2024-09-16T20:36:11.203 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-16T20:36:11.219 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T20:36:11.301 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/dis-2024-09-16_19:45:14-rbd-wip-dis-testing-distro-default-smithi/7907507 branch: wip-dis-testing description: rbd/device/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-bitmap supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nbd} email: idryomov@gmail.com first_in_suite: false job_id: '7907507' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: dis-2024-09-16_19:45:14-rbd-wip-dis-testing-distro-default-smithi no_nested_subset: false openstack: - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-dis-testing ceph: conf: global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - SLOW_OPS - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum - noscrub - nodeep-scrub sha1: 6e8474a02f2b86353394076ab80c6f50ea28e361 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: extra_packages: - rbd-nbd flavor: default sha1: 6e8474a02f2b86353394076ab80c6f50ea28e361 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-dis-testing sha1: 6e8474a02f2b86353394076ab80c6f50ea28e361 owner: scheduled_dis@teuthology priority: 70 repo: https://github.com/ceph/ceph-ci roles: - - mon.a - mon.c - osd.0 - osd.1 - osd.2 - - mon.b - mgr.x - osd.3 - osd.4 - osd.5 - - client.0 seed: 2961 sha1: 6e8474a02f2b86353394076ab80c6f50ea28e361 sleep_before_teardown: 0 subset: 66/100 suite: rbd suite_branch: wip-dis-testing suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_6e8474a02f2b86353394076ab80c6f50ea28e361/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci suite_sha1: 6e8474a02f2b86353394076ab80c6f50ea28e361 targets: smithi029.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP3WniueVCePjB674hgS/OiOwtALk1EjrLxndGnr52FplnqEJL+LFsN++t6kfeeYy8zfkVDXJLqzdonjfYBEyNw= smithi043.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDk1jDLkQEAOSm5cIMX2m6pEXmEeCa+Nnm2oBQ0gvGc/tQctf4vmeQ4WVYa0C45ipJqEFBCB6BrDSs4fvkVmMKs= smithi139.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMqbK+ZfDBSX6DiaHHbJBA9OWjwpcBU86wW0vkxexx3xjQk26MC//JrenBONHIhXcoMihK7NN+/gHMFPJn2KCzs= tasks: - install: null - ceph: null - thrashosds: timeout: 1200 - rbd_fsx: clients: - client.0 holebdy: 512 nbd: true ops: 6000 punch_holes: true readbdy: 512 truncbdy: 512 writebdy: 512 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-16_19:45:14 tube: smithi user: dis verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-16T20:36:11.301 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_6e8474a02f2b86353394076ab80c6f50ea28e361/qa; will attempt to use it 2024-09-16T20:36:11.302 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_6e8474a02f2b86353394076ab80c6f50ea28e361/qa/tasks 2024-09-16T20:36:11.302 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-16T20:36:11.303 INFO:teuthology.task.internal:Checking packages... 2024-09-16T20:36:11.325 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '6e8474a02f2b86353394076ab80c6f50ea28e361' 2024-09-16T20:36:11.325 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-16T20:36:11.326 INFO:teuthology.packaging:ref: None 2024-09-16T20:36:11.326 INFO:teuthology.packaging:tag: None 2024-09-16T20:36:11.326 INFO:teuthology.packaging:branch: wip-dis-testing 2024-09-16T20:36:11.326 INFO:teuthology.packaging:sha1: 6e8474a02f2b86353394076ab80c6f50ea28e361 2024-09-16T20:36:11.326 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-dis-testing 2024-09-16T20:36:11.474 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4883.g6e8474a0 2024-09-16T20:36:11.476 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-16T20:36:11.481 INFO:teuthology.task.internal:no buildpackages task found 2024-09-16T20:36:11.481 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-16T20:36:11.486 INFO:teuthology.task.internal:Saving configuration 2024-09-16T20:36:11.500 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-16T20:36:11.505 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-16T20:36:11.532 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi029.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dis-2024-09-16_19:45:14-rbd-wip-dis-testing-distro-default-smithi/7907507', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 20:30:54.994052', 'locked_by': 'scheduled_dis@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP3WniueVCePjB674hgS/OiOwtALk1EjrLxndGnr52FplnqEJL+LFsN++t6kfeeYy8zfkVDXJLqzdonjfYBEyNw='} 2024-09-16T20:36:11.557 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi043.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dis-2024-09-16_19:45:14-rbd-wip-dis-testing-distro-default-smithi/7907507', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 20:30:54.995257', 'locked_by': 'scheduled_dis@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDk1jDLkQEAOSm5cIMX2m6pEXmEeCa+Nnm2oBQ0gvGc/tQctf4vmeQ4WVYa0C45ipJqEFBCB6BrDSs4fvkVmMKs='} 2024-09-16T20:36:11.575 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi139.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dis-2024-09-16_19:45:14-rbd-wip-dis-testing-distro-default-smithi/7907507', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 20:30:54.992623', 'locked_by': 'scheduled_dis@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMqbK+ZfDBSX6DiaHHbJBA9OWjwpcBU86wW0vkxexx3xjQk26MC//JrenBONHIhXcoMihK7NN+/gHMFPJn2KCzs='} 2024-09-16T20:36:11.576 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-16T20:36:11.581 INFO:teuthology.task.internal:roles: ubuntu@smithi029.front.sepia.ceph.com - ['mon.a', 'mon.c', 'osd.0', 'osd.1', 'osd.2'] 2024-09-16T20:36:11.581 INFO:teuthology.task.internal:roles: ubuntu@smithi043.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.3', 'osd.4', 'osd.5'] 2024-09-16T20:36:11.581 INFO:teuthology.task.internal:roles: ubuntu@smithi139.front.sepia.ceph.com - ['client.0'] 2024-09-16T20:36:11.581 INFO:teuthology.run_tasks:Running task console_log... 2024-09-16T20:36:11.666 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f5bc67c7ac0>, signals=[15]) 2024-09-16T20:36:11.666 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-16T20:36:11.673 INFO:teuthology.task.internal:Opening connections... 2024-09-16T20:36:11.673 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi029.front.sepia.ceph.com 2024-09-16T20:36:11.674 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi029.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T20:36:11.754 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi043.front.sepia.ceph.com 2024-09-16T20:36:11.754 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T20:36:11.831 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi139.front.sepia.ceph.com 2024-09-16T20:36:11.832 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T20:36:11.906 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-16T20:36:11.912 DEBUG:teuthology.orchestra.run.smithi029:> uname -m 2024-09-16T20:36:11.929 INFO:teuthology.orchestra.run.smithi029.stdout:x86_64 2024-09-16T20:36:11.930 DEBUG:teuthology.orchestra.run.smithi029:> cat /etc/os-release 2024-09-16T20:36:11.989 INFO:teuthology.orchestra.run.smithi029.stdout:NAME="CentOS Stream" 2024-09-16T20:36:11.989 INFO:teuthology.orchestra.run.smithi029.stdout:VERSION="9" 2024-09-16T20:36:11.989 INFO:teuthology.orchestra.run.smithi029.stdout:ID="centos" 2024-09-16T20:36:11.989 INFO:teuthology.orchestra.run.smithi029.stdout:ID_LIKE="rhel fedora" 2024-09-16T20:36:11.989 INFO:teuthology.orchestra.run.smithi029.stdout:VERSION_ID="9" 2024-09-16T20:36:11.989 INFO:teuthology.orchestra.run.smithi029.stdout:PLATFORM_ID="platform:el9" 2024-09-16T20:36:11.989 INFO:teuthology.orchestra.run.smithi029.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-16T20:36:11.989 INFO:teuthology.orchestra.run.smithi029.stdout:ANSI_COLOR="0;31" 2024-09-16T20:36:11.990 INFO:teuthology.orchestra.run.smithi029.stdout:LOGO="fedora-logo-icon" 2024-09-16T20:36:11.990 INFO:teuthology.orchestra.run.smithi029.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-16T20:36:11.990 INFO:teuthology.orchestra.run.smithi029.stdout:HOME_URL="https://centos.org/" 2024-09-16T20:36:11.990 INFO:teuthology.orchestra.run.smithi029.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-16T20:36:11.990 INFO:teuthology.orchestra.run.smithi029.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-16T20:36:11.990 INFO:teuthology.orchestra.run.smithi029.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-16T20:36:11.992 INFO:teuthology.lock.ops:Updating smithi029.front.sepia.ceph.com on lock server 2024-09-16T20:36:12.018 DEBUG:teuthology.orchestra.run.smithi043:> uname -m 2024-09-16T20:36:12.037 INFO:teuthology.orchestra.run.smithi043.stdout:x86_64 2024-09-16T20:36:12.037 DEBUG:teuthology.orchestra.run.smithi043:> cat /etc/os-release 2024-09-16T20:36:12.095 INFO:teuthology.orchestra.run.smithi043.stdout:NAME="CentOS Stream" 2024-09-16T20:36:12.095 INFO:teuthology.orchestra.run.smithi043.stdout:VERSION="9" 2024-09-16T20:36:12.095 INFO:teuthology.orchestra.run.smithi043.stdout:ID="centos" 2024-09-16T20:36:12.095 INFO:teuthology.orchestra.run.smithi043.stdout:ID_LIKE="rhel fedora" 2024-09-16T20:36:12.096 INFO:teuthology.orchestra.run.smithi043.stdout:VERSION_ID="9" 2024-09-16T20:36:12.096 INFO:teuthology.orchestra.run.smithi043.stdout:PLATFORM_ID="platform:el9" 2024-09-16T20:36:12.096 INFO:teuthology.orchestra.run.smithi043.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-16T20:36:12.096 INFO:teuthology.orchestra.run.smithi043.stdout:ANSI_COLOR="0;31" 2024-09-16T20:36:12.096 INFO:teuthology.orchestra.run.smithi043.stdout:LOGO="fedora-logo-icon" 2024-09-16T20:36:12.096 INFO:teuthology.orchestra.run.smithi043.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-16T20:36:12.096 INFO:teuthology.orchestra.run.smithi043.stdout:HOME_URL="https://centos.org/" 2024-09-16T20:36:12.096 INFO:teuthology.orchestra.run.smithi043.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-16T20:36:12.096 INFO:teuthology.orchestra.run.smithi043.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-16T20:36:12.096 INFO:teuthology.orchestra.run.smithi043.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-16T20:36:12.097 INFO:teuthology.lock.ops:Updating smithi043.front.sepia.ceph.com on lock server 2024-09-16T20:36:12.118 DEBUG:teuthology.orchestra.run.smithi139:> uname -m 2024-09-16T20:36:12.136 INFO:teuthology.orchestra.run.smithi139.stdout:x86_64 2024-09-16T20:36:12.137 DEBUG:teuthology.orchestra.run.smithi139:> cat /etc/os-release 2024-09-16T20:36:12.195 INFO:teuthology.orchestra.run.smithi139.stdout:NAME="CentOS Stream" 2024-09-16T20:36:12.195 INFO:teuthology.orchestra.run.smithi139.stdout:VERSION="9" 2024-09-16T20:36:12.196 INFO:teuthology.orchestra.run.smithi139.stdout:ID="centos" 2024-09-16T20:36:12.196 INFO:teuthology.orchestra.run.smithi139.stdout:ID_LIKE="rhel fedora" 2024-09-16T20:36:12.196 INFO:teuthology.orchestra.run.smithi139.stdout:VERSION_ID="9" 2024-09-16T20:36:12.196 INFO:teuthology.orchestra.run.smithi139.stdout:PLATFORM_ID="platform:el9" 2024-09-16T20:36:12.196 INFO:teuthology.orchestra.run.smithi139.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-16T20:36:12.196 INFO:teuthology.orchestra.run.smithi139.stdout:ANSI_COLOR="0;31" 2024-09-16T20:36:12.196 INFO:teuthology.orchestra.run.smithi139.stdout:LOGO="fedora-logo-icon" 2024-09-16T20:36:12.196 INFO:teuthology.orchestra.run.smithi139.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-16T20:36:12.196 INFO:teuthology.orchestra.run.smithi139.stdout:HOME_URL="https://centos.org/" 2024-09-16T20:36:12.196 INFO:teuthology.orchestra.run.smithi139.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-16T20:36:12.196 INFO:teuthology.orchestra.run.smithi139.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-16T20:36:12.196 INFO:teuthology.orchestra.run.smithi139.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-16T20:36:12.197 INFO:teuthology.lock.ops:Updating smithi139.front.sepia.ceph.com on lock server 2024-09-16T20:36:12.218 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-16T20:36:12.226 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-16T20:36:12.230 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-16T20:36:12.230 DEBUG:teuthology.orchestra.run.smithi029:> test '!' -e /home/ubuntu/cephtest 2024-09-16T20:36:12.233 DEBUG:teuthology.orchestra.run.smithi043:> test '!' -e /home/ubuntu/cephtest 2024-09-16T20:36:12.235 DEBUG:teuthology.orchestra.run.smithi139:> test '!' -e /home/ubuntu/cephtest 2024-09-16T20:36:12.253 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-16T20:36:12.258 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-16T20:36:12.258 DEBUG:teuthology.orchestra.run.smithi029:> test -z $(ls -A /var/lib/ceph) 2024-09-16T20:36:12.291 DEBUG:teuthology.orchestra.run.smithi043:> test -z $(ls -A /var/lib/ceph) 2024-09-16T20:36:12.297 DEBUG:teuthology.orchestra.run.smithi139:> test -z $(ls -A /var/lib/ceph) 2024-09-16T20:36:12.394 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-16T20:36:12.459 INFO:teuthology.run_tasks:Running task kernel... 2024-09-16T20:36:12.474 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-16T20:36:12.474 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'client.0': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-16T20:36:12.474 DEBUG:teuthology.orchestra.run.smithi029:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T20:36:12.475 DEBUG:teuthology.orchestra.run.smithi043:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T20:36:12.475 DEBUG:teuthology.orchestra.run.smithi139:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T20:36:12.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T20:36:12.492 DEBUG:teuthology.orchestra.run.smithi043:> uname -r 2024-09-16T20:36:12.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T20:36:12.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T20:36:12.494 DEBUG:teuthology.orchestra.run.smithi139:> uname -r 2024-09-16T20:36:12.494 DEBUG:teuthology.orchestra.run.smithi029:> uname -r 2024-09-16T20:36:12.550 INFO:teuthology.orchestra.run.smithi043.stdout:5.14.0-503.el9.x86_64 2024-09-16T20:36:12.550 INFO:teuthology.task.kernel:Running kernel on smithi043: 5.14.0-503.el9.x86_64 2024-09-16T20:36:12.551 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum install -y kernel 2024-09-16T20:36:12.552 INFO:teuthology.orchestra.run.smithi139.stdout:5.14.0-503.el9.x86_64 2024-09-16T20:36:12.552 INFO:teuthology.task.kernel:Running kernel on smithi139: 5.14.0-503.el9.x86_64 2024-09-16T20:36:12.552 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum install -y kernel 2024-09-16T20:36:12.553 INFO:teuthology.orchestra.run.smithi029.stdout:5.14.0-503.el9.x86_64 2024-09-16T20:36:12.553 INFO:teuthology.task.kernel:Running kernel on smithi029: 5.14.0-503.el9.x86_64 2024-09-16T20:36:12.553 DEBUG:teuthology.orchestra.run.smithi029:> sudo yum install -y kernel 2024-09-16T20:36:17.630 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - BaseOS 2.5 MB/s | 8.3 MB 00:03 2024-09-16T20:36:17.904 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - BaseOS 2.3 MB/s | 8.3 MB 00:03 2024-09-16T20:36:18.604 INFO:teuthology.orchestra.run.smithi029.stdout:CentOS Stream 9 - BaseOS 2.6 MB/s | 8.3 MB 00:03 2024-09-16T20:36:22.530 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - AppStream 5.7 MB/s | 20 MB 00:03 2024-09-16T20:36:22.864 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - AppStream 4.9 MB/s | 20 MB 00:04 2024-09-16T20:36:23.893 INFO:teuthology.orchestra.run.smithi029.stdout:CentOS Stream 9 - AppStream 4.9 MB/s | 20 MB 00:04 2024-09-16T20:36:28.552 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 6.5 MB 00:01 2024-09-16T20:36:29.623 INFO:teuthology.orchestra.run.smithi029.stdout:CentOS Stream 9 - CRB 3.6 MB/s | 6.5 MB 00:01 2024-09-16T20:36:30.085 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - Extras packages 66 kB/s | 19 kB 00:00 2024-09-16T20:36:31.158 INFO:teuthology.orchestra.run.smithi029.stdout:CentOS Stream 9 - Extras packages 71 kB/s | 19 kB 00:00 2024-09-16T20:36:31.356 INFO:teuthology.orchestra.run.smithi043.stdout:Extra Packages for Enterprise Linux 20 MB/s | 23 MB 00:01 2024-09-16T20:36:31.675 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - CRB 1.2 MB/s | 6.5 MB 00:05 2024-09-16T20:36:32.291 INFO:teuthology.orchestra.run.smithi029.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-16T20:36:33.170 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - Extras packages 71 kB/s | 19 kB 00:00 2024-09-16T20:36:34.215 INFO:teuthology.orchestra.run.smithi139.stdout:Extra Packages for Enterprise Linux 24 MB/s | 23 MB 00:00 2024-09-16T20:36:37.004 INFO:teuthology.orchestra.run.smithi043.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-16T20:36:37.855 INFO:teuthology.orchestra.run.smithi029.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2024-09-16T20:36:39.102 INFO:teuthology.orchestra.run.smithi043.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-16T20:36:39.102 INFO:teuthology.orchestra.run.smithi043.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-16T20:36:39.203 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-16T20:36:39.207 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T20:36:39.207 INFO:teuthology.orchestra.run.smithi043.stdout: Package Architecture Version Repository Size 2024-09-16T20:36:39.207 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T20:36:39.207 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-16T20:36:39.207 INFO:teuthology.orchestra.run.smithi043.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-16T20:36:39.207 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-09-16T20:36:39.208 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-16T20:36:39.208 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-16T20:36:39.208 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-16T20:36:39.208 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:36:39.208 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-16T20:36:39.208 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T20:36:39.208 INFO:teuthology.orchestra.run.smithi043.stdout:Install 4 Packages 2024-09-16T20:36:39.208 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:36:39.209 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 79 M 2024-09-16T20:36:39.209 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 126 M 2024-09-16T20:36:39.209 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-16T20:36:39.423 INFO:teuthology.orchestra.run.smithi043.stdout:[MIRROR] kernel-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://mirrors.lug.mtu.edu/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-508.el9.x86_64.rpm (IP: 141.219.188.21) 2024-09-16T20:36:39.424 INFO:teuthology.orchestra.run.smithi043.stdout:[MIRROR] kernel-core-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://mirrors.lug.mtu.edu/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-508.el9.x86_64.rpm (IP: 141.219.188.21) 2024-09-16T20:36:39.424 INFO:teuthology.orchestra.run.smithi043.stdout:[MIRROR] kernel-modules-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://mirrors.lug.mtu.edu/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-508.el9.x86_64.rpm (IP: 141.219.188.21) 2024-09-16T20:36:39.564 INFO:teuthology.orchestra.run.smithi043.stdout:[MIRROR] kernel-modules-5.14.0-508.el9.x86_64.rpm: Status code: 404 for https://mirrors.lug.mtu.edu/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-508.el9.x86_64.rpm (IP: 141.219.188.21) 2024-09-16T20:36:39.571 INFO:teuthology.orchestra.run.smithi043.stdout:[MIRROR] kernel-5.14.0-508.el9.x86_64.rpm: Status code: 404 for https://mirrors.lug.mtu.edu/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-508.el9.x86_64.rpm (IP: 141.219.188.21) 2024-09-16T20:36:39.578 INFO:teuthology.orchestra.run.smithi043.stdout:[MIRROR] kernel-core-5.14.0-508.el9.x86_64.rpm: Status code: 404 for https://mirrors.lug.mtu.edu/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-508.el9.x86_64.rpm (IP: 141.219.188.21) 2024-09-16T20:36:39.609 INFO:teuthology.orchestra.run.smithi043.stdout:[MIRROR] kernel-modules-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://mirrors.iu13.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-508.el9.x86_64.rpm (IP: 206.82.17.213) 2024-09-16T20:36:39.614 INFO:teuthology.orchestra.run.smithi043.stdout:[MIRROR] kernel-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://mirrors.iu13.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-508.el9.x86_64.rpm (IP: 206.82.17.213) 2024-09-16T20:36:39.621 INFO:teuthology.orchestra.run.smithi043.stdout:[MIRROR] kernel-core-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://mirrors.iu13.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-508.el9.x86_64.rpm (IP: 206.82.17.213) 2024-09-16T20:36:39.684 INFO:teuthology.orchestra.run.smithi043.stdout:[MIRROR] kernel-modules-5.14.0-508.el9.x86_64.rpm: Status code: 404 for https://mirrors.iu13.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-508.el9.x86_64.rpm (IP: 206.82.17.213) 2024-09-16T20:36:39.689 INFO:teuthology.orchestra.run.smithi043.stdout:[MIRROR] kernel-5.14.0-508.el9.x86_64.rpm: Status code: 404 for https://mirrors.iu13.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-508.el9.x86_64.rpm (IP: 206.82.17.213) 2024-09-16T20:36:39.696 INFO:teuthology.orchestra.run.smithi139.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-09-16T20:36:39.696 INFO:teuthology.orchestra.run.smithi043.stdout:[MIRROR] kernel-core-5.14.0-508.el9.x86_64.rpm: Status code: 404 for https://mirrors.iu13.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-508.el9.x86_64.rpm (IP: 206.82.17.213) 2024-09-16T20:36:39.905 INFO:teuthology.orchestra.run.smithi029.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-16T20:36:39.905 INFO:teuthology.orchestra.run.smithi029.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-16T20:36:39.963 INFO:teuthology.orchestra.run.smithi043.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 57 kB/s | 36 kB 00:00 2024-09-16T20:36:40.006 INFO:teuthology.orchestra.run.smithi043.stdout:[MIRROR] kernel-modules-core-5.14.0-508.el9.x86_64.rpm: Status code: 404 for https://mirrors.lug.mtu.edu/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-core-5.14.0-508.el9.x86_64.rpm (IP: 141.219.188.21) 2024-09-16T20:36:40.016 INFO:teuthology.orchestra.run.smithi029.stdout:Dependencies resolved. 2024-09-16T20:36:40.031 INFO:teuthology.orchestra.run.smithi029.stdout:================================================================================ 2024-09-16T20:36:40.032 INFO:teuthology.orchestra.run.smithi029.stdout: Package Architecture Version Repository Size 2024-09-16T20:36:40.032 INFO:teuthology.orchestra.run.smithi029.stdout:================================================================================ 2024-09-16T20:36:40.032 INFO:teuthology.orchestra.run.smithi029.stdout:Installing: 2024-09-16T20:36:40.032 INFO:teuthology.orchestra.run.smithi029.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-16T20:36:40.032 INFO:teuthology.orchestra.run.smithi029.stdout:Installing dependencies: 2024-09-16T20:36:40.032 INFO:teuthology.orchestra.run.smithi029.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-16T20:36:40.032 INFO:teuthology.orchestra.run.smithi029.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-16T20:36:40.032 INFO:teuthology.orchestra.run.smithi029.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-16T20:36:40.032 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:36:40.032 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction Summary 2024-09-16T20:36:40.032 INFO:teuthology.orchestra.run.smithi029.stdout:================================================================================ 2024-09-16T20:36:40.032 INFO:teuthology.orchestra.run.smithi029.stdout:Install 4 Packages 2024-09-16T20:36:40.033 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:36:40.033 INFO:teuthology.orchestra.run.smithi029.stdout:Total download size: 79 M 2024-09-16T20:36:40.033 INFO:teuthology.orchestra.run.smithi029.stdout:Installed size: 126 M 2024-09-16T20:36:40.033 INFO:teuthology.orchestra.run.smithi029.stdout:Downloading Packages: 2024-09-16T20:36:40.051 INFO:teuthology.orchestra.run.smithi043.stdout:[MIRROR] kernel-modules-core-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://mirrors.lug.mtu.edu/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-core-5.14.0-508.el9.x86_64.rpm (IP: 141.219.188.21) 2024-09-16T20:36:40.073 INFO:teuthology.orchestra.run.smithi043.stdout:[MIRROR] kernel-modules-core-5.14.0-508.el9.x86_64.rpm: Status code: 404 for https://mirrors.iu13.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-core-5.14.0-508.el9.x86_64.rpm (IP: 206.82.17.213) 2024-09-16T20:36:40.085 INFO:teuthology.orchestra.run.smithi029.stdout:[MIRROR] kernel-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://mirror.us.mirhosting.net/centos/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-508.el9.x86_64.rpm (IP: 185.142.32.136) 2024-09-16T20:36:40.086 INFO:teuthology.orchestra.run.smithi029.stdout:[MIRROR] kernel-core-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://mirror.us.mirhosting.net/centos/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-508.el9.x86_64.rpm (IP: 185.142.32.136) 2024-09-16T20:36:40.086 INFO:teuthology.orchestra.run.smithi029.stdout:[MIRROR] kernel-modules-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://mirror.us.mirhosting.net/centos/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-508.el9.x86_64.rpm (IP: 185.142.32.136) 2024-09-16T20:36:40.094 INFO:teuthology.orchestra.run.smithi043.stdout:[MIRROR] kernel-modules-core-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://mirrors.iu13.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-core-5.14.0-508.el9.x86_64.rpm (IP: 206.82.17.213) 2024-09-16T20:36:40.159 INFO:teuthology.orchestra.run.smithi029.stdout:[MIRROR] kernel-5.14.0-508.el9.x86_64.rpm: Status code: 404 for https://mirror.us.mirhosting.net/centos/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-508.el9.x86_64.rpm (IP: 185.142.32.136) 2024-09-16T20:36:40.166 INFO:teuthology.orchestra.run.smithi029.stdout:[MIRROR] kernel-core-5.14.0-508.el9.x86_64.rpm: Status code: 404 for https://mirror.us.mirhosting.net/centos/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-508.el9.x86_64.rpm (IP: 185.142.32.136) 2024-09-16T20:36:40.174 INFO:teuthology.orchestra.run.smithi029.stdout:[MIRROR] kernel-modules-5.14.0-508.el9.x86_64.rpm: Status code: 404 for https://mirror.us.mirhosting.net/centos/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-508.el9.x86_64.rpm (IP: 185.142.32.136) 2024-09-16T20:36:40.190 INFO:teuthology.orchestra.run.smithi029.stdout:[MIRROR] kernel-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-508.el9.x86_64.rpm (IP: 64.50.233.100) 2024-09-16T20:36:40.198 INFO:teuthology.orchestra.run.smithi029.stdout:[MIRROR] kernel-core-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-508.el9.x86_64.rpm (IP: 64.50.233.100) 2024-09-16T20:36:40.205 INFO:teuthology.orchestra.run.smithi029.stdout:[MIRROR] kernel-modules-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-508.el9.x86_64.rpm (IP: 64.50.233.100) 2024-09-16T20:36:40.262 INFO:teuthology.orchestra.run.smithi029.stdout:[MIRROR] kernel-5.14.0-508.el9.x86_64.rpm: Status code: 404 for https://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-508.el9.x86_64.rpm (IP: 64.50.233.100) 2024-09-16T20:36:40.269 INFO:teuthology.orchestra.run.smithi029.stdout:[MIRROR] kernel-core-5.14.0-508.el9.x86_64.rpm: Status code: 404 for https://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-508.el9.x86_64.rpm (IP: 64.50.233.100) 2024-09-16T20:36:40.275 INFO:teuthology.orchestra.run.smithi029.stdout:[MIRROR] kernel-modules-5.14.0-508.el9.x86_64.rpm: Status code: 404 for https://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-508.el9.x86_64.rpm (IP: 64.50.233.100) 2024-09-16T20:36:40.436 INFO:teuthology.orchestra.run.smithi029.stdout:[MIRROR] kernel-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-508.el9.x86_64.rpm (IP: 140.211.166.134) 2024-09-16T20:36:40.445 INFO:teuthology.orchestra.run.smithi029.stdout:[MIRROR] kernel-core-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-508.el9.x86_64.rpm (IP: 140.211.166.134) 2024-09-16T20:36:40.452 INFO:teuthology.orchestra.run.smithi029.stdout:[MIRROR] kernel-modules-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-508.el9.x86_64.rpm (IP: 140.211.166.134) 2024-09-16T20:36:40.794 INFO:teuthology.orchestra.run.smithi029.stdout:[MIRROR] kernel-5.14.0-508.el9.x86_64.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-508.el9.x86_64.rpm (IP: 140.211.166.134) 2024-09-16T20:36:40.805 INFO:teuthology.orchestra.run.smithi029.stdout:[MIRROR] kernel-core-5.14.0-508.el9.x86_64.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-508.el9.x86_64.rpm (IP: 140.211.166.134) 2024-09-16T20:36:40.810 INFO:teuthology.orchestra.run.smithi029.stdout:[MIRROR] kernel-modules-5.14.0-508.el9.x86_64.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-508.el9.x86_64.rpm (IP: 140.211.166.134) 2024-09-16T20:36:41.059 INFO:teuthology.orchestra.run.smithi029.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 36 kB/s | 36 kB 00:01 2024-09-16T20:36:41.133 INFO:teuthology.orchestra.run.smithi029.stdout:[MIRROR] kernel-modules-core-5.14.0-508.el9.x86_64.rpm: Status code: 404 for https://mirror.us.mirhosting.net/centos/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-core-5.14.0-508.el9.x86_64.rpm (IP: 185.142.32.136) 2024-09-16T20:36:41.165 INFO:teuthology.orchestra.run.smithi029.stdout:[MIRROR] kernel-modules-core-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://mirror.us.mirhosting.net/centos/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-core-5.14.0-508.el9.x86_64.rpm (IP: 185.142.32.136) 2024-09-16T20:36:41.180 INFO:teuthology.orchestra.run.smithi029.stdout:[MIRROR] kernel-modules-core-5.14.0-508.el9.x86_64.rpm: Status code: 404 for https://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-core-5.14.0-508.el9.x86_64.rpm (IP: 64.50.233.100) 2024-09-16T20:36:41.213 INFO:teuthology.orchestra.run.smithi029.stdout:[MIRROR] kernel-modules-core-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-core-5.14.0-508.el9.x86_64.rpm (IP: 64.50.233.100) 2024-09-16T20:36:41.300 INFO:teuthology.orchestra.run.smithi029.stdout:[MIRROR] kernel-modules-core-5.14.0-508.el9.x86_64.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-core-5.14.0-508.el9.x86_64.rpm (IP: 140.211.166.134) 2024-09-16T20:36:41.387 INFO:teuthology.orchestra.run.smithi029.stdout:[MIRROR] kernel-modules-core-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-core-5.14.0-508.el9.x86_64.rpm (IP: 140.211.166.134) 2024-09-16T20:36:41.611 INFO:teuthology.orchestra.run.smithi139.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-16T20:36:41.611 INFO:teuthology.orchestra.run.smithi139.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-16T20:36:41.715 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-16T20:36:41.725 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-16T20:36:41.725 INFO:teuthology.orchestra.run.smithi139.stdout: Package Architecture Version Repository Size 2024-09-16T20:36:41.725 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-16T20:36:41.725 INFO:teuthology.orchestra.run.smithi139.stdout:Installing: 2024-09-16T20:36:41.725 INFO:teuthology.orchestra.run.smithi139.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-16T20:36:41.725 INFO:teuthology.orchestra.run.smithi139.stdout:Installing dependencies: 2024-09-16T20:36:41.726 INFO:teuthology.orchestra.run.smithi139.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-16T20:36:41.726 INFO:teuthology.orchestra.run.smithi139.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-16T20:36:41.726 INFO:teuthology.orchestra.run.smithi139.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-16T20:36:41.726 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:36:41.726 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction Summary 2024-09-16T20:36:41.726 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-16T20:36:41.726 INFO:teuthology.orchestra.run.smithi139.stdout:Install 4 Packages 2024-09-16T20:36:41.726 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:36:41.726 INFO:teuthology.orchestra.run.smithi139.stdout:Total download size: 79 M 2024-09-16T20:36:41.726 INFO:teuthology.orchestra.run.smithi139.stdout:Installed size: 126 M 2024-09-16T20:36:41.726 INFO:teuthology.orchestra.run.smithi139.stdout:Downloading Packages: 2024-09-16T20:36:42.064 INFO:teuthology.orchestra.run.smithi139.stdout:[MIRROR] kernel-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://ftp-chi.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-508.el9.x86_64.rpm (IP: 64.50.236.52) 2024-09-16T20:36:42.066 INFO:teuthology.orchestra.run.smithi139.stdout:[MIRROR] kernel-core-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://ftp-chi.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-508.el9.x86_64.rpm (IP: 64.50.236.52) 2024-09-16T20:36:42.066 INFO:teuthology.orchestra.run.smithi139.stdout:[MIRROR] kernel-modules-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://ftp-chi.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-508.el9.x86_64.rpm (IP: 64.50.236.52) 2024-09-16T20:36:42.191 INFO:teuthology.orchestra.run.smithi139.stdout:[MIRROR] kernel-5.14.0-508.el9.x86_64.rpm: Status code: 404 for https://ftp-chi.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-508.el9.x86_64.rpm (IP: 64.50.236.52) 2024-09-16T20:36:42.198 INFO:teuthology.orchestra.run.smithi139.stdout:[MIRROR] kernel-core-5.14.0-508.el9.x86_64.rpm: Status code: 404 for https://ftp-chi.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-508.el9.x86_64.rpm (IP: 64.50.236.52) 2024-09-16T20:36:42.205 INFO:teuthology.orchestra.run.smithi139.stdout:[MIRROR] kernel-modules-5.14.0-508.el9.x86_64.rpm: Status code: 404 for https://ftp-chi.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-508.el9.x86_64.rpm (IP: 64.50.236.52) 2024-09-16T20:36:42.223 INFO:teuthology.orchestra.run.smithi139.stdout:[MIRROR] kernel-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://mirror.us.mirhosting.net/centos/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-508.el9.x86_64.rpm (IP: 185.142.32.136) 2024-09-16T20:36:42.229 INFO:teuthology.orchestra.run.smithi139.stdout:[MIRROR] kernel-core-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://mirror.us.mirhosting.net/centos/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-508.el9.x86_64.rpm (IP: 185.142.32.136) 2024-09-16T20:36:42.236 INFO:teuthology.orchestra.run.smithi139.stdout:[MIRROR] kernel-modules-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://mirror.us.mirhosting.net/centos/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-508.el9.x86_64.rpm (IP: 185.142.32.136) 2024-09-16T20:36:42.295 INFO:teuthology.orchestra.run.smithi139.stdout:[MIRROR] kernel-5.14.0-508.el9.x86_64.rpm: Status code: 404 for https://mirror.us.mirhosting.net/centos/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-508.el9.x86_64.rpm (IP: 185.142.32.136) 2024-09-16T20:36:42.303 INFO:teuthology.orchestra.run.smithi139.stdout:[MIRROR] kernel-core-5.14.0-508.el9.x86_64.rpm: Status code: 404 for https://mirror.us.mirhosting.net/centos/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-508.el9.x86_64.rpm (IP: 185.142.32.136) 2024-09-16T20:36:42.309 INFO:teuthology.orchestra.run.smithi139.stdout:[MIRROR] kernel-modules-5.14.0-508.el9.x86_64.rpm: Status code: 404 for https://mirror.us.mirhosting.net/centos/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-508.el9.x86_64.rpm (IP: 185.142.32.136) 2024-09-16T20:36:42.327 INFO:teuthology.orchestra.run.smithi139.stdout:[MIRROR] kernel-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-508.el9.x86_64.rpm (IP: 64.50.233.100) 2024-09-16T20:36:42.335 INFO:teuthology.orchestra.run.smithi139.stdout:[MIRROR] kernel-core-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-508.el9.x86_64.rpm (IP: 64.50.233.100) 2024-09-16T20:36:42.341 INFO:teuthology.orchestra.run.smithi139.stdout:[MIRROR] kernel-modules-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-508.el9.x86_64.rpm (IP: 64.50.233.100) 2024-09-16T20:36:42.399 INFO:teuthology.orchestra.run.smithi139.stdout:[MIRROR] kernel-5.14.0-508.el9.x86_64.rpm: Status code: 404 for https://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-508.el9.x86_64.rpm (IP: 64.50.233.100) 2024-09-16T20:36:42.406 INFO:teuthology.orchestra.run.smithi139.stdout:[MIRROR] kernel-core-5.14.0-508.el9.x86_64.rpm: Status code: 404 for https://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-508.el9.x86_64.rpm (IP: 64.50.233.100) 2024-09-16T20:36:42.412 INFO:teuthology.orchestra.run.smithi139.stdout:[MIRROR] kernel-modules-5.14.0-508.el9.x86_64.rpm: Status code: 404 for https://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-508.el9.x86_64.rpm (IP: 64.50.233.100) 2024-09-16T20:36:42.457 INFO:teuthology.orchestra.run.smithi139.stdout:[MIRROR] kernel-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-508.el9.x86_64.rpm (IP: 163.237.219.7) 2024-09-16T20:36:42.463 INFO:teuthology.orchestra.run.smithi139.stdout:[MIRROR] kernel-core-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-508.el9.x86_64.rpm (IP: 163.237.219.7) 2024-09-16T20:36:42.471 INFO:teuthology.orchestra.run.smithi139.stdout:[MIRROR] kernel-modules-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-508.el9.x86_64.rpm (IP: 163.237.219.7) 2024-09-16T20:36:42.548 INFO:teuthology.orchestra.run.smithi139.stdout:[MIRROR] kernel-5.14.0-508.el9.x86_64.rpm: Status code: 404 for https://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-508.el9.x86_64.rpm (IP: 163.237.219.7) 2024-09-16T20:36:42.554 INFO:teuthology.orchestra.run.smithi139.stdout:[MIRROR] kernel-core-5.14.0-508.el9.x86_64.rpm: Status code: 404 for https://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-508.el9.x86_64.rpm (IP: 163.237.219.7) 2024-09-16T20:36:42.562 INFO:teuthology.orchestra.run.smithi139.stdout:[MIRROR] kernel-modules-5.14.0-508.el9.x86_64.rpm: Status code: 404 for https://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-508.el9.x86_64.rpm (IP: 163.237.219.7) 2024-09-16T20:36:42.605 INFO:teuthology.orchestra.run.smithi139.stdout:[MIRROR] kernel-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://mirror.team-cymru.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-508.el9.x86_64.rpm (IP: 216.31.2.234) 2024-09-16T20:36:42.612 INFO:teuthology.orchestra.run.smithi139.stdout:[MIRROR] kernel-core-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://mirror.team-cymru.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-508.el9.x86_64.rpm (IP: 216.31.2.234) 2024-09-16T20:36:42.619 INFO:teuthology.orchestra.run.smithi139.stdout:[MIRROR] kernel-modules-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://mirror.team-cymru.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-508.el9.x86_64.rpm (IP: 216.31.2.234) 2024-09-16T20:36:42.704 INFO:teuthology.orchestra.run.smithi139.stdout:[MIRROR] kernel-5.14.0-508.el9.x86_64.rpm: Status code: 404 for https://mirror.team-cymru.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-508.el9.x86_64.rpm (IP: 216.31.2.234) 2024-09-16T20:36:42.710 INFO:teuthology.orchestra.run.smithi139.stdout:[MIRROR] kernel-core-5.14.0-508.el9.x86_64.rpm: Status code: 404 for https://mirror.team-cymru.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-508.el9.x86_64.rpm (IP: 216.31.2.234) 2024-09-16T20:36:42.718 INFO:teuthology.orchestra.run.smithi139.stdout:[MIRROR] kernel-modules-5.14.0-508.el9.x86_64.rpm: Status code: 404 for https://mirror.team-cymru.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-508.el9.x86_64.rpm (IP: 216.31.2.234) 2024-09-16T20:36:42.781 INFO:teuthology.orchestra.run.smithi139.stdout:[MIRROR] kernel-core-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://us.mirrors.cicku.me/centos/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-508.el9.x86_64.rpm (IP: 172.65.90.23) 2024-09-16T20:36:42.782 INFO:teuthology.orchestra.run.smithi139.stdout:[MIRROR] kernel-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://us.mirrors.cicku.me/centos/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-508.el9.x86_64.rpm (IP: 172.65.90.23) 2024-09-16T20:36:42.806 INFO:teuthology.orchestra.run.smithi139.stdout:[MIRROR] kernel-modules-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://us.mirrors.cicku.me/centos/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-508.el9.x86_64.rpm (IP: 172.65.90.23) 2024-09-16T20:36:42.871 INFO:teuthology.orchestra.run.smithi139.stdout:[MIRROR] kernel-core-5.14.0-508.el9.x86_64.rpm: Status code: 404 for https://us.mirrors.cicku.me/centos/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-508.el9.x86_64.rpm (IP: 172.65.90.21) 2024-09-16T20:36:42.875 INFO:teuthology.orchestra.run.smithi139.stdout:[MIRROR] kernel-modules-5.14.0-508.el9.x86_64.rpm: Status code: 404 for https://us.mirrors.cicku.me/centos/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-508.el9.x86_64.rpm (IP: 172.65.90.21) 2024-09-16T20:36:42.921 INFO:teuthology.orchestra.run.smithi139.stdout:[MIRROR] kernel-5.14.0-508.el9.x86_64.rpm: Status code: 404 for https://us.mirrors.cicku.me/centos/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-508.el9.x86_64.rpm (IP: 172.65.90.21) 2024-09-16T20:36:43.180 INFO:teuthology.orchestra.run.smithi043.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 4.1 MB/s | 16 MB 00:03 2024-09-16T20:36:43.267 INFO:teuthology.orchestra.run.smithi029.stdout:(2/4): kernel-modules-core-5.14.0-508.el9.x86_6 13 MB/s | 29 MB 00:02 2024-09-16T20:36:43.497 INFO:teuthology.orchestra.run.smithi139.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 24 kB/s | 36 kB 00:01 2024-09-16T20:36:43.623 INFO:teuthology.orchestra.run.smithi139.stdout:[MIRROR] kernel-modules-core-5.14.0-508.el9.x86_64.rpm: Status code: 404 for https://ftp-chi.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-core-5.14.0-508.el9.x86_64.rpm (IP: 64.50.236.52) 2024-09-16T20:36:43.685 INFO:teuthology.orchestra.run.smithi139.stdout:[MIRROR] kernel-modules-core-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://ftp-chi.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-core-5.14.0-508.el9.x86_64.rpm (IP: 64.50.236.52) 2024-09-16T20:36:43.755 INFO:teuthology.orchestra.run.smithi139.stdout:[MIRROR] kernel-modules-core-5.14.0-508.el9.x86_64.rpm: Status code: 404 for https://mirror.us.mirhosting.net/centos/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-core-5.14.0-508.el9.x86_64.rpm (IP: 185.142.32.136) 2024-09-16T20:36:43.787 INFO:teuthology.orchestra.run.smithi139.stdout:[MIRROR] kernel-modules-core-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://mirror.us.mirhosting.net/centos/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-core-5.14.0-508.el9.x86_64.rpm (IP: 185.142.32.136) 2024-09-16T20:36:43.859 INFO:teuthology.orchestra.run.smithi139.stdout:[MIRROR] kernel-modules-core-5.14.0-508.el9.x86_64.rpm: Status code: 404 for https://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-core-5.14.0-508.el9.x86_64.rpm (IP: 64.50.233.100) 2024-09-16T20:36:43.893 INFO:teuthology.orchestra.run.smithi139.stdout:[MIRROR] kernel-modules-core-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-core-5.14.0-508.el9.x86_64.rpm (IP: 64.50.233.100) 2024-09-16T20:36:43.982 INFO:teuthology.orchestra.run.smithi139.stdout:[MIRROR] kernel-modules-core-5.14.0-508.el9.x86_64.rpm: Status code: 404 for https://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-core-5.14.0-508.el9.x86_64.rpm (IP: 163.237.219.7) 2024-09-16T20:36:44.039 INFO:teuthology.orchestra.run.smithi139.stdout:[MIRROR] kernel-modules-core-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-core-5.14.0-508.el9.x86_64.rpm (IP: 163.237.219.7) 2024-09-16T20:36:44.138 INFO:teuthology.orchestra.run.smithi139.stdout:[MIRROR] kernel-modules-core-5.14.0-508.el9.x86_64.rpm: Status code: 404 for https://mirror.team-cymru.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-core-5.14.0-508.el9.x86_64.rpm (IP: 216.31.2.234) 2024-09-16T20:36:44.196 INFO:teuthology.orchestra.run.smithi139.stdout:[MIRROR] kernel-modules-core-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://mirror.team-cymru.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-core-5.14.0-508.el9.x86_64.rpm (IP: 216.31.2.234) 2024-09-16T20:36:44.286 INFO:teuthology.orchestra.run.smithi139.stdout:[MIRROR] kernel-modules-core-5.14.0-508.el9.x86_64.rpm: Status code: 404 for https://us.mirrors.cicku.me/centos/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-core-5.14.0-508.el9.x86_64.rpm (IP: 172.65.90.21) 2024-09-16T20:36:44.353 INFO:teuthology.orchestra.run.smithi139.stdout:[MIRROR] kernel-modules-core-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://us.mirrors.cicku.me/centos/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-core-5.14.0-508.el9.x86_64.rpm (IP: 172.65.90.23) 2024-09-16T20:36:44.939 INFO:teuthology.orchestra.run.smithi139.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 5.3 MB/s | 16 MB 00:02 2024-09-16T20:36:45.772 INFO:teuthology.orchestra.run.smithi139.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 9.2 MB/s | 35 MB 00:03 2024-09-16T20:36:46.597 INFO:teuthology.orchestra.run.smithi139.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 9.2 MB/s | 29 MB 00:03 2024-09-16T20:36:46.605 INFO:teuthology.orchestra.run.smithi139.stdout:-------------------------------------------------------------------------------- 2024-09-16T20:36:46.605 INFO:teuthology.orchestra.run.smithi139.stdout:Total 16 MB/s | 79 MB 00:04 2024-09-16T20:36:47.292 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction check 2024-09-16T20:36:47.642 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction check succeeded. 2024-09-16T20:36:47.642 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction test 2024-09-16T20:36:48.597 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction test succeeded. 2024-09-16T20:36:48.597 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction 2024-09-16T20:36:49.105 INFO:teuthology.orchestra.run.smithi043.stdout:(3/4): kernel-modules-core-5.14.0-508.el9.x86_6 3.1 MB/s | 29 MB 00:09 2024-09-16T20:36:50.579 INFO:teuthology.orchestra.run.smithi043.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 3.1 MB/s | 35 MB 00:11 2024-09-16T20:36:50.586 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-16T20:36:50.586 INFO:teuthology.orchestra.run.smithi043.stdout:Total 6.9 MB/s | 79 MB 00:11 2024-09-16T20:36:51.039 INFO:teuthology.orchestra.run.smithi139.stdout: Preparing : 1/1 2024-09-16T20:36:51.297 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-16T20:36:51.669 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-16T20:36:51.686 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-16T20:36:51.686 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-16T20:36:51.763 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-16T20:36:52.665 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-16T20:36:52.666 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-16T20:36:54.523 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-16T20:36:54.650 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-16T20:36:55.229 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-16T20:36:55.585 INFO:teuthology.orchestra.run.smithi029.stdout:(3/4): kernel-core-5.14.0-508.el9.x86_64.rpm 1.0 MB/s | 16 MB 00:15 2024-09-16T20:36:55.876 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-16T20:36:55.957 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-16T20:36:58.721 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-16T20:36:58.850 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-16T20:36:59.481 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-16T20:36:59.517 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-16T20:37:02.452 INFO:teuthology.orchestra.run.smithi029.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 1.5 MB/s | 35 MB 00:22 2024-09-16T20:37:02.458 INFO:teuthology.orchestra.run.smithi029.stdout:-------------------------------------------------------------------------------- 2024-09-16T20:37:02.458 INFO:teuthology.orchestra.run.smithi029.stdout:Total 3.5 MB/s | 79 MB 00:22 2024-09-16T20:37:03.175 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction check 2024-09-16T20:37:03.572 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction check succeeded. 2024-09-16T20:37:03.572 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction test 2024-09-16T20:37:03.980 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-16T20:37:04.009 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-16T20:37:04.357 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-16T20:37:04.543 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction test succeeded. 2024-09-16T20:37:04.544 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction 2024-09-16T20:37:07.136 INFO:teuthology.orchestra.run.smithi029.stdout: Preparing : 1/1 2024-09-16T20:37:07.889 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-16T20:37:07.999 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-16T20:37:09.158 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-16T20:37:10.796 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-16T20:37:11.002 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-16T20:37:16.127 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-16T20:37:16.175 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-16T20:37:21.311 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-16T20:37:42.770 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-16T20:37:42.810 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-16T20:37:43.111 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-16T20:37:43.111 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-16T20:37:43.111 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-16T20:37:43.112 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-16T20:37:43.976 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-16T20:37:43.977 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:37:43.977 INFO:teuthology.orchestra.run.smithi139.stdout:Installed: 2024-09-16T20:37:43.977 INFO:teuthology.orchestra.run.smithi139.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-16T20:37:43.977 INFO:teuthology.orchestra.run.smithi139.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-16T20:37:43.977 INFO:teuthology.orchestra.run.smithi139.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-16T20:37:43.977 INFO:teuthology.orchestra.run.smithi139.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-16T20:37:43.977 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:37:43.977 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-16T20:37:44.340 DEBUG:teuthology.orchestra.run.smithi139:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T20:37:44.430 INFO:teuthology.orchestra.run.smithi139.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-16T20:37:44.430 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-16T20:37:44.430 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-16T20:37:44.430 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-16T20:37:44.430 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-16T20:37:44.431 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-16T20:37:44.431 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi139.front.sepia.ceph.com, path=None, version=distro) 2024-09-16T20:37:44.431 DEBUG:teuthology.orchestra.run.smithi139:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T20:37:44.489 INFO:teuthology.orchestra.run.smithi139.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-16T20:37:44.490 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-16T20:37:44.490 DEBUG:teuthology.orchestra.run.smithi139:> sudo rpm -qi grub2-tools 2024-09-16T20:37:44.521 INFO:teuthology.orchestra.run.smithi139.stdout:Name : grub2-tools 2024-09-16T20:37:44.521 INFO:teuthology.orchestra.run.smithi139.stdout:Epoch : 1 2024-09-16T20:37:44.521 INFO:teuthology.orchestra.run.smithi139.stdout:Version : 2.06 2024-09-16T20:37:44.521 INFO:teuthology.orchestra.run.smithi139.stdout:Release : 82.el9 2024-09-16T20:37:44.521 INFO:teuthology.orchestra.run.smithi139.stdout:Architecture: x86_64 2024-09-16T20:37:44.521 INFO:teuthology.orchestra.run.smithi139.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-16T20:37:44.522 INFO:teuthology.orchestra.run.smithi139.stdout:Group : Unspecified 2024-09-16T20:37:44.522 INFO:teuthology.orchestra.run.smithi139.stdout:Size : 8274670 2024-09-16T20:37:44.522 INFO:teuthology.orchestra.run.smithi139.stdout:License : GPLv3+ 2024-09-16T20:37:44.522 INFO:teuthology.orchestra.run.smithi139.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-16T20:37:44.522 INFO:teuthology.orchestra.run.smithi139.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-16T20:37:44.522 INFO:teuthology.orchestra.run.smithi139.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-16T20:37:44.522 INFO:teuthology.orchestra.run.smithi139.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-16T20:37:44.522 INFO:teuthology.orchestra.run.smithi139.stdout:Packager : builder@centos.org 2024-09-16T20:37:44.522 INFO:teuthology.orchestra.run.smithi139.stdout:Vendor : CentOS 2024-09-16T20:37:44.522 INFO:teuthology.orchestra.run.smithi139.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-16T20:37:44.522 INFO:teuthology.orchestra.run.smithi139.stdout:Summary : Support tools for GRUB. 2024-09-16T20:37:44.522 INFO:teuthology.orchestra.run.smithi139.stdout:Description : 2024-09-16T20:37:44.522 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:37:44.523 INFO:teuthology.orchestra.run.smithi139.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-16T20:37:44.523 INFO:teuthology.orchestra.run.smithi139.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-16T20:37:44.523 INFO:teuthology.orchestra.run.smithi139.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-16T20:37:44.523 INFO:teuthology.orchestra.run.smithi139.stdout:hardware devices. 2024-09-16T20:37:44.523 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:37:44.523 INFO:teuthology.orchestra.run.smithi139.stdout:This subpackage provides tools for support of all platforms. 2024-09-16T20:37:44.524 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-16T20:37:44.524 INFO:teuthology.task.kernel:Updating grub on smithi139 to boot 5.14.0-508.el9.x86_64 2024-09-16T20:37:44.524 DEBUG:teuthology.orchestra.run.smithi139:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-16T20:37:45.232 INFO:teuthology.orchestra.run.smithi139.stderr:Generating grub configuration file ... 2024-09-16T20:37:46.563 INFO:teuthology.orchestra.run.smithi139.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-16T20:37:46.602 INFO:teuthology.orchestra.run.smithi139.stderr:done 2024-09-16T20:37:46.605 DEBUG:teuthology.orchestra.run.smithi139:> mktemp 2024-09-16T20:37:46.622 INFO:teuthology.orchestra.run.smithi139.stdout:/tmp/tmp.OS3GGF3Rcz 2024-09-16T20:37:46.622 DEBUG:teuthology.orchestra.run.smithi139:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.OS3GGF3Rcz 2024-09-16T20:37:46.690 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 0666 /tmp/tmp.OS3GGF3Rcz 2024-09-16T20:37:46.832 DEBUG:teuthology.orchestra.remote:smithi139:/tmp/tmp.OS3GGF3Rcz is 6KB 2024-09-16T20:37:46.881 DEBUG:teuthology.orchestra.run.smithi139:> rm -fr /tmp/tmp.OS3GGF3Rcz 2024-09-16T20:37:46.896 DEBUG:teuthology.orchestra.run.smithi139:> sudo /bin/ls /boot/loader/entries || true 2024-09-16T20:37:46.958 INFO:teuthology.orchestra.run.smithi139.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-16T20:37:46.958 INFO:teuthology.orchestra.run.smithi139.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-16T20:37:46.958 INFO:teuthology.orchestra.run.smithi139.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-16T20:37:46.958 INFO:teuthology.orchestra.run.smithi139.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-16T20:37:46.960 DEBUG:teuthology.orchestra.run.smithi139:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-16T20:37:47.115 DEBUG:teuthology.orchestra.run.smithi139:> sudo shutdown -r now 2024-09-16T20:37:48.169 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-16T20:37:48.211 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-16T20:37:48.537 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-16T20:37:48.537 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-16T20:37:48.537 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-16T20:37:48.537 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-16T20:37:49.345 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-16T20:37:49.346 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:37:49.346 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-16T20:37:49.346 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-16T20:37:49.346 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-16T20:37:49.346 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-16T20:37:49.346 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-16T20:37:49.346 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:37:49.346 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-16T20:37:49.726 DEBUG:teuthology.orchestra.run.smithi043:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T20:37:49.831 INFO:teuthology.orchestra.run.smithi043.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-16T20:37:49.831 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-16T20:37:49.832 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-16T20:37:49.832 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-16T20:37:49.832 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-16T20:37:49.832 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-16T20:37:49.832 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi043.front.sepia.ceph.com, path=None, version=distro) 2024-09-16T20:37:49.832 DEBUG:teuthology.orchestra.run.smithi043:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T20:37:49.893 INFO:teuthology.orchestra.run.smithi043.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-16T20:37:49.893 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-16T20:37:49.893 DEBUG:teuthology.orchestra.run.smithi043:> sudo rpm -qi grub2-tools 2024-09-16T20:37:49.929 INFO:teuthology.orchestra.run.smithi043.stdout:Name : grub2-tools 2024-09-16T20:37:49.929 INFO:teuthology.orchestra.run.smithi043.stdout:Epoch : 1 2024-09-16T20:37:49.930 INFO:teuthology.orchestra.run.smithi043.stdout:Version : 2.06 2024-09-16T20:37:49.930 INFO:teuthology.orchestra.run.smithi043.stdout:Release : 82.el9 2024-09-16T20:37:49.930 INFO:teuthology.orchestra.run.smithi043.stdout:Architecture: x86_64 2024-09-16T20:37:49.930 INFO:teuthology.orchestra.run.smithi043.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-16T20:37:49.930 INFO:teuthology.orchestra.run.smithi043.stdout:Group : Unspecified 2024-09-16T20:37:49.930 INFO:teuthology.orchestra.run.smithi043.stdout:Size : 8274670 2024-09-16T20:37:49.930 INFO:teuthology.orchestra.run.smithi043.stdout:License : GPLv3+ 2024-09-16T20:37:49.930 INFO:teuthology.orchestra.run.smithi043.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-16T20:37:49.930 INFO:teuthology.orchestra.run.smithi043.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-16T20:37:49.930 INFO:teuthology.orchestra.run.smithi043.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-16T20:37:49.930 INFO:teuthology.orchestra.run.smithi043.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-16T20:37:49.930 INFO:teuthology.orchestra.run.smithi043.stdout:Packager : builder@centos.org 2024-09-16T20:37:49.931 INFO:teuthology.orchestra.run.smithi043.stdout:Vendor : CentOS 2024-09-16T20:37:49.931 INFO:teuthology.orchestra.run.smithi043.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-16T20:37:49.931 INFO:teuthology.orchestra.run.smithi043.stdout:Summary : Support tools for GRUB. 2024-09-16T20:37:49.931 INFO:teuthology.orchestra.run.smithi043.stdout:Description : 2024-09-16T20:37:49.931 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:37:49.931 INFO:teuthology.orchestra.run.smithi043.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-16T20:37:49.931 INFO:teuthology.orchestra.run.smithi043.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-16T20:37:49.931 INFO:teuthology.orchestra.run.smithi043.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-16T20:37:49.931 INFO:teuthology.orchestra.run.smithi043.stdout:hardware devices. 2024-09-16T20:37:49.931 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:37:49.931 INFO:teuthology.orchestra.run.smithi043.stdout:This subpackage provides tools for support of all platforms. 2024-09-16T20:37:49.932 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-16T20:37:49.932 INFO:teuthology.task.kernel:Updating grub on smithi043 to boot 5.14.0-508.el9.x86_64 2024-09-16T20:37:49.932 DEBUG:teuthology.orchestra.run.smithi043:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-16T20:37:50.750 INFO:teuthology.orchestra.run.smithi043.stderr:Generating grub configuration file ... 2024-09-16T20:37:52.134 INFO:teuthology.orchestra.run.smithi043.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-16T20:37:52.151 INFO:teuthology.orchestra.run.smithi043.stderr:done 2024-09-16T20:37:52.153 DEBUG:teuthology.orchestra.run.smithi043:> mktemp 2024-09-16T20:37:52.168 INFO:teuthology.orchestra.run.smithi043.stdout:/tmp/tmp.PWGifS4lAN 2024-09-16T20:37:52.168 DEBUG:teuthology.orchestra.run.smithi043:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.PWGifS4lAN 2024-09-16T20:37:52.239 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 0666 /tmp/tmp.PWGifS4lAN 2024-09-16T20:37:52.379 DEBUG:teuthology.orchestra.remote:smithi043:/tmp/tmp.PWGifS4lAN is 6KB 2024-09-16T20:37:52.399 DEBUG:teuthology.orchestra.run.smithi043:> rm -fr /tmp/tmp.PWGifS4lAN 2024-09-16T20:37:52.413 DEBUG:teuthology.orchestra.run.smithi043:> sudo /bin/ls /boot/loader/entries || true 2024-09-16T20:37:52.476 INFO:teuthology.orchestra.run.smithi043.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-16T20:37:52.477 INFO:teuthology.orchestra.run.smithi043.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-16T20:37:52.477 INFO:teuthology.orchestra.run.smithi043.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-16T20:37:52.477 INFO:teuthology.orchestra.run.smithi043.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-16T20:37:52.478 DEBUG:teuthology.orchestra.run.smithi043:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-16T20:37:52.642 DEBUG:teuthology.orchestra.run.smithi043:> sudo shutdown -r now 2024-09-16T20:38:03.184 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-16T20:38:03.228 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-16T20:38:03.677 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-16T20:38:03.677 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-16T20:38:03.677 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-16T20:38:03.677 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-16T20:38:04.733 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-16T20:38:04.733 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:38:04.733 INFO:teuthology.orchestra.run.smithi029.stdout:Installed: 2024-09-16T20:38:04.734 INFO:teuthology.orchestra.run.smithi029.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-16T20:38:04.734 INFO:teuthology.orchestra.run.smithi029.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-16T20:38:04.734 INFO:teuthology.orchestra.run.smithi029.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-16T20:38:04.735 INFO:teuthology.orchestra.run.smithi029.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-16T20:38:04.735 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:38:04.735 INFO:teuthology.orchestra.run.smithi029.stdout:Complete! 2024-09-16T20:38:05.225 DEBUG:teuthology.orchestra.run.smithi029:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T20:38:05.325 INFO:teuthology.orchestra.run.smithi029.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-16T20:38:05.326 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-16T20:38:05.326 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-16T20:38:05.326 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-16T20:38:05.326 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-16T20:38:05.326 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-16T20:38:05.326 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi029.front.sepia.ceph.com, path=None, version=distro) 2024-09-16T20:38:05.326 DEBUG:teuthology.orchestra.run.smithi029:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T20:38:05.386 INFO:teuthology.orchestra.run.smithi029.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-16T20:38:05.386 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-16T20:38:05.386 DEBUG:teuthology.orchestra.run.smithi029:> sudo rpm -qi grub2-tools 2024-09-16T20:38:05.417 INFO:teuthology.orchestra.run.smithi029.stdout:Name : grub2-tools 2024-09-16T20:38:05.417 INFO:teuthology.orchestra.run.smithi029.stdout:Epoch : 1 2024-09-16T20:38:05.417 INFO:teuthology.orchestra.run.smithi029.stdout:Version : 2.06 2024-09-16T20:38:05.417 INFO:teuthology.orchestra.run.smithi029.stdout:Release : 82.el9 2024-09-16T20:38:05.418 INFO:teuthology.orchestra.run.smithi029.stdout:Architecture: x86_64 2024-09-16T20:38:05.418 INFO:teuthology.orchestra.run.smithi029.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-16T20:38:05.418 INFO:teuthology.orchestra.run.smithi029.stdout:Group : Unspecified 2024-09-16T20:38:05.418 INFO:teuthology.orchestra.run.smithi029.stdout:Size : 8274670 2024-09-16T20:38:05.418 INFO:teuthology.orchestra.run.smithi029.stdout:License : GPLv3+ 2024-09-16T20:38:05.418 INFO:teuthology.orchestra.run.smithi029.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-16T20:38:05.418 INFO:teuthology.orchestra.run.smithi029.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-16T20:38:05.418 INFO:teuthology.orchestra.run.smithi029.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-16T20:38:05.418 INFO:teuthology.orchestra.run.smithi029.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-16T20:38:05.418 INFO:teuthology.orchestra.run.smithi029.stdout:Packager : builder@centos.org 2024-09-16T20:38:05.418 INFO:teuthology.orchestra.run.smithi029.stdout:Vendor : CentOS 2024-09-16T20:38:05.418 INFO:teuthology.orchestra.run.smithi029.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-16T20:38:05.418 INFO:teuthology.orchestra.run.smithi029.stdout:Summary : Support tools for GRUB. 2024-09-16T20:38:05.418 INFO:teuthology.orchestra.run.smithi029.stdout:Description : 2024-09-16T20:38:05.419 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:38:05.419 INFO:teuthology.orchestra.run.smithi029.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-16T20:38:05.419 INFO:teuthology.orchestra.run.smithi029.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-16T20:38:05.419 INFO:teuthology.orchestra.run.smithi029.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-16T20:38:05.419 INFO:teuthology.orchestra.run.smithi029.stdout:hardware devices. 2024-09-16T20:38:05.419 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:38:05.419 INFO:teuthology.orchestra.run.smithi029.stdout:This subpackage provides tools for support of all platforms. 2024-09-16T20:38:05.420 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-16T20:38:05.420 INFO:teuthology.task.kernel:Updating grub on smithi029 to boot 5.14.0-508.el9.x86_64 2024-09-16T20:38:05.420 DEBUG:teuthology.orchestra.run.smithi029:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-16T20:38:06.229 INFO:teuthology.orchestra.run.smithi029.stderr:Generating grub configuration file ... 2024-09-16T20:38:07.856 INFO:teuthology.orchestra.run.smithi029.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-16T20:38:07.898 INFO:teuthology.orchestra.run.smithi029.stderr:done 2024-09-16T20:38:07.900 DEBUG:teuthology.orchestra.run.smithi029:> mktemp 2024-09-16T20:38:07.915 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/tmp.b3rkerrzLr 2024-09-16T20:38:07.915 DEBUG:teuthology.orchestra.run.smithi029:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.b3rkerrzLr 2024-09-16T20:38:07.994 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod 0666 /tmp/tmp.b3rkerrzLr 2024-09-16T20:38:08.146 DEBUG:teuthology.orchestra.remote:smithi029:/tmp/tmp.b3rkerrzLr is 6KB 2024-09-16T20:38:08.195 DEBUG:teuthology.orchestra.run.smithi029:> rm -fr /tmp/tmp.b3rkerrzLr 2024-09-16T20:38:08.210 DEBUG:teuthology.orchestra.run.smithi029:> sudo /bin/ls /boot/loader/entries || true 2024-09-16T20:38:08.278 INFO:teuthology.orchestra.run.smithi029.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-16T20:38:08.278 INFO:teuthology.orchestra.run.smithi029.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-16T20:38:08.278 INFO:teuthology.orchestra.run.smithi029.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-16T20:38:08.279 INFO:teuthology.orchestra.run.smithi029.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-16T20:38:08.280 DEBUG:teuthology.orchestra.run.smithi029:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-16T20:38:08.439 DEBUG:teuthology.orchestra.run.smithi029:> sudo shutdown -r now 2024-09-16T20:38:17.121 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-16T20:38:17.121 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi139.front.sepia.ceph.com' 2024-09-16T20:38:17.122 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T20:38:22.650 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-16T20:38:22.651 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-09-16T20:38:22.651 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T20:38:35.585 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.139 2024-09-16T20:38:38.445 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-16T20:38:38.445 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi029.front.sepia.ceph.com' 2024-09-16T20:38:38.446 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi029.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T20:38:41.217 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.43 2024-09-16T20:38:44.589 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi139.front.sepia.ceph.com' 2024-09-16T20:38:44.590 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T20:38:47.645 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.139 2024-09-16T20:38:50.220 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-09-16T20:38:50.221 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T20:38:56.833 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.29 2024-09-16T20:38:59.648 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi139.front.sepia.ceph.com' 2024-09-16T20:38:59.648 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T20:39:05.839 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi029.front.sepia.ceph.com' 2024-09-16T20:39:05.840 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi029.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T20:39:18.081 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.139 2024-09-16T20:39:33.097 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi139.front.sepia.ceph.com' 2024-09-16T20:39:33.098 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T20:39:33.426 DEBUG:teuthology.orchestra.run.smithi139:> true 2024-09-16T20:39:33.957 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi139.front.sepia.ceph.com' 2024-09-16T20:39:33.957 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-508.el9.x86_64"... 2024-09-16T20:39:33.957 DEBUG:teuthology.orchestra.run.smithi139:> uname -r 2024-09-16T20:39:33.974 INFO:teuthology.orchestra.run.smithi139.stdout:5.14.0-508.el9.x86_64 2024-09-16T20:39:33.974 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-16T20:39:33.974 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-16T20:39:33.974 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-16T20:39:34.975 INFO:teuthology.task.kernel:Enabling kdb on client.0... 2024-09-16T20:39:34.975 DEBUG:teuthology.orchestra.run.smithi139:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-16T20:39:35.051 INFO:teuthology.orchestra.run.smithi139.stdout:ttyS1 2024-09-16T20:39:35.070 DEBUG:teuthology.parallel:result is None 2024-09-16T20:39:50.223 DEBUG:teuthology.orchestra.remote:timed out 2024-09-16T20:40:02.223 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-09-16T20:40:02.224 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T20:40:02.603 DEBUG:teuthology.orchestra.run.smithi043:> true 2024-09-16T20:40:02.973 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-09-16T20:40:02.973 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-508.el9.x86_64"... 2024-09-16T20:40:02.974 DEBUG:teuthology.orchestra.run.smithi043:> uname -r 2024-09-16T20:40:02.991 INFO:teuthology.orchestra.run.smithi043.stdout:5.14.0-508.el9.x86_64 2024-09-16T20:40:02.991 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-16T20:40:02.992 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-16T20:40:02.992 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-16T20:40:03.993 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-16T20:40:03.993 DEBUG:teuthology.orchestra.run.smithi043:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-16T20:40:04.065 INFO:teuthology.orchestra.run.smithi043.stdout:ttyS1 2024-09-16T20:40:04.084 DEBUG:teuthology.parallel:result is None 2024-09-16T20:40:05.842 DEBUG:teuthology.orchestra.remote:timed out 2024-09-16T20:40:17.843 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi029.front.sepia.ceph.com' 2024-09-16T20:40:17.844 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi029.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T20:40:18.154 DEBUG:teuthology.orchestra.run.smithi029:> true 2024-09-16T20:40:18.554 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi029.front.sepia.ceph.com' 2024-09-16T20:40:18.554 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-508.el9.x86_64"... 2024-09-16T20:40:18.554 DEBUG:teuthology.orchestra.run.smithi029:> uname -r 2024-09-16T20:40:18.571 INFO:teuthology.orchestra.run.smithi029.stdout:5.14.0-508.el9.x86_64 2024-09-16T20:40:18.572 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-16T20:40:18.572 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-16T20:40:18.572 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-16T20:40:19.573 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-16T20:40:19.573 DEBUG:teuthology.orchestra.run.smithi029:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-16T20:40:19.656 INFO:teuthology.orchestra.run.smithi029.stdout:ttyS1 2024-09-16T20:40:19.675 DEBUG:teuthology.parallel:result is None 2024-09-16T20:40:19.675 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-16T20:40:19.682 INFO:teuthology.task.internal:Creating test directory... 2024-09-16T20:40:19.682 DEBUG:teuthology.orchestra.run.smithi029:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T20:40:19.685 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T20:40:19.687 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T20:40:19.706 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-16T20:40:19.713 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-16T20:40:19.723 INFO:teuthology.task.internal:Creating archive directory... 2024-09-16T20:40:19.723 DEBUG:teuthology.orchestra.run.smithi029:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T20:40:19.743 DEBUG:teuthology.orchestra.run.smithi043:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T20:40:19.746 DEBUG:teuthology.orchestra.run.smithi139:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T20:40:19.793 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-16T20:40:19.799 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-16T20:40:19.799 DEBUG:teuthology.orchestra.run.smithi029:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T20:40:19.808 DEBUG:teuthology.orchestra.run.smithi043:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T20:40:19.815 DEBUG:teuthology.orchestra.run.smithi139:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T20:40:19.840 INFO:teuthology.orchestra.run.smithi029.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T20:40:19.852 INFO:teuthology.orchestra.run.smithi029.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T20:40:19.858 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T20:40:19.870 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T20:40:19.874 INFO:teuthology.orchestra.run.smithi139.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T20:40:19.885 INFO:teuthology.orchestra.run.smithi139.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T20:40:19.887 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-16T20:40:19.894 INFO:teuthology.task.internal:Configuring sudo... 2024-09-16T20:40:19.894 DEBUG:teuthology.orchestra.run.smithi029:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T20:40:19.896 DEBUG:teuthology.orchestra.run.smithi043:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T20:40:19.900 DEBUG:teuthology.orchestra.run.smithi139:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T20:40:19.928 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-16T20:40:19.936 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-16T20:40:19.936 DEBUG:teuthology.orchestra.run.smithi029:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T20:40:19.966 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T20:40:19.968 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T20:40:19.985 DEBUG:teuthology.orchestra.run.smithi029:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T20:40:20.065 DEBUG:teuthology.orchestra.run.smithi029:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T20:40:20.118 DEBUG:teuthology.orchestra.run.smithi029:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T20:40:20.202 DEBUG:teuthology.orchestra.run.smithi029:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T20:40:20.248 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-09-16T20:40:20.248 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T20:40:20.317 DEBUG:teuthology.orchestra.run.smithi043:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T20:40:20.362 DEBUG:teuthology.orchestra.run.smithi043:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T20:40:20.414 DEBUG:teuthology.orchestra.run.smithi043:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T20:40:20.496 DEBUG:teuthology.orchestra.run.smithi043:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T20:40:20.544 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-16T20:40:20.545 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T20:40:20.614 DEBUG:teuthology.orchestra.run.smithi139:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T20:40:20.658 DEBUG:teuthology.orchestra.run.smithi139:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T20:40:20.721 DEBUG:teuthology.orchestra.run.smithi139:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T20:40:20.807 DEBUG:teuthology.orchestra.run.smithi139:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T20:40:20.852 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-16T20:40:20.852 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T20:40:20.921 DEBUG:teuthology.orchestra.run.smithi029:> sudo service rsyslog restart 2024-09-16T20:40:20.926 DEBUG:teuthology.orchestra.run.smithi043:> sudo service rsyslog restart 2024-09-16T20:40:20.929 DEBUG:teuthology.orchestra.run.smithi139:> sudo service rsyslog restart 2024-09-16T20:40:20.969 INFO:teuthology.orchestra.run.smithi029.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T20:40:20.981 INFO:teuthology.orchestra.run.smithi043.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T20:40:21.001 INFO:teuthology.orchestra.run.smithi139.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T20:40:21.464 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-16T20:40:21.473 INFO:teuthology.task.internal:Starting timer... 2024-09-16T20:40:21.473 INFO:teuthology.run_tasks:Running task pcp... 2024-09-16T20:40:21.574 INFO:teuthology.run_tasks:Running task selinux... 2024-09-16T20:40:21.582 DEBUG:teuthology.orchestra.run.smithi029:> sudo service auditd rotate 2024-09-16T20:40:21.665 INFO:teuthology.orchestra.run.smithi029.stdout:Rotating logs: 2024-09-16T20:40:21.667 DEBUG:teuthology.orchestra.run.smithi043:> sudo service auditd rotate 2024-09-16T20:40:21.714 INFO:teuthology.orchestra.run.smithi043.stdout:Rotating logs: 2024-09-16T20:40:21.716 DEBUG:teuthology.orchestra.run.smithi139:> sudo service auditd rotate 2024-09-16T20:40:21.779 INFO:teuthology.orchestra.run.smithi139.stdout:Rotating logs: 2024-09-16T20:40:21.781 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-16T20:40:21.782 DEBUG:teuthology.orchestra.run.smithi029:> /usr/sbin/getenforce 2024-09-16T20:40:21.818 INFO:teuthology.orchestra.run.smithi029.stdout:Permissive 2024-09-16T20:40:21.818 DEBUG:teuthology.orchestra.run.smithi043:> /usr/sbin/getenforce 2024-09-16T20:40:21.840 INFO:teuthology.orchestra.run.smithi043.stdout:Permissive 2024-09-16T20:40:21.840 DEBUG:teuthology.orchestra.run.smithi139:> /usr/sbin/getenforce 2024-09-16T20:40:21.868 INFO:teuthology.orchestra.run.smithi139.stdout:Permissive 2024-09-16T20:40:21.869 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi029.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi043.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi139.front.sepia.ceph.com': 'permissive'} 2024-09-16T20:40:21.869 DEBUG:teuthology.orchestra.run.smithi029:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-16T20:40:21.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T20:40:21.900 DEBUG:teuthology.orchestra.run.smithi043:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-16T20:40:21.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T20:40:21.930 DEBUG:teuthology.orchestra.run.smithi139:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-16T20:40:21.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T20:40:21.959 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-16T20:40:21.959 DEBUG:teuthology.orchestra.run.smithi029:> sudo /usr/sbin/setenforce permissive 2024-09-16T20:40:21.999 DEBUG:teuthology.orchestra.run.smithi043:> sudo /usr/sbin/setenforce permissive 2024-09-16T20:40:22.038 DEBUG:teuthology.orchestra.run.smithi139:> sudo /usr/sbin/setenforce permissive 2024-09-16T20:40:22.072 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-16T20:40:22.082 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-16T20:40:22.085 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-16T20:40:22.199 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-16T20:40:22.211 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-16T20:40:22.212 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi029.front.sepia.ceph.com,smithi043.front.sepia.ceph.com,smithi139.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-16T20:44:16.996 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi029.front.sepia.ceph.com'), Remote(name='ubuntu@smithi043.front.sepia.ceph.com'), Remote(name='ubuntu@smithi139.front.sepia.ceph.com')] 2024-09-16T20:44:16.997 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi029.front.sepia.ceph.com' 2024-09-16T20:44:16.998 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi029.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T20:44:17.075 DEBUG:teuthology.orchestra.run.smithi029:> true 2024-09-16T20:44:17.152 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi029.front.sepia.ceph.com' 2024-09-16T20:44:17.153 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-09-16T20:44:17.153 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T20:44:17.237 DEBUG:teuthology.orchestra.run.smithi043:> true 2024-09-16T20:44:17.321 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-09-16T20:44:17.322 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi139.front.sepia.ceph.com' 2024-09-16T20:44:17.322 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T20:44:17.412 DEBUG:teuthology.orchestra.run.smithi139:> true 2024-09-16T20:44:17.489 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi139.front.sepia.ceph.com' 2024-09-16T20:44:17.490 INFO:teuthology.run_tasks:Running task clock... 2024-09-16T20:44:17.501 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-16T20:44:17.501 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T20:44:17.502 DEBUG:teuthology.orchestra.run.smithi029:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T20:44:17.504 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T20:44:17.504 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T20:44:17.507 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T20:44:17.507 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T20:44:17.535 INFO:teuthology.orchestra.run.smithi029.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-16T20:44:17.537 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-16T20:44:17.551 INFO:teuthology.orchestra.run.smithi029.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-16T20:44:17.553 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-16T20:44:17.560 INFO:teuthology.orchestra.run.smithi139.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-16T20:44:17.576 INFO:teuthology.orchestra.run.smithi139.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-16T20:44:17.577 INFO:teuthology.orchestra.run.smithi029.stderr:sudo: ntpd: command not found 2024-09-16T20:44:17.580 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: ntpd: command not found 2024-09-16T20:44:17.589 INFO:teuthology.orchestra.run.smithi029.stdout:506 Cannot talk to daemon 2024-09-16T20:44:17.592 INFO:teuthology.orchestra.run.smithi043.stdout:506 Cannot talk to daemon 2024-09-16T20:44:17.602 INFO:teuthology.orchestra.run.smithi139.stderr:sudo: ntpd: command not found 2024-09-16T20:44:17.605 INFO:teuthology.orchestra.run.smithi029.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-16T20:44:17.607 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-16T20:44:17.613 INFO:teuthology.orchestra.run.smithi139.stdout:506 Cannot talk to daemon 2024-09-16T20:44:17.620 INFO:teuthology.orchestra.run.smithi029.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-16T20:44:17.622 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-16T20:44:17.628 INFO:teuthology.orchestra.run.smithi139.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-16T20:44:17.642 INFO:teuthology.orchestra.run.smithi139.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-16T20:44:17.675 INFO:teuthology.orchestra.run.smithi029.stderr:bash: line 1: ntpq: command not found 2024-09-16T20:44:17.679 INFO:teuthology.orchestra.run.smithi029.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T20:44:17.679 INFO:teuthology.orchestra.run.smithi029.stdout:=============================================================================== 2024-09-16T20:44:17.679 INFO:teuthology.orchestra.run.smithi029.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T20:44:17.679 INFO:teuthology.orchestra.run.smithi029.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T20:44:17.679 INFO:teuthology.orchestra.run.smithi029.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T20:44:17.679 INFO:teuthology.orchestra.run.smithi029.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T20:44:17.687 INFO:teuthology.orchestra.run.smithi043.stderr:bash: line 1: ntpq: command not found 2024-09-16T20:44:17.691 INFO:teuthology.orchestra.run.smithi043.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T20:44:17.691 INFO:teuthology.orchestra.run.smithi043.stdout:=============================================================================== 2024-09-16T20:44:17.692 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T20:44:17.692 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T20:44:17.692 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T20:44:17.692 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T20:44:17.696 INFO:teuthology.orchestra.run.smithi139.stderr:bash: line 1: ntpq: command not found 2024-09-16T20:44:17.700 INFO:teuthology.orchestra.run.smithi139.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T20:44:17.701 INFO:teuthology.orchestra.run.smithi139.stdout:=============================================================================== 2024-09-16T20:44:17.701 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T20:44:17.701 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T20:44:17.701 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T20:44:17.701 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T20:44:17.701 INFO:teuthology.run_tasks:Running task install... 2024-09-16T20:44:17.709 DEBUG:teuthology.task.install:project ceph 2024-09-16T20:44:17.710 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'extra_packages': ['rbd-nbd'], 'flavor': 'default', 'sha1': '6e8474a02f2b86353394076ab80c6f50ea28e361'}} 2024-09-16T20:44:17.710 DEBUG:teuthology.task.install:config {'extra_packages': ['rbd-nbd'], 'flavor': 'default', 'sha1': '6e8474a02f2b86353394076ab80c6f50ea28e361'} 2024-09-16T20:44:17.710 INFO:teuthology.task.install:Using flavor: default 2024-09-16T20:44:17.715 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-16T20:44:17.715 INFO:teuthology.task.install:extra packages: ['rbd-nbd'] 2024-09-16T20:44:17.715 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': ['rbd-nbd'], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '6e8474a02f2b86353394076ab80c6f50ea28e361', 'tag': None, 'wait_for_package': False} 2024-09-16T20:44:17.716 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6e8474a02f2b86353394076ab80c6f50ea28e361 2024-09-16T20:44:17.717 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': ['rbd-nbd'], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '6e8474a02f2b86353394076ab80c6f50ea28e361', 'tag': None, 'wait_for_package': False} 2024-09-16T20:44:17.717 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6e8474a02f2b86353394076ab80c6f50ea28e361 2024-09-16T20:44:17.718 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': ['rbd-nbd'], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '6e8474a02f2b86353394076ab80c6f50ea28e361', 'tag': None, 'wait_for_package': False} 2024-09-16T20:44:17.719 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6e8474a02f2b86353394076ab80c6f50ea28e361 2024-09-16T20:44:17.860 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-dis-testing/6e8474a02f2b86353394076ab80c6f50ea28e361/centos/9/flavors/default/ 2024-09-16T20:44:17.860 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4883.g6e8474a0 2024-09-16T20:44:17.863 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-dis-testing/6e8474a02f2b86353394076ab80c6f50ea28e361/centos/9/flavors/default/ 2024-09-16T20:44:17.863 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4883.g6e8474a0 2024-09-16T20:44:17.869 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-dis-testing/6e8474a02f2b86353394076ab80c6f50ea28e361/centos/9/flavors/default/ 2024-09-16T20:44:17.870 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4883.g6e8474a0 2024-09-16T20:44:17.978 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-dis-testing/6e8474a02f2b86353394076ab80c6f50ea28e361/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-dis-testing/6e8474a02f2b86353394076ab80c6f50ea28e361/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-dis-testing/6e8474a02f2b86353394076ab80c6f50ea28e361/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-16T20:44:17.978 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-16T20:44:17.978 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-16T20:44:17.981 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-dis-testing/6e8474a02f2b86353394076ab80c6f50ea28e361/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-dis-testing/6e8474a02f2b86353394076ab80c6f50ea28e361/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-dis-testing/6e8474a02f2b86353394076ab80c6f50ea28e361/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-16T20:44:17.981 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-09-16T20:44:17.981 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-16T20:44:17.985 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-dis-testing/6e8474a02f2b86353394076ab80c6f50ea28e361/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-dis-testing/6e8474a02f2b86353394076ab80c6f50ea28e361/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-dis-testing/6e8474a02f2b86353394076ab80c6f50ea28e361/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-16T20:44:17.985 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-16T20:44:17.985 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-16T20:44:18.004 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, rbd-nbd on remote rpm x86_64 2024-09-16T20:44:18.005 DEBUG:teuthology.orchestra.run.smithi139:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/6e8474a02f2b86353394076ab80c6f50ea28e361/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-16T20:44:18.007 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, rbd-nbd on remote rpm x86_64 2024-09-16T20:44:18.008 DEBUG:teuthology.orchestra.run.smithi029:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/6e8474a02f2b86353394076ab80c6f50ea28e361/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-16T20:44:18.012 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, rbd-nbd on remote rpm x86_64 2024-09-16T20:44:18.012 DEBUG:teuthology.orchestra.run.smithi043:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/6e8474a02f2b86353394076ab80c6f50ea28e361/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-16T20:44:18.070 DEBUG:teuthology.orchestra.run.smithi139:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-16T20:44:18.074 DEBUG:teuthology.orchestra.run.smithi029:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-16T20:44:18.078 DEBUG:teuthology.orchestra.run.smithi043:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-16T20:44:18.146 DEBUG:teuthology.orchestra.run.smithi139:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-16T20:44:18.151 DEBUG:teuthology.orchestra.run.smithi029:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-16T20:44:18.154 DEBUG:teuthology.orchestra.run.smithi043:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-16T20:44:18.209 INFO:teuthology.orchestra.run.smithi139.stdout:check_obsoletes = 1 2024-09-16T20:44:18.211 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum clean all 2024-09-16T20:44:18.216 INFO:teuthology.orchestra.run.smithi029.stdout:check_obsoletes = 1 2024-09-16T20:44:18.217 DEBUG:teuthology.orchestra.run.smithi029:> sudo yum clean all 2024-09-16T20:44:18.219 INFO:teuthology.orchestra.run.smithi043.stdout:check_obsoletes = 1 2024-09-16T20:44:18.221 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum clean all 2024-09-16T20:44:18.439 INFO:teuthology.orchestra.run.smithi139.stdout:45 files removed 2024-09-16T20:44:18.448 INFO:teuthology.orchestra.run.smithi029.stdout:45 files removed 2024-09-16T20:44:18.466 INFO:teuthology.orchestra.run.smithi043.stdout:45 files removed 2024-09-16T20:44:18.467 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum -y install ceph-radosgw 2024-09-16T20:44:18.475 DEBUG:teuthology.orchestra.run.smithi029:> sudo yum -y install ceph-radosgw 2024-09-16T20:44:18.493 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-radosgw 2024-09-16T20:44:19.018 INFO:teuthology.orchestra.run.smithi139.stdout:ceph packages for x86_64 318 kB/s | 84 kB 00:00 2024-09-16T20:44:19.045 INFO:teuthology.orchestra.run.smithi029.stdout:ceph packages for x86_64 312 kB/s | 84 kB 00:00 2024-09-16T20:44:19.069 INFO:teuthology.orchestra.run.smithi043.stdout:ceph packages for x86_64 305 kB/s | 84 kB 00:00 2024-09-16T20:44:19.319 INFO:teuthology.orchestra.run.smithi139.stdout:ceph noarch packages 94 kB/s | 24 kB 00:00 2024-09-16T20:44:19.338 INFO:teuthology.orchestra.run.smithi029.stdout:ceph noarch packages 98 kB/s | 24 kB 00:00 2024-09-16T20:44:19.361 INFO:teuthology.orchestra.run.smithi043.stdout:ceph noarch packages 98 kB/s | 24 kB 00:00 2024-09-16T20:44:19.594 INFO:teuthology.orchestra.run.smithi139.stdout:ceph source packages 8.1 kB/s | 1.9 kB 00:00 2024-09-16T20:44:19.614 INFO:teuthology.orchestra.run.smithi029.stdout:ceph source packages 8.2 kB/s | 1.9 kB 00:00 2024-09-16T20:44:19.637 INFO:teuthology.orchestra.run.smithi043.stdout:ceph source packages 8.2 kB/s | 1.9 kB 00:00 2024-09-16T20:44:21.021 INFO:teuthology.orchestra.run.smithi029.stdout:CentOS Stream 9 - BaseOS 6.0 MB/s | 8.3 MB 00:01 2024-09-16T20:44:22.183 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - BaseOS 3.2 MB/s | 8.3 MB 00:02 2024-09-16T20:44:24.295 INFO:teuthology.orchestra.run.smithi029.stdout:CentOS Stream 9 - AppStream 9.5 MB/s | 20 MB 00:02 2024-09-16T20:44:25.536 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - BaseOS 1.4 MB/s | 8.3 MB 00:05 2024-09-16T20:44:25.968 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - AppStream 7.5 MB/s | 20 MB 00:02 2024-09-16T20:44:34.120 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - CRB 1.5 MB/s | 6.5 MB 00:04 2024-09-16T20:44:35.607 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - Extras packages 73 kB/s | 19 kB 00:00 2024-09-16T20:44:36.578 INFO:teuthology.orchestra.run.smithi139.stdout:Extra Packages for Enterprise Linux 27 MB/s | 23 MB 00:00 2024-09-16T20:44:37.721 INFO:teuthology.orchestra.run.smithi029.stdout:CentOS Stream 9 - CRB 701 kB/s | 6.5 MB 00:09 2024-09-16T20:44:38.795 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - AppStream 1.7 MB/s | 20 MB 00:12 2024-09-16T20:44:39.536 INFO:teuthology.orchestra.run.smithi029.stdout:CentOS Stream 9 - Extras packages 35 kB/s | 19 kB 00:00 2024-09-16T20:44:40.567 INFO:teuthology.orchestra.run.smithi029.stdout:Extra Packages for Enterprise Linux 25 MB/s | 23 MB 00:00 2024-09-16T20:44:42.008 INFO:teuthology.orchestra.run.smithi139.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-16T20:44:44.271 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-16T20:44:44.281 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-16T20:44:44.282 INFO:teuthology.orchestra.run.smithi139.stdout: Package Arch Version Repository Size 2024-09-16T20:44:44.282 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-16T20:44:44.282 INFO:teuthology.orchestra.run.smithi139.stdout:Installing: 2024-09-16T20:44:44.282 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-radosgw x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 11 M 2024-09-16T20:44:44.282 INFO:teuthology.orchestra.run.smithi139.stdout:Upgrading: 2024-09-16T20:44:44.282 INFO:teuthology.orchestra.run.smithi139.stdout: librados2 x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 3.5 M 2024-09-16T20:44:44.282 INFO:teuthology.orchestra.run.smithi139.stdout: librbd1 x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 3.2 M 2024-09-16T20:44:44.282 INFO:teuthology.orchestra.run.smithi139.stdout:Installing dependencies: 2024-09-16T20:44:44.282 INFO:teuthology.orchestra.run.smithi139.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-16T20:44:44.282 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-base x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 5.5 M 2024-09-16T20:44:44.283 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-common x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 22 M 2024-09-16T20:44:44.283 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-selinux x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 25 k 2024-09-16T20:44:44.283 INFO:teuthology.orchestra.run.smithi139.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-16T20:44:44.283 INFO:teuthology.orchestra.run.smithi139.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-16T20:44:44.283 INFO:teuthology.orchestra.run.smithi139.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-16T20:44:44.283 INFO:teuthology.orchestra.run.smithi139.stdout: libcephfs2 x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 755 k 2024-09-16T20:44:44.283 INFO:teuthology.orchestra.run.smithi139.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-16T20:44:44.283 INFO:teuthology.orchestra.run.smithi139.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-16T20:44:44.283 INFO:teuthology.orchestra.run.smithi139.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-16T20:44:44.283 INFO:teuthology.orchestra.run.smithi139.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-16T20:44:44.283 INFO:teuthology.orchestra.run.smithi139.stdout: libradosstriper1 x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 508 k 2024-09-16T20:44:44.283 INFO:teuthology.orchestra.run.smithi139.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-16T20:44:44.283 INFO:teuthology.orchestra.run.smithi139.stdout: librgw2 x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 5.5 M 2024-09-16T20:44:44.284 INFO:teuthology.orchestra.run.smithi139.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-16T20:44:44.284 INFO:teuthology.orchestra.run.smithi139.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-16T20:44:44.284 INFO:teuthology.orchestra.run.smithi139.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-16T20:44:44.284 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ceph-argparse x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 45 k 2024-09-16T20:44:44.284 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ceph-common x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 143 k 2024-09-16T20:44:44.284 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cephfs x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 162 k 2024-09-16T20:44:44.284 INFO:teuthology.orchestra.run.smithi139.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-16T20:44:44.284 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rados x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 320 k 2024-09-16T20:44:44.284 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rbd x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 307 k 2024-09-16T20:44:44.284 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rgw x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 99 k 2024-09-16T20:44:44.284 INFO:teuthology.orchestra.run.smithi139.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-16T20:44:44.284 INFO:teuthology.orchestra.run.smithi139.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-16T20:44:44.284 INFO:teuthology.orchestra.run.smithi139.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-16T20:44:44.285 INFO:teuthology.orchestra.run.smithi139.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-16T20:44:44.285 INFO:teuthology.orchestra.run.smithi139.stdout:Installing weak dependencies: 2024-09-16T20:44:44.285 INFO:teuthology.orchestra.run.smithi139.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-16T20:44:44.285 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:44:44.285 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction Summary 2024-09-16T20:44:44.285 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-16T20:44:44.285 INFO:teuthology.orchestra.run.smithi139.stdout:Install 31 Packages 2024-09-16T20:44:44.285 INFO:teuthology.orchestra.run.smithi139.stdout:Upgrade 2 Packages 2024-09-16T20:44:44.285 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:44:44.286 INFO:teuthology.orchestra.run.smithi139.stdout:Total download size: 62 M 2024-09-16T20:44:44.286 INFO:teuthology.orchestra.run.smithi139.stdout:Downloading Packages: 2024-09-16T20:44:44.951 INFO:teuthology.orchestra.run.smithi139.stdout:(1/33): ceph-base-19.3.0-4883.g6e8474a0.el9.x86 9.9 MB/s | 5.5 MB 00:00 2024-09-16T20:44:44.993 INFO:teuthology.orchestra.run.smithi139.stdout:(2/33): ceph-selinux-19.3.0-4883.g6e8474a0.el9. 606 kB/s | 25 kB 00:00 2024-09-16T20:44:45.085 INFO:teuthology.orchestra.run.smithi139.stdout:(3/33): libcephfs2-19.3.0-4883.g6e8474a0.el9.x8 8.1 MB/s | 755 kB 00:00 2024-09-16T20:44:45.210 INFO:teuthology.orchestra.run.smithi139.stdout:(4/33): ceph-radosgw-19.3.0-4883.g6e8474a0.el9. 13 MB/s | 11 MB 00:00 2024-09-16T20:44:45.235 INFO:teuthology.orchestra.run.smithi139.stdout:(5/33): libradosstriper1-19.3.0-4883.g6e8474a0. 3.3 MB/s | 508 kB 00:00 2024-09-16T20:44:45.286 INFO:teuthology.orchestra.run.smithi139.stdout:(6/33): python3-ceph-argparse-19.3.0-4883.g6e84 901 kB/s | 45 kB 00:00 2024-09-16T20:44:45.361 INFO:teuthology.orchestra.run.smithi139.stdout:(7/33): python3-ceph-common-19.3.0-4883.g6e8474 1.9 MB/s | 143 kB 00:00 2024-09-16T20:44:45.428 INFO:teuthology.orchestra.run.smithi139.stdout:(8/33): python3-cephfs-19.3.0-4883.g6e8474a0.el 2.4 MB/s | 162 kB 00:00 2024-09-16T20:44:45.504 INFO:teuthology.orchestra.run.smithi139.stdout:(9/33): librgw2-19.3.0-4883.g6e8474a0.el9.x86_6 19 MB/s | 5.5 MB 00:00 2024-09-16T20:44:45.704 INFO:teuthology.orchestra.run.smithi139.stdout:(10/33): ceph-common-19.3.0-4883.g6e8474a0.el9. 17 MB/s | 22 MB 00:01 2024-09-16T20:44:45.729 INFO:teuthology.orchestra.run.smithi139.stdout:(11/33): python3-rados-19.3.0-4883.g6e8474a0.el 1.0 MB/s | 320 kB 00:00 2024-09-16T20:44:45.754 INFO:teuthology.orchestra.run.smithi139.stdout:(12/33): python3-rbd-19.3.0-4883.g6e8474a0.el9. 1.2 MB/s | 307 kB 00:00 2024-09-16T20:44:45.779 INFO:teuthology.orchestra.run.smithi139.stdout:(13/33): python3-rgw-19.3.0-4883.g6e8474a0.el9. 1.3 MB/s | 99 kB 00:00 2024-09-16T20:44:45.863 INFO:teuthology.orchestra.run.smithi139.stdout:(14/33): boost-program-options-1.75.0-8.el9.x86 799 kB/s | 107 kB 00:00 2024-09-16T20:44:45.888 INFO:teuthology.orchestra.run.smithi139.stdout:(15/33): libnbd-1.20.2-2.el9.x86_64.rpm 1.2 MB/s | 171 kB 00:00 2024-09-16T20:44:45.913 INFO:teuthology.orchestra.run.smithi139.stdout:(16/33): librabbitmq-0.11.0-7.el9.x86_64.rpm 905 kB/s | 45 kB 00:00 2024-09-16T20:44:45.939 INFO:teuthology.orchestra.run.smithi139.stdout:(17/33): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.0 MB/s | 160 kB 00:00 2024-09-16T20:44:45.964 INFO:teuthology.orchestra.run.smithi139.stdout:(18/33): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 662 kB 00:00 2024-09-16T20:44:45.989 INFO:teuthology.orchestra.run.smithi139.stdout:(19/33): python3-prettytable-0.7.2-27.el9.noarc 844 kB/s | 42 kB 00:00 2024-09-16T20:44:46.014 INFO:teuthology.orchestra.run.smithi139.stdout:(20/33): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.8 MB/s | 292 kB 00:00 2024-09-16T20:44:46.040 INFO:teuthology.orchestra.run.smithi139.stdout:(21/33): qatlib-service-24.02.0-1.el9.x86_64.rp 714 kB/s | 36 kB 00:00 2024-09-16T20:44:46.065 INFO:teuthology.orchestra.run.smithi139.stdout:(22/33): qatlib-24.02.0-1.el9.x86_64.rpm 2.1 MB/s | 221 kB 00:00 2024-09-16T20:44:46.090 INFO:teuthology.orchestra.run.smithi139.stdout:(23/33): qatzip-libs-1.2.0-1.el9.x86_64.rpm 627 kB/s | 47 kB 00:00 2024-09-16T20:44:46.149 INFO:teuthology.orchestra.run.smithi139.stdout:(24/33): libarrow-doc-9.0.0-11.el9.noarch.rpm 448 kB/s | 26 kB 00:00 2024-09-16T20:44:46.166 INFO:teuthology.orchestra.run.smithi139.stdout:(25/33): liboath-2.6.7-2.el9.x86_64.rpm 2.9 MB/s | 49 kB 00:00 2024-09-16T20:44:46.191 INFO:teuthology.orchestra.run.smithi139.stdout:(26/33): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.0 MB/s | 308 kB 00:00 2024-09-16T20:44:46.216 INFO:teuthology.orchestra.run.smithi139.stdout:(27/33): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-16T20:44:46.250 INFO:teuthology.orchestra.run.smithi139.stdout:(28/33): re2-20211101-20.el9.x86_64.rpm 5.6 MB/s | 191 kB 00:00 2024-09-16T20:44:46.257 INFO:teuthology.orchestra.run.smithi029.stdout:lab-extras 37 kB/s | 1.7 kB 00:00 2024-09-16T20:44:46.300 INFO:teuthology.orchestra.run.smithi139.stdout:(29/33): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.5 MB/s | 839 kB 00:00 2024-09-16T20:44:46.392 INFO:teuthology.orchestra.run.smithi139.stdout:(30/33): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-09-16T20:44:46.484 INFO:teuthology.orchestra.run.smithi139.stdout:(31/33): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-09-16T20:44:46.576 INFO:teuthology.orchestra.run.smithi139.stdout:(32/33): librados2-19.3.0-4883.g6e8474a0.el9.x8 13 MB/s | 3.5 MB 00:00 2024-09-16T20:44:46.667 INFO:teuthology.orchestra.run.smithi139.stdout:(33/33): librbd1-19.3.0-4883.g6e8474a0.el9.x86_ 12 MB/s | 3.2 MB 00:00 2024-09-16T20:44:46.673 INFO:teuthology.orchestra.run.smithi139.stdout:-------------------------------------------------------------------------------- 2024-09-16T20:44:46.673 INFO:teuthology.orchestra.run.smithi139.stdout:Total 26 MB/s | 62 MB 00:02 2024-09-16T20:44:46.787 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction check 2024-09-16T20:44:46.823 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction check succeeded. 2024-09-16T20:44:46.823 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction test 2024-09-16T20:44:47.520 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction test succeeded. 2024-09-16T20:44:47.521 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction 2024-09-16T20:44:48.665 INFO:teuthology.orchestra.run.smithi029.stdout:Dependencies resolved. 2024-09-16T20:44:48.682 INFO:teuthology.orchestra.run.smithi029.stdout:================================================================================ 2024-09-16T20:44:48.682 INFO:teuthology.orchestra.run.smithi029.stdout: Package Arch Version Repository Size 2024-09-16T20:44:48.682 INFO:teuthology.orchestra.run.smithi029.stdout:================================================================================ 2024-09-16T20:44:48.682 INFO:teuthology.orchestra.run.smithi029.stdout:Installing: 2024-09-16T20:44:48.682 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-radosgw x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 11 M 2024-09-16T20:44:48.682 INFO:teuthology.orchestra.run.smithi029.stdout:Upgrading: 2024-09-16T20:44:48.682 INFO:teuthology.orchestra.run.smithi029.stdout: librados2 x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 3.5 M 2024-09-16T20:44:48.682 INFO:teuthology.orchestra.run.smithi029.stdout: librbd1 x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 3.2 M 2024-09-16T20:44:48.682 INFO:teuthology.orchestra.run.smithi029.stdout:Installing dependencies: 2024-09-16T20:44:48.683 INFO:teuthology.orchestra.run.smithi029.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-16T20:44:48.683 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-base x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 5.5 M 2024-09-16T20:44:48.683 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-common x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 22 M 2024-09-16T20:44:48.683 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-selinux x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 25 k 2024-09-16T20:44:48.683 INFO:teuthology.orchestra.run.smithi029.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-16T20:44:48.683 INFO:teuthology.orchestra.run.smithi029.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-16T20:44:48.683 INFO:teuthology.orchestra.run.smithi029.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-16T20:44:48.683 INFO:teuthology.orchestra.run.smithi029.stdout: libcephfs2 x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 755 k 2024-09-16T20:44:48.683 INFO:teuthology.orchestra.run.smithi029.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-16T20:44:48.683 INFO:teuthology.orchestra.run.smithi029.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-16T20:44:48.683 INFO:teuthology.orchestra.run.smithi029.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-16T20:44:48.683 INFO:teuthology.orchestra.run.smithi029.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-16T20:44:48.683 INFO:teuthology.orchestra.run.smithi029.stdout: libradosstriper1 x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 508 k 2024-09-16T20:44:48.684 INFO:teuthology.orchestra.run.smithi029.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-16T20:44:48.684 INFO:teuthology.orchestra.run.smithi029.stdout: librgw2 x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 5.5 M 2024-09-16T20:44:48.684 INFO:teuthology.orchestra.run.smithi029.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-16T20:44:48.684 INFO:teuthology.orchestra.run.smithi029.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-16T20:44:48.684 INFO:teuthology.orchestra.run.smithi029.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-16T20:44:48.684 INFO:teuthology.orchestra.run.smithi029.stdout: python3-ceph-argparse x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 45 k 2024-09-16T20:44:48.684 INFO:teuthology.orchestra.run.smithi029.stdout: python3-ceph-common x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 143 k 2024-09-16T20:44:48.684 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cephfs x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 162 k 2024-09-16T20:44:48.684 INFO:teuthology.orchestra.run.smithi029.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-16T20:44:48.684 INFO:teuthology.orchestra.run.smithi029.stdout: python3-rados x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 320 k 2024-09-16T20:44:48.684 INFO:teuthology.orchestra.run.smithi029.stdout: python3-rbd x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 307 k 2024-09-16T20:44:48.684 INFO:teuthology.orchestra.run.smithi029.stdout: python3-rgw x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 99 k 2024-09-16T20:44:48.684 INFO:teuthology.orchestra.run.smithi029.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-16T20:44:48.685 INFO:teuthology.orchestra.run.smithi029.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-16T20:44:48.685 INFO:teuthology.orchestra.run.smithi029.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-16T20:44:48.685 INFO:teuthology.orchestra.run.smithi029.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-16T20:44:48.685 INFO:teuthology.orchestra.run.smithi029.stdout:Installing weak dependencies: 2024-09-16T20:44:48.685 INFO:teuthology.orchestra.run.smithi029.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-16T20:44:48.685 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:44:48.685 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction Summary 2024-09-16T20:44:48.685 INFO:teuthology.orchestra.run.smithi029.stdout:================================================================================ 2024-09-16T20:44:48.685 INFO:teuthology.orchestra.run.smithi029.stdout:Install 31 Packages 2024-09-16T20:44:48.685 INFO:teuthology.orchestra.run.smithi029.stdout:Upgrade 2 Packages 2024-09-16T20:44:48.685 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:44:48.686 INFO:teuthology.orchestra.run.smithi029.stdout:Total download size: 62 M 2024-09-16T20:44:48.686 INFO:teuthology.orchestra.run.smithi029.stdout:Downloading Packages: 2024-09-16T20:44:49.216 INFO:teuthology.orchestra.run.smithi139.stdout: Preparing : 1/1 2024-09-16T20:44:49.296 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/35 2024-09-16T20:44:49.322 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/35 2024-09-16T20:44:49.356 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/35 2024-09-16T20:44:49.379 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-16T20:44:49.395 INFO:teuthology.orchestra.run.smithi029.stdout:(1/33): ceph-base-19.3.0-4883.g6e8474a0.el9.x86 9.2 MB/s | 5.5 MB 00:00 2024-09-16T20:44:49.399 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-16T20:44:49.446 INFO:teuthology.orchestra.run.smithi029.stdout:(2/33): ceph-selinux-19.3.0-4883.g6e8474a0.el9. 502 kB/s | 25 kB 00:00 2024-09-16T20:44:49.614 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-16T20:44:49.667 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-16T20:44:49.671 INFO:teuthology.orchestra.run.smithi029.stdout:(3/33): ceph-radosgw-19.3.0-4883.g6e8474a0.el9. 12 MB/s | 11 MB 00:00 2024-09-16T20:44:49.722 INFO:teuthology.orchestra.run.smithi029.stdout:(4/33): libcephfs2-19.3.0-4883.g6e8474a0.el9.x8 2.7 MB/s | 755 kB 00:00 2024-09-16T20:44:49.756 INFO:teuthology.orchestra.run.smithi029.stdout:(5/33): libradosstriper1-19.3.0-4883.g6e8474a0. 5.9 MB/s | 508 kB 00:00 2024-09-16T20:44:49.814 INFO:teuthology.orchestra.run.smithi029.stdout:(6/33): python3-ceph-argparse-19.3.0-4883.g6e84 770 kB/s | 45 kB 00:00 2024-09-16T20:44:49.907 INFO:teuthology.orchestra.run.smithi029.stdout:(7/33): python3-ceph-common-19.3.0-4883.g6e8474 1.5 MB/s | 143 kB 00:00 2024-09-16T20:44:50.065 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/35 2024-09-16T20:44:50.115 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : librados2-2:19.3.0-4883.g6e8474a0.el9.x86_64 7/35 2024-09-16T20:44:50.165 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - CRB 900 kB/s | 6.5 MB 00:07 2024-09-16T20:44:50.208 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: librados2-2:19.3.0-4883.g6e8474a0.el9.x86_64 7/35 2024-09-16T20:44:50.295 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-rados-2:19.3.0-4883.g6e8474a0.el9.x86_64 8/35 2024-09-16T20:44:50.307 INFO:teuthology.orchestra.run.smithi029.stdout:(8/33): ceph-common-19.3.0-4883.g6e8474a0.el9.x 14 MB/s | 22 MB 00:01 2024-09-16T20:44:50.325 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libcephfs2-2:19.3.0-4883.g6e8474a0.el9.x86_64 9/35 2024-09-16T20:44:50.341 INFO:teuthology.orchestra.run.smithi029.stdout:(9/33): python3-cephfs-19.3.0-4883.g6e8474a0.el 373 kB/s | 162 kB 00:00 2024-09-16T20:44:50.371 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: libcephfs2-2:19.3.0-4883.g6e8474a0.el9.x86_64 9/35 2024-09-16T20:44:50.375 INFO:teuthology.orchestra.run.smithi029.stdout:(10/33): python3-rados-19.3.0-4883.g6e8474a0.el 4.7 MB/s | 320 kB 00:00 2024-09-16T20:44:50.400 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/35 2024-09-16T20:44:50.444 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-ceph-argparse-2:19.3.0-4883.g6e8474a0.el9. 11/35 2024-09-16T20:44:50.484 INFO:teuthology.orchestra.run.smithi029.stdout:(11/33): librgw2-19.3.0-4883.g6e8474a0.el9.x86_ 7.2 MB/s | 5.5 MB 00:00 2024-09-16T20:44:50.512 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cephfs-2:19.3.0-4883.g6e8474a0.el9.x86_64 12/35 2024-09-16T20:44:50.518 INFO:teuthology.orchestra.run.smithi029.stdout:(12/33): python3-rgw-19.3.0-4883.g6e8474a0.el9. 696 kB/s | 99 kB 00:00 2024-09-16T20:44:50.536 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libradosstriper1-2:19.3.0-4883.g6e8474a0.el9.x86_6 13/35 2024-09-16T20:44:50.554 INFO:teuthology.orchestra.run.smithi029.stdout:(13/33): python3-rbd-19.3.0-4883.g6e8474a0.el9. 1.4 MB/s | 307 kB 00:00 2024-09-16T20:44:50.600 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4883.g6e8474a0.el9.x86_6 13/35 2024-09-16T20:44:50.634 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/35 2024-09-16T20:44:50.671 INFO:teuthology.orchestra.run.smithi029.stdout:(14/33): boost-program-options-1.75.0-8.el9.x86 574 kB/s | 107 kB 00:00 2024-09-16T20:44:50.730 INFO:teuthology.orchestra.run.smithi029.stdout:(15/33): libnbd-1.20.2-2.el9.x86_64.rpm 808 kB/s | 171 kB 00:00 2024-09-16T20:44:50.750 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/35 2024-09-16T20:44:50.755 INFO:teuthology.orchestra.run.smithi029.stdout:(16/33): librabbitmq-0.11.0-7.el9.x86_64.rpm 537 kB/s | 45 kB 00:00 2024-09-16T20:44:50.786 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/35 2024-09-16T20:44:50.789 INFO:teuthology.orchestra.run.smithi029.stdout:(17/33): libpmemobj-1.12.1-1.el9.x86_64.rpm 681 kB/s | 160 kB 00:00 2024-09-16T20:44:50.848 INFO:teuthology.orchestra.run.smithi029.stdout:(18/33): python3-prettytable-0.7.2-27.el9.noarc 722 kB/s | 42 kB 00:00 2024-09-16T20:44:50.849 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/35 2024-09-16T20:44:50.883 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/35 2024-09-16T20:44:50.907 INFO:teuthology.orchestra.run.smithi029.stdout:(19/33): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.9 MB/s | 292 kB 00:00 2024-09-16T20:44:50.912 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/35 2024-09-16T20:44:50.945 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-16T20:44:50.949 INFO:teuthology.orchestra.run.smithi029.stdout:(20/33): librdkafka-1.6.1-102.el9.x86_64.rpm 3.0 MB/s | 662 kB 00:00 2024-09-16T20:44:50.983 INFO:teuthology.orchestra.run.smithi029.stdout:(21/33): qatlib-service-24.02.0-1.el9.x86_64.rp 474 kB/s | 36 kB 00:00 2024-09-16T20:44:50.983 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/35 2024-09-16T20:44:51.017 INFO:teuthology.orchestra.run.smithi029.stdout:(22/33): qatzip-libs-1.2.0-1.el9.x86_64.rpm 705 kB/s | 47 kB 00:00 2024-09-16T20:44:51.051 INFO:teuthology.orchestra.run.smithi029.stdout:(23/33): qatlib-24.02.0-1.el9.x86_64.rpm 1.1 MB/s | 221 kB 00:00 2024-09-16T20:44:51.120 INFO:teuthology.orchestra.run.smithi029.stdout:(24/33): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.2 MB/s | 308 kB 00:00 2024-09-16T20:44:51.152 INFO:teuthology.orchestra.run.smithi029.stdout:(25/33): liboath-2.6.7-2.el9.x86_64.rpm 1.5 MB/s | 49 kB 00:00 2024-09-16T20:44:51.186 INFO:teuthology.orchestra.run.smithi029.stdout:(26/33): libarrow-doc-9.0.0-11.el9.noarch.rpm 219 kB/s | 26 kB 00:00 2024-09-16T20:44:51.220 INFO:teuthology.orchestra.run.smithi029.stdout:(27/33): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-09-16T20:44:51.264 INFO:teuthology.orchestra.run.smithi029.stdout:(28/33): re2-20211101-20.el9.x86_64.rpm 4.3 MB/s | 191 kB 00:00 2024-09-16T20:44:51.300 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 22/35 2024-09-16T20:44:51.331 INFO:teuthology.orchestra.run.smithi029.stdout:(29/33): parquet-libs-9.0.0-11.el9.x86_64.rpm 5.7 MB/s | 839 kB 00:00 2024-09-16T20:44:51.342 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : librbd1-2:19.3.0-4883.g6e8474a0.el9.x86_64 23/35 2024-09-16T20:44:51.389 INFO:teuthology.orchestra.run.smithi029.stdout:(30/33): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-09-16T20:44:51.433 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: librbd1-2:19.3.0-4883.g6e8474a0.el9.x86_64 23/35 2024-09-16T20:44:51.468 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-rbd-2:19.3.0-4883.g6e8474a0.el9.x86_64 24/35 2024-09-16T20:44:51.508 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 25/35 2024-09-16T20:44:51.523 INFO:teuthology.orchestra.run.smithi029.stdout:(31/33): libarrow-9.0.0-11.el9.x86_64.rpm 8.7 MB/s | 4.4 MB 00:00 2024-09-16T20:44:51.674 INFO:teuthology.orchestra.run.smithi029.stdout:(32/33): librados2-19.3.0-4883.g6e8474a0.el9.x8 10 MB/s | 3.5 MB 00:00 2024-09-16T20:44:51.766 INFO:teuthology.orchestra.run.smithi029.stdout:(33/33): librbd1-19.3.0-4883.g6e8474a0.el9.x86_ 8.6 MB/s | 3.2 MB 00:00 2024-09-16T20:44:51.772 INFO:teuthology.orchestra.run.smithi029.stdout:-------------------------------------------------------------------------------- 2024-09-16T20:44:51.772 INFO:teuthology.orchestra.run.smithi029.stdout:Total 20 MB/s | 62 MB 00:03 2024-09-16T20:44:51.869 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - Extras packages 43 kB/s | 19 kB 00:00 2024-09-16T20:44:51.895 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction check 2024-09-16T20:44:51.937 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction check succeeded. 2024-09-16T20:44:51.937 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction test 2024-09-16T20:44:51.998 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 26/35 2024-09-16T20:44:52.028 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : librgw2-2:19.3.0-4883.g6e8474a0.el9.x86_64 27/35 2024-09-16T20:44:52.100 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: librgw2-2:19.3.0-4883.g6e8474a0.el9.x86_64 27/35 2024-09-16T20:44:52.178 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-rgw-2:19.3.0-4883.g6e8474a0.el9.x86_64 28/35 2024-09-16T20:44:52.213 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-ceph-common-2:19.3.0-4883.g6e8474a0.el9.x8 29/35 2024-09-16T20:44:52.700 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction test succeeded. 2024-09-16T20:44:52.700 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction 2024-09-16T20:44:52.814 INFO:teuthology.orchestra.run.smithi043.stdout:Extra Packages for Enterprise Linux 28 MB/s | 23 MB 00:00 2024-09-16T20:44:54.239 INFO:teuthology.orchestra.run.smithi029.stdout: Preparing : 1/1 2024-09-16T20:44:54.344 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/35 2024-09-16T20:44:54.381 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/35 2024-09-16T20:44:54.425 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/35 2024-09-16T20:44:54.457 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-16T20:44:54.481 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-16T20:44:54.602 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-common-2:19.3.0-4883.g6e8474a0.el9.x86_64 30/35 2024-09-16T20:44:54.643 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-common-2:19.3.0-4883.g6e8474a0.el9.x86_64 30/35 2024-09-16T20:44:54.679 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-16T20:44:54.749 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-16T20:44:55.172 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/35 2024-09-16T20:44:55.211 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-common-2:19.3.0-4883.g6e8474a0.el9.x86_64 30/35 2024-09-16T20:44:55.246 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-base-2:19.3.0-4883.g6e8474a0.el9.x86_64 31/35 2024-09-16T20:44:55.328 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-base-2:19.3.0-4883.g6e8474a0.el9.x86_64 31/35 2024-09-16T20:44:55.329 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-16T20:44:55.329 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-16T20:44:55.329 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:44:55.333 INFO:teuthology.orchestra.run.smithi029.stdout: Upgrading : librados2-2:19.3.0-4883.g6e8474a0.el9.x86_64 7/35 2024-09-16T20:44:55.364 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-selinux-2:19.3.0-4883.g6e8474a0.el9.x86_64 32/35 2024-09-16T20:44:55.430 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: librados2-2:19.3.0-4883.g6e8474a0.el9.x86_64 7/35 2024-09-16T20:44:55.576 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-rados-2:19.3.0-4883.g6e8474a0.el9.x86_64 8/35 2024-09-16T20:44:55.678 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libcephfs2-2:19.3.0-4883.g6e8474a0.el9.x86_64 9/35 2024-09-16T20:44:55.742 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: libcephfs2-2:19.3.0-4883.g6e8474a0.el9.x86_64 9/35 2024-09-16T20:44:55.788 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/35 2024-09-16T20:44:55.832 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-ceph-argparse-2:19.3.0-4883.g6e8474a0.el9. 11/35 2024-09-16T20:44:55.911 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-cephfs-2:19.3.0-4883.g6e8474a0.el9.x86_64 12/35 2024-09-16T20:44:55.942 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libradosstriper1-2:19.3.0-4883.g6e8474a0.el9.x86_6 13/35 2024-09-16T20:44:56.007 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4883.g6e8474a0.el9.x86_6 13/35 2024-09-16T20:44:56.050 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/35 2024-09-16T20:44:56.173 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/35 2024-09-16T20:44:56.212 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/35 2024-09-16T20:44:56.261 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/35 2024-09-16T20:44:56.302 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/35 2024-09-16T20:44:56.342 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/35 2024-09-16T20:44:56.384 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-16T20:44:56.421 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/35 2024-09-16T20:44:56.775 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 22/35 2024-09-16T20:44:56.841 INFO:teuthology.orchestra.run.smithi029.stdout: Upgrading : librbd1-2:19.3.0-4883.g6e8474a0.el9.x86_64 23/35 2024-09-16T20:44:56.926 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: librbd1-2:19.3.0-4883.g6e8474a0.el9.x86_64 23/35 2024-09-16T20:44:56.964 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-rbd-2:19.3.0-4883.g6e8474a0.el9.x86_64 24/35 2024-09-16T20:44:57.010 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 25/35 2024-09-16T20:44:57.537 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 26/35 2024-09-16T20:44:57.573 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : librgw2-2:19.3.0-4883.g6e8474a0.el9.x86_64 27/35 2024-09-16T20:44:57.637 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: librgw2-2:19.3.0-4883.g6e8474a0.el9.x86_64 27/35 2024-09-16T20:44:57.718 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-rgw-2:19.3.0-4883.g6e8474a0.el9.x86_64 28/35 2024-09-16T20:44:57.760 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-ceph-common-2:19.3.0-4883.g6e8474a0.el9.x8 29/35 2024-09-16T20:44:58.488 INFO:teuthology.orchestra.run.smithi043.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-09-16T20:45:00.351 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-common-2:19.3.0-4883.g6e8474a0.el9.x86_64 30/35 2024-09-16T20:45:00.394 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-common-2:19.3.0-4883.g6e8474a0.el9.x86_64 30/35 2024-09-16T20:45:00.825 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-16T20:45:00.836 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T20:45:00.836 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-09-16T20:45:00.837 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T20:45:00.837 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-16T20:45:00.837 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-radosgw x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 11 M 2024-09-16T20:45:00.837 INFO:teuthology.orchestra.run.smithi043.stdout:Upgrading: 2024-09-16T20:45:00.837 INFO:teuthology.orchestra.run.smithi043.stdout: librados2 x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 3.5 M 2024-09-16T20:45:00.837 INFO:teuthology.orchestra.run.smithi043.stdout: librbd1 x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 3.2 M 2024-09-16T20:45:00.837 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-09-16T20:45:00.837 INFO:teuthology.orchestra.run.smithi043.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-16T20:45:00.837 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-base x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 5.5 M 2024-09-16T20:45:00.837 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-common x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 22 M 2024-09-16T20:45:00.837 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-selinux x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 25 k 2024-09-16T20:45:00.837 INFO:teuthology.orchestra.run.smithi043.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-16T20:45:00.838 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-16T20:45:00.838 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-16T20:45:00.838 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs2 x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 755 k 2024-09-16T20:45:00.838 INFO:teuthology.orchestra.run.smithi043.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-16T20:45:00.838 INFO:teuthology.orchestra.run.smithi043.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-16T20:45:00.838 INFO:teuthology.orchestra.run.smithi043.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-16T20:45:00.838 INFO:teuthology.orchestra.run.smithi043.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-16T20:45:00.838 INFO:teuthology.orchestra.run.smithi043.stdout: libradosstriper1 x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 508 k 2024-09-16T20:45:00.838 INFO:teuthology.orchestra.run.smithi043.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-16T20:45:00.838 INFO:teuthology.orchestra.run.smithi043.stdout: librgw2 x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 5.5 M 2024-09-16T20:45:00.838 INFO:teuthology.orchestra.run.smithi043.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-16T20:45:00.838 INFO:teuthology.orchestra.run.smithi043.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-16T20:45:00.838 INFO:teuthology.orchestra.run.smithi043.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-16T20:45:00.838 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-argparse x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 45 k 2024-09-16T20:45:00.839 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-common x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 143 k 2024-09-16T20:45:00.839 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cephfs x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 162 k 2024-09-16T20:45:00.839 INFO:teuthology.orchestra.run.smithi043.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-16T20:45:00.839 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rados x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 320 k 2024-09-16T20:45:00.839 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rbd x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 307 k 2024-09-16T20:45:00.839 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rgw x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 99 k 2024-09-16T20:45:00.839 INFO:teuthology.orchestra.run.smithi043.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-16T20:45:00.839 INFO:teuthology.orchestra.run.smithi043.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-16T20:45:00.839 INFO:teuthology.orchestra.run.smithi043.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-16T20:45:00.839 INFO:teuthology.orchestra.run.smithi043.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-16T20:45:00.839 INFO:teuthology.orchestra.run.smithi043.stdout:Installing weak dependencies: 2024-09-16T20:45:00.839 INFO:teuthology.orchestra.run.smithi043.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-16T20:45:00.839 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:45:00.840 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-16T20:45:00.840 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T20:45:00.840 INFO:teuthology.orchestra.run.smithi043.stdout:Install 31 Packages 2024-09-16T20:45:00.840 INFO:teuthology.orchestra.run.smithi043.stdout:Upgrade 2 Packages 2024-09-16T20:45:00.840 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:45:00.840 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 62 M 2024-09-16T20:45:00.840 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-16T20:45:00.991 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-common-2:19.3.0-4883.g6e8474a0.el9.x86_64 30/35 2024-09-16T20:45:01.044 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-base-2:19.3.0-4883.g6e8474a0.el9.x86_64 31/35 2024-09-16T20:45:01.159 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-base-2:19.3.0-4883.g6e8474a0.el9.x86_64 31/35 2024-09-16T20:45:01.159 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-16T20:45:01.159 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-16T20:45:01.159 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:45:01.257 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-selinux-2:19.3.0-4883.g6e8474a0.el9.x86_64 32/35 2024-09-16T20:45:01.654 INFO:teuthology.orchestra.run.smithi043.stdout:(1/33): ceph-base-19.3.0-4883.g6e8474a0.el9.x86 8.7 MB/s | 5.5 MB 00:00 2024-09-16T20:45:01.721 INFO:teuthology.orchestra.run.smithi043.stdout:(2/33): ceph-selinux-19.3.0-4883.g6e8474a0.el9. 377 kB/s | 25 kB 00:00 2024-09-16T20:45:01.830 INFO:teuthology.orchestra.run.smithi043.stdout:(3/33): libcephfs2-19.3.0-4883.g6e8474a0.el9.x8 6.8 MB/s | 755 kB 00:00 2024-09-16T20:45:01.997 INFO:teuthology.orchestra.run.smithi043.stdout:(4/33): ceph-radosgw-19.3.0-4883.g6e8474a0.el9. 11 MB/s | 11 MB 00:00 2024-09-16T20:45:02.039 INFO:teuthology.orchestra.run.smithi043.stdout:(5/33): libradosstriper1-19.3.0-4883.g6e8474a0. 2.4 MB/s | 508 kB 00:00 2024-09-16T20:45:02.247 INFO:teuthology.orchestra.run.smithi043.stdout:(6/33): ceph-common-19.3.0-4883.g6e8474a0.el9.x 18 MB/s | 22 MB 00:01 2024-09-16T20:45:02.273 INFO:teuthology.orchestra.run.smithi043.stdout:(7/33): python3-ceph-argparse-19.3.0-4883.g6e84 193 kB/s | 45 kB 00:00 2024-09-16T20:45:02.298 INFO:teuthology.orchestra.run.smithi043.stdout:(8/33): python3-ceph-common-19.3.0-4883.g6e8474 2.8 MB/s | 143 kB 00:00 2024-09-16T20:45:02.323 INFO:teuthology.orchestra.run.smithi043.stdout:(9/33): python3-cephfs-19.3.0-4883.g6e8474a0.el 3.1 MB/s | 162 kB 00:00 2024-09-16T20:45:02.348 INFO:teuthology.orchestra.run.smithi043.stdout:(10/33): python3-rados-19.3.0-4883.g6e8474a0.el 6.3 MB/s | 320 kB 00:00 2024-09-16T20:45:02.374 INFO:teuthology.orchestra.run.smithi043.stdout:(11/33): python3-rbd-19.3.0-4883.g6e8474a0.el9. 6.0 MB/s | 307 kB 00:00 2024-09-16T20:45:02.399 INFO:teuthology.orchestra.run.smithi043.stdout:(12/33): python3-rgw-19.3.0-4883.g6e8474a0.el9. 1.9 MB/s | 99 kB 00:00 2024-09-16T20:45:02.491 INFO:teuthology.orchestra.run.smithi043.stdout:(13/33): librgw2-19.3.0-4883.g6e8474a0.el9.x86_ 11 MB/s | 5.5 MB 00:00 2024-09-16T20:45:02.566 INFO:teuthology.orchestra.run.smithi043.stdout:(14/33): boost-program-options-1.75.0-8.el9.x86 557 kB/s | 107 kB 00:00 2024-09-16T20:45:02.616 INFO:teuthology.orchestra.run.smithi043.stdout:(15/33): librabbitmq-0.11.0-7.el9.x86_64.rpm 904 kB/s | 45 kB 00:00 2024-09-16T20:45:02.650 INFO:teuthology.orchestra.run.smithi043.stdout:(16/33): libnbd-1.20.2-2.el9.x86_64.rpm 679 kB/s | 171 kB 00:00 2024-09-16T20:45:02.676 INFO:teuthology.orchestra.run.smithi043.stdout:(17/33): libpmemobj-1.12.1-1.el9.x86_64.rpm 869 kB/s | 160 kB 00:00 2024-09-16T20:45:02.734 INFO:teuthology.orchestra.run.smithi043.stdout:(18/33): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-09-16T20:45:02.759 INFO:teuthology.orchestra.run.smithi043.stdout:(19/33): librdkafka-1.6.1-102.el9.x86_64.rpm 4.5 MB/s | 662 kB 00:00 2024-09-16T20:45:02.810 INFO:teuthology.orchestra.run.smithi043.stdout:(20/33): qatlib-service-24.02.0-1.el9.x86_64.rp 721 kB/s | 36 kB 00:00 2024-09-16T20:45:02.835 INFO:teuthology.orchestra.run.smithi043.stdout:(21/33): python3-prettytable-0.7.2-27.el9.noarc 266 kB/s | 42 kB 00:00 2024-09-16T20:45:02.869 INFO:teuthology.orchestra.run.smithi043.stdout:(22/33): qatzip-libs-1.2.0-1.el9.x86_64.rpm 807 kB/s | 47 kB 00:00 2024-09-16T20:45:02.910 INFO:teuthology.orchestra.run.smithi043.stdout:(23/33): gperftools-libs-2.9.1-3.el9.x86_64.rpm 4.0 MB/s | 308 kB 00:00 2024-09-16T20:45:02.936 INFO:teuthology.orchestra.run.smithi043.stdout:(24/33): qatlib-24.02.0-1.el9.x86_64.rpm 1.1 MB/s | 221 kB 00:00 2024-09-16T20:45:02.961 INFO:teuthology.orchestra.run.smithi043.stdout:(25/33): libarrow-doc-9.0.0-11.el9.noarch.rpm 538 kB/s | 26 kB 00:00 2024-09-16T20:45:02.994 INFO:teuthology.orchestra.run.smithi043.stdout:(26/33): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-16T20:45:03.028 INFO:teuthology.orchestra.run.smithi043.stdout:(27/33): liboath-2.6.7-2.el9.x86_64.rpm 528 kB/s | 49 kB 00:00 2024-09-16T20:45:03.062 INFO:teuthology.orchestra.run.smithi043.stdout:(28/33): re2-20211101-20.el9.x86_64.rpm 5.6 MB/s | 191 kB 00:00 2024-09-16T20:45:03.103 INFO:teuthology.orchestra.run.smithi043.stdout:(29/33): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.7 MB/s | 839 kB 00:00 2024-09-16T20:45:03.212 INFO:teuthology.orchestra.run.smithi043.stdout:(30/33): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-09-16T20:45:03.279 INFO:teuthology.orchestra.run.smithi043.stdout:(31/33): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-09-16T20:45:03.338 INFO:teuthology.orchestra.run.smithi043.stdout:(32/33): librados2-19.3.0-4883.g6e8474a0.el9.x8 15 MB/s | 3.5 MB 00:00 2024-09-16T20:45:03.454 INFO:teuthology.orchestra.run.smithi043.stdout:(33/33): librbd1-19.3.0-4883.g6e8474a0.el9.x86_ 13 MB/s | 3.2 MB 00:00 2024-09-16T20:45:03.459 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-16T20:45:03.459 INFO:teuthology.orchestra.run.smithi043.stdout:Total 24 MB/s | 62 MB 00:02 2024-09-16T20:45:03.573 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-16T20:45:03.616 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-16T20:45:03.617 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-16T20:45:04.299 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-16T20:45:04.300 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-16T20:45:05.169 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4883.g6e8474a0.el9.x86_64 32/35 2024-09-16T20:45:05.169 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /sys 2024-09-16T20:45:05.170 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /proc 2024-09-16T20:45:05.170 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /mnt 2024-09-16T20:45:05.170 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /var/tmp 2024-09-16T20:45:05.170 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /home 2024-09-16T20:45:05.170 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /root 2024-09-16T20:45:05.170 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /tmp 2024-09-16T20:45:05.170 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:45:05.858 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-16T20:45:05.941 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/35 2024-09-16T20:45:05.976 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/35 2024-09-16T20:45:06.001 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/35 2024-09-16T20:45:06.024 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-16T20:45:06.049 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-16T20:45:06.195 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-radosgw-2:19.3.0-4883.g6e8474a0.el9.x86_64 33/35 2024-09-16T20:45:06.202 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-16T20:45:06.221 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4883.g6e8474a0.el9.x86_64 33/35 2024-09-16T20:45:06.221 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T20:45:06.221 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-16T20:45:06.222 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T20:45:06.222 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T20:45:06.222 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:45:06.223 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-16T20:45:06.255 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-16T20:45:06.290 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-16T20:45:06.290 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-16T20:45:06.641 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/35 2024-09-16T20:45:06.728 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : librados2-2:19.3.0-4883.g6e8474a0.el9.x86_64 7/35 2024-09-16T20:45:06.821 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librados2-2:19.3.0-4883.g6e8474a0.el9.x86_64 7/35 2024-09-16T20:45:06.911 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rados-2:19.3.0-4883.g6e8474a0.el9.x86_64 8/35 2024-09-16T20:45:06.937 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libcephfs2-2:19.3.0-4883.g6e8474a0.el9.x86_64 9/35 2024-09-16T20:45:07.000 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libcephfs2-2:19.3.0-4883.g6e8474a0.el9.x86_64 9/35 2024-09-16T20:45:07.029 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/35 2024-09-16T20:45:07.182 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-ceph-argparse-2:19.3.0-4883.g6e8474a0.el9. 11/35 2024-09-16T20:45:07.252 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cephfs-2:19.3.0-4883.g6e8474a0.el9.x86_64 12/35 2024-09-16T20:45:07.281 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libradosstriper1-2:19.3.0-4883.g6e8474a0.el9.x86_6 13/35 2024-09-16T20:45:07.337 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4883.g6e8474a0.el9.x86_6 13/35 2024-09-16T20:45:07.371 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/35 2024-09-16T20:45:07.483 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/35 2024-09-16T20:45:07.491 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-16T20:45:07.491 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-base-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/35 2024-09-16T20:45:07.491 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-common-2:19.3.0-4883.g6e8474a0.el9.x86_64 2/35 2024-09-16T20:45:07.491 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-radosgw-2:19.3.0-4883.g6e8474a0.el9.x86_64 3/35 2024-09-16T20:45:07.491 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-selinux-2:19.3.0-4883.g6e8474a0.el9.x86_64 4/35 2024-09-16T20:45:07.491 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libcephfs2-2:19.3.0-4883.g6e8474a0.el9.x86_64 5/35 2024-09-16T20:45:07.491 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libradosstriper1-2:19.3.0-4883.g6e8474a0.el9.x86_6 6/35 2024-09-16T20:45:07.491 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librgw2-2:19.3.0-4883.g6e8474a0.el9.x86_64 7/35 2024-09-16T20:45:07.491 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4883.g6e8474a0.el9. 8/35 2024-09-16T20:45:07.491 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-ceph-common-2:19.3.0-4883.g6e8474a0.el9.x8 9/35 2024-09-16T20:45:07.491 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cephfs-2:19.3.0-4883.g6e8474a0.el9.x86_64 10/35 2024-09-16T20:45:07.491 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-rados-2:19.3.0-4883.g6e8474a0.el9.x86_64 11/35 2024-09-16T20:45:07.492 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-rbd-2:19.3.0-4883.g6e8474a0.el9.x86_64 12/35 2024-09-16T20:45:07.492 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-rgw-2:19.3.0-4883.g6e8474a0.el9.x86_64 13/35 2024-09-16T20:45:07.492 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/35 2024-09-16T20:45:07.492 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 15/35 2024-09-16T20:45:07.492 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 16/35 2024-09-16T20:45:07.492 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 17/35 2024-09-16T20:45:07.492 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 18/35 2024-09-16T20:45:07.492 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 19/35 2024-09-16T20:45:07.492 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-16T20:45:07.492 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 21/35 2024-09-16T20:45:07.492 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 22/35 2024-09-16T20:45:07.492 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 23/35 2024-09-16T20:45:07.492 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 24/35 2024-09-16T20:45:07.493 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 25/35 2024-09-16T20:45:07.493 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 26/35 2024-09-16T20:45:07.493 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 27/35 2024-09-16T20:45:07.494 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 28/35 2024-09-16T20:45:07.494 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 29/35 2024-09-16T20:45:07.494 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : re2-1:20211101-20.el9.x86_64 30/35 2024-09-16T20:45:07.494 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 31/35 2024-09-16T20:45:07.494 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librados2-2:19.3.0-4883.g6e8474a0.el9.x86_64 32/35 2024-09-16T20:45:07.494 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 33/35 2024-09-16T20:45:07.494 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librbd1-2:19.3.0-4883.g6e8474a0.el9.x86_64 34/35 2024-09-16T20:45:07.523 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/35 2024-09-16T20:45:07.568 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/35 2024-09-16T20:45:07.596 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/35 2024-09-16T20:45:07.625 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/35 2024-09-16T20:45:07.658 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-16T20:45:07.695 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/35 2024-09-16T20:45:08.038 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 22/35 2024-09-16T20:45:08.080 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : librbd1-2:19.3.0-4883.g6e8474a0.el9.x86_64 23/35 2024-09-16T20:45:08.156 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librbd1-2:19.3.0-4883.g6e8474a0.el9.x86_64 23/35 2024-09-16T20:45:08.192 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rbd-2:19.3.0-4883.g6e8474a0.el9.x86_64 24/35 2024-09-16T20:45:08.237 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 25/35 2024-09-16T20:45:08.506 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 35/35 2024-09-16T20:45:08.506 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:45:08.506 INFO:teuthology.orchestra.run.smithi139.stdout:Upgraded: 2024-09-16T20:45:08.507 INFO:teuthology.orchestra.run.smithi139.stdout: librados2-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:08.507 INFO:teuthology.orchestra.run.smithi139.stdout: librbd1-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:08.507 INFO:teuthology.orchestra.run.smithi139.stdout:Installed: 2024-09-16T20:45:08.507 INFO:teuthology.orchestra.run.smithi139.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-16T20:45:08.507 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-base-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:08.507 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-common-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:08.507 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-radosgw-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:08.507 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-selinux-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:08.507 INFO:teuthology.orchestra.run.smithi139.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-16T20:45:08.507 INFO:teuthology.orchestra.run.smithi139.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-16T20:45:08.507 INFO:teuthology.orchestra.run.smithi139.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-16T20:45:08.507 INFO:teuthology.orchestra.run.smithi139.stdout: libcephfs2-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:08.508 INFO:teuthology.orchestra.run.smithi139.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-16T20:45:08.508 INFO:teuthology.orchestra.run.smithi139.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-16T20:45:08.508 INFO:teuthology.orchestra.run.smithi139.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-16T20:45:08.508 INFO:teuthology.orchestra.run.smithi139.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-16T20:45:08.508 INFO:teuthology.orchestra.run.smithi139.stdout: libradosstriper1-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:08.508 INFO:teuthology.orchestra.run.smithi139.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-16T20:45:08.508 INFO:teuthology.orchestra.run.smithi139.stdout: librgw2-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:08.508 INFO:teuthology.orchestra.run.smithi139.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-16T20:45:08.508 INFO:teuthology.orchestra.run.smithi139.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-16T20:45:08.508 INFO:teuthology.orchestra.run.smithi139.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-16T20:45:08.508 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ceph-argparse-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:08.508 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ceph-common-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:08.509 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cephfs-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:08.509 INFO:teuthology.orchestra.run.smithi139.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-16T20:45:08.509 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rados-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:08.509 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rbd-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:08.509 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rgw-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:08.509 INFO:teuthology.orchestra.run.smithi139.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-16T20:45:08.509 INFO:teuthology.orchestra.run.smithi139.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-16T20:45:08.509 INFO:teuthology.orchestra.run.smithi139.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-16T20:45:08.510 INFO:teuthology.orchestra.run.smithi139.stdout: re2-1:20211101-20.el9.x86_64 2024-09-16T20:45:08.510 INFO:teuthology.orchestra.run.smithi139.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-16T20:45:08.510 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:45:08.510 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-16T20:45:08.754 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 26/35 2024-09-16T20:45:08.782 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librgw2-2:19.3.0-4883.g6e8474a0.el9.x86_64 27/35 2024-09-16T20:45:08.817 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum -y install ceph-test 2024-09-16T20:45:08.838 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librgw2-2:19.3.0-4883.g6e8474a0.el9.x86_64 27/35 2024-09-16T20:45:08.918 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rgw-2:19.3.0-4883.g6e8474a0.el9.x86_64 28/35 2024-09-16T20:45:08.959 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-ceph-common-2:19.3.0-4883.g6e8474a0.el9.x8 29/35 2024-09-16T20:45:09.401 INFO:teuthology.orchestra.run.smithi139.stdout:Last metadata expiration check: 0:00:28 ago on Mon 16 Sep 2024 08:44:41 PM UTC. 2024-09-16T20:45:09.535 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-16T20:45:09.535 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-16T20:45:09.536 INFO:teuthology.orchestra.run.smithi139.stdout: Package Arch Version Repository Size 2024-09-16T20:45:09.536 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-16T20:45:09.536 INFO:teuthology.orchestra.run.smithi139.stdout:Installing: 2024-09-16T20:45:09.536 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-test x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 50 M 2024-09-16T20:45:09.536 INFO:teuthology.orchestra.run.smithi139.stdout:Installing dependencies: 2024-09-16T20:45:09.536 INFO:teuthology.orchestra.run.smithi139.stdout: libcephsqlite x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 161 k 2024-09-16T20:45:09.536 INFO:teuthology.orchestra.run.smithi139.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-16T20:45:09.536 INFO:teuthology.orchestra.run.smithi139.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-16T20:45:09.536 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:45:09.536 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction Summary 2024-09-16T20:45:09.536 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-16T20:45:09.536 INFO:teuthology.orchestra.run.smithi139.stdout:Install 4 Packages 2024-09-16T20:45:09.537 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:45:09.537 INFO:teuthology.orchestra.run.smithi139.stdout:Total download size: 51 M 2024-09-16T20:45:09.537 INFO:teuthology.orchestra.run.smithi139.stdout:Installed size: 217 M 2024-09-16T20:45:09.537 INFO:teuthology.orchestra.run.smithi139.stdout:Downloading Packages: 2024-09-16T20:45:09.777 INFO:teuthology.orchestra.run.smithi139.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.4 MB/s | 304 kB 00:00 2024-09-16T20:45:09.803 INFO:teuthology.orchestra.run.smithi139.stdout:(2/4): libcephsqlite-19.3.0-4883.g6e8474a0.el9. 664 kB/s | 161 kB 00:00 2024-09-16T20:45:09.828 INFO:teuthology.orchestra.run.smithi139.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-16T20:45:11.128 INFO:teuthology.orchestra.run.smithi139.stdout:(4/4): ceph-test-19.3.0-4883.g6e8474a0.el9.x86_ 32 MB/s | 50 MB 00:01 2024-09-16T20:45:11.130 INFO:teuthology.orchestra.run.smithi139.stdout:-------------------------------------------------------------------------------- 2024-09-16T20:45:11.130 INFO:teuthology.orchestra.run.smithi139.stdout:Total 32 MB/s | 51 MB 00:01 2024-09-16T20:45:11.157 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction check 2024-09-16T20:45:11.177 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction check succeeded. 2024-09-16T20:45:11.177 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction test 2024-09-16T20:45:11.405 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-common-2:19.3.0-4883.g6e8474a0.el9.x86_64 30/35 2024-09-16T20:45:11.439 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-common-2:19.3.0-4883.g6e8474a0.el9.x86_64 30/35 2024-09-16T20:45:11.556 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction test succeeded. 2024-09-16T20:45:11.556 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction 2024-09-16T20:45:12.040 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-common-2:19.3.0-4883.g6e8474a0.el9.x86_64 30/35 2024-09-16T20:45:12.076 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-base-2:19.3.0-4883.g6e8474a0.el9.x86_64 31/35 2024-09-16T20:45:12.112 INFO:teuthology.orchestra.run.smithi139.stdout: Preparing : 1/1 2024-09-16T20:45:12.166 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-base-2:19.3.0-4883.g6e8474a0.el9.x86_64 31/35 2024-09-16T20:45:12.166 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-16T20:45:12.166 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-16T20:45:12.166 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:45:12.191 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-16T20:45:12.210 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-selinux-2:19.3.0-4883.g6e8474a0.el9.x86_64 32/35 2024-09-16T20:45:12.236 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-16T20:45:12.263 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libcephsqlite-2:19.3.0-4883.g6e8474a0.el9.x86_64 3/4 2024-09-16T20:45:13.940 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4883.g6e8474a0.el9.x86_64 32/35 2024-09-16T20:45:13.941 INFO:teuthology.orchestra.run.smithi029.stdout:skipping the directory /sys 2024-09-16T20:45:13.941 INFO:teuthology.orchestra.run.smithi029.stdout:skipping the directory /proc 2024-09-16T20:45:13.941 INFO:teuthology.orchestra.run.smithi029.stdout:skipping the directory /mnt 2024-09-16T20:45:13.941 INFO:teuthology.orchestra.run.smithi029.stdout:skipping the directory /var/tmp 2024-09-16T20:45:13.941 INFO:teuthology.orchestra.run.smithi029.stdout:skipping the directory /home 2024-09-16T20:45:13.941 INFO:teuthology.orchestra.run.smithi029.stdout:skipping the directory /root 2024-09-16T20:45:13.941 INFO:teuthology.orchestra.run.smithi029.stdout:skipping the directory /tmp 2024-09-16T20:45:13.941 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:45:15.008 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-radosgw-2:19.3.0-4883.g6e8474a0.el9.x86_64 33/35 2024-09-16T20:45:15.035 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4883.g6e8474a0.el9.x86_64 33/35 2024-09-16T20:45:15.035 INFO:teuthology.orchestra.run.smithi029.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T20:45:15.035 INFO:teuthology.orchestra.run.smithi029.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-16T20:45:15.035 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T20:45:15.035 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T20:45:15.036 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:45:15.037 INFO:teuthology.orchestra.run.smithi029.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-16T20:45:15.136 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-16T20:45:15.136 INFO:teuthology.orchestra.run.smithi029.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-16T20:45:16.799 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-16T20:45:16.799 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-base-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/35 2024-09-16T20:45:16.799 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-common-2:19.3.0-4883.g6e8474a0.el9.x86_64 2/35 2024-09-16T20:45:16.800 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-radosgw-2:19.3.0-4883.g6e8474a0.el9.x86_64 3/35 2024-09-16T20:45:16.800 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-selinux-2:19.3.0-4883.g6e8474a0.el9.x86_64 4/35 2024-09-16T20:45:16.800 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libcephfs2-2:19.3.0-4883.g6e8474a0.el9.x86_64 5/35 2024-09-16T20:45:16.800 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libradosstriper1-2:19.3.0-4883.g6e8474a0.el9.x86_6 6/35 2024-09-16T20:45:16.800 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : librgw2-2:19.3.0-4883.g6e8474a0.el9.x86_64 7/35 2024-09-16T20:45:16.800 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4883.g6e8474a0.el9. 8/35 2024-09-16T20:45:16.800 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-ceph-common-2:19.3.0-4883.g6e8474a0.el9.x8 9/35 2024-09-16T20:45:16.800 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-cephfs-2:19.3.0-4883.g6e8474a0.el9.x86_64 10/35 2024-09-16T20:45:16.800 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-rados-2:19.3.0-4883.g6e8474a0.el9.x86_64 11/35 2024-09-16T20:45:16.800 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-rbd-2:19.3.0-4883.g6e8474a0.el9.x86_64 12/35 2024-09-16T20:45:16.801 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-rgw-2:19.3.0-4883.g6e8474a0.el9.x86_64 13/35 2024-09-16T20:45:16.801 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/35 2024-09-16T20:45:16.801 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 15/35 2024-09-16T20:45:16.801 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 16/35 2024-09-16T20:45:16.801 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 17/35 2024-09-16T20:45:16.801 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 18/35 2024-09-16T20:45:16.801 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 19/35 2024-09-16T20:45:16.801 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-16T20:45:16.801 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 21/35 2024-09-16T20:45:16.801 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 22/35 2024-09-16T20:45:16.801 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 23/35 2024-09-16T20:45:16.801 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 24/35 2024-09-16T20:45:16.802 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 25/35 2024-09-16T20:45:16.803 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 26/35 2024-09-16T20:45:16.803 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 27/35 2024-09-16T20:45:16.803 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 28/35 2024-09-16T20:45:16.803 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 29/35 2024-09-16T20:45:16.803 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : re2-1:20211101-20.el9.x86_64 30/35 2024-09-16T20:45:16.803 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 31/35 2024-09-16T20:45:16.803 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : librados2-2:19.3.0-4883.g6e8474a0.el9.x86_64 32/35 2024-09-16T20:45:16.803 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 33/35 2024-09-16T20:45:16.803 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : librbd1-2:19.3.0-4883.g6e8474a0.el9.x86_64 34/35 2024-09-16T20:45:17.025 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4883.g6e8474a0.el9.x86_64 3/4 2024-09-16T20:45:17.110 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-test-2:19.3.0-4883.g6e8474a0.el9.x86_64 4/4 2024-09-16T20:45:17.642 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 35/35 2024-09-16T20:45:17.642 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:45:17.642 INFO:teuthology.orchestra.run.smithi029.stdout:Upgraded: 2024-09-16T20:45:17.642 INFO:teuthology.orchestra.run.smithi029.stdout: librados2-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:17.642 INFO:teuthology.orchestra.run.smithi029.stdout: librbd1-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:17.643 INFO:teuthology.orchestra.run.smithi029.stdout:Installed: 2024-09-16T20:45:17.643 INFO:teuthology.orchestra.run.smithi029.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-16T20:45:17.643 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-base-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:17.643 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-common-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:17.643 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-radosgw-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:17.643 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-selinux-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:17.643 INFO:teuthology.orchestra.run.smithi029.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-16T20:45:17.643 INFO:teuthology.orchestra.run.smithi029.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-16T20:45:17.643 INFO:teuthology.orchestra.run.smithi029.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-16T20:45:17.643 INFO:teuthology.orchestra.run.smithi029.stdout: libcephfs2-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:17.643 INFO:teuthology.orchestra.run.smithi029.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-16T20:45:17.643 INFO:teuthology.orchestra.run.smithi029.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-16T20:45:17.644 INFO:teuthology.orchestra.run.smithi029.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-16T20:45:17.644 INFO:teuthology.orchestra.run.smithi029.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-16T20:45:17.644 INFO:teuthology.orchestra.run.smithi029.stdout: libradosstriper1-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:17.644 INFO:teuthology.orchestra.run.smithi029.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-16T20:45:17.644 INFO:teuthology.orchestra.run.smithi029.stdout: librgw2-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:17.644 INFO:teuthology.orchestra.run.smithi029.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-16T20:45:17.644 INFO:teuthology.orchestra.run.smithi029.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-16T20:45:17.644 INFO:teuthology.orchestra.run.smithi029.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-16T20:45:17.644 INFO:teuthology.orchestra.run.smithi029.stdout: python3-ceph-argparse-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:17.644 INFO:teuthology.orchestra.run.smithi029.stdout: python3-ceph-common-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:17.644 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cephfs-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:17.644 INFO:teuthology.orchestra.run.smithi029.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-16T20:45:17.645 INFO:teuthology.orchestra.run.smithi029.stdout: python3-rados-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:17.645 INFO:teuthology.orchestra.run.smithi029.stdout: python3-rbd-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:17.645 INFO:teuthology.orchestra.run.smithi029.stdout: python3-rgw-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:17.645 INFO:teuthology.orchestra.run.smithi029.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-16T20:45:17.645 INFO:teuthology.orchestra.run.smithi029.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-16T20:45:17.645 INFO:teuthology.orchestra.run.smithi029.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-16T20:45:17.645 INFO:teuthology.orchestra.run.smithi029.stdout: re2-1:20211101-20.el9.x86_64 2024-09-16T20:45:17.645 INFO:teuthology.orchestra.run.smithi029.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-16T20:45:17.645 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:45:17.645 INFO:teuthology.orchestra.run.smithi029.stdout:Complete! 2024-09-16T20:45:18.061 DEBUG:teuthology.orchestra.run.smithi029:> sudo yum -y install ceph-test 2024-09-16T20:45:18.666 INFO:teuthology.orchestra.run.smithi029.stdout:Last metadata expiration check: 0:00:32 ago on Mon 16 Sep 2024 08:44:46 PM UTC. 2024-09-16T20:45:18.812 INFO:teuthology.orchestra.run.smithi029.stdout:Dependencies resolved. 2024-09-16T20:45:18.813 INFO:teuthology.orchestra.run.smithi029.stdout:================================================================================ 2024-09-16T20:45:18.813 INFO:teuthology.orchestra.run.smithi029.stdout: Package Arch Version Repository Size 2024-09-16T20:45:18.813 INFO:teuthology.orchestra.run.smithi029.stdout:================================================================================ 2024-09-16T20:45:18.813 INFO:teuthology.orchestra.run.smithi029.stdout:Installing: 2024-09-16T20:45:18.813 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-test x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 50 M 2024-09-16T20:45:18.813 INFO:teuthology.orchestra.run.smithi029.stdout:Installing dependencies: 2024-09-16T20:45:18.814 INFO:teuthology.orchestra.run.smithi029.stdout: libcephsqlite x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 161 k 2024-09-16T20:45:18.814 INFO:teuthology.orchestra.run.smithi029.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-16T20:45:18.814 INFO:teuthology.orchestra.run.smithi029.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-16T20:45:18.814 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:45:18.814 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction Summary 2024-09-16T20:45:18.814 INFO:teuthology.orchestra.run.smithi029.stdout:================================================================================ 2024-09-16T20:45:18.814 INFO:teuthology.orchestra.run.smithi029.stdout:Install 4 Packages 2024-09-16T20:45:18.814 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:45:18.815 INFO:teuthology.orchestra.run.smithi029.stdout:Total download size: 51 M 2024-09-16T20:45:18.815 INFO:teuthology.orchestra.run.smithi029.stdout:Installed size: 217 M 2024-09-16T20:45:18.815 INFO:teuthology.orchestra.run.smithi029.stdout:Downloading Packages: 2024-09-16T20:45:19.146 INFO:teuthology.orchestra.run.smithi029.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.4 MB/s | 304 kB 00:00 2024-09-16T20:45:19.179 INFO:teuthology.orchestra.run.smithi029.stdout:(2/4): libcephsqlite-19.3.0-4883.g6e8474a0.el9. 675 kB/s | 161 kB 00:00 2024-09-16T20:45:19.205 INFO:teuthology.orchestra.run.smithi029.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-16T20:45:19.577 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-test-2:19.3.0-4883.g6e8474a0.el9.x86_64 4/4 2024-09-16T20:45:19.577 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-test-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/4 2024-09-16T20:45:19.577 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libcephsqlite-2:19.3.0-4883.g6e8474a0.el9.x86_64 2/4 2024-09-16T20:45:19.577 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-16T20:45:19.899 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-16T20:45:19.899 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:45:19.900 INFO:teuthology.orchestra.run.smithi139.stdout:Installed: 2024-09-16T20:45:19.900 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-test-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:19.900 INFO:teuthology.orchestra.run.smithi139.stdout: libcephsqlite-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:19.900 INFO:teuthology.orchestra.run.smithi139.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-16T20:45:19.900 INFO:teuthology.orchestra.run.smithi139.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-16T20:45:19.900 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:45:19.900 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-16T20:45:20.151 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum -y install ceph 2024-09-16T20:45:20.739 INFO:teuthology.orchestra.run.smithi139.stdout:Last metadata expiration check: 0:00:39 ago on Mon 16 Sep 2024 08:44:41 PM UTC. 2024-09-16T20:45:20.878 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-16T20:45:20.880 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-16T20:45:20.881 INFO:teuthology.orchestra.run.smithi139.stdout: Package Arch Version Repository Size 2024-09-16T20:45:20.881 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-16T20:45:20.881 INFO:teuthology.orchestra.run.smithi139.stdout:Installing: 2024-09-16T20:45:20.881 INFO:teuthology.orchestra.run.smithi139.stdout: ceph x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 6.6 k 2024-09-16T20:45:20.881 INFO:teuthology.orchestra.run.smithi139.stdout:Installing dependencies: 2024-09-16T20:45:20.881 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mds x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 2.4 M 2024-09-16T20:45:20.881 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 1.5 M 2024-09-16T20:45:20.881 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 291 k 2024-09-16T20:45:20.881 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mon x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 4.8 M 2024-09-16T20:45:20.881 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-osd x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 17 M 2024-09-16T20:45:20.881 INFO:teuthology.orchestra.run.smithi139.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-16T20:45:20.882 INFO:teuthology.orchestra.run.smithi139.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-16T20:45:20.882 INFO:teuthology.orchestra.run.smithi139.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-16T20:45:20.882 INFO:teuthology.orchestra.run.smithi139.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-16T20:45:20.882 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-16T20:45:20.882 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-16T20:45:20.882 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-16T20:45:20.882 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-16T20:45:20.882 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-16T20:45:20.882 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-16T20:45:20.883 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-collections 2024-09-16T20:45:20.883 INFO:teuthology.orchestra.run.smithi139.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-16T20:45:20.883 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-16T20:45:20.883 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-16T20:45:20.883 INFO:teuthology.orchestra.run.smithi139.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-16T20:45:20.883 INFO:teuthology.orchestra.run.smithi139.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-16T20:45:20.883 INFO:teuthology.orchestra.run.smithi139.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-16T20:45:20.883 INFO:teuthology.orchestra.run.smithi139.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-16T20:45:20.883 INFO:teuthology.orchestra.run.smithi139.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-16T20:45:20.883 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-16T20:45:20.883 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-16T20:45:20.883 INFO:teuthology.orchestra.run.smithi139.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-16T20:45:20.884 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-16T20:45:20.884 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-16T20:45:20.884 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-16T20:45:20.884 INFO:teuthology.orchestra.run.smithi139.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-16T20:45:20.884 INFO:teuthology.orchestra.run.smithi139.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-16T20:45:20.884 INFO:teuthology.orchestra.run.smithi139.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-16T20:45:20.884 INFO:teuthology.orchestra.run.smithi139.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-16T20:45:20.884 INFO:teuthology.orchestra.run.smithi139.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-16T20:45:20.884 INFO:teuthology.orchestra.run.smithi139.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-16T20:45:20.884 INFO:teuthology.orchestra.run.smithi139.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-16T20:45:20.884 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:45:20.884 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction Summary 2024-09-16T20:45:20.885 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-16T20:45:20.885 INFO:teuthology.orchestra.run.smithi139.stdout:Install 37 Packages 2024-09-16T20:45:20.885 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:45:20.885 INFO:teuthology.orchestra.run.smithi139.stdout:Total download size: 31 M 2024-09-16T20:45:20.885 INFO:teuthology.orchestra.run.smithi139.stdout:Installed size: 109 M 2024-09-16T20:45:20.886 INFO:teuthology.orchestra.run.smithi139.stdout:Downloading Packages: 2024-09-16T20:45:20.888 INFO:teuthology.orchestra.run.smithi029.stdout:(4/4): ceph-test-19.3.0-4883.g6e8474a0.el9.x86_ 26 MB/s | 50 MB 00:01 2024-09-16T20:45:20.891 INFO:teuthology.orchestra.run.smithi029.stdout:-------------------------------------------------------------------------------- 2024-09-16T20:45:20.891 INFO:teuthology.orchestra.run.smithi029.stdout:Total 24 MB/s | 51 MB 00:02 2024-09-16T20:45:20.919 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction check 2024-09-16T20:45:20.942 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction check succeeded. 2024-09-16T20:45:20.942 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction test 2024-09-16T20:45:21.320 INFO:teuthology.orchestra.run.smithi139.stdout:(1/37): ceph-19.3.0-4883.g6e8474a0.el9.x86_64.r 56 kB/s | 6.6 kB 00:00 2024-09-16T20:45:21.342 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction test succeeded. 2024-09-16T20:45:21.342 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction 2024-09-16T20:45:21.537 INFO:teuthology.orchestra.run.smithi139.stdout:(2/37): ceph-mds-19.3.0-4883.g6e8474a0.el9.x86_ 7.1 MB/s | 2.4 MB 00:00 2024-09-16T20:45:21.570 INFO:teuthology.orchestra.run.smithi139.stdout:(3/37): ceph-mgr-19.3.0-4883.g6e8474a0.el9.x86_ 4.2 MB/s | 1.5 MB 00:00 2024-09-16T20:45:21.670 INFO:teuthology.orchestra.run.smithi139.stdout:(4/37): ceph-mgr-modules-core-19.3.0-4883.g6e84 2.8 MB/s | 291 kB 00:00 2024-09-16T20:45:21.745 INFO:teuthology.orchestra.run.smithi139.stdout:(5/37): ceph-mon-19.3.0-4883.g6e8474a0.el9.x86_ 11 MB/s | 4.8 MB 00:00 2024-09-16T20:45:21.901 INFO:teuthology.orchestra.run.smithi029.stdout: Preparing : 1/1 2024-09-16T20:45:21.998 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-16T20:45:22.049 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-16T20:45:22.085 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libcephsqlite-2:19.3.0-4883.g6e8474a0.el9.x86_64 3/4 2024-09-16T20:45:22.088 INFO:teuthology.orchestra.run.smithi139.stdout:(6/37): ceph-osd-19.3.0-4883.g6e8474a0.el9.x86_ 31 MB/s | 17 MB 00:00 2024-09-16T20:45:22.113 INFO:teuthology.orchestra.run.smithi139.stdout:(7/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 573 kB/s | 253 kB 00:00 2024-09-16T20:45:22.138 INFO:teuthology.orchestra.run.smithi139.stdout:(8/37): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-16T20:45:22.188 INFO:teuthology.orchestra.run.smithi139.stdout:(9/37): python3-requests-2.25.1-8.el9.noarch.rp 2.4 MB/s | 125 kB 00:00 2024-09-16T20:45:22.247 INFO:teuthology.orchestra.run.smithi139.stdout:(10/37): python3-urllib3-1.26.5-6.el9.noarch.rp 3.6 MB/s | 215 kB 00:00 2024-09-16T20:45:22.280 INFO:teuthology.orchestra.run.smithi139.stdout:(11/37): python3-cryptography-36.0.1-4.el9.x86_ 2.3 MB/s | 1.2 MB 00:00 2024-09-16T20:45:22.306 INFO:teuthology.orchestra.run.smithi139.stdout:(12/37): python3-pycparser-2.20-6.el9.noarch.rp 701 kB/s | 135 kB 00:00 2024-09-16T20:45:22.589 INFO:teuthology.orchestra.run.smithi139.stdout:(13/37): python3-markupsafe-1.1.1-12.el9.x86_64 123 kB/s | 35 kB 00:00 2024-09-16T20:45:22.778 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4883.g6e8474a0.el9.x86_64 32/35 2024-09-16T20:45:22.778 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /sys 2024-09-16T20:45:22.779 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /proc 2024-09-16T20:45:22.779 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /mnt 2024-09-16T20:45:22.779 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /var/tmp 2024-09-16T20:45:22.779 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /home 2024-09-16T20:45:22.779 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /root 2024-09-16T20:45:22.779 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /tmp 2024-09-16T20:45:22.779 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:45:22.793 INFO:teuthology.orchestra.run.smithi139.stdout:(14/37): python3-packaging-20.9-5.el9.noarch.rp 379 kB/s | 77 kB 00:00 2024-09-16T20:45:22.826 INFO:teuthology.orchestra.run.smithi139.stdout:(15/37): lua-5.4.4-4.el9.x86_64.rpm 325 kB/s | 188 kB 00:00 2024-09-16T20:45:22.852 INFO:teuthology.orchestra.run.smithi139.stdout:(16/37): python3-mako-1.1.4-6.el9.noarch.rpm 301 kB/s | 172 kB 00:00 2024-09-16T20:45:22.902 INFO:teuthology.orchestra.run.smithi139.stdout:(17/37): python3-pytz-2021.1-5.el9.noarch.rpm 468 kB/s | 51 kB 00:00 2024-09-16T20:45:22.935 INFO:teuthology.orchestra.run.smithi139.stdout:(18/37): python3-toml-0.10.2-6.el9.noarch.rpm 383 kB/s | 42 kB 00:00 2024-09-16T20:45:22.961 INFO:teuthology.orchestra.run.smithi139.stdout:(19/37): lua-devel-5.4.4-4.el9.x86_64.rpm 205 kB/s | 22 kB 00:00 2024-09-16T20:45:23.003 INFO:teuthology.orchestra.run.smithi139.stdout:(20/37): luarocks-3.9.2-5.el9.noarch.rpm 1.5 MB/s | 151 kB 00:00 2024-09-16T20:45:23.028 INFO:teuthology.orchestra.run.smithi139.stdout:(21/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 481 kB/s | 43 kB 00:00 2024-09-16T20:45:23.053 INFO:teuthology.orchestra.run.smithi139.stdout:(22/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 431 kB/s | 11 kB 00:00 2024-09-16T20:45:23.078 INFO:teuthology.orchestra.run.smithi139.stdout:(23/37): python3-cherrypy-18.6.1-2.el9.noarch.r 4.8 MB/s | 358 kB 00:00 2024-09-16T20:45:23.104 INFO:teuthology.orchestra.run.smithi139.stdout:(24/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.2 MB/s | 172 kB 00:00 2024-09-16T20:45:23.129 INFO:teuthology.orchestra.run.smithi139.stdout:(25/37): python3-jaraco-classes-3.2.1-5.el9.noa 234 kB/s | 18 kB 00:00 2024-09-16T20:45:23.154 INFO:teuthology.orchestra.run.smithi139.stdout:(26/37): python3-jaraco-collections-3.0.0-8.el9 308 kB/s | 23 kB 00:00 2024-09-16T20:45:23.179 INFO:teuthology.orchestra.run.smithi139.stdout:(27/37): python3-jaraco-functools-3.5.0-2.el9.n 389 kB/s | 19 kB 00:00 2024-09-16T20:45:23.204 INFO:teuthology.orchestra.run.smithi139.stdout:(28/37): python3-jaraco-text-3.2.0-6.el9.noarch 262 kB/s | 20 kB 00:00 2024-09-16T20:45:23.230 INFO:teuthology.orchestra.run.smithi139.stdout:(29/37): python3-logutils-0.3.5-21.el9.noarch.r 615 kB/s | 46 kB 00:00 2024-09-16T20:45:23.257 INFO:teuthology.orchestra.run.smithi139.stdout:(30/37): python3-portend-3.1.0-2.el9.noarch.rpm 659 kB/s | 16 kB 00:00 2024-09-16T20:45:23.280 INFO:teuthology.orchestra.run.smithi139.stdout:(31/37): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2024-09-16T20:45:23.306 INFO:teuthology.orchestra.run.smithi139.stdout:(32/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.8 MB/s | 90 kB 00:00 2024-09-16T20:45:23.331 INFO:teuthology.orchestra.run.smithi139.stdout:(33/37): python3-tempora-5.0.0-2.el9.noarch.rpm 715 kB/s | 36 kB 00:00 2024-09-16T20:45:23.381 INFO:teuthology.orchestra.run.smithi139.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.5 MB/s | 272 kB 00:00 2024-09-16T20:45:23.406 INFO:teuthology.orchestra.run.smithi139.stdout:(35/37): python3-zc-lockfile-2.0-10.el9.noarch. 801 kB/s | 20 kB 00:00 2024-09-16T20:45:23.431 INFO:teuthology.orchestra.run.smithi139.stdout:(36/37): python3-webob-1.8.8-2.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-09-16T20:45:23.457 INFO:teuthology.orchestra.run.smithi139.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.3 MB/s | 427 kB 00:00 2024-09-16T20:45:23.461 INFO:teuthology.orchestra.run.smithi139.stdout:-------------------------------------------------------------------------------- 2024-09-16T20:45:23.461 INFO:teuthology.orchestra.run.smithi139.stdout:Total 12 MB/s | 31 MB 00:02 2024-09-16T20:45:23.613 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction check 2024-09-16T20:45:23.653 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction check succeeded. 2024-09-16T20:45:23.653 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction test 2024-09-16T20:45:23.811 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-radosgw-2:19.3.0-4883.g6e8474a0.el9.x86_64 33/35 2024-09-16T20:45:23.837 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4883.g6e8474a0.el9.x86_64 33/35 2024-09-16T20:45:23.837 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T20:45:23.837 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-16T20:45:23.837 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T20:45:23.837 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T20:45:23.838 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:45:23.839 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-16T20:45:23.929 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-16T20:45:23.929 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-16T20:45:24.133 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction test succeeded. 2024-09-16T20:45:24.133 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction 2024-09-16T20:45:25.088 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-16T20:45:25.088 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-base-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/35 2024-09-16T20:45:25.088 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-common-2:19.3.0-4883.g6e8474a0.el9.x86_64 2/35 2024-09-16T20:45:25.088 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-radosgw-2:19.3.0-4883.g6e8474a0.el9.x86_64 3/35 2024-09-16T20:45:25.088 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-selinux-2:19.3.0-4883.g6e8474a0.el9.x86_64 4/35 2024-09-16T20:45:25.088 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libcephfs2-2:19.3.0-4883.g6e8474a0.el9.x86_64 5/35 2024-09-16T20:45:25.088 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libradosstriper1-2:19.3.0-4883.g6e8474a0.el9.x86_6 6/35 2024-09-16T20:45:25.088 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librgw2-2:19.3.0-4883.g6e8474a0.el9.x86_64 7/35 2024-09-16T20:45:25.089 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4883.g6e8474a0.el9. 8/35 2024-09-16T20:45:25.089 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-ceph-common-2:19.3.0-4883.g6e8474a0.el9.x8 9/35 2024-09-16T20:45:25.089 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cephfs-2:19.3.0-4883.g6e8474a0.el9.x86_64 10/35 2024-09-16T20:45:25.089 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rados-2:19.3.0-4883.g6e8474a0.el9.x86_64 11/35 2024-09-16T20:45:25.089 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rbd-2:19.3.0-4883.g6e8474a0.el9.x86_64 12/35 2024-09-16T20:45:25.089 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rgw-2:19.3.0-4883.g6e8474a0.el9.x86_64 13/35 2024-09-16T20:45:25.089 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/35 2024-09-16T20:45:25.089 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 15/35 2024-09-16T20:45:25.089 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 16/35 2024-09-16T20:45:25.089 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 17/35 2024-09-16T20:45:25.089 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 18/35 2024-09-16T20:45:25.089 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 19/35 2024-09-16T20:45:25.089 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-16T20:45:25.089 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 21/35 2024-09-16T20:45:25.090 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 22/35 2024-09-16T20:45:25.090 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 23/35 2024-09-16T20:45:25.090 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 24/35 2024-09-16T20:45:25.090 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 25/35 2024-09-16T20:45:25.090 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 26/35 2024-09-16T20:45:25.091 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 27/35 2024-09-16T20:45:25.091 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 28/35 2024-09-16T20:45:25.091 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 29/35 2024-09-16T20:45:25.091 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : re2-1:20211101-20.el9.x86_64 30/35 2024-09-16T20:45:25.091 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 31/35 2024-09-16T20:45:25.091 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librados2-2:19.3.0-4883.g6e8474a0.el9.x86_64 32/35 2024-09-16T20:45:25.092 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 33/35 2024-09-16T20:45:25.092 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librbd1-2:19.3.0-4883.g6e8474a0.el9.x86_64 34/35 2024-09-16T20:45:25.490 INFO:teuthology.orchestra.run.smithi139.stdout: Preparing : 1/1 2024-09-16T20:45:25.543 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-16T20:45:25.584 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-16T20:45:25.612 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-16T20:45:25.686 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-16T20:45:25.728 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-16T20:45:25.761 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-16T20:45:25.855 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-16T20:45:25.932 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-16T20:45:25.989 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-16T20:45:26.023 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-16T20:45:26.053 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-16T20:45:26.087 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-16T20:45:26.121 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-16T20:45:26.144 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 35/35 2024-09-16T20:45:26.145 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:45:26.145 INFO:teuthology.orchestra.run.smithi043.stdout:Upgraded: 2024-09-16T20:45:26.145 INFO:teuthology.orchestra.run.smithi043.stdout: librados2-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:26.145 INFO:teuthology.orchestra.run.smithi043.stdout: librbd1-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:26.145 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-16T20:45:26.145 INFO:teuthology.orchestra.run.smithi043.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-16T20:45:26.145 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-base-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:26.145 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-common-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:26.145 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-radosgw-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:26.145 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-selinux-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:26.145 INFO:teuthology.orchestra.run.smithi043.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-16T20:45:26.145 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-16T20:45:26.145 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-16T20:45:26.146 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs2-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:26.146 INFO:teuthology.orchestra.run.smithi043.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-16T20:45:26.146 INFO:teuthology.orchestra.run.smithi043.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-16T20:45:26.146 INFO:teuthology.orchestra.run.smithi043.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-16T20:45:26.146 INFO:teuthology.orchestra.run.smithi043.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-16T20:45:26.146 INFO:teuthology.orchestra.run.smithi043.stdout: libradosstriper1-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:26.146 INFO:teuthology.orchestra.run.smithi043.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-16T20:45:26.146 INFO:teuthology.orchestra.run.smithi043.stdout: librgw2-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:26.146 INFO:teuthology.orchestra.run.smithi043.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-16T20:45:26.146 INFO:teuthology.orchestra.run.smithi043.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-16T20:45:26.146 INFO:teuthology.orchestra.run.smithi043.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-16T20:45:26.146 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-argparse-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:26.146 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-common-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:26.146 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cephfs-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:26.146 INFO:teuthology.orchestra.run.smithi043.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-16T20:45:26.147 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rados-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:26.147 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rbd-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:26.147 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rgw-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:26.147 INFO:teuthology.orchestra.run.smithi043.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-16T20:45:26.147 INFO:teuthology.orchestra.run.smithi043.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-16T20:45:26.147 INFO:teuthology.orchestra.run.smithi043.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-16T20:45:26.147 INFO:teuthology.orchestra.run.smithi043.stdout: re2-1:20211101-20.el9.x86_64 2024-09-16T20:45:26.147 INFO:teuthology.orchestra.run.smithi043.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-16T20:45:26.147 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:45:26.147 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-16T20:45:26.161 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-16T20:45:26.206 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-16T20:45:26.239 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-16T20:45:26.288 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-16T20:45:26.337 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-16T20:45:26.412 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-16T20:45:26.498 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-test 2024-09-16T20:45:26.563 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-16T20:45:26.649 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-16T20:45:26.707 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-16T20:45:26.744 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-16T20:45:26.803 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-16T20:45:26.867 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-16T20:45:27.078 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-16T20:45:27.098 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:00:29 ago on Mon 16 Sep 2024 08:44:58 PM UTC. 2024-09-16T20:45:27.123 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-16T20:45:27.204 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-16T20:45:27.211 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4883.g6e8474a0.el9.x86_64 3/4 2024-09-16T20:45:27.235 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-16T20:45:27.236 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T20:45:27.236 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-09-16T20:45:27.236 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T20:45:27.236 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-16T20:45:27.236 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-test x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 50 M 2024-09-16T20:45:27.236 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-09-16T20:45:27.237 INFO:teuthology.orchestra.run.smithi043.stdout: libcephsqlite x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 161 k 2024-09-16T20:45:27.237 INFO:teuthology.orchestra.run.smithi043.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-16T20:45:27.237 INFO:teuthology.orchestra.run.smithi043.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-16T20:45:27.237 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:45:27.237 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-16T20:45:27.237 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T20:45:27.237 INFO:teuthology.orchestra.run.smithi043.stdout:Install 4 Packages 2024-09-16T20:45:27.237 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:45:27.238 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 51 M 2024-09-16T20:45:27.238 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 217 M 2024-09-16T20:45:27.238 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-16T20:45:27.297 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-test-2:19.3.0-4883.g6e8474a0.el9.x86_64 4/4 2024-09-16T20:45:27.346 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-16T20:45:27.384 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-16T20:45:27.519 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-16T20:45:27.634 INFO:teuthology.orchestra.run.smithi043.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 2.2 MB/s | 304 kB 00:00 2024-09-16T20:45:27.693 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4883.g6e8474a0.el9. 32/37 2024-09-16T20:45:27.701 INFO:teuthology.orchestra.run.smithi043.stdout:(2/4): libcephsqlite-19.3.0-4883.g6e8474a0.el9. 790 kB/s | 161 kB 00:00 2024-09-16T20:45:27.723 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-2:19.3.0-4883.g6e8474a0.el9.x86_64 33/37 2024-09-16T20:45:27.749 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4883.g6e8474a0.el9.x86_64 33/37 2024-09-16T20:45:27.749 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T20:45:27.749 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-16T20:45:27.749 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T20:45:27.749 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T20:45:27.749 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:45:27.751 INFO:teuthology.orchestra.run.smithi043.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 546 kB/s | 64 kB 00:00 2024-09-16T20:45:29.076 INFO:teuthology.orchestra.run.smithi043.stdout:(4/4): ceph-test-19.3.0-4883.g6e8474a0.el9.x86_ 32 MB/s | 50 MB 00:01 2024-09-16T20:45:29.078 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-16T20:45:29.078 INFO:teuthology.orchestra.run.smithi043.stdout:Total 28 MB/s | 51 MB 00:01 2024-09-16T20:45:29.106 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-16T20:45:29.129 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-16T20:45:29.129 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-16T20:45:29.215 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-osd-2:19.3.0-4883.g6e8474a0.el9.x86_64 34/37 2024-09-16T20:45:29.242 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-osd-2:19.3.0-4883.g6e8474a0.el9.x86_64 34/37 2024-09-16T20:45:29.242 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T20:45:29.242 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-16T20:45:29.242 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T20:45:29.242 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T20:45:29.242 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:45:29.530 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-16T20:45:29.530 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-16T20:45:29.699 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mon-2:19.3.0-4883.g6e8474a0.el9.x86_64 35/37 2024-09-16T20:45:29.723 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mon-2:19.3.0-4883.g6e8474a0.el9.x86_64 35/37 2024-09-16T20:45:29.723 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T20:45:29.723 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-16T20:45:29.724 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T20:45:29.724 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T20:45:29.724 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:45:29.950 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mds-2:19.3.0-4883.g6e8474a0.el9.x86_64 36/37 2024-09-16T20:45:29.974 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mds-2:19.3.0-4883.g6e8474a0.el9.x86_64 36/37 2024-09-16T20:45:29.974 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T20:45:29.974 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-16T20:45:29.974 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T20:45:29.974 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T20:45:29.974 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:45:30.039 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-2:19.3.0-4883.g6e8474a0.el9.x86_64 37/37 2024-09-16T20:45:30.231 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-16T20:45:30.308 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-16T20:45:30.351 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-16T20:45:30.378 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libcephsqlite-2:19.3.0-4883.g6e8474a0.el9.x86_64 3/4 2024-09-16T20:45:31.419 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-test-2:19.3.0-4883.g6e8474a0.el9.x86_64 4/4 2024-09-16T20:45:31.419 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-test-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/4 2024-09-16T20:45:31.419 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libcephsqlite-2:19.3.0-4883.g6e8474a0.el9.x86_64 2/4 2024-09-16T20:45:31.419 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-16T20:45:31.743 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-16T20:45:31.743 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:45:31.743 INFO:teuthology.orchestra.run.smithi029.stdout:Installed: 2024-09-16T20:45:31.743 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-test-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:31.744 INFO:teuthology.orchestra.run.smithi029.stdout: libcephsqlite-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:31.744 INFO:teuthology.orchestra.run.smithi029.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-16T20:45:31.744 INFO:teuthology.orchestra.run.smithi029.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-16T20:45:31.744 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:45:31.744 INFO:teuthology.orchestra.run.smithi029.stdout:Complete! 2024-09-16T20:45:31.849 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-2:19.3.0-4883.g6e8474a0.el9.x86_64 37/37 2024-09-16T20:45:31.849 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/37 2024-09-16T20:45:31.849 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mds-2:19.3.0-4883.g6e8474a0.el9.x86_64 2/37 2024-09-16T20:45:31.849 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-2:19.3.0-4883.g6e8474a0.el9.x86_64 3/37 2024-09-16T20:45:31.849 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mon-2:19.3.0-4883.g6e8474a0.el9.x86_64 4/37 2024-09-16T20:45:31.849 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-osd-2:19.3.0-4883.g6e8474a0.el9.x86_64 5/37 2024-09-16T20:45:31.849 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4883.g6e8474a0.el9. 6/37 2024-09-16T20:45:31.849 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-16T20:45:31.850 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-16T20:45:31.850 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-16T20:45:31.850 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-16T20:45:31.850 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-16T20:45:31.850 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-16T20:45:31.850 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-16T20:45:31.850 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-16T20:45:31.850 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-16T20:45:31.850 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-16T20:45:31.850 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-16T20:45:31.850 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-16T20:45:31.850 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-16T20:45:31.850 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-16T20:45:31.851 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-16T20:45:31.851 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-16T20:45:31.851 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-16T20:45:31.851 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-16T20:45:31.851 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-16T20:45:31.852 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-16T20:45:31.852 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-16T20:45:31.852 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-16T20:45:31.852 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-16T20:45:31.852 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-16T20:45:31.852 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-16T20:45:31.853 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-16T20:45:31.853 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-16T20:45:31.853 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-16T20:45:31.853 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-16T20:45:31.853 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-16T20:45:32.127 DEBUG:teuthology.orchestra.run.smithi029:> sudo yum -y install ceph 2024-09-16T20:45:32.734 INFO:teuthology.orchestra.run.smithi029.stdout:Last metadata expiration check: 0:00:46 ago on Mon 16 Sep 2024 08:44:46 PM UTC. 2024-09-16T20:45:32.882 INFO:teuthology.orchestra.run.smithi029.stdout:Dependencies resolved. 2024-09-16T20:45:32.885 INFO:teuthology.orchestra.run.smithi029.stdout:================================================================================ 2024-09-16T20:45:32.885 INFO:teuthology.orchestra.run.smithi029.stdout: Package Arch Version Repository Size 2024-09-16T20:45:32.885 INFO:teuthology.orchestra.run.smithi029.stdout:================================================================================ 2024-09-16T20:45:32.885 INFO:teuthology.orchestra.run.smithi029.stdout:Installing: 2024-09-16T20:45:32.885 INFO:teuthology.orchestra.run.smithi029.stdout: ceph x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 6.6 k 2024-09-16T20:45:32.885 INFO:teuthology.orchestra.run.smithi029.stdout:Installing dependencies: 2024-09-16T20:45:32.885 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mds x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 2.4 M 2024-09-16T20:45:32.885 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 1.5 M 2024-09-16T20:45:32.885 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 291 k 2024-09-16T20:45:32.886 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mon x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 4.8 M 2024-09-16T20:45:32.886 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-osd x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 17 M 2024-09-16T20:45:32.886 INFO:teuthology.orchestra.run.smithi029.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-16T20:45:32.886 INFO:teuthology.orchestra.run.smithi029.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-16T20:45:32.886 INFO:teuthology.orchestra.run.smithi029.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-16T20:45:32.886 INFO:teuthology.orchestra.run.smithi029.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-16T20:45:32.886 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-16T20:45:32.886 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-16T20:45:32.886 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-16T20:45:32.886 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-16T20:45:32.886 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-16T20:45:32.886 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-16T20:45:32.886 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jaraco-collections 2024-09-16T20:45:32.886 INFO:teuthology.orchestra.run.smithi029.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-16T20:45:32.887 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-16T20:45:32.887 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-16T20:45:32.887 INFO:teuthology.orchestra.run.smithi029.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-16T20:45:32.887 INFO:teuthology.orchestra.run.smithi029.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-16T20:45:32.887 INFO:teuthology.orchestra.run.smithi029.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-16T20:45:32.887 INFO:teuthology.orchestra.run.smithi029.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-16T20:45:32.887 INFO:teuthology.orchestra.run.smithi029.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-16T20:45:32.887 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-16T20:45:32.887 INFO:teuthology.orchestra.run.smithi029.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-16T20:45:32.887 INFO:teuthology.orchestra.run.smithi029.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-16T20:45:32.887 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-16T20:45:32.887 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-16T20:45:32.888 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-16T20:45:32.888 INFO:teuthology.orchestra.run.smithi029.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-16T20:45:32.888 INFO:teuthology.orchestra.run.smithi029.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-16T20:45:32.888 INFO:teuthology.orchestra.run.smithi029.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-16T20:45:32.888 INFO:teuthology.orchestra.run.smithi029.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-16T20:45:32.888 INFO:teuthology.orchestra.run.smithi029.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-16T20:45:32.888 INFO:teuthology.orchestra.run.smithi029.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-16T20:45:32.888 INFO:teuthology.orchestra.run.smithi029.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-16T20:45:32.888 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:45:32.888 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction Summary 2024-09-16T20:45:32.888 INFO:teuthology.orchestra.run.smithi029.stdout:================================================================================ 2024-09-16T20:45:32.888 INFO:teuthology.orchestra.run.smithi029.stdout:Install 37 Packages 2024-09-16T20:45:32.889 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:45:32.889 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-16T20:45:32.889 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:45:32.889 INFO:teuthology.orchestra.run.smithi139.stdout:Installed: 2024-09-16T20:45:32.889 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:32.890 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mds-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:32.890 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:32.890 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-modules-core-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-16T20:45:32.890 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mon-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:32.890 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-osd-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:32.890 INFO:teuthology.orchestra.run.smithi139.stdout: lua-5.4.4-4.el9.x86_64 2024-09-16T20:45:32.890 INFO:teuthology.orchestra.run.smithi139.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-16T20:45:32.890 INFO:teuthology.orchestra.run.smithi139.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-16T20:45:32.890 INFO:teuthology.orchestra.run.smithi139.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-16T20:45:32.890 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-16T20:45:32.890 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-16T20:45:32.891 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-16T20:45:32.891 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-16T20:45:32.891 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-16T20:45:32.891 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-16T20:45:32.891 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-16T20:45:32.891 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-16T20:45:32.891 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-16T20:45:32.891 INFO:teuthology.orchestra.run.smithi139.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-16T20:45:32.891 INFO:teuthology.orchestra.run.smithi139.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-16T20:45:32.891 INFO:teuthology.orchestra.run.smithi139.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-16T20:45:32.891 INFO:teuthology.orchestra.run.smithi139.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-16T20:45:32.892 INFO:teuthology.orchestra.run.smithi139.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-16T20:45:32.892 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-16T20:45:32.892 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ply-3.11-14.el9.noarch 2024-09-16T20:45:32.892 INFO:teuthology.orchestra.run.smithi139.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-16T20:45:32.892 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-16T20:45:32.892 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-16T20:45:32.892 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-16T20:45:32.892 INFO:teuthology.orchestra.run.smithi139.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-16T20:45:32.892 INFO:teuthology.orchestra.run.smithi139.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-16T20:45:32.892 INFO:teuthology.orchestra.run.smithi139.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-16T20:45:32.892 INFO:teuthology.orchestra.run.smithi139.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-16T20:45:32.892 INFO:teuthology.orchestra.run.smithi139.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-16T20:45:32.892 INFO:teuthology.orchestra.run.smithi139.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-16T20:45:32.892 INFO:teuthology.orchestra.run.smithi139.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-16T20:45:32.893 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:45:32.893 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-16T20:45:32.893 INFO:teuthology.orchestra.run.smithi029.stdout:Total download size: 31 M 2024-09-16T20:45:32.893 INFO:teuthology.orchestra.run.smithi029.stdout:Installed size: 109 M 2024-09-16T20:45:32.893 INFO:teuthology.orchestra.run.smithi029.stdout:Downloading Packages: 2024-09-16T20:45:33.231 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum -y install ceph-base 2024-09-16T20:45:33.561 INFO:teuthology.orchestra.run.smithi029.stdout:(1/37): ceph-19.3.0-4883.g6e8474a0.el9.x86_64.r 57 kB/s | 6.6 kB 00:00 2024-09-16T20:45:33.795 INFO:teuthology.orchestra.run.smithi029.stdout:(2/37): ceph-mds-19.3.0-4883.g6e8474a0.el9.x86_ 6.8 MB/s | 2.4 MB 00:00 2024-09-16T20:45:33.816 INFO:teuthology.orchestra.run.smithi139.stdout:Last metadata expiration check: 0:00:52 ago on Mon 16 Sep 2024 08:44:41 PM UTC. 2024-09-16T20:45:33.846 INFO:teuthology.orchestra.run.smithi029.stdout:(3/37): ceph-mgr-19.3.0-4883.g6e8474a0.el9.x86_ 3.8 MB/s | 1.5 MB 00:00 2024-09-16T20:45:33.905 INFO:teuthology.orchestra.run.smithi139.stdout:Package ceph-base-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-16T20:45:33.938 INFO:teuthology.orchestra.run.smithi029.stdout:(4/37): ceph-mgr-modules-core-19.3.0-4883.g6e84 3.1 MB/s | 291 kB 00:00 2024-09-16T20:45:33.947 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-16T20:45:33.948 INFO:teuthology.orchestra.run.smithi139.stdout:Nothing to do. 2024-09-16T20:45:33.948 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-16T20:45:34.014 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum -y install cephadm 2024-09-16T20:45:34.072 INFO:teuthology.orchestra.run.smithi029.stdout:(5/37): ceph-mon-19.3.0-4883.g6e8474a0.el9.x86_ 9.4 MB/s | 4.8 MB 00:00 2024-09-16T20:45:34.489 INFO:teuthology.orchestra.run.smithi029.stdout:(6/37): ceph-osd-19.3.0-4883.g6e8474a0.el9.x86_ 25 MB/s | 17 MB 00:00 2024-09-16T20:45:34.597 INFO:teuthology.orchestra.run.smithi139.stdout:Last metadata expiration check: 0:00:53 ago on Mon 16 Sep 2024 08:44:41 PM UTC. 2024-09-16T20:45:34.727 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-16T20:45:34.728 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-16T20:45:34.728 INFO:teuthology.orchestra.run.smithi139.stdout: Package Arch Version Repository Size 2024-09-16T20:45:34.728 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-16T20:45:34.728 INFO:teuthology.orchestra.run.smithi139.stdout:Installing: 2024-09-16T20:45:34.728 INFO:teuthology.orchestra.run.smithi139.stdout: cephadm noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 784 k 2024-09-16T20:45:34.728 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:45:34.728 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction Summary 2024-09-16T20:45:34.729 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-16T20:45:34.729 INFO:teuthology.orchestra.run.smithi139.stdout:Install 1 Package 2024-09-16T20:45:34.729 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:45:34.729 INFO:teuthology.orchestra.run.smithi139.stdout:Total download size: 784 k 2024-09-16T20:45:34.729 INFO:teuthology.orchestra.run.smithi139.stdout:Installed size: 791 k 2024-09-16T20:45:34.729 INFO:teuthology.orchestra.run.smithi139.stdout:Downloading Packages: 2024-09-16T20:45:34.873 INFO:teuthology.orchestra.run.smithi029.stdout:(7/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 271 kB/s | 253 kB 00:00 2024-09-16T20:45:34.957 INFO:teuthology.orchestra.run.smithi029.stdout:(8/37): python3-ply-3.11-14.el9.noarch.rpm 228 kB/s | 106 kB 00:00 2024-09-16T20:45:34.990 INFO:teuthology.orchestra.run.smithi139.stdout:cephadm-19.3.0-4883.g6e8474a0.el9.noarch.rpm 2.9 MB/s | 784 kB 00:00 2024-09-16T20:45:34.991 INFO:teuthology.orchestra.run.smithi029.stdout:(9/37): python3-pycparser-2.20-6.el9.noarch.rpm 1.1 MB/s | 135 kB 00:00 2024-09-16T20:45:34.991 INFO:teuthology.orchestra.run.smithi139.stdout:-------------------------------------------------------------------------------- 2024-09-16T20:45:34.991 INFO:teuthology.orchestra.run.smithi139.stdout:Total 2.9 MB/s | 784 kB 00:00 2024-09-16T20:45:34.991 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction check 2024-09-16T20:45:34.996 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction check succeeded. 2024-09-16T20:45:34.996 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction test 2024-09-16T20:45:35.002 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction test succeeded. 2024-09-16T20:45:35.002 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction 2024-09-16T20:45:35.075 INFO:teuthology.orchestra.run.smithi139.stdout: Preparing : 1/1 2024-09-16T20:45:35.108 INFO:teuthology.orchestra.run.smithi029.stdout:(10/37): python3-urllib3-1.26.5-6.el9.noarch.rp 1.8 MB/s | 215 kB 00:00 2024-09-16T20:45:35.167 INFO:teuthology.orchestra.run.smithi029.stdout:(11/37): python3-cryptography-36.0.1-4.el9.x86_ 1.1 MB/s | 1.2 MB 00:01 2024-09-16T20:45:35.201 INFO:teuthology.orchestra.run.smithi029.stdout:(12/37): python3-requests-2.25.1-8.el9.noarch.r 513 kB/s | 125 kB 00:00 2024-09-16T20:45:35.285 INFO:teuthology.orchestra.run.smithi029.stdout:(13/37): lua-5.4.4-4.el9.x86_64.rpm 1.0 MB/s | 188 kB 00:00 2024-09-16T20:45:35.310 INFO:teuthology.orchestra.run.smithi029.stdout:(14/37): python3-markupsafe-1.1.1-12.el9.x86_64 318 kB/s | 35 kB 00:00 2024-09-16T20:45:35.344 INFO:teuthology.orchestra.run.smithi029.stdout:(15/37): python3-packaging-20.9-5.el9.noarch.rp 1.3 MB/s | 77 kB 00:00 2024-09-16T20:45:35.379 INFO:teuthology.orchestra.run.smithi029.stdout:(16/37): python3-mako-1.1.4-6.el9.noarch.rpm 810 kB/s | 172 kB 00:00 2024-09-16T20:45:35.405 INFO:teuthology.orchestra.run.smithi029.stdout:(17/37): python3-pytz-2021.1-5.el9.noarch.rpm 540 kB/s | 51 kB 00:00 2024-09-16T20:45:35.418 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4883.g6e8474a0.el9.x86_64 3/4 2024-09-16T20:45:35.430 INFO:teuthology.orchestra.run.smithi029.stdout:(18/37): python3-toml-0.10.2-6.el9.noarch.rpm 484 kB/s | 42 kB 00:00 2024-09-16T20:45:35.489 INFO:teuthology.orchestra.run.smithi029.stdout:(19/37): luarocks-3.9.2-5.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-09-16T20:45:35.505 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-test-2:19.3.0-4883.g6e8474a0.el9.x86_64 4/4 2024-09-16T20:45:35.515 INFO:teuthology.orchestra.run.smithi029.stdout:(20/37): lua-devel-5.4.4-4.el9.x86_64.rpm 165 kB/s | 22 kB 00:00 2024-09-16T20:45:35.537 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 1/1 2024-09-16T20:45:35.549 INFO:teuthology.orchestra.run.smithi029.stdout:(21/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 368 kB/s | 43 kB 00:00 2024-09-16T20:45:35.591 INFO:teuthology.orchestra.run.smithi029.stdout:(22/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.7 MB/s | 172 kB 00:00 2024-09-16T20:45:35.616 INFO:teuthology.orchestra.run.smithi029.stdout:(23/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 180 kB/s | 11 kB 00:00 2024-09-16T20:45:35.642 INFO:teuthology.orchestra.run.smithi029.stdout:(24/37): python3-jaraco-classes-3.2.1-5.el9.noa 364 kB/s | 18 kB 00:00 2024-09-16T20:45:35.644 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 1/1 2024-09-16T20:45:35.664 INFO:teuthology.orchestra.run.smithi029.stdout:(25/37): python3-jaraco-collections-3.0.0-8.el9 486 kB/s | 23 kB 00:00 2024-09-16T20:45:35.690 INFO:teuthology.orchestra.run.smithi029.stdout:(26/37): python3-jaraco-functools-3.5.0-2.el9.n 409 kB/s | 19 kB 00:00 2024-09-16T20:45:35.716 INFO:teuthology.orchestra.run.smithi029.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 386 kB/s | 20 kB 00:00 2024-09-16T20:45:35.741 INFO:teuthology.orchestra.run.smithi029.stdout:(28/37): python3-logutils-0.3.5-21.el9.noarch.r 912 kB/s | 46 kB 00:00 2024-09-16T20:45:35.775 INFO:teuthology.orchestra.run.smithi029.stdout:(29/37): python3-more-itertools-8.12.0-2.el9.no 1.3 MB/s | 79 kB 00:00 2024-09-16T20:45:35.809 INFO:teuthology.orchestra.run.smithi029.stdout:(30/37): python3-cherrypy-18.6.1-2.el9.noarch.r 1.2 MB/s | 358 kB 00:00 2024-09-16T20:45:35.835 INFO:teuthology.orchestra.run.smithi029.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 277 kB/s | 16 kB 00:00 2024-09-16T20:45:35.868 INFO:teuthology.orchestra.run.smithi029.stdout:(32/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.5 MB/s | 90 kB 00:00 2024-09-16T20:45:35.902 INFO:teuthology.orchestra.run.smithi029.stdout:(33/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.7 MB/s | 272 kB 00:00 2024-09-16T20:45:35.913 INFO:teuthology.orchestra.run.smithi029.stdout:(34/37): python3-tempora-5.0.0-2.el9.noarch.rpm 459 kB/s | 36 kB 00:00 2024-09-16T20:45:35.938 INFO:teuthology.orchestra.run.smithi029.stdout:(35/37): python3-zc-lockfile-2.0-10.el9.noarch. 794 kB/s | 20 kB 00:00 2024-09-16T20:45:35.980 INFO:teuthology.orchestra.run.smithi029.stdout:(36/37): python3-webob-1.8.8-2.el9.noarch.rpm 2.0 MB/s | 230 kB 00:00 2024-09-16T20:45:35.992 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 1/1 2024-09-16T20:45:36.014 INFO:teuthology.orchestra.run.smithi029.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.7 MB/s | 427 kB 00:00 2024-09-16T20:45:36.018 INFO:teuthology.orchestra.run.smithi029.stdout:-------------------------------------------------------------------------------- 2024-09-16T20:45:36.018 INFO:teuthology.orchestra.run.smithi029.stdout:Total 9.8 MB/s | 31 MB 00:03 2024-09-16T20:45:36.174 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction check 2024-09-16T20:45:36.218 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction check succeeded. 2024-09-16T20:45:36.218 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction test 2024-09-16T20:45:36.242 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 1/1 2024-09-16T20:45:36.242 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:45:36.242 INFO:teuthology.orchestra.run.smithi139.stdout:Installed: 2024-09-16T20:45:36.242 INFO:teuthology.orchestra.run.smithi139.stdout: cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-16T20:45:36.242 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:45:36.242 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-16T20:45:36.434 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum -y install ceph-immutable-object-cache 2024-09-16T20:45:36.756 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction test succeeded. 2024-09-16T20:45:36.756 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction 2024-09-16T20:45:37.033 INFO:teuthology.orchestra.run.smithi139.stdout:Last metadata expiration check: 0:00:56 ago on Mon 16 Sep 2024 08:44:41 PM UTC. 2024-09-16T20:45:37.171 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-16T20:45:37.172 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-16T20:45:37.172 INFO:teuthology.orchestra.run.smithi139.stdout: Package Arch Version Repo Size 2024-09-16T20:45:37.172 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-16T20:45:37.172 INFO:teuthology.orchestra.run.smithi139.stdout:Installing: 2024-09-16T20:45:37.172 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 145 k 2024-09-16T20:45:37.173 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:45:37.173 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction Summary 2024-09-16T20:45:37.173 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-16T20:45:37.173 INFO:teuthology.orchestra.run.smithi139.stdout:Install 1 Package 2024-09-16T20:45:37.173 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:45:37.173 INFO:teuthology.orchestra.run.smithi139.stdout:Total download size: 145 k 2024-09-16T20:45:37.173 INFO:teuthology.orchestra.run.smithi139.stdout:Installed size: 432 k 2024-09-16T20:45:37.173 INFO:teuthology.orchestra.run.smithi139.stdout:Downloading Packages: 2024-09-16T20:45:37.380 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-immutable-object-cache-19.3.0-4883.g6e8474 698 kB/s | 145 kB 00:00 2024-09-16T20:45:37.381 INFO:teuthology.orchestra.run.smithi139.stdout:-------------------------------------------------------------------------------- 2024-09-16T20:45:37.381 INFO:teuthology.orchestra.run.smithi139.stdout:Total 693 kB/s | 145 kB 00:00 2024-09-16T20:45:37.382 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction check 2024-09-16T20:45:37.389 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction check succeeded. 2024-09-16T20:45:37.389 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction test 2024-09-16T20:45:37.450 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction test succeeded. 2024-09-16T20:45:37.451 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction 2024-09-16T20:45:37.593 INFO:teuthology.orchestra.run.smithi139.stdout: Preparing : 1/1 2024-09-16T20:45:37.665 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4883.g6e8474a0. 1/1 2024-09-16T20:45:37.690 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4883.g6e8474a0. 1/1 2024-09-16T20:45:37.690 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T20:45:37.690 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-16T20:45:37.690 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:45:38.063 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-test-2:19.3.0-4883.g6e8474a0.el9.x86_64 4/4 2024-09-16T20:45:38.063 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-test-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/4 2024-09-16T20:45:38.063 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libcephsqlite-2:19.3.0-4883.g6e8474a0.el9.x86_64 2/4 2024-09-16T20:45:38.063 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-16T20:45:38.184 INFO:teuthology.orchestra.run.smithi029.stdout: Preparing : 1/1 2024-09-16T20:45:38.260 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-16T20:45:38.311 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-16T20:45:38.347 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-16T20:45:38.412 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-16T20:45:38.412 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:45:38.413 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-16T20:45:38.413 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-test-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:38.413 INFO:teuthology.orchestra.run.smithi043.stdout: libcephsqlite-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:38.413 INFO:teuthology.orchestra.run.smithi043.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-16T20:45:38.413 INFO:teuthology.orchestra.run.smithi043.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-16T20:45:38.413 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:45:38.413 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-16T20:45:38.430 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-16T20:45:38.491 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-16T20:45:38.533 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-16T20:45:38.622 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4883.g6e8474a0. 1/1 2024-09-16T20:45:38.622 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:45:38.622 INFO:teuthology.orchestra.run.smithi139.stdout:Installed: 2024-09-16T20:45:38.622 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-immutable-object-cache-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:38.622 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:45:38.622 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-16T20:45:38.629 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-16T20:45:38.669 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph 2024-09-16T20:45:38.765 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-16T20:45:38.813 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-16T20:45:38.865 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-16T20:45:38.866 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum -y install ceph-mgr 2024-09-16T20:45:38.905 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-16T20:45:38.949 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-16T20:45:38.991 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-16T20:45:39.041 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-16T20:45:39.085 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-16T20:45:39.128 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-16T20:45:39.186 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-16T20:45:39.235 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-16T20:45:39.300 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:00:41 ago on Mon 16 Sep 2024 08:44:58 PM UTC. 2024-09-16T20:45:39.322 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-16T20:45:39.448 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-16T20:45:39.450 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T20:45:39.451 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-09-16T20:45:39.451 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T20:45:39.451 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-16T20:45:39.451 INFO:teuthology.orchestra.run.smithi043.stdout: ceph x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 6.6 k 2024-09-16T20:45:39.451 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-09-16T20:45:39.451 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mds x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 2.4 M 2024-09-16T20:45:39.451 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 1.5 M 2024-09-16T20:45:39.451 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 291 k 2024-09-16T20:45:39.451 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mon x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 4.8 M 2024-09-16T20:45:39.451 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-osd x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 17 M 2024-09-16T20:45:39.451 INFO:teuthology.orchestra.run.smithi043.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-16T20:45:39.451 INFO:teuthology.orchestra.run.smithi043.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-16T20:45:39.451 INFO:teuthology.orchestra.run.smithi043.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-16T20:45:39.452 INFO:teuthology.orchestra.run.smithi043.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-16T20:45:39.452 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-16T20:45:39.452 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-16T20:45:39.452 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-16T20:45:39.452 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-16T20:45:39.452 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-16T20:45:39.452 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-16T20:45:39.452 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-collections 2024-09-16T20:45:39.452 INFO:teuthology.orchestra.run.smithi043.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-16T20:45:39.452 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-16T20:45:39.452 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-16T20:45:39.452 INFO:teuthology.orchestra.run.smithi043.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-16T20:45:39.452 INFO:teuthology.orchestra.run.smithi043.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-16T20:45:39.453 INFO:teuthology.orchestra.run.smithi043.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-16T20:45:39.453 INFO:teuthology.orchestra.run.smithi043.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-16T20:45:39.453 INFO:teuthology.orchestra.run.smithi043.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-16T20:45:39.453 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-16T20:45:39.453 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-16T20:45:39.453 INFO:teuthology.orchestra.run.smithi043.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-16T20:45:39.453 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-16T20:45:39.453 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-16T20:45:39.453 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-16T20:45:39.453 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-16T20:45:39.453 INFO:teuthology.orchestra.run.smithi043.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-16T20:45:39.453 INFO:teuthology.orchestra.run.smithi043.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-16T20:45:39.453 INFO:teuthology.orchestra.run.smithi043.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-16T20:45:39.453 INFO:teuthology.orchestra.run.smithi043.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-16T20:45:39.453 INFO:teuthology.orchestra.run.smithi043.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-16T20:45:39.454 INFO:teuthology.orchestra.run.smithi043.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-16T20:45:39.454 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:45:39.454 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-16T20:45:39.454 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T20:45:39.454 INFO:teuthology.orchestra.run.smithi043.stdout:Install 37 Packages 2024-09-16T20:45:39.454 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:45:39.454 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 31 M 2024-09-16T20:45:39.454 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 109 M 2024-09-16T20:45:39.455 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-16T20:45:39.455 INFO:teuthology.orchestra.run.smithi139.stdout:Last metadata expiration check: 0:00:58 ago on Mon 16 Sep 2024 08:44:41 PM UTC. 2024-09-16T20:45:39.483 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-16T20:45:39.549 INFO:teuthology.orchestra.run.smithi139.stdout:Package ceph-mgr-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-16T20:45:39.576 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-16T20:45:39.591 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-16T20:45:39.592 INFO:teuthology.orchestra.run.smithi139.stdout:Nothing to do. 2024-09-16T20:45:39.592 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-16T20:45:39.638 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-16T20:45:39.650 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum -y install ceph-mgr-dashboard 2024-09-16T20:45:39.693 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-16T20:45:39.761 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-16T20:45:39.824 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-16T20:45:40.038 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-16T20:45:40.102 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-16T20:45:40.121 INFO:teuthology.orchestra.run.smithi043.stdout:(1/37): ceph-19.3.0-4883.g6e8474a0.el9.x86_64.r 51 kB/s | 6.6 kB 00:00 2024-09-16T20:45:40.191 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-16T20:45:40.224 INFO:teuthology.orchestra.run.smithi139.stdout:Last metadata expiration check: 0:00:59 ago on Mon 16 Sep 2024 08:44:41 PM UTC. 2024-09-16T20:45:40.291 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-16T20:45:40.355 INFO:teuthology.orchestra.run.smithi043.stdout:(2/37): ceph-mds-19.3.0-4883.g6e8474a0.el9.x86_ 6.6 MB/s | 2.4 MB 00:00 2024-09-16T20:45:40.356 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-16T20:45:40.356 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-16T20:45:40.357 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-16T20:45:40.357 INFO:teuthology.orchestra.run.smithi139.stdout: Package Arch Version Repository Size 2024-09-16T20:45:40.357 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-16T20:45:40.357 INFO:teuthology.orchestra.run.smithi139.stdout:Installing: 2024-09-16T20:45:40.357 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 14 M 2024-09-16T20:45:40.357 INFO:teuthology.orchestra.run.smithi139.stdout:Installing dependencies: 2024-09-16T20:45:40.357 INFO:teuthology.orchestra.run.smithi139.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-16T20:45:40.357 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 34 k 2024-09-16T20:45:40.358 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 16 k 2024-09-16T20:45:40.358 INFO:teuthology.orchestra.run.smithi139.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-16T20:45:40.358 INFO:teuthology.orchestra.run.smithi139.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-16T20:45:40.358 INFO:teuthology.orchestra.run.smithi139.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-16T20:45:40.358 INFO:teuthology.orchestra.run.smithi139.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-16T20:45:40.358 INFO:teuthology.orchestra.run.smithi139.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-16T20:45:40.358 INFO:teuthology.orchestra.run.smithi139.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-16T20:45:40.358 INFO:teuthology.orchestra.run.smithi139.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-16T20:45:40.358 INFO:teuthology.orchestra.run.smithi139.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-16T20:45:40.358 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:45:40.358 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction Summary 2024-09-16T20:45:40.358 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-16T20:45:40.358 INFO:teuthology.orchestra.run.smithi139.stdout:Install 12 Packages 2024-09-16T20:45:40.358 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:45:40.359 INFO:teuthology.orchestra.run.smithi139.stdout:Total download size: 19 M 2024-09-16T20:45:40.359 INFO:teuthology.orchestra.run.smithi139.stdout:Installed size: 159 M 2024-09-16T20:45:40.359 INFO:teuthology.orchestra.run.smithi139.stdout:Downloading Packages: 2024-09-16T20:45:40.388 INFO:teuthology.orchestra.run.smithi043.stdout:(3/37): ceph-mgr-19.3.0-4883.g6e8474a0.el9.x86_ 3.9 MB/s | 1.5 MB 00:00 2024-09-16T20:45:40.480 INFO:teuthology.orchestra.run.smithi043.stdout:(4/37): ceph-mgr-modules-core-19.3.0-4883.g6e84 3.1 MB/s | 291 kB 00:00 2024-09-16T20:45:40.497 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-16T20:45:40.547 INFO:teuthology.orchestra.run.smithi043.stdout:(5/37): ceph-mon-19.3.0-4883.g6e8474a0.el9.x86_ 11 MB/s | 4.8 MB 00:00 2024-09-16T20:45:40.678 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4883.g6e8474a0.el9. 32/37 2024-09-16T20:45:40.709 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-mgr-2:19.3.0-4883.g6e8474a0.el9.x86_64 33/37 2024-09-16T20:45:40.736 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4883.g6e8474a0.el9.x86_64 33/37 2024-09-16T20:45:40.736 INFO:teuthology.orchestra.run.smithi029.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T20:45:40.737 INFO:teuthology.orchestra.run.smithi029.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-16T20:45:40.737 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T20:45:40.737 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T20:45:40.737 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:45:40.889 INFO:teuthology.orchestra.run.smithi043.stdout:(6/37): ceph-osd-19.3.0-4883.g6e8474a0.el9.x86_ 32 MB/s | 17 MB 00:00 2024-09-16T20:45:40.958 INFO:teuthology.orchestra.run.smithi139.stdout:(1/12): ceph-grafana-dashboards-19.3.0-4883.g6e 238 kB/s | 34 kB 00:00 2024-09-16T20:45:40.983 INFO:teuthology.orchestra.run.smithi139.stdout:(2/12): ceph-prometheus-alerts-19.3.0-4883.g6e8 97 kB/s | 16 kB 00:00 2024-09-16T20:45:41.109 INFO:teuthology.orchestra.run.smithi139.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 2.1 MB/s | 269 kB 00:00 2024-09-16T20:45:41.167 INFO:teuthology.orchestra.run.smithi139.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 4.8 MB/s | 1.0 MB 00:00 2024-09-16T20:45:41.206 INFO:teuthology.orchestra.run.smithi043.stdout:(7/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 349 kB/s | 253 kB 00:00 2024-09-16T20:45:41.226 INFO:teuthology.orchestra.run.smithi139.stdout:(5/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 9.2 MB/s | 548 kB 00:00 2024-09-16T20:45:41.251 INFO:teuthology.orchestra.run.smithi139.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 786 kB/s | 19 kB 00:00 2024-09-16T20:45:41.314 INFO:teuthology.orchestra.run.smithi043.stdout:(8/37): python3-pycparser-2.20-6.el9.noarch.rpm 1.2 MB/s | 135 kB 00:00 2024-09-16T20:45:41.318 INFO:teuthology.orchestra.run.smithi139.stdout:(7/12): python3-grpcio-1.46.7-10.el9.x86_64.rpm 31 MB/s | 2.0 MB 00:00 2024-09-16T20:45:41.343 INFO:teuthology.orchestra.run.smithi139.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 5.7 MB/s | 144 kB 00:00 2024-09-16T20:45:41.348 INFO:teuthology.orchestra.run.smithi043.stdout:(9/37): python3-ply-3.11-14.el9.noarch.rpm 232 kB/s | 106 kB 00:00 2024-09-16T20:45:41.368 INFO:teuthology.orchestra.run.smithi139.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 1.2 MB/s | 31 kB 00:00 2024-09-16T20:45:41.394 INFO:teuthology.orchestra.run.smithi139.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 7.4 MB/s | 188 kB 00:00 2024-09-16T20:45:41.432 INFO:teuthology.orchestra.run.smithi043.stdout:(10/37): python3-requests-2.25.1-8.el9.noarch.r 1.0 MB/s | 125 kB 00:00 2024-09-16T20:45:41.482 INFO:teuthology.orchestra.run.smithi043.stdout:(11/37): python3-cryptography-36.0.1-4.el9.x86_ 1.3 MB/s | 1.2 MB 00:00 2024-09-16T20:45:41.561 INFO:teuthology.orchestra.run.smithi139.stdout:(11/12): ceph-mgr-dashboard-19.3.0-4883.g6e8474 19 MB/s | 14 MB 00:00 2024-09-16T20:45:41.582 INFO:teuthology.orchestra.run.smithi043.stdout:(12/37): lua-5.4.4-4.el9.x86_64.rpm 1.2 MB/s | 188 kB 00:00 2024-09-16T20:45:41.602 INFO:teuthology.orchestra.run.smithi139.stdout:(12/12): protobuf-compiler-3.14.0-13.el9.x86_64 1.7 MB/s | 863 kB 00:00 2024-09-16T20:45:41.605 INFO:teuthology.orchestra.run.smithi139.stdout:-------------------------------------------------------------------------------- 2024-09-16T20:45:41.606 INFO:teuthology.orchestra.run.smithi139.stdout:Total 15 MB/s | 19 MB 00:01 2024-09-16T20:45:41.607 INFO:teuthology.orchestra.run.smithi043.stdout:(13/37): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-09-16T20:45:41.633 INFO:teuthology.orchestra.run.smithi043.stdout:(14/37): python3-markupsafe-1.1.1-12.el9.x86_64 695 kB/s | 35 kB 00:00 2024-09-16T20:45:41.654 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction check 2024-09-16T20:45:41.658 INFO:teuthology.orchestra.run.smithi043.stdout:(15/37): python3-packaging-20.9-5.el9.noarch.rp 1.5 MB/s | 77 kB 00:00 2024-09-16T20:45:41.674 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction check succeeded. 2024-09-16T20:45:41.674 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction test 2024-09-16T20:45:41.683 INFO:teuthology.orchestra.run.smithi043.stdout:(16/37): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-16T20:45:41.717 INFO:teuthology.orchestra.run.smithi043.stdout:(17/37): python3-urllib3-1.26.5-6.el9.noarch.rp 582 kB/s | 215 kB 00:00 2024-09-16T20:45:41.742 INFO:teuthology.orchestra.run.smithi043.stdout:(18/37): python3-toml-0.10.2-6.el9.noarch.rpm 496 kB/s | 42 kB 00:00 2024-09-16T20:45:41.792 INFO:teuthology.orchestra.run.smithi043.stdout:(19/37): luarocks-3.9.2-5.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-09-16T20:45:41.818 INFO:teuthology.orchestra.run.smithi043.stdout:(20/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 578 kB/s | 43 kB 00:00 2024-09-16T20:45:41.868 INFO:teuthology.orchestra.run.smithi043.stdout:(21/37): lua-devel-5.4.4-4.el9.x86_64.rpm 121 kB/s | 22 kB 00:00 2024-09-16T20:45:41.893 INFO:teuthology.orchestra.run.smithi043.stdout:(22/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.7 MB/s | 172 kB 00:00 2024-09-16T20:45:41.927 INFO:teuthology.orchestra.run.smithi043.stdout:(23/37): python3-jaraco-classes-3.2.1-5.el9.noa 731 kB/s | 18 kB 00:00 2024-09-16T20:45:41.952 INFO:teuthology.orchestra.run.smithi043.stdout:(24/37): python3-cherrypy-18.6.1-2.el9.noarch.r 2.6 MB/s | 358 kB 00:00 2024-09-16T20:45:41.962 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction test succeeded. 2024-09-16T20:45:41.962 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction 2024-09-16T20:45:41.977 INFO:teuthology.orchestra.run.smithi043.stdout:(25/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 98 kB/s | 11 kB 00:00 2024-09-16T20:45:42.003 INFO:teuthology.orchestra.run.smithi043.stdout:(26/37): python3-jaraco-collections-3.0.0-8.el9 315 kB/s | 23 kB 00:00 2024-09-16T20:45:42.028 INFO:teuthology.orchestra.run.smithi043.stdout:(27/37): python3-jaraco-functools-3.5.0-2.el9.n 258 kB/s | 19 kB 00:00 2024-09-16T20:45:42.053 INFO:teuthology.orchestra.run.smithi043.stdout:(28/37): python3-jaraco-text-3.2.0-6.el9.noarch 261 kB/s | 20 kB 00:00 2024-09-16T20:45:42.078 INFO:teuthology.orchestra.run.smithi043.stdout:(29/37): python3-logutils-0.3.5-21.el9.noarch.r 614 kB/s | 46 kB 00:00 2024-09-16T20:45:42.104 INFO:teuthology.orchestra.run.smithi043.stdout:(30/37): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2024-09-16T20:45:42.129 INFO:teuthology.orchestra.run.smithi043.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 328 kB/s | 16 kB 00:00 2024-09-16T20:45:42.154 INFO:teuthology.orchestra.run.smithi043.stdout:(32/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.7 MB/s | 90 kB 00:00 2024-09-16T20:45:42.179 INFO:teuthology.orchestra.run.smithi043.stdout:(33/37): python3-tempora-5.0.0-2.el9.noarch.rpm 712 kB/s | 36 kB 00:00 2024-09-16T20:45:42.205 INFO:teuthology.orchestra.run.smithi043.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-09-16T20:45:42.230 INFO:teuthology.orchestra.run.smithi043.stdout:(35/37): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-16T20:45:42.263 INFO:teuthology.orchestra.run.smithi043.stdout:(36/37): python3-zc-lockfile-2.0-10.el9.noarch. 342 kB/s | 20 kB 00:00 2024-09-16T20:45:42.289 INFO:teuthology.orchestra.run.smithi043.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.8 MB/s | 427 kB 00:00 2024-09-16T20:45:42.293 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-16T20:45:42.294 INFO:teuthology.orchestra.run.smithi043.stdout:Total 11 MB/s | 31 MB 00:02 2024-09-16T20:45:42.294 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-osd-2:19.3.0-4883.g6e8474a0.el9.x86_64 34/37 2024-09-16T20:45:42.324 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-osd-2:19.3.0-4883.g6e8474a0.el9.x86_64 34/37 2024-09-16T20:45:42.324 INFO:teuthology.orchestra.run.smithi029.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T20:45:42.324 INFO:teuthology.orchestra.run.smithi029.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-16T20:45:42.324 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T20:45:42.324 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T20:45:42.324 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:45:42.447 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-16T20:45:42.491 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-16T20:45:42.491 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-16T20:45:42.565 INFO:teuthology.orchestra.run.smithi139.stdout: Preparing : 1/1 2024-09-16T20:45:42.636 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-16T20:45:42.672 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-16T20:45:42.718 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-16T20:45:42.757 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-16T20:45:42.804 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-mon-2:19.3.0-4883.g6e8474a0.el9.x86_64 35/37 2024-09-16T20:45:42.830 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-mon-2:19.3.0-4883.g6e8474a0.el9.x86_64 35/37 2024-09-16T20:45:42.830 INFO:teuthology.orchestra.run.smithi029.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T20:45:42.830 INFO:teuthology.orchestra.run.smithi029.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-16T20:45:42.830 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T20:45:42.830 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T20:45:42.830 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:45:42.861 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-16T20:45:42.983 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-16T20:45:43.020 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-16T20:45:43.021 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-16T20:45:43.076 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-16T20:45:43.082 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-mds-2:19.3.0-4883.g6e8474a0.el9.x86_64 36/37 2024-09-16T20:45:43.108 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-mds-2:19.3.0-4883.g6e8474a0.el9.x86_64 36/37 2024-09-16T20:45:43.108 INFO:teuthology.orchestra.run.smithi029.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T20:45:43.108 INFO:teuthology.orchestra.run.smithi029.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-16T20:45:43.108 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T20:45:43.108 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T20:45:43.108 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:45:43.145 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-16T20:45:43.173 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-16T20:45:43.174 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-2:19.3.0-4883.g6e8474a0.el9.x86_64 37/37 2024-09-16T20:45:43.205 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4883.g6e8474a0.el9 10/12 2024-09-16T20:45:44.426 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-16T20:45:44.500 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-16T20:45:44.543 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-16T20:45:44.586 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-16T20:45:44.662 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-16T20:45:44.711 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-16T20:45:44.743 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-16T20:45:44.841 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-16T20:45:44.925 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-16T20:45:44.972 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-16T20:45:45.006 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-16T20:45:45.035 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-16T20:45:45.070 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-16T20:45:45.112 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-16T20:45:45.153 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-16T20:45:45.196 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-16T20:45:45.230 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-16T20:45:45.280 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-16T20:45:45.320 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-16T20:45:45.396 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-16T20:45:45.548 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-16T20:45:45.630 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-16T20:45:45.690 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-16T20:45:45.728 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-16T20:45:45.786 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-16T20:45:45.868 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-16T20:45:45.937 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-2:19.3.0-4883.g6e8474a0.el9.x86_64 37/37 2024-09-16T20:45:45.937 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/37 2024-09-16T20:45:45.937 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-mds-2:19.3.0-4883.g6e8474a0.el9.x86_64 2/37 2024-09-16T20:45:45.937 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-mgr-2:19.3.0-4883.g6e8474a0.el9.x86_64 3/37 2024-09-16T20:45:45.937 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-mon-2:19.3.0-4883.g6e8474a0.el9.x86_64 4/37 2024-09-16T20:45:45.937 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-osd-2:19.3.0-4883.g6e8474a0.el9.x86_64 5/37 2024-09-16T20:45:45.937 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4883.g6e8474a0.el9. 6/37 2024-09-16T20:45:45.937 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-16T20:45:45.938 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-16T20:45:45.938 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-16T20:45:45.938 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-16T20:45:45.938 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-16T20:45:45.938 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-16T20:45:45.938 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-16T20:45:45.938 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-16T20:45:45.938 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-16T20:45:45.938 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-16T20:45:45.938 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-16T20:45:45.938 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-16T20:45:45.938 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-16T20:45:45.938 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-16T20:45:45.938 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-16T20:45:45.939 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-16T20:45:45.939 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-16T20:45:45.939 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-16T20:45:45.939 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-16T20:45:45.939 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-16T20:45:45.939 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-16T20:45:45.939 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-16T20:45:45.939 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-16T20:45:45.939 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-16T20:45:45.940 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-16T20:45:45.940 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-16T20:45:45.940 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-16T20:45:45.940 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-16T20:45:45.940 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-16T20:45:45.940 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-16T20:45:46.080 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-16T20:45:46.132 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-16T20:45:46.212 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-16T20:45:46.311 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-16T20:45:46.358 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-16T20:45:46.376 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4883.g6e8474a0.el 11/12 2024-09-16T20:45:46.426 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4883.g6e8474a0.el9.noa 12/12 2024-09-16T20:45:46.499 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-16T20:45:46.668 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4883.g6e8474a0.el9. 32/37 2024-09-16T20:45:46.690 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-2:19.3.0-4883.g6e8474a0.el9.x86_64 33/37 2024-09-16T20:45:46.716 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4883.g6e8474a0.el9.x86_64 33/37 2024-09-16T20:45:46.716 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T20:45:46.716 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-16T20:45:46.717 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T20:45:46.717 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T20:45:46.717 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:45:47.037 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-16T20:45:47.037 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:45:47.037 INFO:teuthology.orchestra.run.smithi029.stdout:Installed: 2024-09-16T20:45:47.037 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:47.037 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mds-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:47.037 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:47.038 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-modules-core-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-16T20:45:47.038 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mon-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:47.038 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-osd-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:47.038 INFO:teuthology.orchestra.run.smithi029.stdout: lua-5.4.4-4.el9.x86_64 2024-09-16T20:45:47.038 INFO:teuthology.orchestra.run.smithi029.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-16T20:45:47.038 INFO:teuthology.orchestra.run.smithi029.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-16T20:45:47.038 INFO:teuthology.orchestra.run.smithi029.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-16T20:45:47.038 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-16T20:45:47.038 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-16T20:45:47.038 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-16T20:45:47.039 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-16T20:45:47.039 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-16T20:45:47.039 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-16T20:45:47.039 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-16T20:45:47.039 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-16T20:45:47.039 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-16T20:45:47.039 INFO:teuthology.orchestra.run.smithi029.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-16T20:45:47.039 INFO:teuthology.orchestra.run.smithi029.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-16T20:45:47.039 INFO:teuthology.orchestra.run.smithi029.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-16T20:45:47.039 INFO:teuthology.orchestra.run.smithi029.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-16T20:45:47.039 INFO:teuthology.orchestra.run.smithi029.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-16T20:45:47.039 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-16T20:45:47.039 INFO:teuthology.orchestra.run.smithi029.stdout: python3-ply-3.11-14.el9.noarch 2024-09-16T20:45:47.040 INFO:teuthology.orchestra.run.smithi029.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-16T20:45:47.040 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-16T20:45:47.040 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-16T20:45:47.040 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-16T20:45:47.040 INFO:teuthology.orchestra.run.smithi029.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-16T20:45:47.040 INFO:teuthology.orchestra.run.smithi029.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-16T20:45:47.040 INFO:teuthology.orchestra.run.smithi029.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-16T20:45:47.040 INFO:teuthology.orchestra.run.smithi029.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-16T20:45:47.040 INFO:teuthology.orchestra.run.smithi029.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-16T20:45:47.040 INFO:teuthology.orchestra.run.smithi029.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-16T20:45:47.040 INFO:teuthology.orchestra.run.smithi029.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-16T20:45:47.040 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:45:47.040 INFO:teuthology.orchestra.run.smithi029.stdout:Complete! 2024-09-16T20:45:47.498 DEBUG:teuthology.orchestra.run.smithi029:> sudo yum -y install ceph-base 2024-09-16T20:45:48.113 INFO:teuthology.orchestra.run.smithi029.stdout:Last metadata expiration check: 0:01:02 ago on Mon 16 Sep 2024 08:44:46 PM UTC. 2024-09-16T20:45:48.207 INFO:teuthology.orchestra.run.smithi029.stdout:Package ceph-base-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-16T20:45:48.251 INFO:teuthology.orchestra.run.smithi029.stdout:Dependencies resolved. 2024-09-16T20:45:48.252 INFO:teuthology.orchestra.run.smithi029.stdout:Nothing to do. 2024-09-16T20:45:48.252 INFO:teuthology.orchestra.run.smithi029.stdout:Complete! 2024-09-16T20:45:48.265 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-osd-2:19.3.0-4883.g6e8474a0.el9.x86_64 34/37 2024-09-16T20:45:48.294 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-osd-2:19.3.0-4883.g6e8474a0.el9.x86_64 34/37 2024-09-16T20:45:48.294 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T20:45:48.294 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-16T20:45:48.294 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T20:45:48.295 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T20:45:48.295 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:45:48.334 DEBUG:teuthology.orchestra.run.smithi029:> sudo yum -y install cephadm 2024-09-16T20:45:48.344 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4883.g6e8474a0.el9.noa 12/12 2024-09-16T20:45:48.344 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4883.g6e8474a0.el 1/12 2024-09-16T20:45:48.344 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4883.g6e8474a0.el9.noa 2/12 2024-09-16T20:45:48.345 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4883.g6e8474a0.el9 3/12 2024-09-16T20:45:48.345 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-16T20:45:48.345 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-16T20:45:48.345 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-16T20:45:48.345 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-16T20:45:48.345 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-16T20:45:48.345 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-16T20:45:48.345 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-16T20:45:48.345 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-16T20:45:48.678 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-16T20:45:48.679 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:45:48.679 INFO:teuthology.orchestra.run.smithi139.stdout:Installed: 2024-09-16T20:45:48.679 INFO:teuthology.orchestra.run.smithi139.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-16T20:45:48.679 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-grafana-dashboards-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-16T20:45:48.679 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-dashboard-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-16T20:45:48.679 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-prometheus-alerts-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-16T20:45:48.679 INFO:teuthology.orchestra.run.smithi139.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-16T20:45:48.679 INFO:teuthology.orchestra.run.smithi139.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-16T20:45:48.679 INFO:teuthology.orchestra.run.smithi139.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-16T20:45:48.679 INFO:teuthology.orchestra.run.smithi139.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-16T20:45:48.679 INFO:teuthology.orchestra.run.smithi139.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-16T20:45:48.679 INFO:teuthology.orchestra.run.smithi139.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-16T20:45:48.680 INFO:teuthology.orchestra.run.smithi139.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-16T20:45:48.680 INFO:teuthology.orchestra.run.smithi139.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-16T20:45:48.680 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:45:48.680 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-16T20:45:48.767 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mon-2:19.3.0-4883.g6e8474a0.el9.x86_64 35/37 2024-09-16T20:45:48.792 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mon-2:19.3.0-4883.g6e8474a0.el9.x86_64 35/37 2024-09-16T20:45:48.792 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T20:45:48.792 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-16T20:45:48.792 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T20:45:48.792 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T20:45:48.792 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:45:48.930 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-16T20:45:48.940 INFO:teuthology.orchestra.run.smithi029.stdout:Last metadata expiration check: 0:01:02 ago on Mon 16 Sep 2024 08:44:46 PM UTC. 2024-09-16T20:45:49.034 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mds-2:19.3.0-4883.g6e8474a0.el9.x86_64 36/37 2024-09-16T20:45:49.059 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mds-2:19.3.0-4883.g6e8474a0.el9.x86_64 36/37 2024-09-16T20:45:49.059 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T20:45:49.059 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-16T20:45:49.059 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T20:45:49.059 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T20:45:49.059 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:45:49.076 INFO:teuthology.orchestra.run.smithi029.stdout:Dependencies resolved. 2024-09-16T20:45:49.077 INFO:teuthology.orchestra.run.smithi029.stdout:================================================================================ 2024-09-16T20:45:49.077 INFO:teuthology.orchestra.run.smithi029.stdout: Package Arch Version Repository Size 2024-09-16T20:45:49.077 INFO:teuthology.orchestra.run.smithi029.stdout:================================================================================ 2024-09-16T20:45:49.077 INFO:teuthology.orchestra.run.smithi029.stdout:Installing: 2024-09-16T20:45:49.077 INFO:teuthology.orchestra.run.smithi029.stdout: cephadm noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 784 k 2024-09-16T20:45:49.077 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:45:49.078 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction Summary 2024-09-16T20:45:49.078 INFO:teuthology.orchestra.run.smithi029.stdout:================================================================================ 2024-09-16T20:45:49.078 INFO:teuthology.orchestra.run.smithi029.stdout:Install 1 Package 2024-09-16T20:45:49.078 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:45:49.078 INFO:teuthology.orchestra.run.smithi029.stdout:Total download size: 784 k 2024-09-16T20:45:49.078 INFO:teuthology.orchestra.run.smithi029.stdout:Installed size: 791 k 2024-09-16T20:45:49.078 INFO:teuthology.orchestra.run.smithi029.stdout:Downloading Packages: 2024-09-16T20:45:49.125 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-2:19.3.0-4883.g6e8474a0.el9.x86_64 37/37 2024-09-16T20:45:49.389 INFO:teuthology.orchestra.run.smithi029.stdout:cephadm-19.3.0-4883.g6e8474a0.el9.noarch.rpm 2.5 MB/s | 784 kB 00:00 2024-09-16T20:45:49.390 INFO:teuthology.orchestra.run.smithi029.stdout:-------------------------------------------------------------------------------- 2024-09-16T20:45:49.390 INFO:teuthology.orchestra.run.smithi029.stdout:Total 2.5 MB/s | 784 kB 00:00 2024-09-16T20:45:49.390 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction check 2024-09-16T20:45:49.395 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction check succeeded. 2024-09-16T20:45:49.396 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction test 2024-09-16T20:45:49.402 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction test succeeded. 2024-09-16T20:45:49.402 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction 2024-09-16T20:45:49.481 INFO:teuthology.orchestra.run.smithi029.stdout: Preparing : 1/1 2024-09-16T20:45:49.511 INFO:teuthology.orchestra.run.smithi139.stdout:Last metadata expiration check: 0:01:08 ago on Mon 16 Sep 2024 08:44:41 PM UTC. 2024-09-16T20:45:49.644 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-16T20:45:49.645 INFO:teuthology.orchestra.run.smithi139.stdout:======================================================================================= 2024-09-16T20:45:49.645 INFO:teuthology.orchestra.run.smithi139.stdout: Package Arch Version Repository Size 2024-09-16T20:45:49.645 INFO:teuthology.orchestra.run.smithi139.stdout:======================================================================================= 2024-09-16T20:45:49.645 INFO:teuthology.orchestra.run.smithi139.stdout:Installing: 2024-09-16T20:45:49.645 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 7.4 M 2024-09-16T20:45:49.645 INFO:teuthology.orchestra.run.smithi139.stdout:Installing dependencies: 2024-09-16T20:45:49.645 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-16T20:45:49.645 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-16T20:45:49.645 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-16T20:45:49.645 INFO:teuthology.orchestra.run.smithi139.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-16T20:45:49.646 INFO:teuthology.orchestra.run.smithi139.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-16T20:45:49.646 INFO:teuthology.orchestra.run.smithi139.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-16T20:45:49.646 INFO:teuthology.orchestra.run.smithi139.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-16T20:45:49.646 INFO:teuthology.orchestra.run.smithi139.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-16T20:45:49.646 INFO:teuthology.orchestra.run.smithi139.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-16T20:45:49.646 INFO:teuthology.orchestra.run.smithi139.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-16T20:45:49.646 INFO:teuthology.orchestra.run.smithi139.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-16T20:45:49.646 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:45:49.646 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction Summary 2024-09-16T20:45:49.646 INFO:teuthology.orchestra.run.smithi139.stdout:======================================================================================= 2024-09-16T20:45:49.646 INFO:teuthology.orchestra.run.smithi139.stdout:Install 12 Packages 2024-09-16T20:45:49.646 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:45:49.647 INFO:teuthology.orchestra.run.smithi139.stdout:Total download size: 38 M 2024-09-16T20:45:49.647 INFO:teuthology.orchestra.run.smithi139.stdout:Installed size: 212 M 2024-09-16T20:45:49.647 INFO:teuthology.orchestra.run.smithi139.stdout:Downloading Packages: 2024-09-16T20:45:49.992 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 1/1 2024-09-16T20:45:50.076 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 1/1 2024-09-16T20:45:50.505 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 1/1 2024-09-16T20:45:50.608 INFO:teuthology.orchestra.run.smithi139.stdout:(1/12): ceph-mgr-diskprediction-local-19.3.0-48 16 MB/s | 7.4 MB 00:00 2024-09-16T20:45:50.800 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 1/1 2024-09-16T20:45:50.801 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:45:50.801 INFO:teuthology.orchestra.run.smithi029.stdout:Installed: 2024-09-16T20:45:50.801 INFO:teuthology.orchestra.run.smithi029.stdout: cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-16T20:45:50.801 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:45:50.801 INFO:teuthology.orchestra.run.smithi029.stdout:Complete! 2024-09-16T20:45:50.911 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-2:19.3.0-4883.g6e8474a0.el9.x86_64 37/37 2024-09-16T20:45:50.911 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/37 2024-09-16T20:45:50.911 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mds-2:19.3.0-4883.g6e8474a0.el9.x86_64 2/37 2024-09-16T20:45:50.911 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-2:19.3.0-4883.g6e8474a0.el9.x86_64 3/37 2024-09-16T20:45:50.911 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mon-2:19.3.0-4883.g6e8474a0.el9.x86_64 4/37 2024-09-16T20:45:50.911 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-osd-2:19.3.0-4883.g6e8474a0.el9.x86_64 5/37 2024-09-16T20:45:50.911 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4883.g6e8474a0.el9. 6/37 2024-09-16T20:45:50.912 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-16T20:45:50.912 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-16T20:45:50.912 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-16T20:45:50.912 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-16T20:45:50.912 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-16T20:45:50.912 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-16T20:45:50.912 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-16T20:45:50.912 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-16T20:45:50.912 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-16T20:45:50.912 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-16T20:45:50.912 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-16T20:45:50.912 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-16T20:45:50.912 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-16T20:45:50.912 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-16T20:45:50.913 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-16T20:45:50.913 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-16T20:45:50.913 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-16T20:45:50.913 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-16T20:45:50.913 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-16T20:45:50.913 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-16T20:45:50.913 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-16T20:45:50.913 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-16T20:45:50.913 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-16T20:45:50.913 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-16T20:45:50.913 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-16T20:45:50.913 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-16T20:45:50.913 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-16T20:45:50.914 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-16T20:45:50.914 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-16T20:45:50.914 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-16T20:45:51.049 INFO:teuthology.orchestra.run.smithi139.stdout:(2/12): libquadmath-11.5.0-2.el9.x86_64.rpm 214 kB/s | 191 kB 00:00 2024-09-16T20:45:51.070 DEBUG:teuthology.orchestra.run.smithi029:> sudo yum -y install ceph-immutable-object-cache 2024-09-16T20:45:51.200 INFO:teuthology.orchestra.run.smithi139.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 55 kB/s | 33 kB 00:00 2024-09-16T20:45:51.679 INFO:teuthology.orchestra.run.smithi029.stdout:Last metadata expiration check: 0:01:05 ago on Mon 16 Sep 2024 08:44:46 PM UTC. 2024-09-16T20:45:51.725 INFO:teuthology.orchestra.run.smithi139.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 35 kB/s | 18 kB 00:00 2024-09-16T20:45:51.816 INFO:teuthology.orchestra.run.smithi029.stdout:Dependencies resolved. 2024-09-16T20:45:51.817 INFO:teuthology.orchestra.run.smithi029.stdout:================================================================================ 2024-09-16T20:45:51.817 INFO:teuthology.orchestra.run.smithi029.stdout: Package Arch Version Repo Size 2024-09-16T20:45:51.817 INFO:teuthology.orchestra.run.smithi029.stdout:================================================================================ 2024-09-16T20:45:51.818 INFO:teuthology.orchestra.run.smithi029.stdout:Installing: 2024-09-16T20:45:51.818 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 145 k 2024-09-16T20:45:51.818 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:45:51.818 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction Summary 2024-09-16T20:45:51.818 INFO:teuthology.orchestra.run.smithi029.stdout:================================================================================ 2024-09-16T20:45:51.818 INFO:teuthology.orchestra.run.smithi029.stdout:Install 1 Package 2024-09-16T20:45:51.818 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:45:51.818 INFO:teuthology.orchestra.run.smithi029.stdout:Total download size: 145 k 2024-09-16T20:45:51.818 INFO:teuthology.orchestra.run.smithi029.stdout:Installed size: 432 k 2024-09-16T20:45:51.818 INFO:teuthology.orchestra.run.smithi029.stdout:Downloading Packages: 2024-09-16T20:45:51.857 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-16T20:45:51.857 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:45:51.858 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-16T20:45:51.858 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:51.858 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mds-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:51.858 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:51.858 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-modules-core-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-16T20:45:51.858 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mon-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:51.858 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-osd-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:51.858 INFO:teuthology.orchestra.run.smithi043.stdout: lua-5.4.4-4.el9.x86_64 2024-09-16T20:45:51.858 INFO:teuthology.orchestra.run.smithi043.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-16T20:45:51.858 INFO:teuthology.orchestra.run.smithi043.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-16T20:45:51.858 INFO:teuthology.orchestra.run.smithi043.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-16T20:45:51.858 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-16T20:45:51.858 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-16T20:45:51.859 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-16T20:45:51.859 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-16T20:45:51.859 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-16T20:45:51.859 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-16T20:45:51.859 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-16T20:45:51.859 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-16T20:45:51.859 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-16T20:45:51.859 INFO:teuthology.orchestra.run.smithi043.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-16T20:45:51.859 INFO:teuthology.orchestra.run.smithi043.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-16T20:45:51.859 INFO:teuthology.orchestra.run.smithi043.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-16T20:45:51.859 INFO:teuthology.orchestra.run.smithi043.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-16T20:45:51.859 INFO:teuthology.orchestra.run.smithi043.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-16T20:45:51.859 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-16T20:45:51.860 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ply-3.11-14.el9.noarch 2024-09-16T20:45:51.860 INFO:teuthology.orchestra.run.smithi043.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-16T20:45:51.860 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-16T20:45:51.860 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-16T20:45:51.860 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-16T20:45:51.860 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-16T20:45:51.860 INFO:teuthology.orchestra.run.smithi043.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-16T20:45:51.860 INFO:teuthology.orchestra.run.smithi043.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-16T20:45:51.860 INFO:teuthology.orchestra.run.smithi043.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-16T20:45:51.860 INFO:teuthology.orchestra.run.smithi043.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-16T20:45:51.860 INFO:teuthology.orchestra.run.smithi043.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-16T20:45:51.860 INFO:teuthology.orchestra.run.smithi043.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-16T20:45:51.860 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:45:51.860 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-16T20:45:51.934 INFO:teuthology.orchestra.run.smithi139.stdout:(5/12): libgfortran-11.5.0-2.el9.x86_64.rpm 451 kB/s | 801 kB 00:01 2024-09-16T20:45:52.047 INFO:teuthology.orchestra.run.smithi029.stdout:ceph-immutable-object-cache-19.3.0-4883.g6e8474 634 kB/s | 145 kB 00:00 2024-09-16T20:45:52.048 INFO:teuthology.orchestra.run.smithi029.stdout:-------------------------------------------------------------------------------- 2024-09-16T20:45:52.048 INFO:teuthology.orchestra.run.smithi029.stdout:Total 629 kB/s | 145 kB 00:00 2024-09-16T20:45:52.048 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction check 2024-09-16T20:45:52.056 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction check succeeded. 2024-09-16T20:45:52.056 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction test 2024-09-16T20:45:52.118 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction test succeeded. 2024-09-16T20:45:52.119 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction 2024-09-16T20:45:52.218 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-base 2024-09-16T20:45:52.271 INFO:teuthology.orchestra.run.smithi029.stdout: Preparing : 1/1 2024-09-16T20:45:52.292 INFO:teuthology.orchestra.run.smithi139.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 68 kB/s | 38 kB 00:00 2024-09-16T20:45:52.349 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4883.g6e8474a0. 1/1 2024-09-16T20:45:52.375 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4883.g6e8474a0. 1/1 2024-09-16T20:45:52.375 INFO:teuthology.orchestra.run.smithi029.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T20:45:52.376 INFO:teuthology.orchestra.run.smithi029.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-16T20:45:52.376 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:45:52.434 INFO:teuthology.orchestra.run.smithi139.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 2.2 MB/s | 3.0 MB 00:01 2024-09-16T20:45:52.828 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:00:54 ago on Mon 16 Sep 2024 08:44:58 PM UTC. 2024-09-16T20:45:52.921 INFO:teuthology.orchestra.run.smithi043.stdout:Package ceph-base-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-16T20:45:52.966 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-16T20:45:52.967 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-09-16T20:45:52.967 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-16T20:45:53.031 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install cephadm 2024-09-16T20:45:53.142 INFO:teuthology.orchestra.run.smithi139.stdout:(8/12): python3-devel-3.9.19-8.el9.x86_64.rpm 288 kB/s | 245 kB 00:00 2024-09-16T20:45:53.383 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4883.g6e8474a0. 1/1 2024-09-16T20:45:53.383 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:45:53.383 INFO:teuthology.orchestra.run.smithi029.stdout:Installed: 2024-09-16T20:45:53.383 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-immutable-object-cache-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:53.383 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:45:53.383 INFO:teuthology.orchestra.run.smithi029.stdout:Complete! 2024-09-16T20:45:53.401 INFO:teuthology.orchestra.run.smithi139.stdout:(9/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 3.4 MB/s | 4.9 MB 00:01 2024-09-16T20:45:53.633 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:00:55 ago on Mon 16 Sep 2024 08:44:58 PM UTC. 2024-09-16T20:45:53.668 DEBUG:teuthology.orchestra.run.smithi029:> sudo yum -y install ceph-mgr 2024-09-16T20:45:53.668 INFO:teuthology.orchestra.run.smithi139.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.1 MB/s | 5.1 MB 00:01 2024-09-16T20:45:53.769 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-16T20:45:53.770 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T20:45:53.770 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-09-16T20:45:53.770 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T20:45:53.770 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-16T20:45:53.771 INFO:teuthology.orchestra.run.smithi043.stdout: cephadm noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 784 k 2024-09-16T20:45:53.771 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:45:53.771 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-16T20:45:53.771 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T20:45:53.771 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-09-16T20:45:53.771 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:45:53.771 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 784 k 2024-09-16T20:45:53.771 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 791 k 2024-09-16T20:45:53.771 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-16T20:45:54.051 INFO:teuthology.orchestra.run.smithi043.stdout:cephadm-19.3.0-4883.g6e8474a0.el9.noarch.rpm 2.7 MB/s | 784 kB 00:00 2024-09-16T20:45:54.051 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-16T20:45:54.051 INFO:teuthology.orchestra.run.smithi043.stdout:Total 2.7 MB/s | 784 kB 00:00 2024-09-16T20:45:54.051 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-16T20:45:54.057 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-16T20:45:54.057 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-16T20:45:54.063 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-16T20:45:54.064 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-16T20:45:54.118 INFO:teuthology.orchestra.run.smithi139.stdout:(11/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 257 kB/s | 250 kB 00:00 2024-09-16T20:45:54.138 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-16T20:45:54.275 INFO:teuthology.orchestra.run.smithi029.stdout:Last metadata expiration check: 0:01:08 ago on Mon 16 Sep 2024 08:44:46 PM UTC. 2024-09-16T20:45:54.367 INFO:teuthology.orchestra.run.smithi029.stdout:Package ceph-mgr-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-16T20:45:54.412 INFO:teuthology.orchestra.run.smithi029.stdout:Dependencies resolved. 2024-09-16T20:45:54.413 INFO:teuthology.orchestra.run.smithi029.stdout:Nothing to do. 2024-09-16T20:45:54.414 INFO:teuthology.orchestra.run.smithi029.stdout:Complete! 2024-09-16T20:45:54.499 DEBUG:teuthology.orchestra.run.smithi029:> sudo yum -y install ceph-mgr-dashboard 2024-09-16T20:45:54.622 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 1/1 2024-09-16T20:45:54.715 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 1/1 2024-09-16T20:45:55.070 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 1/1 2024-09-16T20:45:55.113 INFO:teuthology.orchestra.run.smithi029.stdout:Last metadata expiration check: 0:01:09 ago on Mon 16 Sep 2024 08:44:46 PM UTC. 2024-09-16T20:45:55.254 INFO:teuthology.orchestra.run.smithi029.stdout:Dependencies resolved. 2024-09-16T20:45:55.255 INFO:teuthology.orchestra.run.smithi029.stdout:================================================================================ 2024-09-16T20:45:55.256 INFO:teuthology.orchestra.run.smithi029.stdout: Package Arch Version Repository Size 2024-09-16T20:45:55.256 INFO:teuthology.orchestra.run.smithi029.stdout:================================================================================ 2024-09-16T20:45:55.256 INFO:teuthology.orchestra.run.smithi029.stdout:Installing: 2024-09-16T20:45:55.256 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 14 M 2024-09-16T20:45:55.256 INFO:teuthology.orchestra.run.smithi029.stdout:Installing dependencies: 2024-09-16T20:45:55.256 INFO:teuthology.orchestra.run.smithi029.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-16T20:45:55.256 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 34 k 2024-09-16T20:45:55.256 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 16 k 2024-09-16T20:45:55.256 INFO:teuthology.orchestra.run.smithi029.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-16T20:45:55.256 INFO:teuthology.orchestra.run.smithi029.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-16T20:45:55.256 INFO:teuthology.orchestra.run.smithi029.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-16T20:45:55.256 INFO:teuthology.orchestra.run.smithi029.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-16T20:45:55.257 INFO:teuthology.orchestra.run.smithi029.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-16T20:45:55.257 INFO:teuthology.orchestra.run.smithi029.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-16T20:45:55.257 INFO:teuthology.orchestra.run.smithi029.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-16T20:45:55.257 INFO:teuthology.orchestra.run.smithi029.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-16T20:45:55.257 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:45:55.257 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction Summary 2024-09-16T20:45:55.257 INFO:teuthology.orchestra.run.smithi029.stdout:================================================================================ 2024-09-16T20:45:55.257 INFO:teuthology.orchestra.run.smithi029.stdout:Install 12 Packages 2024-09-16T20:45:55.257 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:45:55.258 INFO:teuthology.orchestra.run.smithi029.stdout:Total download size: 19 M 2024-09-16T20:45:55.258 INFO:teuthology.orchestra.run.smithi029.stdout:Installed size: 159 M 2024-09-16T20:45:55.258 INFO:teuthology.orchestra.run.smithi029.stdout:Downloading Packages: 2024-09-16T20:45:55.302 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 1/1 2024-09-16T20:45:55.302 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:45:55.302 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-16T20:45:55.302 INFO:teuthology.orchestra.run.smithi043.stdout: cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-16T20:45:55.302 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:45:55.302 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-16T20:45:55.402 INFO:teuthology.orchestra.run.smithi139.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 8.2 MB/s | 16 MB 00:01 2024-09-16T20:45:55.403 INFO:teuthology.orchestra.run.smithi139.stdout:-------------------------------------------------------------------------------- 2024-09-16T20:45:55.404 INFO:teuthology.orchestra.run.smithi139.stdout:Total 6.6 MB/s | 38 MB 00:05 2024-09-16T20:45:55.515 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-immutable-object-cache 2024-09-16T20:45:55.763 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction check 2024-09-16T20:45:55.790 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction check succeeded. 2024-09-16T20:45:55.790 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction test 2024-09-16T20:45:56.126 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:00:58 ago on Mon 16 Sep 2024 08:44:58 PM UTC. 2024-09-16T20:45:56.221 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction test succeeded. 2024-09-16T20:45:56.221 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction 2024-09-16T20:45:56.235 INFO:teuthology.orchestra.run.smithi029.stdout:(1/12): ceph-prometheus-alerts-19.3.0-4883.g6e8 81 kB/s | 16 kB 00:00 2024-09-16T20:45:56.260 INFO:teuthology.orchestra.run.smithi029.stdout:(2/12): ceph-grafana-dashboards-19.3.0-4883.g6e 150 kB/s | 34 kB 00:00 2024-09-16T20:45:56.264 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-16T20:45:56.265 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T20:45:56.265 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repo Size 2024-09-16T20:45:56.265 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T20:45:56.265 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-16T20:45:56.265 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 145 k 2024-09-16T20:45:56.265 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:45:56.265 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-16T20:45:56.265 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T20:45:56.266 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-09-16T20:45:56.266 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:45:56.266 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 145 k 2024-09-16T20:45:56.266 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 432 k 2024-09-16T20:45:56.266 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-16T20:45:56.386 INFO:teuthology.orchestra.run.smithi029.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 2.1 MB/s | 269 kB 00:00 2024-09-16T20:45:56.545 INFO:teuthology.orchestra.run.smithi029.stdout:(4/12): protobuf-compiler-3.14.0-13.el9.x86_64. 5.3 MB/s | 863 kB 00:00 2024-09-16T20:45:56.548 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-immutable-object-cache-19.3.0-4883.g6e8474 515 kB/s | 145 kB 00:00 2024-09-16T20:45:56.548 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-16T20:45:56.548 INFO:teuthology.orchestra.run.smithi043.stdout:Total 512 kB/s | 145 kB 00:00 2024-09-16T20:45:56.549 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-16T20:45:56.557 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-16T20:45:56.557 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-16T20:45:56.595 INFO:teuthology.orchestra.run.smithi029.stdout:(5/12): protobuf-3.14.0-13.el9.x86_64.rpm 2.8 MB/s | 1.0 MB 00:00 2024-09-16T20:45:56.618 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-16T20:45:56.618 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-16T20:45:56.664 INFO:teuthology.orchestra.run.smithi029.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 284 kB/s | 19 kB 00:00 2024-09-16T20:45:56.698 INFO:teuthology.orchestra.run.smithi029.stdout:(7/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 3.5 MB/s | 548 kB 00:00 2024-09-16T20:45:56.732 INFO:teuthology.orchestra.run.smithi029.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 4.2 MB/s | 144 kB 00:00 2024-09-16T20:45:56.769 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-16T20:45:56.832 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4883.g6e8474a0. 1/1 2024-09-16T20:45:56.859 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4883.g6e8474a0. 1/1 2024-09-16T20:45:56.859 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T20:45:56.859 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-16T20:45:56.859 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:45:56.974 INFO:teuthology.orchestra.run.smithi029.stdout:(9/12): ceph-mgr-dashboard-19.3.0-4883.g6e8474a 15 MB/s | 14 MB 00:00 2024-09-16T20:45:57.000 INFO:teuthology.orchestra.run.smithi029.stdout:(10/12): python3-repoze-lru-0.7-16.el9.noarch.r 115 kB/s | 31 kB 00:00 2024-09-16T20:45:57.008 INFO:teuthology.orchestra.run.smithi139.stdout: Preparing : 1/1 2024-09-16T20:45:57.067 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-16T20:45:57.100 INFO:teuthology.orchestra.run.smithi029.stdout:(11/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 4.7 MB/s | 2.0 MB 00:00 2024-09-16T20:45:57.116 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-16T20:45:57.125 INFO:teuthology.orchestra.run.smithi029.stdout:(12/12): python3-routes-2.5.1-5.el9.noarch.rpm 1.2 MB/s | 188 kB 00:00 2024-09-16T20:45:57.130 INFO:teuthology.orchestra.run.smithi029.stdout:-------------------------------------------------------------------------------- 2024-09-16T20:45:57.130 INFO:teuthology.orchestra.run.smithi029.stdout:Total 10 MB/s | 19 MB 00:01 2024-09-16T20:45:57.183 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction check 2024-09-16T20:45:57.205 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction check succeeded. 2024-09-16T20:45:57.205 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction test 2024-09-16T20:45:57.216 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-16T20:45:57.251 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-16T20:45:57.427 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-16T20:45:57.458 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-16T20:45:57.492 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction test succeeded. 2024-09-16T20:45:57.493 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction 2024-09-16T20:45:57.534 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-16T20:45:57.851 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4883.g6e8474a0. 1/1 2024-09-16T20:45:57.851 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:45:57.851 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-16T20:45:57.851 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-immutable-object-cache-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:45:57.851 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:45:57.852 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-16T20:45:58.070 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-mgr 2024-09-16T20:45:58.117 INFO:teuthology.orchestra.run.smithi029.stdout: Preparing : 1/1 2024-09-16T20:45:58.193 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-16T20:45:58.236 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-16T20:45:58.280 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-16T20:45:58.321 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-16T20:45:58.337 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-16T20:45:58.376 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-16T20:45:58.487 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-16T20:45:58.655 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-16T20:45:58.679 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:00 ago on Mon 16 Sep 2024 08:44:58 PM UTC. 2024-09-16T20:45:58.764 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-16T20:45:58.772 INFO:teuthology.orchestra.run.smithi043.stdout:Package ceph-mgr-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-16T20:45:58.817 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-16T20:45:58.818 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-09-16T20:45:58.818 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-16T20:45:58.822 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-16T20:45:58.860 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-16T20:45:58.891 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-mgr-dashboard 2024-09-16T20:45:58.900 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4883.g6e8474a0.el9 10/12 2024-09-16T20:45:59.490 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:01 ago on Mon 16 Sep 2024 08:44:58 PM UTC. 2024-09-16T20:45:59.590 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-16T20:45:59.627 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-16T20:45:59.628 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T20:45:59.629 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-09-16T20:45:59.629 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T20:45:59.629 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-16T20:45:59.629 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 14 M 2024-09-16T20:45:59.629 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-09-16T20:45:59.629 INFO:teuthology.orchestra.run.smithi043.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-16T20:45:59.629 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 34 k 2024-09-16T20:45:59.629 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 16 k 2024-09-16T20:45:59.629 INFO:teuthology.orchestra.run.smithi043.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-16T20:45:59.629 INFO:teuthology.orchestra.run.smithi043.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-16T20:45:59.629 INFO:teuthology.orchestra.run.smithi043.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-16T20:45:59.629 INFO:teuthology.orchestra.run.smithi043.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-16T20:45:59.630 INFO:teuthology.orchestra.run.smithi043.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-16T20:45:59.630 INFO:teuthology.orchestra.run.smithi043.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-16T20:45:59.630 INFO:teuthology.orchestra.run.smithi043.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-16T20:45:59.630 INFO:teuthology.orchestra.run.smithi043.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-16T20:45:59.630 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:45:59.630 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-16T20:45:59.630 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T20:45:59.630 INFO:teuthology.orchestra.run.smithi043.stdout:Install 12 Packages 2024-09-16T20:45:59.630 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:45:59.631 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 19 M 2024-09-16T20:45:59.631 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 159 M 2024-09-16T20:45:59.631 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-16T20:46:00.109 INFO:teuthology.orchestra.run.smithi043.stdout:(1/12): ceph-grafana-dashboards-19.3.0-4883.g6e 236 kB/s | 34 kB 00:00 2024-09-16T20:46:00.134 INFO:teuthology.orchestra.run.smithi043.stdout:(2/12): ceph-prometheus-alerts-19.3.0-4883.g6e8 96 kB/s | 16 kB 00:00 2024-09-16T20:46:00.243 INFO:teuthology.orchestra.run.smithi043.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 2.4 MB/s | 269 kB 00:00 2024-09-16T20:46:00.377 INFO:teuthology.orchestra.run.smithi043.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 3.8 MB/s | 1.0 MB 00:00 2024-09-16T20:46:00.427 INFO:teuthology.orchestra.run.smithi043.stdout:(5/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 11 MB/s | 548 kB 00:00 2024-09-16T20:46:00.452 INFO:teuthology.orchestra.run.smithi043.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 786 kB/s | 19 kB 00:00 2024-09-16T20:46:00.635 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-16T20:46:00.636 INFO:teuthology.orchestra.run.smithi043.stdout:(7/12): ceph-mgr-dashboard-19.3.0-4883.g6e8474a 21 MB/s | 14 MB 00:00 2024-09-16T20:46:00.673 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4883.g6e847 12/12 2024-09-16T20:46:00.686 INFO:teuthology.orchestra.run.smithi043.stdout:(8/12): protobuf-compiler-3.14.0-13.el9.x86_64. 1.9 MB/s | 863 kB 00:00 2024-09-16T20:46:00.736 INFO:teuthology.orchestra.run.smithi043.stdout:(9/12): python3-grpcio-1.46.7-10.el9.x86_64.rpm 7.2 MB/s | 2.0 MB 00:00 2024-09-16T20:46:00.770 INFO:teuthology.orchestra.run.smithi043.stdout:(10/12): python3-grpcio-tools-1.46.7-10.el9.x86 1.1 MB/s | 144 kB 00:00 2024-09-16T20:46:00.795 INFO:teuthology.orchestra.run.smithi043.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 3.6 MB/s | 188 kB 00:00 2024-09-16T20:46:00.820 INFO:teuthology.orchestra.run.smithi043.stdout:(12/12): python3-repoze-lru-0.7-16.el9.noarch.r 233 kB/s | 31 kB 00:00 2024-09-16T20:46:00.825 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-16T20:46:00.825 INFO:teuthology.orchestra.run.smithi043.stdout:Total 16 MB/s | 19 MB 00:01 2024-09-16T20:46:00.875 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-16T20:46:00.895 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-16T20:46:00.896 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-16T20:46:01.191 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-16T20:46:01.191 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-16T20:46:01.801 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-16T20:46:01.880 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-16T20:46:01.915 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-16T20:46:01.979 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-16T20:46:02.008 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-16T20:46:02.107 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-16T20:46:02.240 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-16T20:46:02.281 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4883.g6e8474a0.el 11/12 2024-09-16T20:46:02.331 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-16T20:46:02.363 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4883.g6e8474a0.el9.noa 12/12 2024-09-16T20:46:02.380 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-16T20:46:02.458 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-16T20:46:02.491 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4883.g6e8474a0.el9 10/12 2024-09-16T20:46:03.155 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4883.g6e847 12/12 2024-09-16T20:46:03.155 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4883.g6e847 1/12 2024-09-16T20:46:03.155 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-16T20:46:03.155 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-16T20:46:03.155 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-16T20:46:03.156 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-16T20:46:03.156 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-16T20:46:03.156 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-16T20:46:03.156 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-16T20:46:03.156 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-16T20:46:03.156 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-16T20:46:03.156 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-16T20:46:03.512 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-16T20:46:03.512 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:46:03.512 INFO:teuthology.orchestra.run.smithi139.stdout:Installed: 2024-09-16T20:46:03.512 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-16T20:46:03.512 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-16T20:46:03.513 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-16T20:46:03.513 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-16T20:46:03.513 INFO:teuthology.orchestra.run.smithi139.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-16T20:46:03.513 INFO:teuthology.orchestra.run.smithi139.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-16T20:46:03.513 INFO:teuthology.orchestra.run.smithi139.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-16T20:46:03.513 INFO:teuthology.orchestra.run.smithi139.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-16T20:46:03.513 INFO:teuthology.orchestra.run.smithi139.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-16T20:46:03.513 INFO:teuthology.orchestra.run.smithi139.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-16T20:46:03.513 INFO:teuthology.orchestra.run.smithi139.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-16T20:46:03.513 INFO:teuthology.orchestra.run.smithi139.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-16T20:46:03.513 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:46:03.513 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-16T20:46:03.966 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum -y install ceph-mgr-rook 2024-09-16T20:46:04.561 INFO:teuthology.orchestra.run.smithi139.stdout:Last metadata expiration check: 0:01:23 ago on Mon 16 Sep 2024 08:44:41 PM UTC. 2024-09-16T20:46:04.695 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-16T20:46:04.696 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-16T20:46:04.696 INFO:teuthology.orchestra.run.smithi139.stdout: Package Arch Version Repository Size 2024-09-16T20:46:04.696 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-16T20:46:04.696 INFO:teuthology.orchestra.run.smithi139.stdout:Installing: 2024-09-16T20:46:04.696 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-rook noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 49 k 2024-09-16T20:46:04.696 INFO:teuthology.orchestra.run.smithi139.stdout:Installing dependencies: 2024-09-16T20:46:04.696 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-16T20:46:04.696 INFO:teuthology.orchestra.run.smithi139.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-16T20:46:04.696 INFO:teuthology.orchestra.run.smithi139.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-16T20:46:04.696 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-16T20:46:04.697 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-16T20:46:04.697 INFO:teuthology.orchestra.run.smithi139.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-16T20:46:04.697 INFO:teuthology.orchestra.run.smithi139.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-16T20:46:04.697 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-16T20:46:04.697 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-16T20:46:04.697 INFO:teuthology.orchestra.run.smithi139.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-16T20:46:04.697 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-16T20:46:04.697 INFO:teuthology.orchestra.run.smithi139.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-16T20:46:04.697 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:46:04.697 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction Summary 2024-09-16T20:46:04.697 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-16T20:46:04.697 INFO:teuthology.orchestra.run.smithi139.stdout:Install 13 Packages 2024-09-16T20:46:04.697 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:46:04.698 INFO:teuthology.orchestra.run.smithi139.stdout:Total download size: 2.2 M 2024-09-16T20:46:04.698 INFO:teuthology.orchestra.run.smithi139.stdout:Installed size: 26 M 2024-09-16T20:46:04.698 INFO:teuthology.orchestra.run.smithi139.stdout:Downloading Packages: 2024-09-16T20:46:05.298 INFO:teuthology.orchestra.run.smithi139.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 290 kB/s | 26 kB 00:00 2024-09-16T20:46:05.323 INFO:teuthology.orchestra.run.smithi139.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 169 kB/s | 19 kB 00:00 2024-09-16T20:46:05.381 INFO:teuthology.orchestra.run.smithi139.stdout:(3/13): ceph-mgr-rook-19.3.0-4883.g6e8474a0.el9 286 kB/s | 49 kB 00:00 2024-09-16T20:46:05.432 INFO:teuthology.orchestra.run.smithi139.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.6 MB/s | 222 kB 00:00 2024-09-16T20:46:05.457 INFO:teuthology.orchestra.run.smithi139.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-09-16T20:46:05.490 INFO:teuthology.orchestra.run.smithi139.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 916 kB/s | 54 kB 00:00 2024-09-16T20:46:05.524 INFO:teuthology.orchestra.run.smithi139.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 480 kB/s | 32 kB 00:00 2024-09-16T20:46:05.566 INFO:teuthology.orchestra.run.smithi139.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 192 kB/s | 14 kB 00:00 2024-09-16T20:46:05.599 INFO:teuthology.orchestra.run.smithi139.stdout:(9/13): python3-google-auth-2.34.0-1.el9.noarch 2.9 MB/s | 220 kB 00:00 2024-09-16T20:46:05.625 INFO:teuthology.orchestra.run.smithi139.stdout:(10/13): python3-pyasn1-modules-0.4.8-6.el9.noa 1.1 MB/s | 279 kB 00:00 2024-09-16T20:46:05.650 INFO:teuthology.orchestra.run.smithi139.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-09-16T20:46:05.700 INFO:teuthology.orchestra.run.smithi139.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.2 MB/s | 90 kB 00:00 2024-09-16T20:46:05.742 INFO:teuthology.orchestra.run.smithi139.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 5.8 MB/s | 1.0 MB 00:00 2024-09-16T20:46:05.745 INFO:teuthology.orchestra.run.smithi139.stdout:-------------------------------------------------------------------------------- 2024-09-16T20:46:05.745 INFO:teuthology.orchestra.run.smithi139.stdout:Total 2.1 MB/s | 2.2 MB 00:01 2024-09-16T20:46:05.813 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction check 2024-09-16T20:46:05.827 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4883.g6e8474a0.el9.noa 12/12 2024-09-16T20:46:05.827 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4883.g6e8474a0.el 1/12 2024-09-16T20:46:05.827 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4883.g6e8474a0.el9.noa 2/12 2024-09-16T20:46:05.827 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4883.g6e8474a0.el9 3/12 2024-09-16T20:46:05.827 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-16T20:46:05.827 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-16T20:46:05.828 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-16T20:46:05.828 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-16T20:46:05.828 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-16T20:46:05.828 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-16T20:46:05.828 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-16T20:46:05.828 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-16T20:46:05.828 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction check succeeded. 2024-09-16T20:46:05.829 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction test 2024-09-16T20:46:05.845 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4883.g6e8474a0.el 11/12 2024-09-16T20:46:05.902 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4883.g6e8474a0.el9.noa 12/12 2024-09-16T20:46:05.916 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction test succeeded. 2024-09-16T20:46:05.918 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction 2024-09-16T20:46:06.166 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-16T20:46:06.166 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:46:06.166 INFO:teuthology.orchestra.run.smithi029.stdout:Installed: 2024-09-16T20:46:06.166 INFO:teuthology.orchestra.run.smithi029.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-16T20:46:06.166 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-grafana-dashboards-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-16T20:46:06.167 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-dashboard-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-16T20:46:06.167 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-prometheus-alerts-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-16T20:46:06.167 INFO:teuthology.orchestra.run.smithi029.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-16T20:46:06.167 INFO:teuthology.orchestra.run.smithi029.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-16T20:46:06.167 INFO:teuthology.orchestra.run.smithi029.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-16T20:46:06.167 INFO:teuthology.orchestra.run.smithi029.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-16T20:46:06.167 INFO:teuthology.orchestra.run.smithi029.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-16T20:46:06.167 INFO:teuthology.orchestra.run.smithi029.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-16T20:46:06.167 INFO:teuthology.orchestra.run.smithi029.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-16T20:46:06.167 INFO:teuthology.orchestra.run.smithi029.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-16T20:46:06.167 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:46:06.167 INFO:teuthology.orchestra.run.smithi029.stdout:Complete! 2024-09-16T20:46:06.417 INFO:teuthology.orchestra.run.smithi139.stdout: Preparing : 1/1 2024-09-16T20:46:06.493 DEBUG:teuthology.orchestra.run.smithi029:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-16T20:46:06.570 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-16T20:46:06.629 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-16T20:46:06.773 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-16T20:46:06.810 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-16T20:46:06.852 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-16T20:46:06.954 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-16T20:46:07.054 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-16T20:46:07.101 INFO:teuthology.orchestra.run.smithi029.stdout:Last metadata expiration check: 0:01:21 ago on Mon 16 Sep 2024 08:44:46 PM UTC. 2024-09-16T20:46:07.103 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-16T20:46:07.242 INFO:teuthology.orchestra.run.smithi029.stdout:Dependencies resolved. 2024-09-16T20:46:07.243 INFO:teuthology.orchestra.run.smithi029.stdout:======================================================================================= 2024-09-16T20:46:07.244 INFO:teuthology.orchestra.run.smithi029.stdout: Package Arch Version Repository Size 2024-09-16T20:46:07.244 INFO:teuthology.orchestra.run.smithi029.stdout:======================================================================================= 2024-09-16T20:46:07.244 INFO:teuthology.orchestra.run.smithi029.stdout:Installing: 2024-09-16T20:46:07.244 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 7.4 M 2024-09-16T20:46:07.244 INFO:teuthology.orchestra.run.smithi029.stdout:Installing dependencies: 2024-09-16T20:46:07.244 INFO:teuthology.orchestra.run.smithi029.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-16T20:46:07.244 INFO:teuthology.orchestra.run.smithi029.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-16T20:46:07.244 INFO:teuthology.orchestra.run.smithi029.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-16T20:46:07.244 INFO:teuthology.orchestra.run.smithi029.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-16T20:46:07.244 INFO:teuthology.orchestra.run.smithi029.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-16T20:46:07.244 INFO:teuthology.orchestra.run.smithi029.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-16T20:46:07.244 INFO:teuthology.orchestra.run.smithi029.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-16T20:46:07.245 INFO:teuthology.orchestra.run.smithi029.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-16T20:46:07.245 INFO:teuthology.orchestra.run.smithi029.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-16T20:46:07.245 INFO:teuthology.orchestra.run.smithi029.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-16T20:46:07.245 INFO:teuthology.orchestra.run.smithi029.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-16T20:46:07.245 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:46:07.245 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction Summary 2024-09-16T20:46:07.245 INFO:teuthology.orchestra.run.smithi029.stdout:======================================================================================= 2024-09-16T20:46:07.245 INFO:teuthology.orchestra.run.smithi029.stdout:Install 12 Packages 2024-09-16T20:46:07.245 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:46:07.246 INFO:teuthology.orchestra.run.smithi029.stdout:Total download size: 38 M 2024-09-16T20:46:07.246 INFO:teuthology.orchestra.run.smithi029.stdout:Installed size: 212 M 2024-09-16T20:46:07.246 INFO:teuthology.orchestra.run.smithi029.stdout:Downloading Packages: 2024-09-16T20:46:07.630 INFO:teuthology.orchestra.run.smithi029.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.3 MB/s | 191 kB 00:00 2024-09-16T20:46:07.722 INFO:teuthology.orchestra.run.smithi029.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 3.3 MB/s | 801 kB 00:00 2024-09-16T20:46:07.763 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4883.g6e8474a0.el9.noa 12/12 2024-09-16T20:46:07.763 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4883.g6e8474a0.el 1/12 2024-09-16T20:46:07.763 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4883.g6e8474a0.el9.noa 2/12 2024-09-16T20:46:07.763 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4883.g6e8474a0.el9 3/12 2024-09-16T20:46:07.763 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-16T20:46:07.763 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-16T20:46:07.763 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-16T20:46:07.763 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-16T20:46:07.763 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-16T20:46:07.763 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-16T20:46:07.763 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-16T20:46:07.763 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-16T20:46:07.817 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-16T20:46:07.859 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-16T20:46:07.893 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-16T20:46:07.941 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-16T20:46:07.962 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-rook-2:19.3.0-4883.g6e8474a0.el9.noarch 13/13 2024-09-16T20:46:08.040 INFO:teuthology.orchestra.run.smithi029.stdout:(3/12): ceph-mgr-diskprediction-local-19.3.0-48 13 MB/s | 7.4 MB 00:00 2024-09-16T20:46:08.085 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-16T20:46:08.085 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:46:08.085 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-16T20:46:08.085 INFO:teuthology.orchestra.run.smithi043.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-16T20:46:08.085 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-grafana-dashboards-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-16T20:46:08.085 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-dashboard-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-16T20:46:08.085 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-prometheus-alerts-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-16T20:46:08.085 INFO:teuthology.orchestra.run.smithi043.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-16T20:46:08.085 INFO:teuthology.orchestra.run.smithi043.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-16T20:46:08.085 INFO:teuthology.orchestra.run.smithi043.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-16T20:46:08.086 INFO:teuthology.orchestra.run.smithi043.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-16T20:46:08.086 INFO:teuthology.orchestra.run.smithi043.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-16T20:46:08.086 INFO:teuthology.orchestra.run.smithi043.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-16T20:46:08.086 INFO:teuthology.orchestra.run.smithi043.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-16T20:46:08.086 INFO:teuthology.orchestra.run.smithi043.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-16T20:46:08.086 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:46:08.086 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-16T20:46:08.331 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-16T20:46:08.332 INFO:teuthology.orchestra.run.smithi029.stdout:(4/12): flexiblas-3.0.4-8.el9.x86_64.rpm 47 kB/s | 33 kB 00:00 2024-09-16T20:46:08.532 INFO:teuthology.orchestra.run.smithi029.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 38 kB/s | 18 kB 00:00 2024-09-16T20:46:08.663 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4883.g6e8474a0.el9.noarch 13/13 2024-09-16T20:46:08.663 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4883.g6e8474a0.el9.noarch 1/13 2024-09-16T20:46:08.663 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-16T20:46:08.663 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-16T20:46:08.663 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-16T20:46:08.663 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-16T20:46:08.664 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-16T20:46:08.664 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-16T20:46:08.664 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-16T20:46:08.664 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-16T20:46:08.664 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-16T20:46:08.664 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-16T20:46:08.664 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-16T20:46:08.908 INFO:teuthology.orchestra.run.smithi029.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 67 kB/s | 38 kB 00:00 2024-09-16T20:46:08.941 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:10 ago on Mon 16 Sep 2024 08:44:58 PM UTC. 2024-09-16T20:46:09.009 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-16T20:46:09.009 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:46:09.009 INFO:teuthology.orchestra.run.smithi139.stdout:Installed: 2024-09-16T20:46:09.009 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-rook-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-16T20:46:09.009 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-16T20:46:09.009 INFO:teuthology.orchestra.run.smithi139.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-16T20:46:09.009 INFO:teuthology.orchestra.run.smithi139.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-16T20:46:09.010 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-16T20:46:09.010 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-16T20:46:09.010 INFO:teuthology.orchestra.run.smithi139.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-16T20:46:09.010 INFO:teuthology.orchestra.run.smithi139.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-16T20:46:09.010 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-16T20:46:09.010 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-16T20:46:09.010 INFO:teuthology.orchestra.run.smithi139.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-16T20:46:09.010 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-16T20:46:09.010 INFO:teuthology.orchestra.run.smithi139.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-16T20:46:09.010 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:46:09.010 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-16T20:46:09.085 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-16T20:46:09.086 INFO:teuthology.orchestra.run.smithi043.stdout:======================================================================================= 2024-09-16T20:46:09.086 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-09-16T20:46:09.086 INFO:teuthology.orchestra.run.smithi043.stdout:======================================================================================= 2024-09-16T20:46:09.086 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-16T20:46:09.086 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 7.4 M 2024-09-16T20:46:09.086 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-09-16T20:46:09.086 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-16T20:46:09.086 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-16T20:46:09.086 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-16T20:46:09.086 INFO:teuthology.orchestra.run.smithi043.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-16T20:46:09.086 INFO:teuthology.orchestra.run.smithi043.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-16T20:46:09.087 INFO:teuthology.orchestra.run.smithi043.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-16T20:46:09.087 INFO:teuthology.orchestra.run.smithi043.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-16T20:46:09.087 INFO:teuthology.orchestra.run.smithi043.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-16T20:46:09.087 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-16T20:46:09.087 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-16T20:46:09.087 INFO:teuthology.orchestra.run.smithi043.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-16T20:46:09.087 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:46:09.087 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-16T20:46:09.087 INFO:teuthology.orchestra.run.smithi043.stdout:======================================================================================= 2024-09-16T20:46:09.087 INFO:teuthology.orchestra.run.smithi043.stdout:Install 12 Packages 2024-09-16T20:46:09.087 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:46:09.088 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 38 M 2024-09-16T20:46:09.088 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 212 M 2024-09-16T20:46:09.088 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-16T20:46:09.275 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum -y install ceph-mgr-cephadm 2024-09-16T20:46:09.734 INFO:teuthology.orchestra.run.smithi029.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 297 kB/s | 245 kB 00:00 2024-09-16T20:46:09.835 INFO:teuthology.orchestra.run.smithi043.stdout:(1/12): ceph-mgr-diskprediction-local-19.3.0-48 16 MB/s | 7.4 MB 00:00 2024-09-16T20:46:09.856 INFO:teuthology.orchestra.run.smithi139.stdout:Last metadata expiration check: 0:01:28 ago on Mon 16 Sep 2024 08:44:41 PM UTC. 2024-09-16T20:46:09.935 INFO:teuthology.orchestra.run.smithi043.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 328 kB/s | 33 kB 00:00 2024-09-16T20:46:09.990 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-16T20:46:09.991 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-16T20:46:09.991 INFO:teuthology.orchestra.run.smithi139.stdout: Package Arch Version Repository Size 2024-09-16T20:46:09.991 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-16T20:46:09.991 INFO:teuthology.orchestra.run.smithi139.stdout:Installing: 2024-09-16T20:46:09.991 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 154 k 2024-09-16T20:46:09.991 INFO:teuthology.orchestra.run.smithi139.stdout:Installing dependencies: 2024-09-16T20:46:09.991 INFO:teuthology.orchestra.run.smithi139.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-16T20:46:09.991 INFO:teuthology.orchestra.run.smithi139.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-16T20:46:09.991 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-16T20:46:09.991 INFO:teuthology.orchestra.run.smithi139.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-16T20:46:09.991 INFO:teuthology.orchestra.run.smithi139.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-16T20:46:09.991 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:46:09.992 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction Summary 2024-09-16T20:46:09.992 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-16T20:46:09.992 INFO:teuthology.orchestra.run.smithi139.stdout:Install 6 Packages 2024-09-16T20:46:09.992 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:46:09.992 INFO:teuthology.orchestra.run.smithi139.stdout:Total download size: 7.0 M 2024-09-16T20:46:09.992 INFO:teuthology.orchestra.run.smithi139.stdout:Installed size: 33 M 2024-09-16T20:46:09.993 INFO:teuthology.orchestra.run.smithi139.stdout:Downloading Packages: 2024-09-16T20:46:10.152 INFO:teuthology.orchestra.run.smithi043.stdout:(3/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 14 MB/s | 3.0 MB 00:00 2024-09-16T20:46:10.194 INFO:teuthology.orchestra.run.smithi043.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 438 kB/s | 18 kB 00:00 2024-09-16T20:46:10.236 INFO:teuthology.orchestra.run.smithi043.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 928 kB/s | 38 kB 00:00 2024-09-16T20:46:10.269 INFO:teuthology.orchestra.run.smithi043.stdout:(6/12): libquadmath-11.5.0-2.el9.x86_64.rpm 213 kB/s | 191 kB 00:00 2024-09-16T20:46:10.369 INFO:teuthology.orchestra.run.smithi043.stdout:(7/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 37 MB/s | 4.9 MB 00:00 2024-09-16T20:46:10.487 INFO:teuthology.orchestra.run.smithi043.stdout:(8/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 44 MB/s | 5.1 MB 00:00 2024-09-16T20:46:10.511 INFO:teuthology.orchestra.run.smithi043.stdout:(9/12): python3-devel-3.9.19-8.el9.x86_64.rpm 1.0 MB/s | 245 kB 00:00 2024-09-16T20:46:10.537 INFO:teuthology.orchestra.run.smithi043.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 4.9 MB/s | 250 kB 00:00 2024-09-16T20:46:10.626 INFO:teuthology.orchestra.run.smithi139.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.4 MB/s | 248 kB 00:00 2024-09-16T20:46:10.652 INFO:teuthology.orchestra.run.smithi139.stdout:(2/6): ceph-mgr-cephadm-19.3.0-4883.g6e8474a0.e 772 kB/s | 154 kB 00:00 2024-09-16T20:46:10.702 INFO:teuthology.orchestra.run.smithi139.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 1.1 MB/s | 58 kB 00:00 2024-09-16T20:46:10.735 INFO:teuthology.orchestra.run.smithi139.stdout:(4/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 4.9 MB/s | 548 kB 00:00 2024-09-16T20:46:10.760 INFO:teuthology.orchestra.run.smithi139.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.2 MB/s | 74 kB 00:00 2024-09-16T20:46:10.861 INFO:teuthology.orchestra.run.smithi139.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-09-16T20:46:10.863 INFO:teuthology.orchestra.run.smithi139.stdout:-------------------------------------------------------------------------------- 2024-09-16T20:46:10.864 INFO:teuthology.orchestra.run.smithi139.stdout:Total 8.1 MB/s | 7.0 MB 00:00 2024-09-16T20:46:10.879 INFO:teuthology.orchestra.run.smithi043.stdout:(11/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 44 MB/s | 16 MB 00:00 2024-09-16T20:46:10.933 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction check 2024-09-16T20:46:10.944 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction check succeeded. 2024-09-16T20:46:10.944 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction test 2024-09-16T20:46:10.996 INFO:teuthology.orchestra.run.smithi043.stdout:(12/12): libgfortran-11.5.0-2.el9.x86_64.rpm 493 kB/s | 801 kB 00:01 2024-09-16T20:46:10.997 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-16T20:46:10.997 INFO:teuthology.orchestra.run.smithi043.stdout:Total 20 MB/s | 38 MB 00:01 2024-09-16T20:46:11.044 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction test succeeded. 2024-09-16T20:46:11.044 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction 2024-09-16T20:46:11.353 INFO:teuthology.orchestra.run.smithi139.stdout: Preparing : 1/1 2024-09-16T20:46:11.361 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-16T20:46:11.389 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-16T20:46:11.389 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-16T20:46:11.517 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-16T20:46:11.579 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-16T20:46:11.845 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-16T20:46:11.846 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-16T20:46:12.104 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-16T20:46:12.201 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-16T20:46:12.278 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-16T20:46:12.308 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 6/6 2024-09-16T20:46:12.409 INFO:teuthology.orchestra.run.smithi029.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 653 kB/s | 3.0 MB 00:04 2024-09-16T20:46:12.632 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-16T20:46:12.711 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-16T20:46:12.752 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-16T20:46:12.776 INFO:teuthology.orchestra.run.smithi029.stdout:(9/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 1.2 MB/s | 4.9 MB 00:04 2024-09-16T20:46:12.858 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-16T20:46:12.895 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-16T20:46:13.035 INFO:teuthology.orchestra.run.smithi029.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 1.5 MB/s | 5.1 MB 00:03 2024-09-16T20:46:13.087 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-16T20:46:13.119 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-16T20:46:13.137 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 6/6 2024-09-16T20:46:13.137 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 1/6 2024-09-16T20:46:13.137 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-16T20:46:13.137 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-16T20:46:13.138 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-16T20:46:13.138 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-16T20:46:13.198 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-16T20:46:13.406 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-16T20:46:13.407 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:46:13.407 INFO:teuthology.orchestra.run.smithi139.stdout:Installed: 2024-09-16T20:46:13.407 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-16T20:46:13.407 INFO:teuthology.orchestra.run.smithi139.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-16T20:46:13.407 INFO:teuthology.orchestra.run.smithi139.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-16T20:46:13.407 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-16T20:46:13.407 INFO:teuthology.orchestra.run.smithi139.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-16T20:46:13.407 INFO:teuthology.orchestra.run.smithi139.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-16T20:46:13.407 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:46:13.407 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-16T20:46:13.485 INFO:teuthology.orchestra.run.smithi029.stdout:(11/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 233 kB/s | 250 kB 00:01 2024-09-16T20:46:13.656 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum -y install ceph-fuse 2024-09-16T20:46:13.906 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-16T20:46:14.028 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-16T20:46:14.241 INFO:teuthology.orchestra.run.smithi139.stdout:Last metadata expiration check: 0:01:33 ago on Mon 16 Sep 2024 08:44:41 PM UTC. 2024-09-16T20:46:14.376 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-16T20:46:14.377 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-16T20:46:14.377 INFO:teuthology.orchestra.run.smithi139.stdout: Package Arch Version Repository Size 2024-09-16T20:46:14.377 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-16T20:46:14.377 INFO:teuthology.orchestra.run.smithi139.stdout:Installing: 2024-09-16T20:46:14.377 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-fuse x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 899 k 2024-09-16T20:46:14.377 INFO:teuthology.orchestra.run.smithi139.stdout:Installing dependencies: 2024-09-16T20:46:14.377 INFO:teuthology.orchestra.run.smithi139.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-16T20:46:14.378 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:46:14.378 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction Summary 2024-09-16T20:46:14.378 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-16T20:46:14.378 INFO:teuthology.orchestra.run.smithi139.stdout:Install 2 Packages 2024-09-16T20:46:14.378 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:46:14.378 INFO:teuthology.orchestra.run.smithi139.stdout:Total download size: 979 k 2024-09-16T20:46:14.378 INFO:teuthology.orchestra.run.smithi139.stdout:Installed size: 2.9 M 2024-09-16T20:46:14.379 INFO:teuthology.orchestra.run.smithi139.stdout:Downloading Packages: 2024-09-16T20:46:14.762 INFO:teuthology.orchestra.run.smithi139.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 766 kB/s | 80 kB 00:00 2024-09-16T20:46:15.187 INFO:teuthology.orchestra.run.smithi139.stdout:(2/2): ceph-fuse-19.3.0-4883.g6e8474a0.el9.x86_ 1.7 MB/s | 899 kB 00:00 2024-09-16T20:46:15.188 INFO:teuthology.orchestra.run.smithi139.stdout:-------------------------------------------------------------------------------- 2024-09-16T20:46:15.188 INFO:teuthology.orchestra.run.smithi139.stdout:Total 1.2 MB/s | 979 kB 00:00 2024-09-16T20:46:15.200 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction check 2024-09-16T20:46:15.222 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction check succeeded. 2024-09-16T20:46:15.222 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction test 2024-09-16T20:46:15.302 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction test succeeded. 2024-09-16T20:46:15.302 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction 2024-09-16T20:46:15.353 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-16T20:46:15.488 INFO:teuthology.orchestra.run.smithi139.stdout: Preparing : 1/1 2024-09-16T20:46:15.639 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-16T20:46:15.719 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-fuse-2:19.3.0-4883.g6e8474a0.el9.x86_64 2/2 2024-09-16T20:46:16.366 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-fuse-2:19.3.0-4883.g6e8474a0.el9.x86_64 2/2 2024-09-16T20:46:16.366 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-fuse-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/2 2024-09-16T20:46:16.432 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-16T20:46:16.459 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4883.g6e847 12/12 2024-09-16T20:46:16.649 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-16T20:46:16.649 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:46:16.649 INFO:teuthology.orchestra.run.smithi139.stdout:Installed: 2024-09-16T20:46:16.649 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-fuse-2:19.3.0-4883.g6e8474a0.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-16T20:46:16.649 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:46:16.650 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-16T20:46:16.935 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum -y install ceph-volume 2024-09-16T20:46:17.502 INFO:teuthology.orchestra.run.smithi029.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 3.4 MB/s | 16 MB 00:04 2024-09-16T20:46:17.504 INFO:teuthology.orchestra.run.smithi029.stdout:-------------------------------------------------------------------------------- 2024-09-16T20:46:17.504 INFO:teuthology.orchestra.run.smithi029.stdout:Total 3.7 MB/s | 38 MB 00:10 2024-09-16T20:46:17.525 INFO:teuthology.orchestra.run.smithi139.stdout:Last metadata expiration check: 0:01:36 ago on Mon 16 Sep 2024 08:44:41 PM UTC. 2024-09-16T20:46:17.661 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-16T20:46:17.661 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-16T20:46:17.664 INFO:teuthology.orchestra.run.smithi139.stdout: Package Arch Version Repository Size 2024-09-16T20:46:17.664 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-16T20:46:17.664 INFO:teuthology.orchestra.run.smithi139.stdout:Installing: 2024-09-16T20:46:17.664 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-volume noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 285 k 2024-09-16T20:46:17.664 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:46:17.664 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction Summary 2024-09-16T20:46:17.664 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-16T20:46:17.664 INFO:teuthology.orchestra.run.smithi139.stdout:Install 1 Package 2024-09-16T20:46:17.664 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:46:17.664 INFO:teuthology.orchestra.run.smithi139.stdout:Total download size: 285 k 2024-09-16T20:46:17.665 INFO:teuthology.orchestra.run.smithi139.stdout:Installed size: 1.3 M 2024-09-16T20:46:17.665 INFO:teuthology.orchestra.run.smithi139.stdout:Downloading Packages: 2024-09-16T20:46:17.876 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction check 2024-09-16T20:46:17.904 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction check succeeded. 2024-09-16T20:46:17.904 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction test 2024-09-16T20:46:17.916 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-volume-19.3.0-4883.g6e8474a0.el9.noarch.rp 1.1 MB/s | 285 kB 00:00 2024-09-16T20:46:17.916 INFO:teuthology.orchestra.run.smithi139.stdout:-------------------------------------------------------------------------------- 2024-09-16T20:46:17.917 INFO:teuthology.orchestra.run.smithi139.stdout:Total 1.1 MB/s | 285 kB 00:00 2024-09-16T20:46:17.917 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction check 2024-09-16T20:46:17.925 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction check succeeded. 2024-09-16T20:46:17.925 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction test 2024-09-16T20:46:17.952 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction test succeeded. 2024-09-16T20:46:17.952 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction 2024-09-16T20:46:18.174 INFO:teuthology.orchestra.run.smithi139.stdout: Preparing : 1/1 2024-09-16T20:46:18.243 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-volume-2:19.3.0-4883.g6e8474a0.el9.noarch 1/1 2024-09-16T20:46:18.258 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-volume-2:19.3.0-4883.g6e8474a0.el9.noarch 1/1 2024-09-16T20:46:18.258 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T20:46:18.258 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-16T20:46:18.258 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:46:18.359 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction test succeeded. 2024-09-16T20:46:18.359 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction 2024-09-16T20:46:18.924 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4883.g6e847 12/12 2024-09-16T20:46:18.924 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4883.g6e847 1/12 2024-09-16T20:46:18.924 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-16T20:46:18.925 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-16T20:46:18.925 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-16T20:46:18.925 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-16T20:46:18.925 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-16T20:46:18.925 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-16T20:46:18.925 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-16T20:46:18.925 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-16T20:46:18.925 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-16T20:46:18.925 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-16T20:46:19.182 INFO:teuthology.orchestra.run.smithi029.stdout: Preparing : 1/1 2024-09-16T20:46:19.260 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-16T20:46:19.275 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-volume-2:19.3.0-4883.g6e8474a0.el9.noarch 1/1 2024-09-16T20:46:19.275 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:46:19.275 INFO:teuthology.orchestra.run.smithi139.stdout:Installed: 2024-09-16T20:46:19.275 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-volume-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-16T20:46:19.275 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:46:19.275 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-16T20:46:19.312 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-16T20:46:19.318 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-16T20:46:19.318 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:46:19.318 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-16T20:46:19.318 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-16T20:46:19.318 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-16T20:46:19.318 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-16T20:46:19.318 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-16T20:46:19.318 INFO:teuthology.orchestra.run.smithi043.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-16T20:46:19.319 INFO:teuthology.orchestra.run.smithi043.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-16T20:46:19.319 INFO:teuthology.orchestra.run.smithi043.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-16T20:46:19.319 INFO:teuthology.orchestra.run.smithi043.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-16T20:46:19.319 INFO:teuthology.orchestra.run.smithi043.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-16T20:46:19.319 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-16T20:46:19.319 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-16T20:46:19.319 INFO:teuthology.orchestra.run.smithi043.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-16T20:46:19.319 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:46:19.319 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-16T20:46:19.427 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-16T20:46:19.473 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-16T20:46:19.571 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum -y install librados-devel 2024-09-16T20:46:19.673 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-16T20:46:19.705 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-16T20:46:19.715 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-mgr-rook 2024-09-16T20:46:19.793 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-16T20:46:20.161 INFO:teuthology.orchestra.run.smithi139.stdout:Last metadata expiration check: 0:01:39 ago on Mon 16 Sep 2024 08:44:41 PM UTC. 2024-09-16T20:46:20.295 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-16T20:46:20.296 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-16T20:46:20.296 INFO:teuthology.orchestra.run.smithi139.stdout: Package Arch Version Repo Size 2024-09-16T20:46:20.296 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-16T20:46:20.297 INFO:teuthology.orchestra.run.smithi139.stdout:Installing: 2024-09-16T20:46:20.297 INFO:teuthology.orchestra.run.smithi139.stdout: librados-devel x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 127 k 2024-09-16T20:46:20.297 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:46:20.297 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction Summary 2024-09-16T20:46:20.297 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-16T20:46:20.297 INFO:teuthology.orchestra.run.smithi139.stdout:Install 1 Package 2024-09-16T20:46:20.297 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:46:20.297 INFO:teuthology.orchestra.run.smithi139.stdout:Total download size: 127 k 2024-09-16T20:46:20.297 INFO:teuthology.orchestra.run.smithi139.stdout:Installed size: 456 k 2024-09-16T20:46:20.297 INFO:teuthology.orchestra.run.smithi139.stdout:Downloading Packages: 2024-09-16T20:46:20.334 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:22 ago on Mon 16 Sep 2024 08:44:58 PM UTC. 2024-09-16T20:46:20.473 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-16T20:46:20.474 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T20:46:20.474 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-09-16T20:46:20.474 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T20:46:20.474 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-16T20:46:20.474 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-rook noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 49 k 2024-09-16T20:46:20.474 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-09-16T20:46:20.475 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-16T20:46:20.475 INFO:teuthology.orchestra.run.smithi043.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-16T20:46:20.475 INFO:teuthology.orchestra.run.smithi043.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-16T20:46:20.475 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-16T20:46:20.475 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-16T20:46:20.475 INFO:teuthology.orchestra.run.smithi043.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-16T20:46:20.475 INFO:teuthology.orchestra.run.smithi043.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-16T20:46:20.475 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-16T20:46:20.475 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-16T20:46:20.475 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-16T20:46:20.475 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-16T20:46:20.475 INFO:teuthology.orchestra.run.smithi043.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-16T20:46:20.476 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:46:20.476 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-16T20:46:20.476 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T20:46:20.476 INFO:teuthology.orchestra.run.smithi043.stdout:Install 13 Packages 2024-09-16T20:46:20.476 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:46:20.476 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 2.2 M 2024-09-16T20:46:20.476 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 26 M 2024-09-16T20:46:20.477 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-16T20:46:20.508 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-16T20:46:20.512 INFO:teuthology.orchestra.run.smithi139.stdout:librados-devel-19.3.0-4883.g6e8474a0.el9.x86_64 591 kB/s | 127 kB 00:00 2024-09-16T20:46:20.512 INFO:teuthology.orchestra.run.smithi139.stdout:-------------------------------------------------------------------------------- 2024-09-16T20:46:20.513 INFO:teuthology.orchestra.run.smithi139.stdout:Total 587 kB/s | 127 kB 00:00 2024-09-16T20:46:20.513 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction check 2024-09-16T20:46:20.518 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction check succeeded. 2024-09-16T20:46:20.518 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction test 2024-09-16T20:46:20.582 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction test succeeded. 2024-09-16T20:46:20.582 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction 2024-09-16T20:46:20.649 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-16T20:46:20.734 INFO:teuthology.orchestra.run.smithi043.stdout:(1/13): ceph-mgr-rook-19.3.0-4883.g6e8474a0.el9 289 kB/s | 49 kB 00:00 2024-09-16T20:46:20.737 INFO:teuthology.orchestra.run.smithi139.stdout: Preparing : 1/1 2024-09-16T20:46:20.759 INFO:teuthology.orchestra.run.smithi043.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 131 kB/s | 26 kB 00:00 2024-09-16T20:46:20.785 INFO:teuthology.orchestra.run.smithi043.stdout:(3/13): python3-jsonpointer-2.0-4.el9.noarch.rp 87 kB/s | 19 kB 00:00 2024-09-16T20:46:20.859 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : librados-devel-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T20:46:21.035 INFO:teuthology.orchestra.run.smithi043.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 739 kB/s | 222 kB 00:00 2024-09-16T20:46:21.077 INFO:teuthology.orchestra.run.smithi043.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 544 kB/s | 159 kB 00:00 2024-09-16T20:46:21.127 INFO:teuthology.orchestra.run.smithi043.stdout:(6/13): python3-cachetools-4.2.4-1.el9.noarch.r 645 kB/s | 32 kB 00:00 2024-09-16T20:46:21.152 INFO:teuthology.orchestra.run.smithi043.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 457 kB/s | 54 kB 00:00 2024-09-16T20:46:21.177 INFO:teuthology.orchestra.run.smithi043.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 280 kB/s | 14 kB 00:00 2024-09-16T20:46:21.211 INFO:teuthology.orchestra.run.smithi043.stdout:(9/13): python3-pyasn1-modules-0.4.8-6.el9.noar 654 kB/s | 279 kB 00:00 2024-09-16T20:46:21.237 INFO:teuthology.orchestra.run.smithi043.stdout:(10/13): python3-google-auth-2.34.0-1.el9.noarc 2.6 MB/s | 220 kB 00:00 2024-09-16T20:46:21.270 INFO:teuthology.orchestra.run.smithi043.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 3.4 MB/s | 90 kB 00:00 2024-09-16T20:46:21.303 INFO:teuthology.orchestra.run.smithi043.stdout:(12/13): python3-rsa-4.9-2.el9.noarch.rpm 655 kB/s | 59 kB 00:00 2024-09-16T20:46:21.306 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: librados-devel-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T20:46:21.337 INFO:teuthology.orchestra.run.smithi043.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 6.4 MB/s | 1.0 MB 00:00 2024-09-16T20:46:21.340 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-16T20:46:21.340 INFO:teuthology.orchestra.run.smithi043.stdout:Total 2.6 MB/s | 2.2 MB 00:00 2024-09-16T20:46:21.407 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-16T20:46:21.423 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-16T20:46:21.423 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-16T20:46:21.522 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librados-devel-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T20:46:21.522 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:46:21.522 INFO:teuthology.orchestra.run.smithi139.stdout:Installed: 2024-09-16T20:46:21.522 INFO:teuthology.orchestra.run.smithi139.stdout: librados-devel-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:46:21.522 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:46:21.523 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-16T20:46:21.523 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-16T20:46:21.523 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-16T20:46:21.749 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum -y install libcephfs2 2024-09-16T20:46:21.978 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-16T20:46:22.078 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-16T20:46:22.260 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-16T20:46:22.316 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-16T20:46:22.334 INFO:teuthology.orchestra.run.smithi139.stdout:Last metadata expiration check: 0:01:41 ago on Mon 16 Sep 2024 08:44:41 PM UTC. 2024-09-16T20:46:22.371 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-16T20:46:22.406 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-16T20:46:22.424 INFO:teuthology.orchestra.run.smithi139.stdout:Package libcephfs2-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-16T20:46:22.448 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-16T20:46:22.468 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-16T20:46:22.469 INFO:teuthology.orchestra.run.smithi139.stdout:Nothing to do. 2024-09-16T20:46:22.469 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-16T20:46:22.535 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum -y install libcephfs-devel 2024-09-16T20:46:22.555 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-16T20:46:22.666 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-16T20:46:22.718 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-16T20:46:23.099 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-16T20:46:23.117 INFO:teuthology.orchestra.run.smithi139.stdout:Last metadata expiration check: 0:01:42 ago on Mon 16 Sep 2024 08:44:41 PM UTC. 2024-09-16T20:46:23.138 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4883.g6e847 12/12 2024-09-16T20:46:23.252 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-16T20:46:23.252 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-16T20:46:23.252 INFO:teuthology.orchestra.run.smithi139.stdout: Package Arch Version Repo Size 2024-09-16T20:46:23.252 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-16T20:46:23.252 INFO:teuthology.orchestra.run.smithi139.stdout:Installing: 2024-09-16T20:46:23.252 INFO:teuthology.orchestra.run.smithi139.stdout: libcephfs-devel x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 32 k 2024-09-16T20:46:23.252 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:46:23.252 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction Summary 2024-09-16T20:46:23.252 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-16T20:46:23.252 INFO:teuthology.orchestra.run.smithi139.stdout:Install 1 Package 2024-09-16T20:46:23.252 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:46:23.253 INFO:teuthology.orchestra.run.smithi139.stdout:Total download size: 32 k 2024-09-16T20:46:23.253 INFO:teuthology.orchestra.run.smithi139.stdout:Installed size: 142 k 2024-09-16T20:46:23.253 INFO:teuthology.orchestra.run.smithi139.stdout:Downloading Packages: 2024-09-16T20:46:23.406 INFO:teuthology.orchestra.run.smithi139.stdout:libcephfs-devel-19.3.0-4883.g6e8474a0.el9.x86_6 204 kB/s | 32 kB 00:00 2024-09-16T20:46:23.406 INFO:teuthology.orchestra.run.smithi139.stdout:-------------------------------------------------------------------------------- 2024-09-16T20:46:23.407 INFO:teuthology.orchestra.run.smithi139.stdout:Total 202 kB/s | 32 kB 00:00 2024-09-16T20:46:23.407 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction check 2024-09-16T20:46:23.410 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction check succeeded. 2024-09-16T20:46:23.411 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction test 2024-09-16T20:46:23.432 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction test succeeded. 2024-09-16T20:46:23.433 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction 2024-09-16T20:46:23.480 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-16T20:46:23.527 INFO:teuthology.orchestra.run.smithi139.stdout: Preparing : 1/1 2024-09-16T20:46:23.538 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-16T20:46:23.580 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-16T20:46:23.629 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-16T20:46:23.646 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libcephfs-devel-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T20:46:23.657 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-rook-2:19.3.0-4883.g6e8474a0.el9.noarch 13/13 2024-09-16T20:46:24.041 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: libcephfs-devel-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T20:46:24.279 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libcephfs-devel-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T20:46:24.279 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:46:24.279 INFO:teuthology.orchestra.run.smithi139.stdout:Installed: 2024-09-16T20:46:24.279 INFO:teuthology.orchestra.run.smithi139.stdout: libcephfs-devel-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:46:24.279 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:46:24.279 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-16T20:46:24.435 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4883.g6e8474a0.el9.noarch 13/13 2024-09-16T20:46:24.435 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4883.g6e8474a0.el9.noarch 1/13 2024-09-16T20:46:24.435 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-16T20:46:24.435 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-16T20:46:24.435 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-16T20:46:24.435 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-16T20:46:24.435 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-16T20:46:24.435 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-16T20:46:24.435 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-16T20:46:24.435 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-16T20:46:24.435 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-16T20:46:24.435 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-16T20:46:24.436 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-16T20:46:24.478 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum -y install librados2 2024-09-16T20:46:24.766 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-16T20:46:24.766 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:46:24.766 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-16T20:46:24.766 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-rook-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-16T20:46:24.766 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-16T20:46:24.766 INFO:teuthology.orchestra.run.smithi043.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-16T20:46:24.766 INFO:teuthology.orchestra.run.smithi043.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-16T20:46:24.766 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-16T20:46:24.767 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-16T20:46:24.767 INFO:teuthology.orchestra.run.smithi043.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-16T20:46:24.767 INFO:teuthology.orchestra.run.smithi043.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-16T20:46:24.767 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-16T20:46:24.767 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-16T20:46:24.767 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-16T20:46:24.767 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-16T20:46:24.767 INFO:teuthology.orchestra.run.smithi043.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-16T20:46:24.767 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:46:24.767 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-16T20:46:25.013 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-mgr-cephadm 2024-09-16T20:46:25.060 INFO:teuthology.orchestra.run.smithi139.stdout:Last metadata expiration check: 0:01:44 ago on Mon 16 Sep 2024 08:44:41 PM UTC. 2024-09-16T20:46:25.149 INFO:teuthology.orchestra.run.smithi139.stdout:Package librados2-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-16T20:46:25.193 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-16T20:46:25.194 INFO:teuthology.orchestra.run.smithi139.stdout:Nothing to do. 2024-09-16T20:46:25.194 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-16T20:46:25.253 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum -y install librbd1 2024-09-16T20:46:25.619 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:27 ago on Mon 16 Sep 2024 08:44:58 PM UTC. 2024-09-16T20:46:25.756 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-16T20:46:25.756 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T20:46:25.756 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-09-16T20:46:25.756 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T20:46:25.757 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-16T20:46:25.757 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 154 k 2024-09-16T20:46:25.757 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-09-16T20:46:25.757 INFO:teuthology.orchestra.run.smithi043.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-16T20:46:25.757 INFO:teuthology.orchestra.run.smithi043.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-16T20:46:25.757 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-16T20:46:25.757 INFO:teuthology.orchestra.run.smithi043.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-16T20:46:25.757 INFO:teuthology.orchestra.run.smithi043.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-16T20:46:25.757 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:46:25.757 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-16T20:46:25.757 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T20:46:25.757 INFO:teuthology.orchestra.run.smithi043.stdout:Install 6 Packages 2024-09-16T20:46:25.758 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:46:25.758 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 7.0 M 2024-09-16T20:46:25.758 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 33 M 2024-09-16T20:46:25.758 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-16T20:46:25.837 INFO:teuthology.orchestra.run.smithi139.stdout:Last metadata expiration check: 0:01:44 ago on Mon 16 Sep 2024 08:44:41 PM UTC. 2024-09-16T20:46:25.928 INFO:teuthology.orchestra.run.smithi139.stdout:Package librbd1-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-16T20:46:25.971 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-16T20:46:25.972 INFO:teuthology.orchestra.run.smithi139.stdout:Nothing to do. 2024-09-16T20:46:25.972 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-16T20:46:25.980 INFO:teuthology.orchestra.run.smithi043.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.4 MB/s | 248 kB 00:00 2024-09-16T20:46:26.030 INFO:teuthology.orchestra.run.smithi043.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 11 MB/s | 548 kB 00:00 2024-09-16T20:46:26.036 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum -y install python3-rados 2024-09-16T20:46:26.056 INFO:teuthology.orchestra.run.smithi043.stdout:(3/6): ceph-mgr-cephadm-19.3.0-4883.g6e8474a0.e 632 kB/s | 154 kB 00:00 2024-09-16T20:46:26.081 INFO:teuthology.orchestra.run.smithi043.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 1.1 MB/s | 58 kB 00:00 2024-09-16T20:46:26.123 INFO:teuthology.orchestra.run.smithi043.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.1 MB/s | 74 kB 00:00 2024-09-16T20:46:26.198 INFO:teuthology.orchestra.run.smithi043.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-09-16T20:46:26.200 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-16T20:46:26.201 INFO:teuthology.orchestra.run.smithi043.stdout:Total 16 MB/s | 7.0 MB 00:00 2024-09-16T20:46:26.273 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-16T20:46:26.285 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-16T20:46:26.285 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-16T20:46:26.391 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-16T20:46:26.391 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-16T20:46:26.612 INFO:teuthology.orchestra.run.smithi139.stdout:Last metadata expiration check: 0:01:45 ago on Mon 16 Sep 2024 08:44:41 PM UTC. 2024-09-16T20:46:26.701 INFO:teuthology.orchestra.run.smithi139.stdout:Package python3-rados-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-16T20:46:26.708 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-16T20:46:26.744 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-16T20:46:26.745 INFO:teuthology.orchestra.run.smithi139.stdout:Nothing to do. 2024-09-16T20:46:26.745 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-16T20:46:26.808 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum -y install python3-rgw 2024-09-16T20:46:26.887 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-16T20:46:26.951 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-16T20:46:27.386 INFO:teuthology.orchestra.run.smithi139.stdout:Last metadata expiration check: 0:01:46 ago on Mon 16 Sep 2024 08:44:41 PM UTC. 2024-09-16T20:46:27.476 INFO:teuthology.orchestra.run.smithi139.stdout:Package python3-rgw-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-16T20:46:27.478 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4883.g6e847 12/12 2024-09-16T20:46:27.478 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4883.g6e847 1/12 2024-09-16T20:46:27.478 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-16T20:46:27.478 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-16T20:46:27.479 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-16T20:46:27.479 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-16T20:46:27.479 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-16T20:46:27.479 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-16T20:46:27.479 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-16T20:46:27.479 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-16T20:46:27.479 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-16T20:46:27.479 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-16T20:46:27.502 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-16T20:46:27.518 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-16T20:46:27.519 INFO:teuthology.orchestra.run.smithi139.stdout:Nothing to do. 2024-09-16T20:46:27.519 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-16T20:46:27.581 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-16T20:46:27.585 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum -y install python3-cephfs 2024-09-16T20:46:27.665 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-16T20:46:27.695 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 6/6 2024-09-16T20:46:27.882 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-16T20:46:27.882 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:46:27.882 INFO:teuthology.orchestra.run.smithi029.stdout:Installed: 2024-09-16T20:46:27.882 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-16T20:46:27.882 INFO:teuthology.orchestra.run.smithi029.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-16T20:46:27.882 INFO:teuthology.orchestra.run.smithi029.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-16T20:46:27.882 INFO:teuthology.orchestra.run.smithi029.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-16T20:46:27.882 INFO:teuthology.orchestra.run.smithi029.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-16T20:46:27.882 INFO:teuthology.orchestra.run.smithi029.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-16T20:46:27.883 INFO:teuthology.orchestra.run.smithi029.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-16T20:46:27.883 INFO:teuthology.orchestra.run.smithi029.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-16T20:46:27.883 INFO:teuthology.orchestra.run.smithi029.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-16T20:46:27.883 INFO:teuthology.orchestra.run.smithi029.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-16T20:46:27.883 INFO:teuthology.orchestra.run.smithi029.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-16T20:46:27.883 INFO:teuthology.orchestra.run.smithi029.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-16T20:46:27.883 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:46:27.883 INFO:teuthology.orchestra.run.smithi029.stdout:Complete! 2024-09-16T20:46:28.162 INFO:teuthology.orchestra.run.smithi139.stdout:Last metadata expiration check: 0:01:47 ago on Mon 16 Sep 2024 08:44:41 PM UTC. 2024-09-16T20:46:28.251 INFO:teuthology.orchestra.run.smithi139.stdout:Package python3-cephfs-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-16T20:46:28.293 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-16T20:46:28.294 INFO:teuthology.orchestra.run.smithi139.stdout:Nothing to do. 2024-09-16T20:46:28.294 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-16T20:46:28.353 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum -y install python3-rbd 2024-09-16T20:46:28.354 DEBUG:teuthology.orchestra.run.smithi029:> sudo yum -y install ceph-mgr-rook 2024-09-16T20:46:28.498 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 6/6 2024-09-16T20:46:28.498 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 1/6 2024-09-16T20:46:28.499 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-16T20:46:28.499 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-16T20:46:28.499 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-16T20:46:28.499 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-16T20:46:28.773 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-16T20:46:28.773 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:46:28.773 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-16T20:46:28.773 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-16T20:46:28.773 INFO:teuthology.orchestra.run.smithi043.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-16T20:46:28.773 INFO:teuthology.orchestra.run.smithi043.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-16T20:46:28.773 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-16T20:46:28.773 INFO:teuthology.orchestra.run.smithi043.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-16T20:46:28.773 INFO:teuthology.orchestra.run.smithi043.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-16T20:46:28.773 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:46:28.773 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-16T20:46:28.935 INFO:teuthology.orchestra.run.smithi139.stdout:Last metadata expiration check: 0:01:47 ago on Mon 16 Sep 2024 08:44:41 PM UTC. 2024-09-16T20:46:28.980 INFO:teuthology.orchestra.run.smithi029.stdout:Last metadata expiration check: 0:01:42 ago on Mon 16 Sep 2024 08:44:46 PM UTC. 2024-09-16T20:46:29.024 INFO:teuthology.orchestra.run.smithi139.stdout:Package python3-rbd-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-16T20:46:29.033 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-fuse 2024-09-16T20:46:29.067 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-16T20:46:29.068 INFO:teuthology.orchestra.run.smithi139.stdout:Nothing to do. 2024-09-16T20:46:29.068 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-16T20:46:29.124 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum -y install rbd-fuse 2024-09-16T20:46:29.125 INFO:teuthology.orchestra.run.smithi029.stdout:Dependencies resolved. 2024-09-16T20:46:29.125 INFO:teuthology.orchestra.run.smithi029.stdout:================================================================================ 2024-09-16T20:46:29.126 INFO:teuthology.orchestra.run.smithi029.stdout: Package Arch Version Repository Size 2024-09-16T20:46:29.126 INFO:teuthology.orchestra.run.smithi029.stdout:================================================================================ 2024-09-16T20:46:29.126 INFO:teuthology.orchestra.run.smithi029.stdout:Installing: 2024-09-16T20:46:29.126 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-rook noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 49 k 2024-09-16T20:46:29.126 INFO:teuthology.orchestra.run.smithi029.stdout:Installing dependencies: 2024-09-16T20:46:29.126 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-16T20:46:29.126 INFO:teuthology.orchestra.run.smithi029.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-16T20:46:29.126 INFO:teuthology.orchestra.run.smithi029.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-16T20:46:29.126 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-16T20:46:29.126 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-16T20:46:29.126 INFO:teuthology.orchestra.run.smithi029.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-16T20:46:29.127 INFO:teuthology.orchestra.run.smithi029.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-16T20:46:29.127 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-16T20:46:29.127 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-16T20:46:29.127 INFO:teuthology.orchestra.run.smithi029.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-16T20:46:29.127 INFO:teuthology.orchestra.run.smithi029.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-16T20:46:29.127 INFO:teuthology.orchestra.run.smithi029.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-16T20:46:29.127 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:46:29.127 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction Summary 2024-09-16T20:46:29.127 INFO:teuthology.orchestra.run.smithi029.stdout:================================================================================ 2024-09-16T20:46:29.127 INFO:teuthology.orchestra.run.smithi029.stdout:Install 13 Packages 2024-09-16T20:46:29.127 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:46:29.128 INFO:teuthology.orchestra.run.smithi029.stdout:Total download size: 2.2 M 2024-09-16T20:46:29.128 INFO:teuthology.orchestra.run.smithi029.stdout:Installed size: 26 M 2024-09-16T20:46:29.128 INFO:teuthology.orchestra.run.smithi029.stdout:Downloading Packages: 2024-09-16T20:46:29.402 INFO:teuthology.orchestra.run.smithi029.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 164 kB/s | 19 kB 00:00 2024-09-16T20:46:29.428 INFO:teuthology.orchestra.run.smithi029.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 180 kB/s | 26 kB 00:00 2024-09-16T20:46:29.453 INFO:teuthology.orchestra.run.smithi029.stdout:(3/13): ceph-mgr-rook-19.3.0-4883.g6e8474a0.el9 293 kB/s | 49 kB 00:00 2024-09-16T20:46:29.545 INFO:teuthology.orchestra.run.smithi029.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.5 MB/s | 222 kB 00:00 2024-09-16T20:46:29.571 INFO:teuthology.orchestra.run.smithi029.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.1 MB/s | 159 kB 00:00 2024-09-16T20:46:29.605 INFO:teuthology.orchestra.run.smithi029.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 907 kB/s | 54 kB 00:00 2024-09-16T20:46:29.638 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:31 ago on Mon 16 Sep 2024 08:44:58 PM UTC. 2024-09-16T20:46:29.639 INFO:teuthology.orchestra.run.smithi029.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 477 kB/s | 32 kB 00:00 2024-09-16T20:46:29.681 INFO:teuthology.orchestra.run.smithi029.stdout:(8/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.2 MB/s | 279 kB 00:00 2024-09-16T20:46:29.706 INFO:teuthology.orchestra.run.smithi029.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 142 kB/s | 14 kB 00:00 2024-09-16T20:46:29.707 INFO:teuthology.orchestra.run.smithi139.stdout:Last metadata expiration check: 0:01:48 ago on Mon 16 Sep 2024 08:44:41 PM UTC. 2024-09-16T20:46:29.749 INFO:teuthology.orchestra.run.smithi029.stdout:(10/13): python3-google-auth-2.34.0-1.el9.noarc 2.0 MB/s | 220 kB 00:00 2024-09-16T20:46:29.774 INFO:teuthology.orchestra.run.smithi029.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 979 kB/s | 59 kB 00:00 2024-09-16T20:46:29.779 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-16T20:46:29.779 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T20:46:29.780 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-09-16T20:46:29.780 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T20:46:29.780 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-16T20:46:29.780 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-fuse x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 899 k 2024-09-16T20:46:29.780 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-09-16T20:46:29.780 INFO:teuthology.orchestra.run.smithi043.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-16T20:46:29.780 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:46:29.780 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-16T20:46:29.780 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T20:46:29.780 INFO:teuthology.orchestra.run.smithi043.stdout:Install 2 Packages 2024-09-16T20:46:29.780 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:46:29.781 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 979 k 2024-09-16T20:46:29.781 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 2.9 M 2024-09-16T20:46:29.781 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-16T20:46:29.808 INFO:teuthology.orchestra.run.smithi029.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.5 MB/s | 90 kB 00:00 2024-09-16T20:46:29.841 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-16T20:46:29.842 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-16T20:46:29.842 INFO:teuthology.orchestra.run.smithi139.stdout: Package Architecture Version Repository Size 2024-09-16T20:46:29.842 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-16T20:46:29.842 INFO:teuthology.orchestra.run.smithi139.stdout:Installing: 2024-09-16T20:46:29.842 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-fuse x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 86 k 2024-09-16T20:46:29.842 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:46:29.842 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction Summary 2024-09-16T20:46:29.843 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-16T20:46:29.843 INFO:teuthology.orchestra.run.smithi139.stdout:Install 1 Package 2024-09-16T20:46:29.843 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:46:29.843 INFO:teuthology.orchestra.run.smithi139.stdout:Total download size: 86 k 2024-09-16T20:46:29.843 INFO:teuthology.orchestra.run.smithi139.stdout:Installed size: 231 k 2024-09-16T20:46:29.843 INFO:teuthology.orchestra.run.smithi139.stdout:Downloading Packages: 2024-09-16T20:46:29.858 INFO:teuthology.orchestra.run.smithi029.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 5.8 MB/s | 1.0 MB 00:00 2024-09-16T20:46:29.861 INFO:teuthology.orchestra.run.smithi029.stdout:-------------------------------------------------------------------------------- 2024-09-16T20:46:29.861 INFO:teuthology.orchestra.run.smithi029.stdout:Total 3.0 MB/s | 2.2 MB 00:00 2024-09-16T20:46:29.930 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction check 2024-09-16T20:46:29.946 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction check succeeded. 2024-09-16T20:46:29.946 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction test 2024-09-16T20:46:30.025 INFO:teuthology.orchestra.run.smithi043.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 680 kB/s | 80 kB 00:00 2024-09-16T20:46:30.034 INFO:teuthology.orchestra.run.smithi139.stdout:rbd-fuse-19.3.0-4883.g6e8474a0.el9.x86_64.rpm 449 kB/s | 86 kB 00:00 2024-09-16T20:46:30.034 INFO:teuthology.orchestra.run.smithi139.stdout:-------------------------------------------------------------------------------- 2024-09-16T20:46:30.035 INFO:teuthology.orchestra.run.smithi139.stdout:Total 445 kB/s | 86 kB 00:00 2024-09-16T20:46:30.035 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction check 2024-09-16T20:46:30.042 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction check succeeded. 2024-09-16T20:46:30.042 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction test 2024-09-16T20:46:30.043 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction test succeeded. 2024-09-16T20:46:30.043 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction 2024-09-16T20:46:30.105 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction test succeeded. 2024-09-16T20:46:30.106 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction 2024-09-16T20:46:30.200 INFO:teuthology.orchestra.run.smithi043.stdout:(2/2): ceph-fuse-19.3.0-4883.g6e8474a0.el9.x86_ 3.0 MB/s | 899 kB 00:00 2024-09-16T20:46:30.200 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-16T20:46:30.201 INFO:teuthology.orchestra.run.smithi043.stdout:Total 2.3 MB/s | 979 kB 00:00 2024-09-16T20:46:30.213 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-16T20:46:30.236 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-16T20:46:30.236 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-16T20:46:30.246 INFO:teuthology.orchestra.run.smithi139.stdout: Preparing : 1/1 2024-09-16T20:46:30.320 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-16T20:46:30.321 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-16T20:46:30.339 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : rbd-fuse-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T20:46:30.517 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-16T20:46:30.563 INFO:teuthology.orchestra.run.smithi029.stdout: Preparing : 1/1 2024-09-16T20:46:30.649 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-16T20:46:30.705 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: rbd-fuse-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T20:46:30.718 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-fuse-2:19.3.0-4883.g6e8474a0.el9.x86_64 2/2 2024-09-16T20:46:30.754 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-16T20:46:30.815 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-16T20:46:30.880 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-16T20:46:30.924 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-16T20:46:30.948 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : rbd-fuse-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T20:46:30.948 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:46:30.948 INFO:teuthology.orchestra.run.smithi139.stdout:Installed: 2024-09-16T20:46:30.948 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-fuse-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:46:30.948 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:46:30.948 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-16T20:46:30.976 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-16T20:46:31.101 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-16T20:46:31.226 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum -y install rbd-mirror 2024-09-16T20:46:31.228 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-16T20:46:31.290 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-16T20:46:31.463 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-fuse-2:19.3.0-4883.g6e8474a0.el9.x86_64 2/2 2024-09-16T20:46:31.463 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-fuse-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/2 2024-09-16T20:46:31.727 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-16T20:46:31.727 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:46:31.727 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-16T20:46:31.727 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-fuse-2:19.3.0-4883.g6e8474a0.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-16T20:46:31.727 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:46:31.727 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-16T20:46:31.843 INFO:teuthology.orchestra.run.smithi139.stdout:Last metadata expiration check: 0:01:50 ago on Mon 16 Sep 2024 08:44:41 PM UTC. 2024-09-16T20:46:31.971 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-volume 2024-09-16T20:46:31.985 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-16T20:46:31.986 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-16T20:46:31.986 INFO:teuthology.orchestra.run.smithi139.stdout: Package Arch Version Repo Size 2024-09-16T20:46:31.986 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-16T20:46:31.986 INFO:teuthology.orchestra.run.smithi139.stdout:Installing: 2024-09-16T20:46:31.986 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-mirror x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 3.2 M 2024-09-16T20:46:31.986 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:46:31.986 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction Summary 2024-09-16T20:46:31.986 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-16T20:46:31.986 INFO:teuthology.orchestra.run.smithi139.stdout:Install 1 Package 2024-09-16T20:46:31.986 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:46:31.987 INFO:teuthology.orchestra.run.smithi139.stdout:Total download size: 3.2 M 2024-09-16T20:46:31.987 INFO:teuthology.orchestra.run.smithi139.stdout:Installed size: 14 M 2024-09-16T20:46:31.987 INFO:teuthology.orchestra.run.smithi139.stdout:Downloading Packages: 2024-09-16T20:46:32.057 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-16T20:46:32.125 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-16T20:46:32.159 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-16T20:46:32.217 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-16T20:46:32.255 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-mgr-rook-2:19.3.0-4883.g6e8474a0.el9.noarch 13/13 2024-09-16T20:46:32.329 INFO:teuthology.orchestra.run.smithi139.stdout:rbd-mirror-19.3.0-4883.g6e8474a0.el9.x86_64.rpm 9.4 MB/s | 3.2 MB 00:00 2024-09-16T20:46:32.329 INFO:teuthology.orchestra.run.smithi139.stdout:-------------------------------------------------------------------------------- 2024-09-16T20:46:32.330 INFO:teuthology.orchestra.run.smithi139.stdout:Total 9.3 MB/s | 3.2 MB 00:00 2024-09-16T20:46:32.330 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction check 2024-09-16T20:46:32.339 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction check succeeded. 2024-09-16T20:46:32.339 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction test 2024-09-16T20:46:32.422 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction test succeeded. 2024-09-16T20:46:32.422 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction 2024-09-16T20:46:32.588 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:34 ago on Mon 16 Sep 2024 08:44:58 PM UTC. 2024-09-16T20:46:32.727 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-16T20:46:32.728 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T20:46:32.728 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-09-16T20:46:32.728 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T20:46:32.728 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-16T20:46:32.728 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-volume noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 285 k 2024-09-16T20:46:32.728 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:46:32.729 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-16T20:46:32.729 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T20:46:32.729 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-09-16T20:46:32.729 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:46:32.729 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 285 k 2024-09-16T20:46:32.729 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 1.3 M 2024-09-16T20:46:32.729 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-16T20:46:32.861 INFO:teuthology.orchestra.run.smithi139.stdout: Preparing : 1/1 2024-09-16T20:46:32.929 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : rbd-mirror-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T20:46:32.954 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-volume-19.3.0-4883.g6e8474a0.el9.noarch.rp 1.2 MB/s | 285 kB 00:00 2024-09-16T20:46:32.954 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-16T20:46:32.955 INFO:teuthology.orchestra.run.smithi043.stdout:Total 1.2 MB/s | 285 kB 00:00 2024-09-16T20:46:32.955 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-16T20:46:32.956 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T20:46:32.956 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T20:46:32.956 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-16T20:46:32.956 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T20:46:32.956 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T20:46:32.956 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:46:32.964 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-16T20:46:32.964 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-16T20:46:33.000 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-16T20:46:33.001 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-16T20:46:33.230 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-16T20:46:33.306 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-volume-2:19.3.0-4883.g6e8474a0.el9.noarch 1/1 2024-09-16T20:46:33.323 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-volume-2:19.3.0-4883.g6e8474a0.el9.noarch 1/1 2024-09-16T20:46:33.323 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T20:46:33.323 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-16T20:46:33.323 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:46:33.341 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4883.g6e8474a0.el9.noarch 13/13 2024-09-16T20:46:33.341 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4883.g6e8474a0.el9.noarch 1/13 2024-09-16T20:46:33.341 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-16T20:46:33.341 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-16T20:46:33.341 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-16T20:46:33.341 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-16T20:46:33.341 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-16T20:46:33.341 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-16T20:46:33.341 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-16T20:46:33.341 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-16T20:46:33.341 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-16T20:46:33.341 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-16T20:46:33.342 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-16T20:46:33.697 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-16T20:46:33.697 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:46:33.697 INFO:teuthology.orchestra.run.smithi029.stdout:Installed: 2024-09-16T20:46:33.697 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-rook-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-16T20:46:33.697 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-16T20:46:33.698 INFO:teuthology.orchestra.run.smithi029.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-16T20:46:33.698 INFO:teuthology.orchestra.run.smithi029.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-16T20:46:33.698 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-16T20:46:33.698 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-16T20:46:33.698 INFO:teuthology.orchestra.run.smithi029.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-16T20:46:33.698 INFO:teuthology.orchestra.run.smithi029.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-16T20:46:33.698 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-16T20:46:33.698 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-16T20:46:33.698 INFO:teuthology.orchestra.run.smithi029.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-16T20:46:33.698 INFO:teuthology.orchestra.run.smithi029.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-16T20:46:33.698 INFO:teuthology.orchestra.run.smithi029.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-16T20:46:33.698 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:46:33.698 INFO:teuthology.orchestra.run.smithi029.stdout:Complete! 2024-09-16T20:46:34.017 DEBUG:teuthology.orchestra.run.smithi029:> sudo yum -y install ceph-mgr-cephadm 2024-09-16T20:46:34.027 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : rbd-mirror-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T20:46:34.027 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:46:34.027 INFO:teuthology.orchestra.run.smithi139.stdout:Installed: 2024-09-16T20:46:34.028 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-mirror-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:46:34.028 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:46:34.028 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-16T20:46:34.254 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum -y install rbd-nbd 2024-09-16T20:46:34.353 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-volume-2:19.3.0-4883.g6e8474a0.el9.noarch 1/1 2024-09-16T20:46:34.353 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:46:34.354 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-16T20:46:34.354 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-volume-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-16T20:46:34.354 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:46:34.354 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-16T20:46:34.626 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install librados-devel 2024-09-16T20:46:34.627 INFO:teuthology.orchestra.run.smithi029.stdout:Last metadata expiration check: 0:01:48 ago on Mon 16 Sep 2024 08:44:46 PM UTC. 2024-09-16T20:46:34.766 INFO:teuthology.orchestra.run.smithi029.stdout:Dependencies resolved. 2024-09-16T20:46:34.767 INFO:teuthology.orchestra.run.smithi029.stdout:================================================================================ 2024-09-16T20:46:34.767 INFO:teuthology.orchestra.run.smithi029.stdout: Package Arch Version Repository Size 2024-09-16T20:46:34.767 INFO:teuthology.orchestra.run.smithi029.stdout:================================================================================ 2024-09-16T20:46:34.767 INFO:teuthology.orchestra.run.smithi029.stdout:Installing: 2024-09-16T20:46:34.767 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 154 k 2024-09-16T20:46:34.768 INFO:teuthology.orchestra.run.smithi029.stdout:Installing dependencies: 2024-09-16T20:46:34.768 INFO:teuthology.orchestra.run.smithi029.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-16T20:46:34.768 INFO:teuthology.orchestra.run.smithi029.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-16T20:46:34.768 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-16T20:46:34.768 INFO:teuthology.orchestra.run.smithi029.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-16T20:46:34.768 INFO:teuthology.orchestra.run.smithi029.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-16T20:46:34.768 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:46:34.768 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction Summary 2024-09-16T20:46:34.768 INFO:teuthology.orchestra.run.smithi029.stdout:================================================================================ 2024-09-16T20:46:34.768 INFO:teuthology.orchestra.run.smithi029.stdout:Install 6 Packages 2024-09-16T20:46:34.768 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:46:34.769 INFO:teuthology.orchestra.run.smithi029.stdout:Total download size: 7.0 M 2024-09-16T20:46:34.769 INFO:teuthology.orchestra.run.smithi029.stdout:Installed size: 33 M 2024-09-16T20:46:34.769 INFO:teuthology.orchestra.run.smithi029.stdout:Downloading Packages: 2024-09-16T20:46:34.857 INFO:teuthology.orchestra.run.smithi139.stdout:Last metadata expiration check: 0:01:53 ago on Mon 16 Sep 2024 08:44:41 PM UTC. 2024-09-16T20:46:34.994 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-16T20:46:34.995 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-16T20:46:34.995 INFO:teuthology.orchestra.run.smithi139.stdout: Package Architecture Version Repository Size 2024-09-16T20:46:34.995 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-16T20:46:34.995 INFO:teuthology.orchestra.run.smithi139.stdout:Installing: 2024-09-16T20:46:34.995 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-nbd x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 170 k 2024-09-16T20:46:34.995 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:46:34.995 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction Summary 2024-09-16T20:46:34.995 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-16T20:46:34.995 INFO:teuthology.orchestra.run.smithi139.stdout:Install 1 Package 2024-09-16T20:46:34.996 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:46:34.996 INFO:teuthology.orchestra.run.smithi139.stdout:Total download size: 170 k 2024-09-16T20:46:34.996 INFO:teuthology.orchestra.run.smithi139.stdout:Installed size: 490 k 2024-09-16T20:46:34.996 INFO:teuthology.orchestra.run.smithi139.stdout:Downloading Packages: 2024-09-16T20:46:35.187 INFO:teuthology.orchestra.run.smithi029.stdout:(1/6): ceph-mgr-cephadm-19.3.0-4883.g6e8474a0.e 710 kB/s | 154 kB 00:00 2024-09-16T20:46:35.198 INFO:teuthology.orchestra.run.smithi139.stdout:rbd-nbd-19.3.0-4883.g6e8474a0.el9.x86_64.rpm 844 kB/s | 170 kB 00:00 2024-09-16T20:46:35.198 INFO:teuthology.orchestra.run.smithi139.stdout:-------------------------------------------------------------------------------- 2024-09-16T20:46:35.198 INFO:teuthology.orchestra.run.smithi139.stdout:Total 839 kB/s | 170 kB 00:00 2024-09-16T20:46:35.199 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction check 2024-09-16T20:46:35.206 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction check succeeded. 2024-09-16T20:46:35.206 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction test 2024-09-16T20:46:35.236 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:37 ago on Mon 16 Sep 2024 08:44:58 PM UTC. 2024-09-16T20:46:35.262 INFO:teuthology.orchestra.run.smithi029.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 7.1 MB/s | 548 kB 00:00 2024-09-16T20:46:35.271 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction test succeeded. 2024-09-16T20:46:35.271 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction 2024-09-16T20:46:35.296 INFO:teuthology.orchestra.run.smithi029.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 1.7 MB/s | 58 kB 00:00 2024-09-16T20:46:35.330 INFO:teuthology.orchestra.run.smithi029.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.2 MB/s | 74 kB 00:00 2024-09-16T20:46:35.374 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-16T20:46:35.375 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T20:46:35.375 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repo Size 2024-09-16T20:46:35.375 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T20:46:35.375 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-16T20:46:35.375 INFO:teuthology.orchestra.run.smithi043.stdout: librados-devel x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 127 k 2024-09-16T20:46:35.375 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:46:35.375 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-16T20:46:35.375 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T20:46:35.375 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-09-16T20:46:35.375 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:46:35.375 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 127 k 2024-09-16T20:46:35.375 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 456 k 2024-09-16T20:46:35.376 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-16T20:46:35.422 INFO:teuthology.orchestra.run.smithi139.stdout: Preparing : 1/1 2024-09-16T20:46:35.537 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : rbd-nbd-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T20:46:35.592 INFO:teuthology.orchestra.run.smithi043.stdout:librados-devel-19.3.0-4883.g6e8474a0.el9.x86_64 586 kB/s | 127 kB 00:00 2024-09-16T20:46:35.593 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-16T20:46:35.593 INFO:teuthology.orchestra.run.smithi043.stdout:Total 582 kB/s | 127 kB 00:00 2024-09-16T20:46:35.593 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-16T20:46:35.599 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-16T20:46:35.599 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-16T20:46:35.605 INFO:teuthology.orchestra.run.smithi029.stdout:(5/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 390 kB/s | 248 kB 00:00 2024-09-16T20:46:35.668 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-16T20:46:35.668 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-16T20:46:35.823 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-16T20:46:35.927 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librados-devel-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T20:46:35.954 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: rbd-nbd-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T20:46:36.194 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : rbd-nbd-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T20:46:36.194 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:46:36.194 INFO:teuthology.orchestra.run.smithi139.stdout:Installed: 2024-09-16T20:46:36.195 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-nbd-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:46:36.195 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:46:36.195 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-16T20:46:36.388 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librados-devel-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T20:46:36.473 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum -y install rbd-nbd 2024-09-16T20:46:36.556 INFO:teuthology.orchestra.run.smithi029.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 3.8 MB/s | 6.0 MB 00:01 2024-09-16T20:46:36.558 INFO:teuthology.orchestra.run.smithi029.stdout:-------------------------------------------------------------------------------- 2024-09-16T20:46:36.558 INFO:teuthology.orchestra.run.smithi029.stdout:Total 3.9 MB/s | 7.0 MB 00:01 2024-09-16T20:46:36.628 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librados-devel-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T20:46:36.628 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:46:36.628 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-16T20:46:36.628 INFO:teuthology.orchestra.run.smithi043.stdout: librados-devel-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:46:36.628 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:46:36.628 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-16T20:46:36.634 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction check 2024-09-16T20:46:36.647 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction check succeeded. 2024-09-16T20:46:36.647 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction test 2024-09-16T20:46:36.753 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction test succeeded. 2024-09-16T20:46:36.754 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction 2024-09-16T20:46:36.861 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install libcephfs2 2024-09-16T20:46:37.071 INFO:teuthology.orchestra.run.smithi139.stdout:Last metadata expiration check: 0:01:56 ago on Mon 16 Sep 2024 08:44:41 PM UTC. 2024-09-16T20:46:37.072 INFO:teuthology.orchestra.run.smithi029.stdout: Preparing : 1/1 2024-09-16T20:46:37.165 INFO:teuthology.orchestra.run.smithi139.stdout:Package rbd-nbd-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-16T20:46:37.210 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-16T20:46:37.211 INFO:teuthology.orchestra.run.smithi139.stdout:Nothing to do. 2024-09-16T20:46:37.211 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-16T20:46:37.245 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-16T20:46:37.292 DEBUG:teuthology.parallel:result is None 2024-09-16T20:46:37.304 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-16T20:46:37.476 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:39 ago on Mon 16 Sep 2024 08:44:58 PM UTC. 2024-09-16T20:46:37.569 INFO:teuthology.orchestra.run.smithi043.stdout:Package libcephfs2-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-16T20:46:37.615 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-16T20:46:37.617 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-09-16T20:46:37.617 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-16T20:46:37.685 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install libcephfs-devel 2024-09-16T20:46:37.863 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-16T20:46:37.961 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-16T20:46:38.045 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-16T20:46:38.086 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 6/6 2024-09-16T20:46:38.289 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:40 ago on Mon 16 Sep 2024 08:44:58 PM UTC. 2024-09-16T20:46:38.427 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-16T20:46:38.427 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T20:46:38.427 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repo Size 2024-09-16T20:46:38.428 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T20:46:38.428 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-16T20:46:38.428 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs-devel x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 32 k 2024-09-16T20:46:38.428 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:46:38.428 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-16T20:46:38.428 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T20:46:38.428 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-09-16T20:46:38.428 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:46:38.428 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 32 k 2024-09-16T20:46:38.428 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 142 k 2024-09-16T20:46:38.428 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-16T20:46:38.579 INFO:teuthology.orchestra.run.smithi043.stdout:libcephfs-devel-19.3.0-4883.g6e8474a0.el9.x86_6 211 kB/s | 32 kB 00:00 2024-09-16T20:46:38.579 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-16T20:46:38.580 INFO:teuthology.orchestra.run.smithi043.stdout:Total 209 kB/s | 32 kB 00:00 2024-09-16T20:46:38.580 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-16T20:46:38.583 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-16T20:46:38.583 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-16T20:46:38.608 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-16T20:46:38.608 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-16T20:46:38.691 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-16T20:46:38.795 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libcephfs-devel-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T20:46:39.185 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libcephfs-devel-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T20:46:39.228 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 6/6 2024-09-16T20:46:39.228 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 1/6 2024-09-16T20:46:39.228 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-16T20:46:39.228 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-16T20:46:39.228 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-16T20:46:39.229 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-16T20:46:39.419 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libcephfs-devel-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T20:46:39.419 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:46:39.419 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-16T20:46:39.419 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs-devel-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:46:39.419 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:46:39.419 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-16T20:46:39.537 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-16T20:46:39.537 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:46:39.537 INFO:teuthology.orchestra.run.smithi029.stdout:Installed: 2024-09-16T20:46:39.537 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-16T20:46:39.537 INFO:teuthology.orchestra.run.smithi029.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-16T20:46:39.537 INFO:teuthology.orchestra.run.smithi029.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-16T20:46:39.537 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-16T20:46:39.537 INFO:teuthology.orchestra.run.smithi029.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-16T20:46:39.538 INFO:teuthology.orchestra.run.smithi029.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-16T20:46:39.538 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:46:39.538 INFO:teuthology.orchestra.run.smithi029.stdout:Complete! 2024-09-16T20:46:39.576 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install librados2 2024-09-16T20:46:39.845 DEBUG:teuthology.orchestra.run.smithi029:> sudo yum -y install ceph-fuse 2024-09-16T20:46:40.198 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:42 ago on Mon 16 Sep 2024 08:44:58 PM UTC. 2024-09-16T20:46:40.296 INFO:teuthology.orchestra.run.smithi043.stdout:Package librados2-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-16T20:46:40.343 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-16T20:46:40.345 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-09-16T20:46:40.345 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-16T20:46:40.416 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install librbd1 2024-09-16T20:46:40.460 INFO:teuthology.orchestra.run.smithi029.stdout:Last metadata expiration check: 0:01:54 ago on Mon 16 Sep 2024 08:44:46 PM UTC. 2024-09-16T20:46:40.601 INFO:teuthology.orchestra.run.smithi029.stdout:Dependencies resolved. 2024-09-16T20:46:40.601 INFO:teuthology.orchestra.run.smithi029.stdout:================================================================================ 2024-09-16T20:46:40.602 INFO:teuthology.orchestra.run.smithi029.stdout: Package Arch Version Repository Size 2024-09-16T20:46:40.602 INFO:teuthology.orchestra.run.smithi029.stdout:================================================================================ 2024-09-16T20:46:40.602 INFO:teuthology.orchestra.run.smithi029.stdout:Installing: 2024-09-16T20:46:40.602 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-fuse x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 899 k 2024-09-16T20:46:40.602 INFO:teuthology.orchestra.run.smithi029.stdout:Installing dependencies: 2024-09-16T20:46:40.602 INFO:teuthology.orchestra.run.smithi029.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-16T20:46:40.602 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:46:40.602 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction Summary 2024-09-16T20:46:40.602 INFO:teuthology.orchestra.run.smithi029.stdout:================================================================================ 2024-09-16T20:46:40.602 INFO:teuthology.orchestra.run.smithi029.stdout:Install 2 Packages 2024-09-16T20:46:40.603 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:46:40.603 INFO:teuthology.orchestra.run.smithi029.stdout:Total download size: 979 k 2024-09-16T20:46:40.603 INFO:teuthology.orchestra.run.smithi029.stdout:Installed size: 2.9 M 2024-09-16T20:46:40.603 INFO:teuthology.orchestra.run.smithi029.stdout:Downloading Packages: 2024-09-16T20:46:40.756 INFO:teuthology.orchestra.run.smithi029.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 613 kB/s | 80 kB 00:00 2024-09-16T20:46:40.922 INFO:teuthology.orchestra.run.smithi029.stdout:(2/2): ceph-fuse-19.3.0-4883.g6e8474a0.el9.x86_ 3.0 MB/s | 899 kB 00:00 2024-09-16T20:46:40.923 INFO:teuthology.orchestra.run.smithi029.stdout:-------------------------------------------------------------------------------- 2024-09-16T20:46:40.923 INFO:teuthology.orchestra.run.smithi029.stdout:Total 3.0 MB/s | 979 kB 00:00 2024-09-16T20:46:40.936 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction check 2024-09-16T20:46:40.958 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction check succeeded. 2024-09-16T20:46:40.958 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction test 2024-09-16T20:46:41.032 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:43 ago on Mon 16 Sep 2024 08:44:58 PM UTC. 2024-09-16T20:46:41.043 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction test succeeded. 2024-09-16T20:46:41.044 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction 2024-09-16T20:46:41.127 INFO:teuthology.orchestra.run.smithi043.stdout:Package librbd1-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-16T20:46:41.173 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-16T20:46:41.176 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-09-16T20:46:41.176 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-16T20:46:41.238 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install python3-rados 2024-09-16T20:46:41.240 INFO:teuthology.orchestra.run.smithi029.stdout: Preparing : 1/1 2024-09-16T20:46:41.397 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-16T20:46:41.478 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-fuse-2:19.3.0-4883.g6e8474a0.el9.x86_64 2/2 2024-09-16T20:46:41.843 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:43 ago on Mon 16 Sep 2024 08:44:58 PM UTC. 2024-09-16T20:46:41.936 INFO:teuthology.orchestra.run.smithi043.stdout:Package python3-rados-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-16T20:46:41.982 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-16T20:46:41.983 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-09-16T20:46:41.983 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-16T20:46:42.058 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install python3-rgw 2024-09-16T20:46:42.236 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-fuse-2:19.3.0-4883.g6e8474a0.el9.x86_64 2/2 2024-09-16T20:46:42.237 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-fuse-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/2 2024-09-16T20:46:42.512 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-16T20:46:42.512 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:46:42.512 INFO:teuthology.orchestra.run.smithi029.stdout:Installed: 2024-09-16T20:46:42.512 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-fuse-2:19.3.0-4883.g6e8474a0.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-16T20:46:42.513 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:46:42.513 INFO:teuthology.orchestra.run.smithi029.stdout:Complete! 2024-09-16T20:46:42.666 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:44 ago on Mon 16 Sep 2024 08:44:58 PM UTC. 2024-09-16T20:46:42.758 INFO:teuthology.orchestra.run.smithi043.stdout:Package python3-rgw-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-16T20:46:42.804 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-16T20:46:42.805 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-09-16T20:46:42.805 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-16T20:46:42.855 DEBUG:teuthology.orchestra.run.smithi029:> sudo yum -y install ceph-volume 2024-09-16T20:46:42.875 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install python3-cephfs 2024-09-16T20:46:43.471 INFO:teuthology.orchestra.run.smithi029.stdout:Last metadata expiration check: 0:01:57 ago on Mon 16 Sep 2024 08:44:46 PM UTC. 2024-09-16T20:46:43.480 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:45 ago on Mon 16 Sep 2024 08:44:58 PM UTC. 2024-09-16T20:46:43.571 INFO:teuthology.orchestra.run.smithi043.stdout:Package python3-cephfs-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-16T20:46:43.612 INFO:teuthology.orchestra.run.smithi029.stdout:Dependencies resolved. 2024-09-16T20:46:43.613 INFO:teuthology.orchestra.run.smithi029.stdout:================================================================================ 2024-09-16T20:46:43.613 INFO:teuthology.orchestra.run.smithi029.stdout: Package Arch Version Repository Size 2024-09-16T20:46:43.613 INFO:teuthology.orchestra.run.smithi029.stdout:================================================================================ 2024-09-16T20:46:43.613 INFO:teuthology.orchestra.run.smithi029.stdout:Installing: 2024-09-16T20:46:43.613 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-volume noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 285 k 2024-09-16T20:46:43.613 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:46:43.613 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction Summary 2024-09-16T20:46:43.613 INFO:teuthology.orchestra.run.smithi029.stdout:================================================================================ 2024-09-16T20:46:43.613 INFO:teuthology.orchestra.run.smithi029.stdout:Install 1 Package 2024-09-16T20:46:43.613 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:46:43.613 INFO:teuthology.orchestra.run.smithi029.stdout:Total download size: 285 k 2024-09-16T20:46:43.614 INFO:teuthology.orchestra.run.smithi029.stdout:Installed size: 1.3 M 2024-09-16T20:46:43.614 INFO:teuthology.orchestra.run.smithi029.stdout:Downloading Packages: 2024-09-16T20:46:43.617 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-16T20:46:43.618 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-09-16T20:46:43.618 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-16T20:46:43.692 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install python3-rbd 2024-09-16T20:46:43.844 INFO:teuthology.orchestra.run.smithi029.stdout:ceph-volume-19.3.0-4883.g6e8474a0.el9.noarch.rp 1.2 MB/s | 285 kB 00:00 2024-09-16T20:46:43.844 INFO:teuthology.orchestra.run.smithi029.stdout:-------------------------------------------------------------------------------- 2024-09-16T20:46:43.845 INFO:teuthology.orchestra.run.smithi029.stdout:Total 1.2 MB/s | 285 kB 00:00 2024-09-16T20:46:43.845 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction check 2024-09-16T20:46:43.854 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction check succeeded. 2024-09-16T20:46:43.854 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction test 2024-09-16T20:46:43.884 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction test succeeded. 2024-09-16T20:46:43.885 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction 2024-09-16T20:46:44.122 INFO:teuthology.orchestra.run.smithi029.stdout: Preparing : 1/1 2024-09-16T20:46:44.198 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-volume-2:19.3.0-4883.g6e8474a0.el9.noarch 1/1 2024-09-16T20:46:44.215 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-volume-2:19.3.0-4883.g6e8474a0.el9.noarch 1/1 2024-09-16T20:46:44.215 INFO:teuthology.orchestra.run.smithi029.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T20:46:44.215 INFO:teuthology.orchestra.run.smithi029.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-16T20:46:44.215 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:46:44.306 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:46 ago on Mon 16 Sep 2024 08:44:58 PM UTC. 2024-09-16T20:46:44.400 INFO:teuthology.orchestra.run.smithi043.stdout:Package python3-rbd-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-16T20:46:44.446 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-16T20:46:44.447 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-09-16T20:46:44.447 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-16T20:46:44.517 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install rbd-fuse 2024-09-16T20:46:45.133 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:47 ago on Mon 16 Sep 2024 08:44:58 PM UTC. 2024-09-16T20:46:45.275 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-16T20:46:45.275 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T20:46:45.275 INFO:teuthology.orchestra.run.smithi043.stdout: Package Architecture Version Repository Size 2024-09-16T20:46:45.275 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T20:46:45.275 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-16T20:46:45.276 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-fuse x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 86 k 2024-09-16T20:46:45.276 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:46:45.276 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-16T20:46:45.276 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T20:46:45.276 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-09-16T20:46:45.276 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:46:45.276 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 86 k 2024-09-16T20:46:45.276 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 231 k 2024-09-16T20:46:45.276 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-16T20:46:45.345 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-volume-2:19.3.0-4883.g6e8474a0.el9.noarch 1/1 2024-09-16T20:46:45.345 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:46:45.345 INFO:teuthology.orchestra.run.smithi029.stdout:Installed: 2024-09-16T20:46:45.345 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-volume-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-16T20:46:45.345 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:46:45.345 INFO:teuthology.orchestra.run.smithi029.stdout:Complete! 2024-09-16T20:46:45.474 INFO:teuthology.orchestra.run.smithi043.stdout:rbd-fuse-19.3.0-4883.g6e8474a0.el9.x86_64.rpm 434 kB/s | 86 kB 00:00 2024-09-16T20:46:45.474 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-16T20:46:45.475 INFO:teuthology.orchestra.run.smithi043.stdout:Total 430 kB/s | 86 kB 00:00 2024-09-16T20:46:45.475 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-16T20:46:45.482 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-16T20:46:45.482 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-16T20:46:45.550 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-16T20:46:45.550 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-16T20:46:45.709 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-16T20:46:45.734 DEBUG:teuthology.orchestra.run.smithi029:> sudo yum -y install librados-devel 2024-09-16T20:46:45.817 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : rbd-fuse-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T20:46:46.216 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: rbd-fuse-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T20:46:46.347 INFO:teuthology.orchestra.run.smithi029.stdout:Last metadata expiration check: 0:02:00 ago on Mon 16 Sep 2024 08:44:46 PM UTC. 2024-09-16T20:46:46.461 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : rbd-fuse-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T20:46:46.461 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:46:46.461 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-16T20:46:46.462 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-fuse-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:46:46.462 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:46:46.462 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-16T20:46:46.486 INFO:teuthology.orchestra.run.smithi029.stdout:Dependencies resolved. 2024-09-16T20:46:46.486 INFO:teuthology.orchestra.run.smithi029.stdout:================================================================================ 2024-09-16T20:46:46.486 INFO:teuthology.orchestra.run.smithi029.stdout: Package Arch Version Repo Size 2024-09-16T20:46:46.486 INFO:teuthology.orchestra.run.smithi029.stdout:================================================================================ 2024-09-16T20:46:46.486 INFO:teuthology.orchestra.run.smithi029.stdout:Installing: 2024-09-16T20:46:46.487 INFO:teuthology.orchestra.run.smithi029.stdout: librados-devel x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 127 k 2024-09-16T20:46:46.487 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:46:46.487 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction Summary 2024-09-16T20:46:46.487 INFO:teuthology.orchestra.run.smithi029.stdout:================================================================================ 2024-09-16T20:46:46.487 INFO:teuthology.orchestra.run.smithi029.stdout:Install 1 Package 2024-09-16T20:46:46.487 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:46:46.487 INFO:teuthology.orchestra.run.smithi029.stdout:Total download size: 127 k 2024-09-16T20:46:46.487 INFO:teuthology.orchestra.run.smithi029.stdout:Installed size: 456 k 2024-09-16T20:46:46.487 INFO:teuthology.orchestra.run.smithi029.stdout:Downloading Packages: 2024-09-16T20:46:46.699 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install rbd-mirror 2024-09-16T20:46:46.699 INFO:teuthology.orchestra.run.smithi029.stdout:librados-devel-19.3.0-4883.g6e8474a0.el9.x86_64 598 kB/s | 127 kB 00:00 2024-09-16T20:46:46.700 INFO:teuthology.orchestra.run.smithi029.stdout:-------------------------------------------------------------------------------- 2024-09-16T20:46:46.700 INFO:teuthology.orchestra.run.smithi029.stdout:Total 594 kB/s | 127 kB 00:00 2024-09-16T20:46:46.700 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction check 2024-09-16T20:46:46.706 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction check succeeded. 2024-09-16T20:46:46.706 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction test 2024-09-16T20:46:46.773 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction test succeeded. 2024-09-16T20:46:46.774 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction 2024-09-16T20:46:46.931 INFO:teuthology.orchestra.run.smithi029.stdout: Preparing : 1/1 2024-09-16T20:46:47.042 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : librados-devel-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T20:46:47.314 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:49 ago on Mon 16 Sep 2024 08:44:58 PM UTC. 2024-09-16T20:46:47.453 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-16T20:46:47.453 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T20:46:47.454 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repo Size 2024-09-16T20:46:47.454 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T20:46:47.454 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-16T20:46:47.454 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-mirror x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 3.2 M 2024-09-16T20:46:47.454 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:46:47.454 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-16T20:46:47.454 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T20:46:47.454 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-09-16T20:46:47.454 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:46:47.454 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 3.2 M 2024-09-16T20:46:47.454 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 14 M 2024-09-16T20:46:47.454 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-16T20:46:47.535 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: librados-devel-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T20:46:47.785 INFO:teuthology.orchestra.run.smithi043.stdout:rbd-mirror-19.3.0-4883.g6e8474a0.el9.x86_64.rpm 9.7 MB/s | 3.2 MB 00:00 2024-09-16T20:46:47.785 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-16T20:46:47.786 INFO:teuthology.orchestra.run.smithi043.stdout:Total 9.6 MB/s | 3.2 MB 00:00 2024-09-16T20:46:47.786 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-16T20:46:47.796 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-16T20:46:47.796 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-16T20:46:47.828 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : librados-devel-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T20:46:47.828 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:46:47.828 INFO:teuthology.orchestra.run.smithi029.stdout:Installed: 2024-09-16T20:46:47.828 INFO:teuthology.orchestra.run.smithi029.stdout: librados-devel-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:46:47.828 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:46:47.828 INFO:teuthology.orchestra.run.smithi029.stdout:Complete! 2024-09-16T20:46:47.883 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-16T20:46:47.884 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-16T20:46:48.119 DEBUG:teuthology.orchestra.run.smithi029:> sudo yum -y install libcephfs2 2024-09-16T20:46:48.348 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-16T20:46:48.411 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : rbd-mirror-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T20:46:48.438 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T20:46:48.438 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T20:46:48.438 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-16T20:46:48.438 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T20:46:48.438 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T20:46:48.438 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:46:48.732 INFO:teuthology.orchestra.run.smithi029.stdout:Last metadata expiration check: 0:02:02 ago on Mon 16 Sep 2024 08:44:46 PM UTC. 2024-09-16T20:46:48.829 INFO:teuthology.orchestra.run.smithi029.stdout:Package libcephfs2-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-16T20:46:48.875 INFO:teuthology.orchestra.run.smithi029.stdout:Dependencies resolved. 2024-09-16T20:46:48.876 INFO:teuthology.orchestra.run.smithi029.stdout:Nothing to do. 2024-09-16T20:46:48.876 INFO:teuthology.orchestra.run.smithi029.stdout:Complete! 2024-09-16T20:46:48.956 DEBUG:teuthology.orchestra.run.smithi029:> sudo yum -y install libcephfs-devel 2024-09-16T20:46:49.580 INFO:teuthology.orchestra.run.smithi029.stdout:Last metadata expiration check: 0:02:03 ago on Mon 16 Sep 2024 08:44:46 PM UTC. 2024-09-16T20:46:49.602 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : rbd-mirror-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T20:46:49.602 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:46:49.602 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-16T20:46:49.602 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-mirror-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:46:49.602 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:46:49.602 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-16T20:46:49.717 INFO:teuthology.orchestra.run.smithi029.stdout:Dependencies resolved. 2024-09-16T20:46:49.717 INFO:teuthology.orchestra.run.smithi029.stdout:================================================================================ 2024-09-16T20:46:49.717 INFO:teuthology.orchestra.run.smithi029.stdout: Package Arch Version Repo Size 2024-09-16T20:46:49.718 INFO:teuthology.orchestra.run.smithi029.stdout:================================================================================ 2024-09-16T20:46:49.718 INFO:teuthology.orchestra.run.smithi029.stdout:Installing: 2024-09-16T20:46:49.718 INFO:teuthology.orchestra.run.smithi029.stdout: libcephfs-devel x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 32 k 2024-09-16T20:46:49.718 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:46:49.718 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction Summary 2024-09-16T20:46:49.718 INFO:teuthology.orchestra.run.smithi029.stdout:================================================================================ 2024-09-16T20:46:49.718 INFO:teuthology.orchestra.run.smithi029.stdout:Install 1 Package 2024-09-16T20:46:49.718 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:46:49.718 INFO:teuthology.orchestra.run.smithi029.stdout:Total download size: 32 k 2024-09-16T20:46:49.718 INFO:teuthology.orchestra.run.smithi029.stdout:Installed size: 142 k 2024-09-16T20:46:49.718 INFO:teuthology.orchestra.run.smithi029.stdout:Downloading Packages: 2024-09-16T20:46:49.846 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install rbd-nbd 2024-09-16T20:46:50.461 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:52 ago on Mon 16 Sep 2024 08:44:58 PM UTC. 2024-09-16T20:46:50.608 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-16T20:46:50.609 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T20:46:50.609 INFO:teuthology.orchestra.run.smithi043.stdout: Package Architecture Version Repository Size 2024-09-16T20:46:50.609 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T20:46:50.609 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-16T20:46:50.609 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-nbd x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 170 k 2024-09-16T20:46:50.609 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:46:50.609 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-16T20:46:50.609 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T20:46:50.610 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-09-16T20:46:50.610 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:46:50.610 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 170 k 2024-09-16T20:46:50.610 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 490 k 2024-09-16T20:46:50.610 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-16T20:46:50.805 INFO:teuthology.orchestra.run.smithi043.stdout:rbd-nbd-19.3.0-4883.g6e8474a0.el9.x86_64.rpm 875 kB/s | 170 kB 00:00 2024-09-16T20:46:50.805 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-16T20:46:50.805 INFO:teuthology.orchestra.run.smithi043.stdout:Total 868 kB/s | 170 kB 00:00 2024-09-16T20:46:50.806 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-16T20:46:50.814 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-16T20:46:50.814 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-16T20:46:50.881 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-16T20:46:50.882 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-16T20:46:51.041 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-16T20:46:51.157 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : rbd-nbd-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T20:46:51.578 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: rbd-nbd-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T20:46:51.843 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : rbd-nbd-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T20:46:51.843 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:46:51.843 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-16T20:46:51.843 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-nbd-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:46:51.843 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:46:51.843 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-16T20:46:52.051 INFO:teuthology.orchestra.run.smithi029.stdout:libcephfs-devel-19.3.0-4883.g6e8474a0.el9.x86_6 14 kB/s | 32 kB 00:02 2024-09-16T20:46:52.052 INFO:teuthology.orchestra.run.smithi029.stdout:-------------------------------------------------------------------------------- 2024-09-16T20:46:52.052 INFO:teuthology.orchestra.run.smithi029.stdout:Total 14 kB/s | 32 kB 00:02 2024-09-16T20:46:52.053 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction check 2024-09-16T20:46:52.057 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction check succeeded. 2024-09-16T20:46:52.057 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction test 2024-09-16T20:46:52.076 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction test succeeded. 2024-09-16T20:46:52.077 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction 2024-09-16T20:46:52.100 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install rbd-nbd 2024-09-16T20:46:52.170 INFO:teuthology.orchestra.run.smithi029.stdout: Preparing : 1/1 2024-09-16T20:46:52.295 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libcephfs-devel-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T20:46:52.708 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:54 ago on Mon 16 Sep 2024 08:44:58 PM UTC. 2024-09-16T20:46:52.757 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: libcephfs-devel-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T20:46:52.804 INFO:teuthology.orchestra.run.smithi043.stdout:Package rbd-nbd-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-16T20:46:52.851 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-16T20:46:52.852 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-09-16T20:46:52.852 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-16T20:46:52.927 DEBUG:teuthology.parallel:result is None 2024-09-16T20:46:53.010 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libcephfs-devel-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T20:46:53.010 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:46:53.010 INFO:teuthology.orchestra.run.smithi029.stdout:Installed: 2024-09-16T20:46:53.011 INFO:teuthology.orchestra.run.smithi029.stdout: libcephfs-devel-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:46:53.011 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:46:53.011 INFO:teuthology.orchestra.run.smithi029.stdout:Complete! 2024-09-16T20:46:53.244 DEBUG:teuthology.orchestra.run.smithi029:> sudo yum -y install librados2 2024-09-16T20:46:53.853 INFO:teuthology.orchestra.run.smithi029.stdout:Last metadata expiration check: 0:02:07 ago on Mon 16 Sep 2024 08:44:46 PM UTC. 2024-09-16T20:46:53.950 INFO:teuthology.orchestra.run.smithi029.stdout:Package librados2-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-16T20:46:53.995 INFO:teuthology.orchestra.run.smithi029.stdout:Dependencies resolved. 2024-09-16T20:46:53.997 INFO:teuthology.orchestra.run.smithi029.stdout:Nothing to do. 2024-09-16T20:46:53.997 INFO:teuthology.orchestra.run.smithi029.stdout:Complete! 2024-09-16T20:46:54.071 DEBUG:teuthology.orchestra.run.smithi029:> sudo yum -y install librbd1 2024-09-16T20:46:54.678 INFO:teuthology.orchestra.run.smithi029.stdout:Last metadata expiration check: 0:02:08 ago on Mon 16 Sep 2024 08:44:46 PM UTC. 2024-09-16T20:46:54.771 INFO:teuthology.orchestra.run.smithi029.stdout:Package librbd1-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-16T20:46:54.816 INFO:teuthology.orchestra.run.smithi029.stdout:Dependencies resolved. 2024-09-16T20:46:54.817 INFO:teuthology.orchestra.run.smithi029.stdout:Nothing to do. 2024-09-16T20:46:54.818 INFO:teuthology.orchestra.run.smithi029.stdout:Complete! 2024-09-16T20:46:54.896 DEBUG:teuthology.orchestra.run.smithi029:> sudo yum -y install python3-rados 2024-09-16T20:46:55.503 INFO:teuthology.orchestra.run.smithi029.stdout:Last metadata expiration check: 0:02:09 ago on Mon 16 Sep 2024 08:44:46 PM UTC. 2024-09-16T20:46:55.594 INFO:teuthology.orchestra.run.smithi029.stdout:Package python3-rados-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-16T20:46:55.640 INFO:teuthology.orchestra.run.smithi029.stdout:Dependencies resolved. 2024-09-16T20:46:55.641 INFO:teuthology.orchestra.run.smithi029.stdout:Nothing to do. 2024-09-16T20:46:55.641 INFO:teuthology.orchestra.run.smithi029.stdout:Complete! 2024-09-16T20:46:55.722 DEBUG:teuthology.orchestra.run.smithi029:> sudo yum -y install python3-rgw 2024-09-16T20:46:56.329 INFO:teuthology.orchestra.run.smithi029.stdout:Last metadata expiration check: 0:02:10 ago on Mon 16 Sep 2024 08:44:46 PM UTC. 2024-09-16T20:46:56.421 INFO:teuthology.orchestra.run.smithi029.stdout:Package python3-rgw-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-16T20:46:56.467 INFO:teuthology.orchestra.run.smithi029.stdout:Dependencies resolved. 2024-09-16T20:46:56.469 INFO:teuthology.orchestra.run.smithi029.stdout:Nothing to do. 2024-09-16T20:46:56.469 INFO:teuthology.orchestra.run.smithi029.stdout:Complete! 2024-09-16T20:46:56.546 DEBUG:teuthology.orchestra.run.smithi029:> sudo yum -y install python3-cephfs 2024-09-16T20:46:57.152 INFO:teuthology.orchestra.run.smithi029.stdout:Last metadata expiration check: 0:02:11 ago on Mon 16 Sep 2024 08:44:46 PM UTC. 2024-09-16T20:46:57.243 INFO:teuthology.orchestra.run.smithi029.stdout:Package python3-cephfs-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-16T20:46:57.290 INFO:teuthology.orchestra.run.smithi029.stdout:Dependencies resolved. 2024-09-16T20:46:57.291 INFO:teuthology.orchestra.run.smithi029.stdout:Nothing to do. 2024-09-16T20:46:57.291 INFO:teuthology.orchestra.run.smithi029.stdout:Complete! 2024-09-16T20:46:57.363 DEBUG:teuthology.orchestra.run.smithi029:> sudo yum -y install python3-rbd 2024-09-16T20:46:57.973 INFO:teuthology.orchestra.run.smithi029.stdout:Last metadata expiration check: 0:02:11 ago on Mon 16 Sep 2024 08:44:46 PM UTC. 2024-09-16T20:46:58.065 INFO:teuthology.orchestra.run.smithi029.stdout:Package python3-rbd-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-16T20:46:58.111 INFO:teuthology.orchestra.run.smithi029.stdout:Dependencies resolved. 2024-09-16T20:46:58.112 INFO:teuthology.orchestra.run.smithi029.stdout:Nothing to do. 2024-09-16T20:46:58.112 INFO:teuthology.orchestra.run.smithi029.stdout:Complete! 2024-09-16T20:46:58.188 DEBUG:teuthology.orchestra.run.smithi029:> sudo yum -y install rbd-fuse 2024-09-16T20:46:58.805 INFO:teuthology.orchestra.run.smithi029.stdout:Last metadata expiration check: 0:02:12 ago on Mon 16 Sep 2024 08:44:46 PM UTC. 2024-09-16T20:46:58.952 INFO:teuthology.orchestra.run.smithi029.stdout:Dependencies resolved. 2024-09-16T20:46:58.953 INFO:teuthology.orchestra.run.smithi029.stdout:================================================================================ 2024-09-16T20:46:58.953 INFO:teuthology.orchestra.run.smithi029.stdout: Package Architecture Version Repository Size 2024-09-16T20:46:58.953 INFO:teuthology.orchestra.run.smithi029.stdout:================================================================================ 2024-09-16T20:46:58.953 INFO:teuthology.orchestra.run.smithi029.stdout:Installing: 2024-09-16T20:46:58.953 INFO:teuthology.orchestra.run.smithi029.stdout: rbd-fuse x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 86 k 2024-09-16T20:46:58.953 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:46:58.953 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction Summary 2024-09-16T20:46:58.953 INFO:teuthology.orchestra.run.smithi029.stdout:================================================================================ 2024-09-16T20:46:58.953 INFO:teuthology.orchestra.run.smithi029.stdout:Install 1 Package 2024-09-16T20:46:58.953 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:46:58.954 INFO:teuthology.orchestra.run.smithi029.stdout:Total download size: 86 k 2024-09-16T20:46:58.954 INFO:teuthology.orchestra.run.smithi029.stdout:Installed size: 231 k 2024-09-16T20:46:58.954 INFO:teuthology.orchestra.run.smithi029.stdout:Downloading Packages: 2024-09-16T20:46:59.153 INFO:teuthology.orchestra.run.smithi029.stdout:rbd-fuse-19.3.0-4883.g6e8474a0.el9.x86_64.rpm 431 kB/s | 86 kB 00:00 2024-09-16T20:46:59.154 INFO:teuthology.orchestra.run.smithi029.stdout:-------------------------------------------------------------------------------- 2024-09-16T20:46:59.154 INFO:teuthology.orchestra.run.smithi029.stdout:Total 427 kB/s | 86 kB 00:00 2024-09-16T20:46:59.154 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction check 2024-09-16T20:46:59.162 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction check succeeded. 2024-09-16T20:46:59.162 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction test 2024-09-16T20:46:59.230 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction test succeeded. 2024-09-16T20:46:59.231 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction 2024-09-16T20:46:59.383 INFO:teuthology.orchestra.run.smithi029.stdout: Preparing : 1/1 2024-09-16T20:46:59.496 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : rbd-fuse-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T20:46:59.941 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: rbd-fuse-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T20:47:00.285 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : rbd-fuse-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T20:47:00.286 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:47:00.286 INFO:teuthology.orchestra.run.smithi029.stdout:Installed: 2024-09-16T20:47:00.286 INFO:teuthology.orchestra.run.smithi029.stdout: rbd-fuse-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:47:00.286 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:47:00.286 INFO:teuthology.orchestra.run.smithi029.stdout:Complete! 2024-09-16T20:47:00.596 DEBUG:teuthology.orchestra.run.smithi029:> sudo yum -y install rbd-mirror 2024-09-16T20:47:01.206 INFO:teuthology.orchestra.run.smithi029.stdout:Last metadata expiration check: 0:02:15 ago on Mon 16 Sep 2024 08:44:46 PM UTC. 2024-09-16T20:47:01.345 INFO:teuthology.orchestra.run.smithi029.stdout:Dependencies resolved. 2024-09-16T20:47:01.345 INFO:teuthology.orchestra.run.smithi029.stdout:================================================================================ 2024-09-16T20:47:01.345 INFO:teuthology.orchestra.run.smithi029.stdout: Package Arch Version Repo Size 2024-09-16T20:47:01.346 INFO:teuthology.orchestra.run.smithi029.stdout:================================================================================ 2024-09-16T20:47:01.346 INFO:teuthology.orchestra.run.smithi029.stdout:Installing: 2024-09-16T20:47:01.346 INFO:teuthology.orchestra.run.smithi029.stdout: rbd-mirror x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 3.2 M 2024-09-16T20:47:01.346 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:47:01.346 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction Summary 2024-09-16T20:47:01.346 INFO:teuthology.orchestra.run.smithi029.stdout:================================================================================ 2024-09-16T20:47:01.346 INFO:teuthology.orchestra.run.smithi029.stdout:Install 1 Package 2024-09-16T20:47:01.346 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:47:01.346 INFO:teuthology.orchestra.run.smithi029.stdout:Total download size: 3.2 M 2024-09-16T20:47:01.346 INFO:teuthology.orchestra.run.smithi029.stdout:Installed size: 14 M 2024-09-16T20:47:01.346 INFO:teuthology.orchestra.run.smithi029.stdout:Downloading Packages: 2024-09-16T20:47:01.995 INFO:teuthology.orchestra.run.smithi029.stdout:rbd-mirror-19.3.0-4883.g6e8474a0.el9.x86_64.rpm 4.9 MB/s | 3.2 MB 00:00 2024-09-16T20:47:01.996 INFO:teuthology.orchestra.run.smithi029.stdout:-------------------------------------------------------------------------------- 2024-09-16T20:47:01.996 INFO:teuthology.orchestra.run.smithi029.stdout:Total 4.9 MB/s | 3.2 MB 00:00 2024-09-16T20:47:01.997 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction check 2024-09-16T20:47:02.008 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction check succeeded. 2024-09-16T20:47:02.008 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction test 2024-09-16T20:47:02.097 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction test succeeded. 2024-09-16T20:47:02.097 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction 2024-09-16T20:47:02.564 INFO:teuthology.orchestra.run.smithi029.stdout: Preparing : 1/1 2024-09-16T20:47:02.632 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : rbd-mirror-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T20:47:02.659 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T20:47:02.659 INFO:teuthology.orchestra.run.smithi029.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T20:47:02.659 INFO:teuthology.orchestra.run.smithi029.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-16T20:47:02.659 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T20:47:02.659 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T20:47:02.659 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:47:03.959 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : rbd-mirror-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T20:47:03.959 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:47:03.959 INFO:teuthology.orchestra.run.smithi029.stdout:Installed: 2024-09-16T20:47:03.959 INFO:teuthology.orchestra.run.smithi029.stdout: rbd-mirror-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:47:03.959 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:47:03.959 INFO:teuthology.orchestra.run.smithi029.stdout:Complete! 2024-09-16T20:47:04.260 DEBUG:teuthology.orchestra.run.smithi029:> sudo yum -y install rbd-nbd 2024-09-16T20:47:04.874 INFO:teuthology.orchestra.run.smithi029.stdout:Last metadata expiration check: 0:02:18 ago on Mon 16 Sep 2024 08:44:46 PM UTC. 2024-09-16T20:47:05.016 INFO:teuthology.orchestra.run.smithi029.stdout:Dependencies resolved. 2024-09-16T20:47:05.016 INFO:teuthology.orchestra.run.smithi029.stdout:================================================================================ 2024-09-16T20:47:05.017 INFO:teuthology.orchestra.run.smithi029.stdout: Package Architecture Version Repository Size 2024-09-16T20:47:05.017 INFO:teuthology.orchestra.run.smithi029.stdout:================================================================================ 2024-09-16T20:47:05.017 INFO:teuthology.orchestra.run.smithi029.stdout:Installing: 2024-09-16T20:47:05.017 INFO:teuthology.orchestra.run.smithi029.stdout: rbd-nbd x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 170 k 2024-09-16T20:47:05.017 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:47:05.017 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction Summary 2024-09-16T20:47:05.017 INFO:teuthology.orchestra.run.smithi029.stdout:================================================================================ 2024-09-16T20:47:05.017 INFO:teuthology.orchestra.run.smithi029.stdout:Install 1 Package 2024-09-16T20:47:05.017 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:47:05.017 INFO:teuthology.orchestra.run.smithi029.stdout:Total download size: 170 k 2024-09-16T20:47:05.017 INFO:teuthology.orchestra.run.smithi029.stdout:Installed size: 490 k 2024-09-16T20:47:05.017 INFO:teuthology.orchestra.run.smithi029.stdout:Downloading Packages: 2024-09-16T20:47:05.231 INFO:teuthology.orchestra.run.smithi029.stdout:rbd-nbd-19.3.0-4883.g6e8474a0.el9.x86_64.rpm 798 kB/s | 170 kB 00:00 2024-09-16T20:47:05.231 INFO:teuthology.orchestra.run.smithi029.stdout:-------------------------------------------------------------------------------- 2024-09-16T20:47:05.232 INFO:teuthology.orchestra.run.smithi029.stdout:Total 793 kB/s | 170 kB 00:00 2024-09-16T20:47:05.232 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction check 2024-09-16T20:47:05.240 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction check succeeded. 2024-09-16T20:47:05.240 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction test 2024-09-16T20:47:05.309 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction test succeeded. 2024-09-16T20:47:05.309 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction 2024-09-16T20:47:05.468 INFO:teuthology.orchestra.run.smithi029.stdout: Preparing : 1/1 2024-09-16T20:47:05.574 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : rbd-nbd-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T20:47:06.079 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: rbd-nbd-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T20:47:06.376 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : rbd-nbd-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T20:47:06.376 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:47:06.376 INFO:teuthology.orchestra.run.smithi029.stdout:Installed: 2024-09-16T20:47:06.376 INFO:teuthology.orchestra.run.smithi029.stdout: rbd-nbd-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T20:47:06.377 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:47:06.377 INFO:teuthology.orchestra.run.smithi029.stdout:Complete! 2024-09-16T20:47:06.690 DEBUG:teuthology.orchestra.run.smithi029:> sudo yum -y install rbd-nbd 2024-09-16T20:47:07.298 INFO:teuthology.orchestra.run.smithi029.stdout:Last metadata expiration check: 0:02:21 ago on Mon 16 Sep 2024 08:44:46 PM UTC. 2024-09-16T20:47:07.393 INFO:teuthology.orchestra.run.smithi029.stdout:Package rbd-nbd-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-16T20:47:07.440 INFO:teuthology.orchestra.run.smithi029.stdout:Dependencies resolved. 2024-09-16T20:47:07.441 INFO:teuthology.orchestra.run.smithi029.stdout:Nothing to do. 2024-09-16T20:47:07.441 INFO:teuthology.orchestra.run.smithi029.stdout:Complete! 2024-09-16T20:47:07.526 DEBUG:teuthology.parallel:result is None 2024-09-16T20:47:07.527 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6e8474a02f2b86353394076ab80c6f50ea28e361 2024-09-16T20:47:07.675 DEBUG:teuthology.orchestra.run.smithi029:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-16T20:47:07.697 INFO:teuthology.orchestra.run.smithi029.stdout:19.3.0-4883.g6e8474a0.el9 2024-09-16T20:47:07.698 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4883.g6e8474a0.el9 2024-09-16T20:47:07.698 INFO:teuthology.task.install:The correct ceph version 19.3.0-4883.g6e8474a0 is installed. 2024-09-16T20:47:07.700 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6e8474a02f2b86353394076ab80c6f50ea28e361 2024-09-16T20:47:07.843 DEBUG:teuthology.orchestra.run.smithi043:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-16T20:47:07.869 INFO:teuthology.orchestra.run.smithi043.stdout:19.3.0-4883.g6e8474a0.el9 2024-09-16T20:47:07.869 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4883.g6e8474a0.el9 2024-09-16T20:47:07.870 INFO:teuthology.task.install:The correct ceph version 19.3.0-4883.g6e8474a0 is installed. 2024-09-16T20:47:07.871 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6e8474a02f2b86353394076ab80c6f50ea28e361 2024-09-16T20:47:08.014 DEBUG:teuthology.orchestra.run.smithi139:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-16T20:47:08.039 INFO:teuthology.orchestra.run.smithi139.stdout:19.3.0-4883.g6e8474a0.el9 2024-09-16T20:47:08.040 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4883.g6e8474a0.el9 2024-09-16T20:47:08.040 INFO:teuthology.task.install:The correct ceph version 19.3.0-4883.g6e8474a0 is installed. 2024-09-16T20:47:08.041 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-16T20:47:08.042 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-09-16T20:47:08.042 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T20:47:08.069 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-16T20:47:08.069 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T20:47:08.101 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-16T20:47:08.101 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T20:47:08.129 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-16T20:47:08.130 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-09-16T20:47:08.130 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T20:47:08.159 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T20:47:08.227 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-16T20:47:08.227 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T20:47:08.258 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T20:47:08.329 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-16T20:47:08.329 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T20:47:08.358 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T20:47:08.429 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-16T20:47:08.429 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-09-16T20:47:08.429 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T20:47:08.457 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T20:47:08.525 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-16T20:47:08.525 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T20:47:08.556 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T20:47:08.627 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-16T20:47:08.627 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T20:47:08.653 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T20:47:08.718 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-16T20:47:08.718 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-09-16T20:47:08.718 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T20:47:08.746 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T20:47:08.814 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-16T20:47:08.814 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T20:47:08.845 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T20:47:08.916 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-16T20:47:08.916 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T20:47:08.941 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T20:47:09.006 INFO:teuthology.run_tasks:Running task ceph... 2024-09-16T20:47:09.083 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-16T20:47:09.083 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod 777 /var/log/ceph 2024-09-16T20:47:09.086 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 777 /var/log/ceph 2024-09-16T20:47:09.089 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 777 /var/log/ceph 2024-09-16T20:47:09.116 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-16T20:47:09.117 DEBUG:teuthology.orchestra.run.smithi029:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-16T20:47:09.155 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-16T20:47:09.159 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-16T20:47:09.185 INFO:tasks.ceph:Creating extra log directories... 2024-09-16T20:47:09.186 DEBUG:teuthology.orchestra.run.smithi029:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-16T20:47:09.225 DEBUG:teuthology.orchestra.run.smithi043:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-16T20:47:09.228 DEBUG:teuthology.orchestra.run.smithi139:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-16T20:47:09.262 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-16T20:47:09.263 INFO:tasks.ceph:config {'conf': {'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-16T20:47:09.263 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/dis-2024-09-16_19:45:14-rbd-wip-dis-testing-distro-default-smithi/7907507', 'branch': 'wip-dis-testing', 'description': 'rbd/device/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-bitmap supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nbd}', 'email': 'idryomov@gmail.com', 'first_in_suite': False, 'job_id': '7907507', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'dis-2024-09-16_19:45:14-rbd-wip-dis-testing-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-dis-testing'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': '6e8474a02f2b86353394076ab80c6f50ea28e361'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'extra_packages': ['rbd-nbd'], 'flavor': 'default', 'sha1': '6e8474a02f2b86353394076ab80c6f50ea28e361'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-dis-testing', 'sha1': '6e8474a02f2b86353394076ab80c6f50ea28e361'}}, 'owner': 'scheduled_dis@teuthology', 'priority': 70, 'repo': 'https://github.com/ceph/ceph-ci', 'roles': [['mon.a', 'mon.c', 'osd.0', 'osd.1', 'osd.2'], ['mon.b', 'mgr.x', 'osd.3', 'osd.4', 'osd.5'], ['client.0']], 'seed': 2961, 'sha1': '6e8474a02f2b86353394076ab80c6f50ea28e361', 'sleep_before_teardown': 0, 'subset': '66/100', 'suite': 'rbd', 'suite_branch': 'wip-dis-testing', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_6e8474a02f2b86353394076ab80c6f50ea28e361/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci', 'suite_sha1': '6e8474a02f2b86353394076ab80c6f50ea28e361', 'targets': {'smithi029.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP3WniueVCePjB674hgS/OiOwtALk1EjrLxndGnr52FplnqEJL+LFsN++t6kfeeYy8zfkVDXJLqzdonjfYBEyNw=', 'smithi043.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDk1jDLkQEAOSm5cIMX2m6pEXmEeCa+Nnm2oBQ0gvGc/tQctf4vmeQ4WVYa0C45ipJqEFBCB6BrDSs4fvkVmMKs=', 'smithi139.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMqbK+ZfDBSX6DiaHHbJBA9OWjwpcBU86wW0vkxexx3xjQk26MC//JrenBONHIhXcoMihK7NN+/gHMFPJn2KCzs='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'thrashosds': {'timeout': 1200}}, {'rbd_fsx': {'clients': ['client.0'], 'holebdy': 512, 'nbd': True, 'ops': 6000, 'punch_holes': True, 'readbdy': 512, 'truncbdy': 512, 'writebdy': 512}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-16_19:45:14', 'tube': 'smithi', 'user': 'dis', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-16T20:47:09.263 DEBUG:teuthology.orchestra.run.smithi029:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-16T20:47:09.297 DEBUG:teuthology.orchestra.run.smithi043:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-16T20:47:09.299 DEBUG:teuthology.orchestra.run.smithi139:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-16T20:47:09.314 DEBUG:teuthology.orchestra.run.smithi029:> sudo install -d -m0777 -- /var/run/ceph 2024-09-16T20:47:09.355 DEBUG:teuthology.orchestra.run.smithi043:> sudo install -d -m0777 -- /var/run/ceph 2024-09-16T20:47:09.357 DEBUG:teuthology.orchestra.run.smithi139:> sudo install -d -m0777 -- /var/run/ceph 2024-09-16T20:47:09.381 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-09-16T20:47:09.381 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T20:47:09.436 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T20:47:09.436 DEBUG:teuthology.orchestra.run.smithi029:> stat /dev/vg_nvme/lv_1 2024-09-16T20:47:09.493 INFO:teuthology.orchestra.run.smithi029.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T20:47:09.493 INFO:teuthology.orchestra.run.smithi029.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T20:47:09.493 INFO:teuthology.orchestra.run.smithi029.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-09-16T20:47:09.493 INFO:teuthology.orchestra.run.smithi029.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T20:47:09.493 INFO:teuthology.orchestra.run.smithi029.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T20:47:09.493 INFO:teuthology.orchestra.run.smithi029.stdout:Access: 2024-09-16 20:47:02.926908253 +0000 2024-09-16T20:47:09.493 INFO:teuthology.orchestra.run.smithi029.stdout:Modify: 2024-09-16 20:43:56.908680647 +0000 2024-09-16T20:47:09.493 INFO:teuthology.orchestra.run.smithi029.stdout:Change: 2024-09-16 20:43:56.908680647 +0000 2024-09-16T20:47:09.493 INFO:teuthology.orchestra.run.smithi029.stdout: Birth: 2024-09-16 20:43:56.908680647 +0000 2024-09-16T20:47:09.493 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T20:47:09.559 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records in 2024-09-16T20:47:09.559 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records out 2024-09-16T20:47:09.559 INFO:teuthology.orchestra.run.smithi029.stderr:512 bytes copied, 0.000108001 s, 4.7 MB/s 2024-09-16T20:47:09.560 DEBUG:teuthology.orchestra.run.smithi029:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T20:47:09.618 DEBUG:teuthology.orchestra.run.smithi029:> stat /dev/vg_nvme/lv_2 2024-09-16T20:47:09.674 INFO:teuthology.orchestra.run.smithi029.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T20:47:09.674 INFO:teuthology.orchestra.run.smithi029.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T20:47:09.675 INFO:teuthology.orchestra.run.smithi029.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-16T20:47:09.675 INFO:teuthology.orchestra.run.smithi029.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T20:47:09.675 INFO:teuthology.orchestra.run.smithi029.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T20:47:09.675 INFO:teuthology.orchestra.run.smithi029.stdout:Access: 2024-09-16 20:47:02.926908253 +0000 2024-09-16T20:47:09.675 INFO:teuthology.orchestra.run.smithi029.stdout:Modify: 2024-09-16 20:43:57.232675446 +0000 2024-09-16T20:47:09.675 INFO:teuthology.orchestra.run.smithi029.stdout:Change: 2024-09-16 20:43:57.232675446 +0000 2024-09-16T20:47:09.675 INFO:teuthology.orchestra.run.smithi029.stdout: Birth: 2024-09-16 20:43:57.232675446 +0000 2024-09-16T20:47:09.675 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T20:47:09.740 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records in 2024-09-16T20:47:09.740 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records out 2024-09-16T20:47:09.740 INFO:teuthology.orchestra.run.smithi029.stderr:512 bytes copied, 0.000105524 s, 4.9 MB/s 2024-09-16T20:47:09.741 DEBUG:teuthology.orchestra.run.smithi029:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T20:47:09.799 DEBUG:teuthology.orchestra.run.smithi029:> stat /dev/vg_nvme/lv_3 2024-09-16T20:47:09.855 INFO:teuthology.orchestra.run.smithi029.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T20:47:09.856 INFO:teuthology.orchestra.run.smithi029.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T20:47:09.856 INFO:teuthology.orchestra.run.smithi029.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-16T20:47:09.856 INFO:teuthology.orchestra.run.smithi029.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T20:47:09.856 INFO:teuthology.orchestra.run.smithi029.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T20:47:09.856 INFO:teuthology.orchestra.run.smithi029.stdout:Access: 2024-09-16 20:47:02.926908253 +0000 2024-09-16T20:47:09.856 INFO:teuthology.orchestra.run.smithi029.stdout:Modify: 2024-09-16 20:43:57.559670196 +0000 2024-09-16T20:47:09.856 INFO:teuthology.orchestra.run.smithi029.stdout:Change: 2024-09-16 20:43:57.559670196 +0000 2024-09-16T20:47:09.856 INFO:teuthology.orchestra.run.smithi029.stdout: Birth: 2024-09-16 20:43:57.559670196 +0000 2024-09-16T20:47:09.856 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T20:47:09.920 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records in 2024-09-16T20:47:09.920 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records out 2024-09-16T20:47:09.920 INFO:teuthology.orchestra.run.smithi029.stderr:512 bytes copied, 9.6127e-05 s, 5.3 MB/s 2024-09-16T20:47:09.921 DEBUG:teuthology.orchestra.run.smithi029:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T20:47:09.978 DEBUG:teuthology.orchestra.run.smithi029:> stat /dev/vg_nvme/lv_4 2024-09-16T20:47:10.034 INFO:teuthology.orchestra.run.smithi029.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T20:47:10.034 INFO:teuthology.orchestra.run.smithi029.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T20:47:10.035 INFO:teuthology.orchestra.run.smithi029.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-16T20:47:10.035 INFO:teuthology.orchestra.run.smithi029.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T20:47:10.035 INFO:teuthology.orchestra.run.smithi029.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T20:47:10.035 INFO:teuthology.orchestra.run.smithi029.stdout:Access: 2024-09-16 20:47:02.927908238 +0000 2024-09-16T20:47:10.035 INFO:teuthology.orchestra.run.smithi029.stdout:Modify: 2024-09-16 20:43:57.880665043 +0000 2024-09-16T20:47:10.035 INFO:teuthology.orchestra.run.smithi029.stdout:Change: 2024-09-16 20:43:57.880665043 +0000 2024-09-16T20:47:10.035 INFO:teuthology.orchestra.run.smithi029.stdout: Birth: 2024-09-16 20:43:57.880665043 +0000 2024-09-16T20:47:10.035 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T20:47:10.098 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records in 2024-09-16T20:47:10.098 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records out 2024-09-16T20:47:10.098 INFO:teuthology.orchestra.run.smithi029.stderr:512 bytes copied, 9.5665e-05 s, 5.4 MB/s 2024-09-16T20:47:10.099 DEBUG:teuthology.orchestra.run.smithi029:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T20:47:10.155 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-16T20:47:10.156 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-16T20:47:10.156 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T20:47:10.172 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T20:47:10.172 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_1 2024-09-16T20:47:10.228 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T20:47:10.228 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T20:47:10.228 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-16T20:47:10.228 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T20:47:10.228 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T20:47:10.228 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-09-16 20:46:48.708871663 +0000 2024-09-16T20:47:10.228 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-09-16 20:43:49.683289730 +0000 2024-09-16T20:47:10.228 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-09-16 20:43:49.683289730 +0000 2024-09-16T20:47:10.228 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-09-16 20:43:49.683289730 +0000 2024-09-16T20:47:10.229 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T20:47:10.292 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-09-16T20:47:10.292 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-09-16T20:47:10.292 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 9.9556e-05 s, 5.1 MB/s 2024-09-16T20:47:10.293 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T20:47:10.350 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_2 2024-09-16T20:47:10.406 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T20:47:10.406 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T20:47:10.406 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-16T20:47:10.406 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T20:47:10.406 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T20:47:10.406 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-09-16 20:46:48.708871663 +0000 2024-09-16T20:47:10.406 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-09-16 20:43:50.001286297 +0000 2024-09-16T20:47:10.406 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-09-16 20:43:50.001286297 +0000 2024-09-16T20:47:10.406 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-09-16 20:43:50.001286297 +0000 2024-09-16T20:47:10.406 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T20:47:10.470 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-09-16T20:47:10.470 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-09-16T20:47:10.470 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 9.9217e-05 s, 5.2 MB/s 2024-09-16T20:47:10.471 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T20:47:10.528 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_3 2024-09-16T20:47:10.584 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T20:47:10.584 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T20:47:10.584 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-16T20:47:10.584 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T20:47:10.584 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T20:47:10.584 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-09-16 20:46:48.709871651 +0000 2024-09-16T20:47:10.584 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-09-16 20:43:50.312282939 +0000 2024-09-16T20:47:10.585 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-09-16 20:43:50.312282939 +0000 2024-09-16T20:47:10.585 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-09-16 20:43:50.312282939 +0000 2024-09-16T20:47:10.585 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T20:47:10.648 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-09-16T20:47:10.648 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-09-16T20:47:10.648 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000100487 s, 5.1 MB/s 2024-09-16T20:47:10.649 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T20:47:10.706 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_4 2024-09-16T20:47:10.762 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T20:47:10.762 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T20:47:10.763 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-16T20:47:10.763 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T20:47:10.763 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T20:47:10.763 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-09-16 20:46:48.710871640 +0000 2024-09-16T20:47:10.763 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-09-16 20:43:50.641279386 +0000 2024-09-16T20:47:10.763 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-09-16 20:43:50.641279386 +0000 2024-09-16T20:47:10.763 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-09-16 20:43:50.641279386 +0000 2024-09-16T20:47:10.763 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T20:47:10.826 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-09-16T20:47:10.827 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-09-16T20:47:10.827 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 8.114e-05 s, 6.3 MB/s 2024-09-16T20:47:10.828 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T20:47:10.885 INFO:tasks.ceph:osd dev map: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-16T20:47:10.885 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi029.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi043.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'}} 2024-09-16T20:47:10.885 INFO:tasks.ceph:Generating config... 2024-09-16T20:47:10.886 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-16T20:47:10.886 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-09-16T20:47:10.886 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-16T20:47:10.886 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-16T20:47:10.886 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-16T20:47:10.886 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-16T20:47:10.886 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-16T20:47:10.886 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-16T20:47:10.887 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-16T20:47:10.887 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-16T20:47:10.887 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-09-16T20:47:10.887 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-16T20:47:10.887 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-16T20:47:10.887 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-16T20:47:10.887 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-16T20:47:10.887 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-16T20:47:10.887 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-16T20:47:10.887 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-16T20:47:10.887 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-16T20:47:10.887 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-16T20:47:10.887 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-16T20:47:10.887 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-16T20:47:10.888 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-16T20:47:10.888 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-16T20:47:10.888 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-16T20:47:10.888 INFO:tasks.ceph:Setting up mon.a... 2024-09-16T20:47:10.888 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-16T20:47:10.939 INFO:teuthology.orchestra.run.smithi029.stdout:creating /etc/ceph/ceph.keyring 2024-09-16T20:47:10.941 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-16T20:47:10.988 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-16T20:47:11.014 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.29'), ('mon.c', '[v2:172.21.15.29:3301,v1:172.21.15.29:6790]'), ('mon.b', '172.21.15.43')] 2024-09-16T20:47:11.014 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.29,[v2:172.21.15.29:3301,v1:172.21.15.29:6790],172.21.15.43', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-09-16T20:47:11.015 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-09-16T20:47:11.016 DEBUG:teuthology.orchestra.run.smithi029:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-16T20:47:11.071 DEBUG:teuthology.orchestra.run.smithi029:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.29 --addv c '[v2:172.21.15.29:3301,v1:172.21.15.29:6790]' --add b 172.21.15.43 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-16T20:47:11.160 INFO:teuthology.orchestra.run.smithi029.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-16T20:47:11.160 INFO:teuthology.orchestra.run.smithi029.stdout:monmaptool: generated fsid e28af4a7-5fcd-4de8-8f77-a2b575361bb1 2024-09-16T20:47:11.160 INFO:teuthology.orchestra.run.smithi029.stdout:setting min_mon_release = quincy 2024-09-16T20:47:11.160 INFO:teuthology.orchestra.run.smithi029.stdout:epoch 0 2024-09-16T20:47:11.160 INFO:teuthology.orchestra.run.smithi029.stdout:fsid e28af4a7-5fcd-4de8-8f77-a2b575361bb1 2024-09-16T20:47:11.160 INFO:teuthology.orchestra.run.smithi029.stdout:last_changed 2024-09-16T20:47:11.159779+0000 2024-09-16T20:47:11.160 INFO:teuthology.orchestra.run.smithi029.stdout:created 2024-09-16T20:47:11.159779+0000 2024-09-16T20:47:11.161 INFO:teuthology.orchestra.run.smithi029.stdout:min_mon_release 17 (quincy) 2024-09-16T20:47:11.161 INFO:teuthology.orchestra.run.smithi029.stdout:election_strategy: 1 2024-09-16T20:47:11.161 INFO:teuthology.orchestra.run.smithi029.stdout:0: [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] mon.a 2024-09-16T20:47:11.161 INFO:teuthology.orchestra.run.smithi029.stdout:1: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.b 2024-09-16T20:47:11.161 INFO:teuthology.orchestra.run.smithi029.stdout:2: [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] mon.c 2024-09-16T20:47:11.161 INFO:teuthology.orchestra.run.smithi029.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-16T20:47:11.162 DEBUG:teuthology.orchestra.run.smithi029:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-16T20:47:11.178 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID e28af4a7-5fcd-4de8-8f77-a2b575361bb1... 2024-09-16T20:47:11.178 DEBUG:teuthology.orchestra.run.smithi029:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-16T20:47:11.221 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-16T20:47:11.223 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-16T20:47:11.262 INFO:teuthology.orchestra.run.smithi029.stdout:[global] 2024-09-16T20:47:11.262 INFO:teuthology.orchestra.run.smithi029.stdout: chdir = "" 2024-09-16T20:47:11.262 INFO:teuthology.orchestra.run.smithi029.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-16T20:47:11.262 INFO:teuthology.orchestra.run.smithi029.stdout: auth supported = cephx 2024-09-16T20:47:11.262 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:47:11.262 INFO:teuthology.orchestra.run.smithi029.stdout: filestore xattr use omap = true 2024-09-16T20:47:11.262 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:47:11.262 INFO:teuthology.orchestra.run.smithi029.stdout: mon clock drift allowed = 1.000 2024-09-16T20:47:11.262 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:47:11.263 INFO:teuthology.orchestra.run.smithi029.stdout: osd crush chooseleaf type = 0 2024-09-16T20:47:11.263 INFO:teuthology.orchestra.run.smithi029.stdout: auth debug = true 2024-09-16T20:47:11.263 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:47:11.263 INFO:teuthology.orchestra.run.smithi029.stdout: ms die on old message = true 2024-09-16T20:47:11.263 INFO:teuthology.orchestra.run.smithi029.stdout: ms die on bug = true 2024-09-16T20:47:11.263 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:47:11.263 INFO:teuthology.orchestra.run.smithi029.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-16T20:47:11.263 INFO:teuthology.orchestra.run.smithi029.stdout: mon pg warn max object skew = 0 2024-09-16T20:47:11.263 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:47:11.263 INFO:teuthology.orchestra.run.smithi029.stdout: # disable pg_autoscaler by default for new pools 2024-09-16T20:47:11.263 INFO:teuthology.orchestra.run.smithi029.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-16T20:47:11.263 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:47:11.263 INFO:teuthology.orchestra.run.smithi029.stdout: osd pool default size = 2 2024-09-16T20:47:11.263 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:47:11.263 INFO:teuthology.orchestra.run.smithi029.stdout: mon osd allow primary affinity = true 2024-09-16T20:47:11.264 INFO:teuthology.orchestra.run.smithi029.stdout: mon osd allow pg remap = true 2024-09-16T20:47:11.264 INFO:teuthology.orchestra.run.smithi029.stdout: mon warn on legacy crush tunables = false 2024-09-16T20:47:11.264 INFO:teuthology.orchestra.run.smithi029.stdout: mon warn on crush straw calc version zero = false 2024-09-16T20:47:11.264 INFO:teuthology.orchestra.run.smithi029.stdout: mon warn on no sortbitwise = false 2024-09-16T20:47:11.264 INFO:teuthology.orchestra.run.smithi029.stdout: mon warn on osd down out interval zero = false 2024-09-16T20:47:11.264 INFO:teuthology.orchestra.run.smithi029.stdout: mon warn on too few osds = false 2024-09-16T20:47:11.264 INFO:teuthology.orchestra.run.smithi029.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-16T20:47:11.264 INFO:teuthology.orchestra.run.smithi029.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-16T20:47:11.264 INFO:teuthology.orchestra.run.smithi029.stdout: mon_allow_pool_size_one = true 2024-09-16T20:47:11.264 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:47:11.264 INFO:teuthology.orchestra.run.smithi029.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-16T20:47:11.264 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:47:11.264 INFO:teuthology.orchestra.run.smithi029.stdout: osd default data pool replay window = 5 2024-09-16T20:47:11.265 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:47:11.265 INFO:teuthology.orchestra.run.smithi029.stdout: mon allow pool delete = true 2024-09-16T20:47:11.265 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:47:11.265 INFO:teuthology.orchestra.run.smithi029.stdout: mon cluster log file level = debug 2024-09-16T20:47:11.265 INFO:teuthology.orchestra.run.smithi029.stdout: debug asserts on shutdown = true 2024-09-16T20:47:11.265 INFO:teuthology.orchestra.run.smithi029.stdout: mon health detail to clog = false 2024-09-16T20:47:11.265 INFO:teuthology.orchestra.run.smithi029.stdout: mon host = "172.21.15.29,[v2:172.21.15.29:3301,v1:172.21.15.29:6790],172.21.15.43" 2024-09-16T20:47:11.265 INFO:teuthology.orchestra.run.smithi029.stdout: mon client directed command retry = 5 2024-09-16T20:47:11.265 INFO:teuthology.orchestra.run.smithi029.stdout: mon warn on pool no app = False 2024-09-16T20:47:11.265 INFO:teuthology.orchestra.run.smithi029.stdout: ms inject socket failures = 5000 2024-09-16T20:47:11.265 INFO:teuthology.orchestra.run.smithi029.stdout: fsid = e28af4a7-5fcd-4de8-8f77-a2b575361bb1 2024-09-16T20:47:11.265 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:47:11.265 INFO:teuthology.orchestra.run.smithi029.stdout:[osd] 2024-09-16T20:47:11.266 INFO:teuthology.orchestra.run.smithi029.stdout: osd journal size = 100 2024-09-16T20:47:11.266 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:47:11.266 INFO:teuthology.orchestra.run.smithi029.stdout: osd scrub load threshold = 5.0 2024-09-16T20:47:11.266 INFO:teuthology.orchestra.run.smithi029.stdout: osd scrub max interval = 600 2024-09-16T20:47:11.266 INFO:teuthology.orchestra.run.smithi029.stdout: osd mclock profile = high_recovery_ops 2024-09-16T20:47:11.266 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:47:11.266 INFO:teuthology.orchestra.run.smithi029.stdout: osd recover clone overlap = true 2024-09-16T20:47:11.266 INFO:teuthology.orchestra.run.smithi029.stdout: osd recovery max chunk = 1048576 2024-09-16T20:47:11.266 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:47:11.266 INFO:teuthology.orchestra.run.smithi029.stdout: osd debug shutdown = true 2024-09-16T20:47:11.266 INFO:teuthology.orchestra.run.smithi029.stdout: osd debug op order = true 2024-09-16T20:47:11.266 INFO:teuthology.orchestra.run.smithi029.stdout: osd debug verify stray on activate = true 2024-09-16T20:47:11.266 INFO:teuthology.orchestra.run.smithi029.stdout: osd debug trim objects = true 2024-09-16T20:47:11.267 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:47:11.267 INFO:teuthology.orchestra.run.smithi029.stdout: osd open classes on start = true 2024-09-16T20:47:11.267 INFO:teuthology.orchestra.run.smithi029.stdout: osd debug pg log writeout = true 2024-09-16T20:47:11.267 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:47:11.267 INFO:teuthology.orchestra.run.smithi029.stdout: osd deep scrub update digest min age = 30 2024-09-16T20:47:11.267 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:47:11.267 INFO:teuthology.orchestra.run.smithi029.stdout: osd map max advance = 10 2024-09-16T20:47:11.267 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:47:11.267 INFO:teuthology.orchestra.run.smithi029.stdout: journal zero on create = true 2024-09-16T20:47:11.267 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:47:11.267 INFO:teuthology.orchestra.run.smithi029.stdout: filestore ondisk finisher threads = 3 2024-09-16T20:47:11.267 INFO:teuthology.orchestra.run.smithi029.stdout: filestore apply finisher threads = 3 2024-09-16T20:47:11.267 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:47:11.267 INFO:teuthology.orchestra.run.smithi029.stdout: bdev debug aio = true 2024-09-16T20:47:11.268 INFO:teuthology.orchestra.run.smithi029.stdout: osd debug misdirected ops = true 2024-09-16T20:47:11.268 INFO:teuthology.orchestra.run.smithi029.stdout: bdev async discard = True 2024-09-16T20:47:11.268 INFO:teuthology.orchestra.run.smithi029.stdout: bdev enable discard = True 2024-09-16T20:47:11.268 INFO:teuthology.orchestra.run.smithi029.stdout: bluestore allocator = bitmap 2024-09-16T20:47:11.268 INFO:teuthology.orchestra.run.smithi029.stdout: bluestore block size = 96636764160 2024-09-16T20:47:11.268 INFO:teuthology.orchestra.run.smithi029.stdout: bluestore fsck on mount = True 2024-09-16T20:47:11.268 INFO:teuthology.orchestra.run.smithi029.stdout: debug bluefs = 1/20 2024-09-16T20:47:11.268 INFO:teuthology.orchestra.run.smithi029.stdout: debug bluestore = 1/20 2024-09-16T20:47:11.268 INFO:teuthology.orchestra.run.smithi029.stdout: debug ms = 1 2024-09-16T20:47:11.268 INFO:teuthology.orchestra.run.smithi029.stdout: debug osd = 20 2024-09-16T20:47:11.268 INFO:teuthology.orchestra.run.smithi029.stdout: debug rocksdb = 4/10 2024-09-16T20:47:11.268 INFO:teuthology.orchestra.run.smithi029.stdout: mon osd backfillfull_ratio = 0.85 2024-09-16T20:47:11.268 INFO:teuthology.orchestra.run.smithi029.stdout: mon osd full ratio = 0.9 2024-09-16T20:47:11.269 INFO:teuthology.orchestra.run.smithi029.stdout: mon osd nearfull ratio = 0.8 2024-09-16T20:47:11.269 INFO:teuthology.orchestra.run.smithi029.stdout: osd blocked scrub grace period = 3600 2024-09-16T20:47:11.269 INFO:teuthology.orchestra.run.smithi029.stdout: osd failsafe full ratio = 0.95 2024-09-16T20:47:11.269 INFO:teuthology.orchestra.run.smithi029.stdout: osd max markdown count = 1000 2024-09-16T20:47:11.269 INFO:teuthology.orchestra.run.smithi029.stdout: osd objectstore = bluestore 2024-09-16T20:47:11.269 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:47:11.269 INFO:teuthology.orchestra.run.smithi029.stdout:[mgr] 2024-09-16T20:47:11.269 INFO:teuthology.orchestra.run.smithi029.stdout: debug ms = 1 2024-09-16T20:47:11.269 INFO:teuthology.orchestra.run.smithi029.stdout: debug mgr = 20 2024-09-16T20:47:11.269 INFO:teuthology.orchestra.run.smithi029.stdout: debug mon = 20 2024-09-16T20:47:11.269 INFO:teuthology.orchestra.run.smithi029.stdout: debug auth = 20 2024-09-16T20:47:11.270 INFO:teuthology.orchestra.run.smithi029.stdout: mon reweight min pgs per osd = 4 2024-09-16T20:47:11.270 INFO:teuthology.orchestra.run.smithi029.stdout: mon reweight min bytes per osd = 10 2024-09-16T20:47:11.270 INFO:teuthology.orchestra.run.smithi029.stdout: mgr/telemetry/nag = false 2024-09-16T20:47:11.270 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:47:11.270 INFO:teuthology.orchestra.run.smithi029.stdout:[mon] 2024-09-16T20:47:11.270 INFO:teuthology.orchestra.run.smithi029.stdout: debug ms = 1 2024-09-16T20:47:11.270 INFO:teuthology.orchestra.run.smithi029.stdout: debug mon = 20 2024-09-16T20:47:11.270 INFO:teuthology.orchestra.run.smithi029.stdout: debug paxos = 20 2024-09-16T20:47:11.270 INFO:teuthology.orchestra.run.smithi029.stdout: debug auth = 20 2024-09-16T20:47:11.270 INFO:teuthology.orchestra.run.smithi029.stdout: mon data avail warn = 5 2024-09-16T20:47:11.270 INFO:teuthology.orchestra.run.smithi029.stdout: mon mgr mkfs grace = 240 2024-09-16T20:47:11.271 INFO:teuthology.orchestra.run.smithi029.stdout: mon reweight min pgs per osd = 4 2024-09-16T20:47:11.271 INFO:teuthology.orchestra.run.smithi029.stdout: mon osd reporter subtree level = osd 2024-09-16T20:47:11.271 INFO:teuthology.orchestra.run.smithi029.stdout: mon osd prime pg temp = true 2024-09-16T20:47:11.271 INFO:teuthology.orchestra.run.smithi029.stdout: mon reweight min bytes per osd = 10 2024-09-16T20:47:11.271 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:47:11.271 INFO:teuthology.orchestra.run.smithi029.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-16T20:47:11.271 INFO:teuthology.orchestra.run.smithi029.stdout: auth mon ticket ttl = 660 # 11m 2024-09-16T20:47:11.271 INFO:teuthology.orchestra.run.smithi029.stdout: auth service ticket ttl = 240 # 4m 2024-09-16T20:47:11.271 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:47:11.271 INFO:teuthology.orchestra.run.smithi029.stdout: # don't complain about insecure global_id in the test suite 2024-09-16T20:47:11.272 INFO:teuthology.orchestra.run.smithi029.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-16T20:47:11.272 INFO:teuthology.orchestra.run.smithi029.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-16T20:47:11.272 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:47:11.272 INFO:teuthology.orchestra.run.smithi029.stdout: # 1m isn't quite enough 2024-09-16T20:47:11.272 INFO:teuthology.orchestra.run.smithi029.stdout: mon_down_mkfs_grace = 2m 2024-09-16T20:47:11.272 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:47:11.272 INFO:teuthology.orchestra.run.smithi029.stdout: mon_warn_on_filestore_osds = false 2024-09-16T20:47:11.272 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:47:11.272 INFO:teuthology.orchestra.run.smithi029.stdout:[client] 2024-09-16T20:47:11.272 INFO:teuthology.orchestra.run.smithi029.stdout: rgw cache enabled = true 2024-09-16T20:47:11.272 INFO:teuthology.orchestra.run.smithi029.stdout: rgw enable ops log = true 2024-09-16T20:47:11.272 INFO:teuthology.orchestra.run.smithi029.stdout: rgw enable usage log = true 2024-09-16T20:47:11.272 INFO:teuthology.orchestra.run.smithi029.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-16T20:47:11.273 INFO:teuthology.orchestra.run.smithi029.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-16T20:47:11.273 INFO:teuthology.orchestra.run.smithi029.stdout:[mon.a] 2024-09-16T20:47:11.273 INFO:teuthology.orchestra.run.smithi029.stdout:[mon.c] 2024-09-16T20:47:11.273 INFO:teuthology.orchestra.run.smithi029.stdout:[mon.b] 2024-09-16T20:47:11.273 INFO:teuthology.orchestra.run.smithi043.stdout:[global] 2024-09-16T20:47:11.273 INFO:teuthology.orchestra.run.smithi043.stdout: chdir = "" 2024-09-16T20:47:11.273 INFO:teuthology.orchestra.run.smithi043.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-16T20:47:11.274 INFO:teuthology.orchestra.run.smithi043.stdout: auth supported = cephx 2024-09-16T20:47:11.274 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:47:11.274 INFO:teuthology.orchestra.run.smithi043.stdout: filestore xattr use omap = true 2024-09-16T20:47:11.274 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:47:11.274 INFO:teuthology.orchestra.run.smithi043.stdout: mon clock drift allowed = 1.000 2024-09-16T20:47:11.274 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:47:11.274 INFO:teuthology.orchestra.run.smithi043.stdout: osd crush chooseleaf type = 0 2024-09-16T20:47:11.274 INFO:teuthology.orchestra.run.smithi043.stdout: auth debug = true 2024-09-16T20:47:11.274 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:47:11.274 INFO:teuthology.orchestra.run.smithi043.stdout: ms die on old message = true 2024-09-16T20:47:11.274 INFO:teuthology.orchestra.run.smithi043.stdout: ms die on bug = true 2024-09-16T20:47:11.274 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:47:11.275 INFO:teuthology.orchestra.run.smithi043.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-16T20:47:11.275 INFO:teuthology.orchestra.run.smithi043.stdout: mon pg warn max object skew = 0 2024-09-16T20:47:11.275 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:47:11.275 INFO:teuthology.orchestra.run.smithi043.stdout: # disable pg_autoscaler by default for new pools 2024-09-16T20:47:11.275 INFO:teuthology.orchestra.run.smithi043.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-16T20:47:11.275 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:47:11.275 INFO:teuthology.orchestra.run.smithi043.stdout: osd pool default size = 2 2024-09-16T20:47:11.275 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:47:11.275 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd allow primary affinity = true 2024-09-16T20:47:11.275 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd allow pg remap = true 2024-09-16T20:47:11.275 INFO:teuthology.orchestra.run.smithi043.stdout: mon warn on legacy crush tunables = false 2024-09-16T20:47:11.275 INFO:teuthology.orchestra.run.smithi043.stdout: mon warn on crush straw calc version zero = false 2024-09-16T20:47:11.275 INFO:teuthology.orchestra.run.smithi043.stdout: mon warn on no sortbitwise = false 2024-09-16T20:47:11.276 INFO:teuthology.orchestra.run.smithi043.stdout: mon warn on osd down out interval zero = false 2024-09-16T20:47:11.276 INFO:teuthology.orchestra.run.smithi043.stdout: mon warn on too few osds = false 2024-09-16T20:47:11.276 INFO:teuthology.orchestra.run.smithi043.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-16T20:47:11.276 INFO:teuthology.orchestra.run.smithi043.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-16T20:47:11.276 INFO:teuthology.orchestra.run.smithi043.stdout: mon_allow_pool_size_one = true 2024-09-16T20:47:11.276 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:47:11.276 INFO:teuthology.orchestra.run.smithi043.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-16T20:47:11.276 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:47:11.276 INFO:teuthology.orchestra.run.smithi043.stdout: osd default data pool replay window = 5 2024-09-16T20:47:11.276 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:47:11.276 INFO:teuthology.orchestra.run.smithi043.stdout: mon allow pool delete = true 2024-09-16T20:47:11.276 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:47:11.276 INFO:teuthology.orchestra.run.smithi043.stdout: mon cluster log file level = debug 2024-09-16T20:47:11.276 INFO:teuthology.orchestra.run.smithi043.stdout: debug asserts on shutdown = true 2024-09-16T20:47:11.277 INFO:teuthology.orchestra.run.smithi043.stdout: mon health detail to clog = false 2024-09-16T20:47:11.277 INFO:teuthology.orchestra.run.smithi043.stdout: mon host = "172.21.15.29,[v2:172.21.15.29:3301,v1:172.21.15.29:6790],172.21.15.43" 2024-09-16T20:47:11.277 INFO:teuthology.orchestra.run.smithi043.stdout: mon client directed command retry = 5 2024-09-16T20:47:11.277 INFO:teuthology.orchestra.run.smithi043.stdout: mon warn on pool no app = False 2024-09-16T20:47:11.277 INFO:teuthology.orchestra.run.smithi043.stdout: ms inject socket failures = 5000 2024-09-16T20:47:11.277 INFO:teuthology.orchestra.run.smithi043.stdout: fsid = e28af4a7-5fcd-4de8-8f77-a2b575361bb1 2024-09-16T20:47:11.277 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:47:11.277 INFO:teuthology.orchestra.run.smithi043.stdout:[osd] 2024-09-16T20:47:11.277 INFO:teuthology.orchestra.run.smithi043.stdout: osd journal size = 100 2024-09-16T20:47:11.277 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:47:11.277 INFO:teuthology.orchestra.run.smithi043.stdout: osd scrub load threshold = 5.0 2024-09-16T20:47:11.277 INFO:teuthology.orchestra.run.smithi043.stdout: osd scrub max interval = 600 2024-09-16T20:47:11.277 INFO:teuthology.orchestra.run.smithi043.stdout: osd mclock profile = high_recovery_ops 2024-09-16T20:47:11.278 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:47:11.278 INFO:teuthology.orchestra.run.smithi043.stdout: osd recover clone overlap = true 2024-09-16T20:47:11.278 INFO:teuthology.orchestra.run.smithi043.stdout: osd recovery max chunk = 1048576 2024-09-16T20:47:11.278 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:47:11.278 INFO:teuthology.orchestra.run.smithi043.stdout: osd debug shutdown = true 2024-09-16T20:47:11.278 INFO:teuthology.orchestra.run.smithi043.stdout: osd debug op order = true 2024-09-16T20:47:11.278 INFO:teuthology.orchestra.run.smithi043.stdout: osd debug verify stray on activate = true 2024-09-16T20:47:11.278 INFO:teuthology.orchestra.run.smithi043.stdout: osd debug trim objects = true 2024-09-16T20:47:11.278 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:47:11.278 INFO:teuthology.orchestra.run.smithi043.stdout: osd open classes on start = true 2024-09-16T20:47:11.278 INFO:teuthology.orchestra.run.smithi043.stdout: osd debug pg log writeout = true 2024-09-16T20:47:11.278 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:47:11.278 INFO:teuthology.orchestra.run.smithi043.stdout: osd deep scrub update digest min age = 30 2024-09-16T20:47:11.279 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:47:11.279 INFO:teuthology.orchestra.run.smithi043.stdout: osd map max advance = 10 2024-09-16T20:47:11.279 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:47:11.279 INFO:teuthology.orchestra.run.smithi043.stdout: journal zero on create = true 2024-09-16T20:47:11.279 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:47:11.279 INFO:teuthology.orchestra.run.smithi043.stdout: filestore ondisk finisher threads = 3 2024-09-16T20:47:11.279 INFO:teuthology.orchestra.run.smithi043.stdout: filestore apply finisher threads = 3 2024-09-16T20:47:11.279 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:47:11.279 INFO:teuthology.orchestra.run.smithi043.stdout: bdev debug aio = true 2024-09-16T20:47:11.279 INFO:teuthology.orchestra.run.smithi043.stdout: osd debug misdirected ops = true 2024-09-16T20:47:11.279 INFO:teuthology.orchestra.run.smithi043.stdout: bdev async discard = True 2024-09-16T20:47:11.279 INFO:teuthology.orchestra.run.smithi043.stdout: bdev enable discard = True 2024-09-16T20:47:11.279 INFO:teuthology.orchestra.run.smithi043.stdout: bluestore allocator = bitmap 2024-09-16T20:47:11.280 INFO:teuthology.orchestra.run.smithi043.stdout: bluestore block size = 96636764160 2024-09-16T20:47:11.280 INFO:teuthology.orchestra.run.smithi043.stdout: bluestore fsck on mount = True 2024-09-16T20:47:11.280 INFO:teuthology.orchestra.run.smithi043.stdout: debug bluefs = 1/20 2024-09-16T20:47:11.280 INFO:teuthology.orchestra.run.smithi043.stdout: debug bluestore = 1/20 2024-09-16T20:47:11.280 INFO:teuthology.orchestra.run.smithi043.stdout: debug ms = 1 2024-09-16T20:47:11.280 INFO:teuthology.orchestra.run.smithi043.stdout: debug osd = 20 2024-09-16T20:47:11.280 INFO:teuthology.orchestra.run.smithi043.stdout: debug rocksdb = 4/10 2024-09-16T20:47:11.280 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd backfillfull_ratio = 0.85 2024-09-16T20:47:11.280 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd full ratio = 0.9 2024-09-16T20:47:11.280 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd nearfull ratio = 0.8 2024-09-16T20:47:11.280 INFO:teuthology.orchestra.run.smithi043.stdout: osd blocked scrub grace period = 3600 2024-09-16T20:47:11.280 INFO:teuthology.orchestra.run.smithi043.stdout: osd failsafe full ratio = 0.95 2024-09-16T20:47:11.280 INFO:teuthology.orchestra.run.smithi043.stdout: osd max markdown count = 1000 2024-09-16T20:47:11.280 INFO:teuthology.orchestra.run.smithi043.stdout: osd objectstore = bluestore 2024-09-16T20:47:11.281 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:47:11.281 INFO:teuthology.orchestra.run.smithi043.stdout:[mgr] 2024-09-16T20:47:11.281 INFO:teuthology.orchestra.run.smithi043.stdout: debug ms = 1 2024-09-16T20:47:11.281 INFO:teuthology.orchestra.run.smithi043.stdout: debug mgr = 20 2024-09-16T20:47:11.281 INFO:teuthology.orchestra.run.smithi043.stdout: debug mon = 20 2024-09-16T20:47:11.281 INFO:teuthology.orchestra.run.smithi043.stdout: debug auth = 20 2024-09-16T20:47:11.281 INFO:teuthology.orchestra.run.smithi043.stdout: mon reweight min pgs per osd = 4 2024-09-16T20:47:11.281 INFO:teuthology.orchestra.run.smithi043.stdout: mon reweight min bytes per osd = 10 2024-09-16T20:47:11.281 INFO:teuthology.orchestra.run.smithi043.stdout: mgr/telemetry/nag = false 2024-09-16T20:47:11.281 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:47:11.281 INFO:teuthology.orchestra.run.smithi043.stdout:[mon] 2024-09-16T20:47:11.281 INFO:teuthology.orchestra.run.smithi043.stdout: debug ms = 1 2024-09-16T20:47:11.281 INFO:teuthology.orchestra.run.smithi043.stdout: debug mon = 20 2024-09-16T20:47:11.281 INFO:teuthology.orchestra.run.smithi043.stdout: debug paxos = 20 2024-09-16T20:47:11.282 INFO:teuthology.orchestra.run.smithi043.stdout: debug auth = 20 2024-09-16T20:47:11.282 INFO:teuthology.orchestra.run.smithi043.stdout: mon data avail warn = 5 2024-09-16T20:47:11.282 INFO:teuthology.orchestra.run.smithi043.stdout: mon mgr mkfs grace = 240 2024-09-16T20:47:11.282 INFO:teuthology.orchestra.run.smithi043.stdout: mon reweight min pgs per osd = 4 2024-09-16T20:47:11.282 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd reporter subtree level = osd 2024-09-16T20:47:11.282 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd prime pg temp = true 2024-09-16T20:47:11.282 INFO:teuthology.orchestra.run.smithi043.stdout: mon reweight min bytes per osd = 10 2024-09-16T20:47:11.282 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:47:11.282 INFO:teuthology.orchestra.run.smithi043.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-16T20:47:11.282 INFO:teuthology.orchestra.run.smithi043.stdout: auth mon ticket ttl = 660 # 11m 2024-09-16T20:47:11.282 INFO:teuthology.orchestra.run.smithi043.stdout: auth service ticket ttl = 240 # 4m 2024-09-16T20:47:11.282 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:47:11.282 INFO:teuthology.orchestra.run.smithi043.stdout: # don't complain about insecure global_id in the test suite 2024-09-16T20:47:11.282 INFO:teuthology.orchestra.run.smithi043.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-16T20:47:11.282 INFO:teuthology.orchestra.run.smithi043.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-16T20:47:11.282 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:47:11.283 INFO:teuthology.orchestra.run.smithi043.stdout: # 1m isn't quite enough 2024-09-16T20:47:11.283 INFO:teuthology.orchestra.run.smithi043.stdout: mon_down_mkfs_grace = 2m 2024-09-16T20:47:11.283 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:47:11.283 INFO:teuthology.orchestra.run.smithi043.stdout: mon_warn_on_filestore_osds = false 2024-09-16T20:47:11.283 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T20:47:11.283 INFO:teuthology.orchestra.run.smithi043.stdout:[client] 2024-09-16T20:47:11.283 INFO:teuthology.orchestra.run.smithi043.stdout: rgw cache enabled = true 2024-09-16T20:47:11.283 INFO:teuthology.orchestra.run.smithi043.stdout: rgw enable ops log = true 2024-09-16T20:47:11.283 INFO:teuthology.orchestra.run.smithi043.stdout: rgw enable usage log = true 2024-09-16T20:47:11.283 INFO:teuthology.orchestra.run.smithi043.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-16T20:47:11.283 INFO:teuthology.orchestra.run.smithi043.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-16T20:47:11.283 INFO:teuthology.orchestra.run.smithi043.stdout:[mon.a] 2024-09-16T20:47:11.283 INFO:teuthology.orchestra.run.smithi043.stdout:[mon.c] 2024-09-16T20:47:11.283 INFO:teuthology.orchestra.run.smithi043.stdout:[mon.b] 2024-09-16T20:47:11.284 INFO:teuthology.orchestra.run.smithi139.stdout:[global] 2024-09-16T20:47:11.285 INFO:teuthology.orchestra.run.smithi139.stdout: chdir = "" 2024-09-16T20:47:11.285 INFO:teuthology.orchestra.run.smithi139.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-16T20:47:11.285 INFO:teuthology.orchestra.run.smithi139.stdout: auth supported = cephx 2024-09-16T20:47:11.285 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:47:11.285 INFO:teuthology.orchestra.run.smithi139.stdout: filestore xattr use omap = true 2024-09-16T20:47:11.285 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:47:11.285 INFO:teuthology.orchestra.run.smithi139.stdout: mon clock drift allowed = 1.000 2024-09-16T20:47:11.285 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:47:11.285 INFO:teuthology.orchestra.run.smithi139.stdout: osd crush chooseleaf type = 0 2024-09-16T20:47:11.285 INFO:teuthology.orchestra.run.smithi139.stdout: auth debug = true 2024-09-16T20:47:11.285 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:47:11.285 INFO:teuthology.orchestra.run.smithi139.stdout: ms die on old message = true 2024-09-16T20:47:11.285 INFO:teuthology.orchestra.run.smithi139.stdout: ms die on bug = true 2024-09-16T20:47:11.286 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:47:11.286 INFO:teuthology.orchestra.run.smithi139.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-16T20:47:11.286 INFO:teuthology.orchestra.run.smithi139.stdout: mon pg warn max object skew = 0 2024-09-16T20:47:11.286 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:47:11.286 INFO:teuthology.orchestra.run.smithi139.stdout: # disable pg_autoscaler by default for new pools 2024-09-16T20:47:11.286 INFO:teuthology.orchestra.run.smithi139.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-16T20:47:11.286 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:47:11.286 INFO:teuthology.orchestra.run.smithi139.stdout: osd pool default size = 2 2024-09-16T20:47:11.286 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:47:11.286 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd allow primary affinity = true 2024-09-16T20:47:11.286 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd allow pg remap = true 2024-09-16T20:47:11.286 INFO:teuthology.orchestra.run.smithi139.stdout: mon warn on legacy crush tunables = false 2024-09-16T20:47:11.286 INFO:teuthology.orchestra.run.smithi139.stdout: mon warn on crush straw calc version zero = false 2024-09-16T20:47:11.287 INFO:teuthology.orchestra.run.smithi139.stdout: mon warn on no sortbitwise = false 2024-09-16T20:47:11.287 INFO:teuthology.orchestra.run.smithi139.stdout: mon warn on osd down out interval zero = false 2024-09-16T20:47:11.287 INFO:teuthology.orchestra.run.smithi139.stdout: mon warn on too few osds = false 2024-09-16T20:47:11.287 INFO:teuthology.orchestra.run.smithi139.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-16T20:47:11.287 INFO:teuthology.orchestra.run.smithi139.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-16T20:47:11.287 INFO:teuthology.orchestra.run.smithi139.stdout: mon_allow_pool_size_one = true 2024-09-16T20:47:11.287 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:47:11.287 INFO:teuthology.orchestra.run.smithi139.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-16T20:47:11.287 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:47:11.287 INFO:teuthology.orchestra.run.smithi139.stdout: osd default data pool replay window = 5 2024-09-16T20:47:11.287 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:47:11.287 INFO:teuthology.orchestra.run.smithi139.stdout: mon allow pool delete = true 2024-09-16T20:47:11.287 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:47:11.287 INFO:teuthology.orchestra.run.smithi139.stdout: mon cluster log file level = debug 2024-09-16T20:47:11.288 INFO:teuthology.orchestra.run.smithi139.stdout: debug asserts on shutdown = true 2024-09-16T20:47:11.288 INFO:teuthology.orchestra.run.smithi139.stdout: mon health detail to clog = false 2024-09-16T20:47:11.288 INFO:teuthology.orchestra.run.smithi139.stdout: mon host = "172.21.15.29,[v2:172.21.15.29:3301,v1:172.21.15.29:6790],172.21.15.43" 2024-09-16T20:47:11.288 INFO:teuthology.orchestra.run.smithi139.stdout: mon client directed command retry = 5 2024-09-16T20:47:11.288 INFO:teuthology.orchestra.run.smithi139.stdout: mon warn on pool no app = False 2024-09-16T20:47:11.288 INFO:teuthology.orchestra.run.smithi139.stdout: ms inject socket failures = 5000 2024-09-16T20:47:11.288 INFO:teuthology.orchestra.run.smithi139.stdout: fsid = e28af4a7-5fcd-4de8-8f77-a2b575361bb1 2024-09-16T20:47:11.288 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:47:11.288 INFO:teuthology.orchestra.run.smithi139.stdout:[osd] 2024-09-16T20:47:11.288 INFO:teuthology.orchestra.run.smithi139.stdout: osd journal size = 100 2024-09-16T20:47:11.288 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:47:11.288 INFO:teuthology.orchestra.run.smithi139.stdout: osd scrub load threshold = 5.0 2024-09-16T20:47:11.288 INFO:teuthology.orchestra.run.smithi139.stdout: osd scrub max interval = 600 2024-09-16T20:47:11.289 INFO:teuthology.orchestra.run.smithi139.stdout: osd mclock profile = high_recovery_ops 2024-09-16T20:47:11.289 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:47:11.289 INFO:teuthology.orchestra.run.smithi139.stdout: osd recover clone overlap = true 2024-09-16T20:47:11.289 INFO:teuthology.orchestra.run.smithi139.stdout: osd recovery max chunk = 1048576 2024-09-16T20:47:11.289 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:47:11.289 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug shutdown = true 2024-09-16T20:47:11.289 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug op order = true 2024-09-16T20:47:11.289 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug verify stray on activate = true 2024-09-16T20:47:11.289 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug trim objects = true 2024-09-16T20:47:11.289 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:47:11.289 INFO:teuthology.orchestra.run.smithi139.stdout: osd open classes on start = true 2024-09-16T20:47:11.289 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug pg log writeout = true 2024-09-16T20:47:11.289 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:47:11.290 INFO:teuthology.orchestra.run.smithi139.stdout: osd deep scrub update digest min age = 30 2024-09-16T20:47:11.290 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:47:11.290 INFO:teuthology.orchestra.run.smithi139.stdout: osd map max advance = 10 2024-09-16T20:47:11.290 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:47:11.290 INFO:teuthology.orchestra.run.smithi139.stdout: journal zero on create = true 2024-09-16T20:47:11.290 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:47:11.290 INFO:teuthology.orchestra.run.smithi139.stdout: filestore ondisk finisher threads = 3 2024-09-16T20:47:11.290 INFO:teuthology.orchestra.run.smithi139.stdout: filestore apply finisher threads = 3 2024-09-16T20:47:11.290 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:47:11.290 INFO:teuthology.orchestra.run.smithi139.stdout: bdev debug aio = true 2024-09-16T20:47:11.290 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug misdirected ops = true 2024-09-16T20:47:11.290 INFO:teuthology.orchestra.run.smithi139.stdout: bdev async discard = True 2024-09-16T20:47:11.290 INFO:teuthology.orchestra.run.smithi139.stdout: bdev enable discard = True 2024-09-16T20:47:11.291 INFO:teuthology.orchestra.run.smithi139.stdout: bluestore allocator = bitmap 2024-09-16T20:47:11.291 INFO:teuthology.orchestra.run.smithi139.stdout: bluestore block size = 96636764160 2024-09-16T20:47:11.291 INFO:teuthology.orchestra.run.smithi139.stdout: bluestore fsck on mount = True 2024-09-16T20:47:11.291 INFO:teuthology.orchestra.run.smithi139.stdout: debug bluefs = 1/20 2024-09-16T20:47:11.291 INFO:teuthology.orchestra.run.smithi139.stdout: debug bluestore = 1/20 2024-09-16T20:47:11.291 INFO:teuthology.orchestra.run.smithi139.stdout: debug ms = 1 2024-09-16T20:47:11.291 INFO:teuthology.orchestra.run.smithi139.stdout: debug osd = 20 2024-09-16T20:47:11.291 INFO:teuthology.orchestra.run.smithi139.stdout: debug rocksdb = 4/10 2024-09-16T20:47:11.291 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd backfillfull_ratio = 0.85 2024-09-16T20:47:11.291 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd full ratio = 0.9 2024-09-16T20:47:11.291 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd nearfull ratio = 0.8 2024-09-16T20:47:11.291 INFO:teuthology.orchestra.run.smithi139.stdout: osd blocked scrub grace period = 3600 2024-09-16T20:47:11.291 INFO:teuthology.orchestra.run.smithi139.stdout: osd failsafe full ratio = 0.95 2024-09-16T20:47:11.291 INFO:teuthology.orchestra.run.smithi139.stdout: osd max markdown count = 1000 2024-09-16T20:47:11.291 INFO:teuthology.orchestra.run.smithi139.stdout: osd objectstore = bluestore 2024-09-16T20:47:11.292 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:47:11.292 INFO:teuthology.orchestra.run.smithi139.stdout:[mgr] 2024-09-16T20:47:11.292 INFO:teuthology.orchestra.run.smithi139.stdout: debug ms = 1 2024-09-16T20:47:11.292 INFO:teuthology.orchestra.run.smithi139.stdout: debug mgr = 20 2024-09-16T20:47:11.292 INFO:teuthology.orchestra.run.smithi139.stdout: debug mon = 20 2024-09-16T20:47:11.292 INFO:teuthology.orchestra.run.smithi139.stdout: debug auth = 20 2024-09-16T20:47:11.292 INFO:teuthology.orchestra.run.smithi139.stdout: mon reweight min pgs per osd = 4 2024-09-16T20:47:11.292 INFO:teuthology.orchestra.run.smithi139.stdout: mon reweight min bytes per osd = 10 2024-09-16T20:47:11.292 INFO:teuthology.orchestra.run.smithi139.stdout: mgr/telemetry/nag = false 2024-09-16T20:47:11.292 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:47:11.292 INFO:teuthology.orchestra.run.smithi139.stdout:[mon] 2024-09-16T20:47:11.292 INFO:teuthology.orchestra.run.smithi139.stdout: debug ms = 1 2024-09-16T20:47:11.292 INFO:teuthology.orchestra.run.smithi139.stdout: debug mon = 20 2024-09-16T20:47:11.292 INFO:teuthology.orchestra.run.smithi139.stdout: debug paxos = 20 2024-09-16T20:47:11.293 INFO:teuthology.orchestra.run.smithi139.stdout: debug auth = 20 2024-09-16T20:47:11.293 INFO:teuthology.orchestra.run.smithi139.stdout: mon data avail warn = 5 2024-09-16T20:47:11.293 INFO:teuthology.orchestra.run.smithi139.stdout: mon mgr mkfs grace = 240 2024-09-16T20:47:11.293 INFO:teuthology.orchestra.run.smithi139.stdout: mon reweight min pgs per osd = 4 2024-09-16T20:47:11.293 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd reporter subtree level = osd 2024-09-16T20:47:11.293 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd prime pg temp = true 2024-09-16T20:47:11.293 INFO:teuthology.orchestra.run.smithi139.stdout: mon reweight min bytes per osd = 10 2024-09-16T20:47:11.293 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:47:11.293 INFO:teuthology.orchestra.run.smithi139.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-16T20:47:11.293 INFO:teuthology.orchestra.run.smithi139.stdout: auth mon ticket ttl = 660 # 11m 2024-09-16T20:47:11.293 INFO:teuthology.orchestra.run.smithi139.stdout: auth service ticket ttl = 240 # 4m 2024-09-16T20:47:11.293 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:47:11.293 INFO:teuthology.orchestra.run.smithi139.stdout: # don't complain about insecure global_id in the test suite 2024-09-16T20:47:11.294 INFO:teuthology.orchestra.run.smithi139.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-16T20:47:11.294 INFO:teuthology.orchestra.run.smithi139.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-16T20:47:11.294 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:47:11.294 INFO:teuthology.orchestra.run.smithi139.stdout: # 1m isn't quite enough 2024-09-16T20:47:11.294 INFO:teuthology.orchestra.run.smithi139.stdout: mon_down_mkfs_grace = 2m 2024-09-16T20:47:11.294 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:47:11.294 INFO:teuthology.orchestra.run.smithi139.stdout: mon_warn_on_filestore_osds = false 2024-09-16T20:47:11.294 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T20:47:11.294 INFO:teuthology.orchestra.run.smithi139.stdout:[client] 2024-09-16T20:47:11.294 INFO:teuthology.orchestra.run.smithi139.stdout: rgw cache enabled = true 2024-09-16T20:47:11.294 INFO:teuthology.orchestra.run.smithi139.stdout: rgw enable ops log = true 2024-09-16T20:47:11.294 INFO:teuthology.orchestra.run.smithi139.stdout: rgw enable usage log = true 2024-09-16T20:47:11.294 INFO:teuthology.orchestra.run.smithi139.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-16T20:47:11.295 INFO:teuthology.orchestra.run.smithi139.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-16T20:47:11.295 INFO:teuthology.orchestra.run.smithi139.stdout:[mon.a] 2024-09-16T20:47:11.295 INFO:teuthology.orchestra.run.smithi139.stdout:[mon.c] 2024-09-16T20:47:11.295 INFO:teuthology.orchestra.run.smithi139.stdout:[mon.b] 2024-09-16T20:47:11.296 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-16T20:47:11.296 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-16T20:47:11.380 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-16T20:47:11.381 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-09-16T20:47:11.381 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-16T20:47:11.396 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-09-16T20:47:11.397 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-16T20:47:11.452 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi029.front.sepia.ceph.com 2024-09-16T20:47:11.452 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-09-16T20:47:11.452 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-16T20:47:11.452 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-16T20:47:11.528 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-09-16T20:47:11.528 DEBUG:teuthology.orchestra.run.smithi029:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-16T20:47:11.584 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi043.front.sepia.ceph.com 2024-09-16T20:47:11.584 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-16T20:47:11.584 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-16T20:47:11.584 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-16T20:47:11.622 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-16T20:47:11.622 DEBUG:teuthology.orchestra.run.smithi043:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-16T20:47:11.679 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi139.front.sepia.ceph.com 2024-09-16T20:47:11.679 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-16T20:47:11.679 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-16T20:47:11.679 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-16T20:47:11.720 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-16T20:47:11.720 DEBUG:teuthology.orchestra.run.smithi139:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-16T20:47:11.778 INFO:tasks.ceph:Setting up mon nodes... 2024-09-16T20:47:11.778 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-16T20:47:11.778 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-16T20:47:11.840 INFO:teuthology.orchestra.run.smithi043.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-16T20:47:11.843 INFO:tasks.ceph:Setting up mds nodes... 2024-09-16T20:47:11.843 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-16T20:47:11.843 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-16T20:47:11.895 INFO:teuthology.orchestra.run.smithi139.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-16T20:47:11.909 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-16T20:47:11.909 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi029.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi043.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'}} 2024-09-16T20:47:11.910 DEBUG:teuthology.orchestra.run.smithi029:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-16T20:47:11.936 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-16T20:47:11.936 INFO:tasks.ceph:role: osd.0 2024-09-16T20:47:11.936 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi029.front.sepia.ceph.com 2024-09-16T20:47:11.936 DEBUG:teuthology.orchestra.run.smithi029:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-16T20:47:12.003 INFO:teuthology.orchestra.run.smithi029.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T20:47:12.003 INFO:teuthology.orchestra.run.smithi029.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T20:47:12.003 INFO:teuthology.orchestra.run.smithi029.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T20:47:12.003 INFO:teuthology.orchestra.run.smithi029.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T20:47:12.003 INFO:teuthology.orchestra.run.smithi029.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T20:47:12.003 INFO:teuthology.orchestra.run.smithi029.stdout: = sunit=0 swidth=0 blks 2024-09-16T20:47:12.003 INFO:teuthology.orchestra.run.smithi029.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T20:47:12.003 INFO:teuthology.orchestra.run.smithi029.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T20:47:12.003 INFO:teuthology.orchestra.run.smithi029.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T20:47:12.004 INFO:teuthology.orchestra.run.smithi029.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T20:47:12.035 INFO:teuthology.orchestra.run.smithi029.stdout:Discarding blocks...Done. 2024-09-16T20:47:12.038 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi029.front.sepia.ceph.com -o noatime 2024-09-16T20:47:12.038 DEBUG:teuthology.orchestra.run.smithi029:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-16T20:47:12.108 DEBUG:teuthology.orchestra.run.smithi029:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-16T20:47:12.174 DEBUG:teuthology.orchestra.run.smithi029:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-16T20:47:12.247 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-16T20:47:12.247 INFO:tasks.ceph:role: osd.1 2024-09-16T20:47:12.247 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi029.front.sepia.ceph.com 2024-09-16T20:47:12.247 DEBUG:teuthology.orchestra.run.smithi029:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-16T20:47:12.313 INFO:teuthology.orchestra.run.smithi029.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T20:47:12.314 INFO:teuthology.orchestra.run.smithi029.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T20:47:12.314 INFO:teuthology.orchestra.run.smithi029.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T20:47:12.314 INFO:teuthology.orchestra.run.smithi029.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T20:47:12.314 INFO:teuthology.orchestra.run.smithi029.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T20:47:12.314 INFO:teuthology.orchestra.run.smithi029.stdout: = sunit=0 swidth=0 blks 2024-09-16T20:47:12.314 INFO:teuthology.orchestra.run.smithi029.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T20:47:12.314 INFO:teuthology.orchestra.run.smithi029.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T20:47:12.314 INFO:teuthology.orchestra.run.smithi029.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T20:47:12.314 INFO:teuthology.orchestra.run.smithi029.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T20:47:12.346 INFO:teuthology.orchestra.run.smithi029.stdout:Discarding blocks...Done. 2024-09-16T20:47:12.348 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi029.front.sepia.ceph.com -o noatime 2024-09-16T20:47:12.349 DEBUG:teuthology.orchestra.run.smithi029:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-16T20:47:12.420 DEBUG:teuthology.orchestra.run.smithi029:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-16T20:47:12.491 DEBUG:teuthology.orchestra.run.smithi029:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-16T20:47:12.557 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-16T20:47:12.557 INFO:tasks.ceph:role: osd.2 2024-09-16T20:47:12.557 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi029.front.sepia.ceph.com 2024-09-16T20:47:12.558 DEBUG:teuthology.orchestra.run.smithi029:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-16T20:47:12.622 INFO:teuthology.orchestra.run.smithi029.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T20:47:12.623 INFO:teuthology.orchestra.run.smithi029.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T20:47:12.623 INFO:teuthology.orchestra.run.smithi029.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T20:47:12.623 INFO:teuthology.orchestra.run.smithi029.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T20:47:12.623 INFO:teuthology.orchestra.run.smithi029.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T20:47:12.623 INFO:teuthology.orchestra.run.smithi029.stdout: = sunit=0 swidth=0 blks 2024-09-16T20:47:12.623 INFO:teuthology.orchestra.run.smithi029.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T20:47:12.623 INFO:teuthology.orchestra.run.smithi029.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T20:47:12.623 INFO:teuthology.orchestra.run.smithi029.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T20:47:12.623 INFO:teuthology.orchestra.run.smithi029.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T20:47:12.656 INFO:teuthology.orchestra.run.smithi029.stdout:Discarding blocks...Done. 2024-09-16T20:47:12.658 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi029.front.sepia.ceph.com -o noatime 2024-09-16T20:47:12.659 DEBUG:teuthology.orchestra.run.smithi029:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-16T20:47:12.734 DEBUG:teuthology.orchestra.run.smithi029:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-16T20:47:12.805 DEBUG:teuthology.orchestra.run.smithi029:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T20:47:12.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-16T20:47:12.897+0000 7f0e8d72b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-16T20:47:12.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-16T20:47:12.898+0000 7f0e8d72b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-16T20:47:12.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-16T20:47:12.898+0000 7f0e8d72b640 -1 bdev(0x55b44d94f800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-16T20:47:12.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-16T20:47:12.898+0000 7f0e8d72b640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-16T20:47:16.070 DEBUG:teuthology.orchestra.run.smithi029:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-16T20:47:16.101 DEBUG:teuthology.orchestra.run.smithi029:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T20:47:16.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-16T20:47:16.192+0000 7f9254a2c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-16T20:47:16.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-16T20:47:16.192+0000 7f9254a2c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-16T20:47:16.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-16T20:47:16.192+0000 7f9254a2c640 -1 bdev(0x55839043f800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-16T20:47:16.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-16T20:47:16.192+0000 7f9254a2c640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-16T20:47:19.851 DEBUG:teuthology.orchestra.run.smithi029:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-16T20:47:19.882 DEBUG:teuthology.orchestra.run.smithi029:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T20:47:19.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-16T20:47:19.974+0000 7f502322c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-16T20:47:19.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-16T20:47:19.975+0000 7f502322c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-16T20:47:19.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-16T20:47:19.975+0000 7f502322c640 -1 bdev(0x5557a8847800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-16T20:47:19.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-16T20:47:19.975+0000 7f502322c640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-16T20:47:22.887 DEBUG:teuthology.orchestra.run.smithi029:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-16T20:47:22.918 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-16T20:47:22.950 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-16T20:47:22.950 INFO:tasks.ceph:role: osd.3 2024-09-16T20:47:22.950 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi043.front.sepia.ceph.com 2024-09-16T20:47:22.950 DEBUG:teuthology.orchestra.run.smithi043:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-16T20:47:23.021 INFO:teuthology.orchestra.run.smithi043.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T20:47:23.021 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T20:47:23.021 INFO:teuthology.orchestra.run.smithi043.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T20:47:23.021 INFO:teuthology.orchestra.run.smithi043.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T20:47:23.021 INFO:teuthology.orchestra.run.smithi043.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T20:47:23.022 INFO:teuthology.orchestra.run.smithi043.stdout: = sunit=0 swidth=0 blks 2024-09-16T20:47:23.022 INFO:teuthology.orchestra.run.smithi043.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T20:47:23.022 INFO:teuthology.orchestra.run.smithi043.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T20:47:23.022 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T20:47:23.022 INFO:teuthology.orchestra.run.smithi043.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T20:47:23.054 INFO:teuthology.orchestra.run.smithi043.stdout:Discarding blocks...Done. 2024-09-16T20:47:23.056 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi043.front.sepia.ceph.com -o noatime 2024-09-16T20:47:23.056 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-3 2024-09-16T20:47:23.133 DEBUG:teuthology.orchestra.run.smithi043:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-16T20:47:23.204 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-16T20:47:23.271 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-16T20:47:23.272 INFO:tasks.ceph:role: osd.4 2024-09-16T20:47:23.272 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi043.front.sepia.ceph.com 2024-09-16T20:47:23.272 DEBUG:teuthology.orchestra.run.smithi043:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-16T20:47:23.337 INFO:teuthology.orchestra.run.smithi043.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T20:47:23.338 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T20:47:23.338 INFO:teuthology.orchestra.run.smithi043.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T20:47:23.338 INFO:teuthology.orchestra.run.smithi043.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T20:47:23.338 INFO:teuthology.orchestra.run.smithi043.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T20:47:23.338 INFO:teuthology.orchestra.run.smithi043.stdout: = sunit=0 swidth=0 blks 2024-09-16T20:47:23.338 INFO:teuthology.orchestra.run.smithi043.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T20:47:23.338 INFO:teuthology.orchestra.run.smithi043.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T20:47:23.338 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T20:47:23.338 INFO:teuthology.orchestra.run.smithi043.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T20:47:23.370 INFO:teuthology.orchestra.run.smithi043.stdout:Discarding blocks...Done. 2024-09-16T20:47:23.372 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi043.front.sepia.ceph.com -o noatime 2024-09-16T20:47:23.372 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-4 2024-09-16T20:47:23.442 DEBUG:teuthology.orchestra.run.smithi043:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-16T20:47:23.512 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-16T20:47:23.579 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-16T20:47:23.579 INFO:tasks.ceph:role: osd.5 2024-09-16T20:47:23.579 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi043.front.sepia.ceph.com 2024-09-16T20:47:23.579 DEBUG:teuthology.orchestra.run.smithi043:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-16T20:47:23.644 INFO:teuthology.orchestra.run.smithi043.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T20:47:23.644 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T20:47:23.644 INFO:teuthology.orchestra.run.smithi043.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T20:47:23.644 INFO:teuthology.orchestra.run.smithi043.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T20:47:23.644 INFO:teuthology.orchestra.run.smithi043.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T20:47:23.645 INFO:teuthology.orchestra.run.smithi043.stdout: = sunit=0 swidth=0 blks 2024-09-16T20:47:23.645 INFO:teuthology.orchestra.run.smithi043.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T20:47:23.645 INFO:teuthology.orchestra.run.smithi043.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T20:47:23.645 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T20:47:23.645 INFO:teuthology.orchestra.run.smithi043.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T20:47:23.678 INFO:teuthology.orchestra.run.smithi043.stdout:Discarding blocks...Done. 2024-09-16T20:47:23.680 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi043.front.sepia.ceph.com -o noatime 2024-09-16T20:47:23.680 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-5 2024-09-16T20:47:23.750 DEBUG:teuthology.orchestra.run.smithi043:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-16T20:47:23.820 DEBUG:teuthology.orchestra.run.smithi043:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T20:47:23.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T20:47:23.933+0000 7f9e86eb0640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-16T20:47:23.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T20:47:23.933+0000 7f9e86eb0640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-16T20:47:23.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T20:47:23.933+0000 7f9e86eb0640 -1 bdev(0x55eba8411800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-16T20:47:23.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T20:47:23.933+0000 7f9e86eb0640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-16T20:47:26.847 DEBUG:teuthology.orchestra.run.smithi043:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-16T20:47:26.878 DEBUG:teuthology.orchestra.run.smithi043:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T20:47:26.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T20:47:26.969+0000 7fd62101c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-16T20:47:26.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T20:47:26.969+0000 7fd62101c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-16T20:47:26.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T20:47:26.969+0000 7fd62101c640 -1 bdev(0x55de505b3800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-16T20:47:26.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T20:47:26.969+0000 7fd62101c640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-16T20:47:29.883 DEBUG:teuthology.orchestra.run.smithi043:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-16T20:47:29.914 DEBUG:teuthology.orchestra.run.smithi043:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T20:47:30.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T20:47:30.006+0000 7f67bdf69640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-16T20:47:30.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T20:47:30.006+0000 7f67bdf69640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-16T20:47:30.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T20:47:30.006+0000 7f67bdf69640 -1 bdev(0x56106b309800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-16T20:47:30.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T20:47:30.006+0000 7f67bdf69640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-16T20:47:32.920 DEBUG:teuthology.orchestra.run.smithi043:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-16T20:47:32.951 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-16T20:47:32.951 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-09-16T20:47:32.951 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-16T20:47:32.983 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-09-16T20:47:32.983 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-16T20:47:33.053 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-09-16T20:47:33.053 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-16T20:47:33.122 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-16T20:47:33.123 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-16T20:47:33.153 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-16T20:47:33.153 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-16T20:47:33.223 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-16T20:47:33.223 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-16T20:47:33.292 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-16T20:47:33.292 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-16T20:47:33.360 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-16T20:47:33.361 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-16T20:47:33.380 INFO:tasks.ceph:Adding keys to all mons... 2024-09-16T20:47:33.381 DEBUG:teuthology.orchestra.run.smithi029:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-16T20:47:33.383 DEBUG:teuthology.orchestra.run.smithi043:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-16T20:47:33.410 INFO:teuthology.orchestra.run.smithi029.stdout:[osd.0] 2024-09-16T20:47:33.410 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQBQmehm1o2WNRAA9V1sO8Vf0uiMjjpqNvJ7qg== 2024-09-16T20:47:33.410 INFO:teuthology.orchestra.run.smithi029.stdout:[osd.1] 2024-09-16T20:47:33.410 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQBUmehmu+yKCxAAwG7QCQAZxVWQRWgoqYFTgw== 2024-09-16T20:47:33.410 INFO:teuthology.orchestra.run.smithi029.stdout:[osd.2] 2024-09-16T20:47:33.410 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQBXmehmrwstOhAAU0C3jrCxJpTqLsoZF5SMiA== 2024-09-16T20:47:33.410 INFO:teuthology.orchestra.run.smithi029.stdout:[mgr.x] 2024-09-16T20:47:33.410 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQBPmehmWPARMhAA3KIDBpVmWzUWYW4tKWhcBQ== 2024-09-16T20:47:33.411 INFO:teuthology.orchestra.run.smithi029.stdout:[osd.3] 2024-09-16T20:47:33.411 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQBbmehmu4u4NxAAT8zwIjo0A+gKE/tsiZSazA== 2024-09-16T20:47:33.411 INFO:teuthology.orchestra.run.smithi029.stdout:[osd.4] 2024-09-16T20:47:33.411 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQBemehmiT3cORAAHFk26PR8Z1DfzeyICPlJ9A== 2024-09-16T20:47:33.411 INFO:teuthology.orchestra.run.smithi029.stdout:[osd.5] 2024-09-16T20:47:33.411 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQBimehmJJdyABAA0Rf19+ohKbSgS6XQdKhN7A== 2024-09-16T20:47:33.411 INFO:teuthology.orchestra.run.smithi029.stdout:[client.0] 2024-09-16T20:47:33.411 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQBPmehmlmNcNRAA4zLnA+kLlhA950iwGIRVPA== 2024-09-16T20:47:33.428 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.0] 2024-09-16T20:47:33.428 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQBQmehm1o2WNRAA9V1sO8Vf0uiMjjpqNvJ7qg== 2024-09-16T20:47:33.428 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.1] 2024-09-16T20:47:33.428 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQBUmehmu+yKCxAAwG7QCQAZxVWQRWgoqYFTgw== 2024-09-16T20:47:33.428 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.2] 2024-09-16T20:47:33.428 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQBXmehmrwstOhAAU0C3jrCxJpTqLsoZF5SMiA== 2024-09-16T20:47:33.428 INFO:teuthology.orchestra.run.smithi043.stdout:[mgr.x] 2024-09-16T20:47:33.428 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQBPmehmWPARMhAA3KIDBpVmWzUWYW4tKWhcBQ== 2024-09-16T20:47:33.429 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.3] 2024-09-16T20:47:33.429 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQBbmehmu4u4NxAAT8zwIjo0A+gKE/tsiZSazA== 2024-09-16T20:47:33.429 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.4] 2024-09-16T20:47:33.429 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQBemehmiT3cORAAHFk26PR8Z1DfzeyICPlJ9A== 2024-09-16T20:47:33.429 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.5] 2024-09-16T20:47:33.429 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQBimehmJJdyABAA0Rf19+ohKbSgS6XQdKhN7A== 2024-09-16T20:47:33.429 INFO:teuthology.orchestra.run.smithi043.stdout:[client.0] 2024-09-16T20:47:33.429 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQBPmehmlmNcNRAA4zLnA+kLlhA950iwGIRVPA== 2024-09-16T20:47:33.430 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T20:47:33.455 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T20:47:33.526 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T20:47:33.529 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T20:47:33.583 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T20:47:33.585 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T20:47:33.638 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-16T20:47:33.641 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-16T20:47:33.693 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T20:47:33.696 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T20:47:33.748 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T20:47:33.750 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T20:47:33.802 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T20:47:33.805 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T20:47:33.857 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T20:47:33.859 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T20:47:33.914 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-16T20:47:33.915 DEBUG:teuthology.orchestra.run.smithi029:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-16T20:47:33.940 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-16T20:47:34.038 DEBUG:teuthology.orchestra.run.smithi029:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-16T20:47:34.063 DEBUG:teuthology.orchestra.run.smithi029:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-16T20:47:34.129 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-16T20:47:34.226 DEBUG:teuthology.orchestra.run.smithi029:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-16T20:47:34.252 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-16T20:47:34.278 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-16T20:47:34.377 DEBUG:teuthology.orchestra.run.smithi043:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-16T20:47:34.402 DEBUG:teuthology.orchestra.run.smithi029:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-16T20:47:34.404 DEBUG:teuthology.orchestra.run.smithi043:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-16T20:47:34.458 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-16T20:47:34.458 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-16T20:47:34.458 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-16T20:47:34.460 INFO:tasks.ceph.mon.a:Started 2024-09-16T20:47:34.460 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-16T20:47:34.461 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-16T20:47:34.463 INFO:tasks.ceph.mon.c:Started 2024-09-16T20:47:34.463 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-16T20:47:34.463 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-16T20:47:34.501 INFO:tasks.ceph.mon.b:Started 2024-09-16T20:47:34.501 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-16T20:47:34.501 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-16T20:47:34.501 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-16T20:47:34.503 INFO:tasks.ceph.mgr.x:Started 2024-09-16T20:47:34.503 DEBUG:tasks.ceph:set 0 configs 2024-09-16T20:47:34.503 INFO:tasks.ceph:Setting crush tunables to default 2024-09-16T20:47:34.503 DEBUG:teuthology.orchestra.run.smithi029:> sudo ceph --cluster ceph osd crush tunables default 2024-09-16T20:47:39.851 INFO:teuthology.orchestra.run.smithi029.stderr:adjusted tunables profile to default 2024-09-16T20:47:39.863 INFO:tasks.ceph:check_enable_crimson: False 2024-09-16T20:47:39.863 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-16T20:47:39.864 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-09-16T20:47:39.864 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-16T20:47:39.890 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-09-16T20:47:39.891 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-16T20:47:39.957 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-09-16T20:47:39.957 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-16T20:47:40.023 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-16T20:47:40.023 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-16T20:47:40.049 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-16T20:47:40.049 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-16T20:47:40.115 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-16T20:47:40.115 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-16T20:47:40.181 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --cluster ceph osd new 6c3cb2d7-7d23-4566-9ca1-d7b08e12360c 0 2024-09-16T20:47:40.405 INFO:teuthology.orchestra.run.smithi043.stdout:0 2024-09-16T20:47:40.417 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --cluster ceph osd new 5cf8c20c-1ed2-42ae-a8b0-c88cc4e36bce 1 2024-09-16T20:47:40.601 INFO:teuthology.orchestra.run.smithi043.stdout:1 2024-09-16T20:47:40.612 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --cluster ceph osd new 075d16da-82bf-449b-9f6e-542c9e8d810b 2 2024-09-16T20:47:40.796 INFO:teuthology.orchestra.run.smithi043.stdout:2 2024-09-16T20:47:40.808 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --cluster ceph osd new 0612be38-3a24-41b6-aba4-5a2fe985165e 3 2024-09-16T20:47:40.990 INFO:teuthology.orchestra.run.smithi043.stdout:3 2024-09-16T20:47:41.002 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --cluster ceph osd new be590bf8-e9fa-44ca-a471-6f6bcb087ca7 4 2024-09-16T20:47:41.186 INFO:teuthology.orchestra.run.smithi043.stdout:4 2024-09-16T20:47:41.197 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --cluster ceph osd new 4a208e30-031b-4a9f-af52-e26d4a039ea6 5 2024-09-16T20:47:41.381 INFO:teuthology.orchestra.run.smithi043.stdout:5 2024-09-16T20:47:41.392 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-16T20:47:41.392 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-16T20:47:41.394 INFO:tasks.ceph.osd.0:Started 2024-09-16T20:47:41.395 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-16T20:47:41.395 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-16T20:47:41.397 INFO:tasks.ceph.osd.1:Started 2024-09-16T20:47:41.397 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-16T20:47:41.397 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-16T20:47:41.400 INFO:tasks.ceph.osd.2:Started 2024-09-16T20:47:41.400 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-16T20:47:41.400 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-16T20:47:41.402 INFO:tasks.ceph.osd.3:Started 2024-09-16T20:47:41.403 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-16T20:47:41.403 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-16T20:47:41.405 INFO:tasks.ceph.osd.4:Started 2024-09-16T20:47:41.405 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-16T20:47:41.405 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-16T20:47:41.407 INFO:tasks.ceph.osd.5:Started 2024-09-16T20:47:41.407 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T20:47:41.596 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:47:41.596 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":8,"fsid":"e28af4a7-5fcd-4de8-8f77-a2b575361bb1","created":"2024-09-16T20:47:39.760609+0000","modified":"2024-09-16T20:47:41.378753+0000","last_up_change":"0.000000","last_in_change":"2024-09-16T20:47:41.378753+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"6c3cb2d7-7d23-4566-9ca1-d7b08e12360c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"5cf8c20c-1ed2-42ae-a8b0-c88cc4e36bce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"075d16da-82bf-449b-9f6e-542c9e8d810b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"0612be38-3a24-41b6-aba4-5a2fe985165e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"be590bf8-e9fa-44ca-a471-6f6bcb087ca7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"4a208e30-031b-4a9f-af52-e26d4a039ea6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T20:47:41.606 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-16T20:47:41.607 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-16T20:47:42.213 DEBUG:teuthology.orchestra.run.smithi029:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-16T20:47:42.369 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:47:42.368+0000 7f2ccd54d640 -1 Falling back to public interface 2024-09-16T20:47:42.384 INFO:teuthology.misc.health.smithi029.stdout: 2024-09-16T20:47:42.385 INFO:teuthology.misc.health.smithi029.stdout:{"epoch":8,"fsid":"e28af4a7-5fcd-4de8-8f77-a2b575361bb1","created":"2024-09-16T20:47:39.760609+0000","modified":"2024-09-16T20:47:41.378753+0000","last_up_change":"0.000000","last_in_change":"2024-09-16T20:47:41.378753+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"6c3cb2d7-7d23-4566-9ca1-d7b08e12360c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"5cf8c20c-1ed2-42ae-a8b0-c88cc4e36bce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"075d16da-82bf-449b-9f6e-542c9e8d810b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"0612be38-3a24-41b6-aba4-5a2fe985165e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"be590bf8-e9fa-44ca-a471-6f6bcb087ca7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"4a208e30-031b-4a9f-af52-e26d4a039ea6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T20:47:42.394 DEBUG:teuthology.misc:0 of 6 OSDs are up 2024-09-16T20:47:42.396 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:47:42.395+0000 7f8ac6a0c640 -1 Falling back to public interface 2024-09-16T20:47:42.839 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:47:42.838+0000 7f4b29a2c640 -1 Falling back to public interface 2024-09-16T20:47:42.847 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:47:42.846+0000 7f261b80d640 -1 Falling back to public interface 2024-09-16T20:47:43.094 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:47:43.093+0000 7fb27a4ef640 -1 Falling back to public interface 2024-09-16T20:47:43.128 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:47:43.127+0000 7f09b3b89640 -1 Falling back to public interface 2024-09-16T20:47:45.209 INFO:tasks.ceph.mgr.x.smithi043.stderr:2024-09-16T20:47:45.208+0000 7fc0a571c640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-16T20:47:45.209 INFO:tasks.ceph.mgr.x.smithi043.stderr:2024-09-16T20:47:45.208+0000 7fc0a571c640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-16T20:47:45.921 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:47:45.920+0000 7f2ccd54d640 -1 osd.4 0 log_to_monitors true 2024-09-16T20:47:46.404 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:47:46.402+0000 7f261b80d640 -1 osd.2 0 log_to_monitors true 2024-09-16T20:47:46.451 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:47:46.451+0000 7f8ac6a0c640 -1 osd.3 0 log_to_monitors true 2024-09-16T20:47:46.644 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:47:46.643+0000 7f4b29a2c640 -1 osd.1 0 log_to_monitors true 2024-09-16T20:47:46.851 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:47:46.850+0000 7fb27a4ef640 -1 osd.0 0 log_to_monitors true 2024-09-16T20:47:46.896 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:47:46.895+0000 7f09b3b89640 -1 osd.5 0 log_to_monitors true 2024-09-16T20:47:47.979 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:47:47.978+0000 7f2cc94dc640 -1 osd.4 0 waiting for initial osdmap 2024-09-16T20:47:47.997 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:47:47.996+0000 7f2cc12fb640 -1 osd.4 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T20:47:49.000 DEBUG:teuthology.orchestra.run.smithi029:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-16T20:47:49.230 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:47:49.230+0000 7f8ac2286640 -1 osd.3 0 waiting for initial osdmap 2024-09-16T20:47:49.240 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:47:49.240+0000 7f8aba0a5640 -1 osd.3 11 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T20:47:49.256 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:47:49.255+0000 7f09afb18640 -1 osd.5 0 waiting for initial osdmap 2024-09-16T20:47:49.272 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:47:49.272+0000 7f09a7937640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T20:47:49.421 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:47:49.420+0000 7fb27647e640 -1 osd.0 0 waiting for initial osdmap 2024-09-16T20:47:49.435 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:47:49.434+0000 7fb26e29d640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T20:47:49.456 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:47:49.455+0000 7f2617231640 -1 osd.2 0 waiting for initial osdmap 2024-09-16T20:47:49.467 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:47:49.466+0000 7f260f050640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T20:47:49.522 INFO:teuthology.misc.health.smithi029.stdout: 2024-09-16T20:47:49.523 INFO:teuthology.misc.health.smithi029.stdout:{"epoch":12,"fsid":"e28af4a7-5fcd-4de8-8f77-a2b575361bb1","created":"2024-09-16T20:47:39.760609+0000","modified":"2024-09-16T20:47:49.236815+0000","last_up_change":"2024-09-16T20:47:48.207196+0000","last_in_change":"2024-09-16T20:47:41.378753+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"6c3cb2d7-7d23-4566-9ca1-d7b08e12360c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"5cf8c20c-1ed2-42ae-a8b0-c88cc4e36bce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"075d16da-82bf-449b-9f6e-542c9e8d810b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"0612be38-3a24-41b6-aba4-5a2fe985165e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"be590bf8-e9fa-44ca-a471-6f6bcb087ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":2505220646},{"type":"v1","addr":"172.21.15.43:6801","nonce":2505220646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":2505220646},{"type":"v1","addr":"172.21.15.43:6803","nonce":2505220646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":2505220646},{"type":"v1","addr":"172.21.15.43:6807","nonce":2505220646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":2505220646},{"type":"v1","addr":"172.21.15.43:6805","nonce":2505220646}]},"public_addr":"172.21.15.43:6801/2505220646","cluster_addr":"172.21.15.43:6803/2505220646","heartbeat_back_addr":"172.21.15.43:6807/2505220646","heartbeat_front_addr":"172.21.15.43:6805/2505220646","state":["exists","up"]},{"osd":5,"uuid":"4a208e30-031b-4a9f-af52-e26d4a039ea6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T20:47:46.943344+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T20:47:49.534 DEBUG:teuthology.misc:1 of 6 OSDs are up 2024-09-16T20:47:49.549 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:47:49.547+0000 7f4b2503a640 -1 osd.1 0 waiting for initial osdmap 2024-09-16T20:47:49.561 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:47:49.560+0000 7f4b1ce59640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T20:47:56.140 DEBUG:teuthology.orchestra.run.smithi029:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-16T20:47:56.444 INFO:teuthology.misc.health.smithi029.stdout: 2024-09-16T20:47:56.444 INFO:teuthology.misc.health.smithi029.stdout:{"epoch":17,"fsid":"e28af4a7-5fcd-4de8-8f77-a2b575361bb1","created":"2024-09-16T20:47:39.760609+0000","modified":"2024-09-16T20:47:54.250201+0000","last_up_change":"2024-09-16T20:47:50.238013+0000","last_in_change":"2024-09-16T20:47:41.378753+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T20:47:52.220750+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c3cb2d7-7d23-4566-9ca1-d7b08e12360c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":2744025949},{"type":"v1","addr":"172.21.15.29:6817","nonce":2744025949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":2744025949},{"type":"v1","addr":"172.21.15.29:6819","nonce":2744025949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":2744025949},{"type":"v1","addr":"172.21.15.29:6823","nonce":2744025949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":2744025949},{"type":"v1","addr":"172.21.15.29:6821","nonce":2744025949}]},"public_addr":"172.21.15.29:6817/2744025949","cluster_addr":"172.21.15.29:6819/2744025949","heartbeat_back_addr":"172.21.15.29:6823/2744025949","heartbeat_front_addr":"172.21.15.29:6821/2744025949","state":["exists","up"]},{"osd":1,"uuid":"5cf8c20c-1ed2-42ae-a8b0-c88cc4e36bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":208628762},{"type":"v1","addr":"172.21.15.29:6801","nonce":208628762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":208628762},{"type":"v1","addr":"172.21.15.29:6803","nonce":208628762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":208628762},{"type":"v1","addr":"172.21.15.29:6807","nonce":208628762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":208628762},{"type":"v1","addr":"172.21.15.29:6805","nonce":208628762}]},"public_addr":"172.21.15.29:6801/208628762","cluster_addr":"172.21.15.29:6803/208628762","heartbeat_back_addr":"172.21.15.29:6807/208628762","heartbeat_front_addr":"172.21.15.29:6805/208628762","state":["exists","up"]},{"osd":2,"uuid":"075d16da-82bf-449b-9f6e-542c9e8d810b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3447426536},{"type":"v1","addr":"172.21.15.29:6809","nonce":3447426536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3447426536},{"type":"v1","addr":"172.21.15.29:6811","nonce":3447426536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3447426536},{"type":"v1","addr":"172.21.15.29:6815","nonce":3447426536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":3447426536},{"type":"v1","addr":"172.21.15.29:6813","nonce":3447426536}]},"public_addr":"172.21.15.29:6809/3447426536","cluster_addr":"172.21.15.29:6811/3447426536","heartbeat_back_addr":"172.21.15.29:6815/3447426536","heartbeat_front_addr":"172.21.15.29:6813/3447426536","state":["exists","up"]},{"osd":3,"uuid":"0612be38-3a24-41b6-aba4-5a2fe985165e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3687583891},{"type":"v1","addr":"172.21.15.43:6809","nonce":3687583891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3687583891},{"type":"v1","addr":"172.21.15.43:6811","nonce":3687583891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3687583891},{"type":"v1","addr":"172.21.15.43:6815","nonce":3687583891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3687583891},{"type":"v1","addr":"172.21.15.43:6813","nonce":3687583891}]},"public_addr":"172.21.15.43:6809/3687583891","cluster_addr":"172.21.15.43:6811/3687583891","heartbeat_back_addr":"172.21.15.43:6815/3687583891","heartbeat_front_addr":"172.21.15.43:6813/3687583891","state":["exists","up"]},{"osd":4,"uuid":"be590bf8-e9fa-44ca-a471-6f6bcb087ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":2505220646},{"type":"v1","addr":"172.21.15.43:6801","nonce":2505220646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":2505220646},{"type":"v1","addr":"172.21.15.43:6803","nonce":2505220646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":2505220646},{"type":"v1","addr":"172.21.15.43:6807","nonce":2505220646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":2505220646},{"type":"v1","addr":"172.21.15.43:6805","nonce":2505220646}]},"public_addr":"172.21.15.43:6801/2505220646","cluster_addr":"172.21.15.43:6803/2505220646","heartbeat_back_addr":"172.21.15.43:6807/2505220646","heartbeat_front_addr":"172.21.15.43:6805/2505220646","state":["exists","up"]},{"osd":5,"uuid":"4a208e30-031b-4a9f-af52-e26d4a039ea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1517367307},{"type":"v1","addr":"172.21.15.43:6817","nonce":1517367307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":1517367307},{"type":"v1","addr":"172.21.15.43:6819","nonce":1517367307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":1517367307},{"type":"v1","addr":"172.21.15.43:6823","nonce":1517367307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1517367307},{"type":"v1","addr":"172.21.15.43:6821","nonce":1517367307}]},"public_addr":"172.21.15.43:6817/1517367307","cluster_addr":"172.21.15.43:6819/1517367307","heartbeat_back_addr":"172.21.15.43:6823/1517367307","heartbeat_front_addr":"172.21.15.43:6821/1517367307","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T20:47:47.887412+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T20:47:47.658782+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T20:47:47.453752+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T20:47:47.487992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T20:47:46.943344+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T20:47:47.901547+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T20:47:56.455 DEBUG:teuthology.misc:6 of 6 OSDs are up 2024-09-16T20:47:56.456 INFO:tasks.ceph:Creating RBD pool 2024-09-16T20:47:56.456 DEBUG:teuthology.orchestra.run.smithi029:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-16T20:47:57.265 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'rbd' created 2024-09-16T20:47:57.281 DEBUG:teuthology.orchestra.run.smithi029:> rbd --cluster ceph pool init rbd 2024-09-16T20:48:00.285 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-16T20:48:00.286 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-16T20:48:00.286 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-16T20:48:00.595 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:48:00.608 INFO:teuthology.orchestra.run.smithi029.stdout:[{"version":7,"timestamp":"2024-09-16T20:47:49.549403+0000","name":"","changes":[{"name":"osd.1/osd_mclock_max_capacity_iops_ssd","new_value":"4932.912724"}]}] 2024-09-16T20:48:00.608 INFO:tasks.ceph_manager:config epoch is 7 2024-09-16T20:48:00.608 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-16T20:48:00.608 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-16T20:48:00.608 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-16T20:48:00.958 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:48:00.968 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":5,"flags":0,"active_gid":4099,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":4026083129},{"type":"v1","addr":"172.21.15.43:6825","nonce":4026083129}]},"active_addr":"172.21.15.43:6825/4026083129","active_change":"2024-09-16T20:47:44.196482+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":1582633930}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":1277892741}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":3669221069}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":2765656201}]}]} 2024-09-16T20:48:00.971 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-16T20:48:00.971 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-16T20:48:00.971 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T20:48:01.269 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:48:01.269 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":21,"fsid":"e28af4a7-5fcd-4de8-8f77-a2b575361bb1","created":"2024-09-16T20:47:39.760609+0000","modified":"2024-09-16T20:48:00.270717+0000","last_up_change":"2024-09-16T20:47:50.238013+0000","last_in_change":"2024-09-16T20:47:41.378753+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T20:47:52.220750+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T20:47:56.756666+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c3cb2d7-7d23-4566-9ca1-d7b08e12360c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":2744025949},{"type":"v1","addr":"172.21.15.29:6817","nonce":2744025949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":2744025949},{"type":"v1","addr":"172.21.15.29:6819","nonce":2744025949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":2744025949},{"type":"v1","addr":"172.21.15.29:6823","nonce":2744025949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":2744025949},{"type":"v1","addr":"172.21.15.29:6821","nonce":2744025949}]},"public_addr":"172.21.15.29:6817/2744025949","cluster_addr":"172.21.15.29:6819/2744025949","heartbeat_back_addr":"172.21.15.29:6823/2744025949","heartbeat_front_addr":"172.21.15.29:6821/2744025949","state":["exists","up"]},{"osd":1,"uuid":"5cf8c20c-1ed2-42ae-a8b0-c88cc4e36bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":208628762},{"type":"v1","addr":"172.21.15.29:6801","nonce":208628762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":208628762},{"type":"v1","addr":"172.21.15.29:6803","nonce":208628762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":208628762},{"type":"v1","addr":"172.21.15.29:6807","nonce":208628762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":208628762},{"type":"v1","addr":"172.21.15.29:6805","nonce":208628762}]},"public_addr":"172.21.15.29:6801/208628762","cluster_addr":"172.21.15.29:6803/208628762","heartbeat_back_addr":"172.21.15.29:6807/208628762","heartbeat_front_addr":"172.21.15.29:6805/208628762","state":["exists","up"]},{"osd":2,"uuid":"075d16da-82bf-449b-9f6e-542c9e8d810b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3447426536},{"type":"v1","addr":"172.21.15.29:6809","nonce":3447426536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3447426536},{"type":"v1","addr":"172.21.15.29:6811","nonce":3447426536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3447426536},{"type":"v1","addr":"172.21.15.29:6815","nonce":3447426536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":3447426536},{"type":"v1","addr":"172.21.15.29:6813","nonce":3447426536}]},"public_addr":"172.21.15.29:6809/3447426536","cluster_addr":"172.21.15.29:6811/3447426536","heartbeat_back_addr":"172.21.15.29:6815/3447426536","heartbeat_front_addr":"172.21.15.29:6813/3447426536","state":["exists","up"]},{"osd":3,"uuid":"0612be38-3a24-41b6-aba4-5a2fe985165e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3687583891},{"type":"v1","addr":"172.21.15.43:6809","nonce":3687583891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3687583891},{"type":"v1","addr":"172.21.15.43:6811","nonce":3687583891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3687583891},{"type":"v1","addr":"172.21.15.43:6815","nonce":3687583891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3687583891},{"type":"v1","addr":"172.21.15.43:6813","nonce":3687583891}]},"public_addr":"172.21.15.43:6809/3687583891","cluster_addr":"172.21.15.43:6811/3687583891","heartbeat_back_addr":"172.21.15.43:6815/3687583891","heartbeat_front_addr":"172.21.15.43:6813/3687583891","state":["exists","up"]},{"osd":4,"uuid":"be590bf8-e9fa-44ca-a471-6f6bcb087ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":2505220646},{"type":"v1","addr":"172.21.15.43:6801","nonce":2505220646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":2505220646},{"type":"v1","addr":"172.21.15.43:6803","nonce":2505220646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":2505220646},{"type":"v1","addr":"172.21.15.43:6807","nonce":2505220646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":2505220646},{"type":"v1","addr":"172.21.15.43:6805","nonce":2505220646}]},"public_addr":"172.21.15.43:6801/2505220646","cluster_addr":"172.21.15.43:6803/2505220646","heartbeat_back_addr":"172.21.15.43:6807/2505220646","heartbeat_front_addr":"172.21.15.43:6805/2505220646","state":["exists","up"]},{"osd":5,"uuid":"4a208e30-031b-4a9f-af52-e26d4a039ea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1517367307},{"type":"v1","addr":"172.21.15.43:6817","nonce":1517367307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":1517367307},{"type":"v1","addr":"172.21.15.43:6819","nonce":1517367307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":1517367307},{"type":"v1","addr":"172.21.15.43:6823","nonce":1517367307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1517367307},{"type":"v1","addr":"172.21.15.43:6821","nonce":1517367307}]},"public_addr":"172.21.15.43:6817/1517367307","cluster_addr":"172.21.15.43:6819/1517367307","heartbeat_back_addr":"172.21.15.43:6823/1517367307","heartbeat_front_addr":"172.21.15.43:6821/1517367307","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T20:47:47.887412+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T20:47:47.658782+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T20:47:47.453752+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T20:47:47.487992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T20:47:46.943344+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T20:47:47.901547+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T20:48:01.282 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-16T20:48:01.282 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T20:48:01.580 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:48:01.580 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":21,"fsid":"e28af4a7-5fcd-4de8-8f77-a2b575361bb1","created":"2024-09-16T20:47:39.760609+0000","modified":"2024-09-16T20:48:00.270717+0000","last_up_change":"2024-09-16T20:47:50.238013+0000","last_in_change":"2024-09-16T20:47:41.378753+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T20:47:52.220750+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T20:47:56.756666+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c3cb2d7-7d23-4566-9ca1-d7b08e12360c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":2744025949},{"type":"v1","addr":"172.21.15.29:6817","nonce":2744025949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":2744025949},{"type":"v1","addr":"172.21.15.29:6819","nonce":2744025949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":2744025949},{"type":"v1","addr":"172.21.15.29:6823","nonce":2744025949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":2744025949},{"type":"v1","addr":"172.21.15.29:6821","nonce":2744025949}]},"public_addr":"172.21.15.29:6817/2744025949","cluster_addr":"172.21.15.29:6819/2744025949","heartbeat_back_addr":"172.21.15.29:6823/2744025949","heartbeat_front_addr":"172.21.15.29:6821/2744025949","state":["exists","up"]},{"osd":1,"uuid":"5cf8c20c-1ed2-42ae-a8b0-c88cc4e36bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":208628762},{"type":"v1","addr":"172.21.15.29:6801","nonce":208628762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":208628762},{"type":"v1","addr":"172.21.15.29:6803","nonce":208628762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":208628762},{"type":"v1","addr":"172.21.15.29:6807","nonce":208628762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":208628762},{"type":"v1","addr":"172.21.15.29:6805","nonce":208628762}]},"public_addr":"172.21.15.29:6801/208628762","cluster_addr":"172.21.15.29:6803/208628762","heartbeat_back_addr":"172.21.15.29:6807/208628762","heartbeat_front_addr":"172.21.15.29:6805/208628762","state":["exists","up"]},{"osd":2,"uuid":"075d16da-82bf-449b-9f6e-542c9e8d810b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3447426536},{"type":"v1","addr":"172.21.15.29:6809","nonce":3447426536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3447426536},{"type":"v1","addr":"172.21.15.29:6811","nonce":3447426536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3447426536},{"type":"v1","addr":"172.21.15.29:6815","nonce":3447426536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":3447426536},{"type":"v1","addr":"172.21.15.29:6813","nonce":3447426536}]},"public_addr":"172.21.15.29:6809/3447426536","cluster_addr":"172.21.15.29:6811/3447426536","heartbeat_back_addr":"172.21.15.29:6815/3447426536","heartbeat_front_addr":"172.21.15.29:6813/3447426536","state":["exists","up"]},{"osd":3,"uuid":"0612be38-3a24-41b6-aba4-5a2fe985165e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3687583891},{"type":"v1","addr":"172.21.15.43:6809","nonce":3687583891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3687583891},{"type":"v1","addr":"172.21.15.43:6811","nonce":3687583891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3687583891},{"type":"v1","addr":"172.21.15.43:6815","nonce":3687583891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3687583891},{"type":"v1","addr":"172.21.15.43:6813","nonce":3687583891}]},"public_addr":"172.21.15.43:6809/3687583891","cluster_addr":"172.21.15.43:6811/3687583891","heartbeat_back_addr":"172.21.15.43:6815/3687583891","heartbeat_front_addr":"172.21.15.43:6813/3687583891","state":["exists","up"]},{"osd":4,"uuid":"be590bf8-e9fa-44ca-a471-6f6bcb087ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":2505220646},{"type":"v1","addr":"172.21.15.43:6801","nonce":2505220646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":2505220646},{"type":"v1","addr":"172.21.15.43:6803","nonce":2505220646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":2505220646},{"type":"v1","addr":"172.21.15.43:6807","nonce":2505220646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":2505220646},{"type":"v1","addr":"172.21.15.43:6805","nonce":2505220646}]},"public_addr":"172.21.15.43:6801/2505220646","cluster_addr":"172.21.15.43:6803/2505220646","heartbeat_back_addr":"172.21.15.43:6807/2505220646","heartbeat_front_addr":"172.21.15.43:6805/2505220646","state":["exists","up"]},{"osd":5,"uuid":"4a208e30-031b-4a9f-af52-e26d4a039ea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1517367307},{"type":"v1","addr":"172.21.15.43:6817","nonce":1517367307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":1517367307},{"type":"v1","addr":"172.21.15.43:6819","nonce":1517367307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":1517367307},{"type":"v1","addr":"172.21.15.43:6823","nonce":1517367307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1517367307},{"type":"v1","addr":"172.21.15.43:6821","nonce":1517367307}]},"public_addr":"172.21.15.43:6817/1517367307","cluster_addr":"172.21.15.43:6819/1517367307","heartbeat_back_addr":"172.21.15.43:6823/1517367307","heartbeat_front_addr":"172.21.15.43:6821/1517367307","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T20:47:47.887412+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T20:47:47.658782+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T20:47:47.453752+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T20:47:47.487992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T20:47:46.943344+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T20:47:47.901547+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T20:48:01.593 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-16T20:48:01.593 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-16T20:48:01.594 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-16T20:48:01.594 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-16T20:48:01.595 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-16T20:48:01.595 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-16T20:48:01.770 INFO:teuthology.orchestra.run.smithi029.stdout:55834574853 2024-09-16T20:48:01.770 INFO:teuthology.orchestra.run.smithi029.stdout:47244640261 2024-09-16T20:48:01.771 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-16T20:48:01.771 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-16T20:48:01.805 INFO:teuthology.orchestra.run.smithi029.stdout:55834574854 2024-09-16T20:48:01.806 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-16T20:48:01.810 INFO:teuthology.orchestra.run.smithi029.stdout:55834574854 2024-09-16T20:48:01.811 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T20:48:01.816 INFO:teuthology.orchestra.run.smithi029.stdout:55834574854 2024-09-16T20:48:01.817 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-16T20:48:01.823 INFO:teuthology.orchestra.run.smithi029.stdout:55834574854 2024-09-16T20:48:01.823 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-16T20:48:02.155 INFO:teuthology.orchestra.run.smithi029.stdout:55834574852 2024-09-16T20:48:02.171 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574852 for osd.3 2024-09-16T20:48:02.178 INFO:teuthology.orchestra.run.smithi029.stdout:55834574852 2024-09-16T20:48:02.191 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574852 for osd.5 2024-09-16T20:48:02.268 INFO:teuthology.orchestra.run.smithi029.stdout:55834574851 2024-09-16T20:48:02.282 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574851 for osd.1 2024-09-16T20:48:02.300 INFO:teuthology.orchestra.run.smithi029.stdout:47244640261 2024-09-16T20:48:02.307 INFO:teuthology.orchestra.run.smithi029.stdout:55834574854 2024-09-16T20:48:02.314 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640261 for osd.4 2024-09-16T20:48:02.314 DEBUG:teuthology.parallel:result is None 2024-09-16T20:48:02.322 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.0 2024-09-16T20:48:02.322 DEBUG:teuthology.parallel:result is None 2024-09-16T20:48:02.328 INFO:teuthology.orchestra.run.smithi029.stdout:55834574854 2024-09-16T20:48:02.340 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.2 2024-09-16T20:48:02.341 DEBUG:teuthology.parallel:result is None 2024-09-16T20:48:03.172 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-16T20:48:03.192 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-16T20:48:03.283 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-16T20:48:03.542 INFO:teuthology.orchestra.run.smithi029.stdout:55834574854 2024-09-16T20:48:03.555 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.3 2024-09-16T20:48:03.555 DEBUG:teuthology.parallel:result is None 2024-09-16T20:48:03.574 INFO:teuthology.orchestra.run.smithi029.stdout:55834574854 2024-09-16T20:48:03.587 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.5 2024-09-16T20:48:03.587 DEBUG:teuthology.parallel:result is None 2024-09-16T20:48:03.591 INFO:teuthology.orchestra.run.smithi029.stdout:55834574853 2024-09-16T20:48:03.604 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.1 2024-09-16T20:48:03.604 DEBUG:teuthology.parallel:result is None 2024-09-16T20:48:03.604 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T20:48:03.604 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T20:48:03.938 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:48:03.938 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-09-16T20:48:03.950 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-16T20:48:02.205299+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":201932,"kb_used_data":2296,"kb_used_omap":9,"kb_used_meta":199478,"kb_avail":566029108,"statfs":{"total":579820584960,"available":579613806592,"internally_reserved":0,"allocated":2351104,"data_stored":1565086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9567,"internal_metadata":204266145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.942725"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T20:48:00.277670+0000","last_change":"2024-09-16T20:48:00.277670+0000","last_active":"2024-09-16T20:48:00.277670+0000","last_peered":"2024-09-16T20:48:00.277670+0000","last_clean":"2024-09-16T20:48:00.277670+0000","last_became_active":"2024-09-16T20:47:58.274489+0000","last_became_peered":"2024-09-16T20:47:58.274489+0000","last_unstale":"2024-09-16T20:48:00.277670+0000","last_undegraded":"2024-09-16T20:48:00.277670+0000","last_fullsized":"2024-09-16T20:48:00.277670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:27:12.633064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091339500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T20:48:00.276938+0000","last_change":"2024-09-16T20:48:00.277182+0000","last_active":"2024-09-16T20:48:00.276938+0000","last_peered":"2024-09-16T20:48:00.276938+0000","last_clean":"2024-09-16T20:48:00.276938+0000","last_became_active":"2024-09-16T20:47:58.274389+0000","last_became_peered":"2024-09-16T20:47:58.274389+0000","last_unstale":"2024-09-16T20:48:00.276938+0000","last_undegraded":"2024-09-16T20:48:00.276938+0000","last_fullsized":"2024-09-16T20:48:00.276938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:28:02.441830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043659299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T20:48:00.278054+0000","last_change":"2024-09-16T20:48:00.278054+0000","last_active":"2024-09-16T20:48:00.278054+0000","last_peered":"2024-09-16T20:48:00.278054+0000","last_clean":"2024-09-16T20:48:00.278054+0000","last_became_active":"2024-09-16T20:47:58.272861+0000","last_became_peered":"2024-09-16T20:47:58.272861+0000","last_unstale":"2024-09-16T20:48:00.278054+0000","last_undegraded":"2024-09-16T20:48:00.278054+0000","last_fullsized":"2024-09-16T20:48:00.278054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:13:59.135547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012429489999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T20:48:00.833899+0000","last_change":"2024-09-16T20:48:00.833899+0000","last_active":"2024-09-16T20:48:00.833899+0000","last_peered":"2024-09-16T20:48:00.833899+0000","last_clean":"2024-09-16T20:48:00.833899+0000","last_became_active":"2024-09-16T20:47:58.275031+0000","last_became_peered":"2024-09-16T20:47:58.275031+0000","last_unstale":"2024-09-16T20:48:00.833899+0000","last_undegraded":"2024-09-16T20:48:00.833899+0000","last_fullsized":"2024-09-16T20:48:00.833899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:13:08.658660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046986,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T20:48:00.279477+0000","last_change":"2024-09-16T20:48:00.276970+0000","last_active":"2024-09-16T20:48:00.279477+0000","last_peered":"2024-09-16T20:48:00.279477+0000","last_clean":"2024-09-16T20:48:00.279477+0000","last_became_active":"2024-09-16T20:47:58.274617+0000","last_became_peered":"2024-09-16T20:47:58.274617+0000","last_unstale":"2024-09-16T20:48:00.279477+0000","last_undegraded":"2024-09-16T20:48:00.279477+0000","last_fullsized":"2024-09-16T20:48:00.279477+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:28:54.467905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062077499999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T20:48:01.638601+0000","last_change":"2024-09-16T20:48:01.638893+0000","last_active":"2024-09-16T20:48:01.638601+0000","last_peered":"2024-09-16T20:48:01.638601+0000","last_clean":"2024-09-16T20:48:01.638601+0000","last_became_active":"2024-09-16T20:47:58.273088+0000","last_became_peered":"2024-09-16T20:47:58.273088+0000","last_unstale":"2024-09-16T20:48:01.638601+0000","last_undegraded":"2024-09-16T20:48:01.638601+0000","last_fullsized":"2024-09-16T20:48:01.638601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:27:51.451170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000479672,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T20:48:00.408392+0000","last_change":"2024-09-16T20:48:00.408392+0000","last_active":"2024-09-16T20:48:00.408392+0000","last_peered":"2024-09-16T20:48:00.408392+0000","last_clean":"2024-09-16T20:48:00.408392+0000","last_became_active":"2024-09-16T20:47:58.273260+0000","last_became_peered":"2024-09-16T20:47:58.273260+0000","last_unstale":"2024-09-16T20:48:00.408392+0000","last_undegraded":"2024-09-16T20:48:00.408392+0000","last_fullsized":"2024-09-16T20:48:00.408392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:01:53.009880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037636000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T20:48:00.277739+0000","last_change":"2024-09-16T20:48:00.277739+0000","last_active":"2024-09-16T20:48:00.277739+0000","last_peered":"2024-09-16T20:48:00.277739+0000","last_clean":"2024-09-16T20:48:00.277739+0000","last_became_active":"2024-09-16T20:47:58.271174+0000","last_became_peered":"2024-09-16T20:47:58.271174+0000","last_unstale":"2024-09-16T20:48:00.277739+0000","last_undegraded":"2024-09-16T20:48:00.277739+0000","last_fullsized":"2024-09-16T20:48:00.277739+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:11:35.985075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095771799999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T20:48:00.277219+0000","last_change":"2024-09-16T20:47:53.928484+0000","last_active":"2024-09-16T20:48:00.277219+0000","last_peered":"2024-09-16T20:48:00.277219+0000","last_clean":"2024-09-16T20:48:00.277219+0000","last_became_active":"2024-09-16T20:47:53.926422+0000","last_became_peered":"2024-09-16T20:47:53.926422+0000","last_unstale":"2024-09-16T20:48:00.277219+0000","last_undegraded":"2024-09-16T20:48:00.277219+0000","last_fullsized":"2024-09-16T20:48:00.277219+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:52.241408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:52.241408+0000","last_clean_scrub_stamp":"2024-09-16T20:47:52.241408+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:18:04.963138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36112,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335728,"statfs":{"total":96636764160,"available":96599785472,"internally_reserved":0,"allocated":778240,"data_stored":651108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35532,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336308,"statfs":{"total":96636764160,"available":96600379392,"internally_reserved":0,"allocated":184320,"data_stored":60740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":196608,"data_stored":65708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31968,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339872,"statfs":{"total":96636764160,"available":96604028928,"internally_reserved":0,"allocated":790528,"data_stored":656076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":200704,"data_stored":65727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640261,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35544,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336296,"statfs":{"total":96636764160,"available":96600367104,"internally_reserved":0,"allocated":200704,"data_stored":65727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T20:48:03.951 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T20:48:04.247 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:48:04.248 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-09-16T20:48:04.259 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-16T20:48:04.205841+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":201932,"kb_used_data":2296,"kb_used_omap":9,"kb_used_meta":199478,"kb_avail":566029108,"statfs":{"total":579820584960,"available":579613806592,"internally_reserved":0,"allocated":2351104,"data_stored":1565086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9567,"internal_metadata":204266145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001451"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T20:48:00.277670+0000","last_change":"2024-09-16T20:48:00.277670+0000","last_active":"2024-09-16T20:48:00.277670+0000","last_peered":"2024-09-16T20:48:00.277670+0000","last_clean":"2024-09-16T20:48:00.277670+0000","last_became_active":"2024-09-16T20:47:58.274489+0000","last_became_peered":"2024-09-16T20:47:58.274489+0000","last_unstale":"2024-09-16T20:48:00.277670+0000","last_undegraded":"2024-09-16T20:48:00.277670+0000","last_fullsized":"2024-09-16T20:48:00.277670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:27:12.633064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091339500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T20:48:00.276938+0000","last_change":"2024-09-16T20:48:00.277182+0000","last_active":"2024-09-16T20:48:00.276938+0000","last_peered":"2024-09-16T20:48:00.276938+0000","last_clean":"2024-09-16T20:48:00.276938+0000","last_became_active":"2024-09-16T20:47:58.274389+0000","last_became_peered":"2024-09-16T20:47:58.274389+0000","last_unstale":"2024-09-16T20:48:00.276938+0000","last_undegraded":"2024-09-16T20:48:00.276938+0000","last_fullsized":"2024-09-16T20:48:00.276938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:28:02.441830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043659299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T20:48:00.278054+0000","last_change":"2024-09-16T20:48:00.278054+0000","last_active":"2024-09-16T20:48:00.278054+0000","last_peered":"2024-09-16T20:48:00.278054+0000","last_clean":"2024-09-16T20:48:00.278054+0000","last_became_active":"2024-09-16T20:47:58.272861+0000","last_became_peered":"2024-09-16T20:47:58.272861+0000","last_unstale":"2024-09-16T20:48:00.278054+0000","last_undegraded":"2024-09-16T20:48:00.278054+0000","last_fullsized":"2024-09-16T20:48:00.278054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:13:59.135547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012429489999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T20:48:00.833899+0000","last_change":"2024-09-16T20:48:00.833899+0000","last_active":"2024-09-16T20:48:00.833899+0000","last_peered":"2024-09-16T20:48:00.833899+0000","last_clean":"2024-09-16T20:48:00.833899+0000","last_became_active":"2024-09-16T20:47:58.275031+0000","last_became_peered":"2024-09-16T20:47:58.275031+0000","last_unstale":"2024-09-16T20:48:00.833899+0000","last_undegraded":"2024-09-16T20:48:00.833899+0000","last_fullsized":"2024-09-16T20:48:00.833899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:13:08.658660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046986,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T20:48:00.279477+0000","last_change":"2024-09-16T20:48:00.276970+0000","last_active":"2024-09-16T20:48:00.279477+0000","last_peered":"2024-09-16T20:48:00.279477+0000","last_clean":"2024-09-16T20:48:00.279477+0000","last_became_active":"2024-09-16T20:47:58.274617+0000","last_became_peered":"2024-09-16T20:47:58.274617+0000","last_unstale":"2024-09-16T20:48:00.279477+0000","last_undegraded":"2024-09-16T20:48:00.279477+0000","last_fullsized":"2024-09-16T20:48:00.279477+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:28:54.467905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062077499999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T20:48:01.638601+0000","last_change":"2024-09-16T20:48:01.638893+0000","last_active":"2024-09-16T20:48:01.638601+0000","last_peered":"2024-09-16T20:48:01.638601+0000","last_clean":"2024-09-16T20:48:01.638601+0000","last_became_active":"2024-09-16T20:47:58.273088+0000","last_became_peered":"2024-09-16T20:47:58.273088+0000","last_unstale":"2024-09-16T20:48:01.638601+0000","last_undegraded":"2024-09-16T20:48:01.638601+0000","last_fullsized":"2024-09-16T20:48:01.638601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:27:51.451170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000479672,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T20:48:00.408392+0000","last_change":"2024-09-16T20:48:00.408392+0000","last_active":"2024-09-16T20:48:00.408392+0000","last_peered":"2024-09-16T20:48:00.408392+0000","last_clean":"2024-09-16T20:48:00.408392+0000","last_became_active":"2024-09-16T20:47:58.273260+0000","last_became_peered":"2024-09-16T20:47:58.273260+0000","last_unstale":"2024-09-16T20:48:00.408392+0000","last_undegraded":"2024-09-16T20:48:00.408392+0000","last_fullsized":"2024-09-16T20:48:00.408392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:01:53.009880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037636000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T20:48:00.277739+0000","last_change":"2024-09-16T20:48:00.277739+0000","last_active":"2024-09-16T20:48:00.277739+0000","last_peered":"2024-09-16T20:48:00.277739+0000","last_clean":"2024-09-16T20:48:00.277739+0000","last_became_active":"2024-09-16T20:47:58.271174+0000","last_became_peered":"2024-09-16T20:47:58.271174+0000","last_unstale":"2024-09-16T20:48:00.277739+0000","last_undegraded":"2024-09-16T20:48:00.277739+0000","last_fullsized":"2024-09-16T20:48:00.277739+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:11:35.985075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095771799999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T20:48:00.277219+0000","last_change":"2024-09-16T20:47:53.928484+0000","last_active":"2024-09-16T20:48:00.277219+0000","last_peered":"2024-09-16T20:48:00.277219+0000","last_clean":"2024-09-16T20:48:00.277219+0000","last_became_active":"2024-09-16T20:47:53.926422+0000","last_became_peered":"2024-09-16T20:47:53.926422+0000","last_unstale":"2024-09-16T20:48:00.277219+0000","last_undegraded":"2024-09-16T20:48:00.277219+0000","last_fullsized":"2024-09-16T20:48:00.277219+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:52.241408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:52.241408+0000","last_clean_scrub_stamp":"2024-09-16T20:47:52.241408+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:18:04.963138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36112,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335728,"statfs":{"total":96636764160,"available":96599785472,"internally_reserved":0,"allocated":778240,"data_stored":651108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35532,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336308,"statfs":{"total":96636764160,"available":96600379392,"internally_reserved":0,"allocated":184320,"data_stored":60740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":196608,"data_stored":65708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31968,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339872,"statfs":{"total":96636764160,"available":96604028928,"internally_reserved":0,"allocated":790528,"data_stored":656076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":200704,"data_stored":65727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640262,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35544,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336296,"statfs":{"total":96636764160,"available":96600367104,"internally_reserved":0,"allocated":200704,"data_stored":65727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T20:48:04.260 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T20:48:04.260 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-16T20:48:04.261 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-16T20:48:04.261 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-16T20:48:04.597 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:48:04.597 INFO:teuthology.orchestra.run.smithi029.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-16T20:48:04.610 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-16T20:48:04.610 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-16T20:48:04.620 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-16T20:48:04.621 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'wip-dis-testing'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': '6e8474a02f2b86353394076ab80c6f50ea28e361'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'extra_packages': ['rbd-nbd'], 'flavor': 'default', 'sha1': '6e8474a02f2b86353394076ab80c6f50ea28e361'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-dis-testing', 'sha1': '6e8474a02f2b86353394076ab80c6f50ea28e361'}} 2024-09-16T20:48:04.621 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-16T20:48:04.621 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-16T20:48:04.938 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-16T20:48:04.938 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T20:48:04.939 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T20:48:05.234 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:48:05.235 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-09-16T20:48:05.247 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-09-16T20:48:04.546390+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":201932,"kb_used_data":2296,"kb_used_omap":9,"kb_used_meta":199478,"kb_avail":566029108,"statfs":{"total":579820584960,"available":579613806592,"internally_reserved":0,"allocated":2351104,"data_stored":1565086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9567,"internal_metadata":204266145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.279610"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T20:48:00.277670+0000","last_change":"2024-09-16T20:48:00.277670+0000","last_active":"2024-09-16T20:48:00.277670+0000","last_peered":"2024-09-16T20:48:00.277670+0000","last_clean":"2024-09-16T20:48:00.277670+0000","last_became_active":"2024-09-16T20:47:58.274489+0000","last_became_peered":"2024-09-16T20:47:58.274489+0000","last_unstale":"2024-09-16T20:48:00.277670+0000","last_undegraded":"2024-09-16T20:48:00.277670+0000","last_fullsized":"2024-09-16T20:48:00.277670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:27:12.633064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091339500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T20:48:00.276938+0000","last_change":"2024-09-16T20:48:00.277182+0000","last_active":"2024-09-16T20:48:00.276938+0000","last_peered":"2024-09-16T20:48:00.276938+0000","last_clean":"2024-09-16T20:48:00.276938+0000","last_became_active":"2024-09-16T20:47:58.274389+0000","last_became_peered":"2024-09-16T20:47:58.274389+0000","last_unstale":"2024-09-16T20:48:00.276938+0000","last_undegraded":"2024-09-16T20:48:00.276938+0000","last_fullsized":"2024-09-16T20:48:00.276938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:28:02.441830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043659299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T20:48:00.278054+0000","last_change":"2024-09-16T20:48:00.278054+0000","last_active":"2024-09-16T20:48:00.278054+0000","last_peered":"2024-09-16T20:48:00.278054+0000","last_clean":"2024-09-16T20:48:00.278054+0000","last_became_active":"2024-09-16T20:47:58.272861+0000","last_became_peered":"2024-09-16T20:47:58.272861+0000","last_unstale":"2024-09-16T20:48:00.278054+0000","last_undegraded":"2024-09-16T20:48:00.278054+0000","last_fullsized":"2024-09-16T20:48:00.278054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:13:59.135547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012429489999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T20:48:00.833899+0000","last_change":"2024-09-16T20:48:00.833899+0000","last_active":"2024-09-16T20:48:00.833899+0000","last_peered":"2024-09-16T20:48:00.833899+0000","last_clean":"2024-09-16T20:48:00.833899+0000","last_became_active":"2024-09-16T20:47:58.275031+0000","last_became_peered":"2024-09-16T20:47:58.275031+0000","last_unstale":"2024-09-16T20:48:00.833899+0000","last_undegraded":"2024-09-16T20:48:00.833899+0000","last_fullsized":"2024-09-16T20:48:00.833899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:13:08.658660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046986,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T20:48:00.279477+0000","last_change":"2024-09-16T20:48:00.276970+0000","last_active":"2024-09-16T20:48:00.279477+0000","last_peered":"2024-09-16T20:48:00.279477+0000","last_clean":"2024-09-16T20:48:00.279477+0000","last_became_active":"2024-09-16T20:47:58.274617+0000","last_became_peered":"2024-09-16T20:47:58.274617+0000","last_unstale":"2024-09-16T20:48:00.279477+0000","last_undegraded":"2024-09-16T20:48:00.279477+0000","last_fullsized":"2024-09-16T20:48:00.279477+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:28:54.467905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062077499999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T20:48:01.638601+0000","last_change":"2024-09-16T20:48:01.638893+0000","last_active":"2024-09-16T20:48:01.638601+0000","last_peered":"2024-09-16T20:48:01.638601+0000","last_clean":"2024-09-16T20:48:01.638601+0000","last_became_active":"2024-09-16T20:47:58.273088+0000","last_became_peered":"2024-09-16T20:47:58.273088+0000","last_unstale":"2024-09-16T20:48:01.638601+0000","last_undegraded":"2024-09-16T20:48:01.638601+0000","last_fullsized":"2024-09-16T20:48:01.638601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:27:51.451170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000479672,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T20:48:00.408392+0000","last_change":"2024-09-16T20:48:00.408392+0000","last_active":"2024-09-16T20:48:00.408392+0000","last_peered":"2024-09-16T20:48:00.408392+0000","last_clean":"2024-09-16T20:48:00.408392+0000","last_became_active":"2024-09-16T20:47:58.273260+0000","last_became_peered":"2024-09-16T20:47:58.273260+0000","last_unstale":"2024-09-16T20:48:00.408392+0000","last_undegraded":"2024-09-16T20:48:00.408392+0000","last_fullsized":"2024-09-16T20:48:00.408392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:01:53.009880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037636000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T20:48:00.277739+0000","last_change":"2024-09-16T20:48:00.277739+0000","last_active":"2024-09-16T20:48:00.277739+0000","last_peered":"2024-09-16T20:48:00.277739+0000","last_clean":"2024-09-16T20:48:00.277739+0000","last_became_active":"2024-09-16T20:47:58.271174+0000","last_became_peered":"2024-09-16T20:47:58.271174+0000","last_unstale":"2024-09-16T20:48:00.277739+0000","last_undegraded":"2024-09-16T20:48:00.277739+0000","last_fullsized":"2024-09-16T20:48:00.277739+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:11:35.985075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095771799999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T20:48:00.277219+0000","last_change":"2024-09-16T20:47:53.928484+0000","last_active":"2024-09-16T20:48:00.277219+0000","last_peered":"2024-09-16T20:48:00.277219+0000","last_clean":"2024-09-16T20:48:00.277219+0000","last_became_active":"2024-09-16T20:47:53.926422+0000","last_became_peered":"2024-09-16T20:47:53.926422+0000","last_unstale":"2024-09-16T20:48:00.277219+0000","last_undegraded":"2024-09-16T20:48:00.277219+0000","last_fullsized":"2024-09-16T20:48:00.277219+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:52.241408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:52.241408+0000","last_clean_scrub_stamp":"2024-09-16T20:47:52.241408+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:18:04.963138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36112,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335728,"statfs":{"total":96636764160,"available":96599785472,"internally_reserved":0,"allocated":778240,"data_stored":651108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35532,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336308,"statfs":{"total":96636764160,"available":96600379392,"internally_reserved":0,"allocated":184320,"data_stored":60740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":196608,"data_stored":65708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31968,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339872,"statfs":{"total":96636764160,"available":96604028928,"internally_reserved":0,"allocated":790528,"data_stored":656076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":200704,"data_stored":65727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640262,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35544,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336296,"statfs":{"total":96636764160,"available":96600367104,"internally_reserved":0,"allocated":200704,"data_stored":65727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T20:48:05.248 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T20:48:05.543 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:48:05.544 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-09-16T20:48:05.557 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-09-16T20:48:04.546390+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":201932,"kb_used_data":2296,"kb_used_omap":9,"kb_used_meta":199478,"kb_avail":566029108,"statfs":{"total":579820584960,"available":579613806592,"internally_reserved":0,"allocated":2351104,"data_stored":1565086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9567,"internal_metadata":204266145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.279610"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T20:48:00.277670+0000","last_change":"2024-09-16T20:48:00.277670+0000","last_active":"2024-09-16T20:48:00.277670+0000","last_peered":"2024-09-16T20:48:00.277670+0000","last_clean":"2024-09-16T20:48:00.277670+0000","last_became_active":"2024-09-16T20:47:58.274489+0000","last_became_peered":"2024-09-16T20:47:58.274489+0000","last_unstale":"2024-09-16T20:48:00.277670+0000","last_undegraded":"2024-09-16T20:48:00.277670+0000","last_fullsized":"2024-09-16T20:48:00.277670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:27:12.633064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091339500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T20:48:00.276938+0000","last_change":"2024-09-16T20:48:00.277182+0000","last_active":"2024-09-16T20:48:00.276938+0000","last_peered":"2024-09-16T20:48:00.276938+0000","last_clean":"2024-09-16T20:48:00.276938+0000","last_became_active":"2024-09-16T20:47:58.274389+0000","last_became_peered":"2024-09-16T20:47:58.274389+0000","last_unstale":"2024-09-16T20:48:00.276938+0000","last_undegraded":"2024-09-16T20:48:00.276938+0000","last_fullsized":"2024-09-16T20:48:00.276938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:28:02.441830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043659299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T20:48:00.278054+0000","last_change":"2024-09-16T20:48:00.278054+0000","last_active":"2024-09-16T20:48:00.278054+0000","last_peered":"2024-09-16T20:48:00.278054+0000","last_clean":"2024-09-16T20:48:00.278054+0000","last_became_active":"2024-09-16T20:47:58.272861+0000","last_became_peered":"2024-09-16T20:47:58.272861+0000","last_unstale":"2024-09-16T20:48:00.278054+0000","last_undegraded":"2024-09-16T20:48:00.278054+0000","last_fullsized":"2024-09-16T20:48:00.278054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:13:59.135547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012429489999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T20:48:00.833899+0000","last_change":"2024-09-16T20:48:00.833899+0000","last_active":"2024-09-16T20:48:00.833899+0000","last_peered":"2024-09-16T20:48:00.833899+0000","last_clean":"2024-09-16T20:48:00.833899+0000","last_became_active":"2024-09-16T20:47:58.275031+0000","last_became_peered":"2024-09-16T20:47:58.275031+0000","last_unstale":"2024-09-16T20:48:00.833899+0000","last_undegraded":"2024-09-16T20:48:00.833899+0000","last_fullsized":"2024-09-16T20:48:00.833899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:13:08.658660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046986,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T20:48:00.279477+0000","last_change":"2024-09-16T20:48:00.276970+0000","last_active":"2024-09-16T20:48:00.279477+0000","last_peered":"2024-09-16T20:48:00.279477+0000","last_clean":"2024-09-16T20:48:00.279477+0000","last_became_active":"2024-09-16T20:47:58.274617+0000","last_became_peered":"2024-09-16T20:47:58.274617+0000","last_unstale":"2024-09-16T20:48:00.279477+0000","last_undegraded":"2024-09-16T20:48:00.279477+0000","last_fullsized":"2024-09-16T20:48:00.279477+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:28:54.467905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062077499999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T20:48:01.638601+0000","last_change":"2024-09-16T20:48:01.638893+0000","last_active":"2024-09-16T20:48:01.638601+0000","last_peered":"2024-09-16T20:48:01.638601+0000","last_clean":"2024-09-16T20:48:01.638601+0000","last_became_active":"2024-09-16T20:47:58.273088+0000","last_became_peered":"2024-09-16T20:47:58.273088+0000","last_unstale":"2024-09-16T20:48:01.638601+0000","last_undegraded":"2024-09-16T20:48:01.638601+0000","last_fullsized":"2024-09-16T20:48:01.638601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:27:51.451170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000479672,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T20:48:00.408392+0000","last_change":"2024-09-16T20:48:00.408392+0000","last_active":"2024-09-16T20:48:00.408392+0000","last_peered":"2024-09-16T20:48:00.408392+0000","last_clean":"2024-09-16T20:48:00.408392+0000","last_became_active":"2024-09-16T20:47:58.273260+0000","last_became_peered":"2024-09-16T20:47:58.273260+0000","last_unstale":"2024-09-16T20:48:00.408392+0000","last_undegraded":"2024-09-16T20:48:00.408392+0000","last_fullsized":"2024-09-16T20:48:00.408392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:01:53.009880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037636000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T20:48:00.277739+0000","last_change":"2024-09-16T20:48:00.277739+0000","last_active":"2024-09-16T20:48:00.277739+0000","last_peered":"2024-09-16T20:48:00.277739+0000","last_clean":"2024-09-16T20:48:00.277739+0000","last_became_active":"2024-09-16T20:47:58.271174+0000","last_became_peered":"2024-09-16T20:47:58.271174+0000","last_unstale":"2024-09-16T20:48:00.277739+0000","last_undegraded":"2024-09-16T20:48:00.277739+0000","last_fullsized":"2024-09-16T20:48:00.277739+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:11:35.985075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095771799999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T20:48:00.277219+0000","last_change":"2024-09-16T20:47:53.928484+0000","last_active":"2024-09-16T20:48:00.277219+0000","last_peered":"2024-09-16T20:48:00.277219+0000","last_clean":"2024-09-16T20:48:00.277219+0000","last_became_active":"2024-09-16T20:47:53.926422+0000","last_became_peered":"2024-09-16T20:47:53.926422+0000","last_unstale":"2024-09-16T20:48:00.277219+0000","last_undegraded":"2024-09-16T20:48:00.277219+0000","last_fullsized":"2024-09-16T20:48:00.277219+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:52.241408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:52.241408+0000","last_clean_scrub_stamp":"2024-09-16T20:47:52.241408+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:18:04.963138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36112,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335728,"statfs":{"total":96636764160,"available":96599785472,"internally_reserved":0,"allocated":778240,"data_stored":651108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35532,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336308,"statfs":{"total":96636764160,"available":96600379392,"internally_reserved":0,"allocated":184320,"data_stored":60740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":196608,"data_stored":65708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31968,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339872,"statfs":{"total":96636764160,"available":96604028928,"internally_reserved":0,"allocated":790528,"data_stored":656076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":200704,"data_stored":65727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640262,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35544,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336296,"statfs":{"total":96636764160,"available":96600367104,"internally_reserved":0,"allocated":200704,"data_stored":65727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T20:48:05.558 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T20:48:05.558 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-16T20:48:05.853 INFO:teuthology.orchestra.run.smithi029.stdout:epoch 22 2024-09-16T20:48:05.853 INFO:teuthology.orchestra.run.smithi029.stdout:fsid e28af4a7-5fcd-4de8-8f77-a2b575361bb1 2024-09-16T20:48:05.853 INFO:teuthology.orchestra.run.smithi029.stdout:created 2024-09-16T20:47:39.760609+0000 2024-09-16T20:48:05.853 INFO:teuthology.orchestra.run.smithi029.stdout:modified 2024-09-16T20:48:04.544574+0000 2024-09-16T20:48:05.853 INFO:teuthology.orchestra.run.smithi029.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-16T20:48:05.853 INFO:teuthology.orchestra.run.smithi029.stdout:crush_version 5 2024-09-16T20:48:05.853 INFO:teuthology.orchestra.run.smithi029.stdout:full_ratio 0.95 2024-09-16T20:48:05.854 INFO:teuthology.orchestra.run.smithi029.stdout:backfillfull_ratio 0.9 2024-09-16T20:48:05.854 INFO:teuthology.orchestra.run.smithi029.stdout:nearfull_ratio 0.85 2024-09-16T20:48:05.854 INFO:teuthology.orchestra.run.smithi029.stdout:require_min_compat_client luminous 2024-09-16T20:48:05.854 INFO:teuthology.orchestra.run.smithi029.stdout:min_compat_client jewel 2024-09-16T20:48:05.854 INFO:teuthology.orchestra.run.smithi029.stdout:require_osd_release squid 2024-09-16T20:48:05.854 INFO:teuthology.orchestra.run.smithi029.stdout:stretch_mode_enabled false 2024-09-16T20:48:05.854 INFO:teuthology.orchestra.run.smithi029.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 17 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 6.06 2024-09-16T20:48:05.854 INFO:teuthology.orchestra.run.smithi029.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 21 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 2.25 2024-09-16T20:48:05.854 INFO:teuthology.orchestra.run.smithi029.stdout:max_osd 6 2024-09-16T20:48:05.854 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.29:6816/2744025949,v1:172.21.15.29:6817/2744025949] [v2:172.21.15.29:6818/2744025949,v1:172.21.15.29:6819/2744025949] exists,up 6c3cb2d7-7d23-4566-9ca1-d7b08e12360c 2024-09-16T20:48:05.854 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.29:6800/208628762,v1:172.21.15.29:6801/208628762] [v2:172.21.15.29:6802/208628762,v1:172.21.15.29:6803/208628762] exists,up 5cf8c20c-1ed2-42ae-a8b0-c88cc4e36bce 2024-09-16T20:48:05.855 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.29:6808/3447426536,v1:172.21.15.29:6809/3447426536] [v2:172.21.15.29:6810/3447426536,v1:172.21.15.29:6811/3447426536] exists,up 075d16da-82bf-449b-9f6e-542c9e8d810b 2024-09-16T20:48:05.855 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.43:6808/3687583891,v1:172.21.15.43:6809/3687583891] [v2:172.21.15.43:6810/3687583891,v1:172.21.15.43:6811/3687583891] exists,up 0612be38-3a24-41b6-aba4-5a2fe985165e 2024-09-16T20:48:05.855 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4 up in weight 1 up_from 11 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.43:6800/2505220646,v1:172.21.15.43:6801/2505220646] [v2:172.21.15.43:6802/2505220646,v1:172.21.15.43:6803/2505220646] exists,up be590bf8-e9fa-44ca-a471-6f6bcb087ca7 2024-09-16T20:48:05.855 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.43:6816/1517367307,v1:172.21.15.43:6817/1517367307] [v2:172.21.15.43:6818/1517367307,v1:172.21.15.43:6819/1517367307] exists,up 4a208e30-031b-4a9f-af52-e26d4a039ea6 2024-09-16T20:48:05.866 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.29:6816/2744025949,v1:172.21.15.29:6817/2744025949] [v2:172.21.15.29:6818/2744025949,v1:172.21.15.29:6819/2744025949] exists,up 6c3cb2d7-7d23-4566-9ca1-d7b08e12360c', 'osd.1 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.29:6800/208628762,v1:172.21.15.29:6801/208628762] [v2:172.21.15.29:6802/208628762,v1:172.21.15.29:6803/208628762] exists,up 5cf8c20c-1ed2-42ae-a8b0-c88cc4e36bce', 'osd.2 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.29:6808/3447426536,v1:172.21.15.29:6809/3447426536] [v2:172.21.15.29:6810/3447426536,v1:172.21.15.29:6811/3447426536] exists,up 075d16da-82bf-449b-9f6e-542c9e8d810b', 'osd.3 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.43:6808/3687583891,v1:172.21.15.43:6809/3687583891] [v2:172.21.15.43:6810/3687583891,v1:172.21.15.43:6811/3687583891] exists,up 0612be38-3a24-41b6-aba4-5a2fe985165e', 'osd.4 up in weight 1 up_from 11 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.43:6800/2505220646,v1:172.21.15.43:6801/2505220646] [v2:172.21.15.43:6802/2505220646,v1:172.21.15.43:6803/2505220646] exists,up be590bf8-e9fa-44ca-a471-6f6bcb087ca7', 'osd.5 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.43:6816/1517367307,v1:172.21.15.43:6817/1517367307] [v2:172.21.15.43:6818/1517367307,v1:172.21.15.43:6819/1517367307] exists,up 4a208e30-031b-4a9f-af52-e26d4a039ea6'] 2024-09-16T20:48:05.866 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-16T20:48:05.968 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:05.968 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "mon.a", 2024-09-16T20:48:05.969 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster": "ceph", 2024-09-16T20:48:05.969 INFO:teuthology.orchestra.run.smithi029.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-16T20:48:05.969 INFO:teuthology.orchestra.run.smithi029.stdout: "admin_socket_mode": "", 2024-09-16T20:48:05.969 INFO:teuthology.orchestra.run.smithi029.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-16T20:48:05.969 INFO:teuthology.orchestra.run.smithi029.stdout: "auth_client_required": "cephx, none", 2024-09-16T20:48:05.969 INFO:teuthology.orchestra.run.smithi029.stdout: "auth_cluster_required": "cephx", 2024-09-16T20:48:05.969 INFO:teuthology.orchestra.run.smithi029.stdout: "auth_debug": "true", 2024-09-16T20:48:05.969 INFO:teuthology.orchestra.run.smithi029.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-16T20:48:05.969 INFO:teuthology.orchestra.run.smithi029.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-16T20:48:05.969 INFO:teuthology.orchestra.run.smithi029.stdout: "auth_service_required": "cephx", 2024-09-16T20:48:05.969 INFO:teuthology.orchestra.run.smithi029.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-16T20:48:05.969 INFO:teuthology.orchestra.run.smithi029.stdout: "auth_supported": "cephx", 2024-09-16T20:48:05.970 INFO:teuthology.orchestra.run.smithi029.stdout: "bdev_aio": "true", 2024-09-16T20:48:05.970 INFO:teuthology.orchestra.run.smithi029.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-16T20:48:05.970 INFO:teuthology.orchestra.run.smithi029.stdout: "bdev_aio_poll_ms": "250", 2024-09-16T20:48:05.970 INFO:teuthology.orchestra.run.smithi029.stdout: "bdev_aio_reap_max": "16", 2024-09-16T20:48:05.970 INFO:teuthology.orchestra.run.smithi029.stdout: "bdev_async_discard_threads": "0", 2024-09-16T20:48:05.970 INFO:teuthology.orchestra.run.smithi029.stdout: "bdev_block_size": "4096", 2024-09-16T20:48:05.970 INFO:teuthology.orchestra.run.smithi029.stdout: "bdev_debug_aio": "false", 2024-09-16T20:48:05.970 INFO:teuthology.orchestra.run.smithi029.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-16T20:48:05.970 INFO:teuthology.orchestra.run.smithi029.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-16T20:48:05.970 INFO:teuthology.orchestra.run.smithi029.stdout: "bdev_debug_inflight_ios": "false", 2024-09-16T20:48:05.970 INFO:teuthology.orchestra.run.smithi029.stdout: "bdev_enable_discard": "false", 2024-09-16T20:48:05.970 INFO:teuthology.orchestra.run.smithi029.stdout: "bdev_flock_retry": "3", 2024-09-16T20:48:05.971 INFO:teuthology.orchestra.run.smithi029.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-16T20:48:05.971 INFO:teuthology.orchestra.run.smithi029.stdout: "bdev_inject_crash": "0", 2024-09-16T20:48:05.971 INFO:teuthology.orchestra.run.smithi029.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-16T20:48:05.971 INFO:teuthology.orchestra.run.smithi029.stdout: "bdev_ioring": "false", 2024-09-16T20:48:05.971 INFO:teuthology.orchestra.run.smithi029.stdout: "bdev_ioring_hipri": "false", 2024-09-16T20:48:05.971 INFO:teuthology.orchestra.run.smithi029.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-16T20:48:05.971 INFO:teuthology.orchestra.run.smithi029.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-16T20:48:05.971 INFO:teuthology.orchestra.run.smithi029.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-16T20:48:05.971 INFO:teuthology.orchestra.run.smithi029.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-16T20:48:05.971 INFO:teuthology.orchestra.run.smithi029.stdout: "bdev_stalled_read_warn_lifetime": "86400", 2024-09-16T20:48:05.971 INFO:teuthology.orchestra.run.smithi029.stdout: "bdev_stalled_read_warn_threshold": "1", 2024-09-16T20:48:05.971 INFO:teuthology.orchestra.run.smithi029.stdout: "bdev_type": "", 2024-09-16T20:48:05.971 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_alloc_size": "1048576", 2024-09-16T20:48:05.972 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_allocator": "hybrid", 2024-09-16T20:48:05.972 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_buffered_io": "true", 2024-09-16T20:48:05.972 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_check_for_zeros": "false", 2024-09-16T20:48:05.972 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-16T20:48:05.972 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-16T20:48:05.972 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_compact_log_sync": "false", 2024-09-16T20:48:05.972 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-16T20:48:05.972 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-16T20:48:05.972 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-16T20:48:05.972 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-16T20:48:05.972 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_max_log_runway": "4194304", 2024-09-16T20:48:05.972 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_max_prefetch": "1048576", 2024-09-16T20:48:05.973 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_min_flush_size": "524288", 2024-09-16T20:48:05.973 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_min_log_runway": "1048576", 2024-09-16T20:48:05.973 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_replay_recovery": "false", 2024-09-16T20:48:05.973 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-16T20:48:05.973 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-16T20:48:05.973 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_sync_write": "false", 2024-09-16T20:48:05.973 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-16T20:48:05.973 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-16T20:48:05.973 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-16T20:48:05.973 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_allocation_from_file": "true", 2024-09-16T20:48:05.973 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_allocator": "hybrid", 2024-09-16T20:48:05.973 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_async_db_compaction": "true", 2024-09-16T20:48:05.974 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-16T20:48:05.974 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-16T20:48:05.974 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-16T20:48:05.974 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-16T20:48:05.974 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_bdev_label_multi": "true", 2024-09-16T20:48:05.974 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_bdev_label_multi_upgrade": "false", 2024-09-16T20:48:05.974 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_bdev_label_require_all": "true", 2024-09-16T20:48:05.974 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-16T20:48:05.974 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-16T20:48:05.974 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-16T20:48:05.974 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_block_create": "true", 2024-09-16T20:48:05.974 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_block_db_create": "false", 2024-09-16T20:48:05.974 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_block_db_path": "", 2024-09-16T20:48:05.975 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_block_db_size": "0", 2024-09-16T20:48:05.975 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_block_path": "", 2024-09-16T20:48:05.975 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_block_preallocate_file": "false", 2024-09-16T20:48:05.975 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_block_size": "107374182400", 2024-09-16T20:48:05.975 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_block_wal_create": "false", 2024-09-16T20:48:05.975 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_block_wal_path": "", 2024-09-16T20:48:05.975 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_block_wal_size": "100663296", 2024-09-16T20:48:05.975 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_bluefs": "true", 2024-09-16T20:48:05.975 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-16T20:48:05.975 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-16T20:48:05.975 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-16T20:48:05.975 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_btree2_alloc_weight_factor": "2.000000", 2024-09-16T20:48:05.976 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-16T20:48:05.976 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-16T20:48:05.976 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-16T20:48:05.976 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-16T20:48:05.976 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-16T20:48:05.976 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_autotune": "true", 2024-09-16T20:48:05.976 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-16T20:48:05.976 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-16T20:48:05.976 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-16T20:48:05.976 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-16T20:48:05.976 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_size": "0", 2024-09-16T20:48:05.976 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-16T20:48:05.977 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-16T20:48:05.977 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-16T20:48:05.977 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-16T20:48:05.977 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_type": "2q", 2024-09-16T20:48:05.977 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-16T20:48:05.977 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_clone_cow": "true", 2024-09-16T20:48:05.977 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-16T20:48:05.977 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-16T20:48:05.977 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-16T20:48:05.977 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-16T20:48:05.977 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-16T20:48:05.977 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-16T20:48:05.977 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-16T20:48:05.978 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_compression_mode": "none", 2024-09-16T20:48:05.978 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-16T20:48:05.978 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_csum_type": "crc32c", 2024-09-16T20:48:05.978 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_debug_enforce_min_alloc_size": "0", 2024-09-16T20:48:05.978 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-16T20:48:05.978 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_debug_freelist": "false", 2024-09-16T20:48:05.978 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-16T20:48:05.978 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-16T20:48:05.978 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-16T20:48:05.978 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-16T20:48:05.978 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-16T20:48:05.978 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-16T20:48:05.978 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-16T20:48:05.979 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-16T20:48:05.979 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-16T20:48:05.979 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-16T20:48:05.979 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-16T20:48:05.979 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-16T20:48:05.979 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_debug_small_allocations": "0", 2024-09-16T20:48:05.979 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-16T20:48:05.979 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_default_buffered_read": "true", 2024-09-16T20:48:05.979 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_default_buffered_write": "false", 2024-09-16T20:48:05.979 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-16T20:48:05.979 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-16T20:48:05.979 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-16T20:48:05.979 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_elastic_shared_blobs": "true", 2024-09-16T20:48:05.980 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-16T20:48:05.980 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-16T20:48:05.980 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-16T20:48:05.980 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-16T20:48:05.980 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-16T20:48:05.980 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_fail_eio": "false", 2024-09-16T20:48:05.980 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-16T20:48:05.980 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-16T20:48:05.980 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-16T20:48:05.980 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-16T20:48:05.980 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-16T20:48:05.980 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-16T20:48:05.980 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_fsck_on_mount": "false", 2024-09-16T20:48:05.980 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-16T20:48:05.981 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_fsck_on_umount": "false", 2024-09-16T20:48:05.981 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-16T20:48:05.981 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-16T20:48:05.981 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-16T20:48:05.981 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-16T20:48:05.981 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-16T20:48:05.981 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-16T20:48:05.981 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-16T20:48:05.981 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-16T20:48:05.981 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_ignore_data_csum": "false", 2024-09-16T20:48:05.982 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-16T20:48:05.982 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-16T20:48:05.982 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-16T20:48:05.982 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-16T20:48:05.982 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_log_op_age": "5.000000", 2024-09-16T20:48:05.982 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_max_alloc_size": "0", 2024-09-16T20:48:05.982 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_max_blob_size": "0", 2024-09-16T20:48:05.982 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-16T20:48:05.982 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-16T20:48:05.982 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-16T20:48:05.982 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_max_deferred_txc": "32", 2024-09-16T20:48:05.982 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_min_alloc_size": "0", 2024-09-16T20:48:05.982 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-16T20:48:05.982 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-16T20:48:05.983 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_nid_prealloc": "1024", 2024-09-16T20:48:05.983 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-16T20:48:05.983 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-16T20:48:05.983 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-16T20:48:05.983 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-16T20:48:05.983 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_retry_disk_reads": "3", 2024-09-16T20:48:05.983 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_rocksdb_cf": "true", 2024-09-16T20:48:05.983 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-16T20:48:05.983 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-16T20:48:05.983 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-16T20:48:05.983 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_slow_ops_warn_lifetime": "86400", 2024-09-16T20:48:05.983 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_slow_ops_warn_threshold": "1", 2024-09-16T20:48:05.983 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-16T20:48:05.984 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-16T20:48:05.984 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-16T20:48:05.984 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_spdk_mem": "512", 2024-09-16T20:48:05.984 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-16T20:48:05.984 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-16T20:48:05.984 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-16T20:48:05.984 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-16T20:48:05.984 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-16T20:48:05.984 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-16T20:48:05.984 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-16T20:48:05.984 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_tracing": "false", 2024-09-16T20:48:05.984 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-16T20:48:05.984 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-16T20:48:05.985 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-16T20:48:05.985 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-16T20:48:05.985 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-16T20:48:05.985 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-16T20:48:05.985 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-16T20:48:05.985 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-16T20:48:05.985 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-16T20:48:05.985 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_write_v2": "false", 2024-09-16T20:48:05.985 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_write_v2_random": "false", 2024-09-16T20:48:05.985 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_zero_block_detection": "false", 2024-09-16T20:48:05.985 INFO:teuthology.orchestra.run.smithi029.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-16T20:48:05.985 INFO:teuthology.orchestra.run.smithi029.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-16T20:48:05.985 INFO:teuthology.orchestra.run.smithi029.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-16T20:48:05.986 INFO:teuthology.orchestra.run.smithi029.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-16T20:48:05.986 INFO:teuthology.orchestra.run.smithi029.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-16T20:48:05.986 INFO:teuthology.orchestra.run.smithi029.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-16T20:48:05.986 INFO:teuthology.orchestra.run.smithi029.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-16T20:48:05.986 INFO:teuthology.orchestra.run.smithi029.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-16T20:48:05.986 INFO:teuthology.orchestra.run.smithi029.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-16T20:48:05.986 INFO:teuthology.orchestra.run.smithi029.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-16T20:48:05.986 INFO:teuthology.orchestra.run.smithi029.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-16T20:48:05.986 INFO:teuthology.orchestra.run.smithi029.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-16T20:48:05.986 INFO:teuthology.orchestra.run.smithi029.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-16T20:48:05.986 INFO:teuthology.orchestra.run.smithi029.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-16T20:48:05.986 INFO:teuthology.orchestra.run.smithi029.stdout: "cephx_cluster_require_signatures": "false", 2024-09-16T20:48:05.986 INFO:teuthology.orchestra.run.smithi029.stdout: "cephx_cluster_require_version": "2", 2024-09-16T20:48:05.987 INFO:teuthology.orchestra.run.smithi029.stdout: "cephx_require_signatures": "false", 2024-09-16T20:48:05.987 INFO:teuthology.orchestra.run.smithi029.stdout: "cephx_require_version": "2", 2024-09-16T20:48:05.987 INFO:teuthology.orchestra.run.smithi029.stdout: "cephx_service_require_signatures": "false", 2024-09-16T20:48:05.987 INFO:teuthology.orchestra.run.smithi029.stdout: "cephx_service_require_version": "2", 2024-09-16T20:48:05.987 INFO:teuthology.orchestra.run.smithi029.stdout: "cephx_sign_messages": "true", 2024-09-16T20:48:05.987 INFO:teuthology.orchestra.run.smithi029.stdout: "chdir": "", 2024-09-16T20:48:05.987 INFO:teuthology.orchestra.run.smithi029.stdout: "client_acl_type": "", 2024-09-16T20:48:05.987 INFO:teuthology.orchestra.run.smithi029.stdout: "client_asio_thread_count": "2", 2024-09-16T20:48:05.987 INFO:teuthology.orchestra.run.smithi029.stdout: "client_cache_mid": "0.750000", 2024-09-16T20:48:05.987 INFO:teuthology.orchestra.run.smithi029.stdout: "client_cache_size": "16384", 2024-09-16T20:48:05.987 INFO:teuthology.orchestra.run.smithi029.stdout: "client_caps_release_delay": "5", 2024-09-16T20:48:05.987 INFO:teuthology.orchestra.run.smithi029.stdout: "client_check_pool_perm": "true", 2024-09-16T20:48:05.988 INFO:teuthology.orchestra.run.smithi029.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-16T20:48:05.988 INFO:teuthology.orchestra.run.smithi029.stdout: "client_debug_force_sync_read": "false", 2024-09-16T20:48:05.988 INFO:teuthology.orchestra.run.smithi029.stdout: "client_debug_getattr_caps": "false", 2024-09-16T20:48:05.988 INFO:teuthology.orchestra.run.smithi029.stdout: "client_debug_inject_features": "", 2024-09-16T20:48:05.988 INFO:teuthology.orchestra.run.smithi029.stdout: "client_debug_inject_tick_delay": "0", 2024-09-16T20:48:05.988 INFO:teuthology.orchestra.run.smithi029.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-16T20:48:05.988 INFO:teuthology.orchestra.run.smithi029.stdout: "client_die_on_failed_remount": "false", 2024-09-16T20:48:05.988 INFO:teuthology.orchestra.run.smithi029.stdout: "client_dirsize_rbytes": "true", 2024-09-16T20:48:05.988 INFO:teuthology.orchestra.run.smithi029.stdout: "client_force_lazyio": "false", 2024-09-16T20:48:05.988 INFO:teuthology.orchestra.run.smithi029.stdout: "client_fs": "", 2024-09-16T20:48:05.988 INFO:teuthology.orchestra.run.smithi029.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-16T20:48:05.988 INFO:teuthology.orchestra.run.smithi029.stdout: "client_inject_release_failure": "false", 2024-09-16T20:48:05.989 INFO:teuthology.orchestra.run.smithi029.stdout: "client_max_inline_size": "4096", 2024-09-16T20:48:05.989 INFO:teuthology.orchestra.run.smithi029.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-16T20:48:05.989 INFO:teuthology.orchestra.run.smithi029.stdout: "client_mds_namespace": "", 2024-09-16T20:48:05.989 INFO:teuthology.orchestra.run.smithi029.stdout: "client_metadata": "", 2024-09-16T20:48:05.989 INFO:teuthology.orchestra.run.smithi029.stdout: "client_mount_gid": "-1", 2024-09-16T20:48:05.989 INFO:teuthology.orchestra.run.smithi029.stdout: "client_mount_timeout": "300", 2024-09-16T20:48:05.989 INFO:teuthology.orchestra.run.smithi029.stdout: "client_mount_uid": "-1", 2024-09-16T20:48:05.989 INFO:teuthology.orchestra.run.smithi029.stdout: "client_mountpoint": "/", 2024-09-16T20:48:05.989 INFO:teuthology.orchestra.run.smithi029.stdout: "client_notify_timeout": "10", 2024-09-16T20:48:05.989 INFO:teuthology.orchestra.run.smithi029.stdout: "client_oc": "true", 2024-09-16T20:48:05.989 INFO:teuthology.orchestra.run.smithi029.stdout: "client_oc_max_dirty": "104857600", 2024-09-16T20:48:05.989 INFO:teuthology.orchestra.run.smithi029.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-16T20:48:05.990 INFO:teuthology.orchestra.run.smithi029.stdout: "client_oc_max_objects": "1000", 2024-09-16T20:48:05.990 INFO:teuthology.orchestra.run.smithi029.stdout: "client_oc_size": "209715200", 2024-09-16T20:48:05.990 INFO:teuthology.orchestra.run.smithi029.stdout: "client_oc_target_dirty": "8388608", 2024-09-16T20:48:05.990 INFO:teuthology.orchestra.run.smithi029.stdout: "client_permissions": "true", 2024-09-16T20:48:05.990 INFO:teuthology.orchestra.run.smithi029.stdout: "client_quota": "true", 2024-09-16T20:48:05.990 INFO:teuthology.orchestra.run.smithi029.stdout: "client_quota_df": "true", 2024-09-16T20:48:05.990 INFO:teuthology.orchestra.run.smithi029.stdout: "client_readahead_max_bytes": "0", 2024-09-16T20:48:05.990 INFO:teuthology.orchestra.run.smithi029.stdout: "client_readahead_max_periods": "4", 2024-09-16T20:48:05.990 INFO:teuthology.orchestra.run.smithi029.stdout: "client_readahead_min": "131072", 2024-09-16T20:48:05.990 INFO:teuthology.orchestra.run.smithi029.stdout: "client_reconnect_stale": "false", 2024-09-16T20:48:05.990 INFO:teuthology.orchestra.run.smithi029.stdout: "client_shutdown_timeout": "30", 2024-09-16T20:48:05.990 INFO:teuthology.orchestra.run.smithi029.stdout: "client_snapdir": ".snap", 2024-09-16T20:48:05.990 INFO:teuthology.orchestra.run.smithi029.stdout: "client_tick_interval": "1", 2024-09-16T20:48:05.991 INFO:teuthology.orchestra.run.smithi029.stdout: "client_trace": "", 2024-09-16T20:48:05.991 INFO:teuthology.orchestra.run.smithi029.stdout: "client_try_dentry_invalidate": "false", 2024-09-16T20:48:05.991 INFO:teuthology.orchestra.run.smithi029.stdout: "client_use_faked_inos": "false", 2024-09-16T20:48:05.991 INFO:teuthology.orchestra.run.smithi029.stdout: "client_use_random_mds": "false", 2024-09-16T20:48:05.991 INFO:teuthology.orchestra.run.smithi029.stdout: "clog_to_graylog": "false", 2024-09-16T20:48:05.991 INFO:teuthology.orchestra.run.smithi029.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-16T20:48:05.991 INFO:teuthology.orchestra.run.smithi029.stdout: "clog_to_graylog_port": "12201", 2024-09-16T20:48:05.991 INFO:teuthology.orchestra.run.smithi029.stdout: "clog_to_monitors": "default=true", 2024-09-16T20:48:05.991 INFO:teuthology.orchestra.run.smithi029.stdout: "clog_to_syslog": "false", 2024-09-16T20:48:05.991 INFO:teuthology.orchestra.run.smithi029.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-16T20:48:05.991 INFO:teuthology.orchestra.run.smithi029.stdout: "clog_to_syslog_level": "info", 2024-09-16T20:48:05.991 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "-", 2024-09-16T20:48:05.992 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_network": "", 2024-09-16T20:48:05.992 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_network_interface": "", 2024-09-16T20:48:05.992 INFO:teuthology.orchestra.run.smithi029.stdout: "compressor_zlib_isal": "false", 2024-09-16T20:48:05.992 INFO:teuthology.orchestra.run.smithi029.stdout: "compressor_zlib_level": "5", 2024-09-16T20:48:05.992 INFO:teuthology.orchestra.run.smithi029.stdout: "compressor_zlib_winsize": "-15", 2024-09-16T20:48:05.992 INFO:teuthology.orchestra.run.smithi029.stdout: "compressor_zstd_level": "1", 2024-09-16T20:48:05.992 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-16T20:48:05.992 INFO:teuthology.orchestra.run.smithi029.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-16T20:48:05.992 INFO:teuthology.orchestra.run.smithi029.stdout: "crimson_alien_op_num_threads": "6", 2024-09-16T20:48:05.992 INFO:teuthology.orchestra.run.smithi029.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-16T20:48:05.992 INFO:teuthology.orchestra.run.smithi029.stdout: "crimson_osd_obc_lru_size": "512", 2024-09-16T20:48:05.992 INFO:teuthology.orchestra.run.smithi029.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-16T20:48:05.992 INFO:teuthology.orchestra.run.smithi029.stdout: "crimson_osd_stat_interval": "0", 2024-09-16T20:48:05.993 INFO:teuthology.orchestra.run.smithi029.stdout: "crimson_seastar_cpu_cores": "", 2024-09-16T20:48:05.993 INFO:teuthology.orchestra.run.smithi029.stdout: "crimson_seastar_num_threads": "0", 2024-09-16T20:48:05.993 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_location": "", 2024-09-16T20:48:05.993 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_location_hook": "", 2024-09-16T20:48:05.993 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_location_hook_timeout": "10", 2024-09-16T20:48:05.993 INFO:teuthology.orchestra.run.smithi029.stdout: "daemonize": "false", 2024-09-16T20:48:05.993 INFO:teuthology.orchestra.run.smithi029.stdout: "daos_pool": "tank", 2024-09-16T20:48:05.993 INFO:teuthology.orchestra.run.smithi029.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-16T20:48:05.993 INFO:teuthology.orchestra.run.smithi029.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-16T20:48:05.993 INFO:teuthology.orchestra.run.smithi029.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-16T20:48:05.993 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_alienstore": "0/5", 2024-09-16T20:48:05.993 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_allow_any_pool_priority": "false", 2024-09-16T20:48:05.993 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_asok": "1/5", 2024-09-16T20:48:05.993 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_asok_assert_abort": "false", 2024-09-16T20:48:05.994 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_asserts_on_shutdown": "true", 2024-09-16T20:48:05.994 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_auth": "20/20", 2024-09-16T20:48:05.994 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_bdev": "1/3", 2024-09-16T20:48:05.994 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_bluefs": "1/5", 2024-09-16T20:48:05.994 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_bluestore": "1/5", 2024-09-16T20:48:05.994 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_buffer": "0/1", 2024-09-16T20:48:05.994 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_ceph_dedup": "0/5", 2024-09-16T20:48:05.994 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_ceph_exporter": "1/5", 2024-09-16T20:48:05.994 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_cephfs_mirror": "0/5", 2024-09-16T20:48:05.994 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_cephsqlite": "0/5", 2024-09-16T20:48:05.994 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_client": "0/5", 2024-09-16T20:48:05.994 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_compressor": "1/5", 2024-09-16T20:48:05.994 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_context": "0/1", 2024-09-16T20:48:05.994 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_crimson_interrupt": "0/5", 2024-09-16T20:48:05.994 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_crush": "1/1", 2024-09-16T20:48:05.995 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_crypto": "1/5", 2024-09-16T20:48:05.995 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_cyanstore": "0/5", 2024-09-16T20:48:05.995 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_deliberately_leak_memory": "false", 2024-09-16T20:48:05.995 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_disable_randomized_ping": "false", 2024-09-16T20:48:05.995 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_dpdk": "1/5", 2024-09-16T20:48:05.995 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_eventtrace": "1/5", 2024-09-16T20:48:05.995 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_filer": "0/1", 2024-09-16T20:48:05.995 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_filestore": "1/3", 2024-09-16T20:48:05.995 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_finisher": "1/1", 2024-09-16T20:48:05.995 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_fuse": "1/5", 2024-09-16T20:48:05.995 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_heartbeat_testing_span": "0", 2024-09-16T20:48:05.995 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_heartbeatmap": "1/5", 2024-09-16T20:48:05.995 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-16T20:48:05.995 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_javaclient": "1/5", 2024-09-16T20:48:05.996 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_journal": "1/3", 2024-09-16T20:48:05.996 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_journaler": "0/5", 2024-09-16T20:48:05.996 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_kstore": "1/5", 2024-09-16T20:48:05.996 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_lockdep": "0/1", 2024-09-16T20:48:05.996 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_mclock": "1/5", 2024-09-16T20:48:05.996 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_mds": "1/5", 2024-09-16T20:48:05.996 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_mds_balancer": "1/5", 2024-09-16T20:48:05.996 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_mds_locker": "1/5", 2024-09-16T20:48:05.996 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_mds_log": "1/5", 2024-09-16T20:48:05.996 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_mds_log_expire": "1/5", 2024-09-16T20:48:05.997 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_mds_migrator": "1/5", 2024-09-16T20:48:05.997 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_mds_quiesce": "3/5", 2024-09-16T20:48:05.997 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_memstore": "1/5", 2024-09-16T20:48:05.997 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_mgr": "2/5", 2024-09-16T20:48:05.997 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_mgrc": "1/5", 2024-09-16T20:48:05.997 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_mon": "20/20", 2024-09-16T20:48:05.997 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_monc": "0/10", 2024-09-16T20:48:05.997 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_ms": "1/1", 2024-09-16T20:48:05.997 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_none": "0/5", 2024-09-16T20:48:05.997 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_objclass": "0/5", 2024-09-16T20:48:05.997 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_objectcacher": "0/5", 2024-09-16T20:48:05.997 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_objecter": "0/1", 2024-09-16T20:48:05.997 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_optracker": "0/5", 2024-09-16T20:48:05.998 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_osd": "1/5", 2024-09-16T20:48:05.998 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_paxos": "20/20", 2024-09-16T20:48:05.998 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_perfcounter": "1/5", 2024-09-16T20:48:05.998 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_prioritycache": "1/5", 2024-09-16T20:48:05.998 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_rados": "0/5", 2024-09-16T20:48:05.998 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_rbd": "0/5", 2024-09-16T20:48:05.998 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_rbd_mirror": "0/5", 2024-09-16T20:48:05.998 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_rbd_pwl": "0/5", 2024-09-16T20:48:05.998 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_rbd_replay": "0/5", 2024-09-16T20:48:05.998 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_refs": "0/0", 2024-09-16T20:48:05.998 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_reserver": "1/1", 2024-09-16T20:48:05.998 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_rgw": "1/5", 2024-09-16T20:48:05.998 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_rgw_access": "1/5", 2024-09-16T20:48:05.998 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_rgw_datacache": "1/5", 2024-09-16T20:48:05.998 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_rgw_dbstore": "1/5", 2024-09-16T20:48:05.999 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_rgw_flight": "1/5", 2024-09-16T20:48:05.999 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_rgw_lifecycle": "1/5", 2024-09-16T20:48:05.999 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_rgw_notification": "1/5", 2024-09-16T20:48:05.999 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_rgw_sync": "1/5", 2024-09-16T20:48:05.999 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_rocksdb": "4/5", 2024-09-16T20:48:05.999 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_seastore": "0/5", 2024-09-16T20:48:05.999 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_seastore_backref": "0/5", 2024-09-16T20:48:05.999 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_seastore_cache": "0/5", 2024-09-16T20:48:05.999 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_seastore_cleaner": "0/5", 2024-09-16T20:48:05.999 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_seastore_device": "0/5", 2024-09-16T20:48:05.999 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_seastore_epm": "0/5", 2024-09-16T20:48:05.999 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-16T20:48:05.999 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_seastore_journal": "0/5", 2024-09-16T20:48:06.000 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_seastore_lba": "0/5", 2024-09-16T20:48:06.000 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_seastore_odata": "0/5", 2024-09-16T20:48:06.000 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_seastore_omap": "0/5", 2024-09-16T20:48:06.000 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_seastore_onode": "0/5", 2024-09-16T20:48:06.000 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_seastore_t": "0/5", 2024-09-16T20:48:06.000 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_seastore_tm": "0/5", 2024-09-16T20:48:06.000 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_striper": "0/1", 2024-09-16T20:48:06.000 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_test": "0/5", 2024-09-16T20:48:06.000 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_throttle": "1/1", 2024-09-16T20:48:06.000 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_timer": "0/1", 2024-09-16T20:48:06.000 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_tp": "0/5", 2024-09-16T20:48:06.000 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_trace": "1/5", 2024-09-16T20:48:06.000 INFO:teuthology.orchestra.run.smithi029.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-16T20:48:06.001 INFO:teuthology.orchestra.run.smithi029.stdout: "device_failure_prediction_mode": "none", 2024-09-16T20:48:06.001 INFO:teuthology.orchestra.run.smithi029.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-16T20:48:06.001 INFO:teuthology.orchestra.run.smithi029.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-09-16T20:48:06.001 INFO:teuthology.orchestra.run.smithi029.stdout: "err_to_graylog": "false", 2024-09-16T20:48:06.001 INFO:teuthology.orchestra.run.smithi029.stdout: "err_to_journald": "false", 2024-09-16T20:48:06.001 INFO:teuthology.orchestra.run.smithi029.stdout: "err_to_stderr": "true", 2024-09-16T20:48:06.001 INFO:teuthology.orchestra.run.smithi029.stdout: "err_to_syslog": "false", 2024-09-16T20:48:06.001 INFO:teuthology.orchestra.run.smithi029.stdout: "event_tracing": "false", 2024-09-16T20:48:06.001 INFO:teuthology.orchestra.run.smithi029.stdout: "exporter_addr": "0.0.0.0", 2024-09-16T20:48:06.001 INFO:teuthology.orchestra.run.smithi029.stdout: "exporter_cert_file": "", 2024-09-16T20:48:06.001 INFO:teuthology.orchestra.run.smithi029.stdout: "exporter_http_port": "9926", 2024-09-16T20:48:06.001 INFO:teuthology.orchestra.run.smithi029.stdout: "exporter_key_file": "", 2024-09-16T20:48:06.001 INFO:teuthology.orchestra.run.smithi029.stdout: "exporter_prio_limit": "5", 2024-09-16T20:48:06.001 INFO:teuthology.orchestra.run.smithi029.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-16T20:48:06.002 INFO:teuthology.orchestra.run.smithi029.stdout: "exporter_sort_metrics": "true", 2024-09-16T20:48:06.002 INFO:teuthology.orchestra.run.smithi029.stdout: "exporter_stats_period": "5", 2024-09-16T20:48:06.002 INFO:teuthology.orchestra.run.smithi029.stdout: "fake_statfs_for_testing": "0", 2024-09-16T20:48:06.002 INFO:teuthology.orchestra.run.smithi029.stdout: "fatal_signal_handlers": "true", 2024-09-16T20:48:06.002 INFO:teuthology.orchestra.run.smithi029.stdout: "filer_max_purge_ops": "10", 2024-09-16T20:48:06.002 INFO:teuthology.orchestra.run.smithi029.stdout: "filer_max_truncate_ops": "128", 2024-09-16T20:48:06.002 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_apply_finisher_threads": "1", 2024-09-16T20:48:06.002 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_blackhole": "false", 2024-09-16T20:48:06.002 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_btrfs_clone_range": "true", 2024-09-16T20:48:06.002 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_btrfs_snap": "true", 2024-09-16T20:48:06.002 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_caller_concurrency": "10", 2024-09-16T20:48:06.002 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_collect_device_partition_information": "true", 2024-09-16T20:48:06.002 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_commit_timeout": "600.000000", 2024-09-16T20:48:06.002 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_debug_inject_read_err": "false", 2024-09-16T20:48:06.003 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_debug_omap_check": "false", 2024-09-16T20:48:06.003 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_debug_verify_split": "false", 2024-09-16T20:48:06.003 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_dump_file": "", 2024-09-16T20:48:06.003 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-16T20:48:06.003 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-16T20:48:06.003 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_fadvise": "true", 2024-09-16T20:48:06.003 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_fail_eio": "true", 2024-09-16T20:48:06.003 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_fd_cache_shards": "16", 2024-09-16T20:48:06.003 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_fd_cache_size": "128", 2024-09-16T20:48:06.003 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_fiemap": "false", 2024-09-16T20:48:06.003 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_fiemap_threshold": "4096", 2024-09-16T20:48:06.003 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-16T20:48:06.003 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-16T20:48:06.004 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_inject_stall": "0", 2024-09-16T20:48:06.004 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_journal_parallel": "false", 2024-09-16T20:48:06.004 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_journal_trailing": "false", 2024-09-16T20:48:06.004 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_journal_writeahead": "false", 2024-09-16T20:48:06.004 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_kill_at": "0", 2024-09-16T20:48:06.004 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-16T20:48:06.004 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-16T20:48:06.004 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-16T20:48:06.004 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-16T20:48:06.004 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-16T20:48:06.004 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_max_inline_xattrs": "0", 2024-09-16T20:48:06.004 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-16T20:48:06.004 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-16T20:48:06.005 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-16T20:48:06.005 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-16T20:48:06.005 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_max_xattr_value_size": "0", 2024-09-16T20:48:06.005 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-16T20:48:06.005 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-16T20:48:06.005 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-16T20:48:06.005 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_merge_threshold": "-10", 2024-09-16T20:48:06.005 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-16T20:48:06.005 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_odsync_write": "false", 2024-09-16T20:48:06.005 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_omap_backend": "rocksdb", 2024-09-16T20:48:06.005 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_omap_backend_path": "", 2024-09-16T20:48:06.005 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-16T20:48:06.005 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-16T20:48:06.006 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-16T20:48:06.006 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_op_thread_timeout": "60", 2024-09-16T20:48:06.006 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_op_threads": "2", 2024-09-16T20:48:06.006 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_punch_hole": "false", 2024-09-16T20:48:06.006 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-16T20:48:06.006 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-16T20:48:06.006 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-16T20:48:06.006 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-16T20:48:06.006 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-16T20:48:06.006 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-16T20:48:06.006 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-16T20:48:06.006 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-16T20:48:06.006 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-16T20:48:06.006 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_queue_max_ops": "50", 2024-09-16T20:48:06.006 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-16T20:48:06.007 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_seek_data_hole": "false", 2024-09-16T20:48:06.007 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_sloppy_crc": "false", 2024-09-16T20:48:06.007 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-16T20:48:06.007 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_splice": "false", 2024-09-16T20:48:06.007 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_split_multiple": "2", 2024-09-16T20:48:06.007 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_split_rand_factor": "20", 2024-09-16T20:48:06.007 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_update_to": "1000", 2024-09-16T20:48:06.007 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-16T20:48:06.007 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-16T20:48:06.007 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-16T20:48:06.007 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-16T20:48:06.007 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-16T20:48:06.007 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-16T20:48:06.008 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_wbthrottle_enable": "true", 2024-09-16T20:48:06.008 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-16T20:48:06.008 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-16T20:48:06.008 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-16T20:48:06.008 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-16T20:48:06.008 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-16T20:48:06.008 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-16T20:48:06.008 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_xfs_extsize": "false", 2024-09-16T20:48:06.008 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_zfs_snap": "false", 2024-09-16T20:48:06.008 INFO:teuthology.orchestra.run.smithi029.stdout: "fio_dir": "/tmp/fio", 2024-09-16T20:48:06.008 INFO:teuthology.orchestra.run.smithi029.stdout: "fsid": "e28af4a7-5fcd-4de8-8f77-a2b575361bb1", 2024-09-16T20:48:06.008 INFO:teuthology.orchestra.run.smithi029.stdout: "fuse_allow_other": "true", 2024-09-16T20:48:06.008 INFO:teuthology.orchestra.run.smithi029.stdout: "fuse_atomic_o_trunc": "true", 2024-09-16T20:48:06.008 INFO:teuthology.orchestra.run.smithi029.stdout: "fuse_big_writes": "true", 2024-09-16T20:48:06.009 INFO:teuthology.orchestra.run.smithi029.stdout: "fuse_debug": "false", 2024-09-16T20:48:06.009 INFO:teuthology.orchestra.run.smithi029.stdout: "fuse_default_permissions": "false", 2024-09-16T20:48:06.009 INFO:teuthology.orchestra.run.smithi029.stdout: "fuse_disable_pagecache": "false", 2024-09-16T20:48:06.009 INFO:teuthology.orchestra.run.smithi029.stdout: "fuse_max_write": "0", 2024-09-16T20:48:06.009 INFO:teuthology.orchestra.run.smithi029.stdout: "fuse_multithreaded": "true", 2024-09-16T20:48:06.009 INFO:teuthology.orchestra.run.smithi029.stdout: "fuse_require_active_mds": "true", 2024-09-16T20:48:06.009 INFO:teuthology.orchestra.run.smithi029.stdout: "fuse_set_user_groups": "true", 2024-09-16T20:48:06.009 INFO:teuthology.orchestra.run.smithi029.stdout: "fuse_splice_move": "true", 2024-09-16T20:48:06.009 INFO:teuthology.orchestra.run.smithi029.stdout: "fuse_splice_read": "true", 2024-09-16T20:48:06.009 INFO:teuthology.orchestra.run.smithi029.stdout: "fuse_splice_write": "true", 2024-09-16T20:48:06.009 INFO:teuthology.orchestra.run.smithi029.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-16T20:48:06.009 INFO:teuthology.orchestra.run.smithi029.stdout: "fuse_use_invalidate_cb": "true", 2024-09-16T20:48:06.009 INFO:teuthology.orchestra.run.smithi029.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-16T20:48:06.009 INFO:teuthology.orchestra.run.smithi029.stdout: "gss_target_name": "ceph", 2024-09-16T20:48:06.010 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_file": "", 2024-09-16T20:48:06.010 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_inject_failure": "0", 2024-09-16T20:48:06.010 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_interval": "5", 2024-09-16T20:48:06.010 INFO:teuthology.orchestra.run.smithi029.stdout: "host": "smithi029", 2024-09-16T20:48:06.010 INFO:teuthology.orchestra.run.smithi029.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-16T20:48:06.010 INFO:teuthology.orchestra.run.smithi029.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-16T20:48:06.010 INFO:teuthology.orchestra.run.smithi029.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-16T20:48:06.010 INFO:teuthology.orchestra.run.smithi029.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-16T20:48:06.010 INFO:teuthology.orchestra.run.smithi029.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-16T20:48:06.010 INFO:teuthology.orchestra.run.smithi029.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-16T20:48:06.010 INFO:teuthology.orchestra.run.smithi029.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-16T20:48:06.010 INFO:teuthology.orchestra.run.smithi029.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-16T20:48:06.010 INFO:teuthology.orchestra.run.smithi029.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-16T20:48:06.010 INFO:teuthology.orchestra.run.smithi029.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-16T20:48:06.010 INFO:teuthology.orchestra.run.smithi029.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-16T20:48:06.011 INFO:teuthology.orchestra.run.smithi029.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-16T20:48:06.011 INFO:teuthology.orchestra.run.smithi029.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-16T20:48:06.011 INFO:teuthology.orchestra.run.smithi029.stdout: "inject_early_sigterm": "false", 2024-09-16T20:48:06.011 INFO:teuthology.orchestra.run.smithi029.stdout: "jaeger_agent_port": "6799", 2024-09-16T20:48:06.011 INFO:teuthology.orchestra.run.smithi029.stdout: "jaeger_tracing_enable": "false", 2024-09-16T20:48:06.011 INFO:teuthology.orchestra.run.smithi029.stdout: "journal_aio": "true", 2024-09-16T20:48:06.011 INFO:teuthology.orchestra.run.smithi029.stdout: "journal_align_min_size": "65536", 2024-09-16T20:48:06.011 INFO:teuthology.orchestra.run.smithi029.stdout: "journal_block_align": "true", 2024-09-16T20:48:06.011 INFO:teuthology.orchestra.run.smithi029.stdout: "journal_block_size": "4096", 2024-09-16T20:48:06.011 INFO:teuthology.orchestra.run.smithi029.stdout: "journal_dio": "true", 2024-09-16T20:48:06.011 INFO:teuthology.orchestra.run.smithi029.stdout: "journal_discard": "false", 2024-09-16T20:48:06.012 INFO:teuthology.orchestra.run.smithi029.stdout: "journal_force_aio": "false", 2024-09-16T20:48:06.012 INFO:teuthology.orchestra.run.smithi029.stdout: "journal_ignore_corruption": "false", 2024-09-16T20:48:06.012 INFO:teuthology.orchestra.run.smithi029.stdout: "journal_max_write_bytes": "10485760", 2024-09-16T20:48:06.012 INFO:teuthology.orchestra.run.smithi029.stdout: "journal_max_write_entries": "100", 2024-09-16T20:48:06.012 INFO:teuthology.orchestra.run.smithi029.stdout: "journal_replay_from": "0", 2024-09-16T20:48:06.012 INFO:teuthology.orchestra.run.smithi029.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-16T20:48:06.012 INFO:teuthology.orchestra.run.smithi029.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-16T20:48:06.012 INFO:teuthology.orchestra.run.smithi029.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-16T20:48:06.012 INFO:teuthology.orchestra.run.smithi029.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-16T20:48:06.012 INFO:teuthology.orchestra.run.smithi029.stdout: "journal_write_header_frequency": "0", 2024-09-16T20:48:06.012 INFO:teuthology.orchestra.run.smithi029.stdout: "journal_zero_on_create": "false", 2024-09-16T20:48:06.012 INFO:teuthology.orchestra.run.smithi029.stdout: "journaler_prefetch_periods": "10", 2024-09-16T20:48:06.012 INFO:teuthology.orchestra.run.smithi029.stdout: "journaler_prezero_periods": "5", 2024-09-16T20:48:06.013 INFO:teuthology.orchestra.run.smithi029.stdout: "journaler_write_head_interval": "15", 2024-09-16T20:48:06.013 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "", 2024-09-16T20:48:06.013 INFO:teuthology.orchestra.run.smithi029.stdout: "keyfile": "", 2024-09-16T20:48:06.013 INFO:teuthology.orchestra.run.smithi029.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-16T20:48:06.013 INFO:teuthology.orchestra.run.smithi029.stdout: "kstore_backend": "rocksdb", 2024-09-16T20:48:06.013 INFO:teuthology.orchestra.run.smithi029.stdout: "kstore_default_stripe_size": "65536", 2024-09-16T20:48:06.013 INFO:teuthology.orchestra.run.smithi029.stdout: "kstore_fsck_on_mount": "false", 2024-09-16T20:48:06.013 INFO:teuthology.orchestra.run.smithi029.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-16T20:48:06.013 INFO:teuthology.orchestra.run.smithi029.stdout: "kstore_max_bytes": "67108864", 2024-09-16T20:48:06.013 INFO:teuthology.orchestra.run.smithi029.stdout: "kstore_max_ops": "512", 2024-09-16T20:48:06.013 INFO:teuthology.orchestra.run.smithi029.stdout: "kstore_nid_prealloc": "1024", 2024-09-16T20:48:06.013 INFO:teuthology.orchestra.run.smithi029.stdout: "kstore_onode_map_size": "1024", 2024-09-16T20:48:06.013 INFO:teuthology.orchestra.run.smithi029.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-16T20:48:06.014 INFO:teuthology.orchestra.run.smithi029.stdout: "kstore_sync_submit_transaction": "false", 2024-09-16T20:48:06.014 INFO:teuthology.orchestra.run.smithi029.stdout: "kstore_sync_transaction": "false", 2024-09-16T20:48:06.014 INFO:teuthology.orchestra.run.smithi029.stdout: "librados_thread_count": "2", 2024-09-16T20:48:06.014 INFO:teuthology.orchestra.run.smithi029.stdout: "lockdep": "false", 2024-09-16T20:48:06.014 INFO:teuthology.orchestra.run.smithi029.stdout: "lockdep_force_backtrace": "false", 2024-09-16T20:48:06.014 INFO:teuthology.orchestra.run.smithi029.stdout: "log_coarse_timestamps": "true", 2024-09-16T20:48:06.014 INFO:teuthology.orchestra.run.smithi029.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-16T20:48:06.014 INFO:teuthology.orchestra.run.smithi029.stdout: "log_flush_on_exit": "false", 2024-09-16T20:48:06.014 INFO:teuthology.orchestra.run.smithi029.stdout: "log_graylog_host": "127.0.0.1", 2024-09-16T20:48:06.014 INFO:teuthology.orchestra.run.smithi029.stdout: "log_graylog_port": "12201", 2024-09-16T20:48:06.014 INFO:teuthology.orchestra.run.smithi029.stdout: "log_max_new": "1000", 2024-09-16T20:48:06.014 INFO:teuthology.orchestra.run.smithi029.stdout: "log_max_recent": "10000", 2024-09-16T20:48:06.014 INFO:teuthology.orchestra.run.smithi029.stdout: "log_stderr_prefix": "", 2024-09-16T20:48:06.014 INFO:teuthology.orchestra.run.smithi029.stdout: "log_stop_at_utilization": "0.970000", 2024-09-16T20:48:06.014 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": "true", 2024-09-16T20:48:06.015 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_graylog": "false", 2024-09-16T20:48:06.015 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_journald": "false", 2024-09-16T20:48:06.015 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_stderr": "false", 2024-09-16T20:48:06.015 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_syslog": "false", 2024-09-16T20:48:06.015 INFO:teuthology.orchestra.run.smithi029.stdout: "max_rotating_auth_attempts": "10", 2024-09-16T20:48:06.015 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-16T20:48:06.015 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_action_on_write_error": "1", 2024-09-16T20:48:06.015 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_alternate_name_max": "8192", 2024-09-16T20:48:06.015 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_asio_thread_count": "2", 2024-09-16T20:48:06.015 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_bal_export_pin": "true", 2024-09-16T20:48:06.015 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_bal_fragment_dirs": "true", 2024-09-16T20:48:06.015 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-16T20:48:06.015 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_bal_fragment_interval": "5", 2024-09-16T20:48:06.015 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-16T20:48:06.015 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-16T20:48:06.016 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_bal_interval": "10", 2024-09-16T20:48:06.016 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_bal_max": "-1", 2024-09-16T20:48:06.016 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_bal_max_until": "-1", 2024-09-16T20:48:06.016 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_bal_merge_size": "50", 2024-09-16T20:48:06.016 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_bal_midchunk": "0.300000", 2024-09-16T20:48:06.016 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-16T20:48:06.016 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_bal_min_start": "0.200000", 2024-09-16T20:48:06.016 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_bal_minchunk": "0.001000", 2024-09-16T20:48:06.016 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_bal_mode": "0", 2024-09-16T20:48:06.016 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_bal_need_max": "1.200000", 2024-09-16T20:48:06.016 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_bal_need_min": "0.800000", 2024-09-16T20:48:06.016 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_bal_overload_epochs": "2", 2024-09-16T20:48:06.016 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-16T20:48:06.017 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-16T20:48:06.017 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_bal_split_bits": "3", 2024-09-16T20:48:06.017 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-16T20:48:06.017 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_bal_split_size": "10000", 2024-09-16T20:48:06.017 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-16T20:48:06.017 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_bal_target_decay": "10.000000", 2024-09-16T20:48:06.017 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-16T20:48:06.017 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_beacon_grace": "15.000000", 2024-09-16T20:48:06.017 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_beacon_interval": "4.000000", 2024-09-16T20:48:06.017 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-16T20:48:06.017 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-16T20:48:06.017 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_cache_mid": "0.700000", 2024-09-16T20:48:06.017 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-09-16T20:48:06.018 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_cache_quiesce_delay": "0", 2024-09-16T20:48:06.018 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_cache_quiesce_sleep": "200", 2024-09-16T20:48:06.018 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_cache_quiesce_splitauth": "true", 2024-09-16T20:48:06.018 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_cache_quiesce_threshold": "524288", 2024-09-16T20:48:06.018 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_cache_release_free_interval": "10", 2024-09-16T20:48:06.018 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_cache_reservation": "0.050000", 2024-09-16T20:48:06.018 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-16T20:48:06.018 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_cache_trim_interval": "1", 2024-09-16T20:48:06.018 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_cache_trim_threshold": "262144", 2024-09-16T20:48:06.018 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-16T20:48:06.018 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-16T20:48:06.018 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-16T20:48:06.018 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_client_prealloc_inos": "1000", 2024-09-16T20:48:06.018 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-16T20:48:06.019 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_connect_bootstrapping": "false", 2024-09-16T20:48:06.019 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_damage_table_max_entries": "10000", 2024-09-16T20:48:06.019 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-16T20:48:06.019 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_debug_auth_pins": "false", 2024-09-16T20:48:06.019 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_debug_frag": "false", 2024-09-16T20:48:06.019 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_debug_scatterstat": "false", 2024-09-16T20:48:06.019 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_debug_subtrees": "false", 2024-09-16T20:48:06.019 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_decay_halflife": "5.000000", 2024-09-16T20:48:06.019 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_default_dir_hash": "2", 2024-09-16T20:48:06.019 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_defer_session_stale": "true", 2024-09-16T20:48:06.019 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_deny_all_reconnect": "false", 2024-09-16T20:48:06.019 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_dir_keys_per_op": "16384", 2024-09-16T20:48:06.019 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_dir_max_commit_size": "10", 2024-09-16T20:48:06.020 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_dir_max_entries": "0", 2024-09-16T20:48:06.020 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_dir_prefetch": "true", 2024-09-16T20:48:06.020 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-16T20:48:06.020 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-16T20:48:06.020 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_dump_cache_on_map": "false", 2024-09-16T20:48:06.020 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-16T20:48:06.020 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-16T20:48:06.020 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_early_reply": "true", 2024-09-16T20:48:06.020 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_enable_op_tracker": "true", 2024-09-16T20:48:06.020 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_enforce_unique_name": "true", 2024-09-16T20:48:06.020 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-16T20:48:06.020 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-16T20:48:06.020 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_export_ephemeral_random": "true", 2024-09-16T20:48:06.021 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-16T20:48:06.021 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-16T20:48:06.021 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-16T20:48:06.021 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-16T20:48:06.021 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-16T20:48:06.021 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-16T20:48:06.021 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-16T20:48:06.021 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-16T20:48:06.021 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_health_summarize_threshold": "10", 2024-09-16T20:48:06.021 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-16T20:48:06.021 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-16T20:48:06.021 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_inject_health_dummy": "false", 2024-09-16T20:48:06.022 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-16T20:48:06.022 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_inject_migrator_session_race": "false", 2024-09-16T20:48:06.022 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-16T20:48:06.022 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-16T20:48:06.022 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-16T20:48:06.022 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_join_fs": "", 2024-09-16T20:48:06.022 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_journal_format": "1", 2024-09-16T20:48:06.022 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-16T20:48:06.022 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_kill_create_at": "0", 2024-09-16T20:48:06.022 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_kill_dirfrag_at": "0", 2024-09-16T20:48:06.023 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_kill_export_at": "0", 2024-09-16T20:48:06.023 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_kill_import_at": "0", 2024-09-16T20:48:06.023 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_kill_journal_at": "0", 2024-09-16T20:48:06.023 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_kill_journal_expire_at": "0", 2024-09-16T20:48:06.023 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_kill_journal_replay_at": "0", 2024-09-16T20:48:06.023 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_kill_link_at": "0", 2024-09-16T20:48:06.023 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_kill_mdstable_at": "0", 2024-09-16T20:48:06.023 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_kill_openc_at": "0", 2024-09-16T20:48:06.023 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_kill_rename_at": "0", 2024-09-16T20:48:06.023 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_kill_shutdown_at": "0", 2024-09-16T20:48:06.024 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_log_event_large_threshold": "524288", 2024-09-16T20:48:06.024 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_log_events_per_segment": "1024", 2024-09-16T20:48:06.024 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_log_major_segment_event_ratio": "12", 2024-09-16T20:48:06.024 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_log_max_events": "-1", 2024-09-16T20:48:06.024 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_log_max_segments": "128", 2024-09-16T20:48:06.024 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_log_pause": "false", 2024-09-16T20:48:06.024 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_log_segment_size": "0", 2024-09-16T20:48:06.024 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-16T20:48:06.024 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_log_skip_unbounded_events": "false", 2024-09-16T20:48:06.024 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-09-16T20:48:06.024 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_log_trim_threshold": "128", 2024-09-16T20:48:06.025 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-09-16T20:48:06.025 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_log_warn_factor": "2.000000", 2024-09-16T20:48:06.025 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_max_caps_per_client": "1048576", 2024-09-16T20:48:06.025 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_max_completed_flushes": "100000", 2024-09-16T20:48:06.025 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_max_completed_requests": "100000", 2024-09-16T20:48:06.025 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_max_export_size": "20971520", 2024-09-16T20:48:06.025 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_max_file_recover": "32", 2024-09-16T20:48:06.025 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_max_purge_files": "64", 2024-09-16T20:48:06.025 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_max_purge_ops": "8192", 2024-09-16T20:48:06.025 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-16T20:48:06.025 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-16T20:48:06.025 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_max_snaps_per_dir": "100", 2024-09-16T20:48:06.025 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_metrics_update_interval": "2", 2024-09-16T20:48:06.026 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_min_caps_per_client": "100", 2024-09-16T20:48:06.026 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_min_caps_working_set": "10000", 2024-09-16T20:48:06.026 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-16T20:48:06.026 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_numa_node": "-1", 2024-09-16T20:48:06.026 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-16T20:48:06.026 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_op_complaint_time": "30.000000", 2024-09-16T20:48:06.026 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_op_history_duration": "600", 2024-09-16T20:48:06.026 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_op_history_size": "20", 2024-09-16T20:48:06.026 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_op_history_slow_op_size": "20", 2024-09-16T20:48:06.026 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-16T20:48:06.026 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_op_log_threshold": "5", 2024-09-16T20:48:06.026 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_ping_grace": "15", 2024-09-16T20:48:06.026 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_ping_interval": "5", 2024-09-16T20:48:06.026 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-16T20:48:06.027 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-16T20:48:06.027 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_recall_max_caps": "30000", 2024-09-16T20:48:06.027 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-16T20:48:06.027 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-16T20:48:06.027 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-16T20:48:06.027 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_recall_warning_threshold": "262144", 2024-09-16T20:48:06.027 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-16T20:48:06.027 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_replay_interval": "1.000000", 2024-09-16T20:48:06.027 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-16T20:48:06.027 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-16T20:48:06.028 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_root_ino_gid": "0", 2024-09-16T20:48:06.028 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_root_ino_uid": "0", 2024-09-16T20:48:06.028 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-16T20:48:06.028 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_scrub_stats_review_period": "1", 2024-09-16T20:48:06.028 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_server_dispatch_client_request_delay": "0", 2024-09-16T20:48:06.028 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_server_dispatch_killpoint_random": "0.000000", 2024-09-16T20:48:06.028 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-16T20:48:06.028 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-16T20:48:06.028 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-16T20:48:06.028 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-16T20:48:06.028 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-16T20:48:06.028 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-16T20:48:06.028 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-16T20:48:06.028 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-16T20:48:06.029 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-16T20:48:06.029 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_shutdown_check": "0", 2024-09-16T20:48:06.029 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_skip_ino": "0", 2024-09-16T20:48:06.029 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-16T20:48:06.029 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_snap_max_uid": "4294967294", 2024-09-16T20:48:06.029 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_snap_min_uid": "0", 2024-09-16T20:48:06.029 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_snap_rstat": "false", 2024-09-16T20:48:06.029 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_standby_replay_damaged": "false", 2024-09-16T20:48:06.029 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_symlink_recovery": "true", 2024-09-16T20:48:06.029 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-16T20:48:06.029 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_thrash_exports": "0", 2024-09-16T20:48:06.029 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_thrash_fragments": "0", 2024-09-16T20:48:06.029 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_tick_interval": "5.000000", 2024-09-16T20:48:06.030 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_valgrind_exit": "false", 2024-09-16T20:48:06.030 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_verify_backtrace": "1", 2024-09-16T20:48:06.030 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_verify_scatter": "false", 2024-09-16T20:48:06.030 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-16T20:48:06.030 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_wipe_sessions": "false", 2024-09-16T20:48:06.030 INFO:teuthology.orchestra.run.smithi029.stdout: "mempool_debug": "false", 2024-09-16T20:48:06.030 INFO:teuthology.orchestra.run.smithi029.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-16T20:48:06.030 INFO:teuthology.orchestra.run.smithi029.stdout: "memstore_device_bytes": "1073741824", 2024-09-16T20:48:06.030 INFO:teuthology.orchestra.run.smithi029.stdout: "memstore_page_set": "false", 2024-09-16T20:48:06.030 INFO:teuthology.orchestra.run.smithi029.stdout: "memstore_page_size": "65536", 2024-09-16T20:48:06.030 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr_client_bytes": "134217728", 2024-09-16T20:48:06.030 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr_client_messages": "512", 2024-09-16T20:48:06.030 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-16T20:48:06.031 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-16T20:48:06.031 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-16T20:48:06.031 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-16T20:48:06.031 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr_disabled_modules": "", 2024-09-16T20:48:06.031 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr_enable_op_tracker": "true", 2024-09-16T20:48:06.031 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-16T20:48:06.031 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr_max_pg_creating": "1024", 2024-09-16T20:48:06.031 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr_max_pg_num_change": "128", 2024-09-16T20:48:06.031 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr_mds_bytes": "134217728", 2024-09-16T20:48:06.031 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr_mds_messages": "128", 2024-09-16T20:48:06.031 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-16T20:48:06.031 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr_mon_bytes": "134217728", 2024-09-16T20:48:06.031 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr_mon_messages": "128", 2024-09-16T20:48:06.031 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr_num_op_tracker_shard": "32", 2024-09-16T20:48:06.031 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr_op_complaint_time": "30.000000", 2024-09-16T20:48:06.032 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr_op_history_duration": "600", 2024-09-16T20:48:06.032 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr_op_history_size": "20", 2024-09-16T20:48:06.032 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr_op_history_slow_op_size": "20", 2024-09-16T20:48:06.032 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr_op_history_slow_op_threshold": "10.000000", 2024-09-16T20:48:06.032 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr_op_log_threshold": "5", 2024-09-16T20:48:06.032 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr_osd_bytes": "536870912", 2024-09-16T20:48:06.032 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr_osd_messages": "8192", 2024-09-16T20:48:06.032 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr_pool": "true", 2024-09-16T20:48:06.032 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-16T20:48:06.032 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr_standby_modules": "true", 2024-09-16T20:48:06.032 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr_stats_period": "5", 2024-09-16T20:48:06.032 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr_stats_threshold": "5", 2024-09-16T20:48:06.032 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr_tick_period": "2", 2024-09-16T20:48:06.033 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-16T20:48:06.033 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-16T20:48:06.033 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_allow_pool_delete": "true", 2024-09-16T20:48:06.033 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_allow_pool_size_one": "true", 2024-09-16T20:48:06.033 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_auth_validate_all_caps": "true", 2024-09-16T20:48:06.033 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-16T20:48:06.033 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-16T20:48:06.033 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_client_bytes": "104857600", 2024-09-16T20:48:06.033 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_client_directed_command_retry": "5", 2024-09-16T20:48:06.033 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-16T20:48:06.033 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-16T20:48:06.033 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-16T20:48:06.033 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-16T20:48:06.033 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_client_hunt_parallel": "3", 2024-09-16T20:48:06.034 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_client_log_interval": "1.000000", 2024-09-16T20:48:06.034 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-16T20:48:06.034 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_client_ping_interval": "10.000000", 2024-09-16T20:48:06.034 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-16T20:48:06.034 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_client_target_rank": "-1", 2024-09-16T20:48:06.034 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-16T20:48:06.034 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-16T20:48:06.034 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-16T20:48:06.034 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_cluster_log_level": "debug", 2024-09-16T20:48:06.034 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_cluster_log_to_file": "true", 2024-09-16T20:48:06.034 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-16T20:48:06.034 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-16T20:48:06.034 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-16T20:48:06.035 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_cluster_log_to_journald": "false", 2024-09-16T20:48:06.035 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-16T20:48:06.035 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-16T20:48:06.035 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-16T20:48:06.035 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_compact_on_bootstrap": "false", 2024-09-16T20:48:06.035 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_compact_on_start": "false", 2024-09-16T20:48:06.035 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_compact_on_trim": "true", 2024-09-16T20:48:06.035 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-16T20:48:06.035 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-16T20:48:06.035 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-16T20:48:06.035 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_cpu_threads": "4", 2024-09-16T20:48:06.036 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_crush_min_required_version": "hammer", 2024-09-16T20:48:06.036 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_daemon_bytes": "419430400", 2024-09-16T20:48:06.036 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-16T20:48:06.036 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_data_avail_crit": "5", 2024-09-16T20:48:06.036 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_data_avail_warn": "5", 2024-09-16T20:48:06.036 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_data_size_warn": "16106127360", 2024-09-16T20:48:06.036 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-16T20:48:06.036 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-16T20:48:06.036 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_debug_dump_json": "false", 2024-09-16T20:48:06.037 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-16T20:48:06.037 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_debug_dump_transactions": "false", 2024-09-16T20:48:06.037 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_debug_extra_checks": "false", 2024-09-16T20:48:06.037 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-16T20:48:06.037 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-16T20:48:06.037 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_debug_no_require_reef": "false", 2024-09-16T20:48:06.037 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_debug_no_require_squid": "false", 2024-09-16T20:48:06.037 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-16T20:48:06.037 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-16T20:48:06.037 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-16T20:48:06.037 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_down_mkfs_grace": "120", 2024-09-16T20:48:06.037 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_down_uptime_grace": "60", 2024-09-16T20:48:06.037 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_election_default_strategy": "1", 2024-09-16T20:48:06.038 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_election_timeout": "5.000000", 2024-09-16T20:48:06.038 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-16T20:48:06.038 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_elector_ping_divisor": "2", 2024-09-16T20:48:06.038 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-16T20:48:06.038 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_enable_op_tracker": "true", 2024-09-16T20:48:06.038 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_fake_pool_delete": "false", 2024-09-16T20:48:06.038 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_force_quorum_join": "false", 2024-09-16T20:48:06.038 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_fsmap_prune_threshold": "300", 2024-09-16T20:48:06.038 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_globalid_prealloc": "10000", 2024-09-16T20:48:06.038 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_health_detail_to_clog": "false", 2024-09-16T20:48:06.038 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_health_log_update_period": "5", 2024-09-16T20:48:06.038 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_health_max_detail": "50", 2024-09-16T20:48:06.038 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_health_to_clog": "true", 2024-09-16T20:48:06.038 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_health_to_clog_interval": "600", 2024-09-16T20:48:06.039 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-16T20:48:06.039 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_host": "172.21.15.29,[v2:172.21.15.29:3301,v1:172.21.15.29:6790],172.21.15.43", 2024-09-16T20:48:06.039 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_host_override": "", 2024-09-16T20:48:06.039 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_initial_members": "", 2024-09-16T20:48:06.039 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-16T20:48:06.039 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-16T20:48:06.039 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-16T20:48:06.039 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-16T20:48:06.039 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-16T20:48:06.039 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_lease": "5.000000", 2024-09-16T20:48:06.039 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-16T20:48:06.039 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-16T20:48:06.039 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_log_full_interval": "50", 2024-09-16T20:48:06.039 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_log_max": "10000", 2024-09-16T20:48:06.040 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_log_max_summary": "50", 2024-09-16T20:48:06.040 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-16T20:48:06.040 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_max_log_epochs": "500", 2024-09-16T20:48:06.040 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-16T20:48:06.040 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-16T20:48:06.040 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_max_nvmeof_epochs": "500", 2024-09-16T20:48:06.040 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_max_osd": "10000", 2024-09-16T20:48:06.040 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_max_pg_per_osd": "10000", 2024-09-16T20:48:06.040 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_max_pool_pg_num": "65536", 2024-09-16T20:48:06.040 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-16T20:48:06.040 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-16T20:48:06.040 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_mds_force_trim_to": "0", 2024-09-16T20:48:06.040 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_mds_skip_sanity": "false", 2024-09-16T20:48:06.041 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_memory_autotune": "true", 2024-09-16T20:48:06.041 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_memory_target": "2147483648", 2024-09-16T20:48:06.041 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_mgr_beacon_grace": "30", 2024-09-16T20:48:06.041 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-16T20:48:06.041 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_mgr_digest_period": "5", 2024-09-16T20:48:06.041 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_mgr_inactive_grace": "60", 2024-09-16T20:48:06.041 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-16T20:48:06.041 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-16T20:48:06.041 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_min_osdmap_epochs": "500", 2024-09-16T20:48:06.041 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_nvmeofgw_beacon_grace": "10", 2024-09-16T20:48:06.041 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_nvmeofgw_set_group_id_retry": "1000", 2024-09-16T20:48:06.041 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_op_complaint_time": "30", 2024-09-16T20:48:06.041 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_op_history_duration": "600", 2024-09-16T20:48:06.042 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_op_history_size": "20", 2024-09-16T20:48:06.042 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_op_history_slow_op_size": "20", 2024-09-16T20:48:06.042 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-16T20:48:06.042 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_op_log_threshold": "5", 2024-09-16T20:48:06.042 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-16T20:48:06.042 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-16T20:48:06.042 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-16T20:48:06.042 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_auto_mark_in": "false", 2024-09-16T20:48:06.042 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-16T20:48:06.042 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-16T20:48:06.042 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-16T20:48:06.042 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_cache_size": "500", 2024-09-16T20:48:06.042 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-16T20:48:06.042 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-16T20:48:06.043 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-16T20:48:06.043 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_down_out_interval": "600", 2024-09-16T20:48:06.043 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-16T20:48:06.043 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-16T20:48:06.043 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_force_trim_to": "0", 2024-09-16T20:48:06.043 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-16T20:48:06.043 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-16T20:48:06.043 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-16T20:48:06.043 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-16T20:48:06.043 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-16T20:48:06.043 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-16T20:48:06.043 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-16T20:48:06.043 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_min_down_reporters": "2", 2024-09-16T20:48:06.043 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-16T20:48:06.044 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-16T20:48:06.044 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-16T20:48:06.044 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-16T20:48:06.044 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-16T20:48:06.044 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-16T20:48:06.044 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_report_timeout": "900", 2024-09-16T20:48:06.044 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-16T20:48:06.044 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-16T20:48:06.044 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-16T20:48:06.044 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-16T20:48:06.044 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-16T20:48:06.044 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-16T20:48:06.044 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-16T20:48:06.045 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-16T20:48:06.045 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-16T20:48:06.045 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_pg_stuck_threshold": "60", 2024-09-16T20:48:06.045 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-16T20:48:06.045 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-16T20:48:06.045 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-16T20:48:06.045 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-16T20:48:06.045 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-16T20:48:06.045 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-16T20:48:06.045 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_probe_timeout": "2.000000", 2024-09-16T20:48:06.045 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_reweight_max_change": "0.050000", 2024-09-16T20:48:06.045 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_reweight_max_osds": "4", 2024-09-16T20:48:06.045 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-16T20:48:06.046 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-16T20:48:06.046 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-16T20:48:06.046 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-16T20:48:06.046 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-16T20:48:06.046 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_scrub_interval": "86400", 2024-09-16T20:48:06.046 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_scrub_max_keys": "100", 2024-09-16T20:48:06.046 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_scrub_timeout": "300", 2024-09-16T20:48:06.046 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_session_timeout": "300", 2024-09-16T20:48:06.046 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_smart_report_timeout": "5", 2024-09-16T20:48:06.046 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_stat_smooth_intervals": "6", 2024-09-16T20:48:06.046 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-16T20:48:06.046 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_stretch_pool_min_size": "2", 2024-09-16T20:48:06.046 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_stretch_pool_size": "4", 2024-09-16T20:48:06.047 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-16T20:48:06.047 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-16T20:48:06.047 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_sync_debug": "false", 2024-09-16T20:48:06.047 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-16T20:48:06.047 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-16T20:48:06.047 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_sync_provider_kill_at": "0", 2024-09-16T20:48:06.047 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_sync_requester_kill_at": "0", 2024-09-16T20:48:06.047 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_sync_timeout": "60.000000", 2024-09-16T20:48:06.047 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_target_pg_per_osd": "100", 2024-09-16T20:48:06.047 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_tick_interval": "5", 2024-09-16T20:48:06.047 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_timecheck_interval": "300.000000", 2024-09-16T20:48:06.047 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-16T20:48:06.048 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_use_min_delay_socket": "false", 2024-09-16T20:48:06.048 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_warn_older_version_delay": "604800", 2024-09-16T20:48:06.048 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-16T20:48:06.048 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-16T20:48:06.048 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-16T20:48:06.048 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-16T20:48:06.048 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-16T20:48:06.048 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-16T20:48:06.048 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-16T20:48:06.048 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_warn_on_misplaced": "false", 2024-09-16T20:48:06.048 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-16T20:48:06.048 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_warn_on_older_version": "true", 2024-09-16T20:48:06.048 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-16T20:48:06.049 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_warn_on_pool_no_app": "false", 2024-09-16T20:48:06.049 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-16T20:48:06.049 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-16T20:48:06.049 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-16T20:48:06.049 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-16T20:48:06.049 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-16T20:48:06.049 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-16T20:48:06.049 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-16T20:48:06.049 INFO:teuthology.orchestra.run.smithi029.stdout: "monmap": "", 2024-09-16T20:48:06.049 INFO:teuthology.orchestra.run.smithi029.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-16T20:48:06.049 INFO:teuthology.orchestra.run.smithi029.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-16T20:48:06.049 INFO:teuthology.orchestra.run.smithi029.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-16T20:48:06.049 INFO:teuthology.orchestra.run.smithi029.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-16T20:48:06.049 INFO:teuthology.orchestra.run.smithi029.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-16T20:48:06.050 INFO:teuthology.orchestra.run.smithi029.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-16T20:48:06.050 INFO:teuthology.orchestra.run.smithi029.stdout: "motr_tracing_enabled": "false", 2024-09-16T20:48:06.050 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_async_op_threads": "3", 2024-09-16T20:48:06.050 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-16T20:48:06.050 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_async_rdma_cm": "false", 2024-09-16T20:48:06.050 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_async_rdma_device_name": "", 2024-09-16T20:48:06.050 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_async_rdma_dscp": "96", 2024-09-16T20:48:06.050 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-16T20:48:06.050 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-16T20:48:06.050 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_async_rdma_local_gid": "", 2024-09-16T20:48:06.050 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-16T20:48:06.050 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_async_rdma_port_num": "1", 2024-09-16T20:48:06.050 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-16T20:48:06.050 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-16T20:48:06.051 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-16T20:48:06.051 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-16T20:48:06.051 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_async_rdma_sl": "3", 2024-09-16T20:48:06.051 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_async_rdma_support_srq": "true", 2024-09-16T20:48:06.051 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_async_rdma_type": "ib", 2024-09-16T20:48:06.051 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_async_reap_threshold": "5", 2024-09-16T20:48:06.051 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_bind_before_connect": "false", 2024-09-16T20:48:06.051 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_bind_ipv4": "true", 2024-09-16T20:48:06.051 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_bind_ipv6": "false", 2024-09-16T20:48:06.051 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_bind_msgr1": "true", 2024-09-16T20:48:06.051 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_bind_msgr2": "true", 2024-09-16T20:48:06.051 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_bind_port_max": "7568", 2024-09-16T20:48:06.051 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_bind_port_min": "6800", 2024-09-16T20:48:06.051 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-16T20:48:06.052 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_bind_retry_count": "3", 2024-09-16T20:48:06.052 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_bind_retry_delay": "5", 2024-09-16T20:48:06.052 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_blackhole_client": "false", 2024-09-16T20:48:06.052 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_blackhole_mds": "false", 2024-09-16T20:48:06.052 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_blackhole_mgr": "false", 2024-09-16T20:48:06.052 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_blackhole_mon": "false", 2024-09-16T20:48:06.052 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_blackhole_osd": "false", 2024-09-16T20:48:06.052 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_client_mode": "crc secure", 2024-09-16T20:48:06.052 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_client_throttle_retry_time_interval": "5000", 2024-09-16T20:48:06.052 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_cluster_mode": "crc secure", 2024-09-16T20:48:06.052 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_cluster_type": "", 2024-09-16T20:48:06.052 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_compress_secure": "false", 2024-09-16T20:48:06.052 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_connection_idle_timeout": "900", 2024-09-16T20:48:06.052 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_connection_ready_timeout": "10", 2024-09-16T20:48:06.053 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_crc_data": "true", 2024-09-16T20:48:06.053 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_crc_header": "true", 2024-09-16T20:48:06.053 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_die_on_bad_msg": "false", 2024-09-16T20:48:06.053 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_die_on_bug": "true", 2024-09-16T20:48:06.053 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_die_on_old_message": "true", 2024-09-16T20:48:06.053 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_die_on_skipped_message": "false", 2024-09-16T20:48:06.053 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-16T20:48:06.053 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-16T20:48:06.053 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_dpdk_coremask": "0xF", 2024-09-16T20:48:06.053 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-16T20:48:06.053 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-16T20:48:06.053 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_dpdk_enable_tso": "true", 2024-09-16T20:48:06.053 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-16T20:48:06.053 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-16T20:48:06.054 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_dpdk_hugepages": "", 2024-09-16T20:48:06.054 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-16T20:48:06.054 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-16T20:48:06.054 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_dpdk_lro": "true", 2024-09-16T20:48:06.054 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_dpdk_memory_channel": "4", 2024-09-16T20:48:06.054 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-16T20:48:06.054 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_dpdk_pmd": "", 2024-09-16T20:48:06.054 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_dpdk_port_id": "0", 2024-09-16T20:48:06.054 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-16T20:48:06.054 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-16T20:48:06.054 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_dump_on_send": "false", 2024-09-16T20:48:06.054 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_initial_backoff": "0.200000", 2024-09-16T20:48:06.055 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_inject_delay_max": "1.000000", 2024-09-16T20:48:06.055 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-16T20:48:06.055 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_inject_delay_type": "", 2024-09-16T20:48:06.055 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-16T20:48:06.055 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_inject_network_congestion": "0", 2024-09-16T20:48:06.055 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_inject_socket_failures": "5000", 2024-09-16T20:48:06.055 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_learn_addr_from_peer": "true", 2024-09-16T20:48:06.055 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_max_accept_failures": "4", 2024-09-16T20:48:06.055 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_max_backoff": "15.000000", 2024-09-16T20:48:06.055 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_mon_client_mode": "secure crc", 2024-09-16T20:48:06.055 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-16T20:48:06.055 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_mon_service_mode": "secure crc", 2024-09-16T20:48:06.055 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_osd_compress_min_size": "1024", 2024-09-16T20:48:06.055 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_osd_compress_mode": "none", 2024-09-16T20:48:06.055 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-16T20:48:06.056 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-16T20:48:06.056 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_pq_min_cost": "65536", 2024-09-16T20:48:06.056 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_public_type": "", 2024-09-16T20:48:06.056 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_service_mode": "crc secure", 2024-09-16T20:48:06.056 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_tcp_listen_backlog": "512", 2024-09-16T20:48:06.056 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_tcp_nodelay": "true", 2024-09-16T20:48:06.056 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-16T20:48:06.056 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_tcp_rcvbuf": "0", 2024-09-16T20:48:06.056 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_time_events_min_wait_interval": "1000", 2024-09-16T20:48:06.056 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_type": "async+posix", 2024-09-16T20:48:06.056 INFO:teuthology.orchestra.run.smithi029.stdout: "no_config_file": "false", 2024-09-16T20:48:06.056 INFO:teuthology.orchestra.run.smithi029.stdout: "nvmeof_mon_client_disconnect_panic": "100", 2024-09-16T20:48:06.056 INFO:teuthology.orchestra.run.smithi029.stdout: "nvmeof_mon_client_tick_period": "2", 2024-09-16T20:48:06.057 INFO:teuthology.orchestra.run.smithi029.stdout: "objecter_completion_locks_per_session": "32", 2024-09-16T20:48:06.057 INFO:teuthology.orchestra.run.smithi029.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-16T20:48:06.057 INFO:teuthology.orchestra.run.smithi029.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-16T20:48:06.057 INFO:teuthology.orchestra.run.smithi029.stdout: "objecter_inflight_ops": "1024", 2024-09-16T20:48:06.057 INFO:teuthology.orchestra.run.smithi029.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-16T20:48:06.057 INFO:teuthology.orchestra.run.smithi029.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-16T20:48:06.057 INFO:teuthology.orchestra.run.smithi029.stdout: "objecter_tick_interval": "5.000000", 2024-09-16T20:48:06.057 INFO:teuthology.orchestra.run.smithi029.stdout: "objecter_timeout": "10.000000", 2024-09-16T20:48:06.057 INFO:teuthology.orchestra.run.smithi029.stdout: "objectstore_blackhole": "false", 2024-09-16T20:48:06.057 INFO:teuthology.orchestra.run.smithi029.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-09-16T20:48:06.057 INFO:teuthology.orchestra.run.smithi029.stdout: "openssl_engine_opts": "", 2024-09-16T20:48:06.057 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_agent_delay_time": "5.000000", 2024-09-16T20:48:06.057 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_agent_hist_halflife": "1000", 2024-09-16T20:48:06.057 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_agent_max_low_ops": "2", 2024-09-16T20:48:06.058 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_agent_max_ops": "4", 2024-09-16T20:48:06.058 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-16T20:48:06.058 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-16T20:48:06.058 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_agent_slop": "0.020000", 2024-09-16T20:48:06.058 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-16T20:48:06.058 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-16T20:48:06.058 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_asio_thread_count": "2", 2024-09-16T20:48:06.058 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_async_recovery_min_cost": "100", 2024-09-16T20:48:06.058 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-16T20:48:06.058 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-16T20:48:06.058 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_backfill_scan_max": "512", 2024-09-16T20:48:06.058 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_backfill_scan_min": "64", 2024-09-16T20:48:06.058 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_backoff_on_degraded": "false", 2024-09-16T20:48:06.059 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_backoff_on_peering": "false", 2024-09-16T20:48:06.059 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_backoff_on_unfound": "true", 2024-09-16T20:48:06.059 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_beacon_report_interval": "300", 2024-09-16T20:48:06.059 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_bench_duration": "30", 2024-09-16T20:48:06.059 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-16T20:48:06.059 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_bench_max_block_size": "67108864", 2024-09-16T20:48:06.059 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-16T20:48:06.059 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_blkin_trace_all": "false", 2024-09-16T20:48:06.059 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-16T20:48:06.059 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-16T20:48:06.059 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-16T20:48:06.059 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-16T20:48:06.059 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_check_for_log_corruption": "false", 2024-09-16T20:48:06.060 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-16T20:48:06.060 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-16T20:48:06.060 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-09-16T20:48:06.060 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-16T20:48:06.060 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_class_update_on_start": "true", 2024-09-16T20:48:06.060 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_client_message_cap": "256", 2024-09-16T20:48:06.060 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_client_message_size_cap": "524288000", 2024-09-16T20:48:06.060 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_client_op_priority": "63", 2024-09-16T20:48:06.060 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_client_watch_timeout": "30", 2024-09-16T20:48:06.060 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_command_max_records": "256", 2024-09-16T20:48:06.060 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_compact_on_start": "false", 2024-09-16T20:48:06.060 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-16T20:48:06.060 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-16T20:48:06.061 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-16T20:48:06.061 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_crush_update_on_start": "true", 2024-09-16T20:48:06.061 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_crush_update_weight_set": "true", 2024-09-16T20:48:06.061 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-16T20:48:06.061 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-16T20:48:06.061 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-16T20:48:06.061 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-16T20:48:06.061 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-16T20:48:06.061 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_debug_feed_pullee": "-1", 2024-09-16T20:48:06.061 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-16T20:48:06.061 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-16T20:48:06.061 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-16T20:48:06.061 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_debug_misdirected_ops": "false", 2024-09-16T20:48:06.062 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_debug_no_acting_change": "false", 2024-09-16T20:48:06.062 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_debug_no_purge_strays": "false", 2024-09-16T20:48:06.062 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_debug_op_order": "false", 2024-09-16T20:48:06.062 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-16T20:48:06.062 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-16T20:48:06.062 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-16T20:48:06.062 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-16T20:48:06.062 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_debug_shutdown": "false", 2024-09-16T20:48:06.062 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-16T20:48:06.062 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-16T20:48:06.062 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_debug_trim_objects": "false", 2024-09-16T20:48:06.062 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-16T20:48:06.062 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-16T20:48:06.062 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_debug_verify_snaps": "false", 2024-09-16T20:48:06.063 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-16T20:48:06.063 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-16T20:48:06.063 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_deep_scrub_keys": "1024", 2024-09-16T20:48:06.063 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-16T20:48:06.063 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-16T20:48:06.063 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-16T20:48:06.063 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_deep_scrub_stride": "524288", 2024-09-16T20:48:06.063 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-16T20:48:06.063 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-16T20:48:06.063 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_default_notify_timeout": "30", 2024-09-16T20:48:06.063 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_delete_sleep": "0.000000", 2024-09-16T20:48:06.063 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-16T20:48:06.063 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-16T20:48:06.063 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-16T20:48:06.064 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_discard_disconnected_ops": "true", 2024-09-16T20:48:06.064 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_ec_partial_reads": "true", 2024-09-16T20:48:06.064 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_enable_op_tracker": "true", 2024-09-16T20:48:06.064 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-16T20:48:06.064 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-16T20:48:06.064 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-16T20:48:06.064 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-16T20:48:06.064 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_fast_info": "true", 2024-09-16T20:48:06.064 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_fast_shutdown": "true", 2024-09-16T20:48:06.064 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-16T20:48:06.064 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-16T20:48:06.064 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-16T20:48:06.064 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-16T20:48:06.065 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-16T20:48:06.065 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_function_tracing": "false", 2024-09-16T20:48:06.065 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_heartbeat_grace": "20", 2024-09-16T20:48:06.065 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_heartbeat_interval": "6", 2024-09-16T20:48:06.065 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-16T20:48:06.065 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_heartbeat_min_peers": "10", 2024-09-16T20:48:06.065 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_heartbeat_min_size": "2000", 2024-09-16T20:48:06.065 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_heartbeat_stale": "600", 2024-09-16T20:48:06.065 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-16T20:48:06.065 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_hit_set_max_size": "100000", 2024-09-16T20:48:06.065 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_hit_set_min_size": "1000", 2024-09-16T20:48:06.065 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-16T20:48:06.065 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-16T20:48:06.065 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-16T20:48:06.066 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-16T20:48:06.066 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-16T20:48:06.066 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-16T20:48:06.066 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_journal_size": "5120", 2024-09-16T20:48:06.066 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_kill_backfill_at": "0", 2024-09-16T20:48:06.066 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-16T20:48:06.066 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_map_cache_size": "50", 2024-09-16T20:48:06.066 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_map_dedup": "true", 2024-09-16T20:48:06.066 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_map_message_max": "40", 2024-09-16T20:48:06.066 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-16T20:48:06.066 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_map_share_max_epochs": "40", 2024-09-16T20:48:06.066 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_max_attr_name_len": "100", 2024-09-16T20:48:06.066 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_max_attr_size": "0", 2024-09-16T20:48:06.066 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_max_backfills": "1", 2024-09-16T20:48:06.067 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_max_markdown_count": "5", 2024-09-16T20:48:06.067 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_max_markdown_period": "600", 2024-09-16T20:48:06.067 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_max_object_name_len": "2048", 2024-09-16T20:48:06.067 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_max_object_namespace_len": "256", 2024-09-16T20:48:06.067 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_max_object_size": "134217728", 2024-09-16T20:48:06.067 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-16T20:48:06.067 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-16T20:48:06.067 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_max_pg_blocked_by": "16", 2024-09-16T20:48:06.067 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_max_pg_log_entries": "10000", 2024-09-16T20:48:06.067 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-16T20:48:06.067 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_max_pgls": "1024", 2024-09-16T20:48:06.067 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_max_push_cost": "8388608", 2024-09-16T20:48:06.067 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_max_push_objects": "10", 2024-09-16T20:48:06.068 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_max_scrubs": "3", 2024-09-16T20:48:06.068 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-16T20:48:06.068 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_max_trimming_pgs": "2", 2024-09-16T20:48:06.068 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_max_write_op_reply_len": "64", 2024-09-16T20:48:06.068 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_max_write_size": "90", 2024-09-16T20:48:06.068 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-16T20:48:06.068 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-16T20:48:06.068 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-16T20:48:06.068 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-16T20:48:06.068 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-16T20:48:06.068 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-16T20:48:06.068 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-16T20:48:06.068 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-16T20:48:06.069 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mclock_profile": "balanced", 2024-09-16T20:48:06.069 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-16T20:48:06.069 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-16T20:48:06.069 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-16T20:48:06.069 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-16T20:48:06.069 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-16T20:48:06.069 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-16T20:48:06.069 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-16T20:48:06.069 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-16T20:48:06.069 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-16T20:48:06.069 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-16T20:48:06.069 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-16T20:48:06.069 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_memory_base": "805306368", 2024-09-16T20:48:06.070 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_memory_cache_min": "134217728", 2024-09-16T20:48:06.070 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-16T20:48:06.070 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-16T20:48:06.070 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_memory_target": "4294967296", 2024-09-16T20:48:06.070 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_memory_target_autotune": "false", 2024-09-16T20:48:06.070 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-16T20:48:06.070 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_min_pg_log_entries": "250", 2024-09-16T20:48:06.070 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_min_recovery_priority": "0", 2024-09-16T20:48:06.070 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-16T20:48:06.070 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-16T20:48:06.070 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mon_report_interval": "5", 2024-09-16T20:48:06.070 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-16T20:48:06.070 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-16T20:48:06.070 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_num_cache_shards": "32", 2024-09-16T20:48:06.071 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_num_op_tracker_shard": "32", 2024-09-16T20:48:06.071 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_numa_auto_affinity": "true", 2024-09-16T20:48:06.071 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_numa_node": "-1", 2024-09-16T20:48:06.071 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_numa_prefer_iface": "true", 2024-09-16T20:48:06.071 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-16T20:48:06.071 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_objecter_finishers": "1", 2024-09-16T20:48:06.071 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_objectstore": "bluestore", 2024-09-16T20:48:06.071 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_objectstore_fuse": "false", 2024-09-16T20:48:06.071 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_objectstore_ideal_list_max": "64", 2024-09-16T20:48:06.071 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_objectstore_tracing": "false", 2024-09-16T20:48:06.071 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_op_complaint_time": "30.000000", 2024-09-16T20:48:06.071 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_op_history_duration": "600", 2024-09-16T20:48:06.071 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_op_history_size": "20", 2024-09-16T20:48:06.072 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_op_history_slow_op_size": "20", 2024-09-16T20:48:06.072 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-16T20:48:06.072 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_op_log_threshold": "5", 2024-09-16T20:48:06.072 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_op_num_shards": "0", 2024-09-16T20:48:06.072 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_op_num_shards_hdd": "5", 2024-09-16T20:48:06.072 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_op_num_shards_ssd": "8", 2024-09-16T20:48:06.072 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-16T20:48:06.072 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-16T20:48:06.072 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-16T20:48:06.072 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-16T20:48:06.072 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_op_pq_min_cost": "65536", 2024-09-16T20:48:06.072 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-16T20:48:06.072 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_op_queue_cut_off": "high", 2024-09-16T20:48:06.072 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-16T20:48:06.073 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_op_thread_timeout": "15", 2024-09-16T20:48:06.073 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_open_classes_on_start": "true", 2024-09-16T20:48:06.073 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_os_flags": "0", 2024-09-16T20:48:06.073 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_peering_op_priority": "255", 2024-09-16T20:48:06.073 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pg_delete_cost": "1048576", 2024-09-16T20:48:06.073 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pg_delete_priority": "5", 2024-09-16T20:48:06.073 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-16T20:48:06.073 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-16T20:48:06.073 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-16T20:48:06.073 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pg_log_trim_max": "10000", 2024-09-16T20:48:06.073 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pg_log_trim_min": "100", 2024-09-16T20:48:06.073 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-16T20:48:06.073 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-16T20:48:06.073 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-09-16T20:48:06.074 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-09-16T20:48:06.074 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-16T20:48:06.074 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-16T20:48:06.074 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-16T20:48:06.074 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-16T20:48:06.074 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-16T20:48:06.074 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-16T20:48:06.074 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pool_default_crimson": "false", 2024-09-16T20:48:06.074 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-16T20:48:06.074 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-16T20:48:06.074 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-16T20:48:06.074 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-16T20:48:06.074 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-16T20:48:06.075 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-16T20:48:06.075 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-16T20:48:06.075 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-16T20:48:06.075 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pool_default_flags": "0", 2024-09-16T20:48:06.075 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-16T20:48:06.075 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pool_default_min_size": "0", 2024-09-16T20:48:06.075 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-16T20:48:06.075 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pool_default_pg_num": "32", 2024-09-16T20:48:06.075 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pool_default_pgp_num": "0", 2024-09-16T20:48:06.075 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-16T20:48:06.075 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pool_default_read_ratio": "70", 2024-09-16T20:48:06.075 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pool_default_size": "2", 2024-09-16T20:48:06.075 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pool_default_type": "replicated", 2024-09-16T20:48:06.076 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-16T20:48:06.076 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-16T20:48:06.076 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_push_per_object_cost": "1000", 2024-09-16T20:48:06.076 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-16T20:48:06.076 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_recover_clone_overlap": "true", 2024-09-16T20:48:06.076 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-16T20:48:06.076 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_recovery_cost": "20971520", 2024-09-16T20:48:06.076 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-16T20:48:06.076 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_recovery_max_active": "0", 2024-09-16T20:48:06.076 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-16T20:48:06.076 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-16T20:48:06.076 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-16T20:48:06.076 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-16T20:48:06.076 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_recovery_max_single_start": "1", 2024-09-16T20:48:06.077 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_recovery_op_priority": "3", 2024-09-16T20:48:06.077 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-16T20:48:06.077 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_recovery_priority": "5", 2024-09-16T20:48:06.077 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-16T20:48:06.077 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_recovery_sleep": "0.000000", 2024-09-16T20:48:06.077 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-16T20:48:06.077 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-16T20:48:06.077 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-16T20:48:06.077 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_repair_during_recovery": "false", 2024-09-16T20:48:06.077 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_requested_scrub_priority": "120", 2024-09-16T20:48:06.077 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-16T20:48:06.077 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-16T20:48:06.077 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_scrub_auto_repair": "false", 2024-09-16T20:48:06.078 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-16T20:48:06.078 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-16T20:48:06.078 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_scrub_begin_hour": "0", 2024-09-16T20:48:06.078 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_scrub_begin_week_day": "0", 2024-09-16T20:48:06.078 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_scrub_chunk_max": "25", 2024-09-16T20:48:06.078 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_scrub_chunk_min": "5", 2024-09-16T20:48:06.078 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_scrub_cost": "52428800", 2024-09-16T20:48:06.078 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_scrub_disable_reservation_queuing": "false", 2024-09-16T20:48:06.078 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_scrub_during_recovery": "false", 2024-09-16T20:48:06.078 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_scrub_end_hour": "0", 2024-09-16T20:48:06.078 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_scrub_end_week_day": "0", 2024-09-16T20:48:06.078 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_scrub_event_cost": "4096", 2024-09-16T20:48:06.078 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-16T20:48:06.078 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-16T20:48:06.079 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_scrub_invalid_stats": "true", 2024-09-16T20:48:06.079 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-16T20:48:06.079 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-16T20:48:06.079 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_scrub_max_preemptions": "5", 2024-09-16T20:48:06.079 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-16T20:48:06.079 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_scrub_priority": "5", 2024-09-16T20:48:06.079 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_scrub_retry_after_noscrub": "60", 2024-09-16T20:48:06.079 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_scrub_retry_delay": "30", 2024-09-16T20:48:06.079 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_scrub_retry_new_interval": "10", 2024-09-16T20:48:06.079 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_scrub_retry_pg_state": "60", 2024-09-16T20:48:06.079 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_scrub_retry_trimming": "10", 2024-09-16T20:48:06.079 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_scrub_sleep": "0.000000", 2024-09-16T20:48:06.079 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-16T20:48:06.079 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-16T20:48:06.079 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-16T20:48:06.080 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_skip_check_past_interval_bounds": "false", 2024-09-16T20:48:06.080 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_skip_data_digest": "false", 2024-09-16T20:48:06.080 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_smart_report_timeout": "5", 2024-09-16T20:48:06.080 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_snap_trim_cost": "1048576", 2024-09-16T20:48:06.080 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_snap_trim_priority": "5", 2024-09-16T20:48:06.080 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-16T20:48:06.080 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-16T20:48:06.080 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-16T20:48:06.080 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-16T20:48:06.080 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-16T20:48:06.080 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-16T20:48:06.080 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-16T20:48:06.080 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_target_transaction_size": "30", 2024-09-16T20:48:06.080 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-16T20:48:06.081 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-16T20:48:06.081 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-16T20:48:06.081 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-16T20:48:06.081 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-16T20:48:06.081 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-16T20:48:06.081 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-16T20:48:06.081 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-16T20:48:06.081 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-16T20:48:06.081 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-16T20:48:06.081 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_tracing": "false", 2024-09-16T20:48:06.081 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_use_stale_snap": "false", 2024-09-16T20:48:06.081 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-16T20:48:06.082 INFO:teuthology.orchestra.run.smithi029.stdout: "osdc_blkin_trace_all": "false", 2024-09-16T20:48:06.082 INFO:teuthology.orchestra.run.smithi029.stdout: "paxos_kill_at": "0", 2024-09-16T20:48:06.082 INFO:teuthology.orchestra.run.smithi029.stdout: "paxos_max_join_drift": "10", 2024-09-16T20:48:06.082 INFO:teuthology.orchestra.run.smithi029.stdout: "paxos_min": "500", 2024-09-16T20:48:06.082 INFO:teuthology.orchestra.run.smithi029.stdout: "paxos_min_wait": "0.050000", 2024-09-16T20:48:06.082 INFO:teuthology.orchestra.run.smithi029.stdout: "paxos_propose_interval": "1.000000", 2024-09-16T20:48:06.082 INFO:teuthology.orchestra.run.smithi029.stdout: "paxos_service_trim_max": "500", 2024-09-16T20:48:06.082 INFO:teuthology.orchestra.run.smithi029.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-16T20:48:06.082 INFO:teuthology.orchestra.run.smithi029.stdout: "paxos_service_trim_min": "250", 2024-09-16T20:48:06.082 INFO:teuthology.orchestra.run.smithi029.stdout: "paxos_stash_full_interval": "25", 2024-09-16T20:48:06.082 INFO:teuthology.orchestra.run.smithi029.stdout: "paxos_trim_max": "500", 2024-09-16T20:48:06.082 INFO:teuthology.orchestra.run.smithi029.stdout: "paxos_trim_min": "250", 2024-09-16T20:48:06.083 INFO:teuthology.orchestra.run.smithi029.stdout: "perf": "true", 2024-09-16T20:48:06.083 INFO:teuthology.orchestra.run.smithi029.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-16T20:48:06.083 INFO:teuthology.orchestra.run.smithi029.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-16T20:48:06.083 INFO:teuthology.orchestra.run.smithi029.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-09-16T20:48:06.083 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "-", 2024-09-16T20:48:06.083 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrv": "", 2024-09-16T20:48:06.083 INFO:teuthology.orchestra.run.smithi029.stdout: "public_bind_addr": "-", 2024-09-16T20:48:06.083 INFO:teuthology.orchestra.run.smithi029.stdout: "public_network": "", 2024-09-16T20:48:06.083 INFO:teuthology.orchestra.run.smithi029.stdout: "public_network_interface": "", 2024-09-16T20:48:06.083 INFO:teuthology.orchestra.run.smithi029.stdout: "qat_compressor_busy_polling": "false", 2024-09-16T20:48:06.083 INFO:teuthology.orchestra.run.smithi029.stdout: "qat_compressor_enabled": "false", 2024-09-16T20:48:06.083 INFO:teuthology.orchestra.run.smithi029.stdout: "qat_compressor_session_max_number": "256", 2024-09-16T20:48:06.083 INFO:teuthology.orchestra.run.smithi029.stdout: "rados_mon_op_timeout": "0", 2024-09-16T20:48:06.083 INFO:teuthology.orchestra.run.smithi029.stdout: "rados_osd_op_timeout": "0", 2024-09-16T20:48:06.083 INFO:teuthology.orchestra.run.smithi029.stdout: "rados_tracing": "false", 2024-09-16T20:48:06.084 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_atime_update_interval": "60", 2024-09-16T20:48:06.084 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-16T20:48:06.084 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_balance_parent_reads": "false", 2024-09-16T20:48:06.084 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_balance_snap_reads": "false", 2024-09-16T20:48:06.084 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_blkin_trace_all": "false", 2024-09-16T20:48:06.084 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-16T20:48:06.084 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-16T20:48:06.084 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_cache": "true", 2024-09-16T20:48:06.084 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-16T20:48:06.084 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-16T20:48:06.084 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-16T20:48:06.084 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-16T20:48:06.084 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_cache_policy": "writearound", 2024-09-16T20:48:06.084 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_cache_size": "33554432", 2024-09-16T20:48:06.085 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-16T20:48:06.085 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-16T20:48:06.085 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_clone_copy_on_read": "false", 2024-09-16T20:48:06.085 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_compression_hint": "none", 2024-09-16T20:48:06.085 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_concurrent_management_ops": "10", 2024-09-16T20:48:06.085 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-16T20:48:06.085 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_default_clone_format": "auto", 2024-09-16T20:48:06.085 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_default_data_pool": "", 2024-09-16T20:48:06.085 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_default_features": "61", 2024-09-16T20:48:06.085 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_default_format": "2", 2024-09-16T20:48:06.085 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_default_map_options": "", 2024-09-16T20:48:06.085 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_default_order": "22", 2024-09-16T20:48:06.085 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_default_pool": "rbd", 2024-09-16T20:48:06.085 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-16T20:48:06.085 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_default_stripe_count": "0", 2024-09-16T20:48:06.086 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_default_stripe_unit": "0", 2024-09-16T20:48:06.086 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-16T20:48:06.086 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-16T20:48:06.086 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-16T20:48:06.086 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_enable_alloc_hint": "true", 2024-09-16T20:48:06.086 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-16T20:48:06.086 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_io_scheduler": "simple", 2024-09-16T20:48:06.086 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-16T20:48:06.086 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-16T20:48:06.086 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-16T20:48:06.086 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-16T20:48:06.086 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-16T20:48:06.086 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-16T20:48:06.086 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-16T20:48:06.087 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-16T20:48:06.087 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-16T20:48:06.087 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_journal_order": "24", 2024-09-16T20:48:06.087 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_journal_pool": "", 2024-09-16T20:48:06.087 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_journal_splay_width": "4", 2024-09-16T20:48:06.087 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_localize_parent_reads": "false", 2024-09-16T20:48:06.087 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_localize_snap_reads": "false", 2024-09-16T20:48:06.087 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-16T20:48:06.087 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-16T20:48:06.087 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-16T20:48:06.087 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-16T20:48:06.087 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-16T20:48:06.087 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-16T20:48:06.087 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-16T20:48:06.088 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-16T20:48:06.088 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-16T20:48:06.088 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-16T20:48:06.088 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-16T20:48:06.088 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-16T20:48:06.088 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-16T20:48:06.088 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-16T20:48:06.088 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-16T20:48:06.088 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-16T20:48:06.088 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-16T20:48:06.088 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-16T20:48:06.088 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-16T20:48:06.088 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-16T20:48:06.088 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-16T20:48:06.089 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-16T20:48:06.089 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-16T20:48:06.089 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-16T20:48:06.089 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-16T20:48:06.089 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-16T20:48:06.089 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-16T20:48:06.089 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-16T20:48:06.089 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-16T20:48:06.089 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-16T20:48:06.089 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-16T20:48:06.089 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_mtime_update_interval": "60", 2024-09-16T20:48:06.089 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_non_blocking_aio": "true", 2024-09-16T20:48:06.089 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_op_thread_timeout": "60", 2024-09-16T20:48:06.089 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_op_threads": "1", 2024-09-16T20:48:06.090 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_parent_cache_enabled": "false", 2024-09-16T20:48:06.090 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-16T20:48:06.090 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-16T20:48:06.090 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-16T20:48:06.090 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_plugins": "", 2024-09-16T20:48:06.090 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_qos_bps_burst": "0", 2024-09-16T20:48:06.090 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-16T20:48:06.090 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_qos_bps_limit": "0", 2024-09-16T20:48:06.090 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_qos_exclude_ops": "0", 2024-09-16T20:48:06.090 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_qos_iops_burst": "0", 2024-09-16T20:48:06.090 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-16T20:48:06.090 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_qos_iops_limit": "0", 2024-09-16T20:48:06.090 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-16T20:48:06.090 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-16T20:48:06.091 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-16T20:48:06.091 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-16T20:48:06.091 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-16T20:48:06.091 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-16T20:48:06.091 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-16T20:48:06.091 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-16T20:48:06.091 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-16T20:48:06.091 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-16T20:48:06.091 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-16T20:48:06.091 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-16T20:48:06.091 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-16T20:48:06.091 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-16T20:48:06.091 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_read_from_replica_policy": "default", 2024-09-16T20:48:06.091 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-16T20:48:06.091 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-16T20:48:06.092 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-16T20:48:06.092 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-16T20:48:06.092 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_skip_partial_discard": "true", 2024-09-16T20:48:06.092 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-16T20:48:06.092 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_tracing": "false", 2024-09-16T20:48:06.092 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_validate_names": "true", 2024-09-16T20:48:06.092 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_validate_pool": "true", 2024-09-16T20:48:06.092 INFO:teuthology.orchestra.run.smithi029.stdout: "restapi_base_url": "", 2024-09-16T20:48:06.092 INFO:teuthology.orchestra.run.smithi029.stdout: "restapi_log_level": "", 2024-09-16T20:48:06.092 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_account_default_quota_max_objects": "-1", 2024-09-16T20:48:06.092 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_account_default_quota_max_size": "-1", 2024-09-16T20:48:06.092 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_acl_grants_max_num": "100", 2024-09-16T20:48:06.092 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_admin_entry": "admin", 2024-09-16T20:48:06.093 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-16T20:48:06.093 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_asio_assert_yielding": "false", 2024-09-16T20:48:06.093 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_backend_store": "rados", 2024-09-16T20:48:06.093 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_barbican_url": "", 2024-09-16T20:48:06.093 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_beast_enable_async": "true", 2024-09-16T20:48:06.093 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_bucket_counters_cache": "false", 2024-09-16T20:48:06.093 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-09-16T20:48:06.093 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-16T20:48:06.093 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-16T20:48:06.093 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-16T20:48:06.093 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-16T20:48:06.094 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-16T20:48:06.094 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-16T20:48:06.094 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-16T20:48:06.094 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_cache_enabled": "true", 2024-09-16T20:48:06.094 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_cache_expiry_interval": "900", 2024-09-16T20:48:06.094 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_cache_lru_size": "25000", 2024-09-16T20:48:06.094 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_config_store": "rados", 2024-09-16T20:48:06.094 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_content_length_compat": "false", 2024-09-16T20:48:06.095 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_copy_obj_progress": "true", 2024-09-16T20:48:06.095 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-16T20:48:06.095 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_cors_rules_max_num": "100", 2024-09-16T20:48:06.095 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_cross_domain_policy": "", 2024-09-16T20:48:06.095 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-16T20:48:06.095 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_kmip_addr": "", 2024-09-16T20:48:06.095 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-16T20:48:06.095 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-16T20:48:06.095 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-16T20:48:06.095 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-16T20:48:06.095 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_kmip_password": "", 2024-09-16T20:48:06.095 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-16T20:48:06.095 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_kmip_username": "", 2024-09-16T20:48:06.095 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_require_ssl": "true", 2024-09-16T20:48:06.096 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-16T20:48:06.096 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-16T20:48:06.096 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-16T20:48:06.096 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-16T20:48:06.096 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-16T20:48:06.096 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-16T20:48:06.096 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-16T20:48:06.096 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-16T20:48:06.096 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-16T20:48:06.096 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-16T20:48:06.096 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-16T20:48:06.097 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-16T20:48:06.097 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-16T20:48:06.097 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-16T20:48:06.097 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-16T20:48:06.097 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_vault_addr": "", 2024-09-16T20:48:06.097 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_vault_auth": "token", 2024-09-16T20:48:06.098 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_vault_namespace": "", 2024-09-16T20:48:06.098 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_vault_prefix": "", 2024-09-16T20:48:06.098 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-16T20:48:06.098 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-16T20:48:06.098 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-16T20:48:06.098 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-16T20:48:06.099 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_vault_token_file": "", 2024-09-16T20:48:06.099 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-16T20:48:06.099 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_curl_buffersize": "524288", 2024-09-16T20:48:06.099 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-16T20:48:06.099 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_curl_low_speed_time": "300", 2024-09-16T20:48:06.099 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-16T20:48:06.099 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-16T20:48:06.100 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-16T20:48:06.100 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-16T20:48:06.100 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-16T20:48:06.100 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-16T20:48:06.100 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-16T20:48:06.101 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-16T20:48:06.101 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-16T20:48:06.101 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-16T20:48:06.101 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_d4n_address": "127.0.0.1:6379", 2024-09-16T20:48:06.101 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_d4n_l1_datacache_address": "127.0.0.1:6379", 2024-09-16T20:48:06.101 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_d4n_l1_datacache_persistent_path": "/tmp/rgw_d4n_datacache/", 2024-09-16T20:48:06.102 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_d4n_l1_datacache_size": "1073741824", 2024-09-16T20:48:06.102 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_d4n_l1_evict_cache_on_start": "true", 2024-09-16T20:48:06.102 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_d4n_l1_fadvise": "4", 2024-09-16T20:48:06.102 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_d4n_libaio_aio_num": "64", 2024-09-16T20:48:06.102 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_d4n_libaio_aio_threads": "20", 2024-09-16T20:48:06.103 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-16T20:48:06.103 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_data_log_changes_size": "1000", 2024-09-16T20:48:06.103 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_data_log_num_shards": "128", 2024-09-16T20:48:06.103 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-16T20:48:06.103 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_data_log_window": "30", 2024-09-16T20:48:06.103 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-16T20:48:06.104 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-16T20:48:06.104 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-16T20:48:06.104 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-16T20:48:06.104 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-16T20:48:06.104 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-16T20:48:06.105 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-16T20:48:06.105 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-16T20:48:06.105 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-16T20:48:06.105 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-16T20:48:06.106 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-16T20:48:06.106 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-16T20:48:06.106 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-16T20:48:06.106 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_disable_s3select": "false", 2024-09-16T20:48:06.106 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-16T20:48:06.106 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-16T20:48:06.107 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-16T20:48:06.107 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-16T20:48:06.107 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-16T20:48:06.107 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-16T20:48:06.107 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-16T20:48:06.107 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-16T20:48:06.108 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-16T20:48:06.108 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-16T20:48:06.108 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-16T20:48:06.108 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-16T20:48:06.108 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_dns_name": "", 2024-09-16T20:48:06.108 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_dns_s3website_name": "", 2024-09-16T20:48:06.109 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_dynamic_resharding": "true", 2024-09-16T20:48:06.109 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_dynamic_resharding_may_reduce": "true", 2024-09-16T20:48:06.109 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_dynamic_resharding_reduction_wait": "120", 2024-09-16T20:48:06.109 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-16T20:48:06.109 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_enable_gc_threads": "true", 2024-09-16T20:48:06.110 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_enable_lc_threads": "true", 2024-09-16T20:48:06.110 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_enable_mdsearch": "true", 2024-09-16T20:48:06.110 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_enable_ops_log": "false", 2024-09-16T20:48:06.110 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_enable_quota_threads": "true", 2024-09-16T20:48:06.110 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_enable_static_website": "false", 2024-09-16T20:48:06.110 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_enable_usage_log": "false", 2024-09-16T20:48:06.111 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_enforce_swift_acls": "true", 2024-09-16T20:48:06.111 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_exit_timeout_secs": "120", 2024-09-16T20:48:06.111 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_expose_bucket": "false", 2024-09-16T20:48:06.111 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_extended_http_attrs": "", 2024-09-16T20:48:06.111 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_filter": "none", 2024-09-16T20:48:06.111 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-16T20:48:06.112 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_frontends": "beast port=7480", 2024-09-16T20:48:06.112 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-16T20:48:06.112 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_gc_max_deferred": "50", 2024-09-16T20:48:06.112 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-16T20:48:06.112 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_gc_max_objs": "32", 2024-09-16T20:48:06.112 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-16T20:48:06.113 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-16T20:48:06.113 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-16T20:48:06.113 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-16T20:48:06.113 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_gc_processor_period": "3600", 2024-09-16T20:48:06.113 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-16T20:48:06.114 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-16T20:48:06.114 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-16T20:48:06.114 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-16T20:48:06.114 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_init_timeout": "300", 2024-09-16T20:48:06.114 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-16T20:48:06.115 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-16T20:48:06.115 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_kafka_connection_idle": "300", 2024-09-16T20:48:06.115 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-16T20:48:06.115 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_kafka_sleep_timeout": "10", 2024-09-16T20:48:06.115 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-16T20:48:06.115 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-09-16T20:48:06.116 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-16T20:48:06.116 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_keystone_admin_domain": "", 2024-09-16T20:48:06.116 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_keystone_admin_password": "", 2024-09-16T20:48:06.116 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_keystone_admin_password_path": "", 2024-09-16T20:48:06.116 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_keystone_admin_project": "", 2024-09-16T20:48:06.116 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_keystone_admin_tenant": "", 2024-09-16T20:48:06.117 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_keystone_admin_token": "", 2024-09-16T20:48:06.117 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_keystone_admin_token_path": "", 2024-09-16T20:48:06.117 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_keystone_admin_user": "", 2024-09-16T20:48:06.117 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_keystone_barbican_domain": "", 2024-09-16T20:48:06.117 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_keystone_barbican_password": "", 2024-09-16T20:48:06.118 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_keystone_barbican_project": "", 2024-09-16T20:48:06.118 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-16T20:48:06.118 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_keystone_barbican_user": "", 2024-09-16T20:48:06.118 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-16T20:48:06.118 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-16T20:48:06.118 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-16T20:48:06.119 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-16T20:48:06.119 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-16T20:48:06.119 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_keystone_url": "", 2024-09-16T20:48:06.119 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-16T20:48:06.119 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_lc_debug_interval": "-1", 2024-09-16T20:48:06.120 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_lc_lock_max_time": "90", 2024-09-16T20:48:06.120 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_lc_max_objs": "32", 2024-09-16T20:48:06.120 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_lc_max_rules": "1000", 2024-09-16T20:48:06.120 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_lc_max_worker": "3", 2024-09-16T20:48:06.120 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-16T20:48:06.120 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_lc_thread_delay": "0", 2024-09-16T20:48:06.121 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-16T20:48:06.121 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_ldap_dnattr": "uid", 2024-09-16T20:48:06.121 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-16T20:48:06.121 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_ldap_searchfilter": "", 2024-09-16T20:48:06.121 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-16T20:48:06.121 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_ldap_uri": "", 2024-09-16T20:48:06.122 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_lfuda_sync_frequency": "60", 2024-09-16T20:48:06.122 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-16T20:48:06.122 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-16T20:48:06.122 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-16T20:48:06.122 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_log_http_headers": "", 2024-09-16T20:48:06.122 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-16T20:48:06.123 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-16T20:48:06.123 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_log_object_name_utc": "false", 2024-09-16T20:48:06.123 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-09-16T20:48:06.123 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-09-16T20:48:06.123 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_max_attr_name_len": "0", 2024-09-16T20:48:06.124 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_max_attr_size": "0", 2024-09-16T20:48:06.124 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-16T20:48:06.124 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_max_chunk_size": "4194304", 2024-09-16T20:48:06.124 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-16T20:48:06.124 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-16T20:48:06.125 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-16T20:48:06.125 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_max_listing_results": "5000", 2024-09-16T20:48:06.125 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_max_notify_retries": "10", 2024-09-16T20:48:06.125 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-16T20:48:06.125 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_max_put_param_size": "1048576", 2024-09-16T20:48:06.125 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_max_put_size": "5368709120", 2024-09-16T20:48:06.126 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_max_slo_entries": "1000", 2024-09-16T20:48:06.126 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_md_log_max_shards": "64", 2024-09-16T20:48:06.126 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-16T20:48:06.126 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-16T20:48:06.126 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-16T20:48:06.127 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-16T20:48:06.127 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_mp_lock_max_time": "600", 2024-09-16T20:48:06.127 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-16T20:48:06.127 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-16T20:48:06.127 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-16T20:48:06.127 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-16T20:48:06.127 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-16T20:48:06.127 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-16T20:48:06.127 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-16T20:48:06.128 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-16T20:48:06.128 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_nfs_max_gc": "300", 2024-09-16T20:48:06.128 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-16T20:48:06.128 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-16T20:48:06.128 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-16T20:48:06.129 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-16T20:48:06.129 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-16T20:48:06.129 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-16T20:48:06.129 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-16T20:48:06.129 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_num_async_rados_threads": "32", 2024-09-16T20:48:06.129 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_num_control_oids": "8", 2024-09-16T20:48:06.129 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_numa_node": "-1", 2024-09-16T20:48:06.129 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-16T20:48:06.129 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-16T20:48:06.129 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_objexp_chunk_size": "100", 2024-09-16T20:48:06.129 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_objexp_gc_interval": "600", 2024-09-16T20:48:06.129 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-16T20:48:06.130 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-16T20:48:06.130 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-16T20:48:06.130 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_op_thread_timeout": "600", 2024-09-16T20:48:06.130 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_op_tracing": "false", 2024-09-16T20:48:06.130 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_opa_token": "", 2024-09-16T20:48:06.130 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_opa_url": "", 2024-09-16T20:48:06.130 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_opa_verify_ssl": "true", 2024-09-16T20:48:06.130 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-16T20:48:06.130 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-16T20:48:06.130 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_ops_log_rados": "false", 2024-09-16T20:48:06.130 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_ops_log_socket_path": "", 2024-09-16T20:48:06.130 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-16T20:48:06.130 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-16T20:48:06.130 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-16T20:48:06.131 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_period_push_interval": "2.000000", 2024-09-16T20:48:06.131 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-16T20:48:06.131 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-16T20:48:06.131 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-16T20:48:06.131 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-09-16T20:48:06.131 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_posix_cache_lanes": "3", 2024-09-16T20:48:06.131 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-09-16T20:48:06.131 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_posix_cache_max_buckets": "100", 2024-09-16T20:48:06.131 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_posix_cache_partitions": "3", 2024-09-16T20:48:06.131 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-09-16T20:48:06.131 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_print_continue": "true", 2024-09-16T20:48:06.131 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-16T20:48:06.132 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-16T20:48:06.132 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-16T20:48:06.132 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-16T20:48:06.132 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-16T20:48:06.132 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_rados_tracing": "false", 2024-09-16T20:48:06.132 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_realm": "", 2024-09-16T20:48:06.132 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_realm_id": "", 2024-09-16T20:48:06.132 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-16T20:48:06.132 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_region": "", 2024-09-16T20:48:06.132 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-16T20:48:06.132 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-16T20:48:06.132 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-16T20:48:06.132 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-16T20:48:06.133 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_request_uri": "", 2024-09-16T20:48:06.133 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_reshard_batch_size": "64", 2024-09-16T20:48:06.133 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-16T20:48:06.133 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_reshard_debug_interval": "-1", 2024-09-16T20:48:06.133 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_reshard_max_aio": "128", 2024-09-16T20:48:06.133 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_reshard_num_logs": "16", 2024-09-16T20:48:06.133 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_reshard_progress_judge_interval": "120", 2024-09-16T20:48:06.133 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_reshard_progress_judge_ratio": "0.500000", 2024-09-16T20:48:06.133 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_reshard_thread_interval": "600", 2024-09-16T20:48:06.133 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_reshardlog_threshold": "30000", 2024-09-16T20:48:06.133 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_resolve_cname": "false", 2024-09-16T20:48:06.133 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-16T20:48:06.134 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_run_sync_thread": "true", 2024-09-16T20:48:06.134 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-16T20:48:06.134 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-16T20:48:06.134 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-16T20:48:06.134 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-16T20:48:06.134 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-16T20:48:06.134 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-16T20:48:06.134 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-16T20:48:06.134 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-16T20:48:06.134 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-16T20:48:06.134 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_scheduler_type": "throttler", 2024-09-16T20:48:06.134 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_script_uri": "", 2024-09-16T20:48:06.134 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_service_provider_name": "", 2024-09-16T20:48:06.134 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-16T20:48:06.135 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_sts_client_id": "", 2024-09-16T20:48:06.135 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_sts_client_secret": "", 2024-09-16T20:48:06.135 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_sts_entry": "sts", 2024-09-16T20:48:06.135 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_sts_key": "sts", 2024-09-16T20:48:06.135 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-16T20:48:06.135 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_sts_min_session_duration": "900", 2024-09-16T20:48:06.135 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_sts_token_introspection_url": "", 2024-09-16T20:48:06.135 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_swift_account_in_url": "false", 2024-09-16T20:48:06.135 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_swift_auth_entry": "auth", 2024-09-16T20:48:06.135 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_swift_auth_url": "", 2024-09-16T20:48:06.135 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_swift_custom_header": "", 2024-09-16T20:48:06.135 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-16T20:48:06.136 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_swift_need_stats": "true", 2024-09-16T20:48:06.136 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_swift_tenant_name": "", 2024-09-16T20:48:06.136 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_swift_token_expiration": "86400", 2024-09-16T20:48:06.136 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_swift_url": "", 2024-09-16T20:48:06.136 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_swift_url_prefix": "swift", 2024-09-16T20:48:06.136 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-16T20:48:06.136 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-16T20:48:06.136 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-16T20:48:06.136 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_sync_lease_period": "120", 2024-09-16T20:48:06.136 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-16T20:48:06.137 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-16T20:48:06.137 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-16T20:48:06.137 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-16T20:48:06.137 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-16T20:48:06.137 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-16T20:48:06.137 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-16T20:48:06.137 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-16T20:48:06.137 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-16T20:48:06.137 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_thread_pool_size": "512", 2024-09-16T20:48:06.137 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_topic_persistency_max_retries": "0", 2024-09-16T20:48:06.137 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-09-16T20:48:06.137 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-09-16T20:48:06.138 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_topic_require_publish_policy": "false", 2024-09-16T20:48:06.138 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_torrent_comment": "", 2024-09-16T20:48:06.138 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_torrent_createby": "", 2024-09-16T20:48:06.138 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_torrent_encoding": "", 2024-09-16T20:48:06.138 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_torrent_flag": "false", 2024-09-16T20:48:06.138 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_torrent_max_size": "5368709120", 2024-09-16T20:48:06.138 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_torrent_origin": "", 2024-09-16T20:48:06.138 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-16T20:48:06.138 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_torrent_tracker": "", 2024-09-16T20:48:06.138 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_trust_forwarded_https": "false", 2024-09-16T20:48:06.138 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-16T20:48:06.139 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-16T20:48:06.139 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_usage_max_shards": "32", 2024-09-16T20:48:06.139 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_usage_max_user_shards": "1", 2024-09-16T20:48:06.139 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_use_opa_authz": "false", 2024-09-16T20:48:06.139 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_user_counters_cache": "false", 2024-09-16T20:48:06.139 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_user_counters_cache_size": "10000", 2024-09-16T20:48:06.139 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-16T20:48:06.139 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-16T20:48:06.139 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_user_max_buckets": "1000", 2024-09-16T20:48:06.139 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_user_policies_max_num": "100", 2024-09-16T20:48:06.140 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-16T20:48:06.140 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-16T20:48:06.140 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-16T20:48:06.140 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-16T20:48:06.140 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_user_unique_email": "true", 2024-09-16T20:48:06.140 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_verify_ssl": "true", 2024-09-16T20:48:06.140 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-16T20:48:06.140 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_zone": "", 2024-09-16T20:48:06.140 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_zone_id": "", 2024-09-16T20:48:06.140 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-16T20:48:06.140 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_zonegroup": "", 2024-09-16T20:48:06.140 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_zonegroup_id": "", 2024-09-16T20:48:06.141 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-16T20:48:06.141 INFO:teuthology.orchestra.run.smithi029.stdout: "rgwlc_auto_session_clear": "true", 2024-09-16T20:48:06.141 INFO:teuthology.orchestra.run.smithi029.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-16T20:48:06.141 INFO:teuthology.orchestra.run.smithi029.stdout: "rocksdb_block_size": "4096", 2024-09-16T20:48:06.141 INFO:teuthology.orchestra.run.smithi029.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-16T20:48:06.141 INFO:teuthology.orchestra.run.smithi029.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-16T20:48:06.141 INFO:teuthology.orchestra.run.smithi029.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-16T20:48:06.141 INFO:teuthology.orchestra.run.smithi029.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-16T20:48:06.141 INFO:teuthology.orchestra.run.smithi029.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-16T20:48:06.141 INFO:teuthology.orchestra.run.smithi029.stdout: "rocksdb_cache_size": "536870912", 2024-09-16T20:48:06.141 INFO:teuthology.orchestra.run.smithi029.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-16T20:48:06.141 INFO:teuthology.orchestra.run.smithi029.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-16T20:48:06.142 INFO:teuthology.orchestra.run.smithi029.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-16T20:48:06.142 INFO:teuthology.orchestra.run.smithi029.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-16T20:48:06.142 INFO:teuthology.orchestra.run.smithi029.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-16T20:48:06.142 INFO:teuthology.orchestra.run.smithi029.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-16T20:48:06.142 INFO:teuthology.orchestra.run.smithi029.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-16T20:48:06.142 INFO:teuthology.orchestra.run.smithi029.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-16T20:48:06.142 INFO:teuthology.orchestra.run.smithi029.stdout: "rocksdb_index_type": "binary_search", 2024-09-16T20:48:06.142 INFO:teuthology.orchestra.run.smithi029.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-16T20:48:06.142 INFO:teuthology.orchestra.run.smithi029.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-16T20:48:06.142 INFO:teuthology.orchestra.run.smithi029.stdout: "rocksdb_partition_filters": "false", 2024-09-16T20:48:06.142 INFO:teuthology.orchestra.run.smithi029.stdout: "rocksdb_perf": "false", 2024-09-16T20:48:06.142 INFO:teuthology.orchestra.run.smithi029.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-16T20:48:06.143 INFO:teuthology.orchestra.run.smithi029.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-16T20:48:06.143 INFO:teuthology.orchestra.run.smithi029.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-16T20:48:06.143 INFO:teuthology.orchestra.run.smithi029.stdout: "run_dir": "/var/run/ceph", 2024-09-16T20:48:06.143 INFO:teuthology.orchestra.run.smithi029.stdout: "seastore_block_create": "true", 2024-09-16T20:48:06.143 INFO:teuthology.orchestra.run.smithi029.stdout: "seastore_cache_lru_size": "67108864", 2024-09-16T20:48:06.143 INFO:teuthology.orchestra.run.smithi029.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-16T20:48:06.143 INFO:teuthology.orchestra.run.smithi029.stdout: "seastore_data_delta_based_overwrite": "0", 2024-09-16T20:48:06.143 INFO:teuthology.orchestra.run.smithi029.stdout: "seastore_default_max_object_size": "16777216", 2024-09-16T20:48:06.143 INFO:teuthology.orchestra.run.smithi029.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-16T20:48:06.143 INFO:teuthology.orchestra.run.smithi029.stdout: "seastore_device_size": "53687091200", 2024-09-16T20:48:06.143 INFO:teuthology.orchestra.run.smithi029.stdout: "seastore_disable_end_to_end_data_protection": "true", 2024-09-16T20:48:06.143 INFO:teuthology.orchestra.run.smithi029.stdout: "seastore_full_integrity_check": "false", 2024-09-16T20:48:06.143 INFO:teuthology.orchestra.run.smithi029.stdout: "seastore_journal_batch_capacity": "16", 2024-09-16T20:48:06.144 INFO:teuthology.orchestra.run.smithi029.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-16T20:48:06.144 INFO:teuthology.orchestra.run.smithi029.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-16T20:48:06.144 INFO:teuthology.orchestra.run.smithi029.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-16T20:48:06.144 INFO:teuthology.orchestra.run.smithi029.stdout: "seastore_main_device_type": "SSD", 2024-09-16T20:48:06.144 INFO:teuthology.orchestra.run.smithi029.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-16T20:48:06.144 INFO:teuthology.orchestra.run.smithi029.stdout: "seastore_max_data_allocation_size": "32768", 2024-09-16T20:48:06.144 INFO:teuthology.orchestra.run.smithi029.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-16T20:48:06.144 INFO:teuthology.orchestra.run.smithi029.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-16T20:48:06.144 INFO:teuthology.orchestra.run.smithi029.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-16T20:48:06.144 INFO:teuthology.orchestra.run.smithi029.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-16T20:48:06.145 INFO:teuthology.orchestra.run.smithi029.stdout: "seastore_segment_size": "67108864", 2024-09-16T20:48:06.145 INFO:teuthology.orchestra.run.smithi029.stdout: "set_keepcaps": "false", 2024-09-16T20:48:06.145 INFO:teuthology.orchestra.run.smithi029.stdout: "setgroup": "", 2024-09-16T20:48:06.145 INFO:teuthology.orchestra.run.smithi029.stdout: "setuser": "", 2024-09-16T20:48:06.145 INFO:teuthology.orchestra.run.smithi029.stdout: "setuser_match_path": "", 2024-09-16T20:48:06.145 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-16T20:48:06.145 INFO:teuthology.orchestra.run.smithi029.stdout: "thp": "false", 2024-09-16T20:48:06.145 INFO:teuthology.orchestra.run.smithi029.stdout: "threadpool_default_timeout": "60", 2024-09-16T20:48:06.145 INFO:teuthology.orchestra.run.smithi029.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-16T20:48:06.145 INFO:teuthology.orchestra.run.smithi029.stdout: "throttler_perf_counter": "true", 2024-09-16T20:48:06.145 INFO:teuthology.orchestra.run.smithi029.stdout: "tmp_dir": "/tmp", 2024-09-16T20:48:06.145 INFO:teuthology.orchestra.run.smithi029.stdout: "tmp_file_template": "/tmp/ceph-mon.a.XXXXXX", 2024-09-16T20:48:06.146 INFO:teuthology.orchestra.run.smithi029.stdout: "uadk_compressor_enabled": "false", 2024-09-16T20:48:06.146 INFO:teuthology.orchestra.run.smithi029.stdout: "uadk_wd_sync_ctx_num": "2" 2024-09-16T20:48:06.146 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:06.148 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-16T20:48:06.265 INFO:teuthology.orchestra.run.smithi029.stdout:mon.a: {} 2024-09-16T20:48:06.265 INFO:teuthology.orchestra.run.smithi029.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-16T20:48:06.275 INFO:teuthology.orchestra.run.smithi029.stdout:mon.b: {} 2024-09-16T20:48:06.275 INFO:teuthology.orchestra.run.smithi029.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-16T20:48:06.284 INFO:teuthology.orchestra.run.smithi029.stdout:mon.c: {} 2024-09-16T20:48:06.284 INFO:teuthology.orchestra.run.smithi029.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-16T20:48:06.294 DEBUG:teuthology.orchestra.run.smithi029:> type ceph-objectstore-tool 2024-09-16T20:48:06.309 INFO:teuthology.orchestra.run.smithi029.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-16T20:48:06.309 DEBUG:teuthology.orchestra.run.smithi043:> type ceph-objectstore-tool 2024-09-16T20:48:06.327 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-16T20:48:06.328 INFO:teuthology.run_tasks:Running task rbd_fsx... 2024-09-16T20:48:06.339 INFO:tasks.rbd_fsx:starting rbd_fsx... 2024-09-16T20:48:06.339 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-16T20:48:06.339 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-16T20:48:06.339 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T20:48:06.339 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-16T20:48:06.339 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-16T20:48:06.340 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-16T20:48:06.340 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-16T20:48:06.340 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T20:48:06.341 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-16T20:48:06.341 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:48:06.341 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-16T20:48:06.341 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T20:48:06.342 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 512 -w 512 -t 512 -h 512 -l 250000000 -S 0 -N 6000 -M pool_client.0 image_client.0 2024-09-16T20:48:06.352 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:06.349+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:06.401 INFO:teuthology.orchestra.run.smithi139.stdout:mapped writes DISABLED 2024-09-16T20:48:06.401 INFO:teuthology.orchestra.run.smithi139.stdout:mapped reads DISABLED 2024-09-16T20:48:06.401 INFO:teuthology.orchestra.run.smithi139.stdout:Seed set to 5312 2024-09-16T20:48:06.401 INFO:teuthology.orchestra.run.smithi139.stdout:rbd-nbd mode enabled 2024-09-16T20:48:06.442 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:06.440+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:06.469 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:06.469 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:48:06.469 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:48:06.469 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:06.481 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:48:06.487 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:48:06.487 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:48:06.502 INFO:teuthology.orchestra.run.smithi029.stderr:no valid command found; 10 closest matches: 2024-09-16T20:48:06.502 INFO:teuthology.orchestra.run.smithi029.stderr:0 2024-09-16T20:48:06.502 INFO:teuthology.orchestra.run.smithi029.stderr:1 2024-09-16T20:48:06.502 INFO:teuthology.orchestra.run.smithi029.stderr:2 2024-09-16T20:48:06.502 INFO:teuthology.orchestra.run.smithi029.stderr:abort 2024-09-16T20:48:06.502 INFO:teuthology.orchestra.run.smithi029.stderr:assert 2024-09-16T20:48:06.502 INFO:teuthology.orchestra.run.smithi029.stderr:bench [] [] [] [] 2024-09-16T20:48:06.502 INFO:teuthology.orchestra.run.smithi029.stderr:bluefs debug_inject_read_zeros 2024-09-16T20:48:06.503 INFO:teuthology.orchestra.run.smithi029.stderr:bluefs files list 2024-09-16T20:48:06.503 INFO:teuthology.orchestra.run.smithi029.stderr:bluefs stats 2024-09-16T20:48:06.503 INFO:teuthology.orchestra.run.smithi029.stderr:bluestore allocator dump block 2024-09-16T20:48:06.503 INFO:teuthology.orchestra.run.smithi029.stderr:Error EINVAL: invalid command 2024-09-16T20:48:06.506 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T20:48:06.507 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-16T20:48:06.507 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-16T20:48:06.507 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:48:06.508 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:48:06.523 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:48:06.524 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:48:06.541 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:48:06.541 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:48:06.542 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:06.541+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:06.569 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:48:06.570 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:48:06.598 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:06.598 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:06.603 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:48:06.604 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:48:06.608 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:48:06.643 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:06.642+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:06.668 INFO:teuthology.orchestra.run.smithi029.stderr:no valid command found; 10 closest matches: 2024-09-16T20:48:06.668 INFO:teuthology.orchestra.run.smithi029.stderr:0 2024-09-16T20:48:06.668 INFO:teuthology.orchestra.run.smithi029.stderr:1 2024-09-16T20:48:06.668 INFO:teuthology.orchestra.run.smithi029.stderr:2 2024-09-16T20:48:06.668 INFO:teuthology.orchestra.run.smithi029.stderr:abort 2024-09-16T20:48:06.668 INFO:teuthology.orchestra.run.smithi029.stderr:assert 2024-09-16T20:48:06.668 INFO:teuthology.orchestra.run.smithi029.stderr:bench [] [] [] [] 2024-09-16T20:48:06.668 INFO:teuthology.orchestra.run.smithi029.stderr:bluefs debug_inject_read_zeros 2024-09-16T20:48:06.668 INFO:teuthology.orchestra.run.smithi029.stderr:bluefs files list 2024-09-16T20:48:06.669 INFO:teuthology.orchestra.run.smithi029.stderr:bluefs stats 2024-09-16T20:48:06.669 INFO:teuthology.orchestra.run.smithi029.stderr:bluestore allocator dump block 2024-09-16T20:48:06.669 INFO:teuthology.orchestra.run.smithi029.stderr:Error EINVAL: invalid command 2024-09-16T20:48:06.671 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T20:48:06.671 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-16T20:48:06.671 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-16T20:48:06.755 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:06.754+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:06.768 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:06.768 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:06.777 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:48:06.845 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:06.843+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:06.847 INFO:teuthology.orchestra.run.smithi029.stderr:no valid command found; 10 closest matches: 2024-09-16T20:48:06.847 INFO:teuthology.orchestra.run.smithi029.stderr:0 2024-09-16T20:48:06.847 INFO:teuthology.orchestra.run.smithi029.stderr:1 2024-09-16T20:48:06.847 INFO:teuthology.orchestra.run.smithi029.stderr:2 2024-09-16T20:48:06.847 INFO:teuthology.orchestra.run.smithi029.stderr:abort 2024-09-16T20:48:06.847 INFO:teuthology.orchestra.run.smithi029.stderr:assert 2024-09-16T20:48:06.847 INFO:teuthology.orchestra.run.smithi029.stderr:bench [] [] [] [] 2024-09-16T20:48:06.848 INFO:teuthology.orchestra.run.smithi029.stderr:bluefs debug_inject_read_zeros 2024-09-16T20:48:06.848 INFO:teuthology.orchestra.run.smithi029.stderr:bluefs files list 2024-09-16T20:48:06.848 INFO:teuthology.orchestra.run.smithi029.stderr:bluefs stats 2024-09-16T20:48:06.848 INFO:teuthology.orchestra.run.smithi029.stderr:bluestore allocator dump block 2024-09-16T20:48:06.848 INFO:teuthology.orchestra.run.smithi029.stderr:Error EINVAL: invalid command 2024-09-16T20:48:06.850 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T20:48:06.851 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-16T20:48:06.851 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-16T20:48:06.926 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:06.926 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:06.935 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:48:06.976 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:06.975+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:07.028 INFO:teuthology.orchestra.run.smithi029.stderr:no valid command found; 10 closest matches: 2024-09-16T20:48:07.029 INFO:teuthology.orchestra.run.smithi029.stderr:0 2024-09-16T20:48:07.029 INFO:teuthology.orchestra.run.smithi029.stderr:1 2024-09-16T20:48:07.029 INFO:teuthology.orchestra.run.smithi029.stderr:2 2024-09-16T20:48:07.029 INFO:teuthology.orchestra.run.smithi029.stderr:abort 2024-09-16T20:48:07.029 INFO:teuthology.orchestra.run.smithi029.stderr:assert 2024-09-16T20:48:07.029 INFO:teuthology.orchestra.run.smithi029.stderr:bench [] [] [] [] 2024-09-16T20:48:07.029 INFO:teuthology.orchestra.run.smithi029.stderr:bluefs debug_inject_read_zeros 2024-09-16T20:48:07.029 INFO:teuthology.orchestra.run.smithi029.stderr:bluefs files list 2024-09-16T20:48:07.029 INFO:teuthology.orchestra.run.smithi029.stderr:bluefs stats 2024-09-16T20:48:07.029 INFO:teuthology.orchestra.run.smithi029.stderr:bluestore allocator dump block 2024-09-16T20:48:07.029 INFO:teuthology.orchestra.run.smithi029.stderr:Error EINVAL: invalid command 2024-09-16T20:48:07.031 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T20:48:07.032 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-16T20:48:07.032 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 'trim stale osdmaps' 2024-09-16T20:48:07.046 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:07.045+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:07.084 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:07.084 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:07.093 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:48:07.147 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:07.146+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:07.168 INFO:teuthology.orchestra.run.smithi029.stderr:no valid command found; 10 closest matches: 2024-09-16T20:48:07.168 INFO:teuthology.orchestra.run.smithi029.stderr:0 2024-09-16T20:48:07.168 INFO:teuthology.orchestra.run.smithi029.stderr:1 2024-09-16T20:48:07.168 INFO:teuthology.orchestra.run.smithi029.stderr:2 2024-09-16T20:48:07.169 INFO:teuthology.orchestra.run.smithi029.stderr:abort 2024-09-16T20:48:07.169 INFO:teuthology.orchestra.run.smithi029.stderr:assert 2024-09-16T20:48:07.169 INFO:teuthology.orchestra.run.smithi029.stderr:bench [] [] [] [] 2024-09-16T20:48:07.169 INFO:teuthology.orchestra.run.smithi029.stderr:bluefs debug_inject_read_zeros 2024-09-16T20:48:07.169 INFO:teuthology.orchestra.run.smithi029.stderr:bluefs files list 2024-09-16T20:48:07.169 INFO:teuthology.orchestra.run.smithi029.stderr:bluefs stats 2024-09-16T20:48:07.169 INFO:teuthology.orchestra.run.smithi029.stderr:bluestore allocator dump block 2024-09-16T20:48:07.169 INFO:teuthology.orchestra.run.smithi029.stderr:Error EINVAL: invalid command 2024-09-16T20:48:07.171 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T20:48:07.172 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-16T20:48:07.172 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 'trim stale osdmaps' 2024-09-16T20:48:07.202 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:07.202 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:07.212 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:48:07.253 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:07.252+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:07.338 INFO:teuthology.orchestra.run.smithi029.stderr:no valid command found; 10 closest matches: 2024-09-16T20:48:07.338 INFO:teuthology.orchestra.run.smithi029.stderr:0 2024-09-16T20:48:07.338 INFO:teuthology.orchestra.run.smithi029.stderr:1 2024-09-16T20:48:07.338 INFO:teuthology.orchestra.run.smithi029.stderr:2 2024-09-16T20:48:07.338 INFO:teuthology.orchestra.run.smithi029.stderr:abort 2024-09-16T20:48:07.338 INFO:teuthology.orchestra.run.smithi029.stderr:assert 2024-09-16T20:48:07.338 INFO:teuthology.orchestra.run.smithi029.stderr:bench [] [] [] [] 2024-09-16T20:48:07.338 INFO:teuthology.orchestra.run.smithi029.stderr:bluefs debug_inject_read_zeros 2024-09-16T20:48:07.338 INFO:teuthology.orchestra.run.smithi029.stderr:bluefs files list 2024-09-16T20:48:07.339 INFO:teuthology.orchestra.run.smithi029.stderr:bluefs stats 2024-09-16T20:48:07.339 INFO:teuthology.orchestra.run.smithi029.stderr:bluestore allocator dump block 2024-09-16T20:48:07.339 INFO:teuthology.orchestra.run.smithi029.stderr:Error EINVAL: invalid command 2024-09-16T20:48:07.341 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T20:48:07.341 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-16T20:48:07.349 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:07.347+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:07.362 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:07.362 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:07.371 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:48:07.449 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:07.448+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:07.517 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:07.518 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:07.526 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:48:07.553 INFO:teuthology.orchestra.run.smithi029.stderr:noscrub is set 2024-09-16T20:48:07.554 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:07.553+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:07.642 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:07.642 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:07.651 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:48:07.651 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:07.649+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:07.751 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:07.750+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:07.771 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:07.771 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:07.780 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:48:07.852 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:07.850+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:07.888 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:07.888 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:07.897 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:48:07.952 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:07.951+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:08.004 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:08.004 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:08.012 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:48:08.054 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:08.052+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:08.119 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:08.119 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:08.128 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:48:08.155 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:08.154+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:08.234 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:08.235 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:08.243 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:48:08.256 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:08.255+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:08.349 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:08.349 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:08.358 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:48:08.358 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:08.356+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:08.459 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:08.458+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:08.465 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:08.465 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:08.473 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:48:08.560 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:08.559+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:08.581 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:08.581 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:08.590 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:48:08.621 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T20:48:08.661 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:08.660+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:08.697 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:08.697 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:08.706 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:48:08.754 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:48:08.754 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:48:08.762 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:08.761+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:08.772 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:48:08.772 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:48:08.789 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:48:08.789 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:48:08.806 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:48:08.806 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:48:08.817 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:08.817 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:48:08.817 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:48:08.817 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:08.823 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:48:08.823 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:48:08.826 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:48:08.840 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:48:08.840 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:48:08.863 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:08.862+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:08.947 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:08.947 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:48:08.947 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:48:08.947 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:48:08.947 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:08.956 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:48:09.000 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:08.999+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:09.064 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:09.064 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:48:09.064 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:48:09.064 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:48:09.064 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.064 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4245.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e19)", 2024-09-16T20:48:09.065 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:58.278443+0000", 2024-09-16T20:48:09.065 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 10.784752774999999, 2024-09-16T20:48:09.065 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00020536, 2024-09-16T20:48:09.065 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:09.065 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:09.065 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:48:09.065 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:09.065 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4245", 2024-09-16T20:48:09.065 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.29:0/1819464193", 2024-09-16T20:48:09.065 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2 2024-09-16T20:48:09.065 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.066 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:09.066 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.066 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:09.066 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:58.278443+0000", 2024-09-16T20:48:09.066 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:09.066 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.066 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.066 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:09.066 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:58.278443+0000", 2024-09-16T20:48:09.066 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:09.066 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.066 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.066 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:09.066 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:58.278452+0000", 2024-09-16T20:48:09.067 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9870000000000005e-06 2024-09-16T20:48:09.067 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.067 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.067 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:09.067 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:58.278458+0000", 2024-09-16T20:48:09.067 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7830000000000001e-06 2024-09-16T20:48:09.067 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.067 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.067 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:09.067 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:58.278471+0000", 2024-09-16T20:48:09.067 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2225000000000001e-05 2024-09-16T20:48:09.067 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.067 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.067 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:09.067 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:58.278479+0000", 2024-09-16T20:48:09.068 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1880000000000002e-06 2024-09-16T20:48:09.068 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.068 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.068 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:09.068 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:58.278536+0000", 2024-09-16T20:48:09.068 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7156e-05 2024-09-16T20:48:09.068 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.068 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.068 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:09.068 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:58.278648+0000", 2024-09-16T20:48:09.068 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000112021 2024-09-16T20:48:09.068 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:09.069 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:09.069 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:09.069 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.069 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.069 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4245.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e19)", 2024-09-16T20:48:09.069 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:58.278796+0000", 2024-09-16T20:48:09.069 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 10.784399145, 2024-09-16T20:48:09.069 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00012257699999999999, 2024-09-16T20:48:09.069 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:09.069 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:09.069 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:48:09.069 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:09.069 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4245", 2024-09-16T20:48:09.069 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.29:0/1819464193", 2024-09-16T20:48:09.070 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3 2024-09-16T20:48:09.070 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.070 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:09.070 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.070 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:09.070 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:58.278796+0000", 2024-09-16T20:48:09.070 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:09.070 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.070 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.070 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:09.070 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:58.278796+0000", 2024-09-16T20:48:09.070 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:09.070 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.071 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.071 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:09.071 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:58.278797+0000", 2024-09-16T20:48:09.071 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0640000000000001e-06 2024-09-16T20:48:09.071 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.071 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.071 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:09.071 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:58.278802+0000", 2024-09-16T20:48:09.071 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5750000000000002e-06 2024-09-16T20:48:09.071 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.071 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.071 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:09.071 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:58.278814+0000", 2024-09-16T20:48:09.072 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1712999999999999e-05 2024-09-16T20:48:09.072 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.072 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.072 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:09.072 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:58.278819+0000", 2024-09-16T20:48:09.072 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5809999999999996e-06 2024-09-16T20:48:09.072 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.072 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.072 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:09.072 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:58.278855+0000", 2024-09-16T20:48:09.072 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5485999999999999e-05 2024-09-16T20:48:09.072 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.073 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.073 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:09.073 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:58.278919+0000", 2024-09-16T20:48:09.073 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4158000000000005e-05 2024-09-16T20:48:09.073 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:09.073 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:09.073 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:09.073 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.073 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.073 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4245.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-16T20:48:09.073 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:59.273312+0000", 2024-09-16T20:48:09.073 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 9.7898832999999996, 2024-09-16T20:48:09.073 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0036239010000000001, 2024-09-16T20:48:09.074 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:09.074 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:09.074 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:09.074 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:09.074 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4245", 2024-09-16T20:48:09.074 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.29:0/1819464193", 2024-09-16T20:48:09.074 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5 2024-09-16T20:48:09.074 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.074 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:09.074 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.074 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:09.074 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:59.273312+0000", 2024-09-16T20:48:09.074 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:09.075 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.075 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.075 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:09.075 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:59.273312+0000", 2024-09-16T20:48:09.075 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:09.075 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.075 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.075 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:09.075 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:59.273315+0000", 2024-09-16T20:48:09.075 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.4229999999999999e-06 2024-09-16T20:48:09.075 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.075 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.075 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:09.076 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:59.273328+0000", 2024-09-16T20:48:09.076 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3689999999999999e-05 2024-09-16T20:48:09.076 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.076 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.076 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:09.076 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:59.273350+0000", 2024-09-16T20:48:09.076 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.145e-05 2024-09-16T20:48:09.076 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.076 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.076 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:09.076 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:59.273364+0000", 2024-09-16T20:48:09.076 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.432e-05 2024-09-16T20:48:09.076 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.076 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.077 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:09.077 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:59.273439+0000", 2024-09-16T20:48:09.077 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5098999999999994e-05 2024-09-16T20:48:09.077 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.077 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.077 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op must wait for map", 2024-09-16T20:48:09.077 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:59.273444+0000", 2024-09-16T20:48:09.077 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.22e-06 2024-09-16T20:48:09.077 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.077 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.077 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:09.077 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:59.275086+0000", 2024-09-16T20:48:09.077 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0016413669999999999 2024-09-16T20:48:09.077 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.078 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.078 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:09.078 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:59.275213+0000", 2024-09-16T20:48:09.078 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00012698700000000001 2024-09-16T20:48:09.078 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.078 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.078 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 4", 2024-09-16T20:48:09.078 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:59.275341+0000", 2024-09-16T20:48:09.078 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00012876200000000001 2024-09-16T20:48:09.078 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.078 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.078 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:09.078 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:59.276119+0000", 2024-09-16T20:48:09.078 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00077722800000000003 2024-09-16T20:48:09.079 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.079 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.079 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:09.079 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:59.276868+0000", 2024-09-16T20:48:09.079 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00074979199999999999 2024-09-16T20:48:09.079 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.079 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.079 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:09.079 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:59.276906+0000", 2024-09-16T20:48:09.079 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7279e-05 2024-09-16T20:48:09.079 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.079 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.079 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:09.080 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:59.276936+0000", 2024-09-16T20:48:09.080 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0284e-05 2024-09-16T20:48:09.080 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:09.080 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:09.080 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:09.080 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.080 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.080 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4245.0:5 2.2 e20/18 ondisk, result = 0)", 2024-09-16T20:48:09.080 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:59.276793+0000", 2024-09-16T20:48:09.080 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 9.7864023709999994, 2024-09-16T20:48:09.080 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000150786, 2024-09-16T20:48:09.080 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:09.080 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:09.081 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:09.081 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:09.081 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.081 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:09.081 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:59.276793+0000", 2024-09-16T20:48:09.081 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:09.081 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.081 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.081 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:09.081 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:59.276793+0000", 2024-09-16T20:48:09.081 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:09.081 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.081 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.081 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:09.081 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:59.276794+0000", 2024-09-16T20:48:09.082 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3230000000000001e-06 2024-09-16T20:48:09.082 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.082 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.082 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:09.082 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:59.276800+0000", 2024-09-16T20:48:09.082 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6550000000000001e-06 2024-09-16T20:48:09.082 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.082 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.082 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:09.082 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:59.276814+0000", 2024-09-16T20:48:09.082 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3777e-05 2024-09-16T20:48:09.082 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.082 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.082 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:09.082 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:59.276822+0000", 2024-09-16T20:48:09.083 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9039999999999992e-06 2024-09-16T20:48:09.083 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.083 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.083 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:09.083 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:59.276860+0000", 2024-09-16T20:48:09.083 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8118000000000003e-05 2024-09-16T20:48:09.083 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.083 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.083 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:09.083 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:59.276865+0000", 2024-09-16T20:48:09.083 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3179999999999997e-06 2024-09-16T20:48:09.083 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.083 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.083 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:09.083 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:59.276944+0000", 2024-09-16T20:48:09.083 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8690999999999996e-05 2024-09-16T20:48:09.084 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:09.084 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:09.084 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:09.084 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.084 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.084 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4245.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-16T20:48:09.084 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:00.277778+0000", 2024-09-16T20:48:09.084 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 8.785417013, 2024-09-16T20:48:09.084 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0017304390000000001, 2024-09-16T20:48:09.084 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:09.084 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:09.084 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:09.084 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:09.084 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4245", 2024-09-16T20:48:09.085 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.29:0/1819464193", 2024-09-16T20:48:09.085 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 7 2024-09-16T20:48:09.085 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.085 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:09.085 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.085 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:09.085 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:00.277778+0000", 2024-09-16T20:48:09.085 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:09.085 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.085 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.085 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:09.085 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:00.277778+0000", 2024-09-16T20:48:09.085 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:09.085 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.086 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.086 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:09.086 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:00.277781+0000", 2024-09-16T20:48:09.086 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3479999999999998e-06 2024-09-16T20:48:09.086 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.086 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.086 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:09.086 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:00.277793+0000", 2024-09-16T20:48:09.086 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.239e-05 2024-09-16T20:48:09.086 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.086 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.086 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:09.086 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:00.277810+0000", 2024-09-16T20:48:09.086 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.6402e-05 2024-09-16T20:48:09.087 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.087 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.087 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:09.087 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:00.277822+0000", 2024-09-16T20:48:09.087 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2682e-05 2024-09-16T20:48:09.087 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.087 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.087 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:09.087 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:00.277896+0000", 2024-09-16T20:48:09.087 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3289999999999998e-05 2024-09-16T20:48:09.087 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.087 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.087 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:09.088 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:00.278094+0000", 2024-09-16T20:48:09.088 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019882499999999999 2024-09-16T20:48:09.088 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.088 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.088 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 4", 2024-09-16T20:48:09.088 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:00.278317+0000", 2024-09-16T20:48:09.088 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00022229300000000001 2024-09-16T20:48:09.088 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.088 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.088 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:09.088 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:00.279087+0000", 2024-09-16T20:48:09.088 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00077042300000000001 2024-09-16T20:48:09.088 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.089 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.089 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:09.089 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:00.279435+0000", 2024-09-16T20:48:09.089 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00034811400000000001 2024-09-16T20:48:09.089 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.089 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.089 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:09.089 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:00.279475+0000", 2024-09-16T20:48:09.089 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9981999999999998e-05 2024-09-16T20:48:09.089 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.089 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.089 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:09.089 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:00.279509+0000", 2024-09-16T20:48:09.089 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3689999999999998e-05 2024-09-16T20:48:09.090 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:09.090 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:09.090 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:09.090 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.090 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.090 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4245.0:7 2.2 e21/18 ondisk, result = 0)", 2024-09-16T20:48:09.090 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:00.279346+0000", 2024-09-16T20:48:09.090 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 8.7838494179999991, 2024-09-16T20:48:09.090 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00016808, 2024-09-16T20:48:09.090 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:09.090 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:09.090 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:09.090 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:09.090 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.091 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:09.091 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:00.279346+0000", 2024-09-16T20:48:09.091 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:09.091 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.091 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.091 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:09.091 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:00.279346+0000", 2024-09-16T20:48:09.091 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:09.091 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.091 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.091 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:09.091 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:00.279347+0000", 2024-09-16T20:48:09.091 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4200000000000004e-07 2024-09-16T20:48:09.091 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.092 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.092 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:09.092 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:00.279356+0000", 2024-09-16T20:48:09.092 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.7700000000000007e-06 2024-09-16T20:48:09.092 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.092 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.092 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:09.092 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:00.279369+0000", 2024-09-16T20:48:09.092 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3033e-05 2024-09-16T20:48:09.092 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.092 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.092 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:09.092 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:00.279380+0000", 2024-09-16T20:48:09.092 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1138000000000001e-05 2024-09-16T20:48:09.093 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.093 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.093 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:09.093 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:00.279425+0000", 2024-09-16T20:48:09.093 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.494e-05 2024-09-16T20:48:09.093 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.093 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.093 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:09.093 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:00.279431+0000", 2024-09-16T20:48:09.093 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2199999999999997e-06 2024-09-16T20:48:09.093 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.093 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.093 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:09.093 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:00.279514+0000", 2024-09-16T20:48:09.094 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.3036999999999998e-05 2024-09-16T20:48:09.094 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:09.094 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:09.094 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:09.094 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:09.094 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:09.094 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:09.094 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:09.063+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:09.095 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:48:09.179 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:09.178+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:09.242 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:09.243 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:48:09.243 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:48:09.243 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:09.251 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:48:09.265 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:09.264+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:09.366 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:09.365+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:09.398 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:09.398 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:48:09.398 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:48:09.398 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:48:09.398 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:09.407 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:48:09.467 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:09.465+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:09.516 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:09.516 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:48:09.516 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:48:09.516 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:48:09.516 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.516 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:48:09.517 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:48:09.517 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 16.259439263000001, 2024-09-16T20:48:09.517 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67502346999999996, 2024-09-16T20:48:09.517 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:09.517 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:09.517 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:09.517 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:09.517 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:48:09.517 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:48:09.517 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 1 2024-09-16T20:48:09.517 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.517 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:09.517 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.517 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:09.518 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:48:09.518 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:09.518 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.518 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.518 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:09.518 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:48:09.518 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:09.518 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.518 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.518 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:09.518 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254128+0000", 2024-09-16T20:48:09.519 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.463e-06 2024-09-16T20:48:09.519 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.519 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.519 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:09.519 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254135+0000", 2024-09-16T20:48:09.519 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1579999999999999e-06 2024-09-16T20:48:09.519 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.519 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.519 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:09.519 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254146+0000", 2024-09-16T20:48:09.519 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1341e-05 2024-09-16T20:48:09.520 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.520 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.520 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:09.520 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254155+0000", 2024-09-16T20:48:09.520 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.1549999999999996e-06 2024-09-16T20:48:09.520 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.520 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.520 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:09.520 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254221+0000", 2024-09-16T20:48:09.520 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5470999999999997e-05 2024-09-16T20:48:09.520 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.521 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.521 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:48:09.521 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254231+0000", 2024-09-16T20:48:09.521 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0592e-05 2024-09-16T20:48:09.521 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.521 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.521 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:09.521 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926711+0000", 2024-09-16T20:48:09.521 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67247946599999997 2024-09-16T20:48:09.521 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.521 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.522 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:09.522 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926910+0000", 2024-09-16T20:48:09.522 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019934 2024-09-16T20:48:09.522 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.522 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.522 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:48:09.522 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.927159+0000", 2024-09-16T20:48:09.522 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00024876599999999998 2024-09-16T20:48:09.522 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.522 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.522 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:09.522 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.928797+0000", 2024-09-16T20:48:09.522 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001638308 2024-09-16T20:48:09.522 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.523 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.523 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:09.523 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929026+0000", 2024-09-16T20:48:09.523 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00022876099999999999 2024-09-16T20:48:09.523 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.523 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.523 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:09.523 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929102+0000", 2024-09-16T20:48:09.523 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6184000000000001e-05 2024-09-16T20:48:09.523 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.523 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.523 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:09.523 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929150+0000", 2024-09-16T20:48:09.523 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7465e-05 2024-09-16T20:48:09.524 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:09.524 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:09.524 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:09.524 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.524 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.524 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:25 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:48:09.524 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:53.949358+0000", 2024-09-16T20:48:09.524 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.564206929999999, 2024-09-16T20:48:09.524 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0036872879999999999, 2024-09-16T20:48:09.524 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:09.524 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:09.524 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:09.524 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:09.525 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:48:09.525 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:48:09.525 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 25 2024-09-16T20:48:09.525 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.525 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:09.525 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.525 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:09.525 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.949358+0000", 2024-09-16T20:48:09.525 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:09.525 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.525 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.525 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:09.525 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.949358+0000", 2024-09-16T20:48:09.525 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:09.526 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.526 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.526 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:09.526 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.949359+0000", 2024-09-16T20:48:09.526 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7700000000000004e-07 2024-09-16T20:48:09.526 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.526 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.526 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:09.526 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.949365+0000", 2024-09-16T20:48:09.526 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2669999999999997e-06 2024-09-16T20:48:09.526 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.526 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.526 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:09.526 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.949375+0000", 2024-09-16T20:48:09.527 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0188e-05 2024-09-16T20:48:09.527 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.527 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.527 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:09.527 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.949381+0000", 2024-09-16T20:48:09.527 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.105e-06 2024-09-16T20:48:09.527 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.527 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.527 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:09.527 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.952075+0000", 2024-09-16T20:48:09.527 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.002694072 2024-09-16T20:48:09.527 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.527 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.527 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:09.527 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.952133+0000", 2024-09-16T20:48:09.527 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8393000000000001e-05 2024-09-16T20:48:09.528 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.528 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.528 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:48:09.528 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.952214+0000", 2024-09-16T20:48:09.528 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0329999999999998e-05 2024-09-16T20:48:09.528 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.528 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.528 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:09.528 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.952635+0000", 2024-09-16T20:48:09.528 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000421703 2024-09-16T20:48:09.528 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.528 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.528 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:09.528 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.952996+0000", 2024-09-16T20:48:09.529 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00036085199999999999 2024-09-16T20:48:09.529 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.529 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.529 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:09.529 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.953026+0000", 2024-09-16T20:48:09.529 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9583000000000001e-05 2024-09-16T20:48:09.529 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.529 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.529 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:09.529 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.953046+0000", 2024-09-16T20:48:09.529 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0018e-05 2024-09-16T20:48:09.529 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:09.529 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:09.529 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:09.530 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.530 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.530 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:43 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:48:09.530 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:54.058718+0000", 2024-09-16T20:48:09.530 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.4548475, 2024-09-16T20:48:09.530 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0039347039999999998, 2024-09-16T20:48:09.530 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:09.530 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:09.530 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:09.530 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:09.530 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:48:09.530 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:48:09.531 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 43 2024-09-16T20:48:09.531 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.531 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:09.531 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.531 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:09.531 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058718+0000", 2024-09-16T20:48:09.531 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:09.531 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.531 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.531 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:09.532 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058718+0000", 2024-09-16T20:48:09.532 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:09.532 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.532 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.532 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:09.532 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058718+0000", 2024-09-16T20:48:09.532 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9799999999999999e-07 2024-09-16T20:48:09.532 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.532 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.532 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:09.532 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058720+0000", 2024-09-16T20:48:09.532 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.086e-06 2024-09-16T20:48:09.533 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.533 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.533 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:09.533 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058724+0000", 2024-09-16T20:48:09.533 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8260000000000003e-06 2024-09-16T20:48:09.533 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.533 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.533 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:09.533 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058727+0000", 2024-09-16T20:48:09.533 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0539999999999999e-06 2024-09-16T20:48:09.533 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.534 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.534 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:09.534 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.060648+0000", 2024-09-16T20:48:09.534 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001920958 2024-09-16T20:48:09.534 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.534 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.534 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:09.534 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.060711+0000", 2024-09-16T20:48:09.534 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2496999999999995e-05 2024-09-16T20:48:09.534 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.535 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.535 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:48:09.535 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.060789+0000", 2024-09-16T20:48:09.535 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8004999999999996e-05 2024-09-16T20:48:09.535 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.535 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.535 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:09.535 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.062182+0000", 2024-09-16T20:48:09.535 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0013930889999999999 2024-09-16T20:48:09.535 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.535 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.535 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:09.535 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.062597+0000", 2024-09-16T20:48:09.535 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00041566800000000001 2024-09-16T20:48:09.536 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.536 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.536 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:09.536 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.062628+0000", 2024-09-16T20:48:09.536 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0876000000000001e-05 2024-09-16T20:48:09.536 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.536 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.536 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:09.536 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.062653+0000", 2024-09-16T20:48:09.536 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.4346999999999999e-05 2024-09-16T20:48:09.536 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:09.536 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:09.537 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:09.537 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.537 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.537 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:48:09.537 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:54.058734+0000", 2024-09-16T20:48:09.537 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.4548314, 2024-09-16T20:48:09.537 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0040462379999999997, 2024-09-16T20:48:09.537 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:09.537 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:09.537 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:09.537 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:09.537 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:48:09.537 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:48:09.537 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 44 2024-09-16T20:48:09.538 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.538 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:09.538 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.538 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:09.538 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058734+0000", 2024-09-16T20:48:09.538 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:09.538 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.538 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.538 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:09.538 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058734+0000", 2024-09-16T20:48:09.538 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:09.538 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.538 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.538 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:09.538 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058734+0000", 2024-09-16T20:48:09.539 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7399999999999999e-07 2024-09-16T20:48:09.539 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.539 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.539 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:09.539 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058809+0000", 2024-09-16T20:48:09.539 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4856000000000003e-05 2024-09-16T20:48:09.539 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.539 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.539 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:09.539 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058816+0000", 2024-09-16T20:48:09.540 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.002e-06 2024-09-16T20:48:09.540 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.540 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.540 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:09.540 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058820+0000", 2024-09-16T20:48:09.540 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8149999999999999e-06 2024-09-16T20:48:09.540 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.540 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.540 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:09.540 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.061036+0000", 2024-09-16T20:48:09.540 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0022159020000000001 2024-09-16T20:48:09.540 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.540 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.540 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:09.541 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.061087+0000", 2024-09-16T20:48:09.541 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1649e-05 2024-09-16T20:48:09.541 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.541 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.541 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:48:09.541 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.061176+0000", 2024-09-16T20:48:09.541 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8355000000000004e-05 2024-09-16T20:48:09.541 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.541 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.541 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:09.541 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.062664+0000", 2024-09-16T20:48:09.542 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001488259 2024-09-16T20:48:09.542 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.542 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.542 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:09.542 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.062725+0000", 2024-09-16T20:48:09.542 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0430000000000002e-05 2024-09-16T20:48:09.542 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.542 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.542 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:09.542 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.062754+0000", 2024-09-16T20:48:09.542 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9133e-05 2024-09-16T20:48:09.542 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.542 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.542 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:09.543 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.062780+0000", 2024-09-16T20:48:09.543 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6563e-05 2024-09-16T20:48:09.543 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:09.543 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:09.543 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:09.543 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.543 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.543 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:48:09.543 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:54.058833+0000", 2024-09-16T20:48:09.543 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.454732309000001, 2024-09-16T20:48:09.543 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0049980959999999996, 2024-09-16T20:48:09.543 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:09.543 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:09.543 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:09.544 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:09.544 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:48:09.544 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:48:09.544 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 45 2024-09-16T20:48:09.544 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.544 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:09.544 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.544 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:09.544 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058833+0000", 2024-09-16T20:48:09.544 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:09.544 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.545 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.545 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:09.545 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058833+0000", 2024-09-16T20:48:09.545 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:09.545 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.545 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.545 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:09.545 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058833+0000", 2024-09-16T20:48:09.545 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8799999999999998e-07 2024-09-16T20:48:09.545 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.546 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.546 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:09.546 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058837+0000", 2024-09-16T20:48:09.546 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4400000000000001e-06 2024-09-16T20:48:09.546 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.546 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.546 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:09.546 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058842+0000", 2024-09-16T20:48:09.546 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5829999999999997e-06 2024-09-16T20:48:09.546 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.546 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.546 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:09.546 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058847+0000", 2024-09-16T20:48:09.547 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.955e-06 2024-09-16T20:48:09.547 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.547 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.547 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:09.547 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.061841+0000", 2024-09-16T20:48:09.547 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0029938500000000002 2024-09-16T20:48:09.547 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.547 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.547 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:09.547 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.061889+0000", 2024-09-16T20:48:09.547 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7540999999999998e-05 2024-09-16T20:48:09.547 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.547 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.547 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:48:09.548 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.061961+0000", 2024-09-16T20:48:09.548 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2218000000000005e-05 2024-09-16T20:48:09.548 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.548 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.548 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:09.548 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.063325+0000", 2024-09-16T20:48:09.548 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001364105 2024-09-16T20:48:09.548 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.548 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.548 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:09.548 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.063774+0000", 2024-09-16T20:48:09.548 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00044868300000000001 2024-09-16T20:48:09.548 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.549 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.549 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:09.549 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.063804+0000", 2024-09-16T20:48:09.549 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9774e-05 2024-09-16T20:48:09.549 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.549 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.549 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:09.549 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.063831+0000", 2024-09-16T20:48:09.549 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7559000000000002e-05 2024-09-16T20:48:09.549 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:09.549 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:09.550 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:09.550 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.550 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.550 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:48:09.550 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:54.058855+0000", 2024-09-16T20:48:09.550 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.454710659, 2024-09-16T20:48:09.550 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0050797430000000003, 2024-09-16T20:48:09.550 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:09.550 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:09.550 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:09.550 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:09.550 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:48:09.550 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:48:09.550 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 46 2024-09-16T20:48:09.551 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.551 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:09.551 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.551 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:09.551 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058855+0000", 2024-09-16T20:48:09.551 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:09.551 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.551 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.551 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:09.551 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058855+0000", 2024-09-16T20:48:09.551 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:09.551 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.551 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.552 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:09.552 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058855+0000", 2024-09-16T20:48:09.552 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2599999999999998e-07 2024-09-16T20:48:09.552 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.552 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.552 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:09.552 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058857+0000", 2024-09-16T20:48:09.552 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2390000000000001e-06 2024-09-16T20:48:09.552 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.552 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.552 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:09.553 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058861+0000", 2024-09-16T20:48:09.553 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.873e-06 2024-09-16T20:48:09.553 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.553 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.553 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:09.553 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058867+0000", 2024-09-16T20:48:09.553 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.268e-06 2024-09-16T20:48:09.553 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.553 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.553 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:09.553 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.062208+0000", 2024-09-16T20:48:09.553 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0033400740000000002 2024-09-16T20:48:09.553 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.553 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.554 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:09.554 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.062257+0000", 2024-09-16T20:48:09.554 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9539000000000001e-05 2024-09-16T20:48:09.554 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.554 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.554 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:48:09.554 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.062333+0000", 2024-09-16T20:48:09.554 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6087000000000005e-05 2024-09-16T20:48:09.554 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.554 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.554 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:09.554 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.063843+0000", 2024-09-16T20:48:09.554 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001509718 2024-09-16T20:48:09.554 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.554 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.555 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:09.555 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.063884+0000", 2024-09-16T20:48:09.555 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1321e-05 2024-09-16T20:48:09.555 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.555 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.555 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:09.555 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.063913+0000", 2024-09-16T20:48:09.555 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9218e-05 2024-09-16T20:48:09.555 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.555 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.555 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:09.555 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.063934+0000", 2024-09-16T20:48:09.556 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1080000000000001e-05 2024-09-16T20:48:09.556 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:09.556 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:09.556 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:09.556 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.556 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.556 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:48:09.556 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:54.058872+0000", 2024-09-16T20:48:09.556 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.454692991, 2024-09-16T20:48:09.556 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0060982579999999996, 2024-09-16T20:48:09.556 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:09.556 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:09.556 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:09.557 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:09.557 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:48:09.557 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:48:09.557 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 47 2024-09-16T20:48:09.557 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.557 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:09.557 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.557 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:09.557 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058872+0000", 2024-09-16T20:48:09.557 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:09.557 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.557 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.557 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:09.557 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058872+0000", 2024-09-16T20:48:09.558 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:09.558 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.558 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.558 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:09.558 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058873+0000", 2024-09-16T20:48:09.558 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1e-07 2024-09-16T20:48:09.558 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.558 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.558 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:09.558 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058919+0000", 2024-09-16T20:48:09.558 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6570999999999998e-05 2024-09-16T20:48:09.558 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.558 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.558 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:09.558 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058926+0000", 2024-09-16T20:48:09.559 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5509999999999996e-06 2024-09-16T20:48:09.559 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.559 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.559 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:09.559 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058932+0000", 2024-09-16T20:48:09.559 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8780000000000003e-06 2024-09-16T20:48:09.559 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.559 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.559 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:09.559 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.062820+0000", 2024-09-16T20:48:09.559 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0038885790000000001 2024-09-16T20:48:09.559 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.559 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.560 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:09.560 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.062868+0000", 2024-09-16T20:48:09.560 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7948999999999998e-05 2024-09-16T20:48:09.560 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.560 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.560 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:48:09.560 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.062945+0000", 2024-09-16T20:48:09.560 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6495999999999994e-05 2024-09-16T20:48:09.560 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.560 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.560 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:09.560 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.064336+0000", 2024-09-16T20:48:09.560 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001391446 2024-09-16T20:48:09.560 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.561 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.561 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:09.561 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.064915+0000", 2024-09-16T20:48:09.561 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00057899000000000004 2024-09-16T20:48:09.561 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.561 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.561 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:09.561 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.064947+0000", 2024-09-16T20:48:09.561 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2132999999999999e-05 2024-09-16T20:48:09.561 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.561 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.561 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:09.561 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.064971+0000", 2024-09-16T20:48:09.562 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3354999999999999e-05 2024-09-16T20:48:09.562 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:09.562 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:09.562 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:09.562 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.562 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.562 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:48:09.562 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:54.058944+0000", 2024-09-16T20:48:09.562 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.454621282, 2024-09-16T20:48:09.562 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0061376369999999996, 2024-09-16T20:48:09.563 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:09.563 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:09.563 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:09.563 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:09.563 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:48:09.563 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:48:09.563 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 48 2024-09-16T20:48:09.563 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.563 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:09.563 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.563 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:09.563 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058944+0000", 2024-09-16T20:48:09.563 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:09.564 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.564 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.564 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:09.564 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058944+0000", 2024-09-16T20:48:09.564 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:09.565 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.565 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.565 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:09.565 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058945+0000", 2024-09-16T20:48:09.565 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9200000000000002e-07 2024-09-16T20:48:09.565 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.565 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.566 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:09.566 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058948+0000", 2024-09-16T20:48:09.566 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8099999999999999e-06 2024-09-16T20:48:09.566 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.566 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.566 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:09.566 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058963+0000", 2024-09-16T20:48:09.566 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5112e-05 2024-09-16T20:48:09.566 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.566 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.566 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:09.566 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058975+0000", 2024-09-16T20:48:09.566 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1732e-05 2024-09-16T20:48:09.567 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.567 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.567 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:09.567 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.063363+0000", 2024-09-16T20:48:09.567 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0043879330000000001 2024-09-16T20:48:09.567 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.567 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.567 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:09.567 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.063420+0000", 2024-09-16T20:48:09.567 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6745e-05 2024-09-16T20:48:09.567 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.568 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.568 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:48:09.568 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.063496+0000", 2024-09-16T20:48:09.568 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6297999999999994e-05 2024-09-16T20:48:09.568 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.568 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.568 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:09.568 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.064981+0000", 2024-09-16T20:48:09.568 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0014852559999999999 2024-09-16T20:48:09.568 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.568 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.568 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:09.568 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.065028+0000", 2024-09-16T20:48:09.568 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7052000000000003e-05 2024-09-16T20:48:09.569 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.569 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.569 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:09.569 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.065060+0000", 2024-09-16T20:48:09.569 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1152000000000003e-05 2024-09-16T20:48:09.569 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.569 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.569 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:09.569 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.065082+0000", 2024-09-16T20:48:09.569 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2155e-05 2024-09-16T20:48:09.569 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:09.569 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:09.569 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:09.569 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.570 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.570 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:48:09.570 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:54.058988+0000", 2024-09-16T20:48:09.570 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.454577644, 2024-09-16T20:48:09.570 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0069229319999999997, 2024-09-16T20:48:09.570 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:09.570 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:09.570 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:09.570 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:09.570 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:48:09.570 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:48:09.570 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 49 2024-09-16T20:48:09.570 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.571 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:09.571 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.571 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:09.571 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058988+0000", 2024-09-16T20:48:09.571 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:09.571 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.571 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.571 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:09.571 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058988+0000", 2024-09-16T20:48:09.571 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:09.571 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.571 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.571 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:09.571 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058988+0000", 2024-09-16T20:48:09.572 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8299999999999998e-07 2024-09-16T20:48:09.572 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.572 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.572 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:09.572 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058992+0000", 2024-09-16T20:48:09.572 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8569999999999997e-06 2024-09-16T20:48:09.572 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.572 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.572 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:09.572 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058998+0000", 2024-09-16T20:48:09.572 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9020000000000004e-06 2024-09-16T20:48:09.573 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.573 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.573 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:09.573 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.059001+0000", 2024-09-16T20:48:09.573 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.269e-06 2024-09-16T20:48:09.573 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.573 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.573 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:09.573 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.063972+0000", 2024-09-16T20:48:09.573 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0049705690000000002 2024-09-16T20:48:09.574 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.574 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.574 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:09.574 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.064032+0000", 2024-09-16T20:48:09.574 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9834999999999999e-05 2024-09-16T20:48:09.574 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.574 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.574 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:48:09.574 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.064128+0000", 2024-09-16T20:48:09.574 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.6113999999999995e-05 2024-09-16T20:48:09.574 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.574 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.574 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:09.574 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.065476+0000", 2024-09-16T20:48:09.575 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00134826 2024-09-16T20:48:09.575 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.575 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.575 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:09.575 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.065855+0000", 2024-09-16T20:48:09.575 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00037943500000000002 2024-09-16T20:48:09.575 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.575 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.575 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:09.575 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.065886+0000", 2024-09-16T20:48:09.575 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0787999999999997e-05 2024-09-16T20:48:09.575 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.575 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.575 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:09.575 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.065911+0000", 2024-09-16T20:48:09.576 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.4519999999999999e-05 2024-09-16T20:48:09.576 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:09.576 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:09.576 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:09.576 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.576 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.576 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:48:09.576 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:54.059010+0000", 2024-09-16T20:48:09.576 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.454555481, 2024-09-16T20:48:09.577 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0069966389999999998, 2024-09-16T20:48:09.577 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:09.577 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:09.577 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:09.577 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:09.577 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:48:09.577 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:48:09.577 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 50 2024-09-16T20:48:09.577 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.577 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:09.577 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.577 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:09.578 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.059010+0000", 2024-09-16T20:48:09.578 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:09.578 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.578 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.578 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:09.578 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.059010+0000", 2024-09-16T20:48:09.578 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:09.578 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.578 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.578 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:09.578 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.059010+0000", 2024-09-16T20:48:09.578 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.72e-07 2024-09-16T20:48:09.578 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.579 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.579 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:09.579 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.059067+0000", 2024-09-16T20:48:09.579 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6756999999999999e-05 2024-09-16T20:48:09.579 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.579 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.579 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:09.579 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.059075+0000", 2024-09-16T20:48:09.579 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8560000000000007e-06 2024-09-16T20:48:09.579 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.579 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.580 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:09.580 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.059078+0000", 2024-09-16T20:48:09.580 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6720000000000002e-06 2024-09-16T20:48:09.580 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.580 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.580 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:09.580 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.064369+0000", 2024-09-16T20:48:09.580 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.005290431 2024-09-16T20:48:09.580 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.580 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.580 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:09.580 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.064416+0000", 2024-09-16T20:48:09.581 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6961e-05 2024-09-16T20:48:09.581 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.581 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.581 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:48:09.581 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.064491+0000", 2024-09-16T20:48:09.581 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5511000000000003e-05 2024-09-16T20:48:09.581 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.581 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.581 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:09.581 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.065922+0000", 2024-09-16T20:48:09.581 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001430728 2024-09-16T20:48:09.582 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.582 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.582 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:09.582 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.065955+0000", 2024-09-16T20:48:09.582 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2741999999999999e-05 2024-09-16T20:48:09.582 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.582 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.582 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:09.582 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.065983+0000", 2024-09-16T20:48:09.582 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8415000000000002e-05 2024-09-16T20:48:09.582 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.582 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.582 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:09.582 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.066007+0000", 2024-09-16T20:48:09.583 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3294000000000001e-05 2024-09-16T20:48:09.583 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:09.583 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:09.583 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:09.583 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.583 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.583 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:48:09.583 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:54.059085+0000", 2024-09-16T20:48:09.583 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.454480383, 2024-09-16T20:48:09.583 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0070804489999999999, 2024-09-16T20:48:09.583 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:09.584 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:09.584 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:09.584 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:09.584 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:48:09.584 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:48:09.584 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 51 2024-09-16T20:48:09.584 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.584 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:09.584 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.584 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:09.584 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.059085+0000", 2024-09-16T20:48:09.584 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:09.584 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.584 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.585 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:09.585 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.059085+0000", 2024-09-16T20:48:09.585 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:09.585 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.585 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.585 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:09.585 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.059085+0000", 2024-09-16T20:48:09.585 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3300000000000003e-07 2024-09-16T20:48:09.585 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.585 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.585 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:09.585 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.059088+0000", 2024-09-16T20:48:09.585 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7700000000000002e-06 2024-09-16T20:48:09.585 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.585 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.586 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:09.586 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.059092+0000", 2024-09-16T20:48:09.586 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6040000000000001e-06 2024-09-16T20:48:09.586 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.586 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.586 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:09.586 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.059100+0000", 2024-09-16T20:48:09.586 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1089999999999998e-06 2024-09-16T20:48:09.586 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.586 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.586 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:09.586 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.065129+0000", 2024-09-16T20:48:09.587 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.006028594 2024-09-16T20:48:09.587 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.587 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.587 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:09.587 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.065175+0000", 2024-09-16T20:48:09.587 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6842999999999999e-05 2024-09-16T20:48:09.587 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.587 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.587 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:48:09.587 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.065251+0000", 2024-09-16T20:48:09.588 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5278000000000007e-05 2024-09-16T20:48:09.588 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.588 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.588 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:09.588 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.066022+0000", 2024-09-16T20:48:09.588 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00077180800000000002 2024-09-16T20:48:09.588 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.588 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.589 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:09.589 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.066118+0000", 2024-09-16T20:48:09.589 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.5594999999999995e-05 2024-09-16T20:48:09.589 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.589 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.590 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:09.590 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.066145+0000", 2024-09-16T20:48:09.590 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7089000000000001e-05 2024-09-16T20:48:09.590 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.590 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.590 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:09.591 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.066165+0000", 2024-09-16T20:48:09.591 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0326000000000002e-05 2024-09-16T20:48:09.591 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:09.591 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:09.591 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:09.592 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.592 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.592 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:48:09.592 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:54.059107+0000", 2024-09-16T20:48:09.592 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.454458331, 2024-09-16T20:48:09.592 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0073197189999999997, 2024-09-16T20:48:09.593 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:09.593 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:09.593 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:09.593 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:09.593 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:48:09.594 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:48:09.594 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 52 2024-09-16T20:48:09.594 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.594 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:09.594 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.594 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:09.595 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.059107+0000", 2024-09-16T20:48:09.595 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:09.595 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.596 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.596 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:09.596 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.059107+0000", 2024-09-16T20:48:09.596 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:09.596 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.596 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.597 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:09.597 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.059107+0000", 2024-09-16T20:48:09.597 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4700000000000002e-07 2024-09-16T20:48:09.597 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.597 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.598 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:09.598 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.059110+0000", 2024-09-16T20:48:09.598 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8499999999999998e-06 2024-09-16T20:48:09.598 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.598 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.599 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:09.599 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.059116+0000", 2024-09-16T20:48:09.599 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0519999999999998e-06 2024-09-16T20:48:09.599 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.599 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.600 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:09.600 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.059119+0000", 2024-09-16T20:48:09.600 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0290000000000001e-06 2024-09-16T20:48:09.600 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.600 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.600 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:09.601 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.065513+0000", 2024-09-16T20:48:09.601 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0063932260000000001 2024-09-16T20:48:09.601 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.601 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.602 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:09.602 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.065571+0000", 2024-09-16T20:48:09.602 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8758000000000002e-05 2024-09-16T20:48:09.602 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.602 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.602 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:48:09.602 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.065648+0000", 2024-09-16T20:48:09.602 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7039999999999994e-05 2024-09-16T20:48:09.602 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.602 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.602 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:09.602 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.066180+0000", 2024-09-16T20:48:09.603 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00053161099999999998 2024-09-16T20:48:09.603 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.603 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.603 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:09.603 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.066381+0000", 2024-09-16T20:48:09.603 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000200564 2024-09-16T20:48:09.603 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.603 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.603 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:09.603 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.066408+0000", 2024-09-16T20:48:09.603 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.775e-05 2024-09-16T20:48:09.603 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.604 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.604 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:09.604 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.066427+0000", 2024-09-16T20:48:09.604 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8491999999999998e-05 2024-09-16T20:48:09.604 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:09.604 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:09.604 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:09.604 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.604 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.604 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:69 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:48:09.604 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:54.239973+0000", 2024-09-16T20:48:09.604 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.273592145, 2024-09-16T20:48:09.605 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0043631160000000002, 2024-09-16T20:48:09.605 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:09.605 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:09.605 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:09.605 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:09.605 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:48:09.605 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:48:09.605 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 69 2024-09-16T20:48:09.605 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.605 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:09.605 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.605 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:09.605 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.239973+0000", 2024-09-16T20:48:09.606 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:09.606 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.606 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.606 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:09.606 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.239973+0000", 2024-09-16T20:48:09.606 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:09.606 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.606 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.606 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:09.606 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.239974+0000", 2024-09-16T20:48:09.606 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.63e-07 2024-09-16T20:48:09.606 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.606 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.607 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:09.607 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240069+0000", 2024-09-16T20:48:09.607 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4907000000000005e-05 2024-09-16T20:48:09.607 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.607 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.607 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:09.607 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240081+0000", 2024-09-16T20:48:09.607 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2500000000000001e-05 2024-09-16T20:48:09.607 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.607 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.607 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:09.607 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240101+0000", 2024-09-16T20:48:09.607 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.016e-05 2024-09-16T20:48:09.608 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.608 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.608 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:09.608 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.242185+0000", 2024-09-16T20:48:09.608 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0020841179999999998 2024-09-16T20:48:09.608 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.608 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.608 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:09.608 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.242230+0000", 2024-09-16T20:48:09.608 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5077000000000002e-05 2024-09-16T20:48:09.608 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.609 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.609 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:48:09.609 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.242296+0000", 2024-09-16T20:48:09.609 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5732e-05 2024-09-16T20:48:09.609 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.609 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.609 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:09.609 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.243310+0000", 2024-09-16T20:48:09.609 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0010134860000000001 2024-09-16T20:48:09.609 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.609 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.610 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:09.610 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.244283+0000", 2024-09-16T20:48:09.610 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00097367700000000005 2024-09-16T20:48:09.610 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.610 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.610 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:09.610 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.244315+0000", 2024-09-16T20:48:09.610 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2055999999999999e-05 2024-09-16T20:48:09.610 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.610 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.610 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:09.610 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.244336+0000", 2024-09-16T20:48:09.610 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0939999999999999e-05 2024-09-16T20:48:09.611 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:09.611 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:09.611 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:09.611 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.611 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.611 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:48:09.611 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:54.240113+0000", 2024-09-16T20:48:09.611 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.273452055, 2024-09-16T20:48:09.611 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0043271009999999999, 2024-09-16T20:48:09.611 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:09.611 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:09.611 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:09.611 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:09.611 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:48:09.611 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:48:09.612 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 70 2024-09-16T20:48:09.612 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.612 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:09.612 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.612 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:09.612 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240113+0000", 2024-09-16T20:48:09.612 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:09.612 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.612 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.612 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:09.612 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240113+0000", 2024-09-16T20:48:09.612 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:09.613 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.613 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.613 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:09.613 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240114+0000", 2024-09-16T20:48:09.613 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9999999999999998e-07 2024-09-16T20:48:09.614 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.614 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.614 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:09.614 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240117+0000", 2024-09-16T20:48:09.614 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8210000000000002e-06 2024-09-16T20:48:09.614 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.615 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.615 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:09.615 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240122+0000", 2024-09-16T20:48:09.615 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0379999999999999e-06 2024-09-16T20:48:09.615 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.615 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.615 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:09.615 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240143+0000", 2024-09-16T20:48:09.615 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1568000000000001e-05 2024-09-16T20:48:09.615 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.615 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.615 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:09.616 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.242735+0000", 2024-09-16T20:48:09.616 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0025919699999999999 2024-09-16T20:48:09.616 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.616 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.616 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:09.616 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.242783+0000", 2024-09-16T20:48:09.616 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7528999999999998e-05 2024-09-16T20:48:09.616 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.616 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.617 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:48:09.617 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.242856+0000", 2024-09-16T20:48:09.617 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3362999999999995e-05 2024-09-16T20:48:09.617 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.617 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.618 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:09.618 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.244378+0000", 2024-09-16T20:48:09.618 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00152242 2024-09-16T20:48:09.618 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.618 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.618 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:09.618 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.244388+0000", 2024-09-16T20:48:09.618 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8630000000000003e-06 2024-09-16T20:48:09.619 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.619 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.619 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:09.619 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.244418+0000", 2024-09-16T20:48:09.619 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9482e-05 2024-09-16T20:48:09.619 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.619 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.619 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:09.620 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.244440+0000", 2024-09-16T20:48:09.620 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2646999999999998e-05 2024-09-16T20:48:09.620 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:09.620 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:09.620 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:09.620 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.620 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.620 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:48:09.621 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:54.240150+0000", 2024-09-16T20:48:09.621 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.273415669, 2024-09-16T20:48:09.621 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0052309740000000002, 2024-09-16T20:48:09.621 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:09.621 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:09.621 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:09.621 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:09.622 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:48:09.622 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:48:09.622 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 71 2024-09-16T20:48:09.622 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.622 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:09.622 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.622 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:09.622 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240150+0000", 2024-09-16T20:48:09.622 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:09.623 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.623 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.623 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:09.623 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240150+0000", 2024-09-16T20:48:09.623 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:09.623 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.623 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.623 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:09.623 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240150+0000", 2024-09-16T20:48:09.624 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2199999999999999e-07 2024-09-16T20:48:09.624 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.624 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.624 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:09.624 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240153+0000", 2024-09-16T20:48:09.624 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5679999999999998e-06 2024-09-16T20:48:09.624 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.624 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.625 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:09.625 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240157+0000", 2024-09-16T20:48:09.625 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2549999999999999e-06 2024-09-16T20:48:09.625 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.625 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.625 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:09.625 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240167+0000", 2024-09-16T20:48:09.625 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.6879999999999994e-06 2024-09-16T20:48:09.626 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.626 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.626 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:09.626 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.243340+0000", 2024-09-16T20:48:09.626 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.003173122 2024-09-16T20:48:09.626 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.626 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.626 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:09.626 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.243381+0000", 2024-09-16T20:48:09.627 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0821000000000001e-05 2024-09-16T20:48:09.627 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.627 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.627 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:48:09.627 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.243448+0000", 2024-09-16T20:48:09.627 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7148000000000002e-05 2024-09-16T20:48:09.627 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.627 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.627 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:09.628 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.244446+0000", 2024-09-16T20:48:09.628 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00099830199999999996 2024-09-16T20:48:09.628 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.628 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.628 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:09.628 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.245283+0000", 2024-09-16T20:48:09.628 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00083687200000000001 2024-09-16T20:48:09.628 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.628 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.628 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:09.628 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.245341+0000", 2024-09-16T20:48:09.628 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8238e-05 2024-09-16T20:48:09.628 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.629 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.629 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:09.629 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.245381+0000", 2024-09-16T20:48:09.629 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9538e-05 2024-09-16T20:48:09.629 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:09.629 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:09.629 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:09.629 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.629 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.629 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:48:09.629 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:54.240178+0000", 2024-09-16T20:48:09.629 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.273387245, 2024-09-16T20:48:09.629 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0059449899999999998, 2024-09-16T20:48:09.629 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:09.630 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:09.630 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:09.630 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:09.630 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:48:09.630 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:48:09.630 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 72 2024-09-16T20:48:09.630 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.630 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:09.630 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.630 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:09.631 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240178+0000", 2024-09-16T20:48:09.631 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:09.631 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.631 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.631 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:09.631 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240178+0000", 2024-09-16T20:48:09.631 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:09.631 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.631 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.631 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:09.631 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240179+0000", 2024-09-16T20:48:09.631 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.53e-07 2024-09-16T20:48:09.631 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.631 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.632 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:09.632 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240252+0000", 2024-09-16T20:48:09.632 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3279000000000001e-05 2024-09-16T20:48:09.632 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.632 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.632 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:09.632 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240259+0000", 2024-09-16T20:48:09.632 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4270000000000002e-06 2024-09-16T20:48:09.632 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.632 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.632 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:09.632 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240263+0000", 2024-09-16T20:48:09.632 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7050000000000001e-06 2024-09-16T20:48:09.633 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.633 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.633 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:09.633 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.243686+0000", 2024-09-16T20:48:09.633 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0034228330000000001 2024-09-16T20:48:09.633 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.633 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.633 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:09.633 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.243731+0000", 2024-09-16T20:48:09.633 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4917999999999999e-05 2024-09-16T20:48:09.633 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.633 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.633 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:48:09.634 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.243801+0000", 2024-09-16T20:48:09.634 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0320000000000004e-05 2024-09-16T20:48:09.634 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.634 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.634 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:09.634 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.245441+0000", 2024-09-16T20:48:09.634 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001640138 2024-09-16T20:48:09.634 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.634 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.634 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:09.634 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.245996+0000", 2024-09-16T20:48:09.634 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000555128 2024-09-16T20:48:09.635 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.635 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.635 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:09.635 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.246057+0000", 2024-09-16T20:48:09.635 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0458999999999998e-05 2024-09-16T20:48:09.635 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.635 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.635 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:09.635 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.246123+0000", 2024-09-16T20:48:09.635 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6429999999999999e-05 2024-09-16T20:48:09.635 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:09.635 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:09.636 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:09.636 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.636 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.636 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:48:09.636 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:54.240274+0000", 2024-09-16T20:48:09.636 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.273291838, 2024-09-16T20:48:09.636 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0063725079999999998, 2024-09-16T20:48:09.636 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:09.636 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:09.636 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:09.637 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:09.637 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:48:09.637 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:48:09.637 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 73 2024-09-16T20:48:09.637 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.637 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:09.637 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.637 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:09.637 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240274+0000", 2024-09-16T20:48:09.637 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:09.637 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.637 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.637 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:09.638 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240274+0000", 2024-09-16T20:48:09.638 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:09.638 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.638 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.638 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:09.638 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240274+0000", 2024-09-16T20:48:09.638 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6100000000000001e-07 2024-09-16T20:48:09.638 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.638 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.638 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:09.638 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240280+0000", 2024-09-16T20:48:09.638 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6960000000000002e-06 2024-09-16T20:48:09.638 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.638 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.639 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:09.639 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240300+0000", 2024-09-16T20:48:09.639 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0151000000000001e-05 2024-09-16T20:48:09.639 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.639 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.639 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:09.639 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240304+0000", 2024-09-16T20:48:09.639 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8330000000000004e-06 2024-09-16T20:48:09.639 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.639 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.639 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:09.639 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.244483+0000", 2024-09-16T20:48:09.639 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0041787769999999998 2024-09-16T20:48:09.640 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.640 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.640 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:09.640 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.244534+0000", 2024-09-16T20:48:09.640 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1158e-05 2024-09-16T20:48:09.640 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.640 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.640 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:48:09.640 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.244611+0000", 2024-09-16T20:48:09.640 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6888e-05 2024-09-16T20:48:09.640 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.640 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.640 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:09.641 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.246133+0000", 2024-09-16T20:48:09.641 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00152224 2024-09-16T20:48:09.641 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.641 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.641 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:09.641 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.246555+0000", 2024-09-16T20:48:09.641 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00042180899999999999 2024-09-16T20:48:09.641 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.641 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.641 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:09.641 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.246610+0000", 2024-09-16T20:48:09.641 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5470999999999998e-05 2024-09-16T20:48:09.641 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.641 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.642 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:09.642 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.246646+0000", 2024-09-16T20:48:09.642 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5923999999999997e-05 2024-09-16T20:48:09.642 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:09.642 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:09.642 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:09.642 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.642 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.642 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:48:09.642 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:54.240313+0000", 2024-09-16T20:48:09.642 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.273251953000001, 2024-09-16T20:48:09.642 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0064355100000000002, 2024-09-16T20:48:09.642 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:09.642 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:09.642 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:09.643 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:09.643 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:48:09.643 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:48:09.643 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 74 2024-09-16T20:48:09.643 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.643 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:09.643 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.643 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:09.643 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240313+0000", 2024-09-16T20:48:09.643 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:09.643 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.643 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.643 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:09.644 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240313+0000", 2024-09-16T20:48:09.644 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:09.644 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.644 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.644 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:09.644 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240314+0000", 2024-09-16T20:48:09.644 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0500000000000003e-07 2024-09-16T20:48:09.644 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.644 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.644 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:09.644 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240324+0000", 2024-09-16T20:48:09.644 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8309999999999996e-06 2024-09-16T20:48:09.645 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.645 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.645 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:09.645 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240329+0000", 2024-09-16T20:48:09.645 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7870000000000001e-06 2024-09-16T20:48:09.645 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.645 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.645 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:09.645 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240332+0000", 2024-09-16T20:48:09.645 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0429999999999999e-06 2024-09-16T20:48:09.645 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.646 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.646 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:09.646 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.244880+0000", 2024-09-16T20:48:09.646 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0045483770000000001 2024-09-16T20:48:09.646 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.646 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.646 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:09.646 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.244925+0000", 2024-09-16T20:48:09.646 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4626000000000003e-05 2024-09-16T20:48:09.647 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.647 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.647 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:48:09.647 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.244994+0000", 2024-09-16T20:48:09.647 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9443999999999994e-05 2024-09-16T20:48:09.647 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.647 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.647 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:09.647 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.246659+0000", 2024-09-16T20:48:09.648 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001665267 2024-09-16T20:48:09.648 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.648 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.648 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:09.648 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.246694+0000", 2024-09-16T20:48:09.648 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4879999999999998e-05 2024-09-16T20:48:09.648 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.648 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.648 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:09.648 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.246724+0000", 2024-09-16T20:48:09.648 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0168999999999999e-05 2024-09-16T20:48:09.648 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.648 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.648 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:09.649 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.246749+0000", 2024-09-16T20:48:09.649 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.4481000000000002e-05 2024-09-16T20:48:09.649 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:09.649 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:09.649 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:09.649 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.649 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.649 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:48:09.649 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:54.240337+0000", 2024-09-16T20:48:09.649 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.273228358000001, 2024-09-16T20:48:09.649 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0064925900000000003, 2024-09-16T20:48:09.649 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:09.650 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:09.650 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:09.650 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:09.650 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:48:09.650 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:48:09.650 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 75 2024-09-16T20:48:09.650 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.650 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:09.650 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.650 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:09.650 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240337+0000", 2024-09-16T20:48:09.650 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:09.651 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.651 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.651 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:09.651 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240337+0000", 2024-09-16T20:48:09.651 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:09.651 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.651 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.651 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:09.651 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240337+0000", 2024-09-16T20:48:09.651 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9799999999999999e-07 2024-09-16T20:48:09.651 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.651 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.651 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:09.651 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240407+0000", 2024-09-16T20:48:09.652 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9887000000000004e-05 2024-09-16T20:48:09.652 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.652 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.652 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:09.652 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240416+0000", 2024-09-16T20:48:09.652 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.3839999999999997e-06 2024-09-16T20:48:09.652 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.652 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.652 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:09.652 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240426+0000", 2024-09-16T20:48:09.652 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0665999999999999e-05 2024-09-16T20:48:09.652 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.652 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.652 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:09.653 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.245481+0000", 2024-09-16T20:48:09.653 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0050551069999999997 2024-09-16T20:48:09.653 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.653 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.653 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:09.653 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.245552+0000", 2024-09-16T20:48:09.653 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0335999999999999e-05 2024-09-16T20:48:09.653 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.653 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.653 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:48:09.653 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.245625+0000", 2024-09-16T20:48:09.653 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3388999999999998e-05 2024-09-16T20:48:09.653 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.653 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.654 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:09.654 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.246762+0000", 2024-09-16T20:48:09.654 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001137239 2024-09-16T20:48:09.654 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.654 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.654 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:09.654 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.246784+0000", 2024-09-16T20:48:09.654 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1299000000000001e-05 2024-09-16T20:48:09.654 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.654 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.654 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:09.654 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.246808+0000", 2024-09-16T20:48:09.654 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.4244e-05 2024-09-16T20:48:09.654 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.654 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.655 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:09.655 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.246830+0000", 2024-09-16T20:48:09.655 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1741000000000001e-05 2024-09-16T20:48:09.655 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:09.655 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:09.655 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:09.655 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.655 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.655 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:48:09.655 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:54.240434+0000", 2024-09-16T20:48:09.655 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.273131383999999, 2024-09-16T20:48:09.655 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0067000209999999996, 2024-09-16T20:48:09.655 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:09.656 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:09.656 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:09.656 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:09.656 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:48:09.656 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:48:09.656 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 76 2024-09-16T20:48:09.656 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.656 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:09.656 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.656 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:09.656 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240434+0000", 2024-09-16T20:48:09.656 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:09.657 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.657 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.657 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:09.657 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240434+0000", 2024-09-16T20:48:09.657 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:09.657 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.657 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.657 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:09.657 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240434+0000", 2024-09-16T20:48:09.657 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6600000000000002e-07 2024-09-16T20:48:09.657 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.657 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.657 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:09.658 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240440+0000", 2024-09-16T20:48:09.658 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6550000000000001e-06 2024-09-16T20:48:09.658 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.658 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.658 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:09.658 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240445+0000", 2024-09-16T20:48:09.658 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9609999999999996e-06 2024-09-16T20:48:09.658 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.658 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.658 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:09.658 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240448+0000", 2024-09-16T20:48:09.658 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8880000000000001e-06 2024-09-16T20:48:09.658 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.658 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.658 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:09.659 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.246163+0000", 2024-09-16T20:48:09.659 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0057155269999999998 2024-09-16T20:48:09.659 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.659 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.659 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:09.659 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.246206+0000", 2024-09-16T20:48:09.659 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2379000000000002e-05 2024-09-16T20:48:09.659 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.659 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.659 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:48:09.659 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.246279+0000", 2024-09-16T20:48:09.659 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2889e-05 2024-09-16T20:48:09.659 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.659 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.660 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:09.660 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.246840+0000", 2024-09-16T20:48:09.660 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00056115099999999997 2024-09-16T20:48:09.660 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.660 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.660 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:09.660 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.247077+0000", 2024-09-16T20:48:09.660 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00023753700000000001 2024-09-16T20:48:09.660 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.660 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.660 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:09.660 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.247112+0000", 2024-09-16T20:48:09.660 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4282000000000001e-05 2024-09-16T20:48:09.661 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:09.661 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:09.661 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:09.661 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.247134+0000", 2024-09-16T20:48:09.661 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2385999999999999e-05 2024-09-16T20:48:09.661 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:09.661 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:09.661 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:09.661 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:09.661 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:09.661 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:09.662 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T20:48:09.663 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:48:09.663 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:09.661+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:09.763 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:09.762+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:09.815 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:09.816 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:48:09.816 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:48:09.816 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:09.825 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:48:09.864 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:09.862+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:09.932 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:09.932 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:48:09.932 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:48:09.932 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:48:09.932 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:09.941 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:48:09.965 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:09.964+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:10.047 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:10.048 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:48:10.048 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:48:10.048 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:48:10.048 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:10.048 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop(client.4245.0:1 2.3 e19/18 2:c4c92e5a:::rbd_trash:head v 19'1, mlcod=0'0)", 2024-09-16T20:48:10.048 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:58.272273+0000", 2024-09-16T20:48:10.048 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 11.775108278999999, 2024-09-16T20:48:10.048 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0010431570000000001, 2024-09-16T20:48:10.049 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:10.049 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:10.049 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:10.049 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:10.049 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:10.049 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:10.049 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:58.272273+0000", 2024-09-16T20:48:10.049 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:10.049 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:10.049 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:10.049 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:10.049 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:58.272273+0000", 2024-09-16T20:48:10.050 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:10.050 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:10.050 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:10.050 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:10.050 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:58.272274+0000", 2024-09-16T20:48:10.050 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1300000000000001e-07 2024-09-16T20:48:10.050 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:10.051 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:10.051 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:10.051 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:58.272280+0000", 2024-09-16T20:48:10.051 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0510000000000002e-06 2024-09-16T20:48:10.051 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:10.051 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:10.051 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:10.051 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:58.272297+0000", 2024-09-16T20:48:10.051 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.6861e-05 2024-09-16T20:48:10.051 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:10.052 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:10.052 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:10.052 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:58.272307+0000", 2024-09-16T20:48:10.052 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0594999999999999e-05 2024-09-16T20:48:10.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:10.052 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:10.052 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:10.052 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:58.272371+0000", 2024-09-16T20:48:10.052 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3707000000000005e-05 2024-09-16T20:48:10.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:10.053 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:10.053 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:10.053 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:58.272399+0000", 2024-09-16T20:48:10.053 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7742e-05 2024-09-16T20:48:10.053 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:10.053 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:10.053 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:10.053 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:58.273263+0000", 2024-09-16T20:48:10.053 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00086466600000000005 2024-09-16T20:48:10.053 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:10.053 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:10.053 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:10.053 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:58.273317+0000", 2024-09-16T20:48:10.054 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3121999999999997e-05 2024-09-16T20:48:10.054 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:10.054 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:10.054 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:10.054 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:10.054 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:10.054 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:10.056 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:48:10.065 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:10.064+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:10.166 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:10.164+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:10.176 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:10.177 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:48:10.177 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:48:10.177 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:10.186 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:48:10.266 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:10.265+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:10.335 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:10.335 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:48:10.335 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:48:10.335 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:48:10.335 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:10.343 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:48:10.367 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:10.366+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:10.450 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:10.450 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:48:10.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:48:10.450 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:48:10.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:10.451 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:13 3.b 3:d0179084:::rbd_object_map.110e7f949337:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-16T20:48:10.451 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:09.573961+0000", 2024-09-16T20:48:10.451 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.87592269300000003, 2024-09-16T20:48:10.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0040759480000000002, 2024-09-16T20:48:10.451 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:10.451 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:10.451 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:10.451 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:10.451 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:48:10.451 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:48:10.451 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 13 2024-09-16T20:48:10.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:10.451 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:10.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:10.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:10.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:09.573961+0000", 2024-09-16T20:48:10.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:10.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:10.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:10.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:10.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:09.573961+0000", 2024-09-16T20:48:10.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:10.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:10.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:10.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:10.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:09.573963+0000", 2024-09-16T20:48:10.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8729999999999999e-06 2024-09-16T20:48:10.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:10.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:10.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:10.453 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:09.573974+0000", 2024-09-16T20:48:10.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1518999999999999e-05 2024-09-16T20:48:10.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:10.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:10.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:10.453 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:09.573985+0000", 2024-09-16T20:48:10.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1104000000000001e-05 2024-09-16T20:48:10.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:10.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:10.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:10.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:09.573995+0000", 2024-09-16T20:48:10.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2949999999999999e-06 2024-09-16T20:48:10.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:10.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:10.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:10.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:09.574096+0000", 2024-09-16T20:48:10.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101778 2024-09-16T20:48:10.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:10.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:10.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:48:10.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:09.574101+0000", 2024-09-16T20:48:10.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.882e-06 2024-09-16T20:48:10.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:10.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:10.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:10.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:09.576504+0000", 2024-09-16T20:48:10.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0024028040000000001 2024-09-16T20:48:10.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:10.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:10.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:10.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:09.576614+0000", 2024-09-16T20:48:10.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000110165 2024-09-16T20:48:10.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:10.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:10.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-16T20:48:10.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:09.576764+0000", 2024-09-16T20:48:10.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000150192 2024-09-16T20:48:10.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:10.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:10.456 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:10.456 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:09.577756+0000", 2024-09-16T20:48:10.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00099194699999999997 2024-09-16T20:48:10.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:10.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:10.456 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:48:10.456 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:09.577969+0000", 2024-09-16T20:48:10.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000212828 2024-09-16T20:48:10.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:10.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:10.456 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:48:10.456 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:09.578009+0000", 2024-09-16T20:48:10.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9767000000000001e-05 2024-09-16T20:48:10.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:10.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:10.457 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:10.457 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:09.578037+0000", 2024-09-16T20:48:10.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7793999999999999e-05 2024-09-16T20:48:10.457 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:10.457 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:10.457 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:10.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:10.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:10.457 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.4366.0:13 3.b e26/23 ondisk, result = 0)", 2024-09-16T20:48:10.457 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:09.577660+0000", 2024-09-16T20:48:10.457 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.87222404200000003, 2024-09-16T20:48:10.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000102861, 2024-09-16T20:48:10.458 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:10.458 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:10.458 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:10.458 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:10.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:10.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:10.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:09.577660+0000", 2024-09-16T20:48:10.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:10.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:10.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:10.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:10.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:09.577660+0000", 2024-09-16T20:48:10.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:10.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:10.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:10.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:10.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:09.577660+0000", 2024-09-16T20:48:10.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8000000000000004e-07 2024-09-16T20:48:10.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:10.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:10.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:10.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:09.577665+0000", 2024-09-16T20:48:10.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.707e-06 2024-09-16T20:48:10.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:10.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:10.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:10.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:09.577675+0000", 2024-09-16T20:48:10.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0372e-05 2024-09-16T20:48:10.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:10.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:10.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:10.460 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:09.577687+0000", 2024-09-16T20:48:10.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1389999999999999e-05 2024-09-16T20:48:10.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:10.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:10.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:10.460 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:09.577747+0000", 2024-09-16T20:48:10.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9750000000000002e-05 2024-09-16T20:48:10.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:10.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:10.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:10.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:09.577753+0000", 2024-09-16T20:48:10.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0689999999999998e-06 2024-09-16T20:48:10.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:10.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:10.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:10.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:09.577762+0000", 2024-09-16T20:48:10.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6930000000000002e-06 2024-09-16T20:48:10.461 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:10.461 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:10.461 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:10.461 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:10.461 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:10.461 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:10.462 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:48:10.507 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:10.506+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:10.570 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:10.568+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:10.570 INFO:teuthology.orchestra.run.smithi029.stderr:nodeep-scrub is set 2024-09-16T20:48:10.574 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:10.574 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:48:10.574 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:48:10.574 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:10.583 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:48:10.671 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:10.670+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:10.730 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:10.730 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:48:10.730 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:48:10.730 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:48:10.730 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:10.738 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:48:10.781 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:10.780+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:10.845 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:10.845 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:48:10.845 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:48:10.846 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:48:10.846 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:10.846 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4245.0:5 2.2 e20/18 2:5cea7035:::rbd_info:head v 20'1, mlcod=0'0)", 2024-09-16T20:48:10.846 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:47:59.275537+0000", 2024-09-16T20:48:10.846 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 11.569522235999999, 2024-09-16T20:48:10.846 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0011504359999999999, 2024-09-16T20:48:10.846 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:10.846 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:10.846 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:10.846 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:10.846 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:10.846 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:10.846 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:47:59.275537+0000", 2024-09-16T20:48:10.846 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:10.847 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:10.847 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:10.847 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:10.847 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:47:59.275537+0000", 2024-09-16T20:48:10.847 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:10.847 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:10.847 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:10.847 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:10.847 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:47:59.275538+0000", 2024-09-16T20:48:10.847 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9299999999999997e-07 2024-09-16T20:48:10.847 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:10.847 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:10.847 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:10.847 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:47:59.275547+0000", 2024-09-16T20:48:10.848 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6540000000000005e-06 2024-09-16T20:48:10.848 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:10.848 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:10.848 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:10.848 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:47:59.275572+0000", 2024-09-16T20:48:10.848 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5103e-05 2024-09-16T20:48:10.848 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:10.848 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:10.848 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:10.848 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:47:59.275588+0000", 2024-09-16T20:48:10.848 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5693e-05 2024-09-16T20:48:10.848 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:10.848 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:10.849 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:10.849 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:47:59.275726+0000", 2024-09-16T20:48:10.849 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013797500000000001 2024-09-16T20:48:10.849 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:10.849 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:10.849 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:10.849 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:47:59.275797+0000", 2024-09-16T20:48:10.849 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1361999999999999e-05 2024-09-16T20:48:10.849 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:10.849 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:10.849 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:48:10.849 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:47:59.276641+0000", 2024-09-16T20:48:10.849 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00084412599999999995 2024-09-16T20:48:10.850 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:10.850 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:10.850 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:10.850 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:47:59.276688+0000", 2024-09-16T20:48:10.850 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6829999999999997e-05 2024-09-16T20:48:10.850 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:10.850 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:10.850 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:10.850 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:10.850 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:10.850 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4245.0:7 2.2 e21/18 2:5cea7035:::rbd_info:head v 21'2, mlcod=0'0)", 2024-09-16T20:48:10.850 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:00.278458+0000", 2024-09-16T20:48:10.851 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 10.56660196, 2024-09-16T20:48:10.851 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00081276099999999995, 2024-09-16T20:48:10.851 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:10.851 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:10.851 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:10.851 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:10.851 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:10.851 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:10.851 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:00.278458+0000", 2024-09-16T20:48:10.851 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:10.851 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:10.851 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:10.851 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:10.852 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:00.278458+0000", 2024-09-16T20:48:10.852 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:10.852 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:10.852 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:10.852 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:10.852 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:00.278458+0000", 2024-09-16T20:48:10.852 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.91e-07 2024-09-16T20:48:10.852 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:10.852 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:10.852 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:10.852 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:00.278465+0000", 2024-09-16T20:48:10.852 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9140000000000002e-06 2024-09-16T20:48:10.852 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:10.852 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:10.853 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:10.853 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:00.278477+0000", 2024-09-16T20:48:10.853 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2225000000000001e-05 2024-09-16T20:48:10.853 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:10.853 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:10.853 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:10.853 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:00.278487+0000", 2024-09-16T20:48:10.853 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7499999999999998e-06 2024-09-16T20:48:10.853 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:10.853 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:10.853 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:10.853 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:00.278561+0000", 2024-09-16T20:48:10.853 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4127000000000004e-05 2024-09-16T20:48:10.853 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:10.853 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:10.854 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:10.854 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:00.278587+0000", 2024-09-16T20:48:10.854 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5911999999999999e-05 2024-09-16T20:48:10.854 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:10.854 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:10.854 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:48:10.854 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:00.279230+0000", 2024-09-16T20:48:10.854 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00064284499999999996 2024-09-16T20:48:10.854 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:10.854 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:10.854 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:10.854 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:00.279270+0000", 2024-09-16T20:48:10.854 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0797000000000003e-05 2024-09-16T20:48:10.854 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:10.854 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:10.855 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:10.855 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:10.855 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:10.855 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:10.855 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T20:48:10.856 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:48:10.900 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:10.899+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:10.973 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:10.972+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:10.991 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:48:10.992 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:48:11.004 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:11.004 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:48:11.004 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:48:11.004 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:11.009 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:48:11.009 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:48:11.013 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:48:11.025 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:48:11.025 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:48:11.043 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:48:11.043 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:48:11.059 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:48:11.059 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:48:11.074 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:11.073+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:11.076 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:48:11.076 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:48:11.161 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:11.161 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:11.169 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:48:11.213 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:11.211+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:11.277 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:11.277 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:11.277 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:11.276+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:11.285 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:48:11.378 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:11.376+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:11.392 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:11.392 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:11.401 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:48:11.468 INFO:teuthology.orchestra.run.smithi139.stdout:5 write 0x1cd600 thru 0x1d90ec (0xbaed bytes) 2024-09-16T20:48:11.469 INFO:teuthology.orchestra.run.smithi139.stdout:truncating to largest ever: 0xd13be00 2024-09-16T20:48:11.478 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:11.477+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:11.484 INFO:teuthology.orchestra.run.smithi139.stdout:6 trunc from 0x1d9200 to 0xd13be00 2024-09-16T20:48:11.500 INFO:teuthology.orchestra.run.smithi139.stdout:7 write 0x3784800 thru 0x3784aea (0x2eb bytes) 2024-09-16T20:48:11.501 INFO:teuthology.orchestra.run.smithi139.stdout:8 write 0x7c42a00 thru 0x7c45293 (0x2894 bytes) 2024-09-16T20:48:11.501 INFO:teuthology.orchestra.run.smithi139.stdout:9 write 0x1e77a00 thru 0x1e863f3 (0xe9f4 bytes) 2024-09-16T20:48:11.501 INFO:teuthology.orchestra.run.smithi139.stdout:11 write 0x1839e00 thru 0x1846dd3 (0xcfd4 bytes) 2024-09-16T20:48:11.501 INFO:teuthology.orchestra.run.smithi139.stdout:12 read 0x9e9d800 thru 0x9eaa02c (0xc82d bytes) 2024-09-16T20:48:11.504 INFO:teuthology.orchestra.run.smithi139.stdout:14 trunc from 0xd13be00 to 0xa9aca00 2024-09-16T20:48:11.519 INFO:teuthology.orchestra.run.smithi139.stdout:15 write 0x14c8e00 thru 0x14caab4 (0x1cb5 bytes) 2024-09-16T20:48:11.526 INFO:teuthology.orchestra.run.smithi139.stdout:17 read 0x8ad2000 thru 0x8ad60c4 (0x40c5 bytes) 2024-09-16T20:48:11.526 INFO:teuthology.orchestra.run.smithi139.stdout:18 write 0x96e0400 thru 0x96e85f8 (0x81f9 bytes) 2024-09-16T20:48:11.527 INFO:teuthology.orchestra.run.smithi139.stdout:19 read 0x3acd000 thru 0x3ad625e (0x925f bytes) 2024-09-16T20:48:11.527 INFO:teuthology.orchestra.run.smithi139.stdout:22 punch from 0xa870e00 to 0xa875200, (0x4400 bytes) 2024-09-16T20:48:11.527 INFO:teuthology.orchestra.run.smithi139.stdout:24 write 0x6237000 thru 0x6238ac4 (0x1ac5 bytes) 2024-09-16T20:48:11.527 INFO:teuthology.orchestra.run.smithi139.stdout:25 read 0x9e55c00 thru 0x9e65185 (0xf586 bytes) 2024-09-16T20:48:11.538 INFO:teuthology.orchestra.run.smithi139.stdout:26 trunc from 0xa9aca00 to 0x776c00 2024-09-16T20:48:11.563 INFO:teuthology.orchestra.run.smithi139.stdout:27 punch from 0x1f5400 to 0x1fc000, (0x6c00 bytes) 2024-09-16T20:48:11.563 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:11.563 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:11.572 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:48:11.581 INFO:teuthology.orchestra.run.smithi139.stdout:32 write 0x19dc600 thru 0x19e8794 (0xc195 bytes) 2024-09-16T20:48:11.593 INFO:teuthology.orchestra.run.smithi139.stdout:33 trunc from 0x19e8800 to 0x6838c00 2024-09-16T20:48:11.605 INFO:teuthology.orchestra.run.smithi139.stdout:35 read 0x3885800 thru 0x38907f2 (0xaff3 bytes) 2024-09-16T20:48:11.606 INFO:teuthology.orchestra.run.smithi139.stdout:36 write 0x3546e00 thru 0x3552938 (0xbb39 bytes) 2024-09-16T20:48:11.613 INFO:teuthology.orchestra.run.smithi139.stdout:40 trunc from 0x6838c00 to 0xc330200 2024-09-16T20:48:11.617 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:11.616+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:11.641 INFO:teuthology.orchestra.run.smithi139.stdout:42 write 0xde96800 thru 0xde9cd1f (0x6520 bytes) 2024-09-16T20:48:11.642 INFO:teuthology.orchestra.run.smithi139.stdout:47 punch from 0x103f800 to 0x1042400, (0x2c00 bytes) 2024-09-16T20:48:11.643 INFO:teuthology.orchestra.run.smithi139.stdout:49 read 0x6568400 thru 0x6572be7 (0xa7e8 bytes) 2024-09-16T20:48:11.643 INFO:teuthology.orchestra.run.smithi139.stdout:50 write 0x5157200 thru 0x5158fb7 (0x1db8 bytes) 2024-09-16T20:48:11.643 INFO:teuthology.orchestra.run.smithi139.stdout:52 read 0x47be00 thru 0x486b4e (0xad4f bytes) 2024-09-16T20:48:11.643 INFO:teuthology.orchestra.run.smithi139.stdout:59 write 0x6bb6e00 thru 0x6bbe26b (0x746c bytes) 2024-09-16T20:48:11.644 INFO:teuthology.orchestra.run.smithi139.stdout:61 write 0x8337a00 thru 0x833842e (0xa2f bytes) 2024-09-16T20:48:11.644 INFO:teuthology.orchestra.run.smithi139.stdout:63 read 0x7a26e00 thru 0x7a333bb (0xc5bc bytes) 2024-09-16T20:48:11.644 INFO:teuthology.orchestra.run.smithi139.stdout:64 read 0xa320c00 thru 0xa328908 (0x7d09 bytes) 2024-09-16T20:48:11.655 INFO:teuthology.orchestra.run.smithi139.stdout:65 trunc from 0xde9ce00 to 0x45a6400 2024-09-16T20:48:11.678 INFO:teuthology.orchestra.run.smithi139.stdout:66 read 0x38b8400 thru 0x38c0287 (0x7e88 bytes) 2024-09-16T20:48:11.680 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:11.678+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:11.692 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:11.692 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:11.698 INFO:teuthology.orchestra.run.smithi139.stdout:68 write 0x8be2200 thru 0x8bebd74 (0x9b75 bytes) 2024-09-16T20:48:11.698 INFO:teuthology.orchestra.run.smithi139.stdout:70 read 0x23e8000 thru 0x23e9bf4 (0x1bf5 bytes) 2024-09-16T20:48:11.701 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:48:11.703 INFO:teuthology.orchestra.run.smithi139.stdout:71 trunc from 0x8bebe00 to 0x478a800 2024-09-16T20:48:11.726 INFO:teuthology.orchestra.run.smithi139.stdout:72 trunc from 0x478a800 to 0x3911e00 2024-09-16T20:48:11.740 INFO:teuthology.orchestra.run.smithi139.stdout:73 trunc from 0x3911e00 to 0x706ae00 2024-09-16T20:48:11.749 INFO:teuthology.orchestra.run.smithi139.stdout:74 trunc from 0x706ae00 to 0x586dc00 2024-09-16T20:48:11.759 INFO:teuthology.orchestra.run.smithi139.stdout:75 read 0x1fe6200 thru 0x1fedb06 (0x7907 bytes) 2024-09-16T20:48:11.761 INFO:teuthology.orchestra.run.smithi139.stdout:78 read 0x145a000 thru 0x146489e (0xa89f bytes) 2024-09-16T20:48:11.761 INFO:teuthology.orchestra.run.smithi139.stdout:80 write 0x50dd200 thru 0x50df29f (0x20a0 bytes) 2024-09-16T20:48:11.761 INFO:teuthology.orchestra.run.smithi139.stdout:81 punch from 0x57e4c00 to 0x57f1e00, (0xd200 bytes) 2024-09-16T20:48:11.761 INFO:teuthology.orchestra.run.smithi139.stdout:82 read 0x32d3a00 thru 0x32dd373 (0x9974 bytes) 2024-09-16T20:48:11.776 INFO:teuthology.orchestra.run.smithi139.stdout:87 write 0x62bd800 thru 0x62cd301 (0xfb02 bytes) 2024-09-16T20:48:11.776 INFO:teuthology.orchestra.run.smithi139.stdout:89 read 0x51fea00 thru 0x520b715 (0xcd16 bytes) 2024-09-16T20:48:11.776 INFO:teuthology.orchestra.run.smithi139.stdout:92 clone 1 order 21 su 16384 sc 12 2024-09-16T20:48:11.780 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:11.779+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:11.855 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:11.855 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:11.863 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:48:11.907 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:11.906+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:11.970 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:11.971 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:11.979 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:48:11.982 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:11.981+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:12.083 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:12.082+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:12.126 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:12.126 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:12.135 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:48:12.218 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:12.217+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:12.281 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:12.281 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:12.285 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:12.283+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:12.290 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:48:12.342 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-16T20:48:12.342 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T20:48:12.342 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T20:48:12.417 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:12.416+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:12.438 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:12.438 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:12.447 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:48:12.487 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:12.485+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:12.587 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T20:48:12.588 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:12.587+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:12.603 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:12.603 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:12.613 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:48:12.684 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:48:12.685 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-09-16T20:48:12.689 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:12.688+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:12.696 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2024-09-16T20:48:12.575507+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590452,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":130,"num_read_kb":106,"num_write":109,"num_write_kb":1636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":73,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":202312,"kb_used_data":2680,"kb_used_omap":9,"kb_used_meta":199478,"kb_avail":566028728,"statfs":{"total":579820584960,"available":579613417472,"internally_reserved":0,"allocated":2744320,"data_stored":1718686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9567,"internal_metadata":204266145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":65,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":64,"num_read_kb":51,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.368730"},"pg_stats":[{"pgid":"3.1f","version":"26'1","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-16T20:48:10.757408+0000","last_change":"2024-09-16T20:48:09.568824+0000","last_active":"2024-09-16T20:48:10.757408+0000","last_peered":"2024-09-16T20:48:10.757408+0000","last_clean":"2024-09-16T20:48:10.757408+0000","last_became_active":"2024-09-16T20:48:07.571756+0000","last_became_peered":"2024-09-16T20:48:07.571756+0000","last_unstale":"2024-09-16T20:48:10.757408+0000","last_undegraded":"2024-09-16T20:48:10.757408+0000","last_fullsized":"2024-09-16T20:48:10.757408+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:54:05.639489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071634499999999996,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":2,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-16T20:48:10.572172+0000","last_change":"2024-09-16T20:48:09.577092+0000","last_active":"2024-09-16T20:48:10.572172+0000","last_peered":"2024-09-16T20:48:10.572172+0000","last_clean":"2024-09-16T20:48:10.572172+0000","last_became_active":"2024-09-16T20:48:07.566669+0000","last_became_peered":"2024-09-16T20:48:07.566669+0000","last_unstale":"2024-09-16T20:48:10.572172+0000","last_undegraded":"2024-09-16T20:48:10.572172+0000","last_fullsized":"2024-09-16T20:48:10.572172+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:51:10.646280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084864999999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-16T20:48:10.757567+0000","last_change":"2024-09-16T20:48:09.564507+0000","last_active":"2024-09-16T20:48:10.757567+0000","last_peered":"2024-09-16T20:48:10.757567+0000","last_clean":"2024-09-16T20:48:10.757567+0000","last_became_active":"2024-09-16T20:48:07.568025+0000","last_became_peered":"2024-09-16T20:48:07.568025+0000","last_unstale":"2024-09-16T20:48:10.757567+0000","last_undegraded":"2024-09-16T20:48:10.757567+0000","last_fullsized":"2024-09-16T20:48:10.757567+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:41:51.419198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068985699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1c","version":"26'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-16T20:48:10.572519+0000","last_change":"2024-09-16T20:48:09.565661+0000","last_active":"2024-09-16T20:48:10.572519+0000","last_peered":"2024-09-16T20:48:10.572519+0000","last_clean":"2024-09-16T20:48:10.572519+0000","last_became_active":"2024-09-16T20:48:07.565241+0000","last_became_peered":"2024-09-16T20:48:07.565241+0000","last_unstale":"2024-09-16T20:48:10.572519+0000","last_undegraded":"2024-09-16T20:48:10.572519+0000","last_fullsized":"2024-09-16T20:48:10.572519+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:09:50.163599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001045589,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-16T20:48:10.571987+0000","last_change":"2024-09-16T20:48:09.576745+0000","last_active":"2024-09-16T20:48:10.571987+0000","last_peered":"2024-09-16T20:48:10.571987+0000","last_clean":"2024-09-16T20:48:10.571987+0000","last_became_active":"2024-09-16T20:48:07.559076+0000","last_became_peered":"2024-09-16T20:48:07.559076+0000","last_unstale":"2024-09-16T20:48:10.571987+0000","last_undegraded":"2024-09-16T20:48:10.571987+0000","last_fullsized":"2024-09-16T20:48:10.571987+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:41:58.067318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057432399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1a","version":"26'2","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-16T20:48:10.572954+0000","last_change":"2024-09-16T20:48:09.566668+0000","last_active":"2024-09-16T20:48:10.572954+0000","last_peered":"2024-09-16T20:48:10.572954+0000","last_clean":"2024-09-16T20:48:10.572954+0000","last_became_active":"2024-09-16T20:48:07.566792+0000","last_became_peered":"2024-09-16T20:48:07.566792+0000","last_unstale":"2024-09-16T20:48:10.572954+0000","last_undegraded":"2024-09-16T20:48:10.572954+0000","last_fullsized":"2024-09-16T20:48:10.572954+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:38:52.194248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002037849,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-16T20:48:07.567745+0000","last_change":"2024-09-16T20:48:07.565009+0000","last_active":"2024-09-16T20:48:07.567745+0000","last_peered":"2024-09-16T20:48:07.567745+0000","last_clean":"2024-09-16T20:48:07.567745+0000","last_became_active":"2024-09-16T20:48:07.564636+0000","last_became_peered":"2024-09-16T20:48:07.564636+0000","last_unstale":"2024-09-16T20:48:07.567745+0000","last_undegraded":"2024-09-16T20:48:07.567745+0000","last_fullsized":"2024-09-16T20:48:07.567745+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:14:33.902722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-16T20:48:10.571928+0000","last_change":"2024-09-16T20:48:09.577049+0000","last_active":"2024-09-16T20:48:10.571928+0000","last_peered":"2024-09-16T20:48:10.571928+0000","last_clean":"2024-09-16T20:48:10.571928+0000","last_became_active":"2024-09-16T20:48:07.562732+0000","last_became_peered":"2024-09-16T20:48:07.562732+0000","last_unstale":"2024-09-16T20:48:10.571928+0000","last_undegraded":"2024-09-16T20:48:10.571928+0000","last_fullsized":"2024-09-16T20:48:10.571928+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:04:09.184551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011061790000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-16T20:48:10.757478+0000","last_change":"2024-09-16T20:48:09.568656+0000","last_active":"2024-09-16T20:48:10.757478+0000","last_peered":"2024-09-16T20:48:10.757478+0000","last_clean":"2024-09-16T20:48:10.757478+0000","last_became_active":"2024-09-16T20:48:07.567576+0000","last_became_peered":"2024-09-16T20:48:07.567576+0000","last_unstale":"2024-09-16T20:48:10.757478+0000","last_undegraded":"2024-09-16T20:48:10.757478+0000","last_fullsized":"2024-09-16T20:48:10.757478+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:54:41.189621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057915500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-16T20:48:10.757516+0000","last_change":"2024-09-16T20:48:09.564832+0000","last_active":"2024-09-16T20:48:10.757516+0000","last_peered":"2024-09-16T20:48:10.757516+0000","last_clean":"2024-09-16T20:48:10.757516+0000","last_became_active":"2024-09-16T20:48:07.565965+0000","last_became_peered":"2024-09-16T20:48:07.565965+0000","last_unstale":"2024-09-16T20:48:10.757516+0000","last_undegraded":"2024-09-16T20:48:10.757516+0000","last_fullsized":"2024-09-16T20:48:10.757516+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:22:57.905206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010075049999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-16T20:48:10.573006+0000","last_change":"2024-09-16T20:48:09.570733+0000","last_active":"2024-09-16T20:48:10.573006+0000","last_peered":"2024-09-16T20:48:10.573006+0000","last_clean":"2024-09-16T20:48:10.573006+0000","last_became_active":"2024-09-16T20:48:07.562407+0000","last_became_peered":"2024-09-16T20:48:07.562407+0000","last_unstale":"2024-09-16T20:48:10.573006+0000","last_undegraded":"2024-09-16T20:48:10.573006+0000","last_fullsized":"2024-09-16T20:48:10.573006+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:22:01.902557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051100699999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-16T20:48:10.572644+0000","last_change":"2024-09-16T20:48:09.566005+0000","last_active":"2024-09-16T20:48:10.572644+0000","last_peered":"2024-09-16T20:48:10.572644+0000","last_clean":"2024-09-16T20:48:10.572644+0000","last_became_active":"2024-09-16T20:48:07.568236+0000","last_became_peered":"2024-09-16T20:48:07.568236+0000","last_unstale":"2024-09-16T20:48:10.572644+0000","last_undegraded":"2024-09-16T20:48:10.572644+0000","last_fullsized":"2024-09-16T20:48:10.572644+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:58:52.990565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001271749,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":32,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-16T20:48:10.572772+0000","last_change":"2024-09-16T20:48:00.277670+0000","last_active":"2024-09-16T20:48:10.572772+0000","last_peered":"2024-09-16T20:48:10.572772+0000","last_clean":"2024-09-16T20:48:10.572772+0000","last_became_active":"2024-09-16T20:47:58.274489+0000","last_became_peered":"2024-09-16T20:47:58.274489+0000","last_unstale":"2024-09-16T20:48:10.572772+0000","last_undegraded":"2024-09-16T20:48:10.572772+0000","last_fullsized":"2024-09-16T20:48:10.572772+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:27:12.633064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091339500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-16T20:48:07.567917+0000","last_change":"2024-09-16T20:48:07.565209+0000","last_active":"2024-09-16T20:48:07.567917+0000","last_peered":"2024-09-16T20:48:07.567917+0000","last_clean":"2024-09-16T20:48:07.567917+0000","last_became_active":"2024-09-16T20:48:07.564677+0000","last_became_peered":"2024-09-16T20:48:07.564677+0000","last_unstale":"2024-09-16T20:48:07.567917+0000","last_undegraded":"2024-09-16T20:48:07.567917+0000","last_fullsized":"2024-09-16T20:48:07.567917+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:02:28.687657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-16T20:48:07.561164+0000","last_change":"2024-09-16T20:48:00.277182+0000","last_active":"2024-09-16T20:48:07.561164+0000","last_peered":"2024-09-16T20:48:07.561164+0000","last_clean":"2024-09-16T20:48:07.561164+0000","last_became_active":"2024-09-16T20:47:58.274389+0000","last_became_peered":"2024-09-16T20:47:58.274389+0000","last_unstale":"2024-09-16T20:48:07.561164+0000","last_undegraded":"2024-09-16T20:48:07.561164+0000","last_fullsized":"2024-09-16T20:48:07.561164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:28:02.441830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043659299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-16T20:48:07.567878+0000","last_change":"2024-09-16T20:48:07.564858+0000","last_active":"2024-09-16T20:48:07.567878+0000","last_peered":"2024-09-16T20:48:07.567878+0000","last_clean":"2024-09-16T20:48:07.567878+0000","last_became_active":"2024-09-16T20:48:07.564514+0000","last_became_peered":"2024-09-16T20:48:07.564514+0000","last_unstale":"2024-09-16T20:48:07.567878+0000","last_undegraded":"2024-09-16T20:48:07.567878+0000","last_fullsized":"2024-09-16T20:48:07.567878+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:29:35.359712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":32,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-16T20:48:10.573017+0000","last_change":"2024-09-16T20:48:00.278054+0000","last_active":"2024-09-16T20:48:10.573017+0000","last_peered":"2024-09-16T20:48:10.573017+0000","last_clean":"2024-09-16T20:48:10.573017+0000","last_became_active":"2024-09-16T20:47:58.272861+0000","last_became_peered":"2024-09-16T20:47:58.272861+0000","last_unstale":"2024-09-16T20:48:10.573017+0000","last_undegraded":"2024-09-16T20:48:10.573017+0000","last_fullsized":"2024-09-16T20:48:10.573017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:13:59.135547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012429489999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-16T20:48:07.567740+0000","last_change":"2024-09-16T20:48:07.564448+0000","last_active":"2024-09-16T20:48:07.567740+0000","last_peered":"2024-09-16T20:48:07.567740+0000","last_clean":"2024-09-16T20:48:07.567740+0000","last_became_active":"2024-09-16T20:48:07.564072+0000","last_became_peered":"2024-09-16T20:48:07.564072+0000","last_unstale":"2024-09-16T20:48:07.567740+0000","last_undegraded":"2024-09-16T20:48:07.567740+0000","last_fullsized":"2024-09-16T20:48:07.567740+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:08:06.926929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-16T20:48:10.757422+0000","last_change":"2024-09-16T20:48:00.833899+0000","last_active":"2024-09-16T20:48:10.757422+0000","last_peered":"2024-09-16T20:48:10.757422+0000","last_clean":"2024-09-16T20:48:10.757422+0000","last_became_active":"2024-09-16T20:47:58.275031+0000","last_became_peered":"2024-09-16T20:47:58.275031+0000","last_unstale":"2024-09-16T20:48:10.757422+0000","last_undegraded":"2024-09-16T20:48:10.757422+0000","last_fullsized":"2024-09-16T20:48:10.757422+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:13:08.658660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046986,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-16T20:48:10.757474+0000","last_change":"2024-09-16T20:48:09.565091+0000","last_active":"2024-09-16T20:48:10.757474+0000","last_peered":"2024-09-16T20:48:10.757474+0000","last_clean":"2024-09-16T20:48:10.757474+0000","last_became_active":"2024-09-16T20:48:07.568828+0000","last_became_peered":"2024-09-16T20:48:07.568828+0000","last_unstale":"2024-09-16T20:48:10.757474+0000","last_undegraded":"2024-09-16T20:48:10.757474+0000","last_fullsized":"2024-09-16T20:48:10.757474+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:43:33.411184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010471600000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-16T20:48:10.573319+0000","last_change":"2024-09-16T20:48:09.570254+0000","last_active":"2024-09-16T20:48:10.573319+0000","last_peered":"2024-09-16T20:48:10.573319+0000","last_clean":"2024-09-16T20:48:10.573319+0000","last_became_active":"2024-09-16T20:48:07.568143+0000","last_became_peered":"2024-09-16T20:48:07.568143+0000","last_unstale":"2024-09-16T20:48:10.573319+0000","last_undegraded":"2024-09-16T20:48:10.573319+0000","last_fullsized":"2024-09-16T20:48:10.573319+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:32:27.013561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038800899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-16T20:48:10.572442+0000","last_change":"2024-09-16T20:48:00.276970+0000","last_active":"2024-09-16T20:48:10.572442+0000","last_peered":"2024-09-16T20:48:10.572442+0000","last_clean":"2024-09-16T20:48:10.572442+0000","last_became_active":"2024-09-16T20:47:58.274617+0000","last_became_peered":"2024-09-16T20:47:58.274617+0000","last_unstale":"2024-09-16T20:48:10.572442+0000","last_undegraded":"2024-09-16T20:48:10.572442+0000","last_fullsized":"2024-09-16T20:48:10.572442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:28:54.467905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062077499999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-16T20:48:10.757533+0000","last_change":"2024-09-16T20:48:09.568988+0000","last_active":"2024-09-16T20:48:10.757533+0000","last_peered":"2024-09-16T20:48:10.757533+0000","last_clean":"2024-09-16T20:48:10.757533+0000","last_became_active":"2024-09-16T20:48:07.568564+0000","last_became_peered":"2024-09-16T20:48:07.568564+0000","last_unstale":"2024-09-16T20:48:10.757533+0000","last_undegraded":"2024-09-16T20:48:10.757533+0000","last_fullsized":"2024-09-16T20:48:10.757533+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:33:25.687411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088331900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":32,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-16T20:48:10.571941+0000","last_change":"2024-09-16T20:48:01.638893+0000","last_active":"2024-09-16T20:48:10.571941+0000","last_peered":"2024-09-16T20:48:10.571941+0000","last_clean":"2024-09-16T20:48:10.571941+0000","last_became_active":"2024-09-16T20:47:58.273088+0000","last_became_peered":"2024-09-16T20:47:58.273088+0000","last_unstale":"2024-09-16T20:48:10.571941+0000","last_undegraded":"2024-09-16T20:48:10.571941+0000","last_fullsized":"2024-09-16T20:48:10.571941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:27:51.451170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000479672,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-16T20:48:07.567957+0000","last_change":"2024-09-16T20:48:07.566423+0000","last_active":"2024-09-16T20:48:07.567957+0000","last_peered":"2024-09-16T20:48:07.567957+0000","last_clean":"2024-09-16T20:48:07.567957+0000","last_became_active":"2024-09-16T20:48:07.566097+0000","last_became_peered":"2024-09-16T20:48:07.566097+0000","last_unstale":"2024-09-16T20:48:07.567957+0000","last_undegraded":"2024-09-16T20:48:07.567957+0000","last_fullsized":"2024-09-16T20:48:07.567957+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:00:40.258240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":32,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-16T20:48:10.757343+0000","last_change":"2024-09-16T20:48:00.408392+0000","last_active":"2024-09-16T20:48:10.757343+0000","last_peered":"2024-09-16T20:48:10.757343+0000","last_clean":"2024-09-16T20:48:10.757343+0000","last_became_active":"2024-09-16T20:47:58.273260+0000","last_became_peered":"2024-09-16T20:47:58.273260+0000","last_unstale":"2024-09-16T20:48:10.757343+0000","last_undegraded":"2024-09-16T20:48:10.757343+0000","last_fullsized":"2024-09-16T20:48:10.757343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:01:53.009880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037636000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-16T20:48:10.571965+0000","last_change":"2024-09-16T20:48:09.576740+0000","last_active":"2024-09-16T20:48:10.571965+0000","last_peered":"2024-09-16T20:48:10.571965+0000","last_clean":"2024-09-16T20:48:10.571965+0000","last_became_active":"2024-09-16T20:48:07.568016+0000","last_became_peered":"2024-09-16T20:48:07.568016+0000","last_unstale":"2024-09-16T20:48:10.571965+0000","last_undegraded":"2024-09-16T20:48:10.571965+0000","last_fullsized":"2024-09-16T20:48:10.571965+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:40:59.868579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063857599999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-16T20:48:10.572652+0000","last_change":"2024-09-16T20:48:00.277739+0000","last_active":"2024-09-16T20:48:10.572652+0000","last_peered":"2024-09-16T20:48:10.572652+0000","last_clean":"2024-09-16T20:48:10.572652+0000","last_became_active":"2024-09-16T20:47:58.271174+0000","last_became_peered":"2024-09-16T20:47:58.271174+0000","last_unstale":"2024-09-16T20:48:10.572652+0000","last_undegraded":"2024-09-16T20:48:10.572652+0000","last_fullsized":"2024-09-16T20:48:10.572652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:11:35.985075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095771799999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":123,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-16T20:48:10.572893+0000","last_change":"2024-09-16T20:47:53.928484+0000","last_active":"2024-09-16T20:48:10.572893+0000","last_peered":"2024-09-16T20:48:10.572893+0000","last_clean":"2024-09-16T20:48:10.572893+0000","last_became_active":"2024-09-16T20:47:53.926422+0000","last_became_peered":"2024-09-16T20:47:53.926422+0000","last_unstale":"2024-09-16T20:48:10.572893+0000","last_undegraded":"2024-09-16T20:48:10.572893+0000","last_fullsized":"2024-09-16T20:48:10.572893+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:52.241408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:52.241408+0000","last_clean_scrub_stamp":"2024-09-16T20:47:52.241408+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:18:04.963138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-16T20:48:07.568020+0000","last_change":"2024-09-16T20:48:07.566435+0000","last_active":"2024-09-16T20:48:07.568020+0000","last_peered":"2024-09-16T20:48:07.568020+0000","last_clean":"2024-09-16T20:48:07.568020+0000","last_became_active":"2024-09-16T20:48:07.566139+0000","last_became_peered":"2024-09-16T20:48:07.566139+0000","last_unstale":"2024-09-16T20:48:07.568020+0000","last_undegraded":"2024-09-16T20:48:07.568020+0000","last_fullsized":"2024-09-16T20:48:07.568020+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:10:21.779449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-16T20:48:10.572366+0000","last_change":"2024-09-16T20:48:09.570001+0000","last_active":"2024-09-16T20:48:10.572366+0000","last_peered":"2024-09-16T20:48:10.572366+0000","last_clean":"2024-09-16T20:48:10.572366+0000","last_became_active":"2024-09-16T20:48:07.558855+0000","last_became_peered":"2024-09-16T20:48:07.558855+0000","last_unstale":"2024-09-16T20:48:10.572366+0000","last_undegraded":"2024-09-16T20:48:10.572366+0000","last_fullsized":"2024-09-16T20:48:10.572366+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:50:47.866044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044614999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-16T20:48:10.572374+0000","last_change":"2024-09-16T20:48:09.569992+0000","last_active":"2024-09-16T20:48:10.572374+0000","last_peered":"2024-09-16T20:48:10.572374+0000","last_clean":"2024-09-16T20:48:10.572374+0000","last_became_active":"2024-09-16T20:48:07.563972+0000","last_became_peered":"2024-09-16T20:48:07.563972+0000","last_unstale":"2024-09-16T20:48:10.572374+0000","last_undegraded":"2024-09-16T20:48:10.572374+0000","last_fullsized":"2024-09-16T20:48:10.572374+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:41:05.991974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039392600000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.b","version":"26'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-16T20:48:10.572038+0000","last_change":"2024-09-16T20:48:09.577659+0000","last_active":"2024-09-16T20:48:10.572038+0000","last_peered":"2024-09-16T20:48:10.572038+0000","last_clean":"2024-09-16T20:48:10.572038+0000","last_became_active":"2024-09-16T20:48:07.560092+0000","last_became_peered":"2024-09-16T20:48:07.560092+0000","last_unstale":"2024-09-16T20:48:10.572038+0000","last_undegraded":"2024-09-16T20:48:10.572038+0000","last_fullsized":"2024-09-16T20:48:10.572038+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:32:37.558082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014289540000000001,"stat_sum":{"num_bytes":30,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-16T20:48:10.572755+0000","last_change":"2024-09-16T20:48:09.570395+0000","last_active":"2024-09-16T20:48:10.572755+0000","last_peered":"2024-09-16T20:48:10.572755+0000","last_clean":"2024-09-16T20:48:10.572755+0000","last_became_active":"2024-09-16T20:48:07.558578+0000","last_became_peered":"2024-09-16T20:48:07.558578+0000","last_unstale":"2024-09-16T20:48:10.572755+0000","last_undegraded":"2024-09-16T20:48:10.572755+0000","last_fullsized":"2024-09-16T20:48:10.572755+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:02:41.933285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047762999999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.d","version":"26'3","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-16T20:48:10.572955+0000","last_change":"2024-09-16T20:48:09.565667+0000","last_active":"2024-09-16T20:48:10.572955+0000","last_peered":"2024-09-16T20:48:10.572955+0000","last_clean":"2024-09-16T20:48:10.572955+0000","last_became_active":"2024-09-16T20:48:07.567327+0000","last_became_peered":"2024-09-16T20:48:07.567327+0000","last_unstale":"2024-09-16T20:48:10.572955+0000","last_undegraded":"2024-09-16T20:48:10.572955+0000","last_fullsized":"2024-09-16T20:48:10.572955+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:15:38.308261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012087560000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":50,"num_read_kb":44,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-16T20:48:10.757501+0000","last_change":"2024-09-16T20:48:09.565024+0000","last_active":"2024-09-16T20:48:10.757501+0000","last_peered":"2024-09-16T20:48:10.757501+0000","last_clean":"2024-09-16T20:48:10.757501+0000","last_became_active":"2024-09-16T20:48:07.570704+0000","last_became_peered":"2024-09-16T20:48:07.570704+0000","last_unstale":"2024-09-16T20:48:10.757501+0000","last_undegraded":"2024-09-16T20:48:10.757501+0000","last_fullsized":"2024-09-16T20:48:10.757501+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:18:34.023448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011026149999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.f","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-16T20:48:10.572693+0000","last_change":"2024-09-16T20:48:09.570786+0000","last_active":"2024-09-16T20:48:10.572693+0000","last_peered":"2024-09-16T20:48:10.572693+0000","last_clean":"2024-09-16T20:48:10.572693+0000","last_became_active":"2024-09-16T20:48:07.560783+0000","last_became_peered":"2024-09-16T20:48:07.560783+0000","last_unstale":"2024-09-16T20:48:10.572693+0000","last_undegraded":"2024-09-16T20:48:10.572693+0000","last_fullsized":"2024-09-16T20:48:10.572693+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:07:12.580180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040623999999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-16T20:48:10.572836+0000","last_change":"2024-09-16T20:48:09.565413+0000","last_active":"2024-09-16T20:48:10.572836+0000","last_peered":"2024-09-16T20:48:10.572836+0000","last_clean":"2024-09-16T20:48:10.572836+0000","last_became_active":"2024-09-16T20:48:07.565932+0000","last_became_peered":"2024-09-16T20:48:07.565932+0000","last_unstale":"2024-09-16T20:48:10.572836+0000","last_undegraded":"2024-09-16T20:48:10.572836+0000","last_fullsized":"2024-09-16T20:48:10.572836+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:13:06.934866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115576,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-16T20:48:10.757466+0000","last_change":"2024-09-16T20:48:09.564513+0000","last_active":"2024-09-16T20:48:10.757466+0000","last_peered":"2024-09-16T20:48:10.757466+0000","last_clean":"2024-09-16T20:48:10.757466+0000","last_became_active":"2024-09-16T20:48:07.571985+0000","last_became_peered":"2024-09-16T20:48:07.571985+0000","last_unstale":"2024-09-16T20:48:10.757466+0000","last_undegraded":"2024-09-16T20:48:10.757466+0000","last_fullsized":"2024-09-16T20:48:10.757466+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:42:05.182478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077335299999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-16T20:48:10.757450+0000","last_change":"2024-09-16T20:48:09.569080+0000","last_active":"2024-09-16T20:48:10.757450+0000","last_peered":"2024-09-16T20:48:10.757450+0000","last_clean":"2024-09-16T20:48:10.757450+0000","last_became_active":"2024-09-16T20:48:07.567597+0000","last_became_peered":"2024-09-16T20:48:07.567597+0000","last_unstale":"2024-09-16T20:48:10.757450+0000","last_undegraded":"2024-09-16T20:48:10.757450+0000","last_fullsized":"2024-09-16T20:48:10.757450+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:48:12.543650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000840955,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-16T20:48:10.572704+0000","last_change":"2024-09-16T20:48:09.565356+0000","last_active":"2024-09-16T20:48:10.572704+0000","last_peered":"2024-09-16T20:48:10.572704+0000","last_clean":"2024-09-16T20:48:10.572704+0000","last_became_active":"2024-09-16T20:48:07.565717+0000","last_became_peered":"2024-09-16T20:48:07.565717+0000","last_unstale":"2024-09-16T20:48:10.572704+0000","last_undegraded":"2024-09-16T20:48:10.572704+0000","last_fullsized":"2024-09-16T20:48:10.572704+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:06:18.150980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013583239999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":65,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":64,"num_read_kb":51,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36196,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335644,"statfs":{"total":96636764160,"available":96599699456,"internally_reserved":0,"allocated":860160,"data_stored":681856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35604,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336236,"statfs":{"total":96636764160,"available":96600305664,"internally_reserved":0,"allocated":262144,"data_stored":91483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31448,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340392,"statfs":{"total":96636764160,"available":96604561408,"internally_reserved":0,"allocated":258048,"data_stored":91453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32036,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339804,"statfs":{"total":96636764160,"available":96603959296,"internally_reserved":0,"allocated":860160,"data_stored":681856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":278528,"data_stored":96439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640263,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35568,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336272,"statfs":{"total":96636764160,"available":96600342528,"internally_reserved":0,"allocated":225280,"data_stored":75599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":30,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":35,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":30,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":35,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T20:48:12.733 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:12.733 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:12.742 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:48:12.790 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:12.789+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:12.842 INFO:teuthology.orchestra.run.smithi139.stdout:95 read 0x4c24a00 thru 0x4c2b3b5 (0x69b6 bytes) 2024-09-16T20:48:12.845 INFO:teuthology.orchestra.run.smithi139.stdout:99 trunc from 0x62cd400 to 0x585d200 2024-09-16T20:48:12.855 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:12.855 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:12.868 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:48:12.913 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:12.911+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:12.976 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:12.976 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:12.984 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:48:13.026 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:13.024+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:13.087 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T20:48:13.093 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:13.091+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:13.113 INFO:teuthology.orchestra.run.smithi139.stdout:100 write 0x38b1000 thru 0x38b126d (0x26e bytes) 2024-09-16T20:48:13.114 INFO:teuthology.orchestra.run.smithi139.stdout:101 write 0x468a800 thru 0x4696e80 (0xc681 bytes) 2024-09-16T20:48:13.137 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:13.138 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:13.148 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:48:13.157 INFO:teuthology.orchestra.run.smithi139.stdout:102 write 0xcbbda00 thru 0xcbcc04a (0xe64b bytes) 2024-09-16T20:48:13.157 INFO:teuthology.orchestra.run.smithi139.stdout:103 read 0xa920800 thru 0xa920ee6 (0x6e7 bytes) 2024-09-16T20:48:13.157 INFO:teuthology.orchestra.run.smithi139.stdout:106 write 0x121b800 thru 0x1223fe4 (0x87e5 bytes) 2024-09-16T20:48:13.158 INFO:teuthology.orchestra.run.smithi139.stdout:109 read 0x4963a00 thru 0x4971c31 (0xe232 bytes) 2024-09-16T20:48:13.158 INFO:teuthology.orchestra.run.smithi139.stdout:111 write 0x9fe1200 thru 0x9feadae (0x9baf bytes) 2024-09-16T20:48:13.159 INFO:teuthology.orchestra.run.smithi139.stdout:114 punch from 0x337b800 to 0x3386c00, (0xb400 bytes) 2024-09-16T20:48:13.162 INFO:teuthology.orchestra.run.smithi139.stdout:115 punch from 0x1487200 to 0x1493600, (0xc400 bytes) 2024-09-16T20:48:13.163 INFO:teuthology.orchestra.run.smithi139.stdout:117 punch from 0x5cd9a00 to 0x5ce2800, (0x8e00 bytes) 2024-09-16T20:48:13.163 INFO:teuthology.orchestra.run.smithi139.stdout:120 write 0x702a000 thru 0x702c657 (0x2658 bytes) 2024-09-16T20:48:13.163 INFO:teuthology.orchestra.run.smithi139.stdout:121 read 0x38f6e00 thru 0x38f9dba (0x2fbb bytes) 2024-09-16T20:48:13.163 INFO:teuthology.orchestra.run.smithi139.stdout:123 read 0x73daa00 thru 0x73e6932 (0xbf33 bytes) 2024-09-16T20:48:13.163 INFO:teuthology.orchestra.run.smithi139.stdout:124 read 0x5c71a00 thru 0x5c73117 (0x1718 bytes) 2024-09-16T20:48:13.164 INFO:teuthology.orchestra.run.smithi139.stdout:125 read 0x1b43800 thru 0x1b52895 (0xf096 bytes) 2024-09-16T20:48:13.165 INFO:teuthology.orchestra.run.smithi139.stdout:129 write 0x7d5ba00 thru 0x7d65f11 (0xa512 bytes) 2024-09-16T20:48:13.165 INFO:teuthology.orchestra.run.smithi139.stdout:130 write 0x6e10800 thru 0x6e1d8c2 (0xd0c3 bytes) 2024-09-16T20:48:13.165 INFO:teuthology.orchestra.run.smithi139.stdout:131 write 0x7282000 thru 0x7283964 (0x1965 bytes) 2024-09-16T20:48:13.179 INFO:teuthology.orchestra.run.smithi139.stdout:133 trunc from 0xcbcc200 to 0x14ac400 2024-09-16T20:48:13.234 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:13.232+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:13.242 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:48:13.242 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:48:13.249 INFO:teuthology.orchestra.run.smithi139.stdout:134 punch from 0x57f000 to 0x58ba00, (0xca00 bytes) 2024-09-16T20:48:13.253 INFO:teuthology.orchestra.run.smithi139.stdout:136 trunc from 0x14ac400 to 0x4f28600 2024-09-16T20:48:13.262 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:48:13.262 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:48:13.269 INFO:teuthology.orchestra.run.smithi139.stdout:137 read 0x2d91000 thru 0x2d96418 (0x5419 bytes) 2024-09-16T20:48:13.269 INFO:teuthology.orchestra.run.smithi139.stdout:139 read 0x1eaf400 thru 0x1eb5dea (0x69eb bytes) 2024-09-16T20:48:13.269 INFO:teuthology.orchestra.run.smithi139.stdout:141 read 0x40efa00 thru 0x40fb397 (0xb998 bytes) 2024-09-16T20:48:13.270 INFO:teuthology.orchestra.run.smithi139.stdout:143 read 0x152d800 thru 0x1532627 (0x4e28 bytes) 2024-09-16T20:48:13.270 INFO:teuthology.orchestra.run.smithi139.stdout:144 read 0x4968600 thru 0x496fa88 (0x7489 bytes) 2024-09-16T20:48:13.270 INFO:teuthology.orchestra.run.smithi139.stdout:145 read 0x1234e00 thru 0x1241171 (0xc372 bytes) 2024-09-16T20:48:13.271 INFO:teuthology.orchestra.run.smithi139.stdout:147 write 0x3351800 thru 0x33578c9 (0x60ca bytes) 2024-09-16T20:48:13.271 INFO:teuthology.orchestra.run.smithi139.stdout:148 read 0x2490000 thru 0x2498c1a (0x8c1b bytes) 2024-09-16T20:48:13.271 INFO:teuthology.orchestra.run.smithi139.stdout:150 read 0x41cdc00 thru 0x41d1df6 (0x41f7 bytes) 2024-09-16T20:48:13.281 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:48:13.281 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:48:13.295 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:13.294+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:13.297 INFO:teuthology.orchestra.run.smithi139.stdout:151 write 0xc0a0e00 thru 0xc0a8adb (0x7cdc bytes) 2024-09-16T20:48:13.297 INFO:teuthology.orchestra.run.smithi139.stdout:159 read 0xbc1c000 thru 0xbc22ffc (0x6ffd bytes) 2024-09-16T20:48:13.297 INFO:teuthology.orchestra.run.smithi139.stdout:160 punch from 0x853ee00 to 0x8549200, (0xa400 bytes) 2024-09-16T20:48:13.298 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:48:13.298 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:48:13.311 INFO:teuthology.orchestra.run.smithi139.stdout:162 trunc from 0xc0a8c00 to 0x490800 2024-09-16T20:48:13.314 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:13.314 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:13.320 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:48:13.320 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:48:13.325 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:48:13.336 INFO:teuthology.orchestra.run.smithi139.stdout:165 read 0x2fc800 thru 0x30bc24 (0xf425 bytes) 2024-09-16T20:48:13.339 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:48:13.339 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:48:13.354 INFO:teuthology.orchestra.run.smithi139.stdout:166 write 0x35aca00 thru 0x35b5906 (0x8f07 bytes) 2024-09-16T20:48:13.354 INFO:teuthology.orchestra.run.smithi139.stdout:truncating to largest ever: 0xea06e00 2024-09-16T20:48:13.367 INFO:teuthology.orchestra.run.smithi139.stdout:167 trunc from 0x35b5a00 to 0xea06e00 2024-09-16T20:48:13.386 INFO:teuthology.orchestra.run.smithi139.stdout:170 read 0xb1a3000 thru 0xb1a3e2e (0xe2f bytes) 2024-09-16T20:48:13.386 INFO:teuthology.orchestra.run.smithi139.stdout:171 punch from 0x4af5400 to 0x4b04600, (0xf200 bytes) 2024-09-16T20:48:13.386 INFO:teuthology.orchestra.run.smithi139.stdout:173 write 0xdf70000 thru 0xdf7e5d6 (0xe5d7 bytes) 2024-09-16T20:48:13.386 INFO:teuthology.orchestra.run.smithi139.stdout:175 read 0x4a22a00 thru 0x4a25c0b (0x320c bytes) 2024-09-16T20:48:13.390 INFO:teuthology.orchestra.run.smithi139.stdout:176 trunc from 0xea06e00 to 0xace0800 2024-09-16T20:48:13.396 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:13.395+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:13.416 INFO:teuthology.orchestra.run.smithi139.stdout:178 punch from 0x5421000 to 0x5427000, (0x6000 bytes) 2024-09-16T20:48:13.419 INFO:teuthology.orchestra.run.smithi139.stdout:180 trunc from 0xace0800 to 0x7f41800 2024-09-16T20:48:13.443 INFO:teuthology.orchestra.run.smithi139.stdout:181 trunc from 0x7f41800 to 0x2186e00 2024-09-16T20:48:13.465 INFO:teuthology.orchestra.run.smithi139.stdout:182 read 0xdf8000 thru 0xe01781 (0x9782 bytes) 2024-09-16T20:48:13.465 INFO:teuthology.orchestra.run.smithi139.stdout:183 punch from 0x1f97000 to 0x1f98a00, (0x1a00 bytes) 2024-09-16T20:48:13.482 INFO:teuthology.orchestra.run.smithi139.stdout:185 write 0x25d3c00 thru 0x25de1ce (0xa5cf bytes) 2024-09-16T20:48:13.482 INFO:teuthology.orchestra.run.smithi139.stdout:186 read 0x20ac000 thru 0x20b3633 (0x7634 bytes) 2024-09-16T20:48:13.489 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:13.489 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:48:13.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:48:13.489 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:48:13.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.489 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4245.0:5 2.2 e20/18 2:5cea7035:::rbd_info:head v 20'1, mlcod=0'0)", 2024-09-16T20:48:13.489 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:47:59.275537+0000", 2024-09-16T20:48:13.489 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 14.210785763000001, 2024-09-16T20:48:13.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0011504359999999999, 2024-09-16T20:48:13.489 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:13.489 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:13.490 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:13.490 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:13.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:13.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:47:59.275537+0000", 2024-09-16T20:48:13.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:13.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:13.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:47:59.275537+0000", 2024-09-16T20:48:13.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:13.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:13.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:47:59.275538+0000", 2024-09-16T20:48:13.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9299999999999997e-07 2024-09-16T20:48:13.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:13.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:47:59.275547+0000", 2024-09-16T20:48:13.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6540000000000005e-06 2024-09-16T20:48:13.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:13.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:47:59.275572+0000", 2024-09-16T20:48:13.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5103e-05 2024-09-16T20:48:13.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.492 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:13.492 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:47:59.275588+0000", 2024-09-16T20:48:13.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5693e-05 2024-09-16T20:48:13.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.492 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:13.492 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:47:59.275726+0000", 2024-09-16T20:48:13.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013797500000000001 2024-09-16T20:48:13.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.492 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:13.492 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:47:59.275797+0000", 2024-09-16T20:48:13.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1361999999999999e-05 2024-09-16T20:48:13.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.493 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:48:13.493 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:47:59.276641+0000", 2024-09-16T20:48:13.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00084412599999999995 2024-09-16T20:48:13.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.493 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:13.493 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:47:59.276688+0000", 2024-09-16T20:48:13.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6829999999999997e-05 2024-09-16T20:48:13.493 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:13.493 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:13.493 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:13.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.494 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4245.0:7 2.2 e21/18 2:5cea7035:::rbd_info:head v 21'2, mlcod=0'0)", 2024-09-16T20:48:13.494 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:00.278458+0000", 2024-09-16T20:48:13.494 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 13.207865486999999, 2024-09-16T20:48:13.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00081276099999999995, 2024-09-16T20:48:13.494 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:13.494 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:13.494 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:13.494 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:13.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.494 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:13.494 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:00.278458+0000", 2024-09-16T20:48:13.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:13.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:13.495 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:00.278458+0000", 2024-09-16T20:48:13.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:13.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:13.495 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:00.278458+0000", 2024-09-16T20:48:13.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.91e-07 2024-09-16T20:48:13.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:13.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:00.278465+0000", 2024-09-16T20:48:13.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9140000000000002e-06 2024-09-16T20:48:13.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:13.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:00.278477+0000", 2024-09-16T20:48:13.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2225000000000001e-05 2024-09-16T20:48:13.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:13.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:00.278487+0000", 2024-09-16T20:48:13.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7499999999999998e-06 2024-09-16T20:48:13.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:13.497 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:00.278561+0000", 2024-09-16T20:48:13.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4127000000000004e-05 2024-09-16T20:48:13.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:13.497 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:00.278587+0000", 2024-09-16T20:48:13.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5911999999999999e-05 2024-09-16T20:48:13.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:48:13.497 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:00.279230+0000", 2024-09-16T20:48:13.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00064284499999999996 2024-09-16T20:48:13.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.498 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:13.498 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:00.279270+0000", 2024-09-16T20:48:13.498 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0797000000000003e-05 2024-09-16T20:48:13.498 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:13.498 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:13.498 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:13.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.498 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4462.0:2007 3.19 3:9b3df84f:::rbd_data.110e1748a8d2.0000000000000005:head [stat,truncate 393216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T20:48:13.498 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:13.320940+0000", 2024-09-16T20:48:13.498 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.165382897, 2024-09-16T20:48:13.499 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018938410000000001, 2024-09-16T20:48:13.499 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:13.499 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:13.499 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:13.499 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:13.499 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4462", 2024-09-16T20:48:13.499 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:13.499 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 2007 2024-09-16T20:48:13.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.499 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:13.499 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.499 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:13.499 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.320940+0000", 2024-09-16T20:48:13.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:13.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.500 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:13.500 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.320940+0000", 2024-09-16T20:48:13.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:13.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.500 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:13.500 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.320941+0000", 2024-09-16T20:48:13.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1370000000000001e-06 2024-09-16T20:48:13.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:13.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.320946+0000", 2024-09-16T20:48:13.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4329999999999997e-06 2024-09-16T20:48:13.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:13.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.320967+0000", 2024-09-16T20:48:13.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1452999999999999e-05 2024-09-16T20:48:13.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:13.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.320976+0000", 2024-09-16T20:48:13.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6829999999999997e-06 2024-09-16T20:48:13.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.502 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:13.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.321062+0000", 2024-09-16T20:48:13.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6155000000000004e-05 2024-09-16T20:48:13.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.502 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:13.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.321151+0000", 2024-09-16T20:48:13.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8455000000000006e-05 2024-09-16T20:48:13.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3", 2024-09-16T20:48:13.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.321234+0000", 2024-09-16T20:48:13.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3543999999999998e-05 2024-09-16T20:48:13.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:48:13.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.321955+0000", 2024-09-16T20:48:13.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00072084700000000005 2024-09-16T20:48:13.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:13.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.322718+0000", 2024-09-16T20:48:13.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00076296199999999995 2024-09-16T20:48:13.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.504 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:48:13.504 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.322778+0000", 2024-09-16T20:48:13.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0543e-05 2024-09-16T20:48:13.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.504 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:13.504 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.322834+0000", 2024-09-16T20:48:13.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5628999999999999e-05 2024-09-16T20:48:13.504 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:13.504 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:13.504 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:13.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.505 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4462.0:2011 3.19 3:9cf0b687:::rbd_data.110e1748a8d2.0000000000000009:head [stat,truncate 393216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T20:48:13.505 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:13.321012+0000", 2024-09-16T20:48:13.505 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.16531116600000001, 2024-09-16T20:48:13.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0019734269999999998, 2024-09-16T20:48:13.505 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:13.505 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:13.505 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:13.505 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:13.505 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4462", 2024-09-16T20:48:13.505 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:13.505 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 2011 2024-09-16T20:48:13.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.506 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:13.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:13.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.321012+0000", 2024-09-16T20:48:13.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:13.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:13.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.321012+0000", 2024-09-16T20:48:13.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:13.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:13.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.321013+0000", 2024-09-16T20:48:13.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0500000000000003e-07 2024-09-16T20:48:13.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.507 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.507 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:13.507 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.321018+0000", 2024-09-16T20:48:13.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9749999999999998e-06 2024-09-16T20:48:13.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.507 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.507 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:13.507 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.321030+0000", 2024-09-16T20:48:13.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2027e-05 2024-09-16T20:48:13.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.507 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.507 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:13.507 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.321037+0000", 2024-09-16T20:48:13.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0419999999999997e-06 2024-09-16T20:48:13.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.508 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:13.508 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.321566+0000", 2024-09-16T20:48:13.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00052955000000000005 2024-09-16T20:48:13.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.508 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:13.508 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.321635+0000", 2024-09-16T20:48:13.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8603999999999996e-05 2024-09-16T20:48:13.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3", 2024-09-16T20:48:13.509 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.321753+0000", 2024-09-16T20:48:13.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000117953 2024-09-16T20:48:13.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:48:13.509 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.322433+0000", 2024-09-16T20:48:13.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00068059399999999997 2024-09-16T20:48:13.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:13.509 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.322886+0000", 2024-09-16T20:48:13.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00045234499999999999 2024-09-16T20:48:13.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:48:13.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.322945+0000", 2024-09-16T20:48:13.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9014000000000001e-05 2024-09-16T20:48:13.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:13.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.322985+0000", 2024-09-16T20:48:13.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0717999999999999e-05 2024-09-16T20:48:13.510 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:13.510 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:13.510 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:13.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.511 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4462.0:2013 3.19 3:9eda7881:::rbd_data.110e1748a8d2.000000000000000b:head [stat,truncate 393216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T20:48:13.511 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:13.321049+0000", 2024-09-16T20:48:13.511 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.165273955, 2024-09-16T20:48:13.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0021654790000000001, 2024-09-16T20:48:13.511 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:13.511 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:13.511 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:13.511 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:13.511 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4462", 2024-09-16T20:48:13.511 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:13.511 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 2013 2024-09-16T20:48:13.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.511 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:13.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:13.512 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.321049+0000", 2024-09-16T20:48:13.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:13.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.512 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.512 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:13.512 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.321049+0000", 2024-09-16T20:48:13.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:13.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.512 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.512 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:13.512 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.321050+0000", 2024-09-16T20:48:13.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.06e-07 2024-09-16T20:48:13.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.512 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.512 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:13.513 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.321052+0000", 2024-09-16T20:48:13.513 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5270000000000001e-06 2024-09-16T20:48:13.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.513 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:13.513 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.321060+0000", 2024-09-16T20:48:13.513 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4229999999999999e-06 2024-09-16T20:48:13.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.513 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:13.513 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.321065+0000", 2024-09-16T20:48:13.513 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.772e-06 2024-09-16T20:48:13.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:13.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.321985+0000", 2024-09-16T20:48:13.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00091996899999999997 2024-09-16T20:48:13.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:13.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.322058+0000", 2024-09-16T20:48:13.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2286999999999994e-05 2024-09-16T20:48:13.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3", 2024-09-16T20:48:13.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.322183+0000", 2024-09-16T20:48:13.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000125332 2024-09-16T20:48:13.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.515 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:48:13.515 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.322998+0000", 2024-09-16T20:48:13.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000814837 2024-09-16T20:48:13.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.515 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.515 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:13.515 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.323151+0000", 2024-09-16T20:48:13.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015334800000000001 2024-09-16T20:48:13.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.515 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.515 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:48:13.515 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.323180+0000", 2024-09-16T20:48:13.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8668999999999999e-05 2024-09-16T20:48:13.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.515 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.516 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:13.516 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.323215+0000", 2024-09-16T20:48:13.516 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4808999999999998e-05 2024-09-16T20:48:13.516 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:13.516 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:13.516 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:13.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.516 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.516 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4462.0:2117 3.19 3:9eda7881:::rbd_data.110e1748a8d2.000000000000000b:head [read 0~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-16T20:48:13.516 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:13.358299+0000", 2024-09-16T20:48:13.516 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.12802393500000001, 2024-09-16T20:48:13.516 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00039398299999999999, 2024-09-16T20:48:13.516 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:13.516 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:13.517 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:13.517 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:13.517 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4462", 2024-09-16T20:48:13.517 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:13.517 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 2117 2024-09-16T20:48:13.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.517 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:13.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.517 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:13.517 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.358299+0000", 2024-09-16T20:48:13.517 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:13.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.518 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:13.518 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.358299+0000", 2024-09-16T20:48:13.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:13.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.518 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:13.518 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.358300+0000", 2024-09-16T20:48:13.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1600000000000001e-07 2024-09-16T20:48:13.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.518 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:13.518 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.358303+0000", 2024-09-16T20:48:13.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.824e-06 2024-09-16T20:48:13.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.519 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:13.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.358310+0000", 2024-09-16T20:48:13.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.311e-06 2024-09-16T20:48:13.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.519 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:13.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.358322+0000", 2024-09-16T20:48:13.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2306e-05 2024-09-16T20:48:13.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.519 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:13.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.358577+0000", 2024-09-16T20:48:13.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00025505800000000001 2024-09-16T20:48:13.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.520 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:13.520 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.358626+0000", 2024-09-16T20:48:13.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9048000000000002e-05 2024-09-16T20:48:13.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.520 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:13.520 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.358693+0000", 2024-09-16T20:48:13.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6920000000000003e-05 2024-09-16T20:48:13.520 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:13.520 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:13.520 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:13.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.520 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4462.0:2123 3.8 3:15221d58:::rbd_data.110e1748a8d2.000000000000001d:head [set-alloc-hint object_size 2097152 write_size 2097152,write 491520~6656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T20:48:13.520 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:13.370336+0000", 2024-09-16T20:48:13.521 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.115986808, 2024-09-16T20:48:13.521 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001249165, 2024-09-16T20:48:13.521 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:13.521 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:13.521 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:13.521 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:13.521 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4462", 2024-09-16T20:48:13.521 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:13.521 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 2123 2024-09-16T20:48:13.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.521 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:13.521 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.521 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:13.521 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.370336+0000", 2024-09-16T20:48:13.522 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:13.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.522 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:13.522 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.370336+0000", 2024-09-16T20:48:13.522 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:13.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.522 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:13.522 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.370338+0000", 2024-09-16T20:48:13.522 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2860000000000001e-06 2024-09-16T20:48:13.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.522 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:13.522 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.370349+0000", 2024-09-16T20:48:13.522 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1324e-05 2024-09-16T20:48:13.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:13.523 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.370366+0000", 2024-09-16T20:48:13.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6973000000000002e-05 2024-09-16T20:48:13.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:13.523 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.370374+0000", 2024-09-16T20:48:13.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4610000000000008e-06 2024-09-16T20:48:13.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:13.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.370441+0000", 2024-09-16T20:48:13.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6138999999999998e-05 2024-09-16T20:48:13.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.524 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:13.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.370526+0000", 2024-09-16T20:48:13.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.564e-05 2024-09-16T20:48:13.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.524 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-16T20:48:13.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.370607+0000", 2024-09-16T20:48:13.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0914000000000005e-05 2024-09-16T20:48:13.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.525 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.525 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:48:13.525 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.371193+0000", 2024-09-16T20:48:13.525 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00058586599999999999 2024-09-16T20:48:13.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.525 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.525 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:13.525 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.371525+0000", 2024-09-16T20:48:13.525 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00033158900000000003 2024-09-16T20:48:13.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.525 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.525 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:48:13.525 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.371557+0000", 2024-09-16T20:48:13.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2938999999999998e-05 2024-09-16T20:48:13.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.526 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.526 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:13.526 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.371586+0000", 2024-09-16T20:48:13.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8034e-05 2024-09-16T20:48:13.526 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:13.526 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:13.526 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:13.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.526 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.526 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4462.0:2179 3.19 3:9b3df84f:::rbd_data.110e1748a8d2.0000000000000005:head [read 0~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-16T20:48:13.526 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:13.391065+0000", 2024-09-16T20:48:13.527 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.095257963000000001, 2024-09-16T20:48:13.527 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00040012499999999998, 2024-09-16T20:48:13.527 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:13.527 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:13.527 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:13.527 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:13.527 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4462", 2024-09-16T20:48:13.527 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:13.527 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 2179 2024-09-16T20:48:13.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.527 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:13.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.527 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:13.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.391065+0000", 2024-09-16T20:48:13.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:13.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:13.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.391065+0000", 2024-09-16T20:48:13.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:13.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:13.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.391067+0000", 2024-09-16T20:48:13.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2809999999999998e-06 2024-09-16T20:48:13.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.529 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:13.529 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.391075+0000", 2024-09-16T20:48:13.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7959999999999996e-06 2024-09-16T20:48:13.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.529 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:13.529 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.391097+0000", 2024-09-16T20:48:13.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1820000000000001e-05 2024-09-16T20:48:13.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.529 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:13.530 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.391112+0000", 2024-09-16T20:48:13.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4598e-05 2024-09-16T20:48:13.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.530 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:13.530 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.391237+0000", 2024-09-16T20:48:13.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000125172 2024-09-16T20:48:13.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.530 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:13.530 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.391329+0000", 2024-09-16T20:48:13.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2596999999999995e-05 2024-09-16T20:48:13.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:13.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.391465+0000", 2024-09-16T20:48:13.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013586100000000001 2024-09-16T20:48:13.531 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:13.531 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:13.531 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:13.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.531 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4462.0:2182 3.1 3:800f6fd1:::rbd_data.110e1748a8d2.000000000000002f:head [set-alloc-hint object_size 2097152 write_size 2097152,write 245760~16384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T20:48:13.531 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:13.392651+0000", 2024-09-16T20:48:13.531 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.093672444999999993, 2024-09-16T20:48:13.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001516236, 2024-09-16T20:48:13.531 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:13.532 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:13.532 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:13.532 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:13.532 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4462", 2024-09-16T20:48:13.532 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:13.532 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 2182 2024-09-16T20:48:13.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.532 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:13.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.532 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:13.532 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.392651+0000", 2024-09-16T20:48:13.532 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:13.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.533 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:13.533 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.392651+0000", 2024-09-16T20:48:13.533 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:13.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.533 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.533 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:13.533 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.392652+0000", 2024-09-16T20:48:13.533 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0300000000000001e-06 2024-09-16T20:48:13.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.533 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.533 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:13.533 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.392675+0000", 2024-09-16T20:48:13.533 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3234000000000002e-05 2024-09-16T20:48:13.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.534 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:13.534 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.392689+0000", 2024-09-16T20:48:13.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.437e-05 2024-09-16T20:48:13.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.534 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:13.534 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.392695+0000", 2024-09-16T20:48:13.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9320000000000001e-06 2024-09-16T20:48:13.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.534 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:13.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.392751+0000", 2024-09-16T20:48:13.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5717000000000002e-05 2024-09-16T20:48:13.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:13.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.392833+0000", 2024-09-16T20:48:13.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2225000000000006e-05 2024-09-16T20:48:13.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-16T20:48:13.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.392981+0000", 2024-09-16T20:48:13.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014779100000000001 2024-09-16T20:48:13.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:48:13.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.393709+0000", 2024-09-16T20:48:13.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00072800199999999995 2024-09-16T20:48:13.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.536 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:13.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.394061+0000", 2024-09-16T20:48:13.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00035180399999999999 2024-09-16T20:48:13.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.536 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:48:13.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.394122+0000", 2024-09-16T20:48:13.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1265000000000004e-05 2024-09-16T20:48:13.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.536 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:13.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.394167+0000", 2024-09-16T20:48:13.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4866e-05 2024-09-16T20:48:13.537 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:13.537 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:13.537 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:13.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.537 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4462.0:2183 3.6 3:67e3d162:::rbd_data.110e1748a8d2.0000000000000074:head [set-alloc-hint object_size 2097152 write_size 2097152,write 638976~16384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T20:48:13.537 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:13.392741+0000", 2024-09-16T20:48:13.537 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.093582420999999999, 2024-09-16T20:48:13.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017596409999999999, 2024-09-16T20:48:13.537 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:13.537 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:13.537 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:13.537 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:13.538 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4462", 2024-09-16T20:48:13.538 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:13.538 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 2183 2024-09-16T20:48:13.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.538 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:13.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.538 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:13.538 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.392741+0000", 2024-09-16T20:48:13.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:13.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.538 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:13.538 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.392741+0000", 2024-09-16T20:48:13.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:13.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.539 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:13.539 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.392742+0000", 2024-09-16T20:48:13.539 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0999999999999997e-07 2024-09-16T20:48:13.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.539 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:13.539 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.392765+0000", 2024-09-16T20:48:13.539 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3518000000000001e-05 2024-09-16T20:48:13.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.539 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:13.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.392775+0000", 2024-09-16T20:48:13.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0219e-05 2024-09-16T20:48:13.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:13.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.392785+0000", 2024-09-16T20:48:13.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.268e-06 2024-09-16T20:48:13.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:13.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.392835+0000", 2024-09-16T20:48:13.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0139999999999998e-05 2024-09-16T20:48:13.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:13.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.392941+0000", 2024-09-16T20:48:13.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010652799999999999 2024-09-16T20:48:13.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.541 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3", 2024-09-16T20:48:13.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.393049+0000", 2024-09-16T20:48:13.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000107642 2024-09-16T20:48:13.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.541 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:48:13.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.394059+0000", 2024-09-16T20:48:13.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001009879 2024-09-16T20:48:13.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:13.542 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.394440+0000", 2024-09-16T20:48:13.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00038108499999999997 2024-09-16T20:48:13.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:48:13.542 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.394474+0000", 2024-09-16T20:48:13.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4507e-05 2024-09-16T20:48:13.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:13.542 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.394500+0000", 2024-09-16T20:48:13.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6044999999999999e-05 2024-09-16T20:48:13.543 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:13.543 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:13.543 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:13.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.543 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.543 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4462.0:2191 3.19 3:9eda7881:::rbd_data.110e1748a8d2.000000000000000b:head [read 0~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-16T20:48:13.543 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:13.393259+0000", 2024-09-16T20:48:13.543 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.093063857999999999, 2024-09-16T20:48:13.543 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00051406399999999995, 2024-09-16T20:48:13.543 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:13.543 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:13.543 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:13.543 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:13.543 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4462", 2024-09-16T20:48:13.543 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:13.544 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 2191 2024-09-16T20:48:13.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.544 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:13.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.544 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:13.544 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.393259+0000", 2024-09-16T20:48:13.544 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:13.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.544 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:13.544 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.393259+0000", 2024-09-16T20:48:13.544 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:13.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.545 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:13.545 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.393260+0000", 2024-09-16T20:48:13.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6100000000000001e-07 2024-09-16T20:48:13.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.545 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:13.545 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.393262+0000", 2024-09-16T20:48:13.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.356e-06 2024-09-16T20:48:13.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.545 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:13.545 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.393270+0000", 2024-09-16T20:48:13.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7500000000000003e-06 2024-09-16T20:48:13.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:13.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.393275+0000", 2024-09-16T20:48:13.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5280000000000003e-06 2024-09-16T20:48:13.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:13.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.393601+0000", 2024-09-16T20:48:13.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00032606200000000001 2024-09-16T20:48:13.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:13.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.393670+0000", 2024-09-16T20:48:13.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8244000000000006e-05 2024-09-16T20:48:13.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.547 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:13.547 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.393773+0000", 2024-09-16T20:48:13.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000103663 2024-09-16T20:48:13.547 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:13.547 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:13.547 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:13.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.547 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4462.0:2202 3.19 3:9eda7881:::rbd_data.110e1748a8d2.000000000000000b:head [read 16384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-16T20:48:13.547 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:13.394817+0000", 2024-09-16T20:48:13.547 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.091506547999999993, 2024-09-16T20:48:13.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00057552700000000005, 2024-09-16T20:48:13.547 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:13.548 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:13.548 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:13.548 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:13.548 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4462", 2024-09-16T20:48:13.548 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:13.548 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 2202 2024-09-16T20:48:13.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.548 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:13.548 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.548 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:13.548 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.394817+0000", 2024-09-16T20:48:13.548 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:13.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.548 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.548 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:13.549 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.394817+0000", 2024-09-16T20:48:13.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:13.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.549 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:13.549 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.394817+0000", 2024-09-16T20:48:13.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0999999999999999e-07 2024-09-16T20:48:13.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.549 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:13.549 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.394820+0000", 2024-09-16T20:48:13.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8609999999999998e-06 2024-09-16T20:48:13.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.550 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:13.550 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.394830+0000", 2024-09-16T20:48:13.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0173000000000001e-05 2024-09-16T20:48:13.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.550 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:13.550 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.394836+0000", 2024-09-16T20:48:13.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6860000000000003e-06 2024-09-16T20:48:13.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.550 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:13.550 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.395198+0000", 2024-09-16T20:48:13.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000362223 2024-09-16T20:48:13.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.551 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:13.551 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.395277+0000", 2024-09-16T20:48:13.551 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9334999999999999e-05 2024-09-16T20:48:13.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.551 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:13.551 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.395392+0000", 2024-09-16T20:48:13.551 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114739 2024-09-16T20:48:13.551 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:13.551 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:13.551 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:13.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.552 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.552 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4462.0:2226 3.6 3:67e3d162:::rbd_data.110e1748a8d2.0000000000000074:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T20:48:13.552 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:13.400651+0000", 2024-09-16T20:48:13.552 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.085671685999999997, 2024-09-16T20:48:13.552 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00136352, 2024-09-16T20:48:13.552 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:13.552 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:13.552 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:13.552 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:13.552 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4462", 2024-09-16T20:48:13.552 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:13.552 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 2226 2024-09-16T20:48:13.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.552 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:13.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.553 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:13.553 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.400651+0000", 2024-09-16T20:48:13.553 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:13.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.553 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:13.553 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.400651+0000", 2024-09-16T20:48:13.553 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:13.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.553 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:13.553 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.400652+0000", 2024-09-16T20:48:13.553 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0100000000000004e-07 2024-09-16T20:48:13.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.554 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:13.554 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.400656+0000", 2024-09-16T20:48:13.554 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3279999999999999e-06 2024-09-16T20:48:13.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.554 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.554 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:13.554 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.400670+0000", 2024-09-16T20:48:13.554 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3834e-05 2024-09-16T20:48:13.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.554 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.554 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:13.554 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.400678+0000", 2024-09-16T20:48:13.554 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6399999999999997e-06 2024-09-16T20:48:13.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:13.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.400736+0000", 2024-09-16T20:48:13.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8046e-05 2024-09-16T20:48:13.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:13.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.400824+0000", 2024-09-16T20:48:13.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7688000000000005e-05 2024-09-16T20:48:13.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3", 2024-09-16T20:48:13.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.400938+0000", 2024-09-16T20:48:13.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114644 2024-09-16T20:48:13.556 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.556 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.556 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:48:13.556 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.401471+0000", 2024-09-16T20:48:13.556 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00053242000000000003 2024-09-16T20:48:13.556 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.556 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.556 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:13.556 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.401916+0000", 2024-09-16T20:48:13.556 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00044517299999999999 2024-09-16T20:48:13.556 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.556 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.556 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:48:13.557 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.401974+0000", 2024-09-16T20:48:13.557 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8204e-05 2024-09-16T20:48:13.557 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.557 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.557 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:13.557 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.402015+0000", 2024-09-16T20:48:13.557 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0842e-05 2024-09-16T20:48:13.557 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:13.557 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:13.557 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:13.557 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.557 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.557 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4462.0:2293 3.19 3:9eda7881:::rbd_data.110e1748a8d2.000000000000000b:head [read 0~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-16T20:48:13.557 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:13.421537+0000", 2024-09-16T20:48:13.558 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.064785661999999994, 2024-09-16T20:48:13.558 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000445384, 2024-09-16T20:48:13.558 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:13.558 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:13.558 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:13.558 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:13.558 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4462", 2024-09-16T20:48:13.558 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:13.558 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 2293 2024-09-16T20:48:13.558 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.558 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:13.558 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.558 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:13.558 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.421537+0000", 2024-09-16T20:48:13.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:13.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:13.559 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.421537+0000", 2024-09-16T20:48:13.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:13.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:13.559 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.421538+0000", 2024-09-16T20:48:13.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2900000000000003e-07 2024-09-16T20:48:13.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:13.560 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.421542+0000", 2024-09-16T20:48:13.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.456e-06 2024-09-16T20:48:13.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.560 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:13.560 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.421550+0000", 2024-09-16T20:48:13.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1119999999999996e-06 2024-09-16T20:48:13.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.560 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:13.560 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.421556+0000", 2024-09-16T20:48:13.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8980000000000001e-06 2024-09-16T20:48:13.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.561 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:13.561 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.421786+0000", 2024-09-16T20:48:13.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00023035699999999999 2024-09-16T20:48:13.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.561 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.561 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:13.561 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.421871+0000", 2024-09-16T20:48:13.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5375000000000002e-05 2024-09-16T20:48:13.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.561 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.561 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:13.561 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.421983+0000", 2024-09-16T20:48:13.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000111557 2024-09-16T20:48:13.561 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:13.562 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:13.562 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:13.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.562 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4462.0:2301 3.19 3:9cf0b687:::rbd_data.110e1748a8d2.0000000000000009:head [read 16384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-16T20:48:13.562 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:13.422302+0000", 2024-09-16T20:48:13.562 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.064021066000000001, 2024-09-16T20:48:13.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000418529, 2024-09-16T20:48:13.562 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:13.562 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:13.562 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:13.562 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:13.562 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4462", 2024-09-16T20:48:13.562 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:13.563 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 2301 2024-09-16T20:48:13.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.563 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:13.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.563 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:13.563 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.422302+0000", 2024-09-16T20:48:13.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:13.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.563 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:13.563 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.422302+0000", 2024-09-16T20:48:13.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:13.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:13.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.422303+0000", 2024-09-16T20:48:13.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2999999999999999e-07 2024-09-16T20:48:13.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:13.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.422306+0000", 2024-09-16T20:48:13.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5039999999999998e-06 2024-09-16T20:48:13.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:13.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.422327+0000", 2024-09-16T20:48:13.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0445000000000002e-05 2024-09-16T20:48:13.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.565 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:13.565 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.422333+0000", 2024-09-16T20:48:13.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6259999999999997e-06 2024-09-16T20:48:13.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.565 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:13.565 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.422552+0000", 2024-09-16T20:48:13.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000218385 2024-09-16T20:48:13.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.565 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:13.565 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.422624+0000", 2024-09-16T20:48:13.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2417999999999997e-05 2024-09-16T20:48:13.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.566 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.566 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:13.566 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.422721+0000", 2024-09-16T20:48:13.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6321000000000002e-05 2024-09-16T20:48:13.566 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:13.566 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:13.566 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:13.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.566 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.566 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4462.0:2303 3.19 3:9eda7881:::rbd_data.110e1748a8d2.000000000000000b:head [read 16384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-16T20:48:13.566 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:13.422353+0000", 2024-09-16T20:48:13.566 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.063970201000000004, 2024-09-16T20:48:13.567 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00050020200000000005, 2024-09-16T20:48:13.567 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:13.567 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:13.567 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:13.567 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:13.567 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4462", 2024-09-16T20:48:13.567 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:13.567 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 2303 2024-09-16T20:48:13.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.567 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:13.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.567 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:13.567 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.422353+0000", 2024-09-16T20:48:13.567 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:13.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.568 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:13.568 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.422353+0000", 2024-09-16T20:48:13.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:13.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.568 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:13.568 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.422354+0000", 2024-09-16T20:48:13.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8100000000000003e-07 2024-09-16T20:48:13.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.568 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:13.568 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.422357+0000", 2024-09-16T20:48:13.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0170000000000001e-06 2024-09-16T20:48:13.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.569 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:13.569 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.422364+0000", 2024-09-16T20:48:13.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7670000000000003e-06 2024-09-16T20:48:13.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.569 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:13.569 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.422370+0000", 2024-09-16T20:48:13.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.755e-06 2024-09-16T20:48:13.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.569 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:13.570 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.422744+0000", 2024-09-16T20:48:13.570 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00037439599999999998 2024-09-16T20:48:13.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.570 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:13.570 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.422785+0000", 2024-09-16T20:48:13.570 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0503999999999997e-05 2024-09-16T20:48:13.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.570 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:13.570 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.422853+0000", 2024-09-16T20:48:13.570 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8182000000000006e-05 2024-09-16T20:48:13.570 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:13.570 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:13.570 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:13.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.571 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.571 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4462.0:2372 3.19 3:9eda7881:::rbd_data.110e1748a8d2.000000000000000b:head [read 16384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-16T20:48:13.571 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:13.440720+0000", 2024-09-16T20:48:13.571 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.045603496, 2024-09-16T20:48:13.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00056217799999999998, 2024-09-16T20:48:13.571 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:13.571 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:13.571 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:13.571 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:13.571 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4462", 2024-09-16T20:48:13.571 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:13.571 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 2372 2024-09-16T20:48:13.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.572 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:13.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.572 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:13.572 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.440720+0000", 2024-09-16T20:48:13.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:13.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.572 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:13.572 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.440720+0000", 2024-09-16T20:48:13.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:13.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.572 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:13.573 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.440720+0000", 2024-09-16T20:48:13.573 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3900000000000005e-07 2024-09-16T20:48:13.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.573 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.573 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:13.573 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.440725+0000", 2024-09-16T20:48:13.573 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0300000000000001e-06 2024-09-16T20:48:13.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.573 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.573 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:13.573 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.440751+0000", 2024-09-16T20:48:13.573 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6239000000000001e-05 2024-09-16T20:48:13.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.573 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.574 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:13.574 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.440758+0000", 2024-09-16T20:48:13.574 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6549999999999998e-06 2024-09-16T20:48:13.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.574 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:13.574 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.441089+0000", 2024-09-16T20:48:13.574 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000331143 2024-09-16T20:48:13.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.574 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:13.574 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.441170+0000", 2024-09-16T20:48:13.574 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0833000000000003e-05 2024-09-16T20:48:13.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.577 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:13.577 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.441282+0000", 2024-09-16T20:48:13.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000111739 2024-09-16T20:48:13.577 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:13.577 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:13.577 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:13.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.577 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4462.0:2386 3.8 3:15221d58:::rbd_data.110e1748a8d2.000000000000001d:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T20:48:13.577 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:13.447010+0000", 2024-09-16T20:48:13.577 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.039313357, 2024-09-16T20:48:13.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017918299999999999, 2024-09-16T20:48:13.577 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:13.578 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:13.578 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:13.578 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:13.578 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4462", 2024-09-16T20:48:13.578 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:13.578 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 2386 2024-09-16T20:48:13.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.578 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:13.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.578 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:13.578 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.447010+0000", 2024-09-16T20:48:13.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:13.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:13.579 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.447010+0000", 2024-09-16T20:48:13.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:13.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:13.579 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.447011+0000", 2024-09-16T20:48:13.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.291e-06 2024-09-16T20:48:13.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:13.579 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.447016+0000", 2024-09-16T20:48:13.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9699999999999998e-06 2024-09-16T20:48:13.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:13.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.447034+0000", 2024-09-16T20:48:13.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7507999999999999e-05 2024-09-16T20:48:13.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:13.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.447040+0000", 2024-09-16T20:48:13.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5810000000000002e-06 2024-09-16T20:48:13.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:13.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.447119+0000", 2024-09-16T20:48:13.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9152000000000004e-05 2024-09-16T20:48:13.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:13.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.447249+0000", 2024-09-16T20:48:13.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013011399999999999 2024-09-16T20:48:13.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-16T20:48:13.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.447367+0000", 2024-09-16T20:48:13.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000117761 2024-09-16T20:48:13.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:48:13.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.448277+0000", 2024-09-16T20:48:13.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00090998499999999998 2024-09-16T20:48:13.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:13.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.448696+0000", 2024-09-16T20:48:13.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00041867899999999998 2024-09-16T20:48:13.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:48:13.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.448751+0000", 2024-09-16T20:48:13.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5435e-05 2024-09-16T20:48:13.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:13.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.448802+0000", 2024-09-16T20:48:13.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0353999999999999e-05 2024-09-16T20:48:13.583 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:13.583 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:13.583 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:13.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.583 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4462.0:2395 3.1 3:800f6fd1:::rbd_data.110e1748a8d2.000000000000002f:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T20:48:13.583 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:13.447090+0000", 2024-09-16T20:48:13.583 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.039233380999999998, 2024-09-16T20:48:13.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017342410000000001, 2024-09-16T20:48:13.584 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:13.584 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:13.584 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:13.584 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:13.584 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4462", 2024-09-16T20:48:13.584 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:13.584 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 2395 2024-09-16T20:48:13.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.584 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:13.584 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.584 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:13.584 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.447090+0000", 2024-09-16T20:48:13.584 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:13.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:13.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.447090+0000", 2024-09-16T20:48:13.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:13.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:13.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.447090+0000", 2024-09-16T20:48:13.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9599999999999999e-07 2024-09-16T20:48:13.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:13.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.447094+0000", 2024-09-16T20:48:13.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7950000000000001e-06 2024-09-16T20:48:13.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:13.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.447105+0000", 2024-09-16T20:48:13.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0234999999999999e-05 2024-09-16T20:48:13.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:13.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.447113+0000", 2024-09-16T20:48:13.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8559999999999996e-06 2024-09-16T20:48:13.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:13.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.447179+0000", 2024-09-16T20:48:13.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5852000000000006e-05 2024-09-16T20:48:13.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:13.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.447272+0000", 2024-09-16T20:48:13.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3273e-05 2024-09-16T20:48:13.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-16T20:48:13.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.447385+0000", 2024-09-16T20:48:13.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011260999999999999 2024-09-16T20:48:13.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:48:13.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.448001+0000", 2024-09-16T20:48:13.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00061624499999999997 2024-09-16T20:48:13.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:13.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.448730+0000", 2024-09-16T20:48:13.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00072829500000000001 2024-09-16T20:48:13.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:48:13.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.448780+0000", 2024-09-16T20:48:13.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9988999999999999e-05 2024-09-16T20:48:13.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:13.589 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.448824+0000", 2024-09-16T20:48:13.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4394999999999998e-05 2024-09-16T20:48:13.589 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:13.589 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:13.589 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:13.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.589 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4462.0:2464 3.19 3:9eda7881:::rbd_data.110e1748a8d2.000000000000000b:head [read 0~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-16T20:48:13.589 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:13.468468+0000", 2024-09-16T20:48:13.590 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.017855274000000001, 2024-09-16T20:48:13.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00044578199999999998, 2024-09-16T20:48:13.590 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:13.590 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:13.590 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:13.590 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:13.590 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4462", 2024-09-16T20:48:13.590 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:13.590 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 2464 2024-09-16T20:48:13.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.590 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:13.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.590 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:13.591 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.468468+0000", 2024-09-16T20:48:13.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:13.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.591 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:13.591 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.468468+0000", 2024-09-16T20:48:13.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:13.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.591 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:13.591 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.468468+0000", 2024-09-16T20:48:13.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6899999999999997e-07 2024-09-16T20:48:13.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.592 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:13.592 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.468472+0000", 2024-09-16T20:48:13.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7299999999999999e-06 2024-09-16T20:48:13.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.592 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:13.592 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.468481+0000", 2024-09-16T20:48:13.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9279999999999999e-06 2024-09-16T20:48:13.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.592 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:13.592 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.468487+0000", 2024-09-16T20:48:13.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2140000000000001e-06 2024-09-16T20:48:13.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.593 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:13.593 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.468715+0000", 2024-09-16T20:48:13.593 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00022783300000000001 2024-09-16T20:48:13.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.593 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:13.593 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.468810+0000", 2024-09-16T20:48:13.593 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5266000000000006e-05 2024-09-16T20:48:13.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:13.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:13.593 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:13.593 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.468914+0000", 2024-09-16T20:48:13.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000103242 2024-09-16T20:48:13.594 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:13.594 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:13.594 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:13.594 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:13.594 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:13.594 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:13.595 INFO:teuthology.orchestra.run.smithi029.stderr:noscrub is unset 2024-09-16T20:48:13.596 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:48:13.597 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:13.593+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:13.598 INFO:teuthology.orchestra.run.smithi139.stdout:187 write 0x59f6c00 thru 0x59fb0fa (0x44fb bytes) 2024-09-16T20:48:13.598 INFO:teuthology.orchestra.run.smithi139.stdout:188 write 0x1d85400 thru 0x1d917a0 (0xc3a1 bytes) 2024-09-16T20:48:13.598 INFO:teuthology.orchestra.run.smithi139.stdout:191 trunc from 0x59fb200 to 0x48fc800 2024-09-16T20:48:13.598 INFO:teuthology.orchestra.run.smithi139.stdout:192 read 0x16c3a00 thru 0x16c79a3 (0x3fa4 bytes) 2024-09-16T20:48:13.598 INFO:teuthology.orchestra.run.smithi139.stdout:195 trunc from 0x48fc800 to 0xd243400 2024-09-16T20:48:13.598 INFO:teuthology.orchestra.run.smithi139.stdout:197 write 0x39e00 thru 0x3dff4 (0x41f5 bytes) 2024-09-16T20:48:13.599 INFO:teuthology.orchestra.run.smithi139.stdout:203 read 0x8354000 thru 0x8363597 (0xf598 bytes) 2024-09-16T20:48:13.599 INFO:teuthology.orchestra.run.smithi139.stdout:204 punch from 0x45b2400 to 0x45c1200, (0xee00 bytes) 2024-09-16T20:48:13.599 INFO:teuthology.orchestra.run.smithi139.stdout:205 read 0x832f800 thru 0x833480a (0x500b bytes) 2024-09-16T20:48:13.599 INFO:teuthology.orchestra.run.smithi139.stdout:207 write 0x532e000 thru 0x533b3c1 (0xd3c2 bytes) 2024-09-16T20:48:13.599 INFO:teuthology.orchestra.run.smithi139.stdout:208 write 0x275a400 thru 0x2766c90 (0xc891 bytes) 2024-09-16T20:48:13.599 INFO:teuthology.orchestra.run.smithi139.stdout:209 write 0x4e4d800 thru 0x4e50eee (0x36ef bytes) 2024-09-16T20:48:13.599 INFO:teuthology.orchestra.run.smithi139.stdout:210 read 0x417c600 thru 0x41899ea (0xd3eb bytes) 2024-09-16T20:48:13.599 INFO:teuthology.orchestra.run.smithi139.stdout:211 punch from 0x9c8ae00 to 0x9c8b000, (0x200 bytes) 2024-09-16T20:48:13.599 INFO:teuthology.orchestra.run.smithi139.stdout:213 write 0x6af7e00 thru 0x6affa08 (0x7c09 bytes) 2024-09-16T20:48:13.599 INFO:teuthology.orchestra.run.smithi139.stdout:214 write 0x9730e00 thru 0x97383f3 (0x75f4 bytes) 2024-09-16T20:48:13.599 INFO:teuthology.orchestra.run.smithi139.stdout:215 punch from 0x12d6a00 to 0x12dc600, (0x5c00 bytes) 2024-09-16T20:48:13.599 INFO:teuthology.orchestra.run.smithi139.stdout:219 write 0x6147200 thru 0x615176c (0xa56d bytes) 2024-09-16T20:48:13.599 INFO:teuthology.orchestra.run.smithi139.stdout:220 punch from 0x9e62e00 to 0x9e69400, (0x6600 bytes) 2024-09-16T20:48:13.600 INFO:teuthology.orchestra.run.smithi139.stdout:223 trunc from 0xd243400 to 0x2aa8200 2024-09-16T20:48:13.611 INFO:teuthology.orchestra.run.smithi139.stdout:224 read 0xe3fa00 thru 0xe457d8 (0x5dd9 bytes) 2024-09-16T20:48:13.631 INFO:teuthology.orchestra.run.smithi139.stdout:225 write 0x6d96200 thru 0x6d97d48 (0x1b49 bytes) 2024-09-16T20:48:13.638 INFO:teuthology.orchestra.run.smithi139.stdout:227 trunc from 0x6d97e00 to 0x11fb800 2024-09-16T20:48:13.664 INFO:teuthology.orchestra.run.smithi139.stdout:228 clone 2 order 22 su 262144 sc 5 2024-09-16T20:48:13.695 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:13.694+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:13.749 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:13.750 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:48:13.750 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:48:13.750 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:13.758 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:48:13.797 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:13.795+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:13.898 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:13.896+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:13.906 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:13.907 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:48:13.907 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:48:13.907 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:48:13.907 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:13.915 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:48:13.998 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:13.997+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:14.024 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:14.024 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:48:14.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:48:14.024 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:48:14.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.024 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:613 3.17 3:e8e2930f:::rbd_header.110e1748a8d2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-16T20:48:14.024 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:13.593102+0000", 2024-09-16T20:48:14.024 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.42905452500000002, 2024-09-16T20:48:14.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003632939, 2024-09-16T20:48:14.024 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:14.024 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:14.025 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:14.025 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:14.025 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:48:14.025 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:48:14.025 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 613 2024-09-16T20:48:14.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.025 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:14.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:14.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593102+0000", 2024-09-16T20:48:14.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:14.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:14.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593102+0000", 2024-09-16T20:48:14.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:14.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.026 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:14.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593103+0000", 2024-09-16T20:48:14.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5799999999999998e-07 2024-09-16T20:48:14.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.026 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:14.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593106+0000", 2024-09-16T20:48:14.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.022e-06 2024-09-16T20:48:14.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.027 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.027 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:14.027 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593113+0000", 2024-09-16T20:48:14.027 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1779999999999997e-06 2024-09-16T20:48:14.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.027 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.027 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:14.027 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593119+0000", 2024-09-16T20:48:14.027 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5679999999999999e-06 2024-09-16T20:48:14.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.027 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.027 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:14.027 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.596570+0000", 2024-09-16T20:48:14.027 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0034513120000000002 2024-09-16T20:48:14.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.028 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.028 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:14.028 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.596619+0000", 2024-09-16T20:48:14.028 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8310000000000003e-05 2024-09-16T20:48:14.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.028 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.028 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:14.028 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.596735+0000", 2024-09-16T20:48:14.028 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000116791 2024-09-16T20:48:14.028 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:14.029 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:14.029 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:14.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.029 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4462.0:2890 3.17 3:e8e2930f:::rbd_header.110e1748a8d2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-16T20:48:14.029 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:13.593235+0000", 2024-09-16T20:48:14.029 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.42892185799999999, 2024-09-16T20:48:14.029 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003641395, 2024-09-16T20:48:14.029 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:14.029 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:14.029 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:14.029 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:14.029 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4462", 2024-09-16T20:48:14.029 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:14.030 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 2890 2024-09-16T20:48:14.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.030 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:14.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.030 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:14.030 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593235+0000", 2024-09-16T20:48:14.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:14.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.030 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:14.030 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593235+0000", 2024-09-16T20:48:14.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:14.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.031 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.031 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:14.031 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593236+0000", 2024-09-16T20:48:14.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0270000000000001e-06 2024-09-16T20:48:14.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.031 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.031 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:14.031 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593240+0000", 2024-09-16T20:48:14.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.693e-06 2024-09-16T20:48:14.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.031 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.031 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:14.031 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593248+0000", 2024-09-16T20:48:14.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6899999999999998e-06 2024-09-16T20:48:14.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.032 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:14.032 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593255+0000", 2024-09-16T20:48:14.032 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0270000000000001e-06 2024-09-16T20:48:14.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.032 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:14.032 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.596770+0000", 2024-09-16T20:48:14.032 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035158619999999998 2024-09-16T20:48:14.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.032 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:14.032 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.596823+0000", 2024-09-16T20:48:14.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2444999999999997e-05 2024-09-16T20:48:14.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.033 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.033 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:14.033 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.596876+0000", 2024-09-16T20:48:14.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3650999999999998e-05 2024-09-16T20:48:14.033 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:14.033 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:14.033 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:14.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.033 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.033 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:614 3.17 3:e8e2930f:::rbd_header.110e1748a8d2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-16T20:48:14.033 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:13.593271+0000", 2024-09-16T20:48:14.033 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.42888545300000003, 2024-09-16T20:48:14.034 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0038034470000000002, 2024-09-16T20:48:14.034 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:14.034 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:14.034 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:14.034 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:14.034 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:48:14.034 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:48:14.034 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 614 2024-09-16T20:48:14.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.034 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:14.034 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.034 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:14.034 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593271+0000", 2024-09-16T20:48:14.034 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:14.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.035 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:14.035 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593271+0000", 2024-09-16T20:48:14.035 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:14.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.035 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:14.035 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593272+0000", 2024-09-16T20:48:14.035 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4000000000000002e-07 2024-09-16T20:48:14.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.035 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:14.035 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593276+0000", 2024-09-16T20:48:14.035 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1400000000000002e-06 2024-09-16T20:48:14.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.036 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:14.036 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593283+0000", 2024-09-16T20:48:14.036 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1430000000000001e-06 2024-09-16T20:48:14.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.036 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:14.036 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593289+0000", 2024-09-16T20:48:14.036 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3979999999999998e-06 2024-09-16T20:48:14.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.036 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:14.036 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.596911+0000", 2024-09-16T20:48:14.036 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003622428 2024-09-16T20:48:14.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.037 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.037 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:14.037 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.596973+0000", 2024-09-16T20:48:14.037 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1822000000000005e-05 2024-09-16T20:48:14.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.037 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.037 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:14.037 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.597075+0000", 2024-09-16T20:48:14.037 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101976 2024-09-16T20:48:14.037 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:14.037 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:14.037 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:14.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.037 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.038 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4462.0:2891 3.17 3:e8e2930f:::rbd_header.110e1748a8d2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-16T20:48:14.038 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:13.593449+0000", 2024-09-16T20:48:14.038 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.42870772400000001, 2024-09-16T20:48:14.038 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0038032500000000002, 2024-09-16T20:48:14.038 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:14.038 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:14.038 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:14.038 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:14.038 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4462", 2024-09-16T20:48:14.038 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:14.038 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 2891 2024-09-16T20:48:14.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.038 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:14.038 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.039 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:14.039 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593449+0000", 2024-09-16T20:48:14.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:14.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.039 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:14.039 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593449+0000", 2024-09-16T20:48:14.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:14.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.039 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:14.039 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593450+0000", 2024-09-16T20:48:14.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1299999999999999e-07 2024-09-16T20:48:14.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.040 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.040 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:14.040 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593456+0000", 2024-09-16T20:48:14.040 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1009999999999997e-06 2024-09-16T20:48:14.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.040 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.040 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:14.040 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593465+0000", 2024-09-16T20:48:14.040 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6989999999999992e-06 2024-09-16T20:48:14.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.040 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.040 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:14.040 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593471+0000", 2024-09-16T20:48:14.041 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9680000000000003e-06 2024-09-16T20:48:14.041 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.041 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.041 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:14.041 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.597111+0000", 2024-09-16T20:48:14.041 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0036397320000000001 2024-09-16T20:48:14.041 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.041 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.041 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:14.041 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.597182+0000", 2024-09-16T20:48:14.041 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1847999999999994e-05 2024-09-16T20:48:14.041 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.041 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.042 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:14.042 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.597252+0000", 2024-09-16T20:48:14.042 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0088999999999999e-05 2024-09-16T20:48:14.042 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:14.042 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:14.042 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:14.042 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.042 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.042 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:615 3.17 3:e8e2930f:::rbd_header.110e1748a8d2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-16T20:48:14.042 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:13.593488+0000", 2024-09-16T20:48:14.042 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.42866926399999999, 2024-09-16T20:48:14.042 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0039392940000000003, 2024-09-16T20:48:14.042 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:14.042 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:14.042 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:14.042 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:14.043 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:48:14.043 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:48:14.043 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 615 2024-09-16T20:48:14.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.043 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:14.043 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.043 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:14.043 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593488+0000", 2024-09-16T20:48:14.043 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:14.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.043 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.043 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:14.043 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593488+0000", 2024-09-16T20:48:14.043 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:14.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.044 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.044 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:14.044 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593488+0000", 2024-09-16T20:48:14.044 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3e-07 2024-09-16T20:48:14.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.044 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.044 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:14.044 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593491+0000", 2024-09-16T20:48:14.044 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9510000000000002e-06 2024-09-16T20:48:14.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.044 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.044 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:14.044 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593499+0000", 2024-09-16T20:48:14.044 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6000000000000001e-06 2024-09-16T20:48:14.045 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.045 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.045 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:14.045 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593506+0000", 2024-09-16T20:48:14.045 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.413e-06 2024-09-16T20:48:14.045 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.045 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.045 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:14.045 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.597298+0000", 2024-09-16T20:48:14.045 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0037918869999999999 2024-09-16T20:48:14.045 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.045 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.045 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:14.045 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.597359+0000", 2024-09-16T20:48:14.045 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0835999999999998e-05 2024-09-16T20:48:14.046 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.046 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.046 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:14.046 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.597427+0000", 2024-09-16T20:48:14.046 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7977000000000003e-05 2024-09-16T20:48:14.046 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:14.046 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:14.046 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:14.046 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.046 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.046 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4462.0:2892 3.17 3:e8e2930f:::rbd_header.110e1748a8d2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-16T20:48:14.046 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:13.593612+0000", 2024-09-16T20:48:14.046 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.42854537500000001, 2024-09-16T20:48:14.046 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0039343879999999996, 2024-09-16T20:48:14.046 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:14.047 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:14.047 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:14.047 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:14.047 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4462", 2024-09-16T20:48:14.047 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:14.047 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 2892 2024-09-16T20:48:14.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.047 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:14.047 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.047 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:14.047 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593612+0000", 2024-09-16T20:48:14.047 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:14.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.048 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.048 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:14.048 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593612+0000", 2024-09-16T20:48:14.048 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:14.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.048 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.048 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:14.048 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593612+0000", 2024-09-16T20:48:14.048 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5000000000000002e-07 2024-09-16T20:48:14.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.048 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.048 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:14.049 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593616+0000", 2024-09-16T20:48:14.049 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0049999999999998e-06 2024-09-16T20:48:14.049 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.049 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.049 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:14.049 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593625+0000", 2024-09-16T20:48:14.049 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2959999999999999e-06 2024-09-16T20:48:14.049 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.049 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.049 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:14.049 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593630+0000", 2024-09-16T20:48:14.049 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8379999999999998e-06 2024-09-16T20:48:14.049 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.049 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.050 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:14.050 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.597456+0000", 2024-09-16T20:48:14.050 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0038258699999999999 2024-09-16T20:48:14.050 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.050 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.050 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:14.050 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.597496+0000", 2024-09-16T20:48:14.050 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9711e-05 2024-09-16T20:48:14.050 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.050 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.050 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:14.050 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.597546+0000", 2024-09-16T20:48:14.050 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9917999999999999e-05 2024-09-16T20:48:14.050 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:14.051 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:14.051 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:14.051 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.051 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.051 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:616 3.17 3:e8e2930f:::rbd_header.110e1748a8d2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-16T20:48:14.051 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:13.593673+0000", 2024-09-16T20:48:14.051 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.428483797, 2024-09-16T20:48:14.051 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0039918269999999999, 2024-09-16T20:48:14.051 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:14.051 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:14.051 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:14.051 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:14.051 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:48:14.051 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:48:14.051 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 616 2024-09-16T20:48:14.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.052 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:14.052 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.052 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:14.052 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593673+0000", 2024-09-16T20:48:14.052 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:14.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.052 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.052 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:14.052 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593673+0000", 2024-09-16T20:48:14.052 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:14.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.053 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.053 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:14.053 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593674+0000", 2024-09-16T20:48:14.053 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9500000000000002e-07 2024-09-16T20:48:14.053 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.053 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.053 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:14.053 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593677+0000", 2024-09-16T20:48:14.053 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3340000000000002e-06 2024-09-16T20:48:14.053 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.053 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.053 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:14.054 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593686+0000", 2024-09-16T20:48:14.054 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3359999999999995e-06 2024-09-16T20:48:14.054 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.054 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.054 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:14.054 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593691+0000", 2024-09-16T20:48:14.054 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9200000000000001e-06 2024-09-16T20:48:14.054 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.054 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.054 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:14.054 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.597571+0000", 2024-09-16T20:48:14.054 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003880072 2024-09-16T20:48:14.054 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.055 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.055 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:14.055 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.597611+0000", 2024-09-16T20:48:14.055 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9616999999999997e-05 2024-09-16T20:48:14.055 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.055 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.055 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:14.055 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.597665+0000", 2024-09-16T20:48:14.055 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3853000000000001e-05 2024-09-16T20:48:14.055 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:14.056 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:14.056 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:14.056 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.056 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.056 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4462.0:2893 3.17 3:e8e2930f:::rbd_header.110e1748a8d2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-16T20:48:14.056 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:13.593735+0000", 2024-09-16T20:48:14.056 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.428422154, 2024-09-16T20:48:14.056 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0040387699999999997, 2024-09-16T20:48:14.056 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:14.056 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:14.056 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:14.057 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:14.057 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4462", 2024-09-16T20:48:14.057 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:14.057 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 2893 2024-09-16T20:48:14.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.057 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:14.057 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.057 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:14.057 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593735+0000", 2024-09-16T20:48:14.057 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:14.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.057 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.057 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:14.058 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593735+0000", 2024-09-16T20:48:14.058 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:14.058 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.058 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.058 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:14.058 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593736+0000", 2024-09-16T20:48:14.058 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4000000000000001e-07 2024-09-16T20:48:14.058 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.058 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.058 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:14.058 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593739+0000", 2024-09-16T20:48:14.058 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4690000000000002e-06 2024-09-16T20:48:14.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.059 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.059 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:14.059 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593748+0000", 2024-09-16T20:48:14.059 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9020000000000005e-06 2024-09-16T20:48:14.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.059 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.059 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:14.059 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593754+0000", 2024-09-16T20:48:14.059 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0650000000000004e-06 2024-09-16T20:48:14.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.059 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.060 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:14.060 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.597691+0000", 2024-09-16T20:48:14.060 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0039371800000000002 2024-09-16T20:48:14.060 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.060 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.060 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:14.060 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.597728+0000", 2024-09-16T20:48:14.060 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6498000000000002e-05 2024-09-16T20:48:14.060 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.060 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.060 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:14.060 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.597774+0000", 2024-09-16T20:48:14.061 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5915999999999999e-05 2024-09-16T20:48:14.061 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:14.061 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:14.061 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:14.061 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.061 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.061 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:617 3.17 3:e8e2930f:::rbd_header.110e1748a8d2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-16T20:48:14.061 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:13.593795+0000", 2024-09-16T20:48:14.061 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.42836207599999998, 2024-09-16T20:48:14.061 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0040911430000000002, 2024-09-16T20:48:14.061 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:14.061 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:14.061 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:14.062 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:14.062 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:48:14.062 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:48:14.062 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 617 2024-09-16T20:48:14.062 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.062 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:14.062 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.062 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:14.062 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593795+0000", 2024-09-16T20:48:14.062 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:14.062 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.062 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.062 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:14.063 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593795+0000", 2024-09-16T20:48:14.063 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:14.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.063 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.063 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:14.063 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593796+0000", 2024-09-16T20:48:14.063 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0299999999999998e-07 2024-09-16T20:48:14.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.063 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.063 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:14.063 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593799+0000", 2024-09-16T20:48:14.063 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5449999999999999e-06 2024-09-16T20:48:14.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.063 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.064 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:14.064 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593807+0000", 2024-09-16T20:48:14.064 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8120000000000008e-06 2024-09-16T20:48:14.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.064 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:14.064 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593813+0000", 2024-09-16T20:48:14.064 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6919999999999999e-06 2024-09-16T20:48:14.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.064 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:14.064 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.597796+0000", 2024-09-16T20:48:14.064 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0039835010000000004 2024-09-16T20:48:14.065 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.065 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.065 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:14.065 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.597834+0000", 2024-09-16T20:48:14.065 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7441000000000003e-05 2024-09-16T20:48:14.065 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.065 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.065 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:14.065 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.597886+0000", 2024-09-16T20:48:14.065 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2448999999999999e-05 2024-09-16T20:48:14.065 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:14.065 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:14.065 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:14.065 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.066 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.066 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4462.0:2894 3.17 3:e8e2930f:::rbd_header.110e1748a8d2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-16T20:48:14.066 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:13.593850+0000", 2024-09-16T20:48:14.066 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.42830668700000002, 2024-09-16T20:48:14.066 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0041412749999999998, 2024-09-16T20:48:14.066 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:14.066 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:14.066 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:14.066 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:14.066 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4462", 2024-09-16T20:48:14.066 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:14.066 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 2894 2024-09-16T20:48:14.066 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.066 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:14.067 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.067 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:14.067 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593850+0000", 2024-09-16T20:48:14.067 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:14.067 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.067 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.067 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:14.067 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593850+0000", 2024-09-16T20:48:14.067 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:14.067 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.067 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.068 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:14.068 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593851+0000", 2024-09-16T20:48:14.068 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7800000000000001e-07 2024-09-16T20:48:14.068 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.068 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.068 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:14.068 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593854+0000", 2024-09-16T20:48:14.068 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.083e-06 2024-09-16T20:48:14.068 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.068 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.068 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:14.068 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593862+0000", 2024-09-16T20:48:14.069 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9719999999999992e-06 2024-09-16T20:48:14.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.069 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.069 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:14.069 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593868+0000", 2024-09-16T20:48:14.069 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6690000000000003e-06 2024-09-16T20:48:14.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.069 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.069 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:14.069 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.597910+0000", 2024-09-16T20:48:14.069 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0040420339999999999 2024-09-16T20:48:14.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.069 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.070 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:14.070 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.597945+0000", 2024-09-16T20:48:14.070 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5046999999999998e-05 2024-09-16T20:48:14.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.070 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:14.070 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.597992+0000", 2024-09-16T20:48:14.070 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6891999999999998e-05 2024-09-16T20:48:14.070 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:14.070 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:14.070 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:14.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.070 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:618 3.17 3:e8e2930f:::rbd_header.110e1748a8d2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-16T20:48:14.071 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:13.593904+0000", 2024-09-16T20:48:14.071 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.42825337499999999, 2024-09-16T20:48:14.071 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0042103269999999998, 2024-09-16T20:48:14.071 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:14.071 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:14.071 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:14.071 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:14.071 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:48:14.071 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:48:14.071 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 618 2024-09-16T20:48:14.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.072 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:14.072 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.072 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:14.072 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593904+0000", 2024-09-16T20:48:14.072 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:14.072 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.072 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.072 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:14.072 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593904+0000", 2024-09-16T20:48:14.072 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:14.072 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.072 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.072 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:14.073 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593904+0000", 2024-09-16T20:48:14.073 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5000000000000002e-07 2024-09-16T20:48:14.073 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.073 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.073 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:14.073 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593907+0000", 2024-09-16T20:48:14.073 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0589999999999998e-06 2024-09-16T20:48:14.073 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.073 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.073 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:14.073 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593915+0000", 2024-09-16T20:48:14.073 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1699999999999997e-06 2024-09-16T20:48:14.073 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.074 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.074 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:14.074 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.593922+0000", 2024-09-16T20:48:14.074 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7610000000000002e-06 2024-09-16T20:48:14.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.074 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.074 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:14.074 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.598021+0000", 2024-09-16T20:48:14.074 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0040984230000000003 2024-09-16T20:48:14.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.074 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.074 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:14.074 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.598058+0000", 2024-09-16T20:48:14.075 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6989000000000001e-05 2024-09-16T20:48:14.075 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.075 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.075 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:14.075 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.598114+0000", 2024-09-16T20:48:14.075 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6274999999999999e-05 2024-09-16T20:48:14.075 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:14.075 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:14.075 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:14.075 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.075 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.075 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4462.0:2896 3.17 3:e8e2930f:::rbd_header.110e1748a8d2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-16T20:48:14.075 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:13.594357+0000", 2024-09-16T20:48:14.075 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.42779982, 2024-09-16T20:48:14.076 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0038540509999999998, 2024-09-16T20:48:14.076 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:14.076 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:14.076 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:14.076 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:14.076 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4462", 2024-09-16T20:48:14.076 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:14.076 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 2896 2024-09-16T20:48:14.076 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.076 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:14.076 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.076 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:14.077 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.594357+0000", 2024-09-16T20:48:14.077 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:14.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.077 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.077 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:14.077 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.594357+0000", 2024-09-16T20:48:14.077 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:14.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.077 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.077 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:14.077 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.594358+0000", 2024-09-16T20:48:14.077 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1699999999999997e-07 2024-09-16T20:48:14.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.077 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.078 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:14.078 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.594363+0000", 2024-09-16T20:48:14.078 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7040000000000002e-06 2024-09-16T20:48:14.078 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.078 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.078 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:14.078 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.594372+0000", 2024-09-16T20:48:14.078 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.02e-06 2024-09-16T20:48:14.078 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.078 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.078 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:14.078 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.594378+0000", 2024-09-16T20:48:14.078 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2650000000000002e-06 2024-09-16T20:48:14.079 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.079 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.079 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:14.079 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.598130+0000", 2024-09-16T20:48:14.079 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0037524210000000001 2024-09-16T20:48:14.079 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.079 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.079 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:14.079 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.598166+0000", 2024-09-16T20:48:14.079 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5178000000000001e-05 2024-09-16T20:48:14.079 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.079 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.079 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:14.079 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.598211+0000", 2024-09-16T20:48:14.080 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5546000000000001e-05 2024-09-16T20:48:14.080 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:14.080 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:14.080 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:14.080 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.080 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.080 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:619 3.17 3:e8e2930f:::rbd_header.110e1748a8d2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-16T20:48:14.080 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:13.594396+0000", 2024-09-16T20:48:14.080 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.42776102500000002, 2024-09-16T20:48:14.080 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0039273829999999996, 2024-09-16T20:48:14.080 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:14.081 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:14.081 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:14.081 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:14.081 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:48:14.081 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:48:14.081 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 619 2024-09-16T20:48:14.081 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.081 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:14.081 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.081 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:14.081 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.594396+0000", 2024-09-16T20:48:14.081 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:14.081 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.081 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.082 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:14.082 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.594396+0000", 2024-09-16T20:48:14.082 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:14.082 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.082 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.082 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:14.082 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.594397+0000", 2024-09-16T20:48:14.082 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8999999999999996e-07 2024-09-16T20:48:14.082 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.082 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.082 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:14.082 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.594400+0000", 2024-09-16T20:48:14.082 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1820000000000002e-06 2024-09-16T20:48:14.083 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.083 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.083 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:14.083 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.594408+0000", 2024-09-16T20:48:14.083 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2819999999999996e-06 2024-09-16T20:48:14.083 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.083 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.083 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:14.083 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.594414+0000", 2024-09-16T20:48:14.083 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7690000000000002e-06 2024-09-16T20:48:14.083 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.083 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.084 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:14.084 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.598227+0000", 2024-09-16T20:48:14.084 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0038128659999999998 2024-09-16T20:48:14.084 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.084 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.084 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:14.084 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.598261+0000", 2024-09-16T20:48:14.084 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4066999999999998e-05 2024-09-16T20:48:14.084 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.084 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.084 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:14.084 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.598323+0000", 2024-09-16T20:48:14.084 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2626999999999995e-05 2024-09-16T20:48:14.084 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:14.085 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:14.085 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:14.085 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.085 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.085 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:620 3.17 3:e8e2930f:::rbd_header.110e1748a8d2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-16T20:48:14.085 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:13.594970+0000", 2024-09-16T20:48:14.085 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.42718684000000001, 2024-09-16T20:48:14.085 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035713630000000001, 2024-09-16T20:48:14.085 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:14.085 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:14.085 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:14.085 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:14.085 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:48:14.085 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:48:14.086 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 620 2024-09-16T20:48:14.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.086 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:14.086 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.086 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:14.086 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.594970+0000", 2024-09-16T20:48:14.086 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:14.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.086 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.086 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:14.086 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.594970+0000", 2024-09-16T20:48:14.086 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:14.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.086 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.087 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:14.087 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.594971+0000", 2024-09-16T20:48:14.087 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9699999999999996e-07 2024-09-16T20:48:14.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.087 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.087 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:14.087 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.594973+0000", 2024-09-16T20:48:14.087 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6840000000000001e-06 2024-09-16T20:48:14.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.087 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.087 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:14.087 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.594981+0000", 2024-09-16T20:48:14.087 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2130000000000002e-06 2024-09-16T20:48:14.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.087 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.088 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:14.088 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.594986+0000", 2024-09-16T20:48:14.088 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8200000000000002e-06 2024-09-16T20:48:14.088 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.088 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.088 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:14.088 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.598453+0000", 2024-09-16T20:48:14.088 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0034666330000000002 2024-09-16T20:48:14.088 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.088 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.088 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:14.088 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.598488+0000", 2024-09-16T20:48:14.089 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4879999999999998e-05 2024-09-16T20:48:14.089 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.089 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.089 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:14.089 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.598541+0000", 2024-09-16T20:48:14.089 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3535999999999997e-05 2024-09-16T20:48:14.089 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:14.089 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:14.089 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:14.089 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.089 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.089 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:622 3.17 3:e8e2930f:::rbd_header.110e1748a8d2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-16T20:48:14.089 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:13.595366+0000", 2024-09-16T20:48:14.090 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.42679082699999998, 2024-09-16T20:48:14.090 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0036079530000000001, 2024-09-16T20:48:14.090 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:14.090 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:14.090 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:14.090 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:14.090 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:48:14.090 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:48:14.090 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 622 2024-09-16T20:48:14.090 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.090 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:14.090 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.090 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:14.090 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.595366+0000", 2024-09-16T20:48:14.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:14.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:14.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.595366+0000", 2024-09-16T20:48:14.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:14.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:14.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.595367+0000", 2024-09-16T20:48:14.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5499999999999998e-07 2024-09-16T20:48:14.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.092 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:14.092 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.595370+0000", 2024-09-16T20:48:14.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0299999999999998e-06 2024-09-16T20:48:14.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.092 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.092 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:14.092 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.595376+0000", 2024-09-16T20:48:14.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.105e-06 2024-09-16T20:48:14.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.092 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.092 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:14.092 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.595379+0000", 2024-09-16T20:48:14.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3730000000000001e-06 2024-09-16T20:48:14.093 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.093 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.093 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:14.093 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.598889+0000", 2024-09-16T20:48:14.093 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003509528 2024-09-16T20:48:14.093 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.093 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.093 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:14.093 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.598922+0000", 2024-09-16T20:48:14.093 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3417999999999998e-05 2024-09-16T20:48:14.093 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.093 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.093 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:14.094 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.598974+0000", 2024-09-16T20:48:14.094 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1844e-05 2024-09-16T20:48:14.094 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:14.094 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:14.094 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:14.094 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.094 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.094 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4462.0:2902 3.1f 3:f84ca2b1:::rbd_data.110e1748a8d2.0000000000000011:head [truncate 1638400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-16T20:48:14.094 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:13.595448+0000", 2024-09-16T20:48:14.094 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.426708955, 2024-09-16T20:48:14.094 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0042019040000000002, 2024-09-16T20:48:14.094 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:14.094 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:14.094 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:14.095 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:14.095 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4462", 2024-09-16T20:48:14.095 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:14.095 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 2902 2024-09-16T20:48:14.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.095 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:14.095 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:14.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.595448+0000", 2024-09-16T20:48:14.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:14.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.095 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:14.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.595448+0000", 2024-09-16T20:48:14.096 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:14.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.096 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.096 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:14.096 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.595449+0000", 2024-09-16T20:48:14.096 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2300000000000001e-07 2024-09-16T20:48:14.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.096 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.096 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:14.096 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.595452+0000", 2024-09-16T20:48:14.096 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9359999999999999e-06 2024-09-16T20:48:14.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.096 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.096 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:14.097 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.595458+0000", 2024-09-16T20:48:14.097 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.229e-06 2024-09-16T20:48:14.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.097 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.097 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:14.097 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.595462+0000", 2024-09-16T20:48:14.097 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3549999999999998e-06 2024-09-16T20:48:14.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.097 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.097 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:14.097 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.598879+0000", 2024-09-16T20:48:14.097 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0034164680000000002 2024-09-16T20:48:14.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.097 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.098 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:14.098 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.598916+0000", 2024-09-16T20:48:14.098 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7132999999999998e-05 2024-09-16T20:48:14.098 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.098 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.098 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1", 2024-09-16T20:48:14.098 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.598984+0000", 2024-09-16T20:48:14.098 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7874999999999996e-05 2024-09-16T20:48:14.098 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.098 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.098 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:48:14.098 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.599577+0000", 2024-09-16T20:48:14.098 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00059387600000000002 2024-09-16T20:48:14.098 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.099 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.099 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:14.099 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.599606+0000", 2024-09-16T20:48:14.099 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8345999999999999e-05 2024-09-16T20:48:14.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.099 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.099 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:48:14.099 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.599631+0000", 2024-09-16T20:48:14.099 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5437999999999999e-05 2024-09-16T20:48:14.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.099 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.099 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:14.099 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.599650+0000", 2024-09-16T20:48:14.099 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8624999999999999e-05 2024-09-16T20:48:14.100 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:14.100 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:14.100 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:14.100 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.100 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.100 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:623 3.17 3:e8e2930f:::rbd_header.110e1748a8d2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-16T20:48:14.100 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:13.595610+0000", 2024-09-16T20:48:14.100 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.42654693300000002, 2024-09-16T20:48:14.100 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003595422, 2024-09-16T20:48:14.100 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:14.100 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:14.100 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:14.100 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:14.101 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:48:14.101 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:48:14.101 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 623 2024-09-16T20:48:14.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.101 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:14.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.101 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:14.101 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.595610+0000", 2024-09-16T20:48:14.101 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:14.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.101 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:14.101 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.595610+0000", 2024-09-16T20:48:14.102 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:14.102 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.102 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.102 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:14.102 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.595611+0000", 2024-09-16T20:48:14.102 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.75e-07 2024-09-16T20:48:14.102 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.102 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.102 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:14.102 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.595614+0000", 2024-09-16T20:48:14.102 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0690000000000002e-06 2024-09-16T20:48:14.102 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.102 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.102 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:14.103 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.595621+0000", 2024-09-16T20:48:14.103 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.08e-06 2024-09-16T20:48:14.103 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.103 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.103 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:14.103 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.595626+0000", 2024-09-16T20:48:14.103 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5809999999999996e-06 2024-09-16T20:48:14.103 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.103 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.103 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:14.103 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.599089+0000", 2024-09-16T20:48:14.103 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0034623319999999998 2024-09-16T20:48:14.103 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.104 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.104 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:14.104 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.599148+0000", 2024-09-16T20:48:14.104 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9827999999999997e-05 2024-09-16T20:48:14.104 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.104 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.104 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:14.104 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.599205+0000", 2024-09-16T20:48:14.104 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6956999999999997e-05 2024-09-16T20:48:14.104 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:14.104 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:14.104 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:14.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.105 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4462.0:2910 3.17 3:e8e2930f:::rbd_header.110e1748a8d2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-16T20:48:14.105 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:13.595749+0000", 2024-09-16T20:48:14.105 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.42640760700000002, 2024-09-16T20:48:14.105 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035841890000000002, 2024-09-16T20:48:14.105 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:14.105 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:14.105 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:14.105 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:14.105 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4462", 2024-09-16T20:48:14.105 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:14.105 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 2910 2024-09-16T20:48:14.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.106 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:14.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.106 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:14.106 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.595749+0000", 2024-09-16T20:48:14.106 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:14.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.106 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:14.106 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.595749+0000", 2024-09-16T20:48:14.106 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:14.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.107 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:14.107 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.595750+0000", 2024-09-16T20:48:14.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2099999999999996e-07 2024-09-16T20:48:14.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.107 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:14.107 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.595754+0000", 2024-09-16T20:48:14.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5140000000000002e-06 2024-09-16T20:48:14.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.107 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:14.107 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.595760+0000", 2024-09-16T20:48:14.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6640000000000001e-06 2024-09-16T20:48:14.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:14.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.595769+0000", 2024-09-16T20:48:14.108 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4579999999999993e-06 2024-09-16T20:48:14.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:14.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.599231+0000", 2024-09-16T20:48:14.108 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003462732 2024-09-16T20:48:14.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:14.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.599273+0000", 2024-09-16T20:48:14.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1415999999999998e-05 2024-09-16T20:48:14.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.109 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.109 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:14.109 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.599334+0000", 2024-09-16T20:48:14.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0683999999999997e-05 2024-09-16T20:48:14.109 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:14.109 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:14.109 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:14.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.109 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.109 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:624 3.17 3:e8e2930f:::rbd_header.110e1748a8d2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-16T20:48:14.109 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:13.595782+0000", 2024-09-16T20:48:14.109 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.42637499099999998, 2024-09-16T20:48:14.110 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0036672179999999999, 2024-09-16T20:48:14.110 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:14.110 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:14.110 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:14.110 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:14.110 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:48:14.110 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:48:14.110 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 624 2024-09-16T20:48:14.110 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.110 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:14.110 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.110 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:14.110 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.595782+0000", 2024-09-16T20:48:14.110 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:14.110 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.110 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.111 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:14.111 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.595782+0000", 2024-09-16T20:48:14.111 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:14.111 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.111 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.111 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:14.111 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.595782+0000", 2024-09-16T20:48:14.111 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.72e-07 2024-09-16T20:48:14.111 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.111 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.111 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:14.111 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.595784+0000", 2024-09-16T20:48:14.111 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.057e-06 2024-09-16T20:48:14.111 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.112 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:14.112 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.595789+0000", 2024-09-16T20:48:14.112 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3699999999999997e-06 2024-09-16T20:48:14.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.112 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:14.112 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.595793+0000", 2024-09-16T20:48:14.112 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3050000000000003e-06 2024-09-16T20:48:14.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.112 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:14.112 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.599360+0000", 2024-09-16T20:48:14.112 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035670060000000002 2024-09-16T20:48:14.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.113 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.113 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:14.113 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.599396+0000", 2024-09-16T20:48:14.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5568999999999998e-05 2024-09-16T20:48:14.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.113 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.113 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:14.113 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.599449+0000", 2024-09-16T20:48:14.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3538999999999997e-05 2024-09-16T20:48:14.113 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:14.113 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:14.113 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:14.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.113 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.114 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:625 3.17 3:e8e2930f:::rbd_header.110e1748a8d2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-16T20:48:14.114 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:13.596051+0000", 2024-09-16T20:48:14.114 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.42610561899999999, 2024-09-16T20:48:14.114 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003637239, 2024-09-16T20:48:14.114 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:14.114 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:14.114 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:14.114 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:14.114 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:48:14.114 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:48:14.114 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 625 2024-09-16T20:48:14.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.114 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:14.114 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.114 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:14.115 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.596051+0000", 2024-09-16T20:48:14.115 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:14.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.115 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.115 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:14.115 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.596051+0000", 2024-09-16T20:48:14.115 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:14.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.115 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.115 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:14.115 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.596052+0000", 2024-09-16T20:48:14.115 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7899999999999998e-07 2024-09-16T20:48:14.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.116 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.116 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:14.116 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.596057+0000", 2024-09-16T20:48:14.116 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7400000000000004e-06 2024-09-16T20:48:14.116 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.116 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.116 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:14.116 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.596064+0000", 2024-09-16T20:48:14.116 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.588e-06 2024-09-16T20:48:14.116 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.116 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.116 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:14.116 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.596072+0000", 2024-09-16T20:48:14.117 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1810000000000004e-06 2024-09-16T20:48:14.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.117 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:14.117 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.599588+0000", 2024-09-16T20:48:14.117 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035165610000000001 2024-09-16T20:48:14.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.117 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:14.117 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.599634+0000", 2024-09-16T20:48:14.117 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5750000000000001e-05 2024-09-16T20:48:14.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:14.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:14.118 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:14.118 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:13.599689+0000", 2024-09-16T20:48:14.118 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4740000000000001e-05 2024-09-16T20:48:14.118 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:14.118 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:14.118 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:14.118 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:14.118 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:14.118 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:14.119 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:14.118+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:14.120 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:48:14.220 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:14.219+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:14.232 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:14.232 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:48:14.232 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:48:14.232 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:14.241 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:48:14.325 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:14.324+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:14.388 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:14.388 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:48:14.388 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:48:14.388 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:48:14.388 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:14.397 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:48:14.438 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:14.437+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:14.523 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:14.522+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:14.547 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:14.547 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:48:14.547 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:48:14.547 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:48:14.547 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.547 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4245.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-16T20:48:14.548 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:59.273312+0000", 2024-09-16T20:48:14.548 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.271641331, 2024-09-16T20:48:14.548 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0036239010000000001, 2024-09-16T20:48:14.548 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:14.548 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:14.548 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:14.548 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:14.548 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4245", 2024-09-16T20:48:14.548 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.29:0/1819464193", 2024-09-16T20:48:14.548 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5 2024-09-16T20:48:14.548 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.549 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:14.549 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.549 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:14.549 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:59.273312+0000", 2024-09-16T20:48:14.549 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:14.549 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.549 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.549 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:14.549 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:59.273312+0000", 2024-09-16T20:48:14.549 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:14.549 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.549 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.549 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:14.550 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:59.273315+0000", 2024-09-16T20:48:14.550 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.4229999999999999e-06 2024-09-16T20:48:14.550 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.550 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.550 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:14.550 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:59.273328+0000", 2024-09-16T20:48:14.550 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3689999999999999e-05 2024-09-16T20:48:14.550 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.550 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.550 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:14.550 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:59.273350+0000", 2024-09-16T20:48:14.551 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.145e-05 2024-09-16T20:48:14.551 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.551 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.551 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:14.551 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:59.273364+0000", 2024-09-16T20:48:14.551 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.432e-05 2024-09-16T20:48:14.551 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.551 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.551 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:14.551 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:59.273439+0000", 2024-09-16T20:48:14.551 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5098999999999994e-05 2024-09-16T20:48:14.551 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.552 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.552 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op must wait for map", 2024-09-16T20:48:14.552 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:59.273444+0000", 2024-09-16T20:48:14.552 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.22e-06 2024-09-16T20:48:14.552 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.552 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.552 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:14.552 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:59.275086+0000", 2024-09-16T20:48:14.552 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0016413669999999999 2024-09-16T20:48:14.552 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.552 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.552 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:14.552 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:59.275213+0000", 2024-09-16T20:48:14.553 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00012698700000000001 2024-09-16T20:48:14.553 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.553 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.553 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 4", 2024-09-16T20:48:14.553 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:59.275341+0000", 2024-09-16T20:48:14.553 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00012876200000000001 2024-09-16T20:48:14.553 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.553 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.553 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:14.553 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:59.276119+0000", 2024-09-16T20:48:14.553 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00077722800000000003 2024-09-16T20:48:14.553 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.554 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.554 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:14.554 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:59.276868+0000", 2024-09-16T20:48:14.554 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00074979199999999999 2024-09-16T20:48:14.554 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.554 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.554 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:14.554 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:59.276906+0000", 2024-09-16T20:48:14.554 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7279e-05 2024-09-16T20:48:14.555 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.555 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.555 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:14.555 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:59.276936+0000", 2024-09-16T20:48:14.555 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0284e-05 2024-09-16T20:48:14.555 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:14.555 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:14.555 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:14.555 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.555 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.555 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4245.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-16T20:48:14.555 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:00.277778+0000", 2024-09-16T20:48:14.556 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 14.267175044, 2024-09-16T20:48:14.556 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0017304390000000001, 2024-09-16T20:48:14.556 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:14.556 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:14.556 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:14.556 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:14.556 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4245", 2024-09-16T20:48:14.556 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.29:0/1819464193", 2024-09-16T20:48:14.556 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 7 2024-09-16T20:48:14.556 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.556 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:14.556 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.556 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:14.557 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:00.277778+0000", 2024-09-16T20:48:14.557 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:14.557 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.557 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.557 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:14.557 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:00.277778+0000", 2024-09-16T20:48:14.557 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:14.557 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.557 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.557 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:14.557 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:00.277781+0000", 2024-09-16T20:48:14.557 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3479999999999998e-06 2024-09-16T20:48:14.557 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.558 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.558 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:14.558 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:00.277793+0000", 2024-09-16T20:48:14.558 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.239e-05 2024-09-16T20:48:14.558 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.558 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.558 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:14.558 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:00.277810+0000", 2024-09-16T20:48:14.558 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.6402e-05 2024-09-16T20:48:14.558 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.558 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.558 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:14.558 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:00.277822+0000", 2024-09-16T20:48:14.558 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2682e-05 2024-09-16T20:48:14.558 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.559 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.559 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:14.559 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:00.277896+0000", 2024-09-16T20:48:14.559 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3289999999999998e-05 2024-09-16T20:48:14.559 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.559 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.559 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:14.559 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:00.278094+0000", 2024-09-16T20:48:14.559 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019882499999999999 2024-09-16T20:48:14.559 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.559 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.559 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 4", 2024-09-16T20:48:14.559 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:00.278317+0000", 2024-09-16T20:48:14.560 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00022229300000000001 2024-09-16T20:48:14.560 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.560 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.560 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:14.560 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:00.279087+0000", 2024-09-16T20:48:14.560 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00077042300000000001 2024-09-16T20:48:14.560 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.560 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.560 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:14.560 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:00.279435+0000", 2024-09-16T20:48:14.560 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00034811400000000001 2024-09-16T20:48:14.560 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.560 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.560 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:14.561 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:00.279475+0000", 2024-09-16T20:48:14.561 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9981999999999998e-05 2024-09-16T20:48:14.561 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.561 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.561 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:14.561 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:00.279509+0000", 2024-09-16T20:48:14.561 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3689999999999998e-05 2024-09-16T20:48:14.561 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:14.561 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:14.561 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:14.561 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.561 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.561 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4462.0:1929 3.9 3:9289e717:::rbd_data.110e1748a8d2.0000000000000064:head [set-alloc-hint object_size 2097152 write_size 2097152,write 52224~13312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T20:48:14.562 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:13.300652+0000", 2024-09-16T20:48:14.562 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 1.244301777, 2024-09-16T20:48:14.562 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001563114, 2024-09-16T20:48:14.562 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:14.562 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:14.562 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:14.562 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:14.562 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4462", 2024-09-16T20:48:14.562 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:14.562 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 1929 2024-09-16T20:48:14.562 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.562 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:14.562 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.562 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:14.562 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.300652+0000", 2024-09-16T20:48:14.563 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:14.563 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.563 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.563 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:14.563 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.300652+0000", 2024-09-16T20:48:14.563 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:14.563 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.563 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.563 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:14.563 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.300655+0000", 2024-09-16T20:48:14.563 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4290000000000001e-06 2024-09-16T20:48:14.563 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.563 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.564 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:14.564 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.300669+0000", 2024-09-16T20:48:14.564 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4083999999999999e-05 2024-09-16T20:48:14.564 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.564 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.564 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:14.564 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.300682+0000", 2024-09-16T20:48:14.564 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.325e-05 2024-09-16T20:48:14.564 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.564 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.564 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:14.564 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.300690+0000", 2024-09-16T20:48:14.564 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7139999999999993e-06 2024-09-16T20:48:14.565 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.565 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.565 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:14.565 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.300748+0000", 2024-09-16T20:48:14.565 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7333000000000001e-05 2024-09-16T20:48:14.565 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.565 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.565 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:14.565 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.300889+0000", 2024-09-16T20:48:14.565 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014099699999999999 2024-09-16T20:48:14.565 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.565 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.565 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:48:14.565 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.301035+0000", 2024-09-16T20:48:14.566 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014627999999999999 2024-09-16T20:48:14.566 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.566 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.566 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:14.566 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.301830+0000", 2024-09-16T20:48:14.566 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00079474799999999996 2024-09-16T20:48:14.566 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.566 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.566 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:14.566 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.302143+0000", 2024-09-16T20:48:14.566 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00031337200000000002 2024-09-16T20:48:14.566 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.566 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.566 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:14.566 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.302177+0000", 2024-09-16T20:48:14.567 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4251999999999997e-05 2024-09-16T20:48:14.567 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.567 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.567 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:14.567 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.302215+0000", 2024-09-16T20:48:14.567 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7654999999999998e-05 2024-09-16T20:48:14.567 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:14.567 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:14.567 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:14.567 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.567 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.567 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4462.0:1981 3.3 3:c740a9af:::rbd_data.110e1748a8d2.0000000000000021:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T20:48:14.567 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:13.316023+0000", 2024-09-16T20:48:14.567 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 1.2289302339999999, 2024-09-16T20:48:14.568 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0018032910000000001, 2024-09-16T20:48:14.568 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:14.568 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:14.568 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:14.568 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:14.568 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4462", 2024-09-16T20:48:14.568 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:14.568 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 1981 2024-09-16T20:48:14.568 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.568 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:14.568 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.568 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:14.568 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.316023+0000", 2024-09-16T20:48:14.569 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:14.569 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.569 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.569 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:14.569 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.316023+0000", 2024-09-16T20:48:14.569 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:14.569 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.569 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.569 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:14.569 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.316024+0000", 2024-09-16T20:48:14.569 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9299999999999997e-07 2024-09-16T20:48:14.569 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.569 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.570 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:14.570 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.316028+0000", 2024-09-16T20:48:14.570 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8099999999999999e-06 2024-09-16T20:48:14.570 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.570 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.570 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:14.570 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.316038+0000", 2024-09-16T20:48:14.570 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0346999999999999e-05 2024-09-16T20:48:14.570 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.570 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.570 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:14.570 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.316048+0000", 2024-09-16T20:48:14.570 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0189e-05 2024-09-16T20:48:14.570 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.570 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.571 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:14.571 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.316120+0000", 2024-09-16T20:48:14.571 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1373999999999998e-05 2024-09-16T20:48:14.571 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.571 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.571 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:14.571 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.316200+0000", 2024-09-16T20:48:14.571 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0743999999999998e-05 2024-09-16T20:48:14.571 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.571 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.571 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 2", 2024-09-16T20:48:14.571 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.316306+0000", 2024-09-16T20:48:14.571 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010519299999999999 2024-09-16T20:48:14.571 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.572 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.572 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:14.572 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.317404+0000", 2024-09-16T20:48:14.572 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001098562 2024-09-16T20:48:14.572 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.572 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.572 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:14.572 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.317742+0000", 2024-09-16T20:48:14.572 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00033751499999999998 2024-09-16T20:48:14.572 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.572 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.572 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:14.572 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.317790+0000", 2024-09-16T20:48:14.572 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8310000000000003e-05 2024-09-16T20:48:14.573 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.573 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.573 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:14.573 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.317827+0000", 2024-09-16T20:48:14.573 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6454e-05 2024-09-16T20:48:14.573 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:14.573 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:14.573 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:14.573 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.573 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.573 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4462.0:1989 3.9 3:9289e717:::rbd_data.110e1748a8d2.0000000000000064:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T20:48:14.573 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:13.316104+0000", 2024-09-16T20:48:14.574 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 1.2288499210000001, 2024-09-16T20:48:14.574 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0020540060000000001, 2024-09-16T20:48:14.574 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:14.574 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:14.574 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:14.574 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:14.574 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4462", 2024-09-16T20:48:14.574 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:14.574 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 1989 2024-09-16T20:48:14.574 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.574 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:14.574 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.574 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:14.575 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.316104+0000", 2024-09-16T20:48:14.575 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:14.575 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.575 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.575 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:14.575 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.316104+0000", 2024-09-16T20:48:14.575 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:14.575 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.575 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.575 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:14.575 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.316104+0000", 2024-09-16T20:48:14.575 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2399999999999997e-07 2024-09-16T20:48:14.575 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.575 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.576 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:14.576 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.316109+0000", 2024-09-16T20:48:14.576 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2719999999999999e-06 2024-09-16T20:48:14.576 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.576 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.576 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:14.576 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.316118+0000", 2024-09-16T20:48:14.576 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.1689999999999998e-06 2024-09-16T20:48:14.576 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.576 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.576 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:14.576 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.316125+0000", 2024-09-16T20:48:14.576 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8310000000000003e-06 2024-09-16T20:48:14.576 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.577 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.577 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:14.577 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.316182+0000", 2024-09-16T20:48:14.577 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6870999999999998e-05 2024-09-16T20:48:14.577 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.577 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.577 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:14.577 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.316242+0000", 2024-09-16T20:48:14.577 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0718999999999999e-05 2024-09-16T20:48:14.577 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.577 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.577 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:48:14.577 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.316345+0000", 2024-09-16T20:48:14.577 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102995 2024-09-16T20:48:14.578 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.578 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.578 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:14.578 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.317032+0000", 2024-09-16T20:48:14.578 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000686924 2024-09-16T20:48:14.578 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.578 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.578 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:14.578 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.318093+0000", 2024-09-16T20:48:14.578 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0010610229999999999 2024-09-16T20:48:14.578 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.578 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.578 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:14.578 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.318126+0000", 2024-09-16T20:48:14.579 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2425999999999998e-05 2024-09-16T20:48:14.579 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.579 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.579 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:14.579 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.318158+0000", 2024-09-16T20:48:14.579 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1952000000000002e-05 2024-09-16T20:48:14.579 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:14.579 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:14.579 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:14.579 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.579 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.579 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4462.0:2008 3.f 3:f4cae27b:::rbd_data.110e1748a8d2.0000000000000006:head [stat,truncate 393216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T20:48:14.579 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:13.321026+0000", 2024-09-16T20:48:14.579 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 1.223927137, 2024-09-16T20:48:14.580 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0016871239999999999, 2024-09-16T20:48:14.580 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:14.580 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:14.580 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:14.580 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:14.580 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4462", 2024-09-16T20:48:14.580 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:14.580 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2008 2024-09-16T20:48:14.580 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.580 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:14.580 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.580 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:14.580 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.321026+0000", 2024-09-16T20:48:14.580 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:14.580 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.581 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.581 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:14.581 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.321026+0000", 2024-09-16T20:48:14.581 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:14.581 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.581 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.581 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:14.581 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.321027+0000", 2024-09-16T20:48:14.581 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0900000000000002e-07 2024-09-16T20:48:14.581 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.581 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.581 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:14.581 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.321030+0000", 2024-09-16T20:48:14.581 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9720000000000001e-06 2024-09-16T20:48:14.582 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.582 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.582 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:14.582 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.321044+0000", 2024-09-16T20:48:14.582 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4038e-05 2024-09-16T20:48:14.582 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.582 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.582 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:14.582 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.321054+0000", 2024-09-16T20:48:14.582 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0219e-05 2024-09-16T20:48:14.582 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.582 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.582 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:14.582 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.321128+0000", 2024-09-16T20:48:14.583 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3506999999999999e-05 2024-09-16T20:48:14.583 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.583 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.583 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:14.583 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.321207+0000", 2024-09-16T20:48:14.583 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9223999999999999e-05 2024-09-16T20:48:14.583 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.583 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.583 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 1", 2024-09-16T20:48:14.583 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.321323+0000", 2024-09-16T20:48:14.583 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00011612600000000001 2024-09-16T20:48:14.583 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.584 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.584 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:14.584 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.322239+0000", 2024-09-16T20:48:14.584 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00091620600000000003 2024-09-16T20:48:14.584 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.584 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.584 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:14.584 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.322632+0000", 2024-09-16T20:48:14.584 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00039289899999999999 2024-09-16T20:48:14.584 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.584 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.584 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:14.584 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.322687+0000", 2024-09-16T20:48:14.585 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5272999999999997e-05 2024-09-16T20:48:14.585 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.585 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.585 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:14.585 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.322713+0000", 2024-09-16T20:48:14.585 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6151000000000001e-05 2024-09-16T20:48:14.585 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:14.585 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:14.585 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:14.585 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.585 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.585 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4462.0:2384 3.15 3:ac162bd2:::rbd_data.110e1748a8d2.000000000000001c:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T20:48:14.585 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:13.446985+0000", 2024-09-16T20:48:14.586 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 1.097968418, 2024-09-16T20:48:14.586 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001700022, 2024-09-16T20:48:14.586 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:14.586 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:14.586 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:14.586 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:14.586 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4462", 2024-09-16T20:48:14.586 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:14.586 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2384 2024-09-16T20:48:14.586 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.586 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:14.586 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.586 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:14.587 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.446985+0000", 2024-09-16T20:48:14.587 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:14.587 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.587 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.587 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:14.587 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.446985+0000", 2024-09-16T20:48:14.587 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:14.587 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.587 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.587 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:14.587 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.446986+0000", 2024-09-16T20:48:14.587 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.47e-07 2024-09-16T20:48:14.587 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.588 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.588 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:14.588 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.446991+0000", 2024-09-16T20:48:14.588 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.575e-06 2024-09-16T20:48:14.588 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.588 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.588 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:14.588 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.447002+0000", 2024-09-16T20:48:14.588 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0234000000000001e-05 2024-09-16T20:48:14.588 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.588 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.588 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:14.588 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.447007+0000", 2024-09-16T20:48:14.589 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0050000000000004e-06 2024-09-16T20:48:14.589 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.589 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.589 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:14.589 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.447048+0000", 2024-09-16T20:48:14.589 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1288000000000002e-05 2024-09-16T20:48:14.589 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.589 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.589 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:14.589 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.447110+0000", 2024-09-16T20:48:14.589 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1877000000000004e-05 2024-09-16T20:48:14.589 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.589 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.590 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:48:14.590 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.447187+0000", 2024-09-16T20:48:14.590 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6861999999999997e-05 2024-09-16T20:48:14.590 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.590 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.590 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:14.590 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.447753+0000", 2024-09-16T20:48:14.590 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00056631200000000002 2024-09-16T20:48:14.590 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.590 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.590 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:14.590 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.448604+0000", 2024-09-16T20:48:14.590 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00085127399999999995 2024-09-16T20:48:14.591 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.591 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.591 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:14.591 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.448649+0000", 2024-09-16T20:48:14.591 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4867999999999998e-05 2024-09-16T20:48:14.591 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.591 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.591 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:14.591 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.448685+0000", 2024-09-16T20:48:14.591 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5880000000000002e-05 2024-09-16T20:48:14.591 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:14.591 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:14.591 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:14.592 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.592 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.592 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4462.0:2739 3.15 3:add7c518:::rbd_data.110e1748a8d2.0000000000000017:head [write 1327104~16384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T20:48:14.592 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:13.557894+0000", 2024-09-16T20:48:14.592 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.98705933999999995, 2024-09-16T20:48:14.592 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0018448309999999999, 2024-09-16T20:48:14.592 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:14.592 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:14.592 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:14.592 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:14.592 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4462", 2024-09-16T20:48:14.592 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:14.592 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2739 2024-09-16T20:48:14.593 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.593 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:14.593 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.593 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:14.593 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.557894+0000", 2024-09-16T20:48:14.593 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:14.593 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.593 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.593 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:14.593 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.557894+0000", 2024-09-16T20:48:14.593 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:14.593 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.593 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.594 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:14.594 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.557895+0000", 2024-09-16T20:48:14.594 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.9900000000000009e-07 2024-09-16T20:48:14.594 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.594 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.594 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:14.594 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.557909+0000", 2024-09-16T20:48:14.594 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3400999999999999e-05 2024-09-16T20:48:14.594 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.594 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.594 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:14.594 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.557922+0000", 2024-09-16T20:48:14.594 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3281999999999999e-05 2024-09-16T20:48:14.594 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.594 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.595 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:14.595 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.557929+0000", 2024-09-16T20:48:14.595 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7179999999999999e-06 2024-09-16T20:48:14.595 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.595 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.595 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:14.595 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558017+0000", 2024-09-16T20:48:14.595 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8443999999999995e-05 2024-09-16T20:48:14.595 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.595 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.595 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:14.595 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558111+0000", 2024-09-16T20:48:14.595 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4295000000000004e-05 2024-09-16T20:48:14.595 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.596 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.596 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:48:14.596 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558244+0000", 2024-09-16T20:48:14.596 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013249099999999999 2024-09-16T20:48:14.596 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.596 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.596 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:14.596 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558918+0000", 2024-09-16T20:48:14.596 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00067429600000000001 2024-09-16T20:48:14.596 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.596 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.596 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:14.596 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.559649+0000", 2024-09-16T20:48:14.596 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00073138399999999996 2024-09-16T20:48:14.597 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.597 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.597 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:14.597 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.559702+0000", 2024-09-16T20:48:14.597 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2652999999999999e-05 2024-09-16T20:48:14.597 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.597 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.597 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:14.597 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.559739+0000", 2024-09-16T20:48:14.597 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6868e-05 2024-09-16T20:48:14.597 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:14.597 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:14.597 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:14.597 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.598 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.598 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4462.0:2749 3.f 3:f4cae27b:::rbd_data.110e1748a8d2.0000000000000006:head [stat,write 1638400~16384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T20:48:14.598 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:13.558603+0000", 2024-09-16T20:48:14.598 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.98635070499999999, 2024-09-16T20:48:14.598 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0016212329999999999, 2024-09-16T20:48:14.598 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:14.598 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:14.598 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:14.598 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:14.598 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4462", 2024-09-16T20:48:14.598 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:14.598 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2749 2024-09-16T20:48:14.598 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.598 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:14.599 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.599 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:14.599 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558603+0000", 2024-09-16T20:48:14.599 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:14.599 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.599 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.599 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:14.599 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558603+0000", 2024-09-16T20:48:14.599 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:14.599 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.599 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.599 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:14.599 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558604+0000", 2024-09-16T20:48:14.599 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5700000000000002e-07 2024-09-16T20:48:14.600 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.600 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.600 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:14.600 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558616+0000", 2024-09-16T20:48:14.600 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2714999999999999e-05 2024-09-16T20:48:14.600 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.600 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.600 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:14.600 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558626+0000", 2024-09-16T20:48:14.600 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.9830000000000008e-06 2024-09-16T20:48:14.600 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.600 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.600 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:14.600 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558634+0000", 2024-09-16T20:48:14.601 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9149999999999996e-06 2024-09-16T20:48:14.601 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.601 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.601 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:14.601 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558694+0000", 2024-09-16T20:48:14.601 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9471e-05 2024-09-16T20:48:14.601 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.601 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.601 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:14.601 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558792+0000", 2024-09-16T20:48:14.601 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8848e-05 2024-09-16T20:48:14.601 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.602 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.602 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 1", 2024-09-16T20:48:14.602 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558970+0000", 2024-09-16T20:48:14.602 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00017710999999999999 2024-09-16T20:48:14.602 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.602 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.602 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:14.602 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.559880+0000", 2024-09-16T20:48:14.602 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00091052600000000004 2024-09-16T20:48:14.602 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.602 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.602 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:14.602 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.560127+0000", 2024-09-16T20:48:14.602 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00024736499999999999 2024-09-16T20:48:14.602 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.603 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.603 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:14.603 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.560183+0000", 2024-09-16T20:48:14.603 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5096999999999998e-05 2024-09-16T20:48:14.603 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.603 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.603 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:14.603 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.560224+0000", 2024-09-16T20:48:14.603 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1446000000000003e-05 2024-09-16T20:48:14.603 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:14.603 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:14.603 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:14.603 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.603 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.603 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4462.0:2765 3.3 3:c740a9af:::rbd_data.110e1748a8d2.0000000000000021:head [set-alloc-hint object_size 2097152 write_size 2097152,write 1884160~16384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T20:48:14.604 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:13.561890+0000", 2024-09-16T20:48:14.604 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.98306338000000004, 2024-09-16T20:48:14.604 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0016130859999999999, 2024-09-16T20:48:14.604 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:14.604 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:14.604 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:14.604 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:14.604 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4462", 2024-09-16T20:48:14.604 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:14.604 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2765 2024-09-16T20:48:14.604 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.604 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:14.604 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.605 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:14.605 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.561890+0000", 2024-09-16T20:48:14.605 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:14.605 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.605 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.605 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:14.605 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.561890+0000", 2024-09-16T20:48:14.605 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:14.605 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.605 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.605 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:14.605 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.561891+0000", 2024-09-16T20:48:14.605 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.207e-06 2024-09-16T20:48:14.605 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.606 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.606 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:14.606 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.561909+0000", 2024-09-16T20:48:14.606 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7586000000000001e-05 2024-09-16T20:48:14.606 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.606 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.606 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:14.606 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.561924+0000", 2024-09-16T20:48:14.606 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5328999999999998e-05 2024-09-16T20:48:14.606 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.606 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.606 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:14.606 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.561932+0000", 2024-09-16T20:48:14.607 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9889999999999992e-06 2024-09-16T20:48:14.607 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.607 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.607 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:14.607 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.561997+0000", 2024-09-16T20:48:14.607 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5288000000000002e-05 2024-09-16T20:48:14.607 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.607 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.607 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:14.607 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.562050+0000", 2024-09-16T20:48:14.607 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2201999999999999e-05 2024-09-16T20:48:14.607 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.607 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.607 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 2", 2024-09-16T20:48:14.608 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.562150+0000", 2024-09-16T20:48:14.608 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000100739 2024-09-16T20:48:14.608 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.608 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.608 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:14.608 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.562755+0000", 2024-09-16T20:48:14.608 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00060417100000000003 2024-09-16T20:48:14.608 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.608 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.608 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:14.608 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.563387+0000", 2024-09-16T20:48:14.608 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000632725 2024-09-16T20:48:14.609 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.609 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.609 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:14.609 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.563454+0000", 2024-09-16T20:48:14.609 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6290000000000004e-05 2024-09-16T20:48:14.609 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.609 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.609 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:14.609 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.563503+0000", 2024-09-16T20:48:14.609 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.956e-05 2024-09-16T20:48:14.609 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:14.609 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:14.609 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:14.610 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.610 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.610 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4462.0:2789 3.9 3:97f94242:::rbd_data.110e1748a8d2.0000000000000004:head [read 0~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-16T20:48:14.610 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:13.577221+0000", 2024-09-16T20:48:14.610 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.96773263499999995, 2024-09-16T20:48:14.610 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.004288086, 2024-09-16T20:48:14.610 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:14.610 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:14.610 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:14.610 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:14.610 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4462", 2024-09-16T20:48:14.610 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:14.610 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2789 2024-09-16T20:48:14.611 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.611 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:14.611 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.611 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:14.611 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.577221+0000", 2024-09-16T20:48:14.611 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:14.611 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.611 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.611 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:14.611 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.577221+0000", 2024-09-16T20:48:14.611 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:14.611 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.612 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.612 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:14.612 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.577222+0000", 2024-09-16T20:48:14.612 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2100000000000001e-06 2024-09-16T20:48:14.612 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.612 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.612 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:14.612 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.577227+0000", 2024-09-16T20:48:14.612 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1159999999999998e-06 2024-09-16T20:48:14.612 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.612 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.613 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:14.613 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.577243+0000", 2024-09-16T20:48:14.613 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5586e-05 2024-09-16T20:48:14.613 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.613 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.613 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:14.613 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.577251+0000", 2024-09-16T20:48:14.613 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.7260000000000008e-06 2024-09-16T20:48:14.613 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.613 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.613 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:14.613 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.577318+0000", 2024-09-16T20:48:14.614 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6957000000000003e-05 2024-09-16T20:48:14.614 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.614 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.614 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op must wait for map", 2024-09-16T20:48:14.614 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.577325+0000", 2024-09-16T20:48:14.614 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2659999999999998e-06 2024-09-16T20:48:14.614 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.614 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.614 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:14.614 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.581333+0000", 2024-09-16T20:48:14.614 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.004008317 2024-09-16T20:48:14.614 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.614 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.614 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:14.615 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.581408+0000", 2024-09-16T20:48:14.615 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4611999999999996e-05 2024-09-16T20:48:14.615 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.615 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.615 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:14.615 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.581509+0000", 2024-09-16T20:48:14.615 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101296 2024-09-16T20:48:14.615 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:14.615 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:14.615 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:14.615 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.615 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.615 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4462.0:2792 3.f 3:f4cae27b:::rbd_data.110e1748a8d2.0000000000000006:head [read 0~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-16T20:48:14.615 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:13.577270+0000", 2024-09-16T20:48:14.615 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.96768335599999999, 2024-09-16T20:48:14.616 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0045992910000000001, 2024-09-16T20:48:14.616 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:14.616 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:14.616 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:14.616 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:14.616 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4462", 2024-09-16T20:48:14.616 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:14.616 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2792 2024-09-16T20:48:14.616 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.616 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:14.616 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.616 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:14.616 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.577270+0000", 2024-09-16T20:48:14.616 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:14.617 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.617 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.617 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:14.617 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.577270+0000", 2024-09-16T20:48:14.617 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:14.617 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.617 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.617 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:14.617 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.577271+0000", 2024-09-16T20:48:14.617 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6300000000000005e-07 2024-09-16T20:48:14.617 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.617 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.617 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:14.618 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.577274+0000", 2024-09-16T20:48:14.618 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9270000000000001e-06 2024-09-16T20:48:14.618 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.618 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.618 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:14.618 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.577364+0000", 2024-09-16T20:48:14.618 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0158e-05 2024-09-16T20:48:14.618 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.618 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.618 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:14.618 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.577370+0000", 2024-09-16T20:48:14.618 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.37e-06 2024-09-16T20:48:14.618 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.619 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.619 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:14.619 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.577422+0000", 2024-09-16T20:48:14.619 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1949e-05 2024-09-16T20:48:14.619 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.619 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.619 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op must wait for map", 2024-09-16T20:48:14.619 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.577428+0000", 2024-09-16T20:48:14.619 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.519e-06 2024-09-16T20:48:14.619 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.619 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.619 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:14.619 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.581728+0000", 2024-09-16T20:48:14.619 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0043000060000000003 2024-09-16T20:48:14.620 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.620 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.620 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:14.620 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.581782+0000", 2024-09-16T20:48:14.620 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4472000000000003e-05 2024-09-16T20:48:14.620 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.620 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.620 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:14.620 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.581869+0000", 2024-09-16T20:48:14.620 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.7226999999999997e-05 2024-09-16T20:48:14.620 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:14.620 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:14.620 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:14.621 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.621 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.621 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop(client.4462.0:2766 3.e e28/23 3:77990de2:::rbd_data.110e1748a8d2.0000000000000022:head v 28'7, mlcod=28'5)", 2024-09-16T20:48:14.621 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:13.578190+0000", 2024-09-16T20:48:14.621 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.96676376799999997, 2024-09-16T20:48:14.621 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001604526, 2024-09-16T20:48:14.621 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:14.621 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:14.621 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:14.621 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:14.621 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.621 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:14.621 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.578190+0000", 2024-09-16T20:48:14.622 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:14.622 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.622 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.622 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:14.622 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.578190+0000", 2024-09-16T20:48:14.622 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:14.622 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.622 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.622 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:14.622 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.578191+0000", 2024-09-16T20:48:14.622 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1659999999999999e-06 2024-09-16T20:48:14.622 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.622 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.622 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:14.623 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.578228+0000", 2024-09-16T20:48:14.623 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6922000000000003e-05 2024-09-16T20:48:14.623 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.623 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.623 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:14.623 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.578241+0000", 2024-09-16T20:48:14.623 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3536999999999999e-05 2024-09-16T20:48:14.623 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.623 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.623 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:14.623 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.578248+0000", 2024-09-16T20:48:14.623 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8730000000000001e-06 2024-09-16T20:48:14.623 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.623 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.623 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:14.624 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.578303+0000", 2024-09-16T20:48:14.624 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5241000000000002e-05 2024-09-16T20:48:14.624 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.624 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.624 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:14.624 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.578327+0000", 2024-09-16T20:48:14.624 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3496999999999999e-05 2024-09-16T20:48:14.624 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.624 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.624 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:14.624 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.579747+0000", 2024-09-16T20:48:14.624 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0014204490000000001 2024-09-16T20:48:14.624 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.624 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.625 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:14.625 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.579794+0000", 2024-09-16T20:48:14.625 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6841000000000001e-05 2024-09-16T20:48:14.625 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:14.625 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:14.625 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:14.625 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.625 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.625 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4462.0:2826 3.3 3:c740a9af:::rbd_data.110e1748a8d2.0000000000000021:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-16T20:48:14.625 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:13.587689+0000", 2024-09-16T20:48:14.625 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.95726460899999999, 2024-09-16T20:48:14.625 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0029427279999999999, 2024-09-16T20:48:14.626 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:14.626 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:14.626 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:14.626 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:14.626 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4462", 2024-09-16T20:48:14.626 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:14.626 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2826 2024-09-16T20:48:14.626 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.626 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:14.626 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.626 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:14.626 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.587689+0000", 2024-09-16T20:48:14.626 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:14.626 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.627 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.627 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:14.627 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.587689+0000", 2024-09-16T20:48:14.627 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:14.627 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.627 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.627 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:14.627 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.587690+0000", 2024-09-16T20:48:14.627 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.023e-06 2024-09-16T20:48:14.627 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.627 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.627 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:14.627 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.587694+0000", 2024-09-16T20:48:14.628 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8039999999999999e-06 2024-09-16T20:48:14.628 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.628 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.628 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:14.628 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.587704+0000", 2024-09-16T20:48:14.628 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.9210000000000004e-06 2024-09-16T20:48:14.628 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.628 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.628 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:14.628 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.587711+0000", 2024-09-16T20:48:14.628 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4989999999999997e-06 2024-09-16T20:48:14.628 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.628 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.629 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:14.629 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.587777+0000", 2024-09-16T20:48:14.629 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6223999999999995e-05 2024-09-16T20:48:14.629 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.629 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.629 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:14.629 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.587851+0000", 2024-09-16T20:48:14.629 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3516000000000006e-05 2024-09-16T20:48:14.629 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.629 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.629 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 2", 2024-09-16T20:48:14.629 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.587959+0000", 2024-09-16T20:48:14.629 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000108496 2024-09-16T20:48:14.629 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.630 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.630 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:14.630 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.589206+0000", 2024-09-16T20:48:14.630 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0012469250000000001 2024-09-16T20:48:14.630 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.630 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.630 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:14.630 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.590519+0000", 2024-09-16T20:48:14.630 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001312681 2024-09-16T20:48:14.630 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.630 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.630 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:14.630 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.590588+0000", 2024-09-16T20:48:14.631 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8833999999999999e-05 2024-09-16T20:48:14.631 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.631 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.631 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:14.631 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.590632+0000", 2024-09-16T20:48:14.631 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3804999999999998e-05 2024-09-16T20:48:14.631 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:14.631 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:14.631 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:14.631 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.631 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.631 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4462.0:2829 3.3 3:c09ac2c2:::rbd_data.110e1748a8d2.0000000000000023:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-16T20:48:14.632 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:13.587777+0000", 2024-09-16T20:48:14.632 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.95717633700000004, 2024-09-16T20:48:14.632 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0029914009999999999, 2024-09-16T20:48:14.632 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:14.632 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:14.632 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:14.632 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:14.632 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4462", 2024-09-16T20:48:14.632 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:14.632 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2829 2024-09-16T20:48:14.632 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.632 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:14.632 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.632 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:14.633 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.587777+0000", 2024-09-16T20:48:14.633 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:14.633 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.633 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.633 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:14.633 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.587777+0000", 2024-09-16T20:48:14.633 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:14.633 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.633 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.633 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:14.633 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.587778+0000", 2024-09-16T20:48:14.633 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9999999999999997e-07 2024-09-16T20:48:14.633 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.634 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.634 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:14.634 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.587781+0000", 2024-09-16T20:48:14.634 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6260000000000001e-06 2024-09-16T20:48:14.634 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.634 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.634 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:14.634 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.587792+0000", 2024-09-16T20:48:14.634 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1028e-05 2024-09-16T20:48:14.634 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.634 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.634 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:14.634 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.587799+0000", 2024-09-16T20:48:14.635 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.472e-06 2024-09-16T20:48:14.635 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.635 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.635 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:14.635 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.588271+0000", 2024-09-16T20:48:14.635 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00047227 2024-09-16T20:48:14.635 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.635 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.635 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:14.635 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.588344+0000", 2024-09-16T20:48:14.635 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3269000000000006e-05 2024-09-16T20:48:14.635 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.635 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.635 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 2", 2024-09-16T20:48:14.636 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.588468+0000", 2024-09-16T20:48:14.636 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00012324499999999999 2024-09-16T20:48:14.636 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.636 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.636 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:14.636 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.589550+0000", 2024-09-16T20:48:14.636 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001081867 2024-09-16T20:48:14.636 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.636 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.636 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:14.636 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.590670+0000", 2024-09-16T20:48:14.636 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001120476 2024-09-16T20:48:14.636 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.637 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.637 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:14.637 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.590730+0000", 2024-09-16T20:48:14.637 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9673000000000003e-05 2024-09-16T20:48:14.637 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.637 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.637 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:14.637 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.590769+0000", 2024-09-16T20:48:14.637 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8875000000000003e-05 2024-09-16T20:48:14.637 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:14.637 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:14.637 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:14.637 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.637 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.638 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4462.0:2837 3.a 3:5793249b:::rbd_data.110e1748a8d2.0000000000000030:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-16T20:48:14.638 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:13.587813+0000", 2024-09-16T20:48:14.638 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.95714041000000005, 2024-09-16T20:48:14.638 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0020782470000000001, 2024-09-16T20:48:14.638 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:14.638 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:14.638 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:14.638 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:14.638 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4462", 2024-09-16T20:48:14.638 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:14.638 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2837 2024-09-16T20:48:14.638 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.638 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:14.638 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.639 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:14.639 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.587813+0000", 2024-09-16T20:48:14.639 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:14.639 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.639 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.639 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:14.639 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.587813+0000", 2024-09-16T20:48:14.639 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:14.639 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.639 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.639 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:14.639 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.587814+0000", 2024-09-16T20:48:14.639 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6699999999999999e-07 2024-09-16T20:48:14.640 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.640 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.640 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:14.640 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.587816+0000", 2024-09-16T20:48:14.640 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6730000000000001e-06 2024-09-16T20:48:14.640 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.640 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.640 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:14.640 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.587824+0000", 2024-09-16T20:48:14.640 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6590000000000007e-06 2024-09-16T20:48:14.640 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.640 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.640 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:14.641 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.587829+0000", 2024-09-16T20:48:14.641 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3360000000000002e-06 2024-09-16T20:48:14.641 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.641 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.641 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:14.641 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.587885+0000", 2024-09-16T20:48:14.641 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5594999999999999e-05 2024-09-16T20:48:14.641 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.641 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.641 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:14.641 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.587964+0000", 2024-09-16T20:48:14.641 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9336000000000001e-05 2024-09-16T20:48:14.641 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.641 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.642 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 1", 2024-09-16T20:48:14.642 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.588068+0000", 2024-09-16T20:48:14.642 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010404900000000001 2024-09-16T20:48:14.642 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.642 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.642 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:14.642 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.589216+0000", 2024-09-16T20:48:14.642 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001148203 2024-09-16T20:48:14.642 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.642 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.642 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:14.642 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.589796+0000", 2024-09-16T20:48:14.642 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00057953799999999999 2024-09-16T20:48:14.643 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.643 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.643 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:14.643 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.589849+0000", 2024-09-16T20:48:14.643 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3091999999999999e-05 2024-09-16T20:48:14.643 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.643 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.643 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:14.643 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.589891+0000", 2024-09-16T20:48:14.643 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2299000000000003e-05 2024-09-16T20:48:14.643 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:14.643 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:14.643 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:14.643 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.644 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.644 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4462.0:2908 3.15 3:add7c518:::rbd_data.110e1748a8d2.0000000000000017:head [truncate 1622016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-16T20:48:14.644 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:13.595590+0000", 2024-09-16T20:48:14.644 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.94936312599999995, 2024-09-16T20:48:14.644 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001684517, 2024-09-16T20:48:14.644 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:14.644 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:14.644 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:14.644 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:14.644 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4462", 2024-09-16T20:48:14.644 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:14.644 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2908 2024-09-16T20:48:14.644 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.644 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:14.645 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.645 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:14.645 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.595590+0000", 2024-09-16T20:48:14.645 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:14.645 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.645 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.645 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:14.645 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.595590+0000", 2024-09-16T20:48:14.645 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:14.645 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.645 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.645 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:14.645 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.595592+0000", 2024-09-16T20:48:14.645 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.753e-06 2024-09-16T20:48:14.645 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.646 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.646 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:14.646 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.595597+0000", 2024-09-16T20:48:14.646 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0359999999999997e-06 2024-09-16T20:48:14.646 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.646 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.646 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:14.646 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.595613+0000", 2024-09-16T20:48:14.646 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5492999999999999e-05 2024-09-16T20:48:14.646 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.646 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.646 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:14.646 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.595621+0000", 2024-09-16T20:48:14.646 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0649999999999999e-06 2024-09-16T20:48:14.646 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.647 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.647 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:14.647 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.595691+0000", 2024-09-16T20:48:14.647 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0550999999999996e-05 2024-09-16T20:48:14.647 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.647 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.647 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:14.647 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.595776+0000", 2024-09-16T20:48:14.647 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4627000000000001e-05 2024-09-16T20:48:14.647 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.647 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.647 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:48:14.647 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.595890+0000", 2024-09-16T20:48:14.647 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00011427899999999999 2024-09-16T20:48:14.648 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.648 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.648 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:14.648 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.596951+0000", 2024-09-16T20:48:14.648 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0010610839999999999 2024-09-16T20:48:14.648 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.648 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.648 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:14.648 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.597174+0000", 2024-09-16T20:48:14.648 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00022258099999999999 2024-09-16T20:48:14.648 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.648 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.648 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:14.648 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.597231+0000", 2024-09-16T20:48:14.649 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7185000000000002e-05 2024-09-16T20:48:14.649 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.649 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.649 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:14.649 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.597275+0000", 2024-09-16T20:48:14.649 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3863000000000003e-05 2024-09-16T20:48:14.649 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:14.649 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:14.649 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:14.649 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.649 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.649 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4462.0:3062 3.15 3:add7c518:::rbd_data.110e1748a8d2.0000000000000017:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-16T20:48:14.650 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:13.644870+0000", 2024-09-16T20:48:14.650 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.90008365899999998, 2024-09-16T20:48:14.650 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0019350470000000001, 2024-09-16T20:48:14.650 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:14.650 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:14.650 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:14.650 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:14.650 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4462", 2024-09-16T20:48:14.650 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:14.650 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3062 2024-09-16T20:48:14.650 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.650 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:14.650 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.650 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:14.651 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.644870+0000", 2024-09-16T20:48:14.651 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:14.651 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.651 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.651 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:14.651 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.644870+0000", 2024-09-16T20:48:14.651 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:14.651 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.651 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.651 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:14.651 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.644871+0000", 2024-09-16T20:48:14.651 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0100000000000001e-06 2024-09-16T20:48:14.651 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.651 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.651 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:14.652 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.644876+0000", 2024-09-16T20:48:14.652 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2249999999999999e-06 2024-09-16T20:48:14.652 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.652 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.652 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:14.652 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.644891+0000", 2024-09-16T20:48:14.652 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.451e-05 2024-09-16T20:48:14.652 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.652 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.652 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:14.652 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.644898+0000", 2024-09-16T20:48:14.652 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9030000000000004e-06 2024-09-16T20:48:14.652 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.652 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.653 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:14.653 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.644969+0000", 2024-09-16T20:48:14.653 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0815000000000006e-05 2024-09-16T20:48:14.653 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.653 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.653 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:14.653 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.645053+0000", 2024-09-16T20:48:14.653 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.3592999999999997e-05 2024-09-16T20:48:14.653 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.653 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.653 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:48:14.653 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.645184+0000", 2024-09-16T20:48:14.653 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013147699999999999 2024-09-16T20:48:14.653 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.654 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.654 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:14.654 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.646158+0000", 2024-09-16T20:48:14.654 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00097327399999999999 2024-09-16T20:48:14.654 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.654 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.654 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:14.654 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.646749+0000", 2024-09-16T20:48:14.654 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00059180699999999997 2024-09-16T20:48:14.654 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.654 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.654 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:14.655 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.646779+0000", 2024-09-16T20:48:14.655 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9176e-05 2024-09-16T20:48:14.655 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.655 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.655 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:14.655 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.646805+0000", 2024-09-16T20:48:14.655 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6256999999999999e-05 2024-09-16T20:48:14.655 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:14.655 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:14.655 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:14.655 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.655 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.655 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4462.0:3083 3.9 3:97f94242:::rbd_data.110e1748a8d2.0000000000000004:head [stat,truncate 1572864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-16T20:48:14.656 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:13.648793+0000", 2024-09-16T20:48:14.656 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.89616031799999996, 2024-09-16T20:48:14.656 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0021196909999999999, 2024-09-16T20:48:14.656 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:14.656 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:14.656 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:14.656 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:14.656 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4462", 2024-09-16T20:48:14.656 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:14.656 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3083 2024-09-16T20:48:14.656 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.656 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:14.656 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.657 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:14.657 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.648793+0000", 2024-09-16T20:48:14.657 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:14.657 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.657 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.657 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:14.657 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.648793+0000", 2024-09-16T20:48:14.657 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:14.657 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.657 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.657 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:14.657 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.648794+0000", 2024-09-16T20:48:14.657 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.78e-07 2024-09-16T20:48:14.658 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.658 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.658 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:14.658 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.648799+0000", 2024-09-16T20:48:14.658 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6079999999999998e-06 2024-09-16T20:48:14.658 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.658 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.658 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:14.658 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.648813+0000", 2024-09-16T20:48:14.658 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4151999999999999e-05 2024-09-16T20:48:14.658 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.658 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.658 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:14.659 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.648820+0000", 2024-09-16T20:48:14.659 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.52e-06 2024-09-16T20:48:14.659 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.659 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.659 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:14.659 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.648896+0000", 2024-09-16T20:48:14.659 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5737999999999999e-05 2024-09-16T20:48:14.659 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.659 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.659 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:14.659 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.648973+0000", 2024-09-16T20:48:14.659 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6951000000000002e-05 2024-09-16T20:48:14.659 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.659 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.660 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:48:14.660 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.649110+0000", 2024-09-16T20:48:14.660 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013662099999999999 2024-09-16T20:48:14.660 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.660 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.660 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:14.660 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.649784+0000", 2024-09-16T20:48:14.660 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000673896 2024-09-16T20:48:14.660 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.660 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.660 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:14.660 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.650854+0000", 2024-09-16T20:48:14.660 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0010699520000000001 2024-09-16T20:48:14.661 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.661 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.661 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:14.661 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.650889+0000", 2024-09-16T20:48:14.661 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5522999999999999e-05 2024-09-16T20:48:14.661 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.661 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.661 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:14.661 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.650913+0000", 2024-09-16T20:48:14.661 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3751999999999999e-05 2024-09-16T20:48:14.661 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:14.661 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:14.661 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:14.662 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.662 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.662 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4462.0:3085 3.f 3:f4cae27b:::rbd_data.110e1748a8d2.0000000000000006:head [stat,truncate 1572864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-16T20:48:14.662 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:13.648840+0000", 2024-09-16T20:48:14.662 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.89611335700000005, 2024-09-16T20:48:14.662 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0016608599999999999, 2024-09-16T20:48:14.662 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:14.662 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:14.662 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:14.662 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:14.662 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4462", 2024-09-16T20:48:14.662 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:14.662 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3085 2024-09-16T20:48:14.662 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.663 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:14.663 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.663 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:14.663 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.648840+0000", 2024-09-16T20:48:14.663 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:14.663 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.663 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.663 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:14.663 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.648840+0000", 2024-09-16T20:48:14.663 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:14.663 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.663 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.663 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:14.663 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.648841+0000", 2024-09-16T20:48:14.663 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1699999999999998e-07 2024-09-16T20:48:14.664 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.664 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.664 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:14.664 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.648844+0000", 2024-09-16T20:48:14.664 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9220000000000001e-06 2024-09-16T20:48:14.664 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.664 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.664 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:14.664 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.648854+0000", 2024-09-16T20:48:14.664 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0472000000000001e-05 2024-09-16T20:48:14.664 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.664 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.664 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:14.664 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.648860+0000", 2024-09-16T20:48:14.665 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3330000000000002e-06 2024-09-16T20:48:14.665 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.665 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.665 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:14.665 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.648921+0000", 2024-09-16T20:48:14.665 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1131999999999997e-05 2024-09-16T20:48:14.665 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.665 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.665 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:14.665 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.649001+0000", 2024-09-16T20:48:14.665 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9076e-05 2024-09-16T20:48:14.665 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.666 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.666 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 1", 2024-09-16T20:48:14.666 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.649134+0000", 2024-09-16T20:48:14.666 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000133122 2024-09-16T20:48:14.666 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.666 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.666 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:14.666 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.650296+0000", 2024-09-16T20:48:14.666 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001162164 2024-09-16T20:48:14.666 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.666 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.666 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:14.666 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.650413+0000", 2024-09-16T20:48:14.666 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00011722699999999999 2024-09-16T20:48:14.667 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.667 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.667 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:14.667 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.650461+0000", 2024-09-16T20:48:14.667 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7926999999999997e-05 2024-09-16T20:48:14.667 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:14.667 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:14.667 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:14.667 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.650501+0000", 2024-09-16T20:48:14.667 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9968000000000001e-05 2024-09-16T20:48:14.667 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:14.667 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:14.667 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:14.668 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:14.668 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:14.668 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:14.669 INFO:teuthology.orchestra.run.smithi139.stdout:truncating image image_client.0-clone1 from 0x11fb800 (overlap 0x490800) to 0x3a6c00 2024-09-16T20:48:14.670 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:48:14.670 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:14.667+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:14.770 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:14.769+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:14.820 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:14.820 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:48:14.820 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:48:14.820 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:14.829 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:48:14.873 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:14.872+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:14.924 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #0, image image_client.0 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent1 2024-09-16T20:48:14.941 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:14.941 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:48:14.941 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:48:14.942 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:48:14.942 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:14.950 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:48:14.992 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:14.991+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:15.073 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:15.071+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:15.110 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:15.110 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:48:15.110 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:48:15.110 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:48:15.110 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.111 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:48:15.111 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:48:15.111 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 21.854090939999999, 2024-09-16T20:48:15.111 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67502346999999996, 2024-09-16T20:48:15.111 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:15.111 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:15.111 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:15.111 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:15.111 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:48:15.111 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:48:15.111 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 1 2024-09-16T20:48:15.111 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.111 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:15.112 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.112 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:15.112 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:48:15.112 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:15.112 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.112 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.112 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:15.112 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:48:15.112 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:15.112 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.112 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.112 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:15.112 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254128+0000", 2024-09-16T20:48:15.113 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.463e-06 2024-09-16T20:48:15.113 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.113 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.113 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:15.113 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254135+0000", 2024-09-16T20:48:15.113 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1579999999999999e-06 2024-09-16T20:48:15.113 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.113 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.113 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:15.113 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254146+0000", 2024-09-16T20:48:15.113 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1341e-05 2024-09-16T20:48:15.113 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.113 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.114 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:15.114 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254155+0000", 2024-09-16T20:48:15.114 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.1549999999999996e-06 2024-09-16T20:48:15.114 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.114 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.114 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:15.114 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254221+0000", 2024-09-16T20:48:15.114 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5470999999999997e-05 2024-09-16T20:48:15.114 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.114 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.114 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:48:15.114 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254231+0000", 2024-09-16T20:48:15.115 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0592e-05 2024-09-16T20:48:15.115 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.115 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.115 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:15.115 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926711+0000", 2024-09-16T20:48:15.115 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67247946599999997 2024-09-16T20:48:15.115 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.115 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.115 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:15.115 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926910+0000", 2024-09-16T20:48:15.115 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019934 2024-09-16T20:48:15.115 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.115 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.115 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:48:15.116 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.927159+0000", 2024-09-16T20:48:15.116 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00024876599999999998 2024-09-16T20:48:15.116 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.116 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.116 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:15.116 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.928797+0000", 2024-09-16T20:48:15.116 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001638308 2024-09-16T20:48:15.116 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.116 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.116 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:15.116 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929026+0000", 2024-09-16T20:48:15.116 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00022876099999999999 2024-09-16T20:48:15.116 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.117 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.117 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:15.117 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929102+0000", 2024-09-16T20:48:15.117 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6184000000000001e-05 2024-09-16T20:48:15.117 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.117 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.117 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:15.117 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929150+0000", 2024-09-16T20:48:15.117 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7465e-05 2024-09-16T20:48:15.117 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:15.117 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:15.117 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:15.117 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.118 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.118 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:25 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:48:15.118 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:53.949358+0000", 2024-09-16T20:48:15.118 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 21.158858606999999, 2024-09-16T20:48:15.118 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0036872879999999999, 2024-09-16T20:48:15.118 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:15.118 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:15.118 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:15.118 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:15.118 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:48:15.118 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:48:15.118 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 25 2024-09-16T20:48:15.118 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.119 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:15.119 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.119 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:15.119 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.949358+0000", 2024-09-16T20:48:15.119 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:15.119 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.119 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.119 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:15.119 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.949358+0000", 2024-09-16T20:48:15.119 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:15.119 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.119 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.119 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:15.120 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.949359+0000", 2024-09-16T20:48:15.120 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7700000000000004e-07 2024-09-16T20:48:15.120 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.120 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.120 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:15.120 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.949365+0000", 2024-09-16T20:48:15.120 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2669999999999997e-06 2024-09-16T20:48:15.120 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.120 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.120 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:15.120 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.949375+0000", 2024-09-16T20:48:15.120 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0188e-05 2024-09-16T20:48:15.120 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.120 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.121 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:15.121 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.949381+0000", 2024-09-16T20:48:15.121 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.105e-06 2024-09-16T20:48:15.121 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.121 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.121 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:15.121 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.952075+0000", 2024-09-16T20:48:15.121 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.002694072 2024-09-16T20:48:15.121 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.121 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.121 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:15.121 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.952133+0000", 2024-09-16T20:48:15.121 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8393000000000001e-05 2024-09-16T20:48:15.122 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.122 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.122 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:48:15.122 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.952214+0000", 2024-09-16T20:48:15.122 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0329999999999998e-05 2024-09-16T20:48:15.122 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.122 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.122 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:15.122 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.952635+0000", 2024-09-16T20:48:15.122 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000421703 2024-09-16T20:48:15.122 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.122 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.122 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:15.122 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.952996+0000", 2024-09-16T20:48:15.123 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00036085199999999999 2024-09-16T20:48:15.123 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.123 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.123 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:15.123 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.953026+0000", 2024-09-16T20:48:15.123 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9583000000000001e-05 2024-09-16T20:48:15.123 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.123 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.123 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:15.123 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.953046+0000", 2024-09-16T20:48:15.123 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0018e-05 2024-09-16T20:48:15.123 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:15.123 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:15.123 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:15.124 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.124 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.124 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:43 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:48:15.124 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:54.058718+0000", 2024-09-16T20:48:15.124 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 21.049499177000001, 2024-09-16T20:48:15.124 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0039347039999999998, 2024-09-16T20:48:15.124 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:15.124 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:15.124 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:15.124 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:15.124 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:48:15.124 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:48:15.125 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 43 2024-09-16T20:48:15.125 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.125 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:15.125 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.125 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:15.125 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058718+0000", 2024-09-16T20:48:15.125 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:15.125 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.125 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.125 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:15.125 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058718+0000", 2024-09-16T20:48:15.125 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:15.125 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.126 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.126 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:15.126 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058718+0000", 2024-09-16T20:48:15.126 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9799999999999999e-07 2024-09-16T20:48:15.126 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.126 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.126 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:15.126 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058720+0000", 2024-09-16T20:48:15.126 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.086e-06 2024-09-16T20:48:15.126 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.126 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.126 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:15.126 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058724+0000", 2024-09-16T20:48:15.127 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8260000000000003e-06 2024-09-16T20:48:15.127 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.127 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.127 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:15.127 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058727+0000", 2024-09-16T20:48:15.127 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0539999999999999e-06 2024-09-16T20:48:15.127 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.127 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.127 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:15.127 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.060648+0000", 2024-09-16T20:48:15.127 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001920958 2024-09-16T20:48:15.127 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.128 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.128 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:15.128 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.060711+0000", 2024-09-16T20:48:15.128 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2496999999999995e-05 2024-09-16T20:48:15.128 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.128 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.128 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:48:15.128 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.060789+0000", 2024-09-16T20:48:15.128 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8004999999999996e-05 2024-09-16T20:48:15.128 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.128 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.128 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:15.128 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.062182+0000", 2024-09-16T20:48:15.128 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0013930889999999999 2024-09-16T20:48:15.129 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.129 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.129 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:15.129 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.062597+0000", 2024-09-16T20:48:15.129 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00041566800000000001 2024-09-16T20:48:15.129 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.129 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.129 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:15.129 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.062628+0000", 2024-09-16T20:48:15.129 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0876000000000001e-05 2024-09-16T20:48:15.129 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.129 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.129 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:15.130 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.062653+0000", 2024-09-16T20:48:15.130 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.4346999999999999e-05 2024-09-16T20:48:15.130 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:15.130 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:15.130 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:15.130 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.130 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.130 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:48:15.130 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:54.058734+0000", 2024-09-16T20:48:15.130 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 21.049483077000001, 2024-09-16T20:48:15.130 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0040462379999999997, 2024-09-16T20:48:15.130 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:15.130 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:15.130 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:15.130 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:15.130 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:48:15.131 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:48:15.131 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 44 2024-09-16T20:48:15.131 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.131 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:15.131 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.131 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:15.131 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058734+0000", 2024-09-16T20:48:15.131 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:15.131 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.131 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.131 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:15.131 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058734+0000", 2024-09-16T20:48:15.131 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:15.131 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.131 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.132 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:15.132 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058734+0000", 2024-09-16T20:48:15.132 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7399999999999999e-07 2024-09-16T20:48:15.132 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.132 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.132 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:15.132 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058809+0000", 2024-09-16T20:48:15.132 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4856000000000003e-05 2024-09-16T20:48:15.132 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.132 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.132 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:15.132 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058816+0000", 2024-09-16T20:48:15.132 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.002e-06 2024-09-16T20:48:15.132 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.132 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.133 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:15.133 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058820+0000", 2024-09-16T20:48:15.133 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8149999999999999e-06 2024-09-16T20:48:15.133 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.133 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.133 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:15.133 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.061036+0000", 2024-09-16T20:48:15.133 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0022159020000000001 2024-09-16T20:48:15.133 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.133 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.133 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:15.133 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.061087+0000", 2024-09-16T20:48:15.133 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1649e-05 2024-09-16T20:48:15.133 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.134 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.134 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:48:15.134 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.061176+0000", 2024-09-16T20:48:15.134 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8355000000000004e-05 2024-09-16T20:48:15.134 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.134 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.134 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:15.134 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.062664+0000", 2024-09-16T20:48:15.134 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001488259 2024-09-16T20:48:15.134 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.134 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.134 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:15.134 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.062725+0000", 2024-09-16T20:48:15.135 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0430000000000002e-05 2024-09-16T20:48:15.135 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.135 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.135 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:15.135 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.062754+0000", 2024-09-16T20:48:15.135 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9133e-05 2024-09-16T20:48:15.135 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.135 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.135 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:15.135 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.062780+0000", 2024-09-16T20:48:15.135 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6563e-05 2024-09-16T20:48:15.135 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:15.135 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:15.135 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:15.135 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.136 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.136 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:48:15.136 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:54.058833+0000", 2024-09-16T20:48:15.136 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 21.049383985999999, 2024-09-16T20:48:15.136 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0049980959999999996, 2024-09-16T20:48:15.136 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:15.136 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:15.136 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:15.136 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:15.136 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:48:15.136 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:48:15.136 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 45 2024-09-16T20:48:15.136 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.137 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:15.137 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.137 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:15.137 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058833+0000", 2024-09-16T20:48:15.137 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:15.137 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.137 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.137 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:15.137 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058833+0000", 2024-09-16T20:48:15.137 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:15.137 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.137 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.137 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:15.137 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058833+0000", 2024-09-16T20:48:15.137 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8799999999999998e-07 2024-09-16T20:48:15.138 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.138 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.138 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:15.138 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058837+0000", 2024-09-16T20:48:15.138 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4400000000000001e-06 2024-09-16T20:48:15.138 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.138 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.138 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:15.138 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058842+0000", 2024-09-16T20:48:15.138 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5829999999999997e-06 2024-09-16T20:48:15.138 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.138 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.138 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:15.139 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058847+0000", 2024-09-16T20:48:15.139 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.955e-06 2024-09-16T20:48:15.139 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.139 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.139 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:15.139 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.061841+0000", 2024-09-16T20:48:15.139 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0029938500000000002 2024-09-16T20:48:15.139 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.139 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.139 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:15.139 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.061889+0000", 2024-09-16T20:48:15.139 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7540999999999998e-05 2024-09-16T20:48:15.139 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.140 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.140 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:48:15.140 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.061961+0000", 2024-09-16T20:48:15.140 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2218000000000005e-05 2024-09-16T20:48:15.140 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.140 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.140 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:15.140 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.063325+0000", 2024-09-16T20:48:15.140 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001364105 2024-09-16T20:48:15.140 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.140 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.140 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:15.140 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.063774+0000", 2024-09-16T20:48:15.141 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00044868300000000001 2024-09-16T20:48:15.141 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.141 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.141 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:15.141 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.063804+0000", 2024-09-16T20:48:15.141 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9774e-05 2024-09-16T20:48:15.141 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.141 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.141 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:15.141 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.063831+0000", 2024-09-16T20:48:15.141 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7559000000000002e-05 2024-09-16T20:48:15.141 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:15.141 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:15.142 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:15.142 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.142 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.142 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:48:15.142 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:54.058855+0000", 2024-09-16T20:48:15.142 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 21.049362336000002, 2024-09-16T20:48:15.142 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0050797430000000003, 2024-09-16T20:48:15.142 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:15.142 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:15.142 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:15.142 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:15.142 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:48:15.142 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:48:15.142 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 46 2024-09-16T20:48:15.143 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.143 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:15.143 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.143 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:15.143 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058855+0000", 2024-09-16T20:48:15.143 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:15.143 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.143 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.143 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:15.143 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058855+0000", 2024-09-16T20:48:15.143 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:15.143 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.143 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.144 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:15.144 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058855+0000", 2024-09-16T20:48:15.144 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2599999999999998e-07 2024-09-16T20:48:15.144 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.144 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.144 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:15.144 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058857+0000", 2024-09-16T20:48:15.144 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2390000000000001e-06 2024-09-16T20:48:15.144 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.144 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.144 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:15.144 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058861+0000", 2024-09-16T20:48:15.144 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.873e-06 2024-09-16T20:48:15.145 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.145 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.145 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:15.145 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058867+0000", 2024-09-16T20:48:15.145 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.268e-06 2024-09-16T20:48:15.145 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.145 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.145 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:15.145 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.062208+0000", 2024-09-16T20:48:15.145 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0033400740000000002 2024-09-16T20:48:15.145 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.145 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.145 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:15.146 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.062257+0000", 2024-09-16T20:48:15.146 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9539000000000001e-05 2024-09-16T20:48:15.146 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.146 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.146 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:48:15.146 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.062333+0000", 2024-09-16T20:48:15.146 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6087000000000005e-05 2024-09-16T20:48:15.146 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.146 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.146 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:15.146 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.063843+0000", 2024-09-16T20:48:15.146 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001509718 2024-09-16T20:48:15.146 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.147 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.147 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:15.147 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.063884+0000", 2024-09-16T20:48:15.147 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1321e-05 2024-09-16T20:48:15.147 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.147 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.147 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:15.147 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.063913+0000", 2024-09-16T20:48:15.147 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9218e-05 2024-09-16T20:48:15.147 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.147 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.147 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:15.147 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.063934+0000", 2024-09-16T20:48:15.147 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1080000000000001e-05 2024-09-16T20:48:15.148 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:15.148 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:15.148 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:15.148 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.148 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.148 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:48:15.148 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:54.058872+0000", 2024-09-16T20:48:15.148 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 21.049344668, 2024-09-16T20:48:15.148 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0060982579999999996, 2024-09-16T20:48:15.148 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:15.148 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:15.148 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:15.148 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:15.149 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:48:15.149 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:48:15.149 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 47 2024-09-16T20:48:15.149 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.149 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:15.149 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.149 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:15.149 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058872+0000", 2024-09-16T20:48:15.149 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:15.149 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.149 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.149 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:15.149 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058872+0000", 2024-09-16T20:48:15.149 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:15.149 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.150 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.150 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:15.150 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058873+0000", 2024-09-16T20:48:15.150 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1e-07 2024-09-16T20:48:15.150 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.150 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.150 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:15.150 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058919+0000", 2024-09-16T20:48:15.150 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6570999999999998e-05 2024-09-16T20:48:15.150 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.150 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.150 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:15.150 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058926+0000", 2024-09-16T20:48:15.150 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5509999999999996e-06 2024-09-16T20:48:15.150 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.151 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.151 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:15.151 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058932+0000", 2024-09-16T20:48:15.151 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8780000000000003e-06 2024-09-16T20:48:15.151 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.151 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.151 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:15.151 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.062820+0000", 2024-09-16T20:48:15.151 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0038885790000000001 2024-09-16T20:48:15.151 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.151 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.151 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:15.151 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.062868+0000", 2024-09-16T20:48:15.152 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7948999999999998e-05 2024-09-16T20:48:15.152 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.152 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.152 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:48:15.152 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.062945+0000", 2024-09-16T20:48:15.152 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6495999999999994e-05 2024-09-16T20:48:15.152 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.152 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.152 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:15.152 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.064336+0000", 2024-09-16T20:48:15.152 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001391446 2024-09-16T20:48:15.152 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.152 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.152 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:15.153 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.064915+0000", 2024-09-16T20:48:15.153 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00057899000000000004 2024-09-16T20:48:15.153 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.153 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.153 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:15.153 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.064947+0000", 2024-09-16T20:48:15.153 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2132999999999999e-05 2024-09-16T20:48:15.153 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.153 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.153 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:15.153 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.064971+0000", 2024-09-16T20:48:15.153 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3354999999999999e-05 2024-09-16T20:48:15.153 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:15.154 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:15.154 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:15.154 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.154 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.154 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:48:15.154 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:54.058944+0000", 2024-09-16T20:48:15.154 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 21.049272959, 2024-09-16T20:48:15.154 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0061376369999999996, 2024-09-16T20:48:15.154 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:15.154 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:15.154 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:15.154 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:15.154 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:48:15.154 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:48:15.155 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 48 2024-09-16T20:48:15.155 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.155 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:15.155 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.155 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:15.155 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058944+0000", 2024-09-16T20:48:15.155 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:15.155 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.155 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.155 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:15.155 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058944+0000", 2024-09-16T20:48:15.155 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:15.155 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.156 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.156 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:15.156 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058945+0000", 2024-09-16T20:48:15.156 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9200000000000002e-07 2024-09-16T20:48:15.156 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.156 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.156 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:15.156 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058948+0000", 2024-09-16T20:48:15.156 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8099999999999999e-06 2024-09-16T20:48:15.156 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.156 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.156 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:15.156 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058963+0000", 2024-09-16T20:48:15.157 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5112e-05 2024-09-16T20:48:15.157 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.157 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.157 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:15.157 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058975+0000", 2024-09-16T20:48:15.157 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1732e-05 2024-09-16T20:48:15.157 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.157 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.157 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:15.157 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.063363+0000", 2024-09-16T20:48:15.157 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0043879330000000001 2024-09-16T20:48:15.157 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.158 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.158 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:15.158 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.063420+0000", 2024-09-16T20:48:15.158 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6745e-05 2024-09-16T20:48:15.158 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.158 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.158 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:48:15.158 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.063496+0000", 2024-09-16T20:48:15.158 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6297999999999994e-05 2024-09-16T20:48:15.158 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.158 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.158 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:15.158 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.064981+0000", 2024-09-16T20:48:15.159 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0014852559999999999 2024-09-16T20:48:15.159 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.159 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.159 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:15.159 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.065028+0000", 2024-09-16T20:48:15.159 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7052000000000003e-05 2024-09-16T20:48:15.159 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.159 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.159 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:15.159 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.065060+0000", 2024-09-16T20:48:15.159 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1152000000000003e-05 2024-09-16T20:48:15.159 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.159 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.159 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:15.160 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.065082+0000", 2024-09-16T20:48:15.160 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2155e-05 2024-09-16T20:48:15.160 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:15.160 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:15.160 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:15.160 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.160 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.160 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:48:15.160 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:54.058988+0000", 2024-09-16T20:48:15.160 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 21.049229320999999, 2024-09-16T20:48:15.160 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0069229319999999997, 2024-09-16T20:48:15.160 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:15.160 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:15.160 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:15.161 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:15.161 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:48:15.161 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:48:15.161 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 49 2024-09-16T20:48:15.161 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.161 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:15.161 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.161 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:15.161 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058988+0000", 2024-09-16T20:48:15.161 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:15.161 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.161 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.161 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:15.161 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058988+0000", 2024-09-16T20:48:15.162 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:15.162 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.162 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.162 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:15.162 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058988+0000", 2024-09-16T20:48:15.162 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8299999999999998e-07 2024-09-16T20:48:15.162 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.162 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.162 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:15.162 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058992+0000", 2024-09-16T20:48:15.162 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8569999999999997e-06 2024-09-16T20:48:15.162 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.162 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.162 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:15.163 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058998+0000", 2024-09-16T20:48:15.163 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9020000000000004e-06 2024-09-16T20:48:15.163 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.163 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.163 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:15.163 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.059001+0000", 2024-09-16T20:48:15.163 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.269e-06 2024-09-16T20:48:15.163 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.163 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.163 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:15.163 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.063972+0000", 2024-09-16T20:48:15.163 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0049705690000000002 2024-09-16T20:48:15.163 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.164 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.164 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:15.164 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.064032+0000", 2024-09-16T20:48:15.164 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9834999999999999e-05 2024-09-16T20:48:15.164 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.164 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.164 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:48:15.164 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.064128+0000", 2024-09-16T20:48:15.164 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.6113999999999995e-05 2024-09-16T20:48:15.164 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.164 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.164 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:15.164 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.065476+0000", 2024-09-16T20:48:15.164 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00134826 2024-09-16T20:48:15.165 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.165 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.165 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:15.165 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.065855+0000", 2024-09-16T20:48:15.165 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00037943500000000002 2024-09-16T20:48:15.165 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.165 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.165 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:15.165 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.065886+0000", 2024-09-16T20:48:15.165 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0787999999999997e-05 2024-09-16T20:48:15.165 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.165 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.165 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:15.166 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.065911+0000", 2024-09-16T20:48:15.166 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.4519999999999999e-05 2024-09-16T20:48:15.166 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:15.166 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:15.166 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:15.166 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.166 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.166 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:48:15.166 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:54.059010+0000", 2024-09-16T20:48:15.166 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 21.049207158000002, 2024-09-16T20:48:15.166 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0069966389999999998, 2024-09-16T20:48:15.166 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:15.166 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:15.167 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:15.167 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:15.167 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:48:15.167 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:48:15.167 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 50 2024-09-16T20:48:15.167 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.167 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:15.167 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.167 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:15.167 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.059010+0000", 2024-09-16T20:48:15.167 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:15.167 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.167 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.167 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:15.167 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.059010+0000", 2024-09-16T20:48:15.168 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:15.168 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.168 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.168 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:15.168 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.059010+0000", 2024-09-16T20:48:15.168 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.72e-07 2024-09-16T20:48:15.168 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.168 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.168 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:15.168 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.059067+0000", 2024-09-16T20:48:15.168 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6756999999999999e-05 2024-09-16T20:48:15.168 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.168 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.168 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:15.169 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.059075+0000", 2024-09-16T20:48:15.169 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8560000000000007e-06 2024-09-16T20:48:15.169 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.169 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.169 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:15.169 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.059078+0000", 2024-09-16T20:48:15.169 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6720000000000002e-06 2024-09-16T20:48:15.169 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.169 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.169 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:15.169 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.064369+0000", 2024-09-16T20:48:15.169 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.005290431 2024-09-16T20:48:15.169 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.169 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.170 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:15.170 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.064416+0000", 2024-09-16T20:48:15.170 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6961e-05 2024-09-16T20:48:15.170 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.170 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.170 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:48:15.170 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.064491+0000", 2024-09-16T20:48:15.170 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5511000000000003e-05 2024-09-16T20:48:15.170 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.170 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.170 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:15.170 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.065922+0000", 2024-09-16T20:48:15.170 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001430728 2024-09-16T20:48:15.170 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.170 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.171 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:15.171 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.065955+0000", 2024-09-16T20:48:15.171 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2741999999999999e-05 2024-09-16T20:48:15.171 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.171 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.171 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:15.171 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.065983+0000", 2024-09-16T20:48:15.171 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8415000000000002e-05 2024-09-16T20:48:15.171 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.171 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.171 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:15.171 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.066007+0000", 2024-09-16T20:48:15.171 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3294000000000001e-05 2024-09-16T20:48:15.171 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:15.171 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:15.172 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:15.172 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.172 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.172 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:48:15.172 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:54.059085+0000", 2024-09-16T20:48:15.172 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 21.049132060000002, 2024-09-16T20:48:15.172 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0070804489999999999, 2024-09-16T20:48:15.172 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:15.172 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:15.172 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:15.172 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:15.172 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:48:15.172 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:48:15.173 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 51 2024-09-16T20:48:15.173 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.173 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:15.173 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.173 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:15.173 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.059085+0000", 2024-09-16T20:48:15.173 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:15.173 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.173 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.173 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:15.173 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.059085+0000", 2024-09-16T20:48:15.173 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:15.173 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.173 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.174 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:15.174 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.059085+0000", 2024-09-16T20:48:15.174 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3300000000000003e-07 2024-09-16T20:48:15.174 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.174 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.174 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:15.174 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.059088+0000", 2024-09-16T20:48:15.174 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7700000000000002e-06 2024-09-16T20:48:15.174 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.174 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.174 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:15.174 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.059092+0000", 2024-09-16T20:48:15.174 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6040000000000001e-06 2024-09-16T20:48:15.174 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.174 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.175 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:15.175 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.059100+0000", 2024-09-16T20:48:15.175 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1089999999999998e-06 2024-09-16T20:48:15.175 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.175 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.175 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:15.175 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.065129+0000", 2024-09-16T20:48:15.175 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.006028594 2024-09-16T20:48:15.175 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.175 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.175 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:15.175 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.065175+0000", 2024-09-16T20:48:15.175 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6842999999999999e-05 2024-09-16T20:48:15.176 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.176 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.176 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:48:15.176 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.065251+0000", 2024-09-16T20:48:15.176 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5278000000000007e-05 2024-09-16T20:48:15.176 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.176 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.176 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:15.176 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.066022+0000", 2024-09-16T20:48:15.176 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00077180800000000002 2024-09-16T20:48:15.176 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.176 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.176 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:15.177 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.066118+0000", 2024-09-16T20:48:15.177 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.5594999999999995e-05 2024-09-16T20:48:15.177 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.177 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.177 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:15.177 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.066145+0000", 2024-09-16T20:48:15.177 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7089000000000001e-05 2024-09-16T20:48:15.177 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.177 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.177 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:15.177 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.066165+0000", 2024-09-16T20:48:15.177 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0326000000000002e-05 2024-09-16T20:48:15.177 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:15.178 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:15.178 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:15.178 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.178 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.178 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:48:15.178 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:54.059107+0000", 2024-09-16T20:48:15.178 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 21.049110008, 2024-09-16T20:48:15.178 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0073197189999999997, 2024-09-16T20:48:15.178 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:15.178 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:15.178 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:15.178 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:15.178 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:48:15.178 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:48:15.178 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 52 2024-09-16T20:48:15.178 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.179 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:15.179 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.179 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:15.179 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.059107+0000", 2024-09-16T20:48:15.179 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:15.179 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.179 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.179 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:15.179 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.059107+0000", 2024-09-16T20:48:15.179 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:15.179 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.179 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.179 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:15.179 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.059107+0000", 2024-09-16T20:48:15.179 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4700000000000002e-07 2024-09-16T20:48:15.179 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.180 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.180 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:15.180 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.059110+0000", 2024-09-16T20:48:15.180 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8499999999999998e-06 2024-09-16T20:48:15.180 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.180 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.180 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:15.180 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.059116+0000", 2024-09-16T20:48:15.180 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0519999999999998e-06 2024-09-16T20:48:15.180 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.180 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.180 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:15.180 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.059119+0000", 2024-09-16T20:48:15.181 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0290000000000001e-06 2024-09-16T20:48:15.181 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.181 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.181 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:15.181 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.065513+0000", 2024-09-16T20:48:15.181 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0063932260000000001 2024-09-16T20:48:15.181 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.181 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.181 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:15.181 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.065571+0000", 2024-09-16T20:48:15.181 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8758000000000002e-05 2024-09-16T20:48:15.181 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.181 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.181 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:48:15.181 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.065648+0000", 2024-09-16T20:48:15.182 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7039999999999994e-05 2024-09-16T20:48:15.182 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.182 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.182 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:15.182 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.066180+0000", 2024-09-16T20:48:15.182 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00053161099999999998 2024-09-16T20:48:15.182 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.182 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.182 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:15.182 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.066381+0000", 2024-09-16T20:48:15.182 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000200564 2024-09-16T20:48:15.182 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.182 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.182 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:15.182 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.066408+0000", 2024-09-16T20:48:15.183 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.775e-05 2024-09-16T20:48:15.183 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.183 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.183 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:15.183 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.066427+0000", 2024-09-16T20:48:15.183 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8491999999999998e-05 2024-09-16T20:48:15.183 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:15.183 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:15.183 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:15.183 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.183 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.183 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:69 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:48:15.183 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:54.239973+0000", 2024-09-16T20:48:15.183 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 20.868243822, 2024-09-16T20:48:15.184 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0043631160000000002, 2024-09-16T20:48:15.184 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:15.184 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:15.184 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:15.184 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:15.184 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:48:15.184 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:48:15.184 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 69 2024-09-16T20:48:15.184 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.184 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:15.184 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.184 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:15.184 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.239973+0000", 2024-09-16T20:48:15.185 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:15.185 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.185 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.185 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:15.185 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.239973+0000", 2024-09-16T20:48:15.185 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:15.185 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.185 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.185 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:15.185 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.239974+0000", 2024-09-16T20:48:15.185 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.63e-07 2024-09-16T20:48:15.185 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.185 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.185 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:15.186 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240069+0000", 2024-09-16T20:48:15.186 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4907000000000005e-05 2024-09-16T20:48:15.186 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.186 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.186 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:15.186 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240081+0000", 2024-09-16T20:48:15.186 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2500000000000001e-05 2024-09-16T20:48:15.186 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.186 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.186 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:15.186 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240101+0000", 2024-09-16T20:48:15.186 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.016e-05 2024-09-16T20:48:15.187 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.187 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.187 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:15.187 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.242185+0000", 2024-09-16T20:48:15.187 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0020841179999999998 2024-09-16T20:48:15.187 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.187 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.187 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:15.187 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.242230+0000", 2024-09-16T20:48:15.187 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5077000000000002e-05 2024-09-16T20:48:15.187 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.187 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.187 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:48:15.187 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.242296+0000", 2024-09-16T20:48:15.188 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5732e-05 2024-09-16T20:48:15.188 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.188 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.188 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:15.188 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.243310+0000", 2024-09-16T20:48:15.188 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0010134860000000001 2024-09-16T20:48:15.188 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.188 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.188 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:15.188 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.244283+0000", 2024-09-16T20:48:15.188 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00097367700000000005 2024-09-16T20:48:15.188 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.189 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.189 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:15.189 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.244315+0000", 2024-09-16T20:48:15.189 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2055999999999999e-05 2024-09-16T20:48:15.189 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.189 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.189 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:15.189 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.244336+0000", 2024-09-16T20:48:15.189 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0939999999999999e-05 2024-09-16T20:48:15.189 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:15.189 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:15.189 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:15.189 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.189 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.189 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:48:15.190 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:54.240113+0000", 2024-09-16T20:48:15.190 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 20.868103732000002, 2024-09-16T20:48:15.190 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0043271009999999999, 2024-09-16T20:48:15.190 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:15.190 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:15.190 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:15.190 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:15.190 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:48:15.190 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:48:15.190 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 70 2024-09-16T20:48:15.190 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.190 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:15.190 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.190 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:15.190 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240113+0000", 2024-09-16T20:48:15.191 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:15.191 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.191 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.191 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:15.191 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240113+0000", 2024-09-16T20:48:15.191 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:15.191 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.191 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.191 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:15.191 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240114+0000", 2024-09-16T20:48:15.191 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9999999999999998e-07 2024-09-16T20:48:15.191 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.191 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.191 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:15.192 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240117+0000", 2024-09-16T20:48:15.192 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8210000000000002e-06 2024-09-16T20:48:15.192 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.192 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.192 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:15.192 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240122+0000", 2024-09-16T20:48:15.192 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0379999999999999e-06 2024-09-16T20:48:15.192 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.192 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.192 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:15.192 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240143+0000", 2024-09-16T20:48:15.192 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1568000000000001e-05 2024-09-16T20:48:15.192 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.192 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.193 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:15.193 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.242735+0000", 2024-09-16T20:48:15.193 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0025919699999999999 2024-09-16T20:48:15.193 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.193 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.193 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:15.193 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.242783+0000", 2024-09-16T20:48:15.193 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7528999999999998e-05 2024-09-16T20:48:15.193 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.193 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.193 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:48:15.193 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.242856+0000", 2024-09-16T20:48:15.193 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3362999999999995e-05 2024-09-16T20:48:15.194 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.194 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.194 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:15.194 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.244378+0000", 2024-09-16T20:48:15.194 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00152242 2024-09-16T20:48:15.194 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.194 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.194 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:15.194 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.244388+0000", 2024-09-16T20:48:15.194 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8630000000000003e-06 2024-09-16T20:48:15.194 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.194 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.194 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:15.195 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.244418+0000", 2024-09-16T20:48:15.195 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9482e-05 2024-09-16T20:48:15.195 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.195 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.195 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:15.195 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.244440+0000", 2024-09-16T20:48:15.195 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2646999999999998e-05 2024-09-16T20:48:15.195 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:15.195 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:15.195 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:15.195 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.195 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.195 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:48:15.196 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:54.240150+0000", 2024-09-16T20:48:15.196 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 20.868067346, 2024-09-16T20:48:15.196 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0052309740000000002, 2024-09-16T20:48:15.196 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:15.196 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:15.196 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:15.196 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:15.196 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:48:15.196 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:48:15.196 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 71 2024-09-16T20:48:15.196 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.196 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:15.196 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.197 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:15.197 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240150+0000", 2024-09-16T20:48:15.197 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:15.197 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.197 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.197 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:15.197 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240150+0000", 2024-09-16T20:48:15.197 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:15.197 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.197 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.197 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:15.197 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240150+0000", 2024-09-16T20:48:15.197 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2199999999999999e-07 2024-09-16T20:48:15.197 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.198 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.198 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:15.198 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240153+0000", 2024-09-16T20:48:15.198 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5679999999999998e-06 2024-09-16T20:48:15.198 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.198 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.198 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:15.198 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240157+0000", 2024-09-16T20:48:15.198 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2549999999999999e-06 2024-09-16T20:48:15.198 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.198 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.198 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:15.198 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240167+0000", 2024-09-16T20:48:15.199 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.6879999999999994e-06 2024-09-16T20:48:15.199 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.199 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.199 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:15.199 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.243340+0000", 2024-09-16T20:48:15.199 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.003173122 2024-09-16T20:48:15.199 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.199 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.199 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:15.199 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.243381+0000", 2024-09-16T20:48:15.199 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0821000000000001e-05 2024-09-16T20:48:15.199 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.199 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.200 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:48:15.200 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.243448+0000", 2024-09-16T20:48:15.200 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7148000000000002e-05 2024-09-16T20:48:15.200 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.200 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.200 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:15.200 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.244446+0000", 2024-09-16T20:48:15.200 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00099830199999999996 2024-09-16T20:48:15.200 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.200 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.200 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:15.200 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.245283+0000", 2024-09-16T20:48:15.200 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00083687200000000001 2024-09-16T20:48:15.201 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.201 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.201 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:15.201 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.245341+0000", 2024-09-16T20:48:15.201 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8238e-05 2024-09-16T20:48:15.201 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.201 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.201 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:15.201 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.245381+0000", 2024-09-16T20:48:15.201 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9538e-05 2024-09-16T20:48:15.201 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:15.201 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:15.201 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:15.201 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.202 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.202 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:48:15.202 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:54.240178+0000", 2024-09-16T20:48:15.202 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 20.868038922, 2024-09-16T20:48:15.202 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0059449899999999998, 2024-09-16T20:48:15.202 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:15.202 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:15.202 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:15.202 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:15.202 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:48:15.202 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:48:15.202 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 72 2024-09-16T20:48:15.202 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.202 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:15.203 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.203 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:15.203 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240178+0000", 2024-09-16T20:48:15.203 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:15.203 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.203 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.203 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:15.203 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240178+0000", 2024-09-16T20:48:15.203 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:15.203 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.203 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.203 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:15.203 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240179+0000", 2024-09-16T20:48:15.203 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.53e-07 2024-09-16T20:48:15.204 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.204 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.204 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:15.204 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240252+0000", 2024-09-16T20:48:15.204 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3279000000000001e-05 2024-09-16T20:48:15.204 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.204 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.204 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:15.204 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240259+0000", 2024-09-16T20:48:15.204 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4270000000000002e-06 2024-09-16T20:48:15.204 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.204 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.204 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:15.204 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240263+0000", 2024-09-16T20:48:15.205 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7050000000000001e-06 2024-09-16T20:48:15.205 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.205 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.205 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:15.205 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.243686+0000", 2024-09-16T20:48:15.205 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0034228330000000001 2024-09-16T20:48:15.205 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.205 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.205 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:15.205 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.243731+0000", 2024-09-16T20:48:15.205 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4917999999999999e-05 2024-09-16T20:48:15.205 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.205 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.205 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:48:15.206 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.243801+0000", 2024-09-16T20:48:15.206 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0320000000000004e-05 2024-09-16T20:48:15.206 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.206 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.206 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:15.206 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.245441+0000", 2024-09-16T20:48:15.206 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001640138 2024-09-16T20:48:15.206 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.206 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.206 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:15.206 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.245996+0000", 2024-09-16T20:48:15.206 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000555128 2024-09-16T20:48:15.206 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.206 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.207 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:15.207 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.246057+0000", 2024-09-16T20:48:15.207 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0458999999999998e-05 2024-09-16T20:48:15.207 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.207 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.207 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:15.207 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.246123+0000", 2024-09-16T20:48:15.207 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6429999999999999e-05 2024-09-16T20:48:15.207 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:15.207 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:15.207 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:15.207 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.207 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.208 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:48:15.208 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:54.240274+0000", 2024-09-16T20:48:15.208 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 20.867943515, 2024-09-16T20:48:15.208 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0063725079999999998, 2024-09-16T20:48:15.208 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:15.208 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:15.208 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:15.208 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:15.208 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:48:15.208 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:48:15.208 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 73 2024-09-16T20:48:15.208 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.208 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:15.209 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.209 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:15.209 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240274+0000", 2024-09-16T20:48:15.209 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:15.209 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.209 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.209 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:15.209 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240274+0000", 2024-09-16T20:48:15.209 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:15.209 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.209 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.209 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:15.209 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240274+0000", 2024-09-16T20:48:15.210 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6100000000000001e-07 2024-09-16T20:48:15.210 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.210 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.210 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:15.210 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240280+0000", 2024-09-16T20:48:15.210 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6960000000000002e-06 2024-09-16T20:48:15.210 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.210 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.210 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:15.210 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240300+0000", 2024-09-16T20:48:15.210 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0151000000000001e-05 2024-09-16T20:48:15.210 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.210 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.210 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:15.211 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240304+0000", 2024-09-16T20:48:15.211 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8330000000000004e-06 2024-09-16T20:48:15.211 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.211 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.211 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:15.211 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.244483+0000", 2024-09-16T20:48:15.211 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0041787769999999998 2024-09-16T20:48:15.211 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.211 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.211 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:15.211 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.244534+0000", 2024-09-16T20:48:15.211 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1158e-05 2024-09-16T20:48:15.211 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.212 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.212 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:48:15.212 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.244611+0000", 2024-09-16T20:48:15.212 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6888e-05 2024-09-16T20:48:15.212 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.212 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.212 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:15.212 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.246133+0000", 2024-09-16T20:48:15.212 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00152224 2024-09-16T20:48:15.212 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.212 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.212 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:15.212 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.246555+0000", 2024-09-16T20:48:15.212 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00042180899999999999 2024-09-16T20:48:15.213 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.213 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.213 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:15.213 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.246610+0000", 2024-09-16T20:48:15.213 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5470999999999998e-05 2024-09-16T20:48:15.213 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.213 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.213 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:15.213 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.246646+0000", 2024-09-16T20:48:15.213 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5923999999999997e-05 2024-09-16T20:48:15.213 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:15.213 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:15.213 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:15.214 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.214 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.214 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:48:15.214 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:54.240313+0000", 2024-09-16T20:48:15.214 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 20.867903630000001, 2024-09-16T20:48:15.214 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0064355100000000002, 2024-09-16T20:48:15.214 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:15.214 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:15.214 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:15.214 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:15.214 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:48:15.214 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:48:15.214 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 74 2024-09-16T20:48:15.215 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.215 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:15.215 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.215 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:15.215 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240313+0000", 2024-09-16T20:48:15.215 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:15.215 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.215 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.215 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:15.215 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240313+0000", 2024-09-16T20:48:15.215 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:15.215 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.215 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.215 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:15.216 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240314+0000", 2024-09-16T20:48:15.216 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0500000000000003e-07 2024-09-16T20:48:15.216 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.216 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.216 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:15.216 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240324+0000", 2024-09-16T20:48:15.216 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8309999999999996e-06 2024-09-16T20:48:15.216 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.216 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.216 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:15.216 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240329+0000", 2024-09-16T20:48:15.216 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7870000000000001e-06 2024-09-16T20:48:15.216 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.217 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.217 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:15.217 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240332+0000", 2024-09-16T20:48:15.217 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0429999999999999e-06 2024-09-16T20:48:15.217 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.217 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.217 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:15.217 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.244880+0000", 2024-09-16T20:48:15.217 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0045483770000000001 2024-09-16T20:48:15.217 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.217 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.217 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:15.217 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.244925+0000", 2024-09-16T20:48:15.218 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4626000000000003e-05 2024-09-16T20:48:15.218 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.218 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.218 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:48:15.218 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.244994+0000", 2024-09-16T20:48:15.218 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9443999999999994e-05 2024-09-16T20:48:15.218 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.218 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.218 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:15.218 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.246659+0000", 2024-09-16T20:48:15.218 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001665267 2024-09-16T20:48:15.218 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.218 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.218 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:15.219 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.246694+0000", 2024-09-16T20:48:15.219 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4879999999999998e-05 2024-09-16T20:48:15.219 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.219 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.219 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:15.219 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.246724+0000", 2024-09-16T20:48:15.219 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0168999999999999e-05 2024-09-16T20:48:15.219 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.219 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.219 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:15.219 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.246749+0000", 2024-09-16T20:48:15.219 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.4481000000000002e-05 2024-09-16T20:48:15.219 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:15.219 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:15.220 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:15.220 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.220 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.220 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:48:15.220 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:54.240337+0000", 2024-09-16T20:48:15.220 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 20.867880034999999, 2024-09-16T20:48:15.220 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0064925900000000003, 2024-09-16T20:48:15.220 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:15.220 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:15.220 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:15.220 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:15.220 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:48:15.220 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:48:15.221 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 75 2024-09-16T20:48:15.221 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.221 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:15.221 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.221 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:15.221 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240337+0000", 2024-09-16T20:48:15.221 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:15.221 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.221 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.221 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:15.221 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240337+0000", 2024-09-16T20:48:15.221 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:15.221 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.221 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.222 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:15.222 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240337+0000", 2024-09-16T20:48:15.222 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9799999999999999e-07 2024-09-16T20:48:15.222 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.222 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.222 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:15.222 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240407+0000", 2024-09-16T20:48:15.222 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9887000000000004e-05 2024-09-16T20:48:15.222 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.222 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.222 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:15.222 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240416+0000", 2024-09-16T20:48:15.222 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.3839999999999997e-06 2024-09-16T20:48:15.223 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.223 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.223 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:15.223 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240426+0000", 2024-09-16T20:48:15.223 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0665999999999999e-05 2024-09-16T20:48:15.223 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.223 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.223 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:15.223 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.245481+0000", 2024-09-16T20:48:15.223 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0050551069999999997 2024-09-16T20:48:15.223 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.223 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.223 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:15.224 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.245552+0000", 2024-09-16T20:48:15.224 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0335999999999999e-05 2024-09-16T20:48:15.224 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.224 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.224 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:48:15.224 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.245625+0000", 2024-09-16T20:48:15.224 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3388999999999998e-05 2024-09-16T20:48:15.224 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.224 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.224 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:15.224 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.246762+0000", 2024-09-16T20:48:15.224 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001137239 2024-09-16T20:48:15.224 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.225 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.225 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:15.225 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.246784+0000", 2024-09-16T20:48:15.225 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1299000000000001e-05 2024-09-16T20:48:15.225 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.225 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.225 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:15.225 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.246808+0000", 2024-09-16T20:48:15.225 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.4244e-05 2024-09-16T20:48:15.225 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.225 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.225 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:15.225 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.246830+0000", 2024-09-16T20:48:15.225 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1741000000000001e-05 2024-09-16T20:48:15.226 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:15.226 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:15.226 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:15.226 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.226 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.226 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:48:15.226 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:54.240434+0000", 2024-09-16T20:48:15.226 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 20.867783061000001, 2024-09-16T20:48:15.226 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0067000209999999996, 2024-09-16T20:48:15.226 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:15.226 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:15.226 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:15.226 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:15.227 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:48:15.227 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:48:15.227 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 76 2024-09-16T20:48:15.227 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.227 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:15.227 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.227 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:15.227 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240434+0000", 2024-09-16T20:48:15.227 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:15.227 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.227 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.227 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:15.227 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240434+0000", 2024-09-16T20:48:15.227 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:15.228 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.228 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.228 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:15.228 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240434+0000", 2024-09-16T20:48:15.228 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6600000000000002e-07 2024-09-16T20:48:15.228 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.228 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.228 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:15.228 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240440+0000", 2024-09-16T20:48:15.228 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6550000000000001e-06 2024-09-16T20:48:15.228 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.228 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.228 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:15.228 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240445+0000", 2024-09-16T20:48:15.229 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9609999999999996e-06 2024-09-16T20:48:15.229 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.229 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.229 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:15.229 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240448+0000", 2024-09-16T20:48:15.229 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8880000000000001e-06 2024-09-16T20:48:15.229 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.229 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.229 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:15.229 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.246163+0000", 2024-09-16T20:48:15.229 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0057155269999999998 2024-09-16T20:48:15.229 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.229 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.229 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:15.230 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.246206+0000", 2024-09-16T20:48:15.230 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2379000000000002e-05 2024-09-16T20:48:15.230 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.230 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.230 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:48:15.230 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.246279+0000", 2024-09-16T20:48:15.230 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2889e-05 2024-09-16T20:48:15.230 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.230 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.230 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:15.230 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.246840+0000", 2024-09-16T20:48:15.230 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00056115099999999997 2024-09-16T20:48:15.230 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.230 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.231 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:15.231 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.247077+0000", 2024-09-16T20:48:15.231 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00023753700000000001 2024-09-16T20:48:15.231 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.231 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.231 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:15.231 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.247112+0000", 2024-09-16T20:48:15.231 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4282000000000001e-05 2024-09-16T20:48:15.231 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:15.231 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:15.231 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:15.231 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.247134+0000", 2024-09-16T20:48:15.231 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2385999999999999e-05 2024-09-16T20:48:15.231 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:15.232 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:15.232 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:15.232 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:15.232 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:15.232 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:15.233 INFO:teuthology.orchestra.run.smithi139.stdout:229 trunc from 0x11fb800 to 0x3a5c00 2024-09-16T20:48:15.234 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:48:15.234 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:15.232+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:15.325 INFO:teuthology.orchestra.run.smithi139.stdout:230 trunc from 0x3a5c00 to 0x5d73c00 2024-09-16T20:48:15.333 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:15.332+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:15.350 INFO:teuthology.orchestra.run.smithi139.stdout:232 write 0x8026c00 thru 0x8027cb4 (0x10b5 bytes) 2024-09-16T20:48:15.352 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T20:48:15.358 INFO:teuthology.orchestra.run.smithi139.stdout:235 read 0x5c9c600 thru 0x5ca9e7f (0xd880 bytes) 2024-09-16T20:48:15.358 INFO:teuthology.orchestra.run.smithi139.stdout:237 write 0x31d7c00 thru 0x31dff09 (0x830a bytes) 2024-09-16T20:48:15.358 INFO:teuthology.orchestra.run.smithi139.stdout:238 punch from 0x23bea00 to 0x23c5200, (0x6800 bytes) 2024-09-16T20:48:15.359 INFO:teuthology.orchestra.run.smithi139.stdout:239 read 0x5a51000 thru 0x5a56749 (0x574a bytes) 2024-09-16T20:48:15.359 INFO:teuthology.orchestra.run.smithi139.stdout:241 write 0x7d5ce00 thru 0x7d65689 (0x888a bytes) 2024-09-16T20:48:15.359 INFO:teuthology.orchestra.run.smithi139.stdout:243 read 0x3cdd800 thru 0x3ce95ee (0xbdef bytes) 2024-09-16T20:48:15.362 INFO:teuthology.orchestra.run.smithi139.stdout:244 trunc from 0x8027e00 to 0xa98ce00 2024-09-16T20:48:15.376 INFO:teuthology.orchestra.run.smithi139.stdout:245 read 0x3dfac00 thru 0x3e0734c (0xc74d bytes) 2024-09-16T20:48:15.377 INFO:teuthology.orchestra.run.smithi139.stdout:246 read 0x95aaa00 thru 0x95abab7 (0x10b8 bytes) 2024-09-16T20:48:15.377 INFO:teuthology.orchestra.run.smithi139.stdout:247 write 0xa31c200 thru 0xa32a032 (0xde33 bytes) 2024-09-16T20:48:15.380 INFO:teuthology.orchestra.run.smithi139.stdout:248 trunc from 0xa98ce00 to 0x7905600 2024-09-16T20:48:15.402 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:15.402 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:48:15.403 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:48:15.403 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:15.413 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:48:15.420 INFO:teuthology.orchestra.run.smithi139.stdout:249 write 0xd030e00 thru 0xd03e00e (0xd20f bytes) 2024-09-16T20:48:15.420 INFO:teuthology.orchestra.run.smithi139.stdout:251 read 0xb504400 thru 0xb507579 (0x317a bytes) 2024-09-16T20:48:15.421 INFO:teuthology.orchestra.run.smithi139.stdout:254 punch from 0xb235c00 to 0xb23dc00, (0x8000 bytes) 2024-09-16T20:48:15.421 INFO:teuthology.orchestra.run.smithi139.stdout:255 write 0xf4bc00 thru 0xf4d1ab (0x15ac bytes) 2024-09-16T20:48:15.421 INFO:teuthology.orchestra.run.smithi139.stdout:256 write 0xad3a000 thru 0xad3f7d9 (0x57da bytes) 2024-09-16T20:48:15.421 INFO:teuthology.orchestra.run.smithi139.stdout:259 read 0x6907800 thru 0x690acb2 (0x34b3 bytes) 2024-09-16T20:48:15.421 INFO:teuthology.orchestra.run.smithi139.stdout:260 read 0x10efa00 thru 0x10f80c2 (0x86c3 bytes) 2024-09-16T20:48:15.423 INFO:teuthology.orchestra.run.smithi139.stdout:261 trunc from 0xd03e200 to 0xb1f5c00 2024-09-16T20:48:15.440 INFO:teuthology.orchestra.run.smithi139.stdout:262 read 0x4564600 thru 0x456d313 (0x8d14 bytes) 2024-09-16T20:48:15.441 INFO:teuthology.orchestra.run.smithi139.stdout:263 punch from 0x575de00 to 0x576a800, (0xca00 bytes) 2024-09-16T20:48:15.442 INFO:teuthology.orchestra.run.smithi139.stdout:265 write 0x7b56600 thru 0x7b5e61e (0x801f bytes) 2024-09-16T20:48:15.442 INFO:teuthology.orchestra.run.smithi139.stdout:267 read 0xaf6da00 thru 0xaf76b9f (0x91a0 bytes) 2024-09-16T20:48:15.442 INFO:teuthology.orchestra.run.smithi139.stdout:268 write 0xfc5800 thru 0xfd430d (0xeb0e bytes) 2024-09-16T20:48:15.443 INFO:teuthology.orchestra.run.smithi139.stdout:269 punch from 0x62d8000 to 0x62e7a00, (0xfa00 bytes) 2024-09-16T20:48:15.443 INFO:teuthology.orchestra.run.smithi139.stdout:270 write 0x6bad000 thru 0x6baed05 (0x1d06 bytes) 2024-09-16T20:48:15.444 INFO:teuthology.orchestra.run.smithi139.stdout:271 punch from 0x2698e00 to 0x26a6400, (0xd600 bytes) 2024-09-16T20:48:15.444 INFO:teuthology.orchestra.run.smithi139.stdout:277 read 0x169de00 thru 0x16a712f (0x9330 bytes) 2024-09-16T20:48:15.444 INFO:teuthology.orchestra.run.smithi139.stdout:278 write 0x9126400 thru 0x912faf0 (0x96f1 bytes) 2024-09-16T20:48:15.444 INFO:teuthology.orchestra.run.smithi139.stdout:280 clone 3 order 24 su 8388608 sc 11 2024-09-16T20:48:15.456 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:15.455+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:15.497 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:48:15.497 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:48:15.514 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:48:15.515 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:48:15.529 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:15.530 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:48:15.530 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:48:15.530 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:48:15.530 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:15.531 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:48:15.531 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:48:15.535 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:15.534+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:15.539 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:48:15.548 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:48:15.549 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:48:15.566 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:48:15.566 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:48:15.584 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:48:15.584 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:48:15.601 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T20:48:15.636 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:15.635+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:15.663 INFO:teuthology.orchestra.run.smithi139.stdout:truncating image image_client.0-clone2 from 0xb1f5c00 (overlap 0x3a5c00) to 0x1d2e00 2024-09-16T20:48:15.674 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:15.674 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:15.686 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:48:15.737 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:15.736+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:15.797 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:15.797 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:15.806 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:48:15.838 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:15.837+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:15.918 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:15.918 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:15.927 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:48:15.956 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #1, image image_client.0-clone1 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent2 2024-09-16T20:48:15.970 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:15.969+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:16.037 INFO:teuthology.orchestra.run.smithi139.stdout:281 write 0xbb40400 thru 0xbb4bdbc (0xb9bd bytes) 2024-09-16T20:48:16.039 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:16.037+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:16.042 INFO:teuthology.orchestra.run.smithi139.stdout:282 write 0x4161e00 thru 0x4165fa7 (0x41a8 bytes) 2024-09-16T20:48:16.042 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:16.042 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:16.042 INFO:teuthology.orchestra.run.smithi139.stdout:283 trunc from 0xbb4be00 to 0xb19f000 2024-09-16T20:48:16.052 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:48:16.060 INFO:teuthology.orchestra.run.smithi139.stdout:287 read 0x6139000 thru 0x61432c9 (0xa2ca bytes) 2024-09-16T20:48:16.061 INFO:teuthology.orchestra.run.smithi139.stdout:290 trunc from 0xb19f000 to 0xa9a0a00 2024-09-16T20:48:16.087 INFO:teuthology.orchestra.run.smithi139.stdout:293 write 0xb9f0800 thru 0xba00782 (0xff83 bytes) 2024-09-16T20:48:16.087 INFO:teuthology.orchestra.run.smithi139.stdout:297 read 0x499b400 thru 0x49aaff5 (0xfbf6 bytes) 2024-09-16T20:48:16.087 INFO:teuthology.orchestra.run.smithi139.stdout:301 write 0x5572800 thru 0x557fb7b (0xd37c bytes) 2024-09-16T20:48:16.091 INFO:teuthology.orchestra.run.smithi139.stdout:302 trunc from 0xba00800 to 0x8c09800 2024-09-16T20:48:16.114 INFO:teuthology.orchestra.run.smithi139.stdout:303 read 0x8baf000 thru 0x8bb9249 (0xa24a bytes) 2024-09-16T20:48:16.129 INFO:teuthology.orchestra.run.smithi139.stdout:304 write 0xc732e00 thru 0xc741674 (0xe875 bytes) 2024-09-16T20:48:16.130 INFO:teuthology.orchestra.run.smithi139.stdout:306 write 0xaf44800 thru 0xaf4fd4c (0xb54d bytes) 2024-09-16T20:48:16.131 INFO:teuthology.orchestra.run.smithi139.stdout:307 punch from 0x44ad800 to 0x44b0000, (0x2800 bytes) 2024-09-16T20:48:16.131 INFO:teuthology.orchestra.run.smithi139.stdout:308 write 0x3071200 thru 0x307bcb0 (0xaab1 bytes) 2024-09-16T20:48:16.132 INFO:teuthology.orchestra.run.smithi139.stdout:311 write 0x9f2ee00 thru 0x9f38eb1 (0xa0b2 bytes) 2024-09-16T20:48:16.140 INFO:teuthology.orchestra.run.smithi139.stdout:312 trunc from 0xc741800 to 0x7438400 2024-09-16T20:48:16.140 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:16.139+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:16.162 INFO:teuthology.orchestra.run.smithi139.stdout:313 read 0x6717000 thru 0x6723659 (0xc65a bytes) 2024-09-16T20:48:16.169 INFO:teuthology.orchestra.run.smithi139.stdout:314 trunc from 0x7438400 to 0x26ebc00 2024-09-16T20:48:16.211 INFO:teuthology.orchestra.run.smithi139.stdout:324 punch from 0x182d600 to 0x1831c00, (0x4600 bytes) 2024-09-16T20:48:16.213 INFO:teuthology.orchestra.run.smithi139.stdout:325 read 0x1fff000 thru 0x1fffb76 (0xb77 bytes) 2024-09-16T20:48:16.215 INFO:teuthology.orchestra.run.smithi139.stdout:326 read 0x1076200 thru 0x107de44 (0x7c45 bytes) 2024-09-16T20:48:16.217 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:16.217 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:16.221 INFO:teuthology.orchestra.run.smithi139.stdout:327 trunc from 0x26ebc00 to 0x6e1c800 2024-09-16T20:48:16.227 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:48:16.246 INFO:teuthology.orchestra.run.smithi139.stdout:328 write 0xb7ece00 thru 0xb7f88a7 (0xbaa8 bytes) 2024-09-16T20:48:16.246 INFO:teuthology.orchestra.run.smithi139.stdout:330 read 0xb563400 thru 0xb56a114 (0x6d15 bytes) 2024-09-16T20:48:16.246 INFO:teuthology.orchestra.run.smithi139.stdout:334 read 0x1b11c00 thru 0x1b1abd8 (0x8fd9 bytes) 2024-09-16T20:48:16.262 INFO:teuthology.orchestra.run.smithi139.stdout:335 write 0xdbb5600 thru 0xdbc1f50 (0xc951 bytes) 2024-09-16T20:48:16.268 INFO:teuthology.orchestra.run.smithi139.stdout:336 read 0x96b7400 thru 0x96bf686 (0x8287 bytes) 2024-09-16T20:48:16.270 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:16.269+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:16.279 INFO:teuthology.orchestra.run.smithi139.stdout:337 trunc from 0xdbc2000 to 0x46d9200 2024-09-16T20:48:16.298 INFO:teuthology.orchestra.run.smithi139.stdout:339 write 0x373000 thru 0x381ad7 (0xead8 bytes) 2024-09-16T20:48:16.314 INFO:teuthology.orchestra.run.smithi139.stdout:340 trunc from 0x46d9200 to 0xc599800 2024-09-16T20:48:16.325 INFO:teuthology.orchestra.run.smithi139.stdout:342 write 0x1f3f800 thru 0x1f472be (0x7abf bytes) 2024-09-16T20:48:16.332 INFO:teuthology.orchestra.run.smithi139.stdout:343 punch from 0x2621200 to 0x2622800, (0x1600 bytes) 2024-09-16T20:48:16.333 INFO:teuthology.orchestra.run.smithi139.stdout:344 read 0x8c07c00 thru 0x8c1579a (0xdb9b bytes) 2024-09-16T20:48:16.333 INFO:teuthology.orchestra.run.smithi139.stdout:345 read 0x4596a00 thru 0x45a540f (0xea10 bytes) 2024-09-16T20:48:16.333 INFO:teuthology.orchestra.run.smithi139.stdout:347 write 0x2624600 thru 0x262f0bc (0xaabd bytes) 2024-09-16T20:48:16.342 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:16.341+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:16.345 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:16.345 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:16.346 INFO:teuthology.orchestra.run.smithi139.stdout:349 write 0xe7c0400 thru 0xe7c21e3 (0x1de4 bytes) 2024-09-16T20:48:16.354 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:48:16.355 INFO:teuthology.orchestra.run.smithi139.stdout:350 read 0x7ea8000 thru 0x7eb37b7 (0xb7b8 bytes) 2024-09-16T20:48:16.356 INFO:teuthology.orchestra.run.smithi139.stdout:352 punch from 0xa075400 to 0xa07b200, (0x5e00 bytes) 2024-09-16T20:48:16.356 INFO:teuthology.orchestra.run.smithi139.stdout:354 write 0x9d8de00 thru 0x9d9b63d (0xd83e bytes) 2024-09-16T20:48:16.356 INFO:teuthology.orchestra.run.smithi139.stdout:356 write 0x9db6a00 thru 0x9dc1b9c (0xb19d bytes) 2024-09-16T20:48:16.356 INFO:teuthology.orchestra.run.smithi139.stdout:357 read 0x631b200 thru 0x63289fb (0xd7fc bytes) 2024-09-16T20:48:16.356 INFO:teuthology.orchestra.run.smithi139.stdout:358 read 0xd446e00 thru 0xd4485d4 (0x17d5 bytes) 2024-09-16T20:48:16.356 INFO:teuthology.orchestra.run.smithi139.stdout:359 write 0xe20d200 thru 0xe2186de (0xb4df bytes) 2024-09-16T20:48:16.356 INFO:teuthology.orchestra.run.smithi139.stdout:363 read 0x7240a00 thru 0x7245a59 (0x505a bytes) 2024-09-16T20:48:16.359 INFO:teuthology.orchestra.run.smithi139.stdout:367 trunc from 0xe7c2200 to 0xbd88400 2024-09-16T20:48:16.381 INFO:teuthology.orchestra.run.smithi139.stdout:368 write 0x47f2400 thru 0x47fecae (0xc8af bytes) 2024-09-16T20:48:16.381 INFO:teuthology.orchestra.run.smithi139.stdout:369 write 0x8cc0400 thru 0x8cce23f (0xde40 bytes) 2024-09-16T20:48:16.381 INFO:teuthology.orchestra.run.smithi139.stdout:370 read 0x40400 thru 0x4aeea (0xaaeb bytes) 2024-09-16T20:48:16.382 INFO:teuthology.orchestra.run.smithi139.stdout:371 flatten 2024-09-16T20:48:16.395 INFO:teuthology.orchestra.run.smithi139.stdout:374 read 0xa1e000 thru 0xa240d4 (0x60d5 bytes) 2024-09-16T20:48:16.396 INFO:teuthology.orchestra.run.smithi139.stdout:375 read 0xba8fa00 thru 0xba908ef (0xef0 bytes) 2024-09-16T20:48:16.396 INFO:teuthology.orchestra.run.smithi139.stdout:376 write 0x820fe00 thru 0x821fc94 (0xfe95 bytes) 2024-09-16T20:48:16.396 INFO:teuthology.orchestra.run.smithi139.stdout:378 read 0x10e6400 thru 0x10eca07 (0x6608 bytes) 2024-09-16T20:48:16.413 INFO:teuthology.orchestra.run.smithi139.stdout:379 write 0xed48200 thru 0xed4d60b (0x540c bytes) 2024-09-16T20:48:16.419 INFO:teuthology.orchestra.run.smithi139.stdout:381 punch from 0x8187800 to 0x8188a00, (0x1200 bytes) 2024-09-16T20:48:16.420 INFO:teuthology.orchestra.run.smithi139.stdout:384 write 0x29f3a00 thru 0x29fc6db (0x8cdc bytes) 2024-09-16T20:48:16.429 INFO:teuthology.orchestra.run.smithi139.stdout:385 trunc from 0xed4d800 to 0x895c400 2024-09-16T20:48:16.444 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:16.443+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:16.451 INFO:teuthology.orchestra.run.smithi139.stdout:386 trunc from 0x895c400 to 0x3d77e00 2024-09-16T20:48:16.487 INFO:teuthology.orchestra.run.smithi139.stdout:389 write 0x592d600 thru 0x592f105 (0x1b06 bytes) 2024-09-16T20:48:16.494 INFO:teuthology.orchestra.run.smithi139.stdout:390 punch from 0x2a7d000 to 0x2a8ae00, (0xde00 bytes) 2024-09-16T20:48:16.494 INFO:teuthology.orchestra.run.smithi139.stdout:391 read 0x2c26600 thru 0x2c2c95d (0x635e bytes) 2024-09-16T20:48:16.503 INFO:teuthology.orchestra.run.smithi139.stdout:392 trunc from 0x592f200 to 0xb92dc00 2024-09-16T20:48:16.514 INFO:teuthology.orchestra.run.smithi139.stdout:393 clone 4 order 24 su 524288 sc 11 2024-09-16T20:48:16.516 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:16.516 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:16.526 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:48:16.544 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:16.543+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:16.593 INFO:teuthology.orchestra.run.smithi029.stderr:nodeep-scrub is unset 2024-09-16T20:48:16.645 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:16.644+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:16.681 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:16.681 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:16.690 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:48:16.773 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:16.772+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:16.839 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:16.839 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:16.847 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:48:16.848 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:16.846+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:16.948 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:16.947+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:16.991 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #2, image image_client.0-clone2 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent3 2024-09-16T20:48:17.001 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:17.001 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:17.010 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:48:17.047 INFO:teuthology.orchestra.run.smithi139.stdout:394 write 0x1852600 thru 0x1856aba (0x44bb bytes) 2024-09-16T20:48:17.049 INFO:teuthology.orchestra.run.smithi139.stdout:395 write 0x243b000 thru 0x243ba74 (0xa75 bytes) 2024-09-16T20:48:17.054 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:17.053+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:17.073 INFO:teuthology.orchestra.run.smithi139.stdout:396 write 0xe09fa00 thru 0xe0a4ec9 (0x54ca bytes) 2024-09-16T20:48:17.074 INFO:teuthology.orchestra.run.smithi139.stdout:397 write 0x2392c00 thru 0x239ae01 (0x8202 bytes) 2024-09-16T20:48:17.078 INFO:teuthology.orchestra.run.smithi139.stdout:399 read 0xb9cfe00 thru 0xb9d9dde (0x9fdf bytes) 2024-09-16T20:48:17.078 INFO:teuthology.orchestra.run.smithi139.stdout:400 punch from 0xb470600 to 0xb475a00, (0x5400 bytes) 2024-09-16T20:48:17.078 INFO:teuthology.orchestra.run.smithi139.stdout:401 write 0x2b19e00 thru 0x2b2557a (0xb77b bytes) 2024-09-16T20:48:17.080 INFO:teuthology.orchestra.run.smithi139.stdout:402 write 0xdce5800 thru 0xdce8158 (0x2959 bytes) 2024-09-16T20:48:17.080 INFO:teuthology.orchestra.run.smithi139.stdout:405 punch from 0xc494400 to 0xc49d200, (0x8e00 bytes) 2024-09-16T20:48:17.080 INFO:teuthology.orchestra.run.smithi139.stdout:407 punch from 0x8a28c00 to 0x8a33600, (0xaa00 bytes) 2024-09-16T20:48:17.081 INFO:teuthology.orchestra.run.smithi139.stdout:410 read 0x69e8a00 thru 0x69ea6af (0x1cb0 bytes) 2024-09-16T20:48:17.083 INFO:teuthology.orchestra.run.smithi139.stdout:412 write 0xb5e000 thru 0xb6c5e1 (0xe5e2 bytes) 2024-09-16T20:48:17.084 INFO:teuthology.orchestra.run.smithi139.stdout:414 punch from 0xcb07400 to 0xcb09200, (0x1e00 bytes) 2024-09-16T20:48:17.084 INFO:teuthology.orchestra.run.smithi139.stdout:416 write 0x3fac00 thru 0x3ff235 (0x4636 bytes) 2024-09-16T20:48:17.085 INFO:teuthology.orchestra.run.smithi139.stdout:417 write 0xdc2e000 thru 0xdc3807e (0xa07f bytes) 2024-09-16T20:48:17.085 INFO:teuthology.orchestra.run.smithi139.stdout:419 read 0x32da800 thru 0x32e807c (0xd87d bytes) 2024-09-16T20:48:17.086 INFO:teuthology.orchestra.run.smithi139.stdout:420 read 0xa305200 thru 0xa30dd4f (0x8b50 bytes) 2024-09-16T20:48:17.086 INFO:teuthology.orchestra.run.smithi139.stdout:truncating to largest ever: 0xea87200 2024-09-16T20:48:17.087 INFO:teuthology.orchestra.run.smithi139.stdout:422 trunc from 0xe0a5000 to 0xea87200 2024-09-16T20:48:17.102 INFO:teuthology.orchestra.run.smithi139.stdout:424 punch from 0xdf32000 to 0xdf3ac00, (0x8c00 bytes) 2024-09-16T20:48:17.104 INFO:teuthology.orchestra.run.smithi139.stdout:425 read 0x12a6400 thru 0x12aaa54 (0x4655 bytes) 2024-09-16T20:48:17.105 INFO:teuthology.orchestra.run.smithi139.stdout:426 read 0x9f81e00 thru 0x9f90cdd (0xeede bytes) 2024-09-16T20:48:17.105 INFO:teuthology.orchestra.run.smithi139.stdout:427 write 0xbf72800 thru 0xbf78f43 (0x6744 bytes) 2024-09-16T20:48:17.106 INFO:teuthology.orchestra.run.smithi139.stdout:429 write 0x8662600 thru 0x8671008 (0xea09 bytes) 2024-09-16T20:48:17.115 INFO:teuthology.orchestra.run.smithi139.stdout:430 trunc from 0xea87200 to 0x765a600 2024-09-16T20:48:17.135 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:17.136 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:17.143 INFO:teuthology.orchestra.run.smithi139.stdout:431 read 0xadda00 thru 0xae9f4c (0xc54d bytes) 2024-09-16T20:48:17.145 INFO:teuthology.orchestra.run.smithi139.stdout:433 write 0x72f9c00 thru 0x7308b91 (0xef92 bytes) 2024-09-16T20:48:17.145 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:48:17.148 INFO:teuthology.orchestra.run.smithi139.stdout:436 trunc from 0x765a600 to 0x43f4000 2024-09-16T20:48:17.151 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:17.149+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:17.164 INFO:teuthology.orchestra.run.smithi139.stdout:437 trunc from 0x43f4000 to 0x555a000 2024-09-16T20:48:17.179 INFO:teuthology.orchestra.run.smithi139.stdout:442 read 0x41a0600 thru 0x41ae2e1 (0xdce2 bytes) 2024-09-16T20:48:17.193 INFO:teuthology.orchestra.run.smithi139.stdout:443 write 0x8754e00 thru 0x875c668 (0x7869 bytes) 2024-09-16T20:48:17.196 INFO:teuthology.orchestra.run.smithi139.stdout:444 trunc from 0x875c800 to 0x8fcfa00 2024-09-16T20:48:17.205 INFO:teuthology.orchestra.run.smithi139.stdout:445 punch from 0x578ae00 to 0x5799e00, (0xf000 bytes) 2024-09-16T20:48:17.207 INFO:teuthology.orchestra.run.smithi139.stdout:446 read 0x1c16400 thru 0x1c1a603 (0x4204 bytes) 2024-09-16T20:48:17.216 INFO:teuthology.orchestra.run.smithi139.stdout:447 trunc from 0x8fcfa00 to 0x1094400 2024-09-16T20:48:17.235 INFO:teuthology.orchestra.run.smithi139.stdout:450 punch from 0x6ee400 to 0x6f5600, (0x7200 bytes) 2024-09-16T20:48:17.237 INFO:teuthology.orchestra.run.smithi139.stdout:451 read 0xe24e00 thru 0xe2d495 (0x8696 bytes) 2024-09-16T20:48:17.238 INFO:teuthology.orchestra.run.smithi139.stdout:452 read 0x8cdc00 thru 0x8cfe03 (0x2204 bytes) 2024-09-16T20:48:17.238 INFO:teuthology.orchestra.run.smithi139.stdout:453 read 0x822000 thru 0x82bc83 (0x9c84 bytes) 2024-09-16T20:48:17.245 INFO:teuthology.orchestra.run.smithi139.stdout:455 trunc from 0x1094400 to 0x6873600 2024-09-16T20:48:17.251 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:17.250+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:17.273 INFO:teuthology.orchestra.run.smithi139.stdout:457 write 0xcf15c00 thru 0xcf1bff9 (0x63fa bytes) 2024-09-16T20:48:17.274 INFO:teuthology.orchestra.run.smithi139.stdout:458 read 0x47f1200 thru 0x47f71f0 (0x5ff1 bytes) 2024-09-16T20:48:17.275 INFO:teuthology.orchestra.run.smithi139.stdout:459 write 0x3424800 thru 0x343329b (0xea9c bytes) 2024-09-16T20:48:17.276 INFO:teuthology.orchestra.run.smithi139.stdout:460 read 0x8eb6a00 thru 0x8eb9f48 (0x3549 bytes) 2024-09-16T20:48:17.276 INFO:teuthology.orchestra.run.smithi139.stdout:465 read 0x2b59a00 thru 0x2b695cd (0xfbce bytes) 2024-09-16T20:48:17.277 INFO:teuthology.orchestra.run.smithi139.stdout:466 write 0x71d1200 thru 0x71d8ca8 (0x7aa9 bytes) 2024-09-16T20:48:17.283 INFO:teuthology.orchestra.run.smithi139.stdout:468 trunc from 0xcf1c000 to 0x90cf400 2024-09-16T20:48:17.303 INFO:teuthology.orchestra.run.smithi139.stdout:470 read 0x4834400 thru 0x483dd27 (0x9928 bytes) 2024-09-16T20:48:17.309 INFO:teuthology.orchestra.run.smithi139.stdout:473 punch from 0x1303000 to 0x1307a00, (0x4a00 bytes) 2024-09-16T20:48:17.309 INFO:teuthology.orchestra.run.smithi139.stdout:474 read 0x32dcc00 thru 0x32e6131 (0x9532 bytes) 2024-09-16T20:48:17.309 INFO:teuthology.orchestra.run.smithi139.stdout:476 punch from 0x18eae00 to 0x18f6c00, (0xbe00 bytes) 2024-09-16T20:48:17.309 INFO:teuthology.orchestra.run.smithi139.stdout:477 read 0x6503a00 thru 0x6511fde (0xe5df bytes) 2024-09-16T20:48:17.310 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:17.310 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:17.319 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:48:17.322 INFO:teuthology.orchestra.run.smithi139.stdout:478 write 0xbbf1c00 thru 0xbbfac0a (0x900b bytes) 2024-09-16T20:48:17.323 INFO:teuthology.orchestra.run.smithi139.stdout:481 write 0x7fa0000 thru 0x7fa7192 (0x7193 bytes) 2024-09-16T20:48:17.325 INFO:teuthology.orchestra.run.smithi139.stdout:485 punch from 0xb783000 to 0xb791400, (0xe400 bytes) 2024-09-16T20:48:17.325 INFO:teuthology.orchestra.run.smithi139.stdout:486 read 0x1e6a00 thru 0x1e861c (0x1c1d bytes) 2024-09-16T20:48:17.326 INFO:teuthology.orchestra.run.smithi139.stdout:491 read 0xabe3000 thru 0xabef408 (0xc409 bytes) 2024-09-16T20:48:17.327 INFO:teuthology.orchestra.run.smithi139.stdout:495 punch from 0x39d3600 to 0x39d6200, (0x2c00 bytes) 2024-09-16T20:48:17.338 INFO:teuthology.orchestra.run.smithi139.stdout:500 write 0xc749000 thru 0xc74a1df (0x11e0 bytes) 2024-09-16T20:48:17.339 INFO:teuthology.orchestra.run.smithi139.stdout:501 write 0x2a5b000 thru 0x2a61200 (0x6201 bytes) 2024-09-16T20:48:17.340 INFO:teuthology.orchestra.run.smithi139.stdout:504 write 0xc494000 thru 0xc499963 (0x5964 bytes) 2024-09-16T20:48:17.340 INFO:teuthology.orchestra.run.smithi139.stdout:505 read 0x9090800 thru 0x9098296 (0x7a97 bytes) 2024-09-16T20:48:17.340 INFO:teuthology.orchestra.run.smithi139.stdout:506 write 0xc2ae200 thru 0xc2b73bb (0x91bc bytes) 2024-09-16T20:48:17.341 INFO:teuthology.orchestra.run.smithi139.stdout:508 punch from 0x2351a00 to 0x2353200, (0x1800 bytes) 2024-09-16T20:48:17.341 INFO:teuthology.orchestra.run.smithi139.stdout:510 read 0x8c22600 thru 0x8c2dc5b (0xb65c bytes) 2024-09-16T20:48:17.352 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:17.351+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:17.353 INFO:teuthology.orchestra.run.smithi139.stdout:511 write 0xe852c00 thru 0xe858bed (0x5fee bytes) 2024-09-16T20:48:17.354 INFO:teuthology.orchestra.run.smithi139.stdout:516 write 0x8bfcc00 thru 0x8bfed89 (0x218a bytes) 2024-09-16T20:48:17.355 INFO:teuthology.orchestra.run.smithi139.stdout:517 write 0xa6c8e00 thru 0xa6d6583 (0xd784 bytes) 2024-09-16T20:48:17.355 INFO:teuthology.orchestra.run.smithi139.stdout:519 write 0xd562200 thru 0xd566a2f (0x4830 bytes) 2024-09-16T20:48:17.356 INFO:teuthology.orchestra.run.smithi139.stdout:524 read 0x978ce00 thru 0x9796d6e (0x9f6f bytes) 2024-09-16T20:48:17.358 INFO:teuthology.orchestra.run.smithi139.stdout:525 write 0x9411a00 thru 0x941219a (0x79b bytes) 2024-09-16T20:48:17.369 INFO:teuthology.orchestra.run.smithi139.stdout:526 trunc from 0xe858c00 to 0x4213a00 2024-09-16T20:48:17.395 INFO:teuthology.orchestra.run.smithi139.stdout:527 punch from 0x22bb200 to 0x22c5e00, (0xac00 bytes) 2024-09-16T20:48:17.415 INFO:teuthology.orchestra.run.smithi139.stdout:528 write 0xb119c00 thru 0xb120e1b (0x721c bytes) 2024-09-16T20:48:17.415 INFO:teuthology.orchestra.run.smithi139.stdout:532 read 0x3728400 thru 0x3731a1c (0x961d bytes) 2024-09-16T20:48:17.417 INFO:teuthology.orchestra.run.smithi139.stdout:535 punch from 0x7151200 to 0x7154000, (0x2e00 bytes) 2024-09-16T20:48:17.417 INFO:teuthology.orchestra.run.smithi139.stdout:536 read 0x23c3000 thru 0x23ce0f6 (0xb0f7 bytes) 2024-09-16T20:48:17.417 INFO:teuthology.orchestra.run.smithi139.stdout:538 write 0xa901c00 thru 0xa90e73b (0xcb3c bytes) 2024-09-16T20:48:17.418 INFO:teuthology.orchestra.run.smithi139.stdout:543 write 0x7346a00 thru 0x73564b9 (0xfaba bytes) 2024-09-16T20:48:17.419 INFO:teuthology.orchestra.run.smithi139.stdout:544 punch from 0x40db400 to 0x40e2600, (0x7200 bytes) 2024-09-16T20:48:17.419 INFO:teuthology.orchestra.run.smithi139.stdout:545 read 0x41de200 thru 0x41e3bd7 (0x59d8 bytes) 2024-09-16T20:48:17.421 INFO:teuthology.orchestra.run.smithi139.stdout:547 punch from 0x3e49e00 to 0x3e55400, (0xb600 bytes) 2024-09-16T20:48:17.430 INFO:teuthology.orchestra.run.smithi139.stdout:549 write 0xbf2f400 thru 0xbf3d7ae (0xe3af bytes) 2024-09-16T20:48:17.441 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:17.441 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:17.445 INFO:teuthology.orchestra.run.smithi139.stdout:550 write 0xed15400 thru 0xed1d84a (0x844b bytes) 2024-09-16T20:48:17.447 INFO:teuthology.orchestra.run.smithi139.stdout:551 write 0x9510600 thru 0x951c5e8 (0xbfe9 bytes) 2024-09-16T20:48:17.451 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:48:17.452 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:17.451+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:17.464 INFO:teuthology.orchestra.run.smithi139.stdout:553 trunc from 0xed1da00 to 0x986e00 2024-09-16T20:48:17.497 INFO:teuthology.orchestra.run.smithi139.stdout:555 punch from 0x2c0600 to 0x2c4a00, (0x4400 bytes) 2024-09-16T20:48:17.515 INFO:teuthology.orchestra.run.smithi139.stdout:556 write 0x6491a00 thru 0x6498386 (0x6987 bytes) 2024-09-16T20:48:17.516 INFO:teuthology.orchestra.run.smithi139.stdout:558 read 0x1803e00 thru 0x1808e88 (0x5089 bytes) 2024-09-16T20:48:17.517 INFO:teuthology.orchestra.run.smithi139.stdout:559 write 0x23d5c00 thru 0x23d7e43 (0x2244 bytes) 2024-09-16T20:48:17.523 INFO:teuthology.orchestra.run.smithi139.stdout:561 trunc from 0x6498400 to 0xb34d200 2024-09-16T20:48:17.537 INFO:teuthology.orchestra.run.smithi139.stdout:565 trunc from 0xb34d200 to 0x81c4e00 2024-09-16T20:48:17.554 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:17.552+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:17.561 INFO:teuthology.orchestra.run.smithi139.stdout:570 trunc from 0x81c4e00 to 0x23ab400 2024-09-16T20:48:17.569 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:17.569 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:17.579 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:48:17.589 INFO:teuthology.orchestra.run.smithi139.stdout:571 trunc from 0x23ab400 to 0xc2d2a00 2024-09-16T20:48:17.600 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T20:48:17.603 INFO:teuthology.orchestra.run.smithi139.stdout:572 trunc from 0xc2d2a00 to 0x955de00 2024-09-16T20:48:17.615 INFO:teuthology.orchestra.run.smithi139.stdout:skipping zero length punch hole 2024-09-16T20:48:17.615 INFO:teuthology.orchestra.run.smithi139.stdout:575 read 0x402f200 thru 0x40396b3 (0xa4b4 bytes) 2024-09-16T20:48:17.617 INFO:teuthology.orchestra.run.smithi139.stdout:576 punch from 0xc4ec00 to 0xc59400, (0xa800 bytes) 2024-09-16T20:48:17.617 INFO:teuthology.orchestra.run.smithi139.stdout:577 punch from 0x1eb8c00 to 0x1ec2800, (0x9c00 bytes) 2024-09-16T20:48:17.617 INFO:teuthology.orchestra.run.smithi139.stdout:579 punch from 0x8c4b800 to 0x8c56200, (0xaa00 bytes) 2024-09-16T20:48:17.617 INFO:teuthology.orchestra.run.smithi139.stdout:580 write 0x17e9e00 thru 0x17f3a6f (0x9c70 bytes) 2024-09-16T20:48:17.618 INFO:teuthology.orchestra.run.smithi139.stdout:581 read 0x7ebc600 thru 0x7ec92e3 (0xcce4 bytes) 2024-09-16T20:48:17.619 INFO:teuthology.orchestra.run.smithi139.stdout:582 punch from 0x7aa6000 to 0x7aa9400, (0x3400 bytes) 2024-09-16T20:48:17.619 INFO:teuthology.orchestra.run.smithi139.stdout:585 read 0x1776200 thru 0x17832bd (0xd0be bytes) 2024-09-16T20:48:17.621 INFO:teuthology.orchestra.run.smithi139.stdout:586 read 0x3d55600 thru 0x3d5e1d9 (0x8bda bytes) 2024-09-16T20:48:17.621 INFO:teuthology.orchestra.run.smithi139.stdout:587 read 0x76de000 thru 0x76ebe8d (0xde8e bytes) 2024-09-16T20:48:17.621 INFO:teuthology.orchestra.run.smithi139.stdout:588 read 0x6cc1600 thru 0x6cc71e9 (0x5bea bytes) 2024-09-16T20:48:17.622 INFO:teuthology.orchestra.run.smithi139.stdout:590 punch from 0x91a7000 to 0x91b6800, (0xf800 bytes) 2024-09-16T20:48:17.622 INFO:teuthology.orchestra.run.smithi139.stdout:592 read 0x93cae00 thru 0x93cf19a (0x439b bytes) 2024-09-16T20:48:17.637 INFO:teuthology.orchestra.run.smithi139.stdout:594 write 0xd1fe400 thru 0xd1fea79 (0x67a bytes) 2024-09-16T20:48:17.639 INFO:teuthology.orchestra.run.smithi139.stdout:596 trunc from 0xd1fec00 to 0xdf6d600 2024-09-16T20:48:17.654 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:17.653+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:17.655 INFO:teuthology.orchestra.run.smithi139.stdout:600 read 0x933000 thru 0x93494c (0x194d bytes) 2024-09-16T20:48:17.656 INFO:teuthology.orchestra.run.smithi139.stdout:601 write 0x85ea200 thru 0x85f6354 (0xc155 bytes) 2024-09-16T20:48:17.657 INFO:teuthology.orchestra.run.smithi139.stdout:603 read 0x2899200 thru 0x2899c80 (0xa81 bytes) 2024-09-16T20:48:17.657 INFO:teuthology.orchestra.run.smithi139.stdout:605 write 0x2ff0a00 thru 0x2ff5494 (0x4a95 bytes) 2024-09-16T20:48:17.672 INFO:teuthology.orchestra.run.smithi139.stdout:607 write 0xe87e400 thru 0xe885ea9 (0x7aaa bytes) 2024-09-16T20:48:17.673 INFO:teuthology.orchestra.run.smithi139.stdout:608 punch from 0x5a6d400 to 0x5a6f400, (0x2000 bytes) 2024-09-16T20:48:17.674 INFO:teuthology.orchestra.run.smithi139.stdout:609 punch from 0x4fbd200 to 0x4fcba00, (0xe800 bytes) 2024-09-16T20:48:17.674 INFO:teuthology.orchestra.run.smithi139.stdout:610 read 0x276a800 thru 0x2773eda (0x96db bytes) 2024-09-16T20:48:17.674 INFO:teuthology.orchestra.run.smithi139.stdout:611 write 0xd893200 thru 0xd89a384 (0x7185 bytes) 2024-09-16T20:48:17.674 INFO:teuthology.orchestra.run.smithi139.stdout:612 write 0x67cf600 thru 0x67d21cc (0x2bcd bytes) 2024-09-16T20:48:17.675 INFO:teuthology.orchestra.run.smithi139.stdout:614 clone 5 order 18 su 2048 sc 4 2024-09-16T20:48:17.700 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-16T20:48:17.700 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T20:48:17.700 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-16T20:48:17.700 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-16T20:48:17.705 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:17.705 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:17.715 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:48:17.755 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:48:17.756 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:48:17.756 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:17.754+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:17.773 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:48:17.773 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:48:17.790 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:48:17.791 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:48:17.808 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:48:17.808 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:48:17.826 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:48:17.826 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:48:17.843 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:48:17.843 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:48:17.872 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:17.872 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:48:17.872 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:48:17.872 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:48:17.872 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.872 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:48:17.872 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:48:17.872 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 24.612988117, 2024-09-16T20:48:17.872 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67502346999999996, 2024-09-16T20:48:17.872 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:17.873 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:17.873 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:17.873 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:17.873 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:48:17.873 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:48:17.873 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 1 2024-09-16T20:48:17.873 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.873 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:17.873 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.873 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:17.874 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:48:17.874 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:17.874 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.874 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.874 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:17.874 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:48:17.874 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:17.874 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.874 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.874 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:17.874 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254128+0000", 2024-09-16T20:48:17.874 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.463e-06 2024-09-16T20:48:17.874 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.874 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.875 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:17.875 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254135+0000", 2024-09-16T20:48:17.875 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1579999999999999e-06 2024-09-16T20:48:17.875 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.875 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.875 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:17.875 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254146+0000", 2024-09-16T20:48:17.875 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1341e-05 2024-09-16T20:48:17.875 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.875 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.875 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:17.875 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254155+0000", 2024-09-16T20:48:17.875 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.1549999999999996e-06 2024-09-16T20:48:17.876 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.876 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.876 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:17.876 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254221+0000", 2024-09-16T20:48:17.876 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5470999999999997e-05 2024-09-16T20:48:17.876 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.876 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.876 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:48:17.876 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254231+0000", 2024-09-16T20:48:17.876 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0592e-05 2024-09-16T20:48:17.876 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.876 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.877 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:17.877 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926711+0000", 2024-09-16T20:48:17.877 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67247946599999997 2024-09-16T20:48:17.877 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.877 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.877 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:17.877 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926910+0000", 2024-09-16T20:48:17.877 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019934 2024-09-16T20:48:17.877 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.877 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.877 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:48:17.877 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.927159+0000", 2024-09-16T20:48:17.877 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00024876599999999998 2024-09-16T20:48:17.878 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.878 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.878 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:17.878 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.928797+0000", 2024-09-16T20:48:17.878 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001638308 2024-09-16T20:48:17.878 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.878 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.878 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:17.878 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929026+0000", 2024-09-16T20:48:17.878 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00022876099999999999 2024-09-16T20:48:17.878 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.878 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.878 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:17.878 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929102+0000", 2024-09-16T20:48:17.879 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6184000000000001e-05 2024-09-16T20:48:17.879 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.879 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.879 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:17.879 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929150+0000", 2024-09-16T20:48:17.879 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7465e-05 2024-09-16T20:48:17.879 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:17.879 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:17.879 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:17.879 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.879 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.879 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:48:17.879 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:54.058872+0000", 2024-09-16T20:48:17.880 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.808241845000001, 2024-09-16T20:48:17.880 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0060982579999999996, 2024-09-16T20:48:17.880 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:17.880 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:17.880 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:17.880 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:17.880 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:48:17.880 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:48:17.880 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 47 2024-09-16T20:48:17.880 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.880 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:17.880 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.880 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:17.881 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058872+0000", 2024-09-16T20:48:17.881 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:17.881 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.881 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.881 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:17.881 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058872+0000", 2024-09-16T20:48:17.881 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:17.881 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.881 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.881 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:17.881 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058873+0000", 2024-09-16T20:48:17.881 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1e-07 2024-09-16T20:48:17.881 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.881 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.882 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:17.882 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058919+0000", 2024-09-16T20:48:17.882 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6570999999999998e-05 2024-09-16T20:48:17.882 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.882 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.882 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:17.882 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058926+0000", 2024-09-16T20:48:17.882 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5509999999999996e-06 2024-09-16T20:48:17.882 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.882 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.882 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:17.882 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058932+0000", 2024-09-16T20:48:17.882 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8780000000000003e-06 2024-09-16T20:48:17.882 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.883 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.883 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:17.883 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.062820+0000", 2024-09-16T20:48:17.883 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0038885790000000001 2024-09-16T20:48:17.883 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.883 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.883 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:17.883 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.062868+0000", 2024-09-16T20:48:17.883 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7948999999999998e-05 2024-09-16T20:48:17.883 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.883 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.883 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:48:17.883 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.062945+0000", 2024-09-16T20:48:17.883 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6495999999999994e-05 2024-09-16T20:48:17.884 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.884 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.884 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:17.884 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.064336+0000", 2024-09-16T20:48:17.884 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001391446 2024-09-16T20:48:17.884 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.884 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.884 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:17.884 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.064915+0000", 2024-09-16T20:48:17.884 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00057899000000000004 2024-09-16T20:48:17.884 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.884 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.884 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:17.884 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.064947+0000", 2024-09-16T20:48:17.885 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2132999999999999e-05 2024-09-16T20:48:17.885 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.885 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.885 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:17.885 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.064971+0000", 2024-09-16T20:48:17.885 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3354999999999999e-05 2024-09-16T20:48:17.885 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:17.885 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:17.885 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:17.885 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.885 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.885 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:48:17.885 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:54.058944+0000", 2024-09-16T20:48:17.885 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.808170136000001, 2024-09-16T20:48:17.885 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0061376369999999996, 2024-09-16T20:48:17.886 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:17.886 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:17.886 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:17.886 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:17.886 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:48:17.886 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:48:17.886 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 48 2024-09-16T20:48:17.886 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.886 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:17.886 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.886 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:17.886 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058944+0000", 2024-09-16T20:48:17.886 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:17.887 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.887 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.887 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:17.887 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058944+0000", 2024-09-16T20:48:17.887 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:17.887 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.887 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.887 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:17.887 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058945+0000", 2024-09-16T20:48:17.887 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9200000000000002e-07 2024-09-16T20:48:17.887 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.887 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.887 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:17.887 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058948+0000", 2024-09-16T20:48:17.887 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8099999999999999e-06 2024-09-16T20:48:17.888 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.888 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.888 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:17.888 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058963+0000", 2024-09-16T20:48:17.888 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5112e-05 2024-09-16T20:48:17.888 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.888 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.888 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:17.888 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058975+0000", 2024-09-16T20:48:17.888 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1732e-05 2024-09-16T20:48:17.888 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.888 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.888 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:17.889 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.063363+0000", 2024-09-16T20:48:17.889 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0043879330000000001 2024-09-16T20:48:17.889 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.889 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.889 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:17.889 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.063420+0000", 2024-09-16T20:48:17.889 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6745e-05 2024-09-16T20:48:17.889 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.889 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.889 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:48:17.889 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.063496+0000", 2024-09-16T20:48:17.889 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6297999999999994e-05 2024-09-16T20:48:17.889 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.889 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.890 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:17.890 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.064981+0000", 2024-09-16T20:48:17.890 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0014852559999999999 2024-09-16T20:48:17.890 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.890 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.890 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:17.890 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.065028+0000", 2024-09-16T20:48:17.890 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7052000000000003e-05 2024-09-16T20:48:17.890 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.890 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.890 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:17.890 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.065060+0000", 2024-09-16T20:48:17.890 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1152000000000003e-05 2024-09-16T20:48:17.890 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.891 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.891 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:17.891 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.065082+0000", 2024-09-16T20:48:17.891 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2155e-05 2024-09-16T20:48:17.891 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:17.891 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:17.891 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:17.891 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.891 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.891 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:48:17.891 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:54.058988+0000", 2024-09-16T20:48:17.891 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.808126498, 2024-09-16T20:48:17.891 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0069229319999999997, 2024-09-16T20:48:17.891 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:17.892 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:17.892 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:17.892 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:17.892 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:48:17.892 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:48:17.892 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 49 2024-09-16T20:48:17.892 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.892 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:17.892 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.892 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:17.892 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058988+0000", 2024-09-16T20:48:17.892 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:17.892 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.892 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.893 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:17.893 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058988+0000", 2024-09-16T20:48:17.893 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:17.893 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.893 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.893 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:17.893 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058988+0000", 2024-09-16T20:48:17.893 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8299999999999998e-07 2024-09-16T20:48:17.893 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.893 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.893 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:17.893 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058992+0000", 2024-09-16T20:48:17.893 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8569999999999997e-06 2024-09-16T20:48:17.893 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.893 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.894 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:17.894 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.058998+0000", 2024-09-16T20:48:17.894 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9020000000000004e-06 2024-09-16T20:48:17.894 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.894 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.894 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:17.894 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.059001+0000", 2024-09-16T20:48:17.894 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.269e-06 2024-09-16T20:48:17.894 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.894 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.894 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:17.894 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.063972+0000", 2024-09-16T20:48:17.894 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0049705690000000002 2024-09-16T20:48:17.894 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.895 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.895 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:17.895 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.064032+0000", 2024-09-16T20:48:17.895 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9834999999999999e-05 2024-09-16T20:48:17.895 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.895 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.895 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:48:17.895 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.064128+0000", 2024-09-16T20:48:17.895 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.6113999999999995e-05 2024-09-16T20:48:17.895 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.895 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.895 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:17.895 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.065476+0000", 2024-09-16T20:48:17.895 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00134826 2024-09-16T20:48:17.896 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.896 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.896 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:17.896 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.065855+0000", 2024-09-16T20:48:17.896 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00037943500000000002 2024-09-16T20:48:17.896 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.896 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.896 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:17.896 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.065886+0000", 2024-09-16T20:48:17.896 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0787999999999997e-05 2024-09-16T20:48:17.896 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.896 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.896 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:17.896 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.065911+0000", 2024-09-16T20:48:17.897 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.4519999999999999e-05 2024-09-16T20:48:17.897 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:17.897 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:17.897 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:17.897 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.897 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.897 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:48:17.897 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:54.059010+0000", 2024-09-16T20:48:17.897 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.808104334999999, 2024-09-16T20:48:17.897 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0069966389999999998, 2024-09-16T20:48:17.897 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:17.897 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:17.897 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:17.897 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:17.897 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:48:17.898 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:48:17.898 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 50 2024-09-16T20:48:17.898 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.898 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:17.898 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.898 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:17.898 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.059010+0000", 2024-09-16T20:48:17.898 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:17.898 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.898 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.898 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:17.898 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.059010+0000", 2024-09-16T20:48:17.898 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:17.898 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.899 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.899 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:17.899 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.059010+0000", 2024-09-16T20:48:17.899 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.72e-07 2024-09-16T20:48:17.899 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.899 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.899 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:17.899 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.059067+0000", 2024-09-16T20:48:17.899 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6756999999999999e-05 2024-09-16T20:48:17.899 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.899 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.899 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:17.899 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.059075+0000", 2024-09-16T20:48:17.900 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8560000000000007e-06 2024-09-16T20:48:17.900 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.900 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.900 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:17.900 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.059078+0000", 2024-09-16T20:48:17.900 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6720000000000002e-06 2024-09-16T20:48:17.900 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.900 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.900 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:17.900 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.064369+0000", 2024-09-16T20:48:17.900 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.005290431 2024-09-16T20:48:17.900 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.900 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.900 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:17.901 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.064416+0000", 2024-09-16T20:48:17.901 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6961e-05 2024-09-16T20:48:17.901 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.901 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.901 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:48:17.901 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.064491+0000", 2024-09-16T20:48:17.901 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5511000000000003e-05 2024-09-16T20:48:17.901 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.901 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.901 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:17.901 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.065922+0000", 2024-09-16T20:48:17.901 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001430728 2024-09-16T20:48:17.901 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.902 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.902 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:17.902 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.065955+0000", 2024-09-16T20:48:17.902 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2741999999999999e-05 2024-09-16T20:48:17.902 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.902 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.902 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:17.902 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.065983+0000", 2024-09-16T20:48:17.902 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8415000000000002e-05 2024-09-16T20:48:17.902 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.902 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.902 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:17.902 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.066007+0000", 2024-09-16T20:48:17.902 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3294000000000001e-05 2024-09-16T20:48:17.903 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:17.903 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:17.903 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:17.903 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.903 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.903 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:48:17.903 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:54.059085+0000", 2024-09-16T20:48:17.903 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.808029237, 2024-09-16T20:48:17.903 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0070804489999999999, 2024-09-16T20:48:17.903 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:17.903 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:17.903 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:17.903 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:17.903 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:48:17.904 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:48:17.904 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 51 2024-09-16T20:48:17.904 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.904 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:17.904 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.904 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:17.904 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.059085+0000", 2024-09-16T20:48:17.904 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:17.904 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.904 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.904 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:17.904 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.059085+0000", 2024-09-16T20:48:17.904 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:17.905 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.905 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.905 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:17.905 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.059085+0000", 2024-09-16T20:48:17.905 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3300000000000003e-07 2024-09-16T20:48:17.905 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.905 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.905 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:17.905 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.059088+0000", 2024-09-16T20:48:17.905 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7700000000000002e-06 2024-09-16T20:48:17.905 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.905 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.905 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:17.905 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.059092+0000", 2024-09-16T20:48:17.906 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6040000000000001e-06 2024-09-16T20:48:17.906 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.906 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.906 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:17.906 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.059100+0000", 2024-09-16T20:48:17.906 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1089999999999998e-06 2024-09-16T20:48:17.906 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.906 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.906 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:17.906 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.065129+0000", 2024-09-16T20:48:17.906 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.006028594 2024-09-16T20:48:17.906 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.906 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.906 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:17.907 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.065175+0000", 2024-09-16T20:48:17.907 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6842999999999999e-05 2024-09-16T20:48:17.907 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.907 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.907 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:48:17.907 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.065251+0000", 2024-09-16T20:48:17.907 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5278000000000007e-05 2024-09-16T20:48:17.907 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.907 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.907 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:17.907 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.066022+0000", 2024-09-16T20:48:17.907 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00077180800000000002 2024-09-16T20:48:17.907 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.908 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.908 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:17.908 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.066118+0000", 2024-09-16T20:48:17.908 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.5594999999999995e-05 2024-09-16T20:48:17.908 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.908 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.908 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:17.908 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.066145+0000", 2024-09-16T20:48:17.908 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7089000000000001e-05 2024-09-16T20:48:17.908 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.908 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.908 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:17.908 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.066165+0000", 2024-09-16T20:48:17.909 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0326000000000002e-05 2024-09-16T20:48:17.909 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:17.909 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:17.909 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:17.909 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.909 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.909 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:48:17.909 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:54.059107+0000", 2024-09-16T20:48:17.909 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.808007185000001, 2024-09-16T20:48:17.909 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0073197189999999997, 2024-09-16T20:48:17.909 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:17.909 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:17.909 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:17.910 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:17.910 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:48:17.910 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:48:17.910 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 52 2024-09-16T20:48:17.910 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.910 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:17.910 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.910 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:17.910 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.059107+0000", 2024-09-16T20:48:17.910 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:17.910 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.910 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.910 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:17.911 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.059107+0000", 2024-09-16T20:48:17.911 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:17.911 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.911 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.911 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:17.911 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.059107+0000", 2024-09-16T20:48:17.911 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4700000000000002e-07 2024-09-16T20:48:17.911 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.911 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.911 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:17.911 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.059110+0000", 2024-09-16T20:48:17.911 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8499999999999998e-06 2024-09-16T20:48:17.911 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.912 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.912 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:17.912 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.059116+0000", 2024-09-16T20:48:17.912 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0519999999999998e-06 2024-09-16T20:48:17.912 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.912 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.912 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:17.912 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.059119+0000", 2024-09-16T20:48:17.912 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0290000000000001e-06 2024-09-16T20:48:17.912 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.912 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.912 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:17.912 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.065513+0000", 2024-09-16T20:48:17.912 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0063932260000000001 2024-09-16T20:48:17.913 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.913 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.913 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:17.913 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.065571+0000", 2024-09-16T20:48:17.913 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8758000000000002e-05 2024-09-16T20:48:17.913 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.913 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.913 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:48:17.913 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.065648+0000", 2024-09-16T20:48:17.913 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7039999999999994e-05 2024-09-16T20:48:17.913 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.913 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.913 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:17.914 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.066180+0000", 2024-09-16T20:48:17.914 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00053161099999999998 2024-09-16T20:48:17.914 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.914 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.914 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:17.914 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.066381+0000", 2024-09-16T20:48:17.914 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000200564 2024-09-16T20:48:17.914 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.914 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.914 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:17.914 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.066408+0000", 2024-09-16T20:48:17.914 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.775e-05 2024-09-16T20:48:17.914 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.915 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.915 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:17.915 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.066427+0000", 2024-09-16T20:48:17.915 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8491999999999998e-05 2024-09-16T20:48:17.915 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:17.915 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:17.915 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:17.915 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.915 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.915 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:48:17.915 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:54.240178+0000", 2024-09-16T20:48:17.915 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.626936099000002, 2024-09-16T20:48:17.915 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0059449899999999998, 2024-09-16T20:48:17.915 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:17.916 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:17.916 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:17.916 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:17.916 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:48:17.916 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:48:17.916 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 72 2024-09-16T20:48:17.916 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.916 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:17.916 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.916 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:17.916 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240178+0000", 2024-09-16T20:48:17.916 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:17.916 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.917 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.917 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:17.917 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240178+0000", 2024-09-16T20:48:17.917 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:17.917 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.917 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.917 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:17.917 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240179+0000", 2024-09-16T20:48:17.917 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.53e-07 2024-09-16T20:48:17.917 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.917 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.917 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:17.917 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240252+0000", 2024-09-16T20:48:17.918 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3279000000000001e-05 2024-09-16T20:48:17.918 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.918 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.918 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:17.918 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240259+0000", 2024-09-16T20:48:17.918 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4270000000000002e-06 2024-09-16T20:48:17.918 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.918 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.918 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:17.918 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240263+0000", 2024-09-16T20:48:17.918 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7050000000000001e-06 2024-09-16T20:48:17.918 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.918 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.918 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:17.919 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.243686+0000", 2024-09-16T20:48:17.919 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0034228330000000001 2024-09-16T20:48:17.919 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.919 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.919 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:17.919 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.243731+0000", 2024-09-16T20:48:17.919 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4917999999999999e-05 2024-09-16T20:48:17.919 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.919 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.919 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:48:17.919 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.243801+0000", 2024-09-16T20:48:17.919 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0320000000000004e-05 2024-09-16T20:48:17.920 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.920 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.920 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:17.920 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.245441+0000", 2024-09-16T20:48:17.920 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001640138 2024-09-16T20:48:17.920 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.920 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.920 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:17.920 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.245996+0000", 2024-09-16T20:48:17.920 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000555128 2024-09-16T20:48:17.920 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.920 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.920 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:17.920 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.246057+0000", 2024-09-16T20:48:17.920 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0458999999999998e-05 2024-09-16T20:48:17.921 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.921 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.921 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:17.921 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.246123+0000", 2024-09-16T20:48:17.921 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6429999999999999e-05 2024-09-16T20:48:17.921 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:17.921 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:17.921 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:17.921 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.921 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.921 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:48:17.921 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:54.240274+0000", 2024-09-16T20:48:17.921 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.626840692000002, 2024-09-16T20:48:17.922 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0063725079999999998, 2024-09-16T20:48:17.922 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:17.922 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:17.922 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:17.922 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:17.922 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:48:17.922 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:48:17.922 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 73 2024-09-16T20:48:17.922 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.922 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:17.922 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.922 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:17.922 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240274+0000", 2024-09-16T20:48:17.922 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:17.923 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.923 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.923 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:17.923 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240274+0000", 2024-09-16T20:48:17.923 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:17.923 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.923 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.923 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:17.923 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240274+0000", 2024-09-16T20:48:17.923 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6100000000000001e-07 2024-09-16T20:48:17.923 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.923 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.923 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:17.923 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240280+0000", 2024-09-16T20:48:17.924 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6960000000000002e-06 2024-09-16T20:48:17.924 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.924 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.924 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:17.924 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240300+0000", 2024-09-16T20:48:17.924 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0151000000000001e-05 2024-09-16T20:48:17.924 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.924 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.924 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:17.924 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240304+0000", 2024-09-16T20:48:17.924 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8330000000000004e-06 2024-09-16T20:48:17.924 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.924 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.924 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:17.925 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.244483+0000", 2024-09-16T20:48:17.925 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0041787769999999998 2024-09-16T20:48:17.925 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.925 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.925 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:17.925 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.244534+0000", 2024-09-16T20:48:17.925 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1158e-05 2024-09-16T20:48:17.925 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.925 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.925 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:48:17.925 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.244611+0000", 2024-09-16T20:48:17.925 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6888e-05 2024-09-16T20:48:17.925 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.925 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.926 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:17.926 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.246133+0000", 2024-09-16T20:48:17.926 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00152224 2024-09-16T20:48:17.926 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.926 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.926 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:17.926 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.246555+0000", 2024-09-16T20:48:17.926 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00042180899999999999 2024-09-16T20:48:17.926 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.926 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.926 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:17.926 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.246610+0000", 2024-09-16T20:48:17.926 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5470999999999998e-05 2024-09-16T20:48:17.926 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.927 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.927 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:17.927 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.246646+0000", 2024-09-16T20:48:17.927 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5923999999999997e-05 2024-09-16T20:48:17.927 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:17.927 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:17.927 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:17.927 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.927 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.927 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:48:17.927 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:54.240313+0000", 2024-09-16T20:48:17.927 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.626800806999999, 2024-09-16T20:48:17.927 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0064355100000000002, 2024-09-16T20:48:17.927 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:17.928 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:17.928 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:17.928 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:17.928 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:48:17.928 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:48:17.928 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 74 2024-09-16T20:48:17.928 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.928 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:17.928 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.928 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:17.928 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240313+0000", 2024-09-16T20:48:17.928 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:17.928 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.928 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.929 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:17.929 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240313+0000", 2024-09-16T20:48:17.929 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:17.929 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.929 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.929 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:17.929 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240314+0000", 2024-09-16T20:48:17.929 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0500000000000003e-07 2024-09-16T20:48:17.929 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.929 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.929 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:17.929 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240324+0000", 2024-09-16T20:48:17.929 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8309999999999996e-06 2024-09-16T20:48:17.930 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.930 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.930 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:17.930 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240329+0000", 2024-09-16T20:48:17.930 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7870000000000001e-06 2024-09-16T20:48:17.930 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.930 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.930 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:17.930 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240332+0000", 2024-09-16T20:48:17.930 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0429999999999999e-06 2024-09-16T20:48:17.930 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.930 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.930 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:17.930 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.244880+0000", 2024-09-16T20:48:17.931 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0045483770000000001 2024-09-16T20:48:17.931 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.931 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.931 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:17.931 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.244925+0000", 2024-09-16T20:48:17.931 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4626000000000003e-05 2024-09-16T20:48:17.931 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.931 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.931 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:48:17.931 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.244994+0000", 2024-09-16T20:48:17.931 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9443999999999994e-05 2024-09-16T20:48:17.931 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.931 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.931 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:17.931 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.246659+0000", 2024-09-16T20:48:17.932 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001665267 2024-09-16T20:48:17.932 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.932 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.932 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:17.932 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.246694+0000", 2024-09-16T20:48:17.932 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4879999999999998e-05 2024-09-16T20:48:17.932 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.932 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.932 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:17.932 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.246724+0000", 2024-09-16T20:48:17.932 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0168999999999999e-05 2024-09-16T20:48:17.932 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.932 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.932 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:17.933 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.246749+0000", 2024-09-16T20:48:17.933 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.4481000000000002e-05 2024-09-16T20:48:17.933 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:17.933 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:17.933 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:17.933 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.933 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.933 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:48:17.933 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:54.240337+0000", 2024-09-16T20:48:17.933 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.626777212, 2024-09-16T20:48:17.933 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0064925900000000003, 2024-09-16T20:48:17.933 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:17.933 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:17.934 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:17.934 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:17.934 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:48:17.934 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:48:17.934 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 75 2024-09-16T20:48:17.934 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.934 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:17.934 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.934 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:17.934 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240337+0000", 2024-09-16T20:48:17.934 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:17.934 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.934 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.934 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:17.935 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240337+0000", 2024-09-16T20:48:17.935 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:17.935 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.935 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.935 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:17.935 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240337+0000", 2024-09-16T20:48:17.935 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9799999999999999e-07 2024-09-16T20:48:17.935 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.935 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.935 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:17.935 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240407+0000", 2024-09-16T20:48:17.935 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9887000000000004e-05 2024-09-16T20:48:17.935 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.935 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.936 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:17.936 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240416+0000", 2024-09-16T20:48:17.936 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.3839999999999997e-06 2024-09-16T20:48:17.936 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.936 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.936 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:17.936 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240426+0000", 2024-09-16T20:48:17.936 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0665999999999999e-05 2024-09-16T20:48:17.936 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.936 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.936 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:17.936 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.245481+0000", 2024-09-16T20:48:17.936 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0050551069999999997 2024-09-16T20:48:17.936 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.937 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.937 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:17.937 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.245552+0000", 2024-09-16T20:48:17.937 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0335999999999999e-05 2024-09-16T20:48:17.937 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.937 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.937 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:48:17.937 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.245625+0000", 2024-09-16T20:48:17.937 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3388999999999998e-05 2024-09-16T20:48:17.937 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.937 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.937 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:17.937 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.246762+0000", 2024-09-16T20:48:17.937 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001137239 2024-09-16T20:48:17.938 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.938 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.938 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:17.938 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.246784+0000", 2024-09-16T20:48:17.938 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1299000000000001e-05 2024-09-16T20:48:17.938 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.938 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.938 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:17.938 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.246808+0000", 2024-09-16T20:48:17.938 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.4244e-05 2024-09-16T20:48:17.938 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.938 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.938 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:17.939 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.246830+0000", 2024-09-16T20:48:17.939 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1741000000000001e-05 2024-09-16T20:48:17.939 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:17.939 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:17.939 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:17.939 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.939 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.939 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:48:17.939 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:54.240434+0000", 2024-09-16T20:48:17.939 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.626680237999999, 2024-09-16T20:48:17.939 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0067000209999999996, 2024-09-16T20:48:17.939 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:17.939 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:17.940 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:17.940 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:17.940 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:48:17.940 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:48:17.940 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 76 2024-09-16T20:48:17.940 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.940 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:17.940 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.940 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:17.940 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240434+0000", 2024-09-16T20:48:17.940 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:17.940 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.940 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.941 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:17.941 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240434+0000", 2024-09-16T20:48:17.941 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:17.941 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.941 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.941 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:17.941 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240434+0000", 2024-09-16T20:48:17.941 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6600000000000002e-07 2024-09-16T20:48:17.941 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.941 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.941 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:17.941 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240440+0000", 2024-09-16T20:48:17.941 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6550000000000001e-06 2024-09-16T20:48:17.942 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.942 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.942 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:17.942 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240445+0000", 2024-09-16T20:48:17.942 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9609999999999996e-06 2024-09-16T20:48:17.942 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.942 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.942 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:17.942 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.240448+0000", 2024-09-16T20:48:17.942 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8880000000000001e-06 2024-09-16T20:48:17.942 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.942 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.942 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:17.942 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.246163+0000", 2024-09-16T20:48:17.943 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0057155269999999998 2024-09-16T20:48:17.943 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.943 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.943 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:17.943 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.246206+0000", 2024-09-16T20:48:17.943 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2379000000000002e-05 2024-09-16T20:48:17.943 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.943 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.943 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:48:17.943 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.246279+0000", 2024-09-16T20:48:17.943 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2889e-05 2024-09-16T20:48:17.943 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.943 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.944 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:17.944 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.246840+0000", 2024-09-16T20:48:17.944 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00056115099999999997 2024-09-16T20:48:17.944 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.944 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.944 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:17.944 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.247077+0000", 2024-09-16T20:48:17.944 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00023753700000000001 2024-09-16T20:48:17.944 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.944 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.944 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:17.944 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.247112+0000", 2024-09-16T20:48:17.944 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4282000000000001e-05 2024-09-16T20:48:17.945 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.945 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.945 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:17.945 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:54.247134+0000", 2024-09-16T20:48:17.945 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2385999999999999e-05 2024-09-16T20:48:17.945 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:17.945 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:17.945 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:17.945 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.945 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.945 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4439.0:325 3.10 3:08e17bbe:::rbd_data.110e1748a8d2.0000000000000000:4 [read 1556480~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:17.945 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:15.244151+0000", 2024-09-16T20:48:17.945 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 2.622962835, 2024-09-16T20:48:17.946 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0060249750000000001, 2024-09-16T20:48:17.946 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:17.946 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:17.946 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:17.946 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:17.946 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4439", 2024-09-16T20:48:17.946 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:17.946 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 325 2024-09-16T20:48:17.946 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.946 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:17.946 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.946 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:17.946 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.244151+0000", 2024-09-16T20:48:17.946 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:17.947 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.947 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.947 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:17.947 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.244151+0000", 2024-09-16T20:48:17.947 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:17.947 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.947 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.947 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:17.947 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.244152+0000", 2024-09-16T20:48:17.947 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.84e-07 2024-09-16T20:48:17.947 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.947 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.947 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:17.947 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.244154+0000", 2024-09-16T20:48:17.948 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9870000000000002e-06 2024-09-16T20:48:17.948 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.948 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.948 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:17.948 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.244160+0000", 2024-09-16T20:48:17.948 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2990000000000002e-06 2024-09-16T20:48:17.948 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.948 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.948 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:17.948 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.244167+0000", 2024-09-16T20:48:17.948 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4840000000000001e-06 2024-09-16T20:48:17.948 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.948 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.949 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:17.949 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.249992+0000", 2024-09-16T20:48:17.949 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0058258520000000003 2024-09-16T20:48:17.949 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.949 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.949 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:17.949 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.250071+0000", 2024-09-16T20:48:17.949 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9028000000000003e-05 2024-09-16T20:48:17.949 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.949 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.949 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:17.949 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.250176+0000", 2024-09-16T20:48:17.949 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000104941 2024-09-16T20:48:17.950 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:17.950 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:17.950 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:17.950 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.950 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.950 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4439.0:491 3.10 3:08e17bbe:::rbd_data.110e1748a8d2.0000000000000000:4 [read 49152~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:17.950 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:15.244517+0000", 2024-09-16T20:48:17.950 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 2.6225971129999999, 2024-09-16T20:48:17.950 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0059186830000000001, 2024-09-16T20:48:17.950 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:17.950 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:17.950 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:17.950 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:17.951 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4439", 2024-09-16T20:48:17.951 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:17.951 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 491 2024-09-16T20:48:17.951 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.951 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:17.951 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.951 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:17.951 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.244517+0000", 2024-09-16T20:48:17.951 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:17.951 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.951 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.951 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:17.951 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.244517+0000", 2024-09-16T20:48:17.951 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:17.952 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.952 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.952 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:17.952 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.244518+0000", 2024-09-16T20:48:17.952 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9400000000000001e-07 2024-09-16T20:48:17.952 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.952 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.952 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:17.952 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.244519+0000", 2024-09-16T20:48:17.952 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9230000000000001e-06 2024-09-16T20:48:17.952 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.952 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.952 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:17.952 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.244526+0000", 2024-09-16T20:48:17.953 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8449999999999997e-06 2024-09-16T20:48:17.953 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.953 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.953 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:17.953 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.244532+0000", 2024-09-16T20:48:17.953 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2730000000000002e-06 2024-09-16T20:48:17.953 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.953 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.953 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:17.953 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.250217+0000", 2024-09-16T20:48:17.953 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0056855650000000001 2024-09-16T20:48:17.953 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.953 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.953 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:17.954 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.250305+0000", 2024-09-16T20:48:17.954 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8201000000000005e-05 2024-09-16T20:48:17.954 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.954 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.954 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:17.954 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.250436+0000", 2024-09-16T20:48:17.954 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013048199999999999 2024-09-16T20:48:17.954 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:17.954 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:17.954 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:17.954 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.954 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.954 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4439.0:492 3.10 3:08e17bbe:::rbd_data.110e1748a8d2.0000000000000000:4 [read 163840~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:17.954 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:15.244539+0000", 2024-09-16T20:48:17.955 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 2.6225757079999998, 2024-09-16T20:48:17.955 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0063661830000000001, 2024-09-16T20:48:17.955 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:17.955 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:17.955 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:17.955 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:17.955 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4439", 2024-09-16T20:48:17.955 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:17.955 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 492 2024-09-16T20:48:17.955 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.955 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:17.955 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.955 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:17.955 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.244539+0000", 2024-09-16T20:48:17.956 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:17.956 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.956 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.956 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:17.956 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.244539+0000", 2024-09-16T20:48:17.956 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:17.956 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.956 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.956 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:17.956 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.244539+0000", 2024-09-16T20:48:17.956 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9900000000000001e-07 2024-09-16T20:48:17.956 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.956 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.957 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:17.957 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.244541+0000", 2024-09-16T20:48:17.957 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9e-06 2024-09-16T20:48:17.957 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.957 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.957 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:17.957 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.244547+0000", 2024-09-16T20:48:17.957 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.105e-06 2024-09-16T20:48:17.957 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.957 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.957 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:17.957 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.244552+0000", 2024-09-16T20:48:17.957 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2310000000000004e-06 2024-09-16T20:48:17.957 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.958 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.958 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:17.958 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.250715+0000", 2024-09-16T20:48:17.958 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0061623210000000001 2024-09-16T20:48:17.958 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.958 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.958 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:17.958 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.250795+0000", 2024-09-16T20:48:17.958 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0098000000000005e-05 2024-09-16T20:48:17.958 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.958 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.958 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:17.958 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.250905+0000", 2024-09-16T20:48:17.958 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000110029 2024-09-16T20:48:17.959 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:17.959 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:17.959 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:17.959 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.959 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.959 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4439.0:493 3.10 3:08e17bbe:::rbd_data.110e1748a8d2.0000000000000000:4 [read 262144~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:17.959 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:15.244559+0000", 2024-09-16T20:48:17.959 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 2.6225550699999998, 2024-09-16T20:48:17.959 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0065768110000000001, 2024-09-16T20:48:17.959 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:17.959 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:17.959 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:17.959 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:17.959 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4439", 2024-09-16T20:48:17.960 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:17.960 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 493 2024-09-16T20:48:17.960 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.960 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:17.960 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.960 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:17.960 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.244559+0000", 2024-09-16T20:48:17.960 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:17.960 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.960 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.960 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:17.960 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.244559+0000", 2024-09-16T20:48:17.960 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:17.960 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.961 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.961 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:17.961 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.244560+0000", 2024-09-16T20:48:17.961 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.32e-07 2024-09-16T20:48:17.961 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.961 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.961 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:17.961 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.244562+0000", 2024-09-16T20:48:17.961 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9290000000000001e-06 2024-09-16T20:48:17.961 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.961 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.961 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:17.961 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.244568+0000", 2024-09-16T20:48:17.961 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4790000000000001e-06 2024-09-16T20:48:17.962 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.962 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.962 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:17.962 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.244573+0000", 2024-09-16T20:48:17.962 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.87e-06 2024-09-16T20:48:17.962 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.962 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.962 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:17.962 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.250937+0000", 2024-09-16T20:48:17.962 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0063645669999999998 2024-09-16T20:48:17.962 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.962 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.962 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:17.962 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.251016+0000", 2024-09-16T20:48:17.962 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8524999999999998e-05 2024-09-16T20:48:17.963 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.963 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.963 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:17.963 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.251136+0000", 2024-09-16T20:48:17.963 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000120009 2024-09-16T20:48:17.963 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:17.963 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:17.963 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:17.963 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.963 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.963 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4439.0:494 3.10 3:08e17bbe:::rbd_data.110e1748a8d2.0000000000000000:4 [read 376832~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:17.963 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:15.244580+0000", 2024-09-16T20:48:17.963 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 2.6225346279999999, 2024-09-16T20:48:17.963 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0071144939999999999, 2024-09-16T20:48:17.964 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:17.964 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:17.964 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:17.964 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:17.964 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4439", 2024-09-16T20:48:17.964 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:17.964 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 494 2024-09-16T20:48:17.964 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.964 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:17.964 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.964 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:17.964 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.244580+0000", 2024-09-16T20:48:17.964 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:17.964 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.965 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.965 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:17.965 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.244580+0000", 2024-09-16T20:48:17.965 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:17.965 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.965 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.965 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:17.965 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.244580+0000", 2024-09-16T20:48:17.965 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7700000000000005e-07 2024-09-16T20:48:17.965 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.965 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.965 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:17.965 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.244587+0000", 2024-09-16T20:48:17.965 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2019999999999998e-06 2024-09-16T20:48:17.966 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.966 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.966 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:17.966 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.244594+0000", 2024-09-16T20:48:17.966 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7399999999999998e-06 2024-09-16T20:48:17.966 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.966 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.966 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:17.966 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.244599+0000", 2024-09-16T20:48:17.966 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3020000000000002e-06 2024-09-16T20:48:17.966 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.966 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.966 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:17.966 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.251168+0000", 2024-09-16T20:48:17.967 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0065686690000000001 2024-09-16T20:48:17.967 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.967 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.967 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:17.967 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.251245+0000", 2024-09-16T20:48:17.967 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6792999999999995e-05 2024-09-16T20:48:17.967 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.967 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.967 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:17.967 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.251694+0000", 2024-09-16T20:48:17.967 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00044931099999999998 2024-09-16T20:48:17.967 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:17.967 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:17.967 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:17.967 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.968 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.968 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4439.0:495 3.10 3:08e17bbe:::rbd_data.110e1748a8d2.0000000000000000:4 [read 491520~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:17.968 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:15.244607+0000", 2024-09-16T20:48:17.968 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 2.6225068679999999, 2024-09-16T20:48:17.968 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0073286200000000001, 2024-09-16T20:48:17.968 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:17.968 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:17.968 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:17.968 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:17.968 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4439", 2024-09-16T20:48:17.968 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:17.968 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 495 2024-09-16T20:48:17.968 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.969 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:17.969 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.969 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:17.969 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.244607+0000", 2024-09-16T20:48:17.969 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:17.969 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.969 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.969 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:17.969 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.244607+0000", 2024-09-16T20:48:17.969 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:17.969 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.969 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.969 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:17.969 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.244608+0000", 2024-09-16T20:48:17.970 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2e-07 2024-09-16T20:48:17.970 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.970 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.970 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:17.970 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.244610+0000", 2024-09-16T20:48:17.970 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1600000000000001e-06 2024-09-16T20:48:17.970 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.970 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.970 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:17.970 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.244619+0000", 2024-09-16T20:48:17.970 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9400000000000008e-06 2024-09-16T20:48:17.970 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.970 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.971 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:17.971 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.244625+0000", 2024-09-16T20:48:17.971 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.246e-06 2024-09-16T20:48:17.971 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.971 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.971 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:17.971 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.251740+0000", 2024-09-16T20:48:17.971 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0071144540000000001 2024-09-16T20:48:17.971 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.971 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.971 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:17.971 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.251823+0000", 2024-09-16T20:48:17.971 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2966999999999994e-05 2024-09-16T20:48:17.971 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.972 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.972 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:17.972 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.251936+0000", 2024-09-16T20:48:17.972 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000113433 2024-09-16T20:48:17.972 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:17.972 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:17.972 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:17.972 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.972 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.972 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4439.0:496 3.10 3:08e17bbe:::rbd_data.110e1748a8d2.0000000000000000:4 [read 589824~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:17.972 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:15.244632+0000", 2024-09-16T20:48:17.972 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 2.622481879, 2024-09-16T20:48:17.973 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0075440200000000002, 2024-09-16T20:48:17.973 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:17.973 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:17.973 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:17.973 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:17.973 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4439", 2024-09-16T20:48:17.973 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:17.973 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 496 2024-09-16T20:48:17.973 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.973 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:17.973 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.973 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:17.973 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.244632+0000", 2024-09-16T20:48:17.973 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:17.974 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.974 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.974 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:17.974 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.244632+0000", 2024-09-16T20:48:17.974 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:17.974 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.974 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.974 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:17.974 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.244633+0000", 2024-09-16T20:48:17.974 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2399999999999999e-07 2024-09-16T20:48:17.974 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.974 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.974 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:17.975 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.247109+0000", 2024-09-16T20:48:17.975 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.002475905 2024-09-16T20:48:17.975 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.975 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.975 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:17.975 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.247121+0000", 2024-09-16T20:48:17.975 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2330000000000001e-05 2024-09-16T20:48:17.975 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.975 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.975 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:17.975 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.247129+0000", 2024-09-16T20:48:17.975 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.3210000000000005e-06 2024-09-16T20:48:17.975 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.975 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.976 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:17.976 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.251972+0000", 2024-09-16T20:48:17.976 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0048424130000000003 2024-09-16T20:48:17.976 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.976 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.976 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:17.976 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.252052+0000", 2024-09-16T20:48:17.976 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0482000000000006e-05 2024-09-16T20:48:17.976 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.976 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.976 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:17.976 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.252176+0000", 2024-09-16T20:48:17.976 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00012414500000000001 2024-09-16T20:48:17.977 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:17.977 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:17.977 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:17.977 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.977 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.977 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4439.0:500 3.10 3:08e17bbe:::rbd_data.110e1748a8d2.0000000000000000:4 [read 1032192~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:17.977 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:15.247301+0000", 2024-09-16T20:48:17.977 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 2.6198131020000002, 2024-09-16T20:48:17.977 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0059076409999999999, 2024-09-16T20:48:17.977 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:17.977 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:17.977 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:17.977 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:17.977 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4439", 2024-09-16T20:48:17.978 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:17.978 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 500 2024-09-16T20:48:17.978 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.978 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:17.978 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.978 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:17.978 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.247301+0000", 2024-09-16T20:48:17.978 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:17.978 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.978 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.978 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:17.978 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.247301+0000", 2024-09-16T20:48:17.978 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:17.979 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.979 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.979 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:17.979 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.247302+0000", 2024-09-16T20:48:17.979 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9799999999999999e-07 2024-09-16T20:48:17.979 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.979 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.979 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:17.979 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.250254+0000", 2024-09-16T20:48:17.979 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0029528229999999998 2024-09-16T20:48:17.979 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.979 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.979 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:17.979 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.250267+0000", 2024-09-16T20:48:17.979 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2626000000000001e-05 2024-09-16T20:48:17.980 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.980 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.980 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:17.980 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.250276+0000", 2024-09-16T20:48:17.980 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.1200000000000008e-06 2024-09-16T20:48:17.980 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.980 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.980 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:17.980 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.253023+0000", 2024-09-16T20:48:17.980 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0027471180000000002 2024-09-16T20:48:17.980 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.980 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.981 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:17.981 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.253103+0000", 2024-09-16T20:48:17.981 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9424000000000004e-05 2024-09-16T20:48:17.981 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:17.981 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:17.981 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:17.981 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:15.253209+0000", 2024-09-16T20:48:17.981 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010613200000000001 2024-09-16T20:48:17.981 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:17.981 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:17.981 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:17.981 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:17.981 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:17.981 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:17.982 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:17.867+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:17.983 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:48:18.010 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:48:18.010 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:18.010 INFO:teuthology.orchestra.run.smithi029.stdout: "epoch": 32, 2024-09-16T20:48:18.010 INFO:teuthology.orchestra.run.smithi029.stdout: "fsid": "e28af4a7-5fcd-4de8-8f77-a2b575361bb1", 2024-09-16T20:48:18.010 INFO:teuthology.orchestra.run.smithi029.stdout: "created": "2024-09-16T20:47:39.760609+0000", 2024-09-16T20:48:18.010 INFO:teuthology.orchestra.run.smithi029.stdout: "modified": "2024-09-16T20:48:16.589813+0000", 2024-09-16T20:48:18.010 INFO:teuthology.orchestra.run.smithi029.stdout: "last_up_change": "2024-09-16T20:47:50.238013+0000", 2024-09-16T20:48:18.010 INFO:teuthology.orchestra.run.smithi029.stdout: "last_in_change": "2024-09-16T20:47:41.378753+0000", 2024-09-16T20:48:18.010 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-16T20:48:18.010 INFO:teuthology.orchestra.run.smithi029.stdout: "flags_num": 5799936, 2024-09-16T20:48:18.010 INFO:teuthology.orchestra.run.smithi029.stdout: "flags_set": [ 2024-09-16T20:48:18.010 INFO:teuthology.orchestra.run.smithi029.stdout: "pglog_hardlimit", 2024-09-16T20:48:18.011 INFO:teuthology.orchestra.run.smithi029.stdout: "purged_snapdirs", 2024-09-16T20:48:18.011 INFO:teuthology.orchestra.run.smithi029.stdout: "recovery_deletes", 2024-09-16T20:48:18.011 INFO:teuthology.orchestra.run.smithi029.stdout: "sortbitwise" 2024-09-16T20:48:18.011 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-16T20:48:18.011 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_version": 5, 2024-09-16T20:48:18.011 INFO:teuthology.orchestra.run.smithi029.stdout: "full_ratio": 0.94999998807907104, 2024-09-16T20:48:18.011 INFO:teuthology.orchestra.run.smithi029.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-16T20:48:18.011 INFO:teuthology.orchestra.run.smithi029.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-16T20:48:18.011 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_snapshot": "", 2024-09-16T20:48:18.011 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_max": 3, 2024-09-16T20:48:18.011 INFO:teuthology.orchestra.run.smithi029.stdout: "max_osd": 6, 2024-09-16T20:48:18.011 INFO:teuthology.orchestra.run.smithi029.stdout: "require_min_compat_client": "luminous", 2024-09-16T20:48:18.011 INFO:teuthology.orchestra.run.smithi029.stdout: "min_compat_client": "jewel", 2024-09-16T20:48:18.012 INFO:teuthology.orchestra.run.smithi029.stdout: "require_osd_release": "squid", 2024-09-16T20:48:18.012 INFO:teuthology.orchestra.run.smithi029.stdout: "allow_crimson": false, 2024-09-16T20:48:18.012 INFO:teuthology.orchestra.run.smithi029.stdout: "pools": [ 2024-09-16T20:48:18.012 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.012 INFO:teuthology.orchestra.run.smithi029.stdout: "pool": 1, 2024-09-16T20:48:18.012 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_name": ".mgr", 2024-09-16T20:48:18.012 INFO:teuthology.orchestra.run.smithi029.stdout: "create_time": "2024-09-16T20:47:52.220750+0000", 2024-09-16T20:48:18.012 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-16T20:48:18.012 INFO:teuthology.orchestra.run.smithi029.stdout: "flags_names": "hashpspool", 2024-09-16T20:48:18.012 INFO:teuthology.orchestra.run.smithi029.stdout: "type": 1, 2024-09-16T20:48:18.012 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 2, 2024-09-16T20:48:18.012 INFO:teuthology.orchestra.run.smithi029.stdout: "min_size": 1, 2024-09-16T20:48:18.012 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_rule": 0, 2024-09-16T20:48:18.013 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_count": 0, 2024-09-16T20:48:18.013 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_target": 0, 2024-09-16T20:48:18.013 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T20:48:18.013 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T20:48:18.013 INFO:teuthology.orchestra.run.smithi029.stdout: "is_stretch_pool": false, 2024-09-16T20:48:18.013 INFO:teuthology.orchestra.run.smithi029.stdout: "object_hash": 2, 2024-09-16T20:48:18.013 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_autoscale_mode": "off", 2024-09-16T20:48:18.013 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num": 1, 2024-09-16T20:48:18.013 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num": 1, 2024-09-16T20:48:18.013 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num_target": 1, 2024-09-16T20:48:18.013 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_target": 1, 2024-09-16T20:48:18.013 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_pending": 1, 2024-09-16T20:48:18.013 INFO:teuthology.orchestra.run.smithi029.stdout: "last_pg_merge_meta": { 2024-09-16T20:48:18.014 INFO:teuthology.orchestra.run.smithi029.stdout: "source_pgid": "0.0", 2024-09-16T20:48:18.014 INFO:teuthology.orchestra.run.smithi029.stdout: "ready_epoch": 0, 2024-09-16T20:48:18.014 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_started": 0, 2024-09-16T20:48:18.014 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_clean": 0, 2024-09-16T20:48:18.014 INFO:teuthology.orchestra.run.smithi029.stdout: "source_version": "0'0", 2024-09-16T20:48:18.014 INFO:teuthology.orchestra.run.smithi029.stdout: "target_version": "0'0" 2024-09-16T20:48:18.014 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.014 INFO:teuthology.orchestra.run.smithi029.stdout: "last_change": "17", 2024-09-16T20:48:18.014 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend": "0", 2024-09-16T20:48:18.014 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T20:48:18.014 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T20:48:18.014 INFO:teuthology.orchestra.run.smithi029.stdout: "auid": 0, 2024-09-16T20:48:18.014 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_mode": "selfmanaged", 2024-09-16T20:48:18.015 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_seq": 0, 2024-09-16T20:48:18.015 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_epoch": 0, 2024-09-16T20:48:18.015 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_snaps": [], 2024-09-16T20:48:18.015 INFO:teuthology.orchestra.run.smithi029.stdout: "removed_snaps": "[]", 2024-09-16T20:48:18.015 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_bytes": 0, 2024-09-16T20:48:18.015 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_objects": 0, 2024-09-16T20:48:18.015 INFO:teuthology.orchestra.run.smithi029.stdout: "tiers": [], 2024-09-16T20:48:18.015 INFO:teuthology.orchestra.run.smithi029.stdout: "tier_of": -1, 2024-09-16T20:48:18.015 INFO:teuthology.orchestra.run.smithi029.stdout: "read_tier": -1, 2024-09-16T20:48:18.015 INFO:teuthology.orchestra.run.smithi029.stdout: "write_tier": -1, 2024-09-16T20:48:18.015 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_mode": "none", 2024-09-16T20:48:18.015 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_bytes": 0, 2024-09-16T20:48:18.015 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_objects": 0, 2024-09-16T20:48:18.015 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T20:48:18.016 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T20:48:18.016 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T20:48:18.016 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_flush_age": 0, 2024-09-16T20:48:18.016 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_evict_age": 0, 2024-09-16T20:48:18.016 INFO:teuthology.orchestra.run.smithi029.stdout: "erasure_code_profile": "", 2024-09-16T20:48:18.016 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_params": { 2024-09-16T20:48:18.016 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "none" 2024-09-16T20:48:18.016 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.016 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_period": 0, 2024-09-16T20:48:18.016 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_count": 0, 2024-09-16T20:48:18.016 INFO:teuthology.orchestra.run.smithi029.stdout: "use_gmt_hitset": true, 2024-09-16T20:48:18.016 INFO:teuthology.orchestra.run.smithi029.stdout: "min_read_recency_for_promote": 0, 2024-09-16T20:48:18.016 INFO:teuthology.orchestra.run.smithi029.stdout: "min_write_recency_for_promote": 0, 2024-09-16T20:48:18.017 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T20:48:18.017 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_search_last_n": 0, 2024-09-16T20:48:18.017 INFO:teuthology.orchestra.run.smithi029.stdout: "grade_table": [], 2024-09-16T20:48:18.017 INFO:teuthology.orchestra.run.smithi029.stdout: "stripe_width": 0, 2024-09-16T20:48:18.017 INFO:teuthology.orchestra.run.smithi029.stdout: "expected_num_objects": 0, 2024-09-16T20:48:18.017 INFO:teuthology.orchestra.run.smithi029.stdout: "fast_read": false, 2024-09-16T20:48:18.017 INFO:teuthology.orchestra.run.smithi029.stdout: "options": { 2024-09-16T20:48:18.017 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_max": 32, 2024-09-16T20:48:18.017 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_min": 1 2024-09-16T20:48:18.017 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.017 INFO:teuthology.orchestra.run.smithi029.stdout: "application_metadata": { 2024-09-16T20:48:18.017 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": {} 2024-09-16T20:48:18.017 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.018 INFO:teuthology.orchestra.run.smithi029.stdout: "read_balance": { 2024-09-16T20:48:18.018 INFO:teuthology.orchestra.run.smithi029.stdout: "score_type": "Fair distribution", 2024-09-16T20:48:18.018 INFO:teuthology.orchestra.run.smithi029.stdout: "score_acting": 6.059999942779541, 2024-09-16T20:48:18.018 INFO:teuthology.orchestra.run.smithi029.stdout: "score_stable": 6.059999942779541, 2024-09-16T20:48:18.018 INFO:teuthology.orchestra.run.smithi029.stdout: "optimal_score": 0.33000001311302185, 2024-09-16T20:48:18.018 INFO:teuthology.orchestra.run.smithi029.stdout: "raw_score_acting": 2, 2024-09-16T20:48:18.018 INFO:teuthology.orchestra.run.smithi029.stdout: "raw_score_stable": 2, 2024-09-16T20:48:18.018 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity_weighted": 1, 2024-09-16T20:48:18.018 INFO:teuthology.orchestra.run.smithi029.stdout: "average_primary_affinity": 1, 2024-09-16T20:48:18.018 INFO:teuthology.orchestra.run.smithi029.stdout: "average_primary_affinity_weighted": 1 2024-09-16T20:48:18.018 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.018 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.019 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.019 INFO:teuthology.orchestra.run.smithi029.stdout: "pool": 2, 2024-09-16T20:48:18.019 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_name": "rbd", 2024-09-16T20:48:18.019 INFO:teuthology.orchestra.run.smithi029.stdout: "create_time": "2024-09-16T20:47:56.756666+0000", 2024-09-16T20:48:18.019 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 8193, 2024-09-16T20:48:18.019 INFO:teuthology.orchestra.run.smithi029.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T20:48:18.019 INFO:teuthology.orchestra.run.smithi029.stdout: "type": 1, 2024-09-16T20:48:18.019 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 2, 2024-09-16T20:48:18.019 INFO:teuthology.orchestra.run.smithi029.stdout: "min_size": 1, 2024-09-16T20:48:18.019 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_rule": 0, 2024-09-16T20:48:18.019 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_count": 0, 2024-09-16T20:48:18.019 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_target": 0, 2024-09-16T20:48:18.019 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T20:48:18.019 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T20:48:18.020 INFO:teuthology.orchestra.run.smithi029.stdout: "is_stretch_pool": false, 2024-09-16T20:48:18.020 INFO:teuthology.orchestra.run.smithi029.stdout: "object_hash": 2, 2024-09-16T20:48:18.020 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_autoscale_mode": "off", 2024-09-16T20:48:18.020 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num": 8, 2024-09-16T20:48:18.020 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num": 8, 2024-09-16T20:48:18.020 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num_target": 8, 2024-09-16T20:48:18.020 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_target": 8, 2024-09-16T20:48:18.020 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_pending": 8, 2024-09-16T20:48:18.020 INFO:teuthology.orchestra.run.smithi029.stdout: "last_pg_merge_meta": { 2024-09-16T20:48:18.020 INFO:teuthology.orchestra.run.smithi029.stdout: "source_pgid": "0.0", 2024-09-16T20:48:18.020 INFO:teuthology.orchestra.run.smithi029.stdout: "ready_epoch": 0, 2024-09-16T20:48:18.020 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_started": 0, 2024-09-16T20:48:18.020 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_clean": 0, 2024-09-16T20:48:18.021 INFO:teuthology.orchestra.run.smithi029.stdout: "source_version": "0'0", 2024-09-16T20:48:18.021 INFO:teuthology.orchestra.run.smithi029.stdout: "target_version": "0'0" 2024-09-16T20:48:18.021 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.021 INFO:teuthology.orchestra.run.smithi029.stdout: "last_change": "21", 2024-09-16T20:48:18.021 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend": "0", 2024-09-16T20:48:18.021 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T20:48:18.021 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T20:48:18.021 INFO:teuthology.orchestra.run.smithi029.stdout: "auid": 0, 2024-09-16T20:48:18.021 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_mode": "selfmanaged", 2024-09-16T20:48:18.021 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_seq": 2, 2024-09-16T20:48:18.021 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_epoch": 21, 2024-09-16T20:48:18.021 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_snaps": [], 2024-09-16T20:48:18.021 INFO:teuthology.orchestra.run.smithi029.stdout: "removed_snaps": "[]", 2024-09-16T20:48:18.021 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_bytes": 0, 2024-09-16T20:48:18.022 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_objects": 0, 2024-09-16T20:48:18.022 INFO:teuthology.orchestra.run.smithi029.stdout: "tiers": [], 2024-09-16T20:48:18.022 INFO:teuthology.orchestra.run.smithi029.stdout: "tier_of": -1, 2024-09-16T20:48:18.022 INFO:teuthology.orchestra.run.smithi029.stdout: "read_tier": -1, 2024-09-16T20:48:18.022 INFO:teuthology.orchestra.run.smithi029.stdout: "write_tier": -1, 2024-09-16T20:48:18.022 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_mode": "none", 2024-09-16T20:48:18.022 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_bytes": 0, 2024-09-16T20:48:18.022 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_objects": 0, 2024-09-16T20:48:18.022 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T20:48:18.022 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T20:48:18.022 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T20:48:18.022 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_flush_age": 0, 2024-09-16T20:48:18.022 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_evict_age": 0, 2024-09-16T20:48:18.022 INFO:teuthology.orchestra.run.smithi029.stdout: "erasure_code_profile": "", 2024-09-16T20:48:18.023 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_params": { 2024-09-16T20:48:18.023 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "none" 2024-09-16T20:48:18.023 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.023 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_period": 0, 2024-09-16T20:48:18.023 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_count": 0, 2024-09-16T20:48:18.023 INFO:teuthology.orchestra.run.smithi029.stdout: "use_gmt_hitset": true, 2024-09-16T20:48:18.023 INFO:teuthology.orchestra.run.smithi029.stdout: "min_read_recency_for_promote": 0, 2024-09-16T20:48:18.023 INFO:teuthology.orchestra.run.smithi029.stdout: "min_write_recency_for_promote": 0, 2024-09-16T20:48:18.023 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T20:48:18.023 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_search_last_n": 0, 2024-09-16T20:48:18.023 INFO:teuthology.orchestra.run.smithi029.stdout: "grade_table": [], 2024-09-16T20:48:18.023 INFO:teuthology.orchestra.run.smithi029.stdout: "stripe_width": 0, 2024-09-16T20:48:18.023 INFO:teuthology.orchestra.run.smithi029.stdout: "expected_num_objects": 0, 2024-09-16T20:48:18.023 INFO:teuthology.orchestra.run.smithi029.stdout: "fast_read": false, 2024-09-16T20:48:18.023 INFO:teuthology.orchestra.run.smithi029.stdout: "options": {}, 2024-09-16T20:48:18.024 INFO:teuthology.orchestra.run.smithi029.stdout: "application_metadata": { 2024-09-16T20:48:18.024 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd": {} 2024-09-16T20:48:18.024 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.024 INFO:teuthology.orchestra.run.smithi029.stdout: "read_balance": { 2024-09-16T20:48:18.024 INFO:teuthology.orchestra.run.smithi029.stdout: "score_type": "Fair distribution", 2024-09-16T20:48:18.024 INFO:teuthology.orchestra.run.smithi029.stdout: "score_acting": 2.25, 2024-09-16T20:48:18.024 INFO:teuthology.orchestra.run.smithi029.stdout: "score_stable": 2.25, 2024-09-16T20:48:18.024 INFO:teuthology.orchestra.run.smithi029.stdout: "optimal_score": 1, 2024-09-16T20:48:18.024 INFO:teuthology.orchestra.run.smithi029.stdout: "raw_score_acting": 2.25, 2024-09-16T20:48:18.024 INFO:teuthology.orchestra.run.smithi029.stdout: "raw_score_stable": 2.25, 2024-09-16T20:48:18.024 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity_weighted": 1, 2024-09-16T20:48:18.024 INFO:teuthology.orchestra.run.smithi029.stdout: "average_primary_affinity": 1, 2024-09-16T20:48:18.024 INFO:teuthology.orchestra.run.smithi029.stdout: "average_primary_affinity_weighted": 1 2024-09-16T20:48:18.024 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.025 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.025 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.025 INFO:teuthology.orchestra.run.smithi029.stdout: "pool": 3, 2024-09-16T20:48:18.025 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_name": "pool_client.0", 2024-09-16T20:48:18.025 INFO:teuthology.orchestra.run.smithi029.stdout: "create_time": "2024-09-16T20:48:06.418889+0000", 2024-09-16T20:48:18.025 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 8193, 2024-09-16T20:48:18.025 INFO:teuthology.orchestra.run.smithi029.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T20:48:18.025 INFO:teuthology.orchestra.run.smithi029.stdout: "type": 1, 2024-09-16T20:48:18.025 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 2, 2024-09-16T20:48:18.025 INFO:teuthology.orchestra.run.smithi029.stdout: "min_size": 1, 2024-09-16T20:48:18.025 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_rule": 0, 2024-09-16T20:48:18.025 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_count": 0, 2024-09-16T20:48:18.025 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_target": 0, 2024-09-16T20:48:18.025 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T20:48:18.026 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T20:48:18.026 INFO:teuthology.orchestra.run.smithi029.stdout: "is_stretch_pool": false, 2024-09-16T20:48:18.026 INFO:teuthology.orchestra.run.smithi029.stdout: "object_hash": 2, 2024-09-16T20:48:18.026 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_autoscale_mode": "off", 2024-09-16T20:48:18.026 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num": 32, 2024-09-16T20:48:18.026 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num": 32, 2024-09-16T20:48:18.026 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num_target": 32, 2024-09-16T20:48:18.026 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_target": 32, 2024-09-16T20:48:18.026 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_pending": 32, 2024-09-16T20:48:18.026 INFO:teuthology.orchestra.run.smithi029.stdout: "last_pg_merge_meta": { 2024-09-16T20:48:18.026 INFO:teuthology.orchestra.run.smithi029.stdout: "source_pgid": "0.0", 2024-09-16T20:48:18.026 INFO:teuthology.orchestra.run.smithi029.stdout: "ready_epoch": 0, 2024-09-16T20:48:18.026 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_started": 0, 2024-09-16T20:48:18.026 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_clean": 0, 2024-09-16T20:48:18.026 INFO:teuthology.orchestra.run.smithi029.stdout: "source_version": "0'0", 2024-09-16T20:48:18.027 INFO:teuthology.orchestra.run.smithi029.stdout: "target_version": "0'0" 2024-09-16T20:48:18.027 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.027 INFO:teuthology.orchestra.run.smithi029.stdout: "last_change": "32", 2024-09-16T20:48:18.027 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend": "0", 2024-09-16T20:48:18.027 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T20:48:18.027 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T20:48:18.027 INFO:teuthology.orchestra.run.smithi029.stdout: "auid": 0, 2024-09-16T20:48:18.027 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_mode": "selfmanaged", 2024-09-16T20:48:18.027 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_seq": 6, 2024-09-16T20:48:18.027 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_epoch": 32, 2024-09-16T20:48:18.027 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_snaps": [], 2024-09-16T20:48:18.027 INFO:teuthology.orchestra.run.smithi029.stdout: "removed_snaps": "[]", 2024-09-16T20:48:18.027 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_bytes": 0, 2024-09-16T20:48:18.028 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_objects": 0, 2024-09-16T20:48:18.028 INFO:teuthology.orchestra.run.smithi029.stdout: "tiers": [], 2024-09-16T20:48:18.028 INFO:teuthology.orchestra.run.smithi029.stdout: "tier_of": -1, 2024-09-16T20:48:18.028 INFO:teuthology.orchestra.run.smithi029.stdout: "read_tier": -1, 2024-09-16T20:48:18.028 INFO:teuthology.orchestra.run.smithi029.stdout: "write_tier": -1, 2024-09-16T20:48:18.028 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_mode": "none", 2024-09-16T20:48:18.028 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_bytes": 0, 2024-09-16T20:48:18.028 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_objects": 0, 2024-09-16T20:48:18.028 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T20:48:18.028 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T20:48:18.028 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T20:48:18.028 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_flush_age": 0, 2024-09-16T20:48:18.029 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_evict_age": 0, 2024-09-16T20:48:18.029 INFO:teuthology.orchestra.run.smithi029.stdout: "erasure_code_profile": "", 2024-09-16T20:48:18.029 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_params": { 2024-09-16T20:48:18.029 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "none" 2024-09-16T20:48:18.029 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.029 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_period": 0, 2024-09-16T20:48:18.029 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_count": 0, 2024-09-16T20:48:18.029 INFO:teuthology.orchestra.run.smithi029.stdout: "use_gmt_hitset": true, 2024-09-16T20:48:18.029 INFO:teuthology.orchestra.run.smithi029.stdout: "min_read_recency_for_promote": 0, 2024-09-16T20:48:18.029 INFO:teuthology.orchestra.run.smithi029.stdout: "min_write_recency_for_promote": 0, 2024-09-16T20:48:18.029 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T20:48:18.029 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_search_last_n": 0, 2024-09-16T20:48:18.029 INFO:teuthology.orchestra.run.smithi029.stdout: "grade_table": [], 2024-09-16T20:48:18.029 INFO:teuthology.orchestra.run.smithi029.stdout: "stripe_width": 0, 2024-09-16T20:48:18.030 INFO:teuthology.orchestra.run.smithi029.stdout: "expected_num_objects": 0, 2024-09-16T20:48:18.030 INFO:teuthology.orchestra.run.smithi029.stdout: "fast_read": false, 2024-09-16T20:48:18.030 INFO:teuthology.orchestra.run.smithi029.stdout: "options": {}, 2024-09-16T20:48:18.030 INFO:teuthology.orchestra.run.smithi029.stdout: "application_metadata": { 2024-09-16T20:48:18.030 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd": {} 2024-09-16T20:48:18.030 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.030 INFO:teuthology.orchestra.run.smithi029.stdout: "read_balance": { 2024-09-16T20:48:18.030 INFO:teuthology.orchestra.run.smithi029.stdout: "score_type": "Fair distribution", 2024-09-16T20:48:18.030 INFO:teuthology.orchestra.run.smithi029.stdout: "score_acting": 1.1299999952316284, 2024-09-16T20:48:18.030 INFO:teuthology.orchestra.run.smithi029.stdout: "score_stable": 1.1299999952316284, 2024-09-16T20:48:18.030 INFO:teuthology.orchestra.run.smithi029.stdout: "optimal_score": 1, 2024-09-16T20:48:18.030 INFO:teuthology.orchestra.run.smithi029.stdout: "raw_score_acting": 1.1299999952316284, 2024-09-16T20:48:18.030 INFO:teuthology.orchestra.run.smithi029.stdout: "raw_score_stable": 1.1299999952316284, 2024-09-16T20:48:18.030 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity_weighted": 1, 2024-09-16T20:48:18.031 INFO:teuthology.orchestra.run.smithi029.stdout: "average_primary_affinity": 1, 2024-09-16T20:48:18.031 INFO:teuthology.orchestra.run.smithi029.stdout: "average_primary_affinity_weighted": 1 2024-09-16T20:48:18.031 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.031 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.031 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-16T20:48:18.031 INFO:teuthology.orchestra.run.smithi029.stdout: "osds": [ 2024-09-16T20:48:18.031 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.031 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 0, 2024-09-16T20:48:18.031 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "6c3cb2d7-7d23-4566-9ca1-d7b08e12360c", 2024-09-16T20:48:18.031 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-16T20:48:18.031 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-16T20:48:18.031 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-16T20:48:18.031 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-16T20:48:18.031 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-16T20:48:18.032 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-16T20:48:18.032 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 13, 2024-09-16T20:48:18.032 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 23, 2024-09-16T20:48:18.032 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-16T20:48:18.032 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-16T20:48:18.032 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-16T20:48:18.032 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:48:18.032 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.032 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:48:18.032 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6816", 2024-09-16T20:48:18.032 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:48:18.032 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.032 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.032 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:48:18.033 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6817", 2024-09-16T20:48:18.033 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:48:18.033 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.033 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:18.033 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.033 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-16T20:48:18.033 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:48:18.033 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.033 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:48:18.033 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6818", 2024-09-16T20:48:18.033 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:48:18.033 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.033 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.033 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:48:18.034 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6819", 2024-09-16T20:48:18.034 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:48:18.034 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.034 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:18.034 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.034 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-16T20:48:18.034 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:48:18.034 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.034 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:48:18.034 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6822", 2024-09-16T20:48:18.034 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:48:18.034 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.034 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.034 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:48:18.035 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6823", 2024-09-16T20:48:18.035 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:48:18.035 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.035 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:18.035 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.035 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-16T20:48:18.035 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:48:18.035 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.035 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:48:18.035 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6820", 2024-09-16T20:48:18.035 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:48:18.035 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.035 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.035 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:48:18.036 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6821", 2024-09-16T20:48:18.036 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:48:18.036 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.036 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:18.036 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.036 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.29:6817/2744025949", 2024-09-16T20:48:18.036 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.29:6819/2744025949", 2024-09-16T20:48:18.036 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.29:6823/2744025949", 2024-09-16T20:48:18.036 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.29:6821/2744025949", 2024-09-16T20:48:18.036 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-16T20:48:18.036 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-16T20:48:18.036 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-16T20:48:18.036 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:18.036 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.037 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.037 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 1, 2024-09-16T20:48:18.037 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "5cf8c20c-1ed2-42ae-a8b0-c88cc4e36bce", 2024-09-16T20:48:18.037 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-16T20:48:18.037 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-16T20:48:18.037 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-16T20:48:18.037 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-16T20:48:18.037 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-16T20:48:18.037 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-16T20:48:18.037 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 13, 2024-09-16T20:48:18.037 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 23, 2024-09-16T20:48:18.037 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-16T20:48:18.037 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-16T20:48:18.037 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-16T20:48:18.037 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:48:18.038 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.038 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:48:18.038 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6800", 2024-09-16T20:48:18.038 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:48:18.038 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.038 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.038 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:48:18.038 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6801", 2024-09-16T20:48:18.038 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:48:18.038 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.038 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:18.038 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.038 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-16T20:48:18.038 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:48:18.039 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.039 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:48:18.039 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6802", 2024-09-16T20:48:18.039 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:48:18.039 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.039 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.039 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:48:18.039 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6803", 2024-09-16T20:48:18.039 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:48:18.039 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.039 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:18.039 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.039 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-16T20:48:18.039 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:48:18.040 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.040 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:48:18.040 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6806", 2024-09-16T20:48:18.040 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:48:18.040 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.040 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.040 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:48:18.040 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6807", 2024-09-16T20:48:18.040 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:48:18.040 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.040 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:18.040 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.040 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-16T20:48:18.040 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:48:18.041 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.041 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:48:18.041 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6804", 2024-09-16T20:48:18.041 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:48:18.041 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.041 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.041 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:48:18.041 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6805", 2024-09-16T20:48:18.041 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:48:18.041 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.041 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:18.041 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.041 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.29:6801/208628762", 2024-09-16T20:48:18.042 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.29:6803/208628762", 2024-09-16T20:48:18.042 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.29:6807/208628762", 2024-09-16T20:48:18.042 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.29:6805/208628762", 2024-09-16T20:48:18.042 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-16T20:48:18.042 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-16T20:48:18.042 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-16T20:48:18.042 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:18.042 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.042 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.042 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 2, 2024-09-16T20:48:18.042 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "075d16da-82bf-449b-9f6e-542c9e8d810b", 2024-09-16T20:48:18.042 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-16T20:48:18.042 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-16T20:48:18.042 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-16T20:48:18.043 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-16T20:48:18.043 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-16T20:48:18.043 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-16T20:48:18.043 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 13, 2024-09-16T20:48:18.043 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 23, 2024-09-16T20:48:18.043 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-16T20:48:18.043 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-16T20:48:18.043 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-16T20:48:18.043 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:48:18.043 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.043 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:48:18.043 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6808", 2024-09-16T20:48:18.043 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:48:18.043 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.044 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.044 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:48:18.044 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6809", 2024-09-16T20:48:18.044 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:48:18.044 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.044 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:18.044 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.044 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-16T20:48:18.044 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:48:18.044 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.044 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:48:18.044 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6810", 2024-09-16T20:48:18.044 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:48:18.045 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.045 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.045 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:48:18.045 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6811", 2024-09-16T20:48:18.045 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:48:18.045 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.045 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:18.045 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.045 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-16T20:48:18.045 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:48:18.045 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.045 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:48:18.045 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6814", 2024-09-16T20:48:18.045 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:48:18.046 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.046 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.046 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:48:18.046 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6815", 2024-09-16T20:48:18.046 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:48:18.046 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.046 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:18.046 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.046 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-16T20:48:18.046 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:48:18.046 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.046 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:48:18.046 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6812", 2024-09-16T20:48:18.046 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:48:18.047 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.047 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.047 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:48:18.047 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6813", 2024-09-16T20:48:18.047 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:48:18.047 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.047 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:18.047 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.047 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.29:6809/3447426536", 2024-09-16T20:48:18.047 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.29:6811/3447426536", 2024-09-16T20:48:18.047 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.29:6815/3447426536", 2024-09-16T20:48:18.047 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.29:6813/3447426536", 2024-09-16T20:48:18.048 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-16T20:48:18.048 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-16T20:48:18.048 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-16T20:48:18.048 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:18.048 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.048 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.048 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 3, 2024-09-16T20:48:18.048 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "0612be38-3a24-41b6-aba4-5a2fe985165e", 2024-09-16T20:48:18.048 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-16T20:48:18.048 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-16T20:48:18.048 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-16T20:48:18.049 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-16T20:48:18.049 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-16T20:48:18.049 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-16T20:48:18.049 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 13, 2024-09-16T20:48:18.049 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 23, 2024-09-16T20:48:18.049 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-16T20:48:18.049 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-16T20:48:18.049 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-16T20:48:18.049 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:48:18.049 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.049 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:48:18.050 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6808", 2024-09-16T20:48:18.050 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:48:18.050 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.050 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.050 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:48:18.050 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6809", 2024-09-16T20:48:18.050 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:48:18.050 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.050 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:18.050 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.050 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-16T20:48:18.050 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:48:18.050 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.050 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:48:18.051 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6810", 2024-09-16T20:48:18.051 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:48:18.051 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.051 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.051 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:48:18.051 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6811", 2024-09-16T20:48:18.051 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:48:18.051 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.051 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:18.051 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.051 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-16T20:48:18.051 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:48:18.052 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.052 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:48:18.052 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6814", 2024-09-16T20:48:18.052 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:48:18.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.052 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.052 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:48:18.052 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6815", 2024-09-16T20:48:18.052 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:48:18.052 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.052 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:18.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.052 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-16T20:48:18.052 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:48:18.053 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.053 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:48:18.053 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6812", 2024-09-16T20:48:18.053 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:48:18.053 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.053 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.053 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:48:18.053 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6813", 2024-09-16T20:48:18.053 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:48:18.053 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.053 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:18.053 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.054 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.43:6809/3687583891", 2024-09-16T20:48:18.054 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.43:6811/3687583891", 2024-09-16T20:48:18.054 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.43:6815/3687583891", 2024-09-16T20:48:18.054 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.43:6813/3687583891", 2024-09-16T20:48:18.054 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-16T20:48:18.054 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-16T20:48:18.054 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-16T20:48:18.054 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:18.054 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.054 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.054 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 4, 2024-09-16T20:48:18.054 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "be590bf8-e9fa-44ca-a471-6f6bcb087ca7", 2024-09-16T20:48:18.054 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-16T20:48:18.055 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-16T20:48:18.055 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-16T20:48:18.055 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-16T20:48:18.055 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-16T20:48:18.055 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-16T20:48:18.055 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 11, 2024-09-16T20:48:18.055 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 23, 2024-09-16T20:48:18.055 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-16T20:48:18.055 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-16T20:48:18.055 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-16T20:48:18.055 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:48:18.055 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.055 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:48:18.056 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6800", 2024-09-16T20:48:18.056 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:48:18.056 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.056 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.056 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:48:18.056 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6801", 2024-09-16T20:48:18.056 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:48:18.056 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.056 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:18.056 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.056 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-16T20:48:18.056 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:48:18.057 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.057 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:48:18.057 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6802", 2024-09-16T20:48:18.057 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:48:18.057 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.057 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.057 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:48:18.057 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6803", 2024-09-16T20:48:18.057 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:48:18.057 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.057 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:18.057 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.057 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-16T20:48:18.058 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:48:18.058 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.058 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:48:18.058 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6806", 2024-09-16T20:48:18.058 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:48:18.058 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.058 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.058 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:48:18.058 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6807", 2024-09-16T20:48:18.058 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:48:18.058 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.058 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:18.058 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.059 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-16T20:48:18.059 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:48:18.059 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.059 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:48:18.059 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6804", 2024-09-16T20:48:18.059 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:48:18.059 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.059 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.059 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:48:18.059 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6805", 2024-09-16T20:48:18.059 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:48:18.059 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.059 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:18.059 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.060 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.43:6801/2505220646", 2024-09-16T20:48:18.060 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.43:6803/2505220646", 2024-09-16T20:48:18.060 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.43:6807/2505220646", 2024-09-16T20:48:18.060 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.43:6805/2505220646", 2024-09-16T20:48:18.060 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-16T20:48:18.060 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-16T20:48:18.060 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-16T20:48:18.060 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:18.060 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.060 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.060 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 5, 2024-09-16T20:48:18.060 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "4a208e30-031b-4a9f-af52-e26d4a039ea6", 2024-09-16T20:48:18.060 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-16T20:48:18.061 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-16T20:48:18.061 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-16T20:48:18.061 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-16T20:48:18.061 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-16T20:48:18.061 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-16T20:48:18.061 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 13, 2024-09-16T20:48:18.061 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 23, 2024-09-16T20:48:18.061 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-16T20:48:18.061 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-16T20:48:18.061 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-16T20:48:18.061 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:48:18.061 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.061 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:48:18.061 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6816", 2024-09-16T20:48:18.062 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:48:18.062 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.062 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.062 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:48:18.062 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6817", 2024-09-16T20:48:18.062 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:48:18.062 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.062 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:18.062 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.062 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-16T20:48:18.062 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:48:18.062 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.062 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:48:18.062 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6818", 2024-09-16T20:48:18.062 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:48:18.063 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.063 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.063 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:48:18.063 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6819", 2024-09-16T20:48:18.063 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:48:18.063 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.063 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:18.063 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.063 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-16T20:48:18.063 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:48:18.063 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.063 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:48:18.063 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6822", 2024-09-16T20:48:18.063 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:48:18.064 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.064 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.064 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:48:18.064 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6823", 2024-09-16T20:48:18.064 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:48:18.064 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.064 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:18.064 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.064 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-16T20:48:18.064 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:48:18.064 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.064 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:48:18.064 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6820", 2024-09-16T20:48:18.064 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:48:18.065 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.065 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.065 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:48:18.065 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6821", 2024-09-16T20:48:18.065 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:48:18.065 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.065 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:18.065 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.065 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.43:6817/1517367307", 2024-09-16T20:48:18.065 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.43:6819/1517367307", 2024-09-16T20:48:18.065 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.43:6823/1517367307", 2024-09-16T20:48:18.065 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.43:6821/1517367307", 2024-09-16T20:48:18.065 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-16T20:48:18.066 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-16T20:48:18.066 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-16T20:48:18.066 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:18.066 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.066 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-16T20:48:18.066 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_xinfo": [ 2024-09-16T20:48:18.066 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.066 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 0, 2024-09-16T20:48:18.066 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-16T20:48:18.066 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-16T20:48:18.066 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-16T20:48:18.066 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540719139924082687, 2024-09-16T20:48:18.066 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-16T20:48:18.066 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-16T20:47:47.887412+0000", 2024-09-16T20:48:18.067 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-16T20:48:18.067 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.067 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.067 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 1, 2024-09-16T20:48:18.067 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-16T20:48:18.067 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-16T20:48:18.067 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-16T20:48:18.067 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540719139924082687, 2024-09-16T20:48:18.067 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-16T20:48:18.067 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-16T20:47:47.658782+0000", 2024-09-16T20:48:18.067 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-16T20:48:18.067 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.067 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.067 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 2, 2024-09-16T20:48:18.068 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-16T20:48:18.068 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-16T20:48:18.068 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-16T20:48:18.068 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540719139924082687, 2024-09-16T20:48:18.068 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-16T20:48:18.068 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-16T20:47:47.453752+0000", 2024-09-16T20:48:18.068 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-16T20:48:18.068 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.068 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.068 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 3, 2024-09-16T20:48:18.068 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-16T20:48:18.068 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-16T20:48:18.068 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-16T20:48:18.068 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540719139924082687, 2024-09-16T20:48:18.069 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-16T20:48:18.069 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-16T20:47:47.487992+0000", 2024-09-16T20:48:18.069 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-16T20:48:18.069 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.069 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.069 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 4, 2024-09-16T20:48:18.069 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-16T20:48:18.069 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-16T20:48:18.069 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-16T20:48:18.069 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540719139924082687, 2024-09-16T20:48:18.069 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-16T20:48:18.069 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-16T20:47:46.943344+0000", 2024-09-16T20:48:18.069 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-16T20:48:18.070 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.070 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.070 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 5, 2024-09-16T20:48:18.070 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-16T20:48:18.070 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-16T20:48:18.070 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-16T20:48:18.070 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540719139924082687, 2024-09-16T20:48:18.070 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-16T20:48:18.070 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-16T20:47:47.901547+0000", 2024-09-16T20:48:18.070 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-16T20:48:18.070 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.070 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-16T20:48:18.070 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_upmap": [], 2024-09-16T20:48:18.070 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_upmap_items": [], 2024-09-16T20:48:18.070 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_upmap_primaries": [], 2024-09-16T20:48:18.071 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_temp": [], 2024-09-16T20:48:18.071 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_temp": [], 2024-09-16T20:48:18.071 INFO:teuthology.orchestra.run.smithi029.stdout: "blocklist": {}, 2024-09-16T20:48:18.071 INFO:teuthology.orchestra.run.smithi029.stdout: "range_blocklist": {}, 2024-09-16T20:48:18.071 INFO:teuthology.orchestra.run.smithi029.stdout: "erasure_code_profiles": { 2024-09-16T20:48:18.071 INFO:teuthology.orchestra.run.smithi029.stdout: "default": { 2024-09-16T20:48:18.071 INFO:teuthology.orchestra.run.smithi029.stdout: "crush-failure-domain": "osd", 2024-09-16T20:48:18.071 INFO:teuthology.orchestra.run.smithi029.stdout: "k": "2", 2024-09-16T20:48:18.071 INFO:teuthology.orchestra.run.smithi029.stdout: "m": "1", 2024-09-16T20:48:18.071 INFO:teuthology.orchestra.run.smithi029.stdout: "plugin": "jerasure", 2024-09-16T20:48:18.071 INFO:teuthology.orchestra.run.smithi029.stdout: "technique": "reed_sol_van" 2024-09-16T20:48:18.071 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.071 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.071 INFO:teuthology.orchestra.run.smithi029.stdout: "removed_snaps_queue": [ 2024-09-16T20:48:18.072 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.072 INFO:teuthology.orchestra.run.smithi029.stdout: "pool": 3, 2024-09-16T20:48:18.072 INFO:teuthology.orchestra.run.smithi029.stdout: "snaps": [ 2024-09-16T20:48:18.072 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.072 INFO:teuthology.orchestra.run.smithi029.stdout: "begin": 2, 2024-09-16T20:48:18.072 INFO:teuthology.orchestra.run.smithi029.stdout: "length": 1 2024-09-16T20:48:18.072 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.072 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:18.072 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.072 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-16T20:48:18.072 INFO:teuthology.orchestra.run.smithi029.stdout: "new_removed_snaps": [], 2024-09-16T20:48:18.072 INFO:teuthology.orchestra.run.smithi029.stdout: "new_purged_snaps": [], 2024-09-16T20:48:18.072 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_node_flags": {}, 2024-09-16T20:48:18.073 INFO:teuthology.orchestra.run.smithi029.stdout: "device_class_flags": {}, 2024-09-16T20:48:18.073 INFO:teuthology.orchestra.run.smithi029.stdout: "stretch_mode": { 2024-09-16T20:48:18.073 INFO:teuthology.orchestra.run.smithi029.stdout: "stretch_mode_enabled": false, 2024-09-16T20:48:18.073 INFO:teuthology.orchestra.run.smithi029.stdout: "stretch_bucket_count": 0, 2024-09-16T20:48:18.073 INFO:teuthology.orchestra.run.smithi029.stdout: "degraded_stretch_mode": 0, 2024-09-16T20:48:18.073 INFO:teuthology.orchestra.run.smithi029.stdout: "recovering_stretch_mode": 0, 2024-09-16T20:48:18.073 INFO:teuthology.orchestra.run.smithi029.stdout: "stretch_mode_bucket": 0 2024-09-16T20:48:18.073 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.073 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:18.074 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:18.008+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:18.076 INFO:tasks.thrashosds.thrasher:j is {'epoch': 32, 'fsid': 'e28af4a7-5fcd-4de8-8f77-a2b575361bb1', 'created': '2024-09-16T20:47:39.760609+0000', 'modified': '2024-09-16T20:48:16.589813+0000', 'last_up_change': '2024-09-16T20:47:50.238013+0000', 'last_in_change': '2024-09-16T20:47:41.378753+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T20:47:52.220750+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-16T20:47:56.756666+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-16T20:48:06.418889+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 6, 'snap_epoch': 32, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '6c3cb2d7-7d23-4566-9ca1-d7b08e12360c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6816', 'nonce': 2744025949}, {'type': 'v1', 'addr': '172.21.15.29:6817', 'nonce': 2744025949}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6818', 'nonce': 2744025949}, {'type': 'v1', 'addr': '172.21.15.29:6819', 'nonce': 2744025949}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6822', 'nonce': 2744025949}, {'type': 'v1', 'addr': '172.21.15.29:6823', 'nonce': 2744025949}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6820', 'nonce': 2744025949}, {'type': 'v1', 'addr': '172.21.15.29:6821', 'nonce': 2744025949}]}, 'public_addr': '172.21.15.29:6817/2744025949', 'cluster_addr': '172.21.15.29:6819/2744025949', 'heartbeat_back_addr': '172.21.15.29:6823/2744025949', 'heartbeat_front_addr': '172.21.15.29:6821/2744025949', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '5cf8c20c-1ed2-42ae-a8b0-c88cc4e36bce', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6800', 'nonce': 208628762}, {'type': 'v1', 'addr': '172.21.15.29:6801', 'nonce': 208628762}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6802', 'nonce': 208628762}, {'type': 'v1', 'addr': '172.21.15.29:6803', 'nonce': 208628762}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6806', 'nonce': 208628762}, {'type': 'v1', 'addr': '172.21.15.29:6807', 'nonce': 208628762}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6804', 'nonce': 208628762}, {'type': 'v1', 'addr': '172.21.15.29:6805', 'nonce': 208628762}]}, 'public_addr': '172.21.15.29:6801/208628762', 'cluster_addr': '172.21.15.29:6803/208628762', 'heartbeat_back_addr': '172.21.15.29:6807/208628762', 'heartbeat_front_addr': '172.21.15.29:6805/208628762', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '075d16da-82bf-449b-9f6e-542c9e8d810b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6808', 'nonce': 3447426536}, {'type': 'v1', 'addr': '172.21.15.29:6809', 'nonce': 3447426536}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6810', 'nonce': 3447426536}, {'type': 'v1', 'addr': '172.21.15.29:6811', 'nonce': 3447426536}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6814', 'nonce': 3447426536}, {'type': 'v1', 'addr': '172.21.15.29:6815', 'nonce': 3447426536}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6812', 'nonce': 3447426536}, {'type': 'v1', 'addr': '172.21.15.29:6813', 'nonce': 3447426536}]}, 'public_addr': '172.21.15.29:6809/3447426536', 'cluster_addr': '172.21.15.29:6811/3447426536', 'heartbeat_back_addr': '172.21.15.29:6815/3447426536', 'heartbeat_front_addr': '172.21.15.29:6813/3447426536', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '0612be38-3a24-41b6-aba4-5a2fe985165e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6808', 'nonce': 3687583891}, {'type': 'v1', 'addr': '172.21.15.43:6809', 'nonce': 3687583891}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6810', 'nonce': 3687583891}, {'type': 'v1', 'addr': '172.21.15.43:6811', 'nonce': 3687583891}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6814', 'nonce': 3687583891}, {'type': 'v1', 'addr': '172.21.15.43:6815', 'nonce': 3687583891}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6812', 'nonce': 3687583891}, {'type': 'v1', 'addr': '172.21.15.43:6813', 'nonce': 3687583891}]}, 'public_addr': '172.21.15.43:6809/3687583891', 'cluster_addr': '172.21.15.43:6811/3687583891', 'heartbeat_back_addr': '172.21.15.43:6815/3687583891', 'heartbeat_front_addr': '172.21.15.43:6813/3687583891', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'be590bf8-e9fa-44ca-a471-6f6bcb087ca7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6800', 'nonce': 2505220646}, {'type': 'v1', 'addr': '172.21.15.43:6801', 'nonce': 2505220646}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6802', 'nonce': 2505220646}, {'type': 'v1', 'addr': '172.21.15.43:6803', 'nonce': 2505220646}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6806', 'nonce': 2505220646}, {'type': 'v1', 'addr': '172.21.15.43:6807', 'nonce': 2505220646}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6804', 'nonce': 2505220646}, {'type': 'v1', 'addr': '172.21.15.43:6805', 'nonce': 2505220646}]}, 'public_addr': '172.21.15.43:6801/2505220646', 'cluster_addr': '172.21.15.43:6803/2505220646', 'heartbeat_back_addr': '172.21.15.43:6807/2505220646', 'heartbeat_front_addr': '172.21.15.43:6805/2505220646', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '4a208e30-031b-4a9f-af52-e26d4a039ea6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6816', 'nonce': 1517367307}, {'type': 'v1', 'addr': '172.21.15.43:6817', 'nonce': 1517367307}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6818', 'nonce': 1517367307}, {'type': 'v1', 'addr': '172.21.15.43:6819', 'nonce': 1517367307}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6822', 'nonce': 1517367307}, {'type': 'v1', 'addr': '172.21.15.43:6823', 'nonce': 1517367307}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6820', 'nonce': 1517367307}, {'type': 'v1', 'addr': '172.21.15.43:6821', 'nonce': 1517367307}]}, 'public_addr': '172.21.15.43:6817/1517367307', 'cluster_addr': '172.21.15.43:6819/1517367307', 'heartbeat_back_addr': '172.21.15.43:6823/1517367307', 'heartbeat_front_addr': '172.21.15.43:6821/1517367307', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T20:47:47.887412+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T20:47:47.658782+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T20:47:47.453752+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T20:47:47.487992+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T20:47:46.943344+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T20:47:47.901547+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 2, 'length': 1}]}], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-16T20:48:18.077 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T20:48:18.085 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:18.084+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:18.181 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:18.182 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:48:18.182 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:48:18.182 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:18.186 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:18.185+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:18.192 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:48:18.287 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:18.286+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:18.340 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:18.341 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:48:18.341 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:48:18.341 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:48:18.341 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:18.349 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:48:18.388 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:18.386+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:18.416 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:48:18.417 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-09-16T20:48:18.429 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":42,"stamp":"2024-09-16T20:48:18.208629+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":94094195,"num_objects":81,"num_object_clones":24,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":11,"num_read":10492,"num_read_kb":49054,"num_write":1063,"num_write_kb":6433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":912,"ondisk_log_size":912,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":81,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":204604,"kb_used_data":4988,"kb_used_omap":9,"kb_used_meta":199478,"kb_avail":566026436,"statfs":{"total":579820584960,"available":579611070464,"internally_reserved":0,"allocated":5107712,"data_stored":3800948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9567,"internal_metadata":204266145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,9],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":91430143,"num_objects":70,"num_object_clones":24,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":11,"num_read":10181,"num_read_kb":48614,"num_write":928,"num_write_kb":4243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.631914"},"pg_stats":[{"pgid":"3.1f","version":"31'26","reported_seq":247,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-16T20:48:16.172987+0000","last_change":"2024-09-16T20:48:09.568824+0000","last_active":"2024-09-16T20:48:16.172987+0000","last_peered":"2024-09-16T20:48:16.172987+0000","last_clean":"2024-09-16T20:48:16.172987+0000","last_became_active":"2024-09-16T20:48:07.571756+0000","last_became_peered":"2024-09-16T20:48:07.571756+0000","last_unstale":"2024-09-16T20:48:16.172987+0000","last_undegraded":"2024-09-16T20:48:16.172987+0000","last_fullsized":"2024-09-16T20:48:16.172987+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:09:28.123968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071634499999999996,"stat_sum":{"num_bytes":14493200,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":199,"num_read_kb":1517,"num_write":26,"num_write_kb":222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1e","version":"31'24","reported_seq":548,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-16T20:48:16.174116+0000","last_change":"2024-09-16T20:48:09.577092+0000","last_active":"2024-09-16T20:48:16.174116+0000","last_peered":"2024-09-16T20:48:16.174116+0000","last_clean":"2024-09-16T20:48:16.174116+0000","last_became_active":"2024-09-16T20:48:07.566669+0000","last_became_peered":"2024-09-16T20:48:07.566669+0000","last_unstale":"2024-09-16T20:48:16.174116+0000","last_undegraded":"2024-09-16T20:48:16.174116+0000","last_fullsized":"2024-09-16T20:48:16.174116+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:05:47.652868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084864999999999995,"stat_sum":{"num_bytes":9127936,"num_objects":7,"num_object_clones":3,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":503,"num_read_kb":3939,"num_write":21,"num_write_kb":317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1d","version":"29'12","reported_seq":53,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-16T20:48:15.587871+0000","last_change":"2024-09-16T20:48:09.564507+0000","last_active":"2024-09-16T20:48:15.587871+0000","last_peered":"2024-09-16T20:48:15.587871+0000","last_clean":"2024-09-16T20:48:15.587871+0000","last_became_active":"2024-09-16T20:48:07.568025+0000","last_became_peered":"2024-09-16T20:48:07.568025+0000","last_unstale":"2024-09-16T20:48:15.587871+0000","last_undegraded":"2024-09-16T20:48:15.587871+0000","last_fullsized":"2024-09-16T20:48:15.587871+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:31:11.791305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068985699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":61,"num_write":12,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1c","version":"31'9","reported_seq":65,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-16T20:48:16.154757+0000","last_change":"2024-09-16T20:48:09.565661+0000","last_active":"2024-09-16T20:48:16.154757+0000","last_peered":"2024-09-16T20:48:16.154757+0000","last_clean":"2024-09-16T20:48:16.154757+0000","last_became_active":"2024-09-16T20:48:07.565241+0000","last_became_peered":"2024-09-16T20:48:07.565241+0000","last_unstale":"2024-09-16T20:48:16.154757+0000","last_undegraded":"2024-09-16T20:48:16.154757+0000","last_fullsized":"2024-09-16T20:48:16.154757+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:04:56.764618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001045589,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":33,"num_read_kb":31,"num_write":9,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1b","version":"31'63","reported_seq":242,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-16T20:48:16.204213+0000","last_change":"2024-09-16T20:48:09.576745+0000","last_active":"2024-09-16T20:48:16.204213+0000","last_peered":"2024-09-16T20:48:16.204213+0000","last_clean":"2024-09-16T20:48:16.204213+0000","last_became_active":"2024-09-16T20:48:07.559076+0000","last_became_peered":"2024-09-16T20:48:07.559076+0000","last_unstale":"2024-09-16T20:48:16.204213+0000","last_undegraded":"2024-09-16T20:48:16.204213+0000","last_fullsized":"2024-09-16T20:48:16.204213+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:37:16.500792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057432399999999997,"stat_sum":{"num_bytes":5399588,"num_objects":5,"num_object_clones":2,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":396,"num_read_kb":2926,"num_write":97,"num_write_kb":275,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1a","version":"31'14","reported_seq":291,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-16T20:48:16.210611+0000","last_change":"2024-09-16T20:48:09.566668+0000","last_active":"2024-09-16T20:48:16.210611+0000","last_peered":"2024-09-16T20:48:16.210611+0000","last_clean":"2024-09-16T20:48:16.210611+0000","last_became_active":"2024-09-16T20:48:07.566792+0000","last_became_peered":"2024-09-16T20:48:07.566792+0000","last_unstale":"2024-09-16T20:48:16.210611+0000","last_undegraded":"2024-09-16T20:48:16.210611+0000","last_fullsized":"2024-09-16T20:48:16.210611+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:23:02.437723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002037849,"stat_sum":{"num_bytes":1245244,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":258,"num_read_kb":372,"num_write":14,"num_write_kb":179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.19","version":"28'2","reported_seq":193,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-16T20:48:13.217429+0000","last_change":"2024-09-16T20:48:09.564648+0000","last_active":"2024-09-16T20:48:13.217429+0000","last_peered":"2024-09-16T20:48:13.217429+0000","last_clean":"2024-09-16T20:48:13.217429+0000","last_became_active":"2024-09-16T20:48:07.564636+0000","last_became_peered":"2024-09-16T20:48:07.564636+0000","last_unstale":"2024-09-16T20:48:13.217429+0000","last_undegraded":"2024-09-16T20:48:13.217429+0000","last_fullsized":"2024-09-16T20:48:13.217429+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:23:34.024979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057190699999999997,"stat_sum":{"num_bytes":2073600,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":167,"num_read_kb":217,"num_write":2,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.18","version":"31'5","reported_seq":265,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-16T20:48:16.177231+0000","last_change":"2024-09-16T20:48:09.577049+0000","last_active":"2024-09-16T20:48:16.177231+0000","last_peered":"2024-09-16T20:48:16.177231+0000","last_clean":"2024-09-16T20:48:16.177231+0000","last_became_active":"2024-09-16T20:48:07.562732+0000","last_became_peered":"2024-09-16T20:48:07.562732+0000","last_unstale":"2024-09-16T20:48:16.177231+0000","last_undegraded":"2024-09-16T20:48:16.177231+0000","last_fullsized":"2024-09-16T20:48:16.177231+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:04:02.197752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011061790000000001,"stat_sum":{"num_bytes":786432,"num_objects":2,"num_object_clones":1,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":230,"num_read_kb":3560,"num_write":4,"num_write_kb":784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.17","version":"31'68","reported_seq":1192,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-16T20:48:16.013462+0000","last_change":"2024-09-16T20:48:09.568656+0000","last_active":"2024-09-16T20:48:16.013462+0000","last_peered":"2024-09-16T20:48:16.013462+0000","last_clean":"2024-09-16T20:48:16.013462+0000","last_became_active":"2024-09-16T20:48:07.567576+0000","last_became_peered":"2024-09-16T20:48:07.567576+0000","last_unstale":"2024-09-16T20:48:16.013462+0000","last_undegraded":"2024-09-16T20:48:16.013462+0000","last_fullsized":"2024-09-16T20:48:16.013462+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:27:18.194750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057915500000000003,"stat_sum":{"num_bytes":1572864,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1678,"num_read_kb":3726,"num_write":40,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.16","version":"31'164","reported_seq":219,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-16T20:48:16.155009+0000","last_change":"2024-09-16T20:48:09.564832+0000","last_active":"2024-09-16T20:48:16.155009+0000","last_peered":"2024-09-16T20:48:16.155009+0000","last_clean":"2024-09-16T20:48:16.155009+0000","last_became_active":"2024-09-16T20:48:07.565965+0000","last_became_peered":"2024-09-16T20:48:07.565965+0000","last_unstale":"2024-09-16T20:48:16.155009+0000","last_undegraded":"2024-09-16T20:48:16.155009+0000","last_fullsized":"2024-09-16T20:48:16.155009+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:50:33.657233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010075049999999999,"stat_sum":{"num_bytes":649253,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":854,"num_read_kb":776,"num_write":290,"num_write_kb":437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.15","version":"29'11","reported_seq":253,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-16T20:48:15.591421+0000","last_change":"2024-09-16T20:48:09.570733+0000","last_active":"2024-09-16T20:48:15.591421+0000","last_peered":"2024-09-16T20:48:15.591421+0000","last_clean":"2024-09-16T20:48:15.591421+0000","last_became_active":"2024-09-16T20:48:07.562407+0000","last_became_peered":"2024-09-16T20:48:07.562407+0000","last_unstale":"2024-09-16T20:48:15.591421+0000","last_undegraded":"2024-09-16T20:48:15.591421+0000","last_fullsized":"2024-09-16T20:48:15.591421+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:02:29.227584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051100699999999995,"stat_sum":{"num_bytes":1388544,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":210,"num_read_kb":976,"num_write":13,"num_write_kb":238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.14","version":"29'7","reported_seq":78,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-16T20:48:15.588732+0000","last_change":"2024-09-16T20:48:09.566005+0000","last_active":"2024-09-16T20:48:15.588732+0000","last_peered":"2024-09-16T20:48:15.588732+0000","last_clean":"2024-09-16T20:48:15.588732+0000","last_became_active":"2024-09-16T20:48:07.568236+0000","last_became_peered":"2024-09-16T20:48:07.568236+0000","last_unstale":"2024-09-16T20:48:15.588732+0000","last_undegraded":"2024-09-16T20:48:15.588732+0000","last_fullsized":"2024-09-16T20:48:15.588732+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:13:45.398541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001271749,"stat_sum":{"num_bytes":2000896,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":38,"num_read_kb":54,"num_write":7,"num_write_kb":126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":40,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-16T20:48:15.588653+0000","last_change":"2024-09-16T20:48:00.277670+0000","last_active":"2024-09-16T20:48:15.588653+0000","last_peered":"2024-09-16T20:48:15.588653+0000","last_clean":"2024-09-16T20:48:15.588653+0000","last_became_active":"2024-09-16T20:47:58.274489+0000","last_became_peered":"2024-09-16T20:47:58.274489+0000","last_unstale":"2024-09-16T20:48:15.588653+0000","last_undegraded":"2024-09-16T20:48:15.588653+0000","last_fullsized":"2024-09-16T20:48:15.588653+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:27:12.633064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091339500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"28'7","reported_seq":39,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-16T20:48:13.197915+0000","last_change":"2024-09-16T20:48:09.564793+0000","last_active":"2024-09-16T20:48:13.197915+0000","last_peered":"2024-09-16T20:48:13.197915+0000","last_clean":"2024-09-16T20:48:13.197915+0000","last_became_active":"2024-09-16T20:48:07.564677+0000","last_became_peered":"2024-09-16T20:48:07.564677+0000","last_unstale":"2024-09-16T20:48:13.197915+0000","last_undegraded":"2024-09-16T20:48:13.197915+0000","last_fullsized":"2024-09-16T20:48:13.197915+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:28:27.827241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067924999999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":85,"num_write":9,"num_write_kb":484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":34,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-16T20:48:12.577188+0000","last_change":"2024-09-16T20:48:00.277182+0000","last_active":"2024-09-16T20:48:12.577188+0000","last_peered":"2024-09-16T20:48:12.577188+0000","last_clean":"2024-09-16T20:48:12.577188+0000","last_became_active":"2024-09-16T20:47:58.274389+0000","last_became_peered":"2024-09-16T20:47:58.274389+0000","last_unstale":"2024-09-16T20:48:12.577188+0000","last_undegraded":"2024-09-16T20:48:12.577188+0000","last_fullsized":"2024-09-16T20:48:12.577188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:28:02.441830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043659299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"27'2","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-16T20:48:12.577300+0000","last_change":"2024-09-16T20:48:09.565578+0000","last_active":"2024-09-16T20:48:12.577300+0000","last_peered":"2024-09-16T20:48:12.577300+0000","last_clean":"2024-09-16T20:48:12.577300+0000","last_became_active":"2024-09-16T20:48:07.564514+0000","last_became_peered":"2024-09-16T20:48:07.564514+0000","last_unstale":"2024-09-16T20:48:12.577300+0000","last_undegraded":"2024-09-16T20:48:12.577300+0000","last_fullsized":"2024-09-16T20:48:12.577300+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:36:38.545816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012934750000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":40,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-16T20:48:15.588671+0000","last_change":"2024-09-16T20:48:00.278054+0000","last_active":"2024-09-16T20:48:15.588671+0000","last_peered":"2024-09-16T20:48:15.588671+0000","last_clean":"2024-09-16T20:48:15.588671+0000","last_became_active":"2024-09-16T20:47:58.272861+0000","last_became_peered":"2024-09-16T20:47:58.272861+0000","last_unstale":"2024-09-16T20:48:15.588671+0000","last_undegraded":"2024-09-16T20:48:15.588671+0000","last_fullsized":"2024-09-16T20:48:15.588671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:13:59.135547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012429489999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"28'5","reported_seq":30,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-16T20:48:13.199996+0000","last_change":"2024-09-16T20:48:09.564955+0000","last_active":"2024-09-16T20:48:13.199996+0000","last_peered":"2024-09-16T20:48:13.199996+0000","last_clean":"2024-09-16T20:48:13.199996+0000","last_became_active":"2024-09-16T20:48:07.564072+0000","last_became_peered":"2024-09-16T20:48:07.564072+0000","last_unstale":"2024-09-16T20:48:13.199996+0000","last_undegraded":"2024-09-16T20:48:13.199996+0000","last_fullsized":"2024-09-16T20:48:13.199996+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:22:02.353633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076367000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-16T20:48:15.587351+0000","last_change":"2024-09-16T20:48:00.833899+0000","last_active":"2024-09-16T20:48:15.587351+0000","last_peered":"2024-09-16T20:48:15.587351+0000","last_clean":"2024-09-16T20:48:15.587351+0000","last_became_active":"2024-09-16T20:47:58.275031+0000","last_became_peered":"2024-09-16T20:47:58.275031+0000","last_unstale":"2024-09-16T20:48:15.587351+0000","last_undegraded":"2024-09-16T20:48:15.587351+0000","last_fullsized":"2024-09-16T20:48:15.587351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:13:08.658660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046986,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"31'15","reported_seq":274,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-16T20:48:15.981339+0000","last_change":"2024-09-16T20:48:09.565091+0000","last_active":"2024-09-16T20:48:15.981339+0000","last_peered":"2024-09-16T20:48:15.981339+0000","last_clean":"2024-09-16T20:48:15.981339+0000","last_became_active":"2024-09-16T20:48:07.568828+0000","last_became_peered":"2024-09-16T20:48:07.568828+0000","last_unstale":"2024-09-16T20:48:15.981339+0000","last_undegraded":"2024-09-16T20:48:15.981339+0000","last_fullsized":"2024-09-16T20:48:15.981339+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:13:45.978502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010471600000000001,"stat_sum":{"num_bytes":1572880,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":237,"num_read_kb":3347,"num_write":14,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.3","version":"31'25","reported_seq":97,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-16T20:48:16.172961+0000","last_change":"2024-09-16T20:48:09.570254+0000","last_active":"2024-09-16T20:48:16.172961+0000","last_peered":"2024-09-16T20:48:16.172961+0000","last_clean":"2024-09-16T20:48:16.172961+0000","last_became_active":"2024-09-16T20:48:07.568143+0000","last_became_peered":"2024-09-16T20:48:07.568143+0000","last_unstale":"2024-09-16T20:48:16.172961+0000","last_undegraded":"2024-09-16T20:48:16.172961+0000","last_fullsized":"2024-09-16T20:48:16.172961+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:50:44.397219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038800899999999998,"stat_sum":{"num_bytes":2009088,"num_objects":6,"num_object_clones":3,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":3,"num_read":45,"num_read_kb":171,"num_write":22,"num_write_kb":187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-16T20:48:15.590653+0000","last_change":"2024-09-16T20:48:00.276970+0000","last_active":"2024-09-16T20:48:15.590653+0000","last_peered":"2024-09-16T20:48:15.590653+0000","last_clean":"2024-09-16T20:48:15.590653+0000","last_became_active":"2024-09-16T20:47:58.274617+0000","last_became_peered":"2024-09-16T20:47:58.274617+0000","last_unstale":"2024-09-16T20:48:15.590653+0000","last_undegraded":"2024-09-16T20:48:15.590653+0000","last_fullsized":"2024-09-16T20:48:15.590653+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:28:54.467905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062077499999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"31'7","reported_seq":75,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-16T20:48:16.173702+0000","last_change":"2024-09-16T20:48:09.568988+0000","last_active":"2024-09-16T20:48:16.173702+0000","last_peered":"2024-09-16T20:48:16.173702+0000","last_clean":"2024-09-16T20:48:16.173702+0000","last_became_active":"2024-09-16T20:48:07.568564+0000","last_became_peered":"2024-09-16T20:48:07.568564+0000","last_unstale":"2024-09-16T20:48:16.173702+0000","last_undegraded":"2024-09-16T20:48:16.173702+0000","last_fullsized":"2024-09-16T20:48:16.173702+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:42:13.437239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088331900000000003,"stat_sum":{"num_bytes":2104848,"num_objects":5,"num_object_clones":2,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":44,"num_read_kb":35,"num_write":4,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":40,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-16T20:48:15.593557+0000","last_change":"2024-09-16T20:48:01.638893+0000","last_active":"2024-09-16T20:48:15.593557+0000","last_peered":"2024-09-16T20:48:15.593557+0000","last_clean":"2024-09-16T20:48:15.593557+0000","last_became_active":"2024-09-16T20:47:58.273088+0000","last_became_peered":"2024-09-16T20:47:58.273088+0000","last_unstale":"2024-09-16T20:48:15.593557+0000","last_undegraded":"2024-09-16T20:48:15.593557+0000","last_fullsized":"2024-09-16T20:48:15.593557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:27:51.451170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000479672,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"28'6","reported_seq":33,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-16T20:48:13.198974+0000","last_change":"2024-09-16T20:48:09.565596+0000","last_active":"2024-09-16T20:48:13.198974+0000","last_peered":"2024-09-16T20:48:13.198974+0000","last_clean":"2024-09-16T20:48:13.198974+0000","last_became_active":"2024-09-16T20:48:07.566097+0000","last_became_peered":"2024-09-16T20:48:07.566097+0000","last_unstale":"2024-09-16T20:48:13.198974+0000","last_undegraded":"2024-09-16T20:48:13.198974+0000","last_fullsized":"2024-09-16T20:48:13.198974+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:46:08.113010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012840460000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":32,"num_write":6,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":40,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-16T20:48:15.587396+0000","last_change":"2024-09-16T20:48:00.408392+0000","last_active":"2024-09-16T20:48:15.587396+0000","last_peered":"2024-09-16T20:48:15.587396+0000","last_clean":"2024-09-16T20:48:15.587396+0000","last_became_active":"2024-09-16T20:47:58.273260+0000","last_became_peered":"2024-09-16T20:47:58.273260+0000","last_unstale":"2024-09-16T20:48:15.587396+0000","last_undegraded":"2024-09-16T20:48:15.587396+0000","last_fullsized":"2024-09-16T20:48:15.587396+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:01:53.009880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037636000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"31'35","reported_seq":330,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-16T20:48:16.211354+0000","last_change":"2024-09-16T20:48:09.576740+0000","last_active":"2024-09-16T20:48:16.211354+0000","last_peered":"2024-09-16T20:48:16.211354+0000","last_clean":"2024-09-16T20:48:16.211354+0000","last_became_active":"2024-09-16T20:48:07.568016+0000","last_became_peered":"2024-09-16T20:48:07.568016+0000","last_unstale":"2024-09-16T20:48:16.211354+0000","last_undegraded":"2024-09-16T20:48:16.211354+0000","last_fullsized":"2024-09-16T20:48:16.211354+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:16:56.774415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063857599999999997,"stat_sum":{"num_bytes":4094480,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":522,"num_read_kb":542,"num_write":27,"num_write_kb":105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-16T20:48:15.588690+0000","last_change":"2024-09-16T20:48:00.277739+0000","last_active":"2024-09-16T20:48:15.588690+0000","last_peered":"2024-09-16T20:48:15.588690+0000","last_clean":"2024-09-16T20:48:15.588690+0000","last_became_active":"2024-09-16T20:47:58.271174+0000","last_became_peered":"2024-09-16T20:47:58.271174+0000","last_unstale":"2024-09-16T20:48:15.588690+0000","last_undegraded":"2024-09-16T20:48:15.588690+0000","last_fullsized":"2024-09-16T20:48:15.588690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:11:35.985075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095771799999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":131,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-16T20:48:15.588408+0000","last_change":"2024-09-16T20:47:53.928484+0000","last_active":"2024-09-16T20:48:15.588408+0000","last_peered":"2024-09-16T20:48:15.588408+0000","last_clean":"2024-09-16T20:48:15.588408+0000","last_became_active":"2024-09-16T20:47:53.926422+0000","last_became_peered":"2024-09-16T20:47:53.926422+0000","last_unstale":"2024-09-16T20:48:15.588408+0000","last_undegraded":"2024-09-16T20:48:15.588408+0000","last_fullsized":"2024-09-16T20:48:15.588408+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:52.241408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:52.241408+0000","last_clean_scrub_stamp":"2024-09-16T20:47:52.241408+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:18:04.963138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"28'2","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-16T20:48:13.142644+0000","last_change":"2024-09-16T20:48:09.565136+0000","last_active":"2024-09-16T20:48:13.142644+0000","last_peered":"2024-09-16T20:48:13.142644+0000","last_clean":"2024-09-16T20:48:13.142644+0000","last_became_active":"2024-09-16T20:48:07.566139+0000","last_became_peered":"2024-09-16T20:48:07.566139+0000","last_unstale":"2024-09-16T20:48:13.142644+0000","last_undegraded":"2024-09-16T20:48:13.142644+0000","last_fullsized":"2024-09-16T20:48:13.142644+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:06:11.577091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001025172,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.9","version":"31'18","reported_seq":334,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-16T20:48:16.153616+0000","last_change":"2024-09-16T20:48:09.570001+0000","last_active":"2024-09-16T20:48:16.153616+0000","last_peered":"2024-09-16T20:48:16.153616+0000","last_clean":"2024-09-16T20:48:16.153616+0000","last_became_active":"2024-09-16T20:48:07.558855+0000","last_became_peered":"2024-09-16T20:48:07.558855+0000","last_unstale":"2024-09-16T20:48:16.153616+0000","last_undegraded":"2024-09-16T20:48:16.153616+0000","last_fullsized":"2024-09-16T20:48:16.153616+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:57:04.611899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044614999999999998,"stat_sum":{"num_bytes":1572864,"num_objects":2,"num_object_clones":1,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":292,"num_read_kb":3242,"num_write":18,"num_write_kb":152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.a","version":"31'12","reported_seq":64,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-16T20:48:16.218705+0000","last_change":"2024-09-16T20:48:09.569992+0000","last_active":"2024-09-16T20:48:16.218705+0000","last_peered":"2024-09-16T20:48:16.218705+0000","last_clean":"2024-09-16T20:48:16.218705+0000","last_became_active":"2024-09-16T20:48:07.563972+0000","last_became_peered":"2024-09-16T20:48:07.563972+0000","last_unstale":"2024-09-16T20:48:16.218705+0000","last_undegraded":"2024-09-16T20:48:16.218705+0000","last_fullsized":"2024-09-16T20:48:16.218705+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:08:56.455810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039392600000000001,"stat_sum":{"num_bytes":15645696,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":25,"num_read_kb":94,"num_write":12,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.b","version":"31'60","reported_seq":117,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-16T20:48:16.214733+0000","last_change":"2024-09-16T20:48:09.577659+0000","last_active":"2024-09-16T20:48:16.214733+0000","last_peered":"2024-09-16T20:48:16.214733+0000","last_clean":"2024-09-16T20:48:16.214733+0000","last_became_active":"2024-09-16T20:48:07.560092+0000","last_became_peered":"2024-09-16T20:48:07.560092+0000","last_unstale":"2024-09-16T20:48:16.214733+0000","last_undegraded":"2024-09-16T20:48:16.214733+0000","last_fullsized":"2024-09-16T20:48:16.214733+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:56:44.579832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014289540000000001,"stat_sum":{"num_bytes":8388649,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":242,"num_read_kb":310,"num_write":86,"num_write_kb":153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.c","version":"31'34","reported_seq":422,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-16T20:48:15.833596+0000","last_change":"2024-09-16T20:48:09.570395+0000","last_active":"2024-09-16T20:48:15.833596+0000","last_peered":"2024-09-16T20:48:15.833596+0000","last_clean":"2024-09-16T20:48:15.833596+0000","last_became_active":"2024-09-16T20:48:07.558578+0000","last_became_peered":"2024-09-16T20:48:07.558578+0000","last_unstale":"2024-09-16T20:48:15.833596+0000","last_undegraded":"2024-09-16T20:48:15.833596+0000","last_fullsized":"2024-09-16T20:48:15.833596+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:53:38.474803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047762999999999999,"stat_sum":{"num_bytes":46,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":758,"num_read_kb":635,"num_write":22,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.d","version":"31'78","reported_seq":911,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-16T20:48:16.094295+0000","last_change":"2024-09-16T20:48:09.565667+0000","last_active":"2024-09-16T20:48:16.094295+0000","last_peered":"2024-09-16T20:48:16.094295+0000","last_clean":"2024-09-16T20:48:16.094295+0000","last_became_active":"2024-09-16T20:48:07.567327+0000","last_became_peered":"2024-09-16T20:48:07.567327+0000","last_unstale":"2024-09-16T20:48:16.094295+0000","last_undegraded":"2024-09-16T20:48:16.094295+0000","last_fullsized":"2024-09-16T20:48:16.094295+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:06:37.247869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012087560000000001,"stat_sum":{"num_bytes":1570816,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1677,"num_read_kb":3625,"num_write":45,"num_write_kb":184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.e","version":"30'11","reported_seq":220,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-16T20:48:15.977406+0000","last_change":"2024-09-16T20:48:09.565024+0000","last_active":"2024-09-16T20:48:15.977406+0000","last_peered":"2024-09-16T20:48:15.977406+0000","last_clean":"2024-09-16T20:48:15.977406+0000","last_became_active":"2024-09-16T20:48:07.570704+0000","last_became_peered":"2024-09-16T20:48:07.570704+0000","last_unstale":"2024-09-16T20:48:15.977406+0000","last_undegraded":"2024-09-16T20:48:15.977406+0000","last_fullsized":"2024-09-16T20:48:15.977406+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:16:40.971251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011026149999999999,"stat_sum":{"num_bytes":1572864,"num_objects":2,"num_object_clones":1,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":183,"num_read_kb":2830,"num_write":10,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.f","version":"31'14","reported_seq":237,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-16T20:48:16.173040+0000","last_change":"2024-09-16T20:48:09.570786+0000","last_active":"2024-09-16T20:48:16.173040+0000","last_peered":"2024-09-16T20:48:16.173040+0000","last_clean":"2024-09-16T20:48:16.173040+0000","last_became_active":"2024-09-16T20:48:07.560783+0000","last_became_peered":"2024-09-16T20:48:07.560783+0000","last_unstale":"2024-09-16T20:48:16.173040+0000","last_undegraded":"2024-09-16T20:48:16.173040+0000","last_fullsized":"2024-09-16T20:48:16.173040+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:56:13.659550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040623999999999999,"stat_sum":{"num_bytes":4773888,"num_objects":4,"num_object_clones":2,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":202,"num_read_kb":2668,"num_write":12,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.10","version":"30'18","reported_seq":1116,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-16T20:48:15.985054+0000","last_change":"2024-09-16T20:48:09.565413+0000","last_active":"2024-09-16T20:48:15.985054+0000","last_peered":"2024-09-16T20:48:15.985054+0000","last_clean":"2024-09-16T20:48:15.985054+0000","last_became_active":"2024-09-16T20:48:07.565932+0000","last_became_peered":"2024-09-16T20:48:07.565932+0000","last_unstale":"2024-09-16T20:48:15.985054+0000","last_undegraded":"2024-09-16T20:48:15.985054+0000","last_fullsized":"2024-09-16T20:48:15.985054+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:01:11.707107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115576,"stat_sum":{"num_bytes":3653632,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1074,"num_read_kb":8732,"num_write":17,"num_write_kb":135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.11","version":"31'29","reported_seq":378,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-16T20:48:16.068640+0000","last_change":"2024-09-16T20:48:09.564513+0000","last_active":"2024-09-16T20:48:16.068640+0000","last_peered":"2024-09-16T20:48:16.068640+0000","last_clean":"2024-09-16T20:48:16.068640+0000","last_became_active":"2024-09-16T20:48:07.571985+0000","last_became_peered":"2024-09-16T20:48:07.571985+0000","last_unstale":"2024-09-16T20:48:16.068640+0000","last_undegraded":"2024-09-16T20:48:16.068640+0000","last_fullsized":"2024-09-16T20:48:16.068640+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:47:09.638017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077335299999999995,"stat_sum":{"num_bytes":7806464,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":319,"num_read_kb":4325,"num_write":28,"num_write_kb":261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.12","version":"29'8","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-16T20:48:15.587304+0000","last_change":"2024-09-16T20:48:09.569080+0000","last_active":"2024-09-16T20:48:15.587304+0000","last_peered":"2024-09-16T20:48:15.587304+0000","last_clean":"2024-09-16T20:48:15.587304+0000","last_became_active":"2024-09-16T20:48:07.567597+0000","last_became_peered":"2024-09-16T20:48:07.567597+0000","last_unstale":"2024-09-16T20:48:15.587304+0000","last_undegraded":"2024-09-16T20:48:15.587304+0000","last_fullsized":"2024-09-16T20:48:15.587304+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:54:36.384514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000840955,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":8,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.13","version":"31'46","reported_seq":81,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-16T20:48:15.686128+0000","last_change":"2024-09-16T20:48:09.565356+0000","last_active":"2024-09-16T20:48:15.686128+0000","last_peered":"2024-09-16T20:48:15.686128+0000","last_clean":"2024-09-16T20:48:15.686128+0000","last_became_active":"2024-09-16T20:48:07.565717+0000","last_became_peered":"2024-09-16T20:48:07.565717+0000","last_unstale":"2024-09-16T20:48:15.686128+0000","last_undegraded":"2024-09-16T20:48:15.686128+0000","last_fullsized":"2024-09-16T20:48:15.686128+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:34:51.048428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013583239999999999,"stat_sum":{"num_bytes":36,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":213,"num_read_kb":171,"num_write":76,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":93503808,"num_objects":77,"num_object_clones":24,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":11,"num_read":10426,"num_read_kb":48999,"num_write":960,"num_write_kb":4803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3444736,"data_stored":3778725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":837,"ondisk_log_size":837,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36484,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335356,"statfs":{"total":96636764160,"available":96599404544,"internally_reserved":0,"allocated":1155072,"data_stored":921628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36788,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335052,"statfs":{"total":96636764160,"available":96599093248,"internally_reserved":0,"allocated":1478656,"data_stored":1249160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,9],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31644,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340196,"statfs":{"total":96636764160,"available":96604360704,"internally_reserved":0,"allocated":471040,"data_stored":269642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32344,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339496,"statfs":{"total":96636764160,"available":96603643904,"internally_reserved":0,"allocated":1179648,"data_stored":951194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31704,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340136,"statfs":{"total":96636764160,"available":96604299264,"internally_reserved":0,"allocated":532480,"data_stored":301651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640264,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35640,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336200,"statfs":{"total":96636764160,"available":96600268800,"internally_reserved":0,"allocated":290816,"data_stored":107673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":229376,"data_stored":202399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":458752,"data_stored":411248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1212416,"data_stored":1186901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1204224,"data_stored":1679589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":14848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":319488,"data_stored":283740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T20:48:18.431 INFO:tasks.thrashosds.thrasher:Setting 2.4 to [5, 2, 0, 1] 2024-09-16T20:48:18.431 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '2.4', '5', '2', '0', '1'] 2024-09-16T20:48:18.431 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 2.4 5 2 0 1 2024-09-16T20:48:18.500 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:18.501 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:48:18.501 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:48:18.501 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:48:18.501 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.501 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4462.0:2736 3.16 3:6bf08829:::rbd_object_map.110e1748a8d2:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T20:48:18.501 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:13.557899+0000", 2024-09-16T20:48:18.501 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.9390848790000002, 2024-09-16T20:48:18.501 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0087604799999999993, 2024-09-16T20:48:18.501 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:18.501 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:18.501 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:18.501 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:18.502 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4462", 2024-09-16T20:48:18.502 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:18.502 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2736 2024-09-16T20:48:18.502 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.502 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:18.502 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.502 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:18.502 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.557899+0000", 2024-09-16T20:48:18.502 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:18.502 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.502 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.502 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:18.502 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.557899+0000", 2024-09-16T20:48:18.503 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:18.503 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.503 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.503 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:18.503 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.557899+0000", 2024-09-16T20:48:18.503 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1900000000000003e-07 2024-09-16T20:48:18.503 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.503 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.503 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:18.503 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.557903+0000", 2024-09-16T20:48:18.503 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2949999999999998e-06 2024-09-16T20:48:18.503 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.503 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.503 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:18.504 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.557926+0000", 2024-09-16T20:48:18.504 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3742000000000001e-05 2024-09-16T20:48:18.504 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.504 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.504 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:18.504 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.557934+0000", 2024-09-16T20:48:18.504 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.571e-06 2024-09-16T20:48:18.504 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.504 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.504 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:18.504 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.563760+0000", 2024-09-16T20:48:18.504 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0058263819999999997 2024-09-16T20:48:18.504 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.505 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.505 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:18.505 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.563814+0000", 2024-09-16T20:48:18.505 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3693000000000002e-05 2024-09-16T20:48:18.505 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.505 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.505 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:48:18.505 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.564013+0000", 2024-09-16T20:48:18.505 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019927599999999999 2024-09-16T20:48:18.505 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.505 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.505 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:18.505 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.566557+0000", 2024-09-16T20:48:18.506 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.002543848 2024-09-16T20:48:18.506 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.506 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.506 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:18.506 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.566572+0000", 2024-09-16T20:48:18.506 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4555e-05 2024-09-16T20:48:18.506 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.506 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.506 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:18.506 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.566620+0000", 2024-09-16T20:48:18.506 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8578999999999997e-05 2024-09-16T20:48:18.506 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.506 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.506 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:18.507 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.566659+0000", 2024-09-16T20:48:18.507 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9020000000000002e-05 2024-09-16T20:48:18.507 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.507 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:18.507 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.507 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.507 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.507 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4462.0:2740 3.16 3:6bf08829:::rbd_object_map.110e1748a8d2:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T20:48:18.507 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:13.557993+0000", 2024-09-16T20:48:18.507 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.9389905519999999, 2024-09-16T20:48:18.507 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.010694718000000001, 2024-09-16T20:48:18.507 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:18.507 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:18.508 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:18.508 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:18.508 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4462", 2024-09-16T20:48:18.508 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:18.508 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2740 2024-09-16T20:48:18.508 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.508 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:18.508 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.508 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:18.508 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.557993+0000", 2024-09-16T20:48:18.508 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:18.508 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.508 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.508 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:18.509 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.557993+0000", 2024-09-16T20:48:18.509 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:18.509 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.509 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.509 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:18.509 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.557994+0000", 2024-09-16T20:48:18.509 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.018e-06 2024-09-16T20:48:18.509 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.509 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.509 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:18.509 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.557999+0000", 2024-09-16T20:48:18.509 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3329999999999996e-06 2024-09-16T20:48:18.509 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.509 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.510 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:18.510 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558010+0000", 2024-09-16T20:48:18.510 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0716999999999999e-05 2024-09-16T20:48:18.510 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.510 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.510 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:18.510 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558016+0000", 2024-09-16T20:48:18.510 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.207e-06 2024-09-16T20:48:18.510 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.510 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.510 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:18.510 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.564547+0000", 2024-09-16T20:48:18.510 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0065309549999999997 2024-09-16T20:48:18.510 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.511 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.511 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:18.511 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.564622+0000", 2024-09-16T20:48:18.511 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4894000000000005e-05 2024-09-16T20:48:18.511 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.511 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.511 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:48:18.511 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.565028+0000", 2024-09-16T20:48:18.511 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000406272 2024-09-16T20:48:18.511 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.511 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.511 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:18.511 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.566715+0000", 2024-09-16T20:48:18.511 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0016862629999999999 2024-09-16T20:48:18.512 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.512 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.512 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:18.512 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.568585+0000", 2024-09-16T20:48:18.512 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00187046 2024-09-16T20:48:18.512 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.512 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.512 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:18.512 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.568636+0000", 2024-09-16T20:48:18.512 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0469000000000001e-05 2024-09-16T20:48:18.512 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.512 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.512 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:18.513 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.568688+0000", 2024-09-16T20:48:18.513 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2129999999999997e-05 2024-09-16T20:48:18.513 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.513 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:18.513 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.513 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.513 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.513 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4462.0:2742 3.16 3:6bf08829:::rbd_object_map.110e1748a8d2:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T20:48:18.513 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:13.558413+0000", 2024-09-16T20:48:18.513 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.938570253, 2024-09-16T20:48:18.513 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.011077027, 2024-09-16T20:48:18.513 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:18.513 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:18.513 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:18.514 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:18.514 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4462", 2024-09-16T20:48:18.514 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:18.514 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2742 2024-09-16T20:48:18.514 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.514 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:18.514 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.514 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:18.514 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558413+0000", 2024-09-16T20:48:18.514 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:18.514 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.514 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.514 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:18.514 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558413+0000", 2024-09-16T20:48:18.515 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:18.515 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.515 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.515 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:18.515 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558414+0000", 2024-09-16T20:48:18.515 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8299999999999996e-07 2024-09-16T20:48:18.515 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.515 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.515 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:18.515 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558419+0000", 2024-09-16T20:48:18.515 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7509999999999999e-06 2024-09-16T20:48:18.515 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.515 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.516 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:18.516 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558430+0000", 2024-09-16T20:48:18.516 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0746e-05 2024-09-16T20:48:18.516 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.516 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.516 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:18.516 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558437+0000", 2024-09-16T20:48:18.516 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3950000000000003e-06 2024-09-16T20:48:18.516 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.516 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.516 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:18.516 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.565635+0000", 2024-09-16T20:48:18.517 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0071981950000000001 2024-09-16T20:48:18.517 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.517 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.517 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:18.517 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.565720+0000", 2024-09-16T20:48:18.517 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4326999999999994e-05 2024-09-16T20:48:18.517 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.517 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.517 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:48:18.517 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.566105+0000", 2024-09-16T20:48:18.517 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00038549500000000002 2024-09-16T20:48:18.517 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.518 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.518 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:18.518 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.568571+0000", 2024-09-16T20:48:18.518 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0024662909999999998 2024-09-16T20:48:18.518 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.518 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.518 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:18.518 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.569396+0000", 2024-09-16T20:48:18.518 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00082436 2024-09-16T20:48:18.518 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.518 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.518 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:18.518 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.569449+0000", 2024-09-16T20:48:18.518 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3603999999999997e-05 2024-09-16T20:48:18.519 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.519 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.519 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:18.519 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.569490+0000", 2024-09-16T20:48:18.519 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.108e-05 2024-09-16T20:48:18.519 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.519 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:18.519 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.519 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.519 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.519 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4462.0:2743 3.16 3:6bf08829:::rbd_object_map.110e1748a8d2:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T20:48:18.519 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:13.558456+0000", 2024-09-16T20:48:18.520 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.9385272799999997, 2024-09-16T20:48:18.520 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.011122093, 2024-09-16T20:48:18.520 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:18.520 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:18.520 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:18.520 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:18.520 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4462", 2024-09-16T20:48:18.520 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:18.520 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2743 2024-09-16T20:48:18.520 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.520 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:18.520 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.520 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:18.520 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558456+0000", 2024-09-16T20:48:18.521 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:18.521 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.521 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.521 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:18.521 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558456+0000", 2024-09-16T20:48:18.521 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:18.521 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.521 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.521 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:18.521 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558457+0000", 2024-09-16T20:48:18.521 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9999999999999997e-07 2024-09-16T20:48:18.521 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.521 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.522 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:18.522 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558460+0000", 2024-09-16T20:48:18.522 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4379999999999999e-06 2024-09-16T20:48:18.522 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.522 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.522 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:18.522 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558468+0000", 2024-09-16T20:48:18.522 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9109999999999993e-06 2024-09-16T20:48:18.522 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.522 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.522 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:18.522 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558477+0000", 2024-09-16T20:48:18.522 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2479999999999996e-06 2024-09-16T20:48:18.522 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.523 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.523 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:18.523 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.566757+0000", 2024-09-16T20:48:18.523 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0082806519999999995 2024-09-16T20:48:18.523 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.523 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.523 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:18.523 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.566834+0000", 2024-09-16T20:48:18.523 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6939000000000003e-05 2024-09-16T20:48:18.523 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.523 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.523 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:48:18.523 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.567214+0000", 2024-09-16T20:48:18.523 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000379744 2024-09-16T20:48:18.524 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.524 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.524 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:18.524 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.568757+0000", 2024-09-16T20:48:18.524 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001543256 2024-09-16T20:48:18.524 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.524 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.524 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:18.524 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.569501+0000", 2024-09-16T20:48:18.524 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00074416700000000001 2024-09-16T20:48:18.524 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.524 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.524 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:18.524 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.569552+0000", 2024-09-16T20:48:18.525 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0779999999999998e-05 2024-09-16T20:48:18.525 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.525 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.525 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:18.525 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.569578+0000", 2024-09-16T20:48:18.525 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6358000000000001e-05 2024-09-16T20:48:18.525 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.525 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:18.525 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.525 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.525 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.525 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4462.0:2744 3.16 3:6bf08829:::rbd_object_map.110e1748a8d2:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T20:48:18.525 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:13.558492+0000", 2024-09-16T20:48:18.525 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.9384914389999999, 2024-09-16T20:48:18.526 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.013743128, 2024-09-16T20:48:18.526 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:18.526 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:18.526 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:18.526 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:18.526 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4462", 2024-09-16T20:48:18.526 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:18.526 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2744 2024-09-16T20:48:18.526 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.526 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:18.526 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.526 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:18.526 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558492+0000", 2024-09-16T20:48:18.526 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:18.527 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.527 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.527 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:18.527 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558492+0000", 2024-09-16T20:48:18.527 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:18.527 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.527 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.527 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:18.527 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558493+0000", 2024-09-16T20:48:18.527 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5799999999999999e-07 2024-09-16T20:48:18.527 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.527 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.527 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:18.528 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558496+0000", 2024-09-16T20:48:18.528 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2430000000000001e-06 2024-09-16T20:48:18.528 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.528 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.528 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:18.528 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558504+0000", 2024-09-16T20:48:18.528 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8299999999999996e-06 2024-09-16T20:48:18.528 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.528 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.528 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:18.528 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558510+0000", 2024-09-16T20:48:18.528 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8590000000000001e-06 2024-09-16T20:48:18.528 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.529 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.529 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:18.529 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.567594+0000", 2024-09-16T20:48:18.529 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0090840639999999993 2024-09-16T20:48:18.529 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.529 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.529 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:18.529 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.567705+0000", 2024-09-16T20:48:18.529 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000111499 2024-09-16T20:48:18.529 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.529 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.529 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:48:18.529 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.568132+0000", 2024-09-16T20:48:18.530 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00042723700000000002 2024-09-16T20:48:18.530 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.530 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.530 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:18.530 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.570575+0000", 2024-09-16T20:48:18.530 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0024422440000000001 2024-09-16T20:48:18.530 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.530 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.530 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:18.530 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.572164+0000", 2024-09-16T20:48:18.530 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0015896420000000001 2024-09-16T20:48:18.530 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.530 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.531 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:18.531 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.572191+0000", 2024-09-16T20:48:18.531 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6486999999999999e-05 2024-09-16T20:48:18.531 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.531 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.531 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:18.531 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.572235+0000", 2024-09-16T20:48:18.531 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4465000000000002e-05 2024-09-16T20:48:18.531 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.531 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:18.531 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.531 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.531 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.532 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4462.0:2745 3.16 3:6bf08829:::rbd_object_map.110e1748a8d2:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T20:48:18.532 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:13.558520+0000", 2024-09-16T20:48:18.532 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.9384631380000004, 2024-09-16T20:48:18.532 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.013849663999999999, 2024-09-16T20:48:18.532 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:18.532 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:18.532 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:18.532 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:18.532 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4462", 2024-09-16T20:48:18.532 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:18.532 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2745 2024-09-16T20:48:18.532 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.532 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:18.533 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.533 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:18.533 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558520+0000", 2024-09-16T20:48:18.533 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:18.533 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.533 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.533 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:18.533 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558520+0000", 2024-09-16T20:48:18.533 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:18.533 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.533 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.534 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:18.534 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558521+0000", 2024-09-16T20:48:18.534 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1300000000000001e-07 2024-09-16T20:48:18.534 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.534 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.534 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:18.534 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558524+0000", 2024-09-16T20:48:18.534 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7439999999999999e-06 2024-09-16T20:48:18.534 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.534 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.534 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:18.534 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558531+0000", 2024-09-16T20:48:18.534 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3889999999999999e-06 2024-09-16T20:48:18.535 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.535 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.535 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:18.535 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558536+0000", 2024-09-16T20:48:18.535 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3210000000000004e-06 2024-09-16T20:48:18.535 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.535 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.535 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:18.535 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.568806+0000", 2024-09-16T20:48:18.535 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.010269789 2024-09-16T20:48:18.535 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.535 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.535 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:18.536 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.568859+0000", 2024-09-16T20:48:18.536 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3028000000000001e-05 2024-09-16T20:48:18.536 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.536 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.536 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:48:18.536 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.569051+0000", 2024-09-16T20:48:18.536 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019187800000000001 2024-09-16T20:48:18.536 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.536 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.536 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:18.536 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.570633+0000", 2024-09-16T20:48:18.536 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0015821800000000001 2024-09-16T20:48:18.536 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.536 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.537 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:18.537 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.572307+0000", 2024-09-16T20:48:18.537 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00167336 2024-09-16T20:48:18.537 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.537 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.537 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:18.537 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.572331+0000", 2024-09-16T20:48:18.537 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.4561000000000001e-05 2024-09-16T20:48:18.537 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.537 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.537 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:18.537 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.572370+0000", 2024-09-16T20:48:18.537 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9001000000000001e-05 2024-09-16T20:48:18.538 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.538 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:18.538 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.538 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.538 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.538 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4462.0:2746 3.16 3:6bf08829:::rbd_object_map.110e1748a8d2:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T20:48:18.538 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:13.558543+0000", 2024-09-16T20:48:18.538 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.9384401709999999, 2024-09-16T20:48:18.538 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.013910726, 2024-09-16T20:48:18.538 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:18.538 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:18.538 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:18.538 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:18.538 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4462", 2024-09-16T20:48:18.539 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:18.539 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2746 2024-09-16T20:48:18.539 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.539 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:18.539 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.539 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:18.539 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558543+0000", 2024-09-16T20:48:18.539 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:18.539 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.539 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.539 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:18.539 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558543+0000", 2024-09-16T20:48:18.539 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:18.539 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.540 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.540 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:18.540 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558544+0000", 2024-09-16T20:48:18.540 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.77e-07 2024-09-16T20:48:18.540 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.540 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.540 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:18.540 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558546+0000", 2024-09-16T20:48:18.540 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6460000000000002e-06 2024-09-16T20:48:18.540 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.540 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.540 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:18.541 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558554+0000", 2024-09-16T20:48:18.541 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.114e-06 2024-09-16T20:48:18.541 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.541 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.541 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:18.541 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558559+0000", 2024-09-16T20:48:18.541 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3249999999999998e-06 2024-09-16T20:48:18.541 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.541 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.541 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:18.541 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.569645+0000", 2024-09-16T20:48:18.541 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.01108639 2024-09-16T20:48:18.541 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.541 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.542 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:18.542 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.569740+0000", 2024-09-16T20:48:18.542 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.5021999999999999e-05 2024-09-16T20:48:18.542 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.542 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.542 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:48:18.542 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.570140+0000", 2024-09-16T20:48:18.542 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00039964500000000001 2024-09-16T20:48:18.542 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.542 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.542 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:18.542 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.572272+0000", 2024-09-16T20:48:18.542 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0021316550000000001 2024-09-16T20:48:18.542 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.543 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.543 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:18.543 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.572377+0000", 2024-09-16T20:48:18.543 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010546400000000001 2024-09-16T20:48:18.543 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.543 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.543 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:18.543 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.572412+0000", 2024-09-16T20:48:18.543 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4715999999999998e-05 2024-09-16T20:48:18.543 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.543 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.543 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:18.543 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.572454+0000", 2024-09-16T20:48:18.544 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2372e-05 2024-09-16T20:48:18.544 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.544 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:18.544 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.544 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.544 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.544 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4462.0:2747 3.16 3:6bf08829:::rbd_object_map.110e1748a8d2:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T20:48:18.544 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:13.558566+0000", 2024-09-16T20:48:18.544 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.9384178350000001, 2024-09-16T20:48:18.544 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.013998029, 2024-09-16T20:48:18.544 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:18.544 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:18.544 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:18.545 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:18.545 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4462", 2024-09-16T20:48:18.545 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:18.545 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2747 2024-09-16T20:48:18.545 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.545 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:18.545 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.545 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:18.545 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558566+0000", 2024-09-16T20:48:18.545 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:18.545 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.545 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.545 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:18.545 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558566+0000", 2024-09-16T20:48:18.546 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:18.546 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.546 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.546 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:18.546 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558566+0000", 2024-09-16T20:48:18.546 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8000000000000001e-07 2024-09-16T20:48:18.546 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.546 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.546 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:18.546 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558569+0000", 2024-09-16T20:48:18.546 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5019999999999999e-06 2024-09-16T20:48:18.546 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.546 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.546 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:18.547 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558575+0000", 2024-09-16T20:48:18.547 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6800000000000004e-06 2024-09-16T20:48:18.547 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.547 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.547 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:18.547 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558580+0000", 2024-09-16T20:48:18.547 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0000000000000004e-06 2024-09-16T20:48:18.547 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.547 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.547 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:18.547 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.570668+0000", 2024-09-16T20:48:18.547 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.01208735 2024-09-16T20:48:18.547 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.547 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.548 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:18.548 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.570769+0000", 2024-09-16T20:48:18.548 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010153300000000001 2024-09-16T20:48:18.548 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.548 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.548 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:48:18.548 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.571149+0000", 2024-09-16T20:48:18.548 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000380205 2024-09-16T20:48:18.548 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.548 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.548 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:18.548 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.572300+0000", 2024-09-16T20:48:18.548 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0011501409999999999 2024-09-16T20:48:18.549 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.549 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.549 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:18.549 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.572463+0000", 2024-09-16T20:48:18.549 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00016354499999999999 2024-09-16T20:48:18.549 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.549 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.549 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:18.549 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.572522+0000", 2024-09-16T20:48:18.549 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8653999999999998e-05 2024-09-16T20:48:18.549 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.549 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.549 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:18.550 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.572564+0000", 2024-09-16T20:48:18.550 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2039000000000002e-05 2024-09-16T20:48:18.550 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.550 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:18.550 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.550 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.550 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.550 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4462.0:2752 3.16 3:6bf08829:::rbd_object_map.110e1748a8d2:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T20:48:18.550 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:13.558869+0000", 2024-09-16T20:48:18.550 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.9381146290000002, 2024-09-16T20:48:18.550 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.015539270000000001, 2024-09-16T20:48:18.550 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:18.550 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:18.550 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:18.551 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:18.551 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4462", 2024-09-16T20:48:18.551 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:18.551 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2752 2024-09-16T20:48:18.551 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.551 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:18.551 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.551 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:18.551 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558869+0000", 2024-09-16T20:48:18.551 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:18.551 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.551 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.551 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:18.551 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558869+0000", 2024-09-16T20:48:18.552 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:18.552 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.552 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.552 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:18.552 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558870+0000", 2024-09-16T20:48:18.552 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5199999999999995e-07 2024-09-16T20:48:18.552 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.552 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.552 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:18.552 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558875+0000", 2024-09-16T20:48:18.552 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9099999999999996e-06 2024-09-16T20:48:18.552 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.552 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.553 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:18.553 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558888+0000", 2024-09-16T20:48:18.553 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3081000000000001e-05 2024-09-16T20:48:18.553 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.553 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.553 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:18.553 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558902+0000", 2024-09-16T20:48:18.553 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4406000000000001e-05 2024-09-16T20:48:18.553 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.553 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.553 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:18.553 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.571454+0000", 2024-09-16T20:48:18.553 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.012551866 2024-09-16T20:48:18.553 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.554 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.554 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:18.554 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.571516+0000", 2024-09-16T20:48:18.554 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1643999999999995e-05 2024-09-16T20:48:18.554 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.554 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.554 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:48:18.554 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.571874+0000", 2024-09-16T20:48:18.554 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00035782400000000002 2024-09-16T20:48:18.554 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.554 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.554 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:18.554 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.573409+0000", 2024-09-16T20:48:18.555 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001535762 2024-09-16T20:48:18.555 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.555 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.555 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:18.555 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.574304+0000", 2024-09-16T20:48:18.555 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00089457000000000002 2024-09-16T20:48:18.555 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.555 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.555 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:18.555 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.574363+0000", 2024-09-16T20:48:18.555 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9512000000000002e-05 2024-09-16T20:48:18.555 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.555 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.555 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:18.556 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.574408+0000", 2024-09-16T20:48:18.556 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4842999999999997e-05 2024-09-16T20:48:18.556 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.556 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:18.556 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.556 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.556 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.556 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4462.0:2753 3.16 3:6bf08829:::rbd_object_map.110e1748a8d2:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T20:48:18.556 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:13.558938+0000", 2024-09-16T20:48:18.556 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.9380455869999995, 2024-09-16T20:48:18.556 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.016280317999999998, 2024-09-16T20:48:18.556 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:18.556 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:18.557 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:18.557 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:18.557 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4462", 2024-09-16T20:48:18.557 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:18.557 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2753 2024-09-16T20:48:18.557 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.557 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:18.557 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.557 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:18.557 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558938+0000", 2024-09-16T20:48:18.557 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:18.557 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.557 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.558 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:18.558 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558938+0000", 2024-09-16T20:48:18.558 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:18.558 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.558 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.558 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:18.558 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558939+0000", 2024-09-16T20:48:18.558 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3600000000000003e-07 2024-09-16T20:48:18.558 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.558 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.558 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:18.558 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558943+0000", 2024-09-16T20:48:18.558 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1010000000000002e-06 2024-09-16T20:48:18.558 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.559 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.559 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:18.559 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558951+0000", 2024-09-16T20:48:18.559 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.3010000000000007e-06 2024-09-16T20:48:18.559 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.559 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.559 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:18.559 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558957+0000", 2024-09-16T20:48:18.559 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6010000000000002e-06 2024-09-16T20:48:18.559 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.559 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.559 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:18.559 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.572596+0000", 2024-09-16T20:48:18.560 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.013639607 2024-09-16T20:48:18.560 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.560 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.560 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:18.560 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.572654+0000", 2024-09-16T20:48:18.560 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7414000000000003e-05 2024-09-16T20:48:18.560 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.560 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.560 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:48:18.560 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.573062+0000", 2024-09-16T20:48:18.560 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00040791200000000002 2024-09-16T20:48:18.560 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.560 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.560 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:18.561 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.574472+0000", 2024-09-16T20:48:18.561 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0014106570000000001 2024-09-16T20:48:18.561 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.561 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.561 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:18.561 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.575120+0000", 2024-09-16T20:48:18.561 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00064772099999999997 2024-09-16T20:48:18.561 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.561 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.561 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:18.561 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.575177+0000", 2024-09-16T20:48:18.561 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7370999999999997e-05 2024-09-16T20:48:18.561 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.562 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.562 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:18.562 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.575218+0000", 2024-09-16T20:48:18.562 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0997000000000001e-05 2024-09-16T20:48:18.562 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.562 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:18.562 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.562 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.562 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.562 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4462.0:2754 3.16 3:6bf08829:::rbd_object_map.110e1748a8d2:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T20:48:18.562 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:13.558974+0000", 2024-09-16T20:48:18.562 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.9380098910000001, 2024-09-16T20:48:18.562 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.017452303999999998, 2024-09-16T20:48:18.563 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:18.563 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:18.563 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:18.563 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:18.563 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4462", 2024-09-16T20:48:18.563 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:18.563 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2754 2024-09-16T20:48:18.563 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.563 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:18.563 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.563 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:18.563 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558974+0000", 2024-09-16T20:48:18.563 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:18.563 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.564 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.564 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:18.564 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558974+0000", 2024-09-16T20:48:18.564 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:18.564 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.564 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.564 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:18.564 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558974+0000", 2024-09-16T20:48:18.564 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4600000000000004e-07 2024-09-16T20:48:18.564 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.564 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.564 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:18.565 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558978+0000", 2024-09-16T20:48:18.565 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5880000000000002e-06 2024-09-16T20:48:18.565 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.565 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.565 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:18.565 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558986+0000", 2024-09-16T20:48:18.565 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6159999999999996e-06 2024-09-16T20:48:18.565 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.565 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.565 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:18.565 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.558992+0000", 2024-09-16T20:48:18.565 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.584e-06 2024-09-16T20:48:18.565 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.565 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.566 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:18.566 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.573457+0000", 2024-09-16T20:48:18.566 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.014465164000000001 2024-09-16T20:48:18.566 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.566 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.566 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:18.566 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.573522+0000", 2024-09-16T20:48:18.566 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.444e-05 2024-09-16T20:48:18.566 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.566 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.566 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:48:18.566 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.573854+0000", 2024-09-16T20:48:18.566 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00033261500000000003 2024-09-16T20:48:18.566 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.567 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.567 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:18.567 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.576290+0000", 2024-09-16T20:48:18.567 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0024358140000000001 2024-09-16T20:48:18.567 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.567 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.567 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:18.567 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.576333+0000", 2024-09-16T20:48:18.567 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3058e-05 2024-09-16T20:48:18.567 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.567 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.567 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:18.567 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.576387+0000", 2024-09-16T20:48:18.567 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3764000000000002e-05 2024-09-16T20:48:18.568 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.568 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.568 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:18.568 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.576426+0000", 2024-09-16T20:48:18.568 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9014999999999998e-05 2024-09-16T20:48:18.568 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.568 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:18.568 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.568 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.568 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.568 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4462.0:2755 3.16 3:6bf08829:::rbd_object_map.110e1748a8d2:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T20:48:18.568 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:13.559003+0000", 2024-09-16T20:48:18.568 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.9379806989999997, 2024-09-16T20:48:18.569 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.017638179, 2024-09-16T20:48:18.569 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:18.569 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:18.569 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:18.569 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:18.569 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4462", 2024-09-16T20:48:18.569 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:18.569 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2755 2024-09-16T20:48:18.569 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.569 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:18.569 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.569 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:18.569 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.559003+0000", 2024-09-16T20:48:18.570 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:18.570 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.570 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.570 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:18.570 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.559003+0000", 2024-09-16T20:48:18.570 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:18.570 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.570 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.570 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:18.570 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.559003+0000", 2024-09-16T20:48:18.571 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3300000000000002e-07 2024-09-16T20:48:18.571 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.571 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.571 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:18.571 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.559007+0000", 2024-09-16T20:48:18.571 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1750000000000001e-06 2024-09-16T20:48:18.571 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.571 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.571 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:18.571 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.559013+0000", 2024-09-16T20:48:18.571 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4049999999999997e-06 2024-09-16T20:48:18.571 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.571 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.571 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:18.572 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.559018+0000", 2024-09-16T20:48:18.572 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0900000000000004e-06 2024-09-16T20:48:18.572 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.572 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.572 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:18.572 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.574498+0000", 2024-09-16T20:48:18.572 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.015479461999999999 2024-09-16T20:48:18.572 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.572 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.572 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:18.572 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.574544+0000", 2024-09-16T20:48:18.572 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6699999999999997e-05 2024-09-16T20:48:18.572 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.573 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.573 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:48:18.573 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.574746+0000", 2024-09-16T20:48:18.573 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000201893 2024-09-16T20:48:18.573 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.573 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.573 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:18.573 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.576440+0000", 2024-09-16T20:48:18.573 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0016939940000000001 2024-09-16T20:48:18.573 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.573 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.573 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:18.573 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.576554+0000", 2024-09-16T20:48:18.574 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000113789 2024-09-16T20:48:18.574 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.574 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.574 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:18.574 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.576600+0000", 2024-09-16T20:48:18.574 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5957000000000001e-05 2024-09-16T20:48:18.574 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.574 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.574 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:18.574 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.576641+0000", 2024-09-16T20:48:18.574 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1180999999999998e-05 2024-09-16T20:48:18.574 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.574 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:18.574 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.575 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.575 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.575 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4462.0:2756 3.16 3:6bf08829:::rbd_object_map.110e1748a8d2:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T20:48:18.575 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:13.559032+0000", 2024-09-16T20:48:18.575 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.9379515280000001, 2024-09-16T20:48:18.575 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.019715557000000002, 2024-09-16T20:48:18.575 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:18.575 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:18.575 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:18.575 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:18.575 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4462", 2024-09-16T20:48:18.575 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:18.575 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2756 2024-09-16T20:48:18.576 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.576 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:18.576 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.576 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:18.576 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.559032+0000", 2024-09-16T20:48:18.576 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:18.576 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.576 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.576 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:18.576 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.559032+0000", 2024-09-16T20:48:18.576 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:18.576 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.576 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.577 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:18.577 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.559033+0000", 2024-09-16T20:48:18.577 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8899999999999999e-07 2024-09-16T20:48:18.577 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.577 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.577 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:18.577 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.559036+0000", 2024-09-16T20:48:18.577 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.168e-06 2024-09-16T20:48:18.577 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.577 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.577 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:18.577 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.559044+0000", 2024-09-16T20:48:18.577 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1450000000000008e-06 2024-09-16T20:48:18.578 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.578 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.578 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:18.578 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.559049+0000", 2024-09-16T20:48:18.578 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4090000000000002e-06 2024-09-16T20:48:18.578 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.578 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.578 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:18.578 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.575259+0000", 2024-09-16T20:48:18.578 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.016209586000000002 2024-09-16T20:48:18.578 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.578 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.578 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:18.579 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.575344+0000", 2024-09-16T20:48:18.579 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5066000000000002e-05 2024-09-16T20:48:18.579 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.579 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.579 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:48:18.579 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.575862+0000", 2024-09-16T20:48:18.579 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00051812899999999999 2024-09-16T20:48:18.579 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.579 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.579 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:18.579 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.577923+0000", 2024-09-16T20:48:18.579 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0020603919999999999 2024-09-16T20:48:18.579 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.580 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.580 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:18.580 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.578613+0000", 2024-09-16T20:48:18.580 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00069047800000000001 2024-09-16T20:48:18.580 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.580 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.580 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:18.580 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.578703+0000", 2024-09-16T20:48:18.580 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9876000000000005e-05 2024-09-16T20:48:18.580 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.580 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.580 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:18.580 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.578748+0000", 2024-09-16T20:48:18.581 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4719000000000003e-05 2024-09-16T20:48:18.581 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.581 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:18.581 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.581 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.581 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.581 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4462.0:2757 3.16 3:6bf08829:::rbd_object_map.110e1748a8d2:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T20:48:18.581 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:13.559058+0000", 2024-09-16T20:48:18.581 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.9379258769999996, 2024-09-16T20:48:18.581 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.022293453000000001, 2024-09-16T20:48:18.581 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:18.581 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:18.581 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:18.581 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:18.582 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4462", 2024-09-16T20:48:18.582 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:18.582 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2757 2024-09-16T20:48:18.582 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.582 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:18.582 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.582 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:18.582 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.559058+0000", 2024-09-16T20:48:18.582 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:18.582 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.582 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.582 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:18.582 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.559058+0000", 2024-09-16T20:48:18.582 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:18.583 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.583 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.583 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:18.583 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.559058+0000", 2024-09-16T20:48:18.583 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3599999999999999e-07 2024-09-16T20:48:18.583 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.583 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.583 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:18.583 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.559061+0000", 2024-09-16T20:48:18.583 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1109999999999999e-06 2024-09-16T20:48:18.583 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.583 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.583 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:18.583 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.559068+0000", 2024-09-16T20:48:18.584 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2670000000000003e-06 2024-09-16T20:48:18.584 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.584 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.584 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:18.584 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.559073+0000", 2024-09-16T20:48:18.584 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.395e-06 2024-09-16T20:48:18.584 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.584 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.584 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:18.584 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.576686+0000", 2024-09-16T20:48:18.584 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.017613058000000001 2024-09-16T20:48:18.584 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.584 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.585 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:18.585 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.576765+0000", 2024-09-16T20:48:18.585 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9462000000000006e-05 2024-09-16T20:48:18.585 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.585 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.585 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:48:18.585 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.577157+0000", 2024-09-16T20:48:18.585 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00039132000000000001 2024-09-16T20:48:18.585 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.585 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.585 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:18.585 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.580162+0000", 2024-09-16T20:48:18.585 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0030047699999999999 2024-09-16T20:48:18.585 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.586 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.586 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:18.586 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.581257+0000", 2024-09-16T20:48:18.586 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0010951820000000001 2024-09-16T20:48:18.586 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.586 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.586 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:18.586 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.581313+0000", 2024-09-16T20:48:18.586 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6308999999999999e-05 2024-09-16T20:48:18.586 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.586 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.586 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:18.586 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.581351+0000", 2024-09-16T20:48:18.587 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8142999999999997e-05 2024-09-16T20:48:18.587 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.587 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:18.587 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.587 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.587 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.587 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4462.0:2764 3.e 3:72cb10a7:::rbd_data.110e1748a8d2.0000000000000003:head [read 12288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-16T20:48:18.587 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:13.561322+0000", 2024-09-16T20:48:18.587 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.9356611929999996, 2024-09-16T20:48:18.587 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.015597898000000001, 2024-09-16T20:48:18.587 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:18.587 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:18.587 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:18.587 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:18.588 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4462", 2024-09-16T20:48:18.588 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:18.588 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2764 2024-09-16T20:48:18.588 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.588 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:18.588 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.588 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:18.588 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.561322+0000", 2024-09-16T20:48:18.588 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:18.588 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.588 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.588 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:18.589 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.561322+0000", 2024-09-16T20:48:18.589 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:18.589 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.589 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.589 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:18.589 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.561323+0000", 2024-09-16T20:48:18.589 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8800000000000002e-07 2024-09-16T20:48:18.589 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.589 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.589 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:18.589 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.561326+0000", 2024-09-16T20:48:18.589 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1259999999999998e-06 2024-09-16T20:48:18.589 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.590 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.590 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:18.590 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.561332+0000", 2024-09-16T20:48:18.590 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2349999999999996e-06 2024-09-16T20:48:18.590 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.590 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.590 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:18.590 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.561336+0000", 2024-09-16T20:48:18.590 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9519999999999996e-06 2024-09-16T20:48:18.590 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.590 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.590 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:18.590 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.576751+0000", 2024-09-16T20:48:18.591 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.015414663 2024-09-16T20:48:18.591 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.591 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.591 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:18.591 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.576821+0000", 2024-09-16T20:48:18.591 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9925000000000006e-05 2024-09-16T20:48:18.591 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.591 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.591 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:18.591 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.576920+0000", 2024-09-16T20:48:18.591 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.9408999999999996e-05 2024-09-16T20:48:18.591 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.592 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:18.592 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.592 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.592 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.592 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4462.0:2766 3.e 3:77990de2:::rbd_data.110e1748a8d2.0000000000000022:head [set-alloc-hint object_size 2097152 write_size 2097152,write 1884160~16384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T20:48:18.592 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:13.562024+0000", 2024-09-16T20:48:18.592 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.9349593360000004, 2024-09-16T20:48:18.592 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.019411194, 2024-09-16T20:48:18.592 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:18.592 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:18.592 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:18.592 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:18.592 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4462", 2024-09-16T20:48:18.593 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:18.593 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2766 2024-09-16T20:48:18.593 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.593 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:18.593 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.593 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:18.593 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.562024+0000", 2024-09-16T20:48:18.593 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:18.593 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.593 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.593 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:18.593 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.562024+0000", 2024-09-16T20:48:18.593 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:18.594 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.594 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.594 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:18.594 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.562025+0000", 2024-09-16T20:48:18.594 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.3099999999999996e-07 2024-09-16T20:48:18.594 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.594 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.594 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:18.594 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.562040+0000", 2024-09-16T20:48:18.594 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4341e-05 2024-09-16T20:48:18.594 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.594 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.594 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:18.594 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.562051+0000", 2024-09-16T20:48:18.595 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1532000000000001e-05 2024-09-16T20:48:18.595 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.595 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.595 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:18.595 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.562058+0000", 2024-09-16T20:48:18.595 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1849999999999998e-06 2024-09-16T20:48:18.595 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.595 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.595 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:18.595 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.577834+0000", 2024-09-16T20:48:18.595 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.015776032999999998 2024-09-16T20:48:18.595 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.595 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.595 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:18.596 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.577976+0000", 2024-09-16T20:48:18.596 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000141979 2024-09-16T20:48:18.596 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.596 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.596 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:48:18.596 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.578113+0000", 2024-09-16T20:48:18.596 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013640399999999999 2024-09-16T20:48:18.596 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.596 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.596 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:18.596 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.580513+0000", 2024-09-16T20:48:18.596 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.002400367 2024-09-16T20:48:18.597 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.597 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.597 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:18.597 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.581359+0000", 2024-09-16T20:48:18.597 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00084645499999999997 2024-09-16T20:48:18.597 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.597 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.597 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:18.597 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.581402+0000", 2024-09-16T20:48:18.597 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2168e-05 2024-09-16T20:48:18.597 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.597 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.597 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:18.597 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.581435+0000", 2024-09-16T20:48:18.598 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3798999999999999e-05 2024-09-16T20:48:18.598 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.598 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:18.598 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.598 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.598 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.598 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4462.0:2778 3.16 3:6af5b17c:::rbd_data.110e1748a8d2.000000000000004c:head [set-alloc-hint object_size 2097152 write_size 2097152,write 625664~13312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T20:48:18.598 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:13.572407+0000", 2024-09-16T20:48:18.598 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.9245769729999997, 2024-09-16T20:48:18.598 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0091246399999999998, 2024-09-16T20:48:18.598 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:18.598 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:18.599 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:18.599 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:18.599 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4462", 2024-09-16T20:48:18.599 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:18.599 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2778 2024-09-16T20:48:18.599 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.599 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:18.599 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.599 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:18.599 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.572407+0000", 2024-09-16T20:48:18.599 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:18.599 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.599 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.600 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:18.600 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.572407+0000", 2024-09-16T20:48:18.600 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:18.600 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.600 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.600 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:18.600 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.572408+0000", 2024-09-16T20:48:18.600 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0780000000000001e-06 2024-09-16T20:48:18.600 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.601 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.601 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:18.601 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.572426+0000", 2024-09-16T20:48:18.601 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8719000000000002e-05 2024-09-16T20:48:18.601 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.601 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.601 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:18.601 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.572438+0000", 2024-09-16T20:48:18.601 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1908e-05 2024-09-16T20:48:18.601 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.601 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.601 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:18.601 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.572446+0000", 2024-09-16T20:48:18.601 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2919999999999998e-06 2024-09-16T20:48:18.602 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.602 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.602 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:18.602 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.577958+0000", 2024-09-16T20:48:18.602 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0055119390000000004 2024-09-16T20:48:18.602 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.602 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.602 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:18.602 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.578059+0000", 2024-09-16T20:48:18.602 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101712 2024-09-16T20:48:18.602 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.602 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.602 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:48:18.603 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.578195+0000", 2024-09-16T20:48:18.603 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000136212 2024-09-16T20:48:18.603 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.603 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.603 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:18.603 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.580215+0000", 2024-09-16T20:48:18.603 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0020197449999999999 2024-09-16T20:48:18.603 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.603 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.603 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:18.603 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.581444+0000", 2024-09-16T20:48:18.603 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0012285359999999999 2024-09-16T20:48:18.603 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.604 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.604 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:18.604 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.581487+0000", 2024-09-16T20:48:18.604 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.35e-05 2024-09-16T20:48:18.604 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.604 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.604 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:18.604 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:13.581531+0000", 2024-09-16T20:48:18.604 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3998999999999997e-05 2024-09-16T20:48:18.604 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.604 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:18.604 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.604 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.604 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.605 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4462.0:3203 3.16 3:6bf08829:::rbd_object_map.110e1748a8d2:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:18.605 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:14.662916+0000", 2024-09-16T20:48:18.605 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 3.834067734, 2024-09-16T20:48:18.605 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0099898680000000007, 2024-09-16T20:48:18.605 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:18.605 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:18.605 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:18.605 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:18.605 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4462", 2024-09-16T20:48:18.605 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:18.605 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3203 2024-09-16T20:48:18.605 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.605 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:18.606 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.606 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:18.606 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:14.662916+0000", 2024-09-16T20:48:18.606 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:18.606 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.606 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.606 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:18.606 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:14.662916+0000", 2024-09-16T20:48:18.606 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:18.606 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.606 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.606 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:18.606 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:14.662916+0000", 2024-09-16T20:48:18.606 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.28e-07 2024-09-16T20:48:18.607 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.607 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.607 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:18.607 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:14.662918+0000", 2024-09-16T20:48:18.607 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5719999999999999e-06 2024-09-16T20:48:18.607 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.607 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.607 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:18.607 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:14.662925+0000", 2024-09-16T20:48:18.607 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7489999999999998e-06 2024-09-16T20:48:18.607 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.607 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.607 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:18.608 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:14.662930+0000", 2024-09-16T20:48:18.608 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.498e-06 2024-09-16T20:48:18.608 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.608 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.608 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:18.608 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:14.669544+0000", 2024-09-16T20:48:18.608 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0066145910000000004 2024-09-16T20:48:18.608 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.608 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.608 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:18.608 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:14.669626+0000", 2024-09-16T20:48:18.608 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1366000000000007e-05 2024-09-16T20:48:18.608 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.609 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.609 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:48:18.609 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:14.669977+0000", 2024-09-16T20:48:18.609 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00035112700000000001 2024-09-16T20:48:18.609 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.609 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.609 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:18.609 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:14.672150+0000", 2024-09-16T20:48:18.609 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.002173038 2024-09-16T20:48:18.609 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.609 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.609 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:18.609 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:14.672826+0000", 2024-09-16T20:48:18.609 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00067579800000000002 2024-09-16T20:48:18.610 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.610 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.610 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:18.610 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:14.672865+0000", 2024-09-16T20:48:18.610 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9529e-05 2024-09-16T20:48:18.610 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.610 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.610 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:18.610 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:14.672906+0000", 2024-09-16T20:48:18.610 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0371999999999999e-05 2024-09-16T20:48:18.610 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.610 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:18.611 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.611 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.611 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.611 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4462.0:3204 3.16 3:6bf08829:::rbd_object_map.110e1748a8d2:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:18.611 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:14.662934+0000", 2024-09-16T20:48:18.611 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 3.834050006, 2024-09-16T20:48:18.611 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.010064468, 2024-09-16T20:48:18.611 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:18.611 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:18.611 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:18.611 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:18.611 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4462", 2024-09-16T20:48:18.611 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:18.611 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3204 2024-09-16T20:48:18.612 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.612 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:18.612 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.612 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:18.612 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:14.662934+0000", 2024-09-16T20:48:18.612 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:18.612 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.612 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.612 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:18.612 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:14.662934+0000", 2024-09-16T20:48:18.612 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:18.612 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.612 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.612 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:18.613 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:14.662934+0000", 2024-09-16T20:48:18.613 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.4600000000000001e-07 2024-09-16T20:48:18.613 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.613 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.613 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:18.613 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:14.662935+0000", 2024-09-16T20:48:18.613 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.641e-06 2024-09-16T20:48:18.613 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.613 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.613 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:18.613 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:14.662939+0000", 2024-09-16T20:48:18.613 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2329999999999998e-06 2024-09-16T20:48:18.613 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.613 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.614 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:18.614 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:14.662941+0000", 2024-09-16T20:48:18.614 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6639999999999998e-06 2024-09-16T20:48:18.614 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.614 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.614 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:18.614 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:14.670723+0000", 2024-09-16T20:48:18.614 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0077818590000000003 2024-09-16T20:48:18.614 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.614 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.614 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:18.614 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:14.670774+0000", 2024-09-16T20:48:18.614 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0569999999999999e-05 2024-09-16T20:48:18.614 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.615 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.615 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:48:18.615 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:14.671053+0000", 2024-09-16T20:48:18.615 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00027881800000000001 2024-09-16T20:48:18.615 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.615 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.615 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:18.615 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:14.672413+0000", 2024-09-16T20:48:18.615 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0013607809999999999 2024-09-16T20:48:18.615 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.615 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.615 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:18.615 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:14.672913+0000", 2024-09-16T20:48:18.615 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00049993599999999998 2024-09-16T20:48:18.616 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.616 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.616 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:18.616 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:14.672960+0000", 2024-09-16T20:48:18.616 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6292000000000003e-05 2024-09-16T20:48:18.616 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.616 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.616 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:18.616 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:14.672998+0000", 2024-09-16T20:48:18.616 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8427999999999999e-05 2024-09-16T20:48:18.616 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.616 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:18.616 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.617 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.617 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.617 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4462.0:3205 3.16 3:6bf08829:::rbd_object_map.110e1748a8d2:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:18.617 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:14.662944+0000", 2024-09-16T20:48:18.617 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 3.834039185, 2024-09-16T20:48:18.617 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.010144625000000001, 2024-09-16T20:48:18.617 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:18.617 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:18.617 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:18.617 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:18.617 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4462", 2024-09-16T20:48:18.617 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2874010473", 2024-09-16T20:48:18.617 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3205 2024-09-16T20:48:18.617 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.618 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:18.618 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.618 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:18.618 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:14.662944+0000", 2024-09-16T20:48:18.618 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:18.618 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.618 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.618 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:18.618 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:14.662944+0000", 2024-09-16T20:48:18.618 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:18.618 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.618 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.618 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:18.618 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:14.662945+0000", 2024-09-16T20:48:18.619 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.4600000000000001e-07 2024-09-16T20:48:18.619 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.619 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.619 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:18.619 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:14.662946+0000", 2024-09-16T20:48:18.619 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5400000000000001e-06 2024-09-16T20:48:18.619 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.619 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.619 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:18.619 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:14.662949+0000", 2024-09-16T20:48:18.619 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.892e-06 2024-09-16T20:48:18.619 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.619 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.619 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:18.620 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:14.662953+0000", 2024-09-16T20:48:18.620 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.019e-06 2024-09-16T20:48:18.620 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.620 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.620 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:18.620 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:14.671475+0000", 2024-09-16T20:48:18.620 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0085216879999999995 2024-09-16T20:48:18.620 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.620 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.620 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:18.620 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:14.671527+0000", 2024-09-16T20:48:18.620 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1743000000000002e-05 2024-09-16T20:48:18.620 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.620 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.621 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:48:18.621 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:14.671758+0000", 2024-09-16T20:48:18.621 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00023128100000000001 2024-09-16T20:48:18.621 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.621 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.621 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:18.621 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:14.672750+0000", 2024-09-16T20:48:18.621 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00099249400000000002 2024-09-16T20:48:18.621 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.621 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.621 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:18.621 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:14.673008+0000", 2024-09-16T20:48:18.621 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00025771399999999998 2024-09-16T20:48:18.621 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.622 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.622 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:18.622 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:14.673051+0000", 2024-09-16T20:48:18.622 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.286e-05 2024-09-16T20:48:18.622 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:18.622 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:18.622 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:18.622 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:14.673089+0000", 2024-09-16T20:48:18.622 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8148000000000001e-05 2024-09-16T20:48:18.622 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.622 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:18.622 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.622 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:18.623 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:18.623 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:18.623 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:18.498+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:18.624 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:48:18.624 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T20:48:18.665 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:18.664+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:18.671 INFO:teuthology.orchestra.run.smithi139.stdout:truncating image image_client.0-clone4 from 0xe886000 (overlap 0x986e00) to 0x2dba00 2024-09-16T20:48:18.725 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:18.724+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:18.744 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:18.745 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:48:18.745 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:18.745 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4478.0:2174 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:head [read 4096~4096 out=4096b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-16T20:48:18.745 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:18.743979+0000", 2024-09-16T20:48:18.745 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.00019560199999999999, 2024-09-16T20:48:18.745 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00021459499999999999, 2024-09-16T20:48:18.745 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:18.745 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:18.745 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:18.745 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:18.745 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4478", 2024-09-16T20:48:18.745 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/947429854", 2024-09-16T20:48:18.745 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 2174 2024-09-16T20:48:18.745 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:18.746 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:18.746 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:18.746 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:18.746 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.743979+0000", 2024-09-16T20:48:18.746 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:18.746 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:18.746 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:18.746 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:18.746 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.743979+0000", 2024-09-16T20:48:18.746 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:18.746 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:18.746 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:18.746 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:18.746 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.743980+0000", 2024-09-16T20:48:18.747 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2299999999999999e-07 2024-09-16T20:48:18.747 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:18.747 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:18.747 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:18.747 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.743984+0000", 2024-09-16T20:48:18.747 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5510000000000001e-06 2024-09-16T20:48:18.747 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:18.747 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:18.747 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:18.747 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.743999+0000", 2024-09-16T20:48:18.747 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4724e-05 2024-09-16T20:48:18.747 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:18.747 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:18.747 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:18.748 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.744008+0000", 2024-09-16T20:48:18.748 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3419999999999997e-06 2024-09-16T20:48:18.748 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:18.748 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:18.748 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:18.748 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.744058+0000", 2024-09-16T20:48:18.748 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9296999999999999e-05 2024-09-16T20:48:18.748 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:18.748 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:18.748 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:18.748 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.744134+0000", 2024-09-16T20:48:18.748 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6650999999999995e-05 2024-09-16T20:48:18.748 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:18.748 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:18.748 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:18.749 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:18.749 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-16T20:48:18.749 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 1 2024-09-16T20:48:18.749 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:18.753 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:48:18.839 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:18.838+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:18.904 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:18.904 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:48:18.904 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:48:18.904 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:48:18.904 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:18.913 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:48:18.928 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:18.926+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:18.989 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #3, image image_client.0-clone3 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent4 2024-09-16T20:48:19.029 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:19.028+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:19.068 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:19.069 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:48:19.069 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:48:19.069 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:48:19.069 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.069 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:863 3.1e 3:7b30dea9:::rbd_data.110e1748a8d2.0000000000000002:4 [read 851968~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:19.069 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.246541+0000", 2024-09-16T20:48:19.069 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.8201651769999998, 2024-09-16T20:48:19.069 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.009094375, 2024-09-16T20:48:19.069 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:19.069 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:19.069 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:19.069 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:19.070 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:19.070 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:19.070 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 863 2024-09-16T20:48:19.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.070 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:19.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.071 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:19.071 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246541+0000", 2024-09-16T20:48:19.071 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.071 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.071 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:19.071 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246541+0000", 2024-09-16T20:48:19.071 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.071 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.071 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:19.071 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246541+0000", 2024-09-16T20:48:19.071 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1199999999999998e-07 2024-09-16T20:48:19.072 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.072 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.072 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:19.072 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246543+0000", 2024-09-16T20:48:19.072 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1490000000000001e-06 2024-09-16T20:48:19.072 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.072 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.072 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:19.072 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246550+0000", 2024-09-16T20:48:19.072 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4320000000000004e-06 2024-09-16T20:48:19.072 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.072 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.072 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:19.073 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246555+0000", 2024-09-16T20:48:19.073 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2000000000000002e-06 2024-09-16T20:48:19.073 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.073 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.073 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:19.073 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.255463+0000", 2024-09-16T20:48:19.073 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0089078219999999993 2024-09-16T20:48:19.073 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.073 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.073 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:19.073 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.255541+0000", 2024-09-16T20:48:19.073 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8252999999999998e-05 2024-09-16T20:48:19.073 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.073 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.074 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:19.074 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.255635+0000", 2024-09-16T20:48:19.074 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4106999999999999e-05 2024-09-16T20:48:19.074 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.074 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:19.074 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.074 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.074 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:864 3.1e 3:7b30dea9:::rbd_data.110e1748a8d2.0000000000000002:4 [read 966656~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:19.074 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.246561+0000", 2024-09-16T20:48:19.074 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.8201445110000001, 2024-09-16T20:48:19.074 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0092998040000000001, 2024-09-16T20:48:19.074 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:19.074 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:19.075 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:19.075 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:19.075 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:19.075 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:19.075 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 864 2024-09-16T20:48:19.075 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.075 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:19.075 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.075 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:19.075 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246561+0000", 2024-09-16T20:48:19.075 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.075 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.075 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.075 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:19.076 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246561+0000", 2024-09-16T20:48:19.076 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.076 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.076 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.076 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:19.076 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246562+0000", 2024-09-16T20:48:19.076 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6600000000000002e-07 2024-09-16T20:48:19.076 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.076 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.076 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:19.076 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246564+0000", 2024-09-16T20:48:19.076 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7439999999999999e-06 2024-09-16T20:48:19.076 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.077 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.077 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:19.077 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246569+0000", 2024-09-16T20:48:19.077 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8130000000000001e-06 2024-09-16T20:48:19.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.077 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.077 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:19.077 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246574+0000", 2024-09-16T20:48:19.077 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7530000000000001e-06 2024-09-16T20:48:19.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.077 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.077 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:19.077 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.255674+0000", 2024-09-16T20:48:19.077 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0091003790000000005 2024-09-16T20:48:19.078 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.078 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.078 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:19.078 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.255757+0000", 2024-09-16T20:48:19.078 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2950999999999999e-05 2024-09-16T20:48:19.078 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.078 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.078 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:19.078 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.255861+0000", 2024-09-16T20:48:19.078 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000103798 2024-09-16T20:48:19.078 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.078 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:19.078 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.078 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.078 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.079 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:865 3.1e 3:7b30dea9:::rbd_data.110e1748a8d2.0000000000000002:4 [read 1081344~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:19.079 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.246581+0000", 2024-09-16T20:48:19.079 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.8201253140000002, 2024-09-16T20:48:19.079 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0095063049999999996, 2024-09-16T20:48:19.079 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:19.079 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:19.079 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:19.079 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:19.079 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:19.079 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:19.080 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 865 2024-09-16T20:48:19.080 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.080 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:19.080 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.080 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:19.080 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246581+0000", 2024-09-16T20:48:19.080 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.080 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.080 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.080 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:19.080 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246581+0000", 2024-09-16T20:48:19.080 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.080 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.081 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.081 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:19.081 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246581+0000", 2024-09-16T20:48:19.081 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6699999999999999e-07 2024-09-16T20:48:19.081 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.081 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.081 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:19.081 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246583+0000", 2024-09-16T20:48:19.081 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.903e-06 2024-09-16T20:48:19.081 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.081 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.081 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:19.081 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246589+0000", 2024-09-16T20:48:19.081 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1530000000000002e-06 2024-09-16T20:48:19.082 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.082 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.082 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:19.082 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246594+0000", 2024-09-16T20:48:19.082 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0699999999999997e-06 2024-09-16T20:48:19.082 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.082 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.082 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:19.082 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.255902+0000", 2024-09-16T20:48:19.082 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0093078369999999994 2024-09-16T20:48:19.082 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.082 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.082 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:19.083 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.255991+0000", 2024-09-16T20:48:19.083 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9184999999999994e-05 2024-09-16T20:48:19.083 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.083 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.083 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:19.083 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.256087+0000", 2024-09-16T20:48:19.083 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5790000000000003e-05 2024-09-16T20:48:19.083 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.083 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:19.083 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.083 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.083 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.083 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:866 3.1e 3:7b30dea9:::rbd_data.110e1748a8d2.0000000000000002:4 [read 1179648~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:19.083 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.246609+0000", 2024-09-16T20:48:19.084 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.8200969649999998, 2024-09-16T20:48:19.084 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0096725939999999996, 2024-09-16T20:48:19.084 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:19.084 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:19.084 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:19.084 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:19.084 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:19.084 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:19.084 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 866 2024-09-16T20:48:19.084 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.084 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:19.084 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.084 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:19.085 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246609+0000", 2024-09-16T20:48:19.085 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.085 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.085 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.085 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:19.085 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246609+0000", 2024-09-16T20:48:19.085 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.085 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.085 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.085 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:19.085 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246610+0000", 2024-09-16T20:48:19.085 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2099999999999996e-07 2024-09-16T20:48:19.085 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.086 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.086 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:19.086 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246612+0000", 2024-09-16T20:48:19.086 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2979999999999999e-06 2024-09-16T20:48:19.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.086 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.086 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:19.086 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246619+0000", 2024-09-16T20:48:19.086 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9190000000000002e-06 2024-09-16T20:48:19.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.086 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.086 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:19.086 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246624+0000", 2024-09-16T20:48:19.086 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4720000000000003e-06 2024-09-16T20:48:19.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.087 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.087 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:19.087 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.256116+0000", 2024-09-16T20:48:19.087 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0094914979999999993 2024-09-16T20:48:19.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.087 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.087 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:19.087 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.256190+0000", 2024-09-16T20:48:19.087 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3887000000000006e-05 2024-09-16T20:48:19.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.087 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.087 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:19.087 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.256282+0000", 2024-09-16T20:48:19.088 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1898999999999995e-05 2024-09-16T20:48:19.088 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.088 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:19.088 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.088 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.088 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.088 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:867 3.1e 3:7b30dea9:::rbd_data.110e1748a8d2.0000000000000002:4 [read 1294336~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:19.088 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.246632+0000", 2024-09-16T20:48:19.088 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.8200744169999998, 2024-09-16T20:48:19.088 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0098045530000000006, 2024-09-16T20:48:19.088 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:19.088 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:19.088 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:19.089 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:19.089 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:19.089 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:19.089 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 867 2024-09-16T20:48:19.089 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.089 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:19.089 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.089 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:19.089 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246632+0000", 2024-09-16T20:48:19.089 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.089 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.089 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.089 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:19.089 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246632+0000", 2024-09-16T20:48:19.090 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.090 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.090 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.090 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:19.090 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246632+0000", 2024-09-16T20:48:19.090 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2099999999999997e-07 2024-09-16T20:48:19.090 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.090 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.090 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:19.090 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246634+0000", 2024-09-16T20:48:19.090 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.759e-06 2024-09-16T20:48:19.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:19.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246640+0000", 2024-09-16T20:48:19.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.246e-06 2024-09-16T20:48:19.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:19.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246646+0000", 2024-09-16T20:48:19.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9719999999999997e-06 2024-09-16T20:48:19.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:19.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.256317+0000", 2024-09-16T20:48:19.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0096714690000000002 2024-09-16T20:48:19.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.092 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.092 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:19.092 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.256387+0000", 2024-09-16T20:48:19.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9589000000000001e-05 2024-09-16T20:48:19.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.092 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.092 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:19.092 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.256436+0000", 2024-09-16T20:48:19.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8996999999999999e-05 2024-09-16T20:48:19.092 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.092 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:19.093 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.093 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.093 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.093 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:868 3.1e 3:7b30dea9:::rbd_data.110e1748a8d2.0000000000000002:4 [read 1409024~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:19.093 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.246671+0000", 2024-09-16T20:48:19.093 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.8200347180000001, 2024-09-16T20:48:19.093 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0099548169999999995, 2024-09-16T20:48:19.093 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:19.093 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:19.093 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:19.093 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:19.093 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:19.093 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:19.094 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 868 2024-09-16T20:48:19.094 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.094 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:19.094 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.094 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:19.094 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246671+0000", 2024-09-16T20:48:19.094 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.094 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.094 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.094 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:19.094 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246671+0000", 2024-09-16T20:48:19.094 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.094 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.094 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:19.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246672+0000", 2024-09-16T20:48:19.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4000000000000002e-07 2024-09-16T20:48:19.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.095 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:19.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246674+0000", 2024-09-16T20:48:19.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.069e-06 2024-09-16T20:48:19.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.095 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:19.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246680+0000", 2024-09-16T20:48:19.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9970000000000004e-06 2024-09-16T20:48:19.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.096 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.096 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:19.096 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246685+0000", 2024-09-16T20:48:19.096 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0769999999999998e-06 2024-09-16T20:48:19.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.096 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.096 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:19.096 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.256473+0000", 2024-09-16T20:48:19.096 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0097883620000000001 2024-09-16T20:48:19.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.096 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.096 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:19.096 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.256538+0000", 2024-09-16T20:48:19.096 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5068999999999997e-05 2024-09-16T20:48:19.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.097 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.097 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:19.097 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.256626+0000", 2024-09-16T20:48:19.097 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7802999999999999e-05 2024-09-16T20:48:19.097 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.097 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:19.097 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.097 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.097 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:869 3.1e 3:7b30dea9:::rbd_data.110e1748a8d2.0000000000000002:4 [read 1507328~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:19.097 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.246691+0000", 2024-09-16T20:48:19.097 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.8200146889999997, 2024-09-16T20:48:19.097 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.010068172, 2024-09-16T20:48:19.098 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:19.098 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:19.098 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:19.098 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:19.098 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:19.098 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:19.098 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 869 2024-09-16T20:48:19.098 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.098 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:19.098 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.098 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:19.098 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246691+0000", 2024-09-16T20:48:19.098 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.099 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.099 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:19.099 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246691+0000", 2024-09-16T20:48:19.099 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.099 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.099 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:19.099 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246692+0000", 2024-09-16T20:48:19.099 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.89e-07 2024-09-16T20:48:19.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.099 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.099 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:19.099 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246693+0000", 2024-09-16T20:48:19.100 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8300000000000001e-06 2024-09-16T20:48:19.100 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.100 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.100 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:19.100 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246700+0000", 2024-09-16T20:48:19.100 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1129999999999997e-06 2024-09-16T20:48:19.100 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.100 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.100 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:19.100 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246704+0000", 2024-09-16T20:48:19.100 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8620000000000002e-06 2024-09-16T20:48:19.100 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.100 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.101 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:19.101 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.256658+0000", 2024-09-16T20:48:19.101 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0099539320000000004 2024-09-16T20:48:19.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.101 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:19.101 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.256703+0000", 2024-09-16T20:48:19.101 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5056999999999999e-05 2024-09-16T20:48:19.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.101 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:19.101 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.256759+0000", 2024-09-16T20:48:19.101 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5989000000000002e-05 2024-09-16T20:48:19.102 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.102 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:19.102 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.102 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.102 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.102 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:938 3.1b 3:db0c5633:::rbd_data.110e1748a8d2.0000000000000007:4 [read 1392640~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:19.102 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.247004+0000", 2024-09-16T20:48:19.102 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.8197016609999999, 2024-09-16T20:48:19.102 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.009170213, 2024-09-16T20:48:19.102 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:19.102 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:19.102 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:19.102 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:19.102 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:19.103 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:19.103 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 938 2024-09-16T20:48:19.103 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.103 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:19.103 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.103 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:19.103 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247004+0000", 2024-09-16T20:48:19.103 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.103 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.103 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.103 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:19.103 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247004+0000", 2024-09-16T20:48:19.103 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.103 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.104 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.104 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:19.104 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247005+0000", 2024-09-16T20:48:19.104 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2e-07 2024-09-16T20:48:19.104 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.104 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.104 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:19.104 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247007+0000", 2024-09-16T20:48:19.104 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9580000000000001e-06 2024-09-16T20:48:19.104 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.104 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.104 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:19.104 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247012+0000", 2024-09-16T20:48:19.105 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2220000000000001e-06 2024-09-16T20:48:19.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.105 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:19.105 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247015+0000", 2024-09-16T20:48:19.105 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1700000000000001e-06 2024-09-16T20:48:19.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.105 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:19.105 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.256001+0000", 2024-09-16T20:48:19.105 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0089862680000000004 2024-09-16T20:48:19.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.105 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:19.105 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.256077+0000", 2024-09-16T20:48:19.106 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5568000000000006e-05 2024-09-16T20:48:19.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.106 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:19.106 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.256174+0000", 2024-09-16T20:48:19.106 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7507e-05 2024-09-16T20:48:19.106 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.106 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:19.106 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.106 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:939 3.1b 3:db0c5633:::rbd_data.110e1748a8d2.0000000000000007:4 [read 1507328~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:19.106 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.247019+0000", 2024-09-16T20:48:19.106 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.819686935, 2024-09-16T20:48:19.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0093507969999999992, 2024-09-16T20:48:19.107 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:19.107 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:19.107 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:19.107 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:19.107 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:19.107 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:19.107 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 939 2024-09-16T20:48:19.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.107 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:19.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.107 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:19.107 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247019+0000", 2024-09-16T20:48:19.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:19.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247019+0000", 2024-09-16T20:48:19.108 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:19.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247019+0000", 2024-09-16T20:48:19.108 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5600000000000002e-07 2024-09-16T20:48:19.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:19.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247020+0000", 2024-09-16T20:48:19.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1260000000000001e-06 2024-09-16T20:48:19.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.109 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.109 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:19.109 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247024+0000", 2024-09-16T20:48:19.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1300000000000003e-06 2024-09-16T20:48:19.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.109 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.109 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:19.109 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247027+0000", 2024-09-16T20:48:19.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6630000000000002e-06 2024-09-16T20:48:19.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.109 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.109 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:19.110 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.256199+0000", 2024-09-16T20:48:19.110 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0091721400000000005 2024-09-16T20:48:19.110 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.110 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.110 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:19.110 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.256275+0000", 2024-09-16T20:48:19.110 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6081999999999994e-05 2024-09-16T20:48:19.110 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.110 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.110 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:19.110 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.256370+0000", 2024-09-16T20:48:19.110 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4400000000000004e-05 2024-09-16T20:48:19.110 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.110 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:19.111 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.111 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.111 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.111 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:277 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 229376~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-16T20:48:19.111 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:18.933574+0000", 2024-09-16T20:48:19.111 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.13313235600000001, 2024-09-16T20:48:19.111 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0091469129999999996, 2024-09-16T20:48:19.111 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:19.111 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:19.111 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:19.111 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:19.111 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:19.111 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:19.111 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 277 2024-09-16T20:48:19.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.112 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:19.112 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:19.112 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933574+0000", 2024-09-16T20:48:19.112 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.112 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:19.112 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933574+0000", 2024-09-16T20:48:19.112 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.112 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.113 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:19.113 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933574+0000", 2024-09-16T20:48:19.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3900000000000001e-07 2024-09-16T20:48:19.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.113 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.113 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:19.113 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933575+0000", 2024-09-16T20:48:19.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.15e-06 2024-09-16T20:48:19.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.113 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.113 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:19.113 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933579+0000", 2024-09-16T20:48:19.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7280000000000001e-06 2024-09-16T20:48:19.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.114 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.114 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:19.114 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933581+0000", 2024-09-16T20:48:19.114 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6860000000000002e-06 2024-09-16T20:48:19.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.114 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.114 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:19.114 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.942565+0000", 2024-09-16T20:48:19.114 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0089838910000000008 2024-09-16T20:48:19.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.114 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.114 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:19.114 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.942616+0000", 2024-09-16T20:48:19.114 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1214999999999997e-05 2024-09-16T20:48:19.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.115 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.115 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:19.115 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.942720+0000", 2024-09-16T20:48:19.115 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000104004 2024-09-16T20:48:19.115 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.115 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:19.115 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.115 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.115 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:278 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 215040~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-16T20:48:19.115 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:18.933592+0000", 2024-09-16T20:48:19.115 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.133114026, 2024-09-16T20:48:19.115 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0093109009999999999, 2024-09-16T20:48:19.116 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:19.116 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:19.116 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:19.116 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:19.116 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:19.116 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:19.116 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 278 2024-09-16T20:48:19.116 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.116 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:19.116 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.116 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:19.116 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933592+0000", 2024-09-16T20:48:19.116 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.117 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:19.117 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933592+0000", 2024-09-16T20:48:19.117 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.117 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:19.117 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933592+0000", 2024-09-16T20:48:19.117 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3599999999999999e-07 2024-09-16T20:48:19.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.117 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:19.117 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933594+0000", 2024-09-16T20:48:19.118 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2890000000000001e-06 2024-09-16T20:48:19.118 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.118 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.118 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:19.118 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933597+0000", 2024-09-16T20:48:19.118 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2949999999999998e-06 2024-09-16T20:48:19.118 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.118 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.118 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:19.118 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933600+0000", 2024-09-16T20:48:19.118 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6979999999999998e-06 2024-09-16T20:48:19.118 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.118 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.118 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:19.119 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.942751+0000", 2024-09-16T20:48:19.119 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0091511040000000002 2024-09-16T20:48:19.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.119 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.119 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:19.119 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.942797+0000", 2024-09-16T20:48:19.119 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6332000000000003e-05 2024-09-16T20:48:19.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.119 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.119 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:19.119 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.942903+0000", 2024-09-16T20:48:19.119 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000105747 2024-09-16T20:48:19.119 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.119 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:19.120 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.120 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.120 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.120 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:279 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 237568~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-16T20:48:19.120 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:18.933603+0000", 2024-09-16T20:48:19.120 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.13310275299999999, 2024-09-16T20:48:19.120 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0094875810000000001, 2024-09-16T20:48:19.120 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:19.120 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:19.120 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:19.120 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:19.120 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:19.120 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:19.120 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 279 2024-09-16T20:48:19.121 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.121 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:19.121 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.121 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:19.121 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933603+0000", 2024-09-16T20:48:19.121 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.121 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.121 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.121 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:19.121 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933603+0000", 2024-09-16T20:48:19.121 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.121 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.121 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.122 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:19.122 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933604+0000", 2024-09-16T20:48:19.122 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3700000000000001e-07 2024-09-16T20:48:19.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.122 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.122 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:19.122 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933604+0000", 2024-09-16T20:48:19.122 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8700000000000004e-07 2024-09-16T20:48:19.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.122 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.122 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:19.122 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933612+0000", 2024-09-16T20:48:19.122 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3220000000000004e-06 2024-09-16T20:48:19.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.123 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.123 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:19.123 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933616+0000", 2024-09-16T20:48:19.123 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1110000000000001e-06 2024-09-16T20:48:19.123 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.123 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.123 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:19.123 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.942934+0000", 2024-09-16T20:48:19.123 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0093178800000000006 2024-09-16T20:48:19.123 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.123 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.123 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:19.123 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.942985+0000", 2024-09-16T20:48:19.123 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0957e-05 2024-09-16T20:48:19.123 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.124 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.124 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:19.124 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.943091+0000", 2024-09-16T20:48:19.124 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000105987 2024-09-16T20:48:19.124 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.124 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:19.124 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.124 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.124 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.124 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:280 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 223232~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-16T20:48:19.124 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:18.933621+0000", 2024-09-16T20:48:19.124 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.133085282, 2024-09-16T20:48:19.124 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0096694529999999997, 2024-09-16T20:48:19.125 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:19.125 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:19.125 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:19.125 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:19.125 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:19.125 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:19.125 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 280 2024-09-16T20:48:19.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.125 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:19.125 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.125 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:19.125 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933621+0000", 2024-09-16T20:48:19.125 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.126 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.126 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:19.126 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933621+0000", 2024-09-16T20:48:19.126 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.126 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.126 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.126 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:19.126 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933621+0000", 2024-09-16T20:48:19.126 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6199999999999999e-07 2024-09-16T20:48:19.126 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.126 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.126 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:19.126 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933625+0000", 2024-09-16T20:48:19.126 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9230000000000004e-06 2024-09-16T20:48:19.127 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.127 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.127 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:19.127 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933628+0000", 2024-09-16T20:48:19.127 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4759999999999998e-06 2024-09-16T20:48:19.127 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.127 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.127 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:19.127 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933632+0000", 2024-09-16T20:48:19.127 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7280000000000001e-06 2024-09-16T20:48:19.127 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.127 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.127 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:19.128 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.943128+0000", 2024-09-16T20:48:19.128 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0094958909999999994 2024-09-16T20:48:19.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.128 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.128 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:19.128 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.943183+0000", 2024-09-16T20:48:19.128 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4858999999999998e-05 2024-09-16T20:48:19.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.128 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.128 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:19.128 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.943290+0000", 2024-09-16T20:48:19.128 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000107314 2024-09-16T20:48:19.128 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.128 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:19.129 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.129 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.129 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.129 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:281 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 245760~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-16T20:48:19.129 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:18.933636+0000", 2024-09-16T20:48:19.129 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.13307036899999999, 2024-09-16T20:48:19.129 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0098598379999999992, 2024-09-16T20:48:19.129 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:19.129 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:19.129 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:19.130 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:19.130 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:19.130 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:19.130 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 281 2024-09-16T20:48:19.130 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.130 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:19.130 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.130 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:19.130 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933636+0000", 2024-09-16T20:48:19.130 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.130 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.130 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.130 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:19.131 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933636+0000", 2024-09-16T20:48:19.131 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.131 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.131 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.131 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:19.131 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933636+0000", 2024-09-16T20:48:19.131 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6199999999999999e-07 2024-09-16T20:48:19.131 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.131 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.131 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:19.131 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933637+0000", 2024-09-16T20:48:19.131 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.074e-06 2024-09-16T20:48:19.131 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.132 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.132 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:19.132 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933641+0000", 2024-09-16T20:48:19.132 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.216e-06 2024-09-16T20:48:19.132 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.132 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.132 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:19.132 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933645+0000", 2024-09-16T20:48:19.132 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3459999999999998e-06 2024-09-16T20:48:19.132 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.132 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.132 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:19.132 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.943333+0000", 2024-09-16T20:48:19.133 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00968865 2024-09-16T20:48:19.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.133 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:19.133 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.943381+0000", 2024-09-16T20:48:19.133 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7469999999999998e-05 2024-09-16T20:48:19.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.133 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:19.133 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.943495+0000", 2024-09-16T20:48:19.133 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011472 2024-09-16T20:48:19.133 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.133 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:19.134 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.134 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.134 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:282 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 231424~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-16T20:48:19.134 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:18.933648+0000", 2024-09-16T20:48:19.134 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.133058179, 2024-09-16T20:48:19.134 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.010039625, 2024-09-16T20:48:19.134 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:19.134 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:19.134 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:19.134 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:19.134 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:19.134 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:19.134 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 282 2024-09-16T20:48:19.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.135 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:19.135 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.135 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:19.135 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933648+0000", 2024-09-16T20:48:19.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.135 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.135 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:19.135 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933648+0000", 2024-09-16T20:48:19.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.135 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.135 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:19.136 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933648+0000", 2024-09-16T20:48:19.136 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3099999999999999e-07 2024-09-16T20:48:19.136 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.136 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.136 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:19.136 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933653+0000", 2024-09-16T20:48:19.136 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0549999999999999e-06 2024-09-16T20:48:19.136 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.136 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.136 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:19.136 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933656+0000", 2024-09-16T20:48:19.136 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1999999999999999e-06 2024-09-16T20:48:19.136 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.137 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.137 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:19.137 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933659+0000", 2024-09-16T20:48:19.137 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.554e-06 2024-09-16T20:48:19.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.137 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.137 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:19.137 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.943526+0000", 2024-09-16T20:48:19.137 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0098671850000000005 2024-09-16T20:48:19.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.137 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.137 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:19.137 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.943573+0000", 2024-09-16T20:48:19.137 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6869999999999997e-05 2024-09-16T20:48:19.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.138 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:19.138 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.943687+0000", 2024-09-16T20:48:19.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011443 2024-09-16T20:48:19.138 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.138 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:19.138 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.138 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:283 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 253952~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-16T20:48:19.138 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:18.933662+0000", 2024-09-16T20:48:19.138 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.13304371300000001, 2024-09-16T20:48:19.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.010209279, 2024-09-16T20:48:19.139 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:19.139 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:19.139 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:19.139 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:19.139 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:19.139 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:19.139 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 283 2024-09-16T20:48:19.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.139 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:19.139 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.139 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:19.139 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933662+0000", 2024-09-16T20:48:19.139 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.140 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.140 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.140 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:19.140 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933662+0000", 2024-09-16T20:48:19.140 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.140 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.140 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.140 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:19.140 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933662+0000", 2024-09-16T20:48:19.140 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4200000000000002e-07 2024-09-16T20:48:19.140 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.140 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.140 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:19.140 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933664+0000", 2024-09-16T20:48:19.141 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.068e-06 2024-09-16T20:48:19.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.141 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.141 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:19.141 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933667+0000", 2024-09-16T20:48:19.141 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5829999999999998e-06 2024-09-16T20:48:19.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.141 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.141 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:19.141 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933670+0000", 2024-09-16T20:48:19.141 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0630000000000002e-06 2024-09-16T20:48:19.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.142 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.142 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:19.142 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.943719+0000", 2024-09-16T20:48:19.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.010048629999999999 2024-09-16T20:48:19.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.142 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.142 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:19.142 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.943765+0000", 2024-09-16T20:48:19.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6641999999999998e-05 2024-09-16T20:48:19.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.142 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.142 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:19.142 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.943871+0000", 2024-09-16T20:48:19.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000106051 2024-09-16T20:48:19.143 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.143 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:19.143 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.143 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:284 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 239616~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-16T20:48:19.143 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:18.933680+0000", 2024-09-16T20:48:19.143 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.13302613299999999, 2024-09-16T20:48:19.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.010385393999999999, 2024-09-16T20:48:19.143 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:19.143 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:19.143 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:19.143 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:19.143 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:19.144 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:19.144 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 284 2024-09-16T20:48:19.144 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.144 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:19.144 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.144 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:19.144 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933680+0000", 2024-09-16T20:48:19.144 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.144 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.144 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.144 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:19.144 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933680+0000", 2024-09-16T20:48:19.144 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.144 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.145 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.145 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:19.145 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933680+0000", 2024-09-16T20:48:19.145 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8099999999999998e-07 2024-09-16T20:48:19.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.145 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.145 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:19.145 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933682+0000", 2024-09-16T20:48:19.145 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.381e-06 2024-09-16T20:48:19.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.145 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.145 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:19.145 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933685+0000", 2024-09-16T20:48:19.145 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.32e-06 2024-09-16T20:48:19.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.146 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:19.146 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933689+0000", 2024-09-16T20:48:19.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9820000000000002e-06 2024-09-16T20:48:19.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.146 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:19.146 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.943909+0000", 2024-09-16T20:48:19.147 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.01022034 2024-09-16T20:48:19.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.147 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.147 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:19.147 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.943964+0000", 2024-09-16T20:48:19.147 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4617999999999998e-05 2024-09-16T20:48:19.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.147 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.147 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:19.147 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.944065+0000", 2024-09-16T20:48:19.147 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101372 2024-09-16T20:48:19.147 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.147 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:19.148 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.148 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.148 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.148 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:285 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 247808~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-16T20:48:19.148 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:18.933693+0000", 2024-09-16T20:48:19.148 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.13301339000000001, 2024-09-16T20:48:19.148 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.010567497, 2024-09-16T20:48:19.148 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:19.148 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:19.148 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:19.148 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:19.148 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:19.148 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:19.148 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 285 2024-09-16T20:48:19.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.149 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:19.149 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.149 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:19.149 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933693+0000", 2024-09-16T20:48:19.149 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.149 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.149 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:19.149 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933693+0000", 2024-09-16T20:48:19.149 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.149 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.150 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:19.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933693+0000", 2024-09-16T20:48:19.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.28e-07 2024-09-16T20:48:19.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.150 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.150 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:19.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933694+0000", 2024-09-16T20:48:19.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9999999999999995e-07 2024-09-16T20:48:19.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.150 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.150 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:19.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933697+0000", 2024-09-16T20:48:19.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0019999999999998e-06 2024-09-16T20:48:19.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:19.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933699+0000", 2024-09-16T20:48:19.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.667e-06 2024-09-16T20:48:19.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:19.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.944102+0000", 2024-09-16T20:48:19.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.010402183000000001 2024-09-16T20:48:19.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:19.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.944155+0000", 2024-09-16T20:48:19.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3721999999999998e-05 2024-09-16T20:48:19.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.152 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.152 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:19.152 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.944260+0000", 2024-09-16T20:48:19.152 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000104695 2024-09-16T20:48:19.152 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.152 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:19.152 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.152 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.152 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:286 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 256000~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-16T20:48:19.152 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:18.933715+0000", 2024-09-16T20:48:19.152 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.13299117799999999, 2024-09-16T20:48:19.153 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.010732683, 2024-09-16T20:48:19.153 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:19.153 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:19.153 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:19.153 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:19.153 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:19.153 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:19.153 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 286 2024-09-16T20:48:19.153 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.153 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:19.153 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.153 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:19.153 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933715+0000", 2024-09-16T20:48:19.153 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.154 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.154 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.154 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:19.154 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933715+0000", 2024-09-16T20:48:19.154 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.154 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.154 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.154 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:19.154 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933715+0000", 2024-09-16T20:48:19.154 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-16T20:48:19.154 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.154 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.154 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:19.154 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933716+0000", 2024-09-16T20:48:19.154 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1379999999999999e-06 2024-09-16T20:48:19.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.155 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.155 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:19.155 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933720+0000", 2024-09-16T20:48:19.155 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3299999999999999e-06 2024-09-16T20:48:19.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.155 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.155 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:19.155 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933722+0000", 2024-09-16T20:48:19.155 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.836e-06 2024-09-16T20:48:19.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.155 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.155 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:19.156 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.944289+0000", 2024-09-16T20:48:19.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.010566880000000001 2024-09-16T20:48:19.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.156 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.156 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:19.156 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.944342+0000", 2024-09-16T20:48:19.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2918999999999999e-05 2024-09-16T20:48:19.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.156 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.156 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:19.156 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.944447+0000", 2024-09-16T20:48:19.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010521 2024-09-16T20:48:19.156 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.156 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:19.157 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.157 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.157 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.157 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:287 3.1e 3:79e5ec01:::rbd_header.110e891a9b2b:head [watch reconnect cookie 140567972193216 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-16T20:48:19.157 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:18.933737+0000", 2024-09-16T20:48:19.157 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.13296922899999999, 2024-09-16T20:48:19.157 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.011955889000000001, 2024-09-16T20:48:19.157 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:19.157 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:19.157 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:19.157 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:19.157 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:19.157 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:19.157 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 287 2024-09-16T20:48:19.158 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.158 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:19.158 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.158 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:19.158 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933737+0000", 2024-09-16T20:48:19.158 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.158 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.158 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.158 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:19.158 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933737+0000", 2024-09-16T20:48:19.158 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.158 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.158 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.158 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:19.159 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933737+0000", 2024-09-16T20:48:19.159 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6600000000000002e-07 2024-09-16T20:48:19.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.159 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.159 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:19.159 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933739+0000", 2024-09-16T20:48:19.159 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4899999999999999e-06 2024-09-16T20:48:19.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.159 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.159 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:19.159 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933742+0000", 2024-09-16T20:48:19.159 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.709e-06 2024-09-16T20:48:19.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.159 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.160 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:19.160 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.933746+0000", 2024-09-16T20:48:19.160 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9260000000000002e-06 2024-09-16T20:48:19.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.160 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.160 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:19.160 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.944469+0000", 2024-09-16T20:48:19.160 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.010722581 2024-09-16T20:48:19.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.160 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.160 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:19.160 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.944517+0000", 2024-09-16T20:48:19.160 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8269999999999997e-05 2024-09-16T20:48:19.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.161 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.161 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:19.161 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:18.945693+0000", 2024-09-16T20:48:19.161 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0011755470000000001 2024-09-16T20:48:19.161 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.161 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:19.161 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.161 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.161 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:19.161 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:19.163 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:48:19.163 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:19.161+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:19.263 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:19.262+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:19.274 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:19.274 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:48:19.274 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:48:19.274 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:19.283 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:48:19.364 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:19.362+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:19.391 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:19.391 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:48:19.391 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:48:19.391 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:48:19.391 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:19.400 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:48:19.465 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:19.464+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:19.475 INFO:teuthology.orchestra.run.smithi139.stdout:617 write 0x38a2200 thru 0x38a3ef2 (0x1cf3 bytes) 2024-09-16T20:48:19.477 INFO:teuthology.orchestra.run.smithi139.stdout:618 write 0x38bba00 thru 0x38c08c3 (0x4ec4 bytes) 2024-09-16T20:48:19.479 INFO:teuthology.orchestra.run.smithi139.stdout:619 write 0x2c7f600 thru 0x2c7f78f (0x190 bytes) 2024-09-16T20:48:19.491 INFO:teuthology.orchestra.run.smithi139.stdout:621 trunc from 0xe886000 to 0x597f400 2024-09-16T20:48:19.547 INFO:teuthology.orchestra.run.smithi029.stderr:noscrub is set 2024-09-16T20:48:19.551 INFO:teuthology.orchestra.run.smithi029.stderr:set 2.4 pg_upmap_items mapping to [5->2,0->1] 2024-09-16T20:48:19.566 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:19.564+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:19.578 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:19.578 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:48:19.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:48:19.578 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:48:19.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.578 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:962 3.19 3:9cf0b687:::rbd_data.110e1748a8d2.0000000000000009:4 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:19.579 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.241027+0000", 2024-09-16T20:48:19.579 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.335529105, 2024-09-16T20:48:19.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.029355698999999999, 2024-09-16T20:48:19.579 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:19.579 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:19.579 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:19.579 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:19.579 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:19.579 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:19.579 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 962 2024-09-16T20:48:19.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.579 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:19.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:19.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241027+0000", 2024-09-16T20:48:19.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:19.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241027+0000", 2024-09-16T20:48:19.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:19.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241028+0000", 2024-09-16T20:48:19.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4499999999999998e-07 2024-09-16T20:48:19.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:19.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241029+0000", 2024-09-16T20:48:19.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.767e-06 2024-09-16T20:48:19.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:19.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241036+0000", 2024-09-16T20:48:19.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1210000000000003e-06 2024-09-16T20:48:19.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:19.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241048+0000", 2024-09-16T20:48:19.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2142e-05 2024-09-16T20:48:19.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:19.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.270281+0000", 2024-09-16T20:48:19.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.029233362999999998 2024-09-16T20:48:19.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:19.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.270328+0000", 2024-09-16T20:48:19.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7398999999999998e-05 2024-09-16T20:48:19.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:19.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.270383+0000", 2024-09-16T20:48:19.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4561999999999997e-05 2024-09-16T20:48:19.583 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.583 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:19.583 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.583 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:963 3.19 3:9cf0b687:::rbd_data.110e1748a8d2.0000000000000009:4 [read 1064960~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:19.583 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.241055+0000", 2024-09-16T20:48:19.583 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.3355011509999999, 2024-09-16T20:48:19.584 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.029462128000000001, 2024-09-16T20:48:19.584 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:19.584 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:19.584 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:19.584 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:19.584 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:19.584 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:19.584 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 963 2024-09-16T20:48:19.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.584 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:19.584 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.584 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:19.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241055+0000", 2024-09-16T20:48:19.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:19.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241055+0000", 2024-09-16T20:48:19.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:19.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241056+0000", 2024-09-16T20:48:19.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3000000000000001e-07 2024-09-16T20:48:19.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:19.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241058+0000", 2024-09-16T20:48:19.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2280000000000001e-06 2024-09-16T20:48:19.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:19.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241065+0000", 2024-09-16T20:48:19.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7399999999999998e-06 2024-09-16T20:48:19.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:19.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241070+0000", 2024-09-16T20:48:19.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0309999999999998e-06 2024-09-16T20:48:19.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:19.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.270408+0000", 2024-09-16T20:48:19.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.029337698999999998 2024-09-16T20:48:19.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:19.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.270454+0000", 2024-09-16T20:48:19.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6690000000000002e-05 2024-09-16T20:48:19.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:19.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.270517+0000", 2024-09-16T20:48:19.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.321e-05 2024-09-16T20:48:19.588 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.588 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:19.588 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.588 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:964 3.19 3:9cf0b687:::rbd_data.110e1748a8d2.0000000000000009:4 [read 1179648~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:19.588 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.241240+0000", 2024-09-16T20:48:19.588 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.335316153, 2024-09-16T20:48:19.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.029396230999999998, 2024-09-16T20:48:19.588 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:19.589 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:19.589 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:19.589 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:19.589 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:19.589 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:19.589 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 964 2024-09-16T20:48:19.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.589 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:19.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:19.589 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241240+0000", 2024-09-16T20:48:19.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.590 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:19.590 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241240+0000", 2024-09-16T20:48:19.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.590 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:19.590 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241241+0000", 2024-09-16T20:48:19.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0699999999999997e-07 2024-09-16T20:48:19.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.590 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:19.590 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241244+0000", 2024-09-16T20:48:19.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0460000000000001e-06 2024-09-16T20:48:19.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.591 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:19.591 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241254+0000", 2024-09-16T20:48:19.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6840000000000008e-06 2024-09-16T20:48:19.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.591 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:19.591 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241260+0000", 2024-09-16T20:48:19.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1920000000000002e-06 2024-09-16T20:48:19.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.592 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:19.592 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.270541+0000", 2024-09-16T20:48:19.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.029281523 2024-09-16T20:48:19.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.592 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:19.592 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.270584+0000", 2024-09-16T20:48:19.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2682000000000002e-05 2024-09-16T20:48:19.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.592 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:19.592 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.270637+0000", 2024-09-16T20:48:19.593 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2497000000000002e-05 2024-09-16T20:48:19.593 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.593 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:19.593 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.593 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:965 3.19 3:9cf0b687:::rbd_data.110e1748a8d2.0000000000000009:4 [read 1277952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:19.593 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.241270+0000", 2024-09-16T20:48:19.593 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.3352863470000003, 2024-09-16T20:48:19.593 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.029504444000000001, 2024-09-16T20:48:19.593 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:19.593 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:19.593 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:19.593 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:19.594 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:19.594 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:19.594 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 965 2024-09-16T20:48:19.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.594 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:19.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.594 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:19.594 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241270+0000", 2024-09-16T20:48:19.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.594 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:19.594 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241270+0000", 2024-09-16T20:48:19.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.595 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:19.595 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241271+0000", 2024-09-16T20:48:19.595 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.75e-07 2024-09-16T20:48:19.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.595 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:19.595 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241273+0000", 2024-09-16T20:48:19.595 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3149999999999999e-06 2024-09-16T20:48:19.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.595 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:19.595 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241280+0000", 2024-09-16T20:48:19.596 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7429999999999996e-06 2024-09-16T20:48:19.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.596 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.596 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:19.596 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241285+0000", 2024-09-16T20:48:19.596 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0180000000000001e-06 2024-09-16T20:48:19.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.596 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.596 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:19.596 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.270661+0000", 2024-09-16T20:48:19.596 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.029376270999999999 2024-09-16T20:48:19.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.596 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.597 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:19.597 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.270710+0000", 2024-09-16T20:48:19.597 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8979e-05 2024-09-16T20:48:19.597 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.597 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.597 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:19.597 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.270775+0000", 2024-09-16T20:48:19.597 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4642999999999998e-05 2024-09-16T20:48:19.597 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.597 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:19.597 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.597 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.597 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.597 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:966 3.19 3:9cf0b687:::rbd_data.110e1748a8d2.0000000000000009:4 [read 1392640~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:19.598 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.241292+0000", 2024-09-16T20:48:19.598 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.3352645609999998, 2024-09-16T20:48:19.598 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.029601085999999999, 2024-09-16T20:48:19.598 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:19.598 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:19.598 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:19.598 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:19.598 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:19.598 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:19.598 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 966 2024-09-16T20:48:19.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.598 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:19.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.598 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:19.598 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241292+0000", 2024-09-16T20:48:19.599 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.599 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.599 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:19.599 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241292+0000", 2024-09-16T20:48:19.599 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.599 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.599 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:19.599 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241292+0000", 2024-09-16T20:48:19.599 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8599999999999999e-07 2024-09-16T20:48:19.599 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.599 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:19.599 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241294+0000", 2024-09-16T20:48:19.599 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0310000000000001e-06 2024-09-16T20:48:19.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.600 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.600 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:19.600 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241300+0000", 2024-09-16T20:48:19.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0109999999999997e-06 2024-09-16T20:48:19.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.600 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.600 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:19.600 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241305+0000", 2024-09-16T20:48:19.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4850000000000003e-06 2024-09-16T20:48:19.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.600 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.600 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:19.600 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.270798+0000", 2024-09-16T20:48:19.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.029493385 2024-09-16T20:48:19.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.601 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:19.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.270840+0000", 2024-09-16T20:48:19.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2026e-05 2024-09-16T20:48:19.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.601 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:19.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.270893+0000", 2024-09-16T20:48:19.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2762000000000001e-05 2024-09-16T20:48:19.601 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.601 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:19.602 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.602 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:967 3.19 3:9cf0b687:::rbd_data.110e1748a8d2.0000000000000009:4 [read 1507328~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:19.602 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.241403+0000", 2024-09-16T20:48:19.602 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.3351532349999999, 2024-09-16T20:48:19.602 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.029625058999999999, 2024-09-16T20:48:19.602 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:19.602 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:19.602 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:19.602 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:19.602 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:19.602 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:19.602 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 967 2024-09-16T20:48:19.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.603 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:19.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.603 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:19.603 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241403+0000", 2024-09-16T20:48:19.603 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.603 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:19.603 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241403+0000", 2024-09-16T20:48:19.603 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.604 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:19.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241404+0000", 2024-09-16T20:48:19.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8899999999999999e-07 2024-09-16T20:48:19.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.604 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:19.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241407+0000", 2024-09-16T20:48:19.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1370000000000002e-06 2024-09-16T20:48:19.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.604 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:19.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241416+0000", 2024-09-16T20:48:19.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5210000000000008e-06 2024-09-16T20:48:19.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:19.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241431+0000", 2024-09-16T20:48:19.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4164e-05 2024-09-16T20:48:19.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:19.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.270918+0000", 2024-09-16T20:48:19.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0294875 2024-09-16T20:48:19.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:19.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.270965+0000", 2024-09-16T20:48:19.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7213999999999999e-05 2024-09-16T20:48:19.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.606 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:19.606 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.271028+0000", 2024-09-16T20:48:19.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2934000000000004e-05 2024-09-16T20:48:19.606 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.606 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:19.606 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.606 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:982 3.19 3:9eda7881:::rbd_data.110e1748a8d2.000000000000000b:4 [read 81920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:19.606 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.241441+0000", 2024-09-16T20:48:19.607 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.3351154540000003, 2024-09-16T20:48:19.607 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.029707366999999998, 2024-09-16T20:48:19.607 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:19.607 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:19.607 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:19.607 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:19.607 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:19.607 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:19.607 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 982 2024-09-16T20:48:19.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.607 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:19.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.607 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:19.608 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241441+0000", 2024-09-16T20:48:19.608 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.608 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:19.608 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241441+0000", 2024-09-16T20:48:19.608 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.608 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:19.608 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241442+0000", 2024-09-16T20:48:19.608 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4099999999999999e-07 2024-09-16T20:48:19.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:19.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241452+0000", 2024-09-16T20:48:19.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0035e-05 2024-09-16T20:48:19.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:19.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241459+0000", 2024-09-16T20:48:19.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2230000000000001e-06 2024-09-16T20:48:19.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.610 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:19.610 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241464+0000", 2024-09-16T20:48:19.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7319999999999998e-06 2024-09-16T20:48:19.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.610 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:19.610 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.271052+0000", 2024-09-16T20:48:19.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.029588857999999999 2024-09-16T20:48:19.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.611 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:19.611 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.271095+0000", 2024-09-16T20:48:19.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2194000000000003e-05 2024-09-16T20:48:19.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.611 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:19.611 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.271148+0000", 2024-09-16T20:48:19.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3783999999999999e-05 2024-09-16T20:48:19.612 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.612 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:19.612 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.612 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:983 3.19 3:9eda7881:::rbd_data.110e1748a8d2.000000000000000b:4 [read 196608~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:19.612 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.241472+0000", 2024-09-16T20:48:19.612 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.3350840599999998, 2024-09-16T20:48:19.612 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.029807613, 2024-09-16T20:48:19.612 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:19.612 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:19.612 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:19.612 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:19.612 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:19.613 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:19.613 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 983 2024-09-16T20:48:19.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.613 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:19.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.613 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:19.613 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241472+0000", 2024-09-16T20:48:19.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.613 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:19.613 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241472+0000", 2024-09-16T20:48:19.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.614 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:19.614 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241473+0000", 2024-09-16T20:48:19.614 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1999999999999999e-07 2024-09-16T20:48:19.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.614 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:19.614 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241480+0000", 2024-09-16T20:48:19.614 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3320000000000003e-06 2024-09-16T20:48:19.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.614 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:19.615 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241487+0000", 2024-09-16T20:48:19.615 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8680000000000001e-06 2024-09-16T20:48:19.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.615 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.615 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:19.615 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241492+0000", 2024-09-16T20:48:19.615 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0760000000000002e-06 2024-09-16T20:48:19.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.615 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.615 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:19.615 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.271172+0000", 2024-09-16T20:48:19.615 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.029679252999999999 2024-09-16T20:48:19.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.615 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.616 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:19.616 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.271218+0000", 2024-09-16T20:48:19.616 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6770000000000001e-05 2024-09-16T20:48:19.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.616 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.616 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:19.616 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.271280+0000", 2024-09-16T20:48:19.616 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1693999999999996e-05 2024-09-16T20:48:19.616 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.616 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:19.616 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.616 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.617 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:984 3.19 3:9eda7881:::rbd_data.110e1748a8d2.000000000000000b:4 [read 294912~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:19.617 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.241500+0000", 2024-09-16T20:48:19.617 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.3350560460000001, 2024-09-16T20:48:19.617 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.029939342000000001, 2024-09-16T20:48:19.617 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:19.617 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:19.617 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:19.617 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:19.617 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:19.617 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:19.617 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 984 2024-09-16T20:48:19.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.617 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:19.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:19.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241500+0000", 2024-09-16T20:48:19.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:19.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241500+0000", 2024-09-16T20:48:19.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:19.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241501+0000", 2024-09-16T20:48:19.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5799999999999999e-07 2024-09-16T20:48:19.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.619 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:19.619 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241503+0000", 2024-09-16T20:48:19.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.424e-06 2024-09-16T20:48:19.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.619 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:19.619 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241510+0000", 2024-09-16T20:48:19.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6640000000000001e-06 2024-09-16T20:48:19.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.619 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:19.620 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241515+0000", 2024-09-16T20:48:19.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7260000000000002e-06 2024-09-16T20:48:19.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.620 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:19.620 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.271305+0000", 2024-09-16T20:48:19.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.029789777 2024-09-16T20:48:19.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.620 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:19.620 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.271373+0000", 2024-09-16T20:48:19.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8626000000000003e-05 2024-09-16T20:48:19.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.621 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:19.621 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.271440+0000", 2024-09-16T20:48:19.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6567000000000001e-05 2024-09-16T20:48:19.621 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.621 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:19.621 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.621 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:985 3.19 3:9eda7881:::rbd_data.110e1748a8d2.000000000000000b:4 [read 409600~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:19.621 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.241521+0000", 2024-09-16T20:48:19.621 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.3350354400000004, 2024-09-16T20:48:19.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.030053854000000001, 2024-09-16T20:48:19.621 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:19.622 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:19.622 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:19.622 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:19.622 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:19.622 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:19.622 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 985 2024-09-16T20:48:19.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.622 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:19.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.622 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:19.622 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241521+0000", 2024-09-16T20:48:19.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:19.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241521+0000", 2024-09-16T20:48:19.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:19.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241521+0000", 2024-09-16T20:48:19.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6300000000000001e-07 2024-09-16T20:48:19.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.624 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:19.624 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241523+0000", 2024-09-16T20:48:19.624 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8279999999999999e-06 2024-09-16T20:48:19.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.624 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.624 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:19.624 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241529+0000", 2024-09-16T20:48:19.624 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.054e-06 2024-09-16T20:48:19.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.624 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.624 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:19.624 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241534+0000", 2024-09-16T20:48:19.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8609999999999997e-06 2024-09-16T20:48:19.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.625 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:19.625 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.271466+0000", 2024-09-16T20:48:19.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.029931777999999999 2024-09-16T20:48:19.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.625 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:19.625 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.271519+0000", 2024-09-16T20:48:19.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2947e-05 2024-09-16T20:48:19.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.626 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:19.626 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.271575+0000", 2024-09-16T20:48:19.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6023000000000002e-05 2024-09-16T20:48:19.626 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.626 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:19.626 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.626 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:986 3.19 3:9eda7881:::rbd_data.110e1748a8d2.000000000000000b:4 [read 524288~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:19.626 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.241540+0000", 2024-09-16T20:48:19.626 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.3350162860000001, 2024-09-16T20:48:19.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.030156059999999998, 2024-09-16T20:48:19.626 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:19.627 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:19.627 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:19.627 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:19.627 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:19.627 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:19.627 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 986 2024-09-16T20:48:19.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.627 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:19.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.627 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:19.627 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241540+0000", 2024-09-16T20:48:19.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:19.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241540+0000", 2024-09-16T20:48:19.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:19.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241541+0000", 2024-09-16T20:48:19.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.72e-07 2024-09-16T20:48:19.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:19.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241542+0000", 2024-09-16T20:48:19.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.699e-06 2024-09-16T20:48:19.629 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.629 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.629 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:19.629 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241548+0000", 2024-09-16T20:48:19.629 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8889999999999999e-06 2024-09-16T20:48:19.629 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.629 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.629 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:19.629 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241553+0000", 2024-09-16T20:48:19.629 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6519999999999997e-06 2024-09-16T20:48:19.629 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.629 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.629 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:19.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.271599+0000", 2024-09-16T20:48:19.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.030046568999999999 2024-09-16T20:48:19.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:19.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.271643+0000", 2024-09-16T20:48:19.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3169e-05 2024-09-16T20:48:19.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:19.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.271696+0000", 2024-09-16T20:48:19.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3709999999999999e-05 2024-09-16T20:48:19.630 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.631 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:19.631 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.631 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.631 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:987 3.19 3:9eda7881:::rbd_data.110e1748a8d2.000000000000000b:4 [read 622592~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:19.631 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.241559+0000", 2024-09-16T20:48:19.631 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.3349974439999999, 2024-09-16T20:48:19.631 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.030270335999999998, 2024-09-16T20:48:19.631 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:19.631 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:19.631 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:19.631 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:19.631 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:19.632 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:19.632 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 987 2024-09-16T20:48:19.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.632 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:19.632 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.632 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:19.632 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241559+0000", 2024-09-16T20:48:19.632 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.632 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.632 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:19.632 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241559+0000", 2024-09-16T20:48:19.632 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.633 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.633 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:19.633 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241559+0000", 2024-09-16T20:48:19.633 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3299999999999998e-07 2024-09-16T20:48:19.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.633 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.633 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:19.633 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241561+0000", 2024-09-16T20:48:19.633 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9400000000000001e-06 2024-09-16T20:48:19.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.633 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.633 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:19.633 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241568+0000", 2024-09-16T20:48:19.634 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2269999999999998e-06 2024-09-16T20:48:19.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.634 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:19.634 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241572+0000", 2024-09-16T20:48:19.634 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6630000000000001e-06 2024-09-16T20:48:19.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.634 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:19.634 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.271721+0000", 2024-09-16T20:48:19.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.030148878 2024-09-16T20:48:19.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.635 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:19.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.271772+0000", 2024-09-16T20:48:19.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0899999999999997e-05 2024-09-16T20:48:19.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.636 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:19.636 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.271829+0000", 2024-09-16T20:48:19.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7395000000000002e-05 2024-09-16T20:48:19.636 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.636 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:19.636 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.636 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:988 3.19 3:9eda7881:::rbd_data.110e1748a8d2.000000000000000b:4 [read 737280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:19.637 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.241721+0000", 2024-09-16T20:48:19.637 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.334835558, 2024-09-16T20:48:19.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.030229721000000001, 2024-09-16T20:48:19.637 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:19.637 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:19.637 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:19.637 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:19.637 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:19.637 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:19.637 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 988 2024-09-16T20:48:19.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.637 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:19.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.638 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:19.638 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241721+0000", 2024-09-16T20:48:19.638 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.638 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.638 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:19.638 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241721+0000", 2024-09-16T20:48:19.638 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.638 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.638 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:19.638 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241722+0000", 2024-09-16T20:48:19.638 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.06e-07 2024-09-16T20:48:19.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.639 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:19.639 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241732+0000", 2024-09-16T20:48:19.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0696000000000001e-05 2024-09-16T20:48:19.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.639 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:19.639 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241743+0000", 2024-09-16T20:48:19.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0556e-05 2024-09-16T20:48:19.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.639 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:19.640 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241749+0000", 2024-09-16T20:48:19.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.511e-06 2024-09-16T20:48:19.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.640 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:19.640 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.271854+0000", 2024-09-16T20:48:19.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.030104820000000001 2024-09-16T20:48:19.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.640 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:19.640 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.271897+0000", 2024-09-16T20:48:19.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2672000000000001e-05 2024-09-16T20:48:19.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:19.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.271951+0000", 2024-09-16T20:48:19.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3860000000000003e-05 2024-09-16T20:48:19.641 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.641 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:19.641 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.641 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:989 3.19 3:9eda7881:::rbd_data.110e1748a8d2.000000000000000b:4 [read 851968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:19.641 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.241760+0000", 2024-09-16T20:48:19.641 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.3347967509999998, 2024-09-16T20:48:19.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.030321378999999999, 2024-09-16T20:48:19.642 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:19.642 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:19.642 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:19.642 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:19.642 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:19.642 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:19.642 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 989 2024-09-16T20:48:19.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.642 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:19.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.642 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:19.642 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241760+0000", 2024-09-16T20:48:19.642 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.643 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:19.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241760+0000", 2024-09-16T20:48:19.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.643 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:19.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241760+0000", 2024-09-16T20:48:19.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4000000000000002e-07 2024-09-16T20:48:19.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.643 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:19.644 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241763+0000", 2024-09-16T20:48:19.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3250000000000002e-06 2024-09-16T20:48:19.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.644 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:19.644 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241769+0000", 2024-09-16T20:48:19.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3940000000000001e-06 2024-09-16T20:48:19.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.644 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:19.644 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241774+0000", 2024-09-16T20:48:19.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7849999999999999e-06 2024-09-16T20:48:19.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:19.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.271974+0000", 2024-09-16T20:48:19.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.030200569 2024-09-16T20:48:19.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:19.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.272026+0000", 2024-09-16T20:48:19.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1996999999999997e-05 2024-09-16T20:48:19.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:19.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.272081+0000", 2024-09-16T20:48:19.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4768999999999997e-05 2024-09-16T20:48:19.646 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.646 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:19.646 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.646 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:990 3.19 3:9eda7881:::rbd_data.110e1748a8d2.000000000000000b:4 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:19.646 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.241781+0000", 2024-09-16T20:48:19.646 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.334775509, 2024-09-16T20:48:19.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.030420295, 2024-09-16T20:48:19.647 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:19.647 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:19.647 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:19.647 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:19.647 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:19.647 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:19.647 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 990 2024-09-16T20:48:19.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.647 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:19.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.647 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:19.647 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241781+0000", 2024-09-16T20:48:19.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:19.648 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241781+0000", 2024-09-16T20:48:19.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:19.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241781+0000", 2024-09-16T20:48:19.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4700000000000002e-07 2024-09-16T20:48:19.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:19.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241783+0000", 2024-09-16T20:48:19.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.99e-06 2024-09-16T20:48:19.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.650 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:19.650 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241789+0000", 2024-09-16T20:48:19.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6910000000000002e-06 2024-09-16T20:48:19.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.650 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:19.650 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241802+0000", 2024-09-16T20:48:19.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3081000000000001e-05 2024-09-16T20:48:19.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.650 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:19.650 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.272104+0000", 2024-09-16T20:48:19.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.030302086999999998 2024-09-16T20:48:19.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.651 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:19.651 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.272148+0000", 2024-09-16T20:48:19.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3693999999999999e-05 2024-09-16T20:48:19.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.651 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:19.651 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.272201+0000", 2024-09-16T20:48:19.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3405000000000001e-05 2024-09-16T20:48:19.651 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.651 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:19.652 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.652 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:991 3.19 3:9eda7881:::rbd_data.110e1748a8d2.000000000000000b:4 [read 1064960~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:19.652 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.241889+0000", 2024-09-16T20:48:19.652 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.3346674429999998, 2024-09-16T20:48:19.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.030449282000000001, 2024-09-16T20:48:19.652 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:19.652 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:19.652 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:19.652 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:19.652 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:19.652 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:19.653 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 991 2024-09-16T20:48:19.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.653 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:19.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:19.653 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241889+0000", 2024-09-16T20:48:19.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:19.653 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241889+0000", 2024-09-16T20:48:19.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:19.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241890+0000", 2024-09-16T20:48:19.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9500000000000002e-07 2024-09-16T20:48:19.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:19.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241893+0000", 2024-09-16T20:48:19.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2169999999999999e-06 2024-09-16T20:48:19.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:19.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241903+0000", 2024-09-16T20:48:19.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6879999999999994e-06 2024-09-16T20:48:19.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.655 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.655 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:19.655 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241909+0000", 2024-09-16T20:48:19.655 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1079999999999998e-06 2024-09-16T20:48:19.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.655 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.655 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:19.655 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.272226+0000", 2024-09-16T20:48:19.655 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.030317647 2024-09-16T20:48:19.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.655 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.655 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:19.655 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.272279+0000", 2024-09-16T20:48:19.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3023999999999999e-05 2024-09-16T20:48:19.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.656 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:19.656 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.272338+0000", 2024-09-16T20:48:19.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9002999999999997e-05 2024-09-16T20:48:19.656 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.656 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:19.656 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.656 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:992 3.19 3:9eda7881:::rbd_data.110e1748a8d2.000000000000000b:4 [read 1179648~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:19.656 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.241918+0000", 2024-09-16T20:48:19.656 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.33463829, 2024-09-16T20:48:19.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.030545129000000001, 2024-09-16T20:48:19.657 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:19.657 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:19.657 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:19.657 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:19.657 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:19.657 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:19.657 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 992 2024-09-16T20:48:19.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.657 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:19.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.657 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:19.657 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241918+0000", 2024-09-16T20:48:19.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.658 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:19.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241918+0000", 2024-09-16T20:48:19.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.658 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:19.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241919+0000", 2024-09-16T20:48:19.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.58e-07 2024-09-16T20:48:19.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.658 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:19.659 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241921+0000", 2024-09-16T20:48:19.659 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.119e-06 2024-09-16T20:48:19.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.659 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.659 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:19.659 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241927+0000", 2024-09-16T20:48:19.659 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.472e-06 2024-09-16T20:48:19.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.659 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.659 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:19.659 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241932+0000", 2024-09-16T20:48:19.659 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8930000000000004e-06 2024-09-16T20:48:19.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.659 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.660 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:19.660 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.272364+0000", 2024-09-16T20:48:19.660 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.030431395999999999 2024-09-16T20:48:19.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.660 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:19.660 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.272409+0000", 2024-09-16T20:48:19.660 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5491999999999997e-05 2024-09-16T20:48:19.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.660 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:19.660 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.272463+0000", 2024-09-16T20:48:19.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4299000000000003e-05 2024-09-16T20:48:19.661 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.661 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:19.661 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.661 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:993 3.19 3:9eda7881:::rbd_data.110e1748a8d2.000000000000000b:4 [read 1277952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:19.661 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.241940+0000", 2024-09-16T20:48:19.661 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.3346164119999999, 2024-09-16T20:48:19.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0306495, 2024-09-16T20:48:19.661 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:19.661 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:19.661 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:19.662 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:19.662 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:19.662 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:19.662 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 993 2024-09-16T20:48:19.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.662 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:19.662 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.662 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:19.662 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241940+0000", 2024-09-16T20:48:19.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.662 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.662 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:19.662 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241940+0000", 2024-09-16T20:48:19.663 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.663 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:19.663 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241940+0000", 2024-09-16T20:48:19.663 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4000000000000002e-07 2024-09-16T20:48:19.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.663 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:19.663 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241950+0000", 2024-09-16T20:48:19.663 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9210000000000004e-06 2024-09-16T20:48:19.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.664 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:19.664 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241967+0000", 2024-09-16T20:48:19.664 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6124999999999999e-05 2024-09-16T20:48:19.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.664 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.664 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:19.664 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241972+0000", 2024-09-16T20:48:19.664 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9150000000000001e-06 2024-09-16T20:48:19.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.664 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.664 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:19.664 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.272486+0000", 2024-09-16T20:48:19.664 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.030513506999999999 2024-09-16T20:48:19.665 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.665 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.665 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:19.665 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.272533+0000", 2024-09-16T20:48:19.665 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6897000000000002e-05 2024-09-16T20:48:19.665 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.665 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.665 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:19.665 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.272590+0000", 2024-09-16T20:48:19.665 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6694999999999998e-05 2024-09-16T20:48:19.665 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.665 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:19.665 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.665 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.666 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:994 3.19 3:9eda7881:::rbd_data.110e1748a8d2.000000000000000b:4 [read 1392640~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:19.666 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.241982+0000", 2024-09-16T20:48:19.666 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.3345742449999998, 2024-09-16T20:48:19.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.030729910999999999, 2024-09-16T20:48:19.666 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:19.666 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:19.666 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:19.666 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:19.666 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:19.666 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:19.666 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 994 2024-09-16T20:48:19.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.666 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:19.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.667 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:19.667 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241982+0000", 2024-09-16T20:48:19.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.667 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:19.667 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241982+0000", 2024-09-16T20:48:19.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.667 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:19.667 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241983+0000", 2024-09-16T20:48:19.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2e-07 2024-09-16T20:48:19.668 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.668 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.668 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:19.668 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241985+0000", 2024-09-16T20:48:19.668 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5919999999999999e-06 2024-09-16T20:48:19.668 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.668 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.668 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:19.668 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241993+0000", 2024-09-16T20:48:19.668 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.418e-06 2024-09-16T20:48:19.668 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.668 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.668 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:19.668 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.242006+0000", 2024-09-16T20:48:19.669 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3283e-05 2024-09-16T20:48:19.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.669 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.669 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:19.669 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.272613+0000", 2024-09-16T20:48:19.669 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.030606694 2024-09-16T20:48:19.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.669 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.669 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:19.669 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.272657+0000", 2024-09-16T20:48:19.669 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4161e-05 2024-09-16T20:48:19.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.669 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.669 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:19.670 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.272712+0000", 2024-09-16T20:48:19.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5242999999999999e-05 2024-09-16T20:48:19.670 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.670 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:19.670 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.670 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:995 3.19 3:9eda7881:::rbd_data.110e1748a8d2.000000000000000b:4 [read 1507328~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:19.670 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.242014+0000", 2024-09-16T20:48:19.670 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.334542624, 2024-09-16T20:48:19.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.030826637, 2024-09-16T20:48:19.670 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:19.671 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:19.671 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:19.671 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:19.671 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:19.671 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:19.671 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 995 2024-09-16T20:48:19.672 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.672 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:19.672 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.672 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:19.672 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.242014+0000", 2024-09-16T20:48:19.672 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.673 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.673 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.673 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:19.673 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.242014+0000", 2024-09-16T20:48:19.673 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:19.673 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.673 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.673 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:19.673 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.242014+0000", 2024-09-16T20:48:19.673 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9100000000000004e-07 2024-09-16T20:48:19.673 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.673 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.673 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:19.673 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.242021+0000", 2024-09-16T20:48:19.674 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.832e-06 2024-09-16T20:48:19.674 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.674 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.674 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:19.674 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.242029+0000", 2024-09-16T20:48:19.674 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4420000000000001e-06 2024-09-16T20:48:19.674 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.674 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.674 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:19.674 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.242034+0000", 2024-09-16T20:48:19.674 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3589999999999999e-06 2024-09-16T20:48:19.674 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.674 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.674 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:19.675 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.272733+0000", 2024-09-16T20:48:19.675 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.030698791 2024-09-16T20:48:19.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.675 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.675 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:19.675 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.272779+0000", 2024-09-16T20:48:19.675 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5881000000000003e-05 2024-09-16T20:48:19.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:19.675 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:19.675 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:19.675 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.272840+0000", 2024-09-16T20:48:19.675 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1841000000000006e-05 2024-09-16T20:48:19.675 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.675 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:19.676 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.676 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:19.676 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:19.676 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:19.677 INFO:teuthology.orchestra.run.smithi139.stdout:622 read 0x3c76200 thru 0x3c845f5 (0xe3f6 bytes) 2024-09-16T20:48:19.677 INFO:teuthology.orchestra.run.smithi139.stdout:627 read 0x55f2600 thru 0x56022c6 (0xfcc7 bytes) 2024-09-16T20:48:19.677 INFO:teuthology.orchestra.run.smithi139.stdout:628 write 0x145b400 thru 0x145f5f5 (0x41f6 bytes) 2024-09-16T20:48:19.678 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:48:19.678 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:19.675+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:19.708 INFO:teuthology.orchestra.run.smithi139.stdout:629 write 0xcf7d000 thru 0xcf80b3f (0x3b40 bytes) 2024-09-16T20:48:19.728 INFO:teuthology.orchestra.run.smithi139.stdout:630 write 0xde97e00 thru 0xde9becc (0x40cd bytes) 2024-09-16T20:48:19.729 INFO:teuthology.orchestra.run.smithi139.stdout:631 write 0x235c00 thru 0x23ef21 (0x9322 bytes) 2024-09-16T20:48:19.730 INFO:teuthology.orchestra.run.smithi139.stdout:632 punch from 0x30c6e00 to 0x30d5c00, (0xee00 bytes) 2024-09-16T20:48:19.741 INFO:teuthology.orchestra.run.smithi139.stdout:633 punch from 0xda13c00 to 0xda14400, (0x800 bytes) 2024-09-16T20:48:19.741 INFO:teuthology.orchestra.run.smithi139.stdout:634 read 0x58c1200 thru 0x58c7d0a (0x6b0b bytes) 2024-09-16T20:48:19.745 INFO:teuthology.orchestra.run.smithi139.stdout:636 read 0x21f4000 thru 0x2203842 (0xf843 bytes) 2024-09-16T20:48:19.751 INFO:teuthology.orchestra.run.smithi139.stdout:637 write 0x170a600 thru 0x170e1fd (0x3bfe bytes) 2024-09-16T20:48:19.752 INFO:teuthology.orchestra.run.smithi139.stdout:639 write 0x4a9ec00 thru 0x4aa3a02 (0x4e03 bytes) 2024-09-16T20:48:19.753 INFO:teuthology.orchestra.run.smithi139.stdout:640 punch from 0xd0a8000 to 0xd0b5600, (0xd600 bytes) 2024-09-16T20:48:19.777 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:19.776+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:19.787 INFO:teuthology.orchestra.run.smithi139.stdout:643 punch from 0xa303e00 to 0xa30c000, (0x8200 bytes) 2024-09-16T20:48:19.787 INFO:teuthology.orchestra.run.smithi139.stdout:644 write 0x2130c00 thru 0x2131b2c (0xf2d bytes) 2024-09-16T20:48:19.808 INFO:teuthology.orchestra.run.smithi139.stdout:645 write 0xcadf400 thru 0xcadfca1 (0x8a2 bytes) 2024-09-16T20:48:19.808 INFO:teuthology.orchestra.run.smithi139.stdout:647 read 0x6740800 thru 0x674b94a (0xb14b bytes) 2024-09-16T20:48:19.808 INFO:teuthology.orchestra.run.smithi139.stdout:648 write 0x3167200 thru 0x316e594 (0x7395 bytes) 2024-09-16T20:48:19.843 INFO:teuthology.orchestra.run.smithi139.stdout:651 write 0x6ff400 thru 0x708ed9 (0x9ada bytes) 2024-09-16T20:48:19.855 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T20:48:19.856 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:19.856 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:48:19.856 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:48:19.856 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:19.862 INFO:teuthology.orchestra.run.smithi139.stdout:652 write 0xd611200 thru 0xd619fd4 (0x8dd5 bytes) 2024-09-16T20:48:19.863 INFO:teuthology.orchestra.run.smithi139.stdout:653 write 0xdb11a00 thru 0xdb17489 (0x5a8a bytes) 2024-09-16T20:48:19.863 INFO:teuthology.orchestra.run.smithi139.stdout:655 punch from 0x6112800 to 0x6113800, (0x1000 bytes) 2024-09-16T20:48:19.865 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:48:19.900 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:19.899+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:19.907 INFO:teuthology.orchestra.run.smithi139.stdout:659 write 0x91f2200 thru 0x91f87fd (0x65fe bytes) 2024-09-16T20:48:19.908 INFO:teuthology.orchestra.run.smithi139.stdout:660 read 0x5124200 thru 0x5125394 (0x1195 bytes) 2024-09-16T20:48:19.927 INFO:teuthology.orchestra.run.smithi139.stdout:661 write 0x279d800 thru 0x27ac349 (0xeb4a bytes) 2024-09-16T20:48:19.941 INFO:teuthology.orchestra.run.smithi139.stdout:662 trunc from 0xde9c000 to 0x2c94a00 2024-09-16T20:48:19.980 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:19.978+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:20.035 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:48:20.035 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:48:20.045 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:20.045 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:48:20.045 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:48:20.045 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:48:20.045 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:20.059 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:48:20.060 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:48:20.061 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:48:20.077 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:48:20.078 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:48:20.080 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:20.079+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:20.100 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:48:20.101 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:48:20.120 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:48:20.120 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:48:20.137 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:48:20.138 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:48:20.142 INFO:teuthology.orchestra.run.smithi139.stdout:663 write 0xde12a00 thru 0xde19e03 (0x7404 bytes) 2024-09-16T20:48:20.149 INFO:teuthology.orchestra.run.smithi139.stdout:truncating to largest ever: 0xeaf9c00 2024-09-16T20:48:20.149 INFO:teuthology.orchestra.run.smithi139.stdout:664 trunc from 0xde1a000 to 0xeaf9c00 2024-09-16T20:48:20.168 INFO:teuthology.orchestra.run.smithi139.stdout:666 punch from 0xdbe9000 to 0xdbea000, (0x1000 bytes) 2024-09-16T20:48:20.175 INFO:teuthology.orchestra.run.smithi139.stdout:668 read 0x9455c00 thru 0x945f8ef (0x9cf0 bytes) 2024-09-16T20:48:20.175 INFO:teuthology.orchestra.run.smithi139.stdout:673 write 0x5a7f600 thru 0x5a8ae4e (0xb84f bytes) 2024-09-16T20:48:20.175 INFO:teuthology.orchestra.run.smithi139.stdout:674 read 0x81cde00 thru 0x81db82a (0xda2b bytes) 2024-09-16T20:48:20.176 INFO:teuthology.orchestra.run.smithi139.stdout:675 read 0x78f000 thru 0x79d2fa (0xe2fb bytes) 2024-09-16T20:48:20.177 INFO:teuthology.orchestra.run.smithi139.stdout:676 write 0x8172a00 thru 0x817c843 (0x9e44 bytes) 2024-09-16T20:48:20.178 INFO:teuthology.orchestra.run.smithi139.stdout:677 write 0x87ea00 thru 0x87ef1d (0x51e bytes) 2024-09-16T20:48:20.178 INFO:teuthology.orchestra.run.smithi139.stdout:678 read 0x272000 thru 0x275afa (0x3afb bytes) 2024-09-16T20:48:20.178 INFO:teuthology.orchestra.run.smithi139.stdout:679 write 0xd028800 thru 0xd03842c (0xfc2d bytes) 2024-09-16T20:48:20.178 INFO:teuthology.orchestra.run.smithi139.stdout:681 write 0xdf97c00 thru 0xdf99433 (0x1834 bytes) 2024-09-16T20:48:20.178 INFO:teuthology.orchestra.run.smithi139.stdout:685 read 0x16eae00 thru 0x16f6fc1 (0xc1c2 bytes) 2024-09-16T20:48:20.182 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:20.179+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:20.183 INFO:teuthology.orchestra.run.smithi139.stdout:686 write 0x98a4600 thru 0x98a86be (0x40bf bytes) 2024-09-16T20:48:20.183 INFO:teuthology.orchestra.run.smithi139.stdout:687 write 0x20ce600 thru 0x20d3fb6 (0x59b7 bytes) 2024-09-16T20:48:20.184 INFO:teuthology.orchestra.run.smithi139.stdout:688 write 0x3ccb800 thru 0x3cd047d (0x4c7e bytes) 2024-09-16T20:48:20.184 INFO:teuthology.orchestra.run.smithi139.stdout:689 write 0x83d8a00 thru 0x83e362e (0xac2f bytes) 2024-09-16T20:48:20.184 INFO:teuthology.orchestra.run.smithi139.stdout:691 read 0x54e9400 thru 0x54f316a (0x9d6b bytes) 2024-09-16T20:48:20.184 INFO:teuthology.orchestra.run.smithi139.stdout:692 read 0x4c57400 thru 0x4c5ba6b (0x466c bytes) 2024-09-16T20:48:20.185 INFO:teuthology.orchestra.run.smithi139.stdout:693 read 0x7886000 thru 0x788a669 (0x466a bytes) 2024-09-16T20:48:20.190 INFO:teuthology.orchestra.run.smithi139.stdout:694 trunc from 0xeaf9c00 to 0x81b9000 2024-09-16T20:48:20.192 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:20.192 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:20.204 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:48:20.282 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:20.281+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:20.291 INFO:teuthology.orchestra.run.smithi139.stdout:695 read 0x161fa00 thru 0x1623b32 (0x4133 bytes) 2024-09-16T20:48:20.299 INFO:teuthology.orchestra.run.smithi139.stdout:700 trunc from 0x81b9000 to 0x5a95800 2024-09-16T20:48:20.331 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:20.331 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:20.338 INFO:teuthology.orchestra.run.smithi139.stdout:701 punch from 0x416da00 to 0x4174c00, (0x7200 bytes) 2024-09-16T20:48:20.340 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:48:20.347 INFO:teuthology.orchestra.run.smithi139.stdout:702 read 0x44c1600 thru 0x44c7584 (0x5f85 bytes) 2024-09-16T20:48:20.347 INFO:teuthology.orchestra.run.smithi139.stdout:704 read 0x9f4800 thru 0x9f823c (0x3a3d bytes) 2024-09-16T20:48:20.354 INFO:teuthology.orchestra.run.smithi139.stdout:705 trunc from 0x5a95800 to 0x10f9a00 2024-09-16T20:48:20.384 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:20.383+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:20.469 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:20.469 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:20.478 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:48:20.483 INFO:teuthology.orchestra.run.smithi139.stdout:706 write 0xbf92400 thru 0xbf95224 (0x2e25 bytes) 2024-09-16T20:48:20.484 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:20.482+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:20.494 INFO:teuthology.orchestra.run.smithi139.stdout:707 trunc from 0xbf95400 to 0x6215200 2024-09-16T20:48:20.546 INFO:teuthology.orchestra.run.smithi139.stdout:708 write 0x6952200 thru 0x695d0c8 (0xaec9 bytes) 2024-09-16T20:48:20.554 INFO:teuthology.orchestra.run.smithi139.stdout:711 trunc from 0x695d200 to 0x4f8aa00 2024-09-16T20:48:20.584 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:20.583+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:20.613 INFO:teuthology.orchestra.run.smithi139.stdout:713 read 0x34bac00 thru 0x34bd0a2 (0x24a3 bytes) 2024-09-16T20:48:20.613 INFO:teuthology.orchestra.run.smithi139.stdout:715 write 0x4956200 thru 0x495aec8 (0x4cc9 bytes) 2024-09-16T20:48:20.613 INFO:teuthology.orchestra.run.smithi139.stdout:717 read 0x20dfa00 thru 0x20e99bb (0x9fbc bytes) 2024-09-16T20:48:20.613 INFO:teuthology.orchestra.run.smithi139.stdout:719 read 0x65ea00 thru 0x6608de (0x1edf bytes) 2024-09-16T20:48:20.613 INFO:teuthology.orchestra.run.smithi139.stdout:720 write 0x3484a00 thru 0x348f8e7 (0xaee8 bytes) 2024-09-16T20:48:20.619 INFO:teuthology.orchestra.run.smithi139.stdout:721 trunc from 0x4f8aa00 to 0x976ae00 2024-09-16T20:48:20.635 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:20.635 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:20.644 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:48:20.666 INFO:teuthology.orchestra.run.smithi139.stdout:722 trunc from 0x976ae00 to 0xb069c00 2024-09-16T20:48:20.676 INFO:teuthology.orchestra.run.smithi139.stdout:726 write 0x13aec00 thru 0x13b85c3 (0x99c4 bytes) 2024-09-16T20:48:20.676 INFO:teuthology.orchestra.run.smithi139.stdout:728 write 0x2464400 thru 0x2468a90 (0x4691 bytes) 2024-09-16T20:48:20.676 INFO:teuthology.orchestra.run.smithi139.stdout:729 punch from 0x8d26200 to 0x8d35000, (0xee00 bytes) 2024-09-16T20:48:20.680 INFO:teuthology.orchestra.run.smithi139.stdout:730 write 0x7e1400 thru 0x7effa1 (0xeba2 bytes) 2024-09-16T20:48:20.680 INFO:teuthology.orchestra.run.smithi139.stdout:731 read 0x87bd000 thru 0x87c9ff1 (0xcff2 bytes) 2024-09-16T20:48:20.680 INFO:teuthology.orchestra.run.smithi139.stdout:732 write 0x672e400 thru 0x6735d7f (0x7980 bytes) 2024-09-16T20:48:20.680 INFO:teuthology.orchestra.run.smithi139.stdout:733 read 0x8e3b600 thru 0x8e4b316 (0xfd17 bytes) 2024-09-16T20:48:20.680 INFO:teuthology.orchestra.run.smithi139.stdout:735 clone 6 order 22 su 2097152 sc 12 2024-09-16T20:48:20.688 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:20.687+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:20.757 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:20.757 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:20.766 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:48:20.787 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:20.785+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:20.887 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:20.886+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:20.913 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:20.913 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:20.922 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:48:20.988 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:20.986+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:21.028 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:21.028 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:21.037 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:48:21.121 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:21.120+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:21.184 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:21.184 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:21.189 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:21.188+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:21.193 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:48:21.290 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:21.289+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:21.340 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:21.340 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:21.348 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:48:21.391 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:21.390+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:21.455 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:21.455 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:21.463 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:48:21.504 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:21.503+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:21.566 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T20:48:21.582 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:21.583 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:21.592 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:48:21.592 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:21.591+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:21.690 INFO:teuthology.orchestra.run.smithi139.stdout:truncating image image_client.0-clone5 from 0xb069c00 (overlap 0x10f9a00) to 0xd94800 2024-09-16T20:48:21.694 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:21.692+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:21.718 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:21.718 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:21.732 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:48:21.794 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:21.793+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:21.848 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:21.848 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:21.858 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:48:21.898 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:21.897+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:21.996 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:21.995+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:22.006 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:22.006 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:22.015 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:48:22.097 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:22.096+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:22.114 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #4, image image_client.0-clone4 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent5 2024-09-16T20:48:22.148 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T20:48:22.167 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:22.167 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:22.174 INFO:teuthology.orchestra.run.smithi139.stdout:736 write 0x8a76200 thru 0x8a85cf6 (0xfaf7 bytes) 2024-09-16T20:48:22.175 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:48:22.179 INFO:teuthology.orchestra.run.smithi139.stdout:737 trunc from 0xb069c00 to 0x6f64600 2024-09-16T20:48:22.198 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:22.197+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:22.207 INFO:teuthology.orchestra.run.smithi139.stdout:740 write 0xc5200 thru 0xcf820 (0xa621 bytes) 2024-09-16T20:48:22.288 INFO:teuthology.orchestra.run.smithi139.stdout:742 write 0x9a1a400 thru 0x9a1e661 (0x4262 bytes) 2024-09-16T20:48:22.294 INFO:teuthology.orchestra.run.smithi139.stdout:743 read 0x5a1c400 thru 0x5a1e4fb (0x20fc bytes) 2024-09-16T20:48:22.294 INFO:teuthology.orchestra.run.smithi139.stdout:745 read 0x704e600 thru 0x7050c2c (0x262d bytes) 2024-09-16T20:48:22.294 INFO:teuthology.orchestra.run.smithi139.stdout:746 read 0x94b4200 thru 0x94b48f6 (0x6f7 bytes) 2024-09-16T20:48:22.299 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:22.298+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:22.301 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:48:22.301 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:48:22.308 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:22.308 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:22.310 INFO:teuthology.orchestra.run.smithi139.stdout:750 write 0xc16b400 thru 0xc17159a (0x619b bytes) 2024-09-16T20:48:22.312 INFO:teuthology.orchestra.run.smithi139.stdout:751 read 0xa0a0400 thru 0xa0a4b34 (0x4735 bytes) 2024-09-16T20:48:22.312 INFO:teuthology.orchestra.run.smithi139.stdout:752 write 0x1530600 thru 0x153fe56 (0xf857 bytes) 2024-09-16T20:48:22.313 INFO:teuthology.orchestra.run.smithi139.stdout:753 read 0x9e72e00 thru 0x9e74d91 (0x1f92 bytes) 2024-09-16T20:48:22.313 INFO:teuthology.orchestra.run.smithi139.stdout:754 write 0x228bc00 thru 0x229528b (0x968c bytes) 2024-09-16T20:48:22.313 INFO:teuthology.orchestra.run.smithi139.stdout:755 read 0xab2f200 thru 0xab38e4e (0x9c4f bytes) 2024-09-16T20:48:22.317 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:48:22.329 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:48:22.329 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:48:22.360 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:48:22.361 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:48:22.389 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:48:22.389 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:48:22.400 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:22.399+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:22.407 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:48:22.408 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:48:22.413 INFO:teuthology.orchestra.run.smithi139.stdout:756 write 0xe0fda00 thru 0xe10bdb7 (0xe3b8 bytes) 2024-09-16T20:48:22.413 INFO:teuthology.orchestra.run.smithi139.stdout:758 read 0x1edc600 thru 0x1ee9736 (0xd137 bytes) 2024-09-16T20:48:22.413 INFO:teuthology.orchestra.run.smithi139.stdout:760 punch from 0x7549600 to 0x754c600, (0x3000 bytes) 2024-09-16T20:48:22.413 INFO:teuthology.orchestra.run.smithi139.stdout:761 punch from 0x573e400 to 0x5745400, (0x7000 bytes) 2024-09-16T20:48:22.414 INFO:teuthology.orchestra.run.smithi139.stdout:762 punch from 0xb46ca00 to 0xb476200, (0x9800 bytes) 2024-09-16T20:48:22.414 INFO:teuthology.orchestra.run.smithi139.stdout:763 write 0x2a5b000 thru 0x2a68b91 (0xdb92 bytes) 2024-09-16T20:48:22.414 INFO:teuthology.orchestra.run.smithi139.stdout:766 write 0xd2f6c00 thru 0xd2fbe84 (0x5285 bytes) 2024-09-16T20:48:22.414 INFO:teuthology.orchestra.run.smithi139.stdout:768 write 0x2511400 thru 0x251a983 (0x9584 bytes) 2024-09-16T20:48:22.415 INFO:teuthology.orchestra.run.smithi139.stdout:769 write 0xade6400 thru 0xade88e2 (0x24e3 bytes) 2024-09-16T20:48:22.415 INFO:teuthology.orchestra.run.smithi139.stdout:770 write 0xa960800 thru 0xa965094 (0x4895 bytes) 2024-09-16T20:48:22.415 INFO:teuthology.orchestra.run.smithi139.stdout:771 read 0x9413800 thru 0x941c2f0 (0x8af1 bytes) 2024-09-16T20:48:22.415 INFO:teuthology.orchestra.run.smithi139.stdout:774 write 0xa4c0c00 thru 0xa4c3ade (0x2edf bytes) 2024-09-16T20:48:22.415 INFO:teuthology.orchestra.run.smithi139.stdout:776 read 0x95f000 thru 0x96753a (0x853b bytes) 2024-09-16T20:48:22.428 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:48:22.428 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:48:22.468 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:22.468 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:48:22.468 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:48:22.468 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:22.476 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:48:22.520 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:22.519+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:22.583 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:22.583 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:48:22.584 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:48:22.584 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:48:22.584 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:22.592 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:48:22.602 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:22.601+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:22.613 INFO:teuthology.orchestra.run.smithi029.stderr:nodeep-scrub is set 2024-09-16T20:48:22.621 INFO:teuthology.orchestra.run.smithi139.stdout:778 punch from 0x75f1400 to 0x7600600, (0xf200 bytes) 2024-09-16T20:48:22.631 INFO:teuthology.orchestra.run.smithi139.stdout:780 write 0xbae8400 thru 0xbaeb4ec (0x30ed bytes) 2024-09-16T20:48:22.631 INFO:teuthology.orchestra.run.smithi139.stdout:781 write 0xdfc400 thru 0xe0aee7 (0xeae8 bytes) 2024-09-16T20:48:22.673 INFO:teuthology.orchestra.run.smithi139.stdout:782 punch from 0xc02ce00 to 0xc03b200, (0xe400 bytes) 2024-09-16T20:48:22.682 INFO:teuthology.orchestra.run.smithi139.stdout:783 write 0xdad2e00 thru 0xdadb990 (0x8b91 bytes) 2024-09-16T20:48:22.682 INFO:teuthology.orchestra.run.smithi139.stdout:784 read 0xda9c400 thru 0xdaa15d8 (0x51d9 bytes) 2024-09-16T20:48:22.694 INFO:teuthology.orchestra.run.smithi139.stdout:786 trunc from 0xe10be00 to 0x31c8800 2024-09-16T20:48:22.703 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:22.702+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:22.773 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:22.774 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:48:22.774 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:48:22.774 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:48:22.774 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.774 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:780 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 933888~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:22.774 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.239489+0000", 2024-09-16T20:48:22.774 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.5322043460000003, 2024-09-16T20:48:22.774 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00741699, 2024-09-16T20:48:22.774 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:22.774 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:22.774 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:22.774 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:22.775 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:22.775 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:22.775 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 780 2024-09-16T20:48:22.775 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.775 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:22.775 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.775 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:22.775 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239489+0000", 2024-09-16T20:48:22.776 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:22.776 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.776 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.776 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:22.776 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239489+0000", 2024-09-16T20:48:22.776 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:22.776 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.776 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.776 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:22.776 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239490+0000", 2024-09-16T20:48:22.776 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8599999999999998e-07 2024-09-16T20:48:22.777 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.777 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.777 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:22.777 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239492+0000", 2024-09-16T20:48:22.777 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1380000000000001e-06 2024-09-16T20:48:22.777 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.777 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.777 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:22.777 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239499+0000", 2024-09-16T20:48:22.777 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9530000000000002e-06 2024-09-16T20:48:22.777 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.777 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.778 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:22.778 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239504+0000", 2024-09-16T20:48:22.778 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0690000000000001e-06 2024-09-16T20:48:22.778 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.778 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.778 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:22.778 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246714+0000", 2024-09-16T20:48:22.778 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.007209844 2024-09-16T20:48:22.778 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.778 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.778 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:22.778 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246804+0000", 2024-09-16T20:48:22.778 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0109000000000001e-05 2024-09-16T20:48:22.779 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.779 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.779 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:22.779 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246906+0000", 2024-09-16T20:48:22.779 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000102391 2024-09-16T20:48:22.779 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:22.779 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:22.779 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:22.779 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.779 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.779 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:781 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1048576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:22.779 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.239892+0000", 2024-09-16T20:48:22.780 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.5318019759999997, 2024-09-16T20:48:22.780 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0072223799999999996, 2024-09-16T20:48:22.780 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:22.780 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:22.780 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:22.780 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:22.780 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:22.780 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:22.780 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 781 2024-09-16T20:48:22.780 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.780 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:22.780 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.781 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:22.781 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239892+0000", 2024-09-16T20:48:22.781 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:22.781 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.781 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.781 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:22.781 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239892+0000", 2024-09-16T20:48:22.781 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:22.781 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.781 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.781 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:22.781 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239893+0000", 2024-09-16T20:48:22.781 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1399999999999995e-07 2024-09-16T20:48:22.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.782 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.782 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:22.782 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239897+0000", 2024-09-16T20:48:22.782 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4859999999999999e-06 2024-09-16T20:48:22.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.782 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.782 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:22.782 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239907+0000", 2024-09-16T20:48:22.782 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4539999999999996e-06 2024-09-16T20:48:22.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.782 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.783 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:22.783 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239913+0000", 2024-09-16T20:48:22.783 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7880000000000001e-06 2024-09-16T20:48:22.783 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.783 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.783 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:22.783 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246937+0000", 2024-09-16T20:48:22.783 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0070239999999999999 2024-09-16T20:48:22.783 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.783 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.783 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:22.783 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247013+0000", 2024-09-16T20:48:22.784 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5894000000000002e-05 2024-09-16T20:48:22.784 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.784 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.784 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:22.784 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247114+0000", 2024-09-16T20:48:22.784 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000100844 2024-09-16T20:48:22.784 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:22.784 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:22.784 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:22.784 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.785 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.785 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:782 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1146880~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:22.785 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.239925+0000", 2024-09-16T20:48:22.785 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.531769003, 2024-09-16T20:48:22.785 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0074161840000000001, 2024-09-16T20:48:22.785 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:22.785 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:22.785 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:22.785 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:22.785 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:22.785 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:22.785 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 782 2024-09-16T20:48:22.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.786 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:22.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.786 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:22.786 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239925+0000", 2024-09-16T20:48:22.786 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:22.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.786 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:22.786 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239925+0000", 2024-09-16T20:48:22.786 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:22.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.787 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.787 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:22.787 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239925+0000", 2024-09-16T20:48:22.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7199999999999999e-07 2024-09-16T20:48:22.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.787 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.787 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:22.787 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239928+0000", 2024-09-16T20:48:22.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5349999999999999e-06 2024-09-16T20:48:22.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.787 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.787 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:22.787 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239935+0000", 2024-09-16T20:48:22.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7680000000000003e-06 2024-09-16T20:48:22.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.788 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.788 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:22.788 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239940+0000", 2024-09-16T20:48:22.788 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.152e-06 2024-09-16T20:48:22.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.788 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.788 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:22.788 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247143+0000", 2024-09-16T20:48:22.788 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0072037309999999997 2024-09-16T20:48:22.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.788 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.789 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:22.789 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247230+0000", 2024-09-16T20:48:22.789 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6598e-05 2024-09-16T20:48:22.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.789 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.789 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:22.789 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247341+0000", 2024-09-16T20:48:22.789 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011092800000000001 2024-09-16T20:48:22.789 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:22.789 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:22.789 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:22.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.789 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.789 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:783 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1261568~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:22.790 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.239947+0000", 2024-09-16T20:48:22.790 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.5317466809999996, 2024-09-16T20:48:22.790 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0076044789999999999, 2024-09-16T20:48:22.790 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:22.790 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:22.790 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:22.790 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:22.790 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:22.790 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:22.791 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 783 2024-09-16T20:48:22.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.791 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:22.791 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.791 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:22.791 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239947+0000", 2024-09-16T20:48:22.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:22.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.791 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.791 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:22.791 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239947+0000", 2024-09-16T20:48:22.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:22.792 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.792 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.792 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:22.792 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239947+0000", 2024-09-16T20:48:22.792 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9400000000000001e-07 2024-09-16T20:48:22.792 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.792 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.792 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:22.792 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239950+0000", 2024-09-16T20:48:22.792 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2350000000000002e-06 2024-09-16T20:48:22.792 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.792 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.793 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:22.793 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239956+0000", 2024-09-16T20:48:22.793 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3400000000000003e-06 2024-09-16T20:48:22.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.793 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.793 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:22.793 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239961+0000", 2024-09-16T20:48:22.793 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.04e-06 2024-09-16T20:48:22.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.793 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.793 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:22.793 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247371+0000", 2024-09-16T20:48:22.793 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0074096810000000004 2024-09-16T20:48:22.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.794 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:22.794 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247451+0000", 2024-09-16T20:48:22.794 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9845999999999994e-05 2024-09-16T20:48:22.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.794 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:22.794 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247552+0000", 2024-09-16T20:48:22.794 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000100943 2024-09-16T20:48:22.794 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:22.794 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:22.795 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:22.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.795 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.795 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:784 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1376256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:22.795 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.239997+0000", 2024-09-16T20:48:22.795 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.5316970249999997, 2024-09-16T20:48:22.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.007775372, 2024-09-16T20:48:22.796 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:22.796 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:22.796 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:22.796 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:22.796 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:22.796 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:22.796 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 784 2024-09-16T20:48:22.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.796 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:22.796 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.796 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:22.796 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239997+0000", 2024-09-16T20:48:22.797 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:22.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.797 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.797 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:22.797 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239997+0000", 2024-09-16T20:48:22.797 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:22.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.797 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.797 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:22.797 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239997+0000", 2024-09-16T20:48:22.797 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1799999999999995e-07 2024-09-16T20:48:22.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.797 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.797 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:22.798 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240001+0000", 2024-09-16T20:48:22.798 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6009999999999999e-06 2024-09-16T20:48:22.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.798 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.798 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:22.798 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240009+0000", 2024-09-16T20:48:22.798 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5809999999999999e-06 2024-09-16T20:48:22.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.798 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.798 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:22.798 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240015+0000", 2024-09-16T20:48:22.799 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0100000000000001e-06 2024-09-16T20:48:22.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.799 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.799 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:22.799 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247583+0000", 2024-09-16T20:48:22.799 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0075685730000000003 2024-09-16T20:48:22.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.799 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.799 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:22.799 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247672+0000", 2024-09-16T20:48:22.799 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8480000000000007e-05 2024-09-16T20:48:22.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.800 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:22.800 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247772+0000", 2024-09-16T20:48:22.800 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000100509 2024-09-16T20:48:22.800 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:22.800 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:22.800 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:22.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.801 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.801 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:785 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1474560~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:22.801 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240024+0000", 2024-09-16T20:48:22.801 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.5316694389999999, 2024-09-16T20:48:22.801 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.008096776, 2024-09-16T20:48:22.801 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:22.801 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:22.801 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:22.801 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:22.801 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:22.802 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:22.802 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 785 2024-09-16T20:48:22.802 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.802 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:22.802 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.802 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:22.802 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240024+0000", 2024-09-16T20:48:22.802 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:22.802 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.802 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.802 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:22.803 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240024+0000", 2024-09-16T20:48:22.803 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:22.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.803 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.803 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:22.803 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240025+0000", 2024-09-16T20:48:22.803 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5000000000000003e-07 2024-09-16T20:48:22.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.803 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.803 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:22.803 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240027+0000", 2024-09-16T20:48:22.803 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2900000000000001e-06 2024-09-16T20:48:22.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.804 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:22.804 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240036+0000", 2024-09-16T20:48:22.804 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4710000000000007e-06 2024-09-16T20:48:22.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.804 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:22.804 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240041+0000", 2024-09-16T20:48:22.804 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4689999999999996e-06 2024-09-16T20:48:22.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.804 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:22.805 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247821+0000", 2024-09-16T20:48:22.805 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0077797919999999998 2024-09-16T20:48:22.805 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.805 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.805 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:22.805 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247921+0000", 2024-09-16T20:48:22.805 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9814000000000004e-05 2024-09-16T20:48:22.805 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.805 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.805 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:22.805 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248121+0000", 2024-09-16T20:48:22.805 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00020039 2024-09-16T20:48:22.805 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:22.805 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:22.806 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:22.806 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.806 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.806 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:940 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 81920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:22.806 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240569+0000", 2024-09-16T20:48:22.806 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.5311251810000002, 2024-09-16T20:48:22.806 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0078289759999999996, 2024-09-16T20:48:22.806 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:22.806 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:22.806 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:22.806 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:22.806 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:22.807 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:22.807 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 940 2024-09-16T20:48:22.807 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.807 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:22.807 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.807 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:22.807 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240569+0000", 2024-09-16T20:48:22.807 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:22.807 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.807 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.807 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:22.807 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240569+0000", 2024-09-16T20:48:22.807 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:22.807 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.808 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.808 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:22.808 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240569+0000", 2024-09-16T20:48:22.808 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2600000000000001e-07 2024-09-16T20:48:22.808 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.808 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.808 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:22.808 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240575+0000", 2024-09-16T20:48:22.808 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1159999999999998e-06 2024-09-16T20:48:22.808 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.808 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.808 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:22.808 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240587+0000", 2024-09-16T20:48:22.809 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1915e-05 2024-09-16T20:48:22.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.809 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.809 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:22.809 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240594+0000", 2024-09-16T20:48:22.809 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5610000000000001e-06 2024-09-16T20:48:22.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.809 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.809 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:22.809 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248158+0000", 2024-09-16T20:48:22.809 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0075635370000000004 2024-09-16T20:48:22.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.809 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.809 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:22.810 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248278+0000", 2024-09-16T20:48:22.810 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012009400000000001 2024-09-16T20:48:22.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.810 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.810 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:22.810 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248398+0000", 2024-09-16T20:48:22.810 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000119827 2024-09-16T20:48:22.810 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:22.810 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:22.810 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:22.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.810 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.810 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:941 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 196608~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:22.810 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240608+0000", 2024-09-16T20:48:22.811 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.5310856040000003, 2024-09-16T20:48:22.811 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0080495519999999997, 2024-09-16T20:48:22.811 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:22.811 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:22.811 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:22.811 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:22.811 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:22.811 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:22.811 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 941 2024-09-16T20:48:22.811 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.811 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:22.811 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.811 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:22.811 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240608+0000", 2024-09-16T20:48:22.812 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:22.812 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.812 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.812 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:22.812 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240608+0000", 2024-09-16T20:48:22.812 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:22.812 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.812 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.812 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:22.812 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240609+0000", 2024-09-16T20:48:22.812 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7000000000000004e-07 2024-09-16T20:48:22.812 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.813 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.813 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:22.813 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240612+0000", 2024-09-16T20:48:22.813 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8310000000000001e-06 2024-09-16T20:48:22.813 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.813 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.813 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:22.813 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240619+0000", 2024-09-16T20:48:22.813 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8429999999999993e-06 2024-09-16T20:48:22.813 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.813 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.813 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:22.814 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240625+0000", 2024-09-16T20:48:22.814 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5659999999999997e-06 2024-09-16T20:48:22.814 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.814 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.814 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:22.814 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248456+0000", 2024-09-16T20:48:22.814 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0078313940000000002 2024-09-16T20:48:22.814 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.814 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.814 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:22.814 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248548+0000", 2024-09-16T20:48:22.814 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1324999999999997e-05 2024-09-16T20:48:22.814 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.814 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.815 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:22.815 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248658+0000", 2024-09-16T20:48:22.815 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000109923 2024-09-16T20:48:22.815 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:22.815 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:22.815 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:22.815 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.815 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.815 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:942 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 294912~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:22.815 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240675+0000", 2024-09-16T20:48:22.815 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.5310187969999998, 2024-09-16T20:48:22.815 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0082135699999999999, 2024-09-16T20:48:22.815 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:22.815 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:22.816 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:22.816 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:22.816 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:22.816 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:22.816 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 942 2024-09-16T20:48:22.816 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.816 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:22.816 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.816 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:22.816 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240675+0000", 2024-09-16T20:48:22.816 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:22.816 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.816 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.817 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:22.817 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240675+0000", 2024-09-16T20:48:22.817 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:22.817 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.817 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.817 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:22.817 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240676+0000", 2024-09-16T20:48:22.817 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.37e-07 2024-09-16T20:48:22.817 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.817 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.817 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:22.817 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240679+0000", 2024-09-16T20:48:22.817 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4060000000000001e-06 2024-09-16T20:48:22.818 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.818 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.818 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:22.818 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240687+0000", 2024-09-16T20:48:22.818 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4640000000000006e-06 2024-09-16T20:48:22.818 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.818 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.818 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:22.818 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240694+0000", 2024-09-16T20:48:22.818 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.246e-06 2024-09-16T20:48:22.818 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.818 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.818 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:22.818 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248693+0000", 2024-09-16T20:48:22.819 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0079997550000000008 2024-09-16T20:48:22.819 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.819 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.819 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:22.819 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248781+0000", 2024-09-16T20:48:22.819 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7509999999999994e-05 2024-09-16T20:48:22.819 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.819 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.819 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:22.819 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248889+0000", 2024-09-16T20:48:22.819 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000107552 2024-09-16T20:48:22.819 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:22.819 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:22.819 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:22.820 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.820 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.820 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:943 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 409600~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:22.820 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240704+0000", 2024-09-16T20:48:22.820 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.5309898010000005, 2024-09-16T20:48:22.820 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0083978209999999998, 2024-09-16T20:48:22.820 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:22.820 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:22.820 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:22.820 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:22.820 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:22.820 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:22.820 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 943 2024-09-16T20:48:22.821 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.821 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:22.821 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.821 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:22.821 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240704+0000", 2024-09-16T20:48:22.821 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:22.821 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.821 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.821 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:22.821 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240704+0000", 2024-09-16T20:48:22.821 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:22.821 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.821 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.822 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:22.822 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240704+0000", 2024-09-16T20:48:22.822 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7e-07 2024-09-16T20:48:22.822 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.822 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.822 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:22.822 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240707+0000", 2024-09-16T20:48:22.822 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5809999999999999e-06 2024-09-16T20:48:22.822 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.822 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.822 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:22.822 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240713+0000", 2024-09-16T20:48:22.822 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4949999999999996e-06 2024-09-16T20:48:22.823 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.823 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.823 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:22.823 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240718+0000", 2024-09-16T20:48:22.823 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8890000000000001e-06 2024-09-16T20:48:22.823 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.823 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.823 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:22.823 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248920+0000", 2024-09-16T20:48:22.823 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0082018679999999993 2024-09-16T20:48:22.823 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.823 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.823 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:22.824 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249007+0000", 2024-09-16T20:48:22.824 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6327000000000002e-05 2024-09-16T20:48:22.824 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.824 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.824 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:22.824 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249102+0000", 2024-09-16T20:48:22.824 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5191000000000004e-05 2024-09-16T20:48:22.824 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:22.824 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:22.824 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:22.824 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.824 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.824 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:944 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 524288~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:22.825 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240726+0000", 2024-09-16T20:48:22.825 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.5309679960000002, 2024-09-16T20:48:22.825 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0085999960000000004, 2024-09-16T20:48:22.825 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:22.825 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:22.825 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:22.825 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:22.825 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:22.825 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:22.825 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 944 2024-09-16T20:48:22.825 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.825 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:22.825 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.826 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:22.826 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240726+0000", 2024-09-16T20:48:22.826 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:22.826 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.826 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.826 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:22.826 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240726+0000", 2024-09-16T20:48:22.826 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:22.826 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.826 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.826 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:22.826 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240726+0000", 2024-09-16T20:48:22.826 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7099999999999997e-07 2024-09-16T20:48:22.827 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.827 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.827 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:22.827 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240728+0000", 2024-09-16T20:48:22.827 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.125e-06 2024-09-16T20:48:22.827 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.827 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.827 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:22.827 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240734+0000", 2024-09-16T20:48:22.827 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1890000000000004e-06 2024-09-16T20:48:22.827 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.827 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.827 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:22.828 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240739+0000", 2024-09-16T20:48:22.828 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0340000000000004e-06 2024-09-16T20:48:22.828 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.828 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.828 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:22.828 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249131+0000", 2024-09-16T20:48:22.828 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0083919579999999997 2024-09-16T20:48:22.828 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.828 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.828 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:22.828 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249217+0000", 2024-09-16T20:48:22.828 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5154999999999994e-05 2024-09-16T20:48:22.828 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.829 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.829 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:22.829 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249326+0000", 2024-09-16T20:48:22.829 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000109164 2024-09-16T20:48:22.829 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:22.829 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:22.829 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:22.829 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.829 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.829 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:945 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 622592~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:22.829 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240746+0000", 2024-09-16T20:48:22.829 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.5309478570000001, 2024-09-16T20:48:22.830 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0088083320000000003, 2024-09-16T20:48:22.830 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:22.830 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:22.830 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:22.830 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:22.830 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:22.830 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:22.830 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 945 2024-09-16T20:48:22.830 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.830 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:22.830 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.830 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:22.830 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240746+0000", 2024-09-16T20:48:22.831 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:22.831 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.831 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.831 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:22.831 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240746+0000", 2024-09-16T20:48:22.831 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:22.831 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.831 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.831 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:22.831 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240746+0000", 2024-09-16T20:48:22.831 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8500000000000002e-07 2024-09-16T20:48:22.831 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.831 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.832 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:22.832 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240748+0000", 2024-09-16T20:48:22.832 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8449999999999999e-06 2024-09-16T20:48:22.832 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.832 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.832 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:22.832 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240754+0000", 2024-09-16T20:48:22.832 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8830000000000003e-06 2024-09-16T20:48:22.832 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.832 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.832 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:22.832 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240759+0000", 2024-09-16T20:48:22.832 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0220000000000004e-06 2024-09-16T20:48:22.833 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.833 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.833 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:22.833 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249360+0000", 2024-09-16T20:48:22.833 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.008601437 2024-09-16T20:48:22.833 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.833 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.833 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:22.833 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249453+0000", 2024-09-16T20:48:22.833 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2743000000000003e-05 2024-09-16T20:48:22.833 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.833 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.834 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:22.834 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249554+0000", 2024-09-16T20:48:22.834 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101017 2024-09-16T20:48:22.834 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:22.834 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:22.834 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:22.834 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.834 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.834 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:946 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 737280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:22.834 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240766+0000", 2024-09-16T20:48:22.834 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.5309281539999997, 2024-09-16T20:48:22.834 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0089092070000000006, 2024-09-16T20:48:22.835 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:22.835 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:22.835 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:22.835 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:22.835 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:22.835 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:22.835 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 946 2024-09-16T20:48:22.835 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.835 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:22.836 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.836 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:22.836 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240766+0000", 2024-09-16T20:48:22.836 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:22.836 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.836 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.836 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:22.836 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240766+0000", 2024-09-16T20:48:22.836 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:22.836 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.836 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.836 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:22.836 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240766+0000", 2024-09-16T20:48:22.836 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0600000000000001e-07 2024-09-16T20:48:22.837 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.837 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.837 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:22.837 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240768+0000", 2024-09-16T20:48:22.837 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8959999999999999e-06 2024-09-16T20:48:22.837 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.837 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.837 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:22.837 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240774+0000", 2024-09-16T20:48:22.837 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0979999999999999e-06 2024-09-16T20:48:22.837 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.837 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.838 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:22.838 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240779+0000", 2024-09-16T20:48:22.838 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.933e-06 2024-09-16T20:48:22.838 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.838 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.838 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:22.838 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249572+0000", 2024-09-16T20:48:22.838 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0087926380000000002 2024-09-16T20:48:22.838 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.838 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.838 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:22.838 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249612+0000", 2024-09-16T20:48:22.838 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0859000000000003e-05 2024-09-16T20:48:22.838 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.839 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.839 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:22.839 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249675+0000", 2024-09-16T20:48:22.839 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2377000000000003e-05 2024-09-16T20:48:22.839 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:22.839 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:22.839 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:22.839 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.839 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.839 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:947 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 851968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:22.839 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240785+0000", 2024-09-16T20:48:22.839 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.5309085620000005, 2024-09-16T20:48:22.839 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0090960260000000001, 2024-09-16T20:48:22.839 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:22.840 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:22.840 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:22.840 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:22.840 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:22.840 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:22.840 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 947 2024-09-16T20:48:22.840 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.840 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:22.840 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.841 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:22.841 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240785+0000", 2024-09-16T20:48:22.841 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:22.841 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.841 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.841 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:22.841 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240785+0000", 2024-09-16T20:48:22.841 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:22.841 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.841 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.841 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:22.841 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240786+0000", 2024-09-16T20:48:22.842 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-16T20:48:22.842 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.842 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.842 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:22.842 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240787+0000", 2024-09-16T20:48:22.842 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.858e-06 2024-09-16T20:48:22.842 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.842 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.842 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:22.842 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240793+0000", 2024-09-16T20:48:22.843 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7760000000000003e-06 2024-09-16T20:48:22.843 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.843 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.843 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:22.843 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240799+0000", 2024-09-16T20:48:22.843 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1109999999999996e-06 2024-09-16T20:48:22.843 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.843 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.843 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:22.843 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249715+0000", 2024-09-16T20:48:22.844 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0089159789999999992 2024-09-16T20:48:22.844 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.844 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.844 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:22.844 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249766+0000", 2024-09-16T20:48:22.844 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0968999999999999e-05 2024-09-16T20:48:22.844 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.844 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.844 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:22.844 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249881+0000", 2024-09-16T20:48:22.844 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114963 2024-09-16T20:48:22.844 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:22.844 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:22.845 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:22.845 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.845 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.845 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:948 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:22.845 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240806+0000", 2024-09-16T20:48:22.845 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.5308873819999995, 2024-09-16T20:48:22.845 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0092300609999999995, 2024-09-16T20:48:22.845 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:22.845 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:22.845 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:22.845 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:22.845 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:22.845 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:22.845 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 948 2024-09-16T20:48:22.846 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.846 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:22.846 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.846 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:22.846 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240806+0000", 2024-09-16T20:48:22.846 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:22.846 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.846 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.846 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:22.846 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240806+0000", 2024-09-16T20:48:22.846 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:22.846 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.846 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.847 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:22.847 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240807+0000", 2024-09-16T20:48:22.847 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4499999999999998e-07 2024-09-16T20:48:22.847 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.847 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.847 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:22.847 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240809+0000", 2024-09-16T20:48:22.847 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.34e-06 2024-09-16T20:48:22.847 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.847 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.847 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:22.847 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240815+0000", 2024-09-16T20:48:22.847 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1449999999999996e-06 2024-09-16T20:48:22.847 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.848 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.848 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:22.848 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240821+0000", 2024-09-16T20:48:22.848 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0270000000000001e-06 2024-09-16T20:48:22.848 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.848 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.848 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:22.848 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249920+0000", 2024-09-16T20:48:22.848 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0090988460000000007 2024-09-16T20:48:22.848 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.848 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.848 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:22.848 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249966+0000", 2024-09-16T20:48:22.849 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5918999999999999e-05 2024-09-16T20:48:22.849 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.849 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.849 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:22.849 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250036+0000", 2024-09-16T20:48:22.849 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0438999999999994e-05 2024-09-16T20:48:22.849 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:22.849 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:22.849 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:22.849 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.849 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.849 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:949 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1064960~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:22.849 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240833+0000", 2024-09-16T20:48:22.850 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.5308607670000001, 2024-09-16T20:48:22.850 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0093580339999999994, 2024-09-16T20:48:22.850 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:22.850 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:22.850 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:22.850 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:22.850 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:22.850 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:22.850 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 949 2024-09-16T20:48:22.850 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.850 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:22.850 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.850 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:22.850 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240833+0000", 2024-09-16T20:48:22.851 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:22.851 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.851 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.851 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:22.851 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240833+0000", 2024-09-16T20:48:22.851 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:22.851 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.851 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.851 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:22.851 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240833+0000", 2024-09-16T20:48:22.851 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2899999999999999e-07 2024-09-16T20:48:22.851 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.851 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.851 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:22.852 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240836+0000", 2024-09-16T20:48:22.852 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2649999999999999e-06 2024-09-16T20:48:22.852 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.852 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.852 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:22.852 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240842+0000", 2024-09-16T20:48:22.852 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5579999999999997e-06 2024-09-16T20:48:22.852 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.852 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.852 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:22.852 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240847+0000", 2024-09-16T20:48:22.852 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2009999999999998e-06 2024-09-16T20:48:22.852 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.853 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.853 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:22.853 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250073+0000", 2024-09-16T20:48:22.853 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0092258710000000001 2024-09-16T20:48:22.853 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.853 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.853 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:22.853 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250141+0000", 2024-09-16T20:48:22.853 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7849999999999996e-05 2024-09-16T20:48:22.853 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.853 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.853 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:22.853 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250191+0000", 2024-09-16T20:48:22.853 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.986e-05 2024-09-16T20:48:22.854 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:22.854 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:22.854 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:22.854 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.854 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.854 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:950 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1179648~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:22.854 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240855+0000", 2024-09-16T20:48:22.854 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.5308385150000001, 2024-09-16T20:48:22.854 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0095855560000000003, 2024-09-16T20:48:22.854 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:22.854 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:22.854 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:22.854 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:22.854 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:22.855 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:22.855 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 950 2024-09-16T20:48:22.855 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.855 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:22.855 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.855 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:22.855 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240855+0000", 2024-09-16T20:48:22.855 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:22.855 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.855 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.856 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:22.856 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240855+0000", 2024-09-16T20:48:22.856 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:22.856 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.856 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.856 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:22.856 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240856+0000", 2024-09-16T20:48:22.856 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4999999999999998e-07 2024-09-16T20:48:22.856 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.856 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.856 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:22.856 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240858+0000", 2024-09-16T20:48:22.856 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.996e-06 2024-09-16T20:48:22.856 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.857 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.857 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:22.857 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240864+0000", 2024-09-16T20:48:22.857 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9359999999999996e-06 2024-09-16T20:48:22.857 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.857 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.857 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:22.857 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240868+0000", 2024-09-16T20:48:22.857 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7670000000000003e-06 2024-09-16T20:48:22.857 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.857 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.857 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:22.857 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250226+0000", 2024-09-16T20:48:22.858 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0093576100000000006 2024-09-16T20:48:22.858 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.858 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.858 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:22.858 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250289+0000", 2024-09-16T20:48:22.858 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2627999999999997e-05 2024-09-16T20:48:22.858 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.858 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.858 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:22.858 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250441+0000", 2024-09-16T20:48:22.858 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015216900000000001 2024-09-16T20:48:22.858 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:22.858 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:22.858 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:22.859 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.859 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.859 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:951 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1277952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:22.859 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.241044+0000", 2024-09-16T20:48:22.859 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.5306493269999999, 2024-09-16T20:48:22.859 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0096266589999999992, 2024-09-16T20:48:22.859 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:22.859 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:22.859 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:22.859 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:22.859 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:22.859 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:22.859 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 951 2024-09-16T20:48:22.860 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.860 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:22.860 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.860 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:22.860 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241044+0000", 2024-09-16T20:48:22.860 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:22.860 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.860 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.860 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:22.860 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241044+0000", 2024-09-16T20:48:22.860 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:22.860 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.860 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.860 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:22.861 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241045+0000", 2024-09-16T20:48:22.861 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5300000000000004e-07 2024-09-16T20:48:22.861 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.861 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.861 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:22.861 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241048+0000", 2024-09-16T20:48:22.861 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.303e-06 2024-09-16T20:48:22.861 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.861 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.861 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:22.861 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241057+0000", 2024-09-16T20:48:22.861 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5550000000000003e-06 2024-09-16T20:48:22.861 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.861 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.862 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:22.862 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241063+0000", 2024-09-16T20:48:22.862 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3470000000000004e-06 2024-09-16T20:48:22.862 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.862 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.862 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:22.862 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250477+0000", 2024-09-16T20:48:22.862 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0094136159999999997 2024-09-16T20:48:22.862 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.862 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.862 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:22.862 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250562+0000", 2024-09-16T20:48:22.863 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4709000000000005e-05 2024-09-16T20:48:22.863 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.863 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.863 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:22.863 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250671+0000", 2024-09-16T20:48:22.863 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010947599999999999 2024-09-16T20:48:22.863 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:22.863 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:22.863 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:22.863 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.864 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.864 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:952 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1392640~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:22.864 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.241130+0000", 2024-09-16T20:48:22.864 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.5305634609999998, 2024-09-16T20:48:22.864 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0098096290000000003, 2024-09-16T20:48:22.864 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:22.864 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:22.864 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:22.864 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:22.864 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:22.864 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:22.864 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 952 2024-09-16T20:48:22.864 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.865 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:22.865 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.865 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:22.865 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241130+0000", 2024-09-16T20:48:22.865 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:22.865 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.865 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.865 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:22.865 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241130+0000", 2024-09-16T20:48:22.865 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:22.866 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.866 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.866 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:22.866 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241131+0000", 2024-09-16T20:48:22.866 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.61e-07 2024-09-16T20:48:22.866 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.866 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.866 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:22.866 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241134+0000", 2024-09-16T20:48:22.866 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1080000000000001e-06 2024-09-16T20:48:22.866 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.866 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.866 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:22.866 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241142+0000", 2024-09-16T20:48:22.867 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9119999999999998e-06 2024-09-16T20:48:22.867 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.867 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.867 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:22.867 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241148+0000", 2024-09-16T20:48:22.867 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1410000000000002e-06 2024-09-16T20:48:22.867 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.867 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.867 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:22.867 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250731+0000", 2024-09-16T20:48:22.867 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0095833979999999999 2024-09-16T20:48:22.867 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.868 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.868 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:22.868 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250820+0000", 2024-09-16T20:48:22.868 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8997000000000002e-05 2024-09-16T20:48:22.868 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.868 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.868 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:22.868 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250940+0000", 2024-09-16T20:48:22.868 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000119412 2024-09-16T20:48:22.868 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:22.868 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:22.868 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:22.868 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.869 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.869 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:953 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1507328~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:22.869 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.241157+0000", 2024-09-16T20:48:22.869 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.5305365399999999, 2024-09-16T20:48:22.869 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.010039259, 2024-09-16T20:48:22.869 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:22.869 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:22.869 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:22.869 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:22.869 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:22.869 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:22.870 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 953 2024-09-16T20:48:22.870 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.870 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:22.870 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.870 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:22.870 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241157+0000", 2024-09-16T20:48:22.870 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:22.870 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.870 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.870 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:22.870 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241157+0000", 2024-09-16T20:48:22.870 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:22.871 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.871 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.871 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:22.871 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241158+0000", 2024-09-16T20:48:22.871 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.01e-07 2024-09-16T20:48:22.871 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.871 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.871 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:22.871 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241160+0000", 2024-09-16T20:48:22.871 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2879999999999999e-06 2024-09-16T20:48:22.871 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.872 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.872 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:22.872 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241166+0000", 2024-09-16T20:48:22.872 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.421e-06 2024-09-16T20:48:22.872 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.872 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.872 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:22.872 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241171+0000", 2024-09-16T20:48:22.872 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9840000000000001e-06 2024-09-16T20:48:22.872 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.872 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.872 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:22.873 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250988+0000", 2024-09-16T20:48:22.873 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0098171449999999993 2024-09-16T20:48:22.873 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.873 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.873 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:22.873 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.251085+0000", 2024-09-16T20:48:22.873 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6877000000000001e-05 2024-09-16T20:48:22.873 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:22.873 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:22.873 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:22.873 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.251196+0000", 2024-09-16T20:48:22.874 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000111143 2024-09-16T20:48:22.874 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:22.874 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:22.874 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:22.874 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:22.874 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:22.874 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:22.877 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:22.874+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:22.877 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:48:22.878 INFO:teuthology.orchestra.run.smithi139.stdout:789 read 0xac5e00 thru 0xad523d (0xf43e bytes) 2024-09-16T20:48:22.878 INFO:teuthology.orchestra.run.smithi139.stdout:793 write 0x46dcc00 thru 0x46ea838 (0xdc39 bytes) 2024-09-16T20:48:22.878 INFO:teuthology.orchestra.run.smithi139.stdout:794 punch from 0x1e87600 to 0x1e87a00, (0x400 bytes) 2024-09-16T20:48:22.878 INFO:teuthology.orchestra.run.smithi139.stdout:795 read 0x956400 thru 0x95fb53 (0x9754 bytes) 2024-09-16T20:48:22.878 INFO:teuthology.orchestra.run.smithi139.stdout:796 read 0x286a400 thru 0x286f212 (0x4e13 bytes) 2024-09-16T20:48:22.878 INFO:teuthology.orchestra.run.smithi139.stdout:798 trunc from 0x46eaa00 to 0xa516a00 2024-09-16T20:48:22.879 INFO:teuthology.orchestra.run.smithi139.stdout:799 write 0x54b6200 thru 0x54be722 (0x8523 bytes) 2024-09-16T20:48:22.879 INFO:teuthology.orchestra.run.smithi139.stdout:800 write 0x4be8a00 thru 0x4be95d3 (0xbd4 bytes) 2024-09-16T20:48:22.879 INFO:teuthology.orchestra.run.smithi139.stdout:803 trunc from 0xa516a00 to 0x9f17800 2024-09-16T20:48:22.879 INFO:teuthology.orchestra.run.smithi139.stdout:806 write 0x12a6000 thru 0x12b599d (0xf99e bytes) 2024-09-16T20:48:22.879 INFO:teuthology.orchestra.run.smithi139.stdout:810 write 0x1120800 thru 0x112f3ad (0xebae bytes) 2024-09-16T20:48:22.879 INFO:teuthology.orchestra.run.smithi139.stdout:811 trunc from 0x9f17800 to 0xa17c000 2024-09-16T20:48:22.976 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:22.975+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:22.989 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:22.989 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:48:22.989 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:22.989 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5484 3.f 3:f439ce52:::rbd_header.110ed9ca540:head [notify cookie 140455161379360 in=50b out=8b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:22.989 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.988418+0000", 2024-09-16T20:48:22.989 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.00031573400000000002, 2024-09-16T20:48:22.990 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00032403199999999998, 2024-09-16T20:48:22.990 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:22.990 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:22.990 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:22.990 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:22.990 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:22.990 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:22.990 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5484 2024-09-16T20:48:22.990 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:22.990 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:22.990 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:22.990 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:22.990 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.988418+0000", 2024-09-16T20:48:22.990 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:22.991 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:22.991 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:22.991 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:22.991 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.988418+0000", 2024-09-16T20:48:22.991 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:22.991 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:22.991 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:22.991 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:22.991 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.988419+0000", 2024-09-16T20:48:22.991 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3600000000000003e-07 2024-09-16T20:48:22.991 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:22.991 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:22.991 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:22.992 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.988422+0000", 2024-09-16T20:48:22.992 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5190000000000001e-06 2024-09-16T20:48:22.992 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:22.992 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:22.992 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:22.992 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.988430+0000", 2024-09-16T20:48:22.992 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0029999999999994e-06 2024-09-16T20:48:22.992 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:22.992 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:22.992 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:22.992 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.988434+0000", 2024-09-16T20:48:22.992 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5370000000000002e-06 2024-09-16T20:48:22.992 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:22.993 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:22.993 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:22.993 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.988553+0000", 2024-09-16T20:48:22.993 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000119171 2024-09-16T20:48:22.993 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:22.993 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:22.993 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:22.993 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.988629+0000", 2024-09-16T20:48:22.993 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5680999999999996e-05 2024-09-16T20:48:22.993 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:22.993 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:22.993 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:22.993 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.988742+0000", 2024-09-16T20:48:22.994 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000113485 2024-09-16T20:48:22.994 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:22.994 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:22.994 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:22.994 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:22.994 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:22.994 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5485 3.f 3:f439ce52:::rbd_header.110ed9ca540:head [call rbd.metadata_list in=33b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:22.994 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.988603+0000", 2024-09-16T20:48:22.994 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.00013107099999999999, 2024-09-16T20:48:22.994 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000188242, 2024-09-16T20:48:22.994 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:22.994 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:22.994 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:48:22.995 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:22.995 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:22.995 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:22.995 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5485 2024-09-16T20:48:22.995 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:22.995 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:22.995 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:22.995 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:22.995 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.988603+0000", 2024-09-16T20:48:22.995 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:22.995 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:22.995 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:22.995 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:22.996 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.988603+0000", 2024-09-16T20:48:22.996 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:22.996 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:22.996 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:22.996 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:22.996 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.988603+0000", 2024-09-16T20:48:22.996 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.82e-07 2024-09-16T20:48:22.996 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:22.996 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:22.996 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:22.996 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.988607+0000", 2024-09-16T20:48:22.996 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3289999999999998e-06 2024-09-16T20:48:22.997 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:22.997 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:22.997 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:22.997 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.988613+0000", 2024-09-16T20:48:22.997 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8649999999999998e-06 2024-09-16T20:48:22.997 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:22.997 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:22.997 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:22.997 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.988616+0000", 2024-09-16T20:48:22.997 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.546e-06 2024-09-16T20:48:22.997 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:22.997 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:22.998 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:22.998 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.988772+0000", 2024-09-16T20:48:22.998 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00015570800000000001 2024-09-16T20:48:22.998 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:22.998 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:22.998 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:22.998 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:22.998 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:22.998 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:2727 3.f 3.4a739c2f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:22.998 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.988807+0000", 2024-09-16T20:48:22.998 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4294967295.9999266, 2024-09-16T20:48:22.999 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1027e-05, 2024-09-16T20:48:22.999 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:22.999 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:22.999 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:48:22.999 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:22.999 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:48:22.999 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:48:22.999 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2727 2024-09-16T20:48:22.999 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:22.999 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:22.999 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:22.999 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:23.000 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.988807+0000", 2024-09-16T20:48:23.000 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.000 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.000 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.000 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:23.000 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.988807+0000", 2024-09-16T20:48:23.000 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.000 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.000 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.000 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:23.000 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.988808+0000", 2024-09-16T20:48:23.000 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.054e-06 2024-09-16T20:48:23.000 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.001 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.001 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:23.001 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.988814+0000", 2024-09-16T20:48:23.001 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2490000000000001e-06 2024-09-16T20:48:23.001 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.001 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.001 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:23.001 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.988824+0000", 2024-09-16T20:48:23.001 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0210999999999999e-05 2024-09-16T20:48:23.001 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.001 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.001 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:23.002 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.988831+0000", 2024-09-16T20:48:23.002 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7889999999999997e-06 2024-09-16T20:48:23.002 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.002 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:23.002 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.002 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.002 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.002 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5486 3.f 3.4a739c2f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:23.002 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.988820+0000", 2024-09-16T20:48:23.002 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4294967295.9999142, 2024-09-16T20:48:23.002 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6890999999999994e-05, 2024-09-16T20:48:23.002 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:23.002 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:23.003 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:48:23.003 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:23.003 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:23.003 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:23.003 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5486 2024-09-16T20:48:23.003 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.003 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:23.003 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.003 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:23.003 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.988820+0000", 2024-09-16T20:48:23.003 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.004 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.004 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.004 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:23.004 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.988820+0000", 2024-09-16T20:48:23.004 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.004 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.004 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.004 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:23.004 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.988820+0000", 2024-09-16T20:48:23.004 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4499999999999996e-07 2024-09-16T20:48:23.004 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.005 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.005 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:23.005 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.988824+0000", 2024-09-16T20:48:23.005 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2059999999999999e-06 2024-09-16T20:48:23.005 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.005 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.005 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:23.005 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.988835+0000", 2024-09-16T20:48:23.005 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1430999999999999e-05 2024-09-16T20:48:23.005 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.005 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.005 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:23.005 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.988839+0000", 2024-09-16T20:48:23.005 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5769999999999998e-06 2024-09-16T20:48:23.006 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.006 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:23.006 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.006 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.006 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.006 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:2728 3.f 3.4a739c2f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:23.006 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.988848+0000", 2024-09-16T20:48:23.006 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4294967295.9998865, 2024-09-16T20:48:23.006 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5211999999999998e-05, 2024-09-16T20:48:23.006 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:23.006 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:23.006 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:48:23.007 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:23.007 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:48:23.007 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:48:23.007 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2728 2024-09-16T20:48:23.007 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.007 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:23.007 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.007 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:23.007 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.988848+0000", 2024-09-16T20:48:23.007 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.007 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.007 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.007 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:23.007 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.988848+0000", 2024-09-16T20:48:23.008 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.008 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.008 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.008 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:23.008 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.988848+0000", 2024-09-16T20:48:23.008 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5099999999999999e-07 2024-09-16T20:48:23.008 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.008 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.008 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:23.008 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.988852+0000", 2024-09-16T20:48:23.008 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4130000000000002e-06 2024-09-16T20:48:23.008 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.008 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.008 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:23.009 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.988859+0000", 2024-09-16T20:48:23.009 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5089999999999996e-06 2024-09-16T20:48:23.009 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.009 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.009 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:23.009 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.988865+0000", 2024-09-16T20:48:23.009 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7150000000000003e-06 2024-09-16T20:48:23.009 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.009 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:23.009 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.009 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.009 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-16T20:48:23.009 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 5 2024-09-16T20:48:23.010 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:23.011 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:48:23.012 INFO:teuthology.orchestra.run.smithi139.stdout:812 write 0xe08d600 thru 0xe094ad1 (0x74d2 bytes) 2024-09-16T20:48:23.012 INFO:teuthology.orchestra.run.smithi139.stdout:815 punch from 0x9e2f000 to 0x9e31c00, (0x2c00 bytes) 2024-09-16T20:48:23.012 INFO:teuthology.orchestra.run.smithi139.stdout:816 write 0x677e200 thru 0x6782708 (0x4509 bytes) 2024-09-16T20:48:23.012 INFO:teuthology.orchestra.run.smithi139.stdout:817 read 0xaa66c00 thru 0xaa6d90b (0x6d0c bytes) 2024-09-16T20:48:23.012 INFO:teuthology.orchestra.run.smithi139.stdout:818 write 0x9cb2a00 thru 0x9cb6c7c (0x427d bytes) 2024-09-16T20:48:23.012 INFO:teuthology.orchestra.run.smithi139.stdout:820 punch from 0x801b200 to 0x8024a00, (0x9800 bytes) 2024-09-16T20:48:23.013 INFO:teuthology.orchestra.run.smithi139.stdout:821 punch from 0xa570e00 to 0xa579e00, (0x9000 bytes) 2024-09-16T20:48:23.014 INFO:teuthology.orchestra.run.smithi139.stdout:822 trunc from 0xe094c00 to 0x1fc7a00 2024-09-16T20:48:23.077 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:23.076+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:23.121 INFO:teuthology.orchestra.run.smithi139.stdout:823 punch from 0x1070a00 to 0x107aa00, (0xa000 bytes) 2024-09-16T20:48:23.273 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:23.273 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:48:23.273 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:48:23.273 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:48:23.273 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:23.274 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:48:23.276 INFO:teuthology.orchestra.run.smithi139.stdout:825 read 0x1078800 thru 0x1080c9f (0x84a0 bytes) 2024-09-16T20:48:23.276 INFO:teuthology.orchestra.run.smithi139.stdout:826 write 0x6deb600 thru 0x6df91f0 (0xdbf1 bytes) 2024-09-16T20:48:23.276 INFO:teuthology.orchestra.run.smithi139.stdout:829 punch from 0xbe0600 to 0xbec600, (0xc000 bytes) 2024-09-16T20:48:23.276 INFO:teuthology.orchestra.run.smithi139.stdout:831 read 0x2e02000 thru 0x2e0f98e (0xd98f bytes) 2024-09-16T20:48:23.276 INFO:teuthology.orchestra.run.smithi139.stdout:832 read 0x3ddd000 thru 0x3dec2ef (0xf2f0 bytes) 2024-09-16T20:48:23.276 INFO:teuthology.orchestra.run.smithi139.stdout:834 trunc from 0x6df9200 to 0xa851600 2024-09-16T20:48:23.276 INFO:teuthology.orchestra.run.smithi139.stdout:835 write 0x3e3a00 thru 0x3e9e2b (0x642c bytes) 2024-09-16T20:48:23.276 INFO:teuthology.orchestra.run.smithi139.stdout:836 read 0x3dece00 thru 0x3dfbd89 (0xef8a bytes) 2024-09-16T20:48:23.276 INFO:teuthology.orchestra.run.smithi139.stdout:837 read 0x9ecd000 thru 0x9ecf12e (0x212f bytes) 2024-09-16T20:48:23.276 INFO:teuthology.orchestra.run.smithi139.stdout:838 trunc from 0xa851600 to 0x6ff7e00 2024-09-16T20:48:23.276 INFO:teuthology.orchestra.run.smithi139.stdout:839 trunc from 0x6ff7e00 to 0xdb01400 2024-09-16T20:48:23.277 INFO:teuthology.orchestra.run.smithi139.stdout:840 write 0x6c57000 thru 0x6c65535 (0xe536 bytes) 2024-09-16T20:48:23.277 INFO:teuthology.orchestra.run.smithi139.stdout:842 write 0x3ff0e00 thru 0x3ff1c97 (0xe98 bytes) 2024-09-16T20:48:23.277 INFO:teuthology.orchestra.run.smithi139.stdout:843 punch from 0xa789400 to 0xa798000, (0xec00 bytes) 2024-09-16T20:48:23.277 INFO:teuthology.orchestra.run.smithi139.stdout:846 write 0x14ab600 thru 0x14ad58c (0x1f8d bytes) 2024-09-16T20:48:23.277 INFO:teuthology.orchestra.run.smithi139.stdout:848 write 0x5077000 thru 0x508656e (0xf56f bytes) 2024-09-16T20:48:23.277 INFO:teuthology.orchestra.run.smithi139.stdout:849 punch from 0x33a0600 to 0x33a3800, (0x3200 bytes) 2024-09-16T20:48:23.277 INFO:teuthology.orchestra.run.smithi139.stdout:850 read 0x867ba00 thru 0x867dd41 (0x2342 bytes) 2024-09-16T20:48:23.277 INFO:teuthology.orchestra.run.smithi139.stdout:852 write 0xda32e00 thru 0xda33abf (0xcc0 bytes) 2024-09-16T20:48:23.277 INFO:teuthology.orchestra.run.smithi139.stdout:853 read 0x5a10200 thru 0x5a1ca79 (0xc87a bytes) 2024-09-16T20:48:23.277 INFO:teuthology.orchestra.run.smithi139.stdout:855 read 0x388e000 thru 0x3894918 (0x6919 bytes) 2024-09-16T20:48:23.277 INFO:teuthology.orchestra.run.smithi139.stdout:857 trunc from 0xdb01400 to 0x402dc00 2024-09-16T20:48:23.278 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:23.270+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:23.290 INFO:teuthology.orchestra.run.smithi139.stdout:858 write 0x3a3ba00 thru 0x3a46f7e (0xb57f bytes) 2024-09-16T20:48:23.316 INFO:teuthology.orchestra.run.smithi139.stdout:861 write 0xc9d1200 thru 0xc9dead3 (0xd8d4 bytes) 2024-09-16T20:48:23.317 INFO:teuthology.orchestra.run.smithi139.stdout:862 punch from 0x7c69600 to 0x7c6d400, (0x3e00 bytes) 2024-09-16T20:48:23.318 INFO:teuthology.orchestra.run.smithi139.stdout:870 punch from 0x9687400 to 0x9696c00, (0xf800 bytes) 2024-09-16T20:48:23.318 INFO:teuthology.orchestra.run.smithi139.stdout:871 read 0xb854600 thru 0xb858ac1 (0x44c2 bytes) 2024-09-16T20:48:23.319 INFO:teuthology.orchestra.run.smithi139.stdout:872 punch from 0x1920800 to 0x1923c00, (0x3400 bytes) 2024-09-16T20:48:23.319 INFO:teuthology.orchestra.run.smithi139.stdout:873 write 0x941e00 thru 0x94dad0 (0xbcd1 bytes) 2024-09-16T20:48:23.320 INFO:teuthology.orchestra.run.smithi139.stdout:874 punch from 0x300e200 to 0x3014000, (0x5e00 bytes) 2024-09-16T20:48:23.320 INFO:teuthology.orchestra.run.smithi139.stdout:877 punch from 0x1db8200 to 0x1dc6c00, (0xea00 bytes) 2024-09-16T20:48:23.324 INFO:teuthology.orchestra.run.smithi139.stdout:879 punch from 0x1d4c200 to 0x1d58200, (0xc000 bytes) 2024-09-16T20:48:23.324 INFO:teuthology.orchestra.run.smithi139.stdout:882 punch from 0xb390e00 to 0xb39f200, (0xe400 bytes) 2024-09-16T20:48:23.332 INFO:teuthology.orchestra.run.smithi139.stdout:883 trunc from 0xc9dec00 to 0x64cce00 2024-09-16T20:48:23.372 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:23.371+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:23.405 INFO:teuthology.orchestra.run.smithi139.stdout:885 read 0x52acc00 thru 0x52b2973 (0x5d74 bytes) 2024-09-16T20:48:23.414 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:23.414 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:48:23.415 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:48:23.415 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:48:23.415 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.415 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2167 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 907264~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:23.415 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335615+0000", 2024-09-16T20:48:23.415 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 1.077060683, 2024-09-16T20:48:23.415 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046952761000000003, 2024-09-16T20:48:23.415 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:23.415 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:23.415 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:23.415 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:23.415 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:23.416 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:23.416 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2167 2024-09-16T20:48:23.416 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.416 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:23.416 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.416 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:23.416 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335615+0000", 2024-09-16T20:48:23.416 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.416 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.416 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.416 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:23.416 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335615+0000", 2024-09-16T20:48:23.416 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.416 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.417 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.417 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:23.417 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335616+0000", 2024-09-16T20:48:23.417 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2300000000000001e-07 2024-09-16T20:48:23.417 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.417 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.417 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:23.417 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335618+0000", 2024-09-16T20:48:23.417 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3269999999999999e-06 2024-09-16T20:48:23.417 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.417 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.417 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:23.417 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335625+0000", 2024-09-16T20:48:23.418 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1930000000000004e-06 2024-09-16T20:48:23.418 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.418 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.418 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:23.418 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335630+0000", 2024-09-16T20:48:23.418 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2460000000000003e-06 2024-09-16T20:48:23.418 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.418 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.418 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:23.418 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.382377+0000", 2024-09-16T20:48:23.418 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046746256 2024-09-16T20:48:23.418 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.418 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.419 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:23.419 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.382474+0000", 2024-09-16T20:48:23.419 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.7388999999999999e-05 2024-09-16T20:48:23.419 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.419 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.419 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:23.419 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.382568+0000", 2024-09-16T20:48:23.419 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.3826999999999995e-05 2024-09-16T20:48:23.419 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.419 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:23.419 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.419 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.419 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.419 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2170 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 915456~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:23.420 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335637+0000", 2024-09-16T20:48:23.420 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 1.0770385339999999, 2024-09-16T20:48:23.420 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047182843000000002, 2024-09-16T20:48:23.420 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:23.420 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:23.420 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:23.420 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:23.420 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:23.420 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:23.420 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2170 2024-09-16T20:48:23.420 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.420 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:23.421 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.421 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:23.421 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335637+0000", 2024-09-16T20:48:23.421 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.421 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.421 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.421 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:23.421 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335637+0000", 2024-09-16T20:48:23.421 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.421 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.421 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.421 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:23.421 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335638+0000", 2024-09-16T20:48:23.422 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6699999999999999e-07 2024-09-16T20:48:23.422 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.422 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.422 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:23.422 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335639+0000", 2024-09-16T20:48:23.422 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8339999999999999e-06 2024-09-16T20:48:23.422 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.422 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.422 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:23.422 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335646+0000", 2024-09-16T20:48:23.422 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6599999999999998e-06 2024-09-16T20:48:23.422 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.422 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.422 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:23.422 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335651+0000", 2024-09-16T20:48:23.423 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0289999999999996e-06 2024-09-16T20:48:23.423 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.423 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.423 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:23.423 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.382616+0000", 2024-09-16T20:48:23.423 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046965066 2024-09-16T20:48:23.423 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.423 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.423 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:23.423 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.382708+0000", 2024-09-16T20:48:23.423 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2177999999999997e-05 2024-09-16T20:48:23.423 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.423 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.423 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:23.424 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.382820+0000", 2024-09-16T20:48:23.424 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000111709 2024-09-16T20:48:23.424 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.424 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:23.424 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.424 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.424 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.424 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2172 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 923648~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:23.424 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335658+0000", 2024-09-16T20:48:23.424 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 1.077017627, 2024-09-16T20:48:23.424 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047394824000000002, 2024-09-16T20:48:23.424 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:23.424 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:23.425 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:23.425 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:23.425 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:23.425 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:23.425 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2172 2024-09-16T20:48:23.425 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.425 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:23.425 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.425 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:23.425 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335658+0000", 2024-09-16T20:48:23.425 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.426 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.426 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.426 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:23.426 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335658+0000", 2024-09-16T20:48:23.426 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.426 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.426 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.426 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:23.426 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335658+0000", 2024-09-16T20:48:23.426 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8599999999999999e-07 2024-09-16T20:48:23.426 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.426 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.426 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:23.426 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335660+0000", 2024-09-16T20:48:23.427 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.95e-06 2024-09-16T20:48:23.427 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.427 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.427 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:23.427 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335667+0000", 2024-09-16T20:48:23.427 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.815e-06 2024-09-16T20:48:23.427 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.427 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.427 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:23.427 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335672+0000", 2024-09-16T20:48:23.427 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8040000000000001e-06 2024-09-16T20:48:23.427 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.427 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.427 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:23.428 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.382864+0000", 2024-09-16T20:48:23.428 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047192125000000001 2024-09-16T20:48:23.428 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.428 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.428 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:23.428 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.382950+0000", 2024-09-16T20:48:23.428 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5594999999999996e-05 2024-09-16T20:48:23.428 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.428 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.428 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:23.428 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383053+0000", 2024-09-16T20:48:23.428 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000103149 2024-09-16T20:48:23.428 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.428 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:23.429 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.429 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.429 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.429 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2174 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 931840~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:23.429 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335679+0000", 2024-09-16T20:48:23.429 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 1.0769967540000001, 2024-09-16T20:48:23.429 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047619511000000003, 2024-09-16T20:48:23.429 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:23.429 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:23.429 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:23.429 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:23.429 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:23.429 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:23.429 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2174 2024-09-16T20:48:23.430 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.430 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:23.430 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.430 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:23.430 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335679+0000", 2024-09-16T20:48:23.430 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.430 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.430 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.430 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:23.430 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335679+0000", 2024-09-16T20:48:23.430 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.430 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.430 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.430 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:23.431 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335679+0000", 2024-09-16T20:48:23.431 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7800000000000002e-07 2024-09-16T20:48:23.431 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.431 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.431 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:23.431 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335682+0000", 2024-09-16T20:48:23.431 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9490000000000001e-06 2024-09-16T20:48:23.431 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.431 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.431 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:23.431 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335689+0000", 2024-09-16T20:48:23.431 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8220000000000001e-06 2024-09-16T20:48:23.431 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.431 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.432 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:23.432 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335695+0000", 2024-09-16T20:48:23.432 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0569999999999998e-06 2024-09-16T20:48:23.432 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.432 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.432 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:23.432 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383103+0000", 2024-09-16T20:48:23.432 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047408219000000001 2024-09-16T20:48:23.432 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.432 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.432 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:23.432 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383189+0000", 2024-09-16T20:48:23.432 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6007000000000005e-05 2024-09-16T20:48:23.432 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.433 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.433 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:23.433 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383298+0000", 2024-09-16T20:48:23.433 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000109079 2024-09-16T20:48:23.433 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.433 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:23.433 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.433 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.433 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.433 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2176 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 940032~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:23.433 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335702+0000", 2024-09-16T20:48:23.433 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 1.076973526, 2024-09-16T20:48:23.433 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047804450999999998, 2024-09-16T20:48:23.433 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:23.434 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:23.434 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:23.434 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:23.434 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:23.434 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:23.434 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2176 2024-09-16T20:48:23.434 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.434 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:23.434 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.434 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:23.434 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335702+0000", 2024-09-16T20:48:23.434 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.434 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.434 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.435 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:23.435 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335702+0000", 2024-09-16T20:48:23.435 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.435 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.435 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.435 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:23.435 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335703+0000", 2024-09-16T20:48:23.435 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.34e-07 2024-09-16T20:48:23.435 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.435 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.435 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:23.435 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335704+0000", 2024-09-16T20:48:23.435 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.917e-06 2024-09-16T20:48:23.436 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.436 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.436 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:23.436 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335711+0000", 2024-09-16T20:48:23.436 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4400000000000002e-06 2024-09-16T20:48:23.436 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.436 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.436 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:23.436 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335717+0000", 2024-09-16T20:48:23.436 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6679999999999998e-06 2024-09-16T20:48:23.436 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.437 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.437 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:23.437 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383335+0000", 2024-09-16T20:48:23.437 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047618754999999999 2024-09-16T20:48:23.437 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.437 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.437 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:23.437 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383405+0000", 2024-09-16T20:48:23.438 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9286e-05 2024-09-16T20:48:23.438 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.438 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.438 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:23.438 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383507+0000", 2024-09-16T20:48:23.438 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101951 2024-09-16T20:48:23.438 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.438 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:23.438 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.438 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.438 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.439 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2179 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 948224~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:23.439 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335723+0000", 2024-09-16T20:48:23.439 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 1.0769524319999999, 2024-09-16T20:48:23.439 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048007181000000003, 2024-09-16T20:48:23.439 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:23.439 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:23.439 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:23.439 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:23.439 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:23.439 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:23.439 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2179 2024-09-16T20:48:23.440 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.440 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:23.440 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.440 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:23.440 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335723+0000", 2024-09-16T20:48:23.440 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.440 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.440 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.440 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:23.440 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335723+0000", 2024-09-16T20:48:23.440 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.440 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.440 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.441 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:23.441 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335724+0000", 2024-09-16T20:48:23.441 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.58e-07 2024-09-16T20:48:23.441 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.441 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.441 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:23.441 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335726+0000", 2024-09-16T20:48:23.441 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9120000000000001e-06 2024-09-16T20:48:23.441 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.441 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.441 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:23.441 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335732+0000", 2024-09-16T20:48:23.441 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6080000000000001e-06 2024-09-16T20:48:23.442 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.442 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.442 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:23.442 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335737+0000", 2024-09-16T20:48:23.442 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.22e-06 2024-09-16T20:48:23.442 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.442 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.442 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:23.442 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383556+0000", 2024-09-16T20:48:23.442 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047819002999999999 2024-09-16T20:48:23.442 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.442 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.442 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:23.443 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383635+0000", 2024-09-16T20:48:23.443 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9060999999999994e-05 2024-09-16T20:48:23.443 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.443 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.443 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:23.443 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383730+0000", 2024-09-16T20:48:23.443 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4919000000000004e-05 2024-09-16T20:48:23.443 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.443 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:23.443 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.443 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.443 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.443 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2180 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 956416~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:23.444 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335744+0000", 2024-09-16T20:48:23.444 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 1.076931984, 2024-09-16T20:48:23.444 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048255147999999998, 2024-09-16T20:48:23.444 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:23.444 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:23.444 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:23.444 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:23.444 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:23.445 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:23.445 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2180 2024-09-16T20:48:23.445 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.445 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:23.445 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.445 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:23.445 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335744+0000", 2024-09-16T20:48:23.445 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.445 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.445 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.445 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:23.445 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335744+0000", 2024-09-16T20:48:23.446 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.446 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.446 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.446 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:23.446 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335744+0000", 2024-09-16T20:48:23.446 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9000000000000002e-07 2024-09-16T20:48:23.446 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.446 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.446 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:23.446 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335746+0000", 2024-09-16T20:48:23.446 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7570000000000001e-06 2024-09-16T20:48:23.446 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.446 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.447 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:23.447 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335752+0000", 2024-09-16T20:48:23.447 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0079999999999999e-06 2024-09-16T20:48:23.447 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.447 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.447 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:23.447 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335757+0000", 2024-09-16T20:48:23.447 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1649999999999997e-06 2024-09-16T20:48:23.447 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.447 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.447 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:23.447 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383803+0000", 2024-09-16T20:48:23.447 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048046091999999999 2024-09-16T20:48:23.448 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.448 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.448 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:23.448 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383889+0000", 2024-09-16T20:48:23.448 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6048e-05 2024-09-16T20:48:23.448 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.448 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.448 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:23.448 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383999+0000", 2024-09-16T20:48:23.448 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000109688 2024-09-16T20:48:23.448 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.448 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:23.448 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.449 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.449 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.449 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2182 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 964608~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:23.449 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335773+0000", 2024-09-16T20:48:23.449 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 1.076902177, 2024-09-16T20:48:23.449 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048467116999999997, 2024-09-16T20:48:23.449 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:23.449 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:23.449 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:23.449 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:23.449 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:23.449 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:23.449 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2182 2024-09-16T20:48:23.450 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.450 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:23.450 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.450 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:23.450 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335773+0000", 2024-09-16T20:48:23.450 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.450 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.450 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.450 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:23.450 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335773+0000", 2024-09-16T20:48:23.450 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.450 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.450 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.450 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:23.451 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335774+0000", 2024-09-16T20:48:23.451 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3799999999999997e-07 2024-09-16T20:48:23.451 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.451 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.451 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:23.451 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335776+0000", 2024-09-16T20:48:23.451 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.227e-06 2024-09-16T20:48:23.451 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.451 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.451 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:23.451 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335790+0000", 2024-09-16T20:48:23.451 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4119e-05 2024-09-16T20:48:23.451 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.451 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.451 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:23.452 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335796+0000", 2024-09-16T20:48:23.452 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.694e-06 2024-09-16T20:48:23.452 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.452 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.452 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:23.452 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384058+0000", 2024-09-16T20:48:23.452 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048261983000000001 2024-09-16T20:48:23.452 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.452 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.452 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:23.452 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384148+0000", 2024-09-16T20:48:23.452 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9501999999999998e-05 2024-09-16T20:48:23.452 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.453 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.453 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:23.453 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384241+0000", 2024-09-16T20:48:23.453 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.3053999999999994e-05 2024-09-16T20:48:23.453 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.453 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:23.453 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.453 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.453 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.453 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2185 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 972800~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:23.453 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335804+0000", 2024-09-16T20:48:23.453 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 1.076871527, 2024-09-16T20:48:23.453 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048570818000000002, 2024-09-16T20:48:23.453 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:23.454 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:23.454 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:23.454 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:23.454 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:23.454 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:23.454 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2185 2024-09-16T20:48:23.454 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.454 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:23.454 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.454 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:23.454 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335804+0000", 2024-09-16T20:48:23.454 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.454 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.454 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.454 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:23.455 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335804+0000", 2024-09-16T20:48:23.455 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.455 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.455 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.455 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:23.455 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335805+0000", 2024-09-16T20:48:23.455 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8299999999999997e-07 2024-09-16T20:48:23.455 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.455 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.455 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:23.455 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335813+0000", 2024-09-16T20:48:23.455 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6309999999999992e-06 2024-09-16T20:48:23.455 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.455 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.455 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:23.456 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335821+0000", 2024-09-16T20:48:23.456 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0169999999999996e-06 2024-09-16T20:48:23.456 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.456 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.456 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:23.456 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335827+0000", 2024-09-16T20:48:23.456 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0100000000000001e-06 2024-09-16T20:48:23.456 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.456 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.456 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:23.456 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384273+0000", 2024-09-16T20:48:23.456 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048445333 2024-09-16T20:48:23.456 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.457 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.457 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:23.457 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384315+0000", 2024-09-16T20:48:23.457 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2472999999999998e-05 2024-09-16T20:48:23.457 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.457 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.457 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:23.457 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384375+0000", 2024-09-16T20:48:23.457 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9871000000000003e-05 2024-09-16T20:48:23.457 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.458 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:23.458 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.458 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.458 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.458 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2187 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 980992~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:23.458 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335835+0000", 2024-09-16T20:48:23.458 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 1.0768404499999999, 2024-09-16T20:48:23.458 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048684927000000003, 2024-09-16T20:48:23.458 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:23.458 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:23.458 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:23.458 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:23.458 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:23.458 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:23.459 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2187 2024-09-16T20:48:23.459 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.459 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:23.459 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.459 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:23.459 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335835+0000", 2024-09-16T20:48:23.459 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.459 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.459 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.459 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:23.459 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335835+0000", 2024-09-16T20:48:23.459 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.459 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.460 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.460 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:23.460 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335836+0000", 2024-09-16T20:48:23.460 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3599999999999999e-07 2024-09-16T20:48:23.460 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.460 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.460 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:23.460 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335838+0000", 2024-09-16T20:48:23.460 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.996e-06 2024-09-16T20:48:23.460 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.460 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.460 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:23.460 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335844+0000", 2024-09-16T20:48:23.460 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7889999999999997e-06 2024-09-16T20:48:23.461 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.461 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.461 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:23.461 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335849+0000", 2024-09-16T20:48:23.461 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7400000000000004e-06 2024-09-16T20:48:23.461 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.461 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.461 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:23.461 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384411+0000", 2024-09-16T20:48:23.461 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048562069999999999 2024-09-16T20:48:23.461 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.461 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.461 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:23.461 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384460+0000", 2024-09-16T20:48:23.462 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9125999999999997e-05 2024-09-16T20:48:23.462 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.462 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.462 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:23.462 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384520+0000", 2024-09-16T20:48:23.462 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9769999999999999e-05 2024-09-16T20:48:23.462 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.462 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:23.462 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.462 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.462 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.462 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2189 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 989184~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:23.462 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335856+0000", 2024-09-16T20:48:23.462 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 1.0768192050000001, 2024-09-16T20:48:23.463 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048856525999999997, 2024-09-16T20:48:23.463 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:23.463 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:23.463 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:23.463 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:23.463 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:23.463 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:23.463 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2189 2024-09-16T20:48:23.463 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.463 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:23.463 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.463 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:23.463 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335856+0000", 2024-09-16T20:48:23.464 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.464 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.464 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.464 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:23.464 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335856+0000", 2024-09-16T20:48:23.464 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.464 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.464 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.464 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:23.464 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335857+0000", 2024-09-16T20:48:23.464 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6800000000000001e-07 2024-09-16T20:48:23.464 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.464 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.465 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:23.465 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335859+0000", 2024-09-16T20:48:23.465 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.013e-06 2024-09-16T20:48:23.465 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.465 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.465 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:23.465 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335866+0000", 2024-09-16T20:48:23.465 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8750000000000002e-06 2024-09-16T20:48:23.465 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.465 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.465 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:23.465 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335871+0000", 2024-09-16T20:48:23.465 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1719999999999998e-06 2024-09-16T20:48:23.465 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.466 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.466 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:23.466 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384550+0000", 2024-09-16T20:48:23.466 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048679476999999999 2024-09-16T20:48:23.466 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.466 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.466 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:23.466 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384611+0000", 2024-09-16T20:48:23.466 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0897000000000003e-05 2024-09-16T20:48:23.466 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.466 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.466 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:23.466 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384713+0000", 2024-09-16T20:48:23.466 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101724 2024-09-16T20:48:23.466 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.467 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:23.467 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.467 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.467 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.467 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2192 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 997376~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:23.467 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335878+0000", 2024-09-16T20:48:23.467 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 1.07679752, 2024-09-16T20:48:23.467 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049028769999999999, 2024-09-16T20:48:23.467 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:23.467 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:23.467 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:23.467 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:23.468 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:23.468 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:23.468 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2192 2024-09-16T20:48:23.468 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.468 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:23.468 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.468 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:23.468 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335878+0000", 2024-09-16T20:48:23.468 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.468 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.468 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.468 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:23.468 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335878+0000", 2024-09-16T20:48:23.468 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.469 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.469 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.469 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:23.469 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335879+0000", 2024-09-16T20:48:23.469 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6899999999999998e-07 2024-09-16T20:48:23.469 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.469 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.469 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:23.469 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335881+0000", 2024-09-16T20:48:23.469 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.996e-06 2024-09-16T20:48:23.469 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.469 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.469 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:23.470 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335887+0000", 2024-09-16T20:48:23.470 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6370000000000002e-06 2024-09-16T20:48:23.470 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.470 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.470 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:23.470 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335892+0000", 2024-09-16T20:48:23.470 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2120000000000002e-06 2024-09-16T20:48:23.470 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.470 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.470 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:23.470 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384747+0000", 2024-09-16T20:48:23.470 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048854923000000001 2024-09-16T20:48:23.470 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.470 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.471 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:23.471 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384830+0000", 2024-09-16T20:48:23.471 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2359000000000002e-05 2024-09-16T20:48:23.471 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.471 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.471 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:23.471 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384907+0000", 2024-09-16T20:48:23.471 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7174000000000003e-05 2024-09-16T20:48:23.471 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.471 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:23.471 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.471 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.471 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.471 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2194 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1005568~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:23.472 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335899+0000", 2024-09-16T20:48:23.472 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 1.076776599, 2024-09-16T20:48:23.472 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049225393999999999, 2024-09-16T20:48:23.472 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:23.472 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:23.472 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:23.472 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:23.472 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:23.472 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:23.472 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2194 2024-09-16T20:48:23.472 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.472 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:23.472 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.472 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:23.473 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335899+0000", 2024-09-16T20:48:23.473 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.473 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.473 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.473 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:23.473 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335899+0000", 2024-09-16T20:48:23.473 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.473 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.473 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.473 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:23.473 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335899+0000", 2024-09-16T20:48:23.473 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8200000000000001e-07 2024-09-16T20:48:23.473 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.473 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.474 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:23.474 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335901+0000", 2024-09-16T20:48:23.474 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9e-06 2024-09-16T20:48:23.474 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.474 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.474 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:23.474 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335908+0000", 2024-09-16T20:48:23.474 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3559999999999998e-06 2024-09-16T20:48:23.474 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.474 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.474 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:23.474 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335914+0000", 2024-09-16T20:48:23.474 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4729999999999997e-06 2024-09-16T20:48:23.475 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.475 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.475 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:23.475 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384979+0000", 2024-09-16T20:48:23.475 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049064545000000001 2024-09-16T20:48:23.475 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.475 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.475 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:23.475 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385060+0000", 2024-09-16T20:48:23.475 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0804999999999996e-05 2024-09-16T20:48:23.475 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.475 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.475 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:23.476 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385124+0000", 2024-09-16T20:48:23.476 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4932999999999996e-05 2024-09-16T20:48:23.476 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.476 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:23.476 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.476 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.476 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.476 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2197 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1013760~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:23.476 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335921+0000", 2024-09-16T20:48:23.476 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 1.0767543180000001, 2024-09-16T20:48:23.476 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049342184999999997, 2024-09-16T20:48:23.476 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:23.476 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:23.476 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:23.477 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:23.477 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:23.477 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:23.477 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2197 2024-09-16T20:48:23.477 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.477 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:23.477 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.477 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:23.477 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335921+0000", 2024-09-16T20:48:23.477 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.477 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.477 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.477 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:23.478 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335921+0000", 2024-09-16T20:48:23.478 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.478 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.478 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.478 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:23.478 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335922+0000", 2024-09-16T20:48:23.478 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.32e-07 2024-09-16T20:48:23.478 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.478 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.478 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:23.478 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335924+0000", 2024-09-16T20:48:23.478 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.995e-06 2024-09-16T20:48:23.479 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.479 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.479 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:23.479 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335930+0000", 2024-09-16T20:48:23.479 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5139999999999998e-06 2024-09-16T20:48:23.479 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.479 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.479 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:23.479 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335936+0000", 2024-09-16T20:48:23.479 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6160000000000001e-06 2024-09-16T20:48:23.479 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.479 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.479 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:23.480 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385142+0000", 2024-09-16T20:48:23.480 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049206393000000001 2024-09-16T20:48:23.480 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.480 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.480 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:23.480 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385184+0000", 2024-09-16T20:48:23.480 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1928999999999998e-05 2024-09-16T20:48:23.480 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.480 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.480 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:23.480 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385264+0000", 2024-09-16T20:48:23.480 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9306000000000003e-05 2024-09-16T20:48:23.480 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.481 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:23.481 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.481 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.481 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.481 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2198 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1021952~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:23.481 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335942+0000", 2024-09-16T20:48:23.481 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 1.076733258, 2024-09-16T20:48:23.481 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049562483999999997, 2024-09-16T20:48:23.481 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:23.481 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:23.481 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:23.481 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:23.481 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:23.481 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:23.482 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2198 2024-09-16T20:48:23.482 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.482 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:23.482 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.482 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:23.482 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335942+0000", 2024-09-16T20:48:23.482 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.482 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.482 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.482 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:23.482 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335942+0000", 2024-09-16T20:48:23.482 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.482 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.482 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.482 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:23.483 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335943+0000", 2024-09-16T20:48:23.483 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7099999999999997e-07 2024-09-16T20:48:23.483 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.483 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.483 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:23.483 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335945+0000", 2024-09-16T20:48:23.483 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3499999999999999e-06 2024-09-16T20:48:23.483 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.483 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.483 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:23.483 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335952+0000", 2024-09-16T20:48:23.483 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4309999999999999e-06 2024-09-16T20:48:23.483 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.484 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.484 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:23.484 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335957+0000", 2024-09-16T20:48:23.484 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0869999999999997e-06 2024-09-16T20:48:23.484 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.484 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.484 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:23.484 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385312+0000", 2024-09-16T20:48:23.484 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049355121000000002 2024-09-16T20:48:23.484 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.484 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.484 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:23.484 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385405+0000", 2024-09-16T20:48:23.485 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.3344000000000006e-05 2024-09-16T20:48:23.485 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.485 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.485 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:23.485 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385505+0000", 2024-09-16T20:48:23.485 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.9779999999999997e-05 2024-09-16T20:48:23.485 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.485 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:23.485 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.485 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.485 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.485 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2201 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1030144~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:23.485 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335976+0000", 2024-09-16T20:48:23.485 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 1.0766998160000001, 2024-09-16T20:48:23.486 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049740077000000001, 2024-09-16T20:48:23.486 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:23.486 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:23.486 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:23.486 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:23.486 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:23.486 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:23.486 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2201 2024-09-16T20:48:23.486 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.487 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:23.487 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.487 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:23.487 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335976+0000", 2024-09-16T20:48:23.487 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.487 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.487 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.488 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:23.488 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335976+0000", 2024-09-16T20:48:23.488 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.488 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.488 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.488 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:23.488 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335976+0000", 2024-09-16T20:48:23.488 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8599999999999998e-07 2024-09-16T20:48:23.488 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.488 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.488 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:23.488 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335978+0000", 2024-09-16T20:48:23.488 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1500000000000002e-06 2024-09-16T20:48:23.488 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.489 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.489 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:23.489 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335986+0000", 2024-09-16T20:48:23.489 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3989999999999998e-06 2024-09-16T20:48:23.489 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.489 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.489 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:23.489 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335992+0000", 2024-09-16T20:48:23.489 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.682e-06 2024-09-16T20:48:23.489 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.489 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.489 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:23.489 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385537+0000", 2024-09-16T20:48:23.490 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049545828 2024-09-16T20:48:23.490 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.490 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.490 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:23.490 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385619+0000", 2024-09-16T20:48:23.490 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1720999999999999e-05 2024-09-16T20:48:23.490 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.490 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.490 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:23.490 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385716+0000", 2024-09-16T20:48:23.490 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.6811000000000005e-05 2024-09-16T20:48:23.490 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.490 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:23.491 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.491 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.491 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.491 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2203 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1038336~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:23.491 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.336000+0000", 2024-09-16T20:48:23.491 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 1.07667603, 2024-09-16T20:48:23.491 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049949888999999997, 2024-09-16T20:48:23.491 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:23.491 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:23.491 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:23.491 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:23.491 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:23.491 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:23.492 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2203 2024-09-16T20:48:23.492 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.492 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:23.492 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.492 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:23.492 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336000+0000", 2024-09-16T20:48:23.492 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.492 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.492 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.492 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:23.492 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336000+0000", 2024-09-16T20:48:23.492 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.492 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.492 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.493 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:23.493 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336000+0000", 2024-09-16T20:48:23.493 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.58e-07 2024-09-16T20:48:23.493 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.493 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.493 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:23.493 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336002+0000", 2024-09-16T20:48:23.493 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0040000000000002e-06 2024-09-16T20:48:23.493 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.493 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.493 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:23.493 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336009+0000", 2024-09-16T20:48:23.493 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9890000000000003e-06 2024-09-16T20:48:23.494 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.494 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.494 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:23.494 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336014+0000", 2024-09-16T20:48:23.494 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4e-06 2024-09-16T20:48:23.494 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.494 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.494 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:23.494 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385759+0000", 2024-09-16T20:48:23.494 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049744151 2024-09-16T20:48:23.494 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.494 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.494 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:23.494 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385847+0000", 2024-09-16T20:48:23.495 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8189000000000006e-05 2024-09-16T20:48:23.495 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.495 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.495 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:23.495 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385950+0000", 2024-09-16T20:48:23.495 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102698 2024-09-16T20:48:23.495 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.495 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:23.495 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.495 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.495 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.495 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2206 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1046528~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:23.495 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:48:23.495 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 1.076653869, 2024-09-16T20:48:23.496 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050147714000000003, 2024-09-16T20:48:23.496 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:23.496 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:23.496 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:23.496 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:23.496 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:23.496 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:23.496 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2206 2024-09-16T20:48:23.496 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.496 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:23.496 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.496 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:23.496 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:48:23.496 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.497 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.497 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.497 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:23.497 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:48:23.497 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.497 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.497 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.497 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:23.497 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:48:23.497 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8599999999999999e-07 2024-09-16T20:48:23.497 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.497 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.498 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:23.498 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336024+0000", 2024-09-16T20:48:23.498 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.934e-06 2024-09-16T20:48:23.498 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.498 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.498 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:23.498 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336037+0000", 2024-09-16T20:48:23.498 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2496e-05 2024-09-16T20:48:23.498 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.498 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.498 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:23.498 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336042+0000", 2024-09-16T20:48:23.498 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4809999999999997e-06 2024-09-16T20:48:23.498 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.499 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.499 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:23.499 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385980+0000", 2024-09-16T20:48:23.499 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049938376999999999 2024-09-16T20:48:23.499 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.499 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.499 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:23.499 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.386069+0000", 2024-09-16T20:48:23.499 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8368999999999994e-05 2024-09-16T20:48:23.499 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.499 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.499 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:23.499 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.386170+0000", 2024-09-16T20:48:23.499 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000100671 2024-09-16T20:48:23.500 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.500 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:23.500 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.500 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.500 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.500 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5439 3.a 3:5789f5c3:::rbd_data.110ed9ca540.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 1181696~60416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:23.500 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:48:23.500 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.54796528899999997, 2024-09-16T20:48:23.500 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10102045599999999, 2024-09-16T20:48:23.500 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:23.500 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:23.500 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:23.500 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:23.500 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:23.501 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:23.501 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5439 2024-09-16T20:48:23.501 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.501 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:23.501 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.501 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:23.501 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:48:23.501 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.501 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.501 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.501 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:23.501 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:48:23.501 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.501 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.502 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.502 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:23.502 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864711+0000", 2024-09-16T20:48:23.502 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7899999999999997e-07 2024-09-16T20:48:23.502 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.502 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.502 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:23.502 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864760+0000", 2024-09-16T20:48:23.502 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8803e-05 2024-09-16T20:48:23.502 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.502 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.502 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:23.502 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864769+0000", 2024-09-16T20:48:23.502 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0999999999999993e-06 2024-09-16T20:48:23.502 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.503 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.503 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:23.503 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864774+0000", 2024-09-16T20:48:23.503 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8919999999999999e-06 2024-09-16T20:48:23.503 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.503 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.503 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:23.503 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864815+0000", 2024-09-16T20:48:23.503 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1307000000000003e-05 2024-09-16T20:48:23.503 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.503 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.503 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:23.504 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864893+0000", 2024-09-16T20:48:23.504 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7581000000000001e-05 2024-09-16T20:48:23.504 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.504 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.504 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 1", 2024-09-16T20:48:23.504 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864996+0000", 2024-09-16T20:48:23.504 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102725 2024-09-16T20:48:23.504 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.504 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.504 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:23.504 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965442+0000", 2024-09-16T20:48:23.504 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100446012 2024-09-16T20:48:23.504 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.505 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.505 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:23.505 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965618+0000", 2024-09-16T20:48:23.505 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00017653100000000001 2024-09-16T20:48:23.505 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.505 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.505 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:23.505 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965682+0000", 2024-09-16T20:48:23.505 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3905999999999994e-05 2024-09-16T20:48:23.505 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.505 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.505 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:23.505 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965731+0000", 2024-09-16T20:48:23.505 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8819999999999997e-05 2024-09-16T20:48:23.505 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.506 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:23.506 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.506 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.506 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.506 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4600.0:5439 3.a e37/23 ondisk, result = 0)", 2024-09-16T20:48:23.506 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:48:23.506 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.54696914100000005, 2024-09-16T20:48:23.506 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099742634999999996, 2024-09-16T20:48:23.506 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:23.506 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:23.506 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:23.506 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:23.506 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.507 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:23.507 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:48:23.507 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.507 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.507 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.507 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:23.507 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:48:23.507 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.507 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.507 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.507 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:23.507 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:48:23.507 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0900000000000001e-07 2024-09-16T20:48:23.507 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.508 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.508 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:23.508 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865711+0000", 2024-09-16T20:48:23.508 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3739999999999998e-06 2024-09-16T20:48:23.508 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.508 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.508 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:23.508 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965343+0000", 2024-09-16T20:48:23.508 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099632526999999999 2024-09-16T20:48:23.508 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.508 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.508 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:23.508 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965357+0000", 2024-09-16T20:48:23.508 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3713999999999999e-05 2024-09-16T20:48:23.509 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.509 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.509 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:23.509 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965426+0000", 2024-09-16T20:48:23.509 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8745999999999995e-05 2024-09-16T20:48:23.509 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.509 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.509 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:23.509 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965434+0000", 2024-09-16T20:48:23.509 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6910000000000003e-06 2024-09-16T20:48:23.509 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.509 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.509 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:23.509 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965449+0000", 2024-09-16T20:48:23.510 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4874e-05 2024-09-16T20:48:23.510 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.510 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:23.510 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.510 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.510 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:23.510 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:23.511 INFO:teuthology.orchestra.run.smithi139.stdout:888 write 0x870d600 thru 0x8711f56 (0x4957 bytes) 2024-09-16T20:48:23.511 INFO:teuthology.orchestra.run.smithi139.stdout:889 clone 7 order 22 su 32768 sc 12 2024-09-16T20:48:23.512 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:48:23.512 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:23.510+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:23.612 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:23.610+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:23.666 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:23.666 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:48:23.666 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:48:23.666 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:23.675 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:48:23.704 INFO:teuthology.orchestra.run.smithi139.stdout:truncating image image_client.0-clone6 from 0x8712000 (overlap 0x1fc7a00) to 0x988a00 2024-09-16T20:48:23.713 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:23.711+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:23.814 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:23.813+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:23.830 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:23.830 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:48:23.830 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:48:23.830 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:48:23.830 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:23.839 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:48:23.915 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:23.913+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:23.948 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:23.949 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:48:23.949 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:48:23.949 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:48:23.949 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.949 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:48:23.949 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:48:23.949 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 30.692780327000001, 2024-09-16T20:48:23.949 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67502346999999996, 2024-09-16T20:48:23.949 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:23.949 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:23.949 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:23.949 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:23.949 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:48:23.950 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:48:23.950 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 1 2024-09-16T20:48:23.950 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.950 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:23.950 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.950 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:23.950 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:48:23.950 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.950 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.950 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.950 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:23.950 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:48:23.950 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.951 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.951 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.951 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:23.951 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254128+0000", 2024-09-16T20:48:23.951 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.463e-06 2024-09-16T20:48:23.951 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.951 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.951 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:23.951 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254135+0000", 2024-09-16T20:48:23.951 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1579999999999999e-06 2024-09-16T20:48:23.951 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.951 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.951 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:23.952 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254146+0000", 2024-09-16T20:48:23.952 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1341e-05 2024-09-16T20:48:23.952 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.952 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.952 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:23.952 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254155+0000", 2024-09-16T20:48:23.952 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.1549999999999996e-06 2024-09-16T20:48:23.952 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.952 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.952 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:23.952 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254221+0000", 2024-09-16T20:48:23.952 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5470999999999997e-05 2024-09-16T20:48:23.952 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.953 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.953 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:48:23.953 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254231+0000", 2024-09-16T20:48:23.953 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0592e-05 2024-09-16T20:48:23.953 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.953 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.953 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:23.953 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926711+0000", 2024-09-16T20:48:23.953 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67247946599999997 2024-09-16T20:48:23.953 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.953 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.953 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:23.953 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926910+0000", 2024-09-16T20:48:23.954 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019934 2024-09-16T20:48:23.954 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.954 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.954 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:48:23.954 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.927159+0000", 2024-09-16T20:48:23.954 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00024876599999999998 2024-09-16T20:48:23.954 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.954 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.954 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:23.954 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.928797+0000", 2024-09-16T20:48:23.954 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001638308 2024-09-16T20:48:23.954 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.954 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.954 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:23.955 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929026+0000", 2024-09-16T20:48:23.955 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00022876099999999999 2024-09-16T20:48:23.955 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.955 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.955 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:23.955 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929102+0000", 2024-09-16T20:48:23.955 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6184000000000001e-05 2024-09-16T20:48:23.955 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.955 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.955 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:23.955 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929150+0000", 2024-09-16T20:48:23.955 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7465e-05 2024-09-16T20:48:23.955 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.956 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:23.956 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.956 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.956 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.956 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4312 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4618240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:23.956 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745559+0000", 2024-09-16T20:48:23.956 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.201347191, 2024-09-16T20:48:23.956 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046756458000000001, 2024-09-16T20:48:23.956 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:23.956 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:23.956 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:23.956 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:23.956 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:48:23.957 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:23.957 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4312 2024-09-16T20:48:23.957 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.957 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:23.957 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.957 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:23.957 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745559+0000", 2024-09-16T20:48:23.957 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.957 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.957 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.957 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:23.957 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745559+0000", 2024-09-16T20:48:23.957 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.957 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.958 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.958 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:23.958 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745560+0000", 2024-09-16T20:48:23.958 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6899999999999998e-07 2024-09-16T20:48:23.958 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.958 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.958 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:23.958 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745569+0000", 2024-09-16T20:48:23.958 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8950000000000004e-06 2024-09-16T20:48:23.958 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.959 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.959 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:23.959 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745576+0000", 2024-09-16T20:48:23.959 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9949999999999999e-06 2024-09-16T20:48:23.959 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.959 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.959 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:23.959 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745596+0000", 2024-09-16T20:48:23.959 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0767999999999998e-05 2024-09-16T20:48:23.959 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.959 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.959 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:23.960 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792225+0000", 2024-09-16T20:48:23.960 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046628393999999997 2024-09-16T20:48:23.960 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.960 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.960 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:23.960 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792265+0000", 2024-09-16T20:48:23.960 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9912e-05 2024-09-16T20:48:23.960 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.960 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.960 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:23.960 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792316+0000", 2024-09-16T20:48:23.960 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1025e-05 2024-09-16T20:48:23.960 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.961 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:23.961 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.961 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.961 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.961 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4313 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4626432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:23.961 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745608+0000", 2024-09-16T20:48:23.961 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.2012986220000004, 2024-09-16T20:48:23.961 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046829200000000001, 2024-09-16T20:48:23.961 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:23.961 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:23.961 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:23.961 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:23.961 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:48:23.962 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:23.962 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4313 2024-09-16T20:48:23.962 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.962 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:23.962 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.962 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:23.962 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745608+0000", 2024-09-16T20:48:23.962 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.962 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.962 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.962 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:23.962 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745608+0000", 2024-09-16T20:48:23.962 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.962 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.962 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.963 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:23.963 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745609+0000", 2024-09-16T20:48:23.963 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4700000000000001e-07 2024-09-16T20:48:23.963 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.963 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.963 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:23.963 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745611+0000", 2024-09-16T20:48:23.963 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8119999999999999e-06 2024-09-16T20:48:23.963 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.963 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.963 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:23.963 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745619+0000", 2024-09-16T20:48:23.963 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6520000000000006e-06 2024-09-16T20:48:23.963 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.964 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.964 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:23.964 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745625+0000", 2024-09-16T20:48:23.964 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6590000000000004e-06 2024-09-16T20:48:23.964 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.964 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.964 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:23.964 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792344+0000", 2024-09-16T20:48:23.964 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046719318000000003 2024-09-16T20:48:23.964 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.964 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.964 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:23.965 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792384+0000", 2024-09-16T20:48:23.965 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0389000000000002e-05 2024-09-16T20:48:23.965 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.965 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.965 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:23.965 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792437+0000", 2024-09-16T20:48:23.965 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2623000000000001e-05 2024-09-16T20:48:23.965 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.965 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:23.965 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.965 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.965 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.965 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4314 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4634624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:23.965 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745632+0000", 2024-09-16T20:48:23.966 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.201274755, 2024-09-16T20:48:23.966 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046928879999999999, 2024-09-16T20:48:23.966 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:23.966 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:23.966 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:23.966 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:23.966 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:48:23.966 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:23.966 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4314 2024-09-16T20:48:23.966 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.966 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:23.966 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.966 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:23.966 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745632+0000", 2024-09-16T20:48:23.967 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.967 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.967 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.967 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:23.967 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745632+0000", 2024-09-16T20:48:23.967 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.967 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.967 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.967 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:23.967 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745632+0000", 2024-09-16T20:48:23.967 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3700000000000001e-07 2024-09-16T20:48:23.967 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.967 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.967 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:23.968 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745634+0000", 2024-09-16T20:48:23.968 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9759999999999998e-06 2024-09-16T20:48:23.968 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.968 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.968 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:23.968 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745645+0000", 2024-09-16T20:48:23.968 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1097000000000001e-05 2024-09-16T20:48:23.968 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.968 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.968 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:23.968 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745651+0000", 2024-09-16T20:48:23.968 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5090000000000001e-06 2024-09-16T20:48:23.968 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.968 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.969 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:23.969 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792467+0000", 2024-09-16T20:48:23.969 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046816712000000003 2024-09-16T20:48:23.969 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.969 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.969 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:23.969 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792511+0000", 2024-09-16T20:48:23.969 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3256000000000001e-05 2024-09-16T20:48:23.969 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.969 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.969 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:23.969 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792561+0000", 2024-09-16T20:48:23.969 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9892999999999998e-05 2024-09-16T20:48:23.969 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.969 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:23.970 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.970 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.970 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.970 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4315 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4642816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:23.970 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745659+0000", 2024-09-16T20:48:23.970 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.2012476320000003, 2024-09-16T20:48:23.970 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047065501000000003, 2024-09-16T20:48:23.970 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:23.970 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:23.970 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:23.970 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:23.970 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:48:23.970 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:23.970 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4315 2024-09-16T20:48:23.970 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.971 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:23.971 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.971 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:23.971 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745659+0000", 2024-09-16T20:48:23.971 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.971 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.971 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.971 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:23.971 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745659+0000", 2024-09-16T20:48:23.971 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.971 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.971 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.971 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:23.972 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745659+0000", 2024-09-16T20:48:23.972 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7e-07 2024-09-16T20:48:23.972 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.972 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.972 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:23.972 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745662+0000", 2024-09-16T20:48:23.972 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2189999999999998e-06 2024-09-16T20:48:23.972 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.972 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.972 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:23.972 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745668+0000", 2024-09-16T20:48:23.972 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8979999999999999e-06 2024-09-16T20:48:23.972 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.972 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.973 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:23.973 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745675+0000", 2024-09-16T20:48:23.973 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3579999999999999e-06 2024-09-16T20:48:23.973 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.973 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.973 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:23.973 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792593+0000", 2024-09-16T20:48:23.973 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046917888999999997 2024-09-16T20:48:23.973 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.973 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.973 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:23.973 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792632+0000", 2024-09-16T20:48:23.973 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9818999999999999e-05 2024-09-16T20:48:23.974 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.974 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.974 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:23.974 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792724+0000", 2024-09-16T20:48:23.974 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.1848000000000006e-05 2024-09-16T20:48:23.974 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.974 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:23.974 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.974 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.974 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.974 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4316 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4651008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:23.974 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745682+0000", 2024-09-16T20:48:23.974 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.2012243539999998, 2024-09-16T20:48:23.974 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047270334999999997, 2024-09-16T20:48:23.975 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:23.975 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:23.975 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:23.975 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:23.975 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:48:23.975 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:23.975 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4316 2024-09-16T20:48:23.975 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.975 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:23.975 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.975 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:23.975 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745682+0000", 2024-09-16T20:48:23.975 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.975 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.976 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.976 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:23.976 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745682+0000", 2024-09-16T20:48:23.976 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.976 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.976 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.976 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:23.976 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745683+0000", 2024-09-16T20:48:23.976 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.27e-07 2024-09-16T20:48:23.976 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.976 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.976 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:23.976 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745685+0000", 2024-09-16T20:48:23.977 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.464e-06 2024-09-16T20:48:23.977 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.977 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.977 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:23.977 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745692+0000", 2024-09-16T20:48:23.977 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.849e-06 2024-09-16T20:48:23.977 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.977 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.977 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:23.977 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745697+0000", 2024-09-16T20:48:23.977 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0170000000000004e-06 2024-09-16T20:48:23.977 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.977 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.978 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:23.978 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792771+0000", 2024-09-16T20:48:23.978 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047074224999999997 2024-09-16T20:48:23.978 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.978 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.978 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:23.978 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792845+0000", 2024-09-16T20:48:23.978 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3739999999999995e-05 2024-09-16T20:48:23.978 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.978 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.978 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:23.978 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792952+0000", 2024-09-16T20:48:23.978 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000107613 2024-09-16T20:48:23.978 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.979 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:23.979 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.979 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.979 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.979 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4317 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4659200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:23.979 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745704+0000", 2024-09-16T20:48:23.979 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.2012024080000003, 2024-09-16T20:48:23.979 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047411778000000002, 2024-09-16T20:48:23.979 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:23.979 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:23.979 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:23.979 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:23.979 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:48:23.980 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:23.980 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4317 2024-09-16T20:48:23.980 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.980 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:23.980 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.980 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:23.980 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745704+0000", 2024-09-16T20:48:23.980 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.980 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.980 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.980 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:23.980 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745704+0000", 2024-09-16T20:48:23.980 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.981 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.981 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.981 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:23.981 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745704+0000", 2024-09-16T20:48:23.981 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0900000000000002e-07 2024-09-16T20:48:23.981 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.981 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.981 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:23.981 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745707+0000", 2024-09-16T20:48:23.981 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1660000000000001e-06 2024-09-16T20:48:23.981 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.981 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.981 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:23.982 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745722+0000", 2024-09-16T20:48:23.982 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5084e-05 2024-09-16T20:48:23.982 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.982 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.982 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:23.982 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745735+0000", 2024-09-16T20:48:23.982 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3074999999999999e-05 2024-09-16T20:48:23.982 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.982 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.982 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:23.982 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792989+0000", 2024-09-16T20:48:23.982 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047254013999999997 2024-09-16T20:48:23.982 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.983 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.983 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:23.983 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793034+0000", 2024-09-16T20:48:23.983 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4858999999999999e-05 2024-09-16T20:48:23.983 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.983 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.983 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:23.983 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793116+0000", 2024-09-16T20:48:23.983 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2170999999999996e-05 2024-09-16T20:48:23.983 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.983 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:23.983 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.983 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.983 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.984 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4318 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4667392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:23.984 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745812+0000", 2024-09-16T20:48:23.984 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.2010947290000002, 2024-09-16T20:48:23.984 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047532229000000002, 2024-09-16T20:48:23.984 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:23.984 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:23.984 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:23.984 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:23.984 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:48:23.984 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:23.984 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4318 2024-09-16T20:48:23.984 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.984 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:23.984 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.985 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:23.985 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745812+0000", 2024-09-16T20:48:23.985 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.985 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.985 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.985 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:23.985 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745812+0000", 2024-09-16T20:48:23.985 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.985 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.985 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.985 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:23.985 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745812+0000", 2024-09-16T20:48:23.985 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5899999999999996e-07 2024-09-16T20:48:23.986 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.986 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.986 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:23.986 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745815+0000", 2024-09-16T20:48:23.986 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9110000000000002e-06 2024-09-16T20:48:23.986 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.986 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.986 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:23.986 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745823+0000", 2024-09-16T20:48:23.986 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1030000000000002e-06 2024-09-16T20:48:23.986 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.986 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.986 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:23.987 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745829+0000", 2024-09-16T20:48:23.987 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8449999999999999e-06 2024-09-16T20:48:23.987 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.987 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.987 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:23.987 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793159+0000", 2024-09-16T20:48:23.987 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047330009999999999 2024-09-16T20:48:23.987 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.987 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.987 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:23.987 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793251+0000", 2024-09-16T20:48:23.987 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.1991999999999996e-05 2024-09-16T20:48:23.987 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.987 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.988 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:23.988 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793344+0000", 2024-09-16T20:48:23.988 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2808999999999999e-05 2024-09-16T20:48:23.988 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.988 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:23.988 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.988 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.988 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.988 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4319 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4675584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:23.988 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745838+0000", 2024-09-16T20:48:23.988 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.2010685160000003, 2024-09-16T20:48:23.988 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047640204999999998, 2024-09-16T20:48:23.988 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:23.989 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:23.989 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:23.989 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:23.989 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:48:23.989 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:23.989 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4319 2024-09-16T20:48:23.989 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.989 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:23.989 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.989 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:23.989 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745838+0000", 2024-09-16T20:48:23.989 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.989 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.989 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.990 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:23.990 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745838+0000", 2024-09-16T20:48:23.990 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.990 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.990 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.990 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:23.990 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745838+0000", 2024-09-16T20:48:23.990 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6899999999999998e-07 2024-09-16T20:48:23.990 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.990 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.990 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:23.990 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745840+0000", 2024-09-16T20:48:23.990 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0660000000000002e-06 2024-09-16T20:48:23.991 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.991 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.991 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:23.991 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745847+0000", 2024-09-16T20:48:23.991 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8689999999999998e-06 2024-09-16T20:48:23.991 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.991 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.991 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:23.991 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745853+0000", 2024-09-16T20:48:23.991 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2399999999999998e-06 2024-09-16T20:48:23.992 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.992 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.992 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:23.992 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793381+0000", 2024-09-16T20:48:23.992 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047528005999999998 2024-09-16T20:48:23.992 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.992 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.992 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:23.992 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793423+0000", 2024-09-16T20:48:23.992 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2799000000000001e-05 2024-09-16T20:48:23.992 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.992 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.993 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:23.993 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793478+0000", 2024-09-16T20:48:23.993 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4756000000000002e-05 2024-09-16T20:48:23.993 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.993 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:23.993 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:23.993 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.993 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.993 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4320 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4683776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:23.993 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745859+0000", 2024-09-16T20:48:23.993 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.2010472720000003, 2024-09-16T20:48:23.993 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047811399999999997, 2024-09-16T20:48:23.993 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:23.994 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:23.994 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:23.994 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:23.994 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:48:23.994 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:23.996 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4320 2024-09-16T20:48:23.996 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.996 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:23.996 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.997 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:23.997 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745859+0000", 2024-09-16T20:48:23.997 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.997 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.997 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.997 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:23.997 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745859+0000", 2024-09-16T20:48:23.997 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:23.997 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.997 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.997 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:23.997 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745860+0000", 2024-09-16T20:48:23.998 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9099999999999999e-07 2024-09-16T20:48:23.998 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.998 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.998 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:23.998 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745861+0000", 2024-09-16T20:48:23.998 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8589999999999999e-06 2024-09-16T20:48:23.998 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.998 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.998 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:23.998 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745868+0000", 2024-09-16T20:48:23.998 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2249999999999997e-06 2024-09-16T20:48:23.998 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.998 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.998 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:23.999 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745873+0000", 2024-09-16T20:48:23.999 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1290000000000004e-06 2024-09-16T20:48:23.999 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.999 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.999 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:23.999 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793522+0000", 2024-09-16T20:48:23.999 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047648847000000001 2024-09-16T20:48:23.999 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:23.999 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:23.999 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:23.999 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793603+0000", 2024-09-16T20:48:23.999 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1167000000000004e-05 2024-09-16T20:48:23.999 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.000 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.000 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:24.000 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793671+0000", 2024-09-16T20:48:24.000 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7781999999999996e-05 2024-09-16T20:48:24.000 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.000 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:24.000 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.000 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.000 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.000 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4321 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4691968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:24.001 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745879+0000", 2024-09-16T20:48:24.001 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.2010270759999999, 2024-09-16T20:48:24.001 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047950375000000003, 2024-09-16T20:48:24.001 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:24.001 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:24.001 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:24.001 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:24.001 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:48:24.001 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:24.001 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4321 2024-09-16T20:48:24.001 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.002 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:24.002 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.002 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:24.002 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745879+0000", 2024-09-16T20:48:24.002 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.002 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.002 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.002 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:24.002 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745879+0000", 2024-09-16T20:48:24.002 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.002 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.003 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.003 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:24.003 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745880+0000", 2024-09-16T20:48:24.003 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.65e-07 2024-09-16T20:48:24.003 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.003 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.003 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:24.003 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745882+0000", 2024-09-16T20:48:24.003 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.922e-06 2024-09-16T20:48:24.003 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.003 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.004 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:24.004 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745888+0000", 2024-09-16T20:48:24.004 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0510000000000002e-06 2024-09-16T20:48:24.004 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.004 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.004 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:24.004 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745893+0000", 2024-09-16T20:48:24.004 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9579999999999998e-06 2024-09-16T20:48:24.004 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.004 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.004 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:24.005 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793704+0000", 2024-09-16T20:48:24.005 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047811179000000002 2024-09-16T20:48:24.005 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.005 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.005 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:24.005 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793759+0000", 2024-09-16T20:48:24.005 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5260999999999998e-05 2024-09-16T20:48:24.005 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.005 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.005 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:24.005 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793830+0000", 2024-09-16T20:48:24.005 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0638999999999999e-05 2024-09-16T20:48:24.006 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.006 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:24.006 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.006 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.006 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.006 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4322 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4700160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:24.006 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745899+0000", 2024-09-16T20:48:24.006 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.2010076380000001, 2024-09-16T20:48:24.006 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048057280000000001, 2024-09-16T20:48:24.006 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:24.006 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:24.006 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:24.007 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:24.007 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:48:24.007 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:24.007 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4322 2024-09-16T20:48:24.007 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.007 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:24.007 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.007 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:24.007 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745899+0000", 2024-09-16T20:48:24.007 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.007 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.008 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.008 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:24.008 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745899+0000", 2024-09-16T20:48:24.008 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.008 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.008 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.008 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:24.008 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745899+0000", 2024-09-16T20:48:24.008 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9299999999999999e-07 2024-09-16T20:48:24.008 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.008 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.008 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:24.009 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745901+0000", 2024-09-16T20:48:24.009 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0030000000000001e-06 2024-09-16T20:48:24.009 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.009 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.009 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:24.009 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745908+0000", 2024-09-16T20:48:24.009 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3729999999999998e-06 2024-09-16T20:48:24.009 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.009 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.009 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:24.009 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745914+0000", 2024-09-16T20:48:24.010 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.122e-06 2024-09-16T20:48:24.010 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.010 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.010 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:24.010 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793862+0000", 2024-09-16T20:48:24.010 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047948324 2024-09-16T20:48:24.010 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.010 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.010 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:24.010 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793903+0000", 2024-09-16T20:48:24.010 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0465e-05 2024-09-16T20:48:24.011 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.011 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.011 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:24.011 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793956+0000", 2024-09-16T20:48:24.011 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3600000000000002e-05 2024-09-16T20:48:24.011 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.011 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:24.011 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.011 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.011 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.012 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4323 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4708352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:24.012 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745920+0000", 2024-09-16T20:48:24.012 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.2009859970000001, 2024-09-16T20:48:24.012 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048268138000000002, 2024-09-16T20:48:24.012 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:24.012 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:24.012 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:24.012 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:24.012 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:48:24.012 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:24.012 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4323 2024-09-16T20:48:24.012 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.013 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:24.013 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.013 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:24.013 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745920+0000", 2024-09-16T20:48:24.013 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.013 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.013 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.013 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:24.013 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745920+0000", 2024-09-16T20:48:24.013 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.013 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.014 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.014 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:24.014 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745921+0000", 2024-09-16T20:48:24.014 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7e-07 2024-09-16T20:48:24.014 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.014 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.014 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:24.014 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745923+0000", 2024-09-16T20:48:24.014 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7880000000000001e-06 2024-09-16T20:48:24.014 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.014 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.015 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:24.015 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745929+0000", 2024-09-16T20:48:24.015 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0809999999999999e-06 2024-09-16T20:48:24.015 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.015 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.015 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:24.015 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745935+0000", 2024-09-16T20:48:24.015 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5370000000000003e-06 2024-09-16T20:48:24.015 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.015 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.015 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:24.016 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793987+0000", 2024-09-16T20:48:24.016 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048051736999999997 2024-09-16T20:48:24.016 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.016 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.016 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:24.016 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.794070+0000", 2024-09-16T20:48:24.016 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.3277999999999998e-05 2024-09-16T20:48:24.016 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.016 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.016 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:24.016 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.794189+0000", 2024-09-16T20:48:24.017 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000118347 2024-09-16T20:48:24.017 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.017 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:24.017 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.017 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.017 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.017 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4324 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4716544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:24.017 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745946+0000", 2024-09-16T20:48:24.017 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.200960426, 2024-09-16T20:48:24.017 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048468153999999999, 2024-09-16T20:48:24.017 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:24.017 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:24.018 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:24.018 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:24.018 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:48:24.018 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:24.018 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4324 2024-09-16T20:48:24.018 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.018 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:24.018 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.018 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:24.018 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745946+0000", 2024-09-16T20:48:24.018 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.019 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.019 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.019 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:24.019 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745946+0000", 2024-09-16T20:48:24.019 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.019 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.019 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.019 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:24.019 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745946+0000", 2024-09-16T20:48:24.019 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4499999999999997e-07 2024-09-16T20:48:24.020 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.020 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.020 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:24.020 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745949+0000", 2024-09-16T20:48:24.020 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2529999999999999e-06 2024-09-16T20:48:24.020 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.020 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.020 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:24.020 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745955+0000", 2024-09-16T20:48:24.020 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6470000000000001e-06 2024-09-16T20:48:24.021 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.021 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.021 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:24.021 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745960+0000", 2024-09-16T20:48:24.021 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7910000000000004e-06 2024-09-16T20:48:24.021 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.021 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.021 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:24.021 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.794220+0000", 2024-09-16T20:48:24.021 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048260069000000003 2024-09-16T20:48:24.021 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.022 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.022 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:24.022 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.794311+0000", 2024-09-16T20:48:24.022 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0593000000000005e-05 2024-09-16T20:48:24.022 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.022 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.022 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:24.022 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.794414+0000", 2024-09-16T20:48:24.022 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000103356 2024-09-16T20:48:24.022 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.023 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:24.023 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.023 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.023 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.023 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2059 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 1005568~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:24.023 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.342406+0000", 2024-09-16T20:48:24.023 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 1.6044999880000002, 2024-09-16T20:48:24.023 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046803412000000003, 2024-09-16T20:48:24.023 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:24.023 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:24.024 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:24.024 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:24.024 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:24.024 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:24.024 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2059 2024-09-16T20:48:24.024 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.024 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:24.024 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.024 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:24.024 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342406+0000", 2024-09-16T20:48:24.025 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.025 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.025 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.025 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:24.025 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342406+0000", 2024-09-16T20:48:24.025 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.025 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.025 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.025 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:24.025 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342407+0000", 2024-09-16T20:48:24.026 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7399999999999998e-07 2024-09-16T20:48:24.026 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.026 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.026 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:24.026 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342409+0000", 2024-09-16T20:48:24.026 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.4509999999999999e-06 2024-09-16T20:48:24.026 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.026 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.026 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:24.026 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342418+0000", 2024-09-16T20:48:24.026 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8969999999999997e-06 2024-09-16T20:48:24.026 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.027 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.027 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:24.027 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342424+0000", 2024-09-16T20:48:24.027 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5210000000000002e-06 2024-09-16T20:48:24.027 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.027 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.027 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:24.027 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389084+0000", 2024-09-16T20:48:24.027 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046659997000000002 2024-09-16T20:48:24.027 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.027 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.028 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:24.028 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389153+0000", 2024-09-16T20:48:24.028 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9558e-05 2024-09-16T20:48:24.028 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.028 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.028 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:24.028 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389210+0000", 2024-09-16T20:48:24.028 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6514000000000001e-05 2024-09-16T20:48:24.028 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.028 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:24.028 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.029 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.029 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.029 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2061 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 1013760~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:24.029 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.342445+0000", 2024-09-16T20:48:24.029 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 1.604461436, 2024-09-16T20:48:24.029 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046901708, 2024-09-16T20:48:24.029 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:24.029 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:24.029 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:24.029 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:24.029 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:24.029 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:24.030 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2061 2024-09-16T20:48:24.030 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.030 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:24.030 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.030 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:24.030 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342445+0000", 2024-09-16T20:48:24.030 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.030 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.030 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.030 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:24.030 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342445+0000", 2024-09-16T20:48:24.031 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.031 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.031 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.031 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:24.031 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342446+0000", 2024-09-16T20:48:24.031 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4700000000000001e-07 2024-09-16T20:48:24.031 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.031 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.031 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:24.031 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342448+0000", 2024-09-16T20:48:24.031 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6460000000000002e-06 2024-09-16T20:48:24.031 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.032 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.032 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:24.032 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342456+0000", 2024-09-16T20:48:24.032 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7489999999999998e-06 2024-09-16T20:48:24.032 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.032 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.032 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:24.032 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342462+0000", 2024-09-16T20:48:24.032 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5759999999999996e-06 2024-09-16T20:48:24.032 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.032 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.032 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:24.033 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389238+0000", 2024-09-16T20:48:24.033 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046776515999999997 2024-09-16T20:48:24.033 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.033 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.033 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:24.033 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389285+0000", 2024-09-16T20:48:24.033 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6581e-05 2024-09-16T20:48:24.033 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.033 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.033 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:24.033 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389347+0000", 2024-09-16T20:48:24.034 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2093000000000003e-05 2024-09-16T20:48:24.034 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.034 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:24.034 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.034 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.034 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.034 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2063 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 1021952~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:24.034 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.342470+0000", 2024-09-16T20:48:24.034 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 1.6044365470000002, 2024-09-16T20:48:24.034 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047007602000000002, 2024-09-16T20:48:24.034 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:24.034 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:24.035 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:24.035 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:24.035 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:24.035 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:24.035 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2063 2024-09-16T20:48:24.035 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.035 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:24.035 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.035 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:24.035 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342470+0000", 2024-09-16T20:48:24.035 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.035 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.036 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.036 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:24.036 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342470+0000", 2024-09-16T20:48:24.036 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.036 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.036 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.036 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:24.036 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342470+0000", 2024-09-16T20:48:24.036 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0999999999999999e-07 2024-09-16T20:48:24.036 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.036 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.037 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:24.037 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342473+0000", 2024-09-16T20:48:24.037 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.249e-06 2024-09-16T20:48:24.037 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.037 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.037 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:24.037 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342479+0000", 2024-09-16T20:48:24.037 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7549999999999997e-06 2024-09-16T20:48:24.037 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.037 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.037 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:24.038 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342485+0000", 2024-09-16T20:48:24.038 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.344e-06 2024-09-16T20:48:24.038 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.038 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.038 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:24.038 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389364+0000", 2024-09-16T20:48:24.038 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046879542000000003 2024-09-16T20:48:24.038 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.038 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.038 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:24.038 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389420+0000", 2024-09-16T20:48:24.039 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5541000000000002e-05 2024-09-16T20:48:24.039 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.039 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.039 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:24.039 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389478+0000", 2024-09-16T20:48:24.039 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7661000000000002e-05 2024-09-16T20:48:24.039 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.039 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:24.039 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.039 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.039 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.039 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2065 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 1030144~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:24.040 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.342492+0000", 2024-09-16T20:48:24.040 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 1.604414709, 2024-09-16T20:48:24.040 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047127962000000002, 2024-09-16T20:48:24.040 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:24.040 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:24.040 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:24.040 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:24.040 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:24.040 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:24.040 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2065 2024-09-16T20:48:24.041 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.041 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:24.041 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.041 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:24.041 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342492+0000", 2024-09-16T20:48:24.041 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.041 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.041 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.041 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:24.041 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342492+0000", 2024-09-16T20:48:24.041 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.041 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.042 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.042 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:24.042 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342492+0000", 2024-09-16T20:48:24.042 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1100000000000001e-07 2024-09-16T20:48:24.042 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.042 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.042 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:24.042 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342494+0000", 2024-09-16T20:48:24.042 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9039999999999999e-06 2024-09-16T20:48:24.042 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.042 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.042 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:24.043 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342501+0000", 2024-09-16T20:48:24.043 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6610000000000003e-06 2024-09-16T20:48:24.043 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.043 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.043 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:24.043 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342506+0000", 2024-09-16T20:48:24.043 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8760000000000004e-06 2024-09-16T20:48:24.043 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.043 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.043 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:24.043 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389501+0000", 2024-09-16T20:48:24.043 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046995440999999999 2024-09-16T20:48:24.044 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.044 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.044 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:24.044 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389550+0000", 2024-09-16T20:48:24.044 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8949000000000002e-05 2024-09-16T20:48:24.044 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.044 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.044 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:24.044 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389620+0000", 2024-09-16T20:48:24.044 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9720000000000003e-05 2024-09-16T20:48:24.044 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.045 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:24.045 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.045 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.045 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.045 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2068 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 1038336~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:24.045 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.342513+0000", 2024-09-16T20:48:24.045 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 1.604393615, 2024-09-16T20:48:24.045 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047233961999999997, 2024-09-16T20:48:24.045 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:24.045 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:24.045 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:24.046 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:24.046 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:24.046 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:24.046 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2068 2024-09-16T20:48:24.046 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.046 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:24.046 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.046 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:24.046 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342513+0000", 2024-09-16T20:48:24.046 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.046 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.047 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.047 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:24.047 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342513+0000", 2024-09-16T20:48:24.047 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.047 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.047 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.047 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:24.047 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342513+0000", 2024-09-16T20:48:24.047 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.01e-07 2024-09-16T20:48:24.047 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.047 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.047 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:24.047 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342515+0000", 2024-09-16T20:48:24.048 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7400000000000001e-06 2024-09-16T20:48:24.048 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.048 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.048 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:24.048 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342522+0000", 2024-09-16T20:48:24.048 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.516e-06 2024-09-16T20:48:24.048 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.048 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.048 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:24.048 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342527+0000", 2024-09-16T20:48:24.048 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0080000000000002e-06 2024-09-16T20:48:24.049 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.049 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.049 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:24.049 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389644+0000", 2024-09-16T20:48:24.049 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047117289 2024-09-16T20:48:24.049 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.049 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.049 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:24.049 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389690+0000", 2024-09-16T20:48:24.049 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6378000000000002e-05 2024-09-16T20:48:24.049 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.049 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.049 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:24.050 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389747+0000", 2024-09-16T20:48:24.050 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6629999999999998e-05 2024-09-16T20:48:24.050 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.050 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:24.050 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.050 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.050 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.050 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2070 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 1046528~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:24.050 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.342533+0000", 2024-09-16T20:48:24.050 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 1.6043735180000001, 2024-09-16T20:48:24.050 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047345848000000003, 2024-09-16T20:48:24.051 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:24.051 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:24.051 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:24.051 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:24.051 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:24.051 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:24.051 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2070 2024-09-16T20:48:24.051 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.051 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:24.051 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.051 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:24.051 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342533+0000", 2024-09-16T20:48:24.052 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.052 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.052 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:24.052 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342533+0000", 2024-09-16T20:48:24.052 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.052 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.052 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:24.052 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342533+0000", 2024-09-16T20:48:24.052 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7e-07 2024-09-16T20:48:24.053 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.053 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.053 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:24.053 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342535+0000", 2024-09-16T20:48:24.053 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.68e-06 2024-09-16T20:48:24.053 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.053 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.053 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:24.053 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342542+0000", 2024-09-16T20:48:24.053 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5169999999999996e-06 2024-09-16T20:48:24.053 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.053 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.054 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:24.054 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342546+0000", 2024-09-16T20:48:24.054 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8949999999999997e-06 2024-09-16T20:48:24.054 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.054 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.054 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:24.054 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389759+0000", 2024-09-16T20:48:24.054 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047212853999999999 2024-09-16T20:48:24.054 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.054 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.054 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:24.054 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389813+0000", 2024-09-16T20:48:24.055 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3585999999999998e-05 2024-09-16T20:48:24.055 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.055 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.055 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:24.055 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389879+0000", 2024-09-16T20:48:24.055 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5945999999999995e-05 2024-09-16T20:48:24.055 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.055 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:24.055 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.055 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.055 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:24.055 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:24.057 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:48:24.058 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:24.055+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:24.158 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:24.157+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:24.167 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:24.168 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:48:24.168 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.168 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:152 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 327680~32768 out=32768b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:48:24.168 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:48:24.168 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.0042663800000000002, 2024-09-16T20:48:24.168 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0042854310000000001, 2024-09-16T20:48:24.169 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:24.169 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:24.169 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:24.169 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:24.169 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:48:24.169 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:48:24.169 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 152 2024-09-16T20:48:24.169 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.169 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:24.169 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.169 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:24.169 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:48:24.170 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.170 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.170 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.170 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:24.170 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:48:24.170 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.170 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.170 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.170 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:24.170 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163122+0000", 2024-09-16T20:48:24.170 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6400000000000003e-07 2024-09-16T20:48:24.170 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.171 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.171 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:24.171 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163124+0000", 2024-09-16T20:48:24.171 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.565e-06 2024-09-16T20:48:24.171 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.171 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.171 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:24.171 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163136+0000", 2024-09-16T20:48:24.171 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1717e-05 2024-09-16T20:48:24.171 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.171 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.172 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:24.172 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163142+0000", 2024-09-16T20:48:24.172 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1750000000000002e-06 2024-09-16T20:48:24.172 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.172 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.172 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:24.172 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163606+0000", 2024-09-16T20:48:24.172 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00046418799999999998 2024-09-16T20:48:24.172 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.172 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.172 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:24.172 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163656+0000", 2024-09-16T20:48:24.173 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9852999999999999e-05 2024-09-16T20:48:24.173 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.173 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:24.173 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.173 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.173 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.173 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:153 3.1d 3.674d541d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:48:24.173 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:48:24.173 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.0042334269999999997, 2024-09-16T20:48:24.173 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0042874580000000001, 2024-09-16T20:48:24.173 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:24.174 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:24.174 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:48:24.174 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:24.174 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:48:24.174 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:48:24.174 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 153 2024-09-16T20:48:24.174 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.174 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:24.174 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.174 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:24.174 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:48:24.174 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.175 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.175 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.175 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:24.175 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:48:24.175 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.175 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.175 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.175 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:24.175 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163155+0000", 2024-09-16T20:48:24.175 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.37e-07 2024-09-16T20:48:24.175 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.175 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.176 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:24.176 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163157+0000", 2024-09-16T20:48:24.176 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6010000000000002e-06 2024-09-16T20:48:24.176 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.176 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.176 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:24.176 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163165+0000", 2024-09-16T20:48:24.176 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2619999999999998e-06 2024-09-16T20:48:24.176 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.176 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.176 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:24.176 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163171+0000", 2024-09-16T20:48:24.177 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.711e-06 2024-09-16T20:48:24.177 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.177 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:24.177 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.177 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.177 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.177 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:154 3.1d 3.674d541d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:48:24.177 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:48:24.177 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.0042016550000000003, 2024-09-16T20:48:24.177 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0042796800000000001, 2024-09-16T20:48:24.177 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:24.178 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:24.178 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:48:24.178 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:24.178 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:48:24.178 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:48:24.178 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 154 2024-09-16T20:48:24.178 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.178 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:24.178 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.178 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:24.178 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:48:24.178 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.179 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.179 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.179 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:24.179 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:48:24.179 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.179 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.179 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.179 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:24.179 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:48:24.179 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:48:24.179 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.180 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.180 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:24.180 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163189+0000", 2024-09-16T20:48:24.180 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6840000000000001e-06 2024-09-16T20:48:24.180 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.180 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.180 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:24.180 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163196+0000", 2024-09-16T20:48:24.180 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7920000000000004e-06 2024-09-16T20:48:24.180 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.180 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.181 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:24.181 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163201+0000", 2024-09-16T20:48:24.181 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2809999999999999e-06 2024-09-16T20:48:24.181 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.181 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:24.181 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.181 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.181 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-16T20:48:24.181 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 3 2024-09-16T20:48:24.181 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:24.182 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:48:24.282 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:24.258+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:24.291 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:24.291 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:48:24.291 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:48:24.291 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:48:24.291 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:24.300 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:48:24.365 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #5, image image_client.0-clone5 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent6 2024-09-16T20:48:24.389 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:24.382+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:24.439 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T20:48:24.462 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:24.462 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:48:24.462 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:48:24.462 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:48:24.462 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.462 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3709 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1464320~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:24.462 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.638971+0000", 2024-09-16T20:48:24.462 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 1.8212643559999999, 2024-09-16T20:48:24.462 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049858670000000001, 2024-09-16T20:48:24.462 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:24.462 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:24.463 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:24.463 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:24.463 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:24.463 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:24.463 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3709 2024-09-16T20:48:24.463 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.463 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:24.463 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.463 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:24.463 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638971+0000", 2024-09-16T20:48:24.463 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.463 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.463 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.464 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:24.464 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638971+0000", 2024-09-16T20:48:24.464 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.464 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.464 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.464 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:24.464 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638972+0000", 2024-09-16T20:48:24.464 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0400000000000002e-07 2024-09-16T20:48:24.464 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.464 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.464 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:24.464 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638973+0000", 2024-09-16T20:48:24.464 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.578e-06 2024-09-16T20:48:24.465 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.465 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.465 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:24.465 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638979+0000", 2024-09-16T20:48:24.465 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.502e-06 2024-09-16T20:48:24.465 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.465 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.465 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:24.465 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638983+0000", 2024-09-16T20:48:24.465 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2690000000000001e-06 2024-09-16T20:48:24.465 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.465 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.465 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:24.466 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688628+0000", 2024-09-16T20:48:24.466 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049644547999999997 2024-09-16T20:48:24.466 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.466 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.466 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:24.466 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688718+0000", 2024-09-16T20:48:24.466 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0802000000000002e-05 2024-09-16T20:48:24.466 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.466 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.466 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:24.466 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688830+0000", 2024-09-16T20:48:24.466 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000111667 2024-09-16T20:48:24.466 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.466 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:24.467 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.467 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.467 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.467 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3712 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1472512~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:24.467 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:48:24.467 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 1.821247828, 2024-09-16T20:48:24.467 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050004305999999998, 2024-09-16T20:48:24.467 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:24.467 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:24.467 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:24.467 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:24.467 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:24.467 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:24.468 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3712 2024-09-16T20:48:24.468 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.468 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:24.468 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.468 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:24.468 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:48:24.468 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.468 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.468 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.468 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:24.468 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:48:24.468 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.468 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.469 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.469 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:24.469 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:48:24.469 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0199999999999998e-07 2024-09-16T20:48:24.469 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.469 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.469 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:24.469 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638990+0000", 2024-09-16T20:48:24.469 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.618e-06 2024-09-16T20:48:24.469 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.469 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.469 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:24.469 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638995+0000", 2024-09-16T20:48:24.469 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3940000000000004e-06 2024-09-16T20:48:24.470 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.470 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.470 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:24.470 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639000+0000", 2024-09-16T20:48:24.470 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9309999999999999e-06 2024-09-16T20:48:24.470 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.470 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.470 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:24.470 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688867+0000", 2024-09-16T20:48:24.470 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049866955999999997 2024-09-16T20:48:24.470 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.470 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.470 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:24.470 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688933+0000", 2024-09-16T20:48:24.471 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5359999999999998e-05 2024-09-16T20:48:24.471 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.471 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.471 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:24.471 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688992+0000", 2024-09-16T20:48:24.471 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9744999999999998e-05 2024-09-16T20:48:24.471 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.471 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:24.471 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.471 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.471 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.471 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3714 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1480704~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:24.471 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:48:24.472 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 1.8212302380000001, 2024-09-16T20:48:24.472 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050152552000000003, 2024-09-16T20:48:24.472 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:24.472 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:24.472 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:24.472 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:24.472 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:24.472 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:24.472 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3714 2024-09-16T20:48:24.472 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.472 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:24.472 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.472 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:24.472 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:48:24.473 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.473 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.473 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.473 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:24.473 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:48:24.473 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.473 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.473 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.473 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:24.473 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:48:24.473 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0499999999999999e-07 2024-09-16T20:48:24.473 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.473 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.473 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:24.474 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639007+0000", 2024-09-16T20:48:24.474 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5340000000000001e-06 2024-09-16T20:48:24.474 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.474 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.474 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:24.474 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639013+0000", 2024-09-16T20:48:24.474 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3260000000000003e-06 2024-09-16T20:48:24.474 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.474 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.474 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:24.474 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639018+0000", 2024-09-16T20:48:24.474 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2850000000000002e-06 2024-09-16T20:48:24.474 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.475 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.475 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:24.475 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689026+0000", 2024-09-16T20:48:24.475 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.05000798 2024-09-16T20:48:24.475 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.475 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.475 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:24.475 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689077+0000", 2024-09-16T20:48:24.475 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1063999999999998e-05 2024-09-16T20:48:24.475 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.475 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.475 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:24.475 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689158+0000", 2024-09-16T20:48:24.475 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1057999999999995e-05 2024-09-16T20:48:24.476 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.476 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:24.476 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.476 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.476 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.476 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3716 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1488896~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:24.476 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:48:24.476 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 1.8212124279999999, 2024-09-16T20:48:24.476 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050280142, 2024-09-16T20:48:24.476 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:24.476 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:24.476 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:24.477 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:24.477 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:24.477 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:24.477 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3716 2024-09-16T20:48:24.477 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.477 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:24.477 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.477 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:24.477 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:48:24.477 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.477 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.477 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.477 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:24.477 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:48:24.478 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.478 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.478 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.478 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:24.478 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639024+0000", 2024-09-16T20:48:24.478 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9999999999999997e-07 2024-09-16T20:48:24.478 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.478 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.478 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:24.478 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639026+0000", 2024-09-16T20:48:24.478 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.781e-06 2024-09-16T20:48:24.478 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.478 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.479 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:24.479 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639032+0000", 2024-09-16T20:48:24.479 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7780000000000004e-06 2024-09-16T20:48:24.479 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.479 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.479 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:24.479 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639036+0000", 2024-09-16T20:48:24.479 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6909999999999996e-06 2024-09-16T20:48:24.479 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.479 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.479 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:24.479 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689191+0000", 2024-09-16T20:48:24.479 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050155284000000001 2024-09-16T20:48:24.479 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.480 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.480 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:24.480 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689242+0000", 2024-09-16T20:48:24.480 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0662999999999999e-05 2024-09-16T20:48:24.480 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.480 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.480 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:24.480 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689304+0000", 2024-09-16T20:48:24.480 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1345000000000003e-05 2024-09-16T20:48:24.480 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.480 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:24.480 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.480 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.480 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.481 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3719 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1497088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:24.481 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:48:24.481 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 1.82118515, 2024-09-16T20:48:24.481 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050399715999999997, 2024-09-16T20:48:24.481 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:24.481 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:24.481 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:24.481 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:24.481 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:24.481 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:24.481 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3719 2024-09-16T20:48:24.481 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.481 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:24.482 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.482 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:24.482 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:48:24.482 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.482 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.482 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.482 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:24.482 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:48:24.482 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.482 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.482 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.482 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:24.482 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:48:24.482 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9700000000000002e-07 2024-09-16T20:48:24.483 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.483 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.483 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:24.483 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639053+0000", 2024-09-16T20:48:24.483 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2900000000000001e-06 2024-09-16T20:48:24.483 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.483 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.483 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:24.483 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639060+0000", 2024-09-16T20:48:24.483 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7750000000000004e-06 2024-09-16T20:48:24.483 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.483 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.483 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:24.483 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639066+0000", 2024-09-16T20:48:24.484 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0639999999999999e-06 2024-09-16T20:48:24.484 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.484 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.484 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:24.484 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689337+0000", 2024-09-16T20:48:24.484 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050270784999999998 2024-09-16T20:48:24.484 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.484 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.484 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:24.484 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689392+0000", 2024-09-16T20:48:24.484 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4920999999999998e-05 2024-09-16T20:48:24.484 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.484 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.485 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:24.485 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689450+0000", 2024-09-16T20:48:24.485 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8483999999999998e-05 2024-09-16T20:48:24.485 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.485 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:24.485 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.485 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.485 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.485 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3721 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1505280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:24.485 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:48:24.485 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 1.8211620179999999, 2024-09-16T20:48:24.485 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050513608000000002, 2024-09-16T20:48:24.485 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:24.485 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:24.486 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:24.486 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:24.486 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:24.486 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:24.486 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3721 2024-09-16T20:48:24.486 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.486 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:24.486 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.486 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:24.486 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:48:24.486 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.486 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.486 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.486 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:24.487 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:48:24.487 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.487 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.487 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.487 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:24.487 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:48:24.487 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7e-07 2024-09-16T20:48:24.487 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.487 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.487 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:24.487 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639076+0000", 2024-09-16T20:48:24.487 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8899999999999999e-06 2024-09-16T20:48:24.487 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.487 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.488 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:24.488 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639083+0000", 2024-09-16T20:48:24.488 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1210000000000001e-06 2024-09-16T20:48:24.488 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.488 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.488 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:24.488 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639089+0000", 2024-09-16T20:48:24.488 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.643e-06 2024-09-16T20:48:24.488 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.488 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.488 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:24.488 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689474+0000", 2024-09-16T20:48:24.488 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050385679000000003 2024-09-16T20:48:24.489 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.489 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.489 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:24.489 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689533+0000", 2024-09-16T20:48:24.489 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8303000000000001e-05 2024-09-16T20:48:24.489 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.489 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.489 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:24.489 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689587+0000", 2024-09-16T20:48:24.489 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4601999999999997e-05 2024-09-16T20:48:24.489 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.489 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:24.489 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.489 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.490 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.490 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3723 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1513472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:24.490 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:48:24.490 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 1.821131214, 2024-09-16T20:48:24.490 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050699691999999998, 2024-09-16T20:48:24.490 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:24.490 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:24.490 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:24.490 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:24.490 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:24.490 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:24.490 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3723 2024-09-16T20:48:24.490 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.491 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:24.491 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.491 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:24.491 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:48:24.491 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.491 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.491 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.491 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:24.491 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:48:24.491 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.491 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.491 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.491 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:24.491 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:48:24.492 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3399999999999999e-07 2024-09-16T20:48:24.492 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.492 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.492 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:24.492 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639107+0000", 2024-09-16T20:48:24.492 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3860000000000001e-06 2024-09-16T20:48:24.492 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.492 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.492 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:24.492 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639114+0000", 2024-09-16T20:48:24.492 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9800000000000001e-06 2024-09-16T20:48:24.492 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.492 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.493 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:24.493 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639129+0000", 2024-09-16T20:48:24.493 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4887e-05 2024-09-16T20:48:24.493 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.493 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.493 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:24.493 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689625+0000", 2024-09-16T20:48:24.493 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050495848000000003 2024-09-16T20:48:24.493 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.493 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.493 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:24.493 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689720+0000", 2024-09-16T20:48:24.493 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4821000000000006e-05 2024-09-16T20:48:24.494 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.494 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.494 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:24.494 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689804+0000", 2024-09-16T20:48:24.494 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4235999999999998e-05 2024-09-16T20:48:24.494 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.494 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:24.494 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.494 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.494 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.494 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3725 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1521664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:24.494 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:48:24.494 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 1.821097618, 2024-09-16T20:48:24.495 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050797660000000001, 2024-09-16T20:48:24.495 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:24.495 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:24.495 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:24.495 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:24.495 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:24.495 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:24.495 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3725 2024-09-16T20:48:24.495 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.495 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:24.495 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.495 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:24.495 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:48:24.496 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.496 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.496 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.496 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:24.496 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:48:24.496 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.496 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.496 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.496 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:24.496 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639139+0000", 2024-09-16T20:48:24.496 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8999999999999997e-07 2024-09-16T20:48:24.496 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.496 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.497 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:24.497 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639141+0000", 2024-09-16T20:48:24.497 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.182e-06 2024-09-16T20:48:24.497 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.497 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.497 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:24.497 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639154+0000", 2024-09-16T20:48:24.497 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3626e-05 2024-09-16T20:48:24.497 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.497 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.497 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:24.497 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639160+0000", 2024-09-16T20:48:24.497 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6810000000000003e-06 2024-09-16T20:48:24.498 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.498 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.498 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:24.498 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689831+0000", 2024-09-16T20:48:24.498 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050670907000000001 2024-09-16T20:48:24.498 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.498 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.498 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:24.498 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689882+0000", 2024-09-16T20:48:24.498 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0445999999999998e-05 2024-09-16T20:48:24.498 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.498 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.498 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:24.499 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689936+0000", 2024-09-16T20:48:24.499 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4327999999999999e-05 2024-09-16T20:48:24.499 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.499 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:24.499 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.499 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.499 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.499 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3727 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1529856~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:24.499 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:48:24.499 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 1.821062041, 2024-09-16T20:48:24.499 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051016922999999999, 2024-09-16T20:48:24.499 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:24.499 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:24.500 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:24.500 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:24.500 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:24.500 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:24.500 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3727 2024-09-16T20:48:24.500 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.500 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:24.500 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.500 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:24.500 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:48:24.500 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.500 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.501 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.501 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:24.501 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:48:24.501 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.501 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.501 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.501 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:24.501 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:48:24.501 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9999999999999998e-07 2024-09-16T20:48:24.501 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.501 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.501 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:24.501 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639183+0000", 2024-09-16T20:48:24.502 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2399999999999996e-06 2024-09-16T20:48:24.502 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.502 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.502 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:24.502 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639195+0000", 2024-09-16T20:48:24.502 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1113e-05 2024-09-16T20:48:24.502 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.502 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.502 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:24.502 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639206+0000", 2024-09-16T20:48:24.502 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1503e-05 2024-09-16T20:48:24.502 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.502 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.502 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:24.503 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689969+0000", 2024-09-16T20:48:24.503 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050762676 2024-09-16T20:48:24.503 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.503 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.503 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:24.503 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690071+0000", 2024-09-16T20:48:24.503 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102163 2024-09-16T20:48:24.503 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.503 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.503 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:24.503 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690191+0000", 2024-09-16T20:48:24.503 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000119728 2024-09-16T20:48:24.503 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.503 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:24.504 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.504 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.504 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.504 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3730 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1538048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:24.504 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:48:24.504 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 1.8210204779999999, 2024-09-16T20:48:24.504 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051131467999999999, 2024-09-16T20:48:24.504 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:24.504 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:24.504 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:24.504 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:24.504 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:24.504 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:24.504 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3730 2024-09-16T20:48:24.505 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.505 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:24.505 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.505 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:24.505 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:48:24.505 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.505 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.505 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.505 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:24.505 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:48:24.505 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.505 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.505 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.506 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:24.506 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639216+0000", 2024-09-16T20:48:24.506 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.39e-07 2024-09-16T20:48:24.506 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.506 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.506 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:24.506 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639218+0000", 2024-09-16T20:48:24.506 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.537e-06 2024-09-16T20:48:24.506 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.506 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.506 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:24.506 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639229+0000", 2024-09-16T20:48:24.506 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1122e-05 2024-09-16T20:48:24.506 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.507 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.507 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:24.507 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639237+0000", 2024-09-16T20:48:24.507 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8509999999999999e-06 2024-09-16T20:48:24.507 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.507 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.507 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:24.507 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690227+0000", 2024-09-16T20:48:24.507 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050990147 2024-09-16T20:48:24.507 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.507 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.507 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:24.507 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690278+0000", 2024-09-16T20:48:24.507 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0223999999999999e-05 2024-09-16T20:48:24.508 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.508 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.508 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:24.508 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690347+0000", 2024-09-16T20:48:24.508 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9147999999999996e-05 2024-09-16T20:48:24.508 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.508 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:24.508 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.508 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.508 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.508 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3731 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1546240~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:24.508 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:48:24.508 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 1.82098977, 2024-09-16T20:48:24.509 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051276744999999999, 2024-09-16T20:48:24.509 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:24.509 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:24.509 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:24.509 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:24.509 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:24.509 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:24.509 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3731 2024-09-16T20:48:24.509 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.509 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:24.509 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.509 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:24.509 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:48:24.509 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.510 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.510 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.510 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:24.510 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:48:24.510 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.510 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.510 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.510 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:24.510 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:48:24.510 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9900000000000001e-07 2024-09-16T20:48:24.510 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.510 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.510 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:24.511 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639249+0000", 2024-09-16T20:48:24.511 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3630000000000001e-06 2024-09-16T20:48:24.511 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.511 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.511 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:24.511 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639255+0000", 2024-09-16T20:48:24.511 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4960000000000001e-06 2024-09-16T20:48:24.511 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.511 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.511 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:24.511 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639261+0000", 2024-09-16T20:48:24.511 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2839999999999997e-06 2024-09-16T20:48:24.511 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.511 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.512 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:24.512 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690375+0000", 2024-09-16T20:48:24.512 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051114684000000001 2024-09-16T20:48:24.512 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.512 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.512 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:24.512 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690425+0000", 2024-09-16T20:48:24.512 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9379999999999998e-05 2024-09-16T20:48:24.512 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.512 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.512 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:24.512 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690523+0000", 2024-09-16T20:48:24.513 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8139000000000003e-05 2024-09-16T20:48:24.513 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.513 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:24.513 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.513 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.513 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.513 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3732 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1554432~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:24.513 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:48:24.513 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 1.820943988, 2024-09-16T20:48:24.513 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051380096, 2024-09-16T20:48:24.513 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:24.513 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:24.513 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:24.513 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:24.514 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:24.514 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:24.514 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3732 2024-09-16T20:48:24.514 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.514 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:24.514 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.514 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:24.514 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:48:24.514 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.514 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.514 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.514 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:24.514 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:48:24.514 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.515 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.515 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.515 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:24.515 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:48:24.515 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6100000000000001e-07 2024-09-16T20:48:24.515 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.515 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.515 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:24.515 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639295+0000", 2024-09-16T20:48:24.515 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.447e-06 2024-09-16T20:48:24.515 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.515 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.515 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:24.516 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639302+0000", 2024-09-16T20:48:24.516 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0570000000000004e-06 2024-09-16T20:48:24.516 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.516 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.516 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:24.516 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639307+0000", 2024-09-16T20:48:24.516 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1429999999999997e-06 2024-09-16T20:48:24.516 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.516 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.516 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:24.516 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690555+0000", 2024-09-16T20:48:24.516 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051247963000000001 2024-09-16T20:48:24.516 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.517 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.517 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:24.517 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690602+0000", 2024-09-16T20:48:24.517 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7602000000000003e-05 2024-09-16T20:48:24.517 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.517 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.517 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:24.517 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690672+0000", 2024-09-16T20:48:24.517 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9423000000000003e-05 2024-09-16T20:48:24.517 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.517 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:24.517 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.517 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.517 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.518 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3734 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1562624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:24.518 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:48:24.518 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 1.8209223379999999, 2024-09-16T20:48:24.518 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051490610999999999, 2024-09-16T20:48:24.518 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:24.518 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:24.518 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:24.518 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:24.518 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:24.518 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:24.518 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3734 2024-09-16T20:48:24.518 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.518 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:24.518 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.519 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:24.519 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:48:24.519 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.519 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.519 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.519 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:24.519 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:48:24.519 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.519 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.519 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.519 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:24.519 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639314+0000", 2024-09-16T20:48:24.519 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4400000000000001e-07 2024-09-16T20:48:24.520 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.520 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.520 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:24.520 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639316+0000", 2024-09-16T20:48:24.520 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1459999999999999e-06 2024-09-16T20:48:24.520 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.520 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.520 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:24.520 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639322+0000", 2024-09-16T20:48:24.520 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7610000000000004e-06 2024-09-16T20:48:24.520 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.520 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.520 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:24.521 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639327+0000", 2024-09-16T20:48:24.521 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.186e-06 2024-09-16T20:48:24.521 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.521 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.521 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:24.521 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690698+0000", 2024-09-16T20:48:24.521 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051371223000000001 2024-09-16T20:48:24.521 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.521 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.521 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:24.521 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690748+0000", 2024-09-16T20:48:24.521 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0192999999999999e-05 2024-09-16T20:48:24.522 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.522 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.522 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:24.522 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690804+0000", 2024-09-16T20:48:24.522 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5658000000000001e-05 2024-09-16T20:48:24.522 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.522 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:24.522 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.522 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.523 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.523 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3736 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1570816~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:24.523 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:48:24.523 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 1.820903079, 2024-09-16T20:48:24.523 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051596528000000003, 2024-09-16T20:48:24.523 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:24.523 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:24.523 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:24.523 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:24.523 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:24.523 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:24.523 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3736 2024-09-16T20:48:24.523 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.524 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:24.524 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.524 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:24.524 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:48:24.524 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.524 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.524 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.524 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:24.524 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:48:24.524 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.524 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.524 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.524 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:24.525 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:48:24.525 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9999999999999999e-07 2024-09-16T20:48:24.525 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.525 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.525 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:24.525 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639335+0000", 2024-09-16T20:48:24.525 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.635e-06 2024-09-16T20:48:24.525 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.525 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.525 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:24.525 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639340+0000", 2024-09-16T20:48:24.525 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5840000000000002e-06 2024-09-16T20:48:24.525 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.526 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.526 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:24.526 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639346+0000", 2024-09-16T20:48:24.526 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1190000000000002e-06 2024-09-16T20:48:24.526 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.526 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.526 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:24.526 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690817+0000", 2024-09-16T20:48:24.526 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051470721999999997 2024-09-16T20:48:24.526 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.526 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.526 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:24.526 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690875+0000", 2024-09-16T20:48:24.527 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7441000000000001e-05 2024-09-16T20:48:24.527 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.527 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.527 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:24.527 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690929+0000", 2024-09-16T20:48:24.527 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4727e-05 2024-09-16T20:48:24.527 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.527 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:24.527 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.527 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.527 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.527 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5434 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:24.527 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:48:24.528 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 1.5956415129999999, 2024-09-16T20:48:24.528 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099755831000000003, 2024-09-16T20:48:24.528 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:24.528 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:24.528 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:24.528 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:24.528 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:24.528 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:24.528 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5434 2024-09-16T20:48:24.528 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.528 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:24.528 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.529 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:24.529 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:48:24.529 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.529 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.529 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.529 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:24.529 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:48:24.529 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.529 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.529 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.529 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:24.529 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864595+0000", 2024-09-16T20:48:24.529 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1099999999999995e-07 2024-09-16T20:48:24.529 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.530 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.530 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:24.530 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864597+0000", 2024-09-16T20:48:24.530 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3989999999999998e-06 2024-09-16T20:48:24.530 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.530 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.530 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:24.530 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864603+0000", 2024-09-16T20:48:24.530 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9309999999999996e-06 2024-09-16T20:48:24.530 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.530 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.530 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:24.530 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864607+0000", 2024-09-16T20:48:24.530 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6200000000000001e-06 2024-09-16T20:48:24.530 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.531 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.531 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:24.531 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964164+0000", 2024-09-16T20:48:24.531 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099557162000000005 2024-09-16T20:48:24.531 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.531 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.531 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:24.531 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964244+0000", 2024-09-16T20:48:24.531 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9597000000000004e-05 2024-09-16T20:48:24.531 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.531 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.531 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:24.531 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964350+0000", 2024-09-16T20:48:24.531 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010641099999999999 2024-09-16T20:48:24.531 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.532 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:24.532 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.532 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.532 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.532 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5435 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 20480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:24.532 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:48:24.532 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 1.595611691, 2024-09-16T20:48:24.532 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100897102, 2024-09-16T20:48:24.532 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:24.532 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:24.532 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:24.532 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:24.532 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:24.532 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:24.533 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5435 2024-09-16T20:48:24.533 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.533 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:24.533 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.533 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:24.533 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:48:24.533 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.533 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.533 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.533 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:24.533 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:48:24.533 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.534 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.534 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.534 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:24.534 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:48:24.534 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8000000000000001e-07 2024-09-16T20:48:24.534 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.534 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.534 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:24.534 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864626+0000", 2024-09-16T20:48:24.534 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.745e-06 2024-09-16T20:48:24.534 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.534 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.534 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:24.534 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864633+0000", 2024-09-16T20:48:24.535 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3339999999999998e-06 2024-09-16T20:48:24.535 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.535 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.535 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:24.535 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864636+0000", 2024-09-16T20:48:24.535 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1200000000000002e-06 2024-09-16T20:48:24.535 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.535 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.535 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:24.535 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965299+0000", 2024-09-16T20:48:24.535 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10066352000000001 2024-09-16T20:48:24.535 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.535 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.535 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:24.536 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965390+0000", 2024-09-16T20:48:24.536 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0894000000000001e-05 2024-09-16T20:48:24.536 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.536 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.536 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:24.536 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965521+0000", 2024-09-16T20:48:24.536 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013110899999999999 2024-09-16T20:48:24.536 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.536 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:24.536 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.536 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.536 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.536 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:152 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 327680~32768 out=32768b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:48:24.537 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:48:24.537 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.29711474900000001, 2024-09-16T20:48:24.537 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11869405800000001, 2024-09-16T20:48:24.537 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:24.537 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:24.537 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:24.537 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:24.537 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:48:24.537 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:48:24.537 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 152 2024-09-16T20:48:24.537 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.537 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:24.537 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.537 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:24.538 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:48:24.538 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.538 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.538 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.538 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:24.538 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:48:24.538 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.538 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.538 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.538 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:24.538 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163122+0000", 2024-09-16T20:48:24.538 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6400000000000003e-07 2024-09-16T20:48:24.538 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.538 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.539 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:24.539 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163124+0000", 2024-09-16T20:48:24.539 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.565e-06 2024-09-16T20:48:24.539 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.539 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.539 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:24.539 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163136+0000", 2024-09-16T20:48:24.539 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1717e-05 2024-09-16T20:48:24.539 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.539 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.539 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:24.539 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163142+0000", 2024-09-16T20:48:24.539 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1750000000000002e-06 2024-09-16T20:48:24.539 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.539 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.540 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:24.540 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163606+0000", 2024-09-16T20:48:24.540 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00046418799999999998 2024-09-16T20:48:24.540 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.540 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.540 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:24.540 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163656+0000", 2024-09-16T20:48:24.540 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9852999999999999e-05 2024-09-16T20:48:24.540 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.540 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.540 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:24.540 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281815+0000", 2024-09-16T20:48:24.540 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11815909600000001 2024-09-16T20:48:24.540 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.541 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:24.541 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.541 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.541 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.541 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:153 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:48:24.541 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:48:24.541 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.29708179600000001, 2024-09-16T20:48:24.541 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11903462400000001, 2024-09-16T20:48:24.541 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:24.541 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:24.541 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:24.541 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:24.541 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:48:24.541 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:48:24.542 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 153 2024-09-16T20:48:24.542 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.542 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:24.542 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.542 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:24.542 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:48:24.542 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.542 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.542 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.542 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:24.542 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:48:24.542 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.542 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.542 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.542 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:24.543 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163155+0000", 2024-09-16T20:48:24.543 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.37e-07 2024-09-16T20:48:24.543 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.543 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.543 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:24.543 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163157+0000", 2024-09-16T20:48:24.543 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6010000000000002e-06 2024-09-16T20:48:24.543 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.543 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.543 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:24.543 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163165+0000", 2024-09-16T20:48:24.543 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2619999999999998e-06 2024-09-16T20:48:24.543 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.543 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.543 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:24.544 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163171+0000", 2024-09-16T20:48:24.544 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.711e-06 2024-09-16T20:48:24.544 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.544 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.544 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:24.544 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281868+0000", 2024-09-16T20:48:24.544 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118696622 2024-09-16T20:48:24.544 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.544 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.544 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:24.544 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281969+0000", 2024-09-16T20:48:24.544 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101371 2024-09-16T20:48:24.544 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.545 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.545 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:24.545 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282189+0000", 2024-09-16T20:48:24.545 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00021952000000000001 2024-09-16T20:48:24.545 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.545 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:24.545 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.545 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.545 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.545 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:154 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 360448~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:48:24.545 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:48:24.545 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.29705002400000002, 2024-09-16T20:48:24.546 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11932035100000001, 2024-09-16T20:48:24.546 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:24.546 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:24.546 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:24.546 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:24.546 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:48:24.546 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:48:24.546 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 154 2024-09-16T20:48:24.546 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.546 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:24.546 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.546 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:24.547 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:48:24.547 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.547 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.547 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.547 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:24.547 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:48:24.547 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.547 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.547 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.547 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:24.547 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:48:24.547 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:48:24.547 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.547 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.548 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:24.548 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163189+0000", 2024-09-16T20:48:24.548 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6840000000000001e-06 2024-09-16T20:48:24.548 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.548 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.548 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:24.548 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163196+0000", 2024-09-16T20:48:24.548 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7920000000000004e-06 2024-09-16T20:48:24.548 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.548 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.548 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:24.548 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163201+0000", 2024-09-16T20:48:24.548 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2809999999999999e-06 2024-09-16T20:48:24.549 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.549 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.549 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:24.549 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282219+0000", 2024-09-16T20:48:24.549 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.119018343 2024-09-16T20:48:24.549 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.549 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.549 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:24.549 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282305+0000", 2024-09-16T20:48:24.549 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5865000000000005e-05 2024-09-16T20:48:24.549 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.549 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.549 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:24.550 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282506+0000", 2024-09-16T20:48:24.550 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000200927 2024-09-16T20:48:24.550 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.550 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:24.550 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.550 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.550 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.550 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop(client.4366.0:3088 3.1e e38/23 3:79e5ec01:::rbd_header.110e891a9b2b:head v 38'306, mlcod=36'304)", 2024-09-16T20:48:24.550 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:48:24.550 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.29603115600000002, 2024-09-16T20:48:24.550 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118102759, 2024-09-16T20:48:24.550 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:24.550 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:24.550 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:24.551 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:24.551 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.551 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:24.551 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:48:24.551 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.551 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.551 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.551 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:24.551 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:48:24.551 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:24.551 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.551 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.551 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:24.552 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:48:24.552 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.85e-07 2024-09-16T20:48:24.552 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.552 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.552 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:24.552 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164212+0000", 2024-09-16T20:48:24.552 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3999999999999997e-06 2024-09-16T20:48:24.552 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.552 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.552 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:24.552 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281553+0000", 2024-09-16T20:48:24.552 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11734135499999999 2024-09-16T20:48:24.552 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.552 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.553 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:24.553 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281567+0000", 2024-09-16T20:48:24.553 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4188999999999999e-05 2024-09-16T20:48:24.553 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.553 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.553 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:24.553 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281653+0000", 2024-09-16T20:48:24.553 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5385999999999999e-05 2024-09-16T20:48:24.553 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.553 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.553 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:24.553 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281679+0000", 2024-09-16T20:48:24.553 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6689000000000001e-05 2024-09-16T20:48:24.554 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.554 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.554 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:24.554 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282277+0000", 2024-09-16T20:48:24.554 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00059798000000000002 2024-09-16T20:48:24.554 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:24.554 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:24.554 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:24.554 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282307+0000", 2024-09-16T20:48:24.554 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9975e-05 2024-09-16T20:48:24.554 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.554 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:24.554 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.554 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:24.555 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:24.555 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:24.555 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:24.460+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:24.556 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:48:24.569 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-16T20:48:24.569 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T20:48:24.569 INFO:tasks.thrashosds.thrasher:Removing osd 3, in_osds are: [0, 1, 2, 3, 4, 5] 2024-09-16T20:48:24.569 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 3 2024-09-16T20:48:24.570 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:48:24.570 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:48:24.587 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:48:24.587 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:48:24.599 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:24.598+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:24.605 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:48:24.605 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:48:24.623 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:48:24.623 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:48:24.632 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T20:48:24.644 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:48:24.644 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:48:24.661 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:24.660+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:24.670 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:48:24.670 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:48:24.678 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:24.679 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:24.688 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:48:24.762 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:24.761+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:24.842 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:24.842 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:24.852 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:48:24.895 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:24.894+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:24.924 INFO:tasks.ceph.mon.a.smithi029.stderr:2024-09-16T20:48:24.923+0000 7fe0ce96d640 -1 mon.a@0(leader).osd e38 definitely_dead 0 2024-09-16T20:48:24.965 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:24.963+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:24.967 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:24.967 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:24.976 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:48:25.065 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:25.064+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:25.122 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:25.122 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:25.131 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:48:25.173 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:25.171+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:25.268 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:25.266+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:25.293 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:25.293 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:25.302 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:48:25.350 INFO:teuthology.orchestra.run.smithi139.stdout:890 read 0x32fb400 thru 0x3305e5a (0xaa5b bytes) 2024-09-16T20:48:25.352 INFO:teuthology.orchestra.run.smithi139.stdout:891 read 0x22d7c00 thru 0x22e36a7 (0xbaa8 bytes) 2024-09-16T20:48:25.368 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:25.367+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:25.377 INFO:teuthology.orchestra.run.smithi139.stdout:893 write 0xb31de00 thru 0xb31f329 (0x152a bytes) 2024-09-16T20:48:25.377 INFO:teuthology.orchestra.run.smithi139.stdout:895 write 0x272a000 thru 0x2738c20 (0xec21 bytes) 2024-09-16T20:48:25.377 INFO:teuthology.orchestra.run.smithi139.stdout:896 punch from 0x16e8200 to 0x16edc00, (0x5a00 bytes) 2024-09-16T20:48:25.383 INFO:teuthology.orchestra.run.smithi139.stdout:897 read 0x571a000 thru 0x571b285 (0x1286 bytes) 2024-09-16T20:48:25.395 INFO:teuthology.orchestra.run.smithi139.stdout:899 read 0x2583200 thru 0x258ff22 (0xcd23 bytes) 2024-09-16T20:48:25.405 INFO:teuthology.orchestra.run.smithi139.stdout:901 trunc from 0xb31f400 to 0x38a3600 2024-09-16T20:48:25.418 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:25.418 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:25.427 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:48:25.471 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:25.470+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:25.490 INFO:teuthology.orchestra.run.smithi139.stdout:902 trunc from 0x38a3600 to 0x72d7000 2024-09-16T20:48:25.505 INFO:teuthology.orchestra.run.smithi139.stdout:904 read 0x5411800 thru 0x541c21a (0xaa1b bytes) 2024-09-16T20:48:25.505 INFO:teuthology.orchestra.run.smithi139.stdout:905 read 0x275da00 thru 0x2766190 (0x8791 bytes) 2024-09-16T20:48:25.506 INFO:teuthology.orchestra.run.smithi139.stdout:906 read 0x37c3200 thru 0x37c3819 (0x61a bytes) 2024-09-16T20:48:25.506 INFO:teuthology.orchestra.run.smithi139.stdout:907 read 0x17f5a00 thru 0x17fa60f (0x4c10 bytes) 2024-09-16T20:48:25.507 INFO:teuthology.orchestra.run.smithi139.stdout:910 read 0x1d4c00 thru 0x1e3cf3 (0xf0f4 bytes) 2024-09-16T20:48:25.508 INFO:teuthology.orchestra.run.smithi139.stdout:913 read 0x6663a00 thru 0x6667fb2 (0x45b3 bytes) 2024-09-16T20:48:25.508 INFO:teuthology.orchestra.run.smithi139.stdout:914 read 0x6a5b400 thru 0x6a60a89 (0x568a bytes) 2024-09-16T20:48:25.508 INFO:teuthology.orchestra.run.smithi139.stdout:915 write 0x19b9800 thru 0x19c2cf7 (0x94f8 bytes) 2024-09-16T20:48:25.509 INFO:teuthology.orchestra.run.smithi139.stdout:916 write 0x7dd200 thru 0x7eccc2 (0xfac3 bytes) 2024-09-16T20:48:25.547 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:25.548 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:25.557 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:48:25.563 INFO:teuthology.orchestra.run.smithi139.stdout:917 write 0xb92c400 thru 0xb938389 (0xbf8a bytes) 2024-09-16T20:48:25.569 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:25.568+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:25.578 INFO:teuthology.orchestra.run.smithi139.stdout:918 write 0xe2d3a00 thru 0xe2d9c69 (0x626a bytes) 2024-09-16T20:48:25.580 INFO:teuthology.orchestra.run.smithi139.stdout:919 read 0x1a8ba00 thru 0x1a8d5e3 (0x1be4 bytes) 2024-09-16T20:48:25.580 INFO:teuthology.orchestra.run.smithi139.stdout:921 write 0xa7f9c00 thru 0xa8069b5 (0xcdb6 bytes) 2024-09-16T20:48:25.594 INFO:teuthology.orchestra.run.smithi139.stdout:923 trunc from 0xe2d9e00 to 0x1e48200 2024-09-16T20:48:25.629 INFO:teuthology.orchestra.run.smithi029.stderr:marked out osd.3. 2024-09-16T20:48:25.633 INFO:teuthology.orchestra.run.smithi029.stderr:noscrub is unset 2024-09-16T20:48:25.671 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:25.669+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:25.712 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:25.712 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:25.721 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:48:25.772 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:25.771+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:25.867 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:25.868 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:25.873 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:25.872+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:25.876 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:48:25.974 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:25.973+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:25.985 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:25.985 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:25.993 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:48:26.077 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:26.076+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:26.140 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:26.140 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:26.148 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:48:26.189 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:26.188+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:26.276 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:26.275+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:26.295 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:26.295 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:26.303 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:48:26.377 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:26.375+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:26.450 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:26.450 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:26.459 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:48:26.478 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:26.476+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:26.565 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:26.566 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:26.575 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:48:26.578 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:26.577+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:26.678 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:26.677+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:26.689 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T20:48:26.706 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:26.706 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:26.718 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:48:26.779 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:26.778+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:26.820 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:48:26.821 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:48:26.838 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:48:26.838 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:48:26.856 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:48:26.856 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:48:26.867 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:26.867 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:48:26.867 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:48:26.867 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:26.872 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:48:26.872 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:48:26.875 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:48:26.889 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:48:26.889 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:48:26.889 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:26.888+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:26.906 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:48:26.906 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:48:26.982 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:26.981+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:26.983 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:26.983 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:48:26.983 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:48:26.983 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:48:26.983 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:26.992 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:48:27.083 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:27.081+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:27.100 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:27.100 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:48:27.100 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:48:27.100 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:48:27.100 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.100 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3709 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1464320~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:27.100 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.638971+0000", 2024-09-16T20:48:27.100 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.4594859170000003, 2024-09-16T20:48:27.101 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049858670000000001, 2024-09-16T20:48:27.101 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:27.101 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:27.101 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:27.101 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:27.101 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:27.101 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:27.101 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3709 2024-09-16T20:48:27.101 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.101 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:27.101 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.101 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:27.101 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638971+0000", 2024-09-16T20:48:27.102 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:27.102 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.102 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.102 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:27.102 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638971+0000", 2024-09-16T20:48:27.102 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:27.102 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.102 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.102 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:27.102 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638972+0000", 2024-09-16T20:48:27.102 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0400000000000002e-07 2024-09-16T20:48:27.102 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.102 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.102 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:27.103 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638973+0000", 2024-09-16T20:48:27.103 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.578e-06 2024-09-16T20:48:27.103 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.103 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.103 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:27.103 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638979+0000", 2024-09-16T20:48:27.103 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.502e-06 2024-09-16T20:48:27.103 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.103 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.103 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:27.103 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638983+0000", 2024-09-16T20:48:27.103 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2690000000000001e-06 2024-09-16T20:48:27.103 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.103 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.104 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:27.104 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688628+0000", 2024-09-16T20:48:27.104 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049644547999999997 2024-09-16T20:48:27.104 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.104 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.104 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:27.104 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688718+0000", 2024-09-16T20:48:27.104 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0802000000000002e-05 2024-09-16T20:48:27.104 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.104 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.104 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:27.104 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688830+0000", 2024-09-16T20:48:27.104 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000111667 2024-09-16T20:48:27.104 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:27.105 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:27.105 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:27.105 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.105 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.105 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3712 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1472512~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:27.105 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:48:27.105 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.4594693889999997, 2024-09-16T20:48:27.105 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050004305999999998, 2024-09-16T20:48:27.105 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:27.105 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:27.105 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:27.105 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:27.105 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:27.106 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:27.106 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3712 2024-09-16T20:48:27.106 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.106 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:27.106 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.106 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:27.106 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:48:27.106 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:27.106 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.106 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.106 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:27.106 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:48:27.106 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:27.106 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.107 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.107 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:27.107 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:48:27.107 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0199999999999998e-07 2024-09-16T20:48:27.107 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.107 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.107 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:27.107 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638990+0000", 2024-09-16T20:48:27.107 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.618e-06 2024-09-16T20:48:27.107 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.107 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.107 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:27.107 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638995+0000", 2024-09-16T20:48:27.107 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3940000000000004e-06 2024-09-16T20:48:27.108 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.108 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.108 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:27.108 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639000+0000", 2024-09-16T20:48:27.108 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9309999999999999e-06 2024-09-16T20:48:27.108 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.108 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.108 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:27.108 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688867+0000", 2024-09-16T20:48:27.108 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049866955999999997 2024-09-16T20:48:27.108 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.108 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.108 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:27.108 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688933+0000", 2024-09-16T20:48:27.109 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5359999999999998e-05 2024-09-16T20:48:27.109 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.109 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.109 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:27.109 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688992+0000", 2024-09-16T20:48:27.109 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9744999999999998e-05 2024-09-16T20:48:27.109 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:27.109 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:27.109 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:27.109 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.109 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.109 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3714 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1480704~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:27.109 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:48:27.109 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.459451799, 2024-09-16T20:48:27.110 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050152552000000003, 2024-09-16T20:48:27.110 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:27.110 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:27.110 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:27.110 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:27.110 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:27.110 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:27.110 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3714 2024-09-16T20:48:27.110 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.110 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:27.110 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.110 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:27.110 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:48:27.110 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:27.111 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.111 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.111 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:27.111 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:48:27.111 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:27.111 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.111 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.111 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:27.111 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:48:27.111 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0499999999999999e-07 2024-09-16T20:48:27.111 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.111 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.111 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:27.111 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639007+0000", 2024-09-16T20:48:27.111 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5340000000000001e-06 2024-09-16T20:48:27.112 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.112 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.112 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:27.112 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639013+0000", 2024-09-16T20:48:27.112 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3260000000000003e-06 2024-09-16T20:48:27.112 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.112 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.112 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:27.112 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639018+0000", 2024-09-16T20:48:27.112 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2850000000000002e-06 2024-09-16T20:48:27.112 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.112 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.112 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:27.113 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689026+0000", 2024-09-16T20:48:27.113 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.05000798 2024-09-16T20:48:27.113 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.113 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.113 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:27.113 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689077+0000", 2024-09-16T20:48:27.113 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1063999999999998e-05 2024-09-16T20:48:27.113 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.113 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.113 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:27.113 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689158+0000", 2024-09-16T20:48:27.113 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1057999999999995e-05 2024-09-16T20:48:27.113 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:27.113 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:27.113 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:27.114 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.114 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.114 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3716 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1488896~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:27.114 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:48:27.114 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.4594339889999999, 2024-09-16T20:48:27.114 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050280142, 2024-09-16T20:48:27.114 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:27.114 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:27.114 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:27.114 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:27.114 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:27.114 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:27.114 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3716 2024-09-16T20:48:27.115 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.115 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:27.115 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.115 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:27.115 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:48:27.115 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:27.115 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.115 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.115 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:27.115 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:48:27.115 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:27.115 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.115 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.115 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:27.116 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639024+0000", 2024-09-16T20:48:27.116 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9999999999999997e-07 2024-09-16T20:48:27.116 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.116 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.116 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:27.116 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639026+0000", 2024-09-16T20:48:27.116 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.781e-06 2024-09-16T20:48:27.116 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.116 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.116 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:27.116 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639032+0000", 2024-09-16T20:48:27.116 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7780000000000004e-06 2024-09-16T20:48:27.116 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.116 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.117 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:27.117 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639036+0000", 2024-09-16T20:48:27.117 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6909999999999996e-06 2024-09-16T20:48:27.117 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.117 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.117 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:27.117 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689191+0000", 2024-09-16T20:48:27.117 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050155284000000001 2024-09-16T20:48:27.117 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.117 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.117 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:27.117 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689242+0000", 2024-09-16T20:48:27.117 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0662999999999999e-05 2024-09-16T20:48:27.117 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.118 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.118 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:27.118 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689304+0000", 2024-09-16T20:48:27.118 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1345000000000003e-05 2024-09-16T20:48:27.118 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:27.118 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:27.118 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:27.118 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.118 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.118 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3719 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1497088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:27.118 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:48:27.118 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.4594067109999997, 2024-09-16T20:48:27.118 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050399715999999997, 2024-09-16T20:48:27.118 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:27.119 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:27.119 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:27.119 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:27.119 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:27.119 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:27.119 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3719 2024-09-16T20:48:27.119 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.119 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:27.119 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.119 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:27.119 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:48:27.119 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:27.119 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.120 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.120 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:27.120 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:48:27.120 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:27.120 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.120 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.120 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:27.120 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:48:27.120 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9700000000000002e-07 2024-09-16T20:48:27.120 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.120 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.120 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:27.120 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639053+0000", 2024-09-16T20:48:27.120 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2900000000000001e-06 2024-09-16T20:48:27.121 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.121 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.121 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:27.121 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639060+0000", 2024-09-16T20:48:27.121 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7750000000000004e-06 2024-09-16T20:48:27.121 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.121 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.121 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:27.121 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639066+0000", 2024-09-16T20:48:27.121 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0639999999999999e-06 2024-09-16T20:48:27.121 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.121 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.121 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:27.121 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689337+0000", 2024-09-16T20:48:27.122 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050270784999999998 2024-09-16T20:48:27.122 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.122 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.122 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:27.122 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689392+0000", 2024-09-16T20:48:27.122 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4920999999999998e-05 2024-09-16T20:48:27.122 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.122 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.122 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:27.122 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689450+0000", 2024-09-16T20:48:27.122 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8483999999999998e-05 2024-09-16T20:48:27.122 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:27.122 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:27.122 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:27.123 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.123 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.123 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3721 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1505280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:27.123 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:48:27.123 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.4593835789999998, 2024-09-16T20:48:27.123 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050513608000000002, 2024-09-16T20:48:27.123 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:27.123 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:27.123 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:27.123 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:27.123 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:27.123 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:27.123 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3721 2024-09-16T20:48:27.123 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.124 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:27.124 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.124 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:27.124 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:48:27.124 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:27.124 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.124 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.124 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:27.124 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:48:27.124 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:27.124 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.124 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.124 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:27.125 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:48:27.125 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7e-07 2024-09-16T20:48:27.125 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.125 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.125 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:27.125 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639076+0000", 2024-09-16T20:48:27.125 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8899999999999999e-06 2024-09-16T20:48:27.125 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.125 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.125 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:27.125 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639083+0000", 2024-09-16T20:48:27.125 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1210000000000001e-06 2024-09-16T20:48:27.125 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.125 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.125 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:27.126 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639089+0000", 2024-09-16T20:48:27.126 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.643e-06 2024-09-16T20:48:27.126 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.126 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.126 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:27.126 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689474+0000", 2024-09-16T20:48:27.126 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050385679000000003 2024-09-16T20:48:27.126 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.126 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.126 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:27.126 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689533+0000", 2024-09-16T20:48:27.126 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8303000000000001e-05 2024-09-16T20:48:27.126 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.126 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.127 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:27.127 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689587+0000", 2024-09-16T20:48:27.127 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4601999999999997e-05 2024-09-16T20:48:27.127 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:27.127 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:27.127 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:27.127 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.127 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.127 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3723 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1513472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:27.127 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:48:27.127 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.4593527750000002, 2024-09-16T20:48:27.127 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050699691999999998, 2024-09-16T20:48:27.127 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:27.127 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:27.127 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:27.128 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:27.128 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:27.128 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:27.128 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3723 2024-09-16T20:48:27.128 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.128 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:27.128 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.128 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:27.128 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:48:27.128 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:27.128 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.128 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.128 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:27.129 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:48:27.129 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:27.129 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.129 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.129 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:27.129 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:48:27.129 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3399999999999999e-07 2024-09-16T20:48:27.129 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.129 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.129 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:27.129 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639107+0000", 2024-09-16T20:48:27.129 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3860000000000001e-06 2024-09-16T20:48:27.129 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.130 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.130 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:27.130 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639114+0000", 2024-09-16T20:48:27.130 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9800000000000001e-06 2024-09-16T20:48:27.130 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.130 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.130 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:27.130 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639129+0000", 2024-09-16T20:48:27.130 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4887e-05 2024-09-16T20:48:27.130 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.130 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.130 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:27.130 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689625+0000", 2024-09-16T20:48:27.130 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050495848000000003 2024-09-16T20:48:27.131 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.131 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.131 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:27.131 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689720+0000", 2024-09-16T20:48:27.131 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4821000000000006e-05 2024-09-16T20:48:27.131 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.131 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.131 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:27.131 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689804+0000", 2024-09-16T20:48:27.131 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4235999999999998e-05 2024-09-16T20:48:27.131 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:27.131 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:27.131 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:27.131 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.132 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.132 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3725 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1521664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:27.132 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:48:27.132 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.4593191790000004, 2024-09-16T20:48:27.132 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050797660000000001, 2024-09-16T20:48:27.132 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:27.132 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:27.132 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:27.132 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:27.132 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:27.132 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:27.132 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3725 2024-09-16T20:48:27.132 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.133 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:27.133 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.133 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:27.133 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:48:27.133 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:27.133 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.133 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.133 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:27.133 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:48:27.133 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:27.133 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.133 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.133 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:27.133 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639139+0000", 2024-09-16T20:48:27.134 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8999999999999997e-07 2024-09-16T20:48:27.134 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.134 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.134 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:27.134 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639141+0000", 2024-09-16T20:48:27.134 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.182e-06 2024-09-16T20:48:27.134 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.134 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.134 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:27.134 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639154+0000", 2024-09-16T20:48:27.134 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3626e-05 2024-09-16T20:48:27.134 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.134 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.134 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:27.135 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639160+0000", 2024-09-16T20:48:27.135 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6810000000000003e-06 2024-09-16T20:48:27.135 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.135 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.135 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:27.135 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689831+0000", 2024-09-16T20:48:27.135 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050670907000000001 2024-09-16T20:48:27.135 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.135 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.135 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:27.135 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689882+0000", 2024-09-16T20:48:27.135 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0445999999999998e-05 2024-09-16T20:48:27.135 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.135 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.135 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:27.136 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689936+0000", 2024-09-16T20:48:27.136 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4327999999999999e-05 2024-09-16T20:48:27.136 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:27.136 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:27.136 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:27.136 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.136 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.136 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3727 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1529856~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:27.136 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:48:27.136 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.4592836020000002, 2024-09-16T20:48:27.136 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051016922999999999, 2024-09-16T20:48:27.136 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:27.136 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:27.137 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:27.137 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:27.137 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:27.137 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:27.137 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3727 2024-09-16T20:48:27.137 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.137 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:27.137 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.137 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:27.137 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:48:27.137 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:27.137 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.137 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.137 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:27.138 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:48:27.138 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:27.138 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.138 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.138 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:27.138 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:48:27.138 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9999999999999998e-07 2024-09-16T20:48:27.138 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.138 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.138 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:27.138 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639183+0000", 2024-09-16T20:48:27.138 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2399999999999996e-06 2024-09-16T20:48:27.138 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.138 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.139 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:27.139 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639195+0000", 2024-09-16T20:48:27.139 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1113e-05 2024-09-16T20:48:27.139 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.139 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.139 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:27.139 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639206+0000", 2024-09-16T20:48:27.139 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1503e-05 2024-09-16T20:48:27.139 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.139 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.139 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:27.139 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689969+0000", 2024-09-16T20:48:27.139 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050762676 2024-09-16T20:48:27.140 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.140 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.140 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:27.140 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690071+0000", 2024-09-16T20:48:27.140 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102163 2024-09-16T20:48:27.140 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.140 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.140 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:27.140 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690191+0000", 2024-09-16T20:48:27.140 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000119728 2024-09-16T20:48:27.140 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:27.140 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:27.140 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:27.141 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.141 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.141 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3730 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1538048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:27.141 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:48:27.141 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.4592420390000003, 2024-09-16T20:48:27.141 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051131467999999999, 2024-09-16T20:48:27.141 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:27.141 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:27.141 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:27.141 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:27.141 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:27.141 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:27.141 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3730 2024-09-16T20:48:27.141 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.142 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:27.142 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.142 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:27.142 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:48:27.142 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:27.142 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.142 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.142 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:27.142 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:48:27.142 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:27.142 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.142 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.142 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:27.143 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639216+0000", 2024-09-16T20:48:27.143 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.39e-07 2024-09-16T20:48:27.143 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.143 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.143 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:27.143 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639218+0000", 2024-09-16T20:48:27.143 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.537e-06 2024-09-16T20:48:27.143 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.143 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.143 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:27.143 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639229+0000", 2024-09-16T20:48:27.143 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1122e-05 2024-09-16T20:48:27.143 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.143 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.144 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:27.144 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639237+0000", 2024-09-16T20:48:27.144 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8509999999999999e-06 2024-09-16T20:48:27.144 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.144 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.144 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:27.144 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690227+0000", 2024-09-16T20:48:27.144 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050990147 2024-09-16T20:48:27.144 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.144 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.144 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:27.144 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690278+0000", 2024-09-16T20:48:27.144 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0223999999999999e-05 2024-09-16T20:48:27.144 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.145 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.145 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:27.145 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690347+0000", 2024-09-16T20:48:27.145 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9147999999999996e-05 2024-09-16T20:48:27.145 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:27.145 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:27.145 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:27.145 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.145 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.145 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3731 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1546240~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:27.145 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:48:27.145 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.4592113309999997, 2024-09-16T20:48:27.145 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051276744999999999, 2024-09-16T20:48:27.145 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:27.146 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:27.146 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:27.146 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:27.146 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:27.147 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:27.147 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3731 2024-09-16T20:48:27.147 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.147 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:27.147 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.147 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:27.147 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:48:27.147 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:27.147 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.147 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.147 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:27.147 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:48:27.147 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:27.147 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.148 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.148 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:27.148 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:48:27.148 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9900000000000001e-07 2024-09-16T20:48:27.148 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.148 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.148 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:27.148 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639249+0000", 2024-09-16T20:48:27.148 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3630000000000001e-06 2024-09-16T20:48:27.148 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.148 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.148 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:27.148 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639255+0000", 2024-09-16T20:48:27.149 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4960000000000001e-06 2024-09-16T20:48:27.149 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.149 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.149 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:27.149 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639261+0000", 2024-09-16T20:48:27.149 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2839999999999997e-06 2024-09-16T20:48:27.149 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.149 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.149 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:27.149 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690375+0000", 2024-09-16T20:48:27.149 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051114684000000001 2024-09-16T20:48:27.149 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.149 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.149 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:27.150 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690425+0000", 2024-09-16T20:48:27.150 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9379999999999998e-05 2024-09-16T20:48:27.150 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.150 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.150 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:27.150 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690523+0000", 2024-09-16T20:48:27.150 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8139000000000003e-05 2024-09-16T20:48:27.150 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:27.150 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:27.150 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:27.150 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.151 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.151 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3732 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1554432~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:27.151 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:48:27.151 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.4591655489999997, 2024-09-16T20:48:27.151 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051380096, 2024-09-16T20:48:27.151 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:27.151 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:27.151 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:27.151 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:27.151 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:27.151 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:27.151 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3732 2024-09-16T20:48:27.151 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.152 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:27.152 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.152 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:27.152 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:48:27.152 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:27.152 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.152 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.152 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:27.152 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:48:27.152 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:27.152 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.152 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.152 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:27.152 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:48:27.153 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6100000000000001e-07 2024-09-16T20:48:27.153 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.153 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.153 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:27.153 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639295+0000", 2024-09-16T20:48:27.153 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.447e-06 2024-09-16T20:48:27.153 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.153 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.153 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:27.153 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639302+0000", 2024-09-16T20:48:27.153 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0570000000000004e-06 2024-09-16T20:48:27.153 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.153 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.153 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:27.154 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639307+0000", 2024-09-16T20:48:27.154 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1429999999999997e-06 2024-09-16T20:48:27.154 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.154 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.154 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:27.154 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690555+0000", 2024-09-16T20:48:27.154 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051247963000000001 2024-09-16T20:48:27.154 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.154 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.154 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:27.154 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690602+0000", 2024-09-16T20:48:27.154 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7602000000000003e-05 2024-09-16T20:48:27.154 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.154 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.155 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:27.155 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690672+0000", 2024-09-16T20:48:27.155 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9423000000000003e-05 2024-09-16T20:48:27.155 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:27.155 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:27.155 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:27.155 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.155 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.155 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3734 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1562624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:27.155 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:48:27.155 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.4591438989999999, 2024-09-16T20:48:27.155 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051490610999999999, 2024-09-16T20:48:27.155 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:27.155 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:27.156 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:27.156 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:27.156 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:27.156 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:27.156 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3734 2024-09-16T20:48:27.156 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.156 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:27.156 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.156 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:27.156 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:48:27.156 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:27.156 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.156 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.156 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:27.157 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:48:27.157 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:27.157 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.157 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.157 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:27.157 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639314+0000", 2024-09-16T20:48:27.157 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4400000000000001e-07 2024-09-16T20:48:27.157 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.157 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.157 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:27.157 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639316+0000", 2024-09-16T20:48:27.157 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1459999999999999e-06 2024-09-16T20:48:27.157 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.157 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.158 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:27.158 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639322+0000", 2024-09-16T20:48:27.158 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7610000000000004e-06 2024-09-16T20:48:27.158 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.158 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.158 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:27.158 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639327+0000", 2024-09-16T20:48:27.158 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.186e-06 2024-09-16T20:48:27.158 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.158 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.158 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:27.158 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690698+0000", 2024-09-16T20:48:27.158 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051371223000000001 2024-09-16T20:48:27.158 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.159 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.159 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:27.159 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690748+0000", 2024-09-16T20:48:27.159 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0192999999999999e-05 2024-09-16T20:48:27.159 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.159 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.159 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:27.159 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690804+0000", 2024-09-16T20:48:27.159 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5658000000000001e-05 2024-09-16T20:48:27.159 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:27.159 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:27.159 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:27.159 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.160 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.160 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3736 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1570816~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:27.160 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:48:27.160 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.4591246399999998, 2024-09-16T20:48:27.160 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051596528000000003, 2024-09-16T20:48:27.160 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:27.160 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:27.160 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:27.160 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:27.160 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:27.160 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:27.160 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3736 2024-09-16T20:48:27.160 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.160 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:27.161 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.161 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:27.161 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:48:27.161 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:27.161 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.161 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.161 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:27.161 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:48:27.161 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:27.161 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.161 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.161 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:27.161 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:48:27.161 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9999999999999999e-07 2024-09-16T20:48:27.162 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.162 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.162 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:27.162 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639335+0000", 2024-09-16T20:48:27.162 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.635e-06 2024-09-16T20:48:27.162 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.162 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.162 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:27.162 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639340+0000", 2024-09-16T20:48:27.162 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5840000000000002e-06 2024-09-16T20:48:27.162 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.162 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.162 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:27.162 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639346+0000", 2024-09-16T20:48:27.163 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1190000000000002e-06 2024-09-16T20:48:27.163 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.163 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.163 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:27.163 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690817+0000", 2024-09-16T20:48:27.163 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051470721999999997 2024-09-16T20:48:27.163 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.163 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.163 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:27.163 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690875+0000", 2024-09-16T20:48:27.163 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7441000000000001e-05 2024-09-16T20:48:27.163 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.163 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.163 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:27.163 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690929+0000", 2024-09-16T20:48:27.164 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4727e-05 2024-09-16T20:48:27.164 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:27.164 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:27.164 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:27.164 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.164 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.164 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5434 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:27.164 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:48:27.164 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.2338630740000003, 2024-09-16T20:48:27.164 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099755831000000003, 2024-09-16T20:48:27.164 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:27.164 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:27.164 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:27.164 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:27.164 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:27.165 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:27.165 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5434 2024-09-16T20:48:27.165 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.165 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:27.165 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.165 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:27.165 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:48:27.165 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:27.165 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.165 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.165 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:27.165 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:48:27.165 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:27.166 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.166 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.166 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:27.166 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864595+0000", 2024-09-16T20:48:27.166 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1099999999999995e-07 2024-09-16T20:48:27.166 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.166 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.166 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:27.166 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864597+0000", 2024-09-16T20:48:27.166 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3989999999999998e-06 2024-09-16T20:48:27.166 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.166 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.166 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:27.166 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864603+0000", 2024-09-16T20:48:27.167 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9309999999999996e-06 2024-09-16T20:48:27.167 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.167 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.167 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:27.167 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864607+0000", 2024-09-16T20:48:27.167 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6200000000000001e-06 2024-09-16T20:48:27.167 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.167 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.167 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:27.167 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964164+0000", 2024-09-16T20:48:27.167 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099557162000000005 2024-09-16T20:48:27.167 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.167 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.167 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:27.168 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964244+0000", 2024-09-16T20:48:27.168 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9597000000000004e-05 2024-09-16T20:48:27.168 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.168 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.168 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:27.168 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964350+0000", 2024-09-16T20:48:27.168 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010641099999999999 2024-09-16T20:48:27.168 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:27.168 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:27.168 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:27.168 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.168 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.168 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5435 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 20480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:27.168 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:48:27.169 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.2338332520000002, 2024-09-16T20:48:27.169 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100897102, 2024-09-16T20:48:27.169 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:27.169 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:27.169 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:27.169 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:27.169 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:27.169 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:27.169 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5435 2024-09-16T20:48:27.169 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.169 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:27.169 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.169 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:27.169 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:48:27.170 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:27.170 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.170 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.170 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:27.170 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:48:27.170 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:27.170 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.170 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.170 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:27.170 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:48:27.170 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8000000000000001e-07 2024-09-16T20:48:27.170 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.170 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.171 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:27.171 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864626+0000", 2024-09-16T20:48:27.171 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.745e-06 2024-09-16T20:48:27.171 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.171 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.171 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:27.171 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864633+0000", 2024-09-16T20:48:27.171 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3339999999999998e-06 2024-09-16T20:48:27.171 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.171 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.171 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:27.171 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864636+0000", 2024-09-16T20:48:27.171 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1200000000000002e-06 2024-09-16T20:48:27.171 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.172 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.172 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:27.172 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965299+0000", 2024-09-16T20:48:27.172 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10066352000000001 2024-09-16T20:48:27.172 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.172 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.172 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:27.172 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965390+0000", 2024-09-16T20:48:27.172 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0894000000000001e-05 2024-09-16T20:48:27.172 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.172 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.172 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:27.172 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965521+0000", 2024-09-16T20:48:27.172 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013110899999999999 2024-09-16T20:48:27.173 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:27.173 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:27.173 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:27.173 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.173 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.173 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:152 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 327680~32768 out=32768b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:48:27.173 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:48:27.173 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 2.9353363099999998, 2024-09-16T20:48:27.173 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11869405800000001, 2024-09-16T20:48:27.173 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:27.173 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:27.173 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:27.173 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:27.173 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:48:27.173 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:48:27.174 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 152 2024-09-16T20:48:27.174 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.174 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:27.174 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.174 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:27.174 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:48:27.174 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:27.174 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.174 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.174 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:27.174 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:48:27.174 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:27.174 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.174 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.175 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:27.175 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163122+0000", 2024-09-16T20:48:27.175 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6400000000000003e-07 2024-09-16T20:48:27.175 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.175 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.175 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:27.175 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163124+0000", 2024-09-16T20:48:27.175 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.565e-06 2024-09-16T20:48:27.175 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.175 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.175 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:27.175 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163136+0000", 2024-09-16T20:48:27.175 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1717e-05 2024-09-16T20:48:27.176 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.176 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.176 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:27.176 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163142+0000", 2024-09-16T20:48:27.176 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1750000000000002e-06 2024-09-16T20:48:27.176 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.176 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.176 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:27.176 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163606+0000", 2024-09-16T20:48:27.176 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00046418799999999998 2024-09-16T20:48:27.176 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.176 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.176 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:27.176 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163656+0000", 2024-09-16T20:48:27.177 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9852999999999999e-05 2024-09-16T20:48:27.177 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.177 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.177 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:27.177 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281815+0000", 2024-09-16T20:48:27.177 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11815909600000001 2024-09-16T20:48:27.177 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:27.177 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:27.177 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:27.177 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.177 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.177 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:153 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:48:27.177 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:48:27.177 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 2.935303357, 2024-09-16T20:48:27.178 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11903462400000001, 2024-09-16T20:48:27.178 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:27.178 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:27.178 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:27.178 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:27.178 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:48:27.178 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:48:27.178 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 153 2024-09-16T20:48:27.178 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.178 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:27.178 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.178 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:27.178 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:48:27.178 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:27.179 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.179 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.179 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:27.179 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:48:27.179 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:27.179 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.179 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.179 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:27.179 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163155+0000", 2024-09-16T20:48:27.179 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.37e-07 2024-09-16T20:48:27.179 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.179 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.179 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:27.179 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163157+0000", 2024-09-16T20:48:27.180 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6010000000000002e-06 2024-09-16T20:48:27.180 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.180 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.180 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:27.180 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163165+0000", 2024-09-16T20:48:27.180 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2619999999999998e-06 2024-09-16T20:48:27.180 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.180 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.180 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:27.180 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163171+0000", 2024-09-16T20:48:27.180 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.711e-06 2024-09-16T20:48:27.180 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.180 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.180 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:27.181 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281868+0000", 2024-09-16T20:48:27.181 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118696622 2024-09-16T20:48:27.181 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.181 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.181 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:27.181 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281969+0000", 2024-09-16T20:48:27.181 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101371 2024-09-16T20:48:27.181 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.181 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.181 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:27.181 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282189+0000", 2024-09-16T20:48:27.181 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00021952000000000001 2024-09-16T20:48:27.181 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:27.181 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:27.182 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:27.182 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.182 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.182 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:154 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 360448~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:48:27.182 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:48:27.182 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 2.9352715849999997, 2024-09-16T20:48:27.182 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11932035100000001, 2024-09-16T20:48:27.182 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:27.182 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:27.182 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:27.182 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:27.182 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:48:27.182 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:48:27.182 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 154 2024-09-16T20:48:27.183 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.183 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:27.183 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.183 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:27.183 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:48:27.183 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:27.183 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.183 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.183 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:27.183 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:48:27.183 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:27.183 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.183 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.183 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:27.184 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:48:27.184 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:48:27.184 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.184 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.184 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:27.184 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163189+0000", 2024-09-16T20:48:27.184 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6840000000000001e-06 2024-09-16T20:48:27.184 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.184 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.184 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:27.184 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163196+0000", 2024-09-16T20:48:27.184 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7920000000000004e-06 2024-09-16T20:48:27.184 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.184 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.185 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:27.185 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163201+0000", 2024-09-16T20:48:27.185 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2809999999999999e-06 2024-09-16T20:48:27.185 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.185 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.185 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:27.185 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282219+0000", 2024-09-16T20:48:27.185 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.119018343 2024-09-16T20:48:27.185 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.185 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.185 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:27.185 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282305+0000", 2024-09-16T20:48:27.185 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5865000000000005e-05 2024-09-16T20:48:27.185 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.186 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.186 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:27.186 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282506+0000", 2024-09-16T20:48:27.186 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000200927 2024-09-16T20:48:27.186 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:27.186 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:27.186 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:27.186 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.186 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.186 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop(client.4366.0:3088 3.1e e38/23 3:79e5ec01:::rbd_header.110e891a9b2b:head v 38'306, mlcod=36'304)", 2024-09-16T20:48:27.186 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:48:27.186 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 2.9342527170000001, 2024-09-16T20:48:27.186 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118102759, 2024-09-16T20:48:27.187 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:27.187 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:27.187 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:27.187 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:27.187 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.187 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:27.187 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:48:27.187 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:27.187 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.187 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.187 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:27.187 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:48:27.187 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:27.187 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.188 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.188 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:27.188 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:48:27.188 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.85e-07 2024-09-16T20:48:27.188 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.188 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.188 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:27.188 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164212+0000", 2024-09-16T20:48:27.188 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3999999999999997e-06 2024-09-16T20:48:27.188 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.188 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.188 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:27.189 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281553+0000", 2024-09-16T20:48:27.189 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11734135499999999 2024-09-16T20:48:27.189 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.189 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.189 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:27.189 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281567+0000", 2024-09-16T20:48:27.189 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4188999999999999e-05 2024-09-16T20:48:27.189 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.189 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.189 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:27.189 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281653+0000", 2024-09-16T20:48:27.189 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5385999999999999e-05 2024-09-16T20:48:27.189 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.190 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.190 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:27.190 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281679+0000", 2024-09-16T20:48:27.190 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6689000000000001e-05 2024-09-16T20:48:27.190 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.190 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.190 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:27.190 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282277+0000", 2024-09-16T20:48:27.190 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00059798000000000002 2024-09-16T20:48:27.190 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:27.190 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:27.190 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:27.190 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282307+0000", 2024-09-16T20:48:27.190 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9975e-05 2024-09-16T20:48:27.191 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:27.191 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:27.191 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:27.191 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:27.191 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:27.191 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:27.192 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:48:27.193 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:27.191+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:27.294 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:27.292+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:27.310 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:27.311 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:48:27.311 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:48:27.311 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:27.319 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:48:27.394 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:27.392+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:27.428 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:27.428 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:48:27.428 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:48:27.428 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:48:27.428 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:27.436 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:48:27.521 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:27.520+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:27.584 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:27.585 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:48:27.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:48:27.585 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:48:27.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.585 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6412 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3561472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:27.585 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871685+0000", 2024-09-16T20:48:27.585 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.7112031219999997, 2024-09-16T20:48:27.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048146568000000001, 2024-09-16T20:48:27.585 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:27.585 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:27.585 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:27.586 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:27.586 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:27.586 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:27.586 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6412 2024-09-16T20:48:27.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.586 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:27.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:27.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871685+0000", 2024-09-16T20:48:27.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:27.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:27.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871685+0000", 2024-09-16T20:48:27.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:27.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:27.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871685+0000", 2024-09-16T20:48:27.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.58e-07 2024-09-16T20:48:27.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:27.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871687+0000", 2024-09-16T20:48:27.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1830000000000001e-06 2024-09-16T20:48:27.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:27.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871694+0000", 2024-09-16T20:48:27.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7689999999999999e-06 2024-09-16T20:48:27.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:27.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871699+0000", 2024-09-16T20:48:27.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1769999999999997e-06 2024-09-16T20:48:27.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:27.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.919662+0000", 2024-09-16T20:48:27.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047963121999999997 2024-09-16T20:48:27.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:27.589 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.919744+0000", 2024-09-16T20:48:27.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1997999999999997e-05 2024-09-16T20:48:27.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:27.589 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.919831+0000", 2024-09-16T20:48:27.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6860999999999994e-05 2024-09-16T20:48:27.589 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:27.589 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:27.589 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:27.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.590 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6414 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3569664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:27.590 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871708+0000", 2024-09-16T20:48:27.590 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.711180175, 2024-09-16T20:48:27.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048373317999999998, 2024-09-16T20:48:27.590 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:27.590 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:27.590 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:27.590 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:27.590 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:27.590 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:27.590 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6414 2024-09-16T20:48:27.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.591 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:27.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.591 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:27.591 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871708+0000", 2024-09-16T20:48:27.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:27.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.591 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:27.591 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871708+0000", 2024-09-16T20:48:27.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:27.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.592 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:27.592 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871708+0000", 2024-09-16T20:48:27.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9799999999999999e-07 2024-09-16T20:48:27.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.592 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:27.592 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871710+0000", 2024-09-16T20:48:27.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2699999999999999e-06 2024-09-16T20:48:27.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.592 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:27.592 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871718+0000", 2024-09-16T20:48:27.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8499999999999994e-06 2024-09-16T20:48:27.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.593 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:27.593 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871723+0000", 2024-09-16T20:48:27.593 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1819999999999997e-06 2024-09-16T20:48:27.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.593 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:27.593 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.919883+0000", 2024-09-16T20:48:27.593 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048159955999999997 2024-09-16T20:48:27.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.593 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:27.594 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.919969+0000", 2024-09-16T20:48:27.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5808000000000002e-05 2024-09-16T20:48:27.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.594 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:27.594 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920081+0000", 2024-09-16T20:48:27.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000111854 2024-09-16T20:48:27.594 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:27.594 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:27.594 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:27.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.594 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6415 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3577856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:27.595 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871730+0000", 2024-09-16T20:48:27.595 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.7111576599999996, 2024-09-16T20:48:27.595 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.04851482, 2024-09-16T20:48:27.595 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:27.595 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:27.595 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:27.595 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:27.595 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:27.595 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:27.595 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6415 2024-09-16T20:48:27.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.595 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:27.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.596 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:27.596 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871730+0000", 2024-09-16T20:48:27.596 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:27.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.596 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.596 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:27.596 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871730+0000", 2024-09-16T20:48:27.596 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:27.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.596 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.596 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:27.596 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871731+0000", 2024-09-16T20:48:27.596 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6899999999999998e-07 2024-09-16T20:48:27.597 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.597 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.597 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:27.597 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871733+0000", 2024-09-16T20:48:27.597 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0600000000000002e-06 2024-09-16T20:48:27.597 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.597 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.597 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:27.597 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871740+0000", 2024-09-16T20:48:27.597 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3749999999999997e-06 2024-09-16T20:48:27.597 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.597 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.597 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:27.597 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871749+0000", 2024-09-16T20:48:27.598 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9109999999999999e-06 2024-09-16T20:48:27.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.598 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:27.598 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920129+0000", 2024-09-16T20:48:27.598 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048380175999999997 2024-09-16T20:48:27.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.598 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:27.598 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920181+0000", 2024-09-16T20:48:27.598 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2077000000000003e-05 2024-09-16T20:48:27.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.599 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:27.599 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920245+0000", 2024-09-16T20:48:27.599 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3851999999999998e-05 2024-09-16T20:48:27.599 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:27.599 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:27.599 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:27.599 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.599 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6417 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3586048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:27.599 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871756+0000", 2024-09-16T20:48:27.599 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.7111320249999995, 2024-09-16T20:48:27.599 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048628301999999998, 2024-09-16T20:48:27.600 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:27.600 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:27.600 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:27.600 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:27.600 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:27.600 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:27.600 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6417 2024-09-16T20:48:27.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.600 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:27.600 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.600 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:27.600 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871756+0000", 2024-09-16T20:48:27.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:27.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.601 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:27.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871756+0000", 2024-09-16T20:48:27.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:27.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.601 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:27.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871756+0000", 2024-09-16T20:48:27.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0100000000000005e-07 2024-09-16T20:48:27.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.601 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:27.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871758+0000", 2024-09-16T20:48:27.602 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.902e-06 2024-09-16T20:48:27.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.602 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:27.602 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871764+0000", 2024-09-16T20:48:27.602 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8170000000000004e-06 2024-09-16T20:48:27.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.602 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:27.602 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871769+0000", 2024-09-16T20:48:27.602 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6500000000000004e-06 2024-09-16T20:48:27.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.603 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:27.603 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920276+0000", 2024-09-16T20:48:27.603 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048507794999999999 2024-09-16T20:48:27.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.603 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:27.603 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920329+0000", 2024-09-16T20:48:27.603 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2534999999999998e-05 2024-09-16T20:48:27.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.603 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:27.603 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920384+0000", 2024-09-16T20:48:27.603 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5102000000000002e-05 2024-09-16T20:48:27.604 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:27.604 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:27.604 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:27.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.604 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6419 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3594240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:27.604 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871775+0000", 2024-09-16T20:48:27.604 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.711112848, 2024-09-16T20:48:27.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048754724999999999, 2024-09-16T20:48:27.604 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:27.604 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:27.604 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:27.604 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:27.605 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:27.605 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:27.605 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6419 2024-09-16T20:48:27.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.605 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:27.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:27.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871775+0000", 2024-09-16T20:48:27.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:27.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:27.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871775+0000", 2024-09-16T20:48:27.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:27.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.606 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:27.606 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871775+0000", 2024-09-16T20:48:27.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2500000000000001e-07 2024-09-16T20:48:27.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.606 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:27.606 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871777+0000", 2024-09-16T20:48:27.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9649999999999998e-06 2024-09-16T20:48:27.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.607 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:27.607 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871784+0000", 2024-09-16T20:48:27.607 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3419999999999996e-06 2024-09-16T20:48:27.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.607 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:27.607 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871789+0000", 2024-09-16T20:48:27.607 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1889999999999998e-06 2024-09-16T20:48:27.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.607 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:27.607 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920416+0000", 2024-09-16T20:48:27.607 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048627363999999999 2024-09-16T20:48:27.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.608 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:27.608 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920472+0000", 2024-09-16T20:48:27.608 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5429000000000001e-05 2024-09-16T20:48:27.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.608 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:27.608 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920530+0000", 2024-09-16T20:48:27.608 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8111e-05 2024-09-16T20:48:27.608 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:27.608 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:27.608 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:27.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.609 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6420 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3602432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:27.609 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871796+0000", 2024-09-16T20:48:27.609 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.7110919349999998, 2024-09-16T20:48:27.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048857298, 2024-09-16T20:48:27.609 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:27.609 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:27.609 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:27.609 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:27.609 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:27.609 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:27.609 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6420 2024-09-16T20:48:27.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.610 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:27.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.610 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:27.610 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871796+0000", 2024-09-16T20:48:27.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:27.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.610 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:27.610 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871796+0000", 2024-09-16T20:48:27.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:27.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.610 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:27.610 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871854+0000", 2024-09-16T20:48:27.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8187999999999999e-05 2024-09-16T20:48:27.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.611 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:27.611 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871856+0000", 2024-09-16T20:48:27.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3549999999999999e-06 2024-09-16T20:48:27.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.611 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:27.611 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871866+0000", 2024-09-16T20:48:27.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6500000000000008e-06 2024-09-16T20:48:27.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.611 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:27.612 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871872+0000", 2024-09-16T20:48:27.612 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3799999999999999e-06 2024-09-16T20:48:27.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.612 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:27.612 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920558+0000", 2024-09-16T20:48:27.612 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048685797000000003 2024-09-16T20:48:27.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.612 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:27.612 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920603+0000", 2024-09-16T20:48:27.612 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4629999999999998e-05 2024-09-16T20:48:27.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.613 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:27.613 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920653+0000", 2024-09-16T20:48:27.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0297999999999999e-05 2024-09-16T20:48:27.613 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:27.613 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:27.613 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:27.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.613 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6421 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3610624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:27.613 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871882+0000", 2024-09-16T20:48:27.613 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.7110058669999999, 2024-09-16T20:48:27.614 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048913381999999998, 2024-09-16T20:48:27.614 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:27.614 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:27.614 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:27.614 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:27.614 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:27.614 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:27.614 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6421 2024-09-16T20:48:27.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.614 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:27.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.614 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:27.614 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871882+0000", 2024-09-16T20:48:27.614 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:27.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.615 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.615 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:27.615 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871882+0000", 2024-09-16T20:48:27.615 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:27.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.615 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.615 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:27.615 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.873257+0000", 2024-09-16T20:48:27.615 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001374789 2024-09-16T20:48:27.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.615 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.615 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:27.615 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.873261+0000", 2024-09-16T20:48:27.616 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5589999999999999e-06 2024-09-16T20:48:27.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.616 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.616 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:27.616 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.873275+0000", 2024-09-16T20:48:27.616 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3675999999999999e-05 2024-09-16T20:48:27.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.616 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.616 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:27.616 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.873281+0000", 2024-09-16T20:48:27.616 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6930000000000004e-06 2024-09-16T20:48:27.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.617 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:27.617 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920684+0000", 2024-09-16T20:48:27.617 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047402913999999997 2024-09-16T20:48:27.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.617 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:27.617 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920737+0000", 2024-09-16T20:48:27.617 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3600000000000002e-05 2024-09-16T20:48:27.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.617 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:27.617 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920795+0000", 2024-09-16T20:48:27.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8150999999999999e-05 2024-09-16T20:48:27.618 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:27.618 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:27.618 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:27.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.618 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:7491 3.1e 3:7c48f6ae:::rbd_object_map.110e891a9b2b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:27.618 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.925865+0000", 2024-09-16T20:48:27.618 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.6570227229999999, 2024-09-16T20:48:27.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048897293000000001, 2024-09-16T20:48:27.618 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:27.618 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:27.618 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:27.619 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:27.619 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:27.619 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:27.619 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 7491 2024-09-16T20:48:27.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.619 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:27.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.619 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:27.619 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925865+0000", 2024-09-16T20:48:27.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:27.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.620 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:27.620 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925865+0000", 2024-09-16T20:48:27.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:27.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.620 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:27.620 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925866+0000", 2024-09-16T20:48:27.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.39e-07 2024-09-16T20:48:27.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.620 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:27.620 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925868+0000", 2024-09-16T20:48:27.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.508e-06 2024-09-16T20:48:27.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.621 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:27.621 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925874+0000", 2024-09-16T20:48:27.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7919999999999998e-06 2024-09-16T20:48:27.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.621 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:27.621 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925879+0000", 2024-09-16T20:48:27.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8579999999999999e-06 2024-09-16T20:48:27.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.622 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:27.622 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.970875+0000", 2024-09-16T20:48:27.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044996412 2024-09-16T20:48:27.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.622 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:27.622 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.970970+0000", 2024-09-16T20:48:27.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4732e-05 2024-09-16T20:48:27.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.622 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-16T20:48:27.622 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.971275+0000", 2024-09-16T20:48:27.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00030464199999999999 2024-09-16T20:48:27.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:27.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.974598+0000", 2024-09-16T20:48:27.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0033238159999999998 2024-09-16T20:48:27.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:48:27.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.974666+0000", 2024-09-16T20:48:27.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7617e-05 2024-09-16T20:48:27.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:48:27.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.974718+0000", 2024-09-16T20:48:27.624 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2259000000000002e-05 2024-09-16T20:48:27.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.624 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.624 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:27.624 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.974762+0000", 2024-09-16T20:48:27.624 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4218000000000003e-05 2024-09-16T20:48:27.624 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:27.624 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:27.624 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:27.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.624 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.624 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4750 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1488896~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:27.625 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689052+0000", 2024-09-16T20:48:27.625 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.8938361839999995, 2024-09-16T20:48:27.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048208359999999999, 2024-09-16T20:48:27.625 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:27.625 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:27.625 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:27.625 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:27.625 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:27.625 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:27.625 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4750 2024-09-16T20:48:27.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.625 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:27.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.626 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:27.626 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689052+0000", 2024-09-16T20:48:27.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:27.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.626 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:27.626 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689052+0000", 2024-09-16T20:48:27.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:27.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.626 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:27.626 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689052+0000", 2024-09-16T20:48:27.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.65e-07 2024-09-16T20:48:27.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.627 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:27.627 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689058+0000", 2024-09-16T20:48:27.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.835e-06 2024-09-16T20:48:27.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.627 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:27.627 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689064+0000", 2024-09-16T20:48:27.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2729999999999999e-06 2024-09-16T20:48:27.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:27.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689069+0000", 2024-09-16T20:48:27.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1220000000000003e-06 2024-09-16T20:48:27.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:27.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737161+0000", 2024-09-16T20:48:27.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048091776000000003 2024-09-16T20:48:27.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:27.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737210+0000", 2024-09-16T20:48:27.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8575000000000002e-05 2024-09-16T20:48:27.629 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.629 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.629 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:27.629 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737260+0000", 2024-09-16T20:48:27.629 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0414000000000002e-05 2024-09-16T20:48:27.629 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:27.629 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:27.629 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:27.629 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.629 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.629 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4752 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1497088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:27.629 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689076+0000", 2024-09-16T20:48:27.629 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.8938120420000004, 2024-09-16T20:48:27.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048322324999999999, 2024-09-16T20:48:27.630 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:27.630 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:27.630 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:27.630 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:27.630 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:27.630 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:27.630 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4752 2024-09-16T20:48:27.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.630 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:27.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:27.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689076+0000", 2024-09-16T20:48:27.631 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:27.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.631 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.631 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:27.631 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689076+0000", 2024-09-16T20:48:27.631 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:27.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.631 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.631 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:27.631 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689076+0000", 2024-09-16T20:48:27.631 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8000000000000001e-07 2024-09-16T20:48:27.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.631 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.631 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:27.631 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689078+0000", 2024-09-16T20:48:27.632 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8369999999999999e-06 2024-09-16T20:48:27.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.632 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.632 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:27.632 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689084+0000", 2024-09-16T20:48:27.632 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1319999999999999e-06 2024-09-16T20:48:27.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.632 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.632 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:27.632 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689090+0000", 2024-09-16T20:48:27.632 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0430000000000004e-06 2024-09-16T20:48:27.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.633 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.633 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:27.633 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737286+0000", 2024-09-16T20:48:27.633 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048195670000000003 2024-09-16T20:48:27.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.633 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.633 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:27.633 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737337+0000", 2024-09-16T20:48:27.634 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1511000000000002e-05 2024-09-16T20:48:27.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.634 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:27.634 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737398+0000", 2024-09-16T20:48:27.634 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0751999999999997e-05 2024-09-16T20:48:27.634 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:27.634 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:27.634 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:27.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.634 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4754 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1505280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:27.634 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689097+0000", 2024-09-16T20:48:27.634 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.893790525, 2024-09-16T20:48:27.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048429947000000001, 2024-09-16T20:48:27.635 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:27.635 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:27.635 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:27.635 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:27.635 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:27.635 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:27.635 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4754 2024-09-16T20:48:27.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.635 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:27.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.635 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:27.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689097+0000", 2024-09-16T20:48:27.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:27.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.636 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:27.636 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689097+0000", 2024-09-16T20:48:27.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:27.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.636 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:27.636 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689098+0000", 2024-09-16T20:48:27.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.15e-07 2024-09-16T20:48:27.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.637 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:27.637 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689100+0000", 2024-09-16T20:48:27.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8190000000000001e-06 2024-09-16T20:48:27.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.637 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:27.637 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689106+0000", 2024-09-16T20:48:27.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2489999999999998e-06 2024-09-16T20:48:27.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.637 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:27.637 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689111+0000", 2024-09-16T20:48:27.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3179999999999997e-06 2024-09-16T20:48:27.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.638 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.638 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:27.638 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737428+0000", 2024-09-16T20:48:27.638 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048316572000000002 2024-09-16T20:48:27.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.638 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.638 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:27.638 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737475+0000", 2024-09-16T20:48:27.638 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7392000000000003e-05 2024-09-16T20:48:27.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.638 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.638 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:27.639 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737527+0000", 2024-09-16T20:48:27.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2182000000000003e-05 2024-09-16T20:48:27.639 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:27.639 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:27.639 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:27.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.639 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4756 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1513472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:27.639 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689118+0000", 2024-09-16T20:48:27.639 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.8937703040000002, 2024-09-16T20:48:27.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048535146000000001, 2024-09-16T20:48:27.639 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:27.639 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:27.640 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:27.640 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:27.640 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:27.640 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:27.640 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4756 2024-09-16T20:48:27.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.640 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:27.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.640 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:27.640 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689118+0000", 2024-09-16T20:48:27.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:27.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:27.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689118+0000", 2024-09-16T20:48:27.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:27.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:27.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689118+0000", 2024-09-16T20:48:27.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7099999999999997e-07 2024-09-16T20:48:27.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:27.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689120+0000", 2024-09-16T20:48:27.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.787e-06 2024-09-16T20:48:27.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.642 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:27.642 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689126+0000", 2024-09-16T20:48:27.642 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2720000000000003e-06 2024-09-16T20:48:27.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.642 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:27.642 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689134+0000", 2024-09-16T20:48:27.642 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2190000000000004e-06 2024-09-16T20:48:27.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.642 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:27.642 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737553+0000", 2024-09-16T20:48:27.642 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048418993 2024-09-16T20:48:27.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.643 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:27.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737598+0000", 2024-09-16T20:48:27.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4743999999999997e-05 2024-09-16T20:48:27.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.643 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:27.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737653+0000", 2024-09-16T20:48:27.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4759999999999997e-05 2024-09-16T20:48:27.643 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:27.643 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:27.643 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:27.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.644 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4758 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1521664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:27.644 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689141+0000", 2024-09-16T20:48:27.644 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.8937473239999996, 2024-09-16T20:48:27.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048640741000000001, 2024-09-16T20:48:27.644 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:27.644 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:27.644 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:27.644 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:27.644 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:27.644 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:27.644 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4758 2024-09-16T20:48:27.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.645 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:27.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:27.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689141+0000", 2024-09-16T20:48:27.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:27.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:27.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689141+0000", 2024-09-16T20:48:27.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:27.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:27.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689141+0000", 2024-09-16T20:48:27.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8799999999999998e-07 2024-09-16T20:48:27.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.646 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:27.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689143+0000", 2024-09-16T20:48:27.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.945e-06 2024-09-16T20:48:27.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.646 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:27.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689149+0000", 2024-09-16T20:48:27.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9800000000000003e-06 2024-09-16T20:48:27.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.647 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:27.647 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689155+0000", 2024-09-16T20:48:27.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7509999999999997e-06 2024-09-16T20:48:27.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.647 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:27.647 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737681+0000", 2024-09-16T20:48:27.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048526825000000003 2024-09-16T20:48:27.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.647 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:27.647 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737729+0000", 2024-09-16T20:48:27.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7219999999999999e-05 2024-09-16T20:48:27.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:27.648 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737781+0000", 2024-09-16T20:48:27.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2632e-05 2024-09-16T20:48:27.648 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:27.648 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:27.648 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:27.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.648 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4760 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1529856~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:27.648 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689161+0000", 2024-09-16T20:48:27.648 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.893726977, 2024-09-16T20:48:27.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048743981999999998, 2024-09-16T20:48:27.649 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:27.649 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:27.649 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:27.649 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:27.649 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:27.649 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:27.649 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4760 2024-09-16T20:48:27.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.649 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:27.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:27.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689161+0000", 2024-09-16T20:48:27.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:27.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.650 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:27.650 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689161+0000", 2024-09-16T20:48:27.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:27.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.650 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:27.650 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689161+0000", 2024-09-16T20:48:27.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5599999999999995e-07 2024-09-16T20:48:27.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.650 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:27.650 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689163+0000", 2024-09-16T20:48:27.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.818e-06 2024-09-16T20:48:27.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.651 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:27.651 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689169+0000", 2024-09-16T20:48:27.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0809999999999999e-06 2024-09-16T20:48:27.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.651 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:27.651 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689174+0000", 2024-09-16T20:48:27.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.938e-06 2024-09-16T20:48:27.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.651 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:27.652 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737807+0000", 2024-09-16T20:48:27.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048633124 2024-09-16T20:48:27.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.652 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:27.652 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737851+0000", 2024-09-16T20:48:27.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3764000000000003e-05 2024-09-16T20:48:27.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.652 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:27.652 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737905+0000", 2024-09-16T20:48:27.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3701e-05 2024-09-16T20:48:27.653 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:27.653 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:27.653 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:27.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.653 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4762 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1538048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:27.653 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689182+0000", 2024-09-16T20:48:27.653 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.893705755, 2024-09-16T20:48:27.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048850275999999998, 2024-09-16T20:48:27.653 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:27.653 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:27.653 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:27.653 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:27.653 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:27.654 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:27.654 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4762 2024-09-16T20:48:27.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.654 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:27.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:27.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689182+0000", 2024-09-16T20:48:27.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:27.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:27.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689182+0000", 2024-09-16T20:48:27.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:27.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.655 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.655 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:27.655 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689182+0000", 2024-09-16T20:48:27.655 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-16T20:48:27.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.655 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.655 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:27.655 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689184+0000", 2024-09-16T20:48:27.655 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8190000000000001e-06 2024-09-16T20:48:27.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.655 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.655 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:27.656 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689191+0000", 2024-09-16T20:48:27.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2530000000000001e-06 2024-09-16T20:48:27.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.656 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:27.656 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689195+0000", 2024-09-16T20:48:27.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9559999999999996e-06 2024-09-16T20:48:27.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.656 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:27.656 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737933+0000", 2024-09-16T20:48:27.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048737919999999997 2024-09-16T20:48:27.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.657 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:27.657 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737982+0000", 2024-09-16T20:48:27.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8089e-05 2024-09-16T20:48:27.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.657 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:27.657 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738032+0000", 2024-09-16T20:48:27.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0868999999999997e-05 2024-09-16T20:48:27.657 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:27.657 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:27.657 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:27.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.658 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4764 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1546240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:27.658 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689201+0000", 2024-09-16T20:48:27.658 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.8936863979999998, 2024-09-16T20:48:27.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048957550000000002, 2024-09-16T20:48:27.658 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:27.658 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:27.658 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:27.658 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:27.658 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:27.658 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:27.658 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4764 2024-09-16T20:48:27.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.658 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:27.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.659 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:27.659 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689201+0000", 2024-09-16T20:48:27.659 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:27.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.659 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.659 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:27.659 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689201+0000", 2024-09-16T20:48:27.659 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:27.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.659 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.659 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:27.659 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689202+0000", 2024-09-16T20:48:27.659 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4200000000000002e-07 2024-09-16T20:48:27.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.660 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:27.660 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689204+0000", 2024-09-16T20:48:27.660 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9750000000000001e-06 2024-09-16T20:48:27.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.660 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:27.660 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689210+0000", 2024-09-16T20:48:27.660 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1e-06 2024-09-16T20:48:27.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.660 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:27.660 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689215+0000", 2024-09-16T20:48:27.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0180000000000001e-06 2024-09-16T20:48:27.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.661 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:27.661 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738058+0000", 2024-09-16T20:48:27.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048843573000000001 2024-09-16T20:48:27.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.661 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:27.661 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738102+0000", 2024-09-16T20:48:27.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3256000000000001e-05 2024-09-16T20:48:27.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.662 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:27.662 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738159+0000", 2024-09-16T20:48:27.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7286e-05 2024-09-16T20:48:27.662 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:27.662 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:27.662 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:27.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.662 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.662 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4766 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1554432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:27.662 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689221+0000", 2024-09-16T20:48:27.662 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.8936664319999998, 2024-09-16T20:48:27.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049064996, 2024-09-16T20:48:27.662 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:27.663 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:27.663 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:27.663 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:27.663 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:27.663 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:27.663 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4766 2024-09-16T20:48:27.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.663 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:27.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.663 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:27.663 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689221+0000", 2024-09-16T20:48:27.663 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:27.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.664 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.664 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:27.664 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689221+0000", 2024-09-16T20:48:27.664 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:27.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.664 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.664 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:27.664 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689222+0000", 2024-09-16T20:48:27.664 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.34e-07 2024-09-16T20:48:27.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.664 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.664 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:27.665 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689224+0000", 2024-09-16T20:48:27.665 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.846e-06 2024-09-16T20:48:27.665 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.665 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.665 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:27.665 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689247+0000", 2024-09-16T20:48:27.665 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3448e-05 2024-09-16T20:48:27.665 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.665 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.665 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:27.665 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689254+0000", 2024-09-16T20:48:27.665 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6969999999999996e-06 2024-09-16T20:48:27.665 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.665 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.666 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:27.666 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738188+0000", 2024-09-16T20:48:27.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048934520000000002 2024-09-16T20:48:27.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.666 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:27.666 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738234+0000", 2024-09-16T20:48:27.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5695999999999998e-05 2024-09-16T20:48:27.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.666 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:27.666 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738286+0000", 2024-09-16T20:48:27.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2355000000000003e-05 2024-09-16T20:48:27.667 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:27.667 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:27.667 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:27.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.667 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4768 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1562624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:27.667 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689262+0000", 2024-09-16T20:48:27.667 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.8936254779999997, 2024-09-16T20:48:27.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049157157999999999, 2024-09-16T20:48:27.667 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:27.667 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:27.667 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:27.667 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:27.668 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:27.668 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:27.668 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4768 2024-09-16T20:48:27.668 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.668 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:27.668 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.668 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:27.668 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689262+0000", 2024-09-16T20:48:27.668 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:27.668 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.668 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.668 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:27.668 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689262+0000", 2024-09-16T20:48:27.669 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:27.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.669 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.669 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:27.669 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689263+0000", 2024-09-16T20:48:27.669 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6199999999999998e-07 2024-09-16T20:48:27.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.669 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.669 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:27.669 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689270+0000", 2024-09-16T20:48:27.669 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.798e-06 2024-09-16T20:48:27.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.669 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.670 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:27.670 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689277+0000", 2024-09-16T20:48:27.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0890000000000002e-06 2024-09-16T20:48:27.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.670 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:27.670 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689282+0000", 2024-09-16T20:48:27.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0470000000000001e-06 2024-09-16T20:48:27.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.670 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:27.670 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738319+0000", 2024-09-16T20:48:27.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049037139 2024-09-16T20:48:27.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.671 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.671 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:27.671 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738365+0000", 2024-09-16T20:48:27.671 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6356999999999997e-05 2024-09-16T20:48:27.671 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.671 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.671 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:27.671 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738420+0000", 2024-09-16T20:48:27.671 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4265999999999998e-05 2024-09-16T20:48:27.671 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:27.671 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:27.671 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:27.671 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.672 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.672 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4770 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1570816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:27.672 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689289+0000", 2024-09-16T20:48:27.672 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.893598968, 2024-09-16T20:48:27.672 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049253705000000002, 2024-09-16T20:48:27.672 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:27.672 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:27.672 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:27.672 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:27.672 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:27.672 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:27.672 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4770 2024-09-16T20:48:27.672 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.673 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:27.673 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.673 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:27.673 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689289+0000", 2024-09-16T20:48:27.673 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:27.673 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.673 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.673 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:27.673 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689289+0000", 2024-09-16T20:48:27.673 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:27.673 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.673 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.676 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:27.676 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689289+0000", 2024-09-16T20:48:27.676 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0999999999999999e-07 2024-09-16T20:48:27.676 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.676 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.676 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:27.676 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689291+0000", 2024-09-16T20:48:27.676 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0030000000000001e-06 2024-09-16T20:48:27.676 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.676 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.676 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:27.676 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689297+0000", 2024-09-16T20:48:27.677 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9479999999999996e-06 2024-09-16T20:48:27.677 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.677 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.677 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:27.677 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689302+0000", 2024-09-16T20:48:27.677 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7389999999999999e-06 2024-09-16T20:48:27.677 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.677 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.677 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:27.677 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738442+0000", 2024-09-16T20:48:27.677 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049140001000000003 2024-09-16T20:48:27.677 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.677 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.678 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:27.678 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738490+0000", 2024-09-16T20:48:27.678 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7787000000000002e-05 2024-09-16T20:48:27.678 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.678 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.678 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:27.678 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738543+0000", 2024-09-16T20:48:27.678 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2816999999999999e-05 2024-09-16T20:48:27.678 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:27.678 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:27.678 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:27.678 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.679 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.679 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:3088 3.1e 3:79e5ec01:::rbd_header.110e891a9b2b:head [watch watch cookie 140213437171488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:48:27.679 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:48:27.679 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.4190429509999998, 2024-09-16T20:48:27.679 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.11895186200000001, 2024-09-16T20:48:27.679 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:27.679 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:27.679 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:27.679 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:27.679 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:48:27.679 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:48:27.679 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3088 2024-09-16T20:48:27.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.680 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:27.680 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.680 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:27.680 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:48:27.680 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:27.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.680 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.680 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:27.680 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:48:27.680 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:27.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.680 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.680 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:27.681 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163846+0000", 2024-09-16T20:48:27.681 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6799999999999999e-07 2024-09-16T20:48:27.681 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.681 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.681 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:27.681 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163850+0000", 2024-09-16T20:48:27.681 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.929e-06 2024-09-16T20:48:27.681 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.681 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.681 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:27.681 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163862+0000", 2024-09-16T20:48:27.681 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1972e-05 2024-09-16T20:48:27.681 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.682 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.682 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:27.682 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163867+0000", 2024-09-16T20:48:27.682 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.451e-06 2024-09-16T20:48:27.682 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.682 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.682 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:27.682 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163912+0000", 2024-09-16T20:48:27.682 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4456000000000003e-05 2024-09-16T20:48:27.682 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.682 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.682 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:27.682 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163962+0000", 2024-09-16T20:48:27.682 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0578000000000003e-05 2024-09-16T20:48:27.683 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.683 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.683 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-16T20:48:27.683 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164058+0000", 2024-09-16T20:48:27.683 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5934999999999995e-05 2024-09-16T20:48:27.683 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.683 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.683 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:48:27.683 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164570+0000", 2024-09-16T20:48:27.683 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00051241100000000005 2024-09-16T20:48:27.683 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.683 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.683 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:27.684 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282629+0000", 2024-09-16T20:48:27.684 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.118058986 2024-09-16T20:48:27.684 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.684 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.684 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:48:27.684 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282725+0000", 2024-09-16T20:48:27.684 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5063999999999996e-05 2024-09-16T20:48:27.684 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:27.684 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:27.684 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:27.684 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282797+0000", 2024-09-16T20:48:27.684 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2212000000000006e-05 2024-09-16T20:48:27.684 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:27.685 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:27.685 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:27.685 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:27.685 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:27.685 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:27.685 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T20:48:27.687 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:27.685+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:27.687 INFO:teuthology.orchestra.run.smithi139.stdout:924 read 0xa9f000 thru 0xaa0b3f (0x1b40 bytes) 2024-09-16T20:48:27.687 INFO:teuthology.orchestra.run.smithi139.stdout:925 read 0xbd2e00 thru 0xbdf5ce (0xc7cf bytes) 2024-09-16T20:48:27.688 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:48:27.699 INFO:teuthology.orchestra.run.smithi139.stdout:927 trunc from 0x1e48200 to 0xc41c200 2024-09-16T20:48:27.709 INFO:teuthology.orchestra.run.smithi139.stdout:928 punch from 0x74f9000 to 0x7502000, (0x9000 bytes) 2024-09-16T20:48:27.711 INFO:teuthology.orchestra.run.smithi139.stdout:929 read 0x1e83e00 thru 0x1e85875 (0x1a76 bytes) 2024-09-16T20:48:27.711 INFO:teuthology.orchestra.run.smithi139.stdout:932 read 0x1a2e200 thru 0x1a3c75b (0xe55c bytes) 2024-09-16T20:48:27.712 INFO:teuthology.orchestra.run.smithi139.stdout:935 punch from 0x259da00 to 0x25a8c00, (0xb200 bytes) 2024-09-16T20:48:27.712 INFO:teuthology.orchestra.run.smithi139.stdout:938 punch from 0x6d67000 to 0x6d75e00, (0xee00 bytes) 2024-09-16T20:48:27.712 INFO:teuthology.orchestra.run.smithi139.stdout:940 write 0x6c86800 thru 0x6c89efc (0x36fd bytes) 2024-09-16T20:48:27.712 INFO:teuthology.orchestra.run.smithi139.stdout:941 write 0xb67e600 thru 0xb68c4e5 (0xdee6 bytes) 2024-09-16T20:48:27.713 INFO:teuthology.orchestra.run.smithi139.stdout:945 punch from 0x4fcd600 to 0x4fd7600, (0xa000 bytes) 2024-09-16T20:48:27.713 INFO:teuthology.orchestra.run.smithi139.stdout:946 write 0x3e4b400 thru 0x3e51d59 (0x695a bytes) 2024-09-16T20:48:27.724 INFO:teuthology.orchestra.run.smithi139.stdout:949 trunc from 0xc41c200 to 0x5652200 2024-09-16T20:48:27.747 INFO:teuthology.orchestra.run.smithi139.stdout:952 write 0x55bb600 thru 0x55c00f0 (0x4af1 bytes) 2024-09-16T20:48:27.748 INFO:teuthology.orchestra.run.smithi139.stdout:953 punch from 0x8a9a00 to 0x8ae200, (0x4800 bytes) 2024-09-16T20:48:27.750 INFO:teuthology.orchestra.run.smithi139.stdout:955 trunc from 0x5652200 to 0x67bfc00 2024-09-16T20:48:27.761 INFO:teuthology.orchestra.run.smithi139.stdout:skipping zero length punch hole 2024-09-16T20:48:27.770 INFO:teuthology.orchestra.run.smithi139.stdout:957 trunc from 0x67bfc00 to 0xd73e600 2024-09-16T20:48:27.786 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:27.785+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:27.793 INFO:teuthology.orchestra.run.smithi139.stdout:958 trunc from 0xd73e600 to 0x2913800 2024-09-16T20:48:27.812 INFO:teuthology.orchestra.run.smithi139.stdout:959 read 0x157a800 thru 0x1587b7b (0xd37c bytes) 2024-09-16T20:48:27.832 INFO:teuthology.orchestra.run.smithi139.stdout:960 write 0x9e74c00 thru 0x9e781ce (0x35cf bytes) 2024-09-16T20:48:27.838 INFO:teuthology.orchestra.run.smithi139.stdout:962 punch from 0x4af3e00 to 0x4af9c00, (0x5e00 bytes) 2024-09-16T20:48:27.838 INFO:teuthology.orchestra.run.smithi139.stdout:964 punch from 0x77ece00 to 0x77f4800, (0x7a00 bytes) 2024-09-16T20:48:27.839 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:27.839 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:48:27.839 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:48:27.839 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:27.839 INFO:teuthology.orchestra.run.smithi139.stdout:965 trunc from 0x9e78200 to 0x93c5400 2024-09-16T20:48:27.849 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:48:27.856 INFO:teuthology.orchestra.run.smithi139.stdout:966 write 0x5d2600 thru 0x5d4ffc (0x29fd bytes) 2024-09-16T20:48:27.871 INFO:teuthology.orchestra.run.smithi139.stdout:967 write 0xbe6aa00 thru 0xbe75b25 (0xb126 bytes) 2024-09-16T20:48:27.872 INFO:teuthology.orchestra.run.smithi139.stdout:970 read 0xbcf9a00 thru 0xbd00b67 (0x7168 bytes) 2024-09-16T20:48:27.873 INFO:teuthology.orchestra.run.smithi139.stdout:973 read 0x9b3ee00 thru 0x9b40098 (0x1299 bytes) 2024-09-16T20:48:27.874 INFO:teuthology.orchestra.run.smithi139.stdout:974 write 0x63e7800 thru 0x63f4a67 (0xd268 bytes) 2024-09-16T20:48:27.874 INFO:teuthology.orchestra.run.smithi139.stdout:975 clone 8 order 18 su 8192 sc 8 2024-09-16T20:48:27.887 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:27.886+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:27.957 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:27.957 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:48:27.958 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:48:27.959 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:48:27.959 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:27.966 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:48:27.988 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:27.987+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:28.074 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:28.074 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:48:28.074 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:48:28.074 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:48:28.074 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.074 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:4904 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 7309312~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:28.074 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767724+0000", 2024-09-16T20:48:28.075 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.3046774889999995, 2024-09-16T20:48:28.075 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050270214000000001, 2024-09-16T20:48:28.075 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:28.075 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:28.075 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:28.075 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:28.075 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:28.075 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:28.075 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4904 2024-09-16T20:48:28.075 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.075 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:28.075 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.075 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:28.076 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767724+0000", 2024-09-16T20:48:28.076 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.076 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.076 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.076 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:28.076 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767724+0000", 2024-09-16T20:48:28.076 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.076 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.076 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.076 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:28.076 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767724+0000", 2024-09-16T20:48:28.076 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1399999999999998e-07 2024-09-16T20:48:28.076 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.076 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.077 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:28.077 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767725+0000", 2024-09-16T20:48:28.077 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0380000000000001e-06 2024-09-16T20:48:28.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.077 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.077 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:28.077 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767729+0000", 2024-09-16T20:48:28.077 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2490000000000001e-06 2024-09-16T20:48:28.080 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.080 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.080 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:28.080 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767731+0000", 2024-09-16T20:48:28.080 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5940000000000001e-06 2024-09-16T20:48:28.080 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.081 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.081 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:28.081 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.817787+0000", 2024-09-16T20:48:28.081 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050055792000000002 2024-09-16T20:48:28.081 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.081 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.081 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:28.081 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.817883+0000", 2024-09-16T20:48:28.081 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5606999999999995e-05 2024-09-16T20:48:28.081 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.081 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.081 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:28.081 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.817994+0000", 2024-09-16T20:48:28.082 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011162000000000001 2024-09-16T20:48:28.082 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.082 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:28.082 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.082 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.082 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.082 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:4906 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 7317504~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:28.082 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767734+0000", 2024-09-16T20:48:28.082 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.3046672259999994, 2024-09-16T20:48:28.082 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050406608999999998, 2024-09-16T20:48:28.082 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:28.082 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:28.083 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:28.083 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:28.083 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:28.083 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:28.083 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4906 2024-09-16T20:48:28.083 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.083 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:28.083 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.083 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:28.083 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767734+0000", 2024-09-16T20:48:28.083 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.083 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.083 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.084 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:28.084 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767734+0000", 2024-09-16T20:48:28.084 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.084 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.084 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.084 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:28.084 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767735+0000", 2024-09-16T20:48:28.084 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.05e-07 2024-09-16T20:48:28.084 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.084 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.084 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:28.084 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767736+0000", 2024-09-16T20:48:28.085 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0240000000000001e-06 2024-09-16T20:48:28.085 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.085 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.085 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:28.085 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767739+0000", 2024-09-16T20:48:28.085 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.067e-06 2024-09-16T20:48:28.085 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.085 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.085 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:28.085 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767742+0000", 2024-09-16T20:48:28.085 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.23e-06 2024-09-16T20:48:28.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.086 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.086 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:28.086 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818027+0000", 2024-09-16T20:48:28.086 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050285102999999998 2024-09-16T20:48:28.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.086 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.086 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:28.086 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818072+0000", 2024-09-16T20:48:28.086 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5129000000000001e-05 2024-09-16T20:48:28.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.086 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.086 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:28.087 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818141+0000", 2024-09-16T20:48:28.087 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8850999999999995e-05 2024-09-16T20:48:28.087 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.087 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:28.087 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.087 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.087 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:4908 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 7325696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:28.087 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767745+0000", 2024-09-16T20:48:28.088 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.3046568310000008, 2024-09-16T20:48:28.088 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050654230000000001, 2024-09-16T20:48:28.088 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:28.088 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:28.088 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:28.088 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:28.088 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:28.088 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:28.088 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4908 2024-09-16T20:48:28.088 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.088 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:28.088 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.088 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:28.089 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767745+0000", 2024-09-16T20:48:28.089 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.089 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.089 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.089 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:28.089 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767745+0000", 2024-09-16T20:48:28.089 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.089 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.089 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.089 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:28.089 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767745+0000", 2024-09-16T20:48:28.089 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1e-07 2024-09-16T20:48:28.089 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.090 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.090 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:28.090 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767749+0000", 2024-09-16T20:48:28.090 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.532e-06 2024-09-16T20:48:28.090 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.090 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.090 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:28.090 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767753+0000", 2024-09-16T20:48:28.090 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4939999999999999e-06 2024-09-16T20:48:28.090 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:28.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767756+0000", 2024-09-16T20:48:28.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.61e-06 2024-09-16T20:48:28.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:28.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818184+0000", 2024-09-16T20:48:28.093 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050428615000000003 2024-09-16T20:48:28.093 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.093 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.093 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:28.093 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818276+0000", 2024-09-16T20:48:28.093 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2077999999999995e-05 2024-09-16T20:48:28.093 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.093 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.093 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:28.093 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818399+0000", 2024-09-16T20:48:28.093 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012269099999999999 2024-09-16T20:48:28.094 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.094 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:28.094 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.094 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.094 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.094 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:4910 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 7333888~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:28.094 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767759+0000", 2024-09-16T20:48:28.094 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.3046429410000009, 2024-09-16T20:48:28.094 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050815076000000001, 2024-09-16T20:48:28.094 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:28.094 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:28.094 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:28.094 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:28.095 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:28.095 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:28.095 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4910 2024-09-16T20:48:28.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.095 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:28.095 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:28.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767759+0000", 2024-09-16T20:48:28.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.095 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:28.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767759+0000", 2024-09-16T20:48:28.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.096 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.096 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:28.096 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767759+0000", 2024-09-16T20:48:28.096 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2600000000000001e-07 2024-09-16T20:48:28.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.096 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.096 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:28.096 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767760+0000", 2024-09-16T20:48:28.096 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0640000000000001e-06 2024-09-16T20:48:28.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.096 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.096 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:28.097 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767763+0000", 2024-09-16T20:48:28.097 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.994e-06 2024-09-16T20:48:28.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.097 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.097 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:28.097 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767765+0000", 2024-09-16T20:48:28.097 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5220000000000002e-06 2024-09-16T20:48:28.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.097 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.097 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:28.097 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818438+0000", 2024-09-16T20:48:28.097 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050672602999999997 2024-09-16T20:48:28.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.098 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.098 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:28.098 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818481+0000", 2024-09-16T20:48:28.098 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2967999999999999e-05 2024-09-16T20:48:28.098 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.098 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.098 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:28.098 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818574+0000", 2024-09-16T20:48:28.098 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2699000000000001e-05 2024-09-16T20:48:28.098 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.098 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:28.098 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.098 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.099 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.099 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5012 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2101248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:28.099 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767768+0000", 2024-09-16T20:48:28.099 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.3046332920000001, 2024-09-16T20:48:28.099 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051084682999999999, 2024-09-16T20:48:28.099 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:28.099 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:28.099 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:28.099 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:28.099 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:28.099 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:28.099 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5012 2024-09-16T20:48:28.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.100 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:28.100 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.100 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:28.100 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767768+0000", 2024-09-16T20:48:28.100 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.100 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.100 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.100 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:28.100 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767768+0000", 2024-09-16T20:48:28.100 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.101 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:28.101 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767769+0000", 2024-09-16T20:48:28.101 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1500000000000001e-07 2024-09-16T20:48:28.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.101 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:28.101 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767770+0000", 2024-09-16T20:48:28.101 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.015e-06 2024-09-16T20:48:28.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.101 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:28.102 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767773+0000", 2024-09-16T20:48:28.102 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1070000000000001e-06 2024-09-16T20:48:28.102 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.102 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.102 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:28.102 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767775+0000", 2024-09-16T20:48:28.102 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.508e-06 2024-09-16T20:48:28.102 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.102 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.103 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:28.103 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818630+0000", 2024-09-16T20:48:28.103 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050855282000000002 2024-09-16T20:48:28.103 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.103 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.103 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:28.103 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818732+0000", 2024-09-16T20:48:28.103 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101531 2024-09-16T20:48:28.103 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.103 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.103 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:28.103 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818853+0000", 2024-09-16T20:48:28.103 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000121025 2024-09-16T20:48:28.103 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.104 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:28.104 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.104 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.104 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.104 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5014 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2109440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:28.104 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767778+0000", 2024-09-16T20:48:28.104 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.3046233560000005, 2024-09-16T20:48:28.104 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051260776000000001, 2024-09-16T20:48:28.104 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:28.104 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:28.104 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:28.104 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:28.104 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:28.105 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:28.105 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5014 2024-09-16T20:48:28.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.105 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:28.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.105 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:28.105 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767778+0000", 2024-09-16T20:48:28.105 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.105 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:28.105 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767778+0000", 2024-09-16T20:48:28.105 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.106 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:28.106 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767824+0000", 2024-09-16T20:48:28.106 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5890999999999998e-05 2024-09-16T20:48:28.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.106 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:28.106 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767827+0000", 2024-09-16T20:48:28.106 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.644e-06 2024-09-16T20:48:28.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.106 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:28.106 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767832+0000", 2024-09-16T20:48:28.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2689999999999999e-06 2024-09-16T20:48:28.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.107 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:28.107 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767835+0000", 2024-09-16T20:48:28.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2100000000000002e-06 2024-09-16T20:48:28.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.107 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:28.107 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818887+0000", 2024-09-16T20:48:28.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051051276999999999 2024-09-16T20:48:28.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:28.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818953+0000", 2024-09-16T20:48:28.108 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6105999999999994e-05 2024-09-16T20:48:28.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:28.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819039+0000", 2024-09-16T20:48:28.108 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6378999999999994e-05 2024-09-16T20:48:28.108 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.108 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:28.108 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.108 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5016 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2117632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:28.109 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767841+0000", 2024-09-16T20:48:28.109 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.3045606769999996, 2024-09-16T20:48:28.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051433288000000001, 2024-09-16T20:48:28.109 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:28.109 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:28.109 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:28.109 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:28.109 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:28.109 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:28.109 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5016 2024-09-16T20:48:28.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.109 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:28.109 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.110 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:28.110 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767841+0000", 2024-09-16T20:48:28.110 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.110 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.110 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.110 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:28.110 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767841+0000", 2024-09-16T20:48:28.110 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.110 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.110 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.110 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:28.110 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768156+0000", 2024-09-16T20:48:28.110 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00031505699999999999 2024-09-16T20:48:28.111 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.111 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.111 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:28.111 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768159+0000", 2024-09-16T20:48:28.111 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1659999999999998e-06 2024-09-16T20:48:28.111 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.111 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.111 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:28.111 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768166+0000", 2024-09-16T20:48:28.111 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0550000000000002e-06 2024-09-16T20:48:28.111 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.111 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.111 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:28.111 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768171+0000", 2024-09-16T20:48:28.112 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3170000000000003e-06 2024-09-16T20:48:28.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.112 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:28.112 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819084+0000", 2024-09-16T20:48:28.112 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050913355 2024-09-16T20:48:28.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.112 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:28.112 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819170+0000", 2024-09-16T20:48:28.112 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6328000000000004e-05 2024-09-16T20:48:28.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.112 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.113 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:28.113 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819274+0000", 2024-09-16T20:48:28.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010401 2024-09-16T20:48:28.113 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.113 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:28.113 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.113 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.113 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5018 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2125824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:28.113 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768178+0000", 2024-09-16T20:48:28.113 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.3042231429999998, 2024-09-16T20:48:28.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051227013000000002, 2024-09-16T20:48:28.113 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:28.113 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:28.114 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:28.114 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:28.114 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:28.114 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:28.114 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5018 2024-09-16T20:48:28.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.114 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:28.114 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.114 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:28.114 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768178+0000", 2024-09-16T20:48:28.114 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.114 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.114 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:28.115 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768178+0000", 2024-09-16T20:48:28.115 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.115 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.115 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:28.115 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768179+0000", 2024-09-16T20:48:28.115 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9200000000000002e-07 2024-09-16T20:48:28.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.115 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.115 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:28.115 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768180+0000", 2024-09-16T20:48:28.115 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3790000000000001e-06 2024-09-16T20:48:28.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.115 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.116 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:28.116 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768184+0000", 2024-09-16T20:48:28.116 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5470000000000001e-06 2024-09-16T20:48:28.116 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.116 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.116 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:28.116 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768187+0000", 2024-09-16T20:48:28.116 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8119999999999999e-06 2024-09-16T20:48:28.116 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.116 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.116 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:28.116 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819305+0000", 2024-09-16T20:48:28.116 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051118683999999998 2024-09-16T20:48:28.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.117 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:28.117 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819351+0000", 2024-09-16T20:48:28.117 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5757999999999998e-05 2024-09-16T20:48:28.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.117 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:28.117 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819405+0000", 2024-09-16T20:48:28.117 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4441000000000003e-05 2024-09-16T20:48:28.117 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.117 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:28.117 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.118 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.118 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5020 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2134016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:28.118 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768190+0000", 2024-09-16T20:48:28.118 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.3042114649999998, 2024-09-16T20:48:28.118 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051449661000000001, 2024-09-16T20:48:28.118 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:28.118 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:28.118 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:28.118 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:28.118 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:28.118 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:28.118 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5020 2024-09-16T20:48:28.118 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.118 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:28.119 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.119 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:28.119 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768190+0000", 2024-09-16T20:48:28.119 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.119 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.119 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:28.119 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768190+0000", 2024-09-16T20:48:28.119 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.119 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.119 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:28.119 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768282+0000", 2024-09-16T20:48:28.119 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2263999999999996e-05 2024-09-16T20:48:28.120 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.120 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.120 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:28.120 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768286+0000", 2024-09-16T20:48:28.120 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.529e-06 2024-09-16T20:48:28.120 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.120 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.120 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:28.120 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768294+0000", 2024-09-16T20:48:28.120 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6990000000000003e-06 2024-09-16T20:48:28.120 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.120 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.120 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:28.120 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768306+0000", 2024-09-16T20:48:28.121 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2112e-05 2024-09-16T20:48:28.121 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.121 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.121 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:28.121 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819449+0000", 2024-09-16T20:48:28.121 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051143521999999997 2024-09-16T20:48:28.121 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.121 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.121 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:28.121 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819539+0000", 2024-09-16T20:48:28.121 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0022e-05 2024-09-16T20:48:28.121 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.121 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.121 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:28.122 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819640+0000", 2024-09-16T20:48:28.122 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000100513 2024-09-16T20:48:28.122 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.122 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:28.122 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.122 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.122 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5022 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2142208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:28.122 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768326+0000", 2024-09-16T20:48:28.122 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.3040760759999994, 2024-09-16T20:48:28.122 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051436952000000001, 2024-09-16T20:48:28.122 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:28.122 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:28.122 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:28.123 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:28.123 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:28.123 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:28.123 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5022 2024-09-16T20:48:28.123 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.123 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:28.123 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.123 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:28.123 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768326+0000", 2024-09-16T20:48:28.123 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.123 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.123 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.123 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:28.123 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768326+0000", 2024-09-16T20:48:28.124 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.124 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.124 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.124 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:28.124 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768477+0000", 2024-09-16T20:48:28.124 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015109300000000001 2024-09-16T20:48:28.124 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.124 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.124 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:28.124 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768480+0000", 2024-09-16T20:48:28.124 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8580000000000002e-06 2024-09-16T20:48:28.124 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.124 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.125 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:28.125 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768489+0000", 2024-09-16T20:48:28.125 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4600000000000003e-06 2024-09-16T20:48:28.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.125 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.125 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:28.125 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768493+0000", 2024-09-16T20:48:28.125 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2270000000000004e-06 2024-09-16T20:48:28.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.125 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.125 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:28.125 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819672+0000", 2024-09-16T20:48:28.125 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051178605000000002 2024-09-16T20:48:28.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.126 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.126 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:28.126 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819711+0000", 2024-09-16T20:48:28.126 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9357000000000003e-05 2024-09-16T20:48:28.126 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.126 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.126 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:28.126 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819762+0000", 2024-09-16T20:48:28.126 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1351999999999999e-05 2024-09-16T20:48:28.126 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.126 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:28.126 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.126 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.126 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.127 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5024 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2150400~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:28.127 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768501+0000", 2024-09-16T20:48:28.127 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.3039005980000002, 2024-09-16T20:48:28.127 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051402678, 2024-09-16T20:48:28.127 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:28.127 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:28.127 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:28.127 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:28.127 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:28.127 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:28.127 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5024 2024-09-16T20:48:28.127 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.127 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:28.127 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.128 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:28.128 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768501+0000", 2024-09-16T20:48:28.128 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.128 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.128 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:28.128 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768501+0000", 2024-09-16T20:48:28.128 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.128 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.128 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:28.128 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768619+0000", 2024-09-16T20:48:28.128 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118187 2024-09-16T20:48:28.129 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.129 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.129 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:28.129 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768624+0000", 2024-09-16T20:48:28.129 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3259999999999997e-06 2024-09-16T20:48:28.129 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.129 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.129 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:28.129 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768631+0000", 2024-09-16T20:48:28.129 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6490000000000008e-06 2024-09-16T20:48:28.129 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.129 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.129 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:28.129 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768637+0000", 2024-09-16T20:48:28.130 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4210000000000003e-06 2024-09-16T20:48:28.130 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.130 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.130 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:28.130 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819800+0000", 2024-09-16T20:48:28.130 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051163486000000001 2024-09-16T20:48:28.130 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.130 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.130 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:28.130 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819845+0000", 2024-09-16T20:48:28.130 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4830999999999998e-05 2024-09-16T20:48:28.130 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.130 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.131 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:28.131 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819904+0000", 2024-09-16T20:48:28.131 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8777999999999999e-05 2024-09-16T20:48:28.131 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.131 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:28.131 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.131 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.131 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.131 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5026 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2158592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:28.131 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768646+0000", 2024-09-16T20:48:28.131 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.3037551500000006, 2024-09-16T20:48:28.131 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051384386999999997, 2024-09-16T20:48:28.131 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:28.131 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:28.132 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:28.132 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:28.132 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:28.132 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:28.132 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5026 2024-09-16T20:48:28.132 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.132 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:28.132 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.132 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:28.132 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768646+0000", 2024-09-16T20:48:28.132 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.132 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.132 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.133 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:28.133 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768646+0000", 2024-09-16T20:48:28.133 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.133 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:28.133 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768775+0000", 2024-09-16T20:48:28.133 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012813200000000001 2024-09-16T20:48:28.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.133 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:28.133 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768778+0000", 2024-09-16T20:48:28.133 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2150000000000001e-06 2024-09-16T20:48:28.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.134 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.134 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:28.134 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768786+0000", 2024-09-16T20:48:28.134 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3259999999999996e-06 2024-09-16T20:48:28.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.134 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.134 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:28.134 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768790+0000", 2024-09-16T20:48:28.134 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.216e-06 2024-09-16T20:48:28.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.134 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.134 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:28.134 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819939+0000", 2024-09-16T20:48:28.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051149018999999997 2024-09-16T20:48:28.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.135 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.135 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:28.135 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819979+0000", 2024-09-16T20:48:28.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9311999999999999e-05 2024-09-16T20:48:28.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.135 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.135 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:28.135 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820031+0000", 2024-09-16T20:48:28.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2166999999999997e-05 2024-09-16T20:48:28.135 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.135 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:28.135 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.136 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.136 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.136 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5028 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2166784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:28.136 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768798+0000", 2024-09-16T20:48:28.136 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.3036035070000001, 2024-09-16T20:48:28.136 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051388912000000002, 2024-09-16T20:48:28.136 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:28.136 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:28.136 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:28.136 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:28.136 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:28.136 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:28.136 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5028 2024-09-16T20:48:28.136 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.137 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:28.137 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.137 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:28.137 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768798+0000", 2024-09-16T20:48:28.137 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.137 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.137 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:28.137 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768798+0000", 2024-09-16T20:48:28.137 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.137 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.137 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:28.137 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768989+0000", 2024-09-16T20:48:28.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000190951 2024-09-16T20:48:28.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.138 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:28.138 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768993+0000", 2024-09-16T20:48:28.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9770000000000002e-06 2024-09-16T20:48:28.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.138 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:28.138 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.769002+0000", 2024-09-16T20:48:28.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5469999999999997e-06 2024-09-16T20:48:28.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.138 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:28.139 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.769020+0000", 2024-09-16T20:48:28.139 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8216999999999999e-05 2024-09-16T20:48:28.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.139 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.139 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:28.139 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820070+0000", 2024-09-16T20:48:28.139 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051049998999999999 2024-09-16T20:48:28.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.139 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.139 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:28.139 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820119+0000", 2024-09-16T20:48:28.139 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8962000000000003e-05 2024-09-16T20:48:28.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.139 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.140 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:28.140 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820187+0000", 2024-09-16T20:48:28.140 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8258999999999998e-05 2024-09-16T20:48:28.140 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.140 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:28.140 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.140 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.140 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.140 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5030 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2174976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:28.140 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.769031+0000", 2024-09-16T20:48:28.140 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.3033703939999999, 2024-09-16T20:48:28.140 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051289830000000002, 2024-09-16T20:48:28.140 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:28.141 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:28.141 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:28.141 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:28.141 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:28.141 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:28.141 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5030 2024-09-16T20:48:28.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.141 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:28.141 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.141 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:28.141 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.769031+0000", 2024-09-16T20:48:28.141 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.142 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.142 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:28.142 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.769031+0000", 2024-09-16T20:48:28.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.142 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.142 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:28.142 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.773332+0000", 2024-09-16T20:48:28.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0043004039999999999 2024-09-16T20:48:28.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.142 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.142 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:28.142 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.773336+0000", 2024-09-16T20:48:28.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7849999999999999e-06 2024-09-16T20:48:28.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.143 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:28.143 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.773347+0000", 2024-09-16T20:48:28.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0482999999999999e-05 2024-09-16T20:48:28.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.143 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:28.143 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.773354+0000", 2024-09-16T20:48:28.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1559999999999998e-06 2024-09-16T20:48:28.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.143 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:28.144 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820216+0000", 2024-09-16T20:48:28.144 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046861591000000001 2024-09-16T20:48:28.144 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.144 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.144 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:28.144 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820258+0000", 2024-09-16T20:48:28.144 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2653e-05 2024-09-16T20:48:28.144 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.144 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.144 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:28.144 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820321+0000", 2024-09-16T20:48:28.144 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2757999999999998e-05 2024-09-16T20:48:28.144 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.145 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:28.145 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.145 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.145 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5067 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 6918144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:28.145 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.775279+0000", 2024-09-16T20:48:28.145 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.2971221279999998, 2024-09-16T20:48:28.145 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050171387999999997, 2024-09-16T20:48:28.145 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:28.145 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:28.145 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:28.145 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:28.145 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:28.146 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:28.146 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5067 2024-09-16T20:48:28.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.146 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:28.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.146 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:28.146 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775279+0000", 2024-09-16T20:48:28.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.147 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:28.147 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775279+0000", 2024-09-16T20:48:28.147 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.147 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.147 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:28.147 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775432+0000", 2024-09-16T20:48:28.147 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015257399999999999 2024-09-16T20:48:28.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.147 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.147 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:28.147 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775436+0000", 2024-09-16T20:48:28.147 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0180000000000003e-06 2024-09-16T20:48:28.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.148 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.148 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:28.148 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775447+0000", 2024-09-16T20:48:28.148 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.077e-05 2024-09-16T20:48:28.148 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.148 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.148 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:28.148 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775456+0000", 2024-09-16T20:48:28.148 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4089999999999992e-06 2024-09-16T20:48:28.148 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.148 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.148 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:28.148 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825150+0000", 2024-09-16T20:48:28.148 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049693711000000002 2024-09-16T20:48:28.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.149 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.149 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:28.149 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825337+0000", 2024-09-16T20:48:28.149 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000187214 2024-09-16T20:48:28.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.149 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.149 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:28.149 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825451+0000", 2024-09-16T20:48:28.149 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011369199999999999 2024-09-16T20:48:28.149 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.149 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:28.149 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.150 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.150 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5068 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2330624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:28.150 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.775469+0000", 2024-09-16T20:48:28.150 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.2969321669999996, 2024-09-16T20:48:28.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050198334999999997, 2024-09-16T20:48:28.150 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:28.150 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:28.150 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:28.150 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:28.150 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:28.150 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:28.150 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5068 2024-09-16T20:48:28.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.150 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:28.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:28.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775469+0000", 2024-09-16T20:48:28.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:28.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775469+0000", 2024-09-16T20:48:28.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:28.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775720+0000", 2024-09-16T20:48:28.152 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000250661 2024-09-16T20:48:28.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.152 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.152 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:28.152 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775724+0000", 2024-09-16T20:48:28.152 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8700000000000002e-06 2024-09-16T20:48:28.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.152 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.152 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:28.152 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775733+0000", 2024-09-16T20:48:28.152 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4239999999999999e-06 2024-09-16T20:48:28.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.152 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.152 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:28.153 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775740+0000", 2024-09-16T20:48:28.153 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9519999999999997e-06 2024-09-16T20:48:28.153 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.153 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.153 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:28.153 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825484+0000", 2024-09-16T20:48:28.153 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049743385000000001 2024-09-16T20:48:28.153 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.153 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.153 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:28.153 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825572+0000", 2024-09-16T20:48:28.153 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7834e-05 2024-09-16T20:48:28.153 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.154 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.154 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:28.154 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825668+0000", 2024-09-16T20:48:28.154 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6209e-05 2024-09-16T20:48:28.154 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.154 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:28.154 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.154 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.154 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.154 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5069 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 6926336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:28.154 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.775753+0000", 2024-09-16T20:48:28.154 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.2966489439999993, 2024-09-16T20:48:28.154 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050146430999999998, 2024-09-16T20:48:28.154 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:28.155 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:28.155 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:28.155 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:28.155 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:28.155 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:28.155 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5069 2024-09-16T20:48:28.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.155 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:28.155 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.155 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:28.155 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775753+0000", 2024-09-16T20:48:28.155 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.155 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.156 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:28.156 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775753+0000", 2024-09-16T20:48:28.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.156 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.156 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:28.156 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775865+0000", 2024-09-16T20:48:28.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000112402 2024-09-16T20:48:28.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.156 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.156 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:28.156 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775869+0000", 2024-09-16T20:48:28.157 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5130000000000001e-06 2024-09-16T20:48:28.157 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.157 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.157 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:28.157 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775877+0000", 2024-09-16T20:48:28.157 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1359999999999997e-06 2024-09-16T20:48:28.157 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.157 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.157 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:28.157 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775882+0000", 2024-09-16T20:48:28.157 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6710000000000004e-06 2024-09-16T20:48:28.157 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.157 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.158 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:28.158 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825729+0000", 2024-09-16T20:48:28.158 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049846898000000001 2024-09-16T20:48:28.158 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.158 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.158 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:28.158 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825807+0000", 2024-09-16T20:48:28.158 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7875999999999998e-05 2024-09-16T20:48:28.158 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.158 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.158 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:28.158 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825899+0000", 2024-09-16T20:48:28.159 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1935000000000007e-05 2024-09-16T20:48:28.159 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.159 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:28.159 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.159 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.159 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5070 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2338816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:28.159 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.775892+0000", 2024-09-16T20:48:28.159 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.2965090900000007, 2024-09-16T20:48:28.159 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050155311000000001, 2024-09-16T20:48:28.159 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:28.159 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:28.159 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:28.159 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:28.160 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:28.160 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:28.160 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5070 2024-09-16T20:48:28.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.160 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:28.160 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.160 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:28.160 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775892+0000", 2024-09-16T20:48:28.160 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.160 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.160 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:28.161 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775892+0000", 2024-09-16T20:48:28.161 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.161 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.161 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.161 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:28.161 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776142+0000", 2024-09-16T20:48:28.161 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00024950500000000002 2024-09-16T20:48:28.161 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.161 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.161 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:28.161 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776146+0000", 2024-09-16T20:48:28.161 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2819999999999998e-06 2024-09-16T20:48:28.161 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.161 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.162 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:28.162 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776157+0000", 2024-09-16T20:48:28.162 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0807e-05 2024-09-16T20:48:28.162 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.162 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.162 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:28.162 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776165+0000", 2024-09-16T20:48:28.162 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1300000000000001e-06 2024-09-16T20:48:28.162 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.162 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.162 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:28.162 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825935+0000", 2024-09-16T20:48:28.162 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049769300000000002 2024-09-16T20:48:28.162 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.163 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.163 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:28.163 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825991+0000", 2024-09-16T20:48:28.163 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6177000000000001e-05 2024-09-16T20:48:28.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.163 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.163 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:28.163 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826048+0000", 2024-09-16T20:48:28.163 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.711e-05 2024-09-16T20:48:28.163 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.163 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:28.163 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.163 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.164 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5071 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 6934528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:28.164 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.776177+0000", 2024-09-16T20:48:28.164 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.2962246870000005, 2024-09-16T20:48:28.164 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050036886000000003, 2024-09-16T20:48:28.164 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:28.164 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:28.164 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:28.164 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:28.164 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:28.164 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:28.164 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5071 2024-09-16T20:48:28.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.164 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:28.165 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.165 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:28.165 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776177+0000", 2024-09-16T20:48:28.165 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.165 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.165 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.165 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:28.165 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776177+0000", 2024-09-16T20:48:28.165 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.165 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.165 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.165 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:28.165 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776339+0000", 2024-09-16T20:48:28.165 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016238099999999999 2024-09-16T20:48:28.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.166 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.166 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:28.166 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776343+0000", 2024-09-16T20:48:28.166 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9670000000000003e-06 2024-09-16T20:48:28.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.166 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.166 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:28.166 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776354+0000", 2024-09-16T20:48:28.166 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0789e-05 2024-09-16T20:48:28.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.166 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.166 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:28.166 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776361+0000", 2024-09-16T20:48:28.167 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5200000000000003e-06 2024-09-16T20:48:28.167 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.167 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.167 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:28.167 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826078+0000", 2024-09-16T20:48:28.167 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049717513999999997 2024-09-16T20:48:28.167 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.167 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.167 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:28.167 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826141+0000", 2024-09-16T20:48:28.167 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2446000000000005e-05 2024-09-16T20:48:28.167 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.167 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.167 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:28.168 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826214+0000", 2024-09-16T20:48:28.168 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3269000000000006e-05 2024-09-16T20:48:28.168 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.168 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:28.168 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.168 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.168 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5072 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2347008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:28.168 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.776370+0000", 2024-09-16T20:48:28.168 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.2960311260000008, 2024-09-16T20:48:28.168 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050097653999999998, 2024-09-16T20:48:28.168 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:28.168 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:28.169 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:28.169 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:28.169 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:28.169 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:28.169 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5072 2024-09-16T20:48:28.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.169 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:28.169 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.169 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:28.169 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776370+0000", 2024-09-16T20:48:28.169 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.169 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.169 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:28.170 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776370+0000", 2024-09-16T20:48:28.170 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.170 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.170 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.170 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:28.170 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776645+0000", 2024-09-16T20:48:28.170 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000274634 2024-09-16T20:48:28.170 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.170 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.170 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:28.170 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776654+0000", 2024-09-16T20:48:28.170 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9390000000000003e-06 2024-09-16T20:48:28.170 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.170 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.171 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:28.171 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776665+0000", 2024-09-16T20:48:28.171 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0524e-05 2024-09-16T20:48:28.171 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.171 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.171 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:28.171 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776671+0000", 2024-09-16T20:48:28.171 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7390000000000002e-06 2024-09-16T20:48:28.171 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.171 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.171 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:28.171 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826261+0000", 2024-09-16T20:48:28.171 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049589995999999997 2024-09-16T20:48:28.171 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.172 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.172 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:28.172 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826363+0000", 2024-09-16T20:48:28.172 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010158999999999999 2024-09-16T20:48:28.172 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.172 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.172 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:28.172 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826468+0000", 2024-09-16T20:48:28.172 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000105232 2024-09-16T20:48:28.172 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.172 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:28.172 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.172 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.172 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:28.173 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:28.176 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:48:28.177 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:28.174+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:28.276 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:28.274+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:28.324 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:28.325 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:48:28.325 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:48:28.325 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:28.334 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:48:28.376 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:28.375+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:28.477 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:28.476+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:28.488 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:28.489 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:48:28.489 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:48:28.489 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:48:28.489 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:28.496 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:48:28.578 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:28.576+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:28.605 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:28.605 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:48:28.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:48:28.605 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:48:28.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.605 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:780 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 933888~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:28.605 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.239489+0000", 2024-09-16T20:48:28.605 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 13.363882929000001, 2024-09-16T20:48:28.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00741699, 2024-09-16T20:48:28.606 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:28.606 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:28.606 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:28.606 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:28.606 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:28.607 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:28.607 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 780 2024-09-16T20:48:28.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.607 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:28.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.607 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:28.608 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239489+0000", 2024-09-16T20:48:28.608 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.608 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:28.608 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239489+0000", 2024-09-16T20:48:28.608 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:28.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239490+0000", 2024-09-16T20:48:28.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8599999999999998e-07 2024-09-16T20:48:28.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:28.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239492+0000", 2024-09-16T20:48:28.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1380000000000001e-06 2024-09-16T20:48:28.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.610 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:28.610 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239499+0000", 2024-09-16T20:48:28.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9530000000000002e-06 2024-09-16T20:48:28.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.610 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:28.610 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239504+0000", 2024-09-16T20:48:28.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0690000000000001e-06 2024-09-16T20:48:28.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.611 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:28.611 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246714+0000", 2024-09-16T20:48:28.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.007209844 2024-09-16T20:48:28.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.611 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:28.611 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246804+0000", 2024-09-16T20:48:28.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0109000000000001e-05 2024-09-16T20:48:28.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.612 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:28.612 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246906+0000", 2024-09-16T20:48:28.612 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000102391 2024-09-16T20:48:28.612 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.612 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:28.612 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.613 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:781 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1048576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:28.613 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.239892+0000", 2024-09-16T20:48:28.613 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 13.363480558999999, 2024-09-16T20:48:28.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0072223799999999996, 2024-09-16T20:48:28.613 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:28.613 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:28.613 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:28.613 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:28.614 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:28.614 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:28.614 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 781 2024-09-16T20:48:28.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.614 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:28.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.614 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:28.614 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239892+0000", 2024-09-16T20:48:28.614 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.615 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.615 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:28.615 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239892+0000", 2024-09-16T20:48:28.615 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.615 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.615 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:28.615 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239893+0000", 2024-09-16T20:48:28.615 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1399999999999995e-07 2024-09-16T20:48:28.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.616 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.616 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:28.616 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239897+0000", 2024-09-16T20:48:28.616 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4859999999999999e-06 2024-09-16T20:48:28.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.616 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.616 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:28.617 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239907+0000", 2024-09-16T20:48:28.617 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4539999999999996e-06 2024-09-16T20:48:28.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.617 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:28.617 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239913+0000", 2024-09-16T20:48:28.617 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7880000000000001e-06 2024-09-16T20:48:28.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:28.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246937+0000", 2024-09-16T20:48:28.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0070239999999999999 2024-09-16T20:48:28.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:28.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247013+0000", 2024-09-16T20:48:28.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5894000000000002e-05 2024-09-16T20:48:28.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.619 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:28.619 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247114+0000", 2024-09-16T20:48:28.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000100844 2024-09-16T20:48:28.619 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.619 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:28.619 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.619 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:782 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1146880~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:28.619 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.239925+0000", 2024-09-16T20:48:28.620 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 13.363447585999999, 2024-09-16T20:48:28.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0074161840000000001, 2024-09-16T20:48:28.620 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:28.620 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:28.620 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:28.620 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:28.620 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:28.620 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:28.620 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 782 2024-09-16T20:48:28.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.620 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:28.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.621 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:28.621 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239925+0000", 2024-09-16T20:48:28.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.621 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:28.621 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239925+0000", 2024-09-16T20:48:28.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.621 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:28.621 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239925+0000", 2024-09-16T20:48:28.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7199999999999999e-07 2024-09-16T20:48:28.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.622 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:28.622 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239928+0000", 2024-09-16T20:48:28.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5349999999999999e-06 2024-09-16T20:48:28.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:28.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239935+0000", 2024-09-16T20:48:28.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7680000000000003e-06 2024-09-16T20:48:28.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:28.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239940+0000", 2024-09-16T20:48:28.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.152e-06 2024-09-16T20:48:28.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.624 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.624 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:28.624 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247143+0000", 2024-09-16T20:48:28.624 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0072037309999999997 2024-09-16T20:48:28.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.624 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.624 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:28.624 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247230+0000", 2024-09-16T20:48:28.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6598e-05 2024-09-16T20:48:28.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.625 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:28.625 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247341+0000", 2024-09-16T20:48:28.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011092800000000001 2024-09-16T20:48:28.625 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.625 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:28.625 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.626 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:783 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1261568~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:28.626 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.239947+0000", 2024-09-16T20:48:28.626 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 13.363425264, 2024-09-16T20:48:28.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0076044789999999999, 2024-09-16T20:48:28.626 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:28.627 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:28.627 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:28.627 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:28.627 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:28.627 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:28.627 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 783 2024-09-16T20:48:28.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.628 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:28.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:28.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239947+0000", 2024-09-16T20:48:28.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:28.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239947+0000", 2024-09-16T20:48:28.629 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.629 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.629 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.629 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:28.629 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239947+0000", 2024-09-16T20:48:28.629 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9400000000000001e-07 2024-09-16T20:48:28.629 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.629 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.629 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:28.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239950+0000", 2024-09-16T20:48:28.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2350000000000002e-06 2024-09-16T20:48:28.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:28.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239956+0000", 2024-09-16T20:48:28.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3400000000000003e-06 2024-09-16T20:48:28.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.631 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.631 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:28.631 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239961+0000", 2024-09-16T20:48:28.631 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.04e-06 2024-09-16T20:48:28.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.631 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.631 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:28.632 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247371+0000", 2024-09-16T20:48:28.632 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0074096810000000004 2024-09-16T20:48:28.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.632 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.632 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:28.632 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247451+0000", 2024-09-16T20:48:28.632 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9845999999999994e-05 2024-09-16T20:48:28.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.633 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.633 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:28.633 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247552+0000", 2024-09-16T20:48:28.633 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000100943 2024-09-16T20:48:28.633 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.633 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:28.633 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.633 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.633 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:784 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1376256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:28.633 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.239997+0000", 2024-09-16T20:48:28.634 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 13.363375608, 2024-09-16T20:48:28.634 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.007775372, 2024-09-16T20:48:28.634 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:28.634 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:28.634 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:28.634 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:28.634 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:28.634 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:28.634 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 784 2024-09-16T20:48:28.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.634 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:28.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.634 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:28.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239997+0000", 2024-09-16T20:48:28.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.635 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:28.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239997+0000", 2024-09-16T20:48:28.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.635 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:28.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239997+0000", 2024-09-16T20:48:28.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1799999999999995e-07 2024-09-16T20:48:28.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.636 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:28.636 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240001+0000", 2024-09-16T20:48:28.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6009999999999999e-06 2024-09-16T20:48:28.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.636 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:28.636 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240009+0000", 2024-09-16T20:48:28.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5809999999999999e-06 2024-09-16T20:48:28.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.637 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:28.637 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240015+0000", 2024-09-16T20:48:28.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0100000000000001e-06 2024-09-16T20:48:28.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.637 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:28.638 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247583+0000", 2024-09-16T20:48:28.638 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0075685730000000003 2024-09-16T20:48:28.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.638 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.638 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:28.638 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247672+0000", 2024-09-16T20:48:28.638 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8480000000000007e-05 2024-09-16T20:48:28.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.638 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.638 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:28.638 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247772+0000", 2024-09-16T20:48:28.638 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000100509 2024-09-16T20:48:28.639 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.639 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:28.639 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.639 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:785 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1474560~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:28.639 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240024+0000", 2024-09-16T20:48:28.639 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 13.363348022, 2024-09-16T20:48:28.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.008096776, 2024-09-16T20:48:28.639 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:28.639 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:28.640 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:28.640 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:28.640 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:28.640 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:28.640 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 785 2024-09-16T20:48:28.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.640 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:28.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.640 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:28.640 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240024+0000", 2024-09-16T20:48:28.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:28.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240024+0000", 2024-09-16T20:48:28.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:28.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240025+0000", 2024-09-16T20:48:28.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5000000000000003e-07 2024-09-16T20:48:28.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.642 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:28.642 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240027+0000", 2024-09-16T20:48:28.642 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2900000000000001e-06 2024-09-16T20:48:28.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.642 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:28.642 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240036+0000", 2024-09-16T20:48:28.642 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4710000000000007e-06 2024-09-16T20:48:28.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.643 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:28.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240041+0000", 2024-09-16T20:48:28.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4689999999999996e-06 2024-09-16T20:48:28.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.643 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:28.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247821+0000", 2024-09-16T20:48:28.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0077797919999999998 2024-09-16T20:48:28.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.644 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:28.644 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247921+0000", 2024-09-16T20:48:28.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9814000000000004e-05 2024-09-16T20:48:28.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:28.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248121+0000", 2024-09-16T20:48:28.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00020039 2024-09-16T20:48:28.645 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.645 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:28.645 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.646 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:940 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 81920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:28.646 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240569+0000", 2024-09-16T20:48:28.646 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 13.362803764000001, 2024-09-16T20:48:28.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0078289759999999996, 2024-09-16T20:48:28.646 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:28.646 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:28.646 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:28.646 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:28.646 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:28.646 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:28.647 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 940 2024-09-16T20:48:28.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.647 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:28.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.647 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:28.647 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240569+0000", 2024-09-16T20:48:28.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:28.648 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240569+0000", 2024-09-16T20:48:28.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:28.648 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240569+0000", 2024-09-16T20:48:28.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2600000000000001e-07 2024-09-16T20:48:28.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:28.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240575+0000", 2024-09-16T20:48:28.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1159999999999998e-06 2024-09-16T20:48:28.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:28.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240587+0000", 2024-09-16T20:48:28.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1915e-05 2024-09-16T20:48:28.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.650 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:28.650 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240594+0000", 2024-09-16T20:48:28.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5610000000000001e-06 2024-09-16T20:48:28.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.650 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:28.650 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248158+0000", 2024-09-16T20:48:28.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0075635370000000004 2024-09-16T20:48:28.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.650 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:28.650 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248278+0000", 2024-09-16T20:48:28.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012009400000000001 2024-09-16T20:48:28.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.651 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:28.651 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248398+0000", 2024-09-16T20:48:28.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000119827 2024-09-16T20:48:28.651 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.651 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:28.651 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.652 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:941 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 196608~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:28.652 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240608+0000", 2024-09-16T20:48:28.652 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 13.362764187, 2024-09-16T20:48:28.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0080495519999999997, 2024-09-16T20:48:28.652 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:28.652 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:28.652 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:28.652 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:28.652 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:28.652 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:28.652 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 941 2024-09-16T20:48:28.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.653 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:28.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:28.653 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240608+0000", 2024-09-16T20:48:28.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:28.653 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240608+0000", 2024-09-16T20:48:28.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:28.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240609+0000", 2024-09-16T20:48:28.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7000000000000004e-07 2024-09-16T20:48:28.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:28.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240612+0000", 2024-09-16T20:48:28.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8310000000000001e-06 2024-09-16T20:48:28.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.655 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.655 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:28.655 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240619+0000", 2024-09-16T20:48:28.655 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8429999999999993e-06 2024-09-16T20:48:28.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.655 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.655 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:28.656 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240625+0000", 2024-09-16T20:48:28.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5659999999999997e-06 2024-09-16T20:48:28.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.656 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:28.656 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248456+0000", 2024-09-16T20:48:28.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0078313940000000002 2024-09-16T20:48:28.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.656 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:28.656 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248548+0000", 2024-09-16T20:48:28.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1324999999999997e-05 2024-09-16T20:48:28.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.657 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:28.657 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248658+0000", 2024-09-16T20:48:28.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000109923 2024-09-16T20:48:28.657 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.657 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:28.657 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.657 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:942 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 294912~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:28.657 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240675+0000", 2024-09-16T20:48:28.657 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 13.36269738, 2024-09-16T20:48:28.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0082135699999999999, 2024-09-16T20:48:28.658 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:28.658 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:28.658 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:28.658 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:28.658 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:28.658 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:28.658 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 942 2024-09-16T20:48:28.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.658 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:28.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.659 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:28.659 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240675+0000", 2024-09-16T20:48:28.659 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.659 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.659 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:28.659 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240675+0000", 2024-09-16T20:48:28.659 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.659 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.659 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:28.659 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240676+0000", 2024-09-16T20:48:28.660 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.37e-07 2024-09-16T20:48:28.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.660 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:28.660 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240679+0000", 2024-09-16T20:48:28.660 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4060000000000001e-06 2024-09-16T20:48:28.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.660 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:28.660 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240687+0000", 2024-09-16T20:48:28.660 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4640000000000006e-06 2024-09-16T20:48:28.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.661 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:28.661 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240694+0000", 2024-09-16T20:48:28.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.246e-06 2024-09-16T20:48:28.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.661 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:28.661 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248693+0000", 2024-09-16T20:48:28.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0079997550000000008 2024-09-16T20:48:28.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.661 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:28.661 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248781+0000", 2024-09-16T20:48:28.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7509999999999994e-05 2024-09-16T20:48:28.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.662 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.662 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:28.662 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248889+0000", 2024-09-16T20:48:28.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000107552 2024-09-16T20:48:28.662 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.662 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:28.662 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.663 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:943 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 409600~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:28.663 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240704+0000", 2024-09-16T20:48:28.663 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 13.362668383999999, 2024-09-16T20:48:28.663 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0083978209999999998, 2024-09-16T20:48:28.663 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:28.663 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:28.663 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:28.663 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:28.663 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:28.663 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:28.664 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 943 2024-09-16T20:48:28.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.664 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:28.664 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.664 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:28.664 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240704+0000", 2024-09-16T20:48:28.664 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.664 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.664 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:28.664 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240704+0000", 2024-09-16T20:48:28.664 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.665 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.665 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.665 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:28.665 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240704+0000", 2024-09-16T20:48:28.665 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7e-07 2024-09-16T20:48:28.665 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.665 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.665 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:28.665 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240707+0000", 2024-09-16T20:48:28.665 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5809999999999999e-06 2024-09-16T20:48:28.665 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.665 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.666 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:28.666 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240713+0000", 2024-09-16T20:48:28.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4949999999999996e-06 2024-09-16T20:48:28.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.666 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:28.666 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240718+0000", 2024-09-16T20:48:28.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8890000000000001e-06 2024-09-16T20:48:28.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.666 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:28.666 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248920+0000", 2024-09-16T20:48:28.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0082018679999999993 2024-09-16T20:48:28.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.667 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:28.667 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249007+0000", 2024-09-16T20:48:28.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6327000000000002e-05 2024-09-16T20:48:28.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.667 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:28.667 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249102+0000", 2024-09-16T20:48:28.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5191000000000004e-05 2024-09-16T20:48:28.667 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.667 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:28.667 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.668 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.668 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.668 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:944 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 524288~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:28.668 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240726+0000", 2024-09-16T20:48:28.668 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 13.362646579, 2024-09-16T20:48:28.668 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0085999960000000004, 2024-09-16T20:48:28.668 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:28.668 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:28.668 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:28.668 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:28.668 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:28.668 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:28.668 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 944 2024-09-16T20:48:28.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.669 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:28.669 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.669 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:28.669 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240726+0000", 2024-09-16T20:48:28.669 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.669 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.669 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:28.669 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240726+0000", 2024-09-16T20:48:28.669 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.670 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:28.670 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240726+0000", 2024-09-16T20:48:28.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7099999999999997e-07 2024-09-16T20:48:28.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.670 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:28.670 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240728+0000", 2024-09-16T20:48:28.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.125e-06 2024-09-16T20:48:28.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.670 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:28.671 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240734+0000", 2024-09-16T20:48:28.671 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1890000000000004e-06 2024-09-16T20:48:28.671 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.671 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.671 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:28.671 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240739+0000", 2024-09-16T20:48:28.671 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0340000000000004e-06 2024-09-16T20:48:28.671 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.671 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.671 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:28.671 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249131+0000", 2024-09-16T20:48:28.671 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0083919579999999997 2024-09-16T20:48:28.672 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.672 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.672 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:28.672 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249217+0000", 2024-09-16T20:48:28.672 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5154999999999994e-05 2024-09-16T20:48:28.672 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.672 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.672 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:28.672 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249326+0000", 2024-09-16T20:48:28.672 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000109164 2024-09-16T20:48:28.673 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.673 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:28.673 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.673 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.673 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.673 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:945 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 622592~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:28.673 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240746+0000", 2024-09-16T20:48:28.673 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 13.36262644, 2024-09-16T20:48:28.673 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0088083320000000003, 2024-09-16T20:48:28.673 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:28.673 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:28.674 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:28.674 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:28.674 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:28.674 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:28.674 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 945 2024-09-16T20:48:28.674 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.674 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:28.674 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.674 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:28.674 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240746+0000", 2024-09-16T20:48:28.674 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.674 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.675 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.675 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:28.675 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240746+0000", 2024-09-16T20:48:28.675 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.675 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.675 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:28.675 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240746+0000", 2024-09-16T20:48:28.675 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8500000000000002e-07 2024-09-16T20:48:28.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.675 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.675 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:28.676 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240748+0000", 2024-09-16T20:48:28.676 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8449999999999999e-06 2024-09-16T20:48:28.676 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.676 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.676 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:28.676 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240754+0000", 2024-09-16T20:48:28.676 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8830000000000003e-06 2024-09-16T20:48:28.676 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.676 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.677 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:28.677 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240759+0000", 2024-09-16T20:48:28.677 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0220000000000004e-06 2024-09-16T20:48:28.677 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.677 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.677 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:28.677 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249360+0000", 2024-09-16T20:48:28.677 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.008601437 2024-09-16T20:48:28.677 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.677 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.677 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:28.677 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249453+0000", 2024-09-16T20:48:28.678 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2743000000000003e-05 2024-09-16T20:48:28.678 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.678 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.678 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:28.678 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249554+0000", 2024-09-16T20:48:28.678 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101017 2024-09-16T20:48:28.678 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.678 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:28.678 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.678 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.678 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.678 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:946 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 737280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:28.678 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240766+0000", 2024-09-16T20:48:28.679 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 13.362606737, 2024-09-16T20:48:28.679 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0089092070000000006, 2024-09-16T20:48:28.679 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:28.679 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:28.679 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:28.679 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:28.679 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:28.679 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:28.679 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 946 2024-09-16T20:48:28.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.679 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:28.679 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.679 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:28.680 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240766+0000", 2024-09-16T20:48:28.680 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.680 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.680 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:28.680 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240766+0000", 2024-09-16T20:48:28.680 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.680 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.680 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:28.680 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240766+0000", 2024-09-16T20:48:28.680 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0600000000000001e-07 2024-09-16T20:48:28.681 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.681 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.681 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:28.681 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240768+0000", 2024-09-16T20:48:28.681 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8959999999999999e-06 2024-09-16T20:48:28.681 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.681 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.681 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:28.681 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240774+0000", 2024-09-16T20:48:28.681 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0979999999999999e-06 2024-09-16T20:48:28.681 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.682 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.682 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:28.682 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240779+0000", 2024-09-16T20:48:28.682 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.933e-06 2024-09-16T20:48:28.682 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.682 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.682 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:28.682 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249572+0000", 2024-09-16T20:48:28.682 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0087926380000000002 2024-09-16T20:48:28.682 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.683 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.683 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:28.683 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249612+0000", 2024-09-16T20:48:28.683 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0859000000000003e-05 2024-09-16T20:48:28.683 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.683 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.683 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:28.683 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249675+0000", 2024-09-16T20:48:28.683 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2377000000000003e-05 2024-09-16T20:48:28.683 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.683 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:28.683 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.683 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.684 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.684 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:947 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 851968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:28.684 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240785+0000", 2024-09-16T20:48:28.684 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 13.362587144999999, 2024-09-16T20:48:28.684 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0090960260000000001, 2024-09-16T20:48:28.684 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:28.684 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:28.684 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:28.684 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:28.684 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:28.684 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:28.684 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 947 2024-09-16T20:48:28.684 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.684 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:28.685 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.685 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:28.685 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240785+0000", 2024-09-16T20:48:28.685 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.685 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.685 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.685 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:28.685 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240785+0000", 2024-09-16T20:48:28.685 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.685 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.685 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.685 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:28.685 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240786+0000", 2024-09-16T20:48:28.686 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-16T20:48:28.686 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.686 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.686 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:28.686 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240787+0000", 2024-09-16T20:48:28.686 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.858e-06 2024-09-16T20:48:28.686 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.686 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.686 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:28.686 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240793+0000", 2024-09-16T20:48:28.686 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7760000000000003e-06 2024-09-16T20:48:28.686 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.686 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.686 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:28.687 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240799+0000", 2024-09-16T20:48:28.687 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1109999999999996e-06 2024-09-16T20:48:28.687 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.687 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.687 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:28.687 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249715+0000", 2024-09-16T20:48:28.687 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0089159789999999992 2024-09-16T20:48:28.688 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.688 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.688 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:28.688 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249766+0000", 2024-09-16T20:48:28.688 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0968999999999999e-05 2024-09-16T20:48:28.688 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.688 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.688 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:28.688 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249881+0000", 2024-09-16T20:48:28.688 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114963 2024-09-16T20:48:28.688 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.688 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:28.688 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.689 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.689 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.689 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:948 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:28.689 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240806+0000", 2024-09-16T20:48:28.689 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 13.362565965, 2024-09-16T20:48:28.689 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0092300609999999995, 2024-09-16T20:48:28.689 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:28.689 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:28.689 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:28.689 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:28.689 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:28.689 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:28.689 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 948 2024-09-16T20:48:28.689 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.690 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:28.690 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.690 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:28.690 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240806+0000", 2024-09-16T20:48:28.690 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.690 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.690 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.690 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:28.692 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240806+0000", 2024-09-16T20:48:28.692 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.692 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.692 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.692 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:28.692 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240807+0000", 2024-09-16T20:48:28.692 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4499999999999998e-07 2024-09-16T20:48:28.692 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.693 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.693 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:28.693 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240809+0000", 2024-09-16T20:48:28.693 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.34e-06 2024-09-16T20:48:28.693 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.693 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.693 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:28.693 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240815+0000", 2024-09-16T20:48:28.693 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1449999999999996e-06 2024-09-16T20:48:28.693 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.693 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.693 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:28.693 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240821+0000", 2024-09-16T20:48:28.694 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0270000000000001e-06 2024-09-16T20:48:28.694 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.694 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.694 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:28.694 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249920+0000", 2024-09-16T20:48:28.694 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0090988460000000007 2024-09-16T20:48:28.694 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.694 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.694 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:28.694 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249966+0000", 2024-09-16T20:48:28.694 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5918999999999999e-05 2024-09-16T20:48:28.694 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.695 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.695 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:28.695 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250036+0000", 2024-09-16T20:48:28.695 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0438999999999994e-05 2024-09-16T20:48:28.695 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.695 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:28.695 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.695 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.695 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.695 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:949 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1064960~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:28.695 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240833+0000", 2024-09-16T20:48:28.695 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 13.36253935, 2024-09-16T20:48:28.696 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0093580339999999994, 2024-09-16T20:48:28.696 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:28.696 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:28.696 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:28.696 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:28.696 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:28.696 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:28.696 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 949 2024-09-16T20:48:28.696 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.696 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:28.696 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.696 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:28.696 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240833+0000", 2024-09-16T20:48:28.696 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.697 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.697 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.697 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:28.697 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240833+0000", 2024-09-16T20:48:28.697 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.697 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.697 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.697 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:28.697 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240833+0000", 2024-09-16T20:48:28.697 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2899999999999999e-07 2024-09-16T20:48:28.697 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.697 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.697 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:28.697 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240836+0000", 2024-09-16T20:48:28.698 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2649999999999999e-06 2024-09-16T20:48:28.698 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.698 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.698 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:28.698 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240842+0000", 2024-09-16T20:48:28.698 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5579999999999997e-06 2024-09-16T20:48:28.698 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.698 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.698 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:28.698 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240847+0000", 2024-09-16T20:48:28.698 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2009999999999998e-06 2024-09-16T20:48:28.698 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.698 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.699 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:28.699 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250073+0000", 2024-09-16T20:48:28.699 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0092258710000000001 2024-09-16T20:48:28.699 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.699 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.699 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:28.699 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250141+0000", 2024-09-16T20:48:28.699 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7849999999999996e-05 2024-09-16T20:48:28.699 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.699 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.699 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:28.699 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250191+0000", 2024-09-16T20:48:28.700 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.986e-05 2024-09-16T20:48:28.700 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.700 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:28.700 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.700 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.700 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.700 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:950 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1179648~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:28.700 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240855+0000", 2024-09-16T20:48:28.700 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 13.362517098, 2024-09-16T20:48:28.700 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0095855560000000003, 2024-09-16T20:48:28.700 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:28.700 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:28.700 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:28.701 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:28.701 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:28.701 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:28.701 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 950 2024-09-16T20:48:28.701 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.701 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:28.701 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.701 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:28.701 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240855+0000", 2024-09-16T20:48:28.701 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.701 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.701 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.701 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:28.701 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240855+0000", 2024-09-16T20:48:28.702 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.702 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.702 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.702 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:28.702 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240856+0000", 2024-09-16T20:48:28.702 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4999999999999998e-07 2024-09-16T20:48:28.702 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.702 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.702 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:28.702 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240858+0000", 2024-09-16T20:48:28.702 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.996e-06 2024-09-16T20:48:28.702 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.702 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.703 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:28.703 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240864+0000", 2024-09-16T20:48:28.703 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9359999999999996e-06 2024-09-16T20:48:28.703 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.703 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.703 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:28.703 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240868+0000", 2024-09-16T20:48:28.703 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7670000000000003e-06 2024-09-16T20:48:28.703 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.703 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.703 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:28.703 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250226+0000", 2024-09-16T20:48:28.703 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0093576100000000006 2024-09-16T20:48:28.703 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.704 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.704 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:28.704 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250289+0000", 2024-09-16T20:48:28.704 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2627999999999997e-05 2024-09-16T20:48:28.704 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.704 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.704 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:28.704 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250441+0000", 2024-09-16T20:48:28.704 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015216900000000001 2024-09-16T20:48:28.704 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.704 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:28.704 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.704 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.705 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.705 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:951 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1277952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:28.705 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.241044+0000", 2024-09-16T20:48:28.705 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 13.362327909999999, 2024-09-16T20:48:28.705 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0096266589999999992, 2024-09-16T20:48:28.705 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:28.705 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:28.705 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:28.705 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:28.705 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:28.705 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:28.706 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 951 2024-09-16T20:48:28.706 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.706 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:28.706 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.706 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:28.706 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241044+0000", 2024-09-16T20:48:28.706 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.706 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.706 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.706 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:28.706 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241044+0000", 2024-09-16T20:48:28.706 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.707 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.707 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.707 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:28.707 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241045+0000", 2024-09-16T20:48:28.707 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5300000000000004e-07 2024-09-16T20:48:28.707 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.707 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.707 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:28.707 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241048+0000", 2024-09-16T20:48:28.707 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.303e-06 2024-09-16T20:48:28.707 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.707 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.708 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:28.708 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241057+0000", 2024-09-16T20:48:28.708 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5550000000000003e-06 2024-09-16T20:48:28.708 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.708 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.708 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:28.708 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241063+0000", 2024-09-16T20:48:28.708 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3470000000000004e-06 2024-09-16T20:48:28.708 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.708 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.708 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:28.709 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250477+0000", 2024-09-16T20:48:28.709 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0094136159999999997 2024-09-16T20:48:28.709 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.709 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.709 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:28.709 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250562+0000", 2024-09-16T20:48:28.709 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4709000000000005e-05 2024-09-16T20:48:28.709 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.709 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.709 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:28.710 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250671+0000", 2024-09-16T20:48:28.710 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010947599999999999 2024-09-16T20:48:28.710 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.710 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:28.710 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.710 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.710 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.710 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:952 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1392640~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:28.710 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.241130+0000", 2024-09-16T20:48:28.710 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 13.362242044, 2024-09-16T20:48:28.710 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0098096290000000003, 2024-09-16T20:48:28.710 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:28.710 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:28.711 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:28.711 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:28.711 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:28.711 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:28.711 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 952 2024-09-16T20:48:28.711 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.711 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:28.711 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.711 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:28.711 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241130+0000", 2024-09-16T20:48:28.711 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.712 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.712 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.712 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:28.712 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241130+0000", 2024-09-16T20:48:28.712 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.712 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.712 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.712 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:28.712 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241131+0000", 2024-09-16T20:48:28.712 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.61e-07 2024-09-16T20:48:28.712 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.713 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.713 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:28.713 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241134+0000", 2024-09-16T20:48:28.713 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1080000000000001e-06 2024-09-16T20:48:28.713 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.713 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.713 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:28.713 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241142+0000", 2024-09-16T20:48:28.713 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9119999999999998e-06 2024-09-16T20:48:28.713 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.713 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.714 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:28.714 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241148+0000", 2024-09-16T20:48:28.714 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1410000000000002e-06 2024-09-16T20:48:28.714 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.714 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.714 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:28.714 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250731+0000", 2024-09-16T20:48:28.714 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0095833979999999999 2024-09-16T20:48:28.714 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.714 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.714 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:28.715 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250820+0000", 2024-09-16T20:48:28.715 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8997000000000002e-05 2024-09-16T20:48:28.715 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.715 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.715 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:28.715 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250940+0000", 2024-09-16T20:48:28.715 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000119412 2024-09-16T20:48:28.715 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.715 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:28.715 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.715 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.716 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.716 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:953 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1507328~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:28.716 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.241157+0000", 2024-09-16T20:48:28.716 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 13.362215123, 2024-09-16T20:48:28.716 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.010039259, 2024-09-16T20:48:28.716 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:28.716 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:28.716 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:28.716 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:28.716 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:28.717 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:28.717 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 953 2024-09-16T20:48:28.717 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.717 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:28.717 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.717 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:28.717 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241157+0000", 2024-09-16T20:48:28.717 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.717 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.717 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.717 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:28.717 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241157+0000", 2024-09-16T20:48:28.718 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:28.718 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.718 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.718 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:28.718 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241158+0000", 2024-09-16T20:48:28.718 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.01e-07 2024-09-16T20:48:28.718 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.718 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.718 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:28.718 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241160+0000", 2024-09-16T20:48:28.718 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2879999999999999e-06 2024-09-16T20:48:28.718 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.718 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.719 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:28.719 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241166+0000", 2024-09-16T20:48:28.719 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.421e-06 2024-09-16T20:48:28.719 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.719 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.719 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:28.719 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241171+0000", 2024-09-16T20:48:28.719 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9840000000000001e-06 2024-09-16T20:48:28.719 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.719 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.719 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:28.720 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250988+0000", 2024-09-16T20:48:28.720 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0098171449999999993 2024-09-16T20:48:28.720 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.720 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.720 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:28.720 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.251085+0000", 2024-09-16T20:48:28.720 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6877000000000001e-05 2024-09-16T20:48:28.720 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:28.720 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:28.720 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:28.720 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.251196+0000", 2024-09-16T20:48:28.721 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000111143 2024-09-16T20:48:28.721 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.721 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:28.721 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.721 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:28.721 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:28.721 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:28.724 INFO:teuthology.orchestra.run.smithi029.stderr:nodeep-scrub is unset 2024-09-16T20:48:28.724 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:28.722+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:28.724 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:48:28.824 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:28.823+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:28.879 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:28.880 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:48:28.880 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:48:28.880 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:28.888 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:48:28.917 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T20:48:28.929 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:28.928+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:29.026 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:29.024+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:29.038 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:29.038 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:48:29.038 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:48:29.039 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:48:29.039 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:29.047 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:48:29.053 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:48:29.053 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:48:29.070 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:48:29.071 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:48:29.087 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:48:29.088 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:48:29.104 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:48:29.105 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:48:29.122 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:48:29.122 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:48:29.126 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:29.125+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:29.139 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:48:29.139 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:48:29.155 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:29.155 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:29.164 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:48:29.227 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:29.226+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:29.312 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:29.312 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:29.321 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:48:29.328 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:29.326+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:29.428 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:29.428 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:29.429 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:29.427+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:29.437 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:48:29.529 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:29.528+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:29.585 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:29.585 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:29.593 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:48:29.636 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:29.635+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:29.708 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:29.708 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:29.717 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:48:29.730 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:29.729+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:29.746 INFO:teuthology.orchestra.run.smithi139.stdout:truncating image image_client.0-clone7 from 0xbe75c00 (overlap 0x1e48200) to 0x60e600 2024-09-16T20:48:29.831 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:29.830+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:29.874 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:29.874 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:29.883 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:48:29.933 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:29.931+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:29.992 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:29.993 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:30.001 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:48:30.034 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:30.032+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:30.123 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:30.123 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:30.132 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:48:30.135 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:30.133+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:30.137 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #6, image image_client.0-clone6 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent7 2024-09-16T20:48:30.226 INFO:teuthology.orchestra.run.smithi139.stdout:976 write 0xa956c00 thru 0xa963323 (0xc724 bytes) 2024-09-16T20:48:30.226 INFO:teuthology.orchestra.run.smithi139.stdout:978 write 0x84edc00 thru 0x84fccd4 (0xf0d5 bytes) 2024-09-16T20:48:30.227 INFO:teuthology.orchestra.run.smithi139.stdout:983 punch from 0x40e5800 to 0x40ec600, (0x6e00 bytes) 2024-09-16T20:48:30.237 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:30.233+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:30.240 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:30.240 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:30.250 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:48:30.260 INFO:teuthology.orchestra.run.smithi139.stdout:985 read 0x4690200 thru 0x4692712 (0x2513 bytes) 2024-09-16T20:48:30.260 INFO:teuthology.orchestra.run.smithi139.stdout:988 read 0x9319600 thru 0x931b502 (0x1f03 bytes) 2024-09-16T20:48:30.266 INFO:teuthology.orchestra.run.smithi139.stdout:990 trunc from 0xbe75c00 to 0x6de6c00 2024-09-16T20:48:30.294 INFO:teuthology.orchestra.run.smithi139.stdout:992 punch from 0x2b86a00 to 0x2b89800, (0x2e00 bytes) 2024-09-16T20:48:30.295 INFO:teuthology.orchestra.run.smithi139.stdout:994 read 0x582b400 thru 0x5831fea (0x6beb bytes) 2024-09-16T20:48:30.295 INFO:teuthology.orchestra.run.smithi139.stdout:995 write 0x198e400 thru 0x1998384 (0x9f85 bytes) 2024-09-16T20:48:30.298 INFO:teuthology.orchestra.run.smithi139.stdout:999 punch from 0x1bee400 to 0x1bf6400, (0x8000 bytes) 2024-09-16T20:48:30.301 INFO:teuthology.orchestra.run.smithi139.stdout:1001 write 0x31d800 thru 0x32b9c8 (0xe1c9 bytes) 2024-09-16T20:48:30.336 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:30.334+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:30.355 INFO:teuthology.orchestra.run.smithi139.stdout:1003 write 0xc495200 thru 0xc4a2dd2 (0xdbd3 bytes) 2024-09-16T20:48:30.361 INFO:teuthology.orchestra.run.smithi139.stdout:1006 trunc from 0xc4a2e00 to 0xc932c00 2024-09-16T20:48:30.378 INFO:teuthology.orchestra.run.smithi139.stdout:1007 trunc from 0xc932c00 to 0xc5bd000 2024-09-16T20:48:30.405 INFO:teuthology.orchestra.run.smithi139.stdout:1010 trunc from 0xc5bd000 to 0x367fc00 2024-09-16T20:48:30.411 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:30.411 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:30.423 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:48:30.443 INFO:teuthology.orchestra.run.smithi139.stdout:1012 punch from 0x137a000 to 0x1382e00, (0x8e00 bytes) 2024-09-16T20:48:30.444 INFO:teuthology.orchestra.run.smithi139.stdout:1013 read 0x72b800 thru 0x736ed9 (0xb6da bytes) 2024-09-16T20:48:30.455 INFO:teuthology.orchestra.run.smithi139.stdout:1015 trunc from 0x367fc00 to 0x721000 2024-09-16T20:48:30.465 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:30.463+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:30.506 INFO:teuthology.orchestra.run.smithi139.stdout:1016 read 0x1b2000 thru 0x1b8b8a (0x6b8b bytes) 2024-09-16T20:48:30.537 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:30.535+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:30.540 INFO:teuthology.orchestra.run.smithi139.stdout:1017 write 0xed15c00 thru 0xed1dcbf (0x80c0 bytes) 2024-09-16T20:48:30.541 INFO:teuthology.orchestra.run.smithi139.stdout:1018 write 0x3f93000 thru 0x3f96029 (0x302a bytes) 2024-09-16T20:48:30.541 INFO:teuthology.orchestra.run.smithi139.stdout:1020 punch from 0x1cf0800 to 0x1cf2400, (0x1c00 bytes) 2024-09-16T20:48:30.542 INFO:teuthology.orchestra.run.smithi139.stdout:1021 read 0x20a1c00 thru 0x20a3402 (0x1803 bytes) 2024-09-16T20:48:30.547 INFO:teuthology.orchestra.run.smithi139.stdout:1022 trunc from 0xed1de00 to 0xae99c00 2024-09-16T20:48:30.575 INFO:teuthology.orchestra.run.smithi139.stdout:1023 punch from 0x483d400 to 0x484a400, (0xd000 bytes) 2024-09-16T20:48:30.575 INFO:teuthology.orchestra.run.smithi139.stdout:1025 write 0x971800 thru 0x97d84d (0xc04e bytes) 2024-09-16T20:48:30.575 INFO:teuthology.orchestra.run.smithi139.stdout:1030 write 0x5f9a000 thru 0x5fa835b (0xe35c bytes) 2024-09-16T20:48:30.575 INFO:teuthology.orchestra.run.smithi139.stdout:1032 clone 9 order 20 su 524288 sc 7 2024-09-16T20:48:30.580 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:30.580 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:30.590 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:48:30.638 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:30.637+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:30.649 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5] out_osds: [3] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-16T20:48:30.649 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T20:48:30.650 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2024-09-16T20:48:30.650 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T20:48:30.650 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T20:48:30.650 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok config set filestore_inject_stall 3 2024-09-16T20:48:30.725 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T20:48:30.746 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:30.747 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:30.753 INFO:teuthology.orchestra.run.smithi139.stdout:leaving image image_client.0-clone8 intact 2024-09-16T20:48:30.757 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:48:30.769 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:30.768+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:30.787 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:30.787 INFO:teuthology.orchestra.run.smithi043.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-16T20:48:30.787 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:30.840 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:30.838+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:30.865 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:30.865 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:30.874 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:48:30.965 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:30.956+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:31.023 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:31.023 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:31.032 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:48:31.073 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:31.071+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:31.142 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:31.140+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:31.150 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T20:48:31.179 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:31.180 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:31.188 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:48:31.194 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #7, image image_client.0-clone7 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent8 2024-09-16T20:48:31.271 INFO:teuthology.orchestra.run.smithi139.stdout:1035 read 0x8cfee00 thru 0x8cffe2f (0x1030 bytes) 2024-09-16T20:48:31.272 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:31.271+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:31.281 INFO:teuthology.orchestra.run.smithi139.stdout:1036 trunc from 0xae99c00 to 0x3d41c00 2024-09-16T20:48:31.282 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:48:31.282 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:48:31.302 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:48:31.302 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:48:31.319 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:48:31.319 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:48:31.334 INFO:teuthology.orchestra.run.smithi139.stdout:1037 write 0xebe5200 thru 0xebebb82 (0x6983 bytes) 2024-09-16T20:48:31.334 INFO:teuthology.orchestra.run.smithi139.stdout:1040 flatten 2024-09-16T20:48:31.336 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:48:31.337 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:48:31.343 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:31.343 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:31.343 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:31.341+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:31.355 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:48:31.369 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:48:31.369 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:48:31.395 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:48:31.395 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:48:31.399 INFO:teuthology.orchestra.run.smithi139.stdout:1042 read 0xb27de00 thru 0xb28a754 (0xc955 bytes) 2024-09-16T20:48:31.400 INFO:teuthology.orchestra.run.smithi139.stdout:1043 write 0x7f69200 thru 0x7f6eea0 (0x5ca1 bytes) 2024-09-16T20:48:31.400 INFO:teuthology.orchestra.run.smithi139.stdout:1045 read 0x2bfa400 thru 0x2bfa6be (0x2bf bytes) 2024-09-16T20:48:31.400 INFO:teuthology.orchestra.run.smithi139.stdout:1048 write 0x5dd0600 thru 0x5dd3fb4 (0x39b5 bytes) 2024-09-16T20:48:31.400 INFO:teuthology.orchestra.run.smithi139.stdout:1049 punch from 0xbef9600 to 0xbefda00, (0x4400 bytes) 2024-09-16T20:48:31.401 INFO:teuthology.orchestra.run.smithi139.stdout:1051 punch from 0x43a4000 to 0x43ac400, (0x8400 bytes) 2024-09-16T20:48:31.401 INFO:teuthology.orchestra.run.smithi139.stdout:1052 read 0x60b1600 thru 0x60b997d (0x837e bytes) 2024-09-16T20:48:31.402 INFO:teuthology.orchestra.run.smithi139.stdout:1053 trunc from 0xebebc00 to 0xd373e00 2024-09-16T20:48:31.427 INFO:teuthology.orchestra.run.smithi139.stdout:1055 punch from 0xc63d000 to 0xc644000, (0x7000 bytes) 2024-09-16T20:48:31.428 INFO:teuthology.orchestra.run.smithi139.stdout:1056 write 0x2708e00 thru 0x2714da6 (0xbfa7 bytes) 2024-09-16T20:48:31.428 INFO:teuthology.orchestra.run.smithi139.stdout:1063 write 0x16c7e00 thru 0x16c962e (0x182f bytes) 2024-09-16T20:48:31.442 INFO:teuthology.orchestra.run.smithi139.stdout:1064 trunc from 0xd373e00 to 0xb83000 2024-09-16T20:48:31.449 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:31.448+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:31.471 INFO:teuthology.orchestra.run.smithi139.stdout:skipping zero length punch hole 2024-09-16T20:48:31.484 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:31.484 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:48:31.485 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:48:31.485 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:31.493 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:48:31.494 INFO:teuthology.orchestra.run.smithi139.stdout:1070 write 0x704b600 thru 0x7050b0f (0x5510 bytes) 2024-09-16T20:48:31.495 INFO:teuthology.orchestra.run.smithi139.stdout:1072 punch from 0x535d200 to 0x536b400, (0xe200 bytes) 2024-09-16T20:48:31.496 INFO:teuthology.orchestra.run.smithi139.stdout:1073 read 0x1f4d600 thru 0x1f58be6 (0xb5e7 bytes) 2024-09-16T20:48:31.496 INFO:teuthology.orchestra.run.smithi139.stdout:1074 write 0x5513200 thru 0x551721f (0x4020 bytes) 2024-09-16T20:48:31.502 INFO:teuthology.orchestra.run.smithi139.stdout:1075 trunc from 0x7050c00 to 0x1a54800 2024-09-16T20:48:31.523 INFO:teuthology.orchestra.run.smithi139.stdout:1077 read 0xbe4600 thru 0xbe9334 (0x4d35 bytes) 2024-09-16T20:48:31.530 INFO:teuthology.orchestra.run.smithi139.stdout:1079 trunc from 0x1a54800 to 0x68b4800 2024-09-16T20:48:31.544 INFO:teuthology.orchestra.run.smithi139.stdout:1081 read 0x49f7c00 thru 0x4a078d2 (0xfcd3 bytes) 2024-09-16T20:48:31.548 INFO:teuthology.orchestra.run.smithi139.stdout:1084 read 0x1409800 thru 0x14155e7 (0xbde8 bytes) 2024-09-16T20:48:31.548 INFO:teuthology.orchestra.run.smithi139.stdout:1085 read 0xf27200 thru 0xf29518 (0x2319 bytes) 2024-09-16T20:48:31.549 INFO:teuthology.orchestra.run.smithi139.stdout:1087 trunc from 0x68b4800 to 0x9903800 2024-09-16T20:48:31.564 INFO:teuthology.orchestra.run.smithi139.stdout:1088 read 0x1f2c800 thru 0x1f32604 (0x5e05 bytes) 2024-09-16T20:48:31.564 INFO:teuthology.orchestra.run.smithi139.stdout:1089 punch from 0x4299000 to 0x42a1800, (0x8800 bytes) 2024-09-16T20:48:31.564 INFO:teuthology.orchestra.run.smithi139.stdout:1090 read 0x6666e00 thru 0x667268b (0xb88c bytes) 2024-09-16T20:48:31.564 INFO:teuthology.orchestra.run.smithi139.stdout:1093 write 0x5075200 thru 0x5080861 (0xb662 bytes) 2024-09-16T20:48:31.564 INFO:teuthology.orchestra.run.smithi139.stdout:1094 read 0x6c40000 thru 0x6c4f218 (0xf219 bytes) 2024-09-16T20:48:31.577 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:31.576+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:31.586 INFO:teuthology.orchestra.run.smithi139.stdout:1100 write 0xaf2a600 thru 0xaf32b6e (0x856f bytes) 2024-09-16T20:48:31.586 INFO:teuthology.orchestra.run.smithi139.stdout:1101 read 0x678ca00 thru 0x679233e (0x593f bytes) 2024-09-16T20:48:31.586 INFO:teuthology.orchestra.run.smithi139.stdout:1103 write 0x4f69a00 thru 0x4f75782 (0xbd83 bytes) 2024-09-16T20:48:31.610 INFO:teuthology.orchestra.run.smithi139.stdout:1104 write 0xd8fee00 thru 0xd90c651 (0xd852 bytes) 2024-09-16T20:48:31.610 INFO:teuthology.orchestra.run.smithi139.stdout:1105 read 0x678fc00 thru 0x679eecd (0xf2ce bytes) 2024-09-16T20:48:31.615 INFO:teuthology.orchestra.run.smithi139.stdout:1106 trunc from 0xd90c800 to 0x94fda00 2024-09-16T20:48:31.639 INFO:teuthology.orchestra.run.smithi139.stdout:1108 read 0x1935c00 thru 0x1943714 (0xdb15 bytes) 2024-09-16T20:48:31.642 INFO:teuthology.orchestra.run.smithi139.stdout:1109 trunc from 0x94fda00 to 0x767d800 2024-09-16T20:48:31.646 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:31.644+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:31.650 INFO:teuthology.orchestra.run.smithi029.stderr:noscrub is set 2024-09-16T20:48:31.656 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:31.656 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:48:31.656 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:48:31.656 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:48:31.656 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:31.660 INFO:teuthology.orchestra.run.smithi139.stdout:1113 write 0x15a9a00 thru 0x15af823 (0x5e24 bytes) 2024-09-16T20:48:31.664 INFO:teuthology.orchestra.run.smithi139.stdout:1116 trunc from 0x767d800 to 0xad97400 2024-09-16T20:48:31.667 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:48:31.686 INFO:teuthology.orchestra.run.smithi139.stdout:1120 punch from 0x625fe00 to 0x626be00, (0xc000 bytes) 2024-09-16T20:48:31.703 INFO:teuthology.orchestra.run.smithi139.stdout:1121 write 0xc980800 thru 0xc9899ce (0x91cf bytes) 2024-09-16T20:48:31.703 INFO:teuthology.orchestra.run.smithi139.stdout:1123 read 0x61b5e00 thru 0x61b9992 (0x3b93 bytes) 2024-09-16T20:48:31.703 INFO:teuthology.orchestra.run.smithi139.stdout:1124 read 0x8dc6600 thru 0x8dcfce8 (0x96e9 bytes) 2024-09-16T20:48:31.709 INFO:teuthology.orchestra.run.smithi139.stdout:1127 trunc from 0xc989a00 to 0x77ad800 2024-09-16T20:48:31.737 INFO:teuthology.orchestra.run.smithi139.stdout:1130 trunc from 0x77ad800 to 0x2bcfa00 2024-09-16T20:48:31.746 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:31.745+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:31.782 INFO:teuthology.orchestra.run.smithi139.stdout:1131 write 0x98ed200 thru 0x98f46d7 (0x74d8 bytes) 2024-09-16T20:48:31.782 INFO:teuthology.orchestra.run.smithi139.stdout:1132 punch from 0x74d8200 to 0x74de400, (0x6200 bytes) 2024-09-16T20:48:31.782 INFO:teuthology.orchestra.run.smithi139.stdout:1134 punch from 0x1ac3200 to 0x1ac6e00, (0x3c00 bytes) 2024-09-16T20:48:31.782 INFO:teuthology.orchestra.run.smithi139.stdout:1135 read 0x28f1a00 thru 0x28fd4c4 (0xbac5 bytes) 2024-09-16T20:48:31.801 INFO:teuthology.orchestra.run.smithi139.stdout:1136 write 0xac2d600 thru 0xac3553c (0x7f3d bytes) 2024-09-16T20:48:31.802 INFO:teuthology.orchestra.run.smithi139.stdout:1138 trunc from 0xac35600 to 0x9751800 2024-09-16T20:48:31.825 INFO:teuthology.orchestra.run.smithi139.stdout:1139 punch from 0x14f4a00 to 0x14fea00, (0xa000 bytes) 2024-09-16T20:48:31.829 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:31.829 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:48:31.829 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:48:31.829 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:48:31.829 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.830 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2167 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 907264~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:31.830 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335615+0000", 2024-09-16T20:48:31.830 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 9.4919305909999991, 2024-09-16T20:48:31.830 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046952761000000003, 2024-09-16T20:48:31.830 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:31.830 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:31.830 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:31.830 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:31.830 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:31.830 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:31.830 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2167 2024-09-16T20:48:31.830 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.830 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:31.831 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.831 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:31.831 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335615+0000", 2024-09-16T20:48:31.831 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:31.831 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.831 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.831 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:31.831 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335615+0000", 2024-09-16T20:48:31.831 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:31.831 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.831 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.831 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:31.831 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335616+0000", 2024-09-16T20:48:31.832 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2300000000000001e-07 2024-09-16T20:48:31.832 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.832 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.832 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:31.832 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335618+0000", 2024-09-16T20:48:31.832 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3269999999999999e-06 2024-09-16T20:48:31.832 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.832 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.832 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:31.832 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335625+0000", 2024-09-16T20:48:31.832 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1930000000000004e-06 2024-09-16T20:48:31.832 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.832 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.832 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:31.833 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335630+0000", 2024-09-16T20:48:31.833 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2460000000000003e-06 2024-09-16T20:48:31.833 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.833 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.833 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:31.833 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.382377+0000", 2024-09-16T20:48:31.833 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046746256 2024-09-16T20:48:31.833 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.833 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.833 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:31.833 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.382474+0000", 2024-09-16T20:48:31.833 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.7388999999999999e-05 2024-09-16T20:48:31.833 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.833 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.834 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:31.834 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.382568+0000", 2024-09-16T20:48:31.834 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.3826999999999995e-05 2024-09-16T20:48:31.834 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:31.834 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:31.834 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:31.834 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.834 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.834 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2170 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 915456~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:31.834 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335637+0000", 2024-09-16T20:48:31.834 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 9.4919084419999997, 2024-09-16T20:48:31.834 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047182843000000002, 2024-09-16T20:48:31.834 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:31.835 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:31.835 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:31.835 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:31.835 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:31.835 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:31.835 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2170 2024-09-16T20:48:31.835 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.835 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:31.835 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.835 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:31.835 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335637+0000", 2024-09-16T20:48:31.835 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:31.835 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.835 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.836 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:31.836 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335637+0000", 2024-09-16T20:48:31.836 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:31.836 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.836 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.836 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:31.836 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335638+0000", 2024-09-16T20:48:31.836 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6699999999999999e-07 2024-09-16T20:48:31.836 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.836 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.836 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:31.836 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335639+0000", 2024-09-16T20:48:31.836 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8339999999999999e-06 2024-09-16T20:48:31.837 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.837 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.837 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:31.837 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335646+0000", 2024-09-16T20:48:31.837 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6599999999999998e-06 2024-09-16T20:48:31.837 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.837 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.837 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:31.837 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335651+0000", 2024-09-16T20:48:31.837 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0289999999999996e-06 2024-09-16T20:48:31.837 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.837 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.837 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:31.838 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.382616+0000", 2024-09-16T20:48:31.838 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046965066 2024-09-16T20:48:31.838 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.838 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.838 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:31.838 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.382708+0000", 2024-09-16T20:48:31.838 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2177999999999997e-05 2024-09-16T20:48:31.838 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.838 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.838 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:31.838 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.382820+0000", 2024-09-16T20:48:31.838 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000111709 2024-09-16T20:48:31.838 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:31.838 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:31.839 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:31.839 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.839 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.839 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2172 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 923648~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:31.839 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335658+0000", 2024-09-16T20:48:31.839 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 9.491887535, 2024-09-16T20:48:31.839 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047394824000000002, 2024-09-16T20:48:31.839 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:31.839 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:31.839 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:31.839 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:31.839 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:31.839 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:31.839 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2172 2024-09-16T20:48:31.840 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.840 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:31.840 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.840 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:31.840 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335658+0000", 2024-09-16T20:48:31.840 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:31.840 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.840 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.840 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:31.840 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335658+0000", 2024-09-16T20:48:31.840 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:31.840 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.840 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.841 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:31.841 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335658+0000", 2024-09-16T20:48:31.841 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8599999999999999e-07 2024-09-16T20:48:31.841 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.841 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.841 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:31.841 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335660+0000", 2024-09-16T20:48:31.841 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.95e-06 2024-09-16T20:48:31.841 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.841 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.841 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:31.841 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335667+0000", 2024-09-16T20:48:31.841 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.815e-06 2024-09-16T20:48:31.841 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.841 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.842 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:31.842 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335672+0000", 2024-09-16T20:48:31.842 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8040000000000001e-06 2024-09-16T20:48:31.842 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.842 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.842 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:31.842 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.382864+0000", 2024-09-16T20:48:31.842 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047192125000000001 2024-09-16T20:48:31.842 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.842 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.842 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:31.842 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.382950+0000", 2024-09-16T20:48:31.842 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5594999999999996e-05 2024-09-16T20:48:31.842 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.843 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.843 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:31.843 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383053+0000", 2024-09-16T20:48:31.843 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000103149 2024-09-16T20:48:31.843 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:31.843 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:31.843 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:31.843 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.843 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.843 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2174 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 931840~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:31.843 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335679+0000", 2024-09-16T20:48:31.843 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 9.4918666619999996, 2024-09-16T20:48:31.843 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047619511000000003, 2024-09-16T20:48:31.844 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:31.844 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:31.844 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:31.844 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:31.844 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:31.844 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:31.844 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2174 2024-09-16T20:48:31.844 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.844 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:31.844 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.844 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:31.844 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335679+0000", 2024-09-16T20:48:31.844 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:31.844 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.845 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.845 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:31.845 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335679+0000", 2024-09-16T20:48:31.845 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:31.845 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.845 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.845 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:31.845 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335679+0000", 2024-09-16T20:48:31.845 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7800000000000002e-07 2024-09-16T20:48:31.845 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.845 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.845 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:31.845 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335682+0000", 2024-09-16T20:48:31.845 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9490000000000001e-06 2024-09-16T20:48:31.846 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.846 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.846 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:31.846 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335689+0000", 2024-09-16T20:48:31.846 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8220000000000001e-06 2024-09-16T20:48:31.846 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.846 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.846 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:31.846 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335695+0000", 2024-09-16T20:48:31.846 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0569999999999998e-06 2024-09-16T20:48:31.846 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.846 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.847 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:31.847 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383103+0000", 2024-09-16T20:48:31.847 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047408219000000001 2024-09-16T20:48:31.847 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.847 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.847 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:31.847 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383189+0000", 2024-09-16T20:48:31.847 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6007000000000005e-05 2024-09-16T20:48:31.847 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.847 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.847 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:31.848 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383298+0000", 2024-09-16T20:48:31.848 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000109079 2024-09-16T20:48:31.848 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:31.848 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:31.848 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:31.848 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.848 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.848 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2176 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 940032~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:31.848 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335702+0000", 2024-09-16T20:48:31.848 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 9.4918434339999997, 2024-09-16T20:48:31.848 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047804450999999998, 2024-09-16T20:48:31.848 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:31.849 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:31.849 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:31.849 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:31.849 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:31.849 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:31.849 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2176 2024-09-16T20:48:31.849 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.849 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:31.849 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.850 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:31.850 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335702+0000", 2024-09-16T20:48:31.850 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:31.850 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.850 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.850 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:31.850 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335702+0000", 2024-09-16T20:48:31.850 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:31.850 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.850 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.850 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:31.851 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335703+0000", 2024-09-16T20:48:31.851 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.34e-07 2024-09-16T20:48:31.851 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.851 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.851 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:31.851 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335704+0000", 2024-09-16T20:48:31.851 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.917e-06 2024-09-16T20:48:31.851 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.851 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.851 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:31.851 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335711+0000", 2024-09-16T20:48:31.852 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4400000000000002e-06 2024-09-16T20:48:31.852 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.852 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.852 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:31.852 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335717+0000", 2024-09-16T20:48:31.852 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6679999999999998e-06 2024-09-16T20:48:31.852 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.852 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.852 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:31.852 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383335+0000", 2024-09-16T20:48:31.852 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047618754999999999 2024-09-16T20:48:31.852 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.852 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.853 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:31.853 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383405+0000", 2024-09-16T20:48:31.853 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9286e-05 2024-09-16T20:48:31.853 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.853 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.853 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:31.853 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383507+0000", 2024-09-16T20:48:31.853 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101951 2024-09-16T20:48:31.853 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:31.853 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:31.854 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:31.854 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.854 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.854 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2179 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 948224~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:31.854 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335723+0000", 2024-09-16T20:48:31.854 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 9.4918223400000006, 2024-09-16T20:48:31.854 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048007181000000003, 2024-09-16T20:48:31.854 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:31.854 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:31.854 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:31.854 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:31.854 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:31.854 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:31.854 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2179 2024-09-16T20:48:31.855 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.855 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:31.855 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.855 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:31.855 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335723+0000", 2024-09-16T20:48:31.855 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:31.855 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.855 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.855 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:31.855 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335723+0000", 2024-09-16T20:48:31.855 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:31.855 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.855 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.855 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:31.856 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335724+0000", 2024-09-16T20:48:31.856 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.58e-07 2024-09-16T20:48:31.856 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.856 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.856 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:31.856 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335726+0000", 2024-09-16T20:48:31.856 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9120000000000001e-06 2024-09-16T20:48:31.856 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.856 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.856 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:31.856 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335732+0000", 2024-09-16T20:48:31.856 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6080000000000001e-06 2024-09-16T20:48:31.856 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.857 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.857 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:31.857 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335737+0000", 2024-09-16T20:48:31.857 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.22e-06 2024-09-16T20:48:31.857 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.857 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.857 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:31.857 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383556+0000", 2024-09-16T20:48:31.857 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047819002999999999 2024-09-16T20:48:31.857 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.857 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.858 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:31.858 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383635+0000", 2024-09-16T20:48:31.858 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9060999999999994e-05 2024-09-16T20:48:31.858 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.858 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.858 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:31.858 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383730+0000", 2024-09-16T20:48:31.858 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4919000000000004e-05 2024-09-16T20:48:31.858 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:31.858 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:31.858 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:31.858 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.858 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.859 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2180 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 956416~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:31.859 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335744+0000", 2024-09-16T20:48:31.859 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 9.4918018919999998, 2024-09-16T20:48:31.859 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048255147999999998, 2024-09-16T20:48:31.859 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:31.859 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:31.859 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:31.859 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:31.859 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:31.859 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:31.859 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2180 2024-09-16T20:48:31.859 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.859 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:31.860 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.860 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:31.860 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335744+0000", 2024-09-16T20:48:31.860 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:31.860 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.860 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.860 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:31.860 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335744+0000", 2024-09-16T20:48:31.860 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:31.860 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.860 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.860 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:31.860 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335744+0000", 2024-09-16T20:48:31.861 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9000000000000002e-07 2024-09-16T20:48:31.861 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.861 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.861 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:31.861 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335746+0000", 2024-09-16T20:48:31.861 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7570000000000001e-06 2024-09-16T20:48:31.861 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.861 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.861 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:31.861 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335752+0000", 2024-09-16T20:48:31.861 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0079999999999999e-06 2024-09-16T20:48:31.861 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.861 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.861 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:31.862 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335757+0000", 2024-09-16T20:48:31.862 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1649999999999997e-06 2024-09-16T20:48:31.862 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.862 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.862 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:31.862 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383803+0000", 2024-09-16T20:48:31.862 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048046091999999999 2024-09-16T20:48:31.862 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.862 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.862 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:31.862 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383889+0000", 2024-09-16T20:48:31.862 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6048e-05 2024-09-16T20:48:31.862 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.862 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.863 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:31.863 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383999+0000", 2024-09-16T20:48:31.863 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000109688 2024-09-16T20:48:31.863 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:31.863 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:31.863 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:31.863 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.863 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.863 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2182 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 964608~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:31.863 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335773+0000", 2024-09-16T20:48:31.863 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 9.4917720850000009, 2024-09-16T20:48:31.863 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048467116999999997, 2024-09-16T20:48:31.863 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:31.863 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:31.864 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:31.864 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:31.864 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:31.864 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:31.864 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2182 2024-09-16T20:48:31.864 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.864 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:31.864 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.864 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:31.864 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335773+0000", 2024-09-16T20:48:31.864 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:31.864 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.864 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.865 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:31.865 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335773+0000", 2024-09-16T20:48:31.865 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:31.865 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.865 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.865 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:31.865 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335774+0000", 2024-09-16T20:48:31.865 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3799999999999997e-07 2024-09-16T20:48:31.865 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.865 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.865 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:31.865 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335776+0000", 2024-09-16T20:48:31.865 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.227e-06 2024-09-16T20:48:31.865 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.866 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.866 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:31.866 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335790+0000", 2024-09-16T20:48:31.866 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4119e-05 2024-09-16T20:48:31.866 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.866 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.866 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:31.866 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335796+0000", 2024-09-16T20:48:31.866 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.694e-06 2024-09-16T20:48:31.866 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.866 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.866 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:31.866 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384058+0000", 2024-09-16T20:48:31.866 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048261983000000001 2024-09-16T20:48:31.867 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.867 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.867 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:31.867 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384148+0000", 2024-09-16T20:48:31.867 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9501999999999998e-05 2024-09-16T20:48:31.867 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.867 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.867 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:31.867 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384241+0000", 2024-09-16T20:48:31.867 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.3053999999999994e-05 2024-09-16T20:48:31.867 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:31.867 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:31.867 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:31.868 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.868 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.868 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2185 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 972800~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:31.868 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335804+0000", 2024-09-16T20:48:31.868 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 9.4917414349999998, 2024-09-16T20:48:31.868 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048570818000000002, 2024-09-16T20:48:31.868 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:31.868 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:31.868 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:31.868 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:31.869 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:31.869 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:31.869 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2185 2024-09-16T20:48:31.869 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.869 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:31.869 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.869 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:31.869 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335804+0000", 2024-09-16T20:48:31.869 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:31.869 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.869 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.869 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:31.869 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335804+0000", 2024-09-16T20:48:31.869 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:31.870 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.870 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.870 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:31.870 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335805+0000", 2024-09-16T20:48:31.870 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8299999999999997e-07 2024-09-16T20:48:31.870 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.870 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.870 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:31.870 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335813+0000", 2024-09-16T20:48:31.870 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6309999999999992e-06 2024-09-16T20:48:31.870 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.870 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.870 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:31.870 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335821+0000", 2024-09-16T20:48:31.871 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0169999999999996e-06 2024-09-16T20:48:31.871 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.871 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.871 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:31.871 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335827+0000", 2024-09-16T20:48:31.871 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0100000000000001e-06 2024-09-16T20:48:31.871 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.871 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.871 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:31.871 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384273+0000", 2024-09-16T20:48:31.871 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048445333 2024-09-16T20:48:31.871 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.871 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.872 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:31.872 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384315+0000", 2024-09-16T20:48:31.872 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2472999999999998e-05 2024-09-16T20:48:31.872 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.872 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.872 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:31.872 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384375+0000", 2024-09-16T20:48:31.872 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9871000000000003e-05 2024-09-16T20:48:31.872 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:31.872 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:31.872 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:31.872 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.872 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.873 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2187 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 980992~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:31.873 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335835+0000", 2024-09-16T20:48:31.873 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 9.4917103580000006, 2024-09-16T20:48:31.873 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048684927000000003, 2024-09-16T20:48:31.873 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:31.873 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:31.873 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:31.873 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:31.873 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:31.873 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:31.874 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2187 2024-09-16T20:48:31.874 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.874 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:31.874 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.874 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:31.874 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335835+0000", 2024-09-16T20:48:31.874 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:31.874 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.874 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.874 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:31.875 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335835+0000", 2024-09-16T20:48:31.875 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:31.875 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.875 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.875 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:31.875 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335836+0000", 2024-09-16T20:48:31.875 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3599999999999999e-07 2024-09-16T20:48:31.875 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.875 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.875 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:31.875 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335838+0000", 2024-09-16T20:48:31.875 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.996e-06 2024-09-16T20:48:31.875 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.876 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.876 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:31.876 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335844+0000", 2024-09-16T20:48:31.876 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7889999999999997e-06 2024-09-16T20:48:31.876 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.876 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.876 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:31.876 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335849+0000", 2024-09-16T20:48:31.876 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7400000000000004e-06 2024-09-16T20:48:31.876 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.876 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.876 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:31.876 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384411+0000", 2024-09-16T20:48:31.877 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048562069999999999 2024-09-16T20:48:31.877 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.877 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.877 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:31.877 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384460+0000", 2024-09-16T20:48:31.877 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9125999999999997e-05 2024-09-16T20:48:31.877 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.877 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.877 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:31.877 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384520+0000", 2024-09-16T20:48:31.877 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9769999999999999e-05 2024-09-16T20:48:31.877 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:31.877 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:31.877 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:31.878 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.878 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.878 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2189 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 989184~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:31.878 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335856+0000", 2024-09-16T20:48:31.878 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 9.4916891129999996, 2024-09-16T20:48:31.878 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048856525999999997, 2024-09-16T20:48:31.878 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:31.878 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:31.878 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:31.878 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:31.878 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:31.878 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:31.878 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2189 2024-09-16T20:48:31.879 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.879 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:31.879 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.879 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:31.879 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335856+0000", 2024-09-16T20:48:31.879 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:31.879 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.879 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.879 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:31.879 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335856+0000", 2024-09-16T20:48:31.879 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:31.879 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.879 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.879 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:31.880 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335857+0000", 2024-09-16T20:48:31.880 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6800000000000001e-07 2024-09-16T20:48:31.880 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.880 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.880 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:31.880 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335859+0000", 2024-09-16T20:48:31.880 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.013e-06 2024-09-16T20:48:31.880 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.880 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.880 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:31.880 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335866+0000", 2024-09-16T20:48:31.880 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8750000000000002e-06 2024-09-16T20:48:31.880 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.881 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.881 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:31.881 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335871+0000", 2024-09-16T20:48:31.881 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1719999999999998e-06 2024-09-16T20:48:31.881 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.881 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.881 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:31.881 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384550+0000", 2024-09-16T20:48:31.881 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048679476999999999 2024-09-16T20:48:31.881 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.881 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.881 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:31.881 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384611+0000", 2024-09-16T20:48:31.881 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0897000000000003e-05 2024-09-16T20:48:31.882 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.882 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.882 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:31.882 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384713+0000", 2024-09-16T20:48:31.882 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101724 2024-09-16T20:48:31.882 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:31.882 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:31.882 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:31.882 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.882 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.882 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2192 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 997376~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:31.882 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335878+0000", 2024-09-16T20:48:31.882 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 9.4916674279999995, 2024-09-16T20:48:31.883 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049028769999999999, 2024-09-16T20:48:31.883 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:31.883 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:31.883 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:31.883 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:31.883 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:31.883 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:31.883 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2192 2024-09-16T20:48:31.883 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.883 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:31.883 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.883 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:31.883 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335878+0000", 2024-09-16T20:48:31.883 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:31.884 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.884 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.884 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:31.884 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335878+0000", 2024-09-16T20:48:31.884 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:31.884 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.884 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.884 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:31.884 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335879+0000", 2024-09-16T20:48:31.884 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6899999999999998e-07 2024-09-16T20:48:31.884 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.884 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.884 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:31.885 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335881+0000", 2024-09-16T20:48:31.885 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.996e-06 2024-09-16T20:48:31.885 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.885 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.885 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:31.885 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335887+0000", 2024-09-16T20:48:31.885 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6370000000000002e-06 2024-09-16T20:48:31.885 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.885 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.885 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:31.885 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335892+0000", 2024-09-16T20:48:31.885 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2120000000000002e-06 2024-09-16T20:48:31.885 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.885 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.886 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:31.886 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384747+0000", 2024-09-16T20:48:31.886 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048854923000000001 2024-09-16T20:48:31.886 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.886 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.886 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:31.886 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384830+0000", 2024-09-16T20:48:31.886 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2359000000000002e-05 2024-09-16T20:48:31.886 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.886 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.886 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:31.886 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384907+0000", 2024-09-16T20:48:31.886 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7174000000000003e-05 2024-09-16T20:48:31.886 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:31.887 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:31.887 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:31.887 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.887 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.887 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2194 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1005568~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:31.887 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335899+0000", 2024-09-16T20:48:31.887 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 9.4916465070000005, 2024-09-16T20:48:31.887 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049225393999999999, 2024-09-16T20:48:31.887 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:31.887 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:31.887 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:31.887 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:31.887 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:31.887 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:31.888 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2194 2024-09-16T20:48:31.888 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.888 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:31.888 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.888 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:31.888 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335899+0000", 2024-09-16T20:48:31.888 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:31.888 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.888 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.888 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:31.888 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335899+0000", 2024-09-16T20:48:31.888 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:31.888 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.888 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.889 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:31.889 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335899+0000", 2024-09-16T20:48:31.889 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8200000000000001e-07 2024-09-16T20:48:31.889 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.889 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.889 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:31.889 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335901+0000", 2024-09-16T20:48:31.889 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9e-06 2024-09-16T20:48:31.889 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.889 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.889 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:31.889 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335908+0000", 2024-09-16T20:48:31.889 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3559999999999998e-06 2024-09-16T20:48:31.890 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.890 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.890 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:31.890 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335914+0000", 2024-09-16T20:48:31.890 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4729999999999997e-06 2024-09-16T20:48:31.890 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.890 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.890 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:31.890 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384979+0000", 2024-09-16T20:48:31.890 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049064545000000001 2024-09-16T20:48:31.890 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.890 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.890 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:31.891 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385060+0000", 2024-09-16T20:48:31.891 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0804999999999996e-05 2024-09-16T20:48:31.891 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.891 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.891 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:31.891 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385124+0000", 2024-09-16T20:48:31.891 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4932999999999996e-05 2024-09-16T20:48:31.891 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:31.891 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:31.891 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:31.891 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.891 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.891 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2197 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1013760~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:31.891 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335921+0000", 2024-09-16T20:48:31.892 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 9.4916242260000008, 2024-09-16T20:48:31.892 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049342184999999997, 2024-09-16T20:48:31.892 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:31.892 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:31.892 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:31.892 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:31.892 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:31.892 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:31.892 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2197 2024-09-16T20:48:31.892 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.892 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:31.892 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.892 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:31.893 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335921+0000", 2024-09-16T20:48:31.893 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:31.893 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.893 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.893 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:31.893 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335921+0000", 2024-09-16T20:48:31.893 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:31.893 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.893 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.893 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:31.893 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335922+0000", 2024-09-16T20:48:31.893 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.32e-07 2024-09-16T20:48:31.893 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.893 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.894 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:31.894 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335924+0000", 2024-09-16T20:48:31.894 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.995e-06 2024-09-16T20:48:31.894 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.894 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.894 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:31.894 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335930+0000", 2024-09-16T20:48:31.894 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5139999999999998e-06 2024-09-16T20:48:31.894 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.894 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.894 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:31.894 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335936+0000", 2024-09-16T20:48:31.894 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6160000000000001e-06 2024-09-16T20:48:31.895 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.895 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.895 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:31.895 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385142+0000", 2024-09-16T20:48:31.895 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049206393000000001 2024-09-16T20:48:31.895 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.895 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.895 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:31.895 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385184+0000", 2024-09-16T20:48:31.895 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1928999999999998e-05 2024-09-16T20:48:31.895 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.895 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.895 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:31.895 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385264+0000", 2024-09-16T20:48:31.896 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9306000000000003e-05 2024-09-16T20:48:31.896 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:31.896 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:31.896 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:31.896 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.896 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.896 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2198 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1021952~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:31.896 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335942+0000", 2024-09-16T20:48:31.896 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 9.4916031660000009, 2024-09-16T20:48:31.896 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049562483999999997, 2024-09-16T20:48:31.896 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:31.896 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:31.896 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:31.896 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:31.897 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:31.897 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:31.897 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2198 2024-09-16T20:48:31.897 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.897 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:31.897 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.897 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:31.897 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335942+0000", 2024-09-16T20:48:31.897 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:31.897 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.897 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.897 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:31.897 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335942+0000", 2024-09-16T20:48:31.898 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:31.898 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.898 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.898 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:31.898 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335943+0000", 2024-09-16T20:48:31.898 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7099999999999997e-07 2024-09-16T20:48:31.898 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.898 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.898 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:31.898 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335945+0000", 2024-09-16T20:48:31.898 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3499999999999999e-06 2024-09-16T20:48:31.898 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.898 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.898 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:31.899 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335952+0000", 2024-09-16T20:48:31.899 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4309999999999999e-06 2024-09-16T20:48:31.899 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.899 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.899 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:31.899 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335957+0000", 2024-09-16T20:48:31.899 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0869999999999997e-06 2024-09-16T20:48:31.899 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.899 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.899 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:31.899 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385312+0000", 2024-09-16T20:48:31.899 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049355121000000002 2024-09-16T20:48:31.899 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.899 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.900 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:31.900 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385405+0000", 2024-09-16T20:48:31.900 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.3344000000000006e-05 2024-09-16T20:48:31.900 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.900 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.900 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:31.900 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385505+0000", 2024-09-16T20:48:31.900 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.9779999999999997e-05 2024-09-16T20:48:31.900 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:31.900 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:31.900 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:31.900 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.900 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.900 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2201 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1030144~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:31.901 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335976+0000", 2024-09-16T20:48:31.901 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 9.4915697239999997, 2024-09-16T20:48:31.901 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049740077000000001, 2024-09-16T20:48:31.901 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:31.901 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:31.901 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:31.901 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:31.901 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:31.901 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:31.901 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2201 2024-09-16T20:48:31.901 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.901 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:31.901 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.901 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:31.902 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335976+0000", 2024-09-16T20:48:31.902 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:31.902 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.902 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.902 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:31.902 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335976+0000", 2024-09-16T20:48:31.902 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:31.902 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.902 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.902 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:31.902 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335976+0000", 2024-09-16T20:48:31.902 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8599999999999998e-07 2024-09-16T20:48:31.902 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.902 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.903 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:31.903 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335978+0000", 2024-09-16T20:48:31.903 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1500000000000002e-06 2024-09-16T20:48:31.903 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.903 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.903 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:31.903 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335986+0000", 2024-09-16T20:48:31.903 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3989999999999998e-06 2024-09-16T20:48:31.903 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.903 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.903 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:31.903 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335992+0000", 2024-09-16T20:48:31.903 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.682e-06 2024-09-16T20:48:31.904 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.904 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.904 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:31.904 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385537+0000", 2024-09-16T20:48:31.904 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049545828 2024-09-16T20:48:31.904 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.904 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.904 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:31.904 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385619+0000", 2024-09-16T20:48:31.904 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1720999999999999e-05 2024-09-16T20:48:31.904 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.904 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.904 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:31.905 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385716+0000", 2024-09-16T20:48:31.905 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.6811000000000005e-05 2024-09-16T20:48:31.905 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:31.905 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:31.905 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:31.905 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.905 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.905 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2203 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1038336~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:31.905 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.336000+0000", 2024-09-16T20:48:31.905 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 9.4915459379999998, 2024-09-16T20:48:31.905 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049949888999999997, 2024-09-16T20:48:31.905 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:31.905 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:31.906 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:31.906 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:31.906 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:31.906 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:31.906 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2203 2024-09-16T20:48:31.906 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.906 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:31.906 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.906 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:31.906 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336000+0000", 2024-09-16T20:48:31.906 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:31.906 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.906 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.906 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:31.907 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336000+0000", 2024-09-16T20:48:31.907 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:31.907 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.907 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.907 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:31.907 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336000+0000", 2024-09-16T20:48:31.907 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.58e-07 2024-09-16T20:48:31.907 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.907 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.907 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:31.907 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336002+0000", 2024-09-16T20:48:31.907 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0040000000000002e-06 2024-09-16T20:48:31.907 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.907 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.908 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:31.908 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336009+0000", 2024-09-16T20:48:31.908 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9890000000000003e-06 2024-09-16T20:48:31.908 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.908 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.908 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:31.908 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336014+0000", 2024-09-16T20:48:31.908 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4e-06 2024-09-16T20:48:31.908 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.908 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.908 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:31.908 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385759+0000", 2024-09-16T20:48:31.908 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049744151 2024-09-16T20:48:31.909 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.909 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.909 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:31.909 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385847+0000", 2024-09-16T20:48:31.909 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8189000000000006e-05 2024-09-16T20:48:31.909 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.909 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.909 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:31.909 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385950+0000", 2024-09-16T20:48:31.909 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102698 2024-09-16T20:48:31.909 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:31.909 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:31.909 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:31.909 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.910 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.910 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2206 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1046528~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:31.910 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:48:31.910 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 9.4915237769999994, 2024-09-16T20:48:31.910 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050147714000000003, 2024-09-16T20:48:31.910 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:31.910 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:31.910 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:31.910 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:31.910 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:31.910 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:31.910 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2206 2024-09-16T20:48:31.910 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.910 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:31.911 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.911 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:31.911 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:48:31.911 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:31.911 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.911 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.911 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:31.911 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:48:31.911 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:31.911 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.911 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.911 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:31.911 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:48:31.911 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8599999999999999e-07 2024-09-16T20:48:31.912 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.912 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.912 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:31.912 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336024+0000", 2024-09-16T20:48:31.912 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.934e-06 2024-09-16T20:48:31.912 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.912 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.912 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:31.912 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336037+0000", 2024-09-16T20:48:31.912 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2496e-05 2024-09-16T20:48:31.912 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.912 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.912 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:31.913 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336042+0000", 2024-09-16T20:48:31.913 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4809999999999997e-06 2024-09-16T20:48:31.913 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.913 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.913 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:31.913 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385980+0000", 2024-09-16T20:48:31.913 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049938376999999999 2024-09-16T20:48:31.913 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.913 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.913 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:31.913 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.386069+0000", 2024-09-16T20:48:31.913 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8368999999999994e-05 2024-09-16T20:48:31.913 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.913 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.914 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:31.914 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.386170+0000", 2024-09-16T20:48:31.914 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000100671 2024-09-16T20:48:31.914 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:31.914 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:31.914 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:31.914 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.914 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.914 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5439 3.a 3:5789f5c3:::rbd_data.110ed9ca540.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 1181696~60416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:31.914 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:48:31.914 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 8.9628351970000004, 2024-09-16T20:48:31.914 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10102045599999999, 2024-09-16T20:48:31.914 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:31.914 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:31.915 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:31.915 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:31.915 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:31.915 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:31.915 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5439 2024-09-16T20:48:31.915 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.915 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:31.915 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.915 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:31.915 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:48:31.915 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:31.915 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.915 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.915 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:31.916 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:48:31.916 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:31.916 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.916 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.916 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:31.916 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864711+0000", 2024-09-16T20:48:31.916 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7899999999999997e-07 2024-09-16T20:48:31.916 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.916 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.916 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:31.917 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864760+0000", 2024-09-16T20:48:31.917 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8803e-05 2024-09-16T20:48:31.917 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.917 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.917 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:31.917 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864769+0000", 2024-09-16T20:48:31.917 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0999999999999993e-06 2024-09-16T20:48:31.917 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.917 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.917 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:31.917 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864774+0000", 2024-09-16T20:48:31.917 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8919999999999999e-06 2024-09-16T20:48:31.917 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.918 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.918 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:31.918 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864815+0000", 2024-09-16T20:48:31.918 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1307000000000003e-05 2024-09-16T20:48:31.918 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.918 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.918 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:31.918 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864893+0000", 2024-09-16T20:48:31.918 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7581000000000001e-05 2024-09-16T20:48:31.918 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.918 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.918 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 1", 2024-09-16T20:48:31.918 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864996+0000", 2024-09-16T20:48:31.918 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102725 2024-09-16T20:48:31.919 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.919 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.919 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:31.919 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965442+0000", 2024-09-16T20:48:31.919 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100446012 2024-09-16T20:48:31.919 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.919 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.919 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:31.919 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965618+0000", 2024-09-16T20:48:31.919 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00017653100000000001 2024-09-16T20:48:31.919 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.919 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.919 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:31.920 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965682+0000", 2024-09-16T20:48:31.920 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3905999999999994e-05 2024-09-16T20:48:31.920 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.920 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.920 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:31.920 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965731+0000", 2024-09-16T20:48:31.920 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8819999999999997e-05 2024-09-16T20:48:31.920 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:31.920 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:31.920 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:31.920 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.920 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.920 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4600.0:5439 3.a e37/23 ondisk, result = 0)", 2024-09-16T20:48:31.920 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:48:31.921 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 8.9618390489999999, 2024-09-16T20:48:31.921 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099742634999999996, 2024-09-16T20:48:31.921 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:31.921 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:31.921 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:31.921 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:31.921 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.921 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:31.921 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:48:31.921 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:31.922 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.922 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.922 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:31.922 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:48:31.922 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:31.922 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.922 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.922 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:31.922 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:48:31.922 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0900000000000001e-07 2024-09-16T20:48:31.922 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.922 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.922 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:31.922 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865711+0000", 2024-09-16T20:48:31.923 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3739999999999998e-06 2024-09-16T20:48:31.923 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.923 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.923 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:31.923 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965343+0000", 2024-09-16T20:48:31.923 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099632526999999999 2024-09-16T20:48:31.923 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.923 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.923 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:31.923 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965357+0000", 2024-09-16T20:48:31.923 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3713999999999999e-05 2024-09-16T20:48:31.923 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.923 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.923 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:31.924 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965426+0000", 2024-09-16T20:48:31.924 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8745999999999995e-05 2024-09-16T20:48:31.924 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.924 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.924 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:31.924 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965434+0000", 2024-09-16T20:48:31.924 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6910000000000003e-06 2024-09-16T20:48:31.924 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:31.924 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:31.924 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:31.924 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965449+0000", 2024-09-16T20:48:31.924 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4874e-05 2024-09-16T20:48:31.924 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:31.925 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:31.925 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:31.925 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:31.925 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:31.925 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:31.927 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:48:31.928 INFO:teuthology.orchestra.run.smithi139.stdout:1140 write 0xd556a00 thru 0xd55ad19 (0x431a bytes) 2024-09-16T20:48:31.928 INFO:teuthology.orchestra.run.smithi139.stdout:1143 write 0xdcb1600 thru 0xdcb525a (0x3c5b bytes) 2024-09-16T20:48:31.928 INFO:teuthology.orchestra.run.smithi139.stdout:1145 read 0x62f0c00 thru 0x62f8d48 (0x8149 bytes) 2024-09-16T20:48:31.928 INFO:teuthology.orchestra.run.smithi139.stdout:1146 write 0x181800 thru 0x1911c1 (0xf9c2 bytes) 2024-09-16T20:48:31.929 INFO:teuthology.orchestra.run.smithi139.stdout:1147 trunc from 0xdcb5400 to 0x8957000 2024-09-16T20:48:31.929 INFO:teuthology.orchestra.run.smithi139.stdout:1149 read 0x354da00 thru 0x354e6ce (0xccf bytes) 2024-09-16T20:48:31.929 INFO:teuthology.orchestra.run.smithi139.stdout:1151 read 0x47e0600 thru 0x47eb739 (0xb13a bytes) 2024-09-16T20:48:31.929 INFO:teuthology.orchestra.run.smithi139.stdout:1153 punch from 0x79f4400 to 0x7a00e00, (0xca00 bytes) 2024-09-16T20:48:31.929 INFO:teuthology.orchestra.run.smithi139.stdout:1154 write 0x84d0c00 thru 0x84d533d (0x473e bytes) 2024-09-16T20:48:31.929 INFO:teuthology.orchestra.run.smithi139.stdout:1157 trunc from 0x8957000 to 0x953e600 2024-09-16T20:48:31.929 INFO:teuthology.orchestra.run.smithi139.stdout:1160 punch from 0x936e600 to 0x9374e00, (0x6800 bytes) 2024-09-16T20:48:31.929 INFO:teuthology.orchestra.run.smithi139.stdout:1161 read 0x829d200 thru 0x82a6fa0 (0x9da1 bytes) 2024-09-16T20:48:31.929 INFO:teuthology.orchestra.run.smithi139.stdout:1163 read 0x8fc3e00 thru 0x8fc4fb9 (0x11ba bytes) 2024-09-16T20:48:31.929 INFO:teuthology.orchestra.run.smithi139.stdout:1166 write 0x452c00 thru 0x45cac9 (0x9eca bytes) 2024-09-16T20:48:31.929 INFO:teuthology.orchestra.run.smithi139.stdout:1167 trunc from 0x953e600 to 0x6d7a200 2024-09-16T20:48:31.930 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:31.924+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:31.943 INFO:teuthology.orchestra.run.smithi139.stdout:1169 trunc from 0x6d7a200 to 0xd97e200 2024-09-16T20:48:31.958 INFO:teuthology.orchestra.run.smithi139.stdout:1170 write 0x9a83c00 thru 0x9a87bbc (0x3fbd bytes) 2024-09-16T20:48:31.958 INFO:teuthology.orchestra.run.smithi139.stdout:1172 punch from 0x331e800 to 0x3329200, (0xaa00 bytes) 2024-09-16T20:48:31.966 INFO:teuthology.orchestra.run.smithi139.stdout:1173 trunc from 0xd97e200 to 0x6762200 2024-09-16T20:48:32.009 INFO:teuthology.orchestra.run.smithi139.stdout:1174 write 0xe29ba00 thru 0xe2a55fa (0x9bfb bytes) 2024-09-16T20:48:32.009 INFO:teuthology.orchestra.run.smithi139.stdout:1175 read 0x48a2200 thru 0x48aa180 (0x7f81 bytes) 2024-09-16T20:48:32.010 INFO:teuthology.orchestra.run.smithi139.stdout:1177 read 0x2f84200 thru 0x2f85c0d (0x1a0e bytes) 2024-09-16T20:48:32.026 INFO:teuthology.orchestra.run.smithi139.stdout:1179 trunc from 0xe2a5600 to 0x6b4c00 2024-09-16T20:48:32.027 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:32.026+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:32.046 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:32.046 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:48:32.046 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:48:32.046 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:32.055 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:48:32.071 INFO:teuthology.orchestra.run.smithi139.stdout:1181 write 0x84cd800 thru 0x84dbe89 (0xe68a bytes) 2024-09-16T20:48:32.071 INFO:teuthology.orchestra.run.smithi139.stdout:1183 read 0x3358000 thru 0x3358ef2 (0xef3 bytes) 2024-09-16T20:48:32.071 INFO:teuthology.orchestra.run.smithi139.stdout:1184 read 0x6004c00 thru 0x6010a2b (0xbe2c bytes) 2024-09-16T20:48:32.072 INFO:teuthology.orchestra.run.smithi139.stdout:1185 read 0x7056200 thru 0x7061a0d (0xb80e bytes) 2024-09-16T20:48:32.072 INFO:teuthology.orchestra.run.smithi139.stdout:1188 write 0x2aada00 thru 0x2aba4c0 (0xcac1 bytes) 2024-09-16T20:48:32.081 INFO:teuthology.orchestra.run.smithi139.stdout:1190 trunc from 0x84dc000 to 0x46ea00 2024-09-16T20:48:32.107 INFO:teuthology.orchestra.run.smithi139.stdout:1193 read 0x35f400 thru 0x36ec4d (0xf84e bytes) 2024-09-16T20:48:32.108 INFO:teuthology.orchestra.run.smithi139.stdout:1194 read 0x465000 thru 0x467c39 (0x2c3a bytes) 2024-09-16T20:48:32.108 INFO:teuthology.orchestra.run.smithi139.stdout:1195 read 0x22e000 thru 0x23a26b (0xc26c bytes) 2024-09-16T20:48:32.128 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:32.126+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:32.134 INFO:teuthology.orchestra.run.smithi139.stdout:1198 write 0x9186200 thru 0x9189c84 (0x3a85 bytes) 2024-09-16T20:48:32.156 INFO:teuthology.orchestra.run.smithi139.stdout:1201 write 0xcb92600 thru 0xcb96373 (0x3d74 bytes) 2024-09-16T20:48:32.172 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:32.172 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:48:32.172 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:48:32.172 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:48:32.172 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:32.174 INFO:teuthology.orchestra.run.smithi139.stdout:1203 write 0xcfc3e00 thru 0xcfcbca2 (0x7ea3 bytes) 2024-09-16T20:48:32.182 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:48:32.182 INFO:teuthology.orchestra.run.smithi139.stdout:1204 trunc from 0xcfcbe00 to 0x7134a00 2024-09-16T20:48:32.203 INFO:teuthology.orchestra.run.smithi139.stdout:1205 read 0x6bdae00 thru 0x6be0346 (0x5547 bytes) 2024-09-16T20:48:32.203 INFO:teuthology.orchestra.run.smithi139.stdout:1206 read 0x40c7000 thru 0x40cbbb3 (0x4bb4 bytes) 2024-09-16T20:48:32.203 INFO:teuthology.orchestra.run.smithi139.stdout:1207 read 0x1a1d800 thru 0x1a203f5 (0x2bf6 bytes) 2024-09-16T20:48:32.222 INFO:teuthology.orchestra.run.smithi139.stdout:1208 write 0xa0f2600 thru 0xa0f38f5 (0x12f6 bytes) 2024-09-16T20:48:32.222 INFO:teuthology.orchestra.run.smithi139.stdout:1209 write 0xad4200 thru 0xadd2ad (0x90ae bytes) 2024-09-16T20:48:32.222 INFO:teuthology.orchestra.run.smithi139.stdout:1210 write 0x411c800 thru 0x412112e (0x492f bytes) 2024-09-16T20:48:32.222 INFO:teuthology.orchestra.run.smithi139.stdout:1212 write 0x9abf400 thru 0x9ac12a2 (0x1ea3 bytes) 2024-09-16T20:48:32.222 INFO:teuthology.orchestra.run.smithi139.stdout:1213 read 0x8cedc00 thru 0x8cef27f (0x1680 bytes) 2024-09-16T20:48:32.229 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:32.227+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:32.247 INFO:teuthology.orchestra.run.smithi139.stdout:1216 write 0xdbc8400 thru 0xdbd51f8 (0xcdf9 bytes) 2024-09-16T20:48:32.248 INFO:teuthology.orchestra.run.smithi139.stdout:1218 read 0x2fbae00 thru 0x2fc3391 (0x8592 bytes) 2024-09-16T20:48:32.249 INFO:teuthology.orchestra.run.smithi139.stdout:1219 trunc from 0xdbd5200 to 0xcbbf200 2024-09-16T20:48:32.265 INFO:teuthology.orchestra.run.smithi139.stdout:1221 write 0x70e8200 thru 0x70efff5 (0x7df6 bytes) 2024-09-16T20:48:32.269 INFO:teuthology.orchestra.run.smithi139.stdout:1223 read 0x31c400 thru 0x322be9 (0x67ea bytes) 2024-09-16T20:48:32.278 INFO:teuthology.orchestra.run.smithi139.stdout:1225 trunc from 0xcbbf200 to 0x5cd2a00 2024-09-16T20:48:32.301 INFO:teuthology.orchestra.run.smithi139.stdout:1227 read 0x2cc6200 thru 0x2cd1854 (0xb655 bytes) 2024-09-16T20:48:32.301 INFO:teuthology.orchestra.run.smithi139.stdout:1230 read 0x34b1200 thru 0x34bac20 (0x9a21 bytes) 2024-09-16T20:48:32.301 INFO:teuthology.orchestra.run.smithi139.stdout:1231 read 0xa31e00 thru 0xa358ab (0x3aac bytes) 2024-09-16T20:48:32.301 INFO:teuthology.orchestra.run.smithi139.stdout:1232 write 0xc8d000 thru 0xc97f54 (0xaf55 bytes) 2024-09-16T20:48:32.301 INFO:teuthology.orchestra.run.smithi139.stdout:1235 read 0x692000 thru 0x696a6a (0x4a6b bytes) 2024-09-16T20:48:32.322 INFO:teuthology.orchestra.run.smithi139.stdout:1238 write 0xd26a000 thru 0xd2754c7 (0xb4c8 bytes) 2024-09-16T20:48:32.326 INFO:teuthology.orchestra.run.smithi139.stdout:1239 trunc from 0xd275600 to 0xbf53800 2024-09-16T20:48:32.329 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:32.328+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:32.341 INFO:teuthology.orchestra.run.smithi139.stdout:1241 clone 10 order 21 su 65536 sc 8 2024-09-16T20:48:32.344 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:32.344 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:48:32.344 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:48:32.344 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:48:32.344 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.344 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:48:32.344 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:48:32.345 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 39.087694439000003, 2024-09-16T20:48:32.345 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67502346999999996, 2024-09-16T20:48:32.345 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:32.345 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:32.345 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:32.345 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:32.345 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:48:32.345 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:48:32.345 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 1 2024-09-16T20:48:32.345 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.345 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:32.346 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.346 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:32.346 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:48:32.346 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.346 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.346 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.346 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:32.346 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:48:32.346 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.346 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.346 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.347 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:32.347 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254128+0000", 2024-09-16T20:48:32.347 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.463e-06 2024-09-16T20:48:32.347 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.347 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.347 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:32.347 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254135+0000", 2024-09-16T20:48:32.347 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1579999999999999e-06 2024-09-16T20:48:32.347 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.347 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.347 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:32.347 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254146+0000", 2024-09-16T20:48:32.348 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1341e-05 2024-09-16T20:48:32.348 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.348 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.348 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:32.348 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254155+0000", 2024-09-16T20:48:32.348 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.1549999999999996e-06 2024-09-16T20:48:32.348 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.348 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.348 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:32.348 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254221+0000", 2024-09-16T20:48:32.349 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5470999999999997e-05 2024-09-16T20:48:32.349 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.349 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.349 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:48:32.349 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254231+0000", 2024-09-16T20:48:32.349 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0592e-05 2024-09-16T20:48:32.349 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.349 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.349 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:32.349 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926711+0000", 2024-09-16T20:48:32.349 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67247946599999997 2024-09-16T20:48:32.349 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.350 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.350 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:32.350 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926910+0000", 2024-09-16T20:48:32.350 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019934 2024-09-16T20:48:32.350 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.350 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.350 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:48:32.350 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.927159+0000", 2024-09-16T20:48:32.350 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00024876599999999998 2024-09-16T20:48:32.350 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.350 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.350 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:32.351 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.928797+0000", 2024-09-16T20:48:32.351 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001638308 2024-09-16T20:48:32.351 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.351 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.351 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:32.351 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929026+0000", 2024-09-16T20:48:32.351 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00022876099999999999 2024-09-16T20:48:32.352 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.352 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.352 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:32.352 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929102+0000", 2024-09-16T20:48:32.352 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6184000000000001e-05 2024-09-16T20:48:32.352 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.352 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.352 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:32.352 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929150+0000", 2024-09-16T20:48:32.352 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7465e-05 2024-09-16T20:48:32.352 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.353 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:32.353 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.353 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.353 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.353 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4312 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4618240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:32.353 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745559+0000", 2024-09-16T20:48:32.353 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 12.596261303, 2024-09-16T20:48:32.353 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046756458000000001, 2024-09-16T20:48:32.353 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:32.353 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:32.353 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:32.354 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:32.354 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:48:32.354 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:32.354 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4312 2024-09-16T20:48:32.354 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.354 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:32.354 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.354 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:32.354 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745559+0000", 2024-09-16T20:48:32.354 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.354 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.354 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.355 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:32.355 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745559+0000", 2024-09-16T20:48:32.355 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.355 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.355 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.355 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:32.355 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745560+0000", 2024-09-16T20:48:32.355 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6899999999999998e-07 2024-09-16T20:48:32.355 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.355 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.355 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:32.356 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745569+0000", 2024-09-16T20:48:32.356 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8950000000000004e-06 2024-09-16T20:48:32.356 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.356 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.356 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:32.356 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745576+0000", 2024-09-16T20:48:32.356 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9949999999999999e-06 2024-09-16T20:48:32.356 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.356 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.356 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:32.356 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745596+0000", 2024-09-16T20:48:32.357 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0767999999999998e-05 2024-09-16T20:48:32.357 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.357 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.357 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:32.357 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792225+0000", 2024-09-16T20:48:32.357 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046628393999999997 2024-09-16T20:48:32.357 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.357 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.357 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:32.357 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792265+0000", 2024-09-16T20:48:32.358 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9912e-05 2024-09-16T20:48:32.358 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.358 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.358 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:32.358 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792316+0000", 2024-09-16T20:48:32.358 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1025e-05 2024-09-16T20:48:32.358 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.358 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:32.358 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.358 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.358 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.359 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4313 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4626432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:32.359 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745608+0000", 2024-09-16T20:48:32.359 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 12.596212734, 2024-09-16T20:48:32.359 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046829200000000001, 2024-09-16T20:48:32.359 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:32.359 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:32.359 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:32.359 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:32.360 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:48:32.360 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:32.360 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4313 2024-09-16T20:48:32.360 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.360 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:32.360 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.360 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:32.360 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745608+0000", 2024-09-16T20:48:32.360 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.360 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.360 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.360 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:32.361 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745608+0000", 2024-09-16T20:48:32.361 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.361 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.361 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.361 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:32.361 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745609+0000", 2024-09-16T20:48:32.361 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4700000000000001e-07 2024-09-16T20:48:32.365 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.365 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.365 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:32.365 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745611+0000", 2024-09-16T20:48:32.365 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8119999999999999e-06 2024-09-16T20:48:32.365 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.366 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.366 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:32.366 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745619+0000", 2024-09-16T20:48:32.366 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6520000000000006e-06 2024-09-16T20:48:32.366 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.366 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.366 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:32.366 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745625+0000", 2024-09-16T20:48:32.366 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6590000000000004e-06 2024-09-16T20:48:32.366 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.366 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.367 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:32.367 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792344+0000", 2024-09-16T20:48:32.367 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046719318000000003 2024-09-16T20:48:32.367 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.367 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.367 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:32.367 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792384+0000", 2024-09-16T20:48:32.367 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0389000000000002e-05 2024-09-16T20:48:32.367 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.367 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.367 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:32.368 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792437+0000", 2024-09-16T20:48:32.368 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2623000000000001e-05 2024-09-16T20:48:32.368 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.368 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:32.368 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.368 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.368 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.368 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4314 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4634624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:32.368 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745632+0000", 2024-09-16T20:48:32.368 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 12.596188867, 2024-09-16T20:48:32.368 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046928879999999999, 2024-09-16T20:48:32.368 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:32.369 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:32.369 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:32.369 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:32.369 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:48:32.369 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:32.369 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4314 2024-09-16T20:48:32.369 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.369 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:32.369 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.370 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:32.370 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745632+0000", 2024-09-16T20:48:32.370 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.370 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.370 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.370 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:32.370 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745632+0000", 2024-09-16T20:48:32.370 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.370 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.370 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.370 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:32.371 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745632+0000", 2024-09-16T20:48:32.371 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3700000000000001e-07 2024-09-16T20:48:32.371 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.371 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.371 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:32.371 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745634+0000", 2024-09-16T20:48:32.372 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9759999999999998e-06 2024-09-16T20:48:32.372 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.372 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.372 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:32.373 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745645+0000", 2024-09-16T20:48:32.373 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1097000000000001e-05 2024-09-16T20:48:32.373 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.373 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.373 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:32.373 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745651+0000", 2024-09-16T20:48:32.373 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5090000000000001e-06 2024-09-16T20:48:32.373 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.374 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.374 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:32.374 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792467+0000", 2024-09-16T20:48:32.374 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046816712000000003 2024-09-16T20:48:32.374 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.374 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.374 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:32.374 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792511+0000", 2024-09-16T20:48:32.375 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3256000000000001e-05 2024-09-16T20:48:32.375 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.375 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.375 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:32.375 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792561+0000", 2024-09-16T20:48:32.375 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9892999999999998e-05 2024-09-16T20:48:32.375 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.375 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:32.375 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.376 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.376 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.376 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4315 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4642816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:32.376 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745659+0000", 2024-09-16T20:48:32.376 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 12.596161744, 2024-09-16T20:48:32.376 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047065501000000003, 2024-09-16T20:48:32.376 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:32.377 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:32.377 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:32.377 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:32.377 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:48:32.377 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:32.377 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4315 2024-09-16T20:48:32.377 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.377 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:32.377 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.377 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:32.377 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745659+0000", 2024-09-16T20:48:32.378 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.378 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.378 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.378 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:32.378 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745659+0000", 2024-09-16T20:48:32.378 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.379 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.379 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.379 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:32.379 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745659+0000", 2024-09-16T20:48:32.379 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7e-07 2024-09-16T20:48:32.379 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.379 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.379 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:32.379 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745662+0000", 2024-09-16T20:48:32.379 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2189999999999998e-06 2024-09-16T20:48:32.379 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.380 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.380 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:32.380 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745668+0000", 2024-09-16T20:48:32.380 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8979999999999999e-06 2024-09-16T20:48:32.380 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.380 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.380 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:32.380 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745675+0000", 2024-09-16T20:48:32.380 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3579999999999999e-06 2024-09-16T20:48:32.380 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.380 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.380 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:32.381 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792593+0000", 2024-09-16T20:48:32.381 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046917888999999997 2024-09-16T20:48:32.381 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.381 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.381 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:32.381 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792632+0000", 2024-09-16T20:48:32.381 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9818999999999999e-05 2024-09-16T20:48:32.381 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.381 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.381 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:32.381 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792724+0000", 2024-09-16T20:48:32.382 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.1848000000000006e-05 2024-09-16T20:48:32.382 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.382 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:32.382 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.382 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.382 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.382 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4316 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4651008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:32.382 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745682+0000", 2024-09-16T20:48:32.382 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 12.596138465999999, 2024-09-16T20:48:32.382 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047270334999999997, 2024-09-16T20:48:32.382 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:32.382 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:32.383 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:32.383 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:32.383 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:48:32.383 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:32.383 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4316 2024-09-16T20:48:32.383 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.383 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:32.383 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.383 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:32.383 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745682+0000", 2024-09-16T20:48:32.383 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.384 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.384 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.384 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:32.384 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745682+0000", 2024-09-16T20:48:32.384 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.384 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.384 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.384 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:32.384 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745683+0000", 2024-09-16T20:48:32.384 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.27e-07 2024-09-16T20:48:32.384 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.385 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.385 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:32.385 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745685+0000", 2024-09-16T20:48:32.385 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.464e-06 2024-09-16T20:48:32.385 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.385 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.385 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:32.385 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745692+0000", 2024-09-16T20:48:32.385 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.849e-06 2024-09-16T20:48:32.386 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.386 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.386 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:32.386 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745697+0000", 2024-09-16T20:48:32.386 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0170000000000004e-06 2024-09-16T20:48:32.386 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.386 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.386 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:32.386 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792771+0000", 2024-09-16T20:48:32.386 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047074224999999997 2024-09-16T20:48:32.386 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.387 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.387 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:32.387 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792845+0000", 2024-09-16T20:48:32.387 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3739999999999995e-05 2024-09-16T20:48:32.387 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.387 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.387 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:32.387 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792952+0000", 2024-09-16T20:48:32.387 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000107613 2024-09-16T20:48:32.387 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.387 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:32.388 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.388 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.388 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.388 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4317 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4659200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:32.388 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745704+0000", 2024-09-16T20:48:32.388 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 12.596116520000001, 2024-09-16T20:48:32.388 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047411778000000002, 2024-09-16T20:48:32.388 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:32.388 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:32.388 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:32.388 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:32.388 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:48:32.389 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:32.389 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4317 2024-09-16T20:48:32.389 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.389 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:32.389 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.389 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:32.389 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745704+0000", 2024-09-16T20:48:32.389 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.389 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.389 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.389 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:32.390 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745704+0000", 2024-09-16T20:48:32.390 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.390 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.390 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.390 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:32.390 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745704+0000", 2024-09-16T20:48:32.390 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0900000000000002e-07 2024-09-16T20:48:32.390 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.390 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.390 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:32.390 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745707+0000", 2024-09-16T20:48:32.391 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1660000000000001e-06 2024-09-16T20:48:32.391 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.391 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.391 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:32.391 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745722+0000", 2024-09-16T20:48:32.391 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5084e-05 2024-09-16T20:48:32.391 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.391 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.391 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:32.391 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745735+0000", 2024-09-16T20:48:32.391 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3074999999999999e-05 2024-09-16T20:48:32.391 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.392 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.392 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:32.392 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792989+0000", 2024-09-16T20:48:32.392 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047254013999999997 2024-09-16T20:48:32.392 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.392 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.392 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:32.392 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793034+0000", 2024-09-16T20:48:32.392 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4858999999999999e-05 2024-09-16T20:48:32.392 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.393 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.393 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:32.393 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793116+0000", 2024-09-16T20:48:32.393 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2170999999999996e-05 2024-09-16T20:48:32.393 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.393 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:32.393 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.393 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.393 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.393 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4318 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4667392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:32.393 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745812+0000", 2024-09-16T20:48:32.394 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 12.596008841, 2024-09-16T20:48:32.394 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047532229000000002, 2024-09-16T20:48:32.394 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:32.394 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:32.394 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:32.394 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:32.394 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:48:32.394 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:32.394 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4318 2024-09-16T20:48:32.394 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.394 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:32.394 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.395 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:32.395 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745812+0000", 2024-09-16T20:48:32.395 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.395 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.395 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.395 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:32.395 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745812+0000", 2024-09-16T20:48:32.395 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.395 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.395 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.395 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:32.396 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745812+0000", 2024-09-16T20:48:32.396 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5899999999999996e-07 2024-09-16T20:48:32.396 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.396 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.396 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:32.396 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745815+0000", 2024-09-16T20:48:32.396 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9110000000000002e-06 2024-09-16T20:48:32.396 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.396 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.397 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:32.397 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745823+0000", 2024-09-16T20:48:32.397 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1030000000000002e-06 2024-09-16T20:48:32.397 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.397 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.397 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:32.397 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745829+0000", 2024-09-16T20:48:32.397 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8449999999999999e-06 2024-09-16T20:48:32.397 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.397 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.398 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:32.398 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793159+0000", 2024-09-16T20:48:32.398 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047330009999999999 2024-09-16T20:48:32.398 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.398 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.398 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:32.398 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793251+0000", 2024-09-16T20:48:32.398 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.1991999999999996e-05 2024-09-16T20:48:32.398 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.398 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.398 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:32.399 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793344+0000", 2024-09-16T20:48:32.399 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2808999999999999e-05 2024-09-16T20:48:32.399 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.399 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:32.399 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.399 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.399 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.399 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4319 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4675584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:32.399 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745838+0000", 2024-09-16T20:48:32.399 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 12.595982628, 2024-09-16T20:48:32.399 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047640204999999998, 2024-09-16T20:48:32.400 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:32.400 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:32.400 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:32.400 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:32.400 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:48:32.400 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:32.400 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4319 2024-09-16T20:48:32.400 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.400 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:32.400 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.400 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:32.401 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745838+0000", 2024-09-16T20:48:32.401 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.401 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.401 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.401 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:32.401 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745838+0000", 2024-09-16T20:48:32.401 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.401 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.401 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.402 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:32.402 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745838+0000", 2024-09-16T20:48:32.402 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6899999999999998e-07 2024-09-16T20:48:32.402 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.402 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.402 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:32.402 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745840+0000", 2024-09-16T20:48:32.402 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0660000000000002e-06 2024-09-16T20:48:32.402 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.402 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.402 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:32.403 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745847+0000", 2024-09-16T20:48:32.403 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8689999999999998e-06 2024-09-16T20:48:32.403 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.403 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.403 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:32.403 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745853+0000", 2024-09-16T20:48:32.403 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2399999999999998e-06 2024-09-16T20:48:32.403 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.403 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.403 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:32.404 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793381+0000", 2024-09-16T20:48:32.404 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047528005999999998 2024-09-16T20:48:32.404 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.404 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.404 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:32.404 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793423+0000", 2024-09-16T20:48:32.404 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2799000000000001e-05 2024-09-16T20:48:32.404 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.404 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.404 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:32.404 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793478+0000", 2024-09-16T20:48:32.405 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4756000000000002e-05 2024-09-16T20:48:32.405 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.405 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:32.405 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.405 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.405 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.405 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4320 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4683776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:32.405 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745859+0000", 2024-09-16T20:48:32.405 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 12.595961384000001, 2024-09-16T20:48:32.405 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047811399999999997, 2024-09-16T20:48:32.405 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:32.406 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:32.406 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:32.406 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:32.406 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:48:32.406 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:32.406 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4320 2024-09-16T20:48:32.406 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.406 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:32.406 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.406 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:32.407 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745859+0000", 2024-09-16T20:48:32.407 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.407 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.407 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.407 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:32.407 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745859+0000", 2024-09-16T20:48:32.407 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.407 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.407 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.407 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:32.407 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745860+0000", 2024-09-16T20:48:32.408 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9099999999999999e-07 2024-09-16T20:48:32.408 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.408 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.408 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:32.408 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745861+0000", 2024-09-16T20:48:32.408 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8589999999999999e-06 2024-09-16T20:48:32.408 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.409 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.409 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:32.409 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745868+0000", 2024-09-16T20:48:32.409 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2249999999999997e-06 2024-09-16T20:48:32.409 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.409 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.409 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:32.409 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745873+0000", 2024-09-16T20:48:32.409 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1290000000000004e-06 2024-09-16T20:48:32.409 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.409 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.410 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:32.410 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793522+0000", 2024-09-16T20:48:32.410 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047648847000000001 2024-09-16T20:48:32.410 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.410 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.410 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:32.410 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793603+0000", 2024-09-16T20:48:32.410 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1167000000000004e-05 2024-09-16T20:48:32.410 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.410 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.410 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:32.411 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793671+0000", 2024-09-16T20:48:32.411 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7781999999999996e-05 2024-09-16T20:48:32.411 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.411 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:32.411 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.411 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.411 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.411 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4321 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4691968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:32.411 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745879+0000", 2024-09-16T20:48:32.411 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 12.595941187999999, 2024-09-16T20:48:32.411 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047950375000000003, 2024-09-16T20:48:32.412 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:32.412 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:32.412 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:32.412 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:32.412 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:48:32.412 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:32.412 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4321 2024-09-16T20:48:32.412 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.412 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:32.412 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.412 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:32.413 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745879+0000", 2024-09-16T20:48:32.413 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.413 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.413 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.413 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:32.413 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745879+0000", 2024-09-16T20:48:32.413 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.413 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.413 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.413 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:32.413 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745880+0000", 2024-09-16T20:48:32.414 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.65e-07 2024-09-16T20:48:32.414 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.414 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.414 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:32.414 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745882+0000", 2024-09-16T20:48:32.414 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.922e-06 2024-09-16T20:48:32.414 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.414 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.414 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:32.414 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745888+0000", 2024-09-16T20:48:32.414 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0510000000000002e-06 2024-09-16T20:48:32.414 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.415 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.415 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:32.415 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745893+0000", 2024-09-16T20:48:32.415 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9579999999999998e-06 2024-09-16T20:48:32.415 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.415 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.415 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:32.415 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793704+0000", 2024-09-16T20:48:32.415 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047811179000000002 2024-09-16T20:48:32.415 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.415 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.416 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:32.416 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793759+0000", 2024-09-16T20:48:32.416 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5260999999999998e-05 2024-09-16T20:48:32.416 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.416 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.416 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:32.416 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793830+0000", 2024-09-16T20:48:32.416 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0638999999999999e-05 2024-09-16T20:48:32.416 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.416 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:32.416 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.416 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.417 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.417 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4322 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4700160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:32.417 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745899+0000", 2024-09-16T20:48:32.417 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 12.59592175, 2024-09-16T20:48:32.417 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048057280000000001, 2024-09-16T20:48:32.417 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:32.417 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:32.417 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:32.417 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:32.417 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:48:32.417 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:32.418 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4322 2024-09-16T20:48:32.418 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.418 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:32.418 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.418 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:32.418 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745899+0000", 2024-09-16T20:48:32.418 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.418 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.418 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.418 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:32.418 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745899+0000", 2024-09-16T20:48:32.418 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.419 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.419 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.419 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:32.419 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745899+0000", 2024-09-16T20:48:32.419 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9299999999999999e-07 2024-09-16T20:48:32.419 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.419 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.419 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:32.419 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745901+0000", 2024-09-16T20:48:32.420 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0030000000000001e-06 2024-09-16T20:48:32.420 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.420 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.420 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:32.420 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745908+0000", 2024-09-16T20:48:32.420 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3729999999999998e-06 2024-09-16T20:48:32.420 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.420 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.420 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:32.420 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745914+0000", 2024-09-16T20:48:32.420 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.122e-06 2024-09-16T20:48:32.420 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.421 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.421 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:32.421 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793862+0000", 2024-09-16T20:48:32.421 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047948324 2024-09-16T20:48:32.421 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.421 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.421 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:32.421 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793903+0000", 2024-09-16T20:48:32.421 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0465e-05 2024-09-16T20:48:32.421 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.421 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.422 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:32.422 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793956+0000", 2024-09-16T20:48:32.422 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3600000000000002e-05 2024-09-16T20:48:32.422 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.422 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:32.422 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.422 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.422 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.422 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4323 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4708352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:32.423 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745920+0000", 2024-09-16T20:48:32.423 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 12.595900109, 2024-09-16T20:48:32.423 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048268138000000002, 2024-09-16T20:48:32.423 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:32.423 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:32.423 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:32.423 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:32.423 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:48:32.423 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:32.424 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4323 2024-09-16T20:48:32.424 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.424 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:32.424 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.424 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:32.424 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745920+0000", 2024-09-16T20:48:32.424 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.424 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.424 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.425 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:32.425 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745920+0000", 2024-09-16T20:48:32.425 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.425 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.425 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.425 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:32.425 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745921+0000", 2024-09-16T20:48:32.425 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7e-07 2024-09-16T20:48:32.425 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.425 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.425 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:32.426 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745923+0000", 2024-09-16T20:48:32.426 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7880000000000001e-06 2024-09-16T20:48:32.426 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.426 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.426 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:32.426 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745929+0000", 2024-09-16T20:48:32.426 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0809999999999999e-06 2024-09-16T20:48:32.426 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.426 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.427 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:32.427 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745935+0000", 2024-09-16T20:48:32.427 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5370000000000003e-06 2024-09-16T20:48:32.427 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.427 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.427 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:32.427 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793987+0000", 2024-09-16T20:48:32.427 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048051736999999997 2024-09-16T20:48:32.427 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.427 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.427 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:32.428 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.794070+0000", 2024-09-16T20:48:32.428 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.3277999999999998e-05 2024-09-16T20:48:32.428 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.428 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.428 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:32.428 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.794189+0000", 2024-09-16T20:48:32.428 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000118347 2024-09-16T20:48:32.428 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.428 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:32.428 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.429 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.429 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.429 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4324 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4716544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:32.429 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745946+0000", 2024-09-16T20:48:32.429 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 12.595874538, 2024-09-16T20:48:32.429 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048468153999999999, 2024-09-16T20:48:32.429 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:32.429 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:32.429 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:32.429 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:32.429 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:48:32.429 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:32.430 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4324 2024-09-16T20:48:32.430 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.430 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:32.430 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.430 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:32.430 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745946+0000", 2024-09-16T20:48:32.430 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.430 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.430 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.430 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:32.430 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745946+0000", 2024-09-16T20:48:32.430 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.431 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.431 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.431 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:32.431 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745946+0000", 2024-09-16T20:48:32.431 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4499999999999997e-07 2024-09-16T20:48:32.431 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.431 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.431 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:32.431 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745949+0000", 2024-09-16T20:48:32.431 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2529999999999999e-06 2024-09-16T20:48:32.431 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.432 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.432 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:32.432 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745955+0000", 2024-09-16T20:48:32.432 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6470000000000001e-06 2024-09-16T20:48:32.432 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.432 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.432 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:32.432 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745960+0000", 2024-09-16T20:48:32.432 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7910000000000004e-06 2024-09-16T20:48:32.432 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.432 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.433 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:32.433 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.794220+0000", 2024-09-16T20:48:32.433 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048260069000000003 2024-09-16T20:48:32.433 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.433 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.433 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:32.433 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.794311+0000", 2024-09-16T20:48:32.433 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0593000000000005e-05 2024-09-16T20:48:32.433 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.433 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.433 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:32.433 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.794414+0000", 2024-09-16T20:48:32.434 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000103356 2024-09-16T20:48:32.434 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.434 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:32.434 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.434 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.434 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.434 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2059 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 1005568~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:32.434 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.342406+0000", 2024-09-16T20:48:32.434 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 9.9994140999999992, 2024-09-16T20:48:32.434 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046803412000000003, 2024-09-16T20:48:32.434 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:32.435 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:32.435 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:32.435 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:32.435 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:32.435 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:32.435 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2059 2024-09-16T20:48:32.435 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.435 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:32.435 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.435 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:32.435 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342406+0000", 2024-09-16T20:48:32.436 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.436 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.436 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.436 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:32.436 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342406+0000", 2024-09-16T20:48:32.436 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.436 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.437 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.437 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:32.437 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342407+0000", 2024-09-16T20:48:32.437 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7399999999999998e-07 2024-09-16T20:48:32.437 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.437 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.437 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:32.437 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342409+0000", 2024-09-16T20:48:32.437 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.4509999999999999e-06 2024-09-16T20:48:32.437 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.437 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.438 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:32.438 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342418+0000", 2024-09-16T20:48:32.438 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8969999999999997e-06 2024-09-16T20:48:32.438 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.438 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.438 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:32.438 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342424+0000", 2024-09-16T20:48:32.438 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5210000000000002e-06 2024-09-16T20:48:32.438 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.438 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.438 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:32.439 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389084+0000", 2024-09-16T20:48:32.439 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046659997000000002 2024-09-16T20:48:32.439 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.439 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.439 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:32.439 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389153+0000", 2024-09-16T20:48:32.439 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9558e-05 2024-09-16T20:48:32.439 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.439 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.439 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:32.439 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389210+0000", 2024-09-16T20:48:32.440 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6514000000000001e-05 2024-09-16T20:48:32.440 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.440 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:32.440 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.440 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.440 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.440 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2061 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 1013760~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:32.440 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.342445+0000", 2024-09-16T20:48:32.440 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 9.9993755479999997, 2024-09-16T20:48:32.440 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046901708, 2024-09-16T20:48:32.441 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:32.441 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:32.441 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:32.441 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:32.441 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:32.441 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:32.441 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2061 2024-09-16T20:48:32.441 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.441 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:32.441 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.441 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:32.441 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342445+0000", 2024-09-16T20:48:32.442 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.442 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.442 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.442 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:32.442 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342445+0000", 2024-09-16T20:48:32.442 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.442 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.442 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.442 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:32.442 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342446+0000", 2024-09-16T20:48:32.443 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4700000000000001e-07 2024-09-16T20:48:32.443 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.443 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.443 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:32.443 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342448+0000", 2024-09-16T20:48:32.443 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6460000000000002e-06 2024-09-16T20:48:32.443 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.443 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.443 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:32.443 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342456+0000", 2024-09-16T20:48:32.444 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7489999999999998e-06 2024-09-16T20:48:32.444 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.444 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.444 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:32.444 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342462+0000", 2024-09-16T20:48:32.444 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5759999999999996e-06 2024-09-16T20:48:32.444 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.444 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.444 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:32.444 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389238+0000", 2024-09-16T20:48:32.444 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046776515999999997 2024-09-16T20:48:32.445 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.445 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.445 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:32.445 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389285+0000", 2024-09-16T20:48:32.445 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6581e-05 2024-09-16T20:48:32.445 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.445 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.445 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:32.445 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389347+0000", 2024-09-16T20:48:32.445 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2093000000000003e-05 2024-09-16T20:48:32.445 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.446 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:32.446 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.446 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.446 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.446 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2063 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 1021952~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:32.446 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.342470+0000", 2024-09-16T20:48:32.446 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 9.9993506589999992, 2024-09-16T20:48:32.446 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047007602000000002, 2024-09-16T20:48:32.446 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:32.446 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:32.446 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:32.447 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:32.447 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:32.447 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:32.447 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2063 2024-09-16T20:48:32.447 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.447 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:32.447 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.447 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:32.447 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342470+0000", 2024-09-16T20:48:32.447 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.447 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.448 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.448 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:32.448 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342470+0000", 2024-09-16T20:48:32.448 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.448 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.448 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.448 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:32.448 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342470+0000", 2024-09-16T20:48:32.448 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0999999999999999e-07 2024-09-16T20:48:32.448 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.448 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.449 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:32.449 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342473+0000", 2024-09-16T20:48:32.449 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.249e-06 2024-09-16T20:48:32.449 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.449 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.449 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:32.449 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342479+0000", 2024-09-16T20:48:32.449 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7549999999999997e-06 2024-09-16T20:48:32.449 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.450 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.450 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:32.450 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342485+0000", 2024-09-16T20:48:32.450 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.344e-06 2024-09-16T20:48:32.450 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.450 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.450 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:32.450 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389364+0000", 2024-09-16T20:48:32.450 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046879542000000003 2024-09-16T20:48:32.450 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.451 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.451 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:32.451 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389420+0000", 2024-09-16T20:48:32.451 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5541000000000002e-05 2024-09-16T20:48:32.451 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.451 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.451 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:32.451 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389478+0000", 2024-09-16T20:48:32.452 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7661000000000002e-05 2024-09-16T20:48:32.452 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.452 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:32.452 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.452 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.452 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.452 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2065 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 1030144~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:32.452 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.342492+0000", 2024-09-16T20:48:32.452 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 9.9993288210000006, 2024-09-16T20:48:32.452 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047127962000000002, 2024-09-16T20:48:32.452 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:32.453 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:32.453 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:32.453 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:32.453 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:32.453 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:32.453 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2065 2024-09-16T20:48:32.453 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.453 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:32.453 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.453 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:32.453 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342492+0000", 2024-09-16T20:48:32.454 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.454 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.454 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.454 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:32.454 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342492+0000", 2024-09-16T20:48:32.454 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.454 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.454 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.454 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:32.454 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342492+0000", 2024-09-16T20:48:32.454 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1100000000000001e-07 2024-09-16T20:48:32.454 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.455 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.455 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:32.455 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342494+0000", 2024-09-16T20:48:32.455 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9039999999999999e-06 2024-09-16T20:48:32.455 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.455 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.455 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:32.455 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342501+0000", 2024-09-16T20:48:32.455 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6610000000000003e-06 2024-09-16T20:48:32.455 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.455 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.455 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:32.456 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342506+0000", 2024-09-16T20:48:32.456 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8760000000000004e-06 2024-09-16T20:48:32.456 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.456 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.456 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:32.456 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389501+0000", 2024-09-16T20:48:32.456 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046995440999999999 2024-09-16T20:48:32.456 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.456 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.456 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:32.456 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389550+0000", 2024-09-16T20:48:32.457 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8949000000000002e-05 2024-09-16T20:48:32.457 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.457 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.457 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:32.457 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389620+0000", 2024-09-16T20:48:32.457 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9720000000000003e-05 2024-09-16T20:48:32.457 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.457 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:32.457 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.457 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.457 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.458 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2068 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 1038336~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:32.458 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.342513+0000", 2024-09-16T20:48:32.458 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 9.9993077269999997, 2024-09-16T20:48:32.458 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047233961999999997, 2024-09-16T20:48:32.458 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:32.458 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:32.458 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:32.458 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:32.458 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:32.458 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:32.458 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2068 2024-09-16T20:48:32.458 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.459 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:32.459 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.459 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:32.459 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342513+0000", 2024-09-16T20:48:32.459 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.459 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.459 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.459 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:32.459 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342513+0000", 2024-09-16T20:48:32.459 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.459 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.460 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.460 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:32.460 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342513+0000", 2024-09-16T20:48:32.460 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.01e-07 2024-09-16T20:48:32.460 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.460 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.460 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:32.460 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342515+0000", 2024-09-16T20:48:32.460 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7400000000000001e-06 2024-09-16T20:48:32.460 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.460 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.461 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:32.461 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342522+0000", 2024-09-16T20:48:32.461 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.516e-06 2024-09-16T20:48:32.461 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.461 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.461 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:32.461 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342527+0000", 2024-09-16T20:48:32.461 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0080000000000002e-06 2024-09-16T20:48:32.461 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.461 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.461 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:32.462 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389644+0000", 2024-09-16T20:48:32.462 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047117289 2024-09-16T20:48:32.462 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.462 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.462 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:32.462 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389690+0000", 2024-09-16T20:48:32.462 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6378000000000002e-05 2024-09-16T20:48:32.462 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.462 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.462 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:32.462 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389747+0000", 2024-09-16T20:48:32.463 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6629999999999998e-05 2024-09-16T20:48:32.463 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.463 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:32.463 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.463 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.463 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.463 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2070 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 1046528~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:32.463 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.342533+0000", 2024-09-16T20:48:32.463 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 9.9992876299999995, 2024-09-16T20:48:32.463 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047345848000000003, 2024-09-16T20:48:32.464 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:32.464 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:32.464 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:32.464 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:32.464 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:32.464 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:32.464 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2070 2024-09-16T20:48:32.464 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.464 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:32.464 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.465 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:32.465 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342533+0000", 2024-09-16T20:48:32.465 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.465 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.465 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.465 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:32.465 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342533+0000", 2024-09-16T20:48:32.465 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.465 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.465 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.466 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:32.466 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342533+0000", 2024-09-16T20:48:32.466 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7e-07 2024-09-16T20:48:32.466 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.466 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.466 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:32.466 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342535+0000", 2024-09-16T20:48:32.466 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.68e-06 2024-09-16T20:48:32.466 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.467 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.467 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:32.467 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342542+0000", 2024-09-16T20:48:32.467 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5169999999999996e-06 2024-09-16T20:48:32.467 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.467 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.467 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:32.467 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342546+0000", 2024-09-16T20:48:32.467 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8949999999999997e-06 2024-09-16T20:48:32.467 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.468 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.468 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:32.468 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389759+0000", 2024-09-16T20:48:32.468 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047212853999999999 2024-09-16T20:48:32.468 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.468 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.468 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:32.468 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389813+0000", 2024-09-16T20:48:32.468 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3585999999999998e-05 2024-09-16T20:48:32.468 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.469 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.469 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:32.469 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389879+0000", 2024-09-16T20:48:32.469 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5945999999999995e-05 2024-09-16T20:48:32.469 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.469 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:32.469 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.469 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.469 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:32.470 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:32.471 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:48:32.472 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:32.470+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:32.571 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:32.570+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:32.578 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:32.579 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:48:32.579 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:48:32.579 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:32.587 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:48:32.672 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:32.670+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:32.740 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:32.740 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:48:32.740 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:48:32.740 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:48:32.740 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:32.748 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:48:32.792 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:32.791+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:32.859 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:32.859 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:48:32.859 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:48:32.859 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:48:32.859 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.860 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3709 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1464320~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:32.860 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.638971+0000", 2024-09-16T20:48:32.860 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 10.218660183000001, 2024-09-16T20:48:32.860 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049858670000000001, 2024-09-16T20:48:32.860 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:32.860 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:32.860 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:32.860 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:32.860 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:32.860 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:32.860 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3709 2024-09-16T20:48:32.860 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.861 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:32.861 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.861 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:32.861 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638971+0000", 2024-09-16T20:48:32.861 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.861 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.861 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.861 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:32.861 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638971+0000", 2024-09-16T20:48:32.861 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.861 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.861 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.861 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:32.862 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638972+0000", 2024-09-16T20:48:32.862 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0400000000000002e-07 2024-09-16T20:48:32.862 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.862 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.862 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:32.862 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638973+0000", 2024-09-16T20:48:32.862 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.578e-06 2024-09-16T20:48:32.862 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.862 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.862 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:32.862 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638979+0000", 2024-09-16T20:48:32.862 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.502e-06 2024-09-16T20:48:32.862 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.862 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.863 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:32.863 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638983+0000", 2024-09-16T20:48:32.863 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2690000000000001e-06 2024-09-16T20:48:32.863 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.863 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.863 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:32.863 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688628+0000", 2024-09-16T20:48:32.863 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049644547999999997 2024-09-16T20:48:32.863 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.863 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.863 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:32.863 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688718+0000", 2024-09-16T20:48:32.864 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0802000000000002e-05 2024-09-16T20:48:32.864 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.864 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.864 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:32.864 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688830+0000", 2024-09-16T20:48:32.864 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000111667 2024-09-16T20:48:32.864 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.864 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:32.864 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.864 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.864 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.864 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3712 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1472512~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:32.864 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:48:32.865 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 10.218643654999999, 2024-09-16T20:48:32.865 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050004305999999998, 2024-09-16T20:48:32.865 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:32.865 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:32.865 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:32.865 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:32.865 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:32.865 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:32.865 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3712 2024-09-16T20:48:32.865 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.865 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:32.865 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.865 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:32.865 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:48:32.866 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.866 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.866 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.866 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:32.866 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:48:32.866 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.866 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.866 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.866 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:32.866 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:48:32.866 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0199999999999998e-07 2024-09-16T20:48:32.866 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.866 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.866 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:32.867 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638990+0000", 2024-09-16T20:48:32.867 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.618e-06 2024-09-16T20:48:32.867 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.867 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.867 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:32.867 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638995+0000", 2024-09-16T20:48:32.867 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3940000000000004e-06 2024-09-16T20:48:32.867 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.867 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.867 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:32.867 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639000+0000", 2024-09-16T20:48:32.867 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9309999999999999e-06 2024-09-16T20:48:32.867 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.867 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.868 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:32.868 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688867+0000", 2024-09-16T20:48:32.868 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049866955999999997 2024-09-16T20:48:32.868 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.868 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.868 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:32.868 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688933+0000", 2024-09-16T20:48:32.868 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5359999999999998e-05 2024-09-16T20:48:32.868 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.868 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.868 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:32.868 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688992+0000", 2024-09-16T20:48:32.868 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9744999999999998e-05 2024-09-16T20:48:32.869 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.869 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:32.869 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.869 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.869 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.869 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3714 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1480704~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:32.869 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:48:32.869 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 10.218626065, 2024-09-16T20:48:32.869 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050152552000000003, 2024-09-16T20:48:32.869 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:32.869 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:32.869 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:32.869 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:32.870 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:32.870 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:32.870 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3714 2024-09-16T20:48:32.870 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.870 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:32.870 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.870 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:32.870 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:48:32.870 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.870 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.870 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.870 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:32.870 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:48:32.870 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.871 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.871 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.871 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:32.871 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:48:32.871 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0499999999999999e-07 2024-09-16T20:48:32.871 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.871 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.871 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:32.871 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639007+0000", 2024-09-16T20:48:32.871 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5340000000000001e-06 2024-09-16T20:48:32.871 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.871 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.871 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:32.871 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639013+0000", 2024-09-16T20:48:32.872 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3260000000000003e-06 2024-09-16T20:48:32.872 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.872 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.872 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:32.872 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639018+0000", 2024-09-16T20:48:32.872 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2850000000000002e-06 2024-09-16T20:48:32.872 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.872 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.872 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:32.872 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689026+0000", 2024-09-16T20:48:32.872 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.05000798 2024-09-16T20:48:32.873 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.873 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.873 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:32.873 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689077+0000", 2024-09-16T20:48:32.873 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1063999999999998e-05 2024-09-16T20:48:32.873 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.873 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.873 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:32.873 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689158+0000", 2024-09-16T20:48:32.874 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1057999999999995e-05 2024-09-16T20:48:32.874 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.874 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:32.874 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.874 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.874 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.874 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3716 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1488896~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:32.874 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:48:32.874 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 10.218608254999999, 2024-09-16T20:48:32.874 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050280142, 2024-09-16T20:48:32.874 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:32.874 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:32.874 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:32.875 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:32.875 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:32.875 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:32.875 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3716 2024-09-16T20:48:32.875 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.875 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:32.875 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.875 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:32.875 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:48:32.875 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.875 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.875 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.875 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:32.875 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:48:32.876 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.876 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.876 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.876 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:32.876 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639024+0000", 2024-09-16T20:48:32.876 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9999999999999997e-07 2024-09-16T20:48:32.876 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.876 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.876 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:32.876 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639026+0000", 2024-09-16T20:48:32.876 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.781e-06 2024-09-16T20:48:32.876 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.876 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.877 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:32.877 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639032+0000", 2024-09-16T20:48:32.877 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7780000000000004e-06 2024-09-16T20:48:32.877 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.877 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.877 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:32.877 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639036+0000", 2024-09-16T20:48:32.877 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6909999999999996e-06 2024-09-16T20:48:32.877 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.877 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.877 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:32.877 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689191+0000", 2024-09-16T20:48:32.877 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050155284000000001 2024-09-16T20:48:32.877 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.878 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.878 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:32.878 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689242+0000", 2024-09-16T20:48:32.878 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0662999999999999e-05 2024-09-16T20:48:32.878 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.878 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.878 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:32.878 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689304+0000", 2024-09-16T20:48:32.878 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1345000000000003e-05 2024-09-16T20:48:32.878 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.878 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:32.878 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.878 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.878 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.879 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3719 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1497088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:32.879 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:48:32.879 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 10.218580977, 2024-09-16T20:48:32.879 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050399715999999997, 2024-09-16T20:48:32.879 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:32.879 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:32.879 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:32.879 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:32.879 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:32.879 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:32.879 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3719 2024-09-16T20:48:32.879 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.879 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:32.880 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.880 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:32.880 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:48:32.880 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.880 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.880 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.880 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:32.880 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:48:32.880 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.880 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.880 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.880 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:32.880 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:48:32.880 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9700000000000002e-07 2024-09-16T20:48:32.881 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.881 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.881 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:32.881 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639053+0000", 2024-09-16T20:48:32.881 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2900000000000001e-06 2024-09-16T20:48:32.881 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.881 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.881 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:32.881 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639060+0000", 2024-09-16T20:48:32.881 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7750000000000004e-06 2024-09-16T20:48:32.881 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.881 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.881 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:32.881 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639066+0000", 2024-09-16T20:48:32.882 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0639999999999999e-06 2024-09-16T20:48:32.882 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.882 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.882 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:32.882 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689337+0000", 2024-09-16T20:48:32.882 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050270784999999998 2024-09-16T20:48:32.882 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.882 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.882 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:32.882 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689392+0000", 2024-09-16T20:48:32.882 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4920999999999998e-05 2024-09-16T20:48:32.882 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.882 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.882 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:32.883 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689450+0000", 2024-09-16T20:48:32.883 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8483999999999998e-05 2024-09-16T20:48:32.883 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.883 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:32.883 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.883 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.883 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.883 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3721 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1505280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:32.883 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:48:32.883 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 10.218557844999999, 2024-09-16T20:48:32.883 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050513608000000002, 2024-09-16T20:48:32.883 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:32.883 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:32.884 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:32.884 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:32.884 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:32.884 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:32.884 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3721 2024-09-16T20:48:32.884 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.884 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:32.884 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.884 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:32.884 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:48:32.884 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.884 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.884 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.884 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:32.885 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:48:32.885 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.885 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.885 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.885 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:32.885 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:48:32.885 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7e-07 2024-09-16T20:48:32.885 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.885 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.885 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:32.885 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639076+0000", 2024-09-16T20:48:32.885 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8899999999999999e-06 2024-09-16T20:48:32.885 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.885 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.886 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:32.886 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639083+0000", 2024-09-16T20:48:32.886 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1210000000000001e-06 2024-09-16T20:48:32.886 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.886 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.886 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:32.886 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639089+0000", 2024-09-16T20:48:32.886 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.643e-06 2024-09-16T20:48:32.886 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.886 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.886 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:32.886 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689474+0000", 2024-09-16T20:48:32.886 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050385679000000003 2024-09-16T20:48:32.886 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.887 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.887 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:32.887 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689533+0000", 2024-09-16T20:48:32.887 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8303000000000001e-05 2024-09-16T20:48:32.887 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.887 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.887 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:32.887 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689587+0000", 2024-09-16T20:48:32.887 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4601999999999997e-05 2024-09-16T20:48:32.887 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.887 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:32.887 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.887 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.887 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.888 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3723 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1513472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:32.888 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:48:32.888 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 10.218527041, 2024-09-16T20:48:32.888 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050699691999999998, 2024-09-16T20:48:32.888 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:32.888 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:32.888 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:32.888 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:32.888 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:32.888 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:32.888 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3723 2024-09-16T20:48:32.888 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.888 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:32.889 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.889 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:32.889 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:48:32.889 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.889 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.889 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.889 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:32.889 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:48:32.889 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.889 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.889 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.889 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:32.889 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:48:32.889 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3399999999999999e-07 2024-09-16T20:48:32.890 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.890 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.890 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:32.890 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639107+0000", 2024-09-16T20:48:32.890 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3860000000000001e-06 2024-09-16T20:48:32.890 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.890 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.890 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:32.890 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639114+0000", 2024-09-16T20:48:32.890 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9800000000000001e-06 2024-09-16T20:48:32.890 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.890 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.891 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:32.891 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639129+0000", 2024-09-16T20:48:32.891 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4887e-05 2024-09-16T20:48:32.891 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.891 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.891 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:32.891 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689625+0000", 2024-09-16T20:48:32.891 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050495848000000003 2024-09-16T20:48:32.891 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.891 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.891 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:32.892 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689720+0000", 2024-09-16T20:48:32.892 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4821000000000006e-05 2024-09-16T20:48:32.892 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.892 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.892 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:32.892 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689804+0000", 2024-09-16T20:48:32.892 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4235999999999998e-05 2024-09-16T20:48:32.892 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.892 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:32.892 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.892 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.892 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.892 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3725 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1521664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:32.893 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:48:32.893 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 10.218493445, 2024-09-16T20:48:32.893 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050797660000000001, 2024-09-16T20:48:32.893 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:32.893 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:32.893 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:32.893 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:32.893 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:32.893 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:32.893 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3725 2024-09-16T20:48:32.893 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.893 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:32.893 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.894 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:32.894 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:48:32.894 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.894 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.894 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.894 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:32.894 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:48:32.894 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.894 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.894 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.894 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:32.894 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639139+0000", 2024-09-16T20:48:32.894 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8999999999999997e-07 2024-09-16T20:48:32.894 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.895 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.895 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:32.895 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639141+0000", 2024-09-16T20:48:32.895 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.182e-06 2024-09-16T20:48:32.895 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.895 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.895 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:32.895 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639154+0000", 2024-09-16T20:48:32.895 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3626e-05 2024-09-16T20:48:32.895 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.895 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.895 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:32.895 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639160+0000", 2024-09-16T20:48:32.895 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6810000000000003e-06 2024-09-16T20:48:32.896 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.896 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.896 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:32.896 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689831+0000", 2024-09-16T20:48:32.896 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050670907000000001 2024-09-16T20:48:32.896 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.896 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.896 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:32.896 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689882+0000", 2024-09-16T20:48:32.896 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0445999999999998e-05 2024-09-16T20:48:32.896 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.896 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.896 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:32.897 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689936+0000", 2024-09-16T20:48:32.897 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4327999999999999e-05 2024-09-16T20:48:32.897 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.897 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:32.897 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.897 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.897 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.897 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3727 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1529856~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:32.897 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:48:32.897 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 10.218457868, 2024-09-16T20:48:32.897 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051016922999999999, 2024-09-16T20:48:32.897 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:32.897 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:32.897 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:32.898 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:32.898 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:32.898 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:32.898 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3727 2024-09-16T20:48:32.898 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.898 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:32.898 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.898 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:32.898 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:48:32.898 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.898 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.898 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.898 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:32.899 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:48:32.899 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.899 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.899 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.899 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:32.899 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:48:32.899 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9999999999999998e-07 2024-09-16T20:48:32.899 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.899 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.899 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:32.899 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639183+0000", 2024-09-16T20:48:32.899 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2399999999999996e-06 2024-09-16T20:48:32.899 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.899 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.900 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:32.900 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639195+0000", 2024-09-16T20:48:32.900 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1113e-05 2024-09-16T20:48:32.900 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.900 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.900 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:32.900 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639206+0000", 2024-09-16T20:48:32.900 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1503e-05 2024-09-16T20:48:32.900 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.900 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.900 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:32.900 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689969+0000", 2024-09-16T20:48:32.900 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050762676 2024-09-16T20:48:32.900 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.901 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.901 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:32.901 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690071+0000", 2024-09-16T20:48:32.901 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102163 2024-09-16T20:48:32.901 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.901 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.901 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:32.901 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690191+0000", 2024-09-16T20:48:32.901 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000119728 2024-09-16T20:48:32.901 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.901 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:32.901 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.901 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.901 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.902 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3730 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1538048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:32.902 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:48:32.902 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 10.218416305, 2024-09-16T20:48:32.902 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051131467999999999, 2024-09-16T20:48:32.902 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:32.902 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:32.902 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:32.902 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:32.902 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:32.902 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:32.902 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3730 2024-09-16T20:48:32.902 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.902 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:32.902 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.903 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:32.903 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:48:32.903 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.903 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.903 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.903 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:32.903 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:48:32.903 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.903 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.903 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.903 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:32.903 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639216+0000", 2024-09-16T20:48:32.904 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.39e-07 2024-09-16T20:48:32.904 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.904 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.904 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:32.904 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639218+0000", 2024-09-16T20:48:32.904 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.537e-06 2024-09-16T20:48:32.904 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.904 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.904 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:32.904 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639229+0000", 2024-09-16T20:48:32.904 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1122e-05 2024-09-16T20:48:32.904 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.904 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.905 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:32.905 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639237+0000", 2024-09-16T20:48:32.905 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8509999999999999e-06 2024-09-16T20:48:32.905 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.905 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.905 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:32.905 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690227+0000", 2024-09-16T20:48:32.905 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050990147 2024-09-16T20:48:32.905 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.905 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.905 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:32.905 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690278+0000", 2024-09-16T20:48:32.905 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0223999999999999e-05 2024-09-16T20:48:32.906 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.906 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.906 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:32.906 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690347+0000", 2024-09-16T20:48:32.906 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9147999999999996e-05 2024-09-16T20:48:32.906 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.906 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:32.906 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.906 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.906 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.906 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3731 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1546240~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:32.906 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:48:32.906 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 10.218385596999999, 2024-09-16T20:48:32.907 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051276744999999999, 2024-09-16T20:48:32.907 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:32.907 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:32.907 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:32.907 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:32.907 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:32.907 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:32.907 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3731 2024-09-16T20:48:32.907 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.907 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:32.907 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.907 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:32.907 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:48:32.907 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.908 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.908 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.908 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:32.908 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:48:32.908 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.908 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.908 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.908 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:32.908 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:48:32.908 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9900000000000001e-07 2024-09-16T20:48:32.908 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.908 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.908 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:32.909 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639249+0000", 2024-09-16T20:48:32.909 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3630000000000001e-06 2024-09-16T20:48:32.909 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.909 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.909 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:32.909 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639255+0000", 2024-09-16T20:48:32.909 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4960000000000001e-06 2024-09-16T20:48:32.909 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.909 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.909 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:32.909 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639261+0000", 2024-09-16T20:48:32.909 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2839999999999997e-06 2024-09-16T20:48:32.909 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.909 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.910 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:32.910 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690375+0000", 2024-09-16T20:48:32.910 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051114684000000001 2024-09-16T20:48:32.910 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.910 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.910 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:32.910 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690425+0000", 2024-09-16T20:48:32.910 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9379999999999998e-05 2024-09-16T20:48:32.910 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.910 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.910 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:32.910 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690523+0000", 2024-09-16T20:48:32.910 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8139000000000003e-05 2024-09-16T20:48:32.911 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.911 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:32.911 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.911 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.911 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.911 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3732 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1554432~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:32.911 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:48:32.911 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 10.218339815, 2024-09-16T20:48:32.911 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051380096, 2024-09-16T20:48:32.911 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:32.911 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:32.911 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:32.911 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:32.911 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:32.912 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:32.912 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3732 2024-09-16T20:48:32.912 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.912 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:32.912 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.912 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:32.912 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:48:32.912 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.912 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.912 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.912 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:32.912 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:48:32.912 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.913 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.913 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.913 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:32.913 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:48:32.913 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6100000000000001e-07 2024-09-16T20:48:32.913 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.913 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.913 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:32.913 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639295+0000", 2024-09-16T20:48:32.913 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.447e-06 2024-09-16T20:48:32.913 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.913 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.913 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:32.913 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639302+0000", 2024-09-16T20:48:32.914 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0570000000000004e-06 2024-09-16T20:48:32.914 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.914 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.914 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:32.914 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639307+0000", 2024-09-16T20:48:32.914 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1429999999999997e-06 2024-09-16T20:48:32.914 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.914 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.914 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:32.914 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690555+0000", 2024-09-16T20:48:32.914 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051247963000000001 2024-09-16T20:48:32.914 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.914 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.914 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:32.915 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690602+0000", 2024-09-16T20:48:32.915 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7602000000000003e-05 2024-09-16T20:48:32.915 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.915 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.915 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:32.915 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690672+0000", 2024-09-16T20:48:32.915 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9423000000000003e-05 2024-09-16T20:48:32.915 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.915 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:32.915 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.915 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.915 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.915 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3734 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1562624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:32.915 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:48:32.916 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 10.218318164999999, 2024-09-16T20:48:32.916 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051490610999999999, 2024-09-16T20:48:32.916 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:32.916 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:32.916 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:32.916 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:32.916 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:32.916 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:32.916 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3734 2024-09-16T20:48:32.916 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.916 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:32.916 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.916 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:32.916 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:48:32.917 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.917 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.917 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.917 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:32.917 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:48:32.917 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.917 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.917 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.917 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:32.917 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639314+0000", 2024-09-16T20:48:32.917 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4400000000000001e-07 2024-09-16T20:48:32.917 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.917 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.918 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:32.918 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639316+0000", 2024-09-16T20:48:32.918 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1459999999999999e-06 2024-09-16T20:48:32.918 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.918 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.918 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:32.918 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639322+0000", 2024-09-16T20:48:32.918 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7610000000000004e-06 2024-09-16T20:48:32.918 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.918 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.919 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:32.919 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639327+0000", 2024-09-16T20:48:32.919 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.186e-06 2024-09-16T20:48:32.919 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.919 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.919 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:32.919 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690698+0000", 2024-09-16T20:48:32.919 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051371223000000001 2024-09-16T20:48:32.919 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.919 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.919 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:32.919 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690748+0000", 2024-09-16T20:48:32.920 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0192999999999999e-05 2024-09-16T20:48:32.920 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.920 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.920 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:32.920 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690804+0000", 2024-09-16T20:48:32.920 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5658000000000001e-05 2024-09-16T20:48:32.920 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.920 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:32.920 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.920 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.920 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.920 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3736 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1570816~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:32.920 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:48:32.921 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 10.218298905999999, 2024-09-16T20:48:32.921 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051596528000000003, 2024-09-16T20:48:32.921 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:32.921 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:32.921 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:32.921 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:32.921 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:32.921 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:32.921 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3736 2024-09-16T20:48:32.921 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.921 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:32.921 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.921 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:32.922 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:48:32.922 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.922 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.922 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.922 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:32.922 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:48:32.922 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.922 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.922 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.922 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:32.922 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:48:32.922 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9999999999999999e-07 2024-09-16T20:48:32.922 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.922 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.923 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:32.923 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639335+0000", 2024-09-16T20:48:32.923 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.635e-06 2024-09-16T20:48:32.923 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.923 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.923 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:32.923 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639340+0000", 2024-09-16T20:48:32.923 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5840000000000002e-06 2024-09-16T20:48:32.923 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.923 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.923 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:32.923 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639346+0000", 2024-09-16T20:48:32.923 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1190000000000002e-06 2024-09-16T20:48:32.924 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.924 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.924 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:32.924 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690817+0000", 2024-09-16T20:48:32.924 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051470721999999997 2024-09-16T20:48:32.924 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.924 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.924 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:32.924 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690875+0000", 2024-09-16T20:48:32.924 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7441000000000001e-05 2024-09-16T20:48:32.924 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.924 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.924 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:32.924 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690929+0000", 2024-09-16T20:48:32.925 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4727e-05 2024-09-16T20:48:32.925 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.925 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:32.925 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.925 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.925 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.925 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5434 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:32.925 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:48:32.925 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 9.9930373400000008, 2024-09-16T20:48:32.925 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099755831000000003, 2024-09-16T20:48:32.925 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:32.925 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:32.925 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:32.925 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:32.926 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:32.926 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:32.926 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5434 2024-09-16T20:48:32.926 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.926 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:32.926 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.926 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:32.926 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:48:32.926 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.926 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.926 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.926 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:32.926 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:48:32.926 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.927 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.927 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.927 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:32.927 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864595+0000", 2024-09-16T20:48:32.927 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1099999999999995e-07 2024-09-16T20:48:32.927 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.927 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.927 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:32.927 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864597+0000", 2024-09-16T20:48:32.927 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3989999999999998e-06 2024-09-16T20:48:32.927 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.927 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.927 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:32.928 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864603+0000", 2024-09-16T20:48:32.928 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9309999999999996e-06 2024-09-16T20:48:32.928 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.928 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.928 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:32.928 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864607+0000", 2024-09-16T20:48:32.928 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6200000000000001e-06 2024-09-16T20:48:32.928 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.928 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.928 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:32.928 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964164+0000", 2024-09-16T20:48:32.928 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099557162000000005 2024-09-16T20:48:32.928 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.928 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.929 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:32.929 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964244+0000", 2024-09-16T20:48:32.929 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9597000000000004e-05 2024-09-16T20:48:32.929 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.929 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.929 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:32.929 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964350+0000", 2024-09-16T20:48:32.929 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010641099999999999 2024-09-16T20:48:32.929 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.929 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:32.929 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.929 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.929 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.929 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5435 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 20480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:32.930 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:48:32.930 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 9.9930075180000006, 2024-09-16T20:48:32.930 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100897102, 2024-09-16T20:48:32.930 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:32.930 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:32.930 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:32.930 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:32.930 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:32.930 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:32.930 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5435 2024-09-16T20:48:32.930 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.930 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:32.930 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.930 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:32.931 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:48:32.931 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.931 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.931 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.931 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:32.931 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:48:32.931 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.931 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.931 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.931 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:32.931 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:48:32.931 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8000000000000001e-07 2024-09-16T20:48:32.931 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.931 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.932 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:32.932 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864626+0000", 2024-09-16T20:48:32.932 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.745e-06 2024-09-16T20:48:32.932 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.932 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.932 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:32.932 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864633+0000", 2024-09-16T20:48:32.932 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3339999999999998e-06 2024-09-16T20:48:32.932 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.932 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.932 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:32.932 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864636+0000", 2024-09-16T20:48:32.932 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1200000000000002e-06 2024-09-16T20:48:32.933 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.933 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.933 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:32.933 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965299+0000", 2024-09-16T20:48:32.933 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10066352000000001 2024-09-16T20:48:32.933 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.933 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.933 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:32.933 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965390+0000", 2024-09-16T20:48:32.933 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0894000000000001e-05 2024-09-16T20:48:32.933 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.933 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.933 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:32.933 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965521+0000", 2024-09-16T20:48:32.934 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013110899999999999 2024-09-16T20:48:32.934 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.934 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:32.934 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.934 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.934 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.934 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:152 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 327680~32768 out=32768b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:48:32.934 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:48:32.934 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 8.6945105760000008, 2024-09-16T20:48:32.934 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11869405800000001, 2024-09-16T20:48:32.934 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:32.934 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:32.934 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:32.935 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:32.935 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:48:32.935 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:48:32.935 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 152 2024-09-16T20:48:32.935 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.935 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:32.935 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.935 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:32.935 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:48:32.935 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.935 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.935 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.935 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:32.936 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:48:32.936 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.936 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.936 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.936 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:32.936 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163122+0000", 2024-09-16T20:48:32.936 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6400000000000003e-07 2024-09-16T20:48:32.936 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.936 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.936 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:32.936 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163124+0000", 2024-09-16T20:48:32.936 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.565e-06 2024-09-16T20:48:32.936 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.937 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.937 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:32.937 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163136+0000", 2024-09-16T20:48:32.937 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1717e-05 2024-09-16T20:48:32.937 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.937 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.937 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:32.937 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163142+0000", 2024-09-16T20:48:32.937 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1750000000000002e-06 2024-09-16T20:48:32.937 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.937 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.937 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:32.937 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163606+0000", 2024-09-16T20:48:32.937 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00046418799999999998 2024-09-16T20:48:32.938 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.938 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.938 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:32.938 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163656+0000", 2024-09-16T20:48:32.938 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9852999999999999e-05 2024-09-16T20:48:32.938 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.938 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.938 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:32.938 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281815+0000", 2024-09-16T20:48:32.938 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11815909600000001 2024-09-16T20:48:32.938 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.938 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:32.938 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.938 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.939 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.939 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:153 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:48:32.939 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:48:32.939 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 8.6944776230000009, 2024-09-16T20:48:32.939 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11903462400000001, 2024-09-16T20:48:32.939 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:32.939 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:32.939 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:32.939 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:32.939 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:48:32.939 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:48:32.939 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 153 2024-09-16T20:48:32.939 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.939 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:32.940 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.940 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:32.940 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:48:32.940 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.940 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.940 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.940 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:32.940 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:48:32.940 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.940 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.940 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.940 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:32.940 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163155+0000", 2024-09-16T20:48:32.941 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.37e-07 2024-09-16T20:48:32.941 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.941 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.941 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:32.941 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163157+0000", 2024-09-16T20:48:32.941 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6010000000000002e-06 2024-09-16T20:48:32.941 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.941 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.941 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:32.941 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163165+0000", 2024-09-16T20:48:32.941 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2619999999999998e-06 2024-09-16T20:48:32.941 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.941 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.942 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:32.942 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163171+0000", 2024-09-16T20:48:32.942 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.711e-06 2024-09-16T20:48:32.942 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.942 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.942 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:32.942 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281868+0000", 2024-09-16T20:48:32.942 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118696622 2024-09-16T20:48:32.942 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.942 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.942 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:32.942 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281969+0000", 2024-09-16T20:48:32.942 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101371 2024-09-16T20:48:32.942 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.943 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.943 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:32.943 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282189+0000", 2024-09-16T20:48:32.943 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00021952000000000001 2024-09-16T20:48:32.943 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.943 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:32.943 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.943 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.943 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.943 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:154 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 360448~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:48:32.943 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:48:32.943 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 8.6944458509999993, 2024-09-16T20:48:32.943 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11932035100000001, 2024-09-16T20:48:32.943 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:32.944 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:32.944 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:32.944 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:32.944 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:48:32.944 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:48:32.944 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 154 2024-09-16T20:48:32.944 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.944 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:32.944 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.944 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:32.944 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:48:32.944 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.944 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.945 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.945 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:32.945 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:48:32.945 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.945 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.945 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.945 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:32.945 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:48:32.945 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:48:32.945 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.945 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.945 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:32.945 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163189+0000", 2024-09-16T20:48:32.946 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6840000000000001e-06 2024-09-16T20:48:32.946 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.946 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.946 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:32.946 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163196+0000", 2024-09-16T20:48:32.946 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7920000000000004e-06 2024-09-16T20:48:32.946 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.946 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.946 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:32.946 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163201+0000", 2024-09-16T20:48:32.946 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2809999999999999e-06 2024-09-16T20:48:32.946 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.946 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.946 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:32.947 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282219+0000", 2024-09-16T20:48:32.947 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.119018343 2024-09-16T20:48:32.947 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.947 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.947 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:32.947 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282305+0000", 2024-09-16T20:48:32.947 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5865000000000005e-05 2024-09-16T20:48:32.947 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.947 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.947 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:32.947 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282506+0000", 2024-09-16T20:48:32.947 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000200927 2024-09-16T20:48:32.947 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.947 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:32.948 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.948 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.948 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.948 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop(client.4366.0:3088 3.1e e38/23 3:79e5ec01:::rbd_header.110e891a9b2b:head v 38'306, mlcod=36'304)", 2024-09-16T20:48:32.948 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:48:32.948 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 8.6934269830000002, 2024-09-16T20:48:32.948 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118102759, 2024-09-16T20:48:32.948 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:32.948 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:32.948 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:32.948 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:32.948 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.948 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:32.949 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:48:32.949 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.949 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.949 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.949 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:32.949 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:48:32.949 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:32.949 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.949 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.949 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:32.949 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:48:32.949 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.85e-07 2024-09-16T20:48:32.949 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.949 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.949 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:32.950 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164212+0000", 2024-09-16T20:48:32.950 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3999999999999997e-06 2024-09-16T20:48:32.950 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.950 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.950 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:32.950 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281553+0000", 2024-09-16T20:48:32.950 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11734135499999999 2024-09-16T20:48:32.950 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.950 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.950 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:32.950 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281567+0000", 2024-09-16T20:48:32.950 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4188999999999999e-05 2024-09-16T20:48:32.950 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.951 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.951 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:32.951 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281653+0000", 2024-09-16T20:48:32.951 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5385999999999999e-05 2024-09-16T20:48:32.951 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.951 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.951 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:32.951 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281679+0000", 2024-09-16T20:48:32.951 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6689000000000001e-05 2024-09-16T20:48:32.951 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.951 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.951 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:32.951 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282277+0000", 2024-09-16T20:48:32.951 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00059798000000000002 2024-09-16T20:48:32.952 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:32.952 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:32.952 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:32.952 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282307+0000", 2024-09-16T20:48:32.952 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9975e-05 2024-09-16T20:48:32.952 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.952 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:32.952 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.952 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:32.952 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:32.952 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:32.954 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:48:32.954 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:32.952+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:33.054 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:33.052+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:33.067 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:33.068 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:48:33.068 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:48:33.068 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:33.076 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:48:33.155 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:33.154+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:33.225 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:33.225 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:48:33.225 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:48:33.225 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:48:33.226 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:33.234 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:48:33.256 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:33.255+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:33.301 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #8, image image_client.0-clone8 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent9 2024-09-16T20:48:33.345 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:33.345 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:48:33.345 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:48:33.346 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:48:33.346 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.346 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6412 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3561472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:33.346 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871685+0000", 2024-09-16T20:48:33.346 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 13.471695112999999, 2024-09-16T20:48:33.346 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048146568000000001, 2024-09-16T20:48:33.346 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:33.346 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:33.346 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:33.346 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:33.346 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:33.346 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:33.346 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6412 2024-09-16T20:48:33.346 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.347 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:33.347 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.347 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:33.347 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871685+0000", 2024-09-16T20:48:33.347 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:33.347 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.347 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.347 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:33.347 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871685+0000", 2024-09-16T20:48:33.347 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:33.347 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.347 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.347 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:33.348 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871685+0000", 2024-09-16T20:48:33.348 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.58e-07 2024-09-16T20:48:33.348 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.348 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.348 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:33.348 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871687+0000", 2024-09-16T20:48:33.348 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1830000000000001e-06 2024-09-16T20:48:33.348 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.348 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.348 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:33.348 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871694+0000", 2024-09-16T20:48:33.348 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7689999999999999e-06 2024-09-16T20:48:33.348 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.348 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.349 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:33.349 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871699+0000", 2024-09-16T20:48:33.349 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1769999999999997e-06 2024-09-16T20:48:33.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.349 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.349 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:33.349 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.919662+0000", 2024-09-16T20:48:33.349 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047963121999999997 2024-09-16T20:48:33.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.349 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.349 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:33.349 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.919744+0000", 2024-09-16T20:48:33.349 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1997999999999997e-05 2024-09-16T20:48:33.350 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.350 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.350 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:33.350 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.919831+0000", 2024-09-16T20:48:33.350 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6860999999999994e-05 2024-09-16T20:48:33.350 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:33.350 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:33.350 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:33.350 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.350 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.350 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6414 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3569664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:33.350 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871708+0000", 2024-09-16T20:48:33.350 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 13.471672165999999, 2024-09-16T20:48:33.351 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048373317999999998, 2024-09-16T20:48:33.351 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:33.351 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:33.351 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:33.351 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:33.351 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:33.351 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:33.351 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6414 2024-09-16T20:48:33.351 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.351 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:33.351 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.351 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:33.351 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871708+0000", 2024-09-16T20:48:33.351 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:33.352 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.352 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.352 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:33.352 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871708+0000", 2024-09-16T20:48:33.352 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:33.352 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.352 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.352 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:33.352 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871708+0000", 2024-09-16T20:48:33.352 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9799999999999999e-07 2024-09-16T20:48:33.352 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.352 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.352 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:33.353 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871710+0000", 2024-09-16T20:48:33.353 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2699999999999999e-06 2024-09-16T20:48:33.353 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.353 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.353 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:33.353 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871718+0000", 2024-09-16T20:48:33.353 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8499999999999994e-06 2024-09-16T20:48:33.353 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.353 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.353 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:33.353 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871723+0000", 2024-09-16T20:48:33.353 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1819999999999997e-06 2024-09-16T20:48:33.353 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.354 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.354 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:33.354 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.919883+0000", 2024-09-16T20:48:33.354 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048159955999999997 2024-09-16T20:48:33.354 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.354 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.354 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:33.354 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.919969+0000", 2024-09-16T20:48:33.354 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5808000000000002e-05 2024-09-16T20:48:33.354 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.354 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.354 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:33.354 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920081+0000", 2024-09-16T20:48:33.355 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000111854 2024-09-16T20:48:33.355 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:33.355 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:33.355 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:33.355 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.355 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.355 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6415 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3577856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:33.355 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871730+0000", 2024-09-16T20:48:33.355 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 13.471649651, 2024-09-16T20:48:33.355 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.04851482, 2024-09-16T20:48:33.355 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:33.355 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:33.355 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:33.355 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:33.356 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:33.356 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:33.356 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6415 2024-09-16T20:48:33.356 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.356 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:33.356 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.356 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:33.356 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871730+0000", 2024-09-16T20:48:33.356 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:33.356 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.356 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.356 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:33.357 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871730+0000", 2024-09-16T20:48:33.357 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:33.357 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.357 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.357 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:33.357 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871731+0000", 2024-09-16T20:48:33.357 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6899999999999998e-07 2024-09-16T20:48:33.357 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.357 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.357 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:33.357 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871733+0000", 2024-09-16T20:48:33.357 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0600000000000002e-06 2024-09-16T20:48:33.357 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.357 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.358 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:33.358 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871740+0000", 2024-09-16T20:48:33.358 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3749999999999997e-06 2024-09-16T20:48:33.358 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.358 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.358 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:33.358 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871749+0000", 2024-09-16T20:48:33.358 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9109999999999999e-06 2024-09-16T20:48:33.358 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.358 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.358 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:33.358 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920129+0000", 2024-09-16T20:48:33.358 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048380175999999997 2024-09-16T20:48:33.359 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.359 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.359 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:33.359 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920181+0000", 2024-09-16T20:48:33.359 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2077000000000003e-05 2024-09-16T20:48:33.359 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.359 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.359 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:33.359 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920245+0000", 2024-09-16T20:48:33.359 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3851999999999998e-05 2024-09-16T20:48:33.359 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:33.359 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:33.360 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:33.360 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.360 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.360 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6417 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3586048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:33.360 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871756+0000", 2024-09-16T20:48:33.360 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 13.471624016, 2024-09-16T20:48:33.360 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048628301999999998, 2024-09-16T20:48:33.360 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:33.360 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:33.360 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:33.360 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:33.361 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:33.361 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:33.361 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6417 2024-09-16T20:48:33.361 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.361 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:33.361 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.361 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:33.361 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871756+0000", 2024-09-16T20:48:33.361 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:33.361 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.361 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.361 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:33.361 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871756+0000", 2024-09-16T20:48:33.361 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:33.362 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.362 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.362 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:33.362 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871756+0000", 2024-09-16T20:48:33.362 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0100000000000005e-07 2024-09-16T20:48:33.362 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.362 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.362 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:33.362 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871758+0000", 2024-09-16T20:48:33.362 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.902e-06 2024-09-16T20:48:33.362 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.362 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.362 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:33.363 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871764+0000", 2024-09-16T20:48:33.363 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8170000000000004e-06 2024-09-16T20:48:33.363 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.363 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.363 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:33.363 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871769+0000", 2024-09-16T20:48:33.363 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6500000000000004e-06 2024-09-16T20:48:33.363 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.363 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.363 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:33.363 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920276+0000", 2024-09-16T20:48:33.363 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048507794999999999 2024-09-16T20:48:33.363 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.363 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.364 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:33.364 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920329+0000", 2024-09-16T20:48:33.364 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2534999999999998e-05 2024-09-16T20:48:33.364 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.364 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.364 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:33.364 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920384+0000", 2024-09-16T20:48:33.364 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5102000000000002e-05 2024-09-16T20:48:33.364 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:33.364 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:33.364 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:33.364 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.364 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.365 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6419 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3594240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:33.365 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871775+0000", 2024-09-16T20:48:33.365 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 13.471604838999999, 2024-09-16T20:48:33.365 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048754724999999999, 2024-09-16T20:48:33.365 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:33.365 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:33.365 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:33.365 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:33.365 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:33.365 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:33.365 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6419 2024-09-16T20:48:33.365 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.365 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:33.365 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.366 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:33.366 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871775+0000", 2024-09-16T20:48:33.366 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:33.366 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.366 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.366 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:33.366 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871775+0000", 2024-09-16T20:48:33.366 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:33.366 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.366 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.366 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:33.366 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871775+0000", 2024-09-16T20:48:33.366 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2500000000000001e-07 2024-09-16T20:48:33.367 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.367 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.367 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:33.367 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871777+0000", 2024-09-16T20:48:33.367 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9649999999999998e-06 2024-09-16T20:48:33.367 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.367 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.367 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:33.367 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871784+0000", 2024-09-16T20:48:33.367 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3419999999999996e-06 2024-09-16T20:48:33.367 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.367 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.368 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:33.368 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871789+0000", 2024-09-16T20:48:33.368 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1889999999999998e-06 2024-09-16T20:48:33.368 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.368 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.368 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:33.368 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920416+0000", 2024-09-16T20:48:33.368 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048627363999999999 2024-09-16T20:48:33.368 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.368 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.368 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:33.368 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920472+0000", 2024-09-16T20:48:33.368 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5429000000000001e-05 2024-09-16T20:48:33.369 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.369 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.369 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:33.369 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920530+0000", 2024-09-16T20:48:33.369 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8111e-05 2024-09-16T20:48:33.369 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:33.369 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:33.369 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:33.369 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.369 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.369 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6420 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3602432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:33.369 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871796+0000", 2024-09-16T20:48:33.369 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 13.471583925999999, 2024-09-16T20:48:33.370 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048857298, 2024-09-16T20:48:33.370 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:33.370 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:33.370 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:33.370 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:33.370 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:33.370 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:33.370 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6420 2024-09-16T20:48:33.370 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.370 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:33.370 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.370 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:33.370 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871796+0000", 2024-09-16T20:48:33.371 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:33.371 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.371 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.371 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:33.371 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871796+0000", 2024-09-16T20:48:33.371 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:33.371 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.371 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.371 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:33.371 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871854+0000", 2024-09-16T20:48:33.371 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8187999999999999e-05 2024-09-16T20:48:33.371 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.371 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.372 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:33.372 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871856+0000", 2024-09-16T20:48:33.372 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3549999999999999e-06 2024-09-16T20:48:33.372 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.372 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.372 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:33.372 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871866+0000", 2024-09-16T20:48:33.372 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6500000000000008e-06 2024-09-16T20:48:33.372 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.372 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.372 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:33.372 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871872+0000", 2024-09-16T20:48:33.372 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3799999999999999e-06 2024-09-16T20:48:33.373 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.373 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.373 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:33.373 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920558+0000", 2024-09-16T20:48:33.373 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048685797000000003 2024-09-16T20:48:33.373 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.373 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.373 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:33.373 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920603+0000", 2024-09-16T20:48:33.373 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4629999999999998e-05 2024-09-16T20:48:33.373 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.373 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.373 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:33.373 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920653+0000", 2024-09-16T20:48:33.374 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0297999999999999e-05 2024-09-16T20:48:33.374 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:33.374 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:33.374 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:33.374 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.374 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.374 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6421 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3610624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:33.374 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871882+0000", 2024-09-16T20:48:33.374 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 13.471497857999999, 2024-09-16T20:48:33.374 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048913381999999998, 2024-09-16T20:48:33.374 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:33.374 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:33.374 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:33.374 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:33.375 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:33.375 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:33.375 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6421 2024-09-16T20:48:33.375 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.375 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:33.375 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.375 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:33.375 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871882+0000", 2024-09-16T20:48:33.375 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:33.376 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.376 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.376 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:33.376 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871882+0000", 2024-09-16T20:48:33.376 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:33.376 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.376 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.376 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:33.376 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.873257+0000", 2024-09-16T20:48:33.376 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001374789 2024-09-16T20:48:33.376 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.376 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.376 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:33.377 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.873261+0000", 2024-09-16T20:48:33.377 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5589999999999999e-06 2024-09-16T20:48:33.377 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.377 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.377 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:33.377 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.873275+0000", 2024-09-16T20:48:33.377 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3675999999999999e-05 2024-09-16T20:48:33.377 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.377 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.377 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:33.377 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.873281+0000", 2024-09-16T20:48:33.377 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6930000000000004e-06 2024-09-16T20:48:33.377 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.378 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.378 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:33.378 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920684+0000", 2024-09-16T20:48:33.378 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047402913999999997 2024-09-16T20:48:33.378 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.378 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.378 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:33.378 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920737+0000", 2024-09-16T20:48:33.378 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3600000000000002e-05 2024-09-16T20:48:33.378 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.378 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.378 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:33.378 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920795+0000", 2024-09-16T20:48:33.379 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8150999999999999e-05 2024-09-16T20:48:33.379 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:33.379 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:33.379 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:33.379 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.379 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.379 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:7491 3.1e 3:7c48f6ae:::rbd_object_map.110e891a9b2b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:33.379 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.925865+0000", 2024-09-16T20:48:33.379 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 13.417514713999999, 2024-09-16T20:48:33.379 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048897293000000001, 2024-09-16T20:48:33.379 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:33.379 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:33.379 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:33.380 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:33.380 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:33.380 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:33.380 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 7491 2024-09-16T20:48:33.380 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.380 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:33.380 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.380 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:33.380 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925865+0000", 2024-09-16T20:48:33.380 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:33.380 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.380 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.380 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:33.381 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925865+0000", 2024-09-16T20:48:33.381 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:33.381 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.381 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.381 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:33.381 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925866+0000", 2024-09-16T20:48:33.381 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.39e-07 2024-09-16T20:48:33.381 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.381 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.381 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:33.384 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925868+0000", 2024-09-16T20:48:33.384 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.508e-06 2024-09-16T20:48:33.384 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.384 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.384 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:33.384 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925874+0000", 2024-09-16T20:48:33.384 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7919999999999998e-06 2024-09-16T20:48:33.384 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.384 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.384 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:33.385 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925879+0000", 2024-09-16T20:48:33.385 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8579999999999999e-06 2024-09-16T20:48:33.385 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.385 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.385 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:33.385 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.970875+0000", 2024-09-16T20:48:33.385 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044996412 2024-09-16T20:48:33.385 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.385 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.385 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:33.385 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.970970+0000", 2024-09-16T20:48:33.385 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4732e-05 2024-09-16T20:48:33.385 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.385 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.386 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-16T20:48:33.386 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.971275+0000", 2024-09-16T20:48:33.386 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00030464199999999999 2024-09-16T20:48:33.386 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.386 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.386 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:33.386 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.974598+0000", 2024-09-16T20:48:33.386 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0033238159999999998 2024-09-16T20:48:33.386 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.386 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.386 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:48:33.386 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.974666+0000", 2024-09-16T20:48:33.387 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7617e-05 2024-09-16T20:48:33.387 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.387 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.387 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:48:33.387 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.974718+0000", 2024-09-16T20:48:33.387 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2259000000000002e-05 2024-09-16T20:48:33.387 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.387 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.387 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:33.387 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.974762+0000", 2024-09-16T20:48:33.387 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4218000000000003e-05 2024-09-16T20:48:33.387 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:33.387 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:33.388 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:33.388 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.388 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.388 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4750 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1488896~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:33.388 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689052+0000", 2024-09-16T20:48:33.388 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 10.654328175, 2024-09-16T20:48:33.388 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048208359999999999, 2024-09-16T20:48:33.388 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:33.388 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:33.388 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:33.388 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:33.388 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:33.388 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:33.389 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4750 2024-09-16T20:48:33.389 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.389 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:33.389 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.389 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:33.389 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689052+0000", 2024-09-16T20:48:33.389 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:33.389 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.389 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.389 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:33.389 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689052+0000", 2024-09-16T20:48:33.389 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:33.389 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.389 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.390 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:33.390 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689052+0000", 2024-09-16T20:48:33.390 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.65e-07 2024-09-16T20:48:33.390 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.390 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.390 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:33.390 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689058+0000", 2024-09-16T20:48:33.390 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.835e-06 2024-09-16T20:48:33.390 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.390 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.390 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:33.390 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689064+0000", 2024-09-16T20:48:33.390 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2729999999999999e-06 2024-09-16T20:48:33.391 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.391 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.391 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:33.391 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689069+0000", 2024-09-16T20:48:33.391 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1220000000000003e-06 2024-09-16T20:48:33.391 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.391 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.391 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:33.391 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737161+0000", 2024-09-16T20:48:33.391 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048091776000000003 2024-09-16T20:48:33.391 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.391 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.391 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:33.391 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737210+0000", 2024-09-16T20:48:33.392 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8575000000000002e-05 2024-09-16T20:48:33.392 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.392 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.392 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:33.392 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737260+0000", 2024-09-16T20:48:33.392 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0414000000000002e-05 2024-09-16T20:48:33.392 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:33.392 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:33.392 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:33.392 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.392 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.392 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4752 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1497088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:33.392 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689076+0000", 2024-09-16T20:48:33.393 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 10.654304033000001, 2024-09-16T20:48:33.393 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048322324999999999, 2024-09-16T20:48:33.393 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:33.393 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:33.393 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:33.393 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:33.393 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:33.393 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:33.393 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4752 2024-09-16T20:48:33.393 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.393 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:33.393 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.393 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:33.394 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689076+0000", 2024-09-16T20:48:33.394 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:33.394 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.394 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.394 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:33.394 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689076+0000", 2024-09-16T20:48:33.394 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:33.394 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.394 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.394 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:33.394 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689076+0000", 2024-09-16T20:48:33.394 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8000000000000001e-07 2024-09-16T20:48:33.394 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.394 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.395 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:33.395 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689078+0000", 2024-09-16T20:48:33.395 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8369999999999999e-06 2024-09-16T20:48:33.395 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.395 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.395 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:33.395 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689084+0000", 2024-09-16T20:48:33.395 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1319999999999999e-06 2024-09-16T20:48:33.395 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.395 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.395 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:33.395 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689090+0000", 2024-09-16T20:48:33.395 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0430000000000004e-06 2024-09-16T20:48:33.395 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.396 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.396 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:33.396 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737286+0000", 2024-09-16T20:48:33.396 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048195670000000003 2024-09-16T20:48:33.396 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.396 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.396 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:33.396 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737337+0000", 2024-09-16T20:48:33.396 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1511000000000002e-05 2024-09-16T20:48:33.396 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.396 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.396 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:33.396 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737398+0000", 2024-09-16T20:48:33.397 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0751999999999997e-05 2024-09-16T20:48:33.397 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:33.397 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:33.397 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:33.397 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.397 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.397 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4754 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1505280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:33.397 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689097+0000", 2024-09-16T20:48:33.397 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 10.654282516, 2024-09-16T20:48:33.397 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048429947000000001, 2024-09-16T20:48:33.397 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:33.397 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:33.397 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:33.397 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:33.398 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:33.398 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:33.398 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4754 2024-09-16T20:48:33.398 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.398 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:33.398 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.398 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:33.398 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689097+0000", 2024-09-16T20:48:33.398 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:33.398 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.398 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.398 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:33.398 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689097+0000", 2024-09-16T20:48:33.399 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:33.399 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.399 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.399 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:33.399 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689098+0000", 2024-09-16T20:48:33.399 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.15e-07 2024-09-16T20:48:33.399 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.399 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.399 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:33.399 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689100+0000", 2024-09-16T20:48:33.399 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8190000000000001e-06 2024-09-16T20:48:33.399 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.399 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.399 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:33.400 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689106+0000", 2024-09-16T20:48:33.400 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2489999999999998e-06 2024-09-16T20:48:33.400 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.400 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.400 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:33.400 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689111+0000", 2024-09-16T20:48:33.400 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3179999999999997e-06 2024-09-16T20:48:33.400 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.400 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.400 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:33.400 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737428+0000", 2024-09-16T20:48:33.400 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048316572000000002 2024-09-16T20:48:33.400 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.401 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.401 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:33.401 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737475+0000", 2024-09-16T20:48:33.401 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7392000000000003e-05 2024-09-16T20:48:33.401 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.401 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.401 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:33.401 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737527+0000", 2024-09-16T20:48:33.401 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2182000000000003e-05 2024-09-16T20:48:33.401 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:33.401 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:33.401 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:33.401 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.401 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.402 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4756 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1513472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:33.402 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689118+0000", 2024-09-16T20:48:33.402 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 10.654262295000001, 2024-09-16T20:48:33.402 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048535146000000001, 2024-09-16T20:48:33.402 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:33.402 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:33.402 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:33.402 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:33.402 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:33.402 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:33.402 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4756 2024-09-16T20:48:33.402 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.402 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:33.402 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.403 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:33.403 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689118+0000", 2024-09-16T20:48:33.403 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:33.403 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.403 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.403 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:33.403 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689118+0000", 2024-09-16T20:48:33.403 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:33.403 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.403 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.403 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:33.403 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689118+0000", 2024-09-16T20:48:33.403 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7099999999999997e-07 2024-09-16T20:48:33.403 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.404 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.404 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:33.404 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689120+0000", 2024-09-16T20:48:33.404 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.787e-06 2024-09-16T20:48:33.404 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.404 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.404 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:33.404 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689126+0000", 2024-09-16T20:48:33.404 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2720000000000003e-06 2024-09-16T20:48:33.404 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.404 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.404 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:33.404 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689134+0000", 2024-09-16T20:48:33.404 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2190000000000004e-06 2024-09-16T20:48:33.405 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.405 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.405 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:33.405 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737553+0000", 2024-09-16T20:48:33.405 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048418993 2024-09-16T20:48:33.405 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.405 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.405 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:33.405 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737598+0000", 2024-09-16T20:48:33.405 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4743999999999997e-05 2024-09-16T20:48:33.405 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.405 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.405 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:33.405 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737653+0000", 2024-09-16T20:48:33.406 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4759999999999997e-05 2024-09-16T20:48:33.406 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:33.406 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:33.406 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:33.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.406 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.406 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4758 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1521664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:33.406 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689141+0000", 2024-09-16T20:48:33.406 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 10.654239315, 2024-09-16T20:48:33.406 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048640741000000001, 2024-09-16T20:48:33.406 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:33.406 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:33.406 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:33.407 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:33.407 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:33.407 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:33.407 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4758 2024-09-16T20:48:33.407 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.407 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:33.407 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.407 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:33.407 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689141+0000", 2024-09-16T20:48:33.407 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:33.407 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.407 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.407 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:33.407 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689141+0000", 2024-09-16T20:48:33.408 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:33.408 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.408 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.408 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:33.408 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689141+0000", 2024-09-16T20:48:33.408 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8799999999999998e-07 2024-09-16T20:48:33.408 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.408 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.408 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:33.408 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689143+0000", 2024-09-16T20:48:33.408 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.945e-06 2024-09-16T20:48:33.408 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.408 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.409 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:33.409 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689149+0000", 2024-09-16T20:48:33.409 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9800000000000003e-06 2024-09-16T20:48:33.409 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.409 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.409 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:33.409 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689155+0000", 2024-09-16T20:48:33.409 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7509999999999997e-06 2024-09-16T20:48:33.409 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.409 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.409 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:33.409 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737681+0000", 2024-09-16T20:48:33.409 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048526825000000003 2024-09-16T20:48:33.409 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.410 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.410 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:33.410 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737729+0000", 2024-09-16T20:48:33.410 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7219999999999999e-05 2024-09-16T20:48:33.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.410 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.410 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:33.410 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737781+0000", 2024-09-16T20:48:33.410 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2632e-05 2024-09-16T20:48:33.410 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:33.410 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:33.410 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:33.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.411 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.411 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4760 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1529856~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:33.411 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689161+0000", 2024-09-16T20:48:33.411 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 10.654218968, 2024-09-16T20:48:33.411 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048743981999999998, 2024-09-16T20:48:33.411 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:33.411 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:33.411 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:33.411 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:33.411 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:33.411 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:33.411 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4760 2024-09-16T20:48:33.411 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.411 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:33.412 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.412 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:33.412 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689161+0000", 2024-09-16T20:48:33.412 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:33.412 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.412 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.412 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:33.412 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689161+0000", 2024-09-16T20:48:33.412 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:33.412 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.412 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.412 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:33.412 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689161+0000", 2024-09-16T20:48:33.413 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5599999999999995e-07 2024-09-16T20:48:33.413 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.413 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.413 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:33.413 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689163+0000", 2024-09-16T20:48:33.413 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.818e-06 2024-09-16T20:48:33.413 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.413 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.413 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:33.413 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689169+0000", 2024-09-16T20:48:33.414 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0809999999999999e-06 2024-09-16T20:48:33.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.414 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.414 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:33.414 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689174+0000", 2024-09-16T20:48:33.414 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.938e-06 2024-09-16T20:48:33.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.414 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.414 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:33.414 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737807+0000", 2024-09-16T20:48:33.414 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048633124 2024-09-16T20:48:33.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.414 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.414 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:33.415 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737851+0000", 2024-09-16T20:48:33.415 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3764000000000003e-05 2024-09-16T20:48:33.415 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.415 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.415 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:33.415 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737905+0000", 2024-09-16T20:48:33.415 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3701e-05 2024-09-16T20:48:33.415 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:33.415 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:33.415 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:33.415 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.415 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.415 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4762 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1538048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:33.416 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689182+0000", 2024-09-16T20:48:33.416 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 10.654197745999999, 2024-09-16T20:48:33.416 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048850275999999998, 2024-09-16T20:48:33.416 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:33.416 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:33.416 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:33.416 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:33.416 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:33.416 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:33.416 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4762 2024-09-16T20:48:33.416 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.416 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:33.416 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.416 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:33.417 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689182+0000", 2024-09-16T20:48:33.417 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:33.417 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.417 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.417 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:33.417 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689182+0000", 2024-09-16T20:48:33.417 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:33.417 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.417 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.417 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:33.417 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689182+0000", 2024-09-16T20:48:33.417 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-16T20:48:33.417 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.417 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.418 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:33.418 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689184+0000", 2024-09-16T20:48:33.418 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8190000000000001e-06 2024-09-16T20:48:33.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.418 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.418 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:33.418 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689191+0000", 2024-09-16T20:48:33.418 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2530000000000001e-06 2024-09-16T20:48:33.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.418 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.418 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:33.418 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689195+0000", 2024-09-16T20:48:33.418 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9559999999999996e-06 2024-09-16T20:48:33.419 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.419 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.419 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:33.419 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737933+0000", 2024-09-16T20:48:33.419 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048737919999999997 2024-09-16T20:48:33.419 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.419 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.419 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:33.419 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737982+0000", 2024-09-16T20:48:33.419 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8089e-05 2024-09-16T20:48:33.419 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.419 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.419 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:33.419 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738032+0000", 2024-09-16T20:48:33.420 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0868999999999997e-05 2024-09-16T20:48:33.420 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:33.420 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:33.420 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:33.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.420 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.420 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4764 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1546240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:33.420 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689201+0000", 2024-09-16T20:48:33.420 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 10.654178389, 2024-09-16T20:48:33.420 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048957550000000002, 2024-09-16T20:48:33.420 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:33.420 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:33.420 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:33.420 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:33.421 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:33.421 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:33.421 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4764 2024-09-16T20:48:33.421 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.421 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:33.421 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.421 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:33.421 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689201+0000", 2024-09-16T20:48:33.421 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:33.421 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.421 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.421 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:33.421 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689201+0000", 2024-09-16T20:48:33.422 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:33.422 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.422 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.422 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:33.422 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689202+0000", 2024-09-16T20:48:33.422 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4200000000000002e-07 2024-09-16T20:48:33.422 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.422 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.422 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:33.422 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689204+0000", 2024-09-16T20:48:33.422 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9750000000000001e-06 2024-09-16T20:48:33.422 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.422 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.422 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:33.423 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689210+0000", 2024-09-16T20:48:33.423 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1e-06 2024-09-16T20:48:33.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.423 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.423 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:33.423 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689215+0000", 2024-09-16T20:48:33.423 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0180000000000001e-06 2024-09-16T20:48:33.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.423 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.423 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:33.423 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738058+0000", 2024-09-16T20:48:33.423 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048843573000000001 2024-09-16T20:48:33.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.424 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:33.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738102+0000", 2024-09-16T20:48:33.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3256000000000001e-05 2024-09-16T20:48:33.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.424 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:33.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738159+0000", 2024-09-16T20:48:33.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7286e-05 2024-09-16T20:48:33.424 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:33.424 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:33.424 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:33.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.425 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4766 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1554432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:33.425 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689221+0000", 2024-09-16T20:48:33.425 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 10.654158423, 2024-09-16T20:48:33.425 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049064996, 2024-09-16T20:48:33.425 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:33.425 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:33.425 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:33.425 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:33.425 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:33.425 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:33.425 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4766 2024-09-16T20:48:33.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.426 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:33.426 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.426 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:33.426 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689221+0000", 2024-09-16T20:48:33.426 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:33.426 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.426 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.426 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:33.426 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689221+0000", 2024-09-16T20:48:33.426 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:33.426 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.426 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.426 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:33.426 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689222+0000", 2024-09-16T20:48:33.427 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.34e-07 2024-09-16T20:48:33.427 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.427 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.427 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:33.427 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689224+0000", 2024-09-16T20:48:33.427 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.846e-06 2024-09-16T20:48:33.427 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.427 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.427 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:33.427 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689247+0000", 2024-09-16T20:48:33.427 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3448e-05 2024-09-16T20:48:33.427 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.427 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.428 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:33.428 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689254+0000", 2024-09-16T20:48:33.428 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6969999999999996e-06 2024-09-16T20:48:33.428 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.428 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.428 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:33.428 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738188+0000", 2024-09-16T20:48:33.428 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048934520000000002 2024-09-16T20:48:33.428 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.428 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.428 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:33.428 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738234+0000", 2024-09-16T20:48:33.428 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5695999999999998e-05 2024-09-16T20:48:33.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.429 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.429 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:33.429 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738286+0000", 2024-09-16T20:48:33.429 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2355000000000003e-05 2024-09-16T20:48:33.429 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:33.429 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:33.429 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:33.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.429 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.429 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4768 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1562624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:33.429 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689262+0000", 2024-09-16T20:48:33.429 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 10.654117468999999, 2024-09-16T20:48:33.429 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049157157999999999, 2024-09-16T20:48:33.430 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:33.430 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:33.430 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:33.430 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:33.430 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:33.430 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:33.430 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4768 2024-09-16T20:48:33.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.430 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:33.430 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.430 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:33.430 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689262+0000", 2024-09-16T20:48:33.430 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:33.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.431 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.431 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:33.431 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689262+0000", 2024-09-16T20:48:33.431 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:33.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.431 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.431 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:33.431 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689263+0000", 2024-09-16T20:48:33.431 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6199999999999998e-07 2024-09-16T20:48:33.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.431 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.431 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:33.431 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689270+0000", 2024-09-16T20:48:33.432 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.798e-06 2024-09-16T20:48:33.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.432 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.432 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:33.432 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689277+0000", 2024-09-16T20:48:33.432 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0890000000000002e-06 2024-09-16T20:48:33.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.432 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.432 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:33.432 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689282+0000", 2024-09-16T20:48:33.432 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0470000000000001e-06 2024-09-16T20:48:33.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.432 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.433 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:33.433 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738319+0000", 2024-09-16T20:48:33.433 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049037139 2024-09-16T20:48:33.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.433 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.433 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:33.433 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738365+0000", 2024-09-16T20:48:33.433 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6356999999999997e-05 2024-09-16T20:48:33.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.433 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.433 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:33.433 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738420+0000", 2024-09-16T20:48:33.433 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4265999999999998e-05 2024-09-16T20:48:33.434 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:33.434 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:33.434 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:33.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.434 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.434 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4770 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1570816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:33.434 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689289+0000", 2024-09-16T20:48:33.434 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 10.654090958999999, 2024-09-16T20:48:33.434 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049253705000000002, 2024-09-16T20:48:33.434 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:33.434 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:33.434 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:33.434 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:33.434 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:33.435 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:33.435 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4770 2024-09-16T20:48:33.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.435 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:33.435 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.435 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:33.435 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689289+0000", 2024-09-16T20:48:33.435 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:33.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.435 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.435 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:33.435 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689289+0000", 2024-09-16T20:48:33.435 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:33.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.436 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.436 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:33.436 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689289+0000", 2024-09-16T20:48:33.436 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0999999999999999e-07 2024-09-16T20:48:33.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.436 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.436 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:33.436 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689291+0000", 2024-09-16T20:48:33.436 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0030000000000001e-06 2024-09-16T20:48:33.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.436 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.436 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:33.437 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689297+0000", 2024-09-16T20:48:33.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9479999999999996e-06 2024-09-16T20:48:33.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.437 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:33.437 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689302+0000", 2024-09-16T20:48:33.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7389999999999999e-06 2024-09-16T20:48:33.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.437 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:33.437 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738442+0000", 2024-09-16T20:48:33.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049140001000000003 2024-09-16T20:48:33.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:33.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738490+0000", 2024-09-16T20:48:33.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7787000000000002e-05 2024-09-16T20:48:33.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:33.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738543+0000", 2024-09-16T20:48:33.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2816999999999999e-05 2024-09-16T20:48:33.438 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:33.438 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:33.438 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:33.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.439 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:3088 3.1e 3:79e5ec01:::rbd_header.110e891a9b2b:head [watch watch cookie 140213437171488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:48:33.439 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:48:33.439 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 9.1795349420000001, 2024-09-16T20:48:33.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.11895186200000001, 2024-09-16T20:48:33.439 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:33.439 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:33.439 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:33.439 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:33.439 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:48:33.439 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:48:33.439 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3088 2024-09-16T20:48:33.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.439 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:33.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.440 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:33.440 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:48:33.440 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:33.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.440 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:33.440 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:48:33.440 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:33.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.440 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:33.440 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163846+0000", 2024-09-16T20:48:33.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6799999999999999e-07 2024-09-16T20:48:33.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:33.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163850+0000", 2024-09-16T20:48:33.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.929e-06 2024-09-16T20:48:33.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:33.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163862+0000", 2024-09-16T20:48:33.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1972e-05 2024-09-16T20:48:33.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:33.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163867+0000", 2024-09-16T20:48:33.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.451e-06 2024-09-16T20:48:33.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:33.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163912+0000", 2024-09-16T20:48:33.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4456000000000003e-05 2024-09-16T20:48:33.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:33.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163962+0000", 2024-09-16T20:48:33.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0578000000000003e-05 2024-09-16T20:48:33.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-16T20:48:33.443 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164058+0000", 2024-09-16T20:48:33.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5934999999999995e-05 2024-09-16T20:48:33.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.443 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:48:33.443 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164570+0000", 2024-09-16T20:48:33.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00051241100000000005 2024-09-16T20:48:33.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.443 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:33.443 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282629+0000", 2024-09-16T20:48:33.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.118058986 2024-09-16T20:48:33.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:48:33.444 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282725+0000", 2024-09-16T20:48:33.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5063999999999996e-05 2024-09-16T20:48:33.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:33.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:33.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:33.444 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282797+0000", 2024-09-16T20:48:33.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2212000000000006e-05 2024-09-16T20:48:33.444 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:33.444 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:33.444 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:33.444 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:33.444 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:33.444 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:33.445 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T20:48:33.446 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:33.444+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:33.447 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:48:33.546 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:33.545+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:33.555 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:33.555 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:48:33.555 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:48:33.555 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:33.564 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:48:33.572 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:48:33.572 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:48:33.589 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:48:33.590 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:48:33.606 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:48:33.606 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:48:33.623 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:48:33.623 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:48:33.640 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:48:33.641 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:48:33.647 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:33.646+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:33.660 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:48:33.660 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:48:33.668 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T20:48:33.722 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:33.723 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:33.731 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:48:33.776 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:33.774+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:33.849 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:33.848+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:33.857 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:33.857 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:33.867 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:48:33.951 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:33.950+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:34.033 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:34.034 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:34.044 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:48:34.085 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:34.083+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:34.152 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:34.151+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:34.193 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:34.193 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:34.201 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:48:34.253 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:34.252+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:34.265 INFO:teuthology.orchestra.run.smithi139.stdout:1244 punch from 0x7fc0400 to 0x7fcb400, (0xb000 bytes) 2024-09-16T20:48:34.274 INFO:teuthology.orchestra.run.smithi139.stdout:1248 punch from 0x72c0000 to 0x72ca800, (0xa800 bytes) 2024-09-16T20:48:34.290 INFO:teuthology.orchestra.run.smithi139.stdout:1249 trunc from 0xbf53800 to 0x14da00 2024-09-16T20:48:34.311 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:34.311 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:34.320 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:48:34.332 INFO:teuthology.orchestra.run.smithi139.stdout:1250 write 0x64b1800 thru 0x64b8d6b (0x756c bytes) 2024-09-16T20:48:34.333 INFO:teuthology.orchestra.run.smithi139.stdout:1251 read 0x4593a00 thru 0x459c8c3 (0x8ec4 bytes) 2024-09-16T20:48:34.333 INFO:teuthology.orchestra.run.smithi139.stdout:1256 write 0x37c1a00 thru 0x37c3645 (0x1c46 bytes) 2024-09-16T20:48:34.336 INFO:teuthology.orchestra.run.smithi139.stdout:1259 read 0x27c3800 thru 0x27c9c16 (0x6417 bytes) 2024-09-16T20:48:34.337 INFO:teuthology.orchestra.run.smithi139.stdout:1260 read 0x5c85600 thru 0x5c89251 (0x3c52 bytes) 2024-09-16T20:48:34.354 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:34.353+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:34.432 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:34.433 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:34.441 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:48:34.485 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:34.484+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:34.548 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:34.548 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:34.555 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:34.554+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:34.556 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:48:34.611 INFO:teuthology.orchestra.run.smithi139.stdout:1261 write 0xe8b7800 thru 0xe8ba37f (0x2b80 bytes) 2024-09-16T20:48:34.611 INFO:teuthology.orchestra.run.smithi139.stdout:1265 write 0xfdca00 thru 0xfe6893 (0x9e94 bytes) 2024-09-16T20:48:34.613 INFO:teuthology.orchestra.run.smithi139.stdout:1266 punch from 0xdece800 to 0xdedda00, (0xf200 bytes) 2024-09-16T20:48:34.613 INFO:teuthology.orchestra.run.smithi139.stdout:1267 read 0xe522a00 thru 0xe5283da (0x59db bytes) 2024-09-16T20:48:34.614 INFO:teuthology.orchestra.run.smithi139.stdout:1270 write 0xe1dce00 thru 0xe1e0c4c (0x3e4d bytes) 2024-09-16T20:48:34.614 INFO:teuthology.orchestra.run.smithi139.stdout:1272 punch from 0x3c6b600 to 0x3c6ee00, (0x3800 bytes) 2024-09-16T20:48:34.614 INFO:teuthology.orchestra.run.smithi139.stdout:1274 punch from 0x3418000 to 0x341e400, (0x6400 bytes) 2024-09-16T20:48:34.614 INFO:teuthology.orchestra.run.smithi139.stdout:1275 punch from 0x1426e00 to 0x142b800, (0x4a00 bytes) 2024-09-16T20:48:34.614 INFO:teuthology.orchestra.run.smithi139.stdout:1276 read 0x5cff200 thru 0x5d0df9d (0xed9e bytes) 2024-09-16T20:48:34.615 INFO:teuthology.orchestra.run.smithi139.stdout:1279 punch from 0x9183e00 to 0x9191a00, (0xdc00 bytes) 2024-09-16T20:48:34.615 INFO:teuthology.orchestra.run.smithi139.stdout:1280 read 0x2b50600 thru 0x2b5235a (0x1d5b bytes) 2024-09-16T20:48:34.615 INFO:teuthology.orchestra.run.smithi139.stdout:1282 read 0x22fd200 thru 0x22fd9d5 (0x7d6 bytes) 2024-09-16T20:48:34.615 INFO:teuthology.orchestra.run.smithi139.stdout:1283 read 0x2e99200 thru 0x2e9e520 (0x5321 bytes) 2024-09-16T20:48:34.615 INFO:teuthology.orchestra.run.smithi139.stdout:1284 read 0xbd33e00 thru 0xbd3d9cb (0x9bcc bytes) 2024-09-16T20:48:34.615 INFO:teuthology.orchestra.run.smithi139.stdout:1285 read 0x4e25200 thru 0x4e271e0 (0x1fe1 bytes) 2024-09-16T20:48:34.615 INFO:teuthology.orchestra.run.smithi139.stdout:1288 write 0x7f4ae00 thru 0x7f4dce5 (0x2ee6 bytes) 2024-09-16T20:48:34.615 INFO:teuthology.orchestra.run.smithi139.stdout:1289 read 0x8b67600 thru 0x8b6c8a5 (0x52a6 bytes) 2024-09-16T20:48:34.615 INFO:teuthology.orchestra.run.smithi139.stdout:1291 read 0x98da600 thru 0x98dca63 (0x2464 bytes) 2024-09-16T20:48:34.620 INFO:teuthology.orchestra.run.smithi139.stdout:1292 trunc from 0xe8ba400 to 0xa52e800 2024-09-16T20:48:34.640 INFO:teuthology.orchestra.run.smithi139.stdout:1293 read 0x8642800 thru 0x864588f (0x3090 bytes) 2024-09-16T20:48:34.646 INFO:teuthology.orchestra.run.smithi139.stdout:1294 read 0x2b48200 thru 0x2b53e22 (0xbc23 bytes) 2024-09-16T20:48:34.646 INFO:teuthology.orchestra.run.smithi139.stdout:1296 write 0x5098200 thru 0x50a444e (0xc24f bytes) 2024-09-16T20:48:34.646 INFO:teuthology.orchestra.run.smithi139.stdout:1297 read 0x47aba00 thru 0x47b4288 (0x8889 bytes) 2024-09-16T20:48:34.646 INFO:teuthology.orchestra.run.smithi139.stdout:1299 read 0x5f10600 thru 0x5f1ab10 (0xa511 bytes) 2024-09-16T20:48:34.646 INFO:teuthology.orchestra.run.smithi139.stdout:1300 write 0x64b5a00 thru 0x64c2835 (0xce36 bytes) 2024-09-16T20:48:34.656 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:34.655+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:34.661 INFO:teuthology.orchestra.run.smithi029.stderr:nodeep-scrub is set 2024-09-16T20:48:34.669 INFO:teuthology.orchestra.run.smithi139.stdout:1301 write 0xd438000 thru 0xd447f25 (0xff26 bytes) 2024-09-16T20:48:34.674 INFO:teuthology.orchestra.run.smithi139.stdout:1304 punch from 0x70bc400 to 0x70ca000, (0xdc00 bytes) 2024-09-16T20:48:34.675 INFO:teuthology.orchestra.run.smithi139.stdout:1306 write 0x283b400 thru 0x2840ad0 (0x56d1 bytes) 2024-09-16T20:48:34.675 INFO:teuthology.orchestra.run.smithi139.stdout:1311 read 0x9ac8a00 thru 0x9ac9d7b (0x137c bytes) 2024-09-16T20:48:34.675 INFO:teuthology.orchestra.run.smithi139.stdout:1312 punch from 0xd146c00 to 0xd14e400, (0x7800 bytes) 2024-09-16T20:48:34.675 INFO:teuthology.orchestra.run.smithi139.stdout:1313 punch from 0x994de00 to 0x9953a00, (0x5c00 bytes) 2024-09-16T20:48:34.675 INFO:teuthology.orchestra.run.smithi139.stdout:1314 read 0x124c000 thru 0x124c528 (0x529 bytes) 2024-09-16T20:48:34.681 INFO:teuthology.orchestra.run.smithi139.stdout:1317 trunc from 0xd448000 to 0x868a800 2024-09-16T20:48:34.725 INFO:teuthology.orchestra.run.smithi139.stdout:1318 write 0xe162800 thru 0xe164ca5 (0x24a6 bytes) 2024-09-16T20:48:34.725 INFO:teuthology.orchestra.run.smithi139.stdout:1319 write 0xbe50e00 thru 0xbe567c1 (0x59c2 bytes) 2024-09-16T20:48:34.725 INFO:teuthology.orchestra.run.smithi139.stdout:1321 write 0x664ca00 thru 0x66580f8 (0xb6f9 bytes) 2024-09-16T20:48:34.726 INFO:teuthology.orchestra.run.smithi139.stdout:1322 write 0x2a21600 thru 0x2a2768c (0x608d bytes) 2024-09-16T20:48:34.726 INFO:teuthology.orchestra.run.smithi139.stdout:1323 write 0x3a37800 thru 0x3a41bcb (0xa3cc bytes) 2024-09-16T20:48:34.726 INFO:teuthology.orchestra.run.smithi139.stdout:1325 write 0x62eaa00 thru 0x62ee3a7 (0x39a8 bytes) 2024-09-16T20:48:34.726 INFO:teuthology.orchestra.run.smithi139.stdout:1327 punch from 0x9b1ae00 to 0x9b28e00, (0xe000 bytes) 2024-09-16T20:48:34.726 INFO:teuthology.orchestra.run.smithi139.stdout:1329 punch from 0x260200 to 0x263600, (0x3400 bytes) 2024-09-16T20:48:34.727 INFO:teuthology.orchestra.run.smithi139.stdout:1331 write 0xc820800 thru 0xc82bbe7 (0xb3e8 bytes) 2024-09-16T20:48:34.730 INFO:teuthology.orchestra.run.smithi139.stdout:1333 trunc from 0xe164e00 to 0xb447400 2024-09-16T20:48:34.732 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:34.733 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:34.744 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:48:34.757 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:34.756+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:34.778 INFO:teuthology.orchestra.run.smithi139.stdout:1335 write 0xdbda400 thru 0xdbe94f9 (0xf0fa bytes) 2024-09-16T20:48:34.778 INFO:teuthology.orchestra.run.smithi139.stdout:1337 write 0x1f5c800 thru 0x1f681c6 (0xb9c7 bytes) 2024-09-16T20:48:34.778 INFO:teuthology.orchestra.run.smithi139.stdout:1338 punch from 0xd262600 to 0xd26d400, (0xae00 bytes) 2024-09-16T20:48:34.778 INFO:teuthology.orchestra.run.smithi139.stdout:1340 punch from 0xcb1ca00 to 0xcb25e00, (0x9400 bytes) 2024-09-16T20:48:34.778 INFO:teuthology.orchestra.run.smithi139.stdout:1341 read 0x12dfa00 thru 0x12e58e4 (0x5ee5 bytes) 2024-09-16T20:48:34.778 INFO:teuthology.orchestra.run.smithi139.stdout:1343 write 0x4f96000 thru 0x4f96c3f (0xc40 bytes) 2024-09-16T20:48:34.778 INFO:teuthology.orchestra.run.smithi139.stdout:1345 punch from 0x241fa00 to 0x242c400, (0xca00 bytes) 2024-09-16T20:48:34.779 INFO:teuthology.orchestra.run.smithi139.stdout:1346 write 0x885de00 thru 0x8868811 (0xaa12 bytes) 2024-09-16T20:48:34.780 INFO:teuthology.orchestra.run.smithi139.stdout:1347 trunc from 0xdbe9600 to 0xbd90600 2024-09-16T20:48:34.811 INFO:teuthology.orchestra.run.smithi139.stdout:truncating to largest ever: 0xeb04800 2024-09-16T20:48:34.816 INFO:teuthology.orchestra.run.smithi139.stdout:1351 trunc from 0xbd90600 to 0xeb04800 2024-09-16T20:48:34.829 INFO:teuthology.orchestra.run.smithi139.stdout:1352 write 0xc671600 thru 0xc67d32d (0xbd2e bytes) 2024-09-16T20:48:34.829 INFO:teuthology.orchestra.run.smithi139.stdout:1353 write 0xa543800 thru 0xa54e513 (0xad14 bytes) 2024-09-16T20:48:34.830 INFO:teuthology.orchestra.run.smithi139.stdout:1354 read 0x1a67000 thru 0x1a74e92 (0xde93 bytes) 2024-09-16T20:48:34.830 INFO:teuthology.orchestra.run.smithi139.stdout:1357 write 0xd488200 thru 0xd48d70d (0x550e bytes) 2024-09-16T20:48:34.830 INFO:teuthology.orchestra.run.smithi139.stdout:1359 punch from 0x104b600 to 0x1058800, (0xd200 bytes) 2024-09-16T20:48:34.830 INFO:teuthology.orchestra.run.smithi139.stdout:1361 read 0x92d2e00 thru 0x92e0c14 (0xde15 bytes) 2024-09-16T20:48:34.831 INFO:teuthology.orchestra.run.smithi139.stdout:1364 write 0x8431e00 thru 0x8438af4 (0x6cf5 bytes) 2024-09-16T20:48:34.837 INFO:teuthology.orchestra.run.smithi139.stdout:1365 trunc from 0xeb04800 to 0x939ec00 2024-09-16T20:48:34.854 INFO:teuthology.orchestra.run.smithi139.stdout:1366 punch from 0x657ca00 to 0x6580c00, (0x4200 bytes) 2024-09-16T20:48:34.858 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:34.857+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:34.874 INFO:teuthology.orchestra.run.smithi139.stdout:1370 write 0xd251a00 thru 0xd261408 (0xfa09 bytes) 2024-09-16T20:48:34.876 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:34.877 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:34.880 INFO:teuthology.orchestra.run.smithi139.stdout:1371 read 0x75f3a00 thru 0x75f4eab (0x14ac bytes) 2024-09-16T20:48:34.886 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:48:34.893 INFO:teuthology.orchestra.run.smithi139.stdout:1375 trunc from 0xd261600 to 0x18a0600 2024-09-16T20:48:34.924 INFO:teuthology.orchestra.run.smithi139.stdout:1376 punch from 0x467400 to 0x472e00, (0xba00 bytes) 2024-09-16T20:48:34.942 INFO:teuthology.orchestra.run.smithi139.stdout:1379 trunc from 0x18a0600 to 0xb730e00 2024-09-16T20:48:34.959 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:34.958+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:34.962 INFO:teuthology.orchestra.run.smithi139.stdout:1380 trunc from 0xb730e00 to 0x2af3000 2024-09-16T20:48:34.986 INFO:teuthology.orchestra.run.smithi139.stdout:1381 trunc from 0x2af3000 to 0xdb21400 2024-09-16T20:48:34.995 INFO:teuthology.orchestra.run.smithi139.stdout:1382 punch from 0x7bfb200 to 0x7c01200, (0x6000 bytes) 2024-09-16T20:48:34.996 INFO:teuthology.orchestra.run.smithi139.stdout:1383 write 0x81600 thru 0x8d10d (0xbb0e bytes) 2024-09-16T20:48:34.997 INFO:teuthology.orchestra.run.smithi139.stdout:1386 read 0xda17800 thru 0xda2476a (0xcf6b bytes) 2024-09-16T20:48:34.997 INFO:teuthology.orchestra.run.smithi139.stdout:1387 read 0xb06ba00 thru 0xb078e4a (0xd44b bytes) 2024-09-16T20:48:34.998 INFO:teuthology.orchestra.run.smithi139.stdout:1389 write 0xa9e1200 thru 0xa9e5363 (0x4164 bytes) 2024-09-16T20:48:34.998 INFO:teuthology.orchestra.run.smithi139.stdout:1392 read 0x548ba00 thru 0x549599d (0x9f9e bytes) 2024-09-16T20:48:35.006 INFO:teuthology.orchestra.run.smithi139.stdout:1396 trunc from 0xdb21400 to 0x6dc2200 2024-09-16T20:48:35.012 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:35.013 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:35.023 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:48:35.029 INFO:teuthology.orchestra.run.smithi139.stdout:1397 punch from 0x5bb1800 to 0x5bbdc00, (0xc400 bytes) 2024-09-16T20:48:35.029 INFO:teuthology.orchestra.run.smithi139.stdout:1399 punch from 0x4ef000 to 0x4f8800, (0x9800 bytes) 2024-09-16T20:48:35.029 INFO:teuthology.orchestra.run.smithi139.stdout:1401 read 0x640c000 thru 0x6413ad9 (0x7ada bytes) 2024-09-16T20:48:35.034 INFO:teuthology.orchestra.run.smithi139.stdout:1403 trunc from 0x6dc2200 to 0xb442a00 2024-09-16T20:48:35.055 INFO:teuthology.orchestra.run.smithi139.stdout:1404 write 0xe16a200 thru 0xe16c830 (0x2631 bytes) 2024-09-16T20:48:35.062 INFO:teuthology.orchestra.run.smithi139.stdout:1407 trunc from 0xe16ca00 to 0x8a71000 2024-09-16T20:48:35.064 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:35.063+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:35.078 INFO:teuthology.orchestra.run.smithi139.stdout:1408 read 0x6fa1600 thru 0x6fa32ff (0x1d00 bytes) 2024-09-16T20:48:35.085 INFO:teuthology.orchestra.run.smithi139.stdout:1409 write 0x3b55200 thru 0x3b60c09 (0xba0a bytes) 2024-09-16T20:48:35.102 INFO:teuthology.orchestra.run.smithi139.stdout:1410 write 0xbad4c00 thru 0xbae3f45 (0xf346 bytes) 2024-09-16T20:48:35.126 INFO:teuthology.orchestra.run.smithi139.stdout:1414 write 0xd3ad000 thru 0xd3b1ebb (0x4ebc bytes) 2024-09-16T20:48:35.145 INFO:teuthology.orchestra.run.smithi139.stdout:1416 write 0xd8a7200 thru 0xd8aff0c (0x8d0d bytes) 2024-09-16T20:48:35.151 INFO:teuthology.orchestra.run.smithi139.stdout:1419 trunc from 0xd8b0000 to 0x95bd200 2024-09-16T20:48:35.161 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:35.160+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:35.168 INFO:teuthology.orchestra.run.smithi139.stdout:1421 trunc from 0x95bd200 to 0xb518e00 2024-09-16T20:48:35.194 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:35.194 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:35.194 INFO:teuthology.orchestra.run.smithi139.stdout:1422 write 0xd899600 thru 0xd8a6f38 (0xd939 bytes) 2024-09-16T20:48:35.198 INFO:teuthology.orchestra.run.smithi139.stdout:1425 trunc from 0xd8a7000 to 0x9d86c00 2024-09-16T20:48:35.204 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:48:35.222 INFO:teuthology.orchestra.run.smithi139.stdout:1426 clone 11 order 24 su 2097152 sc 6 2024-09-16T20:48:35.288 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:35.287+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:35.351 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:35.351 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:35.359 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:48:35.400 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:35.399+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:35.463 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:35.462+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:35.506 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:35.506 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:35.515 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:48:35.564 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:35.563+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:35.665 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:35.665 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:35.666 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:35.663+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:35.672 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T20:48:35.676 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:48:35.736 INFO:teuthology.orchestra.run.smithi139.stdout:truncating image image_client.0-clone10 from 0x9d86c00 (overlap 0x14da00) to 0x85600 2024-09-16T20:48:35.766 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:35.765+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:35.796 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5] out_osds: [3] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-16T20:48:35.796 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T20:48:35.796 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-16T20:48:35.796 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-16T20:48:35.799 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:35.800 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:35.808 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:48:35.808 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:48:35.809 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:48:35.827 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:48:35.827 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:48:35.843 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:48:35.844 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:48:35.860 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:48:35.861 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:48:35.867 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:35.866+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:35.893 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:48:35.893 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:48:35.918 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:48:35.918 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:48:35.962 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:35.962 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:48:35.962 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:48:35.962 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:48:35.962 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:35.971 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:48:35.973 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:35.972+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:35.998 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #9, image image_client.0-clone9 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent10 2024-09-16T20:48:36.069 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:36.068+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:36.082 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:36.082 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:48:36.082 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:48:36.082 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:48:36.082 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.083 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6412 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3561472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:36.083 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871685+0000", 2024-09-16T20:48:36.083 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 16.208973445000002, 2024-09-16T20:48:36.083 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048146568000000001, 2024-09-16T20:48:36.083 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:36.083 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:36.083 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:36.083 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:36.083 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:36.083 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:36.083 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6412 2024-09-16T20:48:36.083 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.083 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:36.084 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.084 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:36.084 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871685+0000", 2024-09-16T20:48:36.084 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.084 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.084 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.084 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:36.084 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871685+0000", 2024-09-16T20:48:36.084 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.084 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.084 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.084 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:36.084 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871685+0000", 2024-09-16T20:48:36.085 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.58e-07 2024-09-16T20:48:36.085 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.085 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.085 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:36.085 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871687+0000", 2024-09-16T20:48:36.085 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1830000000000001e-06 2024-09-16T20:48:36.085 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.085 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.085 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:36.085 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871694+0000", 2024-09-16T20:48:36.085 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7689999999999999e-06 2024-09-16T20:48:36.085 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.085 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.086 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:36.086 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871699+0000", 2024-09-16T20:48:36.086 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1769999999999997e-06 2024-09-16T20:48:36.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.086 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.086 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:36.086 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.919662+0000", 2024-09-16T20:48:36.086 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047963121999999997 2024-09-16T20:48:36.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.086 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.086 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:36.086 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.919744+0000", 2024-09-16T20:48:36.086 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1997999999999997e-05 2024-09-16T20:48:36.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.087 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.087 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:36.087 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.919831+0000", 2024-09-16T20:48:36.087 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6860999999999994e-05 2024-09-16T20:48:36.087 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.087 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:36.087 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.087 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.087 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6414 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3569664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:36.087 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871708+0000", 2024-09-16T20:48:36.087 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 16.208950498, 2024-09-16T20:48:36.088 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048373317999999998, 2024-09-16T20:48:36.088 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:36.088 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:36.088 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:36.088 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:36.088 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:36.088 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:36.088 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6414 2024-09-16T20:48:36.088 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.088 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:36.088 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.088 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:36.088 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871708+0000", 2024-09-16T20:48:36.089 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.089 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.089 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.089 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:36.089 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871708+0000", 2024-09-16T20:48:36.089 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.089 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.089 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.089 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:36.089 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871708+0000", 2024-09-16T20:48:36.089 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9799999999999999e-07 2024-09-16T20:48:36.089 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.089 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.090 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:36.090 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871710+0000", 2024-09-16T20:48:36.090 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2699999999999999e-06 2024-09-16T20:48:36.090 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.090 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.090 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:36.090 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871718+0000", 2024-09-16T20:48:36.090 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8499999999999994e-06 2024-09-16T20:48:36.090 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.090 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.090 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:36.090 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871723+0000", 2024-09-16T20:48:36.090 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1819999999999997e-06 2024-09-16T20:48:36.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:36.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.919883+0000", 2024-09-16T20:48:36.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048159955999999997 2024-09-16T20:48:36.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:36.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.919969+0000", 2024-09-16T20:48:36.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5808000000000002e-05 2024-09-16T20:48:36.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:36.092 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920081+0000", 2024-09-16T20:48:36.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000111854 2024-09-16T20:48:36.092 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.092 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:36.092 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.092 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.092 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6415 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3577856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:36.092 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871730+0000", 2024-09-16T20:48:36.092 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 16.208927982999999, 2024-09-16T20:48:36.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.04851482, 2024-09-16T20:48:36.092 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:36.092 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:36.093 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:36.093 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:36.093 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:36.093 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:36.093 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6415 2024-09-16T20:48:36.093 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.093 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:36.093 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.093 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:36.093 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871730+0000", 2024-09-16T20:48:36.093 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.093 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.093 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.093 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:36.094 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871730+0000", 2024-09-16T20:48:36.094 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.094 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.094 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.094 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:36.094 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871731+0000", 2024-09-16T20:48:36.094 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6899999999999998e-07 2024-09-16T20:48:36.094 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.094 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.094 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:36.094 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871733+0000", 2024-09-16T20:48:36.094 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0600000000000002e-06 2024-09-16T20:48:36.094 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.094 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:36.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871740+0000", 2024-09-16T20:48:36.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3749999999999997e-06 2024-09-16T20:48:36.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.095 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:36.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871749+0000", 2024-09-16T20:48:36.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9109999999999999e-06 2024-09-16T20:48:36.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.095 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:36.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920129+0000", 2024-09-16T20:48:36.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048380175999999997 2024-09-16T20:48:36.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.096 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.096 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:36.096 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920181+0000", 2024-09-16T20:48:36.096 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2077000000000003e-05 2024-09-16T20:48:36.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.096 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.096 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:36.096 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920245+0000", 2024-09-16T20:48:36.096 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3851999999999998e-05 2024-09-16T20:48:36.096 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.096 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:36.096 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.097 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.097 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6417 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3586048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:36.097 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871756+0000", 2024-09-16T20:48:36.097 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 16.208902347999999, 2024-09-16T20:48:36.097 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048628301999999998, 2024-09-16T20:48:36.097 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:36.097 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:36.097 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:36.097 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:36.097 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:36.097 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:36.097 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6417 2024-09-16T20:48:36.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.098 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:36.098 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.098 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:36.098 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871756+0000", 2024-09-16T20:48:36.098 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.098 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.098 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.098 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:36.098 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871756+0000", 2024-09-16T20:48:36.098 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.098 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.098 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.098 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:36.098 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871756+0000", 2024-09-16T20:48:36.099 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0100000000000005e-07 2024-09-16T20:48:36.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.099 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.099 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:36.099 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871758+0000", 2024-09-16T20:48:36.099 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.902e-06 2024-09-16T20:48:36.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.099 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.099 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:36.099 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871764+0000", 2024-09-16T20:48:36.099 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8170000000000004e-06 2024-09-16T20:48:36.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.099 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.099 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:36.100 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871769+0000", 2024-09-16T20:48:36.100 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6500000000000004e-06 2024-09-16T20:48:36.100 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.100 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.100 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:36.100 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920276+0000", 2024-09-16T20:48:36.100 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048507794999999999 2024-09-16T20:48:36.100 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.100 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.100 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:36.100 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920329+0000", 2024-09-16T20:48:36.100 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2534999999999998e-05 2024-09-16T20:48:36.100 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.101 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:36.101 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920384+0000", 2024-09-16T20:48:36.101 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5102000000000002e-05 2024-09-16T20:48:36.101 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.101 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:36.101 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.101 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6419 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3594240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:36.101 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871775+0000", 2024-09-16T20:48:36.101 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 16.208883171, 2024-09-16T20:48:36.101 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048754724999999999, 2024-09-16T20:48:36.101 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:36.102 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:36.102 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:36.102 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:36.102 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:36.102 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:36.102 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6419 2024-09-16T20:48:36.102 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.102 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:36.102 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.102 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:36.102 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871775+0000", 2024-09-16T20:48:36.102 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.102 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.102 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.103 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:36.103 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871775+0000", 2024-09-16T20:48:36.103 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.103 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.103 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.103 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:36.103 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871775+0000", 2024-09-16T20:48:36.103 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2500000000000001e-07 2024-09-16T20:48:36.103 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.103 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.103 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:36.103 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871777+0000", 2024-09-16T20:48:36.103 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9649999999999998e-06 2024-09-16T20:48:36.103 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.104 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.104 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:36.104 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871784+0000", 2024-09-16T20:48:36.104 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3419999999999996e-06 2024-09-16T20:48:36.104 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.104 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.104 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:36.104 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871789+0000", 2024-09-16T20:48:36.104 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1889999999999998e-06 2024-09-16T20:48:36.104 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.104 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.104 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:36.104 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920416+0000", 2024-09-16T20:48:36.105 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048627363999999999 2024-09-16T20:48:36.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.105 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:36.105 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920472+0000", 2024-09-16T20:48:36.105 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5429000000000001e-05 2024-09-16T20:48:36.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.105 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:36.105 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920530+0000", 2024-09-16T20:48:36.105 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8111e-05 2024-09-16T20:48:36.105 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.105 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:36.106 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.106 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6420 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3602432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:36.106 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871796+0000", 2024-09-16T20:48:36.106 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 16.208862258, 2024-09-16T20:48:36.106 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048857298, 2024-09-16T20:48:36.106 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:36.106 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:36.106 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:36.106 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:36.106 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:36.106 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:36.106 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6420 2024-09-16T20:48:36.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.107 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:36.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.107 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:36.107 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871796+0000", 2024-09-16T20:48:36.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.107 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:36.107 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871796+0000", 2024-09-16T20:48:36.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:36.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871854+0000", 2024-09-16T20:48:36.108 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8187999999999999e-05 2024-09-16T20:48:36.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:36.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871856+0000", 2024-09-16T20:48:36.108 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3549999999999999e-06 2024-09-16T20:48:36.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:36.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871866+0000", 2024-09-16T20:48:36.108 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6500000000000008e-06 2024-09-16T20:48:36.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.109 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.109 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:36.109 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871872+0000", 2024-09-16T20:48:36.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3799999999999999e-06 2024-09-16T20:48:36.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.109 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.109 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:36.109 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920558+0000", 2024-09-16T20:48:36.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048685797000000003 2024-09-16T20:48:36.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.109 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.109 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:36.109 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920603+0000", 2024-09-16T20:48:36.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4629999999999998e-05 2024-09-16T20:48:36.110 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.110 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.110 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:36.110 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920653+0000", 2024-09-16T20:48:36.110 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0297999999999999e-05 2024-09-16T20:48:36.110 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.110 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:36.110 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.110 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.110 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.110 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6421 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3610624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:36.110 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871882+0000", 2024-09-16T20:48:36.110 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 16.208776189999998, 2024-09-16T20:48:36.111 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048913381999999998, 2024-09-16T20:48:36.111 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:36.111 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:36.111 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:36.111 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:36.111 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:36.111 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:36.111 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6421 2024-09-16T20:48:36.111 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.111 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:36.111 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.111 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:36.111 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871882+0000", 2024-09-16T20:48:36.112 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.112 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:36.112 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871882+0000", 2024-09-16T20:48:36.112 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.112 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:36.112 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.873257+0000", 2024-09-16T20:48:36.112 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001374789 2024-09-16T20:48:36.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.112 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:36.113 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.873261+0000", 2024-09-16T20:48:36.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5589999999999999e-06 2024-09-16T20:48:36.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.113 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.113 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:36.113 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.873275+0000", 2024-09-16T20:48:36.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3675999999999999e-05 2024-09-16T20:48:36.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.113 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.113 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:36.113 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.873281+0000", 2024-09-16T20:48:36.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6930000000000004e-06 2024-09-16T20:48:36.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.114 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.114 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:36.114 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920684+0000", 2024-09-16T20:48:36.114 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047402913999999997 2024-09-16T20:48:36.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.114 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.114 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:36.114 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920737+0000", 2024-09-16T20:48:36.114 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3600000000000002e-05 2024-09-16T20:48:36.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.114 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.114 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:36.114 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920795+0000", 2024-09-16T20:48:36.114 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8150999999999999e-05 2024-09-16T20:48:36.115 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.115 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:36.115 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.115 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.115 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:7491 3.1e 3:7c48f6ae:::rbd_object_map.110e891a9b2b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:36.115 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.925865+0000", 2024-09-16T20:48:36.115 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 16.154793046000002, 2024-09-16T20:48:36.115 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048897293000000001, 2024-09-16T20:48:36.115 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:36.115 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:36.115 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:36.115 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:36.116 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:36.116 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:36.116 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 7491 2024-09-16T20:48:36.116 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.116 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:36.116 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.116 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:36.116 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925865+0000", 2024-09-16T20:48:36.116 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.116 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.116 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.116 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:36.116 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925865+0000", 2024-09-16T20:48:36.117 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.117 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:36.117 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925866+0000", 2024-09-16T20:48:36.117 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.39e-07 2024-09-16T20:48:36.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.117 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:36.117 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925868+0000", 2024-09-16T20:48:36.117 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.508e-06 2024-09-16T20:48:36.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.117 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:36.118 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925874+0000", 2024-09-16T20:48:36.118 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7919999999999998e-06 2024-09-16T20:48:36.118 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.118 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.118 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:36.118 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925879+0000", 2024-09-16T20:48:36.118 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8579999999999999e-06 2024-09-16T20:48:36.118 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.118 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.118 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:36.118 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.970875+0000", 2024-09-16T20:48:36.118 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044996412 2024-09-16T20:48:36.118 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.119 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.119 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:36.119 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.970970+0000", 2024-09-16T20:48:36.119 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4732e-05 2024-09-16T20:48:36.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.119 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.119 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-16T20:48:36.119 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.971275+0000", 2024-09-16T20:48:36.119 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00030464199999999999 2024-09-16T20:48:36.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.119 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.119 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:36.119 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.974598+0000", 2024-09-16T20:48:36.120 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0033238159999999998 2024-09-16T20:48:36.120 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.120 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.120 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:48:36.120 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.974666+0000", 2024-09-16T20:48:36.120 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7617e-05 2024-09-16T20:48:36.120 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.120 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.120 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:48:36.120 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.974718+0000", 2024-09-16T20:48:36.120 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2259000000000002e-05 2024-09-16T20:48:36.120 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.120 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.121 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:36.121 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.974762+0000", 2024-09-16T20:48:36.121 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4218000000000003e-05 2024-09-16T20:48:36.121 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.121 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:36.121 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.121 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.121 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.121 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4750 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1488896~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:36.121 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689052+0000", 2024-09-16T20:48:36.121 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 13.391606507000001, 2024-09-16T20:48:36.121 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048208359999999999, 2024-09-16T20:48:36.121 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:36.121 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:36.122 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:36.122 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:36.122 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:36.122 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:36.122 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4750 2024-09-16T20:48:36.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.122 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:36.122 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.122 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:36.122 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689052+0000", 2024-09-16T20:48:36.122 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.122 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.122 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:36.123 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689052+0000", 2024-09-16T20:48:36.123 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.123 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.123 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.123 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:36.123 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689052+0000", 2024-09-16T20:48:36.123 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.65e-07 2024-09-16T20:48:36.123 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.123 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.123 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:36.123 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689058+0000", 2024-09-16T20:48:36.123 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.835e-06 2024-09-16T20:48:36.123 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.124 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.124 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:36.124 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689064+0000", 2024-09-16T20:48:36.124 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2729999999999999e-06 2024-09-16T20:48:36.124 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.124 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.124 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:36.124 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689069+0000", 2024-09-16T20:48:36.124 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1220000000000003e-06 2024-09-16T20:48:36.124 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.124 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.124 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:36.124 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737161+0000", 2024-09-16T20:48:36.124 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048091776000000003 2024-09-16T20:48:36.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.125 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.125 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:36.125 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737210+0000", 2024-09-16T20:48:36.125 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8575000000000002e-05 2024-09-16T20:48:36.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.125 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.125 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:36.125 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737260+0000", 2024-09-16T20:48:36.125 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0414000000000002e-05 2024-09-16T20:48:36.125 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.125 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:36.125 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.126 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.126 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4752 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1497088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:36.126 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689076+0000", 2024-09-16T20:48:36.126 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 13.391582365, 2024-09-16T20:48:36.126 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048322324999999999, 2024-09-16T20:48:36.126 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:36.126 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:36.126 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:36.126 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:36.126 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:36.126 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:36.126 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4752 2024-09-16T20:48:36.126 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.127 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:36.127 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.127 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:36.127 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689076+0000", 2024-09-16T20:48:36.127 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.127 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.127 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.127 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:36.127 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689076+0000", 2024-09-16T20:48:36.127 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.127 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.127 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.127 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:36.127 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689076+0000", 2024-09-16T20:48:36.128 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8000000000000001e-07 2024-09-16T20:48:36.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.128 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.128 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:36.128 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689078+0000", 2024-09-16T20:48:36.128 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8369999999999999e-06 2024-09-16T20:48:36.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.128 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.128 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:36.128 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689084+0000", 2024-09-16T20:48:36.128 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1319999999999999e-06 2024-09-16T20:48:36.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.128 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.129 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:36.129 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689090+0000", 2024-09-16T20:48:36.129 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0430000000000004e-06 2024-09-16T20:48:36.129 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.129 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.129 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:36.129 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737286+0000", 2024-09-16T20:48:36.129 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048195670000000003 2024-09-16T20:48:36.129 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.129 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.129 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:36.129 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737337+0000", 2024-09-16T20:48:36.129 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1511000000000002e-05 2024-09-16T20:48:36.130 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.130 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.130 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:36.130 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737398+0000", 2024-09-16T20:48:36.130 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0751999999999997e-05 2024-09-16T20:48:36.130 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.130 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:36.130 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.130 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.130 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.130 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4754 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1505280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:36.130 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689097+0000", 2024-09-16T20:48:36.130 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 13.391560847999999, 2024-09-16T20:48:36.130 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048429947000000001, 2024-09-16T20:48:36.131 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:36.131 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:36.131 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:36.131 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:36.131 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:36.131 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:36.131 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4754 2024-09-16T20:48:36.131 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.131 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:36.131 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.131 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:36.131 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689097+0000", 2024-09-16T20:48:36.131 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.132 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.132 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.132 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:36.132 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689097+0000", 2024-09-16T20:48:36.132 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.132 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.132 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.132 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:36.132 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689098+0000", 2024-09-16T20:48:36.132 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.15e-07 2024-09-16T20:48:36.132 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.132 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.132 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:36.132 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689100+0000", 2024-09-16T20:48:36.133 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8190000000000001e-06 2024-09-16T20:48:36.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.133 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:36.133 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689106+0000", 2024-09-16T20:48:36.133 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2489999999999998e-06 2024-09-16T20:48:36.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.133 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:36.133 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689111+0000", 2024-09-16T20:48:36.133 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3179999999999997e-06 2024-09-16T20:48:36.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.134 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:36.134 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737428+0000", 2024-09-16T20:48:36.134 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048316572000000002 2024-09-16T20:48:36.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.134 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.134 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:36.134 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737475+0000", 2024-09-16T20:48:36.134 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7392000000000003e-05 2024-09-16T20:48:36.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.134 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.134 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:36.134 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737527+0000", 2024-09-16T20:48:36.134 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2182000000000003e-05 2024-09-16T20:48:36.135 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.135 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:36.135 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.135 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.135 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4756 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1513472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:36.135 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689118+0000", 2024-09-16T20:48:36.135 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 13.391540626999999, 2024-09-16T20:48:36.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048535146000000001, 2024-09-16T20:48:36.135 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:36.135 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:36.135 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:36.135 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:36.136 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:36.136 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:36.136 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4756 2024-09-16T20:48:36.136 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.136 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:36.136 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.136 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:36.136 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689118+0000", 2024-09-16T20:48:36.136 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.136 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.136 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.136 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:36.136 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689118+0000", 2024-09-16T20:48:36.137 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.137 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.137 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:36.137 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689118+0000", 2024-09-16T20:48:36.137 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7099999999999997e-07 2024-09-16T20:48:36.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.137 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.137 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:36.137 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689120+0000", 2024-09-16T20:48:36.137 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.787e-06 2024-09-16T20:48:36.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.137 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.138 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:36.138 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689126+0000", 2024-09-16T20:48:36.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2720000000000003e-06 2024-09-16T20:48:36.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.138 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:36.138 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689134+0000", 2024-09-16T20:48:36.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2190000000000004e-06 2024-09-16T20:48:36.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.138 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:36.138 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737553+0000", 2024-09-16T20:48:36.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048418993 2024-09-16T20:48:36.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.139 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.139 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:36.139 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737598+0000", 2024-09-16T20:48:36.139 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4743999999999997e-05 2024-09-16T20:48:36.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.139 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.139 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:36.139 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737653+0000", 2024-09-16T20:48:36.139 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4759999999999997e-05 2024-09-16T20:48:36.139 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.139 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:36.139 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.139 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.140 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4758 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1521664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:36.140 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689141+0000", 2024-09-16T20:48:36.140 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 13.391517647000001, 2024-09-16T20:48:36.140 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048640741000000001, 2024-09-16T20:48:36.140 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:36.140 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:36.140 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:36.140 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:36.140 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:36.140 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:36.140 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4758 2024-09-16T20:48:36.140 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.140 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:36.141 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.141 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:36.141 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689141+0000", 2024-09-16T20:48:36.141 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.141 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.141 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:36.141 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689141+0000", 2024-09-16T20:48:36.141 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.141 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.141 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:36.141 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689141+0000", 2024-09-16T20:48:36.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8799999999999998e-07 2024-09-16T20:48:36.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.142 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.142 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:36.142 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689143+0000", 2024-09-16T20:48:36.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.945e-06 2024-09-16T20:48:36.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.142 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.142 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:36.142 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689149+0000", 2024-09-16T20:48:36.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9800000000000003e-06 2024-09-16T20:48:36.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.142 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.142 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:36.143 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689155+0000", 2024-09-16T20:48:36.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7509999999999997e-06 2024-09-16T20:48:36.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.143 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:36.143 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737681+0000", 2024-09-16T20:48:36.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048526825000000003 2024-09-16T20:48:36.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.143 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:36.143 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737729+0000", 2024-09-16T20:48:36.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7219999999999999e-05 2024-09-16T20:48:36.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.144 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.144 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:36.144 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737781+0000", 2024-09-16T20:48:36.144 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2632e-05 2024-09-16T20:48:36.144 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.144 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:36.144 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.144 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.144 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.144 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4760 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1529856~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:36.144 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689161+0000", 2024-09-16T20:48:36.144 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 13.391497299999999, 2024-09-16T20:48:36.145 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048743981999999998, 2024-09-16T20:48:36.145 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:36.145 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:36.145 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:36.145 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:36.145 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:36.145 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:36.145 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4760 2024-09-16T20:48:36.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.145 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:36.145 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.145 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:36.145 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689161+0000", 2024-09-16T20:48:36.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.146 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:36.146 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689161+0000", 2024-09-16T20:48:36.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.146 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:36.146 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689161+0000", 2024-09-16T20:48:36.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5599999999999995e-07 2024-09-16T20:48:36.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.146 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:36.147 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689163+0000", 2024-09-16T20:48:36.147 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.818e-06 2024-09-16T20:48:36.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.147 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.147 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:36.147 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689169+0000", 2024-09-16T20:48:36.147 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0809999999999999e-06 2024-09-16T20:48:36.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.147 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.147 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:36.147 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689174+0000", 2024-09-16T20:48:36.147 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.938e-06 2024-09-16T20:48:36.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.147 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.148 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:36.148 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737807+0000", 2024-09-16T20:48:36.148 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048633124 2024-09-16T20:48:36.148 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.148 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.148 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:36.148 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737851+0000", 2024-09-16T20:48:36.148 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3764000000000003e-05 2024-09-16T20:48:36.148 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.148 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.148 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:36.148 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737905+0000", 2024-09-16T20:48:36.148 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3701e-05 2024-09-16T20:48:36.149 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.149 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:36.149 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.149 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.149 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4762 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1538048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:36.149 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689182+0000", 2024-09-16T20:48:36.149 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 13.391476078, 2024-09-16T20:48:36.149 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048850275999999998, 2024-09-16T20:48:36.149 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:36.149 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:36.149 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:36.149 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:36.149 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:36.150 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:36.150 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4762 2024-09-16T20:48:36.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.150 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:36.150 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.150 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:36.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689182+0000", 2024-09-16T20:48:36.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.150 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.150 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:36.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689182+0000", 2024-09-16T20:48:36.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:36.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689182+0000", 2024-09-16T20:48:36.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-16T20:48:36.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:36.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689184+0000", 2024-09-16T20:48:36.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8190000000000001e-06 2024-09-16T20:48:36.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:36.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689191+0000", 2024-09-16T20:48:36.152 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2530000000000001e-06 2024-09-16T20:48:36.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.152 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.152 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:36.152 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689195+0000", 2024-09-16T20:48:36.152 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9559999999999996e-06 2024-09-16T20:48:36.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.152 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.152 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:36.152 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737933+0000", 2024-09-16T20:48:36.152 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048737919999999997 2024-09-16T20:48:36.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.152 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.153 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:36.153 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737982+0000", 2024-09-16T20:48:36.153 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8089e-05 2024-09-16T20:48:36.153 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.153 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.153 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:36.153 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738032+0000", 2024-09-16T20:48:36.153 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0868999999999997e-05 2024-09-16T20:48:36.153 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.153 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:36.153 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.153 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.153 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.153 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4764 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1546240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:36.154 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689201+0000", 2024-09-16T20:48:36.154 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 13.391456721000001, 2024-09-16T20:48:36.154 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048957550000000002, 2024-09-16T20:48:36.154 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:36.154 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:36.154 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:36.154 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:36.154 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:36.154 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:36.154 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4764 2024-09-16T20:48:36.154 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.154 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:36.154 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.155 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:36.155 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689201+0000", 2024-09-16T20:48:36.155 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.155 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.155 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:36.155 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689201+0000", 2024-09-16T20:48:36.155 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.155 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.155 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:36.155 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689202+0000", 2024-09-16T20:48:36.155 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4200000000000002e-07 2024-09-16T20:48:36.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.156 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.156 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:36.156 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689204+0000", 2024-09-16T20:48:36.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9750000000000001e-06 2024-09-16T20:48:36.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.156 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.156 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:36.156 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689210+0000", 2024-09-16T20:48:36.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1e-06 2024-09-16T20:48:36.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.156 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.156 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:36.157 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689215+0000", 2024-09-16T20:48:36.157 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0180000000000001e-06 2024-09-16T20:48:36.157 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.157 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.157 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:36.157 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738058+0000", 2024-09-16T20:48:36.157 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048843573000000001 2024-09-16T20:48:36.157 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.157 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.157 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:36.157 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738102+0000", 2024-09-16T20:48:36.157 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3256000000000001e-05 2024-09-16T20:48:36.157 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.157 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.158 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:36.158 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738159+0000", 2024-09-16T20:48:36.158 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7286e-05 2024-09-16T20:48:36.158 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.158 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:36.158 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.158 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.158 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.158 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4766 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1554432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:36.158 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689221+0000", 2024-09-16T20:48:36.158 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 13.391436755000001, 2024-09-16T20:48:36.158 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049064996, 2024-09-16T20:48:36.158 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:36.159 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:36.159 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:36.159 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:36.159 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:36.159 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:36.159 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4766 2024-09-16T20:48:36.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.159 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:36.159 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.159 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:36.159 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689221+0000", 2024-09-16T20:48:36.159 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.159 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.160 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:36.160 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689221+0000", 2024-09-16T20:48:36.160 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.160 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.160 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:36.160 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689222+0000", 2024-09-16T20:48:36.160 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.34e-07 2024-09-16T20:48:36.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.160 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.160 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:36.160 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689224+0000", 2024-09-16T20:48:36.161 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.846e-06 2024-09-16T20:48:36.161 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.161 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.161 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:36.161 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689247+0000", 2024-09-16T20:48:36.161 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3448e-05 2024-09-16T20:48:36.161 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.161 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.161 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:36.161 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689254+0000", 2024-09-16T20:48:36.161 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6969999999999996e-06 2024-09-16T20:48:36.161 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.161 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.162 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:36.162 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738188+0000", 2024-09-16T20:48:36.162 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048934520000000002 2024-09-16T20:48:36.162 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.162 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.162 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:36.162 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738234+0000", 2024-09-16T20:48:36.162 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5695999999999998e-05 2024-09-16T20:48:36.162 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.162 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.162 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:36.162 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738286+0000", 2024-09-16T20:48:36.162 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2355000000000003e-05 2024-09-16T20:48:36.163 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.163 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:36.163 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.163 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.163 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4768 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1562624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:36.163 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689262+0000", 2024-09-16T20:48:36.163 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 13.391395801, 2024-09-16T20:48:36.163 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049157157999999999, 2024-09-16T20:48:36.163 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:36.163 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:36.163 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:36.163 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:36.164 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:36.164 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:36.164 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4768 2024-09-16T20:48:36.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.164 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:36.164 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.164 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:36.164 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689262+0000", 2024-09-16T20:48:36.164 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.164 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.164 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:36.165 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689262+0000", 2024-09-16T20:48:36.165 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.165 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.165 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.165 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:36.165 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689263+0000", 2024-09-16T20:48:36.165 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6199999999999998e-07 2024-09-16T20:48:36.165 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.165 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.165 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:36.165 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689270+0000", 2024-09-16T20:48:36.165 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.798e-06 2024-09-16T20:48:36.165 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.165 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.166 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:36.166 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689277+0000", 2024-09-16T20:48:36.166 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0890000000000002e-06 2024-09-16T20:48:36.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.166 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.166 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:36.166 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689282+0000", 2024-09-16T20:48:36.166 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0470000000000001e-06 2024-09-16T20:48:36.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.166 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.166 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:36.166 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738319+0000", 2024-09-16T20:48:36.166 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049037139 2024-09-16T20:48:36.167 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.167 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.167 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:36.167 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738365+0000", 2024-09-16T20:48:36.167 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6356999999999997e-05 2024-09-16T20:48:36.167 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.167 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.167 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:36.167 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738420+0000", 2024-09-16T20:48:36.167 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4265999999999998e-05 2024-09-16T20:48:36.167 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.167 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:36.167 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.168 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.168 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4770 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1570816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:36.168 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689289+0000", 2024-09-16T20:48:36.168 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 13.391369291, 2024-09-16T20:48:36.168 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049253705000000002, 2024-09-16T20:48:36.168 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:36.168 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:36.168 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:36.168 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:36.168 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:36.168 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:36.168 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4770 2024-09-16T20:48:36.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.169 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:36.169 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.169 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:36.169 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689289+0000", 2024-09-16T20:48:36.169 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.169 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.169 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:36.169 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689289+0000", 2024-09-16T20:48:36.169 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.169 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.169 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:36.170 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689289+0000", 2024-09-16T20:48:36.170 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0999999999999999e-07 2024-09-16T20:48:36.170 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.170 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.170 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:36.170 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689291+0000", 2024-09-16T20:48:36.170 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0030000000000001e-06 2024-09-16T20:48:36.170 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.170 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.170 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:36.170 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689297+0000", 2024-09-16T20:48:36.170 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9479999999999996e-06 2024-09-16T20:48:36.171 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.171 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.171 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:36.171 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689302+0000", 2024-09-16T20:48:36.171 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7389999999999999e-06 2024-09-16T20:48:36.171 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.171 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.171 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:36.171 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738442+0000", 2024-09-16T20:48:36.171 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049140001000000003 2024-09-16T20:48:36.171 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.171 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.171 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:36.171 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738490+0000", 2024-09-16T20:48:36.172 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7787000000000002e-05 2024-09-16T20:48:36.172 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.172 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.172 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:36.172 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738543+0000", 2024-09-16T20:48:36.172 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2816999999999999e-05 2024-09-16T20:48:36.172 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.172 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:36.172 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.172 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.172 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.172 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:3088 3.1e 3:79e5ec01:::rbd_header.110e891a9b2b:head [watch watch cookie 140213437171488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:48:36.172 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:48:36.173 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 11.916813274000001, 2024-09-16T20:48:36.173 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.11895186200000001, 2024-09-16T20:48:36.173 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:36.173 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:36.173 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:36.173 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:36.173 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:48:36.173 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:48:36.173 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3088 2024-09-16T20:48:36.173 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.173 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:36.173 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.173 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:36.174 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:48:36.174 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.174 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.174 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:36.174 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:48:36.174 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.174 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.174 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:36.174 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163846+0000", 2024-09-16T20:48:36.174 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6799999999999999e-07 2024-09-16T20:48:36.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.175 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.175 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:36.175 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163850+0000", 2024-09-16T20:48:36.175 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.929e-06 2024-09-16T20:48:36.175 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.175 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.175 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:36.175 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163862+0000", 2024-09-16T20:48:36.175 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1972e-05 2024-09-16T20:48:36.175 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.175 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.175 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:36.175 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163867+0000", 2024-09-16T20:48:36.176 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.451e-06 2024-09-16T20:48:36.176 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.176 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.176 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:36.176 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163912+0000", 2024-09-16T20:48:36.176 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4456000000000003e-05 2024-09-16T20:48:36.176 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.176 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.176 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:36.176 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163962+0000", 2024-09-16T20:48:36.176 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0578000000000003e-05 2024-09-16T20:48:36.176 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.176 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.177 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-16T20:48:36.177 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164058+0000", 2024-09-16T20:48:36.177 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5934999999999995e-05 2024-09-16T20:48:36.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.177 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.177 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:48:36.177 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164570+0000", 2024-09-16T20:48:36.177 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00051241100000000005 2024-09-16T20:48:36.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.177 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.178 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:36.178 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282629+0000", 2024-09-16T20:48:36.178 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.118058986 2024-09-16T20:48:36.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.178 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.178 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:48:36.178 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282725+0000", 2024-09-16T20:48:36.178 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5063999999999996e-05 2024-09-16T20:48:36.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.178 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.178 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:36.178 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282797+0000", 2024-09-16T20:48:36.178 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2212000000000006e-05 2024-09-16T20:48:36.179 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.179 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:36.179 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.179 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.179 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:36.179 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:36.181 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:48:36.181 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:48:36.182 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:36.182 INFO:teuthology.orchestra.run.smithi029.stdout: "epoch": 48, 2024-09-16T20:48:36.182 INFO:teuthology.orchestra.run.smithi029.stdout: "fsid": "e28af4a7-5fcd-4de8-8f77-a2b575361bb1", 2024-09-16T20:48:36.182 INFO:teuthology.orchestra.run.smithi029.stdout: "created": "2024-09-16T20:47:39.760609+0000", 2024-09-16T20:48:36.182 INFO:teuthology.orchestra.run.smithi029.stdout: "modified": "2024-09-16T20:48:35.658110+0000", 2024-09-16T20:48:36.182 INFO:teuthology.orchestra.run.smithi029.stdout: "last_up_change": "2024-09-16T20:47:50.238013+0000", 2024-09-16T20:48:36.182 INFO:teuthology.orchestra.run.smithi029.stdout: "last_in_change": "2024-09-16T20:48:25.619359+0000", 2024-09-16T20:48:36.182 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-16T20:48:36.182 INFO:teuthology.orchestra.run.smithi029.stdout: "flags_num": 5806080, 2024-09-16T20:48:36.182 INFO:teuthology.orchestra.run.smithi029.stdout: "flags_set": [ 2024-09-16T20:48:36.182 INFO:teuthology.orchestra.run.smithi029.stdout: "nodeep-scrub", 2024-09-16T20:48:36.182 INFO:teuthology.orchestra.run.smithi029.stdout: "noscrub", 2024-09-16T20:48:36.182 INFO:teuthology.orchestra.run.smithi029.stdout: "pglog_hardlimit", 2024-09-16T20:48:36.183 INFO:teuthology.orchestra.run.smithi029.stdout: "purged_snapdirs", 2024-09-16T20:48:36.183 INFO:teuthology.orchestra.run.smithi029.stdout: "recovery_deletes", 2024-09-16T20:48:36.183 INFO:teuthology.orchestra.run.smithi029.stdout: "sortbitwise" 2024-09-16T20:48:36.183 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-16T20:48:36.183 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_version": 5, 2024-09-16T20:48:36.183 INFO:teuthology.orchestra.run.smithi029.stdout: "full_ratio": 0.94999998807907104, 2024-09-16T20:48:36.183 INFO:teuthology.orchestra.run.smithi029.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-16T20:48:36.183 INFO:teuthology.orchestra.run.smithi029.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-16T20:48:36.183 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_snapshot": "", 2024-09-16T20:48:36.183 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_max": 3, 2024-09-16T20:48:36.183 INFO:teuthology.orchestra.run.smithi029.stdout: "max_osd": 6, 2024-09-16T20:48:36.183 INFO:teuthology.orchestra.run.smithi029.stdout: "require_min_compat_client": "luminous", 2024-09-16T20:48:36.183 INFO:teuthology.orchestra.run.smithi029.stdout: "min_compat_client": "luminous", 2024-09-16T20:48:36.184 INFO:teuthology.orchestra.run.smithi029.stdout: "require_osd_release": "squid", 2024-09-16T20:48:36.184 INFO:teuthology.orchestra.run.smithi029.stdout: "allow_crimson": false, 2024-09-16T20:48:36.184 INFO:teuthology.orchestra.run.smithi029.stdout: "pools": [ 2024-09-16T20:48:36.184 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.184 INFO:teuthology.orchestra.run.smithi029.stdout: "pool": 1, 2024-09-16T20:48:36.184 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_name": ".mgr", 2024-09-16T20:48:36.184 INFO:teuthology.orchestra.run.smithi029.stdout: "create_time": "2024-09-16T20:47:52.220750+0000", 2024-09-16T20:48:36.184 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-16T20:48:36.184 INFO:teuthology.orchestra.run.smithi029.stdout: "flags_names": "hashpspool", 2024-09-16T20:48:36.184 INFO:teuthology.orchestra.run.smithi029.stdout: "type": 1, 2024-09-16T20:48:36.184 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 2, 2024-09-16T20:48:36.184 INFO:teuthology.orchestra.run.smithi029.stdout: "min_size": 1, 2024-09-16T20:48:36.184 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_rule": 0, 2024-09-16T20:48:36.185 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_count": 0, 2024-09-16T20:48:36.185 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_target": 0, 2024-09-16T20:48:36.185 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T20:48:36.185 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T20:48:36.185 INFO:teuthology.orchestra.run.smithi029.stdout: "is_stretch_pool": false, 2024-09-16T20:48:36.185 INFO:teuthology.orchestra.run.smithi029.stdout: "object_hash": 2, 2024-09-16T20:48:36.185 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_autoscale_mode": "off", 2024-09-16T20:48:36.185 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num": 1, 2024-09-16T20:48:36.185 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num": 1, 2024-09-16T20:48:36.185 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num_target": 1, 2024-09-16T20:48:36.185 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_target": 1, 2024-09-16T20:48:36.185 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_pending": 1, 2024-09-16T20:48:36.185 INFO:teuthology.orchestra.run.smithi029.stdout: "last_pg_merge_meta": { 2024-09-16T20:48:36.185 INFO:teuthology.orchestra.run.smithi029.stdout: "source_pgid": "0.0", 2024-09-16T20:48:36.186 INFO:teuthology.orchestra.run.smithi029.stdout: "ready_epoch": 0, 2024-09-16T20:48:36.186 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_started": 0, 2024-09-16T20:48:36.186 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_clean": 0, 2024-09-16T20:48:36.186 INFO:teuthology.orchestra.run.smithi029.stdout: "source_version": "0'0", 2024-09-16T20:48:36.186 INFO:teuthology.orchestra.run.smithi029.stdout: "target_version": "0'0" 2024-09-16T20:48:36.186 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.186 INFO:teuthology.orchestra.run.smithi029.stdout: "last_change": "17", 2024-09-16T20:48:36.186 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend": "0", 2024-09-16T20:48:36.186 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T20:48:36.186 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T20:48:36.186 INFO:teuthology.orchestra.run.smithi029.stdout: "auid": 0, 2024-09-16T20:48:36.186 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_mode": "selfmanaged", 2024-09-16T20:48:36.186 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_seq": 0, 2024-09-16T20:48:36.187 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_epoch": 0, 2024-09-16T20:48:36.187 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_snaps": [], 2024-09-16T20:48:36.187 INFO:teuthology.orchestra.run.smithi029.stdout: "removed_snaps": "[]", 2024-09-16T20:48:36.187 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_bytes": 0, 2024-09-16T20:48:36.187 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_objects": 0, 2024-09-16T20:48:36.187 INFO:teuthology.orchestra.run.smithi029.stdout: "tiers": [], 2024-09-16T20:48:36.187 INFO:teuthology.orchestra.run.smithi029.stdout: "tier_of": -1, 2024-09-16T20:48:36.187 INFO:teuthology.orchestra.run.smithi029.stdout: "read_tier": -1, 2024-09-16T20:48:36.187 INFO:teuthology.orchestra.run.smithi029.stdout: "write_tier": -1, 2024-09-16T20:48:36.187 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_mode": "none", 2024-09-16T20:48:36.187 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_bytes": 0, 2024-09-16T20:48:36.187 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_objects": 0, 2024-09-16T20:48:36.187 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T20:48:36.187 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T20:48:36.188 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T20:48:36.188 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_flush_age": 0, 2024-09-16T20:48:36.188 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_evict_age": 0, 2024-09-16T20:48:36.188 INFO:teuthology.orchestra.run.smithi029.stdout: "erasure_code_profile": "", 2024-09-16T20:48:36.188 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_params": { 2024-09-16T20:48:36.188 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "none" 2024-09-16T20:48:36.188 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.188 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_period": 0, 2024-09-16T20:48:36.188 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_count": 0, 2024-09-16T20:48:36.188 INFO:teuthology.orchestra.run.smithi029.stdout: "use_gmt_hitset": true, 2024-09-16T20:48:36.188 INFO:teuthology.orchestra.run.smithi029.stdout: "min_read_recency_for_promote": 0, 2024-09-16T20:48:36.188 INFO:teuthology.orchestra.run.smithi029.stdout: "min_write_recency_for_promote": 0, 2024-09-16T20:48:36.188 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T20:48:36.189 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_search_last_n": 0, 2024-09-16T20:48:36.189 INFO:teuthology.orchestra.run.smithi029.stdout: "grade_table": [], 2024-09-16T20:48:36.189 INFO:teuthology.orchestra.run.smithi029.stdout: "stripe_width": 0, 2024-09-16T20:48:36.189 INFO:teuthology.orchestra.run.smithi029.stdout: "expected_num_objects": 0, 2024-09-16T20:48:36.189 INFO:teuthology.orchestra.run.smithi029.stdout: "fast_read": false, 2024-09-16T20:48:36.189 INFO:teuthology.orchestra.run.smithi029.stdout: "options": { 2024-09-16T20:48:36.189 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_max": 32, 2024-09-16T20:48:36.189 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_min": 1 2024-09-16T20:48:36.189 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.189 INFO:teuthology.orchestra.run.smithi029.stdout: "application_metadata": { 2024-09-16T20:48:36.189 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": {} 2024-09-16T20:48:36.189 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.189 INFO:teuthology.orchestra.run.smithi029.stdout: "read_balance": { 2024-09-16T20:48:36.189 INFO:teuthology.orchestra.run.smithi029.stdout: "score_type": "Fair distribution", 2024-09-16T20:48:36.190 INFO:teuthology.orchestra.run.smithi029.stdout: "score_acting": 6.059999942779541, 2024-09-16T20:48:36.190 INFO:teuthology.orchestra.run.smithi029.stdout: "score_stable": 6.059999942779541, 2024-09-16T20:48:36.190 INFO:teuthology.orchestra.run.smithi029.stdout: "optimal_score": 0.33000001311302185, 2024-09-16T20:48:36.190 INFO:teuthology.orchestra.run.smithi029.stdout: "raw_score_acting": 2, 2024-09-16T20:48:36.190 INFO:teuthology.orchestra.run.smithi029.stdout: "raw_score_stable": 2, 2024-09-16T20:48:36.190 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity_weighted": 1, 2024-09-16T20:48:36.190 INFO:teuthology.orchestra.run.smithi029.stdout: "average_primary_affinity": 1, 2024-09-16T20:48:36.190 INFO:teuthology.orchestra.run.smithi029.stdout: "average_primary_affinity_weighted": 1 2024-09-16T20:48:36.190 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:36.190 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.190 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.190 INFO:teuthology.orchestra.run.smithi029.stdout: "pool": 2, 2024-09-16T20:48:36.190 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_name": "rbd", 2024-09-16T20:48:36.191 INFO:teuthology.orchestra.run.smithi029.stdout: "create_time": "2024-09-16T20:47:56.756666+0000", 2024-09-16T20:48:36.191 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 8193, 2024-09-16T20:48:36.191 INFO:teuthology.orchestra.run.smithi029.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T20:48:36.191 INFO:teuthology.orchestra.run.smithi029.stdout: "type": 1, 2024-09-16T20:48:36.191 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 2, 2024-09-16T20:48:36.191 INFO:teuthology.orchestra.run.smithi029.stdout: "min_size": 1, 2024-09-16T20:48:36.191 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_rule": 0, 2024-09-16T20:48:36.191 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_count": 0, 2024-09-16T20:48:36.191 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_target": 0, 2024-09-16T20:48:36.191 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T20:48:36.191 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T20:48:36.191 INFO:teuthology.orchestra.run.smithi029.stdout: "is_stretch_pool": false, 2024-09-16T20:48:36.191 INFO:teuthology.orchestra.run.smithi029.stdout: "object_hash": 2, 2024-09-16T20:48:36.191 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_autoscale_mode": "off", 2024-09-16T20:48:36.192 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num": 8, 2024-09-16T20:48:36.192 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num": 8, 2024-09-16T20:48:36.192 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num_target": 8, 2024-09-16T20:48:36.192 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_target": 8, 2024-09-16T20:48:36.192 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_pending": 8, 2024-09-16T20:48:36.192 INFO:teuthology.orchestra.run.smithi029.stdout: "last_pg_merge_meta": { 2024-09-16T20:48:36.192 INFO:teuthology.orchestra.run.smithi029.stdout: "source_pgid": "0.0", 2024-09-16T20:48:36.192 INFO:teuthology.orchestra.run.smithi029.stdout: "ready_epoch": 0, 2024-09-16T20:48:36.192 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_started": 0, 2024-09-16T20:48:36.192 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_clean": 0, 2024-09-16T20:48:36.192 INFO:teuthology.orchestra.run.smithi029.stdout: "source_version": "0'0", 2024-09-16T20:48:36.192 INFO:teuthology.orchestra.run.smithi029.stdout: "target_version": "0'0" 2024-09-16T20:48:36.192 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.193 INFO:teuthology.orchestra.run.smithi029.stdout: "last_change": "21", 2024-09-16T20:48:36.193 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend": "0", 2024-09-16T20:48:36.193 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T20:48:36.193 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T20:48:36.193 INFO:teuthology.orchestra.run.smithi029.stdout: "auid": 0, 2024-09-16T20:48:36.193 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_mode": "selfmanaged", 2024-09-16T20:48:36.193 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_seq": 2, 2024-09-16T20:48:36.193 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_epoch": 21, 2024-09-16T20:48:36.193 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_snaps": [], 2024-09-16T20:48:36.193 INFO:teuthology.orchestra.run.smithi029.stdout: "removed_snaps": "[]", 2024-09-16T20:48:36.193 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_bytes": 0, 2024-09-16T20:48:36.193 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_objects": 0, 2024-09-16T20:48:36.193 INFO:teuthology.orchestra.run.smithi029.stdout: "tiers": [], 2024-09-16T20:48:36.193 INFO:teuthology.orchestra.run.smithi029.stdout: "tier_of": -1, 2024-09-16T20:48:36.194 INFO:teuthology.orchestra.run.smithi029.stdout: "read_tier": -1, 2024-09-16T20:48:36.194 INFO:teuthology.orchestra.run.smithi029.stdout: "write_tier": -1, 2024-09-16T20:48:36.194 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_mode": "none", 2024-09-16T20:48:36.194 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_bytes": 0, 2024-09-16T20:48:36.194 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_objects": 0, 2024-09-16T20:48:36.194 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T20:48:36.194 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T20:48:36.194 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T20:48:36.194 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_flush_age": 0, 2024-09-16T20:48:36.194 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_evict_age": 0, 2024-09-16T20:48:36.194 INFO:teuthology.orchestra.run.smithi029.stdout: "erasure_code_profile": "", 2024-09-16T20:48:36.194 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_params": { 2024-09-16T20:48:36.194 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "none" 2024-09-16T20:48:36.195 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.195 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_period": 0, 2024-09-16T20:48:36.195 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_count": 0, 2024-09-16T20:48:36.195 INFO:teuthology.orchestra.run.smithi029.stdout: "use_gmt_hitset": true, 2024-09-16T20:48:36.195 INFO:teuthology.orchestra.run.smithi029.stdout: "min_read_recency_for_promote": 0, 2024-09-16T20:48:36.195 INFO:teuthology.orchestra.run.smithi029.stdout: "min_write_recency_for_promote": 0, 2024-09-16T20:48:36.195 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T20:48:36.195 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_search_last_n": 0, 2024-09-16T20:48:36.195 INFO:teuthology.orchestra.run.smithi029.stdout: "grade_table": [], 2024-09-16T20:48:36.195 INFO:teuthology.orchestra.run.smithi029.stdout: "stripe_width": 0, 2024-09-16T20:48:36.195 INFO:teuthology.orchestra.run.smithi029.stdout: "expected_num_objects": 0, 2024-09-16T20:48:36.195 INFO:teuthology.orchestra.run.smithi029.stdout: "fast_read": false, 2024-09-16T20:48:36.195 INFO:teuthology.orchestra.run.smithi029.stdout: "options": {}, 2024-09-16T20:48:36.195 INFO:teuthology.orchestra.run.smithi029.stdout: "application_metadata": { 2024-09-16T20:48:36.196 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd": {} 2024-09-16T20:48:36.196 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.196 INFO:teuthology.orchestra.run.smithi029.stdout: "read_balance": { 2024-09-16T20:48:36.196 INFO:teuthology.orchestra.run.smithi029.stdout: "score_type": "Fair distribution", 2024-09-16T20:48:36.196 INFO:teuthology.orchestra.run.smithi029.stdout: "score_acting": 2.2699999809265137, 2024-09-16T20:48:36.196 INFO:teuthology.orchestra.run.smithi029.stdout: "score_stable": 2.2699999809265137, 2024-09-16T20:48:36.196 INFO:teuthology.orchestra.run.smithi029.stdout: "optimal_score": 0.82999998331069946, 2024-09-16T20:48:36.196 INFO:teuthology.orchestra.run.smithi029.stdout: "raw_score_acting": 1.8799999952316284, 2024-09-16T20:48:36.196 INFO:teuthology.orchestra.run.smithi029.stdout: "raw_score_stable": 1.8799999952316284, 2024-09-16T20:48:36.196 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity_weighted": 1, 2024-09-16T20:48:36.196 INFO:teuthology.orchestra.run.smithi029.stdout: "average_primary_affinity": 1, 2024-09-16T20:48:36.196 INFO:teuthology.orchestra.run.smithi029.stdout: "average_primary_affinity_weighted": 1 2024-09-16T20:48:36.196 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:36.196 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.197 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.197 INFO:teuthology.orchestra.run.smithi029.stdout: "pool": 3, 2024-09-16T20:48:36.197 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_name": "pool_client.0", 2024-09-16T20:48:36.197 INFO:teuthology.orchestra.run.smithi029.stdout: "create_time": "2024-09-16T20:48:06.418889+0000", 2024-09-16T20:48:36.197 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 8193, 2024-09-16T20:48:36.197 INFO:teuthology.orchestra.run.smithi029.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T20:48:36.197 INFO:teuthology.orchestra.run.smithi029.stdout: "type": 1, 2024-09-16T20:48:36.197 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 2, 2024-09-16T20:48:36.197 INFO:teuthology.orchestra.run.smithi029.stdout: "min_size": 1, 2024-09-16T20:48:36.197 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_rule": 0, 2024-09-16T20:48:36.197 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_count": 0, 2024-09-16T20:48:36.197 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_target": 0, 2024-09-16T20:48:36.197 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T20:48:36.197 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T20:48:36.198 INFO:teuthology.orchestra.run.smithi029.stdout: "is_stretch_pool": false, 2024-09-16T20:48:36.198 INFO:teuthology.orchestra.run.smithi029.stdout: "object_hash": 2, 2024-09-16T20:48:36.198 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_autoscale_mode": "off", 2024-09-16T20:48:36.198 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num": 32, 2024-09-16T20:48:36.198 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num": 32, 2024-09-16T20:48:36.198 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num_target": 32, 2024-09-16T20:48:36.198 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_target": 32, 2024-09-16T20:48:36.198 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_pending": 32, 2024-09-16T20:48:36.198 INFO:teuthology.orchestra.run.smithi029.stdout: "last_pg_merge_meta": { 2024-09-16T20:48:36.198 INFO:teuthology.orchestra.run.smithi029.stdout: "source_pgid": "0.0", 2024-09-16T20:48:36.198 INFO:teuthology.orchestra.run.smithi029.stdout: "ready_epoch": 0, 2024-09-16T20:48:36.198 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_started": 0, 2024-09-16T20:48:36.198 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_clean": 0, 2024-09-16T20:48:36.198 INFO:teuthology.orchestra.run.smithi029.stdout: "source_version": "0'0", 2024-09-16T20:48:36.199 INFO:teuthology.orchestra.run.smithi029.stdout: "target_version": "0'0" 2024-09-16T20:48:36.199 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.199 INFO:teuthology.orchestra.run.smithi029.stdout: "last_change": "48", 2024-09-16T20:48:36.199 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend": "0", 2024-09-16T20:48:36.199 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T20:48:36.199 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T20:48:36.199 INFO:teuthology.orchestra.run.smithi029.stdout: "auid": 0, 2024-09-16T20:48:36.199 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_mode": "selfmanaged", 2024-09-16T20:48:36.199 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_seq": 13, 2024-09-16T20:48:36.199 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_epoch": 48, 2024-09-16T20:48:36.199 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_snaps": [], 2024-09-16T20:48:36.199 INFO:teuthology.orchestra.run.smithi029.stdout: "removed_snaps": "[]", 2024-09-16T20:48:36.199 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_bytes": 0, 2024-09-16T20:48:36.199 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_objects": 0, 2024-09-16T20:48:36.200 INFO:teuthology.orchestra.run.smithi029.stdout: "tiers": [], 2024-09-16T20:48:36.200 INFO:teuthology.orchestra.run.smithi029.stdout: "tier_of": -1, 2024-09-16T20:48:36.200 INFO:teuthology.orchestra.run.smithi029.stdout: "read_tier": -1, 2024-09-16T20:48:36.200 INFO:teuthology.orchestra.run.smithi029.stdout: "write_tier": -1, 2024-09-16T20:48:36.200 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_mode": "none", 2024-09-16T20:48:36.200 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_bytes": 0, 2024-09-16T20:48:36.200 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_objects": 0, 2024-09-16T20:48:36.200 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T20:48:36.200 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T20:48:36.200 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T20:48:36.200 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_flush_age": 0, 2024-09-16T20:48:36.200 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_evict_age": 0, 2024-09-16T20:48:36.201 INFO:teuthology.orchestra.run.smithi029.stdout: "erasure_code_profile": "", 2024-09-16T20:48:36.201 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_params": { 2024-09-16T20:48:36.201 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "none" 2024-09-16T20:48:36.201 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.201 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_period": 0, 2024-09-16T20:48:36.201 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_count": 0, 2024-09-16T20:48:36.201 INFO:teuthology.orchestra.run.smithi029.stdout: "use_gmt_hitset": true, 2024-09-16T20:48:36.201 INFO:teuthology.orchestra.run.smithi029.stdout: "min_read_recency_for_promote": 0, 2024-09-16T20:48:36.201 INFO:teuthology.orchestra.run.smithi029.stdout: "min_write_recency_for_promote": 0, 2024-09-16T20:48:36.201 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T20:48:36.201 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_search_last_n": 0, 2024-09-16T20:48:36.201 INFO:teuthology.orchestra.run.smithi029.stdout: "grade_table": [], 2024-09-16T20:48:36.201 INFO:teuthology.orchestra.run.smithi029.stdout: "stripe_width": 0, 2024-09-16T20:48:36.201 INFO:teuthology.orchestra.run.smithi029.stdout: "expected_num_objects": 0, 2024-09-16T20:48:36.202 INFO:teuthology.orchestra.run.smithi029.stdout: "fast_read": false, 2024-09-16T20:48:36.202 INFO:teuthology.orchestra.run.smithi029.stdout: "options": {}, 2024-09-16T20:48:36.202 INFO:teuthology.orchestra.run.smithi029.stdout: "application_metadata": { 2024-09-16T20:48:36.202 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd": {} 2024-09-16T20:48:36.202 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.202 INFO:teuthology.orchestra.run.smithi029.stdout: "read_balance": { 2024-09-16T20:48:36.202 INFO:teuthology.orchestra.run.smithi029.stdout: "score_type": "Fair distribution", 2024-09-16T20:48:36.202 INFO:teuthology.orchestra.run.smithi029.stdout: "score_acting": 1.5099999904632568, 2024-09-16T20:48:36.202 INFO:teuthology.orchestra.run.smithi029.stdout: "score_stable": 1.5099999904632568, 2024-09-16T20:48:36.202 INFO:teuthology.orchestra.run.smithi029.stdout: "optimal_score": 0.82999998331069946, 2024-09-16T20:48:36.202 INFO:teuthology.orchestra.run.smithi029.stdout: "raw_score_acting": 1.25, 2024-09-16T20:48:36.202 INFO:teuthology.orchestra.run.smithi029.stdout: "raw_score_stable": 1.25, 2024-09-16T20:48:36.202 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity_weighted": 1, 2024-09-16T20:48:36.203 INFO:teuthology.orchestra.run.smithi029.stdout: "average_primary_affinity": 1, 2024-09-16T20:48:36.203 INFO:teuthology.orchestra.run.smithi029.stdout: "average_primary_affinity_weighted": 1 2024-09-16T20:48:36.203 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:36.203 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:36.203 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-16T20:48:36.203 INFO:teuthology.orchestra.run.smithi029.stdout: "osds": [ 2024-09-16T20:48:36.203 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.203 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 0, 2024-09-16T20:48:36.203 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "6c3cb2d7-7d23-4566-9ca1-d7b08e12360c", 2024-09-16T20:48:36.203 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-16T20:48:36.203 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-16T20:48:36.203 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-16T20:48:36.203 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-16T20:48:36.203 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-16T20:48:36.204 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-16T20:48:36.204 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 13, 2024-09-16T20:48:36.204 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 42, 2024-09-16T20:48:36.204 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-16T20:48:36.204 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-16T20:48:36.204 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-16T20:48:36.204 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:48:36.204 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.204 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:48:36.204 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6816", 2024-09-16T20:48:36.204 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:48:36.204 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.204 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.205 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:48:36.205 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6817", 2024-09-16T20:48:36.205 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:48:36.205 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:36.205 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:36.205 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.205 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-16T20:48:36.205 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:48:36.205 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.205 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:48:36.205 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6818", 2024-09-16T20:48:36.205 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:48:36.205 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.206 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.206 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:48:36.206 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6819", 2024-09-16T20:48:36.206 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:48:36.206 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:36.206 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:36.206 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.206 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-16T20:48:36.206 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:48:36.206 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.206 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:48:36.206 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6822", 2024-09-16T20:48:36.206 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:48:36.206 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.207 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.207 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:48:36.207 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6823", 2024-09-16T20:48:36.207 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:48:36.207 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:36.207 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:36.207 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.207 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-16T20:48:36.207 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:48:36.207 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.207 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:48:36.207 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6820", 2024-09-16T20:48:36.207 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:48:36.207 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.208 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.208 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:48:36.208 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6821", 2024-09-16T20:48:36.208 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:48:36.208 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:36.208 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:36.208 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.208 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.29:6817/2744025949", 2024-09-16T20:48:36.208 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.29:6819/2744025949", 2024-09-16T20:48:36.208 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.29:6823/2744025949", 2024-09-16T20:48:36.208 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.29:6821/2744025949", 2024-09-16T20:48:36.208 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-16T20:48:36.208 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-16T20:48:36.209 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-16T20:48:36.209 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:36.209 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.209 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.209 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 1, 2024-09-16T20:48:36.209 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "5cf8c20c-1ed2-42ae-a8b0-c88cc4e36bce", 2024-09-16T20:48:36.209 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-16T20:48:36.209 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-16T20:48:36.209 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-16T20:48:36.209 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-16T20:48:36.209 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-16T20:48:36.209 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-16T20:48:36.209 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 13, 2024-09-16T20:48:36.210 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 23, 2024-09-16T20:48:36.210 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-16T20:48:36.210 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-16T20:48:36.210 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-16T20:48:36.210 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:48:36.210 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.210 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:48:36.210 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6800", 2024-09-16T20:48:36.210 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:48:36.210 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.210 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.210 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:48:36.210 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6801", 2024-09-16T20:48:36.210 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:48:36.211 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:36.211 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:36.211 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.211 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-16T20:48:36.211 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:48:36.211 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.211 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:48:36.211 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6802", 2024-09-16T20:48:36.211 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:48:36.211 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.211 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.211 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:48:36.211 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6803", 2024-09-16T20:48:36.211 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:48:36.212 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:36.212 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:36.212 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.212 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-16T20:48:36.212 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:48:36.212 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.212 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:48:36.212 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6806", 2024-09-16T20:48:36.212 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:48:36.212 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.212 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.212 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:48:36.212 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6807", 2024-09-16T20:48:36.213 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:48:36.213 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:36.213 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:36.213 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.213 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-16T20:48:36.213 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:48:36.213 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.213 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:48:36.213 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6804", 2024-09-16T20:48:36.213 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:48:36.213 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.213 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.213 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:48:36.213 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6805", 2024-09-16T20:48:36.214 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:48:36.214 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:36.214 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:36.214 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.214 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.29:6801/208628762", 2024-09-16T20:48:36.214 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.29:6803/208628762", 2024-09-16T20:48:36.214 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.29:6807/208628762", 2024-09-16T20:48:36.214 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.29:6805/208628762", 2024-09-16T20:48:36.214 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-16T20:48:36.214 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-16T20:48:36.214 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-16T20:48:36.214 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:36.214 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.214 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.215 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 2, 2024-09-16T20:48:36.215 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "075d16da-82bf-449b-9f6e-542c9e8d810b", 2024-09-16T20:48:36.215 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-16T20:48:36.215 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-16T20:48:36.215 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-16T20:48:36.215 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-16T20:48:36.215 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-16T20:48:36.215 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-16T20:48:36.215 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 13, 2024-09-16T20:48:36.215 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 42, 2024-09-16T20:48:36.215 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-16T20:48:36.215 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-16T20:48:36.215 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-16T20:48:36.215 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:48:36.216 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.216 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:48:36.216 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6808", 2024-09-16T20:48:36.216 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:48:36.216 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.216 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.216 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:48:36.216 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6809", 2024-09-16T20:48:36.216 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:48:36.216 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:36.216 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:36.216 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.216 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-16T20:48:36.216 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:48:36.217 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.217 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:48:36.217 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6810", 2024-09-16T20:48:36.217 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:48:36.217 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.217 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.217 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:48:36.217 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6811", 2024-09-16T20:48:36.217 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:48:36.217 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:36.217 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:36.217 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.218 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-16T20:48:36.218 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:48:36.218 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.218 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:48:36.218 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6814", 2024-09-16T20:48:36.218 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:48:36.218 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.218 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.218 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:48:36.218 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6815", 2024-09-16T20:48:36.218 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:48:36.218 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:36.218 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:36.218 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.219 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-16T20:48:36.219 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:48:36.219 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.219 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:48:36.219 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6812", 2024-09-16T20:48:36.219 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:48:36.219 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.219 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.219 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:48:36.219 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6813", 2024-09-16T20:48:36.219 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:48:36.219 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:36.219 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:36.219 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.220 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.29:6809/3447426536", 2024-09-16T20:48:36.220 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.29:6811/3447426536", 2024-09-16T20:48:36.220 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.29:6815/3447426536", 2024-09-16T20:48:36.220 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.29:6813/3447426536", 2024-09-16T20:48:36.220 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-16T20:48:36.220 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-16T20:48:36.220 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-16T20:48:36.220 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:36.220 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.220 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.220 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 3, 2024-09-16T20:48:36.220 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "0612be38-3a24-41b6-aba4-5a2fe985165e", 2024-09-16T20:48:36.220 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-16T20:48:36.221 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 0, 2024-09-16T20:48:36.221 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 0, 2024-09-16T20:48:36.221 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-16T20:48:36.221 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-16T20:48:36.221 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-16T20:48:36.221 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 13, 2024-09-16T20:48:36.221 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 23, 2024-09-16T20:48:36.221 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-16T20:48:36.221 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-16T20:48:36.221 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-16T20:48:36.221 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:48:36.221 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.221 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:48:36.222 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6808", 2024-09-16T20:48:36.222 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:48:36.222 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.222 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.222 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:48:36.222 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6809", 2024-09-16T20:48:36.222 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:48:36.222 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:36.222 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:36.222 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.222 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-16T20:48:36.222 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:48:36.222 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.222 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:48:36.223 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6810", 2024-09-16T20:48:36.223 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:48:36.223 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.223 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.223 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:48:36.223 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6811", 2024-09-16T20:48:36.223 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:48:36.223 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:36.223 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:36.223 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.223 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-16T20:48:36.223 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:48:36.223 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.224 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:48:36.224 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6814", 2024-09-16T20:48:36.224 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:48:36.224 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.224 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.224 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:48:36.224 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6815", 2024-09-16T20:48:36.224 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:48:36.224 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:36.224 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:36.224 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.224 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-16T20:48:36.224 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:48:36.225 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.225 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:48:36.225 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6812", 2024-09-16T20:48:36.225 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:48:36.225 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.225 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.225 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:48:36.225 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6813", 2024-09-16T20:48:36.225 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:48:36.225 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:36.225 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:36.225 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.225 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.43:6809/3687583891", 2024-09-16T20:48:36.226 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.43:6811/3687583891", 2024-09-16T20:48:36.226 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.43:6815/3687583891", 2024-09-16T20:48:36.226 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.43:6813/3687583891", 2024-09-16T20:48:36.226 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-16T20:48:36.226 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-16T20:48:36.226 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-16T20:48:36.226 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:36.226 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.226 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.226 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 4, 2024-09-16T20:48:36.226 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "be590bf8-e9fa-44ca-a471-6f6bcb087ca7", 2024-09-16T20:48:36.226 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-16T20:48:36.226 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-16T20:48:36.226 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-16T20:48:36.227 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-16T20:48:36.227 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-16T20:48:36.227 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-16T20:48:36.227 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 11, 2024-09-16T20:48:36.227 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 39, 2024-09-16T20:48:36.227 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-16T20:48:36.227 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-16T20:48:36.227 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-16T20:48:36.227 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:48:36.227 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.227 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:48:36.227 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6800", 2024-09-16T20:48:36.227 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:48:36.228 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.228 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.228 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:48:36.228 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6801", 2024-09-16T20:48:36.228 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:48:36.228 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:36.228 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:36.228 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.228 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-16T20:48:36.228 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:48:36.228 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.228 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:48:36.228 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6802", 2024-09-16T20:48:36.229 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:48:36.229 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.229 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.229 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:48:36.229 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6803", 2024-09-16T20:48:36.229 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:48:36.229 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:36.229 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:36.229 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.229 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-16T20:48:36.229 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:48:36.229 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.229 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:48:36.229 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6806", 2024-09-16T20:48:36.230 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:48:36.230 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.230 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.230 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:48:36.230 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6807", 2024-09-16T20:48:36.230 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:48:36.230 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:36.230 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:36.230 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.230 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-16T20:48:36.230 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:48:36.230 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.230 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:48:36.230 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6804", 2024-09-16T20:48:36.231 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:48:36.231 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.231 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.231 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:48:36.231 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6805", 2024-09-16T20:48:36.231 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:48:36.231 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:36.231 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:36.231 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.231 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.43:6801/2505220646", 2024-09-16T20:48:36.231 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.43:6803/2505220646", 2024-09-16T20:48:36.231 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.43:6807/2505220646", 2024-09-16T20:48:36.231 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.43:6805/2505220646", 2024-09-16T20:48:36.232 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-16T20:48:36.232 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-16T20:48:36.232 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-16T20:48:36.232 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:36.232 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.232 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.232 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 5, 2024-09-16T20:48:36.232 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "4a208e30-031b-4a9f-af52-e26d4a039ea6", 2024-09-16T20:48:36.232 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-16T20:48:36.232 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-16T20:48:36.232 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-16T20:48:36.232 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-16T20:48:36.232 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-16T20:48:36.233 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-16T20:48:36.233 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 13, 2024-09-16T20:48:36.233 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 39, 2024-09-16T20:48:36.233 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-16T20:48:36.233 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-16T20:48:36.233 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-16T20:48:36.233 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:48:36.233 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.233 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:48:36.233 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6816", 2024-09-16T20:48:36.233 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:48:36.233 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.233 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.233 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:48:36.234 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6817", 2024-09-16T20:48:36.234 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:48:36.234 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:36.234 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:36.234 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.234 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-16T20:48:36.234 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:48:36.234 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.234 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:48:36.234 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6818", 2024-09-16T20:48:36.234 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:48:36.234 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.234 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.234 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:48:36.234 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6819", 2024-09-16T20:48:36.235 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:48:36.235 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:36.235 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:36.235 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.235 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-16T20:48:36.235 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:48:36.235 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.235 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:48:36.235 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6822", 2024-09-16T20:48:36.235 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:48:36.235 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.235 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.235 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:48:36.236 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6823", 2024-09-16T20:48:36.236 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:48:36.236 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:36.236 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:36.236 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.236 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-16T20:48:36.236 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:48:36.236 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.236 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:48:36.236 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6820", 2024-09-16T20:48:36.236 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:48:36.236 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.236 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.236 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:48:36.237 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6821", 2024-09-16T20:48:36.237 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:48:36.237 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:36.237 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:36.237 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.237 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.43:6817/1517367307", 2024-09-16T20:48:36.237 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.43:6819/1517367307", 2024-09-16T20:48:36.237 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.43:6823/1517367307", 2024-09-16T20:48:36.237 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.43:6821/1517367307", 2024-09-16T20:48:36.237 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-16T20:48:36.237 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-16T20:48:36.237 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-16T20:48:36.237 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:36.238 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:36.238 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-16T20:48:36.238 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_xinfo": [ 2024-09-16T20:48:36.238 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.238 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 0, 2024-09-16T20:48:36.238 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-16T20:48:36.238 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-16T20:48:36.238 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-16T20:48:36.238 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540719139924082687, 2024-09-16T20:48:36.238 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-16T20:48:36.238 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-16T20:47:47.887412+0000", 2024-09-16T20:48:36.238 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-16T20:48:36.238 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.239 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.239 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 1, 2024-09-16T20:48:36.239 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-16T20:48:36.239 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-16T20:48:36.239 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-16T20:48:36.239 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540719139924082687, 2024-09-16T20:48:36.239 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-16T20:48:36.239 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-16T20:47:47.658782+0000", 2024-09-16T20:48:36.239 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-16T20:48:36.239 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.239 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.239 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 2, 2024-09-16T20:48:36.239 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-16T20:48:36.239 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-16T20:48:36.240 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-16T20:48:36.240 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540719139924082687, 2024-09-16T20:48:36.240 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-16T20:48:36.240 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-16T20:47:47.453752+0000", 2024-09-16T20:48:36.240 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-16T20:48:36.240 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.240 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.240 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 3, 2024-09-16T20:48:36.240 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-16T20:48:36.240 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-16T20:48:36.240 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-16T20:48:36.240 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540719139924082687, 2024-09-16T20:48:36.240 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 65536, 2024-09-16T20:48:36.240 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-16T20:47:47.487992+0000", 2024-09-16T20:48:36.241 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-16T20:48:36.241 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.241 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.241 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 4, 2024-09-16T20:48:36.241 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-16T20:48:36.241 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-16T20:48:36.241 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-16T20:48:36.241 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540719139924082687, 2024-09-16T20:48:36.241 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-16T20:48:36.241 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-16T20:47:46.943344+0000", 2024-09-16T20:48:36.241 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-16T20:48:36.241 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.241 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.242 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 5, 2024-09-16T20:48:36.242 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-16T20:48:36.242 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-16T20:48:36.242 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-16T20:48:36.242 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540719139924082687, 2024-09-16T20:48:36.242 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-16T20:48:36.242 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-16T20:47:47.901547+0000", 2024-09-16T20:48:36.242 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-16T20:48:36.242 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:36.242 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-16T20:48:36.244 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_upmap": [], 2024-09-16T20:48:36.244 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_upmap_items": [ 2024-09-16T20:48:36.244 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.244 INFO:teuthology.orchestra.run.smithi029.stdout: "pgid": "2.4", 2024-09-16T20:48:36.244 INFO:teuthology.orchestra.run.smithi029.stdout: "mappings": [ 2024-09-16T20:48:36.244 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:36.244 INFO:teuthology.orchestra.run.smithi029.stdout: "from": 5, 2024-09-16T20:48:36.245 INFO:teuthology.orchestra.run.smithi029.stdout: "to": 2 2024-09-16T20:48:36.245 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:36.245 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:36.245 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:36.245 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-16T20:48:36.245 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_upmap_primaries": [], 2024-09-16T20:48:36.245 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_temp": [], 2024-09-16T20:48:36.245 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_temp": [], 2024-09-16T20:48:36.245 INFO:teuthology.orchestra.run.smithi029.stdout: "blocklist": {}, 2024-09-16T20:48:36.245 INFO:teuthology.orchestra.run.smithi029.stdout: "range_blocklist": {}, 2024-09-16T20:48:36.245 INFO:teuthology.orchestra.run.smithi029.stdout: "erasure_code_profiles": { 2024-09-16T20:48:36.245 INFO:teuthology.orchestra.run.smithi029.stdout: "default": { 2024-09-16T20:48:36.245 INFO:teuthology.orchestra.run.smithi029.stdout: "crush-failure-domain": "osd", 2024-09-16T20:48:36.245 INFO:teuthology.orchestra.run.smithi029.stdout: "k": "2", 2024-09-16T20:48:36.246 INFO:teuthology.orchestra.run.smithi029.stdout: "m": "1", 2024-09-16T20:48:36.246 INFO:teuthology.orchestra.run.smithi029.stdout: "plugin": "jerasure", 2024-09-16T20:48:36.246 INFO:teuthology.orchestra.run.smithi029.stdout: "technique": "reed_sol_van" 2024-09-16T20:48:36.246 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:36.246 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:36.246 INFO:teuthology.orchestra.run.smithi029.stdout: "removed_snaps_queue": [], 2024-09-16T20:48:36.246 INFO:teuthology.orchestra.run.smithi029.stdout: "new_removed_snaps": [], 2024-09-16T20:48:36.246 INFO:teuthology.orchestra.run.smithi029.stdout: "new_purged_snaps": [], 2024-09-16T20:48:36.246 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_node_flags": {}, 2024-09-16T20:48:36.246 INFO:teuthology.orchestra.run.smithi029.stdout: "device_class_flags": {}, 2024-09-16T20:48:36.246 INFO:teuthology.orchestra.run.smithi029.stdout: "stretch_mode": { 2024-09-16T20:48:36.246 INFO:teuthology.orchestra.run.smithi029.stdout: "stretch_mode_enabled": false, 2024-09-16T20:48:36.246 INFO:teuthology.orchestra.run.smithi029.stdout: "stretch_bucket_count": 0, 2024-09-16T20:48:36.246 INFO:teuthology.orchestra.run.smithi029.stdout: "degraded_stretch_mode": 0, 2024-09-16T20:48:36.247 INFO:teuthology.orchestra.run.smithi029.stdout: "recovering_stretch_mode": 0, 2024-09-16T20:48:36.247 INFO:teuthology.orchestra.run.smithi029.stdout: "stretch_mode_bucket": 0 2024-09-16T20:48:36.247 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:36.247 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:36.248 INFO:tasks.thrashosds.thrasher:j is {'epoch': 48, 'fsid': 'e28af4a7-5fcd-4de8-8f77-a2b575361bb1', 'created': '2024-09-16T20:47:39.760609+0000', 'modified': '2024-09-16T20:48:35.658110+0000', 'last_up_change': '2024-09-16T20:47:50.238013+0000', 'last_in_change': '2024-09-16T20:48:25.619359+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T20:47:52.220750+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-16T20:47:56.756666+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.2699999809265137, 'score_stable': 2.2699999809265137, 'optimal_score': 0.8299999833106995, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-16T20:48:06.418889+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '48', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 13, 'snap_epoch': 48, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5099999904632568, 'score_stable': 1.5099999904632568, 'optimal_score': 0.8299999833106995, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '6c3cb2d7-7d23-4566-9ca1-d7b08e12360c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 42, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6816', 'nonce': 2744025949}, {'type': 'v1', 'addr': '172.21.15.29:6817', 'nonce': 2744025949}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6818', 'nonce': 2744025949}, {'type': 'v1', 'addr': '172.21.15.29:6819', 'nonce': 2744025949}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6822', 'nonce': 2744025949}, {'type': 'v1', 'addr': '172.21.15.29:6823', 'nonce': 2744025949}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6820', 'nonce': 2744025949}, {'type': 'v1', 'addr': '172.21.15.29:6821', 'nonce': 2744025949}]}, 'public_addr': '172.21.15.29:6817/2744025949', 'cluster_addr': '172.21.15.29:6819/2744025949', 'heartbeat_back_addr': '172.21.15.29:6823/2744025949', 'heartbeat_front_addr': '172.21.15.29:6821/2744025949', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '5cf8c20c-1ed2-42ae-a8b0-c88cc4e36bce', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6800', 'nonce': 208628762}, {'type': 'v1', 'addr': '172.21.15.29:6801', 'nonce': 208628762}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6802', 'nonce': 208628762}, {'type': 'v1', 'addr': '172.21.15.29:6803', 'nonce': 208628762}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6806', 'nonce': 208628762}, {'type': 'v1', 'addr': '172.21.15.29:6807', 'nonce': 208628762}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6804', 'nonce': 208628762}, {'type': 'v1', 'addr': '172.21.15.29:6805', 'nonce': 208628762}]}, 'public_addr': '172.21.15.29:6801/208628762', 'cluster_addr': '172.21.15.29:6803/208628762', 'heartbeat_back_addr': '172.21.15.29:6807/208628762', 'heartbeat_front_addr': '172.21.15.29:6805/208628762', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '075d16da-82bf-449b-9f6e-542c9e8d810b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 42, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6808', 'nonce': 3447426536}, {'type': 'v1', 'addr': '172.21.15.29:6809', 'nonce': 3447426536}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6810', 'nonce': 3447426536}, {'type': 'v1', 'addr': '172.21.15.29:6811', 'nonce': 3447426536}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6814', 'nonce': 3447426536}, {'type': 'v1', 'addr': '172.21.15.29:6815', 'nonce': 3447426536}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6812', 'nonce': 3447426536}, {'type': 'v1', 'addr': '172.21.15.29:6813', 'nonce': 3447426536}]}, 'public_addr': '172.21.15.29:6809/3447426536', 'cluster_addr': '172.21.15.29:6811/3447426536', 'heartbeat_back_addr': '172.21.15.29:6815/3447426536', 'heartbeat_front_addr': '172.21.15.29:6813/3447426536', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '0612be38-3a24-41b6-aba4-5a2fe985165e', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6808', 'nonce': 3687583891}, {'type': 'v1', 'addr': '172.21.15.43:6809', 'nonce': 3687583891}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6810', 'nonce': 3687583891}, {'type': 'v1', 'addr': '172.21.15.43:6811', 'nonce': 3687583891}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6814', 'nonce': 3687583891}, {'type': 'v1', 'addr': '172.21.15.43:6815', 'nonce': 3687583891}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6812', 'nonce': 3687583891}, {'type': 'v1', 'addr': '172.21.15.43:6813', 'nonce': 3687583891}]}, 'public_addr': '172.21.15.43:6809/3687583891', 'cluster_addr': '172.21.15.43:6811/3687583891', 'heartbeat_back_addr': '172.21.15.43:6815/3687583891', 'heartbeat_front_addr': '172.21.15.43:6813/3687583891', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'be590bf8-e9fa-44ca-a471-6f6bcb087ca7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 39, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6800', 'nonce': 2505220646}, {'type': 'v1', 'addr': '172.21.15.43:6801', 'nonce': 2505220646}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6802', 'nonce': 2505220646}, {'type': 'v1', 'addr': '172.21.15.43:6803', 'nonce': 2505220646}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6806', 'nonce': 2505220646}, {'type': 'v1', 'addr': '172.21.15.43:6807', 'nonce': 2505220646}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6804', 'nonce': 2505220646}, {'type': 'v1', 'addr': '172.21.15.43:6805', 'nonce': 2505220646}]}, 'public_addr': '172.21.15.43:6801/2505220646', 'cluster_addr': '172.21.15.43:6803/2505220646', 'heartbeat_back_addr': '172.21.15.43:6807/2505220646', 'heartbeat_front_addr': '172.21.15.43:6805/2505220646', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '4a208e30-031b-4a9f-af52-e26d4a039ea6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 39, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6816', 'nonce': 1517367307}, {'type': 'v1', 'addr': '172.21.15.43:6817', 'nonce': 1517367307}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6818', 'nonce': 1517367307}, {'type': 'v1', 'addr': '172.21.15.43:6819', 'nonce': 1517367307}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6822', 'nonce': 1517367307}, {'type': 'v1', 'addr': '172.21.15.43:6823', 'nonce': 1517367307}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6820', 'nonce': 1517367307}, {'type': 'v1', 'addr': '172.21.15.43:6821', 'nonce': 1517367307}]}, 'public_addr': '172.21.15.43:6817/1517367307', 'cluster_addr': '172.21.15.43:6819/1517367307', 'heartbeat_back_addr': '172.21.15.43:6823/1517367307', 'heartbeat_front_addr': '172.21.15.43:6821/1517367307', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T20:47:47.887412+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T20:47:47.658782+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T20:47:47.453752+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-16T20:47:47.487992+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T20:47:46.943344+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T20:47:47.901547+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '2.4', 'mappings': [{'from': 5, 'to': 2}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-16T20:48:36.248 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T20:48:36.248 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:36.179+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:36.292 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:36.291+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:36.361 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:36.361 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:48:36.361 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:48:36.361 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:36.369 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:48:36.415 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:36.414+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:36.477 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:36.478 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:48:36.478 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:48:36.478 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:48:36.478 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:36.483 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:36.482+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:36.486 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:48:36.520 INFO:teuthology.orchestra.run.smithi139.stdout:1427 read 0x8c05e00 thru 0x8c12732 (0xc933 bytes) 2024-09-16T20:48:36.525 INFO:teuthology.orchestra.run.smithi139.stdout:1429 trunc from 0x9d86c00 to 0x6bd8000 2024-09-16T20:48:36.550 INFO:teuthology.orchestra.run.smithi139.stdout:1431 punch from 0x3696e00 to 0x36a6000, (0xf200 bytes) 2024-09-16T20:48:36.552 INFO:teuthology.orchestra.run.smithi139.stdout:1432 read 0xe66e00 thru 0xe6f2e3 (0x84e4 bytes) 2024-09-16T20:48:36.553 INFO:teuthology.orchestra.run.smithi139.stdout:1433 punch from 0x44e4e00 to 0x44f3c00, (0xee00 bytes) 2024-09-16T20:48:36.561 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:48:36.561 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-09-16T20:48:36.569 INFO:teuthology.orchestra.run.smithi139.stdout:1437 write 0xba6c800 thru 0xba6e749 (0x1f4a bytes) 2024-09-16T20:48:36.570 INFO:teuthology.orchestra.run.smithi139.stdout:1438 read 0x3d5a000 thru 0x3d5b03a (0x103b bytes) 2024-09-16T20:48:36.574 INFO:teuthology.orchestra.run.smithi139.stdout:1439 read 0xb514400 thru 0xb523855 (0xf456 bytes) 2024-09-16T20:48:36.575 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":67,"stamp":"2024-09-16T20:48:36.211285+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":419269634,"num_objects":357,"num_object_clones":114,"num_object_copies":714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":357,"num_whiteouts":82,"num_read":62870,"num_read_kb":222420,"num_write":3999,"num_write_kb":60328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":5605797,"num_keys_recovered":35,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3280,"ondisk_log_size":3280,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":267736,"kb_used_data":90864,"kb_used_omap":7,"kb_used_meta":176632,"kb_avail":471591464,"statfs":{"total":483183820800,"available":482909659136,"internally_reserved":0,"allocated":93044736,"data_stored":114875416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7974,"internal_metadata":180871386},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2554414,"num_objects":15,"num_object_clones":4,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":2,"num_read":3290,"num_read_kb":8230,"num_write":118,"num_write_kb":2546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.560763"},"pg_stats":[{"pgid":"3.1f","version":"44'82","reported_seq":1059,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-16T20:48:30.996151+0000","last_change":"2024-09-16T20:48:09.568824+0000","last_active":"2024-09-16T20:48:30.996151+0000","last_peered":"2024-09-16T20:48:30.996151+0000","last_clean":"2024-09-16T20:48:30.996151+0000","last_became_active":"2024-09-16T20:48:07.571756+0000","last_became_peered":"2024-09-16T20:48:07.571756+0000","last_unstale":"2024-09-16T20:48:30.996151+0000","last_undegraded":"2024-09-16T20:48:30.996151+0000","last_fullsized":"2024-09-16T20:48:30.996151+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:31:51.187746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071634499999999996,"stat_sum":{"num_bytes":14755344,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":1125,"num_read_kb":2419,"num_write":70,"num_write_kb":680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"43'323","reported_seq":8965,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-16T20:48:31.156806+0000","last_change":"2024-09-16T20:48:29.647639+0000","last_active":"2024-09-16T20:48:31.156806+0000","last_peered":"2024-09-16T20:48:31.156806+0000","last_clean":"2024-09-16T20:48:31.156806+0000","last_became_active":"2024-09-16T20:48:29.646967+0000","last_became_peered":"2024-09-16T20:48:29.646967+0000","last_unstale":"2024-09-16T20:48:31.156806+0000","last_undegraded":"2024-09-16T20:48:31.156806+0000","last_fullsized":"2024-09-16T20:48:31.156806+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:18:13.602874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084864999999999995,"stat_sum":{"num_bytes":24168496,"num_objects":13,"num_object_clones":5,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":2,"num_read":10159,"num_read_kb":24567,"num_write":460,"num_write_kb":1084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":350768,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"44'115","reported_seq":2051,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-16T20:48:31.177683+0000","last_change":"2024-09-16T20:48:09.564507+0000","last_active":"2024-09-16T20:48:31.177683+0000","last_peered":"2024-09-16T20:48:31.177683+0000","last_clean":"2024-09-16T20:48:31.177683+0000","last_became_active":"2024-09-16T20:48:07.568025+0000","last_became_peered":"2024-09-16T20:48:07.568025+0000","last_unstale":"2024-09-16T20:48:31.177683+0000","last_undegraded":"2024-09-16T20:48:31.177683+0000","last_fullsized":"2024-09-16T20:48:31.177683+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:42:31.566208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068985699999999999,"stat_sum":{"num_bytes":11815936,"num_objects":10,"num_object_clones":3,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":2,"num_read":2485,"num_read_kb":20089,"num_write":91,"num_write_kb":2902,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"44'50","reported_seq":418,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-16T20:48:31.217203+0000","last_change":"2024-09-16T20:48:09.565661+0000","last_active":"2024-09-16T20:48:31.217203+0000","last_peered":"2024-09-16T20:48:31.217203+0000","last_clean":"2024-09-16T20:48:31.217203+0000","last_became_active":"2024-09-16T20:48:07.565241+0000","last_became_peered":"2024-09-16T20:48:07.565241+0000","last_unstale":"2024-09-16T20:48:31.217203+0000","last_undegraded":"2024-09-16T20:48:31.217203+0000","last_fullsized":"2024-09-16T20:48:31.217203+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:08:38.954740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001045589,"stat_sum":{"num_bytes":4228096,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":338,"num_read_kb":2104,"num_write":54,"num_write_kb":2827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"44'221","reported_seq":514,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-16T20:48:30.648177+0000","last_change":"2024-09-16T20:48:29.649187+0000","last_active":"2024-09-16T20:48:30.648177+0000","last_peered":"2024-09-16T20:48:30.648177+0000","last_clean":"2024-09-16T20:48:30.648177+0000","last_became_active":"2024-09-16T20:48:29.648642+0000","last_became_peered":"2024-09-16T20:48:29.648642+0000","last_unstale":"2024-09-16T20:48:30.648177+0000","last_undegraded":"2024-09-16T20:48:30.648177+0000","last_fullsized":"2024-09-16T20:48:30.648177+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:15:17.005189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057432399999999997,"stat_sum":{"num_bytes":8535192,"num_objects":14,"num_object_clones":4,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":3,"num_read":1020,"num_read_kb":3719,"num_write":332,"num_write_kb":1533,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":63050,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"43'105","reported_seq":1937,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-16T20:48:31.171511+0000","last_change":"2024-09-16T20:48:09.566668+0000","last_active":"2024-09-16T20:48:31.171511+0000","last_peered":"2024-09-16T20:48:31.171511+0000","last_clean":"2024-09-16T20:48:31.171511+0000","last_became_active":"2024-09-16T20:48:07.566792+0000","last_became_peered":"2024-09-16T20:48:07.566792+0000","last_unstale":"2024-09-16T20:48:31.171511+0000","last_undegraded":"2024-09-16T20:48:31.171511+0000","last_fullsized":"2024-09-16T20:48:31.171511+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:26:01.149882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002037849,"stat_sum":{"num_bytes":1639484,"num_objects":9,"num_object_clones":2,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":2,"num_read":3189,"num_read_kb":2934,"num_write":69,"num_write_kb":324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"43'62","reported_seq":3001,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T20:48:32.657021+0000","last_change":"2024-09-16T20:48:26.701178+0000","last_active":"2024-09-16T20:48:32.657021+0000","last_peered":"2024-09-16T20:48:32.657021+0000","last_clean":"2024-09-16T20:48:32.657021+0000","last_became_active":"2024-09-16T20:48:26.640404+0000","last_became_peered":"2024-09-16T20:48:26.640404+0000","last_unstale":"2024-09-16T20:48:32.657021+0000","last_undegraded":"2024-09-16T20:48:32.657021+0000","last_fullsized":"2024-09-16T20:48:32.657021+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:29:18.172985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057190699999999997,"stat_sum":{"num_bytes":24775680,"num_objects":15,"num_object_clones":7,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":3,"num_read":2862,"num_read_kb":13430,"num_write":61,"num_write_kb":2755,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":2457600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"46'124","reported_seq":1950,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T20:48:33.123109+0000","last_change":"2024-09-16T20:48:26.654513+0000","last_active":"2024-09-16T20:48:33.123109+0000","last_peered":"2024-09-16T20:48:33.123109+0000","last_clean":"2024-09-16T20:48:33.123109+0000","last_became_active":"2024-09-16T20:48:26.638289+0000","last_became_peered":"2024-09-16T20:48:26.638289+0000","last_unstale":"2024-09-16T20:48:33.123109+0000","last_undegraded":"2024-09-16T20:48:33.123109+0000","last_fullsized":"2024-09-16T20:48:33.123109+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:29:22.064455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011061790000000001,"stat_sum":{"num_bytes":1179648,"num_objects":6,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2740,"num_read_kb":6264,"num_write":70,"num_write_kb":922,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":788480,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"43'86","reported_seq":1269,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-16T20:48:30.650553+0000","last_change":"2024-09-16T20:48:09.568656+0000","last_active":"2024-09-16T20:48:30.650553+0000","last_peered":"2024-09-16T20:48:30.650553+0000","last_clean":"2024-09-16T20:48:30.650553+0000","last_became_active":"2024-09-16T20:48:07.567576+0000","last_became_peered":"2024-09-16T20:48:07.567576+0000","last_unstale":"2024-09-16T20:48:30.650553+0000","last_undegraded":"2024-09-16T20:48:30.650553+0000","last_fullsized":"2024-09-16T20:48:30.650553+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:27:50.672429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057915500000000003,"stat_sum":{"num_bytes":1761280,"num_objects":6,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":1759,"num_read_kb":3852,"num_write":58,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"43'303","reported_seq":665,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-16T20:48:31.217048+0000","last_change":"2024-09-16T20:48:29.649028+0000","last_active":"2024-09-16T20:48:31.217048+0000","last_peered":"2024-09-16T20:48:31.217048+0000","last_clean":"2024-09-16T20:48:31.217048+0000","last_became_active":"2024-09-16T20:48:29.648169+0000","last_became_peered":"2024-09-16T20:48:29.648169+0000","last_unstale":"2024-09-16T20:48:31.217048+0000","last_undegraded":"2024-09-16T20:48:31.217048+0000","last_fullsized":"2024-09-16T20:48:31.217048+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:58:34.058126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010075049999999999,"stat_sum":{"num_bytes":5179466,"num_objects":10,"num_object_clones":2,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":1,"num_read":1636,"num_read_kb":3959,"num_write":512,"num_write_kb":3515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":84042,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"43'41","reported_seq":893,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-16T20:48:30.647841+0000","last_change":"2024-09-16T20:48:26.721401+0000","last_active":"2024-09-16T20:48:30.647841+0000","last_peered":"2024-09-16T20:48:30.647841+0000","last_clean":"2024-09-16T20:48:30.647841+0000","last_became_active":"2024-09-16T20:48:26.639185+0000","last_became_peered":"2024-09-16T20:48:26.639185+0000","last_unstale":"2024-09-16T20:48:30.647841+0000","last_undegraded":"2024-09-16T20:48:30.647841+0000","last_fullsized":"2024-09-16T20:48:30.647841+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:09:45.605786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051100699999999995,"stat_sum":{"num_bytes":6275072,"num_objects":10,"num_object_clones":4,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":3,"num_read":687,"num_read_kb":6584,"num_write":43,"num_write_kb":1704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1189888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"44'53","reported_seq":509,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-16T20:48:30.939643+0000","last_change":"2024-09-16T20:48:09.566005+0000","last_active":"2024-09-16T20:48:30.939643+0000","last_peered":"2024-09-16T20:48:30.939643+0000","last_clean":"2024-09-16T20:48:30.939643+0000","last_became_active":"2024-09-16T20:48:07.568236+0000","last_became_peered":"2024-09-16T20:48:07.568236+0000","last_unstale":"2024-09-16T20:48:30.939643+0000","last_undegraded":"2024-09-16T20:48:30.939643+0000","last_fullsized":"2024-09-16T20:48:30.939643+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:22:58.844564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001271749,"stat_sum":{"num_bytes":14450402,"num_objects":17,"num_object_clones":6,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":5,"num_read":420,"num_read_kb":8760,"num_write":178,"num_write_kb":1286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":66,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-16T20:48:30.648590+0000","last_change":"2024-09-16T20:48:00.277670+0000","last_active":"2024-09-16T20:48:30.648590+0000","last_peered":"2024-09-16T20:48:30.648590+0000","last_clean":"2024-09-16T20:48:30.648590+0000","last_became_active":"2024-09-16T20:47:58.274489+0000","last_became_peered":"2024-09-16T20:47:58.274489+0000","last_unstale":"2024-09-16T20:48:30.648590+0000","last_undegraded":"2024-09-16T20:48:30.648590+0000","last_fullsized":"2024-09-16T20:48:30.648590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:27:12.633064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091339500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"44'54","reported_seq":254,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T20:48:32.657247+0000","last_change":"2024-09-16T20:48:26.756724+0000","last_active":"2024-09-16T20:48:32.657247+0000","last_peered":"2024-09-16T20:48:32.657247+0000","last_clean":"2024-09-16T20:48:32.657247+0000","last_became_active":"2024-09-16T20:48:26.640801+0000","last_became_peered":"2024-09-16T20:48:26.640801+0000","last_unstale":"2024-09-16T20:48:32.657247+0000","last_undegraded":"2024-09-16T20:48:32.657247+0000","last_fullsized":"2024-09-16T20:48:32.657247+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:39:13.508962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067924999999999995,"stat_sum":{"num_bytes":1044517,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":110,"num_read_kb":638,"num_write":58,"num_write_kb":1402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":12325,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":70,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T20:48:32.657097+0000","last_change":"2024-09-16T20:48:00.277182+0000","last_active":"2024-09-16T20:48:32.657097+0000","last_peered":"2024-09-16T20:48:32.657097+0000","last_clean":"2024-09-16T20:48:32.657097+0000","last_became_active":"2024-09-16T20:47:58.274389+0000","last_became_peered":"2024-09-16T20:47:58.274389+0000","last_unstale":"2024-09-16T20:48:32.657097+0000","last_undegraded":"2024-09-16T20:48:32.657097+0000","last_fullsized":"2024-09-16T20:48:32.657097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:28:02.441830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043659299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"43'58","reported_seq":4641,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T20:48:32.657260+0000","last_change":"2024-09-16T20:48:26.737968+0000","last_active":"2024-09-16T20:48:32.657260+0000","last_peered":"2024-09-16T20:48:32.657260+0000","last_clean":"2024-09-16T20:48:32.657260+0000","last_became_active":"2024-09-16T20:48:26.640838+0000","last_became_peered":"2024-09-16T20:48:26.640838+0000","last_unstale":"2024-09-16T20:48:32.657260+0000","last_undegraded":"2024-09-16T20:48:32.657260+0000","last_fullsized":"2024-09-16T20:48:32.657260+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:30:27.688992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012934750000000001,"stat_sum":{"num_bytes":32586752,"num_objects":7,"num_object_clones":3,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":2,"num_read":4511,"num_read_kb":9446,"num_write":57,"num_write_kb":486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":165376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":66,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-16T20:48:30.648405+0000","last_change":"2024-09-16T20:48:00.278054+0000","last_active":"2024-09-16T20:48:30.648405+0000","last_peered":"2024-09-16T20:48:30.648405+0000","last_clean":"2024-09-16T20:48:30.648405+0000","last_became_active":"2024-09-16T20:47:58.272861+0000","last_became_peered":"2024-09-16T20:47:58.272861+0000","last_unstale":"2024-09-16T20:48:30.648405+0000","last_undegraded":"2024-09-16T20:48:30.648405+0000","last_fullsized":"2024-09-16T20:48:30.648405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:13:59.135547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012429489999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"45'55","reported_seq":163,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T20:48:32.657069+0000","last_change":"2024-09-16T20:48:09.564955+0000","last_active":"2024-09-16T20:48:32.657069+0000","last_peered":"2024-09-16T20:48:32.657069+0000","last_clean":"2024-09-16T20:48:32.657069+0000","last_became_active":"2024-09-16T20:48:07.564072+0000","last_became_peered":"2024-09-16T20:48:07.564072+0000","last_unstale":"2024-09-16T20:48:32.657069+0000","last_undegraded":"2024-09-16T20:48:32.657069+0000","last_fullsized":"2024-09-16T20:48:32.657069+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:31:45.512453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076367000000000004,"stat_sum":{"num_bytes":847872,"num_objects":8,"num_object_clones":3,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":3,"num_read":51,"num_read_kb":424,"num_write":58,"num_write_kb":795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-16T20:48:30.650336+0000","last_change":"2024-09-16T20:48:26.634201+0000","last_active":"2024-09-16T20:48:30.650336+0000","last_peered":"2024-09-16T20:48:30.650336+0000","last_clean":"2024-09-16T20:48:30.650336+0000","last_became_active":"2024-09-16T20:48:26.633867+0000","last_became_peered":"2024-09-16T20:48:26.633867+0000","last_unstale":"2024-09-16T20:48:30.650336+0000","last_undegraded":"2024-09-16T20:48:30.650336+0000","last_fullsized":"2024-09-16T20:48:30.650336+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:39:54.522672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"43'65","reported_seq":3083,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-16T20:48:31.086531+0000","last_change":"2024-09-16T20:48:26.698332+0000","last_active":"2024-09-16T20:48:31.086531+0000","last_peered":"2024-09-16T20:48:31.086531+0000","last_clean":"2024-09-16T20:48:31.086531+0000","last_became_active":"2024-09-16T20:48:26.641113+0000","last_became_peered":"2024-09-16T20:48:26.641113+0000","last_unstale":"2024-09-16T20:48:31.086531+0000","last_undegraded":"2024-09-16T20:48:31.086531+0000","last_fullsized":"2024-09-16T20:48:31.086531+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:09:11.534371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010471600000000001,"stat_sum":{"num_bytes":17079877,"num_objects":15,"num_object_clones":3,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":2,"num_read":2263,"num_read_kb":7605,"num_write":60,"num_write_kb":409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":12357,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"43'91","reported_seq":2777,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-16T20:48:31.087147+0000","last_change":"2024-09-16T20:48:09.570254+0000","last_active":"2024-09-16T20:48:31.087147+0000","last_peered":"2024-09-16T20:48:31.087147+0000","last_clean":"2024-09-16T20:48:31.087147+0000","last_became_active":"2024-09-16T20:48:07.568143+0000","last_became_peered":"2024-09-16T20:48:07.568143+0000","last_unstale":"2024-09-16T20:48:31.087147+0000","last_undegraded":"2024-09-16T20:48:31.087147+0000","last_fullsized":"2024-09-16T20:48:31.087147+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:44:17.002275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038800899999999998,"stat_sum":{"num_bytes":29556776,"num_objects":23,"num_object_clones":9,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":9,"num_read":2681,"num_read_kb":8106,"num_write":81,"num_write_kb":1091,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-16T20:48:30.648629+0000","last_change":"2024-09-16T20:48:00.276970+0000","last_active":"2024-09-16T20:48:30.648629+0000","last_peered":"2024-09-16T20:48:30.648629+0000","last_clean":"2024-09-16T20:48:30.648629+0000","last_became_active":"2024-09-16T20:47:58.274617+0000","last_became_peered":"2024-09-16T20:47:58.274617+0000","last_unstale":"2024-09-16T20:48:30.648629+0000","last_undegraded":"2024-09-16T20:48:30.648629+0000","last_fullsized":"2024-09-16T20:48:30.648629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:28:54.467905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062077499999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"44'21","reported_seq":195,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-16T20:48:30.983724+0000","last_change":"2024-09-16T20:48:09.568988+0000","last_active":"2024-09-16T20:48:30.983724+0000","last_peered":"2024-09-16T20:48:30.983724+0000","last_clean":"2024-09-16T20:48:30.983724+0000","last_became_active":"2024-09-16T20:48:07.568564+0000","last_became_peered":"2024-09-16T20:48:07.568564+0000","last_unstale":"2024-09-16T20:48:30.983724+0000","last_undegraded":"2024-09-16T20:48:30.983724+0000","last_fullsized":"2024-09-16T20:48:30.983724+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:03:47.653698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088331900000000003,"stat_sum":{"num_bytes":2596384,"num_objects":11,"num_object_clones":2,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":2,"num_read":130,"num_read_kb":344,"num_write":18,"num_write_kb":308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-16T20:48:30.650298+0000","last_change":"2024-09-16T20:48:26.641551+0000","last_active":"2024-09-16T20:48:30.650298+0000","last_peered":"2024-09-16T20:48:30.650298+0000","last_clean":"2024-09-16T20:48:30.650298+0000","last_became_active":"2024-09-16T20:48:26.641155+0000","last_became_peered":"2024-09-16T20:48:26.641155+0000","last_unstale":"2024-09-16T20:48:30.650298+0000","last_undegraded":"2024-09-16T20:48:30.650298+0000","last_fullsized":"2024-09-16T20:48:30.650298+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:29:58.103219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"45'52","reported_seq":1446,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T20:48:33.150943+0000","last_change":"2024-09-16T20:48:09.565596+0000","last_active":"2024-09-16T20:48:33.150943+0000","last_peered":"2024-09-16T20:48:33.150943+0000","last_clean":"2024-09-16T20:48:33.150943+0000","last_became_active":"2024-09-16T20:48:07.566097+0000","last_became_peered":"2024-09-16T20:48:07.566097+0000","last_unstale":"2024-09-16T20:48:33.150943+0000","last_undegraded":"2024-09-16T20:48:33.150943+0000","last_fullsized":"2024-09-16T20:48:33.150943+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:30:19.338140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012840460000000001,"stat_sum":{"num_bytes":16309248,"num_objects":7,"num_object_clones":2,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":2,"num_read":1346,"num_read_kb":12058,"num_write":57,"num_write_kb":4677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":66,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-16T20:48:30.654285+0000","last_change":"2024-09-16T20:48:00.408392+0000","last_active":"2024-09-16T20:48:30.654285+0000","last_peered":"2024-09-16T20:48:30.654285+0000","last_clean":"2024-09-16T20:48:30.654285+0000","last_became_active":"2024-09-16T20:47:58.273260+0000","last_became_peered":"2024-09-16T20:47:58.273260+0000","last_unstale":"2024-09-16T20:48:30.654285+0000","last_undegraded":"2024-09-16T20:48:30.654285+0000","last_fullsized":"2024-09-16T20:48:30.654285+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:01:53.009880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037636000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"43'96","reported_seq":1134,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-16T20:48:31.172051+0000","last_change":"2024-09-16T20:48:26.664373+0000","last_active":"2024-09-16T20:48:31.172051+0000","last_peered":"2024-09-16T20:48:31.172051+0000","last_clean":"2024-09-16T20:48:31.172051+0000","last_became_active":"2024-09-16T20:48:26.639796+0000","last_became_peered":"2024-09-16T20:48:26.639796+0000","last_unstale":"2024-09-16T20:48:31.172051+0000","last_undegraded":"2024-09-16T20:48:31.172051+0000","last_fullsized":"2024-09-16T20:48:31.172051+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:15:19.140741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063857599999999997,"stat_sum":{"num_bytes":10188304,"num_objects":13,"num_object_clones":4,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":4,"num_read":2334,"num_read_kb":2167,"num_write":65,"num_write_kb":445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":109072,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":67,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-16T20:48:30.648724+0000","last_change":"2024-09-16T20:48:00.277739+0000","last_active":"2024-09-16T20:48:30.648724+0000","last_peered":"2024-09-16T20:48:30.648724+0000","last_clean":"2024-09-16T20:48:30.648724+0000","last_became_active":"2024-09-16T20:47:58.271174+0000","last_became_peered":"2024-09-16T20:47:58.271174+0000","last_unstale":"2024-09-16T20:48:30.648724+0000","last_undegraded":"2024-09-16T20:48:30.648724+0000","last_fullsized":"2024-09-16T20:48:30.648724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:11:35.985075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095771799999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":157,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-16T20:48:30.647427+0000","last_change":"2024-09-16T20:47:53.928484+0000","last_active":"2024-09-16T20:48:30.647427+0000","last_peered":"2024-09-16T20:48:30.647427+0000","last_clean":"2024-09-16T20:48:30.647427+0000","last_became_active":"2024-09-16T20:47:53.926422+0000","last_became_peered":"2024-09-16T20:47:53.926422+0000","last_unstale":"2024-09-16T20:48:30.647427+0000","last_undegraded":"2024-09-16T20:48:30.647427+0000","last_fullsized":"2024-09-16T20:48:30.647427+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:52.241408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:52.241408+0000","last_clean_scrub_stamp":"2024-09-16T20:47:52.241408+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:18:04.963138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"45'171","reported_seq":681,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T20:48:33.189602+0000","last_change":"2024-09-16T20:48:09.565136+0000","last_active":"2024-09-16T20:48:33.189602+0000","last_peered":"2024-09-16T20:48:33.189602+0000","last_clean":"2024-09-16T20:48:33.189602+0000","last_became_active":"2024-09-16T20:48:07.566139+0000","last_became_peered":"2024-09-16T20:48:07.566139+0000","last_unstale":"2024-09-16T20:48:33.189602+0000","last_undegraded":"2024-09-16T20:48:33.189602+0000","last_fullsized":"2024-09-16T20:48:33.189602+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:05:11.901556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001025172,"stat_sum":{"num_bytes":6528309,"num_objects":10,"num_object_clones":3,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":3,"num_read":1091,"num_read_kb":5794,"num_write":259,"num_write_kb":2782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"43'58","reported_seq":911,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-16T20:48:31.216856+0000","last_change":"2024-09-16T20:48:26.685452+0000","last_active":"2024-09-16T20:48:31.216856+0000","last_peered":"2024-09-16T20:48:31.216856+0000","last_clean":"2024-09-16T20:48:31.216856+0000","last_became_active":"2024-09-16T20:48:26.641523+0000","last_became_peered":"2024-09-16T20:48:26.641523+0000","last_unstale":"2024-09-16T20:48:31.216856+0000","last_undegraded":"2024-09-16T20:48:31.216856+0000","last_fullsized":"2024-09-16T20:48:31.216856+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:02:21.159446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044614999999999998,"stat_sum":{"num_bytes":14994944,"num_objects":11,"num_object_clones":4,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":3,"num_read":769,"num_read_kb":6012,"num_write":56,"num_write_kb":2732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":2048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"44'72","reported_seq":915,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-16T20:48:30.648563+0000","last_change":"2024-09-16T20:48:09.569992+0000","last_active":"2024-09-16T20:48:30.648563+0000","last_peered":"2024-09-16T20:48:30.648563+0000","last_clean":"2024-09-16T20:48:30.648563+0000","last_became_active":"2024-09-16T20:48:07.563972+0000","last_became_peered":"2024-09-16T20:48:07.563972+0000","last_unstale":"2024-09-16T20:48:30.648563+0000","last_undegraded":"2024-09-16T20:48:30.648563+0000","last_fullsized":"2024-09-16T20:48:30.648563+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:13:52.331552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039392600000000001,"stat_sum":{"num_bytes":24970240,"num_objects":12,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":3,"num_read":812,"num_read_kb":6846,"num_write":197,"num_write_kb":2273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"43'101","reported_seq":2088,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-16T20:48:31.135026+0000","last_change":"2024-09-16T20:48:26.744936+0000","last_active":"2024-09-16T20:48:31.135026+0000","last_peered":"2024-09-16T20:48:31.135026+0000","last_clean":"2024-09-16T20:48:31.135026+0000","last_became_active":"2024-09-16T20:48:26.643566+0000","last_became_peered":"2024-09-16T20:48:26.643566+0000","last_unstale":"2024-09-16T20:48:31.135026+0000","last_undegraded":"2024-09-16T20:48:31.135026+0000","last_fullsized":"2024-09-16T20:48:31.135026+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:49:16.228762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014289540000000001,"stat_sum":{"num_bytes":24842297,"num_objects":10,"num_object_clones":3,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":3,"num_read":2124,"num_read_kb":3965,"num_write":126,"num_write_kb":601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":137785,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"44'234","reported_seq":4442,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-16T20:48:30.655620+0000","last_change":"2024-09-16T20:48:26.665733+0000","last_active":"2024-09-16T20:48:30.655620+0000","last_peered":"2024-09-16T20:48:30.655620+0000","last_clean":"2024-09-16T20:48:30.655620+0000","last_became_active":"2024-09-16T20:48:26.640480+0000","last_became_peered":"2024-09-16T20:48:26.640480+0000","last_unstale":"2024-09-16T20:48:30.655620+0000","last_undegraded":"2024-09-16T20:48:30.655620+0000","last_fullsized":"2024-09-16T20:48:30.655620+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":234,"log_dups_size":0,"ondisk_log_size":234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:29:49.589991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047762999999999999,"stat_sum":{"num_bytes":30515456,"num_objects":11,"num_object_clones":4,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":3,"num_read":5244,"num_read_kb":10777,"num_write":333,"num_write_kb":1009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":233006,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"43'111","reported_seq":1253,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-16T20:48:31.216051+0000","last_change":"2024-09-16T20:48:09.565667+0000","last_active":"2024-09-16T20:48:31.216051+0000","last_peered":"2024-09-16T20:48:31.216051+0000","last_clean":"2024-09-16T20:48:31.216051+0000","last_became_active":"2024-09-16T20:48:07.567327+0000","last_became_peered":"2024-09-16T20:48:07.567327+0000","last_unstale":"2024-09-16T20:48:31.216051+0000","last_undegraded":"2024-09-16T20:48:31.216051+0000","last_fullsized":"2024-09-16T20:48:31.216051+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:11:12.378028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012087560000000001,"stat_sum":{"num_bytes":6169088,"num_objects":12,"num_object_clones":5,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":3,"num_read":2013,"num_read_kb":5804,"num_write":78,"num_write_kb":2963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"43'61","reported_seq":1749,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-16T20:48:31.217825+0000","last_change":"2024-09-16T20:48:09.565024+0000","last_active":"2024-09-16T20:48:31.217825+0000","last_peered":"2024-09-16T20:48:31.217825+0000","last_clean":"2024-09-16T20:48:31.217825+0000","last_became_active":"2024-09-16T20:48:07.570704+0000","last_became_peered":"2024-09-16T20:48:07.570704+0000","last_unstale":"2024-09-16T20:48:31.217825+0000","last_undegraded":"2024-09-16T20:48:31.217825+0000","last_fullsized":"2024-09-16T20:48:31.217825+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:50:23.240844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011026149999999999,"stat_sum":{"num_bytes":26528768,"num_objects":12,"num_object_clones":5,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":2,"num_read":1656,"num_read_kb":8583,"num_write":59,"num_write_kb":2744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"43'109","reported_seq":1432,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-16T20:48:31.216935+0000","last_change":"2024-09-16T20:48:09.570786+0000","last_active":"2024-09-16T20:48:31.216935+0000","last_peered":"2024-09-16T20:48:31.216935+0000","last_clean":"2024-09-16T20:48:31.216935+0000","last_became_active":"2024-09-16T20:48:07.560783+0000","last_became_peered":"2024-09-16T20:48:07.560783+0000","last_unstale":"2024-09-16T20:48:31.216935+0000","last_undegraded":"2024-09-16T20:48:31.216935+0000","last_fullsized":"2024-09-16T20:48:31.216935+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:40:35.292938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040623999999999999,"stat_sum":{"num_bytes":9320464,"num_objects":18,"num_object_clones":7,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":4,"num_read":2174,"num_read_kb":6721,"num_write":74,"num_write_kb":3047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"43'61","reported_seq":1787,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-16T20:48:31.218054+0000","last_change":"2024-09-16T20:48:09.565413+0000","last_active":"2024-09-16T20:48:31.218054+0000","last_peered":"2024-09-16T20:48:31.218054+0000","last_clean":"2024-09-16T20:48:31.218054+0000","last_became_active":"2024-09-16T20:48:07.565932+0000","last_became_peered":"2024-09-16T20:48:07.565932+0000","last_unstale":"2024-09-16T20:48:31.218054+0000","last_undegraded":"2024-09-16T20:48:31.218054+0000","last_fullsized":"2024-09-16T20:48:31.218054+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:51:49.103406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115576,"stat_sum":{"num_bytes":10113024,"num_objects":17,"num_object_clones":7,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":5,"num_read":1686,"num_read_kb":13464,"num_write":57,"num_write_kb":2821,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"43'53","reported_seq":837,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-16T20:48:31.218317+0000","last_change":"2024-09-16T20:48:09.564513+0000","last_active":"2024-09-16T20:48:31.218317+0000","last_peered":"2024-09-16T20:48:31.218317+0000","last_clean":"2024-09-16T20:48:31.218317+0000","last_became_active":"2024-09-16T20:48:07.571985+0000","last_became_peered":"2024-09-16T20:48:07.571985+0000","last_unstale":"2024-09-16T20:48:31.218317+0000","last_undegraded":"2024-09-16T20:48:31.218317+0000","last_fullsized":"2024-09-16T20:48:31.218317+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:32:14.525055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077335299999999995,"stat_sum":{"num_bytes":15750656,"num_objects":13,"num_object_clones":3,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":1,"num_read":736,"num_read_kb":8042,"num_write":56,"num_write_kb":5055,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"43'31","reported_seq":179,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-16T20:48:31.088085+0000","last_change":"2024-09-16T20:48:09.569080+0000","last_active":"2024-09-16T20:48:31.088085+0000","last_peered":"2024-09-16T20:48:31.088085+0000","last_clean":"2024-09-16T20:48:31.088085+0000","last_became_active":"2024-09-16T20:48:07.567597+0000","last_became_peered":"2024-09-16T20:48:07.567597+0000","last_unstale":"2024-09-16T20:48:31.088085+0000","last_undegraded":"2024-09-16T20:48:31.088085+0000","last_fullsized":"2024-09-16T20:48:31.088085+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:42:19.230517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000840955,"stat_sum":{"num_bytes":2230315,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":102,"num_read_kb":337,"num_write":31,"num_write_kb":254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"43'86","reported_seq":2466,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-16T20:48:31.216330+0000","last_change":"2024-09-16T20:48:09.565356+0000","last_active":"2024-09-16T20:48:31.216330+0000","last_peered":"2024-09-16T20:48:31.216330+0000","last_clean":"2024-09-16T20:48:31.216330+0000","last_became_active":"2024-09-16T20:48:07.565717+0000","last_became_peered":"2024-09-16T20:48:07.565717+0000","last_unstale":"2024-09-16T20:48:31.216330+0000","last_undegraded":"2024-09-16T20:48:31.216330+0000","last_fullsized":"2024-09-16T20:48:31.216330+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:33:43.022211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013583239999999999,"stat_sum":{"num_bytes":17741860,"num_objects":6,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":2,"num_read":2551,"num_read_kb":6551,"num_write":118,"num_write_kb":3134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":418679247,"num_objects":353,"num_object_clones":114,"num_object_copies":706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":353,"num_whiteouts":82,"num_read":62804,"num_read_kb":222365,"num_write":3896,"num_write_kb":58698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":5605797,"num_keys_recovered":35,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":95436800,"data_stored":120456015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3205,"ondisk_log_size":3205,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574860,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43164,"kb_used_data":7800,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94328676,"statfs":{"total":96636764160,"available":96592564224,"internally_reserved":0,"allocated":7987200,"data_stored":9176719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":0,"seq":55834574860,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574860,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57684,"kb_used_data":22324,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94314156,"statfs":{"total":96636764160,"available":96577695744,"internally_reserved":0,"allocated":22859776,"data_stored":28557630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62308,"kb_used_data":26964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94309532,"statfs":{"total":96636764160,"available":96572960768,"internally_reserved":0,"allocated":27611136,"data_stored":34410879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574860,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57684,"kb_used_data":22336,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94314156,"statfs":{"total":96636764160,"available":96577695744,"internally_reserved":0,"allocated":22872064,"data_stored":29835273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640268,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46896,"kb_used_data":11440,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94324944,"statfs":{"total":96636764160,"available":96588742656,"internally_reserved":0,"allocated":11714560,"data_stored":12894915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":22388736,"data_stored":29651048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":26644480,"data_stored":33714511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":22507520,"data_stored":28474158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5763072,"data_stored":7508401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":11218944,"data_stored":12705752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":6914048,"data_stored":8402145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T20:48:36.577 INFO:tasks.thrashosds.thrasher:Setting 3.1e to [1, 5, 3, 0] 2024-09-16T20:48:36.577 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.1e', '1', '5', '3', '0'] 2024-09-16T20:48:36.577 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.1e 1 5 3 0 2024-09-16T20:48:36.585 INFO:teuthology.orchestra.run.smithi139.stdout:1440 write 0xbadde00 thru 0xbaeda8c (0xfc8d bytes) 2024-09-16T20:48:36.587 INFO:teuthology.orchestra.run.smithi139.stdout:1446 read 0x71fec00 thru 0x7207d70 (0x9171 bytes) 2024-09-16T20:48:36.589 INFO:teuthology.orchestra.run.smithi139.stdout:1447 trunc from 0xbaedc00 to 0xa4a3400 2024-09-16T20:48:36.602 INFO:teuthology.orchestra.run.smithi139.stdout:1448 write 0x3e48c00 thru 0x3e57bd5 (0xefd6 bytes) 2024-09-16T20:48:36.604 INFO:teuthology.orchestra.run.smithi139.stdout:1449 read 0x5866e00 thru 0x586c541 (0x5742 bytes) 2024-09-16T20:48:36.604 INFO:teuthology.orchestra.run.smithi139.stdout:1453 read 0x3ca7000 thru 0x3cac703 (0x5704 bytes) 2024-09-16T20:48:36.604 INFO:teuthology.orchestra.run.smithi139.stdout:1454 write 0x6ac0200 thru 0x6ac8d78 (0x8b79 bytes) 2024-09-16T20:48:36.605 INFO:teuthology.orchestra.run.smithi139.stdout:1456 write 0x5bd0e00 thru 0x5bdb765 (0xa966 bytes) 2024-09-16T20:48:36.606 INFO:teuthology.orchestra.run.smithi139.stdout:1458 read 0x5d16400 thru 0x5d1c4b6 (0x60b7 bytes) 2024-09-16T20:48:36.607 INFO:teuthology.orchestra.run.smithi139.stdout:1459 read 0x2a46000 thru 0x2a4b52c (0x552d bytes) 2024-09-16T20:48:36.608 INFO:teuthology.orchestra.run.smithi139.stdout:1460 read 0x218d800 thru 0x2192661 (0x4e62 bytes) 2024-09-16T20:48:36.608 INFO:teuthology.orchestra.run.smithi139.stdout:1461 punch from 0x86ea400 to 0x86f7c00, (0xd800 bytes) 2024-09-16T20:48:36.609 INFO:teuthology.orchestra.run.smithi139.stdout:1464 read 0x4a7ec00 thru 0x4a86693 (0x7a94 bytes) 2024-09-16T20:48:36.609 INFO:teuthology.orchestra.run.smithi139.stdout:1466 read 0x7553000 thru 0x755329e (0x29f bytes) 2024-09-16T20:48:36.610 INFO:teuthology.orchestra.run.smithi139.stdout:1473 read 0x2484000 thru 0x248d4d2 (0x94d3 bytes) 2024-09-16T20:48:36.611 INFO:teuthology.orchestra.run.smithi139.stdout:1476 read 0x720a400 thru 0x720e90d (0x450e bytes) 2024-09-16T20:48:36.611 INFO:teuthology.orchestra.run.smithi139.stdout:1478 punch from 0x3167400 to 0x316b400, (0x4000 bytes) 2024-09-16T20:48:36.611 INFO:teuthology.orchestra.run.smithi139.stdout:1479 write 0x4bff800 thru 0x4c02892 (0x3093 bytes) 2024-09-16T20:48:36.611 INFO:teuthology.orchestra.run.smithi139.stdout:1481 write 0x1ae6000 thru 0x1af554f (0xf550 bytes) 2024-09-16T20:48:36.612 INFO:teuthology.orchestra.run.smithi139.stdout:1483 read 0x563bc00 thru 0x563e49b (0x289c bytes) 2024-09-16T20:48:36.612 INFO:teuthology.orchestra.run.smithi139.stdout:1486 write 0x5492000 thru 0x549d975 (0xb976 bytes) 2024-09-16T20:48:36.612 INFO:teuthology.orchestra.run.smithi139.stdout:1489 read 0x8f75600 thru 0x8f761dd (0xbde bytes) 2024-09-16T20:48:36.617 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:36.616+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:36.632 INFO:teuthology.orchestra.run.smithi139.stdout:1490 write 0xe45d600 thru 0xe465a4f (0x8450 bytes) 2024-09-16T20:48:36.634 INFO:teuthology.orchestra.run.smithi139.stdout:1491 punch from 0x3df6400 to 0x3df8200, (0x1e00 bytes) 2024-09-16T20:48:36.635 INFO:teuthology.orchestra.run.smithi139.stdout:1492 punch from 0xae20000 to 0xae2b400, (0xb400 bytes) 2024-09-16T20:48:36.636 INFO:teuthology.orchestra.run.smithi139.stdout:1495 write 0xc139200 thru 0xc13e07b (0x4e7c bytes) 2024-09-16T20:48:36.636 INFO:teuthology.orchestra.run.smithi139.stdout:1496 punch from 0x53f6e00 to 0x53fee00, (0x8000 bytes) 2024-09-16T20:48:36.636 INFO:teuthology.orchestra.run.smithi139.stdout:1497 write 0x24ae400 thru 0x24bb8f8 (0xd4f9 bytes) 2024-09-16T20:48:36.637 INFO:teuthology.orchestra.run.smithi139.stdout:1498 read 0x9cfae00 thru 0x9cfb74c (0x94d bytes) 2024-09-16T20:48:36.638 INFO:teuthology.orchestra.run.smithi139.stdout:1499 punch from 0x6156200 to 0x6159a00, (0x3800 bytes) 2024-09-16T20:48:36.638 INFO:teuthology.orchestra.run.smithi139.stdout:1500 write 0xc9fb600 thru 0xca0285e (0x725f bytes) 2024-09-16T20:48:36.638 INFO:teuthology.orchestra.run.smithi139.stdout:1502 read 0x140d200 thru 0x1412648 (0x5449 bytes) 2024-09-16T20:48:36.643 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:36.643 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:48:36.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:48:36.643 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:48:36.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.643 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:4904 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 7309312~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:36.643 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767724+0000", 2024-09-16T20:48:36.643 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 16.873460976, 2024-09-16T20:48:36.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050270214000000001, 2024-09-16T20:48:36.643 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:36.643 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:36.643 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:36.644 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:36.644 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:36.644 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:36.644 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4904 2024-09-16T20:48:36.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.644 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:36.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.644 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:36.644 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767724+0000", 2024-09-16T20:48:36.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.644 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:36.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767724+0000", 2024-09-16T20:48:36.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:36.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767724+0000", 2024-09-16T20:48:36.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1399999999999998e-07 2024-09-16T20:48:36.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:36.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767725+0000", 2024-09-16T20:48:36.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0380000000000001e-06 2024-09-16T20:48:36.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.646 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:36.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767729+0000", 2024-09-16T20:48:36.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2490000000000001e-06 2024-09-16T20:48:36.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.646 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:36.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767731+0000", 2024-09-16T20:48:36.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5940000000000001e-06 2024-09-16T20:48:36.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.646 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:36.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.817787+0000", 2024-09-16T20:48:36.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050055792000000002 2024-09-16T20:48:36.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.647 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:36.647 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.817883+0000", 2024-09-16T20:48:36.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5606999999999995e-05 2024-09-16T20:48:36.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.647 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:36.647 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.817994+0000", 2024-09-16T20:48:36.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011162000000000001 2024-09-16T20:48:36.647 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.647 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:36.647 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.648 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:4906 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 7317504~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:36.648 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767734+0000", 2024-09-16T20:48:36.648 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 16.873450713, 2024-09-16T20:48:36.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050406608999999998, 2024-09-16T20:48:36.648 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:36.648 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:36.648 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:36.648 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:36.648 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:36.648 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:36.648 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4906 2024-09-16T20:48:36.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.649 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:36.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:36.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767734+0000", 2024-09-16T20:48:36.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:36.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767734+0000", 2024-09-16T20:48:36.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:36.650 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767735+0000", 2024-09-16T20:48:36.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.05e-07 2024-09-16T20:48:36.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.650 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:36.652 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767736+0000", 2024-09-16T20:48:36.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0240000000000001e-06 2024-09-16T20:48:36.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.652 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:36.652 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767739+0000", 2024-09-16T20:48:36.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.067e-06 2024-09-16T20:48:36.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:36.653 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767742+0000", 2024-09-16T20:48:36.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.23e-06 2024-09-16T20:48:36.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:36.653 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818027+0000", 2024-09-16T20:48:36.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050285102999999998 2024-09-16T20:48:36.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:36.653 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818072+0000", 2024-09-16T20:48:36.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5129000000000001e-05 2024-09-16T20:48:36.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:36.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818141+0000", 2024-09-16T20:48:36.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8850999999999995e-05 2024-09-16T20:48:36.654 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.654 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:36.654 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.654 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:4908 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 7325696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:36.654 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767745+0000", 2024-09-16T20:48:36.654 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 16.873440318, 2024-09-16T20:48:36.655 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050654230000000001, 2024-09-16T20:48:36.655 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:36.655 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:36.655 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:36.655 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:36.655 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:36.655 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:36.655 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4908 2024-09-16T20:48:36.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.655 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:36.655 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.655 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:36.655 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767745+0000", 2024-09-16T20:48:36.655 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.656 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:36.656 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767745+0000", 2024-09-16T20:48:36.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.656 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:36.656 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767745+0000", 2024-09-16T20:48:36.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1e-07 2024-09-16T20:48:36.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.656 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:36.656 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767749+0000", 2024-09-16T20:48:36.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.532e-06 2024-09-16T20:48:36.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.657 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:36.657 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767753+0000", 2024-09-16T20:48:36.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4939999999999999e-06 2024-09-16T20:48:36.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.657 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:36.657 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767756+0000", 2024-09-16T20:48:36.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.61e-06 2024-09-16T20:48:36.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.658 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:36.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818184+0000", 2024-09-16T20:48:36.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050428615000000003 2024-09-16T20:48:36.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.658 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:36.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818276+0000", 2024-09-16T20:48:36.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2077999999999995e-05 2024-09-16T20:48:36.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.658 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:36.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818399+0000", 2024-09-16T20:48:36.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012269099999999999 2024-09-16T20:48:36.658 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.659 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:36.659 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.659 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.659 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:4910 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 7333888~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:36.659 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767759+0000", 2024-09-16T20:48:36.659 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 16.873426427999998, 2024-09-16T20:48:36.659 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050815076000000001, 2024-09-16T20:48:36.659 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:36.659 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:36.659 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:36.659 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:36.659 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:36.660 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:36.660 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4910 2024-09-16T20:48:36.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.660 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:36.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.660 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:36.660 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767759+0000", 2024-09-16T20:48:36.660 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.660 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:36.660 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767759+0000", 2024-09-16T20:48:36.660 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.661 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:36.661 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767759+0000", 2024-09-16T20:48:36.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2600000000000001e-07 2024-09-16T20:48:36.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.661 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:36.661 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767760+0000", 2024-09-16T20:48:36.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0640000000000001e-06 2024-09-16T20:48:36.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.661 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:36.662 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767763+0000", 2024-09-16T20:48:36.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.994e-06 2024-09-16T20:48:36.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.662 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.662 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:36.662 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767765+0000", 2024-09-16T20:48:36.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5220000000000002e-06 2024-09-16T20:48:36.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.662 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.662 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:36.662 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818438+0000", 2024-09-16T20:48:36.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050672602999999997 2024-09-16T20:48:36.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.663 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:36.663 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818481+0000", 2024-09-16T20:48:36.663 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2967999999999999e-05 2024-09-16T20:48:36.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.663 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:36.663 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818574+0000", 2024-09-16T20:48:36.663 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2699000000000001e-05 2024-09-16T20:48:36.663 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.663 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:36.663 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.664 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5012 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2101248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:36.664 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767768+0000", 2024-09-16T20:48:36.664 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 16.873416778999999, 2024-09-16T20:48:36.664 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051084682999999999, 2024-09-16T20:48:36.664 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:36.664 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:36.664 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:36.664 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:36.664 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:36.664 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:36.664 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5012 2024-09-16T20:48:36.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.664 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:36.665 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.665 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:36.665 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767768+0000", 2024-09-16T20:48:36.665 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.665 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.665 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.665 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:36.665 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767768+0000", 2024-09-16T20:48:36.665 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.665 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.665 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.665 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:36.665 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767769+0000", 2024-09-16T20:48:36.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1500000000000001e-07 2024-09-16T20:48:36.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.666 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:36.666 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767770+0000", 2024-09-16T20:48:36.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.015e-06 2024-09-16T20:48:36.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.666 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:36.666 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767773+0000", 2024-09-16T20:48:36.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1070000000000001e-06 2024-09-16T20:48:36.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.666 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:36.667 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767775+0000", 2024-09-16T20:48:36.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.508e-06 2024-09-16T20:48:36.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.667 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:36.667 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818630+0000", 2024-09-16T20:48:36.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050855282000000002 2024-09-16T20:48:36.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.667 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:36.667 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818732+0000", 2024-09-16T20:48:36.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101531 2024-09-16T20:48:36.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.668 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:36.668 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818853+0000", 2024-09-16T20:48:36.668 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000121025 2024-09-16T20:48:36.668 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.668 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:36.668 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.668 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.668 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.668 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5014 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2109440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:36.668 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767778+0000", 2024-09-16T20:48:36.668 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 16.873406843000001, 2024-09-16T20:48:36.668 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051260776000000001, 2024-09-16T20:48:36.668 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:36.669 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:36.669 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:36.669 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:36.669 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:36.669 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:36.669 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5014 2024-09-16T20:48:36.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.669 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:36.669 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.669 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:36.669 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767778+0000", 2024-09-16T20:48:36.669 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.670 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:36.670 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767778+0000", 2024-09-16T20:48:36.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.670 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:36.670 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767824+0000", 2024-09-16T20:48:36.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5890999999999998e-05 2024-09-16T20:48:36.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.670 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:36.670 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767827+0000", 2024-09-16T20:48:36.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.644e-06 2024-09-16T20:48:36.671 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.671 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.671 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:36.671 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767832+0000", 2024-09-16T20:48:36.671 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2689999999999999e-06 2024-09-16T20:48:36.671 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.671 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.671 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:36.671 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767835+0000", 2024-09-16T20:48:36.671 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2100000000000002e-06 2024-09-16T20:48:36.671 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.671 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.672 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:36.672 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818887+0000", 2024-09-16T20:48:36.672 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051051276999999999 2024-09-16T20:48:36.672 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.672 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.672 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:36.672 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818953+0000", 2024-09-16T20:48:36.672 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6105999999999994e-05 2024-09-16T20:48:36.672 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.672 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.672 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:36.673 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819039+0000", 2024-09-16T20:48:36.673 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6378999999999994e-05 2024-09-16T20:48:36.673 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.673 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:36.673 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.673 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.673 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.673 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5016 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2117632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:36.673 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767841+0000", 2024-09-16T20:48:36.673 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 16.873344163999999, 2024-09-16T20:48:36.674 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051433288000000001, 2024-09-16T20:48:36.674 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:36.674 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:36.674 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:36.674 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:36.674 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:36.674 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:36.674 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5016 2024-09-16T20:48:36.674 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.674 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:36.674 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.674 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:36.674 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767841+0000", 2024-09-16T20:48:36.675 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.675 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.675 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:36.675 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767841+0000", 2024-09-16T20:48:36.675 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.675 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.675 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:36.675 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768156+0000", 2024-09-16T20:48:36.675 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00031505699999999999 2024-09-16T20:48:36.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.675 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.676 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:36.676 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768159+0000", 2024-09-16T20:48:36.676 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1659999999999998e-06 2024-09-16T20:48:36.676 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.676 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.676 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:36.676 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768166+0000", 2024-09-16T20:48:36.676 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0550000000000002e-06 2024-09-16T20:48:36.676 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.676 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.676 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:36.676 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768171+0000", 2024-09-16T20:48:36.676 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3170000000000003e-06 2024-09-16T20:48:36.676 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.677 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.677 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:36.677 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819084+0000", 2024-09-16T20:48:36.677 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050913355 2024-09-16T20:48:36.677 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.677 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.677 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:36.677 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819170+0000", 2024-09-16T20:48:36.677 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6328000000000004e-05 2024-09-16T20:48:36.677 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.677 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.677 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:36.677 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819274+0000", 2024-09-16T20:48:36.678 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010401 2024-09-16T20:48:36.678 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.678 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:36.678 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.678 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.678 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.678 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5018 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2125824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:36.678 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768178+0000", 2024-09-16T20:48:36.678 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 16.873006629999999, 2024-09-16T20:48:36.678 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051227013000000002, 2024-09-16T20:48:36.678 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:36.678 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:36.678 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:36.678 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:36.679 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:36.679 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:36.679 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5018 2024-09-16T20:48:36.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.679 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:36.679 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.679 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:36.679 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768178+0000", 2024-09-16T20:48:36.679 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.679 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.679 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:36.679 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768178+0000", 2024-09-16T20:48:36.679 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.680 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.680 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:36.680 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768179+0000", 2024-09-16T20:48:36.680 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9200000000000002e-07 2024-09-16T20:48:36.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.680 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.680 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:36.680 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768180+0000", 2024-09-16T20:48:36.680 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3790000000000001e-06 2024-09-16T20:48:36.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.680 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.680 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:36.681 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768184+0000", 2024-09-16T20:48:36.681 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5470000000000001e-06 2024-09-16T20:48:36.681 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.681 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.681 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:36.681 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768187+0000", 2024-09-16T20:48:36.681 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8119999999999999e-06 2024-09-16T20:48:36.681 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.681 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.681 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:36.681 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819305+0000", 2024-09-16T20:48:36.681 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051118683999999998 2024-09-16T20:48:36.681 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.681 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.682 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:36.682 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819351+0000", 2024-09-16T20:48:36.682 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5757999999999998e-05 2024-09-16T20:48:36.682 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.682 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.682 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:36.682 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819405+0000", 2024-09-16T20:48:36.682 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4441000000000003e-05 2024-09-16T20:48:36.682 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.682 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:36.682 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.682 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.682 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.683 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5020 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2134016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:36.683 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768190+0000", 2024-09-16T20:48:36.683 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 16.872994951999999, 2024-09-16T20:48:36.683 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051449661000000001, 2024-09-16T20:48:36.683 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:36.683 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:36.683 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:36.683 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:36.683 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:36.683 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:36.683 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5020 2024-09-16T20:48:36.683 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.683 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:36.683 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.684 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:36.684 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768190+0000", 2024-09-16T20:48:36.684 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.684 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.684 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.684 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:36.684 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768190+0000", 2024-09-16T20:48:36.684 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.684 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.684 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.684 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:36.684 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768282+0000", 2024-09-16T20:48:36.684 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2263999999999996e-05 2024-09-16T20:48:36.685 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.685 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.685 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:36.685 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768286+0000", 2024-09-16T20:48:36.685 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.529e-06 2024-09-16T20:48:36.685 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.685 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.685 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:36.685 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768294+0000", 2024-09-16T20:48:36.685 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6990000000000003e-06 2024-09-16T20:48:36.685 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.685 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.685 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:36.685 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768306+0000", 2024-09-16T20:48:36.686 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2112e-05 2024-09-16T20:48:36.686 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.686 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.686 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:36.686 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819449+0000", 2024-09-16T20:48:36.686 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051143521999999997 2024-09-16T20:48:36.686 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.686 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.686 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:36.686 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819539+0000", 2024-09-16T20:48:36.686 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0022e-05 2024-09-16T20:48:36.686 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.686 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.687 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:36.687 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819640+0000", 2024-09-16T20:48:36.687 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000100513 2024-09-16T20:48:36.687 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.687 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:36.687 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.687 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.687 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.687 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5022 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2142208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:36.687 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768326+0000", 2024-09-16T20:48:36.687 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 16.872859562999999, 2024-09-16T20:48:36.687 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051436952000000001, 2024-09-16T20:48:36.687 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:36.687 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:36.688 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:36.688 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:36.688 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:36.688 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:36.688 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5022 2024-09-16T20:48:36.688 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.688 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:36.688 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.688 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:36.688 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768326+0000", 2024-09-16T20:48:36.688 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.688 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.688 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.689 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:36.689 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768326+0000", 2024-09-16T20:48:36.689 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.689 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.689 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.689 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:36.689 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768477+0000", 2024-09-16T20:48:36.689 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015109300000000001 2024-09-16T20:48:36.689 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.689 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.689 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:36.689 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768480+0000", 2024-09-16T20:48:36.689 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8580000000000002e-06 2024-09-16T20:48:36.689 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.690 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.690 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:36.690 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768489+0000", 2024-09-16T20:48:36.690 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4600000000000003e-06 2024-09-16T20:48:36.690 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.690 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.690 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:36.690 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768493+0000", 2024-09-16T20:48:36.690 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2270000000000004e-06 2024-09-16T20:48:36.690 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.690 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.690 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:36.690 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819672+0000", 2024-09-16T20:48:36.691 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051178605000000002 2024-09-16T20:48:36.691 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.691 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.691 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:36.691 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819711+0000", 2024-09-16T20:48:36.691 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9357000000000003e-05 2024-09-16T20:48:36.691 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.691 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.691 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:36.691 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819762+0000", 2024-09-16T20:48:36.691 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1351999999999999e-05 2024-09-16T20:48:36.691 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.691 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:36.692 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.692 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.692 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.692 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5024 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2150400~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:36.692 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768501+0000", 2024-09-16T20:48:36.692 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 16.872684084999999, 2024-09-16T20:48:36.692 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051402678, 2024-09-16T20:48:36.692 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:36.692 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:36.692 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:36.692 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:36.692 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:36.692 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:36.693 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5024 2024-09-16T20:48:36.693 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.693 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:36.693 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.693 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:36.693 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768501+0000", 2024-09-16T20:48:36.693 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.693 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.693 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.693 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:36.693 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768501+0000", 2024-09-16T20:48:36.693 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.693 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.694 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.694 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:36.694 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768619+0000", 2024-09-16T20:48:36.694 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118187 2024-09-16T20:48:36.694 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.694 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.694 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:36.694 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768624+0000", 2024-09-16T20:48:36.694 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3259999999999997e-06 2024-09-16T20:48:36.694 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.694 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.694 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:36.694 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768631+0000", 2024-09-16T20:48:36.694 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6490000000000008e-06 2024-09-16T20:48:36.695 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.695 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.695 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:36.695 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768637+0000", 2024-09-16T20:48:36.695 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4210000000000003e-06 2024-09-16T20:48:36.695 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.695 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.695 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:36.695 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819800+0000", 2024-09-16T20:48:36.695 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051163486000000001 2024-09-16T20:48:36.695 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.695 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.695 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:36.696 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819845+0000", 2024-09-16T20:48:36.696 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4830999999999998e-05 2024-09-16T20:48:36.696 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.696 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.696 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:36.696 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819904+0000", 2024-09-16T20:48:36.696 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8777999999999999e-05 2024-09-16T20:48:36.696 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.696 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:36.696 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.696 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.696 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.696 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5026 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2158592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:36.696 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768646+0000", 2024-09-16T20:48:36.697 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 16.872538637000002, 2024-09-16T20:48:36.697 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051384386999999997, 2024-09-16T20:48:36.697 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:36.697 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:36.697 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:36.697 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:36.697 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:36.697 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:36.697 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5026 2024-09-16T20:48:36.697 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.697 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:36.697 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.697 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:36.698 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768646+0000", 2024-09-16T20:48:36.698 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.698 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.698 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.698 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:36.698 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768646+0000", 2024-09-16T20:48:36.698 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.698 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.698 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.698 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:36.698 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768775+0000", 2024-09-16T20:48:36.698 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012813200000000001 2024-09-16T20:48:36.698 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.698 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.699 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:36.699 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768778+0000", 2024-09-16T20:48:36.699 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2150000000000001e-06 2024-09-16T20:48:36.699 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.699 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.699 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:36.699 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768786+0000", 2024-09-16T20:48:36.699 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3259999999999996e-06 2024-09-16T20:48:36.699 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.699 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.699 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:36.699 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768790+0000", 2024-09-16T20:48:36.699 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.216e-06 2024-09-16T20:48:36.700 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.700 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.700 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:36.700 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819939+0000", 2024-09-16T20:48:36.700 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051149018999999997 2024-09-16T20:48:36.700 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.700 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.700 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:36.700 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819979+0000", 2024-09-16T20:48:36.700 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9311999999999999e-05 2024-09-16T20:48:36.700 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.700 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.700 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:36.701 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820031+0000", 2024-09-16T20:48:36.701 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2166999999999997e-05 2024-09-16T20:48:36.701 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.701 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:36.701 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.701 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.701 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.701 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5028 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2166784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:36.701 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768798+0000", 2024-09-16T20:48:36.701 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 16.872386993999999, 2024-09-16T20:48:36.701 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051388912000000002, 2024-09-16T20:48:36.701 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:36.701 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:36.701 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:36.702 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:36.702 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:36.702 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:36.702 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5028 2024-09-16T20:48:36.702 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.702 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:36.702 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.702 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:36.702 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768798+0000", 2024-09-16T20:48:36.702 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.702 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.702 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.702 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:36.702 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768798+0000", 2024-09-16T20:48:36.703 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.703 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.703 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.703 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:36.703 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768989+0000", 2024-09-16T20:48:36.703 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000190951 2024-09-16T20:48:36.703 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.703 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.703 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:36.703 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768993+0000", 2024-09-16T20:48:36.703 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9770000000000002e-06 2024-09-16T20:48:36.703 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.703 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.704 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:36.704 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.769002+0000", 2024-09-16T20:48:36.704 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5469999999999997e-06 2024-09-16T20:48:36.704 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.704 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.704 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:36.704 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.769020+0000", 2024-09-16T20:48:36.704 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8216999999999999e-05 2024-09-16T20:48:36.704 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.704 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.704 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:36.704 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820070+0000", 2024-09-16T20:48:36.704 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051049998999999999 2024-09-16T20:48:36.705 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.705 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.705 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:36.705 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820119+0000", 2024-09-16T20:48:36.705 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8962000000000003e-05 2024-09-16T20:48:36.705 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.705 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.705 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:36.705 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820187+0000", 2024-09-16T20:48:36.705 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8258999999999998e-05 2024-09-16T20:48:36.705 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.705 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:36.705 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.706 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.706 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.706 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5030 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2174976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:36.706 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.769031+0000", 2024-09-16T20:48:36.706 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 16.872153880999999, 2024-09-16T20:48:36.706 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051289830000000002, 2024-09-16T20:48:36.706 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:36.706 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:36.706 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:36.706 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:36.706 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:36.706 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:36.706 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5030 2024-09-16T20:48:36.707 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.707 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:36.707 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.707 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:36.707 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.769031+0000", 2024-09-16T20:48:36.707 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.707 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.707 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.707 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:36.707 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.769031+0000", 2024-09-16T20:48:36.707 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.707 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.707 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.707 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:36.708 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.773332+0000", 2024-09-16T20:48:36.708 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0043004039999999999 2024-09-16T20:48:36.708 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.708 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.708 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:36.708 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.773336+0000", 2024-09-16T20:48:36.708 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7849999999999999e-06 2024-09-16T20:48:36.708 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.708 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.708 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:36.708 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.773347+0000", 2024-09-16T20:48:36.708 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0482999999999999e-05 2024-09-16T20:48:36.708 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.708 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.709 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:36.709 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.773354+0000", 2024-09-16T20:48:36.709 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1559999999999998e-06 2024-09-16T20:48:36.709 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.709 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.709 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:36.709 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820216+0000", 2024-09-16T20:48:36.709 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046861591000000001 2024-09-16T20:48:36.709 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.709 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.709 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:36.709 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820258+0000", 2024-09-16T20:48:36.709 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2653e-05 2024-09-16T20:48:36.710 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.710 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.710 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:36.710 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820321+0000", 2024-09-16T20:48:36.710 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2757999999999998e-05 2024-09-16T20:48:36.710 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.710 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:36.710 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.710 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.710 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.710 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5067 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 6918144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:36.710 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.775279+0000", 2024-09-16T20:48:36.710 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 16.865905614999999, 2024-09-16T20:48:36.710 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050171387999999997, 2024-09-16T20:48:36.711 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:36.711 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:36.711 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:36.711 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:36.711 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:36.711 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:36.711 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5067 2024-09-16T20:48:36.711 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.711 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:36.711 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.711 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:36.711 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775279+0000", 2024-09-16T20:48:36.711 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.711 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.712 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.712 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:36.712 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775279+0000", 2024-09-16T20:48:36.712 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.712 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.712 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.712 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:36.712 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775432+0000", 2024-09-16T20:48:36.712 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015257399999999999 2024-09-16T20:48:36.712 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.712 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.712 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:36.712 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775436+0000", 2024-09-16T20:48:36.712 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0180000000000003e-06 2024-09-16T20:48:36.713 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.713 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.713 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:36.713 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775447+0000", 2024-09-16T20:48:36.713 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.077e-05 2024-09-16T20:48:36.713 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.713 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.713 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:36.713 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775456+0000", 2024-09-16T20:48:36.713 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4089999999999992e-06 2024-09-16T20:48:36.713 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.713 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.713 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:36.714 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825150+0000", 2024-09-16T20:48:36.714 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049693711000000002 2024-09-16T20:48:36.714 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.714 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.714 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:36.714 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825337+0000", 2024-09-16T20:48:36.714 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000187214 2024-09-16T20:48:36.714 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.714 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.714 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:36.714 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825451+0000", 2024-09-16T20:48:36.714 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011369199999999999 2024-09-16T20:48:36.714 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.714 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:36.715 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.715 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.715 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.715 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5068 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2330624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:36.715 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.775469+0000", 2024-09-16T20:48:36.715 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 16.865715653999999, 2024-09-16T20:48:36.715 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050198334999999997, 2024-09-16T20:48:36.715 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:36.715 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:36.715 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:36.715 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:36.715 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:36.715 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:36.715 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5068 2024-09-16T20:48:36.716 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.716 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:36.716 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.716 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:36.716 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775469+0000", 2024-09-16T20:48:36.716 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.716 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.716 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.716 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:36.716 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775469+0000", 2024-09-16T20:48:36.716 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.716 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.716 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.717 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:36.717 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775720+0000", 2024-09-16T20:48:36.717 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000250661 2024-09-16T20:48:36.717 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.717 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.717 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:36.717 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775724+0000", 2024-09-16T20:48:36.717 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8700000000000002e-06 2024-09-16T20:48:36.717 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.717 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.717 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:36.717 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775733+0000", 2024-09-16T20:48:36.717 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4239999999999999e-06 2024-09-16T20:48:36.718 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.718 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.718 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:36.718 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775740+0000", 2024-09-16T20:48:36.718 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9519999999999997e-06 2024-09-16T20:48:36.718 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.718 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.718 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:36.718 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825484+0000", 2024-09-16T20:48:36.718 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049743385000000001 2024-09-16T20:48:36.718 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.718 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.718 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:36.719 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825572+0000", 2024-09-16T20:48:36.719 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7834e-05 2024-09-16T20:48:36.719 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.719 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.719 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:36.719 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825668+0000", 2024-09-16T20:48:36.719 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6209e-05 2024-09-16T20:48:36.719 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.719 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:36.719 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.719 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.719 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.719 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5069 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 6926336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:36.719 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.775753+0000", 2024-09-16T20:48:36.720 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 16.865432430999999, 2024-09-16T20:48:36.720 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050146430999999998, 2024-09-16T20:48:36.720 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:36.720 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:36.720 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:36.720 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:36.720 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:36.720 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:36.720 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5069 2024-09-16T20:48:36.720 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.720 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:36.720 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.720 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:36.721 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775753+0000", 2024-09-16T20:48:36.721 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.721 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.721 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.721 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:36.721 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775753+0000", 2024-09-16T20:48:36.721 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.721 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.721 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.722 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:36.722 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775865+0000", 2024-09-16T20:48:36.722 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000112402 2024-09-16T20:48:36.722 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.722 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.722 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:36.722 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775869+0000", 2024-09-16T20:48:36.722 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5130000000000001e-06 2024-09-16T20:48:36.722 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.722 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.722 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:36.722 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775877+0000", 2024-09-16T20:48:36.722 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1359999999999997e-06 2024-09-16T20:48:36.722 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.723 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.723 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:36.723 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775882+0000", 2024-09-16T20:48:36.723 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6710000000000004e-06 2024-09-16T20:48:36.723 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.723 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.723 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:36.723 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825729+0000", 2024-09-16T20:48:36.723 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049846898000000001 2024-09-16T20:48:36.723 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.723 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.723 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:36.723 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825807+0000", 2024-09-16T20:48:36.724 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7875999999999998e-05 2024-09-16T20:48:36.724 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.724 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.724 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:36.724 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825899+0000", 2024-09-16T20:48:36.724 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1935000000000007e-05 2024-09-16T20:48:36.724 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.724 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:36.724 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.724 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.724 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.724 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5070 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2338816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:36.724 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.775892+0000", 2024-09-16T20:48:36.725 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 16.865292576999998, 2024-09-16T20:48:36.725 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050155311000000001, 2024-09-16T20:48:36.725 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:36.725 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:36.725 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:36.725 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:36.725 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:36.725 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:36.725 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5070 2024-09-16T20:48:36.725 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.725 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:36.725 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.725 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:36.726 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775892+0000", 2024-09-16T20:48:36.726 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.726 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.726 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.726 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:36.726 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775892+0000", 2024-09-16T20:48:36.726 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.726 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.726 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.726 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:36.726 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776142+0000", 2024-09-16T20:48:36.726 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00024950500000000002 2024-09-16T20:48:36.726 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.726 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.727 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:36.727 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776146+0000", 2024-09-16T20:48:36.727 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2819999999999998e-06 2024-09-16T20:48:36.727 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.727 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.727 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:36.727 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776157+0000", 2024-09-16T20:48:36.727 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0807e-05 2024-09-16T20:48:36.727 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.727 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.727 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:36.727 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776165+0000", 2024-09-16T20:48:36.727 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1300000000000001e-06 2024-09-16T20:48:36.728 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.728 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.728 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:36.728 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825935+0000", 2024-09-16T20:48:36.728 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049769300000000002 2024-09-16T20:48:36.728 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.728 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.728 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:36.728 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825991+0000", 2024-09-16T20:48:36.728 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6177000000000001e-05 2024-09-16T20:48:36.728 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.728 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.728 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:36.729 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826048+0000", 2024-09-16T20:48:36.729 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.711e-05 2024-09-16T20:48:36.729 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.729 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:36.729 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.729 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.729 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.729 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5071 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 6934528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:36.729 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.776177+0000", 2024-09-16T20:48:36.729 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 16.865008174, 2024-09-16T20:48:36.729 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050036886000000003, 2024-09-16T20:48:36.730 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:36.730 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:36.730 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:36.730 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:36.730 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:36.730 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:36.730 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5071 2024-09-16T20:48:36.730 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.730 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:36.730 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.730 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:36.730 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776177+0000", 2024-09-16T20:48:36.730 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.731 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.731 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.731 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:36.731 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776177+0000", 2024-09-16T20:48:36.731 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.731 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.731 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.731 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:36.731 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776339+0000", 2024-09-16T20:48:36.731 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016238099999999999 2024-09-16T20:48:36.731 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.731 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.731 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:36.731 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776343+0000", 2024-09-16T20:48:36.732 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9670000000000003e-06 2024-09-16T20:48:36.732 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.732 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.732 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:36.732 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776354+0000", 2024-09-16T20:48:36.732 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0789e-05 2024-09-16T20:48:36.732 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.732 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.732 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:36.732 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776361+0000", 2024-09-16T20:48:36.732 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5200000000000003e-06 2024-09-16T20:48:36.732 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.732 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.733 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:36.733 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826078+0000", 2024-09-16T20:48:36.733 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049717513999999997 2024-09-16T20:48:36.733 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.733 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.733 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:36.733 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826141+0000", 2024-09-16T20:48:36.733 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2446000000000005e-05 2024-09-16T20:48:36.733 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.733 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.733 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:36.733 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826214+0000", 2024-09-16T20:48:36.733 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3269000000000006e-05 2024-09-16T20:48:36.734 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.734 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:36.734 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.734 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.734 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.734 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5072 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2347008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:36.734 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.776370+0000", 2024-09-16T20:48:36.734 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 16.864814613, 2024-09-16T20:48:36.734 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050097653999999998, 2024-09-16T20:48:36.734 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:36.734 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:36.734 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:36.734 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:36.734 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:36.735 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:36.735 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5072 2024-09-16T20:48:36.735 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.735 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:36.735 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.735 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:36.735 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776370+0000", 2024-09-16T20:48:36.735 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.735 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.735 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.735 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:36.735 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776370+0000", 2024-09-16T20:48:36.735 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:36.736 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.736 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.736 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:36.736 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776645+0000", 2024-09-16T20:48:36.736 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000274634 2024-09-16T20:48:36.736 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.736 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.736 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:36.736 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776654+0000", 2024-09-16T20:48:36.736 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9390000000000003e-06 2024-09-16T20:48:36.736 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.736 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.736 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:36.736 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776665+0000", 2024-09-16T20:48:36.737 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0524e-05 2024-09-16T20:48:36.737 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.737 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.737 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:36.737 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776671+0000", 2024-09-16T20:48:36.737 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7390000000000002e-06 2024-09-16T20:48:36.737 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.737 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.737 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:36.737 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826261+0000", 2024-09-16T20:48:36.737 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049589995999999997 2024-09-16T20:48:36.737 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.737 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.737 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:36.738 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826363+0000", 2024-09-16T20:48:36.738 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010158999999999999 2024-09-16T20:48:36.738 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:36.738 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:36.738 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:36.738 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826468+0000", 2024-09-16T20:48:36.738 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000105232 2024-09-16T20:48:36.738 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.738 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:36.738 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.738 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:36.738 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:36.738 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:36.739 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T20:48:36.741 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:36.739+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:36.742 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:48:36.742 INFO:teuthology.orchestra.run.smithi139.stdout:1503 trunc from 0xe465c00 to 0x38ae600 2024-09-16T20:48:36.742 INFO:teuthology.orchestra.run.smithi139.stdout:1504 punch from 0x7c5a00 to 0x7d1a00, (0xc000 bytes) 2024-09-16T20:48:36.742 INFO:teuthology.orchestra.run.smithi139.stdout:1505 write 0x57b2800 thru 0x57b9f2e (0x772f bytes) 2024-09-16T20:48:36.742 INFO:teuthology.orchestra.run.smithi139.stdout:1506 read 0x4025600 thru 0x4029fa6 (0x49a7 bytes) 2024-09-16T20:48:36.743 INFO:teuthology.orchestra.run.smithi139.stdout:1507 write 0x9a06000 thru 0x9a13435 (0xd436 bytes) 2024-09-16T20:48:36.743 INFO:teuthology.orchestra.run.smithi139.stdout:1508 read 0x617a800 thru 0x617cb7d (0x237e bytes) 2024-09-16T20:48:36.743 INFO:teuthology.orchestra.run.smithi139.stdout:1509 punch from 0x1cf4e00 to 0x1d04000, (0xf200 bytes) 2024-09-16T20:48:36.743 INFO:teuthology.orchestra.run.smithi139.stdout:1510 write 0xba71000 thru 0xba757d3 (0x47d4 bytes) 2024-09-16T20:48:36.743 INFO:teuthology.orchestra.run.smithi139.stdout:1511 write 0x9e6d400 thru 0x9e74d9a (0x799b bytes) 2024-09-16T20:48:36.743 INFO:teuthology.orchestra.run.smithi139.stdout:1515 trunc from 0xba75800 to 0x4359600 2024-09-16T20:48:36.746 INFO:teuthology.orchestra.run.smithi139.stdout:1516 punch from 0x382c400 to 0x382dc00, (0x1800 bytes) 2024-09-16T20:48:36.748 INFO:teuthology.orchestra.run.smithi139.stdout:1517 punch from 0x3c1a600 to 0x3c1d800, (0x3200 bytes) 2024-09-16T20:48:36.770 INFO:teuthology.orchestra.run.smithi139.stdout:1519 write 0xdaf8a00 thru 0xdaff3ac (0x69ad bytes) 2024-09-16T20:48:36.770 INFO:teuthology.orchestra.run.smithi139.stdout:1520 write 0x5f38400 thru 0x5f468d9 (0xe4da bytes) 2024-09-16T20:48:36.770 INFO:teuthology.orchestra.run.smithi139.stdout:1522 read 0x5bece00 thru 0x5bf407f (0x7280 bytes) 2024-09-16T20:48:36.771 INFO:teuthology.orchestra.run.smithi139.stdout:1523 write 0xa3c5e00 thru 0xa3ce4c3 (0x86c4 bytes) 2024-09-16T20:48:36.771 INFO:teuthology.orchestra.run.smithi139.stdout:1524 read 0x16e6200 thru 0x16f3468 (0xd269 bytes) 2024-09-16T20:48:36.772 INFO:teuthology.orchestra.run.smithi139.stdout:1525 write 0x4497800 thru 0x44a26e7 (0xaee8 bytes) 2024-09-16T20:48:36.774 INFO:teuthology.orchestra.run.smithi139.stdout:1526 punch from 0xd8a3600 to 0xd8ab600, (0x8000 bytes) 2024-09-16T20:48:36.775 INFO:teuthology.orchestra.run.smithi139.stdout:1527 read 0x981e800 thru 0x98227b2 (0x3fb3 bytes) 2024-09-16T20:48:36.775 INFO:teuthology.orchestra.run.smithi139.stdout:1529 read 0x57ff200 thru 0x580b722 (0xc523 bytes) 2024-09-16T20:48:36.776 INFO:teuthology.orchestra.run.smithi139.stdout:1530 punch from 0xaa7a600 to 0xaa7c200, (0x1c00 bytes) 2024-09-16T20:48:36.776 INFO:teuthology.orchestra.run.smithi139.stdout:1531 read 0xbae8600 thru 0xbaec5d8 (0x3fd9 bytes) 2024-09-16T20:48:36.776 INFO:teuthology.orchestra.run.smithi139.stdout:1532 read 0xb184000 thru 0xb193a50 (0xfa51 bytes) 2024-09-16T20:48:36.789 INFO:teuthology.orchestra.run.smithi139.stdout:1533 write 0xdc87200 thru 0xdc8b332 (0x4133 bytes) 2024-09-16T20:48:36.789 INFO:teuthology.orchestra.run.smithi139.stdout:1535 write 0x5f59200 thru 0x5f5b3f1 (0x21f2 bytes) 2024-09-16T20:48:36.798 INFO:teuthology.orchestra.run.smithi139.stdout:1537 trunc from 0xdc8b400 to 0x73ca600 2024-09-16T20:48:36.818 INFO:teuthology.orchestra.run.smithi139.stdout:1540 read 0x4f8b400 thru 0x4f9989b (0xe49c bytes) 2024-09-16T20:48:36.819 INFO:teuthology.orchestra.run.smithi139.stdout:1542 read 0x5970000 thru 0x5972c2f (0x2c30 bytes) 2024-09-16T20:48:36.820 INFO:teuthology.orchestra.run.smithi139.stdout:1543 read 0x20b800 thru 0x219c72 (0xe473 bytes) 2024-09-16T20:48:36.820 INFO:teuthology.orchestra.run.smithi139.stdout:1544 write 0x62a1200 thru 0x62a8832 (0x7633 bytes) 2024-09-16T20:48:36.837 INFO:teuthology.orchestra.run.smithi139.stdout:1545 write 0x97f2000 thru 0x97f7ce0 (0x5ce1 bytes) 2024-09-16T20:48:36.839 INFO:teuthology.orchestra.run.smithi139.stdout:1551 read 0x6244e00 thru 0x624e810 (0x9a11 bytes) 2024-09-16T20:48:36.841 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:36.839+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:36.842 INFO:teuthology.orchestra.run.smithi139.stdout:1553 punch from 0x3a03400 to 0x3a0cc00, (0x9800 bytes) 2024-09-16T20:48:36.843 INFO:teuthology.orchestra.run.smithi139.stdout:1558 punch from 0x2b19200 to 0x2b24200, (0xb000 bytes) 2024-09-16T20:48:36.843 INFO:teuthology.orchestra.run.smithi139.stdout:1559 write 0x47f7e00 thru 0x47fab17 (0x2d18 bytes) 2024-09-16T20:48:36.845 INFO:teuthology.orchestra.run.smithi139.stdout:1560 write 0x4d98e00 thru 0x4d9cc19 (0x3e1a bytes) 2024-09-16T20:48:36.845 INFO:teuthology.orchestra.run.smithi139.stdout:1563 trunc from 0x97f7e00 to 0x9bbbe00 2024-09-16T20:48:36.857 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:36.857 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:48:36.857 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:48:36.857 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:36.866 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:48:36.873 INFO:teuthology.orchestra.run.smithi139.stdout:1564 write 0xe62e000 thru 0xe6324fe (0x44ff bytes) 2024-09-16T20:48:36.874 INFO:teuthology.orchestra.run.smithi139.stdout:1565 read 0xc3e5e00 thru 0xc3ea7d8 (0x49d9 bytes) 2024-09-16T20:48:36.874 INFO:teuthology.orchestra.run.smithi139.stdout:1566 read 0x7a5e600 thru 0x7a6198e (0x338f bytes) 2024-09-16T20:48:36.877 INFO:teuthology.orchestra.run.smithi139.stdout:1569 read 0x10c1800 thru 0x10c4cf2 (0x34f3 bytes) 2024-09-16T20:48:36.878 INFO:teuthology.orchestra.run.smithi139.stdout:1571 read 0xe168800 thru 0xe17395d (0xb15e bytes) 2024-09-16T20:48:36.878 INFO:teuthology.orchestra.run.smithi139.stdout:1572 write 0x9dc7800 thru 0x9dd0a87 (0x9288 bytes) 2024-09-16T20:48:36.878 INFO:teuthology.orchestra.run.smithi139.stdout:1573 read 0x67ae200 thru 0x67b35dd (0x53de bytes) 2024-09-16T20:48:36.879 INFO:teuthology.orchestra.run.smithi139.stdout:1574 read 0x5c21c00 thru 0x5c2fc70 (0xe071 bytes) 2024-09-16T20:48:36.879 INFO:teuthology.orchestra.run.smithi139.stdout:1576 write 0x9184600 thru 0x919044d (0xbe4e bytes) 2024-09-16T20:48:36.879 INFO:teuthology.orchestra.run.smithi139.stdout:1577 write 0x5ebfa00 thru 0x5ec757c (0x7b7d bytes) 2024-09-16T20:48:36.884 INFO:teuthology.orchestra.run.smithi139.stdout:1580 trunc from 0xe632600 to 0xb053c00 2024-09-16T20:48:36.903 INFO:teuthology.orchestra.run.smithi139.stdout:1581 read 0x3a68000 thru 0x3a6da47 (0x5a48 bytes) 2024-09-16T20:48:36.905 INFO:teuthology.orchestra.run.smithi139.stdout:1585 trunc from 0xb053c00 to 0xa9a6400 2024-09-16T20:48:36.916 INFO:teuthology.orchestra.run.smithi139.stdout:1587 read 0x5e9b600 thru 0x5ea83f1 (0xcdf2 bytes) 2024-09-16T20:48:36.928 INFO:teuthology.orchestra.run.smithi139.stdout:1588 trunc from 0xa9a6400 to 0x16b3400 2024-09-16T20:48:36.941 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:36.940+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:36.971 INFO:teuthology.orchestra.run.smithi139.stdout:1589 write 0xa71ba00 thru 0xa72a8a4 (0xeea5 bytes) 2024-09-16T20:48:36.974 INFO:teuthology.orchestra.run.smithi139.stdout:1590 trunc from 0xa72aa00 to 0xc285c00 2024-09-16T20:48:36.987 INFO:teuthology.orchestra.run.smithi139.stdout:1591 trunc from 0xc285c00 to 0xb47d800 2024-09-16T20:48:36.998 INFO:teuthology.orchestra.run.smithi139.stdout:1592 punch from 0x3567200 to 0x3576600, (0xf400 bytes) 2024-09-16T20:48:37.001 INFO:teuthology.orchestra.run.smithi139.stdout:1594 read 0x1fdc800 thru 0x1fde807 (0x2008 bytes) 2024-09-16T20:48:37.001 INFO:teuthology.orchestra.run.smithi139.stdout:1596 read 0xe34600 thru 0xe3c385 (0x7d86 bytes) 2024-09-16T20:48:37.002 INFO:teuthology.orchestra.run.smithi139.stdout:1598 write 0x7325c00 thru 0x732c09d (0x649e bytes) 2024-09-16T20:48:37.002 INFO:teuthology.orchestra.run.smithi139.stdout:1601 write 0x2755c00 thru 0x275f37c (0x977d bytes) 2024-09-16T20:48:37.002 INFO:teuthology.orchestra.run.smithi139.stdout:1604 punch from 0x4ce00 to 0x54a00, (0x7c00 bytes) 2024-09-16T20:48:37.006 INFO:teuthology.orchestra.run.smithi139.stdout:1605 trunc from 0xb47d800 to 0x84fcc00 2024-09-16T20:48:37.017 INFO:teuthology.orchestra.run.smithi139.stdout:1608 read 0x19c2c00 thru 0x19cce7a (0xa27b bytes) 2024-09-16T20:48:37.019 INFO:teuthology.orchestra.run.smithi139.stdout:1610 write 0x21b4400 thru 0x21bfe28 (0xba29 bytes) 2024-09-16T20:48:37.023 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:37.023 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:48:37.023 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:48:37.023 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:48:37.023 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:37.025 INFO:teuthology.orchestra.run.smithi139.stdout:1614 trunc from 0x84fcc00 to 0x4d4d200 2024-09-16T20:48:37.033 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:48:37.043 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:37.041+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:37.047 INFO:teuthology.orchestra.run.smithi139.stdout:1615 trunc from 0x4d4d200 to 0x95ea400 2024-09-16T20:48:37.063 INFO:teuthology.orchestra.run.smithi139.stdout:1619 write 0x697f800 thru 0x6985e1e (0x661f bytes) 2024-09-16T20:48:37.063 INFO:teuthology.orchestra.run.smithi139.stdout:1621 punch from 0x24a7a00 to 0x24ab600, (0x3c00 bytes) 2024-09-16T20:48:37.063 INFO:teuthology.orchestra.run.smithi139.stdout:1622 punch from 0x38f4200 to 0x3900000, (0xbe00 bytes) 2024-09-16T20:48:37.063 INFO:teuthology.orchestra.run.smithi139.stdout:1623 read 0x779de00 thru 0x77ab605 (0xd806 bytes) 2024-09-16T20:48:37.063 INFO:teuthology.orchestra.run.smithi139.stdout:1627 read 0x890e000 thru 0x8911ace (0x3acf bytes) 2024-09-16T20:48:37.063 INFO:teuthology.orchestra.run.smithi139.stdout:1628 write 0x2d61400 thru 0x2d63040 (0x1c41 bytes) 2024-09-16T20:48:37.064 INFO:teuthology.orchestra.run.smithi139.stdout:1630 read 0x515c00 thru 0x518a02 (0x2e03 bytes) 2024-09-16T20:48:37.064 INFO:teuthology.orchestra.run.smithi139.stdout:1631 punch from 0x5d63000 to 0x5d63e00, (0xe00 bytes) 2024-09-16T20:48:37.065 INFO:teuthology.orchestra.run.smithi139.stdout:1632 write 0x8172200 thru 0x8173966 (0x1767 bytes) 2024-09-16T20:48:37.069 INFO:teuthology.orchestra.run.smithi139.stdout:1634 trunc from 0x95ea400 to 0x73a0e00 2024-09-16T20:48:37.108 INFO:teuthology.orchestra.run.smithi139.stdout:1637 write 0xd6de800 thru 0xd6e415c (0x595d bytes) 2024-09-16T20:48:37.109 INFO:teuthology.orchestra.run.smithi139.stdout:1639 punch from 0x2e1d400 to 0x2e1fc00, (0x2800 bytes) 2024-09-16T20:48:37.109 INFO:teuthology.orchestra.run.smithi139.stdout:1640 read 0x8b85c00 thru 0x8b86073 (0x474 bytes) 2024-09-16T20:48:37.117 INFO:teuthology.orchestra.run.smithi139.stdout:1644 trunc from 0xd6e4200 to 0x5b18a00 2024-09-16T20:48:37.144 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:37.142+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:37.148 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:37.148 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:48:37.149 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:48:37.149 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:48:37.149 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.149 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:780 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 933888~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:37.149 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.239489+0000", 2024-09-16T20:48:37.149 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 21.907504770999999, 2024-09-16T20:48:37.149 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00741699, 2024-09-16T20:48:37.149 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:37.149 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:37.149 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:37.149 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:37.149 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:37.150 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:37.150 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 780 2024-09-16T20:48:37.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.150 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:37.150 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.150 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:37.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239489+0000", 2024-09-16T20:48:37.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:37.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.150 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.150 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:37.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239489+0000", 2024-09-16T20:48:37.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:37.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:37.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239490+0000", 2024-09-16T20:48:37.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8599999999999998e-07 2024-09-16T20:48:37.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:37.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239492+0000", 2024-09-16T20:48:37.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1380000000000001e-06 2024-09-16T20:48:37.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:37.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239499+0000", 2024-09-16T20:48:37.152 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9530000000000002e-06 2024-09-16T20:48:37.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.152 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.152 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:37.152 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239504+0000", 2024-09-16T20:48:37.152 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0690000000000001e-06 2024-09-16T20:48:37.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.152 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.152 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:37.152 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246714+0000", 2024-09-16T20:48:37.152 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.007209844 2024-09-16T20:48:37.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.152 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.153 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:37.153 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246804+0000", 2024-09-16T20:48:37.153 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0109000000000001e-05 2024-09-16T20:48:37.153 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.153 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.153 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:37.153 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246906+0000", 2024-09-16T20:48:37.153 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000102391 2024-09-16T20:48:37.153 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:37.153 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:37.153 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:37.153 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.153 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.154 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:781 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1048576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:37.154 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.239892+0000", 2024-09-16T20:48:37.154 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 21.907102401, 2024-09-16T20:48:37.154 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0072223799999999996, 2024-09-16T20:48:37.154 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:37.154 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:37.154 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:37.154 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:37.154 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:37.154 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:37.154 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 781 2024-09-16T20:48:37.154 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.154 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:37.154 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.155 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:37.155 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239892+0000", 2024-09-16T20:48:37.155 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:37.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.155 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.155 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:37.155 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239892+0000", 2024-09-16T20:48:37.155 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:37.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.155 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.155 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:37.155 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239893+0000", 2024-09-16T20:48:37.155 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1399999999999995e-07 2024-09-16T20:48:37.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.156 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.156 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:37.156 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239897+0000", 2024-09-16T20:48:37.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4859999999999999e-06 2024-09-16T20:48:37.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.156 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.156 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:37.156 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239907+0000", 2024-09-16T20:48:37.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4539999999999996e-06 2024-09-16T20:48:37.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.156 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.156 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:37.156 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239913+0000", 2024-09-16T20:48:37.157 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7880000000000001e-06 2024-09-16T20:48:37.157 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.157 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.157 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:37.157 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246937+0000", 2024-09-16T20:48:37.157 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0070239999999999999 2024-09-16T20:48:37.157 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.157 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.157 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:37.157 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247013+0000", 2024-09-16T20:48:37.157 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5894000000000002e-05 2024-09-16T20:48:37.157 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.157 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.158 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:37.158 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247114+0000", 2024-09-16T20:48:37.158 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000100844 2024-09-16T20:48:37.158 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:37.158 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:37.158 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:37.158 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.158 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.158 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:782 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1146880~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:37.158 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.239925+0000", 2024-09-16T20:48:37.158 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 21.907069428, 2024-09-16T20:48:37.158 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0074161840000000001, 2024-09-16T20:48:37.158 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:37.158 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:37.159 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:37.159 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:37.159 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:37.159 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:37.159 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 782 2024-09-16T20:48:37.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.159 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:37.159 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.159 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:37.159 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239925+0000", 2024-09-16T20:48:37.159 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:37.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.159 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.159 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:37.160 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239925+0000", 2024-09-16T20:48:37.160 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:37.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.160 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.160 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:37.160 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239925+0000", 2024-09-16T20:48:37.160 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7199999999999999e-07 2024-09-16T20:48:37.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.160 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.161 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:37.161 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239928+0000", 2024-09-16T20:48:37.161 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5349999999999999e-06 2024-09-16T20:48:37.161 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.161 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.161 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:37.161 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239935+0000", 2024-09-16T20:48:37.161 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7680000000000003e-06 2024-09-16T20:48:37.161 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.161 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.161 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:37.161 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239940+0000", 2024-09-16T20:48:37.161 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.152e-06 2024-09-16T20:48:37.162 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.162 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.162 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:37.162 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247143+0000", 2024-09-16T20:48:37.162 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0072037309999999997 2024-09-16T20:48:37.162 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.162 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.162 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:37.162 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247230+0000", 2024-09-16T20:48:37.162 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6598e-05 2024-09-16T20:48:37.162 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.162 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.162 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:37.162 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247341+0000", 2024-09-16T20:48:37.163 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011092800000000001 2024-09-16T20:48:37.163 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:37.163 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:37.163 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:37.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.163 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.163 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:783 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1261568~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:37.163 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.239947+0000", 2024-09-16T20:48:37.163 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 21.907047106, 2024-09-16T20:48:37.163 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0076044789999999999, 2024-09-16T20:48:37.163 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:37.163 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:37.163 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:37.163 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:37.164 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:37.164 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:37.164 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 783 2024-09-16T20:48:37.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.164 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:37.164 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.164 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:37.164 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239947+0000", 2024-09-16T20:48:37.164 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:37.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.164 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.164 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:37.164 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239947+0000", 2024-09-16T20:48:37.165 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:37.165 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.165 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.165 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:37.165 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239947+0000", 2024-09-16T20:48:37.165 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9400000000000001e-07 2024-09-16T20:48:37.165 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.165 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.165 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:37.165 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239950+0000", 2024-09-16T20:48:37.165 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2350000000000002e-06 2024-09-16T20:48:37.165 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.165 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.165 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:37.166 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239956+0000", 2024-09-16T20:48:37.166 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3400000000000003e-06 2024-09-16T20:48:37.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.166 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.166 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:37.166 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239961+0000", 2024-09-16T20:48:37.166 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.04e-06 2024-09-16T20:48:37.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.166 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.166 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:37.166 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247371+0000", 2024-09-16T20:48:37.166 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0074096810000000004 2024-09-16T20:48:37.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.166 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.166 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:37.167 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247451+0000", 2024-09-16T20:48:37.167 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9845999999999994e-05 2024-09-16T20:48:37.167 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.167 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.167 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:37.167 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247552+0000", 2024-09-16T20:48:37.167 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000100943 2024-09-16T20:48:37.167 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:37.167 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:37.167 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:37.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.168 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.168 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:784 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1376256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:37.168 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.239997+0000", 2024-09-16T20:48:37.168 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 21.906997449999999, 2024-09-16T20:48:37.168 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.007775372, 2024-09-16T20:48:37.168 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:37.168 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:37.168 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:37.168 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:37.168 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:37.168 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:37.169 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 784 2024-09-16T20:48:37.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.169 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:37.169 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.169 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:37.169 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239997+0000", 2024-09-16T20:48:37.169 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:37.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.169 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.169 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:37.169 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239997+0000", 2024-09-16T20:48:37.169 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:37.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.170 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.170 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:37.170 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239997+0000", 2024-09-16T20:48:37.170 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1799999999999995e-07 2024-09-16T20:48:37.170 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.170 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.170 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:37.170 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240001+0000", 2024-09-16T20:48:37.170 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6009999999999999e-06 2024-09-16T20:48:37.170 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.170 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.170 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:37.170 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240009+0000", 2024-09-16T20:48:37.170 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5809999999999999e-06 2024-09-16T20:48:37.171 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.171 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.171 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:37.171 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240015+0000", 2024-09-16T20:48:37.171 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0100000000000001e-06 2024-09-16T20:48:37.171 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.171 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.171 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:37.171 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247583+0000", 2024-09-16T20:48:37.171 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0075685730000000003 2024-09-16T20:48:37.171 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.171 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.171 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:37.171 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247672+0000", 2024-09-16T20:48:37.172 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8480000000000007e-05 2024-09-16T20:48:37.172 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.172 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.172 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:37.172 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247772+0000", 2024-09-16T20:48:37.172 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000100509 2024-09-16T20:48:37.172 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:37.172 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:37.172 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:37.172 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.172 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.172 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:785 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1474560~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:37.172 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240024+0000", 2024-09-16T20:48:37.173 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 21.906969864000001, 2024-09-16T20:48:37.173 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.008096776, 2024-09-16T20:48:37.173 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:37.173 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:37.173 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:37.173 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:37.173 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:37.173 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:37.173 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 785 2024-09-16T20:48:37.173 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.173 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:37.173 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.173 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:37.173 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240024+0000", 2024-09-16T20:48:37.174 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:37.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.174 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.174 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:37.174 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240024+0000", 2024-09-16T20:48:37.174 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:37.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.174 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.174 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:37.174 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240025+0000", 2024-09-16T20:48:37.174 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5000000000000003e-07 2024-09-16T20:48:37.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.174 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.174 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:37.174 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240027+0000", 2024-09-16T20:48:37.175 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2900000000000001e-06 2024-09-16T20:48:37.175 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.175 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.175 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:37.175 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240036+0000", 2024-09-16T20:48:37.175 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4710000000000007e-06 2024-09-16T20:48:37.175 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.175 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.175 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:37.175 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240041+0000", 2024-09-16T20:48:37.175 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4689999999999996e-06 2024-09-16T20:48:37.175 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.175 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.176 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:37.176 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247821+0000", 2024-09-16T20:48:37.176 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0077797919999999998 2024-09-16T20:48:37.176 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.176 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.176 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:37.176 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247921+0000", 2024-09-16T20:48:37.176 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9814000000000004e-05 2024-09-16T20:48:37.176 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.176 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.176 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:37.176 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248121+0000", 2024-09-16T20:48:37.176 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00020039 2024-09-16T20:48:37.176 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:37.177 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:37.177 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:37.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.177 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.177 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:940 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 81920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:37.177 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240569+0000", 2024-09-16T20:48:37.177 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 21.906425605999999, 2024-09-16T20:48:37.177 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0078289759999999996, 2024-09-16T20:48:37.177 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:37.177 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:37.177 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:37.177 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:37.177 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:37.177 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:37.178 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 940 2024-09-16T20:48:37.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.178 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:37.178 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.178 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:37.178 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240569+0000", 2024-09-16T20:48:37.178 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:37.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.178 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.178 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:37.178 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240569+0000", 2024-09-16T20:48:37.178 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:37.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.178 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.179 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:37.179 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240569+0000", 2024-09-16T20:48:37.179 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2600000000000001e-07 2024-09-16T20:48:37.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.179 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.179 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:37.179 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240575+0000", 2024-09-16T20:48:37.179 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1159999999999998e-06 2024-09-16T20:48:37.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.179 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.179 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:37.179 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240587+0000", 2024-09-16T20:48:37.179 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1915e-05 2024-09-16T20:48:37.180 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.180 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.180 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:37.180 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240594+0000", 2024-09-16T20:48:37.180 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5610000000000001e-06 2024-09-16T20:48:37.180 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.180 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.180 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:37.180 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248158+0000", 2024-09-16T20:48:37.180 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0075635370000000004 2024-09-16T20:48:37.180 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.180 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.180 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:37.180 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248278+0000", 2024-09-16T20:48:37.181 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012009400000000001 2024-09-16T20:48:37.181 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.181 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.181 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:37.181 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248398+0000", 2024-09-16T20:48:37.181 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000119827 2024-09-16T20:48:37.181 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:37.181 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:37.181 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:37.181 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.181 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.181 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:941 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 196608~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:37.181 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240608+0000", 2024-09-16T20:48:37.181 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 21.906386029, 2024-09-16T20:48:37.182 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0080495519999999997, 2024-09-16T20:48:37.182 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:37.182 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:37.182 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:37.182 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:37.182 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:37.182 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:37.182 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 941 2024-09-16T20:48:37.182 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.182 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:37.182 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.182 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:37.182 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240608+0000", 2024-09-16T20:48:37.182 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:37.182 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.183 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.183 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:37.183 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240608+0000", 2024-09-16T20:48:37.183 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:37.183 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.183 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.183 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:37.183 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240609+0000", 2024-09-16T20:48:37.183 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7000000000000004e-07 2024-09-16T20:48:37.183 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.183 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.183 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:37.183 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240612+0000", 2024-09-16T20:48:37.183 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8310000000000001e-06 2024-09-16T20:48:37.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.184 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.184 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:37.184 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240619+0000", 2024-09-16T20:48:37.184 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8429999999999993e-06 2024-09-16T20:48:37.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.184 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.184 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:37.184 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240625+0000", 2024-09-16T20:48:37.184 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5659999999999997e-06 2024-09-16T20:48:37.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.184 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.184 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:37.185 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248456+0000", 2024-09-16T20:48:37.185 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0078313940000000002 2024-09-16T20:48:37.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.185 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.185 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:37.185 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248548+0000", 2024-09-16T20:48:37.185 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1324999999999997e-05 2024-09-16T20:48:37.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.185 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.185 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:37.185 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248658+0000", 2024-09-16T20:48:37.185 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000109923 2024-09-16T20:48:37.185 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:37.186 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:37.186 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:37.186 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.186 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.186 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:942 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 294912~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:37.186 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240675+0000", 2024-09-16T20:48:37.186 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 21.906319222, 2024-09-16T20:48:37.186 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0082135699999999999, 2024-09-16T20:48:37.186 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:37.186 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:37.186 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:37.186 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:37.186 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:37.186 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:37.186 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 942 2024-09-16T20:48:37.187 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.187 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:37.187 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.187 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:37.187 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240675+0000", 2024-09-16T20:48:37.187 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:37.187 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.187 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.187 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:37.187 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240675+0000", 2024-09-16T20:48:37.187 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:37.187 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.187 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.188 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:37.188 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240676+0000", 2024-09-16T20:48:37.188 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.37e-07 2024-09-16T20:48:37.188 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.188 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.188 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:37.188 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240679+0000", 2024-09-16T20:48:37.188 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4060000000000001e-06 2024-09-16T20:48:37.188 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.188 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.188 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:37.188 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240687+0000", 2024-09-16T20:48:37.188 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4640000000000006e-06 2024-09-16T20:48:37.188 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.189 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.189 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:37.189 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240694+0000", 2024-09-16T20:48:37.189 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.246e-06 2024-09-16T20:48:37.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.189 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.189 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:37.189 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248693+0000", 2024-09-16T20:48:37.189 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0079997550000000008 2024-09-16T20:48:37.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.189 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.189 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:37.189 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248781+0000", 2024-09-16T20:48:37.189 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7509999999999994e-05 2024-09-16T20:48:37.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.190 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.190 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:37.190 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248889+0000", 2024-09-16T20:48:37.190 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000107552 2024-09-16T20:48:37.190 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:37.190 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:37.190 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:37.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.190 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.190 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:943 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 409600~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:37.190 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240704+0000", 2024-09-16T20:48:37.190 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 21.906290225999999, 2024-09-16T20:48:37.190 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0083978209999999998, 2024-09-16T20:48:37.191 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:37.191 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:37.191 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:37.191 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:37.191 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:37.191 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:37.191 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 943 2024-09-16T20:48:37.191 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.191 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:37.191 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.191 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:37.191 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240704+0000", 2024-09-16T20:48:37.191 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:37.192 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.192 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.192 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:37.192 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240704+0000", 2024-09-16T20:48:37.192 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:37.192 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.192 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.192 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:37.192 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240704+0000", 2024-09-16T20:48:37.192 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7e-07 2024-09-16T20:48:37.192 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.192 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.192 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:37.193 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240707+0000", 2024-09-16T20:48:37.193 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5809999999999999e-06 2024-09-16T20:48:37.193 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.193 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.193 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:37.193 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240713+0000", 2024-09-16T20:48:37.193 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4949999999999996e-06 2024-09-16T20:48:37.193 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.193 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.193 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:37.193 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240718+0000", 2024-09-16T20:48:37.193 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8890000000000001e-06 2024-09-16T20:48:37.193 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.193 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.194 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:37.194 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248920+0000", 2024-09-16T20:48:37.194 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0082018679999999993 2024-09-16T20:48:37.194 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.194 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.194 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:37.194 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249007+0000", 2024-09-16T20:48:37.194 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6327000000000002e-05 2024-09-16T20:48:37.194 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.194 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.194 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:37.194 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249102+0000", 2024-09-16T20:48:37.194 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5191000000000004e-05 2024-09-16T20:48:37.194 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:37.195 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:37.195 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:37.195 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.195 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.195 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:944 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 524288~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:37.195 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240726+0000", 2024-09-16T20:48:37.195 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 21.906268421, 2024-09-16T20:48:37.195 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0085999960000000004, 2024-09-16T20:48:37.195 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:37.195 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:37.195 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:37.195 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:37.195 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:37.195 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:37.196 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 944 2024-09-16T20:48:37.196 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.196 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:37.196 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.196 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:37.196 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240726+0000", 2024-09-16T20:48:37.196 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:37.196 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.196 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.196 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:37.196 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240726+0000", 2024-09-16T20:48:37.196 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:37.196 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.197 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.197 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:37.197 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240726+0000", 2024-09-16T20:48:37.197 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7099999999999997e-07 2024-09-16T20:48:37.197 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.197 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.197 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:37.197 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240728+0000", 2024-09-16T20:48:37.197 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.125e-06 2024-09-16T20:48:37.197 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.197 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.197 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:37.197 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240734+0000", 2024-09-16T20:48:37.197 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1890000000000004e-06 2024-09-16T20:48:37.198 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.198 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.198 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:37.198 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240739+0000", 2024-09-16T20:48:37.198 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0340000000000004e-06 2024-09-16T20:48:37.198 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.198 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.198 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:37.198 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249131+0000", 2024-09-16T20:48:37.198 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0083919579999999997 2024-09-16T20:48:37.198 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.198 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.198 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:37.199 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249217+0000", 2024-09-16T20:48:37.199 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5154999999999994e-05 2024-09-16T20:48:37.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.199 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.199 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:37.199 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249326+0000", 2024-09-16T20:48:37.199 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000109164 2024-09-16T20:48:37.199 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:37.199 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:37.199 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:37.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.199 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.199 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:945 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 622592~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:37.199 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240746+0000", 2024-09-16T20:48:37.200 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 21.906248282, 2024-09-16T20:48:37.200 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0088083320000000003, 2024-09-16T20:48:37.200 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:37.200 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:37.200 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:37.200 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:37.200 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:37.200 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:37.200 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 945 2024-09-16T20:48:37.200 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.200 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:37.200 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.200 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:37.200 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240746+0000", 2024-09-16T20:48:37.201 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:37.201 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.201 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.201 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:37.201 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240746+0000", 2024-09-16T20:48:37.201 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:37.201 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.201 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.201 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:37.201 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240746+0000", 2024-09-16T20:48:37.201 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8500000000000002e-07 2024-09-16T20:48:37.201 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.201 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.202 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:37.202 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240748+0000", 2024-09-16T20:48:37.202 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8449999999999999e-06 2024-09-16T20:48:37.202 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.202 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.202 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:37.202 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240754+0000", 2024-09-16T20:48:37.202 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8830000000000003e-06 2024-09-16T20:48:37.202 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.202 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.202 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:37.202 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240759+0000", 2024-09-16T20:48:37.202 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0220000000000004e-06 2024-09-16T20:48:37.202 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.203 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.203 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:37.203 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249360+0000", 2024-09-16T20:48:37.203 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.008601437 2024-09-16T20:48:37.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.203 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.203 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:37.203 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249453+0000", 2024-09-16T20:48:37.203 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2743000000000003e-05 2024-09-16T20:48:37.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.203 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.203 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:37.203 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249554+0000", 2024-09-16T20:48:37.203 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101017 2024-09-16T20:48:37.204 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:37.204 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:37.204 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:37.204 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.204 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.204 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:946 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 737280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:37.204 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240766+0000", 2024-09-16T20:48:37.204 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 21.906228579, 2024-09-16T20:48:37.204 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0089092070000000006, 2024-09-16T20:48:37.204 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:37.204 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:37.204 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:37.204 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:37.205 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:37.205 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:37.205 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 946 2024-09-16T20:48:37.205 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.205 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:37.205 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.205 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:37.205 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240766+0000", 2024-09-16T20:48:37.205 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:37.205 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.205 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.205 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:37.205 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240766+0000", 2024-09-16T20:48:37.205 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:37.206 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.206 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.206 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:37.206 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240766+0000", 2024-09-16T20:48:37.206 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0600000000000001e-07 2024-09-16T20:48:37.206 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.206 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.206 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:37.206 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240768+0000", 2024-09-16T20:48:37.206 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8959999999999999e-06 2024-09-16T20:48:37.206 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.206 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.206 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:37.207 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240774+0000", 2024-09-16T20:48:37.207 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0979999999999999e-06 2024-09-16T20:48:37.207 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.207 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.207 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:37.207 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240779+0000", 2024-09-16T20:48:37.207 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.933e-06 2024-09-16T20:48:37.207 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.207 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.207 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:37.207 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249572+0000", 2024-09-16T20:48:37.207 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0087926380000000002 2024-09-16T20:48:37.207 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.208 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.208 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:37.208 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249612+0000", 2024-09-16T20:48:37.208 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0859000000000003e-05 2024-09-16T20:48:37.208 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.208 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.208 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:37.208 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249675+0000", 2024-09-16T20:48:37.208 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2377000000000003e-05 2024-09-16T20:48:37.208 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:37.208 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:37.208 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:37.208 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.209 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.209 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:947 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 851968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:37.209 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240785+0000", 2024-09-16T20:48:37.209 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 21.906208986999999, 2024-09-16T20:48:37.209 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0090960260000000001, 2024-09-16T20:48:37.209 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:37.209 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:37.209 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:37.209 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:37.209 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:37.209 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:37.209 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 947 2024-09-16T20:48:37.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.209 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:37.210 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.210 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:37.210 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240785+0000", 2024-09-16T20:48:37.210 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:37.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.210 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.210 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:37.210 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240785+0000", 2024-09-16T20:48:37.210 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:37.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.210 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.210 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:37.210 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240786+0000", 2024-09-16T20:48:37.210 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-16T20:48:37.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.211 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.211 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:37.211 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240787+0000", 2024-09-16T20:48:37.211 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.858e-06 2024-09-16T20:48:37.211 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.211 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.211 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:37.211 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240793+0000", 2024-09-16T20:48:37.211 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7760000000000003e-06 2024-09-16T20:48:37.211 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.211 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.211 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:37.211 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240799+0000", 2024-09-16T20:48:37.211 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1109999999999996e-06 2024-09-16T20:48:37.212 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.212 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.212 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:37.212 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249715+0000", 2024-09-16T20:48:37.212 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0089159789999999992 2024-09-16T20:48:37.212 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.212 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.212 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:37.212 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249766+0000", 2024-09-16T20:48:37.212 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0968999999999999e-05 2024-09-16T20:48:37.212 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.212 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.212 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:37.213 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249881+0000", 2024-09-16T20:48:37.213 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114963 2024-09-16T20:48:37.213 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:37.213 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:37.213 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:37.213 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.213 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.213 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:948 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:37.213 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240806+0000", 2024-09-16T20:48:37.213 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 21.906187806999998, 2024-09-16T20:48:37.213 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0092300609999999995, 2024-09-16T20:48:37.213 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:37.213 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:37.214 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:37.214 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:37.214 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:37.214 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:37.214 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 948 2024-09-16T20:48:37.214 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.214 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:37.214 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.214 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:37.214 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240806+0000", 2024-09-16T20:48:37.214 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:37.214 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.214 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.214 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:37.215 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240806+0000", 2024-09-16T20:48:37.215 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:37.215 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.215 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.215 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:37.215 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240807+0000", 2024-09-16T20:48:37.215 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4499999999999998e-07 2024-09-16T20:48:37.215 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.215 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.215 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:37.215 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240809+0000", 2024-09-16T20:48:37.215 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.34e-06 2024-09-16T20:48:37.215 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.215 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.216 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:37.216 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240815+0000", 2024-09-16T20:48:37.216 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1449999999999996e-06 2024-09-16T20:48:37.216 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.216 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.216 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:37.216 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240821+0000", 2024-09-16T20:48:37.216 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0270000000000001e-06 2024-09-16T20:48:37.216 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.216 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.216 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:37.216 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249920+0000", 2024-09-16T20:48:37.216 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0090988460000000007 2024-09-16T20:48:37.217 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.217 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.217 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:37.217 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249966+0000", 2024-09-16T20:48:37.217 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5918999999999999e-05 2024-09-16T20:48:37.217 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.217 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.217 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:37.217 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250036+0000", 2024-09-16T20:48:37.217 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0438999999999994e-05 2024-09-16T20:48:37.217 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:37.217 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:37.217 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:37.218 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.218 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.218 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:949 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1064960~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:37.218 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240833+0000", 2024-09-16T20:48:37.218 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 21.906161191999999, 2024-09-16T20:48:37.218 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0093580339999999994, 2024-09-16T20:48:37.218 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:37.218 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:37.218 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:37.218 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:37.218 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:37.218 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:37.218 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 949 2024-09-16T20:48:37.219 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.219 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:37.219 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.219 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:37.219 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240833+0000", 2024-09-16T20:48:37.219 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:37.219 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.219 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.219 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:37.219 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240833+0000", 2024-09-16T20:48:37.219 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:37.219 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.219 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.219 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:37.220 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240833+0000", 2024-09-16T20:48:37.220 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2899999999999999e-07 2024-09-16T20:48:37.220 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.220 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.220 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:37.220 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240836+0000", 2024-09-16T20:48:37.220 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2649999999999999e-06 2024-09-16T20:48:37.220 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.220 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.220 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:37.220 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240842+0000", 2024-09-16T20:48:37.220 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5579999999999997e-06 2024-09-16T20:48:37.220 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.221 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.221 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:37.221 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240847+0000", 2024-09-16T20:48:37.221 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2009999999999998e-06 2024-09-16T20:48:37.221 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.221 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.221 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:37.221 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250073+0000", 2024-09-16T20:48:37.221 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0092258710000000001 2024-09-16T20:48:37.221 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.221 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.221 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:37.221 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250141+0000", 2024-09-16T20:48:37.221 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7849999999999996e-05 2024-09-16T20:48:37.222 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.222 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.222 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:37.222 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250191+0000", 2024-09-16T20:48:37.222 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.986e-05 2024-09-16T20:48:37.222 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:37.222 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:37.222 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:37.222 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.222 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.223 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:950 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1179648~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:37.223 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240855+0000", 2024-09-16T20:48:37.223 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 21.906138939999998, 2024-09-16T20:48:37.223 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0095855560000000003, 2024-09-16T20:48:37.223 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:37.223 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:37.223 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:37.223 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:37.223 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:37.223 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:37.223 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 950 2024-09-16T20:48:37.223 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.223 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:37.224 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.224 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:37.224 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240855+0000", 2024-09-16T20:48:37.224 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:37.224 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.224 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.224 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:37.224 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240855+0000", 2024-09-16T20:48:37.224 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:37.224 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.224 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.224 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:37.224 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240856+0000", 2024-09-16T20:48:37.224 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4999999999999998e-07 2024-09-16T20:48:37.225 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.225 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.225 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:37.225 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240858+0000", 2024-09-16T20:48:37.225 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.996e-06 2024-09-16T20:48:37.225 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.225 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.225 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:37.225 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240864+0000", 2024-09-16T20:48:37.225 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9359999999999996e-06 2024-09-16T20:48:37.225 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.225 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.225 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:37.226 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240868+0000", 2024-09-16T20:48:37.226 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7670000000000003e-06 2024-09-16T20:48:37.226 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.226 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.226 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:37.226 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250226+0000", 2024-09-16T20:48:37.226 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0093576100000000006 2024-09-16T20:48:37.226 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.226 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.226 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:37.226 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250289+0000", 2024-09-16T20:48:37.226 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2627999999999997e-05 2024-09-16T20:48:37.226 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.226 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.226 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:37.227 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250441+0000", 2024-09-16T20:48:37.227 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015216900000000001 2024-09-16T20:48:37.227 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:37.227 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:37.227 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:37.227 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.227 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.227 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:951 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1277952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:37.227 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.241044+0000", 2024-09-16T20:48:37.227 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 21.905949752000001, 2024-09-16T20:48:37.227 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0096266589999999992, 2024-09-16T20:48:37.227 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:37.227 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:37.227 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:37.228 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:37.228 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:37.228 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:37.228 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 951 2024-09-16T20:48:37.228 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.228 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:37.228 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.228 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:37.228 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241044+0000", 2024-09-16T20:48:37.228 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:37.228 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.228 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.228 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:37.229 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241044+0000", 2024-09-16T20:48:37.229 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:37.229 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.229 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.229 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:37.229 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241045+0000", 2024-09-16T20:48:37.229 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5300000000000004e-07 2024-09-16T20:48:37.229 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.229 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.229 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:37.229 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241048+0000", 2024-09-16T20:48:37.229 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.303e-06 2024-09-16T20:48:37.229 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.229 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.230 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:37.230 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241057+0000", 2024-09-16T20:48:37.230 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5550000000000003e-06 2024-09-16T20:48:37.230 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.230 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.230 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:37.230 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241063+0000", 2024-09-16T20:48:37.230 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3470000000000004e-06 2024-09-16T20:48:37.230 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.230 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.231 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:37.231 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250477+0000", 2024-09-16T20:48:37.231 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0094136159999999997 2024-09-16T20:48:37.231 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.231 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.231 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:37.231 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250562+0000", 2024-09-16T20:48:37.231 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4709000000000005e-05 2024-09-16T20:48:37.231 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.231 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.231 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:37.231 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250671+0000", 2024-09-16T20:48:37.231 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010947599999999999 2024-09-16T20:48:37.231 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:37.232 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:37.232 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:37.232 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.232 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.232 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:952 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1392640~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:37.232 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.241130+0000", 2024-09-16T20:48:37.232 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 21.905863885999999, 2024-09-16T20:48:37.232 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0098096290000000003, 2024-09-16T20:48:37.232 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:37.232 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:37.232 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:37.232 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:37.232 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:37.232 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:37.233 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 952 2024-09-16T20:48:37.233 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.233 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:37.233 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.233 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:37.233 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241130+0000", 2024-09-16T20:48:37.233 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:37.233 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.234 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.234 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:37.234 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241130+0000", 2024-09-16T20:48:37.234 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:37.234 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.234 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.234 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:37.234 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241131+0000", 2024-09-16T20:48:37.234 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.61e-07 2024-09-16T20:48:37.234 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.234 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.234 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:37.234 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241134+0000", 2024-09-16T20:48:37.234 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1080000000000001e-06 2024-09-16T20:48:37.235 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.235 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.235 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:37.235 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241142+0000", 2024-09-16T20:48:37.235 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9119999999999998e-06 2024-09-16T20:48:37.235 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.235 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.235 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:37.235 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241148+0000", 2024-09-16T20:48:37.235 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1410000000000002e-06 2024-09-16T20:48:37.235 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.235 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.235 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:37.235 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250731+0000", 2024-09-16T20:48:37.236 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0095833979999999999 2024-09-16T20:48:37.236 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.236 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.236 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:37.236 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250820+0000", 2024-09-16T20:48:37.236 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8997000000000002e-05 2024-09-16T20:48:37.236 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.236 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.236 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:37.236 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250940+0000", 2024-09-16T20:48:37.236 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000119412 2024-09-16T20:48:37.236 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:37.236 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:37.236 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:37.237 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.237 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.237 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:953 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1507328~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:37.237 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.241157+0000", 2024-09-16T20:48:37.237 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 21.905836964999999, 2024-09-16T20:48:37.237 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.010039259, 2024-09-16T20:48:37.237 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:37.237 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:37.237 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:37.237 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:37.237 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:37.237 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:37.237 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 953 2024-09-16T20:48:37.238 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.238 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:37.238 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.238 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:37.238 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241157+0000", 2024-09-16T20:48:37.238 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:37.238 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.238 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.238 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:37.238 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241157+0000", 2024-09-16T20:48:37.238 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:37.238 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.238 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.239 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:37.239 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241158+0000", 2024-09-16T20:48:37.239 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.01e-07 2024-09-16T20:48:37.239 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.239 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.239 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:37.239 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241160+0000", 2024-09-16T20:48:37.239 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2879999999999999e-06 2024-09-16T20:48:37.239 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.239 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.239 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:37.239 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241166+0000", 2024-09-16T20:48:37.239 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.421e-06 2024-09-16T20:48:37.239 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.240 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.240 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:37.240 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241171+0000", 2024-09-16T20:48:37.240 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9840000000000001e-06 2024-09-16T20:48:37.240 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.240 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.240 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:37.240 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250988+0000", 2024-09-16T20:48:37.240 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0098171449999999993 2024-09-16T20:48:37.240 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.240 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.240 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:37.240 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.251085+0000", 2024-09-16T20:48:37.240 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6877000000000001e-05 2024-09-16T20:48:37.241 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:37.241 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:37.241 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:37.241 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.251196+0000", 2024-09-16T20:48:37.241 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000111143 2024-09-16T20:48:37.241 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:37.241 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:37.241 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:37.241 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:37.241 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:37.241 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:37.243 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:48:37.245 INFO:teuthology.orchestra.run.smithi139.stdout:1645 write 0xed06a00 thru 0xed078fa (0xefb bytes) 2024-09-16T20:48:37.245 INFO:teuthology.orchestra.run.smithi139.stdout:1648 punch from 0xaddfc00 to 0xade8a00, (0x8e00 bytes) 2024-09-16T20:48:37.245 INFO:teuthology.orchestra.run.smithi139.stdout:1649 read 0x91f3800 thru 0x92008eb (0xd0ec bytes) 2024-09-16T20:48:37.245 INFO:teuthology.orchestra.run.smithi139.stdout:1650 write 0x6685c00 thru 0x66918ed (0xbcee bytes) 2024-09-16T20:48:37.245 INFO:teuthology.orchestra.run.smithi139.stdout:1651 write 0x2434a00 thru 0x2440111 (0xb712 bytes) 2024-09-16T20:48:37.245 INFO:teuthology.orchestra.run.smithi139.stdout:1652 read 0x782e800 thru 0x783bf89 (0xd78a bytes) 2024-09-16T20:48:37.245 INFO:teuthology.orchestra.run.smithi139.stdout:1653 punch from 0xd48ae00 to 0xd48ec00, (0x3e00 bytes) 2024-09-16T20:48:37.245 INFO:teuthology.orchestra.run.smithi139.stdout:1654 punch from 0x8565600 to 0x8570c00, (0xb600 bytes) 2024-09-16T20:48:37.245 INFO:teuthology.orchestra.run.smithi139.stdout:1655 read 0x50c3400 thru 0x50c925a (0x5e5b bytes) 2024-09-16T20:48:37.245 INFO:teuthology.orchestra.run.smithi139.stdout:1656 read 0x91fa400 thru 0x920528e (0xae8f bytes) 2024-09-16T20:48:37.246 INFO:teuthology.orchestra.run.smithi139.stdout:1657 write 0x19baa00 thru 0x19c705c (0xc65d bytes) 2024-09-16T20:48:37.246 INFO:teuthology.orchestra.run.smithi139.stdout:1658 read 0xd0c800 thru 0xd0d541 (0xd42 bytes) 2024-09-16T20:48:37.246 INFO:teuthology.orchestra.run.smithi139.stdout:1659 write 0x98c8000 thru 0x98d77cc (0xf7cd bytes) 2024-09-16T20:48:37.246 INFO:teuthology.orchestra.run.smithi139.stdout:1663 read 0x3287e00 thru 0x328bdbd (0x3fbe bytes) 2024-09-16T20:48:37.246 INFO:teuthology.orchestra.run.smithi139.stdout:1664 read 0xa1eb200 thru 0xa1fa98b (0xf78c bytes) 2024-09-16T20:48:37.246 INFO:teuthology.orchestra.run.smithi139.stdout:1665 write 0x3f50800 thru 0x3f57a1c (0x721d bytes) 2024-09-16T20:48:37.246 INFO:teuthology.orchestra.run.smithi139.stdout:1670 read 0xb8a5c00 thru 0xb8a7adb (0x1edc bytes) 2024-09-16T20:48:37.246 INFO:teuthology.orchestra.run.smithi139.stdout:1674 write 0x9374800 thru 0x9382c32 (0xe433 bytes) 2024-09-16T20:48:37.246 INFO:teuthology.orchestra.run.smithi139.stdout:1676 write 0xd1a400 thru 0xd23d8e (0x998f bytes) 2024-09-16T20:48:37.246 INFO:teuthology.orchestra.run.smithi139.stdout:1679 read 0x59d5e00 thru 0x59e5d9e (0xff9f bytes) 2024-09-16T20:48:37.246 INFO:teuthology.orchestra.run.smithi139.stdout:1680 punch from 0x3668200 to 0x3671e00, (0x9c00 bytes) 2024-09-16T20:48:37.246 INFO:teuthology.orchestra.run.smithi139.stdout:1681 clone 12 order 23 su 262144 sc 10 2024-09-16T20:48:37.285 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:37.283+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:37.348 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:37.347+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:37.353 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:37.353 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:48:37.353 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:48:37.353 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:37.361 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:48:37.449 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:37.448+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:37.468 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:37.468 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:48:37.468 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:48:37.468 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:48:37.468 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:37.477 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:48:37.550 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:37.548+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:37.625 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:37.625 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:48:37.625 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:48:37.625 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:48:37.625 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.625 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2167 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 907264~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:37.625 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335615+0000", 2024-09-16T20:48:37.625 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.28759647, 2024-09-16T20:48:37.625 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046952761000000003, 2024-09-16T20:48:37.626 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:37.626 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:37.626 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:37.626 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:37.626 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:37.626 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:37.626 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2167 2024-09-16T20:48:37.626 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.626 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:37.626 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.626 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:37.626 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335615+0000", 2024-09-16T20:48:37.626 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:37.627 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.627 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.627 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:37.627 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335615+0000", 2024-09-16T20:48:37.627 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:37.627 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.627 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.627 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:37.627 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335616+0000", 2024-09-16T20:48:37.627 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2300000000000001e-07 2024-09-16T20:48:37.627 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.628 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.628 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:37.628 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335618+0000", 2024-09-16T20:48:37.628 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3269999999999999e-06 2024-09-16T20:48:37.628 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.628 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.628 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:37.628 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335625+0000", 2024-09-16T20:48:37.628 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1930000000000004e-06 2024-09-16T20:48:37.628 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.628 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.628 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:37.628 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335630+0000", 2024-09-16T20:48:37.629 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2460000000000003e-06 2024-09-16T20:48:37.629 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.629 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.629 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:37.629 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.382377+0000", 2024-09-16T20:48:37.629 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046746256 2024-09-16T20:48:37.629 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.629 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.629 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:37.629 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.382474+0000", 2024-09-16T20:48:37.629 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.7388999999999999e-05 2024-09-16T20:48:37.629 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.629 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.630 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:37.630 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.382568+0000", 2024-09-16T20:48:37.630 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.3826999999999995e-05 2024-09-16T20:48:37.630 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:37.630 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:37.630 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:37.630 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.630 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.630 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2170 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 915456~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:37.630 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335637+0000", 2024-09-16T20:48:37.630 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.287574320999999, 2024-09-16T20:48:37.630 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047182843000000002, 2024-09-16T20:48:37.631 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:37.631 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:37.631 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:37.631 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:37.631 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:37.631 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:37.631 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2170 2024-09-16T20:48:37.631 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.631 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:37.632 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.632 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:37.632 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335637+0000", 2024-09-16T20:48:37.632 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:37.632 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.632 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.632 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:37.632 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335637+0000", 2024-09-16T20:48:37.632 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:37.632 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.632 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.632 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:37.633 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335638+0000", 2024-09-16T20:48:37.633 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6699999999999999e-07 2024-09-16T20:48:37.633 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.633 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.633 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:37.633 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335639+0000", 2024-09-16T20:48:37.633 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8339999999999999e-06 2024-09-16T20:48:37.633 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.633 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.633 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:37.633 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335646+0000", 2024-09-16T20:48:37.633 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6599999999999998e-06 2024-09-16T20:48:37.634 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.634 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.634 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:37.634 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335651+0000", 2024-09-16T20:48:37.634 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0289999999999996e-06 2024-09-16T20:48:37.634 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.634 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.634 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:37.634 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.382616+0000", 2024-09-16T20:48:37.634 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046965066 2024-09-16T20:48:37.634 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.634 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.634 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:37.634 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.382708+0000", 2024-09-16T20:48:37.635 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2177999999999997e-05 2024-09-16T20:48:37.635 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.635 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.635 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:37.635 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.382820+0000", 2024-09-16T20:48:37.635 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000111709 2024-09-16T20:48:37.635 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:37.635 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:37.635 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:37.636 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.636 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.636 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2172 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 923648~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:37.636 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335658+0000", 2024-09-16T20:48:37.636 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.287553414, 2024-09-16T20:48:37.636 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047394824000000002, 2024-09-16T20:48:37.636 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:37.636 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:37.636 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:37.636 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:37.636 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:37.636 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:37.636 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2172 2024-09-16T20:48:37.637 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.637 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:37.637 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.637 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:37.637 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335658+0000", 2024-09-16T20:48:37.637 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:37.637 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.637 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.637 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:37.637 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335658+0000", 2024-09-16T20:48:37.637 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:37.638 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.638 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.638 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:37.638 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335658+0000", 2024-09-16T20:48:37.638 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8599999999999999e-07 2024-09-16T20:48:37.638 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.638 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.638 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:37.638 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335660+0000", 2024-09-16T20:48:37.638 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.95e-06 2024-09-16T20:48:37.638 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.638 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.638 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:37.638 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335667+0000", 2024-09-16T20:48:37.639 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.815e-06 2024-09-16T20:48:37.639 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.639 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.639 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:37.639 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335672+0000", 2024-09-16T20:48:37.639 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8040000000000001e-06 2024-09-16T20:48:37.639 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.639 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.639 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:37.640 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.382864+0000", 2024-09-16T20:48:37.640 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047192125000000001 2024-09-16T20:48:37.640 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.640 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.640 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:37.640 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.382950+0000", 2024-09-16T20:48:37.640 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5594999999999996e-05 2024-09-16T20:48:37.640 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.640 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.640 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:37.640 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383053+0000", 2024-09-16T20:48:37.640 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000103149 2024-09-16T20:48:37.640 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:37.641 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:37.641 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:37.641 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.641 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.641 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2174 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 931840~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:37.641 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335679+0000", 2024-09-16T20:48:37.641 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.287532540999999, 2024-09-16T20:48:37.641 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047619511000000003, 2024-09-16T20:48:37.641 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:37.641 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:37.641 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:37.641 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:37.642 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:37.642 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:37.642 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2174 2024-09-16T20:48:37.642 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.642 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:37.642 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.642 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:37.642 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335679+0000", 2024-09-16T20:48:37.642 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:37.642 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.642 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.642 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:37.642 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335679+0000", 2024-09-16T20:48:37.643 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:37.643 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.643 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.643 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:37.643 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335679+0000", 2024-09-16T20:48:37.643 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7800000000000002e-07 2024-09-16T20:48:37.643 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.643 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.643 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:37.643 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335682+0000", 2024-09-16T20:48:37.643 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9490000000000001e-06 2024-09-16T20:48:37.643 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.643 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.644 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:37.644 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335689+0000", 2024-09-16T20:48:37.644 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8220000000000001e-06 2024-09-16T20:48:37.644 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.644 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.644 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:37.644 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335695+0000", 2024-09-16T20:48:37.644 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0569999999999998e-06 2024-09-16T20:48:37.644 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.644 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.644 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:37.644 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383103+0000", 2024-09-16T20:48:37.644 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047408219000000001 2024-09-16T20:48:37.645 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.645 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.645 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:37.645 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383189+0000", 2024-09-16T20:48:37.645 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6007000000000005e-05 2024-09-16T20:48:37.645 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.645 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.645 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:37.645 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383298+0000", 2024-09-16T20:48:37.645 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000109079 2024-09-16T20:48:37.645 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:37.645 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:37.645 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:37.646 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.646 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.646 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2176 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 940032~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:37.646 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335702+0000", 2024-09-16T20:48:37.646 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.287509312999999, 2024-09-16T20:48:37.646 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047804450999999998, 2024-09-16T20:48:37.646 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:37.646 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:37.646 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:37.646 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:37.646 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:37.647 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:37.647 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2176 2024-09-16T20:48:37.647 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.647 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:37.647 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.647 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:37.647 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335702+0000", 2024-09-16T20:48:37.647 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:37.647 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.647 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.647 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:37.647 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335702+0000", 2024-09-16T20:48:37.647 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:37.647 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.648 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.648 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:37.648 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335703+0000", 2024-09-16T20:48:37.648 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.34e-07 2024-09-16T20:48:37.648 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.648 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.648 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:37.648 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335704+0000", 2024-09-16T20:48:37.648 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.917e-06 2024-09-16T20:48:37.648 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.648 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.648 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:37.649 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335711+0000", 2024-09-16T20:48:37.649 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4400000000000002e-06 2024-09-16T20:48:37.649 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.649 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.649 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:37.649 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335717+0000", 2024-09-16T20:48:37.649 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6679999999999998e-06 2024-09-16T20:48:37.649 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.649 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.649 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:37.649 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383335+0000", 2024-09-16T20:48:37.649 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047618754999999999 2024-09-16T20:48:37.650 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.650 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.650 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:37.650 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383405+0000", 2024-09-16T20:48:37.650 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9286e-05 2024-09-16T20:48:37.650 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.650 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.650 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:37.650 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383507+0000", 2024-09-16T20:48:37.650 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101951 2024-09-16T20:48:37.650 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:37.650 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:37.651 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:37.651 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.651 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.651 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2179 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 948224~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:37.651 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335723+0000", 2024-09-16T20:48:37.651 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.287488219, 2024-09-16T20:48:37.651 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048007181000000003, 2024-09-16T20:48:37.651 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:37.651 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:37.651 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:37.651 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:37.651 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:37.651 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:37.652 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2179 2024-09-16T20:48:37.652 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.652 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:37.652 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.652 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:37.652 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335723+0000", 2024-09-16T20:48:37.652 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:37.652 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.652 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.652 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:37.652 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335723+0000", 2024-09-16T20:48:37.652 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:37.652 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.652 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.653 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:37.653 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335724+0000", 2024-09-16T20:48:37.653 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.58e-07 2024-09-16T20:48:37.653 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.653 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.653 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:37.653 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335726+0000", 2024-09-16T20:48:37.653 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9120000000000001e-06 2024-09-16T20:48:37.653 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.654 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.654 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:37.654 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335732+0000", 2024-09-16T20:48:37.654 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6080000000000001e-06 2024-09-16T20:48:37.654 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.654 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.654 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:37.654 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335737+0000", 2024-09-16T20:48:37.654 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.22e-06 2024-09-16T20:48:37.654 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.654 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.654 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:37.654 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383556+0000", 2024-09-16T20:48:37.654 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047819002999999999 2024-09-16T20:48:37.655 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.655 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.655 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:37.655 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383635+0000", 2024-09-16T20:48:37.655 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9060999999999994e-05 2024-09-16T20:48:37.655 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.655 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.655 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:37.655 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383730+0000", 2024-09-16T20:48:37.655 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4919000000000004e-05 2024-09-16T20:48:37.655 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:37.655 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:37.655 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:37.655 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.656 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.656 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2180 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 956416~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:37.656 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335744+0000", 2024-09-16T20:48:37.656 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.287467770999999, 2024-09-16T20:48:37.656 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048255147999999998, 2024-09-16T20:48:37.656 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:37.656 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:37.656 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:37.656 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:37.656 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:37.656 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:37.656 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2180 2024-09-16T20:48:37.656 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.657 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:37.657 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.657 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:37.657 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335744+0000", 2024-09-16T20:48:37.657 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:37.657 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.657 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.657 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:37.657 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335744+0000", 2024-09-16T20:48:37.657 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:37.657 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.657 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.657 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:37.658 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335744+0000", 2024-09-16T20:48:37.658 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9000000000000002e-07 2024-09-16T20:48:37.658 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.658 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.658 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:37.658 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335746+0000", 2024-09-16T20:48:37.658 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7570000000000001e-06 2024-09-16T20:48:37.658 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.658 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.658 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:37.658 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335752+0000", 2024-09-16T20:48:37.658 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0079999999999999e-06 2024-09-16T20:48:37.658 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.659 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.659 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:37.659 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335757+0000", 2024-09-16T20:48:37.659 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1649999999999997e-06 2024-09-16T20:48:37.659 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.659 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.659 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:37.659 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383803+0000", 2024-09-16T20:48:37.659 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048046091999999999 2024-09-16T20:48:37.659 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.659 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.659 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:37.659 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383889+0000", 2024-09-16T20:48:37.660 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6048e-05 2024-09-16T20:48:37.660 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.660 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.660 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:37.660 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383999+0000", 2024-09-16T20:48:37.660 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000109688 2024-09-16T20:48:37.660 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:37.660 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:37.660 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:37.660 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.660 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.660 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2182 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 964608~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:37.660 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335773+0000", 2024-09-16T20:48:37.661 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.287437964, 2024-09-16T20:48:37.661 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048467116999999997, 2024-09-16T20:48:37.661 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:37.661 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:37.661 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:37.661 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:37.661 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:37.661 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:37.661 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2182 2024-09-16T20:48:37.661 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.661 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:37.661 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.661 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:37.661 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335773+0000", 2024-09-16T20:48:37.662 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:37.662 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.662 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.662 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:37.662 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335773+0000", 2024-09-16T20:48:37.662 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:37.662 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.662 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.662 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:37.662 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335774+0000", 2024-09-16T20:48:37.662 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3799999999999997e-07 2024-09-16T20:48:37.662 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.662 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.663 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:37.663 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335776+0000", 2024-09-16T20:48:37.663 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.227e-06 2024-09-16T20:48:37.663 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.663 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.663 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:37.663 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335790+0000", 2024-09-16T20:48:37.663 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4119e-05 2024-09-16T20:48:37.663 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.663 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.663 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:37.663 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335796+0000", 2024-09-16T20:48:37.663 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.694e-06 2024-09-16T20:48:37.664 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.664 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.664 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:37.664 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384058+0000", 2024-09-16T20:48:37.664 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048261983000000001 2024-09-16T20:48:37.664 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.664 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.664 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:37.664 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384148+0000", 2024-09-16T20:48:37.664 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9501999999999998e-05 2024-09-16T20:48:37.664 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.664 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.664 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:37.664 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384241+0000", 2024-09-16T20:48:37.665 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.3053999999999994e-05 2024-09-16T20:48:37.665 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:37.665 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:37.665 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:37.665 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.665 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.665 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2185 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 972800~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:37.665 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335804+0000", 2024-09-16T20:48:37.665 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.287407313999999, 2024-09-16T20:48:37.665 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048570818000000002, 2024-09-16T20:48:37.665 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:37.665 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:37.665 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:37.666 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:37.666 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:37.666 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:37.666 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2185 2024-09-16T20:48:37.666 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.666 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:37.666 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.666 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:37.666 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335804+0000", 2024-09-16T20:48:37.666 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:37.666 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.666 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.666 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:37.666 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335804+0000", 2024-09-16T20:48:37.667 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:37.667 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.667 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.667 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:37.667 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335805+0000", 2024-09-16T20:48:37.667 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8299999999999997e-07 2024-09-16T20:48:37.667 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.667 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.667 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:37.667 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335813+0000", 2024-09-16T20:48:37.667 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6309999999999992e-06 2024-09-16T20:48:37.667 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.667 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.668 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:37.668 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335821+0000", 2024-09-16T20:48:37.668 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0169999999999996e-06 2024-09-16T20:48:37.668 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.668 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.668 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:37.668 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335827+0000", 2024-09-16T20:48:37.668 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0100000000000001e-06 2024-09-16T20:48:37.668 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.668 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.669 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:37.669 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384273+0000", 2024-09-16T20:48:37.669 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048445333 2024-09-16T20:48:37.669 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.669 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.669 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:37.669 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384315+0000", 2024-09-16T20:48:37.669 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2472999999999998e-05 2024-09-16T20:48:37.669 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.669 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.669 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:37.669 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384375+0000", 2024-09-16T20:48:37.669 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9871000000000003e-05 2024-09-16T20:48:37.670 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:37.670 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:37.670 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:37.670 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.670 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.670 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2187 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 980992~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:37.670 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335835+0000", 2024-09-16T20:48:37.670 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.287376237, 2024-09-16T20:48:37.670 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048684927000000003, 2024-09-16T20:48:37.670 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:37.670 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:37.672 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:37.672 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:37.672 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:37.672 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:37.672 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2187 2024-09-16T20:48:37.672 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.672 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:37.673 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.673 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:37.673 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335835+0000", 2024-09-16T20:48:37.673 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:37.673 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.673 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.673 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:37.673 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335835+0000", 2024-09-16T20:48:37.673 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:37.673 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.673 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.673 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:37.673 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335836+0000", 2024-09-16T20:48:37.674 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3599999999999999e-07 2024-09-16T20:48:37.674 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.674 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.674 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:37.674 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335838+0000", 2024-09-16T20:48:37.674 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.996e-06 2024-09-16T20:48:37.674 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.674 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.675 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:37.675 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335844+0000", 2024-09-16T20:48:37.675 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7889999999999997e-06 2024-09-16T20:48:37.675 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.675 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.675 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:37.675 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335849+0000", 2024-09-16T20:48:37.676 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7400000000000004e-06 2024-09-16T20:48:37.676 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.676 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.676 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:37.676 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384411+0000", 2024-09-16T20:48:37.676 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048562069999999999 2024-09-16T20:48:37.677 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.677 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.677 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:37.677 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384460+0000", 2024-09-16T20:48:37.677 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9125999999999997e-05 2024-09-16T20:48:37.678 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.678 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.678 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:37.678 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384520+0000", 2024-09-16T20:48:37.678 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9769999999999999e-05 2024-09-16T20:48:37.678 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:37.678 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:37.678 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:37.678 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.679 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.679 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2189 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 989184~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:37.679 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335856+0000", 2024-09-16T20:48:37.679 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.287354991999999, 2024-09-16T20:48:37.679 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048856525999999997, 2024-09-16T20:48:37.679 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:37.679 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:37.679 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:37.679 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:37.679 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:37.679 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:37.679 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2189 2024-09-16T20:48:37.680 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.680 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:37.680 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.680 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:37.680 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335856+0000", 2024-09-16T20:48:37.680 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:37.680 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.680 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.680 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:37.680 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335856+0000", 2024-09-16T20:48:37.681 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:37.681 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.681 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.681 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:37.681 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335857+0000", 2024-09-16T20:48:37.681 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6800000000000001e-07 2024-09-16T20:48:37.681 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.681 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.681 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:37.682 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335859+0000", 2024-09-16T20:48:37.682 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.013e-06 2024-09-16T20:48:37.682 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.682 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.682 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:37.682 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335866+0000", 2024-09-16T20:48:37.682 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8750000000000002e-06 2024-09-16T20:48:37.682 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.682 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.683 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:37.683 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335871+0000", 2024-09-16T20:48:37.683 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1719999999999998e-06 2024-09-16T20:48:37.683 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.683 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.683 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:37.683 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384550+0000", 2024-09-16T20:48:37.683 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048679476999999999 2024-09-16T20:48:37.683 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.683 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.683 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:37.683 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384611+0000", 2024-09-16T20:48:37.684 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0897000000000003e-05 2024-09-16T20:48:37.684 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.684 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.684 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:37.684 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384713+0000", 2024-09-16T20:48:37.684 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101724 2024-09-16T20:48:37.684 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:37.684 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:37.684 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:37.684 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.685 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.685 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2192 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 997376~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:37.685 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335878+0000", 2024-09-16T20:48:37.685 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.287333307000001, 2024-09-16T20:48:37.685 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049028769999999999, 2024-09-16T20:48:37.685 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:37.685 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:37.685 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:37.685 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:37.685 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:37.685 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:37.686 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2192 2024-09-16T20:48:37.686 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.686 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:37.686 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.686 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:37.686 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335878+0000", 2024-09-16T20:48:37.686 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:37.686 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.686 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.686 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:37.686 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335878+0000", 2024-09-16T20:48:37.687 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:37.687 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.687 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.687 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:37.687 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335879+0000", 2024-09-16T20:48:37.687 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6899999999999998e-07 2024-09-16T20:48:37.687 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.687 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.687 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:37.687 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335881+0000", 2024-09-16T20:48:37.688 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.996e-06 2024-09-16T20:48:37.688 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.688 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.688 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:37.688 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335887+0000", 2024-09-16T20:48:37.688 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6370000000000002e-06 2024-09-16T20:48:37.688 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.688 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.688 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:37.688 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335892+0000", 2024-09-16T20:48:37.689 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2120000000000002e-06 2024-09-16T20:48:37.689 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.689 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.689 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:37.689 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384747+0000", 2024-09-16T20:48:37.689 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048854923000000001 2024-09-16T20:48:37.689 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.689 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.689 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:37.689 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384830+0000", 2024-09-16T20:48:37.690 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2359000000000002e-05 2024-09-16T20:48:37.690 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.690 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.690 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:37.690 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384907+0000", 2024-09-16T20:48:37.690 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7174000000000003e-05 2024-09-16T20:48:37.690 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:37.690 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:37.690 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:37.691 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.691 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.691 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2194 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1005568~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:37.691 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335899+0000", 2024-09-16T20:48:37.691 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.287312386, 2024-09-16T20:48:37.691 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049225393999999999, 2024-09-16T20:48:37.691 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:37.691 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:37.691 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:37.692 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:37.692 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:37.692 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:37.692 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2194 2024-09-16T20:48:37.692 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.692 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:37.692 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.692 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:37.692 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335899+0000", 2024-09-16T20:48:37.692 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:37.693 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.693 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.693 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:37.693 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335899+0000", 2024-09-16T20:48:37.693 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:37.693 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.693 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.693 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:37.694 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335899+0000", 2024-09-16T20:48:37.694 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8200000000000001e-07 2024-09-16T20:48:37.694 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.694 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.694 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:37.695 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335901+0000", 2024-09-16T20:48:37.695 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9e-06 2024-09-16T20:48:37.695 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.695 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.695 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:37.695 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335908+0000", 2024-09-16T20:48:37.695 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3559999999999998e-06 2024-09-16T20:48:37.695 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.695 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.696 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:37.696 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335914+0000", 2024-09-16T20:48:37.696 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4729999999999997e-06 2024-09-16T20:48:37.696 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.696 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.696 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:37.696 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384979+0000", 2024-09-16T20:48:37.696 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049064545000000001 2024-09-16T20:48:37.696 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.696 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.697 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:37.697 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385060+0000", 2024-09-16T20:48:37.697 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0804999999999996e-05 2024-09-16T20:48:37.697 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.697 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.697 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:37.697 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385124+0000", 2024-09-16T20:48:37.697 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4932999999999996e-05 2024-09-16T20:48:37.697 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:37.697 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:37.697 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:37.697 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.697 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.698 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2197 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1013760~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:37.698 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335921+0000", 2024-09-16T20:48:37.698 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.287290105, 2024-09-16T20:48:37.698 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049342184999999997, 2024-09-16T20:48:37.698 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:37.698 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:37.698 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:37.698 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:37.698 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:37.698 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:37.698 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2197 2024-09-16T20:48:37.698 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.699 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:37.699 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.699 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:37.699 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335921+0000", 2024-09-16T20:48:37.699 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:37.699 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.699 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.699 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:37.699 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335921+0000", 2024-09-16T20:48:37.700 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:37.700 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.700 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.700 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:37.700 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335922+0000", 2024-09-16T20:48:37.700 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.32e-07 2024-09-16T20:48:37.700 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.700 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.700 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:37.700 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335924+0000", 2024-09-16T20:48:37.700 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.995e-06 2024-09-16T20:48:37.700 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.700 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.701 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:37.701 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335930+0000", 2024-09-16T20:48:37.701 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5139999999999998e-06 2024-09-16T20:48:37.701 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.701 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.701 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:37.701 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335936+0000", 2024-09-16T20:48:37.701 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6160000000000001e-06 2024-09-16T20:48:37.701 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.701 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.701 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:37.701 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385142+0000", 2024-09-16T20:48:37.701 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049206393000000001 2024-09-16T20:48:37.701 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.702 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.702 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:37.702 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385184+0000", 2024-09-16T20:48:37.702 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1928999999999998e-05 2024-09-16T20:48:37.702 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.702 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.702 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:37.702 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385264+0000", 2024-09-16T20:48:37.702 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9306000000000003e-05 2024-09-16T20:48:37.702 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:37.702 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:37.703 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:37.703 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.703 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.703 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2198 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1021952~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:37.703 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335942+0000", 2024-09-16T20:48:37.703 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.287269045, 2024-09-16T20:48:37.703 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049562483999999997, 2024-09-16T20:48:37.703 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:37.703 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:37.703 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:37.703 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:37.703 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:37.703 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:37.704 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2198 2024-09-16T20:48:37.704 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.704 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:37.704 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.704 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:37.704 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335942+0000", 2024-09-16T20:48:37.704 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:37.704 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.704 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.704 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:37.704 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335942+0000", 2024-09-16T20:48:37.705 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:37.705 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.705 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.705 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:37.705 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335943+0000", 2024-09-16T20:48:37.705 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7099999999999997e-07 2024-09-16T20:48:37.705 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.705 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.705 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:37.705 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335945+0000", 2024-09-16T20:48:37.705 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3499999999999999e-06 2024-09-16T20:48:37.705 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.705 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.706 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:37.706 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335952+0000", 2024-09-16T20:48:37.706 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4309999999999999e-06 2024-09-16T20:48:37.706 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.706 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.706 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:37.706 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335957+0000", 2024-09-16T20:48:37.706 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0869999999999997e-06 2024-09-16T20:48:37.706 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.706 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.706 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:37.706 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385312+0000", 2024-09-16T20:48:37.706 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049355121000000002 2024-09-16T20:48:37.707 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.707 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.707 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:37.707 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385405+0000", 2024-09-16T20:48:37.707 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.3344000000000006e-05 2024-09-16T20:48:37.707 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.707 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.707 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:37.707 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385505+0000", 2024-09-16T20:48:37.707 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.9779999999999997e-05 2024-09-16T20:48:37.707 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:37.707 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:37.707 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:37.708 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.708 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.708 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2201 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1030144~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:37.708 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335976+0000", 2024-09-16T20:48:37.708 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.287235602999999, 2024-09-16T20:48:37.708 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049740077000000001, 2024-09-16T20:48:37.708 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:37.708 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:37.708 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:37.708 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:37.708 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:37.708 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:37.709 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2201 2024-09-16T20:48:37.709 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.709 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:37.709 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.709 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:37.709 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335976+0000", 2024-09-16T20:48:37.709 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:37.709 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.709 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.709 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:37.709 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335976+0000", 2024-09-16T20:48:37.709 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:37.709 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.710 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.710 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:37.710 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335976+0000", 2024-09-16T20:48:37.710 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8599999999999998e-07 2024-09-16T20:48:37.710 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.710 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.710 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:37.710 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335978+0000", 2024-09-16T20:48:37.710 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1500000000000002e-06 2024-09-16T20:48:37.710 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.710 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.711 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:37.711 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335986+0000", 2024-09-16T20:48:37.711 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3989999999999998e-06 2024-09-16T20:48:37.711 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.711 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.711 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:37.711 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335992+0000", 2024-09-16T20:48:37.711 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.682e-06 2024-09-16T20:48:37.711 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.711 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.711 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:37.711 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385537+0000", 2024-09-16T20:48:37.712 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049545828 2024-09-16T20:48:37.712 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.712 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.712 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:37.712 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385619+0000", 2024-09-16T20:48:37.712 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1720999999999999e-05 2024-09-16T20:48:37.712 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.712 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.712 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:37.712 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385716+0000", 2024-09-16T20:48:37.712 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.6811000000000005e-05 2024-09-16T20:48:37.712 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:37.713 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:37.713 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:37.713 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.713 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.713 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2203 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1038336~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:37.713 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.336000+0000", 2024-09-16T20:48:37.713 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.287211816999999, 2024-09-16T20:48:37.714 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049949888999999997, 2024-09-16T20:48:37.714 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:37.714 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:37.714 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:37.714 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:37.714 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:37.714 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:37.714 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2203 2024-09-16T20:48:37.714 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.715 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:37.715 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.715 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:37.715 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336000+0000", 2024-09-16T20:48:37.715 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:37.715 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.715 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.715 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:37.715 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336000+0000", 2024-09-16T20:48:37.715 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:37.715 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.715 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.715 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:37.716 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336000+0000", 2024-09-16T20:48:37.716 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.58e-07 2024-09-16T20:48:37.716 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.716 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.716 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:37.716 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336002+0000", 2024-09-16T20:48:37.716 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0040000000000002e-06 2024-09-16T20:48:37.716 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.716 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.716 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:37.716 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336009+0000", 2024-09-16T20:48:37.717 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9890000000000003e-06 2024-09-16T20:48:37.717 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.717 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.717 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:37.717 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336014+0000", 2024-09-16T20:48:37.717 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4e-06 2024-09-16T20:48:37.717 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.717 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.718 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:37.718 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385759+0000", 2024-09-16T20:48:37.718 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049744151 2024-09-16T20:48:37.718 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.718 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.718 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:37.718 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385847+0000", 2024-09-16T20:48:37.719 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8189000000000006e-05 2024-09-16T20:48:37.719 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.719 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.719 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:37.719 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385950+0000", 2024-09-16T20:48:37.719 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102698 2024-09-16T20:48:37.719 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:37.720 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:37.720 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:37.720 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.720 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.720 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2206 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1046528~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:37.721 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:48:37.721 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.287189656000001, 2024-09-16T20:48:37.721 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050147714000000003, 2024-09-16T20:48:37.721 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:37.721 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:37.721 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:37.721 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:37.722 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:37.722 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:37.722 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2206 2024-09-16T20:48:37.722 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.722 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:37.722 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.723 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:37.723 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:48:37.723 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:37.723 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.723 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.723 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:37.724 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:48:37.724 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:37.724 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.724 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.724 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:37.724 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:48:37.725 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8599999999999999e-07 2024-09-16T20:48:37.725 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.725 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.725 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:37.725 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336024+0000", 2024-09-16T20:48:37.725 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.934e-06 2024-09-16T20:48:37.726 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.726 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.726 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:37.726 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336037+0000", 2024-09-16T20:48:37.726 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2496e-05 2024-09-16T20:48:37.726 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.727 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.727 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:37.727 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336042+0000", 2024-09-16T20:48:37.727 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4809999999999997e-06 2024-09-16T20:48:37.727 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.727 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.728 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:37.728 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385980+0000", 2024-09-16T20:48:37.728 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049938376999999999 2024-09-16T20:48:37.728 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.728 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.729 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:37.729 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.386069+0000", 2024-09-16T20:48:37.729 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8368999999999994e-05 2024-09-16T20:48:37.729 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.729 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.729 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:37.730 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.386170+0000", 2024-09-16T20:48:37.730 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000100671 2024-09-16T20:48:37.730 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:37.730 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:37.730 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:37.730 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.731 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.731 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5439 3.a 3:5789f5c3:::rbd_data.110ed9ca540.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 1181696~60416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:37.731 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:48:37.731 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 14.758501076, 2024-09-16T20:48:37.731 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10102045599999999, 2024-09-16T20:48:37.731 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:37.732 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:37.732 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:37.732 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:37.732 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:37.732 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:37.732 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5439 2024-09-16T20:48:37.733 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.733 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:37.733 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.733 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:37.733 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:48:37.733 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:37.734 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.734 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.734 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:37.734 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:48:37.734 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:37.734 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.735 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.735 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:37.735 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864711+0000", 2024-09-16T20:48:37.735 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7899999999999997e-07 2024-09-16T20:48:37.735 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.735 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.736 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:37.736 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864760+0000", 2024-09-16T20:48:37.736 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8803e-05 2024-09-16T20:48:37.736 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.736 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.737 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:37.737 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864769+0000", 2024-09-16T20:48:37.737 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0999999999999993e-06 2024-09-16T20:48:37.737 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.737 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.737 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:37.737 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864774+0000", 2024-09-16T20:48:37.738 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8919999999999999e-06 2024-09-16T20:48:37.738 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.738 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.738 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:37.738 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864815+0000", 2024-09-16T20:48:37.739 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1307000000000003e-05 2024-09-16T20:48:37.739 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.739 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.739 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:37.739 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864893+0000", 2024-09-16T20:48:37.740 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7581000000000001e-05 2024-09-16T20:48:37.740 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.740 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.740 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 1", 2024-09-16T20:48:37.740 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864996+0000", 2024-09-16T20:48:37.740 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102725 2024-09-16T20:48:37.741 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.741 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.741 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:37.741 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965442+0000", 2024-09-16T20:48:37.741 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100446012 2024-09-16T20:48:37.741 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.742 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.742 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:37.742 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965618+0000", 2024-09-16T20:48:37.742 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00017653100000000001 2024-09-16T20:48:37.742 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.743 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.743 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:37.743 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965682+0000", 2024-09-16T20:48:37.743 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3905999999999994e-05 2024-09-16T20:48:37.743 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.743 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.744 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:37.744 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965731+0000", 2024-09-16T20:48:37.744 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8819999999999997e-05 2024-09-16T20:48:37.744 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:37.744 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:37.745 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:37.745 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.745 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.745 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4600.0:5439 3.a e37/23 ondisk, result = 0)", 2024-09-16T20:48:37.745 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:48:37.745 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 14.757504927999999, 2024-09-16T20:48:37.746 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099742634999999996, 2024-09-16T20:48:37.746 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:37.746 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:37.746 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:37.746 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:37.746 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.746 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:37.747 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:48:37.747 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:37.747 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.747 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.747 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:37.747 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:48:37.748 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:37.748 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.748 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.748 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:37.748 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:48:37.748 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0900000000000001e-07 2024-09-16T20:48:37.748 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.748 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.748 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:37.748 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865711+0000", 2024-09-16T20:48:37.749 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3739999999999998e-06 2024-09-16T20:48:37.749 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.749 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.749 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:37.749 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965343+0000", 2024-09-16T20:48:37.749 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099632526999999999 2024-09-16T20:48:37.749 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.749 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.749 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:37.750 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965357+0000", 2024-09-16T20:48:37.750 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3713999999999999e-05 2024-09-16T20:48:37.750 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.750 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.750 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:37.750 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965426+0000", 2024-09-16T20:48:37.750 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8745999999999995e-05 2024-09-16T20:48:37.750 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.750 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.750 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:37.750 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965434+0000", 2024-09-16T20:48:37.750 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6910000000000003e-06 2024-09-16T20:48:37.750 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:37.750 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:37.750 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:37.751 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965449+0000", 2024-09-16T20:48:37.751 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4874e-05 2024-09-16T20:48:37.751 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:37.751 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:37.751 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:37.751 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:37.751 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:37.751 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:37.753 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:48:37.753 INFO:teuthology.orchestra.run.smithi029.stderr:noscrub is unset 2024-09-16T20:48:37.753 INFO:teuthology.orchestra.run.smithi029.stderr:set 3.1e pg_upmap_items mapping to [1->5,3->0] 2024-09-16T20:48:37.755 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:37.751+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:37.854 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:37.852+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:37.901 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:37.901 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:48:37.901 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:48:37.901 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:37.909 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:48:37.932 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T20:48:37.955 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:37.953+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:38.055 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:38.054+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:38.058 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:38.059 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:48:38.059 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:48:38.059 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:48:38.059 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:38.068 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:48:38.073 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:48:38.073 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:48:38.090 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:48:38.090 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:48:38.107 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:48:38.107 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:48:38.125 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:48:38.125 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:48:38.142 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:48:38.142 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:48:38.156 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:38.155+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:38.159 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:48:38.160 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:48:38.180 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:38.180 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:38.189 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:48:38.273 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:38.272+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:38.337 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:38.337 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:38.346 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:48:38.387 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:38.386+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:38.459 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:38.458+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:38.496 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:38.496 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:38.502 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:48:38.561 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:38.559+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:38.609 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:38.609 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:38.617 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:48:38.661 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:38.661+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:38.741 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:38.741 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:38.751 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:48:38.794 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:38.793+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:38.858 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:38.858 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:38.863 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:38.862+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:38.867 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:48:38.964 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:38.964+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:39.014 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:39.014 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:39.023 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:48:39.066 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:39.065+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:39.130 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:39.130 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:39.138 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:48:39.180 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:39.178+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:39.267 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:39.266+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:39.285 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:39.286 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:39.294 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:48:39.378 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:39.377+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:39.442 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:39.442 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:39.451 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:48:39.469 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:39.468+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:39.570 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:39.569+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:39.599 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:39.599 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:39.608 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:48:39.671 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:39.670+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:39.715 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:39.715 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:39.724 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:48:39.755 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T20:48:39.799 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:39.798+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:39.831 INFO:teuthology.orchestra.run.smithi139.stdout:truncating image image_client.0-clone11 from 0xed07a00 (overlap 0x16b3400) to 0xb59a00 2024-09-16T20:48:39.873 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:39.872+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:39.876 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:39.877 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:39.885 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:48:39.973 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:39.973+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:39.996 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:39.997 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:40.005 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:48:40.075 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:40.073+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:40.117 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:40.117 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:40.129 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:48:40.152 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #10, image image_client.0-clone10 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent11 2024-09-16T20:48:40.171 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T20:48:40.207 INFO:teuthology.orchestra.run.smithi139.stdout:1683 punch from 0x953ea00 to 0x9541800, (0x2e00 bytes) 2024-09-16T20:48:40.216 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:40.215+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:40.217 INFO:teuthology.orchestra.run.smithi139.stdout:1685 write 0x6948000 thru 0x694da4b (0x5a4c bytes) 2024-09-16T20:48:40.217 INFO:teuthology.orchestra.run.smithi139.stdout:1686 write 0x13e7a00 thru 0x13eb474 (0x3a75 bytes) 2024-09-16T20:48:40.227 INFO:teuthology.orchestra.run.smithi139.stdout:1690 trunc from 0xed07a00 to 0x6c41c00 2024-09-16T20:48:40.247 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:40.248 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:40.260 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:48:40.276 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:40.275+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:40.282 INFO:teuthology.orchestra.run.smithi139.stdout:1691 write 0xcc02000 thru 0xcc0a177 (0x8178 bytes) 2024-09-16T20:48:40.289 INFO:teuthology.orchestra.run.smithi139.stdout:1692 write 0xc1ec000 thru 0xc1f2f18 (0x6f19 bytes) 2024-09-16T20:48:40.290 INFO:teuthology.orchestra.run.smithi139.stdout:1693 write 0xa3d0400 thru 0xa3dfbc6 (0xf7c7 bytes) 2024-09-16T20:48:40.290 INFO:teuthology.orchestra.run.smithi139.stdout:1694 punch from 0x635d200 to 0x636ba00, (0xe800 bytes) 2024-09-16T20:48:40.291 INFO:teuthology.orchestra.run.smithi139.stdout:1696 write 0x6042600 thru 0x60467fa (0x41fb bytes) 2024-09-16T20:48:40.292 INFO:teuthology.orchestra.run.smithi139.stdout:1697 read 0x5c0ec00 thru 0x5c1da06 (0xee07 bytes) 2024-09-16T20:48:40.292 INFO:teuthology.orchestra.run.smithi139.stdout:1698 read 0xed3a00 thru 0xee2c29 (0xf22a bytes) 2024-09-16T20:48:40.293 INFO:teuthology.orchestra.run.smithi139.stdout:1704 read 0x9721600 thru 0x9726a3e (0x543f bytes) 2024-09-16T20:48:40.295 INFO:teuthology.orchestra.run.smithi139.stdout:1705 trunc from 0xcc0a200 to 0xb284800 2024-09-16T20:48:40.306 INFO:teuthology.orchestra.run.smithi139.stdout:1707 write 0x1873400 thru 0x187692e (0x352f bytes) 2024-09-16T20:48:40.307 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:48:40.307 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:48:40.325 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:48:40.326 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:48:40.327 INFO:teuthology.orchestra.run.smithi139.stdout:1708 trunc from 0xb284800 to 0x12fe600 2024-09-16T20:48:40.350 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:48:40.350 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:48:40.370 INFO:teuthology.orchestra.run.smithi139.stdout:1709 punch from 0x1244c00 to 0x1250c00, (0xc000 bytes) 2024-09-16T20:48:40.373 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:48:40.373 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:48:40.377 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:40.376+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:40.391 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:48:40.391 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:48:40.402 INFO:teuthology.orchestra.run.smithi139.stdout:1710 write 0x9273600 thru 0x927578d (0x218e bytes) 2024-09-16T20:48:40.402 INFO:teuthology.orchestra.run.smithi139.stdout:1711 write 0x26d5800 thru 0x26e4e0c (0xf60d bytes) 2024-09-16T20:48:40.405 INFO:teuthology.orchestra.run.smithi139.stdout:1712 trunc from 0x9275800 to 0xbbcdc00 2024-09-16T20:48:40.410 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:48:40.410 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:48:40.422 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:40.422 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:48:40.422 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:48:40.422 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:40.432 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:48:40.440 INFO:teuthology.orchestra.run.smithi139.stdout:1714 write 0xd044200 thru 0xd04e81e (0xa61f bytes) 2024-09-16T20:48:40.440 INFO:teuthology.orchestra.run.smithi139.stdout:1717 read 0x169e600 thru 0x16a6d98 (0x8799 bytes) 2024-09-16T20:48:40.447 INFO:teuthology.orchestra.run.smithi139.stdout:1718 trunc from 0xd04ea00 to 0x7bca200 2024-09-16T20:48:40.469 INFO:teuthology.orchestra.run.smithi139.stdout:1719 punch from 0x13cec00 to 0x13db600, (0xca00 bytes) 2024-09-16T20:48:40.469 INFO:teuthology.orchestra.run.smithi139.stdout:1721 trunc from 0x7bca200 to 0x845ee00 2024-09-16T20:48:40.477 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:40.476+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:40.485 INFO:teuthology.orchestra.run.smithi139.stdout:1723 read 0x222e200 thru 0x223cd1c (0xeb1d bytes) 2024-09-16T20:48:40.496 INFO:teuthology.orchestra.run.smithi139.stdout:1724 write 0xa6b4800 thru 0xa6bdb56 (0x9357 bytes) 2024-09-16T20:48:40.497 INFO:teuthology.orchestra.run.smithi139.stdout:1725 write 0x1a3aa00 thru 0x1a487c5 (0xddc6 bytes) 2024-09-16T20:48:40.512 INFO:teuthology.orchestra.run.smithi139.stdout:1726 write 0xc826600 thru 0xc82e81c (0x821d bytes) 2024-09-16T20:48:40.513 INFO:teuthology.orchestra.run.smithi139.stdout:1727 punch from 0x599a200 to 0x599ba00, (0x1800 bytes) 2024-09-16T20:48:40.514 INFO:teuthology.orchestra.run.smithi139.stdout:1728 read 0x7050400 thru 0x7054c05 (0x4806 bytes) 2024-09-16T20:48:40.514 INFO:teuthology.orchestra.run.smithi139.stdout:1732 write 0x5b46200 thru 0x5b49e36 (0x3c37 bytes) 2024-09-16T20:48:40.515 INFO:teuthology.orchestra.run.smithi139.stdout:1733 write 0x4efa800 thru 0x4f0217f (0x7980 bytes) 2024-09-16T20:48:40.515 INFO:teuthology.orchestra.run.smithi139.stdout:1735 read 0x3455400 thru 0x345ba1e (0x661f bytes) 2024-09-16T20:48:40.515 INFO:teuthology.orchestra.run.smithi139.stdout:1736 write 0x1aba000 thru 0x1abd75a (0x375b bytes) 2024-09-16T20:48:40.516 INFO:teuthology.orchestra.run.smithi139.stdout:1737 read 0x7696400 thru 0x7699514 (0x3115 bytes) 2024-09-16T20:48:40.516 INFO:teuthology.orchestra.run.smithi139.stdout:1740 write 0x8e9ec00 thru 0x8ea41c0 (0x55c1 bytes) 2024-09-16T20:48:40.523 INFO:teuthology.orchestra.run.smithi139.stdout:1742 trunc from 0xc82ea00 to 0x5fa8c00 2024-09-16T20:48:40.542 INFO:teuthology.orchestra.run.smithi139.stdout:1749 write 0xe5a600 thru 0xe60582 (0x5f83 bytes) 2024-09-16T20:48:40.549 INFO:teuthology.orchestra.run.smithi139.stdout:1750 trunc from 0x5fa8c00 to 0xb87b800 2024-09-16T20:48:40.560 INFO:teuthology.orchestra.run.smithi139.stdout:1751 read 0x716e00 thru 0x722c76 (0xbe77 bytes) 2024-09-16T20:48:40.562 INFO:teuthology.orchestra.run.smithi139.stdout:1752 read 0x9c36e00 thru 0x9c39340 (0x2541 bytes) 2024-09-16T20:48:40.563 INFO:teuthology.orchestra.run.smithi139.stdout:1756 trunc from 0xb87b800 to 0xab5ae00 2024-09-16T20:48:40.572 INFO:teuthology.orchestra.run.smithi139.stdout:1757 write 0x7004c00 thru 0x700757e (0x297f bytes) 2024-09-16T20:48:40.572 INFO:teuthology.orchestra.run.smithi139.stdout:1758 punch from 0x7da1e00 to 0x7dae800, (0xca00 bytes) 2024-09-16T20:48:40.577 INFO:teuthology.orchestra.run.smithi139.stdout:1764 punch from 0x765da00 to 0x766cc00, (0xf200 bytes) 2024-09-16T20:48:40.577 INFO:teuthology.orchestra.run.smithi139.stdout:1767 write 0xa509e00 thru 0xa516fe7 (0xd1e8 bytes) 2024-09-16T20:48:40.577 INFO:teuthology.orchestra.run.smithi139.stdout:1769 write 0x4e7a400 thru 0x4e7b22c (0xe2d bytes) 2024-09-16T20:48:40.578 INFO:teuthology.orchestra.run.smithi139.stdout:1770 punch from 0x1196200 to 0x119a800, (0x4600 bytes) 2024-09-16T20:48:40.578 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:40.577+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:40.581 INFO:teuthology.orchestra.run.smithi139.stdout:1771 trunc from 0xab5ae00 to 0x81c8a00 2024-09-16T20:48:40.591 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:40.591 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:48:40.591 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:48:40.591 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:48:40.591 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:40.601 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:48:40.604 INFO:teuthology.orchestra.run.smithi139.stdout:1773 trunc from 0x81c8a00 to 0xe917e00 2024-09-16T20:48:40.613 INFO:teuthology.orchestra.run.smithi139.stdout:1774 read 0x85b6200 thru 0x85c1cc9 (0xbaca bytes) 2024-09-16T20:48:40.615 INFO:teuthology.orchestra.run.smithi139.stdout:1775 write 0x92b5a00 thru 0x92baaf8 (0x50f9 bytes) 2024-09-16T20:48:40.615 INFO:teuthology.orchestra.run.smithi139.stdout:1778 write 0xc1b1000 thru 0xc1bda20 (0xca21 bytes) 2024-09-16T20:48:40.615 INFO:teuthology.orchestra.run.smithi139.stdout:1780 write 0x6ac5600 thru 0x6ac7efb (0x28fc bytes) 2024-09-16T20:48:40.615 INFO:teuthology.orchestra.run.smithi139.stdout:1783 punch from 0x65e3a00 to 0x65f1000, (0xd600 bytes) 2024-09-16T20:48:40.616 INFO:teuthology.orchestra.run.smithi139.stdout:1784 read 0x634a400 thru 0x6359a60 (0xf661 bytes) 2024-09-16T20:48:40.616 INFO:teuthology.orchestra.run.smithi139.stdout:1785 punch from 0x743200 to 0x743800, (0x600 bytes) 2024-09-16T20:48:40.616 INFO:teuthology.orchestra.run.smithi139.stdout:1788 read 0xa2cf000 thru 0xa2d6e71 (0x7e72 bytes) 2024-09-16T20:48:40.616 INFO:teuthology.orchestra.run.smithi139.stdout:1789 read 0xbe1000 thru 0xbe13e6 (0x3e7 bytes) 2024-09-16T20:48:40.616 INFO:teuthology.orchestra.run.smithi139.stdout:1790 read 0xd352c00 thru 0xd35d355 (0xa756 bytes) 2024-09-16T20:48:40.626 INFO:teuthology.orchestra.run.smithi139.stdout:1791 trunc from 0xe917e00 to 0x6a53c00 2024-09-16T20:48:40.643 INFO:teuthology.orchestra.run.smithi139.stdout:1792 read 0x2b3200 thru 0x2b66f1 (0x34f2 bytes) 2024-09-16T20:48:40.659 INFO:teuthology.orchestra.run.smithi139.stdout:1793 write 0x9eb0c00 thru 0x9eb527a (0x467b bytes) 2024-09-16T20:48:40.667 INFO:teuthology.orchestra.run.smithi139.stdout:1794 write 0x30c5800 thru 0x30c9848 (0x4049 bytes) 2024-09-16T20:48:40.670 INFO:teuthology.orchestra.run.smithi139.stdout:1795 trunc from 0x9eb5400 to 0x77b1e00 2024-09-16T20:48:40.679 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:40.678+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:40.683 INFO:teuthology.orchestra.run.smithi029.stderr:nodeep-scrub is unset 2024-09-16T20:48:40.687 INFO:teuthology.orchestra.run.smithi139.stdout:1799 punch from 0x5929000 to 0x592e000, (0x5000 bytes) 2024-09-16T20:48:40.699 INFO:teuthology.orchestra.run.smithi139.stdout:1800 write 0x1183e00 thru 0x1193c68 (0xfe69 bytes) 2024-09-16T20:48:40.702 INFO:teuthology.orchestra.run.smithi139.stdout:1802 trunc from 0x77b1e00 to 0x4c59400 2024-09-16T20:48:40.720 INFO:teuthology.orchestra.run.smithi139.stdout:1804 read 0x3760200 thru 0x37693b7 (0x91b8 bytes) 2024-09-16T20:48:40.722 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:40.722 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:48:40.722 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:48:40.722 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:48:40.722 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.722 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:48:40.722 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:48:40.722 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 47.466341171000003, 2024-09-16T20:48:40.722 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67502346999999996, 2024-09-16T20:48:40.723 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:40.723 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:40.723 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:40.723 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:40.723 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:48:40.723 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:48:40.723 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 1 2024-09-16T20:48:40.723 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.723 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:40.723 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.723 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:40.723 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:48:40.723 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:40.723 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.724 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.724 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:40.724 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:48:40.724 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:40.724 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.724 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.724 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:40.724 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254128+0000", 2024-09-16T20:48:40.724 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.463e-06 2024-09-16T20:48:40.724 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.724 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.724 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:40.724 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254135+0000", 2024-09-16T20:48:40.724 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1579999999999999e-06 2024-09-16T20:48:40.725 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.725 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.725 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:40.725 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254146+0000", 2024-09-16T20:48:40.725 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1341e-05 2024-09-16T20:48:40.725 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.725 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.725 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:40.725 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254155+0000", 2024-09-16T20:48:40.725 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.1549999999999996e-06 2024-09-16T20:48:40.725 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.725 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.725 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:40.726 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254221+0000", 2024-09-16T20:48:40.726 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5470999999999997e-05 2024-09-16T20:48:40.726 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.726 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.726 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:48:40.726 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254231+0000", 2024-09-16T20:48:40.726 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0592e-05 2024-09-16T20:48:40.726 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.726 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.726 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:40.726 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926711+0000", 2024-09-16T20:48:40.726 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67247946599999997 2024-09-16T20:48:40.726 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.727 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.727 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:40.727 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926910+0000", 2024-09-16T20:48:40.727 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019934 2024-09-16T20:48:40.727 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.727 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.727 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:48:40.727 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.927159+0000", 2024-09-16T20:48:40.727 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00024876599999999998 2024-09-16T20:48:40.727 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.727 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.727 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:40.727 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.928797+0000", 2024-09-16T20:48:40.728 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001638308 2024-09-16T20:48:40.728 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.728 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.728 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:40.728 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929026+0000", 2024-09-16T20:48:40.728 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00022876099999999999 2024-09-16T20:48:40.728 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.728 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.728 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:40.728 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929102+0000", 2024-09-16T20:48:40.728 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6184000000000001e-05 2024-09-16T20:48:40.728 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.728 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.728 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:40.729 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929150+0000", 2024-09-16T20:48:40.729 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7465e-05 2024-09-16T20:48:40.729 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:40.729 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:40.729 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:40.729 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.729 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.729 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4312 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4618240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:40.729 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745559+0000", 2024-09-16T20:48:40.729 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 20.974908034999999, 2024-09-16T20:48:40.729 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046756458000000001, 2024-09-16T20:48:40.729 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:40.729 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:40.729 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:40.730 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:40.730 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:48:40.730 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:40.730 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4312 2024-09-16T20:48:40.730 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.730 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:40.730 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.730 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:40.730 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745559+0000", 2024-09-16T20:48:40.730 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:40.730 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.730 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.730 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:40.730 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745559+0000", 2024-09-16T20:48:40.731 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:40.731 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.731 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.731 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:40.731 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745560+0000", 2024-09-16T20:48:40.731 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6899999999999998e-07 2024-09-16T20:48:40.731 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.731 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.731 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:40.731 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745569+0000", 2024-09-16T20:48:40.731 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8950000000000004e-06 2024-09-16T20:48:40.731 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.731 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.731 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:40.731 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745576+0000", 2024-09-16T20:48:40.732 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9949999999999999e-06 2024-09-16T20:48:40.732 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.732 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.732 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:40.732 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745596+0000", 2024-09-16T20:48:40.732 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0767999999999998e-05 2024-09-16T20:48:40.732 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.732 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.732 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:40.732 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792225+0000", 2024-09-16T20:48:40.732 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046628393999999997 2024-09-16T20:48:40.732 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.732 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.732 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:40.733 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792265+0000", 2024-09-16T20:48:40.733 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9912e-05 2024-09-16T20:48:40.733 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.733 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.733 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:40.733 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792316+0000", 2024-09-16T20:48:40.733 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1025e-05 2024-09-16T20:48:40.733 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:40.733 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:40.733 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:40.733 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.733 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.733 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4313 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4626432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:40.733 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745608+0000", 2024-09-16T20:48:40.734 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 20.974859466000002, 2024-09-16T20:48:40.734 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046829200000000001, 2024-09-16T20:48:40.734 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:40.734 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:40.734 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:40.734 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:40.734 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:48:40.734 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:40.734 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4313 2024-09-16T20:48:40.734 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.734 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:40.734 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.734 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:40.734 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745608+0000", 2024-09-16T20:48:40.734 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:40.735 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.735 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.735 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:40.735 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745608+0000", 2024-09-16T20:48:40.735 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:40.735 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.735 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.735 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:40.735 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745609+0000", 2024-09-16T20:48:40.735 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4700000000000001e-07 2024-09-16T20:48:40.735 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.735 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.735 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:40.736 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745611+0000", 2024-09-16T20:48:40.736 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8119999999999999e-06 2024-09-16T20:48:40.736 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.736 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.736 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:40.736 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745619+0000", 2024-09-16T20:48:40.736 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6520000000000006e-06 2024-09-16T20:48:40.736 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.736 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.736 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:40.736 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745625+0000", 2024-09-16T20:48:40.736 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6590000000000004e-06 2024-09-16T20:48:40.736 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.736 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.737 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:40.737 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792344+0000", 2024-09-16T20:48:40.737 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046719318000000003 2024-09-16T20:48:40.737 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.737 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.737 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:40.737 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792384+0000", 2024-09-16T20:48:40.737 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0389000000000002e-05 2024-09-16T20:48:40.737 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.737 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.737 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:40.737 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792437+0000", 2024-09-16T20:48:40.737 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2623000000000001e-05 2024-09-16T20:48:40.737 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:40.738 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:40.738 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:40.738 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.738 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.738 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4314 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4634624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:40.738 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745632+0000", 2024-09-16T20:48:40.738 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 20.974835598999999, 2024-09-16T20:48:40.738 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046928879999999999, 2024-09-16T20:48:40.738 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:40.738 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:40.738 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:40.738 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:40.738 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:48:40.738 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:40.739 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4314 2024-09-16T20:48:40.739 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.739 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:40.739 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.739 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:40.739 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745632+0000", 2024-09-16T20:48:40.739 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:40.739 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.739 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.739 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:40.739 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745632+0000", 2024-09-16T20:48:40.739 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:40.739 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.740 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.740 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:40.740 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745632+0000", 2024-09-16T20:48:40.740 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3700000000000001e-07 2024-09-16T20:48:40.740 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.740 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.740 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:40.740 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745634+0000", 2024-09-16T20:48:40.740 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9759999999999998e-06 2024-09-16T20:48:40.740 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.740 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.740 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:40.740 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745645+0000", 2024-09-16T20:48:40.741 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1097000000000001e-05 2024-09-16T20:48:40.741 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.741 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.741 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:40.741 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745651+0000", 2024-09-16T20:48:40.741 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5090000000000001e-06 2024-09-16T20:48:40.741 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.741 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.741 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:40.741 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792467+0000", 2024-09-16T20:48:40.743 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046816712000000003 2024-09-16T20:48:40.744 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.744 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.744 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:40.744 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792511+0000", 2024-09-16T20:48:40.744 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3256000000000001e-05 2024-09-16T20:48:40.744 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.744 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.744 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:40.744 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792561+0000", 2024-09-16T20:48:40.744 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9892999999999998e-05 2024-09-16T20:48:40.744 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:40.744 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:40.744 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:40.745 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.745 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.745 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4315 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4642816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:40.745 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745659+0000", 2024-09-16T20:48:40.745 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 20.974808476, 2024-09-16T20:48:40.745 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047065501000000003, 2024-09-16T20:48:40.745 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:40.745 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:40.745 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:40.745 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:40.745 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:48:40.745 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:40.745 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4315 2024-09-16T20:48:40.745 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.746 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:40.746 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.746 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:40.746 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745659+0000", 2024-09-16T20:48:40.746 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:40.746 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.746 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.746 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:40.746 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745659+0000", 2024-09-16T20:48:40.746 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:40.746 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.746 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.746 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:40.747 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745659+0000", 2024-09-16T20:48:40.747 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7e-07 2024-09-16T20:48:40.747 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.747 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.747 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:40.747 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745662+0000", 2024-09-16T20:48:40.747 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2189999999999998e-06 2024-09-16T20:48:40.747 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.747 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.747 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:40.747 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745668+0000", 2024-09-16T20:48:40.747 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8979999999999999e-06 2024-09-16T20:48:40.747 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.747 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.748 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:40.748 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745675+0000", 2024-09-16T20:48:40.748 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3579999999999999e-06 2024-09-16T20:48:40.748 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.748 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.748 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:40.748 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792593+0000", 2024-09-16T20:48:40.748 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046917888999999997 2024-09-16T20:48:40.748 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.748 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.748 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:40.748 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792632+0000", 2024-09-16T20:48:40.748 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9818999999999999e-05 2024-09-16T20:48:40.749 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.749 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.749 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:40.749 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792724+0000", 2024-09-16T20:48:40.749 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.1848000000000006e-05 2024-09-16T20:48:40.749 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:40.749 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:40.749 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:40.749 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.749 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.749 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4316 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4651008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:40.749 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745682+0000", 2024-09-16T20:48:40.749 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 20.974785197999999, 2024-09-16T20:48:40.750 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047270334999999997, 2024-09-16T20:48:40.750 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:40.750 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:40.750 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:40.750 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:40.750 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:48:40.750 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:40.750 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4316 2024-09-16T20:48:40.750 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.750 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:40.750 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.750 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:40.750 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745682+0000", 2024-09-16T20:48:40.750 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:40.750 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.751 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.751 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:40.751 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745682+0000", 2024-09-16T20:48:40.751 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:40.751 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.751 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.751 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:40.751 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745683+0000", 2024-09-16T20:48:40.751 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.27e-07 2024-09-16T20:48:40.751 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.751 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.751 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:40.751 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745685+0000", 2024-09-16T20:48:40.752 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.464e-06 2024-09-16T20:48:40.752 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.752 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.752 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:40.752 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745692+0000", 2024-09-16T20:48:40.752 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.849e-06 2024-09-16T20:48:40.752 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.752 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.752 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:40.752 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745697+0000", 2024-09-16T20:48:40.752 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0170000000000004e-06 2024-09-16T20:48:40.752 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.752 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.753 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:40.753 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792771+0000", 2024-09-16T20:48:40.753 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047074224999999997 2024-09-16T20:48:40.753 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.753 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.753 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:40.753 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792845+0000", 2024-09-16T20:48:40.753 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3739999999999995e-05 2024-09-16T20:48:40.753 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.753 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.753 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:40.753 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792952+0000", 2024-09-16T20:48:40.754 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000107613 2024-09-16T20:48:40.754 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:40.754 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:40.754 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:40.754 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.754 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.754 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4317 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4659200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:40.754 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745704+0000", 2024-09-16T20:48:40.754 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 20.974763251999999, 2024-09-16T20:48:40.754 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047411778000000002, 2024-09-16T20:48:40.754 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:40.754 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:40.754 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:40.754 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:40.755 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:48:40.755 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:40.755 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4317 2024-09-16T20:48:40.755 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.755 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:40.755 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.755 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:40.755 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745704+0000", 2024-09-16T20:48:40.755 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:40.755 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.755 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.755 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:40.755 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745704+0000", 2024-09-16T20:48:40.755 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:40.756 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.756 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.756 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:40.756 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745704+0000", 2024-09-16T20:48:40.756 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0900000000000002e-07 2024-09-16T20:48:40.756 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.756 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.756 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:40.756 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745707+0000", 2024-09-16T20:48:40.756 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1660000000000001e-06 2024-09-16T20:48:40.756 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.756 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.756 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:40.756 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745722+0000", 2024-09-16T20:48:40.757 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5084e-05 2024-09-16T20:48:40.757 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.757 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.757 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:40.757 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745735+0000", 2024-09-16T20:48:40.757 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3074999999999999e-05 2024-09-16T20:48:40.757 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.757 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.757 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:40.757 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792989+0000", 2024-09-16T20:48:40.757 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047254013999999997 2024-09-16T20:48:40.757 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.757 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.757 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:40.758 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793034+0000", 2024-09-16T20:48:40.758 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4858999999999999e-05 2024-09-16T20:48:40.758 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.758 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.758 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:40.758 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793116+0000", 2024-09-16T20:48:40.758 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2170999999999996e-05 2024-09-16T20:48:40.758 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:40.758 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:40.758 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:40.758 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.758 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.758 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4318 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4667392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:40.759 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745812+0000", 2024-09-16T20:48:40.759 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 20.974655573, 2024-09-16T20:48:40.759 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047532229000000002, 2024-09-16T20:48:40.759 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:40.759 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:40.759 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:40.759 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:40.759 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:48:40.759 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:40.759 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4318 2024-09-16T20:48:40.759 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.760 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:40.760 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.760 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:40.760 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745812+0000", 2024-09-16T20:48:40.760 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:40.760 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.760 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.760 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:40.760 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745812+0000", 2024-09-16T20:48:40.760 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:40.760 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.760 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.760 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:40.761 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745812+0000", 2024-09-16T20:48:40.761 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5899999999999996e-07 2024-09-16T20:48:40.761 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.761 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.761 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:40.761 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745815+0000", 2024-09-16T20:48:40.761 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9110000000000002e-06 2024-09-16T20:48:40.761 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.761 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.761 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:40.761 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745823+0000", 2024-09-16T20:48:40.761 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1030000000000002e-06 2024-09-16T20:48:40.761 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.762 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.762 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:40.762 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745829+0000", 2024-09-16T20:48:40.762 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8449999999999999e-06 2024-09-16T20:48:40.762 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.762 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.762 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:40.762 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793159+0000", 2024-09-16T20:48:40.762 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047330009999999999 2024-09-16T20:48:40.762 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.762 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.762 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:40.762 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793251+0000", 2024-09-16T20:48:40.763 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.1991999999999996e-05 2024-09-16T20:48:40.763 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.763 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.763 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:40.763 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793344+0000", 2024-09-16T20:48:40.763 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2808999999999999e-05 2024-09-16T20:48:40.763 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:40.763 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:40.763 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:40.763 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.763 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.763 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4319 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4675584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:40.763 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745838+0000", 2024-09-16T20:48:40.763 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 20.974629360000002, 2024-09-16T20:48:40.764 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047640204999999998, 2024-09-16T20:48:40.764 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:40.764 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:40.764 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:40.764 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:40.764 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:48:40.764 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:40.764 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4319 2024-09-16T20:48:40.764 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.764 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:40.764 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.764 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:40.764 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745838+0000", 2024-09-16T20:48:40.765 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:40.765 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.765 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.765 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:40.765 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745838+0000", 2024-09-16T20:48:40.765 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:40.765 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.765 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.765 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:40.765 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745838+0000", 2024-09-16T20:48:40.765 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6899999999999998e-07 2024-09-16T20:48:40.765 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.765 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.765 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:40.766 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745840+0000", 2024-09-16T20:48:40.766 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0660000000000002e-06 2024-09-16T20:48:40.766 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.766 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.766 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:40.766 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745847+0000", 2024-09-16T20:48:40.766 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8689999999999998e-06 2024-09-16T20:48:40.766 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.766 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.766 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:40.766 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745853+0000", 2024-09-16T20:48:40.766 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2399999999999998e-06 2024-09-16T20:48:40.766 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.766 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.767 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:40.767 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793381+0000", 2024-09-16T20:48:40.767 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047528005999999998 2024-09-16T20:48:40.767 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.767 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.767 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:40.767 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793423+0000", 2024-09-16T20:48:40.767 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2799000000000001e-05 2024-09-16T20:48:40.767 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.767 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.767 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:40.767 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793478+0000", 2024-09-16T20:48:40.767 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4756000000000002e-05 2024-09-16T20:48:40.767 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:40.767 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:40.768 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:40.768 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.768 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.768 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4320 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4683776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:40.768 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745859+0000", 2024-09-16T20:48:40.768 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 20.974608115999999, 2024-09-16T20:48:40.768 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047811399999999997, 2024-09-16T20:48:40.768 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:40.768 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:40.768 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:40.768 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:40.768 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:48:40.768 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:40.769 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4320 2024-09-16T20:48:40.769 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.769 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:40.769 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.769 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:40.769 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745859+0000", 2024-09-16T20:48:40.769 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:40.769 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.769 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.769 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:40.769 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745859+0000", 2024-09-16T20:48:40.769 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:40.769 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.769 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.769 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:40.770 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745860+0000", 2024-09-16T20:48:40.770 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9099999999999999e-07 2024-09-16T20:48:40.770 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.770 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.770 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:40.770 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745861+0000", 2024-09-16T20:48:40.770 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8589999999999999e-06 2024-09-16T20:48:40.770 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.770 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.770 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:40.770 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745868+0000", 2024-09-16T20:48:40.770 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2249999999999997e-06 2024-09-16T20:48:40.770 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.770 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.771 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:40.771 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745873+0000", 2024-09-16T20:48:40.771 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1290000000000004e-06 2024-09-16T20:48:40.771 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.771 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.771 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:40.771 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793522+0000", 2024-09-16T20:48:40.771 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047648847000000001 2024-09-16T20:48:40.771 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.771 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.771 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:40.771 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793603+0000", 2024-09-16T20:48:40.771 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1167000000000004e-05 2024-09-16T20:48:40.771 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.772 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.772 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:40.772 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793671+0000", 2024-09-16T20:48:40.772 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7781999999999996e-05 2024-09-16T20:48:40.772 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:40.772 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:40.772 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:40.772 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.772 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.772 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4321 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4691968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:40.772 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745879+0000", 2024-09-16T20:48:40.772 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 20.974587920000001, 2024-09-16T20:48:40.772 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047950375000000003, 2024-09-16T20:48:40.772 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:40.773 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:40.773 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:40.773 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:40.773 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:48:40.773 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:40.773 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4321 2024-09-16T20:48:40.773 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.773 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:40.773 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.773 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:40.773 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745879+0000", 2024-09-16T20:48:40.773 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:40.773 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.774 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.774 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:40.774 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745879+0000", 2024-09-16T20:48:40.774 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:40.774 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.774 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.774 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:40.774 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745880+0000", 2024-09-16T20:48:40.774 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.65e-07 2024-09-16T20:48:40.774 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.774 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.774 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:40.775 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745882+0000", 2024-09-16T20:48:40.775 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.922e-06 2024-09-16T20:48:40.775 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.775 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.775 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:40.775 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745888+0000", 2024-09-16T20:48:40.775 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0510000000000002e-06 2024-09-16T20:48:40.775 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.775 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.775 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:40.775 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745893+0000", 2024-09-16T20:48:40.775 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9579999999999998e-06 2024-09-16T20:48:40.775 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.776 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.776 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:40.776 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793704+0000", 2024-09-16T20:48:40.776 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047811179000000002 2024-09-16T20:48:40.776 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.776 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.776 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:40.776 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793759+0000", 2024-09-16T20:48:40.776 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5260999999999998e-05 2024-09-16T20:48:40.776 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.776 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.776 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:40.776 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793830+0000", 2024-09-16T20:48:40.776 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0638999999999999e-05 2024-09-16T20:48:40.777 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:40.777 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:40.777 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:40.777 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.777 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.777 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4322 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4700160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:40.777 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745899+0000", 2024-09-16T20:48:40.777 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 20.974568481999999, 2024-09-16T20:48:40.777 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048057280000000001, 2024-09-16T20:48:40.777 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:40.777 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:40.777 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:40.777 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:40.777 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:48:40.778 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:40.778 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4322 2024-09-16T20:48:40.778 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.778 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:40.778 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.778 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:40.778 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745899+0000", 2024-09-16T20:48:40.778 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:40.778 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.778 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.778 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:40.778 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745899+0000", 2024-09-16T20:48:40.778 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:40.778 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.779 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.779 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:40.779 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745899+0000", 2024-09-16T20:48:40.779 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9299999999999999e-07 2024-09-16T20:48:40.779 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.779 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.779 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:40.779 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745901+0000", 2024-09-16T20:48:40.779 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0030000000000001e-06 2024-09-16T20:48:40.779 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.779 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.779 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:40.779 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745908+0000", 2024-09-16T20:48:40.779 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3729999999999998e-06 2024-09-16T20:48:40.780 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.780 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.780 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:40.780 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745914+0000", 2024-09-16T20:48:40.780 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.122e-06 2024-09-16T20:48:40.780 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.780 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.780 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:40.780 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793862+0000", 2024-09-16T20:48:40.780 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047948324 2024-09-16T20:48:40.780 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.780 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.780 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:40.781 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793903+0000", 2024-09-16T20:48:40.781 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0465e-05 2024-09-16T20:48:40.781 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.781 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.781 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:40.781 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793956+0000", 2024-09-16T20:48:40.781 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3600000000000002e-05 2024-09-16T20:48:40.781 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:40.781 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:40.781 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:40.781 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.781 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.781 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4323 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4708352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:40.781 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745920+0000", 2024-09-16T20:48:40.782 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 20.974546840999999, 2024-09-16T20:48:40.782 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048268138000000002, 2024-09-16T20:48:40.782 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:40.782 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:40.782 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:40.782 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:40.782 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:48:40.782 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:40.782 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4323 2024-09-16T20:48:40.782 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.782 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:40.782 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.782 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:40.782 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745920+0000", 2024-09-16T20:48:40.783 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:40.783 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.783 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.783 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:40.783 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745920+0000", 2024-09-16T20:48:40.783 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:40.783 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.783 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.783 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:40.783 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745921+0000", 2024-09-16T20:48:40.784 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7e-07 2024-09-16T20:48:40.784 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.784 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.784 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:40.784 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745923+0000", 2024-09-16T20:48:40.784 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7880000000000001e-06 2024-09-16T20:48:40.784 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.784 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.784 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:40.784 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745929+0000", 2024-09-16T20:48:40.784 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0809999999999999e-06 2024-09-16T20:48:40.784 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.785 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.785 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:40.785 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745935+0000", 2024-09-16T20:48:40.785 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5370000000000003e-06 2024-09-16T20:48:40.785 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.785 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.785 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:40.785 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793987+0000", 2024-09-16T20:48:40.785 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048051736999999997 2024-09-16T20:48:40.785 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.785 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.785 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:40.785 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.794070+0000", 2024-09-16T20:48:40.785 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.3277999999999998e-05 2024-09-16T20:48:40.786 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.786 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.786 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:40.786 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.794189+0000", 2024-09-16T20:48:40.786 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000118347 2024-09-16T20:48:40.786 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:40.786 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:40.786 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:40.786 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.786 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.786 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4324 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4716544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:40.786 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745946+0000", 2024-09-16T20:48:40.786 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 20.97452127, 2024-09-16T20:48:40.787 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048468153999999999, 2024-09-16T20:48:40.787 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:40.787 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:40.787 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:40.787 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:40.787 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:48:40.787 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:40.787 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4324 2024-09-16T20:48:40.787 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.787 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:40.787 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.787 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:40.787 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745946+0000", 2024-09-16T20:48:40.787 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:40.788 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.788 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.788 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:40.788 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745946+0000", 2024-09-16T20:48:40.788 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:40.788 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.788 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.788 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:40.788 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745946+0000", 2024-09-16T20:48:40.788 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4499999999999997e-07 2024-09-16T20:48:40.788 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.788 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.788 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:40.788 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745949+0000", 2024-09-16T20:48:40.789 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2529999999999999e-06 2024-09-16T20:48:40.789 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.789 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.789 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:40.789 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745955+0000", 2024-09-16T20:48:40.789 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6470000000000001e-06 2024-09-16T20:48:40.789 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.789 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.789 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:40.789 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745960+0000", 2024-09-16T20:48:40.789 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7910000000000004e-06 2024-09-16T20:48:40.789 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.789 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.790 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:40.790 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.794220+0000", 2024-09-16T20:48:40.790 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048260069000000003 2024-09-16T20:48:40.790 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.790 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.790 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:40.790 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.794311+0000", 2024-09-16T20:48:40.790 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0593000000000005e-05 2024-09-16T20:48:40.790 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.790 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.790 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:40.790 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.794414+0000", 2024-09-16T20:48:40.790 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000103356 2024-09-16T20:48:40.790 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:40.791 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:40.791 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:40.791 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.791 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.791 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2059 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 1005568~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:40.791 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.342406+0000", 2024-09-16T20:48:40.791 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 18.378060831999999, 2024-09-16T20:48:40.791 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046803412000000003, 2024-09-16T20:48:40.791 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:40.791 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:40.791 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:40.791 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:40.791 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:40.792 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:40.792 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2059 2024-09-16T20:48:40.792 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.792 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:40.792 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.792 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:40.792 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342406+0000", 2024-09-16T20:48:40.792 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:40.792 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.792 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.792 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:40.792 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342406+0000", 2024-09-16T20:48:40.793 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:40.793 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.793 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.793 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:40.793 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342407+0000", 2024-09-16T20:48:40.793 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7399999999999998e-07 2024-09-16T20:48:40.793 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.793 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.793 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:40.793 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342409+0000", 2024-09-16T20:48:40.793 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.4509999999999999e-06 2024-09-16T20:48:40.793 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.793 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.793 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:40.794 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342418+0000", 2024-09-16T20:48:40.794 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8969999999999997e-06 2024-09-16T20:48:40.794 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.794 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.794 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:40.794 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342424+0000", 2024-09-16T20:48:40.794 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5210000000000002e-06 2024-09-16T20:48:40.794 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.794 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.794 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:40.794 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389084+0000", 2024-09-16T20:48:40.794 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046659997000000002 2024-09-16T20:48:40.794 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.794 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.795 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:40.795 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389153+0000", 2024-09-16T20:48:40.795 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9558e-05 2024-09-16T20:48:40.795 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.795 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.795 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:40.795 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389210+0000", 2024-09-16T20:48:40.795 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6514000000000001e-05 2024-09-16T20:48:40.795 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:40.795 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:40.795 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:40.795 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.795 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.796 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2061 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 1013760~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:40.796 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.342445+0000", 2024-09-16T20:48:40.796 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 18.37802228, 2024-09-16T20:48:40.796 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046901708, 2024-09-16T20:48:40.796 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:40.796 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:40.796 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:40.796 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:40.796 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:40.796 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:40.796 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2061 2024-09-16T20:48:40.796 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.796 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:40.796 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.797 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:40.797 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342445+0000", 2024-09-16T20:48:40.797 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:40.797 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.797 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.797 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:40.797 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342445+0000", 2024-09-16T20:48:40.797 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:40.797 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.797 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.797 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:40.797 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342446+0000", 2024-09-16T20:48:40.797 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4700000000000001e-07 2024-09-16T20:48:40.797 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.798 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.798 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:40.798 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342448+0000", 2024-09-16T20:48:40.798 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6460000000000002e-06 2024-09-16T20:48:40.798 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.798 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.798 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:40.798 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342456+0000", 2024-09-16T20:48:40.798 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7489999999999998e-06 2024-09-16T20:48:40.798 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.798 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.798 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:40.798 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342462+0000", 2024-09-16T20:48:40.798 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5759999999999996e-06 2024-09-16T20:48:40.799 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.799 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.799 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:40.799 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389238+0000", 2024-09-16T20:48:40.799 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046776515999999997 2024-09-16T20:48:40.799 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.799 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.799 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:40.799 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389285+0000", 2024-09-16T20:48:40.799 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6581e-05 2024-09-16T20:48:40.799 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.799 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.799 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:40.799 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389347+0000", 2024-09-16T20:48:40.800 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2093000000000003e-05 2024-09-16T20:48:40.800 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:40.800 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:40.800 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:40.800 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.800 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.800 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2063 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 1021952~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:40.800 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.342470+0000", 2024-09-16T20:48:40.800 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 18.377997391000001, 2024-09-16T20:48:40.800 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047007602000000002, 2024-09-16T20:48:40.800 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:40.800 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:40.800 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:40.801 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:40.801 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:40.801 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:40.801 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2063 2024-09-16T20:48:40.801 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.801 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:40.801 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.801 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:40.801 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342470+0000", 2024-09-16T20:48:40.801 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:40.801 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.801 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.801 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:40.802 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342470+0000", 2024-09-16T20:48:40.802 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:40.802 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.802 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.802 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:40.802 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342470+0000", 2024-09-16T20:48:40.802 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0999999999999999e-07 2024-09-16T20:48:40.802 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.802 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.802 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:40.802 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342473+0000", 2024-09-16T20:48:40.802 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.249e-06 2024-09-16T20:48:40.802 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.803 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.803 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:40.803 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342479+0000", 2024-09-16T20:48:40.803 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7549999999999997e-06 2024-09-16T20:48:40.803 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.803 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.803 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:40.803 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342485+0000", 2024-09-16T20:48:40.803 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.344e-06 2024-09-16T20:48:40.803 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.803 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.803 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:40.803 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389364+0000", 2024-09-16T20:48:40.803 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046879542000000003 2024-09-16T20:48:40.804 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.804 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.804 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:40.804 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389420+0000", 2024-09-16T20:48:40.804 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5541000000000002e-05 2024-09-16T20:48:40.804 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.804 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.804 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:40.804 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389478+0000", 2024-09-16T20:48:40.804 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7661000000000002e-05 2024-09-16T20:48:40.804 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:40.804 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:40.804 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:40.805 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.805 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.805 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2065 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 1030144~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:40.805 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.342492+0000", 2024-09-16T20:48:40.805 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 18.377975552999999, 2024-09-16T20:48:40.805 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047127962000000002, 2024-09-16T20:48:40.805 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:40.805 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:40.805 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:40.805 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:40.805 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:40.805 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:40.805 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2065 2024-09-16T20:48:40.806 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.806 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:40.806 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.806 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:40.806 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342492+0000", 2024-09-16T20:48:40.806 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:40.806 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.806 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.806 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:40.806 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342492+0000", 2024-09-16T20:48:40.806 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:40.806 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.806 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.807 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:40.807 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342492+0000", 2024-09-16T20:48:40.807 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1100000000000001e-07 2024-09-16T20:48:40.807 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.807 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.807 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:40.807 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342494+0000", 2024-09-16T20:48:40.807 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9039999999999999e-06 2024-09-16T20:48:40.807 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.807 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.807 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:40.807 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342501+0000", 2024-09-16T20:48:40.807 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6610000000000003e-06 2024-09-16T20:48:40.807 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.808 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.808 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:40.808 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342506+0000", 2024-09-16T20:48:40.808 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8760000000000004e-06 2024-09-16T20:48:40.808 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.808 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.808 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:40.808 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389501+0000", 2024-09-16T20:48:40.808 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046995440999999999 2024-09-16T20:48:40.808 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.808 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.808 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:40.808 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389550+0000", 2024-09-16T20:48:40.809 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8949000000000002e-05 2024-09-16T20:48:40.809 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.809 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.809 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:40.809 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389620+0000", 2024-09-16T20:48:40.809 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9720000000000003e-05 2024-09-16T20:48:40.809 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:40.809 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:40.809 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:40.809 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.809 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.809 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2068 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 1038336~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:40.810 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.342513+0000", 2024-09-16T20:48:40.810 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 18.377954459000001, 2024-09-16T20:48:40.810 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047233961999999997, 2024-09-16T20:48:40.810 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:40.810 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:40.810 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:40.810 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:40.810 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:40.810 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:40.810 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2068 2024-09-16T20:48:40.810 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.811 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:40.811 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.811 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:40.811 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342513+0000", 2024-09-16T20:48:40.811 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:40.811 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.811 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.811 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:40.811 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342513+0000", 2024-09-16T20:48:40.811 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:40.811 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.811 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.812 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:40.812 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342513+0000", 2024-09-16T20:48:40.812 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.01e-07 2024-09-16T20:48:40.812 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.812 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.812 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:40.812 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342515+0000", 2024-09-16T20:48:40.812 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7400000000000001e-06 2024-09-16T20:48:40.812 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.812 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.812 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:40.812 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342522+0000", 2024-09-16T20:48:40.812 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.516e-06 2024-09-16T20:48:40.813 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.813 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.813 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:40.813 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342527+0000", 2024-09-16T20:48:40.813 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0080000000000002e-06 2024-09-16T20:48:40.813 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.813 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.813 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:40.813 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389644+0000", 2024-09-16T20:48:40.813 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047117289 2024-09-16T20:48:40.813 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.813 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.813 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:40.814 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389690+0000", 2024-09-16T20:48:40.814 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6378000000000002e-05 2024-09-16T20:48:40.814 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.814 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.814 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:40.814 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389747+0000", 2024-09-16T20:48:40.814 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6629999999999998e-05 2024-09-16T20:48:40.814 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:40.814 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:40.814 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:40.814 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.814 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.815 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2070 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 1046528~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:40.815 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.342533+0000", 2024-09-16T20:48:40.815 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 18.377934362000001, 2024-09-16T20:48:40.815 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047345848000000003, 2024-09-16T20:48:40.815 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:40.815 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:40.815 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:40.815 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:40.815 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:40.815 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:40.815 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2070 2024-09-16T20:48:40.815 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.815 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:40.816 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.816 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:40.816 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342533+0000", 2024-09-16T20:48:40.816 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:40.816 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.816 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.816 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:40.816 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342533+0000", 2024-09-16T20:48:40.816 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:40.816 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.816 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.816 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:40.816 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342533+0000", 2024-09-16T20:48:40.817 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7e-07 2024-09-16T20:48:40.817 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.817 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.817 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:40.817 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342535+0000", 2024-09-16T20:48:40.817 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.68e-06 2024-09-16T20:48:40.817 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.817 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.817 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:40.817 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342542+0000", 2024-09-16T20:48:40.817 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5169999999999996e-06 2024-09-16T20:48:40.817 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.817 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.817 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:40.818 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342546+0000", 2024-09-16T20:48:40.818 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8949999999999997e-06 2024-09-16T20:48:40.818 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.818 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.818 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:40.818 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389759+0000", 2024-09-16T20:48:40.818 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047212853999999999 2024-09-16T20:48:40.818 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.818 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.818 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:40.819 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389813+0000", 2024-09-16T20:48:40.819 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3585999999999998e-05 2024-09-16T20:48:40.819 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:40.819 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:40.819 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:40.819 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389879+0000", 2024-09-16T20:48:40.819 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5945999999999995e-05 2024-09-16T20:48:40.819 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:40.819 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:40.819 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:40.819 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:40.819 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:40.820 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:40.821 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:40.819+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:40.822 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:48:40.823 INFO:teuthology.orchestra.run.smithi139.stdout:1805 punch from 0x2ae9800 to 0x2af0200, (0x6a00 bytes) 2024-09-16T20:48:40.823 INFO:teuthology.orchestra.run.smithi139.stdout:skipping zero length punch hole 2024-09-16T20:48:40.823 INFO:teuthology.orchestra.run.smithi139.stdout:1811 trunc from 0x4c59400 to 0x8047a00 2024-09-16T20:48:40.824 INFO:teuthology.orchestra.run.smithi139.stdout:1817 trunc from 0x8047a00 to 0x5cf5800 2024-09-16T20:48:40.824 INFO:teuthology.orchestra.run.smithi139.stdout:1820 punch from 0x1633200 to 0x1642400, (0xf200 bytes) 2024-09-16T20:48:40.824 INFO:teuthology.orchestra.run.smithi139.stdout:1821 read 0x20c4200 thru 0x20ccd36 (0x8b37 bytes) 2024-09-16T20:48:40.824 INFO:teuthology.orchestra.run.smithi139.stdout:1822 punch from 0x5034e00 to 0x503aa00, (0x5c00 bytes) 2024-09-16T20:48:40.824 INFO:teuthology.orchestra.run.smithi139.stdout:1823 read 0xe9a600 thru 0xe9f9d6 (0x53d7 bytes) 2024-09-16T20:48:40.824 INFO:teuthology.orchestra.run.smithi139.stdout:1826 trunc from 0x5cf5800 to 0x5f85200 2024-09-16T20:48:40.824 INFO:teuthology.orchestra.run.smithi139.stdout:1827 punch from 0x1d88400 to 0x1d96c00, (0xe800 bytes) 2024-09-16T20:48:40.824 INFO:teuthology.orchestra.run.smithi139.stdout:1828 trunc from 0x5f85200 to 0x8649800 2024-09-16T20:48:40.824 INFO:teuthology.orchestra.run.smithi139.stdout:1829 trunc from 0x8649800 to 0xdb9c800 2024-09-16T20:48:40.824 INFO:teuthology.orchestra.run.smithi139.stdout:1831 read 0xa1ac00 thru 0xa1d4e0 (0x28e1 bytes) 2024-09-16T20:48:40.824 INFO:teuthology.orchestra.run.smithi139.stdout:1832 punch from 0x5cbe000 to 0x5cbe400, (0x400 bytes) 2024-09-16T20:48:40.824 INFO:teuthology.orchestra.run.smithi139.stdout:1833 read 0xb9b7200 thru 0xb9c12bd (0xa0be bytes) 2024-09-16T20:48:40.825 INFO:teuthology.orchestra.run.smithi139.stdout:1834 write 0xc88c800 thru 0xc88fa04 (0x3205 bytes) 2024-09-16T20:48:40.825 INFO:teuthology.orchestra.run.smithi139.stdout:1835 read 0xd66c800 thru 0xd66ed54 (0x2555 bytes) 2024-09-16T20:48:40.831 INFO:teuthology.orchestra.run.smithi139.stdout:1836 trunc from 0xdb9c800 to 0x4fec400 2024-09-16T20:48:40.862 INFO:teuthology.orchestra.run.smithi139.stdout:1837 write 0xa36e400 thru 0xa37c326 (0xdf27 bytes) 2024-09-16T20:48:40.866 INFO:teuthology.orchestra.run.smithi139.stdout:1840 trunc from 0xa37c400 to 0xc6f8c00 2024-09-16T20:48:40.877 INFO:teuthology.orchestra.run.smithi139.stdout:1842 read 0x29f6c00 thru 0x29f90b4 (0x24b5 bytes) 2024-09-16T20:48:40.879 INFO:teuthology.orchestra.run.smithi139.stdout:1844 trunc from 0xc6f8c00 to 0xc596200 2024-09-16T20:48:40.898 INFO:teuthology.orchestra.run.smithi139.stdout:1846 write 0xc9bee00 thru 0xc9c0ce4 (0x1ee5 bytes) 2024-09-16T20:48:40.899 INFO:teuthology.orchestra.run.smithi139.stdout:1847 read 0xc8b1600 thru 0xc8b24b8 (0xeb9 bytes) 2024-09-16T20:48:40.899 INFO:teuthology.orchestra.run.smithi139.stdout:1848 clone 13 order 21 su 524288 sc 4 2024-09-16T20:48:40.921 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:40.920+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:40.975 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:40.975 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:48:40.975 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:48:40.975 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:40.983 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:48:41.022 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:41.021+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:41.123 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:41.122+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:41.130 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:41.130 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:48:41.130 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:48:41.130 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:48:41.130 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:41.139 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:48:41.224 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:41.223+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:41.287 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:41.288 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:48:41.288 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:48:41.288 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:48:41.288 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.288 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3709 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1464320~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:41.288 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.638971+0000", 2024-09-16T20:48:41.288 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 18.647025030000002, 2024-09-16T20:48:41.288 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049858670000000001, 2024-09-16T20:48:41.288 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:41.288 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:41.288 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:41.288 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:41.288 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:41.289 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:41.289 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3709 2024-09-16T20:48:41.289 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.289 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:41.289 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.289 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:41.289 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638971+0000", 2024-09-16T20:48:41.289 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:41.289 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.289 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.289 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:41.289 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638971+0000", 2024-09-16T20:48:41.289 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:41.290 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.290 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.290 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:41.290 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638972+0000", 2024-09-16T20:48:41.290 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0400000000000002e-07 2024-09-16T20:48:41.290 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.290 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.290 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:41.290 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638973+0000", 2024-09-16T20:48:41.290 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.578e-06 2024-09-16T20:48:41.290 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.290 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.290 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:41.290 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638979+0000", 2024-09-16T20:48:41.291 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.502e-06 2024-09-16T20:48:41.291 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.291 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.291 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:41.291 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638983+0000", 2024-09-16T20:48:41.291 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2690000000000001e-06 2024-09-16T20:48:41.291 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.291 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.291 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:41.291 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688628+0000", 2024-09-16T20:48:41.291 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049644547999999997 2024-09-16T20:48:41.291 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.291 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.292 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:41.292 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688718+0000", 2024-09-16T20:48:41.292 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0802000000000002e-05 2024-09-16T20:48:41.292 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.292 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.292 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:41.292 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688830+0000", 2024-09-16T20:48:41.292 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000111667 2024-09-16T20:48:41.292 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:41.292 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:41.292 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:41.292 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.292 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.292 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3712 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1472512~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:41.293 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:48:41.293 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 18.647008501999998, 2024-09-16T20:48:41.293 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050004305999999998, 2024-09-16T20:48:41.293 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:41.293 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:41.293 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:41.293 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:41.293 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:41.293 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:41.293 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3712 2024-09-16T20:48:41.293 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.293 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:41.293 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.294 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:41.294 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:48:41.294 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:41.294 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.294 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.294 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:41.294 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:48:41.294 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:41.294 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.294 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.294 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:41.294 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:48:41.294 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0199999999999998e-07 2024-09-16T20:48:41.294 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.295 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.295 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:41.295 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638990+0000", 2024-09-16T20:48:41.295 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.618e-06 2024-09-16T20:48:41.295 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.295 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.295 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:41.295 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638995+0000", 2024-09-16T20:48:41.295 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3940000000000004e-06 2024-09-16T20:48:41.295 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.295 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.295 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:41.295 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639000+0000", 2024-09-16T20:48:41.295 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9309999999999999e-06 2024-09-16T20:48:41.296 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.296 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.296 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:41.296 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688867+0000", 2024-09-16T20:48:41.296 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049866955999999997 2024-09-16T20:48:41.296 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.296 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.296 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:41.296 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688933+0000", 2024-09-16T20:48:41.296 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5359999999999998e-05 2024-09-16T20:48:41.296 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.296 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.297 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:41.297 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688992+0000", 2024-09-16T20:48:41.297 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9744999999999998e-05 2024-09-16T20:48:41.297 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:41.297 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:41.297 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:41.297 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.297 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.297 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3714 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1480704~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:41.297 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:48:41.297 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 18.646990912, 2024-09-16T20:48:41.297 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050152552000000003, 2024-09-16T20:48:41.297 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:41.298 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:41.298 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:41.298 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:41.298 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:41.298 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:41.298 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3714 2024-09-16T20:48:41.298 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.298 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:41.298 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.298 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:41.298 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:48:41.298 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:41.298 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.298 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.299 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:41.299 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:48:41.299 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:41.299 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.299 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.299 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:41.299 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:48:41.299 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0499999999999999e-07 2024-09-16T20:48:41.299 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.299 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.299 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:41.299 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639007+0000", 2024-09-16T20:48:41.299 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5340000000000001e-06 2024-09-16T20:48:41.300 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.300 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.300 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:41.300 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639013+0000", 2024-09-16T20:48:41.300 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3260000000000003e-06 2024-09-16T20:48:41.300 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.300 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.300 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:41.300 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639018+0000", 2024-09-16T20:48:41.300 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2850000000000002e-06 2024-09-16T20:48:41.300 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.300 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.300 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:41.301 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689026+0000", 2024-09-16T20:48:41.301 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.05000798 2024-09-16T20:48:41.301 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.301 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.301 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:41.301 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689077+0000", 2024-09-16T20:48:41.301 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1063999999999998e-05 2024-09-16T20:48:41.301 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.301 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.301 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:41.301 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689158+0000", 2024-09-16T20:48:41.301 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1057999999999995e-05 2024-09-16T20:48:41.301 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:41.301 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:41.302 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:41.302 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.302 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.302 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3716 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1488896~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:41.302 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:48:41.302 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 18.646973102, 2024-09-16T20:48:41.302 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050280142, 2024-09-16T20:48:41.302 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:41.302 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:41.302 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:41.302 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:41.302 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:41.302 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:41.303 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3716 2024-09-16T20:48:41.303 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.303 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:41.303 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.303 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:41.303 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:48:41.303 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:41.303 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.303 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.303 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:41.303 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:48:41.303 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:41.303 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.303 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.304 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:41.304 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639024+0000", 2024-09-16T20:48:41.304 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9999999999999997e-07 2024-09-16T20:48:41.304 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.304 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.304 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:41.304 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639026+0000", 2024-09-16T20:48:41.304 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.781e-06 2024-09-16T20:48:41.304 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.304 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.304 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:41.304 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639032+0000", 2024-09-16T20:48:41.304 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7780000000000004e-06 2024-09-16T20:48:41.304 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.305 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.305 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:41.305 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639036+0000", 2024-09-16T20:48:41.305 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6909999999999996e-06 2024-09-16T20:48:41.305 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.305 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.305 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:41.305 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689191+0000", 2024-09-16T20:48:41.305 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050155284000000001 2024-09-16T20:48:41.305 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.305 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.305 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:41.305 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689242+0000", 2024-09-16T20:48:41.306 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0662999999999999e-05 2024-09-16T20:48:41.306 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.306 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.306 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:41.306 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689304+0000", 2024-09-16T20:48:41.306 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1345000000000003e-05 2024-09-16T20:48:41.306 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:41.306 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:41.306 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:41.306 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.306 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.306 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3719 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1497088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:41.306 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:48:41.306 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 18.646945823999999, 2024-09-16T20:48:41.307 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050399715999999997, 2024-09-16T20:48:41.307 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:41.307 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:41.307 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:41.307 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:41.307 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:41.307 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:41.307 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3719 2024-09-16T20:48:41.307 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.307 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:41.307 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.307 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:41.307 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:48:41.307 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:41.308 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.308 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.308 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:41.308 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:48:41.308 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:41.308 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.308 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.308 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:41.308 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:48:41.308 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9700000000000002e-07 2024-09-16T20:48:41.308 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.308 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.308 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:41.308 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639053+0000", 2024-09-16T20:48:41.309 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2900000000000001e-06 2024-09-16T20:48:41.309 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.309 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.309 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:41.309 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639060+0000", 2024-09-16T20:48:41.309 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7750000000000004e-06 2024-09-16T20:48:41.309 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.309 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.309 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:41.309 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639066+0000", 2024-09-16T20:48:41.309 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0639999999999999e-06 2024-09-16T20:48:41.309 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.309 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.310 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:41.310 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689337+0000", 2024-09-16T20:48:41.310 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050270784999999998 2024-09-16T20:48:41.310 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.310 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.310 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:41.310 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689392+0000", 2024-09-16T20:48:41.310 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4920999999999998e-05 2024-09-16T20:48:41.310 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.310 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.310 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:41.310 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689450+0000", 2024-09-16T20:48:41.310 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8483999999999998e-05 2024-09-16T20:48:41.310 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:41.311 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:41.311 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:41.311 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.311 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.311 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3721 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1505280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:41.311 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:48:41.311 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 18.646922692, 2024-09-16T20:48:41.311 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050513608000000002, 2024-09-16T20:48:41.311 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:41.311 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:41.311 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:41.311 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:41.311 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:41.311 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:41.312 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3721 2024-09-16T20:48:41.312 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.312 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:41.312 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.312 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:41.312 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:48:41.312 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:41.312 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.312 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.312 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:41.312 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:48:41.312 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:41.312 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.312 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.313 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:41.313 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:48:41.313 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7e-07 2024-09-16T20:48:41.313 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.313 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.313 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:41.313 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639076+0000", 2024-09-16T20:48:41.313 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8899999999999999e-06 2024-09-16T20:48:41.313 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.313 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.313 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:41.313 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639083+0000", 2024-09-16T20:48:41.313 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1210000000000001e-06 2024-09-16T20:48:41.314 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.314 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.314 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:41.314 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639089+0000", 2024-09-16T20:48:41.314 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.643e-06 2024-09-16T20:48:41.314 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.314 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.314 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:41.314 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689474+0000", 2024-09-16T20:48:41.314 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050385679000000003 2024-09-16T20:48:41.314 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.314 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.314 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:41.314 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689533+0000", 2024-09-16T20:48:41.314 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8303000000000001e-05 2024-09-16T20:48:41.315 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.315 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.315 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:41.315 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689587+0000", 2024-09-16T20:48:41.315 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4601999999999997e-05 2024-09-16T20:48:41.315 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:41.315 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:41.315 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:41.315 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.315 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.315 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3723 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1513472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:41.315 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:48:41.315 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 18.646891887999999, 2024-09-16T20:48:41.315 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050699691999999998, 2024-09-16T20:48:41.316 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:41.316 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:41.316 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:41.316 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:41.316 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:41.316 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:41.316 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3723 2024-09-16T20:48:41.316 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.316 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:41.316 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.316 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:41.316 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:48:41.316 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:41.317 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.317 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.317 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:41.317 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:48:41.317 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:41.317 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.317 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.317 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:41.317 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:48:41.317 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3399999999999999e-07 2024-09-16T20:48:41.317 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.317 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.317 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:41.318 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639107+0000", 2024-09-16T20:48:41.318 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3860000000000001e-06 2024-09-16T20:48:41.318 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.318 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.318 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:41.318 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639114+0000", 2024-09-16T20:48:41.318 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9800000000000001e-06 2024-09-16T20:48:41.318 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.318 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.318 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:41.318 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639129+0000", 2024-09-16T20:48:41.318 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4887e-05 2024-09-16T20:48:41.318 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.318 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.319 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:41.319 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689625+0000", 2024-09-16T20:48:41.319 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050495848000000003 2024-09-16T20:48:41.319 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.319 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.319 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:41.319 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689720+0000", 2024-09-16T20:48:41.319 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4821000000000006e-05 2024-09-16T20:48:41.319 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.319 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.319 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:41.319 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689804+0000", 2024-09-16T20:48:41.319 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4235999999999998e-05 2024-09-16T20:48:41.319 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:41.320 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:41.320 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:41.320 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.320 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.320 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3725 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1521664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:41.320 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:48:41.320 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 18.646858292000001, 2024-09-16T20:48:41.320 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050797660000000001, 2024-09-16T20:48:41.320 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:41.320 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:41.320 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:41.320 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:41.320 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:41.320 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:41.321 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3725 2024-09-16T20:48:41.321 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.321 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:41.321 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.321 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:41.321 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:48:41.321 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:41.321 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.321 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.321 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:41.321 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:48:41.321 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:41.321 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.322 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.322 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:41.322 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639139+0000", 2024-09-16T20:48:41.322 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8999999999999997e-07 2024-09-16T20:48:41.322 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.322 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.322 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:41.322 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639141+0000", 2024-09-16T20:48:41.322 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.182e-06 2024-09-16T20:48:41.322 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.322 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.322 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:41.322 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639154+0000", 2024-09-16T20:48:41.322 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3626e-05 2024-09-16T20:48:41.323 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.323 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.323 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:41.323 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639160+0000", 2024-09-16T20:48:41.323 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6810000000000003e-06 2024-09-16T20:48:41.323 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.323 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.323 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:41.323 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689831+0000", 2024-09-16T20:48:41.323 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050670907000000001 2024-09-16T20:48:41.323 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.323 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.323 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:41.323 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689882+0000", 2024-09-16T20:48:41.324 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0445999999999998e-05 2024-09-16T20:48:41.324 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.324 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.324 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:41.324 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689936+0000", 2024-09-16T20:48:41.324 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4327999999999999e-05 2024-09-16T20:48:41.324 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:41.324 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:41.324 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:41.324 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.324 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.324 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3727 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1529856~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:41.324 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:48:41.324 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 18.646822714999999, 2024-09-16T20:48:41.325 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051016922999999999, 2024-09-16T20:48:41.325 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:41.325 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:41.325 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:41.325 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:41.325 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:41.325 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:41.325 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3727 2024-09-16T20:48:41.325 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.325 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:41.325 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.325 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:41.325 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:48:41.325 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:41.326 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.326 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.326 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:41.326 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:48:41.326 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:41.326 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.326 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.326 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:41.326 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:48:41.326 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9999999999999998e-07 2024-09-16T20:48:41.326 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.326 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.326 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:41.327 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639183+0000", 2024-09-16T20:48:41.327 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2399999999999996e-06 2024-09-16T20:48:41.327 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.327 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.327 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:41.327 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639195+0000", 2024-09-16T20:48:41.327 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1113e-05 2024-09-16T20:48:41.327 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.327 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.327 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:41.327 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639206+0000", 2024-09-16T20:48:41.327 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1503e-05 2024-09-16T20:48:41.327 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.327 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.328 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:41.328 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689969+0000", 2024-09-16T20:48:41.328 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050762676 2024-09-16T20:48:41.328 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.328 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.328 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:41.328 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690071+0000", 2024-09-16T20:48:41.328 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102163 2024-09-16T20:48:41.328 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.328 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.328 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:41.328 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690191+0000", 2024-09-16T20:48:41.328 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000119728 2024-09-16T20:48:41.329 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:41.329 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:41.329 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:41.329 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.329 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.329 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3730 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1538048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:41.329 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:48:41.329 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 18.646781151999999, 2024-09-16T20:48:41.329 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051131467999999999, 2024-09-16T20:48:41.329 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:41.329 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:41.329 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:41.329 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:41.329 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:41.330 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:41.330 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3730 2024-09-16T20:48:41.330 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.330 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:41.330 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.330 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:41.330 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:48:41.330 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:41.330 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.330 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.330 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:41.330 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:48:41.330 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:41.331 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.331 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.331 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:41.331 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639216+0000", 2024-09-16T20:48:41.331 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.39e-07 2024-09-16T20:48:41.331 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.331 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.331 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:41.331 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639218+0000", 2024-09-16T20:48:41.331 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.537e-06 2024-09-16T20:48:41.331 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.331 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.331 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:41.332 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639229+0000", 2024-09-16T20:48:41.332 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1122e-05 2024-09-16T20:48:41.332 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.332 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.332 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:41.332 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639237+0000", 2024-09-16T20:48:41.332 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8509999999999999e-06 2024-09-16T20:48:41.332 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.332 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.332 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:41.332 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690227+0000", 2024-09-16T20:48:41.332 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050990147 2024-09-16T20:48:41.332 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.333 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.333 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:41.333 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690278+0000", 2024-09-16T20:48:41.333 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0223999999999999e-05 2024-09-16T20:48:41.333 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.333 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.333 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:41.333 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690347+0000", 2024-09-16T20:48:41.333 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9147999999999996e-05 2024-09-16T20:48:41.333 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:41.333 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:41.333 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:41.333 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.334 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.334 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3731 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1546240~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:41.334 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:48:41.334 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 18.646750443999998, 2024-09-16T20:48:41.334 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051276744999999999, 2024-09-16T20:48:41.334 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:41.334 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:41.334 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:41.334 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:41.334 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:41.334 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:41.334 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3731 2024-09-16T20:48:41.334 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.334 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:41.335 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.335 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:41.335 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:48:41.335 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:41.335 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.335 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.335 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:41.335 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:48:41.335 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:41.335 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.335 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.335 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:41.335 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:48:41.335 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9900000000000001e-07 2024-09-16T20:48:41.336 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.336 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.336 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:41.336 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639249+0000", 2024-09-16T20:48:41.336 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3630000000000001e-06 2024-09-16T20:48:41.336 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.336 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.336 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:41.336 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639255+0000", 2024-09-16T20:48:41.336 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4960000000000001e-06 2024-09-16T20:48:41.336 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.336 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.336 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:41.337 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639261+0000", 2024-09-16T20:48:41.337 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2839999999999997e-06 2024-09-16T20:48:41.337 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.337 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.337 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:41.337 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690375+0000", 2024-09-16T20:48:41.337 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051114684000000001 2024-09-16T20:48:41.337 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.337 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.337 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:41.337 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690425+0000", 2024-09-16T20:48:41.337 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9379999999999998e-05 2024-09-16T20:48:41.337 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.337 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.338 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:41.338 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690523+0000", 2024-09-16T20:48:41.338 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8139000000000003e-05 2024-09-16T20:48:41.338 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:41.338 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:41.338 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:41.338 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.338 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.338 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3732 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1554432~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:41.338 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:48:41.338 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 18.646704662000001, 2024-09-16T20:48:41.338 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051380096, 2024-09-16T20:48:41.338 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:41.338 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:41.339 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:41.339 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:41.339 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:41.339 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:41.339 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3732 2024-09-16T20:48:41.339 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.339 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:41.339 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.339 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:41.339 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:48:41.339 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:41.339 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.339 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.340 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:41.340 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:48:41.340 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:41.340 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.340 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.340 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:41.340 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:48:41.340 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6100000000000001e-07 2024-09-16T20:48:41.340 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.340 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.340 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:41.340 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639295+0000", 2024-09-16T20:48:41.340 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.447e-06 2024-09-16T20:48:41.340 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.341 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.341 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:41.341 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639302+0000", 2024-09-16T20:48:41.341 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0570000000000004e-06 2024-09-16T20:48:41.341 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.341 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.341 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:41.341 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639307+0000", 2024-09-16T20:48:41.341 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1429999999999997e-06 2024-09-16T20:48:41.341 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.341 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.341 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:41.341 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690555+0000", 2024-09-16T20:48:41.341 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051247963000000001 2024-09-16T20:48:41.342 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.342 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.342 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:41.342 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690602+0000", 2024-09-16T20:48:41.342 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7602000000000003e-05 2024-09-16T20:48:41.342 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.342 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.342 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:41.342 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690672+0000", 2024-09-16T20:48:41.342 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9423000000000003e-05 2024-09-16T20:48:41.342 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:41.342 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:41.342 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:41.342 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.343 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.343 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3734 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1562624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:41.343 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:48:41.343 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 18.646683012, 2024-09-16T20:48:41.343 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051490610999999999, 2024-09-16T20:48:41.343 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:41.343 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:41.343 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:41.343 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:41.343 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:41.343 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:41.343 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3734 2024-09-16T20:48:41.343 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.343 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:41.344 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.344 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:41.344 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:48:41.344 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:41.344 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.344 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.344 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:41.344 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:48:41.344 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:41.344 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.344 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.344 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:41.344 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639314+0000", 2024-09-16T20:48:41.345 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4400000000000001e-07 2024-09-16T20:48:41.345 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.345 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.345 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:41.345 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639316+0000", 2024-09-16T20:48:41.345 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1459999999999999e-06 2024-09-16T20:48:41.345 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.345 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.345 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:41.345 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639322+0000", 2024-09-16T20:48:41.345 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7610000000000004e-06 2024-09-16T20:48:41.345 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.345 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.345 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:41.346 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639327+0000", 2024-09-16T20:48:41.346 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.186e-06 2024-09-16T20:48:41.346 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.346 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.346 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:41.346 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690698+0000", 2024-09-16T20:48:41.346 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051371223000000001 2024-09-16T20:48:41.346 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.346 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.346 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:41.346 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690748+0000", 2024-09-16T20:48:41.346 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0192999999999999e-05 2024-09-16T20:48:41.346 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.346 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.347 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:41.347 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690804+0000", 2024-09-16T20:48:41.347 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5658000000000001e-05 2024-09-16T20:48:41.347 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:41.347 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:41.347 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:41.347 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.347 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.347 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3736 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1570816~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:41.347 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:48:41.347 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 18.646663752999999, 2024-09-16T20:48:41.347 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051596528000000003, 2024-09-16T20:48:41.347 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:41.348 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:41.348 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:41.348 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:41.348 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:41.348 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:41.348 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3736 2024-09-16T20:48:41.348 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.348 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:41.348 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.348 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:41.348 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:48:41.348 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:41.348 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.349 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.349 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:41.349 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:48:41.349 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:41.349 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.349 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.349 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:41.349 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:48:41.349 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9999999999999999e-07 2024-09-16T20:48:41.349 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.349 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.349 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:41.350 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639335+0000", 2024-09-16T20:48:41.350 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.635e-06 2024-09-16T20:48:41.350 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.350 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.350 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:41.350 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639340+0000", 2024-09-16T20:48:41.350 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5840000000000002e-06 2024-09-16T20:48:41.350 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.350 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.350 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:41.350 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639346+0000", 2024-09-16T20:48:41.350 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1190000000000002e-06 2024-09-16T20:48:41.350 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.350 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.351 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:41.351 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690817+0000", 2024-09-16T20:48:41.351 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051470721999999997 2024-09-16T20:48:41.351 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.351 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.351 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:41.351 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690875+0000", 2024-09-16T20:48:41.351 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7441000000000001e-05 2024-09-16T20:48:41.351 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.351 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.351 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:41.351 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690929+0000", 2024-09-16T20:48:41.351 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4727e-05 2024-09-16T20:48:41.352 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:41.352 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:41.352 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:41.352 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.352 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.352 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5434 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:41.352 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:48:41.352 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 18.421402186999998, 2024-09-16T20:48:41.352 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099755831000000003, 2024-09-16T20:48:41.352 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:41.352 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:41.352 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:41.352 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:41.353 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:41.353 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:41.353 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5434 2024-09-16T20:48:41.353 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.353 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:41.353 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.353 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:41.353 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:48:41.353 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:41.353 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.353 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.353 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:41.353 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:48:41.353 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:41.354 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.354 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.354 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:41.354 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864595+0000", 2024-09-16T20:48:41.354 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1099999999999995e-07 2024-09-16T20:48:41.354 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.354 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.354 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:41.354 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864597+0000", 2024-09-16T20:48:41.354 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3989999999999998e-06 2024-09-16T20:48:41.354 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.354 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.354 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:41.354 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864603+0000", 2024-09-16T20:48:41.355 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9309999999999996e-06 2024-09-16T20:48:41.355 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.355 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.355 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:41.355 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864607+0000", 2024-09-16T20:48:41.355 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6200000000000001e-06 2024-09-16T20:48:41.355 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.355 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.355 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:41.355 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964164+0000", 2024-09-16T20:48:41.355 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099557162000000005 2024-09-16T20:48:41.355 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.355 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.355 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:41.356 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964244+0000", 2024-09-16T20:48:41.356 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9597000000000004e-05 2024-09-16T20:48:41.356 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.356 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.356 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:41.356 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964350+0000", 2024-09-16T20:48:41.356 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010641099999999999 2024-09-16T20:48:41.356 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:41.356 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:41.356 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:41.356 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.356 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.356 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5435 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 20480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:41.357 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:48:41.357 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 18.421372365, 2024-09-16T20:48:41.357 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100897102, 2024-09-16T20:48:41.357 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:41.357 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:41.357 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:41.357 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:41.357 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:41.357 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:41.357 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5435 2024-09-16T20:48:41.357 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.357 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:41.357 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.357 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:41.358 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:48:41.358 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:41.358 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.358 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.358 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:41.358 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:48:41.358 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:41.358 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.358 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.358 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:41.358 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:48:41.358 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8000000000000001e-07 2024-09-16T20:48:41.358 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.359 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.359 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:41.359 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864626+0000", 2024-09-16T20:48:41.359 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.745e-06 2024-09-16T20:48:41.359 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.359 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.359 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:41.359 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864633+0000", 2024-09-16T20:48:41.359 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3339999999999998e-06 2024-09-16T20:48:41.359 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.359 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.359 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:41.359 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864636+0000", 2024-09-16T20:48:41.360 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1200000000000002e-06 2024-09-16T20:48:41.360 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.360 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.360 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:41.360 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965299+0000", 2024-09-16T20:48:41.360 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10066352000000001 2024-09-16T20:48:41.360 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.360 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.360 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:41.360 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965390+0000", 2024-09-16T20:48:41.360 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0894000000000001e-05 2024-09-16T20:48:41.360 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.360 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.361 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:41.361 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965521+0000", 2024-09-16T20:48:41.361 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013110899999999999 2024-09-16T20:48:41.361 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:41.361 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:41.361 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:41.361 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.361 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.361 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:152 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 327680~32768 out=32768b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:48:41.361 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:48:41.361 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 17.122875423, 2024-09-16T20:48:41.361 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11869405800000001, 2024-09-16T20:48:41.361 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:41.361 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:41.362 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:41.362 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:41.362 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:48:41.362 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:48:41.362 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 152 2024-09-16T20:48:41.362 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.362 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:41.362 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.362 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:41.362 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:48:41.362 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:41.362 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.362 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.362 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:41.363 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:48:41.363 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:41.363 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.363 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.363 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:41.363 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163122+0000", 2024-09-16T20:48:41.363 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6400000000000003e-07 2024-09-16T20:48:41.363 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.363 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.363 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:41.363 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163124+0000", 2024-09-16T20:48:41.363 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.565e-06 2024-09-16T20:48:41.363 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.364 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.364 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:41.364 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163136+0000", 2024-09-16T20:48:41.364 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1717e-05 2024-09-16T20:48:41.364 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.364 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.364 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:41.364 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163142+0000", 2024-09-16T20:48:41.364 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1750000000000002e-06 2024-09-16T20:48:41.364 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.364 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.364 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:41.364 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163606+0000", 2024-09-16T20:48:41.364 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00046418799999999998 2024-09-16T20:48:41.365 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.365 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.365 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:41.365 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163656+0000", 2024-09-16T20:48:41.365 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9852999999999999e-05 2024-09-16T20:48:41.365 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.365 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.365 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:41.365 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281815+0000", 2024-09-16T20:48:41.365 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11815909600000001 2024-09-16T20:48:41.365 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:41.365 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:41.365 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:41.366 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.366 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.366 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:153 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:48:41.366 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:48:41.366 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 17.122842469999998, 2024-09-16T20:48:41.366 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11903462400000001, 2024-09-16T20:48:41.366 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:41.366 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:41.366 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:41.366 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:41.366 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:48:41.366 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:48:41.366 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 153 2024-09-16T20:48:41.366 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.367 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:41.367 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.367 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:41.367 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:48:41.367 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:41.367 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.367 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.367 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:41.367 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:48:41.367 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:41.367 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.367 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.367 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:41.367 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163155+0000", 2024-09-16T20:48:41.368 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.37e-07 2024-09-16T20:48:41.368 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.368 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.368 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:41.368 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163157+0000", 2024-09-16T20:48:41.368 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6010000000000002e-06 2024-09-16T20:48:41.368 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.368 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.368 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:41.368 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163165+0000", 2024-09-16T20:48:41.368 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2619999999999998e-06 2024-09-16T20:48:41.368 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.368 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.369 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:41.369 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163171+0000", 2024-09-16T20:48:41.369 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.711e-06 2024-09-16T20:48:41.369 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.369 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.369 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:41.369 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281868+0000", 2024-09-16T20:48:41.369 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118696622 2024-09-16T20:48:41.369 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.369 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.369 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:41.369 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281969+0000", 2024-09-16T20:48:41.369 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101371 2024-09-16T20:48:41.369 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.370 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.370 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:41.370 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282189+0000", 2024-09-16T20:48:41.370 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00021952000000000001 2024-09-16T20:48:41.370 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:41.370 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:41.370 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:41.370 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.370 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.370 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:154 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 360448~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:48:41.370 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:48:41.370 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 17.122810697999999, 2024-09-16T20:48:41.370 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11932035100000001, 2024-09-16T20:48:41.370 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:41.371 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:41.371 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:41.371 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:41.371 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:48:41.371 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:48:41.371 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 154 2024-09-16T20:48:41.371 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.371 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:41.371 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.371 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:41.371 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:48:41.371 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:41.371 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.371 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.372 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:41.372 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:48:41.372 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:41.372 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.372 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.372 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:41.372 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:48:41.372 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:48:41.372 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.372 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.372 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:41.372 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163189+0000", 2024-09-16T20:48:41.372 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6840000000000001e-06 2024-09-16T20:48:41.372 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.373 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.373 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:41.373 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163196+0000", 2024-09-16T20:48:41.373 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7920000000000004e-06 2024-09-16T20:48:41.373 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.373 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.373 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:41.373 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163201+0000", 2024-09-16T20:48:41.373 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2809999999999999e-06 2024-09-16T20:48:41.373 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.373 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.373 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:41.373 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282219+0000", 2024-09-16T20:48:41.374 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.119018343 2024-09-16T20:48:41.374 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.374 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.374 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:41.374 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282305+0000", 2024-09-16T20:48:41.374 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5865000000000005e-05 2024-09-16T20:48:41.374 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.374 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.374 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:41.374 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282506+0000", 2024-09-16T20:48:41.374 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000200927 2024-09-16T20:48:41.374 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:41.374 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:41.374 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:41.375 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.375 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.375 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop(client.4366.0:3088 3.1e e38/23 3:79e5ec01:::rbd_header.110e891a9b2b:head v 38'306, mlcod=36'304)", 2024-09-16T20:48:41.375 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:48:41.375 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 17.121791829999999, 2024-09-16T20:48:41.375 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118102759, 2024-09-16T20:48:41.375 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:41.375 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:41.375 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:41.375 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:41.375 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.375 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:41.375 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:48:41.375 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:41.376 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.376 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.376 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:41.376 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:48:41.376 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:41.376 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.376 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.376 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:41.376 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:48:41.376 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.85e-07 2024-09-16T20:48:41.376 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.376 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.376 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:41.376 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164212+0000", 2024-09-16T20:48:41.377 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3999999999999997e-06 2024-09-16T20:48:41.377 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.377 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.377 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:41.377 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281553+0000", 2024-09-16T20:48:41.377 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11734135499999999 2024-09-16T20:48:41.377 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.377 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.377 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:41.377 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281567+0000", 2024-09-16T20:48:41.377 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4188999999999999e-05 2024-09-16T20:48:41.377 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.377 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.378 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:41.378 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281653+0000", 2024-09-16T20:48:41.378 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5385999999999999e-05 2024-09-16T20:48:41.378 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.378 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.378 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:41.378 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281679+0000", 2024-09-16T20:48:41.378 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6689000000000001e-05 2024-09-16T20:48:41.378 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.378 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.378 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:41.378 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282277+0000", 2024-09-16T20:48:41.378 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00059798000000000002 2024-09-16T20:48:41.378 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:41.379 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:41.379 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:41.379 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282307+0000", 2024-09-16T20:48:41.379 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9975e-05 2024-09-16T20:48:41.379 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:41.379 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:41.379 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:41.379 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:41.379 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:41.379 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:41.381 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:48:41.381 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:41.380+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:41.481 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:41.480+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:41.491 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:41.491 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:48:41.491 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:48:41.491 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:41.499 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:48:41.583 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:41.583+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:41.605 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:41.606 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:48:41.606 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:48:41.606 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:48:41.606 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:41.614 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:48:41.685 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:41.683+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:41.727 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:41.727 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:48:41.728 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:48:41.728 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:48:41.728 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.728 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6412 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3561472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:41.728 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871685+0000", 2024-09-16T20:48:41.728 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 21.854303632000001, 2024-09-16T20:48:41.728 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048146568000000001, 2024-09-16T20:48:41.728 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:41.728 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:41.728 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:41.728 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:41.728 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:41.728 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:41.729 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6412 2024-09-16T20:48:41.729 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.729 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:41.729 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.729 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:41.729 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871685+0000", 2024-09-16T20:48:41.729 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:41.729 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.729 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.729 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:41.729 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871685+0000", 2024-09-16T20:48:41.729 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:41.729 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.729 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.730 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:41.730 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871685+0000", 2024-09-16T20:48:41.730 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.58e-07 2024-09-16T20:48:41.730 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.730 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.730 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:41.730 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871687+0000", 2024-09-16T20:48:41.730 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1830000000000001e-06 2024-09-16T20:48:41.730 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.730 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.730 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:41.730 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871694+0000", 2024-09-16T20:48:41.730 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7689999999999999e-06 2024-09-16T20:48:41.731 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.731 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.731 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:41.731 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871699+0000", 2024-09-16T20:48:41.731 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1769999999999997e-06 2024-09-16T20:48:41.731 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.731 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.731 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:41.731 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.919662+0000", 2024-09-16T20:48:41.731 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047963121999999997 2024-09-16T20:48:41.731 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.731 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.731 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:41.731 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.919744+0000", 2024-09-16T20:48:41.732 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1997999999999997e-05 2024-09-16T20:48:41.732 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.732 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.732 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:41.732 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.919831+0000", 2024-09-16T20:48:41.732 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6860999999999994e-05 2024-09-16T20:48:41.732 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:41.732 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:41.732 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:41.732 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.732 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.732 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6414 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3569664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:41.732 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871708+0000", 2024-09-16T20:48:41.733 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 21.854280684999999, 2024-09-16T20:48:41.733 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048373317999999998, 2024-09-16T20:48:41.733 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:41.733 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:41.733 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:41.733 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:41.733 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:41.733 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:41.733 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6414 2024-09-16T20:48:41.733 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.733 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:41.733 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.733 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:41.733 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871708+0000", 2024-09-16T20:48:41.734 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:41.734 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.734 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.734 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:41.734 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871708+0000", 2024-09-16T20:48:41.734 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:41.734 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.734 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.734 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:41.734 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871708+0000", 2024-09-16T20:48:41.734 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9799999999999999e-07 2024-09-16T20:48:41.734 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.734 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.734 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:41.734 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871710+0000", 2024-09-16T20:48:41.735 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2699999999999999e-06 2024-09-16T20:48:41.735 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.735 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.735 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:41.735 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871718+0000", 2024-09-16T20:48:41.735 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8499999999999994e-06 2024-09-16T20:48:41.735 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.735 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.735 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:41.735 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871723+0000", 2024-09-16T20:48:41.735 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1819999999999997e-06 2024-09-16T20:48:41.735 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.735 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.735 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:41.736 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.919883+0000", 2024-09-16T20:48:41.736 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048159955999999997 2024-09-16T20:48:41.736 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.736 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.736 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:41.736 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.919969+0000", 2024-09-16T20:48:41.736 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5808000000000002e-05 2024-09-16T20:48:41.736 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.736 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.736 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:41.736 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920081+0000", 2024-09-16T20:48:41.736 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000111854 2024-09-16T20:48:41.736 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:41.736 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:41.737 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:41.737 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.737 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.737 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6415 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3577856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:41.737 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871730+0000", 2024-09-16T20:48:41.737 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 21.854258170000001, 2024-09-16T20:48:41.737 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.04851482, 2024-09-16T20:48:41.737 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:41.737 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:41.737 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:41.737 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:41.737 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:41.737 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:41.737 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6415 2024-09-16T20:48:41.738 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.738 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:41.738 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.738 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:41.738 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871730+0000", 2024-09-16T20:48:41.738 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:41.738 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.738 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.738 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:41.738 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871730+0000", 2024-09-16T20:48:41.738 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:41.738 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.738 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.738 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:41.739 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871731+0000", 2024-09-16T20:48:41.739 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6899999999999998e-07 2024-09-16T20:48:41.739 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.739 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.739 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:41.739 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871733+0000", 2024-09-16T20:48:41.739 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0600000000000002e-06 2024-09-16T20:48:41.739 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.739 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.739 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:41.739 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871740+0000", 2024-09-16T20:48:41.739 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3749999999999997e-06 2024-09-16T20:48:41.739 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.739 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.740 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:41.740 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871749+0000", 2024-09-16T20:48:41.740 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9109999999999999e-06 2024-09-16T20:48:41.740 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.740 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.740 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:41.740 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920129+0000", 2024-09-16T20:48:41.740 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048380175999999997 2024-09-16T20:48:41.740 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.740 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.740 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:41.740 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920181+0000", 2024-09-16T20:48:41.740 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2077000000000003e-05 2024-09-16T20:48:41.740 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.741 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.741 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:41.741 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920245+0000", 2024-09-16T20:48:41.741 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3851999999999998e-05 2024-09-16T20:48:41.741 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:41.741 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:41.741 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:41.741 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.741 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.741 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6417 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3586048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:41.741 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871756+0000", 2024-09-16T20:48:41.742 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 21.854232535000001, 2024-09-16T20:48:41.744 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048628301999999998, 2024-09-16T20:48:41.744 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:41.744 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:41.744 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:41.744 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:41.744 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:41.744 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:41.744 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6417 2024-09-16T20:48:41.744 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.745 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:41.745 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.745 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:41.745 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871756+0000", 2024-09-16T20:48:41.745 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:41.745 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.745 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.745 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:41.745 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871756+0000", 2024-09-16T20:48:41.745 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:41.745 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.745 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.745 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:41.745 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871756+0000", 2024-09-16T20:48:41.746 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0100000000000005e-07 2024-09-16T20:48:41.746 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.746 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.746 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:41.746 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871758+0000", 2024-09-16T20:48:41.746 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.902e-06 2024-09-16T20:48:41.746 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.746 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.746 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:41.746 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871764+0000", 2024-09-16T20:48:41.746 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8170000000000004e-06 2024-09-16T20:48:41.746 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.746 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.747 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:41.747 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871769+0000", 2024-09-16T20:48:41.747 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6500000000000004e-06 2024-09-16T20:48:41.748 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.748 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.748 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:41.748 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920276+0000", 2024-09-16T20:48:41.748 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048507794999999999 2024-09-16T20:48:41.748 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.748 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.748 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:41.748 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920329+0000", 2024-09-16T20:48:41.748 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2534999999999998e-05 2024-09-16T20:48:41.748 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.749 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.749 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:41.749 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920384+0000", 2024-09-16T20:48:41.749 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5102000000000002e-05 2024-09-16T20:48:41.749 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:41.749 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:41.749 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:41.749 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.749 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.749 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6419 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3594240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:41.749 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871775+0000", 2024-09-16T20:48:41.749 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 21.854213357999999, 2024-09-16T20:48:41.749 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048754724999999999, 2024-09-16T20:48:41.750 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:41.750 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:41.750 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:41.750 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:41.750 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:41.750 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:41.750 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6419 2024-09-16T20:48:41.750 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.750 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:41.750 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.750 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:41.750 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871775+0000", 2024-09-16T20:48:41.750 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:41.750 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.751 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.751 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:41.751 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871775+0000", 2024-09-16T20:48:41.751 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:41.751 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.751 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.751 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:41.751 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871775+0000", 2024-09-16T20:48:41.751 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2500000000000001e-07 2024-09-16T20:48:41.751 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.751 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.752 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:41.752 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871777+0000", 2024-09-16T20:48:41.752 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9649999999999998e-06 2024-09-16T20:48:41.752 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.752 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.752 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:41.752 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871784+0000", 2024-09-16T20:48:41.752 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3419999999999996e-06 2024-09-16T20:48:41.752 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.752 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.752 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:41.753 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871789+0000", 2024-09-16T20:48:41.753 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1889999999999998e-06 2024-09-16T20:48:41.753 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.753 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.753 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:41.753 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920416+0000", 2024-09-16T20:48:41.753 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048627363999999999 2024-09-16T20:48:41.753 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.753 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.753 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:41.753 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920472+0000", 2024-09-16T20:48:41.753 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5429000000000001e-05 2024-09-16T20:48:41.753 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.754 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.754 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:41.754 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920530+0000", 2024-09-16T20:48:41.754 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8111e-05 2024-09-16T20:48:41.754 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:41.754 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:41.754 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:41.754 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.754 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.754 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6420 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3602432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:41.754 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871796+0000", 2024-09-16T20:48:41.754 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 21.854192444999999, 2024-09-16T20:48:41.754 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048857298, 2024-09-16T20:48:41.755 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:41.755 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:41.755 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:41.755 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:41.755 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:41.755 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:41.755 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6420 2024-09-16T20:48:41.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.755 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:41.755 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.755 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:41.755 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871796+0000", 2024-09-16T20:48:41.755 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:41.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.755 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.756 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:41.756 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871796+0000", 2024-09-16T20:48:41.756 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:41.756 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.756 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.756 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:41.756 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871854+0000", 2024-09-16T20:48:41.756 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8187999999999999e-05 2024-09-16T20:48:41.756 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.756 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.756 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:41.756 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871856+0000", 2024-09-16T20:48:41.756 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3549999999999999e-06 2024-09-16T20:48:41.757 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.757 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.757 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:41.757 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871866+0000", 2024-09-16T20:48:41.757 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6500000000000008e-06 2024-09-16T20:48:41.757 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.757 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.757 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:41.757 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871872+0000", 2024-09-16T20:48:41.757 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3799999999999999e-06 2024-09-16T20:48:41.757 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.757 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.758 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:41.758 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920558+0000", 2024-09-16T20:48:41.758 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048685797000000003 2024-09-16T20:48:41.758 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.758 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.758 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:41.758 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920603+0000", 2024-09-16T20:48:41.758 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4629999999999998e-05 2024-09-16T20:48:41.758 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.758 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.758 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:41.758 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920653+0000", 2024-09-16T20:48:41.758 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0297999999999999e-05 2024-09-16T20:48:41.758 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:41.759 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:41.759 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:41.759 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.759 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.760 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6421 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3610624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:41.760 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871882+0000", 2024-09-16T20:48:41.760 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 21.854106377000001, 2024-09-16T20:48:41.760 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048913381999999998, 2024-09-16T20:48:41.760 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:41.760 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:41.760 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:41.760 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:41.760 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:41.761 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:41.761 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6421 2024-09-16T20:48:41.761 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.761 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:41.761 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.761 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:41.761 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871882+0000", 2024-09-16T20:48:41.761 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:41.761 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.761 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.761 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:41.761 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871882+0000", 2024-09-16T20:48:41.761 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:41.762 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.762 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.762 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:41.762 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.873257+0000", 2024-09-16T20:48:41.762 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001374789 2024-09-16T20:48:41.762 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.762 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.762 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:41.762 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.873261+0000", 2024-09-16T20:48:41.762 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5589999999999999e-06 2024-09-16T20:48:41.762 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.762 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.762 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:41.763 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.873275+0000", 2024-09-16T20:48:41.763 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3675999999999999e-05 2024-09-16T20:48:41.763 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.763 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.763 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:41.763 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.873281+0000", 2024-09-16T20:48:41.763 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6930000000000004e-06 2024-09-16T20:48:41.763 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.763 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.763 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:41.763 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920684+0000", 2024-09-16T20:48:41.763 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047402913999999997 2024-09-16T20:48:41.763 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.763 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.764 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:41.764 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920737+0000", 2024-09-16T20:48:41.764 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3600000000000002e-05 2024-09-16T20:48:41.764 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.764 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.764 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:41.764 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920795+0000", 2024-09-16T20:48:41.764 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8150999999999999e-05 2024-09-16T20:48:41.764 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:41.764 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:41.764 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:41.764 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.765 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.765 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:7491 3.1e 3:7c48f6ae:::rbd_object_map.110e891a9b2b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:41.765 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.925865+0000", 2024-09-16T20:48:41.765 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 21.800123233000001, 2024-09-16T20:48:41.765 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048897293000000001, 2024-09-16T20:48:41.765 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:41.765 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:41.765 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:41.765 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:41.765 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:41.765 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:41.765 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 7491 2024-09-16T20:48:41.766 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.766 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:41.766 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.766 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:41.766 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925865+0000", 2024-09-16T20:48:41.766 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:41.766 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.766 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.766 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:41.766 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925865+0000", 2024-09-16T20:48:41.766 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:41.766 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.766 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.767 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:41.767 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925866+0000", 2024-09-16T20:48:41.767 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.39e-07 2024-09-16T20:48:41.767 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.767 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.767 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:41.767 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925868+0000", 2024-09-16T20:48:41.767 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.508e-06 2024-09-16T20:48:41.767 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.767 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.767 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:41.767 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925874+0000", 2024-09-16T20:48:41.767 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7919999999999998e-06 2024-09-16T20:48:41.768 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.768 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.768 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:41.768 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925879+0000", 2024-09-16T20:48:41.768 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8579999999999999e-06 2024-09-16T20:48:41.768 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.768 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.768 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:41.768 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.970875+0000", 2024-09-16T20:48:41.768 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044996412 2024-09-16T20:48:41.768 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.768 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.769 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:41.769 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.970970+0000", 2024-09-16T20:48:41.769 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4732e-05 2024-09-16T20:48:41.769 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.769 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.769 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-16T20:48:41.769 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.971275+0000", 2024-09-16T20:48:41.769 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00030464199999999999 2024-09-16T20:48:41.769 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.769 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.769 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:41.769 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.974598+0000", 2024-09-16T20:48:41.769 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0033238159999999998 2024-09-16T20:48:41.769 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.770 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.770 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:48:41.770 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.974666+0000", 2024-09-16T20:48:41.770 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7617e-05 2024-09-16T20:48:41.770 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.770 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.770 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:48:41.770 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.974718+0000", 2024-09-16T20:48:41.770 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2259000000000002e-05 2024-09-16T20:48:41.770 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.770 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.770 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:41.772 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.974762+0000", 2024-09-16T20:48:41.772 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4218000000000003e-05 2024-09-16T20:48:41.772 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:41.772 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:41.772 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:41.772 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.772 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.772 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4750 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1488896~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:41.772 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689052+0000", 2024-09-16T20:48:41.773 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.036936694000001, 2024-09-16T20:48:41.773 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048208359999999999, 2024-09-16T20:48:41.773 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:41.773 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:41.773 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:41.773 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:41.773 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:41.773 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:41.773 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4750 2024-09-16T20:48:41.773 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.773 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:41.773 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.773 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:41.774 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689052+0000", 2024-09-16T20:48:41.774 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:41.774 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.774 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.774 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:41.774 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689052+0000", 2024-09-16T20:48:41.774 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:41.774 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.774 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.774 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:41.774 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689052+0000", 2024-09-16T20:48:41.774 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.65e-07 2024-09-16T20:48:41.774 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.775 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.775 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:41.775 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689058+0000", 2024-09-16T20:48:41.775 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.835e-06 2024-09-16T20:48:41.775 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.775 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.775 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:41.775 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689064+0000", 2024-09-16T20:48:41.775 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2729999999999999e-06 2024-09-16T20:48:41.775 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.775 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.775 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:41.775 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689069+0000", 2024-09-16T20:48:41.775 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1220000000000003e-06 2024-09-16T20:48:41.776 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.776 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.776 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:41.776 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737161+0000", 2024-09-16T20:48:41.776 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048091776000000003 2024-09-16T20:48:41.776 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.776 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.776 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:41.776 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737210+0000", 2024-09-16T20:48:41.776 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8575000000000002e-05 2024-09-16T20:48:41.776 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.776 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.776 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:41.777 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737260+0000", 2024-09-16T20:48:41.777 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0414000000000002e-05 2024-09-16T20:48:41.777 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:41.777 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:41.777 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:41.777 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.777 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.777 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4752 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1497088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:41.777 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689076+0000", 2024-09-16T20:48:41.777 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.036912552, 2024-09-16T20:48:41.777 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048322324999999999, 2024-09-16T20:48:41.777 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:41.777 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:41.777 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:41.778 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:41.778 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:41.778 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:41.778 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4752 2024-09-16T20:48:41.778 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.778 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:41.778 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.778 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:41.778 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689076+0000", 2024-09-16T20:48:41.778 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:41.778 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.778 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.778 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:41.779 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689076+0000", 2024-09-16T20:48:41.779 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:41.779 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.779 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.779 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:41.779 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689076+0000", 2024-09-16T20:48:41.779 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8000000000000001e-07 2024-09-16T20:48:41.779 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.779 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.779 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:41.779 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689078+0000", 2024-09-16T20:48:41.779 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8369999999999999e-06 2024-09-16T20:48:41.779 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.779 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.780 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:41.780 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689084+0000", 2024-09-16T20:48:41.780 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1319999999999999e-06 2024-09-16T20:48:41.780 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.780 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.780 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:41.780 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689090+0000", 2024-09-16T20:48:41.780 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0430000000000004e-06 2024-09-16T20:48:41.780 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.780 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.780 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:41.780 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737286+0000", 2024-09-16T20:48:41.781 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048195670000000003 2024-09-16T20:48:41.781 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.781 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.781 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:41.781 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737337+0000", 2024-09-16T20:48:41.781 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1511000000000002e-05 2024-09-16T20:48:41.781 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.781 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.781 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:41.781 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737398+0000", 2024-09-16T20:48:41.781 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0751999999999997e-05 2024-09-16T20:48:41.781 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:41.781 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:41.782 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:41.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.782 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.782 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4754 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1505280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:41.782 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689097+0000", 2024-09-16T20:48:41.782 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.036891035, 2024-09-16T20:48:41.782 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048429947000000001, 2024-09-16T20:48:41.782 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:41.782 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:41.782 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:41.782 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:41.782 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:41.782 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:41.783 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4754 2024-09-16T20:48:41.783 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.783 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:41.783 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.783 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:41.783 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689097+0000", 2024-09-16T20:48:41.783 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:41.783 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.783 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.783 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:41.783 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689097+0000", 2024-09-16T20:48:41.783 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:41.783 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.784 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.784 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:41.784 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689098+0000", 2024-09-16T20:48:41.784 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.15e-07 2024-09-16T20:48:41.784 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.784 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.784 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:41.784 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689100+0000", 2024-09-16T20:48:41.784 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8190000000000001e-06 2024-09-16T20:48:41.784 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.784 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.784 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:41.784 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689106+0000", 2024-09-16T20:48:41.784 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2489999999999998e-06 2024-09-16T20:48:41.785 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.785 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.785 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:41.785 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689111+0000", 2024-09-16T20:48:41.785 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3179999999999997e-06 2024-09-16T20:48:41.785 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.785 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.785 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:41.785 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737428+0000", 2024-09-16T20:48:41.785 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048316572000000002 2024-09-16T20:48:41.785 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.785 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.785 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:41.786 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737475+0000", 2024-09-16T20:48:41.786 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7392000000000003e-05 2024-09-16T20:48:41.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.786 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:41.786 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737527+0000", 2024-09-16T20:48:41.786 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2182000000000003e-05 2024-09-16T20:48:41.786 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:41.786 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:41.786 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:41.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.786 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4756 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1513472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:41.786 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689118+0000", 2024-09-16T20:48:41.787 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.036870814, 2024-09-16T20:48:41.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048535146000000001, 2024-09-16T20:48:41.787 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:41.787 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:41.787 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:41.787 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:41.787 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:41.787 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:41.787 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4756 2024-09-16T20:48:41.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.787 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:41.787 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.787 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:41.787 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689118+0000", 2024-09-16T20:48:41.788 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:41.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.788 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.788 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:41.788 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689118+0000", 2024-09-16T20:48:41.788 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:41.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.788 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.788 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:41.788 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689118+0000", 2024-09-16T20:48:41.788 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7099999999999997e-07 2024-09-16T20:48:41.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.788 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.789 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:41.789 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689120+0000", 2024-09-16T20:48:41.789 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.787e-06 2024-09-16T20:48:41.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.789 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.789 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:41.789 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689126+0000", 2024-09-16T20:48:41.789 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2720000000000003e-06 2024-09-16T20:48:41.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.789 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.789 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:41.789 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689134+0000", 2024-09-16T20:48:41.789 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2190000000000004e-06 2024-09-16T20:48:41.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.790 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.790 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:41.790 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737553+0000", 2024-09-16T20:48:41.790 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048418993 2024-09-16T20:48:41.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.790 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.790 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:41.790 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737598+0000", 2024-09-16T20:48:41.790 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4743999999999997e-05 2024-09-16T20:48:41.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.790 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.790 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:41.791 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737653+0000", 2024-09-16T20:48:41.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4759999999999997e-05 2024-09-16T20:48:41.791 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:41.791 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:41.791 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:41.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.791 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.791 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4758 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1521664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:41.791 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689141+0000", 2024-09-16T20:48:41.791 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.036847834, 2024-09-16T20:48:41.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048640741000000001, 2024-09-16T20:48:41.791 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:41.792 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:41.792 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:41.792 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:41.792 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:41.792 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:41.792 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4758 2024-09-16T20:48:41.792 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.792 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:41.793 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.793 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:41.793 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689141+0000", 2024-09-16T20:48:41.793 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:41.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.793 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.793 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:41.793 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689141+0000", 2024-09-16T20:48:41.793 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:41.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.793 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.793 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:41.793 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689141+0000", 2024-09-16T20:48:41.794 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8799999999999998e-07 2024-09-16T20:48:41.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.794 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:41.794 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689143+0000", 2024-09-16T20:48:41.794 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.945e-06 2024-09-16T20:48:41.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.794 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:41.794 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689149+0000", 2024-09-16T20:48:41.794 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9800000000000003e-06 2024-09-16T20:48:41.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.795 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:41.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689155+0000", 2024-09-16T20:48:41.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7509999999999997e-06 2024-09-16T20:48:41.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.795 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.795 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:41.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737681+0000", 2024-09-16T20:48:41.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048526825000000003 2024-09-16T20:48:41.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.795 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.795 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:41.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737729+0000", 2024-09-16T20:48:41.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7219999999999999e-05 2024-09-16T20:48:41.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.796 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.796 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:41.796 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737781+0000", 2024-09-16T20:48:41.796 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2632e-05 2024-09-16T20:48:41.796 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:41.796 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:41.796 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:41.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.796 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.796 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4760 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1529856~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:41.796 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689161+0000", 2024-09-16T20:48:41.796 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.036827487, 2024-09-16T20:48:41.797 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048743981999999998, 2024-09-16T20:48:41.797 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:41.797 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:41.797 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:41.797 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:41.797 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:41.797 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:41.797 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4760 2024-09-16T20:48:41.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.797 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:41.797 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.797 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:41.797 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689161+0000", 2024-09-16T20:48:41.798 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:41.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.798 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.798 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:41.798 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689161+0000", 2024-09-16T20:48:41.798 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:41.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.798 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.798 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:41.798 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689161+0000", 2024-09-16T20:48:41.798 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5599999999999995e-07 2024-09-16T20:48:41.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.798 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.798 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:41.799 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689163+0000", 2024-09-16T20:48:41.799 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.818e-06 2024-09-16T20:48:41.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.799 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.799 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:41.799 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689169+0000", 2024-09-16T20:48:41.799 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0809999999999999e-06 2024-09-16T20:48:41.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.799 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.799 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:41.799 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689174+0000", 2024-09-16T20:48:41.799 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.938e-06 2024-09-16T20:48:41.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.799 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.800 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:41.800 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737807+0000", 2024-09-16T20:48:41.800 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048633124 2024-09-16T20:48:41.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.800 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:41.800 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737851+0000", 2024-09-16T20:48:41.800 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3764000000000003e-05 2024-09-16T20:48:41.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.800 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:41.800 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737905+0000", 2024-09-16T20:48:41.800 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3701e-05 2024-09-16T20:48:41.801 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:41.801 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:41.801 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:41.801 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.801 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.801 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4762 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1538048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:41.801 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689182+0000", 2024-09-16T20:48:41.801 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.036806264999999, 2024-09-16T20:48:41.801 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048850275999999998, 2024-09-16T20:48:41.801 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:41.801 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:41.801 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:41.802 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:41.802 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:41.802 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:41.802 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4762 2024-09-16T20:48:41.802 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.802 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:41.802 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.802 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:41.802 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689182+0000", 2024-09-16T20:48:41.802 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:41.802 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.802 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.802 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:41.802 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689182+0000", 2024-09-16T20:48:41.803 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:41.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.803 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.803 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:41.803 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689182+0000", 2024-09-16T20:48:41.803 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-16T20:48:41.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.803 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.803 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:41.803 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689184+0000", 2024-09-16T20:48:41.803 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8190000000000001e-06 2024-09-16T20:48:41.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.803 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.804 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:41.804 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689191+0000", 2024-09-16T20:48:41.804 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2530000000000001e-06 2024-09-16T20:48:41.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.804 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:41.804 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689195+0000", 2024-09-16T20:48:41.804 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9559999999999996e-06 2024-09-16T20:48:41.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.804 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:41.804 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737933+0000", 2024-09-16T20:48:41.804 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048737919999999997 2024-09-16T20:48:41.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.805 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.805 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:41.805 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737982+0000", 2024-09-16T20:48:41.805 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8089e-05 2024-09-16T20:48:41.805 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.805 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.805 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:41.805 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738032+0000", 2024-09-16T20:48:41.805 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0868999999999997e-05 2024-09-16T20:48:41.805 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:41.805 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:41.805 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:41.805 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.805 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.806 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4764 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1546240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:41.806 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689201+0000", 2024-09-16T20:48:41.806 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.036786908, 2024-09-16T20:48:41.806 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048957550000000002, 2024-09-16T20:48:41.806 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:41.806 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:41.806 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:41.806 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:41.806 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:41.806 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:41.806 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4764 2024-09-16T20:48:41.806 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.806 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:41.806 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.807 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:41.807 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689201+0000", 2024-09-16T20:48:41.807 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:41.807 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.807 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.807 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:41.807 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689201+0000", 2024-09-16T20:48:41.807 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:41.807 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.807 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.807 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:41.807 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689202+0000", 2024-09-16T20:48:41.807 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4200000000000002e-07 2024-09-16T20:48:41.808 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.808 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.808 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:41.808 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689204+0000", 2024-09-16T20:48:41.808 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9750000000000001e-06 2024-09-16T20:48:41.808 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.808 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.808 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:41.808 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689210+0000", 2024-09-16T20:48:41.808 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1e-06 2024-09-16T20:48:41.808 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.808 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.808 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:41.808 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689215+0000", 2024-09-16T20:48:41.809 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0180000000000001e-06 2024-09-16T20:48:41.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.809 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.809 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:41.809 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738058+0000", 2024-09-16T20:48:41.809 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048843573000000001 2024-09-16T20:48:41.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.809 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.809 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:41.809 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738102+0000", 2024-09-16T20:48:41.809 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3256000000000001e-05 2024-09-16T20:48:41.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.809 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.810 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:41.810 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738159+0000", 2024-09-16T20:48:41.810 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7286e-05 2024-09-16T20:48:41.810 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:41.810 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:41.810 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:41.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.810 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.810 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4766 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1554432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:41.810 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689221+0000", 2024-09-16T20:48:41.810 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.036766942, 2024-09-16T20:48:41.810 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049064996, 2024-09-16T20:48:41.810 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:41.810 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:41.811 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:41.811 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:41.811 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:41.811 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:41.811 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4766 2024-09-16T20:48:41.811 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.811 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:41.811 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.811 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:41.811 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689221+0000", 2024-09-16T20:48:41.812 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:41.812 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.812 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.812 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:41.812 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689221+0000", 2024-09-16T20:48:41.812 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:41.812 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.812 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.812 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:41.812 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689222+0000", 2024-09-16T20:48:41.812 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.34e-07 2024-09-16T20:48:41.812 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.813 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.813 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:41.813 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689224+0000", 2024-09-16T20:48:41.813 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.846e-06 2024-09-16T20:48:41.813 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.813 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.813 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:41.813 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689247+0000", 2024-09-16T20:48:41.813 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3448e-05 2024-09-16T20:48:41.813 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.813 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.813 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:41.813 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689254+0000", 2024-09-16T20:48:41.814 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6969999999999996e-06 2024-09-16T20:48:41.814 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.814 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.814 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:41.814 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738188+0000", 2024-09-16T20:48:41.814 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048934520000000002 2024-09-16T20:48:41.814 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.814 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.814 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:41.815 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738234+0000", 2024-09-16T20:48:41.815 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5695999999999998e-05 2024-09-16T20:48:41.815 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.815 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.815 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:41.815 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738286+0000", 2024-09-16T20:48:41.815 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2355000000000003e-05 2024-09-16T20:48:41.815 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:41.815 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:41.815 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:41.815 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.815 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.815 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4768 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1562624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:41.815 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689262+0000", 2024-09-16T20:48:41.816 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.036725988000001, 2024-09-16T20:48:41.816 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049157157999999999, 2024-09-16T20:48:41.816 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:41.816 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:41.816 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:41.816 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:41.816 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:41.816 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:41.816 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4768 2024-09-16T20:48:41.816 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.816 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:41.816 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.816 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:41.816 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689262+0000", 2024-09-16T20:48:41.817 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:41.817 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.817 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.817 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:41.817 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689262+0000", 2024-09-16T20:48:41.817 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:41.817 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.817 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.817 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:41.817 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689263+0000", 2024-09-16T20:48:41.817 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6199999999999998e-07 2024-09-16T20:48:41.817 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.817 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.817 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:41.818 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689270+0000", 2024-09-16T20:48:41.818 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.798e-06 2024-09-16T20:48:41.818 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.818 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.818 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:41.818 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689277+0000", 2024-09-16T20:48:41.818 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0890000000000002e-06 2024-09-16T20:48:41.818 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.818 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.818 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:41.818 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689282+0000", 2024-09-16T20:48:41.818 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0470000000000001e-06 2024-09-16T20:48:41.818 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.818 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.819 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:41.819 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738319+0000", 2024-09-16T20:48:41.819 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049037139 2024-09-16T20:48:41.819 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.819 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.819 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:41.819 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738365+0000", 2024-09-16T20:48:41.819 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6356999999999997e-05 2024-09-16T20:48:41.819 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.819 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.819 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:41.819 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738420+0000", 2024-09-16T20:48:41.819 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4265999999999998e-05 2024-09-16T20:48:41.819 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:41.819 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:41.820 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:41.820 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.820 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.820 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4770 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1570816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:41.820 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689289+0000", 2024-09-16T20:48:41.820 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.036699477999999, 2024-09-16T20:48:41.820 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049253705000000002, 2024-09-16T20:48:41.820 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:41.820 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:41.820 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:41.820 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:41.820 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:41.820 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:41.820 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4770 2024-09-16T20:48:41.821 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.821 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:41.821 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.821 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:41.821 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689289+0000", 2024-09-16T20:48:41.821 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:41.821 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.821 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.821 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:41.821 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689289+0000", 2024-09-16T20:48:41.821 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:41.821 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.821 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.821 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:41.822 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689289+0000", 2024-09-16T20:48:41.822 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0999999999999999e-07 2024-09-16T20:48:41.822 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.822 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.822 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:41.822 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689291+0000", 2024-09-16T20:48:41.822 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0030000000000001e-06 2024-09-16T20:48:41.822 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.822 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.822 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:41.822 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689297+0000", 2024-09-16T20:48:41.822 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9479999999999996e-06 2024-09-16T20:48:41.822 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.823 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.823 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:41.823 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689302+0000", 2024-09-16T20:48:41.823 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7389999999999999e-06 2024-09-16T20:48:41.823 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.823 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.823 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:41.823 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738442+0000", 2024-09-16T20:48:41.823 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049140001000000003 2024-09-16T20:48:41.823 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.823 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.823 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:41.823 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738490+0000", 2024-09-16T20:48:41.823 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7787000000000002e-05 2024-09-16T20:48:41.823 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.824 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.824 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:41.824 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738543+0000", 2024-09-16T20:48:41.824 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2816999999999999e-05 2024-09-16T20:48:41.824 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:41.824 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:41.824 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:41.824 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.824 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.824 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:3088 3.1e 3:79e5ec01:::rbd_header.110e891a9b2b:head [watch watch cookie 140213437171488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:48:41.824 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:48:41.824 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 17.562143461000002, 2024-09-16T20:48:41.824 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.11895186200000001, 2024-09-16T20:48:41.824 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:41.825 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:41.825 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:41.825 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:41.825 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:48:41.825 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:48:41.825 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3088 2024-09-16T20:48:41.825 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.825 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:41.825 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.825 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:41.825 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:48:41.825 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:41.825 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.825 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.826 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:41.826 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:48:41.826 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:41.826 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.826 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.826 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:41.826 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163846+0000", 2024-09-16T20:48:41.826 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6799999999999999e-07 2024-09-16T20:48:41.826 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.826 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.826 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:41.826 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163850+0000", 2024-09-16T20:48:41.826 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.929e-06 2024-09-16T20:48:41.826 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.827 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.827 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:41.827 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163862+0000", 2024-09-16T20:48:41.827 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1972e-05 2024-09-16T20:48:41.827 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.827 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.827 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:41.827 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163867+0000", 2024-09-16T20:48:41.827 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.451e-06 2024-09-16T20:48:41.827 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.827 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.827 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:41.827 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163912+0000", 2024-09-16T20:48:41.827 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4456000000000003e-05 2024-09-16T20:48:41.827 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.828 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.828 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:41.828 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163962+0000", 2024-09-16T20:48:41.828 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0578000000000003e-05 2024-09-16T20:48:41.828 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.828 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.828 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-16T20:48:41.828 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164058+0000", 2024-09-16T20:48:41.828 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5934999999999995e-05 2024-09-16T20:48:41.828 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.828 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.828 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:48:41.828 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164570+0000", 2024-09-16T20:48:41.828 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00051241100000000005 2024-09-16T20:48:41.829 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.829 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.829 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:41.829 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282629+0000", 2024-09-16T20:48:41.829 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.118058986 2024-09-16T20:48:41.829 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.829 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.829 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:48:41.829 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282725+0000", 2024-09-16T20:48:41.829 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5063999999999996e-05 2024-09-16T20:48:41.829 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:41.829 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:41.829 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:41.829 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282797+0000", 2024-09-16T20:48:41.830 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2212000000000006e-05 2024-09-16T20:48:41.830 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:41.830 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:41.830 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:41.830 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:41.830 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:41.830 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:41.831 INFO:teuthology.orchestra.run.smithi139.stdout:truncating image image_client.0-clone12 from 0xc9c0e00 (overlap 0x12fe600) to 0x798e00 2024-09-16T20:48:41.832 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:48:41.832 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:41.830+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:41.932 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:41.931+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:41.981 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:41.982 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:48:41.982 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:48:41.982 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:41.990 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:48:42.033 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:42.032+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:42.099 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:42.099 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:48:42.099 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:48:42.100 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:48:42.100 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:42.108 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:48:42.123 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #11, image image_client.0-clone11 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent12 2024-09-16T20:48:42.134 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:42.133+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:42.206 INFO:teuthology.orchestra.run.smithi139.stdout:1849 read 0x3c0000 thru 0x3c8539 (0x853a bytes) 2024-09-16T20:48:42.207 INFO:teuthology.orchestra.run.smithi139.stdout:1850 punch from 0x830aa00 to 0x8317c00, (0xd200 bytes) 2024-09-16T20:48:42.221 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:42.221 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:48:42.221 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:48:42.221 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:48:42.221 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.221 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:4904 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 7309312~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:42.222 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767724+0000", 2024-09-16T20:48:42.222 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.451760338, 2024-09-16T20:48:42.222 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050270214000000001, 2024-09-16T20:48:42.222 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:42.222 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:42.222 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:42.222 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:42.222 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:42.222 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:42.222 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4904 2024-09-16T20:48:42.222 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.222 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:42.222 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.223 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:42.223 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767724+0000", 2024-09-16T20:48:42.223 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:42.223 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.223 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.223 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:42.223 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767724+0000", 2024-09-16T20:48:42.223 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:42.223 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.223 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.223 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:42.223 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767724+0000", 2024-09-16T20:48:42.223 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1399999999999998e-07 2024-09-16T20:48:42.223 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.224 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.224 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:42.224 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767725+0000", 2024-09-16T20:48:42.224 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0380000000000001e-06 2024-09-16T20:48:42.224 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.224 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.224 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:42.224 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767729+0000", 2024-09-16T20:48:42.224 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2490000000000001e-06 2024-09-16T20:48:42.224 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.224 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.224 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:42.224 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767731+0000", 2024-09-16T20:48:42.225 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5940000000000001e-06 2024-09-16T20:48:42.225 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.225 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.225 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:42.225 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.817787+0000", 2024-09-16T20:48:42.225 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050055792000000002 2024-09-16T20:48:42.225 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.225 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.225 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:42.225 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.817883+0000", 2024-09-16T20:48:42.225 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5606999999999995e-05 2024-09-16T20:48:42.225 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.225 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.226 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:42.226 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.817994+0000", 2024-09-16T20:48:42.226 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011162000000000001 2024-09-16T20:48:42.226 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:42.226 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:42.226 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:42.226 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.226 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.226 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:4906 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 7317504~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:42.226 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767734+0000", 2024-09-16T20:48:42.227 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.451750075, 2024-09-16T20:48:42.227 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050406608999999998, 2024-09-16T20:48:42.227 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:42.227 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:42.227 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:42.227 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:42.227 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:42.227 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:42.227 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4906 2024-09-16T20:48:42.227 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.227 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:42.227 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.227 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:42.228 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767734+0000", 2024-09-16T20:48:42.228 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:42.228 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.228 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.228 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:42.228 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767734+0000", 2024-09-16T20:48:42.228 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:42.228 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.228 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.228 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:42.228 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767735+0000", 2024-09-16T20:48:42.228 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.05e-07 2024-09-16T20:48:42.229 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.229 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.229 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:42.229 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767736+0000", 2024-09-16T20:48:42.229 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0240000000000001e-06 2024-09-16T20:48:42.229 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.229 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.229 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:42.229 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767739+0000", 2024-09-16T20:48:42.229 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.067e-06 2024-09-16T20:48:42.229 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.229 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.229 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:42.230 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767742+0000", 2024-09-16T20:48:42.230 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.23e-06 2024-09-16T20:48:42.230 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.230 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.230 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:42.230 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818027+0000", 2024-09-16T20:48:42.230 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050285102999999998 2024-09-16T20:48:42.230 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.230 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.230 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:42.230 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818072+0000", 2024-09-16T20:48:42.230 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5129000000000001e-05 2024-09-16T20:48:42.230 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.231 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.231 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:42.231 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818141+0000", 2024-09-16T20:48:42.231 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8850999999999995e-05 2024-09-16T20:48:42.231 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:42.231 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:42.231 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:42.231 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.231 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.231 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:4908 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 7325696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:42.231 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767745+0000", 2024-09-16T20:48:42.231 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.451739679999999, 2024-09-16T20:48:42.231 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050654230000000001, 2024-09-16T20:48:42.232 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:42.232 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:42.232 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:42.232 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:42.232 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:42.232 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:42.232 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4908 2024-09-16T20:48:42.232 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.232 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:42.232 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.232 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:42.232 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767745+0000", 2024-09-16T20:48:42.232 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:42.233 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.233 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.233 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:42.233 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767745+0000", 2024-09-16T20:48:42.233 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:42.233 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.233 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.233 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:42.233 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767745+0000", 2024-09-16T20:48:42.233 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1e-07 2024-09-16T20:48:42.233 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.233 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.233 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:42.233 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767749+0000", 2024-09-16T20:48:42.234 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.532e-06 2024-09-16T20:48:42.234 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.234 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.234 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:42.234 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767753+0000", 2024-09-16T20:48:42.234 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4939999999999999e-06 2024-09-16T20:48:42.234 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.234 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.234 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:42.234 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767756+0000", 2024-09-16T20:48:42.234 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.61e-06 2024-09-16T20:48:42.234 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.234 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.235 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:42.235 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818184+0000", 2024-09-16T20:48:42.235 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050428615000000003 2024-09-16T20:48:42.235 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.235 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.235 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:42.235 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818276+0000", 2024-09-16T20:48:42.235 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2077999999999995e-05 2024-09-16T20:48:42.235 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.235 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.235 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:42.235 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818399+0000", 2024-09-16T20:48:42.235 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012269099999999999 2024-09-16T20:48:42.236 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:42.236 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:42.236 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:42.236 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.236 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.236 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:4910 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 7333888~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:42.236 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767759+0000", 2024-09-16T20:48:42.236 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.451725790000001, 2024-09-16T20:48:42.236 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050815076000000001, 2024-09-16T20:48:42.236 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:42.236 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:42.236 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:42.236 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:42.237 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:42.237 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:42.237 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4910 2024-09-16T20:48:42.237 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.237 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:42.237 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.237 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:42.237 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767759+0000", 2024-09-16T20:48:42.237 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:42.237 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.237 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.237 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:42.238 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767759+0000", 2024-09-16T20:48:42.238 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:42.238 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.238 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.238 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:42.238 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767759+0000", 2024-09-16T20:48:42.238 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2600000000000001e-07 2024-09-16T20:48:42.238 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.238 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.238 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:42.238 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767760+0000", 2024-09-16T20:48:42.238 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0640000000000001e-06 2024-09-16T20:48:42.238 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.238 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.239 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:42.239 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767763+0000", 2024-09-16T20:48:42.239 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.994e-06 2024-09-16T20:48:42.239 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.239 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.239 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:42.239 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767765+0000", 2024-09-16T20:48:42.239 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5220000000000002e-06 2024-09-16T20:48:42.239 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.239 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.239 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:42.239 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818438+0000", 2024-09-16T20:48:42.240 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050672602999999997 2024-09-16T20:48:42.240 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.240 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.240 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:42.240 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818481+0000", 2024-09-16T20:48:42.240 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2967999999999999e-05 2024-09-16T20:48:42.240 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.240 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.240 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:42.240 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818574+0000", 2024-09-16T20:48:42.240 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2699000000000001e-05 2024-09-16T20:48:42.240 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:42.240 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:42.241 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:42.241 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.241 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.241 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5012 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2101248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:42.241 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767768+0000", 2024-09-16T20:48:42.241 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.451716140999999, 2024-09-16T20:48:42.241 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051084682999999999, 2024-09-16T20:48:42.241 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:42.241 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:42.241 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:42.241 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:42.242 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:42.242 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:42.242 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5012 2024-09-16T20:48:42.242 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.242 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:42.242 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.242 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:42.242 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767768+0000", 2024-09-16T20:48:42.242 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:42.243 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.243 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.243 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:42.243 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767768+0000", 2024-09-16T20:48:42.243 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:42.243 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.243 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.243 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:42.243 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767769+0000", 2024-09-16T20:48:42.243 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1500000000000001e-07 2024-09-16T20:48:42.243 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.243 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.244 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:42.244 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767770+0000", 2024-09-16T20:48:42.244 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.015e-06 2024-09-16T20:48:42.244 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.244 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.244 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:42.244 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767773+0000", 2024-09-16T20:48:42.244 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1070000000000001e-06 2024-09-16T20:48:42.244 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.244 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.244 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:42.244 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767775+0000", 2024-09-16T20:48:42.245 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.508e-06 2024-09-16T20:48:42.245 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.245 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.245 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:42.245 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818630+0000", 2024-09-16T20:48:42.245 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050855282000000002 2024-09-16T20:48:42.245 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.245 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.245 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:42.245 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818732+0000", 2024-09-16T20:48:42.245 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101531 2024-09-16T20:48:42.245 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.245 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.246 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:42.246 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818853+0000", 2024-09-16T20:48:42.246 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000121025 2024-09-16T20:48:42.246 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:42.246 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:42.246 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:42.246 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.246 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.246 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5014 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2109440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:42.246 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767778+0000", 2024-09-16T20:48:42.246 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.451706205000001, 2024-09-16T20:48:42.247 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051260776000000001, 2024-09-16T20:48:42.247 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:42.247 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:42.247 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:42.247 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:42.247 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:42.247 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:42.247 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5014 2024-09-16T20:48:42.247 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.247 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:42.247 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.247 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:42.247 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767778+0000", 2024-09-16T20:48:42.248 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:42.248 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.248 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.248 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:42.248 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767778+0000", 2024-09-16T20:48:42.248 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:42.248 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.248 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.248 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:42.248 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767824+0000", 2024-09-16T20:48:42.248 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5890999999999998e-05 2024-09-16T20:48:42.248 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.249 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.249 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:42.249 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767827+0000", 2024-09-16T20:48:42.249 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.644e-06 2024-09-16T20:48:42.249 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.250 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.250 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:42.250 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767832+0000", 2024-09-16T20:48:42.250 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2689999999999999e-06 2024-09-16T20:48:42.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.250 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.250 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:42.250 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767835+0000", 2024-09-16T20:48:42.250 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2100000000000002e-06 2024-09-16T20:48:42.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.250 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.250 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:42.250 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818887+0000", 2024-09-16T20:48:42.250 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051051276999999999 2024-09-16T20:48:42.251 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.251 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.251 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:42.251 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818953+0000", 2024-09-16T20:48:42.251 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6105999999999994e-05 2024-09-16T20:48:42.251 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.251 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.251 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:42.251 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819039+0000", 2024-09-16T20:48:42.252 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6378999999999994e-05 2024-09-16T20:48:42.252 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:42.252 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:42.252 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:42.252 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.252 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.252 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5016 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2117632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:42.252 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767841+0000", 2024-09-16T20:48:42.252 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.451643526000002, 2024-09-16T20:48:42.252 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051433288000000001, 2024-09-16T20:48:42.252 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:42.252 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:42.252 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:42.253 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:42.253 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:42.253 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:42.253 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5016 2024-09-16T20:48:42.253 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.253 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:42.253 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.253 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:42.253 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767841+0000", 2024-09-16T20:48:42.254 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:42.254 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.254 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.254 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:42.254 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767841+0000", 2024-09-16T20:48:42.254 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:42.254 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.254 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.254 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:42.254 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768156+0000", 2024-09-16T20:48:42.254 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00031505699999999999 2024-09-16T20:48:42.254 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.254 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.255 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:42.255 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768159+0000", 2024-09-16T20:48:42.255 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1659999999999998e-06 2024-09-16T20:48:42.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.255 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.255 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:42.255 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768166+0000", 2024-09-16T20:48:42.255 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0550000000000002e-06 2024-09-16T20:48:42.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.256 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.256 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:42.256 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768171+0000", 2024-09-16T20:48:42.256 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3170000000000003e-06 2024-09-16T20:48:42.256 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.256 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.256 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:42.256 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819084+0000", 2024-09-16T20:48:42.256 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050913355 2024-09-16T20:48:42.256 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.256 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.256 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:42.256 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819170+0000", 2024-09-16T20:48:42.257 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6328000000000004e-05 2024-09-16T20:48:42.257 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.257 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.257 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:42.257 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819274+0000", 2024-09-16T20:48:42.257 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010401 2024-09-16T20:48:42.257 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:42.257 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:42.257 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:42.257 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.257 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.257 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5018 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2125824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:42.257 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768178+0000", 2024-09-16T20:48:42.258 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.451305992000002, 2024-09-16T20:48:42.258 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051227013000000002, 2024-09-16T20:48:42.258 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:42.258 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:42.258 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:42.258 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:42.258 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:42.258 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:42.258 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5018 2024-09-16T20:48:42.258 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.258 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:42.258 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.258 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:42.259 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768178+0000", 2024-09-16T20:48:42.259 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:42.259 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.259 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.259 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:42.259 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768178+0000", 2024-09-16T20:48:42.259 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:42.259 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.259 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.259 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:42.259 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768179+0000", 2024-09-16T20:48:42.259 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9200000000000002e-07 2024-09-16T20:48:42.259 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.260 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.260 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:42.260 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768180+0000", 2024-09-16T20:48:42.260 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3790000000000001e-06 2024-09-16T20:48:42.260 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.260 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.260 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:42.260 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768184+0000", 2024-09-16T20:48:42.260 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5470000000000001e-06 2024-09-16T20:48:42.260 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.260 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.260 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:42.260 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768187+0000", 2024-09-16T20:48:42.261 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8119999999999999e-06 2024-09-16T20:48:42.261 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.261 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.261 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:42.261 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819305+0000", 2024-09-16T20:48:42.261 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051118683999999998 2024-09-16T20:48:42.261 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.261 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.261 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:42.261 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819351+0000", 2024-09-16T20:48:42.261 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5757999999999998e-05 2024-09-16T20:48:42.261 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.261 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.262 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:42.262 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819405+0000", 2024-09-16T20:48:42.262 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4441000000000003e-05 2024-09-16T20:48:42.262 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:42.262 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:42.262 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:42.262 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.262 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.262 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5020 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2134016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:42.262 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768190+0000", 2024-09-16T20:48:42.262 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.451294313999998, 2024-09-16T20:48:42.262 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051449661000000001, 2024-09-16T20:48:42.262 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:42.262 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:42.263 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:42.263 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:42.263 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:42.263 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:42.263 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5020 2024-09-16T20:48:42.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.263 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:42.263 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.263 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:42.263 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768190+0000", 2024-09-16T20:48:42.263 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:42.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.263 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.263 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:42.264 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768190+0000", 2024-09-16T20:48:42.264 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:42.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.264 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.264 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:42.264 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768282+0000", 2024-09-16T20:48:42.264 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2263999999999996e-05 2024-09-16T20:48:42.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.264 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.264 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:42.264 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768286+0000", 2024-09-16T20:48:42.264 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.529e-06 2024-09-16T20:48:42.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.265 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.265 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:42.265 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768294+0000", 2024-09-16T20:48:42.265 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6990000000000003e-06 2024-09-16T20:48:42.265 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.265 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.265 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:42.265 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768306+0000", 2024-09-16T20:48:42.265 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2112e-05 2024-09-16T20:48:42.265 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.265 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.265 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:42.265 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819449+0000", 2024-09-16T20:48:42.266 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051143521999999997 2024-09-16T20:48:42.266 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.266 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.266 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:42.266 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819539+0000", 2024-09-16T20:48:42.266 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0022e-05 2024-09-16T20:48:42.266 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.266 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.266 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:42.266 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819640+0000", 2024-09-16T20:48:42.266 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000100513 2024-09-16T20:48:42.266 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:42.266 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:42.266 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:42.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.267 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.267 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5022 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2142208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:42.267 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768326+0000", 2024-09-16T20:48:42.267 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.451158925000001, 2024-09-16T20:48:42.267 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051436952000000001, 2024-09-16T20:48:42.267 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:42.267 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:42.267 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:42.267 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:42.267 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:42.267 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:42.267 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5022 2024-09-16T20:48:42.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.268 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:42.268 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.268 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:42.268 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768326+0000", 2024-09-16T20:48:42.268 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:42.268 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.268 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.268 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:42.268 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768326+0000", 2024-09-16T20:48:42.268 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:42.268 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.268 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.269 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:42.269 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768477+0000", 2024-09-16T20:48:42.269 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015109300000000001 2024-09-16T20:48:42.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.269 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.269 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:42.269 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768480+0000", 2024-09-16T20:48:42.269 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8580000000000002e-06 2024-09-16T20:48:42.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.269 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.269 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:42.269 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768489+0000", 2024-09-16T20:48:42.270 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4600000000000003e-06 2024-09-16T20:48:42.270 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.270 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.270 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:42.270 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768493+0000", 2024-09-16T20:48:42.270 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2270000000000004e-06 2024-09-16T20:48:42.270 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.270 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.270 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:42.270 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819672+0000", 2024-09-16T20:48:42.270 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051178605000000002 2024-09-16T20:48:42.270 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.270 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.271 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:42.271 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819711+0000", 2024-09-16T20:48:42.271 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9357000000000003e-05 2024-09-16T20:48:42.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.271 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.271 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:42.271 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819762+0000", 2024-09-16T20:48:42.271 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1351999999999999e-05 2024-09-16T20:48:42.271 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:42.271 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:42.271 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:42.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.271 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.271 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5024 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2150400~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:42.272 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768501+0000", 2024-09-16T20:48:42.272 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.450983446999999, 2024-09-16T20:48:42.272 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051402678, 2024-09-16T20:48:42.272 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:42.272 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:42.272 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:42.272 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:42.272 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:42.272 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:42.272 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5024 2024-09-16T20:48:42.272 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.273 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:42.273 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.273 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:42.273 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768501+0000", 2024-09-16T20:48:42.273 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:42.273 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.273 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.273 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:42.273 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768501+0000", 2024-09-16T20:48:42.273 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:42.273 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.273 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.273 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:42.273 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768619+0000", 2024-09-16T20:48:42.274 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118187 2024-09-16T20:48:42.274 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.274 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.274 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:42.274 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768624+0000", 2024-09-16T20:48:42.274 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3259999999999997e-06 2024-09-16T20:48:42.274 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.274 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.274 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:42.274 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768631+0000", 2024-09-16T20:48:42.274 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6490000000000008e-06 2024-09-16T20:48:42.274 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.275 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.275 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:42.275 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768637+0000", 2024-09-16T20:48:42.275 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4210000000000003e-06 2024-09-16T20:48:42.275 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.275 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.275 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:42.275 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819800+0000", 2024-09-16T20:48:42.275 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051163486000000001 2024-09-16T20:48:42.275 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.275 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.275 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:42.275 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819845+0000", 2024-09-16T20:48:42.276 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4830999999999998e-05 2024-09-16T20:48:42.276 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.276 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.276 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:42.276 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819904+0000", 2024-09-16T20:48:42.276 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8777999999999999e-05 2024-09-16T20:48:42.276 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:42.276 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:42.276 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:42.276 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.276 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.276 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5026 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2158592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:42.277 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768646+0000", 2024-09-16T20:48:42.277 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.450837999000001, 2024-09-16T20:48:42.277 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051384386999999997, 2024-09-16T20:48:42.277 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:42.277 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:42.277 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:42.277 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:42.277 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:42.277 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:42.277 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5026 2024-09-16T20:48:42.277 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.277 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:42.277 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.278 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:42.278 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768646+0000", 2024-09-16T20:48:42.278 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:42.278 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.278 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.278 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:42.278 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768646+0000", 2024-09-16T20:48:42.278 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:42.278 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.278 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.278 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:42.278 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768775+0000", 2024-09-16T20:48:42.278 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012813200000000001 2024-09-16T20:48:42.279 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.279 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.279 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:42.279 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768778+0000", 2024-09-16T20:48:42.279 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2150000000000001e-06 2024-09-16T20:48:42.279 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.279 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.279 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:42.279 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768786+0000", 2024-09-16T20:48:42.279 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3259999999999996e-06 2024-09-16T20:48:42.279 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.279 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.280 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:42.280 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768790+0000", 2024-09-16T20:48:42.280 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.216e-06 2024-09-16T20:48:42.280 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.280 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.280 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:42.280 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819939+0000", 2024-09-16T20:48:42.280 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051149018999999997 2024-09-16T20:48:42.280 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.280 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.280 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:42.280 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819979+0000", 2024-09-16T20:48:42.280 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9311999999999999e-05 2024-09-16T20:48:42.280 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.281 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.281 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:42.281 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820031+0000", 2024-09-16T20:48:42.281 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2166999999999997e-05 2024-09-16T20:48:42.281 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:42.281 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:42.281 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:42.281 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.281 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.281 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5028 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2166784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:42.281 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768798+0000", 2024-09-16T20:48:42.281 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.450686355999999, 2024-09-16T20:48:42.281 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051388912000000002, 2024-09-16T20:48:42.281 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:42.282 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:42.282 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:42.282 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:42.282 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:42.282 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:42.282 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5028 2024-09-16T20:48:42.282 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.282 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:42.282 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.282 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:42.282 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768798+0000", 2024-09-16T20:48:42.282 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:42.282 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.282 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.283 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:42.283 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768798+0000", 2024-09-16T20:48:42.283 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:42.283 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.283 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.283 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:42.283 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768989+0000", 2024-09-16T20:48:42.283 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000190951 2024-09-16T20:48:42.283 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.283 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.283 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:42.283 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768993+0000", 2024-09-16T20:48:42.283 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9770000000000002e-06 2024-09-16T20:48:42.283 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.284 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.284 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:42.284 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.769002+0000", 2024-09-16T20:48:42.284 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5469999999999997e-06 2024-09-16T20:48:42.284 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.284 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.284 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:42.284 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.769020+0000", 2024-09-16T20:48:42.284 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8216999999999999e-05 2024-09-16T20:48:42.284 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.284 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.284 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:42.284 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820070+0000", 2024-09-16T20:48:42.285 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051049998999999999 2024-09-16T20:48:42.285 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.285 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.285 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:42.285 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820119+0000", 2024-09-16T20:48:42.285 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8962000000000003e-05 2024-09-16T20:48:42.285 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.285 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.285 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:42.285 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820187+0000", 2024-09-16T20:48:42.285 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8258999999999998e-05 2024-09-16T20:48:42.286 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:42.286 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:42.286 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:42.286 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.286 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.286 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5030 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2174976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:42.286 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.769031+0000", 2024-09-16T20:48:42.286 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.450453242999998, 2024-09-16T20:48:42.286 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051289830000000002, 2024-09-16T20:48:42.286 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:42.286 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:42.286 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:42.287 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:42.287 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:42.287 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:42.287 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5030 2024-09-16T20:48:42.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.287 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:42.287 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.287 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:42.287 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.769031+0000", 2024-09-16T20:48:42.287 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:42.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.287 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.287 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:42.287 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.769031+0000", 2024-09-16T20:48:42.288 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:42.288 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.288 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.288 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:42.288 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.773332+0000", 2024-09-16T20:48:42.288 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0043004039999999999 2024-09-16T20:48:42.288 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.288 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.288 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:42.288 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.773336+0000", 2024-09-16T20:48:42.288 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7849999999999999e-06 2024-09-16T20:48:42.288 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.288 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.289 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:42.289 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.773347+0000", 2024-09-16T20:48:42.289 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0482999999999999e-05 2024-09-16T20:48:42.289 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.289 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.289 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:42.289 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.773354+0000", 2024-09-16T20:48:42.289 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1559999999999998e-06 2024-09-16T20:48:42.289 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.289 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.289 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:42.289 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820216+0000", 2024-09-16T20:48:42.289 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046861591000000001 2024-09-16T20:48:42.289 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.290 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.290 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:42.290 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820258+0000", 2024-09-16T20:48:42.290 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2653e-05 2024-09-16T20:48:42.290 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.290 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.290 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:42.290 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820321+0000", 2024-09-16T20:48:42.290 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2757999999999998e-05 2024-09-16T20:48:42.290 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:42.290 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:42.291 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:42.291 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.291 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.291 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5067 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 6918144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:42.291 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.775279+0000", 2024-09-16T20:48:42.291 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.444204976999998, 2024-09-16T20:48:42.291 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050171387999999997, 2024-09-16T20:48:42.291 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:42.291 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:42.291 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:42.291 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:42.292 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:42.292 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:42.292 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5067 2024-09-16T20:48:42.292 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.292 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:42.292 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.292 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:42.292 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775279+0000", 2024-09-16T20:48:42.292 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:42.292 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.292 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.292 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:42.293 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775279+0000", 2024-09-16T20:48:42.293 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:42.293 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.293 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.293 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:42.293 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775432+0000", 2024-09-16T20:48:42.293 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015257399999999999 2024-09-16T20:48:42.293 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.293 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.293 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:42.294 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775436+0000", 2024-09-16T20:48:42.294 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0180000000000003e-06 2024-09-16T20:48:42.294 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.294 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.294 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:42.294 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775447+0000", 2024-09-16T20:48:42.294 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.077e-05 2024-09-16T20:48:42.294 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.294 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.294 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:42.294 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775456+0000", 2024-09-16T20:48:42.294 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4089999999999992e-06 2024-09-16T20:48:42.294 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.295 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.295 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:42.295 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825150+0000", 2024-09-16T20:48:42.295 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049693711000000002 2024-09-16T20:48:42.295 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.295 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.295 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:42.295 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825337+0000", 2024-09-16T20:48:42.295 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000187214 2024-09-16T20:48:42.295 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.295 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.295 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:42.295 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825451+0000", 2024-09-16T20:48:42.295 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011369199999999999 2024-09-16T20:48:42.296 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:42.296 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:42.296 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:42.296 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.296 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.296 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5068 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2330624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:42.296 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.775469+0000", 2024-09-16T20:48:42.296 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.444015016000002, 2024-09-16T20:48:42.296 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050198334999999997, 2024-09-16T20:48:42.296 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:42.296 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:42.296 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:42.296 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:42.297 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:42.297 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:42.297 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5068 2024-09-16T20:48:42.297 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.297 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:42.297 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.297 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:42.297 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775469+0000", 2024-09-16T20:48:42.297 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:42.297 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.297 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.297 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:42.297 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775469+0000", 2024-09-16T20:48:42.298 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:42.298 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.298 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.298 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:42.298 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775720+0000", 2024-09-16T20:48:42.298 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000250661 2024-09-16T20:48:42.298 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.298 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.298 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:42.298 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775724+0000", 2024-09-16T20:48:42.298 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8700000000000002e-06 2024-09-16T20:48:42.298 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.298 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.298 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:42.299 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775733+0000", 2024-09-16T20:48:42.299 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4239999999999999e-06 2024-09-16T20:48:42.299 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.299 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.299 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:42.299 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775740+0000", 2024-09-16T20:48:42.299 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9519999999999997e-06 2024-09-16T20:48:42.299 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.299 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.299 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:42.299 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825484+0000", 2024-09-16T20:48:42.299 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049743385000000001 2024-09-16T20:48:42.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.300 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.300 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:42.300 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825572+0000", 2024-09-16T20:48:42.300 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7834e-05 2024-09-16T20:48:42.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.300 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.300 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:42.300 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825668+0000", 2024-09-16T20:48:42.300 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6209e-05 2024-09-16T20:48:42.300 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:42.300 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:42.300 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:42.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.301 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.301 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5069 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 6926336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:42.301 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.775753+0000", 2024-09-16T20:48:42.301 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.443731793000001, 2024-09-16T20:48:42.301 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050146430999999998, 2024-09-16T20:48:42.301 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:42.301 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:42.301 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:42.301 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:42.301 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:42.301 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:42.301 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5069 2024-09-16T20:48:42.301 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.302 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:42.302 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.302 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:42.302 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775753+0000", 2024-09-16T20:48:42.302 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:42.302 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.302 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.302 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:42.302 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775753+0000", 2024-09-16T20:48:42.302 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:42.302 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.302 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.302 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:42.302 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775865+0000", 2024-09-16T20:48:42.303 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000112402 2024-09-16T20:48:42.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.303 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.303 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:42.303 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775869+0000", 2024-09-16T20:48:42.303 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5130000000000001e-06 2024-09-16T20:48:42.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.303 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.303 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:42.303 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775877+0000", 2024-09-16T20:48:42.303 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1359999999999997e-06 2024-09-16T20:48:42.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.303 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.304 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:42.304 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775882+0000", 2024-09-16T20:48:42.304 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6710000000000004e-06 2024-09-16T20:48:42.304 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.304 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.304 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:42.304 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825729+0000", 2024-09-16T20:48:42.304 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049846898000000001 2024-09-16T20:48:42.304 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.304 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.304 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:42.304 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825807+0000", 2024-09-16T20:48:42.304 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7875999999999998e-05 2024-09-16T20:48:42.304 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.305 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:42.305 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825899+0000", 2024-09-16T20:48:42.305 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1935000000000007e-05 2024-09-16T20:48:42.305 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:42.305 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:42.305 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:42.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.305 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5070 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2338816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:42.305 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.775892+0000", 2024-09-16T20:48:42.305 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.443591939000001, 2024-09-16T20:48:42.305 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050155311000000001, 2024-09-16T20:48:42.306 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:42.306 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:42.306 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:42.306 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:42.306 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:42.306 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:42.306 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5070 2024-09-16T20:48:42.306 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.306 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:42.306 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.306 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:42.306 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775892+0000", 2024-09-16T20:48:42.306 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:42.306 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.307 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.307 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:42.307 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775892+0000", 2024-09-16T20:48:42.307 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:42.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.307 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.307 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:42.307 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776142+0000", 2024-09-16T20:48:42.307 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00024950500000000002 2024-09-16T20:48:42.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.307 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.307 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:42.307 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776146+0000", 2024-09-16T20:48:42.307 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2819999999999998e-06 2024-09-16T20:48:42.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.308 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.308 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:42.308 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776157+0000", 2024-09-16T20:48:42.308 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0807e-05 2024-09-16T20:48:42.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.308 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.308 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:42.308 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776165+0000", 2024-09-16T20:48:42.308 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1300000000000001e-06 2024-09-16T20:48:42.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.309 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.309 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:42.309 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825935+0000", 2024-09-16T20:48:42.309 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049769300000000002 2024-09-16T20:48:42.309 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.309 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.309 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:42.309 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825991+0000", 2024-09-16T20:48:42.309 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6177000000000001e-05 2024-09-16T20:48:42.309 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.309 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.309 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:42.309 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826048+0000", 2024-09-16T20:48:42.309 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.711e-05 2024-09-16T20:48:42.310 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:42.310 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:42.310 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:42.310 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.310 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.310 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5071 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 6934528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:42.310 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.776177+0000", 2024-09-16T20:48:42.310 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.443307535999999, 2024-09-16T20:48:42.310 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050036886000000003, 2024-09-16T20:48:42.310 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:42.310 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:42.310 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:42.310 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:42.311 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:42.311 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:42.311 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5071 2024-09-16T20:48:42.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.311 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:42.311 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.311 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:42.311 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776177+0000", 2024-09-16T20:48:42.311 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:42.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.311 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.311 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:42.312 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776177+0000", 2024-09-16T20:48:42.312 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:42.312 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.312 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.312 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:42.312 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776339+0000", 2024-09-16T20:48:42.312 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016238099999999999 2024-09-16T20:48:42.312 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.312 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.312 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:42.312 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776343+0000", 2024-09-16T20:48:42.312 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9670000000000003e-06 2024-09-16T20:48:42.312 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.312 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.313 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:42.313 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776354+0000", 2024-09-16T20:48:42.313 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0789e-05 2024-09-16T20:48:42.313 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.313 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.313 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:42.313 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776361+0000", 2024-09-16T20:48:42.313 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5200000000000003e-06 2024-09-16T20:48:42.313 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.313 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.313 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:42.313 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826078+0000", 2024-09-16T20:48:42.314 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049717513999999997 2024-09-16T20:48:42.314 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.314 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.314 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:42.314 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826141+0000", 2024-09-16T20:48:42.314 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2446000000000005e-05 2024-09-16T20:48:42.314 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.314 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.314 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:42.314 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826214+0000", 2024-09-16T20:48:42.314 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3269000000000006e-05 2024-09-16T20:48:42.314 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:42.314 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:42.315 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:42.315 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.315 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.315 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5072 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2347008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:42.315 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.776370+0000", 2024-09-16T20:48:42.315 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.443113974999999, 2024-09-16T20:48:42.315 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050097653999999998, 2024-09-16T20:48:42.315 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:42.315 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:42.315 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:42.315 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:42.315 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:42.316 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:42.316 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5072 2024-09-16T20:48:42.316 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.316 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:42.316 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.316 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:42.316 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776370+0000", 2024-09-16T20:48:42.316 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:42.316 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.317 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.317 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:42.317 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776370+0000", 2024-09-16T20:48:42.317 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:42.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.317 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.317 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:42.317 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776645+0000", 2024-09-16T20:48:42.317 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000274634 2024-09-16T20:48:42.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.317 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.317 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:42.318 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776654+0000", 2024-09-16T20:48:42.318 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9390000000000003e-06 2024-09-16T20:48:42.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.318 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.318 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:42.318 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776665+0000", 2024-09-16T20:48:42.318 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0524e-05 2024-09-16T20:48:42.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.318 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.318 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:42.318 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776671+0000", 2024-09-16T20:48:42.318 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7390000000000002e-06 2024-09-16T20:48:42.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.319 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.319 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:42.319 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826261+0000", 2024-09-16T20:48:42.319 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049589995999999997 2024-09-16T20:48:42.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.319 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.319 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:42.319 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826363+0000", 2024-09-16T20:48:42.319 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010158999999999999 2024-09-16T20:48:42.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:42.319 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:42.319 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:42.319 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826468+0000", 2024-09-16T20:48:42.319 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000105232 2024-09-16T20:48:42.320 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:42.320 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:42.320 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:42.320 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:42.320 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:42.320 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:42.321 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:42.320+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:42.322 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:48:42.323 INFO:teuthology.orchestra.run.smithi139.stdout:1851 trunc from 0xc9c0e00 to 0x2dc8000 2024-09-16T20:48:42.323 INFO:teuthology.orchestra.run.smithi139.stdout:1854 punch from 0x14d2e00 to 0x14d4a00, (0x1c00 bytes) 2024-09-16T20:48:42.324 INFO:teuthology.orchestra.run.smithi139.stdout:1856 read 0x2b6f400 thru 0x2b70d05 (0x1906 bytes) 2024-09-16T20:48:42.324 INFO:teuthology.orchestra.run.smithi139.stdout:1857 trunc from 0x2dc8000 to 0xda0d800 2024-09-16T20:48:42.324 INFO:teuthology.orchestra.run.smithi139.stdout:1858 trunc from 0xda0d800 to 0x7d10800 2024-09-16T20:48:42.324 INFO:teuthology.orchestra.run.smithi139.stdout:1862 write 0x5bb3a00 thru 0x5bbe128 (0xa729 bytes) 2024-09-16T20:48:42.324 INFO:teuthology.orchestra.run.smithi139.stdout:1863 read 0x47b6c00 thru 0x47c2a78 (0xbe79 bytes) 2024-09-16T20:48:42.324 INFO:teuthology.orchestra.run.smithi139.stdout:1865 write 0x2de3c00 thru 0x2df25a7 (0xe9a8 bytes) 2024-09-16T20:48:42.324 INFO:teuthology.orchestra.run.smithi139.stdout:1866 read 0x3fd8c00 thru 0x3fe3802 (0xac03 bytes) 2024-09-16T20:48:42.324 INFO:teuthology.orchestra.run.smithi139.stdout:1867 write 0xa6b9000 thru 0xa6c0a62 (0x7a63 bytes) 2024-09-16T20:48:42.324 INFO:teuthology.orchestra.run.smithi139.stdout:1869 read 0x34f5800 thru 0x34fd018 (0x7819 bytes) 2024-09-16T20:48:42.324 INFO:teuthology.orchestra.run.smithi139.stdout:1872 read 0x43b2c00 thru 0x43be34e (0xb74f bytes) 2024-09-16T20:48:42.324 INFO:teuthology.orchestra.run.smithi139.stdout:1875 trunc from 0xa6c0c00 to 0xc086000 2024-09-16T20:48:42.330 INFO:teuthology.orchestra.run.smithi139.stdout:1876 punch from 0x571fa00 to 0x5723a00, (0x4000 bytes) 2024-09-16T20:48:42.334 INFO:teuthology.orchestra.run.smithi139.stdout:1877 read 0xb9b2000 thru 0xb9c0dcb (0xedcc bytes) 2024-09-16T20:48:42.334 INFO:teuthology.orchestra.run.smithi139.stdout:1878 read 0x7d7a600 thru 0x7d7b1ce (0xbcf bytes) 2024-09-16T20:48:42.334 INFO:teuthology.orchestra.run.smithi139.stdout:1879 clone 14 order 23 su 524288 sc 5 2024-09-16T20:48:42.422 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T20:48:42.423 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:42.421+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:42.432 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:42.432 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:48:42.432 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:48:42.432 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:42.441 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:48:42.523 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:42.522+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:42.545 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:48:42.547 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:48:42.562 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:48:42.562 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:48:42.578 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:48:42.578 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:48:42.588 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:42.588 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:48:42.588 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:48:42.588 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:48:42.588 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:42.596 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:48:42.597 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:48:42.598 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:48:42.613 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:48:42.614 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:48:42.630 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:48:42.630 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:48:42.641 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:42.640+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:42.700 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T20:48:42.708 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:42.708 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:42.718 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:48:42.745 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:42.744+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:42.756 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5] out_osds: [3] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-16T20:48:42.756 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T20:48:42.756 INFO:tasks.thrashosds.thrasher:Removing osd 5, in_osds are: [0, 1, 2, 4, 5] 2024-09-16T20:48:42.757 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 5 2024-09-16T20:48:42.801 INFO:teuthology.orchestra.run.smithi139.stdout:truncating image image_client.0-clone13 from 0xc086000 (overlap 0x2dc8000) to 0x0 2024-09-16T20:48:42.825 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:42.824+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:42.871 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:42.871 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:42.881 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:48:42.926 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:42.925+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:42.995 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:42.995 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:43.005 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:48:43.027 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:43.025+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:43.127 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:43.126+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:43.138 INFO:tasks.ceph.mon.a.smithi029.stderr:2024-09-16T20:48:43.136+0000 7fe0ce96d640 -1 mon.a@0(leader).osd e54 definitely_dead 0 2024-09-16T20:48:43.152 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:43.152 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:43.161 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:48:43.228 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:43.228+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:43.269 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:43.269 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:43.277 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:48:43.329 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:43.328+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:43.389 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:43.389 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:43.398 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:48:43.409 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #12, image image_client.0-clone12 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent13 2024-09-16T20:48:43.430 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:43.429+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:43.482 INFO:teuthology.orchestra.run.smithi139.stdout:1881 write 0x41c8800 thru 0x41ce8f9 (0x60fa bytes) 2024-09-16T20:48:43.483 INFO:teuthology.orchestra.run.smithi139.stdout:1884 write 0x7b9dc00 thru 0x7ba699d (0x8d9e bytes) 2024-09-16T20:48:43.484 INFO:teuthology.orchestra.run.smithi139.stdout:1886 trunc from 0xc086000 to 0xca46000 2024-09-16T20:48:43.506 INFO:teuthology.orchestra.run.smithi139.stdout:1887 read 0x6af9600 thru 0x6afc249 (0x2c4a bytes) 2024-09-16T20:48:43.507 INFO:teuthology.orchestra.run.smithi139.stdout:1889 read 0x4320400 thru 0x4321015 (0xc16 bytes) 2024-09-16T20:48:43.511 INFO:teuthology.orchestra.run.smithi139.stdout:1891 trunc from 0xca46000 to 0x9333200 2024-09-16T20:48:43.513 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:43.513 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:43.524 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:48:43.525 INFO:teuthology.orchestra.run.smithi139.stdout:1892 read 0x217fe00 thru 0x21874c3 (0x76c4 bytes) 2024-09-16T20:48:43.527 INFO:teuthology.orchestra.run.smithi139.stdout:1893 read 0x28a9a00 thru 0x28ac48e (0x2a8f bytes) 2024-09-16T20:48:43.527 INFO:teuthology.orchestra.run.smithi139.stdout:1895 read 0x2aa2400 thru 0x2aae17a (0xbd7b bytes) 2024-09-16T20:48:43.528 INFO:teuthology.orchestra.run.smithi139.stdout:1896 punch from 0x4d93e00 to 0x4da1000, (0xd200 bytes) 2024-09-16T20:48:43.529 INFO:teuthology.orchestra.run.smithi139.stdout:1899 punch from 0x9dda00 to 0x9e2000, (0x4600 bytes) 2024-09-16T20:48:43.530 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:43.529+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:43.531 INFO:teuthology.orchestra.run.smithi139.stdout:1902 trunc from 0x9333200 to 0x761fa00 2024-09-16T20:48:43.555 INFO:teuthology.orchestra.run.smithi139.stdout:1903 read 0x16f2600 thru 0x16fe121 (0xbb22 bytes) 2024-09-16T20:48:43.557 INFO:teuthology.orchestra.run.smithi139.stdout:1907 read 0x4f1e600 thru 0x4f2b3a7 (0xcda8 bytes) 2024-09-16T20:48:43.557 INFO:teuthology.orchestra.run.smithi139.stdout:1908 punch from 0x50b0e00 to 0x50b7000, (0x6200 bytes) 2024-09-16T20:48:43.557 INFO:teuthology.orchestra.run.smithi139.stdout:1911 read 0x402aa00 thru 0x40331e8 (0x87e9 bytes) 2024-09-16T20:48:43.557 INFO:teuthology.orchestra.run.smithi139.stdout:1913 read 0xa4bc00 thru 0xa56f46 (0xb347 bytes) 2024-09-16T20:48:43.558 INFO:teuthology.orchestra.run.smithi139.stdout:1914 read 0x629a400 thru 0x629d414 (0x3015 bytes) 2024-09-16T20:48:43.558 INFO:teuthology.orchestra.run.smithi139.stdout:1915 punch from 0x43e6c00 to 0x43e7a00, (0xe00 bytes) 2024-09-16T20:48:43.558 INFO:teuthology.orchestra.run.smithi139.stdout:1916 write 0x3abc800 thru 0x3ac3068 (0x6869 bytes) 2024-09-16T20:48:43.559 INFO:teuthology.orchestra.run.smithi139.stdout:1917 punch from 0x2afa000 to 0x2afd800, (0x3800 bytes) 2024-09-16T20:48:43.559 INFO:teuthology.orchestra.run.smithi139.stdout:1919 punch from 0x50b400 to 0x518400, (0xd000 bytes) 2024-09-16T20:48:43.559 INFO:teuthology.orchestra.run.smithi139.stdout:1920 read 0x6340a00 thru 0x63508bf (0xfec0 bytes) 2024-09-16T20:48:43.559 INFO:teuthology.orchestra.run.smithi139.stdout:1921 read 0x1107e00 thru 0x110a644 (0x2845 bytes) 2024-09-16T20:48:43.592 INFO:teuthology.orchestra.run.smithi139.stdout:1922 write 0xde5a000 thru 0xde60ee6 (0x6ee7 bytes) 2024-09-16T20:48:43.605 INFO:teuthology.orchestra.run.smithi139.stdout:1924 trunc from 0xde61000 to 0x3de2800 2024-09-16T20:48:43.627 INFO:teuthology.orchestra.run.smithi139.stdout:1925 read 0xb31a00 thru 0xb396b2 (0x7cb3 bytes) 2024-09-16T20:48:43.628 INFO:teuthology.orchestra.run.smithi139.stdout:1926 punch from 0x37f0800 to 0x37f3800, (0x3000 bytes) 2024-09-16T20:48:43.631 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:43.630+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:43.642 INFO:teuthology.orchestra.run.smithi139.stdout:1927 write 0x7ef3a00 thru 0x7efee87 (0xb488 bytes) 2024-09-16T20:48:43.643 INFO:teuthology.orchestra.run.smithi139.stdout:1928 read 0x4e01000 thru 0x4e0f0e8 (0xe0e9 bytes) 2024-09-16T20:48:43.643 INFO:teuthology.orchestra.run.smithi139.stdout:1932 punch from 0x3387000 to 0x3387600, (0x600 bytes) 2024-09-16T20:48:43.644 INFO:teuthology.orchestra.run.smithi139.stdout:1934 write 0x7703c00 thru 0x770dcbe (0xa0bf bytes) 2024-09-16T20:48:43.645 INFO:teuthology.orchestra.run.smithi139.stdout:1935 trunc from 0x7eff000 to 0x6d6d200 2024-09-16T20:48:43.650 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:43.650 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:43.659 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:48:43.662 INFO:teuthology.orchestra.run.smithi139.stdout:1936 punch from 0x1749800 to 0x1755a00, (0xc200 bytes) 2024-09-16T20:48:43.664 INFO:teuthology.orchestra.run.smithi139.stdout:1940 write 0x1d14c00 thru 0x1d22b6f (0xdf70 bytes) 2024-09-16T20:48:43.665 INFO:teuthology.orchestra.run.smithi139.stdout:1941 read 0x588d000 thru 0x5895298 (0x8299 bytes) 2024-09-16T20:48:43.671 INFO:teuthology.orchestra.run.smithi139.stdout:1943 trunc from 0x6d6d200 to 0x1d8f000 2024-09-16T20:48:43.698 INFO:teuthology.orchestra.run.smithi029.stderr:noscrub is set 2024-09-16T20:48:43.702 INFO:teuthology.orchestra.run.smithi029.stderr:marked out osd.5. 2024-09-16T20:48:43.759 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:43.758+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:43.811 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:43.811 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:43.820 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:48:43.860 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:43.859+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:43.934 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:43.933+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:43.966 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:43.967 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:43.975 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:48:44.035 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:44.034+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:44.084 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:44.085 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:44.093 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:48:44.135 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:44.135+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:44.237 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:44.236+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:44.241 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:44.242 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:44.251 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:48:44.338 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:44.336+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:44.357 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:44.358 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:44.366 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:48:44.449 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:44.448+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:44.513 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:44.513 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:44.521 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:48:44.563 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:44.561+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:44.640 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:44.639+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:44.641 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T20:48:44.669 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:44.670 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:44.678 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:48:44.741 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:44.740+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:44.770 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:48:44.771 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:48:44.788 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:48:44.788 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:48:44.804 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:48:44.804 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:48:44.823 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:48:44.823 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:48:44.829 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:44.829 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:44.838 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:48:44.840 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:48:44.840 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:48:44.857 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:48:44.858 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:48:44.881 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:44.880+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:44.944 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:44.943+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:44.945 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:44.945 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:48:44.946 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:48:44.946 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:44.954 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:48:45.044 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:45.043+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:45.100 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:45.100 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:48:45.101 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:48:45.101 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:48:45.101 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:45.109 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:48:45.152 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:45.151+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:45.217 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:45.217 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:48:45.217 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:48:45.217 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:48:45.217 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.217 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:780 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 933888~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:45.217 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.239489+0000", 2024-09-16T20:48:45.218 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 29.975847889000001, 2024-09-16T20:48:45.218 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00741699, 2024-09-16T20:48:45.218 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:45.218 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:45.218 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:45.218 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:45.218 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:45.218 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:45.218 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 780 2024-09-16T20:48:45.218 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.218 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:45.218 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.218 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:45.218 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239489+0000", 2024-09-16T20:48:45.219 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:45.219 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.219 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.219 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:45.219 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239489+0000", 2024-09-16T20:48:45.219 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:45.219 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.219 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.219 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:45.219 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239490+0000", 2024-09-16T20:48:45.219 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8599999999999998e-07 2024-09-16T20:48:45.219 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.219 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.219 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:45.220 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239492+0000", 2024-09-16T20:48:45.220 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1380000000000001e-06 2024-09-16T20:48:45.220 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.220 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.220 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:45.220 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239499+0000", 2024-09-16T20:48:45.220 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9530000000000002e-06 2024-09-16T20:48:45.220 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.220 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.220 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:45.220 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239504+0000", 2024-09-16T20:48:45.220 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0690000000000001e-06 2024-09-16T20:48:45.220 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.221 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.221 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:45.221 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246714+0000", 2024-09-16T20:48:45.221 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.007209844 2024-09-16T20:48:45.221 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.221 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.221 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:45.221 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246804+0000", 2024-09-16T20:48:45.221 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0109000000000001e-05 2024-09-16T20:48:45.221 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.221 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.221 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:45.221 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246906+0000", 2024-09-16T20:48:45.222 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000102391 2024-09-16T20:48:45.222 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:45.222 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:45.222 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:45.222 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.222 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.222 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:781 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1048576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:45.222 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.239892+0000", 2024-09-16T20:48:45.222 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 29.975445519000001, 2024-09-16T20:48:45.222 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0072223799999999996, 2024-09-16T20:48:45.222 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:45.222 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:45.222 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:45.223 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:45.223 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:45.223 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:45.223 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 781 2024-09-16T20:48:45.223 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.223 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:45.223 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.223 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:45.223 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239892+0000", 2024-09-16T20:48:45.223 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:45.223 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.223 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.223 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:45.224 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239892+0000", 2024-09-16T20:48:45.224 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:45.224 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.224 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.224 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:45.224 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239893+0000", 2024-09-16T20:48:45.224 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1399999999999995e-07 2024-09-16T20:48:45.224 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.224 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.224 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:45.224 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239897+0000", 2024-09-16T20:48:45.224 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4859999999999999e-06 2024-09-16T20:48:45.224 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.225 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.225 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:45.225 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239907+0000", 2024-09-16T20:48:45.225 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4539999999999996e-06 2024-09-16T20:48:45.225 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.225 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.225 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:45.225 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239913+0000", 2024-09-16T20:48:45.225 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7880000000000001e-06 2024-09-16T20:48:45.225 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.225 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.225 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:45.225 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246937+0000", 2024-09-16T20:48:45.226 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0070239999999999999 2024-09-16T20:48:45.226 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.226 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.226 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:45.226 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247013+0000", 2024-09-16T20:48:45.226 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5894000000000002e-05 2024-09-16T20:48:45.226 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.226 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.226 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:45.226 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247114+0000", 2024-09-16T20:48:45.226 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000100844 2024-09-16T20:48:45.226 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:45.226 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:45.226 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:45.227 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.227 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.227 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:782 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1146880~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:45.227 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.239925+0000", 2024-09-16T20:48:45.227 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 29.975412546000001, 2024-09-16T20:48:45.227 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0074161840000000001, 2024-09-16T20:48:45.227 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:45.227 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:45.227 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:45.227 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:45.227 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:45.227 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:45.227 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 782 2024-09-16T20:48:45.228 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.228 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:45.228 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.228 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:45.228 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239925+0000", 2024-09-16T20:48:45.228 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:45.228 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.228 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.228 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:45.228 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239925+0000", 2024-09-16T20:48:45.228 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:45.228 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.228 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.228 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:45.229 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239925+0000", 2024-09-16T20:48:45.229 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7199999999999999e-07 2024-09-16T20:48:45.229 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.229 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.229 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:45.229 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239928+0000", 2024-09-16T20:48:45.229 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5349999999999999e-06 2024-09-16T20:48:45.229 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.229 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.229 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:45.229 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239935+0000", 2024-09-16T20:48:45.229 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7680000000000003e-06 2024-09-16T20:48:45.229 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.229 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.229 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:45.230 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239940+0000", 2024-09-16T20:48:45.230 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.152e-06 2024-09-16T20:48:45.230 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.230 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.230 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:45.230 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247143+0000", 2024-09-16T20:48:45.230 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0072037309999999997 2024-09-16T20:48:45.230 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.230 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.230 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:45.230 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247230+0000", 2024-09-16T20:48:45.230 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6598e-05 2024-09-16T20:48:45.230 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.230 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.231 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:45.231 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247341+0000", 2024-09-16T20:48:45.231 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011092800000000001 2024-09-16T20:48:45.231 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:45.231 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:45.231 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:45.231 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.231 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.231 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:783 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1261568~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:45.231 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.239947+0000", 2024-09-16T20:48:45.231 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 29.975390224000002, 2024-09-16T20:48:45.231 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0076044789999999999, 2024-09-16T20:48:45.231 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:45.231 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:45.232 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:45.232 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:45.232 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:45.232 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:45.232 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 783 2024-09-16T20:48:45.232 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.232 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:45.232 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.232 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:45.232 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239947+0000", 2024-09-16T20:48:45.232 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:45.232 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.232 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.233 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:45.233 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239947+0000", 2024-09-16T20:48:45.233 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:45.233 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.233 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.233 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:45.233 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239947+0000", 2024-09-16T20:48:45.233 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9400000000000001e-07 2024-09-16T20:48:45.233 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.233 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.233 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:45.233 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239950+0000", 2024-09-16T20:48:45.233 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2350000000000002e-06 2024-09-16T20:48:45.234 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.234 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.234 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:45.234 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239956+0000", 2024-09-16T20:48:45.234 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3400000000000003e-06 2024-09-16T20:48:45.234 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.234 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.234 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:45.234 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239961+0000", 2024-09-16T20:48:45.234 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.04e-06 2024-09-16T20:48:45.234 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.234 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.234 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:45.234 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247371+0000", 2024-09-16T20:48:45.235 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0074096810000000004 2024-09-16T20:48:45.235 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.235 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.235 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:45.235 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247451+0000", 2024-09-16T20:48:45.235 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9845999999999994e-05 2024-09-16T20:48:45.235 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.235 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.235 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:45.235 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247552+0000", 2024-09-16T20:48:45.235 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000100943 2024-09-16T20:48:45.235 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:45.235 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:45.235 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:45.236 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.236 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.236 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:784 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1376256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:45.236 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.239997+0000", 2024-09-16T20:48:45.236 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 29.975340568, 2024-09-16T20:48:45.236 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.007775372, 2024-09-16T20:48:45.236 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:45.236 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:45.236 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:45.236 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:45.236 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:45.236 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:45.236 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 784 2024-09-16T20:48:45.236 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.237 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:45.237 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.237 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:45.237 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239997+0000", 2024-09-16T20:48:45.237 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:45.237 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.237 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.237 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:45.237 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239997+0000", 2024-09-16T20:48:45.237 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:45.237 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.237 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.237 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:45.237 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239997+0000", 2024-09-16T20:48:45.238 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1799999999999995e-07 2024-09-16T20:48:45.238 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.238 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.238 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:45.238 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240001+0000", 2024-09-16T20:48:45.238 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6009999999999999e-06 2024-09-16T20:48:45.238 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.238 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.238 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:45.238 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240009+0000", 2024-09-16T20:48:45.238 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5809999999999999e-06 2024-09-16T20:48:45.238 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.238 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.239 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:45.239 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240015+0000", 2024-09-16T20:48:45.239 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0100000000000001e-06 2024-09-16T20:48:45.239 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.239 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.239 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:45.239 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247583+0000", 2024-09-16T20:48:45.239 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0075685730000000003 2024-09-16T20:48:45.239 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.239 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.239 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:45.239 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247672+0000", 2024-09-16T20:48:45.239 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8480000000000007e-05 2024-09-16T20:48:45.239 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.240 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.240 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:45.240 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247772+0000", 2024-09-16T20:48:45.240 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000100509 2024-09-16T20:48:45.240 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:45.240 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:45.240 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:45.240 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.240 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.240 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:785 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1474560~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:45.240 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240024+0000", 2024-09-16T20:48:45.240 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 29.975312981999998, 2024-09-16T20:48:45.240 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.008096776, 2024-09-16T20:48:45.240 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:45.241 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:45.241 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:45.241 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:45.241 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:45.241 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:45.241 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 785 2024-09-16T20:48:45.241 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.241 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:45.241 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.241 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:45.241 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240024+0000", 2024-09-16T20:48:45.241 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:45.241 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.241 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.242 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:45.242 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240024+0000", 2024-09-16T20:48:45.242 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:45.242 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.242 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.242 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:45.242 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240025+0000", 2024-09-16T20:48:45.242 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5000000000000003e-07 2024-09-16T20:48:45.242 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.242 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.242 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:45.242 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240027+0000", 2024-09-16T20:48:45.242 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2900000000000001e-06 2024-09-16T20:48:45.242 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.243 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.243 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:45.243 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240036+0000", 2024-09-16T20:48:45.243 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4710000000000007e-06 2024-09-16T20:48:45.243 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.243 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.243 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:45.243 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240041+0000", 2024-09-16T20:48:45.243 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4689999999999996e-06 2024-09-16T20:48:45.243 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.243 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.243 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:45.243 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247821+0000", 2024-09-16T20:48:45.243 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0077797919999999998 2024-09-16T20:48:45.244 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.244 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.244 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:45.244 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247921+0000", 2024-09-16T20:48:45.244 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9814000000000004e-05 2024-09-16T20:48:45.244 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.244 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.244 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:45.244 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248121+0000", 2024-09-16T20:48:45.244 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00020039 2024-09-16T20:48:45.244 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:45.244 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:45.244 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:45.244 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.245 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.245 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:940 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 81920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:45.245 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240569+0000", 2024-09-16T20:48:45.245 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 29.974768724, 2024-09-16T20:48:45.245 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0078289759999999996, 2024-09-16T20:48:45.245 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:45.245 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:45.245 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:45.245 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:45.245 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:45.245 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:45.245 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 940 2024-09-16T20:48:45.245 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.246 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:45.246 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.246 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:45.246 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240569+0000", 2024-09-16T20:48:45.246 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:45.246 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.246 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.246 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:45.246 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240569+0000", 2024-09-16T20:48:45.246 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:45.246 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.246 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.246 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:45.246 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240569+0000", 2024-09-16T20:48:45.247 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2600000000000001e-07 2024-09-16T20:48:45.247 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.247 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.247 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:45.247 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240575+0000", 2024-09-16T20:48:45.247 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1159999999999998e-06 2024-09-16T20:48:45.247 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.247 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.247 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:45.247 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240587+0000", 2024-09-16T20:48:45.247 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1915e-05 2024-09-16T20:48:45.247 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.247 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.247 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:45.248 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240594+0000", 2024-09-16T20:48:45.248 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5610000000000001e-06 2024-09-16T20:48:45.248 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.248 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.248 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:45.248 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248158+0000", 2024-09-16T20:48:45.248 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0075635370000000004 2024-09-16T20:48:45.248 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.248 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.248 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:45.248 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248278+0000", 2024-09-16T20:48:45.248 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012009400000000001 2024-09-16T20:48:45.248 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.249 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.249 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:45.249 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248398+0000", 2024-09-16T20:48:45.249 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000119827 2024-09-16T20:48:45.249 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:45.249 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:45.249 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:45.249 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.249 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.249 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:941 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 196608~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:45.249 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240608+0000", 2024-09-16T20:48:45.249 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 29.974729147000001, 2024-09-16T20:48:45.249 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0080495519999999997, 2024-09-16T20:48:45.249 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:45.250 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:45.250 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:45.250 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:45.250 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:45.250 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:45.250 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 941 2024-09-16T20:48:45.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.250 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:45.250 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.250 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:45.250 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240608+0000", 2024-09-16T20:48:45.250 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:45.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.250 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.251 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:45.251 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240608+0000", 2024-09-16T20:48:45.251 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:45.251 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.251 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.251 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:45.251 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240609+0000", 2024-09-16T20:48:45.251 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7000000000000004e-07 2024-09-16T20:48:45.251 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.251 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.251 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:45.251 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240612+0000", 2024-09-16T20:48:45.251 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8310000000000001e-06 2024-09-16T20:48:45.251 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.252 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.252 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:45.252 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240619+0000", 2024-09-16T20:48:45.252 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8429999999999993e-06 2024-09-16T20:48:45.252 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.252 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.252 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:45.252 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240625+0000", 2024-09-16T20:48:45.252 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5659999999999997e-06 2024-09-16T20:48:45.252 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.252 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.252 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:45.252 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248456+0000", 2024-09-16T20:48:45.252 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0078313940000000002 2024-09-16T20:48:45.253 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.253 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.253 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:45.253 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248548+0000", 2024-09-16T20:48:45.253 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1324999999999997e-05 2024-09-16T20:48:45.253 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.253 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.253 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:45.253 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248658+0000", 2024-09-16T20:48:45.253 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000109923 2024-09-16T20:48:45.253 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:45.253 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:45.253 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:45.254 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.254 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.254 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:942 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 294912~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:45.254 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240675+0000", 2024-09-16T20:48:45.254 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 29.974662339999998, 2024-09-16T20:48:45.254 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0082135699999999999, 2024-09-16T20:48:45.254 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:45.254 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:45.254 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:45.254 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:45.254 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:45.254 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:45.254 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 942 2024-09-16T20:48:45.254 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.255 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:45.255 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.255 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:45.255 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240675+0000", 2024-09-16T20:48:45.255 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:45.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.255 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.255 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:45.255 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240675+0000", 2024-09-16T20:48:45.255 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:45.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.255 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.255 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:45.255 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240676+0000", 2024-09-16T20:48:45.256 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.37e-07 2024-09-16T20:48:45.256 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.256 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.256 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:45.256 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240679+0000", 2024-09-16T20:48:45.256 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4060000000000001e-06 2024-09-16T20:48:45.256 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.256 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.256 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:45.256 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240687+0000", 2024-09-16T20:48:45.256 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4640000000000006e-06 2024-09-16T20:48:45.256 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.256 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.256 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:45.257 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240694+0000", 2024-09-16T20:48:45.257 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.246e-06 2024-09-16T20:48:45.257 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.257 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.257 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:45.257 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248693+0000", 2024-09-16T20:48:45.257 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0079997550000000008 2024-09-16T20:48:45.257 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.257 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.257 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:45.257 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248781+0000", 2024-09-16T20:48:45.257 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7509999999999994e-05 2024-09-16T20:48:45.257 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.257 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.258 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:45.258 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248889+0000", 2024-09-16T20:48:45.258 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000107552 2024-09-16T20:48:45.258 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:45.258 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:45.258 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:45.258 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.258 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.258 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:943 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 409600~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:45.258 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240704+0000", 2024-09-16T20:48:45.258 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 29.974633344000001, 2024-09-16T20:48:45.259 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0083978209999999998, 2024-09-16T20:48:45.259 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:45.259 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:45.259 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:45.259 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:45.259 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:45.259 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:45.259 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 943 2024-09-16T20:48:45.259 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.259 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:45.259 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.259 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:45.259 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240704+0000", 2024-09-16T20:48:45.259 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:45.260 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.260 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.260 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:45.260 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240704+0000", 2024-09-16T20:48:45.260 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:45.260 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.260 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.260 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:45.260 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240704+0000", 2024-09-16T20:48:45.260 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7e-07 2024-09-16T20:48:45.260 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.260 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.260 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:45.261 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240707+0000", 2024-09-16T20:48:45.261 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5809999999999999e-06 2024-09-16T20:48:45.261 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.261 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.261 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:45.261 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240713+0000", 2024-09-16T20:48:45.261 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4949999999999996e-06 2024-09-16T20:48:45.261 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.261 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.261 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:45.261 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240718+0000", 2024-09-16T20:48:45.261 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8890000000000001e-06 2024-09-16T20:48:45.261 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.261 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.262 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:45.262 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248920+0000", 2024-09-16T20:48:45.262 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0082018679999999993 2024-09-16T20:48:45.262 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.262 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.262 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:45.262 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249007+0000", 2024-09-16T20:48:45.262 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6327000000000002e-05 2024-09-16T20:48:45.262 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.262 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.262 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:45.262 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249102+0000", 2024-09-16T20:48:45.262 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5191000000000004e-05 2024-09-16T20:48:45.262 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:45.263 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:45.263 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:45.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.263 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.263 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:944 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 524288~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:45.263 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240726+0000", 2024-09-16T20:48:45.263 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 29.974611539000001, 2024-09-16T20:48:45.263 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0085999960000000004, 2024-09-16T20:48:45.264 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:45.264 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:45.264 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:45.264 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:45.264 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:45.264 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:45.264 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 944 2024-09-16T20:48:45.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.264 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:45.264 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.264 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:45.264 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240726+0000", 2024-09-16T20:48:45.265 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:45.265 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.265 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.265 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:45.265 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240726+0000", 2024-09-16T20:48:45.265 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:45.265 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.265 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.265 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:45.265 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240726+0000", 2024-09-16T20:48:45.265 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7099999999999997e-07 2024-09-16T20:48:45.266 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.266 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.266 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:45.266 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240728+0000", 2024-09-16T20:48:45.266 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.125e-06 2024-09-16T20:48:45.266 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.266 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.266 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:45.266 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240734+0000", 2024-09-16T20:48:45.266 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1890000000000004e-06 2024-09-16T20:48:45.266 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.266 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.266 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:45.266 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240739+0000", 2024-09-16T20:48:45.267 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0340000000000004e-06 2024-09-16T20:48:45.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.267 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.267 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:45.267 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249131+0000", 2024-09-16T20:48:45.267 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0083919579999999997 2024-09-16T20:48:45.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.268 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.268 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:45.268 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249217+0000", 2024-09-16T20:48:45.268 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5154999999999994e-05 2024-09-16T20:48:45.268 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.268 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.268 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:45.268 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249326+0000", 2024-09-16T20:48:45.269 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000109164 2024-09-16T20:48:45.269 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:45.269 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:45.269 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:45.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.269 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.269 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:945 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 622592~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:45.269 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240746+0000", 2024-09-16T20:48:45.269 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 29.974591400000001, 2024-09-16T20:48:45.269 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0088083320000000003, 2024-09-16T20:48:45.270 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:45.270 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:45.270 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:45.270 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:45.270 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:45.270 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:45.270 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 945 2024-09-16T20:48:45.270 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.270 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:45.270 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.270 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:45.270 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240746+0000", 2024-09-16T20:48:45.270 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:45.270 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.271 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.271 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:45.271 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240746+0000", 2024-09-16T20:48:45.271 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:45.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.271 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.271 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:45.271 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240746+0000", 2024-09-16T20:48:45.271 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8500000000000002e-07 2024-09-16T20:48:45.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.271 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.271 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:45.271 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240748+0000", 2024-09-16T20:48:45.271 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8449999999999999e-06 2024-09-16T20:48:45.272 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.272 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.272 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:45.272 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240754+0000", 2024-09-16T20:48:45.272 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8830000000000003e-06 2024-09-16T20:48:45.272 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.272 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.272 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:45.272 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240759+0000", 2024-09-16T20:48:45.272 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0220000000000004e-06 2024-09-16T20:48:45.272 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.272 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.272 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:45.272 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249360+0000", 2024-09-16T20:48:45.273 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.008601437 2024-09-16T20:48:45.273 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.273 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.273 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:45.273 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249453+0000", 2024-09-16T20:48:45.273 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2743000000000003e-05 2024-09-16T20:48:45.273 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.273 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.273 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:45.273 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249554+0000", 2024-09-16T20:48:45.273 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101017 2024-09-16T20:48:45.273 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:45.273 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:45.273 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:45.273 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.274 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.274 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:946 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 737280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:45.274 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240766+0000", 2024-09-16T20:48:45.274 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 29.974571697000002, 2024-09-16T20:48:45.274 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0089092070000000006, 2024-09-16T20:48:45.274 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:45.274 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:45.274 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:45.274 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:45.274 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:45.274 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:45.274 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 946 2024-09-16T20:48:45.274 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.274 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:45.275 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.275 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:45.275 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240766+0000", 2024-09-16T20:48:45.275 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:45.275 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.275 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.275 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:45.275 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240766+0000", 2024-09-16T20:48:45.275 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:45.275 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.275 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.276 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:45.276 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240766+0000", 2024-09-16T20:48:45.276 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0600000000000001e-07 2024-09-16T20:48:45.276 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.276 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.276 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:45.276 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240768+0000", 2024-09-16T20:48:45.276 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8959999999999999e-06 2024-09-16T20:48:45.276 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.276 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.276 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:45.276 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240774+0000", 2024-09-16T20:48:45.276 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0979999999999999e-06 2024-09-16T20:48:45.277 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.277 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.277 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:45.277 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240779+0000", 2024-09-16T20:48:45.277 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.933e-06 2024-09-16T20:48:45.277 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.277 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.277 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:45.277 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249572+0000", 2024-09-16T20:48:45.277 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0087926380000000002 2024-09-16T20:48:45.277 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.277 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.277 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:45.277 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249612+0000", 2024-09-16T20:48:45.278 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0859000000000003e-05 2024-09-16T20:48:45.278 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.278 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.278 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:45.278 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249675+0000", 2024-09-16T20:48:45.278 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2377000000000003e-05 2024-09-16T20:48:45.278 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:45.278 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:45.278 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:45.278 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.278 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.278 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:947 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 851968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:45.278 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240785+0000", 2024-09-16T20:48:45.278 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 29.974552105000001, 2024-09-16T20:48:45.279 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0090960260000000001, 2024-09-16T20:48:45.279 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:45.279 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:45.279 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:45.279 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:45.279 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:45.279 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:45.279 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 947 2024-09-16T20:48:45.279 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.279 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:45.279 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.279 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:45.279 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240785+0000", 2024-09-16T20:48:45.280 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:45.280 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.280 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.280 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:45.280 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240785+0000", 2024-09-16T20:48:45.280 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:45.280 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.280 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.280 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:45.280 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240786+0000", 2024-09-16T20:48:45.280 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-16T20:48:45.280 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.280 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.281 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:45.281 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240787+0000", 2024-09-16T20:48:45.281 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.858e-06 2024-09-16T20:48:45.281 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.281 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.281 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:45.281 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240793+0000", 2024-09-16T20:48:45.281 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7760000000000003e-06 2024-09-16T20:48:45.281 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.281 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.281 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:45.281 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240799+0000", 2024-09-16T20:48:45.281 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1109999999999996e-06 2024-09-16T20:48:45.281 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.282 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.282 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:45.282 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249715+0000", 2024-09-16T20:48:45.282 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0089159789999999992 2024-09-16T20:48:45.282 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.282 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.282 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:45.282 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249766+0000", 2024-09-16T20:48:45.282 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0968999999999999e-05 2024-09-16T20:48:45.282 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.282 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.282 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:45.282 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249881+0000", 2024-09-16T20:48:45.283 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114963 2024-09-16T20:48:45.283 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:45.283 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:45.283 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:45.283 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.283 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.283 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:948 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:45.283 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240806+0000", 2024-09-16T20:48:45.283 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 29.974530925, 2024-09-16T20:48:45.283 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0092300609999999995, 2024-09-16T20:48:45.283 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:45.283 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:45.283 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:45.283 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:45.284 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:45.284 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:45.284 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 948 2024-09-16T20:48:45.284 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.284 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:45.284 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.284 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:45.284 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240806+0000", 2024-09-16T20:48:45.284 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:45.284 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.284 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.284 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:45.284 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240806+0000", 2024-09-16T20:48:45.285 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:45.285 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.285 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.285 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:45.285 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240807+0000", 2024-09-16T20:48:45.285 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4499999999999998e-07 2024-09-16T20:48:45.285 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.285 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.285 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:45.285 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240809+0000", 2024-09-16T20:48:45.285 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.34e-06 2024-09-16T20:48:45.285 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.285 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.286 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:45.286 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240815+0000", 2024-09-16T20:48:45.286 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1449999999999996e-06 2024-09-16T20:48:45.286 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.286 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.286 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:45.286 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240821+0000", 2024-09-16T20:48:45.286 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0270000000000001e-06 2024-09-16T20:48:45.286 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.286 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.286 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:45.286 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249920+0000", 2024-09-16T20:48:45.286 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0090988460000000007 2024-09-16T20:48:45.286 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.287 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.287 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:45.287 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249966+0000", 2024-09-16T20:48:45.287 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5918999999999999e-05 2024-09-16T20:48:45.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.287 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.287 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:45.287 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250036+0000", 2024-09-16T20:48:45.287 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0438999999999994e-05 2024-09-16T20:48:45.287 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:45.287 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:45.287 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:45.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.288 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.288 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:949 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1064960~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:45.288 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240833+0000", 2024-09-16T20:48:45.288 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 29.97450431, 2024-09-16T20:48:45.288 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0093580339999999994, 2024-09-16T20:48:45.288 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:45.288 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:45.288 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:45.288 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:45.288 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:45.289 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:45.289 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 949 2024-09-16T20:48:45.289 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.289 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:45.289 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.289 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:45.289 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240833+0000", 2024-09-16T20:48:45.289 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:45.289 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.289 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.289 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:45.289 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240833+0000", 2024-09-16T20:48:45.289 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:45.290 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.290 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.290 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:45.290 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240833+0000", 2024-09-16T20:48:45.290 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2899999999999999e-07 2024-09-16T20:48:45.290 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.290 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.290 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:45.290 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240836+0000", 2024-09-16T20:48:45.290 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2649999999999999e-06 2024-09-16T20:48:45.290 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.290 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.290 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:45.290 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240842+0000", 2024-09-16T20:48:45.291 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5579999999999997e-06 2024-09-16T20:48:45.291 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.291 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.291 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:45.291 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240847+0000", 2024-09-16T20:48:45.291 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2009999999999998e-06 2024-09-16T20:48:45.291 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.291 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.291 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:45.291 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250073+0000", 2024-09-16T20:48:45.291 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0092258710000000001 2024-09-16T20:48:45.291 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.291 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.292 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:45.292 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250141+0000", 2024-09-16T20:48:45.292 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7849999999999996e-05 2024-09-16T20:48:45.292 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.292 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.292 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:45.292 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250191+0000", 2024-09-16T20:48:45.292 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.986e-05 2024-09-16T20:48:45.292 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:45.292 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:45.292 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:45.292 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.292 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.293 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:950 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1179648~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:45.293 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240855+0000", 2024-09-16T20:48:45.293 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 29.974482058, 2024-09-16T20:48:45.293 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0095855560000000003, 2024-09-16T20:48:45.293 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:45.293 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:45.293 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:45.293 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:45.293 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:45.293 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:45.293 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 950 2024-09-16T20:48:45.293 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.293 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:45.293 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.294 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:45.294 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240855+0000", 2024-09-16T20:48:45.294 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:45.294 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.294 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.294 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:45.294 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240855+0000", 2024-09-16T20:48:45.294 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:45.294 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.294 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.294 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:45.294 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240856+0000", 2024-09-16T20:48:45.294 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4999999999999998e-07 2024-09-16T20:48:45.294 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.295 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.295 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:45.295 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240858+0000", 2024-09-16T20:48:45.295 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.996e-06 2024-09-16T20:48:45.295 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.295 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.295 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:45.295 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240864+0000", 2024-09-16T20:48:45.295 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9359999999999996e-06 2024-09-16T20:48:45.295 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.295 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.296 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:45.296 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240868+0000", 2024-09-16T20:48:45.296 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7670000000000003e-06 2024-09-16T20:48:45.296 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.296 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.296 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:45.296 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250226+0000", 2024-09-16T20:48:45.296 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0093576100000000006 2024-09-16T20:48:45.296 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.296 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.296 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:45.296 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250289+0000", 2024-09-16T20:48:45.296 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2627999999999997e-05 2024-09-16T20:48:45.297 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.297 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.297 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:45.297 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250441+0000", 2024-09-16T20:48:45.297 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015216900000000001 2024-09-16T20:48:45.297 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:45.297 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:45.297 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:45.297 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.297 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.297 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:951 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1277952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:45.297 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.241044+0000", 2024-09-16T20:48:45.297 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 29.974292869999999, 2024-09-16T20:48:45.297 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0096266589999999992, 2024-09-16T20:48:45.298 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:45.298 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:45.298 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:45.298 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:45.298 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:45.298 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:45.298 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 951 2024-09-16T20:48:45.298 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.298 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:45.298 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.298 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:45.298 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241044+0000", 2024-09-16T20:48:45.298 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:45.298 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.299 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.299 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:45.299 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241044+0000", 2024-09-16T20:48:45.299 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:45.299 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.299 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.299 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:45.299 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241045+0000", 2024-09-16T20:48:45.299 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5300000000000004e-07 2024-09-16T20:48:45.299 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.299 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.299 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:45.299 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241048+0000", 2024-09-16T20:48:45.299 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.303e-06 2024-09-16T20:48:45.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.300 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.300 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:45.300 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241057+0000", 2024-09-16T20:48:45.300 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5550000000000003e-06 2024-09-16T20:48:45.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.300 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.300 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:45.300 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241063+0000", 2024-09-16T20:48:45.300 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3470000000000004e-06 2024-09-16T20:48:45.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.300 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.300 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:45.300 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250477+0000", 2024-09-16T20:48:45.301 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0094136159999999997 2024-09-16T20:48:45.301 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.301 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.301 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:45.301 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250562+0000", 2024-09-16T20:48:45.301 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4709000000000005e-05 2024-09-16T20:48:45.301 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.301 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.301 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:45.301 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250671+0000", 2024-09-16T20:48:45.301 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010947599999999999 2024-09-16T20:48:45.301 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:45.301 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:45.301 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:45.302 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.302 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.302 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:952 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1392640~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:45.302 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.241130+0000", 2024-09-16T20:48:45.302 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 29.974207004, 2024-09-16T20:48:45.302 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0098096290000000003, 2024-09-16T20:48:45.302 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:45.302 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:45.302 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:45.302 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:45.302 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:45.303 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:45.303 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 952 2024-09-16T20:48:45.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.303 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:45.303 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.303 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:45.303 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241130+0000", 2024-09-16T20:48:45.303 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:45.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.303 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.303 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:45.303 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241130+0000", 2024-09-16T20:48:45.303 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:45.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.304 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.304 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:45.304 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241131+0000", 2024-09-16T20:48:45.304 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.61e-07 2024-09-16T20:48:45.304 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.304 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.304 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:45.304 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241134+0000", 2024-09-16T20:48:45.304 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1080000000000001e-06 2024-09-16T20:48:45.304 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.304 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.304 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:45.304 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241142+0000", 2024-09-16T20:48:45.304 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9119999999999998e-06 2024-09-16T20:48:45.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.305 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:45.305 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241148+0000", 2024-09-16T20:48:45.305 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1410000000000002e-06 2024-09-16T20:48:45.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.305 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:45.305 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250731+0000", 2024-09-16T20:48:45.305 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0095833979999999999 2024-09-16T20:48:45.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.305 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:45.305 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250820+0000", 2024-09-16T20:48:45.306 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8997000000000002e-05 2024-09-16T20:48:45.306 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.306 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.306 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:45.306 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250940+0000", 2024-09-16T20:48:45.306 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000119412 2024-09-16T20:48:45.306 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:45.306 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:45.306 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:45.306 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.306 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.306 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:953 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1507328~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:45.306 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.241157+0000", 2024-09-16T20:48:45.306 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 29.974180083, 2024-09-16T20:48:45.306 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.010039259, 2024-09-16T20:48:45.307 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:45.307 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:45.307 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:45.307 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:45.307 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:45.307 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:45.307 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 953 2024-09-16T20:48:45.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.307 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:45.307 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.307 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:45.307 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241157+0000", 2024-09-16T20:48:45.307 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:45.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.308 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.308 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:45.308 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241157+0000", 2024-09-16T20:48:45.308 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:45.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.308 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.308 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:45.308 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241158+0000", 2024-09-16T20:48:45.308 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.01e-07 2024-09-16T20:48:45.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.308 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.308 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:45.308 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241160+0000", 2024-09-16T20:48:45.308 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2879999999999999e-06 2024-09-16T20:48:45.309 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.309 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.309 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:45.309 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241166+0000", 2024-09-16T20:48:45.309 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.421e-06 2024-09-16T20:48:45.309 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.309 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.309 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:45.309 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241171+0000", 2024-09-16T20:48:45.309 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9840000000000001e-06 2024-09-16T20:48:45.309 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.309 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.309 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:45.309 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250988+0000", 2024-09-16T20:48:45.309 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0098171449999999993 2024-09-16T20:48:45.310 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.310 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.310 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:45.310 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.251085+0000", 2024-09-16T20:48:45.310 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6877000000000001e-05 2024-09-16T20:48:45.310 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:45.310 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:45.310 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:45.310 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.251196+0000", 2024-09-16T20:48:45.310 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000111143 2024-09-16T20:48:45.310 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:45.310 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:45.310 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:45.310 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:45.311 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:45.311 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:45.312 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:48:45.312 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:45.311+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:45.413 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:45.412+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:45.431 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:45.431 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:48:45.431 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:48:45.432 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:45.441 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:48:45.514 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:45.513+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:45.613 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:45.613 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:48:45.613 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:48:45.613 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:48:45.613 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:45.616 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:45.614+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:45.617 INFO:teuthology.orchestra.run.smithi139.stdout:1945 trunc from 0x1d8f000 to 0x1019c00 2024-09-16T20:48:45.625 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:48:45.638 INFO:teuthology.orchestra.run.smithi139.stdout:1946 read 0xee5800 thru 0xee7b6c (0x236d bytes) 2024-09-16T20:48:45.640 INFO:teuthology.orchestra.run.smithi139.stdout:1947 read 0x8fa200 thru 0x906f57 (0xcd58 bytes) 2024-09-16T20:48:45.665 INFO:teuthology.orchestra.run.smithi139.stdout:1948 write 0x837b600 thru 0x837d68c (0x208d bytes) 2024-09-16T20:48:45.666 INFO:teuthology.orchestra.run.smithi139.stdout:1949 read 0x7e41a00 thru 0x7e4bec0 (0xa4c1 bytes) 2024-09-16T20:48:45.666 INFO:teuthology.orchestra.run.smithi139.stdout:1950 read 0x707ee00 thru 0x7083851 (0x4a52 bytes) 2024-09-16T20:48:45.675 INFO:teuthology.orchestra.run.smithi139.stdout:1951 trunc from 0x837d800 to 0x24d5600 2024-09-16T20:48:45.699 INFO:teuthology.orchestra.run.smithi139.stdout:1956 write 0x22e8800 thru 0x22e9c37 (0x1438 bytes) 2024-09-16T20:48:45.703 INFO:teuthology.orchestra.run.smithi139.stdout:1957 trunc from 0x24d5600 to 0x3514000 2024-09-16T20:48:45.716 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:45.715+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:45.718 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T20:48:45.718 INFO:teuthology.orchestra.run.smithi139.stdout:1961 write 0xd81c00 thru 0xd8ab79 (0x8f7a bytes) 2024-09-16T20:48:45.750 INFO:teuthology.orchestra.run.smithi139.stdout:1965 write 0xe4b5800 thru 0xe4b5a13 (0x214 bytes) 2024-09-16T20:48:45.752 INFO:teuthology.orchestra.run.smithi139.stdout:1967 flatten 2024-09-16T20:48:45.771 INFO:teuthology.orchestra.run.smithi139.stdout:1969 write 0xa09f800 thru 0xa0abe58 (0xc659 bytes) 2024-09-16T20:48:45.772 INFO:teuthology.orchestra.run.smithi139.stdout:1970 write 0x2a22a00 thru 0x2a319f6 (0xeff7 bytes) 2024-09-16T20:48:45.772 INFO:teuthology.orchestra.run.smithi139.stdout:1971 punch from 0xafa7200 to 0xafa8a00, (0x1800 bytes) 2024-09-16T20:48:45.772 INFO:teuthology.orchestra.run.smithi139.stdout:1973 punch from 0x3f7f600 to 0x3f8a600, (0xb000 bytes) 2024-09-16T20:48:45.773 INFO:teuthology.orchestra.run.smithi139.stdout:1974 write 0x478ac00 thru 0x4798d38 (0xe139 bytes) 2024-09-16T20:48:45.789 INFO:teuthology.orchestra.run.smithi139.stdout:1976 trunc from 0xe4b5c00 to 0xdb0200 2024-09-16T20:48:45.815 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:45.815 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:48:45.815 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:48:45.815 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:48:45.815 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.815 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2167 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 907264~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:45.815 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335615+0000", 2024-09-16T20:48:45.815 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.476773607999998, 2024-09-16T20:48:45.815 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046952761000000003, 2024-09-16T20:48:45.815 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:45.815 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:45.816 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:45.816 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:45.816 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:45.816 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:45.816 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2167 2024-09-16T20:48:45.816 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.816 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:45.816 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.816 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:45.816 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335615+0000", 2024-09-16T20:48:45.816 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:45.816 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.816 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.817 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:45.817 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335615+0000", 2024-09-16T20:48:45.817 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:45.817 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.817 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.817 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:45.817 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335616+0000", 2024-09-16T20:48:45.817 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2300000000000001e-07 2024-09-16T20:48:45.817 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.817 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.817 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:45.817 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335618+0000", 2024-09-16T20:48:45.817 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3269999999999999e-06 2024-09-16T20:48:45.817 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.818 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.818 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:45.818 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335625+0000", 2024-09-16T20:48:45.818 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1930000000000004e-06 2024-09-16T20:48:45.818 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.818 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.818 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:45.818 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335630+0000", 2024-09-16T20:48:45.818 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2460000000000003e-06 2024-09-16T20:48:45.818 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.818 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.818 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:45.818 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.382377+0000", 2024-09-16T20:48:45.819 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046746256 2024-09-16T20:48:45.819 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.819 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.819 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:45.819 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.382474+0000", 2024-09-16T20:48:45.819 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.7388999999999999e-05 2024-09-16T20:48:45.819 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.819 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.819 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:45.819 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.382568+0000", 2024-09-16T20:48:45.819 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.3826999999999995e-05 2024-09-16T20:48:45.819 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:45.819 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:45.819 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:45.820 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.820 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.820 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2170 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 915456~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:45.820 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335637+0000", 2024-09-16T20:48:45.820 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.476751458999999, 2024-09-16T20:48:45.820 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047182843000000002, 2024-09-16T20:48:45.820 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:45.820 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:45.820 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:45.820 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:45.820 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:45.820 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:45.820 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2170 2024-09-16T20:48:45.821 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.821 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:45.821 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.821 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:45.821 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335637+0000", 2024-09-16T20:48:45.821 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:45.821 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.821 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.821 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:45.821 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335637+0000", 2024-09-16T20:48:45.821 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:45.821 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.821 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.821 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:45.822 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335638+0000", 2024-09-16T20:48:45.822 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6699999999999999e-07 2024-09-16T20:48:45.822 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.822 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.822 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:45.822 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335639+0000", 2024-09-16T20:48:45.822 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8339999999999999e-06 2024-09-16T20:48:45.822 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.822 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.822 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:45.822 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335646+0000", 2024-09-16T20:48:45.822 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6599999999999998e-06 2024-09-16T20:48:45.822 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.822 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.822 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:45.823 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335651+0000", 2024-09-16T20:48:45.823 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0289999999999996e-06 2024-09-16T20:48:45.823 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.823 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.823 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:45.823 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.382616+0000", 2024-09-16T20:48:45.823 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046965066 2024-09-16T20:48:45.823 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.823 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.823 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:45.823 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.382708+0000", 2024-09-16T20:48:45.823 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2177999999999997e-05 2024-09-16T20:48:45.824 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.824 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.824 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:45.824 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.382820+0000", 2024-09-16T20:48:45.824 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000111709 2024-09-16T20:48:45.824 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:45.824 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:45.824 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:45.824 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.824 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.824 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2172 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 923648~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:45.824 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335658+0000", 2024-09-16T20:48:45.824 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.476730551999999, 2024-09-16T20:48:45.824 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047394824000000002, 2024-09-16T20:48:45.825 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:45.825 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:45.825 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:45.825 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:45.825 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:45.825 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:45.825 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2172 2024-09-16T20:48:45.825 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.825 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:45.825 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.825 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:45.825 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335658+0000", 2024-09-16T20:48:45.825 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:45.826 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.826 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.826 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:45.826 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335658+0000", 2024-09-16T20:48:45.826 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:45.826 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.826 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.826 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:45.826 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335658+0000", 2024-09-16T20:48:45.826 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8599999999999999e-07 2024-09-16T20:48:45.826 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.826 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.826 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:45.826 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335660+0000", 2024-09-16T20:48:45.827 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.95e-06 2024-09-16T20:48:45.827 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.827 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.827 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:45.827 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335667+0000", 2024-09-16T20:48:45.827 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.815e-06 2024-09-16T20:48:45.827 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.827 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.827 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:45.827 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335672+0000", 2024-09-16T20:48:45.827 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8040000000000001e-06 2024-09-16T20:48:45.827 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.827 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.828 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:45.828 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.382864+0000", 2024-09-16T20:48:45.828 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047192125000000001 2024-09-16T20:48:45.828 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.828 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.828 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:45.828 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.382950+0000", 2024-09-16T20:48:45.828 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5594999999999996e-05 2024-09-16T20:48:45.828 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.828 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.828 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:45.828 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383053+0000", 2024-09-16T20:48:45.828 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000103149 2024-09-16T20:48:45.828 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:45.829 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:45.829 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:45.829 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.829 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.829 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2174 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 931840~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:45.829 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335679+0000", 2024-09-16T20:48:45.829 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.476709678999999, 2024-09-16T20:48:45.829 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047619511000000003, 2024-09-16T20:48:45.830 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:45.830 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:45.830 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:45.830 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:45.830 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:45.830 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:45.830 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2174 2024-09-16T20:48:45.830 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.830 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:45.830 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.830 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:45.830 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335679+0000", 2024-09-16T20:48:45.830 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:45.831 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.831 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.831 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:45.831 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335679+0000", 2024-09-16T20:48:45.831 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:45.831 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.831 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.831 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:45.831 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335679+0000", 2024-09-16T20:48:45.832 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7800000000000002e-07 2024-09-16T20:48:45.832 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.832 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.832 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:45.832 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335682+0000", 2024-09-16T20:48:45.832 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9490000000000001e-06 2024-09-16T20:48:45.832 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.832 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.832 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:45.832 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335689+0000", 2024-09-16T20:48:45.833 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8220000000000001e-06 2024-09-16T20:48:45.833 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.833 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.833 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:45.833 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335695+0000", 2024-09-16T20:48:45.833 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0569999999999998e-06 2024-09-16T20:48:45.833 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.833 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.833 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:45.833 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383103+0000", 2024-09-16T20:48:45.834 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047408219000000001 2024-09-16T20:48:45.834 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.834 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.834 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:45.834 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383189+0000", 2024-09-16T20:48:45.834 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6007000000000005e-05 2024-09-16T20:48:45.834 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.834 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.834 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:45.834 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383298+0000", 2024-09-16T20:48:45.834 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000109079 2024-09-16T20:48:45.835 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:45.835 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:45.835 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:45.835 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.835 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.835 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2176 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 940032~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:45.835 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335702+0000", 2024-09-16T20:48:45.835 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.476686450999999, 2024-09-16T20:48:45.835 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047804450999999998, 2024-09-16T20:48:45.835 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:45.835 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:45.836 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:45.836 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:45.836 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:45.836 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:45.836 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2176 2024-09-16T20:48:45.836 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.836 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:45.836 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.836 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:45.837 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335702+0000", 2024-09-16T20:48:45.837 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:45.837 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.837 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.837 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:45.837 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335702+0000", 2024-09-16T20:48:45.837 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:45.837 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.837 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.837 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:45.838 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335703+0000", 2024-09-16T20:48:45.838 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.34e-07 2024-09-16T20:48:45.838 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.838 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.838 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:45.838 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335704+0000", 2024-09-16T20:48:45.838 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.917e-06 2024-09-16T20:48:45.838 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.838 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.838 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:45.838 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335711+0000", 2024-09-16T20:48:45.839 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4400000000000002e-06 2024-09-16T20:48:45.839 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.839 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.839 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:45.839 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335717+0000", 2024-09-16T20:48:45.839 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6679999999999998e-06 2024-09-16T20:48:45.839 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.839 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.839 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:45.839 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383335+0000", 2024-09-16T20:48:45.840 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047618754999999999 2024-09-16T20:48:45.840 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.840 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.840 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:45.840 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383405+0000", 2024-09-16T20:48:45.840 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9286e-05 2024-09-16T20:48:45.840 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.840 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.841 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:45.841 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383507+0000", 2024-09-16T20:48:45.841 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101951 2024-09-16T20:48:45.841 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:45.841 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:45.841 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:45.841 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.842 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.842 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2179 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 948224~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:45.842 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335723+0000", 2024-09-16T20:48:45.842 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.476665357000002, 2024-09-16T20:48:45.842 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048007181000000003, 2024-09-16T20:48:45.842 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:45.842 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:45.843 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:45.843 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:45.843 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:45.843 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:45.843 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2179 2024-09-16T20:48:45.843 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.843 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:45.843 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.843 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:45.843 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335723+0000", 2024-09-16T20:48:45.844 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:45.844 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.844 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.844 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:45.844 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335723+0000", 2024-09-16T20:48:45.844 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:45.844 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.844 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.844 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:45.844 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335724+0000", 2024-09-16T20:48:45.844 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.58e-07 2024-09-16T20:48:45.844 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.844 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.845 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:45.845 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335726+0000", 2024-09-16T20:48:45.845 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9120000000000001e-06 2024-09-16T20:48:45.845 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.845 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.845 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:45.845 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335732+0000", 2024-09-16T20:48:45.845 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6080000000000001e-06 2024-09-16T20:48:45.845 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.845 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.845 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:45.845 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335737+0000", 2024-09-16T20:48:45.845 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.22e-06 2024-09-16T20:48:45.845 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.845 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.846 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:45.846 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383556+0000", 2024-09-16T20:48:45.846 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047819002999999999 2024-09-16T20:48:45.846 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.846 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.846 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:45.846 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383635+0000", 2024-09-16T20:48:45.846 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9060999999999994e-05 2024-09-16T20:48:45.846 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.846 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.846 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:45.846 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383730+0000", 2024-09-16T20:48:45.846 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4919000000000004e-05 2024-09-16T20:48:45.846 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:45.847 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:45.847 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:45.847 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.847 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.847 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2180 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 956416~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:45.847 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335744+0000", 2024-09-16T20:48:45.847 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.476644909000001, 2024-09-16T20:48:45.847 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048255147999999998, 2024-09-16T20:48:45.847 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:45.847 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:45.847 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:45.847 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:45.847 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:45.848 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:45.848 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2180 2024-09-16T20:48:45.848 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.848 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:45.848 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.848 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:45.848 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335744+0000", 2024-09-16T20:48:45.848 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:45.848 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.848 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.848 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:45.848 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335744+0000", 2024-09-16T20:48:45.848 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:45.849 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.849 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.849 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:45.849 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335744+0000", 2024-09-16T20:48:45.849 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9000000000000002e-07 2024-09-16T20:48:45.849 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.849 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.849 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:45.849 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335746+0000", 2024-09-16T20:48:45.849 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7570000000000001e-06 2024-09-16T20:48:45.849 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.849 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.849 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:45.850 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335752+0000", 2024-09-16T20:48:45.850 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0079999999999999e-06 2024-09-16T20:48:45.850 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.850 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.850 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:45.850 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335757+0000", 2024-09-16T20:48:45.850 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1649999999999997e-06 2024-09-16T20:48:45.850 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.850 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.850 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:45.850 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383803+0000", 2024-09-16T20:48:45.850 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048046091999999999 2024-09-16T20:48:45.850 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.850 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.851 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:45.851 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383889+0000", 2024-09-16T20:48:45.851 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6048e-05 2024-09-16T20:48:45.851 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.851 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.851 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:45.851 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383999+0000", 2024-09-16T20:48:45.851 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000109688 2024-09-16T20:48:45.851 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:45.851 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:45.851 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:45.851 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.851 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.851 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2182 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 964608~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:45.852 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335773+0000", 2024-09-16T20:48:45.852 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.476615102, 2024-09-16T20:48:45.852 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048467116999999997, 2024-09-16T20:48:45.852 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:45.852 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:45.852 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:45.852 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:45.852 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:45.852 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:45.852 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2182 2024-09-16T20:48:45.852 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.852 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:45.852 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.852 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:45.853 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335773+0000", 2024-09-16T20:48:45.853 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:45.853 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.853 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.853 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:45.853 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335773+0000", 2024-09-16T20:48:45.853 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:45.853 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.853 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.853 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:45.853 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335774+0000", 2024-09-16T20:48:45.853 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3799999999999997e-07 2024-09-16T20:48:45.853 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.853 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.854 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:45.854 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335776+0000", 2024-09-16T20:48:45.854 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.227e-06 2024-09-16T20:48:45.854 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.854 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.854 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:45.854 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335790+0000", 2024-09-16T20:48:45.854 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4119e-05 2024-09-16T20:48:45.854 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.854 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.854 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:45.854 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335796+0000", 2024-09-16T20:48:45.854 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.694e-06 2024-09-16T20:48:45.855 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.855 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.855 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:45.855 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384058+0000", 2024-09-16T20:48:45.855 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048261983000000001 2024-09-16T20:48:45.855 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.855 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.855 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:45.855 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384148+0000", 2024-09-16T20:48:45.855 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9501999999999998e-05 2024-09-16T20:48:45.855 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.855 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.855 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:45.855 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384241+0000", 2024-09-16T20:48:45.856 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.3053999999999994e-05 2024-09-16T20:48:45.856 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:45.856 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:45.856 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:45.856 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.856 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.856 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2185 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 972800~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:45.856 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335804+0000", 2024-09-16T20:48:45.856 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.476584452000001, 2024-09-16T20:48:45.856 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048570818000000002, 2024-09-16T20:48:45.856 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:45.856 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:45.856 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:45.857 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:45.857 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:45.857 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:45.860 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2185 2024-09-16T20:48:45.860 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.860 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:45.860 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.860 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:45.860 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335804+0000", 2024-09-16T20:48:45.861 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:45.861 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.861 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.861 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:45.861 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335804+0000", 2024-09-16T20:48:45.861 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:45.861 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.861 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.861 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:45.861 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335805+0000", 2024-09-16T20:48:45.861 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8299999999999997e-07 2024-09-16T20:48:45.861 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.861 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.861 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:45.862 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335813+0000", 2024-09-16T20:48:45.862 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6309999999999992e-06 2024-09-16T20:48:45.862 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.862 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.862 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:45.862 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335821+0000", 2024-09-16T20:48:45.862 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0169999999999996e-06 2024-09-16T20:48:45.862 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.862 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.862 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:45.862 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335827+0000", 2024-09-16T20:48:45.862 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0100000000000001e-06 2024-09-16T20:48:45.862 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.863 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.863 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:45.863 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384273+0000", 2024-09-16T20:48:45.863 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048445333 2024-09-16T20:48:45.863 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.863 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.863 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:45.863 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384315+0000", 2024-09-16T20:48:45.864 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2472999999999998e-05 2024-09-16T20:48:45.864 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.864 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.864 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:45.864 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384375+0000", 2024-09-16T20:48:45.864 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9871000000000003e-05 2024-09-16T20:48:45.864 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:45.864 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:45.864 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:45.864 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.864 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.864 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2187 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 980992~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:45.865 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335835+0000", 2024-09-16T20:48:45.865 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.476553375000002, 2024-09-16T20:48:45.865 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048684927000000003, 2024-09-16T20:48:45.865 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:45.865 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:45.865 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:45.865 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:45.865 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:45.865 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:45.865 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2187 2024-09-16T20:48:45.865 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.865 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:45.865 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.866 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:45.866 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335835+0000", 2024-09-16T20:48:45.866 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:45.866 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.866 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.866 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:45.866 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335835+0000", 2024-09-16T20:48:45.866 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:45.866 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.866 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.866 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:45.866 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335836+0000", 2024-09-16T20:48:45.866 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3599999999999999e-07 2024-09-16T20:48:45.866 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.867 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.867 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:45.867 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335838+0000", 2024-09-16T20:48:45.867 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.996e-06 2024-09-16T20:48:45.867 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.867 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.867 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:45.867 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335844+0000", 2024-09-16T20:48:45.867 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7889999999999997e-06 2024-09-16T20:48:45.867 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.868 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.868 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:45.868 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335849+0000", 2024-09-16T20:48:45.868 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7400000000000004e-06 2024-09-16T20:48:45.868 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.868 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.868 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:45.868 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384411+0000", 2024-09-16T20:48:45.868 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048562069999999999 2024-09-16T20:48:45.868 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.868 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.869 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:45.869 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384460+0000", 2024-09-16T20:48:45.869 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9125999999999997e-05 2024-09-16T20:48:45.869 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.869 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.869 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:45.869 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384520+0000", 2024-09-16T20:48:45.869 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9769999999999999e-05 2024-09-16T20:48:45.869 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:45.869 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:45.869 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:45.869 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.869 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.870 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2189 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 989184~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:45.870 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335856+0000", 2024-09-16T20:48:45.870 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.476532129999999, 2024-09-16T20:48:45.870 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048856525999999997, 2024-09-16T20:48:45.870 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:45.870 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:45.870 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:45.871 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:45.871 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:45.871 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:45.871 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2189 2024-09-16T20:48:45.871 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.871 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:45.871 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.871 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:45.871 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335856+0000", 2024-09-16T20:48:45.871 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:45.871 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.871 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.871 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:45.871 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335856+0000", 2024-09-16T20:48:45.872 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:45.872 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.872 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.872 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:45.872 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335857+0000", 2024-09-16T20:48:45.872 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6800000000000001e-07 2024-09-16T20:48:45.872 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.872 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.873 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:45.873 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335859+0000", 2024-09-16T20:48:45.873 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.013e-06 2024-09-16T20:48:45.873 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.873 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.873 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:45.873 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335866+0000", 2024-09-16T20:48:45.873 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8750000000000002e-06 2024-09-16T20:48:45.873 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.873 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.873 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:45.873 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335871+0000", 2024-09-16T20:48:45.873 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1719999999999998e-06 2024-09-16T20:48:45.873 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.874 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.874 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:45.874 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384550+0000", 2024-09-16T20:48:45.874 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048679476999999999 2024-09-16T20:48:45.874 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.874 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.874 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:45.874 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384611+0000", 2024-09-16T20:48:45.874 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0897000000000003e-05 2024-09-16T20:48:45.874 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.874 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.874 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:45.874 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384713+0000", 2024-09-16T20:48:45.874 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101724 2024-09-16T20:48:45.875 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:45.875 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:45.875 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:45.875 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.875 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.875 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2192 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 997376~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:45.875 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335878+0000", 2024-09-16T20:48:45.875 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.476510444999999, 2024-09-16T20:48:45.875 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049028769999999999, 2024-09-16T20:48:45.875 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:45.875 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:45.875 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:45.875 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:45.875 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:45.876 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:45.876 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2192 2024-09-16T20:48:45.876 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.876 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:45.876 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.876 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:45.876 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335878+0000", 2024-09-16T20:48:45.876 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:45.876 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.876 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.876 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:45.876 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335878+0000", 2024-09-16T20:48:45.876 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:45.876 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.877 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.877 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:45.877 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335879+0000", 2024-09-16T20:48:45.877 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6899999999999998e-07 2024-09-16T20:48:45.877 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.877 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.877 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:45.877 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335881+0000", 2024-09-16T20:48:45.877 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.996e-06 2024-09-16T20:48:45.877 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.877 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.877 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:45.877 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335887+0000", 2024-09-16T20:48:45.877 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6370000000000002e-06 2024-09-16T20:48:45.878 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.878 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.878 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:45.878 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335892+0000", 2024-09-16T20:48:45.878 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2120000000000002e-06 2024-09-16T20:48:45.878 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.878 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.878 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:45.878 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384747+0000", 2024-09-16T20:48:45.878 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048854923000000001 2024-09-16T20:48:45.878 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.879 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.879 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:45.879 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384830+0000", 2024-09-16T20:48:45.879 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2359000000000002e-05 2024-09-16T20:48:45.879 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.879 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.879 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:45.879 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384907+0000", 2024-09-16T20:48:45.879 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7174000000000003e-05 2024-09-16T20:48:45.879 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:45.880 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:45.880 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:45.880 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.880 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.880 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2194 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1005568~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:45.880 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335899+0000", 2024-09-16T20:48:45.880 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.476489524000002, 2024-09-16T20:48:45.880 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049225393999999999, 2024-09-16T20:48:45.881 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:45.881 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:45.881 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:45.881 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:45.881 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:45.881 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:45.881 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2194 2024-09-16T20:48:45.881 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.882 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:45.882 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.882 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:45.882 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335899+0000", 2024-09-16T20:48:45.882 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:45.882 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.882 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.882 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:45.882 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335899+0000", 2024-09-16T20:48:45.882 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:45.882 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.883 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.883 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:45.883 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335899+0000", 2024-09-16T20:48:45.883 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8200000000000001e-07 2024-09-16T20:48:45.883 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.883 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.883 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:45.883 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335901+0000", 2024-09-16T20:48:45.884 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9e-06 2024-09-16T20:48:45.884 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.884 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.884 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:45.884 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335908+0000", 2024-09-16T20:48:45.884 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3559999999999998e-06 2024-09-16T20:48:45.884 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.885 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.885 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:45.885 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335914+0000", 2024-09-16T20:48:45.885 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4729999999999997e-06 2024-09-16T20:48:45.885 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.885 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.885 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:45.885 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384979+0000", 2024-09-16T20:48:45.885 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049064545000000001 2024-09-16T20:48:45.885 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.885 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.886 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:45.886 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385060+0000", 2024-09-16T20:48:45.886 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0804999999999996e-05 2024-09-16T20:48:45.886 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.886 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.886 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:45.886 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385124+0000", 2024-09-16T20:48:45.886 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4932999999999996e-05 2024-09-16T20:48:45.886 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:45.886 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:45.886 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:45.886 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.886 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.886 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2197 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1013760~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:45.887 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335921+0000", 2024-09-16T20:48:45.887 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.476467242999998, 2024-09-16T20:48:45.887 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049342184999999997, 2024-09-16T20:48:45.887 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:45.887 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:45.887 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:45.887 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:45.887 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:45.887 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:45.887 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2197 2024-09-16T20:48:45.887 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.887 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:45.887 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.887 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:45.888 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335921+0000", 2024-09-16T20:48:45.888 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:45.888 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.888 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.888 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:45.888 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335921+0000", 2024-09-16T20:48:45.888 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:45.888 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.888 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.888 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:45.888 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335922+0000", 2024-09-16T20:48:45.888 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.32e-07 2024-09-16T20:48:45.888 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.889 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.889 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:45.889 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335924+0000", 2024-09-16T20:48:45.889 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.995e-06 2024-09-16T20:48:45.889 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.889 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.889 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:45.889 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335930+0000", 2024-09-16T20:48:45.889 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5139999999999998e-06 2024-09-16T20:48:45.889 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.889 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.889 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:45.889 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335936+0000", 2024-09-16T20:48:45.890 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6160000000000001e-06 2024-09-16T20:48:45.890 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.890 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.890 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:45.890 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385142+0000", 2024-09-16T20:48:45.890 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049206393000000001 2024-09-16T20:48:45.890 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.890 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.890 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:45.890 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385184+0000", 2024-09-16T20:48:45.890 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1928999999999998e-05 2024-09-16T20:48:45.890 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.890 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.891 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:45.891 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385264+0000", 2024-09-16T20:48:45.891 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9306000000000003e-05 2024-09-16T20:48:45.891 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:45.891 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:45.891 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:45.891 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.891 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.891 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2198 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1021952~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:45.891 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335942+0000", 2024-09-16T20:48:45.891 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.476446183, 2024-09-16T20:48:45.891 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049562483999999997, 2024-09-16T20:48:45.891 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:45.891 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:45.892 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:45.892 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:45.892 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:45.892 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:45.892 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2198 2024-09-16T20:48:45.892 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.892 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:45.892 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.892 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:45.892 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335942+0000", 2024-09-16T20:48:45.892 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:45.892 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.892 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.893 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:45.893 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335942+0000", 2024-09-16T20:48:45.893 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:45.893 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.893 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.893 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:45.893 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335943+0000", 2024-09-16T20:48:45.893 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7099999999999997e-07 2024-09-16T20:48:45.893 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.893 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.893 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:45.893 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335945+0000", 2024-09-16T20:48:45.893 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3499999999999999e-06 2024-09-16T20:48:45.894 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.894 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.894 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:45.894 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335952+0000", 2024-09-16T20:48:45.894 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4309999999999999e-06 2024-09-16T20:48:45.894 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.894 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.894 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:45.894 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335957+0000", 2024-09-16T20:48:45.894 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0869999999999997e-06 2024-09-16T20:48:45.894 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.894 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.894 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:45.894 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385312+0000", 2024-09-16T20:48:45.894 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049355121000000002 2024-09-16T20:48:45.895 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.895 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.895 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:45.895 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385405+0000", 2024-09-16T20:48:45.895 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.3344000000000006e-05 2024-09-16T20:48:45.895 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.895 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.895 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:45.895 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385505+0000", 2024-09-16T20:48:45.895 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.9779999999999997e-05 2024-09-16T20:48:45.895 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:45.895 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:45.895 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:45.895 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.896 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.896 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2201 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1030144~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:45.896 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335976+0000", 2024-09-16T20:48:45.896 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.476412741000001, 2024-09-16T20:48:45.896 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049740077000000001, 2024-09-16T20:48:45.896 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:45.896 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:45.896 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:45.896 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:45.896 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:45.896 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:45.896 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2201 2024-09-16T20:48:45.896 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.896 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:45.897 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.897 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:45.897 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335976+0000", 2024-09-16T20:48:45.897 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:45.897 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.897 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.897 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:45.897 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335976+0000", 2024-09-16T20:48:45.897 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:45.897 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.897 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.897 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:45.897 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335976+0000", 2024-09-16T20:48:45.898 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8599999999999998e-07 2024-09-16T20:48:45.898 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.898 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.898 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:45.898 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335978+0000", 2024-09-16T20:48:45.898 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1500000000000002e-06 2024-09-16T20:48:45.898 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.898 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.898 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:45.898 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335986+0000", 2024-09-16T20:48:45.898 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3989999999999998e-06 2024-09-16T20:48:45.898 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.898 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.899 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:45.899 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335992+0000", 2024-09-16T20:48:45.899 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.682e-06 2024-09-16T20:48:45.899 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.899 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.899 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:45.899 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385537+0000", 2024-09-16T20:48:45.899 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049545828 2024-09-16T20:48:45.899 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.899 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.899 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:45.899 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385619+0000", 2024-09-16T20:48:45.899 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1720999999999999e-05 2024-09-16T20:48:45.900 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.900 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.900 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:45.900 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385716+0000", 2024-09-16T20:48:45.900 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.6811000000000005e-05 2024-09-16T20:48:45.900 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:45.900 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:45.900 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:45.900 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.900 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.900 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2203 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1038336~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:45.900 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.336000+0000", 2024-09-16T20:48:45.901 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.476388955000001, 2024-09-16T20:48:45.901 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049949888999999997, 2024-09-16T20:48:45.901 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:45.901 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:45.901 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:45.901 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:45.901 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:45.901 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:45.901 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2203 2024-09-16T20:48:45.901 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.901 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:45.901 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.901 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:45.902 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336000+0000", 2024-09-16T20:48:45.902 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:45.902 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.902 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.902 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:45.902 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336000+0000", 2024-09-16T20:48:45.902 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:45.902 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.902 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.902 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:45.902 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336000+0000", 2024-09-16T20:48:45.902 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.58e-07 2024-09-16T20:48:45.902 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.902 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.903 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:45.903 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336002+0000", 2024-09-16T20:48:45.903 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0040000000000002e-06 2024-09-16T20:48:45.903 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.903 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.903 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:45.903 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336009+0000", 2024-09-16T20:48:45.903 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9890000000000003e-06 2024-09-16T20:48:45.903 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.903 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.903 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:45.903 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336014+0000", 2024-09-16T20:48:45.903 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4e-06 2024-09-16T20:48:45.904 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.904 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.904 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:45.904 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385759+0000", 2024-09-16T20:48:45.904 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049744151 2024-09-16T20:48:45.904 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.904 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.904 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:45.904 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385847+0000", 2024-09-16T20:48:45.904 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8189000000000006e-05 2024-09-16T20:48:45.904 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.904 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.904 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:45.905 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385950+0000", 2024-09-16T20:48:45.905 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102698 2024-09-16T20:48:45.905 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:45.905 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:45.905 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:45.905 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.905 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.905 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2206 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1046528~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:45.905 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:48:45.905 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.476366794, 2024-09-16T20:48:45.905 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050147714000000003, 2024-09-16T20:48:45.906 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:45.906 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:45.906 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:45.906 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:45.906 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:45.906 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:45.906 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2206 2024-09-16T20:48:45.906 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.907 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:45.907 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.907 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:45.907 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:48:45.907 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:45.907 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.907 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.907 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:45.907 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:48:45.907 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:45.907 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.908 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.908 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:45.908 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:48:45.908 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8599999999999999e-07 2024-09-16T20:48:45.908 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.908 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.908 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:45.909 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336024+0000", 2024-09-16T20:48:45.909 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.934e-06 2024-09-16T20:48:45.909 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.909 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.909 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:45.909 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336037+0000", 2024-09-16T20:48:45.909 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2496e-05 2024-09-16T20:48:45.909 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.909 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.909 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:45.909 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336042+0000", 2024-09-16T20:48:45.909 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4809999999999997e-06 2024-09-16T20:48:45.910 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.910 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.910 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:45.910 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385980+0000", 2024-09-16T20:48:45.910 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049938376999999999 2024-09-16T20:48:45.910 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.911 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.911 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:45.911 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.386069+0000", 2024-09-16T20:48:45.911 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8368999999999994e-05 2024-09-16T20:48:45.911 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.911 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.911 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:45.911 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.386170+0000", 2024-09-16T20:48:45.911 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000100671 2024-09-16T20:48:45.912 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:45.912 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:45.912 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:45.912 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.912 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.912 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5439 3.a 3:5789f5c3:::rbd_data.110ed9ca540.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 1181696~60416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:45.912 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:48:45.912 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 22.947678214, 2024-09-16T20:48:45.913 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10102045599999999, 2024-09-16T20:48:45.913 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:45.913 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:45.913 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:45.913 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:45.914 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:45.914 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:45.914 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5439 2024-09-16T20:48:45.914 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.914 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:45.914 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.915 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:45.915 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:48:45.915 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:45.915 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.915 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.915 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:45.916 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:48:45.916 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:45.916 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.916 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.916 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:45.916 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864711+0000", 2024-09-16T20:48:45.917 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7899999999999997e-07 2024-09-16T20:48:45.917 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.917 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.917 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:45.917 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864760+0000", 2024-09-16T20:48:45.918 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8803e-05 2024-09-16T20:48:45.918 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.918 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.918 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:45.918 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864769+0000", 2024-09-16T20:48:45.918 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0999999999999993e-06 2024-09-16T20:48:45.918 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.918 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.918 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:45.918 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864774+0000", 2024-09-16T20:48:45.918 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8919999999999999e-06 2024-09-16T20:48:45.918 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.918 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.919 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:45.919 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864815+0000", 2024-09-16T20:48:45.919 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1307000000000003e-05 2024-09-16T20:48:45.919 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.919 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.919 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:45.919 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864893+0000", 2024-09-16T20:48:45.919 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7581000000000001e-05 2024-09-16T20:48:45.919 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.919 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.919 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 1", 2024-09-16T20:48:45.919 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864996+0000", 2024-09-16T20:48:45.920 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102725 2024-09-16T20:48:45.920 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.920 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.920 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:45.920 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965442+0000", 2024-09-16T20:48:45.920 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100446012 2024-09-16T20:48:45.920 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.920 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.920 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:45.920 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965618+0000", 2024-09-16T20:48:45.920 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00017653100000000001 2024-09-16T20:48:45.920 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.920 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.921 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:45.921 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965682+0000", 2024-09-16T20:48:45.921 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3905999999999994e-05 2024-09-16T20:48:45.921 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.921 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.921 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:45.921 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965731+0000", 2024-09-16T20:48:45.921 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8819999999999997e-05 2024-09-16T20:48:45.921 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:45.921 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:45.921 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:45.921 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.921 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.921 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4600.0:5439 3.a e37/23 ondisk, result = 0)", 2024-09-16T20:48:45.921 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:48:45.922 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 22.946682066000001, 2024-09-16T20:48:45.922 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099742634999999996, 2024-09-16T20:48:45.922 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:45.922 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:45.922 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:45.922 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:45.922 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.922 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:45.922 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:48:45.922 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:45.922 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.922 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.922 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:45.922 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:48:45.923 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:45.923 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.923 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.923 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:45.923 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:48:45.923 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0900000000000001e-07 2024-09-16T20:48:45.923 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.923 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.923 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:45.923 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865711+0000", 2024-09-16T20:48:45.923 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3739999999999998e-06 2024-09-16T20:48:45.923 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.923 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.923 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:45.923 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965343+0000", 2024-09-16T20:48:45.923 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099632526999999999 2024-09-16T20:48:45.924 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.924 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.924 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:45.924 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965357+0000", 2024-09-16T20:48:45.924 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3713999999999999e-05 2024-09-16T20:48:45.924 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.924 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.924 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:45.924 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965426+0000", 2024-09-16T20:48:45.924 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8745999999999995e-05 2024-09-16T20:48:45.924 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.925 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.925 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:45.925 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965434+0000", 2024-09-16T20:48:45.925 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6910000000000003e-06 2024-09-16T20:48:45.925 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:45.925 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:45.925 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:45.925 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965449+0000", 2024-09-16T20:48:45.925 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4874e-05 2024-09-16T20:48:45.925 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:45.925 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:45.925 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:45.925 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:45.925 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:45.925 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:45.928 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:48:45.928 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:45.925+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:45.930 INFO:teuthology.orchestra.run.smithi139.stdout:1977 read 0x2a6200 thru 0x2aef4a (0x8d4b bytes) 2024-09-16T20:48:45.930 INFO:teuthology.orchestra.run.smithi139.stdout:1978 write 0x4d7d200 thru 0x4d859ae (0x87af bytes) 2024-09-16T20:48:45.930 INFO:teuthology.orchestra.run.smithi139.stdout:1980 read 0x411e800 thru 0x41287a1 (0x9fa2 bytes) 2024-09-16T20:48:45.930 INFO:teuthology.orchestra.run.smithi139.stdout:1984 read 0x1db3e00 thru 0x1dbc48f (0x8690 bytes) 2024-09-16T20:48:45.930 INFO:teuthology.orchestra.run.smithi139.stdout:1986 trunc from 0x4d85a00 to 0x1da0c00 2024-09-16T20:48:45.931 INFO:teuthology.orchestra.run.smithi139.stdout:1987 punch from 0x11d9800 to 0x11e2c00, (0x9400 bytes) 2024-09-16T20:48:45.931 INFO:teuthology.orchestra.run.smithi139.stdout:1988 read 0xa06000 thru 0xa14ed1 (0xeed2 bytes) 2024-09-16T20:48:45.931 INFO:teuthology.orchestra.run.smithi139.stdout:1990 read 0xa400 thru 0x1999a (0xf59b bytes) 2024-09-16T20:48:45.931 INFO:teuthology.orchestra.run.smithi139.stdout:1991 read 0xa84400 thru 0xa91c4b (0xd84c bytes) 2024-09-16T20:48:45.931 INFO:teuthology.orchestra.run.smithi139.stdout:1992 read 0x112a00 thru 0x11dd84 (0xb385 bytes) 2024-09-16T20:48:45.931 INFO:teuthology.orchestra.run.smithi139.stdout:1993 punch from 0x11dec00 to 0x11eb400, (0xc800 bytes) 2024-09-16T20:48:45.931 INFO:teuthology.orchestra.run.smithi139.stdout:1994 read 0x10b7000 thru 0x10c4026 (0xd027 bytes) 2024-09-16T20:48:45.931 INFO:teuthology.orchestra.run.smithi139.stdout:1997 write 0x460b800 thru 0x4611b56 (0x6357 bytes) 2024-09-16T20:48:45.931 INFO:teuthology.orchestra.run.smithi139.stdout:1998 punch from 0x261aa00 to 0x261ae00, (0x400 bytes) 2024-09-16T20:48:45.931 INFO:teuthology.orchestra.run.smithi139.stdout:1999 write 0x7afdc00 thru 0x7b04416 (0x6817 bytes) 2024-09-16T20:48:45.931 INFO:teuthology.orchestra.run.smithi139.stdout:2001 read 0xbb2a00 thru 0xbb7611 (0x4c12 bytes) 2024-09-16T20:48:45.931 INFO:teuthology.orchestra.run.smithi139.stdout:2002 read 0x3a2e600 thru 0x3a31ebd (0x38be bytes) 2024-09-16T20:48:45.932 INFO:teuthology.orchestra.run.smithi139.stdout:2003 read 0x5279200 thru 0x527fecf (0x6cd0 bytes) 2024-09-16T20:48:45.932 INFO:teuthology.orchestra.run.smithi139.stdout:2004 read 0x1d8da00 thru 0x1d924bf (0x4ac0 bytes) 2024-09-16T20:48:45.932 INFO:teuthology.orchestra.run.smithi139.stdout:2006 write 0xa3c7000 thru 0xa3cab9c (0x3b9d bytes) 2024-09-16T20:48:45.932 INFO:teuthology.orchestra.run.smithi139.stdout:2007 read 0x4aa9e00 thru 0x4aab8f1 (0x1af2 bytes) 2024-09-16T20:48:45.932 INFO:teuthology.orchestra.run.smithi139.stdout:2010 read 0x1f10a00 thru 0x1f157cf (0x4dd0 bytes) 2024-09-16T20:48:45.932 INFO:teuthology.orchestra.run.smithi139.stdout:2012 trunc from 0xa3cac00 to 0xd4c6600 2024-09-16T20:48:45.940 INFO:teuthology.orchestra.run.smithi139.stdout:2013 punch from 0xa4e3c00 to 0xa4e9a00, (0x5e00 bytes) 2024-09-16T20:48:45.941 INFO:teuthology.orchestra.run.smithi139.stdout:2014 read 0x6524a00 thru 0x652b98e (0x6f8f bytes) 2024-09-16T20:48:45.942 INFO:teuthology.orchestra.run.smithi139.stdout:2015 read 0xa885c00 thru 0xa88611b (0x51c bytes) 2024-09-16T20:48:45.956 INFO:teuthology.orchestra.run.smithi139.stdout:2017 trunc from 0xd4c6600 to 0x1067200 2024-09-16T20:48:45.976 INFO:teuthology.orchestra.run.smithi139.stdout:2018 read 0x3eda00 thru 0x3fc148 (0xe749 bytes) 2024-09-16T20:48:45.986 INFO:teuthology.orchestra.run.smithi139.stdout:2019 trunc from 0x1067200 to 0x6c59400 2024-09-16T20:48:45.998 INFO:teuthology.orchestra.run.smithi139.stdout:2023 read 0x4616e00 thru 0x4618020 (0x1221 bytes) 2024-09-16T20:48:45.999 INFO:teuthology.orchestra.run.smithi139.stdout:2024 write 0x30d0000 thru 0x30d3ec7 (0x3ec8 bytes) 2024-09-16T20:48:45.999 INFO:teuthology.orchestra.run.smithi139.stdout:2025 write 0x49bec00 thru 0x49cc268 (0xd669 bytes) 2024-09-16T20:48:46.001 INFO:teuthology.orchestra.run.smithi139.stdout:2026 trunc from 0x6c59400 to 0x8507400 2024-09-16T20:48:46.018 INFO:teuthology.orchestra.run.smithi139.stdout:2027 read 0x3037200 thru 0x303e832 (0x7633 bytes) 2024-09-16T20:48:46.020 INFO:teuthology.orchestra.run.smithi139.stdout:2028 read 0xf8f600 thru 0xf9ae02 (0xb803 bytes) 2024-09-16T20:48:46.020 INFO:teuthology.orchestra.run.smithi139.stdout:2029 write 0x1ebba00 thru 0x1ec5475 (0x9a76 bytes) 2024-09-16T20:48:46.021 INFO:teuthology.orchestra.run.smithi139.stdout:2030 read 0x24dc200 thru 0x24e065d (0x445e bytes) 2024-09-16T20:48:46.021 INFO:teuthology.orchestra.run.smithi139.stdout:2031 write 0x6dfaa00 thru 0x6e09b14 (0xf115 bytes) 2024-09-16T20:48:46.023 INFO:teuthology.orchestra.run.smithi139.stdout:2033 trunc from 0x8507400 to 0x99f7600 2024-09-16T20:48:46.027 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:46.025+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:46.050 INFO:teuthology.orchestra.run.smithi139.stdout:2034 write 0xb507000 thru 0xb50ff32 (0x8f33 bytes) 2024-09-16T20:48:46.052 INFO:teuthology.orchestra.run.smithi139.stdout:2035 read 0x2f23c00 thru 0x2f28396 (0x4797 bytes) 2024-09-16T20:48:46.062 INFO:teuthology.orchestra.run.smithi139.stdout:2036 trunc from 0xb510000 to 0x255ea00 2024-09-16T20:48:46.085 INFO:teuthology.orchestra.run.smithi139.stdout:2037 punch from 0x28ac00 to 0x28c800, (0x1c00 bytes) 2024-09-16T20:48:46.086 INFO:teuthology.orchestra.run.smithi139.stdout:2040 read 0xda6200 thru 0xda6c96 (0xa97 bytes) 2024-09-16T20:48:46.087 INFO:teuthology.orchestra.run.smithi139.stdout:2043 punch from 0x104e00 to 0x10cc00, (0x7e00 bytes) 2024-09-16T20:48:46.087 INFO:teuthology.orchestra.run.smithi139.stdout:2045 punch from 0x18b8c00 to 0x18c7800, (0xec00 bytes) 2024-09-16T20:48:46.089 INFO:teuthology.orchestra.run.smithi139.stdout:2047 trunc from 0x255ea00 to 0x3d7e600 2024-09-16T20:48:46.098 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:46.098 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:48:46.098 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.098 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:7320 3.1a 3:5bdb8ab9:::rbd_header.110e7c8851c:head [call rbd.get_size in=19b out=9b,call rbd.get_features in=24b out=16b,call rbd.get_flags in=20b out=8b,call rbd.get_snapcontext in=18b,call lock.get_info in=30b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T20:48:46.098 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.097198+0000", 2024-09-16T20:48:46.098 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.000467474, 2024-09-16T20:48:46.098 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00048772399999999998, 2024-09-16T20:48:46.098 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:46.098 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:46.098 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:46.099 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:46.099 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:48:46.099 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:48:46.099 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 7320 2024-09-16T20:48:46.099 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.099 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:46.099 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.099 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:46.099 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.097198+0000", 2024-09-16T20:48:46.099 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.099 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.099 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.099 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:46.100 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.097198+0000", 2024-09-16T20:48:46.100 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.100 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.100 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.100 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:46.100 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.097198+0000", 2024-09-16T20:48:46.100 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8899999999999999e-07 2024-09-16T20:48:46.100 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.100 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.100 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:46.100 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.097202+0000", 2024-09-16T20:48:46.101 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5020000000000001e-06 2024-09-16T20:48:46.101 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.101 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.101 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:46.101 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.097209+0000", 2024-09-16T20:48:46.101 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8229999999999997e-06 2024-09-16T20:48:46.101 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.101 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.101 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:46.101 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.097212+0000", 2024-09-16T20:48:46.101 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4649999999999999e-06 2024-09-16T20:48:46.101 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.101 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.102 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:46.102 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.097508+0000", 2024-09-16T20:48:46.102 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000296507 2024-09-16T20:48:46.102 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.102 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.102 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:46.102 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.097554+0000", 2024-09-16T20:48:46.102 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5958000000000003e-05 2024-09-16T20:48:46.102 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.102 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:46.102 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.102 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.102 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.102 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4882.0:1680 3.1a 3.9d51dbda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T20:48:46.103 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.097224+0000", 2024-09-16T20:48:46.103 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.000441335, 2024-09-16T20:48:46.103 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00049831599999999995, 2024-09-16T20:48:46.103 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:46.103 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:46.103 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:48:46.103 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:46.103 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4882", 2024-09-16T20:48:46.103 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2436923223", 2024-09-16T20:48:46.103 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 1680 2024-09-16T20:48:46.103 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.103 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:46.103 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.104 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:46.104 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.097224+0000", 2024-09-16T20:48:46.104 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.104 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.104 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.104 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:46.104 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.097224+0000", 2024-09-16T20:48:46.104 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.104 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.104 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.104 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:46.104 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.097224+0000", 2024-09-16T20:48:46.104 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7300000000000002e-07 2024-09-16T20:48:46.105 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.105 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.105 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:46.105 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.097226+0000", 2024-09-16T20:48:46.105 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.159e-06 2024-09-16T20:48:46.105 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.105 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.105 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:46.105 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.097230+0000", 2024-09-16T20:48:46.105 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9589999999999997e-06 2024-09-16T20:48:46.105 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.105 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.105 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:46.106 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.097233+0000", 2024-09-16T20:48:46.106 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7599999999999998e-06 2024-09-16T20:48:46.106 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.106 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:46.106 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.106 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.106 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.106 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4882.0:1681 3.1a 3.ac0e573a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T20:48:46.106 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.097391+0000", 2024-09-16T20:48:46.106 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.00027359200000000002, 2024-09-16T20:48:46.106 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00035559200000000001, 2024-09-16T20:48:46.106 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:46.106 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:46.106 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:48:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4882", 2024-09-16T20:48:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2436923223", 2024-09-16T20:48:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 1681 2024-09-16T20:48:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.097391+0000", 2024-09-16T20:48:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:46.108 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.097391+0000", 2024-09-16T20:48:46.108 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.108 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.108 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.108 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:46.108 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.097392+0000", 2024-09-16T20:48:46.108 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2999999999999999e-07 2024-09-16T20:48:46.108 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.108 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.108 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:46.108 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.097397+0000", 2024-09-16T20:48:46.108 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4050000000000002e-06 2024-09-16T20:48:46.108 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.109 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.109 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:46.109 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.097406+0000", 2024-09-16T20:48:46.109 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.5240000000000006e-06 2024-09-16T20:48:46.109 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.109 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.109 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:46.109 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.097413+0000", 2024-09-16T20:48:46.109 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9589999999999998e-06 2024-09-16T20:48:46.109 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.109 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:46.109 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.109 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.109 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-16T20:48:46.110 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 3 2024-09-16T20:48:46.110 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:46.110 INFO:teuthology.orchestra.run.smithi139.stdout:2049 trunc from 0x3d7e600 to 0xc787a00 2024-09-16T20:48:46.111 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:48:46.121 INFO:teuthology.orchestra.run.smithi139.stdout:2051 punch from 0xb0b8e00 to 0xb0bfc00, (0x6e00 bytes) 2024-09-16T20:48:46.123 INFO:teuthology.orchestra.run.smithi139.stdout:2052 read 0x460bc00 thru 0x461b129 (0xf52a bytes) 2024-09-16T20:48:46.123 INFO:teuthology.orchestra.run.smithi139.stdout:2053 read 0xf0e00 thru 0xff4f6 (0xe6f7 bytes) 2024-09-16T20:48:46.135 INFO:teuthology.orchestra.run.smithi139.stdout:2054 trunc from 0xc787a00 to 0x2a29c00 2024-09-16T20:48:46.150 INFO:teuthology.orchestra.run.smithi139.stdout:2055 punch from 0x1b19e00 to 0x1b25200, (0xb400 bytes) 2024-09-16T20:48:46.155 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:46.154+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:46.171 INFO:teuthology.orchestra.run.smithi139.stdout:2056 write 0x8c5de00 thru 0x8c6697c (0x8b7d bytes) 2024-09-16T20:48:46.187 INFO:teuthology.orchestra.run.smithi139.stdout:2059 write 0x9368600 thru 0x9375022 (0xca23 bytes) 2024-09-16T20:48:46.188 INFO:teuthology.orchestra.run.smithi139.stdout:2060 punch from 0x5acf600 to 0x5ad7c00, (0x8600 bytes) 2024-09-16T20:48:46.188 INFO:teuthology.orchestra.run.smithi139.stdout:2061 write 0x9170200 thru 0x9171927 (0x1728 bytes) 2024-09-16T20:48:46.193 INFO:teuthology.orchestra.run.smithi139.stdout:2063 read 0x92d2e00 thru 0x92d346d (0x66e bytes) 2024-09-16T20:48:46.193 INFO:teuthology.orchestra.run.smithi139.stdout:2064 punch from 0x59efa00 to 0x59ff200, (0xf800 bytes) 2024-09-16T20:48:46.193 INFO:teuthology.orchestra.run.smithi139.stdout:2066 write 0x7ed6200 thru 0x7ee482b (0xe62c bytes) 2024-09-16T20:48:46.193 INFO:teuthology.orchestra.run.smithi139.stdout:2067 read 0x6573a00 thru 0x65802d3 (0xc8d4 bytes) 2024-09-16T20:48:46.196 INFO:teuthology.orchestra.run.smithi139.stdout:2068 trunc from 0x9375200 to 0xc304000 2024-09-16T20:48:46.209 INFO:teuthology.orchestra.run.smithi139.stdout:2070 write 0x7f29200 thru 0x7f307b2 (0x75b3 bytes) 2024-09-16T20:48:46.212 INFO:teuthology.orchestra.run.smithi139.stdout:2071 read 0xae6ae00 thru 0xae6f6d7 (0x48d8 bytes) 2024-09-16T20:48:46.212 INFO:teuthology.orchestra.run.smithi139.stdout:2072 write 0xa9d5800 thru 0xa9e0196 (0xa997 bytes) 2024-09-16T20:48:46.212 INFO:teuthology.orchestra.run.smithi139.stdout:2074 read 0x355ca00 thru 0x35663f5 (0x99f6 bytes) 2024-09-16T20:48:46.212 INFO:teuthology.orchestra.run.smithi139.stdout:2076 write 0x77e600 thru 0x780afe (0x24ff bytes) 2024-09-16T20:48:46.212 INFO:teuthology.orchestra.run.smithi139.stdout:2077 write 0xdbda00 thru 0xdc6b81 (0x9182 bytes) 2024-09-16T20:48:46.213 INFO:teuthology.orchestra.run.smithi139.stdout:2078 write 0x88d1800 thru 0x88ddaf1 (0xc2f2 bytes) 2024-09-16T20:48:46.214 INFO:teuthology.orchestra.run.smithi139.stdout:2082 write 0x6741c00 thru 0x674dddd (0xc1de bytes) 2024-09-16T20:48:46.215 INFO:teuthology.orchestra.run.smithi139.stdout:2083 write 0x67dba00 thru 0x67e47d7 (0x8dd8 bytes) 2024-09-16T20:48:46.215 INFO:teuthology.orchestra.run.smithi139.stdout:2085 read 0xc1ac800 thru 0xc1b11a1 (0x49a2 bytes) 2024-09-16T20:48:46.215 INFO:teuthology.orchestra.run.smithi139.stdout:2086 read 0xb867400 thru 0xb873ad0 (0xc6d1 bytes) 2024-09-16T20:48:46.216 INFO:teuthology.orchestra.run.smithi139.stdout:2089 write 0x96ff200 thru 0x9704518 (0x5319 bytes) 2024-09-16T20:48:46.216 INFO:teuthology.orchestra.run.smithi139.stdout:2090 read 0xa7f5400 thru 0xa8037d0 (0xe3d1 bytes) 2024-09-16T20:48:46.216 INFO:teuthology.orchestra.run.smithi139.stdout:2091 punch from 0xe5f400 to 0xe6d800, (0xe400 bytes) 2024-09-16T20:48:46.216 INFO:teuthology.orchestra.run.smithi139.stdout:2095 read 0x3541a00 thru 0x3550c53 (0xf254 bytes) 2024-09-16T20:48:46.229 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:46.228+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:46.230 INFO:teuthology.orchestra.run.smithi139.stdout:2097 write 0xc3a3400 thru 0xc3a538a (0x1f8b bytes) 2024-09-16T20:48:46.231 INFO:teuthology.orchestra.run.smithi139.stdout:2099 write 0x7f96800 thru 0x7f982b8 (0x1ab9 bytes) 2024-09-16T20:48:46.232 INFO:teuthology.orchestra.run.smithi139.stdout:2100 write 0xad27c00 thru 0xad2ea55 (0x6e56 bytes) 2024-09-16T20:48:46.232 INFO:teuthology.orchestra.run.smithi139.stdout:2101 read 0x2f33000 thru 0x2f39e47 (0x6e48 bytes) 2024-09-16T20:48:46.233 INFO:teuthology.orchestra.run.smithi139.stdout:2102 read 0xb364e00 thru 0xb36aeb2 (0x60b3 bytes) 2024-09-16T20:48:46.236 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:46.236 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:48:46.236 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:48:46.237 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:48:46.237 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:46.238 INFO:teuthology.orchestra.run.smithi139.stdout:2104 trunc from 0xc3a5400 to 0x7063800 2024-09-16T20:48:46.247 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:48:46.259 INFO:teuthology.orchestra.run.smithi139.stdout:2108 trunc from 0x7063800 to 0x6228800 2024-09-16T20:48:46.287 INFO:teuthology.orchestra.run.smithi139.stdout:2109 write 0xa8b6600 thru 0xa8c247f (0xbe80 bytes) 2024-09-16T20:48:46.295 INFO:teuthology.orchestra.run.smithi139.stdout:2110 write 0x2c6800 thru 0x2ca078 (0x3879 bytes) 2024-09-16T20:48:46.295 INFO:teuthology.orchestra.run.smithi139.stdout:2111 write 0x9bac800 thru 0x9bba6e2 (0xdee3 bytes) 2024-09-16T20:48:46.298 INFO:teuthology.orchestra.run.smithi139.stdout:2113 trunc from 0xa8c2600 to 0xc3a3a00 2024-09-16T20:48:46.312 INFO:teuthology.orchestra.run.smithi139.stdout:2114 read 0x584fe00 thru 0x585cf0f (0xd110 bytes) 2024-09-16T20:48:46.325 INFO:teuthology.orchestra.run.smithi139.stdout:2115 write 0xcf97600 thru 0xcf98c8a (0x168b bytes) 2024-09-16T20:48:46.332 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:46.331+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:46.339 INFO:teuthology.orchestra.run.smithi139.stdout:2118 trunc from 0xcf98e00 to 0x207a000 2024-09-16T20:48:46.386 INFO:teuthology.orchestra.run.smithi139.stdout:2121 write 0xcd05600 thru 0xcd10554 (0xaf55 bytes) 2024-09-16T20:48:46.396 INFO:teuthology.orchestra.run.smithi139.stdout:2122 read 0x90b7600 thru 0x90c16d1 (0xa0d2 bytes) 2024-09-16T20:48:46.396 INFO:teuthology.orchestra.run.smithi139.stdout:2123 read 0x7282600 thru 0x728a889 (0x828a bytes) 2024-09-16T20:48:46.396 INFO:teuthology.orchestra.run.smithi139.stdout:2124 write 0x1594000 thru 0x159c49a (0x849b bytes) 2024-09-16T20:48:46.396 INFO:teuthology.orchestra.run.smithi139.stdout:2128 read 0x9feb600 thru 0x9ff952b (0xdf2c bytes) 2024-09-16T20:48:46.396 INFO:teuthology.orchestra.run.smithi139.stdout:2129 read 0x700d200 thru 0x70192f3 (0xc0f4 bytes) 2024-09-16T20:48:46.396 INFO:teuthology.orchestra.run.smithi139.stdout:2131 punch from 0x2b17e00 to 0x2b27000, (0xf200 bytes) 2024-09-16T20:48:46.396 INFO:teuthology.orchestra.run.smithi139.stdout:2132 punch from 0x354e600 to 0x3556000, (0x7a00 bytes) 2024-09-16T20:48:46.396 INFO:teuthology.orchestra.run.smithi139.stdout:2133 read 0x57ae400 thru 0x57af259 (0xe5a bytes) 2024-09-16T20:48:46.410 INFO:teuthology.orchestra.run.smithi139.stdout:2134 trunc from 0xcd10600 to 0x632c00 2024-09-16T20:48:46.414 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:46.414 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:48:46.414 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:48:46.414 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:48:46.415 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.415 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:48:46.415 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:48:46.415 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 53.158441142000001, 2024-09-16T20:48:46.415 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67502346999999996, 2024-09-16T20:48:46.415 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:46.415 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:46.415 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:46.415 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:46.415 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:48:46.415 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:48:46.416 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 1 2024-09-16T20:48:46.416 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.416 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:46.416 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.416 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:46.416 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:48:46.416 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.416 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.416 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.417 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:46.417 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:48:46.417 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.417 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.418 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.418 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:46.418 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254128+0000", 2024-09-16T20:48:46.418 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.463e-06 2024-09-16T20:48:46.418 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.418 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.418 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:46.418 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254135+0000", 2024-09-16T20:48:46.418 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1579999999999999e-06 2024-09-16T20:48:46.419 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.419 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.419 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:46.419 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254146+0000", 2024-09-16T20:48:46.419 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1341e-05 2024-09-16T20:48:46.419 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.419 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.419 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:46.419 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254155+0000", 2024-09-16T20:48:46.419 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.1549999999999996e-06 2024-09-16T20:48:46.420 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.420 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.420 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:46.420 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254221+0000", 2024-09-16T20:48:46.420 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5470999999999997e-05 2024-09-16T20:48:46.420 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.420 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.420 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:48:46.420 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254231+0000", 2024-09-16T20:48:46.421 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0592e-05 2024-09-16T20:48:46.421 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.421 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.421 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:46.421 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926711+0000", 2024-09-16T20:48:46.421 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67247946599999997 2024-09-16T20:48:46.421 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.421 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.421 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:46.421 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926910+0000", 2024-09-16T20:48:46.421 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019934 2024-09-16T20:48:46.422 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.422 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.422 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:48:46.422 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.927159+0000", 2024-09-16T20:48:46.422 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00024876599999999998 2024-09-16T20:48:46.422 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.422 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.422 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:46.422 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.928797+0000", 2024-09-16T20:48:46.422 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001638308 2024-09-16T20:48:46.422 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.422 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.422 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:46.423 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929026+0000", 2024-09-16T20:48:46.423 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00022876099999999999 2024-09-16T20:48:46.423 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.423 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.423 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:46.423 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929102+0000", 2024-09-16T20:48:46.423 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6184000000000001e-05 2024-09-16T20:48:46.423 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.423 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.423 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:46.423 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929150+0000", 2024-09-16T20:48:46.423 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7465e-05 2024-09-16T20:48:46.423 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.424 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:46.424 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.424 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.424 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.424 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4312 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4618240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:46.424 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745559+0000", 2024-09-16T20:48:46.424 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 26.667008006, 2024-09-16T20:48:46.424 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046756458000000001, 2024-09-16T20:48:46.424 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:46.425 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:46.425 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:46.425 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:46.425 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:48:46.425 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:46.425 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4312 2024-09-16T20:48:46.425 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.425 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:46.425 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.425 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:46.425 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745559+0000", 2024-09-16T20:48:46.425 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.425 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.425 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.426 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:46.426 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745559+0000", 2024-09-16T20:48:46.426 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.426 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.426 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.426 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:46.426 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745560+0000", 2024-09-16T20:48:46.426 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6899999999999998e-07 2024-09-16T20:48:46.426 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.426 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.426 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:46.426 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745569+0000", 2024-09-16T20:48:46.427 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8950000000000004e-06 2024-09-16T20:48:46.427 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.427 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.427 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:46.427 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745576+0000", 2024-09-16T20:48:46.427 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9949999999999999e-06 2024-09-16T20:48:46.427 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.427 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.427 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:46.427 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745596+0000", 2024-09-16T20:48:46.427 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0767999999999998e-05 2024-09-16T20:48:46.427 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.427 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.428 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:46.428 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792225+0000", 2024-09-16T20:48:46.428 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046628393999999997 2024-09-16T20:48:46.428 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.428 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.428 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:46.428 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792265+0000", 2024-09-16T20:48:46.428 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9912e-05 2024-09-16T20:48:46.428 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.428 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.428 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:46.428 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792316+0000", 2024-09-16T20:48:46.429 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1025e-05 2024-09-16T20:48:46.429 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.429 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:46.429 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.429 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.429 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.429 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4313 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4626432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:46.429 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745608+0000", 2024-09-16T20:48:46.429 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 26.666959436999999, 2024-09-16T20:48:46.429 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046829200000000001, 2024-09-16T20:48:46.429 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:46.430 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:46.430 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:46.430 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:46.430 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:48:46.430 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:46.430 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4313 2024-09-16T20:48:46.430 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.430 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:46.430 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.430 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:46.430 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745608+0000", 2024-09-16T20:48:46.430 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.430 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.431 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.431 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:46.431 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745608+0000", 2024-09-16T20:48:46.431 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.431 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.431 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.431 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:46.431 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745609+0000", 2024-09-16T20:48:46.431 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4700000000000001e-07 2024-09-16T20:48:46.431 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.431 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.431 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:46.431 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745611+0000", 2024-09-16T20:48:46.431 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8119999999999999e-06 2024-09-16T20:48:46.432 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.432 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.432 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:46.432 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745619+0000", 2024-09-16T20:48:46.432 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6520000000000006e-06 2024-09-16T20:48:46.432 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.432 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.432 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:46.432 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745625+0000", 2024-09-16T20:48:46.432 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6590000000000004e-06 2024-09-16T20:48:46.432 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.432 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.433 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:46.433 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792344+0000", 2024-09-16T20:48:46.433 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046719318000000003 2024-09-16T20:48:46.433 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.433 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.433 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:46.433 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792384+0000", 2024-09-16T20:48:46.433 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0389000000000002e-05 2024-09-16T20:48:46.433 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.433 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.433 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:46.433 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792437+0000", 2024-09-16T20:48:46.433 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2623000000000001e-05 2024-09-16T20:48:46.434 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.434 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:46.434 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.434 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.434 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.434 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4314 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4634624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:46.434 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745632+0000", 2024-09-16T20:48:46.434 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 26.66693557, 2024-09-16T20:48:46.434 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046928879999999999, 2024-09-16T20:48:46.434 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:46.434 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:46.434 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:46.434 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:46.435 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:48:46.435 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:46.435 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4314 2024-09-16T20:48:46.435 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.435 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:46.435 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.435 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:46.435 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745632+0000", 2024-09-16T20:48:46.435 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.435 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.435 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.435 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:46.435 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745632+0000", 2024-09-16T20:48:46.436 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.436 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.436 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.436 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:46.436 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745632+0000", 2024-09-16T20:48:46.436 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3700000000000001e-07 2024-09-16T20:48:46.436 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.436 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.436 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:46.436 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745634+0000", 2024-09-16T20:48:46.436 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9759999999999998e-06 2024-09-16T20:48:46.436 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.436 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.437 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:46.437 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745645+0000", 2024-09-16T20:48:46.437 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1097000000000001e-05 2024-09-16T20:48:46.437 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.437 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.437 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:46.437 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745651+0000", 2024-09-16T20:48:46.437 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5090000000000001e-06 2024-09-16T20:48:46.437 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.437 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.437 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:46.437 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792467+0000", 2024-09-16T20:48:46.438 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046816712000000003 2024-09-16T20:48:46.438 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.438 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.438 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:46.438 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792511+0000", 2024-09-16T20:48:46.438 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3256000000000001e-05 2024-09-16T20:48:46.438 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.438 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.438 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:46.438 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792561+0000", 2024-09-16T20:48:46.438 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9892999999999998e-05 2024-09-16T20:48:46.438 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.438 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:46.439 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.439 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.439 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.439 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4315 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4642816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:46.439 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745659+0000", 2024-09-16T20:48:46.439 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 26.666908447000001, 2024-09-16T20:48:46.439 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047065501000000003, 2024-09-16T20:48:46.439 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:46.439 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:46.439 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:46.439 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:46.439 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:48:46.439 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:46.440 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4315 2024-09-16T20:48:46.440 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.440 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:46.440 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.440 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:46.440 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745659+0000", 2024-09-16T20:48:46.440 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.440 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.440 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.440 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:46.440 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745659+0000", 2024-09-16T20:48:46.440 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.441 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.441 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.441 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:46.441 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745659+0000", 2024-09-16T20:48:46.441 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7e-07 2024-09-16T20:48:46.441 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.441 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.441 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:46.441 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745662+0000", 2024-09-16T20:48:46.441 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2189999999999998e-06 2024-09-16T20:48:46.441 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.441 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.441 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:46.442 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745668+0000", 2024-09-16T20:48:46.442 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8979999999999999e-06 2024-09-16T20:48:46.442 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.442 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.442 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:46.442 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745675+0000", 2024-09-16T20:48:46.442 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3579999999999999e-06 2024-09-16T20:48:46.442 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.442 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.442 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:46.442 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792593+0000", 2024-09-16T20:48:46.442 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046917888999999997 2024-09-16T20:48:46.443 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.443 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.443 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:46.443 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792632+0000", 2024-09-16T20:48:46.443 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9818999999999999e-05 2024-09-16T20:48:46.443 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.443 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.443 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:46.443 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792724+0000", 2024-09-16T20:48:46.443 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.1848000000000006e-05 2024-09-16T20:48:46.443 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.443 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:46.444 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.444 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.444 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.444 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4316 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4651008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:46.444 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745682+0000", 2024-09-16T20:48:46.444 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 26.666885169, 2024-09-16T20:48:46.444 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047270334999999997, 2024-09-16T20:48:46.444 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:46.444 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:46.444 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:46.444 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:46.444 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:48:46.445 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:46.445 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4316 2024-09-16T20:48:46.445 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.445 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:46.445 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.445 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:46.445 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745682+0000", 2024-09-16T20:48:46.445 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.445 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.445 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.445 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:46.445 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745682+0000", 2024-09-16T20:48:46.445 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.446 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.446 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.446 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:46.446 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745683+0000", 2024-09-16T20:48:46.446 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.27e-07 2024-09-16T20:48:46.446 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.446 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.446 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:46.446 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745685+0000", 2024-09-16T20:48:46.446 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.464e-06 2024-09-16T20:48:46.446 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.446 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.446 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:46.447 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745692+0000", 2024-09-16T20:48:46.447 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.849e-06 2024-09-16T20:48:46.447 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.447 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.447 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:46.447 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745697+0000", 2024-09-16T20:48:46.447 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0170000000000004e-06 2024-09-16T20:48:46.447 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.447 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.447 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:46.447 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792771+0000", 2024-09-16T20:48:46.447 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047074224999999997 2024-09-16T20:48:46.447 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.448 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.448 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:46.448 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792845+0000", 2024-09-16T20:48:46.448 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3739999999999995e-05 2024-09-16T20:48:46.448 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.448 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.448 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:46.448 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792952+0000", 2024-09-16T20:48:46.448 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000107613 2024-09-16T20:48:46.448 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.448 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:46.448 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.448 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.449 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.449 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4317 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4659200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:46.449 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745704+0000", 2024-09-16T20:48:46.449 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 26.666863223, 2024-09-16T20:48:46.449 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047411778000000002, 2024-09-16T20:48:46.449 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:46.449 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:46.449 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:46.449 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:46.449 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:48:46.449 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:46.449 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4317 2024-09-16T20:48:46.449 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.449 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:46.450 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.450 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:46.450 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745704+0000", 2024-09-16T20:48:46.450 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.450 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.450 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.450 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:46.450 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745704+0000", 2024-09-16T20:48:46.450 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.450 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.450 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.450 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:46.450 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745704+0000", 2024-09-16T20:48:46.451 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0900000000000002e-07 2024-09-16T20:48:46.451 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.451 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.451 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:46.451 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745707+0000", 2024-09-16T20:48:46.451 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1660000000000001e-06 2024-09-16T20:48:46.451 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.451 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.451 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:46.451 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745722+0000", 2024-09-16T20:48:46.451 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5084e-05 2024-09-16T20:48:46.451 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.451 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.451 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:46.451 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745735+0000", 2024-09-16T20:48:46.452 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3074999999999999e-05 2024-09-16T20:48:46.452 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.452 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.452 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:46.452 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792989+0000", 2024-09-16T20:48:46.452 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047254013999999997 2024-09-16T20:48:46.452 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.452 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.452 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:46.452 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793034+0000", 2024-09-16T20:48:46.452 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4858999999999999e-05 2024-09-16T20:48:46.452 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.452 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.453 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:46.453 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793116+0000", 2024-09-16T20:48:46.453 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2170999999999996e-05 2024-09-16T20:48:46.453 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.453 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:46.453 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.453 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.453 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.453 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4318 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4667392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:46.453 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745812+0000", 2024-09-16T20:48:46.453 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 26.666755544000001, 2024-09-16T20:48:46.453 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047532229000000002, 2024-09-16T20:48:46.453 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:46.454 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:46.454 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:46.454 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:46.454 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:48:46.454 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:46.454 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4318 2024-09-16T20:48:46.454 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.454 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:46.454 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.454 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:46.454 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745812+0000", 2024-09-16T20:48:46.454 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.454 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.455 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.455 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:46.455 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745812+0000", 2024-09-16T20:48:46.455 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.455 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.455 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.455 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:46.455 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745812+0000", 2024-09-16T20:48:46.455 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5899999999999996e-07 2024-09-16T20:48:46.455 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.455 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.455 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:46.455 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745815+0000", 2024-09-16T20:48:46.456 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9110000000000002e-06 2024-09-16T20:48:46.456 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.456 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.456 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:46.456 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745823+0000", 2024-09-16T20:48:46.456 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1030000000000002e-06 2024-09-16T20:48:46.456 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.456 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.456 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:46.456 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745829+0000", 2024-09-16T20:48:46.456 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8449999999999999e-06 2024-09-16T20:48:46.456 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.456 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.457 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:46.457 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793159+0000", 2024-09-16T20:48:46.457 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047330009999999999 2024-09-16T20:48:46.457 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.457 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.457 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:46.457 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793251+0000", 2024-09-16T20:48:46.457 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.1991999999999996e-05 2024-09-16T20:48:46.457 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.457 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.457 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:46.457 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793344+0000", 2024-09-16T20:48:46.457 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2808999999999999e-05 2024-09-16T20:48:46.458 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.458 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:46.458 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.458 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.458 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.458 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4319 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4675584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:46.458 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745838+0000", 2024-09-16T20:48:46.458 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 26.666729330999999, 2024-09-16T20:48:46.458 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047640204999999998, 2024-09-16T20:48:46.458 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:46.458 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:46.458 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:46.458 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:46.458 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:48:46.458 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:46.459 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4319 2024-09-16T20:48:46.459 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.459 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:46.459 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.459 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:46.459 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745838+0000", 2024-09-16T20:48:46.459 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.459 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.459 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.459 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:46.459 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745838+0000", 2024-09-16T20:48:46.459 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.459 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.460 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.460 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:46.460 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745838+0000", 2024-09-16T20:48:46.460 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6899999999999998e-07 2024-09-16T20:48:46.460 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.460 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.460 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:46.460 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745840+0000", 2024-09-16T20:48:46.460 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0660000000000002e-06 2024-09-16T20:48:46.460 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.460 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.460 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:46.460 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745847+0000", 2024-09-16T20:48:46.461 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8689999999999998e-06 2024-09-16T20:48:46.461 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.461 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.461 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:46.461 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745853+0000", 2024-09-16T20:48:46.461 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2399999999999998e-06 2024-09-16T20:48:46.461 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.461 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.461 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:46.461 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793381+0000", 2024-09-16T20:48:46.461 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047528005999999998 2024-09-16T20:48:46.461 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.462 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.462 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:46.462 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793423+0000", 2024-09-16T20:48:46.462 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2799000000000001e-05 2024-09-16T20:48:46.462 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.462 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.462 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:46.462 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793478+0000", 2024-09-16T20:48:46.462 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4756000000000002e-05 2024-09-16T20:48:46.462 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.462 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:46.462 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.462 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.463 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.463 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4320 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4683776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:46.463 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745859+0000", 2024-09-16T20:48:46.463 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 26.666708087, 2024-09-16T20:48:46.463 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047811399999999997, 2024-09-16T20:48:46.463 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:46.463 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:46.463 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:46.463 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:46.463 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:48:46.463 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:46.463 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4320 2024-09-16T20:48:46.463 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.464 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:46.464 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.464 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:46.464 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745859+0000", 2024-09-16T20:48:46.464 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.464 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.464 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.464 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:46.464 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745859+0000", 2024-09-16T20:48:46.464 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.464 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.464 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.464 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:46.465 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745860+0000", 2024-09-16T20:48:46.465 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9099999999999999e-07 2024-09-16T20:48:46.465 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.465 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.465 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:46.465 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745861+0000", 2024-09-16T20:48:46.465 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8589999999999999e-06 2024-09-16T20:48:46.465 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.465 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.465 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:46.465 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745868+0000", 2024-09-16T20:48:46.465 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2249999999999997e-06 2024-09-16T20:48:46.465 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.466 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.466 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:46.466 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745873+0000", 2024-09-16T20:48:46.466 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1290000000000004e-06 2024-09-16T20:48:46.466 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.466 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.466 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:46.466 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793522+0000", 2024-09-16T20:48:46.466 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047648847000000001 2024-09-16T20:48:46.466 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.466 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.466 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:46.466 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793603+0000", 2024-09-16T20:48:46.466 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1167000000000004e-05 2024-09-16T20:48:46.467 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.467 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.467 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:46.467 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793671+0000", 2024-09-16T20:48:46.467 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7781999999999996e-05 2024-09-16T20:48:46.467 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.467 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:46.467 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.467 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.467 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.467 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4321 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4691968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:46.467 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745879+0000", 2024-09-16T20:48:46.467 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 26.666687890999999, 2024-09-16T20:48:46.468 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047950375000000003, 2024-09-16T20:48:46.468 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:46.468 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:46.468 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:46.468 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:46.468 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:48:46.468 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:46.468 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4321 2024-09-16T20:48:46.468 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.468 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:46.468 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.468 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:46.468 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745879+0000", 2024-09-16T20:48:46.469 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.469 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.469 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.469 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:46.469 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745879+0000", 2024-09-16T20:48:46.469 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.469 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.469 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.469 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:46.472 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745880+0000", 2024-09-16T20:48:46.473 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.65e-07 2024-09-16T20:48:46.473 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.473 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.473 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:46.473 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745882+0000", 2024-09-16T20:48:46.473 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.922e-06 2024-09-16T20:48:46.473 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.473 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.473 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:46.473 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745888+0000", 2024-09-16T20:48:46.473 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0510000000000002e-06 2024-09-16T20:48:46.473 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.473 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.474 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:46.474 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745893+0000", 2024-09-16T20:48:46.474 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9579999999999998e-06 2024-09-16T20:48:46.474 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.474 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.474 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:46.474 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793704+0000", 2024-09-16T20:48:46.474 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047811179000000002 2024-09-16T20:48:46.474 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.474 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.474 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:46.474 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793759+0000", 2024-09-16T20:48:46.474 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5260999999999998e-05 2024-09-16T20:48:46.475 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.475 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.475 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:46.475 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793830+0000", 2024-09-16T20:48:46.475 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0638999999999999e-05 2024-09-16T20:48:46.475 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.475 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:46.475 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.476 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.476 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.476 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4322 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4700160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:46.476 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745899+0000", 2024-09-16T20:48:46.476 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 26.666668453, 2024-09-16T20:48:46.476 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048057280000000001, 2024-09-16T20:48:46.476 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:46.476 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:46.476 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:46.476 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:46.476 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:48:46.477 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:46.477 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4322 2024-09-16T20:48:46.477 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.477 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:46.477 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.477 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:46.477 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745899+0000", 2024-09-16T20:48:46.477 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.477 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.477 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.477 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:46.477 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745899+0000", 2024-09-16T20:48:46.477 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.477 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.478 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.478 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:46.478 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745899+0000", 2024-09-16T20:48:46.478 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9299999999999999e-07 2024-09-16T20:48:46.478 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.478 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.478 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:46.478 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745901+0000", 2024-09-16T20:48:46.478 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0030000000000001e-06 2024-09-16T20:48:46.478 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.478 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.478 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:46.478 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745908+0000", 2024-09-16T20:48:46.478 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3729999999999998e-06 2024-09-16T20:48:46.479 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.479 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.479 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:46.479 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745914+0000", 2024-09-16T20:48:46.479 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.122e-06 2024-09-16T20:48:46.479 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.479 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.479 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:46.479 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793862+0000", 2024-09-16T20:48:46.479 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047948324 2024-09-16T20:48:46.479 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.479 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.479 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:46.479 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793903+0000", 2024-09-16T20:48:46.480 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0465e-05 2024-09-16T20:48:46.480 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.480 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.480 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:46.480 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793956+0000", 2024-09-16T20:48:46.480 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3600000000000002e-05 2024-09-16T20:48:46.480 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.480 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:46.480 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.480 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.480 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.480 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4323 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4708352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:46.481 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745920+0000", 2024-09-16T20:48:46.481 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 26.666646812, 2024-09-16T20:48:46.481 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048268138000000002, 2024-09-16T20:48:46.481 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:46.481 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:46.481 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:46.481 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:46.481 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:48:46.481 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:46.481 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4323 2024-09-16T20:48:46.481 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.481 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:46.481 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.482 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:46.482 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745920+0000", 2024-09-16T20:48:46.482 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.482 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.482 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.482 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:46.482 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745920+0000", 2024-09-16T20:48:46.482 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.482 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.482 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.482 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:46.482 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745921+0000", 2024-09-16T20:48:46.482 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7e-07 2024-09-16T20:48:46.482 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.483 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.483 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:46.483 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745923+0000", 2024-09-16T20:48:46.483 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7880000000000001e-06 2024-09-16T20:48:46.483 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.483 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.483 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:46.483 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745929+0000", 2024-09-16T20:48:46.483 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0809999999999999e-06 2024-09-16T20:48:46.483 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.483 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.483 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:46.483 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745935+0000", 2024-09-16T20:48:46.483 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5370000000000003e-06 2024-09-16T20:48:46.484 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.484 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.484 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:46.484 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793987+0000", 2024-09-16T20:48:46.484 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048051736999999997 2024-09-16T20:48:46.484 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.484 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.484 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:46.484 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.794070+0000", 2024-09-16T20:48:46.484 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.3277999999999998e-05 2024-09-16T20:48:46.484 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.484 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.484 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:46.485 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.794189+0000", 2024-09-16T20:48:46.485 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000118347 2024-09-16T20:48:46.485 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.485 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:46.485 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.485 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.485 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.485 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4324 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4716544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:46.485 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745946+0000", 2024-09-16T20:48:46.485 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 26.666621241000001, 2024-09-16T20:48:46.485 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048468153999999999, 2024-09-16T20:48:46.485 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:46.485 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:46.485 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:46.486 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:46.486 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:48:46.486 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:46.486 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4324 2024-09-16T20:48:46.486 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.486 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:46.486 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.486 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:46.486 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745946+0000", 2024-09-16T20:48:46.486 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.486 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.486 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.486 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:46.486 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745946+0000", 2024-09-16T20:48:46.487 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.487 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.487 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.487 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:46.487 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745946+0000", 2024-09-16T20:48:46.487 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4499999999999997e-07 2024-09-16T20:48:46.487 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.487 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.487 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:46.487 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745949+0000", 2024-09-16T20:48:46.487 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2529999999999999e-06 2024-09-16T20:48:46.487 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.487 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.487 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:46.488 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745955+0000", 2024-09-16T20:48:46.488 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6470000000000001e-06 2024-09-16T20:48:46.488 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.488 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.488 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:46.488 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745960+0000", 2024-09-16T20:48:46.488 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7910000000000004e-06 2024-09-16T20:48:46.488 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.488 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.488 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:46.488 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.794220+0000", 2024-09-16T20:48:46.488 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048260069000000003 2024-09-16T20:48:46.488 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.489 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.489 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:46.489 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.794311+0000", 2024-09-16T20:48:46.489 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0593000000000005e-05 2024-09-16T20:48:46.489 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.489 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.489 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:46.489 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.794414+0000", 2024-09-16T20:48:46.489 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000103356 2024-09-16T20:48:46.489 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.489 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:46.489 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.489 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.489 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.490 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2059 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 1005568~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:46.490 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.342406+0000", 2024-09-16T20:48:46.490 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 24.070160803, 2024-09-16T20:48:46.490 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046803412000000003, 2024-09-16T20:48:46.490 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:46.490 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:46.490 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:46.490 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:46.490 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:46.490 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:46.490 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2059 2024-09-16T20:48:46.490 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.490 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:46.491 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.491 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:46.491 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342406+0000", 2024-09-16T20:48:46.491 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.491 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.491 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.491 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:46.491 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342406+0000", 2024-09-16T20:48:46.491 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.492 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.492 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.492 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:46.492 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342407+0000", 2024-09-16T20:48:46.492 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7399999999999998e-07 2024-09-16T20:48:46.492 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.492 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.492 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:46.492 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342409+0000", 2024-09-16T20:48:46.492 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.4509999999999999e-06 2024-09-16T20:48:46.492 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.492 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.492 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:46.493 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342418+0000", 2024-09-16T20:48:46.493 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8969999999999997e-06 2024-09-16T20:48:46.493 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.493 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.493 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:46.493 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342424+0000", 2024-09-16T20:48:46.493 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5210000000000002e-06 2024-09-16T20:48:46.493 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.493 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.493 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:46.493 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389084+0000", 2024-09-16T20:48:46.493 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046659997000000002 2024-09-16T20:48:46.493 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.493 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.494 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:46.494 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389153+0000", 2024-09-16T20:48:46.494 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9558e-05 2024-09-16T20:48:46.494 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.494 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.494 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:46.494 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389210+0000", 2024-09-16T20:48:46.494 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6514000000000001e-05 2024-09-16T20:48:46.494 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.494 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:46.494 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.494 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.494 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.495 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2061 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 1013760~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:46.495 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.342445+0000", 2024-09-16T20:48:46.495 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 24.070122251000001, 2024-09-16T20:48:46.495 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046901708, 2024-09-16T20:48:46.495 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:46.495 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:46.495 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:46.495 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:46.495 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:46.495 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:46.495 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2061 2024-09-16T20:48:46.495 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.495 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:46.496 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.496 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:46.496 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342445+0000", 2024-09-16T20:48:46.496 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.496 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.496 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.496 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:46.496 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342445+0000", 2024-09-16T20:48:46.496 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.496 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.496 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.496 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:46.496 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342446+0000", 2024-09-16T20:48:46.497 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4700000000000001e-07 2024-09-16T20:48:46.497 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.497 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.497 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:46.497 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342448+0000", 2024-09-16T20:48:46.497 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6460000000000002e-06 2024-09-16T20:48:46.497 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.497 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.497 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:46.497 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342456+0000", 2024-09-16T20:48:46.497 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7489999999999998e-06 2024-09-16T20:48:46.497 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.497 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.497 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:46.498 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342462+0000", 2024-09-16T20:48:46.498 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5759999999999996e-06 2024-09-16T20:48:46.498 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.498 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.498 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:46.498 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389238+0000", 2024-09-16T20:48:46.498 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046776515999999997 2024-09-16T20:48:46.498 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.498 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.498 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:46.498 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389285+0000", 2024-09-16T20:48:46.498 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6581e-05 2024-09-16T20:48:46.498 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.498 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.499 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:46.499 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389347+0000", 2024-09-16T20:48:46.499 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2093000000000003e-05 2024-09-16T20:48:46.499 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.499 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:46.499 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.499 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.499 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.499 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2063 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 1021952~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:46.499 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.342470+0000", 2024-09-16T20:48:46.499 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 24.070097361999998, 2024-09-16T20:48:46.499 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047007602000000002, 2024-09-16T20:48:46.499 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:46.500 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:46.500 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:46.500 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:46.500 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:46.500 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:46.500 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2063 2024-09-16T20:48:46.500 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.500 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:46.500 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.500 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:46.500 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342470+0000", 2024-09-16T20:48:46.500 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.500 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.501 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.501 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:46.501 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342470+0000", 2024-09-16T20:48:46.501 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.501 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.501 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.501 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:46.501 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342470+0000", 2024-09-16T20:48:46.501 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0999999999999999e-07 2024-09-16T20:48:46.501 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.501 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.501 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:46.502 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342473+0000", 2024-09-16T20:48:46.502 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.249e-06 2024-09-16T20:48:46.502 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.502 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.502 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:46.502 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342479+0000", 2024-09-16T20:48:46.502 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7549999999999997e-06 2024-09-16T20:48:46.502 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.502 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.502 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:46.503 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342485+0000", 2024-09-16T20:48:46.503 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.344e-06 2024-09-16T20:48:46.503 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.503 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.503 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:46.503 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389364+0000", 2024-09-16T20:48:46.503 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046879542000000003 2024-09-16T20:48:46.503 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.503 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.503 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:46.503 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389420+0000", 2024-09-16T20:48:46.503 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5541000000000002e-05 2024-09-16T20:48:46.503 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.503 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.504 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:46.504 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389478+0000", 2024-09-16T20:48:46.504 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7661000000000002e-05 2024-09-16T20:48:46.504 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.504 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:46.504 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.504 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.504 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.504 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2065 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 1030144~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:46.504 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.342492+0000", 2024-09-16T20:48:46.504 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 24.070075524, 2024-09-16T20:48:46.504 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047127962000000002, 2024-09-16T20:48:46.505 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:46.505 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:46.505 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:46.505 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:46.505 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:46.505 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:46.505 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2065 2024-09-16T20:48:46.505 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.505 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:46.505 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.505 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:46.505 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342492+0000", 2024-09-16T20:48:46.505 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.505 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.506 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.506 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:46.506 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342492+0000", 2024-09-16T20:48:46.506 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.506 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.506 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.506 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:46.506 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342492+0000", 2024-09-16T20:48:46.506 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1100000000000001e-07 2024-09-16T20:48:46.506 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.506 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.506 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:46.506 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342494+0000", 2024-09-16T20:48:46.507 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9039999999999999e-06 2024-09-16T20:48:46.507 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.507 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.507 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:46.507 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342501+0000", 2024-09-16T20:48:46.507 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6610000000000003e-06 2024-09-16T20:48:46.507 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.507 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.507 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:46.507 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342506+0000", 2024-09-16T20:48:46.507 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8760000000000004e-06 2024-09-16T20:48:46.507 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.507 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.507 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:46.507 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389501+0000", 2024-09-16T20:48:46.508 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046995440999999999 2024-09-16T20:48:46.508 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.508 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.508 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:46.508 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389550+0000", 2024-09-16T20:48:46.508 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8949000000000002e-05 2024-09-16T20:48:46.508 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.508 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.508 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:46.508 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389620+0000", 2024-09-16T20:48:46.508 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9720000000000003e-05 2024-09-16T20:48:46.508 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.508 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:46.509 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.509 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.509 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.509 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2068 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 1038336~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:46.509 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.342513+0000", 2024-09-16T20:48:46.509 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 24.070054429999999, 2024-09-16T20:48:46.509 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047233961999999997, 2024-09-16T20:48:46.509 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:46.509 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:46.509 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:46.509 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:46.509 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:46.509 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:46.509 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2068 2024-09-16T20:48:46.510 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.510 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:46.510 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.510 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:46.510 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342513+0000", 2024-09-16T20:48:46.510 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.510 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.510 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.510 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:46.510 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342513+0000", 2024-09-16T20:48:46.510 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.510 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.510 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.511 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:46.511 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342513+0000", 2024-09-16T20:48:46.511 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.01e-07 2024-09-16T20:48:46.511 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.511 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.511 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:46.511 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342515+0000", 2024-09-16T20:48:46.511 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7400000000000001e-06 2024-09-16T20:48:46.511 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.511 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.511 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:46.511 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342522+0000", 2024-09-16T20:48:46.511 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.516e-06 2024-09-16T20:48:46.511 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.512 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.512 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:46.512 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342527+0000", 2024-09-16T20:48:46.512 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0080000000000002e-06 2024-09-16T20:48:46.512 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.512 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.512 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:46.512 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389644+0000", 2024-09-16T20:48:46.512 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047117289 2024-09-16T20:48:46.512 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.512 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.512 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:46.512 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389690+0000", 2024-09-16T20:48:46.513 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6378000000000002e-05 2024-09-16T20:48:46.513 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.513 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.513 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:46.513 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389747+0000", 2024-09-16T20:48:46.513 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6629999999999998e-05 2024-09-16T20:48:46.513 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.513 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:46.513 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.513 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.513 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.513 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2070 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 1046528~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:46.513 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.342533+0000", 2024-09-16T20:48:46.514 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 24.070034332999999, 2024-09-16T20:48:46.514 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047345848000000003, 2024-09-16T20:48:46.514 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:46.514 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:46.514 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:46.514 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:46.514 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:46.514 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:46.514 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2070 2024-09-16T20:48:46.514 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.514 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:46.514 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.514 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:46.514 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342533+0000", 2024-09-16T20:48:46.515 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.515 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.515 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.515 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:46.515 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342533+0000", 2024-09-16T20:48:46.515 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.515 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.515 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.515 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:46.515 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342533+0000", 2024-09-16T20:48:46.515 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7e-07 2024-09-16T20:48:46.515 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.515 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.515 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:46.516 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342535+0000", 2024-09-16T20:48:46.516 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.68e-06 2024-09-16T20:48:46.516 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.516 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.516 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:46.516 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342542+0000", 2024-09-16T20:48:46.516 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5169999999999996e-06 2024-09-16T20:48:46.516 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.516 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.516 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:46.516 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.342546+0000", 2024-09-16T20:48:46.516 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8949999999999997e-06 2024-09-16T20:48:46.516 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.517 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.517 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:46.517 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389759+0000", 2024-09-16T20:48:46.517 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047212853999999999 2024-09-16T20:48:46.517 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.517 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.517 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:46.517 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389813+0000", 2024-09-16T20:48:46.517 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3585999999999998e-05 2024-09-16T20:48:46.517 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.517 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.517 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:46.517 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.389879+0000", 2024-09-16T20:48:46.517 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5945999999999995e-05 2024-09-16T20:48:46.518 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.518 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:46.518 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.518 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.518 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:46.518 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:46.520 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:48:46.520 INFO:teuthology.orchestra.run.smithi139.stdout:2135 read 0x555e00 thru 0x56052c (0xa72d bytes) 2024-09-16T20:48:46.521 INFO:teuthology.orchestra.run.smithi139.stdout:2136 trunc from 0x632c00 to 0x8b33000 2024-09-16T20:48:46.521 INFO:teuthology.orchestra.run.smithi139.stdout:2137 read 0x1f53600 thru 0x1f5fa38 (0xc439 bytes) 2024-09-16T20:48:46.521 INFO:teuthology.orchestra.run.smithi139.stdout:2142 write 0xa413800 thru 0xa42292a (0xf12b bytes) 2024-09-16T20:48:46.521 INFO:teuthology.orchestra.run.smithi139.stdout:2144 punch from 0x678ac00 to 0x6790a00, (0x5e00 bytes) 2024-09-16T20:48:46.521 INFO:teuthology.orchestra.run.smithi139.stdout:2145 read 0x6434c00 thru 0x643ffb9 (0xb3ba bytes) 2024-09-16T20:48:46.521 INFO:teuthology.orchestra.run.smithi139.stdout:2146 write 0x9ff1400 thru 0x9ffcd5e (0xb95f bytes) 2024-09-16T20:48:46.521 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:46.518+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:46.555 INFO:teuthology.orchestra.run.smithi029.stderr:nodeep-scrub is set 2024-09-16T20:48:46.620 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:46.619+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:46.665 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:46.665 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:48:46.665 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:48:46.665 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:46.677 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:48:46.721 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:46.720+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:46.786 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:46.786 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:48:46.786 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:48:46.786 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:48:46.786 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:46.795 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:48:46.822 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:46.821+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:46.869 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T20:48:46.904 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:46.904 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:48:46.904 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:48:46.904 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:48:46.904 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.904 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3709 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1464320~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:46.904 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.638971+0000", 2024-09-16T20:48:46.905 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 24.263301555999998, 2024-09-16T20:48:46.905 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049858670000000001, 2024-09-16T20:48:46.905 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:46.905 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:46.905 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:46.905 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:46.905 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:46.905 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:46.905 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3709 2024-09-16T20:48:46.905 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.905 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:46.905 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.906 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:46.906 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638971+0000", 2024-09-16T20:48:46.906 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.906 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.906 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.906 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:46.906 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638971+0000", 2024-09-16T20:48:46.906 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.906 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.906 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.906 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:46.906 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638972+0000", 2024-09-16T20:48:46.906 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0400000000000002e-07 2024-09-16T20:48:46.907 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.907 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.907 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:46.907 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638973+0000", 2024-09-16T20:48:46.907 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.578e-06 2024-09-16T20:48:46.907 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.907 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.907 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:46.907 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638979+0000", 2024-09-16T20:48:46.907 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.502e-06 2024-09-16T20:48:46.907 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.907 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.908 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:46.908 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638983+0000", 2024-09-16T20:48:46.908 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2690000000000001e-06 2024-09-16T20:48:46.908 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.908 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.908 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:46.908 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688628+0000", 2024-09-16T20:48:46.908 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049644547999999997 2024-09-16T20:48:46.908 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.908 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.908 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:46.908 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688718+0000", 2024-09-16T20:48:46.908 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0802000000000002e-05 2024-09-16T20:48:46.909 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.909 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.909 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:46.909 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688830+0000", 2024-09-16T20:48:46.909 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000111667 2024-09-16T20:48:46.909 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.909 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:46.909 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.909 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.909 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.909 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3712 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1472512~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:46.909 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:48:46.909 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 24.263285027999999, 2024-09-16T20:48:46.910 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050004305999999998, 2024-09-16T20:48:46.910 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:46.910 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:46.910 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:46.910 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:46.910 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:46.910 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:46.910 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3712 2024-09-16T20:48:46.910 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.910 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:46.910 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.910 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:46.910 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:48:46.910 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.911 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.911 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.911 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:46.911 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:48:46.911 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.911 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.911 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.911 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:46.911 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:48:46.911 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0199999999999998e-07 2024-09-16T20:48:46.911 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.911 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.911 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:46.911 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638990+0000", 2024-09-16T20:48:46.912 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.618e-06 2024-09-16T20:48:46.912 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.912 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.912 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:46.912 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638995+0000", 2024-09-16T20:48:46.912 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3940000000000004e-06 2024-09-16T20:48:46.912 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.912 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.912 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:46.912 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639000+0000", 2024-09-16T20:48:46.912 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9309999999999999e-06 2024-09-16T20:48:46.912 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.912 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.913 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:46.913 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688867+0000", 2024-09-16T20:48:46.913 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049866955999999997 2024-09-16T20:48:46.913 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.913 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.913 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:46.913 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688933+0000", 2024-09-16T20:48:46.913 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5359999999999998e-05 2024-09-16T20:48:46.913 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.913 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.913 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:46.913 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688992+0000", 2024-09-16T20:48:46.913 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9744999999999998e-05 2024-09-16T20:48:46.914 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.914 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:46.914 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.914 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.914 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.914 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3714 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1480704~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:46.914 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:48:46.914 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 24.263267438, 2024-09-16T20:48:46.914 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050152552000000003, 2024-09-16T20:48:46.914 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:46.914 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:46.914 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:46.914 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:46.915 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:46.915 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:46.915 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3714 2024-09-16T20:48:46.915 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.915 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:46.915 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.915 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:46.915 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:48:46.915 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.915 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.915 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.915 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:46.916 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:48:46.916 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.916 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.916 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.916 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:46.916 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:48:46.916 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0499999999999999e-07 2024-09-16T20:48:46.916 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.916 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.916 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:46.916 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639007+0000", 2024-09-16T20:48:46.917 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5340000000000001e-06 2024-09-16T20:48:46.917 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.917 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.917 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:46.917 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639013+0000", 2024-09-16T20:48:46.917 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3260000000000003e-06 2024-09-16T20:48:46.917 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.917 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.917 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:46.917 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639018+0000", 2024-09-16T20:48:46.917 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2850000000000002e-06 2024-09-16T20:48:46.917 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.918 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.918 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:46.918 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689026+0000", 2024-09-16T20:48:46.918 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.05000798 2024-09-16T20:48:46.918 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.918 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.918 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:46.918 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689077+0000", 2024-09-16T20:48:46.918 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1063999999999998e-05 2024-09-16T20:48:46.918 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.918 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.918 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:46.918 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689158+0000", 2024-09-16T20:48:46.919 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1057999999999995e-05 2024-09-16T20:48:46.919 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.919 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:46.919 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.919 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.919 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.919 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3716 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1488896~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:46.919 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:48:46.919 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 24.263249628000001, 2024-09-16T20:48:46.919 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050280142, 2024-09-16T20:48:46.919 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:46.920 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:46.920 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:46.920 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:46.920 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:46.920 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:46.920 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3716 2024-09-16T20:48:46.920 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.920 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:46.920 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.920 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:46.920 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:48:46.920 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.921 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.921 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.921 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:46.921 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:48:46.921 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.921 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.921 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.921 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:46.921 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639024+0000", 2024-09-16T20:48:46.921 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9999999999999997e-07 2024-09-16T20:48:46.921 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.921 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.922 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:46.922 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639026+0000", 2024-09-16T20:48:46.922 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.781e-06 2024-09-16T20:48:46.922 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.922 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.922 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:46.922 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639032+0000", 2024-09-16T20:48:46.922 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7780000000000004e-06 2024-09-16T20:48:46.922 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.922 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.922 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:46.922 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639036+0000", 2024-09-16T20:48:46.923 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6909999999999996e-06 2024-09-16T20:48:46.923 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.923 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.923 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:46.923 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689191+0000", 2024-09-16T20:48:46.923 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050155284000000001 2024-09-16T20:48:46.923 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.923 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.923 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:46.923 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689242+0000", 2024-09-16T20:48:46.923 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0662999999999999e-05 2024-09-16T20:48:46.924 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.924 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.924 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:46.924 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689304+0000", 2024-09-16T20:48:46.924 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1345000000000003e-05 2024-09-16T20:48:46.924 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.924 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:46.924 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.924 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.924 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.925 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3719 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1497088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:46.925 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:48:46.925 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 24.263222349999999, 2024-09-16T20:48:46.925 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050399715999999997, 2024-09-16T20:48:46.925 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:46.925 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:46.925 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:46.925 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:46.925 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:46.925 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:46.925 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3719 2024-09-16T20:48:46.926 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.926 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:46.926 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.926 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:46.926 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:48:46.926 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.926 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.926 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.926 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:46.926 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:48:46.927 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.927 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.927 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.927 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:46.927 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:48:46.927 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9700000000000002e-07 2024-09-16T20:48:46.927 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.927 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.927 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:46.927 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639053+0000", 2024-09-16T20:48:46.927 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2900000000000001e-06 2024-09-16T20:48:46.928 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.928 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.928 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:46.928 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639060+0000", 2024-09-16T20:48:46.928 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7750000000000004e-06 2024-09-16T20:48:46.928 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.928 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.928 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:46.928 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639066+0000", 2024-09-16T20:48:46.928 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0639999999999999e-06 2024-09-16T20:48:46.929 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.929 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.929 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:46.929 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689337+0000", 2024-09-16T20:48:46.929 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050270784999999998 2024-09-16T20:48:46.929 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.929 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.929 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:46.929 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689392+0000", 2024-09-16T20:48:46.929 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4920999999999998e-05 2024-09-16T20:48:46.929 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.929 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.930 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:46.930 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689450+0000", 2024-09-16T20:48:46.930 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8483999999999998e-05 2024-09-16T20:48:46.930 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.930 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:46.930 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.930 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.930 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.930 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3721 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1505280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:46.930 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:48:46.930 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 24.263199218, 2024-09-16T20:48:46.931 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050513608000000002, 2024-09-16T20:48:46.931 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:46.931 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:46.931 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:46.931 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:46.931 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:46.931 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:46.931 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3721 2024-09-16T20:48:46.931 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.931 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:46.931 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.931 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:46.931 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:48:46.932 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.932 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.932 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.932 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:46.932 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:48:46.932 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.932 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.932 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.932 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:46.932 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:48:46.932 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7e-07 2024-09-16T20:48:46.932 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.932 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.933 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:46.933 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639076+0000", 2024-09-16T20:48:46.933 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8899999999999999e-06 2024-09-16T20:48:46.933 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.933 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.933 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:46.933 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639083+0000", 2024-09-16T20:48:46.933 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1210000000000001e-06 2024-09-16T20:48:46.933 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.933 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.933 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:46.933 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639089+0000", 2024-09-16T20:48:46.933 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.643e-06 2024-09-16T20:48:46.934 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.934 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.934 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:46.934 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689474+0000", 2024-09-16T20:48:46.934 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050385679000000003 2024-09-16T20:48:46.934 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.934 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.934 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:46.934 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689533+0000", 2024-09-16T20:48:46.934 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8303000000000001e-05 2024-09-16T20:48:46.934 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.934 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.934 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:46.935 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689587+0000", 2024-09-16T20:48:46.935 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4601999999999997e-05 2024-09-16T20:48:46.935 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.935 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:46.935 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.935 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.935 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.935 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3723 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1513472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:46.935 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:48:46.935 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 24.263168413999999, 2024-09-16T20:48:46.935 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050699691999999998, 2024-09-16T20:48:46.935 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:46.935 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:46.936 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:46.936 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:46.936 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:46.936 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:46.936 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3723 2024-09-16T20:48:46.936 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.936 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:46.936 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.936 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:46.936 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:48:46.936 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.936 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.936 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.936 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:46.937 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:48:46.937 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.937 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.937 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.937 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:46.937 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:48:46.937 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3399999999999999e-07 2024-09-16T20:48:46.937 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.937 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.937 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:46.937 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639107+0000", 2024-09-16T20:48:46.937 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3860000000000001e-06 2024-09-16T20:48:46.937 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.937 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.938 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:46.938 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639114+0000", 2024-09-16T20:48:46.938 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9800000000000001e-06 2024-09-16T20:48:46.938 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.938 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.938 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:46.938 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639129+0000", 2024-09-16T20:48:46.938 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4887e-05 2024-09-16T20:48:46.938 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.938 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.938 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:46.939 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689625+0000", 2024-09-16T20:48:46.939 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050495848000000003 2024-09-16T20:48:46.939 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.939 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.939 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:46.939 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689720+0000", 2024-09-16T20:48:46.939 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4821000000000006e-05 2024-09-16T20:48:46.939 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.939 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.939 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:46.940 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689804+0000", 2024-09-16T20:48:46.940 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4235999999999998e-05 2024-09-16T20:48:46.940 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.940 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:46.940 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.940 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.940 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.940 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3725 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1521664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:46.940 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:48:46.940 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 24.263134818000001, 2024-09-16T20:48:46.940 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050797660000000001, 2024-09-16T20:48:46.941 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:46.941 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:46.941 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:46.941 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:46.941 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:46.941 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:46.941 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3725 2024-09-16T20:48:46.941 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.941 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:46.941 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.941 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:46.941 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:48:46.942 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.942 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.942 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.942 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:46.942 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:48:46.942 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.942 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.942 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.942 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:46.942 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639139+0000", 2024-09-16T20:48:46.942 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8999999999999997e-07 2024-09-16T20:48:46.942 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.943 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.943 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:46.943 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639141+0000", 2024-09-16T20:48:46.943 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.182e-06 2024-09-16T20:48:46.943 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.943 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.943 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:46.943 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639154+0000", 2024-09-16T20:48:46.943 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3626e-05 2024-09-16T20:48:46.943 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.944 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.944 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:46.944 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639160+0000", 2024-09-16T20:48:46.944 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6810000000000003e-06 2024-09-16T20:48:46.944 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.944 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.944 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:46.944 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689831+0000", 2024-09-16T20:48:46.944 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050670907000000001 2024-09-16T20:48:46.944 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.944 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.945 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:46.945 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689882+0000", 2024-09-16T20:48:46.945 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0445999999999998e-05 2024-09-16T20:48:46.945 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.945 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.945 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:46.945 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689936+0000", 2024-09-16T20:48:46.945 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4327999999999999e-05 2024-09-16T20:48:46.945 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.945 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:46.945 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.946 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.946 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.946 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3727 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1529856~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:46.946 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:48:46.946 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 24.263099240999999, 2024-09-16T20:48:46.946 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051016922999999999, 2024-09-16T20:48:46.946 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:46.946 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:46.946 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:46.946 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:46.946 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:46.946 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:46.947 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3727 2024-09-16T20:48:46.947 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.947 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:46.947 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.947 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:46.947 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:48:46.947 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.947 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.947 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.947 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:46.947 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:48:46.948 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.948 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.948 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.948 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:46.948 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:48:46.948 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9999999999999998e-07 2024-09-16T20:48:46.948 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.948 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.948 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:46.948 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639183+0000", 2024-09-16T20:48:46.948 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2399999999999996e-06 2024-09-16T20:48:46.948 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.948 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.949 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:46.949 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639195+0000", 2024-09-16T20:48:46.949 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1113e-05 2024-09-16T20:48:46.949 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.949 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.949 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:46.949 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639206+0000", 2024-09-16T20:48:46.949 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1503e-05 2024-09-16T20:48:46.949 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.949 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.949 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:46.949 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689969+0000", 2024-09-16T20:48:46.949 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050762676 2024-09-16T20:48:46.949 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.950 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.950 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:46.950 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690071+0000", 2024-09-16T20:48:46.950 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102163 2024-09-16T20:48:46.950 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.950 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.950 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:46.950 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690191+0000", 2024-09-16T20:48:46.950 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000119728 2024-09-16T20:48:46.950 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.950 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:46.950 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.950 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.951 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.951 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3730 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1538048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:46.951 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:48:46.951 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 24.263057677999999, 2024-09-16T20:48:46.951 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051131467999999999, 2024-09-16T20:48:46.951 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:46.951 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:46.951 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:46.951 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:46.951 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:46.951 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:46.951 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3730 2024-09-16T20:48:46.951 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.951 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:46.952 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.952 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:46.952 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:48:46.952 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.952 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.952 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.952 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:46.952 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:48:46.952 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.952 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.952 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.952 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:46.952 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639216+0000", 2024-09-16T20:48:46.953 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.39e-07 2024-09-16T20:48:46.953 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.953 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.953 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:46.953 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639218+0000", 2024-09-16T20:48:46.953 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.537e-06 2024-09-16T20:48:46.953 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.953 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.953 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:46.953 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639229+0000", 2024-09-16T20:48:46.953 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1122e-05 2024-09-16T20:48:46.953 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.953 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.953 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:46.954 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639237+0000", 2024-09-16T20:48:46.954 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8509999999999999e-06 2024-09-16T20:48:46.954 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.954 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.954 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:46.954 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690227+0000", 2024-09-16T20:48:46.954 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050990147 2024-09-16T20:48:46.954 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.954 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.954 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:46.954 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690278+0000", 2024-09-16T20:48:46.955 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0223999999999999e-05 2024-09-16T20:48:46.955 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.955 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.955 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:46.955 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690347+0000", 2024-09-16T20:48:46.955 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9147999999999996e-05 2024-09-16T20:48:46.955 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.955 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:46.955 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.955 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.955 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.955 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3731 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1546240~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:46.955 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:48:46.956 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 24.263026969999999, 2024-09-16T20:48:46.956 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051276744999999999, 2024-09-16T20:48:46.956 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:46.956 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:46.956 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:46.956 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:46.956 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:46.956 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:46.956 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3731 2024-09-16T20:48:46.956 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.956 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:46.957 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.957 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:46.957 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:48:46.957 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.957 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.957 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.957 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:46.957 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:48:46.957 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.957 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.957 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.957 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:46.957 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:48:46.958 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9900000000000001e-07 2024-09-16T20:48:46.958 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.958 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.958 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:46.958 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639249+0000", 2024-09-16T20:48:46.958 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3630000000000001e-06 2024-09-16T20:48:46.958 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.958 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.958 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:46.958 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639255+0000", 2024-09-16T20:48:46.958 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4960000000000001e-06 2024-09-16T20:48:46.958 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.958 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.958 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:46.959 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639261+0000", 2024-09-16T20:48:46.959 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2839999999999997e-06 2024-09-16T20:48:46.959 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.959 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.959 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:46.959 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690375+0000", 2024-09-16T20:48:46.959 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051114684000000001 2024-09-16T20:48:46.959 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.959 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.959 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:46.959 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690425+0000", 2024-09-16T20:48:46.959 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9379999999999998e-05 2024-09-16T20:48:46.959 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.959 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.959 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:46.960 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690523+0000", 2024-09-16T20:48:46.960 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8139000000000003e-05 2024-09-16T20:48:46.960 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.960 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:46.960 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.960 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.960 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.960 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3732 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1554432~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:46.960 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:48:46.960 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 24.262981188000001, 2024-09-16T20:48:46.960 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051380096, 2024-09-16T20:48:46.960 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:46.960 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:46.960 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:46.960 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:46.960 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:46.961 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:46.961 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3732 2024-09-16T20:48:46.961 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.961 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:46.961 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.961 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:46.961 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:48:46.961 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.961 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.961 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.961 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:46.961 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:48:46.961 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.961 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.961 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.962 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:46.962 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:48:46.962 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6100000000000001e-07 2024-09-16T20:48:46.962 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.962 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.962 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:46.962 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639295+0000", 2024-09-16T20:48:46.962 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.447e-06 2024-09-16T20:48:46.962 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.962 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.962 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:46.962 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639302+0000", 2024-09-16T20:48:46.962 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0570000000000004e-06 2024-09-16T20:48:46.962 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.962 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.962 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:46.963 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639307+0000", 2024-09-16T20:48:46.963 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1429999999999997e-06 2024-09-16T20:48:46.963 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.963 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.963 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:46.963 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690555+0000", 2024-09-16T20:48:46.963 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051247963000000001 2024-09-16T20:48:46.963 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.963 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.963 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:46.963 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690602+0000", 2024-09-16T20:48:46.963 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7602000000000003e-05 2024-09-16T20:48:46.963 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.964 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.964 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:46.964 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690672+0000", 2024-09-16T20:48:46.964 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9423000000000003e-05 2024-09-16T20:48:46.964 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.964 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:46.964 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.964 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.964 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.964 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3734 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1562624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:46.964 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:48:46.964 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 24.262959538, 2024-09-16T20:48:46.964 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051490610999999999, 2024-09-16T20:48:46.964 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:46.965 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:46.965 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:46.965 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:46.965 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:46.965 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:46.965 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3734 2024-09-16T20:48:46.965 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.965 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:46.965 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.965 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:46.965 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:48:46.965 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.965 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.965 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.966 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:46.966 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:48:46.966 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.966 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.966 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.966 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:46.966 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639314+0000", 2024-09-16T20:48:46.966 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4400000000000001e-07 2024-09-16T20:48:46.966 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.966 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.966 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:46.966 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639316+0000", 2024-09-16T20:48:46.966 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1459999999999999e-06 2024-09-16T20:48:46.967 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.967 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.967 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:46.967 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639322+0000", 2024-09-16T20:48:46.967 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7610000000000004e-06 2024-09-16T20:48:46.967 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.967 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.967 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:46.967 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639327+0000", 2024-09-16T20:48:46.967 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.186e-06 2024-09-16T20:48:46.967 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.967 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.967 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:46.967 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690698+0000", 2024-09-16T20:48:46.968 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051371223000000001 2024-09-16T20:48:46.968 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.968 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.968 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:46.968 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690748+0000", 2024-09-16T20:48:46.968 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0192999999999999e-05 2024-09-16T20:48:46.968 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.968 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.968 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:46.968 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690804+0000", 2024-09-16T20:48:46.968 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5658000000000001e-05 2024-09-16T20:48:46.968 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.968 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:46.968 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.969 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.969 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.969 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3736 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1570816~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:46.969 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:48:46.969 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 24.262940278999999, 2024-09-16T20:48:46.969 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051596528000000003, 2024-09-16T20:48:46.969 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:46.969 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:46.969 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:46.969 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:46.970 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:46.970 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:46.970 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3736 2024-09-16T20:48:46.970 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.970 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:46.970 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.970 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:46.970 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:48:46.970 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.970 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.970 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.970 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:46.971 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:48:46.971 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.971 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.971 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.971 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:46.971 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:48:46.971 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9999999999999999e-07 2024-09-16T20:48:46.971 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.971 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.971 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:46.971 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639335+0000", 2024-09-16T20:48:46.971 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.635e-06 2024-09-16T20:48:46.972 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.972 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.972 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:46.972 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639340+0000", 2024-09-16T20:48:46.972 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5840000000000002e-06 2024-09-16T20:48:46.972 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.972 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.972 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:46.972 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639346+0000", 2024-09-16T20:48:46.972 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1190000000000002e-06 2024-09-16T20:48:46.972 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.973 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.973 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:46.973 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690817+0000", 2024-09-16T20:48:46.973 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051470721999999997 2024-09-16T20:48:46.973 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.973 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.973 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:46.973 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690875+0000", 2024-09-16T20:48:46.973 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7441000000000001e-05 2024-09-16T20:48:46.973 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.973 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.974 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:46.974 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690929+0000", 2024-09-16T20:48:46.974 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4727e-05 2024-09-16T20:48:46.974 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.974 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:46.974 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.974 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.974 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.974 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5434 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:46.974 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:48:46.975 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 24.037678712999998, 2024-09-16T20:48:46.975 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099755831000000003, 2024-09-16T20:48:46.975 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:46.975 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:46.975 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:46.975 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:46.975 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:46.975 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:46.975 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5434 2024-09-16T20:48:46.975 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.975 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:46.976 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.976 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:46.976 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:48:46.976 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.976 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.976 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.976 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:46.976 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:48:46.976 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.976 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.977 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.977 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:46.977 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864595+0000", 2024-09-16T20:48:46.977 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1099999999999995e-07 2024-09-16T20:48:46.977 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.977 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.977 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:46.977 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864597+0000", 2024-09-16T20:48:46.977 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3989999999999998e-06 2024-09-16T20:48:46.977 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.977 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.978 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:46.978 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864603+0000", 2024-09-16T20:48:46.978 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9309999999999996e-06 2024-09-16T20:48:46.978 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.978 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.978 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:46.978 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864607+0000", 2024-09-16T20:48:46.978 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6200000000000001e-06 2024-09-16T20:48:46.978 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.978 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.978 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:46.979 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964164+0000", 2024-09-16T20:48:46.979 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099557162000000005 2024-09-16T20:48:46.979 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.979 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.979 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:46.979 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964244+0000", 2024-09-16T20:48:46.979 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9597000000000004e-05 2024-09-16T20:48:46.979 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.979 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.979 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:46.980 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964350+0000", 2024-09-16T20:48:46.980 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010641099999999999 2024-09-16T20:48:46.980 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.980 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:46.980 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.980 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.980 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.980 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5435 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 20480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:46.980 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:48:46.980 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 24.037648891, 2024-09-16T20:48:46.981 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100897102, 2024-09-16T20:48:46.981 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:46.981 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:46.981 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:46.981 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:46.981 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:46.981 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:46.981 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5435 2024-09-16T20:48:46.981 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.981 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:46.981 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.982 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:46.982 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:48:46.982 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.982 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.982 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.982 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:46.982 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:48:46.982 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.982 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.982 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.982 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:46.982 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:48:46.983 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8000000000000001e-07 2024-09-16T20:48:46.983 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.983 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.983 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:46.983 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864626+0000", 2024-09-16T20:48:46.983 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.745e-06 2024-09-16T20:48:46.983 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.983 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.983 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:46.983 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864633+0000", 2024-09-16T20:48:46.983 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3339999999999998e-06 2024-09-16T20:48:46.983 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.983 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.984 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:46.984 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864636+0000", 2024-09-16T20:48:46.984 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1200000000000002e-06 2024-09-16T20:48:46.984 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.984 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.984 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:46.984 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965299+0000", 2024-09-16T20:48:46.984 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10066352000000001 2024-09-16T20:48:46.984 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.984 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.984 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:46.984 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965390+0000", 2024-09-16T20:48:46.984 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0894000000000001e-05 2024-09-16T20:48:46.985 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.985 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.985 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:46.985 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965521+0000", 2024-09-16T20:48:46.985 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013110899999999999 2024-09-16T20:48:46.985 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.985 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:46.985 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.985 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.985 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.985 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:152 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 327680~32768 out=32768b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:48:46.985 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:48:46.985 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 22.739151949, 2024-09-16T20:48:46.986 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11869405800000001, 2024-09-16T20:48:46.986 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:46.986 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:46.986 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:46.986 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:46.986 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:48:46.986 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:48:46.986 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 152 2024-09-16T20:48:46.986 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.986 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:46.986 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.986 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:46.986 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:48:46.987 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.987 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.987 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.987 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:46.987 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:48:46.987 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.987 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.987 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.987 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:46.987 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163122+0000", 2024-09-16T20:48:46.987 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6400000000000003e-07 2024-09-16T20:48:46.987 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.987 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.987 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:46.988 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163124+0000", 2024-09-16T20:48:46.988 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.565e-06 2024-09-16T20:48:46.988 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.988 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.988 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:46.988 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163136+0000", 2024-09-16T20:48:46.988 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1717e-05 2024-09-16T20:48:46.988 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.988 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.988 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:46.988 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163142+0000", 2024-09-16T20:48:46.988 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1750000000000002e-06 2024-09-16T20:48:46.988 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.989 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.989 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:46.989 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163606+0000", 2024-09-16T20:48:46.989 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00046418799999999998 2024-09-16T20:48:46.989 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.989 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.989 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:46.989 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163656+0000", 2024-09-16T20:48:46.989 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9852999999999999e-05 2024-09-16T20:48:46.989 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.989 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.989 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:46.989 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281815+0000", 2024-09-16T20:48:46.989 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11815909600000001 2024-09-16T20:48:46.990 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.990 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:46.990 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.990 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.990 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.990 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:153 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:48:46.990 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:48:46.990 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 22.739118995999998, 2024-09-16T20:48:46.990 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11903462400000001, 2024-09-16T20:48:46.990 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:46.990 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:46.990 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:46.990 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:46.990 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:48:46.991 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:48:46.991 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 153 2024-09-16T20:48:46.991 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.991 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:46.991 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.991 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:46.991 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:48:46.991 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.991 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.991 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.991 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:46.991 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:48:46.991 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.992 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.992 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.992 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:46.992 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163155+0000", 2024-09-16T20:48:46.992 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.37e-07 2024-09-16T20:48:46.992 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.992 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.992 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:46.992 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163157+0000", 2024-09-16T20:48:46.992 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6010000000000002e-06 2024-09-16T20:48:46.992 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.992 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.992 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:46.992 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163165+0000", 2024-09-16T20:48:46.993 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2619999999999998e-06 2024-09-16T20:48:46.993 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.993 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.993 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:46.993 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163171+0000", 2024-09-16T20:48:46.993 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.711e-06 2024-09-16T20:48:46.993 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.993 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.993 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:46.993 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281868+0000", 2024-09-16T20:48:46.993 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118696622 2024-09-16T20:48:46.993 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.994 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.994 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:46.994 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281969+0000", 2024-09-16T20:48:46.994 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101371 2024-09-16T20:48:46.994 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.994 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.994 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:46.994 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282189+0000", 2024-09-16T20:48:46.994 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00021952000000000001 2024-09-16T20:48:46.994 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.994 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:46.994 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.994 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.995 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.995 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:154 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 360448~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:48:46.995 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:48:46.995 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 22.739087223999999, 2024-09-16T20:48:46.995 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11932035100000001, 2024-09-16T20:48:46.995 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:46.995 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:46.995 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:46.995 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:46.995 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:48:46.995 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:48:46.995 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 154 2024-09-16T20:48:46.995 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.995 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:46.996 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.996 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:46.996 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:48:46.996 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.996 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.996 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.996 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:46.996 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:48:46.996 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:46.996 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.996 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.997 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:46.997 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:48:46.997 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:48:46.997 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.997 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.997 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:46.997 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163189+0000", 2024-09-16T20:48:46.997 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6840000000000001e-06 2024-09-16T20:48:46.997 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.997 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.997 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:46.997 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163196+0000", 2024-09-16T20:48:46.997 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7920000000000004e-06 2024-09-16T20:48:46.997 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.998 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.998 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:46.998 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163201+0000", 2024-09-16T20:48:46.998 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2809999999999999e-06 2024-09-16T20:48:46.998 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.998 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.998 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:46.998 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282219+0000", 2024-09-16T20:48:46.998 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.119018343 2024-09-16T20:48:46.998 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.998 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.998 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:46.998 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282305+0000", 2024-09-16T20:48:46.998 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5865000000000005e-05 2024-09-16T20:48:46.999 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.999 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.999 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:46.999 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282506+0000", 2024-09-16T20:48:46.999 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000200927 2024-09-16T20:48:46.999 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.999 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:46.999 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:46.999 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:46.999 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:46.999 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop(client.4366.0:3088 3.1e e38/23 3:79e5ec01:::rbd_header.110e891a9b2b:head v 38'306, mlcod=36'304)", 2024-09-16T20:48:46.999 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:48:46.999 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 22.738068355999999, 2024-09-16T20:48:46.999 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118102759, 2024-09-16T20:48:47.000 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:47.000 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:47.000 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:47.000 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:47.000 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:47.000 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:47.000 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:48:47.000 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:47.000 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:47.000 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:47.000 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:47.000 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:48:47.000 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:47.000 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:47.000 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:47.001 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:47.001 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:48:47.001 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.85e-07 2024-09-16T20:48:47.001 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:47.001 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:47.001 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:47.001 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164212+0000", 2024-09-16T20:48:47.001 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3999999999999997e-06 2024-09-16T20:48:47.001 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:47.001 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:47.001 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:47.001 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281553+0000", 2024-09-16T20:48:47.001 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11734135499999999 2024-09-16T20:48:47.001 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:47.002 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:47.002 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:47.002 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281567+0000", 2024-09-16T20:48:47.002 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4188999999999999e-05 2024-09-16T20:48:47.002 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:47.002 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:47.002 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:47.002 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281653+0000", 2024-09-16T20:48:47.002 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5385999999999999e-05 2024-09-16T20:48:47.002 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:47.002 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:47.002 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:47.002 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281679+0000", 2024-09-16T20:48:47.002 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6689000000000001e-05 2024-09-16T20:48:47.003 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:47.003 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:47.003 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:47.003 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282277+0000", 2024-09-16T20:48:47.003 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00059798000000000002 2024-09-16T20:48:47.003 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:47.003 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:47.003 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:47.003 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282307+0000", 2024-09-16T20:48:47.003 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9975e-05 2024-09-16T20:48:47.003 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:47.003 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:47.003 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:47.004 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:47.004 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:47.004 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:47.005 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:48:47.006 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:48:47.006 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:48:47.006 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:47.004+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:47.008 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:48:47.008 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:48:47.024 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:48:47.024 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:48:47.042 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:48:47.042 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:48:47.059 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:48:47.059 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:48:47.076 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:48:47.076 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:48:47.121 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:47.121 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:47.127 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:47.126+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:47.130 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:48:47.206 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:47.205+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:47.237 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:47.237 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:47.245 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:48:47.308 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:47.306+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:47.353 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:47.353 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:47.361 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:48:47.445 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:47.444+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:47.509 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:47.509 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:47.509 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:47.507+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:47.517 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:48:47.575 INFO:teuthology.orchestra.run.smithi139.stdout:2148 write 0xd5b9600 thru 0xd5c5aa4 (0xc4a5 bytes) 2024-09-16T20:48:47.577 INFO:teuthology.orchestra.run.smithi139.stdout:2149 read 0x1ce2c00 thru 0x1ce482f (0x1c30 bytes) 2024-09-16T20:48:47.577 INFO:teuthology.orchestra.run.smithi139.stdout:2150 write 0xa09d000 thru 0xa0a0db2 (0x3db3 bytes) 2024-09-16T20:48:47.577 INFO:teuthology.orchestra.run.smithi139.stdout:2151 punch from 0xad92600 to 0xad96600, (0x4000 bytes) 2024-09-16T20:48:47.578 INFO:teuthology.orchestra.run.smithi139.stdout:2152 write 0xa547e00 thru 0xa54a20c (0x240d bytes) 2024-09-16T20:48:47.601 INFO:teuthology.orchestra.run.smithi139.stdout:2153 trunc from 0xd5c5c00 to 0x1275000 2024-09-16T20:48:47.609 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:47.608+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:47.624 INFO:teuthology.orchestra.run.smithi139.stdout:2154 read 0x182a00 thru 0x188cda (0x62db bytes) 2024-09-16T20:48:47.629 INFO:teuthology.orchestra.run.smithi139.stdout:2155 punch from 0x340600 to 0x34c800, (0xc200 bytes) 2024-09-16T20:48:47.629 INFO:teuthology.orchestra.run.smithi139.stdout:2157 read 0xdf6a00 thru 0xdffe9f (0x94a0 bytes) 2024-09-16T20:48:47.629 INFO:teuthology.orchestra.run.smithi139.stdout:2158 punch from 0x22ea00 to 0x239600, (0xac00 bytes) 2024-09-16T20:48:47.629 INFO:teuthology.orchestra.run.smithi139.stdout:2159 read 0x981800 thru 0x985b9f (0x43a0 bytes) 2024-09-16T20:48:47.630 INFO:teuthology.orchestra.run.smithi139.stdout:2160 punch from 0x2ad400 to 0x2b8800, (0xb400 bytes) 2024-09-16T20:48:47.630 INFO:teuthology.orchestra.run.smithi139.stdout:2161 read 0x810a00 thru 0x812bc4 (0x21c5 bytes) 2024-09-16T20:48:47.630 INFO:teuthology.orchestra.run.smithi139.stdout:2163 clone 15 order 18 su 2048 sc 10 2024-09-16T20:48:47.671 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:47.671 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:47.679 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:48:47.710 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:47.709+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:47.786 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:47.786 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:47.794 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:48:47.837 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:47.836+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:47.902 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:47.902 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:47.910 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:48:47.952 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:47.950+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:48.013 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:48.012+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:48.058 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:48.058 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:48.067 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:48:48.114 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:48.114+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:48.213 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:48.214 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:48.214 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:48.213+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:48.222 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:48:48.315 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:48.314+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:48.342 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:48.343 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:48.351 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:48:48.436 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:48.435+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:48.500 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:48.500 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:48.509 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:48:48.517 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:48.516+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:48.570 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T20:48:48.618 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:48.617+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:48.668 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:48.668 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:48.678 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:48:48.718 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4] out_osds: [3, 5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-16T20:48:48.719 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T20:48:48.719 INFO:tasks.thrashosds.thrasher:Adding osd 3 2024-09-16T20:48:48.719 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 3 2024-09-16T20:48:48.722 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:48.721+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:48.806 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:48.806 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:48.818 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:48:48.820 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:48.818+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:48.851 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #13, image image_client.0-clone13 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent14 2024-09-16T20:48:48.897 INFO:teuthology.orchestra.run.smithi139.stdout:2167 punch from 0x63800 to 0x68e00, (0x5600 bytes) 2024-09-16T20:48:48.920 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:48.918+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:48.956 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:48.956 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:48.967 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:48:49.021 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:49.020+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:49.047 INFO:teuthology.orchestra.run.smithi139.stdout:2168 punch from 0x8e8000 to 0x8e9200, (0x1200 bytes) 2024-09-16T20:48:49.078 INFO:teuthology.orchestra.run.smithi139.stdout:2170 read 0x1028400 thru 0x102c25c (0x3e5d bytes) 2024-09-16T20:48:49.086 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T20:48:49.087 INFO:teuthology.orchestra.run.smithi139.stdout:2171 trunc from 0x1275000 to 0x70cb200 2024-09-16T20:48:49.101 INFO:teuthology.orchestra.run.smithi139.stdout:2172 write 0x6d3f800 thru 0x6d41172 (0x1973 bytes) 2024-09-16T20:48:49.101 INFO:teuthology.orchestra.run.smithi139.stdout:2174 read 0x6677200 thru 0x6678bcd (0x19ce bytes) 2024-09-16T20:48:49.101 INFO:teuthology.orchestra.run.smithi139.stdout:2175 punch from 0x466ca00 to 0x4678e00, (0xc400 bytes) 2024-09-16T20:48:49.118 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:49.118 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:49.122 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:49.121+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:49.132 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:48:49.139 INFO:teuthology.orchestra.run.smithi139.stdout:2176 write 0x965c200 thru 0x96653a7 (0x91a8 bytes) 2024-09-16T20:48:49.140 INFO:tasks.ceph.mon.a.smithi029.stderr:2024-09-16T20:48:49.138+0000 7fe0ce96d640 -1 mon.a@0(leader).osd e60 definitely_dead 0 2024-09-16T20:48:49.145 INFO:teuthology.orchestra.run.smithi139.stdout:2178 trunc from 0x9665400 to 0x57ba000 2024-09-16T20:48:49.190 INFO:teuthology.orchestra.run.smithi139.stdout:2179 trunc from 0x57ba000 to 0x7d49c00 2024-09-16T20:48:49.223 INFO:teuthology.orchestra.run.smithi139.stdout:2180 write 0xe862a00 thru 0xe867bc8 (0x51c9 bytes) 2024-09-16T20:48:49.223 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:49.222+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:49.229 INFO:teuthology.orchestra.run.smithi139.stdout:2182 read 0x11f1e00 thru 0x11f849f (0x66a0 bytes) 2024-09-16T20:48:49.232 INFO:teuthology.orchestra.run.smithi139.stdout:2183 punch from 0x3cbb200 to 0x3cc9c00, (0xea00 bytes) 2024-09-16T20:48:49.234 INFO:teuthology.orchestra.run.smithi139.stdout:2187 write 0x4502a00 thru 0x450c0f7 (0x96f8 bytes) 2024-09-16T20:48:49.234 INFO:teuthology.orchestra.run.smithi139.stdout:2188 read 0xe423200 thru 0xe427c62 (0x4a63 bytes) 2024-09-16T20:48:49.239 INFO:teuthology.orchestra.run.smithi139.stdout:2189 trunc from 0xe867c00 to 0xa037200 2024-09-16T20:48:49.264 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:48:49.264 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:48:49.290 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:49.290 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:49.291 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:48:49.291 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:48:49.300 INFO:teuthology.orchestra.run.smithi139.stdout:2191 read 0x9377a00 thru 0x937872a (0xd2b bytes) 2024-09-16T20:48:49.301 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:48:49.305 INFO:teuthology.orchestra.run.smithi139.stdout:2192 trunc from 0xa037200 to 0xdd2ac00 2024-09-16T20:48:49.314 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:48:49.315 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:48:49.318 INFO:teuthology.orchestra.run.smithi139.stdout:2194 punch from 0x6105a00 to 0x610d400, (0x7a00 bytes) 2024-09-16T20:48:49.326 INFO:teuthology.orchestra.run.smithi139.stdout:2195 trunc from 0xdd2ac00 to 0x8703a00 2024-09-16T20:48:49.346 INFO:teuthology.orchestra.run.smithi139.stdout:2197 punch from 0x1e82c00 to 0x1e8ec00, (0xc000 bytes) 2024-09-16T20:48:49.348 INFO:teuthology.orchestra.run.smithi139.stdout:2199 punch from 0x3620a00 to 0x3626200, (0x5800 bytes) 2024-09-16T20:48:49.348 INFO:teuthology.orchestra.run.smithi139.stdout:2202 punch from 0x6c5e800 to 0x6c6ce00, (0xe600 bytes) 2024-09-16T20:48:49.359 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:49.357+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:49.359 INFO:teuthology.orchestra.run.smithi139.stdout:2203 trunc from 0x8703a00 to 0xda600 2024-09-16T20:48:49.425 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:49.424+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:49.452 INFO:teuthology.orchestra.run.smithi139.stdout:2206 punch from 0x1a200 to 0x29400, (0xf200 bytes) 2024-09-16T20:48:49.466 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:49.466 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:48:49.466 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:48:49.466 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:48:49.466 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:49.476 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:48:49.478 INFO:teuthology.orchestra.run.smithi139.stdout:2207 write 0xaadf800 thru 0xaae56b0 (0x5eb1 bytes) 2024-09-16T20:48:49.479 INFO:teuthology.orchestra.run.smithi139.stdout:2208 write 0xb94200 thru 0xb9b642 (0x7443 bytes) 2024-09-16T20:48:49.479 INFO:teuthology.orchestra.run.smithi139.stdout:2211 write 0x5995200 thru 0x59997bf (0x45c0 bytes) 2024-09-16T20:48:49.479 INFO:teuthology.orchestra.run.smithi139.stdout:2214 punch from 0x141a000 to 0x1429200, (0xf200 bytes) 2024-09-16T20:48:49.480 INFO:teuthology.orchestra.run.smithi139.stdout:2220 punch from 0x9615800 to 0x9618000, (0x2800 bytes) 2024-09-16T20:48:49.480 INFO:teuthology.orchestra.run.smithi139.stdout:2223 punch from 0x4184a00 to 0x4189c00, (0x5200 bytes) 2024-09-16T20:48:49.483 INFO:teuthology.orchestra.run.smithi139.stdout:2224 trunc from 0xaae5800 to 0x917d400 2024-09-16T20:48:49.549 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:48:49.549 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:48:49.550 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:49.548+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:49.565 INFO:teuthology.orchestra.run.smithi139.stdout:2227 trunc from 0x917d400 to 0xbf20200 2024-09-16T20:48:49.571 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:48:49.571 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:48:49.590 INFO:teuthology.orchestra.run.smithi139.stdout:2231 write 0xe833400 thru 0xe83bd6f (0x8970 bytes) 2024-09-16T20:48:49.601 INFO:teuthology.orchestra.run.smithi139.stdout:2232 trunc from 0xe83be00 to 0x5b13c00 2024-09-16T20:48:49.605 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:48:49.605 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:48:49.640 INFO:teuthology.orchestra.run.smithi139.stdout:2234 trunc from 0x5b13c00 to 0x7c91800 2024-09-16T20:48:49.649 INFO:teuthology.orchestra.run.smithi139.stdout:2235 punch from 0x35d4a00 to 0x35dca00, (0x8000 bytes) 2024-09-16T20:48:49.650 INFO:teuthology.orchestra.run.smithi139.stdout:2237 read 0x74e5600 thru 0x74e8626 (0x3027 bytes) 2024-09-16T20:48:49.664 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:49.663+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:49.672 INFO:teuthology.orchestra.run.smithi139.stdout:2238 write 0xb756200 thru 0xb759c00 (0x3a01 bytes) 2024-09-16T20:48:49.673 INFO:teuthology.orchestra.run.smithi139.stdout:2241 read 0x54a6200 thru 0x54b490b (0xe70c bytes) 2024-09-16T20:48:49.673 INFO:teuthology.orchestra.run.smithi139.stdout:2242 write 0x74c800 thru 0x753baa (0x73ab bytes) 2024-09-16T20:48:49.677 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:49.677 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:48:49.677 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:48:49.677 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:48:49.677 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.677 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3709 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1464320~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:49.677 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.638971+0000", 2024-09-16T20:48:49.677 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 27.036214092000002, 2024-09-16T20:48:49.677 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049858670000000001, 2024-09-16T20:48:49.677 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:49.677 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:49.677 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:49.678 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:49.678 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:49.678 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:49.678 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3709 2024-09-16T20:48:49.678 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.678 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:49.678 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.678 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:49.678 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638971+0000", 2024-09-16T20:48:49.678 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:49.678 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.678 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.678 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:49.678 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638971+0000", 2024-09-16T20:48:49.679 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:49.679 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.679 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.679 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:49.679 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638972+0000", 2024-09-16T20:48:49.679 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0400000000000002e-07 2024-09-16T20:48:49.679 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.679 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.679 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:49.679 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638973+0000", 2024-09-16T20:48:49.679 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.578e-06 2024-09-16T20:48:49.679 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.679 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.679 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:49.680 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638979+0000", 2024-09-16T20:48:49.680 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.502e-06 2024-09-16T20:48:49.680 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.680 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.680 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:49.680 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638983+0000", 2024-09-16T20:48:49.680 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2690000000000001e-06 2024-09-16T20:48:49.680 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.680 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.680 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:49.680 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688628+0000", 2024-09-16T20:48:49.680 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049644547999999997 2024-09-16T20:48:49.680 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.681 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.681 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:49.681 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688718+0000", 2024-09-16T20:48:49.681 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0802000000000002e-05 2024-09-16T20:48:49.681 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.681 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.681 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:49.681 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688830+0000", 2024-09-16T20:48:49.681 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000111667 2024-09-16T20:48:49.681 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:49.681 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:49.681 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:49.681 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.682 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.682 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3712 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1472512~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:49.682 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:48:49.682 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 27.036197563999998, 2024-09-16T20:48:49.682 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050004305999999998, 2024-09-16T20:48:49.682 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:49.682 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:49.682 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:49.682 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:49.682 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:49.682 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:49.682 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3712 2024-09-16T20:48:49.682 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.682 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:49.683 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.683 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:49.683 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:48:49.683 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:49.683 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.683 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.683 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:49.683 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:48:49.683 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:49.683 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.683 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.683 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:49.683 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:48:49.684 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0199999999999998e-07 2024-09-16T20:48:49.684 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.684 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.684 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:49.684 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638990+0000", 2024-09-16T20:48:49.684 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.618e-06 2024-09-16T20:48:49.684 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.684 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.684 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:49.684 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638995+0000", 2024-09-16T20:48:49.684 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3940000000000004e-06 2024-09-16T20:48:49.684 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.684 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.685 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:49.685 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639000+0000", 2024-09-16T20:48:49.685 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9309999999999999e-06 2024-09-16T20:48:49.685 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.685 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.685 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:49.685 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688867+0000", 2024-09-16T20:48:49.685 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049866955999999997 2024-09-16T20:48:49.685 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.685 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.685 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:49.685 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688933+0000", 2024-09-16T20:48:49.685 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5359999999999998e-05 2024-09-16T20:48:49.685 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.686 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.686 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:49.686 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688992+0000", 2024-09-16T20:48:49.686 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9744999999999998e-05 2024-09-16T20:48:49.686 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:49.686 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:49.686 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:49.686 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.686 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.686 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3714 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1480704~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:49.686 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:48:49.686 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 27.036179974, 2024-09-16T20:48:49.686 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050152552000000003, 2024-09-16T20:48:49.687 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:49.687 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:49.687 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:49.687 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:49.687 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:49.687 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:49.687 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3714 2024-09-16T20:48:49.687 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.687 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:49.687 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.687 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:49.687 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:48:49.687 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:49.688 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.688 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.688 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:49.688 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:48:49.688 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:49.688 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.688 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.688 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:49.688 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:48:49.688 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0499999999999999e-07 2024-09-16T20:48:49.688 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.688 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.688 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:49.689 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639007+0000", 2024-09-16T20:48:49.689 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5340000000000001e-06 2024-09-16T20:48:49.689 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.689 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.689 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:49.689 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639013+0000", 2024-09-16T20:48:49.689 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3260000000000003e-06 2024-09-16T20:48:49.689 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.689 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.689 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:49.689 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639018+0000", 2024-09-16T20:48:49.689 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2850000000000002e-06 2024-09-16T20:48:49.689 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.690 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.690 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:49.690 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689026+0000", 2024-09-16T20:48:49.690 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.05000798 2024-09-16T20:48:49.690 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.690 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.690 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:49.690 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689077+0000", 2024-09-16T20:48:49.690 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1063999999999998e-05 2024-09-16T20:48:49.690 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.690 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.690 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:49.690 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689158+0000", 2024-09-16T20:48:49.690 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1057999999999995e-05 2024-09-16T20:48:49.691 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:49.691 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:49.691 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:49.691 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.691 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.691 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3716 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1488896~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:49.691 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:48:49.691 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 27.036162164, 2024-09-16T20:48:49.691 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050280142, 2024-09-16T20:48:49.691 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:49.691 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:49.691 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:49.691 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:49.692 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:49.692 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:49.692 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3716 2024-09-16T20:48:49.692 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.692 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:49.692 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.692 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:49.692 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:48:49.692 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:49.692 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.692 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.692 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:49.692 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:48:49.693 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:49.693 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.693 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.693 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:49.693 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639024+0000", 2024-09-16T20:48:49.693 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9999999999999997e-07 2024-09-16T20:48:49.693 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.693 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.693 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:49.693 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639026+0000", 2024-09-16T20:48:49.693 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.781e-06 2024-09-16T20:48:49.693 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.693 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.693 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:49.694 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639032+0000", 2024-09-16T20:48:49.694 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7780000000000004e-06 2024-09-16T20:48:49.694 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.694 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.694 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:49.694 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639036+0000", 2024-09-16T20:48:49.694 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6909999999999996e-06 2024-09-16T20:48:49.694 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.694 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.694 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:49.694 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689191+0000", 2024-09-16T20:48:49.694 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050155284000000001 2024-09-16T20:48:49.694 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.695 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.695 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:49.695 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689242+0000", 2024-09-16T20:48:49.695 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0662999999999999e-05 2024-09-16T20:48:49.695 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.695 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.695 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:49.695 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689304+0000", 2024-09-16T20:48:49.695 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1345000000000003e-05 2024-09-16T20:48:49.695 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:49.695 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:49.695 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:49.695 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.695 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.696 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3719 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1497088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:49.696 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:48:49.696 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 27.036134885999999, 2024-09-16T20:48:49.696 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050399715999999997, 2024-09-16T20:48:49.696 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:49.696 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:49.696 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:49.696 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:49.696 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:49.696 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:49.696 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3719 2024-09-16T20:48:49.696 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.696 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:49.697 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.697 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:49.697 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:48:49.697 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:49.697 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.697 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.697 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:49.697 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:48:49.697 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:49.697 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.697 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.697 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:49.697 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:48:49.698 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9700000000000002e-07 2024-09-16T20:48:49.698 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.698 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.698 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:49.698 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639053+0000", 2024-09-16T20:48:49.698 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2900000000000001e-06 2024-09-16T20:48:49.698 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.698 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.698 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:49.698 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639060+0000", 2024-09-16T20:48:49.698 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7750000000000004e-06 2024-09-16T20:48:49.698 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.698 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.698 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:49.699 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639066+0000", 2024-09-16T20:48:49.699 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0639999999999999e-06 2024-09-16T20:48:49.699 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.699 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.699 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:49.699 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689337+0000", 2024-09-16T20:48:49.699 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050270784999999998 2024-09-16T20:48:49.699 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.699 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.699 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:49.699 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689392+0000", 2024-09-16T20:48:49.699 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4920999999999998e-05 2024-09-16T20:48:49.699 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.699 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.700 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:49.700 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689450+0000", 2024-09-16T20:48:49.700 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8483999999999998e-05 2024-09-16T20:48:49.700 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:49.700 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:49.700 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:49.700 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.700 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.700 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3721 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1505280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:49.700 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:48:49.700 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 27.036111754, 2024-09-16T20:48:49.700 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050513608000000002, 2024-09-16T20:48:49.700 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:49.701 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:49.701 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:49.701 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:49.701 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:49.701 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:49.701 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3721 2024-09-16T20:48:49.701 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.701 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:49.701 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.701 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:49.701 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:48:49.701 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:49.701 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.701 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.702 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:49.702 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:48:49.702 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:49.702 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.702 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.702 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:49.702 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:48:49.702 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7e-07 2024-09-16T20:48:49.702 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.702 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.702 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:49.702 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639076+0000", 2024-09-16T20:48:49.702 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8899999999999999e-06 2024-09-16T20:48:49.702 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.703 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.703 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:49.703 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639083+0000", 2024-09-16T20:48:49.703 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1210000000000001e-06 2024-09-16T20:48:49.703 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.703 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.703 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:49.703 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639089+0000", 2024-09-16T20:48:49.703 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.643e-06 2024-09-16T20:48:49.703 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.703 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.703 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:49.703 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689474+0000", 2024-09-16T20:48:49.703 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050385679000000003 2024-09-16T20:48:49.704 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.704 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.704 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:49.704 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689533+0000", 2024-09-16T20:48:49.704 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8303000000000001e-05 2024-09-16T20:48:49.704 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.704 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.704 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:49.704 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689587+0000", 2024-09-16T20:48:49.704 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4601999999999997e-05 2024-09-16T20:48:49.704 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:49.704 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:49.704 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:49.705 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.705 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.705 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3723 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1513472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:49.705 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:48:49.705 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 27.036080949999999, 2024-09-16T20:48:49.705 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050699691999999998, 2024-09-16T20:48:49.705 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:49.705 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:49.705 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:49.705 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:49.705 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:49.705 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:49.705 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3723 2024-09-16T20:48:49.706 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.706 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:49.706 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.706 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:49.706 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:48:49.706 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:49.706 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.706 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.706 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:49.706 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:48:49.706 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:49.706 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.706 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.707 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:49.707 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:48:49.707 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3399999999999999e-07 2024-09-16T20:48:49.707 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.707 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.707 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:49.707 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639107+0000", 2024-09-16T20:48:49.707 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3860000000000001e-06 2024-09-16T20:48:49.707 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.707 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.707 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:49.707 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639114+0000", 2024-09-16T20:48:49.707 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9800000000000001e-06 2024-09-16T20:48:49.708 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.708 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.708 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:49.708 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639129+0000", 2024-09-16T20:48:49.708 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4887e-05 2024-09-16T20:48:49.708 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.708 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.708 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:49.708 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689625+0000", 2024-09-16T20:48:49.708 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050495848000000003 2024-09-16T20:48:49.708 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.708 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.708 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:49.708 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689720+0000", 2024-09-16T20:48:49.709 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4821000000000006e-05 2024-09-16T20:48:49.709 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.709 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.709 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:49.709 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689804+0000", 2024-09-16T20:48:49.709 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4235999999999998e-05 2024-09-16T20:48:49.709 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:49.709 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:49.709 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:49.709 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.709 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.709 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3725 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1521664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:49.709 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:48:49.710 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 27.036047354000001, 2024-09-16T20:48:49.710 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050797660000000001, 2024-09-16T20:48:49.710 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:49.710 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:49.710 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:49.710 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:49.710 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:49.710 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:49.710 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3725 2024-09-16T20:48:49.710 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.710 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:49.710 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.710 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:49.711 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:48:49.711 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:49.711 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.711 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.711 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:49.711 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:48:49.711 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:49.711 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.711 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.711 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:49.711 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639139+0000", 2024-09-16T20:48:49.711 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8999999999999997e-07 2024-09-16T20:48:49.711 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.711 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.712 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:49.712 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639141+0000", 2024-09-16T20:48:49.712 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.182e-06 2024-09-16T20:48:49.712 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.712 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.712 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:49.712 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639154+0000", 2024-09-16T20:48:49.712 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3626e-05 2024-09-16T20:48:49.712 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.712 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.712 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:49.712 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639160+0000", 2024-09-16T20:48:49.712 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6810000000000003e-06 2024-09-16T20:48:49.713 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.713 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.713 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:49.713 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689831+0000", 2024-09-16T20:48:49.713 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050670907000000001 2024-09-16T20:48:49.713 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.713 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.713 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:49.713 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689882+0000", 2024-09-16T20:48:49.713 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0445999999999998e-05 2024-09-16T20:48:49.713 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.713 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.713 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:49.713 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689936+0000", 2024-09-16T20:48:49.714 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4327999999999999e-05 2024-09-16T20:48:49.714 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:49.714 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:49.714 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:49.714 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.714 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.714 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3727 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1529856~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:49.714 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:48:49.714 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 27.036011776999999, 2024-09-16T20:48:49.714 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051016922999999999, 2024-09-16T20:48:49.714 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:49.714 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:49.714 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:49.715 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:49.715 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:49.715 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:49.715 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3727 2024-09-16T20:48:49.715 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.715 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:49.715 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.715 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:49.715 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:48:49.715 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:49.715 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.715 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.715 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:49.715 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:48:49.716 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:49.716 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.716 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.716 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:49.716 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:48:49.716 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9999999999999998e-07 2024-09-16T20:48:49.716 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.716 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.716 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:49.716 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639183+0000", 2024-09-16T20:48:49.716 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2399999999999996e-06 2024-09-16T20:48:49.716 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.716 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.717 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:49.717 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639195+0000", 2024-09-16T20:48:49.717 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1113e-05 2024-09-16T20:48:49.717 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.717 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.717 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:49.717 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639206+0000", 2024-09-16T20:48:49.717 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1503e-05 2024-09-16T20:48:49.717 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.717 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.717 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:49.717 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689969+0000", 2024-09-16T20:48:49.717 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050762676 2024-09-16T20:48:49.717 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.717 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.718 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:49.718 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690071+0000", 2024-09-16T20:48:49.718 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102163 2024-09-16T20:48:49.718 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.718 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.718 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:49.718 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690191+0000", 2024-09-16T20:48:49.718 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000119728 2024-09-16T20:48:49.718 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:49.718 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:49.718 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:49.718 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.718 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.719 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3730 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1538048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:49.719 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:48:49.719 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 27.035970213999999, 2024-09-16T20:48:49.719 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051131467999999999, 2024-09-16T20:48:49.719 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:49.719 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:49.719 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:49.719 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:49.719 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:49.719 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:49.719 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3730 2024-09-16T20:48:49.719 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.719 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:49.719 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.720 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:49.720 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:48:49.720 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:49.720 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.720 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.720 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:49.720 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:48:49.720 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:49.720 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.720 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.720 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:49.720 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639216+0000", 2024-09-16T20:48:49.720 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.39e-07 2024-09-16T20:48:49.721 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.721 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.721 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:49.721 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639218+0000", 2024-09-16T20:48:49.721 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.537e-06 2024-09-16T20:48:49.721 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.721 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.721 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:49.721 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639229+0000", 2024-09-16T20:48:49.721 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1122e-05 2024-09-16T20:48:49.721 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.721 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.721 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:49.722 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639237+0000", 2024-09-16T20:48:49.722 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8509999999999999e-06 2024-09-16T20:48:49.722 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.722 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.722 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:49.722 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690227+0000", 2024-09-16T20:48:49.722 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050990147 2024-09-16T20:48:49.722 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.722 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.722 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:49.722 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690278+0000", 2024-09-16T20:48:49.722 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0223999999999999e-05 2024-09-16T20:48:49.722 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.722 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.723 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:49.723 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690347+0000", 2024-09-16T20:48:49.723 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9147999999999996e-05 2024-09-16T20:48:49.723 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:49.723 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:49.723 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:49.723 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.723 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.723 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3731 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1546240~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:49.723 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:48:49.723 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 27.035939505999998, 2024-09-16T20:48:49.723 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051276744999999999, 2024-09-16T20:48:49.723 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:49.724 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:49.724 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:49.724 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:49.724 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:49.724 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:49.724 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3731 2024-09-16T20:48:49.724 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.724 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:49.724 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.724 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:49.724 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:48:49.724 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:49.724 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.724 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.725 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:49.725 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:48:49.725 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:49.725 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.725 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.725 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:49.725 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:48:49.725 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9900000000000001e-07 2024-09-16T20:48:49.725 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.725 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.725 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:49.725 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639249+0000", 2024-09-16T20:48:49.725 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3630000000000001e-06 2024-09-16T20:48:49.726 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.726 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.726 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:49.726 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639255+0000", 2024-09-16T20:48:49.726 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4960000000000001e-06 2024-09-16T20:48:49.726 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.726 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.726 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:49.726 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639261+0000", 2024-09-16T20:48:49.726 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2839999999999997e-06 2024-09-16T20:48:49.726 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.726 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.726 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:49.727 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690375+0000", 2024-09-16T20:48:49.727 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051114684000000001 2024-09-16T20:48:49.727 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.727 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.727 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:49.727 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690425+0000", 2024-09-16T20:48:49.727 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9379999999999998e-05 2024-09-16T20:48:49.727 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.727 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.727 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:49.727 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690523+0000", 2024-09-16T20:48:49.727 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8139000000000003e-05 2024-09-16T20:48:49.727 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:49.727 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:49.728 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:49.728 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.728 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.728 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3732 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1554432~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:49.728 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:48:49.728 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 27.035893724000001, 2024-09-16T20:48:49.728 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051380096, 2024-09-16T20:48:49.728 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:49.728 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:49.728 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:49.728 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:49.728 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:49.728 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:49.728 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3732 2024-09-16T20:48:49.729 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.729 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:49.729 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.729 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:49.729 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:48:49.729 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:49.729 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.729 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.729 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:49.729 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:48:49.729 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:49.729 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.729 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.730 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:49.730 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:48:49.730 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6100000000000001e-07 2024-09-16T20:48:49.730 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.730 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.730 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:49.730 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639295+0000", 2024-09-16T20:48:49.730 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.447e-06 2024-09-16T20:48:49.730 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.730 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.730 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:49.730 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639302+0000", 2024-09-16T20:48:49.730 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0570000000000004e-06 2024-09-16T20:48:49.730 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.731 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.731 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:49.731 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639307+0000", 2024-09-16T20:48:49.731 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1429999999999997e-06 2024-09-16T20:48:49.731 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.731 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.731 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:49.731 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690555+0000", 2024-09-16T20:48:49.731 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051247963000000001 2024-09-16T20:48:49.731 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.731 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.731 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:49.731 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690602+0000", 2024-09-16T20:48:49.731 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7602000000000003e-05 2024-09-16T20:48:49.732 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.732 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.732 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:49.732 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690672+0000", 2024-09-16T20:48:49.732 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9423000000000003e-05 2024-09-16T20:48:49.732 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:49.732 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:49.732 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:49.732 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.732 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.732 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3734 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1562624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:49.732 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:48:49.732 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 27.035872074, 2024-09-16T20:48:49.732 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051490610999999999, 2024-09-16T20:48:49.733 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:49.733 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:49.733 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:49.733 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:49.733 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:49.733 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:49.733 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3734 2024-09-16T20:48:49.733 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.733 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:49.733 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.733 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:49.733 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:48:49.733 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:49.733 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.734 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.734 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:49.734 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:48:49.734 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:49.734 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.734 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.734 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:49.734 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639314+0000", 2024-09-16T20:48:49.734 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4400000000000001e-07 2024-09-16T20:48:49.734 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.734 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.734 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:49.734 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639316+0000", 2024-09-16T20:48:49.735 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1459999999999999e-06 2024-09-16T20:48:49.735 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.735 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.735 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:49.735 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639322+0000", 2024-09-16T20:48:49.735 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7610000000000004e-06 2024-09-16T20:48:49.735 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.735 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.735 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:49.735 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639327+0000", 2024-09-16T20:48:49.735 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.186e-06 2024-09-16T20:48:49.735 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.735 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.735 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:49.736 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690698+0000", 2024-09-16T20:48:49.736 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051371223000000001 2024-09-16T20:48:49.736 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.736 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.736 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:49.736 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690748+0000", 2024-09-16T20:48:49.736 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0192999999999999e-05 2024-09-16T20:48:49.736 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.736 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.736 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:49.736 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690804+0000", 2024-09-16T20:48:49.736 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5658000000000001e-05 2024-09-16T20:48:49.736 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:49.736 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:49.737 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:49.737 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.737 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.737 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3736 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1570816~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:49.737 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:48:49.737 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 27.035852814999998, 2024-09-16T20:48:49.737 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051596528000000003, 2024-09-16T20:48:49.737 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:49.737 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:49.737 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:49.737 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:49.737 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:49.737 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:49.738 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3736 2024-09-16T20:48:49.738 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.738 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:49.738 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.738 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:49.738 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:48:49.738 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:49.738 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.738 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.738 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:49.738 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:48:49.738 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:49.738 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.738 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.739 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:49.739 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:48:49.739 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9999999999999999e-07 2024-09-16T20:48:49.739 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.739 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.739 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:49.739 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639335+0000", 2024-09-16T20:48:49.739 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.635e-06 2024-09-16T20:48:49.739 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.739 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.739 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:49.739 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639340+0000", 2024-09-16T20:48:49.739 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5840000000000002e-06 2024-09-16T20:48:49.739 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.740 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.740 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:49.740 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639346+0000", 2024-09-16T20:48:49.740 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1190000000000002e-06 2024-09-16T20:48:49.740 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.740 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.740 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:49.740 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690817+0000", 2024-09-16T20:48:49.740 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051470721999999997 2024-09-16T20:48:49.740 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.740 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.740 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:49.740 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690875+0000", 2024-09-16T20:48:49.740 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7441000000000001e-05 2024-09-16T20:48:49.741 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.741 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.741 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:49.741 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690929+0000", 2024-09-16T20:48:49.741 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4727e-05 2024-09-16T20:48:49.741 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:49.741 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:49.741 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:49.741 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.741 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.742 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5434 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:49.742 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:48:49.742 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 26.810591249000002, 2024-09-16T20:48:49.742 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099755831000000003, 2024-09-16T20:48:49.742 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:49.742 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:49.742 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:49.742 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:49.742 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:49.742 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:49.742 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5434 2024-09-16T20:48:49.742 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.742 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:49.743 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.743 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:49.743 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:48:49.743 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:49.743 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.743 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.743 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:49.743 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:48:49.743 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:49.743 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.743 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.743 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:49.743 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864595+0000", 2024-09-16T20:48:49.743 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1099999999999995e-07 2024-09-16T20:48:49.744 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.744 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.744 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:49.744 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864597+0000", 2024-09-16T20:48:49.744 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3989999999999998e-06 2024-09-16T20:48:49.744 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.744 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.744 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:49.744 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864603+0000", 2024-09-16T20:48:49.744 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9309999999999996e-06 2024-09-16T20:48:49.744 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.744 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.744 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:49.745 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864607+0000", 2024-09-16T20:48:49.745 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6200000000000001e-06 2024-09-16T20:48:49.745 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.745 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.745 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:49.745 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964164+0000", 2024-09-16T20:48:49.745 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099557162000000005 2024-09-16T20:48:49.745 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.745 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.745 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:49.745 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964244+0000", 2024-09-16T20:48:49.745 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9597000000000004e-05 2024-09-16T20:48:49.745 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.746 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.746 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:49.746 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964350+0000", 2024-09-16T20:48:49.746 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010641099999999999 2024-09-16T20:48:49.746 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:49.746 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:49.746 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:49.746 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.746 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.746 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5435 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 20480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:49.746 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:48:49.746 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 26.810561427, 2024-09-16T20:48:49.746 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100897102, 2024-09-16T20:48:49.746 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:49.746 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:49.747 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:49.747 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:49.747 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:49.747 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:49.747 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5435 2024-09-16T20:48:49.747 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.747 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:49.747 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.747 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:49.747 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:48:49.747 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:49.747 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.747 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.747 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:49.748 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:48:49.748 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:49.748 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.748 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.748 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:49.748 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:48:49.748 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8000000000000001e-07 2024-09-16T20:48:49.748 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.748 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.748 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:49.748 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864626+0000", 2024-09-16T20:48:49.748 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.745e-06 2024-09-16T20:48:49.748 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.749 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.749 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:49.749 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864633+0000", 2024-09-16T20:48:49.749 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3339999999999998e-06 2024-09-16T20:48:49.749 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.749 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.749 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:49.749 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864636+0000", 2024-09-16T20:48:49.749 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1200000000000002e-06 2024-09-16T20:48:49.749 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.749 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.749 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:49.749 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965299+0000", 2024-09-16T20:48:49.749 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10066352000000001 2024-09-16T20:48:49.750 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.750 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.750 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:49.750 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965390+0000", 2024-09-16T20:48:49.750 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0894000000000001e-05 2024-09-16T20:48:49.750 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.750 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.750 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:49.750 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965521+0000", 2024-09-16T20:48:49.750 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013110899999999999 2024-09-16T20:48:49.750 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:49.750 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:49.750 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:49.751 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.751 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.751 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:152 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 327680~32768 out=32768b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:48:49.751 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:48:49.751 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 25.512064485, 2024-09-16T20:48:49.751 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11869405800000001, 2024-09-16T20:48:49.751 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:49.751 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:49.751 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:49.751 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:49.751 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:48:49.751 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:48:49.751 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 152 2024-09-16T20:48:49.752 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.752 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:49.752 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.752 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:49.752 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:48:49.752 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:49.752 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.752 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.752 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:49.752 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:48:49.752 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:49.752 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.752 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.752 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:49.753 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163122+0000", 2024-09-16T20:48:49.753 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6400000000000003e-07 2024-09-16T20:48:49.753 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.753 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.753 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:49.753 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163124+0000", 2024-09-16T20:48:49.753 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.565e-06 2024-09-16T20:48:49.753 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.753 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.753 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:49.753 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163136+0000", 2024-09-16T20:48:49.753 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1717e-05 2024-09-16T20:48:49.753 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.754 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.754 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:49.754 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163142+0000", 2024-09-16T20:48:49.754 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1750000000000002e-06 2024-09-16T20:48:49.754 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.754 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.754 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:49.754 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163606+0000", 2024-09-16T20:48:49.754 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00046418799999999998 2024-09-16T20:48:49.754 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.754 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.754 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:49.754 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163656+0000", 2024-09-16T20:48:49.754 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9852999999999999e-05 2024-09-16T20:48:49.755 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.755 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.755 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:49.755 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281815+0000", 2024-09-16T20:48:49.755 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11815909600000001 2024-09-16T20:48:49.755 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:49.755 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:49.755 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:49.755 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.755 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.755 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:153 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:48:49.755 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:48:49.755 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 25.512031532000002, 2024-09-16T20:48:49.756 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11903462400000001, 2024-09-16T20:48:49.756 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:49.756 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:49.756 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:49.756 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:49.756 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:48:49.756 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:48:49.756 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 153 2024-09-16T20:48:49.756 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.756 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:49.756 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.756 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:49.756 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:48:49.757 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:49.757 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.757 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.757 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:49.757 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:48:49.757 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:49.757 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.757 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.757 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:49.757 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163155+0000", 2024-09-16T20:48:49.757 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.37e-07 2024-09-16T20:48:49.757 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.757 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.757 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:49.758 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163157+0000", 2024-09-16T20:48:49.758 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6010000000000002e-06 2024-09-16T20:48:49.758 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.758 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.758 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:49.758 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163165+0000", 2024-09-16T20:48:49.758 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2619999999999998e-06 2024-09-16T20:48:49.758 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.758 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.758 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:49.758 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163171+0000", 2024-09-16T20:48:49.758 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.711e-06 2024-09-16T20:48:49.758 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.759 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.759 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:49.759 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281868+0000", 2024-09-16T20:48:49.759 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118696622 2024-09-16T20:48:49.759 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.759 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.759 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:49.759 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281969+0000", 2024-09-16T20:48:49.759 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101371 2024-09-16T20:48:49.759 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.759 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.759 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:49.759 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282189+0000", 2024-09-16T20:48:49.759 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00021952000000000001 2024-09-16T20:48:49.760 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:49.760 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:49.760 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:49.760 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.760 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.760 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:154 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 360448~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:48:49.760 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:48:49.760 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 25.511999759999998, 2024-09-16T20:48:49.760 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11932035100000001, 2024-09-16T20:48:49.760 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:49.760 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:49.760 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:49.760 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:49.761 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:48:49.761 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:48:49.761 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 154 2024-09-16T20:48:49.761 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.761 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:49.761 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.761 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:49.761 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:48:49.761 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:49.761 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.761 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.761 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:49.761 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:48:49.761 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:49.762 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.762 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.762 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:49.762 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:48:49.762 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:48:49.762 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.762 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.762 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:49.762 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163189+0000", 2024-09-16T20:48:49.762 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6840000000000001e-06 2024-09-16T20:48:49.762 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.762 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.762 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:49.762 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163196+0000", 2024-09-16T20:48:49.763 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7920000000000004e-06 2024-09-16T20:48:49.763 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.763 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.763 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:49.763 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163201+0000", 2024-09-16T20:48:49.763 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2809999999999999e-06 2024-09-16T20:48:49.763 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.763 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.763 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:49.763 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282219+0000", 2024-09-16T20:48:49.763 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.119018343 2024-09-16T20:48:49.763 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.763 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.764 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:49.764 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282305+0000", 2024-09-16T20:48:49.764 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5865000000000005e-05 2024-09-16T20:48:49.764 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.764 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.764 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:49.764 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282506+0000", 2024-09-16T20:48:49.764 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000200927 2024-09-16T20:48:49.764 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:49.764 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:49.764 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:49.764 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.764 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.764 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop(client.4366.0:3088 3.1e e38/23 3:79e5ec01:::rbd_header.110e891a9b2b:head v 38'306, mlcod=36'304)", 2024-09-16T20:48:49.765 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:48:49.765 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 25.510980891999999, 2024-09-16T20:48:49.765 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118102759, 2024-09-16T20:48:49.765 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:49.765 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:49.765 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:49.765 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:49.765 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.765 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:49.765 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:48:49.765 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:49.765 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.765 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.765 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:49.766 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:48:49.766 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:49.766 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.766 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.766 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:49.766 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:48:49.766 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.85e-07 2024-09-16T20:48:49.766 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.766 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.766 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:49.766 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164212+0000", 2024-09-16T20:48:49.766 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3999999999999997e-06 2024-09-16T20:48:49.766 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.766 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.767 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:49.767 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281553+0000", 2024-09-16T20:48:49.767 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11734135499999999 2024-09-16T20:48:49.767 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.767 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.767 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:49.767 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281567+0000", 2024-09-16T20:48:49.767 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4188999999999999e-05 2024-09-16T20:48:49.767 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.767 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.767 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:49.767 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281653+0000", 2024-09-16T20:48:49.767 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5385999999999999e-05 2024-09-16T20:48:49.767 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.768 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.768 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:49.768 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281679+0000", 2024-09-16T20:48:49.768 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6689000000000001e-05 2024-09-16T20:48:49.768 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.768 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.768 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:49.768 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282277+0000", 2024-09-16T20:48:49.768 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00059798000000000002 2024-09-16T20:48:49.768 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:49.768 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:49.768 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:49.768 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282307+0000", 2024-09-16T20:48:49.769 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9975e-05 2024-09-16T20:48:49.769 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:49.769 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:49.769 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:49.769 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:49.769 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:49.769 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:49.770 INFO:teuthology.orchestra.run.smithi139.stdout:2244 trunc from 0xb759e00 to 0x2bd400 2024-09-16T20:48:49.771 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:48:49.771 INFO:teuthology.orchestra.run.smithi029.stderr:noscrub is unset 2024-09-16T20:48:49.771 INFO:teuthology.orchestra.run.smithi029.stderr:marked in osd.3. 2024-09-16T20:48:49.773 INFO:tasks.thrashosds.thrasher:Added osd 3 2024-09-16T20:48:49.773 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:49.769+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:49.871 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:49.870+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:49.880 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:49.880 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:48:49.880 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:49.880 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:8039 3.1e 3.ac44ff9e (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:48:49.880 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:48:49.880 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.17207336000000001, 2024-09-16T20:48:49.880 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.17209366000000001, 2024-09-16T20:48:49.880 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:49.881 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:49.881 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "waiting for peered", 2024-09-16T20:48:49.881 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:49.881 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:48:49.881 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:48:49.881 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8039 2024-09-16T20:48:49.881 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:49.881 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:49.881 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:49.881 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:49.881 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:48:49.881 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:49.881 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:49.882 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:49.882 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:49.882 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:48:49.882 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:49.882 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:49.882 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:49.882 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:49.882 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707674+0000", 2024-09-16T20:48:49.882 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6530000000000001e-06 2024-09-16T20:48:49.882 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:49.882 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:49.882 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:49.882 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707681+0000", 2024-09-16T20:48:49.882 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9639999999999997e-06 2024-09-16T20:48:49.883 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:49.883 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:49.883 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:49.883 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707700+0000", 2024-09-16T20:48:49.883 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8674000000000001e-05 2024-09-16T20:48:49.883 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:49.883 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:49.883 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:49.883 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707708+0000", 2024-09-16T20:48:49.883 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6810000000000004e-06 2024-09-16T20:48:49.883 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:49.883 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:49.883 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:49.883 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711293+0000", 2024-09-16T20:48:49.884 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035848189999999999 2024-09-16T20:48:49.884 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:49.884 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:49.884 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:48:49.884 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711301+0000", 2024-09-16T20:48:49.884 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4730000000000003e-06 2024-09-16T20:48:49.884 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:49.884 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:49.884 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:49.884 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:49.884 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:49.884 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:8040 3.1e 3.ac44ff9e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:48:49.884 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:48:49.885 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.17201296499999999, 2024-09-16T20:48:49.885 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.17207130500000001, 2024-09-16T20:48:49.885 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:49.885 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:49.885 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "waiting for peered", 2024-09-16T20:48:49.885 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:49.885 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:48:49.885 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:48:49.885 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8040 2024-09-16T20:48:49.885 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:49.885 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:49.885 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:49.886 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:49.886 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:48:49.886 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:49.886 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:49.886 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:49.886 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:49.886 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:48:49.886 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:49.886 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:49.886 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:49.886 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:49.886 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:48:49.886 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8100000000000003e-07 2024-09-16T20:48:49.886 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:49.887 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:49.887 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:49.887 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707737+0000", 2024-09-16T20:48:49.887 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5020000000000001e-06 2024-09-16T20:48:49.887 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:49.887 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:49.887 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:49.887 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707745+0000", 2024-09-16T20:48:49.887 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2169999999999994e-06 2024-09-16T20:48:49.887 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:49.887 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:49.887 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:49.887 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707751+0000", 2024-09-16T20:48:49.887 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.263e-06 2024-09-16T20:48:49.888 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:49.888 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:49.888 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:49.888 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711336+0000", 2024-09-16T20:48:49.888 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035848360000000001 2024-09-16T20:48:49.888 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:49.888 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:49.888 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:48:49.888 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711340+0000", 2024-09-16T20:48:49.888 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1309999999999999e-06 2024-09-16T20:48:49.888 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:49.888 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:49.888 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:49.889 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:49.889 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:49.889 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4927.0:4487 3.1e 3.ac44ff9e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:48:49.889 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:48:49.889 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.17149276299999999, 2024-09-16T20:48:49.889 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.171581384, 2024-09-16T20:48:49.889 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:49.889 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:49.889 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "waiting for peered", 2024-09-16T20:48:49.889 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:49.889 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4927", 2024-09-16T20:48:49.889 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:48:49.889 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4487 2024-09-16T20:48:49.889 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:49.890 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:49.890 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:49.890 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:49.890 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:48:49.890 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:49.890 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:49.890 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:49.890 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:49.890 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:48:49.890 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:49.890 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:49.890 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:49.890 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:49.890 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708254+0000", 2024-09-16T20:48:49.891 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1800000000000005e-07 2024-09-16T20:48:49.891 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:49.891 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:49.891 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:49.891 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708258+0000", 2024-09-16T20:48:49.891 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8689999999999997e-06 2024-09-16T20:48:49.891 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:49.891 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:49.891 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:49.891 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708267+0000", 2024-09-16T20:48:49.891 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9590000000000007e-06 2024-09-16T20:48:49.891 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:49.891 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:49.891 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:49.892 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708274+0000", 2024-09-16T20:48:49.892 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9970000000000001e-06 2024-09-16T20:48:49.892 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:49.892 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:49.892 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:49.892 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711366+0000", 2024-09-16T20:48:49.892 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030919530000000001 2024-09-16T20:48:49.892 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:49.892 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:49.892 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:48:49.892 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711372+0000", 2024-09-16T20:48:49.892 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9410000000000004e-06 2024-09-16T20:48:49.892 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:49.893 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:49.893 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:49.893 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:49.893 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-16T20:48:49.893 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 3 2024-09-16T20:48:49.893 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:49.894 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:48:49.979 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:49.978+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:50.043 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:50.043 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:48:50.043 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:48:50.043 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:48:50.043 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:50.052 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:48:50.073 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:50.072+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:50.174 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:50.173+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:50.200 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:50.200 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:48:50.200 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:48:50.200 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:48:50.200 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.201 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6412 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3561472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:50.201 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871685+0000", 2024-09-16T20:48:50.201 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 30.326762136999999, 2024-09-16T20:48:50.201 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048146568000000001, 2024-09-16T20:48:50.201 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:50.201 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:50.201 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:50.201 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:50.201 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:50.201 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:50.201 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6412 2024-09-16T20:48:50.201 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.202 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:50.202 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.202 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:50.202 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871685+0000", 2024-09-16T20:48:50.202 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.202 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.202 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.202 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:50.202 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871685+0000", 2024-09-16T20:48:50.202 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.202 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.202 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.202 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:50.203 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871685+0000", 2024-09-16T20:48:50.203 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.58e-07 2024-09-16T20:48:50.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.203 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.203 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:50.203 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871687+0000", 2024-09-16T20:48:50.203 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1830000000000001e-06 2024-09-16T20:48:50.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.203 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.203 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:50.203 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871694+0000", 2024-09-16T20:48:50.203 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7689999999999999e-06 2024-09-16T20:48:50.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.203 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.204 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:50.204 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871699+0000", 2024-09-16T20:48:50.204 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1769999999999997e-06 2024-09-16T20:48:50.204 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.204 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.204 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:50.204 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.919662+0000", 2024-09-16T20:48:50.204 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047963121999999997 2024-09-16T20:48:50.204 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.204 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.204 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:50.204 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.919744+0000", 2024-09-16T20:48:50.204 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1997999999999997e-05 2024-09-16T20:48:50.205 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.205 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.205 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:50.205 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.919831+0000", 2024-09-16T20:48:50.205 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6860999999999994e-05 2024-09-16T20:48:50.205 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.205 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:50.205 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.205 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.205 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.205 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6414 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3569664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:50.205 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871708+0000", 2024-09-16T20:48:50.205 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 30.326739190000001, 2024-09-16T20:48:50.206 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048373317999999998, 2024-09-16T20:48:50.206 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:50.206 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:50.206 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:50.206 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:50.206 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:50.206 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:50.206 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6414 2024-09-16T20:48:50.206 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.206 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:50.206 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.206 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:50.206 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871708+0000", 2024-09-16T20:48:50.207 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.207 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.207 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.207 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:50.207 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871708+0000", 2024-09-16T20:48:50.207 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.207 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.207 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.207 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:50.207 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871708+0000", 2024-09-16T20:48:50.207 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9799999999999999e-07 2024-09-16T20:48:50.207 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.207 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.207 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:50.208 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871710+0000", 2024-09-16T20:48:50.208 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2699999999999999e-06 2024-09-16T20:48:50.208 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.208 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.208 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:50.208 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871718+0000", 2024-09-16T20:48:50.208 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8499999999999994e-06 2024-09-16T20:48:50.208 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.208 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.208 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:50.208 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871723+0000", 2024-09-16T20:48:50.208 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1819999999999997e-06 2024-09-16T20:48:50.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.209 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.209 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:50.209 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.919883+0000", 2024-09-16T20:48:50.209 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048159955999999997 2024-09-16T20:48:50.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.209 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.209 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:50.209 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.919969+0000", 2024-09-16T20:48:50.209 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5808000000000002e-05 2024-09-16T20:48:50.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.209 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.209 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:50.210 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920081+0000", 2024-09-16T20:48:50.210 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000111854 2024-09-16T20:48:50.210 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.210 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:50.210 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.210 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.210 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6415 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3577856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:50.210 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871730+0000", 2024-09-16T20:48:50.210 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 30.326716675, 2024-09-16T20:48:50.210 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.04851482, 2024-09-16T20:48:50.210 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:50.210 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:50.211 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:50.211 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:50.211 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:50.211 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:50.211 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6415 2024-09-16T20:48:50.211 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.211 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:50.211 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.211 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:50.211 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871730+0000", 2024-09-16T20:48:50.211 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.211 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.211 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.211 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:50.212 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871730+0000", 2024-09-16T20:48:50.212 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.212 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.212 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.212 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:50.212 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871731+0000", 2024-09-16T20:48:50.212 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6899999999999998e-07 2024-09-16T20:48:50.212 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.212 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.212 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:50.212 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871733+0000", 2024-09-16T20:48:50.212 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0600000000000002e-06 2024-09-16T20:48:50.212 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.213 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.213 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:50.213 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871740+0000", 2024-09-16T20:48:50.213 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3749999999999997e-06 2024-09-16T20:48:50.213 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.213 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.213 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:50.213 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871749+0000", 2024-09-16T20:48:50.213 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9109999999999999e-06 2024-09-16T20:48:50.213 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.213 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.213 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:50.213 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920129+0000", 2024-09-16T20:48:50.214 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048380175999999997 2024-09-16T20:48:50.214 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.214 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.214 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:50.214 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920181+0000", 2024-09-16T20:48:50.214 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2077000000000003e-05 2024-09-16T20:48:50.214 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.214 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.214 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:50.214 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920245+0000", 2024-09-16T20:48:50.214 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3851999999999998e-05 2024-09-16T20:48:50.214 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.214 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:50.214 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.215 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.215 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.215 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6417 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3586048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:50.215 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871756+0000", 2024-09-16T20:48:50.215 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 30.32669104, 2024-09-16T20:48:50.215 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048628301999999998, 2024-09-16T20:48:50.215 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:50.215 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:50.215 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:50.215 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:50.215 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:50.215 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:50.216 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6417 2024-09-16T20:48:50.216 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.216 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:50.216 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.216 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:50.216 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871756+0000", 2024-09-16T20:48:50.216 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.216 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.216 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.216 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:50.216 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871756+0000", 2024-09-16T20:48:50.216 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.216 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.217 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.217 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:50.217 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871756+0000", 2024-09-16T20:48:50.217 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0100000000000005e-07 2024-09-16T20:48:50.217 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.217 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.217 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:50.217 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871758+0000", 2024-09-16T20:48:50.217 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.902e-06 2024-09-16T20:48:50.217 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.217 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.217 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:50.217 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871764+0000", 2024-09-16T20:48:50.217 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8170000000000004e-06 2024-09-16T20:48:50.218 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.218 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.218 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:50.218 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871769+0000", 2024-09-16T20:48:50.218 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6500000000000004e-06 2024-09-16T20:48:50.218 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.218 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.218 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:50.218 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920276+0000", 2024-09-16T20:48:50.218 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048507794999999999 2024-09-16T20:48:50.218 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.218 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.218 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:50.219 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920329+0000", 2024-09-16T20:48:50.219 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2534999999999998e-05 2024-09-16T20:48:50.219 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.219 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.219 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:50.219 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920384+0000", 2024-09-16T20:48:50.219 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5102000000000002e-05 2024-09-16T20:48:50.219 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.219 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:50.219 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.219 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.219 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.219 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6419 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3594240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:50.219 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871775+0000", 2024-09-16T20:48:50.220 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 30.326671863000001, 2024-09-16T20:48:50.220 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048754724999999999, 2024-09-16T20:48:50.220 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:50.220 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:50.220 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:50.220 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:50.220 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:50.220 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:50.220 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6419 2024-09-16T20:48:50.220 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.220 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:50.220 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.220 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:50.221 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871775+0000", 2024-09-16T20:48:50.221 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.221 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.221 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.221 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:50.221 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871775+0000", 2024-09-16T20:48:50.221 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.221 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.221 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.221 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:50.221 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871775+0000", 2024-09-16T20:48:50.221 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2500000000000001e-07 2024-09-16T20:48:50.221 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.222 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.222 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:50.222 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871777+0000", 2024-09-16T20:48:50.222 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9649999999999998e-06 2024-09-16T20:48:50.222 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.222 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.222 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:50.222 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871784+0000", 2024-09-16T20:48:50.222 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3419999999999996e-06 2024-09-16T20:48:50.222 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.222 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.222 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:50.222 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871789+0000", 2024-09-16T20:48:50.223 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1889999999999998e-06 2024-09-16T20:48:50.223 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.223 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.223 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:50.223 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920416+0000", 2024-09-16T20:48:50.223 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048627363999999999 2024-09-16T20:48:50.223 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.223 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.223 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:50.223 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920472+0000", 2024-09-16T20:48:50.223 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5429000000000001e-05 2024-09-16T20:48:50.223 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.223 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.223 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:50.224 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920530+0000", 2024-09-16T20:48:50.224 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8111e-05 2024-09-16T20:48:50.224 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.224 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:50.224 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.224 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.224 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.224 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6420 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3602432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:50.224 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871796+0000", 2024-09-16T20:48:50.224 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 30.326650950000001, 2024-09-16T20:48:50.224 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048857298, 2024-09-16T20:48:50.224 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:50.224 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:50.225 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:50.225 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:50.225 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:50.225 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:50.225 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6420 2024-09-16T20:48:50.225 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.225 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:50.225 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.225 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:50.225 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871796+0000", 2024-09-16T20:48:50.225 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.225 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.225 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.226 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:50.226 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871796+0000", 2024-09-16T20:48:50.226 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.226 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.226 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.226 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:50.226 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871854+0000", 2024-09-16T20:48:50.226 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8187999999999999e-05 2024-09-16T20:48:50.226 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.226 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.226 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:50.226 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871856+0000", 2024-09-16T20:48:50.226 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3549999999999999e-06 2024-09-16T20:48:50.226 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.227 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.227 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:50.227 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871866+0000", 2024-09-16T20:48:50.227 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6500000000000008e-06 2024-09-16T20:48:50.227 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.227 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.227 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:50.227 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871872+0000", 2024-09-16T20:48:50.227 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3799999999999999e-06 2024-09-16T20:48:50.227 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.227 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.227 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:50.227 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920558+0000", 2024-09-16T20:48:50.228 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048685797000000003 2024-09-16T20:48:50.228 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.228 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.228 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:50.228 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920603+0000", 2024-09-16T20:48:50.228 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4629999999999998e-05 2024-09-16T20:48:50.228 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.228 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.228 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:50.228 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920653+0000", 2024-09-16T20:48:50.228 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0297999999999999e-05 2024-09-16T20:48:50.228 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.228 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:50.229 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.229 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.229 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.229 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6421 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3610624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:50.229 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871882+0000", 2024-09-16T20:48:50.229 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 30.326564882, 2024-09-16T20:48:50.229 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048913381999999998, 2024-09-16T20:48:50.229 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:50.229 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:50.229 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:50.229 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:50.229 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:50.229 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:50.230 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6421 2024-09-16T20:48:50.230 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.230 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:50.230 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.230 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:50.230 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871882+0000", 2024-09-16T20:48:50.230 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.230 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.230 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.230 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:50.230 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871882+0000", 2024-09-16T20:48:50.230 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.230 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.230 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.231 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:50.231 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.873257+0000", 2024-09-16T20:48:50.231 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001374789 2024-09-16T20:48:50.231 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.231 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.231 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:50.231 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.873261+0000", 2024-09-16T20:48:50.231 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5589999999999999e-06 2024-09-16T20:48:50.231 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.231 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.231 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:50.231 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.873275+0000", 2024-09-16T20:48:50.231 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3675999999999999e-05 2024-09-16T20:48:50.232 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.232 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.232 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:50.232 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.873281+0000", 2024-09-16T20:48:50.232 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6930000000000004e-06 2024-09-16T20:48:50.232 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.232 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.232 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:50.232 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920684+0000", 2024-09-16T20:48:50.232 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047402913999999997 2024-09-16T20:48:50.232 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.232 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.232 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:50.233 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920737+0000", 2024-09-16T20:48:50.233 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3600000000000002e-05 2024-09-16T20:48:50.233 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.233 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.233 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:50.233 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920795+0000", 2024-09-16T20:48:50.233 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8150999999999999e-05 2024-09-16T20:48:50.233 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.233 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:50.233 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.233 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.233 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.233 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:7491 3.1e 3:7c48f6ae:::rbd_object_map.110e891a9b2b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:50.233 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.925865+0000", 2024-09-16T20:48:50.234 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 30.272581738, 2024-09-16T20:48:50.234 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048897293000000001, 2024-09-16T20:48:50.234 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:50.234 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:50.234 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:50.234 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:50.234 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:50.234 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:50.234 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 7491 2024-09-16T20:48:50.234 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.234 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:50.234 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.234 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:50.235 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925865+0000", 2024-09-16T20:48:50.235 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.235 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.235 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.235 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:50.235 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925865+0000", 2024-09-16T20:48:50.235 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.235 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.235 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.235 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:50.235 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925866+0000", 2024-09-16T20:48:50.235 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.39e-07 2024-09-16T20:48:50.235 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.236 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.236 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:50.236 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925868+0000", 2024-09-16T20:48:50.236 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.508e-06 2024-09-16T20:48:50.236 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.236 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.236 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:50.236 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925874+0000", 2024-09-16T20:48:50.236 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7919999999999998e-06 2024-09-16T20:48:50.236 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.236 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.236 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:50.236 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925879+0000", 2024-09-16T20:48:50.236 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8579999999999999e-06 2024-09-16T20:48:50.237 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.237 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.237 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:50.237 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.970875+0000", 2024-09-16T20:48:50.237 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044996412 2024-09-16T20:48:50.237 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.237 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.237 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:50.237 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.970970+0000", 2024-09-16T20:48:50.237 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4732e-05 2024-09-16T20:48:50.237 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.237 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.237 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-16T20:48:50.238 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.971275+0000", 2024-09-16T20:48:50.238 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00030464199999999999 2024-09-16T20:48:50.238 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.238 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.238 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:50.238 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.974598+0000", 2024-09-16T20:48:50.238 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0033238159999999998 2024-09-16T20:48:50.238 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.238 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.238 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:48:50.238 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.974666+0000", 2024-09-16T20:48:50.238 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7617e-05 2024-09-16T20:48:50.238 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.239 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.239 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:48:50.239 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.974718+0000", 2024-09-16T20:48:50.239 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2259000000000002e-05 2024-09-16T20:48:50.239 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.239 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.239 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:50.239 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.974762+0000", 2024-09-16T20:48:50.239 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4218000000000003e-05 2024-09-16T20:48:50.239 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.239 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:50.239 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.239 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.239 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.240 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4750 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1488896~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:50.240 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689052+0000", 2024-09-16T20:48:50.240 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.509395199, 2024-09-16T20:48:50.240 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048208359999999999, 2024-09-16T20:48:50.240 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:50.240 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:50.240 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:50.240 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:50.240 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:50.240 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:50.240 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4750 2024-09-16T20:48:50.240 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.240 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:50.241 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.241 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:50.241 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689052+0000", 2024-09-16T20:48:50.241 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.241 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.241 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.241 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:50.241 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689052+0000", 2024-09-16T20:48:50.241 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.241 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.241 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.241 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:50.242 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689052+0000", 2024-09-16T20:48:50.242 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.65e-07 2024-09-16T20:48:50.242 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.242 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.242 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:50.242 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689058+0000", 2024-09-16T20:48:50.242 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.835e-06 2024-09-16T20:48:50.242 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.242 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.242 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:50.242 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689064+0000", 2024-09-16T20:48:50.242 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2729999999999999e-06 2024-09-16T20:48:50.242 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.242 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.243 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:50.243 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689069+0000", 2024-09-16T20:48:50.243 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1220000000000003e-06 2024-09-16T20:48:50.243 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.243 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.243 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:50.243 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737161+0000", 2024-09-16T20:48:50.243 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048091776000000003 2024-09-16T20:48:50.243 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.243 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.243 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:50.243 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737210+0000", 2024-09-16T20:48:50.243 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8575000000000002e-05 2024-09-16T20:48:50.244 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.244 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.244 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:50.244 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737260+0000", 2024-09-16T20:48:50.244 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0414000000000002e-05 2024-09-16T20:48:50.244 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.244 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:50.244 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.244 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.244 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.244 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4752 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1497088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:50.244 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689076+0000", 2024-09-16T20:48:50.245 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.509371056999999, 2024-09-16T20:48:50.245 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048322324999999999, 2024-09-16T20:48:50.245 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:50.245 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:50.245 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:50.245 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:50.245 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:50.245 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:50.245 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4752 2024-09-16T20:48:50.245 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.245 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:50.245 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.245 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:50.245 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689076+0000", 2024-09-16T20:48:50.246 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.246 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.246 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.246 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:50.246 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689076+0000", 2024-09-16T20:48:50.246 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.246 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.246 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.246 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:50.246 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689076+0000", 2024-09-16T20:48:50.246 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8000000000000001e-07 2024-09-16T20:48:50.246 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.246 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.247 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:50.247 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689078+0000", 2024-09-16T20:48:50.247 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8369999999999999e-06 2024-09-16T20:48:50.247 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.247 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.247 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:50.247 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689084+0000", 2024-09-16T20:48:50.247 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1319999999999999e-06 2024-09-16T20:48:50.247 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.247 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.247 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:50.247 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689090+0000", 2024-09-16T20:48:50.247 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0430000000000004e-06 2024-09-16T20:48:50.248 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.248 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.248 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:50.248 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737286+0000", 2024-09-16T20:48:50.248 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048195670000000003 2024-09-16T20:48:50.248 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.248 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.248 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:50.248 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737337+0000", 2024-09-16T20:48:50.248 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1511000000000002e-05 2024-09-16T20:48:50.248 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.248 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.248 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:50.249 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737398+0000", 2024-09-16T20:48:50.249 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0751999999999997e-05 2024-09-16T20:48:50.249 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.249 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:50.249 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.249 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.249 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.249 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4754 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1505280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:50.249 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689097+0000", 2024-09-16T20:48:50.249 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.509349539999999, 2024-09-16T20:48:50.249 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048429947000000001, 2024-09-16T20:48:50.249 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:50.249 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:50.250 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:50.250 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:50.250 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:50.250 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:50.250 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4754 2024-09-16T20:48:50.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.250 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:50.250 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.250 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:50.250 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689097+0000", 2024-09-16T20:48:50.250 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.250 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.251 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:50.251 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689097+0000", 2024-09-16T20:48:50.251 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.251 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.251 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.251 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:50.251 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689098+0000", 2024-09-16T20:48:50.251 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.15e-07 2024-09-16T20:48:50.251 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.251 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.251 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:50.251 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689100+0000", 2024-09-16T20:48:50.251 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8190000000000001e-06 2024-09-16T20:48:50.251 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.252 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.252 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:50.252 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689106+0000", 2024-09-16T20:48:50.252 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2489999999999998e-06 2024-09-16T20:48:50.252 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.252 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.252 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:50.252 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689111+0000", 2024-09-16T20:48:50.252 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3179999999999997e-06 2024-09-16T20:48:50.252 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.252 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.252 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:50.252 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737428+0000", 2024-09-16T20:48:50.253 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048316572000000002 2024-09-16T20:48:50.253 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.253 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.253 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:50.253 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737475+0000", 2024-09-16T20:48:50.253 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7392000000000003e-05 2024-09-16T20:48:50.253 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.253 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.253 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:50.253 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737527+0000", 2024-09-16T20:48:50.253 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2182000000000003e-05 2024-09-16T20:48:50.253 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.253 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:50.254 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.254 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.254 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.254 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4756 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1513472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:50.254 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689118+0000", 2024-09-16T20:48:50.254 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.509329318999999, 2024-09-16T20:48:50.254 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048535146000000001, 2024-09-16T20:48:50.254 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:50.254 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:50.254 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:50.254 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:50.254 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:50.254 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:50.254 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4756 2024-09-16T20:48:50.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.255 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:50.255 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.255 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:50.255 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689118+0000", 2024-09-16T20:48:50.255 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.255 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.255 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:50.255 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689118+0000", 2024-09-16T20:48:50.255 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.255 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.256 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:50.256 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689118+0000", 2024-09-16T20:48:50.256 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7099999999999997e-07 2024-09-16T20:48:50.256 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.256 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.256 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:50.256 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689120+0000", 2024-09-16T20:48:50.256 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.787e-06 2024-09-16T20:48:50.256 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.256 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.256 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:50.256 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689126+0000", 2024-09-16T20:48:50.256 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2720000000000003e-06 2024-09-16T20:48:50.256 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.257 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.257 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:50.257 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689134+0000", 2024-09-16T20:48:50.257 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2190000000000004e-06 2024-09-16T20:48:50.257 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.257 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.257 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:50.257 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737553+0000", 2024-09-16T20:48:50.257 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048418993 2024-09-16T20:48:50.257 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.257 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.257 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:50.257 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737598+0000", 2024-09-16T20:48:50.258 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4743999999999997e-05 2024-09-16T20:48:50.258 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.258 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.258 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:50.258 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737653+0000", 2024-09-16T20:48:50.258 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4759999999999997e-05 2024-09-16T20:48:50.258 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.258 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:50.258 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.258 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.258 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.258 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4758 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1521664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:50.258 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689141+0000", 2024-09-16T20:48:50.259 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.509306338999998, 2024-09-16T20:48:50.259 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048640741000000001, 2024-09-16T20:48:50.259 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:50.259 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:50.259 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:50.259 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:50.259 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:50.259 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:50.259 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4758 2024-09-16T20:48:50.259 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.259 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:50.259 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.259 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:50.260 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689141+0000", 2024-09-16T20:48:50.260 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.260 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.260 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.260 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:50.260 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689141+0000", 2024-09-16T20:48:50.260 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.260 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.260 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.260 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:50.260 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689141+0000", 2024-09-16T20:48:50.260 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8799999999999998e-07 2024-09-16T20:48:50.260 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.261 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.261 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:50.261 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689143+0000", 2024-09-16T20:48:50.261 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.945e-06 2024-09-16T20:48:50.261 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.261 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.261 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:50.261 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689149+0000", 2024-09-16T20:48:50.261 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9800000000000003e-06 2024-09-16T20:48:50.261 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.261 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.261 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:50.261 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689155+0000", 2024-09-16T20:48:50.261 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7509999999999997e-06 2024-09-16T20:48:50.262 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.262 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.262 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:50.262 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737681+0000", 2024-09-16T20:48:50.262 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048526825000000003 2024-09-16T20:48:50.262 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.262 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.262 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:50.262 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737729+0000", 2024-09-16T20:48:50.262 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7219999999999999e-05 2024-09-16T20:48:50.262 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.262 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.262 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:50.263 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737781+0000", 2024-09-16T20:48:50.263 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2632e-05 2024-09-16T20:48:50.263 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.263 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:50.263 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.263 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.263 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4760 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1529856~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:50.263 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689161+0000", 2024-09-16T20:48:50.263 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.509285991999999, 2024-09-16T20:48:50.263 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048743981999999998, 2024-09-16T20:48:50.263 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:50.263 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:50.263 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:50.264 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:50.264 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:50.264 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:50.264 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4760 2024-09-16T20:48:50.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.264 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:50.264 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.264 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:50.264 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689161+0000", 2024-09-16T20:48:50.264 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.264 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.264 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:50.265 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689161+0000", 2024-09-16T20:48:50.265 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.265 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.265 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.265 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:50.265 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689161+0000", 2024-09-16T20:48:50.265 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5599999999999995e-07 2024-09-16T20:48:50.265 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.265 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.265 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:50.265 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689163+0000", 2024-09-16T20:48:50.265 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.818e-06 2024-09-16T20:48:50.265 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.266 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.266 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:50.266 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689169+0000", 2024-09-16T20:48:50.266 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0809999999999999e-06 2024-09-16T20:48:50.266 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.266 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.266 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:50.266 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689174+0000", 2024-09-16T20:48:50.266 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.938e-06 2024-09-16T20:48:50.266 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.266 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.266 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:50.267 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737807+0000", 2024-09-16T20:48:50.267 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048633124 2024-09-16T20:48:50.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.267 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.267 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:50.267 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737851+0000", 2024-09-16T20:48:50.267 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3764000000000003e-05 2024-09-16T20:48:50.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.267 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.267 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:50.267 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737905+0000", 2024-09-16T20:48:50.267 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3701e-05 2024-09-16T20:48:50.267 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.268 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:50.268 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.268 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.268 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.268 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4762 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1538048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:50.268 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689182+0000", 2024-09-16T20:48:50.268 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.509264770000001, 2024-09-16T20:48:50.268 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048850275999999998, 2024-09-16T20:48:50.268 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:50.268 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:50.268 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:50.268 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:50.268 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:50.269 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:50.269 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4762 2024-09-16T20:48:50.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.269 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:50.269 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.269 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:50.269 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689182+0000", 2024-09-16T20:48:50.269 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.269 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.269 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:50.269 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689182+0000", 2024-09-16T20:48:50.269 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.270 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.270 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.270 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:50.270 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689182+0000", 2024-09-16T20:48:50.270 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-16T20:48:50.270 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.270 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.270 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:50.270 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689184+0000", 2024-09-16T20:48:50.270 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8190000000000001e-06 2024-09-16T20:48:50.270 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.270 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.270 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:50.270 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689191+0000", 2024-09-16T20:48:50.271 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2530000000000001e-06 2024-09-16T20:48:50.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.271 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.271 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:50.271 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689195+0000", 2024-09-16T20:48:50.271 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9559999999999996e-06 2024-09-16T20:48:50.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.271 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.271 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:50.271 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737933+0000", 2024-09-16T20:48:50.271 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048737919999999997 2024-09-16T20:48:50.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.271 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.272 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:50.272 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737982+0000", 2024-09-16T20:48:50.272 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8089e-05 2024-09-16T20:48:50.272 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.272 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.272 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:50.272 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738032+0000", 2024-09-16T20:48:50.272 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0868999999999997e-05 2024-09-16T20:48:50.272 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.272 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:50.272 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.272 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.272 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.273 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4764 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1546240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:50.273 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689201+0000", 2024-09-16T20:48:50.273 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.509245412999999, 2024-09-16T20:48:50.273 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048957550000000002, 2024-09-16T20:48:50.273 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:50.273 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:50.273 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:50.273 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:50.273 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:50.273 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:50.273 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4764 2024-09-16T20:48:50.273 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.273 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:50.273 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.274 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:50.274 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689201+0000", 2024-09-16T20:48:50.274 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.274 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.274 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.274 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:50.274 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689201+0000", 2024-09-16T20:48:50.274 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.274 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.274 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.274 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:50.274 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689202+0000", 2024-09-16T20:48:50.274 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4200000000000002e-07 2024-09-16T20:48:50.275 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.275 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.275 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:50.275 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689204+0000", 2024-09-16T20:48:50.275 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9750000000000001e-06 2024-09-16T20:48:50.275 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.275 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.275 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:50.275 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689210+0000", 2024-09-16T20:48:50.275 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1e-06 2024-09-16T20:48:50.275 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.275 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.275 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:50.275 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689215+0000", 2024-09-16T20:48:50.276 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0180000000000001e-06 2024-09-16T20:48:50.276 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.276 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.276 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:50.276 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738058+0000", 2024-09-16T20:48:50.276 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048843573000000001 2024-09-16T20:48:50.276 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.276 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.276 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:50.276 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738102+0000", 2024-09-16T20:48:50.276 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3256000000000001e-05 2024-09-16T20:48:50.276 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.276 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.277 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:50.277 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738159+0000", 2024-09-16T20:48:50.277 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7286e-05 2024-09-16T20:48:50.277 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.277 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:50.277 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.277 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.277 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.277 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4766 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1554432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:50.277 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689221+0000", 2024-09-16T20:48:50.277 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.509225446999999, 2024-09-16T20:48:50.277 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049064996, 2024-09-16T20:48:50.278 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:50.278 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:50.278 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:50.278 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:50.278 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:50.278 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:50.278 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4766 2024-09-16T20:48:50.278 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.278 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:50.278 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.278 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:50.278 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689221+0000", 2024-09-16T20:48:50.278 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.279 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.279 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.279 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:50.279 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689221+0000", 2024-09-16T20:48:50.279 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.279 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.279 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.279 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:50.279 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689222+0000", 2024-09-16T20:48:50.279 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.34e-07 2024-09-16T20:48:50.279 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.279 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.279 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:50.279 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689224+0000", 2024-09-16T20:48:50.280 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.846e-06 2024-09-16T20:48:50.280 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.280 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.280 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:50.280 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689247+0000", 2024-09-16T20:48:50.280 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3448e-05 2024-09-16T20:48:50.280 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.280 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.280 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:50.280 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689254+0000", 2024-09-16T20:48:50.280 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6969999999999996e-06 2024-09-16T20:48:50.280 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.280 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.281 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:50.281 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738188+0000", 2024-09-16T20:48:50.281 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048934520000000002 2024-09-16T20:48:50.281 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.281 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.281 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:50.281 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738234+0000", 2024-09-16T20:48:50.281 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5695999999999998e-05 2024-09-16T20:48:50.281 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.281 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.281 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:50.281 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738286+0000", 2024-09-16T20:48:50.281 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2355000000000003e-05 2024-09-16T20:48:50.282 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.282 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:50.282 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.282 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.282 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.282 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4768 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1562624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:50.282 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689262+0000", 2024-09-16T20:48:50.282 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.509184492999999, 2024-09-16T20:48:50.282 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049157157999999999, 2024-09-16T20:48:50.282 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:50.282 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:50.282 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:50.282 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:50.283 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:50.283 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:50.283 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4768 2024-09-16T20:48:50.283 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.283 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:50.283 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.283 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:50.283 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689262+0000", 2024-09-16T20:48:50.283 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.283 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.283 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.283 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:50.283 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689262+0000", 2024-09-16T20:48:50.283 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.284 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.284 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.284 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:50.284 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689263+0000", 2024-09-16T20:48:50.284 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6199999999999998e-07 2024-09-16T20:48:50.284 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.284 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.284 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:50.284 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689270+0000", 2024-09-16T20:48:50.284 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.798e-06 2024-09-16T20:48:50.284 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.284 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.284 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:50.285 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689277+0000", 2024-09-16T20:48:50.285 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0890000000000002e-06 2024-09-16T20:48:50.285 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.285 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.285 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:50.285 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689282+0000", 2024-09-16T20:48:50.285 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0470000000000001e-06 2024-09-16T20:48:50.285 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.285 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.285 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:50.285 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738319+0000", 2024-09-16T20:48:50.285 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049037139 2024-09-16T20:48:50.285 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.286 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.286 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:50.286 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738365+0000", 2024-09-16T20:48:50.286 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6356999999999997e-05 2024-09-16T20:48:50.286 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.286 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.286 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:50.286 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738420+0000", 2024-09-16T20:48:50.286 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4265999999999998e-05 2024-09-16T20:48:50.286 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.286 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:50.286 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.286 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.287 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.287 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4770 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1570816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:50.287 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689289+0000", 2024-09-16T20:48:50.287 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.509157983000001, 2024-09-16T20:48:50.287 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049253705000000002, 2024-09-16T20:48:50.287 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:50.287 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:50.287 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:50.287 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:50.287 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:50.287 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:50.287 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4770 2024-09-16T20:48:50.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.287 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:50.288 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.288 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:50.288 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689289+0000", 2024-09-16T20:48:50.288 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.288 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.288 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.288 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:50.288 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689289+0000", 2024-09-16T20:48:50.288 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.288 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.288 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.288 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:50.288 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689289+0000", 2024-09-16T20:48:50.289 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0999999999999999e-07 2024-09-16T20:48:50.289 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.289 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.289 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:50.289 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689291+0000", 2024-09-16T20:48:50.289 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0030000000000001e-06 2024-09-16T20:48:50.289 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.289 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.289 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:50.289 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689297+0000", 2024-09-16T20:48:50.289 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9479999999999996e-06 2024-09-16T20:48:50.289 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.289 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.289 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:50.290 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689302+0000", 2024-09-16T20:48:50.290 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7389999999999999e-06 2024-09-16T20:48:50.290 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.290 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.290 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:50.290 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738442+0000", 2024-09-16T20:48:50.290 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049140001000000003 2024-09-16T20:48:50.290 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.290 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.290 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:50.290 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738490+0000", 2024-09-16T20:48:50.290 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7787000000000002e-05 2024-09-16T20:48:50.290 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.291 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.291 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:50.291 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738543+0000", 2024-09-16T20:48:50.291 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2816999999999999e-05 2024-09-16T20:48:50.291 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.291 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:50.291 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.291 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.291 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.291 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:3088 3.1e 3:79e5ec01:::rbd_header.110e891a9b2b:head [watch watch cookie 140213437171488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:48:50.291 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:48:50.291 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 26.034601966, 2024-09-16T20:48:50.291 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.11895186200000001, 2024-09-16T20:48:50.291 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:50.292 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:50.292 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:50.292 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:50.292 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:48:50.292 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:48:50.292 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3088 2024-09-16T20:48:50.292 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.292 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:50.292 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.292 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:50.292 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:48:50.292 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.292 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.293 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.293 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:50.293 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:48:50.293 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.293 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.293 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.293 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:50.293 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163846+0000", 2024-09-16T20:48:50.293 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6799999999999999e-07 2024-09-16T20:48:50.293 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.293 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.293 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:50.293 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163850+0000", 2024-09-16T20:48:50.294 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.929e-06 2024-09-16T20:48:50.294 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.294 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.294 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:50.294 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163862+0000", 2024-09-16T20:48:50.294 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1972e-05 2024-09-16T20:48:50.294 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.294 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.294 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:50.294 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163867+0000", 2024-09-16T20:48:50.294 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.451e-06 2024-09-16T20:48:50.294 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.294 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.294 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:50.295 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163912+0000", 2024-09-16T20:48:50.295 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4456000000000003e-05 2024-09-16T20:48:50.295 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.295 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.295 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:50.295 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163962+0000", 2024-09-16T20:48:50.295 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0578000000000003e-05 2024-09-16T20:48:50.295 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.295 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.295 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-16T20:48:50.295 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164058+0000", 2024-09-16T20:48:50.295 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5934999999999995e-05 2024-09-16T20:48:50.295 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.296 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.296 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:48:50.296 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164570+0000", 2024-09-16T20:48:50.296 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00051241100000000005 2024-09-16T20:48:50.296 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.296 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.296 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:50.296 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282629+0000", 2024-09-16T20:48:50.296 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.118058986 2024-09-16T20:48:50.296 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.296 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.296 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:48:50.296 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282725+0000", 2024-09-16T20:48:50.296 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5063999999999996e-05 2024-09-16T20:48:50.297 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.297 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.297 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:50.297 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282797+0000", 2024-09-16T20:48:50.297 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2212000000000006e-05 2024-09-16T20:48:50.297 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.297 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:50.297 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.297 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.297 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:50.297 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:50.299 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:48:50.299 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:50.298+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:50.399 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:50.398+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:50.446 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:50.446 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:48:50.446 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:48:50.446 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:50.454 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:48:50.500 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:50.500+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:50.601 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:50.600+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:50.601 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:50.601 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:48:50.602 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:48:50.602 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:48:50.602 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:50.610 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:48:50.702 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:50.700+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:50.723 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:50.723 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:48:50.723 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:48:50.723 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:48:50.723 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.723 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:4904 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 7309312~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:50.723 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767724+0000", 2024-09-16T20:48:50.723 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 30.953357235999999, 2024-09-16T20:48:50.723 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050270214000000001, 2024-09-16T20:48:50.724 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:50.724 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:50.724 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:50.724 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:50.724 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:50.724 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:50.724 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4904 2024-09-16T20:48:50.724 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.724 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:50.724 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.724 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:50.724 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767724+0000", 2024-09-16T20:48:50.725 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.725 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.725 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.725 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:50.725 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767724+0000", 2024-09-16T20:48:50.725 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.725 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.725 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.725 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:50.725 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767724+0000", 2024-09-16T20:48:50.725 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1399999999999998e-07 2024-09-16T20:48:50.725 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.725 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.726 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:50.726 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767725+0000", 2024-09-16T20:48:50.726 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0380000000000001e-06 2024-09-16T20:48:50.726 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.726 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.726 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:50.726 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767729+0000", 2024-09-16T20:48:50.726 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2490000000000001e-06 2024-09-16T20:48:50.726 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.726 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.726 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:50.726 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767731+0000", 2024-09-16T20:48:50.726 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5940000000000001e-06 2024-09-16T20:48:50.726 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.727 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.727 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:50.727 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.817787+0000", 2024-09-16T20:48:50.727 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050055792000000002 2024-09-16T20:48:50.727 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.727 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.727 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:50.727 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.817883+0000", 2024-09-16T20:48:50.727 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5606999999999995e-05 2024-09-16T20:48:50.727 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.727 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.727 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:50.727 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.817994+0000", 2024-09-16T20:48:50.728 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011162000000000001 2024-09-16T20:48:50.728 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.728 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:50.728 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.728 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.728 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.728 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:4906 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 7317504~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:50.728 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767734+0000", 2024-09-16T20:48:50.728 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 30.953346972999999, 2024-09-16T20:48:50.728 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050406608999999998, 2024-09-16T20:48:50.728 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:50.728 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:50.728 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:50.728 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:50.729 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:50.729 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:50.729 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4906 2024-09-16T20:48:50.729 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.729 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:50.729 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.729 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:50.729 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767734+0000", 2024-09-16T20:48:50.729 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.729 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.729 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.729 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:50.729 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767734+0000", 2024-09-16T20:48:50.730 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.730 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.730 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.730 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:50.730 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767735+0000", 2024-09-16T20:48:50.730 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.05e-07 2024-09-16T20:48:50.730 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.730 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.730 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:50.730 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767736+0000", 2024-09-16T20:48:50.730 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0240000000000001e-06 2024-09-16T20:48:50.730 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.730 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.731 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:50.731 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767739+0000", 2024-09-16T20:48:50.731 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.067e-06 2024-09-16T20:48:50.731 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.731 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.731 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:50.731 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767742+0000", 2024-09-16T20:48:50.731 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.23e-06 2024-09-16T20:48:50.731 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.731 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.731 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:50.731 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818027+0000", 2024-09-16T20:48:50.731 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050285102999999998 2024-09-16T20:48:50.732 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.732 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.732 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:50.732 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818072+0000", 2024-09-16T20:48:50.732 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5129000000000001e-05 2024-09-16T20:48:50.732 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.732 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.732 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:50.732 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818141+0000", 2024-09-16T20:48:50.732 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8850999999999995e-05 2024-09-16T20:48:50.732 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.732 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:50.732 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.732 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.733 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.733 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:4908 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 7325696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:50.733 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767745+0000", 2024-09-16T20:48:50.733 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 30.953336577999998, 2024-09-16T20:48:50.733 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050654230000000001, 2024-09-16T20:48:50.733 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:50.733 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:50.733 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:50.733 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:50.733 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:50.733 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:50.733 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4908 2024-09-16T20:48:50.733 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.734 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:50.734 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.734 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:50.734 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767745+0000", 2024-09-16T20:48:50.734 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.734 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.734 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.734 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:50.734 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767745+0000", 2024-09-16T20:48:50.734 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.734 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.734 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.734 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:50.734 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767745+0000", 2024-09-16T20:48:50.735 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1e-07 2024-09-16T20:48:50.735 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.735 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.735 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:50.735 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767749+0000", 2024-09-16T20:48:50.735 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.532e-06 2024-09-16T20:48:50.735 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.735 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.735 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:50.736 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767753+0000", 2024-09-16T20:48:50.736 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4939999999999999e-06 2024-09-16T20:48:50.736 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.736 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.736 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:50.736 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767756+0000", 2024-09-16T20:48:50.736 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.61e-06 2024-09-16T20:48:50.736 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.736 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.736 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:50.736 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818184+0000", 2024-09-16T20:48:50.736 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050428615000000003 2024-09-16T20:48:50.737 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.737 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.737 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:50.737 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818276+0000", 2024-09-16T20:48:50.737 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2077999999999995e-05 2024-09-16T20:48:50.737 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.737 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.737 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:50.737 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818399+0000", 2024-09-16T20:48:50.737 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012269099999999999 2024-09-16T20:48:50.737 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.737 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:50.737 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.737 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.738 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.738 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:4910 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 7333888~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:50.738 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767759+0000", 2024-09-16T20:48:50.738 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 30.953322688, 2024-09-16T20:48:50.738 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050815076000000001, 2024-09-16T20:48:50.738 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:50.738 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:50.738 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:50.738 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:50.738 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:50.738 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:50.738 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4910 2024-09-16T20:48:50.738 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.739 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:50.739 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.739 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:50.739 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767759+0000", 2024-09-16T20:48:50.739 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.739 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.739 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.739 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:50.739 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767759+0000", 2024-09-16T20:48:50.739 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.739 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.739 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.739 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:50.740 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767759+0000", 2024-09-16T20:48:50.740 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2600000000000001e-07 2024-09-16T20:48:50.740 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.740 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.740 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:50.740 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767760+0000", 2024-09-16T20:48:50.740 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0640000000000001e-06 2024-09-16T20:48:50.740 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.740 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.740 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:50.740 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767763+0000", 2024-09-16T20:48:50.740 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.994e-06 2024-09-16T20:48:50.740 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.740 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.741 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:50.741 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767765+0000", 2024-09-16T20:48:50.741 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5220000000000002e-06 2024-09-16T20:48:50.741 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.741 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.741 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:50.741 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818438+0000", 2024-09-16T20:48:50.741 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050672602999999997 2024-09-16T20:48:50.741 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.741 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.741 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:50.741 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818481+0000", 2024-09-16T20:48:50.741 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2967999999999999e-05 2024-09-16T20:48:50.741 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.742 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.742 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:50.742 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818574+0000", 2024-09-16T20:48:50.742 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2699000000000001e-05 2024-09-16T20:48:50.742 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.742 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:50.742 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.742 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.742 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.742 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5012 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2101248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:50.742 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767768+0000", 2024-09-16T20:48:50.742 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 30.953313039000001, 2024-09-16T20:48:50.742 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051084682999999999, 2024-09-16T20:48:50.742 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:50.743 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:50.743 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:50.743 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:50.743 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:50.743 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:50.743 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5012 2024-09-16T20:48:50.743 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.743 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:50.743 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.743 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:50.743 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767768+0000", 2024-09-16T20:48:50.743 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.743 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.744 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.744 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:50.744 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767768+0000", 2024-09-16T20:48:50.744 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.744 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.744 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.744 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:50.744 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767769+0000", 2024-09-16T20:48:50.744 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1500000000000001e-07 2024-09-16T20:48:50.744 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.744 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.744 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:50.744 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767770+0000", 2024-09-16T20:48:50.744 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.015e-06 2024-09-16T20:48:50.745 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.745 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.745 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:50.745 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767773+0000", 2024-09-16T20:48:50.745 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1070000000000001e-06 2024-09-16T20:48:50.745 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.745 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.745 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:50.745 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767775+0000", 2024-09-16T20:48:50.745 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.508e-06 2024-09-16T20:48:50.745 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.745 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.745 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:50.745 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818630+0000", 2024-09-16T20:48:50.746 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050855282000000002 2024-09-16T20:48:50.746 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.746 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.746 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:50.746 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818732+0000", 2024-09-16T20:48:50.746 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101531 2024-09-16T20:48:50.746 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.746 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.746 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:50.746 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818853+0000", 2024-09-16T20:48:50.746 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000121025 2024-09-16T20:48:50.746 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.746 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:50.747 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.747 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.747 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.747 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5014 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2109440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:50.747 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767778+0000", 2024-09-16T20:48:50.747 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 30.953303103, 2024-09-16T20:48:50.747 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051260776000000001, 2024-09-16T20:48:50.747 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:50.747 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:50.747 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:50.747 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:50.747 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:50.747 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:50.747 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5014 2024-09-16T20:48:50.748 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.748 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:50.748 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.748 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:50.748 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767778+0000", 2024-09-16T20:48:50.748 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.748 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.748 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.748 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:50.748 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767778+0000", 2024-09-16T20:48:50.748 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.748 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.748 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.748 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:50.749 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767824+0000", 2024-09-16T20:48:50.749 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5890999999999998e-05 2024-09-16T20:48:50.749 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.749 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.749 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:50.749 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767827+0000", 2024-09-16T20:48:50.749 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.644e-06 2024-09-16T20:48:50.749 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.749 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.749 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:50.749 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767832+0000", 2024-09-16T20:48:50.749 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2689999999999999e-06 2024-09-16T20:48:50.749 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.750 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.750 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:50.750 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767835+0000", 2024-09-16T20:48:50.750 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2100000000000002e-06 2024-09-16T20:48:50.750 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.750 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.750 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:50.750 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818887+0000", 2024-09-16T20:48:50.750 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051051276999999999 2024-09-16T20:48:50.750 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.750 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.750 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:50.750 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818953+0000", 2024-09-16T20:48:50.750 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6105999999999994e-05 2024-09-16T20:48:50.751 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.751 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.751 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:50.751 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819039+0000", 2024-09-16T20:48:50.751 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6378999999999994e-05 2024-09-16T20:48:50.751 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.751 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:50.751 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.751 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.751 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.751 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5016 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2117632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:50.751 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767841+0000", 2024-09-16T20:48:50.751 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 30.953240424000001, 2024-09-16T20:48:50.751 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051433288000000001, 2024-09-16T20:48:50.752 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:50.752 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:50.752 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:50.752 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:50.752 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:50.752 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:50.752 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5016 2024-09-16T20:48:50.752 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.752 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:50.752 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.752 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:50.752 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767841+0000", 2024-09-16T20:48:50.753 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.753 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.753 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.753 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:50.753 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767841+0000", 2024-09-16T20:48:50.753 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.753 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.753 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.753 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:50.753 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768156+0000", 2024-09-16T20:48:50.753 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00031505699999999999 2024-09-16T20:48:50.753 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.753 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.753 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:50.754 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768159+0000", 2024-09-16T20:48:50.754 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1659999999999998e-06 2024-09-16T20:48:50.754 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.754 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.754 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:50.754 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768166+0000", 2024-09-16T20:48:50.754 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0550000000000002e-06 2024-09-16T20:48:50.754 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.754 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.754 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:50.754 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768171+0000", 2024-09-16T20:48:50.754 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3170000000000003e-06 2024-09-16T20:48:50.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.755 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.755 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:50.755 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819084+0000", 2024-09-16T20:48:50.755 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050913355 2024-09-16T20:48:50.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.755 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.755 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:50.755 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819170+0000", 2024-09-16T20:48:50.755 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6328000000000004e-05 2024-09-16T20:48:50.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.755 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.755 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:50.756 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819274+0000", 2024-09-16T20:48:50.756 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010401 2024-09-16T20:48:50.756 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.756 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:50.756 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.756 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.756 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.756 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5018 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2125824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:50.756 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768178+0000", 2024-09-16T20:48:50.756 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 30.952902890000001, 2024-09-16T20:48:50.756 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051227013000000002, 2024-09-16T20:48:50.756 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:50.756 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:50.756 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:50.757 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:50.757 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:50.757 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:50.757 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5018 2024-09-16T20:48:50.757 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.757 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:50.757 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.757 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:50.757 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768178+0000", 2024-09-16T20:48:50.757 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.757 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.757 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.757 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:50.758 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768178+0000", 2024-09-16T20:48:50.758 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.758 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.758 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.758 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:50.758 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768179+0000", 2024-09-16T20:48:50.758 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9200000000000002e-07 2024-09-16T20:48:50.758 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.758 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.758 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:50.758 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768180+0000", 2024-09-16T20:48:50.758 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3790000000000001e-06 2024-09-16T20:48:50.758 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.758 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.759 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:50.759 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768184+0000", 2024-09-16T20:48:50.759 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5470000000000001e-06 2024-09-16T20:48:50.759 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.759 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.759 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:50.759 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768187+0000", 2024-09-16T20:48:50.759 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8119999999999999e-06 2024-09-16T20:48:50.759 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.759 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.759 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:50.759 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819305+0000", 2024-09-16T20:48:50.759 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051118683999999998 2024-09-16T20:48:50.759 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.760 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.760 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:50.760 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819351+0000", 2024-09-16T20:48:50.760 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5757999999999998e-05 2024-09-16T20:48:50.760 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.760 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.760 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:50.760 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819405+0000", 2024-09-16T20:48:50.760 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4441000000000003e-05 2024-09-16T20:48:50.760 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.760 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:50.760 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.760 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.761 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.761 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5020 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2134016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:50.761 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768190+0000", 2024-09-16T20:48:50.761 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 30.952891212000001, 2024-09-16T20:48:50.761 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051449661000000001, 2024-09-16T20:48:50.761 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:50.761 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:50.761 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:50.761 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:50.761 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:50.761 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:50.761 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5020 2024-09-16T20:48:50.761 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.762 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:50.762 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.762 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:50.762 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768190+0000", 2024-09-16T20:48:50.762 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.762 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.762 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.762 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:50.762 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768190+0000", 2024-09-16T20:48:50.762 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.762 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.762 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.762 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:50.762 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768282+0000", 2024-09-16T20:48:50.763 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2263999999999996e-05 2024-09-16T20:48:50.763 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.763 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.763 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:50.763 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768286+0000", 2024-09-16T20:48:50.763 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.529e-06 2024-09-16T20:48:50.763 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.763 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.763 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:50.763 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768294+0000", 2024-09-16T20:48:50.763 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6990000000000003e-06 2024-09-16T20:48:50.763 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.763 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.763 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:50.764 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768306+0000", 2024-09-16T20:48:50.764 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2112e-05 2024-09-16T20:48:50.764 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.764 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.764 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:50.764 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819449+0000", 2024-09-16T20:48:50.764 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051143521999999997 2024-09-16T20:48:50.764 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.764 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.764 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:50.764 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819539+0000", 2024-09-16T20:48:50.764 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0022e-05 2024-09-16T20:48:50.764 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.765 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.765 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:50.765 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819640+0000", 2024-09-16T20:48:50.765 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000100513 2024-09-16T20:48:50.765 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.765 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:50.765 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.765 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.765 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.765 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5022 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2142208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:50.765 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768326+0000", 2024-09-16T20:48:50.765 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 30.952755823, 2024-09-16T20:48:50.765 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051436952000000001, 2024-09-16T20:48:50.765 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:50.766 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:50.766 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:50.766 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:50.766 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:50.766 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:50.766 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5022 2024-09-16T20:48:50.766 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.766 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:50.766 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.766 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:50.766 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768326+0000", 2024-09-16T20:48:50.766 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.766 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.766 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.767 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:50.767 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768326+0000", 2024-09-16T20:48:50.767 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.767 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.767 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.767 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:50.767 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768477+0000", 2024-09-16T20:48:50.767 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015109300000000001 2024-09-16T20:48:50.767 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.767 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.767 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:50.767 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768480+0000", 2024-09-16T20:48:50.767 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8580000000000002e-06 2024-09-16T20:48:50.768 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.768 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.768 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:50.768 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768489+0000", 2024-09-16T20:48:50.768 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4600000000000003e-06 2024-09-16T20:48:50.768 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.768 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.768 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:50.768 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768493+0000", 2024-09-16T20:48:50.768 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2270000000000004e-06 2024-09-16T20:48:50.768 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.768 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.768 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:50.769 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819672+0000", 2024-09-16T20:48:50.769 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051178605000000002 2024-09-16T20:48:50.769 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.769 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.769 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:50.769 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819711+0000", 2024-09-16T20:48:50.769 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9357000000000003e-05 2024-09-16T20:48:50.769 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.769 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.769 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:50.769 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819762+0000", 2024-09-16T20:48:50.769 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1351999999999999e-05 2024-09-16T20:48:50.769 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.770 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:50.770 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.770 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.770 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.770 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5024 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2150400~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:50.770 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768501+0000", 2024-09-16T20:48:50.770 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 30.952580345000001, 2024-09-16T20:48:50.770 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051402678, 2024-09-16T20:48:50.770 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:50.770 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:50.770 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:50.770 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:50.770 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:50.770 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:50.771 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5024 2024-09-16T20:48:50.771 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.771 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:50.771 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.771 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:50.771 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768501+0000", 2024-09-16T20:48:50.771 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.771 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.771 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.771 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:50.771 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768501+0000", 2024-09-16T20:48:50.771 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.772 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.772 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.772 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:50.772 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768619+0000", 2024-09-16T20:48:50.772 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118187 2024-09-16T20:48:50.772 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.772 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.772 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:50.772 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768624+0000", 2024-09-16T20:48:50.772 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3259999999999997e-06 2024-09-16T20:48:50.772 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.772 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.772 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:50.773 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768631+0000", 2024-09-16T20:48:50.773 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6490000000000008e-06 2024-09-16T20:48:50.773 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.773 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.773 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:50.773 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768637+0000", 2024-09-16T20:48:50.773 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4210000000000003e-06 2024-09-16T20:48:50.773 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.773 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.773 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:50.773 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819800+0000", 2024-09-16T20:48:50.773 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051163486000000001 2024-09-16T20:48:50.773 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.773 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.774 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:50.774 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819845+0000", 2024-09-16T20:48:50.774 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4830999999999998e-05 2024-09-16T20:48:50.774 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.774 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.774 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:50.774 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819904+0000", 2024-09-16T20:48:50.774 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8777999999999999e-05 2024-09-16T20:48:50.774 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.774 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:50.774 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.774 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.774 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.775 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5026 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2158592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:50.775 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768646+0000", 2024-09-16T20:48:50.775 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 30.952434897, 2024-09-16T20:48:50.775 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051384386999999997, 2024-09-16T20:48:50.775 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:50.775 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:50.775 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:50.775 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:50.775 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:50.775 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:50.775 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5026 2024-09-16T20:48:50.775 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.775 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:50.776 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.776 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:50.776 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768646+0000", 2024-09-16T20:48:50.776 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.776 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.776 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.776 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:50.776 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768646+0000", 2024-09-16T20:48:50.776 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.776 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.776 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.776 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:50.776 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768775+0000", 2024-09-16T20:48:50.777 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012813200000000001 2024-09-16T20:48:50.777 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.777 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.777 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:50.777 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768778+0000", 2024-09-16T20:48:50.777 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2150000000000001e-06 2024-09-16T20:48:50.777 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.777 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.777 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:50.777 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768786+0000", 2024-09-16T20:48:50.777 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3259999999999996e-06 2024-09-16T20:48:50.777 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.777 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.777 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:50.778 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768790+0000", 2024-09-16T20:48:50.778 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.216e-06 2024-09-16T20:48:50.778 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.778 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.778 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:50.778 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819939+0000", 2024-09-16T20:48:50.778 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051149018999999997 2024-09-16T20:48:50.778 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.778 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.778 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:50.778 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819979+0000", 2024-09-16T20:48:50.778 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9311999999999999e-05 2024-09-16T20:48:50.778 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.779 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.779 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:50.779 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820031+0000", 2024-09-16T20:48:50.779 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2166999999999997e-05 2024-09-16T20:48:50.779 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.779 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:50.779 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.779 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.779 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.779 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5028 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2166784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:50.779 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768798+0000", 2024-09-16T20:48:50.779 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 30.952283254000001, 2024-09-16T20:48:50.779 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051388912000000002, 2024-09-16T20:48:50.780 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:50.780 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:50.780 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:50.780 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:50.780 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:50.780 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:50.780 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5028 2024-09-16T20:48:50.780 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.780 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:50.780 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.780 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:50.780 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768798+0000", 2024-09-16T20:48:50.780 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.781 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.781 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.781 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:50.781 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768798+0000", 2024-09-16T20:48:50.781 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.781 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.781 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.781 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:50.781 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768989+0000", 2024-09-16T20:48:50.781 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000190951 2024-09-16T20:48:50.781 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.781 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.781 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:50.782 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768993+0000", 2024-09-16T20:48:50.782 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9770000000000002e-06 2024-09-16T20:48:50.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.782 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.782 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:50.782 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.769002+0000", 2024-09-16T20:48:50.782 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5469999999999997e-06 2024-09-16T20:48:50.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.782 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.782 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:50.782 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.769020+0000", 2024-09-16T20:48:50.782 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8216999999999999e-05 2024-09-16T20:48:50.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.783 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.783 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:50.783 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820070+0000", 2024-09-16T20:48:50.783 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051049998999999999 2024-09-16T20:48:50.783 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.783 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.783 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:50.783 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820119+0000", 2024-09-16T20:48:50.783 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8962000000000003e-05 2024-09-16T20:48:50.783 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.783 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.783 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:50.783 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820187+0000", 2024-09-16T20:48:50.783 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8258999999999998e-05 2024-09-16T20:48:50.784 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.784 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:50.784 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.784 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.784 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.784 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5030 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2174976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:50.784 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.769031+0000", 2024-09-16T20:48:50.784 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 30.952050141000001, 2024-09-16T20:48:50.784 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051289830000000002, 2024-09-16T20:48:50.784 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:50.784 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:50.784 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:50.784 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:50.785 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:50.785 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:50.785 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5030 2024-09-16T20:48:50.785 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.785 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:50.785 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.785 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:50.785 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.769031+0000", 2024-09-16T20:48:50.785 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.785 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.785 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.785 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:50.785 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.769031+0000", 2024-09-16T20:48:50.786 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.786 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:50.786 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.773332+0000", 2024-09-16T20:48:50.786 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0043004039999999999 2024-09-16T20:48:50.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.786 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:50.786 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.773336+0000", 2024-09-16T20:48:50.786 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7849999999999999e-06 2024-09-16T20:48:50.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.786 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:50.787 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.773347+0000", 2024-09-16T20:48:50.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0482999999999999e-05 2024-09-16T20:48:50.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.787 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.787 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:50.787 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.773354+0000", 2024-09-16T20:48:50.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1559999999999998e-06 2024-09-16T20:48:50.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.787 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.787 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:50.787 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820216+0000", 2024-09-16T20:48:50.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046861591000000001 2024-09-16T20:48:50.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.788 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.788 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:50.788 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820258+0000", 2024-09-16T20:48:50.788 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2653e-05 2024-09-16T20:48:50.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.788 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.788 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:50.788 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820321+0000", 2024-09-16T20:48:50.788 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2757999999999998e-05 2024-09-16T20:48:50.788 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.788 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:50.788 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.789 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.789 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5067 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 6918144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:50.789 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.775279+0000", 2024-09-16T20:48:50.789 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 30.945801875000001, 2024-09-16T20:48:50.789 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050171387999999997, 2024-09-16T20:48:50.789 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:50.789 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:50.789 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:50.789 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:50.789 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:50.789 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:50.789 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5067 2024-09-16T20:48:50.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.790 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:50.790 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.790 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:50.790 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775279+0000", 2024-09-16T20:48:50.790 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.790 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.790 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:50.790 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775279+0000", 2024-09-16T20:48:50.790 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.790 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.790 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:50.791 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775432+0000", 2024-09-16T20:48:50.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015257399999999999 2024-09-16T20:48:50.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.791 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.791 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:50.791 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775436+0000", 2024-09-16T20:48:50.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0180000000000003e-06 2024-09-16T20:48:50.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.791 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.791 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:50.791 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775447+0000", 2024-09-16T20:48:50.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.077e-05 2024-09-16T20:48:50.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.792 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.792 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:50.792 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775456+0000", 2024-09-16T20:48:50.792 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4089999999999992e-06 2024-09-16T20:48:50.792 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.792 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.792 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:50.792 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825150+0000", 2024-09-16T20:48:50.792 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049693711000000002 2024-09-16T20:48:50.792 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.792 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.792 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:50.792 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825337+0000", 2024-09-16T20:48:50.793 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000187214 2024-09-16T20:48:50.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.793 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.793 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:50.793 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825451+0000", 2024-09-16T20:48:50.793 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011369199999999999 2024-09-16T20:48:50.793 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.793 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:50.793 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.793 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.793 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5068 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2330624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:50.793 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.775469+0000", 2024-09-16T20:48:50.794 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 30.945611914000001, 2024-09-16T20:48:50.794 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050198334999999997, 2024-09-16T20:48:50.794 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:50.794 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:50.794 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:50.794 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:50.794 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:50.794 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:50.794 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5068 2024-09-16T20:48:50.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.794 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:50.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.794 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:50.794 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775469+0000", 2024-09-16T20:48:50.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.795 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.795 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:50.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775469+0000", 2024-09-16T20:48:50.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.795 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.795 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:50.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775720+0000", 2024-09-16T20:48:50.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000250661 2024-09-16T20:48:50.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.795 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.796 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:50.796 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775724+0000", 2024-09-16T20:48:50.796 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8700000000000002e-06 2024-09-16T20:48:50.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.796 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.796 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:50.796 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775733+0000", 2024-09-16T20:48:50.796 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4239999999999999e-06 2024-09-16T20:48:50.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.796 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.796 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:50.796 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775740+0000", 2024-09-16T20:48:50.796 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9519999999999997e-06 2024-09-16T20:48:50.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.797 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.797 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:50.797 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825484+0000", 2024-09-16T20:48:50.797 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049743385000000001 2024-09-16T20:48:50.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.797 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.797 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:50.797 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825572+0000", 2024-09-16T20:48:50.797 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7834e-05 2024-09-16T20:48:50.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.797 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.797 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:50.798 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825668+0000", 2024-09-16T20:48:50.798 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6209e-05 2024-09-16T20:48:50.798 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.798 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:50.798 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.798 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.798 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5069 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 6926336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:50.798 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.775753+0000", 2024-09-16T20:48:50.798 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 30.945328691, 2024-09-16T20:48:50.798 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050146430999999998, 2024-09-16T20:48:50.798 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:50.798 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:50.798 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:50.799 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:50.799 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:50.799 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:50.799 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5069 2024-09-16T20:48:50.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.799 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:50.799 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.799 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:50.799 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775753+0000", 2024-09-16T20:48:50.799 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.799 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.799 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:50.799 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775753+0000", 2024-09-16T20:48:50.800 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.800 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:50.800 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775865+0000", 2024-09-16T20:48:50.800 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000112402 2024-09-16T20:48:50.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.800 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:50.800 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775869+0000", 2024-09-16T20:48:50.800 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5130000000000001e-06 2024-09-16T20:48:50.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.801 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.801 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:50.801 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775877+0000", 2024-09-16T20:48:50.801 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1359999999999997e-06 2024-09-16T20:48:50.801 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.801 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.801 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:50.801 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775882+0000", 2024-09-16T20:48:50.801 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6710000000000004e-06 2024-09-16T20:48:50.801 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.801 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.801 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:50.801 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825729+0000", 2024-09-16T20:48:50.801 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049846898000000001 2024-09-16T20:48:50.802 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.802 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.802 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:50.802 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825807+0000", 2024-09-16T20:48:50.802 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7875999999999998e-05 2024-09-16T20:48:50.802 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.802 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.802 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:50.802 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825899+0000", 2024-09-16T20:48:50.802 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1935000000000007e-05 2024-09-16T20:48:50.802 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.802 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:50.802 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.803 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.803 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5070 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2338816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:50.803 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.775892+0000", 2024-09-16T20:48:50.803 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 30.945188837, 2024-09-16T20:48:50.803 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050155311000000001, 2024-09-16T20:48:50.803 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:50.803 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:50.803 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:50.803 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:50.803 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:50.803 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:50.803 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5070 2024-09-16T20:48:50.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.804 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:50.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.804 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:50.804 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775892+0000", 2024-09-16T20:48:50.804 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.804 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:50.804 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775892+0000", 2024-09-16T20:48:50.804 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.805 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:50.805 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776142+0000", 2024-09-16T20:48:50.805 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00024950500000000002 2024-09-16T20:48:50.805 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.805 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.805 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:50.805 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776146+0000", 2024-09-16T20:48:50.805 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2819999999999998e-06 2024-09-16T20:48:50.805 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.805 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.805 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:50.805 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776157+0000", 2024-09-16T20:48:50.805 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0807e-05 2024-09-16T20:48:50.806 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.806 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.806 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:50.806 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776165+0000", 2024-09-16T20:48:50.806 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1300000000000001e-06 2024-09-16T20:48:50.806 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.806 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.806 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:50.806 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825935+0000", 2024-09-16T20:48:50.806 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049769300000000002 2024-09-16T20:48:50.806 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.806 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.806 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:50.806 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825991+0000", 2024-09-16T20:48:50.807 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6177000000000001e-05 2024-09-16T20:48:50.807 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.807 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.807 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:50.807 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826048+0000", 2024-09-16T20:48:50.807 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.711e-05 2024-09-16T20:48:50.807 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.807 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:50.807 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.807 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.807 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.807 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5071 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 6934528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:50.807 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.776177+0000", 2024-09-16T20:48:50.807 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 30.944904434000001, 2024-09-16T20:48:50.808 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050036886000000003, 2024-09-16T20:48:50.808 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:50.808 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:50.808 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:50.808 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:50.808 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:50.808 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:50.808 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5071 2024-09-16T20:48:50.808 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.808 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:50.808 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.808 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:50.808 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776177+0000", 2024-09-16T20:48:50.809 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.809 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.809 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:50.809 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776177+0000", 2024-09-16T20:48:50.809 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.809 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.809 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:50.809 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776339+0000", 2024-09-16T20:48:50.809 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016238099999999999 2024-09-16T20:48:50.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.809 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.810 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:50.810 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776343+0000", 2024-09-16T20:48:50.810 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9670000000000003e-06 2024-09-16T20:48:50.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.810 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.810 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:50.810 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776354+0000", 2024-09-16T20:48:50.810 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0789e-05 2024-09-16T20:48:50.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.810 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.810 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:50.810 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776361+0000", 2024-09-16T20:48:50.810 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5200000000000003e-06 2024-09-16T20:48:50.811 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.811 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.811 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:50.811 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826078+0000", 2024-09-16T20:48:50.811 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049717513999999997 2024-09-16T20:48:50.811 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.811 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.811 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:50.811 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826141+0000", 2024-09-16T20:48:50.811 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2446000000000005e-05 2024-09-16T20:48:50.811 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.811 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.811 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:50.811 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826214+0000", 2024-09-16T20:48:50.812 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3269000000000006e-05 2024-09-16T20:48:50.812 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.812 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:50.812 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.812 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.812 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.812 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5072 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2347008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:50.812 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.776370+0000", 2024-09-16T20:48:50.812 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 30.944710872999998, 2024-09-16T20:48:50.812 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050097653999999998, 2024-09-16T20:48:50.812 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:50.812 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:50.812 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:50.813 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:50.813 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:50.813 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:50.813 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5072 2024-09-16T20:48:50.813 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.813 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:50.813 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.813 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:50.813 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776370+0000", 2024-09-16T20:48:50.813 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.813 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.813 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.813 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:50.814 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776370+0000", 2024-09-16T20:48:50.814 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:50.814 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.814 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.814 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:50.814 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776645+0000", 2024-09-16T20:48:50.814 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000274634 2024-09-16T20:48:50.814 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.814 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.814 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:50.814 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776654+0000", 2024-09-16T20:48:50.814 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9390000000000003e-06 2024-09-16T20:48:50.814 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.814 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.815 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:50.815 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776665+0000", 2024-09-16T20:48:50.815 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0524e-05 2024-09-16T20:48:50.815 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.815 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.815 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:50.815 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776671+0000", 2024-09-16T20:48:50.815 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7390000000000002e-06 2024-09-16T20:48:50.815 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.815 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.815 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:50.815 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826261+0000", 2024-09-16T20:48:50.815 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049589995999999997 2024-09-16T20:48:50.815 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.816 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.816 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:50.816 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826363+0000", 2024-09-16T20:48:50.816 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010158999999999999 2024-09-16T20:48:50.816 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:50.816 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:50.816 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:50.816 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826468+0000", 2024-09-16T20:48:50.816 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000105232 2024-09-16T20:48:50.816 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.816 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:50.816 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.816 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:50.817 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:50.817 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:50.818 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:48:50.819 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:50.817+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:50.918 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:50.917+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:50.966 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:50.966 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:48:50.966 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:48:50.966 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:50.975 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:48:51.058 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:51.057+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:51.120 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:51.118+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:51.122 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:51.122 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:48:51.122 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:48:51.122 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:48:51.122 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:51.131 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:48:51.220 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:51.219+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:51.280 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:51.280 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:48:51.280 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:48:51.280 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:48:51.280 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.280 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:780 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 933888~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:51.280 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.239489+0000", 2024-09-16T20:48:51.280 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.038708774, 2024-09-16T20:48:51.280 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00741699, 2024-09-16T20:48:51.280 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:51.281 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:51.281 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:51.281 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:51.281 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:51.281 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:51.281 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 780 2024-09-16T20:48:51.281 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.281 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:51.282 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.282 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:51.282 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239489+0000", 2024-09-16T20:48:51.282 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:51.282 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.282 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.282 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:51.282 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239489+0000", 2024-09-16T20:48:51.282 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:51.282 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.282 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.282 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:51.282 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239490+0000", 2024-09-16T20:48:51.282 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8599999999999998e-07 2024-09-16T20:48:51.283 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.283 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.283 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:51.283 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239492+0000", 2024-09-16T20:48:51.283 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1380000000000001e-06 2024-09-16T20:48:51.283 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.283 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.283 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:51.283 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239499+0000", 2024-09-16T20:48:51.283 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9530000000000002e-06 2024-09-16T20:48:51.283 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.283 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.283 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:51.283 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239504+0000", 2024-09-16T20:48:51.284 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0690000000000001e-06 2024-09-16T20:48:51.284 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.284 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.284 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:51.284 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246714+0000", 2024-09-16T20:48:51.284 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.007209844 2024-09-16T20:48:51.284 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.284 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.284 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:51.284 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246804+0000", 2024-09-16T20:48:51.284 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0109000000000001e-05 2024-09-16T20:48:51.284 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.284 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.285 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:51.285 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246906+0000", 2024-09-16T20:48:51.285 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000102391 2024-09-16T20:48:51.285 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:51.285 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:51.285 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:51.285 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.285 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.285 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:781 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1048576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:51.285 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.239892+0000", 2024-09-16T20:48:51.285 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.038306403999997, 2024-09-16T20:48:51.285 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0072223799999999996, 2024-09-16T20:48:51.285 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:51.286 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:51.286 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:51.286 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:51.286 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:51.286 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:51.286 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 781 2024-09-16T20:48:51.286 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.286 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:51.286 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.286 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:51.286 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239892+0000", 2024-09-16T20:48:51.286 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:51.286 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.286 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.287 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:51.287 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239892+0000", 2024-09-16T20:48:51.287 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:51.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.287 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.287 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:51.287 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239893+0000", 2024-09-16T20:48:51.287 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1399999999999995e-07 2024-09-16T20:48:51.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.287 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.287 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:51.287 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239897+0000", 2024-09-16T20:48:51.287 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4859999999999999e-06 2024-09-16T20:48:51.288 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.288 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.288 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:51.288 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239907+0000", 2024-09-16T20:48:51.288 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4539999999999996e-06 2024-09-16T20:48:51.288 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.288 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.288 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:51.288 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239913+0000", 2024-09-16T20:48:51.288 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7880000000000001e-06 2024-09-16T20:48:51.288 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.288 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.288 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:51.288 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246937+0000", 2024-09-16T20:48:51.289 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0070239999999999999 2024-09-16T20:48:51.289 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.289 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.289 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:51.289 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247013+0000", 2024-09-16T20:48:51.289 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5894000000000002e-05 2024-09-16T20:48:51.289 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.289 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.289 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:51.289 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247114+0000", 2024-09-16T20:48:51.289 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000100844 2024-09-16T20:48:51.289 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:51.289 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:51.289 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:51.290 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.290 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.290 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:782 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1146880~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:51.290 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.239925+0000", 2024-09-16T20:48:51.290 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.038273431, 2024-09-16T20:48:51.290 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0074161840000000001, 2024-09-16T20:48:51.290 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:51.290 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:51.290 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:51.290 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:51.290 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:51.290 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:51.290 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 782 2024-09-16T20:48:51.290 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.291 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:51.291 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.291 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:51.291 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239925+0000", 2024-09-16T20:48:51.291 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:51.291 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.291 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.291 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:51.291 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239925+0000", 2024-09-16T20:48:51.291 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:51.291 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.291 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.291 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:51.292 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239925+0000", 2024-09-16T20:48:51.292 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7199999999999999e-07 2024-09-16T20:48:51.292 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.292 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.292 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:51.292 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239928+0000", 2024-09-16T20:48:51.292 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5349999999999999e-06 2024-09-16T20:48:51.292 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.292 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.293 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:51.293 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239935+0000", 2024-09-16T20:48:51.293 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7680000000000003e-06 2024-09-16T20:48:51.293 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.293 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.293 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:51.293 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239940+0000", 2024-09-16T20:48:51.293 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.152e-06 2024-09-16T20:48:51.293 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.293 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.293 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:51.293 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247143+0000", 2024-09-16T20:48:51.293 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0072037309999999997 2024-09-16T20:48:51.293 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.294 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.294 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:51.294 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247230+0000", 2024-09-16T20:48:51.294 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6598e-05 2024-09-16T20:48:51.294 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.294 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.294 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:51.294 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247341+0000", 2024-09-16T20:48:51.294 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011092800000000001 2024-09-16T20:48:51.294 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:51.294 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:51.294 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:51.294 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.294 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.295 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:783 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1261568~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:51.295 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.239947+0000", 2024-09-16T20:48:51.295 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.038251109000001, 2024-09-16T20:48:51.295 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0076044789999999999, 2024-09-16T20:48:51.295 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:51.295 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:51.295 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:51.295 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:51.295 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:51.295 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:51.295 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 783 2024-09-16T20:48:51.295 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.295 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:51.296 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.296 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:51.296 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239947+0000", 2024-09-16T20:48:51.296 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:51.296 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.296 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.296 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:51.296 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239947+0000", 2024-09-16T20:48:51.296 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:51.296 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.296 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.296 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:51.296 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239947+0000", 2024-09-16T20:48:51.297 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9400000000000001e-07 2024-09-16T20:48:51.297 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.297 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.297 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:51.297 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239950+0000", 2024-09-16T20:48:51.297 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2350000000000002e-06 2024-09-16T20:48:51.297 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.297 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.297 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:51.297 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239956+0000", 2024-09-16T20:48:51.297 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3400000000000003e-06 2024-09-16T20:48:51.297 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.297 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.297 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:51.297 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239961+0000", 2024-09-16T20:48:51.298 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.04e-06 2024-09-16T20:48:51.298 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.298 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.298 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:51.298 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247371+0000", 2024-09-16T20:48:51.298 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0074096810000000004 2024-09-16T20:48:51.298 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.298 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.298 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:51.298 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247451+0000", 2024-09-16T20:48:51.298 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9845999999999994e-05 2024-09-16T20:48:51.298 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.298 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.299 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:51.299 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247552+0000", 2024-09-16T20:48:51.299 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000100943 2024-09-16T20:48:51.299 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:51.299 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:51.299 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:51.299 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.299 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.299 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:784 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1376256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:51.299 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.239997+0000", 2024-09-16T20:48:51.299 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.038201452999999, 2024-09-16T20:48:51.299 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.007775372, 2024-09-16T20:48:51.299 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:51.299 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:51.300 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:51.300 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:51.300 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:51.300 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:51.300 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 784 2024-09-16T20:48:51.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.300 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:51.300 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.300 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:51.300 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239997+0000", 2024-09-16T20:48:51.300 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:51.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.300 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.300 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:51.301 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239997+0000", 2024-09-16T20:48:51.301 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:51.301 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.301 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.301 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:51.301 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239997+0000", 2024-09-16T20:48:51.301 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1799999999999995e-07 2024-09-16T20:48:51.301 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.301 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.301 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:51.301 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240001+0000", 2024-09-16T20:48:51.301 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6009999999999999e-06 2024-09-16T20:48:51.301 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.301 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.302 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:51.302 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240009+0000", 2024-09-16T20:48:51.302 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5809999999999999e-06 2024-09-16T20:48:51.302 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.302 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.302 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:51.302 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240015+0000", 2024-09-16T20:48:51.302 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0100000000000001e-06 2024-09-16T20:48:51.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.303 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.303 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:51.303 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247583+0000", 2024-09-16T20:48:51.303 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0075685730000000003 2024-09-16T20:48:51.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.303 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.303 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:51.303 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247672+0000", 2024-09-16T20:48:51.303 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8480000000000007e-05 2024-09-16T20:48:51.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.303 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.303 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:51.303 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247772+0000", 2024-09-16T20:48:51.304 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000100509 2024-09-16T20:48:51.304 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:51.304 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:51.304 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:51.304 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.304 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.304 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:785 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1474560~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:51.304 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240024+0000", 2024-09-16T20:48:51.304 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.038173866999998, 2024-09-16T20:48:51.304 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.008096776, 2024-09-16T20:48:51.304 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:51.304 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:51.304 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:51.305 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:51.305 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:51.305 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:51.305 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 785 2024-09-16T20:48:51.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.305 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:51.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.305 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:51.305 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240024+0000", 2024-09-16T20:48:51.305 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:51.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.305 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:51.305 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240024+0000", 2024-09-16T20:48:51.306 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:51.306 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.306 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.306 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:51.306 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240025+0000", 2024-09-16T20:48:51.306 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5000000000000003e-07 2024-09-16T20:48:51.306 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.306 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.306 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:51.306 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240027+0000", 2024-09-16T20:48:51.306 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2900000000000001e-06 2024-09-16T20:48:51.306 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.306 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.306 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:51.307 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240036+0000", 2024-09-16T20:48:51.307 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4710000000000007e-06 2024-09-16T20:48:51.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.307 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.307 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:51.307 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240041+0000", 2024-09-16T20:48:51.307 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4689999999999996e-06 2024-09-16T20:48:51.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.307 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.307 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:51.307 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247821+0000", 2024-09-16T20:48:51.307 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0077797919999999998 2024-09-16T20:48:51.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.307 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.308 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:51.308 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247921+0000", 2024-09-16T20:48:51.308 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9814000000000004e-05 2024-09-16T20:48:51.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.308 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.308 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:51.308 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248121+0000", 2024-09-16T20:48:51.308 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00020039 2024-09-16T20:48:51.308 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:51.308 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:51.308 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:51.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.308 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.309 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:940 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 81920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:51.309 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240569+0000", 2024-09-16T20:48:51.309 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.037629609, 2024-09-16T20:48:51.309 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0078289759999999996, 2024-09-16T20:48:51.309 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:51.309 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:51.309 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:51.309 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:51.309 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:51.309 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:51.309 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 940 2024-09-16T20:48:51.309 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.309 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:51.309 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.310 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:51.310 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240569+0000", 2024-09-16T20:48:51.310 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:51.310 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.310 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.310 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:51.310 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240569+0000", 2024-09-16T20:48:51.310 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:51.310 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.310 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.310 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:51.310 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240569+0000", 2024-09-16T20:48:51.310 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2600000000000001e-07 2024-09-16T20:48:51.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.311 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.311 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:51.311 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240575+0000", 2024-09-16T20:48:51.311 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1159999999999998e-06 2024-09-16T20:48:51.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.311 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.311 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:51.311 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240587+0000", 2024-09-16T20:48:51.311 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1915e-05 2024-09-16T20:48:51.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.311 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.311 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:51.311 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240594+0000", 2024-09-16T20:48:51.312 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5610000000000001e-06 2024-09-16T20:48:51.312 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.312 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.312 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:51.312 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248158+0000", 2024-09-16T20:48:51.312 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0075635370000000004 2024-09-16T20:48:51.312 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.312 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.312 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:51.313 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248278+0000", 2024-09-16T20:48:51.313 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012009400000000001 2024-09-16T20:48:51.313 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.313 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.313 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:51.313 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248398+0000", 2024-09-16T20:48:51.313 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000119827 2024-09-16T20:48:51.313 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:51.313 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:51.313 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:51.313 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.313 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.313 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:941 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 196608~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:51.314 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240608+0000", 2024-09-16T20:48:51.314 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.037590031999997, 2024-09-16T20:48:51.314 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0080495519999999997, 2024-09-16T20:48:51.314 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:51.314 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:51.314 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:51.314 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:51.314 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:51.314 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:51.314 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 941 2024-09-16T20:48:51.314 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.314 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:51.314 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.315 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:51.315 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240608+0000", 2024-09-16T20:48:51.315 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:51.315 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.315 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.315 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:51.315 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240608+0000", 2024-09-16T20:48:51.315 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:51.315 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.315 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.315 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:51.315 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240609+0000", 2024-09-16T20:48:51.315 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7000000000000004e-07 2024-09-16T20:48:51.315 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.316 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.316 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:51.316 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240612+0000", 2024-09-16T20:48:51.316 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8310000000000001e-06 2024-09-16T20:48:51.316 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.316 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.316 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:51.316 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240619+0000", 2024-09-16T20:48:51.316 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8429999999999993e-06 2024-09-16T20:48:51.316 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.316 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.316 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:51.316 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240625+0000", 2024-09-16T20:48:51.317 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5659999999999997e-06 2024-09-16T20:48:51.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.317 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.317 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:51.317 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248456+0000", 2024-09-16T20:48:51.317 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0078313940000000002 2024-09-16T20:48:51.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.317 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.317 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:51.317 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248548+0000", 2024-09-16T20:48:51.317 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1324999999999997e-05 2024-09-16T20:48:51.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.317 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.318 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:51.318 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248658+0000", 2024-09-16T20:48:51.318 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000109923 2024-09-16T20:48:51.318 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:51.318 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:51.318 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:51.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.318 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.318 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:942 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 294912~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:51.318 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240675+0000", 2024-09-16T20:48:51.318 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.037523225000001, 2024-09-16T20:48:51.318 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0082135699999999999, 2024-09-16T20:48:51.318 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:51.318 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:51.319 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:51.319 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:51.319 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:51.319 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:51.319 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 942 2024-09-16T20:48:51.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.319 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:51.319 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.319 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:51.319 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240675+0000", 2024-09-16T20:48:51.319 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:51.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.319 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.320 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:51.320 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240675+0000", 2024-09-16T20:48:51.320 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:51.320 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.320 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.320 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:51.320 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240676+0000", 2024-09-16T20:48:51.320 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.37e-07 2024-09-16T20:48:51.320 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.320 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.320 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:51.320 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240679+0000", 2024-09-16T20:48:51.320 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4060000000000001e-06 2024-09-16T20:48:51.320 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.321 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.321 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:51.321 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240687+0000", 2024-09-16T20:48:51.321 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4640000000000006e-06 2024-09-16T20:48:51.321 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.321 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.321 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:51.321 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240694+0000", 2024-09-16T20:48:51.321 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.246e-06 2024-09-16T20:48:51.321 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.321 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.321 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:51.321 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248693+0000", 2024-09-16T20:48:51.322 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0079997550000000008 2024-09-16T20:48:51.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.322 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.322 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:51.322 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248781+0000", 2024-09-16T20:48:51.322 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7509999999999994e-05 2024-09-16T20:48:51.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.322 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.322 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:51.322 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248889+0000", 2024-09-16T20:48:51.322 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000107552 2024-09-16T20:48:51.322 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:51.322 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:51.323 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:51.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.323 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.323 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:943 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 409600~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:51.323 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240704+0000", 2024-09-16T20:48:51.323 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.037494229000004, 2024-09-16T20:48:51.323 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0083978209999999998, 2024-09-16T20:48:51.323 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:51.323 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:51.323 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:51.324 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:51.324 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:51.324 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:51.324 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 943 2024-09-16T20:48:51.324 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.324 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:51.324 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.324 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:51.324 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240704+0000", 2024-09-16T20:48:51.324 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:51.324 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.324 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.324 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:51.325 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240704+0000", 2024-09-16T20:48:51.325 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:51.325 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.325 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.325 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:51.325 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240704+0000", 2024-09-16T20:48:51.325 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7e-07 2024-09-16T20:48:51.325 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.325 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.325 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:51.325 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240707+0000", 2024-09-16T20:48:51.325 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5809999999999999e-06 2024-09-16T20:48:51.325 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.326 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.326 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:51.326 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240713+0000", 2024-09-16T20:48:51.326 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4949999999999996e-06 2024-09-16T20:48:51.326 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.326 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.326 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:51.326 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240718+0000", 2024-09-16T20:48:51.326 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8890000000000001e-06 2024-09-16T20:48:51.326 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.326 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.326 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:51.326 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248920+0000", 2024-09-16T20:48:51.326 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0082018679999999993 2024-09-16T20:48:51.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.327 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.327 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:51.327 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249007+0000", 2024-09-16T20:48:51.327 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6327000000000002e-05 2024-09-16T20:48:51.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.327 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.327 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:51.327 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249102+0000", 2024-09-16T20:48:51.327 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5191000000000004e-05 2024-09-16T20:48:51.327 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:51.327 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:51.327 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:51.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.328 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.328 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:944 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 524288~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:51.328 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240726+0000", 2024-09-16T20:48:51.328 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.037472424000001, 2024-09-16T20:48:51.328 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0085999960000000004, 2024-09-16T20:48:51.328 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:51.328 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:51.328 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:51.328 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:51.328 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:51.328 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:51.328 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 944 2024-09-16T20:48:51.328 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.329 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:51.329 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.329 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:51.329 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240726+0000", 2024-09-16T20:48:51.329 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:51.329 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.329 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.329 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:51.329 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240726+0000", 2024-09-16T20:48:51.329 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:51.329 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.329 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.329 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:51.329 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240726+0000", 2024-09-16T20:48:51.330 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7099999999999997e-07 2024-09-16T20:48:51.330 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.330 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.330 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:51.330 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240728+0000", 2024-09-16T20:48:51.330 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.125e-06 2024-09-16T20:48:51.330 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.330 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.330 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:51.330 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240734+0000", 2024-09-16T20:48:51.330 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1890000000000004e-06 2024-09-16T20:48:51.330 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.330 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.331 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:51.331 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240739+0000", 2024-09-16T20:48:51.331 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0340000000000004e-06 2024-09-16T20:48:51.331 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.331 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.331 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:51.331 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249131+0000", 2024-09-16T20:48:51.331 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0083919579999999997 2024-09-16T20:48:51.331 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.331 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.331 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:51.331 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249217+0000", 2024-09-16T20:48:51.331 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5154999999999994e-05 2024-09-16T20:48:51.331 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.332 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.332 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:51.332 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249326+0000", 2024-09-16T20:48:51.332 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000109164 2024-09-16T20:48:51.332 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:51.332 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:51.332 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:51.332 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.332 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.332 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:945 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 622592~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:51.332 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240746+0000", 2024-09-16T20:48:51.332 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.037452285000001, 2024-09-16T20:48:51.332 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0088083320000000003, 2024-09-16T20:48:51.332 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:51.333 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:51.333 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:51.333 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:51.333 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:51.333 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:51.333 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 945 2024-09-16T20:48:51.333 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.333 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:51.333 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.334 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:51.334 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240746+0000", 2024-09-16T20:48:51.334 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:51.334 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.334 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.334 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:51.334 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240746+0000", 2024-09-16T20:48:51.334 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:51.334 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.334 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.334 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:51.334 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240746+0000", 2024-09-16T20:48:51.334 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8500000000000002e-07 2024-09-16T20:48:51.335 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.335 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.335 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:51.335 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240748+0000", 2024-09-16T20:48:51.335 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8449999999999999e-06 2024-09-16T20:48:51.335 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.335 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.335 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:51.335 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240754+0000", 2024-09-16T20:48:51.335 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8830000000000003e-06 2024-09-16T20:48:51.335 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.335 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.335 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:51.335 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240759+0000", 2024-09-16T20:48:51.336 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0220000000000004e-06 2024-09-16T20:48:51.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.336 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.336 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:51.336 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249360+0000", 2024-09-16T20:48:51.336 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.008601437 2024-09-16T20:48:51.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.336 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.336 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:51.336 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249453+0000", 2024-09-16T20:48:51.336 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2743000000000003e-05 2024-09-16T20:48:51.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.336 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.336 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:51.337 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249554+0000", 2024-09-16T20:48:51.337 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101017 2024-09-16T20:48:51.337 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:51.337 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:51.337 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:51.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.337 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.337 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:946 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 737280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:51.337 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240766+0000", 2024-09-16T20:48:51.337 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.037432582000001, 2024-09-16T20:48:51.337 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0089092070000000006, 2024-09-16T20:48:51.337 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:51.337 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:51.337 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:51.338 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:51.338 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:51.338 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:51.338 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 946 2024-09-16T20:48:51.338 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.338 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:51.338 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.338 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:51.338 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240766+0000", 2024-09-16T20:48:51.338 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:51.338 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.338 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.338 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:51.338 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240766+0000", 2024-09-16T20:48:51.339 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:51.339 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.339 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.339 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:51.339 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240766+0000", 2024-09-16T20:48:51.339 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0600000000000001e-07 2024-09-16T20:48:51.339 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.339 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.339 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:51.339 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240768+0000", 2024-09-16T20:48:51.339 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8959999999999999e-06 2024-09-16T20:48:51.339 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.339 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.339 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:51.340 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240774+0000", 2024-09-16T20:48:51.340 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0979999999999999e-06 2024-09-16T20:48:51.340 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.340 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.340 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:51.340 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240779+0000", 2024-09-16T20:48:51.340 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.933e-06 2024-09-16T20:48:51.340 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.340 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.340 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:51.340 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249572+0000", 2024-09-16T20:48:51.340 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0087926380000000002 2024-09-16T20:48:51.340 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.341 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.341 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:51.341 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249612+0000", 2024-09-16T20:48:51.341 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0859000000000003e-05 2024-09-16T20:48:51.341 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.341 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.341 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:51.341 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249675+0000", 2024-09-16T20:48:51.341 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2377000000000003e-05 2024-09-16T20:48:51.341 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:51.341 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:51.341 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:51.341 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.341 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.342 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:947 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 851968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:51.342 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240785+0000", 2024-09-16T20:48:51.342 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.03741299, 2024-09-16T20:48:51.342 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0090960260000000001, 2024-09-16T20:48:51.342 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:51.342 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:51.342 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:51.342 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:51.342 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:51.342 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:51.342 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 947 2024-09-16T20:48:51.342 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.342 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:51.342 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.343 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:51.343 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240785+0000", 2024-09-16T20:48:51.343 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:51.343 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.343 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.343 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:51.343 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240785+0000", 2024-09-16T20:48:51.343 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:51.343 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.344 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.344 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:51.344 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240786+0000", 2024-09-16T20:48:51.344 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-16T20:48:51.344 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.344 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.344 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:51.344 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240787+0000", 2024-09-16T20:48:51.344 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.858e-06 2024-09-16T20:48:51.344 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.344 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.344 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:51.344 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240793+0000", 2024-09-16T20:48:51.345 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7760000000000003e-06 2024-09-16T20:48:51.345 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.345 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.345 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:51.345 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240799+0000", 2024-09-16T20:48:51.345 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1109999999999996e-06 2024-09-16T20:48:51.345 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.345 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.345 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:51.345 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249715+0000", 2024-09-16T20:48:51.345 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0089159789999999992 2024-09-16T20:48:51.345 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.345 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.346 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:51.346 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249766+0000", 2024-09-16T20:48:51.346 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0968999999999999e-05 2024-09-16T20:48:51.346 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.346 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.346 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:51.346 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249881+0000", 2024-09-16T20:48:51.346 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114963 2024-09-16T20:48:51.346 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:51.346 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:51.346 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:51.346 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.346 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.347 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:948 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:51.347 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240806+0000", 2024-09-16T20:48:51.347 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.037391810000003, 2024-09-16T20:48:51.347 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0092300609999999995, 2024-09-16T20:48:51.347 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:51.347 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:51.347 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:51.347 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:51.347 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:51.347 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:51.347 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 948 2024-09-16T20:48:51.347 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.347 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:51.347 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.348 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:51.348 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240806+0000", 2024-09-16T20:48:51.348 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:51.348 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.348 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.348 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:51.348 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240806+0000", 2024-09-16T20:48:51.348 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:51.348 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.348 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.348 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:51.348 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240807+0000", 2024-09-16T20:48:51.348 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4499999999999998e-07 2024-09-16T20:48:51.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.349 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.349 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:51.349 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240809+0000", 2024-09-16T20:48:51.349 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.34e-06 2024-09-16T20:48:51.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.349 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.349 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:51.349 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240815+0000", 2024-09-16T20:48:51.349 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1449999999999996e-06 2024-09-16T20:48:51.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.349 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.349 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:51.349 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240821+0000", 2024-09-16T20:48:51.350 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0270000000000001e-06 2024-09-16T20:48:51.350 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.350 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.350 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:51.350 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249920+0000", 2024-09-16T20:48:51.350 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0090988460000000007 2024-09-16T20:48:51.350 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.350 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.350 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:51.350 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249966+0000", 2024-09-16T20:48:51.351 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5918999999999999e-05 2024-09-16T20:48:51.351 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.351 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.351 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:51.351 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250036+0000", 2024-09-16T20:48:51.351 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0438999999999994e-05 2024-09-16T20:48:51.351 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:51.351 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:51.351 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:51.351 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.351 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.351 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:949 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1064960~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:51.351 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240833+0000", 2024-09-16T20:48:51.352 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.037365195, 2024-09-16T20:48:51.352 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0093580339999999994, 2024-09-16T20:48:51.352 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:51.352 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:51.352 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:51.352 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:51.352 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:51.352 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:51.352 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 949 2024-09-16T20:48:51.352 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.352 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:51.352 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.352 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:51.352 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240833+0000", 2024-09-16T20:48:51.353 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:51.353 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.353 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.353 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:51.353 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240833+0000", 2024-09-16T20:48:51.353 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:51.353 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.353 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.353 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:51.353 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240833+0000", 2024-09-16T20:48:51.353 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2899999999999999e-07 2024-09-16T20:48:51.353 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.353 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.354 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:51.354 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240836+0000", 2024-09-16T20:48:51.354 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2649999999999999e-06 2024-09-16T20:48:51.354 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.354 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.354 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:51.354 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240842+0000", 2024-09-16T20:48:51.354 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5579999999999997e-06 2024-09-16T20:48:51.354 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.354 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.354 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:51.354 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240847+0000", 2024-09-16T20:48:51.355 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2009999999999998e-06 2024-09-16T20:48:51.355 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.355 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.355 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:51.355 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250073+0000", 2024-09-16T20:48:51.355 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0092258710000000001 2024-09-16T20:48:51.355 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.355 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.355 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:51.355 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250141+0000", 2024-09-16T20:48:51.355 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7849999999999996e-05 2024-09-16T20:48:51.355 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.355 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.356 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:51.356 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250191+0000", 2024-09-16T20:48:51.356 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.986e-05 2024-09-16T20:48:51.356 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:51.356 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:51.356 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:51.356 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.356 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.356 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:950 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1179648~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:51.356 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240855+0000", 2024-09-16T20:48:51.356 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.037342942999999, 2024-09-16T20:48:51.356 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0095855560000000003, 2024-09-16T20:48:51.356 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:51.357 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:51.357 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:51.357 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:51.357 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:51.357 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:51.357 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 950 2024-09-16T20:48:51.357 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.357 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:51.357 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.357 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:51.357 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240855+0000", 2024-09-16T20:48:51.357 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:51.358 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.358 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.358 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:51.358 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240855+0000", 2024-09-16T20:48:51.358 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:51.358 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.358 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.358 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:51.358 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240856+0000", 2024-09-16T20:48:51.358 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4999999999999998e-07 2024-09-16T20:48:51.358 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.358 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.358 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:51.358 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240858+0000", 2024-09-16T20:48:51.358 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.996e-06 2024-09-16T20:48:51.359 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.359 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.359 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:51.359 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240864+0000", 2024-09-16T20:48:51.359 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9359999999999996e-06 2024-09-16T20:48:51.359 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.359 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.359 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:51.359 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240868+0000", 2024-09-16T20:48:51.359 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7670000000000003e-06 2024-09-16T20:48:51.359 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.359 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.359 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:51.360 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250226+0000", 2024-09-16T20:48:51.360 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0093576100000000006 2024-09-16T20:48:51.360 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.360 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.360 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:51.360 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250289+0000", 2024-09-16T20:48:51.360 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2627999999999997e-05 2024-09-16T20:48:51.360 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.360 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.360 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:51.360 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250441+0000", 2024-09-16T20:48:51.360 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015216900000000001 2024-09-16T20:48:51.360 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:51.361 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:51.361 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:51.361 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.361 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.361 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:951 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1277952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:51.361 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.241044+0000", 2024-09-16T20:48:51.361 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.037153754999999, 2024-09-16T20:48:51.361 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0096266589999999992, 2024-09-16T20:48:51.361 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:51.361 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:51.361 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:51.361 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:51.361 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:51.361 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:51.362 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 951 2024-09-16T20:48:51.362 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.362 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:51.362 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.362 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:51.362 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241044+0000", 2024-09-16T20:48:51.362 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:51.362 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.362 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.362 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:51.362 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241044+0000", 2024-09-16T20:48:51.362 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:51.362 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.362 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.363 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:51.363 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241045+0000", 2024-09-16T20:48:51.363 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5300000000000004e-07 2024-09-16T20:48:51.363 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.363 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.363 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:51.363 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241048+0000", 2024-09-16T20:48:51.363 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.303e-06 2024-09-16T20:48:51.363 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.363 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.363 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:51.363 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241057+0000", 2024-09-16T20:48:51.363 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5550000000000003e-06 2024-09-16T20:48:51.363 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.364 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.364 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:51.364 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241063+0000", 2024-09-16T20:48:51.364 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3470000000000004e-06 2024-09-16T20:48:51.364 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.364 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.364 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:51.364 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250477+0000", 2024-09-16T20:48:51.364 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0094136159999999997 2024-09-16T20:48:51.364 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.364 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.364 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:51.364 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250562+0000", 2024-09-16T20:48:51.365 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4709000000000005e-05 2024-09-16T20:48:51.365 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.365 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.365 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:51.365 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250671+0000", 2024-09-16T20:48:51.365 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010947599999999999 2024-09-16T20:48:51.365 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:51.365 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:51.365 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:51.365 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.365 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.365 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:952 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1392640~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:51.365 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.241130+0000", 2024-09-16T20:48:51.365 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.037067888999999, 2024-09-16T20:48:51.366 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0098096290000000003, 2024-09-16T20:48:51.366 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:51.366 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:51.366 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:51.366 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:51.366 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:51.366 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:51.366 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 952 2024-09-16T20:48:51.366 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.366 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:51.366 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.366 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:51.366 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241130+0000", 2024-09-16T20:48:51.366 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:51.367 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.367 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.367 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:51.367 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241130+0000", 2024-09-16T20:48:51.367 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:51.367 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.367 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.367 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:51.367 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241131+0000", 2024-09-16T20:48:51.367 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.61e-07 2024-09-16T20:48:51.367 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.367 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.367 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:51.367 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241134+0000", 2024-09-16T20:48:51.368 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1080000000000001e-06 2024-09-16T20:48:51.368 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.368 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.368 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:51.368 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241142+0000", 2024-09-16T20:48:51.368 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9119999999999998e-06 2024-09-16T20:48:51.368 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.368 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.368 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:51.368 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241148+0000", 2024-09-16T20:48:51.368 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1410000000000002e-06 2024-09-16T20:48:51.368 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.368 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.368 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:51.369 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250731+0000", 2024-09-16T20:48:51.369 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0095833979999999999 2024-09-16T20:48:51.369 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.369 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.369 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:51.369 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250820+0000", 2024-09-16T20:48:51.369 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8997000000000002e-05 2024-09-16T20:48:51.369 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.369 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.369 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:51.369 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250940+0000", 2024-09-16T20:48:51.369 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000119412 2024-09-16T20:48:51.369 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:51.370 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:51.370 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:51.370 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.370 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.370 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:953 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1507328~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:51.370 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.241157+0000", 2024-09-16T20:48:51.370 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.037040967999999, 2024-09-16T20:48:51.370 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.010039259, 2024-09-16T20:48:51.370 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:51.370 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:51.370 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:51.370 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:51.370 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:51.370 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:51.371 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 953 2024-09-16T20:48:51.371 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.371 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:51.371 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.371 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:51.371 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241157+0000", 2024-09-16T20:48:51.371 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:51.371 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.371 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.371 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:51.371 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241157+0000", 2024-09-16T20:48:51.371 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:51.371 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.371 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.372 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:51.372 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241158+0000", 2024-09-16T20:48:51.372 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.01e-07 2024-09-16T20:48:51.372 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.372 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.372 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:51.372 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241160+0000", 2024-09-16T20:48:51.372 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2879999999999999e-06 2024-09-16T20:48:51.372 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.372 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.372 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:51.372 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241166+0000", 2024-09-16T20:48:51.372 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.421e-06 2024-09-16T20:48:51.372 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.373 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.373 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:51.373 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241171+0000", 2024-09-16T20:48:51.373 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9840000000000001e-06 2024-09-16T20:48:51.373 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.373 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.373 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:51.373 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250988+0000", 2024-09-16T20:48:51.373 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0098171449999999993 2024-09-16T20:48:51.373 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.373 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.374 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:51.374 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.251085+0000", 2024-09-16T20:48:51.374 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6877000000000001e-05 2024-09-16T20:48:51.374 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:51.374 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:51.374 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:51.374 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.251196+0000", 2024-09-16T20:48:51.374 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000111143 2024-09-16T20:48:51.374 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:51.374 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:51.374 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:51.374 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:51.374 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:51.374 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:51.376 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:48:51.376 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:51.375+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:51.477 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:51.476+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:51.496 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:51.496 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:48:51.496 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:48:51.496 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:51.505 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:48:51.578 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:51.577+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:51.612 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:51.612 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:48:51.613 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:48:51.613 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:48:51.613 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:51.621 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:48:51.623 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T20:48:51.679 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:51.677+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:51.773 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T20:48:51.780 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:51.780 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:48:51.780 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:48:51.780 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:48:51.780 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.780 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2174 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 931840~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:51.780 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335679+0000", 2024-09-16T20:48:51.780 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 29.442396386999999, 2024-09-16T20:48:51.780 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047619511000000003, 2024-09-16T20:48:51.781 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:51.781 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:51.781 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:51.781 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:51.781 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:51.781 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:51.781 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2174 2024-09-16T20:48:51.781 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.781 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:51.781 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.781 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:51.781 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335679+0000", 2024-09-16T20:48:51.782 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:51.782 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.782 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.782 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:51.782 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335679+0000", 2024-09-16T20:48:51.782 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:51.782 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.782 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.782 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:51.782 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335679+0000", 2024-09-16T20:48:51.782 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7800000000000002e-07 2024-09-16T20:48:51.782 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.782 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.783 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:51.783 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335682+0000", 2024-09-16T20:48:51.783 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9490000000000001e-06 2024-09-16T20:48:51.783 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.783 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.783 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:51.783 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335689+0000", 2024-09-16T20:48:51.783 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8220000000000001e-06 2024-09-16T20:48:51.784 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.784 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.784 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:51.784 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335695+0000", 2024-09-16T20:48:51.784 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0569999999999998e-06 2024-09-16T20:48:51.784 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.784 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.784 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:51.784 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383103+0000", 2024-09-16T20:48:51.784 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047408219000000001 2024-09-16T20:48:51.784 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.785 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.785 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:51.785 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383189+0000", 2024-09-16T20:48:51.785 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6007000000000005e-05 2024-09-16T20:48:51.785 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.785 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.785 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:51.785 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383298+0000", 2024-09-16T20:48:51.785 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000109079 2024-09-16T20:48:51.785 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:51.785 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:51.785 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:51.785 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.786 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.786 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2176 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 940032~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:51.786 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335702+0000", 2024-09-16T20:48:51.786 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 29.442373158999999, 2024-09-16T20:48:51.786 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047804450999999998, 2024-09-16T20:48:51.786 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:51.786 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:51.786 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:51.786 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:51.786 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:51.787 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:51.787 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2176 2024-09-16T20:48:51.787 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.787 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:51.787 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.787 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:51.787 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335702+0000", 2024-09-16T20:48:51.787 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:51.788 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.788 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.788 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:51.788 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335702+0000", 2024-09-16T20:48:51.788 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:51.788 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.788 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.788 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:51.788 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335703+0000", 2024-09-16T20:48:51.788 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.34e-07 2024-09-16T20:48:51.789 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.789 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.789 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:51.789 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335704+0000", 2024-09-16T20:48:51.789 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.917e-06 2024-09-16T20:48:51.789 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.789 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.789 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:51.789 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335711+0000", 2024-09-16T20:48:51.789 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4400000000000002e-06 2024-09-16T20:48:51.789 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.790 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.790 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:51.790 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335717+0000", 2024-09-16T20:48:51.790 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6679999999999998e-06 2024-09-16T20:48:51.790 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.790 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.790 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:51.790 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383335+0000", 2024-09-16T20:48:51.790 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047618754999999999 2024-09-16T20:48:51.790 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.791 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.791 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:51.791 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383405+0000", 2024-09-16T20:48:51.791 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9286e-05 2024-09-16T20:48:51.791 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.791 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.791 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:51.791 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383507+0000", 2024-09-16T20:48:51.791 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101951 2024-09-16T20:48:51.791 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:51.791 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:51.792 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:51.792 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.792 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.792 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2179 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 948224~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:51.792 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335723+0000", 2024-09-16T20:48:51.792 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 29.442352065000001, 2024-09-16T20:48:51.792 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048007181000000003, 2024-09-16T20:48:51.792 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:51.792 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:51.792 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:51.793 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:51.793 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:51.793 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:51.793 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2179 2024-09-16T20:48:51.793 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.793 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:51.793 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.793 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:51.793 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335723+0000", 2024-09-16T20:48:51.793 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:51.793 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.794 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.794 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:51.794 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335723+0000", 2024-09-16T20:48:51.794 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:51.794 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.794 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.794 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:51.795 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335724+0000", 2024-09-16T20:48:51.795 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.58e-07 2024-09-16T20:48:51.795 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.795 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.795 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:51.795 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335726+0000", 2024-09-16T20:48:51.795 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9120000000000001e-06 2024-09-16T20:48:51.795 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.795 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.795 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:51.796 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335732+0000", 2024-09-16T20:48:51.796 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6080000000000001e-06 2024-09-16T20:48:51.796 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.796 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.796 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:51.796 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335737+0000", 2024-09-16T20:48:51.796 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.22e-06 2024-09-16T20:48:51.796 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.796 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.796 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:51.796 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383556+0000", 2024-09-16T20:48:51.796 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047819002999999999 2024-09-16T20:48:51.797 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.797 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.797 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:51.797 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383635+0000", 2024-09-16T20:48:51.797 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9060999999999994e-05 2024-09-16T20:48:51.797 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.797 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.797 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:51.797 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383730+0000", 2024-09-16T20:48:51.797 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4919000000000004e-05 2024-09-16T20:48:51.797 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:51.798 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:51.798 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:51.798 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.798 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.798 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2180 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 956416~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:51.798 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335744+0000", 2024-09-16T20:48:51.798 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 29.442331617000001, 2024-09-16T20:48:51.798 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048255147999999998, 2024-09-16T20:48:51.798 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:51.798 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:51.798 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:51.798 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:51.799 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:51.799 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:51.799 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2180 2024-09-16T20:48:51.799 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.799 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:51.799 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.799 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:51.799 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335744+0000", 2024-09-16T20:48:51.799 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:51.799 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.799 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.799 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:51.799 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335744+0000", 2024-09-16T20:48:51.799 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:51.800 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.800 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.800 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:51.800 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335744+0000", 2024-09-16T20:48:51.800 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9000000000000002e-07 2024-09-16T20:48:51.800 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.800 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.800 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:51.800 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335746+0000", 2024-09-16T20:48:51.800 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7570000000000001e-06 2024-09-16T20:48:51.800 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.800 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.800 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:51.801 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335752+0000", 2024-09-16T20:48:51.801 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0079999999999999e-06 2024-09-16T20:48:51.801 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.801 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.801 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:51.801 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335757+0000", 2024-09-16T20:48:51.801 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1649999999999997e-06 2024-09-16T20:48:51.801 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.801 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.801 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:51.801 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383803+0000", 2024-09-16T20:48:51.801 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048046091999999999 2024-09-16T20:48:51.801 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.802 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.802 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:51.802 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383889+0000", 2024-09-16T20:48:51.802 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6048e-05 2024-09-16T20:48:51.802 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.802 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.802 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:51.802 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383999+0000", 2024-09-16T20:48:51.802 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000109688 2024-09-16T20:48:51.802 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:51.802 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:51.802 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:51.802 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.802 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.803 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2182 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 964608~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:51.803 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335773+0000", 2024-09-16T20:48:51.803 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 29.44230181, 2024-09-16T20:48:51.803 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048467116999999997, 2024-09-16T20:48:51.803 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:51.803 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:51.803 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:51.803 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:51.803 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:51.803 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:51.803 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2182 2024-09-16T20:48:51.803 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.803 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:51.803 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.804 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:51.804 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335773+0000", 2024-09-16T20:48:51.804 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:51.804 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.804 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.804 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:51.804 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335773+0000", 2024-09-16T20:48:51.804 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:51.804 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.804 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.804 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:51.804 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335774+0000", 2024-09-16T20:48:51.804 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3799999999999997e-07 2024-09-16T20:48:51.805 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.805 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.805 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:51.805 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335776+0000", 2024-09-16T20:48:51.805 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.227e-06 2024-09-16T20:48:51.805 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.805 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.805 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:51.805 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335790+0000", 2024-09-16T20:48:51.805 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4119e-05 2024-09-16T20:48:51.805 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.805 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.805 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:51.805 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335796+0000", 2024-09-16T20:48:51.806 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.694e-06 2024-09-16T20:48:51.806 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.806 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.806 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:51.806 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384058+0000", 2024-09-16T20:48:51.806 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048261983000000001 2024-09-16T20:48:51.806 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.806 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.806 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:51.806 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384148+0000", 2024-09-16T20:48:51.806 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9501999999999998e-05 2024-09-16T20:48:51.806 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.806 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.806 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:51.807 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384241+0000", 2024-09-16T20:48:51.807 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.3053999999999994e-05 2024-09-16T20:48:51.807 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:51.807 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:51.807 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:51.807 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.807 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.807 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2185 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 972800~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:51.807 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335804+0000", 2024-09-16T20:48:51.807 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 29.442271160000001, 2024-09-16T20:48:51.807 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048570818000000002, 2024-09-16T20:48:51.807 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:51.807 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:51.808 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:51.808 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:51.808 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:51.808 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:51.808 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2185 2024-09-16T20:48:51.808 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.809 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:51.809 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.809 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:51.809 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335804+0000", 2024-09-16T20:48:51.809 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:51.809 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.809 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.809 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:51.809 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335804+0000", 2024-09-16T20:48:51.809 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:51.809 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.809 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.809 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:51.810 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335805+0000", 2024-09-16T20:48:51.810 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8299999999999997e-07 2024-09-16T20:48:51.810 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.810 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.810 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:51.810 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335813+0000", 2024-09-16T20:48:51.810 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6309999999999992e-06 2024-09-16T20:48:51.810 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.810 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.810 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:51.810 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335821+0000", 2024-09-16T20:48:51.810 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0169999999999996e-06 2024-09-16T20:48:51.810 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.810 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.811 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:51.811 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335827+0000", 2024-09-16T20:48:51.811 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0100000000000001e-06 2024-09-16T20:48:51.811 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.811 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.811 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:51.811 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384273+0000", 2024-09-16T20:48:51.811 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048445333 2024-09-16T20:48:51.811 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.811 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.811 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:51.811 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384315+0000", 2024-09-16T20:48:51.811 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2472999999999998e-05 2024-09-16T20:48:51.812 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.812 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.812 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:51.812 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384375+0000", 2024-09-16T20:48:51.812 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9871000000000003e-05 2024-09-16T20:48:51.812 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:51.812 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:51.812 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:51.812 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.812 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.812 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2187 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 980992~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:51.812 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335835+0000", 2024-09-16T20:48:51.812 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 29.442240083000002, 2024-09-16T20:48:51.813 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048684927000000003, 2024-09-16T20:48:51.813 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:51.813 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:51.813 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:51.813 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:51.813 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:51.813 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:51.813 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2187 2024-09-16T20:48:51.813 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.813 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:51.813 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.814 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:51.814 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335835+0000", 2024-09-16T20:48:51.814 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:51.814 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.814 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.814 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:51.814 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335835+0000", 2024-09-16T20:48:51.815 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:51.815 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.815 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.815 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:51.815 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335836+0000", 2024-09-16T20:48:51.815 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3599999999999999e-07 2024-09-16T20:48:51.815 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.815 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.815 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:51.815 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335838+0000", 2024-09-16T20:48:51.815 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.996e-06 2024-09-16T20:48:51.815 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.815 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.815 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:51.816 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335844+0000", 2024-09-16T20:48:51.816 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7889999999999997e-06 2024-09-16T20:48:51.816 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.816 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.816 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:51.816 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335849+0000", 2024-09-16T20:48:51.816 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7400000000000004e-06 2024-09-16T20:48:51.816 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.816 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.816 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:51.816 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384411+0000", 2024-09-16T20:48:51.816 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048562069999999999 2024-09-16T20:48:51.816 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.816 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.817 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:51.817 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384460+0000", 2024-09-16T20:48:51.817 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9125999999999997e-05 2024-09-16T20:48:51.817 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.817 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.817 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:51.817 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384520+0000", 2024-09-16T20:48:51.817 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9769999999999999e-05 2024-09-16T20:48:51.817 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:51.817 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:51.817 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:51.817 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.817 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.817 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2189 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 989184~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:51.818 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335856+0000", 2024-09-16T20:48:51.818 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 29.442218837999999, 2024-09-16T20:48:51.818 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048856525999999997, 2024-09-16T20:48:51.818 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:51.818 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:51.818 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:51.818 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:51.818 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:51.818 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:51.818 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2189 2024-09-16T20:48:51.818 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.818 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:51.818 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.818 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:51.819 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335856+0000", 2024-09-16T20:48:51.819 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:51.819 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.819 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.819 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:51.819 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335856+0000", 2024-09-16T20:48:51.819 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:51.819 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.819 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.819 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:51.819 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335857+0000", 2024-09-16T20:48:51.819 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6800000000000001e-07 2024-09-16T20:48:51.819 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.819 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.820 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:51.820 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335859+0000", 2024-09-16T20:48:51.820 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.013e-06 2024-09-16T20:48:51.820 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.820 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.820 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:51.820 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335866+0000", 2024-09-16T20:48:51.820 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8750000000000002e-06 2024-09-16T20:48:51.820 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.820 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.820 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:51.820 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335871+0000", 2024-09-16T20:48:51.820 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1719999999999998e-06 2024-09-16T20:48:51.820 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.821 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.821 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:51.821 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384550+0000", 2024-09-16T20:48:51.821 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048679476999999999 2024-09-16T20:48:51.821 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.821 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.821 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:51.821 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384611+0000", 2024-09-16T20:48:51.821 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0897000000000003e-05 2024-09-16T20:48:51.821 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.821 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.821 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:51.821 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384713+0000", 2024-09-16T20:48:51.821 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101724 2024-09-16T20:48:51.822 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:51.822 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:51.822 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:51.822 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.822 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.822 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2192 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 997376~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:51.822 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335878+0000", 2024-09-16T20:48:51.822 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 29.442197152999999, 2024-09-16T20:48:51.822 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049028769999999999, 2024-09-16T20:48:51.822 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:51.822 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:51.822 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:51.822 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:51.822 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:51.823 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:51.823 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2192 2024-09-16T20:48:51.823 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.823 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:51.823 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.823 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:51.823 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335878+0000", 2024-09-16T20:48:51.823 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:51.823 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.823 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.823 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:51.823 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335878+0000", 2024-09-16T20:48:51.823 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:51.823 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.824 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.824 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:51.824 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335879+0000", 2024-09-16T20:48:51.824 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6899999999999998e-07 2024-09-16T20:48:51.824 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.824 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.824 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:51.824 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335881+0000", 2024-09-16T20:48:51.824 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.996e-06 2024-09-16T20:48:51.824 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.824 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.824 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:51.824 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335887+0000", 2024-09-16T20:48:51.824 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6370000000000002e-06 2024-09-16T20:48:51.825 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.825 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.825 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:51.825 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335892+0000", 2024-09-16T20:48:51.825 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2120000000000002e-06 2024-09-16T20:48:51.825 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.825 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.825 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:51.825 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384747+0000", 2024-09-16T20:48:51.825 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048854923000000001 2024-09-16T20:48:51.825 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.825 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.825 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:51.825 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384830+0000", 2024-09-16T20:48:51.825 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2359000000000002e-05 2024-09-16T20:48:51.826 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.826 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.826 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:51.826 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384907+0000", 2024-09-16T20:48:51.826 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7174000000000003e-05 2024-09-16T20:48:51.826 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:51.826 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:51.826 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:51.826 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.826 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.826 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2194 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1005568~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:51.826 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335899+0000", 2024-09-16T20:48:51.826 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 29.442176232000001, 2024-09-16T20:48:51.826 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049225393999999999, 2024-09-16T20:48:51.827 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:51.827 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:51.827 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:51.827 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:51.827 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:51.827 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:51.827 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2194 2024-09-16T20:48:51.827 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.827 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:51.827 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.827 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:51.827 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335899+0000", 2024-09-16T20:48:51.827 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:51.827 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.827 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.828 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:51.828 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335899+0000", 2024-09-16T20:48:51.828 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:51.828 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.828 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.828 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:51.828 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335899+0000", 2024-09-16T20:48:51.828 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8200000000000001e-07 2024-09-16T20:48:51.828 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.828 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.828 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:51.828 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335901+0000", 2024-09-16T20:48:51.828 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9e-06 2024-09-16T20:48:51.829 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.829 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.829 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:51.829 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335908+0000", 2024-09-16T20:48:51.829 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3559999999999998e-06 2024-09-16T20:48:51.829 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.829 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.829 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:51.829 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335914+0000", 2024-09-16T20:48:51.829 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4729999999999997e-06 2024-09-16T20:48:51.829 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.829 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.829 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:51.829 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384979+0000", 2024-09-16T20:48:51.830 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049064545000000001 2024-09-16T20:48:51.830 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.830 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.830 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:51.830 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385060+0000", 2024-09-16T20:48:51.830 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0804999999999996e-05 2024-09-16T20:48:51.830 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.830 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.830 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:51.830 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385124+0000", 2024-09-16T20:48:51.830 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4932999999999996e-05 2024-09-16T20:48:51.830 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:51.830 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:51.830 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:51.831 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.831 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.831 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2197 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1013760~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:51.831 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335921+0000", 2024-09-16T20:48:51.831 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 29.442153951000002, 2024-09-16T20:48:51.831 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049342184999999997, 2024-09-16T20:48:51.831 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:51.831 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:51.831 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:51.831 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:51.831 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:51.831 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:51.831 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2197 2024-09-16T20:48:51.831 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.832 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:51.832 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.832 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:51.832 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335921+0000", 2024-09-16T20:48:51.832 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:51.832 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.832 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.832 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:51.832 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335921+0000", 2024-09-16T20:48:51.832 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:51.832 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.832 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.832 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:51.832 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335922+0000", 2024-09-16T20:48:51.833 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.32e-07 2024-09-16T20:48:51.833 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.833 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.833 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:51.833 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335924+0000", 2024-09-16T20:48:51.833 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.995e-06 2024-09-16T20:48:51.833 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.833 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.833 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:51.833 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335930+0000", 2024-09-16T20:48:51.833 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5139999999999998e-06 2024-09-16T20:48:51.833 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.833 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.833 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:51.833 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335936+0000", 2024-09-16T20:48:51.834 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6160000000000001e-06 2024-09-16T20:48:51.834 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.834 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.834 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:51.834 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385142+0000", 2024-09-16T20:48:51.834 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049206393000000001 2024-09-16T20:48:51.834 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.834 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.834 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:51.834 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385184+0000", 2024-09-16T20:48:51.834 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1928999999999998e-05 2024-09-16T20:48:51.834 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.834 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.834 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:51.835 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385264+0000", 2024-09-16T20:48:51.835 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9306000000000003e-05 2024-09-16T20:48:51.835 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:51.835 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:51.835 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:51.835 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.835 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.835 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2198 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1021952~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:51.835 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335942+0000", 2024-09-16T20:48:51.835 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 29.442132891, 2024-09-16T20:48:51.835 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049562483999999997, 2024-09-16T20:48:51.835 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:51.835 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:51.836 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:51.836 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:51.836 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:51.836 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:51.836 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2198 2024-09-16T20:48:51.836 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.836 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:51.836 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.836 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:51.836 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335942+0000", 2024-09-16T20:48:51.836 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:51.836 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.836 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.836 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:51.837 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335942+0000", 2024-09-16T20:48:51.837 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:51.837 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.837 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.837 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:51.837 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335943+0000", 2024-09-16T20:48:51.837 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7099999999999997e-07 2024-09-16T20:48:51.837 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.837 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.837 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:51.837 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335945+0000", 2024-09-16T20:48:51.837 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3499999999999999e-06 2024-09-16T20:48:51.837 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.837 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.838 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:51.838 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335952+0000", 2024-09-16T20:48:51.838 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4309999999999999e-06 2024-09-16T20:48:51.838 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.838 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.838 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:51.838 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335957+0000", 2024-09-16T20:48:51.838 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0869999999999997e-06 2024-09-16T20:48:51.838 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.838 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.838 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:51.838 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385312+0000", 2024-09-16T20:48:51.838 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049355121000000002 2024-09-16T20:48:51.838 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.838 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.839 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:51.839 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385405+0000", 2024-09-16T20:48:51.839 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.3344000000000006e-05 2024-09-16T20:48:51.839 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.839 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.839 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:51.839 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385505+0000", 2024-09-16T20:48:51.839 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.9779999999999997e-05 2024-09-16T20:48:51.839 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:51.839 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:51.839 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:51.839 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.839 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.839 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2201 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1030144~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:51.840 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335976+0000", 2024-09-16T20:48:51.840 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 29.442099449000001, 2024-09-16T20:48:51.840 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049740077000000001, 2024-09-16T20:48:51.840 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:51.840 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:51.840 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:51.840 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:51.840 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:51.840 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:51.840 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2201 2024-09-16T20:48:51.840 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.840 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:51.840 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.840 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:51.841 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335976+0000", 2024-09-16T20:48:51.841 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:51.841 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.841 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.841 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:51.841 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335976+0000", 2024-09-16T20:48:51.841 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:51.841 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.841 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.841 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:51.841 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335976+0000", 2024-09-16T20:48:51.841 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8599999999999998e-07 2024-09-16T20:48:51.841 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.842 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.842 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:51.842 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335978+0000", 2024-09-16T20:48:51.842 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1500000000000002e-06 2024-09-16T20:48:51.842 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.842 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.842 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:51.842 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335986+0000", 2024-09-16T20:48:51.842 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3989999999999998e-06 2024-09-16T20:48:51.842 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.842 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.842 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:51.842 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335992+0000", 2024-09-16T20:48:51.842 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.682e-06 2024-09-16T20:48:51.842 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.843 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.843 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:51.843 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385537+0000", 2024-09-16T20:48:51.843 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049545828 2024-09-16T20:48:51.843 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.843 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.843 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:51.843 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385619+0000", 2024-09-16T20:48:51.843 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1720999999999999e-05 2024-09-16T20:48:51.843 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.843 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.843 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:51.843 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385716+0000", 2024-09-16T20:48:51.843 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.6811000000000005e-05 2024-09-16T20:48:51.844 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:51.844 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:51.844 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:51.844 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.844 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.844 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2203 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1038336~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:51.844 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.336000+0000", 2024-09-16T20:48:51.844 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 29.442075663000001, 2024-09-16T20:48:51.844 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049949888999999997, 2024-09-16T20:48:51.844 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:51.844 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:51.844 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:51.844 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:51.844 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:51.845 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:51.845 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2203 2024-09-16T20:48:51.845 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.845 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:51.845 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.845 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:51.845 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336000+0000", 2024-09-16T20:48:51.845 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:51.845 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.845 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.845 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:51.845 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336000+0000", 2024-09-16T20:48:51.845 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:51.845 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.846 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.846 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:51.846 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336000+0000", 2024-09-16T20:48:51.846 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.58e-07 2024-09-16T20:48:51.846 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.846 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.846 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:51.846 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336002+0000", 2024-09-16T20:48:51.846 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0040000000000002e-06 2024-09-16T20:48:51.846 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.846 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.846 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:51.846 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336009+0000", 2024-09-16T20:48:51.846 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9890000000000003e-06 2024-09-16T20:48:51.847 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.847 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.847 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:51.847 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336014+0000", 2024-09-16T20:48:51.847 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4e-06 2024-09-16T20:48:51.847 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.847 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.847 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:51.847 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385759+0000", 2024-09-16T20:48:51.847 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049744151 2024-09-16T20:48:51.847 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.847 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.847 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:51.847 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385847+0000", 2024-09-16T20:48:51.847 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8189000000000006e-05 2024-09-16T20:48:51.848 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.848 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.848 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:51.848 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385950+0000", 2024-09-16T20:48:51.848 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102698 2024-09-16T20:48:51.848 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:51.848 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:51.848 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:51.848 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.848 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.848 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2206 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1046528~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:51.848 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:48:51.848 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 29.442053502, 2024-09-16T20:48:51.848 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050147714000000003, 2024-09-16T20:48:51.849 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:51.849 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:51.849 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:51.849 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:51.849 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:51.849 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:51.849 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2206 2024-09-16T20:48:51.849 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.849 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:51.849 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.849 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:51.849 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:48:51.849 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:51.849 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.850 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.850 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:51.850 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:48:51.850 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:51.850 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.850 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.850 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:51.850 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:48:51.850 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8599999999999999e-07 2024-09-16T20:48:51.850 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.850 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.850 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:51.850 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336024+0000", 2024-09-16T20:48:51.850 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.934e-06 2024-09-16T20:48:51.851 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.851 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.851 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:51.851 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336037+0000", 2024-09-16T20:48:51.851 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2496e-05 2024-09-16T20:48:51.851 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.851 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.851 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:51.851 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336042+0000", 2024-09-16T20:48:51.851 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4809999999999997e-06 2024-09-16T20:48:51.851 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.851 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.851 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:51.851 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385980+0000", 2024-09-16T20:48:51.851 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049938376999999999 2024-09-16T20:48:51.852 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.852 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.852 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:51.852 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.386069+0000", 2024-09-16T20:48:51.852 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8368999999999994e-05 2024-09-16T20:48:51.852 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.852 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.852 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:51.852 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.386170+0000", 2024-09-16T20:48:51.852 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000100671 2024-09-16T20:48:51.852 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:51.852 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:51.852 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:51.852 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.853 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.853 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5439 3.a 3:5789f5c3:::rbd_data.110ed9ca540.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 1181696~60416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:51.853 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:48:51.853 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 28.913364922, 2024-09-16T20:48:51.853 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10102045599999999, 2024-09-16T20:48:51.853 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:51.853 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:51.853 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:51.853 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:51.853 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:51.853 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:51.853 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5439 2024-09-16T20:48:51.853 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.853 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:51.854 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.854 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:51.854 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:48:51.854 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:51.854 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.854 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.854 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:51.854 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:48:51.854 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:51.854 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.854 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.854 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:51.854 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864711+0000", 2024-09-16T20:48:51.854 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7899999999999997e-07 2024-09-16T20:48:51.854 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.855 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.855 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:51.855 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864760+0000", 2024-09-16T20:48:51.855 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8803e-05 2024-09-16T20:48:51.855 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.855 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.855 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:51.855 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864769+0000", 2024-09-16T20:48:51.855 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0999999999999993e-06 2024-09-16T20:48:51.855 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.855 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.855 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:51.855 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864774+0000", 2024-09-16T20:48:51.855 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8919999999999999e-06 2024-09-16T20:48:51.856 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.856 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.856 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:51.856 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864815+0000", 2024-09-16T20:48:51.856 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1307000000000003e-05 2024-09-16T20:48:51.856 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.856 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.856 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:51.856 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864893+0000", 2024-09-16T20:48:51.856 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7581000000000001e-05 2024-09-16T20:48:51.856 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.856 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.856 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 1", 2024-09-16T20:48:51.857 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864996+0000", 2024-09-16T20:48:51.857 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102725 2024-09-16T20:48:51.857 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.857 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.857 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:51.857 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965442+0000", 2024-09-16T20:48:51.857 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100446012 2024-09-16T20:48:51.857 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.857 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.857 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:51.857 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965618+0000", 2024-09-16T20:48:51.857 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00017653100000000001 2024-09-16T20:48:51.857 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.857 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.857 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:51.858 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965682+0000", 2024-09-16T20:48:51.858 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3905999999999994e-05 2024-09-16T20:48:51.858 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.858 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.858 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:51.858 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965731+0000", 2024-09-16T20:48:51.858 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8819999999999997e-05 2024-09-16T20:48:51.858 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:51.858 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:51.858 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:51.858 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.858 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.858 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4600.0:5439 3.a e37/23 ondisk, result = 0)", 2024-09-16T20:48:51.858 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:48:51.859 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 28.912368774000001, 2024-09-16T20:48:51.859 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099742634999999996, 2024-09-16T20:48:51.859 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:51.859 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:51.859 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:51.859 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:51.859 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.859 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:51.859 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:48:51.859 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:51.859 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.859 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.859 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:51.860 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:48:51.860 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:51.860 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.860 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.860 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:51.860 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:48:51.860 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0900000000000001e-07 2024-09-16T20:48:51.860 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.860 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.860 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:51.860 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865711+0000", 2024-09-16T20:48:51.860 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3739999999999998e-06 2024-09-16T20:48:51.860 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.861 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.861 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:51.861 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965343+0000", 2024-09-16T20:48:51.861 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099632526999999999 2024-09-16T20:48:51.861 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.861 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.861 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:51.861 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965357+0000", 2024-09-16T20:48:51.861 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3713999999999999e-05 2024-09-16T20:48:51.861 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.861 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.861 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:51.861 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965426+0000", 2024-09-16T20:48:51.861 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8745999999999995e-05 2024-09-16T20:48:51.862 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.862 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.862 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:51.862 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965434+0000", 2024-09-16T20:48:51.862 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6910000000000003e-06 2024-09-16T20:48:51.862 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.862 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.862 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:51.862 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965449+0000", 2024-09-16T20:48:51.862 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4874e-05 2024-09-16T20:48:51.862 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:51.862 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:51.862 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:51.862 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.863 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.863 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4882.0:2508 3.12 3:4992771f:::rbd_object_map.110e7c8851c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T20:48:51.863 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:48:51.863 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 5.2965652260000002, 2024-09-16T20:48:51.863 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.125007011, 2024-09-16T20:48:51.863 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:51.863 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:51.863 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:51.863 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:51.863 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4882", 2024-09-16T20:48:51.863 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2436923223", 2024-09-16T20:48:51.863 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2508 2024-09-16T20:48:51.863 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.863 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:51.864 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.864 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:51.864 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:48:51.864 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:51.864 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.864 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.864 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:51.864 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:48:51.864 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:51.864 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.864 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.864 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:51.864 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481511+0000", 2024-09-16T20:48:51.864 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0699999999999996e-07 2024-09-16T20:48:51.865 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.865 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.865 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:51.865 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481515+0000", 2024-09-16T20:48:51.865 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0609999999999997e-06 2024-09-16T20:48:51.865 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.865 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.865 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:51.865 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481523+0000", 2024-09-16T20:48:51.865 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1310000000000006e-06 2024-09-16T20:48:51.865 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.865 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.865 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:51.865 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481528+0000", 2024-09-16T20:48:51.866 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4900000000000002e-06 2024-09-16T20:48:51.866 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.866 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.866 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:51.866 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481565+0000", 2024-09-16T20:48:51.866 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7066000000000001e-05 2024-09-16T20:48:51.866 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.866 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.866 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:51.866 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481628+0000", 2024-09-16T20:48:51.866 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3118000000000001e-05 2024-09-16T20:48:51.866 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.866 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.866 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 2", 2024-09-16T20:48:51.866 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481822+0000", 2024-09-16T20:48:51.867 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019446200000000001 2024-09-16T20:48:51.867 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.867 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.867 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:51.867 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605988+0000", 2024-09-16T20:48:51.867 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.124165888 2024-09-16T20:48:51.867 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.867 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.867 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:51.867 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606427+0000", 2024-09-16T20:48:51.867 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00043854899999999999 2024-09-16T20:48:51.867 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.867 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.867 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:51.868 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606476+0000", 2024-09-16T20:48:51.868 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8971999999999998e-05 2024-09-16T20:48:51.868 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.868 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.868 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:51.868 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606517+0000", 2024-09-16T20:48:51.868 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1467000000000001e-05 2024-09-16T20:48:51.868 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:51.868 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:51.868 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:51.868 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.868 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.868 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4882.0:2508 3.12 e57/55 ondisk, result = 0)", 2024-09-16T20:48:51.868 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:48:51.868 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 5.2955767260000002, 2024-09-16T20:48:51.869 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12349711400000001, 2024-09-16T20:48:51.869 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:51.869 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:51.869 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:51.869 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:51.869 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.869 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:51.869 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:48:51.869 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:51.869 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.869 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.869 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:51.869 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:48:51.869 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:51.870 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.870 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.870 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:51.870 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:48:51.870 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.06e-07 2024-09-16T20:48:51.870 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.870 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.870 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:51.870 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482502+0000", 2024-09-16T20:48:51.870 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.01e-06 2024-09-16T20:48:51.870 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.870 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.870 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:51.870 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605772+0000", 2024-09-16T20:48:51.871 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.123269687 2024-09-16T20:48:51.871 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.871 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.871 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:51.871 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605791+0000", 2024-09-16T20:48:51.871 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8912000000000001e-05 2024-09-16T20:48:51.871 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.871 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.871 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:51.871 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605974+0000", 2024-09-16T20:48:51.871 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000182836 2024-09-16T20:48:51.871 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.871 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.871 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:51.872 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605982+0000", 2024-09-16T20:48:51.872 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1699999999999997e-06 2024-09-16T20:48:51.872 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.872 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.872 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:51.872 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605996+0000", 2024-09-16T20:48:51.872 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3893000000000001e-05 2024-09-16T20:48:51.872 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:51.872 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:51.872 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:51.872 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.872 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.872 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4882.0:2512 3.f 3:f4d895b7:::rbd_data.110e7c8851c.0000000000000000:head [read 16384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T20:48:51.872 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:48:51.873 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 5.2951396959999997, 2024-09-16T20:48:51.873 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12318483, 2024-09-16T20:48:51.873 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:51.873 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:51.873 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:51.873 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:51.873 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4882", 2024-09-16T20:48:51.873 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2436923223", 2024-09-16T20:48:51.873 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2512 2024-09-16T20:48:51.873 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.873 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:51.873 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.873 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:51.873 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:48:51.874 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:51.874 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.874 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.874 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:51.874 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:48:51.874 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:51.874 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.874 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.874 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:51.874 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:48:51.874 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3499999999999995e-07 2024-09-16T20:48:51.874 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.874 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.874 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:51.874 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482943+0000", 2024-09-16T20:48:51.875 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3450000000000002e-06 2024-09-16T20:48:51.875 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.875 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.875 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:51.875 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605772+0000", 2024-09-16T20:48:51.875 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.122829796 2024-09-16T20:48:51.875 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.875 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.875 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:51.875 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605792+0000", 2024-09-16T20:48:51.875 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9916999999999999e-05 2024-09-16T20:48:51.875 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.875 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.875 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:51.876 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605917+0000", 2024-09-16T20:48:51.876 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00012500799999999999 2024-09-16T20:48:51.876 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.876 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.876 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:51.876 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606004+0000", 2024-09-16T20:48:51.876 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6242999999999994e-05 2024-09-16T20:48:51.876 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:51.876 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:51.876 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:51.876 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606120+0000", 2024-09-16T20:48:51.876 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000116786 2024-09-16T20:48:51.876 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:51.876 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:51.877 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:51.877 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:51.877 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:51.877 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:51.877 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:51.778+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:51.878 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:48:51.878 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:48:51.879 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:48:51.879 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:48:51.879 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:48:51.880 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:48:51.880 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:48:51.880 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:48:51.881 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:48:51.881 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:48:51.881 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:48:51.881 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:48:51.882 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:48:51.882 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:51.880+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:51.982 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:51.981+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:51.993 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:51.993 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:51.998 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:48:52.083 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:52.081+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:52.145 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:52.145 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:52.153 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:48:52.197 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:52.196+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:52.285 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:52.283+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:52.310 INFO:teuthology.orchestra.run.smithi139.stdout:2245 read 0xc5800 thru 0xcd032 (0x7833 bytes) 2024-09-16T20:48:52.334 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:52.334 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:52.344 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:48:52.350 INFO:teuthology.orchestra.run.smithi139.stdout:2246 write 0xc512a00 thru 0xc51a075 (0x7676 bytes) 2024-09-16T20:48:52.351 INFO:teuthology.orchestra.run.smithi139.stdout:2248 punch from 0x3faa400 to 0x3fb4c00, (0xa800 bytes) 2024-09-16T20:48:52.352 INFO:teuthology.orchestra.run.smithi139.stdout:2249 read 0x50a9a00 thru 0x50ad7d5 (0x3dd6 bytes) 2024-09-16T20:48:52.353 INFO:teuthology.orchestra.run.smithi139.stdout:2252 read 0x1b6f000 thru 0x1b79288 (0xa289 bytes) 2024-09-16T20:48:52.353 INFO:teuthology.orchestra.run.smithi139.stdout:2256 read 0xab7f400 thru 0xab8993d (0xa53e bytes) 2024-09-16T20:48:52.353 INFO:teuthology.orchestra.run.smithi139.stdout:2258 punch from 0x3f70400 to 0x3f76c00, (0x6800 bytes) 2024-09-16T20:48:52.353 INFO:teuthology.orchestra.run.smithi139.stdout:2259 clone 16 order 23 su 65536 sc 4 2024-09-16T20:48:52.385 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:52.384+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:52.486 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:52.485+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:52.498 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:52.498 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:52.507 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:48:52.587 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:52.586+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:52.665 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:52.666 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:52.675 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:48:52.719 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:52.718+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:52.728 INFO:teuthology.orchestra.run.smithi029.stderr:nodeep-scrub is unset 2024-09-16T20:48:52.789 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:52.788+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:52.796 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:52.796 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:52.806 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:48:52.891 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:52.890+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:52.913 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:52.913 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:52.922 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:48:52.992 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:52.990+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:53.029 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:53.029 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:53.037 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:48:53.120 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:53.119+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:53.183 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:53.184 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:53.192 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:48:53.194 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:53.194+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:53.295 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:53.295+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:53.339 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:53.340 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:53.348 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:48:53.397 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:53.396+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:53.455 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:53.455 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:53.463 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:48:53.508 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:53.507+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:53.571 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:53.571 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:53.579 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:48:53.598 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:53.596+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:53.698 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:53.697+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:53.727 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:53.727 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:53.740 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:48:53.799 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:53.799+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:53.887 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:53.893 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:53.893 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T20:48:53.896 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:48:53.938 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:53.937+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:54.001 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:54.000+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:54.003 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:54.003 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:54.012 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:48:54.029 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:48:54.029 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:48:54.047 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:48:54.048 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:48:54.064 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:48:54.064 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:48:54.082 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:48:54.082 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:48:54.099 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:48:54.099 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:48:54.102 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:54.101+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:54.116 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:48:54.116 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:48:54.122 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:54.122 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:48:54.122 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:48:54.122 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:54.131 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:48:54.216 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:54.215+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:54.279 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:54.279 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:48:54.279 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:48:54.279 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:48:54.279 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:54.287 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:48:54.328 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:54.327+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:54.405 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:54.404+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:54.436 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:54.436 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:48:54.436 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:48:54.436 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:48:54.436 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.436 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2174 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 931840~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:54.436 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335679+0000", 2024-09-16T20:48:54.436 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.098436026999998, 2024-09-16T20:48:54.436 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047619511000000003, 2024-09-16T20:48:54.436 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:54.437 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:54.437 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:54.437 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:54.437 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:54.437 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:54.437 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2174 2024-09-16T20:48:54.437 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.437 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:54.437 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.437 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:54.437 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335679+0000", 2024-09-16T20:48:54.437 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:54.437 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.438 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.438 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:54.438 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335679+0000", 2024-09-16T20:48:54.438 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:54.438 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.438 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.438 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:54.438 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335679+0000", 2024-09-16T20:48:54.438 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7800000000000002e-07 2024-09-16T20:48:54.438 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.438 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.438 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:54.438 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335682+0000", 2024-09-16T20:48:54.438 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9490000000000001e-06 2024-09-16T20:48:54.439 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.439 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.439 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:54.439 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335689+0000", 2024-09-16T20:48:54.439 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8220000000000001e-06 2024-09-16T20:48:54.439 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.439 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.439 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:54.439 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335695+0000", 2024-09-16T20:48:54.439 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0569999999999998e-06 2024-09-16T20:48:54.439 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.439 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.439 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:54.440 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383103+0000", 2024-09-16T20:48:54.440 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047408219000000001 2024-09-16T20:48:54.440 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.440 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.440 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:54.440 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383189+0000", 2024-09-16T20:48:54.440 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6007000000000005e-05 2024-09-16T20:48:54.440 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.440 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.440 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:54.440 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383298+0000", 2024-09-16T20:48:54.440 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000109079 2024-09-16T20:48:54.440 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:54.441 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:54.441 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:54.441 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.441 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.441 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2176 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 940032~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:54.441 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335702+0000", 2024-09-16T20:48:54.441 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.098412799000002, 2024-09-16T20:48:54.441 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047804450999999998, 2024-09-16T20:48:54.441 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:54.441 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:54.441 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:54.441 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:54.442 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:54.442 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:54.442 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2176 2024-09-16T20:48:54.442 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.442 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:54.442 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.442 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:54.443 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335702+0000", 2024-09-16T20:48:54.443 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:54.443 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.443 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.443 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:54.443 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335702+0000", 2024-09-16T20:48:54.443 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:54.443 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.443 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.443 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:54.443 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335703+0000", 2024-09-16T20:48:54.443 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.34e-07 2024-09-16T20:48:54.443 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.444 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.444 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:54.444 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335704+0000", 2024-09-16T20:48:54.444 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.917e-06 2024-09-16T20:48:54.444 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.444 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.444 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:54.444 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335711+0000", 2024-09-16T20:48:54.444 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4400000000000002e-06 2024-09-16T20:48:54.445 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.445 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.445 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:54.445 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335717+0000", 2024-09-16T20:48:54.445 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6679999999999998e-06 2024-09-16T20:48:54.445 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.445 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.445 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:54.445 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383335+0000", 2024-09-16T20:48:54.445 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047618754999999999 2024-09-16T20:48:54.445 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.445 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.445 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:54.446 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383405+0000", 2024-09-16T20:48:54.446 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9286e-05 2024-09-16T20:48:54.446 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.446 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.446 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:54.446 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383507+0000", 2024-09-16T20:48:54.446 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101951 2024-09-16T20:48:54.446 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:54.446 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:54.448 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:54.448 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.448 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.448 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2179 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 948224~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:54.448 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335723+0000", 2024-09-16T20:48:54.449 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.098391704999997, 2024-09-16T20:48:54.449 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048007181000000003, 2024-09-16T20:48:54.449 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:54.449 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:54.449 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:54.449 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:54.449 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:54.449 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:54.449 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2179 2024-09-16T20:48:54.449 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.450 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:54.450 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.450 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:54.450 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335723+0000", 2024-09-16T20:48:54.450 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:54.450 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.450 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.450 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:54.451 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335723+0000", 2024-09-16T20:48:54.451 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:54.451 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.451 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.451 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:54.451 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335724+0000", 2024-09-16T20:48:54.451 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.58e-07 2024-09-16T20:48:54.451 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.451 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.451 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:54.451 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335726+0000", 2024-09-16T20:48:54.451 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9120000000000001e-06 2024-09-16T20:48:54.452 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.452 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.452 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:54.452 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335732+0000", 2024-09-16T20:48:54.452 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6080000000000001e-06 2024-09-16T20:48:54.452 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.452 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.452 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:54.452 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335737+0000", 2024-09-16T20:48:54.452 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.22e-06 2024-09-16T20:48:54.453 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.453 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.453 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:54.453 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383556+0000", 2024-09-16T20:48:54.453 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047819002999999999 2024-09-16T20:48:54.453 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.453 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.453 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:54.453 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383635+0000", 2024-09-16T20:48:54.453 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9060999999999994e-05 2024-09-16T20:48:54.453 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.454 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.454 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:54.454 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383730+0000", 2024-09-16T20:48:54.454 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4919000000000004e-05 2024-09-16T20:48:54.454 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:54.454 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:54.454 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:54.455 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.455 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.455 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2180 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 956416~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:54.455 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335744+0000", 2024-09-16T20:48:54.455 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.098371256999997, 2024-09-16T20:48:54.455 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048255147999999998, 2024-09-16T20:48:54.455 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:54.455 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:54.455 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:54.455 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:54.456 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:54.456 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:54.456 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2180 2024-09-16T20:48:54.456 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.456 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:54.456 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.456 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:54.457 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335744+0000", 2024-09-16T20:48:54.457 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:54.457 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.457 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.457 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:54.457 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335744+0000", 2024-09-16T20:48:54.457 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:54.457 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.457 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.457 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:54.457 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335744+0000", 2024-09-16T20:48:54.458 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9000000000000002e-07 2024-09-16T20:48:54.458 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.458 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.458 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:54.458 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335746+0000", 2024-09-16T20:48:54.458 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7570000000000001e-06 2024-09-16T20:48:54.458 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.458 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.458 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:54.458 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335752+0000", 2024-09-16T20:48:54.458 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0079999999999999e-06 2024-09-16T20:48:54.458 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.458 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.459 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:54.459 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335757+0000", 2024-09-16T20:48:54.459 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1649999999999997e-06 2024-09-16T20:48:54.459 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.459 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.459 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:54.459 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383803+0000", 2024-09-16T20:48:54.459 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048046091999999999 2024-09-16T20:48:54.459 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.459 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.459 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:54.459 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383889+0000", 2024-09-16T20:48:54.459 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6048e-05 2024-09-16T20:48:54.459 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.459 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.460 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:54.460 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383999+0000", 2024-09-16T20:48:54.460 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000109688 2024-09-16T20:48:54.460 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:54.460 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:54.460 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:54.460 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.460 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.460 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2182 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 964608~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:54.460 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335773+0000", 2024-09-16T20:48:54.460 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.098341449999999, 2024-09-16T20:48:54.460 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048467116999999997, 2024-09-16T20:48:54.460 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:54.461 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:54.461 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:54.461 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:54.461 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:54.461 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:54.461 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2182 2024-09-16T20:48:54.461 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.461 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:54.461 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.461 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:54.461 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335773+0000", 2024-09-16T20:48:54.461 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:54.461 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.461 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.462 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:54.462 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335773+0000", 2024-09-16T20:48:54.462 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:54.462 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.462 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.462 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:54.462 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335774+0000", 2024-09-16T20:48:54.462 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3799999999999997e-07 2024-09-16T20:48:54.462 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.462 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.462 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:54.462 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335776+0000", 2024-09-16T20:48:54.462 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.227e-06 2024-09-16T20:48:54.462 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.463 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.463 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:54.463 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335790+0000", 2024-09-16T20:48:54.463 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4119e-05 2024-09-16T20:48:54.463 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.463 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.463 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:54.463 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335796+0000", 2024-09-16T20:48:54.463 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.694e-06 2024-09-16T20:48:54.463 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.463 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.463 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:54.463 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384058+0000", 2024-09-16T20:48:54.463 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048261983000000001 2024-09-16T20:48:54.464 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.464 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.464 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:54.464 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384148+0000", 2024-09-16T20:48:54.464 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9501999999999998e-05 2024-09-16T20:48:54.464 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.464 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.464 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:54.464 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384241+0000", 2024-09-16T20:48:54.464 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.3053999999999994e-05 2024-09-16T20:48:54.464 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:54.464 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:54.464 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:54.464 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.465 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.465 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2185 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 972800~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:54.465 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335804+0000", 2024-09-16T20:48:54.465 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.0983108, 2024-09-16T20:48:54.465 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048570818000000002, 2024-09-16T20:48:54.465 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:54.465 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:54.465 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:54.465 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:54.465 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:54.465 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:54.465 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2185 2024-09-16T20:48:54.465 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.465 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:54.466 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.466 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:54.466 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335804+0000", 2024-09-16T20:48:54.466 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:54.466 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.466 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.466 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:54.466 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335804+0000", 2024-09-16T20:48:54.466 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:54.466 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.466 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.466 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:54.466 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335805+0000", 2024-09-16T20:48:54.466 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8299999999999997e-07 2024-09-16T20:48:54.467 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.467 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.467 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:54.467 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335813+0000", 2024-09-16T20:48:54.467 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6309999999999992e-06 2024-09-16T20:48:54.467 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.467 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.467 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:54.467 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335821+0000", 2024-09-16T20:48:54.467 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0169999999999996e-06 2024-09-16T20:48:54.467 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.467 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.467 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:54.467 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335827+0000", 2024-09-16T20:48:54.467 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0100000000000001e-06 2024-09-16T20:48:54.468 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.468 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.468 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:54.468 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384273+0000", 2024-09-16T20:48:54.468 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048445333 2024-09-16T20:48:54.468 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.468 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.468 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:54.468 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384315+0000", 2024-09-16T20:48:54.468 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2472999999999998e-05 2024-09-16T20:48:54.468 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.468 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.468 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:54.469 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384375+0000", 2024-09-16T20:48:54.469 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9871000000000003e-05 2024-09-16T20:48:54.469 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:54.469 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:54.469 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:54.469 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.469 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.469 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2187 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 980992~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:54.469 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335835+0000", 2024-09-16T20:48:54.469 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.098279722999997, 2024-09-16T20:48:54.469 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048684927000000003, 2024-09-16T20:48:54.469 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:54.469 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:54.469 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:54.470 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:54.470 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:54.470 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:54.470 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2187 2024-09-16T20:48:54.470 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.470 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:54.470 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.470 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:54.470 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335835+0000", 2024-09-16T20:48:54.470 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:54.470 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.470 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.470 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:54.470 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335835+0000", 2024-09-16T20:48:54.471 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:54.471 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.471 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.471 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:54.471 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335836+0000", 2024-09-16T20:48:54.471 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3599999999999999e-07 2024-09-16T20:48:54.471 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.471 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.471 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:54.471 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335838+0000", 2024-09-16T20:48:54.471 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.996e-06 2024-09-16T20:48:54.471 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.471 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.471 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:54.472 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335844+0000", 2024-09-16T20:48:54.472 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7889999999999997e-06 2024-09-16T20:48:54.472 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.472 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.472 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:54.472 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335849+0000", 2024-09-16T20:48:54.472 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7400000000000004e-06 2024-09-16T20:48:54.472 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.472 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.472 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:54.472 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384411+0000", 2024-09-16T20:48:54.472 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048562069999999999 2024-09-16T20:48:54.472 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.472 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.473 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:54.473 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384460+0000", 2024-09-16T20:48:54.473 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9125999999999997e-05 2024-09-16T20:48:54.473 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.473 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.473 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:54.473 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384520+0000", 2024-09-16T20:48:54.473 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9769999999999999e-05 2024-09-16T20:48:54.473 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:54.473 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:54.473 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:54.473 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.473 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.473 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2189 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 989184~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:54.474 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335856+0000", 2024-09-16T20:48:54.474 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.098258477999998, 2024-09-16T20:48:54.474 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048856525999999997, 2024-09-16T20:48:54.474 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:54.474 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:54.474 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:54.474 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:54.474 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:54.474 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:54.474 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2189 2024-09-16T20:48:54.474 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.474 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:54.474 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.474 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:54.474 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335856+0000", 2024-09-16T20:48:54.475 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:54.475 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.475 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.475 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:54.475 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335856+0000", 2024-09-16T20:48:54.475 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:54.475 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.475 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.475 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:54.475 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335857+0000", 2024-09-16T20:48:54.475 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6800000000000001e-07 2024-09-16T20:48:54.475 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.475 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.476 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:54.476 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335859+0000", 2024-09-16T20:48:54.476 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.013e-06 2024-09-16T20:48:54.476 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.476 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.476 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:54.476 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335866+0000", 2024-09-16T20:48:54.476 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8750000000000002e-06 2024-09-16T20:48:54.476 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.476 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.476 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:54.476 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335871+0000", 2024-09-16T20:48:54.476 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1719999999999998e-06 2024-09-16T20:48:54.476 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.477 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.477 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:54.477 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384550+0000", 2024-09-16T20:48:54.477 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048679476999999999 2024-09-16T20:48:54.477 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.477 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.477 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:54.477 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384611+0000", 2024-09-16T20:48:54.477 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0897000000000003e-05 2024-09-16T20:48:54.477 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.477 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.477 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:54.477 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384713+0000", 2024-09-16T20:48:54.477 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101724 2024-09-16T20:48:54.478 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:54.478 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:54.478 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:54.478 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.478 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.478 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2192 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 997376~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:54.478 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335878+0000", 2024-09-16T20:48:54.478 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.098236792999998, 2024-09-16T20:48:54.478 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049028769999999999, 2024-09-16T20:48:54.478 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:54.478 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:54.478 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:54.478 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:54.479 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:54.479 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:54.479 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2192 2024-09-16T20:48:54.479 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.479 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:54.479 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.479 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:54.479 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335878+0000", 2024-09-16T20:48:54.479 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:54.479 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.479 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.479 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:54.479 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335878+0000", 2024-09-16T20:48:54.480 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:54.480 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.480 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.480 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:54.480 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335879+0000", 2024-09-16T20:48:54.480 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6899999999999998e-07 2024-09-16T20:48:54.480 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.480 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.480 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:54.480 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335881+0000", 2024-09-16T20:48:54.480 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.996e-06 2024-09-16T20:48:54.480 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.480 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.480 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:54.480 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335887+0000", 2024-09-16T20:48:54.481 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6370000000000002e-06 2024-09-16T20:48:54.481 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.481 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.481 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:54.481 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335892+0000", 2024-09-16T20:48:54.481 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2120000000000002e-06 2024-09-16T20:48:54.481 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.481 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.481 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:54.481 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384747+0000", 2024-09-16T20:48:54.481 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048854923000000001 2024-09-16T20:48:54.481 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.481 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.481 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:54.482 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384830+0000", 2024-09-16T20:48:54.482 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2359000000000002e-05 2024-09-16T20:48:54.482 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.482 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.482 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:54.482 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384907+0000", 2024-09-16T20:48:54.482 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7174000000000003e-05 2024-09-16T20:48:54.482 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:54.482 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:54.482 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:54.482 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.482 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.482 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2194 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1005568~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:54.482 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335899+0000", 2024-09-16T20:48:54.483 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.098215871999997, 2024-09-16T20:48:54.483 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049225393999999999, 2024-09-16T20:48:54.483 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:54.483 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:54.483 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:54.483 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:54.483 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:54.483 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:54.483 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2194 2024-09-16T20:48:54.483 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.483 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:54.483 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.483 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:54.483 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335899+0000", 2024-09-16T20:48:54.483 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:54.484 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.484 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.484 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:54.484 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335899+0000", 2024-09-16T20:48:54.484 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:54.484 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.484 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.484 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:54.484 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335899+0000", 2024-09-16T20:48:54.484 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8200000000000001e-07 2024-09-16T20:48:54.484 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.484 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.484 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:54.485 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335901+0000", 2024-09-16T20:48:54.485 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9e-06 2024-09-16T20:48:54.485 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.485 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.485 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:54.485 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335908+0000", 2024-09-16T20:48:54.485 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3559999999999998e-06 2024-09-16T20:48:54.485 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.485 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.485 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:54.485 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335914+0000", 2024-09-16T20:48:54.485 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4729999999999997e-06 2024-09-16T20:48:54.485 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.485 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.486 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:54.486 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384979+0000", 2024-09-16T20:48:54.486 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049064545000000001 2024-09-16T20:48:54.486 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.486 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.486 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:54.486 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385060+0000", 2024-09-16T20:48:54.486 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0804999999999996e-05 2024-09-16T20:48:54.486 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.486 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.486 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:54.486 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385124+0000", 2024-09-16T20:48:54.486 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4932999999999996e-05 2024-09-16T20:48:54.486 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:54.486 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:54.487 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:54.487 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.487 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.487 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2197 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1013760~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:54.487 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335921+0000", 2024-09-16T20:48:54.487 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.098193590999998, 2024-09-16T20:48:54.487 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049342184999999997, 2024-09-16T20:48:54.487 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:54.487 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:54.487 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:54.487 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:54.488 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:54.488 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:54.488 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2197 2024-09-16T20:48:54.488 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.488 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:54.488 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.488 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:54.488 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335921+0000", 2024-09-16T20:48:54.488 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:54.488 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.488 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.488 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:54.488 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335921+0000", 2024-09-16T20:48:54.488 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:54.489 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.489 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.489 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:54.489 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335922+0000", 2024-09-16T20:48:54.489 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.32e-07 2024-09-16T20:48:54.489 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.489 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.489 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:54.489 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335924+0000", 2024-09-16T20:48:54.489 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.995e-06 2024-09-16T20:48:54.489 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.489 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.489 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:54.489 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335930+0000", 2024-09-16T20:48:54.490 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5139999999999998e-06 2024-09-16T20:48:54.490 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.490 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.490 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:54.490 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335936+0000", 2024-09-16T20:48:54.490 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6160000000000001e-06 2024-09-16T20:48:54.490 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.490 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.490 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:54.490 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385142+0000", 2024-09-16T20:48:54.490 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049206393000000001 2024-09-16T20:48:54.490 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.490 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.490 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:54.490 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385184+0000", 2024-09-16T20:48:54.491 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1928999999999998e-05 2024-09-16T20:48:54.491 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.491 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.491 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:54.491 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385264+0000", 2024-09-16T20:48:54.491 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9306000000000003e-05 2024-09-16T20:48:54.491 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:54.491 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:54.491 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:54.491 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.491 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.491 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2198 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1021952~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:54.491 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335942+0000", 2024-09-16T20:48:54.491 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.098172531000003, 2024-09-16T20:48:54.492 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049562483999999997, 2024-09-16T20:48:54.492 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:54.492 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:54.492 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:54.492 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:54.492 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:54.492 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:54.492 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2198 2024-09-16T20:48:54.492 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.492 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:54.492 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.492 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:54.492 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335942+0000", 2024-09-16T20:48:54.492 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:54.493 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.493 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.493 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:54.493 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335942+0000", 2024-09-16T20:48:54.493 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:54.493 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.493 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.493 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:54.493 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335943+0000", 2024-09-16T20:48:54.493 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7099999999999997e-07 2024-09-16T20:48:54.493 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.493 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.493 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:54.494 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335945+0000", 2024-09-16T20:48:54.494 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3499999999999999e-06 2024-09-16T20:48:54.494 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.494 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.494 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:54.494 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335952+0000", 2024-09-16T20:48:54.494 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4309999999999999e-06 2024-09-16T20:48:54.494 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.494 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.494 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:54.495 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335957+0000", 2024-09-16T20:48:54.495 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0869999999999997e-06 2024-09-16T20:48:54.495 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.495 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.495 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:54.495 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385312+0000", 2024-09-16T20:48:54.495 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049355121000000002 2024-09-16T20:48:54.495 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.495 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.495 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:54.495 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385405+0000", 2024-09-16T20:48:54.495 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.3344000000000006e-05 2024-09-16T20:48:54.495 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.495 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.496 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:54.496 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385505+0000", 2024-09-16T20:48:54.496 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.9779999999999997e-05 2024-09-16T20:48:54.496 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:54.496 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:54.496 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:54.496 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.496 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.496 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2201 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1030144~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:54.496 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335976+0000", 2024-09-16T20:48:54.496 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.098139089, 2024-09-16T20:48:54.496 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049740077000000001, 2024-09-16T20:48:54.496 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:54.496 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:54.497 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:54.497 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:54.497 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:54.497 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:54.497 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2201 2024-09-16T20:48:54.497 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.497 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:54.497 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.497 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:54.497 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335976+0000", 2024-09-16T20:48:54.497 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:54.497 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.497 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.497 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:54.497 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335976+0000", 2024-09-16T20:48:54.498 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:54.498 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.498 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.498 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:54.498 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335976+0000", 2024-09-16T20:48:54.498 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8599999999999998e-07 2024-09-16T20:48:54.498 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.498 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.498 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:54.498 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335978+0000", 2024-09-16T20:48:54.498 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1500000000000002e-06 2024-09-16T20:48:54.498 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.498 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.499 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:54.499 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335986+0000", 2024-09-16T20:48:54.499 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3989999999999998e-06 2024-09-16T20:48:54.499 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.499 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.499 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:54.499 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335992+0000", 2024-09-16T20:48:54.499 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.682e-06 2024-09-16T20:48:54.499 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.499 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.499 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:54.499 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385537+0000", 2024-09-16T20:48:54.499 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049545828 2024-09-16T20:48:54.499 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.500 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.500 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:54.500 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385619+0000", 2024-09-16T20:48:54.500 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1720999999999999e-05 2024-09-16T20:48:54.500 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.500 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.500 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:54.500 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385716+0000", 2024-09-16T20:48:54.500 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.6811000000000005e-05 2024-09-16T20:48:54.500 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:54.500 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:54.500 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:54.500 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.501 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.501 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2203 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1038336~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:54.501 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.336000+0000", 2024-09-16T20:48:54.501 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.098115303, 2024-09-16T20:48:54.501 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049949888999999997, 2024-09-16T20:48:54.501 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:54.501 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:54.501 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:54.501 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:54.501 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:54.501 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:54.501 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2203 2024-09-16T20:48:54.501 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.502 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:54.502 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.502 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:54.502 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336000+0000", 2024-09-16T20:48:54.502 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:54.502 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.502 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.502 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:54.502 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336000+0000", 2024-09-16T20:48:54.502 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:54.502 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.502 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.502 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:54.502 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336000+0000", 2024-09-16T20:48:54.503 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.58e-07 2024-09-16T20:48:54.503 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.503 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.503 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:54.503 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336002+0000", 2024-09-16T20:48:54.503 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0040000000000002e-06 2024-09-16T20:48:54.503 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.503 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.503 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:54.503 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336009+0000", 2024-09-16T20:48:54.503 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9890000000000003e-06 2024-09-16T20:48:54.503 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.503 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.504 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:54.504 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336014+0000", 2024-09-16T20:48:54.504 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4e-06 2024-09-16T20:48:54.504 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.504 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.504 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:54.504 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385759+0000", 2024-09-16T20:48:54.504 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049744151 2024-09-16T20:48:54.504 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.504 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.504 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:54.504 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385847+0000", 2024-09-16T20:48:54.504 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8189000000000006e-05 2024-09-16T20:48:54.505 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.505 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.505 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:54.505 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385950+0000", 2024-09-16T20:48:54.505 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102698 2024-09-16T20:48:54.505 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:54.505 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:54.505 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:54.505 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.505 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.505 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2206 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1046528~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:48:54.505 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:48:54.505 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.098093142000003, 2024-09-16T20:48:54.505 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050147714000000003, 2024-09-16T20:48:54.506 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:54.506 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:54.506 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:54.506 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:54.506 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:54.506 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:54.506 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2206 2024-09-16T20:48:54.506 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.506 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:54.506 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.506 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:54.506 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:48:54.506 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:54.506 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.506 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.507 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:54.507 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:48:54.507 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:54.507 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.507 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.507 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:54.507 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:48:54.507 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8599999999999999e-07 2024-09-16T20:48:54.507 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.507 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.507 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:54.507 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336024+0000", 2024-09-16T20:48:54.507 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.934e-06 2024-09-16T20:48:54.508 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.508 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.508 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:54.508 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336037+0000", 2024-09-16T20:48:54.508 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2496e-05 2024-09-16T20:48:54.508 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.508 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.508 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:54.508 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336042+0000", 2024-09-16T20:48:54.508 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4809999999999997e-06 2024-09-16T20:48:54.508 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.508 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.508 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:54.509 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385980+0000", 2024-09-16T20:48:54.509 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049938376999999999 2024-09-16T20:48:54.509 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.509 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.509 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:54.509 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.386069+0000", 2024-09-16T20:48:54.509 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8368999999999994e-05 2024-09-16T20:48:54.509 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.509 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.509 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:54.509 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.386170+0000", 2024-09-16T20:48:54.509 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000100671 2024-09-16T20:48:54.509 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:54.509 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:54.510 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:54.510 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.510 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.510 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5439 3.a 3:5789f5c3:::rbd_data.110ed9ca540.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 1181696~60416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:54.510 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:48:54.510 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 31.569404561999999, 2024-09-16T20:48:54.510 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10102045599999999, 2024-09-16T20:48:54.510 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:54.510 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:54.510 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:54.510 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:54.510 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:54.510 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:54.510 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5439 2024-09-16T20:48:54.511 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.511 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:54.511 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.511 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:54.511 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:48:54.511 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:54.511 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.511 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.511 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:54.511 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:48:54.511 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:54.511 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.511 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.511 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:54.511 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864711+0000", 2024-09-16T20:48:54.512 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7899999999999997e-07 2024-09-16T20:48:54.512 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.512 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.512 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:54.512 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864760+0000", 2024-09-16T20:48:54.512 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8803e-05 2024-09-16T20:48:54.512 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.512 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.512 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:54.512 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864769+0000", 2024-09-16T20:48:54.512 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0999999999999993e-06 2024-09-16T20:48:54.512 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.512 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.513 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:54.513 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864774+0000", 2024-09-16T20:48:54.513 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8919999999999999e-06 2024-09-16T20:48:54.513 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.513 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.513 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:54.513 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864815+0000", 2024-09-16T20:48:54.513 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1307000000000003e-05 2024-09-16T20:48:54.513 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.513 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.513 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:54.513 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864893+0000", 2024-09-16T20:48:54.513 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7581000000000001e-05 2024-09-16T20:48:54.513 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.514 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.514 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 1", 2024-09-16T20:48:54.514 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864996+0000", 2024-09-16T20:48:54.514 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102725 2024-09-16T20:48:54.514 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.514 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.514 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:54.514 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965442+0000", 2024-09-16T20:48:54.514 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100446012 2024-09-16T20:48:54.514 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.514 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.514 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:54.514 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965618+0000", 2024-09-16T20:48:54.514 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00017653100000000001 2024-09-16T20:48:54.514 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.515 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.515 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:54.515 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965682+0000", 2024-09-16T20:48:54.515 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3905999999999994e-05 2024-09-16T20:48:54.515 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.515 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.515 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:54.515 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965731+0000", 2024-09-16T20:48:54.515 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8819999999999997e-05 2024-09-16T20:48:54.515 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:54.515 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:54.515 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:54.515 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.515 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.516 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4600.0:5439 3.a e37/23 ondisk, result = 0)", 2024-09-16T20:48:54.516 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:48:54.516 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 31.568408414, 2024-09-16T20:48:54.516 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099742634999999996, 2024-09-16T20:48:54.516 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:54.516 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:54.516 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:54.516 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:54.516 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.516 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:54.516 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:48:54.516 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:54.516 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.516 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.517 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:54.517 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:48:54.517 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:54.517 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.517 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.517 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:54.517 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:48:54.517 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0900000000000001e-07 2024-09-16T20:48:54.517 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.517 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.517 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:54.517 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865711+0000", 2024-09-16T20:48:54.517 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3739999999999998e-06 2024-09-16T20:48:54.517 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.518 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.518 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:54.518 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965343+0000", 2024-09-16T20:48:54.518 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099632526999999999 2024-09-16T20:48:54.518 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.518 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.518 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:54.518 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965357+0000", 2024-09-16T20:48:54.518 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3713999999999999e-05 2024-09-16T20:48:54.518 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.518 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.518 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:54.518 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965426+0000", 2024-09-16T20:48:54.518 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8745999999999995e-05 2024-09-16T20:48:54.518 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.519 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.519 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:54.519 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965434+0000", 2024-09-16T20:48:54.519 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6910000000000003e-06 2024-09-16T20:48:54.519 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.519 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.519 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:54.519 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965449+0000", 2024-09-16T20:48:54.519 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4874e-05 2024-09-16T20:48:54.519 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:54.519 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:54.519 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:54.519 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.519 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.520 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4882.0:2508 3.12 3:4992771f:::rbd_object_map.110e7c8851c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T20:48:54.520 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:48:54.520 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 7.9526048659999997, 2024-09-16T20:48:54.520 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.125007011, 2024-09-16T20:48:54.520 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:54.520 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:54.520 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:54.520 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:54.520 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4882", 2024-09-16T20:48:54.520 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2436923223", 2024-09-16T20:48:54.520 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2508 2024-09-16T20:48:54.520 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.520 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:54.520 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.521 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:54.521 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:48:54.521 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:54.521 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.521 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.521 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:54.521 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:48:54.521 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:54.521 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.521 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.521 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:54.521 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481511+0000", 2024-09-16T20:48:54.521 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0699999999999996e-07 2024-09-16T20:48:54.521 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.522 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.522 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:54.522 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481515+0000", 2024-09-16T20:48:54.522 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0609999999999997e-06 2024-09-16T20:48:54.522 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.522 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.522 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:54.522 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481523+0000", 2024-09-16T20:48:54.522 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1310000000000006e-06 2024-09-16T20:48:54.522 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.522 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.522 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:54.522 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481528+0000", 2024-09-16T20:48:54.522 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4900000000000002e-06 2024-09-16T20:48:54.522 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.523 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.523 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:54.523 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481565+0000", 2024-09-16T20:48:54.523 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7066000000000001e-05 2024-09-16T20:48:54.523 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.523 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.523 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:54.523 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481628+0000", 2024-09-16T20:48:54.523 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3118000000000001e-05 2024-09-16T20:48:54.523 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.523 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.523 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 2", 2024-09-16T20:48:54.523 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481822+0000", 2024-09-16T20:48:54.523 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019446200000000001 2024-09-16T20:48:54.524 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.524 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.524 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:54.524 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605988+0000", 2024-09-16T20:48:54.524 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.124165888 2024-09-16T20:48:54.524 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.524 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.524 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:54.524 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606427+0000", 2024-09-16T20:48:54.524 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00043854899999999999 2024-09-16T20:48:54.524 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.524 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.524 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:54.524 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606476+0000", 2024-09-16T20:48:54.525 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8971999999999998e-05 2024-09-16T20:48:54.525 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.525 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.525 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:54.525 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606517+0000", 2024-09-16T20:48:54.525 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1467000000000001e-05 2024-09-16T20:48:54.525 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:54.525 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:54.525 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:54.525 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.525 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.525 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4882.0:2508 3.12 e57/55 ondisk, result = 0)", 2024-09-16T20:48:54.525 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:48:54.525 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 7.9516163659999997, 2024-09-16T20:48:54.526 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12349711400000001, 2024-09-16T20:48:54.526 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:54.526 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:54.526 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:54.526 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:54.526 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.526 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:54.526 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:48:54.526 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:54.526 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.526 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.526 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:54.526 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:48:54.526 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:54.527 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.527 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.527 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:54.527 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:48:54.527 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.06e-07 2024-09-16T20:48:54.527 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.527 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.527 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:54.527 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482502+0000", 2024-09-16T20:48:54.527 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.01e-06 2024-09-16T20:48:54.527 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.527 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.527 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:54.527 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605772+0000", 2024-09-16T20:48:54.528 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.123269687 2024-09-16T20:48:54.528 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.528 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.528 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:54.528 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605791+0000", 2024-09-16T20:48:54.528 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8912000000000001e-05 2024-09-16T20:48:54.528 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.528 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.528 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:54.528 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605974+0000", 2024-09-16T20:48:54.528 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000182836 2024-09-16T20:48:54.528 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.528 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.528 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:54.529 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605982+0000", 2024-09-16T20:48:54.529 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1699999999999997e-06 2024-09-16T20:48:54.529 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.529 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.529 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:54.529 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605996+0000", 2024-09-16T20:48:54.529 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3893000000000001e-05 2024-09-16T20:48:54.529 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:54.529 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:54.529 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:54.529 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.529 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.529 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4882.0:2512 3.f 3:f4d895b7:::rbd_data.110e7c8851c.0000000000000000:head [read 16384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T20:48:54.529 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:48:54.530 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 7.951179336, 2024-09-16T20:48:54.530 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12318483, 2024-09-16T20:48:54.530 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:54.530 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:54.530 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:54.530 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:54.530 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4882", 2024-09-16T20:48:54.530 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2436923223", 2024-09-16T20:48:54.530 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2512 2024-09-16T20:48:54.530 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.530 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:54.530 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.530 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:54.530 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:48:54.530 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:54.531 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.531 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.531 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:54.531 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:48:54.531 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:54.531 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.531 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.531 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:54.531 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:48:54.531 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3499999999999995e-07 2024-09-16T20:48:54.531 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.531 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.532 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:54.532 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482943+0000", 2024-09-16T20:48:54.532 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3450000000000002e-06 2024-09-16T20:48:54.532 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.532 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.532 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:54.532 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605772+0000", 2024-09-16T20:48:54.532 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.122829796 2024-09-16T20:48:54.532 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.532 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.532 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:54.532 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605792+0000", 2024-09-16T20:48:54.532 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9916999999999999e-05 2024-09-16T20:48:54.532 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.532 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.533 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:54.533 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605917+0000", 2024-09-16T20:48:54.533 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00012500799999999999 2024-09-16T20:48:54.533 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.533 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.533 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:54.533 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606004+0000", 2024-09-16T20:48:54.533 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6242999999999994e-05 2024-09-16T20:48:54.533 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.533 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.533 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:54.533 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606120+0000", 2024-09-16T20:48:54.533 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000116786 2024-09-16T20:48:54.533 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:54.534 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:54.534 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:54.534 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:54.534 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:54.534 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:54.535 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:48:54.536 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:54.534+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:54.636 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:54.635+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:54.643 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:54.643 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:48:54.643 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:48:54.643 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:54.653 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:48:54.737 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:54.736+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:54.744 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T20:48:54.773 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 3] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-16T20:48:54.774 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T20:48:54.774 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-16T20:48:54.774 INFO:tasks.thrashosds.thrasher:Setting osd 3 primary_affinity to 1.000000 2024-09-16T20:48:54.774 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-16T20:48:54.806 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:54.806 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:48:54.806 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:48:54.806 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:48:54.806 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:54.816 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:48:54.838 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:54.837+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:54.939 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:54.938+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:54.969 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:54.970 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:48:54.970 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:48:54.970 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:48:54.970 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.970 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:48:54.970 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:48:54.970 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 61.713220856, 2024-09-16T20:48:54.970 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67502346999999996, 2024-09-16T20:48:54.970 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:54.970 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:54.970 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:54.971 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:54.971 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:48:54.971 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:48:54.971 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 1 2024-09-16T20:48:54.971 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.971 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:54.971 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.971 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:54.971 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:48:54.971 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:54.971 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.971 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.972 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:54.972 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:48:54.972 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:54.972 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.972 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.972 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:54.972 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254128+0000", 2024-09-16T20:48:54.972 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.463e-06 2024-09-16T20:48:54.972 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.972 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.973 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:54.973 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254135+0000", 2024-09-16T20:48:54.973 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1579999999999999e-06 2024-09-16T20:48:54.973 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.973 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.973 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:54.973 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254146+0000", 2024-09-16T20:48:54.973 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1341e-05 2024-09-16T20:48:54.973 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.973 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.974 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:54.974 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254155+0000", 2024-09-16T20:48:54.974 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.1549999999999996e-06 2024-09-16T20:48:54.974 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.974 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.974 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:54.974 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254221+0000", 2024-09-16T20:48:54.974 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5470999999999997e-05 2024-09-16T20:48:54.974 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.974 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.974 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:48:54.975 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254231+0000", 2024-09-16T20:48:54.975 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0592e-05 2024-09-16T20:48:54.975 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.975 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.975 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:54.975 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926711+0000", 2024-09-16T20:48:54.975 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67247946599999997 2024-09-16T20:48:54.975 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.975 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.975 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:54.975 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926910+0000", 2024-09-16T20:48:54.975 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019934 2024-09-16T20:48:54.976 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.976 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.976 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:48:54.976 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.927159+0000", 2024-09-16T20:48:54.976 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00024876599999999998 2024-09-16T20:48:54.976 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.976 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.976 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:54.976 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.928797+0000", 2024-09-16T20:48:54.976 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001638308 2024-09-16T20:48:54.977 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.977 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.977 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:54.977 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929026+0000", 2024-09-16T20:48:54.977 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00022876099999999999 2024-09-16T20:48:54.977 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.977 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.977 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:54.977 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929102+0000", 2024-09-16T20:48:54.977 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6184000000000001e-05 2024-09-16T20:48:54.977 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.977 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.978 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:54.978 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929150+0000", 2024-09-16T20:48:54.978 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7465e-05 2024-09-16T20:48:54.978 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:54.978 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:54.978 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:54.978 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.978 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.978 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4317 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4659200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:54.978 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745704+0000", 2024-09-16T20:48:54.979 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 35.221642936999999, 2024-09-16T20:48:54.979 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047411778000000002, 2024-09-16T20:48:54.979 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:54.979 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:54.979 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:54.979 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:54.979 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:48:54.979 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:54.979 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4317 2024-09-16T20:48:54.979 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.979 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:54.980 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.980 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:54.980 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745704+0000", 2024-09-16T20:48:54.980 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:54.980 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.980 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.980 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:54.980 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745704+0000", 2024-09-16T20:48:54.980 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:54.980 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.981 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.981 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:54.981 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745704+0000", 2024-09-16T20:48:54.981 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0900000000000002e-07 2024-09-16T20:48:54.981 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.981 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.981 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:54.981 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745707+0000", 2024-09-16T20:48:54.981 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1660000000000001e-06 2024-09-16T20:48:54.981 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.981 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.981 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:54.982 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745722+0000", 2024-09-16T20:48:54.982 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5084e-05 2024-09-16T20:48:54.982 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.982 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.982 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:54.982 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745735+0000", 2024-09-16T20:48:54.982 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3074999999999999e-05 2024-09-16T20:48:54.982 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.982 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.982 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:54.982 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792989+0000", 2024-09-16T20:48:54.983 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047254013999999997 2024-09-16T20:48:54.983 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.983 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.983 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:54.983 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793034+0000", 2024-09-16T20:48:54.983 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4858999999999999e-05 2024-09-16T20:48:54.983 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.983 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.983 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:54.983 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793116+0000", 2024-09-16T20:48:54.984 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2170999999999996e-05 2024-09-16T20:48:54.984 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:54.984 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:54.984 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:54.984 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.984 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.984 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4318 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4667392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:54.984 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745812+0000", 2024-09-16T20:48:54.984 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 35.221535258000003, 2024-09-16T20:48:54.984 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047532229000000002, 2024-09-16T20:48:54.984 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:54.985 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:54.985 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:54.985 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:54.985 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:48:54.985 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:54.985 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4318 2024-09-16T20:48:54.985 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.985 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:54.985 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.985 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:54.986 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745812+0000", 2024-09-16T20:48:54.986 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:54.986 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.986 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.986 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:54.986 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745812+0000", 2024-09-16T20:48:54.986 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:54.986 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.986 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.987 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:54.987 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745812+0000", 2024-09-16T20:48:54.987 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5899999999999996e-07 2024-09-16T20:48:54.987 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.987 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.987 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:54.987 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745815+0000", 2024-09-16T20:48:54.987 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9110000000000002e-06 2024-09-16T20:48:54.987 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.987 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.988 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:54.988 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745823+0000", 2024-09-16T20:48:54.988 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1030000000000002e-06 2024-09-16T20:48:54.988 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.988 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.988 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:54.988 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745829+0000", 2024-09-16T20:48:54.988 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8449999999999999e-06 2024-09-16T20:48:54.988 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.988 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.988 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:54.989 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793159+0000", 2024-09-16T20:48:54.989 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047330009999999999 2024-09-16T20:48:54.989 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.989 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.989 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:54.989 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793251+0000", 2024-09-16T20:48:54.989 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.1991999999999996e-05 2024-09-16T20:48:54.989 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.989 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.989 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:54.989 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793344+0000", 2024-09-16T20:48:54.990 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2808999999999999e-05 2024-09-16T20:48:54.990 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:54.990 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:54.990 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:54.990 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.990 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.990 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4319 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4675584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:54.990 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745838+0000", 2024-09-16T20:48:54.991 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 35.221509044999998, 2024-09-16T20:48:54.991 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047640204999999998, 2024-09-16T20:48:54.991 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:54.991 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:54.991 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:54.991 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:54.991 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:48:54.991 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:54.991 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4319 2024-09-16T20:48:54.991 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.991 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:54.992 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.992 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:54.992 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745838+0000", 2024-09-16T20:48:54.992 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:54.992 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.992 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.992 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:54.992 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745838+0000", 2024-09-16T20:48:54.992 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:54.992 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.992 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.992 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:54.993 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745838+0000", 2024-09-16T20:48:54.993 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6899999999999998e-07 2024-09-16T20:48:54.993 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.993 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.993 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:54.993 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745840+0000", 2024-09-16T20:48:54.993 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0660000000000002e-06 2024-09-16T20:48:54.994 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.994 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.994 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:54.994 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745847+0000", 2024-09-16T20:48:54.994 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8689999999999998e-06 2024-09-16T20:48:54.994 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.994 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.994 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:54.994 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745853+0000", 2024-09-16T20:48:54.994 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2399999999999998e-06 2024-09-16T20:48:54.994 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.995 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.995 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:54.995 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793381+0000", 2024-09-16T20:48:54.995 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047528005999999998 2024-09-16T20:48:54.995 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.995 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.995 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:54.995 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793423+0000", 2024-09-16T20:48:54.995 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2799000000000001e-05 2024-09-16T20:48:54.996 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.996 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.996 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:54.996 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793478+0000", 2024-09-16T20:48:54.996 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4756000000000002e-05 2024-09-16T20:48:54.997 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:54.997 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:54.997 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:54.997 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.997 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.997 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4320 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4683776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:54.997 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745859+0000", 2024-09-16T20:48:54.997 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 35.221487801000002, 2024-09-16T20:48:54.997 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047811399999999997, 2024-09-16T20:48:54.997 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:54.998 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:54.998 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:54.998 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:54.998 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:48:54.998 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:54.998 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4320 2024-09-16T20:48:54.998 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.998 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:54.998 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.998 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:54.998 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745859+0000", 2024-09-16T20:48:54.999 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:54.999 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.999 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.999 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:54.999 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745859+0000", 2024-09-16T20:48:54.999 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:54.999 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:54.999 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:54.999 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:54.999 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745860+0000", 2024-09-16T20:48:55.000 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9099999999999999e-07 2024-09-16T20:48:55.000 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.000 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.000 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:55.000 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745861+0000", 2024-09-16T20:48:55.000 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8589999999999999e-06 2024-09-16T20:48:55.000 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.000 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.000 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:55.000 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745868+0000", 2024-09-16T20:48:55.000 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2249999999999997e-06 2024-09-16T20:48:55.001 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.001 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.001 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:55.001 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745873+0000", 2024-09-16T20:48:55.001 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1290000000000004e-06 2024-09-16T20:48:55.001 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.001 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.001 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:55.001 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793522+0000", 2024-09-16T20:48:55.001 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047648847000000001 2024-09-16T20:48:55.001 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.001 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.002 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:55.002 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793603+0000", 2024-09-16T20:48:55.002 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1167000000000004e-05 2024-09-16T20:48:55.002 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.002 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.002 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:55.002 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793671+0000", 2024-09-16T20:48:55.002 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7781999999999996e-05 2024-09-16T20:48:55.002 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.002 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:55.002 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.003 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.003 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.003 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4321 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4691968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:55.003 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745879+0000", 2024-09-16T20:48:55.003 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 35.221467605000001, 2024-09-16T20:48:55.003 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047950375000000003, 2024-09-16T20:48:55.003 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:55.003 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:55.003 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:55.003 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:55.003 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:48:55.004 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:55.004 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4321 2024-09-16T20:48:55.004 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.004 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:55.004 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.004 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:55.004 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745879+0000", 2024-09-16T20:48:55.004 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.004 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.004 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.004 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:55.005 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745879+0000", 2024-09-16T20:48:55.005 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.005 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.005 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.005 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:55.005 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745880+0000", 2024-09-16T20:48:55.005 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.65e-07 2024-09-16T20:48:55.005 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.005 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.005 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:55.005 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745882+0000", 2024-09-16T20:48:55.006 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.922e-06 2024-09-16T20:48:55.006 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.006 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.006 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:55.006 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745888+0000", 2024-09-16T20:48:55.006 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0510000000000002e-06 2024-09-16T20:48:55.006 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.006 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.006 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:55.006 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745893+0000", 2024-09-16T20:48:55.007 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9579999999999998e-06 2024-09-16T20:48:55.007 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.007 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.007 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:55.007 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793704+0000", 2024-09-16T20:48:55.007 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047811179000000002 2024-09-16T20:48:55.007 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.007 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.007 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:55.007 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793759+0000", 2024-09-16T20:48:55.008 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5260999999999998e-05 2024-09-16T20:48:55.008 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.008 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.008 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:55.008 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793830+0000", 2024-09-16T20:48:55.008 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0638999999999999e-05 2024-09-16T20:48:55.008 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.008 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:55.008 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.008 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.008 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.009 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4322 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4700160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:55.009 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745899+0000", 2024-09-16T20:48:55.009 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 35.221448166999998, 2024-09-16T20:48:55.009 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048057280000000001, 2024-09-16T20:48:55.009 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:55.009 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:55.009 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:55.009 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:55.009 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:48:55.009 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:55.009 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4322 2024-09-16T20:48:55.009 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.010 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:55.010 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.010 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:55.010 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745899+0000", 2024-09-16T20:48:55.010 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.010 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.010 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.010 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:55.010 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745899+0000", 2024-09-16T20:48:55.010 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.010 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.011 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.011 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:55.011 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745899+0000", 2024-09-16T20:48:55.011 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9299999999999999e-07 2024-09-16T20:48:55.011 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.011 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.011 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:55.011 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745901+0000", 2024-09-16T20:48:55.011 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0030000000000001e-06 2024-09-16T20:48:55.011 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.012 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.012 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:55.012 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745908+0000", 2024-09-16T20:48:55.012 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3729999999999998e-06 2024-09-16T20:48:55.012 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.012 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.012 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:55.012 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745914+0000", 2024-09-16T20:48:55.013 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.122e-06 2024-09-16T20:48:55.013 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.013 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.013 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:55.013 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793862+0000", 2024-09-16T20:48:55.013 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047948324 2024-09-16T20:48:55.013 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.013 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.013 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:55.013 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793903+0000", 2024-09-16T20:48:55.013 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0465e-05 2024-09-16T20:48:55.013 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.014 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.014 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:55.014 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793956+0000", 2024-09-16T20:48:55.014 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3600000000000002e-05 2024-09-16T20:48:55.014 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.014 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:55.014 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.014 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.014 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.014 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4323 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4708352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:55.014 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745920+0000", 2024-09-16T20:48:55.015 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 35.221426526000002, 2024-09-16T20:48:55.015 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048268138000000002, 2024-09-16T20:48:55.015 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:55.015 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:55.015 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:55.015 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:55.015 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:48:55.015 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:55.015 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4323 2024-09-16T20:48:55.016 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.016 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:55.016 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.016 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:55.016 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745920+0000", 2024-09-16T20:48:55.016 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.016 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.016 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.016 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:55.016 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745920+0000", 2024-09-16T20:48:55.017 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.017 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.017 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.017 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:55.017 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745921+0000", 2024-09-16T20:48:55.017 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7e-07 2024-09-16T20:48:55.017 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.017 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.017 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:55.017 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745923+0000", 2024-09-16T20:48:55.018 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7880000000000001e-06 2024-09-16T20:48:55.018 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.018 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.018 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:55.018 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745929+0000", 2024-09-16T20:48:55.018 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0809999999999999e-06 2024-09-16T20:48:55.018 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.018 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.018 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:55.019 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745935+0000", 2024-09-16T20:48:55.019 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5370000000000003e-06 2024-09-16T20:48:55.019 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.019 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.019 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:55.019 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793987+0000", 2024-09-16T20:48:55.019 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048051736999999997 2024-09-16T20:48:55.019 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.019 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.019 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:55.019 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.794070+0000", 2024-09-16T20:48:55.020 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.3277999999999998e-05 2024-09-16T20:48:55.020 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.020 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.020 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:55.020 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.794189+0000", 2024-09-16T20:48:55.020 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000118347 2024-09-16T20:48:55.020 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.020 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:55.021 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.021 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.021 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.021 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4324 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4716544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:55.021 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745946+0000", 2024-09-16T20:48:55.021 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 35.221400955, 2024-09-16T20:48:55.021 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048468153999999999, 2024-09-16T20:48:55.021 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:55.021 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:55.022 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:55.022 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:55.022 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:48:55.022 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:55.022 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4324 2024-09-16T20:48:55.022 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.022 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:55.022 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.022 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:55.023 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745946+0000", 2024-09-16T20:48:55.023 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.023 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.023 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.023 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:55.023 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745946+0000", 2024-09-16T20:48:55.023 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.023 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.023 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.024 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:55.024 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745946+0000", 2024-09-16T20:48:55.024 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4499999999999997e-07 2024-09-16T20:48:55.024 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.024 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.024 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:55.024 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745949+0000", 2024-09-16T20:48:55.024 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2529999999999999e-06 2024-09-16T20:48:55.024 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.024 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.025 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:55.025 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745955+0000", 2024-09-16T20:48:55.025 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6470000000000001e-06 2024-09-16T20:48:55.025 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.025 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.025 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:55.025 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745960+0000", 2024-09-16T20:48:55.025 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7910000000000004e-06 2024-09-16T20:48:55.025 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.026 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.026 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:55.026 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.794220+0000", 2024-09-16T20:48:55.026 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048260069000000003 2024-09-16T20:48:55.026 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.026 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.026 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:55.026 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.794311+0000", 2024-09-16T20:48:55.027 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0593000000000005e-05 2024-09-16T20:48:55.027 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.027 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.027 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:55.027 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.794414+0000", 2024-09-16T20:48:55.027 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000103356 2024-09-16T20:48:55.027 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.027 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:55.027 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.027 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.028 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.028 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3571 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:48:55.028 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481059+0000", 2024-09-16T20:48:55.028 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 5.486287795, 2024-09-16T20:48:55.028 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049572995000000002, 2024-09-16T20:48:55.028 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:55.028 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:55.028 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:55.028 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:55.028 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:48:55.029 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:48:55.029 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3571 2024-09-16T20:48:55.029 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.029 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:55.029 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.029 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:55.029 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481059+0000", 2024-09-16T20:48:55.029 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.029 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.029 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.030 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:55.030 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481059+0000", 2024-09-16T20:48:55.030 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.030 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.030 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.030 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:55.030 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481060+0000", 2024-09-16T20:48:55.030 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2599999999999998e-07 2024-09-16T20:48:55.030 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.030 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.030 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:55.031 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481063+0000", 2024-09-16T20:48:55.031 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.999e-06 2024-09-16T20:48:55.031 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.031 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.031 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:55.031 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481067+0000", 2024-09-16T20:48:55.031 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3080000000000001e-06 2024-09-16T20:48:55.031 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.031 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.031 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:55.031 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481070+0000", 2024-09-16T20:48:55.032 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9919999999999999e-06 2024-09-16T20:48:55.032 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.032 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.032 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:55.032 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.522087+0000", 2024-09-16T20:48:55.032 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.041017023 2024-09-16T20:48:55.032 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.032 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.032 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:55.032 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.522191+0000", 2024-09-16T20:48:55.032 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010389499999999999 2024-09-16T20:48:55.033 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.033 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.033 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:48:55.033 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.522566+0000", 2024-09-16T20:48:55.033 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00037539399999999998 2024-09-16T20:48:55.033 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.033 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.033 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:55.033 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525494+0000", 2024-09-16T20:48:55.033 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0029273070000000001 2024-09-16T20:48:55.033 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.034 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.034 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:55.034 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530355+0000", 2024-09-16T20:48:55.034 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0048612509999999996 2024-09-16T20:48:55.034 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.034 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.034 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:55.034 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530407+0000", 2024-09-16T20:48:55.034 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1817000000000002e-05 2024-09-16T20:48:55.034 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.034 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.035 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:55.035 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530632+0000", 2024-09-16T20:48:55.035 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00022558300000000001 2024-09-16T20:48:55.035 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.035 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:55.035 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.035 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.035 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.035 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3572 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:48:55.035 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481220+0000", 2024-09-16T20:48:55.036 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 5.4861272269999999, 2024-09-16T20:48:55.036 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050452862000000001, 2024-09-16T20:48:55.036 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:55.036 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:55.036 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:55.036 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:55.036 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:48:55.036 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:48:55.036 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3572 2024-09-16T20:48:55.036 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.036 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:55.037 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.037 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:55.037 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481220+0000", 2024-09-16T20:48:55.037 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.037 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.037 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.037 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:55.037 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481220+0000", 2024-09-16T20:48:55.037 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.037 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.037 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.038 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:55.038 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481221+0000", 2024-09-16T20:48:55.038 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9400000000000004e-07 2024-09-16T20:48:55.038 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.038 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.038 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:55.038 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481225+0000", 2024-09-16T20:48:55.038 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0230000000000003e-06 2024-09-16T20:48:55.038 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.038 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.038 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:55.038 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481234+0000", 2024-09-16T20:48:55.039 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2920000000000001e-06 2024-09-16T20:48:55.039 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.039 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.039 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:55.039 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481240+0000", 2024-09-16T20:48:55.039 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9390000000000002e-06 2024-09-16T20:48:55.039 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.039 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.039 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:55.039 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.523544+0000", 2024-09-16T20:48:55.039 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.042304475000000001 2024-09-16T20:48:55.040 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.040 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.040 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:55.040 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.523624+0000", 2024-09-16T20:48:55.040 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.975e-05 2024-09-16T20:48:55.040 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.040 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.040 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:48:55.040 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.523975+0000", 2024-09-16T20:48:55.040 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00035063200000000003 2024-09-16T20:48:55.040 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.041 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.041 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:55.041 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530337+0000", 2024-09-16T20:48:55.041 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0063623589999999997 2024-09-16T20:48:55.041 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.041 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.041 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:55.041 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531592+0000", 2024-09-16T20:48:55.041 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0012550599999999999 2024-09-16T20:48:55.041 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.041 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.042 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:55.042 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531633+0000", 2024-09-16T20:48:55.042 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1192000000000001e-05 2024-09-16T20:48:55.043 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.043 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.043 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:55.043 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531673+0000", 2024-09-16T20:48:55.043 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9345999999999999e-05 2024-09-16T20:48:55.044 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.044 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:55.044 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.044 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.044 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.044 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3573 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:48:55.044 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481257+0000", 2024-09-16T20:48:55.044 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 5.48609028, 2024-09-16T20:48:55.044 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050511027, 2024-09-16T20:48:55.044 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:55.044 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:55.045 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:55.045 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:55.045 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:48:55.045 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:48:55.045 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3573 2024-09-16T20:48:55.045 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.045 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:55.045 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.045 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:55.045 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481257+0000", 2024-09-16T20:48:55.045 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.046 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.046 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.046 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:55.046 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481257+0000", 2024-09-16T20:48:55.046 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.046 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.046 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.046 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:55.046 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481257+0000", 2024-09-16T20:48:55.046 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.99e-07 2024-09-16T20:48:55.046 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.046 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.047 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:55.047 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481260+0000", 2024-09-16T20:48:55.047 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1709999999999998e-06 2024-09-16T20:48:55.047 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.047 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.047 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:55.047 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481271+0000", 2024-09-16T20:48:55.047 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0795000000000001e-05 2024-09-16T20:48:55.047 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.047 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.047 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:55.047 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481277+0000", 2024-09-16T20:48:55.048 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1670000000000004e-06 2024-09-16T20:48:55.048 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.048 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.048 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:55.048 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.524647+0000", 2024-09-16T20:48:55.048 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.043369810000000002 2024-09-16T20:48:55.048 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.048 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.048 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:55.048 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.524725+0000", 2024-09-16T20:48:55.048 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7286000000000005e-05 2024-09-16T20:48:55.049 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.049 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.049 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:48:55.049 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525062+0000", 2024-09-16T20:48:55.049 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00033739499999999999 2024-09-16T20:48:55.049 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.049 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.049 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:55.049 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530692+0000", 2024-09-16T20:48:55.049 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0056301479999999997 2024-09-16T20:48:55.049 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.050 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.050 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:55.050 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531694+0000", 2024-09-16T20:48:55.050 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0010017419999999999 2024-09-16T20:48:55.050 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.050 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.050 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:55.050 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531737+0000", 2024-09-16T20:48:55.050 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2762999999999997e-05 2024-09-16T20:48:55.050 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.050 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.050 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:55.051 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531768+0000", 2024-09-16T20:48:55.051 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1151e-05 2024-09-16T20:48:55.051 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.051 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:55.051 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.051 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.051 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.051 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3574 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:48:55.051 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:48:55.051 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 5.4860573239999999, 2024-09-16T20:48:55.051 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053579441999999998, 2024-09-16T20:48:55.051 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:55.052 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:55.052 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:55.052 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:55.052 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:48:55.052 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:48:55.052 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3574 2024-09-16T20:48:55.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.052 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:55.052 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.052 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:55.052 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:48:55.053 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.053 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.053 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.053 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:55.053 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:48:55.053 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.053 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.053 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.053 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:55.053 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:48:55.053 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2e-07 2024-09-16T20:48:55.054 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.054 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.054 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:55.054 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481292+0000", 2024-09-16T20:48:55.054 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2630000000000002e-06 2024-09-16T20:48:55.054 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.054 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.054 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:55.054 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481297+0000", 2024-09-16T20:48:55.054 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.296e-06 2024-09-16T20:48:55.055 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.055 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.055 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:55.055 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481300+0000", 2024-09-16T20:48:55.055 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7019999999999999e-06 2024-09-16T20:48:55.055 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.055 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.055 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:55.055 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525533+0000", 2024-09-16T20:48:55.055 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.044232318 2024-09-16T20:48:55.055 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.056 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.056 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:55.056 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525619+0000", 2024-09-16T20:48:55.056 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5892999999999999e-05 2024-09-16T20:48:55.056 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.056 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.056 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:48:55.056 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525987+0000", 2024-09-16T20:48:55.056 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00036836099999999999 2024-09-16T20:48:55.056 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.056 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.056 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:55.057 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531779+0000", 2024-09-16T20:48:55.057 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0057918240000000001 2024-09-16T20:48:55.057 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.060 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.061 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:55.061 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534747+0000", 2024-09-16T20:48:55.061 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0029681600000000001 2024-09-16T20:48:55.061 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.061 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.061 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:55.061 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534832+0000", 2024-09-16T20:48:55.062 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5180999999999996e-05 2024-09-16T20:48:55.062 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.062 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.062 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:55.062 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534869+0000", 2024-09-16T20:48:55.062 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7024000000000003e-05 2024-09-16T20:48:55.062 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.062 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:55.063 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.063 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.063 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.063 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3575 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:48:55.063 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:48:55.063 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 5.4860425839999998, 2024-09-16T20:48:55.063 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053801463000000001, 2024-09-16T20:48:55.063 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:55.064 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:55.064 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:55.064 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:55.064 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:48:55.064 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:48:55.064 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3575 2024-09-16T20:48:55.064 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.064 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:55.064 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.064 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:55.065 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:48:55.065 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.065 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.065 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.065 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:55.065 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:48:55.065 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.065 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.065 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.065 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:55.066 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481305+0000", 2024-09-16T20:48:55.066 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5400000000000002e-07 2024-09-16T20:48:55.066 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.066 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.066 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:55.066 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481307+0000", 2024-09-16T20:48:55.066 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7800000000000001e-06 2024-09-16T20:48:55.066 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.066 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.066 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:55.066 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481312+0000", 2024-09-16T20:48:55.066 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6349999999999997e-06 2024-09-16T20:48:55.067 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.067 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.067 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:55.067 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481315+0000", 2024-09-16T20:48:55.067 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7829999999999999e-06 2024-09-16T20:48:55.067 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.067 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.067 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:55.067 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.526876+0000", 2024-09-16T20:48:55.067 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.045561070000000002 2024-09-16T20:48:55.067 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.067 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.068 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:55.068 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.526954+0000", 2024-09-16T20:48:55.068 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8108000000000005e-05 2024-09-16T20:48:55.068 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.068 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.068 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:48:55.068 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527282+0000", 2024-09-16T20:48:55.068 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000327483 2024-09-16T20:48:55.068 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.068 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.068 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:55.069 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531784+0000", 2024-09-16T20:48:55.069 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0045020169999999997 2024-09-16T20:48:55.069 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.069 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.069 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:55.069 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534921+0000", 2024-09-16T20:48:55.069 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.003137791 2024-09-16T20:48:55.069 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.069 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.069 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:55.069 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534976+0000", 2024-09-16T20:48:55.069 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4953000000000001e-05 2024-09-16T20:48:55.070 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.070 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.070 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:55.070 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535106+0000", 2024-09-16T20:48:55.070 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000129589 2024-09-16T20:48:55.070 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.070 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:55.070 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.070 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.070 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.070 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3576 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:48:55.071 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:48:55.071 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 5.4860286719999998, 2024-09-16T20:48:55.071 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053955436000000002, 2024-09-16T20:48:55.071 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:55.071 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:55.071 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:55.071 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:55.071 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:48:55.071 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:48:55.071 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3576 2024-09-16T20:48:55.071 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.072 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:55.072 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.072 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:55.072 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:48:55.072 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.072 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.072 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.072 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:55.072 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:48:55.072 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.072 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.073 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.073 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:55.073 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481319+0000", 2024-09-16T20:48:55.073 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.53e-07 2024-09-16T20:48:55.073 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.073 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.073 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:55.073 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481320+0000", 2024-09-16T20:48:55.073 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5230000000000001e-06 2024-09-16T20:48:55.073 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.073 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.074 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:55.074 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481330+0000", 2024-09-16T20:48:55.074 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0190000000000001e-05 2024-09-16T20:48:55.074 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.074 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.074 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:55.074 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481333+0000", 2024-09-16T20:48:55.074 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.151e-06 2024-09-16T20:48:55.074 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.074 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.074 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:55.075 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527689+0000", 2024-09-16T20:48:55.075 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046355192000000003 2024-09-16T20:48:55.075 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.075 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.075 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:55.075 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527764+0000", 2024-09-16T20:48:55.075 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5776000000000001e-05 2024-09-16T20:48:55.075 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.075 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.075 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:48:55.075 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528141+0000", 2024-09-16T20:48:55.075 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00037647599999999999 2024-09-16T20:48:55.076 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.076 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.076 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:55.076 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531788+0000", 2024-09-16T20:48:55.076 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0036470679999999998 2024-09-16T20:48:55.076 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.076 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.076 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:55.076 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535174+0000", 2024-09-16T20:48:55.076 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0033862670000000001 2024-09-16T20:48:55.076 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.077 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.077 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:55.077 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535236+0000", 2024-09-16T20:48:55.077 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1797999999999993e-05 2024-09-16T20:48:55.077 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.077 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.077 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:55.077 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535274+0000", 2024-09-16T20:48:55.077 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7741999999999999e-05 2024-09-16T20:48:55.077 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.077 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:55.077 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.078 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.078 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.078 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3577 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:48:55.078 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:48:55.078 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 5.4860070719999996, 2024-09-16T20:48:55.078 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054073807000000002, 2024-09-16T20:48:55.078 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:55.078 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:55.078 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:55.078 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:55.078 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:48:55.078 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:48:55.079 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3577 2024-09-16T20:48:55.079 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.079 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:55.079 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.079 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:55.079 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:48:55.079 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.079 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.079 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.079 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:55.079 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:48:55.080 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.080 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.080 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.080 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:55.080 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:48:55.080 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.96e-07 2024-09-16T20:48:55.080 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.080 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.080 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:55.080 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481342+0000", 2024-09-16T20:48:55.080 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.714e-06 2024-09-16T20:48:55.081 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.081 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.081 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:55.081 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481349+0000", 2024-09-16T20:48:55.081 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4930000000000001e-06 2024-09-16T20:48:55.081 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.081 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.081 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:55.081 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481353+0000", 2024-09-16T20:48:55.081 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4879999999999999e-06 2024-09-16T20:48:55.081 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.081 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.082 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:55.082 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528619+0000", 2024-09-16T20:48:55.082 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047266351999999998 2024-09-16T20:48:55.082 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.082 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.082 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:55.082 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528706+0000", 2024-09-16T20:48:55.082 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6796e-05 2024-09-16T20:48:55.082 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.082 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.082 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:48:55.082 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529148+0000", 2024-09-16T20:48:55.083 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00044182999999999999 2024-09-16T20:48:55.083 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.083 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.083 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:55.083 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531792+0000", 2024-09-16T20:48:55.083 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0026443119999999998 2024-09-16T20:48:55.083 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.083 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.083 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:55.083 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535319+0000", 2024-09-16T20:48:55.084 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0035266439999999998 2024-09-16T20:48:55.084 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.084 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.084 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:55.084 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535372+0000", 2024-09-16T20:48:55.084 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3207000000000001e-05 2024-09-16T20:48:55.084 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.084 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.084 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:55.084 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535414+0000", 2024-09-16T20:48:55.084 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1675000000000003e-05 2024-09-16T20:48:55.085 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.085 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:55.085 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.085 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.085 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.085 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3578 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:48:55.085 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:48:55.085 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 5.4859859029999996, 2024-09-16T20:48:55.085 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054189586999999997, 2024-09-16T20:48:55.086 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:55.086 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:55.086 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:55.086 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:55.086 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:48:55.086 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:48:55.086 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3578 2024-09-16T20:48:55.086 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.086 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:55.087 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.087 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:55.087 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:48:55.087 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.087 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.087 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.087 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:55.087 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:48:55.087 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.087 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.087 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.087 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:55.088 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:48:55.088 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0800000000000001e-07 2024-09-16T20:48:55.088 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.088 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.088 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:55.088 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481363+0000", 2024-09-16T20:48:55.088 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0339999999999999e-06 2024-09-16T20:48:55.088 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.088 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.088 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:55.088 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481371+0000", 2024-09-16T20:48:55.089 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4440000000000002e-06 2024-09-16T20:48:55.089 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.089 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.089 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:55.089 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481374+0000", 2024-09-16T20:48:55.089 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2049999999999998e-06 2024-09-16T20:48:55.089 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.089 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.089 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:55.089 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529555+0000", 2024-09-16T20:48:55.090 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048181202999999999 2024-09-16T20:48:55.090 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.090 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.090 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:55.090 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529616+0000", 2024-09-16T20:48:55.090 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0912000000000002e-05 2024-09-16T20:48:55.090 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.090 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.090 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:48:55.091 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529939+0000", 2024-09-16T20:48:55.091 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00032269800000000001 2024-09-16T20:48:55.091 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.091 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.091 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:55.092 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532602+0000", 2024-09-16T20:48:55.092 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0026634940000000002 2024-09-16T20:48:55.092 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.092 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.092 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:55.092 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535465+0000", 2024-09-16T20:48:55.092 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.002862149 2024-09-16T20:48:55.092 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.092 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.092 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:55.092 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535516+0000", 2024-09-16T20:48:55.093 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1696000000000001e-05 2024-09-16T20:48:55.093 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.093 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.093 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:55.093 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535551+0000", 2024-09-16T20:48:55.093 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4443999999999998e-05 2024-09-16T20:48:55.093 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.093 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:55.093 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.093 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.093 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.094 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3579 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:48:55.094 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:48:55.094 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 5.4859674639999998, 2024-09-16T20:48:55.094 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054295151, 2024-09-16T20:48:55.094 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:55.094 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:55.094 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:55.094 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:55.094 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:48:55.094 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:48:55.094 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3579 2024-09-16T20:48:55.094 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.095 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:55.095 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.095 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:55.095 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:48:55.095 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.095 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.095 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.095 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:55.095 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:48:55.095 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.095 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.096 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.096 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:55.096 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:48:55.096 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3300000000000001e-07 2024-09-16T20:48:55.096 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.096 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.096 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:55.096 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481381+0000", 2024-09-16T20:48:55.096 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.649e-06 2024-09-16T20:48:55.096 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.096 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.097 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:55.097 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481387+0000", 2024-09-16T20:48:55.097 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4519999999999996e-06 2024-09-16T20:48:55.097 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.097 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.097 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:55.097 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481390+0000", 2024-09-16T20:48:55.097 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0170000000000001e-06 2024-09-16T20:48:55.097 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.097 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.097 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:55.098 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530756+0000", 2024-09-16T20:48:55.098 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049365882999999999 2024-09-16T20:48:55.098 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.098 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.098 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:55.098 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530845+0000", 2024-09-16T20:48:55.098 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9197000000000007e-05 2024-09-16T20:48:55.098 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.098 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.098 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:48:55.098 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531191+0000", 2024-09-16T20:48:55.098 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00034562500000000001 2024-09-16T20:48:55.099 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.099 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.099 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:55.099 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534078+0000", 2024-09-16T20:48:55.099 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0028878179999999999 2024-09-16T20:48:55.099 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.099 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.099 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:55.099 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535589+0000", 2024-09-16T20:48:55.099 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00151085 2024-09-16T20:48:55.099 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.099 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.100 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:55.100 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535639+0000", 2024-09-16T20:48:55.100 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9697000000000002e-05 2024-09-16T20:48:55.100 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.100 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.100 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:55.100 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535675+0000", 2024-09-16T20:48:55.100 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5729999999999998e-05 2024-09-16T20:48:55.100 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.100 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:55.101 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.101 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.101 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.101 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3580 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:48:55.101 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:48:55.101 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 5.4859538319999999, 2024-09-16T20:48:55.101 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054397361999999998, 2024-09-16T20:48:55.101 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:55.101 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:55.101 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:55.101 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:55.102 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:48:55.102 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:48:55.102 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3580 2024-09-16T20:48:55.102 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.102 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:55.102 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.102 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:55.102 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:48:55.102 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.102 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.102 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.102 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:55.103 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:48:55.103 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.103 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.103 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.103 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:55.103 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:48:55.103 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2600000000000001e-07 2024-09-16T20:48:55.103 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.103 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.103 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:55.103 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481395+0000", 2024-09-16T20:48:55.103 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.544e-06 2024-09-16T20:48:55.104 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.104 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.104 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:55.104 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481404+0000", 2024-09-16T20:48:55.104 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8850000000000005e-06 2024-09-16T20:48:55.104 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.104 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.104 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:55.104 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481407+0000", 2024-09-16T20:48:55.104 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1540000000000002e-06 2024-09-16T20:48:55.104 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.105 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.105 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:55.105 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531823+0000", 2024-09-16T20:48:55.105 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050415637999999999 2024-09-16T20:48:55.105 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.105 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.105 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:55.105 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531895+0000", 2024-09-16T20:48:55.105 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2179000000000001e-05 2024-09-16T20:48:55.105 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.105 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.105 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:48:55.106 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532236+0000", 2024-09-16T20:48:55.106 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00034073799999999999 2024-09-16T20:48:55.106 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.106 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.106 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:55.106 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534083+0000", 2024-09-16T20:48:55.106 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001847298 2024-09-16T20:48:55.106 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.106 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.106 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:55.106 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535714+0000", 2024-09-16T20:48:55.106 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0016316810000000001 2024-09-16T20:48:55.107 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.107 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.107 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:55.107 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535759+0000", 2024-09-16T20:48:55.107 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4478999999999999e-05 2024-09-16T20:48:55.107 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.107 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.107 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:55.107 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535791+0000", 2024-09-16T20:48:55.107 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.154e-05 2024-09-16T20:48:55.107 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.107 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:55.108 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.108 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.108 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.108 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3581 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:48:55.108 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:48:55.108 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 5.4859318349999997, 2024-09-16T20:48:55.108 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.057094403000000002, 2024-09-16T20:48:55.108 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:55.108 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:55.109 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:55.109 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:55.109 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:48:55.109 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:48:55.109 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3581 2024-09-16T20:48:55.109 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.109 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:55.109 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.109 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:55.109 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:48:55.109 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.109 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.110 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.110 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:55.110 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:48:55.110 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.110 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.110 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.110 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:55.110 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:48:55.110 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8799999999999998e-07 2024-09-16T20:48:55.110 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.110 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.111 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:55.111 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481418+0000", 2024-09-16T20:48:55.111 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1859999999999999e-06 2024-09-16T20:48:55.111 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.111 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.111 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:55.111 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481423+0000", 2024-09-16T20:48:55.111 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8279999999999999e-06 2024-09-16T20:48:55.111 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.111 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.111 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:55.111 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481426+0000", 2024-09-16T20:48:55.112 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0110000000000001e-06 2024-09-16T20:48:55.112 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.112 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.112 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:55.112 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532631+0000", 2024-09-16T20:48:55.112 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051204514999999999 2024-09-16T20:48:55.112 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.112 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.112 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:55.112 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.533381+0000", 2024-09-16T20:48:55.113 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00075029000000000003 2024-09-16T20:48:55.113 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.113 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.113 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:48:55.113 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.533739+0000", 2024-09-16T20:48:55.113 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00035773100000000002 2024-09-16T20:48:55.113 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.113 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.113 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:48:55.113 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535123+0000", 2024-09-16T20:48:55.113 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0013836160000000001 2024-09-16T20:48:55.113 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.114 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.114 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:55.114 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538413+0000", 2024-09-16T20:48:55.114 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0032902510000000001 2024-09-16T20:48:55.114 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.114 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.114 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:55.114 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538470+0000", 2024-09-16T20:48:55.114 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6697999999999998e-05 2024-09-16T20:48:55.114 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.114 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.115 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:55.115 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538510+0000", 2024-09-16T20:48:55.115 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9988999999999999e-05 2024-09-16T20:48:55.115 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.115 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:55.115 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.115 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.115 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:55.115 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:55.117 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:48:55.118 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:55.116+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:55.217 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:55.216+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:55.265 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:55.265 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:48:55.265 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:48:55.265 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:55.273 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:48:55.357 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:55.356+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:55.419 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:55.418+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:55.421 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:55.421 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:48:55.421 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:48:55.421 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:48:55.421 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:55.429 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:48:55.521 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:55.519+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:55.579 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:48:55.579 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:48:55.579 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:48:55.579 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:48:55.579 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.579 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3709 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1464320~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:55.579 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.638971+0000", 2024-09-16T20:48:55.579 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.938280865000003, 2024-09-16T20:48:55.580 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049858670000000001, 2024-09-16T20:48:55.580 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:55.580 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:55.580 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:55.580 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:55.580 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:55.580 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:55.580 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3709 2024-09-16T20:48:55.580 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.580 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:55.580 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.580 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:55.580 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638971+0000", 2024-09-16T20:48:55.580 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.581 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.581 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.581 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:55.581 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638971+0000", 2024-09-16T20:48:55.581 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.581 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.581 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.581 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:55.581 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638972+0000", 2024-09-16T20:48:55.581 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0400000000000002e-07 2024-09-16T20:48:55.581 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.581 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.581 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:55.582 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638973+0000", 2024-09-16T20:48:55.582 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.578e-06 2024-09-16T20:48:55.582 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.582 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.582 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:55.582 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638979+0000", 2024-09-16T20:48:55.582 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.502e-06 2024-09-16T20:48:55.582 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.582 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.582 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:55.582 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638983+0000", 2024-09-16T20:48:55.582 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2690000000000001e-06 2024-09-16T20:48:55.582 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.582 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.582 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:55.583 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688628+0000", 2024-09-16T20:48:55.583 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049644547999999997 2024-09-16T20:48:55.583 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.583 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.583 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:55.583 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688718+0000", 2024-09-16T20:48:55.583 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0802000000000002e-05 2024-09-16T20:48:55.583 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.583 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.583 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:55.583 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688830+0000", 2024-09-16T20:48:55.583 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000111667 2024-09-16T20:48:55.583 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.584 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:55.584 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.584 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.584 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.584 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3712 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1472512~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:55.584 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:48:55.584 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.938264337, 2024-09-16T20:48:55.584 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050004305999999998, 2024-09-16T20:48:55.584 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:55.584 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:55.584 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:55.584 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:55.584 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:55.584 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:55.585 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3712 2024-09-16T20:48:55.585 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.585 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:55.585 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.585 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:55.585 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:48:55.585 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.585 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.585 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.585 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:55.585 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:48:55.585 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.585 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.585 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.586 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:55.586 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:48:55.586 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0199999999999998e-07 2024-09-16T20:48:55.586 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.586 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.586 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:55.586 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638990+0000", 2024-09-16T20:48:55.586 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.618e-06 2024-09-16T20:48:55.586 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.586 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.586 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:55.586 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638995+0000", 2024-09-16T20:48:55.586 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3940000000000004e-06 2024-09-16T20:48:55.586 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.587 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.587 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:55.587 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639000+0000", 2024-09-16T20:48:55.587 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9309999999999999e-06 2024-09-16T20:48:55.587 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.587 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.587 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:55.587 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688867+0000", 2024-09-16T20:48:55.587 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049866955999999997 2024-09-16T20:48:55.587 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.587 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.587 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:55.587 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688933+0000", 2024-09-16T20:48:55.587 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5359999999999998e-05 2024-09-16T20:48:55.588 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.588 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.588 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:55.588 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688992+0000", 2024-09-16T20:48:55.588 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9744999999999998e-05 2024-09-16T20:48:55.588 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.588 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:55.588 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.588 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.588 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.588 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3714 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1480704~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:55.588 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:48:55.588 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.938246747000001, 2024-09-16T20:48:55.588 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050152552000000003, 2024-09-16T20:48:55.589 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:55.589 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:55.589 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:55.589 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:55.589 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:55.589 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:55.589 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3714 2024-09-16T20:48:55.589 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.589 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:55.589 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.589 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:55.589 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:48:55.589 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.589 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.590 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.590 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:55.590 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:48:55.590 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.590 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.590 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.590 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:55.590 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:48:55.590 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0499999999999999e-07 2024-09-16T20:48:55.590 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.590 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.590 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:55.590 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639007+0000", 2024-09-16T20:48:55.590 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5340000000000001e-06 2024-09-16T20:48:55.591 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.591 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.591 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:55.591 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639013+0000", 2024-09-16T20:48:55.591 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3260000000000003e-06 2024-09-16T20:48:55.591 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.591 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.591 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:55.591 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639018+0000", 2024-09-16T20:48:55.591 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2850000000000002e-06 2024-09-16T20:48:55.591 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.591 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.591 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:55.591 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689026+0000", 2024-09-16T20:48:55.592 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.05000798 2024-09-16T20:48:55.592 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.592 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.592 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:55.592 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689077+0000", 2024-09-16T20:48:55.592 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1063999999999998e-05 2024-09-16T20:48:55.592 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.592 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.592 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:55.592 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689158+0000", 2024-09-16T20:48:55.592 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1057999999999995e-05 2024-09-16T20:48:55.592 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.592 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:55.592 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.593 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.593 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.593 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3716 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1488896~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:55.593 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:48:55.593 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.938228936999998, 2024-09-16T20:48:55.593 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050280142, 2024-09-16T20:48:55.593 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:55.593 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:55.593 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:55.593 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:55.593 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:55.593 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:55.593 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3716 2024-09-16T20:48:55.594 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.594 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:55.594 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.594 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:55.594 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:48:55.594 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.594 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.594 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.594 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:55.594 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:48:55.594 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.594 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.594 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.594 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:55.594 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639024+0000", 2024-09-16T20:48:55.595 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9999999999999997e-07 2024-09-16T20:48:55.595 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.595 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.595 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:55.595 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639026+0000", 2024-09-16T20:48:55.595 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.781e-06 2024-09-16T20:48:55.595 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.595 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.595 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:55.595 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639032+0000", 2024-09-16T20:48:55.595 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7780000000000004e-06 2024-09-16T20:48:55.595 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.595 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.595 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:55.596 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639036+0000", 2024-09-16T20:48:55.596 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6909999999999996e-06 2024-09-16T20:48:55.596 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.596 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.596 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:55.596 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689191+0000", 2024-09-16T20:48:55.596 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050155284000000001 2024-09-16T20:48:55.596 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.596 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.596 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:55.597 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689242+0000", 2024-09-16T20:48:55.597 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0662999999999999e-05 2024-09-16T20:48:55.597 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.597 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.597 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:55.597 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689304+0000", 2024-09-16T20:48:55.597 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1345000000000003e-05 2024-09-16T20:48:55.597 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.597 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:55.597 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.597 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.597 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.597 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3719 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1497088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:55.597 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:48:55.598 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.938201659000001, 2024-09-16T20:48:55.598 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050399715999999997, 2024-09-16T20:48:55.598 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:55.598 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:55.598 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:55.598 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:55.598 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:55.598 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:55.598 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3719 2024-09-16T20:48:55.598 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.598 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:55.598 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.598 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:55.598 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:48:55.599 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.599 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.599 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.599 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:55.599 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:48:55.599 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.599 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.599 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.599 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:55.599 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:48:55.599 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9700000000000002e-07 2024-09-16T20:48:55.599 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.599 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.599 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:55.600 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639053+0000", 2024-09-16T20:48:55.600 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2900000000000001e-06 2024-09-16T20:48:55.600 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.600 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.600 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:55.600 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639060+0000", 2024-09-16T20:48:55.600 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7750000000000004e-06 2024-09-16T20:48:55.600 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.600 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.600 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:55.600 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639066+0000", 2024-09-16T20:48:55.600 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0639999999999999e-06 2024-09-16T20:48:55.600 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.601 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.601 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:55.601 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689337+0000", 2024-09-16T20:48:55.601 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050270784999999998 2024-09-16T20:48:55.601 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.601 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.601 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:55.601 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689392+0000", 2024-09-16T20:48:55.601 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4920999999999998e-05 2024-09-16T20:48:55.601 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.601 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.601 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:55.601 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689450+0000", 2024-09-16T20:48:55.601 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8483999999999998e-05 2024-09-16T20:48:55.602 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.602 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:55.602 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.602 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.602 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.602 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3721 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1505280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:55.602 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:48:55.602 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.938178526999998, 2024-09-16T20:48:55.602 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050513608000000002, 2024-09-16T20:48:55.602 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:55.602 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:55.602 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:55.602 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:55.602 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:55.603 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:55.603 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3721 2024-09-16T20:48:55.603 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.603 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:55.603 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.603 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:55.603 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:48:55.603 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.603 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.603 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.603 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:55.603 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:48:55.603 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.603 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.604 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.604 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:55.604 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:48:55.604 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7e-07 2024-09-16T20:48:55.604 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.604 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.604 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:55.604 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639076+0000", 2024-09-16T20:48:55.604 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8899999999999999e-06 2024-09-16T20:48:55.604 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.604 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.604 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:55.604 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639083+0000", 2024-09-16T20:48:55.604 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1210000000000001e-06 2024-09-16T20:48:55.605 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.605 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.605 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:55.605 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639089+0000", 2024-09-16T20:48:55.605 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.643e-06 2024-09-16T20:48:55.605 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.605 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.605 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:55.605 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689474+0000", 2024-09-16T20:48:55.605 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050385679000000003 2024-09-16T20:48:55.605 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.605 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.605 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:55.605 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689533+0000", 2024-09-16T20:48:55.606 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8303000000000001e-05 2024-09-16T20:48:55.606 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.606 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.606 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:55.606 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689587+0000", 2024-09-16T20:48:55.606 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4601999999999997e-05 2024-09-16T20:48:55.606 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.606 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:55.606 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.606 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.606 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.606 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3723 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1513472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:55.606 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:48:55.606 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.938147723, 2024-09-16T20:48:55.607 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050699691999999998, 2024-09-16T20:48:55.607 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:55.607 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:55.607 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:55.607 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:55.607 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:55.607 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:55.607 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3723 2024-09-16T20:48:55.607 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.607 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:55.607 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.607 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:55.607 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:48:55.607 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.608 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.608 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.608 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:55.608 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:48:55.608 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.608 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.608 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.608 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:55.608 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:48:55.608 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3399999999999999e-07 2024-09-16T20:48:55.608 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.608 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.608 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:55.608 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639107+0000", 2024-09-16T20:48:55.609 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3860000000000001e-06 2024-09-16T20:48:55.609 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.609 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.609 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:55.609 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639114+0000", 2024-09-16T20:48:55.609 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9800000000000001e-06 2024-09-16T20:48:55.609 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.609 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.609 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:55.609 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639129+0000", 2024-09-16T20:48:55.610 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4887e-05 2024-09-16T20:48:55.610 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.610 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.610 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:55.610 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689625+0000", 2024-09-16T20:48:55.610 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050495848000000003 2024-09-16T20:48:55.610 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.610 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.610 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:55.610 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689720+0000", 2024-09-16T20:48:55.610 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4821000000000006e-05 2024-09-16T20:48:55.610 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.610 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.611 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:55.611 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689804+0000", 2024-09-16T20:48:55.611 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4235999999999998e-05 2024-09-16T20:48:55.611 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.611 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:55.611 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.611 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.611 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.611 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3725 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1521664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:55.611 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:48:55.611 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.938114126999999, 2024-09-16T20:48:55.611 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050797660000000001, 2024-09-16T20:48:55.611 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:55.611 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:55.612 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:55.612 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:55.612 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:55.612 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:55.612 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3725 2024-09-16T20:48:55.612 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.612 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:55.612 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.612 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:55.612 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:48:55.612 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.612 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.612 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.612 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:55.613 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:48:55.613 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.613 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.613 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.613 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:55.613 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639139+0000", 2024-09-16T20:48:55.613 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8999999999999997e-07 2024-09-16T20:48:55.613 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.613 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.613 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:55.613 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639141+0000", 2024-09-16T20:48:55.613 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.182e-06 2024-09-16T20:48:55.613 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.613 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.614 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:55.614 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639154+0000", 2024-09-16T20:48:55.614 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3626e-05 2024-09-16T20:48:55.614 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.614 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.614 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:55.614 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639160+0000", 2024-09-16T20:48:55.614 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6810000000000003e-06 2024-09-16T20:48:55.614 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.614 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.614 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:55.615 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689831+0000", 2024-09-16T20:48:55.615 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050670907000000001 2024-09-16T20:48:55.615 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.615 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.615 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:55.615 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689882+0000", 2024-09-16T20:48:55.615 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0445999999999998e-05 2024-09-16T20:48:55.615 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.615 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.615 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:55.615 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689936+0000", 2024-09-16T20:48:55.615 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4327999999999999e-05 2024-09-16T20:48:55.615 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.615 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:55.616 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.616 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.616 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.616 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3727 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1529856~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:55.616 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:48:55.616 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.93807855, 2024-09-16T20:48:55.616 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051016922999999999, 2024-09-16T20:48:55.616 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:55.616 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:55.617 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:55.617 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:55.617 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:55.617 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:55.617 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3727 2024-09-16T20:48:55.617 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.617 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:55.617 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.617 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:55.617 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:48:55.617 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.617 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.617 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.617 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:55.618 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:48:55.618 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.618 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.618 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.618 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:55.618 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:48:55.618 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9999999999999998e-07 2024-09-16T20:48:55.618 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.618 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.618 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:55.618 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639183+0000", 2024-09-16T20:48:55.618 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2399999999999996e-06 2024-09-16T20:48:55.618 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.618 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.619 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:55.619 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639195+0000", 2024-09-16T20:48:55.619 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1113e-05 2024-09-16T20:48:55.619 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.619 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.619 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:55.619 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639206+0000", 2024-09-16T20:48:55.619 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1503e-05 2024-09-16T20:48:55.619 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.619 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.619 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:55.619 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689969+0000", 2024-09-16T20:48:55.619 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050762676 2024-09-16T20:48:55.619 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.620 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.620 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:55.620 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690071+0000", 2024-09-16T20:48:55.620 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102163 2024-09-16T20:48:55.620 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.620 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.620 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:55.620 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690191+0000", 2024-09-16T20:48:55.620 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000119728 2024-09-16T20:48:55.620 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.620 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:55.620 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.620 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.620 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.621 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3730 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1538048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:55.621 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:48:55.621 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.938036986999997, 2024-09-16T20:48:55.621 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051131467999999999, 2024-09-16T20:48:55.621 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:55.621 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:55.621 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:55.621 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:55.621 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:55.621 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:55.621 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3730 2024-09-16T20:48:55.621 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.621 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:55.622 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.622 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:55.622 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:48:55.622 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.622 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.622 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.622 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:55.622 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:48:55.622 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.622 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.622 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.622 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:55.622 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639216+0000", 2024-09-16T20:48:55.623 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.39e-07 2024-09-16T20:48:55.623 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.623 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.623 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:55.623 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639218+0000", 2024-09-16T20:48:55.623 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.537e-06 2024-09-16T20:48:55.623 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.623 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.623 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:55.623 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639229+0000", 2024-09-16T20:48:55.623 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1122e-05 2024-09-16T20:48:55.623 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.623 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.623 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:55.624 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639237+0000", 2024-09-16T20:48:55.624 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8509999999999999e-06 2024-09-16T20:48:55.624 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.624 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.624 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:55.624 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690227+0000", 2024-09-16T20:48:55.624 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050990147 2024-09-16T20:48:55.624 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.624 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.624 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:55.624 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690278+0000", 2024-09-16T20:48:55.624 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0223999999999999e-05 2024-09-16T20:48:55.624 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.624 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.625 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:55.625 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690347+0000", 2024-09-16T20:48:55.625 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9147999999999996e-05 2024-09-16T20:48:55.625 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.625 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:55.625 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.625 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.625 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.625 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3731 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1546240~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:55.625 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:48:55.625 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.938006279, 2024-09-16T20:48:55.625 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051276744999999999, 2024-09-16T20:48:55.625 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:55.626 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:55.626 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:55.626 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:55.626 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:55.626 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:55.626 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3731 2024-09-16T20:48:55.626 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.626 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:55.626 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.626 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:55.626 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:48:55.626 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.626 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.626 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.627 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:55.627 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:48:55.627 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.627 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.627 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.627 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:55.627 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:48:55.627 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9900000000000001e-07 2024-09-16T20:48:55.627 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.627 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.627 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:55.627 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639249+0000", 2024-09-16T20:48:55.627 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3630000000000001e-06 2024-09-16T20:48:55.628 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.628 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.628 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:55.628 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639255+0000", 2024-09-16T20:48:55.628 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4960000000000001e-06 2024-09-16T20:48:55.628 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.628 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.628 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:55.628 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639261+0000", 2024-09-16T20:48:55.629 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2839999999999997e-06 2024-09-16T20:48:55.629 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.629 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.629 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:55.629 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690375+0000", 2024-09-16T20:48:55.629 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051114684000000001 2024-09-16T20:48:55.629 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.629 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.629 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:55.629 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690425+0000", 2024-09-16T20:48:55.629 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9379999999999998e-05 2024-09-16T20:48:55.629 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.629 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.629 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:55.630 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690523+0000", 2024-09-16T20:48:55.630 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8139000000000003e-05 2024-09-16T20:48:55.630 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.630 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:55.630 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.630 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.630 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.630 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3732 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1554432~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:55.630 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:48:55.630 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.937960496999999, 2024-09-16T20:48:55.630 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051380096, 2024-09-16T20:48:55.630 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:55.630 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:55.631 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:55.631 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:55.631 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:55.631 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:55.631 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3732 2024-09-16T20:48:55.631 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.631 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:55.631 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.631 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:55.631 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:48:55.631 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.631 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.631 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.631 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:55.632 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:48:55.632 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.632 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.632 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.632 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:55.632 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:48:55.632 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6100000000000001e-07 2024-09-16T20:48:55.632 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.632 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.632 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:55.632 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639295+0000", 2024-09-16T20:48:55.632 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.447e-06 2024-09-16T20:48:55.632 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.632 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.633 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:55.633 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639302+0000", 2024-09-16T20:48:55.633 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0570000000000004e-06 2024-09-16T20:48:55.633 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.633 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.633 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:55.633 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639307+0000", 2024-09-16T20:48:55.633 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1429999999999997e-06 2024-09-16T20:48:55.633 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.633 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.633 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:55.633 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690555+0000", 2024-09-16T20:48:55.633 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051247963000000001 2024-09-16T20:48:55.633 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.634 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.634 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:55.634 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690602+0000", 2024-09-16T20:48:55.634 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7602000000000003e-05 2024-09-16T20:48:55.634 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.634 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.634 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:55.634 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690672+0000", 2024-09-16T20:48:55.634 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9423000000000003e-05 2024-09-16T20:48:55.634 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.634 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:55.634 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.634 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.634 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.634 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3734 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1562624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:55.635 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:48:55.635 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.937938846999998, 2024-09-16T20:48:55.635 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051490610999999999, 2024-09-16T20:48:55.635 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:55.635 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:55.635 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:55.635 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:55.635 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:55.635 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:55.635 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3734 2024-09-16T20:48:55.635 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.635 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:55.635 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.635 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:55.636 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:48:55.636 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.636 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.636 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.636 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:55.636 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:48:55.636 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.636 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.636 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.636 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:55.636 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639314+0000", 2024-09-16T20:48:55.636 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4400000000000001e-07 2024-09-16T20:48:55.636 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.636 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.637 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:55.637 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639316+0000", 2024-09-16T20:48:55.637 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1459999999999999e-06 2024-09-16T20:48:55.637 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.637 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.637 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:55.637 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639322+0000", 2024-09-16T20:48:55.637 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7610000000000004e-06 2024-09-16T20:48:55.637 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.637 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.637 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:55.637 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639327+0000", 2024-09-16T20:48:55.637 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.186e-06 2024-09-16T20:48:55.637 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.638 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.638 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:55.638 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690698+0000", 2024-09-16T20:48:55.638 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051371223000000001 2024-09-16T20:48:55.638 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.638 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.638 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:55.638 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690748+0000", 2024-09-16T20:48:55.638 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0192999999999999e-05 2024-09-16T20:48:55.638 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.638 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.638 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:55.638 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690804+0000", 2024-09-16T20:48:55.638 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5658000000000001e-05 2024-09-16T20:48:55.639 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.639 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:55.639 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.639 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.639 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.639 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3736 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1570816~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:55.639 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:48:55.639 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.937919588, 2024-09-16T20:48:55.639 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051596528000000003, 2024-09-16T20:48:55.639 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:55.639 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:55.639 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:55.639 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:55.639 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:55.640 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:55.640 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3736 2024-09-16T20:48:55.640 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.640 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:55.640 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.640 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:55.640 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:48:55.640 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.640 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.640 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.640 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:55.640 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:48:55.640 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.641 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.641 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.641 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:55.641 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:48:55.641 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9999999999999999e-07 2024-09-16T20:48:55.641 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.641 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.641 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:55.641 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639335+0000", 2024-09-16T20:48:55.641 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.635e-06 2024-09-16T20:48:55.641 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.641 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.641 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:55.641 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639340+0000", 2024-09-16T20:48:55.641 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5840000000000002e-06 2024-09-16T20:48:55.642 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.642 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.642 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:55.642 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639346+0000", 2024-09-16T20:48:55.642 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1190000000000002e-06 2024-09-16T20:48:55.642 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.642 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.642 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:55.642 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690817+0000", 2024-09-16T20:48:55.642 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051470721999999997 2024-09-16T20:48:55.642 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.642 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.642 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:55.642 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690875+0000", 2024-09-16T20:48:55.643 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7441000000000001e-05 2024-09-16T20:48:55.643 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.643 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.643 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:55.643 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690929+0000", 2024-09-16T20:48:55.643 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4727e-05 2024-09-16T20:48:55.643 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.643 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:55.643 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.643 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.643 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.643 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5434 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:55.643 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:48:55.643 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.712658021999999, 2024-09-16T20:48:55.644 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099755831000000003, 2024-09-16T20:48:55.644 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:55.644 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:55.644 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:55.644 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:55.644 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:55.644 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:55.644 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5434 2024-09-16T20:48:55.644 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.644 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:55.644 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.644 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:55.644 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:48:55.645 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.645 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.645 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.645 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:55.645 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:48:55.645 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.645 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.645 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.645 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:55.645 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864595+0000", 2024-09-16T20:48:55.645 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1099999999999995e-07 2024-09-16T20:48:55.645 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.645 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.645 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:55.646 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864597+0000", 2024-09-16T20:48:55.646 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3989999999999998e-06 2024-09-16T20:48:55.646 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.646 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.646 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:55.646 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864603+0000", 2024-09-16T20:48:55.646 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9309999999999996e-06 2024-09-16T20:48:55.646 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.646 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.646 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:55.646 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864607+0000", 2024-09-16T20:48:55.646 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6200000000000001e-06 2024-09-16T20:48:55.646 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.647 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.647 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:55.647 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964164+0000", 2024-09-16T20:48:55.647 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099557162000000005 2024-09-16T20:48:55.647 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.647 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.647 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:55.647 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964244+0000", 2024-09-16T20:48:55.647 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9597000000000004e-05 2024-09-16T20:48:55.647 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.647 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.647 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:55.647 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964350+0000", 2024-09-16T20:48:55.647 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010641099999999999 2024-09-16T20:48:55.647 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.648 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:55.648 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.648 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.648 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.648 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5435 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 20480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:55.648 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:48:55.648 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.712628199999997, 2024-09-16T20:48:55.648 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100897102, 2024-09-16T20:48:55.648 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:55.648 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:55.648 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:55.648 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:55.648 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:48:55.648 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:55.649 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5435 2024-09-16T20:48:55.649 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.649 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:55.649 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.649 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:55.649 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:48:55.649 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.649 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.649 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.649 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:55.649 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:48:55.649 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.649 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.649 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.650 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:55.650 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:48:55.650 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8000000000000001e-07 2024-09-16T20:48:55.650 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.650 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.650 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:55.650 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864626+0000", 2024-09-16T20:48:55.650 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.745e-06 2024-09-16T20:48:55.650 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.650 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.650 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:55.650 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864633+0000", 2024-09-16T20:48:55.650 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3339999999999998e-06 2024-09-16T20:48:55.651 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.651 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.651 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:55.651 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864636+0000", 2024-09-16T20:48:55.651 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1200000000000002e-06 2024-09-16T20:48:55.651 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.651 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.651 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:55.651 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965299+0000", 2024-09-16T20:48:55.651 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10066352000000001 2024-09-16T20:48:55.651 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.651 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.651 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:55.651 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965390+0000", 2024-09-16T20:48:55.652 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0894000000000001e-05 2024-09-16T20:48:55.652 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.652 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.652 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:55.652 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965521+0000", 2024-09-16T20:48:55.652 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013110899999999999 2024-09-16T20:48:55.652 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.652 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:55.652 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.652 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.652 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.652 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:152 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 327680~32768 out=32768b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:48:55.652 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:48:55.652 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 31.414131258000001, 2024-09-16T20:48:55.653 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11869405800000001, 2024-09-16T20:48:55.653 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:55.653 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:55.653 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:55.653 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:55.653 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:48:55.653 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:48:55.653 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 152 2024-09-16T20:48:55.653 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.653 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:55.653 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.653 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:55.653 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:48:55.653 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.654 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.654 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.654 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:55.654 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:48:55.654 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.654 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.654 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.654 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:55.654 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163122+0000", 2024-09-16T20:48:55.654 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6400000000000003e-07 2024-09-16T20:48:55.654 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.654 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.654 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:55.654 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163124+0000", 2024-09-16T20:48:55.655 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.565e-06 2024-09-16T20:48:55.655 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.655 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.655 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:55.655 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163136+0000", 2024-09-16T20:48:55.655 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1717e-05 2024-09-16T20:48:55.655 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.655 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.655 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:55.655 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163142+0000", 2024-09-16T20:48:55.655 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1750000000000002e-06 2024-09-16T20:48:55.655 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.655 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.655 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:55.656 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163606+0000", 2024-09-16T20:48:55.656 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00046418799999999998 2024-09-16T20:48:55.656 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.656 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.656 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:55.656 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163656+0000", 2024-09-16T20:48:55.656 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9852999999999999e-05 2024-09-16T20:48:55.656 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.656 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.656 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:55.656 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281815+0000", 2024-09-16T20:48:55.656 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11815909600000001 2024-09-16T20:48:55.656 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.656 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:55.657 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.657 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.657 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.657 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:153 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:48:55.657 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:48:55.657 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 31.414098305, 2024-09-16T20:48:55.657 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11903462400000001, 2024-09-16T20:48:55.657 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:55.657 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:55.657 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:55.657 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:55.657 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:48:55.657 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:48:55.657 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 153 2024-09-16T20:48:55.658 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.658 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:55.658 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.658 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:55.658 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:48:55.658 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.658 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.658 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.658 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:55.658 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:48:55.658 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.658 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.658 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.658 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:55.659 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163155+0000", 2024-09-16T20:48:55.659 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.37e-07 2024-09-16T20:48:55.659 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.659 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.659 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:55.659 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163157+0000", 2024-09-16T20:48:55.659 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6010000000000002e-06 2024-09-16T20:48:55.659 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.659 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.659 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:55.659 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163165+0000", 2024-09-16T20:48:55.659 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2619999999999998e-06 2024-09-16T20:48:55.659 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.659 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.660 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:55.660 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163171+0000", 2024-09-16T20:48:55.660 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.711e-06 2024-09-16T20:48:55.660 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.660 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.660 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:55.660 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281868+0000", 2024-09-16T20:48:55.660 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118696622 2024-09-16T20:48:55.660 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.660 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.660 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:55.660 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281969+0000", 2024-09-16T20:48:55.660 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101371 2024-09-16T20:48:55.661 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.661 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.661 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:55.661 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282189+0000", 2024-09-16T20:48:55.661 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00021952000000000001 2024-09-16T20:48:55.661 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.661 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:55.661 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.661 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.661 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.661 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:154 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 360448~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:48:55.661 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:48:55.661 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 31.414066533, 2024-09-16T20:48:55.661 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11932035100000001, 2024-09-16T20:48:55.661 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:55.662 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:55.662 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:48:55.662 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:48:55.662 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:48:55.662 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:48:55.662 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 154 2024-09-16T20:48:55.662 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.662 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:55.662 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.662 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:55.663 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:48:55.663 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.663 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.663 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.663 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:55.663 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:48:55.663 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.663 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.663 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.663 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:55.663 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:48:55.663 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:48:55.663 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.664 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.664 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:55.664 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163189+0000", 2024-09-16T20:48:55.664 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6840000000000001e-06 2024-09-16T20:48:55.664 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.664 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.664 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:55.664 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163196+0000", 2024-09-16T20:48:55.664 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7920000000000004e-06 2024-09-16T20:48:55.664 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.664 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.664 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:55.664 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163201+0000", 2024-09-16T20:48:55.664 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2809999999999999e-06 2024-09-16T20:48:55.665 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.665 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.665 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:55.665 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282219+0000", 2024-09-16T20:48:55.665 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.119018343 2024-09-16T20:48:55.665 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.665 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.665 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:55.665 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282305+0000", 2024-09-16T20:48:55.665 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5865000000000005e-05 2024-09-16T20:48:55.665 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.665 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.665 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:55.666 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282506+0000", 2024-09-16T20:48:55.666 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000200927 2024-09-16T20:48:55.666 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.666 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:55.666 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.666 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.666 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.666 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop(client.4366.0:3088 3.1e e38/23 3:79e5ec01:::rbd_header.110e891a9b2b:head v 38'306, mlcod=36'304)", 2024-09-16T20:48:55.666 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:48:55.666 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 31.413047665000001, 2024-09-16T20:48:55.666 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118102759, 2024-09-16T20:48:55.666 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:48:55.666 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:48:55.666 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:55.667 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:48:55.667 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.667 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:48:55.667 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:48:55.667 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.667 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.667 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.667 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:48:55.667 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:48:55.667 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:48:55.667 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.667 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.667 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:48:55.667 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:48:55.668 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.85e-07 2024-09-16T20:48:55.668 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.668 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.668 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:48:55.668 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164212+0000", 2024-09-16T20:48:55.668 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3999999999999997e-06 2024-09-16T20:48:55.668 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.668 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.668 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:48:55.668 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281553+0000", 2024-09-16T20:48:55.668 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11734135499999999 2024-09-16T20:48:55.668 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.668 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.668 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:48:55.669 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281567+0000", 2024-09-16T20:48:55.669 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4188999999999999e-05 2024-09-16T20:48:55.669 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.669 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.669 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:48:55.669 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281653+0000", 2024-09-16T20:48:55.669 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5385999999999999e-05 2024-09-16T20:48:55.669 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.669 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.669 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:48:55.669 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281679+0000", 2024-09-16T20:48:55.669 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6689000000000001e-05 2024-09-16T20:48:55.669 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.669 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.672 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:48:55.672 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282277+0000", 2024-09-16T20:48:55.672 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00059798000000000002 2024-09-16T20:48:55.672 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:48:55.672 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:48:55.672 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:48:55.673 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282307+0000", 2024-09-16T20:48:55.673 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9975e-05 2024-09-16T20:48:55.673 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.673 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:55.673 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.673 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:48:55.673 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:48:55.673 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:48:55.675 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:48:55.675 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:55.674+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:55.734 INFO:teuthology.orchestra.run.smithi029.stderr:noscrub is set 2024-09-16T20:48:55.735 INFO:teuthology.orchestra.run.smithi029.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-16T20:48:55.775 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:55.773+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:55.796 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:55.796 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:48:55.796 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:48:55.796 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:55.805 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:48:55.889 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:55.888+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:55.955 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:55.955 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:48:55.955 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:48:55.955 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:48:55.955 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:55.964 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:48:55.977 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:55.977+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:56.078 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:56.077+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:56.112 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:56.113 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:48:56.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:48:56.113 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:48:56.113 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.113 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6414 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3569664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:56.113 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871708+0000", 2024-09-16T20:48:56.113 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.238744525000001, 2024-09-16T20:48:56.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048373317999999998, 2024-09-16T20:48:56.113 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:56.113 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:56.113 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:56.113 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:56.114 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:56.114 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:56.114 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6414 2024-09-16T20:48:56.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.114 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:56.114 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.114 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:56.114 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871708+0000", 2024-09-16T20:48:56.114 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:56.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.114 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.114 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:56.114 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871708+0000", 2024-09-16T20:48:56.115 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:56.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.115 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.115 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:56.115 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871708+0000", 2024-09-16T20:48:56.115 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9799999999999999e-07 2024-09-16T20:48:56.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.115 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.115 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:56.115 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871710+0000", 2024-09-16T20:48:56.115 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2699999999999999e-06 2024-09-16T20:48:56.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.115 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.115 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:56.116 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871718+0000", 2024-09-16T20:48:56.116 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8499999999999994e-06 2024-09-16T20:48:56.116 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.116 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.116 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:56.116 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871723+0000", 2024-09-16T20:48:56.116 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1819999999999997e-06 2024-09-16T20:48:56.116 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.116 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.116 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:56.116 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.919883+0000", 2024-09-16T20:48:56.116 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048159955999999997 2024-09-16T20:48:56.116 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.117 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:56.117 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.919969+0000", 2024-09-16T20:48:56.117 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5808000000000002e-05 2024-09-16T20:48:56.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.117 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:56.117 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920081+0000", 2024-09-16T20:48:56.117 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000111854 2024-09-16T20:48:56.117 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:56.117 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:56.117 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:56.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.118 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.118 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6415 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3577856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:56.118 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871730+0000", 2024-09-16T20:48:56.118 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.238722009999996, 2024-09-16T20:48:56.118 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.04851482, 2024-09-16T20:48:56.118 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:56.118 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:56.118 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:56.118 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:56.118 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:56.118 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:56.118 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6415 2024-09-16T20:48:56.118 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.119 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:56.119 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.119 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:56.119 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871730+0000", 2024-09-16T20:48:56.119 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:56.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.119 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.119 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:56.119 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871730+0000", 2024-09-16T20:48:56.119 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:56.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.119 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.119 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:56.120 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871731+0000", 2024-09-16T20:48:56.120 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6899999999999998e-07 2024-09-16T20:48:56.120 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.120 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.120 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:56.120 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871733+0000", 2024-09-16T20:48:56.120 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0600000000000002e-06 2024-09-16T20:48:56.120 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.120 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.120 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:56.120 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871740+0000", 2024-09-16T20:48:56.120 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3749999999999997e-06 2024-09-16T20:48:56.120 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.121 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.121 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:56.121 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871749+0000", 2024-09-16T20:48:56.121 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9109999999999999e-06 2024-09-16T20:48:56.121 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.121 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.121 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:56.121 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920129+0000", 2024-09-16T20:48:56.121 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048380175999999997 2024-09-16T20:48:56.121 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.121 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.121 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:56.121 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920181+0000", 2024-09-16T20:48:56.121 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2077000000000003e-05 2024-09-16T20:48:56.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.122 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.122 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:56.122 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920245+0000", 2024-09-16T20:48:56.122 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3851999999999998e-05 2024-09-16T20:48:56.122 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:56.122 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:56.122 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:56.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.122 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.122 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6417 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3586048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:56.122 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871756+0000", 2024-09-16T20:48:56.122 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.238696375000004, 2024-09-16T20:48:56.122 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048628301999999998, 2024-09-16T20:48:56.123 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:56.123 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:56.123 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:56.123 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:56.123 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:56.123 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:56.123 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6417 2024-09-16T20:48:56.123 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.123 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:56.123 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.123 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:56.123 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871756+0000", 2024-09-16T20:48:56.123 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:56.124 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.124 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.124 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:56.124 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871756+0000", 2024-09-16T20:48:56.124 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:56.124 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.124 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.124 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:56.124 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871756+0000", 2024-09-16T20:48:56.124 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0100000000000005e-07 2024-09-16T20:48:56.124 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.124 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.124 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:56.124 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871758+0000", 2024-09-16T20:48:56.124 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.902e-06 2024-09-16T20:48:56.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.125 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.125 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:56.125 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871764+0000", 2024-09-16T20:48:56.125 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8170000000000004e-06 2024-09-16T20:48:56.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.125 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.125 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:56.125 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871769+0000", 2024-09-16T20:48:56.125 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6500000000000004e-06 2024-09-16T20:48:56.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.125 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.125 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:56.125 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920276+0000", 2024-09-16T20:48:56.126 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048507794999999999 2024-09-16T20:48:56.126 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.126 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.126 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:56.126 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920329+0000", 2024-09-16T20:48:56.126 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2534999999999998e-05 2024-09-16T20:48:56.126 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.126 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.126 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:56.126 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920384+0000", 2024-09-16T20:48:56.126 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5102000000000002e-05 2024-09-16T20:48:56.127 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:56.128 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:56.128 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:56.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.128 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.128 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6419 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3594240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:56.128 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871775+0000", 2024-09-16T20:48:56.128 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.238677197999998, 2024-09-16T20:48:56.128 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048754724999999999, 2024-09-16T20:48:56.128 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:56.128 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:56.128 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:56.129 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:56.129 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:56.129 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:56.129 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6419 2024-09-16T20:48:56.129 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.129 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:56.129 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.129 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:56.129 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871775+0000", 2024-09-16T20:48:56.129 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:56.129 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.129 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.129 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:56.130 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871775+0000", 2024-09-16T20:48:56.130 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:56.130 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.130 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.130 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:56.130 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871775+0000", 2024-09-16T20:48:56.130 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2500000000000001e-07 2024-09-16T20:48:56.130 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.130 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.130 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:56.130 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871777+0000", 2024-09-16T20:48:56.130 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9649999999999998e-06 2024-09-16T20:48:56.131 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.131 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.131 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:56.131 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871784+0000", 2024-09-16T20:48:56.131 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3419999999999996e-06 2024-09-16T20:48:56.131 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.131 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.131 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:56.131 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871789+0000", 2024-09-16T20:48:56.131 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1889999999999998e-06 2024-09-16T20:48:56.132 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.132 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.132 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:56.132 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920416+0000", 2024-09-16T20:48:56.132 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048627363999999999 2024-09-16T20:48:56.132 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.132 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.132 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:56.132 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920472+0000", 2024-09-16T20:48:56.132 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5429000000000001e-05 2024-09-16T20:48:56.132 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.132 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.132 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:56.133 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920530+0000", 2024-09-16T20:48:56.133 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8111e-05 2024-09-16T20:48:56.133 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:56.133 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:56.133 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:56.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.133 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6420 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3602432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:56.133 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871796+0000", 2024-09-16T20:48:56.133 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.238656284999998, 2024-09-16T20:48:56.133 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048857298, 2024-09-16T20:48:56.133 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:56.133 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:56.133 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:56.134 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:56.134 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:56.134 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:56.134 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6420 2024-09-16T20:48:56.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.134 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:56.134 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.134 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:56.134 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871796+0000", 2024-09-16T20:48:56.134 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:56.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.134 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.134 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:56.134 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871796+0000", 2024-09-16T20:48:56.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:56.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.135 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.135 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:56.135 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871854+0000", 2024-09-16T20:48:56.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8187999999999999e-05 2024-09-16T20:48:56.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.135 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.135 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:56.135 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871856+0000", 2024-09-16T20:48:56.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3549999999999999e-06 2024-09-16T20:48:56.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.135 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.136 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:56.136 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871866+0000", 2024-09-16T20:48:56.136 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6500000000000008e-06 2024-09-16T20:48:56.136 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.136 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.136 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:56.136 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871872+0000", 2024-09-16T20:48:56.136 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3799999999999999e-06 2024-09-16T20:48:56.136 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.136 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.136 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:56.136 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920558+0000", 2024-09-16T20:48:56.136 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048685797000000003 2024-09-16T20:48:56.136 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.137 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.137 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:56.137 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920603+0000", 2024-09-16T20:48:56.137 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4629999999999998e-05 2024-09-16T20:48:56.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.137 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.137 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:56.137 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920653+0000", 2024-09-16T20:48:56.137 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0297999999999999e-05 2024-09-16T20:48:56.137 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:56.137 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:56.137 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:56.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.138 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6421 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3610624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:56.138 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871882+0000", 2024-09-16T20:48:56.138 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.238570217000003, 2024-09-16T20:48:56.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048913381999999998, 2024-09-16T20:48:56.138 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:56.138 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:56.138 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:56.138 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:56.138 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:56.138 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:56.138 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6421 2024-09-16T20:48:56.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.138 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:56.139 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.139 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:56.139 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871882+0000", 2024-09-16T20:48:56.139 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:56.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.139 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.139 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:56.139 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871882+0000", 2024-09-16T20:48:56.139 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:56.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.139 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.139 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:56.139 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.873257+0000", 2024-09-16T20:48:56.139 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001374789 2024-09-16T20:48:56.140 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.140 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.140 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:56.140 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.873261+0000", 2024-09-16T20:48:56.140 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5589999999999999e-06 2024-09-16T20:48:56.140 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.140 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.140 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:56.140 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.873275+0000", 2024-09-16T20:48:56.140 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3675999999999999e-05 2024-09-16T20:48:56.140 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.140 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.140 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:56.141 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.873281+0000", 2024-09-16T20:48:56.141 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6930000000000004e-06 2024-09-16T20:48:56.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.141 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.141 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:56.141 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920684+0000", 2024-09-16T20:48:56.141 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047402913999999997 2024-09-16T20:48:56.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.141 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.141 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:56.141 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920737+0000", 2024-09-16T20:48:56.141 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3600000000000002e-05 2024-09-16T20:48:56.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.141 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.142 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:56.142 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920795+0000", 2024-09-16T20:48:56.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8150999999999999e-05 2024-09-16T20:48:56.142 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:56.142 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:56.142 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:56.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.142 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.142 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:7491 3.1e 3:7c48f6ae:::rbd_object_map.110e891a9b2b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:56.142 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.925865+0000", 2024-09-16T20:48:56.142 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.184587073000003, 2024-09-16T20:48:56.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048897293000000001, 2024-09-16T20:48:56.142 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:56.142 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:56.143 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:56.143 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:56.143 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:56.143 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:56.143 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 7491 2024-09-16T20:48:56.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.143 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:56.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.143 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:56.143 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925865+0000", 2024-09-16T20:48:56.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:56.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.144 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.144 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:56.144 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925865+0000", 2024-09-16T20:48:56.144 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:56.144 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.144 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.144 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:56.144 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925866+0000", 2024-09-16T20:48:56.144 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.39e-07 2024-09-16T20:48:56.144 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.144 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.144 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:56.144 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925868+0000", 2024-09-16T20:48:56.144 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.508e-06 2024-09-16T20:48:56.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.145 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.145 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:56.145 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925874+0000", 2024-09-16T20:48:56.145 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7919999999999998e-06 2024-09-16T20:48:56.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.145 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.145 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:56.145 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925879+0000", 2024-09-16T20:48:56.145 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8579999999999999e-06 2024-09-16T20:48:56.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.145 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.145 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:56.145 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.970875+0000", 2024-09-16T20:48:56.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044996412 2024-09-16T20:48:56.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.146 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:56.146 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.970970+0000", 2024-09-16T20:48:56.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4732e-05 2024-09-16T20:48:56.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.146 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-16T20:48:56.146 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.971275+0000", 2024-09-16T20:48:56.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00030464199999999999 2024-09-16T20:48:56.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.147 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:56.147 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.974598+0000", 2024-09-16T20:48:56.147 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0033238159999999998 2024-09-16T20:48:56.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.147 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.147 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:48:56.147 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.974666+0000", 2024-09-16T20:48:56.147 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7617e-05 2024-09-16T20:48:56.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.147 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.147 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:48:56.147 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.974718+0000", 2024-09-16T20:48:56.147 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2259000000000002e-05 2024-09-16T20:48:56.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.147 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.148 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:56.148 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.974762+0000", 2024-09-16T20:48:56.148 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4218000000000003e-05 2024-09-16T20:48:56.148 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:56.148 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:56.148 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:56.148 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.148 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.148 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4754 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1505280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:56.148 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689097+0000", 2024-09-16T20:48:56.148 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 33.421354874999999, 2024-09-16T20:48:56.148 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048429947000000001, 2024-09-16T20:48:56.148 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:56.149 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:56.149 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:56.149 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:56.149 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:56.149 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:56.149 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4754 2024-09-16T20:48:56.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.149 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:56.149 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.149 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:56.149 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689097+0000", 2024-09-16T20:48:56.149 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:56.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.149 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.150 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:56.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689097+0000", 2024-09-16T20:48:56.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:56.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.150 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.150 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:56.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689098+0000", 2024-09-16T20:48:56.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.15e-07 2024-09-16T20:48:56.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.150 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.150 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:56.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689100+0000", 2024-09-16T20:48:56.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8190000000000001e-06 2024-09-16T20:48:56.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:56.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689106+0000", 2024-09-16T20:48:56.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2489999999999998e-06 2024-09-16T20:48:56.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:56.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689111+0000", 2024-09-16T20:48:56.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3179999999999997e-06 2024-09-16T20:48:56.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:56.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737428+0000", 2024-09-16T20:48:56.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048316572000000002 2024-09-16T20:48:56.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.152 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.152 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:56.152 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737475+0000", 2024-09-16T20:48:56.152 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7392000000000003e-05 2024-09-16T20:48:56.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.152 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.152 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:56.152 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737527+0000", 2024-09-16T20:48:56.152 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2182000000000003e-05 2024-09-16T20:48:56.152 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:56.152 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:56.152 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:56.153 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.153 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.153 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4756 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1513472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:56.153 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689118+0000", 2024-09-16T20:48:56.153 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 33.421334653999999, 2024-09-16T20:48:56.153 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048535146000000001, 2024-09-16T20:48:56.153 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:56.153 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:56.153 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:56.153 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:56.153 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:56.153 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:56.153 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4756 2024-09-16T20:48:56.153 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.154 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:56.154 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.154 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:56.154 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689118+0000", 2024-09-16T20:48:56.154 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:56.154 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.154 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.154 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:56.154 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689118+0000", 2024-09-16T20:48:56.154 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:56.154 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.154 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.154 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:56.154 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689118+0000", 2024-09-16T20:48:56.155 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7099999999999997e-07 2024-09-16T20:48:56.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.155 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.155 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:56.155 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689120+0000", 2024-09-16T20:48:56.155 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.787e-06 2024-09-16T20:48:56.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.155 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.155 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:56.155 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689126+0000", 2024-09-16T20:48:56.155 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2720000000000003e-06 2024-09-16T20:48:56.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.155 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.155 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:56.156 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689134+0000", 2024-09-16T20:48:56.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2190000000000004e-06 2024-09-16T20:48:56.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.156 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.156 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:56.156 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737553+0000", 2024-09-16T20:48:56.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048418993 2024-09-16T20:48:56.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.156 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.156 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:56.156 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737598+0000", 2024-09-16T20:48:56.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4743999999999997e-05 2024-09-16T20:48:56.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.156 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.157 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:56.157 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737653+0000", 2024-09-16T20:48:56.157 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4759999999999997e-05 2024-09-16T20:48:56.157 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:56.157 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:56.157 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:56.157 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.157 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.157 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4758 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1521664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:56.157 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689141+0000", 2024-09-16T20:48:56.157 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 33.421311674000002, 2024-09-16T20:48:56.157 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048640741000000001, 2024-09-16T20:48:56.157 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:56.158 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:56.158 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:56.158 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:56.158 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:56.158 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:56.158 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4758 2024-09-16T20:48:56.158 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.158 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:56.158 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.158 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:56.158 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689141+0000", 2024-09-16T20:48:56.158 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:56.158 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.158 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.159 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:56.159 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689141+0000", 2024-09-16T20:48:56.159 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:56.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.159 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.159 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:56.159 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689141+0000", 2024-09-16T20:48:56.159 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8799999999999998e-07 2024-09-16T20:48:56.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.159 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.159 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:56.159 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689143+0000", 2024-09-16T20:48:56.159 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.945e-06 2024-09-16T20:48:56.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.160 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.160 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:56.160 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689149+0000", 2024-09-16T20:48:56.160 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9800000000000003e-06 2024-09-16T20:48:56.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.160 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.160 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:56.160 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689155+0000", 2024-09-16T20:48:56.160 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7509999999999997e-06 2024-09-16T20:48:56.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.160 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.160 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:56.160 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737681+0000", 2024-09-16T20:48:56.160 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048526825000000003 2024-09-16T20:48:56.161 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.161 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.161 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:56.161 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737729+0000", 2024-09-16T20:48:56.161 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7219999999999999e-05 2024-09-16T20:48:56.161 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.161 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.161 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:56.161 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737781+0000", 2024-09-16T20:48:56.161 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2632e-05 2024-09-16T20:48:56.161 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:56.161 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:56.161 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:56.161 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.162 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.162 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4760 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1529856~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:56.162 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689161+0000", 2024-09-16T20:48:56.162 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 33.421291326999999, 2024-09-16T20:48:56.162 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048743981999999998, 2024-09-16T20:48:56.162 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:56.162 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:56.162 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:56.162 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:56.162 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:56.162 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:56.162 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4760 2024-09-16T20:48:56.162 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.162 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:56.163 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.163 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:56.163 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689161+0000", 2024-09-16T20:48:56.163 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:56.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.163 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.163 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:56.163 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689161+0000", 2024-09-16T20:48:56.163 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:56.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.163 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.163 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:56.163 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689161+0000", 2024-09-16T20:48:56.163 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5599999999999995e-07 2024-09-16T20:48:56.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.164 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.164 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:56.164 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689163+0000", 2024-09-16T20:48:56.164 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.818e-06 2024-09-16T20:48:56.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.164 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.164 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:56.164 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689169+0000", 2024-09-16T20:48:56.164 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0809999999999999e-06 2024-09-16T20:48:56.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.164 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.164 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:56.165 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689174+0000", 2024-09-16T20:48:56.165 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.938e-06 2024-09-16T20:48:56.165 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.165 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.165 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:56.165 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737807+0000", 2024-09-16T20:48:56.165 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048633124 2024-09-16T20:48:56.165 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.165 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.166 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:56.166 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737851+0000", 2024-09-16T20:48:56.166 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3764000000000003e-05 2024-09-16T20:48:56.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.166 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.166 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:56.166 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737905+0000", 2024-09-16T20:48:56.166 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3701e-05 2024-09-16T20:48:56.166 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:56.166 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:56.166 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:56.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.166 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.166 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4762 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1538048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:56.167 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689182+0000", 2024-09-16T20:48:56.167 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 33.421270104999998, 2024-09-16T20:48:56.167 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048850275999999998, 2024-09-16T20:48:56.167 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:56.167 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:56.167 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:56.167 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:56.167 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:56.167 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:56.167 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4762 2024-09-16T20:48:56.167 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.167 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:56.167 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.167 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:56.168 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689182+0000", 2024-09-16T20:48:56.168 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:56.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.168 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.168 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:56.168 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689182+0000", 2024-09-16T20:48:56.168 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:56.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.168 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.168 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:56.168 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689182+0000", 2024-09-16T20:48:56.168 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-16T20:48:56.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.168 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.169 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:56.169 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689184+0000", 2024-09-16T20:48:56.169 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8190000000000001e-06 2024-09-16T20:48:56.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.169 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.169 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:56.169 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689191+0000", 2024-09-16T20:48:56.169 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2530000000000001e-06 2024-09-16T20:48:56.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.169 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.169 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:56.169 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689195+0000", 2024-09-16T20:48:56.169 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9559999999999996e-06 2024-09-16T20:48:56.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.170 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.170 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:56.170 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737933+0000", 2024-09-16T20:48:56.170 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048737919999999997 2024-09-16T20:48:56.170 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.170 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.170 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:56.170 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737982+0000", 2024-09-16T20:48:56.170 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8089e-05 2024-09-16T20:48:56.170 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.170 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.170 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:56.170 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738032+0000", 2024-09-16T20:48:56.170 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0868999999999997e-05 2024-09-16T20:48:56.171 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:56.171 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:56.171 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:56.171 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.171 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.171 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4764 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1546240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:56.171 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689201+0000", 2024-09-16T20:48:56.171 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 33.421250747999999, 2024-09-16T20:48:56.171 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048957550000000002, 2024-09-16T20:48:56.171 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:56.171 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:56.171 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:56.171 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:56.171 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:56.172 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:56.172 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4764 2024-09-16T20:48:56.172 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.172 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:56.172 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.172 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:56.172 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689201+0000", 2024-09-16T20:48:56.172 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:56.172 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.172 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.172 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:56.172 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689201+0000", 2024-09-16T20:48:56.172 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:56.173 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.173 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.173 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:56.173 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689202+0000", 2024-09-16T20:48:56.173 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4200000000000002e-07 2024-09-16T20:48:56.173 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.173 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.173 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:56.173 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689204+0000", 2024-09-16T20:48:56.173 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9750000000000001e-06 2024-09-16T20:48:56.173 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.173 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.173 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:56.173 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689210+0000", 2024-09-16T20:48:56.174 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1e-06 2024-09-16T20:48:56.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.174 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.174 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:56.174 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689215+0000", 2024-09-16T20:48:56.174 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0180000000000001e-06 2024-09-16T20:48:56.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.174 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.174 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:56.174 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738058+0000", 2024-09-16T20:48:56.174 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048843573000000001 2024-09-16T20:48:56.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.174 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.174 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:56.175 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738102+0000", 2024-09-16T20:48:56.175 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3256000000000001e-05 2024-09-16T20:48:56.175 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.175 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.175 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:56.175 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738159+0000", 2024-09-16T20:48:56.175 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7286e-05 2024-09-16T20:48:56.175 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:56.175 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:56.175 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:56.175 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.175 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.175 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4766 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1554432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:56.175 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689221+0000", 2024-09-16T20:48:56.176 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 33.421230782000002, 2024-09-16T20:48:56.176 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049064996, 2024-09-16T20:48:56.176 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:56.176 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:56.176 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:56.176 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:56.176 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:56.176 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:56.176 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4766 2024-09-16T20:48:56.176 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.176 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:56.176 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.176 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:56.176 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689221+0000", 2024-09-16T20:48:56.177 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:56.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.177 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.177 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:56.177 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689221+0000", 2024-09-16T20:48:56.177 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:56.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.178 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.178 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:56.178 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689222+0000", 2024-09-16T20:48:56.178 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.34e-07 2024-09-16T20:48:56.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.178 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.178 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:56.178 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689224+0000", 2024-09-16T20:48:56.178 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.846e-06 2024-09-16T20:48:56.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.178 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.178 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:56.178 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689247+0000", 2024-09-16T20:48:56.179 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3448e-05 2024-09-16T20:48:56.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.179 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.179 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:56.179 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689254+0000", 2024-09-16T20:48:56.179 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6969999999999996e-06 2024-09-16T20:48:56.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.179 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.179 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:56.179 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738188+0000", 2024-09-16T20:48:56.179 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048934520000000002 2024-09-16T20:48:56.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.179 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.179 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:56.180 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738234+0000", 2024-09-16T20:48:56.180 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5695999999999998e-05 2024-09-16T20:48:56.180 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.180 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.180 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:56.180 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738286+0000", 2024-09-16T20:48:56.180 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2355000000000003e-05 2024-09-16T20:48:56.180 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:56.180 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:56.180 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:56.180 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.180 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.180 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4768 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1562624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:56.181 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689262+0000", 2024-09-16T20:48:56.181 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 33.421189828000003, 2024-09-16T20:48:56.181 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049157157999999999, 2024-09-16T20:48:56.181 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:56.181 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:56.181 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:56.181 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:56.181 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:56.181 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:56.181 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4768 2024-09-16T20:48:56.181 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.181 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:56.181 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.181 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:56.182 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689262+0000", 2024-09-16T20:48:56.182 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:56.182 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.182 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.182 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:56.182 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689262+0000", 2024-09-16T20:48:56.182 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:56.182 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.182 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.182 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:56.182 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689263+0000", 2024-09-16T20:48:56.182 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6199999999999998e-07 2024-09-16T20:48:56.182 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.182 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.183 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:56.183 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689270+0000", 2024-09-16T20:48:56.183 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.798e-06 2024-09-16T20:48:56.183 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.183 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.183 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:56.183 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689277+0000", 2024-09-16T20:48:56.183 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0890000000000002e-06 2024-09-16T20:48:56.183 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.183 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.183 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:56.183 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689282+0000", 2024-09-16T20:48:56.183 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0470000000000001e-06 2024-09-16T20:48:56.183 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.184 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.184 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:56.184 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738319+0000", 2024-09-16T20:48:56.184 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049037139 2024-09-16T20:48:56.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.184 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.184 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:56.184 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738365+0000", 2024-09-16T20:48:56.184 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6356999999999997e-05 2024-09-16T20:48:56.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.184 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.184 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:56.184 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738420+0000", 2024-09-16T20:48:56.184 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4265999999999998e-05 2024-09-16T20:48:56.185 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:56.185 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:56.185 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:56.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.185 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.185 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4770 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1570816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:56.185 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689289+0000", 2024-09-16T20:48:56.185 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 33.421163317999998, 2024-09-16T20:48:56.185 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049253705000000002, 2024-09-16T20:48:56.185 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:56.185 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:56.185 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:56.185 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:56.185 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:56.186 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:56.186 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4770 2024-09-16T20:48:56.186 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.186 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:56.186 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.186 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:56.186 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689289+0000", 2024-09-16T20:48:56.186 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:56.186 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.186 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.186 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:56.186 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689289+0000", 2024-09-16T20:48:56.186 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:56.187 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.187 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.187 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:56.187 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689289+0000", 2024-09-16T20:48:56.187 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0999999999999999e-07 2024-09-16T20:48:56.187 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.187 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.187 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:56.187 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689291+0000", 2024-09-16T20:48:56.187 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0030000000000001e-06 2024-09-16T20:48:56.187 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.187 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.187 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:56.187 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689297+0000", 2024-09-16T20:48:56.188 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9479999999999996e-06 2024-09-16T20:48:56.188 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.188 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.188 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:56.188 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689302+0000", 2024-09-16T20:48:56.188 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7389999999999999e-06 2024-09-16T20:48:56.188 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.188 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.188 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:56.188 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738442+0000", 2024-09-16T20:48:56.188 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049140001000000003 2024-09-16T20:48:56.188 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.188 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.188 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:56.189 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738490+0000", 2024-09-16T20:48:56.189 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7787000000000002e-05 2024-09-16T20:48:56.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.189 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.189 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:56.189 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738543+0000", 2024-09-16T20:48:56.189 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2816999999999999e-05 2024-09-16T20:48:56.189 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:56.189 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:56.189 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:56.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.189 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.189 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:3088 3.1e 3:79e5ec01:::rbd_header.110e891a9b2b:head [watch watch cookie 140213437171488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:48:56.189 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:48:56.190 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 31.946607301, 2024-09-16T20:48:56.190 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.11895186200000001, 2024-09-16T20:48:56.190 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:56.190 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:56.190 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:56.190 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:56.190 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:48:56.190 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:48:56.190 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3088 2024-09-16T20:48:56.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.190 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:56.190 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.190 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:56.190 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:48:56.191 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:56.191 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.191 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.191 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:56.191 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:48:56.191 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:56.191 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.191 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.191 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:56.191 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163846+0000", 2024-09-16T20:48:56.191 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6799999999999999e-07 2024-09-16T20:48:56.191 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.191 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.192 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:56.192 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163850+0000", 2024-09-16T20:48:56.192 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.929e-06 2024-09-16T20:48:56.192 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.192 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.192 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:56.192 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163862+0000", 2024-09-16T20:48:56.192 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1972e-05 2024-09-16T20:48:56.192 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.192 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.192 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:56.192 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163867+0000", 2024-09-16T20:48:56.192 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.451e-06 2024-09-16T20:48:56.192 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.193 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.193 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:56.193 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163912+0000", 2024-09-16T20:48:56.193 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4456000000000003e-05 2024-09-16T20:48:56.193 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.193 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.193 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:56.193 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163962+0000", 2024-09-16T20:48:56.193 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0578000000000003e-05 2024-09-16T20:48:56.193 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.193 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.193 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-16T20:48:56.193 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164058+0000", 2024-09-16T20:48:56.193 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5934999999999995e-05 2024-09-16T20:48:56.194 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.194 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.194 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:48:56.194 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164570+0000", 2024-09-16T20:48:56.194 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00051241100000000005 2024-09-16T20:48:56.194 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.194 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.194 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:56.194 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282629+0000", 2024-09-16T20:48:56.194 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.118058986 2024-09-16T20:48:56.194 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.194 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.194 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:48:56.194 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282725+0000", 2024-09-16T20:48:56.195 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5063999999999996e-05 2024-09-16T20:48:56.195 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.195 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.195 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:56.195 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282797+0000", 2024-09-16T20:48:56.195 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2212000000000006e-05 2024-09-16T20:48:56.195 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:56.195 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:56.195 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:56.195 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.195 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.195 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:8039 3.1e 3.ac44ff9e (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:48:56.195 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:48:56.195 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 6.4027799520000004, 2024-09-16T20:48:56.196 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0053310449999999, 2024-09-16T20:48:56.196 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:56.196 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:56.196 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:48:56.196 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:56.196 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:48:56.196 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:48:56.196 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8039 2024-09-16T20:48:56.196 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.196 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:56.196 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.196 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:56.196 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:48:56.196 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:56.197 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.197 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.197 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:56.197 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:48:56.197 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:56.197 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.197 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.197 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:56.197 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707674+0000", 2024-09-16T20:48:56.197 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6530000000000001e-06 2024-09-16T20:48:56.197 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.197 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.197 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:56.197 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707681+0000", 2024-09-16T20:48:56.198 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9639999999999997e-06 2024-09-16T20:48:56.198 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.198 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.198 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:56.198 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707700+0000", 2024-09-16T20:48:56.198 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8674000000000001e-05 2024-09-16T20:48:56.198 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.198 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.198 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:56.198 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707708+0000", 2024-09-16T20:48:56.198 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6810000000000004e-06 2024-09-16T20:48:56.198 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.198 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.198 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:56.199 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711293+0000", 2024-09-16T20:48:56.199 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035848189999999999 2024-09-16T20:48:56.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.199 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.199 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:48:56.199 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711301+0000", 2024-09-16T20:48:56.199 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4730000000000003e-06 2024-09-16T20:48:56.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.199 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.199 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:56.199 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.712990+0000", 2024-09-16T20:48:56.199 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.001689348 2024-09-16T20:48:56.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.199 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.200 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:56.200 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713003+0000", 2024-09-16T20:48:56.200 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3433e-05 2024-09-16T20:48:56.200 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:56.200 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:56.200 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:56.200 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.200 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.200 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:8040 3.1e 3.ac44ff9e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:48:56.200 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:48:56.200 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 6.4027195570000002, 2024-09-16T20:48:56.200 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0053410780000001, 2024-09-16T20:48:56.200 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:56.201 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:56.201 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:48:56.201 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:56.201 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:48:56.201 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:48:56.201 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8040 2024-09-16T20:48:56.201 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.201 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:56.201 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.201 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:56.201 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:48:56.201 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:56.201 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.201 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.202 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:56.202 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:48:56.202 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:56.202 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.202 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.202 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:56.202 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:48:56.202 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8100000000000003e-07 2024-09-16T20:48:56.202 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.202 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.202 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:56.202 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707737+0000", 2024-09-16T20:48:56.202 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5020000000000001e-06 2024-09-16T20:48:56.202 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.203 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.203 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:56.203 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707745+0000", 2024-09-16T20:48:56.203 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2169999999999994e-06 2024-09-16T20:48:56.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.203 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.203 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:56.203 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707751+0000", 2024-09-16T20:48:56.203 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.263e-06 2024-09-16T20:48:56.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.203 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.203 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:56.203 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711336+0000", 2024-09-16T20:48:56.203 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035848360000000001 2024-09-16T20:48:56.204 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.204 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.204 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:48:56.204 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711340+0000", 2024-09-16T20:48:56.204 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1309999999999999e-06 2024-09-16T20:48:56.204 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.204 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.204 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:56.204 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713060+0000", 2024-09-16T20:48:56.204 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0017195059999999 2024-09-16T20:48:56.204 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.204 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.204 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:56.205 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713074+0000", 2024-09-16T20:48:56.205 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4042e-05 2024-09-16T20:48:56.205 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:56.205 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:56.205 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:56.205 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.205 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.205 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4927.0:4487 3.1e 3.ac44ff9e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:48:56.205 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:48:56.205 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 6.4021993549999996, 2024-09-16T20:48:56.205 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0048840880000001, 2024-09-16T20:48:56.205 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:56.205 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:56.205 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:48:56.206 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:56.206 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4927", 2024-09-16T20:48:56.206 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:48:56.206 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4487 2024-09-16T20:48:56.206 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.206 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:56.206 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.206 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:56.206 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:48:56.206 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:56.206 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.206 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.206 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:56.206 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:48:56.206 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:56.207 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.207 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.207 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:56.207 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708254+0000", 2024-09-16T20:48:56.207 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1800000000000005e-07 2024-09-16T20:48:56.207 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.207 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.207 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:56.207 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708258+0000", 2024-09-16T20:48:56.207 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8689999999999997e-06 2024-09-16T20:48:56.207 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.208 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.208 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:56.208 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708267+0000", 2024-09-16T20:48:56.208 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9590000000000007e-06 2024-09-16T20:48:56.208 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.208 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.208 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:56.208 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708274+0000", 2024-09-16T20:48:56.208 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9970000000000001e-06 2024-09-16T20:48:56.208 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.208 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.208 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:56.209 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711366+0000", 2024-09-16T20:48:56.209 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030919530000000001 2024-09-16T20:48:56.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.209 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.209 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:48:56.209 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711372+0000", 2024-09-16T20:48:56.209 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9410000000000004e-06 2024-09-16T20:48:56.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.209 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.209 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:56.209 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713126+0000", 2024-09-16T20:48:56.209 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0017537249999999 2024-09-16T20:48:56.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:56.209 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:56.210 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:56.210 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713137+0000", 2024-09-16T20:48:56.210 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0825999999999999e-05 2024-09-16T20:48:56.210 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:56.210 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:56.210 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:56.210 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:56.210 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:56.210 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:56.211 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T20:48:56.212 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:48:56.213 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:56.210+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:56.313 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:56.312+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:56.340 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:48:56.340 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:48:56.357 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:48:56.357 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:48:56.360 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:56.360 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:48:56.360 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:48:56.360 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:56.368 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:48:56.373 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:48:56.373 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:48:56.389 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:48:56.390 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:48:56.406 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:48:56.406 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:48:56.422 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:48:56.423 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:48:56.451 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:56.450+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:56.490 INFO:teuthology.orchestra.run.smithi139.stdout:truncating image image_client.0-clone15 from 0xc51a200 (overlap 0xda600) to 0x15c00 2024-09-16T20:48:56.514 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:56.513+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:56.524 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:56.524 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:56.534 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:48:56.615 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:56.615+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:56.681 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:56.682 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:56.693 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:48:56.716 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:56.715+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:56.765 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #14, image image_client.0-clone14 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent15 2024-09-16T20:48:56.807 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:56.807 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:56.816 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:48:56.817 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:56.816+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:56.871 INFO:teuthology.orchestra.run.smithi139.stdout:2261 read 0x7e60000 thru 0x7e67b91 (0x7b92 bytes) 2024-09-16T20:48:56.872 INFO:teuthology.orchestra.run.smithi139.stdout:2262 read 0x2347600 thru 0x234d9ed (0x63ee bytes) 2024-09-16T20:48:56.873 INFO:teuthology.orchestra.run.smithi139.stdout:2263 punch from 0xf19400 to 0xf21a00, (0x8600 bytes) 2024-09-16T20:48:56.892 INFO:teuthology.orchestra.run.smithi139.stdout:2264 read 0x14b0800 thru 0x14b8969 (0x816a bytes) 2024-09-16T20:48:56.892 INFO:teuthology.orchestra.run.smithi139.stdout:2266 read 0x98a8400 thru 0x98b6723 (0xe324 bytes) 2024-09-16T20:48:56.892 INFO:teuthology.orchestra.run.smithi139.stdout:2267 write 0xa29a800 thru 0xa29c23b (0x1a3c bytes) 2024-09-16T20:48:56.892 INFO:teuthology.orchestra.run.smithi139.stdout:2270 read 0x67d3e00 thru 0x67e1759 (0xd95a bytes) 2024-09-16T20:48:56.892 INFO:teuthology.orchestra.run.smithi139.stdout:2271 punch from 0x9d2ea00 to 0x9d33400, (0x4a00 bytes) 2024-09-16T20:48:56.908 INFO:teuthology.orchestra.run.smithi139.stdout:2273 read 0x87b0c00 thru 0x87bf215 (0xe616 bytes) 2024-09-16T20:48:56.918 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:56.916+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:56.919 INFO:teuthology.orchestra.run.smithi139.stdout:2276 write 0xdde7200 thru 0xddf2185 (0xaf86 bytes) 2024-09-16T20:48:56.920 INFO:teuthology.orchestra.run.smithi139.stdout:2280 write 0x637fc00 thru 0x6385d6a (0x616b bytes) 2024-09-16T20:48:56.920 INFO:teuthology.orchestra.run.smithi139.stdout:2281 write 0x8725e00 thru 0x872c3ac (0x65ad bytes) 2024-09-16T20:48:56.921 INFO:teuthology.orchestra.run.smithi139.stdout:2283 write 0xc8cda00 thru 0xc8dae2a (0xd42b bytes) 2024-09-16T20:48:56.921 INFO:teuthology.orchestra.run.smithi139.stdout:2289 read 0x54ab400 thru 0x54b97bc (0xe3bd bytes) 2024-09-16T20:48:56.921 INFO:teuthology.orchestra.run.smithi139.stdout:2290 read 0x12f4e00 thru 0x1300584 (0xb785 bytes) 2024-09-16T20:48:56.921 INFO:teuthology.orchestra.run.smithi139.stdout:2291 read 0x189c400 thru 0x18a3ec8 (0x7ac9 bytes) 2024-09-16T20:48:56.925 INFO:teuthology.orchestra.run.smithi139.stdout:2292 trunc from 0xddf2200 to 0xa7c6800 2024-09-16T20:48:56.927 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:56.927 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:56.939 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:48:56.970 INFO:teuthology.orchestra.run.smithi139.stdout:2293 read 0x58a9c00 thru 0x58b8c3c (0xf03d bytes) 2024-09-16T20:48:56.971 INFO:teuthology.orchestra.run.smithi139.stdout:2295 write 0x3f71e00 thru 0x3f7e6cd (0xc8ce bytes) 2024-09-16T20:48:56.972 INFO:teuthology.orchestra.run.smithi139.stdout:2296 write 0x580e800 thru 0x5814ad5 (0x62d6 bytes) 2024-09-16T20:48:56.972 INFO:teuthology.orchestra.run.smithi139.stdout:2299 read 0xa702200 thru 0xa70f2ee (0xd0ef bytes) 2024-09-16T20:48:56.973 INFO:teuthology.orchestra.run.smithi139.stdout:2300 read 0x777800 thru 0x782262 (0xaa63 bytes) 2024-09-16T20:48:56.979 INFO:teuthology.orchestra.run.smithi139.stdout:2303 trunc from 0xa7c6800 to 0x5332000 2024-09-16T20:48:57.018 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:57.017+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:57.034 INFO:teuthology.orchestra.run.smithi139.stdout:2304 write 0xb726400 thru 0xb7317b4 (0xb3b5 bytes) 2024-09-16T20:48:57.038 INFO:teuthology.orchestra.run.smithi139.stdout:2305 trunc from 0xb731800 to 0x9720c00 2024-09-16T20:48:57.070 INFO:teuthology.orchestra.run.smithi139.stdout:2306 write 0xad78a00 thru 0xad8532a (0xc92b bytes) 2024-09-16T20:48:57.070 INFO:teuthology.orchestra.run.smithi139.stdout:2307 read 0x1f2ca00 thru 0x1f3b381 (0xe982 bytes) 2024-09-16T20:48:57.082 INFO:teuthology.orchestra.run.smithi139.stdout:2311 trunc from 0xad85400 to 0x1103e00 2024-09-16T20:48:57.093 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:57.093 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:57.104 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:48:57.120 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:57.119+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:57.152 INFO:teuthology.orchestra.run.smithi139.stdout:2312 write 0xc7d3400 thru 0xc7d4d2a (0x192b bytes) 2024-09-16T20:48:57.152 INFO:teuthology.orchestra.run.smithi139.stdout:2315 punch from 0x8abb400 to 0x8abd800, (0x2400 bytes) 2024-09-16T20:48:57.152 INFO:teuthology.orchestra.run.smithi139.stdout:2316 punch from 0x8d4fe00 to 0x8d5e200, (0xe400 bytes) 2024-09-16T20:48:57.152 INFO:teuthology.orchestra.run.smithi139.stdout:2317 read 0x53e1600 thru 0x53eca8b (0xb48c bytes) 2024-09-16T20:48:57.171 INFO:teuthology.orchestra.run.smithi139.stdout:2318 write 0xe7b0400 thru 0xe7b0bab (0x7ac bytes) 2024-09-16T20:48:57.187 INFO:teuthology.orchestra.run.smithi139.stdout:2319 trunc from 0xe7b0c00 to 0x13f2e00 2024-09-16T20:48:57.212 INFO:teuthology.orchestra.run.smithi139.stdout:2321 punch from 0xffae00 to 0x1004200, (0x9400 bytes) 2024-09-16T20:48:57.220 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:57.219+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:57.228 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:57.228 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:57.239 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:48:57.241 INFO:teuthology.orchestra.run.smithi139.stdout:2323 write 0xc574c00 thru 0xc57f911 (0xad12 bytes) 2024-09-16T20:48:57.241 INFO:teuthology.orchestra.run.smithi139.stdout:2324 punch from 0x81ba600 to 0x81c8400, (0xde00 bytes) 2024-09-16T20:48:57.241 INFO:teuthology.orchestra.run.smithi139.stdout:2327 punch from 0x8f8b000 to 0x8f95200, (0xa200 bytes) 2024-09-16T20:48:57.241 INFO:teuthology.orchestra.run.smithi139.stdout:2329 read 0x6623400 thru 0x6628097 (0x4c98 bytes) 2024-09-16T20:48:57.241 INFO:teuthology.orchestra.run.smithi139.stdout:2332 punch from 0xa25ca00 to 0xa26c600, (0xfc00 bytes) 2024-09-16T20:48:57.249 INFO:teuthology.orchestra.run.smithi139.stdout:2334 trunc from 0xc57fa00 to 0x5e08200 2024-09-16T20:48:57.283 INFO:teuthology.orchestra.run.smithi139.stdout:2335 write 0x9ffc400 thru 0xa0049d6 (0x85d7 bytes) 2024-09-16T20:48:57.285 INFO:teuthology.orchestra.run.smithi139.stdout:2336 write 0x214ee00 thru 0x21502c7 (0x14c8 bytes) 2024-09-16T20:48:57.285 INFO:teuthology.orchestra.run.smithi139.stdout:2339 read 0x613b800 thru 0x61406ba (0x4ebb bytes) 2024-09-16T20:48:57.285 INFO:teuthology.orchestra.run.smithi139.stdout:2341 punch from 0x10c200 to 0x111600, (0x5400 bytes) 2024-09-16T20:48:57.289 INFO:teuthology.orchestra.run.smithi139.stdout:2344 punch from 0x8bee800 to 0x8befa00, (0x1200 bytes) 2024-09-16T20:48:57.289 INFO:teuthology.orchestra.run.smithi139.stdout:2348 punch from 0x4d25800 to 0x4d31000, (0xb800 bytes) 2024-09-16T20:48:57.289 INFO:teuthology.orchestra.run.smithi139.stdout:2350 read 0x7936200 thru 0x793900c (0x2e0d bytes) 2024-09-16T20:48:57.289 INFO:teuthology.orchestra.run.smithi139.stdout:2357 punch from 0xd2b400 to 0xd30c00, (0x5800 bytes) 2024-09-16T20:48:57.289 INFO:teuthology.orchestra.run.smithi139.stdout:2358 write 0x1342e00 thru 0x134500e (0x220f bytes) 2024-09-16T20:48:57.293 INFO:teuthology.orchestra.run.smithi139.stdout:2359 write 0x589fa00 thru 0x58aa171 (0xa772 bytes) 2024-09-16T20:48:57.294 INFO:teuthology.orchestra.run.smithi139.stdout:2362 punch from 0x6576600 to 0x6580e00, (0xa800 bytes) 2024-09-16T20:48:57.308 INFO:teuthology.orchestra.run.smithi139.stdout:2363 write 0xacf7c00 thru 0xad0373f (0xbb40 bytes) 2024-09-16T20:48:57.319 INFO:teuthology.orchestra.run.smithi139.stdout:2364 trunc from 0xad03800 to 0xdf97c00 2024-09-16T20:48:57.323 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:57.322+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:57.331 INFO:teuthology.orchestra.run.smithi139.stdout:2366 read 0x30f5400 thru 0x310354e (0xe14f bytes) 2024-09-16T20:48:57.332 INFO:teuthology.orchestra.run.smithi139.stdout:2367 write 0xbdd5400 thru 0xbddf28d (0x9e8e bytes) 2024-09-16T20:48:57.332 INFO:teuthology.orchestra.run.smithi139.stdout:2368 read 0xab8c400 thru 0xab9c1d3 (0xfdd4 bytes) 2024-09-16T20:48:57.333 INFO:teuthology.orchestra.run.smithi139.stdout:2371 read 0x20d9400 thru 0x20da40b (0x100c bytes) 2024-09-16T20:48:57.333 INFO:teuthology.orchestra.run.smithi139.stdout:2372 punch from 0x8507000 to 0x8511600, (0xa600 bytes) 2024-09-16T20:48:57.333 INFO:teuthology.orchestra.run.smithi139.stdout:2373 read 0x1efe00 thru 0x1f7061 (0x7262 bytes) 2024-09-16T20:48:57.334 INFO:teuthology.orchestra.run.smithi139.stdout:2374 write 0x5f5ae00 thru 0x5f5ceaa (0x20ab bytes) 2024-09-16T20:48:57.334 INFO:teuthology.orchestra.run.smithi139.stdout:2376 write 0xa8b6600 thru 0xa8c345d (0xce5e bytes) 2024-09-16T20:48:57.344 INFO:teuthology.orchestra.run.smithi139.stdout:2378 trunc from 0xdf97c00 to 0x671b600 2024-09-16T20:48:57.360 INFO:teuthology.orchestra.run.smithi139.stdout:2381 write 0x1788c00 thru 0x178a444 (0x1845 bytes) 2024-09-16T20:48:57.367 INFO:teuthology.orchestra.run.smithi139.stdout:2385 clone 17 order 23 su 32768 sc 14 2024-09-16T20:48:57.406 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:57.407 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:57.415 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:48:57.421 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:57.421+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:57.522 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:57.521+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:57.563 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:57.563 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:57.571 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:48:57.623 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:57.621+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:57.679 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:57.679 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:57.687 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:48:57.724 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:57.723+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:57.750 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T20:48:57.823 INFO:teuthology.orchestra.run.smithi139.stdout:truncating image image_client.0-clone16 from 0x671b600 (overlap 0x1103e00) to 0xbe9000 2024-09-16T20:48:57.824 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:57.823+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:57.852 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:57.852 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:57.863 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:48:57.925 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:57.924+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:58.016 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:58.016 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:58.025 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:48:58.026 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:58.025+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:58.127 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:58.126+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:58.135 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #15, image image_client.0-clone15 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent16 2024-09-16T20:48:58.139 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:58.139 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:58.148 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:48:58.191 INFO:teuthology.orchestra.run.smithi139.stdout:2386 write 0x3c05400 thru 0x3c076a1 (0x22a2 bytes) 2024-09-16T20:48:58.192 INFO:teuthology.orchestra.run.smithi139.stdout:2387 write 0x269c600 thru 0x26a895f (0xc360 bytes) 2024-09-16T20:48:58.193 INFO:teuthology.orchestra.run.smithi139.stdout:2389 punch from 0x2ab5400 to 0x2abee00, (0x9a00 bytes) 2024-09-16T20:48:58.228 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:58.227+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:58.291 INFO:teuthology.orchestra.run.smithi139.stdout:2391 read 0x1ea3a00 thru 0x1eb31fe (0xf7ff bytes) 2024-09-16T20:48:58.292 INFO:teuthology.orchestra.run.smithi139.stdout:2394 write 0x1d46e00 thru 0x1d49b4e (0x2d4f bytes) 2024-09-16T20:48:58.329 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:58.328+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:58.337 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:58.337 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:58.347 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:48:58.348 INFO:teuthology.orchestra.run.smithi139.stdout:2396 write 0xe322e00 thru 0xe32c622 (0x9823 bytes) 2024-09-16T20:48:58.349 INFO:teuthology.orchestra.run.smithi139.stdout:2397 write 0x1195600 thru 0x119b7e4 (0x61e5 bytes) 2024-09-16T20:48:58.364 INFO:teuthology.orchestra.run.smithi139.stdout:2399 write 0xe674000 thru 0xe67f6f0 (0xb6f1 bytes) 2024-09-16T20:48:58.365 INFO:teuthology.orchestra.run.smithi139.stdout:2400 write 0x7ea5200 thru 0x7eab948 (0x6749 bytes) 2024-09-16T20:48:58.369 INFO:teuthology.orchestra.run.smithi139.stdout:2401 trunc from 0xe67f800 to 0xb1a0a00 2024-09-16T20:48:58.388 INFO:teuthology.orchestra.run.smithi139.stdout:2403 punch from 0x23cbc00 to 0x23d9e00, (0xe200 bytes) 2024-09-16T20:48:58.396 INFO:teuthology.orchestra.run.smithi139.stdout:2408 trunc from 0xb1a0a00 to 0x84b6400 2024-09-16T20:48:58.426 INFO:teuthology.orchestra.run.smithi139.stdout:2412 read 0x1013400 thru 0x10226bd (0xf2be bytes) 2024-09-16T20:48:58.427 INFO:teuthology.orchestra.run.smithi139.stdout:2414 punch from 0x3323600 to 0x3327a00, (0x4400 bytes) 2024-09-16T20:48:58.428 INFO:teuthology.orchestra.run.smithi139.stdout:2415 punch from 0x5e3e600 to 0x5e41e00, (0x3800 bytes) 2024-09-16T20:48:58.430 INFO:teuthology.orchestra.run.smithi139.stdout:2416 read 0x5b6be00 thru 0x5b751ec (0x93ed bytes) 2024-09-16T20:48:58.430 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:58.429+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:58.434 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T20:48:58.441 INFO:teuthology.orchestra.run.smithi139.stdout:2417 read 0x596ac00 thru 0x596f25e (0x465f bytes) 2024-09-16T20:48:58.441 INFO:teuthology.orchestra.run.smithi139.stdout:2418 write 0x7bc4600 thru 0x7bc65bc (0x1fbd bytes) 2024-09-16T20:48:58.442 INFO:teuthology.orchestra.run.smithi139.stdout:2420 write 0x8378200 thru 0x8379cc2 (0x1ac3 bytes) 2024-09-16T20:48:58.442 INFO:teuthology.orchestra.run.smithi139.stdout:2421 punch from 0x6761400 to 0x6763e00, (0x2a00 bytes) 2024-09-16T20:48:58.442 INFO:teuthology.orchestra.run.smithi139.stdout:2422 read 0x438f600 thru 0x438fd42 (0x743 bytes) 2024-09-16T20:48:58.442 INFO:teuthology.orchestra.run.smithi139.stdout:2426 punch from 0x531ee00 to 0x5328a00, (0x9c00 bytes) 2024-09-16T20:48:58.448 INFO:teuthology.orchestra.run.smithi139.stdout:2427 trunc from 0x84b6400 to 0xa627e00 2024-09-16T20:48:58.502 INFO:teuthology.orchestra.run.smithi139.stdout:2428 write 0x6f9e600 thru 0x6fa3a58 (0x5459 bytes) 2024-09-16T20:48:58.503 INFO:teuthology.orchestra.run.smithi139.stdout:2432 punch from 0x6973000 to 0x697c200, (0x9200 bytes) 2024-09-16T20:48:58.503 INFO:teuthology.orchestra.run.smithi139.stdout:2437 write 0x2e73a00 thru 0x2e78234 (0x4835 bytes) 2024-09-16T20:48:58.523 INFO:teuthology.orchestra.run.smithi139.stdout:2438 write 0xa6a1600 thru 0xa6a5c66 (0x4667 bytes) 2024-09-16T20:48:58.526 INFO:teuthology.orchestra.run.smithi139.stdout:2439 trunc from 0xa6a5e00 to 0x83b4400 2024-09-16T20:48:58.533 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:58.533 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:58.533 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:58.531+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:58.543 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:48:58.544 INFO:teuthology.orchestra.run.smithi139.stdout:2443 punch from 0x47e9a00 to 0x47f8200, (0xe800 bytes) 2024-09-16T20:48:58.546 INFO:teuthology.orchestra.run.smithi139.stdout:2445 read 0x59ce600 thru 0x59d14f8 (0x2ef9 bytes) 2024-09-16T20:48:58.581 INFO:teuthology.orchestra.run.smithi139.stdout:2446 write 0xa309200 thru 0xa30d76d (0x456e bytes) 2024-09-16T20:48:58.583 INFO:teuthology.orchestra.run.smithi139.stdout:2447 read 0x6005c00 thru 0x600d7fa (0x7bfb bytes) 2024-09-16T20:48:58.583 INFO:teuthology.orchestra.run.smithi139.stdout:2451 punch from 0x12ac00 to 0x12c000, (0x1400 bytes) 2024-09-16T20:48:58.584 INFO:teuthology.orchestra.run.smithi139.stdout:2452 read 0x9814e00 thru 0x9822895 (0xda96 bytes) 2024-09-16T20:48:58.584 INFO:teuthology.orchestra.run.smithi139.stdout:2453 write 0x46d9600 thru 0x46e9572 (0xff73 bytes) 2024-09-16T20:48:58.586 INFO:teuthology.orchestra.run.smithi139.stdout:2454 trunc from 0xa30d800 to 0x8a25c00 2024-09-16T20:48:58.598 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:48:58.598 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:48:58.600 INFO:teuthology.orchestra.run.smithi139.stdout:2456 punch from 0x7544e00 to 0x7552400, (0xd600 bytes) 2024-09-16T20:48:58.614 INFO:teuthology.orchestra.run.smithi139.stdout:2457 write 0xb06ac00 thru 0xb070d1a (0x611b bytes) 2024-09-16T20:48:58.621 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:48:58.622 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:48:58.634 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:58.632+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:58.638 INFO:teuthology.orchestra.run.smithi139.stdout:2460 write 0xda38c00 thru 0xda4209b (0x949c bytes) 2024-09-16T20:48:58.639 INFO:teuthology.orchestra.run.smithi139.stdout:2461 punch from 0x328d600 to 0x328f600, (0x2000 bytes) 2024-09-16T20:48:58.641 INFO:teuthology.orchestra.run.smithi139.stdout:2462 trunc from 0xda42200 to 0xd336800 2024-09-16T20:48:58.645 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:48:58.645 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:48:58.656 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:48:58.656 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:48:58.665 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:48:58.666 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:48:58.666 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:48:58.666 INFO:teuthology.orchestra.run.smithi139.stdout:2463 read 0xb0bba00 thru 0xb0c376f (0x7d70 bytes) 2024-09-16T20:48:58.667 INFO:teuthology.orchestra.run.smithi139.stdout:2465 punch from 0x5172800 to 0x5179a00, (0x7200 bytes) 2024-09-16T20:48:58.668 INFO:teuthology.orchestra.run.smithi139.stdout:2466 read 0x27c6600 thru 0x27cb711 (0x5112 bytes) 2024-09-16T20:48:58.669 INFO:teuthology.orchestra.run.smithi139.stdout:2468 read 0x2270200 thru 0x2270d9b (0xb9c bytes) 2024-09-16T20:48:58.669 INFO:teuthology.orchestra.run.smithi139.stdout:2470 write 0x3972400 thru 0x397e6e6 (0xc2e7 bytes) 2024-09-16T20:48:58.669 INFO:teuthology.orchestra.run.smithi139.stdout:2471 punch from 0x95d3c00 to 0x95dd000, (0x9400 bytes) 2024-09-16T20:48:58.670 INFO:teuthology.orchestra.run.smithi139.stdout:2472 write 0x912fa00 thru 0x913b6b5 (0xbcb6 bytes) 2024-09-16T20:48:58.670 INFO:teuthology.orchestra.run.smithi139.stdout:2476 read 0xbf4ea00 thru 0xbf57c8b (0x928c bytes) 2024-09-16T20:48:58.671 INFO:teuthology.orchestra.run.smithi139.stdout:2477 write 0xb61a00 thru 0xb65f16 (0x4517 bytes) 2024-09-16T20:48:58.671 INFO:teuthology.orchestra.run.smithi139.stdout:2478 read 0xb560600 thru 0xb56e0d7 (0xdad8 bytes) 2024-09-16T20:48:58.672 INFO:teuthology.orchestra.run.smithi139.stdout:2481 write 0xcbcc400 thru 0xcbd2988 (0x6589 bytes) 2024-09-16T20:48:58.672 INFO:teuthology.orchestra.run.smithi139.stdout:2482 read 0x4c7ea00 thru 0x4c8062d (0x1c2e bytes) 2024-09-16T20:48:58.673 INFO:teuthology.orchestra.run.smithi139.stdout:2484 read 0x59ba000 thru 0x59c65fa (0xc5fb bytes) 2024-09-16T20:48:58.674 INFO:teuthology.orchestra.run.smithi139.stdout:2485 read 0xc83b400 thru 0xc83c7f4 (0x13f5 bytes) 2024-09-16T20:48:58.675 INFO:teuthology.orchestra.run.smithi139.stdout:2486 punch from 0x6368600 to 0x636cc00, (0x4600 bytes) 2024-09-16T20:48:58.675 INFO:teuthology.orchestra.run.smithi139.stdout:2487 punch from 0xfaaa00 to 0xfacc00, (0x2200 bytes) 2024-09-16T20:48:58.675 INFO:teuthology.orchestra.run.smithi139.stdout:2489 write 0x707e000 thru 0x708c38b (0xe38c bytes) 2024-09-16T20:48:58.675 INFO:teuthology.orchestra.run.smithi139.stdout:2490 read 0x7d12a00 thru 0x7d19d40 (0x7341 bytes) 2024-09-16T20:48:58.678 INFO:teuthology.orchestra.run.smithi139.stdout:2491 read 0x2ebc600 thru 0x2ebe1ae (0x1baf bytes) 2024-09-16T20:48:58.678 INFO:teuthology.orchestra.run.smithi139.stdout:2494 write 0x2b70200 thru 0x2b769af (0x67b0 bytes) 2024-09-16T20:48:58.679 INFO:teuthology.orchestra.run.smithi139.stdout:2498 read 0x3b46a00 thru 0x3b4d60b (0x6c0c bytes) 2024-09-16T20:48:58.679 INFO:teuthology.orchestra.run.smithi139.stdout:2499 read 0x359fe00 thru 0x35a82d8 (0x84d9 bytes) 2024-09-16T20:48:58.679 INFO:teuthology.orchestra.run.smithi139.stdout:2500 write 0xb0a8200 thru 0xb0acbeb (0x49ec bytes) 2024-09-16T20:48:58.680 INFO:teuthology.orchestra.run.smithi139.stdout:2503 write 0x75e0000 thru 0x75ed41e (0xd41f bytes) 2024-09-16T20:48:58.680 INFO:teuthology.orchestra.run.smithi139.stdout:2504 read 0x23d6e00 thru 0x23dd0eb (0x62ec bytes) 2024-09-16T20:48:58.690 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:48:58.690 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:48:58.692 INFO:teuthology.orchestra.run.smithi139.stdout:2505 write 0xd4f1a00 thru 0xd4fbde0 (0xa3e1 bytes) 2024-09-16T20:48:58.693 INFO:teuthology.orchestra.run.smithi139.stdout:2506 punch from 0x67ef400 to 0x67f3600, (0x4200 bytes) 2024-09-16T20:48:58.708 INFO:teuthology.orchestra.run.smithi139.stdout:2511 trunc from 0xd4fbe00 to 0x281bc00 2024-09-16T20:48:58.711 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:48:58.711 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:48:58.746 INFO:teuthology.orchestra.run.smithi029.stderr:nodeep-scrub is set 2024-09-16T20:48:58.750 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:58.749+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:58.763 INFO:teuthology.orchestra.run.smithi139.stdout:2512 trunc from 0x281bc00 to 0xe8c6000 2024-09-16T20:48:58.772 INFO:teuthology.orchestra.run.smithi139.stdout:2513 write 0xa24c800 thru 0xa24c94d (0x14e bytes) 2024-09-16T20:48:58.778 INFO:teuthology.orchestra.run.smithi139.stdout:2515 write 0x63f8a00 thru 0x63f9674 (0xc75 bytes) 2024-09-16T20:48:58.778 INFO:teuthology.orchestra.run.smithi139.stdout:2516 read 0x379e600 thru 0x37a4dc3 (0x67c4 bytes) 2024-09-16T20:48:58.779 INFO:teuthology.orchestra.run.smithi139.stdout:2519 write 0x7c8de00 thru 0x7c9599a (0x7b9b bytes) 2024-09-16T20:48:58.779 INFO:teuthology.orchestra.run.smithi139.stdout:2520 punch from 0x367e400 to 0x3682800, (0x4400 bytes) 2024-09-16T20:48:58.780 INFO:teuthology.orchestra.run.smithi139.stdout:2521 punch from 0x219800 to 0x223000, (0x9800 bytes) 2024-09-16T20:48:58.780 INFO:teuthology.orchestra.run.smithi139.stdout:skipping zero length punch hole 2024-09-16T20:48:58.780 INFO:teuthology.orchestra.run.smithi139.stdout:2523 read 0x489c400 thru 0x489d27b (0xe7c bytes) 2024-09-16T20:48:58.780 INFO:teuthology.orchestra.run.smithi139.stdout:2524 clone 18 order 25 su 1048576 sc 2 2024-09-16T20:48:58.821 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:58.821 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:48:58.821 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:48:58.821 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:48:58.821 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:58.829 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:48:58.871 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:48:58.870+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:48:58.941 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:58.941+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:58.978 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:58.978 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:48:58.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:48:58.978 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:48:58.978 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:58.978 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6414 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3569664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:58.978 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871708+0000", 2024-09-16T20:48:58.978 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 39.104600712, 2024-09-16T20:48:58.979 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048373317999999998, 2024-09-16T20:48:58.979 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:58.979 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:58.979 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:58.979 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:58.979 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:58.979 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:58.979 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6414 2024-09-16T20:48:58.979 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:58.979 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:58.979 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:58.979 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:58.979 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871708+0000", 2024-09-16T20:48:58.979 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:58.979 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:58.980 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:58.980 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:58.980 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871708+0000", 2024-09-16T20:48:58.980 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:58.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:58.980 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:58.980 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:58.980 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871708+0000", 2024-09-16T20:48:58.980 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9799999999999999e-07 2024-09-16T20:48:58.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:58.980 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:58.980 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:58.980 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871710+0000", 2024-09-16T20:48:58.980 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2699999999999999e-06 2024-09-16T20:48:58.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:58.981 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:58.981 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:58.981 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871718+0000", 2024-09-16T20:48:58.981 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8499999999999994e-06 2024-09-16T20:48:58.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:58.981 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:58.981 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:58.981 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871723+0000", 2024-09-16T20:48:58.981 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1819999999999997e-06 2024-09-16T20:48:58.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:58.981 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:58.981 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:58.981 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.919883+0000", 2024-09-16T20:48:58.981 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048159955999999997 2024-09-16T20:48:58.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:58.982 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:58.982 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:58.982 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.919969+0000", 2024-09-16T20:48:58.982 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5808000000000002e-05 2024-09-16T20:48:58.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:58.982 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:58.982 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:58.982 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920081+0000", 2024-09-16T20:48:58.982 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000111854 2024-09-16T20:48:58.982 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:58.982 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:58.982 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:58.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:58.983 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:58.983 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6415 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3577856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:58.983 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871730+0000", 2024-09-16T20:48:58.983 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 39.104578197000002, 2024-09-16T20:48:58.983 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.04851482, 2024-09-16T20:48:58.983 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:58.983 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:58.983 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:58.983 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:58.983 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:58.983 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:58.983 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6415 2024-09-16T20:48:58.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:58.983 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:58.984 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:58.984 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:58.984 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871730+0000", 2024-09-16T20:48:58.984 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:58.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:58.984 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:58.984 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:58.984 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871730+0000", 2024-09-16T20:48:58.984 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:58.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:58.984 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:58.984 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:58.984 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871731+0000", 2024-09-16T20:48:58.984 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6899999999999998e-07 2024-09-16T20:48:58.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:58.985 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:58.985 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:58.985 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871733+0000", 2024-09-16T20:48:58.985 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0600000000000002e-06 2024-09-16T20:48:58.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:58.985 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:58.985 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:58.985 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871740+0000", 2024-09-16T20:48:58.985 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3749999999999997e-06 2024-09-16T20:48:58.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:58.985 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:58.985 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:58.986 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871749+0000", 2024-09-16T20:48:58.986 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9109999999999999e-06 2024-09-16T20:48:58.986 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:58.986 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:58.986 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:58.986 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920129+0000", 2024-09-16T20:48:58.986 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048380175999999997 2024-09-16T20:48:58.986 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:58.986 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:58.986 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:58.986 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920181+0000", 2024-09-16T20:48:58.986 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2077000000000003e-05 2024-09-16T20:48:58.986 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:58.986 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:58.986 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:58.987 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920245+0000", 2024-09-16T20:48:58.987 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3851999999999998e-05 2024-09-16T20:48:58.987 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:58.987 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:58.987 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:58.987 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:58.987 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:58.987 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6417 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3586048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:58.987 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871756+0000", 2024-09-16T20:48:58.987 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 39.104552562000002, 2024-09-16T20:48:58.987 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048628301999999998, 2024-09-16T20:48:58.987 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:58.987 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:58.988 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:58.988 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:58.988 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:58.988 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:58.988 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6417 2024-09-16T20:48:58.988 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:58.988 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:58.988 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:58.988 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:58.988 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871756+0000", 2024-09-16T20:48:58.988 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:58.988 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:58.988 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:58.989 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:58.989 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871756+0000", 2024-09-16T20:48:58.989 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:58.989 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:58.989 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:58.989 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:58.989 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871756+0000", 2024-09-16T20:48:58.989 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0100000000000005e-07 2024-09-16T20:48:58.989 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:58.989 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:58.989 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:58.989 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871758+0000", 2024-09-16T20:48:58.989 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.902e-06 2024-09-16T20:48:58.989 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:58.990 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:58.990 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:58.990 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871764+0000", 2024-09-16T20:48:58.990 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8170000000000004e-06 2024-09-16T20:48:58.990 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:58.990 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:58.990 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:58.990 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871769+0000", 2024-09-16T20:48:58.990 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6500000000000004e-06 2024-09-16T20:48:58.990 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:58.990 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:58.990 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:58.990 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920276+0000", 2024-09-16T20:48:58.990 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048507794999999999 2024-09-16T20:48:58.991 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:58.991 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:58.991 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:58.991 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920329+0000", 2024-09-16T20:48:58.991 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2534999999999998e-05 2024-09-16T20:48:58.991 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:58.991 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:58.991 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:58.991 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920384+0000", 2024-09-16T20:48:58.991 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5102000000000002e-05 2024-09-16T20:48:58.991 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:58.991 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:58.991 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:58.991 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:58.992 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:58.992 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6419 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3594240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:58.992 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871775+0000", 2024-09-16T20:48:58.992 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 39.104533385000003, 2024-09-16T20:48:58.992 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048754724999999999, 2024-09-16T20:48:58.992 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:58.992 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:58.992 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:58.992 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:58.992 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:58.992 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:58.992 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6419 2024-09-16T20:48:58.992 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:58.993 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:58.993 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:58.993 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:58.993 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871775+0000", 2024-09-16T20:48:58.993 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:58.993 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:58.993 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:58.993 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:58.993 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871775+0000", 2024-09-16T20:48:58.993 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:58.993 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:58.993 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:58.993 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:58.993 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871775+0000", 2024-09-16T20:48:58.993 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2500000000000001e-07 2024-09-16T20:48:58.994 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:58.994 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:58.994 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:58.994 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871777+0000", 2024-09-16T20:48:58.994 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9649999999999998e-06 2024-09-16T20:48:58.994 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:58.994 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:58.994 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:58.994 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871784+0000", 2024-09-16T20:48:58.994 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3419999999999996e-06 2024-09-16T20:48:58.994 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:58.994 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:58.994 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:58.994 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871789+0000", 2024-09-16T20:48:58.995 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1889999999999998e-06 2024-09-16T20:48:58.995 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:58.995 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:58.995 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:58.995 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920416+0000", 2024-09-16T20:48:58.995 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048627363999999999 2024-09-16T20:48:58.995 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:58.995 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:58.995 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:58.995 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920472+0000", 2024-09-16T20:48:58.995 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5429000000000001e-05 2024-09-16T20:48:58.995 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:58.995 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:58.996 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:58.996 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920530+0000", 2024-09-16T20:48:58.996 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8111e-05 2024-09-16T20:48:58.996 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:58.996 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:58.996 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:58.996 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:58.996 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:58.996 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6420 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3602432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:58.996 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871796+0000", 2024-09-16T20:48:58.996 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 39.104512472000003, 2024-09-16T20:48:58.996 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048857298, 2024-09-16T20:48:58.996 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:58.996 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:58.997 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:58.997 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:58.997 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:58.997 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:58.997 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6420 2024-09-16T20:48:58.997 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:58.997 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:58.997 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:58.997 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:58.997 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871796+0000", 2024-09-16T20:48:58.997 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:58.997 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:58.997 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:58.998 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:58.998 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871796+0000", 2024-09-16T20:48:58.998 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:58.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:58.998 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:58.998 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:58.998 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871854+0000", 2024-09-16T20:48:58.998 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8187999999999999e-05 2024-09-16T20:48:58.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:58.998 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:58.998 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:58.998 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871856+0000", 2024-09-16T20:48:58.998 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3549999999999999e-06 2024-09-16T20:48:58.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:58.998 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:58.999 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:58.999 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871866+0000", 2024-09-16T20:48:58.999 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6500000000000008e-06 2024-09-16T20:48:58.999 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:58.999 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:58.999 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:58.999 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871872+0000", 2024-09-16T20:48:58.999 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3799999999999999e-06 2024-09-16T20:48:58.999 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:58.999 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:58.999 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:58.999 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920558+0000", 2024-09-16T20:48:58.999 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048685797000000003 2024-09-16T20:48:59.000 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.000 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.000 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:59.000 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920603+0000", 2024-09-16T20:48:59.000 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4629999999999998e-05 2024-09-16T20:48:59.000 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.000 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.000 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:59.000 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920653+0000", 2024-09-16T20:48:59.000 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0297999999999999e-05 2024-09-16T20:48:59.000 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.000 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:59.000 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.000 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.000 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.001 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6421 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3610624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:59.001 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871882+0000", 2024-09-16T20:48:59.001 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 39.104426404000002, 2024-09-16T20:48:59.001 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048913381999999998, 2024-09-16T20:48:59.001 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:59.001 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:59.001 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:59.001 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:59.001 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:59.001 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:59.001 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6421 2024-09-16T20:48:59.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.001 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:59.001 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.001 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:59.002 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871882+0000", 2024-09-16T20:48:59.002 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.002 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:59.002 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871882+0000", 2024-09-16T20:48:59.002 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.002 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:59.002 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.873257+0000", 2024-09-16T20:48:59.002 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001374789 2024-09-16T20:48:59.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.002 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:59.003 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.873261+0000", 2024-09-16T20:48:59.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5589999999999999e-06 2024-09-16T20:48:59.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.003 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.003 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:59.003 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.873275+0000", 2024-09-16T20:48:59.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3675999999999999e-05 2024-09-16T20:48:59.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.003 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.003 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:59.003 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.873281+0000", 2024-09-16T20:48:59.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6930000000000004e-06 2024-09-16T20:48:59.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.003 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.003 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:59.004 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920684+0000", 2024-09-16T20:48:59.004 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047402913999999997 2024-09-16T20:48:59.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.004 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.004 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:59.004 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920737+0000", 2024-09-16T20:48:59.004 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3600000000000002e-05 2024-09-16T20:48:59.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.004 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.004 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:59.004 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920795+0000", 2024-09-16T20:48:59.004 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8150999999999999e-05 2024-09-16T20:48:59.004 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.005 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:59.005 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.005 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.005 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.005 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:7491 3.1e 3:7c48f6ae:::rbd_object_map.110e891a9b2b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:59.005 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.925865+0000", 2024-09-16T20:48:59.005 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 39.050443260000002, 2024-09-16T20:48:59.005 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048897293000000001, 2024-09-16T20:48:59.005 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:59.005 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:59.005 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:59.005 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:59.005 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:59.005 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:59.006 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 7491 2024-09-16T20:48:59.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.006 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:59.006 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.006 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:59.006 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925865+0000", 2024-09-16T20:48:59.006 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.006 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.006 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:59.006 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925865+0000", 2024-09-16T20:48:59.006 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.006 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.007 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:59.007 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925866+0000", 2024-09-16T20:48:59.007 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.39e-07 2024-09-16T20:48:59.007 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.007 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.007 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:59.007 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925868+0000", 2024-09-16T20:48:59.007 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.508e-06 2024-09-16T20:48:59.007 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.007 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.007 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:59.007 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925874+0000", 2024-09-16T20:48:59.007 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7919999999999998e-06 2024-09-16T20:48:59.007 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.008 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:59.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925879+0000", 2024-09-16T20:48:59.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8579999999999999e-06 2024-09-16T20:48:59.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.008 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:59.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.970875+0000", 2024-09-16T20:48:59.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044996412 2024-09-16T20:48:59.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.008 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:59.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.970970+0000", 2024-09-16T20:48:59.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4732e-05 2024-09-16T20:48:59.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.009 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.009 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-16T20:48:59.009 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.971275+0000", 2024-09-16T20:48:59.009 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00030464199999999999 2024-09-16T20:48:59.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.009 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.009 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:59.009 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.974598+0000", 2024-09-16T20:48:59.009 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0033238159999999998 2024-09-16T20:48:59.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.009 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.009 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:48:59.010 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.974666+0000", 2024-09-16T20:48:59.010 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7617e-05 2024-09-16T20:48:59.010 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.010 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.010 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:48:59.010 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.974718+0000", 2024-09-16T20:48:59.010 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2259000000000002e-05 2024-09-16T20:48:59.010 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.010 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.010 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:59.010 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.974762+0000", 2024-09-16T20:48:59.010 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4218000000000003e-05 2024-09-16T20:48:59.010 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.010 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:59.011 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.011 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.011 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4754 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1505280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:59.011 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689097+0000", 2024-09-16T20:48:59.011 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.287211061999997, 2024-09-16T20:48:59.011 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048429947000000001, 2024-09-16T20:48:59.011 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:59.011 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:59.011 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:59.011 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:59.011 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:59.011 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:59.011 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4754 2024-09-16T20:48:59.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.012 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:59.012 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.012 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:59.012 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689097+0000", 2024-09-16T20:48:59.012 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.012 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.012 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:59.012 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689097+0000", 2024-09-16T20:48:59.012 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.012 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.012 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:59.012 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689098+0000", 2024-09-16T20:48:59.013 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.15e-07 2024-09-16T20:48:59.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.013 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.013 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:59.013 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689100+0000", 2024-09-16T20:48:59.013 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8190000000000001e-06 2024-09-16T20:48:59.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.013 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.013 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:59.013 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689106+0000", 2024-09-16T20:48:59.013 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2489999999999998e-06 2024-09-16T20:48:59.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.013 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.014 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:59.014 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689111+0000", 2024-09-16T20:48:59.014 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3179999999999997e-06 2024-09-16T20:48:59.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.014 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.014 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:59.014 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737428+0000", 2024-09-16T20:48:59.014 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048316572000000002 2024-09-16T20:48:59.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.014 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.014 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:59.014 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737475+0000", 2024-09-16T20:48:59.014 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7392000000000003e-05 2024-09-16T20:48:59.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.015 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.015 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:59.015 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737527+0000", 2024-09-16T20:48:59.015 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2182000000000003e-05 2024-09-16T20:48:59.015 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.015 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:59.015 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.015 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.015 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4756 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1513472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:59.015 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689118+0000", 2024-09-16T20:48:59.015 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.287190840999997, 2024-09-16T20:48:59.015 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048535146000000001, 2024-09-16T20:48:59.015 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:59.016 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:59.016 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:59.016 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:59.016 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:59.016 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:59.016 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4756 2024-09-16T20:48:59.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.016 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:59.016 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.016 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:59.016 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689118+0000", 2024-09-16T20:48:59.016 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.016 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.016 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:59.017 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689118+0000", 2024-09-16T20:48:59.017 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.017 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.017 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.017 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:59.017 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689118+0000", 2024-09-16T20:48:59.017 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7099999999999997e-07 2024-09-16T20:48:59.017 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.017 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.017 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:59.017 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689120+0000", 2024-09-16T20:48:59.017 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.787e-06 2024-09-16T20:48:59.017 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.017 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.017 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:59.018 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689126+0000", 2024-09-16T20:48:59.018 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2720000000000003e-06 2024-09-16T20:48:59.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.018 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.018 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:59.018 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689134+0000", 2024-09-16T20:48:59.018 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2190000000000004e-06 2024-09-16T20:48:59.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.018 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.018 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:59.018 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737553+0000", 2024-09-16T20:48:59.018 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048418993 2024-09-16T20:48:59.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.018 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.018 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:59.019 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737598+0000", 2024-09-16T20:48:59.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4743999999999997e-05 2024-09-16T20:48:59.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.019 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:59.019 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737653+0000", 2024-09-16T20:48:59.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4759999999999997e-05 2024-09-16T20:48:59.019 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.019 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:59.019 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.019 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4758 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1521664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:59.019 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689141+0000", 2024-09-16T20:48:59.020 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.287167861, 2024-09-16T20:48:59.020 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048640741000000001, 2024-09-16T20:48:59.020 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:59.020 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:59.020 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:59.020 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:59.020 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:59.020 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:59.020 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4758 2024-09-16T20:48:59.020 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.020 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:59.020 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.020 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:59.020 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689141+0000", 2024-09-16T20:48:59.021 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.021 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.021 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:59.021 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689141+0000", 2024-09-16T20:48:59.021 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.021 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.021 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:59.021 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689141+0000", 2024-09-16T20:48:59.021 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8799999999999998e-07 2024-09-16T20:48:59.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.021 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.021 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:59.022 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689143+0000", 2024-09-16T20:48:59.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.945e-06 2024-09-16T20:48:59.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.022 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.022 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:59.022 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689149+0000", 2024-09-16T20:48:59.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9800000000000003e-06 2024-09-16T20:48:59.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.022 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.022 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:59.022 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689155+0000", 2024-09-16T20:48:59.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7509999999999997e-06 2024-09-16T20:48:59.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.023 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.023 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:59.023 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737681+0000", 2024-09-16T20:48:59.023 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048526825000000003 2024-09-16T20:48:59.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.023 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.023 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:59.023 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737729+0000", 2024-09-16T20:48:59.023 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7219999999999999e-05 2024-09-16T20:48:59.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.023 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.023 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:59.023 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737781+0000", 2024-09-16T20:48:59.023 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2632e-05 2024-09-16T20:48:59.023 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.024 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:59.024 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.024 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4760 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1529856~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:59.024 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689161+0000", 2024-09-16T20:48:59.024 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.287147513999997, 2024-09-16T20:48:59.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048743981999999998, 2024-09-16T20:48:59.024 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:59.024 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:59.024 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:59.024 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:59.024 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:59.024 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:59.024 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4760 2024-09-16T20:48:59.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.025 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:59.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:59.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689161+0000", 2024-09-16T20:48:59.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:59.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689161+0000", 2024-09-16T20:48:59.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:59.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689161+0000", 2024-09-16T20:48:59.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5599999999999995e-07 2024-09-16T20:48:59.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.026 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:59.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689163+0000", 2024-09-16T20:48:59.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.818e-06 2024-09-16T20:48:59.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.026 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:59.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689169+0000", 2024-09-16T20:48:59.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0809999999999999e-06 2024-09-16T20:48:59.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.027 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:59.027 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689174+0000", 2024-09-16T20:48:59.027 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.938e-06 2024-09-16T20:48:59.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.027 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.027 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:59.027 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737807+0000", 2024-09-16T20:48:59.027 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048633124 2024-09-16T20:48:59.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.027 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.027 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:59.027 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737851+0000", 2024-09-16T20:48:59.027 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3764000000000003e-05 2024-09-16T20:48:59.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.028 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.028 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:59.028 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737905+0000", 2024-09-16T20:48:59.028 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3701e-05 2024-09-16T20:48:59.028 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.028 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:59.028 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.028 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.028 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4762 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1538048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:59.028 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689182+0000", 2024-09-16T20:48:59.028 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.287126292000004, 2024-09-16T20:48:59.028 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048850275999999998, 2024-09-16T20:48:59.028 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:59.029 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:59.029 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:59.029 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:59.029 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:59.029 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:59.029 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4762 2024-09-16T20:48:59.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.029 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:59.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.029 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:59.029 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689182+0000", 2024-09-16T20:48:59.029 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.030 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:59.030 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689182+0000", 2024-09-16T20:48:59.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.030 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:59.030 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689182+0000", 2024-09-16T20:48:59.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-16T20:48:59.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.030 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:59.030 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689184+0000", 2024-09-16T20:48:59.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8190000000000001e-06 2024-09-16T20:48:59.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.031 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.031 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:59.031 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689191+0000", 2024-09-16T20:48:59.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2530000000000001e-06 2024-09-16T20:48:59.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.031 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.031 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:59.031 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689195+0000", 2024-09-16T20:48:59.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9559999999999996e-06 2024-09-16T20:48:59.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.031 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.031 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:59.031 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737933+0000", 2024-09-16T20:48:59.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048737919999999997 2024-09-16T20:48:59.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.032 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:59.032 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737982+0000", 2024-09-16T20:48:59.032 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8089e-05 2024-09-16T20:48:59.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.032 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:59.032 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738032+0000", 2024-09-16T20:48:59.032 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0868999999999997e-05 2024-09-16T20:48:59.032 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.032 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:59.032 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.033 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.033 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4764 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1546240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:59.033 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689201+0000", 2024-09-16T20:48:59.033 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.287106934999997, 2024-09-16T20:48:59.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048957550000000002, 2024-09-16T20:48:59.033 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:59.033 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:59.033 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:59.033 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:59.033 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:59.033 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:59.033 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4764 2024-09-16T20:48:59.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.033 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:59.034 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.034 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:59.034 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689201+0000", 2024-09-16T20:48:59.034 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.034 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.034 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:59.034 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689201+0000", 2024-09-16T20:48:59.034 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.034 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.034 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:59.034 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689202+0000", 2024-09-16T20:48:59.034 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4200000000000002e-07 2024-09-16T20:48:59.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.035 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:59.035 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689204+0000", 2024-09-16T20:48:59.035 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9750000000000001e-06 2024-09-16T20:48:59.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.035 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:59.035 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689210+0000", 2024-09-16T20:48:59.035 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1e-06 2024-09-16T20:48:59.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.035 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:59.035 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689215+0000", 2024-09-16T20:48:59.035 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0180000000000001e-06 2024-09-16T20:48:59.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.036 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:59.036 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738058+0000", 2024-09-16T20:48:59.036 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048843573000000001 2024-09-16T20:48:59.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.036 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:59.036 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738102+0000", 2024-09-16T20:48:59.036 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3256000000000001e-05 2024-09-16T20:48:59.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.036 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:59.036 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738159+0000", 2024-09-16T20:48:59.036 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7286e-05 2024-09-16T20:48:59.037 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.037 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:59.037 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.037 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.037 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4766 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1554432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:59.037 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689221+0000", 2024-09-16T20:48:59.037 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.287086969000001, 2024-09-16T20:48:59.037 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049064996, 2024-09-16T20:48:59.037 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:59.037 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:59.037 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:59.037 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:59.038 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:59.038 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:59.038 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4766 2024-09-16T20:48:59.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.038 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:59.038 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.038 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:59.038 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689221+0000", 2024-09-16T20:48:59.038 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.038 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.038 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:59.038 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689221+0000", 2024-09-16T20:48:59.038 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.039 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:59.039 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689222+0000", 2024-09-16T20:48:59.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.34e-07 2024-09-16T20:48:59.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.039 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:59.039 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689224+0000", 2024-09-16T20:48:59.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.846e-06 2024-09-16T20:48:59.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.039 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:59.039 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689247+0000", 2024-09-16T20:48:59.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3448e-05 2024-09-16T20:48:59.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.040 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.040 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:59.040 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689254+0000", 2024-09-16T20:48:59.040 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6969999999999996e-06 2024-09-16T20:48:59.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.040 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.040 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:59.040 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738188+0000", 2024-09-16T20:48:59.040 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048934520000000002 2024-09-16T20:48:59.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.040 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.040 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:59.040 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738234+0000", 2024-09-16T20:48:59.041 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5695999999999998e-05 2024-09-16T20:48:59.041 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.041 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.041 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:59.041 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738286+0000", 2024-09-16T20:48:59.041 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2355000000000003e-05 2024-09-16T20:48:59.041 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.041 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:59.041 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.041 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.041 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.041 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4768 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1562624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:59.041 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689262+0000", 2024-09-16T20:48:59.042 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.287046015000001, 2024-09-16T20:48:59.042 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049157157999999999, 2024-09-16T20:48:59.042 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:59.042 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:59.042 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:59.042 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:59.042 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:59.042 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:59.042 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4768 2024-09-16T20:48:59.042 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.042 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:59.042 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.042 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:59.042 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689262+0000", 2024-09-16T20:48:59.042 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.043 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.043 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:59.043 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689262+0000", 2024-09-16T20:48:59.043 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.043 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.043 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:59.043 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689263+0000", 2024-09-16T20:48:59.043 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6199999999999998e-07 2024-09-16T20:48:59.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.043 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.043 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:59.043 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689270+0000", 2024-09-16T20:48:59.043 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.798e-06 2024-09-16T20:48:59.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.044 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.044 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:59.044 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689277+0000", 2024-09-16T20:48:59.044 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0890000000000002e-06 2024-09-16T20:48:59.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.044 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.044 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:59.044 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689282+0000", 2024-09-16T20:48:59.044 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0470000000000001e-06 2024-09-16T20:48:59.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.044 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.044 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:59.044 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738319+0000", 2024-09-16T20:48:59.045 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049037139 2024-09-16T20:48:59.045 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.045 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.045 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:59.045 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738365+0000", 2024-09-16T20:48:59.045 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6356999999999997e-05 2024-09-16T20:48:59.045 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.045 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.045 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:59.045 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738420+0000", 2024-09-16T20:48:59.045 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4265999999999998e-05 2024-09-16T20:48:59.045 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.045 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:59.045 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.046 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.046 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.046 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4770 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1570816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:48:59.046 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689289+0000", 2024-09-16T20:48:59.046 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.287019505000004, 2024-09-16T20:48:59.046 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049253705000000002, 2024-09-16T20:48:59.046 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:59.046 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:59.046 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:59.046 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:59.046 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:48:59.046 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:48:59.046 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4770 2024-09-16T20:48:59.046 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.047 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:59.047 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.047 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:59.047 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689289+0000", 2024-09-16T20:48:59.047 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.047 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.047 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:59.047 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689289+0000", 2024-09-16T20:48:59.047 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.047 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.047 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:59.047 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689289+0000", 2024-09-16T20:48:59.048 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0999999999999999e-07 2024-09-16T20:48:59.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.048 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.048 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:59.048 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689291+0000", 2024-09-16T20:48:59.048 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0030000000000001e-06 2024-09-16T20:48:59.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.048 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.048 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:59.048 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689297+0000", 2024-09-16T20:48:59.048 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9479999999999996e-06 2024-09-16T20:48:59.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.048 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.048 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:59.049 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689302+0000", 2024-09-16T20:48:59.049 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7389999999999999e-06 2024-09-16T20:48:59.049 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.049 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.049 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:59.049 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738442+0000", 2024-09-16T20:48:59.049 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049140001000000003 2024-09-16T20:48:59.049 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.049 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.049 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:59.049 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738490+0000", 2024-09-16T20:48:59.049 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7787000000000002e-05 2024-09-16T20:48:59.049 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.049 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.049 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:59.050 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738543+0000", 2024-09-16T20:48:59.050 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2816999999999999e-05 2024-09-16T20:48:59.050 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.050 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:59.050 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.050 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.050 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.050 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:3088 3.1e 3:79e5ec01:::rbd_header.110e891a9b2b:head [watch watch cookie 140213437171488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:48:59.050 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:48:59.050 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 34.812463487999999, 2024-09-16T20:48:59.050 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.11895186200000001, 2024-09-16T20:48:59.050 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:59.050 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:59.051 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:48:59.051 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:59.051 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:48:59.051 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:48:59.051 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3088 2024-09-16T20:48:59.051 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.051 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:59.051 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.051 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:59.051 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:48:59.051 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.051 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.051 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.051 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:59.052 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:48:59.052 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.052 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.052 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:59.052 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163846+0000", 2024-09-16T20:48:59.052 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6799999999999999e-07 2024-09-16T20:48:59.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.052 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.052 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:59.052 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163850+0000", 2024-09-16T20:48:59.052 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.929e-06 2024-09-16T20:48:59.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.053 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.053 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:59.053 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163862+0000", 2024-09-16T20:48:59.053 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1972e-05 2024-09-16T20:48:59.053 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.053 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.053 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:59.053 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163867+0000", 2024-09-16T20:48:59.053 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.451e-06 2024-09-16T20:48:59.053 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.053 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.053 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:59.053 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163912+0000", 2024-09-16T20:48:59.053 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4456000000000003e-05 2024-09-16T20:48:59.054 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.054 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.054 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:59.054 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163962+0000", 2024-09-16T20:48:59.054 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0578000000000003e-05 2024-09-16T20:48:59.054 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.054 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.054 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-16T20:48:59.054 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164058+0000", 2024-09-16T20:48:59.054 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5934999999999995e-05 2024-09-16T20:48:59.054 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.054 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.054 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:48:59.054 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164570+0000", 2024-09-16T20:48:59.055 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00051241100000000005 2024-09-16T20:48:59.055 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.055 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.055 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:48:59.055 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282629+0000", 2024-09-16T20:48:59.055 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.118058986 2024-09-16T20:48:59.055 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.055 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.055 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:48:59.055 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282725+0000", 2024-09-16T20:48:59.055 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5063999999999996e-05 2024-09-16T20:48:59.055 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.055 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.055 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:59.055 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282797+0000", 2024-09-16T20:48:59.056 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2212000000000006e-05 2024-09-16T20:48:59.056 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.056 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:59.056 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.056 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.056 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.056 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:8039 3.1e 3.ac44ff9e (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:48:59.056 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:48:59.056 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 9.2686361389999998, 2024-09-16T20:48:59.056 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0053310449999999, 2024-09-16T20:48:59.056 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:59.056 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:59.056 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:48:59.056 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:59.057 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:48:59.057 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:48:59.057 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8039 2024-09-16T20:48:59.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.057 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:59.057 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.057 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:59.057 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:48:59.057 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.057 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.057 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:59.057 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:48:59.058 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.058 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.058 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.058 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:59.058 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707674+0000", 2024-09-16T20:48:59.058 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6530000000000001e-06 2024-09-16T20:48:59.058 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.058 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.058 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:59.058 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707681+0000", 2024-09-16T20:48:59.058 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9639999999999997e-06 2024-09-16T20:48:59.058 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.058 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.059 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:59.059 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707700+0000", 2024-09-16T20:48:59.059 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8674000000000001e-05 2024-09-16T20:48:59.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.059 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.059 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:59.059 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707708+0000", 2024-09-16T20:48:59.059 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6810000000000004e-06 2024-09-16T20:48:59.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.059 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.060 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:59.060 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711293+0000", 2024-09-16T20:48:59.060 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035848189999999999 2024-09-16T20:48:59.060 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.060 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.060 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:48:59.060 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711301+0000", 2024-09-16T20:48:59.060 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4730000000000003e-06 2024-09-16T20:48:59.060 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.060 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.060 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:59.060 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.712990+0000", 2024-09-16T20:48:59.060 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.001689348 2024-09-16T20:48:59.060 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.061 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.061 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:59.061 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713003+0000", 2024-09-16T20:48:59.061 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3433e-05 2024-09-16T20:48:59.061 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.061 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:59.061 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.061 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.061 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.061 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:8040 3.1e 3.ac44ff9e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:48:59.061 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:48:59.061 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 9.2685757439999996, 2024-09-16T20:48:59.062 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0053410780000001, 2024-09-16T20:48:59.062 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:59.062 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:59.062 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:48:59.062 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:59.062 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:48:59.062 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:48:59.062 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8040 2024-09-16T20:48:59.062 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.062 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:59.063 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.063 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:59.063 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:48:59.063 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.063 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.063 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:59.063 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:48:59.063 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.063 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.063 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:59.063 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:48:59.063 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8100000000000003e-07 2024-09-16T20:48:59.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.064 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:59.064 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707737+0000", 2024-09-16T20:48:59.064 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5020000000000001e-06 2024-09-16T20:48:59.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.064 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:59.064 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707745+0000", 2024-09-16T20:48:59.064 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2169999999999994e-06 2024-09-16T20:48:59.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.064 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:59.064 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707751+0000", 2024-09-16T20:48:59.065 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.263e-06 2024-09-16T20:48:59.065 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.065 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.065 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:59.065 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711336+0000", 2024-09-16T20:48:59.065 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035848360000000001 2024-09-16T20:48:59.065 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.065 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.065 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:48:59.065 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711340+0000", 2024-09-16T20:48:59.065 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1309999999999999e-06 2024-09-16T20:48:59.065 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.065 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.066 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:59.066 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713060+0000", 2024-09-16T20:48:59.066 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0017195059999999 2024-09-16T20:48:59.066 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.066 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.066 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:59.066 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713074+0000", 2024-09-16T20:48:59.066 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4042e-05 2024-09-16T20:48:59.066 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.066 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:59.066 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.066 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.066 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.066 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4927.0:4487 3.1e 3.ac44ff9e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:48:59.067 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:48:59.067 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 9.2680555420000008, 2024-09-16T20:48:59.067 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0048840880000001, 2024-09-16T20:48:59.067 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:59.067 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:59.067 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:48:59.067 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:59.067 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4927", 2024-09-16T20:48:59.067 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:48:59.067 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4487 2024-09-16T20:48:59.067 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.067 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:59.067 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.067 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:59.068 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:48:59.068 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.068 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.068 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.068 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:59.068 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:48:59.068 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.068 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.068 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.068 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:59.068 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708254+0000", 2024-09-16T20:48:59.068 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1800000000000005e-07 2024-09-16T20:48:59.068 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.069 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.069 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:59.069 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708258+0000", 2024-09-16T20:48:59.069 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8689999999999997e-06 2024-09-16T20:48:59.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.069 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.069 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:59.069 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708267+0000", 2024-09-16T20:48:59.069 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9590000000000007e-06 2024-09-16T20:48:59.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.069 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.069 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:59.069 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708274+0000", 2024-09-16T20:48:59.069 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9970000000000001e-06 2024-09-16T20:48:59.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.070 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:59.070 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711366+0000", 2024-09-16T20:48:59.070 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030919530000000001 2024-09-16T20:48:59.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.070 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:48:59.070 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711372+0000", 2024-09-16T20:48:59.070 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9410000000000004e-06 2024-09-16T20:48:59.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.070 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:59.070 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713126+0000", 2024-09-16T20:48:59.071 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0017537249999999 2024-09-16T20:48:59.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.071 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.071 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:59.071 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713137+0000", 2024-09-16T20:48:59.071 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0825999999999999e-05 2024-09-16T20:48:59.071 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.071 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:59.071 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.071 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.071 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:59.071 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:59.073 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:48:59.073 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:48:59.072+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:48:59.173 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:48:59.173+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:48:59.220 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:59.221 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:48:59.221 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:48:59.221 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:59.229 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:48:59.274 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:59.274+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:59.375 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:59.375+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:59.375 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:59.375 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:48:59.376 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:48:59.376 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:48:59.376 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:59.384 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:48:59.476 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:59.475+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:59.492 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:59.492 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:48:59.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:48:59.492 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:48:59.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.492 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:4904 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 7309312~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:59.492 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767724+0000", 2024-09-16T20:48:59.493 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 39.722409245000001, 2024-09-16T20:48:59.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050270214000000001, 2024-09-16T20:48:59.493 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:59.493 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:59.493 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:59.493 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:59.494 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:59.494 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:59.494 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4904 2024-09-16T20:48:59.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.494 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:59.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:59.495 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767724+0000", 2024-09-16T20:48:59.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:59.495 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767724+0000", 2024-09-16T20:48:59.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:59.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767724+0000", 2024-09-16T20:48:59.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1399999999999998e-07 2024-09-16T20:48:59.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:59.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767725+0000", 2024-09-16T20:48:59.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0380000000000001e-06 2024-09-16T20:48:59.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:59.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767729+0000", 2024-09-16T20:48:59.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2490000000000001e-06 2024-09-16T20:48:59.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:59.497 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767731+0000", 2024-09-16T20:48:59.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5940000000000001e-06 2024-09-16T20:48:59.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:59.497 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.817787+0000", 2024-09-16T20:48:59.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050055792000000002 2024-09-16T20:48:59.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:59.498 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.817883+0000", 2024-09-16T20:48:59.498 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5606999999999995e-05 2024-09-16T20:48:59.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.498 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:59.498 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.817994+0000", 2024-09-16T20:48:59.498 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011162000000000001 2024-09-16T20:48:59.498 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.498 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:59.498 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.498 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:4906 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 7317504~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:59.498 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767734+0000", 2024-09-16T20:48:59.499 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 39.722398982000001, 2024-09-16T20:48:59.499 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050406608999999998, 2024-09-16T20:48:59.499 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:59.499 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:59.499 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:59.499 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:59.499 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:59.499 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:59.499 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4906 2024-09-16T20:48:59.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.499 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:59.499 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.499 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:59.500 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767734+0000", 2024-09-16T20:48:59.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.500 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:59.500 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767734+0000", 2024-09-16T20:48:59.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.500 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:59.500 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767735+0000", 2024-09-16T20:48:59.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.05e-07 2024-09-16T20:48:59.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:59.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767736+0000", 2024-09-16T20:48:59.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0240000000000001e-06 2024-09-16T20:48:59.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:59.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767739+0000", 2024-09-16T20:48:59.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.067e-06 2024-09-16T20:48:59.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:59.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767742+0000", 2024-09-16T20:48:59.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.23e-06 2024-09-16T20:48:59.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.502 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:59.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818027+0000", 2024-09-16T20:48:59.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050285102999999998 2024-09-16T20:48:59.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.502 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:59.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818072+0000", 2024-09-16T20:48:59.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5129000000000001e-05 2024-09-16T20:48:59.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.502 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:59.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818141+0000", 2024-09-16T20:48:59.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8850999999999995e-05 2024-09-16T20:48:59.503 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.503 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:59.503 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.503 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:4908 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 7325696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:59.503 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767745+0000", 2024-09-16T20:48:59.503 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 39.722388586999998, 2024-09-16T20:48:59.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050654230000000001, 2024-09-16T20:48:59.503 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:59.503 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:59.503 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:59.503 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:59.504 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:59.504 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:59.504 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4908 2024-09-16T20:48:59.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.504 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:59.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.504 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:59.504 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767745+0000", 2024-09-16T20:48:59.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.504 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:59.504 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767745+0000", 2024-09-16T20:48:59.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:59.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767745+0000", 2024-09-16T20:48:59.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1e-07 2024-09-16T20:48:59.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:59.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767749+0000", 2024-09-16T20:48:59.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.532e-06 2024-09-16T20:48:59.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:59.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767753+0000", 2024-09-16T20:48:59.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4939999999999999e-06 2024-09-16T20:48:59.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:59.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767756+0000", 2024-09-16T20:48:59.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.61e-06 2024-09-16T20:48:59.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:59.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818184+0000", 2024-09-16T20:48:59.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050428615000000003 2024-09-16T20:48:59.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:59.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818276+0000", 2024-09-16T20:48:59.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2077999999999995e-05 2024-09-16T20:48:59.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.507 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.507 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:59.507 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818399+0000", 2024-09-16T20:48:59.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012269099999999999 2024-09-16T20:48:59.507 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.507 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:59.507 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.507 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.507 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:4910 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 7333888~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:59.507 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767759+0000", 2024-09-16T20:48:59.507 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 39.722374696999999, 2024-09-16T20:48:59.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050815076000000001, 2024-09-16T20:48:59.508 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:59.508 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:59.508 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:59.508 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:59.508 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:59.508 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:59.508 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4910 2024-09-16T20:48:59.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.508 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:59.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.508 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:59.508 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767759+0000", 2024-09-16T20:48:59.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:59.509 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767759+0000", 2024-09-16T20:48:59.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:59.509 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767759+0000", 2024-09-16T20:48:59.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2600000000000001e-07 2024-09-16T20:48:59.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:59.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767760+0000", 2024-09-16T20:48:59.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0640000000000001e-06 2024-09-16T20:48:59.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:59.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767763+0000", 2024-09-16T20:48:59.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.994e-06 2024-09-16T20:48:59.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:59.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767765+0000", 2024-09-16T20:48:59.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5220000000000002e-06 2024-09-16T20:48:59.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:59.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818438+0000", 2024-09-16T20:48:59.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050672602999999997 2024-09-16T20:48:59.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:59.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818481+0000", 2024-09-16T20:48:59.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2967999999999999e-05 2024-09-16T20:48:59.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:59.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818574+0000", 2024-09-16T20:48:59.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2699000000000001e-05 2024-09-16T20:48:59.511 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.512 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:59.512 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.512 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.512 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5012 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2101248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:59.512 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767768+0000", 2024-09-16T20:48:59.512 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 39.722365048, 2024-09-16T20:48:59.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051084682999999999, 2024-09-16T20:48:59.512 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:59.512 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:59.512 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:59.512 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:59.512 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:59.512 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:59.513 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5012 2024-09-16T20:48:59.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.513 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:59.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.513 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:59.513 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767768+0000", 2024-09-16T20:48:59.513 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.513 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:59.513 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767768+0000", 2024-09-16T20:48:59.513 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:59.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767769+0000", 2024-09-16T20:48:59.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1500000000000001e-07 2024-09-16T20:48:59.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:59.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767770+0000", 2024-09-16T20:48:59.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.015e-06 2024-09-16T20:48:59.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:59.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767773+0000", 2024-09-16T20:48:59.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1070000000000001e-06 2024-09-16T20:48:59.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.515 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.515 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:59.515 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767775+0000", 2024-09-16T20:48:59.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.508e-06 2024-09-16T20:48:59.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.515 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.515 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:59.515 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818630+0000", 2024-09-16T20:48:59.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050855282000000002 2024-09-16T20:48:59.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.515 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.515 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:59.515 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818732+0000", 2024-09-16T20:48:59.516 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101531 2024-09-16T20:48:59.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.516 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.516 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:59.516 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818853+0000", 2024-09-16T20:48:59.516 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000121025 2024-09-16T20:48:59.516 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.516 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:59.516 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.516 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.516 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5014 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2109440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:59.516 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767778+0000", 2024-09-16T20:48:59.516 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 39.722355112000002, 2024-09-16T20:48:59.517 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051260776000000001, 2024-09-16T20:48:59.517 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:59.517 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:59.517 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:59.517 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:59.517 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:59.517 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:59.517 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5014 2024-09-16T20:48:59.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.517 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:59.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.517 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:59.517 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767778+0000", 2024-09-16T20:48:59.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.518 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:59.518 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767778+0000", 2024-09-16T20:48:59.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.518 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:59.518 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767824+0000", 2024-09-16T20:48:59.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5890999999999998e-05 2024-09-16T20:48:59.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.518 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:59.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767827+0000", 2024-09-16T20:48:59.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.644e-06 2024-09-16T20:48:59.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.519 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:59.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767832+0000", 2024-09-16T20:48:59.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2689999999999999e-06 2024-09-16T20:48:59.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.519 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:59.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767835+0000", 2024-09-16T20:48:59.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2100000000000002e-06 2024-09-16T20:48:59.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.519 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:59.520 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818887+0000", 2024-09-16T20:48:59.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051051276999999999 2024-09-16T20:48:59.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.520 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:59.520 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818953+0000", 2024-09-16T20:48:59.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6105999999999994e-05 2024-09-16T20:48:59.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.520 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:59.520 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819039+0000", 2024-09-16T20:48:59.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6378999999999994e-05 2024-09-16T20:48:59.520 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.521 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:59.521 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.521 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.521 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5016 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2117632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:59.521 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767841+0000", 2024-09-16T20:48:59.521 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 39.722292433, 2024-09-16T20:48:59.521 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051433288000000001, 2024-09-16T20:48:59.521 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:59.521 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:59.521 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:59.521 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:59.521 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:59.521 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:59.522 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5016 2024-09-16T20:48:59.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.522 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:59.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.522 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:59.522 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767841+0000", 2024-09-16T20:48:59.522 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.522 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:59.522 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767841+0000", 2024-09-16T20:48:59.522 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.522 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:59.523 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768156+0000", 2024-09-16T20:48:59.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00031505699999999999 2024-09-16T20:48:59.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:59.523 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768159+0000", 2024-09-16T20:48:59.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1659999999999998e-06 2024-09-16T20:48:59.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:59.523 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768166+0000", 2024-09-16T20:48:59.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0550000000000002e-06 2024-09-16T20:48:59.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.524 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:59.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768171+0000", 2024-09-16T20:48:59.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3170000000000003e-06 2024-09-16T20:48:59.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.524 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:59.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819084+0000", 2024-09-16T20:48:59.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050913355 2024-09-16T20:48:59.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.524 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:59.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819170+0000", 2024-09-16T20:48:59.525 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6328000000000004e-05 2024-09-16T20:48:59.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.525 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.525 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:59.525 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819274+0000", 2024-09-16T20:48:59.525 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010401 2024-09-16T20:48:59.525 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.525 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:59.525 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.525 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.525 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5018 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2125824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:59.525 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768178+0000", 2024-09-16T20:48:59.525 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 39.721954898999996, 2024-09-16T20:48:59.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051227013000000002, 2024-09-16T20:48:59.526 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:59.526 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:59.526 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:59.526 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:59.526 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:59.526 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:59.526 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5018 2024-09-16T20:48:59.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.526 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:59.526 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.526 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:59.526 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768178+0000", 2024-09-16T20:48:59.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.527 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:59.527 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768178+0000", 2024-09-16T20:48:59.527 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.527 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:59.527 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768179+0000", 2024-09-16T20:48:59.527 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9200000000000002e-07 2024-09-16T20:48:59.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.527 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:59.527 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768180+0000", 2024-09-16T20:48:59.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3790000000000001e-06 2024-09-16T20:48:59.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:59.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768184+0000", 2024-09-16T20:48:59.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5470000000000001e-06 2024-09-16T20:48:59.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:59.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768187+0000", 2024-09-16T20:48:59.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8119999999999999e-06 2024-09-16T20:48:59.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:59.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819305+0000", 2024-09-16T20:48:59.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051118683999999998 2024-09-16T20:48:59.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.529 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:59.529 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819351+0000", 2024-09-16T20:48:59.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5757999999999998e-05 2024-09-16T20:48:59.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.529 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:59.529 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819405+0000", 2024-09-16T20:48:59.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4441000000000003e-05 2024-09-16T20:48:59.529 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.529 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:59.530 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.530 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5020 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2134016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:59.530 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768190+0000", 2024-09-16T20:48:59.530 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 39.721943220999997, 2024-09-16T20:48:59.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051449661000000001, 2024-09-16T20:48:59.530 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:59.530 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:59.530 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:59.530 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:59.530 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:59.530 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:59.530 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5020 2024-09-16T20:48:59.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.531 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:59.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:59.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768190+0000", 2024-09-16T20:48:59.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:59.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768190+0000", 2024-09-16T20:48:59.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:59.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768282+0000", 2024-09-16T20:48:59.532 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2263999999999996e-05 2024-09-16T20:48:59.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.532 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:59.532 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768286+0000", 2024-09-16T20:48:59.532 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.529e-06 2024-09-16T20:48:59.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.532 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:59.532 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768294+0000", 2024-09-16T20:48:59.532 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6990000000000003e-06 2024-09-16T20:48:59.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.532 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:59.533 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768306+0000", 2024-09-16T20:48:59.533 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2112e-05 2024-09-16T20:48:59.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.533 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.533 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:59.533 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819449+0000", 2024-09-16T20:48:59.533 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051143521999999997 2024-09-16T20:48:59.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.533 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.533 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:59.533 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819539+0000", 2024-09-16T20:48:59.533 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0022e-05 2024-09-16T20:48:59.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.533 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.534 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:59.534 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819640+0000", 2024-09-16T20:48:59.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000100513 2024-09-16T20:48:59.534 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.534 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:59.534 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.534 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5022 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2142208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:59.534 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768326+0000", 2024-09-16T20:48:59.534 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 39.721807832000003, 2024-09-16T20:48:59.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051436952000000001, 2024-09-16T20:48:59.534 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:59.534 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:59.535 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:59.535 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:59.535 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:59.535 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:59.535 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5022 2024-09-16T20:48:59.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.535 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:59.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:59.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768326+0000", 2024-09-16T20:48:59.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:59.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768326+0000", 2024-09-16T20:48:59.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.536 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:59.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768477+0000", 2024-09-16T20:48:59.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015109300000000001 2024-09-16T20:48:59.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.536 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:59.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768480+0000", 2024-09-16T20:48:59.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8580000000000002e-06 2024-09-16T20:48:59.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:59.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768489+0000", 2024-09-16T20:48:59.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4600000000000003e-06 2024-09-16T20:48:59.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:59.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768493+0000", 2024-09-16T20:48:59.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2270000000000004e-06 2024-09-16T20:48:59.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:59.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819672+0000", 2024-09-16T20:48:59.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051178605000000002 2024-09-16T20:48:59.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.538 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:59.538 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819711+0000", 2024-09-16T20:48:59.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9357000000000003e-05 2024-09-16T20:48:59.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.538 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:59.538 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819762+0000", 2024-09-16T20:48:59.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1351999999999999e-05 2024-09-16T20:48:59.538 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.538 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:59.538 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.539 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5024 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2150400~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:59.539 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768501+0000", 2024-09-16T20:48:59.539 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 39.721632354, 2024-09-16T20:48:59.539 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051402678, 2024-09-16T20:48:59.539 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:59.539 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:59.539 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:59.539 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:59.539 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:59.539 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:59.539 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5024 2024-09-16T20:48:59.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.540 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:59.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:59.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768501+0000", 2024-09-16T20:48:59.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:59.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768501+0000", 2024-09-16T20:48:59.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:59.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768619+0000", 2024-09-16T20:48:59.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118187 2024-09-16T20:48:59.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.541 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:59.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768624+0000", 2024-09-16T20:48:59.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3259999999999997e-06 2024-09-16T20:48:59.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.541 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:59.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768631+0000", 2024-09-16T20:48:59.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6490000000000008e-06 2024-09-16T20:48:59.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:59.542 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768637+0000", 2024-09-16T20:48:59.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4210000000000003e-06 2024-09-16T20:48:59.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:59.542 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819800+0000", 2024-09-16T20:48:59.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051163486000000001 2024-09-16T20:48:59.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:59.542 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819845+0000", 2024-09-16T20:48:59.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4830999999999998e-05 2024-09-16T20:48:59.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.543 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.543 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:59.543 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819904+0000", 2024-09-16T20:48:59.543 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8777999999999999e-05 2024-09-16T20:48:59.543 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.543 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:59.543 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.543 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.543 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5026 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2158592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:59.543 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768646+0000", 2024-09-16T20:48:59.543 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 39.721486906000003, 2024-09-16T20:48:59.543 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051384386999999997, 2024-09-16T20:48:59.544 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:59.544 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:59.544 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:59.544 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:59.544 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:59.544 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:59.544 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5026 2024-09-16T20:48:59.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.544 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:59.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.544 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:59.544 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768646+0000", 2024-09-16T20:48:59.544 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.545 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:59.545 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768646+0000", 2024-09-16T20:48:59.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.545 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:59.545 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768775+0000", 2024-09-16T20:48:59.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012813200000000001 2024-09-16T20:48:59.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.545 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:59.545 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768778+0000", 2024-09-16T20:48:59.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2150000000000001e-06 2024-09-16T20:48:59.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:59.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768786+0000", 2024-09-16T20:48:59.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3259999999999996e-06 2024-09-16T20:48:59.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:59.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768790+0000", 2024-09-16T20:48:59.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.216e-06 2024-09-16T20:48:59.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:59.547 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819939+0000", 2024-09-16T20:48:59.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051149018999999997 2024-09-16T20:48:59.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.547 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:59.547 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819979+0000", 2024-09-16T20:48:59.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9311999999999999e-05 2024-09-16T20:48:59.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.547 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:59.547 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820031+0000", 2024-09-16T20:48:59.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2166999999999997e-05 2024-09-16T20:48:59.547 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.547 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:59.548 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.548 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.548 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5028 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2166784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:59.548 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768798+0000", 2024-09-16T20:48:59.548 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 39.721335263, 2024-09-16T20:48:59.548 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051388912000000002, 2024-09-16T20:48:59.548 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:59.548 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:59.548 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:59.548 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:59.548 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:59.548 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:59.548 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5028 2024-09-16T20:48:59.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.549 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:59.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.549 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:59.549 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768798+0000", 2024-09-16T20:48:59.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.549 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:59.549 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768798+0000", 2024-09-16T20:48:59.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.549 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:59.549 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768989+0000", 2024-09-16T20:48:59.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000190951 2024-09-16T20:48:59.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.550 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:59.550 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768993+0000", 2024-09-16T20:48:59.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9770000000000002e-06 2024-09-16T20:48:59.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.550 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:59.550 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.769002+0000", 2024-09-16T20:48:59.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5469999999999997e-06 2024-09-16T20:48:59.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.550 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:59.551 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.769020+0000", 2024-09-16T20:48:59.551 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8216999999999999e-05 2024-09-16T20:48:59.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.551 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:59.551 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820070+0000", 2024-09-16T20:48:59.551 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051049998999999999 2024-09-16T20:48:59.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.551 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:59.551 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820119+0000", 2024-09-16T20:48:59.551 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8962000000000003e-05 2024-09-16T20:48:59.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.552 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:59.552 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820187+0000", 2024-09-16T20:48:59.552 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8258999999999998e-05 2024-09-16T20:48:59.552 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.552 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:59.552 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.552 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.552 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5030 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2174976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:59.552 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.769031+0000", 2024-09-16T20:48:59.552 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 39.72110215, 2024-09-16T20:48:59.552 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051289830000000002, 2024-09-16T20:48:59.552 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:59.552 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:59.553 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:59.553 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:59.553 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:59.553 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:59.553 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5030 2024-09-16T20:48:59.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.553 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:59.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.553 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:59.556 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.769031+0000", 2024-09-16T20:48:59.557 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.557 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.557 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.557 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:59.557 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.769031+0000", 2024-09-16T20:48:59.557 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.557 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.557 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.557 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:59.557 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.773332+0000", 2024-09-16T20:48:59.557 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0043004039999999999 2024-09-16T20:48:59.557 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.557 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.557 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:59.558 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.773336+0000", 2024-09-16T20:48:59.558 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7849999999999999e-06 2024-09-16T20:48:59.558 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.558 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.558 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:59.558 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.773347+0000", 2024-09-16T20:48:59.558 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0482999999999999e-05 2024-09-16T20:48:59.558 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.558 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.558 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:59.558 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.773354+0000", 2024-09-16T20:48:59.558 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1559999999999998e-06 2024-09-16T20:48:59.558 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:59.559 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820216+0000", 2024-09-16T20:48:59.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046861591000000001 2024-09-16T20:48:59.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:59.559 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820258+0000", 2024-09-16T20:48:59.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2653e-05 2024-09-16T20:48:59.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:59.559 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820321+0000", 2024-09-16T20:48:59.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2757999999999998e-05 2024-09-16T20:48:59.560 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.560 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:59.560 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.560 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5067 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 6918144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:59.560 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.775279+0000", 2024-09-16T20:48:59.560 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 39.714853884, 2024-09-16T20:48:59.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050171387999999997, 2024-09-16T20:48:59.560 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:59.560 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:59.560 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:59.560 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:59.560 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:59.561 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:59.561 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5067 2024-09-16T20:48:59.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.561 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:59.561 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.561 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:59.561 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775279+0000", 2024-09-16T20:48:59.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.561 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.561 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:59.561 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775279+0000", 2024-09-16T20:48:59.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.562 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:59.562 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775432+0000", 2024-09-16T20:48:59.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015257399999999999 2024-09-16T20:48:59.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.562 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:59.562 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775436+0000", 2024-09-16T20:48:59.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0180000000000003e-06 2024-09-16T20:48:59.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.562 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:59.562 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775447+0000", 2024-09-16T20:48:59.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.077e-05 2024-09-16T20:48:59.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.563 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:59.563 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775456+0000", 2024-09-16T20:48:59.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4089999999999992e-06 2024-09-16T20:48:59.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.563 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:59.563 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825150+0000", 2024-09-16T20:48:59.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049693711000000002 2024-09-16T20:48:59.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.563 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:59.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825337+0000", 2024-09-16T20:48:59.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000187214 2024-09-16T20:48:59.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:59.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825451+0000", 2024-09-16T20:48:59.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011369199999999999 2024-09-16T20:48:59.564 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.564 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:59.564 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.564 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5068 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2330624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:59.564 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.775469+0000", 2024-09-16T20:48:59.565 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 39.714663923000003, 2024-09-16T20:48:59.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050198334999999997, 2024-09-16T20:48:59.565 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:59.565 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:59.565 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:59.565 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:59.565 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:59.565 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:59.565 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5068 2024-09-16T20:48:59.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.565 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:59.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.565 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:59.565 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775469+0000", 2024-09-16T20:48:59.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.566 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.566 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:59.566 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775469+0000", 2024-09-16T20:48:59.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.566 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.566 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:59.566 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775720+0000", 2024-09-16T20:48:59.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000250661 2024-09-16T20:48:59.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.566 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.566 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:59.566 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775724+0000", 2024-09-16T20:48:59.567 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8700000000000002e-06 2024-09-16T20:48:59.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.567 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:59.567 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775733+0000", 2024-09-16T20:48:59.567 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4239999999999999e-06 2024-09-16T20:48:59.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.567 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:59.567 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775740+0000", 2024-09-16T20:48:59.567 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9519999999999997e-06 2024-09-16T20:48:59.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.568 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:59.568 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825484+0000", 2024-09-16T20:48:59.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049743385000000001 2024-09-16T20:48:59.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.568 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:59.568 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825572+0000", 2024-09-16T20:48:59.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7834e-05 2024-09-16T20:48:59.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.568 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:59.568 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825668+0000", 2024-09-16T20:48:59.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6209e-05 2024-09-16T20:48:59.568 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.569 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:59.569 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.569 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5069 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 6926336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:59.569 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.775753+0000", 2024-09-16T20:48:59.569 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 39.7143807, 2024-09-16T20:48:59.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050146430999999998, 2024-09-16T20:48:59.569 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:59.569 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:59.569 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:59.569 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:59.569 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:59.569 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:59.570 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5069 2024-09-16T20:48:59.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.570 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:59.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.570 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:59.570 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775753+0000", 2024-09-16T20:48:59.570 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.570 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:59.570 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775753+0000", 2024-09-16T20:48:59.570 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.571 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:59.571 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775865+0000", 2024-09-16T20:48:59.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000112402 2024-09-16T20:48:59.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.571 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.571 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:59.571 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775869+0000", 2024-09-16T20:48:59.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5130000000000001e-06 2024-09-16T20:48:59.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.571 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.571 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:59.571 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775877+0000", 2024-09-16T20:48:59.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1359999999999997e-06 2024-09-16T20:48:59.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.572 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:59.572 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775882+0000", 2024-09-16T20:48:59.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6710000000000004e-06 2024-09-16T20:48:59.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.572 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:59.572 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825729+0000", 2024-09-16T20:48:59.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049846898000000001 2024-09-16T20:48:59.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.572 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:59.572 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825807+0000", 2024-09-16T20:48:59.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7875999999999998e-05 2024-09-16T20:48:59.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.573 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.573 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:59.573 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825899+0000", 2024-09-16T20:48:59.573 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1935000000000007e-05 2024-09-16T20:48:59.573 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.573 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:59.573 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.573 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.573 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5070 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2338816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:59.573 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.775892+0000", 2024-09-16T20:48:59.573 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 39.714240846000003, 2024-09-16T20:48:59.573 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050155311000000001, 2024-09-16T20:48:59.574 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:59.574 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:59.574 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:59.574 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:59.574 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:59.574 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:59.574 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5070 2024-09-16T20:48:59.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.574 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:59.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.574 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:59.574 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775892+0000", 2024-09-16T20:48:59.574 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.575 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:59.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775892+0000", 2024-09-16T20:48:59.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.575 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:59.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776142+0000", 2024-09-16T20:48:59.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00024950500000000002 2024-09-16T20:48:59.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.575 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:59.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776146+0000", 2024-09-16T20:48:59.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2819999999999998e-06 2024-09-16T20:48:59.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:59.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776157+0000", 2024-09-16T20:48:59.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0807e-05 2024-09-16T20:48:59.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:59.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776165+0000", 2024-09-16T20:48:59.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1300000000000001e-06 2024-09-16T20:48:59.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:59.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825935+0000", 2024-09-16T20:48:59.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049769300000000002 2024-09-16T20:48:59.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.577 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:59.577 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825991+0000", 2024-09-16T20:48:59.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6177000000000001e-05 2024-09-16T20:48:59.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.578 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:59.578 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826048+0000", 2024-09-16T20:48:59.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.711e-05 2024-09-16T20:48:59.578 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.578 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:59.578 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.578 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5071 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 6934528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:59.578 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.776177+0000", 2024-09-16T20:48:59.578 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 39.713956443000001, 2024-09-16T20:48:59.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050036886000000003, 2024-09-16T20:48:59.579 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:59.579 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:59.579 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:59.579 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:59.579 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:59.579 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:59.579 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5071 2024-09-16T20:48:59.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.579 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:59.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:59.579 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776177+0000", 2024-09-16T20:48:59.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:59.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776177+0000", 2024-09-16T20:48:59.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:59.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776339+0000", 2024-09-16T20:48:59.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016238099999999999 2024-09-16T20:48:59.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:59.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776343+0000", 2024-09-16T20:48:59.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9670000000000003e-06 2024-09-16T20:48:59.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:59.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776354+0000", 2024-09-16T20:48:59.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0789e-05 2024-09-16T20:48:59.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:59.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776361+0000", 2024-09-16T20:48:59.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5200000000000003e-06 2024-09-16T20:48:59.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:59.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826078+0000", 2024-09-16T20:48:59.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049717513999999997 2024-09-16T20:48:59.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:59.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826141+0000", 2024-09-16T20:48:59.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2446000000000005e-05 2024-09-16T20:48:59.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:59.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826214+0000", 2024-09-16T20:48:59.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3269000000000006e-05 2024-09-16T20:48:59.582 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.582 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:59.583 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.583 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5072 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2347008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:48:59.583 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.776370+0000", 2024-09-16T20:48:59.583 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 39.713762881999997, 2024-09-16T20:48:59.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050097653999999998, 2024-09-16T20:48:59.583 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:59.583 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:59.583 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:59.583 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:59.583 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:48:59.583 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:48:59.583 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5072 2024-09-16T20:48:59.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.584 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:59.584 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.584 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:59.584 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776370+0000", 2024-09-16T20:48:59.584 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.584 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.584 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:59.584 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776370+0000", 2024-09-16T20:48:59.584 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.584 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.584 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:59.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776645+0000", 2024-09-16T20:48:59.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000274634 2024-09-16T20:48:59.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:59.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776654+0000", 2024-09-16T20:48:59.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9390000000000003e-06 2024-09-16T20:48:59.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:59.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776665+0000", 2024-09-16T20:48:59.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0524e-05 2024-09-16T20:48:59.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:59.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776671+0000", 2024-09-16T20:48:59.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7390000000000002e-06 2024-09-16T20:48:59.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:59.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826261+0000", 2024-09-16T20:48:59.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049589995999999997 2024-09-16T20:48:59.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:59.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826363+0000", 2024-09-16T20:48:59.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010158999999999999 2024-09-16T20:48:59.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:59.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826468+0000", 2024-09-16T20:48:59.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000105232 2024-09-16T20:48:59.587 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.587 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:59.587 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.587 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.587 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:59.587 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:59.589 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:48:59.589 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:59.588+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:59.690 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:48:59.689+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:48:59.696 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:59.696 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:48:59.696 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:48:59.697 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:59.706 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:48:59.791 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:48:59.791+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:48:59.814 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:59.814 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:48:59.814 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:48:59.814 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:48:59.814 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:48:59.822 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:48:59.850 INFO:teuthology.orchestra.run.smithi139.stdout:leaving image image_client.0-clone17 intact 2024-09-16T20:48:59.892 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:48:59.892+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:48:59.930 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:48:59.930 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:48:59.930 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:48:59.931 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:48:59.931 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.931 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:780 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 933888~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:59.931 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.239489+0000", 2024-09-16T20:48:59.931 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 44.689050524000002, 2024-09-16T20:48:59.931 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00741699, 2024-09-16T20:48:59.931 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:59.931 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:59.931 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:59.932 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:59.932 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:59.932 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:59.932 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 780 2024-09-16T20:48:59.932 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.932 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:59.932 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.932 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:59.933 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239489+0000", 2024-09-16T20:48:59.933 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.933 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.933 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.933 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:59.933 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239489+0000", 2024-09-16T20:48:59.933 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.934 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.934 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.934 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:59.934 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239490+0000", 2024-09-16T20:48:59.934 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8599999999999998e-07 2024-09-16T20:48:59.934 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.934 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.935 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:59.935 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239492+0000", 2024-09-16T20:48:59.935 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1380000000000001e-06 2024-09-16T20:48:59.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.935 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.935 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:59.935 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239499+0000", 2024-09-16T20:48:59.935 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9530000000000002e-06 2024-09-16T20:48:59.936 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.936 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.936 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:59.936 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239504+0000", 2024-09-16T20:48:59.936 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0690000000000001e-06 2024-09-16T20:48:59.936 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.936 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.936 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:59.936 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246714+0000", 2024-09-16T20:48:59.936 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.007209844 2024-09-16T20:48:59.936 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.936 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.936 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:59.936 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246804+0000", 2024-09-16T20:48:59.937 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0109000000000001e-05 2024-09-16T20:48:59.937 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.937 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.937 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:59.937 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246906+0000", 2024-09-16T20:48:59.937 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000102391 2024-09-16T20:48:59.937 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.937 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:59.937 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.937 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.937 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.937 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:781 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1048576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:59.938 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.239892+0000", 2024-09-16T20:48:59.938 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 44.688648153999999, 2024-09-16T20:48:59.938 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0072223799999999996, 2024-09-16T20:48:59.938 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:59.938 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:59.938 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:59.938 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:59.938 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:59.938 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:59.938 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 781 2024-09-16T20:48:59.938 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.938 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:59.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.939 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:59.939 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239892+0000", 2024-09-16T20:48:59.939 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.939 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:59.939 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239892+0000", 2024-09-16T20:48:59.939 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.939 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:59.939 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239893+0000", 2024-09-16T20:48:59.939 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1399999999999995e-07 2024-09-16T20:48:59.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:59.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239897+0000", 2024-09-16T20:48:59.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4859999999999999e-06 2024-09-16T20:48:59.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:59.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239907+0000", 2024-09-16T20:48:59.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4539999999999996e-06 2024-09-16T20:48:59.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:59.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239913+0000", 2024-09-16T20:48:59.941 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7880000000000001e-06 2024-09-16T20:48:59.941 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.941 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.941 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:59.941 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246937+0000", 2024-09-16T20:48:59.941 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0070239999999999999 2024-09-16T20:48:59.941 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.941 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.941 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:59.941 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247013+0000", 2024-09-16T20:48:59.941 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5894000000000002e-05 2024-09-16T20:48:59.941 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.941 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.942 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:59.942 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247114+0000", 2024-09-16T20:48:59.942 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000100844 2024-09-16T20:48:59.942 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.942 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:59.942 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.942 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.942 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:782 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1146880~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:59.942 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.239925+0000", 2024-09-16T20:48:59.942 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 44.688615181000003, 2024-09-16T20:48:59.942 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0074161840000000001, 2024-09-16T20:48:59.942 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:59.942 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:59.942 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:59.943 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:59.943 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:59.943 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:59.943 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 782 2024-09-16T20:48:59.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.943 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:59.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.943 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:59.943 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239925+0000", 2024-09-16T20:48:59.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.943 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:59.943 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239925+0000", 2024-09-16T20:48:59.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.944 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:59.944 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239925+0000", 2024-09-16T20:48:59.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7199999999999999e-07 2024-09-16T20:48:59.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.944 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:59.944 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239928+0000", 2024-09-16T20:48:59.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5349999999999999e-06 2024-09-16T20:48:59.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.944 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:59.944 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239935+0000", 2024-09-16T20:48:59.945 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7680000000000003e-06 2024-09-16T20:48:59.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.945 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:59.945 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239940+0000", 2024-09-16T20:48:59.945 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.152e-06 2024-09-16T20:48:59.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.945 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:59.945 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247143+0000", 2024-09-16T20:48:59.945 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0072037309999999997 2024-09-16T20:48:59.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.946 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:59.946 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247230+0000", 2024-09-16T20:48:59.946 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6598e-05 2024-09-16T20:48:59.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.946 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.946 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:59.946 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247341+0000", 2024-09-16T20:48:59.946 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011092800000000001 2024-09-16T20:48:59.946 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.946 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:59.946 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.946 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.946 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:783 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1261568~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:59.946 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.239947+0000", 2024-09-16T20:48:59.947 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 44.688592859000003, 2024-09-16T20:48:59.947 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0076044789999999999, 2024-09-16T20:48:59.947 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:59.947 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:59.947 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:59.947 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:59.947 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:59.947 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:59.947 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 783 2024-09-16T20:48:59.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.947 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:59.947 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.947 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:59.947 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239947+0000", 2024-09-16T20:48:59.948 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.948 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:59.948 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239947+0000", 2024-09-16T20:48:59.948 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.948 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:59.948 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239947+0000", 2024-09-16T20:48:59.948 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9400000000000001e-07 2024-09-16T20:48:59.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.949 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:59.949 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239950+0000", 2024-09-16T20:48:59.949 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2350000000000002e-06 2024-09-16T20:48:59.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.949 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.949 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:59.949 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239956+0000", 2024-09-16T20:48:59.949 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3400000000000003e-06 2024-09-16T20:48:59.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.949 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.949 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:59.949 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239961+0000", 2024-09-16T20:48:59.949 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.04e-06 2024-09-16T20:48:59.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.950 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.950 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:59.950 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247371+0000", 2024-09-16T20:48:59.950 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0074096810000000004 2024-09-16T20:48:59.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.950 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.950 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:59.950 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247451+0000", 2024-09-16T20:48:59.950 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9845999999999994e-05 2024-09-16T20:48:59.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.950 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.950 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:59.950 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247552+0000", 2024-09-16T20:48:59.950 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000100943 2024-09-16T20:48:59.950 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.951 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:59.951 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.951 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.951 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:784 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1376256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:59.951 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.239997+0000", 2024-09-16T20:48:59.951 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 44.688543203000002, 2024-09-16T20:48:59.951 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.007775372, 2024-09-16T20:48:59.951 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:59.951 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:59.951 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:59.951 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:59.951 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:59.951 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:59.951 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 784 2024-09-16T20:48:59.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.952 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:59.952 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.952 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:59.952 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239997+0000", 2024-09-16T20:48:59.952 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.952 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.952 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:59.952 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239997+0000", 2024-09-16T20:48:59.952 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.953 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.953 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:59.953 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239997+0000", 2024-09-16T20:48:59.953 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1799999999999995e-07 2024-09-16T20:48:59.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.953 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.953 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:59.953 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240001+0000", 2024-09-16T20:48:59.953 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6009999999999999e-06 2024-09-16T20:48:59.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.953 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.953 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:59.953 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240009+0000", 2024-09-16T20:48:59.953 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5809999999999999e-06 2024-09-16T20:48:59.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.954 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.954 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:59.954 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240015+0000", 2024-09-16T20:48:59.954 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0100000000000001e-06 2024-09-16T20:48:59.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.954 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.954 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:59.954 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247583+0000", 2024-09-16T20:48:59.954 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0075685730000000003 2024-09-16T20:48:59.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.954 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.954 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:59.954 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247672+0000", 2024-09-16T20:48:59.954 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8480000000000007e-05 2024-09-16T20:48:59.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.955 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:59.955 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247772+0000", 2024-09-16T20:48:59.955 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000100509 2024-09-16T20:48:59.955 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.955 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:59.955 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.955 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:785 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1474560~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:59.955 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240024+0000", 2024-09-16T20:48:59.955 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 44.688515617, 2024-09-16T20:48:59.955 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.008096776, 2024-09-16T20:48:59.955 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:59.956 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:59.956 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:59.956 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:59.956 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:59.956 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:59.956 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 785 2024-09-16T20:48:59.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.956 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:59.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.956 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:59.956 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240024+0000", 2024-09-16T20:48:59.956 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.957 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:59.957 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240024+0000", 2024-09-16T20:48:59.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.957 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:59.957 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240025+0000", 2024-09-16T20:48:59.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5000000000000003e-07 2024-09-16T20:48:59.958 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.958 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.958 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:59.958 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240027+0000", 2024-09-16T20:48:59.958 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2900000000000001e-06 2024-09-16T20:48:59.958 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.958 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.958 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:59.958 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240036+0000", 2024-09-16T20:48:59.958 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4710000000000007e-06 2024-09-16T20:48:59.958 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.958 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.958 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:59.958 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240041+0000", 2024-09-16T20:48:59.958 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4689999999999996e-06 2024-09-16T20:48:59.959 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.959 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.959 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:59.959 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247821+0000", 2024-09-16T20:48:59.959 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0077797919999999998 2024-09-16T20:48:59.959 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.959 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.960 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:59.960 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247921+0000", 2024-09-16T20:48:59.960 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9814000000000004e-05 2024-09-16T20:48:59.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.960 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:59.960 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248121+0000", 2024-09-16T20:48:59.960 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00020039 2024-09-16T20:48:59.960 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.960 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:59.960 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.960 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:940 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 81920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:59.960 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240569+0000", 2024-09-16T20:48:59.961 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 44.687971359000002, 2024-09-16T20:48:59.961 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0078289759999999996, 2024-09-16T20:48:59.961 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:59.961 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:59.961 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:59.961 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:59.961 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:59.961 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:59.962 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 940 2024-09-16T20:48:59.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.962 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:59.964 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.964 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:59.965 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240569+0000", 2024-09-16T20:48:59.965 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.965 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.965 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:59.965 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240569+0000", 2024-09-16T20:48:59.965 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.965 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.966 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:59.966 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240569+0000", 2024-09-16T20:48:59.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2600000000000001e-07 2024-09-16T20:48:59.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.966 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:59.966 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240575+0000", 2024-09-16T20:48:59.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1159999999999998e-06 2024-09-16T20:48:59.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.967 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:59.967 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240587+0000", 2024-09-16T20:48:59.967 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1915e-05 2024-09-16T20:48:59.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.967 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:59.967 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240594+0000", 2024-09-16T20:48:59.967 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5610000000000001e-06 2024-09-16T20:48:59.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.967 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:59.968 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248158+0000", 2024-09-16T20:48:59.968 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0075635370000000004 2024-09-16T20:48:59.968 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.968 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.968 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:59.968 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248278+0000", 2024-09-16T20:48:59.968 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012009400000000001 2024-09-16T20:48:59.969 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.969 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.969 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:59.969 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248398+0000", 2024-09-16T20:48:59.969 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000119827 2024-09-16T20:48:59.969 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.969 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:59.970 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.970 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.970 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:941 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 196608~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:59.970 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240608+0000", 2024-09-16T20:48:59.970 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 44.687931782, 2024-09-16T20:48:59.970 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0080495519999999997, 2024-09-16T20:48:59.971 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:59.971 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:59.971 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:59.971 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:59.971 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:59.971 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:59.971 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 941 2024-09-16T20:48:59.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.971 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:59.971 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.971 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:59.972 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240608+0000", 2024-09-16T20:48:59.972 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.972 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.972 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:59.972 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240608+0000", 2024-09-16T20:48:59.972 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.972 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.972 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:59.972 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240609+0000", 2024-09-16T20:48:59.972 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7000000000000004e-07 2024-09-16T20:48:59.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.972 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.973 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:59.973 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240612+0000", 2024-09-16T20:48:59.973 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8310000000000001e-06 2024-09-16T20:48:59.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.973 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.973 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:59.973 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240619+0000", 2024-09-16T20:48:59.973 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8429999999999993e-06 2024-09-16T20:48:59.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.973 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.973 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:59.973 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240625+0000", 2024-09-16T20:48:59.973 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5659999999999997e-06 2024-09-16T20:48:59.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.973 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.974 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:59.974 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248456+0000", 2024-09-16T20:48:59.974 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0078313940000000002 2024-09-16T20:48:59.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.974 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:59.974 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248548+0000", 2024-09-16T20:48:59.974 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1324999999999997e-05 2024-09-16T20:48:59.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.974 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:59.974 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248658+0000", 2024-09-16T20:48:59.974 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000109923 2024-09-16T20:48:59.974 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.975 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:59.975 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.975 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.975 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:942 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 294912~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:59.975 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240675+0000", 2024-09-16T20:48:59.975 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 44.687864974999997, 2024-09-16T20:48:59.975 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0082135699999999999, 2024-09-16T20:48:59.975 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:59.975 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:59.975 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:59.975 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:59.975 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:59.976 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:59.976 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 942 2024-09-16T20:48:59.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.976 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:59.976 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.976 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:59.976 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240675+0000", 2024-09-16T20:48:59.976 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.976 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.976 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:59.976 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240675+0000", 2024-09-16T20:48:59.976 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.977 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:59.977 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240676+0000", 2024-09-16T20:48:59.977 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.37e-07 2024-09-16T20:48:59.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.977 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:59.977 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240679+0000", 2024-09-16T20:48:59.977 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4060000000000001e-06 2024-09-16T20:48:59.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.977 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:59.977 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240687+0000", 2024-09-16T20:48:59.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4640000000000006e-06 2024-09-16T20:48:59.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.978 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.978 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:59.978 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240694+0000", 2024-09-16T20:48:59.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.246e-06 2024-09-16T20:48:59.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.978 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.978 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:59.978 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248693+0000", 2024-09-16T20:48:59.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0079997550000000008 2024-09-16T20:48:59.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.978 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.979 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:59.979 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248781+0000", 2024-09-16T20:48:59.979 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7509999999999994e-05 2024-09-16T20:48:59.979 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.979 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.980 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:59.980 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248889+0000", 2024-09-16T20:48:59.980 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000107552 2024-09-16T20:48:59.980 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.980 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:59.980 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.980 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.981 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:943 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 409600~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:59.981 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240704+0000", 2024-09-16T20:48:59.981 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 44.687835978999999, 2024-09-16T20:48:59.981 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0083978209999999998, 2024-09-16T20:48:59.981 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:59.981 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:59.981 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:59.981 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:59.981 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:59.982 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:59.982 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 943 2024-09-16T20:48:59.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.982 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:59.982 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.982 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:59.982 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240704+0000", 2024-09-16T20:48:59.982 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.982 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.982 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:59.982 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240704+0000", 2024-09-16T20:48:59.982 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.982 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.982 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:59.983 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240704+0000", 2024-09-16T20:48:59.983 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7e-07 2024-09-16T20:48:59.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.983 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.983 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:59.983 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240707+0000", 2024-09-16T20:48:59.983 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5809999999999999e-06 2024-09-16T20:48:59.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.983 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.983 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:59.983 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240713+0000", 2024-09-16T20:48:59.983 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4949999999999996e-06 2024-09-16T20:48:59.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.983 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.983 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:59.984 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240718+0000", 2024-09-16T20:48:59.984 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8890000000000001e-06 2024-09-16T20:48:59.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.984 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.984 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:59.984 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248920+0000", 2024-09-16T20:48:59.984 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0082018679999999993 2024-09-16T20:48:59.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.984 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.984 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:59.984 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249007+0000", 2024-09-16T20:48:59.984 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6327000000000002e-05 2024-09-16T20:48:59.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.984 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.984 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:59.984 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249102+0000", 2024-09-16T20:48:59.985 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5191000000000004e-05 2024-09-16T20:48:59.985 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.985 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:59.985 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.985 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.985 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:944 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 524288~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:59.985 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240726+0000", 2024-09-16T20:48:59.985 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 44.687814174000003, 2024-09-16T20:48:59.985 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0085999960000000004, 2024-09-16T20:48:59.985 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:59.985 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:59.985 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:59.985 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:59.985 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:59.986 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:59.986 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 944 2024-09-16T20:48:59.986 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.986 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:59.986 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.986 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:59.986 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240726+0000", 2024-09-16T20:48:59.986 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.986 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.986 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.986 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:59.986 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240726+0000", 2024-09-16T20:48:59.986 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.986 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.986 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.986 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:59.987 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240726+0000", 2024-09-16T20:48:59.987 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7099999999999997e-07 2024-09-16T20:48:59.987 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.987 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.987 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:59.987 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240728+0000", 2024-09-16T20:48:59.987 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.125e-06 2024-09-16T20:48:59.987 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.987 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.987 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:59.987 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240734+0000", 2024-09-16T20:48:59.987 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1890000000000004e-06 2024-09-16T20:48:59.987 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.987 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.988 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:59.988 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240739+0000", 2024-09-16T20:48:59.988 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0340000000000004e-06 2024-09-16T20:48:59.988 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.988 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.988 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:59.988 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249131+0000", 2024-09-16T20:48:59.988 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0083919579999999997 2024-09-16T20:48:59.988 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.988 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.988 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:59.988 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249217+0000", 2024-09-16T20:48:59.988 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5154999999999994e-05 2024-09-16T20:48:59.989 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.989 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.989 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:59.989 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249326+0000", 2024-09-16T20:48:59.989 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000109164 2024-09-16T20:48:59.989 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.989 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:59.989 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.989 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.989 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.989 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:945 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 622592~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:59.989 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240746+0000", 2024-09-16T20:48:59.989 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 44.687794035000003, 2024-09-16T20:48:59.989 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0088083320000000003, 2024-09-16T20:48:59.989 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:59.989 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:59.990 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:59.990 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:59.990 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:59.990 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:59.990 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 945 2024-09-16T20:48:59.990 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.990 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:59.990 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.990 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:59.990 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240746+0000", 2024-09-16T20:48:59.990 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.990 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.990 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.990 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:59.990 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240746+0000", 2024-09-16T20:48:59.991 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.991 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.991 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.991 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:59.991 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240746+0000", 2024-09-16T20:48:59.991 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8500000000000002e-07 2024-09-16T20:48:59.991 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.991 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.991 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:59.991 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240748+0000", 2024-09-16T20:48:59.991 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8449999999999999e-06 2024-09-16T20:48:59.991 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.991 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.991 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:59.991 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240754+0000", 2024-09-16T20:48:59.991 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8830000000000003e-06 2024-09-16T20:48:59.992 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.992 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.992 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:59.992 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240759+0000", 2024-09-16T20:48:59.992 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0220000000000004e-06 2024-09-16T20:48:59.992 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.992 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.992 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:59.993 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249360+0000", 2024-09-16T20:48:59.993 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.008601437 2024-09-16T20:48:59.993 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.993 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.993 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:59.993 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249453+0000", 2024-09-16T20:48:59.993 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2743000000000003e-05 2024-09-16T20:48:59.993 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.993 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.993 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:59.993 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249554+0000", 2024-09-16T20:48:59.993 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101017 2024-09-16T20:48:59.993 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.994 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:59.994 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.994 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.994 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.994 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:946 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 737280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:59.994 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240766+0000", 2024-09-16T20:48:59.994 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 44.687774332000004, 2024-09-16T20:48:59.994 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0089092070000000006, 2024-09-16T20:48:59.994 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:59.994 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:59.994 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:59.995 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:48:59.995 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:48:59.995 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:48:59.995 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 946 2024-09-16T20:48:59.995 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.995 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:48:59.995 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.995 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:48:59.995 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240766+0000", 2024-09-16T20:48:59.995 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.996 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.996 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.996 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:48:59.996 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240766+0000", 2024-09-16T20:48:59.996 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:48:59.996 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.996 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.996 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:48:59.996 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240766+0000", 2024-09-16T20:48:59.996 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0600000000000001e-07 2024-09-16T20:48:59.996 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.996 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.996 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:48:59.996 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240768+0000", 2024-09-16T20:48:59.996 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8959999999999999e-06 2024-09-16T20:48:59.997 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.997 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.997 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:48:59.997 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240774+0000", 2024-09-16T20:48:59.997 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0979999999999999e-06 2024-09-16T20:48:59.997 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.997 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.997 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:48:59.997 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240779+0000", 2024-09-16T20:48:59.997 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.933e-06 2024-09-16T20:48:59.997 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.997 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.997 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:48:59.997 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249572+0000", 2024-09-16T20:48:59.997 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0087926380000000002 2024-09-16T20:48:59.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.998 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.998 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:48:59.998 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249612+0000", 2024-09-16T20:48:59.998 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0859000000000003e-05 2024-09-16T20:48:59.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.998 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.998 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:48:59.998 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249675+0000", 2024-09-16T20:48:59.998 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2377000000000003e-05 2024-09-16T20:48:59.998 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.998 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:48:59.998 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:48:59.999 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:48:59.999 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:48:59.999 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:947 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 851968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:48:59.999 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240785+0000", 2024-09-16T20:48:59.999 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 44.687754740000003, 2024-09-16T20:48:59.999 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0090960260000000001, 2024-09-16T20:48:59.999 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:48:59.999 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:48:59.999 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:48:59.999 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:00.000 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:49:00.000 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:49:00.000 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 947 2024-09-16T20:49:00.000 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.000 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:00.000 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.000 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:00.000 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240785+0000", 2024-09-16T20:49:00.000 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:00.000 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.001 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.001 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:00.001 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240785+0000", 2024-09-16T20:49:00.001 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:00.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.001 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.001 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:00.001 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240786+0000", 2024-09-16T20:49:00.001 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-16T20:49:00.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.001 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.001 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:00.002 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240787+0000", 2024-09-16T20:49:00.002 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.858e-06 2024-09-16T20:49:00.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.002 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:00.002 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240793+0000", 2024-09-16T20:49:00.002 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7760000000000003e-06 2024-09-16T20:49:00.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.003 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:00.003 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240799+0000", 2024-09-16T20:49:00.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1109999999999996e-06 2024-09-16T20:49:00.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.003 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.003 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:00.003 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249715+0000", 2024-09-16T20:49:00.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0089159789999999992 2024-09-16T20:49:00.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.004 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.004 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:00.004 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249766+0000", 2024-09-16T20:49:00.004 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0968999999999999e-05 2024-09-16T20:49:00.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.004 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.004 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:00.004 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249881+0000", 2024-09-16T20:49:00.004 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114963 2024-09-16T20:49:00.004 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:00.005 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:00.005 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:00.005 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.005 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.005 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:948 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:49:00.005 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240806+0000", 2024-09-16T20:49:00.005 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 44.687733559999998, 2024-09-16T20:49:00.005 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0092300609999999995, 2024-09-16T20:49:00.006 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:00.006 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:00.006 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:00.006 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:00.006 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:49:00.006 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:49:00.006 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 948 2024-09-16T20:49:00.007 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.007 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:00.007 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.007 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:00.007 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240806+0000", 2024-09-16T20:49:00.007 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:00.007 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.007 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.008 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:00.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240806+0000", 2024-09-16T20:49:00.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:00.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.008 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:00.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240807+0000", 2024-09-16T20:49:00.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4499999999999998e-07 2024-09-16T20:49:00.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.008 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:00.009 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240809+0000", 2024-09-16T20:49:00.009 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.34e-06 2024-09-16T20:49:00.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.009 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.009 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:00.009 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240815+0000", 2024-09-16T20:49:00.009 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1449999999999996e-06 2024-09-16T20:49:00.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.009 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.009 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:00.009 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240821+0000", 2024-09-16T20:49:00.009 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0270000000000001e-06 2024-09-16T20:49:00.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.009 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.010 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:00.010 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249920+0000", 2024-09-16T20:49:00.010 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0090988460000000007 2024-09-16T20:49:00.010 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.010 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.010 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:00.010 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249966+0000", 2024-09-16T20:49:00.010 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5918999999999999e-05 2024-09-16T20:49:00.010 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.010 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.011 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:00.011 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250036+0000", 2024-09-16T20:49:00.011 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0438999999999994e-05 2024-09-16T20:49:00.011 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:00.011 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:00.011 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:00.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.011 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.011 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:949 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1064960~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:49:00.011 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240833+0000", 2024-09-16T20:49:00.011 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 44.687706945000002, 2024-09-16T20:49:00.011 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0093580339999999994, 2024-09-16T20:49:00.011 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:00.011 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:00.011 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:00.012 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:00.012 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:49:00.012 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:49:00.012 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 949 2024-09-16T20:49:00.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.012 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:00.012 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.012 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:00.012 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240833+0000", 2024-09-16T20:49:00.012 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:00.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.013 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.013 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:00.013 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240833+0000", 2024-09-16T20:49:00.013 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:00.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.013 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.013 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:00.013 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240833+0000", 2024-09-16T20:49:00.013 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2899999999999999e-07 2024-09-16T20:49:00.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.013 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.013 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:00.013 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240836+0000", 2024-09-16T20:49:00.014 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2649999999999999e-06 2024-09-16T20:49:00.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.014 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.014 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:00.014 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240842+0000", 2024-09-16T20:49:00.014 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5579999999999997e-06 2024-09-16T20:49:00.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.014 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.014 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:00.014 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240847+0000", 2024-09-16T20:49:00.014 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2009999999999998e-06 2024-09-16T20:49:00.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.015 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.015 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:00.015 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250073+0000", 2024-09-16T20:49:00.015 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0092258710000000001 2024-09-16T20:49:00.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.015 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.015 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:00.015 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250141+0000", 2024-09-16T20:49:00.015 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7849999999999996e-05 2024-09-16T20:49:00.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.015 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.015 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:00.016 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250191+0000", 2024-09-16T20:49:00.016 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.986e-05 2024-09-16T20:49:00.016 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:00.016 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:00.016 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:00.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.016 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.016 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:950 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1179648~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:49:00.016 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240855+0000", 2024-09-16T20:49:00.016 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 44.687684693000001, 2024-09-16T20:49:00.016 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0095855560000000003, 2024-09-16T20:49:00.017 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:00.017 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:00.017 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:00.017 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:00.017 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:49:00.017 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:49:00.017 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 950 2024-09-16T20:49:00.017 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.017 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:00.018 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.018 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:00.018 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240855+0000", 2024-09-16T20:49:00.018 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:00.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.018 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.018 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:00.018 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240855+0000", 2024-09-16T20:49:00.018 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:00.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.018 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.019 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:00.019 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240856+0000", 2024-09-16T20:49:00.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4999999999999998e-07 2024-09-16T20:49:00.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.019 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:00.019 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240858+0000", 2024-09-16T20:49:00.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.996e-06 2024-09-16T20:49:00.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.019 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:00.019 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240864+0000", 2024-09-16T20:49:00.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9359999999999996e-06 2024-09-16T20:49:00.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.020 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.020 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:00.020 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240868+0000", 2024-09-16T20:49:00.020 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7670000000000003e-06 2024-09-16T20:49:00.020 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.020 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.020 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:00.020 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250226+0000", 2024-09-16T20:49:00.020 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0093576100000000006 2024-09-16T20:49:00.020 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.020 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.020 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:00.020 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250289+0000", 2024-09-16T20:49:00.020 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2627999999999997e-05 2024-09-16T20:49:00.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.021 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.021 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:00.021 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250441+0000", 2024-09-16T20:49:00.021 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015216900000000001 2024-09-16T20:49:00.021 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:00.021 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:00.021 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:00.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.021 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.021 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:951 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1277952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:49:00.021 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.241044+0000", 2024-09-16T20:49:00.021 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 44.687495505000001, 2024-09-16T20:49:00.021 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0096266589999999992, 2024-09-16T20:49:00.021 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:00.022 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:00.022 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:00.022 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:00.022 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:49:00.022 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:49:00.022 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 951 2024-09-16T20:49:00.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.022 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:00.022 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.022 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:00.022 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241044+0000", 2024-09-16T20:49:00.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:00.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.022 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.022 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:00.023 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241044+0000", 2024-09-16T20:49:00.023 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:00.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.023 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.023 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:00.023 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241045+0000", 2024-09-16T20:49:00.023 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5300000000000004e-07 2024-09-16T20:49:00.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.023 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.023 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:00.023 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241048+0000", 2024-09-16T20:49:00.023 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.303e-06 2024-09-16T20:49:00.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.023 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.024 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:00.024 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241057+0000", 2024-09-16T20:49:00.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5550000000000003e-06 2024-09-16T20:49:00.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.024 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:00.024 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241063+0000", 2024-09-16T20:49:00.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3470000000000004e-06 2024-09-16T20:49:00.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.024 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:00.024 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250477+0000", 2024-09-16T20:49:00.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0094136159999999997 2024-09-16T20:49:00.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:00.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250562+0000", 2024-09-16T20:49:00.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4709000000000005e-05 2024-09-16T20:49:00.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:00.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250671+0000", 2024-09-16T20:49:00.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010947599999999999 2024-09-16T20:49:00.026 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:00.026 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:00.026 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:00.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.026 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:952 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1392640~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:49:00.026 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.241130+0000", 2024-09-16T20:49:00.026 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 44.687409639000002, 2024-09-16T20:49:00.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0098096290000000003, 2024-09-16T20:49:00.026 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:00.026 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:00.026 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:00.026 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:00.026 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:49:00.026 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:49:00.027 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 952 2024-09-16T20:49:00.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.027 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:00.027 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.027 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:00.027 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241130+0000", 2024-09-16T20:49:00.027 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:00.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.027 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.027 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:00.027 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241130+0000", 2024-09-16T20:49:00.027 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:00.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.027 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.027 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:00.028 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241131+0000", 2024-09-16T20:49:00.028 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.61e-07 2024-09-16T20:49:00.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.028 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.028 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:00.028 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241134+0000", 2024-09-16T20:49:00.028 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1080000000000001e-06 2024-09-16T20:49:00.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.028 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.028 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:00.028 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241142+0000", 2024-09-16T20:49:00.028 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9119999999999998e-06 2024-09-16T20:49:00.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.029 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:00.029 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241148+0000", 2024-09-16T20:49:00.029 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1410000000000002e-06 2024-09-16T20:49:00.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.029 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:00.029 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250731+0000", 2024-09-16T20:49:00.029 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0095833979999999999 2024-09-16T20:49:00.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.030 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:00.030 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250820+0000", 2024-09-16T20:49:00.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8997000000000002e-05 2024-09-16T20:49:00.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.030 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:00.031 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250940+0000", 2024-09-16T20:49:00.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000119412 2024-09-16T20:49:00.031 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:00.031 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:00.031 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:00.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.031 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.031 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:953 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1507328~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:49:00.031 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.241157+0000", 2024-09-16T20:49:00.031 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 44.687382718000002, 2024-09-16T20:49:00.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.010039259, 2024-09-16T20:49:00.031 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:00.031 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:00.032 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:00.032 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:00.032 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:49:00.032 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:49:00.032 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 953 2024-09-16T20:49:00.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.032 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:00.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.032 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:00.033 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241157+0000", 2024-09-16T20:49:00.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:00.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.033 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.033 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:00.033 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241157+0000", 2024-09-16T20:49:00.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:00.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.033 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.033 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:00.033 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241158+0000", 2024-09-16T20:49:00.034 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.01e-07 2024-09-16T20:49:00.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.034 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.034 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:00.034 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241160+0000", 2024-09-16T20:49:00.034 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2879999999999999e-06 2024-09-16T20:49:00.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.034 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.034 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:00.034 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241166+0000", 2024-09-16T20:49:00.035 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.421e-06 2024-09-16T20:49:00.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.035 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:00.035 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241171+0000", 2024-09-16T20:49:00.035 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9840000000000001e-06 2024-09-16T20:49:00.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.035 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:00.035 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250988+0000", 2024-09-16T20:49:00.035 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0098171449999999993 2024-09-16T20:49:00.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.036 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:00.036 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.251085+0000", 2024-09-16T20:49:00.036 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6877000000000001e-05 2024-09-16T20:49:00.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:00.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:00.036 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:00.036 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.251196+0000", 2024-09-16T20:49:00.037 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000111143 2024-09-16T20:49:00.037 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:00.037 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:00.037 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:00.037 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:00.037 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:00.037 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:00.039 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:00.038+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:00.040 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:49:00.139 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:00.139+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:00.159 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:00.160 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:49:00.160 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:49:00.160 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:00.170 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:49:00.240 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:00.240+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:00.279 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #16, image image_client.0-clone16 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent17 2024-09-16T20:49:00.283 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:00.283 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:49:00.284 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:49:00.284 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:49:00.284 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:00.295 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:49:00.341 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:00.341+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:00.377 INFO:teuthology.orchestra.run.smithi139.stdout:2525 punch from 0x8f71c00 to 0x8f78c00, (0x7000 bytes) 2024-09-16T20:49:00.394 INFO:teuthology.orchestra.run.smithi139.stdout:2526 write 0x5b3e200 thru 0x5b3fbc4 (0x19c5 bytes) 2024-09-16T20:49:00.403 INFO:teuthology.orchestra.run.smithi139.stdout:2527 trunc from 0xe8c6000 to 0x7a3bc00 2024-09-16T20:49:00.419 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:00.419 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:49:00.419 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:49:00.419 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:49:00.420 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.420 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2174 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 931840~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:00.420 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335679+0000", 2024-09-16T20:49:00.420 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 38.081718731999999, 2024-09-16T20:49:00.420 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047619511000000003, 2024-09-16T20:49:00.420 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:00.420 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:00.420 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:00.420 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:00.420 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:00.420 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:00.420 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2174 2024-09-16T20:49:00.420 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.421 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:00.421 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.421 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:00.421 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335679+0000", 2024-09-16T20:49:00.421 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:00.421 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.421 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.421 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:00.421 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335679+0000", 2024-09-16T20:49:00.421 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:00.421 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.421 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.422 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:00.422 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335679+0000", 2024-09-16T20:49:00.422 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7800000000000002e-07 2024-09-16T20:49:00.422 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.422 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.422 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:00.422 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335682+0000", 2024-09-16T20:49:00.422 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9490000000000001e-06 2024-09-16T20:49:00.422 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.423 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.423 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:00.423 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335689+0000", 2024-09-16T20:49:00.423 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8220000000000001e-06 2024-09-16T20:49:00.423 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.423 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.423 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:00.423 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335695+0000", 2024-09-16T20:49:00.423 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0569999999999998e-06 2024-09-16T20:49:00.423 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.423 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.423 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:00.423 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383103+0000", 2024-09-16T20:49:00.423 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047408219000000001 2024-09-16T20:49:00.423 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.424 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.424 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:00.424 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383189+0000", 2024-09-16T20:49:00.424 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6007000000000005e-05 2024-09-16T20:49:00.424 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.424 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.424 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:00.424 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383298+0000", 2024-09-16T20:49:00.424 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000109079 2024-09-16T20:49:00.425 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:00.425 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:00.425 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:00.425 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.425 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.425 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2176 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 940032~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:00.425 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335702+0000", 2024-09-16T20:49:00.425 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 38.081695504000002, 2024-09-16T20:49:00.425 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047804450999999998, 2024-09-16T20:49:00.425 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:00.425 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:00.425 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:00.425 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:00.426 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:00.426 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:00.426 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2176 2024-09-16T20:49:00.426 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.426 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:00.426 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.426 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:00.426 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335702+0000", 2024-09-16T20:49:00.426 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:00.427 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.427 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.427 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:00.427 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335702+0000", 2024-09-16T20:49:00.427 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:00.427 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.427 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.427 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:00.427 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335703+0000", 2024-09-16T20:49:00.427 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.34e-07 2024-09-16T20:49:00.428 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.428 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.428 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:00.428 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335704+0000", 2024-09-16T20:49:00.428 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.917e-06 2024-09-16T20:49:00.428 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.428 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.429 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:00.429 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335711+0000", 2024-09-16T20:49:00.429 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4400000000000002e-06 2024-09-16T20:49:00.429 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.429 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.429 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:00.429 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335717+0000", 2024-09-16T20:49:00.429 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6679999999999998e-06 2024-09-16T20:49:00.429 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.429 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.429 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:00.429 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383335+0000", 2024-09-16T20:49:00.430 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047618754999999999 2024-09-16T20:49:00.430 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.430 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.430 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:00.430 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383405+0000", 2024-09-16T20:49:00.430 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9286e-05 2024-09-16T20:49:00.430 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.430 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.430 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:00.430 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383507+0000", 2024-09-16T20:49:00.431 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101951 2024-09-16T20:49:00.431 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:00.431 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:00.431 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:00.431 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.431 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.431 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2179 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 948224~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:00.431 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335723+0000", 2024-09-16T20:49:00.432 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 38.081674409999998, 2024-09-16T20:49:00.432 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048007181000000003, 2024-09-16T20:49:00.432 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:00.432 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:00.432 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:00.432 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:00.432 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:00.433 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:00.433 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2179 2024-09-16T20:49:00.433 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.433 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:00.433 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.433 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:00.433 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335723+0000", 2024-09-16T20:49:00.433 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:00.433 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.434 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.434 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:00.434 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335723+0000", 2024-09-16T20:49:00.434 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:00.434 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.434 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.434 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:00.434 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335724+0000", 2024-09-16T20:49:00.434 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.58e-07 2024-09-16T20:49:00.434 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.435 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.435 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:00.435 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335726+0000", 2024-09-16T20:49:00.435 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9120000000000001e-06 2024-09-16T20:49:00.435 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.435 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.435 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:00.435 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335732+0000", 2024-09-16T20:49:00.435 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6080000000000001e-06 2024-09-16T20:49:00.435 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.435 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.435 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:00.435 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335737+0000", 2024-09-16T20:49:00.435 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.22e-06 2024-09-16T20:49:00.436 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.436 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.436 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:00.436 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383556+0000", 2024-09-16T20:49:00.436 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047819002999999999 2024-09-16T20:49:00.436 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.436 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.436 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:00.436 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383635+0000", 2024-09-16T20:49:00.436 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9060999999999994e-05 2024-09-16T20:49:00.436 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.436 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.436 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:00.436 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383730+0000", 2024-09-16T20:49:00.437 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4919000000000004e-05 2024-09-16T20:49:00.437 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:00.437 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:00.437 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:00.437 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.437 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.437 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2180 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 956416~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:00.437 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335744+0000", 2024-09-16T20:49:00.437 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 38.081653961999997, 2024-09-16T20:49:00.437 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048255147999999998, 2024-09-16T20:49:00.437 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:00.437 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:00.437 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:00.437 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:00.438 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:00.438 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:00.438 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2180 2024-09-16T20:49:00.438 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.438 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:00.438 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.438 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:00.438 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335744+0000", 2024-09-16T20:49:00.438 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:00.438 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.438 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.438 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:00.438 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335744+0000", 2024-09-16T20:49:00.438 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:00.439 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.439 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.439 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:00.439 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335744+0000", 2024-09-16T20:49:00.439 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9000000000000002e-07 2024-09-16T20:49:00.439 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.439 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.439 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:00.439 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335746+0000", 2024-09-16T20:49:00.439 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7570000000000001e-06 2024-09-16T20:49:00.439 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.439 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.439 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:00.439 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335752+0000", 2024-09-16T20:49:00.440 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0079999999999999e-06 2024-09-16T20:49:00.440 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.440 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.440 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:00.440 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335757+0000", 2024-09-16T20:49:00.440 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1649999999999997e-06 2024-09-16T20:49:00.440 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.440 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.440 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:00.441 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383803+0000", 2024-09-16T20:49:00.441 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048046091999999999 2024-09-16T20:49:00.441 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.441 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.441 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:00.441 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383889+0000", 2024-09-16T20:49:00.442 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6048e-05 2024-09-16T20:49:00.442 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.442 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.442 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:00.442 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383999+0000", 2024-09-16T20:49:00.442 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000109688 2024-09-16T20:49:00.442 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:00.442 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:00.442 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:00.442 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.442 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.442 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2182 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 964608~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:00.442 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335773+0000", 2024-09-16T20:49:00.442 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 38.081624155, 2024-09-16T20:49:00.443 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048467116999999997, 2024-09-16T20:49:00.443 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:00.443 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:00.443 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:00.443 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:00.443 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:00.443 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:00.443 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2182 2024-09-16T20:49:00.443 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.443 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:00.443 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.443 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:00.443 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335773+0000", 2024-09-16T20:49:00.444 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:00.444 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.444 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.444 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:00.444 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335773+0000", 2024-09-16T20:49:00.444 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:00.444 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.444 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.444 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:00.444 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335774+0000", 2024-09-16T20:49:00.444 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3799999999999997e-07 2024-09-16T20:49:00.444 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.445 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.445 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:00.445 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335776+0000", 2024-09-16T20:49:00.445 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.227e-06 2024-09-16T20:49:00.445 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.445 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.445 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:00.445 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335790+0000", 2024-09-16T20:49:00.445 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4119e-05 2024-09-16T20:49:00.445 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.445 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.445 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:00.445 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335796+0000", 2024-09-16T20:49:00.445 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.694e-06 2024-09-16T20:49:00.446 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.446 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.446 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:00.446 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384058+0000", 2024-09-16T20:49:00.446 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048261983000000001 2024-09-16T20:49:00.446 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.446 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.446 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:00.446 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384148+0000", 2024-09-16T20:49:00.446 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9501999999999998e-05 2024-09-16T20:49:00.446 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.446 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.446 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:00.446 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384241+0000", 2024-09-16T20:49:00.447 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.3053999999999994e-05 2024-09-16T20:49:00.447 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:00.447 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:00.447 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:00.447 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.447 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.447 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2185 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 972800~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:00.447 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335804+0000", 2024-09-16T20:49:00.447 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 38.081593505000001, 2024-09-16T20:49:00.447 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048570818000000002, 2024-09-16T20:49:00.447 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:00.447 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:00.447 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:00.447 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:00.448 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:00.448 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:00.448 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2185 2024-09-16T20:49:00.448 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.448 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:00.448 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.448 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:00.448 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335804+0000", 2024-09-16T20:49:00.448 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:00.448 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.448 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.448 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:00.448 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335804+0000", 2024-09-16T20:49:00.448 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:00.448 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.449 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.449 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:00.449 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335805+0000", 2024-09-16T20:49:00.449 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8299999999999997e-07 2024-09-16T20:49:00.449 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.449 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.449 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:00.449 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335813+0000", 2024-09-16T20:49:00.449 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6309999999999992e-06 2024-09-16T20:49:00.449 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.449 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.449 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:00.449 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335821+0000", 2024-09-16T20:49:00.449 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0169999999999996e-06 2024-09-16T20:49:00.450 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.450 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.450 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:00.450 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335827+0000", 2024-09-16T20:49:00.450 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0100000000000001e-06 2024-09-16T20:49:00.450 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.450 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.450 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:00.450 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384273+0000", 2024-09-16T20:49:00.450 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048445333 2024-09-16T20:49:00.450 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.450 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.450 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:00.450 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384315+0000", 2024-09-16T20:49:00.450 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2472999999999998e-05 2024-09-16T20:49:00.450 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.451 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.451 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:00.451 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384375+0000", 2024-09-16T20:49:00.451 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9871000000000003e-05 2024-09-16T20:49:00.451 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:00.451 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:00.451 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:00.451 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.451 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.451 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2187 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 980992~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:00.451 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335835+0000", 2024-09-16T20:49:00.451 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 38.081562427999998, 2024-09-16T20:49:00.451 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048684927000000003, 2024-09-16T20:49:00.451 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:00.451 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:00.452 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:00.452 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:00.452 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:00.452 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:00.452 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2187 2024-09-16T20:49:00.452 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.452 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:00.452 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.452 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:00.452 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335835+0000", 2024-09-16T20:49:00.452 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:00.452 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.452 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.452 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:00.452 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335835+0000", 2024-09-16T20:49:00.453 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:00.453 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.453 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.453 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:00.453 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335836+0000", 2024-09-16T20:49:00.453 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3599999999999999e-07 2024-09-16T20:49:00.453 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.453 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.453 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:00.453 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335838+0000", 2024-09-16T20:49:00.453 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.996e-06 2024-09-16T20:49:00.453 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.453 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.453 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:00.454 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335844+0000", 2024-09-16T20:49:00.454 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7889999999999997e-06 2024-09-16T20:49:00.454 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.454 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.454 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:00.454 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335849+0000", 2024-09-16T20:49:00.454 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7400000000000004e-06 2024-09-16T20:49:00.454 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.454 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.454 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:00.454 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384411+0000", 2024-09-16T20:49:00.454 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048562069999999999 2024-09-16T20:49:00.454 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.454 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.454 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:00.454 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384460+0000", 2024-09-16T20:49:00.455 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9125999999999997e-05 2024-09-16T20:49:00.455 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.455 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.455 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:00.455 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384520+0000", 2024-09-16T20:49:00.455 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9769999999999999e-05 2024-09-16T20:49:00.455 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:00.455 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:00.455 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:00.455 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.455 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.456 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2189 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 989184~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:00.456 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335856+0000", 2024-09-16T20:49:00.456 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 38.081541182999999, 2024-09-16T20:49:00.456 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048856525999999997, 2024-09-16T20:49:00.456 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:00.456 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:00.456 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:00.456 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:00.456 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:00.456 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:00.456 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2189 2024-09-16T20:49:00.456 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.457 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:00.457 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.457 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:00.457 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335856+0000", 2024-09-16T20:49:00.457 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:00.457 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.457 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.457 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:00.457 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335856+0000", 2024-09-16T20:49:00.457 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:00.457 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.458 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.458 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:00.458 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335857+0000", 2024-09-16T20:49:00.458 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6800000000000001e-07 2024-09-16T20:49:00.458 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.458 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.458 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:00.458 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335859+0000", 2024-09-16T20:49:00.458 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.013e-06 2024-09-16T20:49:00.459 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.459 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.459 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:00.459 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335866+0000", 2024-09-16T20:49:00.459 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8750000000000002e-06 2024-09-16T20:49:00.459 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.459 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.459 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:00.459 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335871+0000", 2024-09-16T20:49:00.459 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1719999999999998e-06 2024-09-16T20:49:00.459 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.459 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.459 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:00.459 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384550+0000", 2024-09-16T20:49:00.460 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048679476999999999 2024-09-16T20:49:00.460 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.460 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.460 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:00.460 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384611+0000", 2024-09-16T20:49:00.460 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0897000000000003e-05 2024-09-16T20:49:00.460 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.460 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.460 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:00.460 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384713+0000", 2024-09-16T20:49:00.460 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101724 2024-09-16T20:49:00.460 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:00.460 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:00.461 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:00.461 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.461 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.461 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2192 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 997376~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:00.461 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335878+0000", 2024-09-16T20:49:00.461 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 38.081519497999999, 2024-09-16T20:49:00.461 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049028769999999999, 2024-09-16T20:49:00.461 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:00.461 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:00.461 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:00.461 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:00.461 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:00.461 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:00.462 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2192 2024-09-16T20:49:00.462 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.462 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:00.462 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.462 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:00.462 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335878+0000", 2024-09-16T20:49:00.462 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:00.462 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.462 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.462 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:00.462 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335878+0000", 2024-09-16T20:49:00.462 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:00.462 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.462 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.463 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:00.463 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335879+0000", 2024-09-16T20:49:00.463 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6899999999999998e-07 2024-09-16T20:49:00.463 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.463 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.463 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:00.463 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335881+0000", 2024-09-16T20:49:00.463 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.996e-06 2024-09-16T20:49:00.463 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.464 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.464 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:00.464 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335887+0000", 2024-09-16T20:49:00.464 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6370000000000002e-06 2024-09-16T20:49:00.464 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.464 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.464 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:00.464 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335892+0000", 2024-09-16T20:49:00.464 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2120000000000002e-06 2024-09-16T20:49:00.464 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.464 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.464 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:00.464 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384747+0000", 2024-09-16T20:49:00.465 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048854923000000001 2024-09-16T20:49:00.465 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.465 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.465 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:00.465 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384830+0000", 2024-09-16T20:49:00.465 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2359000000000002e-05 2024-09-16T20:49:00.465 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.465 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.465 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:00.465 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384907+0000", 2024-09-16T20:49:00.466 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7174000000000003e-05 2024-09-16T20:49:00.466 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:00.466 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:00.466 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:00.466 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.466 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.466 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2194 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1005568~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:00.466 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335899+0000", 2024-09-16T20:49:00.466 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 38.081498576999998, 2024-09-16T20:49:00.466 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049225393999999999, 2024-09-16T20:49:00.466 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:00.466 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:00.467 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:00.467 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:00.467 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:00.467 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:00.467 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2194 2024-09-16T20:49:00.467 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.467 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:00.467 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.467 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:00.467 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335899+0000", 2024-09-16T20:49:00.467 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:00.468 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.468 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.468 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:00.468 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335899+0000", 2024-09-16T20:49:00.468 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:00.468 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.468 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.468 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:00.468 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335899+0000", 2024-09-16T20:49:00.468 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8200000000000001e-07 2024-09-16T20:49:00.469 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.469 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.469 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:00.469 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335901+0000", 2024-09-16T20:49:00.469 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9e-06 2024-09-16T20:49:00.469 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.469 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.469 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:00.469 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335908+0000", 2024-09-16T20:49:00.469 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3559999999999998e-06 2024-09-16T20:49:00.469 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.469 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.470 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:00.470 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335914+0000", 2024-09-16T20:49:00.470 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4729999999999997e-06 2024-09-16T20:49:00.470 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.470 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.470 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:00.470 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384979+0000", 2024-09-16T20:49:00.470 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049064545000000001 2024-09-16T20:49:00.470 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.470 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.470 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:00.470 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385060+0000", 2024-09-16T20:49:00.470 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0804999999999996e-05 2024-09-16T20:49:00.470 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.471 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.471 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:00.471 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385124+0000", 2024-09-16T20:49:00.471 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4932999999999996e-05 2024-09-16T20:49:00.471 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:00.471 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:00.471 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:00.471 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.471 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.471 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2197 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1013760~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:00.471 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335921+0000", 2024-09-16T20:49:00.471 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 38.081476295999998, 2024-09-16T20:49:00.471 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049342184999999997, 2024-09-16T20:49:00.472 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:00.472 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:00.472 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:00.472 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:00.472 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:00.472 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:00.472 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2197 2024-09-16T20:49:00.472 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.472 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:00.472 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.472 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:00.472 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335921+0000", 2024-09-16T20:49:00.472 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:00.472 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.473 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.473 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:00.473 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335921+0000", 2024-09-16T20:49:00.473 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:00.473 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.473 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.473 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:00.473 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335922+0000", 2024-09-16T20:49:00.473 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.32e-07 2024-09-16T20:49:00.473 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.473 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.473 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:00.473 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335924+0000", 2024-09-16T20:49:00.474 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.995e-06 2024-09-16T20:49:00.474 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.474 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.474 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:00.474 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335930+0000", 2024-09-16T20:49:00.474 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5139999999999998e-06 2024-09-16T20:49:00.474 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.474 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.474 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:00.474 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335936+0000", 2024-09-16T20:49:00.474 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6160000000000001e-06 2024-09-16T20:49:00.474 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.474 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.475 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:00.475 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385142+0000", 2024-09-16T20:49:00.475 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049206393000000001 2024-09-16T20:49:00.475 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.475 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.475 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:00.475 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385184+0000", 2024-09-16T20:49:00.475 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1928999999999998e-05 2024-09-16T20:49:00.475 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.475 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.475 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:00.475 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385264+0000", 2024-09-16T20:49:00.475 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9306000000000003e-05 2024-09-16T20:49:00.475 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:00.476 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:00.476 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:00.476 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.476 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.476 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2198 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1021952~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:00.476 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335942+0000", 2024-09-16T20:49:00.476 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 38.081455235999996, 2024-09-16T20:49:00.476 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049562483999999997, 2024-09-16T20:49:00.476 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:00.476 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:00.476 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:00.476 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:00.476 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:00.477 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:00.477 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2198 2024-09-16T20:49:00.477 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.477 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:00.477 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.477 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:00.477 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335942+0000", 2024-09-16T20:49:00.477 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:00.477 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.477 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.477 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:00.477 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335942+0000", 2024-09-16T20:49:00.478 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:00.478 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.478 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.478 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:00.478 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335943+0000", 2024-09-16T20:49:00.478 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7099999999999997e-07 2024-09-16T20:49:00.478 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.478 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.478 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:00.478 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335945+0000", 2024-09-16T20:49:00.478 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3499999999999999e-06 2024-09-16T20:49:00.478 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.478 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.478 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:00.479 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335952+0000", 2024-09-16T20:49:00.479 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4309999999999999e-06 2024-09-16T20:49:00.479 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.479 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.479 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:00.479 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335957+0000", 2024-09-16T20:49:00.479 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0869999999999997e-06 2024-09-16T20:49:00.479 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.479 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.479 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:00.479 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385312+0000", 2024-09-16T20:49:00.479 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049355121000000002 2024-09-16T20:49:00.479 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.479 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.480 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:00.480 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385405+0000", 2024-09-16T20:49:00.480 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.3344000000000006e-05 2024-09-16T20:49:00.480 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.480 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.480 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:00.480 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385505+0000", 2024-09-16T20:49:00.480 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.9779999999999997e-05 2024-09-16T20:49:00.480 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:00.480 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:00.480 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:00.480 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.480 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.481 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2201 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1030144~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:00.481 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335976+0000", 2024-09-16T20:49:00.481 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 38.081421794000001, 2024-09-16T20:49:00.481 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049740077000000001, 2024-09-16T20:49:00.481 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:00.481 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:00.481 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:00.481 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:00.481 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:00.481 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:00.481 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2201 2024-09-16T20:49:00.481 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.481 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:00.482 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.482 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:00.482 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335976+0000", 2024-09-16T20:49:00.482 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:00.482 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.482 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.482 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:00.482 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335976+0000", 2024-09-16T20:49:00.482 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:00.482 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.482 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.482 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:00.482 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335976+0000", 2024-09-16T20:49:00.483 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8599999999999998e-07 2024-09-16T20:49:00.483 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.483 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.483 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:00.483 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335978+0000", 2024-09-16T20:49:00.483 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1500000000000002e-06 2024-09-16T20:49:00.483 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.483 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.483 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:00.483 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335986+0000", 2024-09-16T20:49:00.484 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3989999999999998e-06 2024-09-16T20:49:00.484 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.484 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.484 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:00.484 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335992+0000", 2024-09-16T20:49:00.484 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.682e-06 2024-09-16T20:49:00.484 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.484 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.484 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:00.484 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385537+0000", 2024-09-16T20:49:00.484 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049545828 2024-09-16T20:49:00.484 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.485 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.485 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:00.485 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385619+0000", 2024-09-16T20:49:00.485 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1720999999999999e-05 2024-09-16T20:49:00.485 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.485 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.485 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:00.485 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385716+0000", 2024-09-16T20:49:00.485 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.6811000000000005e-05 2024-09-16T20:49:00.485 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:00.485 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:00.485 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:00.485 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.485 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.486 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2203 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1038336~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:00.486 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.336000+0000", 2024-09-16T20:49:00.486 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 38.081398008000001, 2024-09-16T20:49:00.486 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049949888999999997, 2024-09-16T20:49:00.486 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:00.486 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:00.486 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:00.486 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:00.486 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:00.486 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:00.486 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2203 2024-09-16T20:49:00.486 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.486 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:00.487 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.487 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:00.487 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336000+0000", 2024-09-16T20:49:00.487 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:00.487 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.487 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.487 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:00.487 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336000+0000", 2024-09-16T20:49:00.487 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:00.487 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.487 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.487 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:00.487 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336000+0000", 2024-09-16T20:49:00.488 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.58e-07 2024-09-16T20:49:00.488 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.488 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.488 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:00.488 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336002+0000", 2024-09-16T20:49:00.488 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0040000000000002e-06 2024-09-16T20:49:00.488 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.488 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.488 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:00.488 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336009+0000", 2024-09-16T20:49:00.488 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9890000000000003e-06 2024-09-16T20:49:00.488 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.488 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.488 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:00.489 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336014+0000", 2024-09-16T20:49:00.489 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4e-06 2024-09-16T20:49:00.489 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.489 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.489 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:00.489 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385759+0000", 2024-09-16T20:49:00.489 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049744151 2024-09-16T20:49:00.489 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.489 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.489 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:00.489 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385847+0000", 2024-09-16T20:49:00.489 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8189000000000006e-05 2024-09-16T20:49:00.489 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.489 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.490 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:00.490 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385950+0000", 2024-09-16T20:49:00.490 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102698 2024-09-16T20:49:00.490 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:00.490 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:00.490 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:00.490 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.490 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.490 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2206 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1046528~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:00.490 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:49:00.490 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 38.081375846999997, 2024-09-16T20:49:00.490 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050147714000000003, 2024-09-16T20:49:00.490 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:00.490 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:00.490 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:00.490 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:00.491 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:00.491 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:00.491 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2206 2024-09-16T20:49:00.491 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.491 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:00.491 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.491 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:00.491 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:49:00.491 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:00.491 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.491 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.491 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:00.491 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:49:00.491 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:00.491 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.492 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.492 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:00.492 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:49:00.492 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8599999999999999e-07 2024-09-16T20:49:00.492 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.492 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.492 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:00.492 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336024+0000", 2024-09-16T20:49:00.492 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.934e-06 2024-09-16T20:49:00.492 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.492 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.492 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:00.493 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336037+0000", 2024-09-16T20:49:00.493 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2496e-05 2024-09-16T20:49:00.493 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.493 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.493 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:00.493 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336042+0000", 2024-09-16T20:49:00.493 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4809999999999997e-06 2024-09-16T20:49:00.493 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.493 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.493 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:00.493 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385980+0000", 2024-09-16T20:49:00.493 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049938376999999999 2024-09-16T20:49:00.494 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.494 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.494 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:00.494 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.386069+0000", 2024-09-16T20:49:00.494 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8368999999999994e-05 2024-09-16T20:49:00.494 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.494 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.494 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:00.494 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.386170+0000", 2024-09-16T20:49:00.495 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000100671 2024-09-16T20:49:00.495 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:00.495 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:00.495 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:00.495 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.495 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.495 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5439 3.a 3:5789f5c3:::rbd_data.110ed9ca540.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 1181696~60416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:00.495 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:49:00.495 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 37.552687267000003, 2024-09-16T20:49:00.495 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10102045599999999, 2024-09-16T20:49:00.495 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:00.495 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:00.496 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:00.496 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:00.496 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:00.496 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:00.496 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5439 2024-09-16T20:49:00.496 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.496 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:00.496 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.496 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:00.496 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:49:00.496 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:00.496 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.496 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.496 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:00.497 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:49:00.497 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:00.497 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.497 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.497 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:00.497 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864711+0000", 2024-09-16T20:49:00.497 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7899999999999997e-07 2024-09-16T20:49:00.497 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.497 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.497 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:00.497 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864760+0000", 2024-09-16T20:49:00.497 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8803e-05 2024-09-16T20:49:00.497 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.497 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.498 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:00.498 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864769+0000", 2024-09-16T20:49:00.498 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0999999999999993e-06 2024-09-16T20:49:00.498 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.498 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.498 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:00.498 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864774+0000", 2024-09-16T20:49:00.498 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8919999999999999e-06 2024-09-16T20:49:00.498 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.498 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.498 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:00.498 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864815+0000", 2024-09-16T20:49:00.499 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1307000000000003e-05 2024-09-16T20:49:00.499 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.499 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.499 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:00.499 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864893+0000", 2024-09-16T20:49:00.499 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7581000000000001e-05 2024-09-16T20:49:00.499 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.499 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.499 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 1", 2024-09-16T20:49:00.499 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864996+0000", 2024-09-16T20:49:00.499 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102725 2024-09-16T20:49:00.499 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.499 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.499 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:00.500 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965442+0000", 2024-09-16T20:49:00.500 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100446012 2024-09-16T20:49:00.500 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.500 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.500 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:00.500 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965618+0000", 2024-09-16T20:49:00.500 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00017653100000000001 2024-09-16T20:49:00.500 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.500 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.500 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:00.500 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965682+0000", 2024-09-16T20:49:00.500 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3905999999999994e-05 2024-09-16T20:49:00.500 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.500 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.501 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:00.501 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965731+0000", 2024-09-16T20:49:00.501 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8819999999999997e-05 2024-09-16T20:49:00.501 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:00.501 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:00.501 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:00.501 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.501 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.501 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4600.0:5439 3.a e37/23 ondisk, result = 0)", 2024-09-16T20:49:00.501 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:49:00.501 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 37.551691118999997, 2024-09-16T20:49:00.501 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099742634999999996, 2024-09-16T20:49:00.501 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:00.502 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:00.502 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:00.502 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:00.502 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.502 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:00.502 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:49:00.502 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:00.502 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.502 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.502 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:00.502 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:49:00.502 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:00.502 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.503 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.503 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:00.503 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:49:00.503 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0900000000000001e-07 2024-09-16T20:49:00.503 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.503 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.503 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:00.503 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865711+0000", 2024-09-16T20:49:00.503 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3739999999999998e-06 2024-09-16T20:49:00.503 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.503 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.503 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:00.503 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965343+0000", 2024-09-16T20:49:00.504 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099632526999999999 2024-09-16T20:49:00.504 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.504 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.504 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:00.504 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965357+0000", 2024-09-16T20:49:00.504 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3713999999999999e-05 2024-09-16T20:49:00.504 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.504 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.504 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:00.504 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965426+0000", 2024-09-16T20:49:00.504 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8745999999999995e-05 2024-09-16T20:49:00.504 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.505 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.505 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:00.505 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965434+0000", 2024-09-16T20:49:00.505 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6910000000000003e-06 2024-09-16T20:49:00.505 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.505 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.505 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:00.505 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965449+0000", 2024-09-16T20:49:00.505 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4874e-05 2024-09-16T20:49:00.505 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:00.505 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:00.505 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:00.505 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.505 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.506 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4882.0:2508 3.12 3:4992771f:::rbd_object_map.110e7c8851c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T20:49:00.506 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:49:00.506 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 13.935887571, 2024-09-16T20:49:00.506 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.125007011, 2024-09-16T20:49:00.506 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:00.506 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:00.506 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:00.506 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:00.506 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4882", 2024-09-16T20:49:00.506 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2436923223", 2024-09-16T20:49:00.506 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2508 2024-09-16T20:49:00.506 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.506 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:00.507 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.507 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:00.507 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:49:00.507 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:00.507 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.507 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.507 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:00.507 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:49:00.507 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:00.507 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.507 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.507 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:00.507 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481511+0000", 2024-09-16T20:49:00.507 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0699999999999996e-07 2024-09-16T20:49:00.508 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.508 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.508 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:00.508 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481515+0000", 2024-09-16T20:49:00.508 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0609999999999997e-06 2024-09-16T20:49:00.508 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.508 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.508 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:00.508 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481523+0000", 2024-09-16T20:49:00.508 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1310000000000006e-06 2024-09-16T20:49:00.508 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.508 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.508 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:00.509 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481528+0000", 2024-09-16T20:49:00.509 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4900000000000002e-06 2024-09-16T20:49:00.509 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.509 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.509 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:00.509 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481565+0000", 2024-09-16T20:49:00.509 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7066000000000001e-05 2024-09-16T20:49:00.509 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.509 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.509 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:00.509 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481628+0000", 2024-09-16T20:49:00.509 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3118000000000001e-05 2024-09-16T20:49:00.509 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.510 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.510 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 2", 2024-09-16T20:49:00.510 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481822+0000", 2024-09-16T20:49:00.510 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019446200000000001 2024-09-16T20:49:00.510 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.510 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.510 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:00.510 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605988+0000", 2024-09-16T20:49:00.510 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.124165888 2024-09-16T20:49:00.510 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.510 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.510 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:00.510 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606427+0000", 2024-09-16T20:49:00.510 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00043854899999999999 2024-09-16T20:49:00.510 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.511 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.511 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:00.511 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606476+0000", 2024-09-16T20:49:00.511 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8971999999999998e-05 2024-09-16T20:49:00.511 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.511 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.511 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:00.511 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606517+0000", 2024-09-16T20:49:00.511 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1467000000000001e-05 2024-09-16T20:49:00.511 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:00.511 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:00.511 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:00.511 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.512 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.512 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4882.0:2508 3.12 e57/55 ondisk, result = 0)", 2024-09-16T20:49:00.512 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:49:00.512 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 13.934899071, 2024-09-16T20:49:00.512 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12349711400000001, 2024-09-16T20:49:00.512 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:00.512 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:00.512 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:00.512 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:00.512 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.512 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:00.512 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:49:00.512 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:00.513 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.513 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.513 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:00.513 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:49:00.513 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:00.513 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.513 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.513 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:00.513 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:49:00.513 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.06e-07 2024-09-16T20:49:00.513 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.513 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.513 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:00.513 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482502+0000", 2024-09-16T20:49:00.514 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.01e-06 2024-09-16T20:49:00.514 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.514 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.514 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:00.514 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605772+0000", 2024-09-16T20:49:00.514 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.123269687 2024-09-16T20:49:00.514 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.514 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.514 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:00.514 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605791+0000", 2024-09-16T20:49:00.514 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8912000000000001e-05 2024-09-16T20:49:00.514 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.514 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.514 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:00.514 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605974+0000", 2024-09-16T20:49:00.515 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000182836 2024-09-16T20:49:00.515 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.515 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.515 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:00.515 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605982+0000", 2024-09-16T20:49:00.515 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1699999999999997e-06 2024-09-16T20:49:00.515 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.515 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.515 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:00.515 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605996+0000", 2024-09-16T20:49:00.515 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3893000000000001e-05 2024-09-16T20:49:00.515 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:00.515 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:00.515 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:00.516 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.516 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.516 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4882.0:2512 3.f 3:f4d895b7:::rbd_data.110e7c8851c.0000000000000000:head [read 16384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T20:49:00.516 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:49:00.516 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 13.934462041, 2024-09-16T20:49:00.516 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12318483, 2024-09-16T20:49:00.516 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:00.516 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:00.516 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:00.516 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:00.516 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4882", 2024-09-16T20:49:00.516 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2436923223", 2024-09-16T20:49:00.516 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2512 2024-09-16T20:49:00.516 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.517 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:00.517 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.517 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:00.517 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:49:00.517 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:00.517 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.517 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.517 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:00.517 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:49:00.517 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:00.517 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.517 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.517 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:00.517 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:49:00.518 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3499999999999995e-07 2024-09-16T20:49:00.518 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.518 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.518 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:00.518 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482943+0000", 2024-09-16T20:49:00.518 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3450000000000002e-06 2024-09-16T20:49:00.518 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.518 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.518 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:00.518 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605772+0000", 2024-09-16T20:49:00.518 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.122829796 2024-09-16T20:49:00.518 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.518 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.518 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:00.519 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605792+0000", 2024-09-16T20:49:00.519 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9916999999999999e-05 2024-09-16T20:49:00.519 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.519 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.519 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:00.519 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605917+0000", 2024-09-16T20:49:00.519 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00012500799999999999 2024-09-16T20:49:00.519 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.519 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.519 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:00.519 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606004+0000", 2024-09-16T20:49:00.519 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6242999999999994e-05 2024-09-16T20:49:00.519 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:00.520 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:00.520 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:00.520 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606120+0000", 2024-09-16T20:49:00.520 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000116786 2024-09-16T20:49:00.520 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:00.520 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:00.520 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:00.520 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:00.520 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:00.520 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:00.521 INFO:teuthology.orchestra.run.smithi139.stdout:2528 clone 19 order 18 su 8192 sc 12 2024-09-16T20:49:00.522 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:49:00.522 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:00.521+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:00.622 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:00.622+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:00.670 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:00.670 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:49:00.670 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:49:00.670 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:00.678 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:49:00.723 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:00.723+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:00.728 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T20:49:00.750 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 3] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-16T20:49:00.750 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-16T20:49:00.761 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T20:49:00.824 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:00.824+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:00.860 INFO:teuthology.orchestra.run.smithi139.stdout:truncating image image_client.0-clone18 from 0x7a3bc00 (overlap 0x7a3bc00) to 0x0 2024-09-16T20:49:00.862 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:00.862 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:49:00.862 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:49:00.862 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:49:00.862 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:00.875 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:49:00.925 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:00.925+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:00.946 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:49:00.947 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:49:00.968 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:49:00.968 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:49:00.998 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:49:00.998 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:49:01.019 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:49:01.019 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:49:01.026 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:01.026+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:01.037 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:01.037 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:01.047 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:49:01.047 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:49:01.049 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:49:01.068 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:49:01.069 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:49:01.126 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:01.126+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:01.181 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:01.181 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:01.190 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:49:01.228 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:01.227+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:01.329 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:01.328+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:01.347 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:01.347 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:01.356 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:49:01.401 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #17, image image_client.0-clone17 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent18 2024-09-16T20:49:01.430 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:01.429+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:01.471 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:01.471 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:01.485 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:49:01.531 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:01.530+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:01.598 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:01.599 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:01.607 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:49:01.650 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:01.650+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:01.732 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:01.732+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:01.737 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:01.737 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:01.748 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:49:01.755 INFO:teuthology.orchestra.run.smithi029.stderr:reweighted osd.0 to 1 (10000) 2024-09-16T20:49:01.755 INFO:teuthology.orchestra.run.smithi029.stderr:noscrub is unset 2024-09-16T20:49:01.775 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-16T20:49:01.858 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:01.858+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:01.914 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:01.914 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:01.924 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:49:01.934 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:01.934+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:02.035 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:02.035+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:02.050 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:02.050 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:02.061 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:49:02.136 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:02.136+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:02.174 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:02.174 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:02.183 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:49:02.238 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:02.237+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:02.326 INFO:teuthology.orchestra.run.smithi139.stdout:2529 trunc from 0x7a3bc00 to 0x454ba00 2024-09-16T20:49:02.330 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:02.330 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:02.339 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:49:02.339 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:02.337+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:02.390 INFO:teuthology.orchestra.run.smithi139.stdout:2534 read 0x19b5800 thru 0x19c1c7f (0xc480 bytes) 2024-09-16T20:49:02.392 INFO:teuthology.orchestra.run.smithi139.stdout:2535 flatten 2024-09-16T20:49:02.439 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:02.438+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:02.486 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:02.486 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:02.494 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:49:02.540 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:02.539+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:02.601 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:02.601 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:02.610 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:49:02.641 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:02.641+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:02.742 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:02.742+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:02.760 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:03.621 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:03.622 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T20:49:03.625 INFO:teuthology.orchestra.run.smithi029.stderr:reweighted osd.1 to 1 (10000) 2024-09-16T20:49:03.625 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:03.623+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:03.626 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-16T20:49:03.626 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:49:03.711 INFO:teuthology.orchestra.run.smithi139.stdout:2536 read 0x41fa200 thru 0x41fa4ab (0x2ac bytes) 2024-09-16T20:49:03.712 INFO:teuthology.orchestra.run.smithi139.stdout:2537 read 0xb9c000 thru 0xba6c4b (0xac4c bytes) 2024-09-16T20:49:03.712 INFO:teuthology.orchestra.run.smithi139.stdout:2542 read 0x89cc00 thru 0x89cd50 (0x151 bytes) 2024-09-16T20:49:03.725 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:03.724+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:03.732 INFO:teuthology.orchestra.run.smithi139.stdout:2543 write 0xa708800 thru 0xa70b279 (0x2a7a bytes) 2024-09-16T20:49:03.733 INFO:teuthology.orchestra.run.smithi139.stdout:2547 write 0x5721600 thru 0x5728a50 (0x7451 bytes) 2024-09-16T20:49:03.733 INFO:teuthology.orchestra.run.smithi139.stdout:2548 write 0x9fdcc00 thru 0x9fe81c8 (0xb5c9 bytes) 2024-09-16T20:49:03.733 INFO:teuthology.orchestra.run.smithi139.stdout:2553 write 0x80c8800 thru 0x80cfb5e (0x735f bytes) 2024-09-16T20:49:03.774 INFO:teuthology.orchestra.run.smithi139.stdout:2556 write 0xde2f400 thru 0xde3dee5 (0xeae6 bytes) 2024-09-16T20:49:03.774 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T20:49:03.775 INFO:teuthology.orchestra.run.smithi139.stdout:2557 write 0x49bca00 thru 0x49bd11e (0x71f bytes) 2024-09-16T20:49:03.775 INFO:teuthology.orchestra.run.smithi139.stdout:2559 write 0x55b8400 thru 0x55c7bcc (0xf7cd bytes) 2024-09-16T20:49:03.775 INFO:teuthology.orchestra.run.smithi139.stdout:2560 punch from 0x6b7ea00 to 0x6b8cc00, (0xe200 bytes) 2024-09-16T20:49:03.776 INFO:teuthology.orchestra.run.smithi139.stdout:2562 write 0x3523e00 thru 0x352d976 (0x9b77 bytes) 2024-09-16T20:49:03.777 INFO:teuthology.orchestra.run.smithi139.stdout:2563 write 0x43c7c00 thru 0x43d07d4 (0x8bd5 bytes) 2024-09-16T20:49:03.777 INFO:teuthology.orchestra.run.smithi139.stdout:2565 read 0x861d000 thru 0x8624bb0 (0x7bb1 bytes) 2024-09-16T20:49:03.777 INFO:teuthology.orchestra.run.smithi139.stdout:skipping zero length punch hole 2024-09-16T20:49:03.779 INFO:teuthology.orchestra.run.smithi139.stdout:2570 write 0xb080200 thru 0xb086405 (0x6206 bytes) 2024-09-16T20:49:03.779 INFO:teuthology.orchestra.run.smithi139.stdout:2571 read 0x86fd200 thru 0x870993d (0xc73e bytes) 2024-09-16T20:49:03.779 INFO:teuthology.orchestra.run.smithi139.stdout:2572 write 0x99aa600 thru 0x99b25cd (0x7fce bytes) 2024-09-16T20:49:03.779 INFO:teuthology.orchestra.run.smithi139.stdout:2574 write 0x76c6e00 thru 0x76d3afb (0xccfc bytes) 2024-09-16T20:49:03.780 INFO:teuthology.orchestra.run.smithi139.stdout:2575 write 0x6f04600 thru 0x6f055e1 (0xfe2 bytes) 2024-09-16T20:49:03.780 INFO:teuthology.orchestra.run.smithi139.stdout:2576 punch from 0x99eba00 to 0x99f2800, (0x6e00 bytes) 2024-09-16T20:49:03.780 INFO:teuthology.orchestra.run.smithi139.stdout:2577 write 0x9075200 thru 0x908074e (0xb54f bytes) 2024-09-16T20:49:03.780 INFO:teuthology.orchestra.run.smithi139.stdout:2578 read 0x6cf3e00 thru 0x6d01305 (0xd506 bytes) 2024-09-16T20:49:03.815 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:03.815 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:03.826 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:03.825+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:03.829 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:49:03.847 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:49:03.848 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:49:03.876 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:49:03.876 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:49:03.879 INFO:teuthology.orchestra.run.smithi139.stdout:2580 write 0xe323800 thru 0xe32b086 (0x7887 bytes) 2024-09-16T20:49:03.882 INFO:teuthology.orchestra.run.smithi139.stdout:2581 write 0x5b33200 thru 0x5b33e26 (0xc27 bytes) 2024-09-16T20:49:03.882 INFO:teuthology.orchestra.run.smithi139.stdout:2582 write 0x4fb2600 thru 0x4fb7790 (0x5191 bytes) 2024-09-16T20:49:03.882 INFO:teuthology.orchestra.run.smithi139.stdout:2583 read 0xaff0c00 thru 0xaff6f94 (0x6395 bytes) 2024-09-16T20:49:03.883 INFO:teuthology.orchestra.run.smithi139.stdout:2584 write 0x2c31600 thru 0x2c33326 (0x1d27 bytes) 2024-09-16T20:49:03.883 INFO:teuthology.orchestra.run.smithi139.stdout:2585 write 0xb3b2200 thru 0xb3b75e5 (0x53e6 bytes) 2024-09-16T20:49:03.883 INFO:teuthology.orchestra.run.smithi139.stdout:2586 write 0xa062e00 thru 0xa06b342 (0x8543 bytes) 2024-09-16T20:49:03.883 INFO:teuthology.orchestra.run.smithi139.stdout:2589 read 0x8b8b800 thru 0x8b9620d (0xaa0e bytes) 2024-09-16T20:49:03.883 INFO:teuthology.orchestra.run.smithi139.stdout:2590 read 0x3a4d600 thru 0x3a569d4 (0x93d5 bytes) 2024-09-16T20:49:03.883 INFO:teuthology.orchestra.run.smithi139.stdout:2591 punch from 0x3ff1000 to 0x3ff4c00, (0x3c00 bytes) 2024-09-16T20:49:03.883 INFO:teuthology.orchestra.run.smithi139.stdout:2592 punch from 0x1e13a00 to 0x1e13e00, (0x400 bytes) 2024-09-16T20:49:03.884 INFO:teuthology.orchestra.run.smithi139.stdout:2593 read 0x7f45a00 thru 0x7f548fa (0xeefb bytes) 2024-09-16T20:49:03.884 INFO:teuthology.orchestra.run.smithi139.stdout:2595 write 0xca02e00 thru 0xca09426 (0x6627 bytes) 2024-09-16T20:49:03.907 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:49:03.908 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:49:03.926 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:03.926+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:03.938 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:49:03.939 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:49:03.940 INFO:teuthology.orchestra.run.smithi139.stdout:2596 write 0xe964a00 thru 0xe971a1d (0xd01e bytes) 2024-09-16T20:49:03.940 INFO:teuthology.orchestra.run.smithi139.stdout:2597 read 0x79b3800 thru 0x79c1dad (0xe5ae bytes) 2024-09-16T20:49:03.940 INFO:teuthology.orchestra.run.smithi139.stdout:2599 read 0xe5dc200 thru 0xe5eb313 (0xf114 bytes) 2024-09-16T20:49:03.941 INFO:teuthology.orchestra.run.smithi139.stdout:2600 write 0x2542000 thru 0x254f2c7 (0xd2c8 bytes) 2024-09-16T20:49:03.941 INFO:teuthology.orchestra.run.smithi139.stdout:2601 read 0xc5bb200 thru 0xc5bb6fc (0x4fd bytes) 2024-09-16T20:49:03.941 INFO:teuthology.orchestra.run.smithi139.stdout:2602 read 0x9797e00 thru 0x979ea0f (0x6c10 bytes) 2024-09-16T20:49:03.942 INFO:teuthology.orchestra.run.smithi139.stdout:2605 write 0x4110c00 thru 0x4118377 (0x7778 bytes) 2024-09-16T20:49:03.942 INFO:teuthology.orchestra.run.smithi139.stdout:2611 read 0x5168800 thru 0x516bb87 (0x3388 bytes) 2024-09-16T20:49:03.953 INFO:teuthology.orchestra.run.smithi139.stdout:2612 trunc from 0xe971c00 to 0x4053400 2024-09-16T20:49:03.963 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:49:03.964 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:49:03.991 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:49:03.991 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:49:04.016 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:04.016 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:49:04.016 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:49:04.016 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:49:04.016 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:04.027 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:04.027+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:04.029 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:49:04.096 INFO:teuthology.orchestra.run.smithi139.stdout:2613 punch from 0xd4e400 to 0xd52600, (0x4200 bytes) 2024-09-16T20:49:04.098 INFO:teuthology.orchestra.run.smithi139.stdout:2615 trunc from 0x4053400 to 0x2607a00 2024-09-16T20:49:04.129 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:04.128+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:04.133 INFO:teuthology.orchestra.run.smithi139.stdout:2618 read 0x29d800 thru 0x29f24f (0x1a50 bytes) 2024-09-16T20:49:04.134 INFO:teuthology.orchestra.run.smithi139.stdout:2620 punch from 0x79d000 to 0x7a1a00, (0x4a00 bytes) 2024-09-16T20:49:04.198 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:04.198 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:49:04.198 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:49:04.198 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:49:04.198 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.199 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2174 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 931840~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:04.199 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335679+0000", 2024-09-16T20:49:04.199 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 41.860203167000002, 2024-09-16T20:49:04.199 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047619511000000003, 2024-09-16T20:49:04.199 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:04.199 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:04.199 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:04.199 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:04.199 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:04.199 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:04.200 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2174 2024-09-16T20:49:04.200 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.200 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:04.200 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.200 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:04.200 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335679+0000", 2024-09-16T20:49:04.200 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.200 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.200 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.200 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:04.200 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335679+0000", 2024-09-16T20:49:04.200 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.200 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.201 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.201 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:04.201 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335679+0000", 2024-09-16T20:49:04.201 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7800000000000002e-07 2024-09-16T20:49:04.201 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.201 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.201 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:04.201 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335682+0000", 2024-09-16T20:49:04.201 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9490000000000001e-06 2024-09-16T20:49:04.201 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.202 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.202 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:04.202 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335689+0000", 2024-09-16T20:49:04.202 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8220000000000001e-06 2024-09-16T20:49:04.202 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.202 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.202 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:04.202 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335695+0000", 2024-09-16T20:49:04.202 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0569999999999998e-06 2024-09-16T20:49:04.202 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.202 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.202 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:04.203 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383103+0000", 2024-09-16T20:49:04.203 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047408219000000001 2024-09-16T20:49:04.203 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.203 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.203 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:04.203 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383189+0000", 2024-09-16T20:49:04.203 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6007000000000005e-05 2024-09-16T20:49:04.203 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.203 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.203 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:04.203 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383298+0000", 2024-09-16T20:49:04.204 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000109079 2024-09-16T20:49:04.204 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.204 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:04.204 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.204 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.204 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.204 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2176 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 940032~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:04.204 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335702+0000", 2024-09-16T20:49:04.205 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 41.860179938999998, 2024-09-16T20:49:04.205 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047804450999999998, 2024-09-16T20:49:04.205 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:04.205 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:04.205 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:04.205 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:04.205 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:04.205 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:04.205 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2176 2024-09-16T20:49:04.205 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.205 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:04.205 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.205 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:04.206 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335702+0000", 2024-09-16T20:49:04.206 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.206 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.206 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.206 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:04.206 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335702+0000", 2024-09-16T20:49:04.206 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.206 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.206 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.206 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:04.206 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335703+0000", 2024-09-16T20:49:04.206 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.34e-07 2024-09-16T20:49:04.206 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.207 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.207 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:04.207 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335704+0000", 2024-09-16T20:49:04.207 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.917e-06 2024-09-16T20:49:04.207 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.207 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.207 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:04.207 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335711+0000", 2024-09-16T20:49:04.207 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4400000000000002e-06 2024-09-16T20:49:04.207 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.207 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.207 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:04.207 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335717+0000", 2024-09-16T20:49:04.207 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6679999999999998e-06 2024-09-16T20:49:04.208 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.208 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.208 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:04.208 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383335+0000", 2024-09-16T20:49:04.208 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047618754999999999 2024-09-16T20:49:04.208 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.208 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.208 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:04.208 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383405+0000", 2024-09-16T20:49:04.208 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9286e-05 2024-09-16T20:49:04.208 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.208 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.209 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:04.209 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383507+0000", 2024-09-16T20:49:04.209 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101951 2024-09-16T20:49:04.209 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.209 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:04.209 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.209 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.209 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.209 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2179 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 948224~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:04.210 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335723+0000", 2024-09-16T20:49:04.210 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 41.860158845000001, 2024-09-16T20:49:04.210 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048007181000000003, 2024-09-16T20:49:04.210 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:04.210 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:04.210 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:04.210 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:04.210 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:04.210 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:04.210 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2179 2024-09-16T20:49:04.210 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.211 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:04.211 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.211 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:04.211 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335723+0000", 2024-09-16T20:49:04.211 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.211 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.211 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.211 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:04.211 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335723+0000", 2024-09-16T20:49:04.211 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.211 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.212 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.212 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:04.212 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335724+0000", 2024-09-16T20:49:04.212 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.58e-07 2024-09-16T20:49:04.212 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.212 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.212 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:04.212 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335726+0000", 2024-09-16T20:49:04.212 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9120000000000001e-06 2024-09-16T20:49:04.212 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.212 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.212 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:04.212 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335732+0000", 2024-09-16T20:49:04.213 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6080000000000001e-06 2024-09-16T20:49:04.213 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.213 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.213 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:04.213 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335737+0000", 2024-09-16T20:49:04.213 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.22e-06 2024-09-16T20:49:04.213 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.213 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.213 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:04.213 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383556+0000", 2024-09-16T20:49:04.214 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047819002999999999 2024-09-16T20:49:04.214 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.214 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.214 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:04.214 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383635+0000", 2024-09-16T20:49:04.214 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9060999999999994e-05 2024-09-16T20:49:04.214 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.214 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.214 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:04.214 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383730+0000", 2024-09-16T20:49:04.214 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4919000000000004e-05 2024-09-16T20:49:04.214 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.214 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:04.215 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.215 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.215 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.215 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2180 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 956416~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:04.215 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335744+0000", 2024-09-16T20:49:04.215 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 41.860138397, 2024-09-16T20:49:04.215 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048255147999999998, 2024-09-16T20:49:04.215 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:04.215 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:04.215 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:04.216 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:04.216 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:04.216 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:04.216 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2180 2024-09-16T20:49:04.216 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.216 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:04.216 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.216 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:04.216 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335744+0000", 2024-09-16T20:49:04.216 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.217 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.217 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.217 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:04.217 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335744+0000", 2024-09-16T20:49:04.217 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.217 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.217 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.217 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:04.217 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335744+0000", 2024-09-16T20:49:04.217 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9000000000000002e-07 2024-09-16T20:49:04.217 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.218 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.218 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:04.218 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335746+0000", 2024-09-16T20:49:04.218 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7570000000000001e-06 2024-09-16T20:49:04.218 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.218 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.218 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:04.218 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335752+0000", 2024-09-16T20:49:04.218 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0079999999999999e-06 2024-09-16T20:49:04.218 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.218 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.218 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:04.219 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335757+0000", 2024-09-16T20:49:04.219 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1649999999999997e-06 2024-09-16T20:49:04.219 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.219 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.219 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:04.219 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383803+0000", 2024-09-16T20:49:04.219 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048046091999999999 2024-09-16T20:49:04.219 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.219 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.219 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:04.220 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383889+0000", 2024-09-16T20:49:04.220 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6048e-05 2024-09-16T20:49:04.220 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.220 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.220 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:04.220 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383999+0000", 2024-09-16T20:49:04.220 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000109688 2024-09-16T20:49:04.220 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.220 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:04.220 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.220 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.220 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.221 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2182 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 964608~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:04.221 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335773+0000", 2024-09-16T20:49:04.221 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 41.860108590000003, 2024-09-16T20:49:04.221 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048467116999999997, 2024-09-16T20:49:04.221 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:04.221 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:04.221 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:04.221 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:04.221 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:04.221 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:04.222 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2182 2024-09-16T20:49:04.222 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.222 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:04.222 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.222 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:04.222 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335773+0000", 2024-09-16T20:49:04.222 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.222 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.222 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.222 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:04.223 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335773+0000", 2024-09-16T20:49:04.223 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.223 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.223 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.223 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:04.223 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335774+0000", 2024-09-16T20:49:04.223 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3799999999999997e-07 2024-09-16T20:49:04.223 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.223 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.224 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:04.224 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335776+0000", 2024-09-16T20:49:04.224 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.227e-06 2024-09-16T20:49:04.224 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.224 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.224 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:04.224 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335790+0000", 2024-09-16T20:49:04.224 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4119e-05 2024-09-16T20:49:04.224 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.224 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.224 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:04.224 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335796+0000", 2024-09-16T20:49:04.225 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.694e-06 2024-09-16T20:49:04.225 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.225 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.225 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:04.225 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384058+0000", 2024-09-16T20:49:04.225 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048261983000000001 2024-09-16T20:49:04.225 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.225 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.225 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:04.225 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384148+0000", 2024-09-16T20:49:04.225 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9501999999999998e-05 2024-09-16T20:49:04.226 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.226 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.226 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:04.226 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384241+0000", 2024-09-16T20:49:04.226 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.3053999999999994e-05 2024-09-16T20:49:04.226 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.226 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:04.226 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.226 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.226 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.226 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2185 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 972800~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:04.227 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335804+0000", 2024-09-16T20:49:04.227 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 41.860077939999996, 2024-09-16T20:49:04.227 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048570818000000002, 2024-09-16T20:49:04.227 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:04.227 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:04.227 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:04.227 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:04.227 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:04.227 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:04.227 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2185 2024-09-16T20:49:04.227 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.227 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:04.228 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.228 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:04.228 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335804+0000", 2024-09-16T20:49:04.228 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.228 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.228 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.228 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:04.228 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335804+0000", 2024-09-16T20:49:04.228 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.228 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.228 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.229 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:04.229 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335805+0000", 2024-09-16T20:49:04.229 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8299999999999997e-07 2024-09-16T20:49:04.229 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.229 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.229 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:04.229 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335813+0000", 2024-09-16T20:49:04.229 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6309999999999992e-06 2024-09-16T20:49:04.229 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.229 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.229 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:04.230 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335821+0000", 2024-09-16T20:49:04.230 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0169999999999996e-06 2024-09-16T20:49:04.230 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.230 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.230 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:04.230 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335827+0000", 2024-09-16T20:49:04.230 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0100000000000001e-06 2024-09-16T20:49:04.230 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.230 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.230 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:04.230 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384273+0000", 2024-09-16T20:49:04.231 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048445333 2024-09-16T20:49:04.231 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.231 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.231 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:04.231 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384315+0000", 2024-09-16T20:49:04.231 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2472999999999998e-05 2024-09-16T20:49:04.231 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.231 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.231 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:04.232 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384375+0000", 2024-09-16T20:49:04.232 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9871000000000003e-05 2024-09-16T20:49:04.232 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.232 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:04.232 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.232 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.232 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.232 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2187 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 980992~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:04.232 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335835+0000", 2024-09-16T20:49:04.232 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 41.860046863000001, 2024-09-16T20:49:04.232 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048684927000000003, 2024-09-16T20:49:04.233 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:04.233 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:04.233 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:04.233 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:04.233 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:04.233 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:04.233 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2187 2024-09-16T20:49:04.233 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.233 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:04.233 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.233 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:04.234 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335835+0000", 2024-09-16T20:49:04.234 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.234 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.234 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.234 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:04.234 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335835+0000", 2024-09-16T20:49:04.234 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.234 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.234 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.234 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:04.234 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335836+0000", 2024-09-16T20:49:04.235 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3599999999999999e-07 2024-09-16T20:49:04.235 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.235 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.235 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:04.235 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335838+0000", 2024-09-16T20:49:04.235 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.996e-06 2024-09-16T20:49:04.235 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.235 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.235 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:04.235 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335844+0000", 2024-09-16T20:49:04.236 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7889999999999997e-06 2024-09-16T20:49:04.236 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.236 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.236 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:04.236 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335849+0000", 2024-09-16T20:49:04.236 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7400000000000004e-06 2024-09-16T20:49:04.236 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.236 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.236 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:04.237 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384411+0000", 2024-09-16T20:49:04.237 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048562069999999999 2024-09-16T20:49:04.237 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.237 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.237 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:04.237 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384460+0000", 2024-09-16T20:49:04.237 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9125999999999997e-05 2024-09-16T20:49:04.237 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.237 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.237 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:04.237 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384520+0000", 2024-09-16T20:49:04.238 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9769999999999999e-05 2024-09-16T20:49:04.238 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.238 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:04.238 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.238 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.238 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.238 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2189 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 989184~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:04.238 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335856+0000", 2024-09-16T20:49:04.238 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 41.860025618000002, 2024-09-16T20:49:04.238 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048856525999999997, 2024-09-16T20:49:04.238 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:04.239 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:04.239 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:04.239 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:04.239 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:04.239 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:04.239 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2189 2024-09-16T20:49:04.239 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.239 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:04.239 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.239 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:04.239 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335856+0000", 2024-09-16T20:49:04.239 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.239 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.240 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.240 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:04.240 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335856+0000", 2024-09-16T20:49:04.240 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.240 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.240 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.240 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:04.240 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335857+0000", 2024-09-16T20:49:04.240 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6800000000000001e-07 2024-09-16T20:49:04.240 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.240 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.240 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:04.241 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335859+0000", 2024-09-16T20:49:04.241 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.013e-06 2024-09-16T20:49:04.241 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.241 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.241 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:04.241 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335866+0000", 2024-09-16T20:49:04.241 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8750000000000002e-06 2024-09-16T20:49:04.241 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.241 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.241 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:04.241 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335871+0000", 2024-09-16T20:49:04.242 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1719999999999998e-06 2024-09-16T20:49:04.242 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.242 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.242 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:04.242 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384550+0000", 2024-09-16T20:49:04.242 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048679476999999999 2024-09-16T20:49:04.242 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.242 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.242 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:04.242 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384611+0000", 2024-09-16T20:49:04.242 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0897000000000003e-05 2024-09-16T20:49:04.242 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.243 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.243 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:04.243 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384713+0000", 2024-09-16T20:49:04.243 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101724 2024-09-16T20:49:04.243 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.243 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:04.243 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.243 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.243 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.243 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2192 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 997376~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:04.243 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335878+0000", 2024-09-16T20:49:04.243 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 41.860003933000002, 2024-09-16T20:49:04.243 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049028769999999999, 2024-09-16T20:49:04.243 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:04.244 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:04.244 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:04.244 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:04.244 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:04.244 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:04.244 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2192 2024-09-16T20:49:04.244 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.244 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:04.244 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.245 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:04.245 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335878+0000", 2024-09-16T20:49:04.245 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.245 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.245 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.245 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:04.245 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335878+0000", 2024-09-16T20:49:04.245 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.245 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.245 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.245 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:04.245 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335879+0000", 2024-09-16T20:49:04.246 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6899999999999998e-07 2024-09-16T20:49:04.246 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.246 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.246 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:04.246 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335881+0000", 2024-09-16T20:49:04.246 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.996e-06 2024-09-16T20:49:04.246 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.246 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.246 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:04.246 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335887+0000", 2024-09-16T20:49:04.246 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6370000000000002e-06 2024-09-16T20:49:04.247 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.247 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.247 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:04.247 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335892+0000", 2024-09-16T20:49:04.247 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2120000000000002e-06 2024-09-16T20:49:04.247 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.247 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.247 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:04.247 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384747+0000", 2024-09-16T20:49:04.248 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048854923000000001 2024-09-16T20:49:04.248 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.248 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.248 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:04.248 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384830+0000", 2024-09-16T20:49:04.248 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2359000000000002e-05 2024-09-16T20:49:04.248 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.248 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.248 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:04.248 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384907+0000", 2024-09-16T20:49:04.248 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7174000000000003e-05 2024-09-16T20:49:04.249 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.249 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:04.249 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.249 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.249 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.249 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2194 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1005568~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:04.249 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335899+0000", 2024-09-16T20:49:04.249 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 41.859983012000001, 2024-09-16T20:49:04.249 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049225393999999999, 2024-09-16T20:49:04.249 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:04.249 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:04.249 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:04.250 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:04.250 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:04.250 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:04.250 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2194 2024-09-16T20:49:04.250 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.250 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:04.250 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.250 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:04.250 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335899+0000", 2024-09-16T20:49:04.250 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.250 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.251 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.251 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:04.251 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335899+0000", 2024-09-16T20:49:04.251 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.251 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.251 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.251 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:04.251 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335899+0000", 2024-09-16T20:49:04.251 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8200000000000001e-07 2024-09-16T20:49:04.251 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.251 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.252 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:04.252 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335901+0000", 2024-09-16T20:49:04.252 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9e-06 2024-09-16T20:49:04.252 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.252 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.252 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:04.252 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335908+0000", 2024-09-16T20:49:04.252 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3559999999999998e-06 2024-09-16T20:49:04.252 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.253 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.253 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:04.253 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335914+0000", 2024-09-16T20:49:04.253 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4729999999999997e-06 2024-09-16T20:49:04.253 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.253 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.253 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:04.253 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384979+0000", 2024-09-16T20:49:04.253 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049064545000000001 2024-09-16T20:49:04.254 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.254 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.254 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:04.254 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385060+0000", 2024-09-16T20:49:04.254 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0804999999999996e-05 2024-09-16T20:49:04.254 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.254 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.254 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:04.254 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385124+0000", 2024-09-16T20:49:04.255 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4932999999999996e-05 2024-09-16T20:49:04.255 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.255 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:04.255 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.255 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.255 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.255 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2197 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1013760~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:04.255 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335921+0000", 2024-09-16T20:49:04.255 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 41.859960731000001, 2024-09-16T20:49:04.255 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049342184999999997, 2024-09-16T20:49:04.255 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:04.255 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:04.256 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:04.256 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:04.256 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:04.256 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:04.256 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2197 2024-09-16T20:49:04.256 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.256 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:04.256 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.256 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:04.257 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335921+0000", 2024-09-16T20:49:04.257 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.257 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.257 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.257 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:04.257 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335921+0000", 2024-09-16T20:49:04.257 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.257 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.257 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.257 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:04.258 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335922+0000", 2024-09-16T20:49:04.258 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.32e-07 2024-09-16T20:49:04.258 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.258 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.258 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:04.258 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335924+0000", 2024-09-16T20:49:04.258 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.995e-06 2024-09-16T20:49:04.258 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.258 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.259 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:04.259 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335930+0000", 2024-09-16T20:49:04.259 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5139999999999998e-06 2024-09-16T20:49:04.259 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.259 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.259 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:04.259 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335936+0000", 2024-09-16T20:49:04.259 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6160000000000001e-06 2024-09-16T20:49:04.259 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.259 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.260 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:04.260 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385142+0000", 2024-09-16T20:49:04.260 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049206393000000001 2024-09-16T20:49:04.260 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.260 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.260 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:04.260 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385184+0000", 2024-09-16T20:49:04.260 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1928999999999998e-05 2024-09-16T20:49:04.261 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.261 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.261 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:04.261 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385264+0000", 2024-09-16T20:49:04.261 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9306000000000003e-05 2024-09-16T20:49:04.261 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.261 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:04.261 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.261 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.261 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.261 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2198 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1021952~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:04.261 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335942+0000", 2024-09-16T20:49:04.261 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 41.859939670999999, 2024-09-16T20:49:04.262 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049562483999999997, 2024-09-16T20:49:04.262 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:04.262 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:04.262 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:04.262 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:04.262 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:04.262 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:04.262 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2198 2024-09-16T20:49:04.262 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.262 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:04.262 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.262 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:04.262 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335942+0000", 2024-09-16T20:49:04.262 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.263 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.263 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.263 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:04.263 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335942+0000", 2024-09-16T20:49:04.263 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.263 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.263 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.263 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:04.263 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335943+0000", 2024-09-16T20:49:04.263 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7099999999999997e-07 2024-09-16T20:49:04.263 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.263 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.263 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:04.263 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335945+0000", 2024-09-16T20:49:04.264 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3499999999999999e-06 2024-09-16T20:49:04.264 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.264 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.264 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:04.264 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335952+0000", 2024-09-16T20:49:04.264 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4309999999999999e-06 2024-09-16T20:49:04.264 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.264 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.264 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:04.264 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335957+0000", 2024-09-16T20:49:04.264 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0869999999999997e-06 2024-09-16T20:49:04.265 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.265 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.265 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:04.265 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385312+0000", 2024-09-16T20:49:04.265 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049355121000000002 2024-09-16T20:49:04.265 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.265 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.265 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:04.265 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385405+0000", 2024-09-16T20:49:04.265 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.3344000000000006e-05 2024-09-16T20:49:04.265 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.265 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.266 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:04.266 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385505+0000", 2024-09-16T20:49:04.266 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.9779999999999997e-05 2024-09-16T20:49:04.266 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.266 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:04.266 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.266 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.266 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.266 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2201 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1030144~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:04.266 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335976+0000", 2024-09-16T20:49:04.266 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 41.859906229000003, 2024-09-16T20:49:04.267 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049740077000000001, 2024-09-16T20:49:04.267 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:04.267 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:04.267 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:04.267 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:04.267 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:04.267 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:04.267 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2201 2024-09-16T20:49:04.267 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.267 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:04.267 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.267 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:04.268 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335976+0000", 2024-09-16T20:49:04.268 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.268 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.268 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.268 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:04.268 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335976+0000", 2024-09-16T20:49:04.268 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.268 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.268 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.268 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:04.269 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335976+0000", 2024-09-16T20:49:04.269 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8599999999999998e-07 2024-09-16T20:49:04.269 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.269 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.269 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:04.269 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335978+0000", 2024-09-16T20:49:04.269 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1500000000000002e-06 2024-09-16T20:49:04.269 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.269 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.269 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:04.270 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335986+0000", 2024-09-16T20:49:04.270 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3989999999999998e-06 2024-09-16T20:49:04.270 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.270 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.270 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:04.270 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335992+0000", 2024-09-16T20:49:04.270 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.682e-06 2024-09-16T20:49:04.270 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.270 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.271 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:04.271 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385537+0000", 2024-09-16T20:49:04.271 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049545828 2024-09-16T20:49:04.271 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.271 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.271 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:04.271 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385619+0000", 2024-09-16T20:49:04.271 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1720999999999999e-05 2024-09-16T20:49:04.271 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.272 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.272 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:04.272 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385716+0000", 2024-09-16T20:49:04.272 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.6811000000000005e-05 2024-09-16T20:49:04.272 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.272 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:04.272 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.272 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.272 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.272 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2203 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1038336~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:04.272 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.336000+0000", 2024-09-16T20:49:04.272 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 41.859882443000004, 2024-09-16T20:49:04.273 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049949888999999997, 2024-09-16T20:49:04.273 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:04.273 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:04.273 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:04.273 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:04.273 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:04.273 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:04.273 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2203 2024-09-16T20:49:04.273 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.273 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:04.273 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.274 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:04.274 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336000+0000", 2024-09-16T20:49:04.274 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.274 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.274 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.274 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:04.274 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336000+0000", 2024-09-16T20:49:04.274 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.274 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.275 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.275 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:04.275 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336000+0000", 2024-09-16T20:49:04.275 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.58e-07 2024-09-16T20:49:04.275 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.275 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.275 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:04.275 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336002+0000", 2024-09-16T20:49:04.275 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0040000000000002e-06 2024-09-16T20:49:04.275 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.275 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.276 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:04.276 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336009+0000", 2024-09-16T20:49:04.276 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9890000000000003e-06 2024-09-16T20:49:04.276 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.276 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.276 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:04.276 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336014+0000", 2024-09-16T20:49:04.276 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4e-06 2024-09-16T20:49:04.276 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.276 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.276 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:04.276 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385759+0000", 2024-09-16T20:49:04.277 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049744151 2024-09-16T20:49:04.277 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.277 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.277 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:04.277 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385847+0000", 2024-09-16T20:49:04.277 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8189000000000006e-05 2024-09-16T20:49:04.277 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.277 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.277 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:04.277 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385950+0000", 2024-09-16T20:49:04.278 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102698 2024-09-16T20:49:04.278 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.278 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:04.278 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.278 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.278 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.278 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2206 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1046528~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:04.279 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:49:04.279 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 41.859860282, 2024-09-16T20:49:04.279 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050147714000000003, 2024-09-16T20:49:04.279 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:04.279 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:04.279 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:04.279 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:04.279 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:04.279 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:04.279 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2206 2024-09-16T20:49:04.279 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.279 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:04.280 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.280 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:04.280 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:49:04.280 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.280 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.280 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.280 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:04.280 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:49:04.280 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.280 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.281 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.281 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:04.281 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:49:04.281 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8599999999999999e-07 2024-09-16T20:49:04.281 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.281 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.281 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:04.281 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336024+0000", 2024-09-16T20:49:04.281 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.934e-06 2024-09-16T20:49:04.282 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.282 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.282 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:04.282 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336037+0000", 2024-09-16T20:49:04.282 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2496e-05 2024-09-16T20:49:04.282 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.282 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.282 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:04.282 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336042+0000", 2024-09-16T20:49:04.282 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4809999999999997e-06 2024-09-16T20:49:04.282 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.283 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.283 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:04.283 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385980+0000", 2024-09-16T20:49:04.283 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049938376999999999 2024-09-16T20:49:04.283 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.283 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.283 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:04.283 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.386069+0000", 2024-09-16T20:49:04.283 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8368999999999994e-05 2024-09-16T20:49:04.284 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.284 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.284 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:04.284 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.386170+0000", 2024-09-16T20:49:04.284 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000100671 2024-09-16T20:49:04.284 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.284 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:04.284 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.284 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.284 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.285 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5439 3.a 3:5789f5c3:::rbd_data.110ed9ca540.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 1181696~60416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:04.285 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:49:04.285 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 41.331171701999999, 2024-09-16T20:49:04.285 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10102045599999999, 2024-09-16T20:49:04.285 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:04.285 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:04.285 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:04.285 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:04.285 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:04.285 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:04.285 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5439 2024-09-16T20:49:04.286 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.286 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:04.286 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.286 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:04.286 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:49:04.286 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.286 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.286 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.286 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:04.286 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:49:04.287 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.287 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.287 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.287 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:04.287 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864711+0000", 2024-09-16T20:49:04.287 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7899999999999997e-07 2024-09-16T20:49:04.287 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.287 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.287 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:04.287 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864760+0000", 2024-09-16T20:49:04.287 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8803e-05 2024-09-16T20:49:04.288 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.288 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.288 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:04.288 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864769+0000", 2024-09-16T20:49:04.288 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0999999999999993e-06 2024-09-16T20:49:04.288 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.288 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.288 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:04.288 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864774+0000", 2024-09-16T20:49:04.289 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8919999999999999e-06 2024-09-16T20:49:04.289 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.289 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.289 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:04.289 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864815+0000", 2024-09-16T20:49:04.289 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1307000000000003e-05 2024-09-16T20:49:04.289 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.289 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.289 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:04.289 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864893+0000", 2024-09-16T20:49:04.290 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7581000000000001e-05 2024-09-16T20:49:04.290 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.290 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.290 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 1", 2024-09-16T20:49:04.290 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864996+0000", 2024-09-16T20:49:04.290 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102725 2024-09-16T20:49:04.290 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.290 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.290 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:04.290 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965442+0000", 2024-09-16T20:49:04.291 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100446012 2024-09-16T20:49:04.291 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.291 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.291 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:04.291 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965618+0000", 2024-09-16T20:49:04.291 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00017653100000000001 2024-09-16T20:49:04.291 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.292 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.292 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:04.292 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965682+0000", 2024-09-16T20:49:04.292 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3905999999999994e-05 2024-09-16T20:49:04.292 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.292 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.292 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:04.292 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965731+0000", 2024-09-16T20:49:04.292 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8819999999999997e-05 2024-09-16T20:49:04.293 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.293 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:04.293 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.293 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.293 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.293 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4600.0:5439 3.a e37/23 ondisk, result = 0)", 2024-09-16T20:49:04.293 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:49:04.293 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 41.330175554, 2024-09-16T20:49:04.293 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099742634999999996, 2024-09-16T20:49:04.293 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:04.294 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:04.294 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:04.294 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:04.294 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.294 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:04.294 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:49:04.294 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.294 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.294 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.294 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:04.294 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:49:04.294 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.294 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.295 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.295 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:04.295 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:49:04.295 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0900000000000001e-07 2024-09-16T20:49:04.295 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.295 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.295 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:04.295 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865711+0000", 2024-09-16T20:49:04.295 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3739999999999998e-06 2024-09-16T20:49:04.295 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.295 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.295 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:04.296 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965343+0000", 2024-09-16T20:49:04.296 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099632526999999999 2024-09-16T20:49:04.296 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.296 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.296 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:04.296 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965357+0000", 2024-09-16T20:49:04.296 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3713999999999999e-05 2024-09-16T20:49:04.296 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.296 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.297 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:04.297 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965426+0000", 2024-09-16T20:49:04.297 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8745999999999995e-05 2024-09-16T20:49:04.297 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.297 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.297 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:04.297 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965434+0000", 2024-09-16T20:49:04.297 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6910000000000003e-06 2024-09-16T20:49:04.297 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.297 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.297 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:04.298 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965449+0000", 2024-09-16T20:49:04.298 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4874e-05 2024-09-16T20:49:04.298 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.298 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:04.298 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.298 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.298 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.298 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4882.0:2508 3.12 3:4992771f:::rbd_object_map.110e7c8851c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T20:49:04.298 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:49:04.299 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 17.714372006000001, 2024-09-16T20:49:04.299 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.125007011, 2024-09-16T20:49:04.299 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:04.299 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:04.299 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:04.299 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:04.299 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4882", 2024-09-16T20:49:04.299 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2436923223", 2024-09-16T20:49:04.299 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2508 2024-09-16T20:49:04.299 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.299 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:04.300 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.300 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:04.300 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:49:04.300 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.300 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.300 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.300 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:04.300 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:49:04.300 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.300 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.301 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.301 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:04.301 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481511+0000", 2024-09-16T20:49:04.301 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0699999999999996e-07 2024-09-16T20:49:04.301 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.301 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.301 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:04.301 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481515+0000", 2024-09-16T20:49:04.301 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0609999999999997e-06 2024-09-16T20:49:04.302 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.302 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.302 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:04.302 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481523+0000", 2024-09-16T20:49:04.302 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1310000000000006e-06 2024-09-16T20:49:04.302 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.302 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.302 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:04.302 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481528+0000", 2024-09-16T20:49:04.303 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4900000000000002e-06 2024-09-16T20:49:04.303 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.303 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.303 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:04.303 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481565+0000", 2024-09-16T20:49:04.303 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7066000000000001e-05 2024-09-16T20:49:04.303 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.303 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.303 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:04.304 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481628+0000", 2024-09-16T20:49:04.304 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3118000000000001e-05 2024-09-16T20:49:04.304 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.304 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.304 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 2", 2024-09-16T20:49:04.304 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481822+0000", 2024-09-16T20:49:04.304 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019446200000000001 2024-09-16T20:49:04.304 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.304 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.304 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:04.304 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605988+0000", 2024-09-16T20:49:04.305 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.124165888 2024-09-16T20:49:04.305 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.305 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.305 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:04.305 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606427+0000", 2024-09-16T20:49:04.305 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00043854899999999999 2024-09-16T20:49:04.305 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.305 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.305 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:04.305 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606476+0000", 2024-09-16T20:49:04.305 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8971999999999998e-05 2024-09-16T20:49:04.305 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.305 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.305 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:04.306 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606517+0000", 2024-09-16T20:49:04.306 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1467000000000001e-05 2024-09-16T20:49:04.306 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.306 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:04.306 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.306 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.306 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.306 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4882.0:2508 3.12 e57/55 ondisk, result = 0)", 2024-09-16T20:49:04.306 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:49:04.306 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 17.713383506, 2024-09-16T20:49:04.306 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12349711400000001, 2024-09-16T20:49:04.306 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:04.306 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:04.307 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:04.307 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:04.307 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.307 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:04.307 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:49:04.307 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.307 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.307 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.307 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:04.307 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:49:04.307 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.307 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.308 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.308 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:04.308 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:49:04.308 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.06e-07 2024-09-16T20:49:04.308 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.308 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.308 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:04.308 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482502+0000", 2024-09-16T20:49:04.308 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.01e-06 2024-09-16T20:49:04.308 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.308 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.309 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:04.309 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605772+0000", 2024-09-16T20:49:04.309 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.123269687 2024-09-16T20:49:04.309 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.309 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.309 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:04.309 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605791+0000", 2024-09-16T20:49:04.309 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8912000000000001e-05 2024-09-16T20:49:04.309 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.309 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.309 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:04.309 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605974+0000", 2024-09-16T20:49:04.309 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000182836 2024-09-16T20:49:04.310 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.310 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.310 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:04.310 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605982+0000", 2024-09-16T20:49:04.310 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1699999999999997e-06 2024-09-16T20:49:04.310 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.310 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.310 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:04.310 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605996+0000", 2024-09-16T20:49:04.310 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3893000000000001e-05 2024-09-16T20:49:04.310 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.310 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:04.310 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.310 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.311 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.311 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4882.0:2512 3.f 3:f4d895b7:::rbd_data.110e7c8851c.0000000000000000:head [read 16384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T20:49:04.311 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:49:04.311 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 17.712946475999999, 2024-09-16T20:49:04.311 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12318483, 2024-09-16T20:49:04.311 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:04.311 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:04.311 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:04.311 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:04.312 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4882", 2024-09-16T20:49:04.312 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2436923223", 2024-09-16T20:49:04.312 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2512 2024-09-16T20:49:04.312 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.312 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:04.312 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.312 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:04.312 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:49:04.312 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.313 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.313 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.313 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:04.313 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:49:04.313 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.313 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.313 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.313 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:04.313 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:49:04.313 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3499999999999995e-07 2024-09-16T20:49:04.313 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.313 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.313 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:04.314 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482943+0000", 2024-09-16T20:49:04.314 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3450000000000002e-06 2024-09-16T20:49:04.314 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.314 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.314 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:04.314 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605772+0000", 2024-09-16T20:49:04.314 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.122829796 2024-09-16T20:49:04.314 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.314 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.314 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:04.314 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605792+0000", 2024-09-16T20:49:04.314 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9916999999999999e-05 2024-09-16T20:49:04.314 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.315 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.315 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:04.315 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605917+0000", 2024-09-16T20:49:04.315 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00012500799999999999 2024-09-16T20:49:04.315 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.315 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.315 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:04.315 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606004+0000", 2024-09-16T20:49:04.315 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6242999999999994e-05 2024-09-16T20:49:04.315 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.316 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.316 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:04.316 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606120+0000", 2024-09-16T20:49:04.316 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000116786 2024-09-16T20:49:04.316 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.316 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:04.316 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.317 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.317 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:04.317 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:04.320 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:49:04.320 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:04.318+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:04.321 INFO:teuthology.orchestra.run.smithi139.stdout:2622 write 0x8716e00 thru 0x871877e (0x197f bytes) 2024-09-16T20:49:04.321 INFO:teuthology.orchestra.run.smithi139.stdout:2623 trunc from 0x8718800 to 0xa5f0000 2024-09-16T20:49:04.321 INFO:teuthology.orchestra.run.smithi139.stdout:2626 write 0xc30000 thru 0xc31586 (0x1587 bytes) 2024-09-16T20:49:04.321 INFO:teuthology.orchestra.run.smithi139.stdout:2629 read 0x243200 thru 0x245c34 (0x2a35 bytes) 2024-09-16T20:49:04.321 INFO:teuthology.orchestra.run.smithi139.stdout:2631 write 0x2ab0c00 thru 0x2ab7908 (0x6d09 bytes) 2024-09-16T20:49:04.321 INFO:teuthology.orchestra.run.smithi139.stdout:2634 write 0xc236c00 thru 0xc239b59 (0x2f5a bytes) 2024-09-16T20:49:04.321 INFO:teuthology.orchestra.run.smithi139.stdout:2635 read 0xbf64800 thru 0xbf67bca (0x33cb bytes) 2024-09-16T20:49:04.322 INFO:teuthology.orchestra.run.smithi139.stdout:2638 trunc from 0xc239c00 to 0x525fc00 2024-09-16T20:49:04.322 INFO:teuthology.orchestra.run.smithi139.stdout:2639 write 0x283ba00 thru 0x2849085 (0xd686 bytes) 2024-09-16T20:49:04.322 INFO:teuthology.orchestra.run.smithi139.stdout:2640 read 0x4e92c00 thru 0x4e9f6b5 (0xcab6 bytes) 2024-09-16T20:49:04.345 INFO:teuthology.orchestra.run.smithi139.stdout:2642 write 0xb103600 thru 0xb10cc1d (0x961e bytes) 2024-09-16T20:49:04.360 INFO:teuthology.orchestra.run.smithi139.stdout:2643 trunc from 0xb10ce00 to 0x43f2800 2024-09-16T20:49:04.384 INFO:teuthology.orchestra.run.smithi139.stdout:2644 read 0x1cd5000 thru 0x1ce048e (0xb48f bytes) 2024-09-16T20:49:04.390 INFO:teuthology.orchestra.run.smithi139.stdout:2645 punch from 0x1ae7600 to 0x1ae8000, (0xa00 bytes) 2024-09-16T20:49:04.390 INFO:teuthology.orchestra.run.smithi139.stdout:2646 read 0x1c5e00 thru 0x1d2fad (0xd1ae bytes) 2024-09-16T20:49:04.392 INFO:teuthology.orchestra.run.smithi139.stdout:2649 read 0x3f09600 thru 0x3f0efba (0x59bb bytes) 2024-09-16T20:49:04.392 INFO:teuthology.orchestra.run.smithi139.stdout:2650 read 0x12b1a00 thru 0x12bcadc (0xb0dd bytes) 2024-09-16T20:49:04.392 INFO:teuthology.orchestra.run.smithi139.stdout:2652 read 0x4392600 thru 0x439b3f8 (0x8df9 bytes) 2024-09-16T20:49:04.393 INFO:teuthology.orchestra.run.smithi139.stdout:2653 read 0x2236200 thru 0x2236f40 (0xd41 bytes) 2024-09-16T20:49:04.393 INFO:teuthology.orchestra.run.smithi139.stdout:2655 read 0x495800 thru 0x49c7c1 (0x6fc2 bytes) 2024-09-16T20:49:04.397 INFO:teuthology.orchestra.run.smithi139.stdout:2656 trunc from 0x43f2800 to 0x7aada00 2024-09-16T20:49:04.406 INFO:teuthology.orchestra.run.smithi139.stdout:2659 read 0x762aa00 thru 0x762f3fe (0x49ff bytes) 2024-09-16T20:49:04.413 INFO:teuthology.orchestra.run.smithi139.stdout:2660 write 0x29a9800 thru 0x29b0560 (0x6d61 bytes) 2024-09-16T20:49:04.419 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:04.418+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:04.421 INFO:teuthology.orchestra.run.smithi139.stdout:2663 trunc from 0x7aada00 to 0xdec9e00 2024-09-16T20:49:04.434 INFO:teuthology.orchestra.run.smithi139.stdout:2664 read 0x126d400 thru 0x1270e01 (0x3a02 bytes) 2024-09-16T20:49:04.435 INFO:teuthology.orchestra.run.smithi139.stdout:2666 read 0x8a07a00 thru 0x8a0fe0a (0x840b bytes) 2024-09-16T20:49:04.436 INFO:teuthology.orchestra.run.smithi139.stdout:2671 write 0x2f94200 thru 0x2f95abf (0x18c0 bytes) 2024-09-16T20:49:04.445 INFO:teuthology.orchestra.run.smithi139.stdout:2673 trunc from 0xdec9e00 to 0x5893c00 2024-09-16T20:49:04.464 INFO:teuthology.orchestra.run.smithi139.stdout:2674 punch from 0x69ce00 to 0x6a5800, (0x8a00 bytes) 2024-09-16T20:49:04.471 INFO:teuthology.orchestra.run.smithi139.stdout:2675 read 0x153e000 thru 0x15424e9 (0x44ea bytes) 2024-09-16T20:49:04.490 INFO:teuthology.orchestra.run.smithi139.stdout:2677 write 0x69d8800 thru 0x69e1b4d (0x934e bytes) 2024-09-16T20:49:04.493 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:04.493 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:49:04.493 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:49:04.493 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:04.496 INFO:teuthology.orchestra.run.smithi139.stdout:2678 read 0x2d42200 thru 0x2d4910f (0x6f10 bytes) 2024-09-16T20:49:04.496 INFO:teuthology.orchestra.run.smithi139.stdout:2679 punch from 0x2b82600 to 0x2b8b600, (0x9000 bytes) 2024-09-16T20:49:04.496 INFO:teuthology.orchestra.run.smithi139.stdout:2683 punch from 0x162c400 to 0x163bc00, (0xf800 bytes) 2024-09-16T20:49:04.496 INFO:teuthology.orchestra.run.smithi139.stdout:2685 read 0x122fc00 thru 0x123e29f (0xe6a0 bytes) 2024-09-16T20:49:04.504 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:49:04.523 INFO:teuthology.orchestra.run.smithi139.stdout:2686 write 0x9c4a000 thru 0x9c5556f (0xb570 bytes) 2024-09-16T20:49:04.524 INFO:teuthology.orchestra.run.smithi139.stdout:2687 write 0x649e400 thru 0x64a8db8 (0xa9b9 bytes) 2024-09-16T20:49:04.524 INFO:teuthology.orchestra.run.smithi139.stdout:2688 punch from 0x126a400 to 0x126f600, (0x5200 bytes) 2024-09-16T20:49:04.524 INFO:teuthology.orchestra.run.smithi139.stdout:2689 clone 20 order 19 su 2048 sc 13 2024-09-16T20:49:04.548 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:04.548+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:04.621 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:04.621 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:49:04.621 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:49:04.621 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:49:04.621 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:04.621 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:04.620+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:04.629 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:49:04.722 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:04.722+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:04.768 INFO:teuthology.orchestra.run.smithi029.stderr:reweighted osd.2 to 1 (10000) 2024-09-16T20:49:04.769 INFO:teuthology.orchestra.run.smithi029.stderr:nodeep-scrub is unset 2024-09-16T20:49:04.783 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:04.783 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:49:04.783 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:49:04.783 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:49:04.783 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.783 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:49:04.783 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:49:04.783 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 71.527315541999997, 2024-09-16T20:49:04.783 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67502346999999996, 2024-09-16T20:49:04.784 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:04.784 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:04.784 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:04.784 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:04.784 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:49:04.784 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:49:04.784 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 1 2024-09-16T20:49:04.784 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.784 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:04.784 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.784 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:04.784 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:49:04.785 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.785 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.785 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.785 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:04.785 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:49:04.785 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.785 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.785 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.785 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:04.785 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254128+0000", 2024-09-16T20:49:04.785 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.463e-06 2024-09-16T20:49:04.785 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.785 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.786 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:04.786 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254135+0000", 2024-09-16T20:49:04.786 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1579999999999999e-06 2024-09-16T20:49:04.786 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.786 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.786 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:04.786 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254146+0000", 2024-09-16T20:49:04.786 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1341e-05 2024-09-16T20:49:04.786 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.786 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.786 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:04.786 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254155+0000", 2024-09-16T20:49:04.786 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.1549999999999996e-06 2024-09-16T20:49:04.786 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.787 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.787 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:04.787 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254221+0000", 2024-09-16T20:49:04.787 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5470999999999997e-05 2024-09-16T20:49:04.787 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.787 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.787 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:49:04.787 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254231+0000", 2024-09-16T20:49:04.787 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0592e-05 2024-09-16T20:49:04.787 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.787 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.787 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:04.787 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926711+0000", 2024-09-16T20:49:04.787 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67247946599999997 2024-09-16T20:49:04.788 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.788 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.788 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:04.788 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926910+0000", 2024-09-16T20:49:04.788 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019934 2024-09-16T20:49:04.788 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.788 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.788 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:49:04.788 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.927159+0000", 2024-09-16T20:49:04.788 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00024876599999999998 2024-09-16T20:49:04.788 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.788 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.788 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:04.789 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.928797+0000", 2024-09-16T20:49:04.789 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001638308 2024-09-16T20:49:04.789 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.789 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.789 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:04.789 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929026+0000", 2024-09-16T20:49:04.789 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00022876099999999999 2024-09-16T20:49:04.789 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.789 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.789 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:04.789 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929102+0000", 2024-09-16T20:49:04.789 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6184000000000001e-05 2024-09-16T20:49:04.789 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.789 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.790 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:04.790 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929150+0000", 2024-09-16T20:49:04.790 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7465e-05 2024-09-16T20:49:04.790 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.790 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:04.790 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.790 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.790 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.790 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4317 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4659200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:04.790 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745704+0000", 2024-09-16T20:49:04.790 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 45.035737623000003, 2024-09-16T20:49:04.790 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047411778000000002, 2024-09-16T20:49:04.790 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:04.790 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:04.791 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:04.791 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:04.791 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:49:04.791 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:04.791 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4317 2024-09-16T20:49:04.791 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.791 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:04.791 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.791 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:04.791 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745704+0000", 2024-09-16T20:49:04.791 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.791 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.791 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.792 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:04.792 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745704+0000", 2024-09-16T20:49:04.792 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.792 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.792 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.792 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:04.792 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745704+0000", 2024-09-16T20:49:04.792 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0900000000000002e-07 2024-09-16T20:49:04.792 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.792 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.792 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:04.792 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745707+0000", 2024-09-16T20:49:04.792 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1660000000000001e-06 2024-09-16T20:49:04.793 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.793 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.793 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:04.793 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745722+0000", 2024-09-16T20:49:04.793 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5084e-05 2024-09-16T20:49:04.793 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.793 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.793 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:04.793 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745735+0000", 2024-09-16T20:49:04.793 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3074999999999999e-05 2024-09-16T20:49:04.793 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.793 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.793 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:04.793 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.792989+0000", 2024-09-16T20:49:04.794 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047254013999999997 2024-09-16T20:49:04.794 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.794 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.794 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:04.794 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793034+0000", 2024-09-16T20:49:04.794 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4858999999999999e-05 2024-09-16T20:49:04.794 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.794 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.794 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:04.794 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793116+0000", 2024-09-16T20:49:04.794 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2170999999999996e-05 2024-09-16T20:49:04.794 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.794 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:04.794 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.795 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.795 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.795 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4318 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4667392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:04.795 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745812+0000", 2024-09-16T20:49:04.795 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 45.035629944, 2024-09-16T20:49:04.795 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047532229000000002, 2024-09-16T20:49:04.795 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:04.795 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:04.795 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:04.795 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:04.795 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:49:04.795 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:04.795 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4318 2024-09-16T20:49:04.796 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.796 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:04.796 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.796 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:04.796 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745812+0000", 2024-09-16T20:49:04.796 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.796 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.796 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.796 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:04.796 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745812+0000", 2024-09-16T20:49:04.796 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.796 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.796 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.797 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:04.797 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745812+0000", 2024-09-16T20:49:04.797 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5899999999999996e-07 2024-09-16T20:49:04.797 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.797 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.797 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:04.797 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745815+0000", 2024-09-16T20:49:04.797 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9110000000000002e-06 2024-09-16T20:49:04.797 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.797 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.797 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:04.797 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745823+0000", 2024-09-16T20:49:04.798 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1030000000000002e-06 2024-09-16T20:49:04.798 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.798 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.798 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:04.798 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745829+0000", 2024-09-16T20:49:04.798 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8449999999999999e-06 2024-09-16T20:49:04.798 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.798 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.798 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:04.798 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793159+0000", 2024-09-16T20:49:04.798 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047330009999999999 2024-09-16T20:49:04.798 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.798 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.798 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:04.799 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793251+0000", 2024-09-16T20:49:04.799 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.1991999999999996e-05 2024-09-16T20:49:04.799 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.799 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.799 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:04.799 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793344+0000", 2024-09-16T20:49:04.799 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2808999999999999e-05 2024-09-16T20:49:04.799 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.799 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:04.799 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.799 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.799 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.799 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4319 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4675584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:04.800 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745838+0000", 2024-09-16T20:49:04.800 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 45.035603731000002, 2024-09-16T20:49:04.800 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047640204999999998, 2024-09-16T20:49:04.800 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:04.800 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:04.800 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:04.800 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:04.800 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:49:04.800 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:04.800 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4319 2024-09-16T20:49:04.800 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.800 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:04.800 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.801 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:04.801 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745838+0000", 2024-09-16T20:49:04.801 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.801 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.801 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.801 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:04.801 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745838+0000", 2024-09-16T20:49:04.801 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.801 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.801 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.801 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:04.801 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745838+0000", 2024-09-16T20:49:04.801 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6899999999999998e-07 2024-09-16T20:49:04.801 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.802 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.802 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:04.802 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745840+0000", 2024-09-16T20:49:04.802 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0660000000000002e-06 2024-09-16T20:49:04.802 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.802 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.802 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:04.802 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745847+0000", 2024-09-16T20:49:04.802 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8689999999999998e-06 2024-09-16T20:49:04.802 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.802 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.802 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:04.802 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745853+0000", 2024-09-16T20:49:04.802 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2399999999999998e-06 2024-09-16T20:49:04.803 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.803 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.803 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:04.803 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793381+0000", 2024-09-16T20:49:04.803 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047528005999999998 2024-09-16T20:49:04.803 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.803 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.803 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:04.803 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793423+0000", 2024-09-16T20:49:04.803 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2799000000000001e-05 2024-09-16T20:49:04.803 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.803 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.803 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:04.803 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793478+0000", 2024-09-16T20:49:04.804 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4756000000000002e-05 2024-09-16T20:49:04.804 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.804 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:04.804 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.804 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.804 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.804 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4320 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4683776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:04.804 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745859+0000", 2024-09-16T20:49:04.804 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 45.035582486999999, 2024-09-16T20:49:04.804 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047811399999999997, 2024-09-16T20:49:04.804 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:04.804 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:04.804 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:04.805 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:04.805 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:49:04.805 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:04.805 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4320 2024-09-16T20:49:04.805 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.805 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:04.805 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.805 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:04.805 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745859+0000", 2024-09-16T20:49:04.805 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.805 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.805 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.805 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:04.805 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745859+0000", 2024-09-16T20:49:04.806 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.806 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.806 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.806 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:04.806 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745860+0000", 2024-09-16T20:49:04.806 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9099999999999999e-07 2024-09-16T20:49:04.806 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.806 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.806 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:04.806 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745861+0000", 2024-09-16T20:49:04.806 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8589999999999999e-06 2024-09-16T20:49:04.806 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.806 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.806 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:04.807 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745868+0000", 2024-09-16T20:49:04.807 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2249999999999997e-06 2024-09-16T20:49:04.807 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.807 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.807 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:04.807 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745873+0000", 2024-09-16T20:49:04.807 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1290000000000004e-06 2024-09-16T20:49:04.807 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.807 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.807 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:04.807 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793522+0000", 2024-09-16T20:49:04.807 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047648847000000001 2024-09-16T20:49:04.807 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.808 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.808 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:04.808 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793603+0000", 2024-09-16T20:49:04.808 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1167000000000004e-05 2024-09-16T20:49:04.808 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.808 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.808 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:04.808 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793671+0000", 2024-09-16T20:49:04.808 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7781999999999996e-05 2024-09-16T20:49:04.808 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.808 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:04.808 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.808 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.808 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.809 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4321 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4691968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:04.809 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745879+0000", 2024-09-16T20:49:04.809 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 45.035562290999998, 2024-09-16T20:49:04.809 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047950375000000003, 2024-09-16T20:49:04.809 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:04.809 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:04.809 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:04.809 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:04.809 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:49:04.809 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:04.809 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4321 2024-09-16T20:49:04.809 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.809 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:04.809 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.810 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:04.810 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745879+0000", 2024-09-16T20:49:04.810 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.810 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.810 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.810 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:04.810 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745879+0000", 2024-09-16T20:49:04.810 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.810 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.810 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.810 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:04.810 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745880+0000", 2024-09-16T20:49:04.810 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.65e-07 2024-09-16T20:49:04.810 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.811 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.811 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:04.811 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745882+0000", 2024-09-16T20:49:04.811 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.922e-06 2024-09-16T20:49:04.811 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.811 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.811 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:04.811 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745888+0000", 2024-09-16T20:49:04.812 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0510000000000002e-06 2024-09-16T20:49:04.812 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.812 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.812 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:04.812 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745893+0000", 2024-09-16T20:49:04.812 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9579999999999998e-06 2024-09-16T20:49:04.812 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.812 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.812 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:04.813 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793704+0000", 2024-09-16T20:49:04.813 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047811179000000002 2024-09-16T20:49:04.813 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.813 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.813 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:04.813 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793759+0000", 2024-09-16T20:49:04.813 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5260999999999998e-05 2024-09-16T20:49:04.813 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.813 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.813 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:04.813 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793830+0000", 2024-09-16T20:49:04.813 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0638999999999999e-05 2024-09-16T20:49:04.813 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.813 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:04.814 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.814 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.814 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.814 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4322 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4700160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:04.814 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745899+0000", 2024-09-16T20:49:04.814 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 45.035542853000003, 2024-09-16T20:49:04.814 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048057280000000001, 2024-09-16T20:49:04.814 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:04.814 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:04.814 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:04.814 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:04.814 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:49:04.814 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:04.814 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4322 2024-09-16T20:49:04.815 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.815 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:04.815 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.815 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:04.815 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745899+0000", 2024-09-16T20:49:04.815 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.815 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.815 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.815 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:04.815 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745899+0000", 2024-09-16T20:49:04.815 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.815 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.815 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.816 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:04.816 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745899+0000", 2024-09-16T20:49:04.816 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9299999999999999e-07 2024-09-16T20:49:04.816 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.816 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.816 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:04.816 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745901+0000", 2024-09-16T20:49:04.816 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0030000000000001e-06 2024-09-16T20:49:04.816 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.816 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.816 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:04.816 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745908+0000", 2024-09-16T20:49:04.816 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3729999999999998e-06 2024-09-16T20:49:04.817 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.817 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.817 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:04.817 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745914+0000", 2024-09-16T20:49:04.817 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.122e-06 2024-09-16T20:49:04.817 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.817 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.817 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:04.817 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793862+0000", 2024-09-16T20:49:04.817 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047948324 2024-09-16T20:49:04.817 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.817 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.817 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:04.817 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793903+0000", 2024-09-16T20:49:04.818 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0465e-05 2024-09-16T20:49:04.818 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.818 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.818 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:04.818 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793956+0000", 2024-09-16T20:49:04.818 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3600000000000002e-05 2024-09-16T20:49:04.818 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.818 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:04.818 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.818 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.818 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.818 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4323 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4708352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:04.818 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745920+0000", 2024-09-16T20:49:04.818 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 45.035521211999999, 2024-09-16T20:49:04.819 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048268138000000002, 2024-09-16T20:49:04.819 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:04.819 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:04.819 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:04.819 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:04.819 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:49:04.819 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:04.819 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4323 2024-09-16T20:49:04.819 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.819 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:04.819 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.819 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:04.819 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745920+0000", 2024-09-16T20:49:04.819 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.820 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.820 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.820 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:04.820 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745920+0000", 2024-09-16T20:49:04.820 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.820 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.820 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.820 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:04.820 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745921+0000", 2024-09-16T20:49:04.820 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7e-07 2024-09-16T20:49:04.820 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.820 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.820 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:04.821 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745923+0000", 2024-09-16T20:49:04.821 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7880000000000001e-06 2024-09-16T20:49:04.821 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.821 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.821 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:04.821 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745929+0000", 2024-09-16T20:49:04.821 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0809999999999999e-06 2024-09-16T20:49:04.821 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.821 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.821 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:04.821 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745935+0000", 2024-09-16T20:49:04.821 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5370000000000003e-06 2024-09-16T20:49:04.821 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.821 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.822 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:04.822 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793987+0000", 2024-09-16T20:49:04.822 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048051736999999997 2024-09-16T20:49:04.822 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.822 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.822 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:04.822 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.794070+0000", 2024-09-16T20:49:04.822 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.3277999999999998e-05 2024-09-16T20:49:04.822 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.822 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.822 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:04.822 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.794189+0000", 2024-09-16T20:49:04.822 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000118347 2024-09-16T20:49:04.822 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.823 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:04.823 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.823 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.823 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.823 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4324 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4716544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:04.823 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745946+0000", 2024-09-16T20:49:04.823 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 45.035495640999997, 2024-09-16T20:49:04.823 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048468153999999999, 2024-09-16T20:49:04.823 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:04.823 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:04.823 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:04.823 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:04.823 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:49:04.823 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:04.824 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4324 2024-09-16T20:49:04.824 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.824 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:04.824 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.824 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:04.824 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745946+0000", 2024-09-16T20:49:04.824 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.824 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.824 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.824 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:04.824 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745946+0000", 2024-09-16T20:49:04.824 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.824 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.824 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.825 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:04.825 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745946+0000", 2024-09-16T20:49:04.825 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4499999999999997e-07 2024-09-16T20:49:04.825 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.825 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.825 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:04.825 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745949+0000", 2024-09-16T20:49:04.825 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2529999999999999e-06 2024-09-16T20:49:04.825 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.825 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.825 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:04.825 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745955+0000", 2024-09-16T20:49:04.825 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6470000000000001e-06 2024-09-16T20:49:04.825 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.826 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.826 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:04.826 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745960+0000", 2024-09-16T20:49:04.826 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7910000000000004e-06 2024-09-16T20:49:04.826 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.826 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.826 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:04.826 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.794220+0000", 2024-09-16T20:49:04.826 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048260069000000003 2024-09-16T20:49:04.826 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.826 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.826 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:04.826 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.794311+0000", 2024-09-16T20:49:04.826 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0593000000000005e-05 2024-09-16T20:49:04.827 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.827 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.827 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:04.827 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.794414+0000", 2024-09-16T20:49:04.827 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000103356 2024-09-16T20:49:04.827 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.827 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:04.827 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.827 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.827 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.827 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3571 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:04.827 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481059+0000", 2024-09-16T20:49:04.827 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.300382481, 2024-09-16T20:49:04.827 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049572995000000002, 2024-09-16T20:49:04.828 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:04.828 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:04.828 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:04.828 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:04.828 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:04.828 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:04.828 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3571 2024-09-16T20:49:04.828 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.828 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:04.828 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.828 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:04.828 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481059+0000", 2024-09-16T20:49:04.828 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.828 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.829 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.829 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:04.829 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481059+0000", 2024-09-16T20:49:04.829 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.829 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.829 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.829 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:04.829 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481060+0000", 2024-09-16T20:49:04.829 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2599999999999998e-07 2024-09-16T20:49:04.829 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.829 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.829 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:04.829 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481063+0000", 2024-09-16T20:49:04.830 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.999e-06 2024-09-16T20:49:04.830 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.830 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.830 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:04.830 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481067+0000", 2024-09-16T20:49:04.830 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3080000000000001e-06 2024-09-16T20:49:04.830 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.830 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.830 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:04.830 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481070+0000", 2024-09-16T20:49:04.830 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9919999999999999e-06 2024-09-16T20:49:04.830 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.830 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.830 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:04.830 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.522087+0000", 2024-09-16T20:49:04.831 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.041017023 2024-09-16T20:49:04.831 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.831 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.831 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:04.831 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.522191+0000", 2024-09-16T20:49:04.831 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010389499999999999 2024-09-16T20:49:04.831 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.831 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.831 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:04.832 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.522566+0000", 2024-09-16T20:49:04.832 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00037539399999999998 2024-09-16T20:49:04.832 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.832 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.832 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:04.832 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525494+0000", 2024-09-16T20:49:04.832 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0029273070000000001 2024-09-16T20:49:04.832 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.832 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.832 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:04.832 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530355+0000", 2024-09-16T20:49:04.832 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0048612509999999996 2024-09-16T20:49:04.832 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.833 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.833 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:04.833 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530407+0000", 2024-09-16T20:49:04.833 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1817000000000002e-05 2024-09-16T20:49:04.833 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.833 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.833 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:04.833 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530632+0000", 2024-09-16T20:49:04.833 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00022558300000000001 2024-09-16T20:49:04.833 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.833 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:04.833 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.833 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.834 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.834 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3572 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:04.834 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481220+0000", 2024-09-16T20:49:04.834 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.300221913, 2024-09-16T20:49:04.834 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050452862000000001, 2024-09-16T20:49:04.834 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:04.834 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:04.834 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:04.834 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:04.834 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:04.834 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:04.834 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3572 2024-09-16T20:49:04.834 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.835 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:04.835 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.835 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:04.835 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481220+0000", 2024-09-16T20:49:04.835 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.835 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.835 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.835 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:04.835 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481220+0000", 2024-09-16T20:49:04.835 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.835 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.835 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.835 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:04.835 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481221+0000", 2024-09-16T20:49:04.836 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9400000000000004e-07 2024-09-16T20:49:04.836 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.836 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.836 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:04.836 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481225+0000", 2024-09-16T20:49:04.836 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0230000000000003e-06 2024-09-16T20:49:04.836 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.836 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.836 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:04.836 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481234+0000", 2024-09-16T20:49:04.836 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2920000000000001e-06 2024-09-16T20:49:04.836 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.836 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.837 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:04.837 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481240+0000", 2024-09-16T20:49:04.837 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9390000000000002e-06 2024-09-16T20:49:04.837 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.837 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.837 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:04.837 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.523544+0000", 2024-09-16T20:49:04.837 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.042304475000000001 2024-09-16T20:49:04.837 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.837 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.837 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:04.837 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.523624+0000", 2024-09-16T20:49:04.837 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.975e-05 2024-09-16T20:49:04.837 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.838 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.838 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:04.838 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.523975+0000", 2024-09-16T20:49:04.838 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00035063200000000003 2024-09-16T20:49:04.838 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.838 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.838 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:04.838 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530337+0000", 2024-09-16T20:49:04.838 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0063623589999999997 2024-09-16T20:49:04.838 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.838 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.838 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:04.838 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531592+0000", 2024-09-16T20:49:04.838 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0012550599999999999 2024-09-16T20:49:04.839 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.839 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.839 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:04.839 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531633+0000", 2024-09-16T20:49:04.839 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1192000000000001e-05 2024-09-16T20:49:04.839 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.839 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.839 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:04.839 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531673+0000", 2024-09-16T20:49:04.839 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9345999999999999e-05 2024-09-16T20:49:04.839 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.839 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:04.839 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.839 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.839 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.840 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3573 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:04.840 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481257+0000", 2024-09-16T20:49:04.840 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.300184966, 2024-09-16T20:49:04.840 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050511027, 2024-09-16T20:49:04.840 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:04.840 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:04.841 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-16T20:49:04.841 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:04.842 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:04.842 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:04.842 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:04.842 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3573 2024-09-16T20:49:04.842 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.842 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:04.842 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.842 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:04.842 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481257+0000", 2024-09-16T20:49:04.842 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.842 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.842 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.842 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:04.843 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481257+0000", 2024-09-16T20:49:04.843 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.843 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.843 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.843 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:04.843 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481257+0000", 2024-09-16T20:49:04.843 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.99e-07 2024-09-16T20:49:04.843 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.843 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.843 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:04.843 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481260+0000", 2024-09-16T20:49:04.843 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1709999999999998e-06 2024-09-16T20:49:04.843 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.843 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.844 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:04.844 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481271+0000", 2024-09-16T20:49:04.844 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0795000000000001e-05 2024-09-16T20:49:04.844 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.844 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.844 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:04.844 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481277+0000", 2024-09-16T20:49:04.844 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1670000000000004e-06 2024-09-16T20:49:04.844 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.844 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.844 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:04.844 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.524647+0000", 2024-09-16T20:49:04.844 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.043369810000000002 2024-09-16T20:49:04.844 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.845 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.845 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:04.845 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.524725+0000", 2024-09-16T20:49:04.845 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7286000000000005e-05 2024-09-16T20:49:04.845 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.845 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.845 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:04.845 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525062+0000", 2024-09-16T20:49:04.845 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00033739499999999999 2024-09-16T20:49:04.845 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.845 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.845 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:04.845 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530692+0000", 2024-09-16T20:49:04.846 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0056301479999999997 2024-09-16T20:49:04.846 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.846 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.846 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:04.846 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531694+0000", 2024-09-16T20:49:04.846 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0010017419999999999 2024-09-16T20:49:04.846 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.846 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.846 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:04.846 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531737+0000", 2024-09-16T20:49:04.846 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2762999999999997e-05 2024-09-16T20:49:04.846 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.846 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.846 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:04.847 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531768+0000", 2024-09-16T20:49:04.847 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1151e-05 2024-09-16T20:49:04.847 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.847 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:04.847 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.847 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.847 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.847 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3574 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:04.847 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:49:04.847 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.30015201, 2024-09-16T20:49:04.847 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053579441999999998, 2024-09-16T20:49:04.847 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:04.847 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:04.848 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:04.848 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:04.848 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:04.848 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:04.848 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3574 2024-09-16T20:49:04.848 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.848 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:04.848 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.848 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:04.848 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:49:04.848 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.848 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.848 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.848 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:04.849 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:49:04.849 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.849 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.849 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.849 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:04.849 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:49:04.849 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2e-07 2024-09-16T20:49:04.849 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.849 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.849 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:04.849 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481292+0000", 2024-09-16T20:49:04.849 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2630000000000002e-06 2024-09-16T20:49:04.849 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.849 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.850 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:04.850 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481297+0000", 2024-09-16T20:49:04.850 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.296e-06 2024-09-16T20:49:04.850 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.850 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.850 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:04.850 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481300+0000", 2024-09-16T20:49:04.850 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7019999999999999e-06 2024-09-16T20:49:04.850 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.850 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.850 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:04.850 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525533+0000", 2024-09-16T20:49:04.850 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.044232318 2024-09-16T20:49:04.850 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.851 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.851 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:04.851 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525619+0000", 2024-09-16T20:49:04.851 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5892999999999999e-05 2024-09-16T20:49:04.851 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.851 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.851 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:04.851 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525987+0000", 2024-09-16T20:49:04.851 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00036836099999999999 2024-09-16T20:49:04.851 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.851 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.851 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:04.851 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531779+0000", 2024-09-16T20:49:04.851 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0057918240000000001 2024-09-16T20:49:04.851 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.852 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.852 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:04.852 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534747+0000", 2024-09-16T20:49:04.852 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0029681600000000001 2024-09-16T20:49:04.852 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.852 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.852 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:04.852 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534832+0000", 2024-09-16T20:49:04.852 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5180999999999996e-05 2024-09-16T20:49:04.852 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.852 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.852 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:04.852 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534869+0000", 2024-09-16T20:49:04.852 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7024000000000003e-05 2024-09-16T20:49:04.853 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.853 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:04.853 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.853 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.853 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.853 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3575 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:04.853 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:49:04.853 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.30013727, 2024-09-16T20:49:04.853 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053801463000000001, 2024-09-16T20:49:04.853 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:04.853 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:04.853 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:04.853 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:04.853 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:04.854 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:04.854 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3575 2024-09-16T20:49:04.854 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.854 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:04.854 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.854 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:04.854 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:49:04.854 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.854 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.854 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.854 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:04.854 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:49:04.854 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.854 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.855 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.855 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:04.855 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481305+0000", 2024-09-16T20:49:04.855 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5400000000000002e-07 2024-09-16T20:49:04.855 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.855 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.855 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:04.855 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481307+0000", 2024-09-16T20:49:04.855 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7800000000000001e-06 2024-09-16T20:49:04.855 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.855 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.855 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:04.855 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481312+0000", 2024-09-16T20:49:04.855 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6349999999999997e-06 2024-09-16T20:49:04.856 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.856 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.856 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:04.856 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481315+0000", 2024-09-16T20:49:04.856 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7829999999999999e-06 2024-09-16T20:49:04.856 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.856 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.856 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:04.856 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.526876+0000", 2024-09-16T20:49:04.856 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.045561070000000002 2024-09-16T20:49:04.856 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.856 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.856 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:04.857 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.526954+0000", 2024-09-16T20:49:04.857 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8108000000000005e-05 2024-09-16T20:49:04.857 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.857 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.857 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:04.857 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527282+0000", 2024-09-16T20:49:04.857 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000327483 2024-09-16T20:49:04.857 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.857 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.857 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:04.857 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531784+0000", 2024-09-16T20:49:04.857 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0045020169999999997 2024-09-16T20:49:04.857 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.857 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.858 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:04.858 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534921+0000", 2024-09-16T20:49:04.858 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.003137791 2024-09-16T20:49:04.858 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.858 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.858 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:04.858 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534976+0000", 2024-09-16T20:49:04.858 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4953000000000001e-05 2024-09-16T20:49:04.858 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.858 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.858 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:04.858 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535106+0000", 2024-09-16T20:49:04.858 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000129589 2024-09-16T20:49:04.858 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.858 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:04.859 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.859 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.859 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.859 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3576 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:04.859 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:49:04.859 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.300123358, 2024-09-16T20:49:04.859 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053955436000000002, 2024-09-16T20:49:04.859 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:04.859 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:04.859 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:04.859 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:04.859 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:04.859 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:04.860 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3576 2024-09-16T20:49:04.860 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.860 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:04.860 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.860 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:04.860 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:49:04.860 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.860 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.860 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.860 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:04.860 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:49:04.860 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.860 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.860 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.861 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:04.861 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481319+0000", 2024-09-16T20:49:04.861 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.53e-07 2024-09-16T20:49:04.861 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.861 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.861 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:04.861 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481320+0000", 2024-09-16T20:49:04.861 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5230000000000001e-06 2024-09-16T20:49:04.861 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.861 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.861 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:04.861 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481330+0000", 2024-09-16T20:49:04.861 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0190000000000001e-05 2024-09-16T20:49:04.861 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.862 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.862 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:04.862 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481333+0000", 2024-09-16T20:49:04.862 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.151e-06 2024-09-16T20:49:04.862 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.862 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.862 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:04.862 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527689+0000", 2024-09-16T20:49:04.862 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046355192000000003 2024-09-16T20:49:04.862 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.862 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.862 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:04.862 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527764+0000", 2024-09-16T20:49:04.862 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5776000000000001e-05 2024-09-16T20:49:04.863 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.863 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.863 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:04.863 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528141+0000", 2024-09-16T20:49:04.863 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00037647599999999999 2024-09-16T20:49:04.863 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.863 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.863 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:04.863 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531788+0000", 2024-09-16T20:49:04.863 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0036470679999999998 2024-09-16T20:49:04.863 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.863 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.863 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:04.863 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535174+0000", 2024-09-16T20:49:04.863 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0033862670000000001 2024-09-16T20:49:04.864 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.864 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.864 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:04.864 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535236+0000", 2024-09-16T20:49:04.864 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1797999999999993e-05 2024-09-16T20:49:04.864 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.864 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.864 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:04.864 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535274+0000", 2024-09-16T20:49:04.864 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7741999999999999e-05 2024-09-16T20:49:04.864 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.864 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:04.864 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.865 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.865 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.865 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3577 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:04.865 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:49:04.865 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.300101758, 2024-09-16T20:49:04.865 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054073807000000002, 2024-09-16T20:49:04.865 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:04.865 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:04.865 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:04.865 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:04.865 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:04.865 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:04.865 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3577 2024-09-16T20:49:04.865 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.865 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:04.866 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.866 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:04.866 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:49:04.866 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.866 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.866 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.866 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:04.866 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:49:04.866 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.866 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.866 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.866 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:04.866 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:49:04.866 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.96e-07 2024-09-16T20:49:04.867 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.867 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.867 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:04.867 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481342+0000", 2024-09-16T20:49:04.867 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.714e-06 2024-09-16T20:49:04.867 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.867 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.867 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:04.867 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481349+0000", 2024-09-16T20:49:04.867 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4930000000000001e-06 2024-09-16T20:49:04.867 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.867 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.867 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:04.867 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481353+0000", 2024-09-16T20:49:04.868 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4879999999999999e-06 2024-09-16T20:49:04.868 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.868 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.868 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:04.868 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528619+0000", 2024-09-16T20:49:04.868 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047266351999999998 2024-09-16T20:49:04.868 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.868 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.868 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:04.868 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528706+0000", 2024-09-16T20:49:04.868 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6796e-05 2024-09-16T20:49:04.868 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.868 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.868 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:04.869 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529148+0000", 2024-09-16T20:49:04.869 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00044182999999999999 2024-09-16T20:49:04.869 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.869 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.869 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:04.869 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531792+0000", 2024-09-16T20:49:04.869 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0026443119999999998 2024-09-16T20:49:04.869 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.869 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.869 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:04.869 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535319+0000", 2024-09-16T20:49:04.869 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0035266439999999998 2024-09-16T20:49:04.869 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.870 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.870 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:04.870 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535372+0000", 2024-09-16T20:49:04.870 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3207000000000001e-05 2024-09-16T20:49:04.870 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.870 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.870 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:04.870 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535414+0000", 2024-09-16T20:49:04.870 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1675000000000003e-05 2024-09-16T20:49:04.870 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.870 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:04.870 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.870 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.870 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.870 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3578 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:04.871 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:49:04.871 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.300080589, 2024-09-16T20:49:04.871 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054189586999999997, 2024-09-16T20:49:04.871 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:04.871 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:04.871 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:04.871 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:04.871 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:04.871 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:04.871 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3578 2024-09-16T20:49:04.871 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.871 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:04.871 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.871 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:04.872 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:49:04.872 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.872 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.872 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.872 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:04.872 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:49:04.872 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.872 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.872 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.872 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:04.872 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:49:04.872 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0800000000000001e-07 2024-09-16T20:49:04.872 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.872 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.873 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:04.873 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481363+0000", 2024-09-16T20:49:04.873 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0339999999999999e-06 2024-09-16T20:49:04.873 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.873 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.873 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:04.873 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481371+0000", 2024-09-16T20:49:04.873 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4440000000000002e-06 2024-09-16T20:49:04.873 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.873 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.873 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:04.873 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481374+0000", 2024-09-16T20:49:04.873 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2049999999999998e-06 2024-09-16T20:49:04.873 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.874 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.874 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:04.874 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529555+0000", 2024-09-16T20:49:04.874 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048181202999999999 2024-09-16T20:49:04.874 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.874 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.874 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:04.874 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529616+0000", 2024-09-16T20:49:04.874 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0912000000000002e-05 2024-09-16T20:49:04.874 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.874 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.874 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:04.874 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529939+0000", 2024-09-16T20:49:04.874 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00032269800000000001 2024-09-16T20:49:04.875 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.875 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.875 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:04.875 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532602+0000", 2024-09-16T20:49:04.875 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0026634940000000002 2024-09-16T20:49:04.875 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.875 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.875 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:04.875 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535465+0000", 2024-09-16T20:49:04.875 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.002862149 2024-09-16T20:49:04.875 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.875 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.875 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:04.875 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535516+0000", 2024-09-16T20:49:04.876 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1696000000000001e-05 2024-09-16T20:49:04.876 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.876 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.876 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:04.876 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535551+0000", 2024-09-16T20:49:04.876 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4443999999999998e-05 2024-09-16T20:49:04.876 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.876 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:04.876 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.876 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.876 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.876 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3579 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:04.876 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:49:04.876 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.30006215, 2024-09-16T20:49:04.877 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054295151, 2024-09-16T20:49:04.877 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:04.877 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:04.877 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:04.877 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:04.877 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:04.877 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:04.877 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3579 2024-09-16T20:49:04.877 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.877 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:04.877 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.877 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:04.877 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:49:04.877 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.878 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.878 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.878 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:04.878 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:49:04.878 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.878 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.878 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.878 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:04.878 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:49:04.878 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3300000000000001e-07 2024-09-16T20:49:04.878 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.878 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.878 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:04.878 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481381+0000", 2024-09-16T20:49:04.879 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.649e-06 2024-09-16T20:49:04.879 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.879 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.879 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:04.879 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481387+0000", 2024-09-16T20:49:04.879 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4519999999999996e-06 2024-09-16T20:49:04.879 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.879 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.879 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:04.879 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481390+0000", 2024-09-16T20:49:04.879 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0170000000000001e-06 2024-09-16T20:49:04.879 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.879 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.879 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:04.880 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530756+0000", 2024-09-16T20:49:04.880 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049365882999999999 2024-09-16T20:49:04.880 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.880 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.880 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:04.880 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530845+0000", 2024-09-16T20:49:04.880 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9197000000000007e-05 2024-09-16T20:49:04.880 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.880 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.880 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:04.880 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531191+0000", 2024-09-16T20:49:04.880 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00034562500000000001 2024-09-16T20:49:04.880 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.880 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.881 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:04.881 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534078+0000", 2024-09-16T20:49:04.881 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0028878179999999999 2024-09-16T20:49:04.881 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.881 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.881 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:04.881 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535589+0000", 2024-09-16T20:49:04.881 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00151085 2024-09-16T20:49:04.881 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.881 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.881 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:04.881 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535639+0000", 2024-09-16T20:49:04.881 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9697000000000002e-05 2024-09-16T20:49:04.881 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.882 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.882 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:04.882 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535675+0000", 2024-09-16T20:49:04.882 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5729999999999998e-05 2024-09-16T20:49:04.882 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.882 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:04.882 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.882 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.882 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.882 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3580 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:04.882 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:49:04.882 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.300048518000001, 2024-09-16T20:49:04.882 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054397361999999998, 2024-09-16T20:49:04.882 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:04.883 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:04.883 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:04.883 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:04.883 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:04.883 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:04.883 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3580 2024-09-16T20:49:04.883 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.883 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:04.883 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.883 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:04.883 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:49:04.883 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.883 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.883 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.884 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:04.884 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:49:04.884 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.884 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.884 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.884 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:04.884 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:49:04.884 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2600000000000001e-07 2024-09-16T20:49:04.884 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.884 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.884 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:04.884 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481395+0000", 2024-09-16T20:49:04.884 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.544e-06 2024-09-16T20:49:04.884 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.885 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.885 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:04.885 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481404+0000", 2024-09-16T20:49:04.885 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8850000000000005e-06 2024-09-16T20:49:04.885 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.885 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.885 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:04.885 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481407+0000", 2024-09-16T20:49:04.885 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1540000000000002e-06 2024-09-16T20:49:04.885 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.885 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.885 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:04.885 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531823+0000", 2024-09-16T20:49:04.885 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050415637999999999 2024-09-16T20:49:04.886 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.886 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.886 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:04.886 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531895+0000", 2024-09-16T20:49:04.886 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2179000000000001e-05 2024-09-16T20:49:04.886 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.886 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.886 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:04.886 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532236+0000", 2024-09-16T20:49:04.886 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00034073799999999999 2024-09-16T20:49:04.886 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.886 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.886 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:04.886 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534083+0000", 2024-09-16T20:49:04.887 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001847298 2024-09-16T20:49:04.887 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.887 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.887 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:04.887 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535714+0000", 2024-09-16T20:49:04.887 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0016316810000000001 2024-09-16T20:49:04.887 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.887 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.887 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:04.887 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535759+0000", 2024-09-16T20:49:04.887 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4478999999999999e-05 2024-09-16T20:49:04.887 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.887 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.887 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:04.888 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535791+0000", 2024-09-16T20:49:04.888 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.154e-05 2024-09-16T20:49:04.888 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.888 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:04.888 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.888 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.888 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.888 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3581 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:04.888 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:49:04.888 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.300026520999999, 2024-09-16T20:49:04.888 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.057094403000000002, 2024-09-16T20:49:04.888 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:04.888 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:04.888 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:04.888 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:04.889 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:04.889 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:04.889 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3581 2024-09-16T20:49:04.889 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.889 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:04.889 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.889 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:04.889 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:49:04.889 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.889 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.889 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.889 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:04.889 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:49:04.889 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:04.890 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.890 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.890 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:04.890 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:49:04.890 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8799999999999998e-07 2024-09-16T20:49:04.890 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.890 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.890 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:04.890 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481418+0000", 2024-09-16T20:49:04.890 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1859999999999999e-06 2024-09-16T20:49:04.890 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.890 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.890 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:04.890 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481423+0000", 2024-09-16T20:49:04.891 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8279999999999999e-06 2024-09-16T20:49:04.891 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.891 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.891 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:04.891 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481426+0000", 2024-09-16T20:49:04.891 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0110000000000001e-06 2024-09-16T20:49:04.891 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.891 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.891 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:04.891 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532631+0000", 2024-09-16T20:49:04.891 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051204514999999999 2024-09-16T20:49:04.891 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.891 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.891 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:04.892 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.533381+0000", 2024-09-16T20:49:04.892 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00075029000000000003 2024-09-16T20:49:04.892 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.892 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.892 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:04.892 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.533739+0000", 2024-09-16T20:49:04.892 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00035773100000000002 2024-09-16T20:49:04.892 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.892 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.892 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:04.892 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535123+0000", 2024-09-16T20:49:04.892 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0013836160000000001 2024-09-16T20:49:04.892 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.892 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.893 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:04.893 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538413+0000", 2024-09-16T20:49:04.893 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0032902510000000001 2024-09-16T20:49:04.893 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.893 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.893 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:04.893 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538470+0000", 2024-09-16T20:49:04.893 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6697999999999998e-05 2024-09-16T20:49:04.893 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:04.893 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:04.893 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:04.893 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538510+0000", 2024-09-16T20:49:04.893 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9988999999999999e-05 2024-09-16T20:49:04.893 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.893 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:04.894 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.894 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:04.894 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:04.894 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:04.896 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:49:04.939 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:04.936+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:04.998 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:04.997+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:05.050 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:05.050 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:49:05.050 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:49:05.050 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:05.060 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:49:05.099 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:05.098+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:05.181 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #18, image image_client.0-clone18 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent19 2024-09-16T20:49:05.184 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:05.184 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:49:05.184 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:49:05.184 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:49:05.184 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:05.193 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:49:05.200 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:05.200+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:05.226 INFO:teuthology.orchestra.run.smithi139.stdout:2690 write 0x9382800 thru 0x938d6d3 (0xaed4 bytes) 2024-09-16T20:49:05.227 INFO:teuthology.orchestra.run.smithi139.stdout:2692 trunc from 0x9c55600 to 0x8b23a00 2024-09-16T20:49:05.286 INFO:teuthology.orchestra.run.smithi139.stdout:2695 read 0x3105600 thru 0x310e07a (0x8a7b bytes) 2024-09-16T20:49:05.289 INFO:teuthology.orchestra.run.smithi139.stdout:2696 trunc from 0x8b23a00 to 0x62ef400 2024-09-16T20:49:05.301 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:05.300+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:05.340 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:05.340 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:49:05.340 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:49:05.340 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:49:05.340 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.340 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3709 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1464320~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:05.340 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.638971+0000", 2024-09-16T20:49:05.340 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 42.699045869000003, 2024-09-16T20:49:05.340 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049858670000000001, 2024-09-16T20:49:05.340 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:05.340 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:05.341 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:05.341 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:05.341 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:05.341 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:05.341 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3709 2024-09-16T20:49:05.341 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.341 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:05.341 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.341 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:05.341 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638971+0000", 2024-09-16T20:49:05.341 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:05.341 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.341 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.341 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:05.342 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638971+0000", 2024-09-16T20:49:05.342 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:05.342 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.342 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.342 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:05.342 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638972+0000", 2024-09-16T20:49:05.342 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0400000000000002e-07 2024-09-16T20:49:05.342 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.342 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.342 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:05.342 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638973+0000", 2024-09-16T20:49:05.342 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.578e-06 2024-09-16T20:49:05.342 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.342 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.343 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:05.343 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638979+0000", 2024-09-16T20:49:05.343 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.502e-06 2024-09-16T20:49:05.343 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.343 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.343 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:05.343 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638983+0000", 2024-09-16T20:49:05.343 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2690000000000001e-06 2024-09-16T20:49:05.343 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.343 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.343 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:05.343 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688628+0000", 2024-09-16T20:49:05.343 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049644547999999997 2024-09-16T20:49:05.343 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.344 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.344 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:05.344 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688718+0000", 2024-09-16T20:49:05.344 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0802000000000002e-05 2024-09-16T20:49:05.344 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.344 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.344 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:05.344 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688830+0000", 2024-09-16T20:49:05.344 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000111667 2024-09-16T20:49:05.344 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:05.344 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:05.344 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:05.344 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.344 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.345 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3712 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1472512~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:05.345 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:49:05.345 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 42.699029340999999, 2024-09-16T20:49:05.345 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050004305999999998, 2024-09-16T20:49:05.345 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:05.345 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:05.345 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:05.345 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:05.345 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:05.345 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:05.345 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3712 2024-09-16T20:49:05.345 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.345 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:05.345 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.346 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:05.346 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:49:05.346 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:05.346 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.346 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.346 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:05.346 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:49:05.346 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:05.346 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.346 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.346 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:05.346 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:49:05.346 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0199999999999998e-07 2024-09-16T20:49:05.346 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.347 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.347 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:05.347 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638990+0000", 2024-09-16T20:49:05.347 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.618e-06 2024-09-16T20:49:05.347 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.347 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.347 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:05.347 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638995+0000", 2024-09-16T20:49:05.347 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3940000000000004e-06 2024-09-16T20:49:05.347 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.347 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.347 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:05.347 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639000+0000", 2024-09-16T20:49:05.347 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9309999999999999e-06 2024-09-16T20:49:05.348 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.348 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.348 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:05.348 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688867+0000", 2024-09-16T20:49:05.348 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049866955999999997 2024-09-16T20:49:05.348 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.348 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.348 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:05.348 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688933+0000", 2024-09-16T20:49:05.348 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5359999999999998e-05 2024-09-16T20:49:05.348 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.348 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.348 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:05.348 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688992+0000", 2024-09-16T20:49:05.349 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9744999999999998e-05 2024-09-16T20:49:05.349 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:05.349 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:05.349 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:05.349 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.349 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.349 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3714 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1480704~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:05.349 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:49:05.349 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 42.699011751, 2024-09-16T20:49:05.349 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050152552000000003, 2024-09-16T20:49:05.349 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:05.349 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:05.349 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:05.349 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:05.350 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:05.350 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:05.350 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3714 2024-09-16T20:49:05.350 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.350 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:05.350 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.350 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:05.350 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:49:05.350 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:05.350 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.350 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.350 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:05.350 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:49:05.350 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:05.350 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.351 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.351 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:05.351 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:49:05.351 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0499999999999999e-07 2024-09-16T20:49:05.351 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.351 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.351 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:05.351 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639007+0000", 2024-09-16T20:49:05.351 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5340000000000001e-06 2024-09-16T20:49:05.351 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.351 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.351 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:05.351 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639013+0000", 2024-09-16T20:49:05.351 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3260000000000003e-06 2024-09-16T20:49:05.352 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.352 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.352 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:05.352 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639018+0000", 2024-09-16T20:49:05.352 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2850000000000002e-06 2024-09-16T20:49:05.352 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.352 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.352 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:05.352 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689026+0000", 2024-09-16T20:49:05.352 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.05000798 2024-09-16T20:49:05.352 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.352 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.352 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:05.352 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689077+0000", 2024-09-16T20:49:05.353 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1063999999999998e-05 2024-09-16T20:49:05.353 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.353 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.353 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:05.353 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689158+0000", 2024-09-16T20:49:05.353 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1057999999999995e-05 2024-09-16T20:49:05.353 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:05.353 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:05.353 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:05.353 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.353 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.353 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3716 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1488896~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:05.353 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:49:05.354 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 42.698993940999998, 2024-09-16T20:49:05.354 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050280142, 2024-09-16T20:49:05.354 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:05.354 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:05.354 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:05.354 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:05.354 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:05.354 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:05.354 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3716 2024-09-16T20:49:05.354 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.354 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:05.354 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.354 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:05.354 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:49:05.355 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:05.355 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.355 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.355 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:05.355 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:49:05.355 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:05.355 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.355 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.355 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:05.355 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639024+0000", 2024-09-16T20:49:05.355 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9999999999999997e-07 2024-09-16T20:49:05.355 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.355 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.355 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:05.356 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639026+0000", 2024-09-16T20:49:05.356 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.781e-06 2024-09-16T20:49:05.356 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.356 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.356 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:05.356 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639032+0000", 2024-09-16T20:49:05.356 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7780000000000004e-06 2024-09-16T20:49:05.356 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.356 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.356 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:05.356 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639036+0000", 2024-09-16T20:49:05.356 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6909999999999996e-06 2024-09-16T20:49:05.356 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.356 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.357 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:05.357 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689191+0000", 2024-09-16T20:49:05.357 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050155284000000001 2024-09-16T20:49:05.357 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.357 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.357 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:05.357 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689242+0000", 2024-09-16T20:49:05.357 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0662999999999999e-05 2024-09-16T20:49:05.357 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.357 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.357 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:05.357 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689304+0000", 2024-09-16T20:49:05.357 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1345000000000003e-05 2024-09-16T20:49:05.357 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:05.358 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:05.358 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:05.358 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.358 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.358 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3719 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1497088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:05.358 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:49:05.358 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 42.698966663, 2024-09-16T20:49:05.358 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050399715999999997, 2024-09-16T20:49:05.358 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:05.358 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:05.358 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:05.358 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:05.358 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:05.358 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:05.359 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3719 2024-09-16T20:49:05.359 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.359 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:05.359 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.359 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:05.359 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:49:05.359 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:05.359 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.359 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.359 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:05.359 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:49:05.359 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:05.359 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.359 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.359 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:05.360 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:49:05.360 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9700000000000002e-07 2024-09-16T20:49:05.360 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.360 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.360 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:05.360 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639053+0000", 2024-09-16T20:49:05.360 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2900000000000001e-06 2024-09-16T20:49:05.360 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.360 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.360 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:05.360 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639060+0000", 2024-09-16T20:49:05.360 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7750000000000004e-06 2024-09-16T20:49:05.360 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.361 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.361 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:05.361 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639066+0000", 2024-09-16T20:49:05.361 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0639999999999999e-06 2024-09-16T20:49:05.361 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.361 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.361 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:05.361 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689337+0000", 2024-09-16T20:49:05.361 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050270784999999998 2024-09-16T20:49:05.361 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.361 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.361 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:05.361 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689392+0000", 2024-09-16T20:49:05.361 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4920999999999998e-05 2024-09-16T20:49:05.361 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.362 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.362 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:05.362 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689450+0000", 2024-09-16T20:49:05.362 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8483999999999998e-05 2024-09-16T20:49:05.362 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:05.362 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:05.362 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:05.362 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.362 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.362 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3721 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1505280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:05.362 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:49:05.362 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 42.698943530999998, 2024-09-16T20:49:05.362 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050513608000000002, 2024-09-16T20:49:05.363 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:05.363 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:05.363 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:05.363 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:05.363 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:05.363 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:05.363 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3721 2024-09-16T20:49:05.363 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.363 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:05.363 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.363 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:05.363 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:49:05.363 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:05.363 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.363 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.364 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:05.364 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:49:05.364 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:05.364 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.364 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.364 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:05.364 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:49:05.364 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7e-07 2024-09-16T20:49:05.364 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.364 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.364 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:05.364 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639076+0000", 2024-09-16T20:49:05.364 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8899999999999999e-06 2024-09-16T20:49:05.364 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.365 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.365 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:05.365 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639083+0000", 2024-09-16T20:49:05.365 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1210000000000001e-06 2024-09-16T20:49:05.365 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.365 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.365 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:05.365 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639089+0000", 2024-09-16T20:49:05.365 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.643e-06 2024-09-16T20:49:05.365 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.365 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.365 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:05.365 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689474+0000", 2024-09-16T20:49:05.365 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050385679000000003 2024-09-16T20:49:05.365 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.366 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.366 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:05.366 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689533+0000", 2024-09-16T20:49:05.366 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8303000000000001e-05 2024-09-16T20:49:05.366 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.366 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.366 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:05.366 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689587+0000", 2024-09-16T20:49:05.366 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4601999999999997e-05 2024-09-16T20:49:05.366 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:05.366 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:05.366 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:05.366 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.366 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.367 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3723 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1513472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:05.367 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:49:05.367 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 42.698912727, 2024-09-16T20:49:05.367 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050699691999999998, 2024-09-16T20:49:05.367 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:05.367 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:05.367 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:05.367 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:05.367 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:05.367 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:05.367 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3723 2024-09-16T20:49:05.367 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.367 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:05.367 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.368 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:05.368 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:49:05.368 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:05.368 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.368 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.368 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:05.368 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:49:05.368 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:05.368 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.368 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.368 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:05.368 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:49:05.368 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3399999999999999e-07 2024-09-16T20:49:05.368 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.369 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.369 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:05.369 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639107+0000", 2024-09-16T20:49:05.369 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3860000000000001e-06 2024-09-16T20:49:05.369 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.369 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.369 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:05.369 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639114+0000", 2024-09-16T20:49:05.369 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9800000000000001e-06 2024-09-16T20:49:05.369 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.369 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.369 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:05.369 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639129+0000", 2024-09-16T20:49:05.369 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4887e-05 2024-09-16T20:49:05.369 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.370 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.370 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:05.370 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689625+0000", 2024-09-16T20:49:05.370 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050495848000000003 2024-09-16T20:49:05.370 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.370 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.370 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:05.370 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689720+0000", 2024-09-16T20:49:05.370 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4821000000000006e-05 2024-09-16T20:49:05.370 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.370 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.370 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:05.370 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689804+0000", 2024-09-16T20:49:05.370 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4235999999999998e-05 2024-09-16T20:49:05.371 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:05.371 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:05.371 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:05.371 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.371 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.371 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3725 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1521664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:05.371 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:49:05.371 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 42.698879130999998, 2024-09-16T20:49:05.371 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050797660000000001, 2024-09-16T20:49:05.371 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:05.371 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:05.371 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:05.371 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:05.371 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:05.371 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:05.372 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3725 2024-09-16T20:49:05.372 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.372 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:05.372 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.372 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:05.372 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:49:05.372 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:05.372 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.372 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.372 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:05.372 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:49:05.372 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:05.372 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.372 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.373 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:05.373 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639139+0000", 2024-09-16T20:49:05.373 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8999999999999997e-07 2024-09-16T20:49:05.373 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.373 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.373 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:05.373 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639141+0000", 2024-09-16T20:49:05.373 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.182e-06 2024-09-16T20:49:05.373 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.373 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.373 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:05.373 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639154+0000", 2024-09-16T20:49:05.373 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3626e-05 2024-09-16T20:49:05.373 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.374 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.374 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:05.374 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639160+0000", 2024-09-16T20:49:05.374 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6810000000000003e-06 2024-09-16T20:49:05.374 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.374 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.374 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:05.374 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689831+0000", 2024-09-16T20:49:05.374 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050670907000000001 2024-09-16T20:49:05.374 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.374 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.374 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:05.374 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689882+0000", 2024-09-16T20:49:05.374 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0445999999999998e-05 2024-09-16T20:49:05.375 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.375 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.375 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:05.375 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689936+0000", 2024-09-16T20:49:05.375 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4327999999999999e-05 2024-09-16T20:49:05.375 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:05.375 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:05.375 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:05.375 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.375 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.375 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3727 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1529856~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:05.375 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:49:05.375 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 42.698843554, 2024-09-16T20:49:05.375 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051016922999999999, 2024-09-16T20:49:05.376 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:05.376 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:05.376 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:05.376 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:05.376 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:05.376 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:05.376 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3727 2024-09-16T20:49:05.376 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.376 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:05.376 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.376 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:05.376 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:49:05.376 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:05.376 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.377 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.377 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:05.377 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:49:05.377 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:05.377 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.377 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.377 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:05.377 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:49:05.377 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9999999999999998e-07 2024-09-16T20:49:05.377 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.377 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.377 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:05.377 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639183+0000", 2024-09-16T20:49:05.377 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2399999999999996e-06 2024-09-16T20:49:05.377 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.378 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.378 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:05.378 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639195+0000", 2024-09-16T20:49:05.378 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1113e-05 2024-09-16T20:49:05.378 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.378 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.378 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:05.378 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639206+0000", 2024-09-16T20:49:05.378 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1503e-05 2024-09-16T20:49:05.378 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.378 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.378 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:05.378 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689969+0000", 2024-09-16T20:49:05.378 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050762676 2024-09-16T20:49:05.379 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.379 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.379 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:05.379 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690071+0000", 2024-09-16T20:49:05.379 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102163 2024-09-16T20:49:05.379 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.379 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.379 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:05.379 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690191+0000", 2024-09-16T20:49:05.379 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000119728 2024-09-16T20:49:05.379 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:05.379 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:05.379 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:05.379 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.379 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.380 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3730 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1538048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:05.380 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:49:05.380 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 42.698801991000003, 2024-09-16T20:49:05.380 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051131467999999999, 2024-09-16T20:49:05.380 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:05.380 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:05.380 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:05.380 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:05.380 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:05.380 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:05.380 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3730 2024-09-16T20:49:05.380 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.380 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:05.380 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.381 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:05.381 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:49:05.381 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:05.381 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.381 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.381 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:05.381 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:49:05.381 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:05.381 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.381 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.381 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:05.381 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639216+0000", 2024-09-16T20:49:05.381 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.39e-07 2024-09-16T20:49:05.381 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.382 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.382 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:05.382 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639218+0000", 2024-09-16T20:49:05.382 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.537e-06 2024-09-16T20:49:05.382 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.382 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.382 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:05.382 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639229+0000", 2024-09-16T20:49:05.382 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1122e-05 2024-09-16T20:49:05.382 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.382 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.382 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:05.382 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639237+0000", 2024-09-16T20:49:05.382 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8509999999999999e-06 2024-09-16T20:49:05.383 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.383 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.383 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:05.383 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690227+0000", 2024-09-16T20:49:05.383 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050990147 2024-09-16T20:49:05.383 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.383 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.383 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:05.383 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690278+0000", 2024-09-16T20:49:05.383 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0223999999999999e-05 2024-09-16T20:49:05.383 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.383 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.383 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:05.383 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690347+0000", 2024-09-16T20:49:05.384 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9147999999999996e-05 2024-09-16T20:49:05.384 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:05.384 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:05.384 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:05.384 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.384 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.384 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3731 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1546240~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:05.384 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:49:05.384 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 42.698771282999999, 2024-09-16T20:49:05.384 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051276744999999999, 2024-09-16T20:49:05.384 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:05.384 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:05.384 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:05.384 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:05.385 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:05.385 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:05.385 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3731 2024-09-16T20:49:05.385 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.385 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:05.385 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.385 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:05.385 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:49:05.385 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:05.385 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.385 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.385 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:05.385 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:49:05.385 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:05.386 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.386 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.386 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:05.386 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:49:05.386 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9900000000000001e-07 2024-09-16T20:49:05.386 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.386 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.386 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:05.386 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639249+0000", 2024-09-16T20:49:05.386 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3630000000000001e-06 2024-09-16T20:49:05.386 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.386 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.386 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:05.387 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639255+0000", 2024-09-16T20:49:05.387 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4960000000000001e-06 2024-09-16T20:49:05.387 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.387 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.387 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:05.387 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639261+0000", 2024-09-16T20:49:05.387 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2839999999999997e-06 2024-09-16T20:49:05.387 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.387 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.387 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:05.387 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690375+0000", 2024-09-16T20:49:05.387 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051114684000000001 2024-09-16T20:49:05.387 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.387 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.388 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:05.388 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690425+0000", 2024-09-16T20:49:05.388 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9379999999999998e-05 2024-09-16T20:49:05.388 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.388 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.388 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:05.388 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690523+0000", 2024-09-16T20:49:05.388 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8139000000000003e-05 2024-09-16T20:49:05.388 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:05.388 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:05.388 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:05.388 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.388 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.388 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3732 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1554432~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:05.389 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:49:05.389 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 42.698725500999998, 2024-09-16T20:49:05.389 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051380096, 2024-09-16T20:49:05.389 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:05.389 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:05.389 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:05.389 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:05.389 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:05.389 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:05.389 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3732 2024-09-16T20:49:05.389 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.389 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:05.389 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.389 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:05.390 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:49:05.390 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:05.390 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.390 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.390 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:05.390 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:49:05.390 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:05.390 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.390 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.390 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:05.390 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:49:05.390 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6100000000000001e-07 2024-09-16T20:49:05.390 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.390 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.391 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:05.391 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639295+0000", 2024-09-16T20:49:05.391 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.447e-06 2024-09-16T20:49:05.391 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.391 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.391 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:05.391 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639302+0000", 2024-09-16T20:49:05.391 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0570000000000004e-06 2024-09-16T20:49:05.391 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.391 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.391 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:05.391 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639307+0000", 2024-09-16T20:49:05.391 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1429999999999997e-06 2024-09-16T20:49:05.391 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.391 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.392 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:05.392 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690555+0000", 2024-09-16T20:49:05.392 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051247963000000001 2024-09-16T20:49:05.392 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.392 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.392 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:05.392 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690602+0000", 2024-09-16T20:49:05.392 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7602000000000003e-05 2024-09-16T20:49:05.392 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.392 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.392 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:05.392 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690672+0000", 2024-09-16T20:49:05.392 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9423000000000003e-05 2024-09-16T20:49:05.392 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:05.393 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:05.393 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:05.393 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.393 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.393 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3734 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1562624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:05.393 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:49:05.393 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 42.698703850999998, 2024-09-16T20:49:05.393 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051490610999999999, 2024-09-16T20:49:05.393 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:05.393 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:05.393 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:05.393 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:05.393 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:05.393 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:05.394 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3734 2024-09-16T20:49:05.394 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.394 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:05.394 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.394 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:05.394 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:49:05.394 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:05.394 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.394 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.394 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:05.394 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:49:05.394 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:05.394 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.394 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.394 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:05.395 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639314+0000", 2024-09-16T20:49:05.395 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4400000000000001e-07 2024-09-16T20:49:05.395 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.395 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.395 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:05.395 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639316+0000", 2024-09-16T20:49:05.395 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1459999999999999e-06 2024-09-16T20:49:05.395 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.395 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.395 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:05.395 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639322+0000", 2024-09-16T20:49:05.395 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7610000000000004e-06 2024-09-16T20:49:05.395 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.395 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.396 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:05.396 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639327+0000", 2024-09-16T20:49:05.396 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.186e-06 2024-09-16T20:49:05.396 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.396 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.396 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:05.396 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690698+0000", 2024-09-16T20:49:05.396 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051371223000000001 2024-09-16T20:49:05.396 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.396 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.396 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:05.396 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690748+0000", 2024-09-16T20:49:05.396 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0192999999999999e-05 2024-09-16T20:49:05.397 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.397 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.397 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:05.397 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690804+0000", 2024-09-16T20:49:05.397 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5658000000000001e-05 2024-09-16T20:49:05.397 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:05.397 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:05.397 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:05.397 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.397 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.397 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3736 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1570816~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:05.397 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:49:05.397 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 42.698684591999999, 2024-09-16T20:49:05.398 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051596528000000003, 2024-09-16T20:49:05.398 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:05.398 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:05.398 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:05.398 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:05.398 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:05.398 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:05.398 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3736 2024-09-16T20:49:05.398 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.398 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:05.398 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.398 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:05.398 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:49:05.399 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:05.399 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.399 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.399 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:05.399 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:49:05.399 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:05.399 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.399 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.399 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:05.399 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:49:05.399 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9999999999999999e-07 2024-09-16T20:49:05.399 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.399 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.399 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:05.400 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639335+0000", 2024-09-16T20:49:05.400 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.635e-06 2024-09-16T20:49:05.400 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.400 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.400 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:05.400 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639340+0000", 2024-09-16T20:49:05.400 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5840000000000002e-06 2024-09-16T20:49:05.400 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.400 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.400 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:05.400 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639346+0000", 2024-09-16T20:49:05.400 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1190000000000002e-06 2024-09-16T20:49:05.400 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.400 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.401 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:05.401 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690817+0000", 2024-09-16T20:49:05.401 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051470721999999997 2024-09-16T20:49:05.401 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.401 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.401 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:05.401 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690875+0000", 2024-09-16T20:49:05.401 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7441000000000001e-05 2024-09-16T20:49:05.401 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.401 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.401 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:05.401 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690929+0000", 2024-09-16T20:49:05.401 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4727e-05 2024-09-16T20:49:05.401 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:05.402 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:05.402 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:05.402 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.402 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.402 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5434 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:05.402 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:49:05.402 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 42.473423025999999, 2024-09-16T20:49:05.402 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099755831000000003, 2024-09-16T20:49:05.402 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:05.402 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:05.402 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:05.402 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:05.402 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:05.402 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:05.402 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5434 2024-09-16T20:49:05.403 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.403 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:05.403 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.403 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:05.403 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:49:05.403 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:05.403 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.403 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.403 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:05.403 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:49:05.403 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:05.403 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.403 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.403 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:05.404 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864595+0000", 2024-09-16T20:49:05.404 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1099999999999995e-07 2024-09-16T20:49:05.404 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.404 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.404 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:05.404 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864597+0000", 2024-09-16T20:49:05.404 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3989999999999998e-06 2024-09-16T20:49:05.404 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.404 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.404 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:05.404 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864603+0000", 2024-09-16T20:49:05.404 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9309999999999996e-06 2024-09-16T20:49:05.404 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.404 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.405 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:05.405 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864607+0000", 2024-09-16T20:49:05.405 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6200000000000001e-06 2024-09-16T20:49:05.405 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.405 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.405 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:05.405 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964164+0000", 2024-09-16T20:49:05.405 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099557162000000005 2024-09-16T20:49:05.405 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.405 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.405 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:05.405 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964244+0000", 2024-09-16T20:49:05.405 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9597000000000004e-05 2024-09-16T20:49:05.405 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.405 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.406 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:05.406 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964350+0000", 2024-09-16T20:49:05.406 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010641099999999999 2024-09-16T20:49:05.406 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:05.406 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:05.406 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:05.406 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.406 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.406 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5435 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 20480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:05.406 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:49:05.406 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 42.473393203999997, 2024-09-16T20:49:05.406 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100897102, 2024-09-16T20:49:05.406 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:05.406 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:05.406 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:05.407 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:05.407 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:05.407 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:05.407 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5435 2024-09-16T20:49:05.407 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.407 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:05.407 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.407 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:05.407 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:49:05.407 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:05.407 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.407 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.407 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:05.407 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:49:05.408 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:05.408 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.408 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.408 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:05.408 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:49:05.408 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8000000000000001e-07 2024-09-16T20:49:05.408 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.408 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.408 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:05.408 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864626+0000", 2024-09-16T20:49:05.408 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.745e-06 2024-09-16T20:49:05.408 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.408 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.408 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:05.409 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864633+0000", 2024-09-16T20:49:05.409 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3339999999999998e-06 2024-09-16T20:49:05.409 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.409 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.409 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:05.409 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864636+0000", 2024-09-16T20:49:05.409 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1200000000000002e-06 2024-09-16T20:49:05.409 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.409 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.409 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:05.409 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965299+0000", 2024-09-16T20:49:05.409 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10066352000000001 2024-09-16T20:49:05.410 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.410 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.410 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:05.410 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965390+0000", 2024-09-16T20:49:05.410 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0894000000000001e-05 2024-09-16T20:49:05.410 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.410 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.410 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:05.410 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965521+0000", 2024-09-16T20:49:05.410 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013110899999999999 2024-09-16T20:49:05.410 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:05.410 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:05.410 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:05.410 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.411 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.411 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:152 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 327680~32768 out=32768b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:49:05.411 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:49:05.411 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 41.174896261999997, 2024-09-16T20:49:05.411 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11869405800000001, 2024-09-16T20:49:05.411 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:05.411 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:05.411 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:05.411 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:05.411 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:49:05.411 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:49:05.411 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 152 2024-09-16T20:49:05.411 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.411 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:05.411 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.412 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:05.412 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:49:05.412 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:05.412 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.412 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.412 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:05.412 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:49:05.412 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:05.412 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.412 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.412 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:05.412 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163122+0000", 2024-09-16T20:49:05.412 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6400000000000003e-07 2024-09-16T20:49:05.413 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.413 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.413 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:05.413 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163124+0000", 2024-09-16T20:49:05.413 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.565e-06 2024-09-16T20:49:05.413 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.413 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.413 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:05.413 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163136+0000", 2024-09-16T20:49:05.413 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1717e-05 2024-09-16T20:49:05.413 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.413 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.413 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:05.413 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163142+0000", 2024-09-16T20:49:05.413 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1750000000000002e-06 2024-09-16T20:49:05.414 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.414 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.414 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:05.414 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163606+0000", 2024-09-16T20:49:05.414 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00046418799999999998 2024-09-16T20:49:05.414 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.414 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.414 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:05.414 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163656+0000", 2024-09-16T20:49:05.414 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9852999999999999e-05 2024-09-16T20:49:05.414 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.414 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.414 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:05.414 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281815+0000", 2024-09-16T20:49:05.415 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11815909600000001 2024-09-16T20:49:05.415 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:05.415 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:05.415 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:05.415 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.415 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.415 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:153 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:49:05.415 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:49:05.415 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 41.174863309000003, 2024-09-16T20:49:05.415 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11903462400000001, 2024-09-16T20:49:05.415 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:05.415 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:05.415 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:05.415 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:05.415 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:49:05.416 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:49:05.416 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 153 2024-09-16T20:49:05.416 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.416 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:05.416 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.416 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:05.416 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:49:05.416 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:05.416 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.416 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.416 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:05.416 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:49:05.416 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:05.416 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.417 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.417 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:05.417 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163155+0000", 2024-09-16T20:49:05.417 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.37e-07 2024-09-16T20:49:05.417 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.417 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.417 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:05.417 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163157+0000", 2024-09-16T20:49:05.417 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6010000000000002e-06 2024-09-16T20:49:05.417 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.417 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.417 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:05.417 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163165+0000", 2024-09-16T20:49:05.417 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2619999999999998e-06 2024-09-16T20:49:05.418 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.418 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.418 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:05.418 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163171+0000", 2024-09-16T20:49:05.418 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.711e-06 2024-09-16T20:49:05.418 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.418 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.418 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:05.418 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281868+0000", 2024-09-16T20:49:05.418 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118696622 2024-09-16T20:49:05.418 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.418 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.418 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:05.418 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281969+0000", 2024-09-16T20:49:05.419 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101371 2024-09-16T20:49:05.419 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.419 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.419 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:05.419 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282189+0000", 2024-09-16T20:49:05.419 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00021952000000000001 2024-09-16T20:49:05.419 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:05.419 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:05.419 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:05.419 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.419 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.419 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:154 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 360448~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:49:05.419 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:49:05.419 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 41.174831537000003, 2024-09-16T20:49:05.419 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11932035100000001, 2024-09-16T20:49:05.420 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:05.420 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:05.420 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:05.420 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:05.420 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:49:05.420 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:49:05.420 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 154 2024-09-16T20:49:05.420 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.420 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:05.420 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.420 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:05.420 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:49:05.420 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:05.420 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.421 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.421 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:05.421 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:49:05.421 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:05.421 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.421 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.421 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:05.421 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:49:05.421 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:49:05.421 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.421 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.421 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:05.421 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163189+0000", 2024-09-16T20:49:05.421 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6840000000000001e-06 2024-09-16T20:49:05.421 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.422 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.422 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:05.422 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163196+0000", 2024-09-16T20:49:05.422 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7920000000000004e-06 2024-09-16T20:49:05.422 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.422 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.422 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:05.422 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163201+0000", 2024-09-16T20:49:05.422 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2809999999999999e-06 2024-09-16T20:49:05.422 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.422 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.422 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:05.422 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282219+0000", 2024-09-16T20:49:05.422 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.119018343 2024-09-16T20:49:05.423 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.423 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.423 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:05.423 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282305+0000", 2024-09-16T20:49:05.423 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5865000000000005e-05 2024-09-16T20:49:05.423 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.423 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.423 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:05.423 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282506+0000", 2024-09-16T20:49:05.423 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000200927 2024-09-16T20:49:05.423 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:05.423 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:05.423 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:05.423 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.424 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.424 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop(client.4366.0:3088 3.1e e38/23 3:79e5ec01:::rbd_header.110e891a9b2b:head v 38'306, mlcod=36'304)", 2024-09-16T20:49:05.424 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:49:05.424 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 41.173812669, 2024-09-16T20:49:05.424 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118102759, 2024-09-16T20:49:05.424 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:05.424 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:05.424 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:05.424 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:05.424 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.424 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:05.424 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:49:05.424 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:05.424 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.424 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.425 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:05.425 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:49:05.425 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:05.425 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.425 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.425 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:05.425 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:49:05.425 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.85e-07 2024-09-16T20:49:05.425 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.425 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.425 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:05.425 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164212+0000", 2024-09-16T20:49:05.425 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3999999999999997e-06 2024-09-16T20:49:05.425 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.426 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.426 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:05.426 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281553+0000", 2024-09-16T20:49:05.426 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11734135499999999 2024-09-16T20:49:05.426 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.426 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.426 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:05.426 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281567+0000", 2024-09-16T20:49:05.426 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4188999999999999e-05 2024-09-16T20:49:05.426 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.426 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.426 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:05.426 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281653+0000", 2024-09-16T20:49:05.426 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5385999999999999e-05 2024-09-16T20:49:05.426 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.427 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.427 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:05.427 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281679+0000", 2024-09-16T20:49:05.427 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6689000000000001e-05 2024-09-16T20:49:05.427 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.427 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.427 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:05.427 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282277+0000", 2024-09-16T20:49:05.427 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00059798000000000002 2024-09-16T20:49:05.427 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:05.427 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:05.427 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:05.427 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282307+0000", 2024-09-16T20:49:05.427 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9975e-05 2024-09-16T20:49:05.427 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:05.428 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:05.428 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:05.428 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:05.428 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:05.428 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:05.430 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:49:05.430 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:05.428+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:05.431 INFO:teuthology.orchestra.run.smithi139.stdout:2697 read 0x4315800 thru 0x4318c82 (0x3483 bytes) 2024-09-16T20:49:05.431 INFO:teuthology.orchestra.run.smithi139.stdout:2698 punch from 0x403bc00 to 0x403c000, (0x400 bytes) 2024-09-16T20:49:05.431 INFO:teuthology.orchestra.run.smithi139.stdout:2699 write 0x82e8000 thru 0x82ef470 (0x7471 bytes) 2024-09-16T20:49:05.431 INFO:teuthology.orchestra.run.smithi139.stdout:truncating to largest ever: 0xed89200 2024-09-16T20:49:05.431 INFO:teuthology.orchestra.run.smithi139.stdout:2700 trunc from 0x82ef600 to 0xed89200 2024-09-16T20:49:05.431 INFO:teuthology.orchestra.run.smithi139.stdout:2701 read 0x7b1ec00 thru 0x7b2b4c3 (0xc8c4 bytes) 2024-09-16T20:49:05.431 INFO:teuthology.orchestra.run.smithi139.stdout:2704 clone 21 order 23 su 262144 sc 5 2024-09-16T20:49:05.530 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:05.529+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:05.538 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:05.538 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:49:05.538 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:49:05.538 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:05.546 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:49:05.631 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:05.630+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:05.694 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:05.694 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:49:05.694 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:49:05.694 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:49:05.694 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:05.703 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:49:05.732 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:05.731+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:05.766 INFO:teuthology.orchestra.run.smithi029.stderr:reweighted osd.4 to 1 (10000) 2024-09-16T20:49:05.785 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-16T20:49:05.814 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:05.814 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:49:05.814 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:49:05.814 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:49:05.814 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.814 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6415 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3577856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:05.814 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871730+0000", 2024-09-16T20:49:05.815 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 45.940067943000003, 2024-09-16T20:49:05.815 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.04851482, 2024-09-16T20:49:05.815 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:05.815 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:05.815 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:05.815 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:05.815 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:05.815 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:05.815 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6415 2024-09-16T20:49:05.815 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.815 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:05.815 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.815 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:05.815 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871730+0000", 2024-09-16T20:49:05.816 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:05.816 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.816 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.816 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:05.816 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871730+0000", 2024-09-16T20:49:05.816 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:05.816 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.816 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.816 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:05.816 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871731+0000", 2024-09-16T20:49:05.817 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6899999999999998e-07 2024-09-16T20:49:05.817 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.817 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.817 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:05.817 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871733+0000", 2024-09-16T20:49:05.817 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0600000000000002e-06 2024-09-16T20:49:05.817 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.817 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.817 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:05.817 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871740+0000", 2024-09-16T20:49:05.817 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3749999999999997e-06 2024-09-16T20:49:05.817 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.817 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.817 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:05.818 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871749+0000", 2024-09-16T20:49:05.818 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9109999999999999e-06 2024-09-16T20:49:05.818 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.818 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.818 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:05.818 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920129+0000", 2024-09-16T20:49:05.818 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048380175999999997 2024-09-16T20:49:05.818 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.818 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.818 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:05.818 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920181+0000", 2024-09-16T20:49:05.818 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2077000000000003e-05 2024-09-16T20:49:05.818 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.818 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.819 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:05.819 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920245+0000", 2024-09-16T20:49:05.819 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3851999999999998e-05 2024-09-16T20:49:05.819 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:05.819 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:05.819 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:05.819 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.819 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.819 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6417 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3586048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:05.819 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871756+0000", 2024-09-16T20:49:05.819 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 45.940042308000002, 2024-09-16T20:49:05.819 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048628301999999998, 2024-09-16T20:49:05.819 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:05.819 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:05.819 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:05.820 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:05.820 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:05.820 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:05.820 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6417 2024-09-16T20:49:05.820 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.820 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:05.820 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.820 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:05.820 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871756+0000", 2024-09-16T20:49:05.820 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:05.820 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.820 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.820 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:05.820 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871756+0000", 2024-09-16T20:49:05.821 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:05.821 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.821 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.821 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:05.821 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871756+0000", 2024-09-16T20:49:05.821 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0100000000000005e-07 2024-09-16T20:49:05.821 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.821 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.821 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:05.821 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871758+0000", 2024-09-16T20:49:05.821 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.902e-06 2024-09-16T20:49:05.821 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.821 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.821 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:05.822 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871764+0000", 2024-09-16T20:49:05.822 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8170000000000004e-06 2024-09-16T20:49:05.822 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.822 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.822 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:05.822 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871769+0000", 2024-09-16T20:49:05.822 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6500000000000004e-06 2024-09-16T20:49:05.822 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.822 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.822 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:05.822 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920276+0000", 2024-09-16T20:49:05.822 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048507794999999999 2024-09-16T20:49:05.822 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.822 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.823 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:05.823 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920329+0000", 2024-09-16T20:49:05.823 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2534999999999998e-05 2024-09-16T20:49:05.823 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.823 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.823 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:05.823 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920384+0000", 2024-09-16T20:49:05.823 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5102000000000002e-05 2024-09-16T20:49:05.823 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:05.823 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:05.823 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:05.823 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.823 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.823 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6419 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3594240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:05.824 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871775+0000", 2024-09-16T20:49:05.824 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 45.940023130999997, 2024-09-16T20:49:05.824 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048754724999999999, 2024-09-16T20:49:05.824 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:05.824 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:05.824 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:05.824 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:05.824 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:05.824 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:05.824 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6419 2024-09-16T20:49:05.824 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.824 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:05.824 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.824 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:05.824 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871775+0000", 2024-09-16T20:49:05.825 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:05.825 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.825 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.825 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:05.825 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871775+0000", 2024-09-16T20:49:05.825 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:05.825 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.825 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.825 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:05.825 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871775+0000", 2024-09-16T20:49:05.825 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2500000000000001e-07 2024-09-16T20:49:05.825 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.825 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.825 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:05.826 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871777+0000", 2024-09-16T20:49:05.826 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9649999999999998e-06 2024-09-16T20:49:05.826 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.826 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.826 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:05.826 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871784+0000", 2024-09-16T20:49:05.826 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3419999999999996e-06 2024-09-16T20:49:05.826 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.826 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.826 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:05.826 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871789+0000", 2024-09-16T20:49:05.826 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1889999999999998e-06 2024-09-16T20:49:05.826 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.826 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.826 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:05.827 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920416+0000", 2024-09-16T20:49:05.827 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048627363999999999 2024-09-16T20:49:05.827 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.827 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.827 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:05.827 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920472+0000", 2024-09-16T20:49:05.827 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5429000000000001e-05 2024-09-16T20:49:05.827 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.827 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.827 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:05.827 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920530+0000", 2024-09-16T20:49:05.827 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8111e-05 2024-09-16T20:49:05.827 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:05.827 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:05.827 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:05.828 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.828 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.828 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6420 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3602432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:05.828 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871796+0000", 2024-09-16T20:49:05.828 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 45.940002217999997, 2024-09-16T20:49:05.828 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048857298, 2024-09-16T20:49:05.828 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:05.828 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:05.828 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:05.828 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:05.828 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:05.828 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:05.828 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6420 2024-09-16T20:49:05.829 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.829 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:05.829 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.829 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:05.829 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871796+0000", 2024-09-16T20:49:05.829 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:05.829 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.829 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.829 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:05.829 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871796+0000", 2024-09-16T20:49:05.829 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:05.829 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.829 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.829 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:05.829 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871854+0000", 2024-09-16T20:49:05.830 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8187999999999999e-05 2024-09-16T20:49:05.830 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.830 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.830 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:05.830 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871856+0000", 2024-09-16T20:49:05.830 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3549999999999999e-06 2024-09-16T20:49:05.830 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.830 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.830 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:05.830 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871866+0000", 2024-09-16T20:49:05.830 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6500000000000008e-06 2024-09-16T20:49:05.830 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.830 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.830 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:05.831 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871872+0000", 2024-09-16T20:49:05.831 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3799999999999999e-06 2024-09-16T20:49:05.831 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.831 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.831 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:05.831 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920558+0000", 2024-09-16T20:49:05.831 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048685797000000003 2024-09-16T20:49:05.831 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.831 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.831 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:05.831 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920603+0000", 2024-09-16T20:49:05.831 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4629999999999998e-05 2024-09-16T20:49:05.831 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.831 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.831 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:05.832 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920653+0000", 2024-09-16T20:49:05.832 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0297999999999999e-05 2024-09-16T20:49:05.832 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:05.832 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:05.832 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:05.832 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.832 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.832 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6421 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3610624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:05.832 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871882+0000", 2024-09-16T20:49:05.832 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 45.939916150000002, 2024-09-16T20:49:05.832 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048913381999999998, 2024-09-16T20:49:05.832 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:05.832 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:05.832 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:05.833 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:05.833 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:05.833 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:05.833 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6421 2024-09-16T20:49:05.833 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.833 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:05.833 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.833 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:05.833 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871882+0000", 2024-09-16T20:49:05.833 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:05.833 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.833 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.833 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:05.833 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871882+0000", 2024-09-16T20:49:05.834 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:05.834 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.834 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.834 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:05.834 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.873257+0000", 2024-09-16T20:49:05.834 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001374789 2024-09-16T20:49:05.834 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.834 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.834 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:05.834 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.873261+0000", 2024-09-16T20:49:05.834 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5589999999999999e-06 2024-09-16T20:49:05.834 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.834 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.835 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:05.835 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.873275+0000", 2024-09-16T20:49:05.835 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3675999999999999e-05 2024-09-16T20:49:05.835 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.835 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.835 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:05.835 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.873281+0000", 2024-09-16T20:49:05.835 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6930000000000004e-06 2024-09-16T20:49:05.836 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.836 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.836 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:05.836 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920684+0000", 2024-09-16T20:49:05.836 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047402913999999997 2024-09-16T20:49:05.836 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.836 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.836 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:05.836 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920737+0000", 2024-09-16T20:49:05.836 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3600000000000002e-05 2024-09-16T20:49:05.836 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.836 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.836 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:05.837 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920795+0000", 2024-09-16T20:49:05.837 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8150999999999999e-05 2024-09-16T20:49:05.837 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:05.837 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:05.837 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:05.837 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.837 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.837 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:7491 3.1e 3:7c48f6ae:::rbd_object_map.110e891a9b2b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:05.837 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.925865+0000", 2024-09-16T20:49:05.837 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 45.885933006000002, 2024-09-16T20:49:05.837 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048897293000000001, 2024-09-16T20:49:05.837 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:05.837 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:05.837 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:05.838 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:05.838 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:05.838 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:05.838 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 7491 2024-09-16T20:49:05.838 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.838 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:05.838 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.838 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:05.838 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925865+0000", 2024-09-16T20:49:05.838 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:05.838 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.838 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.838 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:05.838 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925865+0000", 2024-09-16T20:49:05.838 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:05.839 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.839 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.839 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:05.839 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925866+0000", 2024-09-16T20:49:05.839 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.39e-07 2024-09-16T20:49:05.839 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.839 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.839 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:05.839 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925868+0000", 2024-09-16T20:49:05.839 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.508e-06 2024-09-16T20:49:05.839 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.839 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.839 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:05.839 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925874+0000", 2024-09-16T20:49:05.840 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7919999999999998e-06 2024-09-16T20:49:05.840 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.840 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.840 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:05.840 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925879+0000", 2024-09-16T20:49:05.840 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8579999999999999e-06 2024-09-16T20:49:05.840 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.840 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.840 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:05.840 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.970875+0000", 2024-09-16T20:49:05.840 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044996412 2024-09-16T20:49:05.840 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.840 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.840 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:05.841 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.970970+0000", 2024-09-16T20:49:05.841 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4732e-05 2024-09-16T20:49:05.841 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.841 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.841 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-16T20:49:05.841 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.971275+0000", 2024-09-16T20:49:05.841 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00030464199999999999 2024-09-16T20:49:05.841 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.841 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.841 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:05.841 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.974598+0000", 2024-09-16T20:49:05.841 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0033238159999999998 2024-09-16T20:49:05.841 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.841 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.842 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:49:05.842 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.974666+0000", 2024-09-16T20:49:05.842 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7617e-05 2024-09-16T20:49:05.842 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.842 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.842 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:49:05.842 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.974718+0000", 2024-09-16T20:49:05.842 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2259000000000002e-05 2024-09-16T20:49:05.842 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.842 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.842 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:05.842 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.974762+0000", 2024-09-16T20:49:05.842 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4218000000000003e-05 2024-09-16T20:49:05.842 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:05.843 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:05.843 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:05.843 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.843 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.843 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4756 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1513472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:05.843 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689118+0000", 2024-09-16T20:49:05.843 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 43.122680586999998, 2024-09-16T20:49:05.843 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048535146000000001, 2024-09-16T20:49:05.843 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:05.843 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:05.843 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:05.843 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:05.843 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:49:05.843 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:05.844 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4756 2024-09-16T20:49:05.844 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.844 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:05.844 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.844 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:05.844 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689118+0000", 2024-09-16T20:49:05.844 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:05.844 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.844 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.844 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:05.844 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689118+0000", 2024-09-16T20:49:05.844 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:05.844 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.844 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.845 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:05.845 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689118+0000", 2024-09-16T20:49:05.845 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7099999999999997e-07 2024-09-16T20:49:05.845 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.845 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.845 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:05.845 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689120+0000", 2024-09-16T20:49:05.845 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.787e-06 2024-09-16T20:49:05.845 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.845 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.845 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:05.845 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689126+0000", 2024-09-16T20:49:05.845 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2720000000000003e-06 2024-09-16T20:49:05.845 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.846 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.846 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:05.846 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689134+0000", 2024-09-16T20:49:05.846 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2190000000000004e-06 2024-09-16T20:49:05.846 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.846 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.846 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:05.846 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737553+0000", 2024-09-16T20:49:05.846 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048418993 2024-09-16T20:49:05.846 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.846 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.846 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:05.846 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737598+0000", 2024-09-16T20:49:05.846 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4743999999999997e-05 2024-09-16T20:49:05.847 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.847 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.847 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:05.847 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737653+0000", 2024-09-16T20:49:05.847 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4759999999999997e-05 2024-09-16T20:49:05.847 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:05.847 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:05.847 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:05.847 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.847 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.847 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4758 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1521664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:05.847 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689141+0000", 2024-09-16T20:49:05.847 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 43.122657607000001, 2024-09-16T20:49:05.847 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048640741000000001, 2024-09-16T20:49:05.848 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:05.848 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:05.848 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:05.848 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:05.848 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:49:05.848 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:05.848 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4758 2024-09-16T20:49:05.848 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.848 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:05.848 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.848 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:05.848 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689141+0000", 2024-09-16T20:49:05.848 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:05.848 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.849 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.849 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:05.849 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689141+0000", 2024-09-16T20:49:05.849 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:05.849 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.849 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.849 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:05.849 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689141+0000", 2024-09-16T20:49:05.849 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8799999999999998e-07 2024-09-16T20:49:05.849 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.849 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.849 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:05.849 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689143+0000", 2024-09-16T20:49:05.849 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.945e-06 2024-09-16T20:49:05.849 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.850 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.850 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:05.850 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689149+0000", 2024-09-16T20:49:05.850 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9800000000000003e-06 2024-09-16T20:49:05.850 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.850 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.850 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:05.850 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689155+0000", 2024-09-16T20:49:05.850 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7509999999999997e-06 2024-09-16T20:49:05.850 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.850 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.850 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:05.850 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737681+0000", 2024-09-16T20:49:05.850 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048526825000000003 2024-09-16T20:49:05.851 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.851 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.851 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:05.851 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737729+0000", 2024-09-16T20:49:05.851 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7219999999999999e-05 2024-09-16T20:49:05.851 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.851 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.851 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:05.851 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737781+0000", 2024-09-16T20:49:05.851 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2632e-05 2024-09-16T20:49:05.851 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:05.851 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:05.851 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:05.851 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.851 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.852 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4760 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1529856~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:05.852 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689161+0000", 2024-09-16T20:49:05.852 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 43.122637259999998, 2024-09-16T20:49:05.852 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048743981999999998, 2024-09-16T20:49:05.852 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:05.852 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:05.852 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:05.852 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:05.852 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:49:05.852 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:05.852 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4760 2024-09-16T20:49:05.852 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.852 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:05.852 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.853 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:05.853 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689161+0000", 2024-09-16T20:49:05.853 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:05.853 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.853 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.853 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:05.853 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689161+0000", 2024-09-16T20:49:05.853 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:05.853 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.853 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.853 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:05.853 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689161+0000", 2024-09-16T20:49:05.853 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5599999999999995e-07 2024-09-16T20:49:05.854 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.854 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.854 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:05.854 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689163+0000", 2024-09-16T20:49:05.854 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.818e-06 2024-09-16T20:49:05.854 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.854 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.854 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:05.854 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689169+0000", 2024-09-16T20:49:05.854 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0809999999999999e-06 2024-09-16T20:49:05.854 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.854 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.854 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:05.854 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689174+0000", 2024-09-16T20:49:05.855 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.938e-06 2024-09-16T20:49:05.855 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.855 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.855 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:05.855 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737807+0000", 2024-09-16T20:49:05.855 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048633124 2024-09-16T20:49:05.855 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.855 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.855 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:05.855 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737851+0000", 2024-09-16T20:49:05.855 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3764000000000003e-05 2024-09-16T20:49:05.855 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.855 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.855 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:05.855 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737905+0000", 2024-09-16T20:49:05.856 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3701e-05 2024-09-16T20:49:05.856 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:05.856 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:05.856 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:05.856 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.856 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.856 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4762 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1538048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:05.856 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689182+0000", 2024-09-16T20:49:05.856 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 43.122616037999997, 2024-09-16T20:49:05.856 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048850275999999998, 2024-09-16T20:49:05.856 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:05.856 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:05.856 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:05.857 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:05.857 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:49:05.857 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:05.857 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4762 2024-09-16T20:49:05.857 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.857 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:05.857 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.857 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:05.857 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689182+0000", 2024-09-16T20:49:05.857 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:05.857 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.857 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.857 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:05.857 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689182+0000", 2024-09-16T20:49:05.858 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:05.858 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.858 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.858 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:05.858 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689182+0000", 2024-09-16T20:49:05.858 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-16T20:49:05.858 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.858 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.858 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:05.858 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689184+0000", 2024-09-16T20:49:05.858 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8190000000000001e-06 2024-09-16T20:49:05.858 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.858 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.858 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:05.858 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689191+0000", 2024-09-16T20:49:05.859 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2530000000000001e-06 2024-09-16T20:49:05.859 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.859 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.859 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:05.859 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689195+0000", 2024-09-16T20:49:05.859 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9559999999999996e-06 2024-09-16T20:49:05.859 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.859 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.859 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:05.859 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737933+0000", 2024-09-16T20:49:05.859 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048737919999999997 2024-09-16T20:49:05.859 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.859 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.859 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:05.860 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737982+0000", 2024-09-16T20:49:05.860 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8089e-05 2024-09-16T20:49:05.860 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.860 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.860 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:05.860 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738032+0000", 2024-09-16T20:49:05.860 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0868999999999997e-05 2024-09-16T20:49:05.860 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:05.860 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:05.860 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:05.860 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.860 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.860 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4764 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1546240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:05.860 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689201+0000", 2024-09-16T20:49:05.861 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 43.122596680999997, 2024-09-16T20:49:05.861 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048957550000000002, 2024-09-16T20:49:05.861 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:05.861 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:05.861 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:05.861 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:05.861 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:49:05.861 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:05.861 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4764 2024-09-16T20:49:05.861 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.861 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:05.861 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.861 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:05.861 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689201+0000", 2024-09-16T20:49:05.861 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:05.862 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.862 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.862 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:05.862 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689201+0000", 2024-09-16T20:49:05.862 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:05.862 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.862 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.862 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:05.862 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689202+0000", 2024-09-16T20:49:05.862 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4200000000000002e-07 2024-09-16T20:49:05.862 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.862 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.862 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:05.862 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689204+0000", 2024-09-16T20:49:05.863 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9750000000000001e-06 2024-09-16T20:49:05.863 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.863 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.863 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:05.863 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689210+0000", 2024-09-16T20:49:05.863 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1e-06 2024-09-16T20:49:05.863 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.863 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.863 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:05.863 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689215+0000", 2024-09-16T20:49:05.863 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0180000000000001e-06 2024-09-16T20:49:05.863 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.863 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.863 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:05.863 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738058+0000", 2024-09-16T20:49:05.864 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048843573000000001 2024-09-16T20:49:05.864 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.864 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.864 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:05.864 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738102+0000", 2024-09-16T20:49:05.864 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3256000000000001e-05 2024-09-16T20:49:05.864 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.864 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.864 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:05.864 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738159+0000", 2024-09-16T20:49:05.864 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7286e-05 2024-09-16T20:49:05.864 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:05.864 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:05.864 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:05.865 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.865 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.865 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4766 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1554432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:05.865 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689221+0000", 2024-09-16T20:49:05.865 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 43.122576715000001, 2024-09-16T20:49:05.865 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049064996, 2024-09-16T20:49:05.865 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:05.865 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:05.865 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:05.865 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:05.865 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:49:05.865 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:05.865 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4766 2024-09-16T20:49:05.865 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.865 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:05.866 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.866 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:05.866 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689221+0000", 2024-09-16T20:49:05.866 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:05.866 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.866 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.866 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:05.866 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689221+0000", 2024-09-16T20:49:05.866 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:05.866 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.866 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.866 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:05.866 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689222+0000", 2024-09-16T20:49:05.866 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.34e-07 2024-09-16T20:49:05.867 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.867 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.867 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:05.867 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689224+0000", 2024-09-16T20:49:05.867 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.846e-06 2024-09-16T20:49:05.867 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.867 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.867 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:05.867 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689247+0000", 2024-09-16T20:49:05.867 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3448e-05 2024-09-16T20:49:05.867 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.867 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.867 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:05.867 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689254+0000", 2024-09-16T20:49:05.867 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6969999999999996e-06 2024-09-16T20:49:05.868 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.868 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.868 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:05.868 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738188+0000", 2024-09-16T20:49:05.868 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048934520000000002 2024-09-16T20:49:05.868 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.868 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.868 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:05.868 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738234+0000", 2024-09-16T20:49:05.868 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5695999999999998e-05 2024-09-16T20:49:05.868 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.868 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.868 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:05.868 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738286+0000", 2024-09-16T20:49:05.869 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2355000000000003e-05 2024-09-16T20:49:05.869 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:05.869 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:05.869 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:05.869 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.869 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.869 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4768 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1562624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:05.869 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689262+0000", 2024-09-16T20:49:05.869 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 43.122535761000002, 2024-09-16T20:49:05.869 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049157157999999999, 2024-09-16T20:49:05.869 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:05.869 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:05.869 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:05.869 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:05.870 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:49:05.870 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:05.870 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4768 2024-09-16T20:49:05.870 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.870 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:05.870 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.870 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:05.870 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689262+0000", 2024-09-16T20:49:05.870 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:05.870 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.870 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.870 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:05.870 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689262+0000", 2024-09-16T20:49:05.870 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:05.871 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.871 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.871 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:05.871 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689263+0000", 2024-09-16T20:49:05.871 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6199999999999998e-07 2024-09-16T20:49:05.871 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.871 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.871 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:05.871 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689270+0000", 2024-09-16T20:49:05.871 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.798e-06 2024-09-16T20:49:05.871 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.871 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.871 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:05.871 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689277+0000", 2024-09-16T20:49:05.872 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0890000000000002e-06 2024-09-16T20:49:05.872 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.872 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.872 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:05.872 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689282+0000", 2024-09-16T20:49:05.872 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0470000000000001e-06 2024-09-16T20:49:05.872 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.872 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.872 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:05.872 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738319+0000", 2024-09-16T20:49:05.872 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049037139 2024-09-16T20:49:05.872 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.872 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.872 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:05.873 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738365+0000", 2024-09-16T20:49:05.873 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6356999999999997e-05 2024-09-16T20:49:05.873 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.873 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.873 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:05.873 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738420+0000", 2024-09-16T20:49:05.873 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4265999999999998e-05 2024-09-16T20:49:05.873 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:05.873 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:05.873 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:05.873 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.873 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.873 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4770 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1570816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:05.873 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689289+0000", 2024-09-16T20:49:05.873 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 43.122509250999997, 2024-09-16T20:49:05.874 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049253705000000002, 2024-09-16T20:49:05.874 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:05.874 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:05.874 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:05.874 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:05.874 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:49:05.874 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:05.874 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4770 2024-09-16T20:49:05.874 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.874 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:05.874 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.875 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:05.875 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689289+0000", 2024-09-16T20:49:05.875 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:05.875 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.875 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.875 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:05.875 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689289+0000", 2024-09-16T20:49:05.875 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:05.875 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.875 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.875 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:05.875 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689289+0000", 2024-09-16T20:49:05.875 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0999999999999999e-07 2024-09-16T20:49:05.876 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.876 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.876 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:05.876 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689291+0000", 2024-09-16T20:49:05.876 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0030000000000001e-06 2024-09-16T20:49:05.876 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.876 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.876 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:05.876 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689297+0000", 2024-09-16T20:49:05.876 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9479999999999996e-06 2024-09-16T20:49:05.876 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.876 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.876 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:05.876 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689302+0000", 2024-09-16T20:49:05.877 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7389999999999999e-06 2024-09-16T20:49:05.877 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.877 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.877 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:05.877 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738442+0000", 2024-09-16T20:49:05.877 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049140001000000003 2024-09-16T20:49:05.877 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.877 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.877 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:05.877 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738490+0000", 2024-09-16T20:49:05.877 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7787000000000002e-05 2024-09-16T20:49:05.877 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.877 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.877 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:05.878 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738543+0000", 2024-09-16T20:49:05.878 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2816999999999999e-05 2024-09-16T20:49:05.878 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:05.878 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:05.878 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:05.878 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.878 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.878 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:3088 3.1e 3:79e5ec01:::rbd_header.110e891a9b2b:head [watch watch cookie 140213437171488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:49:05.878 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:49:05.878 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 41.647953233999999, 2024-09-16T20:49:05.878 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.11895186200000001, 2024-09-16T20:49:05.878 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:05.878 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:05.878 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:05.878 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:05.879 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:05.879 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:05.879 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3088 2024-09-16T20:49:05.879 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.879 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:05.879 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.879 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:05.879 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:49:05.879 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:05.879 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.879 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.879 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:05.880 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:49:05.880 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:05.880 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.880 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.880 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:05.880 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163846+0000", 2024-09-16T20:49:05.880 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6799999999999999e-07 2024-09-16T20:49:05.880 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.880 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.880 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:05.880 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163850+0000", 2024-09-16T20:49:05.880 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.929e-06 2024-09-16T20:49:05.880 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.880 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.881 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:05.881 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163862+0000", 2024-09-16T20:49:05.881 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1972e-05 2024-09-16T20:49:05.881 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.881 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.881 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:05.881 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163867+0000", 2024-09-16T20:49:05.881 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.451e-06 2024-09-16T20:49:05.881 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.881 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.881 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:05.881 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163912+0000", 2024-09-16T20:49:05.881 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4456000000000003e-05 2024-09-16T20:49:05.881 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.882 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.882 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:05.882 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163962+0000", 2024-09-16T20:49:05.882 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0578000000000003e-05 2024-09-16T20:49:05.882 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.882 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.882 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-16T20:49:05.882 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164058+0000", 2024-09-16T20:49:05.882 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5934999999999995e-05 2024-09-16T20:49:05.882 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.882 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.882 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:49:05.882 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164570+0000", 2024-09-16T20:49:05.882 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00051241100000000005 2024-09-16T20:49:05.882 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.883 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.883 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:05.883 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282629+0000", 2024-09-16T20:49:05.883 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.118058986 2024-09-16T20:49:05.883 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.883 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.883 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:49:05.883 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282725+0000", 2024-09-16T20:49:05.883 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5063999999999996e-05 2024-09-16T20:49:05.884 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.884 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.884 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:05.884 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282797+0000", 2024-09-16T20:49:05.884 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2212000000000006e-05 2024-09-16T20:49:05.884 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:05.884 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:05.884 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:05.884 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.884 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.884 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:8039 3.1e 3.ac44ff9e (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:49:05.884 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:49:05.885 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 16.104125884999998, 2024-09-16T20:49:05.885 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0053310449999999, 2024-09-16T20:49:05.885 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:05.885 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:05.885 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:49:05.885 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:05.885 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:05.885 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:05.885 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8039 2024-09-16T20:49:05.885 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.886 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:05.886 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.886 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:05.886 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:49:05.886 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:05.886 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.886 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.886 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:05.886 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:49:05.886 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:05.886 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.886 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.886 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:05.887 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707674+0000", 2024-09-16T20:49:05.887 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6530000000000001e-06 2024-09-16T20:49:05.887 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.887 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.887 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:05.887 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707681+0000", 2024-09-16T20:49:05.887 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9639999999999997e-06 2024-09-16T20:49:05.887 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.887 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.887 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:05.887 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707700+0000", 2024-09-16T20:49:05.888 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8674000000000001e-05 2024-09-16T20:49:05.888 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.888 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.888 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:05.888 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707708+0000", 2024-09-16T20:49:05.888 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6810000000000004e-06 2024-09-16T20:49:05.888 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.888 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.888 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:05.889 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711293+0000", 2024-09-16T20:49:05.889 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035848189999999999 2024-09-16T20:49:05.889 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.889 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.889 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:49:05.889 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711301+0000", 2024-09-16T20:49:05.889 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4730000000000003e-06 2024-09-16T20:49:05.889 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.889 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.889 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:05.890 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.712990+0000", 2024-09-16T20:49:05.890 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.001689348 2024-09-16T20:49:05.890 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.890 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.890 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:05.890 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713003+0000", 2024-09-16T20:49:05.890 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3433e-05 2024-09-16T20:49:05.890 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:05.890 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:05.891 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:05.891 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.891 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.891 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:8040 3.1e 3.ac44ff9e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:49:05.891 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:49:05.891 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 16.10406549, 2024-09-16T20:49:05.891 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0053410780000001, 2024-09-16T20:49:05.891 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:05.891 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:05.891 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:49:05.891 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:05.891 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:05.891 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:05.892 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8040 2024-09-16T20:49:05.892 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.892 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:05.892 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.892 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:05.892 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:49:05.892 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:05.892 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.892 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.892 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:05.892 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:49:05.892 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:05.892 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.893 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.893 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:05.893 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:49:05.893 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8100000000000003e-07 2024-09-16T20:49:05.893 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.893 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.893 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:05.893 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707737+0000", 2024-09-16T20:49:05.893 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5020000000000001e-06 2024-09-16T20:49:05.893 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.893 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.893 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:05.893 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707745+0000", 2024-09-16T20:49:05.893 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2169999999999994e-06 2024-09-16T20:49:05.894 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.894 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.894 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:05.894 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707751+0000", 2024-09-16T20:49:05.894 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.263e-06 2024-09-16T20:49:05.894 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.894 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.894 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:05.894 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711336+0000", 2024-09-16T20:49:05.894 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035848360000000001 2024-09-16T20:49:05.894 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.894 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.894 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:49:05.894 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711340+0000", 2024-09-16T20:49:05.895 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1309999999999999e-06 2024-09-16T20:49:05.895 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.895 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.895 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:05.895 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713060+0000", 2024-09-16T20:49:05.895 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0017195059999999 2024-09-16T20:49:05.895 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.895 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.895 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:05.895 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713074+0000", 2024-09-16T20:49:05.895 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4042e-05 2024-09-16T20:49:05.895 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:05.895 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:05.895 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:05.895 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.896 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.896 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4927.0:4487 3.1e 3.ac44ff9e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:49:05.896 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:49:05.896 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 16.103545287999999, 2024-09-16T20:49:05.896 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0048840880000001, 2024-09-16T20:49:05.896 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:05.896 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:05.896 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:49:05.896 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:05.896 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4927", 2024-09-16T20:49:05.896 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:05.896 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4487 2024-09-16T20:49:05.896 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.897 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:05.897 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.897 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:05.897 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:49:05.897 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:05.897 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.897 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.897 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:05.897 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:49:05.897 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:05.897 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.897 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.897 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:05.897 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708254+0000", 2024-09-16T20:49:05.897 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1800000000000005e-07 2024-09-16T20:49:05.898 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.898 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.898 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:05.898 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708258+0000", 2024-09-16T20:49:05.898 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8689999999999997e-06 2024-09-16T20:49:05.898 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.898 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.898 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:05.898 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708267+0000", 2024-09-16T20:49:05.898 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9590000000000007e-06 2024-09-16T20:49:05.898 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.898 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.898 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:05.898 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708274+0000", 2024-09-16T20:49:05.899 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9970000000000001e-06 2024-09-16T20:49:05.899 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.899 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.899 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:05.899 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711366+0000", 2024-09-16T20:49:05.899 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030919530000000001 2024-09-16T20:49:05.899 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.899 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.899 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:49:05.899 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711372+0000", 2024-09-16T20:49:05.899 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9410000000000004e-06 2024-09-16T20:49:05.899 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.899 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.899 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:05.900 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713126+0000", 2024-09-16T20:49:05.900 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0017537249999999 2024-09-16T20:49:05.900 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.900 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.900 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:05.900 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713137+0000", 2024-09-16T20:49:05.900 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0825999999999999e-05 2024-09-16T20:49:05.900 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:05.900 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:05.900 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:05.900 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.900 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.900 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5089.0:11703 3.1b 3:dcd4684e:::rbd_data.110e747f0779.000000000000001b:head [set-alloc-hint object_size 262144 write_size 262144,write 139264~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:49:05.901 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:49:05.901 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 1.673809514, 2024-09-16T20:49:05.901 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.084231385000000006, 2024-09-16T20:49:05.901 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:05.901 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:05.901 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:05.901 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:05.901 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5089", 2024-09-16T20:49:05.901 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:49:05.901 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 11703 2024-09-16T20:49:05.901 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.901 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:05.901 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.901 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:05.901 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:49:05.902 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:05.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.902 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.902 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:05.902 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:49:05.902 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:05.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.902 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.902 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:05.902 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:49:05.902 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2600000000000001e-07 2024-09-16T20:49:05.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.902 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.903 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:05.903 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137998+0000", 2024-09-16T20:49:05.903 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9549999999999998e-06 2024-09-16T20:49:05.903 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.903 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.903 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:05.903 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138008+0000", 2024-09-16T20:49:05.903 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3789999999999995e-06 2024-09-16T20:49:05.903 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.903 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.903 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:05.903 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138013+0000", 2024-09-16T20:49:05.903 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2209999999999996e-06 2024-09-16T20:49:05.903 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.903 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.904 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:05.904 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138054+0000", 2024-09-16T20:49:05.904 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1022999999999997e-05 2024-09-16T20:49:05.904 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.904 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.904 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:05.904 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138124+0000", 2024-09-16T20:49:05.904 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0234999999999994e-05 2024-09-16T20:49:05.904 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.904 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.904 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:05.904 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138197+0000", 2024-09-16T20:49:05.904 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2451000000000001e-05 2024-09-16T20:49:05.904 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.905 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.905 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:05.905 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222010+0000", 2024-09-16T20:49:05.905 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.083813433000000007 2024-09-16T20:49:05.905 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.905 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.905 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:49:05.905 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222163+0000", 2024-09-16T20:49:05.905 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015274599999999999 2024-09-16T20:49:05.905 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.905 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.905 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:49:05.905 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222197+0000", 2024-09-16T20:49:05.906 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3683000000000003e-05 2024-09-16T20:49:05.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.906 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.906 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:05.906 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222220+0000", 2024-09-16T20:49:05.906 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3433000000000001e-05 2024-09-16T20:49:05.906 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:05.906 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:05.906 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:05.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.906 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.906 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.5089.0:11702 3.15 e73/64 3:acf33c8b:::rbd_data.110e747f0779.000000000000001a:head v 73'132, mlcod=73'130)", 2024-09-16T20:49:05.906 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:49:05.906 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 1.6735421850000001, 2024-09-16T20:49:05.907 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.084079763000000002, 2024-09-16T20:49:05.907 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:05.907 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:05.907 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:05.907 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:05.907 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.907 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:05.907 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:49:05.907 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:05.907 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.907 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.907 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:05.907 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:49:05.907 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:05.908 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.908 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.908 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:05.908 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:49:05.908 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:49:05.908 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.908 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.908 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:05.908 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138263+0000", 2024-09-16T20:49:05.908 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1e-06 2024-09-16T20:49:05.908 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.908 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.908 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:05.908 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221589+0000", 2024-09-16T20:49:05.909 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.083326002999999996 2024-09-16T20:49:05.909 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.909 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.909 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:05.909 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221614+0000", 2024-09-16T20:49:05.909 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5593999999999999e-05 2024-09-16T20:49:05.909 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.909 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.909 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:05.909 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221714+0000", 2024-09-16T20:49:05.909 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9889000000000005e-05 2024-09-16T20:49:05.909 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.909 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.909 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:05.910 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221727+0000", 2024-09-16T20:49:05.910 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3125999999999999e-05 2024-09-16T20:49:05.910 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.910 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.910 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:49:05.910 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222294+0000", 2024-09-16T20:49:05.910 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00056712199999999998 2024-09-16T20:49:05.910 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:05.910 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:05.910 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:05.910 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222336+0000", 2024-09-16T20:49:05.910 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1470000000000001e-05 2024-09-16T20:49:05.910 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:05.911 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:05.911 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:05.911 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:05.911 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:05.911 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:05.912 INFO:teuthology.orchestra.run.smithi139.stdout:leaving image image_client.0-clone20 intact 2024-09-16T20:49:05.913 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:49:05.913 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:05.911+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:06.013 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:06.012+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:06.018 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T20:49:06.023 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:06.023 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:49:06.023 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:49:06.024 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:06.032 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:49:06.114 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:06.113+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:06.153 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:49:06.153 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:49:06.171 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:49:06.171 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:49:06.181 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:06.182 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:49:06.182 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:49:06.182 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:49:06.182 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:06.188 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:49:06.188 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:49:06.190 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:49:06.199 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #19, image image_client.0-clone19 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent20 2024-09-16T20:49:06.206 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:49:06.206 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:49:06.215 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:06.214+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:06.226 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:49:06.227 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:49:06.249 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:49:06.249 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:49:06.317 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:06.315+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:06.318 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:06.318 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:06.328 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:49:06.417 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:06.416+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:06.465 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:06.465 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:06.475 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:49:06.519 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:06.519+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:06.612 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:06.613 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:06.619 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:06.618+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:06.623 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:49:06.719 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:06.718+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:06.772 INFO:teuthology.orchestra.run.smithi029.stderr:reweighted osd.3 to 1 (10000) 2024-09-16T20:49:06.786 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-16T20:49:06.787 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T20:49:06.793 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:06.793 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:06.802 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:49:06.820 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:06.819+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:06.896 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T20:49:06.927 INFO:teuthology.orchestra.run.smithi139.stdout:2708 write 0x1fa2800 thru 0x1fa3007 (0x808 bytes) 2024-09-16T20:49:06.928 INFO:teuthology.orchestra.run.smithi139.stdout:2710 write 0xa2d200 thru 0xa392c6 (0xc0c7 bytes) 2024-09-16T20:49:06.928 INFO:teuthology.orchestra.run.smithi139.stdout:2711 punch from 0x7945000 to 0x7949c00, (0x4c00 bytes) 2024-09-16T20:49:06.947 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:06.940+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:06.948 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:06.948 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:06.963 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:49:07.051 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:07.046+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:07.123 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:07.121+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:07.189 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:07.189 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:07.201 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:49:07.242 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:07.241+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:07.286 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:49:07.286 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-09-16T20:49:07.299 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":110,"stamp":"2024-09-16T20:49:06.772311+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":933303167,"num_objects":748,"num_object_clones":216,"num_object_copies":1496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":748,"num_whiteouts":156,"num_read":128786,"num_read_kb":558483,"num_write":9335,"num_write_kb":152597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1009,"num_bytes_recovered":31785521,"num_keys_recovered":234,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7547,"ondisk_log_size":7547,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":366560,"kb_used_data":192572,"kb_used_omap":7,"kb_used_meta":173880,"kb_avail":471492640,"statfs":{"total":483183820800,"available":482808463360,"internally_reserved":0,"allocated":197193728,"data_stored":219449762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7972,"internal_metadata":178053340},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18049157,"num_objects":111,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":111,"num_whiteouts":0,"num_read":22693,"num_read_kb":126216,"num_write":1133,"num_write_kb":8980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.556479"},"pg_stats":[{"pgid":"3.1f","version":"74'158","reported_seq":2419,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:06.217231+0000","last_change":"2024-09-16T20:48:53.732229+0000","last_active":"2024-09-16T20:49:06.217231+0000","last_peered":"2024-09-16T20:49:06.217231+0000","last_clean":"2024-09-16T20:49:06.217231+0000","last_became_active":"2024-09-16T20:48:53.731525+0000","last_became_peered":"2024-09-16T20:48:53.731525+0000","last_unstale":"2024-09-16T20:49:06.217231+0000","last_undegraded":"2024-09-16T20:49:06.217231+0000","last_fullsized":"2024-09-16T20:49:06.217231+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:21:53.398218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071634499999999996,"stat_sum":{"num_bytes":27213882,"num_objects":21,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":2,"num_read":2254,"num_read_kb":6692,"num_write":143,"num_write_kb":2005,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":921716,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1e","version":"73'448","reported_seq":11889,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:05.768222+0000","last_change":"2024-09-16T20:48:53.732932+0000","last_active":"2024-09-16T20:49:05.768222+0000","last_peered":"2024-09-16T20:49:05.768222+0000","last_clean":"2024-09-16T20:49:05.768222+0000","last_became_active":"2024-09-16T20:48:53.732081+0000","last_became_peered":"2024-09-16T20:48:53.732081+0000","last_unstale":"2024-09-16T20:49:05.768222+0000","last_undegraded":"2024-09-16T20:49:05.768222+0000","last_fullsized":"2024-09-16T20:49:05.768222+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:17:52.739495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084864999999999995,"stat_sum":{"num_bytes":31008913,"num_objects":26,"num_object_clones":6,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":3,"num_read":13121,"num_read_kb":35383,"num_write":576,"num_write_kb":4766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":1415412,"num_keys_recovered":59,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"74'375","reported_seq":4732,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:06.215869+0000","last_change":"2024-09-16T20:48:09.564507+0000","last_active":"2024-09-16T20:49:06.215869+0000","last_peered":"2024-09-16T20:49:06.215869+0000","last_clean":"2024-09-16T20:49:06.215869+0000","last_became_active":"2024-09-16T20:48:07.568025+0000","last_became_peered":"2024-09-16T20:48:07.568025+0000","last_unstale":"2024-09-16T20:49:06.215869+0000","last_undegraded":"2024-09-16T20:49:06.215869+0000","last_fullsized":"2024-09-16T20:49:06.215869+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:01:22.178335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068985699999999999,"stat_sum":{"num_bytes":52607554,"num_objects":30,"num_object_clones":9,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":5,"num_read":5750,"num_read_kb":56071,"num_write":493,"num_write_kb":7718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'374","reported_seq":1409,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:06.220563+0000","last_change":"2024-09-16T20:48:09.565661+0000","last_active":"2024-09-16T20:49:06.220563+0000","last_peered":"2024-09-16T20:49:06.220563+0000","last_clean":"2024-09-16T20:49:06.220563+0000","last_became_active":"2024-09-16T20:48:07.565241+0000","last_became_peered":"2024-09-16T20:48:07.565241+0000","last_unstale":"2024-09-16T20:49:06.220563+0000","last_undegraded":"2024-09-16T20:49:06.220563+0000","last_fullsized":"2024-09-16T20:49:06.220563+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:30:21.676377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001045589,"stat_sum":{"num_bytes":7610933,"num_objects":24,"num_object_clones":5,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":4,"num_read":2512,"num_read_kb":8045,"num_write":622,"num_write_kb":3689,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"75'498","reported_seq":2675,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:05.776564+0000","last_change":"2024-09-16T20:48:54.735381+0000","last_active":"2024-09-16T20:49:05.776564+0000","last_peered":"2024-09-16T20:49:05.776564+0000","last_clean":"2024-09-16T20:49:05.776564+0000","last_became_active":"2024-09-16T20:48:54.734278+0000","last_became_peered":"2024-09-16T20:48:54.734278+0000","last_unstale":"2024-09-16T20:49:05.776564+0000","last_undegraded":"2024-09-16T20:49:05.776564+0000","last_fullsized":"2024-09-16T20:49:05.776564+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":498,"log_dups_size":0,"ondisk_log_size":498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:35:54.163829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057432399999999997,"stat_sum":{"num_bytes":21458744,"num_objects":32,"num_object_clones":8,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":7,"num_read":3881,"num_read_kb":17904,"num_write":766,"num_write_kb":9062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":1963751,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"74'566","reported_seq":5682,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:06.216200+0000","last_change":"2024-09-16T20:48:47.560036+0000","last_active":"2024-09-16T20:49:06.216200+0000","last_peered":"2024-09-16T20:49:06.216200+0000","last_clean":"2024-09-16T20:49:06.216200+0000","last_became_active":"2024-09-16T20:48:47.558880+0000","last_became_peered":"2024-09-16T20:48:47.558880+0000","last_unstale":"2024-09-16T20:49:06.216200+0000","last_undegraded":"2024-09-16T20:49:06.216200+0000","last_fullsized":"2024-09-16T20:49:06.216200+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":566,"log_dups_size":0,"ondisk_log_size":566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:15:03.077216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002037849,"stat_sum":{"num_bytes":5546288,"num_objects":26,"num_object_clones":6,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":5,"num_read":9674,"num_read_kb":10969,"num_write":708,"num_write_kb":2125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":333388,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"72'116","reported_seq":3743,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T20:49:03.186311+0000","last_change":"2024-09-16T20:48:53.732127+0000","last_active":"2024-09-16T20:49:03.186311+0000","last_peered":"2024-09-16T20:49:03.186311+0000","last_clean":"2024-09-16T20:49:03.186311+0000","last_became_active":"2024-09-16T20:48:53.731485+0000","last_became_peered":"2024-09-16T20:48:53.731485+0000","last_unstale":"2024-09-16T20:49:03.186311+0000","last_undegraded":"2024-09-16T20:49:03.186311+0000","last_fullsized":"2024-09-16T20:49:03.186311+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:30:41.446995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057190699999999997,"stat_sum":{"num_bytes":25220640,"num_objects":25,"num_object_clones":10,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":4,"num_read":3419,"num_read_kb":16323,"num_write":112,"num_write_kb":3634,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":5091328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"75'175","reported_seq":2976,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:06.220280+0000","last_change":"2024-09-16T20:48:53.731382+0000","last_active":"2024-09-16T20:49:06.220280+0000","last_peered":"2024-09-16T20:49:06.220280+0000","last_clean":"2024-09-16T20:49:06.220280+0000","last_became_active":"2024-09-16T20:48:53.730714+0000","last_became_peered":"2024-09-16T20:48:53.730714+0000","last_unstale":"2024-09-16T20:49:06.220280+0000","last_undegraded":"2024-09-16T20:49:06.220280+0000","last_fullsized":"2024-09-16T20:49:06.220280+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:50:44.320055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011061790000000001,"stat_sum":{"num_bytes":6090256,"num_objects":16,"num_object_clones":2,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":1,"num_read":4019,"num_read_kb":12667,"num_write":141,"num_write_kb":4377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2412544,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"75'282","reported_seq":2233,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:05.768726+0000","last_change":"2024-09-16T20:48:47.564078+0000","last_active":"2024-09-16T20:49:05.768726+0000","last_peered":"2024-09-16T20:49:05.768726+0000","last_clean":"2024-09-16T20:49:05.768726+0000","last_became_active":"2024-09-16T20:48:47.563287+0000","last_became_peered":"2024-09-16T20:48:47.563287+0000","last_unstale":"2024-09-16T20:49:05.768726+0000","last_undegraded":"2024-09-16T20:49:05.768726+0000","last_fullsized":"2024-09-16T20:49:05.768726+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:38:22.963563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057915500000000003,"stat_sum":{"num_bytes":10004558,"num_objects":15,"num_object_clones":3,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":2,"num_read":3515,"num_read_kb":5573,"num_write":286,"num_write_kb":784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":34852,"num_keys_recovered":24,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"74'387","reported_seq":1946,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:06.219856+0000","last_change":"2024-09-16T20:48:53.731777+0000","last_active":"2024-09-16T20:49:06.219856+0000","last_peered":"2024-09-16T20:49:06.219856+0000","last_clean":"2024-09-16T20:49:06.219856+0000","last_became_active":"2024-09-16T20:48:53.730899+0000","last_became_peered":"2024-09-16T20:48:53.730899+0000","last_unstale":"2024-09-16T20:49:06.219856+0000","last_undegraded":"2024-09-16T20:49:06.219856+0000","last_fullsized":"2024-09-16T20:49:06.219856+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:17:10.090950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010075049999999999,"stat_sum":{"num_bytes":12332145,"num_objects":23,"num_object_clones":5,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":4,"num_read":2632,"num_read_kb":15665,"num_write":594,"num_write_kb":5717,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":6235436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"74'136","reported_seq":2069,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:05.768631+0000","last_change":"2024-09-16T20:48:53.734382+0000","last_active":"2024-09-16T20:49:05.768631+0000","last_peered":"2024-09-16T20:49:05.768631+0000","last_clean":"2024-09-16T20:49:05.768631+0000","last_became_active":"2024-09-16T20:48:53.733689+0000","last_became_peered":"2024-09-16T20:48:53.733689+0000","last_unstale":"2024-09-16T20:49:05.768631+0000","last_undegraded":"2024-09-16T20:49:05.768631+0000","last_fullsized":"2024-09-16T20:49:05.768631+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:53:34.168240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051100699999999995,"stat_sum":{"num_bytes":37896192,"num_objects":28,"num_object_clones":10,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":7,"num_read":1595,"num_read_kb":9559,"num_write":134,"num_write_kb":2945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":2749952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"75'150","reported_seq":5704,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:06.216179+0000","last_change":"2024-09-16T20:48:09.566005+0000","last_active":"2024-09-16T20:49:06.216179+0000","last_peered":"2024-09-16T20:49:06.216179+0000","last_clean":"2024-09-16T20:49:06.216179+0000","last_became_active":"2024-09-16T20:48:07.568236+0000","last_became_peered":"2024-09-16T20:48:07.568236+0000","last_unstale":"2024-09-16T20:49:06.216179+0000","last_undegraded":"2024-09-16T20:49:06.216179+0000","last_fullsized":"2024-09-16T20:49:06.216179+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:41:07.660360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001271749,"stat_sum":{"num_bytes":56578786,"num_objects":31,"num_object_clones":11,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":8,"num_read":5727,"num_read_kb":32286,"num_write":318,"num_write_kb":9911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":128,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:05.769724+0000","last_change":"2024-09-16T20:48:00.277670+0000","last_active":"2024-09-16T20:49:05.769724+0000","last_peered":"2024-09-16T20:49:05.769724+0000","last_clean":"2024-09-16T20:49:05.769724+0000","last_became_active":"2024-09-16T20:47:58.274489+0000","last_became_peered":"2024-09-16T20:47:58.274489+0000","last_unstale":"2024-09-16T20:49:05.769724+0000","last_undegraded":"2024-09-16T20:49:05.769724+0000","last_fullsized":"2024-09-16T20:49:05.769724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:27:12.633064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091339500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"68'91","reported_seq":1137,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T20:49:02.779431+0000","last_change":"2024-09-16T20:48:50.757942+0000","last_active":"2024-09-16T20:49:02.779431+0000","last_peered":"2024-09-16T20:49:02.779431+0000","last_clean":"2024-09-16T20:49:02.779431+0000","last_became_active":"2024-09-16T20:48:50.719161+0000","last_became_peered":"2024-09-16T20:48:50.719161+0000","last_unstale":"2024-09-16T20:49:02.779431+0000","last_undegraded":"2024-09-16T20:49:02.779431+0000","last_fullsized":"2024-09-16T20:49:02.779431+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:49:32.448065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067924999999999995,"stat_sum":{"num_bytes":5959717,"num_objects":11,"num_object_clones":3,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":2,"num_read":814,"num_read_kb":6959,"num_write":94,"num_write_kb":1612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":466506,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":124,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T20:49:02.779473+0000","last_change":"2024-09-16T20:48:00.277182+0000","last_active":"2024-09-16T20:49:02.779473+0000","last_peered":"2024-09-16T20:49:02.779473+0000","last_clean":"2024-09-16T20:49:02.779473+0000","last_became_active":"2024-09-16T20:47:58.274389+0000","last_became_peered":"2024-09-16T20:47:58.274389+0000","last_unstale":"2024-09-16T20:49:02.779473+0000","last_undegraded":"2024-09-16T20:49:02.779473+0000","last_fullsized":"2024-09-16T20:49:02.779473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:28:02.441830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043659299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"68'104","reported_seq":4956,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T20:49:02.779448+0000","last_change":"2024-09-16T20:48:50.791999+0000","last_active":"2024-09-16T20:49:02.779448+0000","last_peered":"2024-09-16T20:49:02.779448+0000","last_clean":"2024-09-16T20:49:02.779448+0000","last_became_active":"2024-09-16T20:48:50.719868+0000","last_became_peered":"2024-09-16T20:48:50.719868+0000","last_unstale":"2024-09-16T20:49:02.779448+0000","last_undegraded":"2024-09-16T20:49:02.779448+0000","last_fullsized":"2024-09-16T20:49:02.779448+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:12:32.182096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012934750000000001,"stat_sum":{"num_bytes":42991616,"num_objects":15,"num_object_clones":7,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":5,"num_read":4665,"num_read_kb":10156,"num_write":98,"num_write_kb":938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":523776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":128,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:05.768583+0000","last_change":"2024-09-16T20:48:00.278054+0000","last_active":"2024-09-16T20:49:05.768583+0000","last_peered":"2024-09-16T20:49:05.768583+0000","last_clean":"2024-09-16T20:49:05.768583+0000","last_became_active":"2024-09-16T20:47:58.272861+0000","last_became_peered":"2024-09-16T20:47:58.272861+0000","last_unstale":"2024-09-16T20:49:05.768583+0000","last_undegraded":"2024-09-16T20:49:05.768583+0000","last_fullsized":"2024-09-16T20:49:05.768583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:13:59.135547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012429489999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"72'235","reported_seq":1235,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T20:49:03.080275+0000","last_change":"2024-09-16T20:48:47.560529+0000","last_active":"2024-09-16T20:49:03.080275+0000","last_peered":"2024-09-16T20:49:03.080275+0000","last_clean":"2024-09-16T20:49:03.080275+0000","last_became_active":"2024-09-16T20:48:47.559672+0000","last_became_peered":"2024-09-16T20:48:47.559672+0000","last_unstale":"2024-09-16T20:49:03.080275+0000","last_undegraded":"2024-09-16T20:49:03.080275+0000","last_fullsized":"2024-09-16T20:49:03.080275+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":235,"log_dups_size":0,"ondisk_log_size":235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:49:57.790662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076367000000000004,"stat_sum":{"num_bytes":25125413,"num_objects":19,"num_object_clones":7,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":6,"num_read":1271,"num_read_kb":4419,"num_write":305,"num_write_kb":4379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":363045,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:05.767965+0000","last_change":"2024-09-16T20:48:50.717139+0000","last_active":"2024-09-16T20:49:05.767965+0000","last_peered":"2024-09-16T20:49:05.767965+0000","last_clean":"2024-09-16T20:49:05.767965+0000","last_became_active":"2024-09-16T20:48:50.716799+0000","last_became_peered":"2024-09-16T20:48:50.716799+0000","last_unstale":"2024-09-16T20:49:05.767965+0000","last_undegraded":"2024-09-16T20:49:05.767965+0000","last_fullsized":"2024-09-16T20:49:05.767965+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:28:13.275765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.5","version":"74'137","reported_seq":4326,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:05.772967+0000","last_change":"2024-09-16T20:48:53.732314+0000","last_active":"2024-09-16T20:49:05.772967+0000","last_peered":"2024-09-16T20:49:05.772967+0000","last_clean":"2024-09-16T20:49:05.772967+0000","last_became_active":"2024-09-16T20:48:53.731287+0000","last_became_peered":"2024-09-16T20:48:53.731287+0000","last_unstale":"2024-09-16T20:49:05.772967+0000","last_undegraded":"2024-09-16T20:49:05.772967+0000","last_fullsized":"2024-09-16T20:49:05.772967+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:09:07.472511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010471600000000001,"stat_sum":{"num_bytes":51223109,"num_objects":24,"num_object_clones":6,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":6,"num_read":3291,"num_read_kb":17347,"num_write":128,"num_write_kb":871,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":176266,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"73'229","reported_seq":3102,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:05.769163+0000","last_change":"2024-09-16T20:48:09.570254+0000","last_active":"2024-09-16T20:49:05.769163+0000","last_peered":"2024-09-16T20:49:05.769163+0000","last_clean":"2024-09-16T20:49:05.769163+0000","last_became_active":"2024-09-16T20:48:07.568143+0000","last_became_peered":"2024-09-16T20:48:07.568143+0000","last_unstale":"2024-09-16T20:49:05.769163+0000","last_undegraded":"2024-09-16T20:49:05.769163+0000","last_fullsized":"2024-09-16T20:49:05.769163+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":229,"log_dups_size":0,"ondisk_log_size":229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:29:55.819577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038800899999999998,"stat_sum":{"num_bytes":44459123,"num_objects":39,"num_object_clones":14,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":14,"num_read":3139,"num_read_kb":8707,"num_write":260,"num_write_kb":2174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":130,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:05.769325+0000","last_change":"2024-09-16T20:48:00.276970+0000","last_active":"2024-09-16T20:49:05.769325+0000","last_peered":"2024-09-16T20:49:05.769325+0000","last_clean":"2024-09-16T20:49:05.769325+0000","last_became_active":"2024-09-16T20:47:58.274617+0000","last_became_peered":"2024-09-16T20:47:58.274617+0000","last_unstale":"2024-09-16T20:49:05.769325+0000","last_undegraded":"2024-09-16T20:49:05.769325+0000","last_fullsized":"2024-09-16T20:49:05.769325+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:28:54.467905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062077499999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"74'200","reported_seq":2568,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:05.772583+0000","last_change":"2024-09-16T20:48:47.559753+0000","last_active":"2024-09-16T20:49:05.772583+0000","last_peered":"2024-09-16T20:49:05.772583+0000","last_clean":"2024-09-16T20:49:05.772583+0000","last_became_active":"2024-09-16T20:48:47.559154+0000","last_became_peered":"2024-09-16T20:48:47.559154+0000","last_unstale":"2024-09-16T20:49:05.772583+0000","last_undegraded":"2024-09-16T20:49:05.772583+0000","last_fullsized":"2024-09-16T20:49:05.772583+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":200,"log_dups_size":0,"ondisk_log_size":200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:36:41.429805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088331900000000003,"stat_sum":{"num_bytes":7174809,"num_objects":18,"num_object_clones":3,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":3,"num_read":4017,"num_read_kb":9365,"num_write":150,"num_write_kb":3680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":93299,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:05.767894+0000","last_change":"2024-09-16T20:48:50.717648+0000","last_active":"2024-09-16T20:49:05.767894+0000","last_peered":"2024-09-16T20:49:05.767894+0000","last_clean":"2024-09-16T20:49:05.767894+0000","last_became_active":"2024-09-16T20:48:50.717268+0000","last_became_peered":"2024-09-16T20:48:50.717268+0000","last_unstale":"2024-09-16T20:49:05.767894+0000","last_undegraded":"2024-09-16T20:49:05.767894+0000","last_fullsized":"2024-09-16T20:49:05.767894+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:43:37.442628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"73'113","reported_seq":2583,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T20:49:03.163947+0000","last_change":"2024-09-16T20:48:09.565596+0000","last_active":"2024-09-16T20:49:03.163947+0000","last_peered":"2024-09-16T20:49:03.163947+0000","last_clean":"2024-09-16T20:49:03.163947+0000","last_became_active":"2024-09-16T20:48:07.566097+0000","last_became_peered":"2024-09-16T20:48:07.566097+0000","last_unstale":"2024-09-16T20:49:03.163947+0000","last_undegraded":"2024-09-16T20:49:03.163947+0000","last_fullsized":"2024-09-16T20:49:03.163947+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:00:25.046151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012840460000000001,"stat_sum":{"num_bytes":40322560,"num_objects":19,"num_object_clones":6,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":4,"num_read":2390,"num_read_kb":23130,"num_write":120,"num_write_kb":6389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":134,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:05.772640+0000","last_change":"2024-09-16T20:48:44.557490+0000","last_active":"2024-09-16T20:49:05.772640+0000","last_peered":"2024-09-16T20:49:05.772640+0000","last_clean":"2024-09-16T20:49:05.772640+0000","last_became_active":"2024-09-16T20:48:44.556245+0000","last_became_peered":"2024-09-16T20:48:44.556245+0000","last_unstale":"2024-09-16T20:49:05.772640+0000","last_undegraded":"2024-09-16T20:49:05.772640+0000","last_fullsized":"2024-09-16T20:49:05.772640+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:18:01.568883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037636000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"73'249","reported_seq":3261,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:05.767988+0000","last_change":"2024-09-16T20:48:53.732328+0000","last_active":"2024-09-16T20:49:05.767988+0000","last_peered":"2024-09-16T20:49:05.767988+0000","last_clean":"2024-09-16T20:49:05.767988+0000","last_became_active":"2024-09-16T20:48:53.731305+0000","last_became_peered":"2024-09-16T20:48:53.731305+0000","last_unstale":"2024-09-16T20:49:05.767988+0000","last_undegraded":"2024-09-16T20:49:05.767988+0000","last_fullsized":"2024-09-16T20:49:05.767988+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:16:46.494543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063857599999999997,"stat_sum":{"num_bytes":19843103,"num_objects":26,"num_object_clones":7,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":6,"num_read":5467,"num_read_kb":14758,"num_write":184,"num_write_kb":2361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":931887,"num_keys_recovered":34,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":129,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:05.769442+0000","last_change":"2024-09-16T20:48:00.277739+0000","last_active":"2024-09-16T20:49:05.769442+0000","last_peered":"2024-09-16T20:49:05.769442+0000","last_clean":"2024-09-16T20:49:05.769442+0000","last_became_active":"2024-09-16T20:47:58.271174+0000","last_became_peered":"2024-09-16T20:47:58.271174+0000","last_unstale":"2024-09-16T20:49:05.769442+0000","last_undegraded":"2024-09-16T20:49:05.769442+0000","last_fullsized":"2024-09-16T20:49:05.769442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:11:35.985075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095771799999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":231,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:05.769747+0000","last_change":"2024-09-16T20:48:44.564732+0000","last_active":"2024-09-16T20:49:05.769747+0000","last_peered":"2024-09-16T20:49:05.769747+0000","last_clean":"2024-09-16T20:49:05.769747+0000","last_became_active":"2024-09-16T20:48:44.558587+0000","last_became_peered":"2024-09-16T20:48:44.558587+0000","last_unstale":"2024-09-16T20:49:05.769747+0000","last_undegraded":"2024-09-16T20:49:05.769747+0000","last_fullsized":"2024-09-16T20:49:05.769747+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:52.241408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:52.241408+0000","last_clean_scrub_stamp":"2024-09-16T20:47:52.241408+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:22:02.373518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"70'201","reported_seq":813,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T20:49:02.779525+0000","last_change":"2024-09-16T20:48:09.565136+0000","last_active":"2024-09-16T20:49:02.779525+0000","last_peered":"2024-09-16T20:49:02.779525+0000","last_clean":"2024-09-16T20:49:02.779525+0000","last_became_active":"2024-09-16T20:48:07.566139+0000","last_became_peered":"2024-09-16T20:48:07.566139+0000","last_unstale":"2024-09-16T20:49:02.779525+0000","last_undegraded":"2024-09-16T20:49:02.779525+0000","last_fullsized":"2024-09-16T20:49:02.779525+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:58:40.254706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001025172,"stat_sum":{"num_bytes":6528325,"num_objects":11,"num_object_clones":3,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":3,"num_read":1141,"num_read_kb":5971,"num_write":288,"num_write_kb":3234,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'183","reported_seq":2597,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:06.074339+0000","last_change":"2024-09-16T20:48:53.737981+0000","last_active":"2024-09-16T20:49:06.074339+0000","last_peered":"2024-09-16T20:49:06.074339+0000","last_clean":"2024-09-16T20:49:06.074339+0000","last_became_active":"2024-09-16T20:48:53.736558+0000","last_became_peered":"2024-09-16T20:48:53.736558+0000","last_unstale":"2024-09-16T20:49:06.074339+0000","last_undegraded":"2024-09-16T20:49:06.074339+0000","last_fullsized":"2024-09-16T20:49:06.074339+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:45:36.477396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044614999999999998,"stat_sum":{"num_bytes":36836368,"num_objects":24,"num_object_clones":8,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":6,"num_read":3032,"num_read_kb":13794,"num_write":180,"num_write_kb":6558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":2635264,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"75'168","reported_seq":3149,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:05.769257+0000","last_change":"2024-09-16T20:48:09.569992+0000","last_active":"2024-09-16T20:49:05.769257+0000","last_peered":"2024-09-16T20:49:05.769257+0000","last_clean":"2024-09-16T20:49:05.769257+0000","last_became_active":"2024-09-16T20:48:07.563972+0000","last_became_peered":"2024-09-16T20:48:07.563972+0000","last_unstale":"2024-09-16T20:49:05.769257+0000","last_undegraded":"2024-09-16T20:49:05.769257+0000","last_fullsized":"2024-09-16T20:49:05.769257+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:24:50.257643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039392600000000001,"stat_sum":{"num_bytes":37339949,"num_objects":32,"num_object_clones":10,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":4,"num_read":2959,"num_read_kb":24894,"num_write":319,"num_write_kb":6308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"74'169","reported_seq":2824,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:06.219650+0000","last_change":"2024-09-16T20:48:54.735384+0000","last_active":"2024-09-16T20:49:06.219650+0000","last_peered":"2024-09-16T20:49:06.219650+0000","last_clean":"2024-09-16T20:49:06.219650+0000","last_became_active":"2024-09-16T20:48:54.734409+0000","last_became_peered":"2024-09-16T20:48:54.734409+0000","last_unstale":"2024-09-16T20:49:06.219650+0000","last_undegraded":"2024-09-16T20:49:06.219650+0000","last_fullsized":"2024-09-16T20:49:06.219650+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:20:29.099217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014289540000000001,"stat_sum":{"num_bytes":36485193,"num_objects":17,"num_object_clones":4,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":4,"num_read":2733,"num_read_kb":6275,"num_write":202,"num_write_kb":4405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":971906,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"75'293","reported_seq":9213,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:06.215882+0000","last_change":"2024-09-16T20:48:54.735265+0000","last_active":"2024-09-16T20:49:06.215882+0000","last_peered":"2024-09-16T20:49:06.215882+0000","last_clean":"2024-09-16T20:49:06.215882+0000","last_became_active":"2024-09-16T20:48:54.734672+0000","last_became_peered":"2024-09-16T20:48:54.734672+0000","last_unstale":"2024-09-16T20:49:06.215882+0000","last_undegraded":"2024-09-16T20:49:06.215882+0000","last_fullsized":"2024-09-16T20:49:06.215882+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:08:19.969082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047762999999999999,"stat_sum":{"num_bytes":62220699,"num_objects":20,"num_object_clones":6,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":5,"num_read":9821,"num_read_kb":41554,"num_write":448,"num_write_kb":9855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":530734,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"75'261","reported_seq":4740,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:06.177118+0000","last_change":"2024-09-16T20:48:09.565667+0000","last_active":"2024-09-16T20:49:06.177118+0000","last_peered":"2024-09-16T20:49:06.177118+0000","last_clean":"2024-09-16T20:49:06.177118+0000","last_became_active":"2024-09-16T20:48:07.567327+0000","last_became_peered":"2024-09-16T20:48:07.567327+0000","last_unstale":"2024-09-16T20:49:06.177118+0000","last_undegraded":"2024-09-16T20:49:06.177118+0000","last_fullsized":"2024-09-16T20:49:06.177118+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":261,"log_dups_size":0,"ondisk_log_size":261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:21:33.351638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012087560000000001,"stat_sum":{"num_bytes":24419840,"num_objects":22,"num_object_clones":7,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":4,"num_read":4729,"num_read_kb":14889,"num_write":217,"num_write_kb":7270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"74'123","reported_seq":2824,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:06.215922+0000","last_change":"2024-09-16T20:48:09.565024+0000","last_active":"2024-09-16T20:49:06.215922+0000","last_peered":"2024-09-16T20:49:06.215922+0000","last_clean":"2024-09-16T20:49:06.215922+0000","last_became_active":"2024-09-16T20:48:07.570704+0000","last_became_peered":"2024-09-16T20:48:07.570704+0000","last_unstale":"2024-09-16T20:49:06.215922+0000","last_undegraded":"2024-09-16T20:49:06.215922+0000","last_fullsized":"2024-09-16T20:49:06.215922+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:53:48.266020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011026149999999999,"stat_sum":{"num_bytes":43014849,"num_objects":26,"num_object_clones":9,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":5,"num_read":2620,"num_read_kb":11764,"num_write":120,"num_write_kb":3719,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"74'189","reported_seq":3650,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:05.768845+0000","last_change":"2024-09-16T20:48:09.570786+0000","last_active":"2024-09-16T20:49:05.768845+0000","last_peered":"2024-09-16T20:49:05.768845+0000","last_clean":"2024-09-16T20:49:05.768845+0000","last_became_active":"2024-09-16T20:48:07.560783+0000","last_became_peered":"2024-09-16T20:48:07.560783+0000","last_unstale":"2024-09-16T20:49:05.768845+0000","last_undegraded":"2024-09-16T20:49:05.768845+0000","last_fullsized":"2024-09-16T20:49:05.768845+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:42:26.172477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040623999999999999,"stat_sum":{"num_bytes":22190096,"num_objects":33,"num_object_clones":12,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":8,"num_read":4314,"num_read_kb":24664,"num_write":158,"num_write_kb":8322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"74'232","reported_seq":4660,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:05.769793+0000","last_change":"2024-09-16T20:48:47.563240+0000","last_active":"2024-09-16T20:49:05.769793+0000","last_peered":"2024-09-16T20:49:05.769793+0000","last_clean":"2024-09-16T20:49:05.769793+0000","last_became_active":"2024-09-16T20:48:47.562590+0000","last_became_peered":"2024-09-16T20:48:47.562590+0000","last_unstale":"2024-09-16T20:49:05.769793+0000","last_undegraded":"2024-09-16T20:49:05.769793+0000","last_fullsized":"2024-09-16T20:49:05.769793+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:02:27.719797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115576,"stat_sum":{"num_bytes":53984315,"num_objects":34,"num_object_clones":13,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":12,"num_read":5469,"num_read_kb":30684,"num_write":189,"num_write_kb":3250,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":2587707,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"73'134","reported_seq":3460,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:05.772614+0000","last_change":"2024-09-16T20:48:09.564513+0000","last_active":"2024-09-16T20:49:05.772614+0000","last_peered":"2024-09-16T20:49:05.772614+0000","last_clean":"2024-09-16T20:49:05.772614+0000","last_became_active":"2024-09-16T20:48:07.571985+0000","last_became_peered":"2024-09-16T20:48:07.571985+0000","last_unstale":"2024-09-16T20:49:05.772614+0000","last_undegraded":"2024-09-16T20:49:05.772614+0000","last_fullsized":"2024-09-16T20:49:05.772614+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:10:02.137858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077335299999999995,"stat_sum":{"num_bytes":41333248,"num_objects":23,"num_object_clones":4,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":1,"num_read":3232,"num_read_kb":41914,"num_write":167,"num_write_kb":10470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"74'259","reported_seq":1686,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:05.768550+0000","last_change":"2024-09-16T20:48:53.735871+0000","last_active":"2024-09-16T20:49:05.768550+0000","last_peered":"2024-09-16T20:49:05.768550+0000","last_clean":"2024-09-16T20:49:05.768550+0000","last_became_active":"2024-09-16T20:48:53.734866+0000","last_became_peered":"2024-09-16T20:48:53.734866+0000","last_unstale":"2024-09-16T20:49:05.768550+0000","last_undegraded":"2024-09-16T20:49:05.768550+0000","last_fullsized":"2024-09-16T20:49:05.768550+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":259,"log_dups_size":0,"ondisk_log_size":259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:07:25.910251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000840955,"stat_sum":{"num_bytes":15595599,"num_objects":17,"num_object_clones":5,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":3,"num_read":1898,"num_read_kb":7035,"num_write":350,"num_write_kb":1731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":756384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"74'237","reported_seq":3258,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:05.769090+0000","last_change":"2024-09-16T20:48:09.565356+0000","last_active":"2024-09-16T20:49:05.769090+0000","last_peered":"2024-09-16T20:49:05.769090+0000","last_clean":"2024-09-16T20:49:05.769090+0000","last_became_active":"2024-09-16T20:48:07.565717+0000","last_became_peered":"2024-09-16T20:48:07.565717+0000","last_unstale":"2024-09-16T20:49:05.769090+0000","last_undegraded":"2024-09-16T20:49:05.769090+0000","last_fullsized":"2024-09-16T20:49:05.769090+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":237,"log_dups_size":0,"ondisk_log_size":237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:00:49.876591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013583239999999999,"stat_sum":{"num_bytes":22095958,"num_objects":17,"num_object_clones":3,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":3,"num_read":3619,"num_read_kb":13012,"num_write":362,"num_write_kb":6708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":932712780,"num_objects":744,"num_object_clones":216,"num_object_copies":1488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":744,"num_whiteouts":156,"num_read":128720,"num_read_kb":558428,"num_write":9232,"num_write_kb":150967,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1007,"num_bytes_recovered":31195153,"num_keys_recovered":234,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":201371648,"data_stored":227852367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7472,"ondisk_log_size":7472,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":0,"seq":55834574867,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62392,"kb_used_data":35364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94309448,"statfs":{"total":96636764160,"available":96572874752,"internally_reserved":0,"allocated":36212736,"data_stored":39099286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72920,"kb_used_data":45888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94298920,"statfs":{"total":96636764160,"available":96562094080,"internally_reserved":0,"allocated":46989312,"data_stored":52574072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73564,"kb_used_data":42376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94298276,"statfs":{"total":96636764160,"available":96561434624,"internally_reserved":0,"allocated":43393024,"data_stored":48267306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100908,"kb_used_data":51672,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94270932,"statfs":{"total":96636764160,"available":96533434368,"internally_reserved":0,"allocated":52912128,"data_stored":60498315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640274,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56776,"kb_used_data":17272,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94315064,"statfs":{"total":96636764160,"available":96578625536,"internally_reserved":0,"allocated":17686528,"data_stored":19010783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":52047872,"data_stored":60153666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":41971712,"data_stored":47345821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45547520,"data_stored":51644353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":35356672,"data_stored":38754672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":18169856,"data_stored":19992353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8278016,"data_stored":9961502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T20:49:07.301 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T20:49:07.309 INFO:teuthology.orchestra.run.smithi139.stdout:2712 flatten 2024-09-16T20:49:07.344 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:07.340+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:07.402 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:07.402 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:07.416 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:49:07.465 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:07.461+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:07.525 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:07.524+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:07.612 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:07.612 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:07.626 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:49:07.626 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:07.625+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:07.726 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:07.724+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:07.774 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:07.775 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:07.779 INFO:teuthology.orchestra.run.smithi029.stderr:noscrub is set 2024-09-16T20:49:07.786 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:49:07.804 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:49:07.805 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-09-16T20:49:07.817 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":111,"stamp":"2024-09-16T20:49:07.779334+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":933303167,"num_objects":748,"num_object_clones":216,"num_object_copies":1496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":748,"num_whiteouts":156,"num_read":128786,"num_read_kb":558483,"num_write":9335,"num_write_kb":152597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1009,"num_bytes_recovered":31785521,"num_keys_recovered":234,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7547,"ondisk_log_size":7547,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":366560,"kb_used_data":192572,"kb_used_omap":7,"kb_used_meta":173880,"kb_avail":471492640,"statfs":{"total":483183820800,"available":482808463360,"internally_reserved":0,"allocated":197193728,"data_stored":219449762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7972,"internal_metadata":178053340},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18049157,"num_objects":111,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":111,"num_whiteouts":0,"num_read":22693,"num_read_kb":126216,"num_write":1133,"num_write_kb":8980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.018269"},"pg_stats":[{"pgid":"3.1f","version":"74'158","reported_seq":2419,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:06.217231+0000","last_change":"2024-09-16T20:48:53.732229+0000","last_active":"2024-09-16T20:49:06.217231+0000","last_peered":"2024-09-16T20:49:06.217231+0000","last_clean":"2024-09-16T20:49:06.217231+0000","last_became_active":"2024-09-16T20:48:53.731525+0000","last_became_peered":"2024-09-16T20:48:53.731525+0000","last_unstale":"2024-09-16T20:49:06.217231+0000","last_undegraded":"2024-09-16T20:49:06.217231+0000","last_fullsized":"2024-09-16T20:49:06.217231+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:21:53.398218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071634499999999996,"stat_sum":{"num_bytes":27213882,"num_objects":21,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":2,"num_read":2254,"num_read_kb":6692,"num_write":143,"num_write_kb":2005,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":921716,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1e","version":"73'448","reported_seq":11889,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:05.768222+0000","last_change":"2024-09-16T20:48:53.732932+0000","last_active":"2024-09-16T20:49:05.768222+0000","last_peered":"2024-09-16T20:49:05.768222+0000","last_clean":"2024-09-16T20:49:05.768222+0000","last_became_active":"2024-09-16T20:48:53.732081+0000","last_became_peered":"2024-09-16T20:48:53.732081+0000","last_unstale":"2024-09-16T20:49:05.768222+0000","last_undegraded":"2024-09-16T20:49:05.768222+0000","last_fullsized":"2024-09-16T20:49:05.768222+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:17:52.739495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084864999999999995,"stat_sum":{"num_bytes":31008913,"num_objects":26,"num_object_clones":6,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":3,"num_read":13121,"num_read_kb":35383,"num_write":576,"num_write_kb":4766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":1415412,"num_keys_recovered":59,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"74'375","reported_seq":4732,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:06.215869+0000","last_change":"2024-09-16T20:48:09.564507+0000","last_active":"2024-09-16T20:49:06.215869+0000","last_peered":"2024-09-16T20:49:06.215869+0000","last_clean":"2024-09-16T20:49:06.215869+0000","last_became_active":"2024-09-16T20:48:07.568025+0000","last_became_peered":"2024-09-16T20:48:07.568025+0000","last_unstale":"2024-09-16T20:49:06.215869+0000","last_undegraded":"2024-09-16T20:49:06.215869+0000","last_fullsized":"2024-09-16T20:49:06.215869+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:01:22.178335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068985699999999999,"stat_sum":{"num_bytes":52607554,"num_objects":30,"num_object_clones":9,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":5,"num_read":5750,"num_read_kb":56071,"num_write":493,"num_write_kb":7718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'374","reported_seq":1409,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:06.220563+0000","last_change":"2024-09-16T20:48:09.565661+0000","last_active":"2024-09-16T20:49:06.220563+0000","last_peered":"2024-09-16T20:49:06.220563+0000","last_clean":"2024-09-16T20:49:06.220563+0000","last_became_active":"2024-09-16T20:48:07.565241+0000","last_became_peered":"2024-09-16T20:48:07.565241+0000","last_unstale":"2024-09-16T20:49:06.220563+0000","last_undegraded":"2024-09-16T20:49:06.220563+0000","last_fullsized":"2024-09-16T20:49:06.220563+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:30:21.676377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001045589,"stat_sum":{"num_bytes":7610933,"num_objects":24,"num_object_clones":5,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":4,"num_read":2512,"num_read_kb":8045,"num_write":622,"num_write_kb":3689,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"75'498","reported_seq":2675,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:05.776564+0000","last_change":"2024-09-16T20:48:54.735381+0000","last_active":"2024-09-16T20:49:05.776564+0000","last_peered":"2024-09-16T20:49:05.776564+0000","last_clean":"2024-09-16T20:49:05.776564+0000","last_became_active":"2024-09-16T20:48:54.734278+0000","last_became_peered":"2024-09-16T20:48:54.734278+0000","last_unstale":"2024-09-16T20:49:05.776564+0000","last_undegraded":"2024-09-16T20:49:05.776564+0000","last_fullsized":"2024-09-16T20:49:05.776564+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":498,"log_dups_size":0,"ondisk_log_size":498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:35:54.163829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057432399999999997,"stat_sum":{"num_bytes":21458744,"num_objects":32,"num_object_clones":8,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":7,"num_read":3881,"num_read_kb":17904,"num_write":766,"num_write_kb":9062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":1963751,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"74'566","reported_seq":5682,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:06.216200+0000","last_change":"2024-09-16T20:48:47.560036+0000","last_active":"2024-09-16T20:49:06.216200+0000","last_peered":"2024-09-16T20:49:06.216200+0000","last_clean":"2024-09-16T20:49:06.216200+0000","last_became_active":"2024-09-16T20:48:47.558880+0000","last_became_peered":"2024-09-16T20:48:47.558880+0000","last_unstale":"2024-09-16T20:49:06.216200+0000","last_undegraded":"2024-09-16T20:49:06.216200+0000","last_fullsized":"2024-09-16T20:49:06.216200+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":566,"log_dups_size":0,"ondisk_log_size":566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:15:03.077216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002037849,"stat_sum":{"num_bytes":5546288,"num_objects":26,"num_object_clones":6,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":5,"num_read":9674,"num_read_kb":10969,"num_write":708,"num_write_kb":2125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":333388,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"72'116","reported_seq":3743,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T20:49:03.186311+0000","last_change":"2024-09-16T20:48:53.732127+0000","last_active":"2024-09-16T20:49:03.186311+0000","last_peered":"2024-09-16T20:49:03.186311+0000","last_clean":"2024-09-16T20:49:03.186311+0000","last_became_active":"2024-09-16T20:48:53.731485+0000","last_became_peered":"2024-09-16T20:48:53.731485+0000","last_unstale":"2024-09-16T20:49:03.186311+0000","last_undegraded":"2024-09-16T20:49:03.186311+0000","last_fullsized":"2024-09-16T20:49:03.186311+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:30:41.446995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057190699999999997,"stat_sum":{"num_bytes":25220640,"num_objects":25,"num_object_clones":10,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":4,"num_read":3419,"num_read_kb":16323,"num_write":112,"num_write_kb":3634,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":5091328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"75'175","reported_seq":2976,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:06.220280+0000","last_change":"2024-09-16T20:48:53.731382+0000","last_active":"2024-09-16T20:49:06.220280+0000","last_peered":"2024-09-16T20:49:06.220280+0000","last_clean":"2024-09-16T20:49:06.220280+0000","last_became_active":"2024-09-16T20:48:53.730714+0000","last_became_peered":"2024-09-16T20:48:53.730714+0000","last_unstale":"2024-09-16T20:49:06.220280+0000","last_undegraded":"2024-09-16T20:49:06.220280+0000","last_fullsized":"2024-09-16T20:49:06.220280+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:50:44.320055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011061790000000001,"stat_sum":{"num_bytes":6090256,"num_objects":16,"num_object_clones":2,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":1,"num_read":4019,"num_read_kb":12667,"num_write":141,"num_write_kb":4377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2412544,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"75'282","reported_seq":2233,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:05.768726+0000","last_change":"2024-09-16T20:48:47.564078+0000","last_active":"2024-09-16T20:49:05.768726+0000","last_peered":"2024-09-16T20:49:05.768726+0000","last_clean":"2024-09-16T20:49:05.768726+0000","last_became_active":"2024-09-16T20:48:47.563287+0000","last_became_peered":"2024-09-16T20:48:47.563287+0000","last_unstale":"2024-09-16T20:49:05.768726+0000","last_undegraded":"2024-09-16T20:49:05.768726+0000","last_fullsized":"2024-09-16T20:49:05.768726+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:38:22.963563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057915500000000003,"stat_sum":{"num_bytes":10004558,"num_objects":15,"num_object_clones":3,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":2,"num_read":3515,"num_read_kb":5573,"num_write":286,"num_write_kb":784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":34852,"num_keys_recovered":24,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"74'387","reported_seq":1946,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:06.219856+0000","last_change":"2024-09-16T20:48:53.731777+0000","last_active":"2024-09-16T20:49:06.219856+0000","last_peered":"2024-09-16T20:49:06.219856+0000","last_clean":"2024-09-16T20:49:06.219856+0000","last_became_active":"2024-09-16T20:48:53.730899+0000","last_became_peered":"2024-09-16T20:48:53.730899+0000","last_unstale":"2024-09-16T20:49:06.219856+0000","last_undegraded":"2024-09-16T20:49:06.219856+0000","last_fullsized":"2024-09-16T20:49:06.219856+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:17:10.090950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010075049999999999,"stat_sum":{"num_bytes":12332145,"num_objects":23,"num_object_clones":5,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":4,"num_read":2632,"num_read_kb":15665,"num_write":594,"num_write_kb":5717,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":6235436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"74'136","reported_seq":2069,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:05.768631+0000","last_change":"2024-09-16T20:48:53.734382+0000","last_active":"2024-09-16T20:49:05.768631+0000","last_peered":"2024-09-16T20:49:05.768631+0000","last_clean":"2024-09-16T20:49:05.768631+0000","last_became_active":"2024-09-16T20:48:53.733689+0000","last_became_peered":"2024-09-16T20:48:53.733689+0000","last_unstale":"2024-09-16T20:49:05.768631+0000","last_undegraded":"2024-09-16T20:49:05.768631+0000","last_fullsized":"2024-09-16T20:49:05.768631+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:53:34.168240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051100699999999995,"stat_sum":{"num_bytes":37896192,"num_objects":28,"num_object_clones":10,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":7,"num_read":1595,"num_read_kb":9559,"num_write":134,"num_write_kb":2945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":2749952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"75'150","reported_seq":5704,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:06.216179+0000","last_change":"2024-09-16T20:48:09.566005+0000","last_active":"2024-09-16T20:49:06.216179+0000","last_peered":"2024-09-16T20:49:06.216179+0000","last_clean":"2024-09-16T20:49:06.216179+0000","last_became_active":"2024-09-16T20:48:07.568236+0000","last_became_peered":"2024-09-16T20:48:07.568236+0000","last_unstale":"2024-09-16T20:49:06.216179+0000","last_undegraded":"2024-09-16T20:49:06.216179+0000","last_fullsized":"2024-09-16T20:49:06.216179+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:41:07.660360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001271749,"stat_sum":{"num_bytes":56578786,"num_objects":31,"num_object_clones":11,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":8,"num_read":5727,"num_read_kb":32286,"num_write":318,"num_write_kb":9911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":128,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:05.769724+0000","last_change":"2024-09-16T20:48:00.277670+0000","last_active":"2024-09-16T20:49:05.769724+0000","last_peered":"2024-09-16T20:49:05.769724+0000","last_clean":"2024-09-16T20:49:05.769724+0000","last_became_active":"2024-09-16T20:47:58.274489+0000","last_became_peered":"2024-09-16T20:47:58.274489+0000","last_unstale":"2024-09-16T20:49:05.769724+0000","last_undegraded":"2024-09-16T20:49:05.769724+0000","last_fullsized":"2024-09-16T20:49:05.769724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:27:12.633064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091339500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"68'91","reported_seq":1137,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T20:49:02.779431+0000","last_change":"2024-09-16T20:48:50.757942+0000","last_active":"2024-09-16T20:49:02.779431+0000","last_peered":"2024-09-16T20:49:02.779431+0000","last_clean":"2024-09-16T20:49:02.779431+0000","last_became_active":"2024-09-16T20:48:50.719161+0000","last_became_peered":"2024-09-16T20:48:50.719161+0000","last_unstale":"2024-09-16T20:49:02.779431+0000","last_undegraded":"2024-09-16T20:49:02.779431+0000","last_fullsized":"2024-09-16T20:49:02.779431+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:49:32.448065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067924999999999995,"stat_sum":{"num_bytes":5959717,"num_objects":11,"num_object_clones":3,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":2,"num_read":814,"num_read_kb":6959,"num_write":94,"num_write_kb":1612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":466506,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":124,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T20:49:02.779473+0000","last_change":"2024-09-16T20:48:00.277182+0000","last_active":"2024-09-16T20:49:02.779473+0000","last_peered":"2024-09-16T20:49:02.779473+0000","last_clean":"2024-09-16T20:49:02.779473+0000","last_became_active":"2024-09-16T20:47:58.274389+0000","last_became_peered":"2024-09-16T20:47:58.274389+0000","last_unstale":"2024-09-16T20:49:02.779473+0000","last_undegraded":"2024-09-16T20:49:02.779473+0000","last_fullsized":"2024-09-16T20:49:02.779473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:28:02.441830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043659299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"68'104","reported_seq":4956,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T20:49:02.779448+0000","last_change":"2024-09-16T20:48:50.791999+0000","last_active":"2024-09-16T20:49:02.779448+0000","last_peered":"2024-09-16T20:49:02.779448+0000","last_clean":"2024-09-16T20:49:02.779448+0000","last_became_active":"2024-09-16T20:48:50.719868+0000","last_became_peered":"2024-09-16T20:48:50.719868+0000","last_unstale":"2024-09-16T20:49:02.779448+0000","last_undegraded":"2024-09-16T20:49:02.779448+0000","last_fullsized":"2024-09-16T20:49:02.779448+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:12:32.182096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012934750000000001,"stat_sum":{"num_bytes":42991616,"num_objects":15,"num_object_clones":7,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":5,"num_read":4665,"num_read_kb":10156,"num_write":98,"num_write_kb":938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":523776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":128,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:05.768583+0000","last_change":"2024-09-16T20:48:00.278054+0000","last_active":"2024-09-16T20:49:05.768583+0000","last_peered":"2024-09-16T20:49:05.768583+0000","last_clean":"2024-09-16T20:49:05.768583+0000","last_became_active":"2024-09-16T20:47:58.272861+0000","last_became_peered":"2024-09-16T20:47:58.272861+0000","last_unstale":"2024-09-16T20:49:05.768583+0000","last_undegraded":"2024-09-16T20:49:05.768583+0000","last_fullsized":"2024-09-16T20:49:05.768583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:13:59.135547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012429489999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"72'235","reported_seq":1235,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T20:49:03.080275+0000","last_change":"2024-09-16T20:48:47.560529+0000","last_active":"2024-09-16T20:49:03.080275+0000","last_peered":"2024-09-16T20:49:03.080275+0000","last_clean":"2024-09-16T20:49:03.080275+0000","last_became_active":"2024-09-16T20:48:47.559672+0000","last_became_peered":"2024-09-16T20:48:47.559672+0000","last_unstale":"2024-09-16T20:49:03.080275+0000","last_undegraded":"2024-09-16T20:49:03.080275+0000","last_fullsized":"2024-09-16T20:49:03.080275+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":235,"log_dups_size":0,"ondisk_log_size":235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:49:57.790662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076367000000000004,"stat_sum":{"num_bytes":25125413,"num_objects":19,"num_object_clones":7,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":6,"num_read":1271,"num_read_kb":4419,"num_write":305,"num_write_kb":4379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":363045,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:05.767965+0000","last_change":"2024-09-16T20:48:50.717139+0000","last_active":"2024-09-16T20:49:05.767965+0000","last_peered":"2024-09-16T20:49:05.767965+0000","last_clean":"2024-09-16T20:49:05.767965+0000","last_became_active":"2024-09-16T20:48:50.716799+0000","last_became_peered":"2024-09-16T20:48:50.716799+0000","last_unstale":"2024-09-16T20:49:05.767965+0000","last_undegraded":"2024-09-16T20:49:05.767965+0000","last_fullsized":"2024-09-16T20:49:05.767965+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:28:13.275765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.5","version":"74'137","reported_seq":4326,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:05.772967+0000","last_change":"2024-09-16T20:48:53.732314+0000","last_active":"2024-09-16T20:49:05.772967+0000","last_peered":"2024-09-16T20:49:05.772967+0000","last_clean":"2024-09-16T20:49:05.772967+0000","last_became_active":"2024-09-16T20:48:53.731287+0000","last_became_peered":"2024-09-16T20:48:53.731287+0000","last_unstale":"2024-09-16T20:49:05.772967+0000","last_undegraded":"2024-09-16T20:49:05.772967+0000","last_fullsized":"2024-09-16T20:49:05.772967+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:09:07.472511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010471600000000001,"stat_sum":{"num_bytes":51223109,"num_objects":24,"num_object_clones":6,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":6,"num_read":3291,"num_read_kb":17347,"num_write":128,"num_write_kb":871,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":176266,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"73'229","reported_seq":3102,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:05.769163+0000","last_change":"2024-09-16T20:48:09.570254+0000","last_active":"2024-09-16T20:49:05.769163+0000","last_peered":"2024-09-16T20:49:05.769163+0000","last_clean":"2024-09-16T20:49:05.769163+0000","last_became_active":"2024-09-16T20:48:07.568143+0000","last_became_peered":"2024-09-16T20:48:07.568143+0000","last_unstale":"2024-09-16T20:49:05.769163+0000","last_undegraded":"2024-09-16T20:49:05.769163+0000","last_fullsized":"2024-09-16T20:49:05.769163+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":229,"log_dups_size":0,"ondisk_log_size":229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:29:55.819577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038800899999999998,"stat_sum":{"num_bytes":44459123,"num_objects":39,"num_object_clones":14,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":14,"num_read":3139,"num_read_kb":8707,"num_write":260,"num_write_kb":2174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":130,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:05.769325+0000","last_change":"2024-09-16T20:48:00.276970+0000","last_active":"2024-09-16T20:49:05.769325+0000","last_peered":"2024-09-16T20:49:05.769325+0000","last_clean":"2024-09-16T20:49:05.769325+0000","last_became_active":"2024-09-16T20:47:58.274617+0000","last_became_peered":"2024-09-16T20:47:58.274617+0000","last_unstale":"2024-09-16T20:49:05.769325+0000","last_undegraded":"2024-09-16T20:49:05.769325+0000","last_fullsized":"2024-09-16T20:49:05.769325+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:28:54.467905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062077499999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"74'200","reported_seq":2568,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:05.772583+0000","last_change":"2024-09-16T20:48:47.559753+0000","last_active":"2024-09-16T20:49:05.772583+0000","last_peered":"2024-09-16T20:49:05.772583+0000","last_clean":"2024-09-16T20:49:05.772583+0000","last_became_active":"2024-09-16T20:48:47.559154+0000","last_became_peered":"2024-09-16T20:48:47.559154+0000","last_unstale":"2024-09-16T20:49:05.772583+0000","last_undegraded":"2024-09-16T20:49:05.772583+0000","last_fullsized":"2024-09-16T20:49:05.772583+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":200,"log_dups_size":0,"ondisk_log_size":200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:36:41.429805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088331900000000003,"stat_sum":{"num_bytes":7174809,"num_objects":18,"num_object_clones":3,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":3,"num_read":4017,"num_read_kb":9365,"num_write":150,"num_write_kb":3680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":93299,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:05.767894+0000","last_change":"2024-09-16T20:48:50.717648+0000","last_active":"2024-09-16T20:49:05.767894+0000","last_peered":"2024-09-16T20:49:05.767894+0000","last_clean":"2024-09-16T20:49:05.767894+0000","last_became_active":"2024-09-16T20:48:50.717268+0000","last_became_peered":"2024-09-16T20:48:50.717268+0000","last_unstale":"2024-09-16T20:49:05.767894+0000","last_undegraded":"2024-09-16T20:49:05.767894+0000","last_fullsized":"2024-09-16T20:49:05.767894+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:43:37.442628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"73'113","reported_seq":2583,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T20:49:03.163947+0000","last_change":"2024-09-16T20:48:09.565596+0000","last_active":"2024-09-16T20:49:03.163947+0000","last_peered":"2024-09-16T20:49:03.163947+0000","last_clean":"2024-09-16T20:49:03.163947+0000","last_became_active":"2024-09-16T20:48:07.566097+0000","last_became_peered":"2024-09-16T20:48:07.566097+0000","last_unstale":"2024-09-16T20:49:03.163947+0000","last_undegraded":"2024-09-16T20:49:03.163947+0000","last_fullsized":"2024-09-16T20:49:03.163947+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:00:25.046151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012840460000000001,"stat_sum":{"num_bytes":40322560,"num_objects":19,"num_object_clones":6,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":4,"num_read":2390,"num_read_kb":23130,"num_write":120,"num_write_kb":6389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":134,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:05.772640+0000","last_change":"2024-09-16T20:48:44.557490+0000","last_active":"2024-09-16T20:49:05.772640+0000","last_peered":"2024-09-16T20:49:05.772640+0000","last_clean":"2024-09-16T20:49:05.772640+0000","last_became_active":"2024-09-16T20:48:44.556245+0000","last_became_peered":"2024-09-16T20:48:44.556245+0000","last_unstale":"2024-09-16T20:49:05.772640+0000","last_undegraded":"2024-09-16T20:49:05.772640+0000","last_fullsized":"2024-09-16T20:49:05.772640+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:18:01.568883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037636000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"73'249","reported_seq":3261,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:05.767988+0000","last_change":"2024-09-16T20:48:53.732328+0000","last_active":"2024-09-16T20:49:05.767988+0000","last_peered":"2024-09-16T20:49:05.767988+0000","last_clean":"2024-09-16T20:49:05.767988+0000","last_became_active":"2024-09-16T20:48:53.731305+0000","last_became_peered":"2024-09-16T20:48:53.731305+0000","last_unstale":"2024-09-16T20:49:05.767988+0000","last_undegraded":"2024-09-16T20:49:05.767988+0000","last_fullsized":"2024-09-16T20:49:05.767988+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:16:46.494543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063857599999999997,"stat_sum":{"num_bytes":19843103,"num_objects":26,"num_object_clones":7,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":6,"num_read":5467,"num_read_kb":14758,"num_write":184,"num_write_kb":2361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":931887,"num_keys_recovered":34,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":129,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:05.769442+0000","last_change":"2024-09-16T20:48:00.277739+0000","last_active":"2024-09-16T20:49:05.769442+0000","last_peered":"2024-09-16T20:49:05.769442+0000","last_clean":"2024-09-16T20:49:05.769442+0000","last_became_active":"2024-09-16T20:47:58.271174+0000","last_became_peered":"2024-09-16T20:47:58.271174+0000","last_unstale":"2024-09-16T20:49:05.769442+0000","last_undegraded":"2024-09-16T20:49:05.769442+0000","last_fullsized":"2024-09-16T20:49:05.769442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:11:35.985075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095771799999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":231,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:05.769747+0000","last_change":"2024-09-16T20:48:44.564732+0000","last_active":"2024-09-16T20:49:05.769747+0000","last_peered":"2024-09-16T20:49:05.769747+0000","last_clean":"2024-09-16T20:49:05.769747+0000","last_became_active":"2024-09-16T20:48:44.558587+0000","last_became_peered":"2024-09-16T20:48:44.558587+0000","last_unstale":"2024-09-16T20:49:05.769747+0000","last_undegraded":"2024-09-16T20:49:05.769747+0000","last_fullsized":"2024-09-16T20:49:05.769747+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:52.241408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:52.241408+0000","last_clean_scrub_stamp":"2024-09-16T20:47:52.241408+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:22:02.373518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"70'201","reported_seq":813,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T20:49:02.779525+0000","last_change":"2024-09-16T20:48:09.565136+0000","last_active":"2024-09-16T20:49:02.779525+0000","last_peered":"2024-09-16T20:49:02.779525+0000","last_clean":"2024-09-16T20:49:02.779525+0000","last_became_active":"2024-09-16T20:48:07.566139+0000","last_became_peered":"2024-09-16T20:48:07.566139+0000","last_unstale":"2024-09-16T20:49:02.779525+0000","last_undegraded":"2024-09-16T20:49:02.779525+0000","last_fullsized":"2024-09-16T20:49:02.779525+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:58:40.254706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001025172,"stat_sum":{"num_bytes":6528325,"num_objects":11,"num_object_clones":3,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":3,"num_read":1141,"num_read_kb":5971,"num_write":288,"num_write_kb":3234,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'183","reported_seq":2597,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:06.074339+0000","last_change":"2024-09-16T20:48:53.737981+0000","last_active":"2024-09-16T20:49:06.074339+0000","last_peered":"2024-09-16T20:49:06.074339+0000","last_clean":"2024-09-16T20:49:06.074339+0000","last_became_active":"2024-09-16T20:48:53.736558+0000","last_became_peered":"2024-09-16T20:48:53.736558+0000","last_unstale":"2024-09-16T20:49:06.074339+0000","last_undegraded":"2024-09-16T20:49:06.074339+0000","last_fullsized":"2024-09-16T20:49:06.074339+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:45:36.477396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044614999999999998,"stat_sum":{"num_bytes":36836368,"num_objects":24,"num_object_clones":8,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":6,"num_read":3032,"num_read_kb":13794,"num_write":180,"num_write_kb":6558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":2635264,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"75'168","reported_seq":3149,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:05.769257+0000","last_change":"2024-09-16T20:48:09.569992+0000","last_active":"2024-09-16T20:49:05.769257+0000","last_peered":"2024-09-16T20:49:05.769257+0000","last_clean":"2024-09-16T20:49:05.769257+0000","last_became_active":"2024-09-16T20:48:07.563972+0000","last_became_peered":"2024-09-16T20:48:07.563972+0000","last_unstale":"2024-09-16T20:49:05.769257+0000","last_undegraded":"2024-09-16T20:49:05.769257+0000","last_fullsized":"2024-09-16T20:49:05.769257+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:24:50.257643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039392600000000001,"stat_sum":{"num_bytes":37339949,"num_objects":32,"num_object_clones":10,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":4,"num_read":2959,"num_read_kb":24894,"num_write":319,"num_write_kb":6308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"74'169","reported_seq":2824,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:06.219650+0000","last_change":"2024-09-16T20:48:54.735384+0000","last_active":"2024-09-16T20:49:06.219650+0000","last_peered":"2024-09-16T20:49:06.219650+0000","last_clean":"2024-09-16T20:49:06.219650+0000","last_became_active":"2024-09-16T20:48:54.734409+0000","last_became_peered":"2024-09-16T20:48:54.734409+0000","last_unstale":"2024-09-16T20:49:06.219650+0000","last_undegraded":"2024-09-16T20:49:06.219650+0000","last_fullsized":"2024-09-16T20:49:06.219650+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:20:29.099217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014289540000000001,"stat_sum":{"num_bytes":36485193,"num_objects":17,"num_object_clones":4,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":4,"num_read":2733,"num_read_kb":6275,"num_write":202,"num_write_kb":4405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":971906,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"75'293","reported_seq":9213,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:06.215882+0000","last_change":"2024-09-16T20:48:54.735265+0000","last_active":"2024-09-16T20:49:06.215882+0000","last_peered":"2024-09-16T20:49:06.215882+0000","last_clean":"2024-09-16T20:49:06.215882+0000","last_became_active":"2024-09-16T20:48:54.734672+0000","last_became_peered":"2024-09-16T20:48:54.734672+0000","last_unstale":"2024-09-16T20:49:06.215882+0000","last_undegraded":"2024-09-16T20:49:06.215882+0000","last_fullsized":"2024-09-16T20:49:06.215882+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:08:19.969082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047762999999999999,"stat_sum":{"num_bytes":62220699,"num_objects":20,"num_object_clones":6,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":5,"num_read":9821,"num_read_kb":41554,"num_write":448,"num_write_kb":9855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":530734,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"75'261","reported_seq":4740,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:06.177118+0000","last_change":"2024-09-16T20:48:09.565667+0000","last_active":"2024-09-16T20:49:06.177118+0000","last_peered":"2024-09-16T20:49:06.177118+0000","last_clean":"2024-09-16T20:49:06.177118+0000","last_became_active":"2024-09-16T20:48:07.567327+0000","last_became_peered":"2024-09-16T20:48:07.567327+0000","last_unstale":"2024-09-16T20:49:06.177118+0000","last_undegraded":"2024-09-16T20:49:06.177118+0000","last_fullsized":"2024-09-16T20:49:06.177118+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":261,"log_dups_size":0,"ondisk_log_size":261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:21:33.351638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012087560000000001,"stat_sum":{"num_bytes":24419840,"num_objects":22,"num_object_clones":7,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":4,"num_read":4729,"num_read_kb":14889,"num_write":217,"num_write_kb":7270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"74'123","reported_seq":2824,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:06.215922+0000","last_change":"2024-09-16T20:48:09.565024+0000","last_active":"2024-09-16T20:49:06.215922+0000","last_peered":"2024-09-16T20:49:06.215922+0000","last_clean":"2024-09-16T20:49:06.215922+0000","last_became_active":"2024-09-16T20:48:07.570704+0000","last_became_peered":"2024-09-16T20:48:07.570704+0000","last_unstale":"2024-09-16T20:49:06.215922+0000","last_undegraded":"2024-09-16T20:49:06.215922+0000","last_fullsized":"2024-09-16T20:49:06.215922+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:53:48.266020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011026149999999999,"stat_sum":{"num_bytes":43014849,"num_objects":26,"num_object_clones":9,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":5,"num_read":2620,"num_read_kb":11764,"num_write":120,"num_write_kb":3719,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"74'189","reported_seq":3650,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:05.768845+0000","last_change":"2024-09-16T20:48:09.570786+0000","last_active":"2024-09-16T20:49:05.768845+0000","last_peered":"2024-09-16T20:49:05.768845+0000","last_clean":"2024-09-16T20:49:05.768845+0000","last_became_active":"2024-09-16T20:48:07.560783+0000","last_became_peered":"2024-09-16T20:48:07.560783+0000","last_unstale":"2024-09-16T20:49:05.768845+0000","last_undegraded":"2024-09-16T20:49:05.768845+0000","last_fullsized":"2024-09-16T20:49:05.768845+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:42:26.172477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040623999999999999,"stat_sum":{"num_bytes":22190096,"num_objects":33,"num_object_clones":12,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":8,"num_read":4314,"num_read_kb":24664,"num_write":158,"num_write_kb":8322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"74'232","reported_seq":4660,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:05.769793+0000","last_change":"2024-09-16T20:48:47.563240+0000","last_active":"2024-09-16T20:49:05.769793+0000","last_peered":"2024-09-16T20:49:05.769793+0000","last_clean":"2024-09-16T20:49:05.769793+0000","last_became_active":"2024-09-16T20:48:47.562590+0000","last_became_peered":"2024-09-16T20:48:47.562590+0000","last_unstale":"2024-09-16T20:49:05.769793+0000","last_undegraded":"2024-09-16T20:49:05.769793+0000","last_fullsized":"2024-09-16T20:49:05.769793+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:02:27.719797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115576,"stat_sum":{"num_bytes":53984315,"num_objects":34,"num_object_clones":13,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":12,"num_read":5469,"num_read_kb":30684,"num_write":189,"num_write_kb":3250,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":2587707,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"73'134","reported_seq":3460,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:05.772614+0000","last_change":"2024-09-16T20:48:09.564513+0000","last_active":"2024-09-16T20:49:05.772614+0000","last_peered":"2024-09-16T20:49:05.772614+0000","last_clean":"2024-09-16T20:49:05.772614+0000","last_became_active":"2024-09-16T20:48:07.571985+0000","last_became_peered":"2024-09-16T20:48:07.571985+0000","last_unstale":"2024-09-16T20:49:05.772614+0000","last_undegraded":"2024-09-16T20:49:05.772614+0000","last_fullsized":"2024-09-16T20:49:05.772614+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:10:02.137858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077335299999999995,"stat_sum":{"num_bytes":41333248,"num_objects":23,"num_object_clones":4,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":1,"num_read":3232,"num_read_kb":41914,"num_write":167,"num_write_kb":10470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"74'259","reported_seq":1686,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:05.768550+0000","last_change":"2024-09-16T20:48:53.735871+0000","last_active":"2024-09-16T20:49:05.768550+0000","last_peered":"2024-09-16T20:49:05.768550+0000","last_clean":"2024-09-16T20:49:05.768550+0000","last_became_active":"2024-09-16T20:48:53.734866+0000","last_became_peered":"2024-09-16T20:48:53.734866+0000","last_unstale":"2024-09-16T20:49:05.768550+0000","last_undegraded":"2024-09-16T20:49:05.768550+0000","last_fullsized":"2024-09-16T20:49:05.768550+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":259,"log_dups_size":0,"ondisk_log_size":259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:07:25.910251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000840955,"stat_sum":{"num_bytes":15595599,"num_objects":17,"num_object_clones":5,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":3,"num_read":1898,"num_read_kb":7035,"num_write":350,"num_write_kb":1731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":756384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"74'237","reported_seq":3258,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T20:49:05.769090+0000","last_change":"2024-09-16T20:48:09.565356+0000","last_active":"2024-09-16T20:49:05.769090+0000","last_peered":"2024-09-16T20:49:05.769090+0000","last_clean":"2024-09-16T20:49:05.769090+0000","last_became_active":"2024-09-16T20:48:07.565717+0000","last_became_peered":"2024-09-16T20:48:07.565717+0000","last_unstale":"2024-09-16T20:49:05.769090+0000","last_undegraded":"2024-09-16T20:49:05.769090+0000","last_fullsized":"2024-09-16T20:49:05.769090+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":237,"log_dups_size":0,"ondisk_log_size":237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:00:49.876591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013583239999999999,"stat_sum":{"num_bytes":22095958,"num_objects":17,"num_object_clones":3,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":3,"num_read":3619,"num_read_kb":13012,"num_write":362,"num_write_kb":6708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":932712780,"num_objects":744,"num_object_clones":216,"num_object_copies":1488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":744,"num_whiteouts":156,"num_read":128720,"num_read_kb":558428,"num_write":9232,"num_write_kb":150967,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1007,"num_bytes_recovered":31195153,"num_keys_recovered":234,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":201371648,"data_stored":227852367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7472,"ondisk_log_size":7472,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":0,"seq":55834574867,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62392,"kb_used_data":35364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94309448,"statfs":{"total":96636764160,"available":96572874752,"internally_reserved":0,"allocated":36212736,"data_stored":39099286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72920,"kb_used_data":45888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94298920,"statfs":{"total":96636764160,"available":96562094080,"internally_reserved":0,"allocated":46989312,"data_stored":52574072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73564,"kb_used_data":42376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94298276,"statfs":{"total":96636764160,"available":96561434624,"internally_reserved":0,"allocated":43393024,"data_stored":48267306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100908,"kb_used_data":51672,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94270932,"statfs":{"total":96636764160,"available":96533434368,"internally_reserved":0,"allocated":52912128,"data_stored":60498315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640274,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56776,"kb_used_data":17272,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94315064,"statfs":{"total":96636764160,"available":96578625536,"internally_reserved":0,"allocated":17686528,"data_stored":19010783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":52047872,"data_stored":60153666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":41971712,"data_stored":47345821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45547520,"data_stored":51644353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":35356672,"data_stored":38754672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":18169856,"data_stored":19992353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8278016,"data_stored":9961502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T20:49:07.819 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-16T20:49:07.819 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T20:49:07.819 INFO:tasks.thrashosds.thrasher:Adding osd 5 2024-09-16T20:49:07.819 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 5 2024-09-16T20:49:07.826 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:07.826+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:07.911 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:07.911 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:07.921 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:49:07.964 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:07.963+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:08.028 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:08.026+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:08.049 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:08.049 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:08.067 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:49:08.076 INFO:teuthology.orchestra.run.smithi139.stdout:2715 punch from 0x7b5c00 to 0x7b8200, (0x2600 bytes) 2024-09-16T20:49:08.077 INFO:teuthology.orchestra.run.smithi139.stdout:2716 punch from 0x6245a00 to 0x624c200, (0x6800 bytes) 2024-09-16T20:49:08.077 INFO:teuthology.orchestra.run.smithi139.stdout:2717 write 0xd0d0c00 thru 0xd0d5fab (0x53ac bytes) 2024-09-16T20:49:08.077 INFO:teuthology.orchestra.run.smithi139.stdout:2720 punch from 0x55c8c00 to 0x55cac00, (0x2000 bytes) 2024-09-16T20:49:08.078 INFO:teuthology.orchestra.run.smithi139.stdout:2721 write 0xb0fe800 thru 0xb10b306 (0xcb07 bytes) 2024-09-16T20:49:08.078 INFO:teuthology.orchestra.run.smithi139.stdout:2722 read 0x9402c00 thru 0x940a435 (0x7836 bytes) 2024-09-16T20:49:08.078 INFO:teuthology.orchestra.run.smithi139.stdout:2723 read 0xab4ea00 thru 0xab5b7b7 (0xcdb8 bytes) 2024-09-16T20:49:08.078 INFO:teuthology.orchestra.run.smithi139.stdout:2724 read 0xcf87600 thru 0xcf8fddd (0x87de bytes) 2024-09-16T20:49:08.078 INFO:teuthology.orchestra.run.smithi139.stdout:2726 write 0x75e7000 thru 0x75e7892 (0x893 bytes) 2024-09-16T20:49:08.078 INFO:teuthology.orchestra.run.smithi139.stdout:2728 read 0x93a6400 thru 0x93ac3da (0x5fdb bytes) 2024-09-16T20:49:08.079 INFO:teuthology.orchestra.run.smithi139.stdout:2729 write 0xd226200 thru 0xd2356ea (0xf4eb bytes) 2024-09-16T20:49:08.079 INFO:teuthology.orchestra.run.smithi139.stdout:2731 write 0x6744400 thru 0x6750d98 (0xc999 bytes) 2024-09-16T20:49:08.079 INFO:teuthology.orchestra.run.smithi139.stdout:2732 punch from 0xc211e00 to 0xc216200, (0x4400 bytes) 2024-09-16T20:49:08.079 INFO:teuthology.orchestra.run.smithi139.stdout:2733 write 0x5dfe800 thru 0x5e0730a (0x8b0b bytes) 2024-09-16T20:49:08.079 INFO:teuthology.orchestra.run.smithi139.stdout:2734 read 0xbc0ca00 thru 0xbc1862e (0xbc2f bytes) 2024-09-16T20:49:08.083 INFO:teuthology.orchestra.run.smithi139.stdout:2741 trunc from 0xed89200 to 0xbe6be00 2024-09-16T20:49:08.102 INFO:teuthology.orchestra.run.smithi139.stdout:2742 write 0x6c3f400 thru 0x6c42596 (0x3197 bytes) 2024-09-16T20:49:08.104 INFO:teuthology.orchestra.run.smithi139.stdout:2743 read 0xb293000 thru 0xb29971d (0x671e bytes) 2024-09-16T20:49:08.104 INFO:teuthology.orchestra.run.smithi139.stdout:2744 read 0x1e84a00 thru 0x1e851a5 (0x7a6 bytes) 2024-09-16T20:49:08.104 INFO:teuthology.orchestra.run.smithi139.stdout:2745 write 0x10fca00 thru 0x110bc49 (0xf24a bytes) 2024-09-16T20:49:08.105 INFO:teuthology.orchestra.run.smithi139.stdout:2746 punch from 0x7beb200 to 0x7bf9000, (0xde00 bytes) 2024-09-16T20:49:08.105 INFO:teuthology.orchestra.run.smithi139.stdout:2748 read 0x336a000 thru 0x3378ed3 (0xeed4 bytes) 2024-09-16T20:49:08.106 INFO:teuthology.orchestra.run.smithi139.stdout:2751 read 0x5644000 thru 0x564b65d (0x765e bytes) 2024-09-16T20:49:08.107 INFO:teuthology.orchestra.run.smithi139.stdout:2753 write 0x9c6fe00 thru 0x9c759fe (0x5bff bytes) 2024-09-16T20:49:08.107 INFO:teuthology.orchestra.run.smithi139.stdout:2755 read 0x5e87c00 thru 0x5e93eed (0xc2ee bytes) 2024-09-16T20:49:08.122 INFO:teuthology.orchestra.run.smithi139.stdout:2756 write 0xeb92e00 thru 0xeba25d1 (0xf7d2 bytes) 2024-09-16T20:49:08.123 INFO:teuthology.orchestra.run.smithi139.stdout:2757 punch from 0xe6a00 to 0xf1000, (0xa600 bytes) 2024-09-16T20:49:08.125 INFO:teuthology.orchestra.run.smithi139.stdout:2758 write 0x5b2c800 thru 0x5b2f184 (0x2985 bytes) 2024-09-16T20:49:08.126 INFO:teuthology.orchestra.run.smithi139.stdout:2760 write 0xc385200 thru 0xc3858f5 (0x6f6 bytes) 2024-09-16T20:49:08.126 INFO:teuthology.orchestra.run.smithi139.stdout:2763 write 0xe5a6400 thru 0xe5afd63 (0x9964 bytes) 2024-09-16T20:49:08.126 INFO:teuthology.orchestra.run.smithi139.stdout:2765 punch from 0x363e800 to 0x363fc00, (0x1400 bytes) 2024-09-16T20:49:08.128 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:08.128+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:08.133 INFO:teuthology.orchestra.run.smithi139.stdout:2767 trunc from 0xeba2600 to 0x94e8e00 2024-09-16T20:49:08.150 INFO:teuthology.orchestra.run.smithi139.stdout:2770 read 0x3154600 thru 0x31639e9 (0xf3ea bytes) 2024-09-16T20:49:08.151 INFO:teuthology.orchestra.run.smithi139.stdout:2771 read 0x3145400 thru 0x314e982 (0x9583 bytes) 2024-09-16T20:49:08.151 INFO:teuthology.orchestra.run.smithi139.stdout:2772 punch from 0x60c7e00 to 0x60cf800, (0x7a00 bytes) 2024-09-16T20:49:08.152 INFO:teuthology.orchestra.run.smithi139.stdout:2775 punch from 0x3a6400 to 0x3afe00, (0x9a00 bytes) 2024-09-16T20:49:08.152 INFO:teuthology.orchestra.run.smithi139.stdout:2776 read 0x8354e00 thru 0x83565ef (0x17f0 bytes) 2024-09-16T20:49:08.153 INFO:teuthology.orchestra.run.smithi139.stdout:2779 read 0x4bdca00 thru 0x4be51e6 (0x87e7 bytes) 2024-09-16T20:49:08.153 INFO:teuthology.orchestra.run.smithi139.stdout:2781 read 0x8d6be00 thru 0x8d7007e (0x427f bytes) 2024-09-16T20:49:08.163 INFO:teuthology.orchestra.run.smithi139.stdout:2784 write 0xa342400 thru 0xa3499fe (0x75ff bytes) 2024-09-16T20:49:08.164 INFO:teuthology.orchestra.run.smithi139.stdout:2785 punch from 0x5b6be00 to 0x5b71e00, (0x6000 bytes) 2024-09-16T20:49:08.165 INFO:teuthology.orchestra.run.smithi139.stdout:2788 read 0x8a1a000 thru 0x8a275c7 (0xd5c8 bytes) 2024-09-16T20:49:08.165 INFO:teuthology.orchestra.run.smithi139.stdout:2790 write 0x538e200 thru 0x538ee84 (0xc85 bytes) 2024-09-16T20:49:08.168 INFO:teuthology.orchestra.run.smithi139.stdout:2792 punch from 0x8bf5400 to 0x8bf7000, (0x1c00 bytes) 2024-09-16T20:49:08.168 INFO:teuthology.orchestra.run.smithi139.stdout:2795 write 0x11b1200 thru 0x11bdac8 (0xc8c9 bytes) 2024-09-16T20:49:08.169 INFO:teuthology.orchestra.run.smithi139.stdout:2796 write 0x6a0c800 thru 0x6a121f1 (0x59f2 bytes) 2024-09-16T20:49:08.170 INFO:teuthology.orchestra.run.smithi139.stdout:2797 read 0x15c800 thru 0x15cb4d (0x34e bytes) 2024-09-16T20:49:08.170 INFO:teuthology.orchestra.run.smithi139.stdout:2801 clone 22 order 18 su 2048 sc 2 2024-09-16T20:49:08.219 INFO:tasks.ceph.mon.a.smithi029.stderr:2024-09-16T20:49:08.217+0000 7fe0ce96d640 -1 mon.a@0(leader).osd e77 definitely_dead 0 2024-09-16T20:49:08.226 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:08.226 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:08.229 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:08.228+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:08.235 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:49:08.266 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T20:49:08.330 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:08.329+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:08.385 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:08.385 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:08.395 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:49:08.401 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:49:08.401 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:49:08.418 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:49:08.418 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:49:08.430 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:08.429+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:08.434 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:49:08.435 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:49:08.451 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:49:08.451 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:49:08.468 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:49:08.468 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:49:08.484 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:49:08.485 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:49:08.503 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:08.503 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:49:08.503 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:49:08.504 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:08.512 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:49:08.536 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:08.535+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:08.618 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:08.618 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:49:08.618 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:49:08.618 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:49:08.618 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:08.627 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:49:08.633 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:08.631+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:08.734 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:08.732+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:08.736 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:08.736 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:49:08.736 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:49:08.736 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:49:08.736 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.736 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6415 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3577856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:08.737 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871730+0000", 2024-09-16T20:49:08.737 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 48.862672203999999, 2024-09-16T20:49:08.737 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.04851482, 2024-09-16T20:49:08.737 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:08.737 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:08.737 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:08.737 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:08.737 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:08.737 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:08.737 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6415 2024-09-16T20:49:08.737 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.737 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:08.737 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.738 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:08.738 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871730+0000", 2024-09-16T20:49:08.738 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:08.738 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.738 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.738 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:08.738 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871730+0000", 2024-09-16T20:49:08.738 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:08.738 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.738 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.738 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:08.738 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871731+0000", 2024-09-16T20:49:08.738 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6899999999999998e-07 2024-09-16T20:49:08.738 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.739 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.739 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:08.739 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871733+0000", 2024-09-16T20:49:08.739 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0600000000000002e-06 2024-09-16T20:49:08.739 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.739 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.739 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:08.739 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871740+0000", 2024-09-16T20:49:08.739 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3749999999999997e-06 2024-09-16T20:49:08.739 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.739 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.739 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:08.739 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871749+0000", 2024-09-16T20:49:08.739 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9109999999999999e-06 2024-09-16T20:49:08.740 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.740 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.740 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:08.740 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920129+0000", 2024-09-16T20:49:08.740 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048380175999999997 2024-09-16T20:49:08.740 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.740 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.740 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:08.740 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920181+0000", 2024-09-16T20:49:08.740 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2077000000000003e-05 2024-09-16T20:49:08.740 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.740 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.740 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:08.741 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920245+0000", 2024-09-16T20:49:08.741 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3851999999999998e-05 2024-09-16T20:49:08.741 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:08.741 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:08.741 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:08.741 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.741 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.741 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6417 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3586048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:08.741 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871756+0000", 2024-09-16T20:49:08.741 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 48.862646568999999, 2024-09-16T20:49:08.741 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048628301999999998, 2024-09-16T20:49:08.741 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:08.741 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:08.741 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:08.742 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:08.742 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:08.742 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:08.742 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6417 2024-09-16T20:49:08.742 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.742 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:08.742 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.742 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:08.742 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871756+0000", 2024-09-16T20:49:08.742 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:08.742 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.742 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.742 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:08.742 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871756+0000", 2024-09-16T20:49:08.743 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:08.743 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.743 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.743 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:08.743 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871756+0000", 2024-09-16T20:49:08.743 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0100000000000005e-07 2024-09-16T20:49:08.743 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.743 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.743 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:08.743 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871758+0000", 2024-09-16T20:49:08.743 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.902e-06 2024-09-16T20:49:08.743 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.743 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.743 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:08.744 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871764+0000", 2024-09-16T20:49:08.744 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8170000000000004e-06 2024-09-16T20:49:08.744 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.744 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.744 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:08.744 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871769+0000", 2024-09-16T20:49:08.744 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6500000000000004e-06 2024-09-16T20:49:08.744 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.744 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.744 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:08.744 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920276+0000", 2024-09-16T20:49:08.744 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048507794999999999 2024-09-16T20:49:08.744 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.745 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.745 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:08.745 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920329+0000", 2024-09-16T20:49:08.745 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2534999999999998e-05 2024-09-16T20:49:08.745 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.745 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.745 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:08.745 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920384+0000", 2024-09-16T20:49:08.745 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5102000000000002e-05 2024-09-16T20:49:08.745 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:08.745 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:08.745 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:08.745 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.745 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.746 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6419 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3594240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:08.746 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871775+0000", 2024-09-16T20:49:08.746 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 48.862627392, 2024-09-16T20:49:08.746 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048754724999999999, 2024-09-16T20:49:08.746 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:08.746 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:08.746 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:08.746 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:08.746 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:08.746 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:08.746 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6419 2024-09-16T20:49:08.746 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.746 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:08.746 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.747 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:08.747 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871775+0000", 2024-09-16T20:49:08.747 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:08.747 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.747 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.747 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:08.747 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871775+0000", 2024-09-16T20:49:08.747 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:08.747 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.747 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.747 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:08.747 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871775+0000", 2024-09-16T20:49:08.747 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2500000000000001e-07 2024-09-16T20:49:08.747 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.748 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.748 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:08.748 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871777+0000", 2024-09-16T20:49:08.748 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9649999999999998e-06 2024-09-16T20:49:08.748 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.748 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.748 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:08.748 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871784+0000", 2024-09-16T20:49:08.748 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3419999999999996e-06 2024-09-16T20:49:08.748 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.748 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.748 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:08.748 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871789+0000", 2024-09-16T20:49:08.749 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1889999999999998e-06 2024-09-16T20:49:08.749 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.749 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.749 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:08.749 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920416+0000", 2024-09-16T20:49:08.749 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048627363999999999 2024-09-16T20:49:08.749 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.749 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.749 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:08.749 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920472+0000", 2024-09-16T20:49:08.749 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5429000000000001e-05 2024-09-16T20:49:08.749 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.749 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.750 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:08.750 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920530+0000", 2024-09-16T20:49:08.750 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8111e-05 2024-09-16T20:49:08.750 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:08.750 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:08.750 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:08.750 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.750 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.750 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6420 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3602432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:08.750 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871796+0000", 2024-09-16T20:49:08.750 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 48.862606479, 2024-09-16T20:49:08.750 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048857298, 2024-09-16T20:49:08.750 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:08.751 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:08.751 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:08.751 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:08.751 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:08.751 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:08.751 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6420 2024-09-16T20:49:08.751 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.751 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:08.751 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.751 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:08.751 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871796+0000", 2024-09-16T20:49:08.751 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:08.751 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.751 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.752 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:08.752 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871796+0000", 2024-09-16T20:49:08.752 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:08.752 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.752 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.752 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:08.752 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871854+0000", 2024-09-16T20:49:08.752 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8187999999999999e-05 2024-09-16T20:49:08.752 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.752 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.752 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:08.752 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871856+0000", 2024-09-16T20:49:08.752 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3549999999999999e-06 2024-09-16T20:49:08.753 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.753 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.753 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:08.753 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871866+0000", 2024-09-16T20:49:08.753 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6500000000000008e-06 2024-09-16T20:49:08.753 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.753 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.753 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:08.753 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871872+0000", 2024-09-16T20:49:08.753 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3799999999999999e-06 2024-09-16T20:49:08.753 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.753 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.753 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:08.754 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920558+0000", 2024-09-16T20:49:08.754 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048685797000000003 2024-09-16T20:49:08.754 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.754 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.754 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:08.754 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920603+0000", 2024-09-16T20:49:08.754 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4629999999999998e-05 2024-09-16T20:49:08.754 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.755 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.755 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:08.756 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920653+0000", 2024-09-16T20:49:08.756 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0297999999999999e-05 2024-09-16T20:49:08.756 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:08.756 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:08.756 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:08.756 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.756 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.756 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6421 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3610624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:08.756 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871882+0000", 2024-09-16T20:49:08.756 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 48.862520410999998, 2024-09-16T20:49:08.756 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048913381999999998, 2024-09-16T20:49:08.756 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:08.757 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:08.757 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:08.757 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:08.757 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:08.757 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:08.757 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6421 2024-09-16T20:49:08.757 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.757 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:08.757 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.757 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:08.757 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871882+0000", 2024-09-16T20:49:08.757 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:08.757 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.757 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.758 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:08.758 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871882+0000", 2024-09-16T20:49:08.758 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:08.758 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.758 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.758 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:08.758 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.873257+0000", 2024-09-16T20:49:08.758 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001374789 2024-09-16T20:49:08.758 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.758 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.758 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:08.758 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.873261+0000", 2024-09-16T20:49:08.758 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5589999999999999e-06 2024-09-16T20:49:08.758 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.759 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.759 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:08.759 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.873275+0000", 2024-09-16T20:49:08.759 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3675999999999999e-05 2024-09-16T20:49:08.759 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.759 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.759 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:08.759 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.873281+0000", 2024-09-16T20:49:08.759 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6930000000000004e-06 2024-09-16T20:49:08.759 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.759 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.759 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:08.759 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920684+0000", 2024-09-16T20:49:08.760 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047402913999999997 2024-09-16T20:49:08.760 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.760 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.760 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:08.760 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920737+0000", 2024-09-16T20:49:08.760 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3600000000000002e-05 2024-09-16T20:49:08.760 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.760 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.760 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:08.760 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920795+0000", 2024-09-16T20:49:08.760 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8150999999999999e-05 2024-09-16T20:49:08.760 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:08.760 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:08.761 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:08.761 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.761 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.761 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:7491 3.1e 3:7c48f6ae:::rbd_object_map.110e891a9b2b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:08.761 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.925865+0000", 2024-09-16T20:49:08.761 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 48.808537266999998, 2024-09-16T20:49:08.761 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048897293000000001, 2024-09-16T20:49:08.761 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:08.761 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:08.761 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:08.761 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:08.761 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:08.761 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:08.761 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 7491 2024-09-16T20:49:08.762 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.762 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:08.762 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.762 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:08.762 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925865+0000", 2024-09-16T20:49:08.762 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:08.762 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.762 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.762 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:08.762 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925865+0000", 2024-09-16T20:49:08.762 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:08.762 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.762 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.762 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:08.763 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925866+0000", 2024-09-16T20:49:08.763 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.39e-07 2024-09-16T20:49:08.763 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.763 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.763 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:08.763 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925868+0000", 2024-09-16T20:49:08.763 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.508e-06 2024-09-16T20:49:08.763 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.763 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.763 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:08.763 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925874+0000", 2024-09-16T20:49:08.763 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7919999999999998e-06 2024-09-16T20:49:08.763 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.763 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.764 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:08.764 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925879+0000", 2024-09-16T20:49:08.764 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8579999999999999e-06 2024-09-16T20:49:08.764 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.764 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.764 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:08.764 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.970875+0000", 2024-09-16T20:49:08.764 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044996412 2024-09-16T20:49:08.764 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.764 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.764 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:08.764 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.970970+0000", 2024-09-16T20:49:08.764 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4732e-05 2024-09-16T20:49:08.764 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.765 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.765 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-16T20:49:08.765 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.971275+0000", 2024-09-16T20:49:08.765 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00030464199999999999 2024-09-16T20:49:08.765 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.765 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.765 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:08.765 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.974598+0000", 2024-09-16T20:49:08.765 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0033238159999999998 2024-09-16T20:49:08.765 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.765 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.765 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:49:08.765 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.974666+0000", 2024-09-16T20:49:08.766 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7617e-05 2024-09-16T20:49:08.766 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.766 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.766 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:49:08.766 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.974718+0000", 2024-09-16T20:49:08.766 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2259000000000002e-05 2024-09-16T20:49:08.766 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.766 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.766 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:08.766 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.974762+0000", 2024-09-16T20:49:08.766 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4218000000000003e-05 2024-09-16T20:49:08.766 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:08.766 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:08.767 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:08.767 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.767 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.767 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4756 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1513472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:08.767 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689118+0000", 2024-09-16T20:49:08.767 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 46.045284848000001, 2024-09-16T20:49:08.767 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048535146000000001, 2024-09-16T20:49:08.767 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:08.767 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:08.767 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:08.767 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:08.767 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:49:08.767 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:08.767 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4756 2024-09-16T20:49:08.768 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.768 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:08.768 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.768 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:08.768 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689118+0000", 2024-09-16T20:49:08.768 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:08.768 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.768 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.768 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:08.768 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689118+0000", 2024-09-16T20:49:08.768 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:08.768 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.768 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.769 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:08.769 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689118+0000", 2024-09-16T20:49:08.769 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7099999999999997e-07 2024-09-16T20:49:08.769 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.769 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.769 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:08.769 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689120+0000", 2024-09-16T20:49:08.769 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.787e-06 2024-09-16T20:49:08.769 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.769 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.769 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:08.769 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689126+0000", 2024-09-16T20:49:08.769 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2720000000000003e-06 2024-09-16T20:49:08.769 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.770 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.770 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:08.770 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689134+0000", 2024-09-16T20:49:08.770 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2190000000000004e-06 2024-09-16T20:49:08.770 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.770 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.770 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:08.770 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737553+0000", 2024-09-16T20:49:08.770 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048418993 2024-09-16T20:49:08.770 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.770 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.770 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:08.770 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737598+0000", 2024-09-16T20:49:08.770 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4743999999999997e-05 2024-09-16T20:49:08.771 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.771 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.771 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:08.771 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737653+0000", 2024-09-16T20:49:08.771 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4759999999999997e-05 2024-09-16T20:49:08.771 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:08.771 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:08.771 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:08.771 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.771 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.771 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4758 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1521664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:08.771 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689141+0000", 2024-09-16T20:49:08.771 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 46.045261867999997, 2024-09-16T20:49:08.771 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048640741000000001, 2024-09-16T20:49:08.772 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:08.772 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:08.772 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:08.772 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:08.772 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:49:08.772 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:08.772 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4758 2024-09-16T20:49:08.772 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.772 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:08.772 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.772 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:08.772 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689141+0000", 2024-09-16T20:49:08.772 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:08.772 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.773 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.773 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:08.773 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689141+0000", 2024-09-16T20:49:08.773 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:08.773 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.773 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.773 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:08.773 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689141+0000", 2024-09-16T20:49:08.773 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8799999999999998e-07 2024-09-16T20:49:08.773 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.773 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.773 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:08.773 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689143+0000", 2024-09-16T20:49:08.774 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.945e-06 2024-09-16T20:49:08.774 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.774 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.774 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:08.774 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689149+0000", 2024-09-16T20:49:08.774 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9800000000000003e-06 2024-09-16T20:49:08.774 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.774 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.774 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:08.774 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689155+0000", 2024-09-16T20:49:08.774 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7509999999999997e-06 2024-09-16T20:49:08.774 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.774 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.774 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:08.775 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737681+0000", 2024-09-16T20:49:08.775 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048526825000000003 2024-09-16T20:49:08.775 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.775 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.775 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:08.775 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737729+0000", 2024-09-16T20:49:08.775 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7219999999999999e-05 2024-09-16T20:49:08.775 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.775 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.775 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:08.775 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737781+0000", 2024-09-16T20:49:08.775 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2632e-05 2024-09-16T20:49:08.775 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:08.775 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:08.776 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:08.776 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.776 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.776 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4760 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1529856~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:08.776 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689161+0000", 2024-09-16T20:49:08.776 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 46.045241521000001, 2024-09-16T20:49:08.776 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048743981999999998, 2024-09-16T20:49:08.776 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:08.776 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:08.776 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:08.776 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:08.776 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:49:08.776 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:08.776 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4760 2024-09-16T20:49:08.777 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.777 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:08.777 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.777 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:08.777 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689161+0000", 2024-09-16T20:49:08.777 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:08.777 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.777 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.777 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:08.777 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689161+0000", 2024-09-16T20:49:08.777 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:08.777 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.777 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.777 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:08.778 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689161+0000", 2024-09-16T20:49:08.778 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5599999999999995e-07 2024-09-16T20:49:08.778 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.778 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.778 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:08.778 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689163+0000", 2024-09-16T20:49:08.778 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.818e-06 2024-09-16T20:49:08.778 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.778 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.778 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:08.778 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689169+0000", 2024-09-16T20:49:08.778 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0809999999999999e-06 2024-09-16T20:49:08.778 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.778 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.779 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:08.779 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689174+0000", 2024-09-16T20:49:08.779 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.938e-06 2024-09-16T20:49:08.779 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.779 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.779 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:08.779 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737807+0000", 2024-09-16T20:49:08.779 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048633124 2024-09-16T20:49:08.779 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.779 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.779 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:08.779 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737851+0000", 2024-09-16T20:49:08.779 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3764000000000003e-05 2024-09-16T20:49:08.779 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.780 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.780 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:08.780 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737905+0000", 2024-09-16T20:49:08.780 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3701e-05 2024-09-16T20:49:08.780 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:08.780 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:08.780 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:08.780 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.780 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.780 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4762 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1538048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:08.780 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689182+0000", 2024-09-16T20:49:08.780 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 46.045220299, 2024-09-16T20:49:08.780 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048850275999999998, 2024-09-16T20:49:08.780 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:08.781 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:08.781 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:08.781 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:08.781 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:49:08.781 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:08.781 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4762 2024-09-16T20:49:08.781 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.781 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:08.781 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.781 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:08.781 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689182+0000", 2024-09-16T20:49:08.781 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:08.781 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.781 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.782 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:08.782 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689182+0000", 2024-09-16T20:49:08.782 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:08.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.782 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.782 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:08.782 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689182+0000", 2024-09-16T20:49:08.782 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-16T20:49:08.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.782 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.782 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:08.782 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689184+0000", 2024-09-16T20:49:08.782 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8190000000000001e-06 2024-09-16T20:49:08.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.783 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.783 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:08.783 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689191+0000", 2024-09-16T20:49:08.783 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2530000000000001e-06 2024-09-16T20:49:08.783 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.783 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.783 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:08.783 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689195+0000", 2024-09-16T20:49:08.783 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9559999999999996e-06 2024-09-16T20:49:08.783 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.783 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.783 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:08.783 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737933+0000", 2024-09-16T20:49:08.783 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048737919999999997 2024-09-16T20:49:08.784 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.784 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.784 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:08.784 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737982+0000", 2024-09-16T20:49:08.784 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8089e-05 2024-09-16T20:49:08.784 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.784 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.784 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:08.784 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738032+0000", 2024-09-16T20:49:08.784 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0868999999999997e-05 2024-09-16T20:49:08.784 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:08.784 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:08.784 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:08.784 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.785 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.785 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4764 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1546240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:08.785 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689201+0000", 2024-09-16T20:49:08.785 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 46.045200942000001, 2024-09-16T20:49:08.785 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048957550000000002, 2024-09-16T20:49:08.785 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:08.785 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:08.785 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:08.785 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:08.785 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:49:08.785 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:08.785 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4764 2024-09-16T20:49:08.785 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.786 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:08.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.786 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:08.786 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689201+0000", 2024-09-16T20:49:08.786 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:08.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.786 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:08.786 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689201+0000", 2024-09-16T20:49:08.786 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:08.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.786 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:08.786 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689202+0000", 2024-09-16T20:49:08.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4200000000000002e-07 2024-09-16T20:49:08.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.787 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.787 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:08.787 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689204+0000", 2024-09-16T20:49:08.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9750000000000001e-06 2024-09-16T20:49:08.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.787 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.787 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:08.787 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689210+0000", 2024-09-16T20:49:08.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1e-06 2024-09-16T20:49:08.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.787 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.787 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:08.788 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689215+0000", 2024-09-16T20:49:08.788 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0180000000000001e-06 2024-09-16T20:49:08.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.788 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.788 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:08.788 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738058+0000", 2024-09-16T20:49:08.788 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048843573000000001 2024-09-16T20:49:08.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.788 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.788 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:08.788 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738102+0000", 2024-09-16T20:49:08.788 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3256000000000001e-05 2024-09-16T20:49:08.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.789 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.789 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:08.789 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738159+0000", 2024-09-16T20:49:08.789 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7286e-05 2024-09-16T20:49:08.789 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:08.789 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:08.789 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:08.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.789 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.789 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4766 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1554432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:08.789 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689221+0000", 2024-09-16T20:49:08.789 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 46.045180975999997, 2024-09-16T20:49:08.789 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049064996, 2024-09-16T20:49:08.789 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:08.790 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:08.790 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:08.790 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:08.790 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:49:08.790 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:08.790 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4766 2024-09-16T20:49:08.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.790 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:08.790 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.790 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:08.790 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689221+0000", 2024-09-16T20:49:08.790 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:08.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.790 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.791 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:08.791 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689221+0000", 2024-09-16T20:49:08.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:08.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.791 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.791 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:08.791 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689222+0000", 2024-09-16T20:49:08.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.34e-07 2024-09-16T20:49:08.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.791 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.791 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:08.791 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689224+0000", 2024-09-16T20:49:08.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.846e-06 2024-09-16T20:49:08.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.792 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.792 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:08.792 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689247+0000", 2024-09-16T20:49:08.792 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3448e-05 2024-09-16T20:49:08.792 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.792 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.792 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:08.792 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689254+0000", 2024-09-16T20:49:08.792 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6969999999999996e-06 2024-09-16T20:49:08.792 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.792 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.792 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:08.792 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738188+0000", 2024-09-16T20:49:08.792 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048934520000000002 2024-09-16T20:49:08.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.793 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.793 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:08.793 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738234+0000", 2024-09-16T20:49:08.793 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5695999999999998e-05 2024-09-16T20:49:08.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.793 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.793 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:08.793 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738286+0000", 2024-09-16T20:49:08.793 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2355000000000003e-05 2024-09-16T20:49:08.793 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:08.793 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:08.793 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:08.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.794 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4768 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1562624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:08.794 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689262+0000", 2024-09-16T20:49:08.794 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 46.045140021999998, 2024-09-16T20:49:08.794 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049157157999999999, 2024-09-16T20:49:08.794 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:08.794 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:08.794 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:08.794 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:08.794 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:49:08.794 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:08.794 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4768 2024-09-16T20:49:08.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.795 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:08.795 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.795 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:08.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689262+0000", 2024-09-16T20:49:08.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:08.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.795 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.795 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:08.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689262+0000", 2024-09-16T20:49:08.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:08.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.795 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.795 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:08.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689263+0000", 2024-09-16T20:49:08.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6199999999999998e-07 2024-09-16T20:49:08.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.796 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.796 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:08.796 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689270+0000", 2024-09-16T20:49:08.796 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.798e-06 2024-09-16T20:49:08.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.796 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.796 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:08.796 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689277+0000", 2024-09-16T20:49:08.796 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0890000000000002e-06 2024-09-16T20:49:08.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.796 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.796 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:08.797 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689282+0000", 2024-09-16T20:49:08.797 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0470000000000001e-06 2024-09-16T20:49:08.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.797 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.797 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:08.797 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738319+0000", 2024-09-16T20:49:08.797 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049037139 2024-09-16T20:49:08.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.797 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.797 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:08.797 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738365+0000", 2024-09-16T20:49:08.797 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6356999999999997e-05 2024-09-16T20:49:08.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.797 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.798 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:08.798 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738420+0000", 2024-09-16T20:49:08.798 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4265999999999998e-05 2024-09-16T20:49:08.798 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:08.798 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:08.798 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:08.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.798 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.798 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4770 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1570816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:08.798 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689289+0000", 2024-09-16T20:49:08.799 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 46.045113512, 2024-09-16T20:49:08.799 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049253705000000002, 2024-09-16T20:49:08.799 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:08.799 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:08.799 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:08.799 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:08.799 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:49:08.799 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:08.799 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4770 2024-09-16T20:49:08.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.799 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:08.799 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.799 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:08.799 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689289+0000", 2024-09-16T20:49:08.800 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:08.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.800 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:08.800 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689289+0000", 2024-09-16T20:49:08.800 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:08.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.800 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:08.800 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689289+0000", 2024-09-16T20:49:08.800 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0999999999999999e-07 2024-09-16T20:49:08.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.801 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:08.801 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689291+0000", 2024-09-16T20:49:08.801 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0030000000000001e-06 2024-09-16T20:49:08.801 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.801 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.801 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:08.801 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689297+0000", 2024-09-16T20:49:08.801 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9479999999999996e-06 2024-09-16T20:49:08.801 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.801 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.801 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:08.801 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689302+0000", 2024-09-16T20:49:08.801 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7389999999999999e-06 2024-09-16T20:49:08.801 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.802 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.802 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:08.802 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738442+0000", 2024-09-16T20:49:08.802 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049140001000000003 2024-09-16T20:49:08.802 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.802 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.802 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:08.802 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738490+0000", 2024-09-16T20:49:08.802 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7787000000000002e-05 2024-09-16T20:49:08.802 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.802 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.802 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:08.802 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738543+0000", 2024-09-16T20:49:08.803 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2816999999999999e-05 2024-09-16T20:49:08.803 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:08.803 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:08.803 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:08.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.803 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.803 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:3088 3.1e 3:79e5ec01:::rbd_header.110e891a9b2b:head [watch watch cookie 140213437171488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:49:08.803 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:49:08.803 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 44.570557495000003, 2024-09-16T20:49:08.803 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.11895186200000001, 2024-09-16T20:49:08.803 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:08.803 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:08.803 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:08.803 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:08.804 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:08.804 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:08.804 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3088 2024-09-16T20:49:08.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.804 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:08.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.804 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:08.804 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:49:08.804 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:08.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.804 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:08.804 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:49:08.805 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:08.805 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.805 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.805 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:08.805 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163846+0000", 2024-09-16T20:49:08.805 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6799999999999999e-07 2024-09-16T20:49:08.805 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.805 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.805 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:08.805 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163850+0000", 2024-09-16T20:49:08.805 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.929e-06 2024-09-16T20:49:08.805 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.805 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.806 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:08.806 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163862+0000", 2024-09-16T20:49:08.806 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1972e-05 2024-09-16T20:49:08.806 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.806 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.806 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:08.806 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163867+0000", 2024-09-16T20:49:08.806 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.451e-06 2024-09-16T20:49:08.806 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.806 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.806 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:08.806 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163912+0000", 2024-09-16T20:49:08.806 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4456000000000003e-05 2024-09-16T20:49:08.806 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.806 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.807 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:08.807 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163962+0000", 2024-09-16T20:49:08.807 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0578000000000003e-05 2024-09-16T20:49:08.807 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.807 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.807 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-16T20:49:08.807 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164058+0000", 2024-09-16T20:49:08.807 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5934999999999995e-05 2024-09-16T20:49:08.807 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.807 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.807 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:49:08.807 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164570+0000", 2024-09-16T20:49:08.807 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00051241100000000005 2024-09-16T20:49:08.808 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.808 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.808 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:08.808 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282629+0000", 2024-09-16T20:49:08.808 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.118058986 2024-09-16T20:49:08.808 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.808 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.808 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:49:08.808 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282725+0000", 2024-09-16T20:49:08.808 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5063999999999996e-05 2024-09-16T20:49:08.808 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.808 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.808 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:08.808 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282797+0000", 2024-09-16T20:49:08.809 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2212000000000006e-05 2024-09-16T20:49:08.809 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:08.809 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:08.809 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:08.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.809 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.809 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:8039 3.1e 3.ac44ff9e (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:49:08.809 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:49:08.809 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.026730145999998, 2024-09-16T20:49:08.809 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0053310449999999, 2024-09-16T20:49:08.809 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:08.809 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:08.809 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:49:08.810 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:08.810 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:08.810 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:08.810 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8039 2024-09-16T20:49:08.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.810 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:08.810 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.810 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:08.810 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:49:08.810 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:08.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.810 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.810 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:08.810 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:49:08.811 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:08.811 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.811 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.811 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:08.811 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707674+0000", 2024-09-16T20:49:08.811 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6530000000000001e-06 2024-09-16T20:49:08.811 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.811 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.811 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:08.811 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707681+0000", 2024-09-16T20:49:08.811 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9639999999999997e-06 2024-09-16T20:49:08.811 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.811 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.811 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:08.811 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707700+0000", 2024-09-16T20:49:08.812 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8674000000000001e-05 2024-09-16T20:49:08.812 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.812 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.812 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:08.812 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707708+0000", 2024-09-16T20:49:08.812 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6810000000000004e-06 2024-09-16T20:49:08.812 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.812 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.812 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:08.812 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711293+0000", 2024-09-16T20:49:08.812 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035848189999999999 2024-09-16T20:49:08.812 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.812 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.813 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:49:08.813 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711301+0000", 2024-09-16T20:49:08.813 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4730000000000003e-06 2024-09-16T20:49:08.813 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.813 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.813 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:08.813 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.712990+0000", 2024-09-16T20:49:08.813 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.001689348 2024-09-16T20:49:08.813 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.813 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.813 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:08.813 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713003+0000", 2024-09-16T20:49:08.813 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3433e-05 2024-09-16T20:49:08.814 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:08.814 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:08.814 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:08.814 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.814 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.814 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:8040 3.1e 3.ac44ff9e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:49:08.814 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:49:08.814 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.026669751, 2024-09-16T20:49:08.814 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0053410780000001, 2024-09-16T20:49:08.814 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:08.814 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:08.814 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:49:08.814 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:08.814 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:08.815 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:08.815 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8040 2024-09-16T20:49:08.815 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.815 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:08.815 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.815 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:08.815 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:49:08.815 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:08.815 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.815 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.815 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:08.815 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:49:08.815 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:08.816 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.816 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.816 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:08.816 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:49:08.816 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8100000000000003e-07 2024-09-16T20:49:08.816 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.816 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.816 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:08.816 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707737+0000", 2024-09-16T20:49:08.816 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5020000000000001e-06 2024-09-16T20:49:08.816 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.816 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.816 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:08.816 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707745+0000", 2024-09-16T20:49:08.817 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2169999999999994e-06 2024-09-16T20:49:08.817 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.817 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.817 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:08.817 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707751+0000", 2024-09-16T20:49:08.817 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.263e-06 2024-09-16T20:49:08.817 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.817 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.817 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:08.817 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711336+0000", 2024-09-16T20:49:08.817 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035848360000000001 2024-09-16T20:49:08.817 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.817 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.818 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:49:08.818 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711340+0000", 2024-09-16T20:49:08.818 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1309999999999999e-06 2024-09-16T20:49:08.818 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.818 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.818 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:08.818 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713060+0000", 2024-09-16T20:49:08.818 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0017195059999999 2024-09-16T20:49:08.818 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.818 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.818 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:08.818 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713074+0000", 2024-09-16T20:49:08.818 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4042e-05 2024-09-16T20:49:08.818 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:08.819 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:08.819 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:08.819 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.819 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.819 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4927.0:4487 3.1e 3.ac44ff9e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:49:08.819 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:49:08.819 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.026149548999999, 2024-09-16T20:49:08.819 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0048840880000001, 2024-09-16T20:49:08.819 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:08.819 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:08.819 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:49:08.819 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:08.819 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4927", 2024-09-16T20:49:08.819 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:08.820 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4487 2024-09-16T20:49:08.820 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.820 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:08.820 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.820 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:08.820 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:49:08.820 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:08.820 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.820 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.820 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:08.820 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:49:08.820 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:08.820 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.821 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.821 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:08.821 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708254+0000", 2024-09-16T20:49:08.821 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1800000000000005e-07 2024-09-16T20:49:08.821 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.821 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.821 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:08.821 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708258+0000", 2024-09-16T20:49:08.821 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8689999999999997e-06 2024-09-16T20:49:08.821 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.821 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.821 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:08.821 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708267+0000", 2024-09-16T20:49:08.821 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9590000000000007e-06 2024-09-16T20:49:08.822 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.822 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.822 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:08.822 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708274+0000", 2024-09-16T20:49:08.822 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9970000000000001e-06 2024-09-16T20:49:08.822 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.822 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.822 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:08.822 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711366+0000", 2024-09-16T20:49:08.822 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030919530000000001 2024-09-16T20:49:08.822 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.822 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.822 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:49:08.822 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711372+0000", 2024-09-16T20:49:08.823 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9410000000000004e-06 2024-09-16T20:49:08.823 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.823 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.823 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:08.823 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713126+0000", 2024-09-16T20:49:08.823 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0017537249999999 2024-09-16T20:49:08.823 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.823 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.823 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:08.823 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713137+0000", 2024-09-16T20:49:08.823 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0825999999999999e-05 2024-09-16T20:49:08.823 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:08.823 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:08.824 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:08.824 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.824 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.824 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5089.0:11703 3.1b 3:dcd4684e:::rbd_data.110e747f0779.000000000000001b:head [set-alloc-hint object_size 262144 write_size 262144,write 139264~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:49:08.824 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:49:08.824 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.5964137750000003, 2024-09-16T20:49:08.824 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.084231385000000006, 2024-09-16T20:49:08.824 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:08.824 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:08.824 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:08.824 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:08.824 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5089", 2024-09-16T20:49:08.824 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:49:08.825 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 11703 2024-09-16T20:49:08.825 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.825 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:08.825 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.825 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:08.825 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:49:08.825 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:08.825 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.825 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.825 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:08.825 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:49:08.825 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:08.825 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.825 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.826 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:08.826 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:49:08.826 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2600000000000001e-07 2024-09-16T20:49:08.826 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.826 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.826 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:08.826 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137998+0000", 2024-09-16T20:49:08.826 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9549999999999998e-06 2024-09-16T20:49:08.826 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.826 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.826 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:08.826 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138008+0000", 2024-09-16T20:49:08.826 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3789999999999995e-06 2024-09-16T20:49:08.826 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.827 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.827 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:08.827 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138013+0000", 2024-09-16T20:49:08.827 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2209999999999996e-06 2024-09-16T20:49:08.827 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.827 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.828 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:08.828 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138054+0000", 2024-09-16T20:49:08.828 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1022999999999997e-05 2024-09-16T20:49:08.828 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.828 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.828 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:08.828 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138124+0000", 2024-09-16T20:49:08.828 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0234999999999994e-05 2024-09-16T20:49:08.828 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.828 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.828 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:08.828 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138197+0000", 2024-09-16T20:49:08.828 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2451000000000001e-05 2024-09-16T20:49:08.829 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.829 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.829 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:08.829 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222010+0000", 2024-09-16T20:49:08.829 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.083813433000000007 2024-09-16T20:49:08.829 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.829 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.829 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:49:08.829 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222163+0000", 2024-09-16T20:49:08.829 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015274599999999999 2024-09-16T20:49:08.829 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.829 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.829 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:49:08.830 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222197+0000", 2024-09-16T20:49:08.830 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3683000000000003e-05 2024-09-16T20:49:08.830 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.830 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.830 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:08.830 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222220+0000", 2024-09-16T20:49:08.830 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3433000000000001e-05 2024-09-16T20:49:08.830 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:08.830 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:08.830 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:08.830 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.830 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.830 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.5089.0:11702 3.15 e73/64 3:acf33c8b:::rbd_data.110e747f0779.000000000000001a:head v 73'132, mlcod=73'130)", 2024-09-16T20:49:08.831 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:49:08.831 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.5961464459999997, 2024-09-16T20:49:08.831 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.084079763000000002, 2024-09-16T20:49:08.831 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:08.831 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:08.831 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:08.831 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:08.831 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.831 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:08.831 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:49:08.831 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:08.831 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.831 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.831 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:08.832 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:49:08.832 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:08.832 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.832 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.832 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:08.832 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:49:08.832 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:49:08.832 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.832 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.832 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:08.832 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138263+0000", 2024-09-16T20:49:08.832 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1e-06 2024-09-16T20:49:08.832 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.833 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.833 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:08.833 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221589+0000", 2024-09-16T20:49:08.833 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.083326002999999996 2024-09-16T20:49:08.833 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.833 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.833 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:08.833 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221614+0000", 2024-09-16T20:49:08.833 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5593999999999999e-05 2024-09-16T20:49:08.833 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.833 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.833 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:08.833 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221714+0000", 2024-09-16T20:49:08.833 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9889000000000005e-05 2024-09-16T20:49:08.834 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.834 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.834 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:08.834 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221727+0000", 2024-09-16T20:49:08.834 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3125999999999999e-05 2024-09-16T20:49:08.834 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.834 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.834 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:49:08.834 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222294+0000", 2024-09-16T20:49:08.834 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00056712199999999998 2024-09-16T20:49:08.834 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:08.834 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:08.834 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:08.834 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222336+0000", 2024-09-16T20:49:08.835 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1470000000000001e-05 2024-09-16T20:49:08.835 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:08.835 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:08.835 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:08.835 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:08.835 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:08.835 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:08.837 INFO:teuthology.orchestra.run.smithi029.stderr:marked in osd.5. 2024-09-16T20:49:08.837 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:49:08.837 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:08.836+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:08.838 INFO:tasks.thrashosds.thrasher:Added osd 5 2024-09-16T20:49:08.937 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:08.936+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:08.984 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:08.985 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:49:08.985 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:49:08.985 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:08.992 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:49:09.077 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:09.076+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:09.139 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:09.138+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:09.139 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:09.139 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:49:09.139 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:49:09.140 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:49:09.140 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:09.148 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:49:09.240 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:09.238+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:09.296 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:09.296 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:49:09.296 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:49:09.296 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:49:09.296 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.297 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:4904 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 7309312~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:09.297 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767724+0000", 2024-09-16T20:49:09.297 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 49.526944358000002, 2024-09-16T20:49:09.297 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050270214000000001, 2024-09-16T20:49:09.297 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:09.297 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:09.297 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:09.297 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:09.297 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:09.297 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:09.297 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4904 2024-09-16T20:49:09.297 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.297 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:09.298 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.298 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:09.298 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767724+0000", 2024-09-16T20:49:09.298 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.298 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.298 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.298 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:09.298 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767724+0000", 2024-09-16T20:49:09.298 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.298 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.298 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.298 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:09.298 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767724+0000", 2024-09-16T20:49:09.298 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1399999999999998e-07 2024-09-16T20:49:09.298 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.299 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.299 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:09.299 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767725+0000", 2024-09-16T20:49:09.299 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0380000000000001e-06 2024-09-16T20:49:09.299 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.299 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.299 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:09.299 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767729+0000", 2024-09-16T20:49:09.299 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2490000000000001e-06 2024-09-16T20:49:09.299 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.299 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.299 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:09.299 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767731+0000", 2024-09-16T20:49:09.299 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5940000000000001e-06 2024-09-16T20:49:09.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.300 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.300 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:09.300 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.817787+0000", 2024-09-16T20:49:09.300 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050055792000000002 2024-09-16T20:49:09.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.300 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.300 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:09.300 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.817883+0000", 2024-09-16T20:49:09.300 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5606999999999995e-05 2024-09-16T20:49:09.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.300 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.301 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:09.301 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.817994+0000", 2024-09-16T20:49:09.301 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011162000000000001 2024-09-16T20:49:09.301 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.301 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:09.301 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.301 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.301 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.301 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:4906 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 7317504~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:09.301 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767734+0000", 2024-09-16T20:49:09.301 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 49.526934095000001, 2024-09-16T20:49:09.301 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050406608999999998, 2024-09-16T20:49:09.301 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:09.302 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:09.302 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:09.302 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:09.302 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:09.302 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:09.302 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4906 2024-09-16T20:49:09.302 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.302 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:09.302 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.302 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:09.302 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767734+0000", 2024-09-16T20:49:09.302 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.302 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.303 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.303 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:09.303 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767734+0000", 2024-09-16T20:49:09.303 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.303 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.303 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:09.303 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767735+0000", 2024-09-16T20:49:09.303 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.05e-07 2024-09-16T20:49:09.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.303 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.303 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:09.303 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767736+0000", 2024-09-16T20:49:09.303 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0240000000000001e-06 2024-09-16T20:49:09.304 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.304 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.304 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:09.304 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767739+0000", 2024-09-16T20:49:09.304 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.067e-06 2024-09-16T20:49:09.304 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.304 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.304 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:09.304 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767742+0000", 2024-09-16T20:49:09.304 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.23e-06 2024-09-16T20:49:09.304 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.304 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.304 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:09.304 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818027+0000", 2024-09-16T20:49:09.305 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050285102999999998 2024-09-16T20:49:09.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.305 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:09.305 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818072+0000", 2024-09-16T20:49:09.305 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5129000000000001e-05 2024-09-16T20:49:09.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.305 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:09.305 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818141+0000", 2024-09-16T20:49:09.305 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8850999999999995e-05 2024-09-16T20:49:09.305 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.305 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:09.306 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.306 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.306 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.306 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:4908 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 7325696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:09.306 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767745+0000", 2024-09-16T20:49:09.306 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 49.526923699999998, 2024-09-16T20:49:09.306 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050654230000000001, 2024-09-16T20:49:09.306 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:09.306 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:09.306 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:09.306 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:09.306 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:09.306 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:09.306 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4908 2024-09-16T20:49:09.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.307 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:09.307 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.307 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:09.307 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767745+0000", 2024-09-16T20:49:09.307 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.307 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.307 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:09.307 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767745+0000", 2024-09-16T20:49:09.307 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.307 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.307 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:09.308 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767745+0000", 2024-09-16T20:49:09.308 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1e-07 2024-09-16T20:49:09.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.308 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.308 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:09.308 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767749+0000", 2024-09-16T20:49:09.308 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.532e-06 2024-09-16T20:49:09.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.308 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.308 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:09.308 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767753+0000", 2024-09-16T20:49:09.308 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4939999999999999e-06 2024-09-16T20:49:09.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.308 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.309 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:09.309 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767756+0000", 2024-09-16T20:49:09.309 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.61e-06 2024-09-16T20:49:09.309 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.309 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.309 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:09.309 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818184+0000", 2024-09-16T20:49:09.309 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050428615000000003 2024-09-16T20:49:09.309 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.309 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.309 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:09.309 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818276+0000", 2024-09-16T20:49:09.309 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2077999999999995e-05 2024-09-16T20:49:09.309 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.310 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.310 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:09.310 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818399+0000", 2024-09-16T20:49:09.310 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012269099999999999 2024-09-16T20:49:09.310 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.310 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:09.310 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.310 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.310 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.310 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:4910 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 7333888~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:09.310 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767759+0000", 2024-09-16T20:49:09.310 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 49.526909809999999, 2024-09-16T20:49:09.310 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050815076000000001, 2024-09-16T20:49:09.310 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:09.311 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:09.311 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:09.311 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:09.311 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:09.311 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:09.311 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4910 2024-09-16T20:49:09.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.311 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:09.311 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.311 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:09.311 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767759+0000", 2024-09-16T20:49:09.311 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.311 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.312 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:09.312 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767759+0000", 2024-09-16T20:49:09.312 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.312 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.312 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.312 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:09.312 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767759+0000", 2024-09-16T20:49:09.312 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2600000000000001e-07 2024-09-16T20:49:09.312 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.312 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.312 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:09.312 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767760+0000", 2024-09-16T20:49:09.313 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0640000000000001e-06 2024-09-16T20:49:09.313 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.313 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.313 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:09.313 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767763+0000", 2024-09-16T20:49:09.313 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.994e-06 2024-09-16T20:49:09.313 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.313 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.313 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:09.313 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767765+0000", 2024-09-16T20:49:09.313 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5220000000000002e-06 2024-09-16T20:49:09.313 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.313 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.313 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:09.314 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818438+0000", 2024-09-16T20:49:09.314 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050672602999999997 2024-09-16T20:49:09.314 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.314 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.314 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:09.314 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818481+0000", 2024-09-16T20:49:09.314 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2967999999999999e-05 2024-09-16T20:49:09.314 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.314 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.314 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:09.314 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818574+0000", 2024-09-16T20:49:09.314 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2699000000000001e-05 2024-09-16T20:49:09.314 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.314 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:09.314 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.315 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.315 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.315 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5012 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2101248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:09.315 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767768+0000", 2024-09-16T20:49:09.315 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 49.526900161, 2024-09-16T20:49:09.315 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051084682999999999, 2024-09-16T20:49:09.315 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:09.315 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:09.315 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:09.315 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:09.315 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:09.315 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:09.315 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5012 2024-09-16T20:49:09.316 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.316 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:09.316 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.316 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:09.316 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767768+0000", 2024-09-16T20:49:09.316 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.316 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.316 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.316 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:09.316 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767768+0000", 2024-09-16T20:49:09.316 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.316 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.316 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.317 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:09.317 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767769+0000", 2024-09-16T20:49:09.317 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1500000000000001e-07 2024-09-16T20:49:09.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.317 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.317 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:09.317 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767770+0000", 2024-09-16T20:49:09.317 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.015e-06 2024-09-16T20:49:09.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.317 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.317 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:09.317 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767773+0000", 2024-09-16T20:49:09.317 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1070000000000001e-06 2024-09-16T20:49:09.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.318 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.318 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:09.318 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767775+0000", 2024-09-16T20:49:09.318 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.508e-06 2024-09-16T20:49:09.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.318 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.318 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:09.318 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818630+0000", 2024-09-16T20:49:09.318 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050855282000000002 2024-09-16T20:49:09.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.318 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.318 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:09.319 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818732+0000", 2024-09-16T20:49:09.319 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101531 2024-09-16T20:49:09.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.319 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.319 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:09.319 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818853+0000", 2024-09-16T20:49:09.319 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000121025 2024-09-16T20:49:09.319 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.319 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:09.319 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.319 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.319 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5014 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2109440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:09.320 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767778+0000", 2024-09-16T20:49:09.320 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 49.526890225000002, 2024-09-16T20:49:09.320 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051260776000000001, 2024-09-16T20:49:09.320 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:09.320 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:09.320 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:09.320 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:09.320 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:09.320 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:09.320 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5014 2024-09-16T20:49:09.320 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.320 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:09.320 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.320 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:09.321 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767778+0000", 2024-09-16T20:49:09.321 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.321 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.321 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.321 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:09.321 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767778+0000", 2024-09-16T20:49:09.321 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.321 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.321 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.321 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:09.321 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767824+0000", 2024-09-16T20:49:09.321 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5890999999999998e-05 2024-09-16T20:49:09.321 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.321 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.322 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:09.322 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767827+0000", 2024-09-16T20:49:09.322 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.644e-06 2024-09-16T20:49:09.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.322 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.322 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:09.322 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767832+0000", 2024-09-16T20:49:09.322 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2689999999999999e-06 2024-09-16T20:49:09.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.322 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.322 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:09.322 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767835+0000", 2024-09-16T20:49:09.322 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2100000000000002e-06 2024-09-16T20:49:09.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.323 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.323 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:09.323 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818887+0000", 2024-09-16T20:49:09.323 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051051276999999999 2024-09-16T20:49:09.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.323 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.323 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:09.323 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818953+0000", 2024-09-16T20:49:09.323 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6105999999999994e-05 2024-09-16T20:49:09.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.323 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.323 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:09.324 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819039+0000", 2024-09-16T20:49:09.324 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6378999999999994e-05 2024-09-16T20:49:09.324 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.324 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:09.324 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.324 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.324 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.324 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5016 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2117632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:09.324 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767841+0000", 2024-09-16T20:49:09.324 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 49.526827546, 2024-09-16T20:49:09.324 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051433288000000001, 2024-09-16T20:49:09.324 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:09.324 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:09.324 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:09.325 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:09.325 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:09.325 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:09.325 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5016 2024-09-16T20:49:09.325 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.325 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:09.325 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.325 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:09.325 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767841+0000", 2024-09-16T20:49:09.325 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.325 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.325 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.325 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:09.326 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767841+0000", 2024-09-16T20:49:09.326 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.326 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.326 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.326 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:09.326 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768156+0000", 2024-09-16T20:49:09.326 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00031505699999999999 2024-09-16T20:49:09.326 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.326 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.326 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:09.326 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768159+0000", 2024-09-16T20:49:09.326 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1659999999999998e-06 2024-09-16T20:49:09.326 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.326 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.327 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:09.327 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768166+0000", 2024-09-16T20:49:09.327 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0550000000000002e-06 2024-09-16T20:49:09.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.327 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.327 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:09.327 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768171+0000", 2024-09-16T20:49:09.327 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3170000000000003e-06 2024-09-16T20:49:09.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.327 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.328 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:09.328 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819084+0000", 2024-09-16T20:49:09.328 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050913355 2024-09-16T20:49:09.328 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.328 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.328 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:09.328 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819170+0000", 2024-09-16T20:49:09.328 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6328000000000004e-05 2024-09-16T20:49:09.328 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.329 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.329 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:09.329 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819274+0000", 2024-09-16T20:49:09.329 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010401 2024-09-16T20:49:09.329 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.329 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:09.329 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.330 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.330 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.330 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5018 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2125824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:09.330 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768178+0000", 2024-09-16T20:49:09.330 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 49.526490011999996, 2024-09-16T20:49:09.330 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051227013000000002, 2024-09-16T20:49:09.330 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:09.330 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:09.330 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:09.330 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:09.330 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:09.330 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:09.330 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5018 2024-09-16T20:49:09.330 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.331 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:09.331 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.331 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:09.331 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768178+0000", 2024-09-16T20:49:09.331 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.331 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.331 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.331 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:09.331 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768178+0000", 2024-09-16T20:49:09.331 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.331 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.331 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.331 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:09.332 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768179+0000", 2024-09-16T20:49:09.332 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9200000000000002e-07 2024-09-16T20:49:09.332 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.332 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.332 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:09.332 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768180+0000", 2024-09-16T20:49:09.332 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3790000000000001e-06 2024-09-16T20:49:09.332 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.332 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.332 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:09.332 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768184+0000", 2024-09-16T20:49:09.332 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5470000000000001e-06 2024-09-16T20:49:09.332 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.332 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.333 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:09.333 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768187+0000", 2024-09-16T20:49:09.333 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8119999999999999e-06 2024-09-16T20:49:09.333 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.333 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.333 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:09.333 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819305+0000", 2024-09-16T20:49:09.333 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051118683999999998 2024-09-16T20:49:09.333 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.333 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.333 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:09.333 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819351+0000", 2024-09-16T20:49:09.333 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5757999999999998e-05 2024-09-16T20:49:09.334 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.334 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.334 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:09.334 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819405+0000", 2024-09-16T20:49:09.334 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4441000000000003e-05 2024-09-16T20:49:09.334 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.334 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:09.334 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.334 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.334 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.334 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5020 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2134016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:09.334 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768190+0000", 2024-09-16T20:49:09.334 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 49.526478333999997, 2024-09-16T20:49:09.334 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051449661000000001, 2024-09-16T20:49:09.335 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:09.335 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:09.335 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:09.335 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:09.335 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:09.335 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:09.335 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5020 2024-09-16T20:49:09.335 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.335 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:09.335 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.335 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:09.335 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768190+0000", 2024-09-16T20:49:09.335 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.336 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.336 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:09.336 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768190+0000", 2024-09-16T20:49:09.336 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.336 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.336 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:09.336 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768282+0000", 2024-09-16T20:49:09.336 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2263999999999996e-05 2024-09-16T20:49:09.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.336 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.336 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:09.336 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768286+0000", 2024-09-16T20:49:09.337 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.529e-06 2024-09-16T20:49:09.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.337 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.337 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:09.337 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768294+0000", 2024-09-16T20:49:09.337 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6990000000000003e-06 2024-09-16T20:49:09.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.337 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.337 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:09.337 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768306+0000", 2024-09-16T20:49:09.337 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2112e-05 2024-09-16T20:49:09.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.337 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.338 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:09.338 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819449+0000", 2024-09-16T20:49:09.338 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051143521999999997 2024-09-16T20:49:09.338 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.338 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.338 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:09.338 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819539+0000", 2024-09-16T20:49:09.338 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0022e-05 2024-09-16T20:49:09.338 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.339 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.339 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:09.339 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819640+0000", 2024-09-16T20:49:09.339 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000100513 2024-09-16T20:49:09.339 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.339 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:09.339 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.339 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.339 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.339 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5022 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2142208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:09.339 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768326+0000", 2024-09-16T20:49:09.339 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 49.526342945000003, 2024-09-16T20:49:09.339 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051436952000000001, 2024-09-16T20:49:09.340 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:09.340 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:09.340 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:09.340 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:09.340 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:09.340 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:09.340 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5022 2024-09-16T20:49:09.340 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.340 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:09.340 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.340 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:09.340 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768326+0000", 2024-09-16T20:49:09.340 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.341 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.341 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.341 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:09.341 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768326+0000", 2024-09-16T20:49:09.341 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.341 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.341 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.341 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:09.341 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768477+0000", 2024-09-16T20:49:09.341 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015109300000000001 2024-09-16T20:49:09.341 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.341 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.341 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:09.342 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768480+0000", 2024-09-16T20:49:09.342 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8580000000000002e-06 2024-09-16T20:49:09.342 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.342 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.342 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:09.342 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768489+0000", 2024-09-16T20:49:09.342 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4600000000000003e-06 2024-09-16T20:49:09.342 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.342 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.342 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:09.342 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768493+0000", 2024-09-16T20:49:09.342 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2270000000000004e-06 2024-09-16T20:49:09.342 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.342 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.343 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:09.343 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819672+0000", 2024-09-16T20:49:09.343 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051178605000000002 2024-09-16T20:49:09.343 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.343 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.343 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:09.343 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819711+0000", 2024-09-16T20:49:09.343 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9357000000000003e-05 2024-09-16T20:49:09.343 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.343 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.343 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:09.343 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819762+0000", 2024-09-16T20:49:09.343 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1351999999999999e-05 2024-09-16T20:49:09.343 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.344 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:09.344 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.344 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.344 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.344 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5024 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2150400~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:09.344 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768501+0000", 2024-09-16T20:49:09.344 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 49.526167467, 2024-09-16T20:49:09.344 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051402678, 2024-09-16T20:49:09.344 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:09.344 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:09.345 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:09.345 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:09.345 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:09.345 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:09.345 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5024 2024-09-16T20:49:09.345 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.345 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:09.345 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.345 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:09.345 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768501+0000", 2024-09-16T20:49:09.345 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.345 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.345 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.346 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:09.346 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768501+0000", 2024-09-16T20:49:09.346 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.346 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.346 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.346 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:09.346 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768619+0000", 2024-09-16T20:49:09.346 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118187 2024-09-16T20:49:09.346 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.347 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.347 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:09.347 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768624+0000", 2024-09-16T20:49:09.347 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3259999999999997e-06 2024-09-16T20:49:09.347 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.347 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.347 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:09.347 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768631+0000", 2024-09-16T20:49:09.347 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6490000000000008e-06 2024-09-16T20:49:09.347 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.347 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.347 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:09.347 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768637+0000", 2024-09-16T20:49:09.347 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4210000000000003e-06 2024-09-16T20:49:09.348 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.348 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.348 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:09.348 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819800+0000", 2024-09-16T20:49:09.348 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051163486000000001 2024-09-16T20:49:09.348 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.348 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.348 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:09.348 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819845+0000", 2024-09-16T20:49:09.348 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4830999999999998e-05 2024-09-16T20:49:09.348 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.348 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.348 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:09.349 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819904+0000", 2024-09-16T20:49:09.349 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8777999999999999e-05 2024-09-16T20:49:09.349 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.349 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:09.349 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.349 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.349 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5026 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2158592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:09.349 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768646+0000", 2024-09-16T20:49:09.349 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 49.526022019000003, 2024-09-16T20:49:09.349 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051384386999999997, 2024-09-16T20:49:09.349 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:09.349 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:09.350 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:09.350 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:09.350 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:09.350 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:09.350 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5026 2024-09-16T20:49:09.350 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.350 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:09.350 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.350 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:09.350 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768646+0000", 2024-09-16T20:49:09.350 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.350 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.350 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.350 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:09.350 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768646+0000", 2024-09-16T20:49:09.351 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.351 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.351 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.351 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:09.351 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768775+0000", 2024-09-16T20:49:09.351 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012813200000000001 2024-09-16T20:49:09.351 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.351 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.351 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:09.351 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768778+0000", 2024-09-16T20:49:09.351 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2150000000000001e-06 2024-09-16T20:49:09.351 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.351 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.352 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:09.352 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768786+0000", 2024-09-16T20:49:09.352 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3259999999999996e-06 2024-09-16T20:49:09.352 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.352 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.352 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:09.352 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768790+0000", 2024-09-16T20:49:09.352 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.216e-06 2024-09-16T20:49:09.352 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.352 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.352 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:09.352 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819939+0000", 2024-09-16T20:49:09.352 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051149018999999997 2024-09-16T20:49:09.353 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.353 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.353 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:09.353 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819979+0000", 2024-09-16T20:49:09.353 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9311999999999999e-05 2024-09-16T20:49:09.353 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.353 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.353 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:09.353 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820031+0000", 2024-09-16T20:49:09.353 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2166999999999997e-05 2024-09-16T20:49:09.353 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.353 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:09.353 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.353 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.354 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.354 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5028 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2166784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:09.354 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768798+0000", 2024-09-16T20:49:09.354 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 49.525870376, 2024-09-16T20:49:09.354 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051388912000000002, 2024-09-16T20:49:09.354 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:09.354 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:09.354 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:09.354 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:09.354 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:09.354 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:09.354 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5028 2024-09-16T20:49:09.354 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.354 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:09.355 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.355 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:09.355 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768798+0000", 2024-09-16T20:49:09.355 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.355 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.355 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.355 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:09.355 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768798+0000", 2024-09-16T20:49:09.355 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.355 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.355 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.355 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:09.355 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768989+0000", 2024-09-16T20:49:09.355 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000190951 2024-09-16T20:49:09.356 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.356 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.356 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:09.356 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768993+0000", 2024-09-16T20:49:09.356 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9770000000000002e-06 2024-09-16T20:49:09.356 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.356 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.356 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:09.356 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.769002+0000", 2024-09-16T20:49:09.356 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5469999999999997e-06 2024-09-16T20:49:09.356 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.356 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.356 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:09.357 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.769020+0000", 2024-09-16T20:49:09.357 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8216999999999999e-05 2024-09-16T20:49:09.357 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.357 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.357 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:09.357 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820070+0000", 2024-09-16T20:49:09.357 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051049998999999999 2024-09-16T20:49:09.357 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.357 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.357 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:09.357 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820119+0000", 2024-09-16T20:49:09.357 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8962000000000003e-05 2024-09-16T20:49:09.357 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.358 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.358 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:09.358 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820187+0000", 2024-09-16T20:49:09.358 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8258999999999998e-05 2024-09-16T20:49:09.358 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.358 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:09.358 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.358 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.358 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.358 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5030 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2174976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:09.358 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.769031+0000", 2024-09-16T20:49:09.358 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 49.525637263, 2024-09-16T20:49:09.358 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051289830000000002, 2024-09-16T20:49:09.358 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:09.359 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:09.359 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:09.359 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:09.359 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:09.359 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:09.359 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5030 2024-09-16T20:49:09.359 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.359 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:09.359 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.359 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:09.359 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.769031+0000", 2024-09-16T20:49:09.359 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.359 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.359 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.360 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:09.360 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.769031+0000", 2024-09-16T20:49:09.360 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.360 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.360 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.360 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:09.360 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.773332+0000", 2024-09-16T20:49:09.360 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0043004039999999999 2024-09-16T20:49:09.360 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.360 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.360 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:09.360 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.773336+0000", 2024-09-16T20:49:09.360 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7849999999999999e-06 2024-09-16T20:49:09.361 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.361 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.361 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:09.361 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.773347+0000", 2024-09-16T20:49:09.361 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0482999999999999e-05 2024-09-16T20:49:09.361 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.361 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.361 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:09.361 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.773354+0000", 2024-09-16T20:49:09.361 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1559999999999998e-06 2024-09-16T20:49:09.361 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.361 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.361 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:09.361 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820216+0000", 2024-09-16T20:49:09.362 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046861591000000001 2024-09-16T20:49:09.362 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.362 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.362 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:09.362 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820258+0000", 2024-09-16T20:49:09.362 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2653e-05 2024-09-16T20:49:09.362 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.362 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.362 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:09.363 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820321+0000", 2024-09-16T20:49:09.363 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2757999999999998e-05 2024-09-16T20:49:09.363 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.363 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:09.363 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.363 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.363 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.363 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5067 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 6918144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:09.363 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.775279+0000", 2024-09-16T20:49:09.363 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 49.519388997, 2024-09-16T20:49:09.363 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050171387999999997, 2024-09-16T20:49:09.363 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:09.363 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:09.363 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:09.364 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:09.364 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:09.364 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:09.364 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5067 2024-09-16T20:49:09.364 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.364 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:09.364 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.364 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:09.364 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775279+0000", 2024-09-16T20:49:09.364 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.364 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.364 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.364 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:09.365 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775279+0000", 2024-09-16T20:49:09.365 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.365 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.365 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.365 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:09.365 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775432+0000", 2024-09-16T20:49:09.365 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015257399999999999 2024-09-16T20:49:09.365 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.365 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.365 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:09.365 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775436+0000", 2024-09-16T20:49:09.365 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0180000000000003e-06 2024-09-16T20:49:09.365 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.365 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.366 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:09.366 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775447+0000", 2024-09-16T20:49:09.366 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.077e-05 2024-09-16T20:49:09.366 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.366 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.366 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:09.366 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775456+0000", 2024-09-16T20:49:09.366 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4089999999999992e-06 2024-09-16T20:49:09.366 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.366 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.366 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:09.366 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825150+0000", 2024-09-16T20:49:09.366 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049693711000000002 2024-09-16T20:49:09.366 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.367 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.367 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:09.367 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825337+0000", 2024-09-16T20:49:09.367 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000187214 2024-09-16T20:49:09.367 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.367 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.367 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:09.367 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825451+0000", 2024-09-16T20:49:09.367 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011369199999999999 2024-09-16T20:49:09.367 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.367 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:09.367 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.367 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.367 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.368 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5068 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2330624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:09.368 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.775469+0000", 2024-09-16T20:49:09.368 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 49.519199036000003, 2024-09-16T20:49:09.368 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050198334999999997, 2024-09-16T20:49:09.368 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:09.368 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:09.368 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:09.368 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:09.368 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:09.368 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:09.368 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5068 2024-09-16T20:49:09.368 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.368 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:09.369 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.369 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:09.369 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775469+0000", 2024-09-16T20:49:09.369 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.369 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.369 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.369 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:09.369 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775469+0000", 2024-09-16T20:49:09.369 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.369 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.369 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.369 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:09.369 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775720+0000", 2024-09-16T20:49:09.369 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000250661 2024-09-16T20:49:09.370 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.370 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.370 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:09.370 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775724+0000", 2024-09-16T20:49:09.370 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8700000000000002e-06 2024-09-16T20:49:09.370 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.370 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.370 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:09.370 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775733+0000", 2024-09-16T20:49:09.370 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4239999999999999e-06 2024-09-16T20:49:09.370 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.370 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.370 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:09.371 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775740+0000", 2024-09-16T20:49:09.371 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9519999999999997e-06 2024-09-16T20:49:09.371 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.371 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.371 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:09.371 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825484+0000", 2024-09-16T20:49:09.371 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049743385000000001 2024-09-16T20:49:09.371 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.371 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.371 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:09.371 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825572+0000", 2024-09-16T20:49:09.371 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7834e-05 2024-09-16T20:49:09.371 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.371 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.372 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:09.372 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825668+0000", 2024-09-16T20:49:09.372 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6209e-05 2024-09-16T20:49:09.372 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.372 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:09.372 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.372 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.372 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.372 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5069 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 6926336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:09.372 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.775753+0000", 2024-09-16T20:49:09.372 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 49.518915813, 2024-09-16T20:49:09.372 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050146430999999998, 2024-09-16T20:49:09.372 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:09.372 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:09.373 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:09.373 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:09.373 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:09.373 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:09.373 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5069 2024-09-16T20:49:09.373 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.373 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:09.373 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.373 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:09.373 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775753+0000", 2024-09-16T20:49:09.373 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.373 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.373 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.374 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:09.374 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775753+0000", 2024-09-16T20:49:09.374 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.374 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.374 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.374 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:09.374 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775865+0000", 2024-09-16T20:49:09.374 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000112402 2024-09-16T20:49:09.374 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.374 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.374 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:09.374 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775869+0000", 2024-09-16T20:49:09.374 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5130000000000001e-06 2024-09-16T20:49:09.374 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.375 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.375 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:09.375 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775877+0000", 2024-09-16T20:49:09.375 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1359999999999997e-06 2024-09-16T20:49:09.375 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.375 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.375 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:09.375 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775882+0000", 2024-09-16T20:49:09.375 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6710000000000004e-06 2024-09-16T20:49:09.375 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.375 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.375 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:09.375 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825729+0000", 2024-09-16T20:49:09.375 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049846898000000001 2024-09-16T20:49:09.376 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.376 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.376 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:09.376 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825807+0000", 2024-09-16T20:49:09.376 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7875999999999998e-05 2024-09-16T20:49:09.376 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.376 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.376 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:09.376 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825899+0000", 2024-09-16T20:49:09.376 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1935000000000007e-05 2024-09-16T20:49:09.376 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.376 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:09.376 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.377 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.377 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.377 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5070 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2338816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:09.377 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.775892+0000", 2024-09-16T20:49:09.377 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 49.518775959000003, 2024-09-16T20:49:09.377 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050155311000000001, 2024-09-16T20:49:09.377 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:09.377 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:09.377 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:09.377 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:09.377 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:09.377 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:09.377 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5070 2024-09-16T20:49:09.377 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.378 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:09.378 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.378 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:09.378 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775892+0000", 2024-09-16T20:49:09.378 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.378 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.378 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.378 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:09.378 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775892+0000", 2024-09-16T20:49:09.378 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.378 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.378 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.378 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:09.378 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776142+0000", 2024-09-16T20:49:09.379 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00024950500000000002 2024-09-16T20:49:09.379 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.379 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.379 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:09.379 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776146+0000", 2024-09-16T20:49:09.379 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2819999999999998e-06 2024-09-16T20:49:09.379 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.379 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.379 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:09.379 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776157+0000", 2024-09-16T20:49:09.379 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0807e-05 2024-09-16T20:49:09.379 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.379 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.380 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:09.380 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776165+0000", 2024-09-16T20:49:09.380 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1300000000000001e-06 2024-09-16T20:49:09.380 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.380 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.380 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:09.380 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825935+0000", 2024-09-16T20:49:09.380 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049769300000000002 2024-09-16T20:49:09.380 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.380 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.380 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:09.380 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825991+0000", 2024-09-16T20:49:09.380 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6177000000000001e-05 2024-09-16T20:49:09.380 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.381 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.381 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:09.381 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826048+0000", 2024-09-16T20:49:09.381 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.711e-05 2024-09-16T20:49:09.381 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.381 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:09.381 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.381 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.381 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.381 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5071 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 6934528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:09.381 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.776177+0000", 2024-09-16T20:49:09.381 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 49.518491556000001, 2024-09-16T20:49:09.381 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050036886000000003, 2024-09-16T20:49:09.382 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:09.382 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:09.382 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:09.382 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:09.382 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:09.382 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:09.382 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5071 2024-09-16T20:49:09.382 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.382 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:09.382 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.382 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:09.382 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776177+0000", 2024-09-16T20:49:09.382 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.382 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.383 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.383 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:09.383 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776177+0000", 2024-09-16T20:49:09.383 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.383 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.383 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.383 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:09.383 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776339+0000", 2024-09-16T20:49:09.383 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016238099999999999 2024-09-16T20:49:09.383 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.383 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.383 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:09.383 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776343+0000", 2024-09-16T20:49:09.383 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9670000000000003e-06 2024-09-16T20:49:09.383 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.384 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.384 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:09.384 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776354+0000", 2024-09-16T20:49:09.384 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0789e-05 2024-09-16T20:49:09.384 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.384 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.384 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:09.384 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776361+0000", 2024-09-16T20:49:09.384 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5200000000000003e-06 2024-09-16T20:49:09.384 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.384 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.384 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:09.384 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826078+0000", 2024-09-16T20:49:09.385 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049717513999999997 2024-09-16T20:49:09.385 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.385 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.385 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:09.385 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826141+0000", 2024-09-16T20:49:09.385 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2446000000000005e-05 2024-09-16T20:49:09.385 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.385 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.385 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:09.385 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826214+0000", 2024-09-16T20:49:09.385 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3269000000000006e-05 2024-09-16T20:49:09.385 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.385 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:09.386 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.386 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.386 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.386 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5072 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2347008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:09.386 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.776370+0000", 2024-09-16T20:49:09.386 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 49.518297994999998, 2024-09-16T20:49:09.386 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050097653999999998, 2024-09-16T20:49:09.386 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:09.386 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:09.386 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:09.386 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:09.386 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:09.386 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:09.386 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5072 2024-09-16T20:49:09.387 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.387 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:09.387 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.387 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:09.387 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776370+0000", 2024-09-16T20:49:09.387 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.387 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.387 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.387 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:09.387 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776370+0000", 2024-09-16T20:49:09.387 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.387 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.387 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.388 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:09.388 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776645+0000", 2024-09-16T20:49:09.388 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000274634 2024-09-16T20:49:09.388 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.388 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.388 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:09.388 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776654+0000", 2024-09-16T20:49:09.388 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9390000000000003e-06 2024-09-16T20:49:09.388 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.388 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.388 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:09.388 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776665+0000", 2024-09-16T20:49:09.388 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0524e-05 2024-09-16T20:49:09.388 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.389 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.389 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:09.389 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776671+0000", 2024-09-16T20:49:09.389 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7390000000000002e-06 2024-09-16T20:49:09.389 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.389 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.389 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:09.389 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826261+0000", 2024-09-16T20:49:09.389 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049589995999999997 2024-09-16T20:49:09.389 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.389 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.389 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:09.389 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826363+0000", 2024-09-16T20:49:09.390 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010158999999999999 2024-09-16T20:49:09.390 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.390 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.390 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:09.390 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826468+0000", 2024-09-16T20:49:09.390 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000105232 2024-09-16T20:49:09.390 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.390 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:09.390 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.390 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.390 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:09.390 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:09.392 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:49:09.392 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:09.390+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:09.493 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:09.491+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:09.500 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:09.500 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:49:09.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.500 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5286.0:10643 3.17 3.f6273fb7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-16T20:49:09.500 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:49:09.500 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.70937001200000005, 2024-09-16T20:49:09.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.70939039000000004, 2024-09-16T20:49:09.501 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:09.501 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:09.501 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "waiting for peered", 2024-09-16T20:49:09.501 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:09.501 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5286", 2024-09-16T20:49:09.501 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/4135433989", 2024-09-16T20:49:09.501 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 10643 2024-09-16T20:49:09.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.501 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:09.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.502 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:09.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:49:09.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.502 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:09.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:49:09.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.502 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:09.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790489+0000", 2024-09-16T20:49:09.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5910000000000001e-06 2024-09-16T20:49:09.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:09.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790493+0000", 2024-09-16T20:49:09.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2669999999999998e-06 2024-09-16T20:49:09.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:09.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790500+0000", 2024-09-16T20:49:09.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3780000000000003e-06 2024-09-16T20:49:09.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:09.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790505+0000", 2024-09-16T20:49:09.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1560000000000003e-06 2024-09-16T20:49:09.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.504 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:09.504 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790559+0000", 2024-09-16T20:49:09.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3494e-05 2024-09-16T20:49:09.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.504 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:49:09.504 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790566+0000", 2024-09-16T20:49:09.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7649999999999993e-06 2024-09-16T20:49:09.504 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.505 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:09.505 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.505 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.505 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-16T20:49:09.505 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 1 2024-09-16T20:49:09.505 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:09.509 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:49:09.593 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:09.593+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:09.619 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:09.619 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:49:09.619 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:49:09.619 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:49:09.619 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:09.628 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:49:09.694 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:09.693+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:09.778 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:09.778 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:49:09.780 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:49:09.780 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:49:09.780 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.780 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:780 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 933888~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:49:09.780 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.239489+0000", 2024-09-16T20:49:09.780 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 54.536693311000001, 2024-09-16T20:49:09.780 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00741699, 2024-09-16T20:49:09.780 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:09.780 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:09.780 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:09.780 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:09.781 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:49:09.781 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:49:09.781 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 780 2024-09-16T20:49:09.781 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.781 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:09.781 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.781 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:09.781 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239489+0000", 2024-09-16T20:49:09.781 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.781 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.781 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.781 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:09.781 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239489+0000", 2024-09-16T20:49:09.781 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.782 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.782 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:09.782 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239490+0000", 2024-09-16T20:49:09.782 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8599999999999998e-07 2024-09-16T20:49:09.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.782 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.782 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:09.782 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239492+0000", 2024-09-16T20:49:09.782 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1380000000000001e-06 2024-09-16T20:49:09.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.782 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.782 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:09.783 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239499+0000", 2024-09-16T20:49:09.783 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9530000000000002e-06 2024-09-16T20:49:09.783 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.783 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.783 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:09.783 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239504+0000", 2024-09-16T20:49:09.783 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0690000000000001e-06 2024-09-16T20:49:09.783 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.783 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.783 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:09.783 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246714+0000", 2024-09-16T20:49:09.783 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.007209844 2024-09-16T20:49:09.783 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.784 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.784 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:09.784 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246804+0000", 2024-09-16T20:49:09.784 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0109000000000001e-05 2024-09-16T20:49:09.784 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.784 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.784 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:09.784 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.246906+0000", 2024-09-16T20:49:09.784 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000102391 2024-09-16T20:49:09.784 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.784 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:09.784 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.784 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.785 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.785 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:782 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1146880~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:49:09.785 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.239925+0000", 2024-09-16T20:49:09.785 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 54.536257968000001, 2024-09-16T20:49:09.785 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0074161840000000001, 2024-09-16T20:49:09.785 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:09.785 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:09.785 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:09.785 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:09.785 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:49:09.785 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:49:09.785 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 782 2024-09-16T20:49:09.785 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.785 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:09.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.786 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:09.786 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239925+0000", 2024-09-16T20:49:09.786 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.786 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:09.786 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239925+0000", 2024-09-16T20:49:09.786 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.786 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:09.786 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239925+0000", 2024-09-16T20:49:09.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7199999999999999e-07 2024-09-16T20:49:09.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.787 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.787 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:09.787 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239928+0000", 2024-09-16T20:49:09.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5349999999999999e-06 2024-09-16T20:49:09.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.787 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.787 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:09.787 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239935+0000", 2024-09-16T20:49:09.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7680000000000003e-06 2024-09-16T20:49:09.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.787 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.787 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:09.788 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239940+0000", 2024-09-16T20:49:09.788 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.152e-06 2024-09-16T20:49:09.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.788 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.788 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:09.788 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247143+0000", 2024-09-16T20:49:09.788 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0072037309999999997 2024-09-16T20:49:09.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.788 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.788 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:09.788 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247230+0000", 2024-09-16T20:49:09.788 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6598e-05 2024-09-16T20:49:09.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.789 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.789 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:09.789 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247341+0000", 2024-09-16T20:49:09.789 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011092800000000001 2024-09-16T20:49:09.789 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.789 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:09.789 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.789 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.789 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:783 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1261568~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:49:09.789 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.239947+0000", 2024-09-16T20:49:09.789 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 54.536235646000002, 2024-09-16T20:49:09.789 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0076044789999999999, 2024-09-16T20:49:09.789 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:09.790 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:09.790 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:09.790 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:09.790 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:49:09.790 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:49:09.790 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 783 2024-09-16T20:49:09.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.790 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:09.790 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.790 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:09.790 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239947+0000", 2024-09-16T20:49:09.790 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.790 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.791 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:09.791 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239947+0000", 2024-09-16T20:49:09.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.791 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.791 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:09.791 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239947+0000", 2024-09-16T20:49:09.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9400000000000001e-07 2024-09-16T20:49:09.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.791 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.791 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:09.791 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239950+0000", 2024-09-16T20:49:09.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2350000000000002e-06 2024-09-16T20:49:09.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.792 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.792 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:09.792 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239956+0000", 2024-09-16T20:49:09.792 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3400000000000003e-06 2024-09-16T20:49:09.792 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.792 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.792 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:09.792 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239961+0000", 2024-09-16T20:49:09.792 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.04e-06 2024-09-16T20:49:09.792 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.792 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.792 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:09.792 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247371+0000", 2024-09-16T20:49:09.793 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0074096810000000004 2024-09-16T20:49:09.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.793 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.793 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:09.793 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247451+0000", 2024-09-16T20:49:09.793 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9845999999999994e-05 2024-09-16T20:49:09.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.793 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.793 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:09.793 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247552+0000", 2024-09-16T20:49:09.793 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000100943 2024-09-16T20:49:09.793 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.794 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:09.794 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.794 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:784 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1376256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:49:09.794 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.239997+0000", 2024-09-16T20:49:09.794 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 54.53618599, 2024-09-16T20:49:09.794 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.007775372, 2024-09-16T20:49:09.794 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:09.794 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:09.794 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:09.794 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:09.794 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:49:09.795 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:49:09.795 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 784 2024-09-16T20:49:09.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.795 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:09.795 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.795 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:09.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239997+0000", 2024-09-16T20:49:09.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.795 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.795 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:09.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239997+0000", 2024-09-16T20:49:09.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.796 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.796 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:09.796 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.239997+0000", 2024-09-16T20:49:09.796 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1799999999999995e-07 2024-09-16T20:49:09.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.796 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.796 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:09.796 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240001+0000", 2024-09-16T20:49:09.796 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6009999999999999e-06 2024-09-16T20:49:09.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.796 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.796 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:09.796 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240009+0000", 2024-09-16T20:49:09.796 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5809999999999999e-06 2024-09-16T20:49:09.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.797 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.797 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:09.797 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240015+0000", 2024-09-16T20:49:09.797 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0100000000000001e-06 2024-09-16T20:49:09.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.797 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.797 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:09.797 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247583+0000", 2024-09-16T20:49:09.797 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0075685730000000003 2024-09-16T20:49:09.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.797 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.797 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:09.798 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247672+0000", 2024-09-16T20:49:09.798 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8480000000000007e-05 2024-09-16T20:49:09.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.798 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.798 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:09.798 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247772+0000", 2024-09-16T20:49:09.798 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000100509 2024-09-16T20:49:09.798 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.798 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:09.798 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.798 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.798 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:785 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1474560~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:49:09.798 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240024+0000", 2024-09-16T20:49:09.799 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 54.536158403999998, 2024-09-16T20:49:09.799 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.008096776, 2024-09-16T20:49:09.799 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:09.799 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:09.799 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:09.799 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:09.799 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:49:09.799 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:49:09.799 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 785 2024-09-16T20:49:09.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.799 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:09.799 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.799 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:09.799 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240024+0000", 2024-09-16T20:49:09.799 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.800 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:09.800 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240024+0000", 2024-09-16T20:49:09.800 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.800 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:09.800 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240025+0000", 2024-09-16T20:49:09.800 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5000000000000003e-07 2024-09-16T20:49:09.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.800 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:09.801 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240027+0000", 2024-09-16T20:49:09.801 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2900000000000001e-06 2024-09-16T20:49:09.801 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.801 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.801 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:09.801 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240036+0000", 2024-09-16T20:49:09.801 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4710000000000007e-06 2024-09-16T20:49:09.801 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.801 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.801 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:09.801 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240041+0000", 2024-09-16T20:49:09.801 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4689999999999996e-06 2024-09-16T20:49:09.801 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.802 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.802 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:09.802 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247821+0000", 2024-09-16T20:49:09.802 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0077797919999999998 2024-09-16T20:49:09.802 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.802 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.802 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:09.802 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.247921+0000", 2024-09-16T20:49:09.802 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9814000000000004e-05 2024-09-16T20:49:09.802 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.802 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.802 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:09.802 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248121+0000", 2024-09-16T20:49:09.802 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00020039 2024-09-16T20:49:09.803 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.803 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:09.803 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.803 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.803 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:940 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 81920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:49:09.803 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240569+0000", 2024-09-16T20:49:09.803 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 54.535614146, 2024-09-16T20:49:09.803 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0078289759999999996, 2024-09-16T20:49:09.803 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:09.803 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:09.803 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:09.803 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:09.803 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:49:09.804 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:49:09.804 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 940 2024-09-16T20:49:09.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.804 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:09.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.804 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:09.804 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240569+0000", 2024-09-16T20:49:09.804 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.804 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:09.804 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240569+0000", 2024-09-16T20:49:09.804 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.805 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.805 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:09.805 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240569+0000", 2024-09-16T20:49:09.805 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2600000000000001e-07 2024-09-16T20:49:09.805 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.805 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.805 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:09.805 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240575+0000", 2024-09-16T20:49:09.805 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1159999999999998e-06 2024-09-16T20:49:09.805 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.805 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.805 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:09.805 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240587+0000", 2024-09-16T20:49:09.805 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1915e-05 2024-09-16T20:49:09.806 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.806 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.806 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:09.806 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240594+0000", 2024-09-16T20:49:09.806 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5610000000000001e-06 2024-09-16T20:49:09.806 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.806 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.806 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:09.806 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248158+0000", 2024-09-16T20:49:09.806 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0075635370000000004 2024-09-16T20:49:09.806 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.806 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.806 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:09.806 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248278+0000", 2024-09-16T20:49:09.807 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012009400000000001 2024-09-16T20:49:09.807 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.807 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.807 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:09.807 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248398+0000", 2024-09-16T20:49:09.807 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000119827 2024-09-16T20:49:09.807 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.807 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:09.807 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.807 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.807 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.807 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:941 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 196608~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:49:09.807 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240608+0000", 2024-09-16T20:49:09.807 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 54.535574568999998, 2024-09-16T20:49:09.808 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0080495519999999997, 2024-09-16T20:49:09.808 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:09.808 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:09.808 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:09.808 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:09.808 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:49:09.808 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:49:09.808 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 941 2024-09-16T20:49:09.808 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.808 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:09.808 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.808 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:09.808 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240608+0000", 2024-09-16T20:49:09.808 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.809 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.809 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:09.809 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240608+0000", 2024-09-16T20:49:09.809 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.809 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.809 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:09.809 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240609+0000", 2024-09-16T20:49:09.809 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7000000000000004e-07 2024-09-16T20:49:09.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.809 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.809 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:09.810 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240612+0000", 2024-09-16T20:49:09.810 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8310000000000001e-06 2024-09-16T20:49:09.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.810 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.810 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:09.810 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240619+0000", 2024-09-16T20:49:09.810 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8429999999999993e-06 2024-09-16T20:49:09.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.810 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.810 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:09.810 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240625+0000", 2024-09-16T20:49:09.810 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5659999999999997e-06 2024-09-16T20:49:09.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.813 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.813 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:09.813 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248456+0000", 2024-09-16T20:49:09.813 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0078313940000000002 2024-09-16T20:49:09.813 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.813 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.813 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:09.813 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248548+0000", 2024-09-16T20:49:09.813 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1324999999999997e-05 2024-09-16T20:49:09.813 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.813 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.813 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:09.814 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248658+0000", 2024-09-16T20:49:09.814 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000109923 2024-09-16T20:49:09.814 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.814 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:09.814 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.814 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.814 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.814 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:942 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 294912~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:49:09.814 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240675+0000", 2024-09-16T20:49:09.814 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 54.535507762000002, 2024-09-16T20:49:09.814 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0082135699999999999, 2024-09-16T20:49:09.814 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:09.814 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:09.815 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:09.815 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:09.815 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:49:09.815 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:49:09.815 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 942 2024-09-16T20:49:09.815 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.815 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:09.815 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.815 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:09.815 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240675+0000", 2024-09-16T20:49:09.815 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.815 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.815 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.816 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:09.816 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240675+0000", 2024-09-16T20:49:09.816 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.816 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.816 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.816 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:09.816 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240676+0000", 2024-09-16T20:49:09.816 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.37e-07 2024-09-16T20:49:09.816 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.816 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.816 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:09.816 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240679+0000", 2024-09-16T20:49:09.816 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4060000000000001e-06 2024-09-16T20:49:09.817 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.817 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.817 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:09.817 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240687+0000", 2024-09-16T20:49:09.817 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4640000000000006e-06 2024-09-16T20:49:09.817 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.817 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.817 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:09.817 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240694+0000", 2024-09-16T20:49:09.817 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.246e-06 2024-09-16T20:49:09.817 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.817 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.818 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:09.818 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248693+0000", 2024-09-16T20:49:09.818 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0079997550000000008 2024-09-16T20:49:09.818 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.818 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.818 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:09.818 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248781+0000", 2024-09-16T20:49:09.818 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7509999999999994e-05 2024-09-16T20:49:09.818 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.818 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.818 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:09.818 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248889+0000", 2024-09-16T20:49:09.818 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000107552 2024-09-16T20:49:09.818 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.819 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:09.819 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.819 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.819 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.819 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:943 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 409600~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:49:09.819 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240704+0000", 2024-09-16T20:49:09.819 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 54.535478765999997, 2024-09-16T20:49:09.819 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0083978209999999998, 2024-09-16T20:49:09.819 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:09.819 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:09.819 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:09.819 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:09.820 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:49:09.820 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:49:09.820 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 943 2024-09-16T20:49:09.820 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.820 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:09.820 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.820 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:09.820 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240704+0000", 2024-09-16T20:49:09.820 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.820 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.820 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.820 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:09.820 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240704+0000", 2024-09-16T20:49:09.821 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.821 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.821 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.821 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:09.821 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240704+0000", 2024-09-16T20:49:09.821 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7e-07 2024-09-16T20:49:09.821 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.821 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.821 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:09.821 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240707+0000", 2024-09-16T20:49:09.821 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5809999999999999e-06 2024-09-16T20:49:09.821 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.821 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.821 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:09.822 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240713+0000", 2024-09-16T20:49:09.822 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4949999999999996e-06 2024-09-16T20:49:09.822 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.822 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.822 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:09.822 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240718+0000", 2024-09-16T20:49:09.822 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8890000000000001e-06 2024-09-16T20:49:09.822 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.822 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.822 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:09.822 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.248920+0000", 2024-09-16T20:49:09.822 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0082018679999999993 2024-09-16T20:49:09.823 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.823 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.823 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:09.823 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249007+0000", 2024-09-16T20:49:09.823 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6327000000000002e-05 2024-09-16T20:49:09.823 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.823 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.823 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:09.823 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249102+0000", 2024-09-16T20:49:09.823 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5191000000000004e-05 2024-09-16T20:49:09.823 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.823 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:09.823 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.823 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.824 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.824 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:944 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 524288~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:49:09.824 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240726+0000", 2024-09-16T20:49:09.824 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 54.535456961000001, 2024-09-16T20:49:09.824 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0085999960000000004, 2024-09-16T20:49:09.824 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:09.824 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:09.824 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:09.824 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:09.824 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:49:09.824 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:49:09.824 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 944 2024-09-16T20:49:09.824 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.824 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:09.825 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.825 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:09.825 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240726+0000", 2024-09-16T20:49:09.825 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.825 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.825 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.825 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:09.825 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240726+0000", 2024-09-16T20:49:09.825 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.825 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.825 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.825 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:09.825 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240726+0000", 2024-09-16T20:49:09.826 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7099999999999997e-07 2024-09-16T20:49:09.826 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.826 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.826 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:09.826 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240728+0000", 2024-09-16T20:49:09.826 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.125e-06 2024-09-16T20:49:09.826 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.826 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.826 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:09.826 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240734+0000", 2024-09-16T20:49:09.826 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1890000000000004e-06 2024-09-16T20:49:09.826 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.826 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.826 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:09.827 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240739+0000", 2024-09-16T20:49:09.827 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0340000000000004e-06 2024-09-16T20:49:09.827 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.827 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.827 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:09.827 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249131+0000", 2024-09-16T20:49:09.827 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0083919579999999997 2024-09-16T20:49:09.827 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.827 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.827 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:09.827 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249217+0000", 2024-09-16T20:49:09.827 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5154999999999994e-05 2024-09-16T20:49:09.827 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.827 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.828 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:09.828 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249326+0000", 2024-09-16T20:49:09.828 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000109164 2024-09-16T20:49:09.828 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.828 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:09.828 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.828 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.828 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.828 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:945 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 622592~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:49:09.828 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240746+0000", 2024-09-16T20:49:09.828 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 54.535436822000001, 2024-09-16T20:49:09.828 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0088083320000000003, 2024-09-16T20:49:09.828 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:09.829 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:09.829 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:09.829 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:09.829 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:49:09.829 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:49:09.829 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 945 2024-09-16T20:49:09.829 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.829 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:09.829 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.829 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:09.829 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240746+0000", 2024-09-16T20:49:09.829 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.829 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.830 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.830 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:09.830 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240746+0000", 2024-09-16T20:49:09.830 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.830 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.830 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.830 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:09.830 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240746+0000", 2024-09-16T20:49:09.830 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8500000000000002e-07 2024-09-16T20:49:09.830 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.831 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.831 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:09.831 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240748+0000", 2024-09-16T20:49:09.831 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8449999999999999e-06 2024-09-16T20:49:09.831 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.831 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.831 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:09.831 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240754+0000", 2024-09-16T20:49:09.831 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8830000000000003e-06 2024-09-16T20:49:09.831 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.831 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.832 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:09.832 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240759+0000", 2024-09-16T20:49:09.832 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0220000000000004e-06 2024-09-16T20:49:09.832 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.832 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.832 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:09.832 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249360+0000", 2024-09-16T20:49:09.832 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.008601437 2024-09-16T20:49:09.832 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.832 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.832 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:09.832 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249453+0000", 2024-09-16T20:49:09.832 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2743000000000003e-05 2024-09-16T20:49:09.833 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.833 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.833 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:09.833 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249554+0000", 2024-09-16T20:49:09.833 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101017 2024-09-16T20:49:09.833 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.833 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:09.833 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.833 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.833 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.833 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:946 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 737280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:49:09.833 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240766+0000", 2024-09-16T20:49:09.833 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 54.535417119000002, 2024-09-16T20:49:09.834 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0089092070000000006, 2024-09-16T20:49:09.834 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:09.834 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:09.834 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:09.834 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:09.834 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:49:09.834 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:49:09.834 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 946 2024-09-16T20:49:09.834 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.834 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:09.834 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.834 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:09.834 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240766+0000", 2024-09-16T20:49:09.834 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.835 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.835 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.835 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:09.835 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240766+0000", 2024-09-16T20:49:09.835 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.835 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.835 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.835 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:09.835 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240766+0000", 2024-09-16T20:49:09.835 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0600000000000001e-07 2024-09-16T20:49:09.835 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.835 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.835 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:09.835 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240768+0000", 2024-09-16T20:49:09.836 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8959999999999999e-06 2024-09-16T20:49:09.836 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.836 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.836 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:09.836 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240774+0000", 2024-09-16T20:49:09.836 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0979999999999999e-06 2024-09-16T20:49:09.836 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.836 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.836 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:09.836 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240779+0000", 2024-09-16T20:49:09.836 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.933e-06 2024-09-16T20:49:09.836 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.836 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.837 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:09.837 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249572+0000", 2024-09-16T20:49:09.837 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0087926380000000002 2024-09-16T20:49:09.837 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.837 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.837 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:09.837 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249612+0000", 2024-09-16T20:49:09.837 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0859000000000003e-05 2024-09-16T20:49:09.837 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.837 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.837 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:09.837 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249675+0000", 2024-09-16T20:49:09.837 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2377000000000003e-05 2024-09-16T20:49:09.837 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.838 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:09.838 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.838 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.838 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.838 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:947 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 851968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:49:09.838 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240785+0000", 2024-09-16T20:49:09.838 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 54.535397527000001, 2024-09-16T20:49:09.838 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0090960260000000001, 2024-09-16T20:49:09.838 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:09.838 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:09.838 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:09.838 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:09.838 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:49:09.839 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:49:09.839 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 947 2024-09-16T20:49:09.839 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.839 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:09.839 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.839 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:09.839 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240785+0000", 2024-09-16T20:49:09.839 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.839 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.839 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.839 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:09.839 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240785+0000", 2024-09-16T20:49:09.839 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.839 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.840 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.840 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:09.840 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240786+0000", 2024-09-16T20:49:09.840 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-16T20:49:09.840 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.840 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.840 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:09.840 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240787+0000", 2024-09-16T20:49:09.840 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.858e-06 2024-09-16T20:49:09.840 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.840 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.840 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:09.840 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240793+0000", 2024-09-16T20:49:09.841 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7760000000000003e-06 2024-09-16T20:49:09.841 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.841 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.841 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:09.841 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240799+0000", 2024-09-16T20:49:09.841 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1109999999999996e-06 2024-09-16T20:49:09.841 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.841 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.841 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:09.841 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249715+0000", 2024-09-16T20:49:09.841 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0089159789999999992 2024-09-16T20:49:09.841 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.841 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.842 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:09.842 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249766+0000", 2024-09-16T20:49:09.842 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0968999999999999e-05 2024-09-16T20:49:09.842 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.842 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.842 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:09.842 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249881+0000", 2024-09-16T20:49:09.842 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114963 2024-09-16T20:49:09.842 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.842 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:09.842 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.842 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.842 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.842 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:948 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:49:09.843 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240806+0000", 2024-09-16T20:49:09.843 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 54.535376347000003, 2024-09-16T20:49:09.843 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0092300609999999995, 2024-09-16T20:49:09.843 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:09.843 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:09.843 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:09.843 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:09.843 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:49:09.843 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:49:09.843 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 948 2024-09-16T20:49:09.843 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.843 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:09.843 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.843 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:09.844 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240806+0000", 2024-09-16T20:49:09.844 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.844 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.844 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.844 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:09.844 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240806+0000", 2024-09-16T20:49:09.844 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.844 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.844 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.844 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:09.844 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240807+0000", 2024-09-16T20:49:09.844 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4499999999999998e-07 2024-09-16T20:49:09.844 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.845 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.845 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:09.845 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240809+0000", 2024-09-16T20:49:09.845 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.34e-06 2024-09-16T20:49:09.845 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.845 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.845 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:09.845 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240815+0000", 2024-09-16T20:49:09.845 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1449999999999996e-06 2024-09-16T20:49:09.845 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.845 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.845 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:09.845 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240821+0000", 2024-09-16T20:49:09.845 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0270000000000001e-06 2024-09-16T20:49:09.846 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.846 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.846 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:09.846 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249920+0000", 2024-09-16T20:49:09.846 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0090988460000000007 2024-09-16T20:49:09.846 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.846 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.846 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:09.846 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.249966+0000", 2024-09-16T20:49:09.846 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5918999999999999e-05 2024-09-16T20:49:09.846 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.846 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.846 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:09.846 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250036+0000", 2024-09-16T20:49:09.847 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0438999999999994e-05 2024-09-16T20:49:09.847 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.847 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:09.847 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.847 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.847 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.847 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:949 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1064960~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:49:09.847 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240833+0000", 2024-09-16T20:49:09.847 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 54.535349732, 2024-09-16T20:49:09.847 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0093580339999999994, 2024-09-16T20:49:09.847 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:09.847 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:09.847 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:09.848 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:09.848 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:49:09.848 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:49:09.848 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 949 2024-09-16T20:49:09.848 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.848 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:09.848 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.848 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:09.848 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240833+0000", 2024-09-16T20:49:09.848 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.848 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.848 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.848 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:09.849 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240833+0000", 2024-09-16T20:49:09.849 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.849 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.849 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.849 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:09.849 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240833+0000", 2024-09-16T20:49:09.849 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2899999999999999e-07 2024-09-16T20:49:09.849 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.849 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.849 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:09.849 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240836+0000", 2024-09-16T20:49:09.849 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2649999999999999e-06 2024-09-16T20:49:09.849 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.850 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.850 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:09.850 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240842+0000", 2024-09-16T20:49:09.850 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5579999999999997e-06 2024-09-16T20:49:09.850 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.850 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.850 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:09.850 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240847+0000", 2024-09-16T20:49:09.850 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2009999999999998e-06 2024-09-16T20:49:09.850 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.850 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.850 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:09.850 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250073+0000", 2024-09-16T20:49:09.850 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0092258710000000001 2024-09-16T20:49:09.851 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.851 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.851 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:09.851 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250141+0000", 2024-09-16T20:49:09.851 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7849999999999996e-05 2024-09-16T20:49:09.851 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.851 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.851 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:09.851 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250191+0000", 2024-09-16T20:49:09.851 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.986e-05 2024-09-16T20:49:09.851 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.851 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:09.851 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.852 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.852 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.852 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:950 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1179648~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:49:09.852 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.240855+0000", 2024-09-16T20:49:09.852 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 54.535327479999999, 2024-09-16T20:49:09.852 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0095855560000000003, 2024-09-16T20:49:09.852 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:09.852 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:09.852 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:09.852 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:09.852 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:49:09.852 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:49:09.852 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 950 2024-09-16T20:49:09.852 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.853 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:09.853 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.853 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:09.853 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240855+0000", 2024-09-16T20:49:09.853 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.853 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.853 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.853 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:09.853 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240855+0000", 2024-09-16T20:49:09.853 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.853 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.853 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.853 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:09.854 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240856+0000", 2024-09-16T20:49:09.854 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4999999999999998e-07 2024-09-16T20:49:09.854 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.854 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.854 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:09.854 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240858+0000", 2024-09-16T20:49:09.854 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.996e-06 2024-09-16T20:49:09.854 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.854 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.854 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:09.854 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240864+0000", 2024-09-16T20:49:09.854 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9359999999999996e-06 2024-09-16T20:49:09.854 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.855 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.855 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:09.855 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.240868+0000", 2024-09-16T20:49:09.855 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7670000000000003e-06 2024-09-16T20:49:09.855 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.855 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.855 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:09.855 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250226+0000", 2024-09-16T20:49:09.855 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0093576100000000006 2024-09-16T20:49:09.855 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.855 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.855 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:09.855 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250289+0000", 2024-09-16T20:49:09.856 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2627999999999997e-05 2024-09-16T20:49:09.856 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.856 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.856 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:09.856 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250441+0000", 2024-09-16T20:49:09.856 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015216900000000001 2024-09-16T20:49:09.856 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.856 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:09.856 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.856 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.856 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.856 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:951 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1277952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:49:09.856 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.241044+0000", 2024-09-16T20:49:09.857 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 54.535138291999999, 2024-09-16T20:49:09.857 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0096266589999999992, 2024-09-16T20:49:09.857 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:09.857 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:09.857 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:09.857 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:09.857 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:49:09.857 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:49:09.857 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 951 2024-09-16T20:49:09.857 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.857 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:09.857 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.857 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:09.857 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241044+0000", 2024-09-16T20:49:09.858 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.858 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.858 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.858 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:09.858 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241044+0000", 2024-09-16T20:49:09.858 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.858 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.858 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.858 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:09.858 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241045+0000", 2024-09-16T20:49:09.858 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5300000000000004e-07 2024-09-16T20:49:09.858 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.858 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.858 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:09.859 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241048+0000", 2024-09-16T20:49:09.859 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.303e-06 2024-09-16T20:49:09.859 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.859 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.859 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:09.859 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241057+0000", 2024-09-16T20:49:09.859 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5550000000000003e-06 2024-09-16T20:49:09.859 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.859 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.859 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:09.859 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241063+0000", 2024-09-16T20:49:09.859 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3470000000000004e-06 2024-09-16T20:49:09.859 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.860 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.860 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:09.860 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250477+0000", 2024-09-16T20:49:09.860 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0094136159999999997 2024-09-16T20:49:09.860 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.860 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.860 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:09.860 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250562+0000", 2024-09-16T20:49:09.860 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4709000000000005e-05 2024-09-16T20:49:09.860 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.860 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.860 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:09.861 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250671+0000", 2024-09-16T20:49:09.861 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010947599999999999 2024-09-16T20:49:09.861 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.861 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:09.861 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.861 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.861 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.861 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:952 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1392640~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:49:09.861 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.241130+0000", 2024-09-16T20:49:09.861 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 54.535052426, 2024-09-16T20:49:09.861 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0098096290000000003, 2024-09-16T20:49:09.861 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:09.861 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:09.861 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:09.862 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:09.862 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:49:09.862 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:49:09.862 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 952 2024-09-16T20:49:09.862 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.862 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:09.862 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.862 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:09.862 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241130+0000", 2024-09-16T20:49:09.862 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.862 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.862 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.862 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:09.862 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241130+0000", 2024-09-16T20:49:09.863 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.863 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.863 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.863 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:09.863 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241131+0000", 2024-09-16T20:49:09.863 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.61e-07 2024-09-16T20:49:09.863 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.863 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.863 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:09.863 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241134+0000", 2024-09-16T20:49:09.863 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1080000000000001e-06 2024-09-16T20:49:09.863 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.863 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.864 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:09.864 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241142+0000", 2024-09-16T20:49:09.864 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9119999999999998e-06 2024-09-16T20:49:09.864 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.864 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.864 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:09.864 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241148+0000", 2024-09-16T20:49:09.864 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1410000000000002e-06 2024-09-16T20:49:09.864 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.864 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.864 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:09.864 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250731+0000", 2024-09-16T20:49:09.864 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0095833979999999999 2024-09-16T20:49:09.865 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.865 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.865 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:09.865 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250820+0000", 2024-09-16T20:49:09.865 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8997000000000002e-05 2024-09-16T20:49:09.865 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.865 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.865 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:09.865 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250940+0000", 2024-09-16T20:49:09.865 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000119412 2024-09-16T20:49:09.865 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.865 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:09.865 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.866 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.866 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.866 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4439.0:953 3.17 3:e8782132:::rbd_data.110e1748a8d2.0000000000000008:4 [read 1507328~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T20:49:09.866 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:15.241157+0000", 2024-09-16T20:49:09.866 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 54.535025505, 2024-09-16T20:49:09.866 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.010039259, 2024-09-16T20:49:09.866 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:09.866 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:09.866 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:09.866 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:09.866 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4439", 2024-09-16T20:49:09.866 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2553151540", 2024-09-16T20:49:09.866 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 953 2024-09-16T20:49:09.866 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.866 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:09.867 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.867 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:09.867 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241157+0000", 2024-09-16T20:49:09.867 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.867 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.867 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.867 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:09.867 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241157+0000", 2024-09-16T20:49:09.867 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.867 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.867 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.867 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:09.867 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241158+0000", 2024-09-16T20:49:09.868 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.01e-07 2024-09-16T20:49:09.868 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.868 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.868 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:09.868 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241160+0000", 2024-09-16T20:49:09.868 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2879999999999999e-06 2024-09-16T20:49:09.868 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.868 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.868 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:09.868 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241166+0000", 2024-09-16T20:49:09.868 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.421e-06 2024-09-16T20:49:09.868 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.868 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.868 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:09.869 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.241171+0000", 2024-09-16T20:49:09.869 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9840000000000001e-06 2024-09-16T20:49:09.869 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.869 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.869 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:09.869 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.250988+0000", 2024-09-16T20:49:09.869 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0098171449999999993 2024-09-16T20:49:09.869 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.869 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.869 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:09.869 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.251085+0000", 2024-09-16T20:49:09.869 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6877000000000001e-05 2024-09-16T20:49:09.869 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.870 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.870 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:09.870 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:15.251196+0000", 2024-09-16T20:49:09.870 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000111143 2024-09-16T20:49:09.870 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.870 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:09.870 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.870 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.870 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.870 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5286.0:10643 3.17 3.f6273fb7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-16T20:49:09.870 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:49:09.870 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.98569498300000002, 2024-09-16T20:49:09.870 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.76672347200000002, 2024-09-16T20:49:09.871 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:09.871 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:09.871 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:49:09.871 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:09.871 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5286", 2024-09-16T20:49:09.871 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/4135433989", 2024-09-16T20:49:09.871 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 10643 2024-09-16T20:49:09.871 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.871 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:09.871 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.871 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:09.871 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:49:09.871 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.871 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.871 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.872 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:09.872 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:49:09.872 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:09.872 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.872 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.872 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:09.872 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790489+0000", 2024-09-16T20:49:09.872 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5910000000000001e-06 2024-09-16T20:49:09.872 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.872 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.872 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:09.872 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790493+0000", 2024-09-16T20:49:09.872 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2669999999999998e-06 2024-09-16T20:49:09.873 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.873 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.873 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:09.873 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790500+0000", 2024-09-16T20:49:09.873 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3780000000000003e-06 2024-09-16T20:49:09.873 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.873 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.873 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:09.873 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790505+0000", 2024-09-16T20:49:09.873 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1560000000000003e-06 2024-09-16T20:49:09.873 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.873 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.873 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:09.874 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790559+0000", 2024-09-16T20:49:09.874 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3494e-05 2024-09-16T20:49:09.874 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.874 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.874 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:49:09.874 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790566+0000", 2024-09-16T20:49:09.874 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7649999999999993e-06 2024-09-16T20:49:09.874 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.874 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.874 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:09.874 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:09.557198+0000", 2024-09-16T20:49:09.874 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.76663136200000004 2024-09-16T20:49:09.874 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:09.874 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:09.875 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:09.875 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:09.557211+0000", 2024-09-16T20:49:09.875 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3458999999999999e-05 2024-09-16T20:49:09.875 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.875 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:09.875 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.875 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:09.875 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:09.875 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:09.876 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T20:49:09.877 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:09.875+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:09.877 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:49:09.977 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:09.975+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:09.999 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:09.999 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:49:10.000 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.000 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5286.0:10643 3.17 3.f6273fb7 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T20:49:10.000 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:09.557722+0000", 2024-09-16T20:49:10.000 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.44146128899999998, 2024-09-16T20:49:10.000 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.44148323699999997, 2024-09-16T20:49:10.000 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:10.000 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:10.000 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "waiting for peered", 2024-09-16T20:49:10.000 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:10.000 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5286", 2024-09-16T20:49:10.000 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/4135433989", 2024-09-16T20:49:10.000 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 10643 2024-09-16T20:49:10.000 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.000 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:10.001 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.001 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:10.001 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:09.557722+0000", 2024-09-16T20:49:10.001 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:10.001 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.001 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.001 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:10.001 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:09.557722+0000", 2024-09-16T20:49:10.001 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:10.001 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.001 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.001 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:10.001 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:09.557725+0000", 2024-09-16T20:49:10.002 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.492e-06 2024-09-16T20:49:10.002 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.002 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.002 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:10.002 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:09.557733+0000", 2024-09-16T20:49:10.002 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9810000000000003e-06 2024-09-16T20:49:10.002 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.002 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.002 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:10.002 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:09.557748+0000", 2024-09-16T20:49:10.002 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4883e-05 2024-09-16T20:49:10.002 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.002 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.002 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:10.003 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:09.557761+0000", 2024-09-16T20:49:10.003 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2948e-05 2024-09-16T20:49:10.003 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.003 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.003 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:10.003 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:09.559951+0000", 2024-09-16T20:49:10.003 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0021904089999999999 2024-09-16T20:49:10.003 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.003 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.003 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:49:10.003 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:09.559965+0000", 2024-09-16T20:49:10.003 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4071e-05 2024-09-16T20:49:10.003 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:10.003 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:10.004 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:10.004 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:10.004 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-16T20:49:10.004 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 1 2024-09-16T20:49:10.004 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:10.011 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:49:10.077 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:10.077+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:10.159 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:10.159 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:49:10.159 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:49:10.159 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:49:10.159 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:10.167 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:49:10.179 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:10.178+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:10.276 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:10.276 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:49:10.276 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:49:10.277 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:49:10.277 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.277 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2179 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 948224~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:10.277 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335723+0000", 2024-09-16T20:49:10.277 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 47.938952800000003, 2024-09-16T20:49:10.277 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048007181000000003, 2024-09-16T20:49:10.277 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:10.277 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:10.277 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:10.277 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:10.277 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:10.277 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:10.277 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2179 2024-09-16T20:49:10.277 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.278 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:10.278 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.278 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:10.278 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335723+0000", 2024-09-16T20:49:10.278 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:10.278 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.278 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.278 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:10.278 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335723+0000", 2024-09-16T20:49:10.278 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:10.278 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.278 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.278 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:10.278 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335724+0000", 2024-09-16T20:49:10.278 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.58e-07 2024-09-16T20:49:10.279 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.279 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.279 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:10.279 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335726+0000", 2024-09-16T20:49:10.279 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9120000000000001e-06 2024-09-16T20:49:10.279 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.279 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.279 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:10.279 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335732+0000", 2024-09-16T20:49:10.279 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6080000000000001e-06 2024-09-16T20:49:10.279 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.279 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.279 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:10.279 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335737+0000", 2024-09-16T20:49:10.279 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.22e-06 2024-09-16T20:49:10.279 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.280 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.280 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:10.280 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383556+0000", 2024-09-16T20:49:10.280 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047819002999999999 2024-09-16T20:49:10.280 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.280 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.280 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:10.280 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383635+0000", 2024-09-16T20:49:10.280 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9060999999999994e-05 2024-09-16T20:49:10.280 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.280 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.280 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:10.280 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383730+0000", 2024-09-16T20:49:10.280 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4919000000000004e-05 2024-09-16T20:49:10.281 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:10.281 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:10.281 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:10.281 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.281 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.281 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2180 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 956416~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:10.281 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335744+0000", 2024-09-16T20:49:10.281 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 47.938932352000002, 2024-09-16T20:49:10.281 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048255147999999998, 2024-09-16T20:49:10.281 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:10.281 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:10.281 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:10.281 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:10.281 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:10.281 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:10.281 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2180 2024-09-16T20:49:10.282 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.282 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:10.282 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.282 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:10.282 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335744+0000", 2024-09-16T20:49:10.282 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:10.282 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.282 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.282 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:10.282 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335744+0000", 2024-09-16T20:49:10.282 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:10.282 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.282 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.282 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:10.282 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335744+0000", 2024-09-16T20:49:10.282 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9000000000000002e-07 2024-09-16T20:49:10.283 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.283 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.283 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:10.283 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335746+0000", 2024-09-16T20:49:10.283 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7570000000000001e-06 2024-09-16T20:49:10.283 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.283 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.283 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:10.283 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335752+0000", 2024-09-16T20:49:10.283 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0079999999999999e-06 2024-09-16T20:49:10.283 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.283 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.283 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:10.283 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335757+0000", 2024-09-16T20:49:10.283 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1649999999999997e-06 2024-09-16T20:49:10.284 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.284 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.284 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:10.284 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383803+0000", 2024-09-16T20:49:10.284 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048046091999999999 2024-09-16T20:49:10.284 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.284 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.284 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:10.284 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383889+0000", 2024-09-16T20:49:10.284 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6048e-05 2024-09-16T20:49:10.284 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.284 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.284 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:10.284 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383999+0000", 2024-09-16T20:49:10.285 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000109688 2024-09-16T20:49:10.285 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:10.285 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:10.285 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:10.285 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.285 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.285 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2182 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 964608~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:10.285 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335773+0000", 2024-09-16T20:49:10.285 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 47.938902544999998, 2024-09-16T20:49:10.285 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048467116999999997, 2024-09-16T20:49:10.285 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:10.285 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:10.285 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:10.285 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:10.286 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:10.286 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:10.286 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2182 2024-09-16T20:49:10.286 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.286 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:10.286 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.286 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:10.286 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335773+0000", 2024-09-16T20:49:10.286 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:10.286 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.286 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.286 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:10.286 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335773+0000", 2024-09-16T20:49:10.286 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:10.286 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.286 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.286 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:10.287 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335774+0000", 2024-09-16T20:49:10.287 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3799999999999997e-07 2024-09-16T20:49:10.287 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.287 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.287 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:10.287 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335776+0000", 2024-09-16T20:49:10.287 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.227e-06 2024-09-16T20:49:10.287 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.287 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.287 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:10.287 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335790+0000", 2024-09-16T20:49:10.287 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4119e-05 2024-09-16T20:49:10.287 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.287 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.288 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:10.288 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335796+0000", 2024-09-16T20:49:10.288 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.694e-06 2024-09-16T20:49:10.288 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.288 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.288 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:10.288 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384058+0000", 2024-09-16T20:49:10.288 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048261983000000001 2024-09-16T20:49:10.288 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.288 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.288 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:10.288 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384148+0000", 2024-09-16T20:49:10.288 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9501999999999998e-05 2024-09-16T20:49:10.288 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.289 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.289 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:10.289 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384241+0000", 2024-09-16T20:49:10.289 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.3053999999999994e-05 2024-09-16T20:49:10.289 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:10.289 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:10.289 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:10.289 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.289 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.289 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2185 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 972800~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:10.289 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335804+0000", 2024-09-16T20:49:10.289 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 47.938871894999998, 2024-09-16T20:49:10.289 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048570818000000002, 2024-09-16T20:49:10.289 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:10.289 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:10.290 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:10.290 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:10.290 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:10.290 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:10.290 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2185 2024-09-16T20:49:10.290 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.290 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:10.290 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.290 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:10.290 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335804+0000", 2024-09-16T20:49:10.290 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:10.290 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.290 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.290 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:10.291 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335804+0000", 2024-09-16T20:49:10.291 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:10.291 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.291 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.291 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:10.291 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335805+0000", 2024-09-16T20:49:10.291 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8299999999999997e-07 2024-09-16T20:49:10.291 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.291 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.291 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:10.291 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335813+0000", 2024-09-16T20:49:10.291 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6309999999999992e-06 2024-09-16T20:49:10.291 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.291 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.292 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:10.292 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335821+0000", 2024-09-16T20:49:10.292 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0169999999999996e-06 2024-09-16T20:49:10.292 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.292 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.292 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:10.292 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335827+0000", 2024-09-16T20:49:10.292 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0100000000000001e-06 2024-09-16T20:49:10.292 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.292 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.292 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:10.292 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384273+0000", 2024-09-16T20:49:10.292 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048445333 2024-09-16T20:49:10.292 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.292 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.293 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:10.293 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384315+0000", 2024-09-16T20:49:10.293 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2472999999999998e-05 2024-09-16T20:49:10.293 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.293 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.293 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:10.293 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384375+0000", 2024-09-16T20:49:10.293 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9871000000000003e-05 2024-09-16T20:49:10.293 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:10.293 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:10.293 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:10.293 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.293 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.293 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2187 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 980992~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:10.293 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335835+0000", 2024-09-16T20:49:10.294 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 47.938840818000003, 2024-09-16T20:49:10.294 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048684927000000003, 2024-09-16T20:49:10.294 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:10.294 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:10.294 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:10.294 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:10.294 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:10.294 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:10.294 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2187 2024-09-16T20:49:10.294 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.294 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:10.294 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.294 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:10.294 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335835+0000", 2024-09-16T20:49:10.295 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:10.295 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.295 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.295 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:10.295 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335835+0000", 2024-09-16T20:49:10.295 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:10.295 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.295 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.295 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:10.295 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335836+0000", 2024-09-16T20:49:10.295 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3599999999999999e-07 2024-09-16T20:49:10.295 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.295 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.295 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:10.295 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335838+0000", 2024-09-16T20:49:10.296 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.996e-06 2024-09-16T20:49:10.296 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.296 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.296 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:10.296 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335844+0000", 2024-09-16T20:49:10.296 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7889999999999997e-06 2024-09-16T20:49:10.296 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.296 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.296 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:10.296 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335849+0000", 2024-09-16T20:49:10.296 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7400000000000004e-06 2024-09-16T20:49:10.296 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.296 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.296 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:10.296 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384411+0000", 2024-09-16T20:49:10.297 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048562069999999999 2024-09-16T20:49:10.297 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.297 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.297 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:10.297 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384460+0000", 2024-09-16T20:49:10.297 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9125999999999997e-05 2024-09-16T20:49:10.297 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.297 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.297 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:10.297 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384520+0000", 2024-09-16T20:49:10.297 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9769999999999999e-05 2024-09-16T20:49:10.297 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:10.297 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:10.297 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:10.297 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.298 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.298 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2189 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 989184~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:10.298 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335856+0000", 2024-09-16T20:49:10.298 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 47.938819573000004, 2024-09-16T20:49:10.298 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048856525999999997, 2024-09-16T20:49:10.298 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:10.298 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:10.298 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:10.298 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:10.298 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:10.298 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:10.298 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2189 2024-09-16T20:49:10.298 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.298 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:10.298 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.298 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:10.299 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335856+0000", 2024-09-16T20:49:10.299 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:10.299 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.299 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.299 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:10.299 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335856+0000", 2024-09-16T20:49:10.299 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:10.299 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.299 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.299 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:10.299 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335857+0000", 2024-09-16T20:49:10.299 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6800000000000001e-07 2024-09-16T20:49:10.299 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.299 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.299 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:10.300 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335859+0000", 2024-09-16T20:49:10.300 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.013e-06 2024-09-16T20:49:10.300 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.300 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.300 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:10.300 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335866+0000", 2024-09-16T20:49:10.300 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8750000000000002e-06 2024-09-16T20:49:10.300 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.300 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.300 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:10.300 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335871+0000", 2024-09-16T20:49:10.300 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1719999999999998e-06 2024-09-16T20:49:10.300 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.300 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.300 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:10.301 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384550+0000", 2024-09-16T20:49:10.301 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048679476999999999 2024-09-16T20:49:10.301 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.301 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.301 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:10.301 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384611+0000", 2024-09-16T20:49:10.301 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0897000000000003e-05 2024-09-16T20:49:10.301 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.301 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.301 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:10.301 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384713+0000", 2024-09-16T20:49:10.301 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101724 2024-09-16T20:49:10.301 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:10.301 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:10.301 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:10.302 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.302 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.302 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2192 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 997376~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:10.302 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335878+0000", 2024-09-16T20:49:10.302 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 47.938797888000003, 2024-09-16T20:49:10.302 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049028769999999999, 2024-09-16T20:49:10.302 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:10.302 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:10.302 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:10.302 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:10.302 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:10.302 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:10.302 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2192 2024-09-16T20:49:10.302 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.302 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:10.303 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.303 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:10.303 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335878+0000", 2024-09-16T20:49:10.303 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:10.303 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.303 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.303 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:10.303 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335878+0000", 2024-09-16T20:49:10.303 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:10.303 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.303 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.303 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:10.303 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335879+0000", 2024-09-16T20:49:10.303 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6899999999999998e-07 2024-09-16T20:49:10.303 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.304 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.304 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:10.304 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335881+0000", 2024-09-16T20:49:10.304 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.996e-06 2024-09-16T20:49:10.304 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.304 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.304 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:10.304 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335887+0000", 2024-09-16T20:49:10.304 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6370000000000002e-06 2024-09-16T20:49:10.304 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.304 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.304 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:10.304 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335892+0000", 2024-09-16T20:49:10.304 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2120000000000002e-06 2024-09-16T20:49:10.304 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.305 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.305 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:10.305 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384747+0000", 2024-09-16T20:49:10.305 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048854923000000001 2024-09-16T20:49:10.305 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.305 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.305 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:10.305 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384830+0000", 2024-09-16T20:49:10.305 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2359000000000002e-05 2024-09-16T20:49:10.305 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.305 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.305 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:10.305 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384907+0000", 2024-09-16T20:49:10.306 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7174000000000003e-05 2024-09-16T20:49:10.306 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:10.306 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:10.306 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:10.306 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.306 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.306 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2194 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1005568~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:10.306 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335899+0000", 2024-09-16T20:49:10.306 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 47.938776967000003, 2024-09-16T20:49:10.306 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049225393999999999, 2024-09-16T20:49:10.306 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:10.306 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:10.306 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:10.306 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:10.307 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:10.307 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:10.307 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2194 2024-09-16T20:49:10.307 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.307 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:10.307 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.307 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:10.307 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335899+0000", 2024-09-16T20:49:10.307 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:10.307 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.307 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.307 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:10.307 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335899+0000", 2024-09-16T20:49:10.307 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:10.308 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.308 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.308 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:10.308 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335899+0000", 2024-09-16T20:49:10.308 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8200000000000001e-07 2024-09-16T20:49:10.308 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.308 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.308 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:10.308 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335901+0000", 2024-09-16T20:49:10.308 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9e-06 2024-09-16T20:49:10.308 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.308 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.308 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:10.308 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335908+0000", 2024-09-16T20:49:10.309 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3559999999999998e-06 2024-09-16T20:49:10.309 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.309 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.309 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:10.309 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335914+0000", 2024-09-16T20:49:10.309 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4729999999999997e-06 2024-09-16T20:49:10.309 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.309 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.309 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:10.309 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384979+0000", 2024-09-16T20:49:10.309 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049064545000000001 2024-09-16T20:49:10.309 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.309 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.309 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:10.310 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385060+0000", 2024-09-16T20:49:10.310 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0804999999999996e-05 2024-09-16T20:49:10.310 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.310 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.310 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:10.310 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385124+0000", 2024-09-16T20:49:10.310 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4932999999999996e-05 2024-09-16T20:49:10.310 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:10.310 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:10.310 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:10.310 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.310 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.310 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2197 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1013760~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:10.310 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335921+0000", 2024-09-16T20:49:10.310 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 47.938754686000003, 2024-09-16T20:49:10.311 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049342184999999997, 2024-09-16T20:49:10.311 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:10.311 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:10.311 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:10.311 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:10.311 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:10.311 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:10.311 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2197 2024-09-16T20:49:10.311 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.311 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:10.311 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.311 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:10.311 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335921+0000", 2024-09-16T20:49:10.311 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:10.311 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.312 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.312 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:10.312 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335921+0000", 2024-09-16T20:49:10.312 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:10.312 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.312 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.312 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:10.312 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335922+0000", 2024-09-16T20:49:10.312 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.32e-07 2024-09-16T20:49:10.312 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.312 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.312 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:10.312 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335924+0000", 2024-09-16T20:49:10.312 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.995e-06 2024-09-16T20:49:10.312 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.313 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.313 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:10.313 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335930+0000", 2024-09-16T20:49:10.313 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5139999999999998e-06 2024-09-16T20:49:10.313 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.313 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.313 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:10.313 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335936+0000", 2024-09-16T20:49:10.313 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6160000000000001e-06 2024-09-16T20:49:10.313 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.313 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.313 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:10.313 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385142+0000", 2024-09-16T20:49:10.313 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049206393000000001 2024-09-16T20:49:10.313 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.314 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.314 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:10.314 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385184+0000", 2024-09-16T20:49:10.314 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1928999999999998e-05 2024-09-16T20:49:10.314 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.314 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.314 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:10.314 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385264+0000", 2024-09-16T20:49:10.314 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9306000000000003e-05 2024-09-16T20:49:10.314 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:10.314 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:10.314 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:10.314 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.314 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.314 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2198 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1021952~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:10.314 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335942+0000", 2024-09-16T20:49:10.315 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 47.938733626000001, 2024-09-16T20:49:10.315 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049562483999999997, 2024-09-16T20:49:10.315 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:10.315 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:10.315 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:10.315 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:10.315 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:10.315 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:10.315 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2198 2024-09-16T20:49:10.315 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.315 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:10.315 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.315 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:10.315 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335942+0000", 2024-09-16T20:49:10.315 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:10.316 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.316 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.316 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:10.316 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335942+0000", 2024-09-16T20:49:10.316 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:10.316 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.316 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.316 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:10.316 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335943+0000", 2024-09-16T20:49:10.316 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7099999999999997e-07 2024-09-16T20:49:10.316 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.316 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.316 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:10.317 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335945+0000", 2024-09-16T20:49:10.317 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3499999999999999e-06 2024-09-16T20:49:10.317 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.317 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.317 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:10.317 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335952+0000", 2024-09-16T20:49:10.317 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4309999999999999e-06 2024-09-16T20:49:10.317 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.317 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.317 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:10.317 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335957+0000", 2024-09-16T20:49:10.317 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0869999999999997e-06 2024-09-16T20:49:10.317 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.317 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.317 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:10.318 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385312+0000", 2024-09-16T20:49:10.318 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049355121000000002 2024-09-16T20:49:10.318 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.318 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.318 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:10.318 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385405+0000", 2024-09-16T20:49:10.318 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.3344000000000006e-05 2024-09-16T20:49:10.318 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.318 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.318 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:10.318 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385505+0000", 2024-09-16T20:49:10.318 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.9779999999999997e-05 2024-09-16T20:49:10.318 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:10.318 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:10.318 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:10.319 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.319 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.319 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2201 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1030144~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:10.319 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335976+0000", 2024-09-16T20:49:10.319 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 47.938700183999998, 2024-09-16T20:49:10.319 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049740077000000001, 2024-09-16T20:49:10.319 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:10.319 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:10.319 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:10.319 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:10.319 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:10.319 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:10.320 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2201 2024-09-16T20:49:10.320 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.320 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:10.320 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.320 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:10.320 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335976+0000", 2024-09-16T20:49:10.320 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:10.320 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.320 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.320 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:10.320 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335976+0000", 2024-09-16T20:49:10.320 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:10.320 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.320 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.321 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:10.321 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335976+0000", 2024-09-16T20:49:10.321 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8599999999999998e-07 2024-09-16T20:49:10.321 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.321 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.321 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:10.321 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335978+0000", 2024-09-16T20:49:10.321 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1500000000000002e-06 2024-09-16T20:49:10.321 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.321 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.321 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:10.321 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335986+0000", 2024-09-16T20:49:10.321 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3989999999999998e-06 2024-09-16T20:49:10.321 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.322 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.322 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:10.322 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335992+0000", 2024-09-16T20:49:10.322 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.682e-06 2024-09-16T20:49:10.322 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.322 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.322 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:10.322 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385537+0000", 2024-09-16T20:49:10.322 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049545828 2024-09-16T20:49:10.322 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.322 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.322 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:10.322 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385619+0000", 2024-09-16T20:49:10.322 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1720999999999999e-05 2024-09-16T20:49:10.323 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.323 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.323 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:10.323 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385716+0000", 2024-09-16T20:49:10.323 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.6811000000000005e-05 2024-09-16T20:49:10.323 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:10.323 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:10.323 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:10.323 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.323 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.323 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2203 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1038336~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:10.323 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.336000+0000", 2024-09-16T20:49:10.323 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 47.938676397999998, 2024-09-16T20:49:10.323 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049949888999999997, 2024-09-16T20:49:10.324 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:10.324 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:10.324 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:10.324 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:10.324 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:10.324 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:10.324 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2203 2024-09-16T20:49:10.324 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.324 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:10.324 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.324 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:10.324 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336000+0000", 2024-09-16T20:49:10.324 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:10.325 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.325 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.325 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:10.325 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336000+0000", 2024-09-16T20:49:10.325 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:10.325 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.325 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.325 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:10.325 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336000+0000", 2024-09-16T20:49:10.325 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.58e-07 2024-09-16T20:49:10.325 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.325 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.325 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:10.326 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336002+0000", 2024-09-16T20:49:10.326 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0040000000000002e-06 2024-09-16T20:49:10.326 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.326 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.326 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:10.326 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336009+0000", 2024-09-16T20:49:10.326 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9890000000000003e-06 2024-09-16T20:49:10.326 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.326 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.326 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:10.326 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336014+0000", 2024-09-16T20:49:10.326 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4e-06 2024-09-16T20:49:10.326 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.326 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.326 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:10.327 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385759+0000", 2024-09-16T20:49:10.327 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049744151 2024-09-16T20:49:10.327 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.327 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.327 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:10.327 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385847+0000", 2024-09-16T20:49:10.327 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8189000000000006e-05 2024-09-16T20:49:10.327 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.327 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.327 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:10.327 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385950+0000", 2024-09-16T20:49:10.327 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102698 2024-09-16T20:49:10.327 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:10.327 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:10.327 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:10.328 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.328 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.328 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2206 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1046528~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:10.328 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:49:10.328 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 47.938654237000002, 2024-09-16T20:49:10.328 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050147714000000003, 2024-09-16T20:49:10.328 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:10.328 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:10.328 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:10.328 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:10.328 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:10.328 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:10.328 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2206 2024-09-16T20:49:10.328 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.329 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:10.329 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.329 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:10.329 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:49:10.329 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:10.329 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.329 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.329 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:10.329 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:49:10.329 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:10.329 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.329 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.329 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:10.329 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:49:10.330 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8599999999999999e-07 2024-09-16T20:49:10.330 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.330 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.330 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:10.330 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336024+0000", 2024-09-16T20:49:10.330 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.934e-06 2024-09-16T20:49:10.330 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.330 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.330 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:10.330 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336037+0000", 2024-09-16T20:49:10.330 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2496e-05 2024-09-16T20:49:10.330 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.330 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.330 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:10.330 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336042+0000", 2024-09-16T20:49:10.331 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4809999999999997e-06 2024-09-16T20:49:10.331 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.331 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.331 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:10.331 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385980+0000", 2024-09-16T20:49:10.331 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049938376999999999 2024-09-16T20:49:10.331 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.331 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.331 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:10.331 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.386069+0000", 2024-09-16T20:49:10.331 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8368999999999994e-05 2024-09-16T20:49:10.331 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.331 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.331 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:10.332 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.386170+0000", 2024-09-16T20:49:10.332 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000100671 2024-09-16T20:49:10.332 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:10.332 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:10.332 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:10.332 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.332 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.332 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5439 3.a 3:5789f5c3:::rbd_data.110ed9ca540.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 1181696~60416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:10.332 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:49:10.332 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 47.409965657000001, 2024-09-16T20:49:10.332 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10102045599999999, 2024-09-16T20:49:10.333 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:10.333 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:10.333 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:10.333 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:10.333 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:10.333 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:10.333 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5439 2024-09-16T20:49:10.333 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.333 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:10.333 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.333 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:10.333 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:49:10.333 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:10.334 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.334 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.334 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:10.334 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:49:10.334 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:10.334 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.334 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.334 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:10.334 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864711+0000", 2024-09-16T20:49:10.334 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7899999999999997e-07 2024-09-16T20:49:10.334 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.334 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.334 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:10.334 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864760+0000", 2024-09-16T20:49:10.334 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8803e-05 2024-09-16T20:49:10.334 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.335 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.335 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:10.335 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864769+0000", 2024-09-16T20:49:10.335 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0999999999999993e-06 2024-09-16T20:49:10.335 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.335 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.335 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:10.335 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864774+0000", 2024-09-16T20:49:10.335 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8919999999999999e-06 2024-09-16T20:49:10.335 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.335 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.335 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:10.335 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864815+0000", 2024-09-16T20:49:10.335 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1307000000000003e-05 2024-09-16T20:49:10.335 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.336 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.336 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:10.336 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864893+0000", 2024-09-16T20:49:10.336 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7581000000000001e-05 2024-09-16T20:49:10.336 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.336 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.336 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 1", 2024-09-16T20:49:10.336 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864996+0000", 2024-09-16T20:49:10.336 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102725 2024-09-16T20:49:10.336 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.336 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.336 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:10.336 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965442+0000", 2024-09-16T20:49:10.336 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100446012 2024-09-16T20:49:10.337 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.337 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.337 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:10.337 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965618+0000", 2024-09-16T20:49:10.337 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00017653100000000001 2024-09-16T20:49:10.337 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.337 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.337 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:10.337 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965682+0000", 2024-09-16T20:49:10.337 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3905999999999994e-05 2024-09-16T20:49:10.337 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.337 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.337 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:10.337 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965731+0000", 2024-09-16T20:49:10.338 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8819999999999997e-05 2024-09-16T20:49:10.338 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:10.338 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:10.338 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:10.338 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.338 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.338 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4600.0:5439 3.a e37/23 ondisk, result = 0)", 2024-09-16T20:49:10.338 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:49:10.338 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 47.408969509000002, 2024-09-16T20:49:10.338 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099742634999999996, 2024-09-16T20:49:10.338 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:10.338 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:10.338 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:10.338 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:10.339 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.339 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:10.339 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:49:10.339 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:10.339 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.339 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.339 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:10.339 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:49:10.339 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:10.339 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.339 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.339 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:10.339 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:49:10.340 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0900000000000001e-07 2024-09-16T20:49:10.340 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.340 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.340 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:10.340 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865711+0000", 2024-09-16T20:49:10.340 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3739999999999998e-06 2024-09-16T20:49:10.340 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.340 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.340 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:10.340 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965343+0000", 2024-09-16T20:49:10.340 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099632526999999999 2024-09-16T20:49:10.340 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.340 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.340 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:10.341 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965357+0000", 2024-09-16T20:49:10.341 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3713999999999999e-05 2024-09-16T20:49:10.341 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.341 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.341 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:10.341 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965426+0000", 2024-09-16T20:49:10.341 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8745999999999995e-05 2024-09-16T20:49:10.341 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.341 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.341 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:10.341 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965434+0000", 2024-09-16T20:49:10.341 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6910000000000003e-06 2024-09-16T20:49:10.341 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.341 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.342 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:10.342 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965449+0000", 2024-09-16T20:49:10.342 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4874e-05 2024-09-16T20:49:10.342 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:10.342 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:10.342 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:10.342 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.342 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.342 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4882.0:2508 3.12 3:4992771f:::rbd_object_map.110e7c8851c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T20:49:10.342 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:49:10.342 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.793165961, 2024-09-16T20:49:10.342 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.125007011, 2024-09-16T20:49:10.342 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:10.342 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:10.343 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:10.343 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:10.343 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4882", 2024-09-16T20:49:10.343 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2436923223", 2024-09-16T20:49:10.343 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2508 2024-09-16T20:49:10.343 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.343 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:10.343 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.343 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:10.343 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:49:10.343 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:10.343 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.343 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.343 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:10.344 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:49:10.344 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:10.344 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.344 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.344 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:10.344 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481511+0000", 2024-09-16T20:49:10.344 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0699999999999996e-07 2024-09-16T20:49:10.344 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.344 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.344 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:10.344 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481515+0000", 2024-09-16T20:49:10.344 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0609999999999997e-06 2024-09-16T20:49:10.344 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.345 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.345 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:10.345 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481523+0000", 2024-09-16T20:49:10.345 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1310000000000006e-06 2024-09-16T20:49:10.345 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.345 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.345 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:10.345 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481528+0000", 2024-09-16T20:49:10.345 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4900000000000002e-06 2024-09-16T20:49:10.345 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.345 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.345 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:10.345 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481565+0000", 2024-09-16T20:49:10.345 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7066000000000001e-05 2024-09-16T20:49:10.346 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.346 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.346 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:10.346 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481628+0000", 2024-09-16T20:49:10.346 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3118000000000001e-05 2024-09-16T20:49:10.346 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.346 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.346 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 2", 2024-09-16T20:49:10.346 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481822+0000", 2024-09-16T20:49:10.346 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019446200000000001 2024-09-16T20:49:10.346 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.346 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.346 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:10.346 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605988+0000", 2024-09-16T20:49:10.346 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.124165888 2024-09-16T20:49:10.347 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.347 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.347 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:10.347 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606427+0000", 2024-09-16T20:49:10.347 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00043854899999999999 2024-09-16T20:49:10.347 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.347 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.347 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:10.347 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606476+0000", 2024-09-16T20:49:10.347 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8971999999999998e-05 2024-09-16T20:49:10.347 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.347 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.347 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:10.347 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606517+0000", 2024-09-16T20:49:10.347 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1467000000000001e-05 2024-09-16T20:49:10.348 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:10.348 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:10.348 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:10.348 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.348 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.348 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4882.0:2508 3.12 e57/55 ondisk, result = 0)", 2024-09-16T20:49:10.348 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:49:10.348 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.792177461000001, 2024-09-16T20:49:10.348 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12349711400000001, 2024-09-16T20:49:10.348 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:10.348 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:10.348 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:10.348 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:10.348 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.349 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:10.349 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:49:10.349 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:10.349 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.349 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.349 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:10.349 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:49:10.349 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:10.349 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.349 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.349 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:10.349 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:49:10.349 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.06e-07 2024-09-16T20:49:10.349 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.350 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.350 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:10.350 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482502+0000", 2024-09-16T20:49:10.350 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.01e-06 2024-09-16T20:49:10.350 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.350 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.350 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:10.350 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605772+0000", 2024-09-16T20:49:10.350 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.123269687 2024-09-16T20:49:10.350 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.350 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.350 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:10.350 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605791+0000", 2024-09-16T20:49:10.350 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8912000000000001e-05 2024-09-16T20:49:10.350 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.350 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.351 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:10.351 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605974+0000", 2024-09-16T20:49:10.351 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000182836 2024-09-16T20:49:10.351 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.351 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.351 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:10.351 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605982+0000", 2024-09-16T20:49:10.351 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1699999999999997e-06 2024-09-16T20:49:10.351 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.351 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.351 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:10.351 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605996+0000", 2024-09-16T20:49:10.351 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3893000000000001e-05 2024-09-16T20:49:10.351 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:10.351 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:10.352 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:10.352 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.352 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.352 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4882.0:2512 3.f 3:f4d895b7:::rbd_data.110e7c8851c.0000000000000000:head [read 16384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T20:49:10.352 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:49:10.352 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.791740431000001, 2024-09-16T20:49:10.352 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12318483, 2024-09-16T20:49:10.352 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:10.352 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:10.352 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:10.352 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:10.352 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4882", 2024-09-16T20:49:10.352 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2436923223", 2024-09-16T20:49:10.352 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2512 2024-09-16T20:49:10.352 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.353 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:10.353 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.353 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:10.353 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:49:10.353 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:10.353 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.353 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.353 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:10.353 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:49:10.353 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:10.353 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.353 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.353 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:10.353 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:49:10.353 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3499999999999995e-07 2024-09-16T20:49:10.354 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.354 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.354 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:10.354 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482943+0000", 2024-09-16T20:49:10.354 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3450000000000002e-06 2024-09-16T20:49:10.354 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.354 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.354 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:10.354 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605772+0000", 2024-09-16T20:49:10.354 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.122829796 2024-09-16T20:49:10.354 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.354 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.354 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:10.354 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605792+0000", 2024-09-16T20:49:10.354 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9916999999999999e-05 2024-09-16T20:49:10.355 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.355 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.355 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:10.355 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605917+0000", 2024-09-16T20:49:10.355 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00012500799999999999 2024-09-16T20:49:10.355 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.355 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.355 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:10.355 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606004+0000", 2024-09-16T20:49:10.355 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6242999999999994e-05 2024-09-16T20:49:10.355 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.355 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.355 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:10.355 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606120+0000", 2024-09-16T20:49:10.355 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000116786 2024-09-16T20:49:10.356 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:10.356 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:10.356 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:10.356 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.356 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.356 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:11702 3.15 3:acf33c8b:::rbd_data.110e747f0779.000000000000001a:head [set-alloc-hint object_size 262144 write_size 262144,write 143360~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:49:10.356 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:49:10.356 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 6.1367480089999997, 2024-09-16T20:49:10.356 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.084801043000000006, 2024-09-16T20:49:10.356 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:10.356 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:10.356 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:10.356 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:10.357 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:49:10.357 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:49:10.357 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 11702 2024-09-16T20:49:10.357 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.357 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:10.357 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.357 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:10.357 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:49:10.357 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:10.357 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.357 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.357 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:10.357 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:49:10.357 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:10.358 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.358 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.358 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:10.358 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137929+0000", 2024-09-16T20:49:10.358 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8400000000000002e-07 2024-09-16T20:49:10.358 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.358 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.358 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:10.358 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137937+0000", 2024-09-16T20:49:10.358 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0260000000000007e-06 2024-09-16T20:49:10.358 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.358 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.358 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:10.359 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137952+0000", 2024-09-16T20:49:10.359 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4659e-05 2024-09-16T20:49:10.359 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.359 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.359 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:10.359 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137960+0000", 2024-09-16T20:49:10.359 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0169999999999996e-06 2024-09-16T20:49:10.359 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.359 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.359 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:10.359 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138010+0000", 2024-09-16T20:49:10.359 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0021000000000001e-05 2024-09-16T20:49:10.359 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.359 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.360 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:10.360 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138114+0000", 2024-09-16T20:49:10.360 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000103797 2024-09-16T20:49:10.360 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.360 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.360 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:49:10.360 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138233+0000", 2024-09-16T20:49:10.360 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000118971 2024-09-16T20:49:10.360 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.360 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.360 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:10.360 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138953+0000", 2024-09-16T20:49:10.360 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00072026000000000002 2024-09-16T20:49:10.360 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.361 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.361 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:10.361 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222609+0000", 2024-09-16T20:49:10.361 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.083656650999999999 2024-09-16T20:49:10.361 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.361 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.361 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:10.361 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222684+0000", 2024-09-16T20:49:10.361 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4369999999999994e-05 2024-09-16T20:49:10.361 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.361 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.361 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:10.361 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222729+0000", 2024-09-16T20:49:10.361 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5287000000000002e-05 2024-09-16T20:49:10.362 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:10.362 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:10.362 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:10.362 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.362 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.362 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:11704 3.9 3:91220b75:::rbd_data.110e747f0779.000000000000001c:head [set-alloc-hint object_size 262144 write_size 262144,write 139264~6656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:49:10.362 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:49:10.362 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 6.1365100080000001, 2024-09-16T20:49:10.362 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.084531133999999994, 2024-09-16T20:49:10.362 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:10.362 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:10.362 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:10.362 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:10.362 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:49:10.362 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:49:10.362 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 11704 2024-09-16T20:49:10.363 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.363 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:10.363 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.363 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:10.363 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:49:10.363 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:10.363 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.363 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.363 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:10.363 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:49:10.363 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:10.363 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.363 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.364 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:10.364 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138167+0000", 2024-09-16T20:49:10.364 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.33e-07 2024-09-16T20:49:10.364 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.364 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.364 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:10.364 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138179+0000", 2024-09-16T20:49:10.364 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2055999999999999e-05 2024-09-16T20:49:10.364 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.364 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.364 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:10.364 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138193+0000", 2024-09-16T20:49:10.365 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3961000000000001e-05 2024-09-16T20:49:10.365 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.365 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.365 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:10.365 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138200+0000", 2024-09-16T20:49:10.365 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4429999999999997e-06 2024-09-16T20:49:10.365 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.365 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.365 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:10.365 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138258+0000", 2024-09-16T20:49:10.365 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7225999999999997e-05 2024-09-16T20:49:10.365 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.365 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.365 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:10.366 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138364+0000", 2024-09-16T20:49:10.366 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000106013 2024-09-16T20:49:10.366 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.366 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.366 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:49:10.366 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138494+0000", 2024-09-16T20:49:10.366 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013059599999999999 2024-09-16T20:49:10.366 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.366 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.366 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:10.366 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.139220+0000", 2024-09-16T20:49:10.366 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00072539800000000004 2024-09-16T20:49:10.366 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.367 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.367 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:10.367 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222608+0000", 2024-09-16T20:49:10.367 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.083388620999999996 2024-09-16T20:49:10.367 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.367 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.367 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:10.367 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222651+0000", 2024-09-16T20:49:10.367 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2228000000000003e-05 2024-09-16T20:49:10.367 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:10.367 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:10.367 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:10.367 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222697+0000", 2024-09-16T20:49:10.367 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6659000000000002e-05 2024-09-16T20:49:10.368 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:10.368 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:10.368 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:10.368 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:10.368 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:10.368 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:10.369 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:49:10.370 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:10.368+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:10.470 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:10.469+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:10.495 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T20:49:10.518 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:10.518 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:49:10.518 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:49:10.518 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:10.527 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:49:10.554 INFO:teuthology.orchestra.run.smithi029.stderr:nodeep-scrub is set 2024-09-16T20:49:10.573 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:10.572+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:10.625 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:49:10.625 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:49:10.642 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:49:10.643 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:49:10.660 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:49:10.660 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:49:10.669 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:10.669 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:10.671 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:10.670+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:10.678 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:49:10.678 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:49:10.679 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:49:10.695 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:49:10.695 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:49:10.712 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:49:10.713 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:49:10.772 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:10.771+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:10.808 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:10.808 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:10.818 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:49:10.873 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:10.872+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:10.970 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:10.970 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:10.974 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:10.973+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:10.979 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:49:11.074 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:11.074+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:11.095 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:11.095 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:11.105 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:49:11.175 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:11.175+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:11.177 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #20, image image_client.0-clone20 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent21 2024-09-16T20:49:11.222 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:11.223 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:11.231 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:49:11.276 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:11.275+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:11.338 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:11.339 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:11.347 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:49:11.377 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:11.376+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:11.478 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:11.477+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:11.509 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:11.509 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:11.519 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:49:11.579 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:11.578+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:11.644 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:11.645 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:11.653 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:49:11.694 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:11.694+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:11.780 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:11.779+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:11.818 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:11.818 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:11.827 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:49:11.881 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:11.880+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:11.933 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:11.933 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:11.942 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:49:11.982 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:11.981+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:12.048 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:12.048 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:12.056 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:49:12.083 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:12.082+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:12.163 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:12.164 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:12.172 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:49:12.184 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:12.183+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:12.279 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:12.279 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:12.285 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:12.284+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:12.287 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:49:12.386 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:12.385+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:12.434 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:12.434 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:12.443 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:49:12.486 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:12.486+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:12.563 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:12.563 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:12.572 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:49:12.574 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T20:49:12.617 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:12.616+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:12.682 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:12.682 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:12.688 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:12.687+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:12.692 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:49:12.723 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T20:49:12.789 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:12.789+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:12.863 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:12.863 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:12.876 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:49:12.890 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:12.889+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:12.895 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:49:12.896 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:49:12.932 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:49:12.932 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:49:12.971 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:49:12.971 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:49:12.991 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:12.989+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:13.005 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:49:13.005 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:49:13.035 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:49:13.035 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:49:13.050 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:13.050 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:49:13.050 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.050 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5224.0:14053 3.d 3.597eefed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-16T20:49:13.050 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:13.049454+0000", 2024-09-16T20:49:13.050 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.00039957399999999998, 2024-09-16T20:49:13.050 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000414986, 2024-09-16T20:49:13.050 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:13.051 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:13.051 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:49:13.051 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:13.051 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5224", 2024-09-16T20:49:13.051 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3740120172", 2024-09-16T20:49:13.051 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 14053 2024-09-16T20:49:13.051 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.051 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:13.051 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.051 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:13.051 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:13.049454+0000", 2024-09-16T20:49:13.051 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:13.051 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.051 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.052 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:13.052 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:13.049454+0000", 2024-09-16T20:49:13.052 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:13.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.052 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.052 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:13.052 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:13.049455+0000", 2024-09-16T20:49:13.052 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4799999999999998e-07 2024-09-16T20:49:13.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.052 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.052 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:13.052 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:13.049457+0000", 2024-09-16T20:49:13.052 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3609999999999999e-06 2024-09-16T20:49:13.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.053 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.053 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:13.053 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:13.049463+0000", 2024-09-16T20:49:13.053 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8189999999999997e-06 2024-09-16T20:49:13.053 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.053 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.053 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:13.053 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:13.049468+0000", 2024-09-16T20:49:13.053 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1150000000000002e-06 2024-09-16T20:49:13.053 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.053 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.053 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:13.053 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:13.049867+0000", 2024-09-16T20:49:13.053 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000398942 2024-09-16T20:49:13.054 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.054 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.054 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:13.054 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:13.049871+0000", 2024-09-16T20:49:13.054 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1919999999999998e-06 2024-09-16T20:49:13.054 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:13.054 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:13.054 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:13.054 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:13.054 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-16T20:49:13.054 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 1 2024-09-16T20:49:13.054 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:13.062 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:49:13.103 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:13.102+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:13.192 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:13.191+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:13.263 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:49:13.263 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:49:13.266 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:13.266 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:49:13.266 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:49:13.266 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:49:13.266 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:13.278 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:49:13.319 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:13.318+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:13.394 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:13.393+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:13.478 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:13.479 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:49:13.479 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:49:13.479 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:49:13.479 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.479 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:49:13.479 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:49:13.479 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 80.221732012000004, 2024-09-16T20:49:13.479 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67502346999999996, 2024-09-16T20:49:13.479 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:13.479 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:13.479 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:13.479 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:13.479 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:49:13.480 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:49:13.480 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 1 2024-09-16T20:49:13.480 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.480 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:13.480 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.480 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:13.480 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:49:13.480 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:13.480 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.480 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.480 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:13.480 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:49:13.480 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:13.481 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.481 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.481 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:13.481 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254128+0000", 2024-09-16T20:49:13.481 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.463e-06 2024-09-16T20:49:13.481 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.481 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.481 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:13.481 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254135+0000", 2024-09-16T20:49:13.481 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1579999999999999e-06 2024-09-16T20:49:13.481 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.481 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.481 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:13.481 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254146+0000", 2024-09-16T20:49:13.482 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1341e-05 2024-09-16T20:49:13.482 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.482 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.482 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:13.482 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254155+0000", 2024-09-16T20:49:13.482 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.1549999999999996e-06 2024-09-16T20:49:13.482 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.482 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.482 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:13.482 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254221+0000", 2024-09-16T20:49:13.482 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5470999999999997e-05 2024-09-16T20:49:13.482 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.482 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.482 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:49:13.483 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254231+0000", 2024-09-16T20:49:13.483 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0592e-05 2024-09-16T20:49:13.483 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.483 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.483 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:13.483 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926711+0000", 2024-09-16T20:49:13.483 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67247946599999997 2024-09-16T20:49:13.483 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.483 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.483 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:13.483 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926910+0000", 2024-09-16T20:49:13.483 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019934 2024-09-16T20:49:13.483 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.484 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.484 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:49:13.484 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.927159+0000", 2024-09-16T20:49:13.484 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00024876599999999998 2024-09-16T20:49:13.484 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.484 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.484 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:13.484 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.928797+0000", 2024-09-16T20:49:13.484 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001638308 2024-09-16T20:49:13.484 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.484 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.484 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:13.484 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929026+0000", 2024-09-16T20:49:13.484 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00022876099999999999 2024-09-16T20:49:13.485 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.485 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.485 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:13.485 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929102+0000", 2024-09-16T20:49:13.485 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6184000000000001e-05 2024-09-16T20:49:13.485 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.485 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.485 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:13.485 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929150+0000", 2024-09-16T20:49:13.485 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7465e-05 2024-09-16T20:49:13.485 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:13.485 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:13.485 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:13.485 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.486 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.486 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4319 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4675584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:13.486 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745838+0000", 2024-09-16T20:49:13.486 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 53.730020201000002, 2024-09-16T20:49:13.486 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047640204999999998, 2024-09-16T20:49:13.486 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:13.486 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:13.486 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:13.486 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:13.486 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:49:13.486 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:13.486 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4319 2024-09-16T20:49:13.486 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.486 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:13.486 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.487 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:13.487 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745838+0000", 2024-09-16T20:49:13.487 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:13.487 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.487 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.487 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:13.487 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745838+0000", 2024-09-16T20:49:13.487 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:13.487 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.487 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.487 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:13.487 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745838+0000", 2024-09-16T20:49:13.487 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6899999999999998e-07 2024-09-16T20:49:13.487 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.487 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.488 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:13.488 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745840+0000", 2024-09-16T20:49:13.488 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0660000000000002e-06 2024-09-16T20:49:13.488 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.488 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.488 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:13.488 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745847+0000", 2024-09-16T20:49:13.488 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8689999999999998e-06 2024-09-16T20:49:13.488 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.488 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.488 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:13.488 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745853+0000", 2024-09-16T20:49:13.488 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2399999999999998e-06 2024-09-16T20:49:13.489 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.489 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.489 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:13.489 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793381+0000", 2024-09-16T20:49:13.489 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047528005999999998 2024-09-16T20:49:13.489 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.489 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.489 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:13.489 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793423+0000", 2024-09-16T20:49:13.489 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2799000000000001e-05 2024-09-16T20:49:13.489 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.489 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.489 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:13.489 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793478+0000", 2024-09-16T20:49:13.489 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4756000000000002e-05 2024-09-16T20:49:13.490 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:13.490 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:13.490 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:13.490 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.490 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.490 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4320 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4683776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:13.490 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745859+0000", 2024-09-16T20:49:13.490 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 53.729998956999999, 2024-09-16T20:49:13.490 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047811399999999997, 2024-09-16T20:49:13.490 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:13.490 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:13.490 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:13.490 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:13.490 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:49:13.491 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:13.491 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4320 2024-09-16T20:49:13.491 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.491 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:13.491 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.491 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:13.491 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745859+0000", 2024-09-16T20:49:13.491 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:13.491 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.491 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.491 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:13.491 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745859+0000", 2024-09-16T20:49:13.491 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:13.491 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.492 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.492 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:13.492 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745860+0000", 2024-09-16T20:49:13.492 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9099999999999999e-07 2024-09-16T20:49:13.492 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.492 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.492 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:13.492 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745861+0000", 2024-09-16T20:49:13.492 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8589999999999999e-06 2024-09-16T20:49:13.492 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.492 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.492 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:13.493 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745868+0000", 2024-09-16T20:49:13.493 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2249999999999997e-06 2024-09-16T20:49:13.493 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.493 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.493 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:13.493 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745873+0000", 2024-09-16T20:49:13.493 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1290000000000004e-06 2024-09-16T20:49:13.493 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.493 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.493 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:13.493 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793522+0000", 2024-09-16T20:49:13.493 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047648847000000001 2024-09-16T20:49:13.493 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.493 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.493 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:13.494 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793603+0000", 2024-09-16T20:49:13.494 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1167000000000004e-05 2024-09-16T20:49:13.494 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.494 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.494 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:13.494 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793671+0000", 2024-09-16T20:49:13.494 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7781999999999996e-05 2024-09-16T20:49:13.494 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:13.494 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:13.494 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:13.494 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.494 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.494 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4321 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4691968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:13.495 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745879+0000", 2024-09-16T20:49:13.495 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 53.729978760999998, 2024-09-16T20:49:13.495 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047950375000000003, 2024-09-16T20:49:13.495 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:13.495 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:13.495 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:13.495 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:13.495 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:49:13.495 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:13.495 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4321 2024-09-16T20:49:13.495 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.495 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:13.495 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.495 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:13.496 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745879+0000", 2024-09-16T20:49:13.496 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:13.496 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.496 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.496 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:13.496 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745879+0000", 2024-09-16T20:49:13.496 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:13.496 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.496 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.496 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:13.496 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745880+0000", 2024-09-16T20:49:13.496 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.65e-07 2024-09-16T20:49:13.496 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.497 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.497 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:13.497 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745882+0000", 2024-09-16T20:49:13.497 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.922e-06 2024-09-16T20:49:13.497 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.497 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.497 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:13.497 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745888+0000", 2024-09-16T20:49:13.497 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0510000000000002e-06 2024-09-16T20:49:13.497 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.497 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.497 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:13.497 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745893+0000", 2024-09-16T20:49:13.497 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9579999999999998e-06 2024-09-16T20:49:13.497 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.498 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.498 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:13.498 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793704+0000", 2024-09-16T20:49:13.498 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047811179000000002 2024-09-16T20:49:13.498 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.498 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.498 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:13.498 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793759+0000", 2024-09-16T20:49:13.498 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5260999999999998e-05 2024-09-16T20:49:13.498 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.498 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.498 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:13.498 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793830+0000", 2024-09-16T20:49:13.498 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0638999999999999e-05 2024-09-16T20:49:13.499 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:13.499 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:13.499 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:13.499 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.499 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.499 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4322 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4700160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:13.499 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745899+0000", 2024-09-16T20:49:13.499 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 53.729959323000003, 2024-09-16T20:49:13.499 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048057280000000001, 2024-09-16T20:49:13.499 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:13.499 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:13.499 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:13.499 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:13.499 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:49:13.500 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:13.500 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4322 2024-09-16T20:49:13.500 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.500 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:13.500 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.500 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:13.500 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745899+0000", 2024-09-16T20:49:13.500 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:13.500 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.500 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.500 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:13.500 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745899+0000", 2024-09-16T20:49:13.500 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:13.500 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.501 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.501 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:13.501 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745899+0000", 2024-09-16T20:49:13.501 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9299999999999999e-07 2024-09-16T20:49:13.501 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.501 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.501 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:13.501 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745901+0000", 2024-09-16T20:49:13.501 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0030000000000001e-06 2024-09-16T20:49:13.501 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.501 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.501 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:13.501 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745908+0000", 2024-09-16T20:49:13.501 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3729999999999998e-06 2024-09-16T20:49:13.502 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.502 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.502 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:13.502 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745914+0000", 2024-09-16T20:49:13.502 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.122e-06 2024-09-16T20:49:13.502 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.502 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.502 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:13.502 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793862+0000", 2024-09-16T20:49:13.502 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047948324 2024-09-16T20:49:13.502 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.502 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.502 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:13.502 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793903+0000", 2024-09-16T20:49:13.503 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0465e-05 2024-09-16T20:49:13.503 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.503 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.503 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:13.503 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793956+0000", 2024-09-16T20:49:13.503 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3600000000000002e-05 2024-09-16T20:49:13.503 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:13.503 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:13.503 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:13.503 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.503 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.503 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4323 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4708352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:13.503 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745920+0000", 2024-09-16T20:49:13.503 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 53.729937681999999, 2024-09-16T20:49:13.504 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048268138000000002, 2024-09-16T20:49:13.504 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:13.504 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:13.504 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:13.504 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:13.504 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:49:13.504 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:13.504 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4323 2024-09-16T20:49:13.504 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.504 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:13.504 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.504 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:13.504 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745920+0000", 2024-09-16T20:49:13.504 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:13.505 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.505 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.505 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:13.505 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745920+0000", 2024-09-16T20:49:13.505 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:13.505 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.505 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.505 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:13.505 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745921+0000", 2024-09-16T20:49:13.505 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7e-07 2024-09-16T20:49:13.505 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.505 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.505 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:13.505 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745923+0000", 2024-09-16T20:49:13.506 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7880000000000001e-06 2024-09-16T20:49:13.506 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.506 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.506 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:13.506 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745929+0000", 2024-09-16T20:49:13.506 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0809999999999999e-06 2024-09-16T20:49:13.506 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.506 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.506 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:13.506 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745935+0000", 2024-09-16T20:49:13.506 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5370000000000003e-06 2024-09-16T20:49:13.506 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.506 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.506 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:13.507 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793987+0000", 2024-09-16T20:49:13.507 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048051736999999997 2024-09-16T20:49:13.507 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.507 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.507 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:13.507 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.794070+0000", 2024-09-16T20:49:13.507 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.3277999999999998e-05 2024-09-16T20:49:13.507 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.507 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.507 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:13.507 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.794189+0000", 2024-09-16T20:49:13.507 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000118347 2024-09-16T20:49:13.507 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:13.507 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:13.508 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:13.508 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.508 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.508 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4324 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4716544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:13.508 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745946+0000", 2024-09-16T20:49:13.508 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 53.729912110999997, 2024-09-16T20:49:13.508 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048468153999999999, 2024-09-16T20:49:13.508 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:13.508 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:13.508 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:13.508 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:13.508 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:49:13.508 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:13.509 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4324 2024-09-16T20:49:13.509 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.509 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:13.509 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.509 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:13.509 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745946+0000", 2024-09-16T20:49:13.509 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:13.509 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.509 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.509 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:13.509 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745946+0000", 2024-09-16T20:49:13.509 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:13.509 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.509 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.510 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:13.510 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745946+0000", 2024-09-16T20:49:13.510 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4499999999999997e-07 2024-09-16T20:49:13.510 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.510 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.510 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:13.510 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745949+0000", 2024-09-16T20:49:13.510 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2529999999999999e-06 2024-09-16T20:49:13.510 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.510 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.510 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:13.510 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745955+0000", 2024-09-16T20:49:13.510 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6470000000000001e-06 2024-09-16T20:49:13.510 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.510 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.511 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:13.511 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745960+0000", 2024-09-16T20:49:13.511 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7910000000000004e-06 2024-09-16T20:49:13.511 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.511 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.511 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:13.511 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.794220+0000", 2024-09-16T20:49:13.511 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048260069000000003 2024-09-16T20:49:13.511 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.511 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.511 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:13.511 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.794311+0000", 2024-09-16T20:49:13.511 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0593000000000005e-05 2024-09-16T20:49:13.511 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.512 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.512 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:13.512 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.794414+0000", 2024-09-16T20:49:13.512 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000103356 2024-09-16T20:49:13.512 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:13.512 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:13.512 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:13.512 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.512 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.512 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3571 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:13.512 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481059+0000", 2024-09-16T20:49:13.512 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.994798951, 2024-09-16T20:49:13.512 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049572995000000002, 2024-09-16T20:49:13.512 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:13.513 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:13.513 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:13.513 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:13.513 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:13.513 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:13.513 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3571 2024-09-16T20:49:13.513 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.513 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:13.513 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.513 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:13.513 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481059+0000", 2024-09-16T20:49:13.513 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:13.513 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.514 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.514 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:13.514 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481059+0000", 2024-09-16T20:49:13.514 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:13.514 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.514 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.514 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:13.514 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481060+0000", 2024-09-16T20:49:13.514 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2599999999999998e-07 2024-09-16T20:49:13.514 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.514 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.514 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:13.514 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481063+0000", 2024-09-16T20:49:13.514 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.999e-06 2024-09-16T20:49:13.515 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.515 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.515 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:13.515 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481067+0000", 2024-09-16T20:49:13.515 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3080000000000001e-06 2024-09-16T20:49:13.515 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.515 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.515 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:13.515 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481070+0000", 2024-09-16T20:49:13.515 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9919999999999999e-06 2024-09-16T20:49:13.515 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.515 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.515 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:13.515 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.522087+0000", 2024-09-16T20:49:13.516 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.041017023 2024-09-16T20:49:13.516 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.516 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.516 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:13.516 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.522191+0000", 2024-09-16T20:49:13.516 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010389499999999999 2024-09-16T20:49:13.516 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.516 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.516 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:13.516 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.522566+0000", 2024-09-16T20:49:13.516 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00037539399999999998 2024-09-16T20:49:13.516 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.516 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.516 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:13.517 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525494+0000", 2024-09-16T20:49:13.517 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0029273070000000001 2024-09-16T20:49:13.517 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.517 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.517 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:13.517 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530355+0000", 2024-09-16T20:49:13.517 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0048612509999999996 2024-09-16T20:49:13.517 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.517 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.517 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:13.517 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530407+0000", 2024-09-16T20:49:13.517 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1817000000000002e-05 2024-09-16T20:49:13.517 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.517 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.518 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:13.518 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530632+0000", 2024-09-16T20:49:13.518 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00022558300000000001 2024-09-16T20:49:13.518 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:13.518 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:13.518 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:13.518 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.518 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.518 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3572 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:13.518 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481220+0000", 2024-09-16T20:49:13.518 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.994638383000002, 2024-09-16T20:49:13.518 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050452862000000001, 2024-09-16T20:49:13.518 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:13.518 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:13.519 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:13.519 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:13.519 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:13.519 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:13.519 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3572 2024-09-16T20:49:13.519 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.519 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:13.519 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.519 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:13.519 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481220+0000", 2024-09-16T20:49:13.519 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:13.519 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.519 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.519 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:13.519 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481220+0000", 2024-09-16T20:49:13.520 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:13.520 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.520 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.520 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:13.520 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481221+0000", 2024-09-16T20:49:13.520 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9400000000000004e-07 2024-09-16T20:49:13.520 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.520 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.520 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:13.520 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481225+0000", 2024-09-16T20:49:13.520 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0230000000000003e-06 2024-09-16T20:49:13.520 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.520 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.521 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:13.521 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481234+0000", 2024-09-16T20:49:13.521 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2920000000000001e-06 2024-09-16T20:49:13.521 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.521 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.521 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:13.521 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481240+0000", 2024-09-16T20:49:13.521 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9390000000000002e-06 2024-09-16T20:49:13.521 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.521 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.521 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:13.521 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.523544+0000", 2024-09-16T20:49:13.521 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.042304475000000001 2024-09-16T20:49:13.522 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.522 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.522 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:13.522 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.523624+0000", 2024-09-16T20:49:13.522 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.975e-05 2024-09-16T20:49:13.522 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.522 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.522 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:13.522 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.523975+0000", 2024-09-16T20:49:13.522 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00035063200000000003 2024-09-16T20:49:13.522 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.522 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.522 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:13.523 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530337+0000", 2024-09-16T20:49:13.523 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0063623589999999997 2024-09-16T20:49:13.523 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.523 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.523 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:13.523 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531592+0000", 2024-09-16T20:49:13.523 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0012550599999999999 2024-09-16T20:49:13.523 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.523 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.523 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:13.523 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531633+0000", 2024-09-16T20:49:13.523 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1192000000000001e-05 2024-09-16T20:49:13.523 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.523 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.524 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:13.524 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531673+0000", 2024-09-16T20:49:13.524 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9345999999999999e-05 2024-09-16T20:49:13.524 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:13.524 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:13.524 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:13.524 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.524 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.524 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3573 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:13.524 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481257+0000", 2024-09-16T20:49:13.524 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.994601436, 2024-09-16T20:49:13.524 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050511027, 2024-09-16T20:49:13.524 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:13.525 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:13.525 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:13.525 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:13.525 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:13.525 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:13.525 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3573 2024-09-16T20:49:13.525 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.525 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:13.525 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.525 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:13.525 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481257+0000", 2024-09-16T20:49:13.525 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:13.525 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.526 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.526 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:13.526 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481257+0000", 2024-09-16T20:49:13.526 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:13.526 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.526 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.526 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:13.526 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481257+0000", 2024-09-16T20:49:13.526 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.99e-07 2024-09-16T20:49:13.526 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.526 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.526 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:13.526 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481260+0000", 2024-09-16T20:49:13.527 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1709999999999998e-06 2024-09-16T20:49:13.527 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.527 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.527 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:13.527 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481271+0000", 2024-09-16T20:49:13.527 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0795000000000001e-05 2024-09-16T20:49:13.527 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.527 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.527 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:13.527 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481277+0000", 2024-09-16T20:49:13.527 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1670000000000004e-06 2024-09-16T20:49:13.527 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.527 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.527 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:13.528 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.524647+0000", 2024-09-16T20:49:13.528 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.043369810000000002 2024-09-16T20:49:13.528 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.528 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.528 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:13.528 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.524725+0000", 2024-09-16T20:49:13.528 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7286000000000005e-05 2024-09-16T20:49:13.528 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.528 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.528 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:13.528 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525062+0000", 2024-09-16T20:49:13.528 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00033739499999999999 2024-09-16T20:49:13.528 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.529 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.529 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:13.529 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530692+0000", 2024-09-16T20:49:13.529 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0056301479999999997 2024-09-16T20:49:13.529 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.529 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.529 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:13.529 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531694+0000", 2024-09-16T20:49:13.529 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0010017419999999999 2024-09-16T20:49:13.529 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.529 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.529 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:13.529 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531737+0000", 2024-09-16T20:49:13.529 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2762999999999997e-05 2024-09-16T20:49:13.529 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.530 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.530 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:13.530 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531768+0000", 2024-09-16T20:49:13.530 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1151e-05 2024-09-16T20:49:13.530 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:13.530 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:13.530 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:13.530 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.530 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.530 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3574 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:13.530 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:49:13.530 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.994568480000002, 2024-09-16T20:49:13.530 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053579441999999998, 2024-09-16T20:49:13.530 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:13.530 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:13.531 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:13.531 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:13.531 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:13.531 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:13.531 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3574 2024-09-16T20:49:13.531 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.531 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:13.531 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.531 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:13.531 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:49:13.531 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:13.531 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.531 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.531 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:13.532 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:49:13.532 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:13.532 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.532 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.532 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:13.532 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:49:13.532 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2e-07 2024-09-16T20:49:13.532 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.532 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.532 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:13.532 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481292+0000", 2024-09-16T20:49:13.532 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2630000000000002e-06 2024-09-16T20:49:13.532 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.532 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.533 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:13.533 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481297+0000", 2024-09-16T20:49:13.533 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.296e-06 2024-09-16T20:49:13.533 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.533 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.533 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:13.533 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481300+0000", 2024-09-16T20:49:13.533 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7019999999999999e-06 2024-09-16T20:49:13.533 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.533 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.533 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:13.533 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525533+0000", 2024-09-16T20:49:13.533 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.044232318 2024-09-16T20:49:13.533 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.534 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.534 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:13.534 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525619+0000", 2024-09-16T20:49:13.534 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5892999999999999e-05 2024-09-16T20:49:13.534 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.534 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.534 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:13.534 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525987+0000", 2024-09-16T20:49:13.534 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00036836099999999999 2024-09-16T20:49:13.534 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.534 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.534 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:13.534 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531779+0000", 2024-09-16T20:49:13.534 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0057918240000000001 2024-09-16T20:49:13.535 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.535 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.535 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:13.535 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534747+0000", 2024-09-16T20:49:13.535 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0029681600000000001 2024-09-16T20:49:13.535 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.535 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.535 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:13.535 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534832+0000", 2024-09-16T20:49:13.535 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5180999999999996e-05 2024-09-16T20:49:13.535 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.535 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.535 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:13.535 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534869+0000", 2024-09-16T20:49:13.536 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7024000000000003e-05 2024-09-16T20:49:13.536 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:13.536 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:13.536 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:13.536 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.536 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.536 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3575 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:13.536 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:49:13.536 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.994553740000001, 2024-09-16T20:49:13.536 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053801463000000001, 2024-09-16T20:49:13.536 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:13.536 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:13.536 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:13.536 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:13.537 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:13.537 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:13.537 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3575 2024-09-16T20:49:13.537 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.537 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:13.537 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.537 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:13.537 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:49:13.537 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:13.537 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.537 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.537 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:13.537 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:49:13.538 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:13.538 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.538 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.538 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:13.538 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481305+0000", 2024-09-16T20:49:13.538 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5400000000000002e-07 2024-09-16T20:49:13.538 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.538 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.538 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:13.538 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481307+0000", 2024-09-16T20:49:13.538 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7800000000000001e-06 2024-09-16T20:49:13.538 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.538 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.538 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:13.539 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481312+0000", 2024-09-16T20:49:13.539 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6349999999999997e-06 2024-09-16T20:49:13.539 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.539 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.539 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:13.539 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481315+0000", 2024-09-16T20:49:13.539 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7829999999999999e-06 2024-09-16T20:49:13.539 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.539 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.539 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:13.539 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.526876+0000", 2024-09-16T20:49:13.539 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.045561070000000002 2024-09-16T20:49:13.539 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.539 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.540 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:13.540 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.526954+0000", 2024-09-16T20:49:13.540 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8108000000000005e-05 2024-09-16T20:49:13.540 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.540 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.540 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:13.540 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527282+0000", 2024-09-16T20:49:13.540 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000327483 2024-09-16T20:49:13.540 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.540 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.540 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:13.540 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531784+0000", 2024-09-16T20:49:13.540 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0045020169999999997 2024-09-16T20:49:13.540 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.541 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.541 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:13.541 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534921+0000", 2024-09-16T20:49:13.541 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.003137791 2024-09-16T20:49:13.541 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.541 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.541 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:13.541 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534976+0000", 2024-09-16T20:49:13.541 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4953000000000001e-05 2024-09-16T20:49:13.541 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.541 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.541 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:13.542 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535106+0000", 2024-09-16T20:49:13.542 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000129589 2024-09-16T20:49:13.542 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:13.542 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:13.542 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:13.542 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.542 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.542 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3576 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:13.542 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:49:13.542 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.994539828000001, 2024-09-16T20:49:13.542 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053955436000000002, 2024-09-16T20:49:13.542 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:13.542 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:13.542 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:13.543 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:13.543 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:13.543 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:13.543 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3576 2024-09-16T20:49:13.543 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.543 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:13.543 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.543 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:13.543 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:49:13.543 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:13.543 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.543 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.543 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:13.543 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:49:13.544 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:13.544 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.544 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.544 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:13.544 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481319+0000", 2024-09-16T20:49:13.544 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.53e-07 2024-09-16T20:49:13.545 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.545 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.545 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:13.545 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481320+0000", 2024-09-16T20:49:13.545 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5230000000000001e-06 2024-09-16T20:49:13.545 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.545 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.545 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:13.545 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481330+0000", 2024-09-16T20:49:13.545 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0190000000000001e-05 2024-09-16T20:49:13.545 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.545 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.545 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:13.545 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481333+0000", 2024-09-16T20:49:13.546 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.151e-06 2024-09-16T20:49:13.546 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.546 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.546 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:13.546 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527689+0000", 2024-09-16T20:49:13.546 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046355192000000003 2024-09-16T20:49:13.546 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.546 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.546 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:13.546 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527764+0000", 2024-09-16T20:49:13.546 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5776000000000001e-05 2024-09-16T20:49:13.546 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.546 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.546 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:13.546 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528141+0000", 2024-09-16T20:49:13.547 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00037647599999999999 2024-09-16T20:49:13.547 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.547 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.547 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:13.547 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531788+0000", 2024-09-16T20:49:13.547 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0036470679999999998 2024-09-16T20:49:13.547 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.547 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.547 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:13.547 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535174+0000", 2024-09-16T20:49:13.547 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0033862670000000001 2024-09-16T20:49:13.547 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.547 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.547 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:13.548 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535236+0000", 2024-09-16T20:49:13.548 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1797999999999993e-05 2024-09-16T20:49:13.548 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.548 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.548 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:13.548 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535274+0000", 2024-09-16T20:49:13.548 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7741999999999999e-05 2024-09-16T20:49:13.548 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:13.548 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:13.548 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:13.548 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.548 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.548 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3577 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:13.548 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:49:13.549 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.994518228, 2024-09-16T20:49:13.549 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054073807000000002, 2024-09-16T20:49:13.549 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:13.549 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:13.549 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:13.549 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:13.549 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:13.549 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:13.549 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3577 2024-09-16T20:49:13.549 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.549 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:13.549 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.549 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:13.549 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:49:13.550 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:13.550 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.550 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.550 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:13.550 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:49:13.550 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:13.550 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.550 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.550 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:13.550 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:49:13.550 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.96e-07 2024-09-16T20:49:13.550 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.550 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.550 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:13.551 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481342+0000", 2024-09-16T20:49:13.551 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.714e-06 2024-09-16T20:49:13.551 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.551 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.551 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:13.551 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481349+0000", 2024-09-16T20:49:13.551 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4930000000000001e-06 2024-09-16T20:49:13.551 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.551 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.551 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:13.551 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481353+0000", 2024-09-16T20:49:13.551 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4879999999999999e-06 2024-09-16T20:49:13.551 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.552 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.552 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:13.552 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528619+0000", 2024-09-16T20:49:13.552 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047266351999999998 2024-09-16T20:49:13.552 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.552 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.552 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:13.552 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528706+0000", 2024-09-16T20:49:13.552 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6796e-05 2024-09-16T20:49:13.552 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.552 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.552 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:13.552 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529148+0000", 2024-09-16T20:49:13.553 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00044182999999999999 2024-09-16T20:49:13.553 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.553 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.553 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:13.553 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531792+0000", 2024-09-16T20:49:13.553 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0026443119999999998 2024-09-16T20:49:13.553 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.553 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.553 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:13.553 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535319+0000", 2024-09-16T20:49:13.553 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0035266439999999998 2024-09-16T20:49:13.553 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.553 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.553 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:13.554 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535372+0000", 2024-09-16T20:49:13.554 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3207000000000001e-05 2024-09-16T20:49:13.554 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.554 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.554 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:13.554 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535414+0000", 2024-09-16T20:49:13.554 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1675000000000003e-05 2024-09-16T20:49:13.554 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:13.554 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:13.554 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:13.554 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.554 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.554 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3578 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:13.554 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:49:13.555 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.994497059, 2024-09-16T20:49:13.555 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054189586999999997, 2024-09-16T20:49:13.555 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:13.555 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:13.555 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:13.555 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:13.555 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:13.555 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:13.555 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3578 2024-09-16T20:49:13.555 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.555 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:13.555 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.555 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:13.556 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:49:13.556 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:13.556 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.556 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.556 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:13.556 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:49:13.556 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:13.556 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.556 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.556 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:13.556 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:49:13.556 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0800000000000001e-07 2024-09-16T20:49:13.556 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.557 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.557 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:13.557 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481363+0000", 2024-09-16T20:49:13.557 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0339999999999999e-06 2024-09-16T20:49:13.557 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.557 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.557 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:13.557 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481371+0000", 2024-09-16T20:49:13.557 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4440000000000002e-06 2024-09-16T20:49:13.557 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.557 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.557 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:13.557 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481374+0000", 2024-09-16T20:49:13.558 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2049999999999998e-06 2024-09-16T20:49:13.558 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.558 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.558 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:13.558 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529555+0000", 2024-09-16T20:49:13.558 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048181202999999999 2024-09-16T20:49:13.558 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.558 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.558 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:13.558 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529616+0000", 2024-09-16T20:49:13.558 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0912000000000002e-05 2024-09-16T20:49:13.558 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.558 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.558 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:13.559 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529939+0000", 2024-09-16T20:49:13.559 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00032269800000000001 2024-09-16T20:49:13.559 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.559 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.559 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:13.559 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532602+0000", 2024-09-16T20:49:13.559 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0026634940000000002 2024-09-16T20:49:13.559 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.559 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.559 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:13.559 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535465+0000", 2024-09-16T20:49:13.559 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.002862149 2024-09-16T20:49:13.559 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.559 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.560 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:13.560 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535516+0000", 2024-09-16T20:49:13.560 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1696000000000001e-05 2024-09-16T20:49:13.560 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.560 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.560 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:13.560 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535551+0000", 2024-09-16T20:49:13.560 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4443999999999998e-05 2024-09-16T20:49:13.560 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:13.560 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:13.560 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:13.560 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.560 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.561 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3579 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:13.561 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:49:13.561 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.994478619999999, 2024-09-16T20:49:13.561 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054295151, 2024-09-16T20:49:13.561 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:13.561 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:13.561 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:13.561 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:13.561 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:13.561 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:13.561 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3579 2024-09-16T20:49:13.561 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.561 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:13.561 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.562 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:13.562 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:49:13.562 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:13.562 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.562 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.562 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:13.562 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:49:13.562 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:13.562 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.562 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.562 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:13.562 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:49:13.562 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3300000000000001e-07 2024-09-16T20:49:13.563 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.563 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.563 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:13.563 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481381+0000", 2024-09-16T20:49:13.563 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.649e-06 2024-09-16T20:49:13.563 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.563 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.563 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:13.563 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481387+0000", 2024-09-16T20:49:13.563 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4519999999999996e-06 2024-09-16T20:49:13.563 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.563 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.563 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:13.563 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481390+0000", 2024-09-16T20:49:13.563 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0170000000000001e-06 2024-09-16T20:49:13.564 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.564 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.564 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:13.564 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530756+0000", 2024-09-16T20:49:13.564 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049365882999999999 2024-09-16T20:49:13.564 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.564 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.564 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:13.564 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530845+0000", 2024-09-16T20:49:13.564 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9197000000000007e-05 2024-09-16T20:49:13.564 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.564 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.564 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:13.564 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531191+0000", 2024-09-16T20:49:13.565 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00034562500000000001 2024-09-16T20:49:13.565 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.565 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.565 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:13.565 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534078+0000", 2024-09-16T20:49:13.565 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0028878179999999999 2024-09-16T20:49:13.565 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.565 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.565 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:13.565 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535589+0000", 2024-09-16T20:49:13.565 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00151085 2024-09-16T20:49:13.565 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.565 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.565 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:13.566 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535639+0000", 2024-09-16T20:49:13.566 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9697000000000002e-05 2024-09-16T20:49:13.566 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.566 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.566 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:13.566 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535675+0000", 2024-09-16T20:49:13.566 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5729999999999998e-05 2024-09-16T20:49:13.566 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:13.566 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:13.566 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:13.566 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.566 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.566 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3580 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:13.566 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:49:13.566 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.994464988000001, 2024-09-16T20:49:13.567 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054397361999999998, 2024-09-16T20:49:13.567 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:13.567 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:13.567 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:13.567 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:13.567 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:13.567 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:13.567 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3580 2024-09-16T20:49:13.567 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.567 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:13.567 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.567 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:13.567 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:49:13.567 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:13.567 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.568 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.568 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:13.568 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:49:13.568 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:13.568 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.568 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.568 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:13.568 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:49:13.568 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2600000000000001e-07 2024-09-16T20:49:13.568 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.568 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.568 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:13.568 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481395+0000", 2024-09-16T20:49:13.568 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.544e-06 2024-09-16T20:49:13.569 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.569 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.569 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:13.569 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481404+0000", 2024-09-16T20:49:13.569 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8850000000000005e-06 2024-09-16T20:49:13.569 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.569 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.569 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:13.569 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481407+0000", 2024-09-16T20:49:13.569 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1540000000000002e-06 2024-09-16T20:49:13.569 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.569 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.570 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:13.570 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531823+0000", 2024-09-16T20:49:13.570 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050415637999999999 2024-09-16T20:49:13.570 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.570 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.570 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:13.570 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531895+0000", 2024-09-16T20:49:13.570 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2179000000000001e-05 2024-09-16T20:49:13.570 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.570 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.570 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:13.570 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532236+0000", 2024-09-16T20:49:13.570 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00034073799999999999 2024-09-16T20:49:13.570 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.570 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.570 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:13.571 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534083+0000", 2024-09-16T20:49:13.571 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001847298 2024-09-16T20:49:13.571 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.571 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.571 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:13.571 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535714+0000", 2024-09-16T20:49:13.571 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0016316810000000001 2024-09-16T20:49:13.571 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.571 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.571 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:13.571 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535759+0000", 2024-09-16T20:49:13.571 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4478999999999999e-05 2024-09-16T20:49:13.571 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.571 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.571 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:13.572 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535791+0000", 2024-09-16T20:49:13.572 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.154e-05 2024-09-16T20:49:13.572 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:13.572 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:13.572 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:13.572 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.572 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.572 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3581 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:13.572 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:49:13.572 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.994442991, 2024-09-16T20:49:13.572 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.057094403000000002, 2024-09-16T20:49:13.572 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:13.572 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:13.572 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:13.572 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:13.573 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:13.573 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:13.573 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3581 2024-09-16T20:49:13.573 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.573 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:13.573 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.573 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:13.573 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:49:13.573 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:13.573 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.573 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.573 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:13.573 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:49:13.573 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:13.573 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.574 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.574 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:13.574 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:49:13.574 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8799999999999998e-07 2024-09-16T20:49:13.574 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.574 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.574 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:13.574 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481418+0000", 2024-09-16T20:49:13.574 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1859999999999999e-06 2024-09-16T20:49:13.574 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.574 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.574 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:13.574 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481423+0000", 2024-09-16T20:49:13.574 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8279999999999999e-06 2024-09-16T20:49:13.575 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.575 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.575 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:13.575 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481426+0000", 2024-09-16T20:49:13.575 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0110000000000001e-06 2024-09-16T20:49:13.575 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.575 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.575 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:13.575 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532631+0000", 2024-09-16T20:49:13.575 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051204514999999999 2024-09-16T20:49:13.575 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.575 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.575 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:13.575 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.533381+0000", 2024-09-16T20:49:13.575 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00075029000000000003 2024-09-16T20:49:13.576 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.576 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.576 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:13.576 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.533739+0000", 2024-09-16T20:49:13.576 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00035773100000000002 2024-09-16T20:49:13.576 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.576 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.576 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:13.576 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535123+0000", 2024-09-16T20:49:13.576 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0013836160000000001 2024-09-16T20:49:13.576 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.576 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.576 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:13.576 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538413+0000", 2024-09-16T20:49:13.577 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0032902510000000001 2024-09-16T20:49:13.577 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.577 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.577 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:13.577 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538470+0000", 2024-09-16T20:49:13.577 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6697999999999998e-05 2024-09-16T20:49:13.577 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.577 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.577 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:13.577 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538510+0000", 2024-09-16T20:49:13.577 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9988999999999999e-05 2024-09-16T20:49:13.577 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:13.577 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:13.577 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:13.578 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.578 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.578 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:12732 3.d 3:b752444a:::rbd_header.110e747f0779:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-16T20:49:13.578 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:49:13.578 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 8.7073704129999996, 2024-09-16T20:49:13.578 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.065302467000000003, 2024-09-16T20:49:13.578 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:13.578 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:13.578 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:13.578 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:13.578 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:49:13.578 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:49:13.578 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 12732 2024-09-16T20:49:13.578 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.579 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:13.579 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.579 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:13.579 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:49:13.579 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:13.579 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.579 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.579 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:13.579 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:49:13.579 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:13.579 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.579 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.579 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:13.579 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768490+0000", 2024-09-16T20:49:13.579 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2859999999999998e-06 2024-09-16T20:49:13.580 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.580 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.580 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:13.580 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768501+0000", 2024-09-16T20:49:13.580 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1226999999999999e-05 2024-09-16T20:49:13.580 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.580 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.580 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:13.580 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768517+0000", 2024-09-16T20:49:13.580 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.592e-05 2024-09-16T20:49:13.580 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.580 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.580 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:13.580 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768528+0000", 2024-09-16T20:49:13.581 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0689999999999999e-05 2024-09-16T20:49:13.581 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.581 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.581 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:13.581 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768584+0000", 2024-09-16T20:49:13.581 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5889e-05 2024-09-16T20:49:13.581 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.581 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.581 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:13.581 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768720+0000", 2024-09-16T20:49:13.581 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000136718 2024-09-16T20:49:13.581 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.581 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.581 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:13.582 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.769066+0000", 2024-09-16T20:49:13.582 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000346013 2024-09-16T20:49:13.582 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.582 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.582 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:13.582 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833659+0000", 2024-09-16T20:49:13.582 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.064592822999999994 2024-09-16T20:49:13.582 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.582 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.582 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:13.582 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833719+0000", 2024-09-16T20:49:13.582 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9441000000000002e-05 2024-09-16T20:49:13.582 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.582 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.582 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:13.583 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833758+0000", 2024-09-16T20:49:13.583 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9122000000000003e-05 2024-09-16T20:49:13.583 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.583 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.583 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:13.583 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833790+0000", 2024-09-16T20:49:13.583 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2338000000000001e-05 2024-09-16T20:49:13.583 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:13.583 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:13.583 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:13.583 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.583 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.583 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.5089.0:12732 3.d e73/23 ondisk, result = 0)", 2024-09-16T20:49:13.583 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:49:13.583 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 8.7057682429999996, 2024-09-16T20:49:13.584 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.063706456999999994, 2024-09-16T20:49:13.584 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:13.584 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:13.584 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:13.584 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:13.584 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.584 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:13.584 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:49:13.584 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:13.584 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.584 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.584 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:13.584 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:49:13.584 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:13.584 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.585 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.585 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:13.585 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770091+0000", 2024-09-16T20:49:13.585 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.53e-06 2024-09-16T20:49:13.585 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.585 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.585 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:13.585 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770104+0000", 2024-09-16T20:49:13.585 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2203e-05 2024-09-16T20:49:13.585 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.585 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.585 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:13.585 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833637+0000", 2024-09-16T20:49:13.585 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.063533201999999997 2024-09-16T20:49:13.585 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.586 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.586 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:13.586 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833655+0000", 2024-09-16T20:49:13.586 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8369e-05 2024-09-16T20:49:13.586 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.586 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.586 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:13.586 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833707+0000", 2024-09-16T20:49:13.586 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2061999999999997e-05 2024-09-16T20:49:13.586 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.586 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.586 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:13.586 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833715+0000", 2024-09-16T20:49:13.586 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5179999999999998e-06 2024-09-16T20:49:13.586 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:13.586 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:13.587 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:13.587 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833796+0000", 2024-09-16T20:49:13.587 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1573e-05 2024-09-16T20:49:13.587 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:13.587 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:13.587 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:13.587 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:13.587 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:13.587 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:13.588 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:49:13.589 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:13.587+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:13.689 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:13.687+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:13.789 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:13.789 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:49:13.789 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:49:13.790 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:13.790 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:13.789+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:13.797 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:49:13.799 INFO:teuthology.orchestra.run.smithi029.stderr:noscrub is unset 2024-09-16T20:49:13.839 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 3, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-16T20:49:13.839 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T20:49:13.839 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-09-16T20:49:13.839 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T20:49:13.839 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T20:49:13.840 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2024-09-16T20:49:13.900 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:13.899+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:13.912 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:13.912 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:49:13.912 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:49:13.912 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:49:13.912 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:13.921 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:49:13.964 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:13.964 INFO:teuthology.orchestra.run.smithi029.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-16T20:49:13.964 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:13.991 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:13.991+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:14.017 INFO:teuthology.orchestra.run.smithi139.stdout:2802 read 0x4ad0200 thru 0x4ad9f69 (0x9d6a bytes) 2024-09-16T20:49:14.018 INFO:teuthology.orchestra.run.smithi139.stdout:2804 write 0xa1f7c00 thru 0xa2030f2 (0xb4f3 bytes) 2024-09-16T20:49:14.019 INFO:teuthology.orchestra.run.smithi139.stdout:2808 write 0x4cbca00 thru 0x4cc7b58 (0xb159 bytes) 2024-09-16T20:49:14.019 INFO:teuthology.orchestra.run.smithi139.stdout:2809 write 0x5797000 thru 0x57984f9 (0x14fa bytes) 2024-09-16T20:49:14.020 INFO:teuthology.orchestra.run.smithi139.stdout:2811 write 0x2029400 thru 0x203513e (0xbd3f bytes) 2024-09-16T20:49:14.021 INFO:teuthology.orchestra.run.smithi139.stdout:2813 read 0x95d4c00 thru 0x95de114 (0x9515 bytes) 2024-09-16T20:49:14.025 INFO:teuthology.orchestra.run.smithi139.stdout:2815 read 0xd00000 thru 0xd09df7 (0x9df8 bytes) 2024-09-16T20:49:14.083 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:14.084 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:49:14.084 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:49:14.084 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:49:14.084 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.084 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3709 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1464320~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:14.084 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.638971+0000", 2024-09-16T20:49:14.084 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 51.443100141999999, 2024-09-16T20:49:14.084 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049858670000000001, 2024-09-16T20:49:14.084 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:14.084 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:14.084 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:14.084 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:14.085 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:14.085 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:14.085 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3709 2024-09-16T20:49:14.085 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.085 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:14.085 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.085 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:14.085 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638971+0000", 2024-09-16T20:49:14.085 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:14.085 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.085 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.085 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:14.085 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638971+0000", 2024-09-16T20:49:14.086 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:14.086 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.086 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.086 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:14.086 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638972+0000", 2024-09-16T20:49:14.086 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0400000000000002e-07 2024-09-16T20:49:14.086 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.086 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.086 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:14.086 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638973+0000", 2024-09-16T20:49:14.086 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.578e-06 2024-09-16T20:49:14.086 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.086 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.086 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:14.087 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638979+0000", 2024-09-16T20:49:14.087 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.502e-06 2024-09-16T20:49:14.087 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.087 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.087 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:14.087 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638983+0000", 2024-09-16T20:49:14.087 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2690000000000001e-06 2024-09-16T20:49:14.087 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.087 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.087 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:14.087 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688628+0000", 2024-09-16T20:49:14.087 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049644547999999997 2024-09-16T20:49:14.087 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.087 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.087 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:14.087 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688718+0000", 2024-09-16T20:49:14.088 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0802000000000002e-05 2024-09-16T20:49:14.088 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.088 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.088 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:14.088 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688830+0000", 2024-09-16T20:49:14.088 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000111667 2024-09-16T20:49:14.088 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:14.088 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:14.088 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:14.088 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.088 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.088 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3712 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1472512~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:14.088 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:49:14.088 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 51.443083614000003, 2024-09-16T20:49:14.089 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050004305999999998, 2024-09-16T20:49:14.089 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:14.089 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:14.089 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:14.089 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:14.089 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:14.089 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:14.089 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3712 2024-09-16T20:49:14.089 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.089 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:14.089 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.089 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:14.089 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:49:14.089 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:14.089 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.090 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.090 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:14.090 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:49:14.090 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:14.090 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.090 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.090 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:14.090 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:49:14.090 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0199999999999998e-07 2024-09-16T20:49:14.090 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.090 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.090 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:14.090 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638990+0000", 2024-09-16T20:49:14.090 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.618e-06 2024-09-16T20:49:14.091 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.091 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.091 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:14.091 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638995+0000", 2024-09-16T20:49:14.091 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3940000000000004e-06 2024-09-16T20:49:14.091 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.091 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.091 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:14.091 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639000+0000", 2024-09-16T20:49:14.091 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9309999999999999e-06 2024-09-16T20:49:14.091 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.091 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.091 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:14.091 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688867+0000", 2024-09-16T20:49:14.091 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049866955999999997 2024-09-16T20:49:14.092 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.092 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.092 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:14.092 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688933+0000", 2024-09-16T20:49:14.092 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5359999999999998e-05 2024-09-16T20:49:14.092 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.092 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.092 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:14.092 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688992+0000", 2024-09-16T20:49:14.092 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9744999999999998e-05 2024-09-16T20:49:14.092 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:14.092 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:14.092 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:14.093 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.093 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.093 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3714 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1480704~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:14.093 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:49:14.093 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 51.443066023999997, 2024-09-16T20:49:14.093 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050152552000000003, 2024-09-16T20:49:14.093 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:14.093 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:14.093 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:14.093 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:14.093 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:14.093 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:14.093 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3714 2024-09-16T20:49:14.093 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.094 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:14.094 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.094 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:14.094 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:49:14.094 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:14.094 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.094 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.094 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:14.094 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:49:14.094 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:14.094 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.094 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.094 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:14.094 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:49:14.095 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0499999999999999e-07 2024-09-16T20:49:14.095 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.095 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.095 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:14.095 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639007+0000", 2024-09-16T20:49:14.095 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5340000000000001e-06 2024-09-16T20:49:14.095 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.095 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.095 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:14.095 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639013+0000", 2024-09-16T20:49:14.095 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3260000000000003e-06 2024-09-16T20:49:14.095 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.095 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.095 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:14.096 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639018+0000", 2024-09-16T20:49:14.096 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2850000000000002e-06 2024-09-16T20:49:14.096 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.096 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.096 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:14.096 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689026+0000", 2024-09-16T20:49:14.096 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.05000798 2024-09-16T20:49:14.096 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.096 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.096 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:14.096 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689077+0000", 2024-09-16T20:49:14.096 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1063999999999998e-05 2024-09-16T20:49:14.096 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.096 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.097 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:14.097 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689158+0000", 2024-09-16T20:49:14.097 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1057999999999995e-05 2024-09-16T20:49:14.097 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:14.097 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:14.097 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:14.097 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.097 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.097 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3716 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1488896~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:14.097 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:49:14.097 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 51.443048214000001, 2024-09-16T20:49:14.097 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050280142, 2024-09-16T20:49:14.097 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:14.097 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:14.098 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:14.098 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:14.098 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:14.098 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:14.098 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3716 2024-09-16T20:49:14.098 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.098 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:14.098 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.098 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:14.098 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:49:14.098 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:14.098 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.098 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.098 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:14.098 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:49:14.098 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:14.099 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.099 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.099 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:14.099 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639024+0000", 2024-09-16T20:49:14.099 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9999999999999997e-07 2024-09-16T20:49:14.099 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.099 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.099 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:14.099 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639026+0000", 2024-09-16T20:49:14.099 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.781e-06 2024-09-16T20:49:14.099 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.099 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.099 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:14.099 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639032+0000", 2024-09-16T20:49:14.099 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7780000000000004e-06 2024-09-16T20:49:14.100 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.100 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.100 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:14.100 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639036+0000", 2024-09-16T20:49:14.100 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6909999999999996e-06 2024-09-16T20:49:14.100 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.100 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.100 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:14.100 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689191+0000", 2024-09-16T20:49:14.100 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050155284000000001 2024-09-16T20:49:14.100 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.100 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.100 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:14.100 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689242+0000", 2024-09-16T20:49:14.100 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0662999999999999e-05 2024-09-16T20:49:14.101 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.101 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.101 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:14.101 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689304+0000", 2024-09-16T20:49:14.101 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1345000000000003e-05 2024-09-16T20:49:14.101 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:14.101 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:14.101 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:14.101 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.101 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.101 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3719 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1497088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:14.101 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:49:14.101 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 51.443020936000003, 2024-09-16T20:49:14.101 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050399715999999997, 2024-09-16T20:49:14.101 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:14.102 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:14.102 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:14.102 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:14.102 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:14.102 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:14.102 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3719 2024-09-16T20:49:14.102 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.102 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:14.102 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.102 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:14.102 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:49:14.102 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:14.102 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.102 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.102 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:14.103 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:49:14.103 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:14.103 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.103 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.103 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:14.103 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:49:14.103 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9700000000000002e-07 2024-09-16T20:49:14.103 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.103 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.103 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:14.103 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639053+0000", 2024-09-16T20:49:14.103 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2900000000000001e-06 2024-09-16T20:49:14.103 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.103 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.104 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:14.104 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639060+0000", 2024-09-16T20:49:14.104 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7750000000000004e-06 2024-09-16T20:49:14.104 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.104 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.104 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:14.104 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639066+0000", 2024-09-16T20:49:14.104 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0639999999999999e-06 2024-09-16T20:49:14.104 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.104 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.104 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:14.104 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689337+0000", 2024-09-16T20:49:14.104 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050270784999999998 2024-09-16T20:49:14.104 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.105 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.105 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:14.105 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689392+0000", 2024-09-16T20:49:14.105 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4920999999999998e-05 2024-09-16T20:49:14.105 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.105 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.105 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:14.105 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689450+0000", 2024-09-16T20:49:14.105 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8483999999999998e-05 2024-09-16T20:49:14.105 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:14.105 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:14.105 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:14.105 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.105 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.106 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3721 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1505280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:14.106 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:49:14.106 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 51.442997804000001, 2024-09-16T20:49:14.106 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050513608000000002, 2024-09-16T20:49:14.106 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:14.106 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:14.106 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:14.106 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:14.106 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:14.106 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:14.106 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3721 2024-09-16T20:49:14.106 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.106 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:14.106 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.106 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:14.107 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:49:14.107 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:14.107 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.107 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.107 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:14.107 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:49:14.107 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:14.107 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.107 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.107 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:14.107 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:49:14.107 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7e-07 2024-09-16T20:49:14.107 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.107 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.107 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:14.108 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639076+0000", 2024-09-16T20:49:14.108 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8899999999999999e-06 2024-09-16T20:49:14.108 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.108 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.108 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:14.108 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639083+0000", 2024-09-16T20:49:14.108 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1210000000000001e-06 2024-09-16T20:49:14.108 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.108 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.108 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:14.108 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639089+0000", 2024-09-16T20:49:14.108 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.643e-06 2024-09-16T20:49:14.108 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.108 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.108 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:14.109 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689474+0000", 2024-09-16T20:49:14.109 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050385679000000003 2024-09-16T20:49:14.109 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.109 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.109 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:14.109 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689533+0000", 2024-09-16T20:49:14.109 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8303000000000001e-05 2024-09-16T20:49:14.109 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.109 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.109 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:14.109 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689587+0000", 2024-09-16T20:49:14.109 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4601999999999997e-05 2024-09-16T20:49:14.109 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:14.109 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:14.110 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:14.110 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.110 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.110 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3723 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1513472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:14.110 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:49:14.110 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 51.442967000000003, 2024-09-16T20:49:14.110 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050699691999999998, 2024-09-16T20:49:14.110 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:14.110 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:14.110 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:14.110 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:14.110 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:14.110 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:14.110 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3723 2024-09-16T20:49:14.110 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.111 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:14.111 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.111 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:14.111 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:49:14.111 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:14.111 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.111 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.111 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:14.111 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:49:14.111 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:14.111 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.111 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.111 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:14.111 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:49:14.111 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3399999999999999e-07 2024-09-16T20:49:14.112 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.112 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.112 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:14.112 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639107+0000", 2024-09-16T20:49:14.112 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3860000000000001e-06 2024-09-16T20:49:14.112 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.112 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.112 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:14.112 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639114+0000", 2024-09-16T20:49:14.112 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9800000000000001e-06 2024-09-16T20:49:14.112 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.112 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.112 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:14.112 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639129+0000", 2024-09-16T20:49:14.113 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4887e-05 2024-09-16T20:49:14.113 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.113 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.113 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:14.113 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689625+0000", 2024-09-16T20:49:14.113 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050495848000000003 2024-09-16T20:49:14.113 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.113 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.113 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:14.113 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689720+0000", 2024-09-16T20:49:14.113 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4821000000000006e-05 2024-09-16T20:49:14.113 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.113 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.113 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:14.113 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689804+0000", 2024-09-16T20:49:14.114 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4235999999999998e-05 2024-09-16T20:49:14.114 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:14.114 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:14.114 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:14.114 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.114 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.114 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3725 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1521664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:14.114 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:49:14.114 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 51.442933404000001, 2024-09-16T20:49:14.114 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050797660000000001, 2024-09-16T20:49:14.114 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:14.114 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:14.114 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:14.114 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:14.115 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:14.115 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:14.115 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3725 2024-09-16T20:49:14.115 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.115 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:14.115 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.115 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:14.115 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:49:14.115 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:14.115 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.115 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.115 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:14.115 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:49:14.115 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:14.116 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.116 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.116 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:14.116 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639139+0000", 2024-09-16T20:49:14.116 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8999999999999997e-07 2024-09-16T20:49:14.116 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.116 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.116 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:14.116 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639141+0000", 2024-09-16T20:49:14.116 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.182e-06 2024-09-16T20:49:14.116 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.116 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.116 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:14.116 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639154+0000", 2024-09-16T20:49:14.116 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3626e-05 2024-09-16T20:49:14.117 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.117 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.117 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:14.117 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639160+0000", 2024-09-16T20:49:14.117 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6810000000000003e-06 2024-09-16T20:49:14.117 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.117 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.117 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:14.117 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689831+0000", 2024-09-16T20:49:14.117 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050670907000000001 2024-09-16T20:49:14.117 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.117 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.117 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:14.117 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689882+0000", 2024-09-16T20:49:14.118 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0445999999999998e-05 2024-09-16T20:49:14.118 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.118 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.118 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:14.118 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689936+0000", 2024-09-16T20:49:14.118 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4327999999999999e-05 2024-09-16T20:49:14.118 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:14.118 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:14.118 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:14.118 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.118 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.118 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3727 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1529856~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:14.118 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:49:14.118 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 51.442897827000003, 2024-09-16T20:49:14.118 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051016922999999999, 2024-09-16T20:49:14.119 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:14.119 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:14.119 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:14.119 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:14.119 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:14.119 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:14.119 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3727 2024-09-16T20:49:14.119 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.119 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:14.119 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.119 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:14.119 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:49:14.119 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:14.119 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.120 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.120 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:14.120 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:49:14.120 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:14.120 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.120 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.120 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:14.120 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:49:14.120 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9999999999999998e-07 2024-09-16T20:49:14.120 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.120 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.120 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:14.120 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639183+0000", 2024-09-16T20:49:14.120 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2399999999999996e-06 2024-09-16T20:49:14.120 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.121 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.121 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:14.121 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639195+0000", 2024-09-16T20:49:14.121 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1113e-05 2024-09-16T20:49:14.121 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.121 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.121 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:14.121 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639206+0000", 2024-09-16T20:49:14.121 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1503e-05 2024-09-16T20:49:14.121 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.121 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.121 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:14.121 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689969+0000", 2024-09-16T20:49:14.121 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050762676 2024-09-16T20:49:14.121 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.122 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.122 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:14.122 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690071+0000", 2024-09-16T20:49:14.122 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102163 2024-09-16T20:49:14.122 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.122 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.122 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:14.122 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690191+0000", 2024-09-16T20:49:14.122 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000119728 2024-09-16T20:49:14.122 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:14.122 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:14.122 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:14.122 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.122 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.122 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3730 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1538048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:14.122 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:49:14.123 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 51.442856264, 2024-09-16T20:49:14.123 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051131467999999999, 2024-09-16T20:49:14.123 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:14.123 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:14.123 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:14.123 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:14.123 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:14.123 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:14.123 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3730 2024-09-16T20:49:14.123 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.123 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:14.123 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.123 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:14.123 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:49:14.123 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:14.123 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.124 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.124 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:14.124 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:49:14.124 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:14.124 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.124 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.124 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:14.124 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639216+0000", 2024-09-16T20:49:14.124 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.39e-07 2024-09-16T20:49:14.124 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.124 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.124 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:14.124 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639218+0000", 2024-09-16T20:49:14.124 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.537e-06 2024-09-16T20:49:14.125 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.125 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.125 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:14.125 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639229+0000", 2024-09-16T20:49:14.125 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1122e-05 2024-09-16T20:49:14.125 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.125 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.125 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:14.125 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639237+0000", 2024-09-16T20:49:14.125 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8509999999999999e-06 2024-09-16T20:49:14.125 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.125 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.125 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:14.125 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690227+0000", 2024-09-16T20:49:14.125 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050990147 2024-09-16T20:49:14.126 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.126 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.126 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:14.126 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690278+0000", 2024-09-16T20:49:14.126 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0223999999999999e-05 2024-09-16T20:49:14.126 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.126 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.126 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:14.126 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690347+0000", 2024-09-16T20:49:14.126 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9147999999999996e-05 2024-09-16T20:49:14.126 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:14.126 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:14.126 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:14.126 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.126 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.127 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3731 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1546240~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:14.127 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:49:14.127 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 51.442825556000003, 2024-09-16T20:49:14.127 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051276744999999999, 2024-09-16T20:49:14.127 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:14.127 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:14.127 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:14.127 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:14.127 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:14.127 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:14.127 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3731 2024-09-16T20:49:14.127 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.127 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:14.127 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.128 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:14.128 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:49:14.128 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:14.128 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.128 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.128 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:14.128 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:49:14.128 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:14.128 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.128 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.128 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:14.128 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:49:14.128 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9900000000000001e-07 2024-09-16T20:49:14.129 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.129 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.129 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:14.129 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639249+0000", 2024-09-16T20:49:14.129 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3630000000000001e-06 2024-09-16T20:49:14.129 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.129 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.129 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:14.129 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639255+0000", 2024-09-16T20:49:14.129 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4960000000000001e-06 2024-09-16T20:49:14.129 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.129 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.129 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:14.129 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639261+0000", 2024-09-16T20:49:14.130 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2839999999999997e-06 2024-09-16T20:49:14.130 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.130 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.130 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:14.130 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690375+0000", 2024-09-16T20:49:14.130 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051114684000000001 2024-09-16T20:49:14.130 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.130 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.130 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:14.130 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690425+0000", 2024-09-16T20:49:14.130 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9379999999999998e-05 2024-09-16T20:49:14.130 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.130 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.131 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:14.131 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690523+0000", 2024-09-16T20:49:14.131 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8139000000000003e-05 2024-09-16T20:49:14.131 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:14.131 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:14.131 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:14.131 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.131 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.131 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3732 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1554432~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:14.131 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:49:14.131 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 51.442779774000002, 2024-09-16T20:49:14.131 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051380096, 2024-09-16T20:49:14.131 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:14.131 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:14.132 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:14.132 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:14.132 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:14.132 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:14.132 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3732 2024-09-16T20:49:14.132 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.132 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:14.132 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.132 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:14.132 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:49:14.132 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:14.132 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.132 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.133 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:14.133 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:49:14.133 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:14.133 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.133 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.133 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:14.133 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:49:14.133 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6100000000000001e-07 2024-09-16T20:49:14.133 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.133 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.133 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:14.133 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639295+0000", 2024-09-16T20:49:14.133 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.447e-06 2024-09-16T20:49:14.134 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.134 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.134 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:14.134 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639302+0000", 2024-09-16T20:49:14.134 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0570000000000004e-06 2024-09-16T20:49:14.134 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.134 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.134 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:14.134 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639307+0000", 2024-09-16T20:49:14.134 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1429999999999997e-06 2024-09-16T20:49:14.134 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.135 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.135 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:14.135 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690555+0000", 2024-09-16T20:49:14.135 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051247963000000001 2024-09-16T20:49:14.135 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.135 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.135 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:14.135 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690602+0000", 2024-09-16T20:49:14.135 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7602000000000003e-05 2024-09-16T20:49:14.135 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.135 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.135 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:14.135 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690672+0000", 2024-09-16T20:49:14.135 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9423000000000003e-05 2024-09-16T20:49:14.136 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:14.136 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:14.136 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:14.136 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.136 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.136 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3734 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1562624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:14.136 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:49:14.136 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 51.442758124000001, 2024-09-16T20:49:14.136 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051490610999999999, 2024-09-16T20:49:14.136 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:14.136 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:14.136 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:14.136 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:14.136 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:14.137 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:14.137 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3734 2024-09-16T20:49:14.137 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.137 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:14.137 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.137 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:14.137 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:49:14.137 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:14.137 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.137 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.137 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:14.137 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:49:14.137 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:14.138 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.138 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.138 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:14.138 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639314+0000", 2024-09-16T20:49:14.138 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4400000000000001e-07 2024-09-16T20:49:14.138 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.138 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.138 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:14.138 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639316+0000", 2024-09-16T20:49:14.138 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1459999999999999e-06 2024-09-16T20:49:14.138 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.138 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.138 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:14.138 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639322+0000", 2024-09-16T20:49:14.139 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7610000000000004e-06 2024-09-16T20:49:14.139 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.139 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.139 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:14.139 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639327+0000", 2024-09-16T20:49:14.139 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.186e-06 2024-09-16T20:49:14.139 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.139 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.139 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:14.139 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690698+0000", 2024-09-16T20:49:14.139 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051371223000000001 2024-09-16T20:49:14.139 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.139 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.139 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:14.140 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690748+0000", 2024-09-16T20:49:14.140 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0192999999999999e-05 2024-09-16T20:49:14.140 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.140 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.140 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:14.140 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690804+0000", 2024-09-16T20:49:14.140 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5658000000000001e-05 2024-09-16T20:49:14.140 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:14.140 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:14.140 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:14.140 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.140 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.140 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3736 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1570816~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:14.140 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:49:14.140 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 51.442738865000003, 2024-09-16T20:49:14.141 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051596528000000003, 2024-09-16T20:49:14.141 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:14.141 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:14.141 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:14.141 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:14.141 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:14.141 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:14.141 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3736 2024-09-16T20:49:14.141 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.141 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:14.141 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.141 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:14.141 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:49:14.141 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:14.142 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.142 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.142 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:14.142 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:49:14.142 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:14.142 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.142 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.142 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:14.142 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:49:14.142 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9999999999999999e-07 2024-09-16T20:49:14.142 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.142 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.142 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:14.142 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639335+0000", 2024-09-16T20:49:14.142 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.635e-06 2024-09-16T20:49:14.143 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.143 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.143 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:14.143 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639340+0000", 2024-09-16T20:49:14.143 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5840000000000002e-06 2024-09-16T20:49:14.143 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.143 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.143 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:14.143 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639346+0000", 2024-09-16T20:49:14.143 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1190000000000002e-06 2024-09-16T20:49:14.143 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.143 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.143 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:14.143 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690817+0000", 2024-09-16T20:49:14.144 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051470721999999997 2024-09-16T20:49:14.144 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.144 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.144 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:14.144 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690875+0000", 2024-09-16T20:49:14.144 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7441000000000001e-05 2024-09-16T20:49:14.144 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.144 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.144 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:14.144 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690929+0000", 2024-09-16T20:49:14.144 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4727e-05 2024-09-16T20:49:14.144 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:14.144 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:14.144 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:14.145 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.145 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.145 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5434 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:14.145 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:49:14.145 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 51.217477299000002, 2024-09-16T20:49:14.145 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099755831000000003, 2024-09-16T20:49:14.145 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:14.145 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:14.145 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:14.145 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:14.145 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:14.145 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:14.145 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5434 2024-09-16T20:49:14.145 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.146 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:14.146 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.146 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:14.146 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:49:14.146 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:14.146 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.147 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.147 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:14.147 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:49:14.147 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:14.147 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.147 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.147 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:14.147 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864595+0000", 2024-09-16T20:49:14.147 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1099999999999995e-07 2024-09-16T20:49:14.147 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.147 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.147 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:14.147 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864597+0000", 2024-09-16T20:49:14.147 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3989999999999998e-06 2024-09-16T20:49:14.148 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.148 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.148 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:14.148 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864603+0000", 2024-09-16T20:49:14.148 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9309999999999996e-06 2024-09-16T20:49:14.148 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.148 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.148 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:14.148 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864607+0000", 2024-09-16T20:49:14.148 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6200000000000001e-06 2024-09-16T20:49:14.148 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.148 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.148 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:14.149 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964164+0000", 2024-09-16T20:49:14.149 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099557162000000005 2024-09-16T20:49:14.149 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.149 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.149 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:14.149 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964244+0000", 2024-09-16T20:49:14.149 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9597000000000004e-05 2024-09-16T20:49:14.149 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.149 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.149 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:14.149 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964350+0000", 2024-09-16T20:49:14.149 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010641099999999999 2024-09-16T20:49:14.149 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:14.150 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:14.150 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:14.150 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.150 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.150 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5435 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 20480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:14.150 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:49:14.150 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 51.217447477, 2024-09-16T20:49:14.150 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100897102, 2024-09-16T20:49:14.150 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:14.150 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:14.150 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:14.150 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:14.150 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:14.150 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:14.150 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5435 2024-09-16T20:49:14.151 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.151 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:14.151 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.151 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:14.151 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:49:14.151 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:14.151 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.151 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.151 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:14.151 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:49:14.151 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:14.151 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.151 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.151 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:14.151 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:49:14.152 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8000000000000001e-07 2024-09-16T20:49:14.152 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.152 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.152 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:14.152 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864626+0000", 2024-09-16T20:49:14.152 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.745e-06 2024-09-16T20:49:14.152 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.152 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.152 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:14.152 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864633+0000", 2024-09-16T20:49:14.152 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3339999999999998e-06 2024-09-16T20:49:14.152 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.152 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.152 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:14.153 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864636+0000", 2024-09-16T20:49:14.153 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1200000000000002e-06 2024-09-16T20:49:14.153 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.153 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.153 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:14.153 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965299+0000", 2024-09-16T20:49:14.153 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10066352000000001 2024-09-16T20:49:14.153 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.153 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.153 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:14.153 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965390+0000", 2024-09-16T20:49:14.153 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0894000000000001e-05 2024-09-16T20:49:14.153 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.154 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.154 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:14.154 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965521+0000", 2024-09-16T20:49:14.154 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013110899999999999 2024-09-16T20:49:14.154 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:14.154 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:14.154 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:14.154 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.154 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.154 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:152 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 327680~32768 out=32768b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:49:14.154 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:49:14.154 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 49.918950535, 2024-09-16T20:49:14.154 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11869405800000001, 2024-09-16T20:49:14.154 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:14.155 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:14.155 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:14.155 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:14.155 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:49:14.155 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:49:14.155 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 152 2024-09-16T20:49:14.155 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.155 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:14.155 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.155 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:14.155 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:49:14.155 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:14.155 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.156 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.156 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:14.156 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:49:14.156 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:14.156 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.156 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.156 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:14.156 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163122+0000", 2024-09-16T20:49:14.156 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6400000000000003e-07 2024-09-16T20:49:14.156 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.156 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.156 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:14.156 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163124+0000", 2024-09-16T20:49:14.156 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.565e-06 2024-09-16T20:49:14.156 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.157 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.157 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:14.157 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163136+0000", 2024-09-16T20:49:14.157 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1717e-05 2024-09-16T20:49:14.157 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.157 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.157 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:14.157 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163142+0000", 2024-09-16T20:49:14.157 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1750000000000002e-06 2024-09-16T20:49:14.157 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.157 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.157 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:14.157 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163606+0000", 2024-09-16T20:49:14.157 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00046418799999999998 2024-09-16T20:49:14.158 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.158 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.158 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:14.158 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163656+0000", 2024-09-16T20:49:14.158 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9852999999999999e-05 2024-09-16T20:49:14.158 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.158 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.158 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:14.158 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281815+0000", 2024-09-16T20:49:14.158 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11815909600000001 2024-09-16T20:49:14.158 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:14.158 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:14.158 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:14.158 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.158 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.158 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:153 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:49:14.159 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:49:14.159 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 49.918917581999999, 2024-09-16T20:49:14.159 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11903462400000001, 2024-09-16T20:49:14.159 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:14.159 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:14.159 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:14.159 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:14.159 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:49:14.159 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:49:14.159 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 153 2024-09-16T20:49:14.159 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.159 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:14.159 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.159 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:14.160 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:49:14.160 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:14.160 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.160 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.160 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:14.160 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:49:14.160 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:14.160 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.160 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.160 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:14.160 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163155+0000", 2024-09-16T20:49:14.160 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.37e-07 2024-09-16T20:49:14.160 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.160 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.160 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:14.161 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163157+0000", 2024-09-16T20:49:14.161 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6010000000000002e-06 2024-09-16T20:49:14.161 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.161 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.161 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:14.161 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163165+0000", 2024-09-16T20:49:14.161 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2619999999999998e-06 2024-09-16T20:49:14.161 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.161 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.161 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:14.161 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163171+0000", 2024-09-16T20:49:14.161 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.711e-06 2024-09-16T20:49:14.161 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.161 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.161 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:14.162 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281868+0000", 2024-09-16T20:49:14.162 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118696622 2024-09-16T20:49:14.162 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.162 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.162 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:14.162 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281969+0000", 2024-09-16T20:49:14.162 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101371 2024-09-16T20:49:14.162 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.162 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.162 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:14.162 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282189+0000", 2024-09-16T20:49:14.163 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00021952000000000001 2024-09-16T20:49:14.163 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:14.163 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:14.163 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:14.163 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.163 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.163 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:154 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 360448~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:49:14.163 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:49:14.163 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 49.918885809999999, 2024-09-16T20:49:14.163 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11932035100000001, 2024-09-16T20:49:14.163 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:14.163 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:14.163 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:14.163 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:14.163 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:49:14.164 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:49:14.164 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 154 2024-09-16T20:49:14.164 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.164 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:14.164 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.164 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:14.164 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:49:14.164 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:14.164 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.164 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.164 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:14.164 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:49:14.164 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:14.165 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.165 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.165 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:14.165 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:49:14.165 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:49:14.165 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.165 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.165 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:14.165 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163189+0000", 2024-09-16T20:49:14.165 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6840000000000001e-06 2024-09-16T20:49:14.165 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.165 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.165 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:14.165 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163196+0000", 2024-09-16T20:49:14.166 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7920000000000004e-06 2024-09-16T20:49:14.166 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.166 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.166 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:14.166 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163201+0000", 2024-09-16T20:49:14.166 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2809999999999999e-06 2024-09-16T20:49:14.166 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.166 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.166 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:14.166 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282219+0000", 2024-09-16T20:49:14.166 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.119018343 2024-09-16T20:49:14.166 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.166 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.166 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:14.167 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282305+0000", 2024-09-16T20:49:14.167 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5865000000000005e-05 2024-09-16T20:49:14.167 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.167 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.167 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:14.167 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282506+0000", 2024-09-16T20:49:14.167 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000200927 2024-09-16T20:49:14.167 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:14.167 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:14.167 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:14.167 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.167 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.167 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop(client.4366.0:3088 3.1e e38/23 3:79e5ec01:::rbd_header.110e891a9b2b:head v 38'306, mlcod=36'304)", 2024-09-16T20:49:14.167 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:49:14.168 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 49.917866942000003, 2024-09-16T20:49:14.168 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118102759, 2024-09-16T20:49:14.168 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:14.168 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:14.168 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:14.168 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:14.168 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.168 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:14.168 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:49:14.168 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:14.168 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.168 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.168 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:14.168 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:49:14.169 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:14.169 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.169 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.169 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:14.169 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:49:14.169 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.85e-07 2024-09-16T20:49:14.169 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.169 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.169 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:14.169 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164212+0000", 2024-09-16T20:49:14.169 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3999999999999997e-06 2024-09-16T20:49:14.169 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.169 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.169 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:14.170 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281553+0000", 2024-09-16T20:49:14.170 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11734135499999999 2024-09-16T20:49:14.170 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.170 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.170 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:14.170 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281567+0000", 2024-09-16T20:49:14.170 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4188999999999999e-05 2024-09-16T20:49:14.170 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.170 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.170 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:14.170 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281653+0000", 2024-09-16T20:49:14.170 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5385999999999999e-05 2024-09-16T20:49:14.170 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.170 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.171 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:14.171 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281679+0000", 2024-09-16T20:49:14.171 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6689000000000001e-05 2024-09-16T20:49:14.171 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.171 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.171 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:14.171 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282277+0000", 2024-09-16T20:49:14.171 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00059798000000000002 2024-09-16T20:49:14.171 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:14.171 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:14.171 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:14.171 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282307+0000", 2024-09-16T20:49:14.171 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9975e-05 2024-09-16T20:49:14.171 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:14.171 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:14.172 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:14.172 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:14.172 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:14.172 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:14.174 INFO:teuthology.orchestra.run.smithi139.stdout:2816 write 0xc84a600 thru 0xc8560c6 (0xbac7 bytes) 2024-09-16T20:49:14.174 INFO:teuthology.orchestra.run.smithi139.stdout:2818 write 0xd4e800 thru 0xd541b8 (0x59b9 bytes) 2024-09-16T20:49:14.174 INFO:teuthology.orchestra.run.smithi139.stdout:2819 write 0x7432a00 thru 0x7439287 (0x6888 bytes) 2024-09-16T20:49:14.174 INFO:teuthology.orchestra.run.smithi139.stdout:2821 punch from 0x5d97200 to 0x5d9a800, (0x3600 bytes) 2024-09-16T20:49:14.174 INFO:teuthology.orchestra.run.smithi139.stdout:2822 trunc from 0xc856200 to 0x6a6400 2024-09-16T20:49:14.174 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:49:14.175 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:14.172+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:14.230 INFO:teuthology.orchestra.run.smithi139.stdout:2828 read 0x4f7e00 thru 0x4f9d5d (0x1f5e bytes) 2024-09-16T20:49:14.253 INFO:teuthology.orchestra.run.smithi139.stdout:2829 trunc from 0x6a6400 to 0xbe77800 2024-09-16T20:49:14.263 INFO:teuthology.orchestra.run.smithi139.stdout:2830 read 0x5e2a800 thru 0x5e38bb7 (0xe3b8 bytes) 2024-09-16T20:49:14.268 INFO:teuthology.orchestra.run.smithi139.stdout:2832 write 0x636c400 thru 0x63702c6 (0x3ec7 bytes) 2024-09-16T20:49:14.268 INFO:teuthology.orchestra.run.smithi139.stdout:2833 punch from 0x3faf600 to 0x3fb4600, (0x5000 bytes) 2024-09-16T20:49:14.270 INFO:teuthology.orchestra.run.smithi139.stdout:2834 write 0x7436c00 thru 0x7445329 (0xe72a bytes) 2024-09-16T20:49:14.270 INFO:teuthology.orchestra.run.smithi139.stdout:2837 write 0xbae6600 thru 0xbaeda0b (0x740c bytes) 2024-09-16T20:49:14.270 INFO:teuthology.orchestra.run.smithi139.stdout:2839 write 0x4c6d400 thru 0x4c71533 (0x4134 bytes) 2024-09-16T20:49:14.270 INFO:teuthology.orchestra.run.smithi139.stdout:2842 read 0x6214000 thru 0x6217454 (0x3455 bytes) 2024-09-16T20:49:14.270 INFO:teuthology.orchestra.run.smithi139.stdout:2843 punch from 0xb594a00 to 0xb59de00, (0x9400 bytes) 2024-09-16T20:49:14.270 INFO:teuthology.orchestra.run.smithi139.stdout:2847 clone 23 order 19 su 131072 sc 7 2024-09-16T20:49:14.273 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:14.272+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:14.305 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:14.305 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:49:14.305 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:49:14.305 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:14.316 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:49:14.374 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:14.374+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:14.422 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:14.423 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:49:14.423 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:49:14.423 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:49:14.423 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:14.431 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:49:14.475 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:14.474+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:14.576 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:14.575+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:14.578 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:14.578 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:49:14.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:49:14.579 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:49:14.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.579 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6415 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3577856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:14.579 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871730+0000", 2024-09-16T20:49:14.579 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 54.705174753000001, 2024-09-16T20:49:14.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.04851482, 2024-09-16T20:49:14.579 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:14.579 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:14.579 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:14.579 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:14.579 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:14.579 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:14.580 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6415 2024-09-16T20:49:14.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.580 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:14.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:14.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871730+0000", 2024-09-16T20:49:14.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:14.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:14.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871730+0000", 2024-09-16T20:49:14.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:14.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:14.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871731+0000", 2024-09-16T20:49:14.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6899999999999998e-07 2024-09-16T20:49:14.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:14.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871733+0000", 2024-09-16T20:49:14.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0600000000000002e-06 2024-09-16T20:49:14.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:14.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871740+0000", 2024-09-16T20:49:14.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3749999999999997e-06 2024-09-16T20:49:14.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:14.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871749+0000", 2024-09-16T20:49:14.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9109999999999999e-06 2024-09-16T20:49:14.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:14.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920129+0000", 2024-09-16T20:49:14.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048380175999999997 2024-09-16T20:49:14.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:14.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920181+0000", 2024-09-16T20:49:14.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2077000000000003e-05 2024-09-16T20:49:14.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:14.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920245+0000", 2024-09-16T20:49:14.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3851999999999998e-05 2024-09-16T20:49:14.583 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:14.583 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:14.583 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:14.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.583 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6417 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3586048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:14.583 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871756+0000", 2024-09-16T20:49:14.583 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 54.705149118000001, 2024-09-16T20:49:14.584 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048628301999999998, 2024-09-16T20:49:14.584 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:14.584 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:14.584 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:14.584 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:14.584 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:14.584 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:14.584 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6417 2024-09-16T20:49:14.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.584 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:14.584 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.584 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:14.584 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871756+0000", 2024-09-16T20:49:14.584 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:14.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:14.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871756+0000", 2024-09-16T20:49:14.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:14.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:14.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871756+0000", 2024-09-16T20:49:14.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0100000000000005e-07 2024-09-16T20:49:14.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:14.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871758+0000", 2024-09-16T20:49:14.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.902e-06 2024-09-16T20:49:14.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:14.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871764+0000", 2024-09-16T20:49:14.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8170000000000004e-06 2024-09-16T20:49:14.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:14.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871769+0000", 2024-09-16T20:49:14.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6500000000000004e-06 2024-09-16T20:49:14.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:14.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920276+0000", 2024-09-16T20:49:14.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048507794999999999 2024-09-16T20:49:14.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:14.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920329+0000", 2024-09-16T20:49:14.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2534999999999998e-05 2024-09-16T20:49:14.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:14.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920384+0000", 2024-09-16T20:49:14.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5102000000000002e-05 2024-09-16T20:49:14.588 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:14.588 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:14.588 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:14.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.588 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6419 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3594240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:14.588 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871775+0000", 2024-09-16T20:49:14.588 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 54.705129941000003, 2024-09-16T20:49:14.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048754724999999999, 2024-09-16T20:49:14.588 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:14.588 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:14.588 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:14.588 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:14.589 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:14.589 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:14.589 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6419 2024-09-16T20:49:14.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.589 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:14.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:14.589 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871775+0000", 2024-09-16T20:49:14.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:14.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:14.589 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871775+0000", 2024-09-16T20:49:14.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:14.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.590 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:14.590 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871775+0000", 2024-09-16T20:49:14.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2500000000000001e-07 2024-09-16T20:49:14.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.590 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:14.590 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871777+0000", 2024-09-16T20:49:14.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9649999999999998e-06 2024-09-16T20:49:14.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.590 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:14.590 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871784+0000", 2024-09-16T20:49:14.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3419999999999996e-06 2024-09-16T20:49:14.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.591 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:14.591 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871789+0000", 2024-09-16T20:49:14.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1889999999999998e-06 2024-09-16T20:49:14.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.591 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:14.591 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920416+0000", 2024-09-16T20:49:14.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048627363999999999 2024-09-16T20:49:14.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.591 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:14.591 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920472+0000", 2024-09-16T20:49:14.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5429000000000001e-05 2024-09-16T20:49:14.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.592 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:14.592 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920530+0000", 2024-09-16T20:49:14.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8111e-05 2024-09-16T20:49:14.592 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:14.592 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:14.592 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:14.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.592 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6420 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3602432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:14.592 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871796+0000", 2024-09-16T20:49:14.592 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 54.705109028000003, 2024-09-16T20:49:14.593 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048857298, 2024-09-16T20:49:14.593 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:14.593 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:14.593 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:14.593 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:14.593 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:14.593 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:14.593 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6420 2024-09-16T20:49:14.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.593 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:14.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.593 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:14.593 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871796+0000", 2024-09-16T20:49:14.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:14.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.594 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:14.594 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871796+0000", 2024-09-16T20:49:14.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:14.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.594 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:14.594 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871854+0000", 2024-09-16T20:49:14.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8187999999999999e-05 2024-09-16T20:49:14.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.595 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:14.595 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871856+0000", 2024-09-16T20:49:14.595 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3549999999999999e-06 2024-09-16T20:49:14.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.595 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:14.595 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871866+0000", 2024-09-16T20:49:14.595 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6500000000000008e-06 2024-09-16T20:49:14.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.595 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:14.595 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871872+0000", 2024-09-16T20:49:14.595 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3799999999999999e-06 2024-09-16T20:49:14.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.596 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.596 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:14.596 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920558+0000", 2024-09-16T20:49:14.596 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048685797000000003 2024-09-16T20:49:14.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.596 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.596 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:14.596 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920603+0000", 2024-09-16T20:49:14.596 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4629999999999998e-05 2024-09-16T20:49:14.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.596 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.596 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:14.596 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920653+0000", 2024-09-16T20:49:14.597 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0297999999999999e-05 2024-09-16T20:49:14.597 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:14.597 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:14.597 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:14.597 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.597 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.597 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6421 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3610624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:14.597 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871882+0000", 2024-09-16T20:49:14.597 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 54.705022960000001, 2024-09-16T20:49:14.597 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048913381999999998, 2024-09-16T20:49:14.597 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:14.597 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:14.597 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:14.597 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:14.598 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:14.598 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:14.598 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6421 2024-09-16T20:49:14.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.598 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:14.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.598 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:14.598 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871882+0000", 2024-09-16T20:49:14.598 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:14.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.598 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:14.598 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871882+0000", 2024-09-16T20:49:14.599 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:14.599 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.599 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:14.599 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.873257+0000", 2024-09-16T20:49:14.599 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001374789 2024-09-16T20:49:14.599 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.599 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:14.599 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.873261+0000", 2024-09-16T20:49:14.599 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5589999999999999e-06 2024-09-16T20:49:14.599 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.599 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:14.600 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.873275+0000", 2024-09-16T20:49:14.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3675999999999999e-05 2024-09-16T20:49:14.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.600 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.600 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:14.600 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.873281+0000", 2024-09-16T20:49:14.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6930000000000004e-06 2024-09-16T20:49:14.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.600 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.600 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:14.600 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920684+0000", 2024-09-16T20:49:14.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047402913999999997 2024-09-16T20:49:14.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.600 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.601 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:14.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920737+0000", 2024-09-16T20:49:14.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3600000000000002e-05 2024-09-16T20:49:14.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.601 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:14.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920795+0000", 2024-09-16T20:49:14.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8150999999999999e-05 2024-09-16T20:49:14.601 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:14.601 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:14.601 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:14.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.602 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:7491 3.1e 3:7c48f6ae:::rbd_object_map.110e891a9b2b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:14.602 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.925865+0000", 2024-09-16T20:49:14.602 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 54.651039816000001, 2024-09-16T20:49:14.602 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048897293000000001, 2024-09-16T20:49:14.602 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:14.602 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:14.602 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:14.602 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:14.602 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:14.602 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:14.602 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 7491 2024-09-16T20:49:14.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.602 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:14.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.603 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:14.603 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925865+0000", 2024-09-16T20:49:14.603 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:14.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.603 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:14.603 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925865+0000", 2024-09-16T20:49:14.603 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:14.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.603 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:14.603 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925866+0000", 2024-09-16T20:49:14.603 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.39e-07 2024-09-16T20:49:14.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.604 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:14.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925868+0000", 2024-09-16T20:49:14.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.508e-06 2024-09-16T20:49:14.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.604 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:14.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925874+0000", 2024-09-16T20:49:14.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7919999999999998e-06 2024-09-16T20:49:14.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.604 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:14.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925879+0000", 2024-09-16T20:49:14.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8579999999999999e-06 2024-09-16T20:49:14.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:14.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.970875+0000", 2024-09-16T20:49:14.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044996412 2024-09-16T20:49:14.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:14.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.970970+0000", 2024-09-16T20:49:14.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4732e-05 2024-09-16T20:49:14.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-16T20:49:14.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.971275+0000", 2024-09-16T20:49:14.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00030464199999999999 2024-09-16T20:49:14.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.606 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:14.606 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.974598+0000", 2024-09-16T20:49:14.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0033238159999999998 2024-09-16T20:49:14.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.606 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:49:14.606 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.974666+0000", 2024-09-16T20:49:14.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7617e-05 2024-09-16T20:49:14.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.606 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:49:14.607 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.974718+0000", 2024-09-16T20:49:14.607 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2259000000000002e-05 2024-09-16T20:49:14.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.607 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:14.607 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.974762+0000", 2024-09-16T20:49:14.607 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4218000000000003e-05 2024-09-16T20:49:14.607 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:14.607 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:14.607 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:14.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.607 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4756 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1513472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:14.608 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689118+0000", 2024-09-16T20:49:14.608 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.887787396999997, 2024-09-16T20:49:14.608 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048535146000000001, 2024-09-16T20:49:14.608 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:14.608 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:14.608 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:14.608 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:14.608 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:49:14.608 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:14.608 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4756 2024-09-16T20:49:14.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.608 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:14.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:14.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689118+0000", 2024-09-16T20:49:14.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:14.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:14.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689118+0000", 2024-09-16T20:49:14.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:14.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:14.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689118+0000", 2024-09-16T20:49:14.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7099999999999997e-07 2024-09-16T20:49:14.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.610 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:14.610 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689120+0000", 2024-09-16T20:49:14.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.787e-06 2024-09-16T20:49:14.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.610 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:14.610 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689126+0000", 2024-09-16T20:49:14.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2720000000000003e-06 2024-09-16T20:49:14.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.610 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:14.610 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689134+0000", 2024-09-16T20:49:14.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2190000000000004e-06 2024-09-16T20:49:14.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.611 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:14.611 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737553+0000", 2024-09-16T20:49:14.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048418993 2024-09-16T20:49:14.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.611 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:14.611 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737598+0000", 2024-09-16T20:49:14.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4743999999999997e-05 2024-09-16T20:49:14.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.611 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:14.611 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737653+0000", 2024-09-16T20:49:14.612 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4759999999999997e-05 2024-09-16T20:49:14.612 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:14.612 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:14.612 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:14.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.612 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4758 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1521664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:14.612 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689141+0000", 2024-09-16T20:49:14.612 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.887764417, 2024-09-16T20:49:14.612 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048640741000000001, 2024-09-16T20:49:14.612 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:14.612 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:14.612 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:14.613 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:14.613 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:49:14.613 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:14.613 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4758 2024-09-16T20:49:14.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.613 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:14.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.613 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:14.613 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689141+0000", 2024-09-16T20:49:14.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:14.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.613 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:14.613 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689141+0000", 2024-09-16T20:49:14.614 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:14.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.614 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:14.614 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689141+0000", 2024-09-16T20:49:14.614 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8799999999999998e-07 2024-09-16T20:49:14.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.614 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:14.614 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689143+0000", 2024-09-16T20:49:14.614 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.945e-06 2024-09-16T20:49:14.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.615 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:14.615 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689149+0000", 2024-09-16T20:49:14.615 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9800000000000003e-06 2024-09-16T20:49:14.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.615 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.615 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:14.615 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689155+0000", 2024-09-16T20:49:14.615 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7509999999999997e-06 2024-09-16T20:49:14.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.615 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.615 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:14.615 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737681+0000", 2024-09-16T20:49:14.615 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048526825000000003 2024-09-16T20:49:14.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.616 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.616 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:14.616 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737729+0000", 2024-09-16T20:49:14.616 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7219999999999999e-05 2024-09-16T20:49:14.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.616 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.616 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:14.616 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737781+0000", 2024-09-16T20:49:14.616 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2632e-05 2024-09-16T20:49:14.616 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:14.616 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:14.616 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:14.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.617 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4760 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1529856~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:14.617 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689161+0000", 2024-09-16T20:49:14.617 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.887744069999997, 2024-09-16T20:49:14.617 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048743981999999998, 2024-09-16T20:49:14.617 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:14.617 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:14.617 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:14.617 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:14.617 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:49:14.617 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:14.617 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4760 2024-09-16T20:49:14.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.617 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:14.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:14.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689161+0000", 2024-09-16T20:49:14.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:14.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:14.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689161+0000", 2024-09-16T20:49:14.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:14.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:14.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689161+0000", 2024-09-16T20:49:14.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5599999999999995e-07 2024-09-16T20:49:14.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.619 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:14.619 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689163+0000", 2024-09-16T20:49:14.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.818e-06 2024-09-16T20:49:14.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.619 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:14.619 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689169+0000", 2024-09-16T20:49:14.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0809999999999999e-06 2024-09-16T20:49:14.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.619 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:14.619 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689174+0000", 2024-09-16T20:49:14.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.938e-06 2024-09-16T20:49:14.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.620 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:14.620 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737807+0000", 2024-09-16T20:49:14.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048633124 2024-09-16T20:49:14.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.620 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:14.620 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737851+0000", 2024-09-16T20:49:14.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3764000000000003e-05 2024-09-16T20:49:14.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.620 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:14.620 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737905+0000", 2024-09-16T20:49:14.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3701e-05 2024-09-16T20:49:14.621 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:14.621 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:14.621 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:14.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.621 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4762 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1538048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:14.621 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689182+0000", 2024-09-16T20:49:14.621 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.887722848000003, 2024-09-16T20:49:14.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048850275999999998, 2024-09-16T20:49:14.621 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:14.621 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:14.621 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:14.622 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:14.622 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:49:14.622 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:14.622 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4762 2024-09-16T20:49:14.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.622 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:14.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.622 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:14.622 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689182+0000", 2024-09-16T20:49:14.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:14.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.622 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:14.622 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689182+0000", 2024-09-16T20:49:14.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:14.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:14.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689182+0000", 2024-09-16T20:49:14.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-16T20:49:14.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:14.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689184+0000", 2024-09-16T20:49:14.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8190000000000001e-06 2024-09-16T20:49:14.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:14.624 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689191+0000", 2024-09-16T20:49:14.624 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2530000000000001e-06 2024-09-16T20:49:14.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.624 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.624 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:14.624 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689195+0000", 2024-09-16T20:49:14.624 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9559999999999996e-06 2024-09-16T20:49:14.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.624 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.624 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:14.624 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737933+0000", 2024-09-16T20:49:14.624 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048737919999999997 2024-09-16T20:49:14.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.624 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.625 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:14.625 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737982+0000", 2024-09-16T20:49:14.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8089e-05 2024-09-16T20:49:14.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.625 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:14.625 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738032+0000", 2024-09-16T20:49:14.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0868999999999997e-05 2024-09-16T20:49:14.625 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:14.625 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:14.625 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:14.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.625 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4764 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1546240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:14.626 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689201+0000", 2024-09-16T20:49:14.626 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.887703491000003, 2024-09-16T20:49:14.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048957550000000002, 2024-09-16T20:49:14.626 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:14.626 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:14.626 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:14.626 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:14.626 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:49:14.626 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:14.626 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4764 2024-09-16T20:49:14.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.626 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:14.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.627 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:14.627 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689201+0000", 2024-09-16T20:49:14.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:14.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.627 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:14.627 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689201+0000", 2024-09-16T20:49:14.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:14.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.627 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:14.627 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689202+0000", 2024-09-16T20:49:14.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4200000000000002e-07 2024-09-16T20:49:14.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:14.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689204+0000", 2024-09-16T20:49:14.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9750000000000001e-06 2024-09-16T20:49:14.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:14.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689210+0000", 2024-09-16T20:49:14.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1e-06 2024-09-16T20:49:14.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:14.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689215+0000", 2024-09-16T20:49:14.629 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0180000000000001e-06 2024-09-16T20:49:14.629 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.629 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.629 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:14.629 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738058+0000", 2024-09-16T20:49:14.629 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048843573000000001 2024-09-16T20:49:14.629 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.629 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.629 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:14.629 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738102+0000", 2024-09-16T20:49:14.629 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3256000000000001e-05 2024-09-16T20:49:14.629 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.629 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:14.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738159+0000", 2024-09-16T20:49:14.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7286e-05 2024-09-16T20:49:14.630 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:14.630 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:14.630 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:14.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.630 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4766 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1554432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:14.630 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689221+0000", 2024-09-16T20:49:14.630 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.887683525, 2024-09-16T20:49:14.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049064996, 2024-09-16T20:49:14.630 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:14.630 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:14.631 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:14.631 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:14.631 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:49:14.631 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:14.631 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4766 2024-09-16T20:49:14.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.631 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:14.631 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.631 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:14.631 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689221+0000", 2024-09-16T20:49:14.631 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:14.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.631 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.631 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:14.632 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689221+0000", 2024-09-16T20:49:14.632 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:14.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.632 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.632 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:14.632 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689222+0000", 2024-09-16T20:49:14.632 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.34e-07 2024-09-16T20:49:14.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.632 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.632 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:14.632 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689224+0000", 2024-09-16T20:49:14.632 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.846e-06 2024-09-16T20:49:14.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.632 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.633 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:14.633 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689247+0000", 2024-09-16T20:49:14.633 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3448e-05 2024-09-16T20:49:14.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.633 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.633 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:14.633 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689254+0000", 2024-09-16T20:49:14.633 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6969999999999996e-06 2024-09-16T20:49:14.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.633 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.633 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:14.633 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738188+0000", 2024-09-16T20:49:14.633 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048934520000000002 2024-09-16T20:49:14.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.634 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:14.634 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738234+0000", 2024-09-16T20:49:14.634 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5695999999999998e-05 2024-09-16T20:49:14.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.634 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:14.634 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738286+0000", 2024-09-16T20:49:14.634 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2355000000000003e-05 2024-09-16T20:49:14.634 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:14.634 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:14.634 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:14.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.635 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4768 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1562624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:14.635 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689262+0000", 2024-09-16T20:49:14.635 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.887642571000001, 2024-09-16T20:49:14.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049157157999999999, 2024-09-16T20:49:14.635 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:14.635 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:14.635 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:14.635 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:14.635 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:49:14.635 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:14.635 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4768 2024-09-16T20:49:14.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.635 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:14.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.636 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:14.636 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689262+0000", 2024-09-16T20:49:14.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:14.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.636 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:14.636 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689262+0000", 2024-09-16T20:49:14.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:14.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.636 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:14.636 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689263+0000", 2024-09-16T20:49:14.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6199999999999998e-07 2024-09-16T20:49:14.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.637 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:14.637 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689270+0000", 2024-09-16T20:49:14.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.798e-06 2024-09-16T20:49:14.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.637 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:14.637 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689277+0000", 2024-09-16T20:49:14.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0890000000000002e-06 2024-09-16T20:49:14.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.637 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:14.638 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689282+0000", 2024-09-16T20:49:14.638 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0470000000000001e-06 2024-09-16T20:49:14.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.638 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.638 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:14.638 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738319+0000", 2024-09-16T20:49:14.638 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049037139 2024-09-16T20:49:14.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.638 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.638 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:14.638 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738365+0000", 2024-09-16T20:49:14.638 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6356999999999997e-05 2024-09-16T20:49:14.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.638 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.639 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:14.639 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738420+0000", 2024-09-16T20:49:14.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4265999999999998e-05 2024-09-16T20:49:14.639 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:14.639 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:14.639 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:14.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.639 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4770 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1570816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:14.639 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689289+0000", 2024-09-16T20:49:14.639 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.887616061000003, 2024-09-16T20:49:14.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049253705000000002, 2024-09-16T20:49:14.639 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:14.639 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:14.639 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:14.640 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:14.640 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:49:14.640 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:14.640 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4770 2024-09-16T20:49:14.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.640 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:14.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.640 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:14.640 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689289+0000", 2024-09-16T20:49:14.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:14.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.640 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:14.640 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689289+0000", 2024-09-16T20:49:14.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:14.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:14.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689289+0000", 2024-09-16T20:49:14.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0999999999999999e-07 2024-09-16T20:49:14.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:14.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689291+0000", 2024-09-16T20:49:14.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0030000000000001e-06 2024-09-16T20:49:14.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:14.642 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689297+0000", 2024-09-16T20:49:14.642 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9479999999999996e-06 2024-09-16T20:49:14.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.642 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:14.642 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689302+0000", 2024-09-16T20:49:14.642 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7389999999999999e-06 2024-09-16T20:49:14.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.642 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:14.642 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738442+0000", 2024-09-16T20:49:14.642 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049140001000000003 2024-09-16T20:49:14.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.642 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:14.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738490+0000", 2024-09-16T20:49:14.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7787000000000002e-05 2024-09-16T20:49:14.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.643 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:14.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738543+0000", 2024-09-16T20:49:14.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2816999999999999e-05 2024-09-16T20:49:14.643 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:14.643 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:14.643 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:14.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.643 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:3088 3.1e 3:79e5ec01:::rbd_header.110e891a9b2b:head [watch watch cookie 140213437171488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:49:14.643 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:49:14.643 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 50.413060043999998, 2024-09-16T20:49:14.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.11895186200000001, 2024-09-16T20:49:14.644 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:14.644 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:14.644 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:14.644 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:14.644 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:14.644 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:14.644 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3088 2024-09-16T20:49:14.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.644 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:14.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.644 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:14.644 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:49:14.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:14.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:14.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:49:14.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:14.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:14.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163846+0000", 2024-09-16T20:49:14.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6799999999999999e-07 2024-09-16T20:49:14.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:14.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163850+0000", 2024-09-16T20:49:14.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.929e-06 2024-09-16T20:49:14.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.646 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:14.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163862+0000", 2024-09-16T20:49:14.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1972e-05 2024-09-16T20:49:14.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.646 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:14.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163867+0000", 2024-09-16T20:49:14.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.451e-06 2024-09-16T20:49:14.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.646 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:14.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163912+0000", 2024-09-16T20:49:14.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4456000000000003e-05 2024-09-16T20:49:14.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.647 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:14.647 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163962+0000", 2024-09-16T20:49:14.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0578000000000003e-05 2024-09-16T20:49:14.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.647 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-16T20:49:14.647 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164058+0000", 2024-09-16T20:49:14.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5934999999999995e-05 2024-09-16T20:49:14.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:49:14.648 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164570+0000", 2024-09-16T20:49:14.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00051241100000000005 2024-09-16T20:49:14.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:14.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282629+0000", 2024-09-16T20:49:14.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.118058986 2024-09-16T20:49:14.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:49:14.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282725+0000", 2024-09-16T20:49:14.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5063999999999996e-05 2024-09-16T20:49:14.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:14.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282797+0000", 2024-09-16T20:49:14.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2212000000000006e-05 2024-09-16T20:49:14.649 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:14.650 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:14.650 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:14.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.650 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:8039 3.1e 3.ac44ff9e (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:49:14.650 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:49:14.650 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.869232695000001, 2024-09-16T20:49:14.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0053310449999999, 2024-09-16T20:49:14.650 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:14.650 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:14.650 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:49:14.650 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:14.650 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:14.651 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:14.651 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8039 2024-09-16T20:49:14.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.651 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:14.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.651 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:14.651 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:49:14.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:14.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.651 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:14.651 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:49:14.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:14.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.652 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:14.652 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707674+0000", 2024-09-16T20:49:14.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6530000000000001e-06 2024-09-16T20:49:14.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.652 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:14.652 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707681+0000", 2024-09-16T20:49:14.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9639999999999997e-06 2024-09-16T20:49:14.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.652 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:14.652 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707700+0000", 2024-09-16T20:49:14.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8674000000000001e-05 2024-09-16T20:49:14.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:14.653 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707708+0000", 2024-09-16T20:49:14.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6810000000000004e-06 2024-09-16T20:49:14.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:14.653 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711293+0000", 2024-09-16T20:49:14.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035848189999999999 2024-09-16T20:49:14.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:49:14.653 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711301+0000", 2024-09-16T20:49:14.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4730000000000003e-06 2024-09-16T20:49:14.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:14.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.712990+0000", 2024-09-16T20:49:14.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.001689348 2024-09-16T20:49:14.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:14.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713003+0000", 2024-09-16T20:49:14.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3433e-05 2024-09-16T20:49:14.654 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:14.655 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:14.655 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:14.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.655 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.655 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:8040 3.1e 3.ac44ff9e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:49:14.655 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:49:14.655 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.869172299999999, 2024-09-16T20:49:14.655 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0053410780000001, 2024-09-16T20:49:14.655 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:14.655 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:14.655 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:49:14.655 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:14.655 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:14.655 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:14.655 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8040 2024-09-16T20:49:14.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.656 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:14.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.656 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:14.656 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:49:14.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:14.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.656 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:14.656 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:49:14.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:14.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.657 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:14.657 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:49:14.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8100000000000003e-07 2024-09-16T20:49:14.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.657 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:14.657 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707737+0000", 2024-09-16T20:49:14.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5020000000000001e-06 2024-09-16T20:49:14.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.657 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:14.657 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707745+0000", 2024-09-16T20:49:14.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2169999999999994e-06 2024-09-16T20:49:14.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.658 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:14.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707751+0000", 2024-09-16T20:49:14.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.263e-06 2024-09-16T20:49:14.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.658 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:14.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711336+0000", 2024-09-16T20:49:14.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035848360000000001 2024-09-16T20:49:14.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.658 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:49:14.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711340+0000", 2024-09-16T20:49:14.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1309999999999999e-06 2024-09-16T20:49:14.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.659 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.659 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:14.659 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713060+0000", 2024-09-16T20:49:14.659 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0017195059999999 2024-09-16T20:49:14.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.659 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.659 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:14.659 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713074+0000", 2024-09-16T20:49:14.659 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4042e-05 2024-09-16T20:49:14.659 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:14.659 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:14.659 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:14.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.659 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.660 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4927.0:4487 3.1e 3.ac44ff9e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:49:14.660 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:49:14.660 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.868652097999998, 2024-09-16T20:49:14.660 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0048840880000001, 2024-09-16T20:49:14.660 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:14.660 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:14.660 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:49:14.660 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:14.660 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4927", 2024-09-16T20:49:14.660 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:14.660 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4487 2024-09-16T20:49:14.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.660 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:14.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.661 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:14.661 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:49:14.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:14.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.661 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:14.661 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:49:14.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:14.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.661 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:14.661 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708254+0000", 2024-09-16T20:49:14.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1800000000000005e-07 2024-09-16T20:49:14.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.662 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:14.662 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708258+0000", 2024-09-16T20:49:14.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8689999999999997e-06 2024-09-16T20:49:14.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.662 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.662 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:14.662 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708267+0000", 2024-09-16T20:49:14.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9590000000000007e-06 2024-09-16T20:49:14.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.662 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.662 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:14.662 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708274+0000", 2024-09-16T20:49:14.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9970000000000001e-06 2024-09-16T20:49:14.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.662 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.663 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:14.663 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711366+0000", 2024-09-16T20:49:14.663 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030919530000000001 2024-09-16T20:49:14.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.663 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:49:14.663 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711372+0000", 2024-09-16T20:49:14.663 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9410000000000004e-06 2024-09-16T20:49:14.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.663 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:14.663 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713126+0000", 2024-09-16T20:49:14.664 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0017537249999999 2024-09-16T20:49:14.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.664 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.664 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:14.664 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713137+0000", 2024-09-16T20:49:14.664 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0825999999999999e-05 2024-09-16T20:49:14.664 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:14.664 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:14.664 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:14.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.664 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.664 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5089.0:11703 3.1b 3:dcd4684e:::rbd_data.110e747f0779.000000000000001b:head [set-alloc-hint object_size 262144 write_size 262144,write 139264~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:49:14.664 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:49:14.665 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 10.438916324000001, 2024-09-16T20:49:14.665 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.084231385000000006, 2024-09-16T20:49:14.665 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:14.665 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:14.665 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:14.665 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:14.665 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5089", 2024-09-16T20:49:14.665 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:49:14.665 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 11703 2024-09-16T20:49:14.665 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.665 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:14.665 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.665 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:14.666 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:49:14.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:14.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.666 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:14.666 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:49:14.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:14.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.666 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:14.666 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:49:14.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2600000000000001e-07 2024-09-16T20:49:14.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.667 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:14.667 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137998+0000", 2024-09-16T20:49:14.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9549999999999998e-06 2024-09-16T20:49:14.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.667 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:14.667 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138008+0000", 2024-09-16T20:49:14.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3789999999999995e-06 2024-09-16T20:49:14.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.667 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:14.667 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138013+0000", 2024-09-16T20:49:14.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2209999999999996e-06 2024-09-16T20:49:14.668 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.668 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.668 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:14.668 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138054+0000", 2024-09-16T20:49:14.668 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1022999999999997e-05 2024-09-16T20:49:14.668 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.668 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.668 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:14.668 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138124+0000", 2024-09-16T20:49:14.668 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0234999999999994e-05 2024-09-16T20:49:14.668 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.668 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.668 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:14.668 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138197+0000", 2024-09-16T20:49:14.669 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2451000000000001e-05 2024-09-16T20:49:14.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.669 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.669 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:14.669 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222010+0000", 2024-09-16T20:49:14.669 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.083813433000000007 2024-09-16T20:49:14.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.669 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.669 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:49:14.669 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222163+0000", 2024-09-16T20:49:14.669 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015274599999999999 2024-09-16T20:49:14.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.669 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.669 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:49:14.670 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222197+0000", 2024-09-16T20:49:14.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3683000000000003e-05 2024-09-16T20:49:14.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.670 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:14.670 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222220+0000", 2024-09-16T20:49:14.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3433000000000001e-05 2024-09-16T20:49:14.670 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:14.670 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:14.670 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:14.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.670 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.5089.0:11702 3.15 e73/64 3:acf33c8b:::rbd_data.110e747f0779.000000000000001a:head v 73'132, mlcod=73'130)", 2024-09-16T20:49:14.670 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:49:14.671 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 10.438648994999999, 2024-09-16T20:49:14.671 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.084079763000000002, 2024-09-16T20:49:14.671 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:14.671 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:14.671 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:14.671 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:14.671 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.671 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:14.671 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:49:14.671 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:14.671 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.671 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.671 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:14.671 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:49:14.672 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:14.672 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.672 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.672 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:14.672 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:49:14.672 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:49:14.672 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.672 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.672 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:14.672 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138263+0000", 2024-09-16T20:49:14.672 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1e-06 2024-09-16T20:49:14.672 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.672 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.672 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:14.673 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221589+0000", 2024-09-16T20:49:14.673 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.083326002999999996 2024-09-16T20:49:14.673 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.673 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.673 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:14.673 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221614+0000", 2024-09-16T20:49:14.673 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5593999999999999e-05 2024-09-16T20:49:14.673 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.673 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.673 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:14.673 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221714+0000", 2024-09-16T20:49:14.673 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9889000000000005e-05 2024-09-16T20:49:14.673 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.673 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.673 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:14.674 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221727+0000", 2024-09-16T20:49:14.674 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3125999999999999e-05 2024-09-16T20:49:14.674 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.674 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.674 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:49:14.674 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222294+0000", 2024-09-16T20:49:14.674 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00056712199999999998 2024-09-16T20:49:14.674 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.674 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.674 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:14.674 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222336+0000", 2024-09-16T20:49:14.674 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1470000000000001e-05 2024-09-16T20:49:14.674 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:14.675 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:14.675 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:14.675 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:14.675 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:14.675 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:14.676 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:49:14.677 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:14.676+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:14.777 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:14.777+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:14.829 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:14.829 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:49:14.829 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.829 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:10800 3.6 3:6638bce1:::rbd_header.110e40e7373e:head [call rbd.get_size in=19b,call rbd.get_object_prefix in=20b,call rbd.get_features in=24b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:14.829 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.828211+0000", 2024-09-16T20:49:14.829 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.00012102799999999999, 2024-09-16T20:49:14.829 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000148288, 2024-09-16T20:49:14.829 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:14.829 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:14.829 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:14.830 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:14.830 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:14.830 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:14.830 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 10800 2024-09-16T20:49:14.830 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.830 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:14.830 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.830 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:14.830 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.828211+0000", 2024-09-16T20:49:14.830 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:14.830 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.830 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.830 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:14.830 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.828211+0000", 2024-09-16T20:49:14.831 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:14.831 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.831 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.831 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:14.831 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.828211+0000", 2024-09-16T20:49:14.831 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.23e-07 2024-09-16T20:49:14.831 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.831 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.831 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:14.831 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.828216+0000", 2024-09-16T20:49:14.831 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4360000000000004e-06 2024-09-16T20:49:14.831 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.831 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.831 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:14.832 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.828227+0000", 2024-09-16T20:49:14.832 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0899e-05 2024-09-16T20:49:14.832 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.832 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.832 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:14.832 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.828232+0000", 2024-09-16T20:49:14.832 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2100000000000001e-06 2024-09-16T20:49:14.832 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.832 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.832 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:14.832 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.828273+0000", 2024-09-16T20:49:14.832 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1350000000000002e-05 2024-09-16T20:49:14.832 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:14.832 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:14.833 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:14.833 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.828332+0000", 2024-09-16T20:49:14.833 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8758999999999998e-05 2024-09-16T20:49:14.833 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:14.833 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:14.833 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:14.833 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:14.833 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-16T20:49:14.833 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 1 2024-09-16T20:49:14.833 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:14.837 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:49:14.878 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:14.876+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:14.880 INFO:teuthology.orchestra.run.smithi139.stdout:truncating image image_client.0-clone22 from 0xbe77800 (overlap 0x6a6400) to 0x353200 2024-09-16T20:49:14.979 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:14.978+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:15.006 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:15.006 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:49:15.006 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:49:15.006 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:49:15.006 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:15.014 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:49:15.080 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:15.079+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:15.129 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:15.129 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:49:15.129 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:49:15.129 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:49:15.129 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.129 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:4904 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 7309312~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:15.129 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767724+0000", 2024-09-16T20:49:15.129 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 55.359416201000002, 2024-09-16T20:49:15.129 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050270214000000001, 2024-09-16T20:49:15.129 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:15.130 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:15.130 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:15.130 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:15.130 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:15.130 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:15.130 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4904 2024-09-16T20:49:15.130 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.130 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:15.130 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.130 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:15.130 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767724+0000", 2024-09-16T20:49:15.130 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:15.130 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.131 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.131 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:15.131 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767724+0000", 2024-09-16T20:49:15.131 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:15.131 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.131 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.131 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:15.131 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767724+0000", 2024-09-16T20:49:15.131 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1399999999999998e-07 2024-09-16T20:49:15.131 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.131 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.131 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:15.131 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767725+0000", 2024-09-16T20:49:15.131 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0380000000000001e-06 2024-09-16T20:49:15.132 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.132 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.132 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:15.132 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767729+0000", 2024-09-16T20:49:15.132 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2490000000000001e-06 2024-09-16T20:49:15.132 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.132 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.132 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:15.132 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767731+0000", 2024-09-16T20:49:15.132 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5940000000000001e-06 2024-09-16T20:49:15.132 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.132 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.132 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:15.133 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.817787+0000", 2024-09-16T20:49:15.133 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050055792000000002 2024-09-16T20:49:15.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.133 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:15.133 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.817883+0000", 2024-09-16T20:49:15.133 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5606999999999995e-05 2024-09-16T20:49:15.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.133 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:15.133 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.817994+0000", 2024-09-16T20:49:15.133 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011162000000000001 2024-09-16T20:49:15.133 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:15.133 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:15.134 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:15.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.134 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.134 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:4906 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 7317504~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:15.134 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767734+0000", 2024-09-16T20:49:15.134 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 55.359405938000002, 2024-09-16T20:49:15.134 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050406608999999998, 2024-09-16T20:49:15.134 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:15.134 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:15.134 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:15.134 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:15.134 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:15.134 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:15.134 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4906 2024-09-16T20:49:15.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.135 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:15.135 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.135 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:15.135 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767734+0000", 2024-09-16T20:49:15.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:15.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.135 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.135 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:15.135 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767734+0000", 2024-09-16T20:49:15.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:15.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.135 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.135 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:15.135 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767735+0000", 2024-09-16T20:49:15.136 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.05e-07 2024-09-16T20:49:15.136 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.136 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.136 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:15.136 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767736+0000", 2024-09-16T20:49:15.136 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0240000000000001e-06 2024-09-16T20:49:15.136 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.136 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.136 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:15.136 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767739+0000", 2024-09-16T20:49:15.136 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.067e-06 2024-09-16T20:49:15.136 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.136 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.137 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:15.137 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767742+0000", 2024-09-16T20:49:15.137 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.23e-06 2024-09-16T20:49:15.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.137 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.137 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:15.137 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818027+0000", 2024-09-16T20:49:15.137 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050285102999999998 2024-09-16T20:49:15.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.137 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.137 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:15.137 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818072+0000", 2024-09-16T20:49:15.137 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5129000000000001e-05 2024-09-16T20:49:15.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.138 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:15.138 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818141+0000", 2024-09-16T20:49:15.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8850999999999995e-05 2024-09-16T20:49:15.138 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:15.138 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:15.138 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:15.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.138 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:4908 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 7325696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:15.138 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767745+0000", 2024-09-16T20:49:15.138 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 55.359395542999998, 2024-09-16T20:49:15.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050654230000000001, 2024-09-16T20:49:15.138 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:15.139 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:15.139 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:15.139 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:15.139 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:15.139 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:15.139 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4908 2024-09-16T20:49:15.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.139 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:15.139 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.139 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:15.139 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767745+0000", 2024-09-16T20:49:15.139 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:15.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.139 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.140 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:15.140 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767745+0000", 2024-09-16T20:49:15.140 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:15.140 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.140 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.140 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:15.140 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767745+0000", 2024-09-16T20:49:15.140 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1e-07 2024-09-16T20:49:15.140 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.140 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.140 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:15.140 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767749+0000", 2024-09-16T20:49:15.140 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.532e-06 2024-09-16T20:49:15.140 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.141 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.141 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:15.141 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767753+0000", 2024-09-16T20:49:15.141 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4939999999999999e-06 2024-09-16T20:49:15.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.141 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.141 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:15.141 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767756+0000", 2024-09-16T20:49:15.141 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.61e-06 2024-09-16T20:49:15.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.141 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.141 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:15.141 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818184+0000", 2024-09-16T20:49:15.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050428615000000003 2024-09-16T20:49:15.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.142 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.142 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:15.142 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818276+0000", 2024-09-16T20:49:15.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2077999999999995e-05 2024-09-16T20:49:15.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.142 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.142 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:15.142 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818399+0000", 2024-09-16T20:49:15.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012269099999999999 2024-09-16T20:49:15.142 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:15.142 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:15.142 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:15.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.143 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:4910 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 7333888~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:15.143 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767759+0000", 2024-09-16T20:49:15.143 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 55.359381653, 2024-09-16T20:49:15.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050815076000000001, 2024-09-16T20:49:15.143 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:15.143 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:15.143 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:15.143 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:15.143 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:15.143 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:15.143 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4910 2024-09-16T20:49:15.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.144 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:15.144 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.144 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:15.144 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767759+0000", 2024-09-16T20:49:15.144 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:15.144 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.144 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.144 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:15.144 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767759+0000", 2024-09-16T20:49:15.144 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:15.144 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.144 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.144 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:15.145 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767759+0000", 2024-09-16T20:49:15.145 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2600000000000001e-07 2024-09-16T20:49:15.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.145 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.145 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:15.145 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767760+0000", 2024-09-16T20:49:15.145 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0640000000000001e-06 2024-09-16T20:49:15.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.145 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.145 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:15.145 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767763+0000", 2024-09-16T20:49:15.145 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.994e-06 2024-09-16T20:49:15.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.145 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.146 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:15.146 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767765+0000", 2024-09-16T20:49:15.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5220000000000002e-06 2024-09-16T20:49:15.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.146 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:15.146 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818438+0000", 2024-09-16T20:49:15.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050672602999999997 2024-09-16T20:49:15.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.146 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:15.146 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818481+0000", 2024-09-16T20:49:15.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2967999999999999e-05 2024-09-16T20:49:15.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.147 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:15.147 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818574+0000", 2024-09-16T20:49:15.147 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2699000000000001e-05 2024-09-16T20:49:15.147 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:15.147 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:15.147 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:15.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.147 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.147 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5012 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2101248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:15.147 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767768+0000", 2024-09-16T20:49:15.147 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 55.359372004000001, 2024-09-16T20:49:15.147 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051084682999999999, 2024-09-16T20:49:15.147 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:15.147 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:15.148 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:15.148 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:15.148 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:15.148 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:15.148 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5012 2024-09-16T20:49:15.148 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.148 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:15.148 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.148 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:15.148 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767768+0000", 2024-09-16T20:49:15.148 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:15.148 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.148 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.148 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:15.148 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767768+0000", 2024-09-16T20:49:15.149 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:15.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.149 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.149 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:15.149 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767769+0000", 2024-09-16T20:49:15.149 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1500000000000001e-07 2024-09-16T20:49:15.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.149 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.149 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:15.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767770+0000", 2024-09-16T20:49:15.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.015e-06 2024-09-16T20:49:15.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.150 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.150 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:15.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767773+0000", 2024-09-16T20:49:15.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1070000000000001e-06 2024-09-16T20:49:15.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:15.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767775+0000", 2024-09-16T20:49:15.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.508e-06 2024-09-16T20:49:15.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:15.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818630+0000", 2024-09-16T20:49:15.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050855282000000002 2024-09-16T20:49:15.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:15.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818732+0000", 2024-09-16T20:49:15.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101531 2024-09-16T20:49:15.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.152 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.152 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:15.152 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818853+0000", 2024-09-16T20:49:15.152 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000121025 2024-09-16T20:49:15.152 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:15.152 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:15.152 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:15.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.152 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.152 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5014 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2109440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:15.152 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767778+0000", 2024-09-16T20:49:15.152 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 55.359362068000003, 2024-09-16T20:49:15.153 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051260776000000001, 2024-09-16T20:49:15.153 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:15.153 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:15.153 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:15.153 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:15.153 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:15.153 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:15.153 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5014 2024-09-16T20:49:15.153 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.153 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:15.153 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.153 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:15.153 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767778+0000", 2024-09-16T20:49:15.153 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:15.154 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.154 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.154 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:15.154 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767778+0000", 2024-09-16T20:49:15.154 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:15.154 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.154 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.154 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:15.154 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767824+0000", 2024-09-16T20:49:15.154 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5890999999999998e-05 2024-09-16T20:49:15.154 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.154 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.154 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:15.154 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767827+0000", 2024-09-16T20:49:15.154 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.644e-06 2024-09-16T20:49:15.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.155 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.155 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:15.155 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767832+0000", 2024-09-16T20:49:15.155 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2689999999999999e-06 2024-09-16T20:49:15.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.155 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.155 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:15.155 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767835+0000", 2024-09-16T20:49:15.155 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2100000000000002e-06 2024-09-16T20:49:15.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.155 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.155 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:15.155 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818887+0000", 2024-09-16T20:49:15.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051051276999999999 2024-09-16T20:49:15.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.156 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.156 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:15.156 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818953+0000", 2024-09-16T20:49:15.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6105999999999994e-05 2024-09-16T20:49:15.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.156 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.156 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:15.156 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819039+0000", 2024-09-16T20:49:15.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6378999999999994e-05 2024-09-16T20:49:15.156 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:15.156 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:15.157 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:15.157 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.157 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.157 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5016 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2117632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:15.157 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767841+0000", 2024-09-16T20:49:15.157 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 55.359299389, 2024-09-16T20:49:15.157 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051433288000000001, 2024-09-16T20:49:15.157 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:15.157 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:15.157 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:15.157 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:15.157 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:15.157 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:15.157 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5016 2024-09-16T20:49:15.157 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.158 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:15.158 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.158 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:15.158 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767841+0000", 2024-09-16T20:49:15.158 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:15.158 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.158 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.158 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:15.158 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767841+0000", 2024-09-16T20:49:15.158 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:15.158 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.158 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.158 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:15.159 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768156+0000", 2024-09-16T20:49:15.159 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00031505699999999999 2024-09-16T20:49:15.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.159 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.159 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:15.159 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768159+0000", 2024-09-16T20:49:15.159 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1659999999999998e-06 2024-09-16T20:49:15.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.159 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.159 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:15.159 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768166+0000", 2024-09-16T20:49:15.159 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0550000000000002e-06 2024-09-16T20:49:15.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.159 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.160 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:15.160 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768171+0000", 2024-09-16T20:49:15.160 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3170000000000003e-06 2024-09-16T20:49:15.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.160 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.160 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:15.160 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819084+0000", 2024-09-16T20:49:15.160 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050913355 2024-09-16T20:49:15.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.160 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.160 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:15.160 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819170+0000", 2024-09-16T20:49:15.160 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6328000000000004e-05 2024-09-16T20:49:15.161 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.161 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.161 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:15.161 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819274+0000", 2024-09-16T20:49:15.161 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010401 2024-09-16T20:49:15.161 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:15.161 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:15.161 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:15.161 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.161 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.161 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5018 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2125824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:15.161 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768178+0000", 2024-09-16T20:49:15.161 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 55.358961854999997, 2024-09-16T20:49:15.162 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051227013000000002, 2024-09-16T20:49:15.162 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:15.162 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:15.162 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:15.162 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:15.162 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:15.162 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:15.162 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5018 2024-09-16T20:49:15.162 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.162 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:15.162 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.162 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:15.162 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768178+0000", 2024-09-16T20:49:15.162 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:15.162 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.163 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.163 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:15.163 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768178+0000", 2024-09-16T20:49:15.163 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:15.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.163 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.163 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:15.163 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768179+0000", 2024-09-16T20:49:15.163 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9200000000000002e-07 2024-09-16T20:49:15.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.163 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.163 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:15.163 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768180+0000", 2024-09-16T20:49:15.163 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3790000000000001e-06 2024-09-16T20:49:15.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.164 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.164 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:15.164 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768184+0000", 2024-09-16T20:49:15.164 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5470000000000001e-06 2024-09-16T20:49:15.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.164 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.164 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:15.164 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768187+0000", 2024-09-16T20:49:15.164 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8119999999999999e-06 2024-09-16T20:49:15.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.164 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.165 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:15.165 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819305+0000", 2024-09-16T20:49:15.165 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051118683999999998 2024-09-16T20:49:15.165 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.165 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.165 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:15.165 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819351+0000", 2024-09-16T20:49:15.165 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5757999999999998e-05 2024-09-16T20:49:15.165 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.165 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.165 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:15.165 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819405+0000", 2024-09-16T20:49:15.165 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4441000000000003e-05 2024-09-16T20:49:15.165 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:15.165 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:15.166 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:15.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.166 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.166 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5020 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2134016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:15.166 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768190+0000", 2024-09-16T20:49:15.166 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 55.358950176999997, 2024-09-16T20:49:15.166 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051449661000000001, 2024-09-16T20:49:15.166 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:15.166 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:15.166 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:15.166 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:15.166 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:15.166 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:15.167 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5020 2024-09-16T20:49:15.167 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.167 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:15.167 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.167 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:15.167 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768190+0000", 2024-09-16T20:49:15.167 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:15.167 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.167 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.167 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:15.167 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768190+0000", 2024-09-16T20:49:15.167 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:15.167 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.167 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.168 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:15.168 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768282+0000", 2024-09-16T20:49:15.168 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2263999999999996e-05 2024-09-16T20:49:15.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.168 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.168 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:15.168 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768286+0000", 2024-09-16T20:49:15.168 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.529e-06 2024-09-16T20:49:15.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.168 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.168 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:15.168 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768294+0000", 2024-09-16T20:49:15.168 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6990000000000003e-06 2024-09-16T20:49:15.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.169 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.169 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:15.169 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768306+0000", 2024-09-16T20:49:15.169 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2112e-05 2024-09-16T20:49:15.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.169 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.169 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:15.169 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819449+0000", 2024-09-16T20:49:15.169 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051143521999999997 2024-09-16T20:49:15.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.169 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.169 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:15.169 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819539+0000", 2024-09-16T20:49:15.169 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0022e-05 2024-09-16T20:49:15.170 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.170 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.170 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:15.170 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819640+0000", 2024-09-16T20:49:15.170 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000100513 2024-09-16T20:49:15.170 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:15.170 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:15.170 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:15.170 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.170 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.170 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5022 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2142208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:15.170 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768326+0000", 2024-09-16T20:49:15.170 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 55.358814787999997, 2024-09-16T20:49:15.170 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051436952000000001, 2024-09-16T20:49:15.170 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:15.170 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:15.171 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:15.171 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:15.171 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:15.171 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:15.171 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5022 2024-09-16T20:49:15.171 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.171 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:15.171 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.171 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:15.171 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768326+0000", 2024-09-16T20:49:15.171 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:15.171 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.171 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.172 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:15.172 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768326+0000", 2024-09-16T20:49:15.172 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:15.172 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.172 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.172 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:15.172 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768477+0000", 2024-09-16T20:49:15.172 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015109300000000001 2024-09-16T20:49:15.172 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.172 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.172 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:15.172 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768480+0000", 2024-09-16T20:49:15.172 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8580000000000002e-06 2024-09-16T20:49:15.173 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.173 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.173 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:15.173 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768489+0000", 2024-09-16T20:49:15.173 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4600000000000003e-06 2024-09-16T20:49:15.173 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.173 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.173 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:15.173 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768493+0000", 2024-09-16T20:49:15.173 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2270000000000004e-06 2024-09-16T20:49:15.173 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.173 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.173 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:15.173 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819672+0000", 2024-09-16T20:49:15.174 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051178605000000002 2024-09-16T20:49:15.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.174 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.174 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:15.174 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819711+0000", 2024-09-16T20:49:15.174 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9357000000000003e-05 2024-09-16T20:49:15.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.174 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.174 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:15.174 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819762+0000", 2024-09-16T20:49:15.174 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1351999999999999e-05 2024-09-16T20:49:15.174 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:15.174 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:15.174 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:15.175 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.175 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.175 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5024 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2150400~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:15.175 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768501+0000", 2024-09-16T20:49:15.175 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 55.358639310000001, 2024-09-16T20:49:15.175 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051402678, 2024-09-16T20:49:15.175 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:15.175 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:15.175 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:15.175 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:15.175 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:15.175 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:15.175 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5024 2024-09-16T20:49:15.175 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.176 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:15.176 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.176 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:15.176 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768501+0000", 2024-09-16T20:49:15.176 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:15.176 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.176 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.176 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:15.176 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768501+0000", 2024-09-16T20:49:15.176 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:15.176 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.176 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.176 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:15.177 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768619+0000", 2024-09-16T20:49:15.177 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118187 2024-09-16T20:49:15.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.177 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.177 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:15.177 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768624+0000", 2024-09-16T20:49:15.177 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3259999999999997e-06 2024-09-16T20:49:15.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.177 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.177 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:15.177 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768631+0000", 2024-09-16T20:49:15.177 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6490000000000008e-06 2024-09-16T20:49:15.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.177 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.177 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:15.178 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768637+0000", 2024-09-16T20:49:15.178 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4210000000000003e-06 2024-09-16T20:49:15.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.178 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.178 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:15.178 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819800+0000", 2024-09-16T20:49:15.178 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051163486000000001 2024-09-16T20:49:15.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.178 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.178 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:15.178 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819845+0000", 2024-09-16T20:49:15.178 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4830999999999998e-05 2024-09-16T20:49:15.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.178 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.179 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:15.179 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819904+0000", 2024-09-16T20:49:15.179 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8777999999999999e-05 2024-09-16T20:49:15.179 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:15.179 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:15.179 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:15.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.179 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.179 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5026 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2158592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:15.179 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768646+0000", 2024-09-16T20:49:15.179 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 55.358493862000003, 2024-09-16T20:49:15.179 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051384386999999997, 2024-09-16T20:49:15.179 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:15.180 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:15.180 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:15.180 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:15.180 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:15.180 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:15.180 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5026 2024-09-16T20:49:15.180 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.180 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:15.180 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.180 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:15.180 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768646+0000", 2024-09-16T20:49:15.180 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:15.180 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.180 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.181 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:15.181 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768646+0000", 2024-09-16T20:49:15.181 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:15.181 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.181 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.181 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:15.181 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768775+0000", 2024-09-16T20:49:15.181 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012813200000000001 2024-09-16T20:49:15.181 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.181 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.181 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:15.181 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768778+0000", 2024-09-16T20:49:15.181 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2150000000000001e-06 2024-09-16T20:49:15.181 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.182 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.182 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:15.182 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768786+0000", 2024-09-16T20:49:15.182 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3259999999999996e-06 2024-09-16T20:49:15.182 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.182 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.182 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:15.182 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768790+0000", 2024-09-16T20:49:15.182 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.216e-06 2024-09-16T20:49:15.182 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.182 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.182 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:15.182 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819939+0000", 2024-09-16T20:49:15.182 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051149018999999997 2024-09-16T20:49:15.183 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.183 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.183 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:15.183 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819979+0000", 2024-09-16T20:49:15.183 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9311999999999999e-05 2024-09-16T20:49:15.183 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.183 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.183 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:15.183 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820031+0000", 2024-09-16T20:49:15.183 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2166999999999997e-05 2024-09-16T20:49:15.183 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:15.183 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:15.183 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:15.183 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.184 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.184 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5028 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2166784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:15.184 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768798+0000", 2024-09-16T20:49:15.184 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 55.358342219000001, 2024-09-16T20:49:15.184 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051388912000000002, 2024-09-16T20:49:15.184 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:15.184 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:15.184 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:15.184 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:15.184 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:15.184 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:15.184 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5028 2024-09-16T20:49:15.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.184 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:15.184 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.185 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:15.185 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768798+0000", 2024-09-16T20:49:15.185 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:15.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.185 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.185 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:15.185 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768798+0000", 2024-09-16T20:49:15.185 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:15.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.185 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.185 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:15.185 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768989+0000", 2024-09-16T20:49:15.185 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000190951 2024-09-16T20:49:15.186 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.186 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.186 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:15.186 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768993+0000", 2024-09-16T20:49:15.186 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9770000000000002e-06 2024-09-16T20:49:15.186 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.186 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.186 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:15.186 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.769002+0000", 2024-09-16T20:49:15.186 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5469999999999997e-06 2024-09-16T20:49:15.186 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.186 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.186 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:15.187 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.769020+0000", 2024-09-16T20:49:15.187 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8216999999999999e-05 2024-09-16T20:49:15.187 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.187 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.187 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:15.187 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820070+0000", 2024-09-16T20:49:15.187 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051049998999999999 2024-09-16T20:49:15.187 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.187 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.187 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:15.187 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820119+0000", 2024-09-16T20:49:15.187 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8962000000000003e-05 2024-09-16T20:49:15.187 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.187 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.188 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:15.188 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820187+0000", 2024-09-16T20:49:15.188 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8258999999999998e-05 2024-09-16T20:49:15.188 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:15.188 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:15.188 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:15.188 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.188 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.188 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5030 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2174976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:15.188 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.769031+0000", 2024-09-16T20:49:15.188 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 55.358109106000001, 2024-09-16T20:49:15.188 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051289830000000002, 2024-09-16T20:49:15.188 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:15.188 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:15.189 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:15.189 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:15.189 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:15.189 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:15.189 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5030 2024-09-16T20:49:15.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.189 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:15.189 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.189 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:15.189 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.769031+0000", 2024-09-16T20:49:15.189 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:15.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.189 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.189 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:15.190 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.769031+0000", 2024-09-16T20:49:15.190 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:15.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.190 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.190 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:15.190 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.773332+0000", 2024-09-16T20:49:15.190 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0043004039999999999 2024-09-16T20:49:15.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.190 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.190 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:15.190 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.773336+0000", 2024-09-16T20:49:15.190 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7849999999999999e-06 2024-09-16T20:49:15.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.190 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.191 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:15.191 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.773347+0000", 2024-09-16T20:49:15.191 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0482999999999999e-05 2024-09-16T20:49:15.191 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.191 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.191 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:15.191 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.773354+0000", 2024-09-16T20:49:15.191 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1559999999999998e-06 2024-09-16T20:49:15.191 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.191 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.191 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:15.191 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820216+0000", 2024-09-16T20:49:15.191 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046861591000000001 2024-09-16T20:49:15.191 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.192 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.192 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:15.192 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820258+0000", 2024-09-16T20:49:15.192 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2653e-05 2024-09-16T20:49:15.192 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.192 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.192 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:15.192 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820321+0000", 2024-09-16T20:49:15.192 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2757999999999998e-05 2024-09-16T20:49:15.192 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:15.192 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:15.192 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:15.192 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.192 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.193 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5067 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 6918144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:15.193 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.775279+0000", 2024-09-16T20:49:15.193 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 55.351860840000001, 2024-09-16T20:49:15.193 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050171387999999997, 2024-09-16T20:49:15.193 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:15.193 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:15.193 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:15.193 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:15.193 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:15.193 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:15.193 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5067 2024-09-16T20:49:15.193 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.193 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:15.193 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.194 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:15.194 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775279+0000", 2024-09-16T20:49:15.194 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:15.194 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.194 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.194 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:15.194 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775279+0000", 2024-09-16T20:49:15.194 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:15.194 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.194 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.194 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:15.194 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775432+0000", 2024-09-16T20:49:15.194 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015257399999999999 2024-09-16T20:49:15.194 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.195 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.195 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:15.195 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775436+0000", 2024-09-16T20:49:15.195 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0180000000000003e-06 2024-09-16T20:49:15.195 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.195 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.195 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:15.195 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775447+0000", 2024-09-16T20:49:15.195 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.077e-05 2024-09-16T20:49:15.195 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.195 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.195 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:15.195 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775456+0000", 2024-09-16T20:49:15.195 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4089999999999992e-06 2024-09-16T20:49:15.196 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.196 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.196 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:15.196 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825150+0000", 2024-09-16T20:49:15.196 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049693711000000002 2024-09-16T20:49:15.196 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.196 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.196 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:15.196 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825337+0000", 2024-09-16T20:49:15.196 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000187214 2024-09-16T20:49:15.196 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.196 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.196 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:15.196 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825451+0000", 2024-09-16T20:49:15.197 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011369199999999999 2024-09-16T20:49:15.197 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:15.197 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:15.197 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:15.197 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.197 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.197 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5068 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2330624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:15.197 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.775469+0000", 2024-09-16T20:49:15.197 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 55.351670878999997, 2024-09-16T20:49:15.197 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050198334999999997, 2024-09-16T20:49:15.197 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:15.197 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:15.197 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:15.198 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:15.198 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:15.198 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:15.198 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5068 2024-09-16T20:49:15.198 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.198 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:15.198 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.198 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:15.198 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775469+0000", 2024-09-16T20:49:15.198 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:15.198 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.198 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.198 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:15.198 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775469+0000", 2024-09-16T20:49:15.198 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:15.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.199 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.199 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:15.199 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775720+0000", 2024-09-16T20:49:15.199 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000250661 2024-09-16T20:49:15.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.199 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.199 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:15.199 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775724+0000", 2024-09-16T20:49:15.199 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8700000000000002e-06 2024-09-16T20:49:15.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.199 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.199 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:15.199 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775733+0000", 2024-09-16T20:49:15.200 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4239999999999999e-06 2024-09-16T20:49:15.200 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.200 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.200 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:15.200 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775740+0000", 2024-09-16T20:49:15.200 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9519999999999997e-06 2024-09-16T20:49:15.200 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.200 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.200 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:15.200 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825484+0000", 2024-09-16T20:49:15.200 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049743385000000001 2024-09-16T20:49:15.200 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.200 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.200 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:15.201 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825572+0000", 2024-09-16T20:49:15.201 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7834e-05 2024-09-16T20:49:15.201 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.201 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.201 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:15.201 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825668+0000", 2024-09-16T20:49:15.201 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6209e-05 2024-09-16T20:49:15.201 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:15.201 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:15.201 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:15.201 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.201 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.201 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5069 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 6926336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:15.201 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.775753+0000", 2024-09-16T20:49:15.202 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 55.351387656, 2024-09-16T20:49:15.202 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050146430999999998, 2024-09-16T20:49:15.202 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:15.202 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:15.202 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:15.202 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:15.202 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:15.202 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:15.202 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5069 2024-09-16T20:49:15.202 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.202 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:15.202 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.202 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:15.202 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775753+0000", 2024-09-16T20:49:15.203 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:15.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.203 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.203 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:15.203 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775753+0000", 2024-09-16T20:49:15.203 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:15.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.203 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.203 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:15.203 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775865+0000", 2024-09-16T20:49:15.203 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000112402 2024-09-16T20:49:15.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.203 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.203 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:15.204 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775869+0000", 2024-09-16T20:49:15.204 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5130000000000001e-06 2024-09-16T20:49:15.204 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.204 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.204 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:15.204 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775877+0000", 2024-09-16T20:49:15.204 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1359999999999997e-06 2024-09-16T20:49:15.204 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.204 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.204 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:15.204 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775882+0000", 2024-09-16T20:49:15.204 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6710000000000004e-06 2024-09-16T20:49:15.204 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.204 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.205 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:15.205 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825729+0000", 2024-09-16T20:49:15.205 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049846898000000001 2024-09-16T20:49:15.205 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.205 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.205 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:15.205 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825807+0000", 2024-09-16T20:49:15.205 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7875999999999998e-05 2024-09-16T20:49:15.205 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.205 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.205 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:15.205 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825899+0000", 2024-09-16T20:49:15.205 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1935000000000007e-05 2024-09-16T20:49:15.205 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:15.206 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:15.206 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:15.206 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.206 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.206 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5070 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2338816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:15.206 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.775892+0000", 2024-09-16T20:49:15.206 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 55.351247802000003, 2024-09-16T20:49:15.206 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050155311000000001, 2024-09-16T20:49:15.206 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:15.206 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:15.206 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:15.206 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:15.206 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:15.206 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:15.207 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5070 2024-09-16T20:49:15.207 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.207 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:15.207 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.207 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:15.207 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775892+0000", 2024-09-16T20:49:15.207 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:15.207 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.207 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.207 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:15.207 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775892+0000", 2024-09-16T20:49:15.207 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:15.207 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.207 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.208 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:15.208 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776142+0000", 2024-09-16T20:49:15.208 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00024950500000000002 2024-09-16T20:49:15.208 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.208 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.208 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:15.208 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776146+0000", 2024-09-16T20:49:15.208 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2819999999999998e-06 2024-09-16T20:49:15.208 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.208 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.208 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:15.208 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776157+0000", 2024-09-16T20:49:15.208 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0807e-05 2024-09-16T20:49:15.208 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.209 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.209 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:15.209 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776165+0000", 2024-09-16T20:49:15.209 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1300000000000001e-06 2024-09-16T20:49:15.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.209 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.209 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:15.209 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825935+0000", 2024-09-16T20:49:15.209 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049769300000000002 2024-09-16T20:49:15.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.209 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.209 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:15.209 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825991+0000", 2024-09-16T20:49:15.209 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6177000000000001e-05 2024-09-16T20:49:15.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.210 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.210 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:15.210 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826048+0000", 2024-09-16T20:49:15.210 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.711e-05 2024-09-16T20:49:15.210 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:15.210 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:15.210 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:15.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.210 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.210 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5071 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 6934528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:15.210 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.776177+0000", 2024-09-16T20:49:15.210 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 55.350963399000001, 2024-09-16T20:49:15.211 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050036886000000003, 2024-09-16T20:49:15.211 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:15.211 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:15.211 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:15.211 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:15.211 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:15.211 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:15.211 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5071 2024-09-16T20:49:15.211 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.211 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:15.211 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.211 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:15.211 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776177+0000", 2024-09-16T20:49:15.211 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:15.212 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.212 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.212 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:15.212 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776177+0000", 2024-09-16T20:49:15.212 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:15.212 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.212 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.212 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:15.212 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776339+0000", 2024-09-16T20:49:15.212 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016238099999999999 2024-09-16T20:49:15.212 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.212 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.212 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:15.212 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776343+0000", 2024-09-16T20:49:15.212 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9670000000000003e-06 2024-09-16T20:49:15.213 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.213 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.213 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:15.213 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776354+0000", 2024-09-16T20:49:15.213 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0789e-05 2024-09-16T20:49:15.213 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.213 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.213 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:15.213 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776361+0000", 2024-09-16T20:49:15.213 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5200000000000003e-06 2024-09-16T20:49:15.213 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.213 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.213 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:15.213 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826078+0000", 2024-09-16T20:49:15.214 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049717513999999997 2024-09-16T20:49:15.214 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.214 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.214 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:15.214 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826141+0000", 2024-09-16T20:49:15.214 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2446000000000005e-05 2024-09-16T20:49:15.214 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.214 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.214 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:15.214 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826214+0000", 2024-09-16T20:49:15.214 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3269000000000006e-05 2024-09-16T20:49:15.214 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:15.214 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:15.214 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:15.214 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.215 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.215 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5072 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2347008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:15.215 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.776370+0000", 2024-09-16T20:49:15.215 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 55.350769837999998, 2024-09-16T20:49:15.215 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050097653999999998, 2024-09-16T20:49:15.215 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:15.215 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:15.215 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:15.215 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:15.215 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:15.215 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:15.215 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5072 2024-09-16T20:49:15.215 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.215 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:15.215 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.216 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:15.216 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776370+0000", 2024-09-16T20:49:15.216 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:15.216 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.216 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.216 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:15.216 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776370+0000", 2024-09-16T20:49:15.216 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:15.216 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.216 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.216 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:15.216 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776645+0000", 2024-09-16T20:49:15.216 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000274634 2024-09-16T20:49:15.216 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.216 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.217 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:15.217 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776654+0000", 2024-09-16T20:49:15.217 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9390000000000003e-06 2024-09-16T20:49:15.217 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.217 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.217 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:15.217 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776665+0000", 2024-09-16T20:49:15.217 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0524e-05 2024-09-16T20:49:15.217 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.217 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.217 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:15.217 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776671+0000", 2024-09-16T20:49:15.217 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7390000000000002e-06 2024-09-16T20:49:15.217 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.218 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.218 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:15.218 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826261+0000", 2024-09-16T20:49:15.218 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049589995999999997 2024-09-16T20:49:15.218 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.218 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.218 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:15.218 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826363+0000", 2024-09-16T20:49:15.218 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010158999999999999 2024-09-16T20:49:15.218 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:15.218 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:15.218 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:15.218 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826468+0000", 2024-09-16T20:49:15.218 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000105232 2024-09-16T20:49:15.218 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:15.218 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:15.218 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:15.219 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:15.219 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:15.219 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:15.220 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:49:15.220 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:15.219+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:15.278 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T20:49:15.296 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #21, image image_client.0-clone21 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent22 2024-09-16T20:49:15.321 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:15.320+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:15.373 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:15.373 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:49:15.373 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:49:15.374 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:15.382 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:49:15.410 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:49:15.411 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:49:15.422 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:15.421+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:15.429 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:49:15.429 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:49:15.448 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:49:15.448 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:49:15.468 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:49:15.468 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:49:15.488 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:49:15.488 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:49:15.510 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:49:15.510 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:49:15.522 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:15.522+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:15.541 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:15.541 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:15.550 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:49:15.623 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:15.622+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:15.711 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:15.712 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:15.721 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:49:15.724 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:15.723+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:15.817 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T20:49:15.829 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:15.829 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:15.829 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:15.828+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:15.838 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:49:15.864 INFO:teuthology.orchestra.run.smithi139.stdout:2848 read 0xaea7600 thru 0xaeac9de (0x53df bytes) 2024-09-16T20:49:15.865 INFO:teuthology.orchestra.run.smithi139.stdout:2849 punch from 0xa37f600 to 0xa38ea00, (0xf400 bytes) 2024-09-16T20:49:15.874 INFO:teuthology.orchestra.run.smithi139.stdout:2850 write 0x2bb1800 thru 0x2bb86c3 (0x6ec4 bytes) 2024-09-16T20:49:15.874 INFO:teuthology.orchestra.run.smithi139.stdout:2851 read 0x62f8600 thru 0x62f9a75 (0x1476 bytes) 2024-09-16T20:49:15.874 INFO:teuthology.orchestra.run.smithi139.stdout:2853 read 0x4801400 thru 0x480c1e1 (0xade2 bytes) 2024-09-16T20:49:15.897 INFO:teuthology.orchestra.run.smithi139.stdout:2854 write 0xce6f600 thru 0xce789b9 (0x93ba bytes) 2024-09-16T20:49:15.897 INFO:teuthology.orchestra.run.smithi139.stdout:2855 read 0x424d200 thru 0x4254503 (0x7304 bytes) 2024-09-16T20:49:15.897 INFO:teuthology.orchestra.run.smithi139.stdout:2856 punch from 0x8810800 to 0x8816200, (0x5a00 bytes) 2024-09-16T20:49:15.897 INFO:teuthology.orchestra.run.smithi139.stdout:2858 read 0x5ecd600 thru 0x5ecd691 (0x92 bytes) 2024-09-16T20:49:15.897 INFO:teuthology.orchestra.run.smithi139.stdout:2859 write 0xb4c6600 thru 0xb4ce942 (0x8343 bytes) 2024-09-16T20:49:15.897 INFO:teuthology.orchestra.run.smithi139.stdout:2860 punch from 0x7a0ea00 to 0x7a1c200, (0xd800 bytes) 2024-09-16T20:49:15.925 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:15.924+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:15.926 INFO:teuthology.orchestra.run.smithi139.stdout:2861 write 0xd185e00 thru 0xd193f87 (0xe188 bytes) 2024-09-16T20:49:15.927 INFO:teuthology.orchestra.run.smithi139.stdout:2862 trunc from 0xd194000 to 0xc87de00 2024-09-16T20:49:15.956 INFO:teuthology.orchestra.run.smithi139.stdout:2864 punch from 0x3ac6600 to 0x3ad0400, (0x9e00 bytes) 2024-09-16T20:49:15.957 INFO:teuthology.orchestra.run.smithi139.stdout:2865 punch from 0x644e400 to 0x6456e00, (0x8a00 bytes) 2024-09-16T20:49:15.958 INFO:teuthology.orchestra.run.smithi139.stdout:2866 write 0xa1be400 thru 0xa1c4b8e (0x678f bytes) 2024-09-16T20:49:15.961 INFO:teuthology.orchestra.run.smithi139.stdout:2869 read 0xb9aae00 thru 0xb9ae500 (0x3701 bytes) 2024-09-16T20:49:15.961 INFO:teuthology.orchestra.run.smithi139.stdout:2871 trunc from 0xc87de00 to 0xc314800 2024-09-16T20:49:15.992 INFO:teuthology.orchestra.run.smithi139.stdout:2872 write 0x8302c00 thru 0x8310bd0 (0xdfd1 bytes) 2024-09-16T20:49:15.992 INFO:teuthology.orchestra.run.smithi139.stdout:2873 punch from 0x997a200 to 0x9980e00, (0x6c00 bytes) 2024-09-16T20:49:15.993 INFO:teuthology.orchestra.run.smithi139.stdout:2875 write 0x75f7a00 thru 0x7600949 (0x8f4a bytes) 2024-09-16T20:49:15.994 INFO:teuthology.orchestra.run.smithi139.stdout:2876 read 0x52a2e00 thru 0x52a70de (0x42df bytes) 2024-09-16T20:49:15.994 INFO:teuthology.orchestra.run.smithi139.stdout:2877 write 0x103c000 thru 0x103c60e (0x60f bytes) 2024-09-16T20:49:15.994 INFO:teuthology.orchestra.run.smithi139.stdout:2878 read 0xb988000 thru 0xb9935f1 (0xb5f2 bytes) 2024-09-16T20:49:15.994 INFO:teuthology.orchestra.run.smithi139.stdout:2879 write 0xcb0e00 thru 0xcb6db8 (0x5fb9 bytes) 2024-09-16T20:49:15.994 INFO:teuthology.orchestra.run.smithi139.stdout:2880 read 0x8777200 thru 0x8778c1d (0x1a1e bytes) 2024-09-16T20:49:16.015 INFO:teuthology.orchestra.run.smithi139.stdout:2883 write 0xcab1000 thru 0xcab6aa2 (0x5aa3 bytes) 2024-09-16T20:49:16.015 INFO:teuthology.orchestra.run.smithi139.stdout:2884 read 0x9c79000 thru 0x9c7eafe (0x5aff bytes) 2024-09-16T20:49:16.015 INFO:teuthology.orchestra.run.smithi139.stdout:2887 write 0x5ed9600 thru 0x5ee38f1 (0xa2f2 bytes) 2024-09-16T20:49:16.015 INFO:teuthology.orchestra.run.smithi139.stdout:2888 read 0xca3ee00 thru 0xca4a88f (0xba90 bytes) 2024-09-16T20:49:16.015 INFO:teuthology.orchestra.run.smithi139.stdout:2889 read 0x62a000 thru 0x639684 (0xf685 bytes) 2024-09-16T20:49:16.016 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:16.016 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:16.025 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:49:16.066 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:16.065+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:16.128 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:16.127+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:16.172 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:16.172 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:16.180 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:49:16.229 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:16.229+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:16.244 INFO:teuthology.orchestra.run.smithi139.stdout:2890 trunc from 0xcab6c00 to 0x2365600 2024-09-16T20:49:16.304 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:16.305 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:16.308 INFO:teuthology.orchestra.run.smithi139.stdout:2891 write 0x8fa2000 thru 0x8fa432e (0x232f bytes) 2024-09-16T20:49:16.308 INFO:teuthology.orchestra.run.smithi139.stdout:2892 flatten 2024-09-16T20:49:16.317 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:49:16.330 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:16.329+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:16.430 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:16.429+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:16.466 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:16.466 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:16.478 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:49:16.562 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:16.561+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:16.633 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:16.632+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:16.637 INFO:teuthology.orchestra.run.smithi139.stdout:2895 read 0x2788c00 thru 0x278b08f (0x2490 bytes) 2024-09-16T20:49:16.637 INFO:teuthology.orchestra.run.smithi139.stdout:2896 read 0x3dba600 thru 0x3dbea9d (0x449e bytes) 2024-09-16T20:49:16.641 INFO:teuthology.orchestra.run.smithi139.stdout:2899 trunc from 0x8fa4400 to 0xbc1ea00 2024-09-16T20:49:16.647 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:16.648 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:16.651 INFO:teuthology.orchestra.run.smithi139.stdout:2900 read 0x9015000 thru 0x901a21a (0x521b bytes) 2024-09-16T20:49:16.653 INFO:teuthology.orchestra.run.smithi139.stdout:2901 read 0x8325000 thru 0x832f76b (0xa76c bytes) 2024-09-16T20:49:16.653 INFO:teuthology.orchestra.run.smithi139.stdout:2903 write 0xac97c00 thru 0xaca1b53 (0x9f54 bytes) 2024-09-16T20:49:16.653 INFO:teuthology.orchestra.run.smithi139.stdout:2904 read 0x1afe000 thru 0x1b08aeb (0xaaec bytes) 2024-09-16T20:49:16.657 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:49:16.663 INFO:teuthology.orchestra.run.smithi139.stdout:2906 trunc from 0xbc1ea00 to 0x34a2200 2024-09-16T20:49:16.681 INFO:teuthology.orchestra.run.smithi139.stdout:2908 read 0x31baa00 thru 0x31c9ec7 (0xf4c8 bytes) 2024-09-16T20:49:16.682 INFO:teuthology.orchestra.run.smithi139.stdout:2909 read 0x1c8e800 thru 0x1c9e229 (0xfa2a bytes) 2024-09-16T20:49:16.706 INFO:teuthology.orchestra.run.smithi139.stdout:2911 write 0xea8b600 thru 0xea9b4bf (0xfec0 bytes) 2024-09-16T20:49:16.707 INFO:teuthology.orchestra.run.smithi139.stdout:2913 read 0xc8d7400 thru 0xc8dc18c (0x4d8d bytes) 2024-09-16T20:49:16.716 INFO:teuthology.orchestra.run.smithi139.stdout:2915 trunc from 0xea9b600 to 0x7371a00 2024-09-16T20:49:16.734 INFO:teuthology.orchestra.run.smithi139.stdout:2916 read 0x4717600 thru 0x471d3b4 (0x5db5 bytes) 2024-09-16T20:49:16.735 INFO:teuthology.orchestra.run.smithi139.stdout:2918 read 0x601c800 thru 0x601eb03 (0x2304 bytes) 2024-09-16T20:49:16.735 INFO:teuthology.orchestra.run.smithi139.stdout:2919 write 0x2a5d000 thru 0x2a628b3 (0x58b4 bytes) 2024-09-16T20:49:16.735 INFO:teuthology.orchestra.run.smithi139.stdout:2920 read 0x388f200 thru 0x389811c (0x8f1d bytes) 2024-09-16T20:49:16.735 INFO:teuthology.orchestra.run.smithi139.stdout:2923 punch from 0x4285600 to 0x4286400, (0xe00 bytes) 2024-09-16T20:49:16.741 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:16.740+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:16.753 INFO:teuthology.orchestra.run.smithi139.stdout:2926 write 0xd701e00 thru 0xd70d2ec (0xb4ed bytes) 2024-09-16T20:49:16.754 INFO:teuthology.orchestra.run.smithi139.stdout:2927 read 0x325e600 thru 0x3262e7d (0x487e bytes) 2024-09-16T20:49:16.754 INFO:teuthology.orchestra.run.smithi139.stdout:2928 read 0x9bffa00 thru 0x9c0c7b6 (0xcdb7 bytes) 2024-09-16T20:49:16.765 INFO:teuthology.orchestra.run.smithi139.stdout:2931 trunc from 0xd70d400 to 0x4870600 2024-09-16T20:49:16.787 INFO:teuthology.orchestra.run.smithi139.stdout:2933 punch from 0x25a1c00 to 0x25aa000, (0x8400 bytes) 2024-09-16T20:49:16.794 INFO:teuthology.orchestra.run.smithi139.stdout:2936 trunc from 0x4870600 to 0x27b2000 2024-09-16T20:49:16.808 INFO:teuthology.orchestra.run.smithi029.stderr:nodeep-scrub is unset 2024-09-16T20:49:16.814 INFO:teuthology.orchestra.run.smithi139.stdout:2937 read 0xf41600 thru 0xf47d81 (0x6782 bytes) 2024-09-16T20:49:16.822 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:16.822 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:16.832 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:49:16.835 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:16.834+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:16.840 INFO:teuthology.orchestra.run.smithi139.stdout:2939 write 0x92d5c00 thru 0x92daa46 (0x4e47 bytes) 2024-09-16T20:49:16.841 INFO:teuthology.orchestra.run.smithi139.stdout:2941 write 0x80a7c00 thru 0x80b3c41 (0xc042 bytes) 2024-09-16T20:49:16.841 INFO:teuthology.orchestra.run.smithi139.stdout:2942 write 0x52ed600 thru 0x52fccc7 (0xf6c8 bytes) 2024-09-16T20:49:16.841 INFO:teuthology.orchestra.run.smithi139.stdout:2946 write 0x6f7e000 thru 0x6f7f5b9 (0x15ba bytes) 2024-09-16T20:49:16.841 INFO:teuthology.orchestra.run.smithi139.stdout:2947 punch from 0x6434e00 to 0x6437000, (0x2200 bytes) 2024-09-16T20:49:16.857 INFO:teuthology.orchestra.run.smithi139.stdout:2950 write 0xe3d7e00 thru 0xe3dcc4d (0x4e4e bytes) 2024-09-16T20:49:16.864 INFO:teuthology.orchestra.run.smithi139.stdout:2951 read 0x50d1600 thru 0x50d2306 (0xd07 bytes) 2024-09-16T20:49:16.864 INFO:teuthology.orchestra.run.smithi139.stdout:2952 read 0x1038c00 thru 0x1047b4c (0xef4d bytes) 2024-09-16T20:49:16.866 INFO:teuthology.orchestra.run.smithi139.stdout:2953 trunc from 0xe3dce00 to 0xd3c4600 2024-09-16T20:49:16.888 INFO:teuthology.orchestra.run.smithi139.stdout:2954 trunc from 0xd3c4600 to 0xdd68200 2024-09-16T20:49:16.906 INFO:teuthology.orchestra.run.smithi139.stdout:2955 read 0x600be00 thru 0x60154a3 (0x96a4 bytes) 2024-09-16T20:49:16.906 INFO:teuthology.orchestra.run.smithi139.stdout:2958 write 0x3611600 thru 0x362097a (0xf37b bytes) 2024-09-16T20:49:16.906 INFO:teuthology.orchestra.run.smithi139.stdout:2959 punch from 0xbcdc00 to 0xbd9400, (0xb800 bytes) 2024-09-16T20:49:16.907 INFO:teuthology.orchestra.run.smithi139.stdout:2961 read 0x4d7e000 thru 0x4d89112 (0xb113 bytes) 2024-09-16T20:49:16.919 INFO:teuthology.orchestra.run.smithi139.stdout:2962 trunc from 0xdd68200 to 0x2f34200 2024-09-16T20:49:16.936 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:16.935+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:16.945 INFO:teuthology.orchestra.run.smithi139.stdout:2963 punch from 0x15c0000 to 0x15cde00, (0xde00 bytes) 2024-09-16T20:49:16.955 INFO:teuthology.orchestra.run.smithi139.stdout:2964 trunc from 0x2f34200 to 0xbb15a00 2024-09-16T20:49:16.966 INFO:teuthology.orchestra.run.smithi139.stdout:2965 write 0x2c2a600 thru 0x2c31a32 (0x7433 bytes) 2024-09-16T20:49:16.967 INFO:teuthology.orchestra.run.smithi139.stdout:2967 punch from 0x808aa00 to 0x8091400, (0x6a00 bytes) 2024-09-16T20:49:16.967 INFO:teuthology.orchestra.run.smithi139.stdout:2968 write 0x5cde800 thru 0x5ce1b19 (0x331a bytes) 2024-09-16T20:49:16.967 INFO:teuthology.orchestra.run.smithi139.stdout:2970 read 0x2dcfa00 thru 0x2dd5c31 (0x6232 bytes) 2024-09-16T20:49:16.967 INFO:teuthology.orchestra.run.smithi139.stdout:2985 read 0x367f600 thru 0x3688c9b (0x969c bytes) 2024-09-16T20:49:16.968 INFO:teuthology.orchestra.run.smithi139.stdout:2989 read 0x5309e00 thru 0x5313b67 (0x9d68 bytes) 2024-09-16T20:49:16.968 INFO:teuthology.orchestra.run.smithi139.stdout:2993 read 0x78a5e00 thru 0x78aeab1 (0x8cb2 bytes) 2024-09-16T20:49:16.968 INFO:teuthology.orchestra.run.smithi139.stdout:2999 read 0x86d8000 thru 0x86e39ac (0xb9ad bytes) 2024-09-16T20:49:16.982 INFO:teuthology.orchestra.run.smithi139.stdout:3000 write 0xbfc2c00 thru 0xbfcc737 (0x9b38 bytes) 2024-09-16T20:49:16.983 INFO:teuthology.orchestra.run.smithi139.stdout:3002 punch from 0x8a5ce00 to 0x8a60000, (0x3200 bytes) 2024-09-16T20:49:16.983 INFO:teuthology.orchestra.run.smithi139.stdout:3003 read 0x1f85c00 thru 0x1f8a17e (0x457f bytes) 2024-09-16T20:49:16.986 INFO:teuthology.orchestra.run.smithi139.stdout:3004 trunc from 0xbfcc800 to 0xe1d1600 2024-09-16T20:49:16.994 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:16.994 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:16.996 INFO:teuthology.orchestra.run.smithi139.stdout:3005 read 0x8310200 thru 0x831c508 (0xc309 bytes) 2024-09-16T20:49:16.997 INFO:teuthology.orchestra.run.smithi139.stdout:3007 read 0x1631800 thru 0x163e760 (0xcf61 bytes) 2024-09-16T20:49:16.997 INFO:teuthology.orchestra.run.smithi139.stdout:3008 clone 24 order 19 su 8192 sc 7 2024-09-16T20:49:17.004 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:49:17.037 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:17.036+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:17.111 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:17.111 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:17.120 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:49:17.138 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:17.136+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:17.228 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:17.228 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:17.236 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:49:17.238 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:17.237+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:17.339 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:17.339+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:17.342 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:17.342 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:17.351 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:49:17.440 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:17.440+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:17.498 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:17.498 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:17.506 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:49:17.522 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T20:49:17.547 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:17.547+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:17.642 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:17.642+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:17.647 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:49:17.648 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:49:17.654 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:17.654 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:17.663 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:49:17.664 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:49:17.665 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:49:17.681 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:49:17.681 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:49:17.697 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:49:17.698 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:49:17.714 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:49:17.714 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:49:17.731 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:49:17.731 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:49:17.742 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:17.741+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:17.813 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:17.813 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:49:17.813 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:49:17.813 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:49:17.813 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:17.822 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:49:17.843 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:17.842+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:17.931 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:17.932 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:49:17.932 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:49:17.932 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:49:17.932 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.932 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:4904 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 7309312~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:17.932 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767724+0000", 2024-09-16T20:49:17.932 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 58.162145193000001, 2024-09-16T20:49:17.932 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050270214000000001, 2024-09-16T20:49:17.932 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:17.932 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:17.932 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:17.932 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:17.933 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:17.933 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:17.933 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4904 2024-09-16T20:49:17.933 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.933 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:17.933 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.933 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:17.933 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767724+0000", 2024-09-16T20:49:17.933 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:17.933 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.933 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.934 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:17.934 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767724+0000", 2024-09-16T20:49:17.934 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:17.934 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.934 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.934 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:17.934 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767724+0000", 2024-09-16T20:49:17.934 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1399999999999998e-07 2024-09-16T20:49:17.934 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.934 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.934 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:17.934 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767725+0000", 2024-09-16T20:49:17.934 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0380000000000001e-06 2024-09-16T20:49:17.934 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.934 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.935 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:17.935 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767729+0000", 2024-09-16T20:49:17.935 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2490000000000001e-06 2024-09-16T20:49:17.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.935 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.935 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:17.935 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767731+0000", 2024-09-16T20:49:17.935 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5940000000000001e-06 2024-09-16T20:49:17.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.935 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.935 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:17.935 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.817787+0000", 2024-09-16T20:49:17.935 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050055792000000002 2024-09-16T20:49:17.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.936 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.936 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:17.936 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.817883+0000", 2024-09-16T20:49:17.936 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5606999999999995e-05 2024-09-16T20:49:17.936 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.936 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.936 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:17.936 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.817994+0000", 2024-09-16T20:49:17.936 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011162000000000001 2024-09-16T20:49:17.936 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:17.936 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:17.936 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:17.936 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.936 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.937 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:4906 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 7317504~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:17.937 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767734+0000", 2024-09-16T20:49:17.937 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 58.162134930000001, 2024-09-16T20:49:17.937 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050406608999999998, 2024-09-16T20:49:17.937 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:17.937 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:17.937 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:17.937 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:17.937 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:17.937 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:17.937 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4906 2024-09-16T20:49:17.938 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.938 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:17.938 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.938 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:17.938 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767734+0000", 2024-09-16T20:49:17.938 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:17.938 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.938 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.938 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:17.938 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767734+0000", 2024-09-16T20:49:17.938 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:17.938 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.938 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.938 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:17.939 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767735+0000", 2024-09-16T20:49:17.939 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.05e-07 2024-09-16T20:49:17.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.939 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:17.939 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767736+0000", 2024-09-16T20:49:17.939 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0240000000000001e-06 2024-09-16T20:49:17.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.939 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:17.939 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767739+0000", 2024-09-16T20:49:17.939 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.067e-06 2024-09-16T20:49:17.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:17.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767742+0000", 2024-09-16T20:49:17.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.23e-06 2024-09-16T20:49:17.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:17.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818027+0000", 2024-09-16T20:49:17.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050285102999999998 2024-09-16T20:49:17.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:17.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818072+0000", 2024-09-16T20:49:17.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5129000000000001e-05 2024-09-16T20:49:17.941 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.941 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.941 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:17.941 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818141+0000", 2024-09-16T20:49:17.941 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8850999999999995e-05 2024-09-16T20:49:17.941 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:17.941 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:17.941 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:17.941 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.941 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.941 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:4908 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 7325696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:17.941 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767745+0000", 2024-09-16T20:49:17.941 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 58.162124534999997, 2024-09-16T20:49:17.941 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050654230000000001, 2024-09-16T20:49:17.942 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:17.942 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:17.942 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:17.942 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:17.942 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:17.942 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:17.942 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4908 2024-09-16T20:49:17.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.942 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:17.942 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.942 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:17.942 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767745+0000", 2024-09-16T20:49:17.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:17.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.943 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:17.943 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767745+0000", 2024-09-16T20:49:17.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:17.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.943 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:17.943 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767745+0000", 2024-09-16T20:49:17.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1e-07 2024-09-16T20:49:17.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.943 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:17.944 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767749+0000", 2024-09-16T20:49:17.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.532e-06 2024-09-16T20:49:17.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.944 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:17.944 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767753+0000", 2024-09-16T20:49:17.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4939999999999999e-06 2024-09-16T20:49:17.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.944 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:17.944 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767756+0000", 2024-09-16T20:49:17.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.61e-06 2024-09-16T20:49:17.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.945 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:17.945 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818184+0000", 2024-09-16T20:49:17.945 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050428615000000003 2024-09-16T20:49:17.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.945 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:17.945 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818276+0000", 2024-09-16T20:49:17.945 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2077999999999995e-05 2024-09-16T20:49:17.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.945 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:17.945 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818399+0000", 2024-09-16T20:49:17.945 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012269099999999999 2024-09-16T20:49:17.946 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:17.946 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:17.946 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:17.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.946 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.946 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:4910 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 7333888~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:17.946 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767759+0000", 2024-09-16T20:49:17.946 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 58.162110644999998, 2024-09-16T20:49:17.946 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050815076000000001, 2024-09-16T20:49:17.947 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:17.947 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:17.948 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:17.948 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:17.948 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:17.948 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:17.948 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4910 2024-09-16T20:49:17.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.948 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:17.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.948 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:17.948 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767759+0000", 2024-09-16T20:49:17.948 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:17.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.949 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.949 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:17.949 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767759+0000", 2024-09-16T20:49:17.949 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:17.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.949 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.949 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:17.949 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767759+0000", 2024-09-16T20:49:17.949 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2600000000000001e-07 2024-09-16T20:49:17.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.949 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.949 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:17.949 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767760+0000", 2024-09-16T20:49:17.950 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0640000000000001e-06 2024-09-16T20:49:17.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.950 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.950 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:17.950 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767763+0000", 2024-09-16T20:49:17.950 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.994e-06 2024-09-16T20:49:17.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.950 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.950 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:17.950 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767765+0000", 2024-09-16T20:49:17.950 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5220000000000002e-06 2024-09-16T20:49:17.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.950 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.950 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:17.951 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818438+0000", 2024-09-16T20:49:17.951 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050672602999999997 2024-09-16T20:49:17.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.951 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.951 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:17.951 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818481+0000", 2024-09-16T20:49:17.951 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2967999999999999e-05 2024-09-16T20:49:17.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.951 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.951 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:17.951 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818574+0000", 2024-09-16T20:49:17.951 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2699000000000001e-05 2024-09-16T20:49:17.951 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:17.951 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:17.952 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:17.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.952 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.952 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5012 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2101248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:17.952 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767768+0000", 2024-09-16T20:49:17.952 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 58.162100995999999, 2024-09-16T20:49:17.952 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051084682999999999, 2024-09-16T20:49:17.952 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:17.952 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:17.952 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:17.952 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:17.952 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:17.952 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:17.953 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5012 2024-09-16T20:49:17.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.953 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:17.953 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.953 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:17.953 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767768+0000", 2024-09-16T20:49:17.953 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:17.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.953 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.953 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:17.953 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767768+0000", 2024-09-16T20:49:17.953 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:17.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.954 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.954 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:17.954 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767769+0000", 2024-09-16T20:49:17.954 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1500000000000001e-07 2024-09-16T20:49:17.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.954 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.954 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:17.954 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767770+0000", 2024-09-16T20:49:17.954 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.015e-06 2024-09-16T20:49:17.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.954 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.954 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:17.954 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767773+0000", 2024-09-16T20:49:17.955 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1070000000000001e-06 2024-09-16T20:49:17.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.955 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:17.955 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767775+0000", 2024-09-16T20:49:17.955 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.508e-06 2024-09-16T20:49:17.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.955 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:17.955 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818630+0000", 2024-09-16T20:49:17.955 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050855282000000002 2024-09-16T20:49:17.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.955 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:17.956 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818732+0000", 2024-09-16T20:49:17.956 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101531 2024-09-16T20:49:17.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.956 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:17.956 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818853+0000", 2024-09-16T20:49:17.956 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000121025 2024-09-16T20:49:17.956 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:17.956 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:17.956 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:17.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.956 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5014 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2109440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:17.957 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767778+0000", 2024-09-16T20:49:17.957 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 58.162091060000002, 2024-09-16T20:49:17.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051260776000000001, 2024-09-16T20:49:17.957 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:17.957 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:17.957 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:17.957 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:17.957 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:17.957 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:17.957 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5014 2024-09-16T20:49:17.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.957 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:17.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.958 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:17.958 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767778+0000", 2024-09-16T20:49:17.958 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:17.958 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.960 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:17.960 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767778+0000", 2024-09-16T20:49:17.960 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:17.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.961 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.961 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:17.961 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767824+0000", 2024-09-16T20:49:17.961 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5890999999999998e-05 2024-09-16T20:49:17.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.961 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.961 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:17.961 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767827+0000", 2024-09-16T20:49:17.961 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.644e-06 2024-09-16T20:49:17.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.961 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.961 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:17.961 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767832+0000", 2024-09-16T20:49:17.962 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2689999999999999e-06 2024-09-16T20:49:17.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.962 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:17.962 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767835+0000", 2024-09-16T20:49:17.962 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2100000000000002e-06 2024-09-16T20:49:17.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.962 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:17.962 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818887+0000", 2024-09-16T20:49:17.962 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051051276999999999 2024-09-16T20:49:17.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.963 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:17.963 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818953+0000", 2024-09-16T20:49:17.963 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6105999999999994e-05 2024-09-16T20:49:17.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.963 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.963 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:17.963 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819039+0000", 2024-09-16T20:49:17.963 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6378999999999994e-05 2024-09-16T20:49:17.963 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:17.963 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:17.963 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:17.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.963 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.963 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5016 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2117632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:17.964 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767841+0000", 2024-09-16T20:49:17.964 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 58.162028380999999, 2024-09-16T20:49:17.964 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051433288000000001, 2024-09-16T20:49:17.964 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:17.964 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:17.964 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:17.964 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:17.964 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:17.964 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:17.964 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5016 2024-09-16T20:49:17.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.964 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:17.964 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.965 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:17.965 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767841+0000", 2024-09-16T20:49:17.965 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:17.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.965 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.965 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:17.965 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767841+0000", 2024-09-16T20:49:17.965 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:17.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.965 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.965 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:17.965 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768156+0000", 2024-09-16T20:49:17.965 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00031505699999999999 2024-09-16T20:49:17.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.966 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:17.966 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768159+0000", 2024-09-16T20:49:17.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1659999999999998e-06 2024-09-16T20:49:17.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.966 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:17.966 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768166+0000", 2024-09-16T20:49:17.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0550000000000002e-06 2024-09-16T20:49:17.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.966 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:17.967 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768171+0000", 2024-09-16T20:49:17.967 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3170000000000003e-06 2024-09-16T20:49:17.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.967 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:17.967 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819084+0000", 2024-09-16T20:49:17.967 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050913355 2024-09-16T20:49:17.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.967 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:17.967 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819170+0000", 2024-09-16T20:49:17.967 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6328000000000004e-05 2024-09-16T20:49:17.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.968 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:17.968 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819274+0000", 2024-09-16T20:49:17.968 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010401 2024-09-16T20:49:17.968 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:17.968 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:17.968 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:17.968 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.968 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.968 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5018 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2125824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:17.968 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768178+0000", 2024-09-16T20:49:17.968 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 58.161690847000003, 2024-09-16T20:49:17.968 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051227013000000002, 2024-09-16T20:49:17.968 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:17.969 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:17.969 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:17.969 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:17.969 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:17.969 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:17.969 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5018 2024-09-16T20:49:17.969 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.969 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:17.969 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.969 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:17.969 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768178+0000", 2024-09-16T20:49:17.969 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:17.969 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.969 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.970 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:17.970 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768178+0000", 2024-09-16T20:49:17.970 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:17.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.970 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.970 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:17.970 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768179+0000", 2024-09-16T20:49:17.970 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9200000000000002e-07 2024-09-16T20:49:17.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.970 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.970 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:17.970 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768180+0000", 2024-09-16T20:49:17.970 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3790000000000001e-06 2024-09-16T20:49:17.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.971 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.971 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:17.971 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768184+0000", 2024-09-16T20:49:17.971 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5470000000000001e-06 2024-09-16T20:49:17.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.971 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.971 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:17.971 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768187+0000", 2024-09-16T20:49:17.971 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8119999999999999e-06 2024-09-16T20:49:17.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.971 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.971 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:17.971 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819305+0000", 2024-09-16T20:49:17.972 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051118683999999998 2024-09-16T20:49:17.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.972 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.972 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:17.972 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819351+0000", 2024-09-16T20:49:17.972 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5757999999999998e-05 2024-09-16T20:49:17.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.972 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.972 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:17.972 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819405+0000", 2024-09-16T20:49:17.972 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4441000000000003e-05 2024-09-16T20:49:17.972 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:17.972 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:17.972 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:17.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.973 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.973 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5020 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2134016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:17.973 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768190+0000", 2024-09-16T20:49:17.973 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 58.161679169000003, 2024-09-16T20:49:17.973 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051449661000000001, 2024-09-16T20:49:17.973 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:17.973 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:17.973 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:17.973 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:17.973 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:17.973 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:17.973 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5020 2024-09-16T20:49:17.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.974 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:17.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.974 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:17.974 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768190+0000", 2024-09-16T20:49:17.974 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:17.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.974 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:17.974 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768190+0000", 2024-09-16T20:49:17.974 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:17.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.974 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:17.974 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768282+0000", 2024-09-16T20:49:17.975 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2263999999999996e-05 2024-09-16T20:49:17.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.975 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.975 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:17.975 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768286+0000", 2024-09-16T20:49:17.975 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.529e-06 2024-09-16T20:49:17.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.975 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.975 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:17.975 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768294+0000", 2024-09-16T20:49:17.975 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6990000000000003e-06 2024-09-16T20:49:17.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.975 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.975 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:17.976 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768306+0000", 2024-09-16T20:49:17.976 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2112e-05 2024-09-16T20:49:17.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.976 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.976 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:17.976 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819449+0000", 2024-09-16T20:49:17.976 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051143521999999997 2024-09-16T20:49:17.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.976 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.976 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:17.976 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819539+0000", 2024-09-16T20:49:17.976 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0022e-05 2024-09-16T20:49:17.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.976 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.977 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:17.977 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819640+0000", 2024-09-16T20:49:17.977 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000100513 2024-09-16T20:49:17.977 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:17.977 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:17.977 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:17.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.977 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5022 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2142208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:17.977 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768326+0000", 2024-09-16T20:49:17.977 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 58.161543780000002, 2024-09-16T20:49:17.977 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051436952000000001, 2024-09-16T20:49:17.977 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:17.977 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:17.978 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:17.978 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:17.978 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:17.978 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:17.978 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5022 2024-09-16T20:49:17.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.978 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:17.978 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.978 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:17.978 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768326+0000", 2024-09-16T20:49:17.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:17.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.978 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.979 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:17.979 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768326+0000", 2024-09-16T20:49:17.979 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:17.979 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.979 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.979 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:17.979 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768477+0000", 2024-09-16T20:49:17.979 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015109300000000001 2024-09-16T20:49:17.979 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.979 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.979 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:17.979 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768480+0000", 2024-09-16T20:49:17.979 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8580000000000002e-06 2024-09-16T20:49:17.979 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.980 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.980 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:17.980 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768489+0000", 2024-09-16T20:49:17.980 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4600000000000003e-06 2024-09-16T20:49:17.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.980 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.980 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:17.980 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768493+0000", 2024-09-16T20:49:17.980 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2270000000000004e-06 2024-09-16T20:49:17.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.980 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.980 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:17.980 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819672+0000", 2024-09-16T20:49:17.980 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051178605000000002 2024-09-16T20:49:17.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.981 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.981 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:17.981 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819711+0000", 2024-09-16T20:49:17.981 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9357000000000003e-05 2024-09-16T20:49:17.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.981 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.981 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:17.981 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819762+0000", 2024-09-16T20:49:17.981 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1351999999999999e-05 2024-09-16T20:49:17.981 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:17.981 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:17.981 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:17.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.982 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.982 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5024 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2150400~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:17.982 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768501+0000", 2024-09-16T20:49:17.982 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 58.161368302, 2024-09-16T20:49:17.982 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051402678, 2024-09-16T20:49:17.982 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:17.982 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:17.982 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:17.982 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:17.982 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:17.982 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:17.982 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5024 2024-09-16T20:49:17.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.982 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:17.983 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.983 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:17.983 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768501+0000", 2024-09-16T20:49:17.983 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:17.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.983 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.983 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:17.983 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768501+0000", 2024-09-16T20:49:17.983 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:17.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.983 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.983 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:17.983 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768619+0000", 2024-09-16T20:49:17.983 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118187 2024-09-16T20:49:17.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.984 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.984 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:17.984 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768624+0000", 2024-09-16T20:49:17.984 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3259999999999997e-06 2024-09-16T20:49:17.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.984 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.984 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:17.984 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768631+0000", 2024-09-16T20:49:17.984 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6490000000000008e-06 2024-09-16T20:49:17.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.984 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.984 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:17.984 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768637+0000", 2024-09-16T20:49:17.985 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4210000000000003e-06 2024-09-16T20:49:17.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.985 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.985 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:17.985 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819800+0000", 2024-09-16T20:49:17.985 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051163486000000001 2024-09-16T20:49:17.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.985 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.985 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:17.985 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819845+0000", 2024-09-16T20:49:17.985 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4830999999999998e-05 2024-09-16T20:49:17.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.985 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.985 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:17.986 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819904+0000", 2024-09-16T20:49:17.986 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8777999999999999e-05 2024-09-16T20:49:17.986 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:17.986 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:17.986 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:17.986 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.986 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.986 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5026 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2158592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:17.986 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768646+0000", 2024-09-16T20:49:17.986 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 58.161222854000002, 2024-09-16T20:49:17.986 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051384386999999997, 2024-09-16T20:49:17.986 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:17.986 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:17.986 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:17.987 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:17.987 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:17.987 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:17.987 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5026 2024-09-16T20:49:17.987 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.987 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:17.987 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.987 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:17.987 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768646+0000", 2024-09-16T20:49:17.987 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:17.987 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.987 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.987 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:17.988 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768646+0000", 2024-09-16T20:49:17.988 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:17.988 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.988 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.988 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:17.988 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768775+0000", 2024-09-16T20:49:17.988 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012813200000000001 2024-09-16T20:49:17.988 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.988 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.988 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:17.988 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768778+0000", 2024-09-16T20:49:17.988 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2150000000000001e-06 2024-09-16T20:49:17.988 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.988 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.989 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:17.989 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768786+0000", 2024-09-16T20:49:17.989 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3259999999999996e-06 2024-09-16T20:49:17.989 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.989 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.989 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:17.989 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768790+0000", 2024-09-16T20:49:17.989 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.216e-06 2024-09-16T20:49:17.989 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.989 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.989 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:17.989 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819939+0000", 2024-09-16T20:49:17.989 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051149018999999997 2024-09-16T20:49:17.990 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.990 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.990 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:17.990 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819979+0000", 2024-09-16T20:49:17.990 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9311999999999999e-05 2024-09-16T20:49:17.990 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.990 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.990 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:17.990 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820031+0000", 2024-09-16T20:49:17.990 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2166999999999997e-05 2024-09-16T20:49:17.990 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:17.990 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:17.990 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:17.990 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.990 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.991 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5028 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2166784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:17.991 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768798+0000", 2024-09-16T20:49:17.991 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 58.161071210999999, 2024-09-16T20:49:17.991 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051388912000000002, 2024-09-16T20:49:17.991 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:17.991 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:17.991 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:17.991 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:17.991 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:17.991 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:17.991 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5028 2024-09-16T20:49:17.991 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.991 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:17.991 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.992 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:17.992 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768798+0000", 2024-09-16T20:49:17.992 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:17.992 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.992 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.992 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:17.992 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768798+0000", 2024-09-16T20:49:17.992 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:17.992 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.992 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.992 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:17.992 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768989+0000", 2024-09-16T20:49:17.992 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000190951 2024-09-16T20:49:17.992 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.993 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.993 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:17.993 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768993+0000", 2024-09-16T20:49:17.993 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9770000000000002e-06 2024-09-16T20:49:17.993 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.993 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.993 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:17.993 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.769002+0000", 2024-09-16T20:49:17.993 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5469999999999997e-06 2024-09-16T20:49:17.993 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.993 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.993 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:17.993 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.769020+0000", 2024-09-16T20:49:17.994 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8216999999999999e-05 2024-09-16T20:49:17.994 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.994 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.994 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:17.994 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820070+0000", 2024-09-16T20:49:17.994 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051049998999999999 2024-09-16T20:49:17.994 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.994 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.994 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:17.994 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820119+0000", 2024-09-16T20:49:17.994 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8962000000000003e-05 2024-09-16T20:49:17.994 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.994 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.994 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:17.995 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820187+0000", 2024-09-16T20:49:17.995 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8258999999999998e-05 2024-09-16T20:49:17.995 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:17.995 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:17.995 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:17.995 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.995 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.995 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5030 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2174976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:17.995 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.769031+0000", 2024-09-16T20:49:17.995 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 58.160838097999999, 2024-09-16T20:49:17.995 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051289830000000002, 2024-09-16T20:49:17.995 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:17.995 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:17.996 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:17.996 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:17.996 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:17.996 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:17.996 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5030 2024-09-16T20:49:17.996 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.996 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:17.996 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.996 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:17.996 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.769031+0000", 2024-09-16T20:49:17.996 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:17.996 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.996 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.996 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:17.997 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.769031+0000", 2024-09-16T20:49:17.997 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:17.997 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.997 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.997 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:17.997 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.773332+0000", 2024-09-16T20:49:17.997 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0043004039999999999 2024-09-16T20:49:17.997 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.997 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.997 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:17.997 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.773336+0000", 2024-09-16T20:49:17.997 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7849999999999999e-06 2024-09-16T20:49:17.997 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.998 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.998 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:17.998 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.773347+0000", 2024-09-16T20:49:17.998 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0482999999999999e-05 2024-09-16T20:49:17.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.998 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.998 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:17.998 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.773354+0000", 2024-09-16T20:49:17.998 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1559999999999998e-06 2024-09-16T20:49:17.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.998 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.998 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:17.998 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820216+0000", 2024-09-16T20:49:17.998 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046861591000000001 2024-09-16T20:49:17.999 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.999 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.999 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:17.999 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820258+0000", 2024-09-16T20:49:17.999 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2653e-05 2024-09-16T20:49:17.999 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:17.999 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:17.999 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:17.999 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820321+0000", 2024-09-16T20:49:17.999 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2757999999999998e-05 2024-09-16T20:49:17.999 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:17.999 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:17.999 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:17.999 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.000 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.000 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5067 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 6918144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:18.000 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.775279+0000", 2024-09-16T20:49:18.000 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 58.154589831999999, 2024-09-16T20:49:18.000 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050171387999999997, 2024-09-16T20:49:18.000 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:18.000 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:18.000 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:18.000 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:18.000 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:18.000 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:18.000 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5067 2024-09-16T20:49:18.000 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.001 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:18.001 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.001 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:18.001 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775279+0000", 2024-09-16T20:49:18.001 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:18.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.001 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.001 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:18.001 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775279+0000", 2024-09-16T20:49:18.001 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:18.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.001 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.001 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:18.001 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775432+0000", 2024-09-16T20:49:18.002 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015257399999999999 2024-09-16T20:49:18.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.002 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:18.002 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775436+0000", 2024-09-16T20:49:18.002 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0180000000000003e-06 2024-09-16T20:49:18.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.002 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:18.002 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775447+0000", 2024-09-16T20:49:18.002 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.077e-05 2024-09-16T20:49:18.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.002 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:18.003 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775456+0000", 2024-09-16T20:49:18.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4089999999999992e-06 2024-09-16T20:49:18.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.003 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.003 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:18.003 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825150+0000", 2024-09-16T20:49:18.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049693711000000002 2024-09-16T20:49:18.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.003 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.003 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:18.003 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825337+0000", 2024-09-16T20:49:18.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000187214 2024-09-16T20:49:18.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.003 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.004 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:18.004 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825451+0000", 2024-09-16T20:49:18.004 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011369199999999999 2024-09-16T20:49:18.004 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:18.004 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:18.004 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:18.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.004 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.004 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5068 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2330624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:18.004 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.775469+0000", 2024-09-16T20:49:18.004 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 58.154399871000003, 2024-09-16T20:49:18.004 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050198334999999997, 2024-09-16T20:49:18.004 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:18.004 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:18.005 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:18.005 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:18.005 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:18.005 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:18.005 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5068 2024-09-16T20:49:18.005 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.005 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:18.005 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.005 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:18.005 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775469+0000", 2024-09-16T20:49:18.005 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:18.005 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.005 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.005 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:18.006 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775469+0000", 2024-09-16T20:49:18.006 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:18.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.006 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.006 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:18.006 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775720+0000", 2024-09-16T20:49:18.006 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000250661 2024-09-16T20:49:18.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.006 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.006 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:18.006 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775724+0000", 2024-09-16T20:49:18.006 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8700000000000002e-06 2024-09-16T20:49:18.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.006 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.007 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:18.007 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775733+0000", 2024-09-16T20:49:18.007 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4239999999999999e-06 2024-09-16T20:49:18.007 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.007 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.007 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:18.007 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775740+0000", 2024-09-16T20:49:18.007 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9519999999999997e-06 2024-09-16T20:49:18.007 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.007 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.007 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:18.007 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825484+0000", 2024-09-16T20:49:18.007 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049743385000000001 2024-09-16T20:49:18.007 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.008 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:18.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825572+0000", 2024-09-16T20:49:18.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7834e-05 2024-09-16T20:49:18.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.008 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:18.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825668+0000", 2024-09-16T20:49:18.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6209e-05 2024-09-16T20:49:18.008 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:18.008 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:18.008 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:18.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.009 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.009 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5069 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 6926336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:18.009 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.775753+0000", 2024-09-16T20:49:18.009 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 58.154116647999999, 2024-09-16T20:49:18.009 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050146430999999998, 2024-09-16T20:49:18.009 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:18.009 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:18.009 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:18.009 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:18.009 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:18.009 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:18.009 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5069 2024-09-16T20:49:18.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.009 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:18.010 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.010 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:18.010 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775753+0000", 2024-09-16T20:49:18.010 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:18.010 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.010 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.010 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:18.010 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775753+0000", 2024-09-16T20:49:18.010 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:18.010 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.010 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.010 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:18.010 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775865+0000", 2024-09-16T20:49:18.010 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000112402 2024-09-16T20:49:18.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.011 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.011 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:18.011 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775869+0000", 2024-09-16T20:49:18.011 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5130000000000001e-06 2024-09-16T20:49:18.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.011 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.011 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:18.011 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775877+0000", 2024-09-16T20:49:18.011 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1359999999999997e-06 2024-09-16T20:49:18.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.011 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.011 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:18.011 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775882+0000", 2024-09-16T20:49:18.012 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6710000000000004e-06 2024-09-16T20:49:18.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.012 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.012 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:18.012 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825729+0000", 2024-09-16T20:49:18.012 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049846898000000001 2024-09-16T20:49:18.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.012 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.012 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:18.012 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825807+0000", 2024-09-16T20:49:18.012 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7875999999999998e-05 2024-09-16T20:49:18.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.013 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.013 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:18.013 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825899+0000", 2024-09-16T20:49:18.013 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1935000000000007e-05 2024-09-16T20:49:18.013 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:18.013 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:18.013 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:18.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.013 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.013 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5070 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2338816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:18.013 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.775892+0000", 2024-09-16T20:49:18.013 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 58.153976794000002, 2024-09-16T20:49:18.013 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050155311000000001, 2024-09-16T20:49:18.013 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:18.014 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:18.014 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:18.014 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:18.014 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:18.014 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:18.014 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5070 2024-09-16T20:49:18.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.014 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:18.014 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.014 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:18.014 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775892+0000", 2024-09-16T20:49:18.014 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:18.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.014 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.015 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:18.015 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775892+0000", 2024-09-16T20:49:18.015 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:18.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.015 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.015 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:18.015 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776142+0000", 2024-09-16T20:49:18.015 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00024950500000000002 2024-09-16T20:49:18.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.015 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.015 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:18.015 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776146+0000", 2024-09-16T20:49:18.015 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2819999999999998e-06 2024-09-16T20:49:18.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.016 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.016 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:18.016 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776157+0000", 2024-09-16T20:49:18.016 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0807e-05 2024-09-16T20:49:18.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.016 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.016 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:18.016 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776165+0000", 2024-09-16T20:49:18.016 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1300000000000001e-06 2024-09-16T20:49:18.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.016 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.016 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:18.016 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825935+0000", 2024-09-16T20:49:18.016 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049769300000000002 2024-09-16T20:49:18.017 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.017 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.017 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:18.017 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825991+0000", 2024-09-16T20:49:18.017 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6177000000000001e-05 2024-09-16T20:49:18.017 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.017 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.017 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:18.017 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826048+0000", 2024-09-16T20:49:18.017 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.711e-05 2024-09-16T20:49:18.017 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:18.017 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:18.017 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:18.017 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.018 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.018 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5071 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 6934528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:18.018 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.776177+0000", 2024-09-16T20:49:18.018 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 58.153692391, 2024-09-16T20:49:18.018 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050036886000000003, 2024-09-16T20:49:18.018 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:18.018 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:18.018 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:18.018 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:18.018 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:18.018 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:18.018 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5071 2024-09-16T20:49:18.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.018 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:18.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.019 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:18.019 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776177+0000", 2024-09-16T20:49:18.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:18.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.019 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:18.019 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776177+0000", 2024-09-16T20:49:18.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:18.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.019 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:18.019 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776339+0000", 2024-09-16T20:49:18.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016238099999999999 2024-09-16T20:49:18.020 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.020 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.020 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:18.020 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776343+0000", 2024-09-16T20:49:18.020 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9670000000000003e-06 2024-09-16T20:49:18.020 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.020 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.020 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:18.020 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776354+0000", 2024-09-16T20:49:18.020 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0789e-05 2024-09-16T20:49:18.020 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.020 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.020 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:18.020 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776361+0000", 2024-09-16T20:49:18.020 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5200000000000003e-06 2024-09-16T20:49:18.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.021 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.021 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:18.021 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826078+0000", 2024-09-16T20:49:18.021 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049717513999999997 2024-09-16T20:49:18.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.021 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.021 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:18.021 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826141+0000", 2024-09-16T20:49:18.021 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2446000000000005e-05 2024-09-16T20:49:18.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.021 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.021 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:18.022 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826214+0000", 2024-09-16T20:49:18.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3269000000000006e-05 2024-09-16T20:49:18.022 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:18.022 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:18.022 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:18.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.022 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.022 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5072 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2347008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:18.022 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.776370+0000", 2024-09-16T20:49:18.022 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 58.153498829999997, 2024-09-16T20:49:18.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050097653999999998, 2024-09-16T20:49:18.022 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:18.022 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:18.023 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:18.023 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:18.023 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:18.023 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:18.023 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5072 2024-09-16T20:49:18.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.023 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:18.023 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.023 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:18.023 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776370+0000", 2024-09-16T20:49:18.023 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:18.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.023 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.023 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:18.023 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776370+0000", 2024-09-16T20:49:18.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:18.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.024 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:18.024 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776645+0000", 2024-09-16T20:49:18.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000274634 2024-09-16T20:49:18.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.024 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:18.024 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776654+0000", 2024-09-16T20:49:18.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9390000000000003e-06 2024-09-16T20:49:18.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.024 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:18.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776665+0000", 2024-09-16T20:49:18.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0524e-05 2024-09-16T20:49:18.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:18.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776671+0000", 2024-09-16T20:49:18.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7390000000000002e-06 2024-09-16T20:49:18.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:18.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826261+0000", 2024-09-16T20:49:18.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049589995999999997 2024-09-16T20:49:18.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.026 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:18.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826363+0000", 2024-09-16T20:49:18.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010158999999999999 2024-09-16T20:49:18.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.026 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:18.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826468+0000", 2024-09-16T20:49:18.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000105232 2024-09-16T20:49:18.026 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:18.026 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:18.026 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:18.026 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:18.026 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:18.027 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:18.028 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:49:18.028 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:18.027+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:18.128 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:18.127+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:18.180 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:18.181 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:49:18.181 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:49:18.181 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:18.190 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:49:18.207 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #22, image image_client.0-clone22 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent23 2024-09-16T20:49:18.229 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:18.228+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:18.310 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:18.310 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:49:18.310 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:49:18.310 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:49:18.310 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:18.320 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:49:18.363 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:18.363+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:18.431 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:18.430+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:18.437 INFO:teuthology.orchestra.run.smithi139.stdout:3009 read 0xe4aa00 thru 0xe56ccc (0xc2cd bytes) 2024-09-16T20:49:18.437 INFO:teuthology.orchestra.run.smithi139.stdout:3010 punch from 0x9a24600 to 0x9a2c000, (0x7a00 bytes) 2024-09-16T20:49:18.438 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:18.439 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:49:18.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:49:18.439 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:49:18.439 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.439 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5286.0:10643 3.17 3.f6273fb7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-16T20:49:18.439 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:49:18.439 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 9.6464348490000003, 2024-09-16T20:49:18.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.76672347200000002, 2024-09-16T20:49:18.439 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:18.439 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:18.439 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:49:18.439 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:18.439 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5286", 2024-09-16T20:49:18.440 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/4135433989", 2024-09-16T20:49:18.440 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 10643 2024-09-16T20:49:18.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.440 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:18.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.440 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:18.440 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:49:18.440 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:18.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.440 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:18.440 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:49:18.440 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:18.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:18.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790489+0000", 2024-09-16T20:49:18.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5910000000000001e-06 2024-09-16T20:49:18.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:18.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790493+0000", 2024-09-16T20:49:18.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2669999999999998e-06 2024-09-16T20:49:18.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:18.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790500+0000", 2024-09-16T20:49:18.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3780000000000003e-06 2024-09-16T20:49:18.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:18.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790505+0000", 2024-09-16T20:49:18.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1560000000000003e-06 2024-09-16T20:49:18.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:18.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790559+0000", 2024-09-16T20:49:18.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3494e-05 2024-09-16T20:49:18.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:49:18.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790566+0000", 2024-09-16T20:49:18.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7649999999999993e-06 2024-09-16T20:49:18.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.443 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:18.443 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:09.557198+0000", 2024-09-16T20:49:18.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.76663136200000004 2024-09-16T20:49:18.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.443 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:18.443 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:09.557211+0000", 2024-09-16T20:49:18.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3458999999999999e-05 2024-09-16T20:49:18.443 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:18.443 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:18.443 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:18.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.444 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3863 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1232896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:18.444 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896432+0000", 2024-09-16T20:49:18.444 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.540490165, 2024-09-16T20:49:18.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046223693000000003, 2024-09-16T20:49:18.444 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:18.444 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:18.444 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:18.444 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:18.444 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:18.444 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:18.444 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3863 2024-09-16T20:49:18.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.445 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:18.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:18.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896432+0000", 2024-09-16T20:49:18.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:18.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:18.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896432+0000", 2024-09-16T20:49:18.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:18.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:18.446 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896433+0000", 2024-09-16T20:49:18.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.89e-07 2024-09-16T20:49:18.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.446 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:18.446 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896435+0000", 2024-09-16T20:49:18.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.928e-06 2024-09-16T20:49:18.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.446 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:18.446 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896441+0000", 2024-09-16T20:49:18.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.896e-06 2024-09-16T20:49:18.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.446 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:18.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896445+0000", 2024-09-16T20:49:18.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7550000000000002e-06 2024-09-16T20:49:18.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:18.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.942468+0000", 2024-09-16T20:49:18.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046022278 2024-09-16T20:49:18.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:18.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.942553+0000", 2024-09-16T20:49:18.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5807e-05 2024-09-16T20:49:18.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:18.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.942656+0000", 2024-09-16T20:49:18.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010264000000000001 2024-09-16T20:49:18.448 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:18.448 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:18.448 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:18.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.448 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3864 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1236992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:18.448 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896451+0000", 2024-09-16T20:49:18.448 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.5404712950000001, 2024-09-16T20:49:18.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046415605999999998, 2024-09-16T20:49:18.448 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:18.449 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:18.449 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:18.449 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:18.449 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:18.449 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:18.449 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3864 2024-09-16T20:49:18.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.449 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:18.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.449 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:18.449 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896451+0000", 2024-09-16T20:49:18.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:18.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.450 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:18.450 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896451+0000", 2024-09-16T20:49:18.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:18.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.450 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:18.450 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896452+0000", 2024-09-16T20:49:18.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1699999999999999e-07 2024-09-16T20:49:18.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.450 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:18.450 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896453+0000", 2024-09-16T20:49:18.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6950000000000001e-06 2024-09-16T20:49:18.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.451 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:18.451 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896459+0000", 2024-09-16T20:49:18.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6749999999999999e-06 2024-09-16T20:49:18.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.451 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:18.451 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896464+0000", 2024-09-16T20:49:18.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1399999999999999e-06 2024-09-16T20:49:18.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.451 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:18.451 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.942695+0000", 2024-09-16T20:49:18.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046230965999999998 2024-09-16T20:49:18.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:18.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.942771+0000", 2024-09-16T20:49:18.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5859999999999995e-05 2024-09-16T20:49:18.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:18.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.942867+0000", 2024-09-16T20:49:18.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5952999999999994e-05 2024-09-16T20:49:18.452 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:18.452 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:18.452 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:18.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.453 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3866 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1241088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:18.453 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896471+0000", 2024-09-16T20:49:18.453 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.5404520650000002, 2024-09-16T20:49:18.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046590830999999999, 2024-09-16T20:49:18.453 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:18.453 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:18.453 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:18.453 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:18.453 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:18.453 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:18.453 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3866 2024-09-16T20:49:18.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.454 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:18.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:18.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896471+0000", 2024-09-16T20:49:18.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:18.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:18.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896471+0000", 2024-09-16T20:49:18.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:18.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:18.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896471+0000", 2024-09-16T20:49:18.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0900000000000002e-07 2024-09-16T20:49:18.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:18.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896473+0000", 2024-09-16T20:49:18.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8899999999999999e-06 2024-09-16T20:49:18.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:18.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896479+0000", 2024-09-16T20:49:18.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0179999999999998e-06 2024-09-16T20:49:18.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.456 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:18.456 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896486+0000", 2024-09-16T20:49:18.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7240000000000003e-06 2024-09-16T20:49:18.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.456 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:18.456 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.942924+0000", 2024-09-16T20:49:18.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046438266999999998 2024-09-16T20:49:18.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.456 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:18.456 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943001+0000", 2024-09-16T20:49:18.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6987e-05 2024-09-16T20:49:18.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.457 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:18.457 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943061+0000", 2024-09-16T20:49:18.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0535999999999998e-05 2024-09-16T20:49:18.457 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:18.457 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:18.457 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:18.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.457 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3868 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1245184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:18.457 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896492+0000", 2024-09-16T20:49:18.457 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.5404305140000001, 2024-09-16T20:49:18.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046738590000000003, 2024-09-16T20:49:18.458 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:18.458 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:18.458 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:18.458 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:18.458 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:18.458 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:18.458 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3868 2024-09-16T20:49:18.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.458 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:18.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:18.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896492+0000", 2024-09-16T20:49:18.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:18.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:18.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896492+0000", 2024-09-16T20:49:18.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:18.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:18.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896492+0000", 2024-09-16T20:49:18.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9499999999999998e-07 2024-09-16T20:49:18.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:18.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896494+0000", 2024-09-16T20:49:18.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.973e-06 2024-09-16T20:49:18.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:18.460 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896501+0000", 2024-09-16T20:49:18.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7719999999999997e-06 2024-09-16T20:49:18.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:18.460 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896506+0000", 2024-09-16T20:49:18.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2089999999999996e-06 2024-09-16T20:49:18.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:18.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943093+0000", 2024-09-16T20:49:18.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046586799999999998 2024-09-16T20:49:18.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:18.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943148+0000", 2024-09-16T20:49:18.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4385999999999997e-05 2024-09-16T20:49:18.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:18.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943231+0000", 2024-09-16T20:49:18.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3054999999999997e-05 2024-09-16T20:49:18.461 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:18.462 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:18.462 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:18.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.462 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3870 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1249280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:18.462 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896514+0000", 2024-09-16T20:49:18.462 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.5404088869999999, 2024-09-16T20:49:18.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046893815999999998, 2024-09-16T20:49:18.462 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:18.462 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:18.463 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:18.463 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:18.463 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:18.463 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:18.463 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3870 2024-09-16T20:49:18.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.463 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:18.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.463 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:18.463 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896514+0000", 2024-09-16T20:49:18.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:18.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.463 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:18.464 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896514+0000", 2024-09-16T20:49:18.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:18.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.464 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:18.464 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896514+0000", 2024-09-16T20:49:18.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9299999999999999e-07 2024-09-16T20:49:18.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.464 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:18.464 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896521+0000", 2024-09-16T20:49:18.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8199999999999999e-06 2024-09-16T20:49:18.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.465 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:18.465 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896529+0000", 2024-09-16T20:49:18.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0120000000000005e-06 2024-09-16T20:49:18.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.465 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:18.465 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896534+0000", 2024-09-16T20:49:18.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5779999999999998e-06 2024-09-16T20:49:18.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.465 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:18.465 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943286+0000", 2024-09-16T20:49:18.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046751514000000001 2024-09-16T20:49:18.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:18.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943343+0000", 2024-09-16T20:49:18.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7216000000000002e-05 2024-09-16T20:49:18.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:18.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943407+0000", 2024-09-16T20:49:18.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4282999999999994e-05 2024-09-16T20:49:18.466 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:18.466 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:18.466 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:18.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.467 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3872 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1253376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:18.467 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896561+0000", 2024-09-16T20:49:18.467 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.5403614929999998, 2024-09-16T20:49:18.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.04705165, 2024-09-16T20:49:18.467 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:18.467 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:18.467 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:18.467 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:18.467 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:18.467 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:18.467 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3872 2024-09-16T20:49:18.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.467 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:18.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:18.468 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896561+0000", 2024-09-16T20:49:18.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:18.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:18.468 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896561+0000", 2024-09-16T20:49:18.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:18.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:18.468 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896561+0000", 2024-09-16T20:49:18.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4999999999999998e-07 2024-09-16T20:49:18.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.469 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:18.469 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896563+0000", 2024-09-16T20:49:18.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0140000000000001e-06 2024-09-16T20:49:18.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.469 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:18.469 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896569+0000", 2024-09-16T20:49:18.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6450000000000002e-06 2024-09-16T20:49:18.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.469 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:18.469 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896572+0000", 2024-09-16T20:49:18.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.174e-06 2024-09-16T20:49:18.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:18.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943456+0000", 2024-09-16T20:49:18.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046883552000000002 2024-09-16T20:49:18.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:18.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943531+0000", 2024-09-16T20:49:18.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5080000000000006e-05 2024-09-16T20:49:18.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:18.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943613+0000", 2024-09-16T20:49:18.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1835000000000005e-05 2024-09-16T20:49:18.471 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:18.471 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:18.471 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:18.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.471 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3874 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1257472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:18.471 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896577+0000", 2024-09-16T20:49:18.471 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.5403459779999999, 2024-09-16T20:49:18.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047187944000000002, 2024-09-16T20:49:18.471 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:18.471 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:18.471 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:18.472 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:18.472 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:18.472 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:18.472 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3874 2024-09-16T20:49:18.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.472 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:18.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:18.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896577+0000", 2024-09-16T20:49:18.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:18.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:18.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896577+0000", 2024-09-16T20:49:18.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:18.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:18.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896577+0000", 2024-09-16T20:49:18.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1199999999999999e-07 2024-09-16T20:49:18.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:18.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896578+0000", 2024-09-16T20:49:18.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.237e-06 2024-09-16T20:49:18.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:18.474 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896583+0000", 2024-09-16T20:49:18.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5859999999999998e-06 2024-09-16T20:49:18.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:18.474 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896587+0000", 2024-09-16T20:49:18.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3100000000000002e-06 2024-09-16T20:49:18.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:18.474 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943661+0000", 2024-09-16T20:49:18.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047073639 2024-09-16T20:49:18.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:18.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943712+0000", 2024-09-16T20:49:18.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1190999999999998e-05 2024-09-16T20:49:18.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:18.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943765+0000", 2024-09-16T20:49:18.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2769000000000002e-05 2024-09-16T20:49:18.475 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:18.475 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:18.475 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:18.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.476 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3876 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1261568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:18.476 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896593+0000", 2024-09-16T20:49:18.476 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.5403294000000001, 2024-09-16T20:49:18.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047362866000000003, 2024-09-16T20:49:18.476 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:18.476 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:18.476 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:18.476 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:18.476 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:18.476 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:18.476 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3876 2024-09-16T20:49:18.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.476 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:18.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:18.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896593+0000", 2024-09-16T20:49:18.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:18.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:18.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896593+0000", 2024-09-16T20:49:18.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:18.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:18.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896594+0000", 2024-09-16T20:49:18.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1600000000000002e-07 2024-09-16T20:49:18.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:18.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896595+0000", 2024-09-16T20:49:18.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.756e-06 2024-09-16T20:49:18.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:18.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896601+0000", 2024-09-16T20:49:18.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9309999999999996e-06 2024-09-16T20:49:18.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:18.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896606+0000", 2024-09-16T20:49:18.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1089999999999997e-06 2024-09-16T20:49:18.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:18.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943790+0000", 2024-09-16T20:49:18.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047183414 2024-09-16T20:49:18.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:18.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943869+0000", 2024-09-16T20:49:18.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9122000000000006e-05 2024-09-16T20:49:18.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:18.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943956+0000", 2024-09-16T20:49:18.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7118000000000002e-05 2024-09-16T20:49:18.480 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:18.480 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:18.480 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:18.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.480 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3879 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1265664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:18.480 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896613+0000", 2024-09-16T20:49:18.480 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.5403097580000003, 2024-09-16T20:49:18.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047528204999999997, 2024-09-16T20:49:18.480 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:18.480 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:18.480 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:18.480 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:18.481 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:18.481 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:18.481 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3879 2024-09-16T20:49:18.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.481 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:18.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.481 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:18.481 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896613+0000", 2024-09-16T20:49:18.481 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:18.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.481 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:18.481 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896613+0000", 2024-09-16T20:49:18.481 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:18.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.482 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:18.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896613+0000", 2024-09-16T20:49:18.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.84e-07 2024-09-16T20:49:18.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.482 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:18.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896615+0000", 2024-09-16T20:49:18.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9970000000000001e-06 2024-09-16T20:49:18.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.482 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:18.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896622+0000", 2024-09-16T20:49:18.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3500000000000002e-06 2024-09-16T20:49:18.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:18.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896627+0000", 2024-09-16T20:49:18.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2490000000000001e-06 2024-09-16T20:49:18.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:18.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943996+0000", 2024-09-16T20:49:18.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047369175999999999 2024-09-16T20:49:18.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:18.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944082+0000", 2024-09-16T20:49:18.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5859000000000006e-05 2024-09-16T20:49:18.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:18.484 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944141+0000", 2024-09-16T20:49:18.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9190000000000001e-05 2024-09-16T20:49:18.484 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:18.484 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:18.484 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:18.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.484 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3881 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1269760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:18.484 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896634+0000", 2024-09-16T20:49:18.484 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.5402888450000001, 2024-09-16T20:49:18.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047692834000000003, 2024-09-16T20:49:18.485 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:18.485 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:18.485 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:18.485 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:18.485 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:18.485 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:18.485 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3881 2024-09-16T20:49:18.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.485 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:18.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.486 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:18.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896634+0000", 2024-09-16T20:49:18.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:18.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.486 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:18.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896634+0000", 2024-09-16T20:49:18.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:18.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:18.487 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896634+0000", 2024-09-16T20:49:18.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-16T20:49:18.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:18.487 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896636+0000", 2024-09-16T20:49:18.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.216e-06 2024-09-16T20:49:18.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:18.487 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896640+0000", 2024-09-16T20:49:18.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1749999999999998e-06 2024-09-16T20:49:18.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.488 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:18.488 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896643+0000", 2024-09-16T20:49:18.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8779999999999998e-06 2024-09-16T20:49:18.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.488 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:18.488 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944178+0000", 2024-09-16T20:49:18.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047534633 2024-09-16T20:49:18.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.488 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:18.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944231+0000", 2024-09-16T20:49:18.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3338999999999999e-05 2024-09-16T20:49:18.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:18.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944327+0000", 2024-09-16T20:49:18.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5223000000000006e-05 2024-09-16T20:49:18.489 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:18.489 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:18.489 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:18.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.489 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3882 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1273856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:18.489 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896647+0000", 2024-09-16T20:49:18.490 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.540275684, 2024-09-16T20:49:18.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047858886000000003, 2024-09-16T20:49:18.490 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:18.490 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:18.490 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:18.490 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:18.490 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:18.490 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:18.490 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3882 2024-09-16T20:49:18.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.490 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:18.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:18.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896647+0000", 2024-09-16T20:49:18.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:18.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:18.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896647+0000", 2024-09-16T20:49:18.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:18.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:18.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896647+0000", 2024-09-16T20:49:18.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4400000000000001e-07 2024-09-16T20:49:18.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:18.492 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896648+0000", 2024-09-16T20:49:18.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0499999999999999e-06 2024-09-16T20:49:18.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.492 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:18.492 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896651+0000", 2024-09-16T20:49:18.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9809999999999999e-06 2024-09-16T20:49:18.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.492 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:18.492 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896654+0000", 2024-09-16T20:49:18.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4439999999999998e-06 2024-09-16T20:49:18.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.493 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:18.493 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944361+0000", 2024-09-16T20:49:18.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047707408 2024-09-16T20:49:18.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.493 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:18.493 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944405+0000", 2024-09-16T20:49:18.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4107000000000003e-05 2024-09-16T20:49:18.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.494 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:18.494 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944506+0000", 2024-09-16T20:49:18.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000100652 2024-09-16T20:49:18.494 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:18.494 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:18.494 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:18.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.494 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3883 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1277952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:18.494 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896656+0000", 2024-09-16T20:49:18.494 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.5402661740000001, 2024-09-16T20:49:18.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047975265000000003, 2024-09-16T20:49:18.494 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:18.495 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:18.495 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:18.495 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:18.495 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:18.495 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:18.495 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3883 2024-09-16T20:49:18.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.495 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:18.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:18.495 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896656+0000", 2024-09-16T20:49:18.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:18.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:18.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896656+0000", 2024-09-16T20:49:18.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:18.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:18.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896657+0000", 2024-09-16T20:49:18.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7000000000000001e-07 2024-09-16T20:49:18.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:18.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896658+0000", 2024-09-16T20:49:18.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5850000000000001e-06 2024-09-16T20:49:18.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:18.497 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896664+0000", 2024-09-16T20:49:18.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.609e-06 2024-09-16T20:49:18.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:18.497 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896668+0000", 2024-09-16T20:49:18.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.121e-06 2024-09-16T20:49:18.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:18.497 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944542+0000", 2024-09-16T20:49:18.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047873838000000002 2024-09-16T20:49:18.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.498 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:18.498 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944579+0000", 2024-09-16T20:49:18.498 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6943000000000002e-05 2024-09-16T20:49:18.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.498 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:18.498 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944632+0000", 2024-09-16T20:49:18.498 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2899000000000003e-05 2024-09-16T20:49:18.498 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:18.498 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:18.498 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:18.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.499 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.499 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3886 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1282048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:18.499 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896674+0000", 2024-09-16T20:49:18.499 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.5402482050000001, 2024-09-16T20:49:18.499 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048152421000000001, 2024-09-16T20:49:18.499 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:18.499 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:18.499 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:18.499 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:18.499 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:18.499 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:18.499 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3886 2024-09-16T20:49:18.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.499 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:18.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.500 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:18.500 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896674+0000", 2024-09-16T20:49:18.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:18.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.500 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:18.500 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896674+0000", 2024-09-16T20:49:18.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:18.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.500 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:18.500 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896675+0000", 2024-09-16T20:49:18.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6100000000000002e-07 2024-09-16T20:49:18.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:18.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896677+0000", 2024-09-16T20:49:18.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1270000000000001e-06 2024-09-16T20:49:18.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:18.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896683+0000", 2024-09-16T20:49:18.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1970000000000001e-06 2024-09-16T20:49:18.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:18.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896688+0000", 2024-09-16T20:49:18.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.605e-06 2024-09-16T20:49:18.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.502 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:18.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944676+0000", 2024-09-16T20:49:18.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047988391999999998 2024-09-16T20:49:18.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.502 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:18.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944763+0000", 2024-09-16T20:49:18.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7363999999999999e-05 2024-09-16T20:49:18.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:18.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944827+0000", 2024-09-16T20:49:18.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3374999999999995e-05 2024-09-16T20:49:18.503 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:18.503 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:18.503 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:18.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.503 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3888 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1286144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:18.503 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896693+0000", 2024-09-16T20:49:18.503 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.5402290810000001, 2024-09-16T20:49:18.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048336443999999999, 2024-09-16T20:49:18.503 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:18.503 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:18.504 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:18.504 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:18.504 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:18.504 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:18.504 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3888 2024-09-16T20:49:18.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.504 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:18.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.504 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:18.504 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896693+0000", 2024-09-16T20:49:18.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:18.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:18.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896693+0000", 2024-09-16T20:49:18.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:18.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:18.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896694+0000", 2024-09-16T20:49:18.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3200000000000001e-07 2024-09-16T20:49:18.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:18.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896696+0000", 2024-09-16T20:49:18.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8989999999999999e-06 2024-09-16T20:49:18.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:18.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896702+0000", 2024-09-16T20:49:18.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8819999999999998e-06 2024-09-16T20:49:18.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:18.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896707+0000", 2024-09-16T20:49:18.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1529999999999996e-06 2024-09-16T20:49:18.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:18.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944863+0000", 2024-09-16T20:49:18.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048156658999999997 2024-09-16T20:49:18.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.507 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.507 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:18.507 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944944+0000", 2024-09-16T20:49:18.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0090000000000001e-05 2024-09-16T20:49:18.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.507 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.507 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:18.507 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945030+0000", 2024-09-16T20:49:18.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6428999999999995e-05 2024-09-16T20:49:18.507 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:18.507 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:18.507 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:18.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.507 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.508 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3890 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1290240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:18.508 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896713+0000", 2024-09-16T20:49:18.508 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.5402092110000001, 2024-09-16T20:49:18.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048538382999999997, 2024-09-16T20:49:18.508 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:18.508 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:18.508 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:18.508 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:18.508 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:18.508 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:18.508 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3890 2024-09-16T20:49:18.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.508 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:18.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:18.509 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896713+0000", 2024-09-16T20:49:18.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:18.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:18.509 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896713+0000", 2024-09-16T20:49:18.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:18.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:18.509 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896714+0000", 2024-09-16T20:49:18.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2000000000000001e-07 2024-09-16T20:49:18.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:18.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896716+0000", 2024-09-16T20:49:18.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0109999999999999e-06 2024-09-16T20:49:18.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:18.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896720+0000", 2024-09-16T20:49:18.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9049999999999999e-06 2024-09-16T20:49:18.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:18.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896722+0000", 2024-09-16T20:49:18.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.627e-06 2024-09-16T20:49:18.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:18.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945075+0000", 2024-09-16T20:49:18.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048353149999999998 2024-09-16T20:49:18.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:18.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945157+0000", 2024-09-16T20:49:18.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1972999999999996e-05 2024-09-16T20:49:18.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:18.512 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945252+0000", 2024-09-16T20:49:18.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4396999999999998e-05 2024-09-16T20:49:18.512 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:18.512 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:18.512 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:18.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.512 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.512 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3893 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1294336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:18.512 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896726+0000", 2024-09-16T20:49:18.512 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.5401963539999999, 2024-09-16T20:49:18.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048658686999999999, 2024-09-16T20:49:18.512 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:18.512 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:18.512 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:18.513 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:18.513 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:18.513 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:18.513 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3893 2024-09-16T20:49:18.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.513 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:18.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.513 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:18.513 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896726+0000", 2024-09-16T20:49:18.513 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:18.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.513 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:18.513 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896726+0000", 2024-09-16T20:49:18.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:18.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:18.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896727+0000", 2024-09-16T20:49:18.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7899999999999999e-07 2024-09-16T20:49:18.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:18.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896728+0000", 2024-09-16T20:49:18.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.68e-06 2024-09-16T20:49:18.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.515 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:18.515 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896734+0000", 2024-09-16T20:49:18.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9190000000000004e-06 2024-09-16T20:49:18.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.515 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.515 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:18.515 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896739+0000", 2024-09-16T20:49:18.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5549999999999996e-06 2024-09-16T20:49:18.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.515 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.515 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:18.515 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945271+0000", 2024-09-16T20:49:18.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048531795000000003 2024-09-16T20:49:18.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.515 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.516 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:18.516 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945330+0000", 2024-09-16T20:49:18.516 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9386999999999999e-05 2024-09-16T20:49:18.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.516 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.516 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:18.516 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945385+0000", 2024-09-16T20:49:18.516 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4871999999999999e-05 2024-09-16T20:49:18.516 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:18.516 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:18.516 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:18.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.516 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.516 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3895 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1298432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:18.517 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896754+0000", 2024-09-16T20:49:18.517 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.5401682800000001, 2024-09-16T20:49:18.517 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048754440000000003, 2024-09-16T20:49:18.517 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:18.517 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:18.517 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:18.517 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:18.517 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:18.517 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:18.517 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3895 2024-09-16T20:49:18.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.517 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:18.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.517 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:18.518 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896754+0000", 2024-09-16T20:49:18.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:18.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.518 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:18.518 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896754+0000", 2024-09-16T20:49:18.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:18.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.518 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:18.518 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896755+0000", 2024-09-16T20:49:18.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5200000000000002e-07 2024-09-16T20:49:18.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.518 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:18.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896762+0000", 2024-09-16T20:49:18.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8430000000000004e-06 2024-09-16T20:49:18.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.519 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:18.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896782+0000", 2024-09-16T20:49:18.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0693e-05 2024-09-16T20:49:18.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.519 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:18.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896793+0000", 2024-09-16T20:49:18.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0294e-05 2024-09-16T20:49:18.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.520 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:18.520 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945418+0000", 2024-09-16T20:49:18.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048625738000000002 2024-09-16T20:49:18.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.520 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:18.520 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945457+0000", 2024-09-16T20:49:18.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9103999999999997e-05 2024-09-16T20:49:18.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.520 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:18.520 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945509+0000", 2024-09-16T20:49:18.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1316000000000001e-05 2024-09-16T20:49:18.520 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:18.521 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:18.521 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:18.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.521 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.521 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3896 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1302528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:18.521 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896806+0000", 2024-09-16T20:49:18.521 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.5401160680000001, 2024-09-16T20:49:18.521 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048831159999999998, 2024-09-16T20:49:18.521 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:18.521 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:18.521 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:18.521 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:18.521 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:18.521 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:18.521 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3896 2024-09-16T20:49:18.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.522 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:18.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.522 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:18.522 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896806+0000", 2024-09-16T20:49:18.522 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:18.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.522 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:18.522 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896806+0000", 2024-09-16T20:49:18.522 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:18.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:18.523 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896807+0000", 2024-09-16T20:49:18.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3e-07 2024-09-16T20:49:18.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:18.523 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896811+0000", 2024-09-16T20:49:18.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8310000000000003e-06 2024-09-16T20:49:18.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:18.523 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896819+0000", 2024-09-16T20:49:18.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2470000000000008e-06 2024-09-16T20:49:18.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.524 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:18.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896823+0000", 2024-09-16T20:49:18.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7919999999999999e-06 2024-09-16T20:49:18.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.524 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:18.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945535+0000", 2024-09-16T20:49:18.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048712137000000003 2024-09-16T20:49:18.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.524 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:18.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945571+0000", 2024-09-16T20:49:18.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6261999999999999e-05 2024-09-16T20:49:18.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.525 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.525 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:18.525 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945638+0000", 2024-09-16T20:49:18.525 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6161000000000006e-05 2024-09-16T20:49:18.525 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:18.525 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:18.525 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:18.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.525 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.525 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3899 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1306624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:18.525 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896829+0000", 2024-09-16T20:49:18.525 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.5400938669999999, 2024-09-16T20:49:18.525 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049014110999999999, 2024-09-16T20:49:18.526 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:18.526 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:18.526 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:18.526 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:18.526 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:18.526 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:18.526 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3899 2024-09-16T20:49:18.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.526 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:18.526 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.526 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:18.526 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896829+0000", 2024-09-16T20:49:18.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:18.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.527 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:18.527 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896829+0000", 2024-09-16T20:49:18.527 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:18.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.527 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:18.527 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896829+0000", 2024-09-16T20:49:18.527 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8999999999999998e-07 2024-09-16T20:49:18.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.527 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:18.527 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896831+0000", 2024-09-16T20:49:18.527 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.77e-06 2024-09-16T20:49:18.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:18.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896834+0000", 2024-09-16T20:49:18.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4800000000000001e-06 2024-09-16T20:49:18.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:18.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896837+0000", 2024-09-16T20:49:18.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8559999999999998e-06 2024-09-16T20:49:18.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:18.529 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945663+0000", 2024-09-16T20:49:18.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048825529999999999 2024-09-16T20:49:18.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.529 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:18.529 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945746+0000", 2024-09-16T20:49:18.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3621000000000005e-05 2024-09-16T20:49:18.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:18.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:18.529 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:18.529 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945843+0000", 2024-09-16T20:49:18.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6564000000000006e-05 2024-09-16T20:49:18.529 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:18.529 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:18.529 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:18.530 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:18.530 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:18.530 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:18.532 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:49:18.533 INFO:teuthology.orchestra.run.smithi139.stdout:3012 read 0x7040a00 thru 0x704232c (0x192d bytes) 2024-09-16T20:49:18.533 INFO:teuthology.orchestra.run.smithi139.stdout:3013 read 0x6513800 thru 0x651f391 (0xbb92 bytes) 2024-09-16T20:49:18.533 INFO:teuthology.orchestra.run.smithi139.stdout:3014 write 0x909b400 thru 0x90a7867 (0xc468 bytes) 2024-09-16T20:49:18.533 INFO:teuthology.orchestra.run.smithi139.stdout:3015 write 0x721f600 thru 0x722292e (0x332f bytes) 2024-09-16T20:49:18.533 INFO:teuthology.orchestra.run.smithi139.stdout:3016 write 0xcd4ba00 thru 0xcd51247 (0x5848 bytes) 2024-09-16T20:49:18.533 INFO:teuthology.orchestra.run.smithi139.stdout:3017 read 0xb3e3e00 thru 0xb3e6761 (0x2962 bytes) 2024-09-16T20:49:18.533 INFO:teuthology.orchestra.run.smithi139.stdout:3019 punch from 0xb055200 to 0xb056a00, (0x1800 bytes) 2024-09-16T20:49:18.533 INFO:teuthology.orchestra.run.smithi139.stdout:3020 write 0xbae200 thru 0xbb1b5d (0x395e bytes) 2024-09-16T20:49:18.533 INFO:teuthology.orchestra.run.smithi139.stdout:3022 trunc from 0xe1d1600 to 0x6307e00 2024-09-16T20:49:18.533 INFO:teuthology.orchestra.run.smithi139.stdout:3023 read 0x14a9800 thru 0x14afa3d (0x623e bytes) 2024-09-16T20:49:18.533 INFO:teuthology.orchestra.run.smithi139.stdout:3024 read 0x1db5c00 thru 0x1dc08b3 (0xacb4 bytes) 2024-09-16T20:49:18.533 INFO:teuthology.orchestra.run.smithi139.stdout:3025 write 0x7ce4c00 thru 0x7cec413 (0x7814 bytes) 2024-09-16T20:49:18.534 INFO:teuthology.orchestra.run.smithi139.stdout:3027 write 0x529e200 thru 0x52a78ae (0x96af bytes) 2024-09-16T20:49:18.534 INFO:teuthology.orchestra.run.smithi139.stdout:3028 punch from 0x1734c00 to 0x1737000, (0x2400 bytes) 2024-09-16T20:49:18.534 INFO:teuthology.orchestra.run.smithi139.stdout:3029 read 0x2124600 thru 0x2133f07 (0xf908 bytes) 2024-09-16T20:49:18.534 INFO:teuthology.orchestra.run.smithi139.stdout:3031 trunc from 0x7cec600 to 0x94d8800 2024-09-16T20:49:18.560 INFO:teuthology.orchestra.run.smithi139.stdout:3032 write 0x97a4600 thru 0x97b37c7 (0xf1c8 bytes) 2024-09-16T20:49:18.561 INFO:teuthology.orchestra.run.smithi139.stdout:3037 read 0x656d400 thru 0x6576e25 (0x9a26 bytes) 2024-09-16T20:49:18.561 INFO:teuthology.orchestra.run.smithi139.stdout:3038 punch from 0x736ba00 to 0x736f800, (0x3e00 bytes) 2024-09-16T20:49:18.568 INFO:teuthology.orchestra.run.smithi139.stdout:3043 trunc from 0x97b3800 to 0x4d05a00 2024-09-16T20:49:18.572 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:18.572+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:18.621 INFO:teuthology.orchestra.run.smithi139.stdout:3045 write 0xd6e8600 thru 0xd6f7b74 (0xf575 bytes) 2024-09-16T20:49:18.622 INFO:teuthology.orchestra.run.smithi139.stdout:3047 punch from 0x56fb400 to 0x56fe400, (0x3000 bytes) 2024-09-16T20:49:18.627 INFO:teuthology.orchestra.run.smithi139.stdout:3050 write 0x1d46400 thru 0x1d49bef (0x37f0 bytes) 2024-09-16T20:49:18.627 INFO:teuthology.orchestra.run.smithi139.stdout:3051 read 0x35e2200 thru 0x35ed596 (0xb397 bytes) 2024-09-16T20:49:18.627 INFO:teuthology.orchestra.run.smithi139.stdout:3052 read 0x942c00 thru 0x94e36e (0xb76f bytes) 2024-09-16T20:49:18.627 INFO:teuthology.orchestra.run.smithi139.stdout:3054 punch from 0x9e7000 to 0x9f6c00, (0xfc00 bytes) 2024-09-16T20:49:18.628 INFO:teuthology.orchestra.run.smithi139.stdout:3057 read 0x36cbe00 thru 0x36d9bd7 (0xddd8 bytes) 2024-09-16T20:49:18.635 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:18.634+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:18.659 INFO:teuthology.orchestra.run.smithi139.stdout:3060 write 0xe77e200 thru 0xe77e640 (0x441 bytes) 2024-09-16T20:49:18.663 INFO:teuthology.orchestra.run.smithi139.stdout:3066 trunc from 0xe77e800 to 0xacf6600 2024-09-16T20:49:18.667 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:18.667 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:49:18.667 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:49:18.667 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:18.677 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:49:18.704 INFO:teuthology.orchestra.run.smithi139.stdout:3067 trunc from 0xacf6600 to 0x454000 2024-09-16T20:49:18.737 INFO:teuthology.orchestra.run.smithi139.stdout:3068 read 0x420600 thru 0x4295b3 (0x8fb4 bytes) 2024-09-16T20:49:18.753 INFO:teuthology.orchestra.run.smithi139.stdout:3070 write 0x4c6ba00 thru 0x4c706b5 (0x4cb6 bytes) 2024-09-16T20:49:18.758 INFO:teuthology.orchestra.run.smithi139.stdout:3071 trunc from 0x4c70800 to 0x8713e00 2024-09-16T20:49:18.761 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:18.760+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:18.780 INFO:teuthology.orchestra.run.smithi139.stdout:3076 read 0x75dfc00 thru 0x75e61aa (0x65ab bytes) 2024-09-16T20:49:18.780 INFO:teuthology.orchestra.run.smithi139.stdout:3077 read 0x406be00 thru 0x4070d35 (0x4f36 bytes) 2024-09-16T20:49:18.780 INFO:teuthology.orchestra.run.smithi139.stdout:3078 read 0x5113c00 thru 0x511b884 (0x7c85 bytes) 2024-09-16T20:49:18.780 INFO:teuthology.orchestra.run.smithi139.stdout:3080 write 0x3b02200 thru 0x3b0611d (0x3f1e bytes) 2024-09-16T20:49:18.802 INFO:teuthology.orchestra.run.smithi139.stdout:3082 write 0xb136000 thru 0xb13bb8c (0x5b8d bytes) 2024-09-16T20:49:18.809 INFO:teuthology.orchestra.run.smithi139.stdout:3088 trunc from 0xb13bc00 to 0x4e52600 2024-09-16T20:49:18.823 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T20:49:18.834 INFO:teuthology.orchestra.run.smithi139.stdout:3089 write 0x24e2600 thru 0x24f1c6b (0xf66c bytes) 2024-09-16T20:49:18.841 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:18.841 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:49:18.841 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:49:18.841 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:49:18.842 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:18.842 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:18.840+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:18.853 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:49:18.867 INFO:teuthology.orchestra.run.smithi139.stdout:3090 write 0xc570e00 thru 0xc57d014 (0xc215 bytes) 2024-09-16T20:49:18.869 INFO:teuthology.orchestra.run.smithi139.stdout:3091 trunc from 0xc57d200 to 0xe0c5e00 2024-09-16T20:49:18.894 INFO:teuthology.orchestra.run.smithi139.stdout:3092 trunc from 0xe0c5e00 to 0xb19e600 2024-09-16T20:49:18.923 INFO:teuthology.orchestra.run.smithi139.stdout:3097 trunc from 0xb19e600 to 0x5fa9000 2024-09-16T20:49:18.938 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:18.937+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:18.939 INFO:teuthology.orchestra.run.smithi139.stdout:3098 trunc from 0x5fa9000 to 0x6ecb400 2024-09-16T20:49:18.959 INFO:teuthology.orchestra.run.smithi139.stdout:3099 trunc from 0x6ecb400 to 0x50d4000 2024-09-16T20:49:18.972 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 3, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-16T20:49:18.972 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T20:49:18.973 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-16T20:49:18.973 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-16T20:49:18.976 INFO:teuthology.orchestra.run.smithi139.stdout:3100 read 0x3c6ea00 thru 0x3c71144 (0x2745 bytes) 2024-09-16T20:49:18.976 INFO:teuthology.orchestra.run.smithi139.stdout:3103 punch from 0x1daa00 to 0x1dea00, (0x4000 bytes) 2024-09-16T20:49:18.980 INFO:teuthology.orchestra.run.smithi139.stdout:3105 write 0x3f08200 thru 0x3f16b42 (0xe943 bytes) 2024-09-16T20:49:18.981 INFO:teuthology.orchestra.run.smithi139.stdout:3107 trunc from 0x50d4000 to 0x851b200 2024-09-16T20:49:19.011 INFO:teuthology.orchestra.run.smithi139.stdout:3109 clone 25 order 24 su 2097152 sc 10 2024-09-16T20:49:19.027 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:19.027 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:49:19.027 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:49:19.027 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:49:19.027 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.028 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2179 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 948224~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:19.028 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335723+0000", 2024-09-16T20:49:19.028 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 56.688863791000003, 2024-09-16T20:49:19.028 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048007181000000003, 2024-09-16T20:49:19.028 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:19.028 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:19.028 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:19.028 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:19.028 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:19.028 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:19.028 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2179 2024-09-16T20:49:19.028 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.028 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:19.029 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.029 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:19.029 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335723+0000", 2024-09-16T20:49:19.029 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.029 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.029 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.029 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:19.029 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335723+0000", 2024-09-16T20:49:19.029 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.029 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.029 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.029 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:19.029 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335724+0000", 2024-09-16T20:49:19.030 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.58e-07 2024-09-16T20:49:19.030 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.030 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.030 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:19.030 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335726+0000", 2024-09-16T20:49:19.030 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9120000000000001e-06 2024-09-16T20:49:19.030 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.030 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.030 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:19.030 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335732+0000", 2024-09-16T20:49:19.030 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6080000000000001e-06 2024-09-16T20:49:19.030 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.030 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.031 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:19.031 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335737+0000", 2024-09-16T20:49:19.031 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.22e-06 2024-09-16T20:49:19.031 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.031 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.031 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:19.031 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383556+0000", 2024-09-16T20:49:19.031 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047819002999999999 2024-09-16T20:49:19.031 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.031 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.031 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:19.031 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383635+0000", 2024-09-16T20:49:19.031 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9060999999999994e-05 2024-09-16T20:49:19.032 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.032 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.032 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:19.032 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383730+0000", 2024-09-16T20:49:19.032 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4919000000000004e-05 2024-09-16T20:49:19.032 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.032 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.032 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.032 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.032 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.032 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2180 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 956416~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:19.032 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335744+0000", 2024-09-16T20:49:19.032 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 56.688843343000002, 2024-09-16T20:49:19.033 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048255147999999998, 2024-09-16T20:49:19.033 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:19.033 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:19.033 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:19.033 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:19.033 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:19.033 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:19.033 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2180 2024-09-16T20:49:19.033 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.033 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:19.033 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.033 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:19.033 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335744+0000", 2024-09-16T20:49:19.034 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.034 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.034 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.034 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:19.034 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335744+0000", 2024-09-16T20:49:19.034 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.034 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.034 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.034 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:19.034 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335744+0000", 2024-09-16T20:49:19.034 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9000000000000002e-07 2024-09-16T20:49:19.034 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.034 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.035 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:19.035 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335746+0000", 2024-09-16T20:49:19.035 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7570000000000001e-06 2024-09-16T20:49:19.035 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.035 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.035 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:19.035 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335752+0000", 2024-09-16T20:49:19.035 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0079999999999999e-06 2024-09-16T20:49:19.035 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.035 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.035 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:19.035 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335757+0000", 2024-09-16T20:49:19.035 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1649999999999997e-06 2024-09-16T20:49:19.036 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.036 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.036 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:19.036 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383803+0000", 2024-09-16T20:49:19.036 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048046091999999999 2024-09-16T20:49:19.036 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.036 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.036 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:19.036 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383889+0000", 2024-09-16T20:49:19.036 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6048e-05 2024-09-16T20:49:19.036 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.036 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.036 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:19.037 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383999+0000", 2024-09-16T20:49:19.037 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000109688 2024-09-16T20:49:19.037 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.037 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.037 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.037 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.037 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.037 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2182 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 964608~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:19.037 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335773+0000", 2024-09-16T20:49:19.037 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 56.688813535999998, 2024-09-16T20:49:19.037 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048467116999999997, 2024-09-16T20:49:19.037 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:19.037 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:19.038 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:19.038 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:19.038 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:19.038 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:19.038 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2182 2024-09-16T20:49:19.038 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.038 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:19.038 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.038 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:19.038 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335773+0000", 2024-09-16T20:49:19.038 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.038 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.038 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.038 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:19.039 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335773+0000", 2024-09-16T20:49:19.039 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.039 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.039 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.039 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:19.039 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335774+0000", 2024-09-16T20:49:19.039 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3799999999999997e-07 2024-09-16T20:49:19.039 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.039 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.039 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:19.039 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335776+0000", 2024-09-16T20:49:19.039 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.227e-06 2024-09-16T20:49:19.039 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.040 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.040 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:19.040 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335790+0000", 2024-09-16T20:49:19.040 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4119e-05 2024-09-16T20:49:19.040 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.040 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.040 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:19.040 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335796+0000", 2024-09-16T20:49:19.040 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.694e-06 2024-09-16T20:49:19.040 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.040 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.040 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:19.040 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384058+0000", 2024-09-16T20:49:19.040 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048261983000000001 2024-09-16T20:49:19.041 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.041 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.041 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:19.041 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384148+0000", 2024-09-16T20:49:19.041 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9501999999999998e-05 2024-09-16T20:49:19.041 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.041 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.041 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:19.041 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384241+0000", 2024-09-16T20:49:19.041 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.3053999999999994e-05 2024-09-16T20:49:19.043 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.043 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.043 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.043 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.043 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.043 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2185 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 972800~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:19.043 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335804+0000", 2024-09-16T20:49:19.043 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 56.688782885999998, 2024-09-16T20:49:19.043 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048570818000000002, 2024-09-16T20:49:19.044 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:19.044 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:19.044 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:19.044 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:19.044 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:19.044 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:19.044 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2185 2024-09-16T20:49:19.044 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.044 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:19.044 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.044 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:19.045 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335804+0000", 2024-09-16T20:49:19.045 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.045 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.045 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.045 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:19.045 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335804+0000", 2024-09-16T20:49:19.045 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.045 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.045 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.045 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:19.045 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335805+0000", 2024-09-16T20:49:19.045 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8299999999999997e-07 2024-09-16T20:49:19.045 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.046 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.046 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:19.046 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335813+0000", 2024-09-16T20:49:19.046 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6309999999999992e-06 2024-09-16T20:49:19.046 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.046 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.046 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:19.046 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335821+0000", 2024-09-16T20:49:19.046 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0169999999999996e-06 2024-09-16T20:49:19.046 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.046 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.047 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:19.047 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335827+0000", 2024-09-16T20:49:19.047 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0100000000000001e-06 2024-09-16T20:49:19.047 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.047 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.047 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:19.047 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384273+0000", 2024-09-16T20:49:19.047 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048445333 2024-09-16T20:49:19.047 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.047 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.047 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:19.047 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384315+0000", 2024-09-16T20:49:19.047 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2472999999999998e-05 2024-09-16T20:49:19.048 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.048 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.048 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:19.048 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384375+0000", 2024-09-16T20:49:19.048 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9871000000000003e-05 2024-09-16T20:49:19.048 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.048 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.048 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.048 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.048 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.048 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2187 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 980992~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:19.048 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335835+0000", 2024-09-16T20:49:19.049 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 56.688751809000003, 2024-09-16T20:49:19.049 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048684927000000003, 2024-09-16T20:49:19.049 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:19.049 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:19.049 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:19.049 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:19.049 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:19.049 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:19.049 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2187 2024-09-16T20:49:19.049 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.049 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:19.050 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.050 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:19.050 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335835+0000", 2024-09-16T20:49:19.050 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.050 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.050 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.050 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:19.050 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335835+0000", 2024-09-16T20:49:19.050 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.050 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.050 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.050 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:19.050 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335836+0000", 2024-09-16T20:49:19.051 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3599999999999999e-07 2024-09-16T20:49:19.051 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.051 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.051 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:19.051 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335838+0000", 2024-09-16T20:49:19.051 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.996e-06 2024-09-16T20:49:19.051 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.051 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.051 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:19.051 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335844+0000", 2024-09-16T20:49:19.051 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7889999999999997e-06 2024-09-16T20:49:19.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.052 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.052 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:19.052 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335849+0000", 2024-09-16T20:49:19.052 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7400000000000004e-06 2024-09-16T20:49:19.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.052 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.052 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:19.052 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384411+0000", 2024-09-16T20:49:19.052 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048562069999999999 2024-09-16T20:49:19.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.053 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.053 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:19.053 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384460+0000", 2024-09-16T20:49:19.053 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9125999999999997e-05 2024-09-16T20:49:19.053 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.053 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.053 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:19.053 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384520+0000", 2024-09-16T20:49:19.053 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9769999999999999e-05 2024-09-16T20:49:19.053 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.053 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.054 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.054 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.054 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.054 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2189 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 989184~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:19.054 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335856+0000", 2024-09-16T20:49:19.054 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 56.688730563999997, 2024-09-16T20:49:19.054 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048856525999999997, 2024-09-16T20:49:19.054 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:19.054 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:19.054 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:19.054 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:19.055 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:19.055 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:19.055 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2189 2024-09-16T20:49:19.055 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.055 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:19.055 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.055 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:19.055 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335856+0000", 2024-09-16T20:49:19.055 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.055 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.056 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.056 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:19.056 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335856+0000", 2024-09-16T20:49:19.056 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.056 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.056 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.056 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:19.056 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335857+0000", 2024-09-16T20:49:19.056 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6800000000000001e-07 2024-09-16T20:49:19.056 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.056 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.056 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:19.056 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335859+0000", 2024-09-16T20:49:19.057 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.013e-06 2024-09-16T20:49:19.057 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.057 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.057 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:19.057 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335866+0000", 2024-09-16T20:49:19.057 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8750000000000002e-06 2024-09-16T20:49:19.057 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.057 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.057 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:19.057 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335871+0000", 2024-09-16T20:49:19.057 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1719999999999998e-06 2024-09-16T20:49:19.058 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.058 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.058 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:19.058 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384550+0000", 2024-09-16T20:49:19.058 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048679476999999999 2024-09-16T20:49:19.058 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.058 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.058 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:19.058 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384611+0000", 2024-09-16T20:49:19.058 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0897000000000003e-05 2024-09-16T20:49:19.058 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.058 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.059 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:19.059 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384713+0000", 2024-09-16T20:49:19.059 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101724 2024-09-16T20:49:19.059 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.059 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.059 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.059 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.059 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.059 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2192 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 997376~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:19.059 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335878+0000", 2024-09-16T20:49:19.059 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 56.688708879000004, 2024-09-16T20:49:19.059 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049028769999999999, 2024-09-16T20:49:19.060 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:19.060 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:19.060 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:19.060 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:19.060 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:19.060 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:19.060 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2192 2024-09-16T20:49:19.060 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.060 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:19.060 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.060 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:19.061 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335878+0000", 2024-09-16T20:49:19.061 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.061 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.061 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.061 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:19.061 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335878+0000", 2024-09-16T20:49:19.061 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.061 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.061 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.062 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:19.062 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335879+0000", 2024-09-16T20:49:19.062 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6899999999999998e-07 2024-09-16T20:49:19.062 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.062 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.062 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:19.062 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335881+0000", 2024-09-16T20:49:19.062 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.996e-06 2024-09-16T20:49:19.062 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.062 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.063 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:19.063 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335887+0000", 2024-09-16T20:49:19.063 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6370000000000002e-06 2024-09-16T20:49:19.063 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.063 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.063 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:19.063 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335892+0000", 2024-09-16T20:49:19.063 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2120000000000002e-06 2024-09-16T20:49:19.063 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.063 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.063 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:19.064 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384747+0000", 2024-09-16T20:49:19.064 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048854923000000001 2024-09-16T20:49:19.064 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.065 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.065 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:19.065 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384830+0000", 2024-09-16T20:49:19.065 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2359000000000002e-05 2024-09-16T20:49:19.065 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.065 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.065 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:19.065 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384907+0000", 2024-09-16T20:49:19.065 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7174000000000003e-05 2024-09-16T20:49:19.065 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.066 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.066 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.066 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.066 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.066 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2194 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1005568~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:19.066 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335899+0000", 2024-09-16T20:49:19.066 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 56.688687958000003, 2024-09-16T20:49:19.067 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049225393999999999, 2024-09-16T20:49:19.067 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:19.067 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:19.067 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:19.067 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:19.067 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:19.067 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:19.067 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2194 2024-09-16T20:49:19.067 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.068 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:19.068 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.068 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:19.068 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335899+0000", 2024-09-16T20:49:19.068 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.068 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.068 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.068 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:19.068 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335899+0000", 2024-09-16T20:49:19.068 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.068 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.068 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.069 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:19.069 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335899+0000", 2024-09-16T20:49:19.069 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8200000000000001e-07 2024-09-16T20:49:19.069 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.069 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.069 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:19.069 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335901+0000", 2024-09-16T20:49:19.069 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9e-06 2024-09-16T20:49:19.069 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.069 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.069 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:19.069 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335908+0000", 2024-09-16T20:49:19.070 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3559999999999998e-06 2024-09-16T20:49:19.070 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.070 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.070 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:19.070 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335914+0000", 2024-09-16T20:49:19.070 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4729999999999997e-06 2024-09-16T20:49:19.070 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.070 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.070 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:19.070 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384979+0000", 2024-09-16T20:49:19.070 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049064545000000001 2024-09-16T20:49:19.070 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.070 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.070 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:19.071 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385060+0000", 2024-09-16T20:49:19.071 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0804999999999996e-05 2024-09-16T20:49:19.071 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.071 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.071 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:19.071 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385124+0000", 2024-09-16T20:49:19.071 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4932999999999996e-05 2024-09-16T20:49:19.071 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.071 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.071 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.071 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.071 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.071 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2197 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1013760~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:19.071 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335921+0000", 2024-09-16T20:49:19.071 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 56.688665677000003, 2024-09-16T20:49:19.072 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049342184999999997, 2024-09-16T20:49:19.072 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:19.072 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:19.072 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:19.072 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:19.072 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:19.072 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:19.072 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2197 2024-09-16T20:49:19.072 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.072 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:19.072 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.072 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:19.072 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335921+0000", 2024-09-16T20:49:19.072 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.073 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.073 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.073 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:19.073 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335921+0000", 2024-09-16T20:49:19.073 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.073 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.073 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.073 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:19.073 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335922+0000", 2024-09-16T20:49:19.073 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.32e-07 2024-09-16T20:49:19.073 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.073 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.073 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:19.074 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335924+0000", 2024-09-16T20:49:19.074 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.995e-06 2024-09-16T20:49:19.074 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.074 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.074 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:19.074 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335930+0000", 2024-09-16T20:49:19.074 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5139999999999998e-06 2024-09-16T20:49:19.074 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.074 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.074 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:19.074 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335936+0000", 2024-09-16T20:49:19.074 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6160000000000001e-06 2024-09-16T20:49:19.074 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.074 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.074 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:19.075 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385142+0000", 2024-09-16T20:49:19.075 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049206393000000001 2024-09-16T20:49:19.075 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.075 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.075 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:19.075 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385184+0000", 2024-09-16T20:49:19.075 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1928999999999998e-05 2024-09-16T20:49:19.075 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.075 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.075 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:19.075 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385264+0000", 2024-09-16T20:49:19.075 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9306000000000003e-05 2024-09-16T20:49:19.075 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.075 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.075 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.076 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.076 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.076 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2198 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1021952~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:19.076 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335942+0000", 2024-09-16T20:49:19.076 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 56.688644617000001, 2024-09-16T20:49:19.076 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049562483999999997, 2024-09-16T20:49:19.076 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:19.076 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:19.076 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:19.076 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:19.076 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:19.076 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:19.076 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2198 2024-09-16T20:49:19.076 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.077 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:19.077 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.077 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:19.077 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335942+0000", 2024-09-16T20:49:19.077 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.077 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.077 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.077 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:19.077 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335942+0000", 2024-09-16T20:49:19.077 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.077 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.077 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.077 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:19.077 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335943+0000", 2024-09-16T20:49:19.078 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7099999999999997e-07 2024-09-16T20:49:19.078 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.078 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.078 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:19.078 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335945+0000", 2024-09-16T20:49:19.078 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3499999999999999e-06 2024-09-16T20:49:19.078 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.078 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.078 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:19.078 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335952+0000", 2024-09-16T20:49:19.078 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4309999999999999e-06 2024-09-16T20:49:19.078 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.078 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.078 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:19.078 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335957+0000", 2024-09-16T20:49:19.079 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0869999999999997e-06 2024-09-16T20:49:19.079 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.079 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.079 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:19.079 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385312+0000", 2024-09-16T20:49:19.079 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049355121000000002 2024-09-16T20:49:19.079 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.079 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.079 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:19.079 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385405+0000", 2024-09-16T20:49:19.079 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.3344000000000006e-05 2024-09-16T20:49:19.079 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.079 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.079 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:19.080 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385505+0000", 2024-09-16T20:49:19.080 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.9779999999999997e-05 2024-09-16T20:49:19.080 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.080 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.080 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.080 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.080 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.080 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2201 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1030144~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:19.080 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335976+0000", 2024-09-16T20:49:19.080 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 56.688611174999998, 2024-09-16T20:49:19.080 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049740077000000001, 2024-09-16T20:49:19.080 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:19.081 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:19.081 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:19.081 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:19.081 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:19.081 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:19.081 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2201 2024-09-16T20:49:19.081 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.081 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:19.081 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.081 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:19.081 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335976+0000", 2024-09-16T20:49:19.081 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.081 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.081 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.081 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:19.082 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335976+0000", 2024-09-16T20:49:19.082 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.082 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.082 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.082 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:19.082 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335976+0000", 2024-09-16T20:49:19.082 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8599999999999998e-07 2024-09-16T20:49:19.082 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.082 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.082 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:19.082 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335978+0000", 2024-09-16T20:49:19.082 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1500000000000002e-06 2024-09-16T20:49:19.082 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.082 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.083 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:19.083 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335986+0000", 2024-09-16T20:49:19.083 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3989999999999998e-06 2024-09-16T20:49:19.083 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.083 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.083 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:19.083 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335992+0000", 2024-09-16T20:49:19.083 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.682e-06 2024-09-16T20:49:19.083 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.083 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.083 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:19.083 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385537+0000", 2024-09-16T20:49:19.083 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049545828 2024-09-16T20:49:19.083 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.084 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.084 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:19.084 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385619+0000", 2024-09-16T20:49:19.084 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1720999999999999e-05 2024-09-16T20:49:19.084 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.084 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.084 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:19.084 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385716+0000", 2024-09-16T20:49:19.084 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.6811000000000005e-05 2024-09-16T20:49:19.084 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.084 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.084 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.084 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.084 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.084 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2203 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1038336~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:19.085 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.336000+0000", 2024-09-16T20:49:19.085 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 56.688587388999998, 2024-09-16T20:49:19.085 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049949888999999997, 2024-09-16T20:49:19.085 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:19.085 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:19.085 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:19.085 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:19.085 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:19.085 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:19.085 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2203 2024-09-16T20:49:19.085 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.085 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:19.085 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.086 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:19.086 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336000+0000", 2024-09-16T20:49:19.086 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.086 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.086 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.086 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:19.086 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336000+0000", 2024-09-16T20:49:19.086 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.086 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.086 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.086 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:19.086 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336000+0000", 2024-09-16T20:49:19.086 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.58e-07 2024-09-16T20:49:19.086 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.086 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.087 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:19.087 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336002+0000", 2024-09-16T20:49:19.087 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0040000000000002e-06 2024-09-16T20:49:19.087 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.087 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.087 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:19.087 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336009+0000", 2024-09-16T20:49:19.087 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9890000000000003e-06 2024-09-16T20:49:19.087 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.087 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.087 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:19.087 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336014+0000", 2024-09-16T20:49:19.087 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4e-06 2024-09-16T20:49:19.087 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.088 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.088 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:19.088 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385759+0000", 2024-09-16T20:49:19.088 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049744151 2024-09-16T20:49:19.088 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.088 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.088 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:19.088 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385847+0000", 2024-09-16T20:49:19.088 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8189000000000006e-05 2024-09-16T20:49:19.088 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.088 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.088 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:19.088 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385950+0000", 2024-09-16T20:49:19.088 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102698 2024-09-16T20:49:19.088 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.089 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.089 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.089 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.089 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.089 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2206 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1046528~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:19.089 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:49:19.089 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 56.688565228000002, 2024-09-16T20:49:19.089 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050147714000000003, 2024-09-16T20:49:19.089 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:19.089 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:19.089 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:19.089 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:19.089 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:19.089 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:19.090 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2206 2024-09-16T20:49:19.090 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.090 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:19.090 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.090 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:19.090 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:49:19.090 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.090 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.090 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.090 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:19.090 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:49:19.090 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.090 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.090 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.090 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:19.090 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:49:19.091 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8599999999999999e-07 2024-09-16T20:49:19.091 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.091 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.091 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:19.091 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336024+0000", 2024-09-16T20:49:19.091 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.934e-06 2024-09-16T20:49:19.091 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.091 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.091 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:19.091 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336037+0000", 2024-09-16T20:49:19.091 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2496e-05 2024-09-16T20:49:19.091 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.091 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.091 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:19.091 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336042+0000", 2024-09-16T20:49:19.092 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4809999999999997e-06 2024-09-16T20:49:19.092 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.092 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.092 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:19.092 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385980+0000", 2024-09-16T20:49:19.092 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049938376999999999 2024-09-16T20:49:19.092 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.092 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.092 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:19.092 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.386069+0000", 2024-09-16T20:49:19.092 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8368999999999994e-05 2024-09-16T20:49:19.092 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.092 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.092 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:19.092 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.386170+0000", 2024-09-16T20:49:19.093 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000100671 2024-09-16T20:49:19.093 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.093 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.093 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.093 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.093 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.093 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5439 3.a 3:5789f5c3:::rbd_data.110ed9ca540.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 1181696~60416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:19.093 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:49:19.093 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 56.159876648000001, 2024-09-16T20:49:19.093 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10102045599999999, 2024-09-16T20:49:19.093 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:19.093 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:19.093 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:19.093 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:19.093 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:19.094 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:19.094 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5439 2024-09-16T20:49:19.094 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.094 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:19.094 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.094 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:19.094 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:49:19.094 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.094 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.094 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.094 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:19.094 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:49:19.094 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.094 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.095 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.095 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:19.095 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864711+0000", 2024-09-16T20:49:19.095 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7899999999999997e-07 2024-09-16T20:49:19.095 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.095 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.095 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:19.095 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864760+0000", 2024-09-16T20:49:19.095 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8803e-05 2024-09-16T20:49:19.095 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.095 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.095 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:19.095 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864769+0000", 2024-09-16T20:49:19.095 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0999999999999993e-06 2024-09-16T20:49:19.095 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.096 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.096 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:19.096 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864774+0000", 2024-09-16T20:49:19.096 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8919999999999999e-06 2024-09-16T20:49:19.096 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.096 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.096 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:19.096 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864815+0000", 2024-09-16T20:49:19.096 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1307000000000003e-05 2024-09-16T20:49:19.096 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.096 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.096 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:19.097 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864893+0000", 2024-09-16T20:49:19.097 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7581000000000001e-05 2024-09-16T20:49:19.097 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.097 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.097 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 1", 2024-09-16T20:49:19.097 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864996+0000", 2024-09-16T20:49:19.097 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102725 2024-09-16T20:49:19.097 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.097 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.097 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:19.097 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965442+0000", 2024-09-16T20:49:19.097 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100446012 2024-09-16T20:49:19.097 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.097 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.097 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:19.098 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965618+0000", 2024-09-16T20:49:19.098 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00017653100000000001 2024-09-16T20:49:19.098 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.098 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.098 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:19.098 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965682+0000", 2024-09-16T20:49:19.098 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3905999999999994e-05 2024-09-16T20:49:19.098 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.098 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.098 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:19.098 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965731+0000", 2024-09-16T20:49:19.098 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8819999999999997e-05 2024-09-16T20:49:19.098 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.098 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.099 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.099 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.099 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.099 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4600.0:5439 3.a e37/23 ondisk, result = 0)", 2024-09-16T20:49:19.099 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:49:19.099 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 56.158880500000002, 2024-09-16T20:49:19.099 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099742634999999996, 2024-09-16T20:49:19.099 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:19.099 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:19.099 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:19.099 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:19.099 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.099 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:19.100 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:49:19.100 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.100 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.100 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.100 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:19.100 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:49:19.100 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.100 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.100 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.100 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:19.100 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:49:19.100 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0900000000000001e-07 2024-09-16T20:49:19.100 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.100 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.100 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:19.101 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865711+0000", 2024-09-16T20:49:19.101 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3739999999999998e-06 2024-09-16T20:49:19.101 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.101 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.101 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:19.101 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965343+0000", 2024-09-16T20:49:19.101 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099632526999999999 2024-09-16T20:49:19.101 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.101 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.101 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:19.101 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965357+0000", 2024-09-16T20:49:19.101 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3713999999999999e-05 2024-09-16T20:49:19.101 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.101 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.101 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:19.102 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965426+0000", 2024-09-16T20:49:19.102 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8745999999999995e-05 2024-09-16T20:49:19.102 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.102 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.102 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:19.102 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965434+0000", 2024-09-16T20:49:19.102 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6910000000000003e-06 2024-09-16T20:49:19.102 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.102 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.102 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:19.102 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965449+0000", 2024-09-16T20:49:19.102 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4874e-05 2024-09-16T20:49:19.102 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.103 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.103 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.103 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.103 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.103 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4882.0:2508 3.12 3:4992771f:::rbd_object_map.110e7c8851c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T20:49:19.103 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:49:19.103 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.543076952, 2024-09-16T20:49:19.103 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.125007011, 2024-09-16T20:49:19.103 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:19.103 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:19.103 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:19.103 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:19.103 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4882", 2024-09-16T20:49:19.103 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2436923223", 2024-09-16T20:49:19.103 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2508 2024-09-16T20:49:19.103 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.104 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:19.104 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.104 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:19.104 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:49:19.104 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.104 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.104 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.104 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:19.104 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:49:19.104 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.104 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.104 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.104 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:19.104 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481511+0000", 2024-09-16T20:49:19.105 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0699999999999996e-07 2024-09-16T20:49:19.105 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.105 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.105 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:19.105 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481515+0000", 2024-09-16T20:49:19.105 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0609999999999997e-06 2024-09-16T20:49:19.105 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.105 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.105 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:19.105 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481523+0000", 2024-09-16T20:49:19.105 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1310000000000006e-06 2024-09-16T20:49:19.105 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.105 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.105 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:19.106 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481528+0000", 2024-09-16T20:49:19.106 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4900000000000002e-06 2024-09-16T20:49:19.106 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.106 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.106 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:19.106 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481565+0000", 2024-09-16T20:49:19.106 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7066000000000001e-05 2024-09-16T20:49:19.106 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.106 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.106 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:19.106 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481628+0000", 2024-09-16T20:49:19.106 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3118000000000001e-05 2024-09-16T20:49:19.106 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.106 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.107 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 2", 2024-09-16T20:49:19.107 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481822+0000", 2024-09-16T20:49:19.107 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019446200000000001 2024-09-16T20:49:19.107 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.107 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.107 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:19.107 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605988+0000", 2024-09-16T20:49:19.107 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.124165888 2024-09-16T20:49:19.107 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.107 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.107 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:19.107 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606427+0000", 2024-09-16T20:49:19.107 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00043854899999999999 2024-09-16T20:49:19.107 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.107 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.108 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:19.108 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606476+0000", 2024-09-16T20:49:19.108 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8971999999999998e-05 2024-09-16T20:49:19.108 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.108 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.108 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:19.108 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606517+0000", 2024-09-16T20:49:19.108 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1467000000000001e-05 2024-09-16T20:49:19.108 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.108 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.108 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.108 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.109 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.109 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4882.0:2508 3.12 e57/55 ondisk, result = 0)", 2024-09-16T20:49:19.109 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:49:19.109 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.542088452000002, 2024-09-16T20:49:19.109 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12349711400000001, 2024-09-16T20:49:19.109 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:19.109 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:19.109 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:19.109 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:19.109 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.109 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:19.109 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:49:19.109 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.109 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.110 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.110 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:19.110 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:49:19.110 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.110 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.110 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.110 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:19.110 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:49:19.110 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.06e-07 2024-09-16T20:49:19.110 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.110 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.110 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:19.110 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482502+0000", 2024-09-16T20:49:19.110 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.01e-06 2024-09-16T20:49:19.110 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.111 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.111 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:19.111 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605772+0000", 2024-09-16T20:49:19.111 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.123269687 2024-09-16T20:49:19.111 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.111 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.111 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:19.111 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605791+0000", 2024-09-16T20:49:19.111 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8912000000000001e-05 2024-09-16T20:49:19.111 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.111 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.111 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:19.111 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605974+0000", 2024-09-16T20:49:19.112 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000182836 2024-09-16T20:49:19.112 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.112 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.112 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:19.112 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605982+0000", 2024-09-16T20:49:19.112 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1699999999999997e-06 2024-09-16T20:49:19.112 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.112 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.112 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:19.112 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605996+0000", 2024-09-16T20:49:19.112 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3893000000000001e-05 2024-09-16T20:49:19.112 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.112 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.112 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.113 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.113 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.113 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4882.0:2512 3.f 3:f4d895b7:::rbd_data.110e7c8851c.0000000000000000:head [read 16384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T20:49:19.113 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:49:19.113 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.541651422000001, 2024-09-16T20:49:19.113 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12318483, 2024-09-16T20:49:19.113 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:19.113 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:19.113 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:19.113 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:19.114 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4882", 2024-09-16T20:49:19.114 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2436923223", 2024-09-16T20:49:19.114 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2512 2024-09-16T20:49:19.114 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.114 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:19.114 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.114 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:19.114 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:49:19.114 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.114 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.114 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.114 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:19.115 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:49:19.115 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.115 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.115 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.115 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:19.115 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:49:19.115 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3499999999999995e-07 2024-09-16T20:49:19.115 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.115 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.115 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:19.115 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482943+0000", 2024-09-16T20:49:19.115 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3450000000000002e-06 2024-09-16T20:49:19.116 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.116 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.116 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:19.116 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605772+0000", 2024-09-16T20:49:19.116 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.122829796 2024-09-16T20:49:19.116 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.116 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.116 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:19.116 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605792+0000", 2024-09-16T20:49:19.116 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9916999999999999e-05 2024-09-16T20:49:19.116 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.116 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.116 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:19.117 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605917+0000", 2024-09-16T20:49:19.117 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00012500799999999999 2024-09-16T20:49:19.117 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.117 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.117 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:19.117 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606004+0000", 2024-09-16T20:49:19.117 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6242999999999994e-05 2024-09-16T20:49:19.117 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.117 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.117 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:19.117 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606120+0000", 2024-09-16T20:49:19.117 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000116786 2024-09-16T20:49:19.117 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.118 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.118 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.118 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.118 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.118 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:11702 3.15 3:acf33c8b:::rbd_data.110e747f0779.000000000000001a:head [set-alloc-hint object_size 262144 write_size 262144,write 143360~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:49:19.118 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:49:19.118 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 14.886659, 2024-09-16T20:49:19.118 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.084801043000000006, 2024-09-16T20:49:19.118 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:19.118 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:19.118 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:19.119 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:19.119 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:49:19.119 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:49:19.119 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 11702 2024-09-16T20:49:19.119 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.119 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:19.119 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.119 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:19.119 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:49:19.119 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.119 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.120 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.120 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:19.120 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:49:19.120 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.120 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.120 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.120 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:19.120 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137929+0000", 2024-09-16T20:49:19.120 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8400000000000002e-07 2024-09-16T20:49:19.120 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.120 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.120 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:19.120 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137937+0000", 2024-09-16T20:49:19.120 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0260000000000007e-06 2024-09-16T20:49:19.121 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.121 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.121 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:19.121 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137952+0000", 2024-09-16T20:49:19.121 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4659e-05 2024-09-16T20:49:19.121 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.121 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.121 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:19.121 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137960+0000", 2024-09-16T20:49:19.121 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0169999999999996e-06 2024-09-16T20:49:19.121 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.121 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.122 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:19.122 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138010+0000", 2024-09-16T20:49:19.122 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0021000000000001e-05 2024-09-16T20:49:19.122 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.122 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.122 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:19.122 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138114+0000", 2024-09-16T20:49:19.122 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000103797 2024-09-16T20:49:19.122 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.122 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.122 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:49:19.122 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138233+0000", 2024-09-16T20:49:19.122 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000118971 2024-09-16T20:49:19.122 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.122 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.123 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:19.123 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138953+0000", 2024-09-16T20:49:19.123 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00072026000000000002 2024-09-16T20:49:19.123 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.123 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.123 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:19.123 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222609+0000", 2024-09-16T20:49:19.123 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.083656650999999999 2024-09-16T20:49:19.123 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.123 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.123 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:19.123 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222684+0000", 2024-09-16T20:49:19.123 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4369999999999994e-05 2024-09-16T20:49:19.123 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.123 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.124 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:19.124 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222729+0000", 2024-09-16T20:49:19.124 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5287000000000002e-05 2024-09-16T20:49:19.124 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.124 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.124 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.124 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.124 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.124 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:11704 3.9 3:91220b75:::rbd_data.110e747f0779.000000000000001c:head [set-alloc-hint object_size 262144 write_size 262144,write 139264~6656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:49:19.124 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:49:19.124 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 14.886420999, 2024-09-16T20:49:19.124 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.084531133999999994, 2024-09-16T20:49:19.124 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:19.124 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:19.125 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:19.125 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:19.125 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:49:19.125 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:49:19.125 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 11704 2024-09-16T20:49:19.125 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.125 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:19.125 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.125 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:19.125 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:49:19.125 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.125 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.125 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.125 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:19.125 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:49:19.126 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.126 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.126 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.126 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:19.126 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138167+0000", 2024-09-16T20:49:19.126 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.33e-07 2024-09-16T20:49:19.126 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.126 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.126 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:19.126 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138179+0000", 2024-09-16T20:49:19.126 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2055999999999999e-05 2024-09-16T20:49:19.126 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.126 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.126 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:19.127 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138193+0000", 2024-09-16T20:49:19.127 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3961000000000001e-05 2024-09-16T20:49:19.127 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.127 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.127 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:19.127 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138200+0000", 2024-09-16T20:49:19.127 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4429999999999997e-06 2024-09-16T20:49:19.127 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.127 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.127 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:19.127 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138258+0000", 2024-09-16T20:49:19.127 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7225999999999997e-05 2024-09-16T20:49:19.127 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.127 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.127 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:19.128 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138364+0000", 2024-09-16T20:49:19.128 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000106013 2024-09-16T20:49:19.128 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.128 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.128 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:49:19.128 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138494+0000", 2024-09-16T20:49:19.128 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013059599999999999 2024-09-16T20:49:19.128 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.128 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.128 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:19.128 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.139220+0000", 2024-09-16T20:49:19.128 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00072539800000000004 2024-09-16T20:49:19.128 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.128 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.128 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:19.129 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222608+0000", 2024-09-16T20:49:19.129 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.083388620999999996 2024-09-16T20:49:19.129 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.129 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.129 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:19.129 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222651+0000", 2024-09-16T20:49:19.129 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2228000000000003e-05 2024-09-16T20:49:19.129 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.129 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.129 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:19.129 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222697+0000", 2024-09-16T20:49:19.129 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6659000000000002e-05 2024-09-16T20:49:19.129 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.129 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.130 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.130 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.130 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.130 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:19.131 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:49:19.132 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:19.130+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:19.232 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:19.232+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:19.278 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:19.278 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:49:19.278 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:49:19.279 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:19.287 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:49:19.289 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:49:19.290 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:19.290 INFO:teuthology.orchestra.run.smithi029.stdout: "epoch": 86, 2024-09-16T20:49:19.290 INFO:teuthology.orchestra.run.smithi029.stdout: "fsid": "e28af4a7-5fcd-4de8-8f77-a2b575361bb1", 2024-09-16T20:49:19.290 INFO:teuthology.orchestra.run.smithi029.stdout: "created": "2024-09-16T20:47:39.760609+0000", 2024-09-16T20:49:19.290 INFO:teuthology.orchestra.run.smithi029.stdout: "modified": "2024-09-16T20:49:17.807735+0000", 2024-09-16T20:49:19.290 INFO:teuthology.orchestra.run.smithi029.stdout: "last_up_change": "2024-09-16T20:47:50.238013+0000", 2024-09-16T20:49:19.290 INFO:teuthology.orchestra.run.smithi029.stdout: "last_in_change": "2024-09-16T20:49:08.779548+0000", 2024-09-16T20:49:19.290 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-16T20:49:19.290 INFO:teuthology.orchestra.run.smithi029.stdout: "flags_num": 5799936, 2024-09-16T20:49:19.290 INFO:teuthology.orchestra.run.smithi029.stdout: "flags_set": [ 2024-09-16T20:49:19.290 INFO:teuthology.orchestra.run.smithi029.stdout: "pglog_hardlimit", 2024-09-16T20:49:19.291 INFO:teuthology.orchestra.run.smithi029.stdout: "purged_snapdirs", 2024-09-16T20:49:19.291 INFO:teuthology.orchestra.run.smithi029.stdout: "recovery_deletes", 2024-09-16T20:49:19.291 INFO:teuthology.orchestra.run.smithi029.stdout: "sortbitwise" 2024-09-16T20:49:19.291 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-16T20:49:19.291 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_version": 5, 2024-09-16T20:49:19.291 INFO:teuthology.orchestra.run.smithi029.stdout: "full_ratio": 0.94999998807907104, 2024-09-16T20:49:19.291 INFO:teuthology.orchestra.run.smithi029.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-16T20:49:19.291 INFO:teuthology.orchestra.run.smithi029.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-16T20:49:19.291 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_snapshot": "", 2024-09-16T20:49:19.291 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_max": 3, 2024-09-16T20:49:19.291 INFO:teuthology.orchestra.run.smithi029.stdout: "max_osd": 6, 2024-09-16T20:49:19.291 INFO:teuthology.orchestra.run.smithi029.stdout: "require_min_compat_client": "luminous", 2024-09-16T20:49:19.291 INFO:teuthology.orchestra.run.smithi029.stdout: "min_compat_client": "jewel", 2024-09-16T20:49:19.292 INFO:teuthology.orchestra.run.smithi029.stdout: "require_osd_release": "squid", 2024-09-16T20:49:19.292 INFO:teuthology.orchestra.run.smithi029.stdout: "allow_crimson": false, 2024-09-16T20:49:19.292 INFO:teuthology.orchestra.run.smithi029.stdout: "pools": [ 2024-09-16T20:49:19.292 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.292 INFO:teuthology.orchestra.run.smithi029.stdout: "pool": 1, 2024-09-16T20:49:19.292 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_name": ".mgr", 2024-09-16T20:49:19.292 INFO:teuthology.orchestra.run.smithi029.stdout: "create_time": "2024-09-16T20:47:52.220750+0000", 2024-09-16T20:49:19.292 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-16T20:49:19.292 INFO:teuthology.orchestra.run.smithi029.stdout: "flags_names": "hashpspool", 2024-09-16T20:49:19.292 INFO:teuthology.orchestra.run.smithi029.stdout: "type": 1, 2024-09-16T20:49:19.292 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 2, 2024-09-16T20:49:19.292 INFO:teuthology.orchestra.run.smithi029.stdout: "min_size": 1, 2024-09-16T20:49:19.292 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_rule": 0, 2024-09-16T20:49:19.292 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_count": 0, 2024-09-16T20:49:19.293 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_target": 0, 2024-09-16T20:49:19.293 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T20:49:19.293 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T20:49:19.293 INFO:teuthology.orchestra.run.smithi029.stdout: "is_stretch_pool": false, 2024-09-16T20:49:19.293 INFO:teuthology.orchestra.run.smithi029.stdout: "object_hash": 2, 2024-09-16T20:49:19.293 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_autoscale_mode": "off", 2024-09-16T20:49:19.293 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num": 1, 2024-09-16T20:49:19.293 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num": 1, 2024-09-16T20:49:19.293 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num_target": 1, 2024-09-16T20:49:19.293 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_target": 1, 2024-09-16T20:49:19.293 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_pending": 1, 2024-09-16T20:49:19.293 INFO:teuthology.orchestra.run.smithi029.stdout: "last_pg_merge_meta": { 2024-09-16T20:49:19.293 INFO:teuthology.orchestra.run.smithi029.stdout: "source_pgid": "0.0", 2024-09-16T20:49:19.294 INFO:teuthology.orchestra.run.smithi029.stdout: "ready_epoch": 0, 2024-09-16T20:49:19.294 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_started": 0, 2024-09-16T20:49:19.294 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_clean": 0, 2024-09-16T20:49:19.294 INFO:teuthology.orchestra.run.smithi029.stdout: "source_version": "0'0", 2024-09-16T20:49:19.294 INFO:teuthology.orchestra.run.smithi029.stdout: "target_version": "0'0" 2024-09-16T20:49:19.294 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.294 INFO:teuthology.orchestra.run.smithi029.stdout: "last_change": "17", 2024-09-16T20:49:19.294 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend": "0", 2024-09-16T20:49:19.294 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T20:49:19.294 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T20:49:19.294 INFO:teuthology.orchestra.run.smithi029.stdout: "auid": 0, 2024-09-16T20:49:19.294 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_mode": "selfmanaged", 2024-09-16T20:49:19.294 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_seq": 0, 2024-09-16T20:49:19.295 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_epoch": 0, 2024-09-16T20:49:19.295 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_snaps": [], 2024-09-16T20:49:19.295 INFO:teuthology.orchestra.run.smithi029.stdout: "removed_snaps": "[]", 2024-09-16T20:49:19.295 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_bytes": 0, 2024-09-16T20:49:19.295 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_objects": 0, 2024-09-16T20:49:19.295 INFO:teuthology.orchestra.run.smithi029.stdout: "tiers": [], 2024-09-16T20:49:19.295 INFO:teuthology.orchestra.run.smithi029.stdout: "tier_of": -1, 2024-09-16T20:49:19.295 INFO:teuthology.orchestra.run.smithi029.stdout: "read_tier": -1, 2024-09-16T20:49:19.295 INFO:teuthology.orchestra.run.smithi029.stdout: "write_tier": -1, 2024-09-16T20:49:19.295 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_mode": "none", 2024-09-16T20:49:19.295 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_bytes": 0, 2024-09-16T20:49:19.295 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_objects": 0, 2024-09-16T20:49:19.296 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T20:49:19.296 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T20:49:19.296 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T20:49:19.296 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_flush_age": 0, 2024-09-16T20:49:19.296 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_evict_age": 0, 2024-09-16T20:49:19.296 INFO:teuthology.orchestra.run.smithi029.stdout: "erasure_code_profile": "", 2024-09-16T20:49:19.296 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_params": { 2024-09-16T20:49:19.296 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "none" 2024-09-16T20:49:19.296 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.296 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_period": 0, 2024-09-16T20:49:19.296 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_count": 0, 2024-09-16T20:49:19.296 INFO:teuthology.orchestra.run.smithi029.stdout: "use_gmt_hitset": true, 2024-09-16T20:49:19.296 INFO:teuthology.orchestra.run.smithi029.stdout: "min_read_recency_for_promote": 0, 2024-09-16T20:49:19.297 INFO:teuthology.orchestra.run.smithi029.stdout: "min_write_recency_for_promote": 0, 2024-09-16T20:49:19.297 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T20:49:19.297 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_search_last_n": 0, 2024-09-16T20:49:19.297 INFO:teuthology.orchestra.run.smithi029.stdout: "grade_table": [], 2024-09-16T20:49:19.297 INFO:teuthology.orchestra.run.smithi029.stdout: "stripe_width": 0, 2024-09-16T20:49:19.297 INFO:teuthology.orchestra.run.smithi029.stdout: "expected_num_objects": 0, 2024-09-16T20:49:19.297 INFO:teuthology.orchestra.run.smithi029.stdout: "fast_read": false, 2024-09-16T20:49:19.297 INFO:teuthology.orchestra.run.smithi029.stdout: "options": { 2024-09-16T20:49:19.297 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_max": 32, 2024-09-16T20:49:19.297 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_min": 1 2024-09-16T20:49:19.297 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.297 INFO:teuthology.orchestra.run.smithi029.stdout: "application_metadata": { 2024-09-16T20:49:19.297 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": {} 2024-09-16T20:49:19.298 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.298 INFO:teuthology.orchestra.run.smithi029.stdout: "read_balance": { 2024-09-16T20:49:19.298 INFO:teuthology.orchestra.run.smithi029.stdout: "score_type": "Fair distribution", 2024-09-16T20:49:19.298 INFO:teuthology.orchestra.run.smithi029.stdout: "score_acting": 6.059999942779541, 2024-09-16T20:49:19.298 INFO:teuthology.orchestra.run.smithi029.stdout: "score_stable": 6.059999942779541, 2024-09-16T20:49:19.298 INFO:teuthology.orchestra.run.smithi029.stdout: "optimal_score": 0.33000001311302185, 2024-09-16T20:49:19.298 INFO:teuthology.orchestra.run.smithi029.stdout: "raw_score_acting": 2, 2024-09-16T20:49:19.298 INFO:teuthology.orchestra.run.smithi029.stdout: "raw_score_stable": 2, 2024-09-16T20:49:19.298 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity_weighted": 1, 2024-09-16T20:49:19.298 INFO:teuthology.orchestra.run.smithi029.stdout: "average_primary_affinity": 1, 2024-09-16T20:49:19.298 INFO:teuthology.orchestra.run.smithi029.stdout: "average_primary_affinity_weighted": 1 2024-09-16T20:49:19.298 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.298 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.299 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.299 INFO:teuthology.orchestra.run.smithi029.stdout: "pool": 2, 2024-09-16T20:49:19.299 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_name": "rbd", 2024-09-16T20:49:19.299 INFO:teuthology.orchestra.run.smithi029.stdout: "create_time": "2024-09-16T20:47:56.756666+0000", 2024-09-16T20:49:19.299 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 8193, 2024-09-16T20:49:19.299 INFO:teuthology.orchestra.run.smithi029.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T20:49:19.299 INFO:teuthology.orchestra.run.smithi029.stdout: "type": 1, 2024-09-16T20:49:19.299 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 2, 2024-09-16T20:49:19.299 INFO:teuthology.orchestra.run.smithi029.stdout: "min_size": 1, 2024-09-16T20:49:19.299 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_rule": 0, 2024-09-16T20:49:19.299 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_count": 0, 2024-09-16T20:49:19.299 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_target": 0, 2024-09-16T20:49:19.299 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T20:49:19.300 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T20:49:19.300 INFO:teuthology.orchestra.run.smithi029.stdout: "is_stretch_pool": false, 2024-09-16T20:49:19.300 INFO:teuthology.orchestra.run.smithi029.stdout: "object_hash": 2, 2024-09-16T20:49:19.300 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_autoscale_mode": "off", 2024-09-16T20:49:19.300 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num": 8, 2024-09-16T20:49:19.300 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num": 8, 2024-09-16T20:49:19.300 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num_target": 8, 2024-09-16T20:49:19.300 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_target": 8, 2024-09-16T20:49:19.300 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_pending": 8, 2024-09-16T20:49:19.300 INFO:teuthology.orchestra.run.smithi029.stdout: "last_pg_merge_meta": { 2024-09-16T20:49:19.300 INFO:teuthology.orchestra.run.smithi029.stdout: "source_pgid": "0.0", 2024-09-16T20:49:19.300 INFO:teuthology.orchestra.run.smithi029.stdout: "ready_epoch": 0, 2024-09-16T20:49:19.301 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_started": 0, 2024-09-16T20:49:19.301 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_clean": 0, 2024-09-16T20:49:19.301 INFO:teuthology.orchestra.run.smithi029.stdout: "source_version": "0'0", 2024-09-16T20:49:19.301 INFO:teuthology.orchestra.run.smithi029.stdout: "target_version": "0'0" 2024-09-16T20:49:19.301 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.301 INFO:teuthology.orchestra.run.smithi029.stdout: "last_change": "21", 2024-09-16T20:49:19.301 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend": "0", 2024-09-16T20:49:19.301 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T20:49:19.301 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T20:49:19.301 INFO:teuthology.orchestra.run.smithi029.stdout: "auid": 0, 2024-09-16T20:49:19.301 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_mode": "selfmanaged", 2024-09-16T20:49:19.301 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_seq": 2, 2024-09-16T20:49:19.301 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_epoch": 21, 2024-09-16T20:49:19.302 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_snaps": [], 2024-09-16T20:49:19.302 INFO:teuthology.orchestra.run.smithi029.stdout: "removed_snaps": "[]", 2024-09-16T20:49:19.302 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_bytes": 0, 2024-09-16T20:49:19.302 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_objects": 0, 2024-09-16T20:49:19.302 INFO:teuthology.orchestra.run.smithi029.stdout: "tiers": [], 2024-09-16T20:49:19.302 INFO:teuthology.orchestra.run.smithi029.stdout: "tier_of": -1, 2024-09-16T20:49:19.302 INFO:teuthology.orchestra.run.smithi029.stdout: "read_tier": -1, 2024-09-16T20:49:19.302 INFO:teuthology.orchestra.run.smithi029.stdout: "write_tier": -1, 2024-09-16T20:49:19.302 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_mode": "none", 2024-09-16T20:49:19.302 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_bytes": 0, 2024-09-16T20:49:19.302 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_objects": 0, 2024-09-16T20:49:19.302 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T20:49:19.302 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T20:49:19.303 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T20:49:19.303 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_flush_age": 0, 2024-09-16T20:49:19.303 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_evict_age": 0, 2024-09-16T20:49:19.303 INFO:teuthology.orchestra.run.smithi029.stdout: "erasure_code_profile": "", 2024-09-16T20:49:19.303 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_params": { 2024-09-16T20:49:19.303 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "none" 2024-09-16T20:49:19.303 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.303 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_period": 0, 2024-09-16T20:49:19.303 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_count": 0, 2024-09-16T20:49:19.303 INFO:teuthology.orchestra.run.smithi029.stdout: "use_gmt_hitset": true, 2024-09-16T20:49:19.303 INFO:teuthology.orchestra.run.smithi029.stdout: "min_read_recency_for_promote": 0, 2024-09-16T20:49:19.303 INFO:teuthology.orchestra.run.smithi029.stdout: "min_write_recency_for_promote": 0, 2024-09-16T20:49:19.303 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T20:49:19.304 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_search_last_n": 0, 2024-09-16T20:49:19.304 INFO:teuthology.orchestra.run.smithi029.stdout: "grade_table": [], 2024-09-16T20:49:19.304 INFO:teuthology.orchestra.run.smithi029.stdout: "stripe_width": 0, 2024-09-16T20:49:19.304 INFO:teuthology.orchestra.run.smithi029.stdout: "expected_num_objects": 0, 2024-09-16T20:49:19.304 INFO:teuthology.orchestra.run.smithi029.stdout: "fast_read": false, 2024-09-16T20:49:19.304 INFO:teuthology.orchestra.run.smithi029.stdout: "options": {}, 2024-09-16T20:49:19.304 INFO:teuthology.orchestra.run.smithi029.stdout: "application_metadata": { 2024-09-16T20:49:19.304 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd": {} 2024-09-16T20:49:19.304 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.304 INFO:teuthology.orchestra.run.smithi029.stdout: "read_balance": { 2024-09-16T20:49:19.304 INFO:teuthology.orchestra.run.smithi029.stdout: "score_type": "Fair distribution", 2024-09-16T20:49:19.304 INFO:teuthology.orchestra.run.smithi029.stdout: "score_acting": 2.25, 2024-09-16T20:49:19.304 INFO:teuthology.orchestra.run.smithi029.stdout: "score_stable": 2.25, 2024-09-16T20:49:19.305 INFO:teuthology.orchestra.run.smithi029.stdout: "optimal_score": 1, 2024-09-16T20:49:19.305 INFO:teuthology.orchestra.run.smithi029.stdout: "raw_score_acting": 2.25, 2024-09-16T20:49:19.305 INFO:teuthology.orchestra.run.smithi029.stdout: "raw_score_stable": 2.25, 2024-09-16T20:49:19.305 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity_weighted": 1, 2024-09-16T20:49:19.305 INFO:teuthology.orchestra.run.smithi029.stdout: "average_primary_affinity": 1, 2024-09-16T20:49:19.305 INFO:teuthology.orchestra.run.smithi029.stdout: "average_primary_affinity_weighted": 1 2024-09-16T20:49:19.305 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.305 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.305 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.305 INFO:teuthology.orchestra.run.smithi029.stdout: "pool": 3, 2024-09-16T20:49:19.305 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_name": "pool_client.0", 2024-09-16T20:49:19.305 INFO:teuthology.orchestra.run.smithi029.stdout: "create_time": "2024-09-16T20:48:06.418889+0000", 2024-09-16T20:49:19.306 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 8193, 2024-09-16T20:49:19.306 INFO:teuthology.orchestra.run.smithi029.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T20:49:19.306 INFO:teuthology.orchestra.run.smithi029.stdout: "type": 1, 2024-09-16T20:49:19.306 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 2, 2024-09-16T20:49:19.306 INFO:teuthology.orchestra.run.smithi029.stdout: "min_size": 1, 2024-09-16T20:49:19.306 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_rule": 0, 2024-09-16T20:49:19.306 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_count": 0, 2024-09-16T20:49:19.306 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_target": 0, 2024-09-16T20:49:19.306 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T20:49:19.306 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T20:49:19.306 INFO:teuthology.orchestra.run.smithi029.stdout: "is_stretch_pool": false, 2024-09-16T20:49:19.306 INFO:teuthology.orchestra.run.smithi029.stdout: "object_hash": 2, 2024-09-16T20:49:19.306 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_autoscale_mode": "off", 2024-09-16T20:49:19.306 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num": 32, 2024-09-16T20:49:19.307 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num": 32, 2024-09-16T20:49:19.307 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num_target": 32, 2024-09-16T20:49:19.307 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_target": 32, 2024-09-16T20:49:19.307 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_pending": 32, 2024-09-16T20:49:19.307 INFO:teuthology.orchestra.run.smithi029.stdout: "last_pg_merge_meta": { 2024-09-16T20:49:19.307 INFO:teuthology.orchestra.run.smithi029.stdout: "source_pgid": "0.0", 2024-09-16T20:49:19.307 INFO:teuthology.orchestra.run.smithi029.stdout: "ready_epoch": 0, 2024-09-16T20:49:19.307 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_started": 0, 2024-09-16T20:49:19.307 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_clean": 0, 2024-09-16T20:49:19.307 INFO:teuthology.orchestra.run.smithi029.stdout: "source_version": "0'0", 2024-09-16T20:49:19.307 INFO:teuthology.orchestra.run.smithi029.stdout: "target_version": "0'0" 2024-09-16T20:49:19.307 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.307 INFO:teuthology.orchestra.run.smithi029.stdout: "last_change": "86", 2024-09-16T20:49:19.308 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend": "0", 2024-09-16T20:49:19.308 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T20:49:19.308 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T20:49:19.308 INFO:teuthology.orchestra.run.smithi029.stdout: "auid": 0, 2024-09-16T20:49:19.308 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_mode": "selfmanaged", 2024-09-16T20:49:19.308 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_seq": 26, 2024-09-16T20:49:19.308 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_epoch": 86, 2024-09-16T20:49:19.308 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_snaps": [], 2024-09-16T20:49:19.308 INFO:teuthology.orchestra.run.smithi029.stdout: "removed_snaps": "[]", 2024-09-16T20:49:19.308 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_bytes": 0, 2024-09-16T20:49:19.308 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_objects": 0, 2024-09-16T20:49:19.308 INFO:teuthology.orchestra.run.smithi029.stdout: "tiers": [], 2024-09-16T20:49:19.308 INFO:teuthology.orchestra.run.smithi029.stdout: "tier_of": -1, 2024-09-16T20:49:19.309 INFO:teuthology.orchestra.run.smithi029.stdout: "read_tier": -1, 2024-09-16T20:49:19.309 INFO:teuthology.orchestra.run.smithi029.stdout: "write_tier": -1, 2024-09-16T20:49:19.309 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_mode": "none", 2024-09-16T20:49:19.309 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_bytes": 0, 2024-09-16T20:49:19.309 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_objects": 0, 2024-09-16T20:49:19.309 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T20:49:19.309 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T20:49:19.309 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T20:49:19.309 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_flush_age": 0, 2024-09-16T20:49:19.309 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_evict_age": 0, 2024-09-16T20:49:19.309 INFO:teuthology.orchestra.run.smithi029.stdout: "erasure_code_profile": "", 2024-09-16T20:49:19.309 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_params": { 2024-09-16T20:49:19.309 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "none" 2024-09-16T20:49:19.310 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.310 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_period": 0, 2024-09-16T20:49:19.310 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_count": 0, 2024-09-16T20:49:19.310 INFO:teuthology.orchestra.run.smithi029.stdout: "use_gmt_hitset": true, 2024-09-16T20:49:19.310 INFO:teuthology.orchestra.run.smithi029.stdout: "min_read_recency_for_promote": 0, 2024-09-16T20:49:19.310 INFO:teuthology.orchestra.run.smithi029.stdout: "min_write_recency_for_promote": 0, 2024-09-16T20:49:19.310 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T20:49:19.310 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_search_last_n": 0, 2024-09-16T20:49:19.310 INFO:teuthology.orchestra.run.smithi029.stdout: "grade_table": [], 2024-09-16T20:49:19.310 INFO:teuthology.orchestra.run.smithi029.stdout: "stripe_width": 0, 2024-09-16T20:49:19.310 INFO:teuthology.orchestra.run.smithi029.stdout: "expected_num_objects": 0, 2024-09-16T20:49:19.310 INFO:teuthology.orchestra.run.smithi029.stdout: "fast_read": false, 2024-09-16T20:49:19.310 INFO:teuthology.orchestra.run.smithi029.stdout: "options": {}, 2024-09-16T20:49:19.310 INFO:teuthology.orchestra.run.smithi029.stdout: "application_metadata": { 2024-09-16T20:49:19.311 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd": {} 2024-09-16T20:49:19.311 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.311 INFO:teuthology.orchestra.run.smithi029.stdout: "read_balance": { 2024-09-16T20:49:19.311 INFO:teuthology.orchestra.run.smithi029.stdout: "score_type": "Fair distribution", 2024-09-16T20:49:19.311 INFO:teuthology.orchestra.run.smithi029.stdout: "score_acting": 1.1299999952316284, 2024-09-16T20:49:19.311 INFO:teuthology.orchestra.run.smithi029.stdout: "score_stable": 1.1299999952316284, 2024-09-16T20:49:19.311 INFO:teuthology.orchestra.run.smithi029.stdout: "optimal_score": 1, 2024-09-16T20:49:19.311 INFO:teuthology.orchestra.run.smithi029.stdout: "raw_score_acting": 1.1299999952316284, 2024-09-16T20:49:19.311 INFO:teuthology.orchestra.run.smithi029.stdout: "raw_score_stable": 1.1299999952316284, 2024-09-16T20:49:19.311 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity_weighted": 1, 2024-09-16T20:49:19.311 INFO:teuthology.orchestra.run.smithi029.stdout: "average_primary_affinity": 1, 2024-09-16T20:49:19.311 INFO:teuthology.orchestra.run.smithi029.stdout: "average_primary_affinity_weighted": 1 2024-09-16T20:49:19.311 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.312 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.312 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-16T20:49:19.312 INFO:teuthology.orchestra.run.smithi029.stdout: "osds": [ 2024-09-16T20:49:19.312 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.312 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 0, 2024-09-16T20:49:19.312 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "6c3cb2d7-7d23-4566-9ca1-d7b08e12360c", 2024-09-16T20:49:19.312 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-16T20:49:19.312 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-16T20:49:19.312 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-16T20:49:19.312 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-16T20:49:19.312 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-16T20:49:19.312 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-16T20:49:19.312 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 13, 2024-09-16T20:49:19.313 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 80, 2024-09-16T20:49:19.313 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-16T20:49:19.313 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-16T20:49:19.313 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-16T20:49:19.313 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:19.313 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.313 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:19.313 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6816", 2024-09-16T20:49:19.313 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:49:19.313 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.313 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.313 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:19.313 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6817", 2024-09-16T20:49:19.314 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:49:19.314 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.314 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.314 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.314 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-16T20:49:19.314 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:19.314 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.314 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:19.314 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6818", 2024-09-16T20:49:19.314 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:49:19.314 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.314 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.314 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:19.314 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6819", 2024-09-16T20:49:19.315 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:49:19.315 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.315 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.315 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.315 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-16T20:49:19.315 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:19.315 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.315 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:19.315 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6822", 2024-09-16T20:49:19.315 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:49:19.315 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.315 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.315 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:19.315 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6823", 2024-09-16T20:49:19.316 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:49:19.316 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.316 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.316 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.316 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-16T20:49:19.316 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:19.316 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.316 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:19.316 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6820", 2024-09-16T20:49:19.316 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:49:19.316 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.316 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.316 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:19.317 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6821", 2024-09-16T20:49:19.317 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:49:19.317 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.317 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.317 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.317 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.29:6817/2744025949", 2024-09-16T20:49:19.317 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.29:6819/2744025949", 2024-09-16T20:49:19.317 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.29:6823/2744025949", 2024-09-16T20:49:19.317 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.29:6821/2744025949", 2024-09-16T20:49:19.317 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-16T20:49:19.317 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-16T20:49:19.317 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-16T20:49:19.317 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.318 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.318 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.318 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 1, 2024-09-16T20:49:19.318 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "5cf8c20c-1ed2-42ae-a8b0-c88cc4e36bce", 2024-09-16T20:49:19.318 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-16T20:49:19.318 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-16T20:49:19.318 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-16T20:49:19.318 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-16T20:49:19.318 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-16T20:49:19.318 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-16T20:49:19.318 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 13, 2024-09-16T20:49:19.318 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 81, 2024-09-16T20:49:19.318 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-16T20:49:19.319 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-16T20:49:19.319 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-16T20:49:19.319 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:19.319 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.319 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:19.319 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6800", 2024-09-16T20:49:19.319 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:49:19.319 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.319 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.319 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:19.319 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6801", 2024-09-16T20:49:19.319 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:49:19.319 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.319 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.320 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.320 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-16T20:49:19.320 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:19.320 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.320 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:19.320 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6802", 2024-09-16T20:49:19.320 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:49:19.320 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.320 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.320 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:19.320 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6803", 2024-09-16T20:49:19.320 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:49:19.320 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.321 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.321 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.321 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-16T20:49:19.321 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:19.321 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.321 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:19.321 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6806", 2024-09-16T20:49:19.321 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:49:19.321 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.321 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.321 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:19.321 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6807", 2024-09-16T20:49:19.321 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:49:19.322 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.322 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.322 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.322 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-16T20:49:19.322 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:19.322 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.322 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:19.322 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6804", 2024-09-16T20:49:19.322 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:49:19.322 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.322 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.322 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:19.322 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6805", 2024-09-16T20:49:19.323 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:49:19.323 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.323 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.323 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.323 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.29:6801/208628762", 2024-09-16T20:49:19.323 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.29:6803/208628762", 2024-09-16T20:49:19.323 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.29:6807/208628762", 2024-09-16T20:49:19.323 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.29:6805/208628762", 2024-09-16T20:49:19.323 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-16T20:49:19.323 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-16T20:49:19.323 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-16T20:49:19.323 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.323 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.323 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.324 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 2, 2024-09-16T20:49:19.324 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "075d16da-82bf-449b-9f6e-542c9e8d810b", 2024-09-16T20:49:19.324 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-16T20:49:19.324 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-16T20:49:19.324 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-16T20:49:19.324 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-16T20:49:19.324 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-16T20:49:19.324 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-16T20:49:19.324 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 13, 2024-09-16T20:49:19.324 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 80, 2024-09-16T20:49:19.324 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-16T20:49:19.324 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-16T20:49:19.324 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-16T20:49:19.325 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:19.325 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.325 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:19.325 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6808", 2024-09-16T20:49:19.325 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:49:19.325 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.325 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.325 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:19.325 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6809", 2024-09-16T20:49:19.325 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:49:19.325 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.326 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.326 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.326 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-16T20:49:19.326 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:19.326 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.326 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:19.326 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6810", 2024-09-16T20:49:19.326 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:49:19.326 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.326 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.326 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:19.326 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6811", 2024-09-16T20:49:19.327 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:49:19.327 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.327 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.327 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.327 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-16T20:49:19.327 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:19.327 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.327 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:19.327 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6814", 2024-09-16T20:49:19.327 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:49:19.327 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.327 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.327 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:19.328 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6815", 2024-09-16T20:49:19.328 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:49:19.328 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.328 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.328 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.328 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-16T20:49:19.328 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:19.328 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.328 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:19.328 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6812", 2024-09-16T20:49:19.328 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:49:19.328 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.328 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.329 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:19.329 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6813", 2024-09-16T20:49:19.329 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:49:19.329 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.329 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.329 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.329 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.29:6809/3447426536", 2024-09-16T20:49:19.329 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.29:6811/3447426536", 2024-09-16T20:49:19.329 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.29:6815/3447426536", 2024-09-16T20:49:19.329 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.29:6813/3447426536", 2024-09-16T20:49:19.329 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-16T20:49:19.329 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-16T20:49:19.329 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-16T20:49:19.330 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.330 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.330 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.330 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 3, 2024-09-16T20:49:19.330 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "0612be38-3a24-41b6-aba4-5a2fe985165e", 2024-09-16T20:49:19.330 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-16T20:49:19.330 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-16T20:49:19.330 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-16T20:49:19.330 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-16T20:49:19.330 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-16T20:49:19.330 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-16T20:49:19.330 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 13, 2024-09-16T20:49:19.330 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 78, 2024-09-16T20:49:19.331 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-16T20:49:19.331 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-16T20:49:19.331 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-16T20:49:19.331 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:19.331 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.331 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:19.331 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6808", 2024-09-16T20:49:19.331 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:49:19.331 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.331 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.331 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:19.331 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6809", 2024-09-16T20:49:19.331 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:49:19.332 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.332 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.332 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.332 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-16T20:49:19.332 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:19.332 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.332 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:19.332 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6810", 2024-09-16T20:49:19.332 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:49:19.332 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.332 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.332 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:19.333 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6811", 2024-09-16T20:49:19.333 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:49:19.333 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.333 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.333 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.333 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-16T20:49:19.333 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:19.333 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.333 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:19.333 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6814", 2024-09-16T20:49:19.333 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:49:19.333 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.333 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.333 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:19.334 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6815", 2024-09-16T20:49:19.334 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:49:19.334 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.334 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.334 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.334 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-16T20:49:19.334 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:19.334 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.334 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:19.334 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6812", 2024-09-16T20:49:19.334 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:49:19.334 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.334 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.335 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:19.335 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6813", 2024-09-16T20:49:19.335 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:49:19.335 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.335 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.335 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.335 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.43:6809/3687583891", 2024-09-16T20:49:19.335 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.43:6811/3687583891", 2024-09-16T20:49:19.335 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.43:6815/3687583891", 2024-09-16T20:49:19.335 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.43:6813/3687583891", 2024-09-16T20:49:19.335 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-16T20:49:19.335 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-16T20:49:19.335 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-16T20:49:19.336 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.336 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.336 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.336 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 4, 2024-09-16T20:49:19.336 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "be590bf8-e9fa-44ca-a471-6f6bcb087ca7", 2024-09-16T20:49:19.336 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-16T20:49:19.336 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-16T20:49:19.336 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-16T20:49:19.336 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-16T20:49:19.336 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-16T20:49:19.336 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-16T20:49:19.336 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 11, 2024-09-16T20:49:19.336 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 81, 2024-09-16T20:49:19.337 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-16T20:49:19.337 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-16T20:49:19.337 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-16T20:49:19.337 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:19.337 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.337 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:19.337 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6800", 2024-09-16T20:49:19.337 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:49:19.337 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.337 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.337 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:19.337 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6801", 2024-09-16T20:49:19.337 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:49:19.337 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.338 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.338 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.338 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-16T20:49:19.338 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:19.338 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.338 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:19.338 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6802", 2024-09-16T20:49:19.338 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:49:19.338 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.338 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.338 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:19.338 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6803", 2024-09-16T20:49:19.338 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:49:19.339 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.339 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.339 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.339 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-16T20:49:19.339 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:19.339 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.339 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:19.339 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6806", 2024-09-16T20:49:19.339 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:49:19.339 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.339 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.339 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:19.339 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6807", 2024-09-16T20:49:19.340 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:49:19.340 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.340 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.340 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.340 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-16T20:49:19.340 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:19.340 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.340 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:19.340 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6804", 2024-09-16T20:49:19.340 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:49:19.340 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.340 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.340 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:19.341 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6805", 2024-09-16T20:49:19.341 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:49:19.341 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.341 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.341 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.341 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.43:6801/2505220646", 2024-09-16T20:49:19.341 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.43:6803/2505220646", 2024-09-16T20:49:19.341 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.43:6807/2505220646", 2024-09-16T20:49:19.341 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.43:6805/2505220646", 2024-09-16T20:49:19.341 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-16T20:49:19.341 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-16T20:49:19.341 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-16T20:49:19.341 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.341 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.342 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.342 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 5, 2024-09-16T20:49:19.342 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "4a208e30-031b-4a9f-af52-e26d4a039ea6", 2024-09-16T20:49:19.342 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-16T20:49:19.342 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-16T20:49:19.342 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-16T20:49:19.342 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-16T20:49:19.342 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-16T20:49:19.342 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-16T20:49:19.342 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 13, 2024-09-16T20:49:19.342 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 81, 2024-09-16T20:49:19.342 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-16T20:49:19.342 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-16T20:49:19.343 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-16T20:49:19.343 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:19.343 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.343 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:19.343 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6816", 2024-09-16T20:49:19.343 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:49:19.343 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.343 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.343 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:19.343 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6817", 2024-09-16T20:49:19.343 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:49:19.343 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.343 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.343 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.344 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-16T20:49:19.344 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:19.344 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.344 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:19.344 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6818", 2024-09-16T20:49:19.344 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:49:19.344 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.344 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.344 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:19.344 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6819", 2024-09-16T20:49:19.344 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:49:19.344 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.344 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.345 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.345 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-16T20:49:19.345 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:19.345 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.345 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:19.345 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6822", 2024-09-16T20:49:19.345 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:49:19.345 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.345 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.345 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:19.345 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6823", 2024-09-16T20:49:19.346 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:49:19.346 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.346 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.346 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.346 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-16T20:49:19.346 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:19.346 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.346 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:19.346 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6820", 2024-09-16T20:49:19.346 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:49:19.346 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.347 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.347 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:19.347 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6821", 2024-09-16T20:49:19.347 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:49:19.347 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.347 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.347 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.347 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.43:6817/1517367307", 2024-09-16T20:49:19.347 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.43:6819/1517367307", 2024-09-16T20:49:19.347 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.43:6823/1517367307", 2024-09-16T20:49:19.347 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.43:6821/1517367307", 2024-09-16T20:49:19.347 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-16T20:49:19.347 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-16T20:49:19.347 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-16T20:49:19.348 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.348 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.348 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-16T20:49:19.348 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_xinfo": [ 2024-09-16T20:49:19.348 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.348 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 0, 2024-09-16T20:49:19.348 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-16T20:49:19.348 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-16T20:49:19.348 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-16T20:49:19.348 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540719139924082687, 2024-09-16T20:49:19.348 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-16T20:49:19.348 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-16T20:47:47.887412+0000", 2024-09-16T20:49:19.348 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-16T20:49:19.349 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.349 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.349 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 1, 2024-09-16T20:49:19.349 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-16T20:49:19.349 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-16T20:49:19.349 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-16T20:49:19.349 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540719139924082687, 2024-09-16T20:49:19.349 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-16T20:49:19.349 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-16T20:47:47.658782+0000", 2024-09-16T20:49:19.349 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-16T20:49:19.349 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.349 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.349 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 2, 2024-09-16T20:49:19.349 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-16T20:49:19.350 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-16T20:49:19.350 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-16T20:49:19.350 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540719139924082687, 2024-09-16T20:49:19.350 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-16T20:49:19.350 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-16T20:47:47.453752+0000", 2024-09-16T20:49:19.350 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-16T20:49:19.350 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.350 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.350 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 3, 2024-09-16T20:49:19.350 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-16T20:49:19.350 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-16T20:49:19.350 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-16T20:49:19.350 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540719139924082687, 2024-09-16T20:49:19.350 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-16T20:49:19.351 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-16T20:47:47.487992+0000", 2024-09-16T20:49:19.351 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-16T20:49:19.351 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.351 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.351 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 4, 2024-09-16T20:49:19.351 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-16T20:49:19.351 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-16T20:49:19.351 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-16T20:49:19.351 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540719139924082687, 2024-09-16T20:49:19.351 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-16T20:49:19.351 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-16T20:47:46.943344+0000", 2024-09-16T20:49:19.351 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-16T20:49:19.351 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.351 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.352 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 5, 2024-09-16T20:49:19.352 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-16T20:49:19.352 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-16T20:49:19.352 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-16T20:49:19.352 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540719139924082687, 2024-09-16T20:49:19.352 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-16T20:49:19.352 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-16T20:47:47.901547+0000", 2024-09-16T20:49:19.352 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-16T20:49:19.352 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.352 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-16T20:49:19.352 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_upmap": [], 2024-09-16T20:49:19.352 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_upmap_items": [], 2024-09-16T20:49:19.352 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_upmap_primaries": [], 2024-09-16T20:49:19.353 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_temp": [], 2024-09-16T20:49:19.353 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_temp": [], 2024-09-16T20:49:19.353 INFO:teuthology.orchestra.run.smithi029.stdout: "blocklist": {}, 2024-09-16T20:49:19.353 INFO:teuthology.orchestra.run.smithi029.stdout: "range_blocklist": {}, 2024-09-16T20:49:19.353 INFO:teuthology.orchestra.run.smithi029.stdout: "erasure_code_profiles": { 2024-09-16T20:49:19.353 INFO:teuthology.orchestra.run.smithi029.stdout: "default": { 2024-09-16T20:49:19.353 INFO:teuthology.orchestra.run.smithi029.stdout: "crush-failure-domain": "osd", 2024-09-16T20:49:19.353 INFO:teuthology.orchestra.run.smithi029.stdout: "k": "2", 2024-09-16T20:49:19.353 INFO:teuthology.orchestra.run.smithi029.stdout: "m": "1", 2024-09-16T20:49:19.353 INFO:teuthology.orchestra.run.smithi029.stdout: "plugin": "jerasure", 2024-09-16T20:49:19.353 INFO:teuthology.orchestra.run.smithi029.stdout: "technique": "reed_sol_van" 2024-09-16T20:49:19.353 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.353 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.353 INFO:teuthology.orchestra.run.smithi029.stdout: "removed_snaps_queue": [], 2024-09-16T20:49:19.354 INFO:teuthology.orchestra.run.smithi029.stdout: "new_removed_snaps": [], 2024-09-16T20:49:19.354 INFO:teuthology.orchestra.run.smithi029.stdout: "new_purged_snaps": [], 2024-09-16T20:49:19.354 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_node_flags": {}, 2024-09-16T20:49:19.354 INFO:teuthology.orchestra.run.smithi029.stdout: "device_class_flags": {}, 2024-09-16T20:49:19.354 INFO:teuthology.orchestra.run.smithi029.stdout: "stretch_mode": { 2024-09-16T20:49:19.354 INFO:teuthology.orchestra.run.smithi029.stdout: "stretch_mode_enabled": false, 2024-09-16T20:49:19.354 INFO:teuthology.orchestra.run.smithi029.stdout: "stretch_bucket_count": 0, 2024-09-16T20:49:19.354 INFO:teuthology.orchestra.run.smithi029.stdout: "degraded_stretch_mode": 0, 2024-09-16T20:49:19.354 INFO:teuthology.orchestra.run.smithi029.stdout: "recovering_stretch_mode": 0, 2024-09-16T20:49:19.354 INFO:teuthology.orchestra.run.smithi029.stdout: "stretch_mode_bucket": 0 2024-09-16T20:49:19.355 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.355 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:19.357 INFO:tasks.thrashosds.thrasher:j is {'epoch': 86, 'fsid': 'e28af4a7-5fcd-4de8-8f77-a2b575361bb1', 'created': '2024-09-16T20:47:39.760609+0000', 'modified': '2024-09-16T20:49:17.807735+0000', 'last_up_change': '2024-09-16T20:47:50.238013+0000', 'last_in_change': '2024-09-16T20:49:08.779548+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T20:47:52.220750+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-16T20:47:56.756666+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-16T20:48:06.418889+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 26, 'snap_epoch': 86, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '6c3cb2d7-7d23-4566-9ca1-d7b08e12360c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 80, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6816', 'nonce': 2744025949}, {'type': 'v1', 'addr': '172.21.15.29:6817', 'nonce': 2744025949}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6818', 'nonce': 2744025949}, {'type': 'v1', 'addr': '172.21.15.29:6819', 'nonce': 2744025949}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6822', 'nonce': 2744025949}, {'type': 'v1', 'addr': '172.21.15.29:6823', 'nonce': 2744025949}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6820', 'nonce': 2744025949}, {'type': 'v1', 'addr': '172.21.15.29:6821', 'nonce': 2744025949}]}, 'public_addr': '172.21.15.29:6817/2744025949', 'cluster_addr': '172.21.15.29:6819/2744025949', 'heartbeat_back_addr': '172.21.15.29:6823/2744025949', 'heartbeat_front_addr': '172.21.15.29:6821/2744025949', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '5cf8c20c-1ed2-42ae-a8b0-c88cc4e36bce', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 81, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6800', 'nonce': 208628762}, {'type': 'v1', 'addr': '172.21.15.29:6801', 'nonce': 208628762}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6802', 'nonce': 208628762}, {'type': 'v1', 'addr': '172.21.15.29:6803', 'nonce': 208628762}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6806', 'nonce': 208628762}, {'type': 'v1', 'addr': '172.21.15.29:6807', 'nonce': 208628762}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6804', 'nonce': 208628762}, {'type': 'v1', 'addr': '172.21.15.29:6805', 'nonce': 208628762}]}, 'public_addr': '172.21.15.29:6801/208628762', 'cluster_addr': '172.21.15.29:6803/208628762', 'heartbeat_back_addr': '172.21.15.29:6807/208628762', 'heartbeat_front_addr': '172.21.15.29:6805/208628762', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '075d16da-82bf-449b-9f6e-542c9e8d810b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 80, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6808', 'nonce': 3447426536}, {'type': 'v1', 'addr': '172.21.15.29:6809', 'nonce': 3447426536}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6810', 'nonce': 3447426536}, {'type': 'v1', 'addr': '172.21.15.29:6811', 'nonce': 3447426536}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6814', 'nonce': 3447426536}, {'type': 'v1', 'addr': '172.21.15.29:6815', 'nonce': 3447426536}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6812', 'nonce': 3447426536}, {'type': 'v1', 'addr': '172.21.15.29:6813', 'nonce': 3447426536}]}, 'public_addr': '172.21.15.29:6809/3447426536', 'cluster_addr': '172.21.15.29:6811/3447426536', 'heartbeat_back_addr': '172.21.15.29:6815/3447426536', 'heartbeat_front_addr': '172.21.15.29:6813/3447426536', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '0612be38-3a24-41b6-aba4-5a2fe985165e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 78, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6808', 'nonce': 3687583891}, {'type': 'v1', 'addr': '172.21.15.43:6809', 'nonce': 3687583891}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6810', 'nonce': 3687583891}, {'type': 'v1', 'addr': '172.21.15.43:6811', 'nonce': 3687583891}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6814', 'nonce': 3687583891}, {'type': 'v1', 'addr': '172.21.15.43:6815', 'nonce': 3687583891}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6812', 'nonce': 3687583891}, {'type': 'v1', 'addr': '172.21.15.43:6813', 'nonce': 3687583891}]}, 'public_addr': '172.21.15.43:6809/3687583891', 'cluster_addr': '172.21.15.43:6811/3687583891', 'heartbeat_back_addr': '172.21.15.43:6815/3687583891', 'heartbeat_front_addr': '172.21.15.43:6813/3687583891', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'be590bf8-e9fa-44ca-a471-6f6bcb087ca7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 81, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6800', 'nonce': 2505220646}, {'type': 'v1', 'addr': '172.21.15.43:6801', 'nonce': 2505220646}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6802', 'nonce': 2505220646}, {'type': 'v1', 'addr': '172.21.15.43:6803', 'nonce': 2505220646}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6806', 'nonce': 2505220646}, {'type': 'v1', 'addr': '172.21.15.43:6807', 'nonce': 2505220646}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6804', 'nonce': 2505220646}, {'type': 'v1', 'addr': '172.21.15.43:6805', 'nonce': 2505220646}]}, 'public_addr': '172.21.15.43:6801/2505220646', 'cluster_addr': '172.21.15.43:6803/2505220646', 'heartbeat_back_addr': '172.21.15.43:6807/2505220646', 'heartbeat_front_addr': '172.21.15.43:6805/2505220646', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '4a208e30-031b-4a9f-af52-e26d4a039ea6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 81, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6816', 'nonce': 1517367307}, {'type': 'v1', 'addr': '172.21.15.43:6817', 'nonce': 1517367307}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6818', 'nonce': 1517367307}, {'type': 'v1', 'addr': '172.21.15.43:6819', 'nonce': 1517367307}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6822', 'nonce': 1517367307}, {'type': 'v1', 'addr': '172.21.15.43:6823', 'nonce': 1517367307}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6820', 'nonce': 1517367307}, {'type': 'v1', 'addr': '172.21.15.43:6821', 'nonce': 1517367307}]}, 'public_addr': '172.21.15.43:6817/1517367307', 'cluster_addr': '172.21.15.43:6819/1517367307', 'heartbeat_back_addr': '172.21.15.43:6823/1517367307', 'heartbeat_front_addr': '172.21.15.43:6821/1517367307', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T20:47:47.887412+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T20:47:47.658782+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T20:47:47.453752+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T20:47:47.487992+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T20:47:46.943344+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T20:47:47.901547+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-16T20:49:19.357 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T20:49:19.357 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:19.355+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:19.394 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:19.394 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:49:19.394 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:49:19.394 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:49:19.394 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:19.402 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:49:19.457 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:19.457+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:19.558 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:19.557+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:19.564 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:19.564 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:49:19.564 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:49:19.564 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:49:19.564 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.564 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:49:19.565 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:49:19.565 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 86.307351565999994, 2024-09-16T20:49:19.565 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67502346999999996, 2024-09-16T20:49:19.565 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:19.565 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:19.565 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:19.565 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:19.565 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:49:19.565 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:49:19.565 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 1 2024-09-16T20:49:19.566 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.567 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:19.567 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.567 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:19.567 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:49:19.567 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.567 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.567 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.567 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:19.567 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:49:19.567 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.567 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.567 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.568 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:19.568 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254128+0000", 2024-09-16T20:49:19.568 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.463e-06 2024-09-16T20:49:19.568 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.568 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.568 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:19.568 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254135+0000", 2024-09-16T20:49:19.568 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1579999999999999e-06 2024-09-16T20:49:19.568 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.568 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.568 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:19.568 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254146+0000", 2024-09-16T20:49:19.569 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1341e-05 2024-09-16T20:49:19.569 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.569 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.569 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:19.569 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254155+0000", 2024-09-16T20:49:19.569 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.1549999999999996e-06 2024-09-16T20:49:19.569 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.569 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.569 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:19.569 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254221+0000", 2024-09-16T20:49:19.569 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5470999999999997e-05 2024-09-16T20:49:19.569 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.569 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.570 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:49:19.570 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254231+0000", 2024-09-16T20:49:19.570 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0592e-05 2024-09-16T20:49:19.570 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.570 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.570 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:19.570 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926711+0000", 2024-09-16T20:49:19.570 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67247946599999997 2024-09-16T20:49:19.570 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.570 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.570 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:19.570 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926910+0000", 2024-09-16T20:49:19.570 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019934 2024-09-16T20:49:19.571 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.571 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.571 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:49:19.571 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.927159+0000", 2024-09-16T20:49:19.571 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00024876599999999998 2024-09-16T20:49:19.571 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.571 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.571 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:19.571 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.928797+0000", 2024-09-16T20:49:19.571 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001638308 2024-09-16T20:49:19.571 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.571 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.571 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:19.571 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929026+0000", 2024-09-16T20:49:19.572 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00022876099999999999 2024-09-16T20:49:19.572 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.572 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.572 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:19.572 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929102+0000", 2024-09-16T20:49:19.572 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6184000000000001e-05 2024-09-16T20:49:19.572 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.572 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.572 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:19.572 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929150+0000", 2024-09-16T20:49:19.572 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7465e-05 2024-09-16T20:49:19.572 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.572 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.573 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.573 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.573 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.573 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4319 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4675584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:19.573 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745838+0000", 2024-09-16T20:49:19.573 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 59.815639754999999, 2024-09-16T20:49:19.573 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047640204999999998, 2024-09-16T20:49:19.573 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:19.573 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:19.573 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:19.573 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:19.573 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:49:19.573 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:19.574 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4319 2024-09-16T20:49:19.574 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.574 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:19.574 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.574 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:19.574 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745838+0000", 2024-09-16T20:49:19.574 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.574 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.574 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.574 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:19.574 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745838+0000", 2024-09-16T20:49:19.574 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.574 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.574 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.575 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:19.575 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745838+0000", 2024-09-16T20:49:19.575 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6899999999999998e-07 2024-09-16T20:49:19.575 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.575 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.575 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:19.575 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745840+0000", 2024-09-16T20:49:19.575 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0660000000000002e-06 2024-09-16T20:49:19.575 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.575 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.575 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:19.575 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745847+0000", 2024-09-16T20:49:19.575 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8689999999999998e-06 2024-09-16T20:49:19.576 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.576 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.576 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:19.576 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745853+0000", 2024-09-16T20:49:19.576 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2399999999999998e-06 2024-09-16T20:49:19.576 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.576 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.576 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:19.576 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793381+0000", 2024-09-16T20:49:19.576 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047528005999999998 2024-09-16T20:49:19.576 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.576 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.576 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:19.577 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793423+0000", 2024-09-16T20:49:19.577 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2799000000000001e-05 2024-09-16T20:49:19.577 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.577 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.577 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:19.577 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793478+0000", 2024-09-16T20:49:19.577 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4756000000000002e-05 2024-09-16T20:49:19.577 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.577 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.577 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.577 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.577 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.577 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4320 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4683776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:19.577 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745859+0000", 2024-09-16T20:49:19.578 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 59.815618510999997, 2024-09-16T20:49:19.578 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047811399999999997, 2024-09-16T20:49:19.578 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:19.578 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:19.578 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:19.578 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:19.578 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:49:19.578 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:19.578 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4320 2024-09-16T20:49:19.578 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.578 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:19.578 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.578 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:19.578 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745859+0000", 2024-09-16T20:49:19.579 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.579 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.579 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.579 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:19.579 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745859+0000", 2024-09-16T20:49:19.579 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.579 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.579 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.579 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:19.579 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745860+0000", 2024-09-16T20:49:19.579 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9099999999999999e-07 2024-09-16T20:49:19.579 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.579 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.580 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:19.580 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745861+0000", 2024-09-16T20:49:19.580 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8589999999999999e-06 2024-09-16T20:49:19.580 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.580 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.580 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:19.580 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745868+0000", 2024-09-16T20:49:19.580 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2249999999999997e-06 2024-09-16T20:49:19.580 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.580 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.580 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:19.580 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745873+0000", 2024-09-16T20:49:19.580 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1290000000000004e-06 2024-09-16T20:49:19.581 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.581 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.581 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:19.581 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793522+0000", 2024-09-16T20:49:19.581 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047648847000000001 2024-09-16T20:49:19.581 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.581 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.581 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:19.581 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793603+0000", 2024-09-16T20:49:19.581 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1167000000000004e-05 2024-09-16T20:49:19.581 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.581 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.581 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:19.582 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793671+0000", 2024-09-16T20:49:19.582 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7781999999999996e-05 2024-09-16T20:49:19.582 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.582 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.582 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.582 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.582 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.582 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4321 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4691968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:19.582 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745879+0000", 2024-09-16T20:49:19.582 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 59.815598315000003, 2024-09-16T20:49:19.582 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047950375000000003, 2024-09-16T20:49:19.582 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:19.582 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:19.583 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:19.583 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:19.583 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:49:19.583 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:19.583 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4321 2024-09-16T20:49:19.583 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.583 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:19.583 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.583 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:19.583 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745879+0000", 2024-09-16T20:49:19.583 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.583 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.583 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.584 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:19.584 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745879+0000", 2024-09-16T20:49:19.584 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.584 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.584 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.584 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:19.584 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745880+0000", 2024-09-16T20:49:19.584 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.65e-07 2024-09-16T20:49:19.584 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.584 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.584 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:19.584 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745882+0000", 2024-09-16T20:49:19.584 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.922e-06 2024-09-16T20:49:19.585 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.585 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.585 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:19.585 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745888+0000", 2024-09-16T20:49:19.585 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0510000000000002e-06 2024-09-16T20:49:19.585 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.585 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.585 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:19.585 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745893+0000", 2024-09-16T20:49:19.585 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9579999999999998e-06 2024-09-16T20:49:19.585 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.585 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.585 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:19.586 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793704+0000", 2024-09-16T20:49:19.586 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047811179000000002 2024-09-16T20:49:19.586 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.586 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.586 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:19.586 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793759+0000", 2024-09-16T20:49:19.586 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5260999999999998e-05 2024-09-16T20:49:19.586 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.586 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.586 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:19.586 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793830+0000", 2024-09-16T20:49:19.586 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0638999999999999e-05 2024-09-16T20:49:19.586 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.586 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.586 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.587 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.587 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.587 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4322 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4700160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:19.587 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745899+0000", 2024-09-16T20:49:19.587 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 59.815578877, 2024-09-16T20:49:19.587 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048057280000000001, 2024-09-16T20:49:19.587 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:19.587 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:19.587 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:19.587 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:19.587 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:49:19.587 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:19.587 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4322 2024-09-16T20:49:19.588 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.588 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:19.588 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.588 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:19.588 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745899+0000", 2024-09-16T20:49:19.588 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.588 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.588 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.588 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:19.588 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745899+0000", 2024-09-16T20:49:19.588 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.588 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.589 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.589 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:19.589 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745899+0000", 2024-09-16T20:49:19.589 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9299999999999999e-07 2024-09-16T20:49:19.589 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.589 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.589 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:19.589 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745901+0000", 2024-09-16T20:49:19.589 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0030000000000001e-06 2024-09-16T20:49:19.589 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.589 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.589 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:19.589 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745908+0000", 2024-09-16T20:49:19.590 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3729999999999998e-06 2024-09-16T20:49:19.590 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.590 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.590 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:19.590 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745914+0000", 2024-09-16T20:49:19.590 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.122e-06 2024-09-16T20:49:19.590 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.590 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.590 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:19.590 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793862+0000", 2024-09-16T20:49:19.590 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047948324 2024-09-16T20:49:19.590 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.590 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.590 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:19.591 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793903+0000", 2024-09-16T20:49:19.591 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0465e-05 2024-09-16T20:49:19.591 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.591 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.591 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:19.591 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793956+0000", 2024-09-16T20:49:19.591 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3600000000000002e-05 2024-09-16T20:49:19.591 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.591 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.591 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.591 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.591 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.591 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4323 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4708352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:19.591 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745920+0000", 2024-09-16T20:49:19.592 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 59.815557235999997, 2024-09-16T20:49:19.592 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048268138000000002, 2024-09-16T20:49:19.592 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:19.592 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:19.592 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:19.592 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:19.592 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:49:19.592 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:19.592 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4323 2024-09-16T20:49:19.592 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.592 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:19.592 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.592 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:19.592 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745920+0000", 2024-09-16T20:49:19.592 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.593 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.593 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.593 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:19.593 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745920+0000", 2024-09-16T20:49:19.593 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.593 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.593 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.593 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:19.593 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745921+0000", 2024-09-16T20:49:19.593 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7e-07 2024-09-16T20:49:19.593 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.593 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.593 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:19.593 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745923+0000", 2024-09-16T20:49:19.594 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7880000000000001e-06 2024-09-16T20:49:19.594 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.594 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.594 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:19.594 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745929+0000", 2024-09-16T20:49:19.594 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0809999999999999e-06 2024-09-16T20:49:19.594 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.594 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.594 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:19.594 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745935+0000", 2024-09-16T20:49:19.594 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5370000000000003e-06 2024-09-16T20:49:19.594 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.594 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.594 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:19.595 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793987+0000", 2024-09-16T20:49:19.595 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048051736999999997 2024-09-16T20:49:19.595 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.595 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.595 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:19.595 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.794070+0000", 2024-09-16T20:49:19.595 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.3277999999999998e-05 2024-09-16T20:49:19.595 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.595 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.595 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:19.595 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.794189+0000", 2024-09-16T20:49:19.595 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000118347 2024-09-16T20:49:19.596 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.596 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.596 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.596 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.596 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.596 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4324 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4716544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:19.596 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745946+0000", 2024-09-16T20:49:19.596 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 59.815531665000002, 2024-09-16T20:49:19.596 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048468153999999999, 2024-09-16T20:49:19.596 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:19.596 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:19.596 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:19.596 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:19.596 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:49:19.597 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:19.597 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4324 2024-09-16T20:49:19.597 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.597 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:19.597 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.597 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:19.597 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745946+0000", 2024-09-16T20:49:19.597 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.597 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.597 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.597 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:19.597 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745946+0000", 2024-09-16T20:49:19.598 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.598 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.598 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.598 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:19.598 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745946+0000", 2024-09-16T20:49:19.598 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4499999999999997e-07 2024-09-16T20:49:19.598 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.598 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.598 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:19.598 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745949+0000", 2024-09-16T20:49:19.598 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2529999999999999e-06 2024-09-16T20:49:19.598 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.598 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.599 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:19.599 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745955+0000", 2024-09-16T20:49:19.599 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6470000000000001e-06 2024-09-16T20:49:19.599 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.599 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.599 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:19.599 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745960+0000", 2024-09-16T20:49:19.599 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7910000000000004e-06 2024-09-16T20:49:19.599 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.599 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.599 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:19.599 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.794220+0000", 2024-09-16T20:49:19.599 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048260069000000003 2024-09-16T20:49:19.599 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.599 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.600 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:19.600 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.794311+0000", 2024-09-16T20:49:19.600 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0593000000000005e-05 2024-09-16T20:49:19.600 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.600 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.600 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:19.600 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.794414+0000", 2024-09-16T20:49:19.600 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000103356 2024-09-16T20:49:19.600 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.600 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.600 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.600 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.600 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.601 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3571 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:19.601 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481059+0000", 2024-09-16T20:49:19.601 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 30.080418505000001, 2024-09-16T20:49:19.601 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049572995000000002, 2024-09-16T20:49:19.601 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:19.601 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:19.601 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:19.601 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:19.601 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:19.601 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:19.601 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3571 2024-09-16T20:49:19.601 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.601 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:19.601 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.602 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:19.602 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481059+0000", 2024-09-16T20:49:19.602 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.602 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.602 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.602 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:19.602 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481059+0000", 2024-09-16T20:49:19.602 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.602 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.602 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.602 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:19.602 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481060+0000", 2024-09-16T20:49:19.602 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2599999999999998e-07 2024-09-16T20:49:19.602 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.603 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.603 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:19.603 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481063+0000", 2024-09-16T20:49:19.603 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.999e-06 2024-09-16T20:49:19.603 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.603 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.603 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:19.603 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481067+0000", 2024-09-16T20:49:19.603 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3080000000000001e-06 2024-09-16T20:49:19.603 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.603 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.603 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:19.603 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481070+0000", 2024-09-16T20:49:19.604 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9919999999999999e-06 2024-09-16T20:49:19.604 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.604 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.604 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:19.604 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.522087+0000", 2024-09-16T20:49:19.604 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.041017023 2024-09-16T20:49:19.604 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.604 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.604 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:19.604 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.522191+0000", 2024-09-16T20:49:19.604 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010389499999999999 2024-09-16T20:49:19.604 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.605 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.605 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:19.605 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.522566+0000", 2024-09-16T20:49:19.605 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00037539399999999998 2024-09-16T20:49:19.605 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.605 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.605 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:19.605 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525494+0000", 2024-09-16T20:49:19.605 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0029273070000000001 2024-09-16T20:49:19.605 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.605 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.605 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:19.605 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530355+0000", 2024-09-16T20:49:19.605 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0048612509999999996 2024-09-16T20:49:19.606 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.606 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.606 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:19.606 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530407+0000", 2024-09-16T20:49:19.606 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1817000000000002e-05 2024-09-16T20:49:19.606 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.606 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.606 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:19.606 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530632+0000", 2024-09-16T20:49:19.606 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00022558300000000001 2024-09-16T20:49:19.606 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.606 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.606 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.606 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.607 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.607 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3572 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:19.607 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481220+0000", 2024-09-16T20:49:19.607 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 30.080257936999999, 2024-09-16T20:49:19.607 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050452862000000001, 2024-09-16T20:49:19.607 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:19.607 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:19.607 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:19.607 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:19.607 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:19.607 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:19.607 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3572 2024-09-16T20:49:19.607 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.607 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:19.608 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.608 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:19.608 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481220+0000", 2024-09-16T20:49:19.608 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.608 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.608 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.608 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:19.608 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481220+0000", 2024-09-16T20:49:19.608 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.608 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.608 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.608 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:19.608 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481221+0000", 2024-09-16T20:49:19.609 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9400000000000004e-07 2024-09-16T20:49:19.609 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.609 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.609 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:19.609 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481225+0000", 2024-09-16T20:49:19.609 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0230000000000003e-06 2024-09-16T20:49:19.609 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.609 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.609 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:19.609 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481234+0000", 2024-09-16T20:49:19.609 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2920000000000001e-06 2024-09-16T20:49:19.609 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.609 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.610 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:19.610 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481240+0000", 2024-09-16T20:49:19.610 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9390000000000002e-06 2024-09-16T20:49:19.610 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.610 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.610 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:19.610 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.523544+0000", 2024-09-16T20:49:19.610 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.042304475000000001 2024-09-16T20:49:19.610 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.610 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.610 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:19.610 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.523624+0000", 2024-09-16T20:49:19.610 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.975e-05 2024-09-16T20:49:19.611 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.611 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.611 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:19.611 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.523975+0000", 2024-09-16T20:49:19.611 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00035063200000000003 2024-09-16T20:49:19.611 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.611 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.611 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:19.611 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530337+0000", 2024-09-16T20:49:19.611 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0063623589999999997 2024-09-16T20:49:19.611 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.611 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.611 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:19.612 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531592+0000", 2024-09-16T20:49:19.612 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0012550599999999999 2024-09-16T20:49:19.612 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.612 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.612 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:19.612 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531633+0000", 2024-09-16T20:49:19.612 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1192000000000001e-05 2024-09-16T20:49:19.612 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.612 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.612 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:19.612 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531673+0000", 2024-09-16T20:49:19.612 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9345999999999999e-05 2024-09-16T20:49:19.612 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.612 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.613 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.613 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.613 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.613 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3573 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:19.613 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481257+0000", 2024-09-16T20:49:19.613 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 30.080220990000001, 2024-09-16T20:49:19.613 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050511027, 2024-09-16T20:49:19.613 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:19.613 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:19.613 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:19.613 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:19.613 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:19.613 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:19.614 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3573 2024-09-16T20:49:19.614 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.614 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:19.614 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.614 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:19.614 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481257+0000", 2024-09-16T20:49:19.614 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.614 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.614 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.614 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:19.614 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481257+0000", 2024-09-16T20:49:19.614 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.614 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.615 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.615 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:19.615 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481257+0000", 2024-09-16T20:49:19.615 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.99e-07 2024-09-16T20:49:19.615 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.615 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.615 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:19.615 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481260+0000", 2024-09-16T20:49:19.615 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1709999999999998e-06 2024-09-16T20:49:19.615 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.615 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.615 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:19.615 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481271+0000", 2024-09-16T20:49:19.616 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0795000000000001e-05 2024-09-16T20:49:19.616 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.616 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.616 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:19.616 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481277+0000", 2024-09-16T20:49:19.616 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1670000000000004e-06 2024-09-16T20:49:19.616 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.616 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.616 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:19.616 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.524647+0000", 2024-09-16T20:49:19.616 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.043369810000000002 2024-09-16T20:49:19.616 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.616 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.617 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:19.617 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.524725+0000", 2024-09-16T20:49:19.617 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7286000000000005e-05 2024-09-16T20:49:19.617 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.617 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.617 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:19.617 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525062+0000", 2024-09-16T20:49:19.617 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00033739499999999999 2024-09-16T20:49:19.617 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.617 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.617 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:19.617 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530692+0000", 2024-09-16T20:49:19.617 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0056301479999999997 2024-09-16T20:49:19.617 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.618 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.618 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:19.618 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531694+0000", 2024-09-16T20:49:19.618 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0010017419999999999 2024-09-16T20:49:19.618 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.618 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.618 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:19.618 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531737+0000", 2024-09-16T20:49:19.618 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2762999999999997e-05 2024-09-16T20:49:19.618 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.618 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.618 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:19.618 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531768+0000", 2024-09-16T20:49:19.619 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1151e-05 2024-09-16T20:49:19.619 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.619 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.619 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.619 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.619 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.619 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3574 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:19.619 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:49:19.619 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 30.080188033999999, 2024-09-16T20:49:19.619 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053579441999999998, 2024-09-16T20:49:19.619 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:19.619 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:19.619 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:19.619 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:19.620 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:19.620 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:19.620 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3574 2024-09-16T20:49:19.620 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.620 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:19.620 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.620 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:19.620 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:49:19.620 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.620 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.620 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.620 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:19.620 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:49:19.621 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.621 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.621 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.621 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:19.621 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:49:19.621 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2e-07 2024-09-16T20:49:19.621 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.621 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.621 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:19.621 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481292+0000", 2024-09-16T20:49:19.621 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2630000000000002e-06 2024-09-16T20:49:19.621 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.621 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.621 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:19.622 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481297+0000", 2024-09-16T20:49:19.622 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.296e-06 2024-09-16T20:49:19.622 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.622 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.622 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:19.622 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481300+0000", 2024-09-16T20:49:19.622 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7019999999999999e-06 2024-09-16T20:49:19.622 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.622 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.622 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:19.622 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525533+0000", 2024-09-16T20:49:19.622 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.044232318 2024-09-16T20:49:19.622 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.623 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.623 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:19.623 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525619+0000", 2024-09-16T20:49:19.623 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5892999999999999e-05 2024-09-16T20:49:19.623 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.623 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.623 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:19.623 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525987+0000", 2024-09-16T20:49:19.623 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00036836099999999999 2024-09-16T20:49:19.623 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.623 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.623 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:19.623 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531779+0000", 2024-09-16T20:49:19.624 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0057918240000000001 2024-09-16T20:49:19.624 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.624 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.624 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:19.624 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534747+0000", 2024-09-16T20:49:19.624 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0029681600000000001 2024-09-16T20:49:19.624 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.624 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.624 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:19.624 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534832+0000", 2024-09-16T20:49:19.624 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5180999999999996e-05 2024-09-16T20:49:19.624 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.624 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.625 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:19.625 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534869+0000", 2024-09-16T20:49:19.625 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7024000000000003e-05 2024-09-16T20:49:19.625 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.625 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.625 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.625 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.625 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.625 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3575 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:19.625 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:49:19.625 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 30.080173294000002, 2024-09-16T20:49:19.625 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053801463000000001, 2024-09-16T20:49:19.625 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:19.625 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:19.626 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:19.626 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:19.626 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:19.626 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:19.626 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3575 2024-09-16T20:49:19.626 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.626 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:19.626 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.626 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:19.626 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:49:19.626 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.626 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.627 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.627 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:19.627 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:49:19.627 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.627 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.627 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.627 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:19.627 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481305+0000", 2024-09-16T20:49:19.627 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5400000000000002e-07 2024-09-16T20:49:19.627 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.627 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.627 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:19.627 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481307+0000", 2024-09-16T20:49:19.627 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7800000000000001e-06 2024-09-16T20:49:19.628 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.628 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.628 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:19.628 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481312+0000", 2024-09-16T20:49:19.628 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6349999999999997e-06 2024-09-16T20:49:19.628 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.628 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.628 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:19.628 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481315+0000", 2024-09-16T20:49:19.628 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7829999999999999e-06 2024-09-16T20:49:19.628 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.628 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.628 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:19.629 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.526876+0000", 2024-09-16T20:49:19.629 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.045561070000000002 2024-09-16T20:49:19.629 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.629 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.629 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:19.629 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.526954+0000", 2024-09-16T20:49:19.629 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8108000000000005e-05 2024-09-16T20:49:19.629 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.629 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.629 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:19.629 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527282+0000", 2024-09-16T20:49:19.629 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000327483 2024-09-16T20:49:19.629 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.629 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.630 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:19.630 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531784+0000", 2024-09-16T20:49:19.630 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0045020169999999997 2024-09-16T20:49:19.630 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.630 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.630 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:19.630 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534921+0000", 2024-09-16T20:49:19.630 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.003137791 2024-09-16T20:49:19.630 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.630 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.630 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:19.630 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534976+0000", 2024-09-16T20:49:19.630 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4953000000000001e-05 2024-09-16T20:49:19.631 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.631 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.631 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:19.631 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535106+0000", 2024-09-16T20:49:19.631 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000129589 2024-09-16T20:49:19.631 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.631 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.631 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.631 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.631 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.631 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3576 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:19.631 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:49:19.631 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 30.080159382000002, 2024-09-16T20:49:19.631 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053955436000000002, 2024-09-16T20:49:19.632 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:19.632 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:19.632 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:19.632 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:19.632 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:19.632 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:19.632 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3576 2024-09-16T20:49:19.632 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.632 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:19.632 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.632 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:19.632 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:49:19.632 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.633 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.633 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.633 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:19.633 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:49:19.633 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.633 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.633 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.633 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:19.633 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481319+0000", 2024-09-16T20:49:19.633 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.53e-07 2024-09-16T20:49:19.633 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.633 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.633 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:19.633 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481320+0000", 2024-09-16T20:49:19.634 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5230000000000001e-06 2024-09-16T20:49:19.634 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.634 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.634 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:19.634 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481330+0000", 2024-09-16T20:49:19.634 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0190000000000001e-05 2024-09-16T20:49:19.634 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.634 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.634 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:19.634 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481333+0000", 2024-09-16T20:49:19.634 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.151e-06 2024-09-16T20:49:19.634 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.634 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.634 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:19.635 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527689+0000", 2024-09-16T20:49:19.635 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046355192000000003 2024-09-16T20:49:19.635 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.635 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.635 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:19.635 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527764+0000", 2024-09-16T20:49:19.635 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5776000000000001e-05 2024-09-16T20:49:19.635 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.635 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.635 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:19.635 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528141+0000", 2024-09-16T20:49:19.635 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00037647599999999999 2024-09-16T20:49:19.635 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.635 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.636 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:19.636 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531788+0000", 2024-09-16T20:49:19.636 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0036470679999999998 2024-09-16T20:49:19.636 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.636 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.636 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:19.636 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535174+0000", 2024-09-16T20:49:19.636 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0033862670000000001 2024-09-16T20:49:19.636 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.636 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.636 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:19.636 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535236+0000", 2024-09-16T20:49:19.636 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1797999999999993e-05 2024-09-16T20:49:19.637 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.637 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.637 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:19.637 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535274+0000", 2024-09-16T20:49:19.637 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7741999999999999e-05 2024-09-16T20:49:19.637 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.637 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.637 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.637 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.637 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.637 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3577 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:19.637 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:49:19.637 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 30.080137782000001, 2024-09-16T20:49:19.637 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054073807000000002, 2024-09-16T20:49:19.638 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:19.638 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:19.638 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:19.638 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:19.638 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:19.638 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:19.638 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3577 2024-09-16T20:49:19.638 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.638 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:19.638 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.638 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:19.638 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:49:19.638 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.639 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.639 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.639 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:19.639 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:49:19.639 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.639 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.639 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.639 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:19.639 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:49:19.639 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.96e-07 2024-09-16T20:49:19.639 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.639 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.639 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:19.639 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481342+0000", 2024-09-16T20:49:19.640 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.714e-06 2024-09-16T20:49:19.640 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.640 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.640 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:19.640 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481349+0000", 2024-09-16T20:49:19.640 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4930000000000001e-06 2024-09-16T20:49:19.640 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.640 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.640 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:19.640 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481353+0000", 2024-09-16T20:49:19.640 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4879999999999999e-06 2024-09-16T20:49:19.640 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.640 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.641 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:19.641 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528619+0000", 2024-09-16T20:49:19.641 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047266351999999998 2024-09-16T20:49:19.641 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.641 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.641 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:19.641 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528706+0000", 2024-09-16T20:49:19.641 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6796e-05 2024-09-16T20:49:19.641 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.641 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.641 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:19.641 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529148+0000", 2024-09-16T20:49:19.641 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00044182999999999999 2024-09-16T20:49:19.641 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.642 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.642 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:19.642 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531792+0000", 2024-09-16T20:49:19.642 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0026443119999999998 2024-09-16T20:49:19.642 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.642 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.642 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:19.642 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535319+0000", 2024-09-16T20:49:19.642 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0035266439999999998 2024-09-16T20:49:19.642 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.642 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.642 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:19.642 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535372+0000", 2024-09-16T20:49:19.642 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3207000000000001e-05 2024-09-16T20:49:19.643 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.643 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.643 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:19.643 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535414+0000", 2024-09-16T20:49:19.643 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1675000000000003e-05 2024-09-16T20:49:19.643 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.643 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.643 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.643 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.643 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.643 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3578 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:19.643 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:49:19.643 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 30.080116613000001, 2024-09-16T20:49:19.644 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054189586999999997, 2024-09-16T20:49:19.644 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:19.644 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:19.644 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:19.644 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:19.644 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:19.644 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:19.644 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3578 2024-09-16T20:49:19.644 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.644 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:19.644 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.644 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:19.644 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:49:19.644 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.645 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.645 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.645 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:19.645 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:49:19.645 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.645 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.645 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.645 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:19.645 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:49:19.645 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0800000000000001e-07 2024-09-16T20:49:19.645 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.645 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.645 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:19.646 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481363+0000", 2024-09-16T20:49:19.646 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0339999999999999e-06 2024-09-16T20:49:19.646 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.646 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.646 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:19.646 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481371+0000", 2024-09-16T20:49:19.646 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4440000000000002e-06 2024-09-16T20:49:19.646 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.646 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.646 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:19.646 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481374+0000", 2024-09-16T20:49:19.646 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2049999999999998e-06 2024-09-16T20:49:19.646 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.646 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.647 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:19.647 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529555+0000", 2024-09-16T20:49:19.647 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048181202999999999 2024-09-16T20:49:19.647 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.647 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.647 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:19.647 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529616+0000", 2024-09-16T20:49:19.647 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0912000000000002e-05 2024-09-16T20:49:19.647 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.647 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.647 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:19.647 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529939+0000", 2024-09-16T20:49:19.647 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00032269800000000001 2024-09-16T20:49:19.647 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.648 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.648 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:19.648 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532602+0000", 2024-09-16T20:49:19.648 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0026634940000000002 2024-09-16T20:49:19.648 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.648 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.648 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:19.648 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535465+0000", 2024-09-16T20:49:19.648 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.002862149 2024-09-16T20:49:19.648 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.648 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.648 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:19.648 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535516+0000", 2024-09-16T20:49:19.649 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1696000000000001e-05 2024-09-16T20:49:19.649 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.649 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.649 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:19.649 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535551+0000", 2024-09-16T20:49:19.649 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4443999999999998e-05 2024-09-16T20:49:19.649 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.649 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.649 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.649 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.649 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.649 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3579 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:19.649 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:49:19.649 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 30.080098174, 2024-09-16T20:49:19.650 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054295151, 2024-09-16T20:49:19.650 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:19.650 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:19.650 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:19.650 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:19.650 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:19.650 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:19.650 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3579 2024-09-16T20:49:19.650 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.650 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:19.650 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.650 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:19.650 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:49:19.651 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.651 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.651 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.651 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:19.651 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:49:19.651 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.651 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.651 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.651 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:19.651 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:49:19.651 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3300000000000001e-07 2024-09-16T20:49:19.651 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.651 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.651 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:19.652 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481381+0000", 2024-09-16T20:49:19.652 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.649e-06 2024-09-16T20:49:19.652 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.652 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.652 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:19.652 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481387+0000", 2024-09-16T20:49:19.652 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4519999999999996e-06 2024-09-16T20:49:19.652 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.652 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.652 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:19.652 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481390+0000", 2024-09-16T20:49:19.652 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0170000000000001e-06 2024-09-16T20:49:19.652 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.653 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.653 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:19.653 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530756+0000", 2024-09-16T20:49:19.653 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049365882999999999 2024-09-16T20:49:19.653 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.653 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.653 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:19.653 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530845+0000", 2024-09-16T20:49:19.653 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9197000000000007e-05 2024-09-16T20:49:19.653 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.653 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.653 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:19.653 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531191+0000", 2024-09-16T20:49:19.654 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00034562500000000001 2024-09-16T20:49:19.654 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.654 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.654 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:19.654 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534078+0000", 2024-09-16T20:49:19.654 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0028878179999999999 2024-09-16T20:49:19.654 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.654 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.654 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:19.654 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535589+0000", 2024-09-16T20:49:19.654 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00151085 2024-09-16T20:49:19.654 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.654 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.655 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:19.655 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535639+0000", 2024-09-16T20:49:19.655 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9697000000000002e-05 2024-09-16T20:49:19.655 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.655 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.655 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:19.655 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535675+0000", 2024-09-16T20:49:19.655 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5729999999999998e-05 2024-09-16T20:49:19.655 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.655 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.655 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.655 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.655 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.655 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3580 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:19.656 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:49:19.656 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 30.080084542000002, 2024-09-16T20:49:19.656 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054397361999999998, 2024-09-16T20:49:19.656 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:19.656 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:19.656 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:19.656 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:19.656 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:19.656 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:19.656 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3580 2024-09-16T20:49:19.656 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.656 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:19.656 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.657 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:19.657 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:49:19.657 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.657 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.657 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.657 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:19.657 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:49:19.657 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.657 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.657 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.657 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:19.657 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:49:19.657 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2600000000000001e-07 2024-09-16T20:49:19.658 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.658 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.658 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:19.658 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481395+0000", 2024-09-16T20:49:19.658 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.544e-06 2024-09-16T20:49:19.658 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.658 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.658 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:19.658 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481404+0000", 2024-09-16T20:49:19.658 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8850000000000005e-06 2024-09-16T20:49:19.658 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.658 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.658 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:19.658 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481407+0000", 2024-09-16T20:49:19.659 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1540000000000002e-06 2024-09-16T20:49:19.659 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.659 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.659 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:19.659 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531823+0000", 2024-09-16T20:49:19.659 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050415637999999999 2024-09-16T20:49:19.659 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.659 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.659 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:19.659 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531895+0000", 2024-09-16T20:49:19.659 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2179000000000001e-05 2024-09-16T20:49:19.659 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.659 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.659 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:19.660 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532236+0000", 2024-09-16T20:49:19.660 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00034073799999999999 2024-09-16T20:49:19.660 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.660 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.660 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:19.660 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534083+0000", 2024-09-16T20:49:19.660 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001847298 2024-09-16T20:49:19.660 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.660 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.660 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:19.660 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535714+0000", 2024-09-16T20:49:19.660 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0016316810000000001 2024-09-16T20:49:19.661 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.661 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.661 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:19.661 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535759+0000", 2024-09-16T20:49:19.661 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4478999999999999e-05 2024-09-16T20:49:19.661 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.661 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.661 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:19.661 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535791+0000", 2024-09-16T20:49:19.661 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.154e-05 2024-09-16T20:49:19.661 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.661 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.661 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.661 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.662 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.662 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3581 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:19.662 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:49:19.662 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 30.080062545000001, 2024-09-16T20:49:19.662 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.057094403000000002, 2024-09-16T20:49:19.662 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:19.662 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:19.662 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:19.662 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:19.662 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:19.662 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:19.662 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3581 2024-09-16T20:49:19.662 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.663 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:19.663 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.663 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:19.663 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:49:19.663 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.663 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.663 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.663 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:19.663 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:49:19.663 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.663 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.663 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.663 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:19.664 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:49:19.664 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8799999999999998e-07 2024-09-16T20:49:19.664 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.664 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.664 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:19.664 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481418+0000", 2024-09-16T20:49:19.664 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1859999999999999e-06 2024-09-16T20:49:19.664 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.664 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.664 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:19.664 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481423+0000", 2024-09-16T20:49:19.664 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8279999999999999e-06 2024-09-16T20:49:19.664 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.665 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.665 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:19.665 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481426+0000", 2024-09-16T20:49:19.665 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0110000000000001e-06 2024-09-16T20:49:19.665 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.665 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.665 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:19.665 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532631+0000", 2024-09-16T20:49:19.665 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051204514999999999 2024-09-16T20:49:19.665 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.665 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.665 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:19.666 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.533381+0000", 2024-09-16T20:49:19.666 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00075029000000000003 2024-09-16T20:49:19.666 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.666 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.666 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:19.666 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.533739+0000", 2024-09-16T20:49:19.666 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00035773100000000002 2024-09-16T20:49:19.666 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.667 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.667 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:19.667 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535123+0000", 2024-09-16T20:49:19.667 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0013836160000000001 2024-09-16T20:49:19.667 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.667 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.667 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:19.667 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538413+0000", 2024-09-16T20:49:19.667 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0032902510000000001 2024-09-16T20:49:19.667 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.668 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.668 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:19.668 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538470+0000", 2024-09-16T20:49:19.668 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6697999999999998e-05 2024-09-16T20:49:19.668 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.668 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.668 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:19.668 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538510+0000", 2024-09-16T20:49:19.668 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9988999999999999e-05 2024-09-16T20:49:19.668 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.668 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.668 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.668 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.669 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.669 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:12732 3.d 3:b752444a:::rbd_header.110e747f0779:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-16T20:49:19.669 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:49:19.669 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 14.792989967, 2024-09-16T20:49:19.669 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.065302467000000003, 2024-09-16T20:49:19.669 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:19.669 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:19.669 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:19.669 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:19.669 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:49:19.669 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:49:19.669 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 12732 2024-09-16T20:49:19.669 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.670 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:19.670 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.670 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:19.670 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:49:19.670 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.670 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.670 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.670 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:19.670 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:49:19.670 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.670 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.670 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.670 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:19.671 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768490+0000", 2024-09-16T20:49:19.671 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2859999999999998e-06 2024-09-16T20:49:19.671 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.671 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.671 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:19.671 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768501+0000", 2024-09-16T20:49:19.671 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1226999999999999e-05 2024-09-16T20:49:19.671 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.671 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.671 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:19.671 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768517+0000", 2024-09-16T20:49:19.671 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.592e-05 2024-09-16T20:49:19.671 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.671 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.672 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:19.672 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768528+0000", 2024-09-16T20:49:19.672 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0689999999999999e-05 2024-09-16T20:49:19.672 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.672 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.672 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:19.672 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768584+0000", 2024-09-16T20:49:19.672 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5889e-05 2024-09-16T20:49:19.672 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.672 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.672 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:19.672 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768720+0000", 2024-09-16T20:49:19.672 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000136718 2024-09-16T20:49:19.673 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.673 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.673 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:19.673 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.769066+0000", 2024-09-16T20:49:19.673 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000346013 2024-09-16T20:49:19.673 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.673 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.673 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:19.673 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833659+0000", 2024-09-16T20:49:19.673 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.064592822999999994 2024-09-16T20:49:19.673 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.673 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.673 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:19.674 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833719+0000", 2024-09-16T20:49:19.674 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9441000000000002e-05 2024-09-16T20:49:19.674 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.674 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.674 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:19.674 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833758+0000", 2024-09-16T20:49:19.674 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9122000000000003e-05 2024-09-16T20:49:19.674 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.674 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.674 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:19.674 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833790+0000", 2024-09-16T20:49:19.674 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2338000000000001e-05 2024-09-16T20:49:19.675 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.675 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.675 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.675 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.675 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.675 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.5089.0:12732 3.d e73/23 ondisk, result = 0)", 2024-09-16T20:49:19.675 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:49:19.675 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 14.791387797000001, 2024-09-16T20:49:19.675 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.063706456999999994, 2024-09-16T20:49:19.675 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:19.675 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:19.675 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:19.675 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:19.675 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.676 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:19.676 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:49:19.676 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.676 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.676 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.676 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:19.676 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:49:19.676 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:19.676 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.676 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.676 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:19.676 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770091+0000", 2024-09-16T20:49:19.676 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.53e-06 2024-09-16T20:49:19.676 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.677 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.677 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:19.677 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770104+0000", 2024-09-16T20:49:19.677 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2203e-05 2024-09-16T20:49:19.677 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.677 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.677 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:19.677 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833637+0000", 2024-09-16T20:49:19.677 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.063533201999999997 2024-09-16T20:49:19.677 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.677 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.677 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:19.677 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833655+0000", 2024-09-16T20:49:19.678 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8369e-05 2024-09-16T20:49:19.678 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.678 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.678 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:19.678 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833707+0000", 2024-09-16T20:49:19.678 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2061999999999997e-05 2024-09-16T20:49:19.678 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.678 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.678 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:19.678 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833715+0000", 2024-09-16T20:49:19.678 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5179999999999998e-06 2024-09-16T20:49:19.678 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:19.678 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:19.678 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:19.679 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833796+0000", 2024-09-16T20:49:19.679 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1573e-05 2024-09-16T20:49:19.679 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.679 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.679 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.679 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:19.679 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:19.679 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:19.681 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:49:19.681 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:19.680+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:19.705 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:49:19.706 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-09-16T20:49:19.717 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":126,"stamp":"2024-09-16T20:49:18.218506+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1038552325,"num_objects":830,"num_object_clones":226,"num_object_copies":1660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":830,"num_whiteouts":177,"num_read":161165,"num_read_kb":654779,"num_write":9686,"num_write_kb":204445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1347,"num_bytes_recovered":41543504,"num_keys_recovered":341,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7910,"ondisk_log_size":7910,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":85,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":491024,"kb_used_data":298324,"kb_used_omap":9,"kb_used_meta":192566,"kb_avail":565740016,"statfs":{"total":579820584960,"available":579317776384,"internally_reserved":0,"allocated":305483776,"data_stored":326479942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9567,"internal_metadata":197188257},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6610056,"num_objects":47,"num_object_clones":10,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":21,"num_read":13687,"num_read_kb":53482,"num_write":138,"num_write_kb":1156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.418766"},"pg_stats":[{"pgid":"3.1f","version":"84'160","reported_seq":3930,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-16T20:49:16.002704+0000","last_change":"2024-09-16T20:49:12.802355+0000","last_active":"2024-09-16T20:49:16.002704+0000","last_peered":"2024-09-16T20:49:16.002704+0000","last_clean":"2024-09-16T20:49:16.002704+0000","last_became_active":"2024-09-16T20:49:12.801566+0000","last_became_peered":"2024-09-16T20:49:12.801566+0000","last_unstale":"2024-09-16T20:49:16.002704+0000","last_undegraded":"2024-09-16T20:49:16.002704+0000","last_fullsized":"2024-09-16T20:49:16.002704+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:26:54.881777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071634499999999996,"stat_sum":{"num_bytes":28131386,"num_objects":23,"num_object_clones":4,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":2,"num_read":3742,"num_read_kb":9558,"num_write":146,"num_write_kb":2100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":2125998,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"77'452","reported_seq":12913,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-16T20:49:15.692923+0000","last_change":"2024-09-16T20:48:53.732932+0000","last_active":"2024-09-16T20:49:15.692923+0000","last_peered":"2024-09-16T20:49:15.692923+0000","last_clean":"2024-09-16T20:49:15.692923+0000","last_became_active":"2024-09-16T20:48:53.732081+0000","last_became_peered":"2024-09-16T20:48:53.732081+0000","last_unstale":"2024-09-16T20:49:15.692923+0000","last_undegraded":"2024-09-16T20:49:15.692923+0000","last_fullsized":"2024-09-16T20:49:15.692923+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:49:57.676028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084864999999999995,"stat_sum":{"num_bytes":37205649,"num_objects":27,"num_object_clones":6,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":3,"num_read":14122,"num_read_kb":37032,"num_write":580,"num_write_kb":4864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":1415412,"num_keys_recovered":59,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"77'380","reported_seq":6569,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-16T20:49:16.224532+0000","last_change":"2024-09-16T20:48:09.564507+0000","last_active":"2024-09-16T20:49:16.224532+0000","last_peered":"2024-09-16T20:49:16.224532+0000","last_clean":"2024-09-16T20:49:16.224532+0000","last_became_active":"2024-09-16T20:48:07.568025+0000","last_became_peered":"2024-09-16T20:48:07.568025+0000","last_unstale":"2024-09-16T20:49:16.224532+0000","last_undegraded":"2024-09-16T20:49:16.224532+0000","last_fullsized":"2024-09-16T20:49:16.224532+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":380,"log_dups_size":0,"ondisk_log_size":380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:23:36.335684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068985699999999999,"stat_sum":{"num_bytes":60996162,"num_objects":31,"num_object_clones":9,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":5,"num_read":7563,"num_read_kb":67713,"num_write":499,"num_write_kb":16032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"84'380","reported_seq":1916,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-16T20:49:15.958454+0000","last_change":"2024-09-16T20:48:09.565661+0000","last_active":"2024-09-16T20:49:15.958454+0000","last_peered":"2024-09-16T20:49:15.958454+0000","last_clean":"2024-09-16T20:49:15.958454+0000","last_became_active":"2024-09-16T20:48:07.565241+0000","last_became_peered":"2024-09-16T20:48:07.565241+0000","last_unstale":"2024-09-16T20:49:15.958454+0000","last_undegraded":"2024-09-16T20:49:15.958454+0000","last_fullsized":"2024-09-16T20:49:15.958454+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":380,"log_dups_size":0,"ondisk_log_size":380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:03:29.882226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001045589,"stat_sum":{"num_bytes":16392757,"num_objects":26,"num_object_clones":5,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":4,"num_read":3000,"num_read_kb":17096,"num_write":629,"num_write_kb":11899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"84'502","reported_seq":3732,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-16T20:49:14.887911+0000","last_change":"2024-09-16T20:48:54.735381+0000","last_active":"2024-09-16T20:49:14.887911+0000","last_peered":"2024-09-16T20:49:14.887911+0000","last_clean":"2024-09-16T20:49:14.887911+0000","last_became_active":"2024-09-16T20:48:54.734278+0000","last_became_peered":"2024-09-16T20:48:54.734278+0000","last_unstale":"2024-09-16T20:49:14.887911+0000","last_undegraded":"2024-09-16T20:49:14.887911+0000","last_fullsized":"2024-09-16T20:49:14.887911+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:36:25.917939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057432399999999997,"stat_sum":{"num_bytes":21893944,"num_objects":35,"num_object_clones":9,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":8,"num_read":4918,"num_read_kb":19889,"num_write":769,"num_write_kb":9088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":1963751,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"84'604","reported_seq":7228,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-16T20:49:16.013979+0000","last_change":"2024-09-16T20:49:12.803894+0000","last_active":"2024-09-16T20:49:16.013979+0000","last_peered":"2024-09-16T20:49:16.013979+0000","last_clean":"2024-09-16T20:49:16.013979+0000","last_became_active":"2024-09-16T20:49:12.802869+0000","last_became_peered":"2024-09-16T20:49:12.802869+0000","last_unstale":"2024-09-16T20:49:16.013979+0000","last_undegraded":"2024-09-16T20:49:16.013979+0000","last_fullsized":"2024-09-16T20:49:16.013979+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":604,"log_dups_size":0,"ondisk_log_size":604,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:24:55.509071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002037849,"stat_sum":{"num_bytes":5765560,"num_objects":32,"num_object_clones":8,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":8,"num_read":10908,"num_read_kb":13434,"num_write":762,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":1051004,"num_keys_recovered":46,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"80'133","reported_seq":3849,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-16T20:49:12.800504+0000","last_change":"2024-09-16T20:48:53.732127+0000","last_active":"2024-09-16T20:49:12.800504+0000","last_peered":"2024-09-16T20:49:12.800504+0000","last_clean":"2024-09-16T20:49:12.800504+0000","last_became_active":"2024-09-16T20:48:53.731485+0000","last_became_peered":"2024-09-16T20:48:53.731485+0000","last_unstale":"2024-09-16T20:49:12.800504+0000","last_undegraded":"2024-09-16T20:49:12.800504+0000","last_fullsized":"2024-09-16T20:49:12.800504+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:33:13.590127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057190699999999997,"stat_sum":{"num_bytes":25363686,"num_objects":26,"num_object_clones":10,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":4,"num_read":3491,"num_read_kb":16593,"num_write":129,"num_write_kb":3681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":5091328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"84'207","reported_seq":4112,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-16T20:49:15.994925+0000","last_change":"2024-09-16T20:48:53.731382+0000","last_active":"2024-09-16T20:49:15.994925+0000","last_peered":"2024-09-16T20:49:15.994925+0000","last_clean":"2024-09-16T20:49:15.994925+0000","last_became_active":"2024-09-16T20:48:53.730714+0000","last_became_peered":"2024-09-16T20:48:53.730714+0000","last_unstale":"2024-09-16T20:49:15.994925+0000","last_undegraded":"2024-09-16T20:49:15.994925+0000","last_fullsized":"2024-09-16T20:49:15.994925+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:01:37.390582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011061790000000001,"stat_sum":{"num_bytes":6753808,"num_objects":21,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":3,"num_read":5400,"num_read_kb":14394,"num_write":158,"num_write_kb":4484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2412544,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"84'319","reported_seq":2719,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-16T20:49:15.947120+0000","last_change":"2024-09-16T20:49:12.804399+0000","last_active":"2024-09-16T20:49:15.947120+0000","last_peered":"2024-09-16T20:49:15.947120+0000","last_clean":"2024-09-16T20:49:15.947120+0000","last_became_active":"2024-09-16T20:49:12.803628+0000","last_became_peered":"2024-09-16T20:49:12.803628+0000","last_unstale":"2024-09-16T20:49:15.947120+0000","last_undegraded":"2024-09-16T20:49:15.947120+0000","last_fullsized":"2024-09-16T20:49:15.947120+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:18:37.587611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057915500000000003,"stat_sum":{"num_bytes":10328141,"num_objects":16,"num_object_clones":3,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":2,"num_read":4106,"num_read_kb":6823,"num_write":348,"num_write_kb":940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":331889,"num_keys_recovered":60,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"84'411","reported_seq":2870,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-16T20:49:16.014581+0000","last_change":"2024-09-16T20:48:53.731777+0000","last_active":"2024-09-16T20:49:16.014581+0000","last_peered":"2024-09-16T20:49:16.014581+0000","last_clean":"2024-09-16T20:49:16.014581+0000","last_became_active":"2024-09-16T20:48:53.730899+0000","last_became_peered":"2024-09-16T20:48:53.730899+0000","last_unstale":"2024-09-16T20:49:16.014581+0000","last_undegraded":"2024-09-16T20:49:16.014581+0000","last_fullsized":"2024-09-16T20:49:16.014581+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":411,"log_dups_size":0,"ondisk_log_size":411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:34:40.690714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010075049999999999,"stat_sum":{"num_bytes":12532849,"num_objects":26,"num_object_clones":5,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":5,"num_read":3693,"num_read_kb":17624,"num_write":606,"num_write_kb":5784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":6235436,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"84'142","reported_seq":4209,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-16T20:49:14.913581+0000","last_change":"2024-09-16T20:48:53.734382+0000","last_active":"2024-09-16T20:49:14.913581+0000","last_peered":"2024-09-16T20:49:14.913581+0000","last_clean":"2024-09-16T20:49:14.913581+0000","last_became_active":"2024-09-16T20:48:53.733689+0000","last_became_peered":"2024-09-16T20:48:53.733689+0000","last_unstale":"2024-09-16T20:49:14.913581+0000","last_undegraded":"2024-09-16T20:49:14.913581+0000","last_fullsized":"2024-09-16T20:49:14.913581+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:08:23.570385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051100699999999995,"stat_sum":{"num_bytes":38386688,"num_objects":31,"num_object_clones":11,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":8,"num_read":3712,"num_read_kb":12567,"num_write":140,"num_write_kb":3001,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":2749952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"84'174","reported_seq":6724,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-16T20:49:14.889333+0000","last_change":"2024-09-16T20:48:09.566005+0000","last_active":"2024-09-16T20:49:14.889333+0000","last_peered":"2024-09-16T20:49:14.889333+0000","last_clean":"2024-09-16T20:49:14.889333+0000","last_became_active":"2024-09-16T20:48:07.568236+0000","last_became_peered":"2024-09-16T20:48:07.568236+0000","last_unstale":"2024-09-16T20:49:14.889333+0000","last_undegraded":"2024-09-16T20:49:14.889333+0000","last_fullsized":"2024-09-16T20:49:14.889333+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:34:35.796650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001271749,"stat_sum":{"num_bytes":57403618,"num_objects":36,"num_object_clones":13,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":10,"num_read":6753,"num_read_kb":34114,"num_write":323,"num_write_kb":9953,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":146,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-16T20:49:14.805351+0000","last_change":"2024-09-16T20:48:00.277670+0000","last_active":"2024-09-16T20:49:14.805351+0000","last_peered":"2024-09-16T20:49:14.805351+0000","last_clean":"2024-09-16T20:49:14.805351+0000","last_became_active":"2024-09-16T20:47:58.274489+0000","last_became_peered":"2024-09-16T20:47:58.274489+0000","last_unstale":"2024-09-16T20:49:14.805351+0000","last_undegraded":"2024-09-16T20:49:14.805351+0000","last_fullsized":"2024-09-16T20:49:14.805351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:27:12.633064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091339500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"80'110","reported_seq":1584,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-16T20:49:12.914967+0000","last_change":"2024-09-16T20:48:50.757942+0000","last_active":"2024-09-16T20:49:12.914967+0000","last_peered":"2024-09-16T20:49:12.914967+0000","last_clean":"2024-09-16T20:49:12.914967+0000","last_became_active":"2024-09-16T20:48:50.719161+0000","last_became_peered":"2024-09-16T20:48:50.719161+0000","last_unstale":"2024-09-16T20:49:12.914967+0000","last_undegraded":"2024-09-16T20:49:12.914967+0000","last_fullsized":"2024-09-16T20:49:12.914967+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:01:22.147838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067924999999999995,"stat_sum":{"num_bytes":14800933,"num_objects":17,"num_object_clones":3,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":2,"num_read":1286,"num_read_kb":8032,"num_write":113,"num_write_kb":10004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":466506,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":142,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-16T20:49:12.800574+0000","last_change":"2024-09-16T20:48:00.277182+0000","last_active":"2024-09-16T20:49:12.800574+0000","last_peered":"2024-09-16T20:49:12.800574+0000","last_clean":"2024-09-16T20:49:12.800574+0000","last_became_active":"2024-09-16T20:47:58.274389+0000","last_became_peered":"2024-09-16T20:47:58.274389+0000","last_unstale":"2024-09-16T20:49:12.800574+0000","last_undegraded":"2024-09-16T20:49:12.800574+0000","last_fullsized":"2024-09-16T20:49:12.800574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:28:02.441830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043659299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"77'121","reported_seq":5045,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-16T20:49:12.800604+0000","last_change":"2024-09-16T20:48:50.791999+0000","last_active":"2024-09-16T20:49:12.800604+0000","last_peered":"2024-09-16T20:49:12.800604+0000","last_clean":"2024-09-16T20:49:12.800604+0000","last_became_active":"2024-09-16T20:48:50.719868+0000","last_became_peered":"2024-09-16T20:48:50.719868+0000","last_unstale":"2024-09-16T20:49:12.800604+0000","last_undegraded":"2024-09-16T20:49:12.800604+0000","last_fullsized":"2024-09-16T20:49:12.800604+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:02:56.867098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012934750000000001,"stat_sum":{"num_bytes":54601728,"num_objects":18,"num_object_clones":7,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":5,"num_read":4717,"num_read_kb":10416,"num_write":116,"num_write_kb":9218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":523776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":146,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-16T20:49:14.805166+0000","last_change":"2024-09-16T20:48:00.278054+0000","last_active":"2024-09-16T20:49:14.805166+0000","last_peered":"2024-09-16T20:49:14.805166+0000","last_clean":"2024-09-16T20:49:14.805166+0000","last_became_active":"2024-09-16T20:47:58.272861+0000","last_became_peered":"2024-09-16T20:47:58.272861+0000","last_unstale":"2024-09-16T20:49:14.805166+0000","last_undegraded":"2024-09-16T20:49:14.805166+0000","last_fullsized":"2024-09-16T20:49:14.805166+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:13:59.135547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012429489999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"77'251","reported_seq":2028,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-16T20:49:12.802713+0000","last_change":"2024-09-16T20:49:12.802627+0000","last_active":"2024-09-16T20:49:12.802713+0000","last_peered":"2024-09-16T20:49:12.802713+0000","last_clean":"2024-09-16T20:49:12.802713+0000","last_became_active":"2024-09-16T20:49:12.801810+0000","last_became_peered":"2024-09-16T20:49:12.801810+0000","last_unstale":"2024-09-16T20:49:12.802713+0000","last_undegraded":"2024-09-16T20:49:12.802713+0000","last_fullsized":"2024-09-16T20:49:12.802713+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":251,"log_dups_size":0,"ondisk_log_size":251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:46:11.314272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076367000000000004,"stat_sum":{"num_bytes":26111013,"num_objects":24,"num_object_clones":7,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":6,"num_read":1906,"num_read_kb":6895,"num_write":321,"num_write_kb":4472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":2071114,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-16T20:49:14.846688+0000","last_change":"2024-09-16T20:49:09.557800+0000","last_active":"2024-09-16T20:49:14.846688+0000","last_peered":"2024-09-16T20:49:14.846688+0000","last_clean":"2024-09-16T20:49:14.846688+0000","last_became_active":"2024-09-16T20:49:09.557513+0000","last_became_peered":"2024-09-16T20:49:09.557513+0000","last_unstale":"2024-09-16T20:49:14.846688+0000","last_undegraded":"2024-09-16T20:49:14.846688+0000","last_fullsized":"2024-09-16T20:49:14.846688+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:38:59.143723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"84'143","reported_seq":6228,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-16T20:49:16.014838+0000","last_change":"2024-09-16T20:48:53.732314+0000","last_active":"2024-09-16T20:49:16.014838+0000","last_peered":"2024-09-16T20:49:16.014838+0000","last_clean":"2024-09-16T20:49:16.014838+0000","last_became_active":"2024-09-16T20:48:53.731287+0000","last_became_peered":"2024-09-16T20:48:53.731287+0000","last_unstale":"2024-09-16T20:49:16.014838+0000","last_undegraded":"2024-09-16T20:49:16.014838+0000","last_fullsized":"2024-09-16T20:49:16.014838+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:47:01.307119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010471600000000001,"stat_sum":{"num_bytes":59990597,"num_objects":26,"num_object_clones":6,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":6,"num_read":5170,"num_read_kb":30293,"num_write":138,"num_write_kb":9384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":176266,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"84'234","reported_seq":4615,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-16T20:49:14.950962+0000","last_change":"2024-09-16T20:48:09.570254+0000","last_active":"2024-09-16T20:49:14.950962+0000","last_peered":"2024-09-16T20:49:14.950962+0000","last_clean":"2024-09-16T20:49:14.950962+0000","last_became_active":"2024-09-16T20:48:07.568143+0000","last_became_peered":"2024-09-16T20:48:07.568143+0000","last_unstale":"2024-09-16T20:49:14.950962+0000","last_undegraded":"2024-09-16T20:49:14.950962+0000","last_fullsized":"2024-09-16T20:49:14.950962+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":234,"log_dups_size":0,"ondisk_log_size":234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:50:31.454774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038800899999999998,"stat_sum":{"num_bytes":44967539,"num_objects":41,"num_object_clones":14,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":15,"num_read":4630,"num_read_kb":11471,"num_write":264,"num_write_kb":2239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":148,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-16T20:49:14.804525+0000","last_change":"2024-09-16T20:48:00.276970+0000","last_active":"2024-09-16T20:49:14.804525+0000","last_peered":"2024-09-16T20:49:14.804525+0000","last_clean":"2024-09-16T20:49:14.804525+0000","last_became_active":"2024-09-16T20:47:58.274617+0000","last_became_peered":"2024-09-16T20:47:58.274617+0000","last_unstale":"2024-09-16T20:49:14.804525+0000","last_undegraded":"2024-09-16T20:49:14.804525+0000","last_fullsized":"2024-09-16T20:49:14.804525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:28:54.467905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062077499999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"83'209","reported_seq":4013,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-16T20:49:16.006597+0000","last_change":"2024-09-16T20:49:12.801644+0000","last_active":"2024-09-16T20:49:16.006597+0000","last_peered":"2024-09-16T20:49:16.006597+0000","last_clean":"2024-09-16T20:49:16.006597+0000","last_became_active":"2024-09-16T20:49:12.800827+0000","last_became_peered":"2024-09-16T20:49:12.800827+0000","last_unstale":"2024-09-16T20:49:16.006597+0000","last_undegraded":"2024-09-16T20:49:16.006597+0000","last_fullsized":"2024-09-16T20:49:16.006597+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:07:35.547934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088331900000000003,"stat_sum":{"num_bytes":23806658,"num_objects":22,"num_object_clones":3,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":3,"num_read":5566,"num_read_kb":12229,"num_write":159,"num_write_kb":3790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":1931020,"num_keys_recovered":36,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":61,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-16T20:49:14.807592+0000","last_change":"2024-09-16T20:49:09.559342+0000","last_active":"2024-09-16T20:49:14.807592+0000","last_peered":"2024-09-16T20:49:14.807592+0000","last_clean":"2024-09-16T20:49:14.807592+0000","last_became_active":"2024-09-16T20:49:09.558884+0000","last_became_peered":"2024-09-16T20:49:09.558884+0000","last_unstale":"2024-09-16T20:49:14.807592+0000","last_undegraded":"2024-09-16T20:49:14.807592+0000","last_fullsized":"2024-09-16T20:49:14.807592+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:51:23.264062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"77'124","reported_seq":3863,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-16T20:49:12.800683+0000","last_change":"2024-09-16T20:48:09.565596+0000","last_active":"2024-09-16T20:49:12.800683+0000","last_peered":"2024-09-16T20:49:12.800683+0000","last_clean":"2024-09-16T20:49:12.800683+0000","last_became_active":"2024-09-16T20:48:07.566097+0000","last_became_peered":"2024-09-16T20:48:07.566097+0000","last_unstale":"2024-09-16T20:49:12.800683+0000","last_undegraded":"2024-09-16T20:49:12.800683+0000","last_fullsized":"2024-09-16T20:49:12.800683+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:40:52.987905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012840460000000001,"stat_sum":{"num_bytes":48576000,"num_objects":21,"num_object_clones":6,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":4,"num_read":3639,"num_read_kb":26904,"num_write":131,"num_write_kb":6445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":158,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-16T20:49:14.810034+0000","last_change":"2024-09-16T20:49:09.560350+0000","last_active":"2024-09-16T20:49:14.810034+0000","last_peered":"2024-09-16T20:49:14.810034+0000","last_clean":"2024-09-16T20:49:14.810034+0000","last_became_active":"2024-09-16T20:49:09.559789+0000","last_became_peered":"2024-09-16T20:49:09.559789+0000","last_unstale":"2024-09-16T20:49:14.810034+0000","last_undegraded":"2024-09-16T20:49:14.810034+0000","last_fullsized":"2024-09-16T20:49:14.810034+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:17:54.706074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037636000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"84'253","reported_seq":3871,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-16T20:49:14.887901+0000","last_change":"2024-09-16T20:48:53.732328+0000","last_active":"2024-09-16T20:49:14.887901+0000","last_peered":"2024-09-16T20:49:14.887901+0000","last_clean":"2024-09-16T20:49:14.887901+0000","last_became_active":"2024-09-16T20:48:53.731305+0000","last_became_peered":"2024-09-16T20:48:53.731305+0000","last_unstale":"2024-09-16T20:49:14.887901+0000","last_undegraded":"2024-09-16T20:49:14.887901+0000","last_fullsized":"2024-09-16T20:49:14.887901+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":253,"log_dups_size":0,"ondisk_log_size":253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:07:31.421759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063857599999999997,"stat_sum":{"num_bytes":19843103,"num_objects":27,"num_object_clones":7,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":7,"num_read":6055,"num_read_kb":15984,"num_write":187,"num_write_kb":2385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":931887,"num_keys_recovered":34,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":147,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-16T20:49:14.805271+0000","last_change":"2024-09-16T20:48:00.277739+0000","last_active":"2024-09-16T20:49:14.805271+0000","last_peered":"2024-09-16T20:49:14.805271+0000","last_clean":"2024-09-16T20:49:14.805271+0000","last_became_active":"2024-09-16T20:47:58.271174+0000","last_became_peered":"2024-09-16T20:47:58.271174+0000","last_unstale":"2024-09-16T20:49:14.805271+0000","last_undegraded":"2024-09-16T20:49:14.805271+0000","last_fullsized":"2024-09-16T20:49:14.805271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:11:35.985075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095771799999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":261,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-16T20:49:14.804790+0000","last_change":"2024-09-16T20:49:09.565188+0000","last_active":"2024-09-16T20:49:14.804790+0000","last_peered":"2024-09-16T20:49:14.804790+0000","last_clean":"2024-09-16T20:49:14.804790+0000","last_became_active":"2024-09-16T20:49:09.559571+0000","last_became_peered":"2024-09-16T20:49:09.559571+0000","last_unstale":"2024-09-16T20:49:14.804790+0000","last_undegraded":"2024-09-16T20:49:14.804790+0000","last_fullsized":"2024-09-16T20:49:14.804790+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:52.241408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:52.241408+0000","last_clean_scrub_stamp":"2024-09-16T20:47:52.241408+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:01:53.651981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"77'221","reported_seq":1856,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-16T20:49:13.157981+0000","last_change":"2024-09-16T20:48:09.565136+0000","last_active":"2024-09-16T20:49:13.157981+0000","last_peered":"2024-09-16T20:49:13.157981+0000","last_clean":"2024-09-16T20:49:13.157981+0000","last_became_active":"2024-09-16T20:48:07.566139+0000","last_became_peered":"2024-09-16T20:48:07.566139+0000","last_unstale":"2024-09-16T20:49:13.157981+0000","last_undegraded":"2024-09-16T20:49:13.157981+0000","last_fullsized":"2024-09-16T20:49:13.157981+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:19:42.723758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001025172,"stat_sum":{"num_bytes":18950981,"num_objects":19,"num_object_clones":3,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":3,"num_read":2147,"num_read_kb":8670,"num_write":309,"num_write_kb":3616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"84'188","reported_seq":3090,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-16T20:49:15.274159+0000","last_change":"2024-09-16T20:48:53.737981+0000","last_active":"2024-09-16T20:49:15.274159+0000","last_peered":"2024-09-16T20:49:15.274159+0000","last_clean":"2024-09-16T20:49:15.274159+0000","last_became_active":"2024-09-16T20:48:53.736558+0000","last_became_peered":"2024-09-16T20:48:53.736558+0000","last_unstale":"2024-09-16T20:49:15.274159+0000","last_undegraded":"2024-09-16T20:49:15.274159+0000","last_fullsized":"2024-09-16T20:49:15.274159+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:16:05.344631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044614999999999998,"stat_sum":{"num_bytes":36942864,"num_objects":27,"num_object_clones":9,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":8,"num_read":3504,"num_read_kb":14490,"num_write":183,"num_write_kb":6568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":2635264,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"84'177","reported_seq":4446,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-16T20:49:15.000445+0000","last_change":"2024-09-16T20:48:09.569992+0000","last_active":"2024-09-16T20:49:15.000445+0000","last_peered":"2024-09-16T20:49:15.000445+0000","last_clean":"2024-09-16T20:49:15.000445+0000","last_became_active":"2024-09-16T20:48:07.563972+0000","last_became_peered":"2024-09-16T20:48:07.563972+0000","last_unstale":"2024-09-16T20:49:15.000445+0000","last_undegraded":"2024-09-16T20:49:15.000445+0000","last_fullsized":"2024-09-16T20:49:15.000445+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:08:42.945070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039392600000000001,"stat_sum":{"num_bytes":37339949,"num_objects":34,"num_object_clones":10,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":6,"num_read":4232,"num_read_kb":26421,"num_write":326,"num_write_kb":6334,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"84'171","reported_seq":2938,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-16T20:49:15.070261+0000","last_change":"2024-09-16T20:48:54.735384+0000","last_active":"2024-09-16T20:49:15.070261+0000","last_peered":"2024-09-16T20:49:15.070261+0000","last_clean":"2024-09-16T20:49:15.070261+0000","last_became_active":"2024-09-16T20:48:54.734409+0000","last_became_peered":"2024-09-16T20:48:54.734409+0000","last_unstale":"2024-09-16T20:49:15.070261+0000","last_undegraded":"2024-09-16T20:49:15.070261+0000","last_fullsized":"2024-09-16T20:49:15.070261+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:02:56.317496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014289540000000001,"stat_sum":{"num_bytes":37051977,"num_objects":19,"num_object_clones":4,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":4,"num_read":2827,"num_read_kb":6469,"num_write":204,"num_write_kb":4409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":971906,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"75'293","reported_seq":10473,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-16T20:49:14.804859+0000","last_change":"2024-09-16T20:48:54.735265+0000","last_active":"2024-09-16T20:49:14.804859+0000","last_peered":"2024-09-16T20:49:14.804859+0000","last_clean":"2024-09-16T20:49:14.804859+0000","last_became_active":"2024-09-16T20:48:54.734672+0000","last_became_peered":"2024-09-16T20:48:54.734672+0000","last_unstale":"2024-09-16T20:49:14.804859+0000","last_undegraded":"2024-09-16T20:49:14.804859+0000","last_fullsized":"2024-09-16T20:49:14.804859+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:39:39.065420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047762999999999999,"stat_sum":{"num_bytes":62220699,"num_objects":20,"num_object_clones":6,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":5,"num_read":11061,"num_read_kb":44582,"num_write":448,"num_write_kb":9855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":530734,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"84'273","reported_seq":5687,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-16T20:49:14.887810+0000","last_change":"2024-09-16T20:48:09.565667+0000","last_active":"2024-09-16T20:49:14.887810+0000","last_peered":"2024-09-16T20:49:14.887810+0000","last_clean":"2024-09-16T20:49:14.887810+0000","last_became_active":"2024-09-16T20:48:07.567327+0000","last_became_peered":"2024-09-16T20:48:07.567327+0000","last_unstale":"2024-09-16T20:49:14.887810+0000","last_undegraded":"2024-09-16T20:49:14.887810+0000","last_fullsized":"2024-09-16T20:49:14.887810+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":273,"log_dups_size":0,"ondisk_log_size":273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:39.907068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012087560000000001,"stat_sum":{"num_bytes":24419840,"num_objects":23,"num_object_clones":7,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":5,"num_read":5696,"num_read_kb":16374,"num_write":222,"num_write_kb":7315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"74'123","reported_seq":4191,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-16T20:49:14.810197+0000","last_change":"2024-09-16T20:48:09.565024+0000","last_active":"2024-09-16T20:49:14.810197+0000","last_peered":"2024-09-16T20:49:14.810197+0000","last_clean":"2024-09-16T20:49:14.810197+0000","last_became_active":"2024-09-16T20:48:07.570704+0000","last_became_peered":"2024-09-16T20:48:07.570704+0000","last_unstale":"2024-09-16T20:49:14.810197+0000","last_undegraded":"2024-09-16T20:49:14.810197+0000","last_fullsized":"2024-09-16T20:49:14.810197+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:54:40.693369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011026149999999999,"stat_sum":{"num_bytes":43014849,"num_objects":26,"num_object_clones":9,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":5,"num_read":3969,"num_read_kb":14580,"num_write":120,"num_write_kb":3719,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"84'195","reported_seq":4640,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-16T20:49:14.935107+0000","last_change":"2024-09-16T20:48:09.570786+0000","last_active":"2024-09-16T20:49:14.935107+0000","last_peered":"2024-09-16T20:49:14.935107+0000","last_clean":"2024-09-16T20:49:14.935107+0000","last_became_active":"2024-09-16T20:48:07.560783+0000","last_became_peered":"2024-09-16T20:48:07.560783+0000","last_unstale":"2024-09-16T20:49:14.935107+0000","last_undegraded":"2024-09-16T20:49:14.935107+0000","last_fullsized":"2024-09-16T20:49:14.935107+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:35:05.640671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040623999999999999,"stat_sum":{"num_bytes":22771216,"num_objects":37,"num_object_clones":13,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":10,"num_read":5281,"num_read_kb":26316,"num_write":163,"num_write_kb":8388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"84'244","reported_seq":4868,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-16T20:49:15.901151+0000","last_change":"2024-09-16T20:49:12.801692+0000","last_active":"2024-09-16T20:49:15.901151+0000","last_peered":"2024-09-16T20:49:15.901151+0000","last_clean":"2024-09-16T20:49:15.901151+0000","last_became_active":"2024-09-16T20:49:12.801013+0000","last_became_peered":"2024-09-16T20:49:12.801013+0000","last_unstale":"2024-09-16T20:49:15.901151+0000","last_undegraded":"2024-09-16T20:49:15.901151+0000","last_fullsized":"2024-09-16T20:49:15.901151+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":244,"log_dups_size":0,"ondisk_log_size":244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:10:54.095058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115576,"stat_sum":{"num_bytes":53984331,"num_objects":35,"num_object_clones":13,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":12,"num_read":5517,"num_read_kb":30877,"num_write":201,"num_write_kb":3377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":99,"num_bytes_recovered":5210742,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"77'137","reported_seq":5611,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-16T20:49:16.007455+0000","last_change":"2024-09-16T20:48:09.564513+0000","last_active":"2024-09-16T20:49:16.007455+0000","last_peered":"2024-09-16T20:49:16.007455+0000","last_clean":"2024-09-16T20:49:16.007455+0000","last_became_active":"2024-09-16T20:48:07.571985+0000","last_became_peered":"2024-09-16T20:48:07.571985+0000","last_unstale":"2024-09-16T20:49:16.007455+0000","last_undegraded":"2024-09-16T20:49:16.007455+0000","last_fullsized":"2024-09-16T20:49:16.007455+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:15:58.123591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077335299999999995,"stat_sum":{"num_bytes":49721856,"num_objects":24,"num_object_clones":4,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":1,"num_read":5361,"num_read_kb":53815,"num_write":171,"num_write_kb":18752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"84'261","reported_seq":2084,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-16T20:49:15.055234+0000","last_change":"2024-09-16T20:49:12.804699+0000","last_active":"2024-09-16T20:49:15.055234+0000","last_peered":"2024-09-16T20:49:15.055234+0000","last_clean":"2024-09-16T20:49:15.055234+0000","last_became_active":"2024-09-16T20:49:12.803689+0000","last_became_peered":"2024-09-16T20:49:12.803689+0000","last_unstale":"2024-09-16T20:49:15.055234+0000","last_undegraded":"2024-09-16T20:49:15.055234+0000","last_fullsized":"2024-09-16T20:49:15.055234+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":261,"log_dups_size":0,"ondisk_log_size":261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:55:57.256625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000840955,"stat_sum":{"num_bytes":15595599,"num_objects":18,"num_object_clones":5,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":4,"num_read":2267,"num_read_kb":8049,"num_write":351,"num_write_kb":1731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":107,"num_bytes_recovered":1536239,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"84'243","reported_seq":4523,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-16T20:49:14.887446+0000","last_change":"2024-09-16T20:48:09.565356+0000","last_active":"2024-09-16T20:49:14.887446+0000","last_peered":"2024-09-16T20:49:14.887446+0000","last_clean":"2024-09-16T20:49:14.887446+0000","last_became_active":"2024-09-16T20:48:07.565717+0000","last_became_peered":"2024-09-16T20:48:07.565717+0000","last_unstale":"2024-09-16T20:49:14.887446+0000","last_undegraded":"2024-09-16T20:49:14.887446+0000","last_fullsized":"2024-09-16T20:49:14.887446+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":243,"log_dups_size":0,"ondisk_log_size":243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:56:33.549189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013583239999999999,"stat_sum":{"num_bytes":22095958,"num_objects":18,"num_object_clones":3,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":4,"num_read":4860,"num_read_kb":15020,"num_write":367,"num_write_kb":6764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1037961938,"num_objects":826,"num_object_clones":226,"num_object_copies":1652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":826,"num_whiteouts":177,"num_read":161099,"num_read_kb":654724,"num_write":9583,"num_write_kb":202815,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1343,"num_bytes_recovered":40362768,"num_keys_recovered":341,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":296513536,"data_stored":320999925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7835,"ondisk_log_size":7835,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37764,"kb_used_data":10736,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94334076,"statfs":{"total":96636764160,"available":96598093824,"internally_reserved":0,"allocated":10993664,"data_stored":10741355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":89456,"kb_used_data":58268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94282384,"statfs":{"total":96636764160,"available":96545161216,"internally_reserved":0,"allocated":59666432,"data_stored":62507437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":95760,"kb_used_data":68724,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94276080,"statfs":{"total":96636764160,"available":96538705920,"internally_reserved":0,"allocated":70373376,"data_stored":75844918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":90064,"kb_used_data":58880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94281776,"statfs":{"total":96636764160,"available":96544538624,"internally_reserved":0,"allocated":60293120,"data_stored":65053079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":115020,"kb_used_data":65784,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94256820,"statfs":{"total":96636764160,"available":96518983680,"internally_reserved":0,"allocated":67362816,"data_stored":74369449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640276,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62960,"kb_used_data":35932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94308880,"statfs":{"total":96636764160,"available":96572293120,"internally_reserved":0,"allocated":36794368,"data_stored":37963704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66449408,"data_stored":74034438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":58728448,"data_stored":64071911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69640192,"data_stored":75674278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":58900480,"data_stored":62303005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":33226752,"data_stored":35022469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":9568256,"data_stored":9893824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T20:49:19.719 INFO:tasks.thrashosds.thrasher:Setting 3.c to [4, 2, 0, 5] 2024-09-16T20:49:19.719 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.c', '4', '2', '0', '5'] 2024-09-16T20:49:19.719 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.c 4 2 0 5 2024-09-16T20:49:19.743 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T20:49:19.802 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:19.801+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:19.817 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:19.817 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:49:19.817 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:49:19.817 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:19.818 INFO:teuthology.orchestra.run.smithi029.stderr:noscrub is set 2024-09-16T20:49:19.830 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:49:19.882 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:19.881+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:19.900 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:49:19.900 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:49:19.916 INFO:teuthology.orchestra.run.smithi139.stdout:truncating image image_client.0-clone24 from 0x851b200 (overlap 0x454000) to 0xdd800 2024-09-16T20:49:19.921 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:49:19.921 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:49:19.944 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:49:19.945 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:49:19.975 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:49:19.975 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:49:19.982 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:19.981+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:19.997 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:49:19.997 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:49:20.014 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:20.014 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:20.014 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:49:20.014 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:49:20.024 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:49:20.108 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:20.107+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:20.174 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:20.174 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:20.183 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:49:20.184 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:20.184+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:20.272 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #23, image image_client.0-clone23 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent24 2024-09-16T20:49:20.285 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:20.284+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:20.298 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:20.298 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:20.307 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:49:20.392 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:20.392+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:20.468 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:20.469 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:20.478 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:49:20.520 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:20.519+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:20.586 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:20.586+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:20.641 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:20.641 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:20.651 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:49:20.692 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:20.691+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:20.789 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:20.788+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:20.798 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:20.799 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:20.807 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:49:20.821 INFO:teuthology.orchestra.run.smithi029.stderr:set 3.c pg_upmap_items mapping to [4->2,0->5] 2024-09-16T20:49:20.891 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:20.891+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:20.900 INFO:teuthology.orchestra.run.smithi139.stdout:3112 trunc from 0x851b200 to 0x78cf200 2024-09-16T20:49:20.929 INFO:teuthology.orchestra.run.smithi139.stdout:3115 punch from 0x38adc00 to 0x38b4600, (0x6a00 bytes) 2024-09-16T20:49:20.937 INFO:teuthology.orchestra.run.smithi139.stdout:truncating to largest ever: 0xed9f000 2024-09-16T20:49:20.946 INFO:teuthology.orchestra.run.smithi139.stdout:3116 trunc from 0x78cf200 to 0xed9f000 2024-09-16T20:49:20.955 INFO:teuthology.orchestra.run.smithi139.stdout:3117 punch from 0x10a0000 to 0x10ad800, (0xd800 bytes) 2024-09-16T20:49:20.959 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:20.959 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:20.960 INFO:teuthology.orchestra.run.smithi139.stdout:3118 write 0x26be200 thru 0x26ccff3 (0xedf4 bytes) 2024-09-16T20:49:20.961 INFO:teuthology.orchestra.run.smithi139.stdout:3119 trunc from 0xed9f000 to 0xe3ee600 2024-09-16T20:49:20.969 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:49:20.975 INFO:teuthology.orchestra.run.smithi139.stdout:3122 write 0x4752000 thru 0x475ab53 (0x8b54 bytes) 2024-09-16T20:49:20.978 INFO:teuthology.orchestra.run.smithi139.stdout:3125 write 0xbc17600 thru 0xbc1d41a (0x5e1b bytes) 2024-09-16T20:49:20.978 INFO:teuthology.orchestra.run.smithi139.stdout:3127 write 0x2adea00 thru 0x2ae9573 (0xab74 bytes) 2024-09-16T20:49:20.978 INFO:teuthology.orchestra.run.smithi139.stdout:3129 read 0x487da00 thru 0x4880601 (0x2c02 bytes) 2024-09-16T20:49:20.978 INFO:teuthology.orchestra.run.smithi139.stdout:3132 write 0x7927e00 thru 0x7930503 (0x8704 bytes) 2024-09-16T20:49:20.978 INFO:teuthology.orchestra.run.smithi139.stdout:3135 read 0x9eea000 thru 0x9ef9c1f (0xfc20 bytes) 2024-09-16T20:49:20.979 INFO:teuthology.orchestra.run.smithi139.stdout:3136 read 0x5df0a00 thru 0x5dfce9c (0xc49d bytes) 2024-09-16T20:49:20.993 INFO:teuthology.orchestra.run.smithi139.stdout:3138 trunc from 0xe3ee600 to 0x2abbc00 2024-09-16T20:49:21.009 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:21.009+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:21.032 INFO:teuthology.orchestra.run.smithi139.stdout:3141 write 0x43be800 thru 0x43bf9df (0x11e0 bytes) 2024-09-16T20:49:21.033 INFO:teuthology.orchestra.run.smithi139.stdout:3142 read 0x418c400 thru 0x4194e0d (0x8a0e bytes) 2024-09-16T20:49:21.052 INFO:teuthology.orchestra.run.smithi139.stdout:3143 write 0x8f4ec00 thru 0x8f5c635 (0xda36 bytes) 2024-09-16T20:49:21.059 INFO:teuthology.orchestra.run.smithi139.stdout:3144 write 0x8c0c800 thru 0x8c1bf66 (0xf767 bytes) 2024-09-16T20:49:21.060 INFO:teuthology.orchestra.run.smithi139.stdout:3147 read 0x3aa4a00 thru 0x3ab3667 (0xec68 bytes) 2024-09-16T20:49:21.072 INFO:teuthology.orchestra.run.smithi139.stdout:3151 write 0xa052e00 thru 0xa05587e (0x2a7f bytes) 2024-09-16T20:49:21.078 INFO:teuthology.orchestra.run.smithi139.stdout:3152 read 0x857b600 thru 0x858670b (0xb10c bytes) 2024-09-16T20:49:21.078 INFO:teuthology.orchestra.run.smithi139.stdout:3153 read 0x3c5e00 thru 0x3d2edf (0xd0e0 bytes) 2024-09-16T20:49:21.091 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:21.091+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:21.093 INFO:teuthology.orchestra.run.smithi139.stdout:3154 write 0xc48a800 thru 0xc48bf99 (0x179a bytes) 2024-09-16T20:49:21.100 INFO:teuthology.orchestra.run.smithi139.stdout:3156 write 0xa728800 thru 0xa73464d (0xbe4e bytes) 2024-09-16T20:49:21.100 INFO:teuthology.orchestra.run.smithi139.stdout:3159 punch from 0x7ccea00 to 0x7cd9a00, (0xb000 bytes) 2024-09-16T20:49:21.113 INFO:teuthology.orchestra.run.smithi139.stdout:3160 trunc from 0xc48c000 to 0x2f3c400 2024-09-16T20:49:21.119 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:21.119 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:21.127 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:49:21.192 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:21.191+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:21.275 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:21.276 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:21.284 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:49:21.293 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:21.292+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:21.391 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:21.391 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:21.394 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:21.393+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:21.400 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:49:21.495 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:21.495+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:21.547 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:21.547 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:21.556 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:49:21.600 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:21.599+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:21.676 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:21.676 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:21.685 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:49:21.726 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:21.725+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:21.797 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:21.797+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:21.837 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:21.837 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:21.840 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T20:49:21.846 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:49:21.954 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:21.929+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:21.997 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:21.997 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:21.998 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:21.997+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:22.006 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:49:22.027 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T20:49:22.098 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:22.097+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:22.160 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:22.161 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:22.170 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:49:22.174 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:49:22.174 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:49:22.192 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:49:22.192 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:49:22.199 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:22.199+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:22.209 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:49:22.209 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:49:22.227 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:49:22.227 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:49:22.244 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:49:22.244 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:49:22.262 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:49:22.262 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:49:22.283 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:22.283 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:49:22.283 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:49:22.283 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:49:22.283 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:22.292 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:49:22.333 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:22.332+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:22.401 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:22.400+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:22.441 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:22.441 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:49:22.441 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:49:22.441 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:49:22.441 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.441 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:49:22.441 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:49:22.441 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 89.184862785999997, 2024-09-16T20:49:22.442 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67502346999999996, 2024-09-16T20:49:22.442 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:22.442 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:22.442 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:22.442 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:22.442 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:49:22.442 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:49:22.442 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 1 2024-09-16T20:49:22.442 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.442 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:22.442 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.442 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:22.442 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:49:22.442 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:22.442 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.443 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.443 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:22.443 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:49:22.443 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:22.443 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.443 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.443 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:22.443 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254128+0000", 2024-09-16T20:49:22.443 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.463e-06 2024-09-16T20:49:22.443 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.443 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.443 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:22.443 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254135+0000", 2024-09-16T20:49:22.444 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1579999999999999e-06 2024-09-16T20:49:22.444 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.444 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.444 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:22.444 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254146+0000", 2024-09-16T20:49:22.444 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1341e-05 2024-09-16T20:49:22.444 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.444 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.444 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:22.444 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254155+0000", 2024-09-16T20:49:22.444 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.1549999999999996e-06 2024-09-16T20:49:22.444 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.444 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.445 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:22.445 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254221+0000", 2024-09-16T20:49:22.445 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5470999999999997e-05 2024-09-16T20:49:22.445 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.445 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.445 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:49:22.445 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254231+0000", 2024-09-16T20:49:22.445 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0592e-05 2024-09-16T20:49:22.445 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.445 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.445 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:22.445 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926711+0000", 2024-09-16T20:49:22.445 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67247946599999997 2024-09-16T20:49:22.445 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.446 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.446 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:22.446 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926910+0000", 2024-09-16T20:49:22.446 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019934 2024-09-16T20:49:22.446 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.446 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.446 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:49:22.446 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.927159+0000", 2024-09-16T20:49:22.446 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00024876599999999998 2024-09-16T20:49:22.446 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.446 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.446 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:22.446 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.928797+0000", 2024-09-16T20:49:22.447 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001638308 2024-09-16T20:49:22.447 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.447 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.447 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:22.447 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929026+0000", 2024-09-16T20:49:22.447 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00022876099999999999 2024-09-16T20:49:22.447 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.447 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.447 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:22.447 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929102+0000", 2024-09-16T20:49:22.447 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6184000000000001e-05 2024-09-16T20:49:22.447 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.447 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.448 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:22.448 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929150+0000", 2024-09-16T20:49:22.448 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7465e-05 2024-09-16T20:49:22.448 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:22.448 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:22.448 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:22.448 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.448 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.448 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4319 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4675584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:22.448 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745838+0000", 2024-09-16T20:49:22.448 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 62.693150975000002, 2024-09-16T20:49:22.448 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047640204999999998, 2024-09-16T20:49:22.448 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:22.449 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:22.449 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:22.449 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:22.449 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:49:22.449 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:22.449 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4319 2024-09-16T20:49:22.449 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.449 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:22.449 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.449 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:22.449 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745838+0000", 2024-09-16T20:49:22.449 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:22.450 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.450 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.450 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:22.450 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745838+0000", 2024-09-16T20:49:22.450 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:22.450 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.450 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.450 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:22.450 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745838+0000", 2024-09-16T20:49:22.450 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6899999999999998e-07 2024-09-16T20:49:22.450 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.450 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.451 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:22.451 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745840+0000", 2024-09-16T20:49:22.451 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0660000000000002e-06 2024-09-16T20:49:22.451 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.451 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.451 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:22.451 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745847+0000", 2024-09-16T20:49:22.451 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8689999999999998e-06 2024-09-16T20:49:22.451 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.451 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.451 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:22.452 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745853+0000", 2024-09-16T20:49:22.452 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2399999999999998e-06 2024-09-16T20:49:22.452 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.452 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.452 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:22.452 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793381+0000", 2024-09-16T20:49:22.452 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047528005999999998 2024-09-16T20:49:22.452 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.452 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.452 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:22.452 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793423+0000", 2024-09-16T20:49:22.452 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2799000000000001e-05 2024-09-16T20:49:22.453 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.453 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.453 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:22.453 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793478+0000", 2024-09-16T20:49:22.453 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4756000000000002e-05 2024-09-16T20:49:22.453 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:22.453 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:22.453 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:22.453 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.453 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.453 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4320 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4683776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:22.454 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745859+0000", 2024-09-16T20:49:22.454 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 62.693129730999999, 2024-09-16T20:49:22.454 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047811399999999997, 2024-09-16T20:49:22.454 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:22.454 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:22.454 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:22.454 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:22.454 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:49:22.454 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:22.456 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4320 2024-09-16T20:49:22.456 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.456 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:22.456 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.456 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:22.456 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745859+0000", 2024-09-16T20:49:22.456 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:22.456 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.457 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.457 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:22.457 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745859+0000", 2024-09-16T20:49:22.457 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:22.457 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.457 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.457 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:22.457 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745860+0000", 2024-09-16T20:49:22.457 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9099999999999999e-07 2024-09-16T20:49:22.457 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.457 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.457 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:22.458 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745861+0000", 2024-09-16T20:49:22.458 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8589999999999999e-06 2024-09-16T20:49:22.458 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.458 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.458 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:22.458 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745868+0000", 2024-09-16T20:49:22.458 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2249999999999997e-06 2024-09-16T20:49:22.458 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.458 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.458 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:22.458 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745873+0000", 2024-09-16T20:49:22.458 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1290000000000004e-06 2024-09-16T20:49:22.459 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.459 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.459 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:22.459 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793522+0000", 2024-09-16T20:49:22.459 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047648847000000001 2024-09-16T20:49:22.459 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.459 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.459 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:22.459 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793603+0000", 2024-09-16T20:49:22.459 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1167000000000004e-05 2024-09-16T20:49:22.459 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.459 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.459 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:22.460 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793671+0000", 2024-09-16T20:49:22.460 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7781999999999996e-05 2024-09-16T20:49:22.460 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:22.460 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:22.460 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:22.460 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.460 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.460 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4321 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4691968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:22.460 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745879+0000", 2024-09-16T20:49:22.460 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 62.693109534999998, 2024-09-16T20:49:22.460 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047950375000000003, 2024-09-16T20:49:22.460 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:22.460 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:22.461 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:22.461 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:22.461 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:49:22.461 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:22.461 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4321 2024-09-16T20:49:22.461 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.461 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:22.461 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.461 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:22.461 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745879+0000", 2024-09-16T20:49:22.461 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:22.461 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.461 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.461 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:22.462 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745879+0000", 2024-09-16T20:49:22.462 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:22.462 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.462 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.462 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:22.462 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745880+0000", 2024-09-16T20:49:22.462 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.65e-07 2024-09-16T20:49:22.462 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.462 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.462 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:22.462 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745882+0000", 2024-09-16T20:49:22.462 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.922e-06 2024-09-16T20:49:22.462 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.462 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.462 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:22.463 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745888+0000", 2024-09-16T20:49:22.463 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0510000000000002e-06 2024-09-16T20:49:22.463 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.463 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.463 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:22.463 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745893+0000", 2024-09-16T20:49:22.463 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9579999999999998e-06 2024-09-16T20:49:22.463 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.463 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.463 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:22.463 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793704+0000", 2024-09-16T20:49:22.463 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047811179000000002 2024-09-16T20:49:22.463 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.464 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.464 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:22.464 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793759+0000", 2024-09-16T20:49:22.464 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5260999999999998e-05 2024-09-16T20:49:22.464 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.464 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.464 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:22.464 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793830+0000", 2024-09-16T20:49:22.464 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0638999999999999e-05 2024-09-16T20:49:22.464 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:22.464 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:22.464 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:22.465 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.465 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.465 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4322 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4700160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:22.465 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745899+0000", 2024-09-16T20:49:22.465 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 62.693090097000002, 2024-09-16T20:49:22.465 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048057280000000001, 2024-09-16T20:49:22.465 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:22.465 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:22.465 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:22.465 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:22.465 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:49:22.465 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:22.466 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4322 2024-09-16T20:49:22.466 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.466 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:22.466 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.466 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:22.466 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745899+0000", 2024-09-16T20:49:22.466 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:22.466 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.466 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.466 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:22.466 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745899+0000", 2024-09-16T20:49:22.467 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:22.467 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.467 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.467 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:22.467 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745899+0000", 2024-09-16T20:49:22.467 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9299999999999999e-07 2024-09-16T20:49:22.467 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.467 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.467 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:22.467 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745901+0000", 2024-09-16T20:49:22.467 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0030000000000001e-06 2024-09-16T20:49:22.467 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.468 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.468 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:22.468 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745908+0000", 2024-09-16T20:49:22.468 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3729999999999998e-06 2024-09-16T20:49:22.468 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.468 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.468 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:22.468 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745914+0000", 2024-09-16T20:49:22.468 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.122e-06 2024-09-16T20:49:22.468 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.468 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.468 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:22.469 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793862+0000", 2024-09-16T20:49:22.469 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047948324 2024-09-16T20:49:22.469 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.469 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.469 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:22.469 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793903+0000", 2024-09-16T20:49:22.469 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0465e-05 2024-09-16T20:49:22.469 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.469 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.469 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:22.469 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793956+0000", 2024-09-16T20:49:22.469 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3600000000000002e-05 2024-09-16T20:49:22.469 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:22.470 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:22.470 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:22.470 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.470 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.470 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4323 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4708352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:22.470 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745920+0000", 2024-09-16T20:49:22.470 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 62.693068455999999, 2024-09-16T20:49:22.470 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048268138000000002, 2024-09-16T20:49:22.470 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:22.470 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:22.470 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:22.470 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:22.470 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:49:22.470 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:22.470 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4323 2024-09-16T20:49:22.471 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.471 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:22.471 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.471 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:22.471 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745920+0000", 2024-09-16T20:49:22.471 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:22.471 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.471 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.471 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:22.471 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745920+0000", 2024-09-16T20:49:22.471 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:22.471 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.471 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.471 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:22.472 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745921+0000", 2024-09-16T20:49:22.472 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7e-07 2024-09-16T20:49:22.472 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.472 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.472 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:22.472 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745923+0000", 2024-09-16T20:49:22.472 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7880000000000001e-06 2024-09-16T20:49:22.472 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.472 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.472 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:22.472 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745929+0000", 2024-09-16T20:49:22.472 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0809999999999999e-06 2024-09-16T20:49:22.473 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.473 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.473 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:22.473 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745935+0000", 2024-09-16T20:49:22.473 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5370000000000003e-06 2024-09-16T20:49:22.473 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.473 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.473 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:22.473 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793987+0000", 2024-09-16T20:49:22.473 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048051736999999997 2024-09-16T20:49:22.473 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.473 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.473 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:22.473 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.794070+0000", 2024-09-16T20:49:22.474 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.3277999999999998e-05 2024-09-16T20:49:22.474 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.474 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.474 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:22.474 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.794189+0000", 2024-09-16T20:49:22.474 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000118347 2024-09-16T20:49:22.474 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:22.474 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:22.474 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:22.474 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.474 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.474 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4324 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4716544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:22.474 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745946+0000", 2024-09-16T20:49:22.474 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 62.693042884999997, 2024-09-16T20:49:22.474 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048468153999999999, 2024-09-16T20:49:22.475 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:22.475 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:22.475 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:22.475 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:22.475 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:49:22.475 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:22.475 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4324 2024-09-16T20:49:22.475 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.475 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:22.475 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.475 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:22.475 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745946+0000", 2024-09-16T20:49:22.475 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:22.475 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.475 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.476 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:22.476 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745946+0000", 2024-09-16T20:49:22.476 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:22.476 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.476 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.476 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:22.476 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745946+0000", 2024-09-16T20:49:22.476 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4499999999999997e-07 2024-09-16T20:49:22.476 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.476 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.476 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:22.476 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745949+0000", 2024-09-16T20:49:22.476 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2529999999999999e-06 2024-09-16T20:49:22.476 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.477 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.477 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:22.477 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745955+0000", 2024-09-16T20:49:22.477 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6470000000000001e-06 2024-09-16T20:49:22.477 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.477 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.477 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:22.477 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745960+0000", 2024-09-16T20:49:22.477 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7910000000000004e-06 2024-09-16T20:49:22.477 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.477 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.477 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:22.477 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.794220+0000", 2024-09-16T20:49:22.478 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048260069000000003 2024-09-16T20:49:22.478 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.478 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.478 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:22.478 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.794311+0000", 2024-09-16T20:49:22.478 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0593000000000005e-05 2024-09-16T20:49:22.478 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.478 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.478 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:22.478 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.794414+0000", 2024-09-16T20:49:22.478 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000103356 2024-09-16T20:49:22.478 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:22.478 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:22.478 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:22.479 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.479 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.479 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3571 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:22.479 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481059+0000", 2024-09-16T20:49:22.479 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.957929725, 2024-09-16T20:49:22.479 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049572995000000002, 2024-09-16T20:49:22.479 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:22.479 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:22.479 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:22.479 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:22.479 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:22.479 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:22.479 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3571 2024-09-16T20:49:22.479 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.480 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:22.480 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.480 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:22.480 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481059+0000", 2024-09-16T20:49:22.480 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:22.480 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.480 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.480 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:22.480 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481059+0000", 2024-09-16T20:49:22.480 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:22.480 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.480 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.480 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:22.480 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481060+0000", 2024-09-16T20:49:22.481 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2599999999999998e-07 2024-09-16T20:49:22.481 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.481 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.481 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:22.481 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481063+0000", 2024-09-16T20:49:22.481 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.999e-06 2024-09-16T20:49:22.481 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.481 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.481 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:22.481 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481067+0000", 2024-09-16T20:49:22.481 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3080000000000001e-06 2024-09-16T20:49:22.481 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.481 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.481 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:22.482 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481070+0000", 2024-09-16T20:49:22.482 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9919999999999999e-06 2024-09-16T20:49:22.482 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.482 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.482 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:22.482 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.522087+0000", 2024-09-16T20:49:22.482 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.041017023 2024-09-16T20:49:22.482 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.482 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.482 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:22.482 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.522191+0000", 2024-09-16T20:49:22.482 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010389499999999999 2024-09-16T20:49:22.482 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.483 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.483 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:22.483 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.522566+0000", 2024-09-16T20:49:22.483 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00037539399999999998 2024-09-16T20:49:22.483 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.483 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.483 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:22.483 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525494+0000", 2024-09-16T20:49:22.483 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0029273070000000001 2024-09-16T20:49:22.483 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.483 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.483 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:22.483 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530355+0000", 2024-09-16T20:49:22.483 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0048612509999999996 2024-09-16T20:49:22.483 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.484 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.484 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:22.484 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530407+0000", 2024-09-16T20:49:22.484 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1817000000000002e-05 2024-09-16T20:49:22.484 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.484 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.484 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:22.484 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530632+0000", 2024-09-16T20:49:22.484 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00022558300000000001 2024-09-16T20:49:22.484 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:22.484 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:22.484 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:22.484 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.484 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.485 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3572 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:22.485 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481220+0000", 2024-09-16T20:49:22.485 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.957769157000001, 2024-09-16T20:49:22.485 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050452862000000001, 2024-09-16T20:49:22.485 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:22.485 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:22.485 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:22.485 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:22.485 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:22.485 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:22.485 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3572 2024-09-16T20:49:22.485 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.485 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:22.485 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.486 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:22.486 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481220+0000", 2024-09-16T20:49:22.486 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:22.486 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.486 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.486 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:22.486 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481220+0000", 2024-09-16T20:49:22.486 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:22.486 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.486 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.486 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:22.486 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481221+0000", 2024-09-16T20:49:22.486 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9400000000000004e-07 2024-09-16T20:49:22.487 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.487 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.487 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:22.487 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481225+0000", 2024-09-16T20:49:22.487 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0230000000000003e-06 2024-09-16T20:49:22.487 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.487 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.487 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:22.487 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481234+0000", 2024-09-16T20:49:22.487 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2920000000000001e-06 2024-09-16T20:49:22.487 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.487 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.487 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:22.487 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481240+0000", 2024-09-16T20:49:22.488 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9390000000000002e-06 2024-09-16T20:49:22.488 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.488 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.488 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:22.488 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.523544+0000", 2024-09-16T20:49:22.488 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.042304475000000001 2024-09-16T20:49:22.488 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.488 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.488 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:22.488 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.523624+0000", 2024-09-16T20:49:22.488 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.975e-05 2024-09-16T20:49:22.488 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.488 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.489 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:22.489 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.523975+0000", 2024-09-16T20:49:22.489 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00035063200000000003 2024-09-16T20:49:22.489 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.489 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.489 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:22.489 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530337+0000", 2024-09-16T20:49:22.489 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0063623589999999997 2024-09-16T20:49:22.489 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.489 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.489 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:22.489 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531592+0000", 2024-09-16T20:49:22.490 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0012550599999999999 2024-09-16T20:49:22.490 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.490 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.490 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:22.490 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531633+0000", 2024-09-16T20:49:22.490 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1192000000000001e-05 2024-09-16T20:49:22.490 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.490 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.490 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:22.490 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531673+0000", 2024-09-16T20:49:22.490 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9345999999999999e-05 2024-09-16T20:49:22.490 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:22.490 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:22.490 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:22.491 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.491 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.491 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3573 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:22.491 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481257+0000", 2024-09-16T20:49:22.491 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.957732210000003, 2024-09-16T20:49:22.491 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050511027, 2024-09-16T20:49:22.491 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:22.491 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:22.491 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:22.492 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:22.492 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:22.492 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:22.492 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3573 2024-09-16T20:49:22.492 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.492 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:22.492 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.492 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:22.492 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481257+0000", 2024-09-16T20:49:22.492 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:22.492 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.492 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.493 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:22.493 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481257+0000", 2024-09-16T20:49:22.493 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:22.493 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.493 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.493 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:22.493 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481257+0000", 2024-09-16T20:49:22.493 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.99e-07 2024-09-16T20:49:22.493 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.493 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.494 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:22.494 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481260+0000", 2024-09-16T20:49:22.494 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1709999999999998e-06 2024-09-16T20:49:22.494 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.494 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.494 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:22.494 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481271+0000", 2024-09-16T20:49:22.494 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0795000000000001e-05 2024-09-16T20:49:22.494 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.494 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.494 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:22.494 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481277+0000", 2024-09-16T20:49:22.494 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1670000000000004e-06 2024-09-16T20:49:22.495 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.495 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.495 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:22.495 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.524647+0000", 2024-09-16T20:49:22.495 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.043369810000000002 2024-09-16T20:49:22.495 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.495 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.495 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:22.495 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.524725+0000", 2024-09-16T20:49:22.495 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7286000000000005e-05 2024-09-16T20:49:22.495 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.495 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.495 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:22.496 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525062+0000", 2024-09-16T20:49:22.496 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00033739499999999999 2024-09-16T20:49:22.496 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.496 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.496 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:22.496 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530692+0000", 2024-09-16T20:49:22.496 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0056301479999999997 2024-09-16T20:49:22.496 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.496 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.496 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:22.496 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531694+0000", 2024-09-16T20:49:22.496 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0010017419999999999 2024-09-16T20:49:22.496 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.496 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.497 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:22.497 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531737+0000", 2024-09-16T20:49:22.497 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2762999999999997e-05 2024-09-16T20:49:22.497 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.497 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.497 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:22.497 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531768+0000", 2024-09-16T20:49:22.497 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1151e-05 2024-09-16T20:49:22.497 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:22.497 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:22.497 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:22.497 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.497 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.498 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3574 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:22.498 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:49:22.498 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.957699253999998, 2024-09-16T20:49:22.498 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053579441999999998, 2024-09-16T20:49:22.498 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:22.498 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:22.498 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:22.498 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:22.498 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:22.498 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:22.498 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3574 2024-09-16T20:49:22.498 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.498 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:22.498 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.499 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:22.499 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:49:22.499 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:22.499 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.499 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.499 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:22.499 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:49:22.499 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:22.499 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.499 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.499 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:22.499 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:49:22.499 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2e-07 2024-09-16T20:49:22.500 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.500 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.500 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:22.500 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481292+0000", 2024-09-16T20:49:22.500 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2630000000000002e-06 2024-09-16T20:49:22.500 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.500 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.500 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:22.500 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481297+0000", 2024-09-16T20:49:22.500 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.296e-06 2024-09-16T20:49:22.500 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.500 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.500 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:22.500 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481300+0000", 2024-09-16T20:49:22.500 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7019999999999999e-06 2024-09-16T20:49:22.501 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.501 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.501 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:22.501 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525533+0000", 2024-09-16T20:49:22.501 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.044232318 2024-09-16T20:49:22.501 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.501 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.501 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:22.501 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525619+0000", 2024-09-16T20:49:22.501 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5892999999999999e-05 2024-09-16T20:49:22.501 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.501 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.501 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:22.501 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525987+0000", 2024-09-16T20:49:22.502 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00036836099999999999 2024-09-16T20:49:22.502 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.502 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.502 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:22.502 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531779+0000", 2024-09-16T20:49:22.502 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0057918240000000001 2024-09-16T20:49:22.502 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.502 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.502 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:22.502 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534747+0000", 2024-09-16T20:49:22.502 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0029681600000000001 2024-09-16T20:49:22.502 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.502 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.503 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:22.503 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534832+0000", 2024-09-16T20:49:22.503 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5180999999999996e-05 2024-09-16T20:49:22.503 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.503 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.503 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:22.503 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534869+0000", 2024-09-16T20:49:22.503 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7024000000000003e-05 2024-09-16T20:49:22.503 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:22.503 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:22.503 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:22.503 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.503 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.503 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3575 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:22.503 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:49:22.504 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.957684514, 2024-09-16T20:49:22.504 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053801463000000001, 2024-09-16T20:49:22.504 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:22.504 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:22.504 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:22.504 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:22.504 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:22.504 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:22.504 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3575 2024-09-16T20:49:22.504 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.505 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:22.505 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.505 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:22.505 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:49:22.505 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:22.505 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.505 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.505 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:22.505 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:49:22.505 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:22.506 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.506 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.506 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:22.506 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481305+0000", 2024-09-16T20:49:22.506 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5400000000000002e-07 2024-09-16T20:49:22.506 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.506 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.506 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:22.506 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481307+0000", 2024-09-16T20:49:22.506 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7800000000000001e-06 2024-09-16T20:49:22.506 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.506 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.507 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:22.507 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481312+0000", 2024-09-16T20:49:22.507 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6349999999999997e-06 2024-09-16T20:49:22.507 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.507 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.507 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:22.507 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481315+0000", 2024-09-16T20:49:22.507 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7829999999999999e-06 2024-09-16T20:49:22.507 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.507 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.508 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:22.508 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.526876+0000", 2024-09-16T20:49:22.508 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.045561070000000002 2024-09-16T20:49:22.508 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.508 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.508 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:22.508 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.526954+0000", 2024-09-16T20:49:22.508 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8108000000000005e-05 2024-09-16T20:49:22.508 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.509 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.509 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:22.509 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527282+0000", 2024-09-16T20:49:22.509 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000327483 2024-09-16T20:49:22.509 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.509 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.509 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:22.509 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531784+0000", 2024-09-16T20:49:22.509 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0045020169999999997 2024-09-16T20:49:22.509 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.509 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.509 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:22.510 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534921+0000", 2024-09-16T20:49:22.510 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.003137791 2024-09-16T20:49:22.510 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.510 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.510 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:22.510 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534976+0000", 2024-09-16T20:49:22.510 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4953000000000001e-05 2024-09-16T20:49:22.510 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.510 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.510 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:22.510 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535106+0000", 2024-09-16T20:49:22.510 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000129589 2024-09-16T20:49:22.510 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:22.511 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:22.511 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:22.511 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.511 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.511 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3576 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:22.511 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:49:22.511 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.957670602, 2024-09-16T20:49:22.511 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053955436000000002, 2024-09-16T20:49:22.511 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:22.511 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:22.511 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:22.511 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:22.512 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:22.512 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:22.512 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3576 2024-09-16T20:49:22.512 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.512 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:22.512 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.512 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:22.512 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:49:22.512 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:22.512 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.512 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.512 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:22.512 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:49:22.513 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:22.513 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.513 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.513 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:22.513 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481319+0000", 2024-09-16T20:49:22.513 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.53e-07 2024-09-16T20:49:22.513 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.513 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.513 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:22.513 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481320+0000", 2024-09-16T20:49:22.513 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5230000000000001e-06 2024-09-16T20:49:22.513 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.514 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.514 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:22.514 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481330+0000", 2024-09-16T20:49:22.514 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0190000000000001e-05 2024-09-16T20:49:22.514 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.514 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.514 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:22.514 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481333+0000", 2024-09-16T20:49:22.514 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.151e-06 2024-09-16T20:49:22.514 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.514 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.514 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:22.515 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527689+0000", 2024-09-16T20:49:22.515 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046355192000000003 2024-09-16T20:49:22.515 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.515 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.515 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:22.515 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527764+0000", 2024-09-16T20:49:22.515 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5776000000000001e-05 2024-09-16T20:49:22.515 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.515 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.515 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:22.515 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528141+0000", 2024-09-16T20:49:22.515 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00037647599999999999 2024-09-16T20:49:22.515 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.516 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.516 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:22.516 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531788+0000", 2024-09-16T20:49:22.516 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0036470679999999998 2024-09-16T20:49:22.516 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.516 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.516 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:22.516 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535174+0000", 2024-09-16T20:49:22.516 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0033862670000000001 2024-09-16T20:49:22.516 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.516 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.516 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:22.517 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535236+0000", 2024-09-16T20:49:22.517 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1797999999999993e-05 2024-09-16T20:49:22.517 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.517 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.517 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:22.517 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535274+0000", 2024-09-16T20:49:22.517 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7741999999999999e-05 2024-09-16T20:49:22.517 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:22.517 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:22.517 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:22.517 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.517 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.517 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3577 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:22.518 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:49:22.518 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.957649001999997, 2024-09-16T20:49:22.518 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054073807000000002, 2024-09-16T20:49:22.518 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:22.518 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:22.518 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:22.518 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:22.518 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:22.518 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:22.518 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3577 2024-09-16T20:49:22.518 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.518 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:22.519 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.519 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:22.519 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:49:22.519 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:22.519 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.519 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.519 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:22.519 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:49:22.519 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:22.519 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.519 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.519 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:22.519 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:49:22.520 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.96e-07 2024-09-16T20:49:22.520 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.520 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.520 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:22.520 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481342+0000", 2024-09-16T20:49:22.520 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.714e-06 2024-09-16T20:49:22.520 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.520 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.520 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:22.520 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481349+0000", 2024-09-16T20:49:22.520 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4930000000000001e-06 2024-09-16T20:49:22.520 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.520 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.521 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:22.521 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481353+0000", 2024-09-16T20:49:22.521 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4879999999999999e-06 2024-09-16T20:49:22.521 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.521 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.521 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:22.521 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528619+0000", 2024-09-16T20:49:22.521 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047266351999999998 2024-09-16T20:49:22.521 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.521 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.521 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:22.521 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528706+0000", 2024-09-16T20:49:22.522 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6796e-05 2024-09-16T20:49:22.522 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.522 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.522 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:22.522 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529148+0000", 2024-09-16T20:49:22.522 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00044182999999999999 2024-09-16T20:49:22.522 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.522 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.522 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:22.522 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531792+0000", 2024-09-16T20:49:22.522 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0026443119999999998 2024-09-16T20:49:22.522 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.522 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.522 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:22.523 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535319+0000", 2024-09-16T20:49:22.523 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0035266439999999998 2024-09-16T20:49:22.523 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.523 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.523 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:22.523 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535372+0000", 2024-09-16T20:49:22.523 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3207000000000001e-05 2024-09-16T20:49:22.523 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.523 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.523 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:22.524 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535414+0000", 2024-09-16T20:49:22.524 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1675000000000003e-05 2024-09-16T20:49:22.524 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:22.524 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:22.524 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:22.524 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.524 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.524 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3578 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:22.524 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:49:22.524 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.957627832999997, 2024-09-16T20:49:22.524 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054189586999999997, 2024-09-16T20:49:22.524 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:22.524 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:22.525 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:22.525 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:22.525 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:22.525 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:22.525 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3578 2024-09-16T20:49:22.525 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.525 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:22.525 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.525 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:22.525 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:49:22.525 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:22.525 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.525 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.526 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:22.526 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:49:22.526 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:22.526 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.526 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.526 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:22.526 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:49:22.526 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0800000000000001e-07 2024-09-16T20:49:22.526 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.526 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.526 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:22.526 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481363+0000", 2024-09-16T20:49:22.526 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0339999999999999e-06 2024-09-16T20:49:22.527 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.527 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.527 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:22.527 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481371+0000", 2024-09-16T20:49:22.527 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4440000000000002e-06 2024-09-16T20:49:22.527 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.527 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.527 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:22.527 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481374+0000", 2024-09-16T20:49:22.527 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2049999999999998e-06 2024-09-16T20:49:22.527 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.527 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.527 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:22.527 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529555+0000", 2024-09-16T20:49:22.528 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048181202999999999 2024-09-16T20:49:22.528 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.528 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.528 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:22.528 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529616+0000", 2024-09-16T20:49:22.528 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0912000000000002e-05 2024-09-16T20:49:22.528 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.528 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.528 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:22.528 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529939+0000", 2024-09-16T20:49:22.528 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00032269800000000001 2024-09-16T20:49:22.528 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.529 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.529 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:22.529 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532602+0000", 2024-09-16T20:49:22.529 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0026634940000000002 2024-09-16T20:49:22.529 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.529 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.529 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:22.529 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535465+0000", 2024-09-16T20:49:22.529 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.002862149 2024-09-16T20:49:22.529 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.529 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.529 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:22.529 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535516+0000", 2024-09-16T20:49:22.530 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1696000000000001e-05 2024-09-16T20:49:22.530 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.530 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.530 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:22.530 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535551+0000", 2024-09-16T20:49:22.530 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4443999999999998e-05 2024-09-16T20:49:22.530 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:22.530 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:22.530 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:22.530 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.530 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.530 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3579 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:22.530 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:49:22.530 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.957609394000002, 2024-09-16T20:49:22.531 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054295151, 2024-09-16T20:49:22.531 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:22.531 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:22.531 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:22.531 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:22.531 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:22.531 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:22.531 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3579 2024-09-16T20:49:22.531 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.531 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:22.531 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.531 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:22.532 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:49:22.532 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:22.532 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.532 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.532 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:22.532 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:49:22.532 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:22.532 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.532 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.532 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:22.533 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:49:22.533 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3300000000000001e-07 2024-09-16T20:49:22.533 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.533 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.533 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:22.533 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481381+0000", 2024-09-16T20:49:22.533 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.649e-06 2024-09-16T20:49:22.533 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.533 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.533 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:22.533 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481387+0000", 2024-09-16T20:49:22.534 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4519999999999996e-06 2024-09-16T20:49:22.534 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.534 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.534 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:22.534 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481390+0000", 2024-09-16T20:49:22.534 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0170000000000001e-06 2024-09-16T20:49:22.534 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.534 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.534 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:22.534 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530756+0000", 2024-09-16T20:49:22.534 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049365882999999999 2024-09-16T20:49:22.534 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.534 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.534 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:22.535 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530845+0000", 2024-09-16T20:49:22.535 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9197000000000007e-05 2024-09-16T20:49:22.535 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.535 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.535 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:22.535 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531191+0000", 2024-09-16T20:49:22.535 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00034562500000000001 2024-09-16T20:49:22.535 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.535 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.535 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:22.535 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534078+0000", 2024-09-16T20:49:22.535 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0028878179999999999 2024-09-16T20:49:22.535 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.535 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.535 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:22.536 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535589+0000", 2024-09-16T20:49:22.536 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00151085 2024-09-16T20:49:22.536 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.536 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.536 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:22.536 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535639+0000", 2024-09-16T20:49:22.536 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9697000000000002e-05 2024-09-16T20:49:22.536 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.536 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.536 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:22.536 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535675+0000", 2024-09-16T20:49:22.537 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5729999999999998e-05 2024-09-16T20:49:22.537 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:22.537 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:22.537 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:22.537 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.537 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.537 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3580 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:22.537 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:49:22.537 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.957595761999997, 2024-09-16T20:49:22.537 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054397361999999998, 2024-09-16T20:49:22.537 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:22.537 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:22.538 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:22.538 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:22.538 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:22.538 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:22.538 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3580 2024-09-16T20:49:22.538 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.538 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:22.538 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.538 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:22.538 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:49:22.538 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:22.538 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.538 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.539 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:22.539 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:49:22.539 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:22.539 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.539 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.539 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:22.539 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:49:22.539 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2600000000000001e-07 2024-09-16T20:49:22.539 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.539 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.539 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:22.539 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481395+0000", 2024-09-16T20:49:22.539 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.544e-06 2024-09-16T20:49:22.539 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.540 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.540 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:22.540 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481404+0000", 2024-09-16T20:49:22.540 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8850000000000005e-06 2024-09-16T20:49:22.540 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.540 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.540 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:22.540 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481407+0000", 2024-09-16T20:49:22.540 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1540000000000002e-06 2024-09-16T20:49:22.540 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.540 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.540 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:22.540 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531823+0000", 2024-09-16T20:49:22.541 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050415637999999999 2024-09-16T20:49:22.541 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.541 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.541 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:22.541 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531895+0000", 2024-09-16T20:49:22.541 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2179000000000001e-05 2024-09-16T20:49:22.541 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.541 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.541 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:22.541 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532236+0000", 2024-09-16T20:49:22.541 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00034073799999999999 2024-09-16T20:49:22.541 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.542 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.542 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:22.542 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534083+0000", 2024-09-16T20:49:22.542 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001847298 2024-09-16T20:49:22.542 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.542 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.542 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:22.542 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535714+0000", 2024-09-16T20:49:22.542 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0016316810000000001 2024-09-16T20:49:22.542 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.542 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.542 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:22.542 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535759+0000", 2024-09-16T20:49:22.543 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4478999999999999e-05 2024-09-16T20:49:22.543 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.543 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.543 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:22.543 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535791+0000", 2024-09-16T20:49:22.543 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.154e-05 2024-09-16T20:49:22.543 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:22.543 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:22.543 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:22.543 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.543 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.543 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3581 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:22.543 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:49:22.544 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.957573764999999, 2024-09-16T20:49:22.544 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.057094403000000002, 2024-09-16T20:49:22.544 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:22.544 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:22.544 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:22.544 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:22.544 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:22.544 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:22.544 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3581 2024-09-16T20:49:22.544 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.544 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:22.544 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.544 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:22.545 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:49:22.545 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:22.545 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.545 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.545 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:22.545 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:49:22.545 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:22.545 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.545 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.545 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:22.545 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:49:22.545 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8799999999999998e-07 2024-09-16T20:49:22.545 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.546 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.546 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:22.546 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481418+0000", 2024-09-16T20:49:22.546 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1859999999999999e-06 2024-09-16T20:49:22.546 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.546 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.546 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:22.546 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481423+0000", 2024-09-16T20:49:22.546 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8279999999999999e-06 2024-09-16T20:49:22.546 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.546 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.546 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:22.546 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481426+0000", 2024-09-16T20:49:22.546 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0110000000000001e-06 2024-09-16T20:49:22.547 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.547 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.547 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:22.547 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532631+0000", 2024-09-16T20:49:22.547 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051204514999999999 2024-09-16T20:49:22.547 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.547 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.547 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:22.547 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.533381+0000", 2024-09-16T20:49:22.547 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00075029000000000003 2024-09-16T20:49:22.547 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.547 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.547 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:22.547 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.533739+0000", 2024-09-16T20:49:22.548 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00035773100000000002 2024-09-16T20:49:22.548 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.548 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.548 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:22.548 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535123+0000", 2024-09-16T20:49:22.548 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0013836160000000001 2024-09-16T20:49:22.548 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.548 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.548 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:22.548 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538413+0000", 2024-09-16T20:49:22.548 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0032902510000000001 2024-09-16T20:49:22.548 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.548 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.549 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:22.549 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538470+0000", 2024-09-16T20:49:22.549 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6697999999999998e-05 2024-09-16T20:49:22.549 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.549 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.549 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:22.549 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538510+0000", 2024-09-16T20:49:22.549 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9988999999999999e-05 2024-09-16T20:49:22.549 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:22.549 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:22.549 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:22.549 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.549 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.550 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:12732 3.d 3:b752444a:::rbd_header.110e747f0779:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-16T20:49:22.550 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:49:22.550 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 17.670501186999999, 2024-09-16T20:49:22.550 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.065302467000000003, 2024-09-16T20:49:22.550 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:22.550 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:22.550 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:22.550 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:22.550 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:49:22.550 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:49:22.550 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 12732 2024-09-16T20:49:22.550 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.550 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:22.550 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.551 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:22.551 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:49:22.551 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:22.551 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.551 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.551 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:22.551 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:49:22.551 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:22.551 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.551 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.551 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:22.551 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768490+0000", 2024-09-16T20:49:22.551 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2859999999999998e-06 2024-09-16T20:49:22.552 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.552 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.552 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:22.552 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768501+0000", 2024-09-16T20:49:22.552 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1226999999999999e-05 2024-09-16T20:49:22.552 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.552 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.552 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:22.552 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768517+0000", 2024-09-16T20:49:22.552 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.592e-05 2024-09-16T20:49:22.552 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.552 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.552 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:22.553 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768528+0000", 2024-09-16T20:49:22.553 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0689999999999999e-05 2024-09-16T20:49:22.553 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.553 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.553 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:22.553 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768584+0000", 2024-09-16T20:49:22.553 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5889e-05 2024-09-16T20:49:22.553 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.553 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.553 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:22.553 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768720+0000", 2024-09-16T20:49:22.553 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000136718 2024-09-16T20:49:22.553 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.553 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.554 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:22.554 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.769066+0000", 2024-09-16T20:49:22.554 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000346013 2024-09-16T20:49:22.554 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.554 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.554 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:22.554 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833659+0000", 2024-09-16T20:49:22.554 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.064592822999999994 2024-09-16T20:49:22.554 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.554 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.554 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:22.554 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833719+0000", 2024-09-16T20:49:22.554 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9441000000000002e-05 2024-09-16T20:49:22.555 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.555 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.555 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:22.555 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833758+0000", 2024-09-16T20:49:22.555 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9122000000000003e-05 2024-09-16T20:49:22.555 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.555 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.555 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:22.555 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833790+0000", 2024-09-16T20:49:22.555 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2338000000000001e-05 2024-09-16T20:49:22.555 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:22.555 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:22.555 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:22.555 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.556 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.556 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.5089.0:12732 3.d e73/23 ondisk, result = 0)", 2024-09-16T20:49:22.556 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:49:22.556 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 17.668899017000001, 2024-09-16T20:49:22.556 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.063706456999999994, 2024-09-16T20:49:22.556 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:22.556 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:22.556 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:22.556 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:22.556 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.556 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:22.557 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:49:22.557 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:22.557 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.557 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.557 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:22.557 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:49:22.557 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:22.557 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.557 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.557 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:22.557 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770091+0000", 2024-09-16T20:49:22.557 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.53e-06 2024-09-16T20:49:22.558 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.558 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.558 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:22.558 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770104+0000", 2024-09-16T20:49:22.558 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2203e-05 2024-09-16T20:49:22.558 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.558 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.558 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:22.558 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833637+0000", 2024-09-16T20:49:22.558 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.063533201999999997 2024-09-16T20:49:22.558 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.559 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.559 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:22.559 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833655+0000", 2024-09-16T20:49:22.559 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8369e-05 2024-09-16T20:49:22.559 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.559 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.559 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:22.559 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833707+0000", 2024-09-16T20:49:22.559 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2061999999999997e-05 2024-09-16T20:49:22.559 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.559 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.559 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:22.560 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833715+0000", 2024-09-16T20:49:22.560 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5179999999999998e-06 2024-09-16T20:49:22.560 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.560 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.560 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:22.560 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833796+0000", 2024-09-16T20:49:22.560 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1573e-05 2024-09-16T20:49:22.560 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:22.560 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:22.560 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:22.560 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:22.560 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:22.560 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:22.562 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:49:22.563 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:22.561+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:22.710 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:22.710 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:49:22.710 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:49:22.710 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:22.718 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:49:22.755 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:22.661+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:22.763 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:22.762+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:22.827 INFO:teuthology.orchestra.run.smithi029.stderr:nodeep-scrub is set 2024-09-16T20:49:22.828 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:22.828 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:49:22.828 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:49:22.828 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:49:22.828 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:22.839 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:49:22.863 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:22.863+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:22.964 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:22.964+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:22.995 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:22.995 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:49:22.995 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:49:22.995 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:49:22.996 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.996 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3709 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1464320~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:22.996 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.638971+0000", 2024-09-16T20:49:22.996 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 60.354587168999998, 2024-09-16T20:49:22.996 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049858670000000001, 2024-09-16T20:49:22.996 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:22.996 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:22.996 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:22.996 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:22.996 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:22.996 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:22.996 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3709 2024-09-16T20:49:22.996 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.997 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:22.997 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.997 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:22.997 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638971+0000", 2024-09-16T20:49:22.997 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:22.997 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.997 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.997 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:22.997 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638971+0000", 2024-09-16T20:49:22.997 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:22.997 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.997 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.997 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:22.998 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638972+0000", 2024-09-16T20:49:22.998 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0400000000000002e-07 2024-09-16T20:49:22.998 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.998 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.998 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:22.998 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638973+0000", 2024-09-16T20:49:22.998 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.578e-06 2024-09-16T20:49:22.998 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.998 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.998 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:22.998 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638979+0000", 2024-09-16T20:49:22.998 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.502e-06 2024-09-16T20:49:22.998 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.999 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.999 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:22.999 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638983+0000", 2024-09-16T20:49:22.999 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2690000000000001e-06 2024-09-16T20:49:22.999 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.999 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.999 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:22.999 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688628+0000", 2024-09-16T20:49:22.999 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049644547999999997 2024-09-16T20:49:22.999 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:22.999 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:22.999 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:22.999 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688718+0000", 2024-09-16T20:49:22.999 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0802000000000002e-05 2024-09-16T20:49:23.000 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.000 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.000 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:23.000 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688830+0000", 2024-09-16T20:49:23.000 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000111667 2024-09-16T20:49:23.000 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:23.000 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:23.000 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:23.000 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.000 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.000 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3712 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1472512~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:23.000 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:49:23.000 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 60.354570641000002, 2024-09-16T20:49:23.001 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050004305999999998, 2024-09-16T20:49:23.001 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:23.001 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:23.001 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:23.001 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:23.001 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:23.001 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:23.001 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3712 2024-09-16T20:49:23.001 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.001 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:23.001 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.001 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:23.001 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:49:23.001 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:23.002 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.002 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.002 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:23.002 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:49:23.002 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:23.002 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.002 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.002 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:23.002 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:49:23.002 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0199999999999998e-07 2024-09-16T20:49:23.002 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.002 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.002 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:23.002 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638990+0000", 2024-09-16T20:49:23.003 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.618e-06 2024-09-16T20:49:23.003 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.003 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.003 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:23.003 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638995+0000", 2024-09-16T20:49:23.003 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3940000000000004e-06 2024-09-16T20:49:23.003 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.003 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.003 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:23.003 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639000+0000", 2024-09-16T20:49:23.003 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9309999999999999e-06 2024-09-16T20:49:23.003 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.003 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.003 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:23.004 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688867+0000", 2024-09-16T20:49:23.004 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049866955999999997 2024-09-16T20:49:23.004 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.004 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.004 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:23.004 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688933+0000", 2024-09-16T20:49:23.004 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5359999999999998e-05 2024-09-16T20:49:23.004 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.004 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.004 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:23.004 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688992+0000", 2024-09-16T20:49:23.004 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9744999999999998e-05 2024-09-16T20:49:23.004 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:23.004 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:23.005 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:23.005 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.005 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.005 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3714 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1480704~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:23.005 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:49:23.005 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 60.354553051000003, 2024-09-16T20:49:23.005 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050152552000000003, 2024-09-16T20:49:23.005 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:23.005 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:23.005 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:23.005 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:23.005 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:23.005 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:23.006 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3714 2024-09-16T20:49:23.006 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.006 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:23.006 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.006 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:23.006 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:49:23.006 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:23.006 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.006 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.006 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:23.006 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:49:23.006 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:23.006 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.007 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.007 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:23.007 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:49:23.007 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0499999999999999e-07 2024-09-16T20:49:23.007 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.007 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.007 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:23.007 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639007+0000", 2024-09-16T20:49:23.007 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5340000000000001e-06 2024-09-16T20:49:23.007 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.007 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.007 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:23.007 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639013+0000", 2024-09-16T20:49:23.007 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3260000000000003e-06 2024-09-16T20:49:23.008 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.008 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.008 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:23.008 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639018+0000", 2024-09-16T20:49:23.008 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2850000000000002e-06 2024-09-16T20:49:23.008 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.008 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.008 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:23.008 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689026+0000", 2024-09-16T20:49:23.008 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.05000798 2024-09-16T20:49:23.008 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.008 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.008 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:23.009 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689077+0000", 2024-09-16T20:49:23.009 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1063999999999998e-05 2024-09-16T20:49:23.009 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.009 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.009 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:23.009 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689158+0000", 2024-09-16T20:49:23.009 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1057999999999995e-05 2024-09-16T20:49:23.009 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:23.009 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:23.009 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:23.009 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.009 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.009 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3716 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1488896~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:23.010 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:49:23.010 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 60.354535241000001, 2024-09-16T20:49:23.010 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050280142, 2024-09-16T20:49:23.010 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:23.010 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:23.010 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:23.010 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:23.010 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:23.010 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:23.010 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3716 2024-09-16T20:49:23.010 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.010 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:23.010 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.010 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:23.011 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:49:23.011 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:23.011 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.011 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.011 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:23.011 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:49:23.011 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:23.011 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.011 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.011 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:23.011 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639024+0000", 2024-09-16T20:49:23.011 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9999999999999997e-07 2024-09-16T20:49:23.011 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.011 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.012 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:23.012 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639026+0000", 2024-09-16T20:49:23.012 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.781e-06 2024-09-16T20:49:23.012 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.012 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.012 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:23.012 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639032+0000", 2024-09-16T20:49:23.012 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7780000000000004e-06 2024-09-16T20:49:23.012 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.012 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.012 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:23.012 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639036+0000", 2024-09-16T20:49:23.012 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6909999999999996e-06 2024-09-16T20:49:23.013 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.013 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.013 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:23.013 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689191+0000", 2024-09-16T20:49:23.013 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050155284000000001 2024-09-16T20:49:23.013 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.013 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.013 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:23.013 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689242+0000", 2024-09-16T20:49:23.013 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0662999999999999e-05 2024-09-16T20:49:23.013 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.013 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.013 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:23.013 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689304+0000", 2024-09-16T20:49:23.014 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1345000000000003e-05 2024-09-16T20:49:23.014 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:23.014 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:23.014 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:23.014 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.014 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.014 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3719 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1497088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:23.014 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:49:23.014 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 60.354507963000003, 2024-09-16T20:49:23.014 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050399715999999997, 2024-09-16T20:49:23.014 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:23.014 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:23.014 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:23.014 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:23.015 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:23.015 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:23.015 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3719 2024-09-16T20:49:23.015 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.015 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:23.015 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.015 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:23.015 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:49:23.015 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:23.015 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.015 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.015 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:23.015 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:49:23.015 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:23.016 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.016 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.016 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:23.016 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:49:23.016 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9700000000000002e-07 2024-09-16T20:49:23.016 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.016 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.016 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:23.016 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639053+0000", 2024-09-16T20:49:23.016 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2900000000000001e-06 2024-09-16T20:49:23.016 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.016 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.016 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:23.017 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639060+0000", 2024-09-16T20:49:23.017 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7750000000000004e-06 2024-09-16T20:49:23.017 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.017 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.017 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:23.017 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639066+0000", 2024-09-16T20:49:23.017 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0639999999999999e-06 2024-09-16T20:49:23.017 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.017 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.017 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:23.018 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689337+0000", 2024-09-16T20:49:23.018 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050270784999999998 2024-09-16T20:49:23.018 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.018 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.018 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:23.018 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689392+0000", 2024-09-16T20:49:23.018 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4920999999999998e-05 2024-09-16T20:49:23.018 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.018 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.018 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:23.018 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689450+0000", 2024-09-16T20:49:23.018 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8483999999999998e-05 2024-09-16T20:49:23.018 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:23.018 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:23.019 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:23.019 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.019 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.019 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3721 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1505280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:23.019 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:49:23.019 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 60.354484831000001, 2024-09-16T20:49:23.019 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050513608000000002, 2024-09-16T20:49:23.019 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:23.019 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:23.019 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:23.019 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:23.019 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:23.019 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:23.019 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3721 2024-09-16T20:49:23.020 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.020 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:23.020 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.020 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:23.020 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:49:23.020 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:23.020 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.020 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.020 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:23.020 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:49:23.020 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:23.020 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.020 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.021 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:23.021 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:49:23.021 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7e-07 2024-09-16T20:49:23.021 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.021 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.021 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:23.021 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639076+0000", 2024-09-16T20:49:23.021 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8899999999999999e-06 2024-09-16T20:49:23.021 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.021 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.021 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:23.021 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639083+0000", 2024-09-16T20:49:23.021 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1210000000000001e-06 2024-09-16T20:49:23.021 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.022 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.022 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:23.022 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639089+0000", 2024-09-16T20:49:23.022 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.643e-06 2024-09-16T20:49:23.022 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.022 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.022 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:23.022 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689474+0000", 2024-09-16T20:49:23.022 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050385679000000003 2024-09-16T20:49:23.022 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.022 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.022 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:23.022 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689533+0000", 2024-09-16T20:49:23.022 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8303000000000001e-05 2024-09-16T20:49:23.023 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.023 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.023 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:23.023 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689587+0000", 2024-09-16T20:49:23.023 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4601999999999997e-05 2024-09-16T20:49:23.023 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:23.023 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:23.023 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:23.023 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.023 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.023 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3723 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1513472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:23.023 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:49:23.023 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 60.354454027000003, 2024-09-16T20:49:23.023 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050699691999999998, 2024-09-16T20:49:23.024 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:23.024 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:23.024 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:23.024 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:23.024 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:23.024 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:23.024 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3723 2024-09-16T20:49:23.024 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.024 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:23.024 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.024 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:23.024 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:49:23.025 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:23.025 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.025 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.025 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:23.025 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:49:23.025 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:23.025 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.025 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.025 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:23.025 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:49:23.025 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3399999999999999e-07 2024-09-16T20:49:23.025 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.025 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.025 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:23.026 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639107+0000", 2024-09-16T20:49:23.026 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3860000000000001e-06 2024-09-16T20:49:23.026 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.026 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.026 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:23.026 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639114+0000", 2024-09-16T20:49:23.026 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9800000000000001e-06 2024-09-16T20:49:23.026 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.026 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.026 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:23.026 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639129+0000", 2024-09-16T20:49:23.026 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4887e-05 2024-09-16T20:49:23.026 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.027 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.027 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:23.027 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689625+0000", 2024-09-16T20:49:23.027 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050495848000000003 2024-09-16T20:49:23.027 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.027 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.027 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:23.027 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689720+0000", 2024-09-16T20:49:23.027 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4821000000000006e-05 2024-09-16T20:49:23.027 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.027 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.027 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:23.027 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689804+0000", 2024-09-16T20:49:23.027 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4235999999999998e-05 2024-09-16T20:49:23.028 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:23.028 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:23.028 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:23.028 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.028 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.028 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3725 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1521664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:23.028 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:49:23.028 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 60.354420431000001, 2024-09-16T20:49:23.028 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050797660000000001, 2024-09-16T20:49:23.028 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:23.028 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:23.028 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:23.028 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:23.029 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:23.029 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:23.029 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3725 2024-09-16T20:49:23.029 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.029 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:23.029 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.029 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:23.029 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:49:23.029 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:23.029 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.029 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.029 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:23.029 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:49:23.030 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:23.030 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.030 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.030 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:23.030 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639139+0000", 2024-09-16T20:49:23.030 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8999999999999997e-07 2024-09-16T20:49:23.030 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.030 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.030 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:23.030 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639141+0000", 2024-09-16T20:49:23.030 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.182e-06 2024-09-16T20:49:23.030 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.030 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.031 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:23.031 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639154+0000", 2024-09-16T20:49:23.031 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3626e-05 2024-09-16T20:49:23.031 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.031 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.031 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:23.031 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639160+0000", 2024-09-16T20:49:23.031 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6810000000000003e-06 2024-09-16T20:49:23.031 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.031 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.031 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:23.031 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689831+0000", 2024-09-16T20:49:23.031 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050670907000000001 2024-09-16T20:49:23.031 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.032 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.032 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:23.032 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689882+0000", 2024-09-16T20:49:23.032 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0445999999999998e-05 2024-09-16T20:49:23.032 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.032 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.032 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:23.032 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689936+0000", 2024-09-16T20:49:23.032 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4327999999999999e-05 2024-09-16T20:49:23.032 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:23.032 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:23.032 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:23.032 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.032 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.033 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3727 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1529856~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:23.033 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:49:23.033 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 60.354384854000003, 2024-09-16T20:49:23.033 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051016922999999999, 2024-09-16T20:49:23.033 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:23.033 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:23.033 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:23.033 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:23.033 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:23.033 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:23.033 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3727 2024-09-16T20:49:23.033 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.033 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:23.033 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.034 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:23.034 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:49:23.034 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:23.034 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.034 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.034 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:23.034 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:49:23.034 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:23.034 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.034 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.034 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:23.034 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:49:23.034 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9999999999999998e-07 2024-09-16T20:49:23.034 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.035 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.035 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:23.035 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639183+0000", 2024-09-16T20:49:23.035 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2399999999999996e-06 2024-09-16T20:49:23.035 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.035 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.035 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:23.035 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639195+0000", 2024-09-16T20:49:23.035 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1113e-05 2024-09-16T20:49:23.035 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.035 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.035 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:23.036 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639206+0000", 2024-09-16T20:49:23.036 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1503e-05 2024-09-16T20:49:23.036 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.036 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.036 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:23.036 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689969+0000", 2024-09-16T20:49:23.036 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050762676 2024-09-16T20:49:23.036 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.036 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.036 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:23.036 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690071+0000", 2024-09-16T20:49:23.036 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102163 2024-09-16T20:49:23.036 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.036 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.037 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:23.037 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690191+0000", 2024-09-16T20:49:23.037 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000119728 2024-09-16T20:49:23.037 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:23.037 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:23.037 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:23.037 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.037 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.037 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3730 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1538048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:23.037 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:49:23.037 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 60.354343290999999, 2024-09-16T20:49:23.037 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051131467999999999, 2024-09-16T20:49:23.037 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:23.037 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:23.038 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:23.038 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:23.038 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:23.038 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:23.038 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3730 2024-09-16T20:49:23.038 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.038 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:23.038 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.038 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:23.038 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:49:23.038 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:23.038 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.038 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.039 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:23.039 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:49:23.039 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:23.039 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.039 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.039 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:23.039 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639216+0000", 2024-09-16T20:49:23.039 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.39e-07 2024-09-16T20:49:23.039 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.039 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.039 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:23.039 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639218+0000", 2024-09-16T20:49:23.039 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.537e-06 2024-09-16T20:49:23.040 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.040 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.040 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:23.040 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639229+0000", 2024-09-16T20:49:23.040 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1122e-05 2024-09-16T20:49:23.040 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.040 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.040 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:23.040 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639237+0000", 2024-09-16T20:49:23.040 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8509999999999999e-06 2024-09-16T20:49:23.040 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.040 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.041 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:23.041 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690227+0000", 2024-09-16T20:49:23.041 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050990147 2024-09-16T20:49:23.041 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.041 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.041 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:23.041 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690278+0000", 2024-09-16T20:49:23.041 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0223999999999999e-05 2024-09-16T20:49:23.041 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.041 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.041 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:23.041 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690347+0000", 2024-09-16T20:49:23.041 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9147999999999996e-05 2024-09-16T20:49:23.042 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:23.042 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:23.042 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:23.042 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.042 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.042 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3731 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1546240~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:23.042 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:49:23.042 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 60.354312583000002, 2024-09-16T20:49:23.042 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051276744999999999, 2024-09-16T20:49:23.042 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:23.042 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:23.042 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:23.042 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:23.042 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:23.043 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:23.043 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3731 2024-09-16T20:49:23.043 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.043 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:23.043 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.043 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:23.043 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:49:23.043 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:23.043 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.043 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.043 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:23.043 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:49:23.043 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:23.043 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.044 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.044 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:23.044 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:49:23.044 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9900000000000001e-07 2024-09-16T20:49:23.044 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.044 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.044 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:23.044 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639249+0000", 2024-09-16T20:49:23.044 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3630000000000001e-06 2024-09-16T20:49:23.044 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.044 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.044 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:23.044 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639255+0000", 2024-09-16T20:49:23.045 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4960000000000001e-06 2024-09-16T20:49:23.045 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.045 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.045 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:23.045 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639261+0000", 2024-09-16T20:49:23.045 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2839999999999997e-06 2024-09-16T20:49:23.045 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.045 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.045 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:23.045 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690375+0000", 2024-09-16T20:49:23.045 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051114684000000001 2024-09-16T20:49:23.045 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.045 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.046 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:23.046 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690425+0000", 2024-09-16T20:49:23.046 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9379999999999998e-05 2024-09-16T20:49:23.046 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.046 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.046 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:23.046 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690523+0000", 2024-09-16T20:49:23.046 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8139000000000003e-05 2024-09-16T20:49:23.046 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:23.046 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:23.046 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:23.046 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.046 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.047 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3732 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1554432~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:23.047 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:49:23.047 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 60.354266801000001, 2024-09-16T20:49:23.047 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051380096, 2024-09-16T20:49:23.047 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:23.047 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:23.047 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:23.047 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:23.047 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:23.047 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:23.047 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3732 2024-09-16T20:49:23.047 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.047 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:23.048 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.048 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:23.048 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:49:23.048 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:23.048 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.048 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.048 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:23.048 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:49:23.048 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:23.048 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.048 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.048 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:23.048 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:49:23.048 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6100000000000001e-07 2024-09-16T20:49:23.049 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.049 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.049 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:23.049 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639295+0000", 2024-09-16T20:49:23.049 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.447e-06 2024-09-16T20:49:23.049 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.049 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.049 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:23.049 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639302+0000", 2024-09-16T20:49:23.049 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0570000000000004e-06 2024-09-16T20:49:23.049 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.049 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.049 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:23.050 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639307+0000", 2024-09-16T20:49:23.050 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1429999999999997e-06 2024-09-16T20:49:23.050 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.050 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.050 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:23.050 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690555+0000", 2024-09-16T20:49:23.050 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051247963000000001 2024-09-16T20:49:23.050 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.050 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.050 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:23.050 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690602+0000", 2024-09-16T20:49:23.050 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7602000000000003e-05 2024-09-16T20:49:23.050 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.050 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.051 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:23.051 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690672+0000", 2024-09-16T20:49:23.051 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9423000000000003e-05 2024-09-16T20:49:23.051 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:23.051 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:23.051 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:23.051 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.051 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.051 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3734 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1562624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:23.051 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:49:23.051 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 60.354245151000001, 2024-09-16T20:49:23.051 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051490610999999999, 2024-09-16T20:49:23.051 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:23.051 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:23.052 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:23.052 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:23.052 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:23.052 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:23.052 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3734 2024-09-16T20:49:23.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.052 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:23.052 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.052 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:23.052 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:49:23.052 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:23.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.052 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.052 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:23.053 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:49:23.053 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:23.053 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.053 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.053 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:23.053 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639314+0000", 2024-09-16T20:49:23.053 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4400000000000001e-07 2024-09-16T20:49:23.053 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.053 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.053 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:23.053 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639316+0000", 2024-09-16T20:49:23.053 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1459999999999999e-06 2024-09-16T20:49:23.053 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.054 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.054 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:23.054 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639322+0000", 2024-09-16T20:49:23.054 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7610000000000004e-06 2024-09-16T20:49:23.054 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.054 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.054 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:23.054 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639327+0000", 2024-09-16T20:49:23.054 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.186e-06 2024-09-16T20:49:23.054 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.054 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.054 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:23.055 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690698+0000", 2024-09-16T20:49:23.055 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051371223000000001 2024-09-16T20:49:23.055 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.055 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.055 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:23.055 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690748+0000", 2024-09-16T20:49:23.055 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0192999999999999e-05 2024-09-16T20:49:23.055 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.055 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.055 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:23.055 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690804+0000", 2024-09-16T20:49:23.055 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5658000000000001e-05 2024-09-16T20:49:23.055 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:23.055 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:23.056 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:23.056 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.056 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.056 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3736 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1570816~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:23.056 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:49:23.056 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 60.354225892000002, 2024-09-16T20:49:23.056 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051596528000000003, 2024-09-16T20:49:23.056 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:23.056 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:23.056 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:23.056 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:23.056 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:23.056 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:23.056 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3736 2024-09-16T20:49:23.057 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.057 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:23.057 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.057 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:23.057 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:49:23.057 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:23.057 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.057 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.057 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:23.057 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:49:23.057 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:23.057 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.057 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.057 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:23.058 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:49:23.058 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9999999999999999e-07 2024-09-16T20:49:23.058 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.058 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.058 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:23.058 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639335+0000", 2024-09-16T20:49:23.058 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.635e-06 2024-09-16T20:49:23.058 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.058 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.058 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:23.058 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639340+0000", 2024-09-16T20:49:23.058 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5840000000000002e-06 2024-09-16T20:49:23.058 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.058 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.059 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:23.059 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639346+0000", 2024-09-16T20:49:23.059 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1190000000000002e-06 2024-09-16T20:49:23.059 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.059 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.059 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:23.059 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690817+0000", 2024-09-16T20:49:23.059 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051470721999999997 2024-09-16T20:49:23.059 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.059 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.059 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:23.059 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690875+0000", 2024-09-16T20:49:23.059 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7441000000000001e-05 2024-09-16T20:49:23.059 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.059 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.060 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:23.060 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690929+0000", 2024-09-16T20:49:23.060 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4727e-05 2024-09-16T20:49:23.060 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:23.060 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:23.060 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:23.060 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.060 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.060 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5434 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:23.060 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:49:23.060 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 60.128964326000002, 2024-09-16T20:49:23.060 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099755831000000003, 2024-09-16T20:49:23.060 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:23.061 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:23.061 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:23.061 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:23.061 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:23.061 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:23.061 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5434 2024-09-16T20:49:23.061 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.061 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:23.061 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.061 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:23.061 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:49:23.061 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:23.062 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.062 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.062 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:23.062 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:49:23.062 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:23.062 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.062 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.062 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:23.062 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864595+0000", 2024-09-16T20:49:23.062 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1099999999999995e-07 2024-09-16T20:49:23.062 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.062 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.062 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:23.062 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864597+0000", 2024-09-16T20:49:23.063 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3989999999999998e-06 2024-09-16T20:49:23.063 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.063 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.063 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:23.063 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864603+0000", 2024-09-16T20:49:23.063 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9309999999999996e-06 2024-09-16T20:49:23.063 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.063 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.063 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:23.063 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864607+0000", 2024-09-16T20:49:23.063 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6200000000000001e-06 2024-09-16T20:49:23.063 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.063 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.063 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:23.064 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964164+0000", 2024-09-16T20:49:23.064 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099557162000000005 2024-09-16T20:49:23.064 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.064 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.064 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:23.064 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964244+0000", 2024-09-16T20:49:23.064 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9597000000000004e-05 2024-09-16T20:49:23.064 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.064 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.064 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:23.064 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964350+0000", 2024-09-16T20:49:23.064 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010641099999999999 2024-09-16T20:49:23.064 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:23.064 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:23.065 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:23.065 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.065 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.065 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5435 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 20480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:23.065 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:49:23.065 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 60.128934504, 2024-09-16T20:49:23.065 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100897102, 2024-09-16T20:49:23.065 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:23.065 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:23.065 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:23.065 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:23.065 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:23.065 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:23.065 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5435 2024-09-16T20:49:23.066 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.066 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:23.066 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.066 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:23.066 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:49:23.066 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:23.066 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.066 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.066 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:23.066 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:49:23.066 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:23.066 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.066 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.066 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:23.067 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:49:23.067 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8000000000000001e-07 2024-09-16T20:49:23.067 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.067 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.067 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:23.067 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864626+0000", 2024-09-16T20:49:23.067 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.745e-06 2024-09-16T20:49:23.067 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.067 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.067 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:23.067 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864633+0000", 2024-09-16T20:49:23.067 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3339999999999998e-06 2024-09-16T20:49:23.068 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.068 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.068 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:23.068 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864636+0000", 2024-09-16T20:49:23.068 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1200000000000002e-06 2024-09-16T20:49:23.068 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.068 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.068 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:23.068 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965299+0000", 2024-09-16T20:49:23.068 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10066352000000001 2024-09-16T20:49:23.068 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.068 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.068 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:23.068 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965390+0000", 2024-09-16T20:49:23.069 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0894000000000001e-05 2024-09-16T20:49:23.069 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.069 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.069 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:23.069 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965521+0000", 2024-09-16T20:49:23.069 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013110899999999999 2024-09-16T20:49:23.069 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:23.069 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:23.069 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:23.069 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.069 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.069 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:152 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 327680~32768 out=32768b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:49:23.069 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:49:23.069 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 58.830437562, 2024-09-16T20:49:23.070 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11869405800000001, 2024-09-16T20:49:23.070 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:23.070 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:23.070 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:23.070 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:23.070 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:49:23.070 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:49:23.070 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 152 2024-09-16T20:49:23.070 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.070 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:23.070 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.070 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:23.070 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:49:23.070 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:23.071 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.071 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.071 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:23.071 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:49:23.071 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:23.071 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.071 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.071 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:23.071 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163122+0000", 2024-09-16T20:49:23.071 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6400000000000003e-07 2024-09-16T20:49:23.071 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.071 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.071 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:23.072 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163124+0000", 2024-09-16T20:49:23.072 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.565e-06 2024-09-16T20:49:23.072 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.072 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.072 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:23.072 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163136+0000", 2024-09-16T20:49:23.072 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1717e-05 2024-09-16T20:49:23.072 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.072 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.072 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:23.072 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163142+0000", 2024-09-16T20:49:23.072 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1750000000000002e-06 2024-09-16T20:49:23.072 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.072 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.073 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:23.073 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163606+0000", 2024-09-16T20:49:23.073 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00046418799999999998 2024-09-16T20:49:23.073 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.073 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.073 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:23.073 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163656+0000", 2024-09-16T20:49:23.073 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9852999999999999e-05 2024-09-16T20:49:23.073 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.073 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.073 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:23.073 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281815+0000", 2024-09-16T20:49:23.073 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11815909600000001 2024-09-16T20:49:23.074 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:23.074 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:23.074 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:23.074 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.074 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.074 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:153 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:49:23.074 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:49:23.074 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 58.830404608999999, 2024-09-16T20:49:23.074 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11903462400000001, 2024-09-16T20:49:23.074 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:23.074 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:23.074 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:23.074 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:23.074 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:49:23.074 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:49:23.075 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 153 2024-09-16T20:49:23.075 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.075 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:23.075 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.075 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:23.075 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:49:23.075 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:23.075 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.075 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.075 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:23.075 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:49:23.075 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:23.075 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.075 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.076 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:23.076 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163155+0000", 2024-09-16T20:49:23.076 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.37e-07 2024-09-16T20:49:23.076 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.076 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.076 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:23.076 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163157+0000", 2024-09-16T20:49:23.076 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6010000000000002e-06 2024-09-16T20:49:23.076 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.076 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.076 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:23.076 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163165+0000", 2024-09-16T20:49:23.076 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2619999999999998e-06 2024-09-16T20:49:23.076 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.077 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.077 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:23.077 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163171+0000", 2024-09-16T20:49:23.077 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.711e-06 2024-09-16T20:49:23.077 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.077 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.077 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:23.077 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281868+0000", 2024-09-16T20:49:23.077 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118696622 2024-09-16T20:49:23.077 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.077 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.077 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:23.077 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281969+0000", 2024-09-16T20:49:23.078 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101371 2024-09-16T20:49:23.078 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.078 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.078 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:23.078 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282189+0000", 2024-09-16T20:49:23.078 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00021952000000000001 2024-09-16T20:49:23.078 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:23.078 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:23.078 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:23.078 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.078 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.078 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:154 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 360448~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:49:23.078 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:49:23.079 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 58.830372836999999, 2024-09-16T20:49:23.079 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11932035100000001, 2024-09-16T20:49:23.079 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:23.079 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:23.079 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:23.079 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:23.079 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:49:23.079 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:49:23.079 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 154 2024-09-16T20:49:23.080 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.080 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:23.080 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.080 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:23.080 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:49:23.080 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:23.080 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.080 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.080 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:23.080 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:49:23.080 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:23.080 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.080 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.081 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:23.081 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:49:23.081 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:49:23.081 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.081 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.081 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:23.081 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163189+0000", 2024-09-16T20:49:23.081 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6840000000000001e-06 2024-09-16T20:49:23.081 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.081 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.081 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:23.081 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163196+0000", 2024-09-16T20:49:23.081 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7920000000000004e-06 2024-09-16T20:49:23.081 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.082 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.082 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:23.082 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163201+0000", 2024-09-16T20:49:23.082 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2809999999999999e-06 2024-09-16T20:49:23.082 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.082 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.082 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:23.082 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282219+0000", 2024-09-16T20:49:23.082 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.119018343 2024-09-16T20:49:23.082 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.082 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.082 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:23.082 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282305+0000", 2024-09-16T20:49:23.082 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5865000000000005e-05 2024-09-16T20:49:23.083 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.083 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.083 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:23.083 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282506+0000", 2024-09-16T20:49:23.083 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000200927 2024-09-16T20:49:23.083 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:23.083 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:23.083 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:23.083 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.083 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.083 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop(client.4366.0:3088 3.1e e38/23 3:79e5ec01:::rbd_header.110e891a9b2b:head v 38'306, mlcod=36'304)", 2024-09-16T20:49:23.083 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:49:23.083 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 58.829353969000003, 2024-09-16T20:49:23.084 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118102759, 2024-09-16T20:49:23.084 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:23.084 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:23.084 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:23.084 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:23.084 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.084 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:23.084 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:49:23.084 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:23.084 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.084 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.084 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:23.084 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:49:23.084 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:23.085 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.085 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.085 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:23.085 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:49:23.085 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.85e-07 2024-09-16T20:49:23.085 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.085 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.085 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:23.085 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164212+0000", 2024-09-16T20:49:23.085 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3999999999999997e-06 2024-09-16T20:49:23.085 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.085 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.086 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:23.086 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281553+0000", 2024-09-16T20:49:23.086 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11734135499999999 2024-09-16T20:49:23.086 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.086 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.086 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:23.086 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281567+0000", 2024-09-16T20:49:23.086 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4188999999999999e-05 2024-09-16T20:49:23.086 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.086 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.086 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:23.086 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281653+0000", 2024-09-16T20:49:23.086 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5385999999999999e-05 2024-09-16T20:49:23.086 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.087 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.087 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:23.087 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281679+0000", 2024-09-16T20:49:23.087 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6689000000000001e-05 2024-09-16T20:49:23.087 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.087 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.087 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:23.087 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282277+0000", 2024-09-16T20:49:23.087 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00059798000000000002 2024-09-16T20:49:23.087 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:23.087 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:23.087 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:23.087 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282307+0000", 2024-09-16T20:49:23.087 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9975e-05 2024-09-16T20:49:23.088 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:23.088 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:23.088 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:23.088 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:23.088 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:23.088 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:23.089 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:23.088+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:23.090 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:49:23.091 INFO:teuthology.orchestra.run.smithi139.stdout:3164 write 0x951da00 thru 0x952c0f3 (0xe6f4 bytes) 2024-09-16T20:49:23.091 INFO:teuthology.orchestra.run.smithi139.stdout:3168 write 0x6fad400 thru 0x6fb1332 (0x3f33 bytes) 2024-09-16T20:49:23.091 INFO:teuthology.orchestra.run.smithi139.stdout:3169 read 0x49a7000 thru 0x49ac922 (0x5923 bytes) 2024-09-16T20:49:23.091 INFO:teuthology.orchestra.run.smithi139.stdout:3171 write 0xab5000 thru 0xabc1e0 (0x71e1 bytes) 2024-09-16T20:49:23.091 INFO:teuthology.orchestra.run.smithi139.stdout:3172 trunc from 0x952c200 to 0x7e4ac00 2024-09-16T20:49:23.091 INFO:teuthology.orchestra.run.smithi139.stdout:3173 write 0xa465400 thru 0xa46aed6 (0x5ad7 bytes) 2024-09-16T20:49:23.091 INFO:teuthology.orchestra.run.smithi139.stdout:3175 punch from 0x92e7600 to 0x92f2000, (0xaa00 bytes) 2024-09-16T20:49:23.091 INFO:teuthology.orchestra.run.smithi139.stdout:3176 trunc from 0xa46b000 to 0x9509800 2024-09-16T20:49:23.099 INFO:teuthology.orchestra.run.smithi139.stdout:3178 trunc from 0x9509800 to 0x312c600 2024-09-16T20:49:23.119 INFO:teuthology.orchestra.run.smithi139.stdout:3181 read 0x268000 thru 0x26b2ee (0x32ef bytes) 2024-09-16T20:49:23.120 INFO:teuthology.orchestra.run.smithi139.stdout:3183 read 0x2e37200 thru 0x2e42312 (0xb113 bytes) 2024-09-16T20:49:23.138 INFO:teuthology.orchestra.run.smithi139.stdout:3184 write 0x9382c00 thru 0x9383cb1 (0x10b2 bytes) 2024-09-16T20:49:23.153 INFO:teuthology.orchestra.run.smithi139.stdout:3187 write 0x9bbda00 thru 0x9bc1e8c (0x448d bytes) 2024-09-16T20:49:23.153 INFO:teuthology.orchestra.run.smithi139.stdout:3188 punch from 0x8915c00 to 0x891b000, (0x5400 bytes) 2024-09-16T20:49:23.158 INFO:teuthology.orchestra.run.smithi139.stdout:3190 read 0x38f3400 thru 0x38f471d (0x131e bytes) 2024-09-16T20:49:23.159 INFO:teuthology.orchestra.run.smithi139.stdout:3194 punch from 0x3cae800 to 0x3cb1e00, (0x3600 bytes) 2024-09-16T20:49:23.169 INFO:teuthology.orchestra.run.smithi139.stdout:3195 write 0xaf2bc00 thru 0xaf2f22e (0x362f bytes) 2024-09-16T20:49:23.177 INFO:teuthology.orchestra.run.smithi139.stdout:3196 punch from 0x4492c00 to 0x4497600, (0x4a00 bytes) 2024-09-16T20:49:23.179 INFO:teuthology.orchestra.run.smithi139.stdout:3198 write 0x9b85a00 thru 0x9b8d0a4 (0x76a5 bytes) 2024-09-16T20:49:23.179 INFO:teuthology.orchestra.run.smithi139.stdout:3199 read 0x98fee00 thru 0x9904459 (0x565a bytes) 2024-09-16T20:49:23.180 INFO:teuthology.orchestra.run.smithi139.stdout:3200 read 0x42a2600 thru 0x42a8825 (0x6226 bytes) 2024-09-16T20:49:23.180 INFO:teuthology.orchestra.run.smithi139.stdout:3203 read 0x66e5000 thru 0x66ee6f0 (0x96f1 bytes) 2024-09-16T20:49:23.181 INFO:teuthology.orchestra.run.smithi139.stdout:3205 write 0x7303200 thru 0x7312666 (0xf467 bytes) 2024-09-16T20:49:23.181 INFO:teuthology.orchestra.run.smithi139.stdout:3206 write 0x2406800 thru 0x2406bd4 (0x3d5 bytes) 2024-09-16T20:49:23.181 INFO:teuthology.orchestra.run.smithi139.stdout:3207 read 0x3d33a00 thru 0x3d346c8 (0xcc9 bytes) 2024-09-16T20:49:23.182 INFO:teuthology.orchestra.run.smithi139.stdout:3211 write 0x339b600 thru 0x33a24bd (0x6ebe bytes) 2024-09-16T20:49:23.182 INFO:teuthology.orchestra.run.smithi139.stdout:3212 read 0x3c51e00 thru 0x3c52fd3 (0x11d4 bytes) 2024-09-16T20:49:23.190 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:23.190+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:23.196 INFO:teuthology.orchestra.run.smithi139.stdout:3214 write 0xd0ad800 thru 0xd0b97c6 (0xbfc7 bytes) 2024-09-16T20:49:23.203 INFO:teuthology.orchestra.run.smithi139.stdout:3215 punch from 0x65c6800 to 0x65ca200, (0x3a00 bytes) 2024-09-16T20:49:23.204 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:23.204 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:49:23.205 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:49:23.205 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:23.205 INFO:teuthology.orchestra.run.smithi139.stdout:3219 write 0x87bb000 thru 0x87c8a91 (0xda92 bytes) 2024-09-16T20:49:23.205 INFO:teuthology.orchestra.run.smithi139.stdout:3221 read 0x324f800 thru 0x325a6c1 (0xaec2 bytes) 2024-09-16T20:49:23.206 INFO:teuthology.orchestra.run.smithi139.stdout:3222 read 0x9d10200 thru 0x9d1e85f (0xe660 bytes) 2024-09-16T20:49:23.206 INFO:teuthology.orchestra.run.smithi139.stdout:3223 read 0xbf86a00 thru 0xbf94342 (0xd943 bytes) 2024-09-16T20:49:23.207 INFO:teuthology.orchestra.run.smithi139.stdout:3224 punch from 0x1a11200 to 0x1a1d000, (0xbe00 bytes) 2024-09-16T20:49:23.214 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:49:23.216 INFO:teuthology.orchestra.run.smithi139.stdout:3226 trunc from 0xd0b9800 to 0x5c84000 2024-09-16T20:49:23.239 INFO:teuthology.orchestra.run.smithi139.stdout:3227 write 0x4254e00 thru 0x425fb70 (0xad71 bytes) 2024-09-16T20:49:23.240 INFO:teuthology.orchestra.run.smithi139.stdout:3228 punch from 0x567c600 to 0x568a200, (0xdc00 bytes) 2024-09-16T20:49:23.250 INFO:teuthology.orchestra.run.smithi139.stdout:3231 trunc from 0x5c84000 to 0xd5ea200 2024-09-16T20:49:23.259 INFO:teuthology.orchestra.run.smithi139.stdout:3232 punch from 0xc28d200 to 0xc29b800, (0xe600 bytes) 2024-09-16T20:49:23.267 INFO:teuthology.orchestra.run.smithi139.stdout:3233 punch from 0x7af5c00 to 0x7af9c00, (0x4000 bytes) 2024-09-16T20:49:23.267 INFO:teuthology.orchestra.run.smithi139.stdout:3234 read 0xb098400 thru 0xb0a5754 (0xd355 bytes) 2024-09-16T20:49:23.267 INFO:teuthology.orchestra.run.smithi139.stdout:3235 write 0xb5fa200 thru 0xb5fc1d7 (0x1fd8 bytes) 2024-09-16T20:49:23.267 INFO:teuthology.orchestra.run.smithi139.stdout:3236 read 0x935b800 thru 0x936146f (0x5c70 bytes) 2024-09-16T20:49:23.268 INFO:teuthology.orchestra.run.smithi139.stdout:3237 write 0x43eaa00 thru 0x43f6a87 (0xc088 bytes) 2024-09-16T20:49:23.268 INFO:teuthology.orchestra.run.smithi139.stdout:3238 punch from 0x2b08a00 to 0x2b0ae00, (0x2400 bytes) 2024-09-16T20:49:23.280 INFO:teuthology.orchestra.run.smithi139.stdout:3239 trunc from 0xd5ea200 to 0x2da3a00 2024-09-16T20:49:23.290 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:23.289+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:23.301 INFO:teuthology.orchestra.run.smithi139.stdout:3240 read 0x1f15a00 thru 0x1f20781 (0xad82 bytes) 2024-09-16T20:49:23.302 INFO:teuthology.orchestra.run.smithi139.stdout:3241 read 0x1d2f400 thru 0x1d3ea62 (0xf663 bytes) 2024-09-16T20:49:23.303 INFO:teuthology.orchestra.run.smithi139.stdout:3242 punch from 0x263d800 to 0x2645000, (0x7800 bytes) 2024-09-16T20:49:23.326 INFO:teuthology.orchestra.run.smithi139.stdout:3246 write 0x89f5600 thru 0x8a026da (0xd0db bytes) 2024-09-16T20:49:23.326 INFO:teuthology.orchestra.run.smithi139.stdout:3248 write 0x5d10000 thru 0x5d1dea0 (0xdea1 bytes) 2024-09-16T20:49:23.326 INFO:teuthology.orchestra.run.smithi139.stdout:3249 read 0x5a60000 thru 0x5a6b369 (0xb36a bytes) 2024-09-16T20:49:23.327 INFO:teuthology.orchestra.run.smithi139.stdout:3250 clone 26 order 21 su 32768 sc 13 2024-09-16T20:49:23.366 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:23.366 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:49:23.366 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:49:23.366 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:49:23.366 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:23.375 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:49:23.419 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:23.419+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:23.483 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:23.483 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:49:23.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:49:23.484 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:49:23.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.484 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6415 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3577856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:23.484 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871730+0000", 2024-09-16T20:49:23.484 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 63.609757336999998, 2024-09-16T20:49:23.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.04851482, 2024-09-16T20:49:23.484 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:23.484 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:23.484 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:23.484 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:23.484 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:23.485 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:23.485 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6415 2024-09-16T20:49:23.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.485 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:23.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.485 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:23.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871730+0000", 2024-09-16T20:49:23.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:23.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.485 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:23.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871730+0000", 2024-09-16T20:49:23.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:23.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.486 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:23.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871731+0000", 2024-09-16T20:49:23.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6899999999999998e-07 2024-09-16T20:49:23.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.486 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:23.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871733+0000", 2024-09-16T20:49:23.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0600000000000002e-06 2024-09-16T20:49:23.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.486 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:23.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871740+0000", 2024-09-16T20:49:23.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3749999999999997e-06 2024-09-16T20:49:23.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:23.487 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871749+0000", 2024-09-16T20:49:23.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9109999999999999e-06 2024-09-16T20:49:23.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:23.487 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920129+0000", 2024-09-16T20:49:23.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048380175999999997 2024-09-16T20:49:23.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:23.488 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920181+0000", 2024-09-16T20:49:23.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2077000000000003e-05 2024-09-16T20:49:23.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.488 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:23.488 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920245+0000", 2024-09-16T20:49:23.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3851999999999998e-05 2024-09-16T20:49:23.488 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:23.488 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:23.488 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:23.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.488 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6417 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3586048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:23.489 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871756+0000", 2024-09-16T20:49:23.489 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 63.609731701999998, 2024-09-16T20:49:23.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048628301999999998, 2024-09-16T20:49:23.489 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:23.489 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:23.489 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:23.489 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:23.489 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:23.489 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:23.489 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6417 2024-09-16T20:49:23.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.489 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:23.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:23.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871756+0000", 2024-09-16T20:49:23.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:23.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:23.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871756+0000", 2024-09-16T20:49:23.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:23.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:23.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871756+0000", 2024-09-16T20:49:23.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0100000000000005e-07 2024-09-16T20:49:23.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:23.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871758+0000", 2024-09-16T20:49:23.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.902e-06 2024-09-16T20:49:23.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:23.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871764+0000", 2024-09-16T20:49:23.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8170000000000004e-06 2024-09-16T20:49:23.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:23.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871769+0000", 2024-09-16T20:49:23.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6500000000000004e-06 2024-09-16T20:49:23.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.492 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:23.492 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920276+0000", 2024-09-16T20:49:23.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048507794999999999 2024-09-16T20:49:23.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.492 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:23.492 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920329+0000", 2024-09-16T20:49:23.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2534999999999998e-05 2024-09-16T20:49:23.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.493 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:23.493 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920384+0000", 2024-09-16T20:49:23.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5102000000000002e-05 2024-09-16T20:49:23.493 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:23.493 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:23.493 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:23.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.493 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6419 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3594240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:23.493 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871775+0000", 2024-09-16T20:49:23.493 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 63.609712524999999, 2024-09-16T20:49:23.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048754724999999999, 2024-09-16T20:49:23.493 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:23.494 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:23.494 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:23.494 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:23.494 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:23.494 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:23.494 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6419 2024-09-16T20:49:23.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.494 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:23.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.494 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:23.494 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871775+0000", 2024-09-16T20:49:23.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:23.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:23.495 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871775+0000", 2024-09-16T20:49:23.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:23.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:23.495 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871775+0000", 2024-09-16T20:49:23.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2500000000000001e-07 2024-09-16T20:49:23.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:23.495 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871777+0000", 2024-09-16T20:49:23.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9649999999999998e-06 2024-09-16T20:49:23.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:23.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871784+0000", 2024-09-16T20:49:23.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3419999999999996e-06 2024-09-16T20:49:23.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:23.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871789+0000", 2024-09-16T20:49:23.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1889999999999998e-06 2024-09-16T20:49:23.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:23.497 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920416+0000", 2024-09-16T20:49:23.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048627363999999999 2024-09-16T20:49:23.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:23.497 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920472+0000", 2024-09-16T20:49:23.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5429000000000001e-05 2024-09-16T20:49:23.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:23.497 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920530+0000", 2024-09-16T20:49:23.498 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8111e-05 2024-09-16T20:49:23.498 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:23.498 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:23.498 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:23.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.498 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6420 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3602432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:23.498 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871796+0000", 2024-09-16T20:49:23.500 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 63.609691611999999, 2024-09-16T20:49:23.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048857298, 2024-09-16T20:49:23.500 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:23.501 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:23.501 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:23.501 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:23.501 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:23.501 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:23.501 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6420 2024-09-16T20:49:23.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.501 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:23.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:23.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871796+0000", 2024-09-16T20:49:23.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:23.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.502 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:23.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871796+0000", 2024-09-16T20:49:23.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:23.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.502 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:23.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871854+0000", 2024-09-16T20:49:23.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8187999999999999e-05 2024-09-16T20:49:23.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.502 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:23.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871856+0000", 2024-09-16T20:49:23.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3549999999999999e-06 2024-09-16T20:49:23.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:23.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871866+0000", 2024-09-16T20:49:23.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6500000000000008e-06 2024-09-16T20:49:23.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:23.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871872+0000", 2024-09-16T20:49:23.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3799999999999999e-06 2024-09-16T20:49:23.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:23.504 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920558+0000", 2024-09-16T20:49:23.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048685797000000003 2024-09-16T20:49:23.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.504 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:23.504 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920603+0000", 2024-09-16T20:49:23.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4629999999999998e-05 2024-09-16T20:49:23.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.504 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:23.504 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920653+0000", 2024-09-16T20:49:23.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0297999999999999e-05 2024-09-16T20:49:23.505 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:23.505 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:23.505 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:23.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.505 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6421 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3610624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:23.505 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871882+0000", 2024-09-16T20:49:23.505 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 63.609605543999997, 2024-09-16T20:49:23.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048913381999999998, 2024-09-16T20:49:23.506 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:23.507 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:23.507 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:23.507 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:23.507 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:23.507 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:23.507 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6421 2024-09-16T20:49:23.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.507 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:23.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.508 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:23.508 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871882+0000", 2024-09-16T20:49:23.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:23.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:23.509 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871882+0000", 2024-09-16T20:49:23.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:23.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:23.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.873257+0000", 2024-09-16T20:49:23.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001374789 2024-09-16T20:49:23.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:23.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.873261+0000", 2024-09-16T20:49:23.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5589999999999999e-06 2024-09-16T20:49:23.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:23.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.873275+0000", 2024-09-16T20:49:23.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3675999999999999e-05 2024-09-16T20:49:23.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.512 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.512 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:23.512 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.873281+0000", 2024-09-16T20:49:23.513 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6930000000000004e-06 2024-09-16T20:49:23.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.513 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:23.513 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920684+0000", 2024-09-16T20:49:23.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047402913999999997 2024-09-16T20:49:23.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:23.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920737+0000", 2024-09-16T20:49:23.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3600000000000002e-05 2024-09-16T20:49:23.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.515 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.515 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:23.515 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920795+0000", 2024-09-16T20:49:23.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8150999999999999e-05 2024-09-16T20:49:23.515 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:23.516 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:23.516 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:23.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.516 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.516 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:7491 3.1e 3:7c48f6ae:::rbd_object_map.110e891a9b2b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:23.516 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.925865+0000", 2024-09-16T20:49:23.517 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 63.555622399999997, 2024-09-16T20:49:23.517 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048897293000000001, 2024-09-16T20:49:23.517 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:23.517 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:23.517 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:23.517 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:23.518 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:23.518 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:23.518 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 7491 2024-09-16T20:49:23.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.519 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:23.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.519 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:23.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925865+0000", 2024-09-16T20:49:23.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:23.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.520 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:23.520 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925865+0000", 2024-09-16T20:49:23.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:23.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.521 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.521 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:23.521 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925866+0000", 2024-09-16T20:49:23.521 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.39e-07 2024-09-16T20:49:23.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.521 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.522 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:23.522 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925868+0000", 2024-09-16T20:49:23.522 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.508e-06 2024-09-16T20:49:23.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:23.523 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925874+0000", 2024-09-16T20:49:23.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7919999999999998e-06 2024-09-16T20:49:23.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:23.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925879+0000", 2024-09-16T20:49:23.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8579999999999999e-06 2024-09-16T20:49:23.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.524 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:23.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.970875+0000", 2024-09-16T20:49:23.525 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044996412 2024-09-16T20:49:23.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.525 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.525 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:23.525 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.970970+0000", 2024-09-16T20:49:23.525 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4732e-05 2024-09-16T20:49:23.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.526 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.526 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-16T20:49:23.526 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.971275+0000", 2024-09-16T20:49:23.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00030464199999999999 2024-09-16T20:49:23.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.527 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:23.527 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.974598+0000", 2024-09-16T20:49:23.527 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0033238159999999998 2024-09-16T20:49:23.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:49:23.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.974666+0000", 2024-09-16T20:49:23.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7617e-05 2024-09-16T20:49:23.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:49:23.529 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.974718+0000", 2024-09-16T20:49:23.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2259000000000002e-05 2024-09-16T20:49:23.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.529 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:23.530 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.974762+0000", 2024-09-16T20:49:23.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4218000000000003e-05 2024-09-16T20:49:23.530 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:23.530 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:23.530 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:23.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.531 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4756 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1513472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:23.531 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689118+0000", 2024-09-16T20:49:23.531 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 60.792369981, 2024-09-16T20:49:23.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048535146000000001, 2024-09-16T20:49:23.531 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:23.532 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:23.532 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:23.532 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:23.532 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:49:23.532 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:23.532 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4756 2024-09-16T20:49:23.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.533 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:23.533 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.533 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:23.533 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689118+0000", 2024-09-16T20:49:23.533 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:23.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.534 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:23.534 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689118+0000", 2024-09-16T20:49:23.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:23.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:23.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689118+0000", 2024-09-16T20:49:23.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7099999999999997e-07 2024-09-16T20:49:23.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.536 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:23.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689120+0000", 2024-09-16T20:49:23.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.787e-06 2024-09-16T20:49:23.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:23.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689126+0000", 2024-09-16T20:49:23.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2720000000000003e-06 2024-09-16T20:49:23.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.538 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:23.538 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689134+0000", 2024-09-16T20:49:23.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2190000000000004e-06 2024-09-16T20:49:23.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.538 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:23.539 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737553+0000", 2024-09-16T20:49:23.539 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048418993 2024-09-16T20:49:23.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.539 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:23.539 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737598+0000", 2024-09-16T20:49:23.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4743999999999997e-05 2024-09-16T20:49:23.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:23.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737653+0000", 2024-09-16T20:49:23.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4759999999999997e-05 2024-09-16T20:49:23.541 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:23.541 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:23.541 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:23.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.542 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4758 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1521664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:23.542 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689141+0000", 2024-09-16T20:49:23.542 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 60.792347001000003, 2024-09-16T20:49:23.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048640741000000001, 2024-09-16T20:49:23.542 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:23.542 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:23.543 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:23.543 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:23.543 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:49:23.543 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:23.543 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4758 2024-09-16T20:49:23.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.544 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:23.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.544 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:23.544 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689141+0000", 2024-09-16T20:49:23.544 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:23.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.545 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:23.545 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689141+0000", 2024-09-16T20:49:23.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:23.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:23.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689141+0000", 2024-09-16T20:49:23.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8799999999999998e-07 2024-09-16T20:49:23.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:23.547 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689143+0000", 2024-09-16T20:49:23.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.945e-06 2024-09-16T20:49:23.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.547 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:23.547 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689149+0000", 2024-09-16T20:49:23.548 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9800000000000003e-06 2024-09-16T20:49:23.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.548 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.548 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:23.548 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689155+0000", 2024-09-16T20:49:23.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7509999999999997e-06 2024-09-16T20:49:23.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.549 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:23.549 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737681+0000", 2024-09-16T20:49:23.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048526825000000003 2024-09-16T20:49:23.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.550 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:23.550 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737729+0000", 2024-09-16T20:49:23.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7219999999999999e-05 2024-09-16T20:49:23.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.551 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:23.551 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737781+0000", 2024-09-16T20:49:23.551 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2632e-05 2024-09-16T20:49:23.551 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:23.551 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:23.552 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:23.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.552 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.552 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4760 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1529856~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:23.552 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689161+0000", 2024-09-16T20:49:23.552 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 60.792326654, 2024-09-16T20:49:23.553 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048743981999999998, 2024-09-16T20:49:23.553 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:23.553 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:23.553 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:23.553 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:23.553 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:49:23.553 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:23.554 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4760 2024-09-16T20:49:23.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.554 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:23.554 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.554 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:23.554 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689161+0000", 2024-09-16T20:49:23.554 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:23.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.554 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.554 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:23.554 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689161+0000", 2024-09-16T20:49:23.554 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:23.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:23.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689161+0000", 2024-09-16T20:49:23.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5599999999999995e-07 2024-09-16T20:49:23.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:23.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689163+0000", 2024-09-16T20:49:23.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.818e-06 2024-09-16T20:49:23.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:23.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689169+0000", 2024-09-16T20:49:23.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0809999999999999e-06 2024-09-16T20:49:23.556 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.556 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.556 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:23.556 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689174+0000", 2024-09-16T20:49:23.556 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.938e-06 2024-09-16T20:49:23.556 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.556 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.556 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:23.556 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737807+0000", 2024-09-16T20:49:23.556 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048633124 2024-09-16T20:49:23.556 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.556 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.556 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:23.557 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737851+0000", 2024-09-16T20:49:23.557 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3764000000000003e-05 2024-09-16T20:49:23.557 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.557 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.557 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:23.557 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737905+0000", 2024-09-16T20:49:23.557 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3701e-05 2024-09-16T20:49:23.557 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:23.557 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:23.557 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:23.557 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.557 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.558 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4762 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1538048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:23.558 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689182+0000", 2024-09-16T20:49:23.558 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 60.792305431999999, 2024-09-16T20:49:23.558 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048850275999999998, 2024-09-16T20:49:23.558 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:23.558 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:23.558 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:23.558 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:23.558 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:49:23.558 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:23.558 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4762 2024-09-16T20:49:23.558 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.558 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:23.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:23.559 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689182+0000", 2024-09-16T20:49:23.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:23.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:23.559 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689182+0000", 2024-09-16T20:49:23.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:23.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:23.559 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689182+0000", 2024-09-16T20:49:23.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-16T20:49:23.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.560 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:23.560 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689184+0000", 2024-09-16T20:49:23.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8190000000000001e-06 2024-09-16T20:49:23.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.560 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:23.560 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689191+0000", 2024-09-16T20:49:23.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2530000000000001e-06 2024-09-16T20:49:23.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.560 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:23.561 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689195+0000", 2024-09-16T20:49:23.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9559999999999996e-06 2024-09-16T20:49:23.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.561 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.561 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:23.561 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737933+0000", 2024-09-16T20:49:23.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048737919999999997 2024-09-16T20:49:23.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.561 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.561 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:23.561 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737982+0000", 2024-09-16T20:49:23.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8089e-05 2024-09-16T20:49:23.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.562 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:23.562 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738032+0000", 2024-09-16T20:49:23.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0868999999999997e-05 2024-09-16T20:49:23.562 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:23.562 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:23.562 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:23.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.562 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4764 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1546240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:23.562 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689201+0000", 2024-09-16T20:49:23.562 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 60.792286075, 2024-09-16T20:49:23.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048957550000000002, 2024-09-16T20:49:23.562 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:23.563 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:23.563 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:23.563 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:23.563 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:49:23.563 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:23.563 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4764 2024-09-16T20:49:23.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.563 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:23.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.563 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:23.563 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689201+0000", 2024-09-16T20:49:23.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:23.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:23.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689201+0000", 2024-09-16T20:49:23.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:23.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:23.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689202+0000", 2024-09-16T20:49:23.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4200000000000002e-07 2024-09-16T20:49:23.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:23.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689204+0000", 2024-09-16T20:49:23.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9750000000000001e-06 2024-09-16T20:49:23.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.565 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:23.565 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689210+0000", 2024-09-16T20:49:23.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1e-06 2024-09-16T20:49:23.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.565 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:23.565 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689215+0000", 2024-09-16T20:49:23.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0180000000000001e-06 2024-09-16T20:49:23.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.566 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:23.566 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738058+0000", 2024-09-16T20:49:23.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048843573000000001 2024-09-16T20:49:23.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.566 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.566 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:23.566 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738102+0000", 2024-09-16T20:49:23.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3256000000000001e-05 2024-09-16T20:49:23.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.566 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.566 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:23.566 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738159+0000", 2024-09-16T20:49:23.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7286e-05 2024-09-16T20:49:23.566 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:23.567 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:23.567 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:23.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.567 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4766 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1554432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:23.567 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689221+0000", 2024-09-16T20:49:23.567 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 60.792266109000003, 2024-09-16T20:49:23.567 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049064996, 2024-09-16T20:49:23.567 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:23.567 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:23.567 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:23.567 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:23.567 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:49:23.567 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:23.568 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4766 2024-09-16T20:49:23.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.568 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:23.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.568 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:23.568 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689221+0000", 2024-09-16T20:49:23.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:23.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.568 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:23.568 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689221+0000", 2024-09-16T20:49:23.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:23.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.569 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:23.569 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689222+0000", 2024-09-16T20:49:23.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.34e-07 2024-09-16T20:49:23.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.569 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:23.569 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689224+0000", 2024-09-16T20:49:23.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.846e-06 2024-09-16T20:49:23.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.569 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:23.569 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689247+0000", 2024-09-16T20:49:23.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3448e-05 2024-09-16T20:49:23.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.570 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:23.570 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689254+0000", 2024-09-16T20:49:23.570 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6969999999999996e-06 2024-09-16T20:49:23.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.570 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:23.570 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738188+0000", 2024-09-16T20:49:23.570 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048934520000000002 2024-09-16T20:49:23.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.570 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:23.570 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738234+0000", 2024-09-16T20:49:23.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5695999999999998e-05 2024-09-16T20:49:23.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.571 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.571 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:23.571 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738286+0000", 2024-09-16T20:49:23.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2355000000000003e-05 2024-09-16T20:49:23.571 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:23.571 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:23.571 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:23.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.571 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.571 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4768 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1562624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:23.571 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689262+0000", 2024-09-16T20:49:23.571 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 60.792225154999997, 2024-09-16T20:49:23.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049157157999999999, 2024-09-16T20:49:23.572 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:23.572 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:23.572 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:23.572 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:23.572 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:49:23.572 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:23.572 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4768 2024-09-16T20:49:23.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.572 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:23.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.572 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:23.573 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689262+0000", 2024-09-16T20:49:23.573 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:23.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.573 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.573 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:23.573 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689262+0000", 2024-09-16T20:49:23.573 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:23.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.573 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.573 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:23.573 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689263+0000", 2024-09-16T20:49:23.573 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6199999999999998e-07 2024-09-16T20:49:23.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.573 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.574 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:23.574 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689270+0000", 2024-09-16T20:49:23.574 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.798e-06 2024-09-16T20:49:23.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.574 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:23.574 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689277+0000", 2024-09-16T20:49:23.574 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0890000000000002e-06 2024-09-16T20:49:23.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.574 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:23.574 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689282+0000", 2024-09-16T20:49:23.574 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0470000000000001e-06 2024-09-16T20:49:23.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.575 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:23.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738319+0000", 2024-09-16T20:49:23.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049037139 2024-09-16T20:49:23.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.575 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:23.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738365+0000", 2024-09-16T20:49:23.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6356999999999997e-05 2024-09-16T20:49:23.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.575 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:23.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738420+0000", 2024-09-16T20:49:23.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4265999999999998e-05 2024-09-16T20:49:23.576 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:23.576 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:23.576 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:23.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.576 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4770 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1570816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:23.576 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689289+0000", 2024-09-16T20:49:23.576 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 60.792198644999999, 2024-09-16T20:49:23.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049253705000000002, 2024-09-16T20:49:23.576 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:23.576 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:23.576 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:23.577 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:23.577 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:49:23.577 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:23.577 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4770 2024-09-16T20:49:23.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.577 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:23.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.577 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:23.577 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689289+0000", 2024-09-16T20:49:23.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:23.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.577 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:23.578 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689289+0000", 2024-09-16T20:49:23.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:23.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.578 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:23.578 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689289+0000", 2024-09-16T20:49:23.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0999999999999999e-07 2024-09-16T20:49:23.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.578 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:23.578 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689291+0000", 2024-09-16T20:49:23.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0030000000000001e-06 2024-09-16T20:49:23.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:23.579 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689297+0000", 2024-09-16T20:49:23.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9479999999999996e-06 2024-09-16T20:49:23.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:23.579 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689302+0000", 2024-09-16T20:49:23.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7389999999999999e-06 2024-09-16T20:49:23.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:23.579 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738442+0000", 2024-09-16T20:49:23.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049140001000000003 2024-09-16T20:49:23.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:23.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738490+0000", 2024-09-16T20:49:23.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7787000000000002e-05 2024-09-16T20:49:23.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:23.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738543+0000", 2024-09-16T20:49:23.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2816999999999999e-05 2024-09-16T20:49:23.580 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:23.580 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:23.580 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:23.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.581 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:3088 3.1e 3:79e5ec01:::rbd_header.110e891a9b2b:head [watch watch cookie 140213437171488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:49:23.581 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:49:23.581 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 59.317642628000002, 2024-09-16T20:49:23.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.11895186200000001, 2024-09-16T20:49:23.581 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:23.581 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:23.581 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:23.581 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:23.582 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:23.582 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:23.582 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3088 2024-09-16T20:49:23.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.582 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:23.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:23.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:49:23.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:23.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:23.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:49:23.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:23.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:23.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163846+0000", 2024-09-16T20:49:23.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6799999999999999e-07 2024-09-16T20:49:23.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:23.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163850+0000", 2024-09-16T20:49:23.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.929e-06 2024-09-16T20:49:23.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.584 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:23.584 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163862+0000", 2024-09-16T20:49:23.584 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1972e-05 2024-09-16T20:49:23.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.584 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.584 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:23.584 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163867+0000", 2024-09-16T20:49:23.584 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.451e-06 2024-09-16T20:49:23.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.584 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.584 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:23.584 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163912+0000", 2024-09-16T20:49:23.584 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4456000000000003e-05 2024-09-16T20:49:23.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:23.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163962+0000", 2024-09-16T20:49:23.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0578000000000003e-05 2024-09-16T20:49:23.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-16T20:49:23.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164058+0000", 2024-09-16T20:49:23.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5934999999999995e-05 2024-09-16T20:49:23.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:49:23.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164570+0000", 2024-09-16T20:49:23.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00051241100000000005 2024-09-16T20:49:23.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:23.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282629+0000", 2024-09-16T20:49:23.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.118058986 2024-09-16T20:49:23.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:49:23.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282725+0000", 2024-09-16T20:49:23.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5063999999999996e-05 2024-09-16T20:49:23.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:23.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282797+0000", 2024-09-16T20:49:23.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2212000000000006e-05 2024-09-16T20:49:23.587 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:23.587 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:23.587 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:23.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.587 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:8039 3.1e 3.ac44ff9e (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:49:23.587 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:49:23.587 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 33.773815278999997, 2024-09-16T20:49:23.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0053310449999999, 2024-09-16T20:49:23.588 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:23.588 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:23.588 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:49:23.588 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:23.588 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:23.588 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:23.588 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8039 2024-09-16T20:49:23.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.588 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:23.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:23.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:49:23.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:23.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:23.589 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:49:23.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:23.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:23.589 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707674+0000", 2024-09-16T20:49:23.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6530000000000001e-06 2024-09-16T20:49:23.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:23.590 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707681+0000", 2024-09-16T20:49:23.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9639999999999997e-06 2024-09-16T20:49:23.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.590 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:23.590 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707700+0000", 2024-09-16T20:49:23.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8674000000000001e-05 2024-09-16T20:49:23.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.590 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:23.590 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707708+0000", 2024-09-16T20:49:23.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6810000000000004e-06 2024-09-16T20:49:23.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.591 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:23.591 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711293+0000", 2024-09-16T20:49:23.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035848189999999999 2024-09-16T20:49:23.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.591 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:49:23.591 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711301+0000", 2024-09-16T20:49:23.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4730000000000003e-06 2024-09-16T20:49:23.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.591 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:23.591 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.712990+0000", 2024-09-16T20:49:23.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.001689348 2024-09-16T20:49:23.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.592 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:23.592 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713003+0000", 2024-09-16T20:49:23.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3433e-05 2024-09-16T20:49:23.592 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:23.592 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:23.592 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:23.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.592 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:8040 3.1e 3.ac44ff9e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:49:23.593 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:49:23.593 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 33.773754883999999, 2024-09-16T20:49:23.593 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0053410780000001, 2024-09-16T20:49:23.593 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:23.593 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:23.593 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:49:23.593 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:23.593 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:23.593 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:23.593 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8040 2024-09-16T20:49:23.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.593 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:23.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.594 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:23.594 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:49:23.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:23.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.594 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:23.594 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:49:23.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:23.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.594 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:23.594 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:49:23.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8100000000000003e-07 2024-09-16T20:49:23.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.595 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:23.595 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707737+0000", 2024-09-16T20:49:23.595 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5020000000000001e-06 2024-09-16T20:49:23.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.595 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:23.595 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707745+0000", 2024-09-16T20:49:23.595 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2169999999999994e-06 2024-09-16T20:49:23.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.595 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:23.596 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707751+0000", 2024-09-16T20:49:23.596 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.263e-06 2024-09-16T20:49:23.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.596 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.596 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:23.596 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711336+0000", 2024-09-16T20:49:23.596 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035848360000000001 2024-09-16T20:49:23.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.596 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.596 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:49:23.596 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711340+0000", 2024-09-16T20:49:23.596 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1309999999999999e-06 2024-09-16T20:49:23.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.597 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.597 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:23.597 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713060+0000", 2024-09-16T20:49:23.597 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0017195059999999 2024-09-16T20:49:23.597 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.597 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.597 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:23.597 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713074+0000", 2024-09-16T20:49:23.597 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4042e-05 2024-09-16T20:49:23.597 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:23.597 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:23.597 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:23.597 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.598 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4927.0:4487 3.1e 3.ac44ff9e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:49:23.598 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:49:23.598 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 33.773234682000002, 2024-09-16T20:49:23.598 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0048840880000001, 2024-09-16T20:49:23.598 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:23.598 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:23.598 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:49:23.598 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:23.598 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4927", 2024-09-16T20:49:23.598 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:23.598 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4487 2024-09-16T20:49:23.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.599 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:23.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.599 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:23.599 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:49:23.599 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:23.599 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.599 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:23.599 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:49:23.599 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:23.599 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.599 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:23.599 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708254+0000", 2024-09-16T20:49:23.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1800000000000005e-07 2024-09-16T20:49:23.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.600 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.600 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:23.600 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708258+0000", 2024-09-16T20:49:23.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8689999999999997e-06 2024-09-16T20:49:23.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.600 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.600 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:23.600 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708267+0000", 2024-09-16T20:49:23.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9590000000000007e-06 2024-09-16T20:49:23.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.600 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.601 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:23.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708274+0000", 2024-09-16T20:49:23.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9970000000000001e-06 2024-09-16T20:49:23.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.601 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:23.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711366+0000", 2024-09-16T20:49:23.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030919530000000001 2024-09-16T20:49:23.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.601 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:49:23.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711372+0000", 2024-09-16T20:49:23.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9410000000000004e-06 2024-09-16T20:49:23.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.602 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:23.602 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713126+0000", 2024-09-16T20:49:23.602 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0017537249999999 2024-09-16T20:49:23.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.602 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:23.602 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713137+0000", 2024-09-16T20:49:23.602 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0825999999999999e-05 2024-09-16T20:49:23.602 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:23.602 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:23.602 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:23.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.603 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5089.0:11703 3.1b 3:dcd4684e:::rbd_data.110e747f0779.000000000000001b:head [set-alloc-hint object_size 262144 write_size 262144,write 139264~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:49:23.603 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:49:23.603 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.343498908000001, 2024-09-16T20:49:23.603 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.084231385000000006, 2024-09-16T20:49:23.603 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:23.603 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:23.603 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:23.603 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:23.603 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5089", 2024-09-16T20:49:23.603 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:49:23.603 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 11703 2024-09-16T20:49:23.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.604 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:23.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.604 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:23.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:49:23.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:23.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.604 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:23.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:49:23.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:23.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:23.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:49:23.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2600000000000001e-07 2024-09-16T20:49:23.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:23.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137998+0000", 2024-09-16T20:49:23.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9549999999999998e-06 2024-09-16T20:49:23.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:23.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138008+0000", 2024-09-16T20:49:23.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3789999999999995e-06 2024-09-16T20:49:23.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.606 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:23.606 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138013+0000", 2024-09-16T20:49:23.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2209999999999996e-06 2024-09-16T20:49:23.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.606 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:23.606 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138054+0000", 2024-09-16T20:49:23.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1022999999999997e-05 2024-09-16T20:49:23.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.606 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:23.606 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138124+0000", 2024-09-16T20:49:23.607 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0234999999999994e-05 2024-09-16T20:49:23.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.607 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:23.607 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138197+0000", 2024-09-16T20:49:23.607 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2451000000000001e-05 2024-09-16T20:49:23.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.607 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:23.607 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222010+0000", 2024-09-16T20:49:23.607 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.083813433000000007 2024-09-16T20:49:23.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.607 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:49:23.608 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222163+0000", 2024-09-16T20:49:23.608 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015274599999999999 2024-09-16T20:49:23.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.608 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:49:23.608 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222197+0000", 2024-09-16T20:49:23.608 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3683000000000003e-05 2024-09-16T20:49:23.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.608 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:23.608 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222220+0000", 2024-09-16T20:49:23.608 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3433000000000001e-05 2024-09-16T20:49:23.609 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:23.609 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:23.609 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:23.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.609 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.5089.0:11702 3.15 e73/64 3:acf33c8b:::rbd_data.110e747f0779.000000000000001a:head v 73'132, mlcod=73'130)", 2024-09-16T20:49:23.609 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:49:23.609 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.343231579000001, 2024-09-16T20:49:23.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.084079763000000002, 2024-09-16T20:49:23.609 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:23.609 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:23.609 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:23.609 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:23.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.610 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:23.610 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:49:23.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:23.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.610 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:23.610 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:49:23.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:23.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.610 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:23.610 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:49:23.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:49:23.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.611 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:23.611 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138263+0000", 2024-09-16T20:49:23.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1e-06 2024-09-16T20:49:23.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.611 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:23.611 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221589+0000", 2024-09-16T20:49:23.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.083326002999999996 2024-09-16T20:49:23.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.611 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:23.611 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221614+0000", 2024-09-16T20:49:23.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5593999999999999e-05 2024-09-16T20:49:23.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.612 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:23.612 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221714+0000", 2024-09-16T20:49:23.612 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9889000000000005e-05 2024-09-16T20:49:23.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.612 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:23.612 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221727+0000", 2024-09-16T20:49:23.612 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3125999999999999e-05 2024-09-16T20:49:23.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.612 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:49:23.613 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222294+0000", 2024-09-16T20:49:23.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00056712199999999998 2024-09-16T20:49:23.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:23.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:23.613 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:23.613 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222336+0000", 2024-09-16T20:49:23.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1470000000000001e-05 2024-09-16T20:49:23.613 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:23.613 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:23.613 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:23.613 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:23.613 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:23.613 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:23.615 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:49:23.615 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:23.614+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:23.715 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:23.714+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:23.762 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:23.763 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:49:23.763 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:49:23.763 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:23.771 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:49:23.816 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:23.815+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:23.879 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:23.879 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:49:23.879 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:49:23.880 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:49:23.880 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:23.888 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:49:23.917 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:23.916+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:23.920 INFO:teuthology.orchestra.run.smithi139.stdout:truncating image image_client.0-clone25 from 0x8a02800 (overlap 0x2abbc00) to 0x0 2024-09-16T20:49:24.000 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:24.000 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:49:24.000 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:49:24.000 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:49:24.000 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.000 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:4904 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 7309312~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:24.000 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767724+0000", 2024-09-16T20:49:24.000 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 64.230513028999994, 2024-09-16T20:49:24.000 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050270214000000001, 2024-09-16T20:49:24.000 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:24.001 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:24.001 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:24.001 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:24.001 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:24.001 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:24.001 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4904 2024-09-16T20:49:24.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.001 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:24.001 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.001 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:24.001 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767724+0000", 2024-09-16T20:49:24.001 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.001 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.002 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:24.002 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767724+0000", 2024-09-16T20:49:24.002 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.002 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:24.002 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767724+0000", 2024-09-16T20:49:24.002 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1399999999999998e-07 2024-09-16T20:49:24.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.002 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:24.002 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767725+0000", 2024-09-16T20:49:24.002 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0380000000000001e-06 2024-09-16T20:49:24.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.003 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.003 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:24.003 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767729+0000", 2024-09-16T20:49:24.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2490000000000001e-06 2024-09-16T20:49:24.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.003 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.003 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:24.003 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767731+0000", 2024-09-16T20:49:24.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5940000000000001e-06 2024-09-16T20:49:24.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.003 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.003 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:24.003 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.817787+0000", 2024-09-16T20:49:24.004 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050055792000000002 2024-09-16T20:49:24.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.004 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.004 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:24.004 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.817883+0000", 2024-09-16T20:49:24.004 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5606999999999995e-05 2024-09-16T20:49:24.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.004 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.004 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:24.004 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.817994+0000", 2024-09-16T20:49:24.004 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011162000000000001 2024-09-16T20:49:24.004 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.004 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:24.005 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.005 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.005 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.005 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:4906 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 7317504~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:24.005 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767734+0000", 2024-09-16T20:49:24.005 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 64.230502766000001, 2024-09-16T20:49:24.005 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050406608999999998, 2024-09-16T20:49:24.005 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:24.005 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:24.005 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:24.005 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:24.005 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:24.005 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:24.006 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4906 2024-09-16T20:49:24.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.006 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:24.006 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.006 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:24.006 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767734+0000", 2024-09-16T20:49:24.006 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.006 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.006 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:24.006 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767734+0000", 2024-09-16T20:49:24.006 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.006 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.007 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:24.007 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767735+0000", 2024-09-16T20:49:24.007 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.05e-07 2024-09-16T20:49:24.007 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.007 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.007 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:24.007 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767736+0000", 2024-09-16T20:49:24.007 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0240000000000001e-06 2024-09-16T20:49:24.007 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.007 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.007 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:24.007 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767739+0000", 2024-09-16T20:49:24.007 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.067e-06 2024-09-16T20:49:24.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.008 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:24.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767742+0000", 2024-09-16T20:49:24.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.23e-06 2024-09-16T20:49:24.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.008 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:24.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818027+0000", 2024-09-16T20:49:24.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050285102999999998 2024-09-16T20:49:24.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.008 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:24.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818072+0000", 2024-09-16T20:49:24.009 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5129000000000001e-05 2024-09-16T20:49:24.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.009 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.009 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:24.009 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818141+0000", 2024-09-16T20:49:24.009 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8850999999999995e-05 2024-09-16T20:49:24.009 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.009 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:24.009 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.009 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.009 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:4908 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 7325696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:24.009 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767745+0000", 2024-09-16T20:49:24.009 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 64.230492370999997, 2024-09-16T20:49:24.010 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050654230000000001, 2024-09-16T20:49:24.010 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:24.010 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:24.010 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:24.010 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:24.010 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:24.010 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:24.010 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4908 2024-09-16T20:49:24.010 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.010 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:24.010 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.010 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:24.010 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767745+0000", 2024-09-16T20:49:24.010 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.011 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.011 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:24.011 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767745+0000", 2024-09-16T20:49:24.011 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.011 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.011 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:24.011 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767745+0000", 2024-09-16T20:49:24.011 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1e-07 2024-09-16T20:49:24.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.011 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.011 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:24.012 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767749+0000", 2024-09-16T20:49:24.012 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.532e-06 2024-09-16T20:49:24.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.012 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.012 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:24.012 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767753+0000", 2024-09-16T20:49:24.012 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4939999999999999e-06 2024-09-16T20:49:24.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.012 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.012 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:24.012 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767756+0000", 2024-09-16T20:49:24.012 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.61e-06 2024-09-16T20:49:24.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.012 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.012 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:24.013 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818184+0000", 2024-09-16T20:49:24.013 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050428615000000003 2024-09-16T20:49:24.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.013 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.013 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:24.013 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818276+0000", 2024-09-16T20:49:24.013 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2077999999999995e-05 2024-09-16T20:49:24.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.013 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.013 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:24.013 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818399+0000", 2024-09-16T20:49:24.013 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012269099999999999 2024-09-16T20:49:24.013 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.013 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:24.014 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.014 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.014 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:4910 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 7333888~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:24.014 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767759+0000", 2024-09-16T20:49:24.014 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 64.230478481000006, 2024-09-16T20:49:24.014 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050815076000000001, 2024-09-16T20:49:24.014 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:24.014 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:24.014 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:24.014 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:24.015 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:24.015 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:24.015 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4910 2024-09-16T20:49:24.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.015 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:24.015 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.015 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:24.015 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767759+0000", 2024-09-16T20:49:24.015 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.015 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.015 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:24.015 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767759+0000", 2024-09-16T20:49:24.015 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.016 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.016 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:24.016 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767759+0000", 2024-09-16T20:49:24.016 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2600000000000001e-07 2024-09-16T20:49:24.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.016 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.016 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:24.016 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767760+0000", 2024-09-16T20:49:24.016 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0640000000000001e-06 2024-09-16T20:49:24.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.016 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.016 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:24.016 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767763+0000", 2024-09-16T20:49:24.017 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.994e-06 2024-09-16T20:49:24.017 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.017 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.017 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:24.017 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767765+0000", 2024-09-16T20:49:24.017 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5220000000000002e-06 2024-09-16T20:49:24.017 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.017 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.017 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:24.017 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818438+0000", 2024-09-16T20:49:24.017 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050672602999999997 2024-09-16T20:49:24.017 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.017 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.018 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:24.018 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818481+0000", 2024-09-16T20:49:24.018 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2967999999999999e-05 2024-09-16T20:49:24.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.018 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.018 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:24.018 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818574+0000", 2024-09-16T20:49:24.018 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2699000000000001e-05 2024-09-16T20:49:24.018 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.019 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:24.019 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.019 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5012 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2101248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:24.019 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767768+0000", 2024-09-16T20:49:24.019 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 64.230468832, 2024-09-16T20:49:24.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051084682999999999, 2024-09-16T20:49:24.019 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:24.019 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:24.019 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:24.019 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:24.020 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:24.020 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:24.020 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5012 2024-09-16T20:49:24.020 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.020 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:24.020 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.020 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:24.020 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767768+0000", 2024-09-16T20:49:24.020 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.020 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.020 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.020 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:24.020 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767768+0000", 2024-09-16T20:49:24.021 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.021 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.021 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:24.021 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767769+0000", 2024-09-16T20:49:24.021 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1500000000000001e-07 2024-09-16T20:49:24.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.021 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.021 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:24.021 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767770+0000", 2024-09-16T20:49:24.021 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.015e-06 2024-09-16T20:49:24.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.021 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.021 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:24.022 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767773+0000", 2024-09-16T20:49:24.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1070000000000001e-06 2024-09-16T20:49:24.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.022 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.022 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:24.022 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767775+0000", 2024-09-16T20:49:24.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.508e-06 2024-09-16T20:49:24.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.022 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.022 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:24.022 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818630+0000", 2024-09-16T20:49:24.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050855282000000002 2024-09-16T20:49:24.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.022 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.023 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:24.023 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818732+0000", 2024-09-16T20:49:24.023 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101531 2024-09-16T20:49:24.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.023 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.023 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:24.023 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818853+0000", 2024-09-16T20:49:24.023 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000121025 2024-09-16T20:49:24.023 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.023 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:24.023 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.023 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.024 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5014 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2109440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:24.024 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767778+0000", 2024-09-16T20:49:24.024 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 64.230458896000002, 2024-09-16T20:49:24.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051260776000000001, 2024-09-16T20:49:24.024 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:24.024 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:24.024 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:24.024 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:24.024 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:24.024 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:24.024 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5014 2024-09-16T20:49:24.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.024 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:24.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:24.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767778+0000", 2024-09-16T20:49:24.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:24.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767778+0000", 2024-09-16T20:49:24.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:24.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767824+0000", 2024-09-16T20:49:24.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5890999999999998e-05 2024-09-16T20:49:24.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.026 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:24.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767827+0000", 2024-09-16T20:49:24.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.644e-06 2024-09-16T20:49:24.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.026 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:24.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767832+0000", 2024-09-16T20:49:24.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2689999999999999e-06 2024-09-16T20:49:24.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.026 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:24.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767835+0000", 2024-09-16T20:49:24.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2100000000000002e-06 2024-09-16T20:49:24.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.027 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.027 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:24.027 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818887+0000", 2024-09-16T20:49:24.027 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051051276999999999 2024-09-16T20:49:24.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.027 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.027 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:24.027 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818953+0000", 2024-09-16T20:49:24.027 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6105999999999994e-05 2024-09-16T20:49:24.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.027 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.027 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:24.027 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819039+0000", 2024-09-16T20:49:24.028 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6378999999999994e-05 2024-09-16T20:49:24.028 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.028 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:24.028 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.028 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.028 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5016 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2117632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:24.028 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767841+0000", 2024-09-16T20:49:24.028 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 64.230396217000006, 2024-09-16T20:49:24.028 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051433288000000001, 2024-09-16T20:49:24.028 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:24.028 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:24.028 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:24.028 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:24.029 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:24.029 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:24.029 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5016 2024-09-16T20:49:24.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.029 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:24.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.029 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:24.029 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767841+0000", 2024-09-16T20:49:24.029 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.029 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:24.029 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767841+0000", 2024-09-16T20:49:24.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.030 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:24.030 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768156+0000", 2024-09-16T20:49:24.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00031505699999999999 2024-09-16T20:49:24.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.030 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:24.030 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768159+0000", 2024-09-16T20:49:24.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1659999999999998e-06 2024-09-16T20:49:24.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.030 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:24.031 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768166+0000", 2024-09-16T20:49:24.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0550000000000002e-06 2024-09-16T20:49:24.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.031 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.031 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:24.031 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768171+0000", 2024-09-16T20:49:24.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3170000000000003e-06 2024-09-16T20:49:24.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.031 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.031 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:24.031 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819084+0000", 2024-09-16T20:49:24.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050913355 2024-09-16T20:49:24.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.031 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.032 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:24.032 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819170+0000", 2024-09-16T20:49:24.032 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6328000000000004e-05 2024-09-16T20:49:24.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.032 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:24.032 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819274+0000", 2024-09-16T20:49:24.032 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010401 2024-09-16T20:49:24.032 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.032 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:24.032 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.032 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5018 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2125824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:24.033 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768178+0000", 2024-09-16T20:49:24.033 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 64.230058682999996, 2024-09-16T20:49:24.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051227013000000002, 2024-09-16T20:49:24.033 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:24.033 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:24.033 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:24.033 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:24.033 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:24.033 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:24.033 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5018 2024-09-16T20:49:24.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.033 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:24.033 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.034 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:24.034 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768178+0000", 2024-09-16T20:49:24.034 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.034 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.034 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:24.034 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768178+0000", 2024-09-16T20:49:24.034 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.034 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.034 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:24.034 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768179+0000", 2024-09-16T20:49:24.034 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9200000000000002e-07 2024-09-16T20:49:24.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.035 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:24.035 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768180+0000", 2024-09-16T20:49:24.035 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3790000000000001e-06 2024-09-16T20:49:24.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.035 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:24.035 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768184+0000", 2024-09-16T20:49:24.035 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5470000000000001e-06 2024-09-16T20:49:24.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.035 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:24.035 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768187+0000", 2024-09-16T20:49:24.035 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8119999999999999e-06 2024-09-16T20:49:24.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.036 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:24.036 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819305+0000", 2024-09-16T20:49:24.036 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051118683999999998 2024-09-16T20:49:24.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.036 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:24.036 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819351+0000", 2024-09-16T20:49:24.036 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5757999999999998e-05 2024-09-16T20:49:24.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.036 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:24.037 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819405+0000", 2024-09-16T20:49:24.037 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4441000000000003e-05 2024-09-16T20:49:24.037 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.037 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:24.037 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.037 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.037 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5020 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2134016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:24.037 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768190+0000", 2024-09-16T20:49:24.037 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 64.230047005000003, 2024-09-16T20:49:24.037 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051449661000000001, 2024-09-16T20:49:24.037 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:24.037 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:24.038 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:24.038 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:24.038 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:24.038 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:24.038 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5020 2024-09-16T20:49:24.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.038 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:24.038 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.038 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:24.038 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768190+0000", 2024-09-16T20:49:24.038 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.038 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.038 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:24.039 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768190+0000", 2024-09-16T20:49:24.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.039 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:24.039 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768282+0000", 2024-09-16T20:49:24.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2263999999999996e-05 2024-09-16T20:49:24.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.039 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:24.039 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768286+0000", 2024-09-16T20:49:24.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.529e-06 2024-09-16T20:49:24.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.040 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:24.040 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768294+0000", 2024-09-16T20:49:24.040 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6990000000000003e-06 2024-09-16T20:49:24.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.040 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.040 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:24.040 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768306+0000", 2024-09-16T20:49:24.040 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2112e-05 2024-09-16T20:49:24.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.040 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.040 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:24.040 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819449+0000", 2024-09-16T20:49:24.040 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051143521999999997 2024-09-16T20:49:24.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.041 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.041 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:24.041 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819539+0000", 2024-09-16T20:49:24.041 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0022e-05 2024-09-16T20:49:24.041 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.041 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.041 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:24.041 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819640+0000", 2024-09-16T20:49:24.041 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000100513 2024-09-16T20:49:24.041 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.041 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:24.041 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.041 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.041 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.042 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5022 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2142208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:24.042 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768326+0000", 2024-09-16T20:49:24.042 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 64.229911615999995, 2024-09-16T20:49:24.042 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051436952000000001, 2024-09-16T20:49:24.042 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:24.042 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:24.042 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:24.042 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:24.042 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:24.042 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:24.042 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5022 2024-09-16T20:49:24.042 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.042 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:24.043 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.043 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:24.043 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768326+0000", 2024-09-16T20:49:24.043 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.043 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.043 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:24.043 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768326+0000", 2024-09-16T20:49:24.043 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.043 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.043 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:24.043 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768477+0000", 2024-09-16T20:49:24.043 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015109300000000001 2024-09-16T20:49:24.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.044 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.044 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:24.044 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768480+0000", 2024-09-16T20:49:24.044 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8580000000000002e-06 2024-09-16T20:49:24.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.044 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.044 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:24.044 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768489+0000", 2024-09-16T20:49:24.044 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4600000000000003e-06 2024-09-16T20:49:24.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.044 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.044 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:24.045 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768493+0000", 2024-09-16T20:49:24.045 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2270000000000004e-06 2024-09-16T20:49:24.045 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.045 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.045 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:24.045 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819672+0000", 2024-09-16T20:49:24.045 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051178605000000002 2024-09-16T20:49:24.045 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.045 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.045 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:24.045 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819711+0000", 2024-09-16T20:49:24.045 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9357000000000003e-05 2024-09-16T20:49:24.045 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.045 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.046 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:24.046 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819762+0000", 2024-09-16T20:49:24.046 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1351999999999999e-05 2024-09-16T20:49:24.046 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.046 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:24.046 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.046 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.046 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.046 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5024 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2150400~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:24.046 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768501+0000", 2024-09-16T20:49:24.046 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 64.229736138000007, 2024-09-16T20:49:24.046 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051402678, 2024-09-16T20:49:24.046 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:24.047 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:24.047 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:24.047 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:24.047 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:24.047 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:24.047 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5024 2024-09-16T20:49:24.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.047 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:24.047 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.047 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:24.047 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768501+0000", 2024-09-16T20:49:24.047 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.047 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.048 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:24.048 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768501+0000", 2024-09-16T20:49:24.048 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.048 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.048 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:24.048 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768619+0000", 2024-09-16T20:49:24.048 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118187 2024-09-16T20:49:24.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.048 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.048 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:24.048 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768624+0000", 2024-09-16T20:49:24.048 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3259999999999997e-06 2024-09-16T20:49:24.049 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.049 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.049 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:24.049 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768631+0000", 2024-09-16T20:49:24.049 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6490000000000008e-06 2024-09-16T20:49:24.049 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.049 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.049 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:24.049 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768637+0000", 2024-09-16T20:49:24.049 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4210000000000003e-06 2024-09-16T20:49:24.049 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.049 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.049 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:24.049 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819800+0000", 2024-09-16T20:49:24.050 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051163486000000001 2024-09-16T20:49:24.050 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.050 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.050 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:24.050 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819845+0000", 2024-09-16T20:49:24.050 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4830999999999998e-05 2024-09-16T20:49:24.050 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.050 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.050 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:24.050 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819904+0000", 2024-09-16T20:49:24.050 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8777999999999999e-05 2024-09-16T20:49:24.050 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.050 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:24.051 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.051 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.051 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.051 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5026 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2158592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:24.051 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768646+0000", 2024-09-16T20:49:24.051 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 64.229590689999995, 2024-09-16T20:49:24.051 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051384386999999997, 2024-09-16T20:49:24.051 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:24.051 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:24.051 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:24.051 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:24.051 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:24.051 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:24.052 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5026 2024-09-16T20:49:24.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.052 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:24.052 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.052 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:24.052 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768646+0000", 2024-09-16T20:49:24.052 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.052 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.052 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:24.052 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768646+0000", 2024-09-16T20:49:24.052 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.053 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.053 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:24.053 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768775+0000", 2024-09-16T20:49:24.053 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012813200000000001 2024-09-16T20:49:24.053 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.053 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.053 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:24.053 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768778+0000", 2024-09-16T20:49:24.053 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2150000000000001e-06 2024-09-16T20:49:24.053 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.053 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.053 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:24.053 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768786+0000", 2024-09-16T20:49:24.054 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3259999999999996e-06 2024-09-16T20:49:24.054 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.054 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.054 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:24.054 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768790+0000", 2024-09-16T20:49:24.054 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.216e-06 2024-09-16T20:49:24.054 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.054 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.054 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:24.054 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819939+0000", 2024-09-16T20:49:24.054 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051149018999999997 2024-09-16T20:49:24.054 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.054 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.054 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:24.054 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819979+0000", 2024-09-16T20:49:24.055 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9311999999999999e-05 2024-09-16T20:49:24.055 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.055 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.055 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:24.055 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820031+0000", 2024-09-16T20:49:24.055 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2166999999999997e-05 2024-09-16T20:49:24.055 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.055 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:24.055 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.055 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.055 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.055 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5028 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2166784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:24.055 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768798+0000", 2024-09-16T20:49:24.056 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 64.229439047, 2024-09-16T20:49:24.056 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051388912000000002, 2024-09-16T20:49:24.056 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:24.056 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:24.056 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:24.056 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:24.056 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:24.056 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:24.056 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5028 2024-09-16T20:49:24.056 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.056 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:24.056 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.056 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:24.056 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768798+0000", 2024-09-16T20:49:24.057 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.057 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.057 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:24.057 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768798+0000", 2024-09-16T20:49:24.057 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.057 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.057 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:24.057 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768989+0000", 2024-09-16T20:49:24.057 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000190951 2024-09-16T20:49:24.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.057 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.057 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:24.058 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768993+0000", 2024-09-16T20:49:24.058 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9770000000000002e-06 2024-09-16T20:49:24.058 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.058 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.058 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:24.058 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.769002+0000", 2024-09-16T20:49:24.058 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5469999999999997e-06 2024-09-16T20:49:24.058 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.058 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.058 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:24.058 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.769020+0000", 2024-09-16T20:49:24.058 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8216999999999999e-05 2024-09-16T20:49:24.058 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.059 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.059 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:24.059 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820070+0000", 2024-09-16T20:49:24.059 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051049998999999999 2024-09-16T20:49:24.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.059 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.059 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:24.059 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820119+0000", 2024-09-16T20:49:24.059 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8962000000000003e-05 2024-09-16T20:49:24.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.059 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.059 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:24.059 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820187+0000", 2024-09-16T20:49:24.059 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8258999999999998e-05 2024-09-16T20:49:24.060 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.060 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:24.060 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.060 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.060 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.060 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5030 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2174976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:24.060 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.769031+0000", 2024-09-16T20:49:24.060 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 64.229205934000007, 2024-09-16T20:49:24.060 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051289830000000002, 2024-09-16T20:49:24.060 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:24.060 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:24.060 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:24.060 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:24.061 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:24.061 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:24.061 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5030 2024-09-16T20:49:24.061 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.061 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:24.061 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.061 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:24.061 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.769031+0000", 2024-09-16T20:49:24.061 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.061 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.061 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.061 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:24.061 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.769031+0000", 2024-09-16T20:49:24.061 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.062 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.062 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.062 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:24.062 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.773332+0000", 2024-09-16T20:49:24.062 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0043004039999999999 2024-09-16T20:49:24.062 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.062 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.062 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:24.062 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.773336+0000", 2024-09-16T20:49:24.062 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7849999999999999e-06 2024-09-16T20:49:24.062 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.062 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.062 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:24.063 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.773347+0000", 2024-09-16T20:49:24.063 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0482999999999999e-05 2024-09-16T20:49:24.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.063 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.063 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:24.063 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.773354+0000", 2024-09-16T20:49:24.063 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1559999999999998e-06 2024-09-16T20:49:24.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.063 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.063 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:24.063 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820216+0000", 2024-09-16T20:49:24.063 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046861591000000001 2024-09-16T20:49:24.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.063 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.064 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:24.064 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820258+0000", 2024-09-16T20:49:24.064 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2653e-05 2024-09-16T20:49:24.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.064 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:24.064 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820321+0000", 2024-09-16T20:49:24.064 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2757999999999998e-05 2024-09-16T20:49:24.064 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.064 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:24.064 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.064 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5067 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 6918144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:24.065 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.775279+0000", 2024-09-16T20:49:24.065 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 64.222957668000006, 2024-09-16T20:49:24.065 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050171387999999997, 2024-09-16T20:49:24.065 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:24.065 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:24.065 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:24.065 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:24.065 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:24.065 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:24.065 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5067 2024-09-16T20:49:24.065 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.065 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:24.065 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.066 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:24.066 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775279+0000", 2024-09-16T20:49:24.066 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.066 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.066 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.066 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:24.066 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775279+0000", 2024-09-16T20:49:24.066 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.066 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.066 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.066 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:24.066 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775432+0000", 2024-09-16T20:49:24.066 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015257399999999999 2024-09-16T20:49:24.066 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.067 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.067 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:24.067 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775436+0000", 2024-09-16T20:49:24.067 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0180000000000003e-06 2024-09-16T20:49:24.067 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.067 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.067 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:24.067 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775447+0000", 2024-09-16T20:49:24.067 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.077e-05 2024-09-16T20:49:24.067 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.067 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.067 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:24.067 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775456+0000", 2024-09-16T20:49:24.068 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4089999999999992e-06 2024-09-16T20:49:24.068 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.068 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.068 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:24.068 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825150+0000", 2024-09-16T20:49:24.068 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049693711000000002 2024-09-16T20:49:24.068 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.068 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.068 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:24.068 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825337+0000", 2024-09-16T20:49:24.068 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000187214 2024-09-16T20:49:24.068 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.068 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.068 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:24.069 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825451+0000", 2024-09-16T20:49:24.069 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011369199999999999 2024-09-16T20:49:24.069 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.069 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:24.069 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.069 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.069 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5068 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2330624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:24.069 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.775469+0000", 2024-09-16T20:49:24.069 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 64.222767707000003, 2024-09-16T20:49:24.069 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050198334999999997, 2024-09-16T20:49:24.069 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:24.069 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:24.070 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:24.070 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:24.070 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:24.070 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:24.070 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5068 2024-09-16T20:49:24.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.070 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:24.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.070 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:24.070 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775469+0000", 2024-09-16T20:49:24.070 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.070 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:24.070 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775469+0000", 2024-09-16T20:49:24.071 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.071 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.071 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:24.071 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775720+0000", 2024-09-16T20:49:24.071 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000250661 2024-09-16T20:49:24.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.071 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.071 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:24.071 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775724+0000", 2024-09-16T20:49:24.071 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8700000000000002e-06 2024-09-16T20:49:24.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.071 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.071 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:24.072 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775733+0000", 2024-09-16T20:49:24.072 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4239999999999999e-06 2024-09-16T20:49:24.072 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.072 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.072 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:24.072 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775740+0000", 2024-09-16T20:49:24.072 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9519999999999997e-06 2024-09-16T20:49:24.072 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.072 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.072 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:24.072 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825484+0000", 2024-09-16T20:49:24.072 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049743385000000001 2024-09-16T20:49:24.073 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.073 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.073 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:24.073 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825572+0000", 2024-09-16T20:49:24.073 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7834e-05 2024-09-16T20:49:24.073 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.073 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.073 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:24.073 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825668+0000", 2024-09-16T20:49:24.073 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6209e-05 2024-09-16T20:49:24.073 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.073 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:24.073 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.074 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.074 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5069 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 6926336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:24.074 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.775753+0000", 2024-09-16T20:49:24.074 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 64.222484484000006, 2024-09-16T20:49:24.074 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050146430999999998, 2024-09-16T20:49:24.074 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:24.074 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:24.074 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:24.074 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:24.074 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:24.074 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:24.074 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5069 2024-09-16T20:49:24.075 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.075 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:24.075 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.075 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:24.075 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775753+0000", 2024-09-16T20:49:24.075 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.075 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.075 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.075 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:24.075 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775753+0000", 2024-09-16T20:49:24.075 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.075 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.075 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.075 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:24.076 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775865+0000", 2024-09-16T20:49:24.076 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000112402 2024-09-16T20:49:24.076 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.076 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.076 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:24.076 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775869+0000", 2024-09-16T20:49:24.076 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5130000000000001e-06 2024-09-16T20:49:24.076 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.076 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.076 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:24.076 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775877+0000", 2024-09-16T20:49:24.076 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1359999999999997e-06 2024-09-16T20:49:24.076 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.076 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.077 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:24.077 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775882+0000", 2024-09-16T20:49:24.077 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6710000000000004e-06 2024-09-16T20:49:24.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.077 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.077 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:24.077 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825729+0000", 2024-09-16T20:49:24.077 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049846898000000001 2024-09-16T20:49:24.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.077 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.077 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:24.077 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825807+0000", 2024-09-16T20:49:24.077 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7875999999999998e-05 2024-09-16T20:49:24.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.078 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.078 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:24.078 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825899+0000", 2024-09-16T20:49:24.078 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1935000000000007e-05 2024-09-16T20:49:24.078 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.078 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:24.078 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.078 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.078 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.078 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5070 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2338816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:24.078 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.775892+0000", 2024-09-16T20:49:24.078 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 64.222344629999995, 2024-09-16T20:49:24.078 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050155311000000001, 2024-09-16T20:49:24.079 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:24.079 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:24.079 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:24.079 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:24.079 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:24.079 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:24.079 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5070 2024-09-16T20:49:24.079 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.079 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:24.079 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.079 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:24.079 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775892+0000", 2024-09-16T20:49:24.079 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.079 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.079 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.080 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:24.080 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775892+0000", 2024-09-16T20:49:24.080 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.080 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.080 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.080 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:24.080 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776142+0000", 2024-09-16T20:49:24.080 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00024950500000000002 2024-09-16T20:49:24.080 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.080 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.080 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:24.080 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776146+0000", 2024-09-16T20:49:24.080 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2819999999999998e-06 2024-09-16T20:49:24.081 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.081 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.081 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:24.081 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776157+0000", 2024-09-16T20:49:24.081 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0807e-05 2024-09-16T20:49:24.081 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.081 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.081 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:24.081 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776165+0000", 2024-09-16T20:49:24.081 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1300000000000001e-06 2024-09-16T20:49:24.081 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.081 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.081 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:24.082 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825935+0000", 2024-09-16T20:49:24.082 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049769300000000002 2024-09-16T20:49:24.082 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.082 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.082 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:24.082 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825991+0000", 2024-09-16T20:49:24.082 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6177000000000001e-05 2024-09-16T20:49:24.082 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.082 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.082 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:24.082 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826048+0000", 2024-09-16T20:49:24.082 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.711e-05 2024-09-16T20:49:24.082 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.082 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:24.083 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.083 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.084 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.084 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5071 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 6934528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:24.084 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.776177+0000", 2024-09-16T20:49:24.084 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 64.222060227, 2024-09-16T20:49:24.084 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050036886000000003, 2024-09-16T20:49:24.084 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:24.084 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:24.084 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:24.084 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:24.084 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:24.084 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:24.084 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5071 2024-09-16T20:49:24.085 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.085 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:24.085 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.085 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:24.085 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776177+0000", 2024-09-16T20:49:24.085 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.085 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.085 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.085 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:24.085 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776177+0000", 2024-09-16T20:49:24.085 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.085 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.085 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.085 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:24.086 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776339+0000", 2024-09-16T20:49:24.086 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016238099999999999 2024-09-16T20:49:24.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.086 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.086 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:24.086 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776343+0000", 2024-09-16T20:49:24.086 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9670000000000003e-06 2024-09-16T20:49:24.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.086 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.086 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:24.086 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776354+0000", 2024-09-16T20:49:24.086 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0789e-05 2024-09-16T20:49:24.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.086 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.087 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:24.087 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776361+0000", 2024-09-16T20:49:24.087 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5200000000000003e-06 2024-09-16T20:49:24.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.087 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.087 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:24.087 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826078+0000", 2024-09-16T20:49:24.087 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049717513999999997 2024-09-16T20:49:24.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.087 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.087 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:24.087 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826141+0000", 2024-09-16T20:49:24.087 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2446000000000005e-05 2024-09-16T20:49:24.088 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.088 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.088 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:24.088 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826214+0000", 2024-09-16T20:49:24.088 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3269000000000006e-05 2024-09-16T20:49:24.088 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.088 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:24.088 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.088 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.088 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.088 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5072 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2347008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:24.088 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.776370+0000", 2024-09-16T20:49:24.088 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 64.221866665999997, 2024-09-16T20:49:24.088 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050097653999999998, 2024-09-16T20:49:24.089 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:24.089 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:24.089 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:24.089 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:24.089 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:24.089 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:24.089 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5072 2024-09-16T20:49:24.089 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.089 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:24.089 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.089 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:24.089 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776370+0000", 2024-09-16T20:49:24.089 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.090 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.090 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.090 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:24.090 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776370+0000", 2024-09-16T20:49:24.090 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.090 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.090 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.090 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:24.090 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776645+0000", 2024-09-16T20:49:24.090 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000274634 2024-09-16T20:49:24.090 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.090 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.090 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:24.090 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776654+0000", 2024-09-16T20:49:24.090 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9390000000000003e-06 2024-09-16T20:49:24.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:24.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776665+0000", 2024-09-16T20:49:24.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0524e-05 2024-09-16T20:49:24.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:24.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776671+0000", 2024-09-16T20:49:24.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7390000000000002e-06 2024-09-16T20:49:24.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:24.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826261+0000", 2024-09-16T20:49:24.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049589995999999997 2024-09-16T20:49:24.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.092 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.092 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:24.092 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826363+0000", 2024-09-16T20:49:24.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010158999999999999 2024-09-16T20:49:24.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.092 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.092 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:24.092 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826468+0000", 2024-09-16T20:49:24.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000105232 2024-09-16T20:49:24.092 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.092 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:24.092 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.093 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.093 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:24.093 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:24.094 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:49:24.160 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:24.093+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:24.195 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:24.194+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:24.202 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:24.202 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:49:24.202 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.202 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5138.0:61 3.c 3.f0b864cc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-16T20:49:24.202 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:24.136346+0000", 2024-09-16T20:49:24.202 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.065610652000000005, 2024-09-16T20:49:24.202 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065630453000000005, 2024-09-16T20:49:24.203 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:24.203 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:24.203 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "waiting for peered", 2024-09-16T20:49:24.203 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:24.203 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5138", 2024-09-16T20:49:24.203 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:49:24.203 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 61 2024-09-16T20:49:24.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.203 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:24.203 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.203 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:24.203 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:24.136346+0000", 2024-09-16T20:49:24.203 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.204 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.204 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.204 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:24.204 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:24.136346+0000", 2024-09-16T20:49:24.204 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.204 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.204 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.204 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:24.204 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:24.136347+0000", 2024-09-16T20:49:24.204 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6899999999999996e-07 2024-09-16T20:49:24.204 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.204 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.204 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:24.204 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:24.136351+0000", 2024-09-16T20:49:24.205 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3970000000000004e-06 2024-09-16T20:49:24.205 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.205 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.205 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:24.205 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:24.136363+0000", 2024-09-16T20:49:24.205 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1884e-05 2024-09-16T20:49:24.205 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.205 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.205 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:24.205 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:24.136369+0000", 2024-09-16T20:49:24.205 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4140000000000002e-06 2024-09-16T20:49:24.205 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.205 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.205 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:24.206 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:24.136421+0000", 2024-09-16T20:49:24.206 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2003000000000003e-05 2024-09-16T20:49:24.206 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.206 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.206 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:49:24.206 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:24.136428+0000", 2024-09-16T20:49:24.206 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0110000000000003e-06 2024-09-16T20:49:24.206 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.206 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:24.206 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.206 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.206 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-16T20:49:24.206 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 1 2024-09-16T20:49:24.206 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:24.211 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:49:24.274 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T20:49:24.318 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:24.317+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:24.357 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:24.357 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:49:24.357 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:49:24.358 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:49:24.358 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:24.366 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:49:24.399 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:49:24.399 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:49:24.410 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:24.410+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:24.415 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:49:24.416 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:49:24.432 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:49:24.432 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:49:24.449 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:49:24.449 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:49:24.466 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:49:24.466 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:49:24.475 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:24.475 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:49:24.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:49:24.475 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:49:24.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.475 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5286.0:10643 3.17 3.f6273fb7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-16T20:49:24.476 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:49:24.476 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 15.683198595, 2024-09-16T20:49:24.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.76672347200000002, 2024-09-16T20:49:24.476 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:24.476 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:24.476 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:49:24.476 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:24.476 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5286", 2024-09-16T20:49:24.476 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/4135433989", 2024-09-16T20:49:24.476 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 10643 2024-09-16T20:49:24.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.476 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:24.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:24.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:49:24.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:24.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:49:24.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:24.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790489+0000", 2024-09-16T20:49:24.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5910000000000001e-06 2024-09-16T20:49:24.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:24.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790493+0000", 2024-09-16T20:49:24.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2669999999999998e-06 2024-09-16T20:49:24.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:24.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790500+0000", 2024-09-16T20:49:24.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3780000000000003e-06 2024-09-16T20:49:24.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:24.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790505+0000", 2024-09-16T20:49:24.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1560000000000003e-06 2024-09-16T20:49:24.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:24.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790559+0000", 2024-09-16T20:49:24.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3494e-05 2024-09-16T20:49:24.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:49:24.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790566+0000", 2024-09-16T20:49:24.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7649999999999993e-06 2024-09-16T20:49:24.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:24.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:09.557198+0000", 2024-09-16T20:49:24.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.76663136200000004 2024-09-16T20:49:24.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:24.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:09.557211+0000", 2024-09-16T20:49:24.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3458999999999999e-05 2024-09-16T20:49:24.480 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.480 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:24.480 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.480 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3863 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1232896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:24.480 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896432+0000", 2024-09-16T20:49:24.481 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 9.5772539109999997, 2024-09-16T20:49:24.481 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046223693000000003, 2024-09-16T20:49:24.481 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:24.481 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:24.481 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:24.481 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:24.481 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:24.481 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:24.481 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3863 2024-09-16T20:49:24.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.481 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:24.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.481 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:24.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896432+0000", 2024-09-16T20:49:24.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.482 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:24.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896432+0000", 2024-09-16T20:49:24.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.482 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:24.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896433+0000", 2024-09-16T20:49:24.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.89e-07 2024-09-16T20:49:24.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:24.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896435+0000", 2024-09-16T20:49:24.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.928e-06 2024-09-16T20:49:24.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:24.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896441+0000", 2024-09-16T20:49:24.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.896e-06 2024-09-16T20:49:24.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:24.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896445+0000", 2024-09-16T20:49:24.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7550000000000002e-06 2024-09-16T20:49:24.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:24.484 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.942468+0000", 2024-09-16T20:49:24.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046022278 2024-09-16T20:49:24.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:24.484 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.942553+0000", 2024-09-16T20:49:24.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5807e-05 2024-09-16T20:49:24.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.485 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:24.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.942656+0000", 2024-09-16T20:49:24.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010264000000000001 2024-09-16T20:49:24.485 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.485 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:24.485 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.486 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3864 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1236992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:24.486 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896451+0000", 2024-09-16T20:49:24.486 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 9.5772350409999998, 2024-09-16T20:49:24.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046415605999999998, 2024-09-16T20:49:24.486 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:24.486 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:24.486 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:24.486 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:24.486 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:24.486 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:24.486 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3864 2024-09-16T20:49:24.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.487 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:24.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:24.487 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896451+0000", 2024-09-16T20:49:24.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:24.487 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896451+0000", 2024-09-16T20:49:24.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:24.487 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896452+0000", 2024-09-16T20:49:24.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1699999999999999e-07 2024-09-16T20:49:24.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.488 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:24.488 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896453+0000", 2024-09-16T20:49:24.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6950000000000001e-06 2024-09-16T20:49:24.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.488 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:24.488 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896459+0000", 2024-09-16T20:49:24.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6749999999999999e-06 2024-09-16T20:49:24.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:24.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896464+0000", 2024-09-16T20:49:24.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1399999999999999e-06 2024-09-16T20:49:24.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:24.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.942695+0000", 2024-09-16T20:49:24.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046230965999999998 2024-09-16T20:49:24.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:24.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.942771+0000", 2024-09-16T20:49:24.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5859999999999995e-05 2024-09-16T20:49:24.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:24.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.942867+0000", 2024-09-16T20:49:24.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5952999999999994e-05 2024-09-16T20:49:24.490 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.490 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:24.490 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.490 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3866 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1241088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:24.490 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896471+0000", 2024-09-16T20:49:24.490 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 9.5772158110000003, 2024-09-16T20:49:24.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046590830999999999, 2024-09-16T20:49:24.491 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:24.491 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:24.491 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:24.491 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:24.491 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:24.491 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:24.491 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3866 2024-09-16T20:49:24.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.491 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:24.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:24.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896471+0000", 2024-09-16T20:49:24.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.492 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:24.492 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896471+0000", 2024-09-16T20:49:24.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.492 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:24.492 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896471+0000", 2024-09-16T20:49:24.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0900000000000002e-07 2024-09-16T20:49:24.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.492 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:24.493 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896473+0000", 2024-09-16T20:49:24.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8899999999999999e-06 2024-09-16T20:49:24.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.493 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:24.493 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896479+0000", 2024-09-16T20:49:24.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0179999999999998e-06 2024-09-16T20:49:24.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.493 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:24.493 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896486+0000", 2024-09-16T20:49:24.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7240000000000003e-06 2024-09-16T20:49:24.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.494 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:24.494 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.942924+0000", 2024-09-16T20:49:24.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046438266999999998 2024-09-16T20:49:24.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.494 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:24.494 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943001+0000", 2024-09-16T20:49:24.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6987e-05 2024-09-16T20:49:24.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.494 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:24.494 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943061+0000", 2024-09-16T20:49:24.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0535999999999998e-05 2024-09-16T20:49:24.494 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.495 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:24.495 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.495 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3868 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1245184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:24.495 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896492+0000", 2024-09-16T20:49:24.495 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 9.5771942600000006, 2024-09-16T20:49:24.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046738590000000003, 2024-09-16T20:49:24.495 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:24.495 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:24.495 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:24.495 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:24.495 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:24.495 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:24.496 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3868 2024-09-16T20:49:24.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.496 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:24.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:24.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896492+0000", 2024-09-16T20:49:24.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:24.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896492+0000", 2024-09-16T20:49:24.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:24.497 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896492+0000", 2024-09-16T20:49:24.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9499999999999998e-07 2024-09-16T20:49:24.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:24.497 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896494+0000", 2024-09-16T20:49:24.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.973e-06 2024-09-16T20:49:24.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:24.497 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896501+0000", 2024-09-16T20:49:24.498 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7719999999999997e-06 2024-09-16T20:49:24.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.498 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:24.498 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896506+0000", 2024-09-16T20:49:24.498 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2089999999999996e-06 2024-09-16T20:49:24.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.498 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:24.498 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943093+0000", 2024-09-16T20:49:24.498 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046586799999999998 2024-09-16T20:49:24.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.498 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:24.499 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943148+0000", 2024-09-16T20:49:24.499 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4385999999999997e-05 2024-09-16T20:49:24.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.499 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.499 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:24.499 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943231+0000", 2024-09-16T20:49:24.499 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3054999999999997e-05 2024-09-16T20:49:24.499 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.499 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:24.499 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.499 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.499 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3870 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1249280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:24.500 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896514+0000", 2024-09-16T20:49:24.500 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 9.577172633, 2024-09-16T20:49:24.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046893815999999998, 2024-09-16T20:49:24.500 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:24.500 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:24.500 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:24.500 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:24.500 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:24.500 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:24.500 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3870 2024-09-16T20:49:24.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.500 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:24.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:24.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896514+0000", 2024-09-16T20:49:24.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:24.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896514+0000", 2024-09-16T20:49:24.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:24.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896514+0000", 2024-09-16T20:49:24.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9299999999999999e-07 2024-09-16T20:49:24.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.502 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:24.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896521+0000", 2024-09-16T20:49:24.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8199999999999999e-06 2024-09-16T20:49:24.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.502 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:24.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896529+0000", 2024-09-16T20:49:24.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0120000000000005e-06 2024-09-16T20:49:24.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.502 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:24.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896534+0000", 2024-09-16T20:49:24.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5779999999999998e-06 2024-09-16T20:49:24.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:24.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943286+0000", 2024-09-16T20:49:24.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046751514000000001 2024-09-16T20:49:24.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:24.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943343+0000", 2024-09-16T20:49:24.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7216000000000002e-05 2024-09-16T20:49:24.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.504 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:24.504 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943407+0000", 2024-09-16T20:49:24.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4282999999999994e-05 2024-09-16T20:49:24.504 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.504 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:24.504 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.504 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3872 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1253376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:24.504 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896561+0000", 2024-09-16T20:49:24.504 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 9.5771252390000008, 2024-09-16T20:49:24.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.04705165, 2024-09-16T20:49:24.504 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:24.505 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:24.505 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:24.505 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:24.505 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:24.505 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:24.505 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3872 2024-09-16T20:49:24.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.505 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:24.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:24.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896561+0000", 2024-09-16T20:49:24.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:24.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896561+0000", 2024-09-16T20:49:24.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:24.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896561+0000", 2024-09-16T20:49:24.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4999999999999998e-07 2024-09-16T20:49:24.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:24.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896563+0000", 2024-09-16T20:49:24.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0140000000000001e-06 2024-09-16T20:49:24.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.507 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.507 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:24.507 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896569+0000", 2024-09-16T20:49:24.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6450000000000002e-06 2024-09-16T20:49:24.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.507 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.507 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:24.507 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896572+0000", 2024-09-16T20:49:24.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.174e-06 2024-09-16T20:49:24.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.507 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.507 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:24.508 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943456+0000", 2024-09-16T20:49:24.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046883552000000002 2024-09-16T20:49:24.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.508 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:24.508 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943531+0000", 2024-09-16T20:49:24.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5080000000000006e-05 2024-09-16T20:49:24.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.508 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:24.508 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943613+0000", 2024-09-16T20:49:24.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1835000000000005e-05 2024-09-16T20:49:24.508 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.509 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:24.509 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.509 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3874 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1257472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:24.509 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896577+0000", 2024-09-16T20:49:24.509 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 9.5771097239999996, 2024-09-16T20:49:24.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047187944000000002, 2024-09-16T20:49:24.509 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:24.509 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:24.509 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:24.509 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:24.509 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:24.509 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:24.510 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3874 2024-09-16T20:49:24.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.510 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:24.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:24.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896577+0000", 2024-09-16T20:49:24.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:24.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896577+0000", 2024-09-16T20:49:24.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:24.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896577+0000", 2024-09-16T20:49:24.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1199999999999999e-07 2024-09-16T20:49:24.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:24.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896578+0000", 2024-09-16T20:49:24.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.237e-06 2024-09-16T20:49:24.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:24.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896583+0000", 2024-09-16T20:49:24.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5859999999999998e-06 2024-09-16T20:49:24.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.512 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.512 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:24.512 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896587+0000", 2024-09-16T20:49:24.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3100000000000002e-06 2024-09-16T20:49:24.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.512 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.512 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:24.512 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943661+0000", 2024-09-16T20:49:24.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047073639 2024-09-16T20:49:24.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.512 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.512 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:24.513 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943712+0000", 2024-09-16T20:49:24.513 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1190999999999998e-05 2024-09-16T20:49:24.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.513 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:24.513 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943765+0000", 2024-09-16T20:49:24.513 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2769000000000002e-05 2024-09-16T20:49:24.513 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.513 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:24.513 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.513 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3876 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1261568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:24.514 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896593+0000", 2024-09-16T20:49:24.514 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 9.5770931459999993, 2024-09-16T20:49:24.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047362866000000003, 2024-09-16T20:49:24.514 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:24.514 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:24.514 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:24.514 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:24.514 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:24.514 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:24.514 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3876 2024-09-16T20:49:24.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.514 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:24.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.515 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:24.515 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896593+0000", 2024-09-16T20:49:24.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.515 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.515 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:24.515 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896593+0000", 2024-09-16T20:49:24.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.515 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.515 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:24.515 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896594+0000", 2024-09-16T20:49:24.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1600000000000002e-07 2024-09-16T20:49:24.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.516 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.516 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:24.516 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896595+0000", 2024-09-16T20:49:24.516 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.756e-06 2024-09-16T20:49:24.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.516 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.516 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:24.516 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896601+0000", 2024-09-16T20:49:24.516 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9309999999999996e-06 2024-09-16T20:49:24.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.516 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.516 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:24.517 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896606+0000", 2024-09-16T20:49:24.517 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1089999999999997e-06 2024-09-16T20:49:24.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.517 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:24.517 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943790+0000", 2024-09-16T20:49:24.517 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047183414 2024-09-16T20:49:24.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.517 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:24.517 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943869+0000", 2024-09-16T20:49:24.517 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9122000000000006e-05 2024-09-16T20:49:24.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.518 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:24.518 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943956+0000", 2024-09-16T20:49:24.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7118000000000002e-05 2024-09-16T20:49:24.518 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.518 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:24.518 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.518 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3879 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1265664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:24.518 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896613+0000", 2024-09-16T20:49:24.518 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 9.5770735039999995, 2024-09-16T20:49:24.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047528204999999997, 2024-09-16T20:49:24.518 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:24.519 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:24.519 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:24.519 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:24.519 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:24.519 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:24.519 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3879 2024-09-16T20:49:24.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.519 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:24.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.519 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:24.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896613+0000", 2024-09-16T20:49:24.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.520 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:24.520 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896613+0000", 2024-09-16T20:49:24.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.520 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:24.520 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896613+0000", 2024-09-16T20:49:24.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.84e-07 2024-09-16T20:49:24.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.520 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:24.520 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896615+0000", 2024-09-16T20:49:24.521 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9970000000000001e-06 2024-09-16T20:49:24.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.521 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.521 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:24.521 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896622+0000", 2024-09-16T20:49:24.521 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3500000000000002e-06 2024-09-16T20:49:24.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.521 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.521 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:24.521 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896627+0000", 2024-09-16T20:49:24.521 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2490000000000001e-06 2024-09-16T20:49:24.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.521 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.522 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:24.522 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943996+0000", 2024-09-16T20:49:24.522 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047369175999999999 2024-09-16T20:49:24.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.522 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:24.522 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944082+0000", 2024-09-16T20:49:24.522 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5859000000000006e-05 2024-09-16T20:49:24.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.522 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:24.522 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944141+0000", 2024-09-16T20:49:24.522 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9190000000000001e-05 2024-09-16T20:49:24.522 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.523 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:24.523 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.523 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3881 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1269760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:24.523 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896634+0000", 2024-09-16T20:49:24.523 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 9.5770525909999993, 2024-09-16T20:49:24.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047692834000000003, 2024-09-16T20:49:24.523 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:24.523 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:24.523 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:24.523 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:24.523 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:24.524 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:24.524 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3881 2024-09-16T20:49:24.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.524 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:24.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.524 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:24.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896634+0000", 2024-09-16T20:49:24.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.524 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:24.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896634+0000", 2024-09-16T20:49:24.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.525 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.525 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:24.525 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896634+0000", 2024-09-16T20:49:24.525 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-16T20:49:24.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.525 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.525 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:24.525 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896636+0000", 2024-09-16T20:49:24.525 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.216e-06 2024-09-16T20:49:24.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.525 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.525 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:24.525 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896640+0000", 2024-09-16T20:49:24.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1749999999999998e-06 2024-09-16T20:49:24.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.526 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.526 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:24.526 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896643+0000", 2024-09-16T20:49:24.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8779999999999998e-06 2024-09-16T20:49:24.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.526 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.526 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:24.526 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944178+0000", 2024-09-16T20:49:24.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047534633 2024-09-16T20:49:24.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.526 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.527 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:24.527 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944231+0000", 2024-09-16T20:49:24.527 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3338999999999999e-05 2024-09-16T20:49:24.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.527 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:24.527 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944327+0000", 2024-09-16T20:49:24.527 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5223000000000006e-05 2024-09-16T20:49:24.527 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.527 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:24.527 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.527 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3882 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1273856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:24.528 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896647+0000", 2024-09-16T20:49:24.528 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 9.5770394299999992, 2024-09-16T20:49:24.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047858886000000003, 2024-09-16T20:49:24.528 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:24.528 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:24.528 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:24.528 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:24.528 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:24.528 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:24.528 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3882 2024-09-16T20:49:24.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.528 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:24.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.529 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:24.529 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896647+0000", 2024-09-16T20:49:24.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.529 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:24.529 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896647+0000", 2024-09-16T20:49:24.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.529 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:24.529 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896647+0000", 2024-09-16T20:49:24.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4400000000000001e-07 2024-09-16T20:49:24.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.530 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:24.530 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896648+0000", 2024-09-16T20:49:24.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0499999999999999e-06 2024-09-16T20:49:24.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.530 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:24.530 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896651+0000", 2024-09-16T20:49:24.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9809999999999999e-06 2024-09-16T20:49:24.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.530 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:24.530 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896654+0000", 2024-09-16T20:49:24.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4439999999999998e-06 2024-09-16T20:49:24.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:24.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944361+0000", 2024-09-16T20:49:24.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047707408 2024-09-16T20:49:24.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:24.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944405+0000", 2024-09-16T20:49:24.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4107000000000003e-05 2024-09-16T20:49:24.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:24.532 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944506+0000", 2024-09-16T20:49:24.532 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000100652 2024-09-16T20:49:24.532 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.532 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:24.532 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.532 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3883 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1277952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:24.532 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896656+0000", 2024-09-16T20:49:24.532 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 9.5770299199999993, 2024-09-16T20:49:24.532 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047975265000000003, 2024-09-16T20:49:24.532 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:24.532 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:24.532 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:24.533 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:24.533 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:24.533 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:24.533 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3883 2024-09-16T20:49:24.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.533 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:24.533 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.533 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:24.533 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896656+0000", 2024-09-16T20:49:24.533 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.533 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.533 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:24.534 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896656+0000", 2024-09-16T20:49:24.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.534 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:24.534 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896657+0000", 2024-09-16T20:49:24.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7000000000000001e-07 2024-09-16T20:49:24.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.534 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:24.534 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896658+0000", 2024-09-16T20:49:24.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5850000000000001e-06 2024-09-16T20:49:24.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:24.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896664+0000", 2024-09-16T20:49:24.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.609e-06 2024-09-16T20:49:24.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:24.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896668+0000", 2024-09-16T20:49:24.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.121e-06 2024-09-16T20:49:24.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:24.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944542+0000", 2024-09-16T20:49:24.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047873838000000002 2024-09-16T20:49:24.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.536 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:24.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944579+0000", 2024-09-16T20:49:24.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6943000000000002e-05 2024-09-16T20:49:24.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.536 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:24.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944632+0000", 2024-09-16T20:49:24.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2899000000000003e-05 2024-09-16T20:49:24.536 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.536 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:24.536 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.537 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3886 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1282048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:24.537 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896674+0000", 2024-09-16T20:49:24.537 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 9.5770119509999994, 2024-09-16T20:49:24.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048152421000000001, 2024-09-16T20:49:24.537 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:24.537 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:24.537 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:24.537 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:24.537 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:24.537 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:24.537 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3886 2024-09-16T20:49:24.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.538 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:24.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.538 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:24.538 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896674+0000", 2024-09-16T20:49:24.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.538 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:24.538 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896674+0000", 2024-09-16T20:49:24.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.539 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:24.539 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896675+0000", 2024-09-16T20:49:24.539 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6100000000000002e-07 2024-09-16T20:49:24.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.539 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:24.539 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896677+0000", 2024-09-16T20:49:24.539 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1270000000000001e-06 2024-09-16T20:49:24.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.539 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:24.539 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896683+0000", 2024-09-16T20:49:24.539 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1970000000000001e-06 2024-09-16T20:49:24.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:24.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896688+0000", 2024-09-16T20:49:24.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.605e-06 2024-09-16T20:49:24.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:24.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944676+0000", 2024-09-16T20:49:24.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047988391999999998 2024-09-16T20:49:24.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:24.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944763+0000", 2024-09-16T20:49:24.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7363999999999999e-05 2024-09-16T20:49:24.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.541 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:24.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944827+0000", 2024-09-16T20:49:24.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3374999999999995e-05 2024-09-16T20:49:24.541 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.541 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:24.541 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.541 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3888 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1286144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:24.542 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896693+0000", 2024-09-16T20:49:24.542 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 9.5769928269999998, 2024-09-16T20:49:24.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048336443999999999, 2024-09-16T20:49:24.542 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:24.542 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:24.542 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:24.542 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:24.542 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:24.542 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:24.542 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3888 2024-09-16T20:49:24.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.542 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:24.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:24.543 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896693+0000", 2024-09-16T20:49:24.543 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.543 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.543 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:24.543 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896693+0000", 2024-09-16T20:49:24.543 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.543 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.543 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:24.543 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896694+0000", 2024-09-16T20:49:24.543 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3200000000000001e-07 2024-09-16T20:49:24.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.543 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.544 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:24.544 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896696+0000", 2024-09-16T20:49:24.544 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8989999999999999e-06 2024-09-16T20:49:24.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.544 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:24.544 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896702+0000", 2024-09-16T20:49:24.544 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8819999999999998e-06 2024-09-16T20:49:24.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.544 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:24.544 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896707+0000", 2024-09-16T20:49:24.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1529999999999996e-06 2024-09-16T20:49:24.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.545 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:24.545 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944863+0000", 2024-09-16T20:49:24.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048156658999999997 2024-09-16T20:49:24.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.545 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:24.545 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944944+0000", 2024-09-16T20:49:24.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0090000000000001e-05 2024-09-16T20:49:24.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:24.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945030+0000", 2024-09-16T20:49:24.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6428999999999995e-05 2024-09-16T20:49:24.546 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.546 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:24.546 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.546 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3890 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1290240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:24.546 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896713+0000", 2024-09-16T20:49:24.546 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 9.5769729570000006, 2024-09-16T20:49:24.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048538382999999997, 2024-09-16T20:49:24.546 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:24.546 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:24.547 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:24.547 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:24.547 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:24.547 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:24.547 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3890 2024-09-16T20:49:24.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.547 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:24.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.547 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:24.547 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896713+0000", 2024-09-16T20:49:24.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.547 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:24.548 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896713+0000", 2024-09-16T20:49:24.548 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.548 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.548 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:24.548 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896714+0000", 2024-09-16T20:49:24.548 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2000000000000001e-07 2024-09-16T20:49:24.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.548 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.548 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:24.548 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896716+0000", 2024-09-16T20:49:24.548 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0109999999999999e-06 2024-09-16T20:49:24.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.549 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:24.549 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896720+0000", 2024-09-16T20:49:24.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9049999999999999e-06 2024-09-16T20:49:24.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.549 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:24.549 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896722+0000", 2024-09-16T20:49:24.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.627e-06 2024-09-16T20:49:24.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.549 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:24.549 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945075+0000", 2024-09-16T20:49:24.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048353149999999998 2024-09-16T20:49:24.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.550 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:24.550 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945157+0000", 2024-09-16T20:49:24.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1972999999999996e-05 2024-09-16T20:49:24.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.550 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:24.550 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945252+0000", 2024-09-16T20:49:24.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4396999999999998e-05 2024-09-16T20:49:24.550 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.550 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:24.550 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.551 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3893 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1294336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:24.551 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896726+0000", 2024-09-16T20:49:24.551 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 9.5769601000000009, 2024-09-16T20:49:24.551 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048658686999999999, 2024-09-16T20:49:24.551 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:24.551 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:24.551 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:24.551 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:24.551 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:24.551 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:24.551 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3893 2024-09-16T20:49:24.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.552 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:24.552 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.552 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:24.552 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896726+0000", 2024-09-16T20:49:24.552 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.552 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.552 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:24.552 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896726+0000", 2024-09-16T20:49:24.552 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.552 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.552 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:24.553 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896727+0000", 2024-09-16T20:49:24.553 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7899999999999999e-07 2024-09-16T20:49:24.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.553 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:24.553 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896728+0000", 2024-09-16T20:49:24.553 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.68e-06 2024-09-16T20:49:24.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.553 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:24.553 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896734+0000", 2024-09-16T20:49:24.553 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9190000000000004e-06 2024-09-16T20:49:24.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.554 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.554 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:24.554 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896739+0000", 2024-09-16T20:49:24.554 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5549999999999996e-06 2024-09-16T20:49:24.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.554 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.554 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:24.554 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945271+0000", 2024-09-16T20:49:24.554 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048531795000000003 2024-09-16T20:49:24.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.554 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.554 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:24.554 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945330+0000", 2024-09-16T20:49:24.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9386999999999999e-05 2024-09-16T20:49:24.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:24.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945385+0000", 2024-09-16T20:49:24.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4871999999999999e-05 2024-09-16T20:49:24.555 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.555 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:24.555 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.555 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3895 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1298432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:24.555 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896754+0000", 2024-09-16T20:49:24.556 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 9.5769320259999997, 2024-09-16T20:49:24.556 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048754440000000003, 2024-09-16T20:49:24.556 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:24.556 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:24.556 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:24.556 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:24.556 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:24.556 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:24.556 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3895 2024-09-16T20:49:24.556 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.556 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:24.556 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.556 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:24.556 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896754+0000", 2024-09-16T20:49:24.557 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.557 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.557 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.557 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:24.557 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896754+0000", 2024-09-16T20:49:24.557 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.557 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.557 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.557 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:24.557 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896755+0000", 2024-09-16T20:49:24.557 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5200000000000002e-07 2024-09-16T20:49:24.557 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.557 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.558 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:24.558 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896762+0000", 2024-09-16T20:49:24.558 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8430000000000004e-06 2024-09-16T20:49:24.558 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.558 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.558 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:24.558 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896782+0000", 2024-09-16T20:49:24.558 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0693e-05 2024-09-16T20:49:24.558 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.558 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.558 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:24.558 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896793+0000", 2024-09-16T20:49:24.558 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0294e-05 2024-09-16T20:49:24.558 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:24.559 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945418+0000", 2024-09-16T20:49:24.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048625738000000002 2024-09-16T20:49:24.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:24.559 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945457+0000", 2024-09-16T20:49:24.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9103999999999997e-05 2024-09-16T20:49:24.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:24.559 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945509+0000", 2024-09-16T20:49:24.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1316000000000001e-05 2024-09-16T20:49:24.560 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.560 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:24.560 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.560 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3896 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1302528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:24.560 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896806+0000", 2024-09-16T20:49:24.560 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 9.5768798139999998, 2024-09-16T20:49:24.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048831159999999998, 2024-09-16T20:49:24.560 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:24.560 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:24.560 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:24.561 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:24.561 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:24.561 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:24.561 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3896 2024-09-16T20:49:24.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.561 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:24.561 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.561 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:24.561 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896806+0000", 2024-09-16T20:49:24.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.561 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.561 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:24.561 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896806+0000", 2024-09-16T20:49:24.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.562 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:24.562 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896807+0000", 2024-09-16T20:49:24.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3e-07 2024-09-16T20:49:24.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.562 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:24.562 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896811+0000", 2024-09-16T20:49:24.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8310000000000003e-06 2024-09-16T20:49:24.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.562 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:24.563 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896819+0000", 2024-09-16T20:49:24.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2470000000000008e-06 2024-09-16T20:49:24.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.563 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:24.563 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896823+0000", 2024-09-16T20:49:24.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7919999999999999e-06 2024-09-16T20:49:24.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.563 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:24.563 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945535+0000", 2024-09-16T20:49:24.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048712137000000003 2024-09-16T20:49:24.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:24.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945571+0000", 2024-09-16T20:49:24.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6261999999999999e-05 2024-09-16T20:49:24.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:24.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945638+0000", 2024-09-16T20:49:24.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6161000000000006e-05 2024-09-16T20:49:24.564 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.564 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:24.564 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.565 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3899 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1306624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:24.565 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896829+0000", 2024-09-16T20:49:24.565 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 9.5768576129999996, 2024-09-16T20:49:24.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049014110999999999, 2024-09-16T20:49:24.565 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:24.565 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:24.565 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:24.565 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:24.565 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:24.565 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:24.565 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3899 2024-09-16T20:49:24.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.565 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:24.566 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.566 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:24.566 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896829+0000", 2024-09-16T20:49:24.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.566 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.566 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:24.566 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896829+0000", 2024-09-16T20:49:24.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:24.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.566 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.566 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:24.566 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896829+0000", 2024-09-16T20:49:24.567 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8999999999999998e-07 2024-09-16T20:49:24.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.567 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:24.567 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896831+0000", 2024-09-16T20:49:24.567 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.77e-06 2024-09-16T20:49:24.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.567 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:24.567 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896834+0000", 2024-09-16T20:49:24.567 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4800000000000001e-06 2024-09-16T20:49:24.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.567 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:24.568 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896837+0000", 2024-09-16T20:49:24.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8559999999999998e-06 2024-09-16T20:49:24.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.568 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:24.568 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945663+0000", 2024-09-16T20:49:24.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048825529999999999 2024-09-16T20:49:24.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.568 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:24.568 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945746+0000", 2024-09-16T20:49:24.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3621000000000005e-05 2024-09-16T20:49:24.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:24.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:24.569 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:24.569 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945843+0000", 2024-09-16T20:49:24.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6564000000000006e-05 2024-09-16T20:49:24.569 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.569 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:24.569 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.569 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:24.569 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:24.569 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:24.570 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:49:24.571 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:24.570+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:24.572 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:49:24.572 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:49:24.632 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #24, image image_client.0-clone24 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent25 2024-09-16T20:49:24.671 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:24.670+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:24.686 INFO:teuthology.orchestra.run.smithi139.stdout:3252 read 0x77c0c00 thru 0x77c311f (0x2520 bytes) 2024-09-16T20:49:24.686 INFO:teuthology.orchestra.run.smithi139.stdout:3254 write 0x1bee600 thru 0x1bf7097 (0x8a98 bytes) 2024-09-16T20:49:24.688 INFO:teuthology.orchestra.run.smithi139.stdout:3255 punch from 0x13e8a00 to 0x13ea400, (0x1a00 bytes) 2024-09-16T20:49:24.724 INFO:teuthology.orchestra.run.smithi139.stdout:3258 read 0x5aa9200 thru 0x5ab6fc6 (0xddc7 bytes) 2024-09-16T20:49:24.724 INFO:teuthology.orchestra.run.smithi139.stdout:3259 read 0x7adc00 thru 0x7ae345 (0x746 bytes) 2024-09-16T20:49:24.725 INFO:teuthology.orchestra.run.smithi139.stdout:3260 punch from 0x7d5d000 to 0x7d65e00, (0x8e00 bytes) 2024-09-16T20:49:24.729 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:24.729 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:24.738 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:49:24.747 INFO:teuthology.orchestra.run.smithi139.stdout:3262 read 0x2258c00 thru 0x225ca14 (0x3e15 bytes) 2024-09-16T20:49:24.772 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:24.772+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:24.843 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T20:49:24.857 INFO:teuthology.orchestra.run.smithi139.stdout:3265 write 0xe795000 thru 0xe79b38e (0x638f bytes) 2024-09-16T20:49:24.857 INFO:teuthology.orchestra.run.smithi139.stdout:3266 punch from 0xb76c000 to 0xb774800, (0x8800 bytes) 2024-09-16T20:49:24.858 INFO:teuthology.orchestra.run.smithi139.stdout:3267 read 0x2c49800 thru 0x2c4e4a8 (0x4ca9 bytes) 2024-09-16T20:49:24.858 INFO:teuthology.orchestra.run.smithi139.stdout:3271 punch from 0x648d800 to 0x6499800, (0xc000 bytes) 2024-09-16T20:49:24.860 INFO:teuthology.orchestra.run.smithi139.stdout:3273 read 0x9b12800 thru 0x9b1d845 (0xb046 bytes) 2024-09-16T20:49:24.860 INFO:teuthology.orchestra.run.smithi139.stdout:3274 write 0x893c00 thru 0x898795 (0x4b96 bytes) 2024-09-16T20:49:24.878 INFO:teuthology.orchestra.run.smithi139.stdout:3277 trunc from 0xe79b400 to 0x74b4c00 2024-09-16T20:49:24.887 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:24.886+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:24.901 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:24.901 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:24.915 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:49:24.933 INFO:teuthology.orchestra.run.smithi139.stdout:3279 read 0x4d1fe00 thru 0x4d29159 (0x935a bytes) 2024-09-16T20:49:24.937 INFO:teuthology.orchestra.run.smithi139.stdout:3280 trunc from 0x74b4c00 to 0x4f01a00 2024-09-16T20:49:24.974 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:24.974+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:24.994 INFO:teuthology.orchestra.run.smithi139.stdout:3282 trunc from 0x4f01a00 to 0x40d1a00 2024-09-16T20:49:25.075 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:25.075+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:25.091 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:25.091 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:25.100 INFO:teuthology.orchestra.run.smithi139.stdout:3283 write 0xce5e400 thru 0xce6ccfa (0xe8fb bytes) 2024-09-16T20:49:25.100 INFO:teuthology.orchestra.run.smithi139.stdout:3285 read 0x1a0f800 thru 0x1a173d3 (0x7bd4 bytes) 2024-09-16T20:49:25.101 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:49:25.101 INFO:teuthology.orchestra.run.smithi139.stdout:3288 write 0x18a3400 thru 0x18ad1e0 (0x9de1 bytes) 2024-09-16T20:49:25.102 INFO:teuthology.orchestra.run.smithi139.stdout:3289 write 0x8c5ac00 thru 0x8c6857c (0xd97d bytes) 2024-09-16T20:49:25.102 INFO:teuthology.orchestra.run.smithi139.stdout:3290 punch from 0x2935a00 to 0x2943a00, (0xe000 bytes) 2024-09-16T20:49:25.104 INFO:teuthology.orchestra.run.smithi139.stdout:3291 read 0x1e4c800 thru 0x1e4e528 (0x1d29 bytes) 2024-09-16T20:49:25.112 INFO:teuthology.orchestra.run.smithi139.stdout:3294 read 0x6d36200 thru 0x6d389a8 (0x27a9 bytes) 2024-09-16T20:49:25.112 INFO:teuthology.orchestra.run.smithi139.stdout:3295 write 0xa8a0600 thru 0xa8a58e9 (0x52ea bytes) 2024-09-16T20:49:25.112 INFO:teuthology.orchestra.run.smithi139.stdout:3296 write 0x795fe00 thru 0x7968f02 (0x9103 bytes) 2024-09-16T20:49:25.126 INFO:teuthology.orchestra.run.smithi139.stdout:3297 trunc from 0xce6ce00 to 0x2509800 2024-09-16T20:49:25.176 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:25.176+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:25.198 INFO:teuthology.orchestra.run.smithi139.stdout:3298 write 0x5033a00 thru 0x503e015 (0xa616 bytes) 2024-09-16T20:49:25.198 INFO:teuthology.orchestra.run.smithi139.stdout:3301 read 0x371de00 thru 0x372d409 (0xf60a bytes) 2024-09-16T20:49:25.198 INFO:teuthology.orchestra.run.smithi139.stdout:3302 read 0x24fe800 thru 0x250d3a2 (0xeba3 bytes) 2024-09-16T20:49:25.199 INFO:teuthology.orchestra.run.smithi139.stdout:3303 read 0x48c2800 thru 0x48cbe46 (0x9647 bytes) 2024-09-16T20:49:25.199 INFO:teuthology.orchestra.run.smithi139.stdout:3304 punch from 0x1143600 to 0x1147600, (0x4000 bytes) 2024-09-16T20:49:25.199 INFO:teuthology.orchestra.run.smithi139.stdout:3305 read 0x30e3400 thru 0x30ebfe9 (0x8bea bytes) 2024-09-16T20:49:25.203 INFO:teuthology.orchestra.run.smithi139.stdout:3306 read 0x3539200 thru 0x3548bf9 (0xf9fa bytes) 2024-09-16T20:49:25.203 INFO:teuthology.orchestra.run.smithi139.stdout:3307 punch from 0x4ecc600 to 0x4edba00, (0xf400 bytes) 2024-09-16T20:49:25.216 INFO:teuthology.orchestra.run.smithi139.stdout:3308 write 0x5869e00 thru 0x5877ad7 (0xdcd8 bytes) 2024-09-16T20:49:25.223 INFO:teuthology.orchestra.run.smithi139.stdout:3310 read 0xafc800 thru 0xb07272 (0xaa73 bytes) 2024-09-16T20:49:25.224 INFO:teuthology.orchestra.run.smithi139.stdout:3312 read 0x3848600 thru 0x3848617 (0x18 bytes) 2024-09-16T20:49:25.224 INFO:teuthology.orchestra.run.smithi139.stdout:3313 read 0x550ee00 thru 0x5517d6d (0x8f6e bytes) 2024-09-16T20:49:25.225 INFO:teuthology.orchestra.run.smithi139.stdout:3315 write 0x45f1800 thru 0x45f7e7a (0x667b bytes) 2024-09-16T20:49:25.234 INFO:teuthology.orchestra.run.smithi139.stdout:3316 trunc from 0x5877c00 to 0xd3a9000 2024-09-16T20:49:25.236 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:25.236 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:25.246 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:49:25.247 INFO:teuthology.orchestra.run.smithi139.stdout:3318 read 0x884e400 thru 0x885a66a (0xc26b bytes) 2024-09-16T20:49:25.259 INFO:teuthology.orchestra.run.smithi139.stdout:3322 trunc from 0xd3a9000 to 0x388ce00 2024-09-16T20:49:25.277 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:25.276+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:25.287 INFO:teuthology.orchestra.run.smithi139.stdout:3323 write 0x6ef4a00 thru 0x6eff218 (0xa819 bytes) 2024-09-16T20:49:25.304 INFO:teuthology.orchestra.run.smithi139.stdout:3324 write 0xab8d000 thru 0xab94d7e (0x7d7f bytes) 2024-09-16T20:49:25.304 INFO:teuthology.orchestra.run.smithi139.stdout:3325 read 0x37b3200 thru 0x37b59c5 (0x27c6 bytes) 2024-09-16T20:49:25.304 INFO:teuthology.orchestra.run.smithi139.stdout:3326 punch from 0x1e01c00 to 0x1e0cc00, (0xb000 bytes) 2024-09-16T20:49:25.305 INFO:teuthology.orchestra.run.smithi139.stdout:3328 write 0xa9d9600 thru 0xa9dd335 (0x3d36 bytes) 2024-09-16T20:49:25.305 INFO:teuthology.orchestra.run.smithi139.stdout:3329 read 0x99eb400 thru 0x99efa9c (0x469d bytes) 2024-09-16T20:49:25.305 INFO:teuthology.orchestra.run.smithi139.stdout:3330 punch from 0x660ae00 to 0x6613600, (0x8800 bytes) 2024-09-16T20:49:25.306 INFO:teuthology.orchestra.run.smithi139.stdout:3331 trunc from 0xab94e00 to 0xa1fd800 2024-09-16T20:49:25.321 INFO:teuthology.orchestra.run.smithi139.stdout:3333 punch from 0x1177200 to 0x117bc00, (0x4a00 bytes) 2024-09-16T20:49:25.322 INFO:teuthology.orchestra.run.smithi139.stdout:3334 write 0x978600 thru 0x982bb0 (0xa5b1 bytes) 2024-09-16T20:49:25.331 INFO:teuthology.orchestra.run.smithi139.stdout:3335 trunc from 0xa1fd800 to 0xdeb8e00 2024-09-16T20:49:25.365 INFO:teuthology.orchestra.run.smithi139.stdout:3336 read 0x2730c00 thru 0x2737d37 (0x7138 bytes) 2024-09-16T20:49:25.366 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:25.366 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:25.367 INFO:teuthology.orchestra.run.smithi139.stdout:3337 read 0x8aec200 thru 0x8afaa4b (0xe84c bytes) 2024-09-16T20:49:25.367 INFO:teuthology.orchestra.run.smithi139.stdout:3339 read 0x1924600 thru 0x192c5b5 (0x7fb6 bytes) 2024-09-16T20:49:25.368 INFO:teuthology.orchestra.run.smithi139.stdout:3343 punch from 0x8f68a00 to 0x8f69a00, (0x1000 bytes) 2024-09-16T20:49:25.375 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:49:25.378 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:25.377+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:25.380 INFO:teuthology.orchestra.run.smithi139.stdout:3346 trunc from 0xdeb8e00 to 0x3649200 2024-09-16T20:49:25.413 INFO:teuthology.orchestra.run.smithi139.stdout:3352 write 0xb1a9a00 thru 0xb1aa09e (0x69f bytes) 2024-09-16T20:49:25.414 INFO:teuthology.orchestra.run.smithi139.stdout:3353 write 0xa0ba600 thru 0xa0bb9a0 (0x13a1 bytes) 2024-09-16T20:49:25.414 INFO:teuthology.orchestra.run.smithi139.stdout:3357 read 0xa95ce00 thru 0xa961d26 (0x4f27 bytes) 2024-09-16T20:49:25.427 INFO:teuthology.orchestra.run.smithi139.stdout:3363 write 0xb5b0200 thru 0xb5bd534 (0xd335 bytes) 2024-09-16T20:49:25.428 INFO:teuthology.orchestra.run.smithi139.stdout:3365 punch from 0x6109000 to 0x6115e00, (0xce00 bytes) 2024-09-16T20:49:25.432 INFO:teuthology.orchestra.run.smithi139.stdout:3366 read 0x1d79200 thru 0x1d7bcee (0x2aef bytes) 2024-09-16T20:49:25.432 INFO:teuthology.orchestra.run.smithi139.stdout:3367 punch from 0x333c200 to 0x3342200, (0x6000 bytes) 2024-09-16T20:49:25.444 INFO:teuthology.orchestra.run.smithi139.stdout:3371 write 0xe39f000 thru 0xe3ac0b7 (0xd0b8 bytes) 2024-09-16T20:49:25.446 INFO:teuthology.orchestra.run.smithi139.stdout:3374 read 0x3811000 thru 0x3814431 (0x3432 bytes) 2024-09-16T20:49:25.446 INFO:teuthology.orchestra.run.smithi139.stdout:3377 write 0x9957800 thru 0x9966589 (0xed8a bytes) 2024-09-16T20:49:25.446 INFO:teuthology.orchestra.run.smithi139.stdout:3378 read 0xdff6600 thru 0xe005938 (0xf339 bytes) 2024-09-16T20:49:25.446 INFO:teuthology.orchestra.run.smithi139.stdout:3379 read 0x9a59800 thru 0x9a62ea9 (0x96aa bytes) 2024-09-16T20:49:25.446 INFO:teuthology.orchestra.run.smithi139.stdout:3380 read 0x1e31800 thru 0x1e35cdc (0x44dd bytes) 2024-09-16T20:49:25.455 INFO:teuthology.orchestra.run.smithi139.stdout:3382 trunc from 0xe3ac200 to 0x69b3000 2024-09-16T20:49:25.478 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:25.477+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:25.488 INFO:teuthology.orchestra.run.smithi139.stdout:3385 trunc from 0x69b3000 to 0xe409800 2024-09-16T20:49:25.492 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:25.492 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:25.501 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:49:25.507 INFO:teuthology.orchestra.run.smithi139.stdout:3386 write 0xebfe400 thru 0xec096a0 (0xb2a1 bytes) 2024-09-16T20:49:25.513 INFO:teuthology.orchestra.run.smithi139.stdout:3388 trunc from 0xec09800 to 0xb1c0600 2024-09-16T20:49:25.529 INFO:teuthology.orchestra.run.smithi139.stdout:3390 punch from 0x6900200 to 0x6905000, (0x4e00 bytes) 2024-09-16T20:49:25.538 INFO:teuthology.orchestra.run.smithi139.stdout:3391 write 0x5b9f800 thru 0x5ba7514 (0x7d15 bytes) 2024-09-16T20:49:25.538 INFO:teuthology.orchestra.run.smithi139.stdout:3394 write 0x3369e00 thru 0x336d1eb (0x33ec bytes) 2024-09-16T20:49:25.538 INFO:teuthology.orchestra.run.smithi139.stdout:3398 read 0x7d28c00 thru 0x7d2e070 (0x5471 bytes) 2024-09-16T20:49:25.538 INFO:teuthology.orchestra.run.smithi139.stdout:3399 read 0x8e65200 thru 0x8e67351 (0x2152 bytes) 2024-09-16T20:49:25.549 INFO:teuthology.orchestra.run.smithi139.stdout:3403 trunc from 0xb1c0600 to 0x1fd5a00 2024-09-16T20:49:25.571 INFO:teuthology.orchestra.run.smithi139.stdout:3404 read 0x19ef600 thru 0x19ef76e (0x16f bytes) 2024-09-16T20:49:25.573 INFO:teuthology.orchestra.run.smithi139.stdout:3405 read 0x7d4c00 thru 0x7d4fdd (0x3de bytes) 2024-09-16T20:49:25.573 INFO:teuthology.orchestra.run.smithi139.stdout:3407 punch from 0x968c00 to 0x974600, (0xba00 bytes) 2024-09-16T20:49:25.574 INFO:teuthology.orchestra.run.smithi139.stdout:3408 trunc from 0x1fd5a00 to 0x22b5600 2024-09-16T20:49:25.587 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:25.586+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:25.611 INFO:teuthology.orchestra.run.smithi139.stdout:3409 write 0x420f400 thru 0x42193e2 (0x9fe3 bytes) 2024-09-16T20:49:25.612 INFO:teuthology.orchestra.run.smithi139.stdout:3410 write 0x258c200 thru 0x258e999 (0x279a bytes) 2024-09-16T20:49:25.614 INFO:teuthology.orchestra.run.smithi139.stdout:3411 trunc from 0x4219400 to 0x5bc2800 2024-09-16T20:49:25.627 INFO:teuthology.orchestra.run.smithi139.stdout:3415 read 0x3b7ca00 thru 0x3b7ea4d (0x204e bytes) 2024-09-16T20:49:25.627 INFO:teuthology.orchestra.run.smithi139.stdout:3417 write 0x382c800 thru 0x3832515 (0x5d16 bytes) 2024-09-16T20:49:25.631 INFO:teuthology.orchestra.run.smithi139.stdout:3419 trunc from 0x5bc2800 to 0x2c8ec00 2024-09-16T20:49:25.652 INFO:teuthology.orchestra.run.smithi139.stdout:3420 read 0x20ec600 thru 0x20f2a34 (0x6435 bytes) 2024-09-16T20:49:25.657 INFO:teuthology.orchestra.run.smithi139.stdout:3421 read 0x2258600 thru 0x2261492 (0x8e93 bytes) 2024-09-16T20:49:25.657 INFO:teuthology.orchestra.run.smithi139.stdout:3422 punch from 0x26aca00 to 0x26bba00, (0xf000 bytes) 2024-09-16T20:49:25.659 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:25.659 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:25.668 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:49:25.680 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:25.679+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:25.680 INFO:teuthology.orchestra.run.smithi139.stdout:3423 write 0xe827c00 thru 0xe82c81c (0x4c1d bytes) 2024-09-16T20:49:25.688 INFO:teuthology.orchestra.run.smithi139.stdout:3425 read 0x7e3be00 thru 0x7e4789a (0xba9b bytes) 2024-09-16T20:49:25.688 INFO:teuthology.orchestra.run.smithi139.stdout:3427 write 0x5d4fa00 thru 0x5d5682d (0x6e2e bytes) 2024-09-16T20:49:25.705 INFO:teuthology.orchestra.run.smithi139.stdout:3428 trunc from 0xe82ca00 to 0x237e00 2024-09-16T20:49:25.745 INFO:teuthology.orchestra.run.smithi139.stdout:3431 punch from 0x181600 to 0x18bc00, (0xa600 bytes) 2024-09-16T20:49:25.747 INFO:teuthology.orchestra.run.smithi139.stdout:3433 read 0x99600 thru 0xa89a4 (0xf3a5 bytes) 2024-09-16T20:49:25.759 INFO:teuthology.orchestra.run.smithi139.stdout:3434 write 0x23cfc00 thru 0x23d8f89 (0x938a bytes) 2024-09-16T20:49:25.776 INFO:teuthology.orchestra.run.smithi139.stdout:3435 trunc from 0x23d9000 to 0xaea8c00 2024-09-16T20:49:25.781 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:25.780+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:25.788 INFO:teuthology.orchestra.run.smithi139.stdout:3436 write 0x2a7c000 thru 0x2a7f0c1 (0x30c2 bytes) 2024-09-16T20:49:25.789 INFO:teuthology.orchestra.run.smithi139.stdout:3445 punch from 0x3df2200 to 0x3df3200, (0x1000 bytes) 2024-09-16T20:49:25.790 INFO:teuthology.orchestra.run.smithi139.stdout:3449 read 0x6e68800 thru 0x6e7754a (0xed4b bytes) 2024-09-16T20:49:25.790 INFO:teuthology.orchestra.run.smithi139.stdout:3450 write 0x4cb9600 thru 0x4cbebf7 (0x55f8 bytes) 2024-09-16T20:49:25.790 INFO:teuthology.orchestra.run.smithi139.stdout:3451 read 0x797a000 thru 0x7981a8f (0x7a90 bytes) 2024-09-16T20:49:25.790 INFO:teuthology.orchestra.run.smithi139.stdout:3453 read 0x95c0800 thru 0x95ce8eb (0xe0ec bytes) 2024-09-16T20:49:25.806 INFO:teuthology.orchestra.run.smithi139.stdout:3455 write 0xebafa00 thru 0xebb61a5 (0x67a6 bytes) 2024-09-16T20:49:25.807 INFO:teuthology.orchestra.run.smithi139.stdout:3456 read 0x72b3a00 thru 0x72c1c8f (0xe290 bytes) 2024-09-16T20:49:25.807 INFO:teuthology.orchestra.run.smithi139.stdout:3458 read 0xea04400 thru 0xea12685 (0xe286 bytes) 2024-09-16T20:49:25.807 INFO:teuthology.orchestra.run.smithi139.stdout:3459 punch from 0xce9e00 to 0xcf2800, (0x8a00 bytes) 2024-09-16T20:49:25.812 INFO:teuthology.orchestra.run.smithi139.stdout:3460 punch from 0x29b3a00 to 0x29c1000, (0xd600 bytes) 2024-09-16T20:49:25.812 INFO:teuthology.orchestra.run.smithi139.stdout:3463 read 0xa5d4200 thru 0xa5d6157 (0x1f58 bytes) 2024-09-16T20:49:25.812 INFO:teuthology.orchestra.run.smithi139.stdout:3466 write 0x47b6400 thru 0x47ba155 (0x3d56 bytes) 2024-09-16T20:49:25.814 INFO:teuthology.orchestra.run.smithi139.stdout:3467 trunc from 0xebb6200 to 0xd275000 2024-09-16T20:49:25.829 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:25.829 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:25.836 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 3, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-16T20:49:25.836 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T20:49:25.836 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-16T20:49:25.836 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-16T20:49:25.838 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:49:25.840 INFO:teuthology.orchestra.run.smithi139.stdout:3468 trunc from 0xd275000 to 0x328de00 2024-09-16T20:49:25.841 INFO:teuthology.orchestra.run.smithi029.stderr:noscrub is unset 2024-09-16T20:49:25.878 INFO:teuthology.orchestra.run.smithi139.stdout:3470 write 0x4133e00 thru 0x41423ec (0xe5ed bytes) 2024-09-16T20:49:25.882 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:25.881+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:25.885 INFO:teuthology.orchestra.run.smithi139.stdout:3471 punch from 0x3cc1000 to 0x3cc3c00, (0x2c00 bytes) 2024-09-16T20:49:25.888 INFO:teuthology.orchestra.run.smithi139.stdout:3472 trunc from 0x4142400 to 0x64fd600 2024-09-16T20:49:25.898 INFO:teuthology.orchestra.run.smithi139.stdout:3473 read 0x14ad000 thru 0x14b0bde (0x3bdf bytes) 2024-09-16T20:49:25.911 INFO:teuthology.orchestra.run.smithi139.stdout:3474 write 0x8d7ec00 thru 0x8d84e9f (0x62a0 bytes) 2024-09-16T20:49:25.912 INFO:teuthology.orchestra.run.smithi139.stdout:3475 read 0x2e3d200 thru 0x2e48568 (0xb369 bytes) 2024-09-16T20:49:25.919 INFO:teuthology.orchestra.run.smithi139.stdout:3476 trunc from 0x8d85000 to 0xeb10a00 2024-09-16T20:49:25.932 INFO:teuthology.orchestra.run.smithi139.stdout:3477 read 0x351b400 thru 0x3526d66 (0xb967 bytes) 2024-09-16T20:49:25.936 INFO:teuthology.orchestra.run.smithi139.stdout:3483 trunc from 0xeb10a00 to 0xccaae00 2024-09-16T20:49:25.944 INFO:teuthology.orchestra.run.smithi139.stdout:3486 write 0x303d200 thru 0x30460a6 (0x8ea7 bytes) 2024-09-16T20:49:25.946 INFO:teuthology.orchestra.run.smithi139.stdout:3487 read 0x52fc800 thru 0x5305a3d (0x923e bytes) 2024-09-16T20:49:25.947 INFO:teuthology.orchestra.run.smithi139.stdout:3488 punch from 0x5064e00 to 0x5074600, (0xf800 bytes) 2024-09-16T20:49:25.947 INFO:teuthology.orchestra.run.smithi139.stdout:3493 punch from 0x9bd0a00 to 0x9bd0e00, (0x400 bytes) 2024-09-16T20:49:25.947 INFO:teuthology.orchestra.run.smithi139.stdout:3496 punch from 0x3175400 to 0x317ac00, (0x5800 bytes) 2024-09-16T20:49:25.947 INFO:teuthology.orchestra.run.smithi139.stdout:3497 punch from 0x2777a00 to 0x2783a00, (0xc000 bytes) 2024-09-16T20:49:25.960 INFO:teuthology.orchestra.run.smithi139.stdout:3498 trunc from 0xccaae00 to 0xd2e400 2024-09-16T20:49:25.962 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:25.962 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:25.975 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:49:25.983 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:25.981+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:25.999 INFO:teuthology.orchestra.run.smithi139.stdout:3500 write 0x1d28600 thru 0x1d36b6d (0xe56e bytes) 2024-09-16T20:49:26.005 INFO:teuthology.orchestra.run.smithi139.stdout:3501 read 0x1bd7200 thru 0x1be57e2 (0xe5e3 bytes) 2024-09-16T20:49:26.005 INFO:teuthology.orchestra.run.smithi139.stdout:3502 read 0x110c400 thru 0x1116755 (0xa356 bytes) 2024-09-16T20:49:26.005 INFO:teuthology.orchestra.run.smithi139.stdout:3504 read 0x7f3200 thru 0x7fbff4 (0x8df5 bytes) 2024-09-16T20:49:26.030 INFO:teuthology.orchestra.run.smithi139.stdout:3506 write 0xa6f8200 thru 0xa706955 (0xe756 bytes) 2024-09-16T20:49:26.037 INFO:teuthology.orchestra.run.smithi139.stdout:3508 read 0x5246400 thru 0x5248276 (0x1e77 bytes) 2024-09-16T20:49:26.053 INFO:teuthology.orchestra.run.smithi139.stdout:3509 write 0xdc8c600 thru 0xdc8f365 (0x2d66 bytes) 2024-09-16T20:49:26.058 INFO:teuthology.orchestra.run.smithi139.stdout:3511 trunc from 0xdc8f400 to 0x9c85400 2024-09-16T20:49:26.079 INFO:teuthology.orchestra.run.smithi139.stdout:3514 read 0x9a40c00 thru 0x9a49dca (0x91cb bytes) 2024-09-16T20:49:26.083 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:26.082+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:26.085 INFO:teuthology.orchestra.run.smithi139.stdout:3515 write 0x4dc6200 thru 0x4dcf206 (0x9007 bytes) 2024-09-16T20:49:26.085 INFO:teuthology.orchestra.run.smithi139.stdout:3516 punch from 0x166ea00 to 0x1679200, (0xa800 bytes) 2024-09-16T20:49:26.089 INFO:teuthology.orchestra.run.smithi139.stdout:3517 trunc from 0x9c85400 to 0xd2fba00 2024-09-16T20:49:26.097 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:26.097 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:26.099 INFO:teuthology.orchestra.run.smithi139.stdout:3518 read 0xaac8000 thru 0xaad3e1d (0xbe1e bytes) 2024-09-16T20:49:26.106 INFO:teuthology.orchestra.run.smithi139.stdout:3520 read 0x3f4d600 thru 0x3f5558b (0x7f8c bytes) 2024-09-16T20:49:26.106 INFO:teuthology.orchestra.run.smithi139.stdout:3521 read 0x6516400 thru 0x651ed37 (0x8938 bytes) 2024-09-16T20:49:26.107 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:49:26.112 INFO:teuthology.orchestra.run.smithi139.stdout:3522 trunc from 0xd2fba00 to 0x7b9f600 2024-09-16T20:49:26.125 INFO:teuthology.orchestra.run.smithi139.stdout:3528 read 0x4bdd200 thru 0x4beb340 (0xe141 bytes) 2024-09-16T20:49:26.132 INFO:teuthology.orchestra.run.smithi139.stdout:3529 read 0xe07600 thru 0xe14e93 (0xd894 bytes) 2024-09-16T20:49:26.133 INFO:teuthology.orchestra.run.smithi139.stdout:3530 write 0x49df400 thru 0x49e9c0b (0xa80c bytes) 2024-09-16T20:49:26.133 INFO:teuthology.orchestra.run.smithi139.stdout:3534 punch from 0x2675c00 to 0x2677200, (0x1600 bytes) 2024-09-16T20:49:26.133 INFO:teuthology.orchestra.run.smithi139.stdout:3535 write 0x42d4600 thru 0x42e08c3 (0xc2c4 bytes) 2024-09-16T20:49:26.134 INFO:teuthology.orchestra.run.smithi139.stdout:3536 read 0x3850800 thru 0x386047b (0xfc7c bytes) 2024-09-16T20:49:26.134 INFO:teuthology.orchestra.run.smithi139.stdout:3537 read 0x56ea800 thru 0x56f0831 (0x6032 bytes) 2024-09-16T20:49:26.134 INFO:teuthology.orchestra.run.smithi139.stdout:3538 read 0x56afa00 thru 0x56bc2eb (0xc8ec bytes) 2024-09-16T20:49:26.134 INFO:teuthology.orchestra.run.smithi139.stdout:3539 read 0xe03800 thru 0xe0bddb (0x85dc bytes) 2024-09-16T20:49:26.135 INFO:teuthology.orchestra.run.smithi139.stdout:3540 write 0x4157a00 thru 0x415ec77 (0x7278 bytes) 2024-09-16T20:49:26.154 INFO:teuthology.orchestra.run.smithi139.stdout:3542 write 0xd8d9800 thru 0xd8e86c4 (0xeec5 bytes) 2024-09-16T20:49:26.161 INFO:teuthology.orchestra.run.smithi139.stdout:3543 read 0xcaf2200 thru 0xcafc4b2 (0xa2b3 bytes) 2024-09-16T20:49:26.161 INFO:teuthology.orchestra.run.smithi139.stdout:3544 read 0x3112c00 thru 0x311c3ca (0x97cb bytes) 2024-09-16T20:49:26.168 INFO:teuthology.orchestra.run.smithi139.stdout:3546 trunc from 0xd8e8800 to 0x83ce600 2024-09-16T20:49:26.173 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:49:26.173 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:26.173 INFO:teuthology.orchestra.run.smithi029.stdout: "epoch": 93, 2024-09-16T20:49:26.173 INFO:teuthology.orchestra.run.smithi029.stdout: "fsid": "e28af4a7-5fcd-4de8-8f77-a2b575361bb1", 2024-09-16T20:49:26.173 INFO:teuthology.orchestra.run.smithi029.stdout: "created": "2024-09-16T20:47:39.760609+0000", 2024-09-16T20:49:26.173 INFO:teuthology.orchestra.run.smithi029.stdout: "modified": "2024-09-16T20:49:25.838278+0000", 2024-09-16T20:49:26.173 INFO:teuthology.orchestra.run.smithi029.stdout: "last_up_change": "2024-09-16T20:47:50.238013+0000", 2024-09-16T20:49:26.174 INFO:teuthology.orchestra.run.smithi029.stdout: "last_in_change": "2024-09-16T20:49:08.779548+0000", 2024-09-16T20:49:26.174 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-16T20:49:26.174 INFO:teuthology.orchestra.run.smithi029.stdout: "flags_num": 5804032, 2024-09-16T20:49:26.174 INFO:teuthology.orchestra.run.smithi029.stdout: "flags_set": [ 2024-09-16T20:49:26.174 INFO:teuthology.orchestra.run.smithi029.stdout: "nodeep-scrub", 2024-09-16T20:49:26.174 INFO:teuthology.orchestra.run.smithi029.stdout: "pglog_hardlimit", 2024-09-16T20:49:26.174 INFO:teuthology.orchestra.run.smithi029.stdout: "purged_snapdirs", 2024-09-16T20:49:26.174 INFO:teuthology.orchestra.run.smithi029.stdout: "recovery_deletes", 2024-09-16T20:49:26.174 INFO:teuthology.orchestra.run.smithi029.stdout: "sortbitwise" 2024-09-16T20:49:26.174 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-16T20:49:26.174 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_version": 5, 2024-09-16T20:49:26.174 INFO:teuthology.orchestra.run.smithi029.stdout: "full_ratio": 0.94999998807907104, 2024-09-16T20:49:26.174 INFO:teuthology.orchestra.run.smithi029.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-16T20:49:26.174 INFO:teuthology.orchestra.run.smithi029.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-16T20:49:26.175 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_snapshot": "", 2024-09-16T20:49:26.175 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_max": 3, 2024-09-16T20:49:26.175 INFO:teuthology.orchestra.run.smithi029.stdout: "max_osd": 6, 2024-09-16T20:49:26.175 INFO:teuthology.orchestra.run.smithi029.stdout: "require_min_compat_client": "luminous", 2024-09-16T20:49:26.175 INFO:teuthology.orchestra.run.smithi029.stdout: "min_compat_client": "luminous", 2024-09-16T20:49:26.175 INFO:teuthology.orchestra.run.smithi029.stdout: "require_osd_release": "squid", 2024-09-16T20:49:26.175 INFO:teuthology.orchestra.run.smithi029.stdout: "allow_crimson": false, 2024-09-16T20:49:26.175 INFO:teuthology.orchestra.run.smithi029.stdout: "pools": [ 2024-09-16T20:49:26.175 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.175 INFO:teuthology.orchestra.run.smithi029.stdout: "pool": 1, 2024-09-16T20:49:26.175 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_name": ".mgr", 2024-09-16T20:49:26.175 INFO:teuthology.orchestra.run.smithi029.stdout: "create_time": "2024-09-16T20:47:52.220750+0000", 2024-09-16T20:49:26.175 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-16T20:49:26.175 INFO:teuthology.orchestra.run.smithi029.stdout: "flags_names": "hashpspool", 2024-09-16T20:49:26.176 INFO:teuthology.orchestra.run.smithi029.stdout: "type": 1, 2024-09-16T20:49:26.176 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 2, 2024-09-16T20:49:26.176 INFO:teuthology.orchestra.run.smithi029.stdout: "min_size": 1, 2024-09-16T20:49:26.176 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_rule": 0, 2024-09-16T20:49:26.176 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_count": 0, 2024-09-16T20:49:26.176 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_target": 0, 2024-09-16T20:49:26.176 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T20:49:26.176 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T20:49:26.176 INFO:teuthology.orchestra.run.smithi029.stdout: "is_stretch_pool": false, 2024-09-16T20:49:26.176 INFO:teuthology.orchestra.run.smithi029.stdout: "object_hash": 2, 2024-09-16T20:49:26.176 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_autoscale_mode": "off", 2024-09-16T20:49:26.176 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num": 1, 2024-09-16T20:49:26.176 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num": 1, 2024-09-16T20:49:26.177 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num_target": 1, 2024-09-16T20:49:26.177 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_target": 1, 2024-09-16T20:49:26.177 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_pending": 1, 2024-09-16T20:49:26.177 INFO:teuthology.orchestra.run.smithi029.stdout: "last_pg_merge_meta": { 2024-09-16T20:49:26.177 INFO:teuthology.orchestra.run.smithi029.stdout: "source_pgid": "0.0", 2024-09-16T20:49:26.177 INFO:teuthology.orchestra.run.smithi029.stdout: "ready_epoch": 0, 2024-09-16T20:49:26.177 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_started": 0, 2024-09-16T20:49:26.177 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_clean": 0, 2024-09-16T20:49:26.177 INFO:teuthology.orchestra.run.smithi029.stdout: "source_version": "0'0", 2024-09-16T20:49:26.177 INFO:teuthology.orchestra.run.smithi029.stdout: "target_version": "0'0" 2024-09-16T20:49:26.177 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.177 INFO:teuthology.orchestra.run.smithi029.stdout: "last_change": "17", 2024-09-16T20:49:26.177 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend": "0", 2024-09-16T20:49:26.178 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T20:49:26.178 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T20:49:26.178 INFO:teuthology.orchestra.run.smithi029.stdout: "auid": 0, 2024-09-16T20:49:26.178 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_mode": "selfmanaged", 2024-09-16T20:49:26.178 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_seq": 0, 2024-09-16T20:49:26.178 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_epoch": 0, 2024-09-16T20:49:26.178 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_snaps": [], 2024-09-16T20:49:26.178 INFO:teuthology.orchestra.run.smithi029.stdout: "removed_snaps": "[]", 2024-09-16T20:49:26.178 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_bytes": 0, 2024-09-16T20:49:26.178 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_objects": 0, 2024-09-16T20:49:26.178 INFO:teuthology.orchestra.run.smithi029.stdout: "tiers": [], 2024-09-16T20:49:26.178 INFO:teuthology.orchestra.run.smithi029.stdout: "tier_of": -1, 2024-09-16T20:49:26.178 INFO:teuthology.orchestra.run.smithi029.stdout: "read_tier": -1, 2024-09-16T20:49:26.179 INFO:teuthology.orchestra.run.smithi029.stdout: "write_tier": -1, 2024-09-16T20:49:26.179 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_mode": "none", 2024-09-16T20:49:26.179 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_bytes": 0, 2024-09-16T20:49:26.179 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_objects": 0, 2024-09-16T20:49:26.179 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T20:49:26.179 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T20:49:26.179 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T20:49:26.179 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_flush_age": 0, 2024-09-16T20:49:26.179 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_evict_age": 0, 2024-09-16T20:49:26.179 INFO:teuthology.orchestra.run.smithi029.stdout: "erasure_code_profile": "", 2024-09-16T20:49:26.179 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_params": { 2024-09-16T20:49:26.179 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "none" 2024-09-16T20:49:26.179 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.180 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_period": 0, 2024-09-16T20:49:26.180 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_count": 0, 2024-09-16T20:49:26.180 INFO:teuthology.orchestra.run.smithi029.stdout: "use_gmt_hitset": true, 2024-09-16T20:49:26.180 INFO:teuthology.orchestra.run.smithi029.stdout: "min_read_recency_for_promote": 0, 2024-09-16T20:49:26.180 INFO:teuthology.orchestra.run.smithi029.stdout: "min_write_recency_for_promote": 0, 2024-09-16T20:49:26.180 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T20:49:26.180 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_search_last_n": 0, 2024-09-16T20:49:26.180 INFO:teuthology.orchestra.run.smithi029.stdout: "grade_table": [], 2024-09-16T20:49:26.180 INFO:teuthology.orchestra.run.smithi029.stdout: "stripe_width": 0, 2024-09-16T20:49:26.180 INFO:teuthology.orchestra.run.smithi029.stdout: "expected_num_objects": 0, 2024-09-16T20:49:26.180 INFO:teuthology.orchestra.run.smithi029.stdout: "fast_read": false, 2024-09-16T20:49:26.180 INFO:teuthology.orchestra.run.smithi029.stdout: "options": { 2024-09-16T20:49:26.180 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_max": 32, 2024-09-16T20:49:26.180 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_min": 1 2024-09-16T20:49:26.181 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.181 INFO:teuthology.orchestra.run.smithi029.stdout: "application_metadata": { 2024-09-16T20:49:26.181 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": {} 2024-09-16T20:49:26.181 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.181 INFO:teuthology.orchestra.run.smithi029.stdout: "read_balance": { 2024-09-16T20:49:26.181 INFO:teuthology.orchestra.run.smithi029.stdout: "score_type": "Fair distribution", 2024-09-16T20:49:26.181 INFO:teuthology.orchestra.run.smithi029.stdout: "score_acting": 6.059999942779541, 2024-09-16T20:49:26.181 INFO:teuthology.orchestra.run.smithi029.stdout: "score_stable": 6.059999942779541, 2024-09-16T20:49:26.181 INFO:teuthology.orchestra.run.smithi029.stdout: "optimal_score": 0.33000001311302185, 2024-09-16T20:49:26.181 INFO:teuthology.orchestra.run.smithi029.stdout: "raw_score_acting": 2, 2024-09-16T20:49:26.181 INFO:teuthology.orchestra.run.smithi029.stdout: "raw_score_stable": 2, 2024-09-16T20:49:26.181 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity_weighted": 1, 2024-09-16T20:49:26.181 INFO:teuthology.orchestra.run.smithi029.stdout: "average_primary_affinity": 1, 2024-09-16T20:49:26.182 INFO:teuthology.orchestra.run.smithi029.stdout: "average_primary_affinity_weighted": 1 2024-09-16T20:49:26.182 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:26.182 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.182 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.182 INFO:teuthology.orchestra.run.smithi029.stdout: "pool": 2, 2024-09-16T20:49:26.182 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_name": "rbd", 2024-09-16T20:49:26.182 INFO:teuthology.orchestra.run.smithi029.stdout: "create_time": "2024-09-16T20:47:56.756666+0000", 2024-09-16T20:49:26.182 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 8193, 2024-09-16T20:49:26.182 INFO:teuthology.orchestra.run.smithi029.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T20:49:26.182 INFO:teuthology.orchestra.run.smithi029.stdout: "type": 1, 2024-09-16T20:49:26.182 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 2, 2024-09-16T20:49:26.182 INFO:teuthology.orchestra.run.smithi029.stdout: "min_size": 1, 2024-09-16T20:49:26.182 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_rule": 0, 2024-09-16T20:49:26.182 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_count": 0, 2024-09-16T20:49:26.183 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_target": 0, 2024-09-16T20:49:26.183 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T20:49:26.183 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T20:49:26.183 INFO:teuthology.orchestra.run.smithi029.stdout: "is_stretch_pool": false, 2024-09-16T20:49:26.183 INFO:teuthology.orchestra.run.smithi029.stdout: "object_hash": 2, 2024-09-16T20:49:26.183 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_autoscale_mode": "off", 2024-09-16T20:49:26.183 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num": 8, 2024-09-16T20:49:26.183 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num": 8, 2024-09-16T20:49:26.183 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num_target": 8, 2024-09-16T20:49:26.183 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_target": 8, 2024-09-16T20:49:26.183 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_pending": 8, 2024-09-16T20:49:26.183 INFO:teuthology.orchestra.run.smithi029.stdout: "last_pg_merge_meta": { 2024-09-16T20:49:26.183 INFO:teuthology.orchestra.run.smithi029.stdout: "source_pgid": "0.0", 2024-09-16T20:49:26.183 INFO:teuthology.orchestra.run.smithi029.stdout: "ready_epoch": 0, 2024-09-16T20:49:26.184 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_started": 0, 2024-09-16T20:49:26.184 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_clean": 0, 2024-09-16T20:49:26.184 INFO:teuthology.orchestra.run.smithi029.stdout: "source_version": "0'0", 2024-09-16T20:49:26.184 INFO:teuthology.orchestra.run.smithi029.stdout: "target_version": "0'0" 2024-09-16T20:49:26.184 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.184 INFO:teuthology.orchestra.run.smithi029.stdout: "last_change": "21", 2024-09-16T20:49:26.184 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend": "0", 2024-09-16T20:49:26.184 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T20:49:26.184 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T20:49:26.184 INFO:teuthology.orchestra.run.smithi029.stdout: "auid": 0, 2024-09-16T20:49:26.184 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_mode": "selfmanaged", 2024-09-16T20:49:26.184 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_seq": 2, 2024-09-16T20:49:26.184 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_epoch": 21, 2024-09-16T20:49:26.185 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_snaps": [], 2024-09-16T20:49:26.185 INFO:teuthology.orchestra.run.smithi029.stdout: "removed_snaps": "[]", 2024-09-16T20:49:26.185 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_bytes": 0, 2024-09-16T20:49:26.185 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_objects": 0, 2024-09-16T20:49:26.185 INFO:teuthology.orchestra.run.smithi029.stdout: "tiers": [], 2024-09-16T20:49:26.185 INFO:teuthology.orchestra.run.smithi029.stdout: "tier_of": -1, 2024-09-16T20:49:26.185 INFO:teuthology.orchestra.run.smithi029.stdout: "read_tier": -1, 2024-09-16T20:49:26.185 INFO:teuthology.orchestra.run.smithi029.stdout: "write_tier": -1, 2024-09-16T20:49:26.185 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_mode": "none", 2024-09-16T20:49:26.185 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_bytes": 0, 2024-09-16T20:49:26.185 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_objects": 0, 2024-09-16T20:49:26.185 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T20:49:26.185 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T20:49:26.186 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T20:49:26.186 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_flush_age": 0, 2024-09-16T20:49:26.186 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_evict_age": 0, 2024-09-16T20:49:26.186 INFO:teuthology.orchestra.run.smithi029.stdout: "erasure_code_profile": "", 2024-09-16T20:49:26.186 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_params": { 2024-09-16T20:49:26.186 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "none" 2024-09-16T20:49:26.186 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.186 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_period": 0, 2024-09-16T20:49:26.186 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_count": 0, 2024-09-16T20:49:26.186 INFO:teuthology.orchestra.run.smithi029.stdout: "use_gmt_hitset": true, 2024-09-16T20:49:26.186 INFO:teuthology.orchestra.run.smithi029.stdout: "min_read_recency_for_promote": 0, 2024-09-16T20:49:26.186 INFO:teuthology.orchestra.run.smithi029.stdout: "min_write_recency_for_promote": 0, 2024-09-16T20:49:26.186 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T20:49:26.186 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_search_last_n": 0, 2024-09-16T20:49:26.187 INFO:teuthology.orchestra.run.smithi029.stdout: "grade_table": [], 2024-09-16T20:49:26.187 INFO:teuthology.orchestra.run.smithi029.stdout: "stripe_width": 0, 2024-09-16T20:49:26.187 INFO:teuthology.orchestra.run.smithi029.stdout: "expected_num_objects": 0, 2024-09-16T20:49:26.187 INFO:teuthology.orchestra.run.smithi029.stdout: "fast_read": false, 2024-09-16T20:49:26.187 INFO:teuthology.orchestra.run.smithi029.stdout: "options": {}, 2024-09-16T20:49:26.187 INFO:teuthology.orchestra.run.smithi029.stdout: "application_metadata": { 2024-09-16T20:49:26.187 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd": {} 2024-09-16T20:49:26.187 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.187 INFO:teuthology.orchestra.run.smithi029.stdout: "read_balance": { 2024-09-16T20:49:26.187 INFO:teuthology.orchestra.run.smithi029.stdout: "score_type": "Fair distribution", 2024-09-16T20:49:26.187 INFO:teuthology.orchestra.run.smithi029.stdout: "score_acting": 2.25, 2024-09-16T20:49:26.187 INFO:teuthology.orchestra.run.smithi029.stdout: "score_stable": 2.25, 2024-09-16T20:49:26.187 INFO:teuthology.orchestra.run.smithi029.stdout: "optimal_score": 1, 2024-09-16T20:49:26.187 INFO:teuthology.orchestra.run.smithi029.stdout: "raw_score_acting": 2.25, 2024-09-16T20:49:26.188 INFO:teuthology.orchestra.run.smithi029.stdout: "raw_score_stable": 2.25, 2024-09-16T20:49:26.188 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity_weighted": 1, 2024-09-16T20:49:26.188 INFO:teuthology.orchestra.run.smithi029.stdout: "average_primary_affinity": 1, 2024-09-16T20:49:26.188 INFO:teuthology.orchestra.run.smithi029.stdout: "average_primary_affinity_weighted": 1 2024-09-16T20:49:26.188 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:26.188 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.188 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.188 INFO:teuthology.orchestra.run.smithi029.stdout: "pool": 3, 2024-09-16T20:49:26.188 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_name": "pool_client.0", 2024-09-16T20:49:26.188 INFO:teuthology.orchestra.run.smithi029.stdout: "create_time": "2024-09-16T20:48:06.418889+0000", 2024-09-16T20:49:26.188 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 8193, 2024-09-16T20:49:26.189 INFO:teuthology.orchestra.run.smithi029.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T20:49:26.189 INFO:teuthology.orchestra.run.smithi029.stdout: "type": 1, 2024-09-16T20:49:26.189 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 2, 2024-09-16T20:49:26.189 INFO:teuthology.orchestra.run.smithi029.stdout: "min_size": 1, 2024-09-16T20:49:26.189 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_rule": 0, 2024-09-16T20:49:26.189 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_count": 0, 2024-09-16T20:49:26.189 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_target": 0, 2024-09-16T20:49:26.189 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T20:49:26.189 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T20:49:26.189 INFO:teuthology.orchestra.run.smithi029.stdout: "is_stretch_pool": false, 2024-09-16T20:49:26.189 INFO:teuthology.orchestra.run.smithi029.stdout: "object_hash": 2, 2024-09-16T20:49:26.189 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_autoscale_mode": "off", 2024-09-16T20:49:26.189 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num": 32, 2024-09-16T20:49:26.190 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num": 32, 2024-09-16T20:49:26.190 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num_target": 32, 2024-09-16T20:49:26.190 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_target": 32, 2024-09-16T20:49:26.190 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_pending": 32, 2024-09-16T20:49:26.190 INFO:teuthology.orchestra.run.smithi029.stdout: "last_pg_merge_meta": { 2024-09-16T20:49:26.190 INFO:teuthology.orchestra.run.smithi029.stdout: "source_pgid": "0.0", 2024-09-16T20:49:26.190 INFO:teuthology.orchestra.run.smithi029.stdout: "ready_epoch": 0, 2024-09-16T20:49:26.190 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_started": 0, 2024-09-16T20:49:26.190 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_clean": 0, 2024-09-16T20:49:26.190 INFO:teuthology.orchestra.run.smithi029.stdout: "source_version": "0'0", 2024-09-16T20:49:26.190 INFO:teuthology.orchestra.run.smithi029.stdout: "target_version": "0'0" 2024-09-16T20:49:26.190 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.190 INFO:teuthology.orchestra.run.smithi029.stdout: "last_change": "91", 2024-09-16T20:49:26.191 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend": "0", 2024-09-16T20:49:26.191 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T20:49:26.191 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T20:49:26.191 INFO:teuthology.orchestra.run.smithi029.stdout: "auid": 0, 2024-09-16T20:49:26.191 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_mode": "selfmanaged", 2024-09-16T20:49:26.191 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_seq": 28, 2024-09-16T20:49:26.191 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_epoch": 91, 2024-09-16T20:49:26.191 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_snaps": [], 2024-09-16T20:49:26.192 INFO:teuthology.orchestra.run.smithi029.stdout: "removed_snaps": "[]", 2024-09-16T20:49:26.192 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_bytes": 0, 2024-09-16T20:49:26.192 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_objects": 0, 2024-09-16T20:49:26.192 INFO:teuthology.orchestra.run.smithi029.stdout: "tiers": [], 2024-09-16T20:49:26.192 INFO:teuthology.orchestra.run.smithi029.stdout: "tier_of": -1, 2024-09-16T20:49:26.192 INFO:teuthology.orchestra.run.smithi029.stdout: "read_tier": -1, 2024-09-16T20:49:26.192 INFO:teuthology.orchestra.run.smithi029.stdout: "write_tier": -1, 2024-09-16T20:49:26.192 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_mode": "none", 2024-09-16T20:49:26.192 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_bytes": 0, 2024-09-16T20:49:26.192 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_objects": 0, 2024-09-16T20:49:26.192 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T20:49:26.192 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T20:49:26.193 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T20:49:26.193 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_flush_age": 0, 2024-09-16T20:49:26.193 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_evict_age": 0, 2024-09-16T20:49:26.193 INFO:teuthology.orchestra.run.smithi029.stdout: "erasure_code_profile": "", 2024-09-16T20:49:26.193 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_params": { 2024-09-16T20:49:26.193 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "none" 2024-09-16T20:49:26.193 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.193 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_period": 0, 2024-09-16T20:49:26.193 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_count": 0, 2024-09-16T20:49:26.193 INFO:teuthology.orchestra.run.smithi029.stdout: "use_gmt_hitset": true, 2024-09-16T20:49:26.193 INFO:teuthology.orchestra.run.smithi029.stdout: "min_read_recency_for_promote": 0, 2024-09-16T20:49:26.193 INFO:teuthology.orchestra.run.smithi029.stdout: "min_write_recency_for_promote": 0, 2024-09-16T20:49:26.193 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T20:49:26.193 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_search_last_n": 0, 2024-09-16T20:49:26.194 INFO:teuthology.orchestra.run.smithi029.stdout: "grade_table": [], 2024-09-16T20:49:26.194 INFO:teuthology.orchestra.run.smithi029.stdout: "stripe_width": 0, 2024-09-16T20:49:26.194 INFO:teuthology.orchestra.run.smithi029.stdout: "expected_num_objects": 0, 2024-09-16T20:49:26.194 INFO:teuthology.orchestra.run.smithi029.stdout: "fast_read": false, 2024-09-16T20:49:26.194 INFO:teuthology.orchestra.run.smithi029.stdout: "options": {}, 2024-09-16T20:49:26.194 INFO:teuthology.orchestra.run.smithi029.stdout: "application_metadata": { 2024-09-16T20:49:26.194 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd": {} 2024-09-16T20:49:26.194 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.194 INFO:teuthology.orchestra.run.smithi029.stdout: "read_balance": { 2024-09-16T20:49:26.194 INFO:teuthology.orchestra.run.smithi029.stdout: "score_type": "Fair distribution", 2024-09-16T20:49:26.194 INFO:teuthology.orchestra.run.smithi029.stdout: "score_acting": 1.1299999952316284, 2024-09-16T20:49:26.194 INFO:teuthology.orchestra.run.smithi029.stdout: "score_stable": 1.1299999952316284, 2024-09-16T20:49:26.194 INFO:teuthology.orchestra.run.smithi029.stdout: "optimal_score": 1, 2024-09-16T20:49:26.194 INFO:teuthology.orchestra.run.smithi029.stdout: "raw_score_acting": 1.1299999952316284, 2024-09-16T20:49:26.195 INFO:teuthology.orchestra.run.smithi029.stdout: "raw_score_stable": 1.1299999952316284, 2024-09-16T20:49:26.195 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity_weighted": 1, 2024-09-16T20:49:26.195 INFO:teuthology.orchestra.run.smithi029.stdout: "average_primary_affinity": 1, 2024-09-16T20:49:26.195 INFO:teuthology.orchestra.run.smithi029.stdout: "average_primary_affinity_weighted": 1 2024-09-16T20:49:26.195 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:26.195 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:26.195 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-16T20:49:26.195 INFO:teuthology.orchestra.run.smithi029.stdout: "osds": [ 2024-09-16T20:49:26.195 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.195 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 0, 2024-09-16T20:49:26.195 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "6c3cb2d7-7d23-4566-9ca1-d7b08e12360c", 2024-09-16T20:49:26.195 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-16T20:49:26.195 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-16T20:49:26.196 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-16T20:49:26.196 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-16T20:49:26.196 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-16T20:49:26.196 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-16T20:49:26.196 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 13, 2024-09-16T20:49:26.196 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 90, 2024-09-16T20:49:26.196 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-16T20:49:26.196 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-16T20:49:26.196 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-16T20:49:26.196 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:26.196 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.197 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:26.197 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6816", 2024-09-16T20:49:26.197 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:49:26.197 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.197 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.197 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:26.197 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6817", 2024-09-16T20:49:26.197 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:49:26.197 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:26.197 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:26.197 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.197 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-16T20:49:26.197 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:26.198 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.198 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:26.198 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6818", 2024-09-16T20:49:26.198 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:49:26.198 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.198 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.198 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:26.198 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6819", 2024-09-16T20:49:26.198 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:49:26.198 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:26.198 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:26.198 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.198 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-16T20:49:26.199 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:26.199 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.199 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:26.200 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6822", 2024-09-16T20:49:26.200 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:49:26.200 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.200 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.200 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:26.200 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6823", 2024-09-16T20:49:26.200 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:49:26.200 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:26.200 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:26.200 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.200 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-16T20:49:26.200 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:26.201 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.201 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:26.201 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6820", 2024-09-16T20:49:26.201 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:49:26.201 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.201 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.201 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:26.201 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6821", 2024-09-16T20:49:26.201 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:49:26.201 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:26.201 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:26.201 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.201 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.29:6817/2744025949", 2024-09-16T20:49:26.202 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.29:6819/2744025949", 2024-09-16T20:49:26.202 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.29:6823/2744025949", 2024-09-16T20:49:26.202 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.29:6821/2744025949", 2024-09-16T20:49:26.202 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-16T20:49:26.202 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-16T20:49:26.202 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-16T20:49:26.202 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:26.202 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.202 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.202 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 1, 2024-09-16T20:49:26.202 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "5cf8c20c-1ed2-42ae-a8b0-c88cc4e36bce", 2024-09-16T20:49:26.202 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-16T20:49:26.203 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-16T20:49:26.203 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-16T20:49:26.203 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-16T20:49:26.203 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-16T20:49:26.203 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-16T20:49:26.203 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 13, 2024-09-16T20:49:26.203 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 81, 2024-09-16T20:49:26.203 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-16T20:49:26.203 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-16T20:49:26.203 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-16T20:49:26.203 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:26.203 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.203 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:26.203 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6800", 2024-09-16T20:49:26.204 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:49:26.204 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.204 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.204 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:26.204 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6801", 2024-09-16T20:49:26.204 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:49:26.204 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:26.204 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:26.204 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.205 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-16T20:49:26.205 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:26.205 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.205 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:26.205 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6802", 2024-09-16T20:49:26.205 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:49:26.205 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.205 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.205 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:26.206 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6803", 2024-09-16T20:49:26.206 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:49:26.206 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:26.206 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:26.206 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.206 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-16T20:49:26.206 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:26.206 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.206 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:26.206 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6806", 2024-09-16T20:49:26.206 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:49:26.206 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.207 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.207 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:26.207 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6807", 2024-09-16T20:49:26.207 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:49:26.207 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:26.207 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:26.207 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.207 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-16T20:49:26.207 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:26.207 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.207 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:26.207 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6804", 2024-09-16T20:49:26.207 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:49:26.208 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.208 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.208 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:26.208 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6805", 2024-09-16T20:49:26.208 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:49:26.208 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:26.208 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:26.208 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.208 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.29:6801/208628762", 2024-09-16T20:49:26.208 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.29:6803/208628762", 2024-09-16T20:49:26.208 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.29:6807/208628762", 2024-09-16T20:49:26.208 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.29:6805/208628762", 2024-09-16T20:49:26.208 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-16T20:49:26.209 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-16T20:49:26.209 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-16T20:49:26.209 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:26.209 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.209 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.209 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 2, 2024-09-16T20:49:26.209 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "075d16da-82bf-449b-9f6e-542c9e8d810b", 2024-09-16T20:49:26.209 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-16T20:49:26.209 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-16T20:49:26.209 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-16T20:49:26.209 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-16T20:49:26.209 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-16T20:49:26.209 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-16T20:49:26.209 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 13, 2024-09-16T20:49:26.210 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 80, 2024-09-16T20:49:26.210 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-16T20:49:26.210 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-16T20:49:26.210 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-16T20:49:26.210 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:26.210 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.210 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:26.210 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6808", 2024-09-16T20:49:26.210 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:49:26.210 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.210 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.210 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:26.211 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6809", 2024-09-16T20:49:26.211 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:49:26.211 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:26.211 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:26.211 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.211 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-16T20:49:26.211 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:26.211 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.211 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:26.211 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6810", 2024-09-16T20:49:26.211 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:49:26.211 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.211 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.211 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:26.212 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6811", 2024-09-16T20:49:26.212 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:49:26.212 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:26.212 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:26.212 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.212 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-16T20:49:26.212 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:26.212 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.212 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:26.212 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6814", 2024-09-16T20:49:26.212 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:49:26.212 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.212 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.212 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:26.213 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6815", 2024-09-16T20:49:26.213 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:49:26.213 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:26.213 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:26.213 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.213 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-16T20:49:26.213 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:26.213 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.213 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:26.213 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6812", 2024-09-16T20:49:26.213 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:49:26.213 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.214 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.214 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:26.214 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6813", 2024-09-16T20:49:26.214 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:49:26.214 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:26.214 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:26.214 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.214 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.29:6809/3447426536", 2024-09-16T20:49:26.214 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.29:6811/3447426536", 2024-09-16T20:49:26.214 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.29:6815/3447426536", 2024-09-16T20:49:26.214 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.29:6813/3447426536", 2024-09-16T20:49:26.214 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-16T20:49:26.214 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-16T20:49:26.214 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-16T20:49:26.214 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:26.215 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.215 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.215 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 3, 2024-09-16T20:49:26.215 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "0612be38-3a24-41b6-aba4-5a2fe985165e", 2024-09-16T20:49:26.215 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-16T20:49:26.215 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-16T20:49:26.215 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-16T20:49:26.215 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-16T20:49:26.215 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-16T20:49:26.215 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-16T20:49:26.215 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 13, 2024-09-16T20:49:26.215 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 78, 2024-09-16T20:49:26.215 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-16T20:49:26.215 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-16T20:49:26.215 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-16T20:49:26.216 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:26.216 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.216 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:26.216 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6808", 2024-09-16T20:49:26.216 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:49:26.216 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.216 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.216 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:26.216 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6809", 2024-09-16T20:49:26.216 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:49:26.216 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:26.216 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:26.216 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.217 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-16T20:49:26.217 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:26.217 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.217 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:26.217 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6810", 2024-09-16T20:49:26.217 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:49:26.217 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.217 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.217 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:26.217 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6811", 2024-09-16T20:49:26.217 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:49:26.217 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:26.217 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:26.217 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.218 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-16T20:49:26.218 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:26.218 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.218 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:26.218 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6814", 2024-09-16T20:49:26.218 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:49:26.218 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.218 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.218 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:26.218 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6815", 2024-09-16T20:49:26.218 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:49:26.218 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:26.218 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:26.218 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.218 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-16T20:49:26.219 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:26.219 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.219 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:26.219 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6812", 2024-09-16T20:49:26.219 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:49:26.219 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.219 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.219 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:26.219 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6813", 2024-09-16T20:49:26.219 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:49:26.219 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:26.219 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:26.219 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.220 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.43:6809/3687583891", 2024-09-16T20:49:26.220 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.43:6811/3687583891", 2024-09-16T20:49:26.220 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.43:6815/3687583891", 2024-09-16T20:49:26.220 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.43:6813/3687583891", 2024-09-16T20:49:26.220 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-16T20:49:26.220 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-16T20:49:26.220 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-16T20:49:26.220 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:26.220 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.220 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.220 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 4, 2024-09-16T20:49:26.220 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "be590bf8-e9fa-44ca-a471-6f6bcb087ca7", 2024-09-16T20:49:26.220 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-16T20:49:26.221 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-16T20:49:26.221 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-16T20:49:26.221 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-16T20:49:26.221 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-16T20:49:26.221 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-16T20:49:26.221 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 11, 2024-09-16T20:49:26.221 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 81, 2024-09-16T20:49:26.221 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-16T20:49:26.221 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-16T20:49:26.221 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-16T20:49:26.221 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:26.221 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.221 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:26.221 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6800", 2024-09-16T20:49:26.221 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:49:26.222 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.222 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.222 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:26.222 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6801", 2024-09-16T20:49:26.222 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:49:26.222 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:26.222 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:26.222 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.222 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-16T20:49:26.222 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:26.222 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.222 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:26.222 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6802", 2024-09-16T20:49:26.223 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:49:26.223 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.223 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.223 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:26.223 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6803", 2024-09-16T20:49:26.223 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:49:26.223 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:26.223 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:26.223 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.223 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-16T20:49:26.223 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:26.223 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.223 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:26.224 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6806", 2024-09-16T20:49:26.224 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:49:26.224 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.224 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.224 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:26.224 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6807", 2024-09-16T20:49:26.224 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:49:26.224 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:26.224 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:26.224 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.224 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-16T20:49:26.224 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:26.225 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.225 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:26.225 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6804", 2024-09-16T20:49:26.225 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:49:26.225 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.225 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.225 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:26.225 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6805", 2024-09-16T20:49:26.225 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:49:26.225 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:26.225 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:26.225 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.225 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.43:6801/2505220646", 2024-09-16T20:49:26.225 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.43:6803/2505220646", 2024-09-16T20:49:26.226 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.43:6807/2505220646", 2024-09-16T20:49:26.226 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.43:6805/2505220646", 2024-09-16T20:49:26.226 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-16T20:49:26.226 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-16T20:49:26.226 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-16T20:49:26.226 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:26.226 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.226 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.226 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 5, 2024-09-16T20:49:26.226 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "4a208e30-031b-4a9f-af52-e26d4a039ea6", 2024-09-16T20:49:26.226 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-16T20:49:26.226 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-16T20:49:26.226 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-16T20:49:26.226 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-16T20:49:26.226 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-16T20:49:26.226 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-16T20:49:26.227 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 13, 2024-09-16T20:49:26.227 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 91, 2024-09-16T20:49:26.227 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-16T20:49:26.227 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-16T20:49:26.227 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-16T20:49:26.227 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:26.227 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.227 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:26.227 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6816", 2024-09-16T20:49:26.227 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:49:26.227 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.227 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.227 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:26.227 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6817", 2024-09-16T20:49:26.228 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:49:26.228 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:26.228 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:26.228 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.228 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-16T20:49:26.228 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:26.228 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.228 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:26.228 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6818", 2024-09-16T20:49:26.228 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:49:26.228 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.228 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.228 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:26.228 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6819", 2024-09-16T20:49:26.229 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:49:26.229 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:26.229 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:26.229 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.229 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-16T20:49:26.229 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:26.229 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.229 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:26.229 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6822", 2024-09-16T20:49:26.229 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:49:26.229 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.229 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.229 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:26.229 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6823", 2024-09-16T20:49:26.230 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:49:26.230 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:26.230 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:26.230 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.230 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-16T20:49:26.230 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:26.230 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.230 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:26.230 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6820", 2024-09-16T20:49:26.230 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:49:26.230 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.230 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.230 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:26.230 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6821", 2024-09-16T20:49:26.230 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:49:26.231 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:26.231 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:26.231 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.231 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.43:6817/1517367307", 2024-09-16T20:49:26.231 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.43:6819/1517367307", 2024-09-16T20:49:26.231 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.43:6823/1517367307", 2024-09-16T20:49:26.231 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.43:6821/1517367307", 2024-09-16T20:49:26.231 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-16T20:49:26.231 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-16T20:49:26.231 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-16T20:49:26.231 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:26.231 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:26.231 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-16T20:49:26.231 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_xinfo": [ 2024-09-16T20:49:26.232 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.232 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 0, 2024-09-16T20:49:26.232 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-16T20:49:26.232 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-16T20:49:26.232 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-16T20:49:26.232 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540719139924082687, 2024-09-16T20:49:26.232 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-16T20:49:26.232 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-16T20:47:47.887412+0000", 2024-09-16T20:49:26.232 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-16T20:49:26.232 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.232 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.232 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 1, 2024-09-16T20:49:26.232 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-16T20:49:26.233 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-16T20:49:26.233 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-16T20:49:26.233 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540719139924082687, 2024-09-16T20:49:26.233 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-16T20:49:26.233 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-16T20:47:47.658782+0000", 2024-09-16T20:49:26.233 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-16T20:49:26.233 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.233 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.233 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 2, 2024-09-16T20:49:26.233 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-16T20:49:26.233 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-16T20:49:26.233 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-16T20:49:26.233 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540719139924082687, 2024-09-16T20:49:26.233 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-16T20:49:26.234 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-16T20:47:47.453752+0000", 2024-09-16T20:49:26.234 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-16T20:49:26.234 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.234 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.234 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 3, 2024-09-16T20:49:26.234 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-16T20:49:26.234 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-16T20:49:26.234 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-16T20:49:26.234 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540719139924082687, 2024-09-16T20:49:26.234 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-16T20:49:26.234 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-16T20:47:47.487992+0000", 2024-09-16T20:49:26.234 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-16T20:49:26.234 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.235 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.235 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 4, 2024-09-16T20:49:26.235 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-16T20:49:26.235 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-16T20:49:26.235 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-16T20:49:26.235 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540719139924082687, 2024-09-16T20:49:26.235 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-16T20:49:26.235 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-16T20:47:46.943344+0000", 2024-09-16T20:49:26.235 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-16T20:49:26.235 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.235 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.235 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 5, 2024-09-16T20:49:26.235 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-16T20:49:26.235 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-16T20:49:26.236 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-16T20:49:26.236 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540719139924082687, 2024-09-16T20:49:26.236 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-16T20:49:26.236 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-16T20:47:47.901547+0000", 2024-09-16T20:49:26.236 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-16T20:49:26.236 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:26.236 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-16T20:49:26.236 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_upmap": [], 2024-09-16T20:49:26.236 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_upmap_items": [ 2024-09-16T20:49:26.236 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.236 INFO:teuthology.orchestra.run.smithi029.stdout: "pgid": "3.c", 2024-09-16T20:49:26.236 INFO:teuthology.orchestra.run.smithi029.stdout: "mappings": [ 2024-09-16T20:49:26.236 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:26.236 INFO:teuthology.orchestra.run.smithi029.stdout: "from": 0, 2024-09-16T20:49:26.237 INFO:teuthology.orchestra.run.smithi029.stdout: "to": 5 2024-09-16T20:49:26.237 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:26.237 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:26.237 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:26.237 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-16T20:49:26.237 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_upmap_primaries": [], 2024-09-16T20:49:26.237 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_temp": [], 2024-09-16T20:49:26.237 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_temp": [], 2024-09-16T20:49:26.237 INFO:teuthology.orchestra.run.smithi029.stdout: "blocklist": {}, 2024-09-16T20:49:26.237 INFO:teuthology.orchestra.run.smithi029.stdout: "range_blocklist": {}, 2024-09-16T20:49:26.237 INFO:teuthology.orchestra.run.smithi029.stdout: "erasure_code_profiles": { 2024-09-16T20:49:26.237 INFO:teuthology.orchestra.run.smithi029.stdout: "default": { 2024-09-16T20:49:26.237 INFO:teuthology.orchestra.run.smithi029.stdout: "crush-failure-domain": "osd", 2024-09-16T20:49:26.237 INFO:teuthology.orchestra.run.smithi029.stdout: "k": "2", 2024-09-16T20:49:26.238 INFO:teuthology.orchestra.run.smithi029.stdout: "m": "1", 2024-09-16T20:49:26.238 INFO:teuthology.orchestra.run.smithi029.stdout: "plugin": "jerasure", 2024-09-16T20:49:26.238 INFO:teuthology.orchestra.run.smithi029.stdout: "technique": "reed_sol_van" 2024-09-16T20:49:26.238 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:26.238 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:26.238 INFO:teuthology.orchestra.run.smithi029.stdout: "removed_snaps_queue": [], 2024-09-16T20:49:26.238 INFO:teuthology.orchestra.run.smithi029.stdout: "new_removed_snaps": [], 2024-09-16T20:49:26.238 INFO:teuthology.orchestra.run.smithi029.stdout: "new_purged_snaps": [], 2024-09-16T20:49:26.238 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_node_flags": {}, 2024-09-16T20:49:26.238 INFO:teuthology.orchestra.run.smithi029.stdout: "device_class_flags": {}, 2024-09-16T20:49:26.238 INFO:teuthology.orchestra.run.smithi029.stdout: "stretch_mode": { 2024-09-16T20:49:26.238 INFO:teuthology.orchestra.run.smithi029.stdout: "stretch_mode_enabled": false, 2024-09-16T20:49:26.238 INFO:teuthology.orchestra.run.smithi029.stdout: "stretch_bucket_count": 0, 2024-09-16T20:49:26.238 INFO:teuthology.orchestra.run.smithi029.stdout: "degraded_stretch_mode": 0, 2024-09-16T20:49:26.239 INFO:teuthology.orchestra.run.smithi029.stdout: "recovering_stretch_mode": 0, 2024-09-16T20:49:26.239 INFO:teuthology.orchestra.run.smithi029.stdout: "stretch_mode_bucket": 0 2024-09-16T20:49:26.239 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:26.239 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:26.242 INFO:tasks.thrashosds.thrasher:j is {'epoch': 93, 'fsid': 'e28af4a7-5fcd-4de8-8f77-a2b575361bb1', 'created': '2024-09-16T20:47:39.760609+0000', 'modified': '2024-09-16T20:49:25.838278+0000', 'last_up_change': '2024-09-16T20:47:50.238013+0000', 'last_in_change': '2024-09-16T20:49:08.779548+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T20:47:52.220750+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-16T20:47:56.756666+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-16T20:48:06.418889+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 28, 'snap_epoch': 91, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '6c3cb2d7-7d23-4566-9ca1-d7b08e12360c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 90, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6816', 'nonce': 2744025949}, {'type': 'v1', 'addr': '172.21.15.29:6817', 'nonce': 2744025949}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6818', 'nonce': 2744025949}, {'type': 'v1', 'addr': '172.21.15.29:6819', 'nonce': 2744025949}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6822', 'nonce': 2744025949}, {'type': 'v1', 'addr': '172.21.15.29:6823', 'nonce': 2744025949}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6820', 'nonce': 2744025949}, {'type': 'v1', 'addr': '172.21.15.29:6821', 'nonce': 2744025949}]}, 'public_addr': '172.21.15.29:6817/2744025949', 'cluster_addr': '172.21.15.29:6819/2744025949', 'heartbeat_back_addr': '172.21.15.29:6823/2744025949', 'heartbeat_front_addr': '172.21.15.29:6821/2744025949', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '5cf8c20c-1ed2-42ae-a8b0-c88cc4e36bce', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 81, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6800', 'nonce': 208628762}, {'type': 'v1', 'addr': '172.21.15.29:6801', 'nonce': 208628762}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6802', 'nonce': 208628762}, {'type': 'v1', 'addr': '172.21.15.29:6803', 'nonce': 208628762}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6806', 'nonce': 208628762}, {'type': 'v1', 'addr': '172.21.15.29:6807', 'nonce': 208628762}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6804', 'nonce': 208628762}, {'type': 'v1', 'addr': '172.21.15.29:6805', 'nonce': 208628762}]}, 'public_addr': '172.21.15.29:6801/208628762', 'cluster_addr': '172.21.15.29:6803/208628762', 'heartbeat_back_addr': '172.21.15.29:6807/208628762', 'heartbeat_front_addr': '172.21.15.29:6805/208628762', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '075d16da-82bf-449b-9f6e-542c9e8d810b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 80, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6808', 'nonce': 3447426536}, {'type': 'v1', 'addr': '172.21.15.29:6809', 'nonce': 3447426536}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6810', 'nonce': 3447426536}, {'type': 'v1', 'addr': '172.21.15.29:6811', 'nonce': 3447426536}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6814', 'nonce': 3447426536}, {'type': 'v1', 'addr': '172.21.15.29:6815', 'nonce': 3447426536}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6812', 'nonce': 3447426536}, {'type': 'v1', 'addr': '172.21.15.29:6813', 'nonce': 3447426536}]}, 'public_addr': '172.21.15.29:6809/3447426536', 'cluster_addr': '172.21.15.29:6811/3447426536', 'heartbeat_back_addr': '172.21.15.29:6815/3447426536', 'heartbeat_front_addr': '172.21.15.29:6813/3447426536', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '0612be38-3a24-41b6-aba4-5a2fe985165e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 78, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6808', 'nonce': 3687583891}, {'type': 'v1', 'addr': '172.21.15.43:6809', 'nonce': 3687583891}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6810', 'nonce': 3687583891}, {'type': 'v1', 'addr': '172.21.15.43:6811', 'nonce': 3687583891}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6814', 'nonce': 3687583891}, {'type': 'v1', 'addr': '172.21.15.43:6815', 'nonce': 3687583891}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6812', 'nonce': 3687583891}, {'type': 'v1', 'addr': '172.21.15.43:6813', 'nonce': 3687583891}]}, 'public_addr': '172.21.15.43:6809/3687583891', 'cluster_addr': '172.21.15.43:6811/3687583891', 'heartbeat_back_addr': '172.21.15.43:6815/3687583891', 'heartbeat_front_addr': '172.21.15.43:6813/3687583891', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'be590bf8-e9fa-44ca-a471-6f6bcb087ca7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 81, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6800', 'nonce': 2505220646}, {'type': 'v1', 'addr': '172.21.15.43:6801', 'nonce': 2505220646}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6802', 'nonce': 2505220646}, {'type': 'v1', 'addr': '172.21.15.43:6803', 'nonce': 2505220646}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6806', 'nonce': 2505220646}, {'type': 'v1', 'addr': '172.21.15.43:6807', 'nonce': 2505220646}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6804', 'nonce': 2505220646}, {'type': 'v1', 'addr': '172.21.15.43:6805', 'nonce': 2505220646}]}, 'public_addr': '172.21.15.43:6801/2505220646', 'cluster_addr': '172.21.15.43:6803/2505220646', 'heartbeat_back_addr': '172.21.15.43:6807/2505220646', 'heartbeat_front_addr': '172.21.15.43:6805/2505220646', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '4a208e30-031b-4a9f-af52-e26d4a039ea6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 91, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6816', 'nonce': 1517367307}, {'type': 'v1', 'addr': '172.21.15.43:6817', 'nonce': 1517367307}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6818', 'nonce': 1517367307}, {'type': 'v1', 'addr': '172.21.15.43:6819', 'nonce': 1517367307}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6822', 'nonce': 1517367307}, {'type': 'v1', 'addr': '172.21.15.43:6823', 'nonce': 1517367307}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6820', 'nonce': 1517367307}, {'type': 'v1', 'addr': '172.21.15.43:6821', 'nonce': 1517367307}]}, 'public_addr': '172.21.15.43:6817/1517367307', 'cluster_addr': '172.21.15.43:6819/1517367307', 'heartbeat_back_addr': '172.21.15.43:6823/1517367307', 'heartbeat_front_addr': '172.21.15.43:6821/1517367307', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T20:47:47.887412+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T20:47:47.658782+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T20:47:47.453752+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T20:47:47.487992+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T20:47:46.943344+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T20:47:47.901547+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.c', 'mappings': [{'from': 0, 'to': 5}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-16T20:49:26.242 INFO:tasks.thrashosds.thrasher:Clearing pg_upmap on 3.c 2024-09-16T20:49:26.242 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd rm-pg-upmap-items 3.c 2024-09-16T20:49:26.242 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:26.239+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:26.242 INFO:teuthology.orchestra.run.smithi139.stdout:3547 write 0xc1b2800 thru 0xc1bbc47 (0x9448 bytes) 2024-09-16T20:49:26.243 INFO:teuthology.orchestra.run.smithi139.stdout:3550 punch from 0x67daa00 to 0x67e8200, (0xd800 bytes) 2024-09-16T20:49:26.243 INFO:teuthology.orchestra.run.smithi139.stdout:3554 read 0x72c7800 thru 0x72c7c1d (0x41e bytes) 2024-09-16T20:49:26.243 INFO:teuthology.orchestra.run.smithi139.stdout:3561 write 0x7813400 thru 0x7816434 (0x3035 bytes) 2024-09-16T20:49:26.243 INFO:teuthology.orchestra.run.smithi139.stdout:3562 read 0x9cc4200 thru 0x9cc7b19 (0x391a bytes) 2024-09-16T20:49:26.243 INFO:teuthology.orchestra.run.smithi139.stdout:3563 write 0x8623400 thru 0x862f80c (0xc40d bytes) 2024-09-16T20:49:26.243 INFO:teuthology.orchestra.run.smithi139.stdout:3564 write 0xd23000 thru 0xd2eb10 (0xbb11 bytes) 2024-09-16T20:49:26.243 INFO:teuthology.orchestra.run.smithi139.stdout:3565 trunc from 0xc1bbe00 to 0x3087a00 2024-09-16T20:49:26.271 INFO:teuthology.orchestra.run.smithi139.stdout:3566 write 0xb30b000 thru 0xb30bf95 (0xf96 bytes) 2024-09-16T20:49:26.274 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:26.274 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:26.279 INFO:teuthology.orchestra.run.smithi139.stdout:3567 trunc from 0xb30c000 to 0x4d59000 2024-09-16T20:49:26.284 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:49:26.312 INFO:teuthology.orchestra.run.smithi139.stdout:3568 write 0x8eec200 thru 0x8ef3bb9 (0x79ba bytes) 2024-09-16T20:49:26.313 INFO:teuthology.orchestra.run.smithi139.stdout:3569 read 0x5387c00 thru 0x5391653 (0x9a54 bytes) 2024-09-16T20:49:26.313 INFO:teuthology.orchestra.run.smithi139.stdout:3574 read 0x63d0200 thru 0x63dc25e (0xc05f bytes) 2024-09-16T20:49:26.316 INFO:teuthology.orchestra.run.smithi139.stdout:3575 trunc from 0x8ef3c00 to 0xb0fe800 2024-09-16T20:49:26.327 INFO:teuthology.orchestra.run.smithi139.stdout:3576 write 0x1067800 thru 0x107062c (0x8e2d bytes) 2024-09-16T20:49:26.329 INFO:teuthology.orchestra.run.smithi139.stdout:3577 read 0x928b800 thru 0x929729a (0xba9b bytes) 2024-09-16T20:49:26.329 INFO:teuthology.orchestra.run.smithi139.stdout:3578 punch from 0xa2ad200 to 0xa2b9400, (0xc200 bytes) 2024-09-16T20:49:26.333 INFO:teuthology.orchestra.run.smithi139.stdout:3580 read 0x60b0600 thru 0x60bfcac (0xf6ad bytes) 2024-09-16T20:49:26.333 INFO:teuthology.orchestra.run.smithi139.stdout:3581 write 0x3247600 thru 0x32569ac (0xf3ad bytes) 2024-09-16T20:49:26.333 INFO:teuthology.orchestra.run.smithi139.stdout:3585 trunc from 0xb0fe800 to 0xc697400 2024-09-16T20:49:26.341 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:26.339+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:26.344 INFO:teuthology.orchestra.run.smithi139.stdout:3590 read 0x722dc00 thru 0x7233de7 (0x61e8 bytes) 2024-09-16T20:49:26.360 INFO:teuthology.orchestra.run.smithi139.stdout:3591 trunc from 0xc697400 to 0x793200 2024-09-16T20:49:26.383 INFO:teuthology.orchestra.run.smithi139.stdout:truncating to largest ever: 0xeddf000 2024-09-16T20:49:26.400 INFO:teuthology.orchestra.run.smithi139.stdout:3593 trunc from 0x793200 to 0xeddf000 2024-09-16T20:49:26.406 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:26.406 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:26.410 INFO:teuthology.orchestra.run.smithi139.stdout:3594 punch from 0x3e8fc00 to 0x3e93c00, (0x4000 bytes) 2024-09-16T20:49:26.412 INFO:teuthology.orchestra.run.smithi139.stdout:3596 read 0xc9f2800 thru 0xc9f3951 (0x1152 bytes) 2024-09-16T20:49:26.412 INFO:teuthology.orchestra.run.smithi139.stdout:3598 punch from 0x6d1fa00 to 0x6d2a200, (0xa800 bytes) 2024-09-16T20:49:26.412 INFO:teuthology.orchestra.run.smithi139.stdout:3605 read 0x733c600 thru 0x734c37d (0xfd7e bytes) 2024-09-16T20:49:26.413 INFO:teuthology.orchestra.run.smithi139.stdout:3606 write 0xe5d000 thru 0xe61caf (0x4cb0 bytes) 2024-09-16T20:49:26.416 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:49:26.417 INFO:teuthology.orchestra.run.smithi139.stdout:3607 read 0x396a800 thru 0x39780fc (0xd8fd bytes) 2024-09-16T20:49:26.417 INFO:teuthology.orchestra.run.smithi139.stdout:3609 write 0x4dfb200 thru 0x4e03b52 (0x8953 bytes) 2024-09-16T20:49:26.417 INFO:teuthology.orchestra.run.smithi139.stdout:3610 write 0xe3e6800 thru 0xe3f15ac (0xadad bytes) 2024-09-16T20:49:26.417 INFO:teuthology.orchestra.run.smithi139.stdout:3613 write 0x5e94c00 thru 0x5ea29df (0xdde0 bytes) 2024-09-16T20:49:26.417 INFO:teuthology.orchestra.run.smithi139.stdout:3616 read 0x4969000 thru 0x496bd27 (0x2d28 bytes) 2024-09-16T20:49:26.425 INFO:teuthology.orchestra.run.smithi139.stdout:3617 trunc from 0xeddf000 to 0x7035400 2024-09-16T20:49:26.450 INFO:teuthology.orchestra.run.smithi139.stdout:3619 write 0x35c4c00 thru 0x35d3f83 (0xf384 bytes) 2024-09-16T20:49:26.450 INFO:teuthology.orchestra.run.smithi139.stdout:3621 write 0x13bc00 thru 0x14adbd (0xf1be bytes) 2024-09-16T20:49:26.451 INFO:teuthology.orchestra.run.smithi139.stdout:3623 read 0x3b5a000 thru 0x3b60347 (0x6348 bytes) 2024-09-16T20:49:26.452 INFO:teuthology.orchestra.run.smithi139.stdout:3624 write 0x4bf4200 thru 0x4bff1c8 (0xafc9 bytes) 2024-09-16T20:49:26.452 INFO:teuthology.orchestra.run.smithi139.stdout:3625 write 0x4818200 thru 0x48225eb (0xa3ec bytes) 2024-09-16T20:49:26.452 INFO:teuthology.orchestra.run.smithi139.stdout:3626 punch from 0x6dd2600 to 0x6ddec00, (0xc600 bytes) 2024-09-16T20:49:26.453 INFO:teuthology.orchestra.run.smithi139.stdout:3628 read 0x360a000 thru 0x36111e9 (0x71ea bytes) 2024-09-16T20:49:26.455 INFO:teuthology.orchestra.run.smithi139.stdout:3636 read 0x3435200 thru 0x3437988 (0x2789 bytes) 2024-09-16T20:49:26.458 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:26.457+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:26.471 INFO:teuthology.orchestra.run.smithi139.stdout:3637 write 0x7a96400 thru 0x7aa362f (0xd230 bytes) 2024-09-16T20:49:26.472 INFO:teuthology.orchestra.run.smithi139.stdout:3638 punch from 0x37f9200 to 0x3808200, (0xf000 bytes) 2024-09-16T20:49:26.473 INFO:teuthology.orchestra.run.smithi139.stdout:3642 trunc from 0x7aa3800 to 0x73d7e00 2024-09-16T20:49:26.499 INFO:teuthology.orchestra.run.smithi139.stdout:3644 write 0x7da2e00 thru 0x7da3074 (0x275 bytes) 2024-09-16T20:49:26.506 INFO:teuthology.orchestra.run.smithi139.stdout:3646 trunc from 0x7da3200 to 0x2e7b000 2024-09-16T20:49:26.532 INFO:teuthology.orchestra.run.smithi139.stdout:3647 trunc from 0x2e7b000 to 0x1e4a800 2024-09-16T20:49:26.542 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:26.541+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:26.543 INFO:teuthology.orchestra.run.smithi139.stdout:3650 read 0x15be000 thru 0x15c088e (0x288f bytes) 2024-09-16T20:49:26.550 INFO:teuthology.orchestra.run.smithi139.stdout:3653 read 0x4aec00 thru 0x4b03a6 (0x17a7 bytes) 2024-09-16T20:49:26.550 INFO:teuthology.orchestra.run.smithi139.stdout:3655 punch from 0xf45600 to 0xf49200, (0x3c00 bytes) 2024-09-16T20:49:26.551 INFO:teuthology.orchestra.run.smithi139.stdout:3658 punch from 0x112ee00 to 0x1136000, (0x7200 bytes) 2024-09-16T20:49:26.551 INFO:teuthology.orchestra.run.smithi139.stdout:3660 read 0xa9e800 thru 0xaa519b (0x699c bytes) 2024-09-16T20:49:26.551 INFO:teuthology.orchestra.run.smithi139.stdout:3661 read 0xa29000 thru 0xa327af (0x97b0 bytes) 2024-09-16T20:49:26.561 INFO:teuthology.orchestra.run.smithi139.stdout:3663 write 0x1ea2a00 thru 0x1eb1149 (0xe74a bytes) 2024-09-16T20:49:26.574 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T20:49:26.576 INFO:teuthology.orchestra.run.smithi139.stdout:3664 trunc from 0x1eb1200 to 0x9371c00 2024-09-16T20:49:26.589 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:26.590 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:26.592 INFO:teuthology.orchestra.run.smithi139.stdout:3667 trunc from 0x9371c00 to 0x82fc600 2024-09-16T20:49:26.599 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:49:26.611 INFO:teuthology.orchestra.run.smithi139.stdout:3668 trunc from 0x82fc600 to 0x2d8600 2024-09-16T20:49:26.635 INFO:teuthology.orchestra.run.smithi139.stdout:3671 read 0xe200 thru 0xf5d7 (0x13d8 bytes) 2024-09-16T20:49:26.643 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:26.641+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:26.644 INFO:teuthology.orchestra.run.smithi139.stdout:3675 trunc from 0x2d8600 to 0x8152200 2024-09-16T20:49:26.654 INFO:teuthology.orchestra.run.smithi139.stdout:3677 punch from 0x6f90800 to 0x6f9b600, (0xae00 bytes) 2024-09-16T20:49:26.668 INFO:teuthology.orchestra.run.smithi139.stdout:3678 trunc from 0x8152200 to 0xe8d5000 2024-09-16T20:49:26.677 INFO:teuthology.orchestra.run.smithi139.stdout:3679 write 0xa92f000 thru 0xa9319ae (0x29af bytes) 2024-09-16T20:49:26.682 INFO:teuthology.orchestra.run.smithi139.stdout:3680 write 0x7f48400 thru 0x7f4aa26 (0x2627 bytes) 2024-09-16T20:49:26.683 INFO:teuthology.orchestra.run.smithi139.stdout:3681 write 0x3bf6c00 thru 0x3bfae84 (0x4285 bytes) 2024-09-16T20:49:26.683 INFO:teuthology.orchestra.run.smithi139.stdout:3684 write 0x33f1800 thru 0x33f2287 (0xa88 bytes) 2024-09-16T20:49:26.683 INFO:teuthology.orchestra.run.smithi139.stdout:3685 punch from 0x9cee000 to 0x9cf6800, (0x8800 bytes) 2024-09-16T20:49:26.683 INFO:teuthology.orchestra.run.smithi139.stdout:3686 read 0x89ad400 thru 0x89b112b (0x3d2c bytes) 2024-09-16T20:49:26.683 INFO:teuthology.orchestra.run.smithi139.stdout:3687 write 0x4206800 thru 0x42069e1 (0x1e2 bytes) 2024-09-16T20:49:26.688 INFO:teuthology.orchestra.run.smithi139.stdout:3688 trunc from 0xe8d5000 to 0xa658800 2024-09-16T20:49:26.705 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:49:26.705 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:49:26.706 INFO:teuthology.orchestra.run.smithi139.stdout:3691 write 0x9941400 thru 0x9945006 (0x3c07 bytes) 2024-09-16T20:49:26.711 INFO:teuthology.orchestra.run.smithi139.stdout:3694 write 0x7052c00 thru 0x7061bd2 (0xefd3 bytes) 2024-09-16T20:49:26.725 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:49:26.726 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:49:26.738 INFO:teuthology.orchestra.run.smithi139.stdout:3697 write 0xea07e00 thru 0xea11935 (0x9b36 bytes) 2024-09-16T20:49:26.738 INFO:teuthology.orchestra.run.smithi139.stdout:3698 read 0x9fc8c00 thru 0x9fcbf39 (0x333a bytes) 2024-09-16T20:49:26.739 INFO:teuthology.orchestra.run.smithi139.stdout:3699 write 0x92ba600 thru 0x92c422e (0x9c2f bytes) 2024-09-16T20:49:26.739 INFO:teuthology.orchestra.run.smithi139.stdout:3700 punch from 0x82f7400 to 0x8301200, (0x9e00 bytes) 2024-09-16T20:49:26.743 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:49:26.743 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:49:26.743 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:26.742+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:26.749 INFO:teuthology.orchestra.run.smithi139.stdout:3703 trunc from 0xea11a00 to 0x6c52400 2024-09-16T20:49:26.763 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:49:26.764 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:49:26.769 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:26.769 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:26.774 INFO:teuthology.orchestra.run.smithi139.stdout:3706 trunc from 0x6c52400 to 0xd25a800 2024-09-16T20:49:26.778 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:49:26.781 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:49:26.782 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:49:26.784 INFO:teuthology.orchestra.run.smithi139.stdout:3707 punch from 0x38e8800 to 0x38f0a00, (0x8200 bytes) 2024-09-16T20:49:26.785 INFO:teuthology.orchestra.run.smithi139.stdout:3708 write 0x7f9a400 thru 0x7fa2d07 (0x8908 bytes) 2024-09-16T20:49:26.785 INFO:teuthology.orchestra.run.smithi139.stdout:3709 read 0x629ea00 thru 0x629fb0a (0x110b bytes) 2024-09-16T20:49:26.785 INFO:teuthology.orchestra.run.smithi139.stdout:3710 write 0xae51c00 thru 0xae5834d (0x674e bytes) 2024-09-16T20:49:26.785 INFO:teuthology.orchestra.run.smithi139.stdout:3713 read 0x6afbc00 thru 0x6b0b294 (0xf695 bytes) 2024-09-16T20:49:26.786 INFO:teuthology.orchestra.run.smithi139.stdout:3714 write 0x7f28800 thru 0x7f2b435 (0x2c36 bytes) 2024-09-16T20:49:26.786 INFO:teuthology.orchestra.run.smithi139.stdout:3715 punch from 0xa6a7600 to 0xa6ad400, (0x5e00 bytes) 2024-09-16T20:49:26.792 INFO:teuthology.orchestra.run.smithi139.stdout:3718 trunc from 0xd25a800 to 0x7ea0800 2024-09-16T20:49:26.802 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:49:26.802 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:49:26.825 INFO:teuthology.orchestra.run.smithi139.stdout:3720 trunc from 0x7ea0800 to 0x1bc3600 2024-09-16T20:49:26.844 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:26.842+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:26.846 INFO:teuthology.orchestra.run.smithi029.stderr:clear 3.c pg_upmap_items mapping 2024-09-16T20:49:26.849 INFO:teuthology.orchestra.run.smithi139.stdout:3721 read 0xf80c00 thru 0xf89dbb (0x91bc bytes) 2024-09-16T20:49:26.850 INFO:teuthology.orchestra.run.smithi139.stdout:3724 read 0x10e0800 thru 0x10ef8e6 (0xf0e7 bytes) 2024-09-16T20:49:26.898 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:26.898 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:49:26.898 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:49:26.898 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:49:26.898 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.899 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:4904 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 7309312~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:26.899 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767724+0000", 2024-09-16T20:49:26.899 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 67.128816365999995, 2024-09-16T20:49:26.899 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050270214000000001, 2024-09-16T20:49:26.899 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:26.899 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:26.899 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:26.899 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:26.899 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:26.899 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:26.899 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4904 2024-09-16T20:49:26.899 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.899 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:26.900 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.900 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:26.900 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767724+0000", 2024-09-16T20:49:26.900 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:26.900 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.900 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.900 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:26.900 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767724+0000", 2024-09-16T20:49:26.900 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:26.900 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.900 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.900 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:26.901 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767724+0000", 2024-09-16T20:49:26.901 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1399999999999998e-07 2024-09-16T20:49:26.901 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.901 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.901 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:26.901 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767725+0000", 2024-09-16T20:49:26.901 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0380000000000001e-06 2024-09-16T20:49:26.901 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.901 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.901 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:26.901 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767729+0000", 2024-09-16T20:49:26.901 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2490000000000001e-06 2024-09-16T20:49:26.901 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.902 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.902 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:26.902 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767731+0000", 2024-09-16T20:49:26.902 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5940000000000001e-06 2024-09-16T20:49:26.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.902 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.902 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:26.902 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.817787+0000", 2024-09-16T20:49:26.902 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050055792000000002 2024-09-16T20:49:26.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.902 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.902 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:26.902 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.817883+0000", 2024-09-16T20:49:26.903 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5606999999999995e-05 2024-09-16T20:49:26.903 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.903 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.903 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:26.903 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.817994+0000", 2024-09-16T20:49:26.903 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011162000000000001 2024-09-16T20:49:26.903 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:26.903 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:26.903 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:26.903 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.903 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.903 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:4906 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 7317504~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:26.903 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767734+0000", 2024-09-16T20:49:26.904 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 67.128806103000002, 2024-09-16T20:49:26.904 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050406608999999998, 2024-09-16T20:49:26.904 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:26.904 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:26.904 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:26.904 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:26.904 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:26.904 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:26.904 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4906 2024-09-16T20:49:26.904 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.904 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:26.904 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.904 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:26.905 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767734+0000", 2024-09-16T20:49:26.905 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:26.905 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.905 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.905 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:26.905 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767734+0000", 2024-09-16T20:49:26.905 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:26.905 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.905 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.905 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:26.905 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767735+0000", 2024-09-16T20:49:26.905 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.05e-07 2024-09-16T20:49:26.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.906 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.906 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:26.906 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767736+0000", 2024-09-16T20:49:26.906 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0240000000000001e-06 2024-09-16T20:49:26.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.906 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.906 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:26.906 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767739+0000", 2024-09-16T20:49:26.906 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.067e-06 2024-09-16T20:49:26.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.906 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.906 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:26.907 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767742+0000", 2024-09-16T20:49:26.907 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.23e-06 2024-09-16T20:49:26.907 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.907 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.907 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:26.907 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818027+0000", 2024-09-16T20:49:26.907 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050285102999999998 2024-09-16T20:49:26.907 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.907 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.907 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:26.907 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818072+0000", 2024-09-16T20:49:26.907 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5129000000000001e-05 2024-09-16T20:49:26.908 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.908 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.908 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:26.908 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818141+0000", 2024-09-16T20:49:26.908 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8850999999999995e-05 2024-09-16T20:49:26.908 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:26.908 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:26.908 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:26.908 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.908 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.908 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:4908 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 7325696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:26.908 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767745+0000", 2024-09-16T20:49:26.908 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 67.128795707999998, 2024-09-16T20:49:26.908 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050654230000000001, 2024-09-16T20:49:26.909 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:26.909 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:26.909 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:26.909 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:26.909 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:26.909 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:26.909 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4908 2024-09-16T20:49:26.909 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.909 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:26.909 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.909 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:26.909 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767745+0000", 2024-09-16T20:49:26.909 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:26.910 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.910 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.910 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:26.910 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767745+0000", 2024-09-16T20:49:26.910 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:26.910 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.910 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.910 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:26.910 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767745+0000", 2024-09-16T20:49:26.910 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1e-07 2024-09-16T20:49:26.910 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.910 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.910 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:26.911 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767749+0000", 2024-09-16T20:49:26.911 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.532e-06 2024-09-16T20:49:26.911 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.911 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.911 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:26.911 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767753+0000", 2024-09-16T20:49:26.911 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4939999999999999e-06 2024-09-16T20:49:26.911 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.911 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.911 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:26.911 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767756+0000", 2024-09-16T20:49:26.911 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.61e-06 2024-09-16T20:49:26.912 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.912 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.912 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:26.912 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818184+0000", 2024-09-16T20:49:26.912 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050428615000000003 2024-09-16T20:49:26.912 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.912 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.912 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:26.912 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818276+0000", 2024-09-16T20:49:26.912 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2077999999999995e-05 2024-09-16T20:49:26.912 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.912 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.912 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:26.912 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818399+0000", 2024-09-16T20:49:26.913 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012269099999999999 2024-09-16T20:49:26.913 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:26.913 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:26.913 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:26.913 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.913 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.913 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:4910 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 7333888~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:26.913 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767759+0000", 2024-09-16T20:49:26.913 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 67.128781817999993, 2024-09-16T20:49:26.913 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050815076000000001, 2024-09-16T20:49:26.913 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:26.913 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:26.913 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:26.913 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:26.914 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:26.914 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:26.914 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4910 2024-09-16T20:49:26.914 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.914 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:26.914 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.914 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:26.914 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767759+0000", 2024-09-16T20:49:26.914 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:26.914 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.914 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.914 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:26.914 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767759+0000", 2024-09-16T20:49:26.914 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:26.915 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.915 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.915 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:26.915 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767759+0000", 2024-09-16T20:49:26.915 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2600000000000001e-07 2024-09-16T20:49:26.915 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.915 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.915 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:26.915 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767760+0000", 2024-09-16T20:49:26.915 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0640000000000001e-06 2024-09-16T20:49:26.915 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.915 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.915 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:26.915 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767763+0000", 2024-09-16T20:49:26.916 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.994e-06 2024-09-16T20:49:26.916 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.916 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.916 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:26.916 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767765+0000", 2024-09-16T20:49:26.916 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5220000000000002e-06 2024-09-16T20:49:26.916 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.916 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.916 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:26.916 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818438+0000", 2024-09-16T20:49:26.916 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050672602999999997 2024-09-16T20:49:26.916 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.916 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.917 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:26.917 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818481+0000", 2024-09-16T20:49:26.917 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2967999999999999e-05 2024-09-16T20:49:26.917 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.917 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.917 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:26.917 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818574+0000", 2024-09-16T20:49:26.917 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2699000000000001e-05 2024-09-16T20:49:26.917 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:26.917 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:26.917 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:26.917 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.917 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.918 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5012 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2101248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:26.918 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767768+0000", 2024-09-16T20:49:26.918 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 67.128772169000001, 2024-09-16T20:49:26.918 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051084682999999999, 2024-09-16T20:49:26.918 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:26.918 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:26.918 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:26.918 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:26.918 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:26.918 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:26.918 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5012 2024-09-16T20:49:26.918 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.918 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:26.919 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.919 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:26.919 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767768+0000", 2024-09-16T20:49:26.919 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:26.919 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.919 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.919 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:26.919 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767768+0000", 2024-09-16T20:49:26.919 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:26.919 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.919 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.919 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:26.919 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767769+0000", 2024-09-16T20:49:26.920 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1500000000000001e-07 2024-09-16T20:49:26.920 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.920 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.920 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:26.920 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767770+0000", 2024-09-16T20:49:26.920 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.015e-06 2024-09-16T20:49:26.920 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.920 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.920 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:26.920 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767773+0000", 2024-09-16T20:49:26.920 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1070000000000001e-06 2024-09-16T20:49:26.920 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.920 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.921 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:26.921 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767775+0000", 2024-09-16T20:49:26.921 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.508e-06 2024-09-16T20:49:26.921 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.921 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.921 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:26.921 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818630+0000", 2024-09-16T20:49:26.921 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050855282000000002 2024-09-16T20:49:26.921 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.921 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.921 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:26.921 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818732+0000", 2024-09-16T20:49:26.921 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101531 2024-09-16T20:49:26.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.922 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.922 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:26.922 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818853+0000", 2024-09-16T20:49:26.922 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000121025 2024-09-16T20:49:26.922 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:26.922 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:26.922 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:26.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.922 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.922 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5014 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2109440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:26.922 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767778+0000", 2024-09-16T20:49:26.922 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 67.128762233000003, 2024-09-16T20:49:26.922 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051260776000000001, 2024-09-16T20:49:26.922 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:26.923 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:26.923 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:26.923 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:26.923 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:26.923 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:26.923 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5014 2024-09-16T20:49:26.923 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.923 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:26.923 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.923 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:26.923 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767778+0000", 2024-09-16T20:49:26.923 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:26.923 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.923 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.923 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:26.924 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767778+0000", 2024-09-16T20:49:26.924 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:26.924 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.924 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.924 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:26.924 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767824+0000", 2024-09-16T20:49:26.924 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5890999999999998e-05 2024-09-16T20:49:26.924 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.924 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.924 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:26.924 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767827+0000", 2024-09-16T20:49:26.924 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.644e-06 2024-09-16T20:49:26.924 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.925 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.925 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:26.925 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767832+0000", 2024-09-16T20:49:26.925 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2689999999999999e-06 2024-09-16T20:49:26.925 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.925 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.925 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:26.925 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767835+0000", 2024-09-16T20:49:26.925 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2100000000000002e-06 2024-09-16T20:49:26.925 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.925 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.925 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:26.925 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818887+0000", 2024-09-16T20:49:26.925 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051051276999999999 2024-09-16T20:49:26.926 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.926 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.926 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:26.926 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818953+0000", 2024-09-16T20:49:26.926 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6105999999999994e-05 2024-09-16T20:49:26.926 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.926 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.926 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:26.926 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819039+0000", 2024-09-16T20:49:26.926 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6378999999999994e-05 2024-09-16T20:49:26.926 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:26.926 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:26.926 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:26.927 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.927 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.927 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5016 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2117632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:26.927 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767841+0000", 2024-09-16T20:49:26.927 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 67.128699553999994, 2024-09-16T20:49:26.927 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051433288000000001, 2024-09-16T20:49:26.927 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:26.927 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:26.927 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:26.927 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:26.927 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:26.927 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:26.927 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5016 2024-09-16T20:49:26.928 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.928 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:26.928 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.928 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:26.928 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767841+0000", 2024-09-16T20:49:26.928 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:26.928 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.928 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.928 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:26.928 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767841+0000", 2024-09-16T20:49:26.928 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:26.928 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.928 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.929 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:26.929 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768156+0000", 2024-09-16T20:49:26.929 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00031505699999999999 2024-09-16T20:49:26.929 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.929 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.929 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:26.929 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768159+0000", 2024-09-16T20:49:26.929 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1659999999999998e-06 2024-09-16T20:49:26.929 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.929 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.929 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:26.929 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768166+0000", 2024-09-16T20:49:26.929 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0550000000000002e-06 2024-09-16T20:49:26.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.930 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.930 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:26.930 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768171+0000", 2024-09-16T20:49:26.930 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3170000000000003e-06 2024-09-16T20:49:26.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.930 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.930 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:26.930 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819084+0000", 2024-09-16T20:49:26.930 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050913355 2024-09-16T20:49:26.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.930 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.930 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:26.931 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819170+0000", 2024-09-16T20:49:26.931 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6328000000000004e-05 2024-09-16T20:49:26.931 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.931 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.931 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:26.931 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819274+0000", 2024-09-16T20:49:26.931 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010401 2024-09-16T20:49:26.931 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:26.931 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:26.931 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:26.931 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.931 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.931 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5018 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2125824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:26.931 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768178+0000", 2024-09-16T20:49:26.932 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 67.128362019999997, 2024-09-16T20:49:26.932 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051227013000000002, 2024-09-16T20:49:26.932 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:26.932 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:26.932 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:26.932 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:26.932 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:26.932 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:26.933 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5018 2024-09-16T20:49:26.933 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.933 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:26.933 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.933 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:26.933 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768178+0000", 2024-09-16T20:49:26.933 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:26.933 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.933 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.933 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:26.933 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768178+0000", 2024-09-16T20:49:26.933 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:26.933 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.934 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.934 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:26.934 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768179+0000", 2024-09-16T20:49:26.934 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9200000000000002e-07 2024-09-16T20:49:26.934 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.934 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.934 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:26.934 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768180+0000", 2024-09-16T20:49:26.934 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3790000000000001e-06 2024-09-16T20:49:26.934 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.934 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.934 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:26.934 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768184+0000", 2024-09-16T20:49:26.934 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5470000000000001e-06 2024-09-16T20:49:26.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.935 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.935 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:26.935 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768187+0000", 2024-09-16T20:49:26.935 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8119999999999999e-06 2024-09-16T20:49:26.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.935 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.935 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:26.935 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819305+0000", 2024-09-16T20:49:26.935 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051118683999999998 2024-09-16T20:49:26.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.935 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.935 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:26.936 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819351+0000", 2024-09-16T20:49:26.936 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5757999999999998e-05 2024-09-16T20:49:26.936 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.936 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.936 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:26.936 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819405+0000", 2024-09-16T20:49:26.936 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4441000000000003e-05 2024-09-16T20:49:26.936 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:26.936 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:26.936 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:26.936 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.936 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.936 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5020 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2134016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:26.937 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768190+0000", 2024-09-16T20:49:26.937 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 67.128350342000005, 2024-09-16T20:49:26.937 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051449661000000001, 2024-09-16T20:49:26.937 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:26.937 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:26.937 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:26.937 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:26.937 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:26.937 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:26.937 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5020 2024-09-16T20:49:26.937 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.937 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:26.937 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.937 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:26.938 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768190+0000", 2024-09-16T20:49:26.938 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:26.938 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.938 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.938 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:26.938 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768190+0000", 2024-09-16T20:49:26.938 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:26.938 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.938 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.938 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:26.938 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768282+0000", 2024-09-16T20:49:26.938 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2263999999999996e-05 2024-09-16T20:49:26.938 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.939 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:26.939 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768286+0000", 2024-09-16T20:49:26.939 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.529e-06 2024-09-16T20:49:26.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.939 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:26.939 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768294+0000", 2024-09-16T20:49:26.939 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6990000000000003e-06 2024-09-16T20:49:26.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.939 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:26.939 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768306+0000", 2024-09-16T20:49:26.939 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2112e-05 2024-09-16T20:49:26.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:26.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819449+0000", 2024-09-16T20:49:26.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051143521999999997 2024-09-16T20:49:26.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:26.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819539+0000", 2024-09-16T20:49:26.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0022e-05 2024-09-16T20:49:26.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:26.941 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819640+0000", 2024-09-16T20:49:26.941 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000100513 2024-09-16T20:49:26.941 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:26.941 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:26.941 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:26.941 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.941 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.941 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5022 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2142208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:26.941 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768326+0000", 2024-09-16T20:49:26.941 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 67.128214952999997, 2024-09-16T20:49:26.941 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051436952000000001, 2024-09-16T20:49:26.941 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:26.941 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:26.941 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:26.942 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:26.942 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:26.942 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:26.942 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5022 2024-09-16T20:49:26.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.942 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:26.942 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.942 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:26.942 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768326+0000", 2024-09-16T20:49:26.942 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:26.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.942 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.942 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:26.942 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768326+0000", 2024-09-16T20:49:26.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:26.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.943 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:26.943 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768477+0000", 2024-09-16T20:49:26.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015109300000000001 2024-09-16T20:49:26.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.943 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:26.943 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768480+0000", 2024-09-16T20:49:26.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8580000000000002e-06 2024-09-16T20:49:26.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.944 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:26.944 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768489+0000", 2024-09-16T20:49:26.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4600000000000003e-06 2024-09-16T20:49:26.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.944 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:26.944 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768493+0000", 2024-09-16T20:49:26.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2270000000000004e-06 2024-09-16T20:49:26.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.944 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:26.944 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819672+0000", 2024-09-16T20:49:26.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051178605000000002 2024-09-16T20:49:26.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.945 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:26.945 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819711+0000", 2024-09-16T20:49:26.945 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9357000000000003e-05 2024-09-16T20:49:26.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.945 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:26.945 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819762+0000", 2024-09-16T20:49:26.945 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1351999999999999e-05 2024-09-16T20:49:26.945 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:26.945 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:26.945 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:26.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.946 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.946 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5024 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2150400~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:26.946 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768501+0000", 2024-09-16T20:49:26.946 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 67.128039474999994, 2024-09-16T20:49:26.946 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051402678, 2024-09-16T20:49:26.946 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:26.946 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:26.946 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:26.946 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:26.946 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:26.946 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:26.946 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5024 2024-09-16T20:49:26.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.947 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:26.947 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.947 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:26.947 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768501+0000", 2024-09-16T20:49:26.947 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:26.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.947 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.947 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:26.947 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768501+0000", 2024-09-16T20:49:26.947 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:26.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.947 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.947 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:26.948 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768619+0000", 2024-09-16T20:49:26.948 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118187 2024-09-16T20:49:26.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.948 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:26.948 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768624+0000", 2024-09-16T20:49:26.948 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3259999999999997e-06 2024-09-16T20:49:26.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.948 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:26.948 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768631+0000", 2024-09-16T20:49:26.948 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6490000000000008e-06 2024-09-16T20:49:26.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.949 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.949 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:26.949 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768637+0000", 2024-09-16T20:49:26.949 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4210000000000003e-06 2024-09-16T20:49:26.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.949 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.949 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:26.949 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819800+0000", 2024-09-16T20:49:26.949 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051163486000000001 2024-09-16T20:49:26.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.949 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.949 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:26.950 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819845+0000", 2024-09-16T20:49:26.950 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4830999999999998e-05 2024-09-16T20:49:26.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.950 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.950 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:26.950 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819904+0000", 2024-09-16T20:49:26.950 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8777999999999999e-05 2024-09-16T20:49:26.950 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:26.950 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:26.950 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:26.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.950 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.950 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5026 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2158592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:26.951 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768646+0000", 2024-09-16T20:49:26.951 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 67.127894026999996, 2024-09-16T20:49:26.951 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051384386999999997, 2024-09-16T20:49:26.951 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:26.951 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:26.951 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:26.951 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:26.951 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:26.951 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:26.951 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5026 2024-09-16T20:49:26.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.951 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:26.951 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.951 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:26.952 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768646+0000", 2024-09-16T20:49:26.952 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:26.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.952 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.952 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:26.952 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768646+0000", 2024-09-16T20:49:26.952 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:26.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.952 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.952 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:26.952 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768775+0000", 2024-09-16T20:49:26.952 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012813200000000001 2024-09-16T20:49:26.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.953 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.953 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:26.953 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768778+0000", 2024-09-16T20:49:26.953 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2150000000000001e-06 2024-09-16T20:49:26.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.953 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.953 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:26.953 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768786+0000", 2024-09-16T20:49:26.953 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3259999999999996e-06 2024-09-16T20:49:26.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.953 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.953 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:26.953 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768790+0000", 2024-09-16T20:49:26.954 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.216e-06 2024-09-16T20:49:26.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.954 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.954 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:26.954 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819939+0000", 2024-09-16T20:49:26.954 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051149018999999997 2024-09-16T20:49:26.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.954 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.956 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:26.956 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819979+0000", 2024-09-16T20:49:26.956 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9311999999999999e-05 2024-09-16T20:49:26.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.956 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:26.956 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820031+0000", 2024-09-16T20:49:26.956 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2166999999999997e-05 2024-09-16T20:49:26.956 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:26.956 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:26.956 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:26.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.957 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5028 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2166784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:26.957 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768798+0000", 2024-09-16T20:49:26.957 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 67.127742384000001, 2024-09-16T20:49:26.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051388912000000002, 2024-09-16T20:49:26.957 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:26.957 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:26.957 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:26.957 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:26.957 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:26.957 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:26.957 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5028 2024-09-16T20:49:26.958 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.958 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:26.958 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.958 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:26.958 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768798+0000", 2024-09-16T20:49:26.958 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:26.958 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.958 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.958 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:26.958 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768798+0000", 2024-09-16T20:49:26.958 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:26.958 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.958 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.959 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:26.959 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768989+0000", 2024-09-16T20:49:26.959 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000190951 2024-09-16T20:49:26.959 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.959 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.959 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:26.959 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768993+0000", 2024-09-16T20:49:26.959 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9770000000000002e-06 2024-09-16T20:49:26.959 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.959 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.959 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:26.959 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.769002+0000", 2024-09-16T20:49:26.959 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5469999999999997e-06 2024-09-16T20:49:26.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.960 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:26.960 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.769020+0000", 2024-09-16T20:49:26.960 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8216999999999999e-05 2024-09-16T20:49:26.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.960 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:26.960 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820070+0000", 2024-09-16T20:49:26.960 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051049998999999999 2024-09-16T20:49:26.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.960 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:26.961 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820119+0000", 2024-09-16T20:49:26.961 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8962000000000003e-05 2024-09-16T20:49:26.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.961 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.961 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:26.961 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820187+0000", 2024-09-16T20:49:26.961 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8258999999999998e-05 2024-09-16T20:49:26.961 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:26.961 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:26.961 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:26.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.961 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.961 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5030 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2174976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:26.962 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.769031+0000", 2024-09-16T20:49:26.962 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 67.127509270999994, 2024-09-16T20:49:26.962 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051289830000000002, 2024-09-16T20:49:26.962 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:26.962 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:26.962 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:26.962 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:26.962 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:26.962 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:26.962 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5030 2024-09-16T20:49:26.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.962 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:26.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.963 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:26.963 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.769031+0000", 2024-09-16T20:49:26.963 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:26.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.963 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.963 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:26.963 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.769031+0000", 2024-09-16T20:49:26.963 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:26.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.963 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.963 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:26.963 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.773332+0000", 2024-09-16T20:49:26.963 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0043004039999999999 2024-09-16T20:49:26.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.964 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.964 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:26.964 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.773336+0000", 2024-09-16T20:49:26.964 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7849999999999999e-06 2024-09-16T20:49:26.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.964 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.964 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:26.964 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.773347+0000", 2024-09-16T20:49:26.964 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0482999999999999e-05 2024-09-16T20:49:26.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.964 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.964 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:26.965 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.773354+0000", 2024-09-16T20:49:26.965 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1559999999999998e-06 2024-09-16T20:49:26.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.965 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.965 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:26.965 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820216+0000", 2024-09-16T20:49:26.965 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046861591000000001 2024-09-16T20:49:26.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.965 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.965 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:26.965 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820258+0000", 2024-09-16T20:49:26.965 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2653e-05 2024-09-16T20:49:26.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.965 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.966 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:26.966 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820321+0000", 2024-09-16T20:49:26.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2757999999999998e-05 2024-09-16T20:49:26.966 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:26.966 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:26.966 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:26.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.966 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5067 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 6918144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:26.966 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.775279+0000", 2024-09-16T20:49:26.966 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 67.121261004999994, 2024-09-16T20:49:26.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050171387999999997, 2024-09-16T20:49:26.966 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:26.966 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:26.967 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:26.967 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:26.967 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:26.967 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:26.967 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5067 2024-09-16T20:49:26.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.967 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:26.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.967 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:26.967 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775279+0000", 2024-09-16T20:49:26.967 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:26.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.968 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:26.968 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775279+0000", 2024-09-16T20:49:26.968 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:26.968 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.968 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.968 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:26.968 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775432+0000", 2024-09-16T20:49:26.968 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015257399999999999 2024-09-16T20:49:26.968 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.968 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.968 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:26.968 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775436+0000", 2024-09-16T20:49:26.968 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0180000000000003e-06 2024-09-16T20:49:26.969 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.969 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.969 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:26.969 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775447+0000", 2024-09-16T20:49:26.969 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.077e-05 2024-09-16T20:49:26.969 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.969 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.969 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:26.969 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775456+0000", 2024-09-16T20:49:26.969 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4089999999999992e-06 2024-09-16T20:49:26.969 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.969 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.969 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:26.970 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825150+0000", 2024-09-16T20:49:26.970 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049693711000000002 2024-09-16T20:49:26.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.970 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.970 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:26.970 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825337+0000", 2024-09-16T20:49:26.970 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000187214 2024-09-16T20:49:26.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.970 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.970 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:26.970 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825451+0000", 2024-09-16T20:49:26.970 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011369199999999999 2024-09-16T20:49:26.970 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:26.971 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:26.971 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:26.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.971 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.971 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5068 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2330624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:26.971 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.775469+0000", 2024-09-16T20:49:26.971 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 67.121071044000004, 2024-09-16T20:49:26.971 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050198334999999997, 2024-09-16T20:49:26.971 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:26.971 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:26.971 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:26.971 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:26.971 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:26.972 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:26.972 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5068 2024-09-16T20:49:26.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.972 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:26.972 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.972 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:26.972 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775469+0000", 2024-09-16T20:49:26.972 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:26.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.972 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.972 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:26.972 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775469+0000", 2024-09-16T20:49:26.972 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:26.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.973 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.973 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:26.973 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775720+0000", 2024-09-16T20:49:26.973 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000250661 2024-09-16T20:49:26.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.973 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.973 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:26.973 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775724+0000", 2024-09-16T20:49:26.973 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8700000000000002e-06 2024-09-16T20:49:26.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.973 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.973 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:26.973 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775733+0000", 2024-09-16T20:49:26.974 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4239999999999999e-06 2024-09-16T20:49:26.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.974 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:26.974 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775740+0000", 2024-09-16T20:49:26.974 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9519999999999997e-06 2024-09-16T20:49:26.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.974 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:26.974 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825484+0000", 2024-09-16T20:49:26.974 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049743385000000001 2024-09-16T20:49:26.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.975 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:26.975 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825572+0000", 2024-09-16T20:49:26.975 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7834e-05 2024-09-16T20:49:26.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.975 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.975 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:26.975 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825668+0000", 2024-09-16T20:49:26.975 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6209e-05 2024-09-16T20:49:26.975 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:26.975 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:26.975 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:26.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.975 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.976 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5069 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 6926336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:26.976 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.775753+0000", 2024-09-16T20:49:26.976 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 67.120787820999993, 2024-09-16T20:49:26.976 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050146430999999998, 2024-09-16T20:49:26.976 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:26.976 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:26.976 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:26.976 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:26.976 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:26.976 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:26.976 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5069 2024-09-16T20:49:26.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.976 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:26.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.977 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:26.977 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775753+0000", 2024-09-16T20:49:26.977 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:26.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.977 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:26.977 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775753+0000", 2024-09-16T20:49:26.977 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:26.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.977 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:26.977 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775865+0000", 2024-09-16T20:49:26.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000112402 2024-09-16T20:49:26.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.978 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.978 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:26.978 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775869+0000", 2024-09-16T20:49:26.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5130000000000001e-06 2024-09-16T20:49:26.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.978 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.978 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:26.978 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775877+0000", 2024-09-16T20:49:26.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1359999999999997e-06 2024-09-16T20:49:26.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.978 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.979 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:26.979 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775882+0000", 2024-09-16T20:49:26.979 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6710000000000004e-06 2024-09-16T20:49:26.979 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.979 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.979 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:26.979 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825729+0000", 2024-09-16T20:49:26.979 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049846898000000001 2024-09-16T20:49:26.979 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.979 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.979 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:26.979 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825807+0000", 2024-09-16T20:49:26.979 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7875999999999998e-05 2024-09-16T20:49:26.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.980 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.980 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:26.980 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825899+0000", 2024-09-16T20:49:26.980 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1935000000000007e-05 2024-09-16T20:49:26.980 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:26.980 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:26.980 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:26.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.980 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.980 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5070 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2338816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:26.980 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.775892+0000", 2024-09-16T20:49:26.980 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 67.120647966999996, 2024-09-16T20:49:26.981 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050155311000000001, 2024-09-16T20:49:26.981 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:26.981 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:26.981 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:26.981 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:26.981 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:26.981 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:26.981 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5070 2024-09-16T20:49:26.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.981 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:26.981 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.981 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:26.981 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775892+0000", 2024-09-16T20:49:26.982 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:26.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.982 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.982 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:26.982 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775892+0000", 2024-09-16T20:49:26.982 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:26.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.982 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.982 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:26.982 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776142+0000", 2024-09-16T20:49:26.982 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00024950500000000002 2024-09-16T20:49:26.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.982 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.982 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:26.983 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776146+0000", 2024-09-16T20:49:26.983 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2819999999999998e-06 2024-09-16T20:49:26.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.983 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.983 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:26.983 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776157+0000", 2024-09-16T20:49:26.983 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0807e-05 2024-09-16T20:49:26.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.983 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.983 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:26.983 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776165+0000", 2024-09-16T20:49:26.983 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1300000000000001e-06 2024-09-16T20:49:26.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.983 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.984 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:26.984 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825935+0000", 2024-09-16T20:49:26.984 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049769300000000002 2024-09-16T20:49:26.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.984 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.984 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:26.984 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825991+0000", 2024-09-16T20:49:26.984 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6177000000000001e-05 2024-09-16T20:49:26.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.984 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.984 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:26.984 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826048+0000", 2024-09-16T20:49:26.985 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.711e-05 2024-09-16T20:49:26.985 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:26.985 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:26.985 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:26.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.985 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.985 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5071 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 6934528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:26.985 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.776177+0000", 2024-09-16T20:49:26.985 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 67.120363564000002, 2024-09-16T20:49:26.985 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050036886000000003, 2024-09-16T20:49:26.985 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:26.985 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:26.985 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:26.985 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:26.986 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:26.986 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:26.986 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5071 2024-09-16T20:49:26.986 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.986 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:26.986 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.986 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:26.986 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776177+0000", 2024-09-16T20:49:26.986 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:26.986 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.986 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.986 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:26.986 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776177+0000", 2024-09-16T20:49:26.987 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:26.987 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.987 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.987 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:26.987 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776339+0000", 2024-09-16T20:49:26.987 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016238099999999999 2024-09-16T20:49:26.987 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.987 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.987 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:26.987 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776343+0000", 2024-09-16T20:49:26.987 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9670000000000003e-06 2024-09-16T20:49:26.987 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.987 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.987 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:26.988 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776354+0000", 2024-09-16T20:49:26.988 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0789e-05 2024-09-16T20:49:26.988 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.988 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.988 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:26.988 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776361+0000", 2024-09-16T20:49:26.988 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5200000000000003e-06 2024-09-16T20:49:26.988 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.988 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.988 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:26.988 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826078+0000", 2024-09-16T20:49:26.988 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049717513999999997 2024-09-16T20:49:26.989 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.989 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.989 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:26.989 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826141+0000", 2024-09-16T20:49:26.989 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2446000000000005e-05 2024-09-16T20:49:26.989 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.989 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.989 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:26.989 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826214+0000", 2024-09-16T20:49:26.989 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3269000000000006e-05 2024-09-16T20:49:26.989 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:26.989 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:26.989 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:26.990 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.990 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.990 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5072 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2347008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:26.990 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.776370+0000", 2024-09-16T20:49:26.990 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 67.120170002999998, 2024-09-16T20:49:26.990 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050097653999999998, 2024-09-16T20:49:26.990 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:26.990 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:26.990 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:26.990 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:26.990 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:26.990 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:26.990 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5072 2024-09-16T20:49:26.990 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.991 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:26.991 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.991 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:26.991 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776370+0000", 2024-09-16T20:49:26.991 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:26.991 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.991 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.991 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:26.991 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776370+0000", 2024-09-16T20:49:26.991 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:26.991 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.991 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.991 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:26.992 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776645+0000", 2024-09-16T20:49:26.992 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000274634 2024-09-16T20:49:26.992 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.992 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.992 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:26.992 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776654+0000", 2024-09-16T20:49:26.992 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9390000000000003e-06 2024-09-16T20:49:26.992 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.992 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.992 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:26.992 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776665+0000", 2024-09-16T20:49:26.992 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0524e-05 2024-09-16T20:49:26.992 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.992 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.993 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:26.993 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776671+0000", 2024-09-16T20:49:26.993 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7390000000000002e-06 2024-09-16T20:49:26.993 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.993 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.993 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:26.993 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826261+0000", 2024-09-16T20:49:26.993 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049589995999999997 2024-09-16T20:49:26.993 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.993 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.993 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:26.993 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826363+0000", 2024-09-16T20:49:26.993 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010158999999999999 2024-09-16T20:49:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826468+0000", 2024-09-16T20:49:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000105232 2024-09-16T20:49:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:26.995 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:26.996 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:49:26.997 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:26.995+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:27.097 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:27.095+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:27.146 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:27.146 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:49:27.146 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:49:27.146 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:27.154 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:49:27.198 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:27.196+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:27.298 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:27.297+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:27.301 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:27.301 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:49:27.301 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:49:27.301 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:49:27.301 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:27.310 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:49:27.399 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:27.398+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:27.458 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:27.458 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:49:27.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:49:27.458 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:49:27.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.459 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5286.0:10643 3.17 3.f6273fb7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-16T20:49:27.459 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:49:27.459 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.665782947, 2024-09-16T20:49:27.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.76672347200000002, 2024-09-16T20:49:27.459 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:27.459 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:27.459 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:49:27.459 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:27.460 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5286", 2024-09-16T20:49:27.460 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/4135433989", 2024-09-16T20:49:27.460 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 10643 2024-09-16T20:49:27.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.460 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:27.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:27.460 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:49:27.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:27.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:27.460 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:49:27.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:27.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:27.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790489+0000", 2024-09-16T20:49:27.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5910000000000001e-06 2024-09-16T20:49:27.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:27.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790493+0000", 2024-09-16T20:49:27.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2669999999999998e-06 2024-09-16T20:49:27.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.462 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:27.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790500+0000", 2024-09-16T20:49:27.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3780000000000003e-06 2024-09-16T20:49:27.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.462 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:27.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790505+0000", 2024-09-16T20:49:27.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1560000000000003e-06 2024-09-16T20:49:27.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.462 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:27.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790559+0000", 2024-09-16T20:49:27.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3494e-05 2024-09-16T20:49:27.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.463 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:49:27.463 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790566+0000", 2024-09-16T20:49:27.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7649999999999993e-06 2024-09-16T20:49:27.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.463 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:27.463 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:09.557198+0000", 2024-09-16T20:49:27.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.76663136200000004 2024-09-16T20:49:27.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.464 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:27.464 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:09.557211+0000", 2024-09-16T20:49:27.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3458999999999999e-05 2024-09-16T20:49:27.464 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:27.464 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:27.464 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:27.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.464 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3863 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1232896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:27.464 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896432+0000", 2024-09-16T20:49:27.464 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 12.559838263, 2024-09-16T20:49:27.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046223693000000003, 2024-09-16T20:49:27.464 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:27.464 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:27.465 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:27.465 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:27.465 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:27.465 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:27.465 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3863 2024-09-16T20:49:27.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.465 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:27.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.465 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:27.465 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896432+0000", 2024-09-16T20:49:27.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:27.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.465 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:27.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896432+0000", 2024-09-16T20:49:27.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:27.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:27.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896433+0000", 2024-09-16T20:49:27.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.89e-07 2024-09-16T20:49:27.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:27.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896435+0000", 2024-09-16T20:49:27.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.928e-06 2024-09-16T20:49:27.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:27.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896441+0000", 2024-09-16T20:49:27.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.896e-06 2024-09-16T20:49:27.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.467 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:27.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896445+0000", 2024-09-16T20:49:27.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7550000000000002e-06 2024-09-16T20:49:27.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.467 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:27.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.942468+0000", 2024-09-16T20:49:27.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046022278 2024-09-16T20:49:27.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:27.468 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.942553+0000", 2024-09-16T20:49:27.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5807e-05 2024-09-16T20:49:27.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:27.468 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.942656+0000", 2024-09-16T20:49:27.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010264000000000001 2024-09-16T20:49:27.468 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:27.468 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:27.468 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:27.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.469 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3864 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1236992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:27.469 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896451+0000", 2024-09-16T20:49:27.469 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 12.559819393, 2024-09-16T20:49:27.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046415605999999998, 2024-09-16T20:49:27.469 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:27.469 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:27.469 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:27.469 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:27.469 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:27.469 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:27.470 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3864 2024-09-16T20:49:27.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.470 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:27.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:27.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896451+0000", 2024-09-16T20:49:27.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:27.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:27.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896451+0000", 2024-09-16T20:49:27.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:27.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:27.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896452+0000", 2024-09-16T20:49:27.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1699999999999999e-07 2024-09-16T20:49:27.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:27.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896453+0000", 2024-09-16T20:49:27.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6950000000000001e-06 2024-09-16T20:49:27.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:27.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896459+0000", 2024-09-16T20:49:27.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6749999999999999e-06 2024-09-16T20:49:27.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:27.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896464+0000", 2024-09-16T20:49:27.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1399999999999999e-06 2024-09-16T20:49:27.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:27.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.942695+0000", 2024-09-16T20:49:27.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046230965999999998 2024-09-16T20:49:27.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:27.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.942771+0000", 2024-09-16T20:49:27.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5859999999999995e-05 2024-09-16T20:49:27.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:27.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.942867+0000", 2024-09-16T20:49:27.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5952999999999994e-05 2024-09-16T20:49:27.473 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:27.473 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:27.473 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:27.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.474 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3866 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1241088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:27.474 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896471+0000", 2024-09-16T20:49:27.474 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 12.559800163, 2024-09-16T20:49:27.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046590830999999999, 2024-09-16T20:49:27.474 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:27.474 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:27.474 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:27.474 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:27.474 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:27.474 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:27.474 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3866 2024-09-16T20:49:27.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.475 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:27.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:27.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896471+0000", 2024-09-16T20:49:27.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:27.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:27.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896471+0000", 2024-09-16T20:49:27.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:27.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.476 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:27.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896471+0000", 2024-09-16T20:49:27.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0900000000000002e-07 2024-09-16T20:49:27.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.476 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:27.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896473+0000", 2024-09-16T20:49:27.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8899999999999999e-06 2024-09-16T20:49:27.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.476 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:27.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896479+0000", 2024-09-16T20:49:27.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0179999999999998e-06 2024-09-16T20:49:27.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:27.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896486+0000", 2024-09-16T20:49:27.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7240000000000003e-06 2024-09-16T20:49:27.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:27.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.942924+0000", 2024-09-16T20:49:27.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046438266999999998 2024-09-16T20:49:27.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:27.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943001+0000", 2024-09-16T20:49:27.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6987e-05 2024-09-16T20:49:27.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:27.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943061+0000", 2024-09-16T20:49:27.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0535999999999998e-05 2024-09-16T20:49:27.478 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:27.478 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:27.478 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:27.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.478 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3868 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1245184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:27.479 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896492+0000", 2024-09-16T20:49:27.479 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 12.559778612000001, 2024-09-16T20:49:27.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046738590000000003, 2024-09-16T20:49:27.479 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:27.479 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:27.479 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:27.479 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:27.479 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:27.479 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:27.479 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3868 2024-09-16T20:49:27.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.479 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:27.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:27.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896492+0000", 2024-09-16T20:49:27.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:27.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:27.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896492+0000", 2024-09-16T20:49:27.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:27.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:27.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896492+0000", 2024-09-16T20:49:27.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9499999999999998e-07 2024-09-16T20:49:27.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.481 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:27.481 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896494+0000", 2024-09-16T20:49:27.481 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.973e-06 2024-09-16T20:49:27.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.481 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:27.481 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896501+0000", 2024-09-16T20:49:27.481 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7719999999999997e-06 2024-09-16T20:49:27.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.481 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:27.481 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896506+0000", 2024-09-16T20:49:27.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2089999999999996e-06 2024-09-16T20:49:27.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.482 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:27.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943093+0000", 2024-09-16T20:49:27.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046586799999999998 2024-09-16T20:49:27.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.482 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:27.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943148+0000", 2024-09-16T20:49:27.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4385999999999997e-05 2024-09-16T20:49:27.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:27.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943231+0000", 2024-09-16T20:49:27.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3054999999999997e-05 2024-09-16T20:49:27.483 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:27.483 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:27.483 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:27.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.483 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3870 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1249280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:27.483 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896514+0000", 2024-09-16T20:49:27.483 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 12.559756985, 2024-09-16T20:49:27.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046893815999999998, 2024-09-16T20:49:27.483 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:27.483 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:27.484 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:27.484 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:27.484 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:27.484 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:27.484 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3870 2024-09-16T20:49:27.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.484 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:27.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:27.484 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896514+0000", 2024-09-16T20:49:27.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:27.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:27.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896514+0000", 2024-09-16T20:49:27.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:27.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.488 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:27.488 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896514+0000", 2024-09-16T20:49:27.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9299999999999999e-07 2024-09-16T20:49:27.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.488 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:27.488 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896521+0000", 2024-09-16T20:49:27.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8199999999999999e-06 2024-09-16T20:49:27.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:27.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896529+0000", 2024-09-16T20:49:27.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0120000000000005e-06 2024-09-16T20:49:27.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:27.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896534+0000", 2024-09-16T20:49:27.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5779999999999998e-06 2024-09-16T20:49:27.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:27.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943286+0000", 2024-09-16T20:49:27.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046751514000000001 2024-09-16T20:49:27.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:27.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943343+0000", 2024-09-16T20:49:27.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7216000000000002e-05 2024-09-16T20:49:27.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:27.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943407+0000", 2024-09-16T20:49:27.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4282999999999994e-05 2024-09-16T20:49:27.490 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:27.490 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:27.490 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:27.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.491 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3872 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1253376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:27.491 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896561+0000", 2024-09-16T20:49:27.491 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 12.559709591000001, 2024-09-16T20:49:27.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.04705165, 2024-09-16T20:49:27.491 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:27.491 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:27.491 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:27.491 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:27.491 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:27.491 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:27.491 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3872 2024-09-16T20:49:27.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.492 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:27.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.492 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:27.492 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896561+0000", 2024-09-16T20:49:27.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:27.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.492 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:27.492 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896561+0000", 2024-09-16T20:49:27.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:27.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.492 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:27.492 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896561+0000", 2024-09-16T20:49:27.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4999999999999998e-07 2024-09-16T20:49:27.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.493 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:27.493 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896563+0000", 2024-09-16T20:49:27.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0140000000000001e-06 2024-09-16T20:49:27.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.493 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:27.493 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896569+0000", 2024-09-16T20:49:27.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6450000000000002e-06 2024-09-16T20:49:27.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.494 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:27.494 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896572+0000", 2024-09-16T20:49:27.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.174e-06 2024-09-16T20:49:27.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.494 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:27.494 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943456+0000", 2024-09-16T20:49:27.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046883552000000002 2024-09-16T20:49:27.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.494 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:27.494 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943531+0000", 2024-09-16T20:49:27.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5080000000000006e-05 2024-09-16T20:49:27.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:27.495 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943613+0000", 2024-09-16T20:49:27.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1835000000000005e-05 2024-09-16T20:49:27.495 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:27.495 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:27.495 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:27.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.495 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3874 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1257472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:27.495 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896577+0000", 2024-09-16T20:49:27.496 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 12.559694076, 2024-09-16T20:49:27.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047187944000000002, 2024-09-16T20:49:27.496 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:27.496 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:27.496 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:27.496 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:27.496 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:27.496 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:27.496 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3874 2024-09-16T20:49:27.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.496 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:27.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:27.497 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896577+0000", 2024-09-16T20:49:27.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:27.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:27.497 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896577+0000", 2024-09-16T20:49:27.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:27.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:27.497 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896577+0000", 2024-09-16T20:49:27.498 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1199999999999999e-07 2024-09-16T20:49:27.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.498 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:27.498 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896578+0000", 2024-09-16T20:49:27.498 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.237e-06 2024-09-16T20:49:27.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.498 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:27.498 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896583+0000", 2024-09-16T20:49:27.498 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5859999999999998e-06 2024-09-16T20:49:27.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.499 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.499 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:27.499 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896587+0000", 2024-09-16T20:49:27.499 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3100000000000002e-06 2024-09-16T20:49:27.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.499 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.499 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:27.499 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943661+0000", 2024-09-16T20:49:27.499 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047073639 2024-09-16T20:49:27.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.499 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.499 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:27.500 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943712+0000", 2024-09-16T20:49:27.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1190999999999998e-05 2024-09-16T20:49:27.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.500 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:27.500 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943765+0000", 2024-09-16T20:49:27.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2769000000000002e-05 2024-09-16T20:49:27.500 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:27.500 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:27.500 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:27.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.500 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3876 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1261568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:27.500 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896593+0000", 2024-09-16T20:49:27.501 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 12.559677497999999, 2024-09-16T20:49:27.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047362866000000003, 2024-09-16T20:49:27.501 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:27.501 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:27.501 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:27.501 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:27.501 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:27.501 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:27.501 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3876 2024-09-16T20:49:27.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.501 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:27.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:27.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896593+0000", 2024-09-16T20:49:27.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:27.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.502 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:27.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896593+0000", 2024-09-16T20:49:27.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:27.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.502 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:27.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896594+0000", 2024-09-16T20:49:27.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1600000000000002e-07 2024-09-16T20:49:27.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:27.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896595+0000", 2024-09-16T20:49:27.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.756e-06 2024-09-16T20:49:27.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:27.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896601+0000", 2024-09-16T20:49:27.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9309999999999996e-06 2024-09-16T20:49:27.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:27.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896606+0000", 2024-09-16T20:49:27.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1089999999999997e-06 2024-09-16T20:49:27.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.504 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:27.504 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943790+0000", 2024-09-16T20:49:27.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047183414 2024-09-16T20:49:27.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.504 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:27.504 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943869+0000", 2024-09-16T20:49:27.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9122000000000006e-05 2024-09-16T20:49:27.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:27.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943956+0000", 2024-09-16T20:49:27.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7118000000000002e-05 2024-09-16T20:49:27.505 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:27.505 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:27.505 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:27.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.505 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3879 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1265664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:27.505 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896613+0000", 2024-09-16T20:49:27.505 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 12.559657855999999, 2024-09-16T20:49:27.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047528204999999997, 2024-09-16T20:49:27.506 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:27.506 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:27.506 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:27.506 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:27.506 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:27.506 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:27.506 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3879 2024-09-16T20:49:27.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.506 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:27.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:27.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896613+0000", 2024-09-16T20:49:27.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:27.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.507 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.507 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:27.507 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896613+0000", 2024-09-16T20:49:27.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:27.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.507 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.507 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:27.507 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896613+0000", 2024-09-16T20:49:27.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.84e-07 2024-09-16T20:49:27.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.508 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:27.508 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896615+0000", 2024-09-16T20:49:27.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9970000000000001e-06 2024-09-16T20:49:27.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.508 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:27.508 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896622+0000", 2024-09-16T20:49:27.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3500000000000002e-06 2024-09-16T20:49:27.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.508 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:27.508 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896627+0000", 2024-09-16T20:49:27.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2490000000000001e-06 2024-09-16T20:49:27.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:27.509 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943996+0000", 2024-09-16T20:49:27.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047369175999999999 2024-09-16T20:49:27.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:27.509 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944082+0000", 2024-09-16T20:49:27.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5859000000000006e-05 2024-09-16T20:49:27.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:27.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944141+0000", 2024-09-16T20:49:27.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9190000000000001e-05 2024-09-16T20:49:27.510 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:27.510 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:27.510 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:27.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.510 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3881 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1269760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:27.510 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896634+0000", 2024-09-16T20:49:27.510 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 12.559636942999999, 2024-09-16T20:49:27.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047692834000000003, 2024-09-16T20:49:27.510 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:27.510 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:27.511 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:27.511 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:27.511 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:27.511 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:27.511 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3881 2024-09-16T20:49:27.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.511 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:27.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:27.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896634+0000", 2024-09-16T20:49:27.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:27.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.512 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:27.512 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896634+0000", 2024-09-16T20:49:27.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:27.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.512 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.512 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:27.512 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896634+0000", 2024-09-16T20:49:27.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-16T20:49:27.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.512 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.512 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:27.512 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896636+0000", 2024-09-16T20:49:27.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.216e-06 2024-09-16T20:49:27.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.513 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:27.513 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896640+0000", 2024-09-16T20:49:27.513 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1749999999999998e-06 2024-09-16T20:49:27.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.513 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:27.513 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896643+0000", 2024-09-16T20:49:27.513 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8779999999999998e-06 2024-09-16T20:49:27.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.513 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:27.513 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944178+0000", 2024-09-16T20:49:27.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047534633 2024-09-16T20:49:27.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:27.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944231+0000", 2024-09-16T20:49:27.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3338999999999999e-05 2024-09-16T20:49:27.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:27.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944327+0000", 2024-09-16T20:49:27.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5223000000000006e-05 2024-09-16T20:49:27.514 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:27.514 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:27.514 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:27.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.515 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.515 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3882 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1273856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:27.515 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896647+0000", 2024-09-16T20:49:27.515 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 12.559623781999999, 2024-09-16T20:49:27.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047858886000000003, 2024-09-16T20:49:27.515 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:27.515 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:27.515 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:27.515 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:27.515 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:27.515 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:27.515 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3882 2024-09-16T20:49:27.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.516 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:27.516 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.516 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:27.516 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896647+0000", 2024-09-16T20:49:27.516 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:27.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.516 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.516 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:27.516 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896647+0000", 2024-09-16T20:49:27.516 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:27.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.517 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:27.517 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896647+0000", 2024-09-16T20:49:27.517 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4400000000000001e-07 2024-09-16T20:49:27.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.517 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:27.517 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896648+0000", 2024-09-16T20:49:27.517 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0499999999999999e-06 2024-09-16T20:49:27.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.517 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:27.517 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896651+0000", 2024-09-16T20:49:27.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9809999999999999e-06 2024-09-16T20:49:27.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.518 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:27.518 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896654+0000", 2024-09-16T20:49:27.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4439999999999998e-06 2024-09-16T20:49:27.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.518 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:27.518 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944361+0000", 2024-09-16T20:49:27.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047707408 2024-09-16T20:49:27.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.518 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:27.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944405+0000", 2024-09-16T20:49:27.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4107000000000003e-05 2024-09-16T20:49:27.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.519 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:27.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944506+0000", 2024-09-16T20:49:27.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000100652 2024-09-16T20:49:27.519 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:27.519 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:27.519 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:27.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.519 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3883 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1277952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:27.520 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896656+0000", 2024-09-16T20:49:27.520 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 12.559614271999999, 2024-09-16T20:49:27.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047975265000000003, 2024-09-16T20:49:27.520 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:27.520 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:27.520 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:27.520 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:27.520 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:27.520 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:27.520 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3883 2024-09-16T20:49:27.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.520 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:27.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.521 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:27.521 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896656+0000", 2024-09-16T20:49:27.521 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:27.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.521 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.521 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:27.521 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896656+0000", 2024-09-16T20:49:27.521 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:27.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.521 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.521 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:27.521 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896657+0000", 2024-09-16T20:49:27.521 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7000000000000001e-07 2024-09-16T20:49:27.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.522 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:27.522 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896658+0000", 2024-09-16T20:49:27.522 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5850000000000001e-06 2024-09-16T20:49:27.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.522 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:27.522 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896664+0000", 2024-09-16T20:49:27.522 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.609e-06 2024-09-16T20:49:27.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.522 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:27.523 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896668+0000", 2024-09-16T20:49:27.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.121e-06 2024-09-16T20:49:27.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:27.523 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944542+0000", 2024-09-16T20:49:27.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047873838000000002 2024-09-16T20:49:27.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:27.523 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944579+0000", 2024-09-16T20:49:27.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6943000000000002e-05 2024-09-16T20:49:27.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.524 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:27.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944632+0000", 2024-09-16T20:49:27.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2899000000000003e-05 2024-09-16T20:49:27.524 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:27.524 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:27.524 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:27.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.524 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3886 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1282048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:27.524 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896674+0000", 2024-09-16T20:49:27.524 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 12.559596302999999, 2024-09-16T20:49:27.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048152421000000001, 2024-09-16T20:49:27.525 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:27.525 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:27.525 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:27.525 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:27.525 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:27.525 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:27.525 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3886 2024-09-16T20:49:27.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.525 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:27.525 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.525 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:27.525 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896674+0000", 2024-09-16T20:49:27.525 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:27.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.526 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.526 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:27.526 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896674+0000", 2024-09-16T20:49:27.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:27.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.526 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.526 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:27.526 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896675+0000", 2024-09-16T20:49:27.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6100000000000002e-07 2024-09-16T20:49:27.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.526 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.527 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:27.527 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896677+0000", 2024-09-16T20:49:27.527 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1270000000000001e-06 2024-09-16T20:49:27.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.527 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:27.527 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896683+0000", 2024-09-16T20:49:27.527 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1970000000000001e-06 2024-09-16T20:49:27.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.527 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:27.527 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896688+0000", 2024-09-16T20:49:27.527 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.605e-06 2024-09-16T20:49:27.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:27.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944676+0000", 2024-09-16T20:49:27.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047988391999999998 2024-09-16T20:49:27.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:27.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944763+0000", 2024-09-16T20:49:27.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7363999999999999e-05 2024-09-16T20:49:27.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:27.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944827+0000", 2024-09-16T20:49:27.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3374999999999995e-05 2024-09-16T20:49:27.529 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:27.529 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:27.529 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:27.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.529 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3888 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1286144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:27.529 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896693+0000", 2024-09-16T20:49:27.529 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 12.559577179, 2024-09-16T20:49:27.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048336443999999999, 2024-09-16T20:49:27.529 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:27.529 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:27.529 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:27.530 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:27.530 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:27.530 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:27.530 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3888 2024-09-16T20:49:27.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.530 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:27.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.530 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:27.530 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896693+0000", 2024-09-16T20:49:27.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:27.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.530 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:27.530 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896693+0000", 2024-09-16T20:49:27.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:27.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:27.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896694+0000", 2024-09-16T20:49:27.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3200000000000001e-07 2024-09-16T20:49:27.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:27.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896696+0000", 2024-09-16T20:49:27.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8989999999999999e-06 2024-09-16T20:49:27.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:27.532 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896702+0000", 2024-09-16T20:49:27.532 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8819999999999998e-06 2024-09-16T20:49:27.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.532 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:27.532 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896707+0000", 2024-09-16T20:49:27.532 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1529999999999996e-06 2024-09-16T20:49:27.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.532 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:27.532 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944863+0000", 2024-09-16T20:49:27.532 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048156658999999997 2024-09-16T20:49:27.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.533 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:27.533 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944944+0000", 2024-09-16T20:49:27.533 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0090000000000001e-05 2024-09-16T20:49:27.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.533 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.533 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:27.533 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945030+0000", 2024-09-16T20:49:27.533 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6428999999999995e-05 2024-09-16T20:49:27.533 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:27.533 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:27.533 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:27.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.533 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.534 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3890 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1290240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:27.534 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896713+0000", 2024-09-16T20:49:27.534 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 12.559557309000001, 2024-09-16T20:49:27.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048538382999999997, 2024-09-16T20:49:27.534 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:27.534 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:27.534 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:27.534 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:27.534 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:27.534 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:27.534 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3890 2024-09-16T20:49:27.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.534 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:27.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.534 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:27.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896713+0000", 2024-09-16T20:49:27.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:27.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:27.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896713+0000", 2024-09-16T20:49:27.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:27.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:27.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896714+0000", 2024-09-16T20:49:27.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2000000000000001e-07 2024-09-16T20:49:27.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:27.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896716+0000", 2024-09-16T20:49:27.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0109999999999999e-06 2024-09-16T20:49:27.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.536 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:27.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896720+0000", 2024-09-16T20:49:27.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9049999999999999e-06 2024-09-16T20:49:27.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.536 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:27.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896722+0000", 2024-09-16T20:49:27.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.627e-06 2024-09-16T20:49:27.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:27.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945075+0000", 2024-09-16T20:49:27.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048353149999999998 2024-09-16T20:49:27.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:27.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945157+0000", 2024-09-16T20:49:27.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1972999999999996e-05 2024-09-16T20:49:27.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:27.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945252+0000", 2024-09-16T20:49:27.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4396999999999998e-05 2024-09-16T20:49:27.538 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:27.538 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:27.538 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:27.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.538 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3893 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1294336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:27.538 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896726+0000", 2024-09-16T20:49:27.538 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 12.559544452000001, 2024-09-16T20:49:27.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048658686999999999, 2024-09-16T20:49:27.538 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:27.538 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:27.538 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:27.539 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:27.539 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:27.539 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:27.539 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3893 2024-09-16T20:49:27.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.539 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:27.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.539 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:27.539 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896726+0000", 2024-09-16T20:49:27.539 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:27.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.539 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:27.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896726+0000", 2024-09-16T20:49:27.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:27.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:27.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896727+0000", 2024-09-16T20:49:27.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7899999999999999e-07 2024-09-16T20:49:27.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:27.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896728+0000", 2024-09-16T20:49:27.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.68e-06 2024-09-16T20:49:27.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.541 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:27.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896734+0000", 2024-09-16T20:49:27.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9190000000000004e-06 2024-09-16T20:49:27.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.541 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:27.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896739+0000", 2024-09-16T20:49:27.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5549999999999996e-06 2024-09-16T20:49:27.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.541 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:27.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945271+0000", 2024-09-16T20:49:27.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048531795000000003 2024-09-16T20:49:27.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:27.542 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945330+0000", 2024-09-16T20:49:27.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9386999999999999e-05 2024-09-16T20:49:27.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:27.542 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945385+0000", 2024-09-16T20:49:27.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4871999999999999e-05 2024-09-16T20:49:27.542 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:27.542 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:27.542 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:27.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.543 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3895 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1298432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:27.543 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896754+0000", 2024-09-16T20:49:27.543 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 12.559516378, 2024-09-16T20:49:27.543 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048754440000000003, 2024-09-16T20:49:27.543 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:27.543 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:27.543 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:27.543 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:27.543 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:27.543 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:27.543 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3895 2024-09-16T20:49:27.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.544 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:27.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.544 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:27.544 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896754+0000", 2024-09-16T20:49:27.544 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:27.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.544 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:27.544 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896754+0000", 2024-09-16T20:49:27.544 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:27.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.545 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:27.545 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896755+0000", 2024-09-16T20:49:27.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5200000000000002e-07 2024-09-16T20:49:27.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.545 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:27.545 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896762+0000", 2024-09-16T20:49:27.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8430000000000004e-06 2024-09-16T20:49:27.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.545 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:27.545 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896782+0000", 2024-09-16T20:49:27.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0693e-05 2024-09-16T20:49:27.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:27.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896793+0000", 2024-09-16T20:49:27.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0294e-05 2024-09-16T20:49:27.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:27.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945418+0000", 2024-09-16T20:49:27.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048625738000000002 2024-09-16T20:49:27.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:27.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945457+0000", 2024-09-16T20:49:27.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9103999999999997e-05 2024-09-16T20:49:27.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.547 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:27.547 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945509+0000", 2024-09-16T20:49:27.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1316000000000001e-05 2024-09-16T20:49:27.547 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:27.547 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:27.547 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:27.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.547 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3896 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1302528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:27.547 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896806+0000", 2024-09-16T20:49:27.548 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 12.559464166, 2024-09-16T20:49:27.548 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048831159999999998, 2024-09-16T20:49:27.548 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:27.548 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:27.548 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:27.548 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:27.548 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:27.548 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:27.548 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3896 2024-09-16T20:49:27.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.548 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:27.548 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.548 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:27.549 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896806+0000", 2024-09-16T20:49:27.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:27.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.549 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:27.549 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896806+0000", 2024-09-16T20:49:27.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:27.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.549 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:27.549 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896807+0000", 2024-09-16T20:49:27.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3e-07 2024-09-16T20:49:27.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.550 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:27.550 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896811+0000", 2024-09-16T20:49:27.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8310000000000003e-06 2024-09-16T20:49:27.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.550 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:27.550 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896819+0000", 2024-09-16T20:49:27.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2470000000000008e-06 2024-09-16T20:49:27.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.550 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:27.550 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896823+0000", 2024-09-16T20:49:27.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7919999999999999e-06 2024-09-16T20:49:27.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.551 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:27.551 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945535+0000", 2024-09-16T20:49:27.551 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048712137000000003 2024-09-16T20:49:27.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.551 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:27.551 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945571+0000", 2024-09-16T20:49:27.551 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6261999999999999e-05 2024-09-16T20:49:27.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.551 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:27.551 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945638+0000", 2024-09-16T20:49:27.551 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6161000000000006e-05 2024-09-16T20:49:27.551 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:27.551 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:27.552 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:27.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.552 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.552 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3899 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1306624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:27.552 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896829+0000", 2024-09-16T20:49:27.552 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 12.559441965, 2024-09-16T20:49:27.552 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049014110999999999, 2024-09-16T20:49:27.552 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:27.552 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:27.552 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:27.552 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:27.552 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:27.552 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:27.552 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3899 2024-09-16T20:49:27.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.553 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:27.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.553 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:27.553 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896829+0000", 2024-09-16T20:49:27.553 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:27.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.553 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:27.553 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896829+0000", 2024-09-16T20:49:27.553 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:27.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.553 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:27.554 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896829+0000", 2024-09-16T20:49:27.554 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8999999999999998e-07 2024-09-16T20:49:27.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.554 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.554 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:27.554 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896831+0000", 2024-09-16T20:49:27.554 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.77e-06 2024-09-16T20:49:27.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.554 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.554 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:27.554 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896834+0000", 2024-09-16T20:49:27.554 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4800000000000001e-06 2024-09-16T20:49:27.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.554 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:27.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896837+0000", 2024-09-16T20:49:27.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8559999999999998e-06 2024-09-16T20:49:27.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:27.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945663+0000", 2024-09-16T20:49:27.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048825529999999999 2024-09-16T20:49:27.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:27.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945746+0000", 2024-09-16T20:49:27.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3621000000000005e-05 2024-09-16T20:49:27.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:27.556 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:27.556 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:27.556 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945843+0000", 2024-09-16T20:49:27.556 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6564000000000006e-05 2024-09-16T20:49:27.556 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:27.556 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:27.556 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:27.556 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:27.556 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:27.556 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:27.558 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:27.556+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:27.558 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:49:27.659 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:27.657+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:27.678 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:27.679 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:49:27.679 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.679 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5138.0:9358 3.c 3.c25699cc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T20:49:27.679 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:49:27.679 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.82794994099999997, 2024-09-16T20:49:27.679 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.82796935599999999, 2024-09-16T20:49:27.679 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:27.679 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:27.679 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "waiting for peered", 2024-09-16T20:49:27.679 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:27.679 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5138", 2024-09-16T20:49:27.679 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:49:27.679 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 9358 2024-09-16T20:49:27.680 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.680 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:27.680 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.680 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:27.680 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:49:27.680 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:27.680 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.680 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.680 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:27.680 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:49:27.680 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:27.680 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.680 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.681 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:27.681 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850314+0000", 2024-09-16T20:49:27.681 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4099999999999999e-07 2024-09-16T20:49:27.681 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.681 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.681 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:27.681 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850316+0000", 2024-09-16T20:49:27.681 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2469999999999998e-06 2024-09-16T20:49:27.681 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.681 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.681 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:27.681 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850322+0000", 2024-09-16T20:49:27.681 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3470000000000004e-06 2024-09-16T20:49:27.682 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.682 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.682 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:27.682 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850327+0000", 2024-09-16T20:49:27.682 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9450000000000001e-06 2024-09-16T20:49:27.682 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.682 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.682 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:27.682 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850376+0000", 2024-09-16T20:49:27.682 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8257000000000002e-05 2024-09-16T20:49:27.682 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.682 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.682 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:49:27.683 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850385+0000", 2024-09-16T20:49:27.683 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9800000000000004e-06 2024-09-16T20:49:27.683 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:27.683 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:27.683 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:27.683 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.683 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.683 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5138.0:9382 3.c 3.c25699cc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T20:49:27.683 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:49:27.683 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.81960292499999998, 2024-09-16T20:49:27.683 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.81965997599999996, 2024-09-16T20:49:27.683 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:27.683 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:27.683 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "waiting for peered", 2024-09-16T20:49:27.684 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:27.684 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5138", 2024-09-16T20:49:27.684 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:49:27.684 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 9382 2024-09-16T20:49:27.684 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.684 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:27.684 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.684 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:27.684 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:49:27.684 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:27.684 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.684 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.684 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:27.684 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:49:27.685 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:27.685 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.685 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.685 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:27.685 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858661+0000", 2024-09-16T20:49:27.685 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0200000000000002e-07 2024-09-16T20:49:27.685 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.685 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.685 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:27.685 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858664+0000", 2024-09-16T20:49:27.685 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.038e-06 2024-09-16T20:49:27.685 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.685 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.685 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:27.686 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858671+0000", 2024-09-16T20:49:27.686 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0110000000000003e-06 2024-09-16T20:49:27.686 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.686 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.686 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:27.686 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858677+0000", 2024-09-16T20:49:27.686 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0959999999999997e-06 2024-09-16T20:49:27.686 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.686 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.686 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:27.686 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858732+0000", 2024-09-16T20:49:27.686 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4543000000000003e-05 2024-09-16T20:49:27.686 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.686 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.687 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:49:27.687 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858739+0000", 2024-09-16T20:49:27.687 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5429999999999996e-06 2024-09-16T20:49:27.687 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:27.687 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:27.687 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:27.687 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:27.687 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-16T20:49:27.687 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 2 2024-09-16T20:49:27.687 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:27.688 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:49:27.760 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:27.759+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:27.796 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:27.796 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:49:27.796 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:49:27.796 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:49:27.796 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:27.805 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:49:27.859 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T20:49:27.889 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:27.888+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:27.956 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:27.957 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:49:27.957 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:49:27.957 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:49:27.957 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.957 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2179 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 948224~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:27.957 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335723+0000", 2024-09-16T20:49:27.957 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 65.618927467999995, 2024-09-16T20:49:27.957 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048007181000000003, 2024-09-16T20:49:27.957 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:27.957 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:27.958 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:27.958 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:27.958 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:27.958 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:27.958 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2179 2024-09-16T20:49:27.958 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.958 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:27.958 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.958 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:27.958 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335723+0000", 2024-09-16T20:49:27.958 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:27.958 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.958 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.958 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:27.959 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335723+0000", 2024-09-16T20:49:27.959 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:27.959 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.959 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.959 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:27.959 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335724+0000", 2024-09-16T20:49:27.959 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.58e-07 2024-09-16T20:49:27.959 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.959 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.959 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:27.959 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335726+0000", 2024-09-16T20:49:27.959 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9120000000000001e-06 2024-09-16T20:49:27.959 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.960 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.960 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:27.960 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335732+0000", 2024-09-16T20:49:27.960 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6080000000000001e-06 2024-09-16T20:49:27.960 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.960 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.960 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:27.960 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335737+0000", 2024-09-16T20:49:27.960 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.22e-06 2024-09-16T20:49:27.960 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.960 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.960 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:27.960 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383556+0000", 2024-09-16T20:49:27.961 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047819002999999999 2024-09-16T20:49:27.961 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.961 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.961 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:27.961 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383635+0000", 2024-09-16T20:49:27.961 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9060999999999994e-05 2024-09-16T20:49:27.961 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.961 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.961 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:27.961 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383730+0000", 2024-09-16T20:49:27.961 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4919000000000004e-05 2024-09-16T20:49:27.961 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:27.962 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:27.962 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:27.962 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.962 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.962 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2180 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 956416~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:27.962 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335744+0000", 2024-09-16T20:49:27.962 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 65.618907019999995, 2024-09-16T20:49:27.962 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048255147999999998, 2024-09-16T20:49:27.962 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:27.962 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:27.962 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:27.962 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:27.962 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:27.963 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:27.963 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2180 2024-09-16T20:49:27.963 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.963 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:27.963 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.963 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:27.963 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335744+0000", 2024-09-16T20:49:27.963 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:27.963 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.963 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.963 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:27.964 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335744+0000", 2024-09-16T20:49:27.964 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:27.964 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.964 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.964 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:27.964 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335744+0000", 2024-09-16T20:49:27.964 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9000000000000002e-07 2024-09-16T20:49:27.964 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.964 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.964 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:27.964 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335746+0000", 2024-09-16T20:49:27.964 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7570000000000001e-06 2024-09-16T20:49:27.964 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.965 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.965 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:27.965 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335752+0000", 2024-09-16T20:49:27.965 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0079999999999999e-06 2024-09-16T20:49:27.965 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.965 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.965 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:27.965 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335757+0000", 2024-09-16T20:49:27.965 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1649999999999997e-06 2024-09-16T20:49:27.965 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.965 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.965 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:27.965 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383803+0000", 2024-09-16T20:49:27.966 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048046091999999999 2024-09-16T20:49:27.966 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.966 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.966 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:27.966 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383889+0000", 2024-09-16T20:49:27.966 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6048e-05 2024-09-16T20:49:27.966 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.966 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.966 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:27.966 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.383999+0000", 2024-09-16T20:49:27.966 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000109688 2024-09-16T20:49:27.967 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:27.967 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:27.967 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:27.967 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.967 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.967 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2182 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 964608~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:27.967 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335773+0000", 2024-09-16T20:49:27.967 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 65.618877213000005, 2024-09-16T20:49:27.967 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048467116999999997, 2024-09-16T20:49:27.967 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:27.967 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:27.967 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:27.967 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:27.967 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:27.968 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:27.968 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2182 2024-09-16T20:49:27.968 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.968 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:27.968 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.968 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:27.968 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335773+0000", 2024-09-16T20:49:27.968 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:27.968 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.968 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.968 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:27.968 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335773+0000", 2024-09-16T20:49:27.968 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:27.969 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.969 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.969 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:27.969 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335774+0000", 2024-09-16T20:49:27.969 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3799999999999997e-07 2024-09-16T20:49:27.969 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.969 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.969 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:27.969 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335776+0000", 2024-09-16T20:49:27.969 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.227e-06 2024-09-16T20:49:27.969 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.969 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.969 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:27.970 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335790+0000", 2024-09-16T20:49:27.970 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4119e-05 2024-09-16T20:49:27.970 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.970 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.970 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:27.970 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335796+0000", 2024-09-16T20:49:27.970 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.694e-06 2024-09-16T20:49:27.970 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.970 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.970 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:27.970 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384058+0000", 2024-09-16T20:49:27.970 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048261983000000001 2024-09-16T20:49:27.970 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.970 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.971 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:27.971 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384148+0000", 2024-09-16T20:49:27.971 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9501999999999998e-05 2024-09-16T20:49:27.971 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.971 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.971 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:27.971 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384241+0000", 2024-09-16T20:49:27.971 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.3053999999999994e-05 2024-09-16T20:49:27.971 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:27.971 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:27.971 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:27.971 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.971 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.971 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2185 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 972800~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:27.972 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335804+0000", 2024-09-16T20:49:27.972 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 65.618846563000005, 2024-09-16T20:49:27.972 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048570818000000002, 2024-09-16T20:49:27.972 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:27.972 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:27.972 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:27.972 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:27.972 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:27.972 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:27.972 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2185 2024-09-16T20:49:27.972 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.972 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:27.973 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.973 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:27.973 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335804+0000", 2024-09-16T20:49:27.973 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:27.973 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.973 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.973 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:27.973 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335804+0000", 2024-09-16T20:49:27.973 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:27.973 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.973 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.973 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:27.973 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335805+0000", 2024-09-16T20:49:27.974 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8299999999999997e-07 2024-09-16T20:49:27.974 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.974 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.974 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:27.974 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335813+0000", 2024-09-16T20:49:27.974 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6309999999999992e-06 2024-09-16T20:49:27.974 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.974 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.974 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:27.974 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335821+0000", 2024-09-16T20:49:27.974 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0169999999999996e-06 2024-09-16T20:49:27.975 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.975 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.975 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:27.975 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335827+0000", 2024-09-16T20:49:27.975 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0100000000000001e-06 2024-09-16T20:49:27.975 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.975 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.975 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:27.975 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384273+0000", 2024-09-16T20:49:27.975 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048445333 2024-09-16T20:49:27.975 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.975 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.975 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:27.975 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384315+0000", 2024-09-16T20:49:27.976 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2472999999999998e-05 2024-09-16T20:49:27.976 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.976 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.976 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:27.976 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384375+0000", 2024-09-16T20:49:27.976 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9871000000000003e-05 2024-09-16T20:49:27.976 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:27.976 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:27.976 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:27.976 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.976 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.976 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2187 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 980992~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:27.976 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335835+0000", 2024-09-16T20:49:27.977 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 65.618815486000003, 2024-09-16T20:49:27.977 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048684927000000003, 2024-09-16T20:49:27.977 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:27.977 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:27.977 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:27.977 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:27.977 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:27.977 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:27.977 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2187 2024-09-16T20:49:27.977 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.977 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:27.977 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.977 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:27.978 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335835+0000", 2024-09-16T20:49:27.978 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:27.978 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.978 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.978 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:27.978 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335835+0000", 2024-09-16T20:49:27.978 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:27.980 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.980 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.980 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:27.980 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335836+0000", 2024-09-16T20:49:27.980 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3599999999999999e-07 2024-09-16T20:49:27.980 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.980 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.980 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:27.980 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335838+0000", 2024-09-16T20:49:27.980 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.996e-06 2024-09-16T20:49:27.980 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.981 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.981 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:27.981 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335844+0000", 2024-09-16T20:49:27.981 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7889999999999997e-06 2024-09-16T20:49:27.981 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.981 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.981 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:27.981 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335849+0000", 2024-09-16T20:49:27.981 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7400000000000004e-06 2024-09-16T20:49:27.981 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.981 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.981 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:27.981 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384411+0000", 2024-09-16T20:49:27.982 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048562069999999999 2024-09-16T20:49:27.982 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.982 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.982 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:27.982 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384460+0000", 2024-09-16T20:49:27.982 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9125999999999997e-05 2024-09-16T20:49:27.982 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.982 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.982 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:27.982 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384520+0000", 2024-09-16T20:49:27.982 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9769999999999999e-05 2024-09-16T20:49:27.982 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:27.982 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:27.982 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:27.983 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.983 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.983 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2189 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 989184~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:27.983 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335856+0000", 2024-09-16T20:49:27.983 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 65.618794241000003, 2024-09-16T20:49:27.983 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048856525999999997, 2024-09-16T20:49:27.983 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:27.983 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:27.983 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:27.983 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:27.984 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:27.984 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:27.984 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2189 2024-09-16T20:49:27.984 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.984 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:27.984 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.984 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:27.984 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335856+0000", 2024-09-16T20:49:27.984 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:27.984 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.984 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.984 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:27.984 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335856+0000", 2024-09-16T20:49:27.984 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:27.985 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.985 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.985 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:27.985 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335857+0000", 2024-09-16T20:49:27.985 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6800000000000001e-07 2024-09-16T20:49:27.985 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.985 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.985 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:27.985 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335859+0000", 2024-09-16T20:49:27.985 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.013e-06 2024-09-16T20:49:27.985 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.985 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.985 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:27.986 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335866+0000", 2024-09-16T20:49:27.986 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8750000000000002e-06 2024-09-16T20:49:27.986 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.986 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.986 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:27.986 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335871+0000", 2024-09-16T20:49:27.986 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1719999999999998e-06 2024-09-16T20:49:27.986 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.986 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.986 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:27.986 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384550+0000", 2024-09-16T20:49:27.986 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048679476999999999 2024-09-16T20:49:27.986 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.986 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.987 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:27.987 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384611+0000", 2024-09-16T20:49:27.987 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0897000000000003e-05 2024-09-16T20:49:27.987 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.987 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.987 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:27.987 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384713+0000", 2024-09-16T20:49:27.987 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101724 2024-09-16T20:49:27.987 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:27.987 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:27.987 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:27.988 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.988 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.988 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2192 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 997376~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:27.988 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335878+0000", 2024-09-16T20:49:27.988 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 65.618772555999996, 2024-09-16T20:49:27.988 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049028769999999999, 2024-09-16T20:49:27.988 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:27.988 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:27.988 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:27.988 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:27.988 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:27.988 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:27.988 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2192 2024-09-16T20:49:27.989 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.989 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:27.989 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.989 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:27.989 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335878+0000", 2024-09-16T20:49:27.989 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:27.989 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.989 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.989 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:27.989 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335878+0000", 2024-09-16T20:49:27.989 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:27.989 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.989 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.989 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:27.990 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335879+0000", 2024-09-16T20:49:27.990 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6899999999999998e-07 2024-09-16T20:49:27.990 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.990 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.990 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:27.990 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335881+0000", 2024-09-16T20:49:27.990 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.996e-06 2024-09-16T20:49:27.990 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.990 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.990 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:27.990 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335887+0000", 2024-09-16T20:49:27.990 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6370000000000002e-06 2024-09-16T20:49:27.990 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.991 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.991 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:27.991 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335892+0000", 2024-09-16T20:49:27.991 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2120000000000002e-06 2024-09-16T20:49:27.991 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.991 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.991 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:27.991 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384747+0000", 2024-09-16T20:49:27.991 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048854923000000001 2024-09-16T20:49:27.991 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.991 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.991 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:27.991 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384830+0000", 2024-09-16T20:49:27.991 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2359000000000002e-05 2024-09-16T20:49:27.992 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.992 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.992 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:27.992 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384907+0000", 2024-09-16T20:49:27.992 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7174000000000003e-05 2024-09-16T20:49:27.992 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:27.992 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:27.992 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:27.992 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.992 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.992 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2194 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1005568~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:27.992 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335899+0000", 2024-09-16T20:49:27.992 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 65.618751634999995, 2024-09-16T20:49:27.993 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049225393999999999, 2024-09-16T20:49:27.993 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:27.993 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:27.993 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:27.993 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:27.993 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:27.993 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:27.993 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2194 2024-09-16T20:49:27.993 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.993 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:27.993 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.993 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:27.993 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335899+0000", 2024-09-16T20:49:27.993 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:27.994 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.994 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.994 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:27.994 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335899+0000", 2024-09-16T20:49:27.994 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:27.994 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.994 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.994 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:27.994 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335899+0000", 2024-09-16T20:49:27.994 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8200000000000001e-07 2024-09-16T20:49:27.994 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.994 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.994 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:27.994 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335901+0000", 2024-09-16T20:49:27.995 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9e-06 2024-09-16T20:49:27.995 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.995 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.995 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:27.995 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335908+0000", 2024-09-16T20:49:27.995 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3559999999999998e-06 2024-09-16T20:49:27.995 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.995 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.995 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:27.995 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335914+0000", 2024-09-16T20:49:27.995 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4729999999999997e-06 2024-09-16T20:49:27.995 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.995 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.995 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:27.996 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.384979+0000", 2024-09-16T20:49:27.996 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049064545000000001 2024-09-16T20:49:27.996 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.996 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.996 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:27.996 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385060+0000", 2024-09-16T20:49:27.996 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0804999999999996e-05 2024-09-16T20:49:27.996 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.996 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.997 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:27.997 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385124+0000", 2024-09-16T20:49:27.997 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4932999999999996e-05 2024-09-16T20:49:27.997 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:27.997 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:27.997 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:27.997 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.997 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.998 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2197 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1013760~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:27.998 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335921+0000", 2024-09-16T20:49:27.998 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 65.618729353999996, 2024-09-16T20:49:27.998 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049342184999999997, 2024-09-16T20:49:27.998 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:27.998 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:27.998 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:27.998 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:27.998 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:27.998 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:27.998 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2197 2024-09-16T20:49:27.999 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.999 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:27.999 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.999 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:27.999 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335921+0000", 2024-09-16T20:49:27.999 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:27.999 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.999 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:27.999 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:27.999 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335921+0000", 2024-09-16T20:49:27.999 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:27.999 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:27.999 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.000 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:28.000 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335922+0000", 2024-09-16T20:49:28.000 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.32e-07 2024-09-16T20:49:28.000 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.000 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.000 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:28.000 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335924+0000", 2024-09-16T20:49:28.000 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.995e-06 2024-09-16T20:49:28.000 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.000 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.000 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:28.000 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335930+0000", 2024-09-16T20:49:28.000 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5139999999999998e-06 2024-09-16T20:49:28.000 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.001 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.001 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:28.001 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335936+0000", 2024-09-16T20:49:28.001 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6160000000000001e-06 2024-09-16T20:49:28.001 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.001 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.001 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:28.001 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385142+0000", 2024-09-16T20:49:28.001 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049206393000000001 2024-09-16T20:49:28.001 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.001 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.001 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:28.001 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385184+0000", 2024-09-16T20:49:28.001 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1928999999999998e-05 2024-09-16T20:49:28.002 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.002 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.002 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:28.002 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385264+0000", 2024-09-16T20:49:28.002 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9306000000000003e-05 2024-09-16T20:49:28.002 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.002 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:28.002 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.002 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.002 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.002 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2198 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1021952~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:28.002 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335942+0000", 2024-09-16T20:49:28.002 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 65.618708294000001, 2024-09-16T20:49:28.003 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049562483999999997, 2024-09-16T20:49:28.003 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:28.003 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:28.003 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:28.003 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:28.003 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:28.003 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:28.003 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2198 2024-09-16T20:49:28.003 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.003 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:28.003 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.003 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:28.003 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335942+0000", 2024-09-16T20:49:28.003 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.004 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.004 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.004 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:28.004 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335942+0000", 2024-09-16T20:49:28.004 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.004 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.004 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.004 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:28.004 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335943+0000", 2024-09-16T20:49:28.004 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7099999999999997e-07 2024-09-16T20:49:28.004 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.004 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.004 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:28.005 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335945+0000", 2024-09-16T20:49:28.005 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3499999999999999e-06 2024-09-16T20:49:28.005 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.005 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.005 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:28.005 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335952+0000", 2024-09-16T20:49:28.005 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4309999999999999e-06 2024-09-16T20:49:28.005 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.005 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.005 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:28.005 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335957+0000", 2024-09-16T20:49:28.005 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0869999999999997e-06 2024-09-16T20:49:28.005 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.005 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.006 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:28.006 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385312+0000", 2024-09-16T20:49:28.006 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049355121000000002 2024-09-16T20:49:28.006 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.006 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.006 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:28.006 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385405+0000", 2024-09-16T20:49:28.006 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.3344000000000006e-05 2024-09-16T20:49:28.006 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.006 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.006 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:28.006 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385505+0000", 2024-09-16T20:49:28.006 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.9779999999999997e-05 2024-09-16T20:49:28.007 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.007 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:28.007 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.007 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.007 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.007 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2201 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1030144~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:28.007 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.335976+0000", 2024-09-16T20:49:28.007 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 65.618674851999998, 2024-09-16T20:49:28.007 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049740077000000001, 2024-09-16T20:49:28.007 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:28.007 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:28.007 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:28.007 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:28.007 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:28.008 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:28.008 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2201 2024-09-16T20:49:28.008 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.008 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:28.008 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.008 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:28.008 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335976+0000", 2024-09-16T20:49:28.008 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.008 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.008 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.008 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:28.008 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335976+0000", 2024-09-16T20:49:28.008 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.009 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.009 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.009 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:28.009 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335976+0000", 2024-09-16T20:49:28.009 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8599999999999998e-07 2024-09-16T20:49:28.009 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.009 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.009 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:28.009 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335978+0000", 2024-09-16T20:49:28.009 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1500000000000002e-06 2024-09-16T20:49:28.010 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.010 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.010 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:28.010 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335986+0000", 2024-09-16T20:49:28.010 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3989999999999998e-06 2024-09-16T20:49:28.010 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.010 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.010 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:28.010 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.335992+0000", 2024-09-16T20:49:28.010 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.682e-06 2024-09-16T20:49:28.010 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.011 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.011 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:28.011 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385537+0000", 2024-09-16T20:49:28.011 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049545828 2024-09-16T20:49:28.011 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.011 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.011 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:28.011 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385619+0000", 2024-09-16T20:49:28.011 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1720999999999999e-05 2024-09-16T20:49:28.011 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.011 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.011 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:28.012 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385716+0000", 2024-09-16T20:49:28.012 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.6811000000000005e-05 2024-09-16T20:49:28.012 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.012 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:28.012 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.012 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.012 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.012 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2203 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1038336~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:28.012 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.336000+0000", 2024-09-16T20:49:28.012 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 65.618651065999998, 2024-09-16T20:49:28.013 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049949888999999997, 2024-09-16T20:49:28.013 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:28.013 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:28.013 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:28.013 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:28.013 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:28.013 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:28.013 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2203 2024-09-16T20:49:28.013 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.013 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:28.013 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.013 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:28.013 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336000+0000", 2024-09-16T20:49:28.014 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.014 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.014 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.014 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:28.014 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336000+0000", 2024-09-16T20:49:28.014 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.014 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.014 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.014 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:28.014 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336000+0000", 2024-09-16T20:49:28.014 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.58e-07 2024-09-16T20:49:28.015 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.015 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.015 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:28.015 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336002+0000", 2024-09-16T20:49:28.015 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0040000000000002e-06 2024-09-16T20:49:28.015 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.015 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.015 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:28.015 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336009+0000", 2024-09-16T20:49:28.015 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9890000000000003e-06 2024-09-16T20:49:28.015 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.015 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.015 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:28.015 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336014+0000", 2024-09-16T20:49:28.016 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4e-06 2024-09-16T20:49:28.016 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.016 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.016 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:28.016 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385759+0000", 2024-09-16T20:49:28.016 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049744151 2024-09-16T20:49:28.016 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.016 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.016 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:28.016 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385847+0000", 2024-09-16T20:49:28.016 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8189000000000006e-05 2024-09-16T20:49:28.016 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.017 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.017 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:28.017 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385950+0000", 2024-09-16T20:49:28.017 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102698 2024-09-16T20:49:28.017 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.017 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:28.017 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.017 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.017 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.017 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2206 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1046528~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:28.018 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:49:28.018 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 65.618628904999994, 2024-09-16T20:49:28.018 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050147714000000003, 2024-09-16T20:49:28.018 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:28.018 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:28.018 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:28.018 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:28.018 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:28.018 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:28.018 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2206 2024-09-16T20:49:28.018 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.018 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:28.018 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.019 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:28.019 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:49:28.019 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.019 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.019 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.019 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:28.019 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:49:28.019 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.019 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.019 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.019 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:28.019 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:49:28.019 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8599999999999999e-07 2024-09-16T20:49:28.020 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.020 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.020 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:28.020 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336024+0000", 2024-09-16T20:49:28.020 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.934e-06 2024-09-16T20:49:28.020 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.020 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.020 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:28.020 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336037+0000", 2024-09-16T20:49:28.020 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2496e-05 2024-09-16T20:49:28.020 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.020 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.020 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:28.021 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336042+0000", 2024-09-16T20:49:28.021 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4809999999999997e-06 2024-09-16T20:49:28.021 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.021 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.021 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:28.021 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385980+0000", 2024-09-16T20:49:28.021 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049938376999999999 2024-09-16T20:49:28.021 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.021 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.021 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:28.022 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.386069+0000", 2024-09-16T20:49:28.022 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8368999999999994e-05 2024-09-16T20:49:28.022 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.022 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.022 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:28.022 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.386170+0000", 2024-09-16T20:49:28.022 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000100671 2024-09-16T20:49:28.022 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.022 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:28.022 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.022 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.022 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.022 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5439 3.a 3:5789f5c3:::rbd_data.110ed9ca540.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 1181696~60416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:28.023 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:49:28.023 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 65.089940325000001, 2024-09-16T20:49:28.023 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10102045599999999, 2024-09-16T20:49:28.023 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:28.023 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:28.023 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:28.023 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:28.023 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:28.023 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:28.023 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5439 2024-09-16T20:49:28.023 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.023 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:28.023 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.023 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:28.024 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:49:28.024 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.024 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.024 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.024 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:28.024 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:49:28.024 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.024 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.024 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.024 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:28.024 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864711+0000", 2024-09-16T20:49:28.024 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7899999999999997e-07 2024-09-16T20:49:28.024 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.025 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.025 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:28.025 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864760+0000", 2024-09-16T20:49:28.025 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8803e-05 2024-09-16T20:49:28.025 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.025 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.025 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:28.025 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864769+0000", 2024-09-16T20:49:28.025 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0999999999999993e-06 2024-09-16T20:49:28.025 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.025 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.025 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:28.025 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864774+0000", 2024-09-16T20:49:28.026 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8919999999999999e-06 2024-09-16T20:49:28.026 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.026 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.026 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:28.026 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864815+0000", 2024-09-16T20:49:28.026 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1307000000000003e-05 2024-09-16T20:49:28.026 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.026 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.026 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:28.026 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864893+0000", 2024-09-16T20:49:28.026 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7581000000000001e-05 2024-09-16T20:49:28.026 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.026 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.026 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 1", 2024-09-16T20:49:28.027 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864996+0000", 2024-09-16T20:49:28.027 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102725 2024-09-16T20:49:28.027 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.027 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.027 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:28.027 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965442+0000", 2024-09-16T20:49:28.027 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100446012 2024-09-16T20:49:28.027 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.027 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.027 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:28.027 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965618+0000", 2024-09-16T20:49:28.027 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00017653100000000001 2024-09-16T20:49:28.027 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.027 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.028 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:28.028 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965682+0000", 2024-09-16T20:49:28.028 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3905999999999994e-05 2024-09-16T20:49:28.028 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.028 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.028 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:28.028 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965731+0000", 2024-09-16T20:49:28.028 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8819999999999997e-05 2024-09-16T20:49:28.028 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.028 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:28.029 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.029 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.029 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.029 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4600.0:5439 3.a e37/23 ondisk, result = 0)", 2024-09-16T20:49:28.029 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:49:28.029 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 65.088944177000002, 2024-09-16T20:49:28.029 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099742634999999996, 2024-09-16T20:49:28.029 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:28.029 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:28.029 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:28.029 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:28.029 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.029 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:28.030 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:49:28.030 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.030 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.030 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.030 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:28.030 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:49:28.030 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.030 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.030 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.030 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:28.030 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:49:28.030 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0900000000000001e-07 2024-09-16T20:49:28.030 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.030 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.031 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:28.031 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865711+0000", 2024-09-16T20:49:28.031 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3739999999999998e-06 2024-09-16T20:49:28.031 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.031 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.031 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:28.031 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965343+0000", 2024-09-16T20:49:28.031 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099632526999999999 2024-09-16T20:49:28.031 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.031 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.031 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:28.031 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965357+0000", 2024-09-16T20:49:28.031 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3713999999999999e-05 2024-09-16T20:49:28.032 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.032 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.032 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:28.032 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965426+0000", 2024-09-16T20:49:28.032 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8745999999999995e-05 2024-09-16T20:49:28.032 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.032 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.032 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:28.032 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965434+0000", 2024-09-16T20:49:28.032 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6910000000000003e-06 2024-09-16T20:49:28.033 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.033 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.033 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:28.033 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965449+0000", 2024-09-16T20:49:28.033 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4874e-05 2024-09-16T20:49:28.033 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.033 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:28.034 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.034 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.034 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.034 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4882.0:2508 3.12 3:4992771f:::rbd_object_map.110e7c8851c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T20:49:28.034 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:49:28.034 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 41.473140629, 2024-09-16T20:49:28.034 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.125007011, 2024-09-16T20:49:28.034 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:28.034 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:28.034 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:28.034 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:28.034 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4882", 2024-09-16T20:49:28.035 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2436923223", 2024-09-16T20:49:28.035 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2508 2024-09-16T20:49:28.035 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.035 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:28.035 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.035 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:28.035 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:49:28.035 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.035 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.035 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.035 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:28.035 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:49:28.035 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.036 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.036 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.036 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:28.036 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481511+0000", 2024-09-16T20:49:28.036 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0699999999999996e-07 2024-09-16T20:49:28.036 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.036 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.036 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:28.036 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481515+0000", 2024-09-16T20:49:28.036 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0609999999999997e-06 2024-09-16T20:49:28.036 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.036 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.036 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:28.037 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481523+0000", 2024-09-16T20:49:28.037 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1310000000000006e-06 2024-09-16T20:49:28.037 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.037 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.037 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:28.037 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481528+0000", 2024-09-16T20:49:28.037 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4900000000000002e-06 2024-09-16T20:49:28.037 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.037 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.037 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:28.037 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481565+0000", 2024-09-16T20:49:28.037 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7066000000000001e-05 2024-09-16T20:49:28.037 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.037 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.038 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:28.038 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481628+0000", 2024-09-16T20:49:28.038 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3118000000000001e-05 2024-09-16T20:49:28.038 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.038 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.038 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 2", 2024-09-16T20:49:28.038 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481822+0000", 2024-09-16T20:49:28.038 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019446200000000001 2024-09-16T20:49:28.038 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.038 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.038 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:28.038 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605988+0000", 2024-09-16T20:49:28.038 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.124165888 2024-09-16T20:49:28.038 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.039 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.039 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:28.039 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606427+0000", 2024-09-16T20:49:28.039 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00043854899999999999 2024-09-16T20:49:28.039 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.039 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.039 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:28.039 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606476+0000", 2024-09-16T20:49:28.039 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8971999999999998e-05 2024-09-16T20:49:28.039 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.039 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.039 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:28.039 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606517+0000", 2024-09-16T20:49:28.039 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1467000000000001e-05 2024-09-16T20:49:28.040 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.040 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:28.040 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.040 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.040 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.040 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4882.0:2508 3.12 e57/55 ondisk, result = 0)", 2024-09-16T20:49:28.040 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:49:28.040 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 41.472152129000001, 2024-09-16T20:49:28.040 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12349711400000001, 2024-09-16T20:49:28.040 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:28.040 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:28.040 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:28.040 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:28.040 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.041 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:28.041 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:49:28.041 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.041 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.041 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.041 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:28.041 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:49:28.041 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.041 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.041 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.041 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:28.041 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:49:28.041 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.06e-07 2024-09-16T20:49:28.042 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.042 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.042 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:28.042 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482502+0000", 2024-09-16T20:49:28.042 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.01e-06 2024-09-16T20:49:28.042 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.042 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.042 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:28.042 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605772+0000", 2024-09-16T20:49:28.042 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.123269687 2024-09-16T20:49:28.042 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.042 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.042 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:28.042 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605791+0000", 2024-09-16T20:49:28.043 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8912000000000001e-05 2024-09-16T20:49:28.043 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.043 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.043 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:28.043 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605974+0000", 2024-09-16T20:49:28.043 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000182836 2024-09-16T20:49:28.043 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.043 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.043 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:28.043 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605982+0000", 2024-09-16T20:49:28.043 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1699999999999997e-06 2024-09-16T20:49:28.043 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.043 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.043 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:28.044 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605996+0000", 2024-09-16T20:49:28.044 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3893000000000001e-05 2024-09-16T20:49:28.044 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.044 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:28.044 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.044 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.044 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.044 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4882.0:2512 3.f 3:f4d895b7:::rbd_data.110e7c8851c.0000000000000000:head [read 16384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T20:49:28.044 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:49:28.044 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 41.471715099000001, 2024-09-16T20:49:28.044 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12318483, 2024-09-16T20:49:28.044 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:28.044 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:28.044 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:28.045 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:28.045 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4882", 2024-09-16T20:49:28.045 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2436923223", 2024-09-16T20:49:28.045 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2512 2024-09-16T20:49:28.045 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.045 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:28.045 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.045 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:28.045 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:49:28.045 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.045 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.045 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.046 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:28.046 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:49:28.046 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.046 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.046 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.046 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:28.046 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:49:28.046 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3499999999999995e-07 2024-09-16T20:49:28.046 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.046 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.046 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:28.046 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482943+0000", 2024-09-16T20:49:28.046 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3450000000000002e-06 2024-09-16T20:49:28.046 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.046 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.047 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:28.047 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605772+0000", 2024-09-16T20:49:28.047 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.122829796 2024-09-16T20:49:28.047 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.047 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.047 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:28.047 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605792+0000", 2024-09-16T20:49:28.047 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9916999999999999e-05 2024-09-16T20:49:28.047 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.047 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.047 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:28.047 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605917+0000", 2024-09-16T20:49:28.047 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00012500799999999999 2024-09-16T20:49:28.047 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.048 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.048 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:28.048 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606004+0000", 2024-09-16T20:49:28.048 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6242999999999994e-05 2024-09-16T20:49:28.048 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.048 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.048 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:28.048 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606120+0000", 2024-09-16T20:49:28.048 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000116786 2024-09-16T20:49:28.048 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.048 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:28.048 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.048 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.049 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.049 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:11702 3.15 3:acf33c8b:::rbd_data.110e747f0779.000000000000001a:head [set-alloc-hint object_size 262144 write_size 262144,write 143360~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:49:28.049 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:49:28.049 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.816722677000001, 2024-09-16T20:49:28.049 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.084801043000000006, 2024-09-16T20:49:28.049 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:28.049 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:28.049 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:28.049 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:28.049 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:49:28.049 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:49:28.049 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 11702 2024-09-16T20:49:28.049 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.050 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:28.050 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.050 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:28.050 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:49:28.050 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.050 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.050 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.050 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:28.050 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:49:28.050 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.050 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.050 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.050 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:28.050 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137929+0000", 2024-09-16T20:49:28.051 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8400000000000002e-07 2024-09-16T20:49:28.051 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.051 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.051 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:28.051 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137937+0000", 2024-09-16T20:49:28.051 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0260000000000007e-06 2024-09-16T20:49:28.051 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.051 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.051 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:28.051 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137952+0000", 2024-09-16T20:49:28.051 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4659e-05 2024-09-16T20:49:28.051 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.051 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.051 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:28.052 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137960+0000", 2024-09-16T20:49:28.052 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0169999999999996e-06 2024-09-16T20:49:28.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.052 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.052 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:28.052 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138010+0000", 2024-09-16T20:49:28.052 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0021000000000001e-05 2024-09-16T20:49:28.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.052 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.052 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:28.052 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138114+0000", 2024-09-16T20:49:28.052 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000103797 2024-09-16T20:49:28.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.053 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.053 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:49:28.053 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138233+0000", 2024-09-16T20:49:28.053 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000118971 2024-09-16T20:49:28.053 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.053 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.053 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:28.053 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138953+0000", 2024-09-16T20:49:28.053 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00072026000000000002 2024-09-16T20:49:28.053 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.053 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.053 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:28.053 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222609+0000", 2024-09-16T20:49:28.053 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.083656650999999999 2024-09-16T20:49:28.054 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.054 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.054 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:28.054 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222684+0000", 2024-09-16T20:49:28.054 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4369999999999994e-05 2024-09-16T20:49:28.054 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.054 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.054 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:28.054 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222729+0000", 2024-09-16T20:49:28.054 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5287000000000002e-05 2024-09-16T20:49:28.054 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.054 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:28.054 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.054 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.055 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.055 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:11704 3.9 3:91220b75:::rbd_data.110e747f0779.000000000000001c:head [set-alloc-hint object_size 262144 write_size 262144,write 139264~6656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:49:28.055 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:49:28.055 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.816484676000002, 2024-09-16T20:49:28.055 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.084531133999999994, 2024-09-16T20:49:28.055 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:28.055 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:28.055 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:28.055 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:28.055 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:49:28.055 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:49:28.055 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 11704 2024-09-16T20:49:28.055 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.056 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:28.056 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.056 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:28.056 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:49:28.056 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.056 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.056 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.056 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:28.056 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:49:28.056 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.056 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.056 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.056 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:28.056 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138167+0000", 2024-09-16T20:49:28.057 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.33e-07 2024-09-16T20:49:28.057 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.057 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.057 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:28.057 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138179+0000", 2024-09-16T20:49:28.057 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2055999999999999e-05 2024-09-16T20:49:28.057 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.057 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.057 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:28.057 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138193+0000", 2024-09-16T20:49:28.057 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3961000000000001e-05 2024-09-16T20:49:28.057 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.057 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.057 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:28.058 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138200+0000", 2024-09-16T20:49:28.058 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4429999999999997e-06 2024-09-16T20:49:28.058 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.058 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.058 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:28.058 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138258+0000", 2024-09-16T20:49:28.058 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7225999999999997e-05 2024-09-16T20:49:28.058 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.058 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.058 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:28.058 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138364+0000", 2024-09-16T20:49:28.058 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000106013 2024-09-16T20:49:28.058 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.058 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.059 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:49:28.059 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138494+0000", 2024-09-16T20:49:28.059 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013059599999999999 2024-09-16T20:49:28.059 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.059 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.059 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:28.059 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.139220+0000", 2024-09-16T20:49:28.059 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00072539800000000004 2024-09-16T20:49:28.059 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.059 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.059 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:28.059 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222608+0000", 2024-09-16T20:49:28.059 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.083388620999999996 2024-09-16T20:49:28.059 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.060 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.060 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:28.060 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222651+0000", 2024-09-16T20:49:28.060 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2228000000000003e-05 2024-09-16T20:49:28.060 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.060 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.060 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:28.060 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222697+0000", 2024-09-16T20:49:28.060 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6659000000000002e-05 2024-09-16T20:49:28.060 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.060 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:28.060 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.060 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.061 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:28.061 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:28.062 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:49:28.063 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:28.060+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:28.163 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:28.162+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:28.169 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:28.169 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:49:28.169 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:49:28.170 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:28.178 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:49:28.264 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:28.263+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:28.285 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:28.285 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:49:28.285 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:49:28.285 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:49:28.285 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:28.293 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:49:28.378 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:28.377+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:28.443 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:28.443 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:49:28.443 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:49:28.444 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:49:28.444 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.444 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:49:28.444 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:49:28.444 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 95.186535641999996, 2024-09-16T20:49:28.444 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67502346999999996, 2024-09-16T20:49:28.444 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:28.445 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:28.445 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:28.445 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:28.445 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:49:28.445 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:49:28.445 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 1 2024-09-16T20:49:28.445 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.445 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:28.445 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.445 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:28.445 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:49:28.445 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.445 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.445 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.446 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:28.446 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:49:28.446 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.446 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.446 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.446 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:28.446 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254128+0000", 2024-09-16T20:49:28.446 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.463e-06 2024-09-16T20:49:28.446 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.446 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.446 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:28.446 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254135+0000", 2024-09-16T20:49:28.446 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1579999999999999e-06 2024-09-16T20:49:28.446 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.447 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.447 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:28.447 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254146+0000", 2024-09-16T20:49:28.447 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1341e-05 2024-09-16T20:49:28.447 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.447 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.447 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:28.447 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254155+0000", 2024-09-16T20:49:28.447 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.1549999999999996e-06 2024-09-16T20:49:28.447 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.447 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.448 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:28.448 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254221+0000", 2024-09-16T20:49:28.448 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5470999999999997e-05 2024-09-16T20:49:28.448 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.448 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.448 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:49:28.448 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254231+0000", 2024-09-16T20:49:28.448 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0592e-05 2024-09-16T20:49:28.448 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.449 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.449 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:28.449 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926711+0000", 2024-09-16T20:49:28.449 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67247946599999997 2024-09-16T20:49:28.449 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.449 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.449 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:28.449 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926910+0000", 2024-09-16T20:49:28.449 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019934 2024-09-16T20:49:28.449 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.449 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.450 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:49:28.450 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.927159+0000", 2024-09-16T20:49:28.450 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00024876599999999998 2024-09-16T20:49:28.450 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.450 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.450 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:28.450 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.928797+0000", 2024-09-16T20:49:28.450 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001638308 2024-09-16T20:49:28.450 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.450 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.450 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:28.450 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929026+0000", 2024-09-16T20:49:28.450 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00022876099999999999 2024-09-16T20:49:28.450 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.451 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.451 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:28.451 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929102+0000", 2024-09-16T20:49:28.451 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6184000000000001e-05 2024-09-16T20:49:28.451 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.451 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.451 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:28.451 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929150+0000", 2024-09-16T20:49:28.451 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7465e-05 2024-09-16T20:49:28.451 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.451 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:28.451 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.451 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.451 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.452 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4319 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4675584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:28.452 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745838+0000", 2024-09-16T20:49:28.452 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 68.694823830999994, 2024-09-16T20:49:28.452 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047640204999999998, 2024-09-16T20:49:28.452 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:28.452 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:28.452 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:28.452 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:28.452 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:49:28.452 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:28.452 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4319 2024-09-16T20:49:28.452 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.452 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:28.453 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.453 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:28.453 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745838+0000", 2024-09-16T20:49:28.453 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.453 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.453 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.453 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:28.453 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745838+0000", 2024-09-16T20:49:28.453 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.453 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.453 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.453 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:28.453 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745838+0000", 2024-09-16T20:49:28.454 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6899999999999998e-07 2024-09-16T20:49:28.454 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.454 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.454 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:28.454 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745840+0000", 2024-09-16T20:49:28.454 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0660000000000002e-06 2024-09-16T20:49:28.454 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.454 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.454 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:28.454 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745847+0000", 2024-09-16T20:49:28.454 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8689999999999998e-06 2024-09-16T20:49:28.455 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.455 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.455 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:28.455 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745853+0000", 2024-09-16T20:49:28.455 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2399999999999998e-06 2024-09-16T20:49:28.455 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.455 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.455 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:28.455 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793381+0000", 2024-09-16T20:49:28.455 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047528005999999998 2024-09-16T20:49:28.455 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.455 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.455 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:28.455 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793423+0000", 2024-09-16T20:49:28.456 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2799000000000001e-05 2024-09-16T20:49:28.456 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.456 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.456 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:28.456 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793478+0000", 2024-09-16T20:49:28.456 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4756000000000002e-05 2024-09-16T20:49:28.456 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.456 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:28.456 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.456 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.456 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.456 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4320 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4683776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:28.456 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745859+0000", 2024-09-16T20:49:28.457 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 68.694802586999998, 2024-09-16T20:49:28.457 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047811399999999997, 2024-09-16T20:49:28.457 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:28.457 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:28.457 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:28.457 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:28.457 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:49:28.457 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:28.457 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4320 2024-09-16T20:49:28.457 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.457 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:28.457 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.457 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:28.457 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745859+0000", 2024-09-16T20:49:28.458 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.458 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.458 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.458 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:28.458 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745859+0000", 2024-09-16T20:49:28.458 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.458 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.458 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.458 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:28.458 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745860+0000", 2024-09-16T20:49:28.458 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9099999999999999e-07 2024-09-16T20:49:28.458 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.459 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.459 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:28.459 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745861+0000", 2024-09-16T20:49:28.459 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8589999999999999e-06 2024-09-16T20:49:28.459 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.459 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.459 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:28.459 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745868+0000", 2024-09-16T20:49:28.459 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2249999999999997e-06 2024-09-16T20:49:28.459 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.459 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.459 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:28.459 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745873+0000", 2024-09-16T20:49:28.460 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1290000000000004e-06 2024-09-16T20:49:28.460 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.460 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.460 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:28.460 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793522+0000", 2024-09-16T20:49:28.460 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047648847000000001 2024-09-16T20:49:28.460 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.460 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.460 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:28.460 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793603+0000", 2024-09-16T20:49:28.460 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1167000000000004e-05 2024-09-16T20:49:28.460 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.461 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.461 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:28.461 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793671+0000", 2024-09-16T20:49:28.461 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7781999999999996e-05 2024-09-16T20:49:28.461 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.461 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:28.461 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.461 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.461 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.461 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4321 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4691968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:28.461 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745879+0000", 2024-09-16T20:49:28.461 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 68.694782391000004, 2024-09-16T20:49:28.462 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047950375000000003, 2024-09-16T20:49:28.462 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:28.462 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:28.462 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:28.462 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:28.462 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:49:28.462 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:28.462 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4321 2024-09-16T20:49:28.462 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.462 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:28.462 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.462 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:28.462 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745879+0000", 2024-09-16T20:49:28.462 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.463 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.463 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.463 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:28.463 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745879+0000", 2024-09-16T20:49:28.463 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.463 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.463 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.463 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:28.463 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745880+0000", 2024-09-16T20:49:28.463 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.65e-07 2024-09-16T20:49:28.463 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.463 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.463 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:28.463 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745882+0000", 2024-09-16T20:49:28.464 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.922e-06 2024-09-16T20:49:28.464 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.464 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.464 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:28.464 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745888+0000", 2024-09-16T20:49:28.464 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0510000000000002e-06 2024-09-16T20:49:28.464 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.464 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.464 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:28.464 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745893+0000", 2024-09-16T20:49:28.464 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9579999999999998e-06 2024-09-16T20:49:28.464 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.464 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.465 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:28.465 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793704+0000", 2024-09-16T20:49:28.465 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047811179000000002 2024-09-16T20:49:28.465 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.465 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.465 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:28.465 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793759+0000", 2024-09-16T20:49:28.465 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5260999999999998e-05 2024-09-16T20:49:28.465 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.465 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.465 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:28.465 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793830+0000", 2024-09-16T20:49:28.465 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0638999999999999e-05 2024-09-16T20:49:28.466 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.466 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:28.466 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.466 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.466 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.466 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4322 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4700160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:28.466 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745899+0000", 2024-09-16T20:49:28.466 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 68.694762952999994, 2024-09-16T20:49:28.466 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048057280000000001, 2024-09-16T20:49:28.466 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:28.466 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:28.466 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:28.466 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:28.466 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:49:28.467 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:28.467 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4322 2024-09-16T20:49:28.467 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.467 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:28.467 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.467 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:28.467 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745899+0000", 2024-09-16T20:49:28.467 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.467 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.467 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.467 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:28.467 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745899+0000", 2024-09-16T20:49:28.467 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.467 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.468 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.468 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:28.468 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745899+0000", 2024-09-16T20:49:28.468 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9299999999999999e-07 2024-09-16T20:49:28.468 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.468 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.468 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:28.468 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745901+0000", 2024-09-16T20:49:28.468 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0030000000000001e-06 2024-09-16T20:49:28.468 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.468 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.468 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:28.468 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745908+0000", 2024-09-16T20:49:28.469 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3729999999999998e-06 2024-09-16T20:49:28.469 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.469 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.469 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:28.469 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745914+0000", 2024-09-16T20:49:28.469 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.122e-06 2024-09-16T20:49:28.469 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.469 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.469 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:28.469 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793862+0000", 2024-09-16T20:49:28.469 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047948324 2024-09-16T20:49:28.469 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.469 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.469 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:28.470 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793903+0000", 2024-09-16T20:49:28.470 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0465e-05 2024-09-16T20:49:28.470 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.470 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.470 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:28.470 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793956+0000", 2024-09-16T20:49:28.470 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3600000000000002e-05 2024-09-16T20:49:28.470 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.470 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:28.470 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.470 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.470 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.470 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4323 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4708352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:28.470 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745920+0000", 2024-09-16T20:49:28.470 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 68.694741312000005, 2024-09-16T20:49:28.471 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048268138000000002, 2024-09-16T20:49:28.471 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:28.471 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:28.471 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:28.471 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:28.471 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:49:28.471 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:28.471 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4323 2024-09-16T20:49:28.471 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.471 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:28.471 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.471 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:28.471 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745920+0000", 2024-09-16T20:49:28.471 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.472 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.472 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.472 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:28.472 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745920+0000", 2024-09-16T20:49:28.472 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.472 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.472 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.472 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:28.472 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745921+0000", 2024-09-16T20:49:28.472 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7e-07 2024-09-16T20:49:28.472 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.472 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.472 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:28.473 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745923+0000", 2024-09-16T20:49:28.473 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7880000000000001e-06 2024-09-16T20:49:28.473 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.473 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.473 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:28.473 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745929+0000", 2024-09-16T20:49:28.473 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0809999999999999e-06 2024-09-16T20:49:28.473 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.473 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.473 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:28.473 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745935+0000", 2024-09-16T20:49:28.473 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5370000000000003e-06 2024-09-16T20:49:28.473 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.474 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.474 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:28.474 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.793987+0000", 2024-09-16T20:49:28.474 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048051736999999997 2024-09-16T20:49:28.474 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.474 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.474 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:28.474 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.794070+0000", 2024-09-16T20:49:28.474 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.3277999999999998e-05 2024-09-16T20:49:28.474 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.474 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.474 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:28.474 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.794189+0000", 2024-09-16T20:49:28.474 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000118347 2024-09-16T20:49:28.475 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.475 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:28.475 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.475 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.475 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.475 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4520.0:4324 3.13 3:ce4ddb94:::rbd_data.110e3f481dff.0000000000000009:7 [read 4716544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:28.475 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:19.745946+0000", 2024-09-16T20:49:28.475 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 68.694715740999996, 2024-09-16T20:49:28.475 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048468153999999999, 2024-09-16T20:49:28.475 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:28.475 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:28.475 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:28.475 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:28.475 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4520", 2024-09-16T20:49:28.476 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:28.476 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4324 2024-09-16T20:49:28.476 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.476 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:28.476 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.476 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:28.476 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745946+0000", 2024-09-16T20:49:28.476 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.476 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.476 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.477 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:28.477 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745946+0000", 2024-09-16T20:49:28.477 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.477 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.477 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.477 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:28.477 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745946+0000", 2024-09-16T20:49:28.477 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4499999999999997e-07 2024-09-16T20:49:28.477 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.477 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.477 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:28.478 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745949+0000", 2024-09-16T20:49:28.478 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2529999999999999e-06 2024-09-16T20:49:28.478 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.478 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.478 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:28.478 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745955+0000", 2024-09-16T20:49:28.478 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6470000000000001e-06 2024-09-16T20:49:28.478 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.478 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.478 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:28.479 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.745960+0000", 2024-09-16T20:49:28.479 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7910000000000004e-06 2024-09-16T20:49:28.479 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.479 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.479 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:28.479 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.794220+0000", 2024-09-16T20:49:28.479 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048260069000000003 2024-09-16T20:49:28.479 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.479 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.479 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:28.479 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.794311+0000", 2024-09-16T20:49:28.479 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0593000000000005e-05 2024-09-16T20:49:28.480 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.480 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.480 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:28.480 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:19.794414+0000", 2024-09-16T20:49:28.480 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000103356 2024-09-16T20:49:28.480 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.480 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:28.480 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.480 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.480 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.480 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3571 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:28.480 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481059+0000", 2024-09-16T20:49:28.481 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 38.959602580999999, 2024-09-16T20:49:28.481 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049572995000000002, 2024-09-16T20:49:28.481 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:28.481 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:28.481 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:28.481 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:28.481 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:28.481 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:28.481 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3571 2024-09-16T20:49:28.481 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.481 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:28.481 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.482 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:28.482 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481059+0000", 2024-09-16T20:49:28.482 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.482 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.482 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.482 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:28.482 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481059+0000", 2024-09-16T20:49:28.482 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.482 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.482 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.482 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:28.482 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481060+0000", 2024-09-16T20:49:28.483 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2599999999999998e-07 2024-09-16T20:49:28.483 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.483 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.483 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:28.483 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481063+0000", 2024-09-16T20:49:28.483 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.999e-06 2024-09-16T20:49:28.483 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.483 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.483 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:28.483 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481067+0000", 2024-09-16T20:49:28.483 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3080000000000001e-06 2024-09-16T20:49:28.484 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.484 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.484 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:28.484 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481070+0000", 2024-09-16T20:49:28.484 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9919999999999999e-06 2024-09-16T20:49:28.484 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.484 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.484 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:28.484 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.522087+0000", 2024-09-16T20:49:28.484 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.041017023 2024-09-16T20:49:28.484 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.484 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.485 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:28.485 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.522191+0000", 2024-09-16T20:49:28.485 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010389499999999999 2024-09-16T20:49:28.485 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.485 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.485 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:28.485 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.522566+0000", 2024-09-16T20:49:28.485 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00037539399999999998 2024-09-16T20:49:28.485 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.485 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.485 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:28.486 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525494+0000", 2024-09-16T20:49:28.486 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0029273070000000001 2024-09-16T20:49:28.486 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.486 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.486 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:28.486 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530355+0000", 2024-09-16T20:49:28.486 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0048612509999999996 2024-09-16T20:49:28.486 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.486 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.486 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:28.486 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530407+0000", 2024-09-16T20:49:28.487 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1817000000000002e-05 2024-09-16T20:49:28.487 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.487 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.487 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:28.487 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530632+0000", 2024-09-16T20:49:28.487 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00022558300000000001 2024-09-16T20:49:28.487 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.487 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:28.487 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.487 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.487 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.488 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3572 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:28.488 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481220+0000", 2024-09-16T20:49:28.488 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 38.959442013, 2024-09-16T20:49:28.488 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050452862000000001, 2024-09-16T20:49:28.488 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:28.488 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:28.488 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:28.488 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:28.488 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:28.488 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:28.489 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3572 2024-09-16T20:49:28.489 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.489 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:28.489 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.489 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:28.489 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481220+0000", 2024-09-16T20:49:28.489 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.489 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.489 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.489 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:28.489 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481220+0000", 2024-09-16T20:49:28.490 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.490 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.490 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.490 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:28.490 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481221+0000", 2024-09-16T20:49:28.490 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9400000000000004e-07 2024-09-16T20:49:28.490 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.490 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.490 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:28.490 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481225+0000", 2024-09-16T20:49:28.491 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0230000000000003e-06 2024-09-16T20:49:28.491 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.491 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.491 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:28.491 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481234+0000", 2024-09-16T20:49:28.491 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2920000000000001e-06 2024-09-16T20:49:28.491 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.491 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.491 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:28.491 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481240+0000", 2024-09-16T20:49:28.491 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9390000000000002e-06 2024-09-16T20:49:28.491 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.492 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.492 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:28.492 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.523544+0000", 2024-09-16T20:49:28.492 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.042304475000000001 2024-09-16T20:49:28.492 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.492 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.492 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:28.492 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.523624+0000", 2024-09-16T20:49:28.492 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.975e-05 2024-09-16T20:49:28.492 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.493 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.493 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:28.493 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.523975+0000", 2024-09-16T20:49:28.493 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00035063200000000003 2024-09-16T20:49:28.493 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.493 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.493 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:28.493 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530337+0000", 2024-09-16T20:49:28.493 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0063623589999999997 2024-09-16T20:49:28.493 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.494 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.494 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:28.494 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531592+0000", 2024-09-16T20:49:28.494 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0012550599999999999 2024-09-16T20:49:28.494 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.494 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.494 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:28.494 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531633+0000", 2024-09-16T20:49:28.494 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1192000000000001e-05 2024-09-16T20:49:28.494 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.495 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.495 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:28.495 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531673+0000", 2024-09-16T20:49:28.495 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9345999999999999e-05 2024-09-16T20:49:28.495 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.495 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:28.495 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.495 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.495 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.495 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3573 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:28.495 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481257+0000", 2024-09-16T20:49:28.496 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 38.959405066000002, 2024-09-16T20:49:28.496 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050511027, 2024-09-16T20:49:28.496 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:28.496 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:28.496 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:28.496 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:28.496 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:28.496 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:28.496 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3573 2024-09-16T20:49:28.496 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.496 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:28.497 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.497 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:28.497 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481257+0000", 2024-09-16T20:49:28.497 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.497 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.497 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.497 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:28.497 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481257+0000", 2024-09-16T20:49:28.497 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.497 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.497 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.498 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:28.498 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481257+0000", 2024-09-16T20:49:28.498 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.99e-07 2024-09-16T20:49:28.498 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.498 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.498 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:28.498 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481260+0000", 2024-09-16T20:49:28.498 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1709999999999998e-06 2024-09-16T20:49:28.498 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.498 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.498 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:28.498 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481271+0000", 2024-09-16T20:49:28.499 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0795000000000001e-05 2024-09-16T20:49:28.499 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.499 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.499 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:28.499 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481277+0000", 2024-09-16T20:49:28.499 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1670000000000004e-06 2024-09-16T20:49:28.499 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.499 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.499 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:28.499 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.524647+0000", 2024-09-16T20:49:28.499 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.043369810000000002 2024-09-16T20:49:28.500 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.500 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.500 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:28.500 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.524725+0000", 2024-09-16T20:49:28.500 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7286000000000005e-05 2024-09-16T20:49:28.500 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.500 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.501 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:28.501 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525062+0000", 2024-09-16T20:49:28.501 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00033739499999999999 2024-09-16T20:49:28.501 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.501 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.501 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:28.501 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530692+0000", 2024-09-16T20:49:28.501 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0056301479999999997 2024-09-16T20:49:28.501 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.501 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.501 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:28.502 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531694+0000", 2024-09-16T20:49:28.502 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0010017419999999999 2024-09-16T20:49:28.502 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.502 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.502 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:28.502 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531737+0000", 2024-09-16T20:49:28.502 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2762999999999997e-05 2024-09-16T20:49:28.502 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.502 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.502 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:28.502 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531768+0000", 2024-09-16T20:49:28.503 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1151e-05 2024-09-16T20:49:28.503 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.503 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:28.503 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.503 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.503 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.503 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3574 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:28.503 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:49:28.503 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 38.959372109999997, 2024-09-16T20:49:28.503 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053579441999999998, 2024-09-16T20:49:28.503 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:28.504 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:28.504 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:28.504 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:28.504 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:28.504 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:28.504 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3574 2024-09-16T20:49:28.504 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.504 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:28.504 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.504 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:28.504 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:49:28.505 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.505 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.505 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.505 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:28.505 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:49:28.505 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.505 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.505 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.505 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:28.505 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:49:28.506 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2e-07 2024-09-16T20:49:28.506 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.506 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.506 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:28.506 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481292+0000", 2024-09-16T20:49:28.506 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2630000000000002e-06 2024-09-16T20:49:28.506 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.506 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.506 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:28.506 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481297+0000", 2024-09-16T20:49:28.506 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.296e-06 2024-09-16T20:49:28.507 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.507 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.507 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:28.507 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481300+0000", 2024-09-16T20:49:28.507 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7019999999999999e-06 2024-09-16T20:49:28.507 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.507 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.507 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:28.507 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525533+0000", 2024-09-16T20:49:28.507 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.044232318 2024-09-16T20:49:28.507 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.508 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.508 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:28.508 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525619+0000", 2024-09-16T20:49:28.508 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5892999999999999e-05 2024-09-16T20:49:28.508 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.508 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.508 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:28.508 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525987+0000", 2024-09-16T20:49:28.508 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00036836099999999999 2024-09-16T20:49:28.508 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.509 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.509 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:28.509 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531779+0000", 2024-09-16T20:49:28.509 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0057918240000000001 2024-09-16T20:49:28.509 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.509 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.509 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:28.509 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534747+0000", 2024-09-16T20:49:28.509 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0029681600000000001 2024-09-16T20:49:28.509 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.509 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.509 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:28.509 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534832+0000", 2024-09-16T20:49:28.510 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5180999999999996e-05 2024-09-16T20:49:28.510 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.510 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.510 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:28.510 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534869+0000", 2024-09-16T20:49:28.510 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7024000000000003e-05 2024-09-16T20:49:28.510 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.510 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:28.510 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.510 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.510 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.510 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3575 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:28.510 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:49:28.511 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 38.959357369999999, 2024-09-16T20:49:28.511 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053801463000000001, 2024-09-16T20:49:28.511 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:28.511 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:28.511 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:28.511 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:28.511 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:28.511 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:28.511 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3575 2024-09-16T20:49:28.511 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.511 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:28.512 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.512 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:28.512 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:49:28.512 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.512 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.512 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.512 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:28.512 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:49:28.512 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.512 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.512 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.512 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:28.512 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481305+0000", 2024-09-16T20:49:28.513 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5400000000000002e-07 2024-09-16T20:49:28.513 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.513 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.513 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:28.513 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481307+0000", 2024-09-16T20:49:28.513 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7800000000000001e-06 2024-09-16T20:49:28.513 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.513 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.513 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:28.513 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481312+0000", 2024-09-16T20:49:28.513 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6349999999999997e-06 2024-09-16T20:49:28.513 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.514 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.514 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:28.514 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481315+0000", 2024-09-16T20:49:28.514 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7829999999999999e-06 2024-09-16T20:49:28.514 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.514 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.514 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:28.514 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.526876+0000", 2024-09-16T20:49:28.514 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.045561070000000002 2024-09-16T20:49:28.514 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.514 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.514 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:28.514 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.526954+0000", 2024-09-16T20:49:28.515 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8108000000000005e-05 2024-09-16T20:49:28.515 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.515 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.515 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:28.515 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527282+0000", 2024-09-16T20:49:28.515 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000327483 2024-09-16T20:49:28.515 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.515 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.515 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:28.515 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531784+0000", 2024-09-16T20:49:28.515 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0045020169999999997 2024-09-16T20:49:28.516 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.516 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.516 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:28.516 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534921+0000", 2024-09-16T20:49:28.516 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.003137791 2024-09-16T20:49:28.516 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.516 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.516 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:28.516 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534976+0000", 2024-09-16T20:49:28.516 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4953000000000001e-05 2024-09-16T20:49:28.516 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.517 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.517 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:28.517 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535106+0000", 2024-09-16T20:49:28.517 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000129589 2024-09-16T20:49:28.517 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.517 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:28.517 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.517 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.517 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.517 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3576 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:28.517 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:49:28.518 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 38.959343457999999, 2024-09-16T20:49:28.518 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053955436000000002, 2024-09-16T20:49:28.518 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:28.518 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:28.518 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:28.518 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:28.518 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:28.518 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:28.518 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3576 2024-09-16T20:49:28.518 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.518 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:28.518 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.519 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:28.519 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:49:28.519 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.519 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.519 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.519 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:28.519 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:49:28.519 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.519 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.519 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.519 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:28.519 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481319+0000", 2024-09-16T20:49:28.520 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.53e-07 2024-09-16T20:49:28.520 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.520 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.520 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:28.520 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481320+0000", 2024-09-16T20:49:28.520 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5230000000000001e-06 2024-09-16T20:49:28.520 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.520 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.520 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:28.520 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481330+0000", 2024-09-16T20:49:28.520 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0190000000000001e-05 2024-09-16T20:49:28.520 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.521 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.521 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:28.521 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481333+0000", 2024-09-16T20:49:28.521 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.151e-06 2024-09-16T20:49:28.521 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.521 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.521 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:28.521 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527689+0000", 2024-09-16T20:49:28.521 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046355192000000003 2024-09-16T20:49:28.521 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.521 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.521 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:28.522 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527764+0000", 2024-09-16T20:49:28.522 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5776000000000001e-05 2024-09-16T20:49:28.522 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.522 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.522 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:28.522 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528141+0000", 2024-09-16T20:49:28.522 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00037647599999999999 2024-09-16T20:49:28.522 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.522 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.522 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:28.522 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531788+0000", 2024-09-16T20:49:28.522 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0036470679999999998 2024-09-16T20:49:28.523 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.523 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.523 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:28.523 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535174+0000", 2024-09-16T20:49:28.523 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0033862670000000001 2024-09-16T20:49:28.523 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.523 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.523 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:28.523 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535236+0000", 2024-09-16T20:49:28.523 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1797999999999993e-05 2024-09-16T20:49:28.523 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.524 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.524 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:28.524 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535274+0000", 2024-09-16T20:49:28.524 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7741999999999999e-05 2024-09-16T20:49:28.524 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.524 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:28.524 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.524 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.524 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.524 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3577 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:28.524 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:49:28.524 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 38.959321858000003, 2024-09-16T20:49:28.525 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054073807000000002, 2024-09-16T20:49:28.525 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:28.525 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:28.525 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:28.525 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:28.525 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:28.525 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:28.525 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3577 2024-09-16T20:49:28.525 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.525 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:28.525 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.526 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:28.526 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:49:28.526 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.526 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.526 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.526 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:28.526 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:49:28.526 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.526 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.526 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.526 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:28.526 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:49:28.526 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.96e-07 2024-09-16T20:49:28.527 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.527 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.527 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:28.527 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481342+0000", 2024-09-16T20:49:28.527 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.714e-06 2024-09-16T20:49:28.527 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.527 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.527 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:28.527 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481349+0000", 2024-09-16T20:49:28.527 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4930000000000001e-06 2024-09-16T20:49:28.527 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.527 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.527 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:28.528 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481353+0000", 2024-09-16T20:49:28.528 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4879999999999999e-06 2024-09-16T20:49:28.528 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.528 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.528 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:28.528 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528619+0000", 2024-09-16T20:49:28.528 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047266351999999998 2024-09-16T20:49:28.528 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.528 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.528 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:28.528 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528706+0000", 2024-09-16T20:49:28.529 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6796e-05 2024-09-16T20:49:28.529 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.529 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.529 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:28.529 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529148+0000", 2024-09-16T20:49:28.529 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00044182999999999999 2024-09-16T20:49:28.529 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.529 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.529 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:28.529 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531792+0000", 2024-09-16T20:49:28.529 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0026443119999999998 2024-09-16T20:49:28.530 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.530 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.530 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:28.530 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535319+0000", 2024-09-16T20:49:28.530 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0035266439999999998 2024-09-16T20:49:28.530 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.530 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.530 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:28.530 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535372+0000", 2024-09-16T20:49:28.530 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3207000000000001e-05 2024-09-16T20:49:28.530 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.530 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.530 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:28.531 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535414+0000", 2024-09-16T20:49:28.531 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1675000000000003e-05 2024-09-16T20:49:28.531 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.531 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:28.531 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.531 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.531 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.531 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3578 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:28.531 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:49:28.531 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 38.959300689000003, 2024-09-16T20:49:28.531 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054189586999999997, 2024-09-16T20:49:28.531 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:28.531 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:28.532 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:28.532 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:28.532 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:28.532 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:28.532 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3578 2024-09-16T20:49:28.532 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.532 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:28.532 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.532 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:28.532 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:49:28.532 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.532 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.533 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.533 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:28.533 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:49:28.533 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.533 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.533 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.533 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:28.533 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:49:28.533 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0800000000000001e-07 2024-09-16T20:49:28.533 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.533 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.533 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:28.534 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481363+0000", 2024-09-16T20:49:28.534 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0339999999999999e-06 2024-09-16T20:49:28.534 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.534 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.534 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:28.534 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481371+0000", 2024-09-16T20:49:28.534 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4440000000000002e-06 2024-09-16T20:49:28.534 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.534 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.534 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:28.534 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481374+0000", 2024-09-16T20:49:28.534 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2049999999999998e-06 2024-09-16T20:49:28.534 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.535 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.535 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:28.535 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529555+0000", 2024-09-16T20:49:28.535 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048181202999999999 2024-09-16T20:49:28.535 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.535 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.535 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:28.535 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529616+0000", 2024-09-16T20:49:28.535 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0912000000000002e-05 2024-09-16T20:49:28.535 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.535 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.535 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:28.535 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529939+0000", 2024-09-16T20:49:28.536 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00032269800000000001 2024-09-16T20:49:28.536 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.536 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.536 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:28.536 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532602+0000", 2024-09-16T20:49:28.536 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0026634940000000002 2024-09-16T20:49:28.536 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.536 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.536 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:28.536 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535465+0000", 2024-09-16T20:49:28.536 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.002862149 2024-09-16T20:49:28.536 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.536 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.537 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:28.537 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535516+0000", 2024-09-16T20:49:28.537 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1696000000000001e-05 2024-09-16T20:49:28.537 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.537 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.537 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:28.537 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535551+0000", 2024-09-16T20:49:28.537 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4443999999999998e-05 2024-09-16T20:49:28.537 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.537 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:28.537 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.537 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.537 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.538 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3579 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:28.538 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:49:28.538 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 38.959282250000001, 2024-09-16T20:49:28.538 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054295151, 2024-09-16T20:49:28.538 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:28.538 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:28.538 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:28.538 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:28.538 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:28.538 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:28.538 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3579 2024-09-16T20:49:28.539 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.539 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:28.539 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.539 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:28.539 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:49:28.539 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.539 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.539 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.539 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:28.539 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:49:28.539 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.539 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.540 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.540 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:28.540 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:49:28.540 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3300000000000001e-07 2024-09-16T20:49:28.540 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.540 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.540 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:28.540 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481381+0000", 2024-09-16T20:49:28.540 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.649e-06 2024-09-16T20:49:28.540 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.540 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.540 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:28.540 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481387+0000", 2024-09-16T20:49:28.541 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4519999999999996e-06 2024-09-16T20:49:28.541 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.541 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.541 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:28.541 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481390+0000", 2024-09-16T20:49:28.541 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0170000000000001e-06 2024-09-16T20:49:28.541 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.541 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.541 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:28.541 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530756+0000", 2024-09-16T20:49:28.541 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049365882999999999 2024-09-16T20:49:28.542 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.542 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.542 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:28.542 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530845+0000", 2024-09-16T20:49:28.542 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9197000000000007e-05 2024-09-16T20:49:28.542 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.542 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.542 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:28.542 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531191+0000", 2024-09-16T20:49:28.542 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00034562500000000001 2024-09-16T20:49:28.542 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.543 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.543 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:28.543 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534078+0000", 2024-09-16T20:49:28.543 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0028878179999999999 2024-09-16T20:49:28.543 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.543 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.543 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:28.543 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535589+0000", 2024-09-16T20:49:28.543 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00151085 2024-09-16T20:49:28.543 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.543 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.544 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:28.544 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535639+0000", 2024-09-16T20:49:28.544 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9697000000000002e-05 2024-09-16T20:49:28.544 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.544 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.544 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:28.544 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535675+0000", 2024-09-16T20:49:28.544 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5729999999999998e-05 2024-09-16T20:49:28.544 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.544 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:28.544 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.545 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.545 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.545 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3580 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:28.545 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:49:28.545 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 38.959268618000003, 2024-09-16T20:49:28.545 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054397361999999998, 2024-09-16T20:49:28.545 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:28.545 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:28.545 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:28.545 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:28.546 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:28.546 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:28.546 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3580 2024-09-16T20:49:28.546 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.546 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:28.546 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.546 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:28.546 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:49:28.546 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.546 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.546 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.547 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:28.547 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:49:28.547 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.547 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.547 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.547 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:28.547 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:49:28.547 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2600000000000001e-07 2024-09-16T20:49:28.547 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.547 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.548 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:28.548 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481395+0000", 2024-09-16T20:49:28.548 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.544e-06 2024-09-16T20:49:28.548 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.548 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.548 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:28.548 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481404+0000", 2024-09-16T20:49:28.548 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8850000000000005e-06 2024-09-16T20:49:28.548 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.548 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.549 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:28.549 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481407+0000", 2024-09-16T20:49:28.549 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1540000000000002e-06 2024-09-16T20:49:28.549 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.549 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.549 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:28.549 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531823+0000", 2024-09-16T20:49:28.549 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050415637999999999 2024-09-16T20:49:28.550 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.550 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.550 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:28.550 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531895+0000", 2024-09-16T20:49:28.550 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2179000000000001e-05 2024-09-16T20:49:28.550 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.550 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.550 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:28.550 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532236+0000", 2024-09-16T20:49:28.551 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00034073799999999999 2024-09-16T20:49:28.551 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.551 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.551 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:28.551 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534083+0000", 2024-09-16T20:49:28.551 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001847298 2024-09-16T20:49:28.551 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.551 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.551 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:28.551 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535714+0000", 2024-09-16T20:49:28.551 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0016316810000000001 2024-09-16T20:49:28.552 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.552 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.552 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:28.552 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535759+0000", 2024-09-16T20:49:28.552 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4478999999999999e-05 2024-09-16T20:49:28.552 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.552 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.552 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:28.552 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535791+0000", 2024-09-16T20:49:28.552 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.154e-05 2024-09-16T20:49:28.552 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.553 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:28.553 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.553 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.553 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.553 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3581 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:28.553 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:49:28.553 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 38.959246620999998, 2024-09-16T20:49:28.553 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.057094403000000002, 2024-09-16T20:49:28.553 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:28.553 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:28.554 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:28.554 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:28.554 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:28.554 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:28.554 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3581 2024-09-16T20:49:28.554 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.554 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:28.554 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.554 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:28.554 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:49:28.555 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.555 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.555 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.555 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:28.555 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:49:28.555 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.555 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.555 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.555 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:28.555 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:49:28.555 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8799999999999998e-07 2024-09-16T20:49:28.556 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.556 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.556 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:28.556 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481418+0000", 2024-09-16T20:49:28.556 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1859999999999999e-06 2024-09-16T20:49:28.556 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.556 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.556 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:28.556 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481423+0000", 2024-09-16T20:49:28.556 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8279999999999999e-06 2024-09-16T20:49:28.557 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.557 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.557 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:28.557 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481426+0000", 2024-09-16T20:49:28.557 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0110000000000001e-06 2024-09-16T20:49:28.557 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.557 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.557 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:28.557 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532631+0000", 2024-09-16T20:49:28.557 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051204514999999999 2024-09-16T20:49:28.557 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.558 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.558 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:28.558 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.533381+0000", 2024-09-16T20:49:28.558 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00075029000000000003 2024-09-16T20:49:28.558 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.558 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.558 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:28.558 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.533739+0000", 2024-09-16T20:49:28.558 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00035773100000000002 2024-09-16T20:49:28.558 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.558 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.558 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:28.559 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535123+0000", 2024-09-16T20:49:28.559 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0013836160000000001 2024-09-16T20:49:28.559 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.559 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.559 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:28.559 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538413+0000", 2024-09-16T20:49:28.559 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0032902510000000001 2024-09-16T20:49:28.559 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.559 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.559 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:28.559 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538470+0000", 2024-09-16T20:49:28.560 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6697999999999998e-05 2024-09-16T20:49:28.560 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.560 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.560 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:28.560 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538510+0000", 2024-09-16T20:49:28.560 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9988999999999999e-05 2024-09-16T20:49:28.560 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.560 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:28.560 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.560 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.560 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.561 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:12732 3.d 3:b752444a:::rbd_header.110e747f0779:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-16T20:49:28.561 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:49:28.561 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.672174042999998, 2024-09-16T20:49:28.561 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.065302467000000003, 2024-09-16T20:49:28.561 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:28.561 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:28.561 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:28.561 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:28.561 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:49:28.561 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:49:28.561 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 12732 2024-09-16T20:49:28.561 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.562 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:28.562 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.562 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:28.562 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:49:28.562 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.562 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.562 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.562 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:28.562 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:49:28.562 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.562 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.562 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.562 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:28.563 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768490+0000", 2024-09-16T20:49:28.563 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2859999999999998e-06 2024-09-16T20:49:28.563 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.563 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.563 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:28.563 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768501+0000", 2024-09-16T20:49:28.563 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1226999999999999e-05 2024-09-16T20:49:28.563 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.563 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.563 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:28.563 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768517+0000", 2024-09-16T20:49:28.563 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.592e-05 2024-09-16T20:49:28.564 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.564 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.564 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:28.564 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768528+0000", 2024-09-16T20:49:28.564 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0689999999999999e-05 2024-09-16T20:49:28.564 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.564 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.564 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:28.564 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768584+0000", 2024-09-16T20:49:28.564 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5889e-05 2024-09-16T20:49:28.564 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.565 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.565 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:28.565 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768720+0000", 2024-09-16T20:49:28.565 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000136718 2024-09-16T20:49:28.565 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.565 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.565 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:28.565 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.769066+0000", 2024-09-16T20:49:28.565 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000346013 2024-09-16T20:49:28.565 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.565 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.565 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:28.565 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833659+0000", 2024-09-16T20:49:28.566 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.064592822999999994 2024-09-16T20:49:28.566 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.566 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.566 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:28.566 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833719+0000", 2024-09-16T20:49:28.566 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9441000000000002e-05 2024-09-16T20:49:28.566 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.566 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.566 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:28.566 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833758+0000", 2024-09-16T20:49:28.566 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9122000000000003e-05 2024-09-16T20:49:28.566 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.567 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.567 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:28.567 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833790+0000", 2024-09-16T20:49:28.567 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2338000000000001e-05 2024-09-16T20:49:28.567 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.567 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:28.567 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.567 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.567 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.567 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.5089.0:12732 3.d e73/23 ondisk, result = 0)", 2024-09-16T20:49:28.567 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:49:28.567 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.670571873, 2024-09-16T20:49:28.567 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.063706456999999994, 2024-09-16T20:49:28.568 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:28.568 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:28.568 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:28.568 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:28.568 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.568 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:28.568 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:49:28.568 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.568 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.568 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.568 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:28.569 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:49:28.569 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.569 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.569 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.569 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:28.569 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770091+0000", 2024-09-16T20:49:28.569 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.53e-06 2024-09-16T20:49:28.569 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.569 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.569 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:28.569 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770104+0000", 2024-09-16T20:49:28.569 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2203e-05 2024-09-16T20:49:28.570 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.570 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.570 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:28.570 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833637+0000", 2024-09-16T20:49:28.570 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.063533201999999997 2024-09-16T20:49:28.570 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.570 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.570 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:28.570 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833655+0000", 2024-09-16T20:49:28.570 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8369e-05 2024-09-16T20:49:28.570 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.570 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.570 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:28.571 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833707+0000", 2024-09-16T20:49:28.571 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2061999999999997e-05 2024-09-16T20:49:28.571 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.571 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.571 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:28.571 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833715+0000", 2024-09-16T20:49:28.571 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5179999999999998e-06 2024-09-16T20:49:28.571 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.571 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.571 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:28.571 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833796+0000", 2024-09-16T20:49:28.571 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1573e-05 2024-09-16T20:49:28.571 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.572 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:28.572 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.572 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.572 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:28.572 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:28.573 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:49:28.574 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:28.572+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:28.674 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:28.673+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:28.721 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:28.721 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:49:28.721 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:49:28.721 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:28.729 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:49:28.775 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:28.773+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:28.814 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T20:49:28.837 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:28.837 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:49:28.837 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:49:28.837 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:49:28.838 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:28.847 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:49:28.852 INFO:teuthology.orchestra.run.smithi029.stderr:nodeep-scrub is unset 2024-09-16T20:49:28.869 INFO:teuthology.orchestra.run.smithi139.stdout:3726 punch from 0xfd4800 to 0xfdae00, (0x6600 bytes) 2024-09-16T20:49:28.902 INFO:teuthology.orchestra.run.smithi139.stdout:3729 write 0xe3e6c00 thru 0xe3ef9be (0x8dbf bytes) 2024-09-16T20:49:28.907 INFO:teuthology.orchestra.run.smithi139.stdout:3730 write 0x6b96400 thru 0x6b9b0c5 (0x4cc6 bytes) 2024-09-16T20:49:28.907 INFO:teuthology.orchestra.run.smithi139.stdout:3732 write 0x5608600 thru 0x560ee90 (0x6891 bytes) 2024-09-16T20:49:28.913 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:28.912+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:28.924 INFO:teuthology.orchestra.run.smithi139.stdout:3733 trunc from 0xe3efa00 to 0x2115200 2024-09-16T20:49:28.951 INFO:teuthology.orchestra.run.smithi139.stdout:3735 punch from 0x1918e00 to 0x191b800, (0x2a00 bytes) 2024-09-16T20:49:28.957 INFO:teuthology.orchestra.run.smithi139.stdout:3736 read 0x1dc0a00 thru 0x1dc5927 (0x4f28 bytes) 2024-09-16T20:49:28.957 INFO:teuthology.orchestra.run.smithi139.stdout:3737 read 0x1dcaa00 thru 0x1dce498 (0x3a99 bytes) 2024-09-16T20:49:28.957 INFO:teuthology.orchestra.run.smithi139.stdout:3739 punch from 0xcf1600 to 0xd00c00, (0xf600 bytes) 2024-09-16T20:49:28.962 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:49:28.962 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:49:28.976 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:28.975+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:28.978 INFO:teuthology.orchestra.run.smithi139.stdout:3740 write 0xa3b9800 thru 0xa3c059e (0x6d9f bytes) 2024-09-16T20:49:28.980 INFO:teuthology.orchestra.run.smithi139.stdout:3742 punch from 0x4dfce00 to 0x4e02e00, (0x6000 bytes) 2024-09-16T20:49:28.983 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:28.983 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:49:28.983 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:49:28.983 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:49:28.984 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.984 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3709 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1464320~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:28.984 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.638971+0000", 2024-09-16T20:49:28.984 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 66.342711764000001, 2024-09-16T20:49:28.984 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049858670000000001, 2024-09-16T20:49:28.984 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:28.984 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:28.984 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:28.984 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:28.984 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:28.984 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:28.984 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3709 2024-09-16T20:49:28.984 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.985 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:28.985 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.985 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:28.985 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638971+0000", 2024-09-16T20:49:28.985 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.985 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.985 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.985 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:28.985 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638971+0000", 2024-09-16T20:49:28.985 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.985 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.985 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.985 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:28.985 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638972+0000", 2024-09-16T20:49:28.985 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0400000000000002e-07 2024-09-16T20:49:28.986 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.986 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.986 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:28.986 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638973+0000", 2024-09-16T20:49:28.986 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.578e-06 2024-09-16T20:49:28.986 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.986 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.986 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:28.986 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638979+0000", 2024-09-16T20:49:28.986 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.502e-06 2024-09-16T20:49:28.986 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.986 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.986 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:28.987 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638983+0000", 2024-09-16T20:49:28.987 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2690000000000001e-06 2024-09-16T20:49:28.987 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.987 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.987 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:28.987 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688628+0000", 2024-09-16T20:49:28.987 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049644547999999997 2024-09-16T20:49:28.987 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.987 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.987 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:28.988 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688718+0000", 2024-09-16T20:49:28.988 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0802000000000002e-05 2024-09-16T20:49:28.988 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.988 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.988 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:28.988 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688830+0000", 2024-09-16T20:49:28.988 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000111667 2024-09-16T20:49:28.988 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.988 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:28.988 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.988 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.988 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.988 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3712 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1472512~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:28.989 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:49:28.989 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 66.342695235999997, 2024-09-16T20:49:28.989 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050004305999999998, 2024-09-16T20:49:28.989 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:28.989 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:28.989 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:28.989 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:28.989 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:28.989 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:28.989 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3712 2024-09-16T20:49:28.989 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.989 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:28.989 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.990 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:28.990 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:49:28.990 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.990 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.990 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.990 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:28.990 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:49:28.990 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.990 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.990 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.990 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:28.990 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:49:28.990 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0199999999999998e-07 2024-09-16T20:49:28.990 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.990 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.991 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:28.991 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638990+0000", 2024-09-16T20:49:28.991 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.618e-06 2024-09-16T20:49:28.991 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.991 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.991 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:28.991 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638995+0000", 2024-09-16T20:49:28.991 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3940000000000004e-06 2024-09-16T20:49:28.991 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.991 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.991 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:28.991 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639000+0000", 2024-09-16T20:49:28.991 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9309999999999999e-06 2024-09-16T20:49:28.991 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.992 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.992 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:28.992 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688867+0000", 2024-09-16T20:49:28.992 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049866955999999997 2024-09-16T20:49:28.992 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.992 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.992 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:28.992 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688933+0000", 2024-09-16T20:49:28.992 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5359999999999998e-05 2024-09-16T20:49:28.992 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.992 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.992 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:28.992 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688992+0000", 2024-09-16T20:49:28.992 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9744999999999998e-05 2024-09-16T20:49:28.993 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.993 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:28.993 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.993 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.993 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.993 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3714 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1480704~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:28.993 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:49:28.993 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 66.342677645999999, 2024-09-16T20:49:28.993 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050152552000000003, 2024-09-16T20:49:28.993 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:28.993 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:28.993 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:28.993 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:28.994 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:28.994 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:28.994 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3714 2024-09-16T20:49:28.994 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.994 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:28.994 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.994 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:28.994 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:49:28.994 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.994 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.994 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.994 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:28.994 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:49:28.994 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.995 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.995 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.995 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:28.995 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:49:28.995 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0499999999999999e-07 2024-09-16T20:49:28.995 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.995 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.995 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:28.995 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639007+0000", 2024-09-16T20:49:28.995 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5340000000000001e-06 2024-09-16T20:49:28.995 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.995 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.995 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:28.995 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639013+0000", 2024-09-16T20:49:28.996 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3260000000000003e-06 2024-09-16T20:49:28.996 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.996 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.996 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:28.996 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639018+0000", 2024-09-16T20:49:28.996 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2850000000000002e-06 2024-09-16T20:49:28.996 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.996 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.996 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:28.996 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689026+0000", 2024-09-16T20:49:28.996 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.05000798 2024-09-16T20:49:28.996 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.997 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.997 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:28.997 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689077+0000", 2024-09-16T20:49:28.997 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1063999999999998e-05 2024-09-16T20:49:28.997 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.997 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.997 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:28.997 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689158+0000", 2024-09-16T20:49:28.997 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1057999999999995e-05 2024-09-16T20:49:28.997 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.997 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:28.997 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:28.997 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.998 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.998 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3716 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1488896~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:28.998 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:49:28.998 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 66.342659835999996, 2024-09-16T20:49:28.998 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050280142, 2024-09-16T20:49:28.998 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:28.998 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:28.998 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:28.998 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:28.998 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:28.998 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:28.998 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3716 2024-09-16T20:49:28.998 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.998 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:28.999 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.999 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:28.999 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:49:28.999 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.999 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.999 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.999 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:28.999 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:49:28.999 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:28.999 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:28.999 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:28.999 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:28.999 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639024+0000", 2024-09-16T20:49:29.000 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9999999999999997e-07 2024-09-16T20:49:29.000 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.000 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.000 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:29.000 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639026+0000", 2024-09-16T20:49:29.000 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.781e-06 2024-09-16T20:49:29.000 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.000 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.000 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:29.000 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639032+0000", 2024-09-16T20:49:29.000 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7780000000000004e-06 2024-09-16T20:49:29.000 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.000 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.000 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:29.000 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639036+0000", 2024-09-16T20:49:29.001 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6909999999999996e-06 2024-09-16T20:49:29.001 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.001 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.001 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:29.001 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689191+0000", 2024-09-16T20:49:29.001 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050155284000000001 2024-09-16T20:49:29.001 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.001 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.001 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:29.002 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689242+0000", 2024-09-16T20:49:29.002 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0662999999999999e-05 2024-09-16T20:49:29.002 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.002 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.002 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:29.002 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689304+0000", 2024-09-16T20:49:29.002 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1345000000000003e-05 2024-09-16T20:49:29.002 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:29.002 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:29.002 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:29.003 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.003 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.003 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3719 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1497088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:29.004 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:49:29.004 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 66.342632558000005, 2024-09-16T20:49:29.005 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050399715999999997, 2024-09-16T20:49:29.005 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:29.005 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:29.005 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:29.005 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:29.005 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:29.005 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:29.005 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3719 2024-09-16T20:49:29.005 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.005 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:29.005 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.005 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:29.006 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:49:29.006 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:29.006 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.006 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.006 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:29.006 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:49:29.006 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:29.006 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.006 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.006 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:29.006 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:49:29.006 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9700000000000002e-07 2024-09-16T20:49:29.006 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.006 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.007 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:29.007 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639053+0000", 2024-09-16T20:49:29.007 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2900000000000001e-06 2024-09-16T20:49:29.007 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.007 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.007 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:29.007 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639060+0000", 2024-09-16T20:49:29.007 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7750000000000004e-06 2024-09-16T20:49:29.007 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.007 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.007 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:29.007 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639066+0000", 2024-09-16T20:49:29.008 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0639999999999999e-06 2024-09-16T20:49:29.008 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.008 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.008 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:29.008 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689337+0000", 2024-09-16T20:49:29.008 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050270784999999998 2024-09-16T20:49:29.008 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.008 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.008 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:29.008 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689392+0000", 2024-09-16T20:49:29.008 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4920999999999998e-05 2024-09-16T20:49:29.008 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.008 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.008 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:29.009 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689450+0000", 2024-09-16T20:49:29.009 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8483999999999998e-05 2024-09-16T20:49:29.009 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:29.009 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:29.009 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:29.009 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.009 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.009 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3721 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1505280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:29.009 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:49:29.009 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 66.342609425999996, 2024-09-16T20:49:29.009 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050513608000000002, 2024-09-16T20:49:29.009 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:29.009 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:29.009 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:29.010 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:29.010 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:29.010 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:29.010 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3721 2024-09-16T20:49:29.010 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.010 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:29.010 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.010 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:29.010 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:49:29.010 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:29.010 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.010 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.010 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:29.010 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:49:29.010 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:29.011 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.011 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.011 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:29.011 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:49:29.011 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7e-07 2024-09-16T20:49:29.011 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.011 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.011 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:29.011 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639076+0000", 2024-09-16T20:49:29.011 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8899999999999999e-06 2024-09-16T20:49:29.011 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.011 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.011 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:29.011 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639083+0000", 2024-09-16T20:49:29.011 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1210000000000001e-06 2024-09-16T20:49:29.011 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.012 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.012 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:29.012 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639089+0000", 2024-09-16T20:49:29.012 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.643e-06 2024-09-16T20:49:29.012 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.012 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.012 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:29.012 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689474+0000", 2024-09-16T20:49:29.012 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050385679000000003 2024-09-16T20:49:29.012 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.012 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.012 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:29.012 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689533+0000", 2024-09-16T20:49:29.012 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8303000000000001e-05 2024-09-16T20:49:29.013 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.013 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.013 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:29.013 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689587+0000", 2024-09-16T20:49:29.013 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4601999999999997e-05 2024-09-16T20:49:29.013 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:29.013 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:29.013 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:29.013 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.013 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.013 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3723 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1513472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:29.013 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:49:29.013 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 66.342578622000005, 2024-09-16T20:49:29.013 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050699691999999998, 2024-09-16T20:49:29.014 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:29.014 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:29.014 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:29.014 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:29.014 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:29.014 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:29.014 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3723 2024-09-16T20:49:29.014 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.014 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:29.014 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.014 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:29.014 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:49:29.014 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:29.015 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.015 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.015 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:29.015 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:49:29.015 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:29.015 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.015 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.015 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:29.015 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:49:29.015 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3399999999999999e-07 2024-09-16T20:49:29.015 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.015 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.015 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:29.015 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639107+0000", 2024-09-16T20:49:29.015 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3860000000000001e-06 2024-09-16T20:49:29.016 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.016 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.016 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:29.016 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639114+0000", 2024-09-16T20:49:29.016 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9800000000000001e-06 2024-09-16T20:49:29.016 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.016 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.016 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:29.016 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639129+0000", 2024-09-16T20:49:29.016 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4887e-05 2024-09-16T20:49:29.016 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.016 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.016 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:29.016 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689625+0000", 2024-09-16T20:49:29.016 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050495848000000003 2024-09-16T20:49:29.017 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.017 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.017 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:29.017 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689720+0000", 2024-09-16T20:49:29.017 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4821000000000006e-05 2024-09-16T20:49:29.017 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.017 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.017 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:29.017 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689804+0000", 2024-09-16T20:49:29.017 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4235999999999998e-05 2024-09-16T20:49:29.017 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:29.018 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:29.018 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:29.018 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.018 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.018 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3725 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1521664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:29.018 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:49:29.018 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 66.342545025999996, 2024-09-16T20:49:29.018 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050797660000000001, 2024-09-16T20:49:29.018 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:29.018 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:29.018 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:29.018 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:29.019 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:29.019 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:29.019 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3725 2024-09-16T20:49:29.019 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.019 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:29.019 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.019 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:29.019 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:49:29.019 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:29.019 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.019 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.019 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:29.019 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:49:29.019 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:29.020 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.020 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.020 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:29.020 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639139+0000", 2024-09-16T20:49:29.020 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8999999999999997e-07 2024-09-16T20:49:29.020 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.020 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.020 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:29.020 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639141+0000", 2024-09-16T20:49:29.020 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.182e-06 2024-09-16T20:49:29.020 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.020 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.020 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:29.020 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639154+0000", 2024-09-16T20:49:29.021 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3626e-05 2024-09-16T20:49:29.021 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.021 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.021 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:29.021 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639160+0000", 2024-09-16T20:49:29.021 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6810000000000003e-06 2024-09-16T20:49:29.021 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.021 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.021 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:29.021 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689831+0000", 2024-09-16T20:49:29.021 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050670907000000001 2024-09-16T20:49:29.021 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.021 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.022 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:29.022 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689882+0000", 2024-09-16T20:49:29.022 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0445999999999998e-05 2024-09-16T20:49:29.022 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.022 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.022 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:29.022 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689936+0000", 2024-09-16T20:49:29.022 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4327999999999999e-05 2024-09-16T20:49:29.022 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:29.022 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:29.022 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:29.022 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.022 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.022 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3727 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1529856~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:29.022 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:49:29.023 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 66.342509449000005, 2024-09-16T20:49:29.023 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051016922999999999, 2024-09-16T20:49:29.023 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:29.023 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:29.023 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:29.023 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:29.023 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:29.023 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:29.023 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3727 2024-09-16T20:49:29.023 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.023 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:29.023 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.023 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:29.023 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:49:29.024 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:29.024 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.024 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.024 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:29.024 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:49:29.024 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:29.024 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.024 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.024 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:29.024 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:49:29.024 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9999999999999998e-07 2024-09-16T20:49:29.024 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.024 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.024 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:29.024 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639183+0000", 2024-09-16T20:49:29.025 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2399999999999996e-06 2024-09-16T20:49:29.025 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.025 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.025 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:29.025 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639195+0000", 2024-09-16T20:49:29.025 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1113e-05 2024-09-16T20:49:29.025 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.025 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.025 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:29.025 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639206+0000", 2024-09-16T20:49:29.025 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1503e-05 2024-09-16T20:49:29.026 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.026 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.026 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:29.026 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689969+0000", 2024-09-16T20:49:29.026 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050762676 2024-09-16T20:49:29.026 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.026 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.026 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:29.026 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690071+0000", 2024-09-16T20:49:29.026 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102163 2024-09-16T20:49:29.026 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.026 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.026 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:29.026 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690191+0000", 2024-09-16T20:49:29.026 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000119728 2024-09-16T20:49:29.027 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:29.027 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:29.027 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:29.027 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.027 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.027 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3730 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1538048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:29.027 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:49:29.027 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 66.342467885999994, 2024-09-16T20:49:29.027 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051131467999999999, 2024-09-16T20:49:29.027 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:29.027 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:29.027 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:29.027 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:29.027 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:29.027 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:29.028 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3730 2024-09-16T20:49:29.028 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.028 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:29.028 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.028 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:29.028 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:49:29.028 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:29.028 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.028 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.028 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:29.028 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:49:29.029 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:29.029 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.029 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.029 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:29.029 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639216+0000", 2024-09-16T20:49:29.029 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.39e-07 2024-09-16T20:49:29.029 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.029 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.029 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:29.029 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639218+0000", 2024-09-16T20:49:29.029 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.537e-06 2024-09-16T20:49:29.029 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.029 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.029 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:29.030 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639229+0000", 2024-09-16T20:49:29.030 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1122e-05 2024-09-16T20:49:29.030 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.030 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.030 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:29.030 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639237+0000", 2024-09-16T20:49:29.030 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8509999999999999e-06 2024-09-16T20:49:29.030 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.030 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.030 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:29.030 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690227+0000", 2024-09-16T20:49:29.030 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050990147 2024-09-16T20:49:29.030 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.030 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.030 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:29.031 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690278+0000", 2024-09-16T20:49:29.031 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0223999999999999e-05 2024-09-16T20:49:29.031 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.031 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.031 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:29.031 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690347+0000", 2024-09-16T20:49:29.031 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9147999999999996e-05 2024-09-16T20:49:29.031 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:29.031 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:29.031 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:29.031 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.031 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.031 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3731 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1546240~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:29.031 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:49:29.032 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 66.342437177999997, 2024-09-16T20:49:29.032 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051276744999999999, 2024-09-16T20:49:29.032 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:29.032 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:29.032 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:29.032 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:29.032 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:29.032 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:29.032 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3731 2024-09-16T20:49:29.032 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.032 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:29.032 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.032 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:29.032 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:49:29.033 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:29.033 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.033 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.033 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:29.033 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:49:29.033 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:29.033 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.033 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.033 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:29.033 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:49:29.033 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9900000000000001e-07 2024-09-16T20:49:29.033 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.033 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.034 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:29.034 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639249+0000", 2024-09-16T20:49:29.034 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3630000000000001e-06 2024-09-16T20:49:29.034 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.034 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.034 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:29.034 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639255+0000", 2024-09-16T20:49:29.034 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4960000000000001e-06 2024-09-16T20:49:29.034 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.034 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.034 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:29.034 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639261+0000", 2024-09-16T20:49:29.034 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2839999999999997e-06 2024-09-16T20:49:29.034 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.035 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.035 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:29.035 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690375+0000", 2024-09-16T20:49:29.035 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051114684000000001 2024-09-16T20:49:29.035 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.035 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.035 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:29.035 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690425+0000", 2024-09-16T20:49:29.035 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9379999999999998e-05 2024-09-16T20:49:29.035 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.035 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.035 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:29.035 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690523+0000", 2024-09-16T20:49:29.035 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8139000000000003e-05 2024-09-16T20:49:29.035 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:29.036 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:29.036 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:29.036 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.036 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.036 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3732 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1554432~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:29.036 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:49:29.036 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 66.342391395999996, 2024-09-16T20:49:29.036 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051380096, 2024-09-16T20:49:29.036 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:29.036 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:29.036 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:29.036 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:29.036 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:29.036 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:29.036 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3732 2024-09-16T20:49:29.037 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.037 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:29.037 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.037 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:29.037 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:49:29.037 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:29.037 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.037 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.037 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:29.037 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:49:29.037 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:29.037 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.038 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.038 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:29.038 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:49:29.038 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6100000000000001e-07 2024-09-16T20:49:29.038 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.038 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.038 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:29.038 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639295+0000", 2024-09-16T20:49:29.038 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.447e-06 2024-09-16T20:49:29.038 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.038 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.038 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:29.038 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639302+0000", 2024-09-16T20:49:29.038 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0570000000000004e-06 2024-09-16T20:49:29.039 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.039 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.039 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:29.039 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639307+0000", 2024-09-16T20:49:29.039 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1429999999999997e-06 2024-09-16T20:49:29.039 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.039 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.039 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:29.039 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690555+0000", 2024-09-16T20:49:29.039 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051247963000000001 2024-09-16T20:49:29.039 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.039 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.039 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:29.039 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690602+0000", 2024-09-16T20:49:29.040 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7602000000000003e-05 2024-09-16T20:49:29.040 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.040 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.040 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:29.040 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690672+0000", 2024-09-16T20:49:29.040 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9423000000000003e-05 2024-09-16T20:49:29.040 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:29.040 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:29.040 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:29.040 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.040 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.040 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3734 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1562624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:29.040 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:49:29.040 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 66.342369746000003, 2024-09-16T20:49:29.040 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051490610999999999, 2024-09-16T20:49:29.041 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:29.041 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:29.041 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:29.041 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:29.041 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:29.041 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:29.041 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3734 2024-09-16T20:49:29.041 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.041 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:29.041 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.041 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:29.041 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:49:29.041 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:29.041 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.041 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.041 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:29.042 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:49:29.042 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:29.042 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.042 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.042 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:29.042 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639314+0000", 2024-09-16T20:49:29.042 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4400000000000001e-07 2024-09-16T20:49:29.042 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.042 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.042 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:29.042 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639316+0000", 2024-09-16T20:49:29.042 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1459999999999999e-06 2024-09-16T20:49:29.042 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.043 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.043 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:29.043 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639322+0000", 2024-09-16T20:49:29.043 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7610000000000004e-06 2024-09-16T20:49:29.043 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.043 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.043 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:29.043 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639327+0000", 2024-09-16T20:49:29.043 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.186e-06 2024-09-16T20:49:29.043 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.043 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.043 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:29.043 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690698+0000", 2024-09-16T20:49:29.044 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051371223000000001 2024-09-16T20:49:29.044 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.044 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.044 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:29.044 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690748+0000", 2024-09-16T20:49:29.044 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0192999999999999e-05 2024-09-16T20:49:29.044 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.044 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.044 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:29.044 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690804+0000", 2024-09-16T20:49:29.044 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5658000000000001e-05 2024-09-16T20:49:29.044 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:29.044 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:29.044 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:29.045 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.045 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.045 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3736 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1570816~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:29.045 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:49:29.045 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 66.342350487000004, 2024-09-16T20:49:29.045 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051596528000000003, 2024-09-16T20:49:29.045 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:29.045 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:29.045 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:29.045 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:29.045 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:29.045 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:29.045 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3736 2024-09-16T20:49:29.045 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.045 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:29.045 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.046 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:29.046 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:49:29.046 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:29.046 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.046 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.046 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:29.046 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:49:29.046 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:29.046 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.046 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.046 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:29.046 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:49:29.046 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9999999999999999e-07 2024-09-16T20:49:29.046 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.046 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.047 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:29.047 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639335+0000", 2024-09-16T20:49:29.047 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.635e-06 2024-09-16T20:49:29.047 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.047 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.047 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:29.047 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639340+0000", 2024-09-16T20:49:29.047 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5840000000000002e-06 2024-09-16T20:49:29.047 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.047 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.047 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:29.047 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639346+0000", 2024-09-16T20:49:29.047 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1190000000000002e-06 2024-09-16T20:49:29.048 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.048 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.048 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:29.048 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690817+0000", 2024-09-16T20:49:29.048 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051470721999999997 2024-09-16T20:49:29.048 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.048 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.048 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:29.048 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690875+0000", 2024-09-16T20:49:29.048 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7441000000000001e-05 2024-09-16T20:49:29.048 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.048 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.049 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:29.049 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690929+0000", 2024-09-16T20:49:29.049 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4727e-05 2024-09-16T20:49:29.049 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:29.049 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:29.049 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:29.049 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.049 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.049 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5434 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:29.049 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:49:29.049 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 66.117088921000004, 2024-09-16T20:49:29.049 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099755831000000003, 2024-09-16T20:49:29.049 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:29.049 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:29.050 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:29.050 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:29.050 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:29.050 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:29.050 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5434 2024-09-16T20:49:29.050 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.050 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:29.050 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.050 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:29.050 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:49:29.050 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:29.050 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.050 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.050 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:29.050 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:49:29.051 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:29.051 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.051 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.051 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:29.051 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864595+0000", 2024-09-16T20:49:29.051 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1099999999999995e-07 2024-09-16T20:49:29.051 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.051 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.051 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:29.051 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864597+0000", 2024-09-16T20:49:29.051 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3989999999999998e-06 2024-09-16T20:49:29.051 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.051 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.051 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:29.051 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864603+0000", 2024-09-16T20:49:29.052 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9309999999999996e-06 2024-09-16T20:49:29.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.052 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.052 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:29.052 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864607+0000", 2024-09-16T20:49:29.052 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6200000000000001e-06 2024-09-16T20:49:29.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.052 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.052 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:29.052 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964164+0000", 2024-09-16T20:49:29.052 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099557162000000005 2024-09-16T20:49:29.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.052 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.052 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:29.052 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964244+0000", 2024-09-16T20:49:29.053 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9597000000000004e-05 2024-09-16T20:49:29.053 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.053 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.053 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:29.053 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964350+0000", 2024-09-16T20:49:29.053 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010641099999999999 2024-09-16T20:49:29.053 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:29.053 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:29.053 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:29.053 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.053 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.053 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5435 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 20480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:29.054 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:49:29.054 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 66.117059099000002, 2024-09-16T20:49:29.054 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100897102, 2024-09-16T20:49:29.054 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:29.054 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:29.054 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:29.054 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:29.054 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:29.054 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:29.054 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5435 2024-09-16T20:49:29.054 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.054 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:29.054 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.054 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:29.055 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:49:29.055 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:29.055 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.055 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.055 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:29.055 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:49:29.055 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:29.055 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.055 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.055 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:29.055 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:49:29.055 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8000000000000001e-07 2024-09-16T20:49:29.055 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.056 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.056 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:29.056 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864626+0000", 2024-09-16T20:49:29.056 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.745e-06 2024-09-16T20:49:29.056 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.056 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.056 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:29.056 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864633+0000", 2024-09-16T20:49:29.056 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3339999999999998e-06 2024-09-16T20:49:29.056 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.056 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.056 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:29.056 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864636+0000", 2024-09-16T20:49:29.057 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1200000000000002e-06 2024-09-16T20:49:29.057 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.057 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.057 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:29.057 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965299+0000", 2024-09-16T20:49:29.057 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10066352000000001 2024-09-16T20:49:29.057 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.057 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.057 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:29.057 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965390+0000", 2024-09-16T20:49:29.057 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0894000000000001e-05 2024-09-16T20:49:29.057 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.057 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.057 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:29.058 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965521+0000", 2024-09-16T20:49:29.058 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013110899999999999 2024-09-16T20:49:29.058 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:29.058 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:29.058 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:29.058 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.058 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.058 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:152 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 327680~32768 out=32768b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:49:29.058 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:49:29.058 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 64.818562157000002, 2024-09-16T20:49:29.058 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11869405800000001, 2024-09-16T20:49:29.058 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:29.058 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:29.058 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:29.058 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:29.059 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:49:29.059 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:49:29.059 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 152 2024-09-16T20:49:29.059 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.059 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:29.059 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.059 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:29.059 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:49:29.059 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:29.060 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.060 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.060 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:29.060 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:49:29.060 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:29.060 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.060 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.060 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:29.060 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163122+0000", 2024-09-16T20:49:29.060 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6400000000000003e-07 2024-09-16T20:49:29.060 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.060 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.061 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:29.061 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163124+0000", 2024-09-16T20:49:29.061 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.565e-06 2024-09-16T20:49:29.061 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.061 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.061 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:29.061 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163136+0000", 2024-09-16T20:49:29.061 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1717e-05 2024-09-16T20:49:29.061 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.061 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.061 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:29.062 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163142+0000", 2024-09-16T20:49:29.062 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1750000000000002e-06 2024-09-16T20:49:29.062 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.062 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.062 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:29.062 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163606+0000", 2024-09-16T20:49:29.062 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00046418799999999998 2024-09-16T20:49:29.062 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.062 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.062 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:29.062 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163656+0000", 2024-09-16T20:49:29.062 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9852999999999999e-05 2024-09-16T20:49:29.062 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.062 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.063 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:29.063 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281815+0000", 2024-09-16T20:49:29.063 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11815909600000001 2024-09-16T20:49:29.063 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:29.063 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:29.063 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:29.063 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.063 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.063 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:153 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:49:29.063 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:49:29.063 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 64.818529204000001, 2024-09-16T20:49:29.063 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11903462400000001, 2024-09-16T20:49:29.063 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:29.064 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:29.064 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:29.064 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:29.064 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:49:29.064 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:49:29.064 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 153 2024-09-16T20:49:29.064 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.064 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:29.064 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.064 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:29.064 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:49:29.064 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:29.065 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.065 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.065 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:29.065 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:49:29.065 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:29.065 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.065 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.065 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:29.065 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163155+0000", 2024-09-16T20:49:29.065 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.37e-07 2024-09-16T20:49:29.065 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.065 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.065 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:29.066 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163157+0000", 2024-09-16T20:49:29.066 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6010000000000002e-06 2024-09-16T20:49:29.066 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.066 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.066 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:29.066 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163165+0000", 2024-09-16T20:49:29.066 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2619999999999998e-06 2024-09-16T20:49:29.066 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.066 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.066 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:29.066 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163171+0000", 2024-09-16T20:49:29.066 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.711e-06 2024-09-16T20:49:29.066 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.067 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.067 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:29.067 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281868+0000", 2024-09-16T20:49:29.067 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118696622 2024-09-16T20:49:29.067 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.067 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.067 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:29.067 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281969+0000", 2024-09-16T20:49:29.067 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101371 2024-09-16T20:49:29.067 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.067 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.067 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:29.067 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282189+0000", 2024-09-16T20:49:29.067 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00021952000000000001 2024-09-16T20:49:29.068 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:29.068 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:29.068 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:29.068 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.068 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.068 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:154 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 360448~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:49:29.068 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:49:29.068 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 64.818497432000001, 2024-09-16T20:49:29.068 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11932035100000001, 2024-09-16T20:49:29.068 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:29.068 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:29.068 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:29.069 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:29.069 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:49:29.069 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:49:29.069 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 154 2024-09-16T20:49:29.069 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.069 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:29.069 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.069 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:29.069 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:49:29.069 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:29.069 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.069 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.070 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:29.070 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:49:29.070 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:29.070 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.070 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.070 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:29.070 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:49:29.070 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:49:29.070 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.070 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.070 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:29.070 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163189+0000", 2024-09-16T20:49:29.070 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6840000000000001e-06 2024-09-16T20:49:29.070 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.071 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.071 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:29.071 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163196+0000", 2024-09-16T20:49:29.071 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7920000000000004e-06 2024-09-16T20:49:29.071 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.071 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.071 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:29.071 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163201+0000", 2024-09-16T20:49:29.071 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2809999999999999e-06 2024-09-16T20:49:29.071 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.071 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.071 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:29.071 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282219+0000", 2024-09-16T20:49:29.071 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.119018343 2024-09-16T20:49:29.072 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.072 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.072 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:29.072 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282305+0000", 2024-09-16T20:49:29.072 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5865000000000005e-05 2024-09-16T20:49:29.072 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.072 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.072 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:29.072 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282506+0000", 2024-09-16T20:49:29.072 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000200927 2024-09-16T20:49:29.072 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:29.072 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:29.072 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:29.072 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.073 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.073 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop(client.4366.0:3088 3.1e e38/23 3:79e5ec01:::rbd_header.110e891a9b2b:head v 38'306, mlcod=36'304)", 2024-09-16T20:49:29.073 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:49:29.073 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 64.817478563999998, 2024-09-16T20:49:29.073 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118102759, 2024-09-16T20:49:29.073 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:29.073 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:29.073 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:29.073 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:29.073 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.073 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:29.073 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:49:29.073 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:29.074 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.074 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.074 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:29.074 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:49:29.074 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:29.074 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.074 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.074 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:29.074 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:49:29.074 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.85e-07 2024-09-16T20:49:29.074 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.074 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.074 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:29.074 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164212+0000", 2024-09-16T20:49:29.075 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3999999999999997e-06 2024-09-16T20:49:29.075 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.075 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.075 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:29.075 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281553+0000", 2024-09-16T20:49:29.075 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11734135499999999 2024-09-16T20:49:29.075 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.075 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.075 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:29.075 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281567+0000", 2024-09-16T20:49:29.075 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4188999999999999e-05 2024-09-16T20:49:29.075 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.075 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.075 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:29.076 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281653+0000", 2024-09-16T20:49:29.076 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5385999999999999e-05 2024-09-16T20:49:29.076 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.076 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.076 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:29.076 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281679+0000", 2024-09-16T20:49:29.076 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6689000000000001e-05 2024-09-16T20:49:29.076 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.076 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.076 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:29.076 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282277+0000", 2024-09-16T20:49:29.076 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00059798000000000002 2024-09-16T20:49:29.076 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:29.076 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:29.076 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:29.077 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282307+0000", 2024-09-16T20:49:29.077 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9975e-05 2024-09-16T20:49:29.077 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:29.077 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:29.077 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:29.077 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:29.077 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:29.077 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:29.078 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:49:29.079 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:49:29.080 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:49:29.080 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:49:29.081 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:49:29.081 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:49:29.082 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:49:29.082 INFO:teuthology.orchestra.run.smithi139.stdout:3743 punch from 0x123c400 to 0x124a400, (0xe000 bytes) 2024-09-16T20:49:29.082 INFO:teuthology.orchestra.run.smithi139.stdout:3744 read 0x98d2600 thru 0x98dcc75 (0xa676 bytes) 2024-09-16T20:49:29.082 INFO:teuthology.orchestra.run.smithi139.stdout:3747 write 0x9c22400 thru 0x9c30949 (0xe54a bytes) 2024-09-16T20:49:29.082 INFO:teuthology.orchestra.run.smithi139.stdout:3749 write 0x5e4da00 thru 0x5e5bbf6 (0xe1f7 bytes) 2024-09-16T20:49:29.082 INFO:teuthology.orchestra.run.smithi139.stdout:3750 write 0x796ac00 thru 0x7974424 (0x9825 bytes) 2024-09-16T20:49:29.082 INFO:teuthology.orchestra.run.smithi139.stdout:3751 read 0x6569a00 thru 0x656a284 (0x885 bytes) 2024-09-16T20:49:29.082 INFO:teuthology.orchestra.run.smithi139.stdout:3752 punch from 0x9581e00 to 0x958d400, (0xb600 bytes) 2024-09-16T20:49:29.082 INFO:teuthology.orchestra.run.smithi139.stdout:3753 read 0x4962200 thru 0x4966b74 (0x4975 bytes) 2024-09-16T20:49:29.082 INFO:teuthology.orchestra.run.smithi139.stdout:3754 write 0xd897e00 thru 0xd8a262f (0xa830 bytes) 2024-09-16T20:49:29.083 INFO:teuthology.orchestra.run.smithi139.stdout:3755 read 0xb055800 thru 0xb05d0c9 (0x78ca bytes) 2024-09-16T20:49:29.083 INFO:teuthology.orchestra.run.smithi139.stdout:3756 trunc from 0xd8a2800 to 0xb9f4e00 2024-09-16T20:49:29.083 INFO:teuthology.orchestra.run.smithi139.stdout:3757 read 0x7752800 thru 0x7755b54 (0x3355 bytes) 2024-09-16T20:49:29.083 INFO:teuthology.orchestra.run.smithi139.stdout:3758 read 0xb6d000 thru 0xb77946 (0xa947 bytes) 2024-09-16T20:49:29.083 INFO:teuthology.orchestra.run.smithi139.stdout:3759 write 0x1b68800 thru 0x1b6947a (0xc7b bytes) 2024-09-16T20:49:29.083 INFO:teuthology.orchestra.run.smithi139.stdout:3760 clone 27 order 19 su 4096 sc 8 2024-09-16T20:49:29.083 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:49:29.083 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:49:29.084 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:49:29.084 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:49:29.085 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:29.077+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:29.179 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:29.177+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:29.192 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:29.192 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:29.202 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:49:29.286 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:29.285+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:29.350 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:29.350 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:29.359 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:49:29.400 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:29.398+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:29.482 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:29.481+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:29.506 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:29.506 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:29.515 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:49:29.583 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:29.582+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:29.626 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:29.627 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:29.636 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:49:29.650 INFO:teuthology.orchestra.run.smithi139.stdout:truncating image image_client.0-clone26 from 0xb9f4e00 (overlap 0x237e00) to 0x18d800 2024-09-16T20:49:29.684 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:29.683+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:29.784 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:29.784 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:29.785 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:29.783+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:29.792 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:49:29.886 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:29.885+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:29.939 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:29.940 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:29.948 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:49:29.993 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:29.992+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:30.056 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:30.056 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:30.064 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:49:30.088 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:30.087+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:30.189 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:30.188+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:30.213 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:30.213 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:30.221 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:49:30.291 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:30.289+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:30.369 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:30.369 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:30.377 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:49:30.391 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:30.390+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:30.492 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:30.491+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:30.497 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:30.498 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:30.507 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:49:30.593 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:30.591+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:30.618 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:30.618 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:30.627 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:49:30.693 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:30.692+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:30.734 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:30.735 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:30.744 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:49:30.793 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:30.792+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:30.853 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:30.853 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:30.863 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:49:30.873 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T20:49:30.894 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:30.893+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:30.973 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:30.973 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:30.982 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:49:30.995 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:30.994+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:31.085 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T20:49:31.120 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:31.120 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:31.120 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:31.119+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:31.130 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:49:31.196 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #25, image image_client.0-clone25 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent26 2024-09-16T20:49:31.197 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:31.195+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:31.224 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:49:31.224 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:49:31.242 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:49:31.243 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:49:31.243 INFO:teuthology.orchestra.run.smithi139.stdout:3761 trunc from 0xb9f4e00 to 0xa0ada00 2024-09-16T20:49:31.268 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:49:31.268 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:49:31.292 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:49:31.292 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:49:31.293 INFO:teuthology.orchestra.run.smithi139.stdout:3763 read 0x9ef9600 thru 0x9f05ee7 (0xc8e8 bytes) 2024-09-16T20:49:31.294 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:31.294 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:49:31.294 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:49:31.295 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:31.297 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:31.296+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:31.302 INFO:teuthology.orchestra.run.smithi139.stdout:3768 trunc from 0xa0ada00 to 0x4cc2400 2024-09-16T20:49:31.304 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:49:31.314 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:49:31.314 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:49:31.336 INFO:teuthology.orchestra.run.smithi139.stdout:3769 read 0x67fc00 thru 0x681dba (0x21bb bytes) 2024-09-16T20:49:31.339 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:49:31.339 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:49:31.342 INFO:teuthology.orchestra.run.smithi139.stdout:3772 punch from 0x2f60200 to 0x2f67a00, (0x7800 bytes) 2024-09-16T20:49:31.345 INFO:teuthology.orchestra.run.smithi139.stdout:3774 write 0xdcc200 thru 0xdccf09 (0xd0a bytes) 2024-09-16T20:49:31.378 INFO:teuthology.orchestra.run.smithi139.stdout:3775 write 0xa364a00 thru 0xa3676b5 (0x2cb6 bytes) 2024-09-16T20:49:31.398 INFO:teuthology.orchestra.run.smithi139.stdout:3776 write 0xd898a00 thru 0xd8a5077 (0xc678 bytes) 2024-09-16T20:49:31.398 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:31.396+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:31.404 INFO:teuthology.orchestra.run.smithi139.stdout:3778 write 0x97df600 thru 0x97e5bd9 (0x65da bytes) 2024-09-16T20:49:31.404 INFO:teuthology.orchestra.run.smithi139.stdout:3780 clone 28 order 20 su 16384 sc 2 2024-09-16T20:49:31.452 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:31.452 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:49:31.452 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:49:31.452 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:49:31.453 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:31.461 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:49:31.499 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:31.498+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:31.571 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:31.571 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:49:31.571 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:49:31.571 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:49:31.571 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.571 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3709 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1464320~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:31.571 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.638971+0000", 2024-09-16T20:49:31.571 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 68.929715129000002, 2024-09-16T20:49:31.571 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049858670000000001, 2024-09-16T20:49:31.572 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:31.572 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:31.572 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:31.572 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:31.572 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:31.572 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:31.572 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3709 2024-09-16T20:49:31.572 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.572 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:31.572 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.572 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:31.573 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638971+0000", 2024-09-16T20:49:31.573 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:31.573 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.573 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.573 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:31.573 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638971+0000", 2024-09-16T20:49:31.573 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:31.573 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.573 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.573 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:31.573 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638972+0000", 2024-09-16T20:49:31.573 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0400000000000002e-07 2024-09-16T20:49:31.573 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.574 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.574 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:31.574 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638973+0000", 2024-09-16T20:49:31.574 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.578e-06 2024-09-16T20:49:31.574 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.574 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.574 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:31.574 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638979+0000", 2024-09-16T20:49:31.574 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.502e-06 2024-09-16T20:49:31.574 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.574 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.574 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:31.574 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638983+0000", 2024-09-16T20:49:31.575 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2690000000000001e-06 2024-09-16T20:49:31.575 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.575 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.575 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:31.575 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688628+0000", 2024-09-16T20:49:31.575 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049644547999999997 2024-09-16T20:49:31.575 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.575 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.575 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:31.575 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688718+0000", 2024-09-16T20:49:31.576 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0802000000000002e-05 2024-09-16T20:49:31.576 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.576 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.576 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:31.576 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688830+0000", 2024-09-16T20:49:31.576 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000111667 2024-09-16T20:49:31.576 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:31.576 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:31.576 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:31.576 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.577 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.577 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3712 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1472512~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:31.577 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:49:31.577 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 68.929698600999998, 2024-09-16T20:49:31.577 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050004305999999998, 2024-09-16T20:49:31.577 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:31.577 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:31.577 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:31.577 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:31.577 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:31.577 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:31.577 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3712 2024-09-16T20:49:31.578 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.578 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:31.578 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.578 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:31.578 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:49:31.578 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:31.578 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.578 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.578 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:31.578 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:49:31.578 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:31.578 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.578 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.578 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:31.579 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:49:31.579 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0199999999999998e-07 2024-09-16T20:49:31.579 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.579 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.579 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:31.579 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638990+0000", 2024-09-16T20:49:31.579 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.618e-06 2024-09-16T20:49:31.579 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.579 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.579 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:31.579 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638995+0000", 2024-09-16T20:49:31.579 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3940000000000004e-06 2024-09-16T20:49:31.579 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.579 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.580 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:31.580 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639000+0000", 2024-09-16T20:49:31.580 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9309999999999999e-06 2024-09-16T20:49:31.580 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.580 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.580 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:31.580 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688867+0000", 2024-09-16T20:49:31.580 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049866955999999997 2024-09-16T20:49:31.580 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.580 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.580 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:31.580 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688933+0000", 2024-09-16T20:49:31.580 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5359999999999998e-05 2024-09-16T20:49:31.580 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.581 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.581 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:31.581 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688992+0000", 2024-09-16T20:49:31.581 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9744999999999998e-05 2024-09-16T20:49:31.581 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:31.581 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:31.581 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:31.581 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.581 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.581 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3714 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1480704~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:31.581 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:49:31.581 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 68.929681011, 2024-09-16T20:49:31.581 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050152552000000003, 2024-09-16T20:49:31.581 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:31.582 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:31.582 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:31.582 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:31.582 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:31.582 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:31.582 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3714 2024-09-16T20:49:31.582 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.582 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:31.582 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.582 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:31.582 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:49:31.582 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:31.582 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.583 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.583 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:31.583 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:49:31.583 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:31.583 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.583 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.583 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:31.583 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:49:31.583 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0499999999999999e-07 2024-09-16T20:49:31.583 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.583 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.583 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:31.583 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639007+0000", 2024-09-16T20:49:31.583 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5340000000000001e-06 2024-09-16T20:49:31.584 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.584 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.584 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:31.584 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639013+0000", 2024-09-16T20:49:31.584 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3260000000000003e-06 2024-09-16T20:49:31.584 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.584 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.584 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:31.584 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639018+0000", 2024-09-16T20:49:31.584 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2850000000000002e-06 2024-09-16T20:49:31.584 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.585 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.585 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:31.585 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689026+0000", 2024-09-16T20:49:31.585 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.05000798 2024-09-16T20:49:31.585 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.585 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.585 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:31.585 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689077+0000", 2024-09-16T20:49:31.585 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1063999999999998e-05 2024-09-16T20:49:31.585 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.585 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.585 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:31.585 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689158+0000", 2024-09-16T20:49:31.585 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1057999999999995e-05 2024-09-16T20:49:31.586 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:31.586 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:31.586 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:31.586 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.586 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.586 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3716 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1488896~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:31.586 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:49:31.586 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 68.929663200999997, 2024-09-16T20:49:31.586 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050280142, 2024-09-16T20:49:31.586 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:31.586 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:31.586 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:31.586 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:31.587 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:31.587 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:31.587 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3716 2024-09-16T20:49:31.587 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.587 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:31.587 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.587 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:31.588 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:49:31.588 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:31.588 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.588 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.588 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:31.588 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:49:31.589 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:31.589 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.589 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.589 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:31.589 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639024+0000", 2024-09-16T20:49:31.589 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9999999999999997e-07 2024-09-16T20:49:31.589 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.590 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.590 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:31.590 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639026+0000", 2024-09-16T20:49:31.590 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.781e-06 2024-09-16T20:49:31.590 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.590 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.590 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:31.591 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639032+0000", 2024-09-16T20:49:31.591 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7780000000000004e-06 2024-09-16T20:49:31.591 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.591 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.591 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:31.591 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639036+0000", 2024-09-16T20:49:31.591 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6909999999999996e-06 2024-09-16T20:49:31.591 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.592 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.592 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:31.592 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689191+0000", 2024-09-16T20:49:31.592 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050155284000000001 2024-09-16T20:49:31.592 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.592 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.593 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:31.593 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689242+0000", 2024-09-16T20:49:31.593 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0662999999999999e-05 2024-09-16T20:49:31.593 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.593 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.593 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:31.594 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689304+0000", 2024-09-16T20:49:31.594 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1345000000000003e-05 2024-09-16T20:49:31.594 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:31.594 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:31.594 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:31.594 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.594 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.594 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3719 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1497088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:31.595 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:49:31.595 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 68.929635923000006, 2024-09-16T20:49:31.595 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050399715999999997, 2024-09-16T20:49:31.595 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:31.595 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:31.595 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:31.595 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:31.595 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:31.595 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:31.595 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3719 2024-09-16T20:49:31.595 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.595 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:31.595 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.596 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:31.596 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:49:31.596 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:31.596 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.596 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.596 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:31.596 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:49:31.596 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:31.596 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.596 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.596 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:31.596 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:49:31.596 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9700000000000002e-07 2024-09-16T20:49:31.597 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.597 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.597 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:31.597 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639053+0000", 2024-09-16T20:49:31.597 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2900000000000001e-06 2024-09-16T20:49:31.597 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.597 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.597 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:31.597 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639060+0000", 2024-09-16T20:49:31.597 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7750000000000004e-06 2024-09-16T20:49:31.597 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.597 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.597 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:31.598 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639066+0000", 2024-09-16T20:49:31.598 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0639999999999999e-06 2024-09-16T20:49:31.598 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.598 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.598 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:31.598 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689337+0000", 2024-09-16T20:49:31.598 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050270784999999998 2024-09-16T20:49:31.598 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.598 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.598 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:31.598 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689392+0000", 2024-09-16T20:49:31.598 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4920999999999998e-05 2024-09-16T20:49:31.598 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.599 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.599 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:31.599 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689450+0000", 2024-09-16T20:49:31.599 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8483999999999998e-05 2024-09-16T20:49:31.599 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:31.599 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:31.599 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:31.599 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.599 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.599 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3721 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1505280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:31.599 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:49:31.599 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 68.929612790999997, 2024-09-16T20:49:31.599 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050513608000000002, 2024-09-16T20:49:31.600 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:31.600 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:31.600 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:31.600 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:31.600 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:31.600 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:31.600 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3721 2024-09-16T20:49:31.600 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.600 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:31.600 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.600 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:31.600 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:49:31.601 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:31.601 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.601 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.601 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:31.601 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:49:31.601 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:31.601 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.601 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.601 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:31.601 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:49:31.601 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7e-07 2024-09-16T20:49:31.601 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.602 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.602 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:31.602 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639076+0000", 2024-09-16T20:49:31.602 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8899999999999999e-06 2024-09-16T20:49:31.602 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.602 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.602 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:31.602 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639083+0000", 2024-09-16T20:49:31.602 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1210000000000001e-06 2024-09-16T20:49:31.602 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.602 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.602 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:31.602 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639089+0000", 2024-09-16T20:49:31.603 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.643e-06 2024-09-16T20:49:31.603 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.603 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.603 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:31.603 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689474+0000", 2024-09-16T20:49:31.603 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050385679000000003 2024-09-16T20:49:31.603 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.603 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.603 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:31.603 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689533+0000", 2024-09-16T20:49:31.603 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8303000000000001e-05 2024-09-16T20:49:31.604 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.604 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.604 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:31.604 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689587+0000", 2024-09-16T20:49:31.604 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4601999999999997e-05 2024-09-16T20:49:31.604 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:31.604 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:31.604 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:31.604 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.604 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.604 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3723 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1513472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:31.605 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:49:31.605 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 68.929581987000006, 2024-09-16T20:49:31.605 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050699691999999998, 2024-09-16T20:49:31.605 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:31.605 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:31.605 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:31.605 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:31.605 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:31.605 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:31.605 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3723 2024-09-16T20:49:31.605 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.605 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:31.606 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.606 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:31.606 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:49:31.606 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:31.606 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.606 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.606 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:31.606 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:49:31.606 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:31.606 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.606 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.606 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:31.606 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:49:31.606 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3399999999999999e-07 2024-09-16T20:49:31.606 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.607 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.607 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:31.607 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639107+0000", 2024-09-16T20:49:31.607 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3860000000000001e-06 2024-09-16T20:49:31.607 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.607 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.607 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:31.607 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639114+0000", 2024-09-16T20:49:31.607 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9800000000000001e-06 2024-09-16T20:49:31.607 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.607 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.607 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:31.607 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639129+0000", 2024-09-16T20:49:31.607 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4887e-05 2024-09-16T20:49:31.607 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.608 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.608 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:31.608 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689625+0000", 2024-09-16T20:49:31.608 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050495848000000003 2024-09-16T20:49:31.608 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.608 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.608 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:31.608 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689720+0000", 2024-09-16T20:49:31.608 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4821000000000006e-05 2024-09-16T20:49:31.608 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.608 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.608 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:31.608 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689804+0000", 2024-09-16T20:49:31.609 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4235999999999998e-05 2024-09-16T20:49:31.609 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:31.609 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:31.609 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:31.609 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.609 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.609 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3725 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1521664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:31.609 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:49:31.609 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 68.929548390999997, 2024-09-16T20:49:31.609 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050797660000000001, 2024-09-16T20:49:31.609 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:31.609 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:31.609 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:31.610 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:31.610 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:31.610 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:31.610 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3725 2024-09-16T20:49:31.610 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.610 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:31.610 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.610 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:31.610 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:49:31.610 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:31.610 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.610 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.611 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:31.611 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:49:31.611 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:31.611 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.612 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.612 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:31.612 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639139+0000", 2024-09-16T20:49:31.612 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8999999999999997e-07 2024-09-16T20:49:31.612 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.612 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.612 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:31.612 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639141+0000", 2024-09-16T20:49:31.612 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.182e-06 2024-09-16T20:49:31.613 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.613 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.613 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:31.613 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639154+0000", 2024-09-16T20:49:31.613 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3626e-05 2024-09-16T20:49:31.613 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.613 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.613 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:31.613 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639160+0000", 2024-09-16T20:49:31.613 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6810000000000003e-06 2024-09-16T20:49:31.613 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.613 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.613 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:31.614 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689831+0000", 2024-09-16T20:49:31.614 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050670907000000001 2024-09-16T20:49:31.614 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.614 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.614 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:31.614 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689882+0000", 2024-09-16T20:49:31.614 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0445999999999998e-05 2024-09-16T20:49:31.614 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.614 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.614 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:31.614 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689936+0000", 2024-09-16T20:49:31.614 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4327999999999999e-05 2024-09-16T20:49:31.614 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:31.614 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:31.615 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:31.615 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.615 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.615 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3727 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1529856~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:31.615 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:49:31.615 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 68.929512814000006, 2024-09-16T20:49:31.615 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051016922999999999, 2024-09-16T20:49:31.615 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:31.615 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:31.615 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:31.615 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:31.615 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:31.615 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:31.615 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3727 2024-09-16T20:49:31.616 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.616 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:31.616 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.616 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:31.616 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:49:31.616 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:31.616 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.616 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.616 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:31.616 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:49:31.616 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:31.616 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.616 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.616 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:31.617 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:49:31.617 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9999999999999998e-07 2024-09-16T20:49:31.617 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.617 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.617 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:31.617 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639183+0000", 2024-09-16T20:49:31.617 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2399999999999996e-06 2024-09-16T20:49:31.617 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.617 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.617 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:31.617 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639195+0000", 2024-09-16T20:49:31.617 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1113e-05 2024-09-16T20:49:31.617 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.617 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.618 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:31.618 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639206+0000", 2024-09-16T20:49:31.618 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1503e-05 2024-09-16T20:49:31.618 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.618 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.618 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:31.618 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689969+0000", 2024-09-16T20:49:31.618 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050762676 2024-09-16T20:49:31.618 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.618 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.618 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:31.618 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690071+0000", 2024-09-16T20:49:31.618 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102163 2024-09-16T20:49:31.618 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.618 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.619 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:31.619 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690191+0000", 2024-09-16T20:49:31.619 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000119728 2024-09-16T20:49:31.619 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:31.619 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:31.619 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:31.619 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.619 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.619 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3730 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1538048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:31.619 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:49:31.619 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 68.929471250999995, 2024-09-16T20:49:31.619 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051131467999999999, 2024-09-16T20:49:31.619 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:31.619 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:31.619 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:31.619 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:31.620 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:31.620 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:31.620 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3730 2024-09-16T20:49:31.620 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.620 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:31.620 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.620 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:31.620 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:49:31.620 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:31.620 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.620 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.620 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:31.621 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:49:31.621 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:31.621 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.621 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.621 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:31.621 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639216+0000", 2024-09-16T20:49:31.621 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.39e-07 2024-09-16T20:49:31.621 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.621 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.621 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:31.621 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639218+0000", 2024-09-16T20:49:31.621 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.537e-06 2024-09-16T20:49:31.621 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.621 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.622 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:31.622 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639229+0000", 2024-09-16T20:49:31.622 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1122e-05 2024-09-16T20:49:31.622 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.622 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.622 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:31.622 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639237+0000", 2024-09-16T20:49:31.622 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8509999999999999e-06 2024-09-16T20:49:31.622 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.622 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.622 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:31.622 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690227+0000", 2024-09-16T20:49:31.622 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050990147 2024-09-16T20:49:31.622 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.623 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.623 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:31.623 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690278+0000", 2024-09-16T20:49:31.623 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0223999999999999e-05 2024-09-16T20:49:31.623 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.623 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.623 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:31.623 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690347+0000", 2024-09-16T20:49:31.623 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9147999999999996e-05 2024-09-16T20:49:31.623 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:31.623 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:31.623 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:31.623 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.623 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.623 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3731 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1546240~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:31.624 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:49:31.624 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 68.929440542999998, 2024-09-16T20:49:31.624 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051276744999999999, 2024-09-16T20:49:31.624 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:31.624 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:31.624 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:31.624 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:31.624 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:31.624 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:31.624 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3731 2024-09-16T20:49:31.624 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.624 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:31.625 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.625 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:31.625 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:49:31.625 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:31.625 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.625 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.625 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:31.625 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:49:31.625 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:31.625 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.626 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.626 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:31.626 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:49:31.626 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9900000000000001e-07 2024-09-16T20:49:31.626 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.626 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.626 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:31.626 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639249+0000", 2024-09-16T20:49:31.626 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3630000000000001e-06 2024-09-16T20:49:31.626 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.626 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.626 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:31.626 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639255+0000", 2024-09-16T20:49:31.627 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4960000000000001e-06 2024-09-16T20:49:31.627 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.627 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.627 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:31.627 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639261+0000", 2024-09-16T20:49:31.627 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2839999999999997e-06 2024-09-16T20:49:31.627 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.627 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.627 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:31.627 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690375+0000", 2024-09-16T20:49:31.627 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051114684000000001 2024-09-16T20:49:31.627 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.627 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.628 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:31.628 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690425+0000", 2024-09-16T20:49:31.628 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9379999999999998e-05 2024-09-16T20:49:31.628 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.628 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.628 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:31.628 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690523+0000", 2024-09-16T20:49:31.628 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8139000000000003e-05 2024-09-16T20:49:31.629 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:31.629 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:31.629 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:31.629 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.629 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.629 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3732 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1554432~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:31.629 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:49:31.629 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 68.929394760999998, 2024-09-16T20:49:31.629 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051380096, 2024-09-16T20:49:31.630 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:31.630 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:31.630 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:31.630 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:31.630 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:31.630 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:31.630 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3732 2024-09-16T20:49:31.630 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.630 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:31.630 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.630 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:31.630 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:49:31.630 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:31.631 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.631 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.631 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:31.631 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:49:31.631 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:31.631 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.631 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.631 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:31.631 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:49:31.631 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6100000000000001e-07 2024-09-16T20:49:31.631 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.631 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.631 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:31.631 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639295+0000", 2024-09-16T20:49:31.631 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.447e-06 2024-09-16T20:49:31.632 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.632 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.632 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:31.632 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639302+0000", 2024-09-16T20:49:31.632 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0570000000000004e-06 2024-09-16T20:49:31.632 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.632 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.632 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:31.632 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639307+0000", 2024-09-16T20:49:31.632 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1429999999999997e-06 2024-09-16T20:49:31.632 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.632 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.632 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:31.632 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690555+0000", 2024-09-16T20:49:31.632 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051247963000000001 2024-09-16T20:49:31.633 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.633 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.633 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:31.633 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690602+0000", 2024-09-16T20:49:31.633 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7602000000000003e-05 2024-09-16T20:49:31.633 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.633 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.633 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:31.633 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690672+0000", 2024-09-16T20:49:31.633 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9423000000000003e-05 2024-09-16T20:49:31.633 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:31.633 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:31.633 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:31.633 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.634 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.634 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3734 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1562624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:31.634 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:49:31.634 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 68.929373111000004, 2024-09-16T20:49:31.634 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051490610999999999, 2024-09-16T20:49:31.634 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:31.634 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:31.634 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:31.634 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:31.634 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:31.634 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:31.634 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3734 2024-09-16T20:49:31.634 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.634 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:31.634 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.634 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:31.635 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:49:31.635 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:31.635 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.635 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.635 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:31.635 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:49:31.635 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:31.635 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.635 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.635 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:31.635 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639314+0000", 2024-09-16T20:49:31.635 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4400000000000001e-07 2024-09-16T20:49:31.635 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.635 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.636 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:31.636 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639316+0000", 2024-09-16T20:49:31.636 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1459999999999999e-06 2024-09-16T20:49:31.636 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.636 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.636 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:31.636 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639322+0000", 2024-09-16T20:49:31.636 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7610000000000004e-06 2024-09-16T20:49:31.636 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.636 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.636 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:31.636 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639327+0000", 2024-09-16T20:49:31.636 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.186e-06 2024-09-16T20:49:31.636 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.637 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.637 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:31.637 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690698+0000", 2024-09-16T20:49:31.637 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051371223000000001 2024-09-16T20:49:31.637 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.637 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.637 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:31.637 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690748+0000", 2024-09-16T20:49:31.637 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0192999999999999e-05 2024-09-16T20:49:31.637 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.637 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.637 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:31.637 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690804+0000", 2024-09-16T20:49:31.637 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5658000000000001e-05 2024-09-16T20:49:31.638 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:31.638 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:31.638 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:31.638 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.638 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.638 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3736 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1570816~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:31.638 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:49:31.638 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 68.929353852000006, 2024-09-16T20:49:31.638 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051596528000000003, 2024-09-16T20:49:31.638 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:31.638 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:31.638 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:31.638 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:31.638 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:31.639 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:31.639 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3736 2024-09-16T20:49:31.639 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.639 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:31.639 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.639 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:31.639 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:49:31.639 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:31.639 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.639 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.639 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:31.639 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:49:31.639 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:31.639 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.640 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.640 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:31.640 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:49:31.640 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9999999999999999e-07 2024-09-16T20:49:31.640 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.640 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.640 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:31.640 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639335+0000", 2024-09-16T20:49:31.640 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.635e-06 2024-09-16T20:49:31.640 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.640 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.640 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:31.640 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639340+0000", 2024-09-16T20:49:31.641 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5840000000000002e-06 2024-09-16T20:49:31.641 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.641 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.641 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:31.641 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639346+0000", 2024-09-16T20:49:31.641 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1190000000000002e-06 2024-09-16T20:49:31.641 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.641 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.641 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:31.641 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690817+0000", 2024-09-16T20:49:31.641 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051470721999999997 2024-09-16T20:49:31.641 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.641 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.641 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:31.642 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690875+0000", 2024-09-16T20:49:31.642 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7441000000000001e-05 2024-09-16T20:49:31.642 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.642 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.642 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:31.642 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690929+0000", 2024-09-16T20:49:31.642 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4727e-05 2024-09-16T20:49:31.642 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:31.642 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:31.642 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:31.642 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.642 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.643 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5434 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:31.643 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:49:31.643 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 68.704092286000005, 2024-09-16T20:49:31.643 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099755831000000003, 2024-09-16T20:49:31.643 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:31.643 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:31.643 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:31.643 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:31.643 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:31.643 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:31.644 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5434 2024-09-16T20:49:31.644 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.644 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:31.644 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.644 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:31.644 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:49:31.644 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:31.644 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.644 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.644 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:31.644 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:49:31.644 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:31.644 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.645 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.645 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:31.645 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864595+0000", 2024-09-16T20:49:31.645 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1099999999999995e-07 2024-09-16T20:49:31.645 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.645 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.645 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:31.645 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864597+0000", 2024-09-16T20:49:31.645 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3989999999999998e-06 2024-09-16T20:49:31.645 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.645 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.645 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:31.645 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864603+0000", 2024-09-16T20:49:31.645 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9309999999999996e-06 2024-09-16T20:49:31.646 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.646 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.646 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:31.646 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864607+0000", 2024-09-16T20:49:31.646 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6200000000000001e-06 2024-09-16T20:49:31.646 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.646 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.646 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:31.646 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964164+0000", 2024-09-16T20:49:31.647 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099557162000000005 2024-09-16T20:49:31.647 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.647 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.647 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:31.647 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964244+0000", 2024-09-16T20:49:31.647 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9597000000000004e-05 2024-09-16T20:49:31.647 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.647 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.647 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:31.647 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964350+0000", 2024-09-16T20:49:31.647 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010641099999999999 2024-09-16T20:49:31.648 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:31.648 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:31.648 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:31.648 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.648 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.648 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5435 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 20480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:31.648 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:49:31.648 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 68.704062464000003, 2024-09-16T20:49:31.648 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100897102, 2024-09-16T20:49:31.648 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:31.648 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:31.648 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:31.649 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:31.649 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:31.649 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:31.649 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5435 2024-09-16T20:49:31.649 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.649 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:31.649 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.649 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:31.649 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:49:31.649 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:31.649 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.649 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.649 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:31.650 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:49:31.650 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:31.650 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.650 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.650 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:31.650 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:49:31.650 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8000000000000001e-07 2024-09-16T20:49:31.650 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.650 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.650 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:31.650 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864626+0000", 2024-09-16T20:49:31.650 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.745e-06 2024-09-16T20:49:31.650 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.650 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.651 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:31.651 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864633+0000", 2024-09-16T20:49:31.651 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3339999999999998e-06 2024-09-16T20:49:31.651 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.651 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.651 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:31.651 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864636+0000", 2024-09-16T20:49:31.651 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1200000000000002e-06 2024-09-16T20:49:31.651 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.651 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.652 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:31.652 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965299+0000", 2024-09-16T20:49:31.652 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10066352000000001 2024-09-16T20:49:31.652 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.652 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.652 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:31.652 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965390+0000", 2024-09-16T20:49:31.652 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0894000000000001e-05 2024-09-16T20:49:31.652 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.652 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.653 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:31.653 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965521+0000", 2024-09-16T20:49:31.653 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013110899999999999 2024-09-16T20:49:31.653 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:31.653 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:31.653 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:31.653 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.653 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.653 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:152 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 327680~32768 out=32768b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:49:31.653 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:49:31.653 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 67.405565522000003, 2024-09-16T20:49:31.654 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11869405800000001, 2024-09-16T20:49:31.654 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:31.654 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:31.654 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:31.654 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:31.654 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:49:31.654 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:49:31.654 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 152 2024-09-16T20:49:31.654 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.654 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:31.654 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.654 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:31.654 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:49:31.654 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:31.655 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.655 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.655 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:31.655 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:49:31.655 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:31.655 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.655 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.655 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:31.655 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163122+0000", 2024-09-16T20:49:31.655 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6400000000000003e-07 2024-09-16T20:49:31.655 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.655 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.656 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:31.656 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163124+0000", 2024-09-16T20:49:31.656 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.565e-06 2024-09-16T20:49:31.656 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.656 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.656 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:31.656 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163136+0000", 2024-09-16T20:49:31.656 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1717e-05 2024-09-16T20:49:31.656 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.656 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.656 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:31.657 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163142+0000", 2024-09-16T20:49:31.657 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1750000000000002e-06 2024-09-16T20:49:31.657 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.657 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.657 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:31.657 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163606+0000", 2024-09-16T20:49:31.657 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00046418799999999998 2024-09-16T20:49:31.657 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.657 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.658 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:31.658 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163656+0000", 2024-09-16T20:49:31.658 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9852999999999999e-05 2024-09-16T20:49:31.658 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.658 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.658 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:31.658 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281815+0000", 2024-09-16T20:49:31.658 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11815909600000001 2024-09-16T20:49:31.658 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:31.658 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:31.658 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:31.658 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.658 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.658 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:153 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:49:31.659 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:49:31.659 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 67.405532569000002, 2024-09-16T20:49:31.659 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11903462400000001, 2024-09-16T20:49:31.659 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:31.659 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:31.659 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:31.659 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:31.659 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:49:31.659 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:49:31.659 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 153 2024-09-16T20:49:31.659 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.659 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:31.659 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.659 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:31.660 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:49:31.660 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:31.660 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.660 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.660 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:31.660 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:49:31.660 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:31.660 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.660 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.660 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:31.660 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163155+0000", 2024-09-16T20:49:31.660 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.37e-07 2024-09-16T20:49:31.660 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.660 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.661 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:31.661 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163157+0000", 2024-09-16T20:49:31.661 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6010000000000002e-06 2024-09-16T20:49:31.661 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.661 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.661 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:31.661 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163165+0000", 2024-09-16T20:49:31.661 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2619999999999998e-06 2024-09-16T20:49:31.661 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.661 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.661 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:31.661 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163171+0000", 2024-09-16T20:49:31.661 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.711e-06 2024-09-16T20:49:31.662 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.662 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.662 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:31.662 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281868+0000", 2024-09-16T20:49:31.662 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118696622 2024-09-16T20:49:31.662 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.662 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.662 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:31.662 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281969+0000", 2024-09-16T20:49:31.662 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101371 2024-09-16T20:49:31.662 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.662 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.662 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:31.663 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282189+0000", 2024-09-16T20:49:31.663 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00021952000000000001 2024-09-16T20:49:31.663 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:31.663 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:31.663 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:31.663 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.663 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.663 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:154 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 360448~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:49:31.663 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:49:31.663 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 67.405500797000002, 2024-09-16T20:49:31.663 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11932035100000001, 2024-09-16T20:49:31.663 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:31.663 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:31.664 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:31.664 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:31.664 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:49:31.664 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:49:31.664 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 154 2024-09-16T20:49:31.664 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.664 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:31.664 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.664 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:31.664 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:49:31.664 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:31.664 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.664 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.664 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:31.665 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:49:31.665 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:31.665 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.665 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.665 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:31.665 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:49:31.665 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:49:31.665 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.665 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.665 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:31.665 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163189+0000", 2024-09-16T20:49:31.665 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6840000000000001e-06 2024-09-16T20:49:31.665 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.666 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.666 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:31.666 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163196+0000", 2024-09-16T20:49:31.666 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7920000000000004e-06 2024-09-16T20:49:31.666 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.666 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.666 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:31.666 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163201+0000", 2024-09-16T20:49:31.666 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2809999999999999e-06 2024-09-16T20:49:31.666 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.666 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.666 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:31.666 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282219+0000", 2024-09-16T20:49:31.666 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.119018343 2024-09-16T20:49:31.667 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.667 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.667 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:31.667 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282305+0000", 2024-09-16T20:49:31.667 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5865000000000005e-05 2024-09-16T20:49:31.667 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.667 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.667 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:31.667 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282506+0000", 2024-09-16T20:49:31.667 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000200927 2024-09-16T20:49:31.667 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:31.667 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:31.667 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:31.668 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.668 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.668 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop(client.4366.0:3088 3.1e e38/23 3:79e5ec01:::rbd_header.110e891a9b2b:head v 38'306, mlcod=36'304)", 2024-09-16T20:49:31.668 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:49:31.668 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 67.404481928999999, 2024-09-16T20:49:31.668 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118102759, 2024-09-16T20:49:31.668 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:31.668 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:31.668 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:31.668 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:31.668 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.668 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:31.668 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:49:31.669 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:31.669 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.669 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.669 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:31.669 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:49:31.669 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:31.669 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.669 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.669 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:31.669 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:49:31.669 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.85e-07 2024-09-16T20:49:31.669 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.669 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.670 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:31.670 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164212+0000", 2024-09-16T20:49:31.670 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3999999999999997e-06 2024-09-16T20:49:31.670 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.670 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.670 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:31.670 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281553+0000", 2024-09-16T20:49:31.670 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11734135499999999 2024-09-16T20:49:31.670 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.670 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.670 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:31.670 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281567+0000", 2024-09-16T20:49:31.670 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4188999999999999e-05 2024-09-16T20:49:31.670 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.671 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.671 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:31.671 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281653+0000", 2024-09-16T20:49:31.671 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5385999999999999e-05 2024-09-16T20:49:31.671 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.671 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.671 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:31.671 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281679+0000", 2024-09-16T20:49:31.671 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6689000000000001e-05 2024-09-16T20:49:31.671 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.672 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.672 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:31.672 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282277+0000", 2024-09-16T20:49:31.672 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00059798000000000002 2024-09-16T20:49:31.672 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:31.672 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:31.672 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:31.672 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282307+0000", 2024-09-16T20:49:31.672 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9975e-05 2024-09-16T20:49:31.672 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:31.672 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:31.672 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:31.672 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:31.673 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:31.673 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:31.674 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:49:31.675 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:31.673+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:31.775 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:31.774+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:31.783 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:31.783 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:49:31.783 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:49:31.783 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:31.791 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:49:31.858 INFO:teuthology.orchestra.run.smithi029.stderr:noscrub is set 2024-09-16T20:49:31.861 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 3, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-16T20:49:31.861 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T20:49:31.861 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-16T20:49:31.861 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-16T20:49:31.876 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:31.874+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:31.941 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:31.941 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:49:31.941 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:49:31.941 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:49:31.942 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:31.950 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:49:31.950 INFO:teuthology.orchestra.run.smithi139.stdout:truncating image image_client.0-clone27 from 0xd8a5200 (overlap 0x4cc2400) to 0x2e0e200 2024-09-16T20:49:31.992 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:31.990+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:32.078 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:32.076+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:32.116 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:32.116 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:49:32.116 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:49:32.116 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:49:32.116 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.117 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6419 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3594240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:32.117 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871775+0000", 2024-09-16T20:49:32.117 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 72.242600922999998, 2024-09-16T20:49:32.117 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048754724999999999, 2024-09-16T20:49:32.117 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:32.117 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:32.117 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:32.117 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:32.117 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:32.117 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:32.117 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6419 2024-09-16T20:49:32.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.117 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:32.118 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.118 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:32.118 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871775+0000", 2024-09-16T20:49:32.118 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.118 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.118 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.118 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:32.118 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871775+0000", 2024-09-16T20:49:32.118 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.118 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.118 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.118 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:32.118 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871775+0000", 2024-09-16T20:49:32.119 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2500000000000001e-07 2024-09-16T20:49:32.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.119 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.119 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:32.119 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871777+0000", 2024-09-16T20:49:32.119 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9649999999999998e-06 2024-09-16T20:49:32.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.119 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.119 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:32.119 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871784+0000", 2024-09-16T20:49:32.119 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3419999999999996e-06 2024-09-16T20:49:32.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.119 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.120 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:32.120 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871789+0000", 2024-09-16T20:49:32.120 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1889999999999998e-06 2024-09-16T20:49:32.120 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.120 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.120 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:32.120 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920416+0000", 2024-09-16T20:49:32.120 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048627363999999999 2024-09-16T20:49:32.120 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.120 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.120 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:32.120 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920472+0000", 2024-09-16T20:49:32.120 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5429000000000001e-05 2024-09-16T20:49:32.121 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.121 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.121 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:32.121 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920530+0000", 2024-09-16T20:49:32.121 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8111e-05 2024-09-16T20:49:32.121 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.121 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:32.121 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.121 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.121 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.121 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6420 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3602432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:32.121 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871796+0000", 2024-09-16T20:49:32.121 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 72.242580009999998, 2024-09-16T20:49:32.122 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048857298, 2024-09-16T20:49:32.122 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:32.122 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:32.122 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:32.122 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:32.122 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:32.122 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:32.122 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6420 2024-09-16T20:49:32.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.122 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:32.122 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.122 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:32.123 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871796+0000", 2024-09-16T20:49:32.123 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.123 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.123 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.123 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:32.123 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871796+0000", 2024-09-16T20:49:32.123 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.123 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.123 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.123 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:32.123 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871854+0000", 2024-09-16T20:49:32.123 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8187999999999999e-05 2024-09-16T20:49:32.123 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.124 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.124 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:32.124 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871856+0000", 2024-09-16T20:49:32.124 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3549999999999999e-06 2024-09-16T20:49:32.124 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.124 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.124 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:32.124 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871866+0000", 2024-09-16T20:49:32.124 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6500000000000008e-06 2024-09-16T20:49:32.124 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.124 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.124 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:32.124 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871872+0000", 2024-09-16T20:49:32.125 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3799999999999999e-06 2024-09-16T20:49:32.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.125 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.125 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:32.125 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920558+0000", 2024-09-16T20:49:32.125 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048685797000000003 2024-09-16T20:49:32.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.125 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.125 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:32.125 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920603+0000", 2024-09-16T20:49:32.125 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4629999999999998e-05 2024-09-16T20:49:32.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.125 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.126 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:32.126 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920653+0000", 2024-09-16T20:49:32.126 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0297999999999999e-05 2024-09-16T20:49:32.126 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.126 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:32.126 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.126 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.126 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.126 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6421 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3610624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:32.126 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871882+0000", 2024-09-16T20:49:32.126 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 72.242493941999996, 2024-09-16T20:49:32.126 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048913381999999998, 2024-09-16T20:49:32.126 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:32.127 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:32.127 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:32.127 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:32.127 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:32.127 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:32.127 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6421 2024-09-16T20:49:32.127 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.127 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:32.127 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.127 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:32.127 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871882+0000", 2024-09-16T20:49:32.127 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.127 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.128 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.128 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:32.128 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871882+0000", 2024-09-16T20:49:32.128 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.128 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.128 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:32.128 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.873257+0000", 2024-09-16T20:49:32.128 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001374789 2024-09-16T20:49:32.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.128 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.128 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:32.128 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.873261+0000", 2024-09-16T20:49:32.129 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5589999999999999e-06 2024-09-16T20:49:32.129 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.129 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.129 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:32.129 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.873275+0000", 2024-09-16T20:49:32.129 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3675999999999999e-05 2024-09-16T20:49:32.129 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.129 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.129 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:32.129 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.873281+0000", 2024-09-16T20:49:32.129 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6930000000000004e-06 2024-09-16T20:49:32.129 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.129 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.130 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:32.130 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920684+0000", 2024-09-16T20:49:32.130 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047402913999999997 2024-09-16T20:49:32.130 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.130 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.130 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:32.130 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920737+0000", 2024-09-16T20:49:32.130 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3600000000000002e-05 2024-09-16T20:49:32.130 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.130 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.130 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:32.130 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920795+0000", 2024-09-16T20:49:32.130 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8150999999999999e-05 2024-09-16T20:49:32.131 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.131 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:32.131 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.131 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.131 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.131 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:7491 3.1e 3:7c48f6ae:::rbd_object_map.110e891a9b2b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:32.131 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.925865+0000", 2024-09-16T20:49:32.131 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 72.188510797999996, 2024-09-16T20:49:32.131 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048897293000000001, 2024-09-16T20:49:32.131 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:32.131 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:32.131 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:32.131 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:32.131 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:32.132 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:32.132 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 7491 2024-09-16T20:49:32.132 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.132 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:32.132 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.132 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:32.132 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925865+0000", 2024-09-16T20:49:32.132 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.132 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.132 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.132 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:32.132 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925865+0000", 2024-09-16T20:49:32.132 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.133 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:32.133 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925866+0000", 2024-09-16T20:49:32.133 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.39e-07 2024-09-16T20:49:32.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.133 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:32.133 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925868+0000", 2024-09-16T20:49:32.133 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.508e-06 2024-09-16T20:49:32.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.133 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:32.133 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925874+0000", 2024-09-16T20:49:32.133 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7919999999999998e-06 2024-09-16T20:49:32.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.134 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.134 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:32.134 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925879+0000", 2024-09-16T20:49:32.134 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8579999999999999e-06 2024-09-16T20:49:32.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.134 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.134 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:32.134 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.970875+0000", 2024-09-16T20:49:32.134 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044996412 2024-09-16T20:49:32.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.134 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.134 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:32.135 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.970970+0000", 2024-09-16T20:49:32.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4732e-05 2024-09-16T20:49:32.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.135 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.135 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-16T20:49:32.135 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.971275+0000", 2024-09-16T20:49:32.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00030464199999999999 2024-09-16T20:49:32.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.135 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.135 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:32.135 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.974598+0000", 2024-09-16T20:49:32.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0033238159999999998 2024-09-16T20:49:32.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.135 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.135 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:49:32.136 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.974666+0000", 2024-09-16T20:49:32.136 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7617e-05 2024-09-16T20:49:32.136 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.136 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.136 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:49:32.136 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.974718+0000", 2024-09-16T20:49:32.136 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2259000000000002e-05 2024-09-16T20:49:32.136 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.136 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.136 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:32.136 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.974762+0000", 2024-09-16T20:49:32.136 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4218000000000003e-05 2024-09-16T20:49:32.136 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.136 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:32.137 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.137 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.137 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4758 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1521664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:32.137 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689141+0000", 2024-09-16T20:49:32.137 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 69.425235399000002, 2024-09-16T20:49:32.137 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048640741000000001, 2024-09-16T20:49:32.137 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:32.137 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:32.137 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:32.137 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:32.137 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:49:32.137 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:32.137 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4758 2024-09-16T20:49:32.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.138 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:32.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.138 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:32.138 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689141+0000", 2024-09-16T20:49:32.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.138 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:32.138 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689141+0000", 2024-09-16T20:49:32.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.138 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:32.139 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689141+0000", 2024-09-16T20:49:32.139 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8799999999999998e-07 2024-09-16T20:49:32.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.139 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.139 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:32.139 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689143+0000", 2024-09-16T20:49:32.139 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.945e-06 2024-09-16T20:49:32.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.139 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.139 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:32.139 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689149+0000", 2024-09-16T20:49:32.139 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9800000000000003e-06 2024-09-16T20:49:32.140 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.140 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.140 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:32.140 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689155+0000", 2024-09-16T20:49:32.140 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7509999999999997e-06 2024-09-16T20:49:32.140 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.140 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.140 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:32.140 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737681+0000", 2024-09-16T20:49:32.140 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048526825000000003 2024-09-16T20:49:32.140 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.140 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.140 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:32.140 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737729+0000", 2024-09-16T20:49:32.141 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7219999999999999e-05 2024-09-16T20:49:32.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.141 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.141 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:32.141 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737781+0000", 2024-09-16T20:49:32.141 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2632e-05 2024-09-16T20:49:32.141 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.141 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:32.141 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.141 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.141 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4760 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1529856~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:32.142 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689161+0000", 2024-09-16T20:49:32.142 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 69.425215051999999, 2024-09-16T20:49:32.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048743981999999998, 2024-09-16T20:49:32.142 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:32.142 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:32.142 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:32.142 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:32.142 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:49:32.142 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:32.142 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4760 2024-09-16T20:49:32.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.142 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:32.142 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.143 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:32.143 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689161+0000", 2024-09-16T20:49:32.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.143 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:32.143 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689161+0000", 2024-09-16T20:49:32.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.143 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:32.143 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689161+0000", 2024-09-16T20:49:32.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5599999999999995e-07 2024-09-16T20:49:32.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.144 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.144 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:32.144 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689163+0000", 2024-09-16T20:49:32.144 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.818e-06 2024-09-16T20:49:32.144 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.144 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.144 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:32.144 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689169+0000", 2024-09-16T20:49:32.144 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0809999999999999e-06 2024-09-16T20:49:32.144 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.144 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.144 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:32.144 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689174+0000", 2024-09-16T20:49:32.145 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.938e-06 2024-09-16T20:49:32.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.145 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.145 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:32.145 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737807+0000", 2024-09-16T20:49:32.145 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048633124 2024-09-16T20:49:32.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.145 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.145 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:32.145 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737851+0000", 2024-09-16T20:49:32.145 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3764000000000003e-05 2024-09-16T20:49:32.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.146 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:32.146 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737905+0000", 2024-09-16T20:49:32.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3701e-05 2024-09-16T20:49:32.146 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.146 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:32.146 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.146 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4762 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1538048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:32.146 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689182+0000", 2024-09-16T20:49:32.146 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 69.425193829999998, 2024-09-16T20:49:32.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048850275999999998, 2024-09-16T20:49:32.146 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:32.147 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:32.147 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:32.147 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:32.147 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:49:32.147 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:32.147 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4762 2024-09-16T20:49:32.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.147 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:32.147 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.147 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:32.147 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689182+0000", 2024-09-16T20:49:32.147 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.147 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.148 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:32.148 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689182+0000", 2024-09-16T20:49:32.148 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.148 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.148 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.148 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:32.148 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689182+0000", 2024-09-16T20:49:32.148 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-16T20:49:32.148 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.148 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.148 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:32.148 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689184+0000", 2024-09-16T20:49:32.148 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8190000000000001e-06 2024-09-16T20:49:32.148 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.149 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.149 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:32.149 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689191+0000", 2024-09-16T20:49:32.149 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2530000000000001e-06 2024-09-16T20:49:32.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.149 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.149 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:32.149 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689195+0000", 2024-09-16T20:49:32.149 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9559999999999996e-06 2024-09-16T20:49:32.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.149 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.149 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:32.149 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737933+0000", 2024-09-16T20:49:32.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048737919999999997 2024-09-16T20:49:32.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.150 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.150 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:32.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737982+0000", 2024-09-16T20:49:32.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8089e-05 2024-09-16T20:49:32.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.150 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.150 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:32.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738032+0000", 2024-09-16T20:49:32.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0868999999999997e-05 2024-09-16T20:49:32.150 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.150 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:32.150 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.151 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4764 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1546240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:32.151 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689201+0000", 2024-09-16T20:49:32.151 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 69.425174472999998, 2024-09-16T20:49:32.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048957550000000002, 2024-09-16T20:49:32.151 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:32.151 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:32.151 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:32.151 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:32.151 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:49:32.151 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:32.151 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4764 2024-09-16T20:49:32.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.152 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:32.152 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.152 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:32.152 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689201+0000", 2024-09-16T20:49:32.152 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.152 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.152 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:32.152 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689201+0000", 2024-09-16T20:49:32.152 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.153 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.153 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:32.153 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689202+0000", 2024-09-16T20:49:32.153 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4200000000000002e-07 2024-09-16T20:49:32.153 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.153 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.153 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:32.153 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689204+0000", 2024-09-16T20:49:32.153 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9750000000000001e-06 2024-09-16T20:49:32.153 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.153 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.153 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:32.153 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689210+0000", 2024-09-16T20:49:32.153 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1e-06 2024-09-16T20:49:32.154 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.154 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.154 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:32.154 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689215+0000", 2024-09-16T20:49:32.154 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0180000000000001e-06 2024-09-16T20:49:32.154 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.154 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.154 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:32.154 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738058+0000", 2024-09-16T20:49:32.154 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048843573000000001 2024-09-16T20:49:32.154 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.154 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.154 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:32.155 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738102+0000", 2024-09-16T20:49:32.155 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3256000000000001e-05 2024-09-16T20:49:32.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.155 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.155 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:32.155 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738159+0000", 2024-09-16T20:49:32.155 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7286e-05 2024-09-16T20:49:32.155 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.155 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:32.155 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.155 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.155 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4766 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1554432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:32.156 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689221+0000", 2024-09-16T20:49:32.156 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 69.425154507000002, 2024-09-16T20:49:32.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049064996, 2024-09-16T20:49:32.156 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:32.156 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:32.156 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:32.156 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:32.156 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:49:32.156 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:32.156 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4766 2024-09-16T20:49:32.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.156 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:32.156 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.156 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:32.156 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689221+0000", 2024-09-16T20:49:32.157 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.157 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.157 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.157 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:32.157 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689221+0000", 2024-09-16T20:49:32.157 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.157 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.157 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.157 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:32.157 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689222+0000", 2024-09-16T20:49:32.157 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.34e-07 2024-09-16T20:49:32.157 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.157 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.158 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:32.158 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689224+0000", 2024-09-16T20:49:32.158 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.846e-06 2024-09-16T20:49:32.158 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.158 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.158 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:32.158 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689247+0000", 2024-09-16T20:49:32.158 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3448e-05 2024-09-16T20:49:32.158 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.158 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.158 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:32.158 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689254+0000", 2024-09-16T20:49:32.158 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6969999999999996e-06 2024-09-16T20:49:32.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.159 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.159 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:32.159 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738188+0000", 2024-09-16T20:49:32.159 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048934520000000002 2024-09-16T20:49:32.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.159 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.159 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:32.159 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738234+0000", 2024-09-16T20:49:32.159 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5695999999999998e-05 2024-09-16T20:49:32.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.159 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.159 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:32.159 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738286+0000", 2024-09-16T20:49:32.160 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2355000000000003e-05 2024-09-16T20:49:32.160 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.160 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:32.160 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.160 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.160 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4768 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1562624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:32.160 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689262+0000", 2024-09-16T20:49:32.160 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 69.425113553000003, 2024-09-16T20:49:32.160 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049157157999999999, 2024-09-16T20:49:32.160 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:32.160 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:32.161 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:32.161 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:32.161 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:49:32.161 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:32.161 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4768 2024-09-16T20:49:32.161 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.161 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:32.161 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.161 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:32.161 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689262+0000", 2024-09-16T20:49:32.161 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.161 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.162 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.162 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:32.162 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689262+0000", 2024-09-16T20:49:32.162 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.162 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.162 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.162 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:32.162 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689263+0000", 2024-09-16T20:49:32.162 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6199999999999998e-07 2024-09-16T20:49:32.162 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.162 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.162 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:32.162 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689270+0000", 2024-09-16T20:49:32.162 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.798e-06 2024-09-16T20:49:32.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.163 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.163 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:32.163 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689277+0000", 2024-09-16T20:49:32.163 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0890000000000002e-06 2024-09-16T20:49:32.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.163 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.163 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:32.163 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689282+0000", 2024-09-16T20:49:32.163 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0470000000000001e-06 2024-09-16T20:49:32.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.163 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.163 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:32.164 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738319+0000", 2024-09-16T20:49:32.164 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049037139 2024-09-16T20:49:32.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.164 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.164 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:32.164 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738365+0000", 2024-09-16T20:49:32.164 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6356999999999997e-05 2024-09-16T20:49:32.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.164 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.164 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:32.164 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738420+0000", 2024-09-16T20:49:32.164 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4265999999999998e-05 2024-09-16T20:49:32.164 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.165 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:32.165 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.165 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.165 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.165 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4770 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1570816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:32.165 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689289+0000", 2024-09-16T20:49:32.165 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 69.425087043000005, 2024-09-16T20:49:32.165 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049253705000000002, 2024-09-16T20:49:32.165 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:32.165 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:32.165 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:32.165 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:32.165 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:49:32.166 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:32.166 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4770 2024-09-16T20:49:32.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.166 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:32.166 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.166 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:32.166 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689289+0000", 2024-09-16T20:49:32.166 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.166 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.166 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:32.166 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689289+0000", 2024-09-16T20:49:32.166 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.167 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.167 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:32.167 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689289+0000", 2024-09-16T20:49:32.167 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0999999999999999e-07 2024-09-16T20:49:32.167 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.167 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.167 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:32.167 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689291+0000", 2024-09-16T20:49:32.167 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0030000000000001e-06 2024-09-16T20:49:32.167 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.167 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.167 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:32.167 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689297+0000", 2024-09-16T20:49:32.168 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9479999999999996e-06 2024-09-16T20:49:32.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.168 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.168 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:32.168 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689302+0000", 2024-09-16T20:49:32.168 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7389999999999999e-06 2024-09-16T20:49:32.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.168 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.168 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:32.168 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738442+0000", 2024-09-16T20:49:32.168 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049140001000000003 2024-09-16T20:49:32.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.168 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.168 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:32.169 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738490+0000", 2024-09-16T20:49:32.169 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7787000000000002e-05 2024-09-16T20:49:32.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.169 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.169 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:32.169 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738543+0000", 2024-09-16T20:49:32.169 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2816999999999999e-05 2024-09-16T20:49:32.169 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.169 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:32.169 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.169 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.169 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:3088 3.1e 3:79e5ec01:::rbd_header.110e891a9b2b:head [watch watch cookie 140213437171488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:49:32.170 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:49:32.170 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 67.950531025999993, 2024-09-16T20:49:32.170 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.11895186200000001, 2024-09-16T20:49:32.170 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:32.170 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:32.170 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:32.170 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:32.170 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:32.170 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:32.170 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3088 2024-09-16T20:49:32.170 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.170 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:32.171 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.171 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:32.171 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:49:32.171 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.171 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.171 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.171 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:32.171 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:49:32.171 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.171 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.171 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.171 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:32.171 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163846+0000", 2024-09-16T20:49:32.171 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6799999999999999e-07 2024-09-16T20:49:32.172 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.172 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.172 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:32.172 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163850+0000", 2024-09-16T20:49:32.172 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.929e-06 2024-09-16T20:49:32.172 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.172 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.172 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:32.172 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163862+0000", 2024-09-16T20:49:32.172 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1972e-05 2024-09-16T20:49:32.172 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.172 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.172 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:32.172 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163867+0000", 2024-09-16T20:49:32.172 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.451e-06 2024-09-16T20:49:32.173 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.173 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.173 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:32.173 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163912+0000", 2024-09-16T20:49:32.173 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4456000000000003e-05 2024-09-16T20:49:32.173 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.173 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.173 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:32.173 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163962+0000", 2024-09-16T20:49:32.173 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0578000000000003e-05 2024-09-16T20:49:32.173 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.173 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.173 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-16T20:49:32.174 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164058+0000", 2024-09-16T20:49:32.174 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5934999999999995e-05 2024-09-16T20:49:32.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.174 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.174 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:49:32.174 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164570+0000", 2024-09-16T20:49:32.174 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00051241100000000005 2024-09-16T20:49:32.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.174 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.174 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:32.174 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282629+0000", 2024-09-16T20:49:32.174 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.118058986 2024-09-16T20:49:32.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.174 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.175 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:49:32.175 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282725+0000", 2024-09-16T20:49:32.175 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5063999999999996e-05 2024-09-16T20:49:32.175 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.175 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.175 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:32.175 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282797+0000", 2024-09-16T20:49:32.175 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2212000000000006e-05 2024-09-16T20:49:32.175 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.175 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:32.175 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.175 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.176 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.176 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:8039 3.1e 3.ac44ff9e (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:49:32.176 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:49:32.176 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 42.406703677000003, 2024-09-16T20:49:32.176 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0053310449999999, 2024-09-16T20:49:32.176 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:32.176 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:32.176 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:49:32.176 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:32.176 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:32.176 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:32.176 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8039 2024-09-16T20:49:32.176 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.177 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:32.177 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.177 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:32.177 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:49:32.177 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.177 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.177 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:32.177 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:49:32.177 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.177 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.177 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:32.177 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707674+0000", 2024-09-16T20:49:32.178 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6530000000000001e-06 2024-09-16T20:49:32.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.178 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.178 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:32.178 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707681+0000", 2024-09-16T20:49:32.178 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9639999999999997e-06 2024-09-16T20:49:32.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.178 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.178 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:32.178 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707700+0000", 2024-09-16T20:49:32.178 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8674000000000001e-05 2024-09-16T20:49:32.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.178 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.178 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:32.179 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707708+0000", 2024-09-16T20:49:32.179 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6810000000000004e-06 2024-09-16T20:49:32.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.179 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.179 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:32.179 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711293+0000", 2024-09-16T20:49:32.179 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035848189999999999 2024-09-16T20:49:32.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.179 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.179 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:49:32.179 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711301+0000", 2024-09-16T20:49:32.179 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4730000000000003e-06 2024-09-16T20:49:32.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.179 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.180 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:32.180 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.712990+0000", 2024-09-16T20:49:32.180 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.001689348 2024-09-16T20:49:32.180 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.180 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.180 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:32.180 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713003+0000", 2024-09-16T20:49:32.180 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3433e-05 2024-09-16T20:49:32.180 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.180 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:32.180 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.180 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.180 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.180 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:8040 3.1e 3.ac44ff9e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:49:32.181 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:49:32.181 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 42.406643281999997, 2024-09-16T20:49:32.181 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0053410780000001, 2024-09-16T20:49:32.181 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:32.181 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:32.181 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:49:32.181 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:32.181 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:32.181 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:32.181 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8040 2024-09-16T20:49:32.181 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.181 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:32.181 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.182 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:32.182 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:49:32.182 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.182 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.182 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.182 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:32.182 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:49:32.182 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.182 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.182 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.182 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:32.182 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:49:32.182 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8100000000000003e-07 2024-09-16T20:49:32.183 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.183 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.183 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:32.183 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707737+0000", 2024-09-16T20:49:32.183 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5020000000000001e-06 2024-09-16T20:49:32.183 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.183 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.183 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:32.183 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707745+0000", 2024-09-16T20:49:32.183 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2169999999999994e-06 2024-09-16T20:49:32.183 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.183 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.183 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:32.184 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707751+0000", 2024-09-16T20:49:32.184 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.263e-06 2024-09-16T20:49:32.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.184 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.184 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:32.184 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711336+0000", 2024-09-16T20:49:32.184 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035848360000000001 2024-09-16T20:49:32.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.184 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.184 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:49:32.184 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711340+0000", 2024-09-16T20:49:32.184 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1309999999999999e-06 2024-09-16T20:49:32.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.184 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.185 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:32.185 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713060+0000", 2024-09-16T20:49:32.185 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0017195059999999 2024-09-16T20:49:32.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.185 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.185 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:32.185 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713074+0000", 2024-09-16T20:49:32.185 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4042e-05 2024-09-16T20:49:32.185 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.185 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:32.185 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.185 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.185 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4927.0:4487 3.1e 3.ac44ff9e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:49:32.186 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:49:32.186 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 42.40612308, 2024-09-16T20:49:32.186 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0048840880000001, 2024-09-16T20:49:32.186 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:32.186 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:32.186 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:49:32.186 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:32.186 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4927", 2024-09-16T20:49:32.186 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:32.186 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4487 2024-09-16T20:49:32.186 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.186 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:32.186 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.186 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:32.187 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:49:32.187 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.187 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.187 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.187 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:32.187 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:49:32.187 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.187 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.187 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.187 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:32.187 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708254+0000", 2024-09-16T20:49:32.187 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1800000000000005e-07 2024-09-16T20:49:32.187 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.188 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.188 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:32.188 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708258+0000", 2024-09-16T20:49:32.188 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8689999999999997e-06 2024-09-16T20:49:32.188 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.188 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.188 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:32.188 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708267+0000", 2024-09-16T20:49:32.188 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9590000000000007e-06 2024-09-16T20:49:32.188 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.188 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.188 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:32.188 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708274+0000", 2024-09-16T20:49:32.188 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9970000000000001e-06 2024-09-16T20:49:32.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.189 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.189 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:32.189 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711366+0000", 2024-09-16T20:49:32.189 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030919530000000001 2024-09-16T20:49:32.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.189 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.189 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:49:32.189 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711372+0000", 2024-09-16T20:49:32.189 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9410000000000004e-06 2024-09-16T20:49:32.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.189 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.189 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:32.190 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713126+0000", 2024-09-16T20:49:32.190 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0017537249999999 2024-09-16T20:49:32.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.190 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.190 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:32.190 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713137+0000", 2024-09-16T20:49:32.190 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0825999999999999e-05 2024-09-16T20:49:32.190 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.190 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:32.190 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.190 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.190 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5089.0:11703 3.1b 3:dcd4684e:::rbd_data.110e747f0779.000000000000001b:head [set-alloc-hint object_size 262144 write_size 262144,write 139264~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:49:32.191 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:49:32.191 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.976387305999999, 2024-09-16T20:49:32.191 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.084231385000000006, 2024-09-16T20:49:32.191 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:32.191 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:32.191 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:32.191 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:32.191 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5089", 2024-09-16T20:49:32.191 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:49:32.191 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 11703 2024-09-16T20:49:32.191 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.191 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:32.191 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.191 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:32.192 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:49:32.192 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.192 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.192 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.192 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:32.192 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:49:32.192 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.192 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.192 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.192 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:32.192 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:49:32.192 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2600000000000001e-07 2024-09-16T20:49:32.193 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.193 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.193 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:32.193 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137998+0000", 2024-09-16T20:49:32.193 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9549999999999998e-06 2024-09-16T20:49:32.193 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.193 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.193 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:32.193 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138008+0000", 2024-09-16T20:49:32.193 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3789999999999995e-06 2024-09-16T20:49:32.193 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.193 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.194 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:32.194 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138013+0000", 2024-09-16T20:49:32.194 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2209999999999996e-06 2024-09-16T20:49:32.194 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.194 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.194 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:32.194 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138054+0000", 2024-09-16T20:49:32.194 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1022999999999997e-05 2024-09-16T20:49:32.194 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.194 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.194 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:32.194 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138124+0000", 2024-09-16T20:49:32.194 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0234999999999994e-05 2024-09-16T20:49:32.194 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.195 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.195 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:32.195 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138197+0000", 2024-09-16T20:49:32.195 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2451000000000001e-05 2024-09-16T20:49:32.195 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.195 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.195 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:32.195 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222010+0000", 2024-09-16T20:49:32.195 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.083813433000000007 2024-09-16T20:49:32.195 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.195 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.195 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:49:32.195 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222163+0000", 2024-09-16T20:49:32.195 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015274599999999999 2024-09-16T20:49:32.196 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.196 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.196 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:49:32.196 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222197+0000", 2024-09-16T20:49:32.196 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3683000000000003e-05 2024-09-16T20:49:32.196 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.196 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.196 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:32.196 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222220+0000", 2024-09-16T20:49:32.196 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3433000000000001e-05 2024-09-16T20:49:32.196 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.196 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:32.196 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.197 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.197 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.197 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.5089.0:11702 3.15 e73/64 3:acf33c8b:::rbd_data.110e747f0779.000000000000001a:head v 73'132, mlcod=73'130)", 2024-09-16T20:49:32.197 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:49:32.197 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.976119977, 2024-09-16T20:49:32.197 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.084079763000000002, 2024-09-16T20:49:32.197 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:32.197 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:32.197 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:32.197 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:32.197 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.197 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:32.197 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:49:32.197 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.198 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.198 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.198 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:32.198 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:49:32.198 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.198 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.198 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.198 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:32.198 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:49:32.198 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:49:32.198 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.198 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.198 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:32.199 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138263+0000", 2024-09-16T20:49:32.199 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1e-06 2024-09-16T20:49:32.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.199 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.199 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:32.199 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221589+0000", 2024-09-16T20:49:32.199 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.083326002999999996 2024-09-16T20:49:32.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.199 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.199 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:32.199 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221614+0000", 2024-09-16T20:49:32.199 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5593999999999999e-05 2024-09-16T20:49:32.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.199 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.200 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:32.200 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221714+0000", 2024-09-16T20:49:32.200 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9889000000000005e-05 2024-09-16T20:49:32.200 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.200 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.200 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:32.200 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221727+0000", 2024-09-16T20:49:32.200 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3125999999999999e-05 2024-09-16T20:49:32.200 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.200 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.200 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:49:32.200 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222294+0000", 2024-09-16T20:49:32.200 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00056712199999999998 2024-09-16T20:49:32.200 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.201 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.201 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:32.201 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222336+0000", 2024-09-16T20:49:32.201 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1470000000000001e-05 2024-09-16T20:49:32.201 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.201 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:32.201 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.201 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.201 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.201 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5138.0:9358 3.c 3:33996a43:::rbd_data.110e6372565d.0000000000000007:head [read 1343488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T20:49:32.201 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:49:32.201 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.2618445669999998, 2024-09-16T20:49:32.201 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0167493569999999, 2024-09-16T20:49:32.202 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:32.202 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:32.202 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:32.202 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:32.202 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5138", 2024-09-16T20:49:32.202 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:49:32.202 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9358 2024-09-16T20:49:32.202 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.202 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:32.202 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.202 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:32.202 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:49:32.202 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.202 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.203 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.203 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:32.203 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:49:32.203 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.203 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.203 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:32.203 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852533+0000", 2024-09-16T20:49:32.203 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.646e-06 2024-09-16T20:49:32.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.203 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.203 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:32.203 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852539+0000", 2024-09-16T20:49:32.204 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3389999999999998e-06 2024-09-16T20:49:32.204 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.204 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.204 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:32.204 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852552+0000", 2024-09-16T20:49:32.204 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.271e-05 2024-09-16T20:49:32.204 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.204 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.204 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:32.204 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852563+0000", 2024-09-16T20:49:32.204 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1139e-05 2024-09-16T20:49:32.204 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.204 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.205 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:32.205 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852883+0000", 2024-09-16T20:49:32.205 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00032010600000000001 2024-09-16T20:49:32.205 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.205 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.205 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:49:32.205 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852890+0000", 2024-09-16T20:49:32.205 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7669999999999998e-06 2024-09-16T20:49:32.205 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.205 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.205 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:32.205 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869059+0000", 2024-09-16T20:49:32.205 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.016169353 2024-09-16T20:49:32.205 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.206 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.206 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:32.206 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869178+0000", 2024-09-16T20:49:32.206 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118193 2024-09-16T20:49:32.206 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.206 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.206 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:32.206 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869281+0000", 2024-09-16T20:49:32.206 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010310399999999999 2024-09-16T20:49:32.206 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.206 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:32.206 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.206 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.207 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.207 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5138.0:9382 3.c 3:33996a43:::rbd_data.110e6372565d.0000000000000007:head [read 0~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T20:49:32.207 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:49:32.207 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.2617936900000002, 2024-09-16T20:49:32.207 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.017136034, 2024-09-16T20:49:32.207 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:32.207 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:32.207 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:32.207 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:32.207 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5138", 2024-09-16T20:49:32.207 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:49:32.207 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9382 2024-09-16T20:49:32.207 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.207 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:32.208 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.208 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:32.208 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:49:32.208 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.208 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.208 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.208 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:32.208 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:49:32.208 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.208 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.208 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.208 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:32.208 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852583+0000", 2024-09-16T20:49:32.209 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.51e-07 2024-09-16T20:49:32.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.209 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.209 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:32.209 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852586+0000", 2024-09-16T20:49:32.209 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0280000000000001e-06 2024-09-16T20:49:32.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.209 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.209 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:32.209 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852595+0000", 2024-09-16T20:49:32.209 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2799999999999992e-06 2024-09-16T20:49:32.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.209 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.210 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:32.210 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852601+0000", 2024-09-16T20:49:32.210 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3810000000000004e-06 2024-09-16T20:49:32.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.210 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.210 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:32.210 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852921+0000", 2024-09-16T20:49:32.210 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00031979699999999997 2024-09-16T20:49:32.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.210 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.210 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:49:32.210 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852930+0000", 2024-09-16T20:49:32.210 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6489999999999997e-06 2024-09-16T20:49:32.211 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.211 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.211 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:32.211 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869434+0000", 2024-09-16T20:49:32.211 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0165038829999999 2024-09-16T20:49:32.211 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.211 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.211 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:32.211 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869495+0000", 2024-09-16T20:49:32.211 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1509999999999999e-05 2024-09-16T20:49:32.211 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.211 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.211 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:32.212 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869718+0000", 2024-09-16T20:49:32.212 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000222955 2024-09-16T20:49:32.212 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.212 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:32.212 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.212 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.212 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.212 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5425.0:2601 3.1e 3:7dc19323:::rbd_data.110e6372565d.0000000000000026:1d [read 1171456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-16T20:49:32.212 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:31.967612+0000", 2024-09-16T20:49:32.212 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.14676366800000001, 2024-09-16T20:49:32.212 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048708179999999997, 2024-09-16T20:49:32.212 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:32.213 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:32.213 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:32.213 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:32.213 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5425", 2024-09-16T20:49:32.213 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1974102553", 2024-09-16T20:49:32.213 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 2601 2024-09-16T20:49:32.213 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.213 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:32.213 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.213 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:32.214 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:31.967612+0000", 2024-09-16T20:49:32.214 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.214 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.214 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.214 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:32.214 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:31.967612+0000", 2024-09-16T20:49:32.214 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.214 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.214 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.214 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:32.214 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:31.967842+0000", 2024-09-16T20:49:32.214 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00022931899999999999 2024-09-16T20:49:32.214 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.214 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.215 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:32.215 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:31.967845+0000", 2024-09-16T20:49:32.215 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2320000000000001e-06 2024-09-16T20:49:32.215 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.215 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.215 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:32.215 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:31.967854+0000", 2024-09-16T20:49:32.215 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1849999999999993e-06 2024-09-16T20:49:32.215 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.215 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.215 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:32.215 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:31.967864+0000", 2024-09-16T20:49:32.215 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6639999999999993e-06 2024-09-16T20:49:32.216 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.216 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.216 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:32.216 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:32.016212+0000", 2024-09-16T20:49:32.216 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048348820000000001 2024-09-16T20:49:32.216 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.216 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.216 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:32.216 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:32.016257+0000", 2024-09-16T20:49:32.216 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4674e-05 2024-09-16T20:49:32.216 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.216 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.216 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:32.217 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:32.016320+0000", 2024-09-16T20:49:32.217 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3286000000000004e-05 2024-09-16T20:49:32.217 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.217 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:32.217 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.217 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.217 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:32.217 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:32.219 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:49:32.220 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:32.217+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:32.220 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:49:32.220 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:32.220 INFO:teuthology.orchestra.run.smithi029.stdout: "epoch": 99, 2024-09-16T20:49:32.220 INFO:teuthology.orchestra.run.smithi029.stdout: "fsid": "e28af4a7-5fcd-4de8-8f77-a2b575361bb1", 2024-09-16T20:49:32.220 INFO:teuthology.orchestra.run.smithi029.stdout: "created": "2024-09-16T20:47:39.760609+0000", 2024-09-16T20:49:32.220 INFO:teuthology.orchestra.run.smithi029.stdout: "modified": "2024-09-16T20:49:31.853964+0000", 2024-09-16T20:49:32.220 INFO:teuthology.orchestra.run.smithi029.stdout: "last_up_change": "2024-09-16T20:47:50.238013+0000", 2024-09-16T20:49:32.220 INFO:teuthology.orchestra.run.smithi029.stdout: "last_in_change": "2024-09-16T20:49:08.779548+0000", 2024-09-16T20:49:32.221 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-16T20:49:32.221 INFO:teuthology.orchestra.run.smithi029.stdout: "flags_num": 5801984, 2024-09-16T20:49:32.221 INFO:teuthology.orchestra.run.smithi029.stdout: "flags_set": [ 2024-09-16T20:49:32.221 INFO:teuthology.orchestra.run.smithi029.stdout: "noscrub", 2024-09-16T20:49:32.221 INFO:teuthology.orchestra.run.smithi029.stdout: "pglog_hardlimit", 2024-09-16T20:49:32.221 INFO:teuthology.orchestra.run.smithi029.stdout: "purged_snapdirs", 2024-09-16T20:49:32.221 INFO:teuthology.orchestra.run.smithi029.stdout: "recovery_deletes", 2024-09-16T20:49:32.221 INFO:teuthology.orchestra.run.smithi029.stdout: "sortbitwise" 2024-09-16T20:49:32.221 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-16T20:49:32.221 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_version": 5, 2024-09-16T20:49:32.221 INFO:teuthology.orchestra.run.smithi029.stdout: "full_ratio": 0.94999998807907104, 2024-09-16T20:49:32.221 INFO:teuthology.orchestra.run.smithi029.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-16T20:49:32.221 INFO:teuthology.orchestra.run.smithi029.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-16T20:49:32.222 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_snapshot": "", 2024-09-16T20:49:32.222 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_max": 3, 2024-09-16T20:49:32.222 INFO:teuthology.orchestra.run.smithi029.stdout: "max_osd": 6, 2024-09-16T20:49:32.222 INFO:teuthology.orchestra.run.smithi029.stdout: "require_min_compat_client": "luminous", 2024-09-16T20:49:32.222 INFO:teuthology.orchestra.run.smithi029.stdout: "min_compat_client": "jewel", 2024-09-16T20:49:32.222 INFO:teuthology.orchestra.run.smithi029.stdout: "require_osd_release": "squid", 2024-09-16T20:49:32.222 INFO:teuthology.orchestra.run.smithi029.stdout: "allow_crimson": false, 2024-09-16T20:49:32.222 INFO:teuthology.orchestra.run.smithi029.stdout: "pools": [ 2024-09-16T20:49:32.222 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.222 INFO:teuthology.orchestra.run.smithi029.stdout: "pool": 1, 2024-09-16T20:49:32.222 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_name": ".mgr", 2024-09-16T20:49:32.222 INFO:teuthology.orchestra.run.smithi029.stdout: "create_time": "2024-09-16T20:47:52.220750+0000", 2024-09-16T20:49:32.222 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-16T20:49:32.223 INFO:teuthology.orchestra.run.smithi029.stdout: "flags_names": "hashpspool", 2024-09-16T20:49:32.223 INFO:teuthology.orchestra.run.smithi029.stdout: "type": 1, 2024-09-16T20:49:32.223 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 2, 2024-09-16T20:49:32.223 INFO:teuthology.orchestra.run.smithi029.stdout: "min_size": 1, 2024-09-16T20:49:32.223 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_rule": 0, 2024-09-16T20:49:32.223 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_count": 0, 2024-09-16T20:49:32.223 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_target": 0, 2024-09-16T20:49:32.223 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T20:49:32.223 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T20:49:32.223 INFO:teuthology.orchestra.run.smithi029.stdout: "is_stretch_pool": false, 2024-09-16T20:49:32.223 INFO:teuthology.orchestra.run.smithi029.stdout: "object_hash": 2, 2024-09-16T20:49:32.223 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_autoscale_mode": "off", 2024-09-16T20:49:32.223 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num": 1, 2024-09-16T20:49:32.223 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num": 1, 2024-09-16T20:49:32.224 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num_target": 1, 2024-09-16T20:49:32.224 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_target": 1, 2024-09-16T20:49:32.224 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_pending": 1, 2024-09-16T20:49:32.224 INFO:teuthology.orchestra.run.smithi029.stdout: "last_pg_merge_meta": { 2024-09-16T20:49:32.224 INFO:teuthology.orchestra.run.smithi029.stdout: "source_pgid": "0.0", 2024-09-16T20:49:32.224 INFO:teuthology.orchestra.run.smithi029.stdout: "ready_epoch": 0, 2024-09-16T20:49:32.224 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_started": 0, 2024-09-16T20:49:32.224 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_clean": 0, 2024-09-16T20:49:32.224 INFO:teuthology.orchestra.run.smithi029.stdout: "source_version": "0'0", 2024-09-16T20:49:32.224 INFO:teuthology.orchestra.run.smithi029.stdout: "target_version": "0'0" 2024-09-16T20:49:32.224 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.224 INFO:teuthology.orchestra.run.smithi029.stdout: "last_change": "17", 2024-09-16T20:49:32.224 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend": "0", 2024-09-16T20:49:32.224 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T20:49:32.225 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T20:49:32.225 INFO:teuthology.orchestra.run.smithi029.stdout: "auid": 0, 2024-09-16T20:49:32.225 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_mode": "selfmanaged", 2024-09-16T20:49:32.225 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_seq": 0, 2024-09-16T20:49:32.225 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_epoch": 0, 2024-09-16T20:49:32.225 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_snaps": [], 2024-09-16T20:49:32.225 INFO:teuthology.orchestra.run.smithi029.stdout: "removed_snaps": "[]", 2024-09-16T20:49:32.225 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_bytes": 0, 2024-09-16T20:49:32.225 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_objects": 0, 2024-09-16T20:49:32.225 INFO:teuthology.orchestra.run.smithi029.stdout: "tiers": [], 2024-09-16T20:49:32.225 INFO:teuthology.orchestra.run.smithi029.stdout: "tier_of": -1, 2024-09-16T20:49:32.225 INFO:teuthology.orchestra.run.smithi029.stdout: "read_tier": -1, 2024-09-16T20:49:32.225 INFO:teuthology.orchestra.run.smithi029.stdout: "write_tier": -1, 2024-09-16T20:49:32.226 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_mode": "none", 2024-09-16T20:49:32.226 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_bytes": 0, 2024-09-16T20:49:32.226 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_objects": 0, 2024-09-16T20:49:32.226 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T20:49:32.226 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T20:49:32.226 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T20:49:32.226 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_flush_age": 0, 2024-09-16T20:49:32.226 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_evict_age": 0, 2024-09-16T20:49:32.226 INFO:teuthology.orchestra.run.smithi029.stdout: "erasure_code_profile": "", 2024-09-16T20:49:32.226 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_params": { 2024-09-16T20:49:32.226 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "none" 2024-09-16T20:49:32.226 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.226 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_period": 0, 2024-09-16T20:49:32.226 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_count": 0, 2024-09-16T20:49:32.227 INFO:teuthology.orchestra.run.smithi029.stdout: "use_gmt_hitset": true, 2024-09-16T20:49:32.227 INFO:teuthology.orchestra.run.smithi029.stdout: "min_read_recency_for_promote": 0, 2024-09-16T20:49:32.227 INFO:teuthology.orchestra.run.smithi029.stdout: "min_write_recency_for_promote": 0, 2024-09-16T20:49:32.227 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T20:49:32.227 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_search_last_n": 0, 2024-09-16T20:49:32.227 INFO:teuthology.orchestra.run.smithi029.stdout: "grade_table": [], 2024-09-16T20:49:32.227 INFO:teuthology.orchestra.run.smithi029.stdout: "stripe_width": 0, 2024-09-16T20:49:32.227 INFO:teuthology.orchestra.run.smithi029.stdout: "expected_num_objects": 0, 2024-09-16T20:49:32.227 INFO:teuthology.orchestra.run.smithi029.stdout: "fast_read": false, 2024-09-16T20:49:32.227 INFO:teuthology.orchestra.run.smithi029.stdout: "options": { 2024-09-16T20:49:32.227 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_max": 32, 2024-09-16T20:49:32.227 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_min": 1 2024-09-16T20:49:32.227 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.228 INFO:teuthology.orchestra.run.smithi029.stdout: "application_metadata": { 2024-09-16T20:49:32.228 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": {} 2024-09-16T20:49:32.228 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.228 INFO:teuthology.orchestra.run.smithi029.stdout: "read_balance": { 2024-09-16T20:49:32.228 INFO:teuthology.orchestra.run.smithi029.stdout: "score_type": "Fair distribution", 2024-09-16T20:49:32.228 INFO:teuthology.orchestra.run.smithi029.stdout: "score_acting": 6.059999942779541, 2024-09-16T20:49:32.228 INFO:teuthology.orchestra.run.smithi029.stdout: "score_stable": 6.059999942779541, 2024-09-16T20:49:32.228 INFO:teuthology.orchestra.run.smithi029.stdout: "optimal_score": 0.33000001311302185, 2024-09-16T20:49:32.228 INFO:teuthology.orchestra.run.smithi029.stdout: "raw_score_acting": 2, 2024-09-16T20:49:32.228 INFO:teuthology.orchestra.run.smithi029.stdout: "raw_score_stable": 2, 2024-09-16T20:49:32.228 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity_weighted": 1, 2024-09-16T20:49:32.228 INFO:teuthology.orchestra.run.smithi029.stdout: "average_primary_affinity": 1, 2024-09-16T20:49:32.228 INFO:teuthology.orchestra.run.smithi029.stdout: "average_primary_affinity_weighted": 1 2024-09-16T20:49:32.228 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:32.229 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.229 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.229 INFO:teuthology.orchestra.run.smithi029.stdout: "pool": 2, 2024-09-16T20:49:32.229 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_name": "rbd", 2024-09-16T20:49:32.229 INFO:teuthology.orchestra.run.smithi029.stdout: "create_time": "2024-09-16T20:47:56.756666+0000", 2024-09-16T20:49:32.229 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 8193, 2024-09-16T20:49:32.229 INFO:teuthology.orchestra.run.smithi029.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T20:49:32.229 INFO:teuthology.orchestra.run.smithi029.stdout: "type": 1, 2024-09-16T20:49:32.229 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 2, 2024-09-16T20:49:32.229 INFO:teuthology.orchestra.run.smithi029.stdout: "min_size": 1, 2024-09-16T20:49:32.229 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_rule": 0, 2024-09-16T20:49:32.229 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_count": 0, 2024-09-16T20:49:32.230 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_target": 0, 2024-09-16T20:49:32.230 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T20:49:32.230 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T20:49:32.230 INFO:teuthology.orchestra.run.smithi029.stdout: "is_stretch_pool": false, 2024-09-16T20:49:32.230 INFO:teuthology.orchestra.run.smithi029.stdout: "object_hash": 2, 2024-09-16T20:49:32.230 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_autoscale_mode": "off", 2024-09-16T20:49:32.230 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num": 8, 2024-09-16T20:49:32.230 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num": 8, 2024-09-16T20:49:32.230 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num_target": 8, 2024-09-16T20:49:32.230 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_target": 8, 2024-09-16T20:49:32.230 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_pending": 8, 2024-09-16T20:49:32.230 INFO:teuthology.orchestra.run.smithi029.stdout: "last_pg_merge_meta": { 2024-09-16T20:49:32.230 INFO:teuthology.orchestra.run.smithi029.stdout: "source_pgid": "0.0", 2024-09-16T20:49:32.231 INFO:teuthology.orchestra.run.smithi029.stdout: "ready_epoch": 0, 2024-09-16T20:49:32.231 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_started": 0, 2024-09-16T20:49:32.231 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_clean": 0, 2024-09-16T20:49:32.231 INFO:teuthology.orchestra.run.smithi029.stdout: "source_version": "0'0", 2024-09-16T20:49:32.231 INFO:teuthology.orchestra.run.smithi029.stdout: "target_version": "0'0" 2024-09-16T20:49:32.231 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.231 INFO:teuthology.orchestra.run.smithi029.stdout: "last_change": "21", 2024-09-16T20:49:32.231 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend": "0", 2024-09-16T20:49:32.231 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T20:49:32.231 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T20:49:32.231 INFO:teuthology.orchestra.run.smithi029.stdout: "auid": 0, 2024-09-16T20:49:32.231 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_mode": "selfmanaged", 2024-09-16T20:49:32.231 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_seq": 2, 2024-09-16T20:49:32.232 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_epoch": 21, 2024-09-16T20:49:32.232 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_snaps": [], 2024-09-16T20:49:32.232 INFO:teuthology.orchestra.run.smithi029.stdout: "removed_snaps": "[]", 2024-09-16T20:49:32.232 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_bytes": 0, 2024-09-16T20:49:32.232 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_objects": 0, 2024-09-16T20:49:32.232 INFO:teuthology.orchestra.run.smithi029.stdout: "tiers": [], 2024-09-16T20:49:32.232 INFO:teuthology.orchestra.run.smithi029.stdout: "tier_of": -1, 2024-09-16T20:49:32.232 INFO:teuthology.orchestra.run.smithi029.stdout: "read_tier": -1, 2024-09-16T20:49:32.232 INFO:teuthology.orchestra.run.smithi029.stdout: "write_tier": -1, 2024-09-16T20:49:32.232 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_mode": "none", 2024-09-16T20:49:32.232 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_bytes": 0, 2024-09-16T20:49:32.232 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_objects": 0, 2024-09-16T20:49:32.232 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T20:49:32.233 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T20:49:32.233 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T20:49:32.233 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_flush_age": 0, 2024-09-16T20:49:32.233 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_evict_age": 0, 2024-09-16T20:49:32.233 INFO:teuthology.orchestra.run.smithi029.stdout: "erasure_code_profile": "", 2024-09-16T20:49:32.233 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_params": { 2024-09-16T20:49:32.233 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "none" 2024-09-16T20:49:32.233 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.233 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_period": 0, 2024-09-16T20:49:32.233 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_count": 0, 2024-09-16T20:49:32.233 INFO:teuthology.orchestra.run.smithi029.stdout: "use_gmt_hitset": true, 2024-09-16T20:49:32.233 INFO:teuthology.orchestra.run.smithi029.stdout: "min_read_recency_for_promote": 0, 2024-09-16T20:49:32.233 INFO:teuthology.orchestra.run.smithi029.stdout: "min_write_recency_for_promote": 0, 2024-09-16T20:49:32.233 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T20:49:32.234 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_search_last_n": 0, 2024-09-16T20:49:32.234 INFO:teuthology.orchestra.run.smithi029.stdout: "grade_table": [], 2024-09-16T20:49:32.234 INFO:teuthology.orchestra.run.smithi029.stdout: "stripe_width": 0, 2024-09-16T20:49:32.234 INFO:teuthology.orchestra.run.smithi029.stdout: "expected_num_objects": 0, 2024-09-16T20:49:32.234 INFO:teuthology.orchestra.run.smithi029.stdout: "fast_read": false, 2024-09-16T20:49:32.234 INFO:teuthology.orchestra.run.smithi029.stdout: "options": {}, 2024-09-16T20:49:32.234 INFO:teuthology.orchestra.run.smithi029.stdout: "application_metadata": { 2024-09-16T20:49:32.234 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd": {} 2024-09-16T20:49:32.234 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.234 INFO:teuthology.orchestra.run.smithi029.stdout: "read_balance": { 2024-09-16T20:49:32.234 INFO:teuthology.orchestra.run.smithi029.stdout: "score_type": "Fair distribution", 2024-09-16T20:49:32.234 INFO:teuthology.orchestra.run.smithi029.stdout: "score_acting": 2.25, 2024-09-16T20:49:32.235 INFO:teuthology.orchestra.run.smithi029.stdout: "score_stable": 2.25, 2024-09-16T20:49:32.235 INFO:teuthology.orchestra.run.smithi029.stdout: "optimal_score": 1, 2024-09-16T20:49:32.235 INFO:teuthology.orchestra.run.smithi029.stdout: "raw_score_acting": 2.25, 2024-09-16T20:49:32.235 INFO:teuthology.orchestra.run.smithi029.stdout: "raw_score_stable": 2.25, 2024-09-16T20:49:32.235 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity_weighted": 1, 2024-09-16T20:49:32.235 INFO:teuthology.orchestra.run.smithi029.stdout: "average_primary_affinity": 1, 2024-09-16T20:49:32.235 INFO:teuthology.orchestra.run.smithi029.stdout: "average_primary_affinity_weighted": 1 2024-09-16T20:49:32.235 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:32.235 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.235 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.235 INFO:teuthology.orchestra.run.smithi029.stdout: "pool": 3, 2024-09-16T20:49:32.235 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_name": "pool_client.0", 2024-09-16T20:49:32.235 INFO:teuthology.orchestra.run.smithi029.stdout: "create_time": "2024-09-16T20:48:06.418889+0000", 2024-09-16T20:49:32.236 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 8193, 2024-09-16T20:49:32.236 INFO:teuthology.orchestra.run.smithi029.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T20:49:32.236 INFO:teuthology.orchestra.run.smithi029.stdout: "type": 1, 2024-09-16T20:49:32.236 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 2, 2024-09-16T20:49:32.236 INFO:teuthology.orchestra.run.smithi029.stdout: "min_size": 1, 2024-09-16T20:49:32.236 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_rule": 0, 2024-09-16T20:49:32.236 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_count": 0, 2024-09-16T20:49:32.236 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_target": 0, 2024-09-16T20:49:32.236 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T20:49:32.236 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T20:49:32.236 INFO:teuthology.orchestra.run.smithi029.stdout: "is_stretch_pool": false, 2024-09-16T20:49:32.236 INFO:teuthology.orchestra.run.smithi029.stdout: "object_hash": 2, 2024-09-16T20:49:32.236 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_autoscale_mode": "off", 2024-09-16T20:49:32.236 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num": 32, 2024-09-16T20:49:32.237 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num": 32, 2024-09-16T20:49:32.237 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num_target": 32, 2024-09-16T20:49:32.237 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_target": 32, 2024-09-16T20:49:32.237 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_pending": 32, 2024-09-16T20:49:32.237 INFO:teuthology.orchestra.run.smithi029.stdout: "last_pg_merge_meta": { 2024-09-16T20:49:32.237 INFO:teuthology.orchestra.run.smithi029.stdout: "source_pgid": "0.0", 2024-09-16T20:49:32.237 INFO:teuthology.orchestra.run.smithi029.stdout: "ready_epoch": 0, 2024-09-16T20:49:32.237 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_started": 0, 2024-09-16T20:49:32.237 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_clean": 0, 2024-09-16T20:49:32.237 INFO:teuthology.orchestra.run.smithi029.stdout: "source_version": "0'0", 2024-09-16T20:49:32.237 INFO:teuthology.orchestra.run.smithi029.stdout: "target_version": "0'0" 2024-09-16T20:49:32.237 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.237 INFO:teuthology.orchestra.run.smithi029.stdout: "last_change": "99", 2024-09-16T20:49:32.238 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend": "0", 2024-09-16T20:49:32.238 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T20:49:32.238 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T20:49:32.238 INFO:teuthology.orchestra.run.smithi029.stdout: "auid": 0, 2024-09-16T20:49:32.238 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_mode": "selfmanaged", 2024-09-16T20:49:32.238 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_seq": 30, 2024-09-16T20:49:32.238 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_epoch": 99, 2024-09-16T20:49:32.238 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_snaps": [], 2024-09-16T20:49:32.238 INFO:teuthology.orchestra.run.smithi029.stdout: "removed_snaps": "[]", 2024-09-16T20:49:32.238 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_bytes": 0, 2024-09-16T20:49:32.238 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_objects": 0, 2024-09-16T20:49:32.238 INFO:teuthology.orchestra.run.smithi029.stdout: "tiers": [], 2024-09-16T20:49:32.238 INFO:teuthology.orchestra.run.smithi029.stdout: "tier_of": -1, 2024-09-16T20:49:32.238 INFO:teuthology.orchestra.run.smithi029.stdout: "read_tier": -1, 2024-09-16T20:49:32.238 INFO:teuthology.orchestra.run.smithi029.stdout: "write_tier": -1, 2024-09-16T20:49:32.239 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_mode": "none", 2024-09-16T20:49:32.239 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_bytes": 0, 2024-09-16T20:49:32.239 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_objects": 0, 2024-09-16T20:49:32.239 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T20:49:32.239 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T20:49:32.239 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T20:49:32.239 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_flush_age": 0, 2024-09-16T20:49:32.239 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_evict_age": 0, 2024-09-16T20:49:32.239 INFO:teuthology.orchestra.run.smithi029.stdout: "erasure_code_profile": "", 2024-09-16T20:49:32.239 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_params": { 2024-09-16T20:49:32.239 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "none" 2024-09-16T20:49:32.239 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.239 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_period": 0, 2024-09-16T20:49:32.240 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_count": 0, 2024-09-16T20:49:32.240 INFO:teuthology.orchestra.run.smithi029.stdout: "use_gmt_hitset": true, 2024-09-16T20:49:32.240 INFO:teuthology.orchestra.run.smithi029.stdout: "min_read_recency_for_promote": 0, 2024-09-16T20:49:32.240 INFO:teuthology.orchestra.run.smithi029.stdout: "min_write_recency_for_promote": 0, 2024-09-16T20:49:32.240 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T20:49:32.240 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_search_last_n": 0, 2024-09-16T20:49:32.240 INFO:teuthology.orchestra.run.smithi029.stdout: "grade_table": [], 2024-09-16T20:49:32.240 INFO:teuthology.orchestra.run.smithi029.stdout: "stripe_width": 0, 2024-09-16T20:49:32.240 INFO:teuthology.orchestra.run.smithi029.stdout: "expected_num_objects": 0, 2024-09-16T20:49:32.240 INFO:teuthology.orchestra.run.smithi029.stdout: "fast_read": false, 2024-09-16T20:49:32.240 INFO:teuthology.orchestra.run.smithi029.stdout: "options": {}, 2024-09-16T20:49:32.240 INFO:teuthology.orchestra.run.smithi029.stdout: "application_metadata": { 2024-09-16T20:49:32.240 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd": {} 2024-09-16T20:49:32.240 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.241 INFO:teuthology.orchestra.run.smithi029.stdout: "read_balance": { 2024-09-16T20:49:32.241 INFO:teuthology.orchestra.run.smithi029.stdout: "score_type": "Fair distribution", 2024-09-16T20:49:32.241 INFO:teuthology.orchestra.run.smithi029.stdout: "score_acting": 1.1299999952316284, 2024-09-16T20:49:32.241 INFO:teuthology.orchestra.run.smithi029.stdout: "score_stable": 1.1299999952316284, 2024-09-16T20:49:32.241 INFO:teuthology.orchestra.run.smithi029.stdout: "optimal_score": 1, 2024-09-16T20:49:32.241 INFO:teuthology.orchestra.run.smithi029.stdout: "raw_score_acting": 1.1299999952316284, 2024-09-16T20:49:32.241 INFO:teuthology.orchestra.run.smithi029.stdout: "raw_score_stable": 1.1299999952316284, 2024-09-16T20:49:32.241 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity_weighted": 1, 2024-09-16T20:49:32.241 INFO:teuthology.orchestra.run.smithi029.stdout: "average_primary_affinity": 1, 2024-09-16T20:49:32.241 INFO:teuthology.orchestra.run.smithi029.stdout: "average_primary_affinity_weighted": 1 2024-09-16T20:49:32.241 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:32.241 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:32.242 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-16T20:49:32.242 INFO:teuthology.orchestra.run.smithi029.stdout: "osds": [ 2024-09-16T20:49:32.242 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.242 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 0, 2024-09-16T20:49:32.242 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "6c3cb2d7-7d23-4566-9ca1-d7b08e12360c", 2024-09-16T20:49:32.242 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-16T20:49:32.242 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-16T20:49:32.242 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-16T20:49:32.242 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-16T20:49:32.242 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-16T20:49:32.242 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-16T20:49:32.242 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 13, 2024-09-16T20:49:32.243 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 97, 2024-09-16T20:49:32.243 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-16T20:49:32.243 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-16T20:49:32.243 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-16T20:49:32.243 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:32.243 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.243 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:32.243 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6816", 2024-09-16T20:49:32.243 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:49:32.243 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.243 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.243 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:32.244 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6817", 2024-09-16T20:49:32.244 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:49:32.244 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:32.244 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:32.244 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.244 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-16T20:49:32.244 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:32.244 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.244 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:32.244 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6818", 2024-09-16T20:49:32.244 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:49:32.244 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.245 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.245 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:32.245 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6819", 2024-09-16T20:49:32.245 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:49:32.245 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:32.245 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:32.245 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.245 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-16T20:49:32.245 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:32.245 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.245 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:32.245 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6822", 2024-09-16T20:49:32.246 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:49:32.246 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.246 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.246 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:32.246 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6823", 2024-09-16T20:49:32.246 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:49:32.246 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:32.246 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:32.246 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.246 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-16T20:49:32.246 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:32.246 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.246 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:32.247 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6820", 2024-09-16T20:49:32.247 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:49:32.247 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.247 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.247 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:32.247 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6821", 2024-09-16T20:49:32.247 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:49:32.247 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:32.247 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:32.247 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.247 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.29:6817/2744025949", 2024-09-16T20:49:32.247 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.29:6819/2744025949", 2024-09-16T20:49:32.247 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.29:6823/2744025949", 2024-09-16T20:49:32.248 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.29:6821/2744025949", 2024-09-16T20:49:32.248 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-16T20:49:32.248 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-16T20:49:32.248 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-16T20:49:32.248 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:32.248 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.248 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.248 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 1, 2024-09-16T20:49:32.248 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "5cf8c20c-1ed2-42ae-a8b0-c88cc4e36bce", 2024-09-16T20:49:32.248 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-16T20:49:32.248 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-16T20:49:32.248 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-16T20:49:32.248 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-16T20:49:32.249 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-16T20:49:32.249 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-16T20:49:32.249 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 13, 2024-09-16T20:49:32.249 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 81, 2024-09-16T20:49:32.249 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-16T20:49:32.249 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-16T20:49:32.249 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-16T20:49:32.249 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:32.249 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.249 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:32.249 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6800", 2024-09-16T20:49:32.249 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:49:32.249 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.250 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.250 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:32.250 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6801", 2024-09-16T20:49:32.250 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:49:32.250 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:32.250 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:32.250 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.250 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-16T20:49:32.250 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:32.250 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.250 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:32.250 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6802", 2024-09-16T20:49:32.250 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:49:32.251 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.251 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.251 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:32.251 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6803", 2024-09-16T20:49:32.251 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:49:32.251 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:32.251 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:32.251 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.251 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-16T20:49:32.251 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:32.251 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.251 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:32.251 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6806", 2024-09-16T20:49:32.252 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:49:32.252 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.252 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.252 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:32.252 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6807", 2024-09-16T20:49:32.252 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:49:32.252 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:32.252 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:32.252 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.252 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-16T20:49:32.252 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:32.252 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.252 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:32.253 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6804", 2024-09-16T20:49:32.253 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:49:32.253 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.253 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.253 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:32.253 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6805", 2024-09-16T20:49:32.253 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:49:32.253 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:32.253 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:32.253 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.253 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.29:6801/208628762", 2024-09-16T20:49:32.253 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.29:6803/208628762", 2024-09-16T20:49:32.253 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.29:6807/208628762", 2024-09-16T20:49:32.254 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.29:6805/208628762", 2024-09-16T20:49:32.254 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-16T20:49:32.254 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-16T20:49:32.254 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-16T20:49:32.254 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:32.254 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.254 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.254 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 2, 2024-09-16T20:49:32.254 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "075d16da-82bf-449b-9f6e-542c9e8d810b", 2024-09-16T20:49:32.254 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-16T20:49:32.254 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-16T20:49:32.254 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-16T20:49:32.254 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-16T20:49:32.254 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-16T20:49:32.255 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-16T20:49:32.255 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 13, 2024-09-16T20:49:32.255 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 80, 2024-09-16T20:49:32.255 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-16T20:49:32.255 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-16T20:49:32.255 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-16T20:49:32.255 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:32.255 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.255 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:32.255 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6808", 2024-09-16T20:49:32.255 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:49:32.255 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.256 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.256 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:32.256 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6809", 2024-09-16T20:49:32.256 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:49:32.256 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:32.256 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:32.256 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.256 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-16T20:49:32.256 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:32.256 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.256 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:32.256 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6810", 2024-09-16T20:49:32.256 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:49:32.256 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.256 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.257 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:32.257 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6811", 2024-09-16T20:49:32.257 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:49:32.257 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:32.257 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:32.257 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.257 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-16T20:49:32.257 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:32.257 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.257 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:32.257 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6814", 2024-09-16T20:49:32.257 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:49:32.257 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.258 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.258 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:32.258 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6815", 2024-09-16T20:49:32.258 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:49:32.258 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:32.258 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:32.258 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.258 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-16T20:49:32.258 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:32.258 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.258 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:32.258 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6812", 2024-09-16T20:49:32.258 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:49:32.259 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.259 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.259 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:32.259 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6813", 2024-09-16T20:49:32.259 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:49:32.259 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:32.259 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:32.259 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.259 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.29:6809/3447426536", 2024-09-16T20:49:32.259 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.29:6811/3447426536", 2024-09-16T20:49:32.259 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.29:6815/3447426536", 2024-09-16T20:49:32.259 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.29:6813/3447426536", 2024-09-16T20:49:32.259 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-16T20:49:32.259 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-16T20:49:32.259 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-16T20:49:32.260 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:32.260 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.260 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.260 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 3, 2024-09-16T20:49:32.260 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "0612be38-3a24-41b6-aba4-5a2fe985165e", 2024-09-16T20:49:32.260 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-16T20:49:32.260 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-16T20:49:32.260 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-16T20:49:32.260 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-16T20:49:32.260 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-16T20:49:32.260 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-16T20:49:32.260 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 13, 2024-09-16T20:49:32.260 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 96, 2024-09-16T20:49:32.261 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-16T20:49:32.261 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-16T20:49:32.261 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-16T20:49:32.261 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:32.261 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.261 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:32.261 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6808", 2024-09-16T20:49:32.261 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:49:32.261 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.261 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.261 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:32.261 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6809", 2024-09-16T20:49:32.261 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:49:32.262 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:32.262 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:32.262 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.262 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-16T20:49:32.262 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:32.262 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.262 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:32.262 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6810", 2024-09-16T20:49:32.262 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:49:32.262 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.262 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.262 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:32.262 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6811", 2024-09-16T20:49:32.262 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:49:32.263 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:32.263 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:32.263 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.263 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-16T20:49:32.263 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:32.263 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.263 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:32.263 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6814", 2024-09-16T20:49:32.263 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:49:32.263 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.263 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.263 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:32.263 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6815", 2024-09-16T20:49:32.264 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:49:32.264 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:32.264 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:32.264 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.264 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-16T20:49:32.264 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:32.264 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.264 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:32.264 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6812", 2024-09-16T20:49:32.264 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:49:32.264 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.264 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.264 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:32.265 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6813", 2024-09-16T20:49:32.265 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:49:32.265 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:32.265 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:32.265 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.265 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.43:6809/3687583891", 2024-09-16T20:49:32.265 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.43:6811/3687583891", 2024-09-16T20:49:32.265 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.43:6815/3687583891", 2024-09-16T20:49:32.265 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.43:6813/3687583891", 2024-09-16T20:49:32.265 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-16T20:49:32.265 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-16T20:49:32.265 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-16T20:49:32.265 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:32.265 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.266 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.266 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 4, 2024-09-16T20:49:32.266 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "be590bf8-e9fa-44ca-a471-6f6bcb087ca7", 2024-09-16T20:49:32.266 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-16T20:49:32.266 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-16T20:49:32.266 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-16T20:49:32.266 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-16T20:49:32.266 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-16T20:49:32.266 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-16T20:49:32.266 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 11, 2024-09-16T20:49:32.266 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 81, 2024-09-16T20:49:32.266 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-16T20:49:32.266 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-16T20:49:32.267 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-16T20:49:32.267 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:32.267 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.267 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:32.267 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6800", 2024-09-16T20:49:32.267 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:49:32.267 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.267 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.267 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:32.267 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6801", 2024-09-16T20:49:32.267 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:49:32.267 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:32.267 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:32.268 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.268 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-16T20:49:32.268 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:32.268 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.268 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:32.268 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6802", 2024-09-16T20:49:32.268 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:49:32.268 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.268 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.268 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:32.268 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6803", 2024-09-16T20:49:32.268 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:49:32.268 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:32.269 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:32.269 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.269 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-16T20:49:32.269 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:32.269 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.269 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:32.269 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6806", 2024-09-16T20:49:32.269 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:49:32.269 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.269 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.269 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:32.269 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6807", 2024-09-16T20:49:32.269 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:49:32.270 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:32.270 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:32.270 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.270 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-16T20:49:32.270 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:32.270 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.270 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:32.270 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6804", 2024-09-16T20:49:32.270 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:49:32.270 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.270 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.270 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:32.270 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6805", 2024-09-16T20:49:32.271 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:49:32.271 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:32.271 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:32.271 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.271 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.43:6801/2505220646", 2024-09-16T20:49:32.271 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.43:6803/2505220646", 2024-09-16T20:49:32.271 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.43:6807/2505220646", 2024-09-16T20:49:32.271 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.43:6805/2505220646", 2024-09-16T20:49:32.271 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-16T20:49:32.271 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-16T20:49:32.271 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-16T20:49:32.271 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:32.271 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.272 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.272 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 5, 2024-09-16T20:49:32.272 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "4a208e30-031b-4a9f-af52-e26d4a039ea6", 2024-09-16T20:49:32.272 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-16T20:49:32.272 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-16T20:49:32.272 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-16T20:49:32.272 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-16T20:49:32.272 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-16T20:49:32.272 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-16T20:49:32.272 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 13, 2024-09-16T20:49:32.272 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 91, 2024-09-16T20:49:32.272 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-16T20:49:32.272 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-16T20:49:32.272 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-16T20:49:32.272 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:32.273 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.273 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:32.273 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6816", 2024-09-16T20:49:32.273 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:49:32.273 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.273 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.273 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:32.273 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6817", 2024-09-16T20:49:32.273 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:49:32.273 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:32.273 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:32.273 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.273 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-16T20:49:32.274 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:32.274 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.274 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:32.274 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6818", 2024-09-16T20:49:32.274 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:49:32.274 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.274 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.274 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:32.274 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6819", 2024-09-16T20:49:32.274 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:49:32.274 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:32.274 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:32.274 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.275 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-16T20:49:32.275 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:32.275 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.275 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:32.275 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6822", 2024-09-16T20:49:32.275 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:49:32.275 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.275 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.275 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:32.275 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6823", 2024-09-16T20:49:32.275 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:49:32.275 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:32.275 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:32.276 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.276 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-16T20:49:32.276 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:32.276 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.276 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:32.276 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6820", 2024-09-16T20:49:32.276 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:49:32.276 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.276 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.276 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:32.276 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6821", 2024-09-16T20:49:32.276 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:49:32.276 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:32.277 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:32.277 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.277 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.43:6817/1517367307", 2024-09-16T20:49:32.277 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.43:6819/1517367307", 2024-09-16T20:49:32.277 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.43:6823/1517367307", 2024-09-16T20:49:32.277 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.43:6821/1517367307", 2024-09-16T20:49:32.277 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-16T20:49:32.277 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-16T20:49:32.277 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-16T20:49:32.277 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:32.277 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:32.277 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-16T20:49:32.277 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_xinfo": [ 2024-09-16T20:49:32.278 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.278 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 0, 2024-09-16T20:49:32.278 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-16T20:49:32.278 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-16T20:49:32.278 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-16T20:49:32.278 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540719139924082687, 2024-09-16T20:49:32.278 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-16T20:49:32.278 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-16T20:47:47.887412+0000", 2024-09-16T20:49:32.278 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-16T20:49:32.278 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.278 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.278 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 1, 2024-09-16T20:49:32.278 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-16T20:49:32.279 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-16T20:49:32.279 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-16T20:49:32.279 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540719139924082687, 2024-09-16T20:49:32.279 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-16T20:49:32.279 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-16T20:47:47.658782+0000", 2024-09-16T20:49:32.279 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-16T20:49:32.279 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.279 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.279 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 2, 2024-09-16T20:49:32.279 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-16T20:49:32.279 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-16T20:49:32.279 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-16T20:49:32.279 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540719139924082687, 2024-09-16T20:49:32.279 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-16T20:49:32.280 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-16T20:47:47.453752+0000", 2024-09-16T20:49:32.280 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-16T20:49:32.280 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.280 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.280 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 3, 2024-09-16T20:49:32.280 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-16T20:49:32.280 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-16T20:49:32.280 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-16T20:49:32.280 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540719139924082687, 2024-09-16T20:49:32.280 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-16T20:49:32.280 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-16T20:47:47.487992+0000", 2024-09-16T20:49:32.280 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-16T20:49:32.281 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.281 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.281 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 4, 2024-09-16T20:49:32.281 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-16T20:49:32.281 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-16T20:49:32.281 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-16T20:49:32.281 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540719139924082687, 2024-09-16T20:49:32.281 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-16T20:49:32.281 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-16T20:47:46.943344+0000", 2024-09-16T20:49:32.281 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-16T20:49:32.281 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.281 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:32.281 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 5, 2024-09-16T20:49:32.281 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-16T20:49:32.282 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-16T20:49:32.282 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-16T20:49:32.282 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540719139924082687, 2024-09-16T20:49:32.282 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-16T20:49:32.282 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-16T20:47:47.901547+0000", 2024-09-16T20:49:32.282 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-16T20:49:32.282 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:32.282 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-16T20:49:32.282 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_upmap": [], 2024-09-16T20:49:32.282 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_upmap_items": [], 2024-09-16T20:49:32.282 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_upmap_primaries": [], 2024-09-16T20:49:32.282 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_temp": [], 2024-09-16T20:49:32.282 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_temp": [], 2024-09-16T20:49:32.283 INFO:teuthology.orchestra.run.smithi029.stdout: "blocklist": {}, 2024-09-16T20:49:32.283 INFO:teuthology.orchestra.run.smithi029.stdout: "range_blocklist": {}, 2024-09-16T20:49:32.283 INFO:teuthology.orchestra.run.smithi029.stdout: "erasure_code_profiles": { 2024-09-16T20:49:32.283 INFO:teuthology.orchestra.run.smithi029.stdout: "default": { 2024-09-16T20:49:32.283 INFO:teuthology.orchestra.run.smithi029.stdout: "crush-failure-domain": "osd", 2024-09-16T20:49:32.283 INFO:teuthology.orchestra.run.smithi029.stdout: "k": "2", 2024-09-16T20:49:32.283 INFO:teuthology.orchestra.run.smithi029.stdout: "m": "1", 2024-09-16T20:49:32.283 INFO:teuthology.orchestra.run.smithi029.stdout: "plugin": "jerasure", 2024-09-16T20:49:32.283 INFO:teuthology.orchestra.run.smithi029.stdout: "technique": "reed_sol_van" 2024-09-16T20:49:32.283 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:32.283 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:32.283 INFO:teuthology.orchestra.run.smithi029.stdout: "removed_snaps_queue": [], 2024-09-16T20:49:32.283 INFO:teuthology.orchestra.run.smithi029.stdout: "new_removed_snaps": [], 2024-09-16T20:49:32.283 INFO:teuthology.orchestra.run.smithi029.stdout: "new_purged_snaps": [], 2024-09-16T20:49:32.284 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_node_flags": {}, 2024-09-16T20:49:32.284 INFO:teuthology.orchestra.run.smithi029.stdout: "device_class_flags": {}, 2024-09-16T20:49:32.284 INFO:teuthology.orchestra.run.smithi029.stdout: "stretch_mode": { 2024-09-16T20:49:32.284 INFO:teuthology.orchestra.run.smithi029.stdout: "stretch_mode_enabled": false, 2024-09-16T20:49:32.284 INFO:teuthology.orchestra.run.smithi029.stdout: "stretch_bucket_count": 0, 2024-09-16T20:49:32.284 INFO:teuthology.orchestra.run.smithi029.stdout: "degraded_stretch_mode": 0, 2024-09-16T20:49:32.284 INFO:teuthology.orchestra.run.smithi029.stdout: "recovering_stretch_mode": 0, 2024-09-16T20:49:32.284 INFO:teuthology.orchestra.run.smithi029.stdout: "stretch_mode_bucket": 0 2024-09-16T20:49:32.284 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:32.284 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:32.285 INFO:tasks.thrashosds.thrasher:j is {'epoch': 99, 'fsid': 'e28af4a7-5fcd-4de8-8f77-a2b575361bb1', 'created': '2024-09-16T20:47:39.760609+0000', 'modified': '2024-09-16T20:49:31.853964+0000', 'last_up_change': '2024-09-16T20:47:50.238013+0000', 'last_in_change': '2024-09-16T20:49:08.779548+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T20:47:52.220750+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-16T20:47:56.756666+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-16T20:48:06.418889+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 30, 'snap_epoch': 99, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '6c3cb2d7-7d23-4566-9ca1-d7b08e12360c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 97, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6816', 'nonce': 2744025949}, {'type': 'v1', 'addr': '172.21.15.29:6817', 'nonce': 2744025949}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6818', 'nonce': 2744025949}, {'type': 'v1', 'addr': '172.21.15.29:6819', 'nonce': 2744025949}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6822', 'nonce': 2744025949}, {'type': 'v1', 'addr': '172.21.15.29:6823', 'nonce': 2744025949}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6820', 'nonce': 2744025949}, {'type': 'v1', 'addr': '172.21.15.29:6821', 'nonce': 2744025949}]}, 'public_addr': '172.21.15.29:6817/2744025949', 'cluster_addr': '172.21.15.29:6819/2744025949', 'heartbeat_back_addr': '172.21.15.29:6823/2744025949', 'heartbeat_front_addr': '172.21.15.29:6821/2744025949', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '5cf8c20c-1ed2-42ae-a8b0-c88cc4e36bce', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 81, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6800', 'nonce': 208628762}, {'type': 'v1', 'addr': '172.21.15.29:6801', 'nonce': 208628762}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6802', 'nonce': 208628762}, {'type': 'v1', 'addr': '172.21.15.29:6803', 'nonce': 208628762}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6806', 'nonce': 208628762}, {'type': 'v1', 'addr': '172.21.15.29:6807', 'nonce': 208628762}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6804', 'nonce': 208628762}, {'type': 'v1', 'addr': '172.21.15.29:6805', 'nonce': 208628762}]}, 'public_addr': '172.21.15.29:6801/208628762', 'cluster_addr': '172.21.15.29:6803/208628762', 'heartbeat_back_addr': '172.21.15.29:6807/208628762', 'heartbeat_front_addr': '172.21.15.29:6805/208628762', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '075d16da-82bf-449b-9f6e-542c9e8d810b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 80, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6808', 'nonce': 3447426536}, {'type': 'v1', 'addr': '172.21.15.29:6809', 'nonce': 3447426536}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6810', 'nonce': 3447426536}, {'type': 'v1', 'addr': '172.21.15.29:6811', 'nonce': 3447426536}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6814', 'nonce': 3447426536}, {'type': 'v1', 'addr': '172.21.15.29:6815', 'nonce': 3447426536}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6812', 'nonce': 3447426536}, {'type': 'v1', 'addr': '172.21.15.29:6813', 'nonce': 3447426536}]}, 'public_addr': '172.21.15.29:6809/3447426536', 'cluster_addr': '172.21.15.29:6811/3447426536', 'heartbeat_back_addr': '172.21.15.29:6815/3447426536', 'heartbeat_front_addr': '172.21.15.29:6813/3447426536', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '0612be38-3a24-41b6-aba4-5a2fe985165e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 96, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6808', 'nonce': 3687583891}, {'type': 'v1', 'addr': '172.21.15.43:6809', 'nonce': 3687583891}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6810', 'nonce': 3687583891}, {'type': 'v1', 'addr': '172.21.15.43:6811', 'nonce': 3687583891}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6814', 'nonce': 3687583891}, {'type': 'v1', 'addr': '172.21.15.43:6815', 'nonce': 3687583891}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6812', 'nonce': 3687583891}, {'type': 'v1', 'addr': '172.21.15.43:6813', 'nonce': 3687583891}]}, 'public_addr': '172.21.15.43:6809/3687583891', 'cluster_addr': '172.21.15.43:6811/3687583891', 'heartbeat_back_addr': '172.21.15.43:6815/3687583891', 'heartbeat_front_addr': '172.21.15.43:6813/3687583891', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'be590bf8-e9fa-44ca-a471-6f6bcb087ca7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 81, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6800', 'nonce': 2505220646}, {'type': 'v1', 'addr': '172.21.15.43:6801', 'nonce': 2505220646}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6802', 'nonce': 2505220646}, {'type': 'v1', 'addr': '172.21.15.43:6803', 'nonce': 2505220646}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6806', 'nonce': 2505220646}, {'type': 'v1', 'addr': '172.21.15.43:6807', 'nonce': 2505220646}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6804', 'nonce': 2505220646}, {'type': 'v1', 'addr': '172.21.15.43:6805', 'nonce': 2505220646}]}, 'public_addr': '172.21.15.43:6801/2505220646', 'cluster_addr': '172.21.15.43:6803/2505220646', 'heartbeat_back_addr': '172.21.15.43:6807/2505220646', 'heartbeat_front_addr': '172.21.15.43:6805/2505220646', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '4a208e30-031b-4a9f-af52-e26d4a039ea6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 91, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6816', 'nonce': 1517367307}, {'type': 'v1', 'addr': '172.21.15.43:6817', 'nonce': 1517367307}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6818', 'nonce': 1517367307}, {'type': 'v1', 'addr': '172.21.15.43:6819', 'nonce': 1517367307}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6822', 'nonce': 1517367307}, {'type': 'v1', 'addr': '172.21.15.43:6823', 'nonce': 1517367307}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6820', 'nonce': 1517367307}, {'type': 'v1', 'addr': '172.21.15.43:6821', 'nonce': 1517367307}]}, 'public_addr': '172.21.15.43:6817/1517367307', 'cluster_addr': '172.21.15.43:6819/1517367307', 'heartbeat_back_addr': '172.21.15.43:6823/1517367307', 'heartbeat_front_addr': '172.21.15.43:6821/1517367307', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T20:47:47.887412+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T20:47:47.658782+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T20:47:47.453752+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T20:47:47.487992+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T20:47:46.943344+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T20:47:47.901547+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-16T20:49:32.285 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T20:49:32.329 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:32.328+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:32.392 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:32.392 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:49:32.392 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:49:32.392 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:32.401 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:49:32.443 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:32.441+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:32.521 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:32.520+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:32.553 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:32.553 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:49:32.553 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:49:32.553 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:49:32.553 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:32.562 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:49:32.598 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:49:32.598 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-09-16T20:49:32.610 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":146,"stamp":"2024-09-16T20:49:32.220909+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1140603369,"num_objects":996,"num_object_clones":297,"num_object_copies":1992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":996,"num_whiteouts":231,"num_read":193554,"num_read_kb":824469,"num_write":12382,"num_write_kb":233961,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1460,"num_bytes_recovered":59731590,"num_keys_recovered":365,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10230,"ondisk_log_size":10230,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":520164,"kb_used_data":310820,"kb_used_omap":9,"kb_used_meta":209206,"kb_avail":565710876,"statfs":{"total":579820584960,"available":579287937024,"internally_reserved":0,"allocated":318279680,"data_stored":340715533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9567,"internal_metadata":214227617},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,2],"upper_bound":2048},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-11525387,"num_objects":28,"num_object_clones":23,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":12,"num_read":6225,"num_read_kb":24363,"num_write":1068,"num_write_kb":6697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":9175963,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000703"},"pg_stats":[{"pgid":"3.1f","version":"97'194","reported_seq":4291,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-16T20:49:31.144922+0000","last_change":"2024-09-16T20:49:12.802355+0000","last_active":"2024-09-16T20:49:31.144922+0000","last_peered":"2024-09-16T20:49:31.144922+0000","last_clean":"2024-09-16T20:49:31.144922+0000","last_became_active":"2024-09-16T20:49:12.801566+0000","last_became_peered":"2024-09-16T20:49:12.801566+0000","last_unstale":"2024-09-16T20:49:31.144922+0000","last_undegraded":"2024-09-16T20:49:31.144922+0000","last_fullsized":"2024-09-16T20:49:31.144922+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:40:41.175616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071634499999999996,"stat_sum":{"num_bytes":28770465,"num_objects":26,"num_object_clones":6,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":3,"num_read":4079,"num_read_kb":13488,"num_write":182,"num_write_kb":2259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":2125998,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"97'497","reported_seq":13625,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-16T20:49:30.557706+0000","last_change":"2024-09-16T20:48:53.732932+0000","last_active":"2024-09-16T20:49:30.557706+0000","last_peered":"2024-09-16T20:49:30.557706+0000","last_clean":"2024-09-16T20:49:30.557706+0000","last_became_active":"2024-09-16T20:48:53.732081+0000","last_became_peered":"2024-09-16T20:48:53.732081+0000","last_unstale":"2024-09-16T20:49:30.557706+0000","last_undegraded":"2024-09-16T20:49:30.557706+0000","last_fullsized":"2024-09-16T20:49:30.557706+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":497,"log_dups_size":0,"ondisk_log_size":497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:35:24.553528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084864999999999995,"stat_sum":{"num_bytes":45215905,"num_objects":32,"num_object_clones":8,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":5,"num_read":14778,"num_read_kb":43309,"num_write":622,"num_write_kb":5115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":1415412,"num_keys_recovered":59,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"97'418","reported_seq":7744,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-16T20:49:31.145675+0000","last_change":"2024-09-16T20:48:09.564507+0000","last_active":"2024-09-16T20:49:31.145675+0000","last_peered":"2024-09-16T20:49:31.145675+0000","last_clean":"2024-09-16T20:49:31.145675+0000","last_became_active":"2024-09-16T20:48:07.568025+0000","last_became_peered":"2024-09-16T20:48:07.568025+0000","last_unstale":"2024-09-16T20:49:31.145675+0000","last_undegraded":"2024-09-16T20:49:31.145675+0000","last_fullsized":"2024-09-16T20:49:31.145675+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":418,"log_dups_size":0,"ondisk_log_size":418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:10:11.415794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068985699999999999,"stat_sum":{"num_bytes":62503490,"num_objects":35,"num_object_clones":11,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":6,"num_read":8682,"num_read_kb":71333,"num_write":537,"num_write_kb":18363,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"97'396","reported_seq":2083,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-16T20:49:30.885639+0000","last_change":"2024-09-16T20:48:09.565661+0000","last_active":"2024-09-16T20:49:30.885639+0000","last_peered":"2024-09-16T20:49:30.885639+0000","last_clean":"2024-09-16T20:49:30.885639+0000","last_became_active":"2024-09-16T20:48:07.565241+0000","last_became_peered":"2024-09-16T20:48:07.565241+0000","last_unstale":"2024-09-16T20:49:30.885639+0000","last_undegraded":"2024-09-16T20:49:30.885639+0000","last_fullsized":"2024-09-16T20:49:30.885639+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":396,"log_dups_size":0,"ondisk_log_size":396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:15:18.667304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001045589,"stat_sum":{"num_bytes":16523829,"num_objects":26,"num_object_clones":5,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":4,"num_read":3130,"num_read_kb":18240,"num_write":646,"num_write_kb":12499,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"96'543","reported_seq":3860,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-16T20:49:30.557588+0000","last_change":"2024-09-16T20:48:54.735381+0000","last_active":"2024-09-16T20:49:30.557588+0000","last_peered":"2024-09-16T20:49:30.557588+0000","last_clean":"2024-09-16T20:49:30.557588+0000","last_became_active":"2024-09-16T20:48:54.734278+0000","last_became_peered":"2024-09-16T20:48:54.734278+0000","last_unstale":"2024-09-16T20:49:30.557588+0000","last_undegraded":"2024-09-16T20:49:30.557588+0000","last_fullsized":"2024-09-16T20:49:30.557588+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":543,"log_dups_size":0,"ondisk_log_size":543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:26:33.651233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057432399999999997,"stat_sum":{"num_bytes":22188856,"num_objects":37,"num_object_clones":10,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":9,"num_read":4980,"num_read_kb":20310,"num_write":810,"num_write_kb":10152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":1963751,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"97'722","reported_seq":9370,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-16T20:49:31.182388+0000","last_change":"2024-09-16T20:49:12.803894+0000","last_active":"2024-09-16T20:49:31.182388+0000","last_peered":"2024-09-16T20:49:31.182388+0000","last_clean":"2024-09-16T20:49:31.182388+0000","last_became_active":"2024-09-16T20:49:12.802869+0000","last_became_peered":"2024-09-16T20:49:12.802869+0000","last_unstale":"2024-09-16T20:49:31.182388+0000","last_undegraded":"2024-09-16T20:49:31.182388+0000","last_fullsized":"2024-09-16T20:49:31.182388+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":722,"log_dups_size":0,"ondisk_log_size":722,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:46:44.989391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002037849,"stat_sum":{"num_bytes":12147140,"num_objects":43,"num_object_clones":12,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":10,"num_read":13278,"num_read_kb":31323,"num_write":925,"num_write_kb":5783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":1051004,"num_keys_recovered":46,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"93'256","reported_seq":4307,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-16T20:49:27.933888+0000","last_change":"2024-09-16T20:48:53.732127+0000","last_active":"2024-09-16T20:49:27.933888+0000","last_peered":"2024-09-16T20:49:27.933888+0000","last_clean":"2024-09-16T20:49:27.933888+0000","last_became_active":"2024-09-16T20:48:53.731485+0000","last_became_peered":"2024-09-16T20:48:53.731485+0000","last_unstale":"2024-09-16T20:49:27.933888+0000","last_undegraded":"2024-09-16T20:49:27.933888+0000","last_fullsized":"2024-09-16T20:49:27.933888+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":256,"log_dups_size":0,"ondisk_log_size":256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:11:58.913893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057190699999999997,"stat_sum":{"num_bytes":26571350,"num_objects":33,"num_object_clones":12,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":5,"num_read":4074,"num_read_kb":19463,"num_write":301,"num_write_kb":6010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":5091328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"93'253","reported_seq":5325,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-16T20:49:30.557421+0000","last_change":"2024-09-16T20:48:53.731382+0000","last_active":"2024-09-16T20:49:30.557421+0000","last_peered":"2024-09-16T20:49:30.557421+0000","last_clean":"2024-09-16T20:49:30.557421+0000","last_became_active":"2024-09-16T20:48:53.730714+0000","last_became_peered":"2024-09-16T20:48:53.730714+0000","last_unstale":"2024-09-16T20:49:30.557421+0000","last_undegraded":"2024-09-16T20:49:30.557421+0000","last_fullsized":"2024-09-16T20:49:30.557421+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":253,"log_dups_size":0,"ondisk_log_size":253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:51:19.362636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011061790000000001,"stat_sum":{"num_bytes":17353744,"num_objects":28,"num_object_clones":8,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":7,"num_read":6617,"num_read_kb":25371,"num_write":203,"num_write_kb":5445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2412544,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"98'545","reported_seq":5618,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-16T20:49:30.876143+0000","last_change":"2024-09-16T20:49:12.804399+0000","last_active":"2024-09-16T20:49:30.876143+0000","last_peered":"2024-09-16T20:49:30.876143+0000","last_clean":"2024-09-16T20:49:30.876143+0000","last_became_active":"2024-09-16T20:49:12.803628+0000","last_became_peered":"2024-09-16T20:49:12.803628+0000","last_unstale":"2024-09-16T20:49:30.876143+0000","last_undegraded":"2024-09-16T20:49:30.876143+0000","last_fullsized":"2024-09-16T20:49:30.876143+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":545,"log_dups_size":0,"ondisk_log_size":545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:59:45.715774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057915500000000003,"stat_sum":{"num_bytes":10021001,"num_objects":19,"num_object_clones":4,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":3,"num_read":8436,"num_read_kb":10581,"num_write":487,"num_write_kb":2847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":331889,"num_keys_recovered":60,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"98'623","reported_seq":6512,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-16T20:49:31.184733+0000","last_change":"2024-09-16T20:48:53.731777+0000","last_active":"2024-09-16T20:49:31.184733+0000","last_peered":"2024-09-16T20:49:31.184733+0000","last_clean":"2024-09-16T20:49:31.184733+0000","last_became_active":"2024-09-16T20:48:53.730899+0000","last_became_peered":"2024-09-16T20:48:53.730899+0000","last_unstale":"2024-09-16T20:49:31.184733+0000","last_undegraded":"2024-09-16T20:49:31.184733+0000","last_fullsized":"2024-09-16T20:49:31.184733+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:14:21.255624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010075049999999999,"stat_sum":{"num_bytes":24310897,"num_objects":37,"num_object_clones":10,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":10,"num_read":9092,"num_read_kb":31983,"num_write":723,"num_write_kb":6040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":6235436,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"97'180","reported_seq":4982,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-16T20:49:31.145076+0000","last_change":"2024-09-16T20:48:53.734382+0000","last_active":"2024-09-16T20:49:31.145076+0000","last_peered":"2024-09-16T20:49:31.145076+0000","last_clean":"2024-09-16T20:49:31.145076+0000","last_became_active":"2024-09-16T20:48:53.733689+0000","last_became_peered":"2024-09-16T20:48:53.733689+0000","last_unstale":"2024-09-16T20:49:31.145076+0000","last_undegraded":"2024-09-16T20:49:31.145076+0000","last_fullsized":"2024-09-16T20:49:31.145076+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:02:24.985746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051100699999999995,"stat_sum":{"num_bytes":39831552,"num_objects":36,"num_object_clones":14,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":10,"num_read":4437,"num_read_kb":16868,"num_write":175,"num_write_kb":4678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":2749952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"97'200","reported_seq":6862,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-16T20:49:31.142279+0000","last_change":"2024-09-16T20:48:09.566005+0000","last_active":"2024-09-16T20:49:31.142279+0000","last_peered":"2024-09-16T20:49:31.142279+0000","last_clean":"2024-09-16T20:49:31.142279+0000","last_became_active":"2024-09-16T20:48:07.568236+0000","last_became_peered":"2024-09-16T20:48:07.568236+0000","last_unstale":"2024-09-16T20:49:31.142279+0000","last_undegraded":"2024-09-16T20:49:31.142279+0000","last_fullsized":"2024-09-16T20:49:31.142279+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":200,"log_dups_size":0,"ondisk_log_size":200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:00:37.510497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001271749,"stat_sum":{"num_bytes":57149154,"num_objects":37,"num_object_clones":14,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":10,"num_read":6842,"num_read_kb":35846,"num_write":347,"num_write_kb":10088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":174,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-16T20:49:30.557558+0000","last_change":"2024-09-16T20:48:00.277670+0000","last_active":"2024-09-16T20:49:30.557558+0000","last_peered":"2024-09-16T20:49:30.557558+0000","last_clean":"2024-09-16T20:49:30.557558+0000","last_became_active":"2024-09-16T20:47:58.274489+0000","last_became_peered":"2024-09-16T20:47:58.274489+0000","last_unstale":"2024-09-16T20:49:30.557558+0000","last_undegraded":"2024-09-16T20:49:30.557558+0000","last_fullsized":"2024-09-16T20:49:30.557558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:27:12.633064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091339500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"93'508","reported_seq":2601,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-16T20:49:27.933976+0000","last_change":"2024-09-16T20:48:50.757942+0000","last_active":"2024-09-16T20:49:27.933976+0000","last_peered":"2024-09-16T20:49:27.933976+0000","last_clean":"2024-09-16T20:49:27.933976+0000","last_became_active":"2024-09-16T20:48:50.719161+0000","last_became_peered":"2024-09-16T20:48:50.719161+0000","last_unstale":"2024-09-16T20:49:27.933976+0000","last_undegraded":"2024-09-16T20:49:27.933976+0000","last_fullsized":"2024-09-16T20:49:27.933976+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:55:00.369583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067924999999999995,"stat_sum":{"num_bytes":15341646,"num_objects":22,"num_object_clones":5,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":3,"num_read":3965,"num_read_kb":18980,"num_write":790,"num_write_kb":10993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":466506,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":167,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-16T20:49:27.934026+0000","last_change":"2024-09-16T20:48:00.277182+0000","last_active":"2024-09-16T20:49:27.934026+0000","last_peered":"2024-09-16T20:49:27.934026+0000","last_clean":"2024-09-16T20:49:27.934026+0000","last_became_active":"2024-09-16T20:47:58.274389+0000","last_became_peered":"2024-09-16T20:47:58.274389+0000","last_unstale":"2024-09-16T20:49:27.934026+0000","last_undegraded":"2024-09-16T20:49:27.934026+0000","last_fullsized":"2024-09-16T20:49:27.934026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:28:02.441830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043659299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"93'181","reported_seq":7957,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-16T20:49:27.933960+0000","last_change":"2024-09-16T20:48:50.791999+0000","last_active":"2024-09-16T20:49:27.933960+0000","last_peered":"2024-09-16T20:49:27.933960+0000","last_clean":"2024-09-16T20:49:27.933960+0000","last_became_active":"2024-09-16T20:48:50.719868+0000","last_became_peered":"2024-09-16T20:48:50.719868+0000","last_unstale":"2024-09-16T20:49:27.933960+0000","last_undegraded":"2024-09-16T20:49:27.933960+0000","last_fullsized":"2024-09-16T20:49:27.933960+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:39:31.932757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012934750000000001,"stat_sum":{"num_bytes":59212288,"num_objects":24,"num_object_clones":9,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":7,"num_read":7574,"num_read_kb":28554,"num_write":174,"num_write_kb":11713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":523776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":174,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-16T20:49:30.557512+0000","last_change":"2024-09-16T20:48:00.278054+0000","last_active":"2024-09-16T20:49:30.557512+0000","last_peered":"2024-09-16T20:49:30.557512+0000","last_clean":"2024-09-16T20:49:30.557512+0000","last_became_active":"2024-09-16T20:47:58.272861+0000","last_became_peered":"2024-09-16T20:47:58.272861+0000","last_unstale":"2024-09-16T20:49:30.557512+0000","last_undegraded":"2024-09-16T20:49:30.557512+0000","last_fullsized":"2024-09-16T20:49:30.557512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:13:59.135547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012429489999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"93'280","reported_seq":2492,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-16T20:49:27.933882+0000","last_change":"2024-09-16T20:49:12.802627+0000","last_active":"2024-09-16T20:49:27.933882+0000","last_peered":"2024-09-16T20:49:27.933882+0000","last_clean":"2024-09-16T20:49:27.933882+0000","last_became_active":"2024-09-16T20:49:12.801810+0000","last_became_peered":"2024-09-16T20:49:12.801810+0000","last_unstale":"2024-09-16T20:49:27.933882+0000","last_undegraded":"2024-09-16T20:49:27.933882+0000","last_fullsized":"2024-09-16T20:49:27.933882+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":280,"log_dups_size":0,"ondisk_log_size":280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:48:21.026069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076367000000000004,"stat_sum":{"num_bytes":26866725,"num_objects":28,"num_object_clones":9,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":7,"num_read":2320,"num_read_kb":8330,"num_write":359,"num_write_kb":4805,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":2071114,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":49,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-16T20:49:30.564017+0000","last_change":"2024-09-16T20:49:09.557800+0000","last_active":"2024-09-16T20:49:30.564017+0000","last_peered":"2024-09-16T20:49:30.564017+0000","last_clean":"2024-09-16T20:49:30.564017+0000","last_became_active":"2024-09-16T20:49:09.557513+0000","last_became_peered":"2024-09-16T20:49:09.557513+0000","last_unstale":"2024-09-16T20:49:30.564017+0000","last_undegraded":"2024-09-16T20:49:30.564017+0000","last_fullsized":"2024-09-16T20:49:30.564017+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:38:59.143723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"97'181","reported_seq":7827,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-16T20:49:30.859782+0000","last_change":"2024-09-16T20:48:53.732314+0000","last_active":"2024-09-16T20:49:30.859782+0000","last_peered":"2024-09-16T20:49:30.859782+0000","last_clean":"2024-09-16T20:49:30.859782+0000","last_became_active":"2024-09-16T20:48:53.731287+0000","last_became_peered":"2024-09-16T20:48:53.731287+0000","last_unstale":"2024-09-16T20:49:30.859782+0000","last_undegraded":"2024-09-16T20:49:30.859782+0000","last_fullsized":"2024-09-16T20:49:30.859782+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:58:46.719402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010471600000000001,"stat_sum":{"num_bytes":59998805,"num_objects":29,"num_object_clones":7,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":7,"num_read":6709,"num_read_kb":33629,"num_write":175,"num_write_kb":9637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":176266,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"97'273","reported_seq":4718,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-16T20:49:30.559542+0000","last_change":"2024-09-16T20:48:09.570254+0000","last_active":"2024-09-16T20:49:30.559542+0000","last_peered":"2024-09-16T20:49:30.559542+0000","last_clean":"2024-09-16T20:49:30.559542+0000","last_became_active":"2024-09-16T20:48:07.568143+0000","last_became_peered":"2024-09-16T20:48:07.568143+0000","last_unstale":"2024-09-16T20:49:30.559542+0000","last_undegraded":"2024-09-16T20:49:30.559542+0000","last_fullsized":"2024-09-16T20:49:30.559542+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":273,"log_dups_size":0,"ondisk_log_size":273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:31:48.720457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038800899999999998,"stat_sum":{"num_bytes":46402675,"num_objects":45,"num_object_clones":16,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":17,"num_read":4671,"num_read_kb":11644,"num_write":300,"num_write_kb":2462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":176,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-16T20:49:30.559452+0000","last_change":"2024-09-16T20:48:00.276970+0000","last_active":"2024-09-16T20:49:30.559452+0000","last_peered":"2024-09-16T20:49:30.559452+0000","last_clean":"2024-09-16T20:49:30.559452+0000","last_became_active":"2024-09-16T20:47:58.274617+0000","last_became_peered":"2024-09-16T20:47:58.274617+0000","last_unstale":"2024-09-16T20:49:30.559452+0000","last_undegraded":"2024-09-16T20:49:30.559452+0000","last_fullsized":"2024-09-16T20:49:30.559452+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:28:54.467905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062077499999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"96'266","reported_seq":5759,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-16T20:49:30.810866+0000","last_change":"2024-09-16T20:49:12.801644+0000","last_active":"2024-09-16T20:49:30.810866+0000","last_peered":"2024-09-16T20:49:30.810866+0000","last_clean":"2024-09-16T20:49:30.810866+0000","last_became_active":"2024-09-16T20:49:12.800827+0000","last_became_peered":"2024-09-16T20:49:12.800827+0000","last_unstale":"2024-09-16T20:49:30.810866+0000","last_undegraded":"2024-09-16T20:49:30.810866+0000","last_fullsized":"2024-09-16T20:49:30.810866+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":266,"log_dups_size":0,"ondisk_log_size":266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:55:08.623741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088331900000000003,"stat_sum":{"num_bytes":36399847,"num_objects":25,"num_object_clones":4,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":4,"num_read":7249,"num_read_kb":27413,"num_write":215,"num_write_kb":4140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":1931020,"num_keys_recovered":36,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":89,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-16T20:49:30.557506+0000","last_change":"2024-09-16T20:49:09.559342+0000","last_active":"2024-09-16T20:49:30.557506+0000","last_peered":"2024-09-16T20:49:30.557506+0000","last_clean":"2024-09-16T20:49:30.557506+0000","last_became_active":"2024-09-16T20:49:09.558884+0000","last_became_peered":"2024-09-16T20:49:09.558884+0000","last_unstale":"2024-09-16T20:49:30.557506+0000","last_undegraded":"2024-09-16T20:49:30.557506+0000","last_fullsized":"2024-09-16T20:49:30.557506+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:51:23.264062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"93'163","reported_seq":4258,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-16T20:49:27.933900+0000","last_change":"2024-09-16T20:48:09.565596+0000","last_active":"2024-09-16T20:49:27.933900+0000","last_peered":"2024-09-16T20:49:27.933900+0000","last_clean":"2024-09-16T20:49:27.933900+0000","last_became_active":"2024-09-16T20:48:07.566097+0000","last_became_peered":"2024-09-16T20:48:07.566097+0000","last_unstale":"2024-09-16T20:49:27.933900+0000","last_undegraded":"2024-09-16T20:49:27.933900+0000","last_fullsized":"2024-09-16T20:49:27.933900+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:50:23.648082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012840460000000001,"stat_sum":{"num_bytes":48874496,"num_objects":27,"num_object_clones":8,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":7,"num_read":3975,"num_read_kb":27655,"num_write":167,"num_write_kb":6769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":186,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-16T20:49:30.557928+0000","last_change":"2024-09-16T20:49:09.560350+0000","last_active":"2024-09-16T20:49:30.557928+0000","last_peered":"2024-09-16T20:49:30.557928+0000","last_clean":"2024-09-16T20:49:30.557928+0000","last_became_active":"2024-09-16T20:49:09.559789+0000","last_became_peered":"2024-09-16T20:49:09.559789+0000","last_unstale":"2024-09-16T20:49:30.557928+0000","last_undegraded":"2024-09-16T20:49:30.557928+0000","last_fullsized":"2024-09-16T20:49:30.557928+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:17:54.706074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037636000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"97'293","reported_seq":3980,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-16T20:49:30.557454+0000","last_change":"2024-09-16T20:48:53.732328+0000","last_active":"2024-09-16T20:49:30.557454+0000","last_peered":"2024-09-16T20:49:30.557454+0000","last_clean":"2024-09-16T20:49:30.557454+0000","last_became_active":"2024-09-16T20:48:53.731305+0000","last_became_peered":"2024-09-16T20:48:53.731305+0000","last_unstale":"2024-09-16T20:49:30.557454+0000","last_undegraded":"2024-09-16T20:49:30.557454+0000","last_fullsized":"2024-09-16T20:49:30.557454+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:33:26.511712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063857599999999997,"stat_sum":{"num_bytes":22530079,"num_objects":38,"num_object_clones":12,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":12,"num_read":6106,"num_read_kb":16252,"num_write":222,"num_write_kb":2521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":931887,"num_keys_recovered":34,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":175,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-16T20:49:30.557616+0000","last_change":"2024-09-16T20:48:00.277739+0000","last_active":"2024-09-16T20:49:30.557616+0000","last_peered":"2024-09-16T20:49:30.557616+0000","last_clean":"2024-09-16T20:49:30.557616+0000","last_became_active":"2024-09-16T20:47:58.271174+0000","last_became_peered":"2024-09-16T20:47:58.271174+0000","last_unstale":"2024-09-16T20:49:30.557616+0000","last_undegraded":"2024-09-16T20:49:30.557616+0000","last_fullsized":"2024-09-16T20:49:30.557616+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:11:35.985075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095771799999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":289,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-16T20:49:30.557440+0000","last_change":"2024-09-16T20:49:09.565188+0000","last_active":"2024-09-16T20:49:30.557440+0000","last_peered":"2024-09-16T20:49:30.557440+0000","last_clean":"2024-09-16T20:49:30.557440+0000","last_became_active":"2024-09-16T20:49:09.559571+0000","last_became_peered":"2024-09-16T20:49:09.559571+0000","last_unstale":"2024-09-16T20:49:30.557440+0000","last_undegraded":"2024-09-16T20:49:30.557440+0000","last_fullsized":"2024-09-16T20:49:30.557440+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:52.241408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:52.241408+0000","last_clean_scrub_stamp":"2024-09-16T20:47:52.241408+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:01:53.651981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"93'439","reported_seq":2696,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-16T20:49:27.933883+0000","last_change":"2024-09-16T20:48:09.565136+0000","last_active":"2024-09-16T20:49:27.933883+0000","last_peered":"2024-09-16T20:49:27.933883+0000","last_clean":"2024-09-16T20:49:27.933883+0000","last_became_active":"2024-09-16T20:48:07.566139+0000","last_became_peered":"2024-09-16T20:48:07.566139+0000","last_unstale":"2024-09-16T20:49:27.933883+0000","last_undegraded":"2024-09-16T20:49:27.933883+0000","last_fullsized":"2024-09-16T20:49:27.933883+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:23:10.058173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001025172,"stat_sum":{"num_bytes":20286538,"num_objects":29,"num_object_clones":7,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":6,"num_read":3648,"num_read_kb":10935,"num_write":672,"num_write_kb":4642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"97'224","reported_seq":4442,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-16T20:49:31.139766+0000","last_change":"2024-09-16T20:48:53.737981+0000","last_active":"2024-09-16T20:49:31.139766+0000","last_peered":"2024-09-16T20:49:31.139766+0000","last_clean":"2024-09-16T20:49:31.139766+0000","last_became_active":"2024-09-16T20:48:53.736558+0000","last_became_peered":"2024-09-16T20:48:53.736558+0000","last_unstale":"2024-09-16T20:49:31.139766+0000","last_undegraded":"2024-09-16T20:49:31.139766+0000","last_fullsized":"2024-09-16T20:49:31.139766+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":224,"log_dups_size":0,"ondisk_log_size":224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:10:14.663756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044614999999999998,"stat_sum":{"num_bytes":43987984,"num_objects":33,"num_object_clones":12,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":9,"num_read":4815,"num_read_kb":31620,"num_write":236,"num_write_kb":9955,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":2635264,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"97'194","reported_seq":4502,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-16T20:49:30.559393+0000","last_change":"2024-09-16T20:48:09.569992+0000","last_active":"2024-09-16T20:49:30.559393+0000","last_peered":"2024-09-16T20:49:30.559393+0000","last_clean":"2024-09-16T20:49:30.559393+0000","last_became_active":"2024-09-16T20:48:07.563972+0000","last_became_peered":"2024-09-16T20:48:07.563972+0000","last_unstale":"2024-09-16T20:49:30.559393+0000","last_undegraded":"2024-09-16T20:49:30.559393+0000","last_fullsized":"2024-09-16T20:49:30.559393+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:01:22.755198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039392600000000001,"stat_sum":{"num_bytes":37776173,"num_objects":36,"num_object_clones":11,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":7,"num_read":4247,"num_read_kb":26512,"num_write":341,"num_write_kb":6379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"97'222","reported_seq":3222,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-16T20:49:31.138762+0000","last_change":"2024-09-16T20:48:54.735384+0000","last_active":"2024-09-16T20:49:31.138762+0000","last_peered":"2024-09-16T20:49:31.138762+0000","last_clean":"2024-09-16T20:49:31.138762+0000","last_became_active":"2024-09-16T20:48:54.734409+0000","last_became_peered":"2024-09-16T20:48:54.734409+0000","last_unstale":"2024-09-16T20:49:31.138762+0000","last_undegraded":"2024-09-16T20:49:31.138762+0000","last_fullsized":"2024-09-16T20:49:31.138762+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":222,"log_dups_size":0,"ondisk_log_size":222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:48:04.021977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014289540000000001,"stat_sum":{"num_bytes":37007433,"num_objects":22,"num_object_clones":6,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":5,"num_read":3045,"num_read_kb":9198,"num_write":253,"num_write_kb":4647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":971906,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"98'317","reported_seq":10617,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-16T20:49:31.139698+0000","last_change":"2024-09-16T20:49:30.561716+0000","last_active":"2024-09-16T20:49:31.139698+0000","last_peered":"2024-09-16T20:49:31.139698+0000","last_clean":"2024-09-16T20:49:31.139698+0000","last_became_active":"2024-09-16T20:49:30.560372+0000","last_became_peered":"2024-09-16T20:49:30.560372+0000","last_unstale":"2024-09-16T20:49:31.139698+0000","last_undegraded":"2024-09-16T20:49:31.139698+0000","last_fullsized":"2024-09-16T20:49:31.139698+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":317,"log_dups_size":0,"ondisk_log_size":317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:41:23.208636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047762999999999999,"stat_sum":{"num_bytes":62472091,"num_objects":23,"num_object_clones":7,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":5,"num_read":11137,"num_read_kb":45792,"num_write":472,"num_write_kb":10143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":163,"num_bytes_recovered":18718820,"num_keys_recovered":48,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"97'320","reported_seq":5866,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-16T20:49:31.142495+0000","last_change":"2024-09-16T20:48:09.565667+0000","last_active":"2024-09-16T20:49:31.142495+0000","last_peered":"2024-09-16T20:49:31.142495+0000","last_clean":"2024-09-16T20:49:31.142495+0000","last_became_active":"2024-09-16T20:48:07.567327+0000","last_became_peered":"2024-09-16T20:48:07.567327+0000","last_unstale":"2024-09-16T20:49:31.142495+0000","last_undegraded":"2024-09-16T20:49:31.142495+0000","last_fullsized":"2024-09-16T20:49:31.142495+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":320,"log_dups_size":0,"ondisk_log_size":320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:49:30.548789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012087560000000001,"stat_sum":{"num_bytes":27892864,"num_objects":34,"num_object_clones":11,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":8,"num_read":5818,"num_read_kb":17887,"num_write":266,"num_write_kb":8499,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"92'159","reported_seq":4771,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-16T20:49:31.176757+0000","last_change":"2024-09-16T20:48:09.565024+0000","last_active":"2024-09-16T20:49:31.176757+0000","last_peered":"2024-09-16T20:49:31.176757+0000","last_clean":"2024-09-16T20:49:31.176757+0000","last_became_active":"2024-09-16T20:48:07.570704+0000","last_became_peered":"2024-09-16T20:48:07.570704+0000","last_unstale":"2024-09-16T20:49:31.176757+0000","last_undegraded":"2024-09-16T20:49:31.176757+0000","last_fullsized":"2024-09-16T20:49:31.176757+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:55:50.590081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011026149999999999,"stat_sum":{"num_bytes":57826001,"num_objects":31,"num_object_clones":11,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":7,"num_read":4510,"num_read_kb":28452,"num_write":155,"num_write_kb":4059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"97'238","reported_seq":4855,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-16T20:49:31.142834+0000","last_change":"2024-09-16T20:48:09.570786+0000","last_active":"2024-09-16T20:49:31.142834+0000","last_peered":"2024-09-16T20:49:31.142834+0000","last_clean":"2024-09-16T20:49:31.142834+0000","last_became_active":"2024-09-16T20:48:07.560783+0000","last_became_peered":"2024-09-16T20:48:07.560783+0000","last_unstale":"2024-09-16T20:49:31.142834+0000","last_undegraded":"2024-09-16T20:49:31.142834+0000","last_fullsized":"2024-09-16T20:49:31.142834+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":238,"log_dups_size":0,"ondisk_log_size":238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:05:51.358204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040623999999999999,"stat_sum":{"num_bytes":24971444,"num_objects":44,"num_object_clones":16,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":11,"num_read":5459,"num_read_kb":28124,"num_write":215,"num_write_kb":9849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"93'271","reported_seq":4974,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-16T20:49:30.557320+0000","last_change":"2024-09-16T20:49:12.801692+0000","last_active":"2024-09-16T20:49:30.557320+0000","last_peered":"2024-09-16T20:49:30.557320+0000","last_clean":"2024-09-16T20:49:30.557320+0000","last_became_active":"2024-09-16T20:49:12.801013+0000","last_became_peered":"2024-09-16T20:49:12.801013+0000","last_unstale":"2024-09-16T20:49:30.557320+0000","last_undegraded":"2024-09-16T20:49:30.557320+0000","last_fullsized":"2024-09-16T20:49:30.557320+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:15:31.348237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115576,"stat_sum":{"num_bytes":53984331,"num_objects":35,"num_object_clones":13,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":12,"num_read":5569,"num_read_kb":31208,"num_write":227,"num_write_kb":3546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":99,"num_bytes_recovered":5210742,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"91'210","reported_seq":6690,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-16T20:49:30.557402+0000","last_change":"2024-09-16T20:48:09.564513+0000","last_active":"2024-09-16T20:49:30.557402+0000","last_peered":"2024-09-16T20:49:30.557402+0000","last_clean":"2024-09-16T20:49:30.557402+0000","last_became_active":"2024-09-16T20:48:07.571985+0000","last_became_peered":"2024-09-16T20:48:07.571985+0000","last_unstale":"2024-09-16T20:49:30.557402+0000","last_undegraded":"2024-09-16T20:49:30.557402+0000","last_fullsized":"2024-09-16T20:49:30.557402+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:04:22.714057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077335299999999995,"stat_sum":{"num_bytes":50754078,"num_objects":30,"num_object_clones":6,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":3,"num_read":6603,"num_read_kb":56009,"num_write":273,"num_write_kb":18882,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"97'289","reported_seq":2166,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-16T20:49:30.563892+0000","last_change":"2024-09-16T20:49:12.804699+0000","last_active":"2024-09-16T20:49:30.563892+0000","last_peered":"2024-09-16T20:49:30.563892+0000","last_clean":"2024-09-16T20:49:30.563892+0000","last_became_active":"2024-09-16T20:49:12.803689+0000","last_became_peered":"2024-09-16T20:49:12.803689+0000","last_unstale":"2024-09-16T20:49:30.563892+0000","last_undegraded":"2024-09-16T20:49:30.563892+0000","last_fullsized":"2024-09-16T20:49:30.563892+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":289,"log_dups_size":0,"ondisk_log_size":289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:41:21.859986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000840955,"stat_sum":{"num_bytes":15943247,"num_objects":21,"num_object_clones":6,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":5,"num_read":2293,"num_read_kb":8198,"num_write":378,"num_write_kb":1926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":107,"num_bytes_recovered":1536239,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"97'300","reported_seq":5079,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-16T20:49:30.557427+0000","last_change":"2024-09-16T20:48:09.565356+0000","last_active":"2024-09-16T20:49:30.557427+0000","last_peered":"2024-09-16T20:49:30.557427+0000","last_clean":"2024-09-16T20:49:30.557427+0000","last_became_active":"2024-09-16T20:48:07.565717+0000","last_became_peered":"2024-09-16T20:48:07.565717+0000","last_unstale":"2024-09-16T20:49:30.557427+0000","last_undegraded":"2024-09-16T20:49:30.557427+0000","last_fullsized":"2024-09-16T20:49:30.557427+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":300,"log_dups_size":0,"ondisk_log_size":300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:30:58.967258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013583239999999999,"stat_sum":{"num_bytes":28896854,"num_objects":27,"num_object_clones":7,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":7,"num_read":5350,"num_read_kb":19907,"num_write":431,"num_write_kb":7485,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1140012982,"num_objects":992,"num_object_clones":297,"num_object_copies":1984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":992,"num_whiteouts":231,"num_read":193488,"num_read_kb":824414,"num_write":12279,"num_write_kb":232331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1456,"num_bytes_recovered":58550854,"num_keys_recovered":365,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":310276096,"data_stored":336784745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10155,"ondisk_log_size":10155,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39704,"kb_used_data":12672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94332136,"statfs":{"total":96636764160,"available":96596107264,"internally_reserved":0,"allocated":12976128,"data_stored":12822074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":97016,"kb_used_data":61668,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94274824,"statfs":{"total":96636764160,"available":96537419776,"internally_reserved":0,"allocated":63148032,"data_stored":66407657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,2],"upper_bound":2048},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":101268,"kb_used_data":70080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94270572,"statfs":{"total":96636764160,"available":96533065728,"internally_reserved":0,"allocated":71761920,"data_stored":77248707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":92748,"kb_used_data":61560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94279092,"statfs":{"total":96636764160,"available":96541790208,"internally_reserved":0,"allocated":63037440,"data_stored":68086338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122672,"kb_used_data":69272,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94249168,"statfs":{"total":96636764160,"available":96511148032,"internally_reserved":0,"allocated":70934528,"data_stored":78410767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640279,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66756,"kb_used_data":35568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94305084,"statfs":{"total":96636764160,"available":96568406016,"internally_reserved":0,"allocated":36421632,"data_stored":37739990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69787648,"data_stored":77949046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61300736,"data_stored":67034268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":70619136,"data_stored":76787005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":62005248,"data_stored":65945955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":35323904,"data_stored":37298467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":11239424,"data_stored":11770004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T20:49:32.612 INFO:tasks.thrashosds.thrasher:Setting 3.2 to [5, 0] 2024-09-16T20:49:32.612 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.2', '5', '0'] 2024-09-16T20:49:32.612 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.2 5 0 2024-09-16T20:49:32.622 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:32.621+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:32.639 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #26, image image_client.0-clone26 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent27 2024-09-16T20:49:32.673 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:32.673 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:49:32.673 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:49:32.673 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:49:32.673 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.673 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:4904 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 7309312~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:32.673 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767724+0000", 2024-09-16T20:49:32.673 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 72.903823256999999, 2024-09-16T20:49:32.674 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050270214000000001, 2024-09-16T20:49:32.674 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:32.674 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:32.674 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:32.674 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:32.674 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:32.674 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:32.674 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4904 2024-09-16T20:49:32.674 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.674 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:32.674 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.674 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:32.674 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767724+0000", 2024-09-16T20:49:32.675 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.675 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.675 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:32.675 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767724+0000", 2024-09-16T20:49:32.675 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.675 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.675 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:32.675 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767724+0000", 2024-09-16T20:49:32.675 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1399999999999998e-07 2024-09-16T20:49:32.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.675 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.676 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:32.676 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767725+0000", 2024-09-16T20:49:32.676 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0380000000000001e-06 2024-09-16T20:49:32.676 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.676 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.676 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:32.676 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767729+0000", 2024-09-16T20:49:32.676 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2490000000000001e-06 2024-09-16T20:49:32.676 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.676 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.676 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:32.676 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767731+0000", 2024-09-16T20:49:32.676 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5940000000000001e-06 2024-09-16T20:49:32.676 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.677 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.677 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:32.677 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.817787+0000", 2024-09-16T20:49:32.677 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050055792000000002 2024-09-16T20:49:32.677 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.677 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.677 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:32.677 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.817883+0000", 2024-09-16T20:49:32.677 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5606999999999995e-05 2024-09-16T20:49:32.677 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.677 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.677 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:32.678 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.817994+0000", 2024-09-16T20:49:32.678 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011162000000000001 2024-09-16T20:49:32.678 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.678 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:32.678 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.678 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.678 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.678 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:4906 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 7317504~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:32.678 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767734+0000", 2024-09-16T20:49:32.678 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 72.903812994000006, 2024-09-16T20:49:32.678 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050406608999999998, 2024-09-16T20:49:32.678 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:32.678 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:32.678 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:32.679 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:32.679 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:32.679 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:32.679 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4906 2024-09-16T20:49:32.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.679 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:32.679 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.679 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:32.679 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767734+0000", 2024-09-16T20:49:32.679 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.679 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.679 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:32.680 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767734+0000", 2024-09-16T20:49:32.680 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.680 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.680 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:32.680 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767735+0000", 2024-09-16T20:49:32.680 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.05e-07 2024-09-16T20:49:32.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.680 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.680 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:32.680 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767736+0000", 2024-09-16T20:49:32.680 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0240000000000001e-06 2024-09-16T20:49:32.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.680 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.681 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:32.681 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767739+0000", 2024-09-16T20:49:32.681 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.067e-06 2024-09-16T20:49:32.681 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.681 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.681 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:32.681 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767742+0000", 2024-09-16T20:49:32.681 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.23e-06 2024-09-16T20:49:32.684 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.684 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.684 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:32.684 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818027+0000", 2024-09-16T20:49:32.684 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050285102999999998 2024-09-16T20:49:32.684 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.685 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.685 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:32.685 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818072+0000", 2024-09-16T20:49:32.685 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5129000000000001e-05 2024-09-16T20:49:32.685 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.685 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.685 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:32.685 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818141+0000", 2024-09-16T20:49:32.685 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8850999999999995e-05 2024-09-16T20:49:32.685 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.685 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:32.685 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.685 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.685 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.686 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:4908 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 7325696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:32.686 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767745+0000", 2024-09-16T20:49:32.686 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 72.903802599000002, 2024-09-16T20:49:32.686 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050654230000000001, 2024-09-16T20:49:32.686 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:32.686 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:32.686 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:32.686 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:32.686 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:32.686 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:32.686 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4908 2024-09-16T20:49:32.686 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.686 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:32.687 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.687 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:32.687 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767745+0000", 2024-09-16T20:49:32.687 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.687 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.687 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.687 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:32.687 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767745+0000", 2024-09-16T20:49:32.687 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.687 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.687 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.687 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:32.687 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767745+0000", 2024-09-16T20:49:32.688 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1e-07 2024-09-16T20:49:32.688 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.688 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.688 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:32.688 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767749+0000", 2024-09-16T20:49:32.688 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.532e-06 2024-09-16T20:49:32.688 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.688 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.688 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:32.688 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767753+0000", 2024-09-16T20:49:32.688 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4939999999999999e-06 2024-09-16T20:49:32.688 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.688 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.689 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:32.689 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767756+0000", 2024-09-16T20:49:32.689 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.61e-06 2024-09-16T20:49:32.689 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.689 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.689 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:32.689 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818184+0000", 2024-09-16T20:49:32.689 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050428615000000003 2024-09-16T20:49:32.689 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.689 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.689 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:32.689 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818276+0000", 2024-09-16T20:49:32.689 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2077999999999995e-05 2024-09-16T20:49:32.689 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.690 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.690 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:32.690 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818399+0000", 2024-09-16T20:49:32.690 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012269099999999999 2024-09-16T20:49:32.690 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.690 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:32.690 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.690 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.690 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.690 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:4910 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 7333888~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:32.690 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767759+0000", 2024-09-16T20:49:32.690 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 72.903788708999997, 2024-09-16T20:49:32.690 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050815076000000001, 2024-09-16T20:49:32.690 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:32.691 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:32.691 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:32.691 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:32.691 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:32.691 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:32.691 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4910 2024-09-16T20:49:32.691 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.691 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:32.691 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.691 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:32.691 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767759+0000", 2024-09-16T20:49:32.691 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.691 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.692 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.692 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:32.692 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767759+0000", 2024-09-16T20:49:32.692 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.692 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.692 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.692 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:32.692 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767759+0000", 2024-09-16T20:49:32.692 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2600000000000001e-07 2024-09-16T20:49:32.692 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.692 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.692 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:32.692 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767760+0000", 2024-09-16T20:49:32.693 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0640000000000001e-06 2024-09-16T20:49:32.693 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.693 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.693 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:32.693 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767763+0000", 2024-09-16T20:49:32.693 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.994e-06 2024-09-16T20:49:32.693 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.693 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.693 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:32.693 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767765+0000", 2024-09-16T20:49:32.693 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5220000000000002e-06 2024-09-16T20:49:32.693 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.693 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.693 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:32.694 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818438+0000", 2024-09-16T20:49:32.694 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050672602999999997 2024-09-16T20:49:32.694 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.694 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.694 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:32.694 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818481+0000", 2024-09-16T20:49:32.694 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2967999999999999e-05 2024-09-16T20:49:32.694 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.694 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.694 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:32.694 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818574+0000", 2024-09-16T20:49:32.694 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2699000000000001e-05 2024-09-16T20:49:32.694 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.695 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:32.695 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.695 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.695 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.695 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5012 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2101248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:32.695 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767768+0000", 2024-09-16T20:49:32.695 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 72.903779060000005, 2024-09-16T20:49:32.695 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051084682999999999, 2024-09-16T20:49:32.695 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:32.695 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:32.695 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:32.695 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:32.695 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:32.695 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:32.696 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5012 2024-09-16T20:49:32.696 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.696 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:32.696 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.696 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:32.696 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767768+0000", 2024-09-16T20:49:32.696 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.696 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.696 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.696 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:32.696 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767768+0000", 2024-09-16T20:49:32.696 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.696 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.697 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.697 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:32.697 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767769+0000", 2024-09-16T20:49:32.697 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1500000000000001e-07 2024-09-16T20:49:32.697 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.697 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.697 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:32.697 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767770+0000", 2024-09-16T20:49:32.697 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.015e-06 2024-09-16T20:49:32.697 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.697 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.697 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:32.697 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767773+0000", 2024-09-16T20:49:32.697 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1070000000000001e-06 2024-09-16T20:49:32.698 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.698 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.698 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:32.698 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767775+0000", 2024-09-16T20:49:32.698 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.508e-06 2024-09-16T20:49:32.698 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.698 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.698 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:32.698 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818630+0000", 2024-09-16T20:49:32.698 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050855282000000002 2024-09-16T20:49:32.698 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.698 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.698 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:32.698 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818732+0000", 2024-09-16T20:49:32.699 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101531 2024-09-16T20:49:32.699 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.699 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.699 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:32.699 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818853+0000", 2024-09-16T20:49:32.699 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000121025 2024-09-16T20:49:32.699 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.699 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:32.699 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.699 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.699 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.699 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5014 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2109440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:32.699 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767778+0000", 2024-09-16T20:49:32.700 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 72.903769123999993, 2024-09-16T20:49:32.700 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051260776000000001, 2024-09-16T20:49:32.700 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:32.700 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:32.700 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:32.700 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:32.700 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:32.700 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:32.700 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5014 2024-09-16T20:49:32.700 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.700 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:32.700 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.700 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:32.700 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767778+0000", 2024-09-16T20:49:32.701 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.701 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.701 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.701 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:32.701 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767778+0000", 2024-09-16T20:49:32.701 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.701 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.701 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.701 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:32.701 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767824+0000", 2024-09-16T20:49:32.701 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5890999999999998e-05 2024-09-16T20:49:32.701 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.701 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.702 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:32.702 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767827+0000", 2024-09-16T20:49:32.702 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.644e-06 2024-09-16T20:49:32.702 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.702 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.702 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:32.702 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767832+0000", 2024-09-16T20:49:32.702 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2689999999999999e-06 2024-09-16T20:49:32.702 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.702 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.702 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:32.702 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767835+0000", 2024-09-16T20:49:32.702 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2100000000000002e-06 2024-09-16T20:49:32.702 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.703 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.703 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:32.703 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818887+0000", 2024-09-16T20:49:32.703 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051051276999999999 2024-09-16T20:49:32.703 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.703 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.703 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:32.703 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818953+0000", 2024-09-16T20:49:32.703 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6105999999999994e-05 2024-09-16T20:49:32.703 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.703 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.703 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:32.703 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819039+0000", 2024-09-16T20:49:32.703 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6378999999999994e-05 2024-09-16T20:49:32.704 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.704 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:32.704 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.704 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.704 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.704 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5016 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2117632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:32.704 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767841+0000", 2024-09-16T20:49:32.704 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 72.903706444999997, 2024-09-16T20:49:32.704 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051433288000000001, 2024-09-16T20:49:32.704 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:32.704 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:32.704 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:32.704 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:32.704 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:32.705 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:32.705 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5016 2024-09-16T20:49:32.705 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.705 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:32.705 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.705 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:32.705 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767841+0000", 2024-09-16T20:49:32.705 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.705 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.705 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.705 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:32.705 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767841+0000", 2024-09-16T20:49:32.705 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.706 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.706 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.706 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:32.706 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768156+0000", 2024-09-16T20:49:32.706 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00031505699999999999 2024-09-16T20:49:32.706 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.706 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.706 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:32.706 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768159+0000", 2024-09-16T20:49:32.706 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1659999999999998e-06 2024-09-16T20:49:32.706 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.706 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.706 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:32.707 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768166+0000", 2024-09-16T20:49:32.707 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0550000000000002e-06 2024-09-16T20:49:32.707 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.707 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.707 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:32.707 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768171+0000", 2024-09-16T20:49:32.707 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3170000000000003e-06 2024-09-16T20:49:32.707 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.707 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.707 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:32.707 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819084+0000", 2024-09-16T20:49:32.707 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050913355 2024-09-16T20:49:32.707 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.708 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.708 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:32.708 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819170+0000", 2024-09-16T20:49:32.708 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6328000000000004e-05 2024-09-16T20:49:32.708 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.708 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.708 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:32.708 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819274+0000", 2024-09-16T20:49:32.708 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010401 2024-09-16T20:49:32.708 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.708 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:32.708 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.708 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.708 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.709 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5018 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2125824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:32.709 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768178+0000", 2024-09-16T20:49:32.709 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 72.903368911000001, 2024-09-16T20:49:32.709 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051227013000000002, 2024-09-16T20:49:32.709 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:32.709 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:32.709 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:32.709 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:32.709 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:32.709 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:32.709 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5018 2024-09-16T20:49:32.709 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.709 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:32.710 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.710 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:32.710 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768178+0000", 2024-09-16T20:49:32.710 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.710 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.710 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.710 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:32.710 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768178+0000", 2024-09-16T20:49:32.710 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.710 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.710 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.710 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:32.710 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768179+0000", 2024-09-16T20:49:32.711 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9200000000000002e-07 2024-09-16T20:49:32.711 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.711 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.711 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:32.711 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768180+0000", 2024-09-16T20:49:32.711 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3790000000000001e-06 2024-09-16T20:49:32.711 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.711 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.711 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:32.711 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768184+0000", 2024-09-16T20:49:32.711 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5470000000000001e-06 2024-09-16T20:49:32.711 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.711 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.712 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:32.712 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768187+0000", 2024-09-16T20:49:32.712 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8119999999999999e-06 2024-09-16T20:49:32.712 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.712 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.712 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:32.712 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819305+0000", 2024-09-16T20:49:32.712 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051118683999999998 2024-09-16T20:49:32.712 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.712 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.712 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:32.712 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819351+0000", 2024-09-16T20:49:32.712 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5757999999999998e-05 2024-09-16T20:49:32.713 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.713 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.713 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:32.713 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819405+0000", 2024-09-16T20:49:32.713 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4441000000000003e-05 2024-09-16T20:49:32.713 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.713 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:32.713 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.713 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.713 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.713 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5020 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2134016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:32.713 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768190+0000", 2024-09-16T20:49:32.713 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 72.903357232999994, 2024-09-16T20:49:32.714 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051449661000000001, 2024-09-16T20:49:32.714 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:32.714 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:32.714 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:32.714 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:32.714 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:32.714 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:32.714 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5020 2024-09-16T20:49:32.715 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.715 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:32.715 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.715 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:32.715 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768190+0000", 2024-09-16T20:49:32.715 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.715 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.715 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.716 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:32.716 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768190+0000", 2024-09-16T20:49:32.716 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.716 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.716 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.716 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:32.716 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768282+0000", 2024-09-16T20:49:32.716 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2263999999999996e-05 2024-09-16T20:49:32.716 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.716 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.716 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:32.716 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768286+0000", 2024-09-16T20:49:32.716 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.529e-06 2024-09-16T20:49:32.717 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.717 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.717 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:32.717 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768294+0000", 2024-09-16T20:49:32.717 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6990000000000003e-06 2024-09-16T20:49:32.717 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.717 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.717 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:32.717 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768306+0000", 2024-09-16T20:49:32.717 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2112e-05 2024-09-16T20:49:32.717 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.717 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.718 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:32.718 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819449+0000", 2024-09-16T20:49:32.718 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051143521999999997 2024-09-16T20:49:32.718 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.718 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.718 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:32.718 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819539+0000", 2024-09-16T20:49:32.718 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0022e-05 2024-09-16T20:49:32.718 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.718 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.718 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:32.718 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819640+0000", 2024-09-16T20:49:32.718 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000100513 2024-09-16T20:49:32.719 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.719 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:32.719 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.719 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.719 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.719 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5022 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2142208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:32.719 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768326+0000", 2024-09-16T20:49:32.719 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 72.903221844000001, 2024-09-16T20:49:32.719 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051436952000000001, 2024-09-16T20:49:32.719 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:32.719 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:32.719 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:32.719 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:32.720 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:32.720 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:32.720 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5022 2024-09-16T20:49:32.720 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.720 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:32.720 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.720 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:32.720 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768326+0000", 2024-09-16T20:49:32.720 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.720 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.720 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.720 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:32.720 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768326+0000", 2024-09-16T20:49:32.721 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.721 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.721 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.721 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:32.721 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768477+0000", 2024-09-16T20:49:32.721 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015109300000000001 2024-09-16T20:49:32.721 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.721 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.721 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:32.721 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768480+0000", 2024-09-16T20:49:32.721 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8580000000000002e-06 2024-09-16T20:49:32.721 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.721 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.722 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:32.722 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768489+0000", 2024-09-16T20:49:32.722 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4600000000000003e-06 2024-09-16T20:49:32.722 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.722 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.722 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:32.722 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768493+0000", 2024-09-16T20:49:32.722 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2270000000000004e-06 2024-09-16T20:49:32.722 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.722 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.722 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:32.722 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819672+0000", 2024-09-16T20:49:32.722 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051178605000000002 2024-09-16T20:49:32.723 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.723 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.723 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:32.723 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819711+0000", 2024-09-16T20:49:32.723 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9357000000000003e-05 2024-09-16T20:49:32.723 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.723 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.723 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:32.723 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819762+0000", 2024-09-16T20:49:32.723 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1351999999999999e-05 2024-09-16T20:49:32.723 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.723 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:32.723 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.723 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.724 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.724 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5024 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2150400~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:32.724 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768501+0000", 2024-09-16T20:49:32.724 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 72.903046365999998, 2024-09-16T20:49:32.724 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051402678, 2024-09-16T20:49:32.724 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:32.724 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:32.724 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:32.724 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:32.724 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:32.724 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:32.724 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5024 2024-09-16T20:49:32.724 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.724 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:32.725 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.725 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:32.725 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768501+0000", 2024-09-16T20:49:32.725 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.725 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.725 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.725 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:32.725 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768501+0000", 2024-09-16T20:49:32.725 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.725 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.725 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.725 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:32.726 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768619+0000", 2024-09-16T20:49:32.726 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118187 2024-09-16T20:49:32.726 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.726 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.726 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:32.726 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768624+0000", 2024-09-16T20:49:32.726 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3259999999999997e-06 2024-09-16T20:49:32.726 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.726 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.726 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:32.726 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768631+0000", 2024-09-16T20:49:32.726 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6490000000000008e-06 2024-09-16T20:49:32.726 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.726 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.727 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:32.727 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768637+0000", 2024-09-16T20:49:32.727 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4210000000000003e-06 2024-09-16T20:49:32.727 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.727 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.727 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:32.727 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819800+0000", 2024-09-16T20:49:32.727 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051163486000000001 2024-09-16T20:49:32.727 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.727 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.727 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:32.727 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819845+0000", 2024-09-16T20:49:32.727 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4830999999999998e-05 2024-09-16T20:49:32.728 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.728 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.728 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:32.728 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819904+0000", 2024-09-16T20:49:32.728 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8777999999999999e-05 2024-09-16T20:49:32.728 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.728 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:32.728 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.728 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.728 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.728 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5026 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2158592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:32.728 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768646+0000", 2024-09-16T20:49:32.728 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 72.902900918, 2024-09-16T20:49:32.729 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051384386999999997, 2024-09-16T20:49:32.729 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:32.729 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:32.729 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:32.729 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:32.729 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:32.729 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:32.729 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5026 2024-09-16T20:49:32.729 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.729 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:32.729 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.729 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:32.729 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768646+0000", 2024-09-16T20:49:32.730 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.730 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.730 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.730 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:32.730 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768646+0000", 2024-09-16T20:49:32.730 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.730 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.730 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.730 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:32.730 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768775+0000", 2024-09-16T20:49:32.730 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012813200000000001 2024-09-16T20:49:32.730 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.730 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.731 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:32.731 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768778+0000", 2024-09-16T20:49:32.731 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2150000000000001e-06 2024-09-16T20:49:32.731 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.731 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.731 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:32.731 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768786+0000", 2024-09-16T20:49:32.731 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3259999999999996e-06 2024-09-16T20:49:32.731 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.731 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.731 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:32.731 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768790+0000", 2024-09-16T20:49:32.731 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.216e-06 2024-09-16T20:49:32.731 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.732 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.732 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:32.732 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819939+0000", 2024-09-16T20:49:32.732 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051149018999999997 2024-09-16T20:49:32.732 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.732 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.732 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:32.732 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819979+0000", 2024-09-16T20:49:32.732 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9311999999999999e-05 2024-09-16T20:49:32.732 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.732 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.732 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:32.732 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820031+0000", 2024-09-16T20:49:32.733 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2166999999999997e-05 2024-09-16T20:49:32.733 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.733 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:32.733 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.733 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.733 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.733 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5028 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2166784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:32.733 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768798+0000", 2024-09-16T20:49:32.733 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 72.902749275000005, 2024-09-16T20:49:32.733 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051388912000000002, 2024-09-16T20:49:32.733 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:32.733 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:32.733 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:32.734 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:32.734 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:32.734 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:32.734 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5028 2024-09-16T20:49:32.734 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.734 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:32.734 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.734 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:32.734 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768798+0000", 2024-09-16T20:49:32.734 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.734 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.734 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.734 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:32.735 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768798+0000", 2024-09-16T20:49:32.735 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.735 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.735 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.735 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:32.735 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768989+0000", 2024-09-16T20:49:32.735 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000190951 2024-09-16T20:49:32.735 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.735 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.735 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:32.735 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768993+0000", 2024-09-16T20:49:32.735 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9770000000000002e-06 2024-09-16T20:49:32.735 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.736 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.736 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:32.736 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.769002+0000", 2024-09-16T20:49:32.736 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5469999999999997e-06 2024-09-16T20:49:32.736 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.736 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.736 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:32.736 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.769020+0000", 2024-09-16T20:49:32.736 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8216999999999999e-05 2024-09-16T20:49:32.736 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.736 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.736 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:32.736 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820070+0000", 2024-09-16T20:49:32.737 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051049998999999999 2024-09-16T20:49:32.737 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.737 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.737 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:32.737 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820119+0000", 2024-09-16T20:49:32.737 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8962000000000003e-05 2024-09-16T20:49:32.737 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.737 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.737 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:32.737 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820187+0000", 2024-09-16T20:49:32.737 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8258999999999998e-05 2024-09-16T20:49:32.737 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.737 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:32.738 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.738 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.738 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.738 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5030 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2174976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:32.738 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.769031+0000", 2024-09-16T20:49:32.738 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 72.902516161999998, 2024-09-16T20:49:32.738 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051289830000000002, 2024-09-16T20:49:32.738 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:32.738 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:32.738 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:32.738 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:32.738 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:32.738 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:32.739 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5030 2024-09-16T20:49:32.739 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.739 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:32.739 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.739 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:32.739 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.769031+0000", 2024-09-16T20:49:32.739 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.739 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.739 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.739 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:32.739 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.769031+0000", 2024-09-16T20:49:32.739 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.739 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.739 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.740 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:32.740 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.773332+0000", 2024-09-16T20:49:32.740 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0043004039999999999 2024-09-16T20:49:32.740 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.740 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.740 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:32.740 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.773336+0000", 2024-09-16T20:49:32.740 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7849999999999999e-06 2024-09-16T20:49:32.740 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.740 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.740 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:32.740 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.773347+0000", 2024-09-16T20:49:32.740 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0482999999999999e-05 2024-09-16T20:49:32.741 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.741 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.741 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:32.741 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.773354+0000", 2024-09-16T20:49:32.741 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1559999999999998e-06 2024-09-16T20:49:32.741 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.741 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.741 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:32.741 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820216+0000", 2024-09-16T20:49:32.741 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046861591000000001 2024-09-16T20:49:32.741 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.741 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.741 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:32.742 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820258+0000", 2024-09-16T20:49:32.742 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2653e-05 2024-09-16T20:49:32.742 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.742 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.742 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:32.742 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820321+0000", 2024-09-16T20:49:32.742 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2757999999999998e-05 2024-09-16T20:49:32.742 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.742 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:32.742 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.742 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.742 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.742 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5067 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 6918144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:32.743 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.775279+0000", 2024-09-16T20:49:32.743 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 72.896267895999998, 2024-09-16T20:49:32.743 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050171387999999997, 2024-09-16T20:49:32.743 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:32.743 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:32.743 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:32.743 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:32.743 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:32.743 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:32.743 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5067 2024-09-16T20:49:32.743 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.743 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:32.743 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.744 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:32.744 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775279+0000", 2024-09-16T20:49:32.744 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.744 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.744 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.744 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:32.744 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775279+0000", 2024-09-16T20:49:32.744 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.744 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.744 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.744 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:32.744 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775432+0000", 2024-09-16T20:49:32.744 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015257399999999999 2024-09-16T20:49:32.745 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.745 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.745 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:32.745 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775436+0000", 2024-09-16T20:49:32.745 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0180000000000003e-06 2024-09-16T20:49:32.745 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.745 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.745 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:32.745 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775447+0000", 2024-09-16T20:49:32.745 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.077e-05 2024-09-16T20:49:32.745 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.745 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.745 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:32.745 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775456+0000", 2024-09-16T20:49:32.745 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4089999999999992e-06 2024-09-16T20:49:32.746 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.746 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.746 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:32.746 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825150+0000", 2024-09-16T20:49:32.746 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049693711000000002 2024-09-16T20:49:32.746 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.746 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.746 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:32.746 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825337+0000", 2024-09-16T20:49:32.746 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000187214 2024-09-16T20:49:32.746 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.746 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.746 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:32.747 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825451+0000", 2024-09-16T20:49:32.747 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011369199999999999 2024-09-16T20:49:32.747 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.747 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:32.747 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.747 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.747 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.747 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5068 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2330624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:32.747 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.775469+0000", 2024-09-16T20:49:32.747 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 72.896077934999994, 2024-09-16T20:49:32.747 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050198334999999997, 2024-09-16T20:49:32.747 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:32.747 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:32.747 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:32.747 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:32.748 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:32.748 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:32.748 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5068 2024-09-16T20:49:32.748 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.748 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:32.748 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.748 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:32.748 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775469+0000", 2024-09-16T20:49:32.748 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.748 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.748 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.748 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:32.748 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775469+0000", 2024-09-16T20:49:32.748 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.748 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.749 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.749 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:32.749 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775720+0000", 2024-09-16T20:49:32.749 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000250661 2024-09-16T20:49:32.749 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.749 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.749 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:32.749 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775724+0000", 2024-09-16T20:49:32.749 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8700000000000002e-06 2024-09-16T20:49:32.749 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.749 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.749 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:32.749 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775733+0000", 2024-09-16T20:49:32.749 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4239999999999999e-06 2024-09-16T20:49:32.750 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.750 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.750 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:32.750 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775740+0000", 2024-09-16T20:49:32.750 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9519999999999997e-06 2024-09-16T20:49:32.750 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.750 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.750 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:32.750 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825484+0000", 2024-09-16T20:49:32.750 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049743385000000001 2024-09-16T20:49:32.750 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.750 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.751 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:32.751 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825572+0000", 2024-09-16T20:49:32.751 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7834e-05 2024-09-16T20:49:32.751 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.751 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.751 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:32.751 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825668+0000", 2024-09-16T20:49:32.751 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6209e-05 2024-09-16T20:49:32.751 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.751 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:32.751 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.751 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.751 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.752 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5069 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 6926336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:32.752 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.775753+0000", 2024-09-16T20:49:32.752 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 72.895794711999997, 2024-09-16T20:49:32.752 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050146430999999998, 2024-09-16T20:49:32.752 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:32.752 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:32.752 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:32.752 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:32.752 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:32.752 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:32.752 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5069 2024-09-16T20:49:32.752 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.753 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:32.753 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.753 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:32.753 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775753+0000", 2024-09-16T20:49:32.753 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.753 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.753 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.753 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:32.753 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775753+0000", 2024-09-16T20:49:32.753 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.753 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.753 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.753 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:32.753 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775865+0000", 2024-09-16T20:49:32.754 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000112402 2024-09-16T20:49:32.754 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.754 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.754 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:32.754 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775869+0000", 2024-09-16T20:49:32.754 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5130000000000001e-06 2024-09-16T20:49:32.754 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.754 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.754 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:32.754 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775877+0000", 2024-09-16T20:49:32.754 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1359999999999997e-06 2024-09-16T20:49:32.754 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.754 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.755 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:32.755 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775882+0000", 2024-09-16T20:49:32.755 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6710000000000004e-06 2024-09-16T20:49:32.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.755 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.755 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:32.755 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825729+0000", 2024-09-16T20:49:32.755 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049846898000000001 2024-09-16T20:49:32.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.755 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.755 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:32.755 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825807+0000", 2024-09-16T20:49:32.755 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7875999999999998e-05 2024-09-16T20:49:32.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.755 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.756 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:32.756 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825899+0000", 2024-09-16T20:49:32.756 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1935000000000007e-05 2024-09-16T20:49:32.756 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.756 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:32.756 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.756 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.756 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.756 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5070 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2338816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:32.756 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.775892+0000", 2024-09-16T20:49:32.756 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 72.895654858, 2024-09-16T20:49:32.756 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050155311000000001, 2024-09-16T20:49:32.756 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:32.756 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:32.757 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:32.757 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:32.757 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:32.757 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:32.757 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5070 2024-09-16T20:49:32.757 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.757 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:32.757 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.757 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:32.757 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775892+0000", 2024-09-16T20:49:32.757 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.757 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.757 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.757 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:32.758 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775892+0000", 2024-09-16T20:49:32.758 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.758 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.758 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.758 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:32.758 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776142+0000", 2024-09-16T20:49:32.758 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00024950500000000002 2024-09-16T20:49:32.758 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.758 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.758 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:32.758 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776146+0000", 2024-09-16T20:49:32.758 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2819999999999998e-06 2024-09-16T20:49:32.758 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.758 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.759 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:32.759 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776157+0000", 2024-09-16T20:49:32.759 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0807e-05 2024-09-16T20:49:32.759 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.759 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.759 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:32.759 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776165+0000", 2024-09-16T20:49:32.759 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1300000000000001e-06 2024-09-16T20:49:32.759 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.759 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.759 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:32.759 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825935+0000", 2024-09-16T20:49:32.760 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049769300000000002 2024-09-16T20:49:32.760 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.760 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.760 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:32.760 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825991+0000", 2024-09-16T20:49:32.760 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6177000000000001e-05 2024-09-16T20:49:32.760 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.760 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.760 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:32.760 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826048+0000", 2024-09-16T20:49:32.760 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.711e-05 2024-09-16T20:49:32.760 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.760 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:32.761 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.761 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.761 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.761 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5071 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 6934528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:32.761 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.776177+0000", 2024-09-16T20:49:32.761 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 72.895370455000005, 2024-09-16T20:49:32.761 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050036886000000003, 2024-09-16T20:49:32.761 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:32.761 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:32.761 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:32.761 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:32.761 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:32.761 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:32.762 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5071 2024-09-16T20:49:32.762 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.762 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:32.762 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.762 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:32.762 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776177+0000", 2024-09-16T20:49:32.762 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.762 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.762 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.762 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:32.762 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776177+0000", 2024-09-16T20:49:32.762 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.762 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.763 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.763 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:32.763 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776339+0000", 2024-09-16T20:49:32.763 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016238099999999999 2024-09-16T20:49:32.763 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.763 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.763 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:32.763 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776343+0000", 2024-09-16T20:49:32.763 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9670000000000003e-06 2024-09-16T20:49:32.763 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.763 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.763 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:32.763 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776354+0000", 2024-09-16T20:49:32.763 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0789e-05 2024-09-16T20:49:32.764 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.764 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.764 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:32.764 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776361+0000", 2024-09-16T20:49:32.764 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5200000000000003e-06 2024-09-16T20:49:32.764 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.764 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.764 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:32.764 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826078+0000", 2024-09-16T20:49:32.764 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049717513999999997 2024-09-16T20:49:32.764 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.764 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.765 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:32.765 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826141+0000", 2024-09-16T20:49:32.765 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2446000000000005e-05 2024-09-16T20:49:32.765 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.765 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.765 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:32.765 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826214+0000", 2024-09-16T20:49:32.765 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3269000000000006e-05 2024-09-16T20:49:32.765 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.765 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:32.765 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.765 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.765 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.766 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5072 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2347008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:32.766 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.776370+0000", 2024-09-16T20:49:32.766 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 72.895176894000002, 2024-09-16T20:49:32.766 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050097653999999998, 2024-09-16T20:49:32.766 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:32.766 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:32.766 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:32.766 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:32.766 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:32.766 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:32.766 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5072 2024-09-16T20:49:32.766 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.766 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:32.766 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.767 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:32.767 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776370+0000", 2024-09-16T20:49:32.767 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.767 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.767 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.767 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:32.767 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776370+0000", 2024-09-16T20:49:32.767 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:32.767 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.767 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.767 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:32.767 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776645+0000", 2024-09-16T20:49:32.767 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000274634 2024-09-16T20:49:32.767 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.768 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.768 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:32.768 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776654+0000", 2024-09-16T20:49:32.768 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9390000000000003e-06 2024-09-16T20:49:32.768 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.768 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.768 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:32.768 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776665+0000", 2024-09-16T20:49:32.768 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0524e-05 2024-09-16T20:49:32.768 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.768 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.768 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:32.768 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776671+0000", 2024-09-16T20:49:32.768 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7390000000000002e-06 2024-09-16T20:49:32.769 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.769 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.769 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:32.769 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826261+0000", 2024-09-16T20:49:32.769 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049589995999999997 2024-09-16T20:49:32.769 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.769 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.769 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:32.769 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826363+0000", 2024-09-16T20:49:32.769 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010158999999999999 2024-09-16T20:49:32.769 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:32.769 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:32.769 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:32.770 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826468+0000", 2024-09-16T20:49:32.770 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000105232 2024-09-16T20:49:32.770 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.770 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:32.770 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.770 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:32.770 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:32.770 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:32.772 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:32.770+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:32.773 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:49:32.774 INFO:teuthology.orchestra.run.smithi139.stdout:3781 read 0xcaea400 thru 0xcaf1519 (0x711a bytes) 2024-09-16T20:49:32.774 INFO:teuthology.orchestra.run.smithi139.stdout:3782 punch from 0xbbd200 to 0xbca200, (0xd000 bytes) 2024-09-16T20:49:32.774 INFO:teuthology.orchestra.run.smithi139.stdout:3783 write 0xae9e400 thru 0xaea67b4 (0x83b5 bytes) 2024-09-16T20:49:32.774 INFO:teuthology.orchestra.run.smithi139.stdout:3785 trunc from 0xd8a5200 to 0xce13200 2024-09-16T20:49:32.774 INFO:teuthology.orchestra.run.smithi139.stdout:3787 read 0x6201a00 thru 0x620fe85 (0xe486 bytes) 2024-09-16T20:49:32.774 INFO:teuthology.orchestra.run.smithi139.stdout:3788 trunc from 0xce13200 to 0xdadb000 2024-09-16T20:49:32.774 INFO:teuthology.orchestra.run.smithi139.stdout:3789 punch from 0xbbd3c00 to 0xbbe1e00, (0xe200 bytes) 2024-09-16T20:49:32.774 INFO:teuthology.orchestra.run.smithi139.stdout:3790 read 0xb44e000 thru 0xb4557cc (0x77cd bytes) 2024-09-16T20:49:32.774 INFO:teuthology.orchestra.run.smithi139.stdout:3791 write 0x11af400 thru 0x11b0d2f (0x1930 bytes) 2024-09-16T20:49:32.774 INFO:teuthology.orchestra.run.smithi139.stdout:3793 write 0x8efd600 thru 0x8f09d46 (0xc747 bytes) 2024-09-16T20:49:32.774 INFO:teuthology.orchestra.run.smithi139.stdout:3794 read 0x1ba8a00 thru 0x1bb47e0 (0xbde1 bytes) 2024-09-16T20:49:32.775 INFO:teuthology.orchestra.run.smithi139.stdout:3798 write 0xc9a6200 thru 0xc9b5250 (0xf051 bytes) 2024-09-16T20:49:32.775 INFO:teuthology.orchestra.run.smithi139.stdout:3799 write 0xc28fe00 thru 0xc299ac1 (0x9cc2 bytes) 2024-09-16T20:49:32.775 INFO:teuthology.orchestra.run.smithi139.stdout:3803 write 0x7136800 thru 0x713e5a8 (0x7da9 bytes) 2024-09-16T20:49:32.775 INFO:teuthology.orchestra.run.smithi139.stdout:3804 trunc from 0xdadb000 to 0x64cf400 2024-09-16T20:49:32.837 INFO:teuthology.orchestra.run.smithi139.stdout:3805 write 0xa6e8800 thru 0xa6ef684 (0x6e85 bytes) 2024-09-16T20:49:32.842 INFO:teuthology.orchestra.run.smithi139.stdout:3808 trunc from 0xa6ef800 to 0x5f9c600 2024-09-16T20:49:32.872 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:32.871+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:32.883 INFO:teuthology.orchestra.run.smithi139.stdout:3810 write 0xbfa1a00 thru 0xbfab5b5 (0x9bb6 bytes) 2024-09-16T20:49:32.889 INFO:teuthology.orchestra.run.smithi139.stdout:3814 punch from 0x9c58800 to 0x9c63c00, (0xb400 bytes) 2024-09-16T20:49:32.891 INFO:teuthology.orchestra.run.smithi139.stdout:3815 write 0x7492a00 thru 0x74a09ce (0xdfcf bytes) 2024-09-16T20:49:32.891 INFO:teuthology.orchestra.run.smithi139.stdout:3818 write 0x3af200 thru 0x3b7336 (0x8137 bytes) 2024-09-16T20:49:32.893 INFO:teuthology.orchestra.run.smithi139.stdout:3819 read 0x20fb400 thru 0x2108617 (0xd218 bytes) 2024-09-16T20:49:32.893 INFO:teuthology.orchestra.run.smithi139.stdout:3820 punch from 0x5a75400 to 0x5a85200, (0xfe00 bytes) 2024-09-16T20:49:32.904 INFO:teuthology.orchestra.run.smithi139.stdout:3823 trunc from 0xbfab600 to 0x591e800 2024-09-16T20:49:32.937 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:32.937 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:49:32.937 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:49:32.937 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:32.946 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:49:32.948 INFO:teuthology.orchestra.run.smithi139.stdout:3824 trunc from 0x591e800 to 0xdc05000 2024-09-16T20:49:32.967 INFO:teuthology.orchestra.run.smithi139.stdout:3830 trunc from 0xdc05000 to 0x505b600 2024-09-16T20:49:32.989 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:32.988+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:32.990 INFO:teuthology.orchestra.run.smithi139.stdout:3834 read 0x131e400 thru 0x13288ff (0xa500 bytes) 2024-09-16T20:49:32.991 INFO:teuthology.orchestra.run.smithi139.stdout:3835 punch from 0x1649c00 to 0x1656c00, (0xd000 bytes) 2024-09-16T20:49:32.994 INFO:teuthology.orchestra.run.smithi139.stdout:3836 read 0x1765600 thru 0x176c476 (0x6e77 bytes) 2024-09-16T20:49:33.011 INFO:teuthology.orchestra.run.smithi139.stdout:3838 trunc from 0x505b600 to 0x5d99600 2024-09-16T20:49:33.033 INFO:teuthology.orchestra.run.smithi139.stdout:3839 read 0x36ea600 thru 0x36eb447 (0xe48 bytes) 2024-09-16T20:49:33.043 INFO:teuthology.orchestra.run.smithi139.stdout:3840 trunc from 0x5d99600 to 0xec42e00 2024-09-16T20:49:33.063 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:33.063 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:49:33.063 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:49:33.063 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:49:33.063 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:33.064 INFO:teuthology.orchestra.run.smithi139.stdout:3841 trunc from 0xec42e00 to 0xc1d9600 2024-09-16T20:49:33.073 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:49:33.074 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:33.073+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:33.074 INFO:teuthology.orchestra.run.smithi139.stdout:3842 punch from 0xa5bfa00 to 0xa5c8600, (0x8c00 bytes) 2024-09-16T20:49:33.078 INFO:teuthology.orchestra.run.smithi139.stdout:3846 write 0x8cfb200 thru 0x8d03aa2 (0x88a3 bytes) 2024-09-16T20:49:33.078 INFO:teuthology.orchestra.run.smithi139.stdout:3847 write 0x62e5800 thru 0x62e7ed8 (0x26d9 bytes) 2024-09-16T20:49:33.078 INFO:teuthology.orchestra.run.smithi139.stdout:3851 write 0x2f6f200 thru 0x2f73ba0 (0x49a1 bytes) 2024-09-16T20:49:33.079 INFO:teuthology.orchestra.run.smithi139.stdout:3853 write 0x93a4c00 thru 0x93a8b9e (0x3f9f bytes) 2024-09-16T20:49:33.079 INFO:teuthology.orchestra.run.smithi139.stdout:3857 write 0x1bed200 thru 0x1bfafbf (0xddc0 bytes) 2024-09-16T20:49:33.079 INFO:teuthology.orchestra.run.smithi139.stdout:3860 write 0x3818800 thru 0x3818c10 (0x411 bytes) 2024-09-16T20:49:33.079 INFO:teuthology.orchestra.run.smithi139.stdout:3862 read 0xa8e3a00 thru 0xa8e4643 (0xc44 bytes) 2024-09-16T20:49:33.079 INFO:teuthology.orchestra.run.smithi139.stdout:3863 punch from 0x225d000 to 0x225ec00, (0x1c00 bytes) 2024-09-16T20:49:33.081 INFO:teuthology.orchestra.run.smithi139.stdout:3866 write 0x202bc00 thru 0x202f035 (0x3436 bytes) 2024-09-16T20:49:33.081 INFO:teuthology.orchestra.run.smithi139.stdout:3869 write 0x4b76e00 thru 0x4b7e789 (0x798a bytes) 2024-09-16T20:49:33.081 INFO:teuthology.orchestra.run.smithi139.stdout:3872 write 0x8508600 thru 0x850d41a (0x4e1b bytes) 2024-09-16T20:49:33.081 INFO:teuthology.orchestra.run.smithi139.stdout:3873 write 0xaaf6600 thru 0xaaf68c3 (0x2c4 bytes) 2024-09-16T20:49:33.082 INFO:teuthology.orchestra.run.smithi139.stdout:3874 punch from 0xa0a6200 to 0xa0aec00, (0x8a00 bytes) 2024-09-16T20:49:33.084 INFO:teuthology.orchestra.run.smithi139.stdout:3876 punch from 0x6682a00 to 0x668f200, (0xc800 bytes) 2024-09-16T20:49:33.084 INFO:teuthology.orchestra.run.smithi139.stdout:3878 write 0xb2b200 thru 0xb2cc60 (0x1a61 bytes) 2024-09-16T20:49:33.094 INFO:teuthology.orchestra.run.smithi139.stdout:3879 write 0x131ac00 thru 0x131df6c (0x336d bytes) 2024-09-16T20:49:33.098 INFO:teuthology.orchestra.run.smithi139.stdout:3880 read 0x6fa4a00 thru 0x6faa4f2 (0x5af3 bytes) 2024-09-16T20:49:33.098 INFO:teuthology.orchestra.run.smithi139.stdout:3882 write 0x26b1800 thru 0x26bcbfc (0xb3fd bytes) 2024-09-16T20:49:33.098 INFO:teuthology.orchestra.run.smithi139.stdout:3886 read 0x48e9400 thru 0x48ec4fb (0x30fc bytes) 2024-09-16T20:49:33.098 INFO:teuthology.orchestra.run.smithi139.stdout:3887 write 0x3d32c00 thru 0x3d42b63 (0xff64 bytes) 2024-09-16T20:49:33.098 INFO:teuthology.orchestra.run.smithi139.stdout:3891 read 0xbb45000 thru 0xbb4f6c6 (0xa6c7 bytes) 2024-09-16T20:49:33.105 INFO:teuthology.orchestra.run.smithi139.stdout:3895 trunc from 0xc1d9600 to 0x5be9e00 2024-09-16T20:49:33.149 INFO:teuthology.orchestra.run.smithi139.stdout:3896 punch from 0x880200 to 0x881600, (0x1400 bytes) 2024-09-16T20:49:33.163 INFO:teuthology.orchestra.run.smithi139.stdout:3897 trunc from 0x5be9e00 to 0xe31be00 2024-09-16T20:49:33.175 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:33.174+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:33.176 INFO:teuthology.orchestra.run.smithi139.stdout:3899 write 0x1c15400 thru 0x1c1d6a6 (0x82a7 bytes) 2024-09-16T20:49:33.180 INFO:teuthology.orchestra.run.smithi139.stdout:3900 trunc from 0xe31be00 to 0xb15c600 2024-09-16T20:49:33.203 INFO:teuthology.orchestra.run.smithi139.stdout:3902 read 0x1edac00 thru 0x1edb753 (0xb54 bytes) 2024-09-16T20:49:33.204 INFO:teuthology.orchestra.run.smithi139.stdout:3903 write 0x6def400 thru 0x6df3c7c (0x487d bytes) 2024-09-16T20:49:33.204 INFO:teuthology.orchestra.run.smithi139.stdout:3907 read 0x52d0800 thru 0x52dcd29 (0xc52a bytes) 2024-09-16T20:49:33.204 INFO:teuthology.orchestra.run.smithi139.stdout:3909 punch from 0x29c1200 to 0x29c6200, (0x5000 bytes) 2024-09-16T20:49:33.205 INFO:teuthology.orchestra.run.smithi139.stdout:3910 read 0x556f400 thru 0x5575c10 (0x6811 bytes) 2024-09-16T20:49:33.224 INFO:teuthology.orchestra.run.smithi139.stdout:3911 write 0xe2f3e00 thru 0xe3034d3 (0xf6d4 bytes) 2024-09-16T20:49:33.224 INFO:teuthology.orchestra.run.smithi139.stdout:3912 punch from 0x2c6ec00 to 0x2c7dc00, (0xf000 bytes) 2024-09-16T20:49:33.226 INFO:teuthology.orchestra.run.smithi139.stdout:3916 read 0xa275600 thru 0xa281e22 (0xc823 bytes) 2024-09-16T20:49:33.226 INFO:teuthology.orchestra.run.smithi139.stdout:3917 read 0x6c12e00 thru 0x6c218ba (0xeabb bytes) 2024-09-16T20:49:33.227 INFO:teuthology.orchestra.run.smithi139.stdout:3919 write 0x1621e00 thru 0x162a112 (0x8313 bytes) 2024-09-16T20:49:33.228 INFO:teuthology.orchestra.run.smithi139.stdout:3921 write 0xd091800 thru 0xd093d39 (0x253a bytes) 2024-09-16T20:49:33.228 INFO:teuthology.orchestra.run.smithi139.stdout:3922 read 0x766ca00 thru 0x76772f8 (0xa8f9 bytes) 2024-09-16T20:49:33.228 INFO:teuthology.orchestra.run.smithi139.stdout:3923 punch from 0xb7c5e00 to 0xb7d2e00, (0xd000 bytes) 2024-09-16T20:49:33.229 INFO:teuthology.orchestra.run.smithi139.stdout:3924 trunc from 0xe303600 to 0xe863000 2024-09-16T20:49:33.236 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:33.236 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:49:33.236 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:49:33.236 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:49:33.236 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.236 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5286.0:10643 3.17 3.f6273fb7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-16T20:49:33.236 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:49:33.236 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.443742325999999, 2024-09-16T20:49:33.237 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.76672347200000002, 2024-09-16T20:49:33.237 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:33.237 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:33.237 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:49:33.237 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:33.237 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5286", 2024-09-16T20:49:33.237 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/4135433989", 2024-09-16T20:49:33.237 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 10643 2024-09-16T20:49:33.237 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.237 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:33.237 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.238 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:33.238 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:49:33.238 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:33.238 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.238 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.238 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:33.238 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:49:33.238 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:33.238 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.238 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.238 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:33.238 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790489+0000", 2024-09-16T20:49:33.239 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5910000000000001e-06 2024-09-16T20:49:33.239 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.239 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.239 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:33.239 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790493+0000", 2024-09-16T20:49:33.239 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2669999999999998e-06 2024-09-16T20:49:33.239 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.239 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.239 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:33.239 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790500+0000", 2024-09-16T20:49:33.239 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3780000000000003e-06 2024-09-16T20:49:33.239 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.240 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.240 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:33.240 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790505+0000", 2024-09-16T20:49:33.240 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1560000000000003e-06 2024-09-16T20:49:33.240 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.240 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.240 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:33.240 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790559+0000", 2024-09-16T20:49:33.240 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3494e-05 2024-09-16T20:49:33.240 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.240 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.240 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:49:33.240 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790566+0000", 2024-09-16T20:49:33.241 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7649999999999993e-06 2024-09-16T20:49:33.241 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.241 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.241 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:33.241 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:09.557198+0000", 2024-09-16T20:49:33.241 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.76663136200000004 2024-09-16T20:49:33.241 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.241 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.241 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:33.241 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:09.557211+0000", 2024-09-16T20:49:33.241 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3458999999999999e-05 2024-09-16T20:49:33.241 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:33.241 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:33.241 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:33.242 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.242 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.242 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3863 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1232896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:33.242 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896432+0000", 2024-09-16T20:49:33.242 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.337797642000002, 2024-09-16T20:49:33.242 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046223693000000003, 2024-09-16T20:49:33.242 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:33.242 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:33.242 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:33.242 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:33.242 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:33.242 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:33.242 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3863 2024-09-16T20:49:33.242 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.242 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:33.243 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.243 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:33.243 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896432+0000", 2024-09-16T20:49:33.243 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:33.243 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.243 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.243 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:33.243 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896432+0000", 2024-09-16T20:49:33.243 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:33.243 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.243 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.243 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:33.243 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896433+0000", 2024-09-16T20:49:33.243 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.89e-07 2024-09-16T20:49:33.244 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.244 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.244 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:33.244 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896435+0000", 2024-09-16T20:49:33.244 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.928e-06 2024-09-16T20:49:33.244 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.244 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.244 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:33.244 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896441+0000", 2024-09-16T20:49:33.244 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.896e-06 2024-09-16T20:49:33.244 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.245 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.245 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:33.245 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896445+0000", 2024-09-16T20:49:33.245 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7550000000000002e-06 2024-09-16T20:49:33.245 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.245 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.245 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:33.245 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.942468+0000", 2024-09-16T20:49:33.245 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046022278 2024-09-16T20:49:33.245 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.245 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.246 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:33.246 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.942553+0000", 2024-09-16T20:49:33.246 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5807e-05 2024-09-16T20:49:33.246 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.246 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.246 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:33.246 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.942656+0000", 2024-09-16T20:49:33.246 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010264000000000001 2024-09-16T20:49:33.246 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:33.246 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:33.246 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:33.246 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.247 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.247 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3864 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1236992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:33.247 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896451+0000", 2024-09-16T20:49:33.247 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.337778772, 2024-09-16T20:49:33.247 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046415605999999998, 2024-09-16T20:49:33.247 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:33.247 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:33.247 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:33.247 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:33.247 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:33.247 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:33.247 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3864 2024-09-16T20:49:33.247 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.247 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:33.248 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.248 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:33.248 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896451+0000", 2024-09-16T20:49:33.248 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:33.248 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.248 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.248 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:33.248 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896451+0000", 2024-09-16T20:49:33.248 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:33.248 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.248 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.248 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:33.248 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896452+0000", 2024-09-16T20:49:33.248 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1699999999999999e-07 2024-09-16T20:49:33.249 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.249 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.249 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:33.249 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896453+0000", 2024-09-16T20:49:33.249 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6950000000000001e-06 2024-09-16T20:49:33.249 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.249 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.249 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:33.249 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896459+0000", 2024-09-16T20:49:33.250 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6749999999999999e-06 2024-09-16T20:49:33.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.250 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.250 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:33.250 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896464+0000", 2024-09-16T20:49:33.250 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1399999999999999e-06 2024-09-16T20:49:33.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.250 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.250 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:33.250 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.942695+0000", 2024-09-16T20:49:33.250 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046230965999999998 2024-09-16T20:49:33.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.251 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.251 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:33.251 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.942771+0000", 2024-09-16T20:49:33.251 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5859999999999995e-05 2024-09-16T20:49:33.251 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.251 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.251 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:33.251 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.942867+0000", 2024-09-16T20:49:33.251 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5952999999999994e-05 2024-09-16T20:49:33.251 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:33.251 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:33.251 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:33.251 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.251 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.252 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3866 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1241088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:33.252 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896471+0000", 2024-09-16T20:49:33.252 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.337759542000001, 2024-09-16T20:49:33.252 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046590830999999999, 2024-09-16T20:49:33.252 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:33.252 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:33.252 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:33.252 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:33.252 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:33.252 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:33.252 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3866 2024-09-16T20:49:33.252 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.252 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:33.252 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.253 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:33.253 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896471+0000", 2024-09-16T20:49:33.253 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:33.253 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.253 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.253 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:33.253 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896471+0000", 2024-09-16T20:49:33.253 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:33.253 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.253 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.253 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:33.253 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896471+0000", 2024-09-16T20:49:33.253 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0900000000000002e-07 2024-09-16T20:49:33.253 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.254 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.254 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:33.254 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896473+0000", 2024-09-16T20:49:33.254 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8899999999999999e-06 2024-09-16T20:49:33.254 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.254 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.254 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:33.254 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896479+0000", 2024-09-16T20:49:33.254 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0179999999999998e-06 2024-09-16T20:49:33.254 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.254 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.254 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:33.254 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896486+0000", 2024-09-16T20:49:33.254 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7240000000000003e-06 2024-09-16T20:49:33.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.255 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.255 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:33.255 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.942924+0000", 2024-09-16T20:49:33.255 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046438266999999998 2024-09-16T20:49:33.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.255 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.255 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:33.255 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943001+0000", 2024-09-16T20:49:33.255 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6987e-05 2024-09-16T20:49:33.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.255 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.255 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:33.255 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943061+0000", 2024-09-16T20:49:33.256 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0535999999999998e-05 2024-09-16T20:49:33.256 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:33.256 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:33.256 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:33.256 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.256 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.256 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3868 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1245184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:33.256 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896492+0000", 2024-09-16T20:49:33.256 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.337737991000001, 2024-09-16T20:49:33.256 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046738590000000003, 2024-09-16T20:49:33.256 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:33.256 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:33.256 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:33.257 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:33.257 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:33.257 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:33.257 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3868 2024-09-16T20:49:33.257 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.257 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:33.257 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.257 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:33.257 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896492+0000", 2024-09-16T20:49:33.257 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:33.257 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.257 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.257 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:33.257 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896492+0000", 2024-09-16T20:49:33.258 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:33.258 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.258 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.258 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:33.258 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896492+0000", 2024-09-16T20:49:33.258 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9499999999999998e-07 2024-09-16T20:49:33.258 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.258 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.258 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:33.258 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896494+0000", 2024-09-16T20:49:33.258 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.973e-06 2024-09-16T20:49:33.258 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.258 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.258 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:33.259 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896501+0000", 2024-09-16T20:49:33.259 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7719999999999997e-06 2024-09-16T20:49:33.259 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.259 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.259 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:33.259 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896506+0000", 2024-09-16T20:49:33.259 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2089999999999996e-06 2024-09-16T20:49:33.259 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.259 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.259 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:33.259 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943093+0000", 2024-09-16T20:49:33.260 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046586799999999998 2024-09-16T20:49:33.260 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.260 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.260 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:33.260 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943148+0000", 2024-09-16T20:49:33.260 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4385999999999997e-05 2024-09-16T20:49:33.260 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.260 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.260 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:33.260 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943231+0000", 2024-09-16T20:49:33.260 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3054999999999997e-05 2024-09-16T20:49:33.260 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:33.261 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:33.261 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:33.261 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.261 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.261 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3870 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1249280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:33.261 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896514+0000", 2024-09-16T20:49:33.261 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.337716363999998, 2024-09-16T20:49:33.261 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046893815999999998, 2024-09-16T20:49:33.261 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:33.261 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:33.261 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:33.261 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:33.261 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:33.262 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:33.262 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3870 2024-09-16T20:49:33.262 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.262 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:33.262 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.262 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:33.262 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896514+0000", 2024-09-16T20:49:33.262 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:33.262 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.263 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.263 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:33.263 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896514+0000", 2024-09-16T20:49:33.263 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:33.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.263 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.263 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:33.263 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896514+0000", 2024-09-16T20:49:33.263 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9299999999999999e-07 2024-09-16T20:49:33.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.263 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.263 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:33.263 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896521+0000", 2024-09-16T20:49:33.264 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8199999999999999e-06 2024-09-16T20:49:33.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.264 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.264 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:33.264 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896529+0000", 2024-09-16T20:49:33.264 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0120000000000005e-06 2024-09-16T20:49:33.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.264 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.264 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:33.264 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896534+0000", 2024-09-16T20:49:33.264 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5779999999999998e-06 2024-09-16T20:49:33.265 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.265 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.265 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:33.265 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943286+0000", 2024-09-16T20:49:33.265 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046751514000000001 2024-09-16T20:49:33.265 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.265 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.265 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:33.265 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943343+0000", 2024-09-16T20:49:33.265 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7216000000000002e-05 2024-09-16T20:49:33.265 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.265 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.265 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:33.265 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943407+0000", 2024-09-16T20:49:33.266 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4282999999999994e-05 2024-09-16T20:49:33.266 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:33.266 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:33.266 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:33.266 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.266 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.266 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3872 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1253376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:33.266 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896561+0000", 2024-09-16T20:49:33.266 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.337668969999999, 2024-09-16T20:49:33.266 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.04705165, 2024-09-16T20:49:33.266 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:33.266 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:33.266 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:33.266 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:33.266 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:33.267 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:33.267 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3872 2024-09-16T20:49:33.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.267 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:33.267 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.267 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:33.267 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896561+0000", 2024-09-16T20:49:33.267 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:33.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.267 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.267 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:33.268 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896561+0000", 2024-09-16T20:49:33.268 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:33.268 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.268 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.268 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:33.268 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896561+0000", 2024-09-16T20:49:33.268 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4999999999999998e-07 2024-09-16T20:49:33.268 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.268 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.268 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:33.268 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896563+0000", 2024-09-16T20:49:33.268 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0140000000000001e-06 2024-09-16T20:49:33.268 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.268 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.269 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:33.269 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896569+0000", 2024-09-16T20:49:33.269 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6450000000000002e-06 2024-09-16T20:49:33.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.269 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.269 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:33.269 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896572+0000", 2024-09-16T20:49:33.269 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.174e-06 2024-09-16T20:49:33.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.269 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.269 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:33.269 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943456+0000", 2024-09-16T20:49:33.269 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046883552000000002 2024-09-16T20:49:33.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.270 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.270 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:33.270 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943531+0000", 2024-09-16T20:49:33.270 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5080000000000006e-05 2024-09-16T20:49:33.270 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.270 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.270 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:33.270 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943613+0000", 2024-09-16T20:49:33.270 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1835000000000005e-05 2024-09-16T20:49:33.271 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:33.271 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:33.271 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:33.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.271 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.271 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3874 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1257472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:33.271 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896577+0000", 2024-09-16T20:49:33.271 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.337653455000002, 2024-09-16T20:49:33.271 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047187944000000002, 2024-09-16T20:49:33.271 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:33.271 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:33.271 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:33.271 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:33.271 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:33.272 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:33.272 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3874 2024-09-16T20:49:33.272 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.272 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:33.272 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.272 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:33.272 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896577+0000", 2024-09-16T20:49:33.272 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:33.272 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.273 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.273 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:33.273 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896577+0000", 2024-09-16T20:49:33.273 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:33.273 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.273 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.273 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:33.273 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896577+0000", 2024-09-16T20:49:33.273 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1199999999999999e-07 2024-09-16T20:49:33.273 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.273 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.273 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:33.274 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896578+0000", 2024-09-16T20:49:33.274 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.237e-06 2024-09-16T20:49:33.274 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.274 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.274 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:33.274 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896583+0000", 2024-09-16T20:49:33.274 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5859999999999998e-06 2024-09-16T20:49:33.274 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.274 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.274 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:33.274 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896587+0000", 2024-09-16T20:49:33.274 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3100000000000002e-06 2024-09-16T20:49:33.274 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.274 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.275 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:33.275 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943661+0000", 2024-09-16T20:49:33.275 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047073639 2024-09-16T20:49:33.275 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.275 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.275 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:33.275 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943712+0000", 2024-09-16T20:49:33.275 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1190999999999998e-05 2024-09-16T20:49:33.275 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.275 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.275 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:33.275 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943765+0000", 2024-09-16T20:49:33.275 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2769000000000002e-05 2024-09-16T20:49:33.276 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:33.276 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:33.276 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:33.276 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.276 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.276 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3876 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1261568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:33.276 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896593+0000", 2024-09-16T20:49:33.276 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.337636877000001, 2024-09-16T20:49:33.276 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047362866000000003, 2024-09-16T20:49:33.276 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:33.276 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:33.276 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:33.277 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:33.277 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:33.277 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:33.277 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3876 2024-09-16T20:49:33.277 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.277 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:33.277 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.277 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:33.277 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896593+0000", 2024-09-16T20:49:33.277 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:33.277 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.278 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.278 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:33.278 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896593+0000", 2024-09-16T20:49:33.278 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:33.278 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.278 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.278 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:33.278 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896594+0000", 2024-09-16T20:49:33.278 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1600000000000002e-07 2024-09-16T20:49:33.278 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.278 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.278 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:33.278 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896595+0000", 2024-09-16T20:49:33.278 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.756e-06 2024-09-16T20:49:33.279 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.279 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.279 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:33.279 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896601+0000", 2024-09-16T20:49:33.279 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9309999999999996e-06 2024-09-16T20:49:33.279 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.279 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.279 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:33.279 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896606+0000", 2024-09-16T20:49:33.279 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1089999999999997e-06 2024-09-16T20:49:33.279 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.279 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.279 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:33.279 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943790+0000", 2024-09-16T20:49:33.279 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047183414 2024-09-16T20:49:33.280 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.280 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.280 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:33.280 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943869+0000", 2024-09-16T20:49:33.280 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9122000000000006e-05 2024-09-16T20:49:33.280 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.280 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.280 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:33.280 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943956+0000", 2024-09-16T20:49:33.280 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7118000000000002e-05 2024-09-16T20:49:33.281 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:33.281 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:33.281 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:33.281 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.281 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.281 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3879 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1265664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:33.281 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896613+0000", 2024-09-16T20:49:33.281 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.337617235, 2024-09-16T20:49:33.281 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047528204999999997, 2024-09-16T20:49:33.281 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:33.281 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:33.281 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:33.281 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:33.282 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:33.282 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:33.282 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3879 2024-09-16T20:49:33.282 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.282 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:33.282 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.282 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:33.282 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896613+0000", 2024-09-16T20:49:33.282 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:33.282 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.282 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.282 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:33.282 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896613+0000", 2024-09-16T20:49:33.283 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:33.283 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.283 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.283 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:33.283 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896613+0000", 2024-09-16T20:49:33.283 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.84e-07 2024-09-16T20:49:33.283 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.283 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.283 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:33.283 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896615+0000", 2024-09-16T20:49:33.283 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9970000000000001e-06 2024-09-16T20:49:33.283 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.283 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.283 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:33.284 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896622+0000", 2024-09-16T20:49:33.284 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3500000000000002e-06 2024-09-16T20:49:33.284 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.284 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.284 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:33.284 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896627+0000", 2024-09-16T20:49:33.284 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2490000000000001e-06 2024-09-16T20:49:33.284 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.284 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.284 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:33.284 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943996+0000", 2024-09-16T20:49:33.284 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047369175999999999 2024-09-16T20:49:33.284 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.285 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.285 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:33.285 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944082+0000", 2024-09-16T20:49:33.285 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5859000000000006e-05 2024-09-16T20:49:33.285 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.285 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.285 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:33.285 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944141+0000", 2024-09-16T20:49:33.285 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9190000000000001e-05 2024-09-16T20:49:33.285 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:33.285 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:33.286 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:33.286 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.286 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.286 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3881 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1269760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:33.286 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896634+0000", 2024-09-16T20:49:33.286 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.337596322, 2024-09-16T20:49:33.286 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047692834000000003, 2024-09-16T20:49:33.286 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:33.286 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:33.286 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:33.286 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:33.286 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:33.286 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:33.286 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3881 2024-09-16T20:49:33.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.287 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:33.287 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.287 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:33.287 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896634+0000", 2024-09-16T20:49:33.287 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:33.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.287 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.287 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:33.287 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896634+0000", 2024-09-16T20:49:33.287 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:33.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.287 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.288 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:33.288 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896634+0000", 2024-09-16T20:49:33.288 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-16T20:49:33.288 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.288 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.288 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:33.288 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896636+0000", 2024-09-16T20:49:33.288 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.216e-06 2024-09-16T20:49:33.288 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.288 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.288 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:33.288 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896640+0000", 2024-09-16T20:49:33.289 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1749999999999998e-06 2024-09-16T20:49:33.289 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.289 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.289 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:33.289 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896643+0000", 2024-09-16T20:49:33.289 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8779999999999998e-06 2024-09-16T20:49:33.289 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.289 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.289 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:33.289 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944178+0000", 2024-09-16T20:49:33.289 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047534633 2024-09-16T20:49:33.289 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.289 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.290 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:33.290 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944231+0000", 2024-09-16T20:49:33.290 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3338999999999999e-05 2024-09-16T20:49:33.290 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.290 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.290 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:33.290 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944327+0000", 2024-09-16T20:49:33.290 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5223000000000006e-05 2024-09-16T20:49:33.290 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:33.290 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:33.290 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:33.290 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.291 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.291 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3882 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1273856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:33.291 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896647+0000", 2024-09-16T20:49:33.291 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.337583161000001, 2024-09-16T20:49:33.291 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047858886000000003, 2024-09-16T20:49:33.291 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:33.291 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:33.291 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:33.291 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:33.292 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:33.292 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:33.292 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3882 2024-09-16T20:49:33.292 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.292 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:33.292 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.292 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:33.292 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896647+0000", 2024-09-16T20:49:33.292 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:33.292 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.292 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.292 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:33.292 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896647+0000", 2024-09-16T20:49:33.293 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:33.293 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.293 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.293 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:33.293 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896647+0000", 2024-09-16T20:49:33.293 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4400000000000001e-07 2024-09-16T20:49:33.293 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.293 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.293 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:33.293 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896648+0000", 2024-09-16T20:49:33.293 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0499999999999999e-06 2024-09-16T20:49:33.293 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.293 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.294 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:33.294 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896651+0000", 2024-09-16T20:49:33.294 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9809999999999999e-06 2024-09-16T20:49:33.294 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.294 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.294 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:33.294 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896654+0000", 2024-09-16T20:49:33.294 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4439999999999998e-06 2024-09-16T20:49:33.294 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.294 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.294 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:33.294 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944361+0000", 2024-09-16T20:49:33.294 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047707408 2024-09-16T20:49:33.295 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.295 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.295 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:33.295 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944405+0000", 2024-09-16T20:49:33.295 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4107000000000003e-05 2024-09-16T20:49:33.295 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.295 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.295 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:33.295 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944506+0000", 2024-09-16T20:49:33.295 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000100652 2024-09-16T20:49:33.295 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:33.295 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:33.295 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:33.295 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.295 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.296 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3883 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1277952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:33.296 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896656+0000", 2024-09-16T20:49:33.296 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.337573651, 2024-09-16T20:49:33.296 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047975265000000003, 2024-09-16T20:49:33.296 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:33.296 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:33.296 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:33.296 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:33.296 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:33.296 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:33.296 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3883 2024-09-16T20:49:33.296 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.296 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:33.297 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.297 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:33.297 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896656+0000", 2024-09-16T20:49:33.297 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:33.297 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.297 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.297 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:33.297 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896656+0000", 2024-09-16T20:49:33.297 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:33.297 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.297 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.297 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:33.297 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896657+0000", 2024-09-16T20:49:33.297 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7000000000000001e-07 2024-09-16T20:49:33.298 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.298 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.298 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:33.298 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896658+0000", 2024-09-16T20:49:33.298 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5850000000000001e-06 2024-09-16T20:49:33.298 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.298 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.298 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:33.298 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896664+0000", 2024-09-16T20:49:33.298 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.609e-06 2024-09-16T20:49:33.298 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.298 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.298 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:33.298 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896668+0000", 2024-09-16T20:49:33.299 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.121e-06 2024-09-16T20:49:33.299 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.299 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.299 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:33.299 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944542+0000", 2024-09-16T20:49:33.299 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047873838000000002 2024-09-16T20:49:33.299 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.299 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.299 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:33.299 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944579+0000", 2024-09-16T20:49:33.299 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6943000000000002e-05 2024-09-16T20:49:33.299 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.299 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.299 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:33.300 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944632+0000", 2024-09-16T20:49:33.300 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2899000000000003e-05 2024-09-16T20:49:33.300 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:33.300 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:33.300 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:33.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.300 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.300 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3886 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1282048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:33.300 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896674+0000", 2024-09-16T20:49:33.300 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.337555682000001, 2024-09-16T20:49:33.300 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048152421000000001, 2024-09-16T20:49:33.300 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:33.300 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:33.300 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:33.300 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:33.301 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:33.301 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:33.301 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3886 2024-09-16T20:49:33.301 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.301 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:33.301 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.301 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:33.301 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896674+0000", 2024-09-16T20:49:33.301 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:33.301 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.302 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.302 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:33.302 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896674+0000", 2024-09-16T20:49:33.302 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:33.302 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.302 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.302 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:33.302 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896675+0000", 2024-09-16T20:49:33.302 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6100000000000002e-07 2024-09-16T20:49:33.302 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.302 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.302 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:33.303 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896677+0000", 2024-09-16T20:49:33.303 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1270000000000001e-06 2024-09-16T20:49:33.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.303 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.303 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:33.303 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896683+0000", 2024-09-16T20:49:33.303 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1970000000000001e-06 2024-09-16T20:49:33.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.303 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.303 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:33.303 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896688+0000", 2024-09-16T20:49:33.303 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.605e-06 2024-09-16T20:49:33.304 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.304 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.304 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:33.304 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944676+0000", 2024-09-16T20:49:33.304 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047988391999999998 2024-09-16T20:49:33.304 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.304 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.304 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:33.304 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944763+0000", 2024-09-16T20:49:33.304 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7363999999999999e-05 2024-09-16T20:49:33.304 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.304 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.304 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:33.305 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944827+0000", 2024-09-16T20:49:33.305 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3374999999999995e-05 2024-09-16T20:49:33.305 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:33.305 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:33.305 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:33.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.305 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3888 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1286144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:33.305 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896693+0000", 2024-09-16T20:49:33.305 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.337536558, 2024-09-16T20:49:33.305 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048336443999999999, 2024-09-16T20:49:33.305 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:33.305 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:33.306 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:33.306 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:33.306 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:33.306 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:33.306 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3888 2024-09-16T20:49:33.306 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.306 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:33.306 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.306 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:33.306 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896693+0000", 2024-09-16T20:49:33.306 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:33.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.307 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.307 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:33.307 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896693+0000", 2024-09-16T20:49:33.307 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:33.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.307 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.307 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:33.307 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896694+0000", 2024-09-16T20:49:33.307 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3200000000000001e-07 2024-09-16T20:49:33.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.307 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.307 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:33.307 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896696+0000", 2024-09-16T20:49:33.308 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8989999999999999e-06 2024-09-16T20:49:33.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.308 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.308 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:33.308 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896702+0000", 2024-09-16T20:49:33.308 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8819999999999998e-06 2024-09-16T20:49:33.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.308 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.308 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:33.308 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896707+0000", 2024-09-16T20:49:33.308 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1529999999999996e-06 2024-09-16T20:49:33.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.308 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.309 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:33.309 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944863+0000", 2024-09-16T20:49:33.309 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048156658999999997 2024-09-16T20:49:33.309 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.309 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.309 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:33.309 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944944+0000", 2024-09-16T20:49:33.309 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0090000000000001e-05 2024-09-16T20:49:33.309 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.309 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.309 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:33.309 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945030+0000", 2024-09-16T20:49:33.309 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6428999999999995e-05 2024-09-16T20:49:33.309 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:33.310 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:33.310 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:33.310 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.310 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.310 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3890 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1290240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:33.310 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896713+0000", 2024-09-16T20:49:33.310 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.337516688000001, 2024-09-16T20:49:33.310 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048538382999999997, 2024-09-16T20:49:33.310 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:33.310 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:33.310 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:33.310 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:33.310 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:33.310 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:33.311 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3890 2024-09-16T20:49:33.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.311 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:33.311 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.311 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:33.311 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896713+0000", 2024-09-16T20:49:33.311 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:33.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.311 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.311 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:33.311 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896713+0000", 2024-09-16T20:49:33.311 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:33.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.312 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.312 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:33.312 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896714+0000", 2024-09-16T20:49:33.312 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2000000000000001e-07 2024-09-16T20:49:33.312 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.312 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.312 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:33.312 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896716+0000", 2024-09-16T20:49:33.312 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0109999999999999e-06 2024-09-16T20:49:33.312 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.313 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.313 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:33.313 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896720+0000", 2024-09-16T20:49:33.313 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9049999999999999e-06 2024-09-16T20:49:33.313 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.313 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.313 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:33.313 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896722+0000", 2024-09-16T20:49:33.313 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.627e-06 2024-09-16T20:49:33.313 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.313 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.313 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:33.313 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945075+0000", 2024-09-16T20:49:33.313 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048353149999999998 2024-09-16T20:49:33.314 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.314 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.314 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:33.314 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945157+0000", 2024-09-16T20:49:33.314 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1972999999999996e-05 2024-09-16T20:49:33.314 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.314 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.314 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:33.314 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945252+0000", 2024-09-16T20:49:33.314 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4396999999999998e-05 2024-09-16T20:49:33.314 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:33.314 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:33.314 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:33.315 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.315 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.315 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3893 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1294336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:33.315 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896726+0000", 2024-09-16T20:49:33.315 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.337503830999999, 2024-09-16T20:49:33.315 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048658686999999999, 2024-09-16T20:49:33.315 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:33.315 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:33.315 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:33.315 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:33.315 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:33.315 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:33.315 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3893 2024-09-16T20:49:33.315 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.316 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:33.316 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.316 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:33.316 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896726+0000", 2024-09-16T20:49:33.316 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:33.316 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.316 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.316 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:33.316 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896726+0000", 2024-09-16T20:49:33.316 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:33.316 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.316 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.316 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:33.317 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896727+0000", 2024-09-16T20:49:33.317 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7899999999999999e-07 2024-09-16T20:49:33.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.317 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.317 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:33.317 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896728+0000", 2024-09-16T20:49:33.317 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.68e-06 2024-09-16T20:49:33.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.317 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.317 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:33.317 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896734+0000", 2024-09-16T20:49:33.318 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9190000000000004e-06 2024-09-16T20:49:33.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.318 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.318 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:33.318 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896739+0000", 2024-09-16T20:49:33.318 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5549999999999996e-06 2024-09-16T20:49:33.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.318 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.318 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:33.318 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945271+0000", 2024-09-16T20:49:33.318 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048531795000000003 2024-09-16T20:49:33.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.318 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.319 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:33.319 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945330+0000", 2024-09-16T20:49:33.319 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9386999999999999e-05 2024-09-16T20:49:33.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.319 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.319 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:33.319 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945385+0000", 2024-09-16T20:49:33.319 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4871999999999999e-05 2024-09-16T20:49:33.319 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:33.319 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:33.319 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:33.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.319 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.319 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3895 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1298432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:33.319 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896754+0000", 2024-09-16T20:49:33.320 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.337475757, 2024-09-16T20:49:33.320 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048754440000000003, 2024-09-16T20:49:33.320 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:33.320 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:33.320 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:33.320 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:33.320 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:33.320 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:33.321 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3895 2024-09-16T20:49:33.321 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.321 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:33.321 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.321 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:33.321 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896754+0000", 2024-09-16T20:49:33.321 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:33.321 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.321 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.321 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:33.321 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896754+0000", 2024-09-16T20:49:33.321 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:33.321 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.321 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.322 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:33.322 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896755+0000", 2024-09-16T20:49:33.322 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5200000000000002e-07 2024-09-16T20:49:33.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.322 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.322 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:33.322 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896762+0000", 2024-09-16T20:49:33.322 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8430000000000004e-06 2024-09-16T20:49:33.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.322 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.323 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:33.323 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896782+0000", 2024-09-16T20:49:33.323 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0693e-05 2024-09-16T20:49:33.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.323 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.323 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:33.323 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896793+0000", 2024-09-16T20:49:33.323 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0294e-05 2024-09-16T20:49:33.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.323 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.324 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:33.324 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945418+0000", 2024-09-16T20:49:33.324 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048625738000000002 2024-09-16T20:49:33.324 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.324 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.324 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:33.324 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945457+0000", 2024-09-16T20:49:33.324 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9103999999999997e-05 2024-09-16T20:49:33.324 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.324 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.324 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:33.324 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945509+0000", 2024-09-16T20:49:33.325 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1316000000000001e-05 2024-09-16T20:49:33.325 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:33.325 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:33.325 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:33.325 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.325 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.325 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3896 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1302528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:33.325 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896806+0000", 2024-09-16T20:49:33.325 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.337423545, 2024-09-16T20:49:33.325 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048831159999999998, 2024-09-16T20:49:33.325 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:33.325 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:33.325 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:33.325 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:33.325 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:33.326 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:33.326 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3896 2024-09-16T20:49:33.326 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.326 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:33.326 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.326 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:33.326 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896806+0000", 2024-09-16T20:49:33.326 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:33.326 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.326 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.326 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:33.326 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896806+0000", 2024-09-16T20:49:33.327 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:33.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.327 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.327 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:33.327 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896807+0000", 2024-09-16T20:49:33.327 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3e-07 2024-09-16T20:49:33.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.327 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.327 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:33.327 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896811+0000", 2024-09-16T20:49:33.327 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8310000000000003e-06 2024-09-16T20:49:33.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.327 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.328 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:33.328 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896819+0000", 2024-09-16T20:49:33.328 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2470000000000008e-06 2024-09-16T20:49:33.328 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.328 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.328 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:33.328 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896823+0000", 2024-09-16T20:49:33.328 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7919999999999999e-06 2024-09-16T20:49:33.328 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.328 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.328 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:33.328 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945535+0000", 2024-09-16T20:49:33.329 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048712137000000003 2024-09-16T20:49:33.329 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.329 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.329 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:33.329 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945571+0000", 2024-09-16T20:49:33.329 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6261999999999999e-05 2024-09-16T20:49:33.329 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.329 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.329 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:33.329 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945638+0000", 2024-09-16T20:49:33.329 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6161000000000006e-05 2024-09-16T20:49:33.329 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:33.329 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:33.330 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:33.330 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.330 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.330 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3899 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1306624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:33.330 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896829+0000", 2024-09-16T20:49:33.330 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.337401344, 2024-09-16T20:49:33.330 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049014110999999999, 2024-09-16T20:49:33.330 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:33.330 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:33.330 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:33.330 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:33.330 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:33.330 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:33.330 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3899 2024-09-16T20:49:33.331 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.331 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:33.331 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.331 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:33.331 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896829+0000", 2024-09-16T20:49:33.331 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:33.331 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.331 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.331 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:33.331 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896829+0000", 2024-09-16T20:49:33.331 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:33.331 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.331 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.331 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:33.332 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896829+0000", 2024-09-16T20:49:33.332 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8999999999999998e-07 2024-09-16T20:49:33.332 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.332 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.332 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:33.332 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896831+0000", 2024-09-16T20:49:33.332 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.77e-06 2024-09-16T20:49:33.332 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.332 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.332 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:33.332 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896834+0000", 2024-09-16T20:49:33.332 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4800000000000001e-06 2024-09-16T20:49:33.332 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.332 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.333 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:33.333 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896837+0000", 2024-09-16T20:49:33.333 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8559999999999998e-06 2024-09-16T20:49:33.333 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.333 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.333 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:33.333 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945663+0000", 2024-09-16T20:49:33.333 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048825529999999999 2024-09-16T20:49:33.333 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.333 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.334 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:33.334 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945746+0000", 2024-09-16T20:49:33.334 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3621000000000005e-05 2024-09-16T20:49:33.334 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:33.334 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:33.334 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:33.334 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945843+0000", 2024-09-16T20:49:33.334 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6564000000000006e-05 2024-09-16T20:49:33.334 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:33.334 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:33.334 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:33.334 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:33.334 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:33.334 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:33.337 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:33.334+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:33.337 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:49:33.338 INFO:teuthology.orchestra.run.smithi139.stdout:3928 write 0x73a400 thru 0x73c232 (0x1e33 bytes) 2024-09-16T20:49:33.338 INFO:teuthology.orchestra.run.smithi139.stdout:3930 trunc from 0xe863000 to 0x8781400 2024-09-16T20:49:33.338 INFO:teuthology.orchestra.run.smithi139.stdout:3931 write 0x593200 thru 0x5934e2 (0x2e3 bytes) 2024-09-16T20:49:33.338 INFO:teuthology.orchestra.run.smithi139.stdout:3932 write 0x4381200 thru 0x4387a3f (0x6840 bytes) 2024-09-16T20:49:33.338 INFO:teuthology.orchestra.run.smithi139.stdout:3934 write 0xcfae400 thru 0xcfb0e24 (0x2a25 bytes) 2024-09-16T20:49:33.338 INFO:teuthology.orchestra.run.smithi139.stdout:3935 write 0x8b9a400 thru 0x8ba839d (0xdf9e bytes) 2024-09-16T20:49:33.338 INFO:teuthology.orchestra.run.smithi139.stdout:3937 read 0x74e9200 thru 0x74e9ac2 (0x8c3 bytes) 2024-09-16T20:49:33.338 INFO:teuthology.orchestra.run.smithi139.stdout:3938 trunc from 0xcfb1000 to 0x6b74c00 2024-09-16T20:49:33.353 INFO:teuthology.orchestra.run.smithi139.stdout:3939 punch from 0x64db200 to 0x64e0800, (0x5600 bytes) 2024-09-16T20:49:33.355 INFO:teuthology.orchestra.run.smithi139.stdout:3940 read 0x34e9000 thru 0x34f1b2d (0x8b2e bytes) 2024-09-16T20:49:33.355 INFO:teuthology.orchestra.run.smithi139.stdout:3943 read 0x51bba00 thru 0x51c4ff8 (0x95f9 bytes) 2024-09-16T20:49:33.355 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T20:49:33.356 INFO:teuthology.orchestra.run.smithi139.stdout:3945 punch from 0x6158600 to 0x6163e00, (0xb800 bytes) 2024-09-16T20:49:33.357 INFO:teuthology.orchestra.run.smithi139.stdout:3946 trunc from 0x6b74c00 to 0x80fee00 2024-09-16T20:49:33.373 INFO:teuthology.orchestra.run.smithi139.stdout:3947 write 0x7672e00 thru 0x767f4f4 (0xc6f5 bytes) 2024-09-16T20:49:33.374 INFO:teuthology.orchestra.run.smithi139.stdout:3948 read 0x32fc600 thru 0x330a7ac (0xe1ad bytes) 2024-09-16T20:49:33.379 INFO:teuthology.orchestra.run.smithi139.stdout:3949 trunc from 0x80fee00 to 0x9818400 2024-09-16T20:49:33.395 INFO:teuthology.orchestra.run.smithi139.stdout:3951 punch from 0x553000 to 0x554800, (0x1800 bytes) 2024-09-16T20:49:33.399 INFO:teuthology.orchestra.run.smithi139.stdout:3952 write 0x4f20400 thru 0x4f23787 (0x3388 bytes) 2024-09-16T20:49:33.425 INFO:teuthology.orchestra.run.smithi139.stdout:3955 write 0xcd27a00 thru 0xcd32125 (0xa726 bytes) 2024-09-16T20:49:33.426 INFO:teuthology.orchestra.run.smithi139.stdout:3956 punch from 0x6d9200 to 0x6e4800, (0xb600 bytes) 2024-09-16T20:49:33.429 INFO:teuthology.orchestra.run.smithi139.stdout:3957 clone 29 order 19 su 4096 sc 3 2024-09-16T20:49:33.437 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:33.436+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:33.474 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:33.474 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:49:33.475 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:49:33.475 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:33.483 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:49:33.525 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:49:33.526 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:49:33.542 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:49:33.543 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:49:33.543 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:33.541+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:33.561 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:49:33.561 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:49:33.578 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:49:33.578 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:49:33.595 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:49:33.595 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:49:33.612 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:49:33.612 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:49:33.634 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:33.634 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:33.640 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:33.638+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:33.643 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:49:33.741 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:33.740+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:33.791 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:33.846 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:33.847 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:49:33.848 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:33.846+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:33.857 INFO:teuthology.orchestra.run.smithi029.stderr:set 3.2 pg_upmap mapping to [5,0] 2024-09-16T20:49:33.874 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T20:49:33.958 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:33.957+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:33.965 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:33.966 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:33.975 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:49:34.049 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:34.047+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:34.124 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:34.124 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:34.133 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:49:34.149 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:34.148+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:34.240 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:34.240 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:34.249 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:49:34.250 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:34.249+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:34.351 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:34.350+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:34.357 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:34.357 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:34.365 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:49:34.452 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:34.450+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:34.512 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:34.512 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:34.521 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:49:34.555 INFO:teuthology.orchestra.run.smithi029.stderr:nodeep-scrub is set 2024-09-16T20:49:34.556 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:34.554+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:34.631 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:34.631 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:34.640 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:49:34.654 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:34.653+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:34.754 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:34.754+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:34.759 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:34.760 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:34.769 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:49:34.856 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:34.854+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:34.876 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:34.876 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:34.885 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:49:34.969 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:34.968+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:35.033 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:35.034 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:35.042 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:49:35.084 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:35.082+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:35.157 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:35.156+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:35.189 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:35.189 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:35.199 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:49:35.258 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:35.257+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:35.306 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:35.306 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:35.315 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:49:35.360 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:35.358+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:35.459 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:35.458+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:35.463 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:35.463 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:35.472 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:49:35.560 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:35.559+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:35.621 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:35.621 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:35.624 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T20:49:35.630 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:49:35.673 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:35.672+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:35.736 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:35.737 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:35.746 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:49:35.761 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:49:35.762 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:49:35.779 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:49:35.780 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:49:35.787 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:35.785+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:35.796 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:49:35.796 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:49:35.813 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:49:35.813 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:49:35.833 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:49:35.833 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:49:35.857 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:49:35.858 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:49:35.863 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:35.861+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:35.867 INFO:teuthology.orchestra.run.smithi139.stdout:truncating image image_client.0-clone28 from 0xcd32200 (overlap 0x505b600) to 0x383fe00 2024-09-16T20:49:35.914 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:35.914 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:49:35.914 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:49:35.914 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:49:35.915 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.915 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5286.0:10643 3.17 3.f6273fb7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-16T20:49:35.915 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:49:35.915 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.121535694999999, 2024-09-16T20:49:35.915 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.76672347200000002, 2024-09-16T20:49:35.915 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:35.915 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:35.915 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:49:35.915 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:35.915 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5286", 2024-09-16T20:49:35.915 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/4135433989", 2024-09-16T20:49:35.915 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 10643 2024-09-16T20:49:35.915 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.915 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:35.916 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.916 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:35.916 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:49:35.916 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:35.916 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.916 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.916 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:35.916 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:49:35.916 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:35.916 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.916 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.916 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:35.916 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790489+0000", 2024-09-16T20:49:35.916 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5910000000000001e-06 2024-09-16T20:49:35.917 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.917 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.917 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:35.917 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790493+0000", 2024-09-16T20:49:35.917 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2669999999999998e-06 2024-09-16T20:49:35.917 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.917 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.917 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:35.917 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790500+0000", 2024-09-16T20:49:35.917 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3780000000000003e-06 2024-09-16T20:49:35.917 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.917 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.917 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:35.917 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790505+0000", 2024-09-16T20:49:35.917 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1560000000000003e-06 2024-09-16T20:49:35.918 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.918 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.918 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:35.918 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790559+0000", 2024-09-16T20:49:35.918 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3494e-05 2024-09-16T20:49:35.918 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.918 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.918 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:49:35.918 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790566+0000", 2024-09-16T20:49:35.918 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7649999999999993e-06 2024-09-16T20:49:35.918 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.918 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.918 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:35.918 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:09.557198+0000", 2024-09-16T20:49:35.919 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.76663136200000004 2024-09-16T20:49:35.919 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.919 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.919 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:35.919 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:09.557211+0000", 2024-09-16T20:49:35.919 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3458999999999999e-05 2024-09-16T20:49:35.919 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:35.919 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:35.919 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:35.919 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.919 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.919 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3863 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1232896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:35.919 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896432+0000", 2024-09-16T20:49:35.919 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 21.015591011000001, 2024-09-16T20:49:35.920 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046223693000000003, 2024-09-16T20:49:35.920 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:35.920 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:35.920 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:35.920 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:35.920 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:35.920 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:35.920 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3863 2024-09-16T20:49:35.920 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.920 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:35.920 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.920 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:35.920 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896432+0000", 2024-09-16T20:49:35.920 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:35.920 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.921 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.921 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:35.921 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896432+0000", 2024-09-16T20:49:35.921 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:35.921 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.921 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.921 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:35.921 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896433+0000", 2024-09-16T20:49:35.921 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.89e-07 2024-09-16T20:49:35.921 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.921 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.921 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:35.921 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896435+0000", 2024-09-16T20:49:35.921 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.928e-06 2024-09-16T20:49:35.921 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.922 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.922 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:35.922 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896441+0000", 2024-09-16T20:49:35.922 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.896e-06 2024-09-16T20:49:35.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.922 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.922 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:35.922 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896445+0000", 2024-09-16T20:49:35.922 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7550000000000002e-06 2024-09-16T20:49:35.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.922 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.922 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:35.922 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.942468+0000", 2024-09-16T20:49:35.922 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046022278 2024-09-16T20:49:35.923 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.923 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.923 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:35.923 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.942553+0000", 2024-09-16T20:49:35.923 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5807e-05 2024-09-16T20:49:35.923 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.923 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.923 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:35.923 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.942656+0000", 2024-09-16T20:49:35.923 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010264000000000001 2024-09-16T20:49:35.923 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:35.923 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:35.923 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:35.923 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.924 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.924 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3864 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1236992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:35.924 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896451+0000", 2024-09-16T20:49:35.924 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 21.015572141, 2024-09-16T20:49:35.924 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046415605999999998, 2024-09-16T20:49:35.924 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:35.924 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:35.924 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:35.924 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:35.924 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:35.924 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:35.924 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3864 2024-09-16T20:49:35.924 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.924 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:35.925 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.925 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:35.925 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896451+0000", 2024-09-16T20:49:35.925 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:35.925 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.925 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.925 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:35.925 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896451+0000", 2024-09-16T20:49:35.925 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:35.925 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.925 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.925 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:35.925 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896452+0000", 2024-09-16T20:49:35.926 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1699999999999999e-07 2024-09-16T20:49:35.926 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.926 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.926 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:35.926 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896453+0000", 2024-09-16T20:49:35.926 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6950000000000001e-06 2024-09-16T20:49:35.926 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.926 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.926 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:35.926 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896459+0000", 2024-09-16T20:49:35.926 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6749999999999999e-06 2024-09-16T20:49:35.926 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.926 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.926 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:35.926 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896464+0000", 2024-09-16T20:49:35.927 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1399999999999999e-06 2024-09-16T20:49:35.927 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.927 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.927 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:35.927 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.942695+0000", 2024-09-16T20:49:35.927 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046230965999999998 2024-09-16T20:49:35.927 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.927 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.927 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:35.927 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.942771+0000", 2024-09-16T20:49:35.928 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5859999999999995e-05 2024-09-16T20:49:35.928 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.928 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.928 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:35.928 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.942867+0000", 2024-09-16T20:49:35.928 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5952999999999994e-05 2024-09-16T20:49:35.928 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:35.928 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:35.928 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:35.928 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.928 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.928 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3866 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1241088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:35.928 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896471+0000", 2024-09-16T20:49:35.929 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 21.015552911, 2024-09-16T20:49:35.929 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046590830999999999, 2024-09-16T20:49:35.929 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:35.929 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:35.929 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:35.929 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:35.929 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:35.929 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:35.929 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3866 2024-09-16T20:49:35.929 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.929 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:35.929 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.929 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:35.930 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896471+0000", 2024-09-16T20:49:35.930 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:35.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.930 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.930 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:35.930 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896471+0000", 2024-09-16T20:49:35.930 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:35.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.930 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.930 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:35.930 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896471+0000", 2024-09-16T20:49:35.930 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0900000000000002e-07 2024-09-16T20:49:35.931 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.931 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.931 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:35.931 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896473+0000", 2024-09-16T20:49:35.931 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8899999999999999e-06 2024-09-16T20:49:35.931 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.931 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.931 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:35.931 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896479+0000", 2024-09-16T20:49:35.931 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0179999999999998e-06 2024-09-16T20:49:35.931 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.931 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.932 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:35.932 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896486+0000", 2024-09-16T20:49:35.932 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7240000000000003e-06 2024-09-16T20:49:35.932 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.932 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.932 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:35.932 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.942924+0000", 2024-09-16T20:49:35.932 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046438266999999998 2024-09-16T20:49:35.932 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.932 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.932 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:35.932 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943001+0000", 2024-09-16T20:49:35.933 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6987e-05 2024-09-16T20:49:35.933 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.933 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.933 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:35.933 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943061+0000", 2024-09-16T20:49:35.933 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0535999999999998e-05 2024-09-16T20:49:35.933 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:35.933 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:35.933 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:35.933 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.933 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.933 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3868 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1245184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:35.933 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896492+0000", 2024-09-16T20:49:35.934 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 21.015531360000001, 2024-09-16T20:49:35.934 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046738590000000003, 2024-09-16T20:49:35.934 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:35.934 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:35.934 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:35.934 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:35.934 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:35.934 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:35.934 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3868 2024-09-16T20:49:35.934 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.934 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:35.934 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.935 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:35.935 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896492+0000", 2024-09-16T20:49:35.935 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:35.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.935 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.935 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:35.935 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896492+0000", 2024-09-16T20:49:35.935 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:35.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.935 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.935 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:35.935 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896492+0000", 2024-09-16T20:49:35.935 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9499999999999998e-07 2024-09-16T20:49:35.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.936 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.936 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:35.936 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896494+0000", 2024-09-16T20:49:35.936 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.973e-06 2024-09-16T20:49:35.936 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.936 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.936 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:35.936 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896501+0000", 2024-09-16T20:49:35.936 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7719999999999997e-06 2024-09-16T20:49:35.936 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.936 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.936 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:35.936 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896506+0000", 2024-09-16T20:49:35.936 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2089999999999996e-06 2024-09-16T20:49:35.936 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.937 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.937 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:35.937 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943093+0000", 2024-09-16T20:49:35.937 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046586799999999998 2024-09-16T20:49:35.937 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.937 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.937 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:35.937 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943148+0000", 2024-09-16T20:49:35.937 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4385999999999997e-05 2024-09-16T20:49:35.937 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.937 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.937 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:35.937 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943231+0000", 2024-09-16T20:49:35.938 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3054999999999997e-05 2024-09-16T20:49:35.938 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:35.938 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:35.938 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:35.938 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.938 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.938 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3870 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1249280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:35.938 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896514+0000", 2024-09-16T20:49:35.938 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 21.015509732999998, 2024-09-16T20:49:35.938 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046893815999999998, 2024-09-16T20:49:35.938 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:35.938 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:35.938 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:35.938 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:35.939 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:35.939 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:35.939 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3870 2024-09-16T20:49:35.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.939 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:35.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.939 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:35.939 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896514+0000", 2024-09-16T20:49:35.939 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:35.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.939 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:35.939 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896514+0000", 2024-09-16T20:49:35.939 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:35.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:35.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896514+0000", 2024-09-16T20:49:35.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9299999999999999e-07 2024-09-16T20:49:35.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:35.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896521+0000", 2024-09-16T20:49:35.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8199999999999999e-06 2024-09-16T20:49:35.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:35.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896529+0000", 2024-09-16T20:49:35.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0120000000000005e-06 2024-09-16T20:49:35.941 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.941 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.941 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:35.941 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896534+0000", 2024-09-16T20:49:35.941 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5779999999999998e-06 2024-09-16T20:49:35.941 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.941 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.941 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:35.941 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943286+0000", 2024-09-16T20:49:35.941 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046751514000000001 2024-09-16T20:49:35.941 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.941 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.941 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:35.941 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943343+0000", 2024-09-16T20:49:35.941 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7216000000000002e-05 2024-09-16T20:49:35.941 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.942 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.942 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:35.942 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943407+0000", 2024-09-16T20:49:35.942 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4282999999999994e-05 2024-09-16T20:49:35.942 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:35.942 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:35.942 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:35.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.942 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.942 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3872 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1253376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:35.942 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896561+0000", 2024-09-16T20:49:35.942 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 21.015462338999999, 2024-09-16T20:49:35.942 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.04705165, 2024-09-16T20:49:35.942 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:35.942 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:35.943 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:35.943 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:35.943 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:35.943 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:35.943 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3872 2024-09-16T20:49:35.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.943 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:35.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.943 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:35.943 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896561+0000", 2024-09-16T20:49:35.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:35.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.943 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:35.943 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896561+0000", 2024-09-16T20:49:35.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:35.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.944 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:35.944 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896561+0000", 2024-09-16T20:49:35.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4999999999999998e-07 2024-09-16T20:49:35.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.944 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:35.944 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896563+0000", 2024-09-16T20:49:35.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0140000000000001e-06 2024-09-16T20:49:35.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.944 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:35.945 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896569+0000", 2024-09-16T20:49:35.945 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6450000000000002e-06 2024-09-16T20:49:35.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.945 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:35.945 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896572+0000", 2024-09-16T20:49:35.945 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.174e-06 2024-09-16T20:49:35.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.945 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:35.945 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943456+0000", 2024-09-16T20:49:35.945 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046883552000000002 2024-09-16T20:49:35.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.946 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:35.946 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943531+0000", 2024-09-16T20:49:35.946 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5080000000000006e-05 2024-09-16T20:49:35.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.946 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.946 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:35.946 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943613+0000", 2024-09-16T20:49:35.946 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1835000000000005e-05 2024-09-16T20:49:35.946 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:35.946 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:35.946 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:35.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.946 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.946 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3874 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1257472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:35.946 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896577+0000", 2024-09-16T20:49:35.947 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 21.015446824000001, 2024-09-16T20:49:35.947 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047187944000000002, 2024-09-16T20:49:35.947 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:35.947 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:35.947 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:35.947 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:35.947 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:35.947 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:35.947 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3874 2024-09-16T20:49:35.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.947 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:35.947 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.947 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:35.947 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896577+0000", 2024-09-16T20:49:35.948 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:35.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.948 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:35.948 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896577+0000", 2024-09-16T20:49:35.948 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:35.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.948 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:35.948 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896577+0000", 2024-09-16T20:49:35.948 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1199999999999999e-07 2024-09-16T20:49:35.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.949 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:35.949 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896578+0000", 2024-09-16T20:49:35.949 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.237e-06 2024-09-16T20:49:35.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.949 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.949 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:35.952 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896583+0000", 2024-09-16T20:49:35.952 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5859999999999998e-06 2024-09-16T20:49:35.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.952 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.952 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:35.952 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896587+0000", 2024-09-16T20:49:35.952 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3100000000000002e-06 2024-09-16T20:49:35.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.952 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.952 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:35.953 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943661+0000", 2024-09-16T20:49:35.953 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047073639 2024-09-16T20:49:35.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.953 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.953 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:35.953 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943712+0000", 2024-09-16T20:49:35.953 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1190999999999998e-05 2024-09-16T20:49:35.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.953 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.953 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:35.953 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943765+0000", 2024-09-16T20:49:35.953 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2769000000000002e-05 2024-09-16T20:49:35.953 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:35.953 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:35.953 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:35.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.954 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.954 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3876 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1261568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:35.954 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896593+0000", 2024-09-16T20:49:35.954 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 21.015430246000001, 2024-09-16T20:49:35.954 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047362866000000003, 2024-09-16T20:49:35.954 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:35.954 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:35.954 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:35.954 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:35.954 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:35.954 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:35.954 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3876 2024-09-16T20:49:35.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.955 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:35.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.955 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:35.955 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896593+0000", 2024-09-16T20:49:35.955 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:35.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.955 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:35.955 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896593+0000", 2024-09-16T20:49:35.955 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:35.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.955 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:35.955 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896594+0000", 2024-09-16T20:49:35.955 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1600000000000002e-07 2024-09-16T20:49:35.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.956 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:35.956 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896595+0000", 2024-09-16T20:49:35.956 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.756e-06 2024-09-16T20:49:35.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.956 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:35.956 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896601+0000", 2024-09-16T20:49:35.956 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9309999999999996e-06 2024-09-16T20:49:35.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.956 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:35.956 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896606+0000", 2024-09-16T20:49:35.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1089999999999997e-06 2024-09-16T20:49:35.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.957 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:35.957 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943790+0000", 2024-09-16T20:49:35.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047183414 2024-09-16T20:49:35.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.957 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:35.957 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943869+0000", 2024-09-16T20:49:35.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9122000000000006e-05 2024-09-16T20:49:35.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.957 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:35.958 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943956+0000", 2024-09-16T20:49:35.958 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7118000000000002e-05 2024-09-16T20:49:35.958 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:35.958 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:35.958 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:35.958 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.958 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.958 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3879 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1265664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:35.958 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896613+0000", 2024-09-16T20:49:35.958 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 21.015410603999999, 2024-09-16T20:49:35.958 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047528204999999997, 2024-09-16T20:49:35.958 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:35.958 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:35.958 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:35.959 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:35.959 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:35.959 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:35.959 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3879 2024-09-16T20:49:35.959 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.959 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:35.959 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.959 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:35.959 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896613+0000", 2024-09-16T20:49:35.959 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:35.959 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.959 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.959 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:35.959 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896613+0000", 2024-09-16T20:49:35.959 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:35.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.960 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:35.960 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896613+0000", 2024-09-16T20:49:35.960 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.84e-07 2024-09-16T20:49:35.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.960 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:35.960 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896615+0000", 2024-09-16T20:49:35.960 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9970000000000001e-06 2024-09-16T20:49:35.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.960 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:35.960 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896622+0000", 2024-09-16T20:49:35.961 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3500000000000002e-06 2024-09-16T20:49:35.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.961 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.961 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:35.961 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896627+0000", 2024-09-16T20:49:35.961 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2490000000000001e-06 2024-09-16T20:49:35.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.961 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.961 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:35.961 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943996+0000", 2024-09-16T20:49:35.961 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047369175999999999 2024-09-16T20:49:35.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.961 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.961 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:35.962 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944082+0000", 2024-09-16T20:49:35.962 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5859000000000006e-05 2024-09-16T20:49:35.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.962 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:35.962 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944141+0000", 2024-09-16T20:49:35.962 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9190000000000001e-05 2024-09-16T20:49:35.962 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:35.962 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:35.962 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:35.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.962 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3881 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1269760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:35.963 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896634+0000", 2024-09-16T20:49:35.963 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 21.015389690999999, 2024-09-16T20:49:35.963 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047692834000000003, 2024-09-16T20:49:35.963 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:35.963 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:35.963 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:35.963 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:35.963 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:35.963 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:35.963 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3881 2024-09-16T20:49:35.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.963 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:35.963 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.963 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:35.964 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896634+0000", 2024-09-16T20:49:35.964 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:35.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.964 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.964 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:35.964 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896634+0000", 2024-09-16T20:49:35.964 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:35.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.964 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.964 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:35.964 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896634+0000", 2024-09-16T20:49:35.964 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-16T20:49:35.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.964 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.965 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:35.965 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896636+0000", 2024-09-16T20:49:35.965 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.216e-06 2024-09-16T20:49:35.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.965 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.965 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:35.965 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896640+0000", 2024-09-16T20:49:35.965 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1749999999999998e-06 2024-09-16T20:49:35.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.965 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.965 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:35.965 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896643+0000", 2024-09-16T20:49:35.965 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8779999999999998e-06 2024-09-16T20:49:35.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.966 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:35.966 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944178+0000", 2024-09-16T20:49:35.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047534633 2024-09-16T20:49:35.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.966 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:35.966 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944231+0000", 2024-09-16T20:49:35.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3338999999999999e-05 2024-09-16T20:49:35.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.966 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:35.966 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944327+0000", 2024-09-16T20:49:35.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5223000000000006e-05 2024-09-16T20:49:35.967 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:35.967 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:35.967 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:35.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.967 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3882 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1273856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:35.967 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896647+0000", 2024-09-16T20:49:35.967 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 21.015376530000001, 2024-09-16T20:49:35.967 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047858886000000003, 2024-09-16T20:49:35.967 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:35.967 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:35.967 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:35.967 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:35.967 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:35.968 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:35.968 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3882 2024-09-16T20:49:35.968 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.968 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:35.968 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.968 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:35.968 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896647+0000", 2024-09-16T20:49:35.968 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:35.968 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.968 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.968 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:35.968 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896647+0000", 2024-09-16T20:49:35.968 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:35.968 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.969 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.969 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:35.969 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896647+0000", 2024-09-16T20:49:35.969 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4400000000000001e-07 2024-09-16T20:49:35.969 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.969 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.969 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:35.969 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896648+0000", 2024-09-16T20:49:35.969 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0499999999999999e-06 2024-09-16T20:49:35.969 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.969 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.969 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:35.969 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896651+0000", 2024-09-16T20:49:35.969 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9809999999999999e-06 2024-09-16T20:49:35.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.970 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.970 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:35.970 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896654+0000", 2024-09-16T20:49:35.970 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4439999999999998e-06 2024-09-16T20:49:35.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.970 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.970 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:35.970 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944361+0000", 2024-09-16T20:49:35.970 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047707408 2024-09-16T20:49:35.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.970 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.970 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:35.970 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944405+0000", 2024-09-16T20:49:35.970 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4107000000000003e-05 2024-09-16T20:49:35.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.971 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.971 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:35.971 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944506+0000", 2024-09-16T20:49:35.971 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000100652 2024-09-16T20:49:35.971 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:35.971 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:35.971 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:35.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.971 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.971 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3883 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1277952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:35.971 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896656+0000", 2024-09-16T20:49:35.971 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 21.015367019999999, 2024-09-16T20:49:35.971 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047975265000000003, 2024-09-16T20:49:35.972 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:35.972 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:35.972 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:35.972 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:35.972 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:35.972 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:35.972 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3883 2024-09-16T20:49:35.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.972 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:35.972 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.972 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:35.973 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896656+0000", 2024-09-16T20:49:35.973 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:35.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.973 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.973 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:35.973 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896656+0000", 2024-09-16T20:49:35.973 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:35.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.973 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.973 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:35.973 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896657+0000", 2024-09-16T20:49:35.973 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7000000000000001e-07 2024-09-16T20:49:35.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.973 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.974 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:35.974 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896658+0000", 2024-09-16T20:49:35.974 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5850000000000001e-06 2024-09-16T20:49:35.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.974 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:35.974 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896664+0000", 2024-09-16T20:49:35.974 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.609e-06 2024-09-16T20:49:35.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.974 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:35.974 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896668+0000", 2024-09-16T20:49:35.974 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.121e-06 2024-09-16T20:49:35.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.975 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.975 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:35.975 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944542+0000", 2024-09-16T20:49:35.975 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047873838000000002 2024-09-16T20:49:35.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.975 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.975 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:35.975 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944579+0000", 2024-09-16T20:49:35.975 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6943000000000002e-05 2024-09-16T20:49:35.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.975 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.975 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:35.976 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944632+0000", 2024-09-16T20:49:35.976 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2899000000000003e-05 2024-09-16T20:49:35.976 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:35.976 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:35.976 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:35.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.976 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.976 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3886 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1282048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:35.976 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896674+0000", 2024-09-16T20:49:35.976 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 21.015349051000001, 2024-09-16T20:49:35.976 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048152421000000001, 2024-09-16T20:49:35.977 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:35.977 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:35.977 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:35.977 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:35.977 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:35.977 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:35.977 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3886 2024-09-16T20:49:35.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.977 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:35.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.977 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:35.977 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896674+0000", 2024-09-16T20:49:35.977 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:35.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.978 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.978 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:35.978 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896674+0000", 2024-09-16T20:49:35.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:35.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.978 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.978 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:35.978 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896675+0000", 2024-09-16T20:49:35.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6100000000000002e-07 2024-09-16T20:49:35.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.978 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.978 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:35.978 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896677+0000", 2024-09-16T20:49:35.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1270000000000001e-06 2024-09-16T20:49:35.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.978 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.979 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:35.979 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896683+0000", 2024-09-16T20:49:35.979 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1970000000000001e-06 2024-09-16T20:49:35.979 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.979 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.979 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:35.979 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896688+0000", 2024-09-16T20:49:35.979 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.605e-06 2024-09-16T20:49:35.979 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.979 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.979 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:35.979 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944676+0000", 2024-09-16T20:49:35.979 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047988391999999998 2024-09-16T20:49:35.979 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.979 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.980 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:35.980 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944763+0000", 2024-09-16T20:49:35.980 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7363999999999999e-05 2024-09-16T20:49:35.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.980 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.980 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:35.980 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944827+0000", 2024-09-16T20:49:35.980 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3374999999999995e-05 2024-09-16T20:49:35.980 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:35.980 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:35.980 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:35.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.980 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.980 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3888 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1286144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:35.981 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896693+0000", 2024-09-16T20:49:35.981 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 21.015329927, 2024-09-16T20:49:35.981 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048336443999999999, 2024-09-16T20:49:35.981 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:35.981 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:35.981 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:35.981 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:35.981 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:35.981 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:35.981 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3888 2024-09-16T20:49:35.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.981 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:35.981 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.981 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:35.982 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896693+0000", 2024-09-16T20:49:35.982 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:35.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.982 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.982 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:35.982 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896693+0000", 2024-09-16T20:49:35.982 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:35.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.982 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.982 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:35.982 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896694+0000", 2024-09-16T20:49:35.982 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3200000000000001e-07 2024-09-16T20:49:35.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.983 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.983 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:35.983 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896696+0000", 2024-09-16T20:49:35.983 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8989999999999999e-06 2024-09-16T20:49:35.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.983 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.983 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:35.983 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896702+0000", 2024-09-16T20:49:35.983 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8819999999999998e-06 2024-09-16T20:49:35.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.983 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.984 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:35.984 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896707+0000", 2024-09-16T20:49:35.984 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1529999999999996e-06 2024-09-16T20:49:35.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.984 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.984 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:35.984 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944863+0000", 2024-09-16T20:49:35.984 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048156658999999997 2024-09-16T20:49:35.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.984 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.984 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:35.984 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944944+0000", 2024-09-16T20:49:35.984 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0090000000000001e-05 2024-09-16T20:49:35.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.985 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.985 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:35.985 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945030+0000", 2024-09-16T20:49:35.985 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6428999999999995e-05 2024-09-16T20:49:35.985 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:35.985 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:35.985 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:35.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.985 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.985 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3890 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1290240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:35.985 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896713+0000", 2024-09-16T20:49:35.985 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 21.015310057000001, 2024-09-16T20:49:35.986 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048538382999999997, 2024-09-16T20:49:35.986 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:35.986 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:35.986 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:35.986 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:35.986 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:35.986 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:35.986 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3890 2024-09-16T20:49:35.986 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.986 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:35.986 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.986 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:35.986 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896713+0000", 2024-09-16T20:49:35.987 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:35.987 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.987 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.987 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:35.987 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896713+0000", 2024-09-16T20:49:35.987 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:35.987 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.987 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.987 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:35.987 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896714+0000", 2024-09-16T20:49:35.987 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2000000000000001e-07 2024-09-16T20:49:35.987 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.987 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.987 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:35.988 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896716+0000", 2024-09-16T20:49:35.988 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0109999999999999e-06 2024-09-16T20:49:35.988 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.988 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.988 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:35.988 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896720+0000", 2024-09-16T20:49:35.988 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9049999999999999e-06 2024-09-16T20:49:35.988 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.988 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.988 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:35.988 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896722+0000", 2024-09-16T20:49:35.988 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.627e-06 2024-09-16T20:49:35.988 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.989 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.989 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:35.989 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945075+0000", 2024-09-16T20:49:35.989 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048353149999999998 2024-09-16T20:49:35.989 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.989 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.989 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:35.989 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945157+0000", 2024-09-16T20:49:35.989 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1972999999999996e-05 2024-09-16T20:49:35.989 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.989 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.989 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:35.989 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945252+0000", 2024-09-16T20:49:35.990 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4396999999999998e-05 2024-09-16T20:49:35.990 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:35.990 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:35.990 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:35.990 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.990 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.990 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3893 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1294336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:35.990 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896726+0000", 2024-09-16T20:49:35.990 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 21.015297199999999, 2024-09-16T20:49:35.990 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048658686999999999, 2024-09-16T20:49:35.990 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:35.990 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:35.990 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:35.990 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:35.990 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:35.991 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:35.991 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3893 2024-09-16T20:49:35.991 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.991 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:35.991 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.991 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:35.991 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896726+0000", 2024-09-16T20:49:35.991 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:35.991 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.991 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.991 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:35.991 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896726+0000", 2024-09-16T20:49:35.991 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:35.991 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.992 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.992 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:35.992 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896727+0000", 2024-09-16T20:49:35.992 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7899999999999999e-07 2024-09-16T20:49:35.992 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.992 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.992 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:35.992 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896728+0000", 2024-09-16T20:49:35.992 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.68e-06 2024-09-16T20:49:35.992 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.992 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.992 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:35.992 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896734+0000", 2024-09-16T20:49:35.993 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9190000000000004e-06 2024-09-16T20:49:35.993 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.993 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.993 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:35.993 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896739+0000", 2024-09-16T20:49:35.993 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5549999999999996e-06 2024-09-16T20:49:35.993 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.993 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.993 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:35.993 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945271+0000", 2024-09-16T20:49:35.993 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048531795000000003 2024-09-16T20:49:35.993 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.993 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.993 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:35.994 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945330+0000", 2024-09-16T20:49:35.994 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9386999999999999e-05 2024-09-16T20:49:35.994 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.994 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.994 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:35.994 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945385+0000", 2024-09-16T20:49:35.994 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4871999999999999e-05 2024-09-16T20:49:35.994 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:35.994 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:35.994 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:35.994 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.994 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.994 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3895 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1298432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:35.994 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896754+0000", 2024-09-16T20:49:35.995 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 21.015269126, 2024-09-16T20:49:35.995 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048754440000000003, 2024-09-16T20:49:35.995 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:35.995 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:35.995 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:35.995 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:35.995 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:35.995 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:35.995 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3895 2024-09-16T20:49:35.995 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.995 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:35.995 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.995 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:35.995 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896754+0000", 2024-09-16T20:49:35.996 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:35.996 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.996 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.996 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:35.996 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896754+0000", 2024-09-16T20:49:35.996 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:35.996 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.996 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.996 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:35.996 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896755+0000", 2024-09-16T20:49:35.996 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5200000000000002e-07 2024-09-16T20:49:35.996 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.997 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.997 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:35.997 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896762+0000", 2024-09-16T20:49:35.997 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8430000000000004e-06 2024-09-16T20:49:35.997 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.997 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.997 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:35.997 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896782+0000", 2024-09-16T20:49:35.997 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0693e-05 2024-09-16T20:49:35.997 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.997 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.997 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:35.998 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896793+0000", 2024-09-16T20:49:35.998 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0294e-05 2024-09-16T20:49:35.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.998 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.998 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:35.998 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945418+0000", 2024-09-16T20:49:35.998 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048625738000000002 2024-09-16T20:49:35.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.998 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.998 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:35.998 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945457+0000", 2024-09-16T20:49:35.998 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9103999999999997e-05 2024-09-16T20:49:35.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.999 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.999 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:35.999 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945509+0000", 2024-09-16T20:49:35.999 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1316000000000001e-05 2024-09-16T20:49:35.999 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:35.999 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:35.999 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:35.999 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:35.999 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:35.999 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3896 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1302528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:35.999 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896806+0000", 2024-09-16T20:49:35.999 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 21.015216914, 2024-09-16T20:49:35.999 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048831159999999998, 2024-09-16T20:49:36.000 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:36.000 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:36.000 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:36.000 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:36.000 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:36.000 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:36.000 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3896 2024-09-16T20:49:36.000 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:36.000 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:36.000 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:36.000 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:36.000 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896806+0000", 2024-09-16T20:49:36.000 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:36.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:36.001 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:36.001 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:36.001 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896806+0000", 2024-09-16T20:49:36.001 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:36.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:36.001 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:36.001 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:36.001 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896807+0000", 2024-09-16T20:49:36.001 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3e-07 2024-09-16T20:49:36.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:36.001 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:36.001 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:36.002 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896811+0000", 2024-09-16T20:49:36.002 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8310000000000003e-06 2024-09-16T20:49:36.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:36.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:36.002 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:36.002 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896819+0000", 2024-09-16T20:49:36.002 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2470000000000008e-06 2024-09-16T20:49:36.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:36.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:36.002 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:36.002 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896823+0000", 2024-09-16T20:49:36.002 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7919999999999999e-06 2024-09-16T20:49:36.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:36.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:36.003 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:36.003 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945535+0000", 2024-09-16T20:49:36.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048712137000000003 2024-09-16T20:49:36.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:36.003 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:36.003 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:36.003 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945571+0000", 2024-09-16T20:49:36.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6261999999999999e-05 2024-09-16T20:49:36.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:36.003 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:36.003 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:36.003 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945638+0000", 2024-09-16T20:49:36.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6161000000000006e-05 2024-09-16T20:49:36.003 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:36.003 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:36.004 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:36.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:36.004 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:36.004 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3899 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1306624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:36.004 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896829+0000", 2024-09-16T20:49:36.004 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 21.015194713, 2024-09-16T20:49:36.004 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049014110999999999, 2024-09-16T20:49:36.004 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:36.004 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:36.004 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:36.004 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:36.004 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:36.004 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:36.004 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3899 2024-09-16T20:49:36.005 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:36.005 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:36.005 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:36.005 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:36.005 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896829+0000", 2024-09-16T20:49:36.005 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:36.005 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:36.005 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:36.005 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:36.005 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896829+0000", 2024-09-16T20:49:36.005 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:36.005 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:36.006 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:36.006 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:36.006 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896829+0000", 2024-09-16T20:49:36.006 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8999999999999998e-07 2024-09-16T20:49:36.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:36.006 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:36.006 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:36.006 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896831+0000", 2024-09-16T20:49:36.006 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.77e-06 2024-09-16T20:49:36.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:36.006 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:36.006 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:36.006 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896834+0000", 2024-09-16T20:49:36.006 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4800000000000001e-06 2024-09-16T20:49:36.007 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:36.007 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:36.007 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:36.007 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896837+0000", 2024-09-16T20:49:36.007 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8559999999999998e-06 2024-09-16T20:49:36.007 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:36.007 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:36.007 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:36.007 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945663+0000", 2024-09-16T20:49:36.007 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048825529999999999 2024-09-16T20:49:36.007 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:36.007 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:36.007 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:36.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945746+0000", 2024-09-16T20:49:36.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3621000000000005e-05 2024-09-16T20:49:36.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:36.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:36.008 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:36.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945843+0000", 2024-09-16T20:49:36.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6564000000000006e-05 2024-09-16T20:49:36.008 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:36.008 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:36.008 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:36.008 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:36.008 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:36.008 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:36.010 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:36.008+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:36.010 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:49:36.111 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:36.110+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:36.123 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:36.123 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:49:36.123 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:49:36.123 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:36.132 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:49:36.216 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:36.215+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:36.288 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:36.288 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:49:36.288 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:49:36.289 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:49:36.289 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:36.297 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:49:36.312 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:36.311+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:36.414 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:36.413+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:36.446 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:36.446 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:49:36.447 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:49:36.447 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:49:36.447 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.447 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2203 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1038336~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:36.447 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.336000+0000", 2024-09-16T20:49:36.447 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 74.108812682000007, 2024-09-16T20:49:36.447 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049949888999999997, 2024-09-16T20:49:36.447 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:36.447 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:36.447 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:36.447 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:36.447 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:36.448 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:36.448 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2203 2024-09-16T20:49:36.448 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.448 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:36.448 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.448 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:36.448 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336000+0000", 2024-09-16T20:49:36.448 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:36.448 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.448 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.448 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:36.448 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336000+0000", 2024-09-16T20:49:36.448 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:36.449 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.449 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.449 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:36.449 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336000+0000", 2024-09-16T20:49:36.449 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.58e-07 2024-09-16T20:49:36.449 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.449 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.449 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:36.449 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336002+0000", 2024-09-16T20:49:36.449 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0040000000000002e-06 2024-09-16T20:49:36.449 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.449 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.450 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:36.450 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336009+0000", 2024-09-16T20:49:36.450 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9890000000000003e-06 2024-09-16T20:49:36.450 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.450 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.450 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:36.450 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336014+0000", 2024-09-16T20:49:36.450 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4e-06 2024-09-16T20:49:36.450 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.450 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.450 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:36.450 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385759+0000", 2024-09-16T20:49:36.450 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049744151 2024-09-16T20:49:36.451 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.451 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.451 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:36.451 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385847+0000", 2024-09-16T20:49:36.451 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8189000000000006e-05 2024-09-16T20:49:36.451 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.451 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.451 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:36.451 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385950+0000", 2024-09-16T20:49:36.451 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102698 2024-09-16T20:49:36.451 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:36.451 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:36.451 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:36.451 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.452 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.452 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2206 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1046528~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:36.452 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:49:36.452 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 74.108790521000003, 2024-09-16T20:49:36.452 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050147714000000003, 2024-09-16T20:49:36.452 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:36.452 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:36.452 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:36.452 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:36.452 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:36.452 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:36.452 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2206 2024-09-16T20:49:36.452 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.452 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:36.453 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.453 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:36.453 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:49:36.453 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:36.453 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.453 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.453 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:36.453 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:49:36.453 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:36.453 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.453 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.453 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:36.453 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:49:36.454 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8599999999999999e-07 2024-09-16T20:49:36.454 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.454 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.454 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:36.454 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336024+0000", 2024-09-16T20:49:36.454 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.934e-06 2024-09-16T20:49:36.454 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.454 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.454 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:36.454 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336037+0000", 2024-09-16T20:49:36.454 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2496e-05 2024-09-16T20:49:36.454 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.454 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.455 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:36.455 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336042+0000", 2024-09-16T20:49:36.455 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4809999999999997e-06 2024-09-16T20:49:36.455 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.455 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.455 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:36.455 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385980+0000", 2024-09-16T20:49:36.455 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049938376999999999 2024-09-16T20:49:36.455 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.455 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.455 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:36.455 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.386069+0000", 2024-09-16T20:49:36.455 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8368999999999994e-05 2024-09-16T20:49:36.455 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.456 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.456 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:36.456 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.386170+0000", 2024-09-16T20:49:36.456 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000100671 2024-09-16T20:49:36.456 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:36.456 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:36.456 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:36.456 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.456 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.456 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5439 3.a 3:5789f5c3:::rbd_data.110ed9ca540.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 1181696~60416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:36.456 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:49:36.456 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 73.580101940999995, 2024-09-16T20:49:36.456 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10102045599999999, 2024-09-16T20:49:36.456 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:36.457 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:36.457 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:36.457 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:36.457 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:36.457 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:36.457 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5439 2024-09-16T20:49:36.457 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.457 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:36.457 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.457 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:36.457 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:49:36.457 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:36.457 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.458 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.458 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:36.458 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:49:36.458 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:36.458 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.458 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.458 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:36.458 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864711+0000", 2024-09-16T20:49:36.458 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7899999999999997e-07 2024-09-16T20:49:36.458 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.458 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.458 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:36.458 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864760+0000", 2024-09-16T20:49:36.459 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8803e-05 2024-09-16T20:49:36.459 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.459 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.459 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:36.459 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864769+0000", 2024-09-16T20:49:36.459 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0999999999999993e-06 2024-09-16T20:49:36.459 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.459 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.459 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:36.459 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864774+0000", 2024-09-16T20:49:36.459 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8919999999999999e-06 2024-09-16T20:49:36.459 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.459 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.459 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:36.460 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864815+0000", 2024-09-16T20:49:36.460 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1307000000000003e-05 2024-09-16T20:49:36.460 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.460 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.460 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:36.460 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864893+0000", 2024-09-16T20:49:36.460 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7581000000000001e-05 2024-09-16T20:49:36.460 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.460 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.460 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 1", 2024-09-16T20:49:36.460 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864996+0000", 2024-09-16T20:49:36.460 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102725 2024-09-16T20:49:36.460 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.460 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.460 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:36.461 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965442+0000", 2024-09-16T20:49:36.461 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100446012 2024-09-16T20:49:36.461 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.461 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.461 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:36.461 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965618+0000", 2024-09-16T20:49:36.461 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00017653100000000001 2024-09-16T20:49:36.461 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.461 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.461 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:36.461 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965682+0000", 2024-09-16T20:49:36.461 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3905999999999994e-05 2024-09-16T20:49:36.461 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.461 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.462 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:36.462 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965731+0000", 2024-09-16T20:49:36.462 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8819999999999997e-05 2024-09-16T20:49:36.462 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:36.462 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:36.462 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:36.462 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.462 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.462 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4600.0:5439 3.a e37/23 ondisk, result = 0)", 2024-09-16T20:49:36.462 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:49:36.462 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 73.579105792999997, 2024-09-16T20:49:36.462 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099742634999999996, 2024-09-16T20:49:36.463 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:36.463 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:36.463 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:36.463 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:36.463 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.463 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:36.463 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:49:36.463 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:36.463 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.463 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.463 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:36.463 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:49:36.463 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:36.463 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.464 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.464 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:36.464 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:49:36.464 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0900000000000001e-07 2024-09-16T20:49:36.464 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.464 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.464 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:36.464 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865711+0000", 2024-09-16T20:49:36.464 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3739999999999998e-06 2024-09-16T20:49:36.464 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.464 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.464 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:36.464 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965343+0000", 2024-09-16T20:49:36.465 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099632526999999999 2024-09-16T20:49:36.465 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.465 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.465 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:36.465 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965357+0000", 2024-09-16T20:49:36.465 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3713999999999999e-05 2024-09-16T20:49:36.465 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.465 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.465 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:36.465 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965426+0000", 2024-09-16T20:49:36.465 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8745999999999995e-05 2024-09-16T20:49:36.465 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.465 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.465 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:36.465 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965434+0000", 2024-09-16T20:49:36.466 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6910000000000003e-06 2024-09-16T20:49:36.466 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.466 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.466 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:36.466 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965449+0000", 2024-09-16T20:49:36.466 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4874e-05 2024-09-16T20:49:36.466 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:36.466 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:36.466 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:36.466 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.466 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.466 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4882.0:2508 3.12 3:4992771f:::rbd_object_map.110e7c8851c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T20:49:36.467 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:49:36.467 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 49.963302245000001, 2024-09-16T20:49:36.467 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.125007011, 2024-09-16T20:49:36.467 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:36.467 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:36.467 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:36.467 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:36.467 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4882", 2024-09-16T20:49:36.467 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2436923223", 2024-09-16T20:49:36.467 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2508 2024-09-16T20:49:36.467 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.467 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:36.467 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.468 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:36.468 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:49:36.468 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:36.468 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.468 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.468 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:36.468 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:49:36.468 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:36.468 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.468 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.468 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:36.468 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481511+0000", 2024-09-16T20:49:36.468 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0699999999999996e-07 2024-09-16T20:49:36.468 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.469 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.469 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:36.469 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481515+0000", 2024-09-16T20:49:36.469 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0609999999999997e-06 2024-09-16T20:49:36.469 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.469 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.469 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:36.469 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481523+0000", 2024-09-16T20:49:36.469 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1310000000000006e-06 2024-09-16T20:49:36.469 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.469 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.469 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:36.469 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481528+0000", 2024-09-16T20:49:36.470 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4900000000000002e-06 2024-09-16T20:49:36.470 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.470 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.470 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:36.470 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481565+0000", 2024-09-16T20:49:36.470 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7066000000000001e-05 2024-09-16T20:49:36.470 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.470 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.470 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:36.470 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481628+0000", 2024-09-16T20:49:36.470 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3118000000000001e-05 2024-09-16T20:49:36.471 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.471 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.471 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 2", 2024-09-16T20:49:36.471 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481822+0000", 2024-09-16T20:49:36.471 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019446200000000001 2024-09-16T20:49:36.471 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.471 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.471 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:36.471 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605988+0000", 2024-09-16T20:49:36.471 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.124165888 2024-09-16T20:49:36.471 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.471 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.472 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:36.472 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606427+0000", 2024-09-16T20:49:36.472 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00043854899999999999 2024-09-16T20:49:36.472 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.472 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.472 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:36.472 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606476+0000", 2024-09-16T20:49:36.472 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8971999999999998e-05 2024-09-16T20:49:36.472 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.472 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.472 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:36.472 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606517+0000", 2024-09-16T20:49:36.473 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1467000000000001e-05 2024-09-16T20:49:36.473 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:36.473 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:36.473 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:36.473 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.473 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.473 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4882.0:2508 3.12 e57/55 ondisk, result = 0)", 2024-09-16T20:49:36.473 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:49:36.473 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 49.962313745000003, 2024-09-16T20:49:36.473 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12349711400000001, 2024-09-16T20:49:36.473 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:36.473 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:36.473 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:36.474 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:36.474 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.474 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:36.474 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:49:36.474 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:36.474 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.474 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.474 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:36.474 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:49:36.474 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:36.474 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.475 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.475 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:36.475 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:49:36.475 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.06e-07 2024-09-16T20:49:36.475 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.475 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.475 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:36.475 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482502+0000", 2024-09-16T20:49:36.475 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.01e-06 2024-09-16T20:49:36.475 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.475 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.476 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:36.476 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605772+0000", 2024-09-16T20:49:36.476 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.123269687 2024-09-16T20:49:36.476 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.476 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.476 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:36.476 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605791+0000", 2024-09-16T20:49:36.476 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8912000000000001e-05 2024-09-16T20:49:36.476 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.476 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.476 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:36.476 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605974+0000", 2024-09-16T20:49:36.476 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000182836 2024-09-16T20:49:36.477 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.477 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.477 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:36.477 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605982+0000", 2024-09-16T20:49:36.477 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1699999999999997e-06 2024-09-16T20:49:36.477 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.477 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.477 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:36.477 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605996+0000", 2024-09-16T20:49:36.477 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3893000000000001e-05 2024-09-16T20:49:36.477 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:36.477 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:36.478 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:36.478 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.478 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.478 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4882.0:2512 3.f 3:f4d895b7:::rbd_data.110e7c8851c.0000000000000000:head [read 16384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T20:49:36.478 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:49:36.478 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 49.961876715000002, 2024-09-16T20:49:36.478 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12318483, 2024-09-16T20:49:36.478 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:36.478 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:36.478 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:36.478 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:36.478 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4882", 2024-09-16T20:49:36.478 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2436923223", 2024-09-16T20:49:36.479 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2512 2024-09-16T20:49:36.479 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.479 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:36.479 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.479 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:36.479 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:49:36.479 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:36.479 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.479 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.479 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:36.479 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:49:36.479 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:36.479 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.480 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.480 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:36.480 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:49:36.480 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3499999999999995e-07 2024-09-16T20:49:36.480 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.480 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.480 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:36.480 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482943+0000", 2024-09-16T20:49:36.480 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3450000000000002e-06 2024-09-16T20:49:36.480 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.480 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.480 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:36.480 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605772+0000", 2024-09-16T20:49:36.481 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.122829796 2024-09-16T20:49:36.481 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.481 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.481 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:36.481 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605792+0000", 2024-09-16T20:49:36.481 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9916999999999999e-05 2024-09-16T20:49:36.481 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.481 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.481 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:36.481 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605917+0000", 2024-09-16T20:49:36.481 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00012500799999999999 2024-09-16T20:49:36.481 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.481 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.481 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:36.482 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606004+0000", 2024-09-16T20:49:36.482 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6242999999999994e-05 2024-09-16T20:49:36.482 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.482 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.482 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:36.482 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606120+0000", 2024-09-16T20:49:36.482 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000116786 2024-09-16T20:49:36.482 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:36.482 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:36.482 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:36.482 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.482 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.482 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:11702 3.15 3:acf33c8b:::rbd_data.110e747f0779.000000000000001a:head [set-alloc-hint object_size 262144 write_size 262144,write 143360~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:49:36.483 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:49:36.483 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.306884293000003, 2024-09-16T20:49:36.483 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.084801043000000006, 2024-09-16T20:49:36.483 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:36.483 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:36.483 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:36.483 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:36.483 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:49:36.483 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:49:36.483 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 11702 2024-09-16T20:49:36.483 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.483 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:36.483 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.483 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:36.484 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:49:36.484 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:36.484 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.484 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.484 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:36.484 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:49:36.484 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:36.484 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.484 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.484 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:36.484 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137929+0000", 2024-09-16T20:49:36.484 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8400000000000002e-07 2024-09-16T20:49:36.485 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.485 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.485 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:36.485 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137937+0000", 2024-09-16T20:49:36.485 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0260000000000007e-06 2024-09-16T20:49:36.485 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.485 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.485 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:36.485 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137952+0000", 2024-09-16T20:49:36.485 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4659e-05 2024-09-16T20:49:36.485 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.485 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.485 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:36.486 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137960+0000", 2024-09-16T20:49:36.486 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0169999999999996e-06 2024-09-16T20:49:36.486 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.486 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.486 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:36.486 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138010+0000", 2024-09-16T20:49:36.486 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0021000000000001e-05 2024-09-16T20:49:36.486 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.486 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.486 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:36.486 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138114+0000", 2024-09-16T20:49:36.486 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000103797 2024-09-16T20:49:36.486 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.486 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.487 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:49:36.487 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138233+0000", 2024-09-16T20:49:36.487 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000118971 2024-09-16T20:49:36.487 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.487 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.487 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:36.487 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138953+0000", 2024-09-16T20:49:36.487 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00072026000000000002 2024-09-16T20:49:36.487 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.487 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.487 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:36.487 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222609+0000", 2024-09-16T20:49:36.487 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.083656650999999999 2024-09-16T20:49:36.487 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.488 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.488 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:36.488 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222684+0000", 2024-09-16T20:49:36.488 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4369999999999994e-05 2024-09-16T20:49:36.488 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.488 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.488 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:36.488 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222729+0000", 2024-09-16T20:49:36.488 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5287000000000002e-05 2024-09-16T20:49:36.488 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:36.488 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:36.488 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:36.488 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.489 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.489 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:11704 3.9 3:91220b75:::rbd_data.110e747f0779.000000000000001c:head [set-alloc-hint object_size 262144 write_size 262144,write 139264~6656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:49:36.489 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:49:36.489 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.306646292000003, 2024-09-16T20:49:36.489 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.084531133999999994, 2024-09-16T20:49:36.489 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:36.489 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:36.489 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:36.489 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:36.489 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:49:36.489 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:49:36.489 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 11704 2024-09-16T20:49:36.489 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.489 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:36.490 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.490 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:36.490 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:49:36.490 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:36.490 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.490 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.490 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:36.490 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:49:36.490 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:36.490 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.490 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.490 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:36.490 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138167+0000", 2024-09-16T20:49:36.490 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.33e-07 2024-09-16T20:49:36.491 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.491 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.491 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:36.491 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138179+0000", 2024-09-16T20:49:36.491 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2055999999999999e-05 2024-09-16T20:49:36.491 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.491 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.491 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:36.491 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138193+0000", 2024-09-16T20:49:36.491 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3961000000000001e-05 2024-09-16T20:49:36.491 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.491 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.491 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:36.492 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138200+0000", 2024-09-16T20:49:36.492 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4429999999999997e-06 2024-09-16T20:49:36.492 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.492 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.492 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:36.492 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138258+0000", 2024-09-16T20:49:36.492 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7225999999999997e-05 2024-09-16T20:49:36.492 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.492 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.492 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:36.492 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138364+0000", 2024-09-16T20:49:36.492 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000106013 2024-09-16T20:49:36.492 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.492 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.492 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:49:36.493 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138494+0000", 2024-09-16T20:49:36.493 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013059599999999999 2024-09-16T20:49:36.493 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.493 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.493 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:36.493 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.139220+0000", 2024-09-16T20:49:36.493 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00072539800000000004 2024-09-16T20:49:36.493 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.493 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.493 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:36.493 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222608+0000", 2024-09-16T20:49:36.493 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.083388620999999996 2024-09-16T20:49:36.493 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.494 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.494 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:36.494 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222651+0000", 2024-09-16T20:49:36.494 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2228000000000003e-05 2024-09-16T20:49:36.494 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.494 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.494 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:36.494 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222697+0000", 2024-09-16T20:49:36.494 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6659000000000002e-05 2024-09-16T20:49:36.494 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:36.494 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:36.494 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:36.494 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.494 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.495 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5138.0:9358 3.c 3.c25699cc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T20:49:36.495 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:49:36.495 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 9.5944989869999997, 2024-09-16T20:49:36.495 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0018657580000001, 2024-09-16T20:49:36.495 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:36.495 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:36.495 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:49:36.495 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:36.495 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5138", 2024-09-16T20:49:36.495 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:49:36.495 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 9358 2024-09-16T20:49:36.495 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.496 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:36.496 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.496 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:36.496 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:49:36.496 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:36.496 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.496 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.496 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:36.496 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:49:36.496 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:36.496 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.496 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.497 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:36.497 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850314+0000", 2024-09-16T20:49:36.497 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4099999999999999e-07 2024-09-16T20:49:36.497 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.497 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.497 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:36.497 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850316+0000", 2024-09-16T20:49:36.497 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2469999999999998e-06 2024-09-16T20:49:36.497 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.497 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.497 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:36.497 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850322+0000", 2024-09-16T20:49:36.497 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3470000000000004e-06 2024-09-16T20:49:36.498 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.498 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.498 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:36.498 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850327+0000", 2024-09-16T20:49:36.498 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9450000000000001e-06 2024-09-16T20:49:36.498 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.498 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.498 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:36.498 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850376+0000", 2024-09-16T20:49:36.498 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8257000000000002e-05 2024-09-16T20:49:36.498 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.498 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.498 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:49:36.498 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850385+0000", 2024-09-16T20:49:36.499 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9800000000000004e-06 2024-09-16T20:49:36.499 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.499 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.499 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:36.499 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852167+0000", 2024-09-16T20:49:36.499 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0017825659999999 2024-09-16T20:49:36.499 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.499 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.499 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:36.499 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852179+0000", 2024-09-16T20:49:36.499 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1975e-05 2024-09-16T20:49:36.499 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:36.500 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:36.500 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:36.500 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.500 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.500 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5138.0:9382 3.c 3.c25699cc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T20:49:36.500 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:49:36.500 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 9.5861519709999996, 2024-09-16T20:49:36.500 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.99358404700000003, 2024-09-16T20:49:36.500 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:36.500 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:36.500 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:49:36.500 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:36.500 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5138", 2024-09-16T20:49:36.500 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:49:36.501 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 9382 2024-09-16T20:49:36.501 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.501 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:36.501 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.501 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:36.501 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:49:36.501 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:36.501 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.501 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.501 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:36.501 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:49:36.501 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:36.502 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.502 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.502 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:36.502 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858661+0000", 2024-09-16T20:49:36.502 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0200000000000002e-07 2024-09-16T20:49:36.502 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.502 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.502 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:36.502 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858664+0000", 2024-09-16T20:49:36.502 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.038e-06 2024-09-16T20:49:36.502 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.502 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.502 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:36.503 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858671+0000", 2024-09-16T20:49:36.503 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0110000000000003e-06 2024-09-16T20:49:36.503 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.503 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.503 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:36.503 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858677+0000", 2024-09-16T20:49:36.503 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0959999999999997e-06 2024-09-16T20:49:36.503 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.503 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.503 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:36.503 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858732+0000", 2024-09-16T20:49:36.503 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4543000000000003e-05 2024-09-16T20:49:36.503 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.504 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.504 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:49:36.504 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858739+0000", 2024-09-16T20:49:36.504 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5429999999999996e-06 2024-09-16T20:49:36.504 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.504 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.504 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:36.504 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852236+0000", 2024-09-16T20:49:36.504 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.993496453 2024-09-16T20:49:36.504 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.504 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.505 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:36.505 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852244+0000", 2024-09-16T20:49:36.505 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8610000000000004e-06 2024-09-16T20:49:36.505 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:36.505 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:36.505 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:36.505 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.505 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.505 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:360 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 364544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:49:36.505 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246705+0000", 2024-09-16T20:49:36.505 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.19810778300000001, 2024-09-16T20:49:36.505 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050128442000000002, 2024-09-16T20:49:36.505 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:36.505 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:36.506 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:36.506 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:36.506 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:49:36.506 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:49:36.506 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 360 2024-09-16T20:49:36.506 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.506 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:36.506 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.506 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:36.506 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246705+0000", 2024-09-16T20:49:36.506 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:36.506 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.506 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.506 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:36.506 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246705+0000", 2024-09-16T20:49:36.507 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:36.507 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.507 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.507 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:36.507 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246705+0000", 2024-09-16T20:49:36.507 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3999999999999998e-07 2024-09-16T20:49:36.507 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.507 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.507 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:36.507 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246706+0000", 2024-09-16T20:49:36.507 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1710000000000001e-06 2024-09-16T20:49:36.507 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.507 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.507 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:36.508 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246709+0000", 2024-09-16T20:49:36.508 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0350000000000002e-06 2024-09-16T20:49:36.508 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.508 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.508 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:36.508 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246712+0000", 2024-09-16T20:49:36.508 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5239999999999999e-06 2024-09-16T20:49:36.508 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.508 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.508 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:36.508 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.296641+0000", 2024-09-16T20:49:36.508 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049929310999999997 2024-09-16T20:49:36.508 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.509 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.509 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:36.509 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.296741+0000", 2024-09-16T20:49:36.509 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.9976999999999995e-05 2024-09-16T20:49:36.509 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.509 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.509 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:36.509 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.296833+0000", 2024-09-16T20:49:36.509 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2183999999999997e-05 2024-09-16T20:49:36.509 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:36.509 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:36.509 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:36.509 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.509 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.510 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:361 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 372736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:49:36.510 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246714+0000", 2024-09-16T20:49:36.510 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.19809782300000001, 2024-09-16T20:49:36.510 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050338229999999998, 2024-09-16T20:49:36.510 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:36.510 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:36.510 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:36.510 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:36.510 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:49:36.510 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:49:36.510 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 361 2024-09-16T20:49:36.510 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.510 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:36.510 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.511 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:36.511 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246714+0000", 2024-09-16T20:49:36.511 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:36.511 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.511 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.511 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:36.511 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246714+0000", 2024-09-16T20:49:36.511 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:36.511 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.511 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.511 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:36.511 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246715+0000", 2024-09-16T20:49:36.511 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1899999999999999e-07 2024-09-16T20:49:36.511 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.511 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.512 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:36.512 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246716+0000", 2024-09-16T20:49:36.512 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1659999999999999e-06 2024-09-16T20:49:36.512 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.512 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.512 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:36.512 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246721+0000", 2024-09-16T20:49:36.512 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9830000000000004e-06 2024-09-16T20:49:36.512 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.512 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.512 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:36.512 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246723+0000", 2024-09-16T20:49:36.512 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5679999999999998e-06 2024-09-16T20:49:36.513 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.513 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.513 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:36.513 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.296875+0000", 2024-09-16T20:49:36.513 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050151683000000002 2024-09-16T20:49:36.513 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.513 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.513 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:36.513 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.296959+0000", 2024-09-16T20:49:36.513 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4079999999999995e-05 2024-09-16T20:49:36.513 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.513 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.513 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:36.513 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297053+0000", 2024-09-16T20:49:36.514 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.3530999999999996e-05 2024-09-16T20:49:36.514 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:36.514 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:36.514 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:36.514 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.514 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.514 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:362 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 385024~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:49:36.514 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246727+0000", 2024-09-16T20:49:36.514 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.198085714, 2024-09-16T20:49:36.514 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050546528, 2024-09-16T20:49:36.514 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:36.514 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:36.514 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:36.515 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:36.515 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:49:36.515 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:49:36.515 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 362 2024-09-16T20:49:36.515 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.515 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:36.515 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.515 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:36.515 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246727+0000", 2024-09-16T20:49:36.515 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:36.515 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.515 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.515 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:36.516 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246727+0000", 2024-09-16T20:49:36.516 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:36.516 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.516 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.516 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:36.516 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246727+0000", 2024-09-16T20:49:36.516 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.35e-07 2024-09-16T20:49:36.516 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.516 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.516 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:36.516 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246728+0000", 2024-09-16T20:49:36.516 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.175e-06 2024-09-16T20:49:36.517 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.517 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.517 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:36.517 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246734+0000", 2024-09-16T20:49:36.517 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2160000000000003e-06 2024-09-16T20:49:36.517 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.517 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.517 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:36.517 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246737+0000", 2024-09-16T20:49:36.517 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9450000000000002e-06 2024-09-16T20:49:36.517 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.517 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.517 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:36.518 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297103+0000", 2024-09-16T20:49:36.518 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050365518999999997 2024-09-16T20:49:36.518 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.518 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.518 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:36.518 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297183+0000", 2024-09-16T20:49:36.518 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0208000000000002e-05 2024-09-16T20:49:36.518 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.518 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.518 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:36.518 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297273+0000", 2024-09-16T20:49:36.518 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0229999999999995e-05 2024-09-16T20:49:36.518 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:36.519 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:36.519 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:36.519 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.519 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.519 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:368 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 327680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:49:36.519 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246746+0000", 2024-09-16T20:49:36.519 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.19806643400000001, 2024-09-16T20:49:36.519 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050730747999999999, 2024-09-16T20:49:36.519 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:36.519 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:36.519 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:36.519 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:36.519 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:49:36.520 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:49:36.520 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 368 2024-09-16T20:49:36.520 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.520 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:36.520 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.520 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:36.520 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246746+0000", 2024-09-16T20:49:36.520 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:36.520 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.520 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.520 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:36.520 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246746+0000", 2024-09-16T20:49:36.520 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:36.520 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.521 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.521 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:36.521 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246746+0000", 2024-09-16T20:49:36.521 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8099999999999999e-07 2024-09-16T20:49:36.521 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.521 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.521 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:36.521 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246754+0000", 2024-09-16T20:49:36.521 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4440000000000002e-06 2024-09-16T20:49:36.521 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.521 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.521 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:36.521 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246758+0000", 2024-09-16T20:49:36.521 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3900000000000003e-06 2024-09-16T20:49:36.522 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.522 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.522 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:36.522 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246761+0000", 2024-09-16T20:49:36.522 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.101e-06 2024-09-16T20:49:36.522 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.522 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.522 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:36.522 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297310+0000", 2024-09-16T20:49:36.522 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050549166 2024-09-16T20:49:36.522 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.522 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.522 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:36.522 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297387+0000", 2024-09-16T20:49:36.522 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6658999999999999e-05 2024-09-16T20:49:36.523 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.523 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.523 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:36.523 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297477+0000", 2024-09-16T20:49:36.523 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9707e-05 2024-09-16T20:49:36.523 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:36.523 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:36.523 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:36.523 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.523 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.523 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:369 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 339968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:49:36.523 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246766+0000", 2024-09-16T20:49:36.523 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.19804670899999999, 2024-09-16T20:49:36.523 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050911291999999997, 2024-09-16T20:49:36.524 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:36.524 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:36.524 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:36.524 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:36.524 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:49:36.524 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:49:36.524 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 369 2024-09-16T20:49:36.524 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.524 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:36.524 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.524 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:36.524 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246766+0000", 2024-09-16T20:49:36.524 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:36.524 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.524 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.525 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:36.525 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246766+0000", 2024-09-16T20:49:36.525 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:36.525 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.525 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.525 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:36.525 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246766+0000", 2024-09-16T20:49:36.525 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3200000000000001e-07 2024-09-16T20:49:36.525 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.525 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.525 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:36.525 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246767+0000", 2024-09-16T20:49:36.525 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.252e-06 2024-09-16T20:49:36.526 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.526 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.526 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:36.526 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246770+0000", 2024-09-16T20:49:36.526 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1049999999999999e-06 2024-09-16T20:49:36.526 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.526 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.526 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:36.526 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246773+0000", 2024-09-16T20:49:36.526 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.396e-06 2024-09-16T20:49:36.526 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.526 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.526 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:36.526 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297513+0000", 2024-09-16T20:49:36.526 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050740800000000003 2024-09-16T20:49:36.527 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.527 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.527 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:36.527 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297589+0000", 2024-09-16T20:49:36.527 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5318e-05 2024-09-16T20:49:36.527 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.527 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.527 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:36.527 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297677+0000", 2024-09-16T20:49:36.527 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8189000000000006e-05 2024-09-16T20:49:36.527 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:36.527 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:36.527 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:36.527 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.527 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.528 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:370 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 348160~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:49:36.528 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246776+0000", 2024-09-16T20:49:36.528 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.19803669400000001, 2024-09-16T20:49:36.528 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051096643999999997, 2024-09-16T20:49:36.528 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:36.528 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:36.528 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:36.528 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:36.528 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:49:36.528 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:49:36.528 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 370 2024-09-16T20:49:36.528 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.528 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:36.528 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.529 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:36.529 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246776+0000", 2024-09-16T20:49:36.529 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:36.529 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.529 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.529 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:36.529 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246776+0000", 2024-09-16T20:49:36.529 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:36.529 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.529 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.529 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:36.529 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246776+0000", 2024-09-16T20:49:36.529 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2499999999999999e-07 2024-09-16T20:49:36.529 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.530 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.530 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:36.530 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246777+0000", 2024-09-16T20:49:36.530 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1510000000000001e-06 2024-09-16T20:49:36.530 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.530 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.530 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:36.530 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246780+0000", 2024-09-16T20:49:36.530 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9610000000000001e-06 2024-09-16T20:49:36.530 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.530 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.530 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:36.530 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246783+0000", 2024-09-16T20:49:36.530 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6129999999999998e-06 2024-09-16T20:49:36.530 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.531 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.531 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:36.531 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297713+0000", 2024-09-16T20:49:36.531 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050929950000000002 2024-09-16T20:49:36.531 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.531 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.531 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:36.531 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297786+0000", 2024-09-16T20:49:36.531 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3096000000000006e-05 2024-09-16T20:49:36.531 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.531 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.531 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:36.531 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297872+0000", 2024-09-16T20:49:36.532 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6648000000000001e-05 2024-09-16T20:49:36.532 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:36.532 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:36.532 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:36.532 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.532 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.532 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:371 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 380928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:49:36.532 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246791+0000", 2024-09-16T20:49:36.532 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.198021058, 2024-09-16T20:49:36.532 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051287209, 2024-09-16T20:49:36.532 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:36.532 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:36.532 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:36.532 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:36.532 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:49:36.533 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:49:36.533 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 371 2024-09-16T20:49:36.533 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.533 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:36.533 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.533 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:36.533 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246791+0000", 2024-09-16T20:49:36.533 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:36.533 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.533 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.533 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:36.533 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246791+0000", 2024-09-16T20:49:36.533 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:36.533 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.534 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.534 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:36.534 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246792+0000", 2024-09-16T20:49:36.534 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8200000000000001e-07 2024-09-16T20:49:36.534 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.534 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.534 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:36.534 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246793+0000", 2024-09-16T20:49:36.534 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.28e-06 2024-09-16T20:49:36.534 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.534 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.534 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:36.534 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246796+0000", 2024-09-16T20:49:36.535 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.597e-06 2024-09-16T20:49:36.535 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.535 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.535 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:36.535 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246799+0000", 2024-09-16T20:49:36.535 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7089999999999998e-06 2024-09-16T20:49:36.535 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.535 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.535 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:36.535 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297910+0000", 2024-09-16T20:49:36.535 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051110598 2024-09-16T20:49:36.535 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.535 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.535 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:36.536 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297986+0000", 2024-09-16T20:49:36.536 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6711000000000005e-05 2024-09-16T20:49:36.536 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.536 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.536 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:36.536 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.298078+0000", 2024-09-16T20:49:36.536 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2032000000000002e-05 2024-09-16T20:49:36.536 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:36.536 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:36.536 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:36.536 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.536 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.536 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:373 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 376832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:49:36.536 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246904+0000", 2024-09-16T20:49:36.537 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.197908636, 2024-09-16T20:49:36.537 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051310617000000003, 2024-09-16T20:49:36.537 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:36.537 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:36.537 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:36.537 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:36.537 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:49:36.537 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:49:36.537 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 373 2024-09-16T20:49:36.537 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.537 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:36.537 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.537 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:36.537 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246904+0000", 2024-09-16T20:49:36.538 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:36.538 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.538 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.538 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:36.538 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246904+0000", 2024-09-16T20:49:36.538 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:36.538 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.538 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.538 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:36.538 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246904+0000", 2024-09-16T20:49:36.538 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8000000000000001e-07 2024-09-16T20:49:36.538 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.538 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.538 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:36.539 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246908+0000", 2024-09-16T20:49:36.539 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4039999999999997e-06 2024-09-16T20:49:36.539 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.539 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.539 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:36.539 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246913+0000", 2024-09-16T20:49:36.539 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0030000000000002e-06 2024-09-16T20:49:36.539 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.539 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.539 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:36.539 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246917+0000", 2024-09-16T20:49:36.539 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.303e-06 2024-09-16T20:49:36.539 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.539 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.539 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:36.540 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.298114+0000", 2024-09-16T20:49:36.540 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051197724 2024-09-16T20:49:36.540 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.540 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.540 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:36.540 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.298159+0000", 2024-09-16T20:49:36.540 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4882999999999997e-05 2024-09-16T20:49:36.540 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.540 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.540 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:36.540 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.298214+0000", 2024-09-16T20:49:36.540 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4920000000000003e-05 2024-09-16T20:49:36.540 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:36.540 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:36.541 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:36.541 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.541 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.541 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:374 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 389120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:49:36.541 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246927+0000", 2024-09-16T20:49:36.541 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.197885587, 2024-09-16T20:49:36.541 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051498164999999999, 2024-09-16T20:49:36.541 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:36.541 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:36.541 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:36.541 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:36.541 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:49:36.541 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:49:36.542 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 374 2024-09-16T20:49:36.542 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.542 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:36.542 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.542 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:36.542 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246927+0000", 2024-09-16T20:49:36.542 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:36.542 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.542 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.542 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:36.542 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246927+0000", 2024-09-16T20:49:36.542 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:36.542 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.542 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.542 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:36.542 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246927+0000", 2024-09-16T20:49:36.543 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7e-07 2024-09-16T20:49:36.543 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.543 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.543 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:36.543 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246929+0000", 2024-09-16T20:49:36.543 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.607e-06 2024-09-16T20:49:36.543 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.543 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.543 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:36.543 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246932+0000", 2024-09-16T20:49:36.543 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4539999999999999e-06 2024-09-16T20:49:36.543 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.543 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.543 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:36.543 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246935+0000", 2024-09-16T20:49:36.544 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6570000000000001e-06 2024-09-16T20:49:36.544 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.544 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.544 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:36.544 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.298242+0000", 2024-09-16T20:49:36.544 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051306887000000002 2024-09-16T20:49:36.544 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.544 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.544 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:36.544 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.298327+0000", 2024-09-16T20:49:36.544 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5068999999999995e-05 2024-09-16T20:49:36.544 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:36.544 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:36.545 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:36.545 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.298425+0000", 2024-09-16T20:49:36.545 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8021000000000002e-05 2024-09-16T20:49:36.545 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:36.545 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:36.545 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:36.545 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:36.545 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:36.545 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:36.546 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #27, image image_client.0-clone27 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent28 2024-09-16T20:49:36.546 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:36.545+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:36.547 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:49:36.571 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T20:49:36.647 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:36.646+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:36.696 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:36.697 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:49:36.697 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:49:36.697 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:36.708 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:49:36.749 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:36.747+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:36.850 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:36.849+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:36.856 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:36.857 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:49:36.857 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:49:36.857 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:49:36.857 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:36.865 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:49:36.950 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:36.949+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:37.014 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:37.014 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:49:37.014 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:49:37.014 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:49:37.014 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.014 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:49:37.014 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:49:37.014 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 103.757801855, 2024-09-16T20:49:37.014 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67502346999999996, 2024-09-16T20:49:37.014 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:37.014 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:37.014 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:37.014 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:37.015 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:49:37.015 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:49:37.015 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 1 2024-09-16T20:49:37.015 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.015 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:37.015 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.015 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:37.015 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:49:37.015 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.015 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.015 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.015 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:37.015 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:49:37.015 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.015 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.016 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.016 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:37.016 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254128+0000", 2024-09-16T20:49:37.016 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.463e-06 2024-09-16T20:49:37.016 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.016 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.016 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:37.016 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254135+0000", 2024-09-16T20:49:37.016 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1579999999999999e-06 2024-09-16T20:49:37.016 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.016 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.016 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:37.016 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254146+0000", 2024-09-16T20:49:37.016 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1341e-05 2024-09-16T20:49:37.017 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.017 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.017 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:37.017 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254155+0000", 2024-09-16T20:49:37.017 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.1549999999999996e-06 2024-09-16T20:49:37.017 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.017 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.017 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:37.017 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254221+0000", 2024-09-16T20:49:37.017 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5470999999999997e-05 2024-09-16T20:49:37.017 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.017 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.017 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:49:37.018 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254231+0000", 2024-09-16T20:49:37.018 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0592e-05 2024-09-16T20:49:37.018 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.018 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.018 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:37.018 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926711+0000", 2024-09-16T20:49:37.018 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67247946599999997 2024-09-16T20:49:37.018 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.018 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.018 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:37.018 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926910+0000", 2024-09-16T20:49:37.018 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019934 2024-09-16T20:49:37.018 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.018 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.018 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:49:37.019 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.927159+0000", 2024-09-16T20:49:37.019 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00024876599999999998 2024-09-16T20:49:37.019 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.019 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.019 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:37.019 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.928797+0000", 2024-09-16T20:49:37.019 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001638308 2024-09-16T20:49:37.019 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.019 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.019 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:37.019 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929026+0000", 2024-09-16T20:49:37.019 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00022876099999999999 2024-09-16T20:49:37.019 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.019 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.020 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:37.020 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929102+0000", 2024-09-16T20:49:37.020 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6184000000000001e-05 2024-09-16T20:49:37.020 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.020 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.020 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:37.020 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929150+0000", 2024-09-16T20:49:37.020 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7465e-05 2024-09-16T20:49:37.020 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.020 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:37.020 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.020 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.020 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.020 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3571 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:37.021 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481059+0000", 2024-09-16T20:49:37.021 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 47.530868794, 2024-09-16T20:49:37.021 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049572995000000002, 2024-09-16T20:49:37.021 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:37.021 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:37.021 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:37.021 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:37.021 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:37.021 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:37.021 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3571 2024-09-16T20:49:37.021 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.021 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:37.021 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.022 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:37.022 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481059+0000", 2024-09-16T20:49:37.022 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.022 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.022 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.022 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:37.022 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481059+0000", 2024-09-16T20:49:37.022 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.022 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.022 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.022 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:37.022 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481060+0000", 2024-09-16T20:49:37.022 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2599999999999998e-07 2024-09-16T20:49:37.022 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.023 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.023 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:37.023 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481063+0000", 2024-09-16T20:49:37.023 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.999e-06 2024-09-16T20:49:37.023 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.023 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.023 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:37.023 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481067+0000", 2024-09-16T20:49:37.023 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3080000000000001e-06 2024-09-16T20:49:37.023 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.023 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.023 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:37.023 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481070+0000", 2024-09-16T20:49:37.024 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9919999999999999e-06 2024-09-16T20:49:37.024 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.024 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.024 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:37.024 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.522087+0000", 2024-09-16T20:49:37.024 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.041017023 2024-09-16T20:49:37.024 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.024 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.024 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:37.024 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.522191+0000", 2024-09-16T20:49:37.024 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010389499999999999 2024-09-16T20:49:37.024 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.024 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.025 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:37.025 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.522566+0000", 2024-09-16T20:49:37.025 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00037539399999999998 2024-09-16T20:49:37.025 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.025 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.025 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:37.025 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525494+0000", 2024-09-16T20:49:37.025 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0029273070000000001 2024-09-16T20:49:37.025 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.025 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.025 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:37.025 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530355+0000", 2024-09-16T20:49:37.025 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0048612509999999996 2024-09-16T20:49:37.025 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.025 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.026 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:37.026 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530407+0000", 2024-09-16T20:49:37.026 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1817000000000002e-05 2024-09-16T20:49:37.026 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.026 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.026 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:37.026 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530632+0000", 2024-09-16T20:49:37.026 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00022558300000000001 2024-09-16T20:49:37.026 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.026 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:37.026 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.026 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.026 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.026 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3572 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:37.026 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481220+0000", 2024-09-16T20:49:37.026 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 47.530708226000002, 2024-09-16T20:49:37.027 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050452862000000001, 2024-09-16T20:49:37.027 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:37.027 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:37.027 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:37.027 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:37.027 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:37.027 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:37.027 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3572 2024-09-16T20:49:37.027 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.027 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:37.027 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.027 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:37.027 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481220+0000", 2024-09-16T20:49:37.027 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.027 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.028 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.028 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:37.028 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481220+0000", 2024-09-16T20:49:37.028 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.028 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.028 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.028 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:37.028 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481221+0000", 2024-09-16T20:49:37.028 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9400000000000004e-07 2024-09-16T20:49:37.028 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.028 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.028 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:37.028 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481225+0000", 2024-09-16T20:49:37.028 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0230000000000003e-06 2024-09-16T20:49:37.028 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.029 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.029 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:37.029 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481234+0000", 2024-09-16T20:49:37.029 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2920000000000001e-06 2024-09-16T20:49:37.029 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.029 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.029 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:37.029 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481240+0000", 2024-09-16T20:49:37.029 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9390000000000002e-06 2024-09-16T20:49:37.029 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.029 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.029 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:37.029 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.523544+0000", 2024-09-16T20:49:37.030 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.042304475000000001 2024-09-16T20:49:37.030 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.030 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.030 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:37.030 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.523624+0000", 2024-09-16T20:49:37.030 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.975e-05 2024-09-16T20:49:37.030 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.030 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.030 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:37.030 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.523975+0000", 2024-09-16T20:49:37.030 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00035063200000000003 2024-09-16T20:49:37.030 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.030 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.030 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:37.031 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530337+0000", 2024-09-16T20:49:37.031 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0063623589999999997 2024-09-16T20:49:37.031 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.031 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.031 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:37.031 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531592+0000", 2024-09-16T20:49:37.031 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0012550599999999999 2024-09-16T20:49:37.031 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.031 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.031 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:37.031 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531633+0000", 2024-09-16T20:49:37.031 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1192000000000001e-05 2024-09-16T20:49:37.032 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.032 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.032 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:37.032 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531673+0000", 2024-09-16T20:49:37.032 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9345999999999999e-05 2024-09-16T20:49:37.032 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.032 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:37.032 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.032 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.032 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.032 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3573 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:37.032 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481257+0000", 2024-09-16T20:49:37.032 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 47.530671279000003, 2024-09-16T20:49:37.032 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050511027, 2024-09-16T20:49:37.033 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:37.033 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:37.033 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:37.033 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:37.033 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:37.033 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:37.033 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3573 2024-09-16T20:49:37.033 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.033 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:37.033 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.033 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:37.033 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481257+0000", 2024-09-16T20:49:37.033 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.033 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.034 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.034 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:37.034 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481257+0000", 2024-09-16T20:49:37.034 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.034 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.034 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.034 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:37.034 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481257+0000", 2024-09-16T20:49:37.034 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.99e-07 2024-09-16T20:49:37.034 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.034 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.034 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:37.034 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481260+0000", 2024-09-16T20:49:37.034 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1709999999999998e-06 2024-09-16T20:49:37.035 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.035 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.035 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:37.035 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481271+0000", 2024-09-16T20:49:37.035 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0795000000000001e-05 2024-09-16T20:49:37.035 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.035 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.035 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:37.035 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481277+0000", 2024-09-16T20:49:37.035 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1670000000000004e-06 2024-09-16T20:49:37.035 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.035 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.035 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:37.035 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.524647+0000", 2024-09-16T20:49:37.035 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.043369810000000002 2024-09-16T20:49:37.036 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.036 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.036 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:37.036 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.524725+0000", 2024-09-16T20:49:37.036 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7286000000000005e-05 2024-09-16T20:49:37.036 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.036 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.036 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:37.036 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525062+0000", 2024-09-16T20:49:37.036 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00033739499999999999 2024-09-16T20:49:37.036 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.036 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.036 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:37.036 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530692+0000", 2024-09-16T20:49:37.036 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0056301479999999997 2024-09-16T20:49:37.037 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.037 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.037 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:37.037 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531694+0000", 2024-09-16T20:49:37.037 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0010017419999999999 2024-09-16T20:49:37.037 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.037 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.037 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:37.037 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531737+0000", 2024-09-16T20:49:37.037 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2762999999999997e-05 2024-09-16T20:49:37.037 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.037 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.037 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:37.037 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531768+0000", 2024-09-16T20:49:37.037 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1151e-05 2024-09-16T20:49:37.038 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.038 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:37.038 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.038 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.038 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.038 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3574 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:37.038 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:49:37.038 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 47.530638322999998, 2024-09-16T20:49:37.038 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053579441999999998, 2024-09-16T20:49:37.038 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:37.038 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:37.038 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:37.038 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:37.038 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:37.038 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:37.039 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3574 2024-09-16T20:49:37.039 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.039 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:37.039 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.039 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:37.039 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:49:37.039 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.039 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.039 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.039 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:37.039 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:49:37.039 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.039 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.040 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.040 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:37.040 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:49:37.040 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2e-07 2024-09-16T20:49:37.040 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.040 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.040 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:37.040 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481292+0000", 2024-09-16T20:49:37.040 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2630000000000002e-06 2024-09-16T20:49:37.040 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.040 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.040 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:37.041 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481297+0000", 2024-09-16T20:49:37.041 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.296e-06 2024-09-16T20:49:37.041 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.041 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.041 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:37.041 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481300+0000", 2024-09-16T20:49:37.041 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7019999999999999e-06 2024-09-16T20:49:37.041 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.041 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.041 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:37.041 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525533+0000", 2024-09-16T20:49:37.041 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.044232318 2024-09-16T20:49:37.041 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.041 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.042 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:37.042 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525619+0000", 2024-09-16T20:49:37.042 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5892999999999999e-05 2024-09-16T20:49:37.042 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.042 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.042 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:37.042 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525987+0000", 2024-09-16T20:49:37.042 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00036836099999999999 2024-09-16T20:49:37.042 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.042 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.042 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:37.042 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531779+0000", 2024-09-16T20:49:37.042 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0057918240000000001 2024-09-16T20:49:37.042 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.042 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.042 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:37.043 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534747+0000", 2024-09-16T20:49:37.043 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0029681600000000001 2024-09-16T20:49:37.043 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.043 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.043 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:37.043 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534832+0000", 2024-09-16T20:49:37.043 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5180999999999996e-05 2024-09-16T20:49:37.043 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.043 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.043 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:37.043 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534869+0000", 2024-09-16T20:49:37.043 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7024000000000003e-05 2024-09-16T20:49:37.043 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.044 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:37.044 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.044 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.044 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.044 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3575 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:37.044 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:49:37.044 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 47.530623583000001, 2024-09-16T20:49:37.044 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053801463000000001, 2024-09-16T20:49:37.044 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:37.044 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:37.044 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:37.044 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:37.044 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:37.044 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:37.044 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3575 2024-09-16T20:49:37.045 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.045 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:37.045 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.045 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:37.045 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:49:37.045 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.045 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.045 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.045 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:37.045 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:49:37.045 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.045 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.045 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.046 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:37.046 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481305+0000", 2024-09-16T20:49:37.046 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5400000000000002e-07 2024-09-16T20:49:37.046 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.046 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.046 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:37.046 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481307+0000", 2024-09-16T20:49:37.046 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7800000000000001e-06 2024-09-16T20:49:37.046 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.046 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.046 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:37.046 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481312+0000", 2024-09-16T20:49:37.046 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6349999999999997e-06 2024-09-16T20:49:37.046 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.046 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.047 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:37.047 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481315+0000", 2024-09-16T20:49:37.047 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7829999999999999e-06 2024-09-16T20:49:37.047 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.047 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.047 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:37.047 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.526876+0000", 2024-09-16T20:49:37.047 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.045561070000000002 2024-09-16T20:49:37.047 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.047 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.047 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:37.047 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.526954+0000", 2024-09-16T20:49:37.047 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8108000000000005e-05 2024-09-16T20:49:37.047 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.048 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.048 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:37.048 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527282+0000", 2024-09-16T20:49:37.048 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000327483 2024-09-16T20:49:37.048 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.048 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.048 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:37.048 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531784+0000", 2024-09-16T20:49:37.048 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0045020169999999997 2024-09-16T20:49:37.048 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.048 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.048 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:37.048 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534921+0000", 2024-09-16T20:49:37.048 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.003137791 2024-09-16T20:49:37.049 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.049 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.049 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:37.049 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534976+0000", 2024-09-16T20:49:37.049 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4953000000000001e-05 2024-09-16T20:49:37.049 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.049 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.049 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:37.049 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535106+0000", 2024-09-16T20:49:37.049 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000129589 2024-09-16T20:49:37.049 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.049 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:37.049 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.049 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.050 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.050 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3576 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:37.050 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:49:37.050 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 47.530609671000001, 2024-09-16T20:49:37.050 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053955436000000002, 2024-09-16T20:49:37.050 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:37.050 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:37.050 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:37.050 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:37.050 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:37.050 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:37.050 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3576 2024-09-16T20:49:37.050 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.050 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:37.051 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.051 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:37.051 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:49:37.051 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.051 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.051 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.051 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:37.051 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:49:37.051 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.051 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.051 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.051 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:37.051 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481319+0000", 2024-09-16T20:49:37.052 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.53e-07 2024-09-16T20:49:37.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.052 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.052 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:37.052 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481320+0000", 2024-09-16T20:49:37.052 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5230000000000001e-06 2024-09-16T20:49:37.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.052 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.052 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:37.052 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481330+0000", 2024-09-16T20:49:37.052 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0190000000000001e-05 2024-09-16T20:49:37.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.053 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.053 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:37.053 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481333+0000", 2024-09-16T20:49:37.053 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.151e-06 2024-09-16T20:49:37.053 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.053 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.053 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:37.053 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527689+0000", 2024-09-16T20:49:37.053 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046355192000000003 2024-09-16T20:49:37.053 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.053 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.053 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:37.053 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527764+0000", 2024-09-16T20:49:37.053 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5776000000000001e-05 2024-09-16T20:49:37.054 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.054 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.054 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:37.054 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528141+0000", 2024-09-16T20:49:37.054 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00037647599999999999 2024-09-16T20:49:37.054 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.054 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.054 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:37.054 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531788+0000", 2024-09-16T20:49:37.054 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0036470679999999998 2024-09-16T20:49:37.054 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.054 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.055 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:37.055 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535174+0000", 2024-09-16T20:49:37.055 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0033862670000000001 2024-09-16T20:49:37.055 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.055 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.055 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:37.055 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535236+0000", 2024-09-16T20:49:37.055 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1797999999999993e-05 2024-09-16T20:49:37.055 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.055 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.055 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:37.055 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535274+0000", 2024-09-16T20:49:37.055 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7741999999999999e-05 2024-09-16T20:49:37.055 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.056 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:37.056 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.056 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.056 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.056 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3577 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:37.056 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:49:37.056 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 47.530588070999997, 2024-09-16T20:49:37.056 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054073807000000002, 2024-09-16T20:49:37.056 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:37.056 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:37.056 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:37.056 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:37.056 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:37.056 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:37.057 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3577 2024-09-16T20:49:37.057 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.057 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:37.057 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.057 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:37.057 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:49:37.057 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.057 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.057 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.057 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:37.057 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:49:37.057 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.057 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.058 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.058 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:37.058 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:49:37.058 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.96e-07 2024-09-16T20:49:37.058 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.058 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.058 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:37.058 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481342+0000", 2024-09-16T20:49:37.058 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.714e-06 2024-09-16T20:49:37.058 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.058 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.059 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:37.059 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481349+0000", 2024-09-16T20:49:37.059 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4930000000000001e-06 2024-09-16T20:49:37.059 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.059 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.059 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:37.059 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481353+0000", 2024-09-16T20:49:37.059 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4879999999999999e-06 2024-09-16T20:49:37.059 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.059 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.059 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:37.059 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528619+0000", 2024-09-16T20:49:37.059 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047266351999999998 2024-09-16T20:49:37.059 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.060 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.060 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:37.060 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528706+0000", 2024-09-16T20:49:37.060 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6796e-05 2024-09-16T20:49:37.060 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.060 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.060 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:37.060 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529148+0000", 2024-09-16T20:49:37.060 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00044182999999999999 2024-09-16T20:49:37.060 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.060 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.060 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:37.060 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531792+0000", 2024-09-16T20:49:37.061 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0026443119999999998 2024-09-16T20:49:37.061 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.061 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.061 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:37.061 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535319+0000", 2024-09-16T20:49:37.061 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0035266439999999998 2024-09-16T20:49:37.061 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.061 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.061 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:37.061 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535372+0000", 2024-09-16T20:49:37.061 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3207000000000001e-05 2024-09-16T20:49:37.061 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.061 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.061 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:37.062 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535414+0000", 2024-09-16T20:49:37.062 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1675000000000003e-05 2024-09-16T20:49:37.062 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.062 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:37.062 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.062 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.062 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.062 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3578 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:37.062 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:49:37.062 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 47.530566901999997, 2024-09-16T20:49:37.062 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054189586999999997, 2024-09-16T20:49:37.062 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:37.062 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:37.063 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:37.063 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:37.063 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:37.063 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:37.063 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3578 2024-09-16T20:49:37.063 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.063 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:37.063 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.063 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:37.063 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:49:37.063 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.063 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.063 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.063 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:37.063 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:49:37.064 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.064 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.064 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.064 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:37.064 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:49:37.064 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0800000000000001e-07 2024-09-16T20:49:37.064 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.064 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.064 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:37.064 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481363+0000", 2024-09-16T20:49:37.064 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0339999999999999e-06 2024-09-16T20:49:37.065 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.065 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.065 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:37.065 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481371+0000", 2024-09-16T20:49:37.065 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4440000000000002e-06 2024-09-16T20:49:37.065 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.065 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.065 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:37.065 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481374+0000", 2024-09-16T20:49:37.065 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2049999999999998e-06 2024-09-16T20:49:37.065 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.065 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.065 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:37.065 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529555+0000", 2024-09-16T20:49:37.066 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048181202999999999 2024-09-16T20:49:37.066 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.066 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.066 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:37.066 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529616+0000", 2024-09-16T20:49:37.066 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0912000000000002e-05 2024-09-16T20:49:37.066 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.066 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.066 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:37.066 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529939+0000", 2024-09-16T20:49:37.066 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00032269800000000001 2024-09-16T20:49:37.066 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.066 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.066 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:37.067 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532602+0000", 2024-09-16T20:49:37.067 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0026634940000000002 2024-09-16T20:49:37.067 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.067 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.067 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:37.067 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535465+0000", 2024-09-16T20:49:37.067 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.002862149 2024-09-16T20:49:37.067 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.067 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.067 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:37.067 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535516+0000", 2024-09-16T20:49:37.067 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1696000000000001e-05 2024-09-16T20:49:37.067 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.067 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.067 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:37.068 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535551+0000", 2024-09-16T20:49:37.068 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4443999999999998e-05 2024-09-16T20:49:37.068 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.068 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:37.068 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.068 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.068 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.068 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3579 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:37.068 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:49:37.068 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 47.530548463000002, 2024-09-16T20:49:37.068 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054295151, 2024-09-16T20:49:37.068 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:37.068 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:37.069 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:37.069 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:37.069 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:37.069 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:37.069 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3579 2024-09-16T20:49:37.069 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.069 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:37.069 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.069 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:37.069 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:49:37.069 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.069 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.069 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.069 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:37.070 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:49:37.070 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.070 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.070 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.070 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:37.070 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:49:37.070 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3300000000000001e-07 2024-09-16T20:49:37.070 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.070 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.070 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:37.070 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481381+0000", 2024-09-16T20:49:37.070 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.649e-06 2024-09-16T20:49:37.070 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.070 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.071 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:37.071 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481387+0000", 2024-09-16T20:49:37.071 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4519999999999996e-06 2024-09-16T20:49:37.071 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.071 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.071 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:37.071 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481390+0000", 2024-09-16T20:49:37.071 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0170000000000001e-06 2024-09-16T20:49:37.071 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.071 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.071 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:37.071 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530756+0000", 2024-09-16T20:49:37.071 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049365882999999999 2024-09-16T20:49:37.071 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.072 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.072 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:37.072 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530845+0000", 2024-09-16T20:49:37.072 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9197000000000007e-05 2024-09-16T20:49:37.072 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.072 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.072 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:37.072 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531191+0000", 2024-09-16T20:49:37.072 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00034562500000000001 2024-09-16T20:49:37.072 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.072 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.072 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:37.072 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534078+0000", 2024-09-16T20:49:37.072 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0028878179999999999 2024-09-16T20:49:37.073 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.073 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.073 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:37.073 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535589+0000", 2024-09-16T20:49:37.073 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00151085 2024-09-16T20:49:37.073 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.073 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.073 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:37.073 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535639+0000", 2024-09-16T20:49:37.073 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9697000000000002e-05 2024-09-16T20:49:37.073 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.073 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.073 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:37.074 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535675+0000", 2024-09-16T20:49:37.074 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5729999999999998e-05 2024-09-16T20:49:37.074 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.074 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:37.074 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.074 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.074 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.074 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3580 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:37.074 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:49:37.074 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 47.530534830999997, 2024-09-16T20:49:37.074 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054397361999999998, 2024-09-16T20:49:37.074 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:37.074 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:37.074 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:37.074 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:37.074 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:37.075 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:37.075 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3580 2024-09-16T20:49:37.075 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.075 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:37.075 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.075 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:37.075 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:49:37.075 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.075 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.075 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.075 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:37.075 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:49:37.075 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.075 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.075 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.075 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:37.076 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:49:37.076 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2600000000000001e-07 2024-09-16T20:49:37.076 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.076 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.076 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:37.076 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481395+0000", 2024-09-16T20:49:37.076 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.544e-06 2024-09-16T20:49:37.076 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.076 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.076 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:37.076 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481404+0000", 2024-09-16T20:49:37.076 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8850000000000005e-06 2024-09-16T20:49:37.076 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.076 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.077 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:37.077 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481407+0000", 2024-09-16T20:49:37.077 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1540000000000002e-06 2024-09-16T20:49:37.077 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.077 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.077 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:37.077 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531823+0000", 2024-09-16T20:49:37.077 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050415637999999999 2024-09-16T20:49:37.077 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.077 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.077 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:37.077 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531895+0000", 2024-09-16T20:49:37.077 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2179000000000001e-05 2024-09-16T20:49:37.077 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.077 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.078 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:37.078 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532236+0000", 2024-09-16T20:49:37.078 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00034073799999999999 2024-09-16T20:49:37.078 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.078 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.078 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:37.078 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534083+0000", 2024-09-16T20:49:37.078 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001847298 2024-09-16T20:49:37.078 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.078 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.078 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:37.078 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535714+0000", 2024-09-16T20:49:37.078 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0016316810000000001 2024-09-16T20:49:37.078 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.079 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.079 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:37.079 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535759+0000", 2024-09-16T20:49:37.079 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4478999999999999e-05 2024-09-16T20:49:37.079 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.079 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.079 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:37.079 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535791+0000", 2024-09-16T20:49:37.079 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.154e-05 2024-09-16T20:49:37.079 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.079 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:37.079 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.079 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.079 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.080 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3581 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:37.080 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:49:37.080 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 47.530512834, 2024-09-16T20:49:37.080 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.057094403000000002, 2024-09-16T20:49:37.080 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:37.080 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:37.080 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:37.080 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:37.080 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:37.080 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:37.080 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3581 2024-09-16T20:49:37.080 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.080 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:37.081 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.081 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:37.081 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:49:37.081 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.081 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.081 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.081 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:37.081 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:49:37.081 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.081 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.081 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.081 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:37.081 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:49:37.082 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8799999999999998e-07 2024-09-16T20:49:37.082 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.082 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.082 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:37.082 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481418+0000", 2024-09-16T20:49:37.082 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1859999999999999e-06 2024-09-16T20:49:37.082 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.082 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.082 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:37.082 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481423+0000", 2024-09-16T20:49:37.082 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8279999999999999e-06 2024-09-16T20:49:37.082 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.082 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.083 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:37.083 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481426+0000", 2024-09-16T20:49:37.083 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0110000000000001e-06 2024-09-16T20:49:37.083 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.083 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.083 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:37.083 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532631+0000", 2024-09-16T20:49:37.083 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051204514999999999 2024-09-16T20:49:37.083 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.083 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.083 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:37.083 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.533381+0000", 2024-09-16T20:49:37.083 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00075029000000000003 2024-09-16T20:49:37.083 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.084 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.084 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:37.084 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.533739+0000", 2024-09-16T20:49:37.084 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00035773100000000002 2024-09-16T20:49:37.084 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.084 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.084 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:37.084 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535123+0000", 2024-09-16T20:49:37.084 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0013836160000000001 2024-09-16T20:49:37.084 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.084 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.084 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:37.084 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538413+0000", 2024-09-16T20:49:37.084 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0032902510000000001 2024-09-16T20:49:37.084 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.085 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.085 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:37.085 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538470+0000", 2024-09-16T20:49:37.085 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6697999999999998e-05 2024-09-16T20:49:37.085 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.085 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.085 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:37.085 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538510+0000", 2024-09-16T20:49:37.085 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9988999999999999e-05 2024-09-16T20:49:37.085 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.085 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:37.085 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.086 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.086 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.086 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:12732 3.d 3:b752444a:::rbd_header.110e747f0779:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-16T20:49:37.086 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:49:37.086 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.243440256, 2024-09-16T20:49:37.086 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.065302467000000003, 2024-09-16T20:49:37.086 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:37.086 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:37.086 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:37.086 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:37.086 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:49:37.086 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:49:37.086 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 12732 2024-09-16T20:49:37.086 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.087 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:37.087 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.087 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:37.087 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:49:37.087 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.087 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.087 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.087 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:37.087 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:49:37.087 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.087 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.087 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.087 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:37.087 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768490+0000", 2024-09-16T20:49:37.087 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2859999999999998e-06 2024-09-16T20:49:37.088 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.088 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.088 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:37.088 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768501+0000", 2024-09-16T20:49:37.088 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1226999999999999e-05 2024-09-16T20:49:37.088 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.088 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.088 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:37.088 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768517+0000", 2024-09-16T20:49:37.088 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.592e-05 2024-09-16T20:49:37.088 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.088 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.088 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:37.088 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768528+0000", 2024-09-16T20:49:37.089 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0689999999999999e-05 2024-09-16T20:49:37.089 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.089 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.089 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:37.089 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768584+0000", 2024-09-16T20:49:37.089 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5889e-05 2024-09-16T20:49:37.089 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.089 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.089 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:37.089 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768720+0000", 2024-09-16T20:49:37.089 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000136718 2024-09-16T20:49:37.089 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.090 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.090 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:37.090 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.769066+0000", 2024-09-16T20:49:37.090 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000346013 2024-09-16T20:49:37.090 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.090 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.090 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:37.090 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833659+0000", 2024-09-16T20:49:37.090 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.064592822999999994 2024-09-16T20:49:37.090 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.090 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.090 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:37.090 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833719+0000", 2024-09-16T20:49:37.090 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9441000000000002e-05 2024-09-16T20:49:37.091 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.091 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.091 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:37.091 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833758+0000", 2024-09-16T20:49:37.091 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9122000000000003e-05 2024-09-16T20:49:37.091 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.091 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.091 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:37.091 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833790+0000", 2024-09-16T20:49:37.091 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2338000000000001e-05 2024-09-16T20:49:37.091 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.091 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:37.092 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.092 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.092 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.092 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.5089.0:12732 3.d e73/23 ondisk, result = 0)", 2024-09-16T20:49:37.092 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:49:37.092 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.241838086000001, 2024-09-16T20:49:37.092 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.063706456999999994, 2024-09-16T20:49:37.092 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:37.092 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:37.092 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:37.092 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:37.093 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.093 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:37.093 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:49:37.093 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.093 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.093 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.093 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:37.093 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:49:37.093 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.093 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.093 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.093 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:37.093 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770091+0000", 2024-09-16T20:49:37.094 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.53e-06 2024-09-16T20:49:37.094 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.094 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.094 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:37.094 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770104+0000", 2024-09-16T20:49:37.094 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2203e-05 2024-09-16T20:49:37.094 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.094 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.094 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:37.094 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833637+0000", 2024-09-16T20:49:37.094 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.063533201999999997 2024-09-16T20:49:37.094 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.094 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.095 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:37.095 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833655+0000", 2024-09-16T20:49:37.095 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8369e-05 2024-09-16T20:49:37.095 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.095 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.095 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:37.095 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833707+0000", 2024-09-16T20:49:37.095 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2061999999999997e-05 2024-09-16T20:49:37.095 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.095 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.095 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:37.096 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833715+0000", 2024-09-16T20:49:37.096 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5179999999999998e-06 2024-09-16T20:49:37.096 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.096 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.096 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:37.096 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833796+0000", 2024-09-16T20:49:37.096 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1573e-05 2024-09-16T20:49:37.096 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.096 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:37.096 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.096 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.096 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.097 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:363 3.10 3:0ac2a405:::rbd_data.110e39315da6.00000000000000cc:1f [read 385024~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:49:37.097 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246644+0000", 2024-09-16T20:49:37.097 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.76528399999999996, 2024-09-16T20:49:37.097 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050074721000000003, 2024-09-16T20:49:37.097 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:37.097 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:37.097 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:37.097 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:37.097 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:49:37.097 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:49:37.097 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 363 2024-09-16T20:49:37.097 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.097 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:37.098 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.098 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:37.098 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246644+0000", 2024-09-16T20:49:37.098 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.098 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.098 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.098 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:37.098 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246644+0000", 2024-09-16T20:49:37.098 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.098 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.098 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.098 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:37.098 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246645+0000", 2024-09-16T20:49:37.099 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.44e-07 2024-09-16T20:49:37.099 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.099 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.099 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:37.099 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246647+0000", 2024-09-16T20:49:37.099 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.712e-06 2024-09-16T20:49:37.099 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.099 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.099 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:37.099 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246655+0000", 2024-09-16T20:49:37.099 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8299999999999996e-06 2024-09-16T20:49:37.099 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.099 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.100 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:37.100 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246660+0000", 2024-09-16T20:49:37.100 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5040000000000004e-06 2024-09-16T20:49:37.100 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.100 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.100 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:37.100 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.247858+0000", 2024-09-16T20:49:37.100 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0011981019999999999 2024-09-16T20:49:37.100 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.100 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.100 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:37.100 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.247928+0000", 2024-09-16T20:49:37.101 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9919999999999995e-05 2024-09-16T20:49:37.101 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.101 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.101 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:37.101 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.296719+0000", 2024-09-16T20:49:37.101 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048791108999999999 2024-09-16T20:49:37.101 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.101 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:37.101 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.101 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.101 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.101 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:364 3.10 3:0ac2a405:::rbd_data.110e39315da6.00000000000000cc:1f [read 331776~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:49:37.102 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246686+0000", 2024-09-16T20:49:37.102 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.76524196200000005, 2024-09-16T20:49:37.102 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050212992999999997, 2024-09-16T20:49:37.102 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:37.102 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:37.102 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:37.102 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:37.102 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:49:37.102 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:49:37.102 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 364 2024-09-16T20:49:37.102 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.102 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:37.102 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.102 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:37.103 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246686+0000", 2024-09-16T20:49:37.103 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.103 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.103 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.103 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:37.103 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246686+0000", 2024-09-16T20:49:37.103 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.103 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.103 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.103 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:37.103 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246687+0000", 2024-09-16T20:49:37.103 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3600000000000004e-07 2024-09-16T20:49:37.103 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.103 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.104 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:37.104 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246689+0000", 2024-09-16T20:49:37.104 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.4049999999999998e-06 2024-09-16T20:49:37.104 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.104 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.104 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:37.104 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246694+0000", 2024-09-16T20:49:37.104 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7079999999999996e-06 2024-09-16T20:49:37.104 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.104 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.104 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:37.104 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246697+0000", 2024-09-16T20:49:37.104 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1190000000000001e-06 2024-09-16T20:49:37.105 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.105 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.105 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:37.105 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.296755+0000", 2024-09-16T20:49:37.105 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050058004000000003 2024-09-16T20:49:37.105 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.105 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.105 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:37.105 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.296821+0000", 2024-09-16T20:49:37.105 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5759000000000005e-05 2024-09-16T20:49:37.105 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.105 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.105 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:37.106 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.296899+0000", 2024-09-16T20:49:37.106 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8461999999999995e-05 2024-09-16T20:49:37.106 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.106 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:37.106 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.106 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.106 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.106 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:365 3.10 3:0ac2a405:::rbd_data.110e39315da6.00000000000000cc:1f [read 352256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:49:37.106 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246724+0000", 2024-09-16T20:49:37.106 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.765204471, 2024-09-16T20:49:37.106 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050319864999999998, 2024-09-16T20:49:37.106 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:37.106 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:37.107 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:37.107 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:37.107 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:49:37.107 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:49:37.107 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 365 2024-09-16T20:49:37.107 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.107 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:37.107 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.107 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:37.107 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246724+0000", 2024-09-16T20:49:37.107 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.107 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.107 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.107 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:37.107 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246724+0000", 2024-09-16T20:49:37.108 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.108 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.108 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.108 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:37.108 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246724+0000", 2024-09-16T20:49:37.108 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3900000000000005e-07 2024-09-16T20:49:37.108 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.108 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.108 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:37.108 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246727+0000", 2024-09-16T20:49:37.108 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9270000000000001e-06 2024-09-16T20:49:37.108 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.108 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.108 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:37.109 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246734+0000", 2024-09-16T20:49:37.109 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5339999999999996e-06 2024-09-16T20:49:37.109 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.109 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.109 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:37.109 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246739+0000", 2024-09-16T20:49:37.109 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4650000000000002e-06 2024-09-16T20:49:37.109 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.109 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.109 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:37.109 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.296935+0000", 2024-09-16T20:49:37.109 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050196159999999997 2024-09-16T20:49:37.109 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.109 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.110 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:37.110 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.296984+0000", 2024-09-16T20:49:37.110 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9190000000000002e-05 2024-09-16T20:49:37.110 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.110 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.110 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:37.110 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297043+0000", 2024-09-16T20:49:37.110 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9049999999999999e-05 2024-09-16T20:49:37.110 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.110 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:37.110 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.110 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.110 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.110 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:366 3.10 3:0ac2a405:::rbd_data.110e39315da6.00000000000000cc:1f [read 360448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:49:37.111 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246748+0000", 2024-09-16T20:49:37.111 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.76517987099999996, 2024-09-16T20:49:37.111 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050466466000000001, 2024-09-16T20:49:37.111 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:37.111 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:37.111 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:37.111 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:37.111 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:49:37.111 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:49:37.111 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 366 2024-09-16T20:49:37.111 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.111 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:37.111 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.111 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:37.112 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246748+0000", 2024-09-16T20:49:37.112 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.112 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.112 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.112 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:37.112 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246748+0000", 2024-09-16T20:49:37.112 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.112 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.112 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.112 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:37.112 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246749+0000", 2024-09-16T20:49:37.112 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.58e-07 2024-09-16T20:49:37.112 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.112 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.112 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:37.113 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246751+0000", 2024-09-16T20:49:37.113 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3719999999999999e-06 2024-09-16T20:49:37.113 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.113 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.113 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:37.113 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246757+0000", 2024-09-16T20:49:37.113 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1789999999999996e-06 2024-09-16T20:49:37.113 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.113 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.113 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:37.113 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246762+0000", 2024-09-16T20:49:37.113 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3000000000000001e-06 2024-09-16T20:49:37.113 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.114 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.114 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:37.114 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297086+0000", 2024-09-16T20:49:37.114 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050323248000000001 2024-09-16T20:49:37.114 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.114 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.114 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:37.114 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297160+0000", 2024-09-16T20:49:37.114 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4615000000000003e-05 2024-09-16T20:49:37.114 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.114 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.114 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:37.114 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297215+0000", 2024-09-16T20:49:37.114 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4293999999999999e-05 2024-09-16T20:49:37.114 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.115 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:37.115 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.115 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.115 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.115 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:367 3.10 3:0ac2a405:::rbd_data.110e39315da6.00000000000000cc:1f [read 372736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:49:37.115 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246769+0000", 2024-09-16T20:49:37.115 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.76515882700000004, 2024-09-16T20:49:37.115 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050565820999999997, 2024-09-16T20:49:37.115 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:37.115 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:37.115 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:37.115 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:37.115 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:49:37.115 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:49:37.116 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 367 2024-09-16T20:49:37.116 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.116 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:37.116 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.116 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:37.116 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246769+0000", 2024-09-16T20:49:37.116 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.116 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.116 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.116 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:37.116 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246769+0000", 2024-09-16T20:49:37.116 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.116 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.116 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.116 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:37.117 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246770+0000", 2024-09-16T20:49:37.117 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0499999999999999e-07 2024-09-16T20:49:37.117 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.117 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.117 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:37.117 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246772+0000", 2024-09-16T20:49:37.117 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0959999999999999e-06 2024-09-16T20:49:37.117 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.117 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.117 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:37.117 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246778+0000", 2024-09-16T20:49:37.117 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2210000000000002e-06 2024-09-16T20:49:37.117 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.117 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.117 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:37.118 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246783+0000", 2024-09-16T20:49:37.118 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9080000000000003e-06 2024-09-16T20:49:37.118 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.118 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.118 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:37.118 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297243+0000", 2024-09-16T20:49:37.118 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050460307000000003 2024-09-16T20:49:37.118 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.118 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.118 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:37.118 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297284+0000", 2024-09-16T20:49:37.118 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1213e-05 2024-09-16T20:49:37.118 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.118 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.118 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:37.119 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297335+0000", 2024-09-16T20:49:37.119 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0671000000000003e-05 2024-09-16T20:49:37.119 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.119 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:37.119 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.119 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.119 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.119 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:372 3.10 3:0ac2a405:::rbd_data.110e39315da6.00000000000000cc:1f [read 380928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:49:37.119 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246793+0000", 2024-09-16T20:49:37.119 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.76513486100000005, 2024-09-16T20:49:37.119 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050657788000000002, 2024-09-16T20:49:37.119 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:37.119 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:37.119 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:37.119 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:37.120 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:49:37.120 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:49:37.120 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 372 2024-09-16T20:49:37.120 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.120 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:37.120 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.120 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:37.120 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246793+0000", 2024-09-16T20:49:37.120 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.120 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.120 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.120 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:37.120 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246793+0000", 2024-09-16T20:49:37.120 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.120 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.120 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.121 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:37.121 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246794+0000", 2024-09-16T20:49:37.121 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5499999999999998e-07 2024-09-16T20:49:37.121 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.121 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.121 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:37.121 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246796+0000", 2024-09-16T20:49:37.121 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2840000000000001e-06 2024-09-16T20:49:37.121 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.121 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.121 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:37.121 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246802+0000", 2024-09-16T20:49:37.121 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6000000000000003e-06 2024-09-16T20:49:37.121 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.121 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.122 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:37.122 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246808+0000", 2024-09-16T20:49:37.122 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3800000000000002e-06 2024-09-16T20:49:37.122 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.122 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.122 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:37.122 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297358+0000", 2024-09-16T20:49:37.122 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050550204000000001 2024-09-16T20:49:37.122 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.122 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.122 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:37.122 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297401+0000", 2024-09-16T20:49:37.122 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3278999999999997e-05 2024-09-16T20:49:37.122 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.122 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.123 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:37.123 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297451+0000", 2024-09-16T20:49:37.123 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9586000000000003e-05 2024-09-16T20:49:37.123 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.123 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:37.123 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.123 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.123 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:37.123 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:37.124 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:49:37.125 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:37.123+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:37.225 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:37.223+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:37.274 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:37.274 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:49:37.274 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:49:37.274 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:37.283 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:49:37.326 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:37.324+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:37.427 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:37.425+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:37.430 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:37.430 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:49:37.430 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:49:37.430 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:49:37.430 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:37.438 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:49:37.527 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:37.526+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:37.546 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:37.547 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:49:37.547 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:49:37.547 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:49:37.547 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.547 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3712 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1472512~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:37.547 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:49:37.547 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 74.906220265000002, 2024-09-16T20:49:37.547 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050004305999999998, 2024-09-16T20:49:37.547 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:37.547 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:37.547 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:37.547 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:37.547 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:37.547 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:37.548 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3712 2024-09-16T20:49:37.548 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.548 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:37.548 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.548 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:37.548 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:49:37.548 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.548 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.548 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.548 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:37.548 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:49:37.548 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.548 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.548 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.549 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:37.549 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:49:37.549 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0199999999999998e-07 2024-09-16T20:49:37.549 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.549 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.549 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:37.549 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638990+0000", 2024-09-16T20:49:37.549 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.618e-06 2024-09-16T20:49:37.549 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.549 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.549 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:37.549 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638995+0000", 2024-09-16T20:49:37.549 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3940000000000004e-06 2024-09-16T20:49:37.550 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.550 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.550 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:37.550 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639000+0000", 2024-09-16T20:49:37.550 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9309999999999999e-06 2024-09-16T20:49:37.550 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.550 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.550 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:37.550 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688867+0000", 2024-09-16T20:49:37.550 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049866955999999997 2024-09-16T20:49:37.550 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.550 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.550 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:37.550 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688933+0000", 2024-09-16T20:49:37.550 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5359999999999998e-05 2024-09-16T20:49:37.551 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.551 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.551 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:37.551 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688992+0000", 2024-09-16T20:49:37.551 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9744999999999998e-05 2024-09-16T20:49:37.551 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.551 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:37.551 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.551 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.551 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.551 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3714 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1480704~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:37.551 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:49:37.551 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 74.906202675000003, 2024-09-16T20:49:37.551 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050152552000000003, 2024-09-16T20:49:37.551 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:37.552 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:37.552 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:37.552 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:37.552 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:37.552 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:37.552 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3714 2024-09-16T20:49:37.552 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.552 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:37.552 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.552 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:37.552 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:49:37.552 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.552 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.552 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.553 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:37.553 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:49:37.553 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.553 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.553 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.553 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:37.553 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:49:37.553 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0499999999999999e-07 2024-09-16T20:49:37.553 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.553 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.553 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:37.553 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639007+0000", 2024-09-16T20:49:37.553 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5340000000000001e-06 2024-09-16T20:49:37.553 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.554 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.554 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:37.554 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639013+0000", 2024-09-16T20:49:37.554 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3260000000000003e-06 2024-09-16T20:49:37.554 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.554 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.554 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:37.554 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639018+0000", 2024-09-16T20:49:37.554 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2850000000000002e-06 2024-09-16T20:49:37.554 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.554 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.554 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:37.554 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689026+0000", 2024-09-16T20:49:37.554 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.05000798 2024-09-16T20:49:37.555 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.555 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.555 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:37.555 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689077+0000", 2024-09-16T20:49:37.555 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1063999999999998e-05 2024-09-16T20:49:37.555 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.555 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.555 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:37.555 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689158+0000", 2024-09-16T20:49:37.555 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1057999999999995e-05 2024-09-16T20:49:37.555 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.555 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:37.555 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.555 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.556 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.556 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3716 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1488896~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:37.556 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:49:37.556 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 74.906184865, 2024-09-16T20:49:37.556 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050280142, 2024-09-16T20:49:37.556 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:37.556 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:37.556 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:37.556 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:37.556 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:37.556 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:37.556 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3716 2024-09-16T20:49:37.556 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.556 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:37.557 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.557 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:37.557 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:49:37.557 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.557 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.557 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.557 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:37.557 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:49:37.557 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.557 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.557 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.557 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:37.557 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639024+0000", 2024-09-16T20:49:37.557 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9999999999999997e-07 2024-09-16T20:49:37.557 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.557 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.558 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:37.558 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639026+0000", 2024-09-16T20:49:37.558 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.781e-06 2024-09-16T20:49:37.558 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.558 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.558 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:37.558 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639032+0000", 2024-09-16T20:49:37.558 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7780000000000004e-06 2024-09-16T20:49:37.558 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.558 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.558 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:37.558 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639036+0000", 2024-09-16T20:49:37.558 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6909999999999996e-06 2024-09-16T20:49:37.558 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.558 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.559 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:37.559 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689191+0000", 2024-09-16T20:49:37.559 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050155284000000001 2024-09-16T20:49:37.559 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.559 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.559 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:37.559 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689242+0000", 2024-09-16T20:49:37.559 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0662999999999999e-05 2024-09-16T20:49:37.559 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.559 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.559 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:37.559 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689304+0000", 2024-09-16T20:49:37.560 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1345000000000003e-05 2024-09-16T20:49:37.560 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.560 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:37.560 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.560 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.560 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.560 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3719 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1497088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:37.560 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:49:37.560 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 74.906157586999996, 2024-09-16T20:49:37.560 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050399715999999997, 2024-09-16T20:49:37.560 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:37.560 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:37.560 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:37.560 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:37.561 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:37.561 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:37.561 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3719 2024-09-16T20:49:37.561 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.561 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:37.561 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.561 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:37.561 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:49:37.561 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.561 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.561 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.561 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:37.562 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:49:37.562 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.562 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.562 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.562 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:37.562 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:49:37.562 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9700000000000002e-07 2024-09-16T20:49:37.562 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.562 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.562 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:37.562 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639053+0000", 2024-09-16T20:49:37.562 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2900000000000001e-06 2024-09-16T20:49:37.562 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.562 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.563 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:37.563 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639060+0000", 2024-09-16T20:49:37.563 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7750000000000004e-06 2024-09-16T20:49:37.563 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.563 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.563 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:37.563 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639066+0000", 2024-09-16T20:49:37.563 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0639999999999999e-06 2024-09-16T20:49:37.563 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.563 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.563 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:37.563 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689337+0000", 2024-09-16T20:49:37.563 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050270784999999998 2024-09-16T20:49:37.563 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.564 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.564 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:37.564 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689392+0000", 2024-09-16T20:49:37.564 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4920999999999998e-05 2024-09-16T20:49:37.564 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.564 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.564 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:37.564 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689450+0000", 2024-09-16T20:49:37.564 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8483999999999998e-05 2024-09-16T20:49:37.564 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.564 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:37.564 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.564 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.564 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.565 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3721 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1505280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:37.565 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:49:37.565 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 74.906134455, 2024-09-16T20:49:37.565 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050513608000000002, 2024-09-16T20:49:37.565 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:37.565 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:37.565 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:37.565 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:37.565 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:37.565 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:37.565 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3721 2024-09-16T20:49:37.565 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.565 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:37.565 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.566 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:37.566 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:49:37.566 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.566 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.566 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.566 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:37.566 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:49:37.566 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.566 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.566 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.566 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:37.566 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:49:37.566 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7e-07 2024-09-16T20:49:37.566 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.567 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.567 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:37.567 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639076+0000", 2024-09-16T20:49:37.567 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8899999999999999e-06 2024-09-16T20:49:37.567 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.567 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.567 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:37.567 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639083+0000", 2024-09-16T20:49:37.567 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1210000000000001e-06 2024-09-16T20:49:37.567 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.567 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.567 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:37.567 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639089+0000", 2024-09-16T20:49:37.567 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.643e-06 2024-09-16T20:49:37.568 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.568 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.568 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:37.568 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689474+0000", 2024-09-16T20:49:37.568 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050385679000000003 2024-09-16T20:49:37.568 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.568 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.568 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:37.568 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689533+0000", 2024-09-16T20:49:37.568 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8303000000000001e-05 2024-09-16T20:49:37.568 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.568 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.568 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:37.569 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689587+0000", 2024-09-16T20:49:37.569 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4601999999999997e-05 2024-09-16T20:49:37.569 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.569 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:37.569 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.569 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.569 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.569 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3723 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1513472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:37.569 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:49:37.569 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 74.906103650999995, 2024-09-16T20:49:37.569 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050699691999999998, 2024-09-16T20:49:37.569 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:37.569 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:37.569 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:37.570 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:37.570 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:37.570 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:37.570 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3723 2024-09-16T20:49:37.570 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.570 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:37.570 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.570 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:37.570 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:49:37.570 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.570 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.570 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.570 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:37.570 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:49:37.571 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.571 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.571 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.571 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:37.571 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:49:37.571 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3399999999999999e-07 2024-09-16T20:49:37.571 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.571 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.571 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:37.571 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639107+0000", 2024-09-16T20:49:37.571 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3860000000000001e-06 2024-09-16T20:49:37.571 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.571 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.571 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:37.571 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639114+0000", 2024-09-16T20:49:37.572 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9800000000000001e-06 2024-09-16T20:49:37.572 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.572 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.572 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:37.572 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639129+0000", 2024-09-16T20:49:37.572 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4887e-05 2024-09-16T20:49:37.572 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.572 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.572 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:37.572 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689625+0000", 2024-09-16T20:49:37.572 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050495848000000003 2024-09-16T20:49:37.572 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.572 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.573 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:37.573 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689720+0000", 2024-09-16T20:49:37.573 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4821000000000006e-05 2024-09-16T20:49:37.573 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.573 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.573 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:37.573 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689804+0000", 2024-09-16T20:49:37.573 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4235999999999998e-05 2024-09-16T20:49:37.573 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.573 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:37.573 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.573 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.573 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.573 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3725 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1521664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:37.574 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:49:37.574 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 74.906070055000001, 2024-09-16T20:49:37.574 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050797660000000001, 2024-09-16T20:49:37.574 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:37.574 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:37.574 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:37.574 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:37.574 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:37.574 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:37.574 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3725 2024-09-16T20:49:37.574 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.574 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:37.574 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.574 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:37.574 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:49:37.575 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.575 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.575 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.575 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:37.575 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:49:37.575 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.575 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.575 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.575 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:37.575 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639139+0000", 2024-09-16T20:49:37.575 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8999999999999997e-07 2024-09-16T20:49:37.575 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.575 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.575 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:37.575 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639141+0000", 2024-09-16T20:49:37.576 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.182e-06 2024-09-16T20:49:37.576 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.576 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.576 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:37.576 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639154+0000", 2024-09-16T20:49:37.576 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3626e-05 2024-09-16T20:49:37.576 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.576 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.576 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:37.576 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639160+0000", 2024-09-16T20:49:37.576 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6810000000000003e-06 2024-09-16T20:49:37.576 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.576 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.576 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:37.577 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689831+0000", 2024-09-16T20:49:37.577 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050670907000000001 2024-09-16T20:49:37.577 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.577 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.577 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:37.577 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689882+0000", 2024-09-16T20:49:37.577 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0445999999999998e-05 2024-09-16T20:49:37.577 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.577 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.577 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:37.577 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689936+0000", 2024-09-16T20:49:37.577 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4327999999999999e-05 2024-09-16T20:49:37.577 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.577 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:37.577 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.578 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.578 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.578 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3727 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1529856~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:37.578 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:49:37.578 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 74.906034477999995, 2024-09-16T20:49:37.578 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051016922999999999, 2024-09-16T20:49:37.578 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:37.578 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:37.578 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:37.578 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:37.578 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:37.578 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:37.578 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3727 2024-09-16T20:49:37.579 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.579 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:37.579 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.579 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:37.579 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:49:37.579 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.579 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.579 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.579 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:37.579 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:49:37.579 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.579 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.579 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.579 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:37.580 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:49:37.580 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9999999999999998e-07 2024-09-16T20:49:37.580 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.580 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.580 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:37.580 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639183+0000", 2024-09-16T20:49:37.580 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2399999999999996e-06 2024-09-16T20:49:37.580 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.580 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.580 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:37.580 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639195+0000", 2024-09-16T20:49:37.580 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1113e-05 2024-09-16T20:49:37.580 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.580 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.581 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:37.581 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639206+0000", 2024-09-16T20:49:37.581 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1503e-05 2024-09-16T20:49:37.581 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.581 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.581 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:37.581 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689969+0000", 2024-09-16T20:49:37.581 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050762676 2024-09-16T20:49:37.581 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.581 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.581 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:37.581 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690071+0000", 2024-09-16T20:49:37.581 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102163 2024-09-16T20:49:37.581 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.581 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.582 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:37.582 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690191+0000", 2024-09-16T20:49:37.582 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000119728 2024-09-16T20:49:37.582 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.582 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:37.582 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.582 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.582 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.582 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3730 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1538048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:37.582 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:49:37.582 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 74.905992914999999, 2024-09-16T20:49:37.582 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051131467999999999, 2024-09-16T20:49:37.582 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:37.582 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:37.582 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:37.583 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:37.583 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:37.583 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:37.583 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3730 2024-09-16T20:49:37.583 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.583 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:37.583 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.583 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:37.583 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:49:37.583 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.583 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.583 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.583 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:37.583 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:49:37.583 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.584 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.584 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.584 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:37.584 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639216+0000", 2024-09-16T20:49:37.584 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.39e-07 2024-09-16T20:49:37.584 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.584 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.584 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:37.584 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639218+0000", 2024-09-16T20:49:37.584 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.537e-06 2024-09-16T20:49:37.584 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.584 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.584 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:37.584 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639229+0000", 2024-09-16T20:49:37.585 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1122e-05 2024-09-16T20:49:37.585 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.585 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.585 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:37.585 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639237+0000", 2024-09-16T20:49:37.585 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8509999999999999e-06 2024-09-16T20:49:37.585 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.585 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.585 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:37.585 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690227+0000", 2024-09-16T20:49:37.585 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050990147 2024-09-16T20:49:37.585 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.585 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.585 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:37.586 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690278+0000", 2024-09-16T20:49:37.586 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0223999999999999e-05 2024-09-16T20:49:37.586 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.586 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.586 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:37.586 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690347+0000", 2024-09-16T20:49:37.586 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9147999999999996e-05 2024-09-16T20:49:37.586 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.586 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:37.586 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.586 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.586 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.586 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3731 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1546240~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:37.587 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:49:37.587 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 74.905962207000002, 2024-09-16T20:49:37.587 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051276744999999999, 2024-09-16T20:49:37.587 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:37.587 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:37.587 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:37.587 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:37.587 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:37.587 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:37.587 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3731 2024-09-16T20:49:37.587 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.587 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:37.587 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.587 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:37.588 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:49:37.588 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.588 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.588 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.588 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:37.588 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:49:37.588 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.588 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.588 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.588 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:37.588 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:49:37.588 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9900000000000001e-07 2024-09-16T20:49:37.588 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.588 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.589 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:37.589 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639249+0000", 2024-09-16T20:49:37.589 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3630000000000001e-06 2024-09-16T20:49:37.589 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.589 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.589 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:37.589 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639255+0000", 2024-09-16T20:49:37.589 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4960000000000001e-06 2024-09-16T20:49:37.589 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.589 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.589 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:37.589 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639261+0000", 2024-09-16T20:49:37.589 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2839999999999997e-06 2024-09-16T20:49:37.589 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.590 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.590 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:37.590 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690375+0000", 2024-09-16T20:49:37.590 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051114684000000001 2024-09-16T20:49:37.590 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.590 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.590 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:37.590 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690425+0000", 2024-09-16T20:49:37.590 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9379999999999998e-05 2024-09-16T20:49:37.590 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.590 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.590 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:37.590 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690523+0000", 2024-09-16T20:49:37.590 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8139000000000003e-05 2024-09-16T20:49:37.590 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.591 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:37.591 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.591 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.591 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.591 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3732 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1554432~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:37.591 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:49:37.591 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 74.905916425000001, 2024-09-16T20:49:37.591 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051380096, 2024-09-16T20:49:37.591 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:37.591 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:37.591 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:37.591 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:37.591 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:37.591 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:37.591 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3732 2024-09-16T20:49:37.591 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.592 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:37.592 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.592 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:37.592 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:49:37.592 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.592 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.592 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.592 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:37.592 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:49:37.592 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.592 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.592 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.592 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:37.592 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:49:37.592 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6100000000000001e-07 2024-09-16T20:49:37.593 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.593 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.593 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:37.593 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639295+0000", 2024-09-16T20:49:37.593 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.447e-06 2024-09-16T20:49:37.593 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.593 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.593 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:37.593 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639302+0000", 2024-09-16T20:49:37.593 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0570000000000004e-06 2024-09-16T20:49:37.593 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.593 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.593 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:37.593 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639307+0000", 2024-09-16T20:49:37.594 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1429999999999997e-06 2024-09-16T20:49:37.594 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.594 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.594 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:37.594 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690555+0000", 2024-09-16T20:49:37.594 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051247963000000001 2024-09-16T20:49:37.594 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.594 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.594 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:37.594 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690602+0000", 2024-09-16T20:49:37.594 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7602000000000003e-05 2024-09-16T20:49:37.594 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.594 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.594 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:37.595 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690672+0000", 2024-09-16T20:49:37.595 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9423000000000003e-05 2024-09-16T20:49:37.595 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.595 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:37.595 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.595 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.595 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.595 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3734 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1562624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:37.595 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:49:37.595 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 74.905894774999993, 2024-09-16T20:49:37.595 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051490610999999999, 2024-09-16T20:49:37.595 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:37.595 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:37.595 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:37.595 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:37.596 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:37.596 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:37.596 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3734 2024-09-16T20:49:37.596 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.596 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:37.596 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.596 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:37.596 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:49:37.596 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.596 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.596 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.596 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:37.596 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:49:37.597 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.597 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.597 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.597 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:37.597 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639314+0000", 2024-09-16T20:49:37.597 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4400000000000001e-07 2024-09-16T20:49:37.597 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.597 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.597 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:37.597 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639316+0000", 2024-09-16T20:49:37.597 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1459999999999999e-06 2024-09-16T20:49:37.597 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.597 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.597 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:37.598 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639322+0000", 2024-09-16T20:49:37.598 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7610000000000004e-06 2024-09-16T20:49:37.598 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.598 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.598 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:37.598 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639327+0000", 2024-09-16T20:49:37.598 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.186e-06 2024-09-16T20:49:37.598 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.598 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.598 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:37.598 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690698+0000", 2024-09-16T20:49:37.598 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051371223000000001 2024-09-16T20:49:37.599 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.599 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.599 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:37.599 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690748+0000", 2024-09-16T20:49:37.599 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0192999999999999e-05 2024-09-16T20:49:37.599 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.599 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.599 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:37.599 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690804+0000", 2024-09-16T20:49:37.599 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5658000000000001e-05 2024-09-16T20:49:37.599 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.599 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:37.599 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.600 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.600 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.600 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3736 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1570816~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:37.600 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:49:37.600 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 74.905875515999995, 2024-09-16T20:49:37.600 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051596528000000003, 2024-09-16T20:49:37.600 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:37.600 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:37.600 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:37.600 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:37.600 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:37.600 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:37.600 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3736 2024-09-16T20:49:37.601 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.601 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:37.601 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.601 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:37.601 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:49:37.601 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.601 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.601 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.601 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:37.601 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:49:37.601 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.601 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.601 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.601 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:37.602 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:49:37.602 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9999999999999999e-07 2024-09-16T20:49:37.602 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.602 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.602 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:37.602 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639335+0000", 2024-09-16T20:49:37.602 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.635e-06 2024-09-16T20:49:37.602 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.602 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.602 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:37.602 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639340+0000", 2024-09-16T20:49:37.602 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5840000000000002e-06 2024-09-16T20:49:37.602 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.602 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.602 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:37.603 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639346+0000", 2024-09-16T20:49:37.603 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1190000000000002e-06 2024-09-16T20:49:37.603 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.603 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.603 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:37.603 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690817+0000", 2024-09-16T20:49:37.603 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051470721999999997 2024-09-16T20:49:37.603 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.603 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.603 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:37.603 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690875+0000", 2024-09-16T20:49:37.603 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7441000000000001e-05 2024-09-16T20:49:37.603 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.603 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.603 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:37.604 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690929+0000", 2024-09-16T20:49:37.604 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4727e-05 2024-09-16T20:49:37.604 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.604 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:37.604 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.604 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.604 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.604 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5434 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:37.604 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:49:37.604 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 74.680613949999994, 2024-09-16T20:49:37.604 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099755831000000003, 2024-09-16T20:49:37.604 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:37.604 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:37.604 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:37.604 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:37.605 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:37.605 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:37.605 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5434 2024-09-16T20:49:37.605 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.605 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:37.605 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.605 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:37.605 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:49:37.605 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.605 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.605 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.605 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:37.605 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:49:37.605 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.605 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.606 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.606 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:37.606 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864595+0000", 2024-09-16T20:49:37.606 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1099999999999995e-07 2024-09-16T20:49:37.606 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.606 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.606 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:37.606 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864597+0000", 2024-09-16T20:49:37.606 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3989999999999998e-06 2024-09-16T20:49:37.606 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.606 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.606 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:37.606 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864603+0000", 2024-09-16T20:49:37.606 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9309999999999996e-06 2024-09-16T20:49:37.606 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.606 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.607 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:37.607 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864607+0000", 2024-09-16T20:49:37.607 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6200000000000001e-06 2024-09-16T20:49:37.607 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.607 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.607 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:37.607 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964164+0000", 2024-09-16T20:49:37.607 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099557162000000005 2024-09-16T20:49:37.607 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.607 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.607 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:37.607 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964244+0000", 2024-09-16T20:49:37.607 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9597000000000004e-05 2024-09-16T20:49:37.607 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.608 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.608 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:37.608 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964350+0000", 2024-09-16T20:49:37.608 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010641099999999999 2024-09-16T20:49:37.608 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.608 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:37.608 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.608 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.608 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.608 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5435 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 20480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:37.608 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:49:37.608 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 74.680584128000007, 2024-09-16T20:49:37.608 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100897102, 2024-09-16T20:49:37.608 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:37.609 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:37.609 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:37.609 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:37.609 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:37.609 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:37.609 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5435 2024-09-16T20:49:37.609 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.609 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:37.609 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.609 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:37.609 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:49:37.609 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.609 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.609 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.610 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:37.610 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:49:37.610 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.610 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.610 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.610 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:37.610 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:49:37.610 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8000000000000001e-07 2024-09-16T20:49:37.610 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.610 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.610 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:37.610 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864626+0000", 2024-09-16T20:49:37.610 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.745e-06 2024-09-16T20:49:37.610 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.610 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.610 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:37.611 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864633+0000", 2024-09-16T20:49:37.611 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3339999999999998e-06 2024-09-16T20:49:37.611 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.611 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.611 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:37.611 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864636+0000", 2024-09-16T20:49:37.611 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1200000000000002e-06 2024-09-16T20:49:37.611 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.611 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.611 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:37.611 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965299+0000", 2024-09-16T20:49:37.611 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10066352000000001 2024-09-16T20:49:37.611 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.611 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.612 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:37.612 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965390+0000", 2024-09-16T20:49:37.612 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0894000000000001e-05 2024-09-16T20:49:37.612 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.612 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.612 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:37.612 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965521+0000", 2024-09-16T20:49:37.612 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013110899999999999 2024-09-16T20:49:37.612 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.612 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:37.612 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.612 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.612 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.613 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:152 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 327680~32768 out=32768b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:49:37.613 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:49:37.613 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 73.382087186000007, 2024-09-16T20:49:37.613 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11869405800000001, 2024-09-16T20:49:37.613 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:37.613 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:37.613 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:37.613 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:37.613 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:49:37.613 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:49:37.613 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 152 2024-09-16T20:49:37.613 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.613 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:37.614 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.614 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:37.614 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:49:37.614 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.614 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.614 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.614 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:37.614 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:49:37.614 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.614 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.614 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.614 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:37.614 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163122+0000", 2024-09-16T20:49:37.614 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6400000000000003e-07 2024-09-16T20:49:37.614 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.614 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.615 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:37.615 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163124+0000", 2024-09-16T20:49:37.615 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.565e-06 2024-09-16T20:49:37.615 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.615 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.615 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:37.615 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163136+0000", 2024-09-16T20:49:37.615 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1717e-05 2024-09-16T20:49:37.615 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.615 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.615 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:37.615 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163142+0000", 2024-09-16T20:49:37.615 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1750000000000002e-06 2024-09-16T20:49:37.615 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.615 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.616 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:37.616 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163606+0000", 2024-09-16T20:49:37.616 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00046418799999999998 2024-09-16T20:49:37.616 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.616 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.616 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:37.616 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163656+0000", 2024-09-16T20:49:37.616 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9852999999999999e-05 2024-09-16T20:49:37.616 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.616 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.616 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:37.616 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281815+0000", 2024-09-16T20:49:37.616 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11815909600000001 2024-09-16T20:49:37.616 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.616 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:37.617 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.617 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.617 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.617 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:153 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:49:37.617 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:49:37.617 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 73.382054233000005, 2024-09-16T20:49:37.617 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11903462400000001, 2024-09-16T20:49:37.617 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:37.617 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:37.617 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:37.617 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:37.617 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:49:37.617 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:49:37.618 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 153 2024-09-16T20:49:37.618 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.618 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:37.618 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.618 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:37.618 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:49:37.618 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.618 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.618 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.618 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:37.618 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:49:37.618 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.618 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.618 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.618 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:37.619 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163155+0000", 2024-09-16T20:49:37.619 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.37e-07 2024-09-16T20:49:37.619 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.619 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.619 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:37.619 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163157+0000", 2024-09-16T20:49:37.619 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6010000000000002e-06 2024-09-16T20:49:37.619 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.619 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.619 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:37.619 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163165+0000", 2024-09-16T20:49:37.619 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2619999999999998e-06 2024-09-16T20:49:37.619 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.619 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.620 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:37.620 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163171+0000", 2024-09-16T20:49:37.620 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.711e-06 2024-09-16T20:49:37.620 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.620 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.620 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:37.620 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281868+0000", 2024-09-16T20:49:37.620 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118696622 2024-09-16T20:49:37.620 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.620 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.620 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:37.620 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281969+0000", 2024-09-16T20:49:37.620 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101371 2024-09-16T20:49:37.620 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.620 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.621 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:37.621 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282189+0000", 2024-09-16T20:49:37.621 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00021952000000000001 2024-09-16T20:49:37.621 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.621 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:37.621 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.621 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.621 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.621 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:154 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 360448~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:49:37.621 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:49:37.621 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 73.382022461000005, 2024-09-16T20:49:37.621 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11932035100000001, 2024-09-16T20:49:37.621 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:37.621 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:37.622 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:37.622 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:37.622 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:49:37.622 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:49:37.622 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 154 2024-09-16T20:49:37.622 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.622 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:37.622 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.622 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:37.622 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:49:37.622 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.622 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.622 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.622 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:37.622 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:49:37.622 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.623 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.623 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.623 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:37.623 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:49:37.623 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:49:37.623 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.623 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.623 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:37.623 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163189+0000", 2024-09-16T20:49:37.623 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6840000000000001e-06 2024-09-16T20:49:37.623 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.623 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.623 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:37.623 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163196+0000", 2024-09-16T20:49:37.624 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7920000000000004e-06 2024-09-16T20:49:37.624 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.624 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.624 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:37.624 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163201+0000", 2024-09-16T20:49:37.624 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2809999999999999e-06 2024-09-16T20:49:37.624 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.624 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.624 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:37.624 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282219+0000", 2024-09-16T20:49:37.624 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.119018343 2024-09-16T20:49:37.624 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.624 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.624 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:37.624 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282305+0000", 2024-09-16T20:49:37.625 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5865000000000005e-05 2024-09-16T20:49:37.625 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.625 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.625 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:37.625 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282506+0000", 2024-09-16T20:49:37.625 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000200927 2024-09-16T20:49:37.625 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.625 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:37.625 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.625 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.625 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.625 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop(client.4366.0:3088 3.1e e38/23 3:79e5ec01:::rbd_header.110e891a9b2b:head v 38'306, mlcod=36'304)", 2024-09-16T20:49:37.625 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:49:37.625 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 73.381003593000003, 2024-09-16T20:49:37.625 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118102759, 2024-09-16T20:49:37.626 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:37.626 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:37.626 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:37.626 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:37.626 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.626 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:37.626 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:49:37.626 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.626 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.626 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.626 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:37.626 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:49:37.626 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.626 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.626 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.627 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:37.627 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:49:37.627 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.85e-07 2024-09-16T20:49:37.627 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.627 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.627 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:37.627 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164212+0000", 2024-09-16T20:49:37.627 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3999999999999997e-06 2024-09-16T20:49:37.627 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.627 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.627 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:37.627 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281553+0000", 2024-09-16T20:49:37.627 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11734135499999999 2024-09-16T20:49:37.627 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.627 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.628 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:37.628 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281567+0000", 2024-09-16T20:49:37.628 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4188999999999999e-05 2024-09-16T20:49:37.628 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.628 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.628 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:37.628 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281653+0000", 2024-09-16T20:49:37.628 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5385999999999999e-05 2024-09-16T20:49:37.628 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.628 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.628 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:37.628 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281679+0000", 2024-09-16T20:49:37.628 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6689000000000001e-05 2024-09-16T20:49:37.628 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.629 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.629 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:37.629 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282277+0000", 2024-09-16T20:49:37.629 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00059798000000000002 2024-09-16T20:49:37.629 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.629 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.629 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:37.629 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282307+0000", 2024-09-16T20:49:37.629 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9975e-05 2024-09-16T20:49:37.629 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.629 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:37.629 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.629 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.629 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.629 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5255.0:44 3.16 3:6f5476df:::rbd_header.110e25db1f60:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:49:37.630 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:49:37.630 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 1.224700538, 2024-09-16T20:49:37.630 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.121007509, 2024-09-16T20:49:37.630 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:37.630 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:37.630 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:37.630 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:37.630 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5255", 2024-09-16T20:49:37.630 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:49:37.630 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 44 2024-09-16T20:49:37.630 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.630 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:37.630 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.630 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:37.631 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:49:37.631 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.631 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.631 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.631 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:37.631 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:49:37.631 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:37.631 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.631 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.631 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:37.631 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:49:37.631 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0500000000000003e-07 2024-09-16T20:49:37.631 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.631 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.631 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:37.632 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320512+0000", 2024-09-16T20:49:37.632 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2540000000000001e-06 2024-09-16T20:49:37.632 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.632 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.632 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:37.632 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320518+0000", 2024-09-16T20:49:37.632 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1689999999999997e-06 2024-09-16T20:49:37.632 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.632 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.632 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:37.632 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320521+0000", 2024-09-16T20:49:37.632 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5200000000000002e-06 2024-09-16T20:49:37.633 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.633 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.633 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:37.633 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320548+0000", 2024-09-16T20:49:37.633 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6662999999999999e-05 2024-09-16T20:49:37.633 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.633 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.633 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:37.633 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320587+0000", 2024-09-16T20:49:37.633 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8866999999999999e-05 2024-09-16T20:49:37.633 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:37.633 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:37.633 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:37.634 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.441515+0000", 2024-09-16T20:49:37.634 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.120928331 2024-09-16T20:49:37.634 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.634 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:37.634 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.634 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:37.634 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:37.634 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:37.636 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:49:37.636 INFO:teuthology.orchestra.run.smithi029.stderr:noscrub is unset 2024-09-16T20:49:37.637 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:37.634+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:37.736 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:37.734+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:37.780 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:37.780 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:49:37.780 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:49:37.780 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:37.788 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:49:37.837 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:37.836+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:37.872 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T20:49:37.935 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:37.935 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:49:37.935 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:49:37.935 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:49:37.936 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:37.937 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:37.936+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:37.944 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:49:38.018 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:49:38.018 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:49:38.038 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:38.037+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:38.045 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:49:38.046 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:49:38.068 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:49:38.068 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:49:38.093 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:49:38.093 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:49:38.110 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:38.110 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:38.118 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:49:38.118 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:49:38.119 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:49:38.139 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:38.137+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:38.142 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:49:38.142 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:49:38.222 INFO:teuthology.orchestra.run.smithi139.stdout:3958 read 0x92a6600 thru 0x92affea (0x99eb bytes) 2024-09-16T20:49:38.223 INFO:teuthology.orchestra.run.smithi139.stdout:3959 read 0x765e000 thru 0x7661fe0 (0x3fe1 bytes) 2024-09-16T20:49:38.224 INFO:teuthology.orchestra.run.smithi139.stdout:3960 punch from 0x47c5c00 to 0x47c7600, (0x1a00 bytes) 2024-09-16T20:49:38.240 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:38.238+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:38.241 INFO:teuthology.orchestra.run.smithi139.stdout:3964 trunc from 0xcd32200 to 0x9de9e00 2024-09-16T20:49:38.258 INFO:teuthology.orchestra.run.smithi139.stdout:3965 write 0x5274a00 thru 0x5277712 (0x2d13 bytes) 2024-09-16T20:49:38.263 INFO:teuthology.orchestra.run.smithi139.stdout:3967 trunc from 0x9de9e00 to 0x735fe00 2024-09-16T20:49:38.276 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:38.276 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:38.287 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:49:38.304 INFO:teuthology.orchestra.run.smithi139.stdout:3968 write 0x2ed7e00 thru 0x2ee11f7 (0x93f8 bytes) 2024-09-16T20:49:38.306 INFO:teuthology.orchestra.run.smithi139.stdout:3969 read 0x6535c00 thru 0x653912f (0x3530 bytes) 2024-09-16T20:49:38.306 INFO:teuthology.orchestra.run.smithi139.stdout:3970 write 0x241da00 thru 0x24230e3 (0x56e4 bytes) 2024-09-16T20:49:38.307 INFO:teuthology.orchestra.run.smithi139.stdout:3971 read 0x6318800 thru 0x6326efd (0xe6fe bytes) 2024-09-16T20:49:38.315 INFO:teuthology.orchestra.run.smithi139.stdout:3972 trunc from 0x735fe00 to 0xe8f4800 2024-09-16T20:49:38.371 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:38.370+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:38.381 INFO:teuthology.orchestra.run.smithi139.stdout:3973 trunc from 0xe8f4800 to 0x4a4e000 2024-09-16T20:49:38.426 INFO:teuthology.orchestra.run.smithi139.stdout:3975 read 0x40d3200 thru 0x40df092 (0xbe93 bytes) 2024-09-16T20:49:38.428 INFO:teuthology.orchestra.run.smithi139.stdout:3976 read 0x11b6c00 thru 0x11c5e6a (0xf26b bytes) 2024-09-16T20:49:38.432 INFO:teuthology.orchestra.run.smithi139.stdout:3977 write 0x3ef4400 thru 0x3efe221 (0x9e22 bytes) 2024-09-16T20:49:38.435 INFO:teuthology.orchestra.run.smithi139.stdout:3978 trunc from 0x4a4e000 to 0x28f2000 2024-09-16T20:49:38.442 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:38.441+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:38.453 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:38.454 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:38.465 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:49:38.519 INFO:teuthology.orchestra.run.smithi139.stdout:3979 write 0x9462600 thru 0x9469170 (0x6b71 bytes) 2024-09-16T20:49:38.520 INFO:teuthology.orchestra.run.smithi139.stdout:3980 read 0x8fe0800 thru 0x8fea7fe (0x9fff bytes) 2024-09-16T20:49:38.520 INFO:teuthology.orchestra.run.smithi139.stdout:3982 read 0x35d0e00 thru 0x35d9e14 (0x9015 bytes) 2024-09-16T20:49:38.520 INFO:teuthology.orchestra.run.smithi139.stdout:3983 read 0x7b6400 thru 0x7c1900 (0xb501 bytes) 2024-09-16T20:49:38.522 INFO:teuthology.orchestra.run.smithi139.stdout:3985 read 0x65fd600 thru 0x660cb62 (0xf563 bytes) 2024-09-16T20:49:38.532 INFO:teuthology.orchestra.run.smithi139.stdout:3986 trunc from 0x9469200 to 0xf13c00 2024-09-16T20:49:38.551 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:38.549+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:38.579 INFO:teuthology.orchestra.run.smithi139.stdout:3987 trunc from 0xf13c00 to 0x438b000 2024-09-16T20:49:38.613 INFO:teuthology.orchestra.run.smithi139.stdout:3989 write 0x606fa00 thru 0x6071e22 (0x2423 bytes) 2024-09-16T20:49:38.613 INFO:teuthology.orchestra.run.smithi139.stdout:3990 punch from 0x4834400 to 0x4837600, (0x3200 bytes) 2024-09-16T20:49:38.619 INFO:teuthology.orchestra.run.smithi139.stdout:3991 trunc from 0x6072000 to 0x15c1a00 2024-09-16T20:49:38.640 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:38.640 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:38.643 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:38.642+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:38.650 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:49:38.660 INFO:teuthology.orchestra.run.smithi139.stdout:3992 write 0x60e3a00 thru 0x60ef912 (0xbf13 bytes) 2024-09-16T20:49:38.662 INFO:teuthology.orchestra.run.smithi139.stdout:3993 write 0x54aa400 thru 0x54b7799 (0xd39a bytes) 2024-09-16T20:49:38.682 INFO:teuthology.orchestra.run.smithi139.stdout:3994 write 0x831f400 thru 0x832c86b (0xd46c bytes) 2024-09-16T20:49:38.687 INFO:teuthology.orchestra.run.smithi139.stdout:3995 read 0x640c400 thru 0x641105b (0x4c5c bytes) 2024-09-16T20:49:38.688 INFO:teuthology.orchestra.run.smithi139.stdout:3996 clone 30 order 20 su 32768 sc 5 2024-09-16T20:49:38.744 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:38.743+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:38.799 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:38.800 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:38.808 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:49:38.850 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:38.848+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:38.874 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 3, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-16T20:49:38.875 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T20:49:38.875 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T20:49:38.946 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:38.945+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:38.955 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:38.956 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:38.964 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:49:39.047 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:39.046+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:39.072 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:39.072 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:39.080 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:49:39.148 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:39.147+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:39.175 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:49:39.175 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-09-16T20:49:39.187 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":154,"stamp":"2024-09-16T20:49:38.221980+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1180192983,"num_objects":1220,"num_object_clones":355,"num_object_copies":2440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1220,"num_whiteouts":349,"num_read":208869,"num_read_kb":855387,"num_write":13283,"num_write_kb":237419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1535,"num_bytes_recovered":60983461,"num_keys_recovered":388,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11089,"ondisk_log_size":11089,"up":82,"acting":81,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":546784,"kb_used_data":315212,"kb_used_omap":9,"kb_used_meta":231414,"kb_avail":565684256,"statfs":{"total":579820584960,"available":579260678144,"internally_reserved":0,"allocated":322777088,"data_stored":344916179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9567,"internal_metadata":236968609},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":33951457,"num_objects":185,"num_object_clones":45,"num_object_copies":370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":185,"num_whiteouts":98,"num_read":13255,"num_read_kb":27358,"num_write":728,"num_write_kb":2911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":1251871,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000690"},"pg_stats":[{"pgid":"3.1f","version":"102'210","reported_seq":5040,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T20:49:36.189976+0000","last_change":"2024-09-16T20:49:12.802355+0000","last_active":"2024-09-16T20:49:36.189976+0000","last_peered":"2024-09-16T20:49:36.189976+0000","last_clean":"2024-09-16T20:49:36.189976+0000","last_became_active":"2024-09-16T20:49:12.801566+0000","last_became_peered":"2024-09-16T20:49:12.801566+0000","last_unstale":"2024-09-16T20:49:36.189976+0000","last_undegraded":"2024-09-16T20:49:36.189976+0000","last_fullsized":"2024-09-16T20:49:36.189976+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:28:47.236440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071634499999999996,"stat_sum":{"num_bytes":30388385,"num_objects":34,"num_object_clones":7,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":8,"num_read":4819,"num_read_kb":15272,"num_write":193,"num_write_kb":2299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":2125998,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"99'512","reported_seq":14614,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T20:49:35.929050+0000","last_change":"2024-09-16T20:48:53.732932+0000","last_active":"2024-09-16T20:49:35.929050+0000","last_peered":"2024-09-16T20:49:35.929050+0000","last_clean":"2024-09-16T20:49:35.929050+0000","last_became_active":"2024-09-16T20:48:53.732081+0000","last_became_peered":"2024-09-16T20:48:53.732081+0000","last_unstale":"2024-09-16T20:49:35.929050+0000","last_undegraded":"2024-09-16T20:49:35.929050+0000","last_fullsized":"2024-09-16T20:49:35.929050+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:53:53.422816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084864999999999995,"stat_sum":{"num_bytes":45455521,"num_objects":38,"num_object_clones":10,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":8,"num_read":15748,"num_read_kb":47241,"num_write":634,"num_write_kb":5182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":1415412,"num_keys_recovered":59,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"102'440","reported_seq":8363,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T20:49:35.979737+0000","last_change":"2024-09-16T20:48:09.564507+0000","last_active":"2024-09-16T20:49:35.979737+0000","last_peered":"2024-09-16T20:49:35.979737+0000","last_clean":"2024-09-16T20:49:35.979737+0000","last_became_active":"2024-09-16T20:48:07.568025+0000","last_became_peered":"2024-09-16T20:48:07.568025+0000","last_unstale":"2024-09-16T20:49:35.979737+0000","last_undegraded":"2024-09-16T20:49:35.979737+0000","last_fullsized":"2024-09-16T20:49:35.979737+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:25:26.677803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068985699999999999,"stat_sum":{"num_bytes":63760962,"num_objects":44,"num_object_clones":14,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":10,"num_read":9281,"num_read_kb":73119,"num_write":554,"num_write_kb":18391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"102'423","reported_seq":2181,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T20:49:36.218252+0000","last_change":"2024-09-16T20:48:09.565661+0000","last_active":"2024-09-16T20:49:36.218252+0000","last_peered":"2024-09-16T20:49:36.218252+0000","last_clean":"2024-09-16T20:49:36.218252+0000","last_became_active":"2024-09-16T20:48:07.565241+0000","last_became_peered":"2024-09-16T20:48:07.565241+0000","last_unstale":"2024-09-16T20:49:36.218252+0000","last_undegraded":"2024-09-16T20:49:36.218252+0000","last_fullsized":"2024-09-16T20:49:36.218252+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":423,"log_dups_size":0,"ondisk_log_size":423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:05:49.511697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001045589,"stat_sum":{"num_bytes":18588213,"num_objects":40,"num_object_clones":10,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":12,"num_read":3207,"num_read_kb":18423,"num_write":664,"num_write_kb":12548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"99'567","reported_seq":3902,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T20:49:35.883327+0000","last_change":"2024-09-16T20:48:54.735381+0000","last_active":"2024-09-16T20:49:35.883327+0000","last_peered":"2024-09-16T20:49:35.883327+0000","last_clean":"2024-09-16T20:49:35.883327+0000","last_became_active":"2024-09-16T20:48:54.734278+0000","last_became_peered":"2024-09-16T20:48:54.734278+0000","last_unstale":"2024-09-16T20:49:35.883327+0000","last_undegraded":"2024-09-16T20:49:35.883327+0000","last_fullsized":"2024-09-16T20:49:35.883327+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":567,"log_dups_size":0,"ondisk_log_size":567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:03:26.496944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057432399999999997,"stat_sum":{"num_bytes":22731576,"num_objects":42,"num_object_clones":12,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":12,"num_read":4995,"num_read_kb":20374,"num_write":831,"num_write_kb":10248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":1963751,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"102'738","reported_seq":10778,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T20:49:36.221889+0000","last_change":"2024-09-16T20:49:12.803894+0000","last_active":"2024-09-16T20:49:36.221889+0000","last_peered":"2024-09-16T20:49:36.221889+0000","last_clean":"2024-09-16T20:49:36.221889+0000","last_became_active":"2024-09-16T20:49:12.802869+0000","last_became_peered":"2024-09-16T20:49:12.802869+0000","last_unstale":"2024-09-16T20:49:36.221889+0000","last_undegraded":"2024-09-16T20:49:36.221889+0000","last_fullsized":"2024-09-16T20:49:36.221889+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":738,"log_dups_size":0,"ondisk_log_size":738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:08:36.615372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002037849,"stat_sum":{"num_bytes":12511684,"num_objects":49,"num_object_clones":13,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":14,"num_read":14668,"num_read_kb":34310,"num_write":937,"num_write_kb":5855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":1051004,"num_keys_recovered":46,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"99'278","reported_seq":4920,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T20:49:33.197894+0000","last_change":"2024-09-16T20:48:53.732127+0000","last_active":"2024-09-16T20:49:33.197894+0000","last_peered":"2024-09-16T20:49:33.197894+0000","last_clean":"2024-09-16T20:49:33.197894+0000","last_became_active":"2024-09-16T20:48:53.731485+0000","last_became_peered":"2024-09-16T20:48:53.731485+0000","last_unstale":"2024-09-16T20:49:33.197894+0000","last_undegraded":"2024-09-16T20:49:33.197894+0000","last_fullsized":"2024-09-16T20:49:33.197894+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":278,"log_dups_size":0,"ondisk_log_size":278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:05:53.878910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057190699999999997,"stat_sum":{"num_bytes":27539542,"num_objects":44,"num_object_clones":16,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":11,"num_read":4665,"num_read_kb":20869,"num_write":316,"num_write_kb":6095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":5091328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"99'256","reported_seq":5338,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T20:49:35.563587+0000","last_change":"2024-09-16T20:48:53.731382+0000","last_active":"2024-09-16T20:49:35.563587+0000","last_peered":"2024-09-16T20:49:35.563587+0000","last_clean":"2024-09-16T20:49:35.563587+0000","last_became_active":"2024-09-16T20:48:53.730714+0000","last_became_peered":"2024-09-16T20:48:53.730714+0000","last_unstale":"2024-09-16T20:49:35.563587+0000","last_undegraded":"2024-09-16T20:49:35.563587+0000","last_fullsized":"2024-09-16T20:49:35.563587+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":256,"log_dups_size":0,"ondisk_log_size":256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:39:25.815393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011061790000000001,"stat_sum":{"num_bytes":18242064,"num_objects":30,"num_object_clones":8,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":8,"num_read":6619,"num_read_kb":25371,"num_write":205,"num_write_kb":5449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2412544,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"102'717","reported_seq":5900,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T20:49:36.222251+0000","last_change":"2024-09-16T20:49:12.804399+0000","last_active":"2024-09-16T20:49:36.222251+0000","last_peered":"2024-09-16T20:49:36.222251+0000","last_clean":"2024-09-16T20:49:36.222251+0000","last_became_active":"2024-09-16T20:49:12.803628+0000","last_became_peered":"2024-09-16T20:49:12.803628+0000","last_unstale":"2024-09-16T20:49:36.222251+0000","last_undegraded":"2024-09-16T20:49:36.222251+0000","last_fullsized":"2024-09-16T20:49:36.222251+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":717,"log_dups_size":0,"ondisk_log_size":717,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:40:49.260501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057915500000000003,"stat_sum":{"num_bytes":11319482,"num_objects":29,"num_object_clones":6,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":8,"num_read":9573,"num_read_kb":11515,"num_write":770,"num_write_kb":3189,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":331889,"num_keys_recovered":60,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"102'636","reported_seq":6798,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T20:49:36.225660+0000","last_change":"2024-09-16T20:48:53.731777+0000","last_active":"2024-09-16T20:49:36.225660+0000","last_peered":"2024-09-16T20:49:36.225660+0000","last_clean":"2024-09-16T20:49:36.225660+0000","last_became_active":"2024-09-16T20:48:53.730899+0000","last_became_peered":"2024-09-16T20:48:53.730899+0000","last_unstale":"2024-09-16T20:49:36.225660+0000","last_undegraded":"2024-09-16T20:49:36.225660+0000","last_fullsized":"2024-09-16T20:49:36.225660+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":636,"log_dups_size":0,"ondisk_log_size":636,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:26:07.279147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010075049999999999,"stat_sum":{"num_bytes":25646193,"num_objects":43,"num_object_clones":13,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":12,"num_read":9952,"num_read_kb":32834,"num_write":731,"num_write_kb":6078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":6235436,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"99'197","reported_seq":6393,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T20:49:36.219372+0000","last_change":"2024-09-16T20:48:53.734382+0000","last_active":"2024-09-16T20:49:36.219372+0000","last_peered":"2024-09-16T20:49:36.219372+0000","last_clean":"2024-09-16T20:49:36.219372+0000","last_became_active":"2024-09-16T20:48:53.733689+0000","last_became_peered":"2024-09-16T20:48:53.733689+0000","last_unstale":"2024-09-16T20:49:36.219372+0000","last_undegraded":"2024-09-16T20:49:36.219372+0000","last_fullsized":"2024-09-16T20:49:36.219372+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:47:21.103635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051100699999999995,"stat_sum":{"num_bytes":41359360,"num_objects":45,"num_object_clones":16,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":16,"num_read":5829,"num_read_kb":19946,"num_write":186,"num_write_kb":4701,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":2749952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"102'209","reported_seq":7378,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T20:49:35.952535+0000","last_change":"2024-09-16T20:48:09.566005+0000","last_active":"2024-09-16T20:49:35.952535+0000","last_peered":"2024-09-16T20:49:35.952535+0000","last_clean":"2024-09-16T20:49:35.952535+0000","last_became_active":"2024-09-16T20:48:07.568236+0000","last_became_peered":"2024-09-16T20:48:07.568236+0000","last_unstale":"2024-09-16T20:49:35.952535+0000","last_undegraded":"2024-09-16T20:49:35.952535+0000","last_fullsized":"2024-09-16T20:49:35.952535+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:48:04.932094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001271749,"stat_sum":{"num_bytes":58703074,"num_objects":41,"num_object_clones":15,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":12,"num_read":7343,"num_read_kb":36778,"num_write":354,"num_write_kb":10138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":182,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T20:49:35.559136+0000","last_change":"2024-09-16T20:48:00.277670+0000","last_active":"2024-09-16T20:49:35.559136+0000","last_peered":"2024-09-16T20:49:35.559136+0000","last_clean":"2024-09-16T20:49:35.559136+0000","last_became_active":"2024-09-16T20:47:58.274489+0000","last_became_peered":"2024-09-16T20:47:58.274489+0000","last_unstale":"2024-09-16T20:49:35.559136+0000","last_undegraded":"2024-09-16T20:49:35.559136+0000","last_fullsized":"2024-09-16T20:49:35.559136+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:27:12.633064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091339500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"99'575","reported_seq":2680,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T20:49:33.138151+0000","last_change":"2024-09-16T20:48:50.757942+0000","last_active":"2024-09-16T20:49:33.138151+0000","last_peered":"2024-09-16T20:49:33.138151+0000","last_clean":"2024-09-16T20:49:33.138151+0000","last_became_active":"2024-09-16T20:48:50.719161+0000","last_became_peered":"2024-09-16T20:48:50.719161+0000","last_unstale":"2024-09-16T20:49:33.138151+0000","last_undegraded":"2024-09-16T20:49:33.138151+0000","last_fullsized":"2024-09-16T20:49:33.138151+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:26:43.345988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067924999999999995,"stat_sum":{"num_bytes":15788107,"num_objects":26,"num_object_clones":6,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":5,"num_read":4247,"num_read_kb":19214,"num_write":896,"num_write_kb":11196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":466506,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":175,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T20:49:31.859223+0000","last_change":"2024-09-16T20:48:00.277182+0000","last_active":"2024-09-16T20:49:31.859223+0000","last_peered":"2024-09-16T20:49:31.859223+0000","last_clean":"2024-09-16T20:49:31.859223+0000","last_became_active":"2024-09-16T20:47:58.274389+0000","last_became_peered":"2024-09-16T20:47:58.274389+0000","last_unstale":"2024-09-16T20:49:31.859223+0000","last_undegraded":"2024-09-16T20:49:31.859223+0000","last_fullsized":"2024-09-16T20:49:31.859223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:28:02.441830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043659299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"99'217","reported_seq":8876,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T20:49:33.225735+0000","last_change":"2024-09-16T20:48:50.791999+0000","last_active":"2024-09-16T20:49:33.225735+0000","last_peered":"2024-09-16T20:49:33.225735+0000","last_clean":"2024-09-16T20:49:33.225735+0000","last_became_active":"2024-09-16T20:48:50.719868+0000","last_became_peered":"2024-09-16T20:48:50.719868+0000","last_unstale":"2024-09-16T20:49:33.225735+0000","last_undegraded":"2024-09-16T20:49:33.225735+0000","last_fullsized":"2024-09-16T20:49:33.225735+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":217,"log_dups_size":0,"ondisk_log_size":217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:45:17.129854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012934750000000001,"stat_sum":{"num_bytes":59310592,"num_objects":28,"num_object_clones":11,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":8,"num_read":8577,"num_read_kb":30053,"num_write":198,"num_write_kb":11788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":523776,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":182,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T20:49:35.559153+0000","last_change":"2024-09-16T20:48:00.278054+0000","last_active":"2024-09-16T20:49:35.559153+0000","last_peered":"2024-09-16T20:49:35.559153+0000","last_clean":"2024-09-16T20:49:35.559153+0000","last_became_active":"2024-09-16T20:47:58.272861+0000","last_became_peered":"2024-09-16T20:47:58.272861+0000","last_unstale":"2024-09-16T20:49:35.559153+0000","last_undegraded":"2024-09-16T20:49:35.559153+0000","last_fullsized":"2024-09-16T20:49:35.559153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:13:59.135547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012429489999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"99'286","reported_seq":2527,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T20:49:32.536928+0000","last_change":"2024-09-16T20:49:12.802627+0000","last_active":"2024-09-16T20:49:32.536928+0000","last_peered":"2024-09-16T20:49:32.536928+0000","last_clean":"2024-09-16T20:49:32.536928+0000","last_became_active":"2024-09-16T20:49:12.801810+0000","last_became_peered":"2024-09-16T20:49:12.801810+0000","last_unstale":"2024-09-16T20:49:32.536928+0000","last_undegraded":"2024-09-16T20:49:32.536928+0000","last_fullsized":"2024-09-16T20:49:32.536928+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:57:15.190062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076367000000000004,"stat_sum":{"num_bytes":28558373,"num_objects":32,"num_object_clones":11,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":9,"num_read":2341,"num_read_kb":8446,"num_write":363,"num_write_kb":4838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":2071114,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":57,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T20:49:35.558688+0000","last_change":"2024-09-16T20:49:09.557800+0000","last_active":"2024-09-16T20:49:35.558688+0000","last_peered":"2024-09-16T20:49:35.558688+0000","last_clean":"2024-09-16T20:49:35.558688+0000","last_became_active":"2024-09-16T20:49:09.557513+0000","last_became_peered":"2024-09-16T20:49:09.557513+0000","last_unstale":"2024-09-16T20:49:35.558688+0000","last_undegraded":"2024-09-16T20:49:35.558688+0000","last_fullsized":"2024-09-16T20:49:35.558688+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:38:59.143723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"102'192","reported_seq":7867,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T20:49:36.211133+0000","last_change":"2024-09-16T20:48:53.732314+0000","last_active":"2024-09-16T20:49:36.211133+0000","last_peered":"2024-09-16T20:49:36.211133+0000","last_clean":"2024-09-16T20:49:36.211133+0000","last_became_active":"2024-09-16T20:48:53.731287+0000","last_became_peered":"2024-09-16T20:48:53.731287+0000","last_unstale":"2024-09-16T20:49:36.211133+0000","last_undegraded":"2024-09-16T20:49:36.211133+0000","last_fullsized":"2024-09-16T20:49:36.211133+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":192,"log_dups_size":0,"ondisk_log_size":192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:22:20.661820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010471600000000001,"stat_sum":{"num_bytes":61696085,"num_objects":37,"num_object_clones":8,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":9,"num_read":6738,"num_read_kb":33698,"num_write":183,"num_write_kb":9685,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":176266,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"99'282","reported_seq":4736,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T20:49:35.825016+0000","last_change":"2024-09-16T20:48:09.570254+0000","last_active":"2024-09-16T20:49:35.825016+0000","last_peered":"2024-09-16T20:49:35.825016+0000","last_clean":"2024-09-16T20:49:35.825016+0000","last_became_active":"2024-09-16T20:48:07.568143+0000","last_became_peered":"2024-09-16T20:48:07.568143+0000","last_unstale":"2024-09-16T20:49:35.825016+0000","last_undegraded":"2024-09-16T20:49:35.825016+0000","last_fullsized":"2024-09-16T20:49:35.825016+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:56:12.039884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038800899999999998,"stat_sum":{"num_bytes":46913651,"num_objects":49,"num_object_clones":17,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":20,"num_read":4673,"num_read_kb":11644,"num_write":306,"num_write_kb":2488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":184,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T20:49:35.829269+0000","last_change":"2024-09-16T20:48:00.276970+0000","last_active":"2024-09-16T20:49:35.829269+0000","last_peered":"2024-09-16T20:49:35.829269+0000","last_clean":"2024-09-16T20:49:35.829269+0000","last_became_active":"2024-09-16T20:47:58.274617+0000","last_became_peered":"2024-09-16T20:47:58.274617+0000","last_unstale":"2024-09-16T20:49:35.829269+0000","last_undegraded":"2024-09-16T20:49:35.829269+0000","last_fullsized":"2024-09-16T20:49:35.829269+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:28:54.467905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062077499999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"99'281","reported_seq":5819,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T20:49:36.086382+0000","last_change":"2024-09-16T20:49:12.801644+0000","last_active":"2024-09-16T20:49:36.086382+0000","last_peered":"2024-09-16T20:49:36.086382+0000","last_clean":"2024-09-16T20:49:36.086382+0000","last_became_active":"2024-09-16T20:49:12.800827+0000","last_became_peered":"2024-09-16T20:49:12.800827+0000","last_unstale":"2024-09-16T20:49:36.086382+0000","last_undegraded":"2024-09-16T20:49:36.086382+0000","last_fullsized":"2024-09-16T20:49:36.086382+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":281,"log_dups_size":0,"ondisk_log_size":281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:16:02.846537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088331900000000003,"stat_sum":{"num_bytes":38779623,"num_objects":32,"num_object_clones":5,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":7,"num_read":7301,"num_read_kb":27521,"num_write":226,"num_write_kb":4204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":1931020,"num_keys_recovered":36,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":97,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T20:49:35.563684+0000","last_change":"2024-09-16T20:49:09.559342+0000","last_active":"2024-09-16T20:49:35.563684+0000","last_peered":"2024-09-16T20:49:35.563684+0000","last_clean":"2024-09-16T20:49:35.563684+0000","last_became_active":"2024-09-16T20:49:09.558884+0000","last_became_peered":"2024-09-16T20:49:09.558884+0000","last_unstale":"2024-09-16T20:49:35.563684+0000","last_undegraded":"2024-09-16T20:49:35.563684+0000","last_fullsized":"2024-09-16T20:49:35.563684+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:51:23.264062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"99'182","reported_seq":4409,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T20:49:33.106981+0000","last_change":"2024-09-16T20:48:09.565596+0000","last_active":"2024-09-16T20:49:33.106981+0000","last_peered":"2024-09-16T20:49:33.106981+0000","last_clean":"2024-09-16T20:49:33.106981+0000","last_became_active":"2024-09-16T20:48:07.566097+0000","last_became_peered":"2024-09-16T20:48:07.566097+0000","last_unstale":"2024-09-16T20:49:33.106981+0000","last_undegraded":"2024-09-16T20:49:33.106981+0000","last_fullsized":"2024-09-16T20:49:33.106981+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:44:18.983095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012840460000000001,"stat_sum":{"num_bytes":51222016,"num_objects":37,"num_object_clones":11,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":12,"num_read":4105,"num_read_kb":27829,"num_write":181,"num_write_kb":6876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":194,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T20:49:35.558765+0000","last_change":"2024-09-16T20:49:09.560350+0000","last_active":"2024-09-16T20:49:35.558765+0000","last_peered":"2024-09-16T20:49:35.558765+0000","last_clean":"2024-09-16T20:49:35.558765+0000","last_became_active":"2024-09-16T20:49:09.559789+0000","last_became_peered":"2024-09-16T20:49:09.559789+0000","last_unstale":"2024-09-16T20:49:35.558765+0000","last_undegraded":"2024-09-16T20:49:35.558765+0000","last_fullsized":"2024-09-16T20:49:35.558765+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:17:54.706074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037636000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"102'312","reported_seq":4265,"reported_epoch":102,"state":"active+undersized+remapped","last_fresh":"2024-09-16T20:49:36.193681+0000","last_change":"2024-09-16T20:49:35.959510+0000","last_active":"2024-09-16T20:49:36.193681+0000","last_peered":"2024-09-16T20:49:36.193681+0000","last_clean":"2024-09-16T20:49:32.052802+0000","last_became_active":"2024-09-16T20:49:35.829992+0000","last_became_peered":"2024-09-16T20:49:35.829992+0000","last_unstale":"2024-09-16T20:49:36.193681+0000","last_undegraded":"2024-09-16T20:49:36.193681+0000","last_fullsized":"2024-09-16T20:49:35.558976+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063857599999999997,"stat_sum":{"num_bytes":23778847,"num_objects":45,"num_object_clones":14,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":16,"num_read":6203,"num_read_kb":16341,"num_write":234,"num_write_kb":2559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":127,"num_bytes_recovered":2183758,"num_keys_recovered":57,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[2],"avail_no_missing":["2","0","3","5"],"object_location_counts":[{"shards":"2","objects":45}],"blocked_by":[],"up_primary":5,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":183,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T20:49:35.559343+0000","last_change":"2024-09-16T20:48:00.277739+0000","last_active":"2024-09-16T20:49:35.559343+0000","last_peered":"2024-09-16T20:49:35.559343+0000","last_clean":"2024-09-16T20:49:35.559343+0000","last_became_active":"2024-09-16T20:47:58.271174+0000","last_became_peered":"2024-09-16T20:47:58.271174+0000","last_unstale":"2024-09-16T20:49:35.559343+0000","last_undegraded":"2024-09-16T20:49:35.559343+0000","last_fullsized":"2024-09-16T20:49:35.559343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:11:35.985075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095771799999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":297,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T20:49:35.559212+0000","last_change":"2024-09-16T20:49:09.565188+0000","last_active":"2024-09-16T20:49:35.559212+0000","last_peered":"2024-09-16T20:49:35.559212+0000","last_clean":"2024-09-16T20:49:35.559212+0000","last_became_active":"2024-09-16T20:49:09.559571+0000","last_became_peered":"2024-09-16T20:49:09.559571+0000","last_unstale":"2024-09-16T20:49:35.559212+0000","last_undegraded":"2024-09-16T20:49:35.559212+0000","last_fullsized":"2024-09-16T20:49:35.559212+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:52.241408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:52.241408+0000","last_clean_scrub_stamp":"2024-09-16T20:47:52.241408+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:01:53.651981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"99'453","reported_seq":2743,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T20:49:32.846060+0000","last_change":"2024-09-16T20:48:09.565136+0000","last_active":"2024-09-16T20:49:32.846060+0000","last_peered":"2024-09-16T20:49:32.846060+0000","last_clean":"2024-09-16T20:49:32.846060+0000","last_became_active":"2024-09-16T20:48:07.566139+0000","last_became_peered":"2024-09-16T20:48:07.566139+0000","last_unstale":"2024-09-16T20:49:32.846060+0000","last_undegraded":"2024-09-16T20:49:32.846060+0000","last_fullsized":"2024-09-16T20:49:32.846060+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":453,"log_dups_size":0,"ondisk_log_size":453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:19:44.604387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001025172,"stat_sum":{"num_bytes":20372570,"num_objects":35,"num_object_clones":8,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":10,"num_read":3681,"num_read_kb":11066,"num_write":682,"num_write_kb":4686,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"102'240","reported_seq":5030,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T20:49:36.133951+0000","last_change":"2024-09-16T20:48:53.737981+0000","last_active":"2024-09-16T20:49:36.133951+0000","last_peered":"2024-09-16T20:49:36.133951+0000","last_clean":"2024-09-16T20:49:36.133951+0000","last_became_active":"2024-09-16T20:48:53.736558+0000","last_became_peered":"2024-09-16T20:48:53.736558+0000","last_unstale":"2024-09-16T20:49:36.133951+0000","last_undegraded":"2024-09-16T20:49:36.133951+0000","last_fullsized":"2024-09-16T20:49:36.133951+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":240,"log_dups_size":0,"ondisk_log_size":240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:25:08.985916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044614999999999998,"stat_sum":{"num_bytes":45768736,"num_objects":41,"num_object_clones":15,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":13,"num_read":5392,"num_read_kb":32660,"num_write":248,"num_write_kb":10068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":2635264,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"102'206","reported_seq":4545,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T20:49:35.853625+0000","last_change":"2024-09-16T20:48:09.569992+0000","last_active":"2024-09-16T20:49:35.853625+0000","last_peered":"2024-09-16T20:49:35.853625+0000","last_clean":"2024-09-16T20:49:35.853625+0000","last_became_active":"2024-09-16T20:48:07.563972+0000","last_became_peered":"2024-09-16T20:48:07.563972+0000","last_unstale":"2024-09-16T20:49:35.853625+0000","last_undegraded":"2024-09-16T20:49:35.853625+0000","last_fullsized":"2024-09-16T20:49:35.853625+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:14:14.931146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039392600000000001,"stat_sum":{"num_bytes":38660397,"num_objects":42,"num_object_clones":13,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":11,"num_read":4276,"num_read_kb":26651,"num_write":349,"num_write_kb":6401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"102'242","reported_seq":3818,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T20:49:35.992336+0000","last_change":"2024-09-16T20:48:54.735384+0000","last_active":"2024-09-16T20:49:35.992336+0000","last_peered":"2024-09-16T20:49:35.992336+0000","last_clean":"2024-09-16T20:49:35.992336+0000","last_became_active":"2024-09-16T20:48:54.734409+0000","last_became_peered":"2024-09-16T20:48:54.734409+0000","last_unstale":"2024-09-16T20:49:35.992336+0000","last_undegraded":"2024-09-16T20:49:35.992336+0000","last_fullsized":"2024-09-16T20:49:35.992336+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":242,"log_dups_size":0,"ondisk_log_size":242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:30:19.007269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014289540000000001,"stat_sum":{"num_bytes":38032969,"num_objects":31,"num_object_clones":7,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":12,"num_read":3623,"num_read_kb":10413,"num_write":266,"num_write_kb":4656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":971906,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"102'332","reported_seq":11184,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T20:49:35.991789+0000","last_change":"2024-09-16T20:49:30.561716+0000","last_active":"2024-09-16T20:49:35.991789+0000","last_peered":"2024-09-16T20:49:35.991789+0000","last_clean":"2024-09-16T20:49:35.991789+0000","last_became_active":"2024-09-16T20:49:30.560372+0000","last_became_peered":"2024-09-16T20:49:30.560372+0000","last_unstale":"2024-09-16T20:49:35.991789+0000","last_undegraded":"2024-09-16T20:49:35.991789+0000","last_fullsized":"2024-09-16T20:49:35.991789+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:39:30.892614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047762999999999999,"stat_sum":{"num_bytes":62739867,"num_objects":29,"num_object_clones":7,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":10,"num_read":11686,"num_read_kb":46824,"num_write":482,"num_write_kb":10168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":163,"num_bytes_recovered":18718820,"num_keys_recovered":48,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"102'396","reported_seq":6524,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T20:49:35.881190+0000","last_change":"2024-09-16T20:48:09.565667+0000","last_active":"2024-09-16T20:49:35.881190+0000","last_peered":"2024-09-16T20:49:35.881190+0000","last_clean":"2024-09-16T20:49:35.881190+0000","last_became_active":"2024-09-16T20:48:07.567327+0000","last_became_peered":"2024-09-16T20:48:07.567327+0000","last_unstale":"2024-09-16T20:49:35.881190+0000","last_undegraded":"2024-09-16T20:49:35.881190+0000","last_fullsized":"2024-09-16T20:49:35.881190+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":396,"log_dups_size":0,"ondisk_log_size":396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:40:23.997217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012087560000000001,"stat_sum":{"num_bytes":28744762,"num_objects":39,"num_object_clones":12,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":11,"num_read":6754,"num_read_kb":20517,"num_write":395,"num_write_kb":8689,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"102'179","reported_seq":4816,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T20:49:36.087678+0000","last_change":"2024-09-16T20:48:09.565024+0000","last_active":"2024-09-16T20:49:36.087678+0000","last_peered":"2024-09-16T20:49:36.087678+0000","last_clean":"2024-09-16T20:49:36.087678+0000","last_became_active":"2024-09-16T20:48:07.570704+0000","last_became_peered":"2024-09-16T20:48:07.570704+0000","last_unstale":"2024-09-16T20:49:36.087678+0000","last_undegraded":"2024-09-16T20:49:36.087678+0000","last_fullsized":"2024-09-16T20:49:36.087678+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:39:14.632814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011026149999999999,"stat_sum":{"num_bytes":58874209,"num_objects":39,"num_object_clones":13,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":11,"num_read":4542,"num_read_kb":28527,"num_write":171,"num_write_kb":4178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"102'255","reported_seq":5842,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T20:49:36.215295+0000","last_change":"2024-09-16T20:48:09.570786+0000","last_active":"2024-09-16T20:49:36.215295+0000","last_peered":"2024-09-16T20:49:36.215295+0000","last_clean":"2024-09-16T20:49:36.215295+0000","last_became_active":"2024-09-16T20:48:07.560783+0000","last_became_peered":"2024-09-16T20:48:07.560783+0000","last_unstale":"2024-09-16T20:49:36.215295+0000","last_undegraded":"2024-09-16T20:49:36.215295+0000","last_fullsized":"2024-09-16T20:49:36.215295+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":255,"log_dups_size":0,"ondisk_log_size":255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:15:31.818078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040623999999999999,"stat_sum":{"num_bytes":26040586,"num_objects":50,"num_object_clones":18,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":14,"num_read":6439,"num_read_kb":30679,"num_write":229,"num_write_kb":9902,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"102'361","reported_seq":6039,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T20:49:36.216538+0000","last_change":"2024-09-16T20:49:12.801692+0000","last_active":"2024-09-16T20:49:36.216538+0000","last_peered":"2024-09-16T20:49:36.216538+0000","last_clean":"2024-09-16T20:49:36.216538+0000","last_became_active":"2024-09-16T20:49:12.801013+0000","last_became_peered":"2024-09-16T20:49:12.801013+0000","last_unstale":"2024-09-16T20:49:36.216538+0000","last_undegraded":"2024-09-16T20:49:36.216538+0000","last_fullsized":"2024-09-16T20:49:36.216538+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":361,"log_dups_size":0,"ondisk_log_size":361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:45:16.309372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115576,"stat_sum":{"num_bytes":56909387,"num_objects":45,"num_object_clones":15,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":16,"num_read":7154,"num_read_kb":32807,"num_write":284,"num_write_kb":4752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":99,"num_bytes_recovered":5210742,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"99'225","reported_seq":6730,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T20:49:35.881527+0000","last_change":"2024-09-16T20:48:09.564513+0000","last_active":"2024-09-16T20:49:35.881527+0000","last_peered":"2024-09-16T20:49:35.881527+0000","last_clean":"2024-09-16T20:49:35.881527+0000","last_became_active":"2024-09-16T20:48:07.571985+0000","last_became_peered":"2024-09-16T20:48:07.571985+0000","last_unstale":"2024-09-16T20:49:35.881527+0000","last_undegraded":"2024-09-16T20:49:35.881527+0000","last_fullsized":"2024-09-16T20:49:35.881527+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":225,"log_dups_size":0,"ondisk_log_size":225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:31:12.768451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077335299999999995,"stat_sum":{"num_bytes":52686366,"num_objects":38,"num_object_clones":9,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":5,"num_read":6630,"num_read_kb":56093,"num_write":284,"num_write_kb":18928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"102'306","reported_seq":2236,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T20:49:35.880566+0000","last_change":"2024-09-16T20:49:12.804699+0000","last_active":"2024-09-16T20:49:35.880566+0000","last_peered":"2024-09-16T20:49:35.880566+0000","last_clean":"2024-09-16T20:49:35.880566+0000","last_became_active":"2024-09-16T20:49:12.803689+0000","last_became_peered":"2024-09-16T20:49:12.803689+0000","last_unstale":"2024-09-16T20:49:35.880566+0000","last_undegraded":"2024-09-16T20:49:35.880566+0000","last_fullsized":"2024-09-16T20:49:35.880566+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":306,"log_dups_size":0,"ondisk_log_size":306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:30:27.585314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000840955,"stat_sum":{"num_bytes":19391055,"num_objects":31,"num_object_clones":8,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":9,"num_read":2348,"num_read_kb":8230,"num_write":391,"num_write_kb":2030,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":107,"num_bytes_recovered":1536239,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"102'309","reported_seq":5138,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T20:49:35.879727+0000","last_change":"2024-09-16T20:48:09.565356+0000","last_active":"2024-09-16T20:49:35.879727+0000","last_peered":"2024-09-16T20:49:35.879727+0000","last_clean":"2024-09-16T20:49:35.879727+0000","last_became_active":"2024-09-16T20:48:07.565717+0000","last_became_peered":"2024-09-16T20:48:07.565717+0000","last_unstale":"2024-09-16T20:49:35.879727+0000","last_undegraded":"2024-09-16T20:49:35.879727+0000","last_fullsized":"2024-09-16T20:49:35.879727+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":309,"log_dups_size":0,"ondisk_log_size":309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:12:38.689303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013583239999999999,"stat_sum":{"num_bytes":29088342,"num_objects":31,"num_object_clones":7,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":10,"num_read":5394,"num_read_kb":20067,"num_write":437,"num_write_kb":7524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1179602596,"num_objects":1216,"num_object_clones":355,"num_object_copies":2432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1216,"num_whiteouts":349,"num_read":208803,"num_read_kb":855332,"num_write":13180,"num_write_kb":235789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1531,"num_bytes_recovered":59802725,"num_keys_recovered":388,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":317104128,"data_stored":343432207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11014,"ondisk_log_size":11014,"up":64,"acting":63,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45308,"kb_used_data":14112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94326532,"statfs":{"total":96636764160,"available":96590368768,"internally_reserved":0,"allocated":14450688,"data_stored":14225687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":97424,"kb_used_data":62068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94274416,"statfs":{"total":96636764160,"available":96537001984,"internally_reserved":0,"allocated":63557632,"data_stored":66748800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":101712,"kb_used_data":70516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94270128,"statfs":{"total":96636764160,"available":96532611072,"internally_reserved":0,"allocated":72208384,"data_stored":77616106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":112196,"kb_used_data":62964,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94259644,"statfs":{"total":96636764160,"available":96521875456,"internally_reserved":0,"allocated":64475136,"data_stored":69455577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123172,"kb_used_data":69772,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94248668,"statfs":{"total":96636764160,"available":96510636032,"internally_reserved":0,"allocated":71446528,"data_stored":78830105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640280,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66972,"kb_used_data":35780,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94304868,"statfs":{"total":96636764160,"available":96568184832,"internally_reserved":0,"allocated":36638720,"data_stored":38039904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":71548928,"data_stored":79616374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":62689280,"data_stored":68382892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":71016448,"data_stored":77135325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":62369792,"data_stored":66266620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":35528704,"data_stored":37615920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":13950976,"data_stored":14415076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T20:49:39.190 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:39.190 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:39.199 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:49:39.283 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:39.282+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:39.346 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:39.346 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:39.350 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:39.348+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:39.355 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:49:39.451 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:39.449+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:39.502 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:39.503 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:39.511 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:49:39.552 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:39.550+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:39.638 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T20:49:39.653 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:39.651+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:39.663 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:39.663 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:39.672 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:49:39.735 INFO:teuthology.orchestra.run.smithi139.stdout:truncating image image_client.0-clone29 from 0x832ca00 (overlap 0xf13c00) to 0xd91c00 2024-09-16T20:49:39.756 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:39.755+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:39.836 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:39.837 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:39.846 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:49:39.854 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:39.853+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:39.955 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:39.954+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:39.996 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:39.996 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:40.006 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:49:40.056 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:40.054+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:40.120 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:40.121 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:40.130 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:49:40.155 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T20:49:40.156 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:40.155+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:40.188 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #28, image image_client.0-clone28 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent29 2024-09-16T20:49:40.290 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:40.290 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:40.293 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:40.256+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:40.300 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:49:40.359 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:40.357+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:40.419 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:49:40.419 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:49:40.449 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:49:40.449 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:49:40.460 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:40.458+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:40.478 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:49:40.478 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:49:40.485 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:40.485 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:49:40.485 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:49:40.486 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:49:40.486 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.486 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3712 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1472512~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:40.486 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:49:40.486 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 77.844423144999993, 2024-09-16T20:49:40.486 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050004305999999998, 2024-09-16T20:49:40.486 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:40.486 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:40.486 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:40.486 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:40.486 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:40.486 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:40.486 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3712 2024-09-16T20:49:40.486 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.486 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:40.487 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.487 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:40.487 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:49:40.487 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:40.487 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.487 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.487 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:40.487 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:49:40.487 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:40.487 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.487 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.487 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:40.487 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:49:40.487 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0199999999999998e-07 2024-09-16T20:49:40.488 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.488 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.488 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:40.488 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638990+0000", 2024-09-16T20:49:40.488 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.618e-06 2024-09-16T20:49:40.488 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.488 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.488 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:40.488 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638995+0000", 2024-09-16T20:49:40.488 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3940000000000004e-06 2024-09-16T20:49:40.488 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.488 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.488 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:40.488 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639000+0000", 2024-09-16T20:49:40.489 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9309999999999999e-06 2024-09-16T20:49:40.489 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.489 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.489 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:40.489 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688867+0000", 2024-09-16T20:49:40.489 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049866955999999997 2024-09-16T20:49:40.489 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.489 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.489 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:40.489 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688933+0000", 2024-09-16T20:49:40.490 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5359999999999998e-05 2024-09-16T20:49:40.490 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.490 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.490 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:40.490 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688992+0000", 2024-09-16T20:49:40.490 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9744999999999998e-05 2024-09-16T20:49:40.490 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:40.491 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:40.491 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:40.491 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.491 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.491 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3714 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1480704~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:40.491 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:49:40.491 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 77.844405554999994, 2024-09-16T20:49:40.491 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050152552000000003, 2024-09-16T20:49:40.491 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:40.491 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:40.491 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:40.491 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:40.491 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:40.491 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:40.492 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3714 2024-09-16T20:49:40.492 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.492 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:40.492 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.492 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:40.492 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:49:40.492 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:40.492 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.492 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.492 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:40.492 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:49:40.493 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:40.493 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.493 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.493 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:40.493 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:49:40.493 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0499999999999999e-07 2024-09-16T20:49:40.493 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.493 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.493 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:40.494 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639007+0000", 2024-09-16T20:49:40.494 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5340000000000001e-06 2024-09-16T20:49:40.494 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.494 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.494 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:40.494 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639013+0000", 2024-09-16T20:49:40.494 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3260000000000003e-06 2024-09-16T20:49:40.494 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.494 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.494 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:40.494 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639018+0000", 2024-09-16T20:49:40.495 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2850000000000002e-06 2024-09-16T20:49:40.495 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.495 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.495 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:40.495 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689026+0000", 2024-09-16T20:49:40.495 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.05000798 2024-09-16T20:49:40.495 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.495 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.495 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:40.495 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689077+0000", 2024-09-16T20:49:40.496 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1063999999999998e-05 2024-09-16T20:49:40.496 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.496 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.496 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:40.496 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689158+0000", 2024-09-16T20:49:40.496 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1057999999999995e-05 2024-09-16T20:49:40.496 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:40.497 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:40.497 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:40.497 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.497 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.497 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3716 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1488896~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:40.497 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:49:40.497 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 77.844387745000006, 2024-09-16T20:49:40.497 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050280142, 2024-09-16T20:49:40.497 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:40.497 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:40.497 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:40.497 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:40.497 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:40.498 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:40.498 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3716 2024-09-16T20:49:40.498 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.498 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:40.498 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.498 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:40.498 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:49:40.498 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:40.498 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.498 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.498 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:40.498 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:49:40.498 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:40.498 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.498 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.499 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:40.499 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639024+0000", 2024-09-16T20:49:40.499 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9999999999999997e-07 2024-09-16T20:49:40.499 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.499 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.499 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:40.499 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639026+0000", 2024-09-16T20:49:40.499 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.781e-06 2024-09-16T20:49:40.499 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.499 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.500 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:40.500 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639032+0000", 2024-09-16T20:49:40.500 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7780000000000004e-06 2024-09-16T20:49:40.500 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.500 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.500 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:40.500 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639036+0000", 2024-09-16T20:49:40.500 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6909999999999996e-06 2024-09-16T20:49:40.501 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.501 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.501 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:40.501 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689191+0000", 2024-09-16T20:49:40.501 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050155284000000001 2024-09-16T20:49:40.501 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.501 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.501 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:40.501 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689242+0000", 2024-09-16T20:49:40.501 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0662999999999999e-05 2024-09-16T20:49:40.501 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.502 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.502 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:40.502 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689304+0000", 2024-09-16T20:49:40.502 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1345000000000003e-05 2024-09-16T20:49:40.502 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:40.502 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:40.502 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:40.502 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.502 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.502 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3719 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1497088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:40.502 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:49:40.502 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 77.844360467000001, 2024-09-16T20:49:40.502 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050399715999999997, 2024-09-16T20:49:40.502 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:40.502 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:40.503 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:40.503 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:40.503 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:40.503 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:40.503 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3719 2024-09-16T20:49:40.503 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.503 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:40.503 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.503 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:40.503 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:49:40.503 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:40.503 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.503 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.503 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:40.504 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:49:40.504 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:40.504 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.504 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.504 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:40.504 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:49:40.504 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9700000000000002e-07 2024-09-16T20:49:40.504 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.504 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.504 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:40.504 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639053+0000", 2024-09-16T20:49:40.504 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2900000000000001e-06 2024-09-16T20:49:40.505 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.505 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.505 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:40.505 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639060+0000", 2024-09-16T20:49:40.505 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7750000000000004e-06 2024-09-16T20:49:40.505 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.505 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.505 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:40.505 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639066+0000", 2024-09-16T20:49:40.506 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0639999999999999e-06 2024-09-16T20:49:40.506 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.506 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.506 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:40.506 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689337+0000", 2024-09-16T20:49:40.506 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050270784999999998 2024-09-16T20:49:40.506 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.506 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.506 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:40.506 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689392+0000", 2024-09-16T20:49:40.506 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4920999999999998e-05 2024-09-16T20:49:40.506 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.506 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.506 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:40.507 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689450+0000", 2024-09-16T20:49:40.507 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8483999999999998e-05 2024-09-16T20:49:40.507 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:40.507 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:40.507 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:40.507 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.507 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.507 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3721 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1505280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:40.507 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:49:40.507 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 77.844337335000006, 2024-09-16T20:49:40.507 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050513608000000002, 2024-09-16T20:49:40.507 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:40.507 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:40.507 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:40.507 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:40.508 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:40.508 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:40.508 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3721 2024-09-16T20:49:40.508 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.508 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:40.508 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.508 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:40.508 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:49:40.508 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:40.508 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.508 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.508 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:40.508 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:49:40.509 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:40.509 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.509 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.509 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:40.509 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:49:40.509 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7e-07 2024-09-16T20:49:40.509 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.509 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.509 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:40.509 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639076+0000", 2024-09-16T20:49:40.510 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8899999999999999e-06 2024-09-16T20:49:40.510 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.510 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.510 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:40.510 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639083+0000", 2024-09-16T20:49:40.510 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1210000000000001e-06 2024-09-16T20:49:40.510 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.510 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.510 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:40.510 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639089+0000", 2024-09-16T20:49:40.510 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.643e-06 2024-09-16T20:49:40.510 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.510 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.510 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:40.510 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689474+0000", 2024-09-16T20:49:40.511 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050385679000000003 2024-09-16T20:49:40.511 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.511 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.511 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:40.511 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689533+0000", 2024-09-16T20:49:40.511 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8303000000000001e-05 2024-09-16T20:49:40.511 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.511 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.511 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:40.511 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689587+0000", 2024-09-16T20:49:40.511 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4601999999999997e-05 2024-09-16T20:49:40.511 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:40.511 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:40.511 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:40.512 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.512 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.512 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3723 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1513472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:40.512 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:49:40.512 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 77.844306531000001, 2024-09-16T20:49:40.512 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050699691999999998, 2024-09-16T20:49:40.512 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:40.512 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:40.512 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:40.512 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:40.512 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:40.512 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:40.512 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3723 2024-09-16T20:49:40.512 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.513 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:40.513 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.513 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:40.513 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:49:40.513 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:40.513 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.513 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.513 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:40.513 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:49:40.513 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:40.513 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.513 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.513 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:40.513 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:49:40.514 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3399999999999999e-07 2024-09-16T20:49:40.514 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.514 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.514 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:40.514 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639107+0000", 2024-09-16T20:49:40.514 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3860000000000001e-06 2024-09-16T20:49:40.514 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.514 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.514 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:40.514 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639114+0000", 2024-09-16T20:49:40.514 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9800000000000001e-06 2024-09-16T20:49:40.514 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.514 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.514 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:40.514 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639129+0000", 2024-09-16T20:49:40.514 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4887e-05 2024-09-16T20:49:40.515 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.515 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.515 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:40.515 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689625+0000", 2024-09-16T20:49:40.515 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050495848000000003 2024-09-16T20:49:40.515 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.515 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.515 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:40.515 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689720+0000", 2024-09-16T20:49:40.515 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4821000000000006e-05 2024-09-16T20:49:40.515 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.515 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.515 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:40.515 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689804+0000", 2024-09-16T20:49:40.516 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4235999999999998e-05 2024-09-16T20:49:40.516 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:40.516 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:40.516 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:40.516 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.516 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.516 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3725 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1521664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:40.516 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:49:40.516 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 77.844272935000006, 2024-09-16T20:49:40.516 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050797660000000001, 2024-09-16T20:49:40.516 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:40.517 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:40.517 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:40.517 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:40.517 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:40.517 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:40.517 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3725 2024-09-16T20:49:40.517 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.517 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:40.517 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.517 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:40.517 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:49:40.517 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:40.518 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.518 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.518 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:40.518 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:49:40.518 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:40.518 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.518 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.518 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:40.518 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639139+0000", 2024-09-16T20:49:40.518 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8999999999999997e-07 2024-09-16T20:49:40.518 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.518 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.518 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:40.518 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639141+0000", 2024-09-16T20:49:40.519 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.182e-06 2024-09-16T20:49:40.519 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.519 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.519 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:40.519 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639154+0000", 2024-09-16T20:49:40.519 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3626e-05 2024-09-16T20:49:40.519 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.519 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.519 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:40.519 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639160+0000", 2024-09-16T20:49:40.519 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6810000000000003e-06 2024-09-16T20:49:40.519 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.519 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.519 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:40.519 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689831+0000", 2024-09-16T20:49:40.520 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050670907000000001 2024-09-16T20:49:40.520 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.520 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.520 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:40.520 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689882+0000", 2024-09-16T20:49:40.520 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0445999999999998e-05 2024-09-16T20:49:40.520 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.520 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.520 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:40.520 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689936+0000", 2024-09-16T20:49:40.520 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4327999999999999e-05 2024-09-16T20:49:40.520 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:40.520 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:40.520 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:40.521 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.521 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.521 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3727 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1529856~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:40.521 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:49:40.521 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 77.844237358000001, 2024-09-16T20:49:40.521 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051016922999999999, 2024-09-16T20:49:40.521 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:40.521 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:40.521 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:40.521 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:40.521 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:40.522 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:40.522 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3727 2024-09-16T20:49:40.522 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.522 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:40.522 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.522 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:40.522 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:49:40.522 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:40.522 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.522 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.522 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:40.522 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:49:40.522 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:40.522 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.522 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.523 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:40.523 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:49:40.523 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9999999999999998e-07 2024-09-16T20:49:40.523 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.523 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.523 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:40.523 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639183+0000", 2024-09-16T20:49:40.523 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2399999999999996e-06 2024-09-16T20:49:40.523 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.523 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.523 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:40.523 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639195+0000", 2024-09-16T20:49:40.523 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1113e-05 2024-09-16T20:49:40.523 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.524 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.524 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:40.524 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639206+0000", 2024-09-16T20:49:40.524 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1503e-05 2024-09-16T20:49:40.524 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.524 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.524 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:40.524 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689969+0000", 2024-09-16T20:49:40.524 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050762676 2024-09-16T20:49:40.524 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.524 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.524 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:40.524 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690071+0000", 2024-09-16T20:49:40.524 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102163 2024-09-16T20:49:40.524 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.525 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.525 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:40.525 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690191+0000", 2024-09-16T20:49:40.525 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000119728 2024-09-16T20:49:40.525 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:40.525 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:40.525 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:40.525 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.525 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.525 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3730 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1538048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:40.525 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:49:40.525 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 77.844195795000005, 2024-09-16T20:49:40.525 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051131467999999999, 2024-09-16T20:49:40.525 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:40.526 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:40.526 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:40.526 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:40.526 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:40.526 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:40.526 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3730 2024-09-16T20:49:40.526 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.526 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:40.526 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.526 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:40.526 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:49:40.526 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:40.526 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.527 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.527 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:40.527 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:49:40.527 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:40.527 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.527 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.527 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:40.527 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639216+0000", 2024-09-16T20:49:40.527 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.39e-07 2024-09-16T20:49:40.527 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.527 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.527 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:40.527 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639218+0000", 2024-09-16T20:49:40.527 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.537e-06 2024-09-16T20:49:40.527 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.528 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.528 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:40.528 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639229+0000", 2024-09-16T20:49:40.528 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1122e-05 2024-09-16T20:49:40.528 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.528 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.528 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:40.528 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639237+0000", 2024-09-16T20:49:40.528 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8509999999999999e-06 2024-09-16T20:49:40.528 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.528 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.528 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:40.528 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690227+0000", 2024-09-16T20:49:40.528 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050990147 2024-09-16T20:49:40.528 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.529 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.529 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:40.529 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690278+0000", 2024-09-16T20:49:40.529 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0223999999999999e-05 2024-09-16T20:49:40.529 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.529 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.529 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:40.529 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690347+0000", 2024-09-16T20:49:40.529 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9147999999999996e-05 2024-09-16T20:49:40.530 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:40.530 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:40.530 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:40.530 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.530 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.530 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3731 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1546240~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:40.530 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:49:40.530 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 77.844165086999993, 2024-09-16T20:49:40.530 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051276744999999999, 2024-09-16T20:49:40.530 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:40.530 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:40.530 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:40.530 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:40.530 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:40.530 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:40.530 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3731 2024-09-16T20:49:40.531 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.531 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:40.531 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.531 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:40.531 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:49:40.531 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:40.531 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.531 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.531 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:40.531 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:49:40.531 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:40.531 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.531 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.531 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:40.532 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:49:40.532 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9900000000000001e-07 2024-09-16T20:49:40.532 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.532 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.532 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:40.532 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639249+0000", 2024-09-16T20:49:40.532 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3630000000000001e-06 2024-09-16T20:49:40.532 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.532 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.532 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:40.532 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639255+0000", 2024-09-16T20:49:40.532 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4960000000000001e-06 2024-09-16T20:49:40.532 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.532 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.532 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:40.533 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639261+0000", 2024-09-16T20:49:40.533 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2839999999999997e-06 2024-09-16T20:49:40.533 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.533 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.533 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:40.533 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690375+0000", 2024-09-16T20:49:40.533 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051114684000000001 2024-09-16T20:49:40.533 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.533 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.533 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:40.533 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690425+0000", 2024-09-16T20:49:40.533 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9379999999999998e-05 2024-09-16T20:49:40.533 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.534 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.534 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:40.534 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690523+0000", 2024-09-16T20:49:40.534 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8139000000000003e-05 2024-09-16T20:49:40.534 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:40.534 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:40.534 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:40.534 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.534 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.534 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3732 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1554432~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:40.534 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:49:40.534 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 77.844119305000007, 2024-09-16T20:49:40.534 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051380096, 2024-09-16T20:49:40.535 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:40.535 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:40.535 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:40.535 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:40.535 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:40.535 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:40.535 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3732 2024-09-16T20:49:40.535 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.535 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:40.535 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.535 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:40.535 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:49:40.535 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:40.535 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.535 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.536 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:40.536 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:49:40.536 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:40.536 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.536 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.536 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:40.536 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:49:40.536 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6100000000000001e-07 2024-09-16T20:49:40.536 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.536 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.536 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:40.536 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639295+0000", 2024-09-16T20:49:40.537 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.447e-06 2024-09-16T20:49:40.537 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.537 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.537 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:40.537 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639302+0000", 2024-09-16T20:49:40.537 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0570000000000004e-06 2024-09-16T20:49:40.537 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.537 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.537 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:40.537 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639307+0000", 2024-09-16T20:49:40.537 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1429999999999997e-06 2024-09-16T20:49:40.537 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.537 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.537 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:40.538 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690555+0000", 2024-09-16T20:49:40.538 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051247963000000001 2024-09-16T20:49:40.538 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.538 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.538 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:40.538 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690602+0000", 2024-09-16T20:49:40.538 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7602000000000003e-05 2024-09-16T20:49:40.538 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.538 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.538 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:40.538 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690672+0000", 2024-09-16T20:49:40.538 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9423000000000003e-05 2024-09-16T20:49:40.538 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:40.538 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:40.538 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:40.538 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.539 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.539 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3734 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1562624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:40.539 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:49:40.539 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 77.844097654999999, 2024-09-16T20:49:40.539 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051490610999999999, 2024-09-16T20:49:40.539 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:40.539 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:40.539 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:40.539 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:40.539 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:40.539 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:40.539 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3734 2024-09-16T20:49:40.539 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.539 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:40.539 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.540 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:40.540 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:49:40.540 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:40.540 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.540 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.540 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:40.540 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:49:40.540 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:40.540 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.540 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.540 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:40.540 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639314+0000", 2024-09-16T20:49:40.540 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4400000000000001e-07 2024-09-16T20:49:40.540 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.540 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.541 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:40.541 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639316+0000", 2024-09-16T20:49:40.541 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1459999999999999e-06 2024-09-16T20:49:40.541 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.541 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.541 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:40.541 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639322+0000", 2024-09-16T20:49:40.541 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7610000000000004e-06 2024-09-16T20:49:40.541 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.541 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.541 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:40.541 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639327+0000", 2024-09-16T20:49:40.541 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.186e-06 2024-09-16T20:49:40.541 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.542 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.542 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:40.542 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690698+0000", 2024-09-16T20:49:40.542 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051371223000000001 2024-09-16T20:49:40.542 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.542 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.542 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:40.542 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690748+0000", 2024-09-16T20:49:40.542 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0192999999999999e-05 2024-09-16T20:49:40.542 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.542 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.542 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:40.542 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690804+0000", 2024-09-16T20:49:40.542 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5658000000000001e-05 2024-09-16T20:49:40.542 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:40.543 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:40.543 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:40.543 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.543 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.543 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3736 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1570816~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:40.543 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:49:40.543 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 77.844078396, 2024-09-16T20:49:40.543 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051596528000000003, 2024-09-16T20:49:40.543 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:40.543 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:40.543 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:40.544 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:40.544 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:40.544 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:40.544 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3736 2024-09-16T20:49:40.544 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.544 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:40.544 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.544 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:40.544 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:49:40.544 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:40.545 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.545 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.545 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:40.545 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:49:40.545 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:40.545 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.545 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.545 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:40.545 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:49:40.545 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9999999999999999e-07 2024-09-16T20:49:40.545 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.545 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.545 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:40.545 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639335+0000", 2024-09-16T20:49:40.546 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.635e-06 2024-09-16T20:49:40.546 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.546 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.546 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:40.546 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639340+0000", 2024-09-16T20:49:40.546 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5840000000000002e-06 2024-09-16T20:49:40.546 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.546 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.546 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:40.546 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639346+0000", 2024-09-16T20:49:40.546 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1190000000000002e-06 2024-09-16T20:49:40.546 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.546 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.546 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:40.546 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690817+0000", 2024-09-16T20:49:40.547 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051470721999999997 2024-09-16T20:49:40.547 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.547 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.547 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:40.547 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690875+0000", 2024-09-16T20:49:40.547 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7441000000000001e-05 2024-09-16T20:49:40.547 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.547 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.547 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:40.547 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690929+0000", 2024-09-16T20:49:40.547 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4727e-05 2024-09-16T20:49:40.547 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:40.547 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:40.547 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:40.547 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.547 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.548 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5434 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:40.548 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:49:40.548 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 77.61881683, 2024-09-16T20:49:40.548 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099755831000000003, 2024-09-16T20:49:40.548 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:40.548 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:40.548 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:40.548 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:40.548 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:40.548 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:40.548 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5434 2024-09-16T20:49:40.548 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.548 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:40.548 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.549 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:40.549 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:49:40.549 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:40.549 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.549 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.549 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:40.549 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:49:40.549 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:40.549 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.549 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.549 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:40.549 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864595+0000", 2024-09-16T20:49:40.549 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1099999999999995e-07 2024-09-16T20:49:40.549 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.550 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.550 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:40.550 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864597+0000", 2024-09-16T20:49:40.550 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3989999999999998e-06 2024-09-16T20:49:40.550 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.550 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.550 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:40.550 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864603+0000", 2024-09-16T20:49:40.550 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9309999999999996e-06 2024-09-16T20:49:40.550 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.550 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.550 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:40.550 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864607+0000", 2024-09-16T20:49:40.551 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6200000000000001e-06 2024-09-16T20:49:40.551 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.551 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.551 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:40.551 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964164+0000", 2024-09-16T20:49:40.551 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099557162000000005 2024-09-16T20:49:40.551 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.551 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.551 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:40.551 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964244+0000", 2024-09-16T20:49:40.551 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9597000000000004e-05 2024-09-16T20:49:40.551 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.551 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.551 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:40.551 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964350+0000", 2024-09-16T20:49:40.552 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010641099999999999 2024-09-16T20:49:40.552 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:40.552 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:40.552 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:40.552 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.552 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.552 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5435 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 20480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:40.552 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:49:40.552 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 77.618787007999998, 2024-09-16T20:49:40.552 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100897102, 2024-09-16T20:49:40.552 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:40.552 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:40.552 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:40.552 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:40.552 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:40.553 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:40.553 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5435 2024-09-16T20:49:40.553 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.553 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:40.553 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.553 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:40.553 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:49:40.553 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:40.553 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.553 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.553 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:40.553 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:49:40.553 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:40.554 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.554 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.554 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:40.554 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:49:40.554 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8000000000000001e-07 2024-09-16T20:49:40.554 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.554 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.554 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:40.554 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864626+0000", 2024-09-16T20:49:40.554 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.745e-06 2024-09-16T20:49:40.554 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.555 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.555 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:40.555 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864633+0000", 2024-09-16T20:49:40.555 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3339999999999998e-06 2024-09-16T20:49:40.555 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.555 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.555 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:40.555 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864636+0000", 2024-09-16T20:49:40.555 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1200000000000002e-06 2024-09-16T20:49:40.555 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.555 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.555 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:40.555 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965299+0000", 2024-09-16T20:49:40.555 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10066352000000001 2024-09-16T20:49:40.555 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.556 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.556 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:40.556 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965390+0000", 2024-09-16T20:49:40.556 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0894000000000001e-05 2024-09-16T20:49:40.556 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.556 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.556 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:40.556 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965521+0000", 2024-09-16T20:49:40.556 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013110899999999999 2024-09-16T20:49:40.556 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:40.556 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:40.556 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:40.556 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.556 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.556 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:152 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 327680~32768 out=32768b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:49:40.557 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:49:40.557 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 76.320290065999998, 2024-09-16T20:49:40.557 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11869405800000001, 2024-09-16T20:49:40.557 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:40.557 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:40.557 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:40.557 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:40.557 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:49:40.557 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:49:40.557 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 152 2024-09-16T20:49:40.557 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.557 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:40.557 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.557 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:40.557 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:49:40.558 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:40.558 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.558 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.558 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:40.558 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:49:40.558 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:40.558 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.558 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.558 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:40.558 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163122+0000", 2024-09-16T20:49:40.558 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6400000000000003e-07 2024-09-16T20:49:40.558 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.558 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.558 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:40.558 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163124+0000", 2024-09-16T20:49:40.558 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.565e-06 2024-09-16T20:49:40.559 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.559 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.559 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:40.559 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163136+0000", 2024-09-16T20:49:40.559 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1717e-05 2024-09-16T20:49:40.559 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.559 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.559 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:40.559 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163142+0000", 2024-09-16T20:49:40.559 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1750000000000002e-06 2024-09-16T20:49:40.559 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.559 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.559 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:40.560 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163606+0000", 2024-09-16T20:49:40.560 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00046418799999999998 2024-09-16T20:49:40.560 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.560 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.560 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:40.560 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163656+0000", 2024-09-16T20:49:40.560 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9852999999999999e-05 2024-09-16T20:49:40.560 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.560 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.560 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:40.560 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281815+0000", 2024-09-16T20:49:40.560 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11815909600000001 2024-09-16T20:49:40.560 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:40.560 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:40.561 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:40.561 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.561 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.561 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:153 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:49:40.561 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:49:40.561 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 76.320257112999997, 2024-09-16T20:49:40.561 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11903462400000001, 2024-09-16T20:49:40.561 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:40.561 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:40.561 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:40.561 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:40.561 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:49:40.561 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:49:40.562 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 153 2024-09-16T20:49:40.562 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.562 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:40.562 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.562 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:40.562 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:49:40.562 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:40.562 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.562 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.562 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:40.562 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:49:40.562 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:40.562 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.562 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.562 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:40.562 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163155+0000", 2024-09-16T20:49:40.563 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.37e-07 2024-09-16T20:49:40.563 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.563 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.563 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:40.563 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163157+0000", 2024-09-16T20:49:40.563 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6010000000000002e-06 2024-09-16T20:49:40.563 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.563 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.563 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:40.563 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163165+0000", 2024-09-16T20:49:40.563 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2619999999999998e-06 2024-09-16T20:49:40.563 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.563 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.563 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:40.563 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163171+0000", 2024-09-16T20:49:40.564 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.711e-06 2024-09-16T20:49:40.564 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.564 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.564 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:40.564 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281868+0000", 2024-09-16T20:49:40.564 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118696622 2024-09-16T20:49:40.564 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.564 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.564 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:40.564 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281969+0000", 2024-09-16T20:49:40.564 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101371 2024-09-16T20:49:40.564 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.564 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.564 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:40.564 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282189+0000", 2024-09-16T20:49:40.565 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00021952000000000001 2024-09-16T20:49:40.565 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:40.565 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:40.565 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:40.565 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.565 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.565 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:154 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 360448~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:49:40.565 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:49:40.565 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 76.320225340999997, 2024-09-16T20:49:40.565 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11932035100000001, 2024-09-16T20:49:40.565 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:40.565 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:40.565 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:40.565 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:40.565 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:49:40.566 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:49:40.566 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 154 2024-09-16T20:49:40.566 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.566 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:40.566 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.566 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:40.566 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:49:40.566 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:40.566 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.566 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.566 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:40.566 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:49:40.566 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:40.566 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.567 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.567 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:40.567 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:49:40.567 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:49:40.567 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.567 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.567 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:40.567 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163189+0000", 2024-09-16T20:49:40.567 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6840000000000001e-06 2024-09-16T20:49:40.567 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.567 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.567 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:40.568 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163196+0000", 2024-09-16T20:49:40.568 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7920000000000004e-06 2024-09-16T20:49:40.568 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.568 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.568 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:40.568 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163201+0000", 2024-09-16T20:49:40.568 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2809999999999999e-06 2024-09-16T20:49:40.568 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.568 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.568 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:40.568 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282219+0000", 2024-09-16T20:49:40.568 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.119018343 2024-09-16T20:49:40.568 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.568 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.568 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:40.569 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282305+0000", 2024-09-16T20:49:40.569 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5865000000000005e-05 2024-09-16T20:49:40.569 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.569 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.569 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:40.569 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282506+0000", 2024-09-16T20:49:40.569 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000200927 2024-09-16T20:49:40.569 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:40.569 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:40.569 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:40.569 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.570 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.570 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop(client.4366.0:3088 3.1e e38/23 3:79e5ec01:::rbd_header.110e891a9b2b:head v 38'306, mlcod=36'304)", 2024-09-16T20:49:40.570 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:49:40.570 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 76.319206472999994, 2024-09-16T20:49:40.570 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118102759, 2024-09-16T20:49:40.570 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:40.571 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:40.571 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:40.571 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:40.571 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.571 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:40.571 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:49:40.571 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:40.571 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.572 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.572 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:40.572 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:49:40.572 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:40.572 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.572 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.573 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:40.573 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:49:40.573 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.85e-07 2024-09-16T20:49:40.573 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.573 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.573 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:40.573 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164212+0000", 2024-09-16T20:49:40.573 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3999999999999997e-06 2024-09-16T20:49:40.574 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.574 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.574 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:40.574 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281553+0000", 2024-09-16T20:49:40.574 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11734135499999999 2024-09-16T20:49:40.574 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.574 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.575 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:40.575 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281567+0000", 2024-09-16T20:49:40.575 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4188999999999999e-05 2024-09-16T20:49:40.575 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.575 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.575 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:40.575 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281653+0000", 2024-09-16T20:49:40.576 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5385999999999999e-05 2024-09-16T20:49:40.576 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.576 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.576 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:40.576 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281679+0000", 2024-09-16T20:49:40.576 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6689000000000001e-05 2024-09-16T20:49:40.577 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.577 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.577 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:40.577 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282277+0000", 2024-09-16T20:49:40.577 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00059798000000000002 2024-09-16T20:49:40.577 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.577 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.577 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:40.578 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282307+0000", 2024-09-16T20:49:40.578 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9975e-05 2024-09-16T20:49:40.578 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:40.578 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:40.578 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:40.578 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.578 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.578 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5255.0:44 3.16 3:6f5476df:::rbd_header.110e25db1f60:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:49:40.578 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:49:40.578 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.162903418, 2024-09-16T20:49:40.579 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.121007509, 2024-09-16T20:49:40.579 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:40.579 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:40.579 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:40.579 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:40.579 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5255", 2024-09-16T20:49:40.579 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:49:40.579 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 44 2024-09-16T20:49:40.579 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.579 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:40.579 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.579 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:40.579 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:49:40.580 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:40.580 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.580 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.580 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:40.580 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:49:40.580 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:40.580 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.580 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.580 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:40.580 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:49:40.580 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0500000000000003e-07 2024-09-16T20:49:40.580 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.580 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.581 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:40.581 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320512+0000", 2024-09-16T20:49:40.581 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2540000000000001e-06 2024-09-16T20:49:40.581 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.581 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.581 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:40.581 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320518+0000", 2024-09-16T20:49:40.581 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1689999999999997e-06 2024-09-16T20:49:40.581 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.581 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.581 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:40.581 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320521+0000", 2024-09-16T20:49:40.582 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5200000000000002e-06 2024-09-16T20:49:40.582 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.582 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.582 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:40.582 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320548+0000", 2024-09-16T20:49:40.582 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6662999999999999e-05 2024-09-16T20:49:40.582 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.582 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.582 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:40.582 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320587+0000", 2024-09-16T20:49:40.582 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8866999999999999e-05 2024-09-16T20:49:40.582 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:40.583 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:40.583 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:40.583 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.441515+0000", 2024-09-16T20:49:40.583 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.120928331 2024-09-16T20:49:40.583 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:40.583 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:40.583 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:40.583 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:40.583 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:40.583 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:40.585 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:49:40.585 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:49:40.586 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:49:40.586 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:49:40.586 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:49:40.587 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:49:40.587 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:49:40.588 INFO:teuthology.orchestra.run.smithi029.stderr:nodeep-scrub is unset 2024-09-16T20:49:40.588 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:40.583+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:40.686 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:40.685+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:40.695 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:40.696 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:49:40.696 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:49:40.696 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:40.704 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:49:40.786 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:40.785+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:40.816 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:40.816 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:49:40.817 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:49:40.817 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:49:40.817 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:40.826 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:49:40.887 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:40.886+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:40.938 INFO:teuthology.orchestra.run.smithi139.stdout:3997 read 0x4dd6400 thru 0x4ddaea9 (0x4aaa bytes) 2024-09-16T20:49:40.939 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:40.939 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:49:40.939 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:49:40.939 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:49:40.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.939 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6419 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3594240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:40.939 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871775+0000", 2024-09-16T20:49:40.939 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.064887944999995, 2024-09-16T20:49:40.939 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048754724999999999, 2024-09-16T20:49:40.939 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:40.939 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:40.939 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:40.940 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:40.940 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:40.940 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:40.940 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6419 2024-09-16T20:49:40.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.940 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:40.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:40.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871775+0000", 2024-09-16T20:49:40.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:40.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:40.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871775+0000", 2024-09-16T20:49:40.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:40.941 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.941 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.941 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:40.941 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871775+0000", 2024-09-16T20:49:40.941 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2500000000000001e-07 2024-09-16T20:49:40.941 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.941 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.941 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:40.941 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871777+0000", 2024-09-16T20:49:40.941 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9649999999999998e-06 2024-09-16T20:49:40.941 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.941 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.941 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:40.941 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871784+0000", 2024-09-16T20:49:40.941 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3419999999999996e-06 2024-09-16T20:49:40.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.942 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.942 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:40.942 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871789+0000", 2024-09-16T20:49:40.942 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1889999999999998e-06 2024-09-16T20:49:40.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.942 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.942 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:40.942 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920416+0000", 2024-09-16T20:49:40.942 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048627363999999999 2024-09-16T20:49:40.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.942 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.942 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:40.943 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920472+0000", 2024-09-16T20:49:40.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5429000000000001e-05 2024-09-16T20:49:40.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.943 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:40.943 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920530+0000", 2024-09-16T20:49:40.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8111e-05 2024-09-16T20:49:40.943 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:40.943 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:40.943 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:40.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.943 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6420 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3602432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:40.943 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871796+0000", 2024-09-16T20:49:40.943 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.064867031999995, 2024-09-16T20:49:40.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048857298, 2024-09-16T20:49:40.944 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:40.944 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:40.944 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:40.944 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:40.944 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:40.944 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:40.944 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6420 2024-09-16T20:49:40.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.944 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:40.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.944 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:40.944 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871796+0000", 2024-09-16T20:49:40.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:40.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.945 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:40.945 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871796+0000", 2024-09-16T20:49:40.945 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:40.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.945 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:40.945 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871854+0000", 2024-09-16T20:49:40.945 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8187999999999999e-05 2024-09-16T20:49:40.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.946 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:40.946 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871856+0000", 2024-09-16T20:49:40.946 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3549999999999999e-06 2024-09-16T20:49:40.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.946 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.946 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:40.946 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871866+0000", 2024-09-16T20:49:40.946 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6500000000000008e-06 2024-09-16T20:49:40.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.946 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.946 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:40.947 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871872+0000", 2024-09-16T20:49:40.947 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3799999999999999e-06 2024-09-16T20:49:40.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.947 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.947 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:40.947 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920558+0000", 2024-09-16T20:49:40.947 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048685797000000003 2024-09-16T20:49:40.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.948 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:40.948 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920603+0000", 2024-09-16T20:49:40.948 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4629999999999998e-05 2024-09-16T20:49:40.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.948 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:40.948 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920653+0000", 2024-09-16T20:49:40.948 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0297999999999999e-05 2024-09-16T20:49:40.948 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:40.949 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:40.949 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:40.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.949 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.949 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6421 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3610624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:40.949 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871882+0000", 2024-09-16T20:49:40.949 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.064780963999993, 2024-09-16T20:49:40.949 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048913381999999998, 2024-09-16T20:49:40.949 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:40.949 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:40.949 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:40.949 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:40.949 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:40.949 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:40.949 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6421 2024-09-16T20:49:40.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.950 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:40.950 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.950 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:40.950 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871882+0000", 2024-09-16T20:49:40.950 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:40.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.950 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.950 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:40.950 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871882+0000", 2024-09-16T20:49:40.950 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:40.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.950 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.950 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:40.950 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.873257+0000", 2024-09-16T20:49:40.951 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001374789 2024-09-16T20:49:40.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.951 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.951 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:40.951 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.873261+0000", 2024-09-16T20:49:40.951 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5589999999999999e-06 2024-09-16T20:49:40.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.951 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.951 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:40.951 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.873275+0000", 2024-09-16T20:49:40.951 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3675999999999999e-05 2024-09-16T20:49:40.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.951 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.951 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:40.951 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.873281+0000", 2024-09-16T20:49:40.952 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6930000000000004e-06 2024-09-16T20:49:40.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.952 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.952 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:40.952 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920684+0000", 2024-09-16T20:49:40.952 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047402913999999997 2024-09-16T20:49:40.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.952 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.952 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:40.952 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920737+0000", 2024-09-16T20:49:40.952 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3600000000000002e-05 2024-09-16T20:49:40.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.952 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.952 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:40.953 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920795+0000", 2024-09-16T20:49:40.953 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8150999999999999e-05 2024-09-16T20:49:40.953 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:40.953 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:40.953 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:40.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.953 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.953 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:7491 3.1e 3:7c48f6ae:::rbd_object_map.110e891a9b2b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:40.953 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.925865+0000", 2024-09-16T20:49:40.953 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.010797819999993, 2024-09-16T20:49:40.953 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048897293000000001, 2024-09-16T20:49:40.953 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:40.953 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:40.953 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:40.954 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:40.954 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:40.954 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:40.954 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 7491 2024-09-16T20:49:40.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.954 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:40.954 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.954 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:40.954 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925865+0000", 2024-09-16T20:49:40.954 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:40.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.954 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.954 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:40.954 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925865+0000", 2024-09-16T20:49:40.955 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:40.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.955 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:40.955 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925866+0000", 2024-09-16T20:49:40.955 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.39e-07 2024-09-16T20:49:40.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.955 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:40.955 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925868+0000", 2024-09-16T20:49:40.955 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.508e-06 2024-09-16T20:49:40.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.955 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:40.955 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925874+0000", 2024-09-16T20:49:40.956 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7919999999999998e-06 2024-09-16T20:49:40.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.956 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:40.956 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925879+0000", 2024-09-16T20:49:40.956 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8579999999999999e-06 2024-09-16T20:49:40.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.956 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:40.956 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.970875+0000", 2024-09-16T20:49:40.956 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044996412 2024-09-16T20:49:40.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.956 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:40.956 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.970970+0000", 2024-09-16T20:49:40.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4732e-05 2024-09-16T20:49:40.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.957 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-16T20:49:40.957 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.971275+0000", 2024-09-16T20:49:40.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00030464199999999999 2024-09-16T20:49:40.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.957 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:40.957 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.974598+0000", 2024-09-16T20:49:40.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0033238159999999998 2024-09-16T20:49:40.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.958 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:49:40.958 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.974666+0000", 2024-09-16T20:49:40.958 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7617e-05 2024-09-16T20:49:40.958 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.958 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.958 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:49:40.958 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.974718+0000", 2024-09-16T20:49:40.958 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2259000000000002e-05 2024-09-16T20:49:40.958 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.958 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.958 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:40.959 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.974762+0000", 2024-09-16T20:49:40.959 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4218000000000003e-05 2024-09-16T20:49:40.959 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:40.959 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:40.959 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:40.959 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.959 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.959 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4758 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1521664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:40.959 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689141+0000", 2024-09-16T20:49:40.959 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 78.247522420999999, 2024-09-16T20:49:40.959 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048640741000000001, 2024-09-16T20:49:40.959 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:40.960 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:40.960 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:40.960 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:40.960 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:49:40.960 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:40.960 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4758 2024-09-16T20:49:40.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.960 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:40.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.960 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:40.961 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689141+0000", 2024-09-16T20:49:40.961 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:40.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.961 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.961 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:40.961 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689141+0000", 2024-09-16T20:49:40.961 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:40.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.961 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.961 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:40.961 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689141+0000", 2024-09-16T20:49:40.962 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8799999999999998e-07 2024-09-16T20:49:40.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.962 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:40.962 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689143+0000", 2024-09-16T20:49:40.962 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.945e-06 2024-09-16T20:49:40.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.962 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:40.962 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689149+0000", 2024-09-16T20:49:40.962 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9800000000000003e-06 2024-09-16T20:49:40.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.963 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.963 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:40.963 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689155+0000", 2024-09-16T20:49:40.963 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7509999999999997e-06 2024-09-16T20:49:40.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.963 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.963 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:40.963 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737681+0000", 2024-09-16T20:49:40.963 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048526825000000003 2024-09-16T20:49:40.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.964 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.964 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:40.964 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737729+0000", 2024-09-16T20:49:40.964 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7219999999999999e-05 2024-09-16T20:49:40.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.964 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.964 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:40.964 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737781+0000", 2024-09-16T20:49:40.964 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2632e-05 2024-09-16T20:49:40.964 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:40.965 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:40.965 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:40.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.965 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.965 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4760 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1529856~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:40.965 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689161+0000", 2024-09-16T20:49:40.965 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 78.247502073999996, 2024-09-16T20:49:40.965 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048743981999999998, 2024-09-16T20:49:40.965 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:40.965 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:40.965 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:40.965 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:40.965 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:49:40.966 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:40.966 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4760 2024-09-16T20:49:40.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.966 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:40.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.966 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:40.966 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689161+0000", 2024-09-16T20:49:40.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:40.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.966 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:40.966 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689161+0000", 2024-09-16T20:49:40.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:40.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.967 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:40.967 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689161+0000", 2024-09-16T20:49:40.967 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5599999999999995e-07 2024-09-16T20:49:40.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.967 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:40.967 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689163+0000", 2024-09-16T20:49:40.967 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.818e-06 2024-09-16T20:49:40.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.967 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:40.967 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689169+0000", 2024-09-16T20:49:40.967 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0809999999999999e-06 2024-09-16T20:49:40.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.968 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.968 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:40.968 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689174+0000", 2024-09-16T20:49:40.968 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.938e-06 2024-09-16T20:49:40.968 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.968 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.968 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:40.968 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737807+0000", 2024-09-16T20:49:40.968 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048633124 2024-09-16T20:49:40.968 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.968 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.968 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:40.968 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737851+0000", 2024-09-16T20:49:40.969 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3764000000000003e-05 2024-09-16T20:49:40.969 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.969 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.969 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:40.969 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737905+0000", 2024-09-16T20:49:40.969 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3701e-05 2024-09-16T20:49:40.969 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:40.969 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:40.969 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:40.969 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.969 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.969 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4762 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1538048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:40.970 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689182+0000", 2024-09-16T20:49:40.970 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 78.247480851999995, 2024-09-16T20:49:40.970 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048850275999999998, 2024-09-16T20:49:40.970 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:40.970 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:40.970 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:40.970 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:40.970 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:49:40.970 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:40.970 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4762 2024-09-16T20:49:40.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.970 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:40.970 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.971 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:40.971 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689182+0000", 2024-09-16T20:49:40.971 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:40.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.971 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.971 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:40.971 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689182+0000", 2024-09-16T20:49:40.971 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:40.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.971 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.971 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:40.971 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689182+0000", 2024-09-16T20:49:40.972 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-16T20:49:40.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.972 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.972 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:40.972 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689184+0000", 2024-09-16T20:49:40.972 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8190000000000001e-06 2024-09-16T20:49:40.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.972 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.972 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:40.972 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689191+0000", 2024-09-16T20:49:40.973 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2530000000000001e-06 2024-09-16T20:49:40.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.973 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.973 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:40.973 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689195+0000", 2024-09-16T20:49:40.973 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9559999999999996e-06 2024-09-16T20:49:40.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.973 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.973 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:40.973 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737933+0000", 2024-09-16T20:49:40.973 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048737919999999997 2024-09-16T20:49:40.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.974 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:40.974 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737982+0000", 2024-09-16T20:49:40.974 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8089e-05 2024-09-16T20:49:40.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.974 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:40.974 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738032+0000", 2024-09-16T20:49:40.974 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0868999999999997e-05 2024-09-16T20:49:40.974 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:40.974 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:40.974 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:40.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.974 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4764 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1546240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:40.974 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689201+0000", 2024-09-16T20:49:40.975 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 78.247461494999996, 2024-09-16T20:49:40.975 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048957550000000002, 2024-09-16T20:49:40.975 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:40.975 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:40.975 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:40.975 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:40.975 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:49:40.975 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:40.975 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4764 2024-09-16T20:49:40.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.975 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:40.975 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.975 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:40.976 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689201+0000", 2024-09-16T20:49:40.976 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:40.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.976 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.976 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:40.976 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689201+0000", 2024-09-16T20:49:40.976 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:40.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.976 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.976 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:40.976 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689202+0000", 2024-09-16T20:49:40.976 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4200000000000002e-07 2024-09-16T20:49:40.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.977 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:40.977 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689204+0000", 2024-09-16T20:49:40.977 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9750000000000001e-06 2024-09-16T20:49:40.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.977 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:40.977 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689210+0000", 2024-09-16T20:49:40.977 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1e-06 2024-09-16T20:49:40.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.978 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:40.978 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689215+0000", 2024-09-16T20:49:40.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0180000000000001e-06 2024-09-16T20:49:40.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.978 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.978 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:40.978 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738058+0000", 2024-09-16T20:49:40.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048843573000000001 2024-09-16T20:49:40.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.978 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.978 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:40.978 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738102+0000", 2024-09-16T20:49:40.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3256000000000001e-05 2024-09-16T20:49:40.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.979 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.979 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:40.979 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738159+0000", 2024-09-16T20:49:40.979 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7286e-05 2024-09-16T20:49:40.979 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:40.979 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:40.979 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:40.979 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.979 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.979 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4766 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1554432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:40.979 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689221+0000", 2024-09-16T20:49:40.979 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 78.247441529, 2024-09-16T20:49:40.979 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049064996, 2024-09-16T20:49:40.979 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:40.979 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:40.980 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:40.980 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:40.980 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:49:40.980 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:40.980 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4766 2024-09-16T20:49:40.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.980 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:40.980 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.980 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:40.980 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689221+0000", 2024-09-16T20:49:40.980 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:40.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.980 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.981 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:40.981 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689221+0000", 2024-09-16T20:49:40.981 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:40.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.981 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.981 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:40.981 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689222+0000", 2024-09-16T20:49:40.981 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.34e-07 2024-09-16T20:49:40.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.981 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.982 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:40.982 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689224+0000", 2024-09-16T20:49:40.982 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.846e-06 2024-09-16T20:49:40.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.982 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.982 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:40.982 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689247+0000", 2024-09-16T20:49:40.982 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3448e-05 2024-09-16T20:49:40.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.982 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.982 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:40.982 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689254+0000", 2024-09-16T20:49:40.982 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6969999999999996e-06 2024-09-16T20:49:40.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.983 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.983 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:40.983 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738188+0000", 2024-09-16T20:49:40.983 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048934520000000002 2024-09-16T20:49:40.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.983 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.983 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:40.983 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738234+0000", 2024-09-16T20:49:40.983 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5695999999999998e-05 2024-09-16T20:49:40.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.983 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.983 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:40.983 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738286+0000", 2024-09-16T20:49:40.984 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2355000000000003e-05 2024-09-16T20:49:40.984 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:40.984 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:40.984 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:40.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.984 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.984 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4768 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1562624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:40.984 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689262+0000", 2024-09-16T20:49:40.984 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 78.247400575, 2024-09-16T20:49:40.984 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049157157999999999, 2024-09-16T20:49:40.984 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:40.984 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:40.984 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:40.985 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:40.985 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:49:40.985 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:40.985 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4768 2024-09-16T20:49:40.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.985 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:40.985 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.985 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:40.985 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689262+0000", 2024-09-16T20:49:40.985 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:40.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.986 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.986 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:40.986 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689262+0000", 2024-09-16T20:49:40.986 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:40.986 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.986 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.986 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:40.986 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689263+0000", 2024-09-16T20:49:40.986 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6199999999999998e-07 2024-09-16T20:49:40.986 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.986 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.986 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:40.986 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689270+0000", 2024-09-16T20:49:40.986 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.798e-06 2024-09-16T20:49:40.987 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.987 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.987 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:40.987 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689277+0000", 2024-09-16T20:49:40.987 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0890000000000002e-06 2024-09-16T20:49:40.987 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.987 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.987 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:40.987 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689282+0000", 2024-09-16T20:49:40.987 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0470000000000001e-06 2024-09-16T20:49:40.988 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.988 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.988 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:40.988 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738319+0000", 2024-09-16T20:49:40.988 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049037139 2024-09-16T20:49:40.988 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.988 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.988 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:40.988 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738365+0000", 2024-09-16T20:49:40.988 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6356999999999997e-05 2024-09-16T20:49:40.988 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.988 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.988 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:40.988 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738420+0000", 2024-09-16T20:49:40.989 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4265999999999998e-05 2024-09-16T20:49:40.989 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:40.989 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:40.989 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:40.989 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.989 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.989 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4770 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1570816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:40.989 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689289+0000", 2024-09-16T20:49:40.989 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 78.247374065000002, 2024-09-16T20:49:40.989 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049253705000000002, 2024-09-16T20:49:40.989 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:40.990 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:40.990 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:40.990 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:40.990 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:49:40.990 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:40.990 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4770 2024-09-16T20:49:40.990 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.990 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:40.990 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.990 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:40.990 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689289+0000", 2024-09-16T20:49:40.990 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:40.990 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.990 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.991 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:40.991 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689289+0000", 2024-09-16T20:49:40.991 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:40.991 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.991 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.991 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:40.991 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689289+0000", 2024-09-16T20:49:40.991 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0999999999999999e-07 2024-09-16T20:49:40.991 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.991 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.991 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:40.991 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689291+0000", 2024-09-16T20:49:40.991 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0030000000000001e-06 2024-09-16T20:49:40.992 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.992 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.992 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:40.992 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689297+0000", 2024-09-16T20:49:40.992 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9479999999999996e-06 2024-09-16T20:49:40.992 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.992 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.992 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:40.992 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689302+0000", 2024-09-16T20:49:40.992 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7389999999999999e-06 2024-09-16T20:49:40.992 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.992 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.992 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:40.993 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738442+0000", 2024-09-16T20:49:40.993 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049140001000000003 2024-09-16T20:49:40.993 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.993 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.993 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:40.993 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738490+0000", 2024-09-16T20:49:40.993 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7787000000000002e-05 2024-09-16T20:49:40.993 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.993 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.993 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:40.994 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738543+0000", 2024-09-16T20:49:40.994 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2816999999999999e-05 2024-09-16T20:49:40.994 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:40.994 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:40.994 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:40.994 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.994 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.994 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:3088 3.1e 3:79e5ec01:::rbd_header.110e891a9b2b:head [watch watch cookie 140213437171488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:49:40.994 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:49:40.994 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 76.772818048000005, 2024-09-16T20:49:40.994 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.11895186200000001, 2024-09-16T20:49:40.994 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:40.995 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:40.995 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:40.995 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:40.995 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:40.995 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:40.995 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3088 2024-09-16T20:49:40.995 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.995 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:40.995 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.995 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:40.995 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:49:40.995 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:40.995 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.996 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.996 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:40.996 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:49:40.996 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:40.996 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.996 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.996 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:40.996 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163846+0000", 2024-09-16T20:49:40.996 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6799999999999999e-07 2024-09-16T20:49:40.996 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.996 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.997 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:40.997 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163850+0000", 2024-09-16T20:49:40.997 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.929e-06 2024-09-16T20:49:40.997 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.997 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.997 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:40.997 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163862+0000", 2024-09-16T20:49:40.997 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1972e-05 2024-09-16T20:49:40.997 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.997 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.997 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:40.997 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163867+0000", 2024-09-16T20:49:40.998 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.451e-06 2024-09-16T20:49:40.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.998 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.998 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:40.998 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163912+0000", 2024-09-16T20:49:40.998 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4456000000000003e-05 2024-09-16T20:49:40.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.998 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.998 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:40.998 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163962+0000", 2024-09-16T20:49:40.998 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0578000000000003e-05 2024-09-16T20:49:40.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.998 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.998 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-16T20:49:40.999 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164058+0000", 2024-09-16T20:49:40.999 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5934999999999995e-05 2024-09-16T20:49:40.999 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.999 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.999 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:49:40.999 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164570+0000", 2024-09-16T20:49:40.999 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00051241100000000005 2024-09-16T20:49:40.999 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:40.999 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:40.999 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:40.999 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282629+0000", 2024-09-16T20:49:41.000 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.118058986 2024-09-16T20:49:41.000 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.000 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.000 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:49:41.000 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282725+0000", 2024-09-16T20:49:41.000 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5063999999999996e-05 2024-09-16T20:49:41.000 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.000 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.000 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:41.000 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282797+0000", 2024-09-16T20:49:41.000 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2212000000000006e-05 2024-09-16T20:49:41.001 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:41.001 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:41.001 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:41.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.001 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.001 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:8039 3.1e 3.ac44ff9e (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:49:41.001 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:49:41.001 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.228990699000001, 2024-09-16T20:49:41.001 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0053310449999999, 2024-09-16T20:49:41.001 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:41.001 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:41.001 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:49:41.002 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:41.002 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:41.002 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:41.002 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8039 2024-09-16T20:49:41.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.002 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:41.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.002 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:41.002 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:49:41.002 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:41.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.003 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:41.003 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:49:41.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:41.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.003 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.003 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:41.003 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707674+0000", 2024-09-16T20:49:41.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6530000000000001e-06 2024-09-16T20:49:41.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.003 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.003 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:41.003 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707681+0000", 2024-09-16T20:49:41.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9639999999999997e-06 2024-09-16T20:49:41.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.004 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.004 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:41.004 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707700+0000", 2024-09-16T20:49:41.004 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8674000000000001e-05 2024-09-16T20:49:41.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.004 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.004 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:41.004 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707708+0000", 2024-09-16T20:49:41.004 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6810000000000004e-06 2024-09-16T20:49:41.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.004 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.004 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:41.005 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711293+0000", 2024-09-16T20:49:41.005 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035848189999999999 2024-09-16T20:49:41.005 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.005 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.005 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:49:41.005 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711301+0000", 2024-09-16T20:49:41.005 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4730000000000003e-06 2024-09-16T20:49:41.005 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.005 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.005 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:41.005 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.712990+0000", 2024-09-16T20:49:41.005 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.001689348 2024-09-16T20:49:41.005 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.005 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.006 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:41.006 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713003+0000", 2024-09-16T20:49:41.006 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3433e-05 2024-09-16T20:49:41.006 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:41.006 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:41.006 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:41.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.006 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.006 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:8040 3.1e 3.ac44ff9e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:49:41.006 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:49:41.006 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.228930304000002, 2024-09-16T20:49:41.006 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0053410780000001, 2024-09-16T20:49:41.006 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:41.006 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:41.006 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:49:41.007 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:41.007 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:41.007 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:41.007 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8040 2024-09-16T20:49:41.007 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.007 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:41.007 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.007 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:41.007 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:49:41.007 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:41.007 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.007 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.007 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:41.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:49:41.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:41.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.008 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:41.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:49:41.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8100000000000003e-07 2024-09-16T20:49:41.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.008 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:41.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707737+0000", 2024-09-16T20:49:41.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5020000000000001e-06 2024-09-16T20:49:41.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.009 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.009 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:41.009 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707745+0000", 2024-09-16T20:49:41.009 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2169999999999994e-06 2024-09-16T20:49:41.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.009 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.009 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:41.009 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707751+0000", 2024-09-16T20:49:41.009 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.263e-06 2024-09-16T20:49:41.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.009 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.009 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:41.009 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711336+0000", 2024-09-16T20:49:41.010 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035848360000000001 2024-09-16T20:49:41.010 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.010 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.010 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:49:41.010 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711340+0000", 2024-09-16T20:49:41.010 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1309999999999999e-06 2024-09-16T20:49:41.010 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.010 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.010 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:41.010 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713060+0000", 2024-09-16T20:49:41.010 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0017195059999999 2024-09-16T20:49:41.010 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.010 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.010 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:41.010 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713074+0000", 2024-09-16T20:49:41.011 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4042e-05 2024-09-16T20:49:41.011 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:41.011 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:41.011 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:41.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.011 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.011 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4927.0:4487 3.1e 3.ac44ff9e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:49:41.011 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:49:41.011 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.228410101999998, 2024-09-16T20:49:41.011 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0048840880000001, 2024-09-16T20:49:41.011 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:41.011 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:41.011 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:49:41.011 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:41.011 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4927", 2024-09-16T20:49:41.012 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:41.012 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4487 2024-09-16T20:49:41.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.012 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:41.012 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.012 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:41.012 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:49:41.012 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:41.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.012 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.012 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:41.012 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:49:41.012 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:41.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.012 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.013 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:41.013 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708254+0000", 2024-09-16T20:49:41.013 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1800000000000005e-07 2024-09-16T20:49:41.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.013 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.013 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:41.013 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708258+0000", 2024-09-16T20:49:41.013 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8689999999999997e-06 2024-09-16T20:49:41.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.013 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.013 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:41.013 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708267+0000", 2024-09-16T20:49:41.014 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9590000000000007e-06 2024-09-16T20:49:41.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.014 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.014 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:41.014 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708274+0000", 2024-09-16T20:49:41.014 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9970000000000001e-06 2024-09-16T20:49:41.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.014 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.014 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:41.014 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711366+0000", 2024-09-16T20:49:41.014 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030919530000000001 2024-09-16T20:49:41.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.015 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.015 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:49:41.015 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711372+0000", 2024-09-16T20:49:41.015 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9410000000000004e-06 2024-09-16T20:49:41.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.015 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.015 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:41.015 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713126+0000", 2024-09-16T20:49:41.015 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0017537249999999 2024-09-16T20:49:41.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.015 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.015 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:41.015 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713137+0000", 2024-09-16T20:49:41.015 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0825999999999999e-05 2024-09-16T20:49:41.016 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:41.016 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:41.016 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:41.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.016 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.016 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5089.0:11703 3.1b 3:dcd4684e:::rbd_data.110e747f0779.000000000000001b:head [set-alloc-hint object_size 262144 write_size 262144,write 139264~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:49:41.016 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:49:41.016 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.798674327999997, 2024-09-16T20:49:41.016 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.084231385000000006, 2024-09-16T20:49:41.016 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:41.016 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:41.016 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:41.017 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:41.017 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5089", 2024-09-16T20:49:41.017 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:49:41.017 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 11703 2024-09-16T20:49:41.017 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.017 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:41.017 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.017 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:41.017 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:49:41.017 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:41.017 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.017 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.017 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:41.017 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:49:41.018 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:41.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.018 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.018 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:41.018 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:49:41.018 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2600000000000001e-07 2024-09-16T20:49:41.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.018 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.018 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:41.018 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137998+0000", 2024-09-16T20:49:41.018 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9549999999999998e-06 2024-09-16T20:49:41.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.018 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.018 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:41.018 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138008+0000", 2024-09-16T20:49:41.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3789999999999995e-06 2024-09-16T20:49:41.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.019 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:41.019 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138013+0000", 2024-09-16T20:49:41.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2209999999999996e-06 2024-09-16T20:49:41.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.019 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:41.019 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138054+0000", 2024-09-16T20:49:41.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1022999999999997e-05 2024-09-16T20:49:41.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.019 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:41.020 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138124+0000", 2024-09-16T20:49:41.020 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0234999999999994e-05 2024-09-16T20:49:41.020 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.020 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.020 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:41.020 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138197+0000", 2024-09-16T20:49:41.020 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2451000000000001e-05 2024-09-16T20:49:41.020 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.020 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.020 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:41.020 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222010+0000", 2024-09-16T20:49:41.020 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.083813433000000007 2024-09-16T20:49:41.020 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.020 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.021 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:49:41.021 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222163+0000", 2024-09-16T20:49:41.021 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015274599999999999 2024-09-16T20:49:41.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.021 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.021 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:49:41.021 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222197+0000", 2024-09-16T20:49:41.021 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3683000000000003e-05 2024-09-16T20:49:41.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.021 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.021 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:41.021 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222220+0000", 2024-09-16T20:49:41.021 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3433000000000001e-05 2024-09-16T20:49:41.022 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:41.022 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:41.022 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:41.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.022 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.022 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.5089.0:11702 3.15 e73/64 3:acf33c8b:::rbd_data.110e747f0779.000000000000001a:head v 73'132, mlcod=73'130)", 2024-09-16T20:49:41.022 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:49:41.022 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.798406999000001, 2024-09-16T20:49:41.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.084079763000000002, 2024-09-16T20:49:41.022 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:41.022 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:41.022 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:41.023 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:41.023 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.023 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:41.023 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:49:41.023 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:41.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.023 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.023 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:41.023 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:49:41.023 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:41.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.023 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.023 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:41.024 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:49:41.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:49:41.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.024 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:41.024 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138263+0000", 2024-09-16T20:49:41.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1e-06 2024-09-16T20:49:41.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.024 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:41.024 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221589+0000", 2024-09-16T20:49:41.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.083326002999999996 2024-09-16T20:49:41.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:41.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221614+0000", 2024-09-16T20:49:41.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5593999999999999e-05 2024-09-16T20:49:41.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:41.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221714+0000", 2024-09-16T20:49:41.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9889000000000005e-05 2024-09-16T20:49:41.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:41.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221727+0000", 2024-09-16T20:49:41.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3125999999999999e-05 2024-09-16T20:49:41.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.026 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:49:41.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222294+0000", 2024-09-16T20:49:41.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00056712199999999998 2024-09-16T20:49:41.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.026 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:41.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222336+0000", 2024-09-16T20:49:41.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1470000000000001e-05 2024-09-16T20:49:41.026 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:41.026 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:41.026 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:41.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.027 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.027 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5138.0:9358 3.c 3:33996a43:::rbd_data.110e6372565d.0000000000000007:head [read 1343488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T20:49:41.027 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:49:41.027 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 13.084131589, 2024-09-16T20:49:41.027 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0167493569999999, 2024-09-16T20:49:41.027 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:41.027 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:41.027 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:41.027 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:41.027 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5138", 2024-09-16T20:49:41.027 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:49:41.027 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9358 2024-09-16T20:49:41.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.027 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:41.027 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.027 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:41.028 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:49:41.028 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:41.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.028 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.028 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:41.028 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:49:41.028 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:41.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.028 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.028 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:41.028 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852533+0000", 2024-09-16T20:49:41.028 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.646e-06 2024-09-16T20:49:41.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.028 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.029 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:41.029 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852539+0000", 2024-09-16T20:49:41.029 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3389999999999998e-06 2024-09-16T20:49:41.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.029 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:41.029 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852552+0000", 2024-09-16T20:49:41.029 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.271e-05 2024-09-16T20:49:41.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.029 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:41.029 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852563+0000", 2024-09-16T20:49:41.029 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1139e-05 2024-09-16T20:49:41.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.030 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:41.030 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852883+0000", 2024-09-16T20:49:41.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00032010600000000001 2024-09-16T20:49:41.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.030 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:49:41.030 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852890+0000", 2024-09-16T20:49:41.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7669999999999998e-06 2024-09-16T20:49:41.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.030 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:41.030 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869059+0000", 2024-09-16T20:49:41.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.016169353 2024-09-16T20:49:41.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.031 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.031 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:41.031 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869178+0000", 2024-09-16T20:49:41.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118193 2024-09-16T20:49:41.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.031 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.031 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:41.031 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869281+0000", 2024-09-16T20:49:41.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010310399999999999 2024-09-16T20:49:41.031 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:41.031 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:41.031 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:41.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.032 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5138.0:9382 3.c 3:33996a43:::rbd_data.110e6372565d.0000000000000007:head [read 0~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T20:49:41.032 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:49:41.032 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 13.084080712, 2024-09-16T20:49:41.032 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.017136034, 2024-09-16T20:49:41.032 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:41.032 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:41.032 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:41.032 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:41.032 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5138", 2024-09-16T20:49:41.032 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:49:41.032 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9382 2024-09-16T20:49:41.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.032 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:41.033 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.033 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:41.033 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:49:41.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:41.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.033 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.033 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:41.033 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:49:41.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:41.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.033 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.033 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:41.033 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852583+0000", 2024-09-16T20:49:41.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.51e-07 2024-09-16T20:49:41.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.034 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.034 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:41.034 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852586+0000", 2024-09-16T20:49:41.034 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0280000000000001e-06 2024-09-16T20:49:41.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.034 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.034 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:41.034 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852595+0000", 2024-09-16T20:49:41.034 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2799999999999992e-06 2024-09-16T20:49:41.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.034 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.034 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:41.034 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852601+0000", 2024-09-16T20:49:41.034 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3810000000000004e-06 2024-09-16T20:49:41.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.035 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:41.035 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852921+0000", 2024-09-16T20:49:41.035 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00031979699999999997 2024-09-16T20:49:41.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.035 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:49:41.035 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852930+0000", 2024-09-16T20:49:41.035 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6489999999999997e-06 2024-09-16T20:49:41.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.035 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:41.036 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869434+0000", 2024-09-16T20:49:41.036 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0165038829999999 2024-09-16T20:49:41.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.036 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:41.036 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869495+0000", 2024-09-16T20:49:41.036 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1509999999999999e-05 2024-09-16T20:49:41.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.036 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:41.036 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869718+0000", 2024-09-16T20:49:41.036 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000222955 2024-09-16T20:49:41.036 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:41.037 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:41.037 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:41.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.037 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.037 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5425.0:2601 3.1e 3:7dc19323:::rbd_data.110e6372565d.0000000000000026:1d [read 1171456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-16T20:49:41.037 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:31.967612+0000", 2024-09-16T20:49:41.037 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.9690506899999995, 2024-09-16T20:49:41.037 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048708179999999997, 2024-09-16T20:49:41.037 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:41.037 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:41.038 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:41.038 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:41.038 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5425", 2024-09-16T20:49:41.038 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1974102553", 2024-09-16T20:49:41.038 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 2601 2024-09-16T20:49:41.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.038 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:41.038 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.038 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:41.038 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:31.967612+0000", 2024-09-16T20:49:41.038 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:41.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.038 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.038 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:41.039 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:31.967612+0000", 2024-09-16T20:49:41.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:41.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.039 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:41.039 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:31.967842+0000", 2024-09-16T20:49:41.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00022931899999999999 2024-09-16T20:49:41.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.039 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:41.040 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:31.967845+0000", 2024-09-16T20:49:41.040 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2320000000000001e-06 2024-09-16T20:49:41.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.040 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.040 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:41.040 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:31.967854+0000", 2024-09-16T20:49:41.040 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1849999999999993e-06 2024-09-16T20:49:41.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.040 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.040 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:41.040 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:31.967864+0000", 2024-09-16T20:49:41.040 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6639999999999993e-06 2024-09-16T20:49:41.041 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.041 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.041 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:41.041 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:32.016212+0000", 2024-09-16T20:49:41.041 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048348820000000001 2024-09-16T20:49:41.041 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.041 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.041 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:41.041 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:32.016257+0000", 2024-09-16T20:49:41.041 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4674e-05 2024-09-16T20:49:41.041 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.041 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.041 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:41.042 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:32.016320+0000", 2024-09-16T20:49:41.042 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3286000000000004e-05 2024-09-16T20:49:41.042 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:41.042 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:41.042 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:41.042 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:41.042 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:41.042 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:41.044 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:49:41.046 INFO:teuthology.orchestra.run.smithi139.stdout:3998 punch from 0x1fa3e00 to 0x1fa7c00, (0x3e00 bytes) 2024-09-16T20:49:41.046 INFO:teuthology.orchestra.run.smithi139.stdout:3999 read 0x379c600 thru 0x37ab03f (0xea40 bytes) 2024-09-16T20:49:41.046 INFO:teuthology.orchestra.run.smithi139.stdout:4004 read 0x5073c00 thru 0x5081f61 (0xe362 bytes) 2024-09-16T20:49:41.046 INFO:teuthology.orchestra.run.smithi139.stdout:4005 read 0x4292000 thru 0x4296e70 (0x4e71 bytes) 2024-09-16T20:49:41.047 INFO:teuthology.orchestra.run.smithi139.stdout:4006 trunc from 0x832ca00 to 0x326a600 2024-09-16T20:49:41.047 INFO:teuthology.orchestra.run.smithi139.stdout:4007 punch from 0x22ec400 to 0x22efa00, (0x3600 bytes) 2024-09-16T20:49:41.047 INFO:teuthology.orchestra.run.smithi139.stdout:4008 write 0x5874a00 thru 0x588379b (0xed9c bytes) 2024-09-16T20:49:41.047 INFO:teuthology.orchestra.run.smithi139.stdout:4009 write 0x66e7800 thru 0x66f5d3e (0xe53f bytes) 2024-09-16T20:49:41.047 INFO:teuthology.orchestra.run.smithi139.stdout:4010 write 0xb738e00 thru 0xb743124 (0xa325 bytes) 2024-09-16T20:49:41.047 INFO:teuthology.orchestra.run.smithi139.stdout:4011 read 0x8f5da00 thru 0x8f6997c (0xbf7d bytes) 2024-09-16T20:49:41.047 INFO:teuthology.orchestra.run.smithi139.stdout:4012 read 0x3baa800 thru 0x3bb15b5 (0x6db6 bytes) 2024-09-16T20:49:41.047 INFO:teuthology.orchestra.run.smithi139.stdout:4013 read 0x3ccc400 thru 0x3cd4226 (0x7e27 bytes) 2024-09-16T20:49:41.047 INFO:teuthology.orchestra.run.smithi139.stdout:4016 punch from 0x7b74200 to 0x7b7bc00, (0x7a00 bytes) 2024-09-16T20:49:41.047 INFO:teuthology.orchestra.run.smithi139.stdout:4017 read 0x818f000 thru 0x8193de9 (0x4dea bytes) 2024-09-16T20:49:41.047 INFO:teuthology.orchestra.run.smithi139.stdout:4020 read 0x13a3000 thru 0x13a5e53 (0x2e54 bytes) 2024-09-16T20:49:41.047 INFO:teuthology.orchestra.run.smithi139.stdout:4022 write 0x62c8000 thru 0x62c8393 (0x394 bytes) 2024-09-16T20:49:41.047 INFO:teuthology.orchestra.run.smithi139.stdout:4023 read 0x941c00 thru 0x94eb18 (0xcf19 bytes) 2024-09-16T20:49:41.047 INFO:teuthology.orchestra.run.smithi139.stdout:4025 read 0x30d8e00 thru 0x30dc45f (0x3660 bytes) 2024-09-16T20:49:41.048 INFO:teuthology.orchestra.run.smithi139.stdout:4027 write 0x1717a00 thru 0x172623d (0xe83e bytes) 2024-09-16T20:49:41.048 INFO:teuthology.orchestra.run.smithi139.stdout:4028 punch from 0x88d7a00 to 0x88d8000, (0x600 bytes) 2024-09-16T20:49:41.048 INFO:teuthology.orchestra.run.smithi139.stdout:4029 trunc from 0xb743200 to 0x572f200 2024-09-16T20:49:41.048 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:41.042+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:41.053 INFO:teuthology.orchestra.run.smithi139.stdout:4030 read 0x315a000 thru 0x3165dd9 (0xbdda bytes) 2024-09-16T20:49:41.071 INFO:teuthology.orchestra.run.smithi139.stdout:4035 write 0x61ca400 thru 0x61d8979 (0xe57a bytes) 2024-09-16T20:49:41.100 INFO:teuthology.orchestra.run.smithi139.stdout:4036 write 0xb5b9000 thru 0xb5c4c75 (0xbc76 bytes) 2024-09-16T20:49:41.101 INFO:teuthology.orchestra.run.smithi139.stdout:4037 punch from 0x76a5200 to 0x76ad400, (0x8200 bytes) 2024-09-16T20:49:41.102 INFO:teuthology.orchestra.run.smithi139.stdout:4038 clone 31 order 23 su 4194304 sc 4 2024-09-16T20:49:41.144 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:41.143+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:41.160 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:41.160 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:49:41.161 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:49:41.161 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:41.169 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:49:41.246 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:41.244+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:41.316 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:41.317 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:49:41.317 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:49:41.317 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:49:41.317 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:41.325 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:49:41.368 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:41.367+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:41.433 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:41.433 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:49:41.433 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:49:41.433 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:49:41.434 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.434 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:4904 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 7309312~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:41.434 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767724+0000", 2024-09-16T20:49:41.434 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.663966716999994, 2024-09-16T20:49:41.434 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050270214000000001, 2024-09-16T20:49:41.434 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:41.434 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:41.434 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:41.434 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:41.434 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:41.434 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:41.434 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4904 2024-09-16T20:49:41.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.434 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:41.435 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.435 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:41.435 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767724+0000", 2024-09-16T20:49:41.435 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:41.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.435 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.435 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:41.435 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767724+0000", 2024-09-16T20:49:41.435 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:41.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.435 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.435 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:41.435 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767724+0000", 2024-09-16T20:49:41.435 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1399999999999998e-07 2024-09-16T20:49:41.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.436 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.436 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:41.436 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767725+0000", 2024-09-16T20:49:41.436 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0380000000000001e-06 2024-09-16T20:49:41.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.436 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.436 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:41.436 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767729+0000", 2024-09-16T20:49:41.436 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2490000000000001e-06 2024-09-16T20:49:41.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.436 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.436 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:41.436 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767731+0000", 2024-09-16T20:49:41.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5940000000000001e-06 2024-09-16T20:49:41.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.437 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:41.437 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.817787+0000", 2024-09-16T20:49:41.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050055792000000002 2024-09-16T20:49:41.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.437 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:41.437 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.817883+0000", 2024-09-16T20:49:41.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5606999999999995e-05 2024-09-16T20:49:41.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:41.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.817994+0000", 2024-09-16T20:49:41.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011162000000000001 2024-09-16T20:49:41.438 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:41.438 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:41.438 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:41.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.438 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:4906 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 7317504~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:41.438 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767734+0000", 2024-09-16T20:49:41.438 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.663956454000001, 2024-09-16T20:49:41.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050406608999999998, 2024-09-16T20:49:41.439 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:41.439 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:41.439 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:41.439 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:41.439 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:41.439 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:41.439 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4906 2024-09-16T20:49:41.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.439 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:41.439 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.439 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:41.439 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767734+0000", 2024-09-16T20:49:41.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:41.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.440 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:41.440 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767734+0000", 2024-09-16T20:49:41.440 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:41.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.440 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:41.440 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767735+0000", 2024-09-16T20:49:41.440 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.05e-07 2024-09-16T20:49:41.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:41.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767736+0000", 2024-09-16T20:49:41.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0240000000000001e-06 2024-09-16T20:49:41.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:41.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767739+0000", 2024-09-16T20:49:41.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.067e-06 2024-09-16T20:49:41.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:41.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767742+0000", 2024-09-16T20:49:41.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.23e-06 2024-09-16T20:49:41.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:41.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818027+0000", 2024-09-16T20:49:41.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050285102999999998 2024-09-16T20:49:41.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:41.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818072+0000", 2024-09-16T20:49:41.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5129000000000001e-05 2024-09-16T20:49:41.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:41.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818141+0000", 2024-09-16T20:49:41.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8850999999999995e-05 2024-09-16T20:49:41.442 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:41.443 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:41.443 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:41.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.443 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:4908 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 7325696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:41.443 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767745+0000", 2024-09-16T20:49:41.443 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.663946058999997, 2024-09-16T20:49:41.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050654230000000001, 2024-09-16T20:49:41.443 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:41.443 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:41.443 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:41.443 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:41.443 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:41.443 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:41.443 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4908 2024-09-16T20:49:41.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.444 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:41.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:41.444 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767745+0000", 2024-09-16T20:49:41.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:41.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:41.444 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767745+0000", 2024-09-16T20:49:41.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:41.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:41.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767745+0000", 2024-09-16T20:49:41.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1e-07 2024-09-16T20:49:41.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:41.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767749+0000", 2024-09-16T20:49:41.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.532e-06 2024-09-16T20:49:41.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:41.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767753+0000", 2024-09-16T20:49:41.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4939999999999999e-06 2024-09-16T20:49:41.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.446 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:41.446 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767756+0000", 2024-09-16T20:49:41.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.61e-06 2024-09-16T20:49:41.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.446 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:41.446 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818184+0000", 2024-09-16T20:49:41.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050428615000000003 2024-09-16T20:49:41.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.446 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:41.446 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818276+0000", 2024-09-16T20:49:41.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2077999999999995e-05 2024-09-16T20:49:41.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:41.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818399+0000", 2024-09-16T20:49:41.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012269099999999999 2024-09-16T20:49:41.448 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:41.448 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:41.448 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:41.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.448 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:4910 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 7333888~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:41.449 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767759+0000", 2024-09-16T20:49:41.449 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.663932169000006, 2024-09-16T20:49:41.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050815076000000001, 2024-09-16T20:49:41.449 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:41.449 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:41.449 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:41.449 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:41.449 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:41.449 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:41.449 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4910 2024-09-16T20:49:41.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.450 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:41.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.450 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:41.450 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767759+0000", 2024-09-16T20:49:41.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:41.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.450 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:41.450 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767759+0000", 2024-09-16T20:49:41.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:41.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.450 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:41.450 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767759+0000", 2024-09-16T20:49:41.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2600000000000001e-07 2024-09-16T20:49:41.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.451 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:41.451 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767760+0000", 2024-09-16T20:49:41.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0640000000000001e-06 2024-09-16T20:49:41.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.451 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:41.451 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767763+0000", 2024-09-16T20:49:41.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.994e-06 2024-09-16T20:49:41.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.451 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:41.451 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767765+0000", 2024-09-16T20:49:41.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5220000000000002e-06 2024-09-16T20:49:41.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:41.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818438+0000", 2024-09-16T20:49:41.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050672602999999997 2024-09-16T20:49:41.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:41.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818481+0000", 2024-09-16T20:49:41.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2967999999999999e-05 2024-09-16T20:49:41.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:41.453 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818574+0000", 2024-09-16T20:49:41.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2699000000000001e-05 2024-09-16T20:49:41.453 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:41.453 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:41.453 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:41.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.453 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5012 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2101248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:41.453 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767768+0000", 2024-09-16T20:49:41.453 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.66392252, 2024-09-16T20:49:41.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051084682999999999, 2024-09-16T20:49:41.453 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:41.453 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:41.454 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:41.454 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:41.454 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:41.454 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:41.454 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5012 2024-09-16T20:49:41.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.454 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:41.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:41.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767768+0000", 2024-09-16T20:49:41.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:41.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:41.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767768+0000", 2024-09-16T20:49:41.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:41.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:41.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767769+0000", 2024-09-16T20:49:41.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1500000000000001e-07 2024-09-16T20:49:41.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:41.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767770+0000", 2024-09-16T20:49:41.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.015e-06 2024-09-16T20:49:41.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:41.456 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767773+0000", 2024-09-16T20:49:41.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1070000000000001e-06 2024-09-16T20:49:41.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.456 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:41.456 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767775+0000", 2024-09-16T20:49:41.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.508e-06 2024-09-16T20:49:41.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.456 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:41.456 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818630+0000", 2024-09-16T20:49:41.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050855282000000002 2024-09-16T20:49:41.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.456 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:41.457 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818732+0000", 2024-09-16T20:49:41.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101531 2024-09-16T20:49:41.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.457 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:41.457 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818853+0000", 2024-09-16T20:49:41.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000121025 2024-09-16T20:49:41.457 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:41.457 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:41.457 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:41.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.457 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5014 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2109440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:41.458 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767778+0000", 2024-09-16T20:49:41.458 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.663912584000002, 2024-09-16T20:49:41.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051260776000000001, 2024-09-16T20:49:41.458 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:41.458 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:41.458 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:41.458 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:41.458 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:41.458 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:41.458 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5014 2024-09-16T20:49:41.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.458 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:41.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:41.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767778+0000", 2024-09-16T20:49:41.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:41.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:41.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767778+0000", 2024-09-16T20:49:41.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:41.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:41.460 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767824+0000", 2024-09-16T20:49:41.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5890999999999998e-05 2024-09-16T20:49:41.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:41.460 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767827+0000", 2024-09-16T20:49:41.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.644e-06 2024-09-16T20:49:41.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:41.460 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767832+0000", 2024-09-16T20:49:41.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2689999999999999e-06 2024-09-16T20:49:41.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:41.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767835+0000", 2024-09-16T20:49:41.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2100000000000002e-06 2024-09-16T20:49:41.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:41.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818887+0000", 2024-09-16T20:49:41.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051051276999999999 2024-09-16T20:49:41.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:41.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818953+0000", 2024-09-16T20:49:41.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6105999999999994e-05 2024-09-16T20:49:41.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:41.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819039+0000", 2024-09-16T20:49:41.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6378999999999994e-05 2024-09-16T20:49:41.462 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:41.462 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:41.462 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:41.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.462 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5016 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2117632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:41.462 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767841+0000", 2024-09-16T20:49:41.462 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.663849905000006, 2024-09-16T20:49:41.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051433288000000001, 2024-09-16T20:49:41.462 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:41.462 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:41.462 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:41.462 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:41.463 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:41.463 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:41.463 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5016 2024-09-16T20:49:41.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.463 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:41.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.463 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:41.463 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767841+0000", 2024-09-16T20:49:41.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:41.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.463 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:41.463 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767841+0000", 2024-09-16T20:49:41.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:41.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.464 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:41.464 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768156+0000", 2024-09-16T20:49:41.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00031505699999999999 2024-09-16T20:49:41.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.464 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:41.464 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768159+0000", 2024-09-16T20:49:41.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1659999999999998e-06 2024-09-16T20:49:41.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.464 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:41.465 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768166+0000", 2024-09-16T20:49:41.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0550000000000002e-06 2024-09-16T20:49:41.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.465 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:41.465 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768171+0000", 2024-09-16T20:49:41.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3170000000000003e-06 2024-09-16T20:49:41.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.465 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:41.465 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819084+0000", 2024-09-16T20:49:41.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050913355 2024-09-16T20:49:41.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:41.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819170+0000", 2024-09-16T20:49:41.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6328000000000004e-05 2024-09-16T20:49:41.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:41.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819274+0000", 2024-09-16T20:49:41.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010401 2024-09-16T20:49:41.467 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:41.467 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:41.467 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:41.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.467 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5018 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2125824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:41.467 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768178+0000", 2024-09-16T20:49:41.467 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.663512370999996, 2024-09-16T20:49:41.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051227013000000002, 2024-09-16T20:49:41.468 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:41.468 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:41.468 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:41.468 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:41.468 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:41.468 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:41.468 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5018 2024-09-16T20:49:41.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.468 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:41.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.469 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:41.469 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768178+0000", 2024-09-16T20:49:41.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:41.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.469 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:41.469 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768178+0000", 2024-09-16T20:49:41.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:41.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.469 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:41.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768179+0000", 2024-09-16T20:49:41.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9200000000000002e-07 2024-09-16T20:49:41.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:41.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768180+0000", 2024-09-16T20:49:41.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3790000000000001e-06 2024-09-16T20:49:41.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:41.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768184+0000", 2024-09-16T20:49:41.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5470000000000001e-06 2024-09-16T20:49:41.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:41.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768187+0000", 2024-09-16T20:49:41.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8119999999999999e-06 2024-09-16T20:49:41.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:41.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819305+0000", 2024-09-16T20:49:41.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051118683999999998 2024-09-16T20:49:41.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:41.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819351+0000", 2024-09-16T20:49:41.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5757999999999998e-05 2024-09-16T20:49:41.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:41.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819405+0000", 2024-09-16T20:49:41.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4441000000000003e-05 2024-09-16T20:49:41.472 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:41.472 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:41.472 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:41.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.473 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5020 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2134016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:41.473 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768190+0000", 2024-09-16T20:49:41.473 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.663500693000003, 2024-09-16T20:49:41.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051449661000000001, 2024-09-16T20:49:41.473 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:41.473 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:41.473 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:41.473 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:41.473 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:41.473 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:41.473 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5020 2024-09-16T20:49:41.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.474 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:41.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:41.474 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768190+0000", 2024-09-16T20:49:41.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:41.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:41.474 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768190+0000", 2024-09-16T20:49:41.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:41.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:41.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768282+0000", 2024-09-16T20:49:41.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2263999999999996e-05 2024-09-16T20:49:41.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:41.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768286+0000", 2024-09-16T20:49:41.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.529e-06 2024-09-16T20:49:41.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:41.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768294+0000", 2024-09-16T20:49:41.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6990000000000003e-06 2024-09-16T20:49:41.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.476 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:41.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768306+0000", 2024-09-16T20:49:41.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2112e-05 2024-09-16T20:49:41.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.476 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:41.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819449+0000", 2024-09-16T20:49:41.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051143521999999997 2024-09-16T20:49:41.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:41.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819539+0000", 2024-09-16T20:49:41.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0022e-05 2024-09-16T20:49:41.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:41.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819640+0000", 2024-09-16T20:49:41.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000100513 2024-09-16T20:49:41.477 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:41.477 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:41.477 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:41.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.478 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5022 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2142208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:41.478 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768326+0000", 2024-09-16T20:49:41.478 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.663365303999996, 2024-09-16T20:49:41.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051436952000000001, 2024-09-16T20:49:41.478 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:41.478 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:41.478 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:41.478 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:41.478 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:41.478 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:41.479 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5022 2024-09-16T20:49:41.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.479 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:41.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:41.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768326+0000", 2024-09-16T20:49:41.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:41.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:41.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768326+0000", 2024-09-16T20:49:41.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:41.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:41.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768477+0000", 2024-09-16T20:49:41.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015109300000000001 2024-09-16T20:49:41.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:41.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768480+0000", 2024-09-16T20:49:41.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8580000000000002e-06 2024-09-16T20:49:41.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.481 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:41.481 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768489+0000", 2024-09-16T20:49:41.481 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4600000000000003e-06 2024-09-16T20:49:41.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.481 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:41.481 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768493+0000", 2024-09-16T20:49:41.481 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2270000000000004e-06 2024-09-16T20:49:41.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.481 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:41.481 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819672+0000", 2024-09-16T20:49:41.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051178605000000002 2024-09-16T20:49:41.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.482 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:41.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819711+0000", 2024-09-16T20:49:41.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9357000000000003e-05 2024-09-16T20:49:41.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.482 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:41.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819762+0000", 2024-09-16T20:49:41.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1351999999999999e-05 2024-09-16T20:49:41.482 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:41.483 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:41.483 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:41.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.483 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5024 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2150400~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:41.483 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768501+0000", 2024-09-16T20:49:41.483 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.663189826000007, 2024-09-16T20:49:41.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051402678, 2024-09-16T20:49:41.483 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:41.483 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:41.483 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:41.483 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:41.483 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:41.484 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:41.484 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5024 2024-09-16T20:49:41.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.484 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:41.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:41.484 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768501+0000", 2024-09-16T20:49:41.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:41.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:41.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768501+0000", 2024-09-16T20:49:41.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:41.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.485 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:41.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768619+0000", 2024-09-16T20:49:41.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118187 2024-09-16T20:49:41.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.485 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:41.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768624+0000", 2024-09-16T20:49:41.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3259999999999997e-06 2024-09-16T20:49:41.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.486 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:41.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768631+0000", 2024-09-16T20:49:41.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6490000000000008e-06 2024-09-16T20:49:41.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.486 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:41.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768637+0000", 2024-09-16T20:49:41.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4210000000000003e-06 2024-09-16T20:49:41.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.486 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:41.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819800+0000", 2024-09-16T20:49:41.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051163486000000001 2024-09-16T20:49:41.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:41.487 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819845+0000", 2024-09-16T20:49:41.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4830999999999998e-05 2024-09-16T20:49:41.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:41.487 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819904+0000", 2024-09-16T20:49:41.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8777999999999999e-05 2024-09-16T20:49:41.487 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:41.488 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:41.488 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:41.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.488 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5026 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2158592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:41.488 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768646+0000", 2024-09-16T20:49:41.488 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.663044377999995, 2024-09-16T20:49:41.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051384386999999997, 2024-09-16T20:49:41.488 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:41.488 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:41.488 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:41.488 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:41.489 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:41.489 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:41.489 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5026 2024-09-16T20:49:41.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.489 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:41.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:41.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768646+0000", 2024-09-16T20:49:41.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:41.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:41.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768646+0000", 2024-09-16T20:49:41.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:41.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:41.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768775+0000", 2024-09-16T20:49:41.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012813200000000001 2024-09-16T20:49:41.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:41.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768778+0000", 2024-09-16T20:49:41.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2150000000000001e-06 2024-09-16T20:49:41.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:41.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768786+0000", 2024-09-16T20:49:41.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3259999999999996e-06 2024-09-16T20:49:41.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:41.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768790+0000", 2024-09-16T20:49:41.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.216e-06 2024-09-16T20:49:41.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:41.492 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819939+0000", 2024-09-16T20:49:41.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051149018999999997 2024-09-16T20:49:41.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.492 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:41.492 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819979+0000", 2024-09-16T20:49:41.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9311999999999999e-05 2024-09-16T20:49:41.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.492 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:41.492 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820031+0000", 2024-09-16T20:49:41.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2166999999999997e-05 2024-09-16T20:49:41.492 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:41.493 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:41.493 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:41.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.493 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5028 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2166784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:41.493 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768798+0000", 2024-09-16T20:49:41.493 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.662892735, 2024-09-16T20:49:41.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051388912000000002, 2024-09-16T20:49:41.493 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:41.493 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:41.493 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:41.493 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:41.494 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:41.494 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:41.494 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5028 2024-09-16T20:49:41.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.494 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:41.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.494 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:41.494 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768798+0000", 2024-09-16T20:49:41.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:41.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.494 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:41.495 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768798+0000", 2024-09-16T20:49:41.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:41.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:41.495 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768989+0000", 2024-09-16T20:49:41.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000190951 2024-09-16T20:49:41.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:41.495 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768993+0000", 2024-09-16T20:49:41.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9770000000000002e-06 2024-09-16T20:49:41.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:41.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.769002+0000", 2024-09-16T20:49:41.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5469999999999997e-06 2024-09-16T20:49:41.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:41.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.769020+0000", 2024-09-16T20:49:41.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8216999999999999e-05 2024-09-16T20:49:41.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:41.497 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820070+0000", 2024-09-16T20:49:41.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051049998999999999 2024-09-16T20:49:41.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:41.497 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820119+0000", 2024-09-16T20:49:41.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8962000000000003e-05 2024-09-16T20:49:41.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.498 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:41.498 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820187+0000", 2024-09-16T20:49:41.498 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8258999999999998e-05 2024-09-16T20:49:41.498 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:41.498 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:41.498 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:41.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.498 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5030 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2174976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:41.498 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.769031+0000", 2024-09-16T20:49:41.499 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.662659622000007, 2024-09-16T20:49:41.499 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051289830000000002, 2024-09-16T20:49:41.499 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:41.499 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:41.499 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:41.499 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:41.499 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:41.499 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:41.499 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5030 2024-09-16T20:49:41.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.499 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:41.499 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.500 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:41.500 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.769031+0000", 2024-09-16T20:49:41.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:41.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.500 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:41.500 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.769031+0000", 2024-09-16T20:49:41.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:41.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:41.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.773332+0000", 2024-09-16T20:49:41.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0043004039999999999 2024-09-16T20:49:41.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:41.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.773336+0000", 2024-09-16T20:49:41.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7849999999999999e-06 2024-09-16T20:49:41.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:41.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.773347+0000", 2024-09-16T20:49:41.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0482999999999999e-05 2024-09-16T20:49:41.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.502 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:41.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.773354+0000", 2024-09-16T20:49:41.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1559999999999998e-06 2024-09-16T20:49:41.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.502 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:41.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820216+0000", 2024-09-16T20:49:41.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046861591000000001 2024-09-16T20:49:41.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:41.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820258+0000", 2024-09-16T20:49:41.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2653e-05 2024-09-16T20:49:41.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:41.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820321+0000", 2024-09-16T20:49:41.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2757999999999998e-05 2024-09-16T20:49:41.503 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:41.503 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:41.504 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:41.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.504 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5067 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 6918144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:41.504 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.775279+0000", 2024-09-16T20:49:41.504 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.656411356000007, 2024-09-16T20:49:41.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050171387999999997, 2024-09-16T20:49:41.504 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:41.504 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:41.504 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:41.504 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:41.505 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:41.505 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:41.505 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5067 2024-09-16T20:49:41.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.505 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:41.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:41.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775279+0000", 2024-09-16T20:49:41.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:41.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:41.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775279+0000", 2024-09-16T20:49:41.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:41.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:41.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775432+0000", 2024-09-16T20:49:41.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015257399999999999 2024-09-16T20:49:41.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:41.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775436+0000", 2024-09-16T20:49:41.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0180000000000003e-06 2024-09-16T20:49:41.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.507 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.507 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:41.507 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775447+0000", 2024-09-16T20:49:41.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.077e-05 2024-09-16T20:49:41.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.507 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.507 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:41.507 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775456+0000", 2024-09-16T20:49:41.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4089999999999992e-06 2024-09-16T20:49:41.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.508 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:41.508 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825150+0000", 2024-09-16T20:49:41.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049693711000000002 2024-09-16T20:49:41.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.508 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:41.508 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825337+0000", 2024-09-16T20:49:41.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000187214 2024-09-16T20:49:41.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:41.509 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825451+0000", 2024-09-16T20:49:41.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011369199999999999 2024-09-16T20:49:41.509 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:41.509 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:41.509 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:41.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.509 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5068 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2330624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:41.509 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.775469+0000", 2024-09-16T20:49:41.510 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.656221395000003, 2024-09-16T20:49:41.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050198334999999997, 2024-09-16T20:49:41.510 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:41.510 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:41.510 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:41.510 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:41.510 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:41.510 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:41.510 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5068 2024-09-16T20:49:41.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.510 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:41.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:41.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775469+0000", 2024-09-16T20:49:41.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:41.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:41.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775469+0000", 2024-09-16T20:49:41.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:41.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.512 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:41.512 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775720+0000", 2024-09-16T20:49:41.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000250661 2024-09-16T20:49:41.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.512 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.512 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:41.512 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775724+0000", 2024-09-16T20:49:41.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8700000000000002e-06 2024-09-16T20:49:41.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.512 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.512 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:41.512 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775733+0000", 2024-09-16T20:49:41.513 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4239999999999999e-06 2024-09-16T20:49:41.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.513 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:41.513 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775740+0000", 2024-09-16T20:49:41.513 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9519999999999997e-06 2024-09-16T20:49:41.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.513 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:41.513 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825484+0000", 2024-09-16T20:49:41.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049743385000000001 2024-09-16T20:49:41.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:41.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825572+0000", 2024-09-16T20:49:41.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7834e-05 2024-09-16T20:49:41.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:41.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825668+0000", 2024-09-16T20:49:41.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6209e-05 2024-09-16T20:49:41.515 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:41.515 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:41.515 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:41.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.515 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.515 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5069 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 6926336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:41.515 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.775753+0000", 2024-09-16T20:49:41.515 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.655938172000006, 2024-09-16T20:49:41.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050146430999999998, 2024-09-16T20:49:41.515 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:41.515 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:41.515 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:41.516 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:41.516 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:41.516 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:41.516 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5069 2024-09-16T20:49:41.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.516 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:41.516 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.516 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:41.516 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775753+0000", 2024-09-16T20:49:41.516 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:41.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.517 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:41.517 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775753+0000", 2024-09-16T20:49:41.517 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:41.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.517 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:41.517 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775865+0000", 2024-09-16T20:49:41.517 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000112402 2024-09-16T20:49:41.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.517 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:41.518 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775869+0000", 2024-09-16T20:49:41.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5130000000000001e-06 2024-09-16T20:49:41.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.518 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:41.518 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775877+0000", 2024-09-16T20:49:41.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1359999999999997e-06 2024-09-16T20:49:41.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.518 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:41.518 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775882+0000", 2024-09-16T20:49:41.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6710000000000004e-06 2024-09-16T20:49:41.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.519 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:41.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825729+0000", 2024-09-16T20:49:41.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049846898000000001 2024-09-16T20:49:41.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.519 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:41.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825807+0000", 2024-09-16T20:49:41.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7875999999999998e-05 2024-09-16T20:49:41.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.520 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:41.520 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825899+0000", 2024-09-16T20:49:41.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1935000000000007e-05 2024-09-16T20:49:41.520 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:41.520 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:41.520 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:41.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.520 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5070 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2338816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:41.520 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.775892+0000", 2024-09-16T20:49:41.520 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.655798317999995, 2024-09-16T20:49:41.521 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050155311000000001, 2024-09-16T20:49:41.521 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:41.521 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:41.521 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:41.521 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:41.521 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:41.521 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:41.521 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5070 2024-09-16T20:49:41.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.521 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:41.521 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.522 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:41.522 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775892+0000", 2024-09-16T20:49:41.522 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:41.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.522 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:41.522 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775892+0000", 2024-09-16T20:49:41.522 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:41.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.522 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:41.522 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776142+0000", 2024-09-16T20:49:41.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00024950500000000002 2024-09-16T20:49:41.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:41.523 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776146+0000", 2024-09-16T20:49:41.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2819999999999998e-06 2024-09-16T20:49:41.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:41.523 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776157+0000", 2024-09-16T20:49:41.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0807e-05 2024-09-16T20:49:41.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.524 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:41.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776165+0000", 2024-09-16T20:49:41.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1300000000000001e-06 2024-09-16T20:49:41.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.524 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:41.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825935+0000", 2024-09-16T20:49:41.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049769300000000002 2024-09-16T20:49:41.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.524 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:41.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825991+0000", 2024-09-16T20:49:41.525 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6177000000000001e-05 2024-09-16T20:49:41.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.525 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.525 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:41.525 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826048+0000", 2024-09-16T20:49:41.525 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.711e-05 2024-09-16T20:49:41.525 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:41.525 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:41.525 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:41.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.525 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.525 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5071 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 6934528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:41.526 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.776177+0000", 2024-09-16T20:49:41.526 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.655513915, 2024-09-16T20:49:41.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050036886000000003, 2024-09-16T20:49:41.526 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:41.526 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:41.526 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:41.526 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:41.526 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:41.526 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:41.526 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5071 2024-09-16T20:49:41.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.526 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:41.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.527 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:41.527 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776177+0000", 2024-09-16T20:49:41.527 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:41.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.527 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:41.527 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776177+0000", 2024-09-16T20:49:41.527 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:41.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.527 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:41.527 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776339+0000", 2024-09-16T20:49:41.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016238099999999999 2024-09-16T20:49:41.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:41.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776343+0000", 2024-09-16T20:49:41.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9670000000000003e-06 2024-09-16T20:49:41.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:41.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776354+0000", 2024-09-16T20:49:41.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0789e-05 2024-09-16T20:49:41.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.529 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:41.529 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776361+0000", 2024-09-16T20:49:41.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5200000000000003e-06 2024-09-16T20:49:41.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.529 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:41.529 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826078+0000", 2024-09-16T20:49:41.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049717513999999997 2024-09-16T20:49:41.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.530 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:41.530 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826141+0000", 2024-09-16T20:49:41.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2446000000000005e-05 2024-09-16T20:49:41.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.530 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:41.530 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826214+0000", 2024-09-16T20:49:41.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3269000000000006e-05 2024-09-16T20:49:41.530 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:41.530 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:41.531 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:41.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.531 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5072 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2347008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:41.531 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.776370+0000", 2024-09-16T20:49:41.531 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.655320353999997, 2024-09-16T20:49:41.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050097653999999998, 2024-09-16T20:49:41.531 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:41.531 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:41.531 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:41.531 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:41.531 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:41.531 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:41.532 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5072 2024-09-16T20:49:41.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.532 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:41.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.532 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:41.532 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776370+0000", 2024-09-16T20:49:41.532 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:41.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.532 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:41.532 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776370+0000", 2024-09-16T20:49:41.532 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:41.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.533 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.533 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:41.533 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776645+0000", 2024-09-16T20:49:41.533 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000274634 2024-09-16T20:49:41.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.533 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.533 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:41.533 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776654+0000", 2024-09-16T20:49:41.533 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9390000000000003e-06 2024-09-16T20:49:41.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.533 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.534 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:41.534 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776665+0000", 2024-09-16T20:49:41.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0524e-05 2024-09-16T20:49:41.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.534 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:41.534 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776671+0000", 2024-09-16T20:49:41.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7390000000000002e-06 2024-09-16T20:49:41.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.534 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:41.534 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826261+0000", 2024-09-16T20:49:41.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049589995999999997 2024-09-16T20:49:41.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:41.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826363+0000", 2024-09-16T20:49:41.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010158999999999999 2024-09-16T20:49:41.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:41.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:41.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:41.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826468+0000", 2024-09-16T20:49:41.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000105232 2024-09-16T20:49:41.535 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:41.535 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:41.535 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:41.535 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:41.536 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:41.536 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:41.537 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:49:41.538 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:41.535+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:41.638 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:41.637+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:41.691 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:41.691 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:49:41.692 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:49:41.692 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:41.700 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:49:41.717 INFO:teuthology.orchestra.run.smithi139.stdout:truncating image image_client.0-clone30 from 0xb5c4e00 (overlap 0x326a600) to 0x2d5fa00 2024-09-16T20:49:41.741 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:41.740+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:41.839 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:41.838+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:41.850 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:41.851 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:49:41.851 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:49:41.851 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:49:41.851 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:41.860 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:49:41.940 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:41.939+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:42.011 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:42.011 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:49:42.011 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:49:42.011 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:49:42.011 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.011 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5286.0:10643 3.17 3.f6273fb7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-16T20:49:42.012 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:49:42.012 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 33.219125402000003, 2024-09-16T20:49:42.012 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.76672347200000002, 2024-09-16T20:49:42.012 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:42.012 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:42.012 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:49:42.012 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:42.012 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5286", 2024-09-16T20:49:42.012 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/4135433989", 2024-09-16T20:49:42.012 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 10643 2024-09-16T20:49:42.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.012 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:42.012 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.012 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:42.013 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:49:42.013 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:42.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.013 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.013 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:42.013 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:49:42.013 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:42.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.013 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.013 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:42.013 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790489+0000", 2024-09-16T20:49:42.013 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5910000000000001e-06 2024-09-16T20:49:42.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.014 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.014 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:42.014 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790493+0000", 2024-09-16T20:49:42.014 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2669999999999998e-06 2024-09-16T20:49:42.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.014 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.014 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:42.014 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790500+0000", 2024-09-16T20:49:42.014 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3780000000000003e-06 2024-09-16T20:49:42.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.014 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.014 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:42.014 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790505+0000", 2024-09-16T20:49:42.014 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1560000000000003e-06 2024-09-16T20:49:42.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.015 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.015 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:42.015 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790559+0000", 2024-09-16T20:49:42.015 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3494e-05 2024-09-16T20:49:42.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.015 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.015 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:49:42.015 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790566+0000", 2024-09-16T20:49:42.015 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7649999999999993e-06 2024-09-16T20:49:42.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.015 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.015 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:42.015 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:09.557198+0000", 2024-09-16T20:49:42.015 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.76663136200000004 2024-09-16T20:49:42.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.016 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.016 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:42.016 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:09.557211+0000", 2024-09-16T20:49:42.016 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3458999999999999e-05 2024-09-16T20:49:42.016 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:42.016 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:42.016 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:42.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.016 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.016 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3870 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1249280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:42.016 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896514+0000", 2024-09-16T20:49:42.016 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.113099439999999, 2024-09-16T20:49:42.017 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046893815999999998, 2024-09-16T20:49:42.017 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:42.017 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:42.017 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:42.017 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:42.017 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:42.017 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:42.017 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3870 2024-09-16T20:49:42.017 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.017 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:42.017 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.017 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:42.017 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896514+0000", 2024-09-16T20:49:42.018 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:42.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.018 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.018 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:42.018 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896514+0000", 2024-09-16T20:49:42.018 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:42.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.018 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.018 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:42.018 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896514+0000", 2024-09-16T20:49:42.018 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9299999999999999e-07 2024-09-16T20:49:42.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.018 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.018 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:42.018 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896521+0000", 2024-09-16T20:49:42.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8199999999999999e-06 2024-09-16T20:49:42.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.019 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:42.019 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896529+0000", 2024-09-16T20:49:42.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0120000000000005e-06 2024-09-16T20:49:42.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.019 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:42.019 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896534+0000", 2024-09-16T20:49:42.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5779999999999998e-06 2024-09-16T20:49:42.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.020 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:42.020 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943286+0000", 2024-09-16T20:49:42.020 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046751514000000001 2024-09-16T20:49:42.020 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.020 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.020 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:42.020 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943343+0000", 2024-09-16T20:49:42.020 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7216000000000002e-05 2024-09-16T20:49:42.020 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.020 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.020 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:42.020 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943407+0000", 2024-09-16T20:49:42.021 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4282999999999994e-05 2024-09-16T20:49:42.021 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:42.021 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:42.021 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:42.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.021 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.021 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3872 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1253376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:42.021 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896561+0000", 2024-09-16T20:49:42.021 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.113052046, 2024-09-16T20:49:42.021 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.04705165, 2024-09-16T20:49:42.021 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:42.021 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:42.021 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:42.021 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:42.021 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:42.022 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:42.022 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3872 2024-09-16T20:49:42.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.022 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:42.022 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.022 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:42.022 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896561+0000", 2024-09-16T20:49:42.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:42.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.022 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.022 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:42.022 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896561+0000", 2024-09-16T20:49:42.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:42.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.022 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.023 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:42.023 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896561+0000", 2024-09-16T20:49:42.023 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4999999999999998e-07 2024-09-16T20:49:42.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.023 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.023 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:42.023 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896563+0000", 2024-09-16T20:49:42.023 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0140000000000001e-06 2024-09-16T20:49:42.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.023 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.023 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:42.023 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896569+0000", 2024-09-16T20:49:42.023 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6450000000000002e-06 2024-09-16T20:49:42.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.024 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:42.024 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896572+0000", 2024-09-16T20:49:42.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.174e-06 2024-09-16T20:49:42.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.024 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:42.024 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943456+0000", 2024-09-16T20:49:42.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046883552000000002 2024-09-16T20:49:42.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.024 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:42.024 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943531+0000", 2024-09-16T20:49:42.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5080000000000006e-05 2024-09-16T20:49:42.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:42.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943613+0000", 2024-09-16T20:49:42.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1835000000000005e-05 2024-09-16T20:49:42.025 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:42.025 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:42.025 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:42.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.025 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3874 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1257472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:42.025 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896577+0000", 2024-09-16T20:49:42.025 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.113036530999999, 2024-09-16T20:49:42.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047187944000000002, 2024-09-16T20:49:42.026 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:42.026 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:42.026 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:42.026 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:42.026 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:42.026 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:42.026 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3874 2024-09-16T20:49:42.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.026 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:42.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.026 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:42.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896577+0000", 2024-09-16T20:49:42.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:42.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.027 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.027 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:42.027 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896577+0000", 2024-09-16T20:49:42.027 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:42.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.027 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.027 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:42.027 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896577+0000", 2024-09-16T20:49:42.027 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1199999999999999e-07 2024-09-16T20:49:42.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.027 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.027 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:42.028 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896578+0000", 2024-09-16T20:49:42.028 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.237e-06 2024-09-16T20:49:42.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.028 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.028 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:42.028 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896583+0000", 2024-09-16T20:49:42.028 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5859999999999998e-06 2024-09-16T20:49:42.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.028 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.028 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:42.028 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896587+0000", 2024-09-16T20:49:42.028 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3100000000000002e-06 2024-09-16T20:49:42.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.029 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:42.029 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943661+0000", 2024-09-16T20:49:42.029 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047073639 2024-09-16T20:49:42.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.029 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:42.029 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943712+0000", 2024-09-16T20:49:42.029 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1190999999999998e-05 2024-09-16T20:49:42.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.029 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:42.029 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943765+0000", 2024-09-16T20:49:42.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2769000000000002e-05 2024-09-16T20:49:42.030 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:42.030 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:42.030 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:42.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.030 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3876 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1261568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:42.030 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896593+0000", 2024-09-16T20:49:42.030 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.113019952999998, 2024-09-16T20:49:42.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047362866000000003, 2024-09-16T20:49:42.030 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:42.030 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:42.030 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:42.030 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:42.031 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:42.031 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:42.031 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3876 2024-09-16T20:49:42.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.031 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:42.031 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.031 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:42.031 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896593+0000", 2024-09-16T20:49:42.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:42.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.031 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.031 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:42.031 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896593+0000", 2024-09-16T20:49:42.032 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:42.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.032 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:42.032 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896594+0000", 2024-09-16T20:49:42.032 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1600000000000002e-07 2024-09-16T20:49:42.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.032 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:42.032 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896595+0000", 2024-09-16T20:49:42.032 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.756e-06 2024-09-16T20:49:42.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.033 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:42.033 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896601+0000", 2024-09-16T20:49:42.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9309999999999996e-06 2024-09-16T20:49:42.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.033 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.033 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:42.033 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896606+0000", 2024-09-16T20:49:42.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1089999999999997e-06 2024-09-16T20:49:42.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.033 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.033 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:42.033 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943790+0000", 2024-09-16T20:49:42.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047183414 2024-09-16T20:49:42.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.034 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.034 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:42.034 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943869+0000", 2024-09-16T20:49:42.034 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9122000000000006e-05 2024-09-16T20:49:42.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.034 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.034 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:42.034 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943956+0000", 2024-09-16T20:49:42.034 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7118000000000002e-05 2024-09-16T20:49:42.034 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:42.034 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:42.034 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:42.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.035 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3879 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1265664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:42.035 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896613+0000", 2024-09-16T20:49:42.035 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.113000311, 2024-09-16T20:49:42.035 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047528204999999997, 2024-09-16T20:49:42.035 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:42.035 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:42.035 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:42.035 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:42.035 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:42.035 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:42.035 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3879 2024-09-16T20:49:42.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.036 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:42.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.036 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:42.036 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896613+0000", 2024-09-16T20:49:42.036 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:42.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.036 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:42.036 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896613+0000", 2024-09-16T20:49:42.036 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:42.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.036 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:42.037 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896613+0000", 2024-09-16T20:49:42.037 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.84e-07 2024-09-16T20:49:42.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.037 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.037 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:42.037 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896615+0000", 2024-09-16T20:49:42.037 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9970000000000001e-06 2024-09-16T20:49:42.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.037 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.037 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:42.037 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896622+0000", 2024-09-16T20:49:42.037 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3500000000000002e-06 2024-09-16T20:49:42.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.038 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.038 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:42.038 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896627+0000", 2024-09-16T20:49:42.038 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2490000000000001e-06 2024-09-16T20:49:42.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.038 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.038 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:42.038 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943996+0000", 2024-09-16T20:49:42.038 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047369175999999999 2024-09-16T20:49:42.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.038 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.038 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:42.038 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944082+0000", 2024-09-16T20:49:42.038 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5859000000000006e-05 2024-09-16T20:49:42.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.039 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:42.039 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944141+0000", 2024-09-16T20:49:42.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9190000000000001e-05 2024-09-16T20:49:42.039 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:42.039 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:42.039 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:42.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.039 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3881 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1269760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:42.039 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896634+0000", 2024-09-16T20:49:42.039 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.112979398, 2024-09-16T20:49:42.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047692834000000003, 2024-09-16T20:49:42.039 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:42.040 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:42.040 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:42.040 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:42.040 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:42.040 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:42.040 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3881 2024-09-16T20:49:42.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.040 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:42.040 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.040 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:42.040 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896634+0000", 2024-09-16T20:49:42.040 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:42.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.040 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.040 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:42.041 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896634+0000", 2024-09-16T20:49:42.041 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:42.041 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.041 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.041 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:42.041 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896634+0000", 2024-09-16T20:49:42.041 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-16T20:49:42.041 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.041 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.041 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:42.041 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896636+0000", 2024-09-16T20:49:42.041 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.216e-06 2024-09-16T20:49:42.042 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.042 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.042 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:42.042 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896640+0000", 2024-09-16T20:49:42.042 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1749999999999998e-06 2024-09-16T20:49:42.042 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.042 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.042 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:42.042 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896643+0000", 2024-09-16T20:49:42.042 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8779999999999998e-06 2024-09-16T20:49:42.042 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.042 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.042 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:42.043 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944178+0000", 2024-09-16T20:49:42.043 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047534633 2024-09-16T20:49:42.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.043 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.043 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:42.043 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944231+0000", 2024-09-16T20:49:42.043 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3338999999999999e-05 2024-09-16T20:49:42.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.043 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.043 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:42.043 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944327+0000", 2024-09-16T20:49:42.043 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5223000000000006e-05 2024-09-16T20:49:42.043 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:42.043 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:42.043 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:42.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.044 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.044 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3882 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1273856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:42.044 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896647+0000", 2024-09-16T20:49:42.044 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.112966236999998, 2024-09-16T20:49:42.044 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047858886000000003, 2024-09-16T20:49:42.044 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:42.044 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:42.044 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:42.044 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:42.044 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:42.044 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:42.044 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3882 2024-09-16T20:49:42.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.045 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:42.045 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.045 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:42.045 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896647+0000", 2024-09-16T20:49:42.045 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:42.045 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.045 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.045 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:42.045 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896647+0000", 2024-09-16T20:49:42.045 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:42.045 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.045 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.045 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:42.045 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896647+0000", 2024-09-16T20:49:42.046 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4400000000000001e-07 2024-09-16T20:49:42.046 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.046 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.046 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:42.046 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896648+0000", 2024-09-16T20:49:42.046 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0499999999999999e-06 2024-09-16T20:49:42.046 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.046 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.046 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:42.046 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896651+0000", 2024-09-16T20:49:42.046 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9809999999999999e-06 2024-09-16T20:49:42.046 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.046 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.046 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:42.047 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896654+0000", 2024-09-16T20:49:42.047 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4439999999999998e-06 2024-09-16T20:49:42.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.047 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.047 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:42.047 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944361+0000", 2024-09-16T20:49:42.047 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047707408 2024-09-16T20:49:42.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.047 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.047 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:42.047 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944405+0000", 2024-09-16T20:49:42.047 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4107000000000003e-05 2024-09-16T20:49:42.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.047 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.048 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:42.048 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944506+0000", 2024-09-16T20:49:42.048 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000100652 2024-09-16T20:49:42.048 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:42.048 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:42.048 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:42.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.048 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.048 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3883 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1277952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:42.048 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896656+0000", 2024-09-16T20:49:42.048 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.112956727, 2024-09-16T20:49:42.048 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047975265000000003, 2024-09-16T20:49:42.048 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:42.049 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:42.049 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:42.049 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:42.049 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:42.049 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:42.049 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3883 2024-09-16T20:49:42.049 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.049 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:42.049 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.049 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:42.049 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896656+0000", 2024-09-16T20:49:42.049 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:42.050 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.050 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.050 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:42.050 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896656+0000", 2024-09-16T20:49:42.050 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:42.050 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.050 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.050 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:42.050 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896657+0000", 2024-09-16T20:49:42.050 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7000000000000001e-07 2024-09-16T20:49:42.050 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.050 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.050 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:42.050 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896658+0000", 2024-09-16T20:49:42.051 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5850000000000001e-06 2024-09-16T20:49:42.051 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.051 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.051 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:42.051 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896664+0000", 2024-09-16T20:49:42.051 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.609e-06 2024-09-16T20:49:42.051 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.051 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.051 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:42.051 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896668+0000", 2024-09-16T20:49:42.051 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.121e-06 2024-09-16T20:49:42.051 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.051 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.051 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:42.051 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944542+0000", 2024-09-16T20:49:42.052 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047873838000000002 2024-09-16T20:49:42.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.052 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.052 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:42.052 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944579+0000", 2024-09-16T20:49:42.052 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6943000000000002e-05 2024-09-16T20:49:42.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.052 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.052 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:42.052 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944632+0000", 2024-09-16T20:49:42.052 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2899000000000003e-05 2024-09-16T20:49:42.052 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:42.052 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:42.052 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:42.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.053 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.053 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3886 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1282048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:42.053 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896674+0000", 2024-09-16T20:49:42.053 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.112938757999999, 2024-09-16T20:49:42.053 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048152421000000001, 2024-09-16T20:49:42.053 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:42.053 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:42.053 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:42.053 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:42.053 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:42.053 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:42.053 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3886 2024-09-16T20:49:42.053 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.053 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:42.054 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.054 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:42.054 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896674+0000", 2024-09-16T20:49:42.054 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:42.054 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.054 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.054 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:42.054 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896674+0000", 2024-09-16T20:49:42.054 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:42.054 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.054 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.054 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:42.054 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896675+0000", 2024-09-16T20:49:42.054 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6100000000000002e-07 2024-09-16T20:49:42.055 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.055 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.055 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:42.055 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896677+0000", 2024-09-16T20:49:42.055 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1270000000000001e-06 2024-09-16T20:49:42.055 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.055 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.055 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:42.055 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896683+0000", 2024-09-16T20:49:42.055 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1970000000000001e-06 2024-09-16T20:49:42.055 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.055 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.055 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:42.055 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896688+0000", 2024-09-16T20:49:42.055 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.605e-06 2024-09-16T20:49:42.056 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.056 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.056 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:42.056 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944676+0000", 2024-09-16T20:49:42.056 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047988391999999998 2024-09-16T20:49:42.056 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.056 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.056 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:42.056 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944763+0000", 2024-09-16T20:49:42.056 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7363999999999999e-05 2024-09-16T20:49:42.056 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.056 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.056 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:42.056 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944827+0000", 2024-09-16T20:49:42.057 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3374999999999995e-05 2024-09-16T20:49:42.057 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:42.057 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:42.057 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:42.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.057 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.057 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3888 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1286144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:42.057 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896693+0000", 2024-09-16T20:49:42.057 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.112919634000001, 2024-09-16T20:49:42.057 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048336443999999999, 2024-09-16T20:49:42.057 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:42.057 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:42.057 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:42.057 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:42.058 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:42.058 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:42.058 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3888 2024-09-16T20:49:42.058 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.058 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:42.058 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.058 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:42.058 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896693+0000", 2024-09-16T20:49:42.058 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:42.058 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.058 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.058 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:42.058 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896693+0000", 2024-09-16T20:49:42.058 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:42.058 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.059 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.059 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:42.059 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896694+0000", 2024-09-16T20:49:42.059 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3200000000000001e-07 2024-09-16T20:49:42.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.059 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.059 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:42.059 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896696+0000", 2024-09-16T20:49:42.059 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8989999999999999e-06 2024-09-16T20:49:42.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.059 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.059 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:42.059 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896702+0000", 2024-09-16T20:49:42.059 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8819999999999998e-06 2024-09-16T20:49:42.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.060 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.060 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:42.060 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896707+0000", 2024-09-16T20:49:42.060 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1529999999999996e-06 2024-09-16T20:49:42.060 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.060 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.060 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:42.060 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944863+0000", 2024-09-16T20:49:42.060 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048156658999999997 2024-09-16T20:49:42.060 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.060 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.060 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:42.060 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944944+0000", 2024-09-16T20:49:42.060 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0090000000000001e-05 2024-09-16T20:49:42.061 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.061 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.061 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:42.061 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945030+0000", 2024-09-16T20:49:42.061 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6428999999999995e-05 2024-09-16T20:49:42.061 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:42.061 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:42.061 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:42.061 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.061 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.061 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3890 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1290240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:42.061 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896713+0000", 2024-09-16T20:49:42.061 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.112899764000002, 2024-09-16T20:49:42.062 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048538382999999997, 2024-09-16T20:49:42.062 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:42.062 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:42.062 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:42.062 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:42.062 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:42.062 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:42.062 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3890 2024-09-16T20:49:42.062 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.062 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:42.062 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.062 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:42.062 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896713+0000", 2024-09-16T20:49:42.062 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:42.062 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.063 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.063 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:42.063 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896713+0000", 2024-09-16T20:49:42.063 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:42.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.063 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.063 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:42.063 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896714+0000", 2024-09-16T20:49:42.063 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2000000000000001e-07 2024-09-16T20:49:42.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.063 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.063 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:42.063 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896716+0000", 2024-09-16T20:49:42.063 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0109999999999999e-06 2024-09-16T20:49:42.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.064 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:42.064 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896720+0000", 2024-09-16T20:49:42.064 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9049999999999999e-06 2024-09-16T20:49:42.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.064 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:42.064 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896722+0000", 2024-09-16T20:49:42.064 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.627e-06 2024-09-16T20:49:42.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.064 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:42.064 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945075+0000", 2024-09-16T20:49:42.064 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048353149999999998 2024-09-16T20:49:42.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.065 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.065 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:42.065 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945157+0000", 2024-09-16T20:49:42.065 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1972999999999996e-05 2024-09-16T20:49:42.065 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.065 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.065 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:42.065 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945252+0000", 2024-09-16T20:49:42.065 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4396999999999998e-05 2024-09-16T20:49:42.065 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:42.065 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:42.065 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:42.065 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.065 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.065 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3893 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1294336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:42.066 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896726+0000", 2024-09-16T20:49:42.066 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.112886907, 2024-09-16T20:49:42.066 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048658686999999999, 2024-09-16T20:49:42.066 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:42.066 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:42.066 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:42.066 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:42.066 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:42.066 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:42.066 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3893 2024-09-16T20:49:42.066 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.066 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:42.066 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.066 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:42.066 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896726+0000", 2024-09-16T20:49:42.067 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:42.067 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.067 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.067 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:42.067 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896726+0000", 2024-09-16T20:49:42.067 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:42.067 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.067 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.067 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:42.067 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896727+0000", 2024-09-16T20:49:42.067 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7899999999999999e-07 2024-09-16T20:49:42.067 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.067 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.067 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:42.068 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896728+0000", 2024-09-16T20:49:42.068 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.68e-06 2024-09-16T20:49:42.068 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.068 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.068 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:42.068 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896734+0000", 2024-09-16T20:49:42.068 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9190000000000004e-06 2024-09-16T20:49:42.068 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.068 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.068 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:42.068 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896739+0000", 2024-09-16T20:49:42.068 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5549999999999996e-06 2024-09-16T20:49:42.068 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.068 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.069 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:42.069 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945271+0000", 2024-09-16T20:49:42.069 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048531795000000003 2024-09-16T20:49:42.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.069 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.069 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:42.069 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945330+0000", 2024-09-16T20:49:42.069 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9386999999999999e-05 2024-09-16T20:49:42.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.069 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.069 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:42.069 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945385+0000", 2024-09-16T20:49:42.070 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4871999999999999e-05 2024-09-16T20:49:42.070 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:42.070 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:42.070 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:42.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.070 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3895 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1298432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:42.070 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896754+0000", 2024-09-16T20:49:42.070 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.112858833000001, 2024-09-16T20:49:42.070 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048754440000000003, 2024-09-16T20:49:42.070 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:42.070 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:42.070 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:42.070 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:42.070 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:42.071 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:42.071 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3895 2024-09-16T20:49:42.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.071 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:42.071 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.071 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:42.071 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896754+0000", 2024-09-16T20:49:42.071 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:42.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.071 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.071 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:42.071 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896754+0000", 2024-09-16T20:49:42.071 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:42.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.072 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.072 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:42.072 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896755+0000", 2024-09-16T20:49:42.072 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5200000000000002e-07 2024-09-16T20:49:42.072 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.072 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.072 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:42.072 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896762+0000", 2024-09-16T20:49:42.072 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8430000000000004e-06 2024-09-16T20:49:42.072 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.072 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.072 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:42.072 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896782+0000", 2024-09-16T20:49:42.072 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0693e-05 2024-09-16T20:49:42.073 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.073 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.073 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:42.073 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896793+0000", 2024-09-16T20:49:42.073 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0294e-05 2024-09-16T20:49:42.073 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.073 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.073 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:42.073 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945418+0000", 2024-09-16T20:49:42.073 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048625738000000002 2024-09-16T20:49:42.073 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.073 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.073 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:42.074 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945457+0000", 2024-09-16T20:49:42.074 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9103999999999997e-05 2024-09-16T20:49:42.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.074 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.074 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:42.074 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945509+0000", 2024-09-16T20:49:42.074 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1316000000000001e-05 2024-09-16T20:49:42.074 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:42.074 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:42.074 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:42.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.074 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.074 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3896 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1302528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:42.074 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896806+0000", 2024-09-16T20:49:42.075 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.112806621000001, 2024-09-16T20:49:42.075 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048831159999999998, 2024-09-16T20:49:42.075 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:42.075 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:42.075 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:42.075 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:42.075 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:42.075 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:42.075 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3896 2024-09-16T20:49:42.075 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.075 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:42.075 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.075 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:42.075 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896806+0000", 2024-09-16T20:49:42.075 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:42.076 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.076 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.076 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:42.076 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896806+0000", 2024-09-16T20:49:42.076 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:42.076 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.076 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.076 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:42.076 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896807+0000", 2024-09-16T20:49:42.076 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3e-07 2024-09-16T20:49:42.076 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.076 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.076 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:42.077 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896811+0000", 2024-09-16T20:49:42.077 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8310000000000003e-06 2024-09-16T20:49:42.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.077 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.077 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:42.077 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896819+0000", 2024-09-16T20:49:42.077 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2470000000000008e-06 2024-09-16T20:49:42.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.077 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.077 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:42.077 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896823+0000", 2024-09-16T20:49:42.077 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7919999999999999e-06 2024-09-16T20:49:42.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.078 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.078 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:42.078 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945535+0000", 2024-09-16T20:49:42.078 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048712137000000003 2024-09-16T20:49:42.078 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.078 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.078 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:42.078 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945571+0000", 2024-09-16T20:49:42.078 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6261999999999999e-05 2024-09-16T20:49:42.078 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.078 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.078 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:42.078 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945638+0000", 2024-09-16T20:49:42.078 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6161000000000006e-05 2024-09-16T20:49:42.078 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:42.079 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:42.079 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:42.079 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.079 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.079 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3899 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1306624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:42.079 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896829+0000", 2024-09-16T20:49:42.079 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.112784420000001, 2024-09-16T20:49:42.079 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049014110999999999, 2024-09-16T20:49:42.079 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:42.079 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:42.079 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:42.079 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:42.079 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:42.079 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:42.080 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3899 2024-09-16T20:49:42.080 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.080 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:42.080 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.080 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:42.080 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896829+0000", 2024-09-16T20:49:42.080 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:42.080 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.080 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.080 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:42.080 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896829+0000", 2024-09-16T20:49:42.080 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:42.080 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.080 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.080 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:42.081 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896829+0000", 2024-09-16T20:49:42.081 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8999999999999998e-07 2024-09-16T20:49:42.081 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.081 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.081 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:42.081 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896831+0000", 2024-09-16T20:49:42.081 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.77e-06 2024-09-16T20:49:42.081 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.081 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.081 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:42.081 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896834+0000", 2024-09-16T20:49:42.081 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4800000000000001e-06 2024-09-16T20:49:42.081 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.081 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.082 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:42.082 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896837+0000", 2024-09-16T20:49:42.082 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8559999999999998e-06 2024-09-16T20:49:42.082 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.082 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.082 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:42.082 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945663+0000", 2024-09-16T20:49:42.083 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048825529999999999 2024-09-16T20:49:42.083 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.083 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.084 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:42.084 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945746+0000", 2024-09-16T20:49:42.084 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3621000000000005e-05 2024-09-16T20:49:42.084 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.084 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.084 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:42.084 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945843+0000", 2024-09-16T20:49:42.084 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6564000000000006e-05 2024-09-16T20:49:42.084 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:42.084 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:42.084 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:42.084 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.084 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.085 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5512.0:450 3.2 3:44e0cffa:::rbd_header.110ef0d7a535:head [watch reconnect cookie 140263298018912 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:49:42.085 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:49:42.085 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.4471024889999997, 2024-09-16T20:49:42.085 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0084581939999999, 2024-09-16T20:49:42.085 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:42.085 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:42.085 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:42.085 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:42.085 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5512", 2024-09-16T20:49:42.085 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:49:42.085 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 450 2024-09-16T20:49:42.085 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.085 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:42.085 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.086 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:42.086 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:49:42.086 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:42.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.086 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.086 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:42.086 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:49:42.086 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:42.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.086 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.086 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:42.086 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562512+0000", 2024-09-16T20:49:42.086 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.009e-06 2024-09-16T20:49:42.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.087 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.087 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:42.087 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562517+0000", 2024-09-16T20:49:42.087 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9339999999999997e-06 2024-09-16T20:49:42.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.087 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.087 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:42.087 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562530+0000", 2024-09-16T20:49:42.087 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3495e-05 2024-09-16T20:49:42.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.087 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.087 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:42.087 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562536+0000", 2024-09-16T20:49:42.087 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5690000000000004e-06 2024-09-16T20:49:42.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.088 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.088 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:42.088 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562591+0000", 2024-09-16T20:49:42.088 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5761999999999999e-05 2024-09-16T20:49:42.088 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.088 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.088 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:49:42.088 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562599+0000", 2024-09-16T20:49:42.088 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9179999999999994e-06 2024-09-16T20:49:42.088 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.088 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.088 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:42.088 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.565971+0000", 2024-09-16T20:49:42.088 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003371546 2024-09-16T20:49:42.088 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.089 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.089 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:49:42.089 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566019+0000", 2024-09-16T20:49:42.089 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7849000000000002e-05 2024-09-16T20:49:42.089 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.089 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.089 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:42.089 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.569382+0000", 2024-09-16T20:49:42.089 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.003362833 2024-09-16T20:49:42.089 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.089 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.089 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:42.089 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.569529+0000", 2024-09-16T20:49:42.090 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000147823 2024-09-16T20:49:42.090 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.090 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.090 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:42.090 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570969+0000", 2024-09-16T20:49:42.090 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001439456 2024-09-16T20:49:42.090 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:42.090 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:42.090 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:42.090 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.090 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.090 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:14996 3.2 3:44e0cffa:::rbd_header.110ef0d7a535:head [watch reconnect cookie 140213436269840 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:49:42.090 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:49:42.091 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.4470540769999998, 2024-09-16T20:49:42.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.008813943, 2024-09-16T20:49:42.091 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:42.091 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:42.091 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:42.091 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:42.091 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:42.091 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:42.091 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 14996 2024-09-16T20:49:42.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.091 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:42.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:42.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:49:42.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:42.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.092 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.092 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:42.092 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:49:42.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:42.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.092 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.092 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:42.092 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562560+0000", 2024-09-16T20:49:42.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.06e-07 2024-09-16T20:49:42.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.092 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.092 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:42.093 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562563+0000", 2024-09-16T20:49:42.093 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.495e-06 2024-09-16T20:49:42.093 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.093 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.093 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:42.093 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562571+0000", 2024-09-16T20:49:42.093 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1950000000000003e-06 2024-09-16T20:49:42.093 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.093 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.093 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:42.093 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562579+0000", 2024-09-16T20:49:42.093 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2649999999999999e-06 2024-09-16T20:49:42.093 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.093 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.093 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:42.094 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562673+0000", 2024-09-16T20:49:42.094 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4118999999999998e-05 2024-09-16T20:49:42.094 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.094 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.094 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:49:42.094 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562677+0000", 2024-09-16T20:49:42.094 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5179999999999998e-06 2024-09-16T20:49:42.094 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.094 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.094 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:42.094 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566461+0000", 2024-09-16T20:49:42.094 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0037837259999999998 2024-09-16T20:49:42.094 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.094 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.094 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:49:42.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566468+0000", 2024-09-16T20:49:42.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3350000000000001e-06 2024-09-16T20:49:42.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.095 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:42.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570306+0000", 2024-09-16T20:49:42.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.003837997 2024-09-16T20:49:42.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.095 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:42.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570392+0000", 2024-09-16T20:49:42.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5872999999999996e-05 2024-09-16T20:49:42.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.096 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.096 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:42.096 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571373+0000", 2024-09-16T20:49:42.096 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00098071399999999993 2024-09-16T20:49:42.096 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:42.096 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:42.096 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:42.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.096 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.096 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5255.0:1931 3.2 3:47882819:::rbd_data.110eb5f95afc.0000000000000009:head [read 217088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:49:42.096 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:49:42.096 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.4470153269999999, 2024-09-16T20:49:42.096 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0082752880000001, 2024-09-16T20:49:42.097 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:42.097 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:42.097 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:49:42.097 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:42.097 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5255", 2024-09-16T20:49:42.097 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:49:42.097 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1931 2024-09-16T20:49:42.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.097 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:42.097 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.097 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:42.097 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:49:42.097 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:42.098 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.098 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.098 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:42.098 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:49:42.098 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:42.098 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.098 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.098 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:42.098 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:49:42.098 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9699999999999996e-07 2024-09-16T20:49:42.098 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.098 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.098 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:42.098 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562601+0000", 2024-09-16T20:49:42.099 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8600000000000001e-06 2024-09-16T20:49:42.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.099 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.099 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:42.099 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562608+0000", 2024-09-16T20:49:42.099 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9e-06 2024-09-16T20:49:42.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.099 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.099 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:42.099 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562615+0000", 2024-09-16T20:49:42.099 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8179999999999998e-06 2024-09-16T20:49:42.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.099 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.100 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:42.100 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562703+0000", 2024-09-16T20:49:42.100 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7811000000000003e-05 2024-09-16T20:49:42.100 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.100 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.100 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:49:42.100 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562706+0000", 2024-09-16T20:49:42.100 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4920000000000002e-06 2024-09-16T20:49:42.100 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.100 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.100 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:42.100 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566685+0000", 2024-09-16T20:49:42.100 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0039792780000000002 2024-09-16T20:49:42.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.101 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:49:42.101 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566693+0000", 2024-09-16T20:49:42.101 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4730000000000003e-06 2024-09-16T20:49:42.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.101 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:42.101 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570737+0000", 2024-09-16T20:49:42.101 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0040436749999999 2024-09-16T20:49:42.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.101 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:42.101 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570873+0000", 2024-09-16T20:49:42.101 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000136484 2024-09-16T20:49:42.102 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:42.102 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:42.102 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:42.102 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.102 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.102 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5255.0:1939 3.2 3:47882819:::rbd_data.110eb5f95afc.0000000000000009:head [read 233472~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:49:42.102 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:49:42.102 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.446860461, 2024-09-16T20:49:42.102 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0084657589999999, 2024-09-16T20:49:42.102 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:42.102 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:42.102 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:49:42.102 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:42.102 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5255", 2024-09-16T20:49:42.102 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:49:42.103 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1939 2024-09-16T20:49:42.103 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.103 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:42.103 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.103 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:42.103 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:49:42.103 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:42.103 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.103 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.103 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:42.103 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:49:42.103 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:42.103 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.103 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.103 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:42.104 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:49:42.104 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5199999999999997e-07 2024-09-16T20:49:42.104 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.104 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.104 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:42.104 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562757+0000", 2024-09-16T20:49:42.104 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4979999999999998e-06 2024-09-16T20:49:42.104 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.104 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.104 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:42.104 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562764+0000", 2024-09-16T20:49:42.104 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1470000000000004e-06 2024-09-16T20:49:42.104 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.104 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.105 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:42.105 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562768+0000", 2024-09-16T20:49:42.105 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7929999999999999e-06 2024-09-16T20:49:42.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.105 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:42.105 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562805+0000", 2024-09-16T20:49:42.105 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7477e-05 2024-09-16T20:49:42.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.105 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting_for_map not empty", 2024-09-16T20:49:42.105 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562810+0000", 2024-09-16T20:49:42.105 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0819999999999998e-06 2024-09-16T20:49:42.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.106 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:42.106 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566717+0000", 2024-09-16T20:49:42.106 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0039072600000000001 2024-09-16T20:49:42.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.106 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:49:42.106 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566724+0000", 2024-09-16T20:49:42.106 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4880000000000004e-06 2024-09-16T20:49:42.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.106 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:42.106 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571009+0000", 2024-09-16T20:49:42.106 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.004284677 2024-09-16T20:49:42.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:42.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:42.107 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:42.107 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571218+0000", 2024-09-16T20:49:42.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00020978500000000001 2024-09-16T20:49:42.107 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:42.107 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:42.107 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:42.107 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:42.107 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:42.107 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:42.109 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:49:42.109 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:42.108+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:42.210 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:42.208+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:42.216 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:42.216 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:49:42.216 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:49:42.216 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:42.225 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:49:42.311 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:42.309+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:42.332 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:42.332 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:49:42.332 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:49:42.332 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:49:42.332 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:42.340 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:49:42.412 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:42.411+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:42.416 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #29, image image_client.0-clone29 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent30 2024-09-16T20:49:42.464 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:42.464 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:49:42.465 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:49:42.465 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:49:42.465 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.465 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2203 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1038336~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:42.465 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.336000+0000", 2024-09-16T20:49:42.465 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 80.126737063999997, 2024-09-16T20:49:42.465 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049949888999999997, 2024-09-16T20:49:42.465 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:42.465 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:42.465 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:42.466 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:42.466 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:42.466 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:42.466 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2203 2024-09-16T20:49:42.466 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.466 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:42.466 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.466 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:42.466 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336000+0000", 2024-09-16T20:49:42.466 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:42.466 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.467 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.467 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:42.467 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336000+0000", 2024-09-16T20:49:42.467 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:42.467 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.467 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.467 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:42.467 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336000+0000", 2024-09-16T20:49:42.467 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.58e-07 2024-09-16T20:49:42.467 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.467 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.468 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:42.468 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336002+0000", 2024-09-16T20:49:42.468 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0040000000000002e-06 2024-09-16T20:49:42.468 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.468 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.468 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:42.468 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336009+0000", 2024-09-16T20:49:42.468 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9890000000000003e-06 2024-09-16T20:49:42.468 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.468 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.468 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:42.469 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336014+0000", 2024-09-16T20:49:42.469 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4e-06 2024-09-16T20:49:42.469 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.469 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.469 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:42.469 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385759+0000", 2024-09-16T20:49:42.469 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049744151 2024-09-16T20:49:42.469 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.469 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.469 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:42.469 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385847+0000", 2024-09-16T20:49:42.470 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8189000000000006e-05 2024-09-16T20:49:42.470 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.470 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.470 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:42.470 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385950+0000", 2024-09-16T20:49:42.470 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102698 2024-09-16T20:49:42.470 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:42.470 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:42.470 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:42.470 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.471 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.471 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2206 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1046528~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:42.471 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:49:42.471 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 80.126714903000007, 2024-09-16T20:49:42.471 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050147714000000003, 2024-09-16T20:49:42.471 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:42.471 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:42.471 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:42.471 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:42.471 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:42.472 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:42.472 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2206 2024-09-16T20:49:42.472 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.472 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:42.472 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.472 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:42.472 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:49:42.472 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:42.472 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.472 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.472 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:42.472 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:49:42.473 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:42.473 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.473 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.473 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:42.473 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:49:42.473 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8599999999999999e-07 2024-09-16T20:49:42.473 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.473 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.473 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:42.473 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336024+0000", 2024-09-16T20:49:42.473 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.934e-06 2024-09-16T20:49:42.473 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.474 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.474 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:42.474 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336037+0000", 2024-09-16T20:49:42.474 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2496e-05 2024-09-16T20:49:42.474 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.474 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.474 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:42.474 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336042+0000", 2024-09-16T20:49:42.474 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4809999999999997e-06 2024-09-16T20:49:42.474 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.474 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.474 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:42.474 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385980+0000", 2024-09-16T20:49:42.475 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049938376999999999 2024-09-16T20:49:42.475 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.475 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.475 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:42.475 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.386069+0000", 2024-09-16T20:49:42.475 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8368999999999994e-05 2024-09-16T20:49:42.475 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.475 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.475 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:42.475 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.386170+0000", 2024-09-16T20:49:42.475 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000100671 2024-09-16T20:49:42.475 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:42.475 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:42.476 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:42.476 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.476 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.476 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5439 3.a 3:5789f5c3:::rbd_data.110ed9ca540.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 1181696~60416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:42.476 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:49:42.476 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 79.598026322999999, 2024-09-16T20:49:42.476 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10102045599999999, 2024-09-16T20:49:42.476 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:42.476 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:42.476 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:42.476 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:42.477 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:42.477 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:42.477 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5439 2024-09-16T20:49:42.477 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.477 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:42.477 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.477 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:42.477 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:49:42.477 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:42.477 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.477 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.478 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:42.478 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:49:42.478 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:42.478 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.478 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.478 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:42.478 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864711+0000", 2024-09-16T20:49:42.478 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7899999999999997e-07 2024-09-16T20:49:42.478 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.479 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.479 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:42.479 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864760+0000", 2024-09-16T20:49:42.479 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8803e-05 2024-09-16T20:49:42.479 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.479 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.479 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:42.479 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864769+0000", 2024-09-16T20:49:42.480 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0999999999999993e-06 2024-09-16T20:49:42.480 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.480 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.480 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:42.480 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864774+0000", 2024-09-16T20:49:42.480 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8919999999999999e-06 2024-09-16T20:49:42.480 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.480 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.480 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:42.480 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864815+0000", 2024-09-16T20:49:42.480 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1307000000000003e-05 2024-09-16T20:49:42.481 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.481 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.481 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:42.481 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864893+0000", 2024-09-16T20:49:42.481 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7581000000000001e-05 2024-09-16T20:49:42.481 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.481 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.481 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 1", 2024-09-16T20:49:42.481 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864996+0000", 2024-09-16T20:49:42.481 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102725 2024-09-16T20:49:42.481 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.482 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.482 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:42.482 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965442+0000", 2024-09-16T20:49:42.482 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100446012 2024-09-16T20:49:42.482 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.482 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.482 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:42.482 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965618+0000", 2024-09-16T20:49:42.482 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00017653100000000001 2024-09-16T20:49:42.482 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.483 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.483 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:42.483 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965682+0000", 2024-09-16T20:49:42.483 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3905999999999994e-05 2024-09-16T20:49:42.483 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.483 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.483 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:42.483 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965731+0000", 2024-09-16T20:49:42.483 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8819999999999997e-05 2024-09-16T20:49:42.483 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:42.483 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:42.484 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:42.484 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.484 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.484 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4600.0:5439 3.a e37/23 ondisk, result = 0)", 2024-09-16T20:49:42.484 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:49:42.484 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 79.597030175, 2024-09-16T20:49:42.484 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099742634999999996, 2024-09-16T20:49:42.484 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:42.484 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:42.484 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:42.485 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:42.485 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.485 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:42.485 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:49:42.485 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:42.485 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.485 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.485 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:42.485 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:49:42.485 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:42.485 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.486 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.486 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:42.486 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:49:42.486 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0900000000000001e-07 2024-09-16T20:49:42.486 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.486 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.486 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:42.486 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865711+0000", 2024-09-16T20:49:42.486 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3739999999999998e-06 2024-09-16T20:49:42.486 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.486 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.487 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:42.487 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965343+0000", 2024-09-16T20:49:42.487 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099632526999999999 2024-09-16T20:49:42.487 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.487 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.487 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:42.487 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965357+0000", 2024-09-16T20:49:42.487 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3713999999999999e-05 2024-09-16T20:49:42.487 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.487 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.487 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:42.488 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965426+0000", 2024-09-16T20:49:42.488 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8745999999999995e-05 2024-09-16T20:49:42.488 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.488 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.488 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:42.488 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965434+0000", 2024-09-16T20:49:42.488 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6910000000000003e-06 2024-09-16T20:49:42.488 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.488 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.488 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:42.488 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965449+0000", 2024-09-16T20:49:42.489 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4874e-05 2024-09-16T20:49:42.489 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:42.489 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:42.489 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:42.489 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.489 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.489 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4882.0:2508 3.12 3:4992771f:::rbd_object_map.110e7c8851c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T20:49:42.489 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:49:42.489 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 55.981226626999998, 2024-09-16T20:49:42.489 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.125007011, 2024-09-16T20:49:42.489 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:42.489 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:42.490 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:42.490 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:42.490 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4882", 2024-09-16T20:49:42.490 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2436923223", 2024-09-16T20:49:42.490 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2508 2024-09-16T20:49:42.490 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.490 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:42.490 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.490 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:42.490 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:49:42.490 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:42.490 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.491 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.491 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:42.491 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:49:42.491 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:42.491 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.491 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.491 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:42.491 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481511+0000", 2024-09-16T20:49:42.491 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0699999999999996e-07 2024-09-16T20:49:42.491 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.491 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.491 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:42.491 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481515+0000", 2024-09-16T20:49:42.492 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0609999999999997e-06 2024-09-16T20:49:42.492 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.492 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.492 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:42.492 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481523+0000", 2024-09-16T20:49:42.492 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1310000000000006e-06 2024-09-16T20:49:42.492 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.492 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.492 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:42.492 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481528+0000", 2024-09-16T20:49:42.492 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4900000000000002e-06 2024-09-16T20:49:42.492 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.493 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.493 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:42.493 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481565+0000", 2024-09-16T20:49:42.493 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7066000000000001e-05 2024-09-16T20:49:42.493 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.493 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.493 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:42.493 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481628+0000", 2024-09-16T20:49:42.493 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3118000000000001e-05 2024-09-16T20:49:42.493 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.493 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.493 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 2", 2024-09-16T20:49:42.494 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481822+0000", 2024-09-16T20:49:42.494 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019446200000000001 2024-09-16T20:49:42.494 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.494 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.494 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:42.494 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605988+0000", 2024-09-16T20:49:42.494 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.124165888 2024-09-16T20:49:42.494 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.494 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.494 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:42.495 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606427+0000", 2024-09-16T20:49:42.495 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00043854899999999999 2024-09-16T20:49:42.495 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.495 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.495 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:42.495 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606476+0000", 2024-09-16T20:49:42.495 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8971999999999998e-05 2024-09-16T20:49:42.495 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.495 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.495 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:42.495 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606517+0000", 2024-09-16T20:49:42.496 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1467000000000001e-05 2024-09-16T20:49:42.496 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:42.496 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:42.496 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:42.496 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.496 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.496 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4882.0:2508 3.12 e57/55 ondisk, result = 0)", 2024-09-16T20:49:42.496 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:49:42.496 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 55.980238127, 2024-09-16T20:49:42.497 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12349711400000001, 2024-09-16T20:49:42.497 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:42.497 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:42.497 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:42.497 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:42.497 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.497 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:42.497 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:49:42.497 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:42.497 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.497 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.497 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:42.498 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:49:42.498 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:42.498 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.498 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.498 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:42.498 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:49:42.498 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.06e-07 2024-09-16T20:49:42.498 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.498 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.498 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:42.498 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482502+0000", 2024-09-16T20:49:42.498 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.01e-06 2024-09-16T20:49:42.498 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.499 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.499 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:42.499 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605772+0000", 2024-09-16T20:49:42.499 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.123269687 2024-09-16T20:49:42.499 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.499 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.499 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:42.499 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605791+0000", 2024-09-16T20:49:42.499 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8912000000000001e-05 2024-09-16T20:49:42.499 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.500 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.500 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:42.500 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605974+0000", 2024-09-16T20:49:42.500 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000182836 2024-09-16T20:49:42.500 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.500 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.500 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:42.500 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605982+0000", 2024-09-16T20:49:42.500 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1699999999999997e-06 2024-09-16T20:49:42.500 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.500 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.501 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:42.501 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605996+0000", 2024-09-16T20:49:42.501 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3893000000000001e-05 2024-09-16T20:49:42.501 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:42.501 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:42.501 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:42.501 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.501 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.501 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4882.0:2512 3.f 3:f4d895b7:::rbd_data.110e7c8851c.0000000000000000:head [read 16384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T20:49:42.501 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:49:42.502 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 55.979801096999999, 2024-09-16T20:49:42.502 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12318483, 2024-09-16T20:49:42.502 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:42.502 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:42.502 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:42.502 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:42.502 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4882", 2024-09-16T20:49:42.502 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2436923223", 2024-09-16T20:49:42.502 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2512 2024-09-16T20:49:42.502 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.502 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:42.502 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.502 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:42.503 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:49:42.503 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:42.503 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.503 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.503 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:42.503 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:49:42.503 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:42.503 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.503 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.503 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:42.503 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:49:42.503 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3499999999999995e-07 2024-09-16T20:49:42.504 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.504 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.504 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:42.504 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482943+0000", 2024-09-16T20:49:42.504 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3450000000000002e-06 2024-09-16T20:49:42.504 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.504 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.504 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:42.504 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605772+0000", 2024-09-16T20:49:42.504 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.122829796 2024-09-16T20:49:42.505 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.505 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.505 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:42.505 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605792+0000", 2024-09-16T20:49:42.505 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9916999999999999e-05 2024-09-16T20:49:42.505 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.505 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.505 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:42.505 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605917+0000", 2024-09-16T20:49:42.505 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00012500799999999999 2024-09-16T20:49:42.505 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.506 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.506 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:42.506 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606004+0000", 2024-09-16T20:49:42.506 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6242999999999994e-05 2024-09-16T20:49:42.506 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.506 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.506 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:42.506 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606120+0000", 2024-09-16T20:49:42.506 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000116786 2024-09-16T20:49:42.506 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:42.506 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:42.506 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:42.506 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.507 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.507 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:11702 3.15 3:acf33c8b:::rbd_data.110e747f0779.000000000000001a:head [set-alloc-hint object_size 262144 write_size 262144,write 143360~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:49:42.507 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:49:42.507 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 38.324808675, 2024-09-16T20:49:42.507 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.084801043000000006, 2024-09-16T20:49:42.507 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:42.507 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:42.507 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:42.507 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:42.507 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:49:42.507 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:49:42.507 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 11702 2024-09-16T20:49:42.508 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.508 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:42.508 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.508 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:42.508 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:49:42.508 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:42.508 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.508 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.508 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:42.508 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:49:42.508 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:42.509 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.509 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.509 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:42.509 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137929+0000", 2024-09-16T20:49:42.509 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8400000000000002e-07 2024-09-16T20:49:42.509 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.509 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.509 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:42.509 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137937+0000", 2024-09-16T20:49:42.509 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0260000000000007e-06 2024-09-16T20:49:42.509 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.510 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.510 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:42.510 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137952+0000", 2024-09-16T20:49:42.510 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4659e-05 2024-09-16T20:49:42.510 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.510 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.510 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:42.510 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137960+0000", 2024-09-16T20:49:42.510 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0169999999999996e-06 2024-09-16T20:49:42.510 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.510 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.510 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:42.511 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138010+0000", 2024-09-16T20:49:42.511 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0021000000000001e-05 2024-09-16T20:49:42.511 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.511 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.511 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:42.511 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138114+0000", 2024-09-16T20:49:42.511 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000103797 2024-09-16T20:49:42.511 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.511 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.511 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:49:42.511 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138233+0000", 2024-09-16T20:49:42.511 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000118971 2024-09-16T20:49:42.512 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.512 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.512 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:42.512 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138953+0000", 2024-09-16T20:49:42.512 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00072026000000000002 2024-09-16T20:49:42.512 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.512 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.512 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:42.512 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222609+0000", 2024-09-16T20:49:42.512 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.083656650999999999 2024-09-16T20:49:42.512 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.512 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.513 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:42.513 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222684+0000", 2024-09-16T20:49:42.513 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4369999999999994e-05 2024-09-16T20:49:42.513 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.513 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.513 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:42.513 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222729+0000", 2024-09-16T20:49:42.513 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5287000000000002e-05 2024-09-16T20:49:42.513 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:42.513 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:42.513 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:42.514 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.514 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.514 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:11704 3.9 3:91220b75:::rbd_data.110e747f0779.000000000000001c:head [set-alloc-hint object_size 262144 write_size 262144,write 139264~6656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:49:42.514 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:49:42.514 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 38.324570674, 2024-09-16T20:49:42.514 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.084531133999999994, 2024-09-16T20:49:42.514 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:42.514 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:42.514 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:42.514 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:42.514 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:49:42.514 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:49:42.514 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 11704 2024-09-16T20:49:42.515 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.515 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:42.515 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.515 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:42.515 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:49:42.515 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:42.515 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.515 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.515 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:42.515 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:49:42.515 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:42.516 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.516 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.516 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:42.516 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138167+0000", 2024-09-16T20:49:42.516 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.33e-07 2024-09-16T20:49:42.516 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.516 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.516 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:42.516 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138179+0000", 2024-09-16T20:49:42.516 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2055999999999999e-05 2024-09-16T20:49:42.516 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.516 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.517 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:42.517 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138193+0000", 2024-09-16T20:49:42.517 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3961000000000001e-05 2024-09-16T20:49:42.517 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.517 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.517 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:42.517 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138200+0000", 2024-09-16T20:49:42.517 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4429999999999997e-06 2024-09-16T20:49:42.517 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.517 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.517 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:42.518 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138258+0000", 2024-09-16T20:49:42.518 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7225999999999997e-05 2024-09-16T20:49:42.518 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.518 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.518 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:42.518 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138364+0000", 2024-09-16T20:49:42.518 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000106013 2024-09-16T20:49:42.518 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.518 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.518 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:49:42.519 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138494+0000", 2024-09-16T20:49:42.519 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013059599999999999 2024-09-16T20:49:42.519 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.519 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.519 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:42.519 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.139220+0000", 2024-09-16T20:49:42.519 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00072539800000000004 2024-09-16T20:49:42.519 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.519 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.519 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:42.519 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222608+0000", 2024-09-16T20:49:42.519 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.083388620999999996 2024-09-16T20:49:42.520 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.520 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.520 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:42.520 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222651+0000", 2024-09-16T20:49:42.520 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2228000000000003e-05 2024-09-16T20:49:42.520 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.520 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.520 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:42.520 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222697+0000", 2024-09-16T20:49:42.520 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6659000000000002e-05 2024-09-16T20:49:42.521 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:42.521 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:42.521 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:42.521 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.521 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.521 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5138.0:9358 3.c 3.c25699cc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T20:49:42.521 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:49:42.521 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.612423369, 2024-09-16T20:49:42.521 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0018657580000001, 2024-09-16T20:49:42.521 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:42.521 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:42.521 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:49:42.522 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:42.522 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5138", 2024-09-16T20:49:42.522 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:49:42.522 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 9358 2024-09-16T20:49:42.522 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.522 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:42.522 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.522 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:42.522 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:49:42.522 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:42.523 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.523 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.523 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:42.523 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:49:42.523 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:42.523 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.523 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.523 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:42.523 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850314+0000", 2024-09-16T20:49:42.523 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4099999999999999e-07 2024-09-16T20:49:42.523 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.523 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.523 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:42.524 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850316+0000", 2024-09-16T20:49:42.524 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2469999999999998e-06 2024-09-16T20:49:42.524 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.524 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.524 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:42.524 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850322+0000", 2024-09-16T20:49:42.524 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3470000000000004e-06 2024-09-16T20:49:42.524 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.524 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.524 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:42.524 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850327+0000", 2024-09-16T20:49:42.524 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9450000000000001e-06 2024-09-16T20:49:42.524 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.525 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.525 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:42.525 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850376+0000", 2024-09-16T20:49:42.525 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8257000000000002e-05 2024-09-16T20:49:42.525 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.525 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.525 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:49:42.525 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850385+0000", 2024-09-16T20:49:42.525 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9800000000000004e-06 2024-09-16T20:49:42.525 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.525 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.525 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:42.526 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852167+0000", 2024-09-16T20:49:42.526 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0017825659999999 2024-09-16T20:49:42.526 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.526 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.526 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:42.526 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852179+0000", 2024-09-16T20:49:42.526 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1975e-05 2024-09-16T20:49:42.526 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:42.526 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:42.526 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:42.526 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.526 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.526 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5138.0:9382 3.c 3.c25699cc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T20:49:42.527 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:49:42.527 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.604076353, 2024-09-16T20:49:42.527 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.99358404700000003, 2024-09-16T20:49:42.527 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:42.527 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:42.527 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:49:42.527 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:42.527 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5138", 2024-09-16T20:49:42.527 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:49:42.527 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 9382 2024-09-16T20:49:42.527 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.527 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:42.527 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.528 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:42.528 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:49:42.528 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:42.528 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.528 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.528 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:42.528 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:49:42.528 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:42.528 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.528 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.528 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:42.528 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858661+0000", 2024-09-16T20:49:42.529 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0200000000000002e-07 2024-09-16T20:49:42.529 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.529 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.529 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:42.529 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858664+0000", 2024-09-16T20:49:42.529 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.038e-06 2024-09-16T20:49:42.529 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.529 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.529 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:42.529 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858671+0000", 2024-09-16T20:49:42.529 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0110000000000003e-06 2024-09-16T20:49:42.529 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.529 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.530 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:42.530 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858677+0000", 2024-09-16T20:49:42.530 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0959999999999997e-06 2024-09-16T20:49:42.530 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.530 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.530 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:42.530 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858732+0000", 2024-09-16T20:49:42.530 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4543000000000003e-05 2024-09-16T20:49:42.530 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.530 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.530 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:49:42.530 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858739+0000", 2024-09-16T20:49:42.530 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5429999999999996e-06 2024-09-16T20:49:42.531 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.531 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.531 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:42.531 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852236+0000", 2024-09-16T20:49:42.531 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.993496453 2024-09-16T20:49:42.531 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.531 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.531 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:42.531 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852244+0000", 2024-09-16T20:49:42.531 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8610000000000004e-06 2024-09-16T20:49:42.531 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:42.531 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:42.532 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:42.532 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.532 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.532 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:360 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 364544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:49:42.532 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246705+0000", 2024-09-16T20:49:42.532 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 6.2160321649999997, 2024-09-16T20:49:42.532 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050128442000000002, 2024-09-16T20:49:42.532 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:42.532 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:42.532 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:42.532 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:42.533 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:49:42.533 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:49:42.533 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 360 2024-09-16T20:49:42.533 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.533 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:42.533 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.533 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:42.533 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246705+0000", 2024-09-16T20:49:42.533 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:42.533 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.533 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.533 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:42.534 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246705+0000", 2024-09-16T20:49:42.534 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:42.534 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.534 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.534 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:42.534 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246705+0000", 2024-09-16T20:49:42.534 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3999999999999998e-07 2024-09-16T20:49:42.534 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.534 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.534 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:42.534 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246706+0000", 2024-09-16T20:49:42.535 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1710000000000001e-06 2024-09-16T20:49:42.535 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.535 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.535 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:42.535 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246709+0000", 2024-09-16T20:49:42.535 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0350000000000002e-06 2024-09-16T20:49:42.535 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.535 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.535 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:42.535 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246712+0000", 2024-09-16T20:49:42.535 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5239999999999999e-06 2024-09-16T20:49:42.535 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.535 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.536 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:42.536 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.296641+0000", 2024-09-16T20:49:42.536 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049929310999999997 2024-09-16T20:49:42.536 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.536 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.536 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:42.536 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.296741+0000", 2024-09-16T20:49:42.536 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.9976999999999995e-05 2024-09-16T20:49:42.536 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.536 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.536 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:42.536 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.296833+0000", 2024-09-16T20:49:42.537 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2183999999999997e-05 2024-09-16T20:49:42.537 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:42.537 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:42.537 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:42.537 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.537 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.537 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:361 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 372736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:49:42.537 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246714+0000", 2024-09-16T20:49:42.537 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 6.2160222049999998, 2024-09-16T20:49:42.537 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050338229999999998, 2024-09-16T20:49:42.537 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:42.537 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:42.537 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:42.538 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:42.538 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:49:42.538 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:49:42.538 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 361 2024-09-16T20:49:42.538 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.538 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:42.538 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.538 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:42.538 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246714+0000", 2024-09-16T20:49:42.538 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:42.538 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.539 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.539 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:42.539 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246714+0000", 2024-09-16T20:49:42.539 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:42.539 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.539 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.539 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:42.539 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246715+0000", 2024-09-16T20:49:42.539 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1899999999999999e-07 2024-09-16T20:49:42.539 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.539 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.539 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:42.540 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246716+0000", 2024-09-16T20:49:42.540 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1659999999999999e-06 2024-09-16T20:49:42.540 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.540 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.540 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:42.540 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246721+0000", 2024-09-16T20:49:42.540 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9830000000000004e-06 2024-09-16T20:49:42.540 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.540 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.540 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:42.540 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246723+0000", 2024-09-16T20:49:42.540 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5679999999999998e-06 2024-09-16T20:49:42.541 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.541 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.541 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:42.541 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.296875+0000", 2024-09-16T20:49:42.541 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050151683000000002 2024-09-16T20:49:42.541 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.541 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.541 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:42.541 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.296959+0000", 2024-09-16T20:49:42.541 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4079999999999995e-05 2024-09-16T20:49:42.541 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.541 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.542 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:42.542 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297053+0000", 2024-09-16T20:49:42.542 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.3530999999999996e-05 2024-09-16T20:49:42.542 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:42.542 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:42.542 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:42.542 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.542 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.542 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:362 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 385024~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:49:42.542 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246727+0000", 2024-09-16T20:49:42.542 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 6.2160100959999998, 2024-09-16T20:49:42.542 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050546528, 2024-09-16T20:49:42.542 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:42.543 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:42.543 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:42.543 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:42.543 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:49:42.543 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:49:42.543 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 362 2024-09-16T20:49:42.543 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.543 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:42.543 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.543 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:42.543 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246727+0000", 2024-09-16T20:49:42.543 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:42.544 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.544 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.544 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:42.544 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246727+0000", 2024-09-16T20:49:42.544 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:42.544 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.544 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.544 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:42.544 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246727+0000", 2024-09-16T20:49:42.544 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.35e-07 2024-09-16T20:49:42.544 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.545 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.545 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:42.545 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246728+0000", 2024-09-16T20:49:42.545 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.175e-06 2024-09-16T20:49:42.545 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.545 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.545 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:42.545 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246734+0000", 2024-09-16T20:49:42.545 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2160000000000003e-06 2024-09-16T20:49:42.545 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.545 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.545 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:42.546 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246737+0000", 2024-09-16T20:49:42.546 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9450000000000002e-06 2024-09-16T20:49:42.546 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.546 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.546 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:42.546 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297103+0000", 2024-09-16T20:49:42.546 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050365518999999997 2024-09-16T20:49:42.546 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.546 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.546 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:42.546 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297183+0000", 2024-09-16T20:49:42.546 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0208000000000002e-05 2024-09-16T20:49:42.546 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.547 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.547 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:42.547 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297273+0000", 2024-09-16T20:49:42.547 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0229999999999995e-05 2024-09-16T20:49:42.547 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:42.547 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:42.547 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:42.547 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.547 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.547 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:368 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 327680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:49:42.547 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246746+0000", 2024-09-16T20:49:42.547 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 6.2159908159999997, 2024-09-16T20:49:42.547 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050730747999999999, 2024-09-16T20:49:42.547 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:42.548 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:42.548 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:42.548 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:42.548 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:49:42.548 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:49:42.548 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 368 2024-09-16T20:49:42.548 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.548 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:42.548 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.548 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:42.548 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246746+0000", 2024-09-16T20:49:42.549 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:42.549 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.549 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.549 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:42.549 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246746+0000", 2024-09-16T20:49:42.549 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:42.549 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.549 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.549 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:42.549 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246746+0000", 2024-09-16T20:49:42.549 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8099999999999999e-07 2024-09-16T20:49:42.550 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.550 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.550 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:42.550 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246754+0000", 2024-09-16T20:49:42.550 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4440000000000002e-06 2024-09-16T20:49:42.550 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.550 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.550 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:42.550 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246758+0000", 2024-09-16T20:49:42.550 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3900000000000003e-06 2024-09-16T20:49:42.550 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.550 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.551 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:42.551 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246761+0000", 2024-09-16T20:49:42.551 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.101e-06 2024-09-16T20:49:42.551 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.551 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.551 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:42.551 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297310+0000", 2024-09-16T20:49:42.551 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050549166 2024-09-16T20:49:42.552 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.552 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.552 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:42.552 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297387+0000", 2024-09-16T20:49:42.552 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6658999999999999e-05 2024-09-16T20:49:42.552 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.552 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.552 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:42.552 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297477+0000", 2024-09-16T20:49:42.552 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9707e-05 2024-09-16T20:49:42.552 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:42.552 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:42.553 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:42.553 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.553 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.553 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:369 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 339968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:49:42.553 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246766+0000", 2024-09-16T20:49:42.553 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 6.2159710910000001, 2024-09-16T20:49:42.553 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050911291999999997, 2024-09-16T20:49:42.553 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:42.554 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:42.554 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:42.554 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:42.554 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:49:42.554 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:49:42.554 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 369 2024-09-16T20:49:42.554 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.554 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:42.554 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.554 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:42.554 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246766+0000", 2024-09-16T20:49:42.554 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:42.554 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.555 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.555 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:42.555 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246766+0000", 2024-09-16T20:49:42.555 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:42.555 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.555 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.555 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:42.555 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246766+0000", 2024-09-16T20:49:42.555 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3200000000000001e-07 2024-09-16T20:49:42.555 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.555 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.555 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:42.555 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246767+0000", 2024-09-16T20:49:42.556 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.252e-06 2024-09-16T20:49:42.556 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.556 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.556 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:42.556 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246770+0000", 2024-09-16T20:49:42.556 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1049999999999999e-06 2024-09-16T20:49:42.556 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.556 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.556 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:42.556 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246773+0000", 2024-09-16T20:49:42.556 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.396e-06 2024-09-16T20:49:42.556 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.557 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.557 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:42.557 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297513+0000", 2024-09-16T20:49:42.557 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050740800000000003 2024-09-16T20:49:42.557 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.557 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.557 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:42.557 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297589+0000", 2024-09-16T20:49:42.557 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5318e-05 2024-09-16T20:49:42.557 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.557 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.557 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:42.557 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297677+0000", 2024-09-16T20:49:42.558 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8189000000000006e-05 2024-09-16T20:49:42.558 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:42.558 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:42.558 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:42.558 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.558 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.558 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:370 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 348160~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:49:42.558 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246776+0000", 2024-09-16T20:49:42.558 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 6.2159610760000001, 2024-09-16T20:49:42.558 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051096643999999997, 2024-09-16T20:49:42.558 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:42.559 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:42.559 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:42.559 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:42.559 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:49:42.559 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:49:42.559 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 370 2024-09-16T20:49:42.559 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.559 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:42.559 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.559 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:42.559 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246776+0000", 2024-09-16T20:49:42.559 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:42.559 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.560 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.560 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:42.560 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246776+0000", 2024-09-16T20:49:42.560 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:42.560 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.560 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.560 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:42.560 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246776+0000", 2024-09-16T20:49:42.560 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2499999999999999e-07 2024-09-16T20:49:42.560 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.560 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.560 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:42.561 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246777+0000", 2024-09-16T20:49:42.561 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1510000000000001e-06 2024-09-16T20:49:42.561 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.561 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.561 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:42.561 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246780+0000", 2024-09-16T20:49:42.561 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9610000000000001e-06 2024-09-16T20:49:42.561 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.561 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.561 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:42.561 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246783+0000", 2024-09-16T20:49:42.561 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6129999999999998e-06 2024-09-16T20:49:42.562 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.562 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.562 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:42.562 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297713+0000", 2024-09-16T20:49:42.562 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050929950000000002 2024-09-16T20:49:42.562 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.562 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.562 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:42.562 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297786+0000", 2024-09-16T20:49:42.562 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3096000000000006e-05 2024-09-16T20:49:42.562 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.562 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.562 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:42.562 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297872+0000", 2024-09-16T20:49:42.563 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6648000000000001e-05 2024-09-16T20:49:42.563 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:42.563 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:42.563 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:42.563 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.563 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.563 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:371 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 380928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:49:42.563 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246791+0000", 2024-09-16T20:49:42.563 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 6.2159454399999996, 2024-09-16T20:49:42.563 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051287209, 2024-09-16T20:49:42.563 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:42.563 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:42.563 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:42.564 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:42.564 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:49:42.564 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:49:42.564 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 371 2024-09-16T20:49:42.564 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.564 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:42.564 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.564 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:42.564 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246791+0000", 2024-09-16T20:49:42.564 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:42.564 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.564 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.564 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:42.565 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246791+0000", 2024-09-16T20:49:42.565 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:42.565 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.565 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.565 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:42.565 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246792+0000", 2024-09-16T20:49:42.565 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8200000000000001e-07 2024-09-16T20:49:42.565 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.565 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.565 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:42.565 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246793+0000", 2024-09-16T20:49:42.565 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.28e-06 2024-09-16T20:49:42.566 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.566 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.566 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:42.566 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246796+0000", 2024-09-16T20:49:42.566 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.597e-06 2024-09-16T20:49:42.566 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.566 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.566 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:42.566 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246799+0000", 2024-09-16T20:49:42.566 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7089999999999998e-06 2024-09-16T20:49:42.566 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.566 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.567 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:42.567 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297910+0000", 2024-09-16T20:49:42.567 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051110598 2024-09-16T20:49:42.567 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.567 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.567 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:42.567 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297986+0000", 2024-09-16T20:49:42.567 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6711000000000005e-05 2024-09-16T20:49:42.567 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.567 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.567 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:42.567 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.298078+0000", 2024-09-16T20:49:42.567 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2032000000000002e-05 2024-09-16T20:49:42.568 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:42.568 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:42.568 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:42.568 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.568 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.568 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:373 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 376832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:49:42.568 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246904+0000", 2024-09-16T20:49:42.568 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 6.2158330179999997, 2024-09-16T20:49:42.568 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051310617000000003, 2024-09-16T20:49:42.568 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:42.568 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:42.568 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:42.569 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:42.569 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:49:42.569 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:49:42.569 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 373 2024-09-16T20:49:42.569 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.569 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:42.569 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.569 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:42.569 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246904+0000", 2024-09-16T20:49:42.569 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:42.569 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.570 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.570 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:42.570 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246904+0000", 2024-09-16T20:49:42.570 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:42.570 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.570 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.570 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:42.570 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246904+0000", 2024-09-16T20:49:42.570 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8000000000000001e-07 2024-09-16T20:49:42.570 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.570 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.570 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:42.571 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246908+0000", 2024-09-16T20:49:42.571 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4039999999999997e-06 2024-09-16T20:49:42.571 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.571 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.571 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:42.571 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246913+0000", 2024-09-16T20:49:42.571 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0030000000000002e-06 2024-09-16T20:49:42.571 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.571 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.571 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:42.571 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246917+0000", 2024-09-16T20:49:42.571 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.303e-06 2024-09-16T20:49:42.572 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.572 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.572 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:42.572 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.298114+0000", 2024-09-16T20:49:42.572 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051197724 2024-09-16T20:49:42.572 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.572 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.572 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:42.572 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.298159+0000", 2024-09-16T20:49:42.572 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4882999999999997e-05 2024-09-16T20:49:42.572 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.573 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.573 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:42.573 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.298214+0000", 2024-09-16T20:49:42.573 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4920000000000003e-05 2024-09-16T20:49:42.573 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:42.573 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:42.573 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:42.573 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.573 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.573 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:374 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 389120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:49:42.573 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246927+0000", 2024-09-16T20:49:42.573 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 6.2158099690000004, 2024-09-16T20:49:42.574 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051498164999999999, 2024-09-16T20:49:42.574 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:42.574 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:42.574 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:42.574 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:42.574 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:49:42.574 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:49:42.574 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 374 2024-09-16T20:49:42.574 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.574 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:42.574 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.574 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:42.574 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246927+0000", 2024-09-16T20:49:42.575 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:42.575 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.575 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.575 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:42.575 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246927+0000", 2024-09-16T20:49:42.575 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:42.575 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.575 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.575 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:42.575 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246927+0000", 2024-09-16T20:49:42.575 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7e-07 2024-09-16T20:49:42.576 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.576 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.576 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:42.576 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246929+0000", 2024-09-16T20:49:42.576 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.607e-06 2024-09-16T20:49:42.576 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.576 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.576 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:42.576 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246932+0000", 2024-09-16T20:49:42.576 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4539999999999999e-06 2024-09-16T20:49:42.576 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.577 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.577 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:42.577 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246935+0000", 2024-09-16T20:49:42.577 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6570000000000001e-06 2024-09-16T20:49:42.577 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.577 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.577 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:42.577 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.298242+0000", 2024-09-16T20:49:42.577 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051306887000000002 2024-09-16T20:49:42.577 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.577 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.577 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:42.578 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.298327+0000", 2024-09-16T20:49:42.578 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5068999999999995e-05 2024-09-16T20:49:42.578 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:42.578 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:42.578 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:42.578 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.298425+0000", 2024-09-16T20:49:42.578 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8021000000000002e-05 2024-09-16T20:49:42.578 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:42.578 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:42.578 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:42.578 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:42.578 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:42.579 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:42.580 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:49:42.581 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:42.578+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:42.588 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T20:49:42.594 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T20:49:42.681 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:42.679+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:42.739 INFO:teuthology.orchestra.run.smithi139.stdout:4039 read 0x131a800 thru 0x13272bb (0xcabc bytes) 2024-09-16T20:49:42.739 INFO:teuthology.orchestra.run.smithi139.stdout:4041 punch from 0xaff1a00 to 0xb000600, (0xec00 bytes) 2024-09-16T20:49:42.747 INFO:teuthology.orchestra.run.smithi139.stdout:4045 read 0x3dac400 thru 0x3dae221 (0x1e22 bytes) 2024-09-16T20:49:42.748 INFO:teuthology.orchestra.run.smithi139.stdout:4051 punch from 0xb41c400 to 0xb425000, (0x8c00 bytes) 2024-09-16T20:49:42.752 INFO:teuthology.orchestra.run.smithi139.stdout:4053 write 0xf21c00 thru 0xf2a8dc (0x8cdd bytes) 2024-09-16T20:49:42.752 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:42.752 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:49:42.752 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:49:42.752 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:42.753 INFO:teuthology.orchestra.run.smithi139.stdout:4054 clone 32 order 20 su 524288 sc 15 2024-09-16T20:49:42.768 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:49:42.782 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:42.780+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:42.824 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:49:42.824 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:49:42.858 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:49:42.858 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:49:42.877 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:49:42.877 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:49:42.882 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:42.881+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:42.888 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:42.888 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:42.894 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:49:42.894 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:49:42.897 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:49:42.911 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:49:42.911 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:49:42.946 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:49:42.946 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:49:42.999 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:42.998+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:43.006 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:43.006 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:43.015 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:49:43.084 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:43.083+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:43.162 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:43.162 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:43.171 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:49:43.185 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:43.183+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:43.278 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:43.278 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:43.286 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:43.284+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:43.287 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:49:43.386 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:43.385+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:43.435 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:43.435 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:43.443 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:49:43.488 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:43.486+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:43.558 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:43.558 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:43.567 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:49:43.591 INFO:teuthology.orchestra.run.smithi029.stderr:noscrub is set 2024-09-16T20:49:43.608 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:43.607+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:43.688 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:43.687+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:43.717 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:43.717 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:43.723 INFO:teuthology.orchestra.run.smithi139.stdout:truncating image image_client.0-clone31 from 0xb5c4e00 (overlap 0xb5c4e00) to 0x245a800 2024-09-16T20:49:43.726 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:49:43.789 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:43.787+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:43.875 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:43.875 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:43.884 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:49:43.927 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:43.926+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:43.990 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:43.988+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:43.990 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:43.991 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:43.999 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:49:44.091 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:44.090+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:44.147 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:44.147 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:44.155 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:49:44.190 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 3, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-16T20:49:44.190 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T20:49:44.190 INFO:tasks.thrashosds.thrasher:inject_pause on osd.2 2024-09-16T20:49:44.190 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T20:49:44.190 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T20:49:44.190 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok config set filestore_inject_stall 3 2024-09-16T20:49:44.192 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:44.191+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:44.266 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:44.266 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:44.276 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:49:44.308 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:44.308 INFO:teuthology.orchestra.run.smithi029.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-16T20:49:44.308 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:44.319 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:44.318+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:44.385 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:44.385 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:44.393 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:44.391+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:44.394 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:49:44.435 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #30, image image_client.0-clone30 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent31 2024-09-16T20:49:44.493 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:44.492+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:44.572 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:44.573 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:44.583 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:49:44.594 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:44.593+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:44.695 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:44.694+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:44.714 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:44.714 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:44.725 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:49:44.796 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:44.795+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:44.833 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:44.833 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:44.841 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:49:44.897 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:44.896+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:44.948 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:44.949 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:44.957 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:49:44.958 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T20:49:44.999 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:44.996+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:45.099 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:45.098+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:45.107 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:45.107 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:45.116 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:49:45.121 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:49:45.121 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:49:45.139 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:49:45.139 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:49:45.156 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:49:45.156 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:49:45.173 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:49:45.173 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:49:45.191 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:49:45.191 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:49:45.200 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:45.199+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:45.208 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:49:45.208 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:49:45.230 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:45.231 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:49:45.231 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:49:45.231 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:45.239 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:49:45.323 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:45.322+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:45.386 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:45.386 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:49:45.386 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:49:45.386 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:49:45.386 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:45.394 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:49:45.435 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:45.434+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:45.503 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:45.502+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:45.543 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:45.543 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:49:45.543 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:49:45.543 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:49:45.543 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.543 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:49:45.543 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:49:45.543 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 112.286734677, 2024-09-16T20:49:45.543 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67502346999999996, 2024-09-16T20:49:45.544 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:45.544 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:45.544 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:45.544 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:45.544 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:49:45.544 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:49:45.544 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 1 2024-09-16T20:49:45.544 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.544 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:45.545 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.545 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:45.545 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:49:45.545 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:45.545 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.545 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.545 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:45.545 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:49:45.545 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:45.545 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.545 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.545 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:45.546 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254128+0000", 2024-09-16T20:49:45.546 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.463e-06 2024-09-16T20:49:45.546 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.546 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.546 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:45.546 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254135+0000", 2024-09-16T20:49:45.546 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1579999999999999e-06 2024-09-16T20:49:45.546 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.546 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.546 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:45.546 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254146+0000", 2024-09-16T20:49:45.547 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1341e-05 2024-09-16T20:49:45.547 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.547 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.547 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:45.547 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254155+0000", 2024-09-16T20:49:45.547 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.1549999999999996e-06 2024-09-16T20:49:45.547 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.547 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.547 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:45.547 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254221+0000", 2024-09-16T20:49:45.547 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5470999999999997e-05 2024-09-16T20:49:45.548 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.548 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.548 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:49:45.548 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254231+0000", 2024-09-16T20:49:45.548 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0592e-05 2024-09-16T20:49:45.548 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.548 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.548 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:45.548 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926711+0000", 2024-09-16T20:49:45.548 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67247946599999997 2024-09-16T20:49:45.548 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.548 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.549 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:45.549 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926910+0000", 2024-09-16T20:49:45.549 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019934 2024-09-16T20:49:45.549 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.549 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.549 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:49:45.549 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.927159+0000", 2024-09-16T20:49:45.549 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00024876599999999998 2024-09-16T20:49:45.549 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.549 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.549 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:45.549 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.928797+0000", 2024-09-16T20:49:45.550 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001638308 2024-09-16T20:49:45.550 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.550 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.550 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:45.550 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929026+0000", 2024-09-16T20:49:45.550 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00022876099999999999 2024-09-16T20:49:45.550 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.550 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.550 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:45.550 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929102+0000", 2024-09-16T20:49:45.550 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6184000000000001e-05 2024-09-16T20:49:45.551 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.551 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.551 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:45.551 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929150+0000", 2024-09-16T20:49:45.551 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7465e-05 2024-09-16T20:49:45.551 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:45.551 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:45.551 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:45.551 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.551 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.551 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3571 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:45.551 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481059+0000", 2024-09-16T20:49:45.552 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 56.059801616000001, 2024-09-16T20:49:45.552 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049572995000000002, 2024-09-16T20:49:45.552 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:45.552 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:45.552 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:45.552 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:45.552 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:45.552 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:45.552 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3571 2024-09-16T20:49:45.552 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.552 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:45.552 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.552 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:45.553 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481059+0000", 2024-09-16T20:49:45.553 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:45.553 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.553 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.553 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:45.553 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481059+0000", 2024-09-16T20:49:45.553 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:45.553 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.553 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.553 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:45.553 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481060+0000", 2024-09-16T20:49:45.554 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2599999999999998e-07 2024-09-16T20:49:45.554 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.554 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.554 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:45.554 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481063+0000", 2024-09-16T20:49:45.554 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.999e-06 2024-09-16T20:49:45.554 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.554 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.554 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:45.554 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481067+0000", 2024-09-16T20:49:45.554 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3080000000000001e-06 2024-09-16T20:49:45.554 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.554 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.555 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:45.555 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481070+0000", 2024-09-16T20:49:45.555 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9919999999999999e-06 2024-09-16T20:49:45.555 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.555 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.555 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:45.555 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.522087+0000", 2024-09-16T20:49:45.555 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.041017023 2024-09-16T20:49:45.555 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.555 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.555 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:45.555 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.522191+0000", 2024-09-16T20:49:45.556 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010389499999999999 2024-09-16T20:49:45.556 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.556 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.556 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:45.556 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.522566+0000", 2024-09-16T20:49:45.556 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00037539399999999998 2024-09-16T20:49:45.556 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.556 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.556 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:45.556 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525494+0000", 2024-09-16T20:49:45.556 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0029273070000000001 2024-09-16T20:49:45.556 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.556 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.557 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:45.557 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530355+0000", 2024-09-16T20:49:45.557 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0048612509999999996 2024-09-16T20:49:45.557 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.557 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.557 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:45.557 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530407+0000", 2024-09-16T20:49:45.557 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1817000000000002e-05 2024-09-16T20:49:45.557 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.557 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.557 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:45.558 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530632+0000", 2024-09-16T20:49:45.558 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00022558300000000001 2024-09-16T20:49:45.558 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:45.558 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:45.558 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:45.558 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.558 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.558 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3572 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:45.558 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481220+0000", 2024-09-16T20:49:45.558 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 56.059641048000003, 2024-09-16T20:49:45.558 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050452862000000001, 2024-09-16T20:49:45.558 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:45.559 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:45.559 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:45.559 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:45.559 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:45.559 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:45.559 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3572 2024-09-16T20:49:45.559 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.559 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:45.559 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.559 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:45.559 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481220+0000", 2024-09-16T20:49:45.560 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:45.560 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.560 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.560 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:45.560 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481220+0000", 2024-09-16T20:49:45.560 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:45.560 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.560 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.560 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:45.560 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481221+0000", 2024-09-16T20:49:45.560 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9400000000000004e-07 2024-09-16T20:49:45.560 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.561 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.561 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:45.561 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481225+0000", 2024-09-16T20:49:45.561 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0230000000000003e-06 2024-09-16T20:49:45.561 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.561 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.561 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:45.561 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481234+0000", 2024-09-16T20:49:45.561 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2920000000000001e-06 2024-09-16T20:49:45.561 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.561 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.562 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:45.562 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481240+0000", 2024-09-16T20:49:45.562 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9390000000000002e-06 2024-09-16T20:49:45.562 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.562 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.562 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:45.562 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.523544+0000", 2024-09-16T20:49:45.562 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.042304475000000001 2024-09-16T20:49:45.562 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.562 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.562 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:45.562 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.523624+0000", 2024-09-16T20:49:45.563 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.975e-05 2024-09-16T20:49:45.563 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.563 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.563 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:45.563 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.523975+0000", 2024-09-16T20:49:45.563 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00035063200000000003 2024-09-16T20:49:45.563 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.563 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.563 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:45.563 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530337+0000", 2024-09-16T20:49:45.564 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0063623589999999997 2024-09-16T20:49:45.564 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.564 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.564 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:45.564 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531592+0000", 2024-09-16T20:49:45.564 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0012550599999999999 2024-09-16T20:49:45.564 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.564 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.564 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:45.564 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531633+0000", 2024-09-16T20:49:45.564 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1192000000000001e-05 2024-09-16T20:49:45.564 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.565 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.565 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:45.565 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531673+0000", 2024-09-16T20:49:45.565 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9345999999999999e-05 2024-09-16T20:49:45.565 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:45.565 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:45.565 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:45.565 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.565 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.565 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3573 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:45.565 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481257+0000", 2024-09-16T20:49:45.566 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 56.059604100999998, 2024-09-16T20:49:45.566 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050511027, 2024-09-16T20:49:45.566 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:45.566 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:45.566 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:45.566 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:45.566 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:45.566 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:45.566 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3573 2024-09-16T20:49:45.566 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.567 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:45.567 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.567 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:45.567 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481257+0000", 2024-09-16T20:49:45.567 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:45.567 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.567 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.567 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:45.567 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481257+0000", 2024-09-16T20:49:45.568 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:45.568 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.568 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.568 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:45.568 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481257+0000", 2024-09-16T20:49:45.568 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.99e-07 2024-09-16T20:49:45.568 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.568 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.568 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:45.569 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481260+0000", 2024-09-16T20:49:45.569 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1709999999999998e-06 2024-09-16T20:49:45.569 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.569 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.569 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:45.569 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481271+0000", 2024-09-16T20:49:45.569 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0795000000000001e-05 2024-09-16T20:49:45.569 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.569 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.569 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:45.570 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481277+0000", 2024-09-16T20:49:45.570 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1670000000000004e-06 2024-09-16T20:49:45.570 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.570 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.570 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:45.570 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.524647+0000", 2024-09-16T20:49:45.570 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.043369810000000002 2024-09-16T20:49:45.570 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.570 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.570 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:45.570 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.524725+0000", 2024-09-16T20:49:45.571 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7286000000000005e-05 2024-09-16T20:49:45.571 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.571 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.571 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:45.571 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525062+0000", 2024-09-16T20:49:45.571 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00033739499999999999 2024-09-16T20:49:45.571 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.571 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.571 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:45.571 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530692+0000", 2024-09-16T20:49:45.571 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0056301479999999997 2024-09-16T20:49:45.572 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.572 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.572 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:45.572 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531694+0000", 2024-09-16T20:49:45.572 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0010017419999999999 2024-09-16T20:49:45.572 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.572 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.572 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:45.572 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531737+0000", 2024-09-16T20:49:45.572 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2762999999999997e-05 2024-09-16T20:49:45.573 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.573 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.573 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:45.573 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531768+0000", 2024-09-16T20:49:45.573 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1151e-05 2024-09-16T20:49:45.573 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:45.573 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:45.573 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:45.573 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.573 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.573 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3574 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:45.574 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:49:45.574 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 56.059571145, 2024-09-16T20:49:45.574 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053579441999999998, 2024-09-16T20:49:45.574 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:45.574 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:45.574 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:45.574 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:45.574 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:45.574 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:45.574 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3574 2024-09-16T20:49:45.575 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.575 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:45.575 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.575 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:45.575 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:49:45.575 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:45.575 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.575 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.575 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:45.575 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:49:45.576 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:45.576 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.576 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.576 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:45.576 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:49:45.576 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2e-07 2024-09-16T20:49:45.576 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.576 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.576 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:45.576 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481292+0000", 2024-09-16T20:49:45.577 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2630000000000002e-06 2024-09-16T20:49:45.577 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.577 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.577 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:45.577 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481297+0000", 2024-09-16T20:49:45.577 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.296e-06 2024-09-16T20:49:45.577 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.577 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.577 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:45.577 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481300+0000", 2024-09-16T20:49:45.578 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7019999999999999e-06 2024-09-16T20:49:45.578 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.578 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.578 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:45.578 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525533+0000", 2024-09-16T20:49:45.578 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.044232318 2024-09-16T20:49:45.578 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.578 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.578 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:45.578 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525619+0000", 2024-09-16T20:49:45.578 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5892999999999999e-05 2024-09-16T20:49:45.579 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.579 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.579 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:45.579 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525987+0000", 2024-09-16T20:49:45.579 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00036836099999999999 2024-09-16T20:49:45.579 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.579 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.579 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:45.579 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531779+0000", 2024-09-16T20:49:45.579 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0057918240000000001 2024-09-16T20:49:45.580 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.580 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.580 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:45.580 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534747+0000", 2024-09-16T20:49:45.580 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0029681600000000001 2024-09-16T20:49:45.580 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.580 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.580 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:45.580 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534832+0000", 2024-09-16T20:49:45.581 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5180999999999996e-05 2024-09-16T20:49:45.581 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.581 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.581 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:45.581 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534869+0000", 2024-09-16T20:49:45.581 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7024000000000003e-05 2024-09-16T20:49:45.581 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:45.581 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:45.581 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:45.581 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.581 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.581 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3575 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:45.582 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:49:45.582 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 56.059556405000002, 2024-09-16T20:49:45.582 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053801463000000001, 2024-09-16T20:49:45.582 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:45.582 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:45.582 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:45.582 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:45.582 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:45.582 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:45.582 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3575 2024-09-16T20:49:45.582 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.582 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:45.583 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.583 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:45.583 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:49:45.583 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:45.583 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.583 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.583 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:45.583 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:49:45.583 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:45.583 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.583 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.583 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:45.583 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481305+0000", 2024-09-16T20:49:45.584 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5400000000000002e-07 2024-09-16T20:49:45.584 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.584 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.584 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:45.584 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481307+0000", 2024-09-16T20:49:45.584 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7800000000000001e-06 2024-09-16T20:49:45.584 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.584 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.584 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:45.584 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481312+0000", 2024-09-16T20:49:45.584 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6349999999999997e-06 2024-09-16T20:49:45.585 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.585 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.585 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:45.585 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481315+0000", 2024-09-16T20:49:45.585 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7829999999999999e-06 2024-09-16T20:49:45.585 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.585 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.585 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:45.585 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.526876+0000", 2024-09-16T20:49:45.585 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.045561070000000002 2024-09-16T20:49:45.585 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.586 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.586 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:45.586 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.526954+0000", 2024-09-16T20:49:45.586 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8108000000000005e-05 2024-09-16T20:49:45.586 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.586 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.586 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:45.586 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527282+0000", 2024-09-16T20:49:45.586 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000327483 2024-09-16T20:49:45.586 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.586 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.586 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:45.587 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531784+0000", 2024-09-16T20:49:45.587 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0045020169999999997 2024-09-16T20:49:45.587 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.587 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.587 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:45.587 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534921+0000", 2024-09-16T20:49:45.587 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.003137791 2024-09-16T20:49:45.587 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.587 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.587 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:45.587 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534976+0000", 2024-09-16T20:49:45.587 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4953000000000001e-05 2024-09-16T20:49:45.587 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.588 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.588 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:45.588 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535106+0000", 2024-09-16T20:49:45.588 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000129589 2024-09-16T20:49:45.588 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:45.588 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:45.588 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:45.588 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.588 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.588 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3576 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:45.588 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:49:45.589 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 56.059542493000002, 2024-09-16T20:49:45.589 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053955436000000002, 2024-09-16T20:49:45.589 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:45.589 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:45.589 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:45.589 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:45.589 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:45.589 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:45.589 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3576 2024-09-16T20:49:45.589 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.589 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:45.589 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.590 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:45.590 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:49:45.590 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:45.590 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.590 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.590 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:45.590 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:49:45.590 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:45.590 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.590 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.591 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:45.591 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481319+0000", 2024-09-16T20:49:45.591 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.53e-07 2024-09-16T20:49:45.591 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.591 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.591 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:45.591 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481320+0000", 2024-09-16T20:49:45.591 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5230000000000001e-06 2024-09-16T20:49:45.591 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.591 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.591 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:45.591 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481330+0000", 2024-09-16T20:49:45.592 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0190000000000001e-05 2024-09-16T20:49:45.592 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.592 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.592 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:45.592 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481333+0000", 2024-09-16T20:49:45.592 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.151e-06 2024-09-16T20:49:45.592 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.592 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.592 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:45.592 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527689+0000", 2024-09-16T20:49:45.592 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046355192000000003 2024-09-16T20:49:45.592 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.592 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.593 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:45.593 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527764+0000", 2024-09-16T20:49:45.593 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5776000000000001e-05 2024-09-16T20:49:45.593 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.593 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.593 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:45.593 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528141+0000", 2024-09-16T20:49:45.593 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00037647599999999999 2024-09-16T20:49:45.593 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.593 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.594 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:45.594 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531788+0000", 2024-09-16T20:49:45.594 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0036470679999999998 2024-09-16T20:49:45.594 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.594 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.594 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:45.594 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535174+0000", 2024-09-16T20:49:45.594 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0033862670000000001 2024-09-16T20:49:45.594 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.594 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.594 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:45.594 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535236+0000", 2024-09-16T20:49:45.594 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1797999999999993e-05 2024-09-16T20:49:45.595 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.595 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.595 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:45.595 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535274+0000", 2024-09-16T20:49:45.595 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7741999999999999e-05 2024-09-16T20:49:45.595 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:45.595 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:45.595 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:45.595 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.595 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.595 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3577 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:45.595 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:49:45.595 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 56.059520892999998, 2024-09-16T20:49:45.596 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054073807000000002, 2024-09-16T20:49:45.596 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:45.596 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:45.596 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:45.596 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:45.596 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:45.596 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:45.596 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3577 2024-09-16T20:49:45.596 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.596 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:45.596 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.596 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:45.597 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:49:45.597 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:45.597 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.597 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.597 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:45.597 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:49:45.597 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:45.597 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.597 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.597 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:45.597 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:49:45.597 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.96e-07 2024-09-16T20:49:45.598 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.598 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.598 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:45.598 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481342+0000", 2024-09-16T20:49:45.598 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.714e-06 2024-09-16T20:49:45.598 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.598 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.598 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:45.598 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481349+0000", 2024-09-16T20:49:45.598 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4930000000000001e-06 2024-09-16T20:49:45.598 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.598 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.598 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:45.599 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481353+0000", 2024-09-16T20:49:45.599 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4879999999999999e-06 2024-09-16T20:49:45.599 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.599 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.599 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:45.599 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528619+0000", 2024-09-16T20:49:45.599 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047266351999999998 2024-09-16T20:49:45.599 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.599 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.599 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:45.599 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528706+0000", 2024-09-16T20:49:45.599 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6796e-05 2024-09-16T20:49:45.600 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.600 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.600 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:45.600 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529148+0000", 2024-09-16T20:49:45.600 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00044182999999999999 2024-09-16T20:49:45.600 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.600 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.600 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:45.600 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531792+0000", 2024-09-16T20:49:45.600 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0026443119999999998 2024-09-16T20:49:45.600 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.600 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.601 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:45.601 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535319+0000", 2024-09-16T20:49:45.601 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0035266439999999998 2024-09-16T20:49:45.601 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.601 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.601 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:45.601 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535372+0000", 2024-09-16T20:49:45.601 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3207000000000001e-05 2024-09-16T20:49:45.601 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.601 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.601 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:45.601 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535414+0000", 2024-09-16T20:49:45.601 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1675000000000003e-05 2024-09-16T20:49:45.602 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:45.602 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:45.602 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:45.602 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.602 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.602 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3578 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:45.602 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:49:45.602 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 56.059499723999998, 2024-09-16T20:49:45.602 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054189586999999997, 2024-09-16T20:49:45.602 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:45.602 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:45.602 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:45.603 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:45.603 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:45.603 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:45.603 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3578 2024-09-16T20:49:45.603 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.603 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:45.603 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.603 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:45.603 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:49:45.603 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:45.603 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.603 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.603 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:45.603 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:49:45.604 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:45.604 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.604 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.604 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:45.604 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:49:45.604 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0800000000000001e-07 2024-09-16T20:49:45.604 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.604 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.604 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:45.604 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481363+0000", 2024-09-16T20:49:45.604 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0339999999999999e-06 2024-09-16T20:49:45.604 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.605 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.605 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:45.605 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481371+0000", 2024-09-16T20:49:45.605 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4440000000000002e-06 2024-09-16T20:49:45.605 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.605 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.605 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:45.605 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481374+0000", 2024-09-16T20:49:45.605 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2049999999999998e-06 2024-09-16T20:49:45.605 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.605 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.605 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:45.605 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529555+0000", 2024-09-16T20:49:45.606 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048181202999999999 2024-09-16T20:49:45.606 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.606 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.606 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:45.606 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529616+0000", 2024-09-16T20:49:45.606 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0912000000000002e-05 2024-09-16T20:49:45.606 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.606 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.606 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:45.606 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529939+0000", 2024-09-16T20:49:45.606 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00032269800000000001 2024-09-16T20:49:45.606 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.607 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.607 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:45.607 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532602+0000", 2024-09-16T20:49:45.607 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0026634940000000002 2024-09-16T20:49:45.607 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.607 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.607 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:45.607 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535465+0000", 2024-09-16T20:49:45.607 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.002862149 2024-09-16T20:49:45.607 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.607 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.607 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:45.607 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535516+0000", 2024-09-16T20:49:45.608 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1696000000000001e-05 2024-09-16T20:49:45.608 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.608 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.608 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:45.608 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535551+0000", 2024-09-16T20:49:45.608 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4443999999999998e-05 2024-09-16T20:49:45.608 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:45.608 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:45.608 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:45.608 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.608 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.608 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3579 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:45.609 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:49:45.609 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 56.059481284999997, 2024-09-16T20:49:45.609 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054295151, 2024-09-16T20:49:45.609 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:45.609 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:45.609 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:45.609 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:45.609 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:45.609 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:45.609 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3579 2024-09-16T20:49:45.609 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.609 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:45.609 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.610 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:45.610 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:49:45.610 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:45.610 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.610 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.610 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:45.610 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:49:45.610 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:45.610 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.610 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.610 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:45.610 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:49:45.610 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3300000000000001e-07 2024-09-16T20:49:45.611 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.611 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.611 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:45.611 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481381+0000", 2024-09-16T20:49:45.611 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.649e-06 2024-09-16T20:49:45.611 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.611 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.611 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:45.611 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481387+0000", 2024-09-16T20:49:45.611 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4519999999999996e-06 2024-09-16T20:49:45.611 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.611 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.612 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:45.612 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481390+0000", 2024-09-16T20:49:45.612 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0170000000000001e-06 2024-09-16T20:49:45.612 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.612 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.612 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:45.612 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530756+0000", 2024-09-16T20:49:45.612 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049365882999999999 2024-09-16T20:49:45.612 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.612 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.612 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:45.612 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530845+0000", 2024-09-16T20:49:45.613 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9197000000000007e-05 2024-09-16T20:49:45.613 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.613 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.613 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:45.613 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531191+0000", 2024-09-16T20:49:45.613 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00034562500000000001 2024-09-16T20:49:45.613 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.613 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.613 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:45.613 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534078+0000", 2024-09-16T20:49:45.613 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0028878179999999999 2024-09-16T20:49:45.613 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.613 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.614 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:45.614 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535589+0000", 2024-09-16T20:49:45.614 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00151085 2024-09-16T20:49:45.614 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.614 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.614 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:45.614 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535639+0000", 2024-09-16T20:49:45.614 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9697000000000002e-05 2024-09-16T20:49:45.614 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.614 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.614 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:45.614 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535675+0000", 2024-09-16T20:49:45.615 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5729999999999998e-05 2024-09-16T20:49:45.615 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:45.615 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:45.615 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:45.615 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.615 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.615 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3580 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:45.615 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:49:45.615 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 56.059467652999999, 2024-09-16T20:49:45.615 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054397361999999998, 2024-09-16T20:49:45.615 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:45.615 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:45.616 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:45.616 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:45.616 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:45.616 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:45.616 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3580 2024-09-16T20:49:45.616 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.616 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:45.616 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.616 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:45.616 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:49:45.616 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:45.616 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.617 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.617 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:45.617 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:49:45.617 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:45.617 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.617 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.617 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:45.617 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:49:45.617 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2600000000000001e-07 2024-09-16T20:49:45.617 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.617 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.617 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:45.618 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481395+0000", 2024-09-16T20:49:45.618 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.544e-06 2024-09-16T20:49:45.618 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.618 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.618 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:45.618 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481404+0000", 2024-09-16T20:49:45.618 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8850000000000005e-06 2024-09-16T20:49:45.618 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.618 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.618 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:45.618 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481407+0000", 2024-09-16T20:49:45.618 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1540000000000002e-06 2024-09-16T20:49:45.618 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.619 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.619 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:45.619 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531823+0000", 2024-09-16T20:49:45.619 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050415637999999999 2024-09-16T20:49:45.619 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.619 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.619 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:45.619 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531895+0000", 2024-09-16T20:49:45.619 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2179000000000001e-05 2024-09-16T20:49:45.619 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.619 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.619 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:45.619 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532236+0000", 2024-09-16T20:49:45.620 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00034073799999999999 2024-09-16T20:49:45.620 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.620 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.620 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:45.620 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534083+0000", 2024-09-16T20:49:45.620 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001847298 2024-09-16T20:49:45.620 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.620 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.620 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:45.620 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535714+0000", 2024-09-16T20:49:45.620 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0016316810000000001 2024-09-16T20:49:45.620 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.620 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.621 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:45.621 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535759+0000", 2024-09-16T20:49:45.621 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4478999999999999e-05 2024-09-16T20:49:45.621 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.621 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.621 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:45.621 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535791+0000", 2024-09-16T20:49:45.621 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.154e-05 2024-09-16T20:49:45.621 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:45.621 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:45.621 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:45.621 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.621 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.622 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3581 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:45.622 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:49:45.622 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 56.059445656000001, 2024-09-16T20:49:45.622 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.057094403000000002, 2024-09-16T20:49:45.622 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:45.622 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:45.622 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:45.622 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:45.622 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:45.622 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:45.622 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3581 2024-09-16T20:49:45.622 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.622 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:45.623 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.623 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:45.623 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:49:45.623 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:45.623 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.623 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.623 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:45.623 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:49:45.623 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:45.623 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.623 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.623 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:45.624 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:49:45.624 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8799999999999998e-07 2024-09-16T20:49:45.624 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.624 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.624 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:45.624 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481418+0000", 2024-09-16T20:49:45.624 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1859999999999999e-06 2024-09-16T20:49:45.624 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.624 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.624 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:45.624 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481423+0000", 2024-09-16T20:49:45.624 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8279999999999999e-06 2024-09-16T20:49:45.625 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.625 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.625 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:45.625 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481426+0000", 2024-09-16T20:49:45.625 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0110000000000001e-06 2024-09-16T20:49:45.625 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.625 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.625 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:45.625 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532631+0000", 2024-09-16T20:49:45.625 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051204514999999999 2024-09-16T20:49:45.625 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.626 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.626 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:45.626 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.533381+0000", 2024-09-16T20:49:45.626 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00075029000000000003 2024-09-16T20:49:45.626 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.626 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.626 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:45.626 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.533739+0000", 2024-09-16T20:49:45.626 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00035773100000000002 2024-09-16T20:49:45.626 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.627 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.627 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:45.627 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535123+0000", 2024-09-16T20:49:45.627 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0013836160000000001 2024-09-16T20:49:45.627 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.627 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.627 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:45.627 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538413+0000", 2024-09-16T20:49:45.627 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0032902510000000001 2024-09-16T20:49:45.627 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.627 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.627 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:45.627 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538470+0000", 2024-09-16T20:49:45.628 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6697999999999998e-05 2024-09-16T20:49:45.628 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.628 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.628 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:45.628 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538510+0000", 2024-09-16T20:49:45.628 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9988999999999999e-05 2024-09-16T20:49:45.628 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:45.628 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:45.628 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:45.628 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.628 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.629 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:12732 3.d 3:b752444a:::rbd_header.110e747f0779:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-16T20:49:45.629 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:49:45.629 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 40.772373078000001, 2024-09-16T20:49:45.629 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.065302467000000003, 2024-09-16T20:49:45.629 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:45.629 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:45.629 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:45.629 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:45.629 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:49:45.629 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:49:45.629 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 12732 2024-09-16T20:49:45.630 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.630 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:45.630 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.630 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:45.630 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:49:45.630 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:45.630 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.630 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.630 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:45.630 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:49:45.631 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:45.631 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.631 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.632 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:45.632 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768490+0000", 2024-09-16T20:49:45.633 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2859999999999998e-06 2024-09-16T20:49:45.633 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.633 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.633 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:45.633 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768501+0000", 2024-09-16T20:49:45.633 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1226999999999999e-05 2024-09-16T20:49:45.633 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.633 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.633 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:45.633 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768517+0000", 2024-09-16T20:49:45.633 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.592e-05 2024-09-16T20:49:45.633 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.633 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.634 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:45.634 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768528+0000", 2024-09-16T20:49:45.634 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0689999999999999e-05 2024-09-16T20:49:45.634 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.634 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.634 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:45.634 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768584+0000", 2024-09-16T20:49:45.634 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5889e-05 2024-09-16T20:49:45.634 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.634 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.634 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:45.634 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768720+0000", 2024-09-16T20:49:45.634 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000136718 2024-09-16T20:49:45.635 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.635 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.635 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:45.635 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.769066+0000", 2024-09-16T20:49:45.635 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000346013 2024-09-16T20:49:45.635 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.635 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.635 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:45.635 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833659+0000", 2024-09-16T20:49:45.635 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.064592822999999994 2024-09-16T20:49:45.635 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.635 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.635 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:45.636 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833719+0000", 2024-09-16T20:49:45.636 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9441000000000002e-05 2024-09-16T20:49:45.636 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.636 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.636 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:45.636 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833758+0000", 2024-09-16T20:49:45.636 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9122000000000003e-05 2024-09-16T20:49:45.636 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.636 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.636 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:45.636 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833790+0000", 2024-09-16T20:49:45.636 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2338000000000001e-05 2024-09-16T20:49:45.636 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:45.637 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:45.637 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:45.637 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.637 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.637 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.5089.0:12732 3.d e73/23 ondisk, result = 0)", 2024-09-16T20:49:45.637 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:49:45.637 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 40.770770908000003, 2024-09-16T20:49:45.637 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.063706456999999994, 2024-09-16T20:49:45.637 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:45.637 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:45.637 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:45.637 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:45.637 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.637 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:45.638 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:49:45.638 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:45.638 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.638 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.638 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:45.638 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:49:45.638 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:45.638 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.638 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.638 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:45.638 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770091+0000", 2024-09-16T20:49:45.638 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.53e-06 2024-09-16T20:49:45.638 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.638 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.639 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:45.639 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770104+0000", 2024-09-16T20:49:45.639 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2203e-05 2024-09-16T20:49:45.639 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.639 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.639 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:45.639 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833637+0000", 2024-09-16T20:49:45.639 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.063533201999999997 2024-09-16T20:49:45.639 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.639 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.639 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:45.639 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833655+0000", 2024-09-16T20:49:45.639 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8369e-05 2024-09-16T20:49:45.640 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.640 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.640 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:45.640 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833707+0000", 2024-09-16T20:49:45.640 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2061999999999997e-05 2024-09-16T20:49:45.640 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.640 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.640 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:45.640 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833715+0000", 2024-09-16T20:49:45.640 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5179999999999998e-06 2024-09-16T20:49:45.640 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.640 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.640 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:45.640 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833796+0000", 2024-09-16T20:49:45.641 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1573e-05 2024-09-16T20:49:45.641 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:45.641 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:45.641 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:45.641 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.641 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.641 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:363 3.10 3:0ac2a405:::rbd_data.110e39315da6.00000000000000cc:1f [read 385024~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:49:45.641 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246644+0000", 2024-09-16T20:49:45.641 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 9.2942168219999992, 2024-09-16T20:49:45.641 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050074721000000003, 2024-09-16T20:49:45.641 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:45.641 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:45.641 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:45.641 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:45.642 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:49:45.642 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:49:45.642 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 363 2024-09-16T20:49:45.642 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.642 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:45.642 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.642 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:45.642 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246644+0000", 2024-09-16T20:49:45.642 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:45.642 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.642 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.642 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:45.642 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246644+0000", 2024-09-16T20:49:45.642 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:45.643 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.643 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.643 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:45.643 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246645+0000", 2024-09-16T20:49:45.643 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.44e-07 2024-09-16T20:49:45.643 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.643 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.643 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:45.643 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246647+0000", 2024-09-16T20:49:45.643 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.712e-06 2024-09-16T20:49:45.643 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.643 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.643 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:45.643 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246655+0000", 2024-09-16T20:49:45.643 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8299999999999996e-06 2024-09-16T20:49:45.644 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.644 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.644 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:45.644 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246660+0000", 2024-09-16T20:49:45.644 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5040000000000004e-06 2024-09-16T20:49:45.644 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.644 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.644 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:45.644 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.247858+0000", 2024-09-16T20:49:45.644 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0011981019999999999 2024-09-16T20:49:45.644 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.644 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.644 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:45.645 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.247928+0000", 2024-09-16T20:49:45.645 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9919999999999995e-05 2024-09-16T20:49:45.645 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.645 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.645 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:45.645 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.296719+0000", 2024-09-16T20:49:45.645 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048791108999999999 2024-09-16T20:49:45.645 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:45.645 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:45.645 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:45.645 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.645 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.645 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:364 3.10 3:0ac2a405:::rbd_data.110e39315da6.00000000000000cc:1f [read 331776~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:49:45.645 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246686+0000", 2024-09-16T20:49:45.646 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 9.2941747840000009, 2024-09-16T20:49:45.646 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050212992999999997, 2024-09-16T20:49:45.646 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:45.646 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:45.646 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:45.646 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:45.646 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:49:45.646 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:49:45.646 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 364 2024-09-16T20:49:45.646 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.646 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:45.646 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.646 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:45.646 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246686+0000", 2024-09-16T20:49:45.646 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:45.647 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.647 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.647 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:45.647 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246686+0000", 2024-09-16T20:49:45.647 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:45.647 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.647 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.647 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:45.647 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246687+0000", 2024-09-16T20:49:45.647 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3600000000000004e-07 2024-09-16T20:49:45.647 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.647 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.647 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:45.648 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246689+0000", 2024-09-16T20:49:45.648 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.4049999999999998e-06 2024-09-16T20:49:45.648 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.648 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.648 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:45.648 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246694+0000", 2024-09-16T20:49:45.648 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7079999999999996e-06 2024-09-16T20:49:45.648 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.648 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.648 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:45.648 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246697+0000", 2024-09-16T20:49:45.648 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1190000000000001e-06 2024-09-16T20:49:45.648 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.648 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.649 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:45.649 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.296755+0000", 2024-09-16T20:49:45.649 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050058004000000003 2024-09-16T20:49:45.649 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.649 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.649 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:45.649 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.296821+0000", 2024-09-16T20:49:45.649 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5759000000000005e-05 2024-09-16T20:49:45.649 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.649 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.649 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:45.649 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.296899+0000", 2024-09-16T20:49:45.649 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8461999999999995e-05 2024-09-16T20:49:45.649 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:45.650 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:45.650 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:45.650 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.650 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.650 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:365 3.10 3:0ac2a405:::rbd_data.110e39315da6.00000000000000cc:1f [read 352256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:49:45.650 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246724+0000", 2024-09-16T20:49:45.650 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 9.2941372930000004, 2024-09-16T20:49:45.650 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050319864999999998, 2024-09-16T20:49:45.650 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:45.650 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:45.650 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:45.650 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:45.650 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:49:45.650 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:49:45.650 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 365 2024-09-16T20:49:45.651 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.651 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:45.651 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.651 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:45.651 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246724+0000", 2024-09-16T20:49:45.651 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:45.651 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.651 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.651 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:45.651 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246724+0000", 2024-09-16T20:49:45.651 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:45.651 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.651 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.651 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:45.651 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246724+0000", 2024-09-16T20:49:45.652 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3900000000000005e-07 2024-09-16T20:49:45.652 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.652 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.652 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:45.652 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246727+0000", 2024-09-16T20:49:45.652 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9270000000000001e-06 2024-09-16T20:49:45.652 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.652 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.652 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:45.652 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246734+0000", 2024-09-16T20:49:45.652 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5339999999999996e-06 2024-09-16T20:49:45.652 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.652 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.652 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:45.653 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246739+0000", 2024-09-16T20:49:45.653 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4650000000000002e-06 2024-09-16T20:49:45.653 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.653 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.653 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:45.653 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.296935+0000", 2024-09-16T20:49:45.653 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050196159999999997 2024-09-16T20:49:45.653 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.653 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.653 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:45.653 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.296984+0000", 2024-09-16T20:49:45.653 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9190000000000002e-05 2024-09-16T20:49:45.653 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.653 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.654 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:45.654 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297043+0000", 2024-09-16T20:49:45.654 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9049999999999999e-05 2024-09-16T20:49:45.654 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:45.654 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:45.654 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:45.654 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.654 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.654 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:366 3.10 3:0ac2a405:::rbd_data.110e39315da6.00000000000000cc:1f [read 360448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:49:45.654 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246748+0000", 2024-09-16T20:49:45.654 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 9.2941126930000006, 2024-09-16T20:49:45.654 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050466466000000001, 2024-09-16T20:49:45.654 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:45.655 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:45.655 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:45.655 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:45.655 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:49:45.655 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:49:45.655 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 366 2024-09-16T20:49:45.655 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.655 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:45.655 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.655 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:45.655 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246748+0000", 2024-09-16T20:49:45.655 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:45.655 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.655 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.656 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:45.656 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246748+0000", 2024-09-16T20:49:45.656 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:45.656 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.656 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.656 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:45.656 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246749+0000", 2024-09-16T20:49:45.656 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.58e-07 2024-09-16T20:49:45.656 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.656 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.656 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:45.656 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246751+0000", 2024-09-16T20:49:45.656 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3719999999999999e-06 2024-09-16T20:49:45.657 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.657 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.657 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:45.657 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246757+0000", 2024-09-16T20:49:45.657 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1789999999999996e-06 2024-09-16T20:49:45.657 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.657 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.657 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:45.657 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246762+0000", 2024-09-16T20:49:45.657 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3000000000000001e-06 2024-09-16T20:49:45.657 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.657 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.657 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:45.657 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297086+0000", 2024-09-16T20:49:45.658 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050323248000000001 2024-09-16T20:49:45.658 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.658 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.658 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:45.658 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297160+0000", 2024-09-16T20:49:45.658 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4615000000000003e-05 2024-09-16T20:49:45.658 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.658 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.658 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:45.658 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297215+0000", 2024-09-16T20:49:45.658 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4293999999999999e-05 2024-09-16T20:49:45.658 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:45.658 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:45.659 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:45.659 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.659 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.659 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:367 3.10 3:0ac2a405:::rbd_data.110e39315da6.00000000000000cc:1f [read 372736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:49:45.659 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246769+0000", 2024-09-16T20:49:45.659 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 9.2940916490000003, 2024-09-16T20:49:45.659 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050565820999999997, 2024-09-16T20:49:45.659 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:45.659 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:45.659 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:45.659 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:45.659 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:49:45.659 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:49:45.660 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 367 2024-09-16T20:49:45.660 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.660 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:45.660 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.660 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:45.660 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246769+0000", 2024-09-16T20:49:45.660 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:45.660 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.660 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.660 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:45.660 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246769+0000", 2024-09-16T20:49:45.660 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:45.660 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.661 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.661 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:45.661 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246770+0000", 2024-09-16T20:49:45.661 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0499999999999999e-07 2024-09-16T20:49:45.661 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.661 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.661 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:45.661 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246772+0000", 2024-09-16T20:49:45.661 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0959999999999999e-06 2024-09-16T20:49:45.661 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.661 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.661 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:45.661 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246778+0000", 2024-09-16T20:49:45.662 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2210000000000002e-06 2024-09-16T20:49:45.662 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.662 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.662 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:45.662 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246783+0000", 2024-09-16T20:49:45.662 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9080000000000003e-06 2024-09-16T20:49:45.662 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.662 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.662 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:45.662 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297243+0000", 2024-09-16T20:49:45.662 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050460307000000003 2024-09-16T20:49:45.662 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.662 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.662 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:45.663 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297284+0000", 2024-09-16T20:49:45.663 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1213e-05 2024-09-16T20:49:45.663 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.663 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.663 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:45.663 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297335+0000", 2024-09-16T20:49:45.663 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0671000000000003e-05 2024-09-16T20:49:45.663 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:45.663 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:45.663 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:45.663 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.663 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.663 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:372 3.10 3:0ac2a405:::rbd_data.110e39315da6.00000000000000cc:1f [read 380928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:49:45.664 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246793+0000", 2024-09-16T20:49:45.664 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 9.2940676829999997, 2024-09-16T20:49:45.664 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050657788000000002, 2024-09-16T20:49:45.664 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:45.664 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:45.664 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:45.664 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:45.664 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:49:45.664 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:49:45.664 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 372 2024-09-16T20:49:45.664 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.664 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:45.664 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.664 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:45.665 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246793+0000", 2024-09-16T20:49:45.665 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:45.665 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.665 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.665 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:45.665 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246793+0000", 2024-09-16T20:49:45.665 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:45.665 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.665 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.665 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:45.665 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246794+0000", 2024-09-16T20:49:45.665 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5499999999999998e-07 2024-09-16T20:49:45.665 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.665 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.665 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:45.666 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246796+0000", 2024-09-16T20:49:45.666 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2840000000000001e-06 2024-09-16T20:49:45.666 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.666 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.666 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:45.666 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246802+0000", 2024-09-16T20:49:45.666 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6000000000000003e-06 2024-09-16T20:49:45.666 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.666 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.666 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:45.666 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246808+0000", 2024-09-16T20:49:45.666 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3800000000000002e-06 2024-09-16T20:49:45.666 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.667 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.667 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:45.667 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297358+0000", 2024-09-16T20:49:45.667 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050550204000000001 2024-09-16T20:49:45.667 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.667 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.667 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:45.667 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297401+0000", 2024-09-16T20:49:45.667 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3278999999999997e-05 2024-09-16T20:49:45.667 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:45.667 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:45.667 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:45.667 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297451+0000", 2024-09-16T20:49:45.668 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9586000000000003e-05 2024-09-16T20:49:45.668 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:45.668 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:45.668 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:45.668 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:45.668 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:45.668 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:45.668 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T20:49:45.670 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:49:45.670 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:45.668+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:45.770 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:45.768+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:45.821 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:45.821 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:49:45.821 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:49:45.821 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:45.831 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:49:45.872 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:45.871+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:45.972 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:45.971+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:45.978 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:45.978 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:49:45.978 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:49:45.979 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:49:45.979 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:45.987 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:49:46.072 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:46.071+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:46.095 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:46.096 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:49:46.096 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:49:46.096 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:49:46.096 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.096 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3712 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1472512~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:46.096 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:49:46.096 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 83.455004295999998, 2024-09-16T20:49:46.096 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050004305999999998, 2024-09-16T20:49:46.096 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:46.096 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:46.096 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:46.096 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:46.096 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:46.097 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:46.097 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3712 2024-09-16T20:49:46.097 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.097 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:46.097 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.097 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:46.097 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:49:46.097 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:46.097 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.097 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.097 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:46.097 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:49:46.097 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:46.098 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.098 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.098 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:46.098 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:49:46.098 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0199999999999998e-07 2024-09-16T20:49:46.098 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.098 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.098 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:46.098 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638990+0000", 2024-09-16T20:49:46.098 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.618e-06 2024-09-16T20:49:46.098 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.098 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.098 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:46.099 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638995+0000", 2024-09-16T20:49:46.099 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3940000000000004e-06 2024-09-16T20:49:46.099 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.099 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.099 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:46.099 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639000+0000", 2024-09-16T20:49:46.099 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9309999999999999e-06 2024-09-16T20:49:46.099 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.099 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.099 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:46.099 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688867+0000", 2024-09-16T20:49:46.099 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049866955999999997 2024-09-16T20:49:46.099 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.100 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.100 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:46.100 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688933+0000", 2024-09-16T20:49:46.100 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5359999999999998e-05 2024-09-16T20:49:46.100 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.100 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.100 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:46.100 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688992+0000", 2024-09-16T20:49:46.100 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9744999999999998e-05 2024-09-16T20:49:46.100 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:46.100 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:46.100 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:46.100 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.101 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.101 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3714 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1480704~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:46.101 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:49:46.101 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 83.454986706, 2024-09-16T20:49:46.101 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050152552000000003, 2024-09-16T20:49:46.101 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:46.101 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:46.101 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:46.101 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:46.101 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:46.101 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:46.101 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3714 2024-09-16T20:49:46.101 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.101 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:46.102 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.102 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:46.102 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:49:46.102 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:46.102 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.102 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.102 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:46.102 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:49:46.102 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:46.102 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.102 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.102 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:46.102 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:49:46.102 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0499999999999999e-07 2024-09-16T20:49:46.103 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.103 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.103 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:46.103 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639007+0000", 2024-09-16T20:49:46.103 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5340000000000001e-06 2024-09-16T20:49:46.103 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.103 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.103 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:46.103 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639013+0000", 2024-09-16T20:49:46.103 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3260000000000003e-06 2024-09-16T20:49:46.103 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.103 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.103 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:46.104 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639018+0000", 2024-09-16T20:49:46.104 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2850000000000002e-06 2024-09-16T20:49:46.104 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.104 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.104 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:46.104 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689026+0000", 2024-09-16T20:49:46.104 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.05000798 2024-09-16T20:49:46.104 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.104 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.104 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:46.104 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689077+0000", 2024-09-16T20:49:46.104 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1063999999999998e-05 2024-09-16T20:49:46.104 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.104 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.105 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:46.105 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689158+0000", 2024-09-16T20:49:46.105 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1057999999999995e-05 2024-09-16T20:49:46.105 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:46.105 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:46.105 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:46.105 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.105 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.105 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3716 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1488896~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:46.105 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:49:46.105 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 83.454968895999997, 2024-09-16T20:49:46.105 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050280142, 2024-09-16T20:49:46.105 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:46.106 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:46.106 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:46.106 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:46.106 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:46.106 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:46.106 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3716 2024-09-16T20:49:46.106 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.106 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:46.106 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.106 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:46.106 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:49:46.106 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:46.106 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:49:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639024+0000", 2024-09-16T20:49:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9999999999999997e-07 2024-09-16T20:49:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639026+0000", 2024-09-16T20:49:46.108 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.781e-06 2024-09-16T20:49:46.108 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.108 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.108 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:46.108 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639032+0000", 2024-09-16T20:49:46.108 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7780000000000004e-06 2024-09-16T20:49:46.108 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.108 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.108 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:46.108 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639036+0000", 2024-09-16T20:49:46.108 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6909999999999996e-06 2024-09-16T20:49:46.108 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.108 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.108 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:46.109 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689191+0000", 2024-09-16T20:49:46.109 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050155284000000001 2024-09-16T20:49:46.109 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.109 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.109 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:46.109 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689242+0000", 2024-09-16T20:49:46.109 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0662999999999999e-05 2024-09-16T20:49:46.109 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.109 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.109 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:46.109 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689304+0000", 2024-09-16T20:49:46.109 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1345000000000003e-05 2024-09-16T20:49:46.109 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:46.109 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:46.110 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:46.110 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.110 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.110 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3719 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1497088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:46.110 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:49:46.110 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 83.454941618000007, 2024-09-16T20:49:46.110 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050399715999999997, 2024-09-16T20:49:46.110 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:46.110 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:46.110 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:46.110 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:46.110 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:46.111 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:46.111 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3719 2024-09-16T20:49:46.111 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.111 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:46.111 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.111 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:46.111 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:49:46.111 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:46.111 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.111 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.111 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:46.111 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:49:46.111 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:46.111 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.112 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.112 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:46.112 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:49:46.112 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9700000000000002e-07 2024-09-16T20:49:46.112 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.112 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.112 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:46.112 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639053+0000", 2024-09-16T20:49:46.112 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2900000000000001e-06 2024-09-16T20:49:46.112 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.112 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.112 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:46.112 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639060+0000", 2024-09-16T20:49:46.112 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7750000000000004e-06 2024-09-16T20:49:46.113 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.113 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.113 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:46.113 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639066+0000", 2024-09-16T20:49:46.113 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0639999999999999e-06 2024-09-16T20:49:46.113 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.113 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.113 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:46.113 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689337+0000", 2024-09-16T20:49:46.113 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050270784999999998 2024-09-16T20:49:46.113 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.113 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.113 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:46.113 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689392+0000", 2024-09-16T20:49:46.114 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4920999999999998e-05 2024-09-16T20:49:46.114 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.114 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.114 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:46.114 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689450+0000", 2024-09-16T20:49:46.114 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8483999999999998e-05 2024-09-16T20:49:46.114 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:46.114 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:46.114 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:46.114 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.114 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.114 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3721 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1505280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:46.114 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:49:46.114 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 83.454918485999997, 2024-09-16T20:49:46.115 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050513608000000002, 2024-09-16T20:49:46.115 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:46.115 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:46.115 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:46.115 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:46.115 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:46.115 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:46.115 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3721 2024-09-16T20:49:46.115 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.115 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:46.115 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.115 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:46.115 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:49:46.115 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:46.116 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.116 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.116 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:46.116 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:49:46.116 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:46.116 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.116 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.116 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:46.116 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:49:46.116 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7e-07 2024-09-16T20:49:46.116 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.116 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.116 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:46.116 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639076+0000", 2024-09-16T20:49:46.117 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8899999999999999e-06 2024-09-16T20:49:46.117 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.117 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.117 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:46.117 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639083+0000", 2024-09-16T20:49:46.117 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1210000000000001e-06 2024-09-16T20:49:46.117 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.117 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.117 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:46.117 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639089+0000", 2024-09-16T20:49:46.117 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.643e-06 2024-09-16T20:49:46.117 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.117 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.118 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:46.118 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689474+0000", 2024-09-16T20:49:46.118 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050385679000000003 2024-09-16T20:49:46.118 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.118 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.118 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:46.118 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689533+0000", 2024-09-16T20:49:46.118 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8303000000000001e-05 2024-09-16T20:49:46.118 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.118 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.118 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:46.118 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689587+0000", 2024-09-16T20:49:46.118 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4601999999999997e-05 2024-09-16T20:49:46.118 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:46.119 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:46.119 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:46.119 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.119 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.119 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3723 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1513472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:46.119 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:49:46.119 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 83.454887682000006, 2024-09-16T20:49:46.119 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050699691999999998, 2024-09-16T20:49:46.119 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:46.119 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:46.119 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:46.119 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:46.119 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:46.119 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:46.120 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3723 2024-09-16T20:49:46.120 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.120 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:46.120 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.120 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:46.120 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:49:46.120 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:46.120 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.120 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.120 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:46.120 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:49:46.120 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:46.120 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.120 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.121 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:46.121 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:49:46.121 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3399999999999999e-07 2024-09-16T20:49:46.121 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.121 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.121 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:46.121 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639107+0000", 2024-09-16T20:49:46.121 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3860000000000001e-06 2024-09-16T20:49:46.121 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.121 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.121 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:46.121 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639114+0000", 2024-09-16T20:49:46.121 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9800000000000001e-06 2024-09-16T20:49:46.121 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.122 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.122 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:46.122 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639129+0000", 2024-09-16T20:49:46.122 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4887e-05 2024-09-16T20:49:46.122 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.122 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.122 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:46.122 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689625+0000", 2024-09-16T20:49:46.122 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050495848000000003 2024-09-16T20:49:46.122 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.122 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.122 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:46.122 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689720+0000", 2024-09-16T20:49:46.122 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4821000000000006e-05 2024-09-16T20:49:46.122 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.123 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.123 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:46.123 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689804+0000", 2024-09-16T20:49:46.123 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4235999999999998e-05 2024-09-16T20:49:46.123 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:46.123 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:46.123 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:46.123 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.123 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.123 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3725 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1521664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:46.123 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:49:46.123 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 83.454854085999997, 2024-09-16T20:49:46.123 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050797660000000001, 2024-09-16T20:49:46.123 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:46.124 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:46.124 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:46.124 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:46.124 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:46.124 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:46.124 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3725 2024-09-16T20:49:46.124 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.124 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:46.124 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.124 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:46.124 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:49:46.124 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:46.124 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.125 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.125 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:46.125 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:49:46.125 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:46.125 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.125 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.125 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:46.125 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639139+0000", 2024-09-16T20:49:46.125 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8999999999999997e-07 2024-09-16T20:49:46.125 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.125 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.125 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:46.125 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639141+0000", 2024-09-16T20:49:46.125 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.182e-06 2024-09-16T20:49:46.126 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.126 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.126 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:46.126 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639154+0000", 2024-09-16T20:49:46.126 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3626e-05 2024-09-16T20:49:46.126 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.126 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.126 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:46.126 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639160+0000", 2024-09-16T20:49:46.126 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6810000000000003e-06 2024-09-16T20:49:46.126 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.126 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.126 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:46.127 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689831+0000", 2024-09-16T20:49:46.127 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050670907000000001 2024-09-16T20:49:46.127 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.127 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.127 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:46.127 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689882+0000", 2024-09-16T20:49:46.127 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0445999999999998e-05 2024-09-16T20:49:46.127 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.127 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.127 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:46.127 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689936+0000", 2024-09-16T20:49:46.127 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4327999999999999e-05 2024-09-16T20:49:46.127 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:46.127 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:46.127 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:46.128 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.128 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.128 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3727 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1529856~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:46.128 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:49:46.128 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 83.454818509000006, 2024-09-16T20:49:46.128 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051016922999999999, 2024-09-16T20:49:46.128 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:46.128 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:46.128 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:46.128 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:46.128 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:46.128 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:46.128 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3727 2024-09-16T20:49:46.129 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.129 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:46.129 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.129 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:46.129 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:49:46.129 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:46.129 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.129 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.129 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:46.129 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:49:46.129 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:46.129 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.129 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.129 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:46.130 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:49:46.130 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9999999999999998e-07 2024-09-16T20:49:46.130 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.130 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.130 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:46.130 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639183+0000", 2024-09-16T20:49:46.130 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2399999999999996e-06 2024-09-16T20:49:46.130 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.130 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.130 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:46.130 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639195+0000", 2024-09-16T20:49:46.130 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1113e-05 2024-09-16T20:49:46.130 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.130 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.131 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:46.131 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639206+0000", 2024-09-16T20:49:46.131 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1503e-05 2024-09-16T20:49:46.131 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.131 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.131 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:46.131 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689969+0000", 2024-09-16T20:49:46.131 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050762676 2024-09-16T20:49:46.131 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.131 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.131 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:46.131 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690071+0000", 2024-09-16T20:49:46.131 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102163 2024-09-16T20:49:46.131 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.131 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.132 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:46.132 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690191+0000", 2024-09-16T20:49:46.132 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000119728 2024-09-16T20:49:46.132 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:46.132 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:46.132 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:46.132 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.132 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.132 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3730 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1538048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:46.132 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:49:46.132 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 83.454776945999996, 2024-09-16T20:49:46.132 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051131467999999999, 2024-09-16T20:49:46.132 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:46.132 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:46.133 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:46.133 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:46.133 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:46.133 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:46.133 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3730 2024-09-16T20:49:46.133 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.133 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:46.133 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.133 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:46.133 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:49:46.133 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:46.133 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.133 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.133 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:46.134 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:49:46.134 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:46.134 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.134 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.134 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:46.134 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639216+0000", 2024-09-16T20:49:46.134 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.39e-07 2024-09-16T20:49:46.134 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.134 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.134 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:46.134 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639218+0000", 2024-09-16T20:49:46.134 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.537e-06 2024-09-16T20:49:46.134 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.134 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.134 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:46.135 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639229+0000", 2024-09-16T20:49:46.135 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1122e-05 2024-09-16T20:49:46.135 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.135 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.135 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:46.135 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639237+0000", 2024-09-16T20:49:46.135 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8509999999999999e-06 2024-09-16T20:49:46.135 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.135 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.135 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:46.135 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690227+0000", 2024-09-16T20:49:46.135 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050990147 2024-09-16T20:49:46.135 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.135 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.136 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:46.136 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690278+0000", 2024-09-16T20:49:46.136 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0223999999999999e-05 2024-09-16T20:49:46.136 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.136 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.136 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:46.136 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690347+0000", 2024-09-16T20:49:46.136 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9147999999999996e-05 2024-09-16T20:49:46.136 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:46.136 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:46.136 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:46.136 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.136 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.136 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3731 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1546240~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:46.137 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:49:46.137 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 83.454746237999998, 2024-09-16T20:49:46.137 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051276744999999999, 2024-09-16T20:49:46.137 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:46.137 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:46.137 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:46.137 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:46.137 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:46.137 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:46.137 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3731 2024-09-16T20:49:46.137 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.137 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:46.137 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.137 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:46.137 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:49:46.138 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:46.138 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.138 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.138 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:46.138 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:49:46.138 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:46.138 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.138 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.138 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:46.138 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:49:46.138 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9900000000000001e-07 2024-09-16T20:49:46.138 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.138 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.138 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:46.139 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639249+0000", 2024-09-16T20:49:46.139 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3630000000000001e-06 2024-09-16T20:49:46.139 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.139 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.139 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:46.139 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639255+0000", 2024-09-16T20:49:46.139 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4960000000000001e-06 2024-09-16T20:49:46.139 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.139 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.139 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:46.139 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639261+0000", 2024-09-16T20:49:46.139 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2839999999999997e-06 2024-09-16T20:49:46.139 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.139 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.140 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:46.140 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690375+0000", 2024-09-16T20:49:46.140 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051114684000000001 2024-09-16T20:49:46.140 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.140 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.140 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:46.140 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690425+0000", 2024-09-16T20:49:46.140 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9379999999999998e-05 2024-09-16T20:49:46.140 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.140 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.140 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:46.140 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690523+0000", 2024-09-16T20:49:46.140 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8139000000000003e-05 2024-09-16T20:49:46.140 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:46.141 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:46.141 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:46.141 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.141 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.141 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3732 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1554432~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:46.141 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:49:46.141 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 83.454700455999998, 2024-09-16T20:49:46.141 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051380096, 2024-09-16T20:49:46.141 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:46.141 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:46.141 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:46.141 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:46.141 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:46.141 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:46.142 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3732 2024-09-16T20:49:46.142 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.142 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:46.142 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.142 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:46.142 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:49:46.142 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:46.142 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.142 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.142 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:46.142 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:49:46.142 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:46.142 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.142 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.142 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:46.143 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:49:46.143 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6100000000000001e-07 2024-09-16T20:49:46.143 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.143 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.143 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:46.143 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639295+0000", 2024-09-16T20:49:46.143 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.447e-06 2024-09-16T20:49:46.143 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.143 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.143 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:46.143 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639302+0000", 2024-09-16T20:49:46.143 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0570000000000004e-06 2024-09-16T20:49:46.143 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.143 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.144 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:46.144 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639307+0000", 2024-09-16T20:49:46.144 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1429999999999997e-06 2024-09-16T20:49:46.144 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.144 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.144 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:46.144 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690555+0000", 2024-09-16T20:49:46.144 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051247963000000001 2024-09-16T20:49:46.144 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.144 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.144 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:46.144 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690602+0000", 2024-09-16T20:49:46.144 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7602000000000003e-05 2024-09-16T20:49:46.144 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.145 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.145 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:46.145 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690672+0000", 2024-09-16T20:49:46.145 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9423000000000003e-05 2024-09-16T20:49:46.145 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:46.145 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:46.145 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:46.145 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.145 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.145 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3734 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1562624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:46.145 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:49:46.145 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 83.454678806000004, 2024-09-16T20:49:46.145 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051490610999999999, 2024-09-16T20:49:46.145 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:46.146 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:46.146 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:46.146 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:46.146 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:46.146 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:46.146 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3734 2024-09-16T20:49:46.146 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.146 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:46.146 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.146 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:46.146 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:49:46.146 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:46.146 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.146 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.146 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:46.147 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:49:46.147 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:46.147 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.147 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.147 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:46.147 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639314+0000", 2024-09-16T20:49:46.147 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4400000000000001e-07 2024-09-16T20:49:46.147 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.147 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.147 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:46.147 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639316+0000", 2024-09-16T20:49:46.147 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1459999999999999e-06 2024-09-16T20:49:46.147 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.147 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.147 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:46.148 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639322+0000", 2024-09-16T20:49:46.148 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7610000000000004e-06 2024-09-16T20:49:46.148 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.148 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.148 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:46.148 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639327+0000", 2024-09-16T20:49:46.148 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.186e-06 2024-09-16T20:49:46.148 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.148 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.148 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:46.148 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690698+0000", 2024-09-16T20:49:46.148 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051371223000000001 2024-09-16T20:49:46.148 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.149 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.149 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:46.149 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690748+0000", 2024-09-16T20:49:46.149 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0192999999999999e-05 2024-09-16T20:49:46.149 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.149 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.149 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:46.149 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690804+0000", 2024-09-16T20:49:46.149 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5658000000000001e-05 2024-09-16T20:49:46.149 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:46.149 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:46.149 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:46.149 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.149 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.150 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3736 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1570816~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:46.150 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:49:46.150 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 83.454659547000006, 2024-09-16T20:49:46.150 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051596528000000003, 2024-09-16T20:49:46.150 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:46.150 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:46.150 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:46.150 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:46.150 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:46.150 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:46.150 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3736 2024-09-16T20:49:46.150 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.150 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:46.150 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.150 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:46.151 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:49:46.151 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:46.151 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.151 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.151 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:46.151 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:49:46.151 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:46.151 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.151 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.151 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:46.151 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:49:46.151 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9999999999999999e-07 2024-09-16T20:49:46.151 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.151 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.152 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:46.152 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639335+0000", 2024-09-16T20:49:46.152 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.635e-06 2024-09-16T20:49:46.152 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.152 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.152 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:46.152 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639340+0000", 2024-09-16T20:49:46.152 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5840000000000002e-06 2024-09-16T20:49:46.152 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.152 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.152 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:46.152 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639346+0000", 2024-09-16T20:49:46.152 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1190000000000002e-06 2024-09-16T20:49:46.152 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.153 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.153 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:46.153 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690817+0000", 2024-09-16T20:49:46.153 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051470721999999997 2024-09-16T20:49:46.153 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.153 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.153 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:46.153 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690875+0000", 2024-09-16T20:49:46.153 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7441000000000001e-05 2024-09-16T20:49:46.153 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.153 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.153 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:46.153 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690929+0000", 2024-09-16T20:49:46.153 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4727e-05 2024-09-16T20:49:46.154 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:46.154 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:46.154 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:46.154 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.154 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.154 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5434 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:46.154 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:49:46.154 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 83.229397981000005, 2024-09-16T20:49:46.154 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099755831000000003, 2024-09-16T20:49:46.154 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:46.154 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:46.154 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:46.154 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:46.154 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:46.154 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:46.155 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5434 2024-09-16T20:49:46.155 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.155 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:46.155 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.155 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:46.155 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:49:46.155 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:46.155 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.155 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.155 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:46.155 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:49:46.155 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:46.155 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.155 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.156 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:46.156 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864595+0000", 2024-09-16T20:49:46.156 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1099999999999995e-07 2024-09-16T20:49:46.156 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.156 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.156 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:46.156 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864597+0000", 2024-09-16T20:49:46.156 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3989999999999998e-06 2024-09-16T20:49:46.156 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.156 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.156 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:46.156 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864603+0000", 2024-09-16T20:49:46.156 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9309999999999996e-06 2024-09-16T20:49:46.156 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.157 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.157 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:46.157 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864607+0000", 2024-09-16T20:49:46.157 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6200000000000001e-06 2024-09-16T20:49:46.157 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.157 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.157 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:46.157 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964164+0000", 2024-09-16T20:49:46.157 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099557162000000005 2024-09-16T20:49:46.157 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.157 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.157 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:46.157 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964244+0000", 2024-09-16T20:49:46.157 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9597000000000004e-05 2024-09-16T20:49:46.158 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.158 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.158 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:46.158 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964350+0000", 2024-09-16T20:49:46.158 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010641099999999999 2024-09-16T20:49:46.158 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:46.158 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:46.158 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:46.158 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.158 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.158 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5435 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 20480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:46.158 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:49:46.158 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 83.229368159000003, 2024-09-16T20:49:46.158 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100897102, 2024-09-16T20:49:46.159 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:46.159 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:46.159 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:46.159 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:46.159 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:46.159 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:46.159 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5435 2024-09-16T20:49:46.159 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.159 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:46.159 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.159 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:46.159 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:49:46.159 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:46.159 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.160 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.160 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:46.160 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:49:46.160 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:46.160 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.160 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.160 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:46.160 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:49:46.160 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8000000000000001e-07 2024-09-16T20:49:46.160 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.160 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.160 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:46.160 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864626+0000", 2024-09-16T20:49:46.160 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.745e-06 2024-09-16T20:49:46.161 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.161 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.161 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:46.161 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864633+0000", 2024-09-16T20:49:46.161 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3339999999999998e-06 2024-09-16T20:49:46.161 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.161 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.161 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:46.161 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864636+0000", 2024-09-16T20:49:46.161 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1200000000000002e-06 2024-09-16T20:49:46.161 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.161 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.161 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:46.161 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965299+0000", 2024-09-16T20:49:46.162 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10066352000000001 2024-09-16T20:49:46.162 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.162 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.162 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:46.162 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965390+0000", 2024-09-16T20:49:46.162 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0894000000000001e-05 2024-09-16T20:49:46.162 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.162 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.162 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:46.162 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965521+0000", 2024-09-16T20:49:46.162 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013110899999999999 2024-09-16T20:49:46.162 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:46.162 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:46.162 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:46.163 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.163 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.163 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:152 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 327680~32768 out=32768b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:49:46.163 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:49:46.163 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 81.930871217000004, 2024-09-16T20:49:46.163 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11869405800000001, 2024-09-16T20:49:46.163 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:46.163 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:46.163 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:46.163 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:46.163 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:49:46.163 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:49:46.163 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 152 2024-09-16T20:49:46.163 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.164 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:46.164 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.164 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:46.164 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:49:46.164 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:46.164 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.164 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.164 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:46.164 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:49:46.164 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:46.164 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.164 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.165 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:46.165 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163122+0000", 2024-09-16T20:49:46.165 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6400000000000003e-07 2024-09-16T20:49:46.165 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.165 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.165 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:46.165 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163124+0000", 2024-09-16T20:49:46.165 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.565e-06 2024-09-16T20:49:46.165 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.165 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.165 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:46.165 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163136+0000", 2024-09-16T20:49:46.165 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1717e-05 2024-09-16T20:49:46.165 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.166 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.166 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:46.166 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163142+0000", 2024-09-16T20:49:46.166 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1750000000000002e-06 2024-09-16T20:49:46.166 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.166 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.166 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:46.166 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163606+0000", 2024-09-16T20:49:46.166 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00046418799999999998 2024-09-16T20:49:46.166 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.166 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.166 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:46.166 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163656+0000", 2024-09-16T20:49:46.166 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9852999999999999e-05 2024-09-16T20:49:46.167 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.167 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.167 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:46.167 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281815+0000", 2024-09-16T20:49:46.167 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11815909600000001 2024-09-16T20:49:46.167 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:46.167 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:46.167 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:46.167 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.167 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.167 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:153 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:49:46.167 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:49:46.167 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 81.930838264000002, 2024-09-16T20:49:46.167 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11903462400000001, 2024-09-16T20:49:46.168 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:46.168 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:46.168 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:46.168 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:46.168 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:49:46.168 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:49:46.168 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 153 2024-09-16T20:49:46.168 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.168 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:46.168 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.168 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:46.168 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:49:46.168 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:46.168 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.169 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.169 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:46.172 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:49:46.172 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:46.172 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.172 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.172 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:46.172 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163155+0000", 2024-09-16T20:49:46.172 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.37e-07 2024-09-16T20:49:46.172 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.172 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.173 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:46.173 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163157+0000", 2024-09-16T20:49:46.173 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6010000000000002e-06 2024-09-16T20:49:46.173 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.173 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.173 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:46.173 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163165+0000", 2024-09-16T20:49:46.173 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2619999999999998e-06 2024-09-16T20:49:46.173 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.173 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.173 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:46.173 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163171+0000", 2024-09-16T20:49:46.173 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.711e-06 2024-09-16T20:49:46.174 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.174 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.174 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:46.174 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281868+0000", 2024-09-16T20:49:46.174 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118696622 2024-09-16T20:49:46.174 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.174 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.174 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:46.174 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281969+0000", 2024-09-16T20:49:46.174 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101371 2024-09-16T20:49:46.174 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.174 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.174 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:46.174 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282189+0000", 2024-09-16T20:49:46.175 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00021952000000000001 2024-09-16T20:49:46.175 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:46.175 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:46.175 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:46.175 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.175 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.175 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:154 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 360448~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:49:46.175 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:49:46.175 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 81.930806492000002, 2024-09-16T20:49:46.175 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11932035100000001, 2024-09-16T20:49:46.175 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:46.175 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:46.175 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:46.175 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:46.176 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:49:46.176 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:49:46.176 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 154 2024-09-16T20:49:46.176 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.176 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:46.176 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.176 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:46.176 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:49:46.176 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:46.176 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.176 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.176 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:46.176 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:49:46.176 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:46.176 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.177 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.177 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:46.177 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:49:46.177 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:49:46.177 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.177 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.177 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:46.177 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163189+0000", 2024-09-16T20:49:46.177 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6840000000000001e-06 2024-09-16T20:49:46.177 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.177 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.177 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:46.177 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163196+0000", 2024-09-16T20:49:46.177 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7920000000000004e-06 2024-09-16T20:49:46.177 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.177 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.178 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:46.178 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163201+0000", 2024-09-16T20:49:46.178 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2809999999999999e-06 2024-09-16T20:49:46.178 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.178 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.178 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:46.178 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282219+0000", 2024-09-16T20:49:46.178 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.119018343 2024-09-16T20:49:46.178 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.178 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.178 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:46.178 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282305+0000", 2024-09-16T20:49:46.178 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5865000000000005e-05 2024-09-16T20:49:46.178 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.179 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.179 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:46.179 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282506+0000", 2024-09-16T20:49:46.179 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000200927 2024-09-16T20:49:46.179 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:46.179 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:46.179 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:46.179 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.179 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.179 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop(client.4366.0:3088 3.1e e38/23 3:79e5ec01:::rbd_header.110e891a9b2b:head v 38'306, mlcod=36'304)", 2024-09-16T20:49:46.179 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:49:46.179 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 81.929787623999999, 2024-09-16T20:49:46.179 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118102759, 2024-09-16T20:49:46.179 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:46.179 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:46.180 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:46.180 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:46.180 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.180 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:46.180 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:49:46.180 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:46.180 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.180 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.180 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:46.180 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:49:46.180 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:46.180 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.180 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.180 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:46.181 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:49:46.181 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.85e-07 2024-09-16T20:49:46.181 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.181 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.181 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:46.181 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164212+0000", 2024-09-16T20:49:46.181 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3999999999999997e-06 2024-09-16T20:49:46.181 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.181 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.181 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:46.181 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281553+0000", 2024-09-16T20:49:46.181 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11734135499999999 2024-09-16T20:49:46.181 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.181 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.182 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:46.182 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281567+0000", 2024-09-16T20:49:46.182 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4188999999999999e-05 2024-09-16T20:49:46.182 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.182 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.182 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:46.182 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281653+0000", 2024-09-16T20:49:46.182 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5385999999999999e-05 2024-09-16T20:49:46.182 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.182 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.182 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:46.182 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281679+0000", 2024-09-16T20:49:46.182 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6689000000000001e-05 2024-09-16T20:49:46.183 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.183 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.183 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:46.183 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282277+0000", 2024-09-16T20:49:46.183 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00059798000000000002 2024-09-16T20:49:46.183 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.183 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.183 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:46.183 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282307+0000", 2024-09-16T20:49:46.183 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9975e-05 2024-09-16T20:49:46.183 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:46.183 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:46.183 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:46.183 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.184 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.184 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5255.0:44 3.16 3:6f5476df:::rbd_header.110e25db1f60:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:49:46.184 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:49:46.184 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 9.7734845690000007, 2024-09-16T20:49:46.184 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.121007509, 2024-09-16T20:49:46.184 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:46.184 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:46.184 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:46.184 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:46.184 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5255", 2024-09-16T20:49:46.184 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:49:46.184 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 44 2024-09-16T20:49:46.184 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.184 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:46.185 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.185 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:46.185 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:49:46.185 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:46.185 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.185 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.185 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:46.185 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:49:46.185 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:46.185 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.185 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.185 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:46.185 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:49:46.186 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0500000000000003e-07 2024-09-16T20:49:46.186 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.186 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.186 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:46.186 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320512+0000", 2024-09-16T20:49:46.186 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2540000000000001e-06 2024-09-16T20:49:46.186 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.186 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.186 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:46.186 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320518+0000", 2024-09-16T20:49:46.186 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1689999999999997e-06 2024-09-16T20:49:46.186 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.186 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.186 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:46.186 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320521+0000", 2024-09-16T20:49:46.187 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5200000000000002e-06 2024-09-16T20:49:46.187 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.187 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.187 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:46.187 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320548+0000", 2024-09-16T20:49:46.187 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6662999999999999e-05 2024-09-16T20:49:46.187 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.187 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.187 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:46.187 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320587+0000", 2024-09-16T20:49:46.187 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8866999999999999e-05 2024-09-16T20:49:46.187 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:46.187 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:46.187 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:46.188 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.441515+0000", 2024-09-16T20:49:46.188 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.120928331 2024-09-16T20:49:46.188 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:46.188 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:46.188 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:46.188 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:46.188 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:46.188 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:46.189 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:49:46.190 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:46.188+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:46.290 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:46.289+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:46.310 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:46.310 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:49:46.310 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:49:46.311 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:46.319 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:49:46.391 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:46.390+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:46.467 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:46.467 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:49:46.468 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:49:46.468 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:49:46.468 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:46.476 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:49:46.519 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:46.518+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:46.584 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:46.584 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:49:46.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:49:46.585 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:49:46.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.585 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6419 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3594240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:46.585 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871775+0000", 2024-09-16T20:49:46.585 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 86.710786237999997, 2024-09-16T20:49:46.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048754724999999999, 2024-09-16T20:49:46.585 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:46.585 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:46.585 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:46.585 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:46.585 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:46.585 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:46.585 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6419 2024-09-16T20:49:46.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.586 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:46.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:46.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871775+0000", 2024-09-16T20:49:46.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:46.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:46.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871775+0000", 2024-09-16T20:49:46.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:46.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:46.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871775+0000", 2024-09-16T20:49:46.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2500000000000001e-07 2024-09-16T20:49:46.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:46.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871777+0000", 2024-09-16T20:49:46.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9649999999999998e-06 2024-09-16T20:49:46.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:46.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871784+0000", 2024-09-16T20:49:46.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3419999999999996e-06 2024-09-16T20:49:46.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:46.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871789+0000", 2024-09-16T20:49:46.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1889999999999998e-06 2024-09-16T20:49:46.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:46.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920416+0000", 2024-09-16T20:49:46.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048627363999999999 2024-09-16T20:49:46.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:46.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920472+0000", 2024-09-16T20:49:46.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5429000000000001e-05 2024-09-16T20:49:46.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:46.589 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920530+0000", 2024-09-16T20:49:46.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8111e-05 2024-09-16T20:49:46.589 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:46.589 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:46.589 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:46.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.589 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6420 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3602432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:46.589 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871796+0000", 2024-09-16T20:49:46.590 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 86.710765324999997, 2024-09-16T20:49:46.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048857298, 2024-09-16T20:49:46.590 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:46.590 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:46.590 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:46.590 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:46.590 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:46.590 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:46.590 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6420 2024-09-16T20:49:46.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.590 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:46.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.591 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:46.591 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871796+0000", 2024-09-16T20:49:46.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:46.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.591 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:46.591 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871796+0000", 2024-09-16T20:49:46.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:46.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.591 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:46.591 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871854+0000", 2024-09-16T20:49:46.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8187999999999999e-05 2024-09-16T20:49:46.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.592 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:46.592 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871856+0000", 2024-09-16T20:49:46.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3549999999999999e-06 2024-09-16T20:49:46.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.592 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:46.592 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871866+0000", 2024-09-16T20:49:46.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6500000000000008e-06 2024-09-16T20:49:46.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.592 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:46.592 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871872+0000", 2024-09-16T20:49:46.593 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3799999999999999e-06 2024-09-16T20:49:46.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.593 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:46.593 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920558+0000", 2024-09-16T20:49:46.593 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048685797000000003 2024-09-16T20:49:46.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.593 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:46.593 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920603+0000", 2024-09-16T20:49:46.593 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4629999999999998e-05 2024-09-16T20:49:46.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.593 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:46.594 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920653+0000", 2024-09-16T20:49:46.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0297999999999999e-05 2024-09-16T20:49:46.594 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:46.594 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:46.594 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:46.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.594 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:6421 3.1e 3:7d17938d:::rbd_data.110e3f481dff.0000000000000000:7 [read 3610624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:46.594 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.871882+0000", 2024-09-16T20:49:46.594 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 86.710679256999995, 2024-09-16T20:49:46.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048913381999999998, 2024-09-16T20:49:46.594 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:46.595 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:46.595 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:46.595 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:46.595 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:46.595 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:46.595 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6421 2024-09-16T20:49:46.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.595 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:46.596 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.596 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:46.596 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871882+0000", 2024-09-16T20:49:46.596 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:46.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.596 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.596 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:46.596 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.871882+0000", 2024-09-16T20:49:46.596 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:46.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.596 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.596 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:46.596 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.873257+0000", 2024-09-16T20:49:46.597 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001374789 2024-09-16T20:49:46.597 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.597 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.597 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:46.597 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.873261+0000", 2024-09-16T20:49:46.597 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5589999999999999e-06 2024-09-16T20:49:46.597 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.597 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.597 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:46.597 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.873275+0000", 2024-09-16T20:49:46.597 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3675999999999999e-05 2024-09-16T20:49:46.597 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.597 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.598 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:46.598 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.873281+0000", 2024-09-16T20:49:46.598 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6930000000000004e-06 2024-09-16T20:49:46.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.598 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:46.598 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920684+0000", 2024-09-16T20:49:46.598 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047402913999999997 2024-09-16T20:49:46.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.598 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:46.598 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920737+0000", 2024-09-16T20:49:46.598 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3600000000000002e-05 2024-09-16T20:49:46.599 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.599 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:46.599 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.920795+0000", 2024-09-16T20:49:46.599 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8150999999999999e-05 2024-09-16T20:49:46.599 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:46.599 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:46.599 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:46.599 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.599 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:7491 3.1e 3:7c48f6ae:::rbd_object_map.110e891a9b2b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:46.599 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.925865+0000", 2024-09-16T20:49:46.600 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 86.656696112999995, 2024-09-16T20:49:46.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048897293000000001, 2024-09-16T20:49:46.600 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:46.600 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:46.600 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:46.600 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:46.600 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:46.600 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:46.600 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 7491 2024-09-16T20:49:46.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.601 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:46.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.601 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:46.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925865+0000", 2024-09-16T20:49:46.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:46.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.601 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:46.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925865+0000", 2024-09-16T20:49:46.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:46.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.601 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:46.602 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925866+0000", 2024-09-16T20:49:46.602 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.39e-07 2024-09-16T20:49:46.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.602 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:46.602 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925868+0000", 2024-09-16T20:49:46.602 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.508e-06 2024-09-16T20:49:46.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.602 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:46.602 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925874+0000", 2024-09-16T20:49:46.602 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7919999999999998e-06 2024-09-16T20:49:46.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.603 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:46.603 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.925879+0000", 2024-09-16T20:49:46.603 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8579999999999999e-06 2024-09-16T20:49:46.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.603 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:46.603 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.970875+0000", 2024-09-16T20:49:46.603 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044996412 2024-09-16T20:49:46.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.603 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:46.603 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.970970+0000", 2024-09-16T20:49:46.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4732e-05 2024-09-16T20:49:46.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.604 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-16T20:49:46.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.971275+0000", 2024-09-16T20:49:46.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00030464199999999999 2024-09-16T20:49:46.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.604 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:46.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.974598+0000", 2024-09-16T20:49:46.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0033238159999999998 2024-09-16T20:49:46.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:49:46.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.974666+0000", 2024-09-16T20:49:46.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7617e-05 2024-09-16T20:49:46.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:49:46.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.974718+0000", 2024-09-16T20:49:46.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2259000000000002e-05 2024-09-16T20:49:46.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:46.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.974762+0000", 2024-09-16T20:49:46.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4218000000000003e-05 2024-09-16T20:49:46.606 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:46.606 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:46.606 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:46.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.606 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4758 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1521664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:46.606 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689141+0000", 2024-09-16T20:49:46.606 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 83.893420714000001, 2024-09-16T20:49:46.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048640741000000001, 2024-09-16T20:49:46.606 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:46.606 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:46.606 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:46.606 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:46.607 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:49:46.607 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:46.607 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4758 2024-09-16T20:49:46.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.607 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:46.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.607 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:46.607 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689141+0000", 2024-09-16T20:49:46.607 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:46.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.607 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:46.607 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689141+0000", 2024-09-16T20:49:46.608 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:46.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.608 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:46.608 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689141+0000", 2024-09-16T20:49:46.608 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8799999999999998e-07 2024-09-16T20:49:46.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.608 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:46.608 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689143+0000", 2024-09-16T20:49:46.608 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.945e-06 2024-09-16T20:49:46.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.608 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:46.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689149+0000", 2024-09-16T20:49:46.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9800000000000003e-06 2024-09-16T20:49:46.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:46.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689155+0000", 2024-09-16T20:49:46.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7509999999999997e-06 2024-09-16T20:49:46.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:46.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737681+0000", 2024-09-16T20:49:46.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048526825000000003 2024-09-16T20:49:46.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.610 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:46.610 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737729+0000", 2024-09-16T20:49:46.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7219999999999999e-05 2024-09-16T20:49:46.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.610 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:46.610 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737781+0000", 2024-09-16T20:49:46.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2632e-05 2024-09-16T20:49:46.610 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:46.610 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:46.610 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:46.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.611 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4760 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1529856~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:46.611 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689161+0000", 2024-09-16T20:49:46.611 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 83.893400366999998, 2024-09-16T20:49:46.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048743981999999998, 2024-09-16T20:49:46.611 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:46.611 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:46.611 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:46.611 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:46.611 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:49:46.611 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:46.611 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4760 2024-09-16T20:49:46.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.612 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:46.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.612 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:46.612 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689161+0000", 2024-09-16T20:49:46.612 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:46.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.612 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:46.612 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689161+0000", 2024-09-16T20:49:46.612 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:46.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.612 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:46.612 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689161+0000", 2024-09-16T20:49:46.612 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5599999999999995e-07 2024-09-16T20:49:46.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.613 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:46.613 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689163+0000", 2024-09-16T20:49:46.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.818e-06 2024-09-16T20:49:46.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.613 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:46.613 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689169+0000", 2024-09-16T20:49:46.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0809999999999999e-06 2024-09-16T20:49:46.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.613 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:46.613 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689174+0000", 2024-09-16T20:49:46.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.938e-06 2024-09-16T20:49:46.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.614 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:46.614 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737807+0000", 2024-09-16T20:49:46.614 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048633124 2024-09-16T20:49:46.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.614 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:46.614 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737851+0000", 2024-09-16T20:49:46.614 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3764000000000003e-05 2024-09-16T20:49:46.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.614 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:46.614 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737905+0000", 2024-09-16T20:49:46.614 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3701e-05 2024-09-16T20:49:46.615 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:46.615 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:46.615 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:46.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.615 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.615 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4762 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1538048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:46.615 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689182+0000", 2024-09-16T20:49:46.615 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 83.893379144999997, 2024-09-16T20:49:46.615 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048850275999999998, 2024-09-16T20:49:46.615 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:46.615 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:46.615 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:46.615 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:46.615 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:49:46.616 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:46.616 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4762 2024-09-16T20:49:46.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.616 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:46.616 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.616 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:46.616 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689182+0000", 2024-09-16T20:49:46.616 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:46.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.616 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.616 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:46.616 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689182+0000", 2024-09-16T20:49:46.616 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:46.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.617 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:46.617 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689182+0000", 2024-09-16T20:49:46.617 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-16T20:49:46.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.617 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:46.617 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689184+0000", 2024-09-16T20:49:46.617 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8190000000000001e-06 2024-09-16T20:49:46.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.617 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:46.617 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689191+0000", 2024-09-16T20:49:46.617 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2530000000000001e-06 2024-09-16T20:49:46.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:46.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689195+0000", 2024-09-16T20:49:46.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9559999999999996e-06 2024-09-16T20:49:46.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:46.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737933+0000", 2024-09-16T20:49:46.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048737919999999997 2024-09-16T20:49:46.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:46.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.737982+0000", 2024-09-16T20:49:46.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8089e-05 2024-09-16T20:49:46.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.619 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:46.619 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738032+0000", 2024-09-16T20:49:46.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0868999999999997e-05 2024-09-16T20:49:46.619 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:46.619 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:46.619 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:46.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.619 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4764 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1546240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:46.619 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689201+0000", 2024-09-16T20:49:46.620 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 83.893359787999998, 2024-09-16T20:49:46.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048957550000000002, 2024-09-16T20:49:46.620 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:46.620 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:46.620 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:46.620 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:46.620 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:49:46.620 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:46.620 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4764 2024-09-16T20:49:46.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.620 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:46.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.620 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:46.620 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689201+0000", 2024-09-16T20:49:46.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:46.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.621 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:46.621 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689201+0000", 2024-09-16T20:49:46.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:46.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.621 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:46.621 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689202+0000", 2024-09-16T20:49:46.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4200000000000002e-07 2024-09-16T20:49:46.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.622 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:46.622 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689204+0000", 2024-09-16T20:49:46.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9750000000000001e-06 2024-09-16T20:49:46.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.622 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:46.622 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689210+0000", 2024-09-16T20:49:46.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1e-06 2024-09-16T20:49:46.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.622 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:46.622 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689215+0000", 2024-09-16T20:49:46.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0180000000000001e-06 2024-09-16T20:49:46.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:46.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738058+0000", 2024-09-16T20:49:46.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048843573000000001 2024-09-16T20:49:46.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:46.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738102+0000", 2024-09-16T20:49:46.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3256000000000001e-05 2024-09-16T20:49:46.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:46.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738159+0000", 2024-09-16T20:49:46.624 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7286e-05 2024-09-16T20:49:46.624 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:46.624 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:46.624 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:46.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.624 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.624 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4766 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1554432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:46.624 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689221+0000", 2024-09-16T20:49:46.624 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 83.893339822000002, 2024-09-16T20:49:46.624 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049064996, 2024-09-16T20:49:46.624 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:46.624 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:46.624 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:46.625 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:46.625 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:49:46.625 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:46.625 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4766 2024-09-16T20:49:46.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.625 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:46.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.625 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:46.625 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689221+0000", 2024-09-16T20:49:46.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:46.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.625 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:46.625 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689221+0000", 2024-09-16T20:49:46.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:46.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.626 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:46.626 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689222+0000", 2024-09-16T20:49:46.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.34e-07 2024-09-16T20:49:46.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.626 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:46.626 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689224+0000", 2024-09-16T20:49:46.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.846e-06 2024-09-16T20:49:46.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.626 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:46.627 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689247+0000", 2024-09-16T20:49:46.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3448e-05 2024-09-16T20:49:46.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.627 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:46.627 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689254+0000", 2024-09-16T20:49:46.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6969999999999996e-06 2024-09-16T20:49:46.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.627 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:46.627 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738188+0000", 2024-09-16T20:49:46.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048934520000000002 2024-09-16T20:49:46.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:46.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738234+0000", 2024-09-16T20:49:46.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5695999999999998e-05 2024-09-16T20:49:46.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:46.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738286+0000", 2024-09-16T20:49:46.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2355000000000003e-05 2024-09-16T20:49:46.628 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:46.628 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:46.628 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:46.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.629 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.629 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4768 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1562624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:46.629 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689262+0000", 2024-09-16T20:49:46.629 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 83.893298868000002, 2024-09-16T20:49:46.629 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049157157999999999, 2024-09-16T20:49:46.629 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:46.629 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:46.629 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:46.629 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:46.629 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:49:46.629 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:46.629 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4768 2024-09-16T20:49:46.629 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.629 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:46.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:46.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689262+0000", 2024-09-16T20:49:46.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:46.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:46.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689262+0000", 2024-09-16T20:49:46.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:46.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:46.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689263+0000", 2024-09-16T20:49:46.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6199999999999998e-07 2024-09-16T20:49:46.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.631 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:46.631 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689270+0000", 2024-09-16T20:49:46.631 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.798e-06 2024-09-16T20:49:46.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.631 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.631 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:46.631 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689277+0000", 2024-09-16T20:49:46.631 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0890000000000002e-06 2024-09-16T20:49:46.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.631 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.631 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:46.631 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689282+0000", 2024-09-16T20:49:46.631 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0470000000000001e-06 2024-09-16T20:49:46.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.631 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.632 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:46.632 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738319+0000", 2024-09-16T20:49:46.632 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049037139 2024-09-16T20:49:46.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.632 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.632 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:46.632 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738365+0000", 2024-09-16T20:49:46.632 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6356999999999997e-05 2024-09-16T20:49:46.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.632 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.632 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:46.632 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738420+0000", 2024-09-16T20:49:46.632 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4265999999999998e-05 2024-09-16T20:49:46.633 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:46.633 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:46.633 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:46.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.633 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.633 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4600.0:4770 3.b 3:d36f4eda:::rbd_data.110e3f481dff.0000000000000008:7 [read 1570816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:46.633 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:22.689289+0000", 2024-09-16T20:49:46.633 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 83.893272358000004, 2024-09-16T20:49:46.633 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049253705000000002, 2024-09-16T20:49:46.633 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:46.633 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:46.633 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:46.633 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:46.633 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4600", 2024-09-16T20:49:46.634 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:46.634 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4770 2024-09-16T20:49:46.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.634 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:46.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.634 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:46.634 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689289+0000", 2024-09-16T20:49:46.634 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:46.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.634 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:46.634 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689289+0000", 2024-09-16T20:49:46.634 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:46.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.635 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:46.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689289+0000", 2024-09-16T20:49:46.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0999999999999999e-07 2024-09-16T20:49:46.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.635 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:46.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689291+0000", 2024-09-16T20:49:46.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0030000000000001e-06 2024-09-16T20:49:46.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.635 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:46.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689297+0000", 2024-09-16T20:49:46.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9479999999999996e-06 2024-09-16T20:49:46.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.635 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:46.636 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.689302+0000", 2024-09-16T20:49:46.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7389999999999999e-06 2024-09-16T20:49:46.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.636 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:46.636 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738442+0000", 2024-09-16T20:49:46.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049140001000000003 2024-09-16T20:49:46.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.636 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:46.636 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738490+0000", 2024-09-16T20:49:46.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7787000000000002e-05 2024-09-16T20:49:46.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.636 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:46.637 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:22.738543+0000", 2024-09-16T20:49:46.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2816999999999999e-05 2024-09-16T20:49:46.637 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:46.637 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:46.637 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:46.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.637 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:3088 3.1e 3:79e5ec01:::rbd_header.110e891a9b2b:head [watch watch cookie 140213437171488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:49:46.637 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:49:46.637 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 82.418716341000007, 2024-09-16T20:49:46.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.11895186200000001, 2024-09-16T20:49:46.637 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:46.637 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:46.637 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:46.638 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:46.638 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:46.638 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:46.638 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3088 2024-09-16T20:49:46.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.638 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:46.638 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.638 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:46.638 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:49:46.638 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:46.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.638 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.638 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:46.638 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:49:46.638 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:46.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.639 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:46.639 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163846+0000", 2024-09-16T20:49:46.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6799999999999999e-07 2024-09-16T20:49:46.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.639 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:46.639 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163850+0000", 2024-09-16T20:49:46.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.929e-06 2024-09-16T20:49:46.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.639 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:46.639 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163862+0000", 2024-09-16T20:49:46.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1972e-05 2024-09-16T20:49:46.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.640 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:46.640 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163867+0000", 2024-09-16T20:49:46.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.451e-06 2024-09-16T20:49:46.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.640 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:46.640 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163912+0000", 2024-09-16T20:49:46.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4456000000000003e-05 2024-09-16T20:49:46.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.640 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:46.640 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163962+0000", 2024-09-16T20:49:46.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0578000000000003e-05 2024-09-16T20:49:46.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-16T20:49:46.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164058+0000", 2024-09-16T20:49:46.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5934999999999995e-05 2024-09-16T20:49:46.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:49:46.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164570+0000", 2024-09-16T20:49:46.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00051241100000000005 2024-09-16T20:49:46.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:46.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282629+0000", 2024-09-16T20:49:46.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.118058986 2024-09-16T20:49:46.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.642 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:49:46.642 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282725+0000", 2024-09-16T20:49:46.642 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5063999999999996e-05 2024-09-16T20:49:46.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.642 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:46.642 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282797+0000", 2024-09-16T20:49:46.642 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2212000000000006e-05 2024-09-16T20:49:46.642 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:46.642 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:46.642 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:46.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.643 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:8039 3.1e 3.ac44ff9e (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:49:46.643 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:49:46.643 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 56.874888992000002, 2024-09-16T20:49:46.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0053310449999999, 2024-09-16T20:49:46.643 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:46.643 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:46.643 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:49:46.643 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:46.643 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:46.643 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:46.643 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8039 2024-09-16T20:49:46.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.643 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:46.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.643 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:46.644 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:49:46.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:46.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.644 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:46.644 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:49:46.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:46.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.644 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:46.644 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707674+0000", 2024-09-16T20:49:46.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6530000000000001e-06 2024-09-16T20:49:46.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:46.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707681+0000", 2024-09-16T20:49:46.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9639999999999997e-06 2024-09-16T20:49:46.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:46.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707700+0000", 2024-09-16T20:49:46.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8674000000000001e-05 2024-09-16T20:49:46.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:46.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707708+0000", 2024-09-16T20:49:46.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6810000000000004e-06 2024-09-16T20:49:46.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.646 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:46.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711293+0000", 2024-09-16T20:49:46.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035848189999999999 2024-09-16T20:49:46.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.646 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:49:46.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711301+0000", 2024-09-16T20:49:46.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4730000000000003e-06 2024-09-16T20:49:46.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.646 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:46.647 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.712990+0000", 2024-09-16T20:49:46.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.001689348 2024-09-16T20:49:46.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.647 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:46.647 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713003+0000", 2024-09-16T20:49:46.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3433e-05 2024-09-16T20:49:46.647 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:46.647 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:46.647 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:46.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.647 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:8040 3.1e 3.ac44ff9e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:49:46.647 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:49:46.647 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 56.874828596999997, 2024-09-16T20:49:46.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0053410780000001, 2024-09-16T20:49:46.648 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:46.648 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:46.648 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:49:46.648 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:46.648 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:46.648 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:46.648 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8040 2024-09-16T20:49:46.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.648 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:46.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:46.648 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:49:46.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:46.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:46.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:49:46.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:46.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:46.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:49:46.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8100000000000003e-07 2024-09-16T20:49:46.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:46.650 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707737+0000", 2024-09-16T20:49:46.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5020000000000001e-06 2024-09-16T20:49:46.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.650 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:46.650 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707745+0000", 2024-09-16T20:49:46.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2169999999999994e-06 2024-09-16T20:49:46.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.650 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:46.650 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707751+0000", 2024-09-16T20:49:46.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.263e-06 2024-09-16T20:49:46.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.651 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:46.651 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711336+0000", 2024-09-16T20:49:46.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035848360000000001 2024-09-16T20:49:46.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.651 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:49:46.651 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711340+0000", 2024-09-16T20:49:46.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1309999999999999e-06 2024-09-16T20:49:46.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.651 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:46.651 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713060+0000", 2024-09-16T20:49:46.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0017195059999999 2024-09-16T20:49:46.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.652 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:46.652 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713074+0000", 2024-09-16T20:49:46.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4042e-05 2024-09-16T20:49:46.652 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:46.652 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:46.652 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:46.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.653 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4927.0:4487 3.1e 3.ac44ff9e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:49:46.653 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:49:46.653 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 56.874308395, 2024-09-16T20:49:46.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0048840880000001, 2024-09-16T20:49:46.653 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:46.653 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:46.653 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:49:46.653 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:46.653 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4927", 2024-09-16T20:49:46.653 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:46.653 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4487 2024-09-16T20:49:46.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.653 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:46.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:46.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:49:46.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:46.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:46.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:49:46.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:46.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:46.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708254+0000", 2024-09-16T20:49:46.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1800000000000005e-07 2024-09-16T20:49:46.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.655 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:46.655 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708258+0000", 2024-09-16T20:49:46.655 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8689999999999997e-06 2024-09-16T20:49:46.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.655 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.655 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:46.655 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708267+0000", 2024-09-16T20:49:46.655 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9590000000000007e-06 2024-09-16T20:49:46.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.655 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.655 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:46.656 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708274+0000", 2024-09-16T20:49:46.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9970000000000001e-06 2024-09-16T20:49:46.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.656 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:46.656 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711366+0000", 2024-09-16T20:49:46.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030919530000000001 2024-09-16T20:49:46.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.656 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:49:46.656 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711372+0000", 2024-09-16T20:49:46.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9410000000000004e-06 2024-09-16T20:49:46.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.657 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:46.657 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713126+0000", 2024-09-16T20:49:46.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0017537249999999 2024-09-16T20:49:46.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.657 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:46.657 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713137+0000", 2024-09-16T20:49:46.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0825999999999999e-05 2024-09-16T20:49:46.657 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:46.657 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:46.657 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:46.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.657 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5089.0:11703 3.1b 3:dcd4684e:::rbd_data.110e747f0779.000000000000001b:head [set-alloc-hint object_size 262144 write_size 262144,write 139264~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:49:46.657 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:49:46.657 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 42.444572620999999, 2024-09-16T20:49:46.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.084231385000000006, 2024-09-16T20:49:46.658 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:46.658 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:46.658 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:46.658 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:46.658 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5089", 2024-09-16T20:49:46.658 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:49:46.658 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 11703 2024-09-16T20:49:46.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.658 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:46.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.658 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:46.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:49:46.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:46.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.659 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.659 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:46.659 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:49:46.659 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:46.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.659 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.659 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:46.659 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:49:46.659 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2600000000000001e-07 2024-09-16T20:49:46.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.659 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.659 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:46.659 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137998+0000", 2024-09-16T20:49:46.659 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9549999999999998e-06 2024-09-16T20:49:46.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.660 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:46.660 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138008+0000", 2024-09-16T20:49:46.660 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3789999999999995e-06 2024-09-16T20:49:46.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.660 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:46.660 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138013+0000", 2024-09-16T20:49:46.660 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2209999999999996e-06 2024-09-16T20:49:46.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.660 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:46.660 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138054+0000", 2024-09-16T20:49:46.660 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1022999999999997e-05 2024-09-16T20:49:46.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.661 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:46.661 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138124+0000", 2024-09-16T20:49:46.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0234999999999994e-05 2024-09-16T20:49:46.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.661 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:46.661 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138197+0000", 2024-09-16T20:49:46.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2451000000000001e-05 2024-09-16T20:49:46.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.661 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:46.661 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222010+0000", 2024-09-16T20:49:46.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.083813433000000007 2024-09-16T20:49:46.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.662 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.662 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:49:46.662 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222163+0000", 2024-09-16T20:49:46.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015274599999999999 2024-09-16T20:49:46.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.662 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.662 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:49:46.662 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222197+0000", 2024-09-16T20:49:46.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3683000000000003e-05 2024-09-16T20:49:46.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.662 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.662 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:46.662 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222220+0000", 2024-09-16T20:49:46.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3433000000000001e-05 2024-09-16T20:49:46.662 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:46.663 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:46.663 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:46.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.663 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.5089.0:11702 3.15 e73/64 3:acf33c8b:::rbd_data.110e747f0779.000000000000001a:head v 73'132, mlcod=73'130)", 2024-09-16T20:49:46.663 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:49:46.663 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 42.444305292000003, 2024-09-16T20:49:46.663 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.084079763000000002, 2024-09-16T20:49:46.663 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:46.663 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:46.663 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:46.663 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:46.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.663 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:46.663 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:49:46.664 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:46.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.664 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.664 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:46.664 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:49:46.664 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:46.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.664 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.664 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:46.664 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:49:46.664 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:49:46.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.664 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.664 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:46.665 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138263+0000", 2024-09-16T20:49:46.665 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1e-06 2024-09-16T20:49:46.665 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.665 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.665 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:46.665 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221589+0000", 2024-09-16T20:49:46.665 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.083326002999999996 2024-09-16T20:49:46.665 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.665 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.665 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:46.666 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221614+0000", 2024-09-16T20:49:46.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5593999999999999e-05 2024-09-16T20:49:46.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.666 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:46.666 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221714+0000", 2024-09-16T20:49:46.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9889000000000005e-05 2024-09-16T20:49:46.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.666 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:46.666 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221727+0000", 2024-09-16T20:49:46.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3125999999999999e-05 2024-09-16T20:49:46.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.666 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:49:46.667 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222294+0000", 2024-09-16T20:49:46.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00056712199999999998 2024-09-16T20:49:46.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.667 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:46.667 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222336+0000", 2024-09-16T20:49:46.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1470000000000001e-05 2024-09-16T20:49:46.667 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:46.667 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:46.667 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:46.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.667 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5138.0:9358 3.c 3:33996a43:::rbd_data.110e6372565d.0000000000000007:head [read 1343488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T20:49:46.667 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:49:46.668 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.730029882, 2024-09-16T20:49:46.668 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0167493569999999, 2024-09-16T20:49:46.668 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:46.668 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:46.668 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:46.668 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:46.668 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5138", 2024-09-16T20:49:46.668 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:49:46.668 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9358 2024-09-16T20:49:46.668 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.668 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:46.668 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.668 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:46.668 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:49:46.668 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:46.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.669 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.669 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:46.669 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:49:46.669 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:46.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.669 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.669 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:46.669 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852533+0000", 2024-09-16T20:49:46.669 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.646e-06 2024-09-16T20:49:46.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.669 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.669 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:46.669 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852539+0000", 2024-09-16T20:49:46.669 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3389999999999998e-06 2024-09-16T20:49:46.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.670 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:46.670 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852552+0000", 2024-09-16T20:49:46.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.271e-05 2024-09-16T20:49:46.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.670 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:46.670 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852563+0000", 2024-09-16T20:49:46.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1139e-05 2024-09-16T20:49:46.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.670 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:46.670 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852883+0000", 2024-09-16T20:49:46.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00032010600000000001 2024-09-16T20:49:46.671 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.671 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.671 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:49:46.671 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852890+0000", 2024-09-16T20:49:46.671 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7669999999999998e-06 2024-09-16T20:49:46.671 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.671 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.671 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:46.671 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869059+0000", 2024-09-16T20:49:46.671 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.016169353 2024-09-16T20:49:46.671 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.671 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.671 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:46.671 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869178+0000", 2024-09-16T20:49:46.671 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118193 2024-09-16T20:49:46.672 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.672 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.672 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:46.672 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869281+0000", 2024-09-16T20:49:46.672 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010310399999999999 2024-09-16T20:49:46.672 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:46.672 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:46.672 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:46.672 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.672 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.672 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5138.0:9382 3.c 3:33996a43:::rbd_data.110e6372565d.0000000000000007:head [read 0~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T20:49:46.672 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:49:46.672 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.729979005000001, 2024-09-16T20:49:46.672 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.017136034, 2024-09-16T20:49:46.672 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:46.673 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:46.673 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:46.673 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:46.673 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5138", 2024-09-16T20:49:46.673 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:49:46.673 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9382 2024-09-16T20:49:46.673 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.673 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:46.673 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.673 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:46.673 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:49:46.673 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:46.673 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.673 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.674 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:46.674 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:49:46.674 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:46.674 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.674 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.674 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:46.674 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852583+0000", 2024-09-16T20:49:46.674 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.51e-07 2024-09-16T20:49:46.674 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.674 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.674 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:46.674 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852586+0000", 2024-09-16T20:49:46.674 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0280000000000001e-06 2024-09-16T20:49:46.674 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.674 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.675 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:46.675 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852595+0000", 2024-09-16T20:49:46.675 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2799999999999992e-06 2024-09-16T20:49:46.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.675 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.675 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:46.675 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852601+0000", 2024-09-16T20:49:46.675 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3810000000000004e-06 2024-09-16T20:49:46.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.675 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.675 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:46.675 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852921+0000", 2024-09-16T20:49:46.675 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00031979699999999997 2024-09-16T20:49:46.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.675 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.675 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:49:46.676 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852930+0000", 2024-09-16T20:49:46.676 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6489999999999997e-06 2024-09-16T20:49:46.676 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.676 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.676 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:46.676 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869434+0000", 2024-09-16T20:49:46.676 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0165038829999999 2024-09-16T20:49:46.676 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.676 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.676 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:46.676 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869495+0000", 2024-09-16T20:49:46.676 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1509999999999999e-05 2024-09-16T20:49:46.676 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.676 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.676 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:46.677 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869718+0000", 2024-09-16T20:49:46.677 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000222955 2024-09-16T20:49:46.677 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:46.677 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:46.677 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:46.677 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.677 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.677 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5425.0:2601 3.1e 3:7dc19323:::rbd_data.110e6372565d.0000000000000026:1d [read 1171456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-16T20:49:46.677 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:31.967612+0000", 2024-09-16T20:49:46.677 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 14.614948983, 2024-09-16T20:49:46.677 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048708179999999997, 2024-09-16T20:49:46.677 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:46.677 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:46.677 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:46.677 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:46.678 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5425", 2024-09-16T20:49:46.678 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1974102553", 2024-09-16T20:49:46.678 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 2601 2024-09-16T20:49:46.678 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.678 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:46.678 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.678 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:46.678 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:31.967612+0000", 2024-09-16T20:49:46.678 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:46.678 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.678 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.678 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:46.678 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:31.967612+0000", 2024-09-16T20:49:46.678 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:46.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.679 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.679 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:46.679 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:31.967842+0000", 2024-09-16T20:49:46.679 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00022931899999999999 2024-09-16T20:49:46.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.679 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.679 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:46.679 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:31.967845+0000", 2024-09-16T20:49:46.679 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2320000000000001e-06 2024-09-16T20:49:46.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.679 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.679 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:46.679 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:31.967854+0000", 2024-09-16T20:49:46.680 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1849999999999993e-06 2024-09-16T20:49:46.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.680 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.680 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:46.680 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:31.967864+0000", 2024-09-16T20:49:46.680 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6639999999999993e-06 2024-09-16T20:49:46.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.680 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.680 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:46.680 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:32.016212+0000", 2024-09-16T20:49:46.680 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048348820000000001 2024-09-16T20:49:46.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.680 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.680 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:46.681 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:32.016257+0000", 2024-09-16T20:49:46.681 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4674e-05 2024-09-16T20:49:46.681 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:46.681 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:46.681 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:46.681 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:32.016320+0000", 2024-09-16T20:49:46.681 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3286000000000004e-05 2024-09-16T20:49:46.681 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:46.681 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:46.681 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:46.681 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:46.681 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:46.681 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:46.683 INFO:teuthology.orchestra.run.smithi029.stderr:nodeep-scrub is set 2024-09-16T20:49:46.684 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:49:46.684 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:46.681+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:46.783 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:46.782+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:46.801 INFO:teuthology.orchestra.run.smithi139.stdout:4056 write 0x1c18a00 thru 0x1c1b201 (0x2802 bytes) 2024-09-16T20:49:46.803 INFO:teuthology.orchestra.run.smithi139.stdout:4057 read 0x16f9c00 thru 0x16fe623 (0x4a24 bytes) 2024-09-16T20:49:46.822 INFO:teuthology.orchestra.run.smithi139.stdout:4062 punch from 0x6e44200 to 0x6e4fc00, (0xba00 bytes) 2024-09-16T20:49:46.822 INFO:teuthology.orchestra.run.smithi139.stdout:4063 flatten 2024-09-16T20:49:46.845 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:46.845 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:49:46.845 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:49:46.845 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:46.858 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:49:46.899 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:46.898+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:46.984 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:46.983+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:47.017 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:47.017 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:49:47.018 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:49:47.018 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:49:47.018 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:47.026 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:49:47.029 INFO:teuthology.orchestra.run.smithi139.stdout:4064 read 0x4f89e00 thru 0x4f9253f (0x8740 bytes) 2024-09-16T20:49:47.030 INFO:teuthology.orchestra.run.smithi139.stdout:4066 punch from 0x238fa00 to 0x2398e00, (0x9400 bytes) 2024-09-16T20:49:47.030 INFO:teuthology.orchestra.run.smithi139.stdout:4069 punch from 0x325e000 to 0x326ca00, (0xea00 bytes) 2024-09-16T20:49:47.030 INFO:teuthology.orchestra.run.smithi139.stdout:4070 write 0x364ec00 thru 0x365543a (0x683b bytes) 2024-09-16T20:49:47.030 INFO:teuthology.orchestra.run.smithi139.stdout:4071 write 0x87200 thru 0x8ce16 (0x5c17 bytes) 2024-09-16T20:49:47.038 INFO:teuthology.orchestra.run.smithi139.stdout:4073 trunc from 0xb5c4e00 to 0x5275c00 2024-09-16T20:49:47.055 INFO:teuthology.orchestra.run.smithi139.stdout:4076 read 0x351ea00 thru 0x352d60d (0xec0e bytes) 2024-09-16T20:49:47.057 INFO:teuthology.orchestra.run.smithi139.stdout:4083 read 0x1534600 thru 0x1540b09 (0xc50a bytes) 2024-09-16T20:49:47.057 INFO:teuthology.orchestra.run.smithi139.stdout:4085 punch from 0x4b18800 to 0x4b20e00, (0x8600 bytes) 2024-09-16T20:49:47.075 INFO:teuthology.orchestra.run.smithi139.stdout:4087 write 0xaae2200 thru 0xaae7200 (0x5001 bytes) 2024-09-16T20:49:47.076 INFO:teuthology.orchestra.run.smithi139.stdout:4089 read 0x8926c00 thru 0x892879f (0x1ba0 bytes) 2024-09-16T20:49:47.076 INFO:teuthology.orchestra.run.smithi139.stdout:4091 write 0x98c00 thru 0x9c41c (0x381d bytes) 2024-09-16T20:49:47.077 INFO:teuthology.orchestra.run.smithi139.stdout:4092 read 0xa84a000 thru 0xa84f4dc (0x54dd bytes) 2024-09-16T20:49:47.077 INFO:teuthology.orchestra.run.smithi139.stdout:4093 write 0x5614c00 thru 0x561ad6e (0x616f bytes) 2024-09-16T20:49:47.077 INFO:teuthology.orchestra.run.smithi139.stdout:4094 read 0x6507000 thru 0x650fcdb (0x8cdc bytes) 2024-09-16T20:49:47.077 INFO:teuthology.orchestra.run.smithi139.stdout:4095 clone 33 order 18 su 1024 sc 5 2024-09-16T20:49:47.110 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:47.109+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:47.175 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:47.175 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:49:47.175 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:49:47.176 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:49:47.176 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.176 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:4904 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 7309312~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:47.176 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767724+0000", 2024-09-16T20:49:47.176 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 87.406202331000003, 2024-09-16T20:49:47.176 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050270214000000001, 2024-09-16T20:49:47.176 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:47.176 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:47.176 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:47.176 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:47.177 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:47.177 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:47.177 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4904 2024-09-16T20:49:47.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.177 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:47.177 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.177 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:47.177 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767724+0000", 2024-09-16T20:49:47.177 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:47.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.177 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.177 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:47.178 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767724+0000", 2024-09-16T20:49:47.178 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:47.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.178 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.178 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:47.178 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767724+0000", 2024-09-16T20:49:47.178 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1399999999999998e-07 2024-09-16T20:49:47.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.178 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.178 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:47.178 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767725+0000", 2024-09-16T20:49:47.179 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0380000000000001e-06 2024-09-16T20:49:47.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.179 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.179 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:47.179 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767729+0000", 2024-09-16T20:49:47.179 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2490000000000001e-06 2024-09-16T20:49:47.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.179 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.179 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:47.179 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767731+0000", 2024-09-16T20:49:47.179 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5940000000000001e-06 2024-09-16T20:49:47.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.180 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.180 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:47.180 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.817787+0000", 2024-09-16T20:49:47.180 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050055792000000002 2024-09-16T20:49:47.180 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.180 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.180 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:47.180 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.817883+0000", 2024-09-16T20:49:47.180 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5606999999999995e-05 2024-09-16T20:49:47.180 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.180 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.180 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:47.181 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.817994+0000", 2024-09-16T20:49:47.181 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011162000000000001 2024-09-16T20:49:47.181 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:47.181 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:47.181 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:47.181 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.181 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.181 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:4906 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 7317504~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:47.181 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767734+0000", 2024-09-16T20:49:47.181 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 87.406192067999996, 2024-09-16T20:49:47.181 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050406608999999998, 2024-09-16T20:49:47.182 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:47.182 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:47.182 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:47.182 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:47.182 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:47.182 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:47.182 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4906 2024-09-16T20:49:47.182 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.182 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:47.182 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.182 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:47.183 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767734+0000", 2024-09-16T20:49:47.183 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:47.183 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.183 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.183 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:47.183 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767734+0000", 2024-09-16T20:49:47.183 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:47.183 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.183 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.183 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:47.184 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767735+0000", 2024-09-16T20:49:47.184 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.05e-07 2024-09-16T20:49:47.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.184 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.184 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:47.184 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767736+0000", 2024-09-16T20:49:47.184 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0240000000000001e-06 2024-09-16T20:49:47.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.184 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.184 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:47.185 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767739+0000", 2024-09-16T20:49:47.185 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.067e-06 2024-09-16T20:49:47.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.185 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.185 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:47.185 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767742+0000", 2024-09-16T20:49:47.185 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.23e-06 2024-09-16T20:49:47.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.185 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.185 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:47.185 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818027+0000", 2024-09-16T20:49:47.185 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050285102999999998 2024-09-16T20:49:47.186 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.186 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.186 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:47.186 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818072+0000", 2024-09-16T20:49:47.186 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5129000000000001e-05 2024-09-16T20:49:47.186 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.186 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.186 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:47.186 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818141+0000", 2024-09-16T20:49:47.186 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8850999999999995e-05 2024-09-16T20:49:47.186 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:47.186 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:47.187 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:47.187 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.187 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.187 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:4908 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 7325696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:47.187 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767745+0000", 2024-09-16T20:49:47.187 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 87.406181673000006, 2024-09-16T20:49:47.187 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050654230000000001, 2024-09-16T20:49:47.187 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:47.187 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:47.187 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:47.187 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:47.188 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:47.188 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:47.188 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4908 2024-09-16T20:49:47.188 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.188 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:47.188 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.188 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:47.188 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767745+0000", 2024-09-16T20:49:47.188 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:47.188 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.188 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.189 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:47.189 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767745+0000", 2024-09-16T20:49:47.189 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:47.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.189 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.189 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:47.189 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767745+0000", 2024-09-16T20:49:47.189 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1e-07 2024-09-16T20:49:47.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.189 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.189 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:47.189 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767749+0000", 2024-09-16T20:49:47.190 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.532e-06 2024-09-16T20:49:47.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.190 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.190 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:47.190 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767753+0000", 2024-09-16T20:49:47.190 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4939999999999999e-06 2024-09-16T20:49:47.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.190 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.190 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:47.190 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767756+0000", 2024-09-16T20:49:47.190 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.61e-06 2024-09-16T20:49:47.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.190 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.191 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:47.191 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818184+0000", 2024-09-16T20:49:47.191 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050428615000000003 2024-09-16T20:49:47.191 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.191 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.191 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:47.191 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818276+0000", 2024-09-16T20:49:47.191 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2077999999999995e-05 2024-09-16T20:49:47.191 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.191 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.191 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:47.192 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818399+0000", 2024-09-16T20:49:47.192 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012269099999999999 2024-09-16T20:49:47.192 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:47.192 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:47.192 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:47.192 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.192 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.192 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:4910 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 7333888~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:47.192 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767759+0000", 2024-09-16T20:49:47.192 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 87.406167783000001, 2024-09-16T20:49:47.193 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050815076000000001, 2024-09-16T20:49:47.193 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:47.193 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:47.193 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:47.193 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:47.193 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:47.193 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:47.193 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4910 2024-09-16T20:49:47.193 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.193 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:47.193 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.193 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:47.194 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767759+0000", 2024-09-16T20:49:47.194 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:47.194 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.194 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.194 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:47.194 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767759+0000", 2024-09-16T20:49:47.194 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:47.194 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.194 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.194 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:47.194 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767759+0000", 2024-09-16T20:49:47.194 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2600000000000001e-07 2024-09-16T20:49:47.194 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.195 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.195 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:47.195 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767760+0000", 2024-09-16T20:49:47.195 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0640000000000001e-06 2024-09-16T20:49:47.195 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.195 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.195 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:47.195 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767763+0000", 2024-09-16T20:49:47.195 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.994e-06 2024-09-16T20:49:47.195 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.195 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.195 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:47.196 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767765+0000", 2024-09-16T20:49:47.196 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5220000000000002e-06 2024-09-16T20:49:47.196 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.196 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.196 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:47.196 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818438+0000", 2024-09-16T20:49:47.196 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050672602999999997 2024-09-16T20:49:47.196 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.196 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.197 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:47.197 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818481+0000", 2024-09-16T20:49:47.197 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2967999999999999e-05 2024-09-16T20:49:47.197 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.197 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.197 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:47.197 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818574+0000", 2024-09-16T20:49:47.197 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2699000000000001e-05 2024-09-16T20:49:47.197 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:47.197 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:47.197 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:47.197 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.198 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.198 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5012 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2101248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:47.198 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767768+0000", 2024-09-16T20:49:47.198 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 87.406158133999995, 2024-09-16T20:49:47.198 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051084682999999999, 2024-09-16T20:49:47.198 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:47.198 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:47.198 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:47.198 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:47.198 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:47.198 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:47.198 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5012 2024-09-16T20:49:47.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.199 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:47.199 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.199 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:47.199 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767768+0000", 2024-09-16T20:49:47.199 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:47.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.199 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.199 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:47.199 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767768+0000", 2024-09-16T20:49:47.199 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:47.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.200 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.200 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:47.200 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767769+0000", 2024-09-16T20:49:47.200 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1500000000000001e-07 2024-09-16T20:49:47.200 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.200 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.200 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:47.200 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767770+0000", 2024-09-16T20:49:47.200 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.015e-06 2024-09-16T20:49:47.200 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.200 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.200 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:47.200 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767773+0000", 2024-09-16T20:49:47.201 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1070000000000001e-06 2024-09-16T20:49:47.201 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.201 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.201 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:47.201 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767775+0000", 2024-09-16T20:49:47.201 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.508e-06 2024-09-16T20:49:47.201 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.201 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.202 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:47.202 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818630+0000", 2024-09-16T20:49:47.202 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050855282000000002 2024-09-16T20:49:47.202 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.202 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.202 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:47.202 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818732+0000", 2024-09-16T20:49:47.202 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101531 2024-09-16T20:49:47.202 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.202 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.202 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:47.202 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818853+0000", 2024-09-16T20:49:47.203 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000121025 2024-09-16T20:49:47.203 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:47.203 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:47.203 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:47.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.203 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.203 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5014 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2109440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:47.203 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767778+0000", 2024-09-16T20:49:47.203 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 87.406148197999997, 2024-09-16T20:49:47.203 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051260776000000001, 2024-09-16T20:49:47.203 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:47.203 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:47.204 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:47.204 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:47.204 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:47.204 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:47.204 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5014 2024-09-16T20:49:47.204 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.204 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:47.204 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.204 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:47.204 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767778+0000", 2024-09-16T20:49:47.204 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:47.204 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.205 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.205 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:47.205 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767778+0000", 2024-09-16T20:49:47.205 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:47.205 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.205 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.205 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:47.205 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767824+0000", 2024-09-16T20:49:47.205 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5890999999999998e-05 2024-09-16T20:49:47.205 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.205 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.205 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:47.206 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767827+0000", 2024-09-16T20:49:47.206 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.644e-06 2024-09-16T20:49:47.206 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.206 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.206 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:47.206 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767832+0000", 2024-09-16T20:49:47.206 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2689999999999999e-06 2024-09-16T20:49:47.206 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.206 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.206 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:47.206 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767835+0000", 2024-09-16T20:49:47.206 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2100000000000002e-06 2024-09-16T20:49:47.207 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.207 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.207 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:47.207 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818887+0000", 2024-09-16T20:49:47.207 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051051276999999999 2024-09-16T20:49:47.207 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.207 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.207 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:47.207 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818953+0000", 2024-09-16T20:49:47.207 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6105999999999994e-05 2024-09-16T20:49:47.207 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.207 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.207 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:47.208 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819039+0000", 2024-09-16T20:49:47.208 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6378999999999994e-05 2024-09-16T20:49:47.208 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:47.208 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:47.208 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:47.208 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.208 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.208 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5016 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2117632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:47.208 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767841+0000", 2024-09-16T20:49:47.208 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 87.406085519000001, 2024-09-16T20:49:47.208 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051433288000000001, 2024-09-16T20:49:47.208 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:47.209 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:47.209 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:47.209 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:47.209 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:47.209 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:47.209 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5016 2024-09-16T20:49:47.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.209 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:47.209 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.209 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:47.209 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767841+0000", 2024-09-16T20:49:47.210 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:47.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.210 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.210 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:47.210 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767841+0000", 2024-09-16T20:49:47.210 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:47.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.210 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.210 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:47.210 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768156+0000", 2024-09-16T20:49:47.210 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00031505699999999999 2024-09-16T20:49:47.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.211 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.211 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:47.211 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768159+0000", 2024-09-16T20:49:47.211 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1659999999999998e-06 2024-09-16T20:49:47.211 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.211 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.211 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:47.211 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768166+0000", 2024-09-16T20:49:47.211 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0550000000000002e-06 2024-09-16T20:49:47.211 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.211 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.211 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:47.212 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768171+0000", 2024-09-16T20:49:47.212 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3170000000000003e-06 2024-09-16T20:49:47.212 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.212 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.212 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:47.212 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819084+0000", 2024-09-16T20:49:47.212 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050913355 2024-09-16T20:49:47.212 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.212 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.212 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:47.213 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819170+0000", 2024-09-16T20:49:47.213 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6328000000000004e-05 2024-09-16T20:49:47.213 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.213 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.213 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:47.213 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819274+0000", 2024-09-16T20:49:47.213 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010401 2024-09-16T20:49:47.213 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:47.213 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:47.213 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:47.213 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.214 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.214 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5018 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2125824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:47.214 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768178+0000", 2024-09-16T20:49:47.214 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 87.405747985000005, 2024-09-16T20:49:47.214 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051227013000000002, 2024-09-16T20:49:47.214 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:47.214 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:47.214 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:47.214 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:47.214 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:47.214 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:47.215 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5018 2024-09-16T20:49:47.215 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.215 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:47.215 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.215 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:47.215 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768178+0000", 2024-09-16T20:49:47.215 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:47.215 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.215 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.216 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:47.216 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768178+0000", 2024-09-16T20:49:47.216 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:47.216 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.216 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.216 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:47.216 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768179+0000", 2024-09-16T20:49:47.216 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9200000000000002e-07 2024-09-16T20:49:47.216 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.217 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.217 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:47.217 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768180+0000", 2024-09-16T20:49:47.217 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3790000000000001e-06 2024-09-16T20:49:47.217 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.217 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.217 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:47.217 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768184+0000", 2024-09-16T20:49:47.217 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5470000000000001e-06 2024-09-16T20:49:47.217 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.218 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.218 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:47.218 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768187+0000", 2024-09-16T20:49:47.218 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8119999999999999e-06 2024-09-16T20:49:47.218 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.218 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.218 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:47.218 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819305+0000", 2024-09-16T20:49:47.218 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051118683999999998 2024-09-16T20:49:47.218 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.218 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.219 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:47.219 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819351+0000", 2024-09-16T20:49:47.219 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5757999999999998e-05 2024-09-16T20:49:47.219 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.219 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.219 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:47.219 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819405+0000", 2024-09-16T20:49:47.219 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4441000000000003e-05 2024-09-16T20:49:47.219 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:47.219 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:47.219 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:47.220 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.220 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.220 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5020 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2134016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:47.220 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768190+0000", 2024-09-16T20:49:47.220 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 87.405736306999998, 2024-09-16T20:49:47.220 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051449661000000001, 2024-09-16T20:49:47.220 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:47.220 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:47.220 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:47.220 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:47.221 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:47.221 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:47.221 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5020 2024-09-16T20:49:47.221 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.221 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:47.221 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.221 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:47.221 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768190+0000", 2024-09-16T20:49:47.221 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:47.221 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.222 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.222 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:47.222 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768190+0000", 2024-09-16T20:49:47.222 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:47.222 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.222 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.222 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:47.222 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768282+0000", 2024-09-16T20:49:47.222 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2263999999999996e-05 2024-09-16T20:49:47.222 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.222 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.223 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:47.223 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768286+0000", 2024-09-16T20:49:47.223 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.529e-06 2024-09-16T20:49:47.223 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.223 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.223 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:47.223 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768294+0000", 2024-09-16T20:49:47.223 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6990000000000003e-06 2024-09-16T20:49:47.223 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.223 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.223 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:47.223 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768306+0000", 2024-09-16T20:49:47.223 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2112e-05 2024-09-16T20:49:47.224 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.224 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.224 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:47.224 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819449+0000", 2024-09-16T20:49:47.224 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051143521999999997 2024-09-16T20:49:47.224 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.224 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.224 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:47.224 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819539+0000", 2024-09-16T20:49:47.224 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0022e-05 2024-09-16T20:49:47.224 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.224 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.225 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:47.225 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819640+0000", 2024-09-16T20:49:47.225 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000100513 2024-09-16T20:49:47.225 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:47.225 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:47.225 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:47.225 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.225 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.225 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5022 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2142208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:47.225 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768326+0000", 2024-09-16T20:49:47.225 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 87.405600918000005, 2024-09-16T20:49:47.225 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051436952000000001, 2024-09-16T20:49:47.226 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:47.226 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:47.226 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:47.226 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:47.226 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:47.226 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:47.226 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5022 2024-09-16T20:49:47.226 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.226 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:47.226 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.226 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:47.226 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768326+0000", 2024-09-16T20:49:47.227 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:47.227 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.227 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.227 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:47.227 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768326+0000", 2024-09-16T20:49:47.227 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:47.227 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.227 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.227 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:47.227 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768477+0000", 2024-09-16T20:49:47.227 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015109300000000001 2024-09-16T20:49:47.227 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.228 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.228 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:47.228 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768480+0000", 2024-09-16T20:49:47.228 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8580000000000002e-06 2024-09-16T20:49:47.228 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.228 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.228 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:47.228 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768489+0000", 2024-09-16T20:49:47.228 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4600000000000003e-06 2024-09-16T20:49:47.228 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.228 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.228 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:47.229 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768493+0000", 2024-09-16T20:49:47.229 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2270000000000004e-06 2024-09-16T20:49:47.229 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.229 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.229 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:47.229 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819672+0000", 2024-09-16T20:49:47.229 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051178605000000002 2024-09-16T20:49:47.229 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.229 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.229 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:47.229 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819711+0000", 2024-09-16T20:49:47.229 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9357000000000003e-05 2024-09-16T20:49:47.230 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.230 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.230 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:47.230 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819762+0000", 2024-09-16T20:49:47.230 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1351999999999999e-05 2024-09-16T20:49:47.230 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:47.230 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:47.230 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:47.230 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.230 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.230 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5024 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2150400~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:47.231 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768501+0000", 2024-09-16T20:49:47.231 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 87.405425440000002, 2024-09-16T20:49:47.231 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051402678, 2024-09-16T20:49:47.231 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:47.231 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:47.231 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:47.231 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:47.231 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:47.231 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:47.231 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5024 2024-09-16T20:49:47.231 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.232 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:47.232 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.232 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:47.232 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768501+0000", 2024-09-16T20:49:47.232 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:47.232 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.232 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.232 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:47.232 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768501+0000", 2024-09-16T20:49:47.232 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:47.232 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.232 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.233 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:47.233 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768619+0000", 2024-09-16T20:49:47.233 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118187 2024-09-16T20:49:47.233 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.233 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.233 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:47.233 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768624+0000", 2024-09-16T20:49:47.233 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3259999999999997e-06 2024-09-16T20:49:47.233 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.233 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.233 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:47.233 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768631+0000", 2024-09-16T20:49:47.234 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6490000000000008e-06 2024-09-16T20:49:47.234 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.234 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.234 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:47.234 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768637+0000", 2024-09-16T20:49:47.234 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4210000000000003e-06 2024-09-16T20:49:47.234 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.234 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.234 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:47.234 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819800+0000", 2024-09-16T20:49:47.234 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051163486000000001 2024-09-16T20:49:47.234 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.235 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.235 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:47.235 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819845+0000", 2024-09-16T20:49:47.235 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4830999999999998e-05 2024-09-16T20:49:47.235 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.235 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.235 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:47.235 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819904+0000", 2024-09-16T20:49:47.235 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8777999999999999e-05 2024-09-16T20:49:47.235 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:47.235 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:47.235 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:47.235 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.236 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.236 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5026 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2158592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:47.236 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768646+0000", 2024-09-16T20:49:47.236 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 87.405279992000004, 2024-09-16T20:49:47.236 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051384386999999997, 2024-09-16T20:49:47.236 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:47.236 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:47.236 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:47.236 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:47.236 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:47.236 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:47.236 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5026 2024-09-16T20:49:47.236 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.237 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:47.237 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.237 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:47.237 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768646+0000", 2024-09-16T20:49:47.237 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:47.237 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.237 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.237 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:47.237 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768646+0000", 2024-09-16T20:49:47.237 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:47.237 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.237 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.237 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:47.238 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768775+0000", 2024-09-16T20:49:47.238 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012813200000000001 2024-09-16T20:49:47.238 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.238 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.238 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:47.238 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768778+0000", 2024-09-16T20:49:47.238 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2150000000000001e-06 2024-09-16T20:49:47.238 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.238 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.238 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:47.238 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768786+0000", 2024-09-16T20:49:47.239 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3259999999999996e-06 2024-09-16T20:49:47.239 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.239 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.239 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:47.239 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768790+0000", 2024-09-16T20:49:47.239 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.216e-06 2024-09-16T20:49:47.239 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.239 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.239 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:47.239 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819939+0000", 2024-09-16T20:49:47.239 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051149018999999997 2024-09-16T20:49:47.239 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.240 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.240 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:47.240 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819979+0000", 2024-09-16T20:49:47.240 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9311999999999999e-05 2024-09-16T20:49:47.240 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.240 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.240 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:47.240 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820031+0000", 2024-09-16T20:49:47.240 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2166999999999997e-05 2024-09-16T20:49:47.240 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:47.240 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:47.240 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:47.241 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.241 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.241 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5028 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2166784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:47.241 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768798+0000", 2024-09-16T20:49:47.241 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 87.405128348999995, 2024-09-16T20:49:47.241 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051388912000000002, 2024-09-16T20:49:47.241 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:47.241 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:47.241 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:47.241 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:47.241 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:47.241 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:47.242 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5028 2024-09-16T20:49:47.242 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.242 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:47.242 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.242 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:47.242 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768798+0000", 2024-09-16T20:49:47.242 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:47.242 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.242 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.242 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:47.242 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768798+0000", 2024-09-16T20:49:47.243 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:47.243 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.243 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.243 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:47.243 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768989+0000", 2024-09-16T20:49:47.243 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000190951 2024-09-16T20:49:47.243 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.243 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.243 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:47.243 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768993+0000", 2024-09-16T20:49:47.243 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9770000000000002e-06 2024-09-16T20:49:47.244 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.244 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.244 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:47.244 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.769002+0000", 2024-09-16T20:49:47.244 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5469999999999997e-06 2024-09-16T20:49:47.244 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.244 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.244 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:47.244 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.769020+0000", 2024-09-16T20:49:47.244 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8216999999999999e-05 2024-09-16T20:49:47.244 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.245 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.245 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:47.245 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820070+0000", 2024-09-16T20:49:47.245 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051049998999999999 2024-09-16T20:49:47.245 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.245 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.245 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:47.245 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820119+0000", 2024-09-16T20:49:47.245 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8962000000000003e-05 2024-09-16T20:49:47.245 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.245 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.245 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:47.246 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820187+0000", 2024-09-16T20:49:47.246 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8258999999999998e-05 2024-09-16T20:49:47.246 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:47.246 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:47.246 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:47.246 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.246 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.246 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5030 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2174976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:47.246 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.769031+0000", 2024-09-16T20:49:47.246 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 87.404895236000002, 2024-09-16T20:49:47.246 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051289830000000002, 2024-09-16T20:49:47.246 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:47.246 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:47.247 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:47.247 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:47.247 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:47.247 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:47.247 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5030 2024-09-16T20:49:47.247 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.247 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:47.247 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.247 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:47.247 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.769031+0000", 2024-09-16T20:49:47.247 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:47.248 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.248 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.248 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:47.248 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.769031+0000", 2024-09-16T20:49:47.248 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:47.248 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.248 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.248 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:47.248 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.773332+0000", 2024-09-16T20:49:47.248 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0043004039999999999 2024-09-16T20:49:47.248 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.248 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.249 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:47.249 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.773336+0000", 2024-09-16T20:49:47.249 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7849999999999999e-06 2024-09-16T20:49:47.249 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.249 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.249 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:47.249 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.773347+0000", 2024-09-16T20:49:47.249 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0482999999999999e-05 2024-09-16T20:49:47.249 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.249 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.249 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:47.249 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.773354+0000", 2024-09-16T20:49:47.249 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1559999999999998e-06 2024-09-16T20:49:47.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.250 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.250 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:47.250 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820216+0000", 2024-09-16T20:49:47.250 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046861591000000001 2024-09-16T20:49:47.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.250 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.250 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:47.250 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820258+0000", 2024-09-16T20:49:47.250 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2653e-05 2024-09-16T20:49:47.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.250 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.250 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:47.251 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820321+0000", 2024-09-16T20:49:47.251 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2757999999999998e-05 2024-09-16T20:49:47.251 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:47.251 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:47.251 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:47.251 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.251 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.251 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5067 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 6918144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:47.251 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.775279+0000", 2024-09-16T20:49:47.251 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 87.398646970000001, 2024-09-16T20:49:47.251 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050171387999999997, 2024-09-16T20:49:47.251 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:47.251 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:47.252 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:47.252 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:47.252 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:47.252 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:47.252 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5067 2024-09-16T20:49:47.252 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.252 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:47.252 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.252 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:47.252 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775279+0000", 2024-09-16T20:49:47.252 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:47.252 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.252 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.253 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:47.253 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775279+0000", 2024-09-16T20:49:47.253 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:47.253 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.253 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.253 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:47.253 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775432+0000", 2024-09-16T20:49:47.253 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015257399999999999 2024-09-16T20:49:47.253 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.253 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.253 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:47.253 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775436+0000", 2024-09-16T20:49:47.254 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0180000000000003e-06 2024-09-16T20:49:47.254 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.254 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.254 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:47.254 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775447+0000", 2024-09-16T20:49:47.254 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.077e-05 2024-09-16T20:49:47.254 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.254 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.254 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:47.254 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775456+0000", 2024-09-16T20:49:47.254 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4089999999999992e-06 2024-09-16T20:49:47.254 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.254 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.255 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:47.255 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825150+0000", 2024-09-16T20:49:47.255 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049693711000000002 2024-09-16T20:49:47.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.255 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.255 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:47.255 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825337+0000", 2024-09-16T20:49:47.255 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000187214 2024-09-16T20:49:47.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.255 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.255 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:47.255 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825451+0000", 2024-09-16T20:49:47.255 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011369199999999999 2024-09-16T20:49:47.255 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:47.256 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:47.256 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:47.256 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.256 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.256 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5068 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2330624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:47.256 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.775469+0000", 2024-09-16T20:49:47.256 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 87.398457008999998, 2024-09-16T20:49:47.256 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050198334999999997, 2024-09-16T20:49:47.256 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:47.256 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:47.256 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:47.256 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:47.256 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:47.257 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:47.257 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5068 2024-09-16T20:49:47.257 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.257 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:47.257 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.257 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:47.257 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775469+0000", 2024-09-16T20:49:47.257 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:47.257 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.257 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.257 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:47.257 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775469+0000", 2024-09-16T20:49:47.258 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:47.258 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.258 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.258 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:47.258 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775720+0000", 2024-09-16T20:49:47.258 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000250661 2024-09-16T20:49:47.258 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.258 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.258 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:47.258 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775724+0000", 2024-09-16T20:49:47.258 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8700000000000002e-06 2024-09-16T20:49:47.258 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.259 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.259 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:47.259 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775733+0000", 2024-09-16T20:49:47.259 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4239999999999999e-06 2024-09-16T20:49:47.259 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.259 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.259 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:47.259 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775740+0000", 2024-09-16T20:49:47.259 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9519999999999997e-06 2024-09-16T20:49:47.259 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.259 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.259 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:47.260 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825484+0000", 2024-09-16T20:49:47.260 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049743385000000001 2024-09-16T20:49:47.260 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.260 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.260 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:47.260 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825572+0000", 2024-09-16T20:49:47.260 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7834e-05 2024-09-16T20:49:47.260 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.260 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.260 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:47.260 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825668+0000", 2024-09-16T20:49:47.261 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6209e-05 2024-09-16T20:49:47.261 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:47.261 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:47.261 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:47.261 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.261 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.261 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5069 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 6926336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:47.261 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.775753+0000", 2024-09-16T20:49:47.261 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 87.398173786000001, 2024-09-16T20:49:47.261 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050146430999999998, 2024-09-16T20:49:47.261 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:47.261 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:47.261 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:47.262 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:47.262 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:47.262 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:47.262 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5069 2024-09-16T20:49:47.262 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.262 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:47.262 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.262 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:47.262 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775753+0000", 2024-09-16T20:49:47.262 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:47.262 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.262 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.262 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:47.263 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775753+0000", 2024-09-16T20:49:47.263 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:47.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.263 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.263 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:47.263 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775865+0000", 2024-09-16T20:49:47.263 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000112402 2024-09-16T20:49:47.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.263 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.263 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:47.263 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775869+0000", 2024-09-16T20:49:47.263 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5130000000000001e-06 2024-09-16T20:49:47.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.264 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.264 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:47.264 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775877+0000", 2024-09-16T20:49:47.264 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1359999999999997e-06 2024-09-16T20:49:47.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.264 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.264 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:47.264 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775882+0000", 2024-09-16T20:49:47.264 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6710000000000004e-06 2024-09-16T20:49:47.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.265 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.265 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:47.265 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825729+0000", 2024-09-16T20:49:47.265 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049846898000000001 2024-09-16T20:49:47.265 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.265 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.265 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:47.265 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825807+0000", 2024-09-16T20:49:47.265 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7875999999999998e-05 2024-09-16T20:49:47.265 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.265 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.265 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:47.266 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825899+0000", 2024-09-16T20:49:47.266 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1935000000000007e-05 2024-09-16T20:49:47.266 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:47.266 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:47.266 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:47.266 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.266 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.266 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5070 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2338816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:47.266 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.775892+0000", 2024-09-16T20:49:47.266 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 87.398033932000004, 2024-09-16T20:49:47.266 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050155311000000001, 2024-09-16T20:49:47.266 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:47.266 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:47.267 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:47.267 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:47.267 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:47.267 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:47.267 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5070 2024-09-16T20:49:47.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.267 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:47.267 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.268 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:47.268 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775892+0000", 2024-09-16T20:49:47.268 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:47.268 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.268 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.268 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:47.268 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775892+0000", 2024-09-16T20:49:47.268 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:47.268 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.268 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.268 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:47.268 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776142+0000", 2024-09-16T20:49:47.269 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00024950500000000002 2024-09-16T20:49:47.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.269 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.269 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:47.269 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776146+0000", 2024-09-16T20:49:47.269 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2819999999999998e-06 2024-09-16T20:49:47.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.269 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.269 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:47.269 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776157+0000", 2024-09-16T20:49:47.269 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0807e-05 2024-09-16T20:49:47.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.270 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.270 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:47.270 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776165+0000", 2024-09-16T20:49:47.270 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1300000000000001e-06 2024-09-16T20:49:47.270 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.270 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.270 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:47.270 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825935+0000", 2024-09-16T20:49:47.270 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049769300000000002 2024-09-16T20:49:47.270 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.270 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.271 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:47.271 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825991+0000", 2024-09-16T20:49:47.271 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6177000000000001e-05 2024-09-16T20:49:47.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.271 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.271 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:47.271 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826048+0000", 2024-09-16T20:49:47.271 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.711e-05 2024-09-16T20:49:47.271 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:47.271 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:47.271 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:47.272 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.272 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.272 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5071 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 6934528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:47.272 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.776177+0000", 2024-09-16T20:49:47.272 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 87.397749528999995, 2024-09-16T20:49:47.272 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050036886000000003, 2024-09-16T20:49:47.272 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:47.272 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:47.272 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:47.273 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:47.273 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:47.273 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:47.273 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5071 2024-09-16T20:49:47.273 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.273 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:47.273 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.273 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:47.273 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776177+0000", 2024-09-16T20:49:47.273 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:47.273 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.273 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.273 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:47.274 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776177+0000", 2024-09-16T20:49:47.274 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:47.274 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.274 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.274 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:47.274 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776339+0000", 2024-09-16T20:49:47.274 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016238099999999999 2024-09-16T20:49:47.274 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.274 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.274 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:47.274 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776343+0000", 2024-09-16T20:49:47.274 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9670000000000003e-06 2024-09-16T20:49:47.275 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.275 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.275 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:47.275 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776354+0000", 2024-09-16T20:49:47.275 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0789e-05 2024-09-16T20:49:47.275 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.275 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.275 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:47.275 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776361+0000", 2024-09-16T20:49:47.275 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5200000000000003e-06 2024-09-16T20:49:47.275 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.275 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.276 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:47.276 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826078+0000", 2024-09-16T20:49:47.276 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049717513999999997 2024-09-16T20:49:47.276 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.276 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.276 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:47.276 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826141+0000", 2024-09-16T20:49:47.276 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2446000000000005e-05 2024-09-16T20:49:47.276 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.276 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.276 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:47.276 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826214+0000", 2024-09-16T20:49:47.277 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3269000000000006e-05 2024-09-16T20:49:47.277 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:47.277 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:47.277 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:47.277 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.277 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.277 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5072 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2347008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:47.277 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.776370+0000", 2024-09-16T20:49:47.277 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 87.397555968000006, 2024-09-16T20:49:47.277 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050097653999999998, 2024-09-16T20:49:47.277 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:47.277 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:47.277 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:47.278 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:47.278 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:47.278 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:47.278 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5072 2024-09-16T20:49:47.278 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.278 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:47.278 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.278 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:47.278 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776370+0000", 2024-09-16T20:49:47.278 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:47.278 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.278 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.279 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:47.279 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776370+0000", 2024-09-16T20:49:47.279 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:47.279 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.279 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.279 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:47.279 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776645+0000", 2024-09-16T20:49:47.279 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000274634 2024-09-16T20:49:47.279 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.279 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.279 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:47.279 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776654+0000", 2024-09-16T20:49:47.279 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9390000000000003e-06 2024-09-16T20:49:47.280 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.280 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.280 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:47.280 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776665+0000", 2024-09-16T20:49:47.280 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0524e-05 2024-09-16T20:49:47.280 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.280 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.280 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:47.280 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776671+0000", 2024-09-16T20:49:47.280 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7390000000000002e-06 2024-09-16T20:49:47.280 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.280 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.281 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:47.281 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826261+0000", 2024-09-16T20:49:47.281 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049589995999999997 2024-09-16T20:49:47.281 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.281 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.281 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:47.281 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826363+0000", 2024-09-16T20:49:47.281 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010158999999999999 2024-09-16T20:49:47.281 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:47.281 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:47.281 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:47.281 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826468+0000", 2024-09-16T20:49:47.282 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000105232 2024-09-16T20:49:47.282 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:47.282 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:47.282 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:47.282 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:47.282 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:47.282 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:47.282 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T20:49:47.284 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:49:47.285 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:47.282+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:47.384 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:47.383+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:47.406 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:49:47.406 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:49:47.423 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:49:47.423 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:49:47.432 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:47.432 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:49:47.432 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:49:47.432 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:47.440 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:49:47.440 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:49:47.441 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:49:47.456 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:49:47.456 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:49:47.473 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:49:47.474 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:49:47.485 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:47.484+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:47.490 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:49:47.490 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:49:47.548 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:47.548 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:47.556 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:49:47.586 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:47.585+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:47.668 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:47.668 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:47.677 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:49:47.721 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:47.720+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:47.785 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:47.785 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:47.786 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:47.785+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:47.794 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:49:47.887 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:47.886+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:47.941 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:47.941 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:47.949 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:49:47.993 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:47.992+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:48.056 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:48.057 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:48.065 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:49:48.106 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:48.105+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:48.189 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:48.188+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:48.212 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:48.212 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:48.221 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:49:48.290 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:48.289+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:48.327 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:48.327 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:48.335 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:49:48.419 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:48.418+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:48.482 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:48.482 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:48.491 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:49:48.492 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:48.491+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:48.593 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:48.592+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:48.643 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:48.643 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:48.651 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:49:48.685 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T20:49:48.695 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:48.694+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:48.761 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:48.762 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:48.771 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:49:48.814 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:48.813+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:48.886 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:48.886 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:48.896 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:49:48.897 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:48.894+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:48.997 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:48.996+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:49.014 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:49.014 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:49.025 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:49:49.110 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:49.109+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:49.139 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #31, image image_client.0-clone31 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent32 2024-09-16T20:49:49.185 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:49.185 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:49.196 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:49:49.198 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:49.197+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:49.300 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:49.298+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:49.317 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 3, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-16T20:49:49.318 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T20:49:49.318 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-09-16T20:49:49.318 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T20:49:49.318 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T20:49:49.318 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2024-09-16T20:49:49.359 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:49.359 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:49.370 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:49:49.412 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:49.411+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:49.439 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:49.439 INFO:teuthology.orchestra.run.smithi029.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-16T20:49:49.439 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:49.500 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:49.499+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:49.501 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T20:49:49.517 INFO:teuthology.orchestra.run.smithi139.stdout:4098 trunc from 0xaae7400 to 0x6d76a00 2024-09-16T20:49:49.522 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:49.522 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:49.532 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:49:49.553 INFO:teuthology.orchestra.run.smithi139.stdout:4101 write 0x9aa0200 thru 0x9aac922 (0xc723 bytes) 2024-09-16T20:49:49.611 INFO:teuthology.orchestra.run.smithi029.stderr:noscrub is unset 2024-09-16T20:49:49.616 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:49.615+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:49.659 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:49:49.659 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:49:49.678 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:49:49.678 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:49:49.683 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:49.683 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:49.692 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:49:49.694 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:49:49.695 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:49:49.712 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:49:49.712 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:49:49.728 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:49:49.729 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:49:49.732 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:49.731+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:49.745 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:49:49.745 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:49:49.803 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:49.802+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:49.827 INFO:teuthology.orchestra.run.smithi139.stdout:4104 write 0xa24f600 thru 0xa25f0da (0xfadb bytes) 2024-09-16T20:49:49.827 INFO:teuthology.orchestra.run.smithi139.stdout:4105 read 0x9ba4600 thru 0x9bad187 (0x8b88 bytes) 2024-09-16T20:49:49.827 INFO:teuthology.orchestra.run.smithi139.stdout:4106 write 0xf11600 thru 0xf1880a (0x720b bytes) 2024-09-16T20:49:49.828 INFO:teuthology.orchestra.run.smithi139.stdout:4107 write 0x64d8800 thru 0x64da2ce (0x1acf bytes) 2024-09-16T20:49:49.828 INFO:teuthology.orchestra.run.smithi139.stdout:4109 read 0x16a0000 thru 0x16ad274 (0xd275 bytes) 2024-09-16T20:49:49.840 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:49.840 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:49:49.840 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:49:49.841 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:49:49.841 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.841 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:4904 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 7309312~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:49.841 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767724+0000", 2024-09-16T20:49:49.841 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 90.070824586000001, 2024-09-16T20:49:49.841 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050270214000000001, 2024-09-16T20:49:49.841 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:49.841 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:49.841 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:49.841 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:49.841 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:49.841 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:49.841 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4904 2024-09-16T20:49:49.841 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.842 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:49.842 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.842 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:49.842 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767724+0000", 2024-09-16T20:49:49.842 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:49.842 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.842 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.842 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:49.842 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767724+0000", 2024-09-16T20:49:49.842 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:49.842 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.842 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.842 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:49.842 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767724+0000", 2024-09-16T20:49:49.842 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1399999999999998e-07 2024-09-16T20:49:49.843 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.843 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.843 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:49.843 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767725+0000", 2024-09-16T20:49:49.843 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0380000000000001e-06 2024-09-16T20:49:49.843 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.843 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.843 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:49.843 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767729+0000", 2024-09-16T20:49:49.843 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2490000000000001e-06 2024-09-16T20:49:49.843 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.843 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.843 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:49.843 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767731+0000", 2024-09-16T20:49:49.844 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5940000000000001e-06 2024-09-16T20:49:49.844 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.844 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.844 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:49.844 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.817787+0000", 2024-09-16T20:49:49.844 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050055792000000002 2024-09-16T20:49:49.844 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.844 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.844 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:49.844 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.817883+0000", 2024-09-16T20:49:49.844 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5606999999999995e-05 2024-09-16T20:49:49.844 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.844 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.844 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:49.845 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.817994+0000", 2024-09-16T20:49:49.845 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011162000000000001 2024-09-16T20:49:49.845 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:49.845 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:49.845 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:49.845 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.845 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.845 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:4906 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 7317504~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:49.845 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767734+0000", 2024-09-16T20:49:49.845 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 90.070814322999993, 2024-09-16T20:49:49.845 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050406608999999998, 2024-09-16T20:49:49.845 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:49.845 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:49.845 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:49.846 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:49.846 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:49.846 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:49.846 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4906 2024-09-16T20:49:49.846 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.846 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:49.846 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.846 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:49.846 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767734+0000", 2024-09-16T20:49:49.846 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:49.846 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.846 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.846 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:49.846 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767734+0000", 2024-09-16T20:49:49.846 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:49.847 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.847 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.847 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:49.847 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767735+0000", 2024-09-16T20:49:49.847 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.05e-07 2024-09-16T20:49:49.847 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.847 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.847 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:49.847 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767736+0000", 2024-09-16T20:49:49.847 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0240000000000001e-06 2024-09-16T20:49:49.847 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.847 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.847 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:49.847 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767739+0000", 2024-09-16T20:49:49.847 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.067e-06 2024-09-16T20:49:49.848 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.848 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.848 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:49.848 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767742+0000", 2024-09-16T20:49:49.848 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.23e-06 2024-09-16T20:49:49.848 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.848 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.848 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:49.848 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818027+0000", 2024-09-16T20:49:49.848 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050285102999999998 2024-09-16T20:49:49.848 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.848 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.848 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:49.848 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818072+0000", 2024-09-16T20:49:49.849 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5129000000000001e-05 2024-09-16T20:49:49.849 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.849 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.849 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:49.849 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818141+0000", 2024-09-16T20:49:49.849 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8850999999999995e-05 2024-09-16T20:49:49.849 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:49.849 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:49.849 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:49.849 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.849 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.849 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:4908 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 7325696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:49.849 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767745+0000", 2024-09-16T20:49:49.849 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 90.070803928000004, 2024-09-16T20:49:49.849 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050654230000000001, 2024-09-16T20:49:49.850 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:49.850 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:49.850 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:49.850 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:49.850 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:49.850 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:49.850 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4908 2024-09-16T20:49:49.850 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.850 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:49.850 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.850 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:49.850 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767745+0000", 2024-09-16T20:49:49.850 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:49.850 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.850 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.851 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:49.851 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767745+0000", 2024-09-16T20:49:49.851 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:49.851 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.851 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.851 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:49.851 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767745+0000", 2024-09-16T20:49:49.851 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1e-07 2024-09-16T20:49:49.851 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.851 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.851 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:49.851 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767749+0000", 2024-09-16T20:49:49.851 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.532e-06 2024-09-16T20:49:49.851 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.851 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.852 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:49.852 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767753+0000", 2024-09-16T20:49:49.852 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4939999999999999e-06 2024-09-16T20:49:49.852 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.852 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.852 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:49.852 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767756+0000", 2024-09-16T20:49:49.852 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.61e-06 2024-09-16T20:49:49.852 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.852 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.852 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:49.852 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818184+0000", 2024-09-16T20:49:49.852 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050428615000000003 2024-09-16T20:49:49.853 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.853 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.853 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:49.853 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818276+0000", 2024-09-16T20:49:49.853 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2077999999999995e-05 2024-09-16T20:49:49.853 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.853 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.853 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:49.853 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818399+0000", 2024-09-16T20:49:49.853 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012269099999999999 2024-09-16T20:49:49.853 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:49.853 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:49.853 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:49.854 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.854 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.854 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:4910 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 7333888~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:49.854 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767759+0000", 2024-09-16T20:49:49.854 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 90.070790037999998, 2024-09-16T20:49:49.854 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050815076000000001, 2024-09-16T20:49:49.854 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:49.854 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:49.854 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:49.854 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:49.854 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:49.854 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:49.854 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4910 2024-09-16T20:49:49.854 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.854 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:49.855 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.855 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:49.855 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767759+0000", 2024-09-16T20:49:49.855 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:49.855 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.855 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.855 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:49.855 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767759+0000", 2024-09-16T20:49:49.855 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:49.855 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.855 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.855 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:49.855 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767759+0000", 2024-09-16T20:49:49.855 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2600000000000001e-07 2024-09-16T20:49:49.856 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.856 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.856 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:49.856 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767760+0000", 2024-09-16T20:49:49.856 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0640000000000001e-06 2024-09-16T20:49:49.856 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.856 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.856 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:49.856 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767763+0000", 2024-09-16T20:49:49.856 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.994e-06 2024-09-16T20:49:49.856 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.856 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.856 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:49.857 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767765+0000", 2024-09-16T20:49:49.857 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5220000000000002e-06 2024-09-16T20:49:49.857 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.857 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.857 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:49.857 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818438+0000", 2024-09-16T20:49:49.857 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050672602999999997 2024-09-16T20:49:49.857 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.857 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.857 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:49.857 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818481+0000", 2024-09-16T20:49:49.857 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2967999999999999e-05 2024-09-16T20:49:49.857 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.858 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.858 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:49.858 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818574+0000", 2024-09-16T20:49:49.858 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2699000000000001e-05 2024-09-16T20:49:49.858 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:49.858 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:49.858 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:49.858 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.858 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.858 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5012 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2101248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:49.858 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767768+0000", 2024-09-16T20:49:49.858 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 90.070780389000006, 2024-09-16T20:49:49.858 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051084682999999999, 2024-09-16T20:49:49.858 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:49.859 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:49.859 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:49.859 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:49.859 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:49.859 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:49.859 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5012 2024-09-16T20:49:49.859 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.859 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:49.859 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.859 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:49.859 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767768+0000", 2024-09-16T20:49:49.859 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:49.859 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.859 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.859 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:49.859 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767768+0000", 2024-09-16T20:49:49.860 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:49.860 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.860 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.860 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:49.860 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767769+0000", 2024-09-16T20:49:49.860 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1500000000000001e-07 2024-09-16T20:49:49.860 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.860 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.860 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:49.860 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767770+0000", 2024-09-16T20:49:49.860 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.015e-06 2024-09-16T20:49:49.860 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.860 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.860 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:49.860 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767773+0000", 2024-09-16T20:49:49.861 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1070000000000001e-06 2024-09-16T20:49:49.861 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.861 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.861 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:49.861 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767775+0000", 2024-09-16T20:49:49.861 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.508e-06 2024-09-16T20:49:49.861 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.861 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.861 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:49.861 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818630+0000", 2024-09-16T20:49:49.861 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050855282000000002 2024-09-16T20:49:49.861 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.861 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.861 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:49.861 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818732+0000", 2024-09-16T20:49:49.862 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101531 2024-09-16T20:49:49.862 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.862 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.862 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:49.862 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818853+0000", 2024-09-16T20:49:49.862 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000121025 2024-09-16T20:49:49.862 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:49.862 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:49.862 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:49.862 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.862 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.862 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5014 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2109440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:49.862 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767778+0000", 2024-09-16T20:49:49.862 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 90.070770452999994, 2024-09-16T20:49:49.863 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051260776000000001, 2024-09-16T20:49:49.863 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:49.863 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:49.863 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:49.863 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:49.863 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:49.863 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:49.863 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5014 2024-09-16T20:49:49.863 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.863 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:49.863 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.863 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:49.863 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767778+0000", 2024-09-16T20:49:49.863 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:49.864 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.864 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.864 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:49.864 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767778+0000", 2024-09-16T20:49:49.864 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:49.864 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.864 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.864 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:49.864 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767824+0000", 2024-09-16T20:49:49.864 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5890999999999998e-05 2024-09-16T20:49:49.864 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.864 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.864 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:49.864 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767827+0000", 2024-09-16T20:49:49.865 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.644e-06 2024-09-16T20:49:49.865 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.865 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.865 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:49.865 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767832+0000", 2024-09-16T20:49:49.865 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2689999999999999e-06 2024-09-16T20:49:49.865 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.865 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.865 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:49.865 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767835+0000", 2024-09-16T20:49:49.865 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2100000000000002e-06 2024-09-16T20:49:49.865 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.865 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.865 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:49.866 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818887+0000", 2024-09-16T20:49:49.866 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051051276999999999 2024-09-16T20:49:49.866 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.866 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.866 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:49.866 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.818953+0000", 2024-09-16T20:49:49.866 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6105999999999994e-05 2024-09-16T20:49:49.866 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.866 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.866 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:49.866 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819039+0000", 2024-09-16T20:49:49.866 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6378999999999994e-05 2024-09-16T20:49:49.866 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:49.866 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:49.867 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:49.867 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.867 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.867 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5016 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2117632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:49.867 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.767841+0000", 2024-09-16T20:49:49.867 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 90.070707773999999, 2024-09-16T20:49:49.867 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051433288000000001, 2024-09-16T20:49:49.867 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:49.867 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:49.867 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:49.867 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:49.867 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:49.867 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:49.867 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5016 2024-09-16T20:49:49.868 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.868 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:49.868 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.868 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:49.868 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767841+0000", 2024-09-16T20:49:49.868 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:49.868 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.868 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.868 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:49.868 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.767841+0000", 2024-09-16T20:49:49.868 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:49.868 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.868 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.868 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:49.869 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768156+0000", 2024-09-16T20:49:49.869 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00031505699999999999 2024-09-16T20:49:49.869 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.869 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.869 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:49.869 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768159+0000", 2024-09-16T20:49:49.869 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1659999999999998e-06 2024-09-16T20:49:49.869 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.869 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.869 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:49.869 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768166+0000", 2024-09-16T20:49:49.869 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0550000000000002e-06 2024-09-16T20:49:49.869 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.869 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.869 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:49.870 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768171+0000", 2024-09-16T20:49:49.870 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3170000000000003e-06 2024-09-16T20:49:49.870 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.870 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.870 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:49.870 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819084+0000", 2024-09-16T20:49:49.870 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050913355 2024-09-16T20:49:49.870 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.870 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.870 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:49.870 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819170+0000", 2024-09-16T20:49:49.870 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6328000000000004e-05 2024-09-16T20:49:49.870 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.870 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.871 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:49.871 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819274+0000", 2024-09-16T20:49:49.871 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010401 2024-09-16T20:49:49.871 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:49.871 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:49.871 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:49.871 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.871 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.871 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5018 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2125824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:49.871 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768178+0000", 2024-09-16T20:49:49.871 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 90.070370240000003, 2024-09-16T20:49:49.871 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051227013000000002, 2024-09-16T20:49:49.871 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:49.871 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:49.872 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:49.872 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:49.872 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:49.872 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:49.872 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5018 2024-09-16T20:49:49.872 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.872 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:49.872 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.872 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:49.872 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768178+0000", 2024-09-16T20:49:49.872 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:49.872 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.872 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.872 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:49.873 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768178+0000", 2024-09-16T20:49:49.873 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:49.873 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.873 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.873 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:49.873 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768179+0000", 2024-09-16T20:49:49.873 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9200000000000002e-07 2024-09-16T20:49:49.873 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.873 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.873 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:49.873 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768180+0000", 2024-09-16T20:49:49.873 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3790000000000001e-06 2024-09-16T20:49:49.873 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.874 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.874 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:49.874 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768184+0000", 2024-09-16T20:49:49.874 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5470000000000001e-06 2024-09-16T20:49:49.874 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.874 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.874 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:49.874 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768187+0000", 2024-09-16T20:49:49.874 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8119999999999999e-06 2024-09-16T20:49:49.874 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.874 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.874 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:49.874 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819305+0000", 2024-09-16T20:49:49.874 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051118683999999998 2024-09-16T20:49:49.875 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.875 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.875 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:49.875 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819351+0000", 2024-09-16T20:49:49.875 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5757999999999998e-05 2024-09-16T20:49:49.875 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.875 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.875 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:49.875 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819405+0000", 2024-09-16T20:49:49.875 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4441000000000003e-05 2024-09-16T20:49:49.875 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:49.875 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:49.875 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:49.875 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.876 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.876 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5020 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2134016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:49.876 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768190+0000", 2024-09-16T20:49:49.876 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 90.070358561999996, 2024-09-16T20:49:49.876 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051449661000000001, 2024-09-16T20:49:49.876 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:49.876 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:49.876 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:49.876 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:49.876 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:49.876 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:49.876 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5020 2024-09-16T20:49:49.876 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.876 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:49.877 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.877 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:49.877 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768190+0000", 2024-09-16T20:49:49.877 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:49.877 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.877 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.877 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:49.877 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768190+0000", 2024-09-16T20:49:49.877 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:49.877 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.877 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.877 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:49.877 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768282+0000", 2024-09-16T20:49:49.878 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2263999999999996e-05 2024-09-16T20:49:49.878 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.878 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.878 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:49.878 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768286+0000", 2024-09-16T20:49:49.878 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.529e-06 2024-09-16T20:49:49.878 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.878 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.878 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:49.878 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768294+0000", 2024-09-16T20:49:49.878 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6990000000000003e-06 2024-09-16T20:49:49.878 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.879 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.879 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:49.879 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768306+0000", 2024-09-16T20:49:49.879 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2112e-05 2024-09-16T20:49:49.879 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.879 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.879 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:49.879 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819449+0000", 2024-09-16T20:49:49.879 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051143521999999997 2024-09-16T20:49:49.879 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.879 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.879 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:49.879 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819539+0000", 2024-09-16T20:49:49.879 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0022e-05 2024-09-16T20:49:49.880 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.880 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.880 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:49.880 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819640+0000", 2024-09-16T20:49:49.880 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000100513 2024-09-16T20:49:49.880 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:49.880 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:49.880 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:49.880 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.880 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.880 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5022 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2142208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:49.880 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768326+0000", 2024-09-16T20:49:49.880 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 90.070223173000002, 2024-09-16T20:49:49.880 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051436952000000001, 2024-09-16T20:49:49.880 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:49.881 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:49.881 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:49.881 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:49.881 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:49.881 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:49.881 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5022 2024-09-16T20:49:49.881 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.881 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:49.881 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.881 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:49.881 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768326+0000", 2024-09-16T20:49:49.881 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:49.881 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.881 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.881 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:49.882 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768326+0000", 2024-09-16T20:49:49.882 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:49.882 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.882 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.882 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:49.882 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768477+0000", 2024-09-16T20:49:49.882 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015109300000000001 2024-09-16T20:49:49.882 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.882 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.882 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:49.882 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768480+0000", 2024-09-16T20:49:49.882 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8580000000000002e-06 2024-09-16T20:49:49.883 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.883 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.883 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:49.883 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768489+0000", 2024-09-16T20:49:49.883 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4600000000000003e-06 2024-09-16T20:49:49.883 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.883 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.883 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:49.883 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768493+0000", 2024-09-16T20:49:49.883 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2270000000000004e-06 2024-09-16T20:49:49.883 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.883 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.883 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:49.883 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819672+0000", 2024-09-16T20:49:49.884 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051178605000000002 2024-09-16T20:49:49.884 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.884 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.884 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:49.884 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819711+0000", 2024-09-16T20:49:49.884 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9357000000000003e-05 2024-09-16T20:49:49.884 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.884 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.884 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:49.884 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819762+0000", 2024-09-16T20:49:49.884 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1351999999999999e-05 2024-09-16T20:49:49.884 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:49.885 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:49.885 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:49.885 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.885 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.885 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5024 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2150400~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:49.885 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768501+0000", 2024-09-16T20:49:49.885 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 90.070047695, 2024-09-16T20:49:49.885 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051402678, 2024-09-16T20:49:49.885 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:49.885 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:49.885 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:49.885 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:49.885 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:49.885 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:49.886 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5024 2024-09-16T20:49:49.886 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.886 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:49.886 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.886 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:49.886 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768501+0000", 2024-09-16T20:49:49.886 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:49.886 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.886 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.886 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:49.886 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768501+0000", 2024-09-16T20:49:49.886 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:49.886 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.886 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.887 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:49.887 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768619+0000", 2024-09-16T20:49:49.887 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118187 2024-09-16T20:49:49.887 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.887 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.887 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:49.887 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768624+0000", 2024-09-16T20:49:49.887 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3259999999999997e-06 2024-09-16T20:49:49.887 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.887 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.887 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:49.887 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768631+0000", 2024-09-16T20:49:49.887 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6490000000000008e-06 2024-09-16T20:49:49.888 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.888 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.888 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:49.888 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768637+0000", 2024-09-16T20:49:49.888 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4210000000000003e-06 2024-09-16T20:49:49.888 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.888 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.888 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:49.888 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819800+0000", 2024-09-16T20:49:49.888 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051163486000000001 2024-09-16T20:49:49.888 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.888 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.888 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:49.889 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819845+0000", 2024-09-16T20:49:49.889 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4830999999999998e-05 2024-09-16T20:49:49.889 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.889 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.889 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:49.889 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819904+0000", 2024-09-16T20:49:49.889 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8777999999999999e-05 2024-09-16T20:49:49.889 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:49.889 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:49.889 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:49.889 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.889 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.889 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5026 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2158592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:49.889 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768646+0000", 2024-09-16T20:49:49.890 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 90.069902247000002, 2024-09-16T20:49:49.890 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051384386999999997, 2024-09-16T20:49:49.890 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:49.890 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:49.890 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:49.890 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:49.890 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:49.890 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:49.890 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5026 2024-09-16T20:49:49.890 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.890 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:49.890 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.890 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:49.890 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768646+0000", 2024-09-16T20:49:49.890 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:49.890 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.891 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.891 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:49.891 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768646+0000", 2024-09-16T20:49:49.891 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:49.891 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.891 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.891 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:49.891 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768775+0000", 2024-09-16T20:49:49.891 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012813200000000001 2024-09-16T20:49:49.891 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.891 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.891 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:49.891 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768778+0000", 2024-09-16T20:49:49.891 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2150000000000001e-06 2024-09-16T20:49:49.891 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.892 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.892 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:49.892 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768786+0000", 2024-09-16T20:49:49.892 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3259999999999996e-06 2024-09-16T20:49:49.892 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.892 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.892 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:49.892 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768790+0000", 2024-09-16T20:49:49.892 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.216e-06 2024-09-16T20:49:49.892 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.892 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.892 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:49.892 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819939+0000", 2024-09-16T20:49:49.892 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051149018999999997 2024-09-16T20:49:49.893 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.893 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.893 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:49.893 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.819979+0000", 2024-09-16T20:49:49.893 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9311999999999999e-05 2024-09-16T20:49:49.893 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.893 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.893 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:49.893 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820031+0000", 2024-09-16T20:49:49.893 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2166999999999997e-05 2024-09-16T20:49:49.893 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:49.893 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:49.893 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:49.894 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.894 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.894 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5028 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2166784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:49.894 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.768798+0000", 2024-09-16T20:49:49.894 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 90.069750604000006, 2024-09-16T20:49:49.894 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051388912000000002, 2024-09-16T20:49:49.894 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:49.894 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:49.894 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:49.894 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:49.894 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:49.894 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:49.894 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5028 2024-09-16T20:49:49.894 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.895 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:49.895 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.895 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:49.895 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768798+0000", 2024-09-16T20:49:49.895 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:49.895 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.895 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.895 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:49.895 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768798+0000", 2024-09-16T20:49:49.895 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:49.895 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.895 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.895 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:49.895 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768989+0000", 2024-09-16T20:49:49.895 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000190951 2024-09-16T20:49:49.896 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.896 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.896 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:49.896 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.768993+0000", 2024-09-16T20:49:49.896 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9770000000000002e-06 2024-09-16T20:49:49.896 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.896 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.896 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:49.896 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.769002+0000", 2024-09-16T20:49:49.896 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5469999999999997e-06 2024-09-16T20:49:49.896 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.896 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.896 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:49.896 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.769020+0000", 2024-09-16T20:49:49.896 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8216999999999999e-05 2024-09-16T20:49:49.897 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.897 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.897 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:49.897 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820070+0000", 2024-09-16T20:49:49.897 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051049998999999999 2024-09-16T20:49:49.897 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.897 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.897 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:49.897 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820119+0000", 2024-09-16T20:49:49.897 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8962000000000003e-05 2024-09-16T20:49:49.897 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.897 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.897 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:49.897 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820187+0000", 2024-09-16T20:49:49.898 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8258999999999998e-05 2024-09-16T20:49:49.898 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:49.898 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:49.898 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:49.898 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.898 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.898 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5030 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2174976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:49.898 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.769031+0000", 2024-09-16T20:49:49.898 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 90.069517490999999, 2024-09-16T20:49:49.898 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051289830000000002, 2024-09-16T20:49:49.898 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:49.898 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:49.898 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:49.898 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:49.898 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:49.899 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:49.899 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5030 2024-09-16T20:49:49.899 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.899 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:49.899 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.899 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:49.899 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.769031+0000", 2024-09-16T20:49:49.899 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:49.899 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.899 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.899 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:49.899 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.769031+0000", 2024-09-16T20:49:49.899 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:49.900 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.900 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.900 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:49.900 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.773332+0000", 2024-09-16T20:49:49.900 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0043004039999999999 2024-09-16T20:49:49.900 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.900 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.900 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:49.900 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.773336+0000", 2024-09-16T20:49:49.900 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7849999999999999e-06 2024-09-16T20:49:49.900 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.900 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.900 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:49.901 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.773347+0000", 2024-09-16T20:49:49.901 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0482999999999999e-05 2024-09-16T20:49:49.901 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.901 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.901 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:49.901 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.773354+0000", 2024-09-16T20:49:49.901 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1559999999999998e-06 2024-09-16T20:49:49.901 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.901 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.901 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:49.901 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820216+0000", 2024-09-16T20:49:49.901 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046861591000000001 2024-09-16T20:49:49.901 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.901 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.902 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:49.902 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820258+0000", 2024-09-16T20:49:49.902 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2653e-05 2024-09-16T20:49:49.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.902 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.902 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:49.902 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.820321+0000", 2024-09-16T20:49:49.902 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2757999999999998e-05 2024-09-16T20:49:49.902 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:49.902 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:49.902 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:49.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.902 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.902 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5067 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 6918144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:49.903 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.775279+0000", 2024-09-16T20:49:49.903 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 90.063269224999999, 2024-09-16T20:49:49.903 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050171387999999997, 2024-09-16T20:49:49.903 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:49.903 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:49.903 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:49.903 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:49.903 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:49.903 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:49.903 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5067 2024-09-16T20:49:49.903 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.903 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:49.903 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.904 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:49.904 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775279+0000", 2024-09-16T20:49:49.904 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:49.904 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.904 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.904 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:49.904 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775279+0000", 2024-09-16T20:49:49.904 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:49.904 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.904 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.904 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:49.904 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775432+0000", 2024-09-16T20:49:49.904 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015257399999999999 2024-09-16T20:49:49.905 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.905 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.905 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:49.905 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775436+0000", 2024-09-16T20:49:49.905 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0180000000000003e-06 2024-09-16T20:49:49.905 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.905 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.905 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:49.905 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775447+0000", 2024-09-16T20:49:49.905 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.077e-05 2024-09-16T20:49:49.905 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.906 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.906 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:49.906 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775456+0000", 2024-09-16T20:49:49.906 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4089999999999992e-06 2024-09-16T20:49:49.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.906 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.906 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:49.906 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825150+0000", 2024-09-16T20:49:49.906 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049693711000000002 2024-09-16T20:49:49.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.906 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.906 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:49.906 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825337+0000", 2024-09-16T20:49:49.906 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000187214 2024-09-16T20:49:49.907 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.907 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.907 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:49.907 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825451+0000", 2024-09-16T20:49:49.907 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011369199999999999 2024-09-16T20:49:49.907 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:49.907 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:49.907 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:49.907 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.907 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.907 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5068 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2330624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:49.907 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.775469+0000", 2024-09-16T20:49:49.907 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 90.063079263999995, 2024-09-16T20:49:49.908 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050198334999999997, 2024-09-16T20:49:49.908 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:49.908 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:49.908 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:49.908 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:49.908 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:49.908 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:49.908 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5068 2024-09-16T20:49:49.908 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.908 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:49.908 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.908 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:49.908 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775469+0000", 2024-09-16T20:49:49.908 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:49.909 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.909 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.909 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:49.909 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775469+0000", 2024-09-16T20:49:49.909 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:49.909 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.909 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.909 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:49.909 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775720+0000", 2024-09-16T20:49:49.909 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000250661 2024-09-16T20:49:49.909 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.909 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.909 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:49.910 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775724+0000", 2024-09-16T20:49:49.910 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8700000000000002e-06 2024-09-16T20:49:49.910 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.910 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.910 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:49.910 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775733+0000", 2024-09-16T20:49:49.910 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4239999999999999e-06 2024-09-16T20:49:49.910 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.910 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.910 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:49.910 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775740+0000", 2024-09-16T20:49:49.910 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9519999999999997e-06 2024-09-16T20:49:49.910 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.910 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.911 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:49.911 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825484+0000", 2024-09-16T20:49:49.911 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049743385000000001 2024-09-16T20:49:49.911 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.911 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.911 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:49.911 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825572+0000", 2024-09-16T20:49:49.911 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7834e-05 2024-09-16T20:49:49.911 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.911 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.911 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:49.911 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825668+0000", 2024-09-16T20:49:49.911 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6209e-05 2024-09-16T20:49:49.911 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:49.912 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:49.912 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:49.912 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.912 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.912 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5069 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 6926336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:49.912 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.775753+0000", 2024-09-16T20:49:49.912 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 90.062796040999999, 2024-09-16T20:49:49.912 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050146430999999998, 2024-09-16T20:49:49.912 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:49.912 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:49.912 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:49.912 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:49.912 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:49.913 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:49.913 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5069 2024-09-16T20:49:49.913 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.913 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:49.913 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.913 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:49.913 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775753+0000", 2024-09-16T20:49:49.913 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:49.913 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.913 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.913 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:49.913 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775753+0000", 2024-09-16T20:49:49.913 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:49.913 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.914 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.914 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:49.914 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775865+0000", 2024-09-16T20:49:49.914 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000112402 2024-09-16T20:49:49.914 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.914 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.914 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:49.914 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775869+0000", 2024-09-16T20:49:49.914 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5130000000000001e-06 2024-09-16T20:49:49.914 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.914 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.914 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:49.914 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775877+0000", 2024-09-16T20:49:49.914 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1359999999999997e-06 2024-09-16T20:49:49.915 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.915 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.915 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:49.915 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775882+0000", 2024-09-16T20:49:49.915 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6710000000000004e-06 2024-09-16T20:49:49.915 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.915 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.915 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:49.915 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825729+0000", 2024-09-16T20:49:49.915 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049846898000000001 2024-09-16T20:49:49.915 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.915 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.915 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:49.916 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825807+0000", 2024-09-16T20:49:49.916 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7875999999999998e-05 2024-09-16T20:49:49.916 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.916 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.916 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:49.916 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825899+0000", 2024-09-16T20:49:49.916 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1935000000000007e-05 2024-09-16T20:49:49.916 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:49.916 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:49.916 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:49.916 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.916 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.916 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5070 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2338816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:49.917 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.775892+0000", 2024-09-16T20:49:49.917 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 90.062656187000002, 2024-09-16T20:49:49.917 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050155311000000001, 2024-09-16T20:49:49.917 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:49.917 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:49.917 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:49.917 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:49.917 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:49.917 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:49.917 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5070 2024-09-16T20:49:49.917 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.917 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:49.917 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.917 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:49.918 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775892+0000", 2024-09-16T20:49:49.918 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:49.918 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.918 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.918 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:49.918 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.775892+0000", 2024-09-16T20:49:49.918 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:49.918 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.918 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.918 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:49.918 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776142+0000", 2024-09-16T20:49:49.918 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00024950500000000002 2024-09-16T20:49:49.918 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.919 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.919 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:49.919 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776146+0000", 2024-09-16T20:49:49.919 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2819999999999998e-06 2024-09-16T20:49:49.919 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.919 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.919 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:49.919 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776157+0000", 2024-09-16T20:49:49.919 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0807e-05 2024-09-16T20:49:49.919 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.919 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.919 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:49.919 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776165+0000", 2024-09-16T20:49:49.919 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1300000000000001e-06 2024-09-16T20:49:49.920 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.920 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.920 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:49.920 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825935+0000", 2024-09-16T20:49:49.920 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049769300000000002 2024-09-16T20:49:49.920 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.920 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.920 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:49.920 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.825991+0000", 2024-09-16T20:49:49.920 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6177000000000001e-05 2024-09-16T20:49:49.920 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.920 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.920 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:49.920 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826048+0000", 2024-09-16T20:49:49.921 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.711e-05 2024-09-16T20:49:49.921 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:49.921 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:49.921 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:49.921 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.921 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.921 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5071 3.7 3:e00df26b:::rbd_data.110e3f481dff.0000000000000006:7 [read 6934528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:49.921 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.776177+0000", 2024-09-16T20:49:49.921 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 90.062371784000007, 2024-09-16T20:49:49.921 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050036886000000003, 2024-09-16T20:49:49.921 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:49.921 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:49.921 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:49.921 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:49.922 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:49.922 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:49.922 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5071 2024-09-16T20:49:49.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.922 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:49.922 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.922 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:49.922 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776177+0000", 2024-09-16T20:49:49.922 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:49.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.922 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.922 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:49.923 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776177+0000", 2024-09-16T20:49:49.923 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:49.923 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.923 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.923 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:49.923 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776339+0000", 2024-09-16T20:49:49.923 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016238099999999999 2024-09-16T20:49:49.923 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.923 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.923 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:49.923 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776343+0000", 2024-09-16T20:49:49.923 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9670000000000003e-06 2024-09-16T20:49:49.923 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.923 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.923 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:49.924 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776354+0000", 2024-09-16T20:49:49.924 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0789e-05 2024-09-16T20:49:49.924 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.924 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.924 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:49.924 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776361+0000", 2024-09-16T20:49:49.924 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5200000000000003e-06 2024-09-16T20:49:49.924 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.924 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.924 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:49.924 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826078+0000", 2024-09-16T20:49:49.924 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049717513999999997 2024-09-16T20:49:49.924 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.924 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.924 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:49.925 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826141+0000", 2024-09-16T20:49:49.925 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2446000000000005e-05 2024-09-16T20:49:49.925 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.925 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.925 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:49.925 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826214+0000", 2024-09-16T20:49:49.925 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3269000000000006e-05 2024-09-16T20:49:49.925 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:49.925 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:49.925 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:49.925 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.925 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.925 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4520.0:5072 3.7 3:e362722f:::rbd_data.110e3f481dff.0000000000000002:7 [read 2347008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T20:49:49.925 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:19.776370+0000", 2024-09-16T20:49:49.926 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 90.062178223000004, 2024-09-16T20:49:49.926 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050097653999999998, 2024-09-16T20:49:49.926 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:49.926 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:49.926 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:49.926 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:49.926 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4520", 2024-09-16T20:49:49.926 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/569304580", 2024-09-16T20:49:49.926 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5072 2024-09-16T20:49:49.926 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.926 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:49.926 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.926 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:49.926 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776370+0000", 2024-09-16T20:49:49.927 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:49.927 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.927 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.927 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:49.927 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776370+0000", 2024-09-16T20:49:49.927 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:49.927 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.927 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.927 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:49.927 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776645+0000", 2024-09-16T20:49:49.927 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000274634 2024-09-16T20:49:49.927 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.927 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.927 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:49.927 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776654+0000", 2024-09-16T20:49:49.928 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9390000000000003e-06 2024-09-16T20:49:49.928 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.928 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.928 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:49.928 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776665+0000", 2024-09-16T20:49:49.928 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0524e-05 2024-09-16T20:49:49.928 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.928 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.928 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:49.928 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.776671+0000", 2024-09-16T20:49:49.928 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7390000000000002e-06 2024-09-16T20:49:49.928 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.928 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.928 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:49.928 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826261+0000", 2024-09-16T20:49:49.929 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049589995999999997 2024-09-16T20:49:49.929 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.929 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.929 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:49.929 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826363+0000", 2024-09-16T20:49:49.929 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010158999999999999 2024-09-16T20:49:49.929 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:49.929 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:49.929 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:49.929 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:19.826468+0000", 2024-09-16T20:49:49.929 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000105232 2024-09-16T20:49:49.929 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:49.929 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:49.930 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:49.930 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:49.930 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:49.930 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:49.931 INFO:teuthology.orchestra.run.smithi139.stdout:4110 trunc from 0xa25f200 to 0x5a1f400 2024-09-16T20:49:49.931 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:49:49.932 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:49.930+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:50.024 INFO:teuthology.orchestra.run.smithi139.stdout:4111 read 0x1c7a00 thru 0x1cd7b1 (0x5db2 bytes) 2024-09-16T20:49:50.025 INFO:teuthology.orchestra.run.smithi139.stdout:4112 write 0x3424000 thru 0x3427325 (0x3326 bytes) 2024-09-16T20:49:50.025 INFO:teuthology.orchestra.run.smithi139.stdout:4114 write 0x362a00 thru 0x366dac (0x43ad bytes) 2024-09-16T20:49:50.032 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:50.030+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:50.036 INFO:teuthology.orchestra.run.smithi139.stdout:4115 trunc from 0x5a1f400 to 0xea13a00 2024-09-16T20:49:50.063 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:50.063 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:49:50.063 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:49:50.063 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:50.072 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:49:50.117 INFO:teuthology.orchestra.run.smithi139.stdout:4117 punch from 0x20aea00 to 0x20b5800, (0x6e00 bytes) 2024-09-16T20:49:50.128 INFO:teuthology.orchestra.run.smithi139.stdout:4118 write 0x26cac00 thru 0x26d9ec8 (0xf2c9 bytes) 2024-09-16T20:49:50.129 INFO:teuthology.orchestra.run.smithi139.stdout:4119 read 0xb481c00 thru 0xb488142 (0x6543 bytes) 2024-09-16T20:49:50.129 INFO:teuthology.orchestra.run.smithi139.stdout:4121 write 0x10ee000 thru 0x10f1a6a (0x3a6b bytes) 2024-09-16T20:49:50.129 INFO:teuthology.orchestra.run.smithi139.stdout:4123 write 0xde75600 thru 0xde78998 (0x3399 bytes) 2024-09-16T20:49:50.129 INFO:teuthology.orchestra.run.smithi139.stdout:4124 write 0x371a600 thru 0x3722394 (0x7d95 bytes) 2024-09-16T20:49:50.129 INFO:teuthology.orchestra.run.smithi139.stdout:4125 write 0x8e9e400 thru 0x8ea598d (0x758e bytes) 2024-09-16T20:49:50.129 INFO:teuthology.orchestra.run.smithi139.stdout:4128 write 0x6a83800 thru 0x6a849bf (0x11c0 bytes) 2024-09-16T20:49:50.129 INFO:teuthology.orchestra.run.smithi139.stdout:4130 punch from 0x18f7a00 to 0x1905200, (0xd800 bytes) 2024-09-16T20:49:50.132 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:50.131+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:50.139 INFO:teuthology.orchestra.run.smithi139.stdout:4131 flatten 2024-09-16T20:49:50.186 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:50.186 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:49:50.186 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:49:50.186 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:49:50.186 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:50.196 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:49:50.233 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:50.232+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:50.334 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:50.333+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:50.378 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:50.378 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:49:50.378 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:49:50.378 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:49:50.379 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.379 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5286.0:10643 3.17 3.f6273fb7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-16T20:49:50.379 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:49:50.379 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 41.585160512999998, 2024-09-16T20:49:50.379 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.76672347200000002, 2024-09-16T20:49:50.379 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:50.379 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:50.379 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:49:50.379 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:50.379 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5286", 2024-09-16T20:49:50.379 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/4135433989", 2024-09-16T20:49:50.379 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 10643 2024-09-16T20:49:50.379 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.380 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:50.380 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.380 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:50.380 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:49:50.380 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:50.380 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.380 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.380 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:50.380 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:49:50.380 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:50.380 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.380 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.381 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:50.381 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790489+0000", 2024-09-16T20:49:50.381 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5910000000000001e-06 2024-09-16T20:49:50.381 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.381 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.381 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:50.381 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790493+0000", 2024-09-16T20:49:50.381 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2669999999999998e-06 2024-09-16T20:49:50.381 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.381 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.381 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:50.382 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790500+0000", 2024-09-16T20:49:50.382 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3780000000000003e-06 2024-09-16T20:49:50.382 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.382 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.382 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:50.382 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790505+0000", 2024-09-16T20:49:50.382 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1560000000000003e-06 2024-09-16T20:49:50.382 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.382 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.382 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:50.382 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790559+0000", 2024-09-16T20:49:50.382 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3494e-05 2024-09-16T20:49:50.383 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.383 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.383 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:49:50.383 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790566+0000", 2024-09-16T20:49:50.383 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7649999999999993e-06 2024-09-16T20:49:50.383 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.383 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.383 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:50.383 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:09.557198+0000", 2024-09-16T20:49:50.383 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.76663136200000004 2024-09-16T20:49:50.383 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.384 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.384 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:50.384 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:09.557211+0000", 2024-09-16T20:49:50.384 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3458999999999999e-05 2024-09-16T20:49:50.384 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:50.384 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:50.384 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:50.384 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.384 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.384 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3870 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1249280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:50.384 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896514+0000", 2024-09-16T20:49:50.384 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 35.479134551000001, 2024-09-16T20:49:50.385 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046893815999999998, 2024-09-16T20:49:50.385 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:50.385 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:50.385 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:50.385 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:50.385 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:50.385 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:50.385 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3870 2024-09-16T20:49:50.385 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.385 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:50.385 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.385 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:50.386 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896514+0000", 2024-09-16T20:49:50.386 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:50.386 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.386 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.386 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:50.386 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896514+0000", 2024-09-16T20:49:50.386 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:50.386 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.386 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.386 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:50.386 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896514+0000", 2024-09-16T20:49:50.386 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9299999999999999e-07 2024-09-16T20:49:50.386 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.386 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.387 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:50.387 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896521+0000", 2024-09-16T20:49:50.387 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8199999999999999e-06 2024-09-16T20:49:50.387 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.387 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.387 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:50.387 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896529+0000", 2024-09-16T20:49:50.387 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0120000000000005e-06 2024-09-16T20:49:50.387 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.387 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.387 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:50.388 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896534+0000", 2024-09-16T20:49:50.388 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5779999999999998e-06 2024-09-16T20:49:50.388 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.388 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.388 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:50.388 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943286+0000", 2024-09-16T20:49:50.388 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046751514000000001 2024-09-16T20:49:50.388 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.388 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.388 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:50.388 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943343+0000", 2024-09-16T20:49:50.389 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7216000000000002e-05 2024-09-16T20:49:50.389 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.389 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.389 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:50.389 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943407+0000", 2024-09-16T20:49:50.389 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4282999999999994e-05 2024-09-16T20:49:50.389 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:50.389 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:50.389 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:50.389 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.389 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.390 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3872 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1253376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:50.390 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896561+0000", 2024-09-16T20:49:50.390 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 35.479087157000002, 2024-09-16T20:49:50.390 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.04705165, 2024-09-16T20:49:50.390 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:50.390 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:50.390 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:50.390 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:50.390 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:50.390 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:50.390 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3872 2024-09-16T20:49:50.390 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.390 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:50.391 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.391 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:50.391 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896561+0000", 2024-09-16T20:49:50.391 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:50.391 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.391 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.391 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:50.391 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896561+0000", 2024-09-16T20:49:50.391 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:50.391 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.391 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.391 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:50.392 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896561+0000", 2024-09-16T20:49:50.392 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4999999999999998e-07 2024-09-16T20:49:50.392 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.392 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.392 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:50.392 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896563+0000", 2024-09-16T20:49:50.392 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0140000000000001e-06 2024-09-16T20:49:50.392 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.392 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.392 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:50.392 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896569+0000", 2024-09-16T20:49:50.392 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6450000000000002e-06 2024-09-16T20:49:50.393 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.393 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.393 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:50.393 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896572+0000", 2024-09-16T20:49:50.393 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.174e-06 2024-09-16T20:49:50.393 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.393 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.393 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:50.393 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943456+0000", 2024-09-16T20:49:50.393 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046883552000000002 2024-09-16T20:49:50.393 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.393 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.394 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:50.394 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943531+0000", 2024-09-16T20:49:50.394 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5080000000000006e-05 2024-09-16T20:49:50.394 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.394 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.394 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:50.394 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943613+0000", 2024-09-16T20:49:50.394 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1835000000000005e-05 2024-09-16T20:49:50.394 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:50.394 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:50.394 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:50.394 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.394 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.395 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3874 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1257472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:50.395 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896577+0000", 2024-09-16T20:49:50.395 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 35.479071642000001, 2024-09-16T20:49:50.395 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047187944000000002, 2024-09-16T20:49:50.395 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:50.395 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:50.395 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:50.395 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:50.395 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:50.395 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:50.395 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3874 2024-09-16T20:49:50.395 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.396 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:50.396 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.396 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:50.396 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896577+0000", 2024-09-16T20:49:50.396 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:50.396 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.396 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.396 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:50.396 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896577+0000", 2024-09-16T20:49:50.396 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:50.396 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.396 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.397 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:50.397 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896577+0000", 2024-09-16T20:49:50.397 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1199999999999999e-07 2024-09-16T20:49:50.397 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.397 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.397 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:50.397 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896578+0000", 2024-09-16T20:49:50.397 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.237e-06 2024-09-16T20:49:50.397 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.397 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.397 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:50.398 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896583+0000", 2024-09-16T20:49:50.398 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5859999999999998e-06 2024-09-16T20:49:50.398 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.398 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.398 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:50.398 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896587+0000", 2024-09-16T20:49:50.398 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3100000000000002e-06 2024-09-16T20:49:50.398 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.398 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.398 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:50.398 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943661+0000", 2024-09-16T20:49:50.398 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047073639 2024-09-16T20:49:50.398 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.399 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.399 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:50.399 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943712+0000", 2024-09-16T20:49:50.399 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1190999999999998e-05 2024-09-16T20:49:50.399 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.399 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.399 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:50.399 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943765+0000", 2024-09-16T20:49:50.399 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2769000000000002e-05 2024-09-16T20:49:50.399 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:50.399 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:50.399 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:50.399 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.400 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.400 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3876 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1261568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:50.400 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896593+0000", 2024-09-16T20:49:50.400 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 35.479055064000001, 2024-09-16T20:49:50.400 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047362866000000003, 2024-09-16T20:49:50.400 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:50.400 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:50.400 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:50.400 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:50.400 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:50.400 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:50.400 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3876 2024-09-16T20:49:50.401 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.401 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:50.401 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.401 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:50.401 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896593+0000", 2024-09-16T20:49:50.401 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:50.401 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.401 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.401 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:50.401 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896593+0000", 2024-09-16T20:49:50.401 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:50.401 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.401 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.401 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:50.402 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896594+0000", 2024-09-16T20:49:50.402 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1600000000000002e-07 2024-09-16T20:49:50.402 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.402 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.402 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:50.402 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896595+0000", 2024-09-16T20:49:50.402 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.756e-06 2024-09-16T20:49:50.402 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.402 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.402 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:50.402 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896601+0000", 2024-09-16T20:49:50.402 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9309999999999996e-06 2024-09-16T20:49:50.403 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.403 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.403 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:50.403 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896606+0000", 2024-09-16T20:49:50.403 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1089999999999997e-06 2024-09-16T20:49:50.403 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.403 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.403 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:50.403 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943790+0000", 2024-09-16T20:49:50.403 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047183414 2024-09-16T20:49:50.403 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.403 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.403 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:50.404 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943869+0000", 2024-09-16T20:49:50.404 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9122000000000006e-05 2024-09-16T20:49:50.404 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.404 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.404 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:50.404 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943956+0000", 2024-09-16T20:49:50.404 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7118000000000002e-05 2024-09-16T20:49:50.404 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:50.404 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:50.404 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:50.404 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.404 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.405 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3879 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1265664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:50.405 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896613+0000", 2024-09-16T20:49:50.405 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 35.479035422000003, 2024-09-16T20:49:50.405 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047528204999999997, 2024-09-16T20:49:50.405 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:50.405 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:50.405 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:50.405 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:50.405 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:50.406 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:50.406 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3879 2024-09-16T20:49:50.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.406 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:50.406 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.406 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:50.406 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896613+0000", 2024-09-16T20:49:50.406 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:50.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.406 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.406 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:50.407 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896613+0000", 2024-09-16T20:49:50.407 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:50.407 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.407 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.407 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:50.407 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896613+0000", 2024-09-16T20:49:50.407 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.84e-07 2024-09-16T20:49:50.407 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.407 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.407 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:50.407 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896615+0000", 2024-09-16T20:49:50.407 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9970000000000001e-06 2024-09-16T20:49:50.408 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.408 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.408 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:50.408 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896622+0000", 2024-09-16T20:49:50.408 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3500000000000002e-06 2024-09-16T20:49:50.408 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.408 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.408 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:50.408 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896627+0000", 2024-09-16T20:49:50.408 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2490000000000001e-06 2024-09-16T20:49:50.408 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.408 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.409 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:50.409 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.943996+0000", 2024-09-16T20:49:50.409 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047369175999999999 2024-09-16T20:49:50.409 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.409 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.409 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:50.409 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944082+0000", 2024-09-16T20:49:50.409 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5859000000000006e-05 2024-09-16T20:49:50.409 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.409 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.409 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:50.409 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944141+0000", 2024-09-16T20:49:50.410 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9190000000000001e-05 2024-09-16T20:49:50.410 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:50.410 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:50.410 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:50.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.410 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.410 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3881 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1269760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:50.410 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896634+0000", 2024-09-16T20:49:50.410 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 35.479014509000002, 2024-09-16T20:49:50.410 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047692834000000003, 2024-09-16T20:49:50.410 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:50.410 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:50.410 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:50.411 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:50.411 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:50.411 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:50.411 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3881 2024-09-16T20:49:50.411 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.411 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:50.411 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.411 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:50.411 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896634+0000", 2024-09-16T20:49:50.411 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:50.411 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.412 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.412 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:50.412 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896634+0000", 2024-09-16T20:49:50.412 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:50.412 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.412 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.412 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:50.412 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896634+0000", 2024-09-16T20:49:50.412 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-16T20:49:50.412 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.412 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.412 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:50.413 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896636+0000", 2024-09-16T20:49:50.413 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.216e-06 2024-09-16T20:49:50.413 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.413 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.413 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:50.413 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896640+0000", 2024-09-16T20:49:50.413 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1749999999999998e-06 2024-09-16T20:49:50.413 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.413 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.413 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:50.413 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896643+0000", 2024-09-16T20:49:50.413 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8779999999999998e-06 2024-09-16T20:49:50.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.414 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.414 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:50.414 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944178+0000", 2024-09-16T20:49:50.414 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047534633 2024-09-16T20:49:50.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.414 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.414 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:50.414 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944231+0000", 2024-09-16T20:49:50.414 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3338999999999999e-05 2024-09-16T20:49:50.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.414 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.414 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:50.415 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944327+0000", 2024-09-16T20:49:50.415 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5223000000000006e-05 2024-09-16T20:49:50.415 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:50.415 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:50.415 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:50.415 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.415 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.415 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3882 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1273856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:50.415 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896647+0000", 2024-09-16T20:49:50.415 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 35.479001347999997, 2024-09-16T20:49:50.415 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047858886000000003, 2024-09-16T20:49:50.415 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:50.416 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:50.416 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:50.416 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:50.416 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:50.416 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:50.416 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3882 2024-09-16T20:49:50.416 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.416 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:50.416 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.416 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:50.416 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896647+0000", 2024-09-16T20:49:50.416 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:50.416 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.417 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.417 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:50.417 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896647+0000", 2024-09-16T20:49:50.417 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:50.417 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.417 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.417 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:50.417 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896647+0000", 2024-09-16T20:49:50.417 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4400000000000001e-07 2024-09-16T20:49:50.417 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.417 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.417 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:50.418 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896648+0000", 2024-09-16T20:49:50.418 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0499999999999999e-06 2024-09-16T20:49:50.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.418 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.418 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:50.418 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896651+0000", 2024-09-16T20:49:50.418 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9809999999999999e-06 2024-09-16T20:49:50.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.418 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.418 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:50.418 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896654+0000", 2024-09-16T20:49:50.418 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4439999999999998e-06 2024-09-16T20:49:50.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.419 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.419 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:50.419 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944361+0000", 2024-09-16T20:49:50.419 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047707408 2024-09-16T20:49:50.419 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.419 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.419 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:50.419 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944405+0000", 2024-09-16T20:49:50.419 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4107000000000003e-05 2024-09-16T20:49:50.419 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.419 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.419 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:50.419 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944506+0000", 2024-09-16T20:49:50.420 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000100652 2024-09-16T20:49:50.420 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:50.420 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:50.420 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:50.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.420 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.420 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3883 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1277952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:50.420 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896656+0000", 2024-09-16T20:49:50.420 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 35.478991837999999, 2024-09-16T20:49:50.420 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047975265000000003, 2024-09-16T20:49:50.420 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:50.420 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:50.421 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:50.421 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:50.421 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:50.421 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:50.421 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3883 2024-09-16T20:49:50.421 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.421 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:50.421 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.421 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:50.421 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896656+0000", 2024-09-16T20:49:50.421 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:50.421 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.421 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.422 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:50.422 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896656+0000", 2024-09-16T20:49:50.422 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:50.422 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.422 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.422 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:50.422 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896657+0000", 2024-09-16T20:49:50.422 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7000000000000001e-07 2024-09-16T20:49:50.422 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.422 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.422 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:50.422 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896658+0000", 2024-09-16T20:49:50.422 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5850000000000001e-06 2024-09-16T20:49:50.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.423 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.423 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:50.423 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896664+0000", 2024-09-16T20:49:50.423 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.609e-06 2024-09-16T20:49:50.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.423 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.423 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:50.423 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896668+0000", 2024-09-16T20:49:50.423 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.121e-06 2024-09-16T20:49:50.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.423 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.423 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:50.423 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944542+0000", 2024-09-16T20:49:50.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047873838000000002 2024-09-16T20:49:50.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.424 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:50.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944579+0000", 2024-09-16T20:49:50.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6943000000000002e-05 2024-09-16T20:49:50.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.424 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:50.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944632+0000", 2024-09-16T20:49:50.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2899000000000003e-05 2024-09-16T20:49:50.424 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:50.425 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:50.425 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:50.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.425 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3886 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1282048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:50.425 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896674+0000", 2024-09-16T20:49:50.425 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 35.478973869000001, 2024-09-16T20:49:50.425 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048152421000000001, 2024-09-16T20:49:50.425 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:50.425 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:50.425 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:50.425 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:50.425 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:50.425 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:50.426 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3886 2024-09-16T20:49:50.426 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.426 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:50.426 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.426 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:50.426 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896674+0000", 2024-09-16T20:49:50.426 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:50.426 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.426 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.426 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:50.426 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896674+0000", 2024-09-16T20:49:50.426 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:50.426 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.427 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.427 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:50.427 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896675+0000", 2024-09-16T20:49:50.427 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6100000000000002e-07 2024-09-16T20:49:50.427 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.427 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.427 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:50.427 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896677+0000", 2024-09-16T20:49:50.427 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1270000000000001e-06 2024-09-16T20:49:50.427 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.427 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.427 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:50.427 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896683+0000", 2024-09-16T20:49:50.428 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1970000000000001e-06 2024-09-16T20:49:50.428 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.428 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.428 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:50.428 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896688+0000", 2024-09-16T20:49:50.428 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.605e-06 2024-09-16T20:49:50.428 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.428 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.428 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:50.428 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944676+0000", 2024-09-16T20:49:50.428 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047988391999999998 2024-09-16T20:49:50.428 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.429 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.429 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:50.429 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944763+0000", 2024-09-16T20:49:50.429 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7363999999999999e-05 2024-09-16T20:49:50.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.429 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.429 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:50.429 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944827+0000", 2024-09-16T20:49:50.429 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3374999999999995e-05 2024-09-16T20:49:50.429 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:50.430 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:50.430 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:50.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.430 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.430 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3888 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1286144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:50.430 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896693+0000", 2024-09-16T20:49:50.430 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 35.478954745000003, 2024-09-16T20:49:50.430 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048336443999999999, 2024-09-16T20:49:50.430 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:50.430 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:50.430 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:50.431 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:50.431 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:50.431 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:50.431 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3888 2024-09-16T20:49:50.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.431 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:50.431 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.431 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:50.431 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896693+0000", 2024-09-16T20:49:50.431 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:50.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.432 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.432 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:50.432 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896693+0000", 2024-09-16T20:49:50.432 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:50.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.432 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.432 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:50.432 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896694+0000", 2024-09-16T20:49:50.432 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3200000000000001e-07 2024-09-16T20:49:50.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.433 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.433 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:50.433 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896696+0000", 2024-09-16T20:49:50.433 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8989999999999999e-06 2024-09-16T20:49:50.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.433 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.433 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:50.433 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896702+0000", 2024-09-16T20:49:50.433 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8819999999999998e-06 2024-09-16T20:49:50.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.433 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.433 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:50.434 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896707+0000", 2024-09-16T20:49:50.434 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1529999999999996e-06 2024-09-16T20:49:50.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.434 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.434 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:50.434 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944863+0000", 2024-09-16T20:49:50.434 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048156658999999997 2024-09-16T20:49:50.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.434 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.434 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:50.434 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.944944+0000", 2024-09-16T20:49:50.435 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0090000000000001e-05 2024-09-16T20:49:50.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.435 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.435 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:50.435 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945030+0000", 2024-09-16T20:49:50.435 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6428999999999995e-05 2024-09-16T20:49:50.435 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:50.435 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:50.435 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:50.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.435 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.435 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3890 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1290240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:50.436 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896713+0000", 2024-09-16T20:49:50.436 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 35.478934875, 2024-09-16T20:49:50.436 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048538382999999997, 2024-09-16T20:49:50.436 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:50.436 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:50.436 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:50.436 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:50.436 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:50.436 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:50.436 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3890 2024-09-16T20:49:50.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.436 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:50.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.437 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:50.437 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896713+0000", 2024-09-16T20:49:50.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:50.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.437 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:50.437 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896713+0000", 2024-09-16T20:49:50.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:50.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.437 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:50.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896714+0000", 2024-09-16T20:49:50.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2000000000000001e-07 2024-09-16T20:49:50.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:50.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896716+0000", 2024-09-16T20:49:50.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0109999999999999e-06 2024-09-16T20:49:50.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:50.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896720+0000", 2024-09-16T20:49:50.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9049999999999999e-06 2024-09-16T20:49:50.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.439 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.439 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:50.439 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896722+0000", 2024-09-16T20:49:50.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.627e-06 2024-09-16T20:49:50.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.439 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.439 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:50.439 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945075+0000", 2024-09-16T20:49:50.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048353149999999998 2024-09-16T20:49:50.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.439 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.440 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:50.440 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945157+0000", 2024-09-16T20:49:50.440 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1972999999999996e-05 2024-09-16T20:49:50.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.440 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:50.440 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945252+0000", 2024-09-16T20:49:50.440 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4396999999999998e-05 2024-09-16T20:49:50.440 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:50.440 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:50.440 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:50.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.441 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3893 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1294336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:50.441 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896726+0000", 2024-09-16T20:49:50.441 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 35.478922017999999, 2024-09-16T20:49:50.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048658686999999999, 2024-09-16T20:49:50.441 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:50.441 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:50.441 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:50.441 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:50.441 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:50.441 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:50.441 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3893 2024-09-16T20:49:50.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.442 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:50.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:50.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896726+0000", 2024-09-16T20:49:50.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:50.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:50.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896726+0000", 2024-09-16T20:49:50.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:50.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:50.443 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896727+0000", 2024-09-16T20:49:50.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7899999999999999e-07 2024-09-16T20:49:50.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.443 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:50.443 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896728+0000", 2024-09-16T20:49:50.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.68e-06 2024-09-16T20:49:50.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.443 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:50.443 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896734+0000", 2024-09-16T20:49:50.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9190000000000004e-06 2024-09-16T20:49:50.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:50.444 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896739+0000", 2024-09-16T20:49:50.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5549999999999996e-06 2024-09-16T20:49:50.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:50.444 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945271+0000", 2024-09-16T20:49:50.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048531795000000003 2024-09-16T20:49:50.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:50.444 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945330+0000", 2024-09-16T20:49:50.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9386999999999999e-05 2024-09-16T20:49:50.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:50.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945385+0000", 2024-09-16T20:49:50.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4871999999999999e-05 2024-09-16T20:49:50.445 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:50.445 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:50.445 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:50.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.445 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3895 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1298432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:50.445 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896754+0000", 2024-09-16T20:49:50.446 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 35.478893943999999, 2024-09-16T20:49:50.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048754440000000003, 2024-09-16T20:49:50.446 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:50.446 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:50.446 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:50.446 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:50.446 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:50.446 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:50.446 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3895 2024-09-16T20:49:50.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.446 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:50.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.446 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:50.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896754+0000", 2024-09-16T20:49:50.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:50.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:50.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896754+0000", 2024-09-16T20:49:50.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:50.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:50.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896755+0000", 2024-09-16T20:49:50.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5200000000000002e-07 2024-09-16T20:49:50.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:50.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896762+0000", 2024-09-16T20:49:50.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8430000000000004e-06 2024-09-16T20:49:50.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:50.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896782+0000", 2024-09-16T20:49:50.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0693e-05 2024-09-16T20:49:50.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.449 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:50.449 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896793+0000", 2024-09-16T20:49:50.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0294e-05 2024-09-16T20:49:50.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.449 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:50.449 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945418+0000", 2024-09-16T20:49:50.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048625738000000002 2024-09-16T20:49:50.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.450 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:50.450 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945457+0000", 2024-09-16T20:49:50.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9103999999999997e-05 2024-09-16T20:49:50.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.450 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:50.450 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945509+0000", 2024-09-16T20:49:50.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1316000000000001e-05 2024-09-16T20:49:50.450 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:50.450 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:50.450 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:50.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.451 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3896 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1302528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:50.451 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896806+0000", 2024-09-16T20:49:50.451 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 35.478841731999999, 2024-09-16T20:49:50.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048831159999999998, 2024-09-16T20:49:50.451 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:50.451 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:50.451 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:50.451 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:50.451 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:50.451 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:50.451 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3896 2024-09-16T20:49:50.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.451 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:50.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:50.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896806+0000", 2024-09-16T20:49:50.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:50.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:50.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896806+0000", 2024-09-16T20:49:50.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:50.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:50.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896807+0000", 2024-09-16T20:49:50.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3e-07 2024-09-16T20:49:50.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:50.453 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896811+0000", 2024-09-16T20:49:50.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8310000000000003e-06 2024-09-16T20:49:50.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:50.453 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896819+0000", 2024-09-16T20:49:50.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2470000000000008e-06 2024-09-16T20:49:50.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:50.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896823+0000", 2024-09-16T20:49:50.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7919999999999999e-06 2024-09-16T20:49:50.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:50.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945535+0000", 2024-09-16T20:49:50.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048712137000000003 2024-09-16T20:49:50.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:50.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945571+0000", 2024-09-16T20:49:50.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6261999999999999e-05 2024-09-16T20:49:50.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:50.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945638+0000", 2024-09-16T20:49:50.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6161000000000006e-05 2024-09-16T20:49:50.455 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:50.455 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:50.455 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:50.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.455 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5352.0:3899 3.0 3:01da6156:::rbd_data.110e84dd14d5.0000000000000001:18 [read 1306624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-16T20:49:50.455 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:14.896829+0000", 2024-09-16T20:49:50.455 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 35.478819530999999, 2024-09-16T20:49:50.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049014110999999999, 2024-09-16T20:49:50.456 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:50.456 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:50.456 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:50.456 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:50.456 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5352", 2024-09-16T20:49:50.456 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/550119856", 2024-09-16T20:49:50.456 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3899 2024-09-16T20:49:50.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.456 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:50.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.457 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:50.457 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896829+0000", 2024-09-16T20:49:50.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:50.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.457 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:50.457 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896829+0000", 2024-09-16T20:49:50.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:50.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.457 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:50.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896829+0000", 2024-09-16T20:49:50.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8999999999999998e-07 2024-09-16T20:49:50.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:50.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896831+0000", 2024-09-16T20:49:50.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.77e-06 2024-09-16T20:49:50.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:50.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896834+0000", 2024-09-16T20:49:50.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4800000000000001e-06 2024-09-16T20:49:50.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:50.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.896837+0000", 2024-09-16T20:49:50.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8559999999999998e-06 2024-09-16T20:49:50.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:50.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945663+0000", 2024-09-16T20:49:50.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048825529999999999 2024-09-16T20:49:50.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:50.460 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945746+0000", 2024-09-16T20:49:50.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3621000000000005e-05 2024-09-16T20:49:50.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:50.460 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:14.945843+0000", 2024-09-16T20:49:50.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6564000000000006e-05 2024-09-16T20:49:50.460 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:50.460 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:50.460 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:50.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.461 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5512.0:450 3.2 3:44e0cffa:::rbd_header.110ef0d7a535:head [watch reconnect cookie 140263298018912 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:49:50.461 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:49:50.461 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 13.813137599999999, 2024-09-16T20:49:50.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0084581939999999, 2024-09-16T20:49:50.461 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:50.461 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:50.461 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:50.461 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:50.461 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5512", 2024-09-16T20:49:50.461 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:49:50.461 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 450 2024-09-16T20:49:50.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.462 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:50.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.462 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:50.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:49:50.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:50.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.462 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:50.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:49:50.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:50.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.463 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:50.463 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562512+0000", 2024-09-16T20:49:50.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.009e-06 2024-09-16T20:49:50.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.463 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:50.463 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562517+0000", 2024-09-16T20:49:50.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9339999999999997e-06 2024-09-16T20:49:50.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.463 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:50.463 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562530+0000", 2024-09-16T20:49:50.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3495e-05 2024-09-16T20:49:50.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.464 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:50.464 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562536+0000", 2024-09-16T20:49:50.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5690000000000004e-06 2024-09-16T20:49:50.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.464 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:50.464 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562591+0000", 2024-09-16T20:49:50.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5761999999999999e-05 2024-09-16T20:49:50.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.464 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:49:50.465 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562599+0000", 2024-09-16T20:49:50.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9179999999999994e-06 2024-09-16T20:49:50.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.465 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:50.465 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.565971+0000", 2024-09-16T20:49:50.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003371546 2024-09-16T20:49:50.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.465 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:49:50.465 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566019+0000", 2024-09-16T20:49:50.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7849000000000002e-05 2024-09-16T20:49:50.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:50.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.569382+0000", 2024-09-16T20:49:50.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.003362833 2024-09-16T20:49:50.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:50.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.569529+0000", 2024-09-16T20:49:50.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000147823 2024-09-16T20:49:50.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:50.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570969+0000", 2024-09-16T20:49:50.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001439456 2024-09-16T20:49:50.467 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:50.467 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:50.467 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:50.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.467 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:14996 3.2 3:44e0cffa:::rbd_header.110ef0d7a535:head [watch reconnect cookie 140213436269840 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:49:50.467 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:49:50.467 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 13.813089187999999, 2024-09-16T20:49:50.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.008813943, 2024-09-16T20:49:50.467 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:50.467 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:50.468 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:50.468 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:50.468 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:50.468 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:50.468 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 14996 2024-09-16T20:49:50.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.468 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:50.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:50.468 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:49:50.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:50.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.469 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:50.469 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:49:50.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:50.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.469 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:50.469 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562560+0000", 2024-09-16T20:49:50.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.06e-07 2024-09-16T20:49:50.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.469 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:50.469 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562563+0000", 2024-09-16T20:49:50.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.495e-06 2024-09-16T20:49:50.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:50.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562571+0000", 2024-09-16T20:49:50.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1950000000000003e-06 2024-09-16T20:49:50.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:50.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562579+0000", 2024-09-16T20:49:50.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2649999999999999e-06 2024-09-16T20:49:50.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:50.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562673+0000", 2024-09-16T20:49:50.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4118999999999998e-05 2024-09-16T20:49:50.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:49:50.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562677+0000", 2024-09-16T20:49:50.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5179999999999998e-06 2024-09-16T20:49:50.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:50.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566461+0000", 2024-09-16T20:49:50.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0037837259999999998 2024-09-16T20:49:50.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:49:50.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566468+0000", 2024-09-16T20:49:50.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3350000000000001e-06 2024-09-16T20:49:50.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:50.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570306+0000", 2024-09-16T20:49:50.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.003837997 2024-09-16T20:49:50.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:50.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570392+0000", 2024-09-16T20:49:50.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5872999999999996e-05 2024-09-16T20:49:50.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:50.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571373+0000", 2024-09-16T20:49:50.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00098071399999999993 2024-09-16T20:49:50.473 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:50.473 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:50.473 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:50.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.474 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5255.0:1931 3.2 3:47882819:::rbd_data.110eb5f95afc.0000000000000009:head [read 217088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:49:50.474 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:49:50.474 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 13.813050437999999, 2024-09-16T20:49:50.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0082752880000001, 2024-09-16T20:49:50.474 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:50.474 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:50.474 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:49:50.474 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:50.474 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5255", 2024-09-16T20:49:50.474 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:49:50.474 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1931 2024-09-16T20:49:50.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.474 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:50.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:50.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:49:50.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:50.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:50.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:49:50.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:50.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:50.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:49:50.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9699999999999996e-07 2024-09-16T20:49:50.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.476 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:50.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562601+0000", 2024-09-16T20:49:50.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8600000000000001e-06 2024-09-16T20:49:50.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.476 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:50.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562608+0000", 2024-09-16T20:49:50.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9e-06 2024-09-16T20:49:50.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:50.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562615+0000", 2024-09-16T20:49:50.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8179999999999998e-06 2024-09-16T20:49:50.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:50.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562703+0000", 2024-09-16T20:49:50.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7811000000000003e-05 2024-09-16T20:49:50.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:49:50.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562706+0000", 2024-09-16T20:49:50.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4920000000000002e-06 2024-09-16T20:49:50.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:50.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566685+0000", 2024-09-16T20:49:50.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0039792780000000002 2024-09-16T20:49:50.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:49:50.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566693+0000", 2024-09-16T20:49:50.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4730000000000003e-06 2024-09-16T20:49:50.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:50.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570737+0000", 2024-09-16T20:49:50.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0040436749999999 2024-09-16T20:49:50.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:50.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570873+0000", 2024-09-16T20:49:50.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000136484 2024-09-16T20:49:50.480 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:50.480 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:50.480 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:50.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.480 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5255.0:1939 3.2 3:47882819:::rbd_data.110eb5f95afc.0000000000000009:head [read 233472~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:49:50.480 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:49:50.480 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 13.812895572, 2024-09-16T20:49:50.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0084657589999999, 2024-09-16T20:49:50.481 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:50.481 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:50.481 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:49:50.481 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:50.481 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5255", 2024-09-16T20:49:50.481 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:49:50.481 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1939 2024-09-16T20:49:50.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.481 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:50.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.481 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:50.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:49:50.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:50.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.482 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:50.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:49:50.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:50.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.482 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:50.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:49:50.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5199999999999997e-07 2024-09-16T20:49:50.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:50.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562757+0000", 2024-09-16T20:49:50.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4979999999999998e-06 2024-09-16T20:49:50.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:50.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562764+0000", 2024-09-16T20:49:50.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1470000000000004e-06 2024-09-16T20:49:50.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:50.484 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562768+0000", 2024-09-16T20:49:50.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7929999999999999e-06 2024-09-16T20:49:50.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:50.484 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562805+0000", 2024-09-16T20:49:50.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7477e-05 2024-09-16T20:49:50.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting_for_map not empty", 2024-09-16T20:49:50.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562810+0000", 2024-09-16T20:49:50.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0819999999999998e-06 2024-09-16T20:49:50.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.485 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:50.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566717+0000", 2024-09-16T20:49:50.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0039072600000000001 2024-09-16T20:49:50.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.485 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:49:50.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566724+0000", 2024-09-16T20:49:50.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4880000000000004e-06 2024-09-16T20:49:50.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.486 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:50.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571009+0000", 2024-09-16T20:49:50.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.004284677 2024-09-16T20:49:50.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:50.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:50.486 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:50.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571218+0000", 2024-09-16T20:49:50.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00020978500000000001 2024-09-16T20:49:50.486 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:50.486 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:50.487 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:50.487 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:50.487 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:50.487 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:50.488 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:50.486+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:50.489 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:49:50.588 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:50.587+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:50.632 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:50.633 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:49:50.633 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:49:50.633 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:50.641 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:49:50.726 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:50.725+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:50.790 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:50.789+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:50.817 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:50.817 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:49:50.817 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:49:50.817 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:49:50.817 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:50.827 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:49:50.891 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:50.890+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:50.993 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:50.991+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:51.018 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:51.018 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:49:51.019 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:49:51.019 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:49:51.019 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.019 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2206 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1046528~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:51.019 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:49:51.019 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 88.680297928000002, 2024-09-16T20:49:51.019 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050147714000000003, 2024-09-16T20:49:51.019 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.019 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.019 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:51.019 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.019 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:51.020 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:51.020 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2206 2024-09-16T20:49:51.020 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.020 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.020 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.020 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.020 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:49:51.020 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.020 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.020 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.020 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.020 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:49:51.020 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.021 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.021 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.021 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.021 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:49:51.021 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8599999999999999e-07 2024-09-16T20:49:51.021 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.021 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.021 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.021 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336024+0000", 2024-09-16T20:49:51.021 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.934e-06 2024-09-16T20:49:51.021 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.021 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.021 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.021 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336037+0000", 2024-09-16T20:49:51.022 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2496e-05 2024-09-16T20:49:51.022 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.022 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.022 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.022 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336042+0000", 2024-09-16T20:49:51.022 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4809999999999997e-06 2024-09-16T20:49:51.022 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.022 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.022 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:51.022 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385980+0000", 2024-09-16T20:49:51.022 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049938376999999999 2024-09-16T20:49:51.022 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.022 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.022 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:51.022 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.386069+0000", 2024-09-16T20:49:51.023 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8368999999999994e-05 2024-09-16T20:49:51.023 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.023 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.023 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:51.023 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.386170+0000", 2024-09-16T20:49:51.023 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000100671 2024-09-16T20:49:51.023 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.023 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.023 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.023 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.023 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.023 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5439 3.a 3:5789f5c3:::rbd_data.110ed9ca540.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 1181696~60416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:51.023 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:49:51.023 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 88.151609347999994, 2024-09-16T20:49:51.024 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10102045599999999, 2024-09-16T20:49:51.024 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.024 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.024 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:51.024 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.024 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:51.024 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:51.024 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5439 2024-09-16T20:49:51.024 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.024 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.024 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.024 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.024 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:49:51.024 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.024 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.025 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.025 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.025 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:49:51.025 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.025 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.025 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.025 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.025 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864711+0000", 2024-09-16T20:49:51.025 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7899999999999997e-07 2024-09-16T20:49:51.025 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.025 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.025 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.025 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864760+0000", 2024-09-16T20:49:51.025 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8803e-05 2024-09-16T20:49:51.025 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.025 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.026 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.026 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864769+0000", 2024-09-16T20:49:51.026 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0999999999999993e-06 2024-09-16T20:49:51.026 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.026 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.026 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.026 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864774+0000", 2024-09-16T20:49:51.026 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8919999999999999e-06 2024-09-16T20:49:51.026 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.026 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.026 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:51.026 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864815+0000", 2024-09-16T20:49:51.026 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1307000000000003e-05 2024-09-16T20:49:51.026 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.027 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.027 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:51.027 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864893+0000", 2024-09-16T20:49:51.027 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7581000000000001e-05 2024-09-16T20:49:51.027 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.027 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.027 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 1", 2024-09-16T20:49:51.027 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864996+0000", 2024-09-16T20:49:51.027 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102725 2024-09-16T20:49:51.027 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.027 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.027 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:51.027 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965442+0000", 2024-09-16T20:49:51.027 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100446012 2024-09-16T20:49:51.027 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.027 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.028 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:51.028 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965618+0000", 2024-09-16T20:49:51.028 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00017653100000000001 2024-09-16T20:49:51.028 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.028 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.028 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:51.028 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965682+0000", 2024-09-16T20:49:51.028 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3905999999999994e-05 2024-09-16T20:49:51.028 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.028 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.028 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:51.028 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965731+0000", 2024-09-16T20:49:51.028 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8819999999999997e-05 2024-09-16T20:49:51.028 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.029 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.029 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.029 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.029 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.029 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4600.0:5439 3.a e37/23 ondisk, result = 0)", 2024-09-16T20:49:51.029 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:49:51.029 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 88.150613199999995, 2024-09-16T20:49:51.029 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099742634999999996, 2024-09-16T20:49:51.029 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.029 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.029 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:51.029 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.029 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.029 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.029 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:49:51.030 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.030 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.030 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.030 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.030 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:49:51.030 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.030 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.030 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.030 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.030 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:49:51.030 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0900000000000001e-07 2024-09-16T20:49:51.030 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.030 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.030 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.030 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865711+0000", 2024-09-16T20:49:51.031 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3739999999999998e-06 2024-09-16T20:49:51.031 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.031 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.031 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.031 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965343+0000", 2024-09-16T20:49:51.031 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099632526999999999 2024-09-16T20:49:51.031 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.031 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.031 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.031 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965357+0000", 2024-09-16T20:49:51.031 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3713999999999999e-05 2024-09-16T20:49:51.031 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.031 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.031 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:51.031 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965426+0000", 2024-09-16T20:49:51.031 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8745999999999995e-05 2024-09-16T20:49:51.032 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.032 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.032 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:51.032 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965434+0000", 2024-09-16T20:49:51.032 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6910000000000003e-06 2024-09-16T20:49:51.032 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.032 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.032 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:51.032 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965449+0000", 2024-09-16T20:49:51.032 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4874e-05 2024-09-16T20:49:51.032 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.032 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.032 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.032 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.032 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.033 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4882.0:2508 3.12 3:4992771f:::rbd_object_map.110e7c8851c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T20:49:51.033 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:49:51.033 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 64.534809652000007, 2024-09-16T20:49:51.033 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.125007011, 2024-09-16T20:49:51.033 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.033 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.033 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:51.033 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.033 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4882", 2024-09-16T20:49:51.033 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2436923223", 2024-09-16T20:49:51.033 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2508 2024-09-16T20:49:51.033 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.033 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.033 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.033 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.034 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:49:51.034 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.034 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.034 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.034 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.034 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:49:51.034 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.034 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.034 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.034 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.034 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481511+0000", 2024-09-16T20:49:51.034 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0699999999999996e-07 2024-09-16T20:49:51.035 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.035 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.035 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.035 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481515+0000", 2024-09-16T20:49:51.035 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0609999999999997e-06 2024-09-16T20:49:51.035 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.035 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.035 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.035 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481523+0000", 2024-09-16T20:49:51.035 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1310000000000006e-06 2024-09-16T20:49:51.035 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.035 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.035 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.035 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481528+0000", 2024-09-16T20:49:51.035 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4900000000000002e-06 2024-09-16T20:49:51.036 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.036 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.036 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:51.036 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481565+0000", 2024-09-16T20:49:51.036 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7066000000000001e-05 2024-09-16T20:49:51.036 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.036 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.036 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:51.036 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481628+0000", 2024-09-16T20:49:51.036 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3118000000000001e-05 2024-09-16T20:49:51.036 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.036 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.036 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 2", 2024-09-16T20:49:51.036 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481822+0000", 2024-09-16T20:49:51.036 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019446200000000001 2024-09-16T20:49:51.037 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.037 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.037 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:51.037 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605988+0000", 2024-09-16T20:49:51.037 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.124165888 2024-09-16T20:49:51.037 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.037 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.037 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:51.037 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606427+0000", 2024-09-16T20:49:51.037 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00043854899999999999 2024-09-16T20:49:51.037 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.037 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.037 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:51.037 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606476+0000", 2024-09-16T20:49:51.037 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8971999999999998e-05 2024-09-16T20:49:51.038 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.038 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.038 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:51.038 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606517+0000", 2024-09-16T20:49:51.038 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1467000000000001e-05 2024-09-16T20:49:51.038 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.038 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.038 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.038 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.038 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.038 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4882.0:2508 3.12 e57/55 ondisk, result = 0)", 2024-09-16T20:49:51.038 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:49:51.038 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 64.533821152000002, 2024-09-16T20:49:51.039 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12349711400000001, 2024-09-16T20:49:51.039 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.039 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.039 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:51.039 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.039 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.039 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.039 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:49:51.039 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.039 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.039 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.039 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.039 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:49:51.039 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.039 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.040 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.040 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.040 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:49:51.040 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.06e-07 2024-09-16T20:49:51.040 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.040 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.040 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.040 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482502+0000", 2024-09-16T20:49:51.040 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.01e-06 2024-09-16T20:49:51.040 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.040 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.040 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.040 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605772+0000", 2024-09-16T20:49:51.040 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.123269687 2024-09-16T20:49:51.040 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.041 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.041 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.041 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605791+0000", 2024-09-16T20:49:51.041 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8912000000000001e-05 2024-09-16T20:49:51.041 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.041 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.041 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:51.041 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605974+0000", 2024-09-16T20:49:51.041 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000182836 2024-09-16T20:49:51.041 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.041 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.041 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:51.041 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605982+0000", 2024-09-16T20:49:51.041 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1699999999999997e-06 2024-09-16T20:49:51.041 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.042 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.042 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:51.042 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605996+0000", 2024-09-16T20:49:51.042 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3893000000000001e-05 2024-09-16T20:49:51.042 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.042 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.042 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.042 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.042 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.042 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4882.0:2512 3.f 3:f4d895b7:::rbd_data.110e7c8851c.0000000000000000:head [read 16384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T20:49:51.042 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:49:51.042 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 64.533384122000001, 2024-09-16T20:49:51.042 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12318483, 2024-09-16T20:49:51.042 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.042 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.042 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:51.043 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.043 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4882", 2024-09-16T20:49:51.043 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2436923223", 2024-09-16T20:49:51.043 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2512 2024-09-16T20:49:51.043 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.043 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.043 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.043 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.043 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:49:51.043 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.043 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.043 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.043 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.043 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:49:51.043 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.044 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.044 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.044 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.044 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:49:51.044 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3499999999999995e-07 2024-09-16T20:49:51.044 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.044 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.044 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.044 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482943+0000", 2024-09-16T20:49:51.044 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3450000000000002e-06 2024-09-16T20:49:51.044 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.044 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.044 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.044 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605772+0000", 2024-09-16T20:49:51.045 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.122829796 2024-09-16T20:49:51.045 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.045 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.045 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.045 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605792+0000", 2024-09-16T20:49:51.045 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9916999999999999e-05 2024-09-16T20:49:51.045 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.045 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.045 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:51.045 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605917+0000", 2024-09-16T20:49:51.045 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00012500799999999999 2024-09-16T20:49:51.045 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.045 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.045 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:51.046 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606004+0000", 2024-09-16T20:49:51.046 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6242999999999994e-05 2024-09-16T20:49:51.046 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.046 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.046 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:51.046 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606120+0000", 2024-09-16T20:49:51.046 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000116786 2024-09-16T20:49:51.046 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.046 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.046 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.046 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.046 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.046 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:11702 3.15 3:acf33c8b:::rbd_data.110e747f0779.000000000000001a:head [set-alloc-hint object_size 262144 write_size 262144,write 143360~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:49:51.046 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:49:51.047 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 46.878391700000002, 2024-09-16T20:49:51.047 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.084801043000000006, 2024-09-16T20:49:51.047 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.047 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.047 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:51.047 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.047 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:49:51.047 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:49:51.047 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 11702 2024-09-16T20:49:51.047 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.047 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.047 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.047 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.047 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:49:51.048 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.048 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.048 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.048 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.048 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:49:51.048 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.048 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.048 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.048 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.048 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137929+0000", 2024-09-16T20:49:51.048 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8400000000000002e-07 2024-09-16T20:49:51.048 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.048 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.048 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.049 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137937+0000", 2024-09-16T20:49:51.049 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0260000000000007e-06 2024-09-16T20:49:51.049 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.049 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.049 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.049 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137952+0000", 2024-09-16T20:49:51.049 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4659e-05 2024-09-16T20:49:51.049 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.049 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.049 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.049 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137960+0000", 2024-09-16T20:49:51.049 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0169999999999996e-06 2024-09-16T20:49:51.049 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.049 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.050 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:51.050 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138010+0000", 2024-09-16T20:49:51.050 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0021000000000001e-05 2024-09-16T20:49:51.050 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.050 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.050 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:51.050 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138114+0000", 2024-09-16T20:49:51.050 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000103797 2024-09-16T20:49:51.050 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.050 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.050 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:49:51.050 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138233+0000", 2024-09-16T20:49:51.050 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000118971 2024-09-16T20:49:51.050 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.050 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.051 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:51.051 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138953+0000", 2024-09-16T20:49:51.051 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00072026000000000002 2024-09-16T20:49:51.051 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.051 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.051 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:51.051 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222609+0000", 2024-09-16T20:49:51.051 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.083656650999999999 2024-09-16T20:49:51.051 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.051 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.051 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:51.051 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222684+0000", 2024-09-16T20:49:51.051 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4369999999999994e-05 2024-09-16T20:49:51.051 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.051 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.052 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:51.052 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222729+0000", 2024-09-16T20:49:51.052 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5287000000000002e-05 2024-09-16T20:49:51.052 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.052 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.052 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.052 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.052 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:11704 3.9 3:91220b75:::rbd_data.110e747f0779.000000000000001c:head [set-alloc-hint object_size 262144 write_size 262144,write 139264~6656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:49:51.052 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:49:51.052 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 46.878153699000002, 2024-09-16T20:49:51.052 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.084531133999999994, 2024-09-16T20:49:51.052 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.052 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.052 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:51.053 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.053 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:49:51.053 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:49:51.053 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 11704 2024-09-16T20:49:51.053 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.053 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.053 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.053 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.053 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:49:51.053 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.053 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.053 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.053 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.053 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:49:51.053 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.054 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.054 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.054 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.054 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138167+0000", 2024-09-16T20:49:51.054 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.33e-07 2024-09-16T20:49:51.054 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.054 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.054 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.054 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138179+0000", 2024-09-16T20:49:51.054 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2055999999999999e-05 2024-09-16T20:49:51.054 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.054 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.054 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.054 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138193+0000", 2024-09-16T20:49:51.054 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3961000000000001e-05 2024-09-16T20:49:51.054 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.055 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.055 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.055 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138200+0000", 2024-09-16T20:49:51.055 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4429999999999997e-06 2024-09-16T20:49:51.055 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.055 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.055 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:51.055 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138258+0000", 2024-09-16T20:49:51.055 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7225999999999997e-05 2024-09-16T20:49:51.055 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.055 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.055 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:51.055 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138364+0000", 2024-09-16T20:49:51.055 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000106013 2024-09-16T20:49:51.055 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.056 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.056 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:49:51.056 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138494+0000", 2024-09-16T20:49:51.056 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013059599999999999 2024-09-16T20:49:51.056 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.056 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.056 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:51.056 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.139220+0000", 2024-09-16T20:49:51.056 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00072539800000000004 2024-09-16T20:49:51.056 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.056 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.056 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:51.056 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222608+0000", 2024-09-16T20:49:51.056 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.083388620999999996 2024-09-16T20:49:51.056 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.056 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.057 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:51.057 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222651+0000", 2024-09-16T20:49:51.057 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2228000000000003e-05 2024-09-16T20:49:51.057 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.057 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.057 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:51.057 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222697+0000", 2024-09-16T20:49:51.057 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6659000000000002e-05 2024-09-16T20:49:51.057 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.057 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.057 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.057 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.057 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.057 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5138.0:9358 3.c 3.c25699cc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T20:49:51.058 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:49:51.058 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 24.166006394, 2024-09-16T20:49:51.058 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0018657580000001, 2024-09-16T20:49:51.058 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.058 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.058 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:49:51.058 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.058 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5138", 2024-09-16T20:49:51.058 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:49:51.058 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 9358 2024-09-16T20:49:51.058 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.058 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.058 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.058 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.059 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:49:51.059 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.059 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.059 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.059 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.059 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:49:51.059 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.059 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.059 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.059 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.059 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850314+0000", 2024-09-16T20:49:51.059 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4099999999999999e-07 2024-09-16T20:49:51.059 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.059 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.059 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.060 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850316+0000", 2024-09-16T20:49:51.060 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2469999999999998e-06 2024-09-16T20:49:51.060 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.060 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.060 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.060 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850322+0000", 2024-09-16T20:49:51.060 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3470000000000004e-06 2024-09-16T20:49:51.060 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.060 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.060 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.060 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850327+0000", 2024-09-16T20:49:51.060 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9450000000000001e-06 2024-09-16T20:49:51.060 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.060 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.060 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:51.061 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850376+0000", 2024-09-16T20:49:51.061 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8257000000000002e-05 2024-09-16T20:49:51.061 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.061 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.061 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:49:51.061 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850385+0000", 2024-09-16T20:49:51.061 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9800000000000004e-06 2024-09-16T20:49:51.061 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.061 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.061 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:51.061 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852167+0000", 2024-09-16T20:49:51.061 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0017825659999999 2024-09-16T20:49:51.061 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.061 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.061 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:51.061 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852179+0000", 2024-09-16T20:49:51.062 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1975e-05 2024-09-16T20:49:51.062 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.062 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.062 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.062 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.062 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.062 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5138.0:9382 3.c 3.c25699cc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T20:49:51.062 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:49:51.062 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 24.157659378000002, 2024-09-16T20:49:51.062 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.99358404700000003, 2024-09-16T20:49:51.062 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.062 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.062 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:49:51.063 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.063 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5138", 2024-09-16T20:49:51.063 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:49:51.063 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 9382 2024-09-16T20:49:51.063 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.063 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.063 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.063 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.063 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:49:51.063 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.063 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.063 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.063 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.063 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:49:51.064 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.064 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.064 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.064 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.064 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858661+0000", 2024-09-16T20:49:51.064 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0200000000000002e-07 2024-09-16T20:49:51.064 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.064 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.064 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.064 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858664+0000", 2024-09-16T20:49:51.064 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.038e-06 2024-09-16T20:49:51.064 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.064 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.064 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.065 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858671+0000", 2024-09-16T20:49:51.065 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0110000000000003e-06 2024-09-16T20:49:51.065 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.065 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.065 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.065 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858677+0000", 2024-09-16T20:49:51.065 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0959999999999997e-06 2024-09-16T20:49:51.065 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.065 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.065 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:51.065 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858732+0000", 2024-09-16T20:49:51.065 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4543000000000003e-05 2024-09-16T20:49:51.065 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.065 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.065 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:49:51.066 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858739+0000", 2024-09-16T20:49:51.066 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5429999999999996e-06 2024-09-16T20:49:51.066 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.066 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.066 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:51.066 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852236+0000", 2024-09-16T20:49:51.066 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.993496453 2024-09-16T20:49:51.066 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.066 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.066 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:51.066 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852244+0000", 2024-09-16T20:49:51.066 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8610000000000004e-06 2024-09-16T20:49:51.066 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.066 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.067 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.067 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.067 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.067 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:361 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 372736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:49:51.067 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246714+0000", 2024-09-16T20:49:51.067 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 14.76960523, 2024-09-16T20:49:51.067 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050338229999999998, 2024-09-16T20:49:51.067 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.067 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.067 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:51.067 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.067 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:49:51.067 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:49:51.067 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 361 2024-09-16T20:49:51.067 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.068 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.068 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.068 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.068 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246714+0000", 2024-09-16T20:49:51.068 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.068 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.068 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.068 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.068 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246714+0000", 2024-09-16T20:49:51.068 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.068 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.068 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.068 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.068 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246715+0000", 2024-09-16T20:49:51.069 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1899999999999999e-07 2024-09-16T20:49:51.069 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.069 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.069 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.069 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246716+0000", 2024-09-16T20:49:51.069 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1659999999999999e-06 2024-09-16T20:49:51.069 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.069 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.069 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.069 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246721+0000", 2024-09-16T20:49:51.069 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9830000000000004e-06 2024-09-16T20:49:51.069 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.069 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.069 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.069 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246723+0000", 2024-09-16T20:49:51.070 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5679999999999998e-06 2024-09-16T20:49:51.070 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.070 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.070 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:51.070 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.296875+0000", 2024-09-16T20:49:51.070 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050151683000000002 2024-09-16T20:49:51.070 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.070 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.070 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:51.070 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.296959+0000", 2024-09-16T20:49:51.070 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4079999999999995e-05 2024-09-16T20:49:51.070 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.070 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.070 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:51.070 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297053+0000", 2024-09-16T20:49:51.070 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.3530999999999996e-05 2024-09-16T20:49:51.071 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.071 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.071 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.071 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.071 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.071 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:362 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 385024~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:49:51.071 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246727+0000", 2024-09-16T20:49:51.071 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 14.769593121, 2024-09-16T20:49:51.071 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050546528, 2024-09-16T20:49:51.071 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.071 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.071 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:51.071 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.071 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:49:51.071 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:49:51.072 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 362 2024-09-16T20:49:51.072 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.072 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.072 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.072 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.072 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246727+0000", 2024-09-16T20:49:51.072 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.072 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.072 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.072 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.072 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246727+0000", 2024-09-16T20:49:51.072 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.072 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.072 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.072 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.073 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246727+0000", 2024-09-16T20:49:51.073 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.35e-07 2024-09-16T20:49:51.073 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.073 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.073 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.073 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246728+0000", 2024-09-16T20:49:51.073 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.175e-06 2024-09-16T20:49:51.073 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.073 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.073 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.073 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246734+0000", 2024-09-16T20:49:51.073 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2160000000000003e-06 2024-09-16T20:49:51.073 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.073 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.073 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.074 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246737+0000", 2024-09-16T20:49:51.074 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9450000000000002e-06 2024-09-16T20:49:51.074 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.074 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.074 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:51.074 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297103+0000", 2024-09-16T20:49:51.074 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050365518999999997 2024-09-16T20:49:51.074 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.074 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.074 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:51.074 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297183+0000", 2024-09-16T20:49:51.074 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0208000000000002e-05 2024-09-16T20:49:51.074 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.074 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.074 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:51.074 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297273+0000", 2024-09-16T20:49:51.075 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0229999999999995e-05 2024-09-16T20:49:51.075 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.075 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.075 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.075 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.075 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.075 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:368 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 327680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:49:51.075 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246746+0000", 2024-09-16T20:49:51.075 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 14.769573841, 2024-09-16T20:49:51.075 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050730747999999999, 2024-09-16T20:49:51.075 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.075 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.075 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:51.075 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.075 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:49:51.076 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:49:51.076 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 368 2024-09-16T20:49:51.076 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.076 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.076 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.076 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.076 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246746+0000", 2024-09-16T20:49:51.076 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.076 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.076 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.076 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.076 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246746+0000", 2024-09-16T20:49:51.076 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.076 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.076 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.076 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.077 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246746+0000", 2024-09-16T20:49:51.077 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8099999999999999e-07 2024-09-16T20:49:51.077 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.077 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.077 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.077 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246754+0000", 2024-09-16T20:49:51.077 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4440000000000002e-06 2024-09-16T20:49:51.077 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.077 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.077 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.077 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246758+0000", 2024-09-16T20:49:51.077 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3900000000000003e-06 2024-09-16T20:49:51.077 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.077 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.077 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.078 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246761+0000", 2024-09-16T20:49:51.078 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.101e-06 2024-09-16T20:49:51.078 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.078 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.078 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:51.078 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297310+0000", 2024-09-16T20:49:51.078 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050549166 2024-09-16T20:49:51.078 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.078 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.078 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:51.078 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297387+0000", 2024-09-16T20:49:51.078 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6658999999999999e-05 2024-09-16T20:49:51.078 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.078 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.079 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:51.079 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297477+0000", 2024-09-16T20:49:51.079 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9707e-05 2024-09-16T20:49:51.079 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.079 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.079 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.079 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.079 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.079 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:369 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 339968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:49:51.079 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246766+0000", 2024-09-16T20:49:51.079 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 14.769554116, 2024-09-16T20:49:51.079 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050911291999999997, 2024-09-16T20:49:51.079 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.079 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.080 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:51.080 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.080 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:49:51.080 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:49:51.080 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 369 2024-09-16T20:49:51.080 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.080 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.080 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.080 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.080 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246766+0000", 2024-09-16T20:49:51.080 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.080 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.080 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.080 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.081 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246766+0000", 2024-09-16T20:49:51.081 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.081 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.081 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.081 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.081 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246766+0000", 2024-09-16T20:49:51.081 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3200000000000001e-07 2024-09-16T20:49:51.081 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.081 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.081 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.081 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246767+0000", 2024-09-16T20:49:51.081 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.252e-06 2024-09-16T20:49:51.082 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.082 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.082 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.082 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246770+0000", 2024-09-16T20:49:51.082 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1049999999999999e-06 2024-09-16T20:49:51.082 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.082 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.082 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.082 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246773+0000", 2024-09-16T20:49:51.082 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.396e-06 2024-09-16T20:49:51.082 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.082 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.082 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:51.082 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297513+0000", 2024-09-16T20:49:51.082 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050740800000000003 2024-09-16T20:49:51.083 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.083 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.083 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:51.083 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297589+0000", 2024-09-16T20:49:51.083 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5318e-05 2024-09-16T20:49:51.083 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.083 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.083 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:51.083 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297677+0000", 2024-09-16T20:49:51.083 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8189000000000006e-05 2024-09-16T20:49:51.083 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.083 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.083 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.083 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.084 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.084 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:370 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 348160~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:49:51.084 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246776+0000", 2024-09-16T20:49:51.084 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 14.769544100999999, 2024-09-16T20:49:51.084 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051096643999999997, 2024-09-16T20:49:51.084 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.084 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.084 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:51.084 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.084 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:49:51.084 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:49:51.084 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 370 2024-09-16T20:49:51.084 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.085 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.085 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.085 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.085 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246776+0000", 2024-09-16T20:49:51.085 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.085 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.085 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.085 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.085 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246776+0000", 2024-09-16T20:49:51.085 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.085 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.085 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.085 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.086 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246776+0000", 2024-09-16T20:49:51.086 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2499999999999999e-07 2024-09-16T20:49:51.086 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.086 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.086 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.086 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246777+0000", 2024-09-16T20:49:51.086 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1510000000000001e-06 2024-09-16T20:49:51.086 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.086 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.086 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.086 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246780+0000", 2024-09-16T20:49:51.086 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9610000000000001e-06 2024-09-16T20:49:51.086 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.086 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.086 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.087 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246783+0000", 2024-09-16T20:49:51.087 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6129999999999998e-06 2024-09-16T20:49:51.087 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.087 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.087 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:51.087 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297713+0000", 2024-09-16T20:49:51.087 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050929950000000002 2024-09-16T20:49:51.087 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.087 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.087 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:51.087 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297786+0000", 2024-09-16T20:49:51.087 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3096000000000006e-05 2024-09-16T20:49:51.087 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.087 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.087 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:51.087 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297872+0000", 2024-09-16T20:49:51.088 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6648000000000001e-05 2024-09-16T20:49:51.088 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.088 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.088 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.088 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.088 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.088 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:371 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 380928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:49:51.088 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246791+0000", 2024-09-16T20:49:51.088 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 14.769528465, 2024-09-16T20:49:51.088 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051287209, 2024-09-16T20:49:51.088 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.088 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.088 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:51.088 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.089 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:49:51.089 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:49:51.089 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 371 2024-09-16T20:49:51.089 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.089 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.089 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.089 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.089 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246791+0000", 2024-09-16T20:49:51.089 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.089 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.089 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.089 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.089 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246791+0000", 2024-09-16T20:49:51.090 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.090 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.090 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.090 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.090 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246792+0000", 2024-09-16T20:49:51.090 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8200000000000001e-07 2024-09-16T20:49:51.090 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.090 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.090 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.090 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246793+0000", 2024-09-16T20:49:51.090 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.28e-06 2024-09-16T20:49:51.090 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.090 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.090 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.090 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246796+0000", 2024-09-16T20:49:51.091 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.597e-06 2024-09-16T20:49:51.091 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.091 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.091 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.091 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246799+0000", 2024-09-16T20:49:51.091 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7089999999999998e-06 2024-09-16T20:49:51.091 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.091 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.091 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:51.091 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297910+0000", 2024-09-16T20:49:51.091 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051110598 2024-09-16T20:49:51.091 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.091 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.091 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:51.091 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297986+0000", 2024-09-16T20:49:51.092 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6711000000000005e-05 2024-09-16T20:49:51.092 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.092 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.092 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:51.092 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.298078+0000", 2024-09-16T20:49:51.092 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2032000000000002e-05 2024-09-16T20:49:51.092 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.092 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.092 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.092 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.092 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.092 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:373 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 376832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:49:51.092 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246904+0000", 2024-09-16T20:49:51.092 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 14.769416043, 2024-09-16T20:49:51.093 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051310617000000003, 2024-09-16T20:49:51.093 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.093 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.093 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:51.093 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.093 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:49:51.093 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:49:51.093 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 373 2024-09-16T20:49:51.093 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.093 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.093 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.093 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.093 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246904+0000", 2024-09-16T20:49:51.093 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.093 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.094 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.094 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.094 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246904+0000", 2024-09-16T20:49:51.094 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.094 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.094 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.094 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.094 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246904+0000", 2024-09-16T20:49:51.094 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8000000000000001e-07 2024-09-16T20:49:51.094 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.094 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.094 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.094 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246908+0000", 2024-09-16T20:49:51.094 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4039999999999997e-06 2024-09-16T20:49:51.094 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.095 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.095 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.095 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246913+0000", 2024-09-16T20:49:51.095 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0030000000000002e-06 2024-09-16T20:49:51.095 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.095 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.095 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.095 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246917+0000", 2024-09-16T20:49:51.095 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.303e-06 2024-09-16T20:49:51.095 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.095 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.095 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:51.095 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.298114+0000", 2024-09-16T20:49:51.095 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051197724 2024-09-16T20:49:51.096 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.096 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.096 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:51.096 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.298159+0000", 2024-09-16T20:49:51.096 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4882999999999997e-05 2024-09-16T20:49:51.096 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.096 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.096 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:51.096 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.298214+0000", 2024-09-16T20:49:51.096 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4920000000000003e-05 2024-09-16T20:49:51.096 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.096 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.096 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.096 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.096 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.097 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:374 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 389120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:49:51.097 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246927+0000", 2024-09-16T20:49:51.097 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 14.769392994, 2024-09-16T20:49:51.097 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051498164999999999, 2024-09-16T20:49:51.097 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.097 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.097 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:51.097 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.097 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:49:51.097 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:49:51.097 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 374 2024-09-16T20:49:51.097 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.097 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.098 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.098 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.098 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246927+0000", 2024-09-16T20:49:51.098 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.098 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.098 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.098 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.098 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246927+0000", 2024-09-16T20:49:51.098 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.098 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.098 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.098 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.098 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246927+0000", 2024-09-16T20:49:51.098 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7e-07 2024-09-16T20:49:51.099 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.099 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.099 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.099 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246929+0000", 2024-09-16T20:49:51.099 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.607e-06 2024-09-16T20:49:51.099 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.099 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.099 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.099 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246932+0000", 2024-09-16T20:49:51.099 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4539999999999999e-06 2024-09-16T20:49:51.099 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.099 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.099 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.100 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246935+0000", 2024-09-16T20:49:51.100 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6570000000000001e-06 2024-09-16T20:49:51.100 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.100 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.100 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:51.100 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.298242+0000", 2024-09-16T20:49:51.100 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051306887000000002 2024-09-16T20:49:51.100 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.100 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.100 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:51.100 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.298327+0000", 2024-09-16T20:49:51.100 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5068999999999995e-05 2024-09-16T20:49:51.100 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.101 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.101 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:51.101 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.298425+0000", 2024-09-16T20:49:51.101 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8021000000000002e-05 2024-09-16T20:49:51.101 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.101 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.101 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.101 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.101 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.101 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5763.0:4895 3.a 3:5384606c:::rbd_data.110e1b19a241.0000000000000018:head [read 0~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-16T20:49:51.101 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:45.598212+0000", 2024-09-16T20:49:51.101 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 5.4181073370000004, 2024-09-16T20:49:51.101 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0375864829999999, 2024-09-16T20:49:51.101 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.102 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.102 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:51.102 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.102 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5763", 2024-09-16T20:49:51.102 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/623897567", 2024-09-16T20:49:51.102 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4895 2024-09-16T20:49:51.102 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.102 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.102 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.102 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.102 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598212+0000", 2024-09-16T20:49:51.102 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.102 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.103 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.103 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.103 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598212+0000", 2024-09-16T20:49:51.103 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.103 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.103 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.103 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.103 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598214+0000", 2024-09-16T20:49:51.103 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0090000000000002e-06 2024-09-16T20:49:51.103 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.103 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.103 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.103 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598222+0000", 2024-09-16T20:49:51.103 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6699999999999994e-06 2024-09-16T20:49:51.104 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.104 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.104 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.104 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598241+0000", 2024-09-16T20:49:51.104 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8896999999999999e-05 2024-09-16T20:49:51.104 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.104 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.104 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.104 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598250+0000", 2024-09-16T20:49:51.104 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6649999999999992e-06 2024-09-16T20:49:51.104 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.104 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.104 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:51.104 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598325+0000", 2024-09-16T20:49:51.104 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5189000000000001e-05 2024-09-16T20:49:51.105 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.105 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.105 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op must wait for map", 2024-09-16T20:49:51.105 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598333+0000", 2024-09-16T20:49:51.105 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.3170000000000002e-06 2024-09-16T20:49:51.105 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.105 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.105 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:51.105 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601355+0000", 2024-09-16T20:49:51.105 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0030215400000000001 2024-09-16T20:49:51.105 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.105 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.105 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:49:51.105 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601365+0000", 2024-09-16T20:49:51.106 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0215e-05 2024-09-16T20:49:51.106 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.106 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.106 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:51.106 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635506+0000", 2024-09-16T20:49:51.106 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0341408240000001 2024-09-16T20:49:51.106 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.106 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.106 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:51.106 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635655+0000", 2024-09-16T20:49:51.106 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000149412 2024-09-16T20:49:51.106 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.106 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.106 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:51.107 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635799+0000", 2024-09-16T20:49:51.107 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000143745 2024-09-16T20:49:51.107 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.107 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.107 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.107 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.107 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.107 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5763.0:4900 3.a 3:5384606c:::rbd_data.110e1b19a241.0000000000000018:head [read 8192~24576] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-16T20:49:51.107 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:49:51.107 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 5.4180488609999999, 2024-09-16T20:49:51.107 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0379340800000001, 2024-09-16T20:49:51.107 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.107 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.107 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:51.108 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.108 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5763", 2024-09-16T20:49:51.108 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/623897567", 2024-09-16T20:49:51.108 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4900 2024-09-16T20:49:51.108 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.108 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.108 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.108 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.108 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:49:51.108 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.108 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.108 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.108 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.109 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:49:51.109 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.109 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.109 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.109 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.109 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:49:51.109 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4300000000000003e-07 2024-09-16T20:49:51.109 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.109 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.109 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.109 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598274+0000", 2024-09-16T20:49:51.109 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.976e-06 2024-09-16T20:49:51.109 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.109 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.110 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.110 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598282+0000", 2024-09-16T20:49:51.110 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0029999999999994e-06 2024-09-16T20:49:51.110 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.110 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.110 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.110 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598289+0000", 2024-09-16T20:49:51.110 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2389999999999999e-06 2024-09-16T20:49:51.110 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.110 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.110 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:51.110 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598430+0000", 2024-09-16T20:49:51.110 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014119600000000001 2024-09-16T20:49:51.110 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.111 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.111 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting_for_map not empty", 2024-09-16T20:49:51.111 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598435+0000", 2024-09-16T20:49:51.111 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3489999999999999e-06 2024-09-16T20:49:51.111 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.111 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.111 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:51.111 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601429+0000", 2024-09-16T20:49:51.111 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0029942720000000001 2024-09-16T20:49:51.111 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.111 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.111 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:49:51.111 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601438+0000", 2024-09-16T20:49:51.111 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4759999999999998e-06 2024-09-16T20:49:51.111 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.112 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.112 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:51.112 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635987+0000", 2024-09-16T20:49:51.112 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0345495010000001 2024-09-16T20:49:51.112 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.112 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.112 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:51.112 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.636056+0000", 2024-09-16T20:49:51.112 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8610999999999998e-05 2024-09-16T20:49:51.112 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.112 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.112 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:51.112 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.636205+0000", 2024-09-16T20:49:51.112 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014891399999999999 2024-09-16T20:49:51.112 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.113 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.113 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.113 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.113 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.113 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:51.114 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:49:51.115 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:51.113+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:51.215 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:51.214+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:51.306 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:51.306 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:49:51.306 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.306 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18780 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.306 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.264754+0000", 2024-09-16T20:49:51.307 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.029577846000000001, 2024-09-16T20:49:51.307 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.029596357, 2024-09-16T20:49:51.307 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.307 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.307 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.307 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.307 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.307 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.307 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18780 2024-09-16T20:49:51.307 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.307 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.307 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.307 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.308 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.264754+0000", 2024-09-16T20:49:51.308 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.308 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.308 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.308 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.308 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.264754+0000", 2024-09-16T20:49:51.308 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.308 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.308 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.308 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.308 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.264754+0000", 2024-09-16T20:49:51.308 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3200000000000001e-07 2024-09-16T20:49:51.309 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.309 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.309 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.309 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.264755+0000", 2024-09-16T20:49:51.309 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1030000000000001e-06 2024-09-16T20:49:51.309 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.309 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.309 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.309 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.264759+0000", 2024-09-16T20:49:51.309 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1470000000000003e-06 2024-09-16T20:49:51.309 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.309 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.309 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.310 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.264763+0000", 2024-09-16T20:49:51.310 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2989999999999998e-06 2024-09-16T20:49:51.310 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.310 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.310 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.310 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.310 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.310 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18813 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.310 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.265194+0000", 2024-09-16T20:49:51.310 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.029137690000000001, 2024-09-16T20:49:51.310 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.029213981, 2024-09-16T20:49:51.310 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.310 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.311 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.311 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.311 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.311 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.311 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18813 2024-09-16T20:49:51.311 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.311 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.311 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.311 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.311 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265194+0000", 2024-09-16T20:49:51.311 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.311 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.311 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.311 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.312 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265194+0000", 2024-09-16T20:49:51.312 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.312 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.312 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.312 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.312 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265194+0000", 2024-09-16T20:49:51.312 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6199999999999999e-07 2024-09-16T20:49:51.312 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.312 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.312 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.312 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265195+0000", 2024-09-16T20:49:51.312 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.009e-06 2024-09-16T20:49:51.312 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.313 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.313 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.313 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265198+0000", 2024-09-16T20:49:51.313 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0400000000000001e-06 2024-09-16T20:49:51.313 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.313 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.313 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.313 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265202+0000", 2024-09-16T20:49:51.313 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7469999999999999e-06 2024-09-16T20:49:51.313 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.313 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.313 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.313 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.314 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.314 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18845 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.314 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.265617+0000", 2024-09-16T20:49:51.314 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.028714205, 2024-09-16T20:49:51.314 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.028840963000000001, 2024-09-16T20:49:51.314 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.314 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.314 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.314 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.314 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.314 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.314 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18845 2024-09-16T20:49:51.314 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.315 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.315 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.315 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.315 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265617+0000", 2024-09-16T20:49:51.315 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.315 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.315 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.315 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.315 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265617+0000", 2024-09-16T20:49:51.315 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.315 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.315 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.315 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.315 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265618+0000", 2024-09-16T20:49:51.316 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.77e-07 2024-09-16T20:49:51.316 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.316 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.316 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.316 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265619+0000", 2024-09-16T20:49:51.316 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.39e-06 2024-09-16T20:49:51.316 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.316 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.316 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.316 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265623+0000", 2024-09-16T20:49:51.316 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3150000000000002e-06 2024-09-16T20:49:51.316 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.316 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.316 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.317 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265626+0000", 2024-09-16T20:49:51.317 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.999e-06 2024-09-16T20:49:51.317 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.317 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.317 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.317 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.317 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.317 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18880 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.317 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.271483+0000", 2024-09-16T20:49:51.317 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.022848833999999998, 2024-09-16T20:49:51.317 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.023014303, 2024-09-16T20:49:51.317 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.317 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.317 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.318 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.318 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.318 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.318 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18880 2024-09-16T20:49:51.318 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.318 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.318 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.318 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.318 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271483+0000", 2024-09-16T20:49:51.318 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.318 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.318 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.318 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.318 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271483+0000", 2024-09-16T20:49:51.318 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.319 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.319 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.319 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.319 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271483+0000", 2024-09-16T20:49:51.319 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5199999999999998e-07 2024-09-16T20:49:51.319 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.319 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.319 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.319 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271484+0000", 2024-09-16T20:49:51.319 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0380000000000001e-06 2024-09-16T20:49:51.319 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.319 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.319 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.319 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271487+0000", 2024-09-16T20:49:51.319 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1520000000000001e-06 2024-09-16T20:49:51.320 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.320 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.320 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.320 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271490+0000", 2024-09-16T20:49:51.320 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6589999999999999e-06 2024-09-16T20:49:51.320 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.320 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.320 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.320 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.320 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.320 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18911 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.320 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.274834+0000", 2024-09-16T20:49:51.320 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.019497035999999999, 2024-09-16T20:49:51.320 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.019687626, 2024-09-16T20:49:51.321 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.321 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.321 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.321 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.321 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.321 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.321 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18911 2024-09-16T20:49:51.321 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.321 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.321 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.321 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.321 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274834+0000", 2024-09-16T20:49:51.321 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.321 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.322 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.322 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.322 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274834+0000", 2024-09-16T20:49:51.322 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.322 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.322 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.322 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.322 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274835+0000", 2024-09-16T20:49:51.322 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.4400000000000001e-07 2024-09-16T20:49:51.322 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.322 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.322 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.322 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274836+0000", 2024-09-16T20:49:51.322 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.26e-06 2024-09-16T20:49:51.322 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.323 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.323 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.323 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274840+0000", 2024-09-16T20:49:51.323 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8659999999999999e-06 2024-09-16T20:49:51.323 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.323 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.323 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.323 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274844+0000", 2024-09-16T20:49:51.323 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9940000000000002e-06 2024-09-16T20:49:51.323 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.323 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.323 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.323 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.323 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.324 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18944 3.10 3.996ee350 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.324 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.282914+0000", 2024-09-16T20:49:51.324 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.011417543, 2024-09-16T20:49:51.324 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.011636547000000001, 2024-09-16T20:49:51.324 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.324 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.324 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.324 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.324 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.324 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.324 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18944 2024-09-16T20:49:51.324 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.325 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.325 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.325 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.325 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282914+0000", 2024-09-16T20:49:51.325 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.325 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.325 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.325 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.325 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282914+0000", 2024-09-16T20:49:51.325 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.325 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.325 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.326 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.326 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282914+0000", 2024-09-16T20:49:51.326 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5599999999999995e-07 2024-09-16T20:49:51.326 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.326 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.326 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.326 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282917+0000", 2024-09-16T20:49:51.326 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2419999999999999e-06 2024-09-16T20:49:51.326 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.326 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.326 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.326 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282926+0000", 2024-09-16T20:49:51.326 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.1099999999999992e-06 2024-09-16T20:49:51.326 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.327 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.327 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.327 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282931+0000", 2024-09-16T20:49:51.327 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4689999999999996e-06 2024-09-16T20:49:51.327 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.327 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.327 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.327 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.327 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.327 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18781 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.327 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.264771+0000", 2024-09-16T20:49:51.327 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.029559963000000002, 2024-09-16T20:49:51.327 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.029803791, 2024-09-16T20:49:51.328 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.328 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.328 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.328 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.328 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.328 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.328 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18781 2024-09-16T20:49:51.328 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.328 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.328 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.328 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.328 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.264771+0000", 2024-09-16T20:49:51.328 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.329 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.329 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.329 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.329 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.264771+0000", 2024-09-16T20:49:51.329 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.329 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.329 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.329 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.329 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.264772+0000", 2024-09-16T20:49:51.329 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5199999999999998e-07 2024-09-16T20:49:51.329 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.329 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.329 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.330 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.264774+0000", 2024-09-16T20:49:51.330 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7680000000000001e-06 2024-09-16T20:49:51.330 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.330 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.330 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.330 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.264779+0000", 2024-09-16T20:49:51.330 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5169999999999999e-06 2024-09-16T20:49:51.330 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.330 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.330 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.330 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.264783+0000", 2024-09-16T20:49:51.330 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2339999999999996e-06 2024-09-16T20:49:51.330 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.330 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.330 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.331 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.331 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.331 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18814 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.331 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.265221+0000", 2024-09-16T20:49:51.331 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.029109981, 2024-09-16T20:49:51.331 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.029397996999999999, 2024-09-16T20:49:51.331 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.331 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.331 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.331 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.331 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.331 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.331 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18814 2024-09-16T20:49:51.331 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.331 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.332 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.332 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.332 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265221+0000", 2024-09-16T20:49:51.332 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.332 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.332 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.332 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.332 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265221+0000", 2024-09-16T20:49:51.332 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.332 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.332 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.332 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.332 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265222+0000", 2024-09-16T20:49:51.332 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2399999999999999e-07 2024-09-16T20:49:51.333 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.333 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.333 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.333 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265223+0000", 2024-09-16T20:49:51.333 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3430000000000001e-06 2024-09-16T20:49:51.333 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.333 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.333 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.333 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265226+0000", 2024-09-16T20:49:51.333 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3009999999999998e-06 2024-09-16T20:49:51.333 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.333 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.333 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.333 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265229+0000", 2024-09-16T20:49:51.334 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8430000000000001e-06 2024-09-16T20:49:51.334 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.334 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.334 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.334 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.334 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.334 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18846 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.334 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.265630+0000", 2024-09-16T20:49:51.334 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.028701706, 2024-09-16T20:49:51.334 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.029037901000000001, 2024-09-16T20:49:51.334 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.334 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.334 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.335 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.335 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.335 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.335 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18846 2024-09-16T20:49:51.335 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.335 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.335 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.335 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.335 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265630+0000", 2024-09-16T20:49:51.335 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.335 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.335 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.335 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.335 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265630+0000", 2024-09-16T20:49:51.335 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.336 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.336 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.336 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.336 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265630+0000", 2024-09-16T20:49:51.336 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.17e-07 2024-09-16T20:49:51.336 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.336 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.336 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.336 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265631+0000", 2024-09-16T20:49:51.336 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.062e-06 2024-09-16T20:49:51.336 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.336 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.337 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.337 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265635+0000", 2024-09-16T20:49:51.337 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0029999999999996e-06 2024-09-16T20:49:51.337 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.337 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.337 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.337 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265638+0000", 2024-09-16T20:49:51.337 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.695e-06 2024-09-16T20:49:51.337 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.337 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.337 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.337 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.337 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.337 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18881 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.338 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.271496+0000", 2024-09-16T20:49:51.338 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.022835304000000001, 2024-09-16T20:49:51.338 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.023222757, 2024-09-16T20:49:51.338 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.338 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.338 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.338 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.338 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.338 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.338 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18881 2024-09-16T20:49:51.338 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.338 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.338 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.338 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.338 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271496+0000", 2024-09-16T20:49:51.339 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.339 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.339 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.339 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.339 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271496+0000", 2024-09-16T20:49:51.339 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.339 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.339 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.339 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.339 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271496+0000", 2024-09-16T20:49:51.339 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3099999999999999e-07 2024-09-16T20:49:51.339 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.339 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.339 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.340 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271498+0000", 2024-09-16T20:49:51.340 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2410000000000001e-06 2024-09-16T20:49:51.340 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.340 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.340 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.340 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271501+0000", 2024-09-16T20:49:51.340 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.444e-06 2024-09-16T20:49:51.340 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.340 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.340 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.340 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271504+0000", 2024-09-16T20:49:51.340 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6979999999999998e-06 2024-09-16T20:49:51.340 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.340 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.341 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.341 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.341 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.341 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18912 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.341 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.274847+0000", 2024-09-16T20:49:51.341 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.019484561000000001, 2024-09-16T20:49:51.341 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.019919808000000001, 2024-09-16T20:49:51.341 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.341 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.341 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.341 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.341 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.341 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.341 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18912 2024-09-16T20:49:51.342 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.342 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.342 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.342 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.342 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274847+0000", 2024-09-16T20:49:51.342 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.342 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.342 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.342 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.342 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274847+0000", 2024-09-16T20:49:51.342 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.342 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.342 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.342 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.342 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274847+0000", 2024-09-16T20:49:51.342 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3200000000000001e-07 2024-09-16T20:49:51.343 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.343 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.343 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.343 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274848+0000", 2024-09-16T20:49:51.343 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.102e-06 2024-09-16T20:49:51.343 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.343 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.343 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.343 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274851+0000", 2024-09-16T20:49:51.343 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1300000000000001e-06 2024-09-16T20:49:51.343 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.343 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.343 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.343 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274854+0000", 2024-09-16T20:49:51.343 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7769999999999998e-06 2024-09-16T20:49:51.344 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.344 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.344 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.344 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.344 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.344 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18945 3.10 3.996ee350 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.344 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.282938+0000", 2024-09-16T20:49:51.344 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.011392928, 2024-09-16T20:49:51.344 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.011868583, 2024-09-16T20:49:51.344 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.344 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.344 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.344 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.344 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.344 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.345 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18945 2024-09-16T20:49:51.345 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.345 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.345 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.345 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.345 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282938+0000", 2024-09-16T20:49:51.345 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.345 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.345 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.345 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.345 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282938+0000", 2024-09-16T20:49:51.345 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.345 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.345 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.345 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.346 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282939+0000", 2024-09-16T20:49:51.346 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.89e-07 2024-09-16T20:49:51.346 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.346 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.346 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.346 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282941+0000", 2024-09-16T20:49:51.346 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0719999999999998e-06 2024-09-16T20:49:51.346 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.346 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.346 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.346 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282948+0000", 2024-09-16T20:49:51.346 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6930000000000001e-06 2024-09-16T20:49:51.346 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.346 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.346 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.347 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282953+0000", 2024-09-16T20:49:51.347 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2780000000000001e-06 2024-09-16T20:49:51.347 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.347 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.347 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.347 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.347 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.347 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18782 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.347 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.264796+0000", 2024-09-16T20:49:51.347 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.029535713000000002, 2024-09-16T20:49:51.347 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.030036236000000001, 2024-09-16T20:49:51.347 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.347 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.347 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.347 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.348 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.348 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.348 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18782 2024-09-16T20:49:51.348 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.348 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.348 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.348 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.348 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.264796+0000", 2024-09-16T20:49:51.348 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.348 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.348 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.348 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.348 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.264796+0000", 2024-09-16T20:49:51.348 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.348 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.349 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.349 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.349 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.264796+0000", 2024-09-16T20:49:51.349 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8500000000000002e-07 2024-09-16T20:49:51.349 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.349 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.349 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.349 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.264804+0000", 2024-09-16T20:49:51.349 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.841e-06 2024-09-16T20:49:51.349 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.349 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.349 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.349 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.264809+0000", 2024-09-16T20:49:51.350 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3990000000000003e-06 2024-09-16T20:49:51.350 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.350 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.350 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.350 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.264813+0000", 2024-09-16T20:49:51.350 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.343e-06 2024-09-16T20:49:51.350 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.350 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.350 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.350 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.350 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.350 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18815 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.350 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.265233+0000", 2024-09-16T20:49:51.350 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.029098807000000001, 2024-09-16T20:49:51.351 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.029623178999999999, 2024-09-16T20:49:51.351 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.351 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.351 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.351 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.351 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.351 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.351 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18815 2024-09-16T20:49:51.351 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.351 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.351 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.351 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.351 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265233+0000", 2024-09-16T20:49:51.352 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.352 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.352 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.352 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.352 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265233+0000", 2024-09-16T20:49:51.352 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.352 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.352 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.352 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.352 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265233+0000", 2024-09-16T20:49:51.352 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0600000000000001e-07 2024-09-16T20:49:51.352 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.352 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.352 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.353 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265234+0000", 2024-09-16T20:49:51.353 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0279999999999999e-06 2024-09-16T20:49:51.353 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.353 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.353 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.353 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265237+0000", 2024-09-16T20:49:51.353 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0029999999999999e-06 2024-09-16T20:49:51.353 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.353 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.353 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.353 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265240+0000", 2024-09-16T20:49:51.353 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.616e-06 2024-09-16T20:49:51.353 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.353 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.354 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.354 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.354 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.354 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18847 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.354 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.265641+0000", 2024-09-16T20:49:51.354 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.028690506000000001, 2024-09-16T20:49:51.354 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.029238487000000001, 2024-09-16T20:49:51.354 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.354 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.354 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.354 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.354 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.354 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.354 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18847 2024-09-16T20:49:51.354 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.355 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.355 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.355 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.355 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265641+0000", 2024-09-16T20:49:51.355 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.355 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.355 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.355 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.355 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265641+0000", 2024-09-16T20:49:51.355 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.355 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.355 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.355 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.355 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265641+0000", 2024-09-16T20:49:51.356 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1199999999999999e-07 2024-09-16T20:49:51.356 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.356 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.356 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.356 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265642+0000", 2024-09-16T20:49:51.356 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0049999999999999e-06 2024-09-16T20:49:51.356 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.356 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.356 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.356 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265645+0000", 2024-09-16T20:49:51.356 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.932e-06 2024-09-16T20:49:51.356 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.356 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.356 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.356 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265648+0000", 2024-09-16T20:49:51.357 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2389999999999998e-06 2024-09-16T20:49:51.357 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.357 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.357 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.357 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.357 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.357 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18882 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.357 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.271517+0000", 2024-09-16T20:49:51.357 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.022814852, 2024-09-16T20:49:51.357 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.023402373000000001, 2024-09-16T20:49:51.357 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.357 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.357 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.358 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.358 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.358 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.358 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18882 2024-09-16T20:49:51.358 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.358 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.358 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.358 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.358 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271517+0000", 2024-09-16T20:49:51.358 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.358 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.358 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.358 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.359 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271517+0000", 2024-09-16T20:49:51.359 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.359 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.359 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.359 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.359 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271517+0000", 2024-09-16T20:49:51.359 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0699999999999997e-07 2024-09-16T20:49:51.359 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.359 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.359 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.359 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271519+0000", 2024-09-16T20:49:51.359 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.95e-06 2024-09-16T20:49:51.359 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.359 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.359 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.360 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271525+0000", 2024-09-16T20:49:51.360 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2439999999999998e-06 2024-09-16T20:49:51.360 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.360 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.360 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.360 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271531+0000", 2024-09-16T20:49:51.360 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2719999999999997e-06 2024-09-16T20:49:51.360 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.360 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.360 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.360 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.360 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.360 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18913 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.361 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.274857+0000", 2024-09-16T20:49:51.361 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.019474078999999998, 2024-09-16T20:49:51.361 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.020109554000000002, 2024-09-16T20:49:51.361 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.361 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.361 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.361 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.361 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.361 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.361 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18913 2024-09-16T20:49:51.361 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.361 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.361 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.361 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.362 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274857+0000", 2024-09-16T20:49:51.362 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.362 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.362 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.362 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.362 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274857+0000", 2024-09-16T20:49:51.362 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.362 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.362 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.362 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.362 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274858+0000", 2024-09-16T20:49:51.362 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3699999999999999e-07 2024-09-16T20:49:51.362 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.362 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.362 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.363 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274859+0000", 2024-09-16T20:49:51.363 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.063e-06 2024-09-16T20:49:51.363 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.363 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.363 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.363 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274864+0000", 2024-09-16T20:49:51.363 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2789999999999998e-06 2024-09-16T20:49:51.363 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.363 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.363 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.363 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274867+0000", 2024-09-16T20:49:51.363 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4309999999999998e-06 2024-09-16T20:49:51.363 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.363 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.363 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.364 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.364 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.364 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18946 3.10 3.996ee350 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.364 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.282960+0000", 2024-09-16T20:49:51.364 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.011371503999999999, 2024-09-16T20:49:51.364 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.012058714, 2024-09-16T20:49:51.364 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.364 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.364 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.364 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.364 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.364 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.364 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18946 2024-09-16T20:49:51.364 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.365 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.365 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.365 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.365 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282960+0000", 2024-09-16T20:49:51.365 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.365 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.365 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.365 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.365 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282960+0000", 2024-09-16T20:49:51.365 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.365 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.365 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.365 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.366 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282960+0000", 2024-09-16T20:49:51.366 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2e-07 2024-09-16T20:49:51.366 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.366 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.366 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.366 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282962+0000", 2024-09-16T20:49:51.366 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1270000000000001e-06 2024-09-16T20:49:51.366 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.366 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.366 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.366 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282969+0000", 2024-09-16T20:49:51.366 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0789999999999997e-06 2024-09-16T20:49:51.366 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.366 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.367 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.367 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282974+0000", 2024-09-16T20:49:51.367 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0309999999999998e-06 2024-09-16T20:49:51.367 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.367 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.367 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.367 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.367 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.367 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18783 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.367 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.264817+0000", 2024-09-16T20:49:51.367 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.029513942000000001, 2024-09-16T20:49:51.367 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.030249508000000001, 2024-09-16T20:49:51.367 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.367 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.368 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.368 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.368 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.368 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.368 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18783 2024-09-16T20:49:51.368 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.368 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.368 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.368 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.368 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.264817+0000", 2024-09-16T20:49:51.368 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.368 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.369 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.369 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.369 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.264817+0000", 2024-09-16T20:49:51.369 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.369 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.369 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.369 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.369 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.264818+0000", 2024-09-16T20:49:51.369 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6899999999999999e-07 2024-09-16T20:49:51.369 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.369 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.369 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.369 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.264819+0000", 2024-09-16T20:49:51.370 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.097e-06 2024-09-16T20:49:51.370 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.370 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.370 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.370 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.264822+0000", 2024-09-16T20:49:51.370 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0599999999999999e-06 2024-09-16T20:49:51.370 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.370 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.370 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.370 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.264825+0000", 2024-09-16T20:49:51.370 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6249999999999999e-06 2024-09-16T20:49:51.370 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.370 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.371 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.371 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.371 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.371 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18816 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.371 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.265243+0000", 2024-09-16T20:49:51.371 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.029088831999999998, 2024-09-16T20:49:51.371 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.02987366, 2024-09-16T20:49:51.371 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.371 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.371 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.371 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.371 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.371 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.371 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18816 2024-09-16T20:49:51.372 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.372 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.372 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.372 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.372 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265243+0000", 2024-09-16T20:49:51.372 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.372 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.372 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.372 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.372 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265243+0000", 2024-09-16T20:49:51.372 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.372 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.372 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.373 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.373 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265243+0000", 2024-09-16T20:49:51.373 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3300000000000001e-07 2024-09-16T20:49:51.373 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.373 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.373 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.373 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265244+0000", 2024-09-16T20:49:51.373 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.097e-06 2024-09-16T20:49:51.373 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.373 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.373 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.373 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265248+0000", 2024-09-16T20:49:51.373 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1699999999999999e-06 2024-09-16T20:49:51.373 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.373 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.374 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.374 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265251+0000", 2024-09-16T20:49:51.374 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.621e-06 2024-09-16T20:49:51.374 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.374 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.374 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.374 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.374 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.374 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18848 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.374 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.265651+0000", 2024-09-16T20:49:51.374 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.028680039000000001, 2024-09-16T20:49:51.374 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.029492059000000001, 2024-09-16T20:49:51.374 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.374 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.374 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.374 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.375 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.375 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.375 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18848 2024-09-16T20:49:51.375 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.375 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.375 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.375 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.375 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265651+0000", 2024-09-16T20:49:51.375 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.375 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.375 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.375 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.375 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265651+0000", 2024-09-16T20:49:51.375 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.375 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.376 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.376 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.376 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265652+0000", 2024-09-16T20:49:51.376 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1e-07 2024-09-16T20:49:51.376 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.376 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.376 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.376 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265653+0000", 2024-09-16T20:49:51.376 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.029e-06 2024-09-16T20:49:51.376 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.376 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.376 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.376 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265657+0000", 2024-09-16T20:49:51.376 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.941e-06 2024-09-16T20:49:51.377 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.377 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.377 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.377 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265659+0000", 2024-09-16T20:49:51.377 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.712e-06 2024-09-16T20:49:51.377 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.377 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.377 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.377 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.377 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.377 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18883 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.377 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.271538+0000", 2024-09-16T20:49:51.377 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.022793731000000001, 2024-09-16T20:49:51.377 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.023629376000000001, 2024-09-16T20:49:51.378 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.378 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.378 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.378 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.378 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.378 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.378 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18883 2024-09-16T20:49:51.378 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.378 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.378 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.378 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.378 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271538+0000", 2024-09-16T20:49:51.378 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.378 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.379 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.379 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.379 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271538+0000", 2024-09-16T20:49:51.379 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.379 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.379 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.379 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.379 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271538+0000", 2024-09-16T20:49:51.379 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9299999999999999e-07 2024-09-16T20:49:51.379 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.379 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.379 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.379 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271540+0000", 2024-09-16T20:49:51.380 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9640000000000002e-06 2024-09-16T20:49:51.380 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.380 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.380 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.380 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271546+0000", 2024-09-16T20:49:51.380 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9349999999999999e-06 2024-09-16T20:49:51.380 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.380 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.380 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.380 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271551+0000", 2024-09-16T20:49:51.380 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0860000000000001e-06 2024-09-16T20:49:51.380 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.380 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.381 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.381 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.381 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.381 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18914 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.381 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.274871+0000", 2024-09-16T20:49:51.381 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.019460892, 2024-09-16T20:49:51.381 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.020319803000000001, 2024-09-16T20:49:51.381 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.381 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.381 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.381 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.381 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.381 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.382 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18914 2024-09-16T20:49:51.382 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.382 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.382 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.382 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.382 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274871+0000", 2024-09-16T20:49:51.382 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.382 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.382 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.382 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.382 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274871+0000", 2024-09-16T20:49:51.382 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.382 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.383 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.383 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.383 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274871+0000", 2024-09-16T20:49:51.383 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3300000000000001e-07 2024-09-16T20:49:51.383 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.383 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.383 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.383 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274872+0000", 2024-09-16T20:49:51.383 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.9999999999999995e-07 2024-09-16T20:49:51.383 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.383 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.383 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.383 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274875+0000", 2024-09-16T20:49:51.383 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.162e-06 2024-09-16T20:49:51.384 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.384 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.384 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.384 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274878+0000", 2024-09-16T20:49:51.384 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3569999999999998e-06 2024-09-16T20:49:51.384 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.384 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.384 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.384 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.384 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.384 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18947 3.10 3.996ee350 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.384 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.282980+0000", 2024-09-16T20:49:51.385 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.011351188999999999, 2024-09-16T20:49:51.385 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.012242095, 2024-09-16T20:49:51.385 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.385 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.385 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.385 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.385 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.385 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.385 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18947 2024-09-16T20:49:51.385 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.385 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.385 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.385 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.385 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282980+0000", 2024-09-16T20:49:51.386 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.386 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.386 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.386 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.386 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282980+0000", 2024-09-16T20:49:51.386 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.386 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.386 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.386 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.386 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282981+0000", 2024-09-16T20:49:51.386 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.84e-07 2024-09-16T20:49:51.386 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.386 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.386 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.387 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282982+0000", 2024-09-16T20:49:51.387 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7719999999999999e-06 2024-09-16T20:49:51.387 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.387 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.387 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.387 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282988+0000", 2024-09-16T20:49:51.387 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9649999999999996e-06 2024-09-16T20:49:51.387 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.387 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.387 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.387 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282998+0000", 2024-09-16T20:49:51.387 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0003999999999999e-05 2024-09-16T20:49:51.387 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.388 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.388 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.388 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.388 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.388 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18784 3.10 3:0ac77699:::rbd_data.110ea93691bb.0000000000000052:23 [read 920576~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.388 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.264828+0000", 2024-09-16T20:49:51.388 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.029503728, 2024-09-16T20:49:51.388 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.030447485999999999, 2024-09-16T20:49:51.388 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.388 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.388 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:49:51.388 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.388 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.389 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.389 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18784 2024-09-16T20:49:51.389 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.389 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.389 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.389 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.389 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.264828+0000", 2024-09-16T20:49:51.389 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.389 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.389 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.389 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.389 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.264828+0000", 2024-09-16T20:49:51.389 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.389 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.390 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.390 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.390 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.264828+0000", 2024-09-16T20:49:51.390 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1500000000000001e-07 2024-09-16T20:49:51.390 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.390 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.390 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.390 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.264829+0000", 2024-09-16T20:49:51.390 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0499999999999999e-06 2024-09-16T20:49:51.390 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.390 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.390 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.390 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.264832+0000", 2024-09-16T20:49:51.390 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0350000000000002e-06 2024-09-16T20:49:51.390 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.390 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.391 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.391 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.264834+0000", 2024-09-16T20:49:51.391 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.4899999999999999e-06 2024-09-16T20:49:51.391 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.391 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.391 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:51.391 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.295230+0000", 2024-09-16T20:49:51.391 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.030395421999999998 2024-09-16T20:49:51.391 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.391 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.391 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.391 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.391 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.391 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18817 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.391 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.265254+0000", 2024-09-16T20:49:51.392 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.029077671999999999, 2024-09-16T20:49:51.392 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.030076105999999998, 2024-09-16T20:49:51.392 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.392 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.392 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.392 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.392 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.392 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.392 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18817 2024-09-16T20:49:51.392 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.392 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.392 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.392 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.393 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265254+0000", 2024-09-16T20:49:51.393 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.393 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.393 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.393 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.393 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265254+0000", 2024-09-16T20:49:51.393 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.393 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.393 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.393 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.393 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265254+0000", 2024-09-16T20:49:51.393 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1299999999999999e-07 2024-09-16T20:49:51.393 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.393 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.393 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.394 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265255+0000", 2024-09-16T20:49:51.394 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0380000000000001e-06 2024-09-16T20:49:51.394 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.394 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.394 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.394 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265258+0000", 2024-09-16T20:49:51.394 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9189999999999999e-06 2024-09-16T20:49:51.394 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.394 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.394 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.394 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265261+0000", 2024-09-16T20:49:51.394 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.633e-06 2024-09-16T20:49:51.394 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.394 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.394 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.395 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.395 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.395 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18849 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.395 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.265663+0000", 2024-09-16T20:49:51.395 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.028668263999999999, 2024-09-16T20:49:51.395 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.029715418, 2024-09-16T20:49:51.395 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.395 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.395 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.395 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.395 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.395 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.395 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18849 2024-09-16T20:49:51.395 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.395 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.396 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.396 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.396 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265663+0000", 2024-09-16T20:49:51.396 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.396 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.396 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.396 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.396 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265663+0000", 2024-09-16T20:49:51.396 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.396 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.396 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.396 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.396 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265663+0000", 2024-09-16T20:49:51.396 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2399999999999999e-07 2024-09-16T20:49:51.397 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.397 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.397 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.397 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265665+0000", 2024-09-16T20:49:51.397 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1349999999999999e-06 2024-09-16T20:49:51.397 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.397 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.397 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.397 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265667+0000", 2024-09-16T20:49:51.397 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9569999999999998e-06 2024-09-16T20:49:51.397 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.397 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.397 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.397 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265670+0000", 2024-09-16T20:49:51.398 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6740000000000001e-06 2024-09-16T20:49:51.398 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.398 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.398 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.398 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.398 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.398 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18884 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.398 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.271558+0000", 2024-09-16T20:49:51.398 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.022773801, 2024-09-16T20:49:51.398 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.023862458999999999, 2024-09-16T20:49:51.398 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.398 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.398 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.399 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.399 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.399 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.399 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18884 2024-09-16T20:49:51.399 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.399 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.399 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.399 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.399 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271558+0000", 2024-09-16T20:49:51.399 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.399 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.399 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.399 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.400 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271558+0000", 2024-09-16T20:49:51.400 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.400 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.400 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.400 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.400 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271558+0000", 2024-09-16T20:49:51.400 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8799999999999998e-07 2024-09-16T20:49:51.400 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.400 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.400 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.400 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271560+0000", 2024-09-16T20:49:51.400 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9180000000000001e-06 2024-09-16T20:49:51.400 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.400 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.401 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.401 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271566+0000", 2024-09-16T20:49:51.401 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1709999999999999e-06 2024-09-16T20:49:51.401 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.401 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.401 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.401 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271571+0000", 2024-09-16T20:49:51.401 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7210000000000002e-06 2024-09-16T20:49:51.401 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.401 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.401 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.401 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.401 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.402 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18915 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.402 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.274881+0000", 2024-09-16T20:49:51.402 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.019450081000000001, 2024-09-16T20:49:51.402 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.020563387999999998, 2024-09-16T20:49:51.402 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.402 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.402 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.402 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.402 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.402 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.402 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18915 2024-09-16T20:49:51.402 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.402 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.402 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.403 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.403 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274881+0000", 2024-09-16T20:49:51.403 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.403 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.403 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.403 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.403 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274881+0000", 2024-09-16T20:49:51.403 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.403 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.403 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.403 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.403 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274882+0000", 2024-09-16T20:49:51.403 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9099999999999999e-07 2024-09-16T20:49:51.403 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.404 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.404 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.404 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274883+0000", 2024-09-16T20:49:51.404 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0699999999999999e-06 2024-09-16T20:49:51.404 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.404 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.404 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.404 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274886+0000", 2024-09-16T20:49:51.404 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1109999999999999e-06 2024-09-16T20:49:51.404 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.404 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.404 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.405 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274889+0000", 2024-09-16T20:49:51.405 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7980000000000001e-06 2024-09-16T20:49:51.405 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.405 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.405 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.405 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.405 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.405 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18948 3.10 3.996ee350 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.405 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.283029+0000", 2024-09-16T20:49:51.405 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.011302748, 2024-09-16T20:49:51.405 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.012439561, 2024-09-16T20:49:51.405 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.405 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.405 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.405 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.406 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.406 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.406 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18948 2024-09-16T20:49:51.406 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.406 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.406 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.406 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.406 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283029+0000", 2024-09-16T20:49:51.406 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.406 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.406 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.406 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.406 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283029+0000", 2024-09-16T20:49:51.406 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.406 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.407 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.407 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.407 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283029+0000", 2024-09-16T20:49:51.407 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7399999999999998e-07 2024-09-16T20:49:51.407 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.407 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.407 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.407 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283031+0000", 2024-09-16T20:49:51.407 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3259999999999998e-06 2024-09-16T20:49:51.407 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.407 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.407 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.407 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283042+0000", 2024-09-16T20:49:51.407 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0693000000000001e-05 2024-09-16T20:49:51.407 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.408 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.408 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.408 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283051+0000", 2024-09-16T20:49:51.408 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.3639999999999999e-06 2024-09-16T20:49:51.408 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.408 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.408 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.408 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.408 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.408 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18785 3.10 3:0ac77699:::rbd_data.110ea93691bb.0000000000000052:23 [read 925696~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.408 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.264837+0000", 2024-09-16T20:49:51.408 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.029493954999999999, 2024-09-16T20:49:51.408 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.030660364999999998, 2024-09-16T20:49:51.409 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.409 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.409 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:51.409 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.409 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.409 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.409 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18785 2024-09-16T20:49:51.409 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.409 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.409 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.409 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.409 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.264837+0000", 2024-09-16T20:49:51.409 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.409 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.409 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.410 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.410 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.264837+0000", 2024-09-16T20:49:51.410 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.410 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.410 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.410 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.410 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.264838+0000", 2024-09-16T20:49:51.410 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1199999999999999e-07 2024-09-16T20:49:51.410 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.410 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.410 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.410 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.264839+0000", 2024-09-16T20:49:51.410 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.015e-06 2024-09-16T20:49:51.411 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.411 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.411 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.411 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.264846+0000", 2024-09-16T20:49:51.411 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9999999999999999e-06 2024-09-16T20:49:51.411 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.411 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.411 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.411 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.264849+0000", 2024-09-16T20:49:51.411 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.004e-06 2024-09-16T20:49:51.411 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.411 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.411 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:51.411 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.295420+0000", 2024-09-16T20:49:51.412 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.030571672000000001 2024-09-16T20:49:51.412 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.412 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.412 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:51.412 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.295460+0000", 2024-09-16T20:49:51.412 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9276000000000001e-05 2024-09-16T20:49:51.412 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.412 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.412 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:51.412 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.295498+0000", 2024-09-16T20:49:51.412 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8186000000000003e-05 2024-09-16T20:49:51.412 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.412 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.412 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.412 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.413 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.413 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18818 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.413 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.265264+0000", 2024-09-16T20:49:51.413 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.029067863999999999, 2024-09-16T20:49:51.413 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.030298449000000002, 2024-09-16T20:49:51.413 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.413 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.413 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.413 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.413 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.413 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.413 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18818 2024-09-16T20:49:51.413 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.413 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.414 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.414 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.414 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265264+0000", 2024-09-16T20:49:51.414 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.414 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.414 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.414 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.414 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265264+0000", 2024-09-16T20:49:51.414 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.414 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.414 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.414 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.414 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265264+0000", 2024-09-16T20:49:51.415 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.34e-07 2024-09-16T20:49:51.415 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.415 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.415 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.415 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265267+0000", 2024-09-16T20:49:51.415 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0800000000000002e-06 2024-09-16T20:49:51.415 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.415 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.415 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.415 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265270+0000", 2024-09-16T20:49:51.415 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2880000000000001e-06 2024-09-16T20:49:51.415 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.415 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.415 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.416 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265273+0000", 2024-09-16T20:49:51.416 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7319999999999998e-06 2024-09-16T20:49:51.416 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.416 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.416 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.416 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.416 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.416 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18850 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.416 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.265673+0000", 2024-09-16T20:49:51.416 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.028658205999999999, 2024-09-16T20:49:51.416 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.029937027000000001, 2024-09-16T20:49:51.416 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.416 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.416 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.417 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.417 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.417 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.417 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18850 2024-09-16T20:49:51.417 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.417 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.417 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.417 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.417 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265673+0000", 2024-09-16T20:49:51.417 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.417 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.417 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.417 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.417 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265673+0000", 2024-09-16T20:49:51.417 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.418 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.418 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.418 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.418 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265673+0000", 2024-09-16T20:49:51.418 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.16e-07 2024-09-16T20:49:51.418 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.418 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.418 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.418 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265674+0000", 2024-09-16T20:49:51.418 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.7000000000000003e-07 2024-09-16T20:49:51.418 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.418 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.418 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.418 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265677+0000", 2024-09-16T20:49:51.419 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9040000000000001e-06 2024-09-16T20:49:51.419 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.419 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.419 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.419 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265683+0000", 2024-09-16T20:49:51.419 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2190000000000001e-06 2024-09-16T20:49:51.419 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.419 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.419 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.419 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.419 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.419 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18885 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.419 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.271577+0000", 2024-09-16T20:49:51.419 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.022753981, 2024-09-16T20:49:51.420 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.024081201999999999, 2024-09-16T20:49:51.420 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.420 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.420 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.420 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.420 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.420 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.420 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18885 2024-09-16T20:49:51.420 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.420 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.420 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.420 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.420 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271577+0000", 2024-09-16T20:49:51.420 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.421 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.421 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.421 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.421 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271577+0000", 2024-09-16T20:49:51.421 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.421 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.421 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.421 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.421 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271578+0000", 2024-09-16T20:49:51.421 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0600000000000001e-07 2024-09-16T20:49:51.421 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.421 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.421 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.421 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271580+0000", 2024-09-16T20:49:51.422 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.843e-06 2024-09-16T20:49:51.422 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.422 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.422 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.422 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271586+0000", 2024-09-16T20:49:51.422 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3559999999999998e-06 2024-09-16T20:49:51.422 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.422 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.422 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.422 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271591+0000", 2024-09-16T20:49:51.422 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6419999999999998e-06 2024-09-16T20:49:51.422 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.422 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.422 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.422 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.423 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.423 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18949 3.10 3.996ee350 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.423 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.283062+0000", 2024-09-16T20:49:51.423 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.011269491, 2024-09-16T20:49:51.423 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.012652778999999999, 2024-09-16T20:49:51.423 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.423 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.423 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.423 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.423 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.423 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.423 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18949 2024-09-16T20:49:51.423 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.424 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.424 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.424 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.424 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283062+0000", 2024-09-16T20:49:51.424 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.424 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.424 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.424 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.424 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283062+0000", 2024-09-16T20:49:51.424 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.424 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.424 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.424 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.424 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283062+0000", 2024-09-16T20:49:51.425 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4199999999999996e-07 2024-09-16T20:49:51.425 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.425 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.425 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.425 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283065+0000", 2024-09-16T20:49:51.425 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.4229999999999999e-06 2024-09-16T20:49:51.425 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.425 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.425 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.425 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283072+0000", 2024-09-16T20:49:51.425 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6579999999999996e-06 2024-09-16T20:49:51.425 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.425 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.425 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.426 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283077+0000", 2024-09-16T20:49:51.426 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4330000000000003e-06 2024-09-16T20:49:51.426 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.426 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.426 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.426 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.426 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.426 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18819 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.426 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.265276+0000", 2024-09-16T20:49:51.426 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.029055279999999999, 2024-09-16T20:49:51.426 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.030489241, 2024-09-16T20:49:51.426 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.426 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.426 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.427 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.427 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.427 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.427 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18819 2024-09-16T20:49:51.427 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.427 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.427 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.427 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.427 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265276+0000", 2024-09-16T20:49:51.427 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.427 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.427 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.427 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.427 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265276+0000", 2024-09-16T20:49:51.428 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.428 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.428 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.428 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.428 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265276+0000", 2024-09-16T20:49:51.428 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1899999999999999e-07 2024-09-16T20:49:51.428 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.428 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.428 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.428 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265277+0000", 2024-09-16T20:49:51.428 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0070000000000001e-06 2024-09-16T20:49:51.428 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.428 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.428 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.429 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265280+0000", 2024-09-16T20:49:51.429 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9550000000000001e-06 2024-09-16T20:49:51.429 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.429 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.429 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.429 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265283+0000", 2024-09-16T20:49:51.429 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.4899999999999999e-06 2024-09-16T20:49:51.429 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.429 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.429 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.429 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.429 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.429 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18851 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.429 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.265692+0000", 2024-09-16T20:49:51.430 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.028639228999999999, 2024-09-16T20:49:51.430 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.030121776999999999, 2024-09-16T20:49:51.430 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.430 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.430 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.430 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.430 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.430 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.430 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18851 2024-09-16T20:49:51.430 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.430 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.430 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.430 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.430 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265692+0000", 2024-09-16T20:49:51.430 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.431 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.431 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.431 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.431 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265692+0000", 2024-09-16T20:49:51.431 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.431 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.431 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.431 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.431 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265692+0000", 2024-09-16T20:49:51.431 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5400000000000002e-07 2024-09-16T20:49:51.431 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.431 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.431 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.431 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265694+0000", 2024-09-16T20:49:51.431 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.268e-06 2024-09-16T20:49:51.431 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.432 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.432 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.432 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265697+0000", 2024-09-16T20:49:51.432 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3100000000000001e-06 2024-09-16T20:49:51.432 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.432 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.432 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.432 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265700+0000", 2024-09-16T20:49:51.432 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0089999999999999e-06 2024-09-16T20:49:51.432 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.432 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.432 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.432 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.432 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.433 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18886 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.433 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.271597+0000", 2024-09-16T20:49:51.433 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.022734312, 2024-09-16T20:49:51.433 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.024265881, 2024-09-16T20:49:51.433 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.433 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.433 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.433 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.433 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.433 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.433 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18886 2024-09-16T20:49:51.433 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.433 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.433 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.434 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.434 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271597+0000", 2024-09-16T20:49:51.434 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.434 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.434 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.434 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.434 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271597+0000", 2024-09-16T20:49:51.434 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.434 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.434 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.434 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.434 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271597+0000", 2024-09-16T20:49:51.434 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5100000000000001e-07 2024-09-16T20:49:51.434 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.434 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.434 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.435 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271599+0000", 2024-09-16T20:49:51.435 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9379999999999999e-06 2024-09-16T20:49:51.435 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.435 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.435 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.435 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271605+0000", 2024-09-16T20:49:51.435 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8109999999999999e-06 2024-09-16T20:49:51.435 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.435 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.435 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.435 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271610+0000", 2024-09-16T20:49:51.435 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9459999999999997e-06 2024-09-16T20:49:51.435 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.435 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.436 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.436 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.436 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.436 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18950 3.10 3.996ee350 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.436 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.283084+0000", 2024-09-16T20:49:51.436 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.011247073999999999, 2024-09-16T20:49:51.436 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.012821129000000001, 2024-09-16T20:49:51.436 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.436 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.436 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.436 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.436 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.436 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.437 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18950 2024-09-16T20:49:51.437 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.437 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.437 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.437 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.437 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283084+0000", 2024-09-16T20:49:51.437 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.437 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.437 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.437 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.437 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283084+0000", 2024-09-16T20:49:51.437 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.437 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.437 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.438 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.438 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283085+0000", 2024-09-16T20:49:51.438 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.34e-07 2024-09-16T20:49:51.438 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.438 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.438 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.438 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283087+0000", 2024-09-16T20:49:51.438 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1550000000000001e-06 2024-09-16T20:49:51.438 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.438 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.438 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.438 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283093+0000", 2024-09-16T20:49:51.438 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5300000000000002e-06 2024-09-16T20:49:51.438 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.438 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.439 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.439 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283108+0000", 2024-09-16T20:49:51.439 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4637000000000001e-05 2024-09-16T20:49:51.439 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.439 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.439 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.439 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.439 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.439 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18820 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.439 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.265286+0000", 2024-09-16T20:49:51.439 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.029045620000000001, 2024-09-16T20:49:51.439 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.030644719000000001, 2024-09-16T20:49:51.439 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.439 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.439 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.440 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.440 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.440 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.440 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18820 2024-09-16T20:49:51.440 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.440 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.440 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.440 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.440 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265286+0000", 2024-09-16T20:49:51.440 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.440 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.440 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.440 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.440 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265286+0000", 2024-09-16T20:49:51.441 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.441 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.441 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.441 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.441 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265286+0000", 2024-09-16T20:49:51.441 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2100000000000001e-07 2024-09-16T20:49:51.441 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.441 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.441 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.441 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265287+0000", 2024-09-16T20:49:51.441 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.1500000000000003e-07 2024-09-16T20:49:51.441 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.441 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.441 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.441 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265290+0000", 2024-09-16T20:49:51.442 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.886e-06 2024-09-16T20:49:51.442 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.442 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.442 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.442 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265292+0000", 2024-09-16T20:49:51.442 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.525e-06 2024-09-16T20:49:51.442 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.442 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.442 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.442 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.442 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.442 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18852 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.442 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.265703+0000", 2024-09-16T20:49:51.442 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.028628147, 2024-09-16T20:49:51.443 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.030251093, 2024-09-16T20:49:51.443 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.443 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.443 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.443 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.443 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.443 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.443 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18852 2024-09-16T20:49:51.443 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.443 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.443 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.443 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.443 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265703+0000", 2024-09-16T20:49:51.443 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.443 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.444 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.444 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.444 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265703+0000", 2024-09-16T20:49:51.444 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.444 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.444 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.444 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.444 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265703+0000", 2024-09-16T20:49:51.444 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.28e-07 2024-09-16T20:49:51.444 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.444 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.444 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.444 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265705+0000", 2024-09-16T20:49:51.444 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.029e-06 2024-09-16T20:49:51.445 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.445 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.445 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.445 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265708+0000", 2024-09-16T20:49:51.445 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.005e-06 2024-09-16T20:49:51.445 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.445 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.445 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.445 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265710+0000", 2024-09-16T20:49:51.445 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5160000000000001e-06 2024-09-16T20:49:51.445 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.445 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.445 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.445 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.446 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.446 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18887 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.446 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.271617+0000", 2024-09-16T20:49:51.446 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.022714681, 2024-09-16T20:49:51.446 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.024370267000000001, 2024-09-16T20:49:51.446 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.446 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.446 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.446 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.446 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.446 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.446 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18887 2024-09-16T20:49:51.446 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.446 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.446 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.446 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.447 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271617+0000", 2024-09-16T20:49:51.447 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.447 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.447 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.447 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.447 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271617+0000", 2024-09-16T20:49:51.447 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.447 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.447 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.447 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.447 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271617+0000", 2024-09-16T20:49:51.447 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0999999999999999e-07 2024-09-16T20:49:51.447 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.447 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.447 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.448 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271619+0000", 2024-09-16T20:49:51.448 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7990000000000001e-06 2024-09-16T20:49:51.448 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.448 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.448 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.448 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271625+0000", 2024-09-16T20:49:51.448 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1199999999999999e-06 2024-09-16T20:49:51.448 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.448 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.448 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.448 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271630+0000", 2024-09-16T20:49:51.448 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.977e-06 2024-09-16T20:49:51.448 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.448 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.448 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.449 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.449 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.449 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18919 3.10 3.996ee350 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.449 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.274926+0000", 2024-09-16T20:49:51.449 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.019405848, 2024-09-16T20:49:51.449 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.021108656999999999, 2024-09-16T20:49:51.449 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.449 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.449 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.449 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.449 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.449 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.449 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18919 2024-09-16T20:49:51.449 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.450 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.450 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.450 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.450 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274926+0000", 2024-09-16T20:49:51.450 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.450 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.450 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.450 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.450 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274926+0000", 2024-09-16T20:49:51.450 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.450 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.450 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.450 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.450 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277287+0000", 2024-09-16T20:49:51.450 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0023613850000000001 2024-09-16T20:49:51.451 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.451 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.451 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.451 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277291+0000", 2024-09-16T20:49:51.451 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7670000000000001e-06 2024-09-16T20:49:51.451 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.451 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.451 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.451 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277301+0000", 2024-09-16T20:49:51.451 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0373999999999999e-05 2024-09-16T20:49:51.451 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.451 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.451 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.451 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277306+0000", 2024-09-16T20:49:51.451 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4079999999999997e-06 2024-09-16T20:49:51.452 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.452 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.452 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.452 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.452 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.452 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18951 3.10 3.996ee350 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.452 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.283116+0000", 2024-09-16T20:49:51.452 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.011214956999999999, 2024-09-16T20:49:51.452 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.012966248999999999, 2024-09-16T20:49:51.452 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.452 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.452 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.452 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.452 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.452 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.453 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18951 2024-09-16T20:49:51.453 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.453 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.453 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.453 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.453 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283116+0000", 2024-09-16T20:49:51.453 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.453 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.453 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.453 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.453 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283116+0000", 2024-09-16T20:49:51.453 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.453 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.453 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.453 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.454 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283117+0000", 2024-09-16T20:49:51.454 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.89e-07 2024-09-16T20:49:51.454 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.454 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.454 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.454 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283119+0000", 2024-09-16T20:49:51.454 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2910000000000002e-06 2024-09-16T20:49:51.454 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.454 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.454 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.454 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283126+0000", 2024-09-16T20:49:51.454 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8959999999999997e-06 2024-09-16T20:49:51.454 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.454 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.455 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.455 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283131+0000", 2024-09-16T20:49:51.455 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3600000000000004e-06 2024-09-16T20:49:51.455 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.455 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.455 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.455 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.455 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.455 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18821 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.455 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.265295+0000", 2024-09-16T20:49:51.455 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.029036038, 2024-09-16T20:49:51.455 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.030840721000000001, 2024-09-16T20:49:51.455 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.455 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.455 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.456 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.456 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.456 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.456 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18821 2024-09-16T20:49:51.456 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.456 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.456 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.456 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.456 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265295+0000", 2024-09-16T20:49:51.456 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.456 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.456 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.456 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.456 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265295+0000", 2024-09-16T20:49:51.457 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.457 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.457 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.457 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.457 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265296+0000", 2024-09-16T20:49:51.457 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5899999999999998e-07 2024-09-16T20:49:51.457 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.457 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.457 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.457 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265297+0000", 2024-09-16T20:49:51.457 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.066e-06 2024-09-16T20:49:51.457 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.457 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.457 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.458 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265300+0000", 2024-09-16T20:49:51.458 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.914e-06 2024-09-16T20:49:51.458 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.458 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.458 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.458 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265304+0000", 2024-09-16T20:49:51.458 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9899999999999999e-06 2024-09-16T20:49:51.458 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.458 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.458 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.458 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.458 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.459 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18853 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.459 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.265713+0000", 2024-09-16T20:49:51.459 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.028618344, 2024-09-16T20:49:51.459 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.030470349000000001, 2024-09-16T20:49:51.459 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.459 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.459 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.459 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.459 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.459 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.459 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18853 2024-09-16T20:49:51.459 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.459 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.459 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.459 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.460 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265713+0000", 2024-09-16T20:49:51.460 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.460 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.460 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.460 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.460 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265713+0000", 2024-09-16T20:49:51.460 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.460 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.460 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.460 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.460 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265713+0000", 2024-09-16T20:49:51.460 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2700000000000001e-07 2024-09-16T20:49:51.460 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.460 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.460 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.461 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265714+0000", 2024-09-16T20:49:51.461 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.102e-06 2024-09-16T20:49:51.461 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.461 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.461 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.461 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265721+0000", 2024-09-16T20:49:51.461 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5810000000000002e-06 2024-09-16T20:49:51.461 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.461 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.461 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.461 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265725+0000", 2024-09-16T20:49:51.461 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.664e-06 2024-09-16T20:49:51.461 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.461 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.461 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.462 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.462 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.462 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18888 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.462 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.271636+0000", 2024-09-16T20:49:51.462 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.022695235000000001, 2024-09-16T20:49:51.462 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.024577588000000001, 2024-09-16T20:49:51.462 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.462 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.462 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.462 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.462 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.462 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.462 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18888 2024-09-16T20:49:51.462 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.463 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.463 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.463 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.463 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271636+0000", 2024-09-16T20:49:51.463 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.463 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.463 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.463 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.463 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271636+0000", 2024-09-16T20:49:51.463 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.463 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.463 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.463 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.463 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271637+0000", 2024-09-16T20:49:51.463 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5100000000000001e-07 2024-09-16T20:49:51.464 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.464 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.464 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.464 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271638+0000", 2024-09-16T20:49:51.464 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8509999999999999e-06 2024-09-16T20:49:51.464 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.464 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.464 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.464 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271644+0000", 2024-09-16T20:49:51.464 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8000000000000004e-06 2024-09-16T20:49:51.464 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.464 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.464 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.464 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271649+0000", 2024-09-16T20:49:51.464 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7079999999999996e-06 2024-09-16T20:49:51.465 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.465 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.465 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.465 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.465 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.465 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18920 3.10 3.996ee350 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.465 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.277326+0000", 2024-09-16T20:49:51.465 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.017005437000000002, 2024-09-16T20:49:51.465 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.018913803999999999, 2024-09-16T20:49:51.465 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.465 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.465 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.465 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.465 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.466 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.466 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18920 2024-09-16T20:49:51.466 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.466 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.466 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.466 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.466 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277326+0000", 2024-09-16T20:49:51.466 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.466 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.466 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.466 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.466 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277326+0000", 2024-09-16T20:49:51.466 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.466 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.467 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.467 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.467 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277327+0000", 2024-09-16T20:49:51.467 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5400000000000001e-07 2024-09-16T20:49:51.467 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.467 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.467 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.467 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277329+0000", 2024-09-16T20:49:51.467 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.875e-06 2024-09-16T20:49:51.467 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.467 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.467 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.467 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277336+0000", 2024-09-16T20:49:51.467 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0400000000000004e-06 2024-09-16T20:49:51.467 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.468 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.468 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.468 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277342+0000", 2024-09-16T20:49:51.468 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8239999999999996e-06 2024-09-16T20:49:51.468 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.468 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.468 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.468 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.468 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.468 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18952 3.10 3.996ee350 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.468 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.283165+0000", 2024-09-16T20:49:51.468 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.011165951, 2024-09-16T20:49:51.468 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.013098186, 2024-09-16T20:49:51.469 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.469 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.469 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.469 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.469 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.469 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.469 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18952 2024-09-16T20:49:51.469 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.469 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.469 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.469 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.469 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283165+0000", 2024-09-16T20:49:51.469 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.469 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.469 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.470 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.470 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283165+0000", 2024-09-16T20:49:51.470 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.470 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.470 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.470 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.470 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283166+0000", 2024-09-16T20:49:51.470 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2900000000000003e-07 2024-09-16T20:49:51.470 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.470 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.470 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.470 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283169+0000", 2024-09-16T20:49:51.470 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.4540000000000001e-06 2024-09-16T20:49:51.470 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.470 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.470 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.471 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283184+0000", 2024-09-16T20:49:51.471 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5216e-05 2024-09-16T20:49:51.471 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.471 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.471 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.471 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283190+0000", 2024-09-16T20:49:51.471 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6229999999999999e-06 2024-09-16T20:49:51.471 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.471 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.471 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.471 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.471 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.471 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18822 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.471 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.265307+0000", 2024-09-16T20:49:51.471 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.029024444, 2024-09-16T20:49:51.472 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.030987243000000001, 2024-09-16T20:49:51.472 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.472 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.472 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.472 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.472 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.472 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.472 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18822 2024-09-16T20:49:51.472 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.472 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.472 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.472 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.472 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265307+0000", 2024-09-16T20:49:51.472 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.472 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.473 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.473 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.473 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265307+0000", 2024-09-16T20:49:51.473 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.473 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.473 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.473 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.473 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265307+0000", 2024-09-16T20:49:51.473 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3200000000000001e-07 2024-09-16T20:49:51.473 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.473 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.473 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.473 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265308+0000", 2024-09-16T20:49:51.473 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0300000000000001e-06 2024-09-16T20:49:51.473 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.473 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.474 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.474 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265311+0000", 2024-09-16T20:49:51.474 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8669999999999998e-06 2024-09-16T20:49:51.474 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.474 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.474 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.474 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265314+0000", 2024-09-16T20:49:51.474 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6139999999999999e-06 2024-09-16T20:49:51.474 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.474 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.474 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.474 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.474 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.474 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18854 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.474 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.265728+0000", 2024-09-16T20:49:51.474 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.028603364999999999, 2024-09-16T20:49:51.475 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.030614953, 2024-09-16T20:49:51.475 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.475 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.475 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.475 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.475 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.475 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.475 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18854 2024-09-16T20:49:51.475 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.475 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.475 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.475 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.475 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265728+0000", 2024-09-16T20:49:51.475 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.475 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.476 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.476 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.476 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265728+0000", 2024-09-16T20:49:51.476 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.476 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.476 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.476 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.476 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265728+0000", 2024-09-16T20:49:51.476 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2399999999999999e-07 2024-09-16T20:49:51.476 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.476 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.476 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.476 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265732+0000", 2024-09-16T20:49:51.476 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4419999999999998e-06 2024-09-16T20:49:51.476 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.477 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.477 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.477 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265735+0000", 2024-09-16T20:49:51.477 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.258e-06 2024-09-16T20:49:51.477 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.477 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.477 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.477 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265738+0000", 2024-09-16T20:49:51.477 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7829999999999999e-06 2024-09-16T20:49:51.477 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.477 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.477 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.477 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.477 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.478 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18889 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.478 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.271658+0000", 2024-09-16T20:49:51.478 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.022673544, 2024-09-16T20:49:51.478 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.024733633000000001, 2024-09-16T20:49:51.478 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.478 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.478 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.478 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.478 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.478 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.478 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18889 2024-09-16T20:49:51.478 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.478 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.478 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.479 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.479 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271658+0000", 2024-09-16T20:49:51.479 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.479 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.479 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.479 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.479 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271658+0000", 2024-09-16T20:49:51.479 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.479 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.479 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.479 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.479 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271658+0000", 2024-09-16T20:49:51.479 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0200000000000003e-07 2024-09-16T20:49:51.480 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.480 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.480 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.480 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271660+0000", 2024-09-16T20:49:51.480 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0150000000000002e-06 2024-09-16T20:49:51.480 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.480 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.480 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.480 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271667+0000", 2024-09-16T20:49:51.480 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3130000000000004e-06 2024-09-16T20:49:51.480 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.480 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.480 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.480 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271672+0000", 2024-09-16T20:49:51.481 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9409999999999998e-06 2024-09-16T20:49:51.481 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.481 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.481 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.481 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.481 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.481 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18921 3.10 3.996ee350 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.481 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.277351+0000", 2024-09-16T20:49:51.481 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.016979957, 2024-09-16T20:49:51.481 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.019087929999999999, 2024-09-16T20:49:51.481 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.481 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.481 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.482 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.482 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.482 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.482 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18921 2024-09-16T20:49:51.482 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.482 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.482 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.482 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.482 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277351+0000", 2024-09-16T20:49:51.482 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.482 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.482 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.482 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.483 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277351+0000", 2024-09-16T20:49:51.483 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.483 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.483 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.483 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.483 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277352+0000", 2024-09-16T20:49:51.483 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8400000000000005e-07 2024-09-16T20:49:51.483 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.483 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.483 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.483 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277354+0000", 2024-09-16T20:49:51.483 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.21e-06 2024-09-16T20:49:51.483 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.483 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.483 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.484 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277361+0000", 2024-09-16T20:49:51.484 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6250000000000001e-06 2024-09-16T20:49:51.484 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.484 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.484 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.484 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277366+0000", 2024-09-16T20:49:51.484 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.451e-06 2024-09-16T20:49:51.484 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.484 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.484 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.484 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.484 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.484 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18953 3.10 3.996ee350 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.485 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.283202+0000", 2024-09-16T20:49:51.485 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.011129780000000001, 2024-09-16T20:49:51.485 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.013284486999999999, 2024-09-16T20:49:51.485 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.485 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.485 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.485 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.485 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.485 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.485 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18953 2024-09-16T20:49:51.485 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.485 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.485 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.486 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.486 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283202+0000", 2024-09-16T20:49:51.486 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.486 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.486 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.486 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.486 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283202+0000", 2024-09-16T20:49:51.486 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.486 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.486 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.486 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.486 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283202+0000", 2024-09-16T20:49:51.486 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2900000000000004e-07 2024-09-16T20:49:51.486 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.487 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.487 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.487 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283204+0000", 2024-09-16T20:49:51.487 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1500000000000002e-06 2024-09-16T20:49:51.487 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.487 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.487 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.487 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283213+0000", 2024-09-16T20:49:51.487 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0550000000000005e-06 2024-09-16T20:49:51.487 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.487 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.487 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.487 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283219+0000", 2024-09-16T20:49:51.487 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.604e-06 2024-09-16T20:49:51.488 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.488 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.488 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.488 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.488 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.488 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18823 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.488 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.265322+0000", 2024-09-16T20:49:51.488 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.029009895000000001, 2024-09-16T20:49:51.488 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.031194237999999999, 2024-09-16T20:49:51.488 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.488 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.488 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.489 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.489 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.489 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.489 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18823 2024-09-16T20:49:51.489 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.489 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.489 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.489 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.489 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265322+0000", 2024-09-16T20:49:51.489 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.489 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.489 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.489 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.489 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265322+0000", 2024-09-16T20:49:51.489 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.490 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.490 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.490 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.490 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265322+0000", 2024-09-16T20:49:51.490 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2999999999999999e-07 2024-09-16T20:49:51.490 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.490 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.490 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.490 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265323+0000", 2024-09-16T20:49:51.490 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.08e-06 2024-09-16T20:49:51.490 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.490 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.490 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.490 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265328+0000", 2024-09-16T20:49:51.490 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9799999999999998e-06 2024-09-16T20:49:51.491 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.491 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.491 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.491 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265331+0000", 2024-09-16T20:49:51.491 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9239999999999999e-06 2024-09-16T20:49:51.491 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.491 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.491 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.491 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.491 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.491 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18855 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.491 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.265741+0000", 2024-09-16T20:49:51.491 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.02859039, 2024-09-16T20:49:51.491 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.030799251, 2024-09-16T20:49:51.491 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.492 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.492 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.492 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.492 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.492 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.492 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18855 2024-09-16T20:49:51.492 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.492 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.492 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.492 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.492 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265741+0000", 2024-09-16T20:49:51.492 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.492 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.492 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.492 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.493 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265741+0000", 2024-09-16T20:49:51.493 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.493 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.493 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.493 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.493 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265741+0000", 2024-09-16T20:49:51.493 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2499999999999999e-07 2024-09-16T20:49:51.493 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.493 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.493 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.493 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265742+0000", 2024-09-16T20:49:51.493 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0950000000000001e-06 2024-09-16T20:49:51.493 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.493 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.493 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.494 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265746+0000", 2024-09-16T20:49:51.494 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1750000000000001e-06 2024-09-16T20:49:51.494 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.494 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.494 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.494 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265748+0000", 2024-09-16T20:49:51.494 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6230000000000001e-06 2024-09-16T20:49:51.494 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.494 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.494 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.494 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.494 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.494 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18890 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.495 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.271698+0000", 2024-09-16T20:49:51.495 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.022633304, 2024-09-16T20:49:51.495 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.024866253000000001, 2024-09-16T20:49:51.495 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.495 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.495 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.495 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.495 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.495 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.495 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18890 2024-09-16T20:49:51.495 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.495 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.495 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.495 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.495 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271698+0000", 2024-09-16T20:49:51.496 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.496 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.496 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.496 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.496 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271698+0000", 2024-09-16T20:49:51.496 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.496 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.496 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.496 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.496 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271698+0000", 2024-09-16T20:49:51.496 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7300000000000002e-07 2024-09-16T20:49:51.496 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.496 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.496 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.497 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271700+0000", 2024-09-16T20:49:51.497 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9089999999999998e-06 2024-09-16T20:49:51.497 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.497 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.497 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.497 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271706+0000", 2024-09-16T20:49:51.497 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0979999999999999e-06 2024-09-16T20:49:51.497 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.497 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.497 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.497 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271711+0000", 2024-09-16T20:49:51.497 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5499999999999996e-06 2024-09-16T20:49:51.497 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.497 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.498 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.498 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.498 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.498 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18922 3.10 3.996ee350 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.498 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.277374+0000", 2024-09-16T20:49:51.498 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.016957587, 2024-09-16T20:49:51.498 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.019217292, 2024-09-16T20:49:51.498 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.498 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.498 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.498 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.498 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.498 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.498 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18922 2024-09-16T20:49:51.499 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.499 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.499 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.499 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.499 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277374+0000", 2024-09-16T20:49:51.499 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.499 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.499 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.499 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.499 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277374+0000", 2024-09-16T20:49:51.499 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.499 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.499 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.500 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.500 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277374+0000", 2024-09-16T20:49:51.500 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4099999999999999e-07 2024-09-16T20:49:51.500 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.500 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.500 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.500 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277376+0000", 2024-09-16T20:49:51.500 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1040000000000001e-06 2024-09-16T20:49:51.500 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.500 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.500 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.500 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277385+0000", 2024-09-16T20:49:51.500 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.7250000000000003e-06 2024-09-16T20:49:51.500 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.501 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.501 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.501 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277390+0000", 2024-09-16T20:49:51.501 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3510000000000001e-06 2024-09-16T20:49:51.501 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.501 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.501 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.501 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.501 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.501 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18954 3.10 3.996ee350 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.501 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.283229+0000", 2024-09-16T20:49:51.501 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.011102489, 2024-09-16T20:49:51.501 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.01340731, 2024-09-16T20:49:51.501 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.501 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.502 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.502 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.502 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.502 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.502 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18954 2024-09-16T20:49:51.502 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.502 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.502 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.502 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.502 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283229+0000", 2024-09-16T20:49:51.502 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.502 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.502 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.502 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.502 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283229+0000", 2024-09-16T20:49:51.503 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.503 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.503 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.503 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.503 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283229+0000", 2024-09-16T20:49:51.503 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7999999999999996e-07 2024-09-16T20:49:51.503 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.503 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.503 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.503 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283236+0000", 2024-09-16T20:49:51.503 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5420000000000002e-06 2024-09-16T20:49:51.503 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.503 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.503 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.503 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283245+0000", 2024-09-16T20:49:51.504 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6300000000000004e-06 2024-09-16T20:49:51.504 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.504 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.504 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.504 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283250+0000", 2024-09-16T20:49:51.504 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8300000000000001e-06 2024-09-16T20:49:51.504 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.504 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.504 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.504 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.504 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.504 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18824 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.504 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.265334+0000", 2024-09-16T20:49:51.504 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.028997321, 2024-09-16T20:49:51.504 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.031352047000000001, 2024-09-16T20:49:51.505 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.505 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.505 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.505 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.505 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.505 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.505 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18824 2024-09-16T20:49:51.505 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.505 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.505 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.505 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.505 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265334+0000", 2024-09-16T20:49:51.505 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.506 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.506 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.506 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.506 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265334+0000", 2024-09-16T20:49:51.506 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.506 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.506 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.506 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.506 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265334+0000", 2024-09-16T20:49:51.506 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5100000000000001e-07 2024-09-16T20:49:51.506 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.506 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.506 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.506 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265335+0000", 2024-09-16T20:49:51.506 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0440000000000001e-06 2024-09-16T20:49:51.507 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.507 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.507 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.507 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265338+0000", 2024-09-16T20:49:51.507 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.022e-06 2024-09-16T20:49:51.507 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.507 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.507 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.507 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265341+0000", 2024-09-16T20:49:51.507 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6189999999999998e-06 2024-09-16T20:49:51.507 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.507 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.507 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.507 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.507 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.507 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18856 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.508 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.265751+0000", 2024-09-16T20:49:51.508 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.028580227, 2024-09-16T20:49:51.508 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.030983156000000001, 2024-09-16T20:49:51.508 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.508 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.508 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.508 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.508 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.508 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.508 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18856 2024-09-16T20:49:51.508 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.508 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.508 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.509 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.509 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265751+0000", 2024-09-16T20:49:51.509 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.509 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.509 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.509 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.509 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265751+0000", 2024-09-16T20:49:51.509 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.509 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.509 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.509 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.509 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265751+0000", 2024-09-16T20:49:51.509 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1799999999999999e-07 2024-09-16T20:49:51.509 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.510 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.510 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.510 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265752+0000", 2024-09-16T20:49:51.510 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0610000000000001e-06 2024-09-16T20:49:51.510 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.510 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.510 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.510 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265757+0000", 2024-09-16T20:49:51.510 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.566e-06 2024-09-16T20:49:51.510 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.510 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.510 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.510 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265760+0000", 2024-09-16T20:49:51.510 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8839999999999998e-06 2024-09-16T20:49:51.510 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.511 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.511 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.511 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.511 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.511 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18891 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.511 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.271716+0000", 2024-09-16T20:49:51.511 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.022615185999999999, 2024-09-16T20:49:51.511 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.025065054999999999, 2024-09-16T20:49:51.511 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.511 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.511 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.511 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.511 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.511 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.511 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18891 2024-09-16T20:49:51.512 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.512 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.512 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.512 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.512 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271716+0000", 2024-09-16T20:49:51.512 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.512 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.512 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.512 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.512 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271716+0000", 2024-09-16T20:49:51.512 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.512 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.512 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.512 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.512 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271716+0000", 2024-09-16T20:49:51.513 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.4499999999999998e-07 2024-09-16T20:49:51.513 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.513 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.513 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.513 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271718+0000", 2024-09-16T20:49:51.513 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.099e-06 2024-09-16T20:49:51.513 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.513 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.513 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.513 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271721+0000", 2024-09-16T20:49:51.513 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.337e-06 2024-09-16T20:49:51.513 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.513 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.513 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.513 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271723+0000", 2024-09-16T20:49:51.514 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5909999999999998e-06 2024-09-16T20:49:51.514 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.514 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.514 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.514 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.514 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.514 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18923 3.10 3.996ee350 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.514 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.277397+0000", 2024-09-16T20:49:51.514 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.016934055, 2024-09-16T20:49:51.514 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.019420840000000002, 2024-09-16T20:49:51.514 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.514 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.514 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.514 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.514 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.515 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.515 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18923 2024-09-16T20:49:51.515 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.515 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.515 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.515 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.515 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277397+0000", 2024-09-16T20:49:51.515 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.515 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.515 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.515 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.515 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277397+0000", 2024-09-16T20:49:51.515 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.515 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.515 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.515 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.516 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277398+0000", 2024-09-16T20:49:51.516 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.58e-07 2024-09-16T20:49:51.516 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.516 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.516 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.516 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277400+0000", 2024-09-16T20:49:51.516 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8679999999999999e-06 2024-09-16T20:49:51.516 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.516 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.516 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.516 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277407+0000", 2024-09-16T20:49:51.516 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.204e-06 2024-09-16T20:49:51.516 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.516 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.516 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.517 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277412+0000", 2024-09-16T20:49:51.517 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0390000000000004e-06 2024-09-16T20:49:51.517 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.517 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.517 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.517 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.517 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.517 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18955 3.10 3.996ee350 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.517 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.283265+0000", 2024-09-16T20:49:51.517 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.011066229, 2024-09-16T20:49:51.517 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.01357798, 2024-09-16T20:49:51.517 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.517 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.517 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.517 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.518 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.518 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.518 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18955 2024-09-16T20:49:51.518 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.518 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.518 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.518 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.518 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283265+0000", 2024-09-16T20:49:51.518 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.518 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.518 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.518 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.519 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283265+0000", 2024-09-16T20:49:51.519 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.519 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.519 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.519 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.519 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283266+0000", 2024-09-16T20:49:51.519 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4000000000000001e-07 2024-09-16T20:49:51.519 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.519 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.519 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.519 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283268+0000", 2024-09-16T20:49:51.519 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2819999999999999e-06 2024-09-16T20:49:51.519 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.520 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.520 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.520 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283276+0000", 2024-09-16T20:49:51.520 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5830000000000001e-06 2024-09-16T20:49:51.520 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.520 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.520 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.520 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283283+0000", 2024-09-16T20:49:51.520 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8639999999999998e-06 2024-09-16T20:49:51.520 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.520 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.520 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.520 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.521 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.521 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18825 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.521 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.265344+0000", 2024-09-16T20:49:51.521 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.028987103, 2024-09-16T20:49:51.521 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.03152452, 2024-09-16T20:49:51.521 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.521 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.521 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.521 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.521 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.521 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.521 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18825 2024-09-16T20:49:51.521 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.521 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.522 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.522 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.522 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265344+0000", 2024-09-16T20:49:51.522 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.522 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.522 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.522 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.522 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265344+0000", 2024-09-16T20:49:51.522 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.522 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.522 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.522 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.522 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265345+0000", 2024-09-16T20:49:51.523 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1e-07 2024-09-16T20:49:51.523 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.523 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.523 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.523 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265346+0000", 2024-09-16T20:49:51.523 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.082e-06 2024-09-16T20:49:51.523 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.523 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.523 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.523 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265353+0000", 2024-09-16T20:49:51.523 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3679999999999996e-06 2024-09-16T20:49:51.523 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.523 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.524 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.524 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265358+0000", 2024-09-16T20:49:51.524 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4700000000000004e-06 2024-09-16T20:49:51.524 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.524 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.524 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.524 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.524 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.524 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18858 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.524 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.265764+0000", 2024-09-16T20:49:51.524 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.028567608000000001, 2024-09-16T20:49:51.524 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.031144068, 2024-09-16T20:49:51.524 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.525 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.525 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.525 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.525 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.525 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.525 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18858 2024-09-16T20:49:51.525 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.525 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.525 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.525 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.525 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265764+0000", 2024-09-16T20:49:51.525 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.525 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.526 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.526 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.526 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265764+0000", 2024-09-16T20:49:51.526 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.526 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.526 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.526 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.526 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265835+0000", 2024-09-16T20:49:51.526 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1599000000000003e-05 2024-09-16T20:49:51.526 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.526 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.526 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.526 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265838+0000", 2024-09-16T20:49:51.526 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7269999999999999e-06 2024-09-16T20:49:51.527 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.527 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.527 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.527 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265845+0000", 2024-09-16T20:49:51.527 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2830000000000004e-06 2024-09-16T20:49:51.527 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.527 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.527 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.527 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265852+0000", 2024-09-16T20:49:51.527 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5300000000000002e-06 2024-09-16T20:49:51.527 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.527 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.527 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.527 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.528 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.528 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18892 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.528 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.271727+0000", 2024-09-16T20:49:51.528 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.022604735000000001, 2024-09-16T20:49:51.528 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.025228813999999999, 2024-09-16T20:49:51.528 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.528 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.528 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.528 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.528 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.528 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.528 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18892 2024-09-16T20:49:51.529 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.529 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.529 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.529 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.529 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271727+0000", 2024-09-16T20:49:51.529 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.529 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.529 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.529 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.529 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271727+0000", 2024-09-16T20:49:51.529 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.529 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.529 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.529 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.530 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271727+0000", 2024-09-16T20:49:51.530 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3099999999999999e-07 2024-09-16T20:49:51.530 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.530 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.530 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.530 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271732+0000", 2024-09-16T20:49:51.530 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4929999999999998e-06 2024-09-16T20:49:51.530 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.530 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.530 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.530 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271736+0000", 2024-09-16T20:49:51.530 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.608e-06 2024-09-16T20:49:51.530 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.531 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.531 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.531 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271740+0000", 2024-09-16T20:49:51.531 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1280000000000001e-06 2024-09-16T20:49:51.531 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.531 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.531 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.531 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.531 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.531 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18924 3.10 3.996ee350 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.531 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.277419+0000", 2024-09-16T20:49:51.531 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.016912685, 2024-09-16T20:49:51.531 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.019585446999999999, 2024-09-16T20:49:51.531 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.531 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.532 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.532 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.532 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.532 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.532 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18924 2024-09-16T20:49:51.532 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.532 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.532 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.532 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.532 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277419+0000", 2024-09-16T20:49:51.532 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.532 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.532 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.532 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.533 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277419+0000", 2024-09-16T20:49:51.533 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.533 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.533 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.533 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.533 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277419+0000", 2024-09-16T20:49:51.533 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.08e-07 2024-09-16T20:49:51.533 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.533 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.533 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.533 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277421+0000", 2024-09-16T20:49:51.533 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0540000000000002e-06 2024-09-16T20:49:51.533 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.533 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.533 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.534 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277427+0000", 2024-09-16T20:49:51.534 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1399999999999997e-06 2024-09-16T20:49:51.534 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.534 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.534 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.534 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277432+0000", 2024-09-16T20:49:51.534 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7299999999999996e-06 2024-09-16T20:49:51.534 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.534 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.534 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.534 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.534 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.534 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18956 3.10 3.996ee350 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.534 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.283291+0000", 2024-09-16T20:49:51.534 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.011040664, 2024-09-16T20:49:51.535 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.013776968000000001, 2024-09-16T20:49:51.535 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.535 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.535 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.535 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.535 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.535 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.535 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18956 2024-09-16T20:49:51.535 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.535 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.535 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.535 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.535 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283291+0000", 2024-09-16T20:49:51.535 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.535 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.536 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.536 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.536 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283291+0000", 2024-09-16T20:49:51.536 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.536 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.536 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.536 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.536 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283291+0000", 2024-09-16T20:49:51.536 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1399999999999997e-07 2024-09-16T20:49:51.536 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.536 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.536 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.536 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283293+0000", 2024-09-16T20:49:51.536 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0490000000000002e-06 2024-09-16T20:49:51.536 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.537 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.537 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.537 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283300+0000", 2024-09-16T20:49:51.537 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7270000000000001e-06 2024-09-16T20:49:51.537 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.537 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.537 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.537 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283305+0000", 2024-09-16T20:49:51.537 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3210000000000004e-06 2024-09-16T20:49:51.537 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.537 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.537 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.537 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.537 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.538 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18826 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.538 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.265361+0000", 2024-09-16T20:49:51.538 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.028970171999999999, 2024-09-16T20:49:51.538 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.031752372000000001, 2024-09-16T20:49:51.538 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.538 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.538 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.538 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.538 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.538 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.538 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18826 2024-09-16T20:49:51.538 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.538 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.538 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.539 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.539 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265361+0000", 2024-09-16T20:49:51.539 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.539 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.539 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.539 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.539 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265361+0000", 2024-09-16T20:49:51.539 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.539 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.539 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.539 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.539 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265361+0000", 2024-09-16T20:49:51.539 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5600000000000002e-07 2024-09-16T20:49:51.539 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.539 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.540 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.540 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265363+0000", 2024-09-16T20:49:51.540 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.097e-06 2024-09-16T20:49:51.540 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.540 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.540 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.540 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265369+0000", 2024-09-16T20:49:51.540 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4330000000000001e-06 2024-09-16T20:49:51.540 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.540 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.540 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.540 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265376+0000", 2024-09-16T20:49:51.540 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1709999999999996e-06 2024-09-16T20:49:51.540 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.541 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.541 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.541 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.541 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.541 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18859 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.541 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.265858+0000", 2024-09-16T20:49:51.541 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.028473127000000001, 2024-09-16T20:49:51.541 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.031280334, 2024-09-16T20:49:51.541 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.541 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.541 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.541 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.541 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.541 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.541 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18859 2024-09-16T20:49:51.542 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.542 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.542 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.542 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.542 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265858+0000", 2024-09-16T20:49:51.542 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.542 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.542 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.542 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.542 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265858+0000", 2024-09-16T20:49:51.542 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.542 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.542 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.542 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.543 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.266294+0000", 2024-09-16T20:49:51.543 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00043545699999999999 2024-09-16T20:49:51.543 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.543 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.543 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.543 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.266296+0000", 2024-09-16T20:49:51.543 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.571e-06 2024-09-16T20:49:51.543 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.543 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.543 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.543 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.266302+0000", 2024-09-16T20:49:51.543 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.575e-06 2024-09-16T20:49:51.543 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.543 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.544 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.544 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.266307+0000", 2024-09-16T20:49:51.544 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1680000000000003e-06 2024-09-16T20:49:51.544 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.544 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.544 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.544 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.544 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.544 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18893 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.544 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.271744+0000", 2024-09-16T20:49:51.544 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.022587672, 2024-09-16T20:49:51.544 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.025419039000000001, 2024-09-16T20:49:51.544 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.545 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.545 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.545 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.545 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.545 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.545 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18893 2024-09-16T20:49:51.545 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.545 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.545 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.545 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.545 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271744+0000", 2024-09-16T20:49:51.545 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.545 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.545 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.546 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.546 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271744+0000", 2024-09-16T20:49:51.546 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.546 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.546 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.546 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.546 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271744+0000", 2024-09-16T20:49:51.546 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.35e-07 2024-09-16T20:49:51.546 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.546 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.546 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.546 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271745+0000", 2024-09-16T20:49:51.546 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0589999999999999e-06 2024-09-16T20:49:51.546 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.547 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.547 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.547 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271748+0000", 2024-09-16T20:49:51.547 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1700000000000001e-06 2024-09-16T20:49:51.547 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.547 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.547 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.547 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271751+0000", 2024-09-16T20:49:51.547 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5339999999999998e-06 2024-09-16T20:49:51.547 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.547 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.547 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.547 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.547 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.548 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18925 3.10 3.996ee350 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.548 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.277439+0000", 2024-09-16T20:49:51.548 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.016892759, 2024-09-16T20:49:51.548 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.019748089999999999, 2024-09-16T20:49:51.548 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.548 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.548 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.548 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.548 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.548 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.548 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18925 2024-09-16T20:49:51.548 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.548 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.549 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.549 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.549 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277439+0000", 2024-09-16T20:49:51.549 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.549 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.549 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.549 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.549 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277439+0000", 2024-09-16T20:49:51.549 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.549 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.549 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.549 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.549 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277439+0000", 2024-09-16T20:49:51.549 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.51e-07 2024-09-16T20:49:51.550 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.550 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.550 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.550 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277441+0000", 2024-09-16T20:49:51.550 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9319999999999999e-06 2024-09-16T20:49:51.550 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.550 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.550 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.550 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277447+0000", 2024-09-16T20:49:51.550 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1890000000000004e-06 2024-09-16T20:49:51.550 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.550 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.550 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.550 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277452+0000", 2024-09-16T20:49:51.551 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0490000000000003e-06 2024-09-16T20:49:51.551 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.551 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.551 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.551 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.551 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.551 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18957 3.10 3.996ee350 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.551 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.283313+0000", 2024-09-16T20:49:51.551 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.011018258, 2024-09-16T20:49:51.551 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.01391937, 2024-09-16T20:49:51.551 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.551 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.551 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.551 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.551 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.551 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.552 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18957 2024-09-16T20:49:51.552 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.552 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.552 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.552 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.552 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283313+0000", 2024-09-16T20:49:51.552 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.552 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.552 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.552 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.552 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283313+0000", 2024-09-16T20:49:51.552 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.552 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.552 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.553 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.553 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283314+0000", 2024-09-16T20:49:51.553 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3499999999999996e-07 2024-09-16T20:49:51.553 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.553 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.553 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.553 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283316+0000", 2024-09-16T20:49:51.553 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1689999999999999e-06 2024-09-16T20:49:51.553 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.553 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.553 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.553 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283324+0000", 2024-09-16T20:49:51.553 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9009999999999994e-06 2024-09-16T20:49:51.553 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.554 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.554 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.554 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283329+0000", 2024-09-16T20:49:51.554 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2630000000000003e-06 2024-09-16T20:49:51.554 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.554 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.554 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.554 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.554 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.554 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18827 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.554 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.265380+0000", 2024-09-16T20:49:51.554 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.028950931999999999, 2024-09-16T20:49:51.554 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.031900382999999997, 2024-09-16T20:49:51.554 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.554 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.555 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.555 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.555 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.555 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.555 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18827 2024-09-16T20:49:51.555 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.555 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.555 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.555 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.555 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265380+0000", 2024-09-16T20:49:51.555 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.555 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.555 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.555 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.556 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265380+0000", 2024-09-16T20:49:51.556 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.556 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.556 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.556 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.556 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265381+0000", 2024-09-16T20:49:51.556 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5800000000000001e-07 2024-09-16T20:49:51.556 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.556 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.556 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.556 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265382+0000", 2024-09-16T20:49:51.556 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0690000000000001e-06 2024-09-16T20:49:51.556 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.556 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.557 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.557 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265385+0000", 2024-09-16T20:49:51.557 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1049999999999999e-06 2024-09-16T20:49:51.557 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.557 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.557 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.557 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265390+0000", 2024-09-16T20:49:51.557 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1680000000000003e-06 2024-09-16T20:49:51.557 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.557 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.557 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.557 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.557 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.557 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18860 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.558 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.266314+0000", 2024-09-16T20:49:51.558 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.028017852999999999, 2024-09-16T20:49:51.558 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.031015190000000002, 2024-09-16T20:49:51.558 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.558 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.558 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.558 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.558 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.558 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.558 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18860 2024-09-16T20:49:51.558 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.558 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.558 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.559 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.559 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.266314+0000", 2024-09-16T20:49:51.559 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.559 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.559 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.559 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.559 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.266314+0000", 2024-09-16T20:49:51.559 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.559 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.559 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.559 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.559 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.266314+0000", 2024-09-16T20:49:51.559 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1899999999999998e-07 2024-09-16T20:49:51.560 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.560 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.560 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.560 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.266315+0000", 2024-09-16T20:49:51.560 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1480000000000001e-06 2024-09-16T20:49:51.560 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.560 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.560 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.560 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.266318+0000", 2024-09-16T20:49:51.560 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3799999999999998e-06 2024-09-16T20:49:51.560 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.560 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.560 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.560 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.266321+0000", 2024-09-16T20:49:51.561 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7659999999999999e-06 2024-09-16T20:49:51.561 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.561 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.561 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.561 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.561 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.561 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18894 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.561 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.271754+0000", 2024-09-16T20:49:51.561 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.022577597000000001, 2024-09-16T20:49:51.561 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.025622273000000001, 2024-09-16T20:49:51.561 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.561 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.561 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.562 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.562 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.562 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.562 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18894 2024-09-16T20:49:51.562 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.562 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.562 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.562 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.562 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271754+0000", 2024-09-16T20:49:51.562 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.562 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.562 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.562 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.562 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271754+0000", 2024-09-16T20:49:51.562 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.563 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.563 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.563 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.563 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271754+0000", 2024-09-16T20:49:51.563 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.4400000000000001e-07 2024-09-16T20:49:51.563 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.563 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.563 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.563 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271755+0000", 2024-09-16T20:49:51.563 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0279999999999999e-06 2024-09-16T20:49:51.563 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.563 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.563 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.564 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271759+0000", 2024-09-16T20:49:51.564 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2069999999999997e-06 2024-09-16T20:49:51.564 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.564 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.564 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.564 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271764+0000", 2024-09-16T20:49:51.564 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6970000000000001e-06 2024-09-16T20:49:51.564 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.564 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.564 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.564 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.564 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.564 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18926 3.10 3.996ee350 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.565 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.277459+0000", 2024-09-16T20:49:51.565 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.016872714, 2024-09-16T20:49:51.565 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.019953847, 2024-09-16T20:49:51.565 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.565 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.565 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.565 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.565 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.565 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.565 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18926 2024-09-16T20:49:51.565 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.565 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.566 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.566 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.566 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277459+0000", 2024-09-16T20:49:51.566 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.566 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.566 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.566 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.566 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277459+0000", 2024-09-16T20:49:51.566 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.566 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.566 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.566 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.566 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277459+0000", 2024-09-16T20:49:51.567 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9499999999999998e-07 2024-09-16T20:49:51.567 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.567 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.567 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.567 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277461+0000", 2024-09-16T20:49:51.567 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9300000000000002e-06 2024-09-16T20:49:51.567 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.567 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.567 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.567 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277467+0000", 2024-09-16T20:49:51.567 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0800000000000002e-06 2024-09-16T20:49:51.567 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.567 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.567 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.568 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277472+0000", 2024-09-16T20:49:51.568 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0900000000000004e-06 2024-09-16T20:49:51.568 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.568 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.568 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.568 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.568 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.568 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18958 3.10 3.996ee350 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.568 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.283336+0000", 2024-09-16T20:49:51.568 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.010994943, 2024-09-16T20:49:51.568 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.014101137, 2024-09-16T20:49:51.568 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.568 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.569 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.569 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.569 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.569 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.569 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18958 2024-09-16T20:49:51.569 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.569 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.569 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.569 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.569 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283336+0000", 2024-09-16T20:49:51.569 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.569 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.570 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.570 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.570 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283336+0000", 2024-09-16T20:49:51.570 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.570 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.570 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.570 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.570 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283337+0000", 2024-09-16T20:49:51.570 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1199999999999998e-07 2024-09-16T20:49:51.570 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.570 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.570 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.570 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283339+0000", 2024-09-16T20:49:51.570 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9980000000000002e-06 2024-09-16T20:49:51.571 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.571 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.571 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.571 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283345+0000", 2024-09-16T20:49:51.571 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.348e-06 2024-09-16T20:49:51.571 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.571 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.571 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.571 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283351+0000", 2024-09-16T20:49:51.571 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.083e-06 2024-09-16T20:49:51.571 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.571 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.571 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.571 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.572 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.572 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18828 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.572 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.265395+0000", 2024-09-16T20:49:51.572 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.028936343, 2024-09-16T20:49:51.572 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.032066456, 2024-09-16T20:49:51.572 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.572 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.572 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.572 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.572 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.572 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.572 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18828 2024-09-16T20:49:51.572 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.572 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.573 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.573 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.573 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265395+0000", 2024-09-16T20:49:51.573 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.573 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.573 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.573 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.573 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265395+0000", 2024-09-16T20:49:51.573 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.573 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.573 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.573 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.573 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265395+0000", 2024-09-16T20:49:51.573 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2999999999999999e-07 2024-09-16T20:49:51.573 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.574 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.574 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.574 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265396+0000", 2024-09-16T20:49:51.574 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.032e-06 2024-09-16T20:49:51.574 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.574 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.574 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.574 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265399+0000", 2024-09-16T20:49:51.574 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0340000000000001e-06 2024-09-16T20:49:51.574 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.574 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.574 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.574 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265403+0000", 2024-09-16T20:49:51.574 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6849999999999999e-06 2024-09-16T20:49:51.574 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.575 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.575 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.575 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.575 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.575 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18861 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.575 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.266324+0000", 2024-09-16T20:49:51.575 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.028007014, 2024-09-16T20:49:51.575 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.031160531000000002, 2024-09-16T20:49:51.575 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.575 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.575 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.575 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.575 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.575 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.575 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18861 2024-09-16T20:49:51.576 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.576 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.576 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.576 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.576 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.266324+0000", 2024-09-16T20:49:51.576 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.576 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.576 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.576 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.576 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.266324+0000", 2024-09-16T20:49:51.576 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.576 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.576 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.576 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.577 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.266325+0000", 2024-09-16T20:49:51.577 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.4400000000000001e-07 2024-09-16T20:49:51.577 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.577 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.577 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.577 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.266326+0000", 2024-09-16T20:49:51.577 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0550000000000001e-06 2024-09-16T20:49:51.577 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.577 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.577 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.577 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.266329+0000", 2024-09-16T20:49:51.577 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0920000000000002e-06 2024-09-16T20:49:51.577 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.577 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.578 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.578 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.266332+0000", 2024-09-16T20:49:51.578 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7549999999999999e-06 2024-09-16T20:49:51.578 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.578 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.578 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.578 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.578 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.578 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18895 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.578 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.271780+0000", 2024-09-16T20:49:51.578 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.022551135999999999, 2024-09-16T20:49:51.578 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.025750361999999999, 2024-09-16T20:49:51.578 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.578 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.578 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.579 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.579 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.579 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.579 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18895 2024-09-16T20:49:51.579 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.579 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.579 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.579 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.579 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271780+0000", 2024-09-16T20:49:51.579 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.579 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.579 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.579 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.579 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271780+0000", 2024-09-16T20:49:51.579 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.580 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.580 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.580 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.580 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271781+0000", 2024-09-16T20:49:51.580 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1099999999999996e-07 2024-09-16T20:49:51.580 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.580 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.580 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.580 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271783+0000", 2024-09-16T20:49:51.580 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.137e-06 2024-09-16T20:49:51.580 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.580 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.580 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.581 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271789+0000", 2024-09-16T20:49:51.581 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3199999999999996e-06 2024-09-16T20:49:51.581 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.581 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.581 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.581 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271794+0000", 2024-09-16T20:49:51.581 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1830000000000002e-06 2024-09-16T20:49:51.581 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.581 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.581 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.581 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.581 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.582 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18927 3.10 3.996ee350 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.582 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.277479+0000", 2024-09-16T20:49:51.582 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.016852459, 2024-09-16T20:49:51.582 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.020099866000000001, 2024-09-16T20:49:51.582 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.582 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.582 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.582 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.582 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.582 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.582 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18927 2024-09-16T20:49:51.582 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.582 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.582 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.583 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.583 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277479+0000", 2024-09-16T20:49:51.583 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.583 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.583 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.583 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.583 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277479+0000", 2024-09-16T20:49:51.583 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.583 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.583 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.583 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.583 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277479+0000", 2024-09-16T20:49:51.583 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2500000000000001e-07 2024-09-16T20:49:51.583 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.583 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.584 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.584 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277482+0000", 2024-09-16T20:49:51.584 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1950000000000002e-06 2024-09-16T20:49:51.584 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.584 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.584 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.584 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277489+0000", 2024-09-16T20:49:51.584 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.238e-06 2024-09-16T20:49:51.584 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.584 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.584 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.584 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277494+0000", 2024-09-16T20:49:51.584 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0000000000000004e-06 2024-09-16T20:49:51.585 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.585 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.585 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.585 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.585 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.585 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18959 3.10 3.996ee350 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.585 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.283358+0000", 2024-09-16T20:49:51.585 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.010973475, 2024-09-16T20:49:51.585 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.014270885000000001, 2024-09-16T20:49:51.585 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.585 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.585 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.585 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.586 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.586 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.586 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18959 2024-09-16T20:49:51.586 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.586 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.586 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.586 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.586 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283358+0000", 2024-09-16T20:49:51.586 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.586 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.586 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.586 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.586 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283358+0000", 2024-09-16T20:49:51.586 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.586 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.587 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.587 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.587 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283358+0000", 2024-09-16T20:49:51.587 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0200000000000003e-07 2024-09-16T20:49:51.587 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.587 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.587 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.587 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283360+0000", 2024-09-16T20:49:51.587 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8640000000000001e-06 2024-09-16T20:49:51.587 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.587 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.587 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.587 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283373+0000", 2024-09-16T20:49:51.587 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3118999999999999e-05 2024-09-16T20:49:51.588 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.588 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.588 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.588 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283386+0000", 2024-09-16T20:49:51.588 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2211e-05 2024-09-16T20:49:51.588 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.588 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.588 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.588 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.588 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.588 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18829 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.588 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.265406+0000", 2024-09-16T20:49:51.588 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.028925222, 2024-09-16T20:49:51.589 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.032270403000000003, 2024-09-16T20:49:51.589 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.589 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.589 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.589 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.589 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.589 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.589 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18829 2024-09-16T20:49:51.589 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.589 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.589 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.589 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.589 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265406+0000", 2024-09-16T20:49:51.589 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.589 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.590 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.590 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.590 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265406+0000", 2024-09-16T20:49:51.590 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.590 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.590 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.590 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.590 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265406+0000", 2024-09-16T20:49:51.590 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.35e-07 2024-09-16T20:49:51.590 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.590 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.590 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.590 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265407+0000", 2024-09-16T20:49:51.590 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4499999999999995e-07 2024-09-16T20:49:51.591 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.591 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.591 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.591 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265412+0000", 2024-09-16T20:49:51.591 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9699999999999998e-06 2024-09-16T20:49:51.591 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.591 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.591 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.591 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265415+0000", 2024-09-16T20:49:51.591 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6970000000000002e-06 2024-09-16T20:49:51.591 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.591 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.591 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.591 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.591 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.592 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18896 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.592 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.271802+0000", 2024-09-16T20:49:51.592 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.022529793999999999, 2024-09-16T20:49:51.592 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.025909518999999999, 2024-09-16T20:49:51.592 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.592 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.592 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.592 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.592 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.592 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.592 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18896 2024-09-16T20:49:51.592 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.593 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.593 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.593 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.593 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271802+0000", 2024-09-16T20:49:51.593 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.593 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.593 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.593 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.593 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271802+0000", 2024-09-16T20:49:51.593 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.593 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.593 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.593 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.594 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271802+0000", 2024-09-16T20:49:51.594 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6499999999999999e-07 2024-09-16T20:49:51.594 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.594 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.594 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.594 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271804+0000", 2024-09-16T20:49:51.594 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9850000000000001e-06 2024-09-16T20:49:51.594 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.594 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.594 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.594 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271810+0000", 2024-09-16T20:49:51.594 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0680000000000002e-06 2024-09-16T20:49:51.594 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.595 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.595 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.595 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271815+0000", 2024-09-16T20:49:51.595 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0939999999999998e-06 2024-09-16T20:49:51.595 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.595 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.595 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.595 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.595 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.595 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18928 3.10 3.996ee350 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.595 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.277532+0000", 2024-09-16T20:49:51.595 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.016798957999999999, 2024-09-16T20:49:51.595 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.020203036000000001, 2024-09-16T20:49:51.595 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.596 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.596 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.596 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.596 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.596 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.596 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18928 2024-09-16T20:49:51.596 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.596 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.596 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.596 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.596 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277532+0000", 2024-09-16T20:49:51.596 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.596 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.596 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.597 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.597 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277532+0000", 2024-09-16T20:49:51.597 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.597 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.597 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.597 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.597 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277533+0000", 2024-09-16T20:49:51.597 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.23e-07 2024-09-16T20:49:51.597 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.597 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.597 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.597 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277537+0000", 2024-09-16T20:49:51.597 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9249999999999997e-06 2024-09-16T20:49:51.597 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.598 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.598 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.598 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277551+0000", 2024-09-16T20:49:51.598 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.361e-05 2024-09-16T20:49:51.598 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.598 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.598 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.598 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277559+0000", 2024-09-16T20:49:51.598 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8599999999999993e-06 2024-09-16T20:49:51.598 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.598 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.598 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.598 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.598 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.599 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18960 3.10 3.996ee350 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.599 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.283396+0000", 2024-09-16T20:49:51.599 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.01093512, 2024-09-16T20:49:51.599 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.014364681000000001, 2024-09-16T20:49:51.599 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.599 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.599 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.599 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.599 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.599 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.599 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18960 2024-09-16T20:49:51.599 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.599 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.599 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.599 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.600 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283396+0000", 2024-09-16T20:49:51.600 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.600 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.600 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.600 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.600 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283396+0000", 2024-09-16T20:49:51.600 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.600 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.600 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.600 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.600 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283397+0000", 2024-09-16T20:49:51.600 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5700000000000002e-07 2024-09-16T20:49:51.600 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.601 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.601 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.601 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283399+0000", 2024-09-16T20:49:51.601 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5519999999999999e-06 2024-09-16T20:49:51.601 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.601 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.601 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.601 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283406+0000", 2024-09-16T20:49:51.601 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9079999999999998e-06 2024-09-16T20:49:51.601 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.601 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.601 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.601 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283412+0000", 2024-09-16T20:49:51.601 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4620000000000004e-06 2024-09-16T20:49:51.601 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.602 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.602 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.602 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.602 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.602 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18830 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.602 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.265419+0000", 2024-09-16T20:49:51.602 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.028912601999999999, 2024-09-16T20:49:51.602 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.032368310999999997, 2024-09-16T20:49:51.602 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.602 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.602 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.602 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.602 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.602 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.603 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18830 2024-09-16T20:49:51.603 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.603 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.603 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.603 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.603 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265419+0000", 2024-09-16T20:49:51.603 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.603 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.603 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.603 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.603 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265419+0000", 2024-09-16T20:49:51.603 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.603 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.603 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.604 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.604 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265419+0000", 2024-09-16T20:49:51.604 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2700000000000001e-07 2024-09-16T20:49:51.604 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.604 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.604 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.604 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265423+0000", 2024-09-16T20:49:51.604 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9280000000000003e-06 2024-09-16T20:49:51.604 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.604 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.604 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.604 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265427+0000", 2024-09-16T20:49:51.604 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3810000000000001e-06 2024-09-16T20:49:51.604 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.605 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.605 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.605 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265430+0000", 2024-09-16T20:49:51.605 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9790000000000002e-06 2024-09-16T20:49:51.605 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.605 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.605 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.605 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.605 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.605 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18865 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.605 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.271182+0000", 2024-09-16T20:49:51.605 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.023149459000000001, 2024-09-16T20:49:51.605 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.026651162999999999, 2024-09-16T20:49:51.606 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.606 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.606 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.606 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.606 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.606 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.606 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18865 2024-09-16T20:49:51.606 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.606 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.606 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.606 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.606 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271182+0000", 2024-09-16T20:49:51.606 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.606 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.607 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.607 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.607 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271182+0000", 2024-09-16T20:49:51.607 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.607 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.607 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.607 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.607 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271183+0000", 2024-09-16T20:49:51.607 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0999999999999998e-07 2024-09-16T20:49:51.607 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.607 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.607 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.607 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271185+0000", 2024-09-16T20:49:51.608 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6350000000000002e-06 2024-09-16T20:49:51.608 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.608 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.608 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.608 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271192+0000", 2024-09-16T20:49:51.608 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8770000000000004e-06 2024-09-16T20:49:51.608 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.608 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.608 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.608 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271196+0000", 2024-09-16T20:49:51.608 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4479999999999999e-06 2024-09-16T20:49:51.608 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.608 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.609 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.609 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.609 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.609 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18897 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.609 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.271822+0000", 2024-09-16T20:49:51.609 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.022509698000000002, 2024-09-16T20:49:51.609 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.026059638999999999, 2024-09-16T20:49:51.609 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.609 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.609 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.609 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.609 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.609 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.610 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18897 2024-09-16T20:49:51.610 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.610 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.610 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.610 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.610 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271822+0000", 2024-09-16T20:49:51.610 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.610 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.610 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.610 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.610 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271822+0000", 2024-09-16T20:49:51.610 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.610 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.610 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.611 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.611 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271822+0000", 2024-09-16T20:49:51.611 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9200000000000002e-07 2024-09-16T20:49:51.611 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.611 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.611 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.611 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271824+0000", 2024-09-16T20:49:51.611 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9520000000000001e-06 2024-09-16T20:49:51.611 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.611 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.611 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.611 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271830+0000", 2024-09-16T20:49:51.611 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1569999999999997e-06 2024-09-16T20:49:51.611 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.612 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.612 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.612 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271835+0000", 2024-09-16T20:49:51.612 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1109999999999999e-06 2024-09-16T20:49:51.612 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.612 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.612 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.612 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.612 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.612 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18929 3.10 3.996ee350 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.612 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.277576+0000", 2024-09-16T20:49:51.612 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.016755630000000001, 2024-09-16T20:49:51.612 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.020353871999999999, 2024-09-16T20:49:51.612 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.613 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.613 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.613 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.613 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.613 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.613 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18929 2024-09-16T20:49:51.613 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.613 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.613 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.613 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.613 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277576+0000", 2024-09-16T20:49:51.613 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.613 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.613 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.614 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.614 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277576+0000", 2024-09-16T20:49:51.614 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.614 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.614 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.614 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.614 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277576+0000", 2024-09-16T20:49:51.614 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5499999999999998e-07 2024-09-16T20:49:51.614 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.614 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.614 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.614 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277579+0000", 2024-09-16T20:49:51.614 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9059999999999998e-06 2024-09-16T20:49:51.614 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.615 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.615 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.615 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277587+0000", 2024-09-16T20:49:51.615 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3579999999999997e-06 2024-09-16T20:49:51.615 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.615 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.615 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.615 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277592+0000", 2024-09-16T20:49:51.615 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7910000000000001e-06 2024-09-16T20:49:51.615 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.615 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.615 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.615 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.615 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.615 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18961 3.10 3.996ee350 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.616 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.283419+0000", 2024-09-16T20:49:51.616 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.010912692, 2024-09-16T20:49:51.616 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.014558033, 2024-09-16T20:49:51.616 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.616 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.616 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.616 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.616 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.616 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.616 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18961 2024-09-16T20:49:51.616 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.616 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.616 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.616 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.617 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283419+0000", 2024-09-16T20:49:51.617 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.617 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.617 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.617 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.617 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283419+0000", 2024-09-16T20:49:51.617 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.617 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.617 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.617 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.617 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283419+0000", 2024-09-16T20:49:51.617 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4300000000000003e-07 2024-09-16T20:49:51.617 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.617 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.618 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.618 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283421+0000", 2024-09-16T20:49:51.618 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.136e-06 2024-09-16T20:49:51.618 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.618 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.618 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.618 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283429+0000", 2024-09-16T20:49:51.618 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1180000000000003e-06 2024-09-16T20:49:51.618 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.618 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.618 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.618 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283435+0000", 2024-09-16T20:49:51.618 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.905e-06 2024-09-16T20:49:51.618 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.619 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.619 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.619 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.619 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.619 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18831 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.619 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.265434+0000", 2024-09-16T20:49:51.619 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.028897783999999999, 2024-09-16T20:49:51.619 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.032577683000000003, 2024-09-16T20:49:51.619 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.619 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.619 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.619 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.619 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.620 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.620 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18831 2024-09-16T20:49:51.620 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.620 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.620 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.620 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.620 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265434+0000", 2024-09-16T20:49:51.620 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.620 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.620 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.620 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.620 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265434+0000", 2024-09-16T20:49:51.620 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.621 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.621 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.621 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.621 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265434+0000", 2024-09-16T20:49:51.621 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2999999999999999e-07 2024-09-16T20:49:51.621 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.621 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.621 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.621 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265435+0000", 2024-09-16T20:49:51.621 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0130000000000001e-06 2024-09-16T20:49:51.621 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.621 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.621 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.621 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265440+0000", 2024-09-16T20:49:51.622 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1039999999999998e-06 2024-09-16T20:49:51.622 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.622 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.622 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.622 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265443+0000", 2024-09-16T20:49:51.622 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6460000000000002e-06 2024-09-16T20:49:51.622 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.622 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.622 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.622 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.622 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.622 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18866 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.622 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.271202+0000", 2024-09-16T20:49:51.622 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.023129646, 2024-09-16T20:49:51.623 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.026834464999999998, 2024-09-16T20:49:51.623 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.623 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.623 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.623 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.623 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.623 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.623 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18866 2024-09-16T20:49:51.623 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.623 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.623 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.623 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.623 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271202+0000", 2024-09-16T20:49:51.623 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.624 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.624 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.624 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.624 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271202+0000", 2024-09-16T20:49:51.624 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.624 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.624 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.624 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.624 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271202+0000", 2024-09-16T20:49:51.624 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0999999999999998e-07 2024-09-16T20:49:51.624 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.624 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.624 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.625 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271204+0000", 2024-09-16T20:49:51.625 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0600000000000002e-06 2024-09-16T20:49:51.625 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.625 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.625 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.625 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271210+0000", 2024-09-16T20:49:51.625 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6180000000000003e-06 2024-09-16T20:49:51.625 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.625 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.625 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.625 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271215+0000", 2024-09-16T20:49:51.625 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3249999999999998e-06 2024-09-16T20:49:51.625 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.625 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.626 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.626 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.626 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.626 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18898 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.626 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.271841+0000", 2024-09-16T20:49:51.626 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.022489967, 2024-09-16T20:49:51.626 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.026219039999999999, 2024-09-16T20:49:51.626 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.626 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.626 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.626 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.626 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.626 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.626 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18898 2024-09-16T20:49:51.627 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.627 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.627 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.627 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.627 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271841+0000", 2024-09-16T20:49:51.627 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.627 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.627 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.627 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.627 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271841+0000", 2024-09-16T20:49:51.627 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.627 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.627 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.627 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.628 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271842+0000", 2024-09-16T20:49:51.628 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.89e-07 2024-09-16T20:49:51.628 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.628 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.628 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.628 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271844+0000", 2024-09-16T20:49:51.628 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.956e-06 2024-09-16T20:49:51.628 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.628 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.628 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.628 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271850+0000", 2024-09-16T20:49:51.628 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9939999999999997e-06 2024-09-16T20:49:51.628 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.629 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.629 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.629 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271855+0000", 2024-09-16T20:49:51.629 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9840000000000001e-06 2024-09-16T20:49:51.629 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.629 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.629 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.629 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.629 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.629 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18930 3.10 3.996ee350 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.629 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.277601+0000", 2024-09-16T20:49:51.629 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.016730636, 2024-09-16T20:49:51.629 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.020483245000000001, 2024-09-16T20:49:51.630 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.630 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.630 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.630 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.630 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.630 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.630 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18930 2024-09-16T20:49:51.630 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.630 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.630 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.630 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.630 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277601+0000", 2024-09-16T20:49:51.630 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.630 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.631 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.631 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.631 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277601+0000", 2024-09-16T20:49:51.631 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.631 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.631 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.631 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.631 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277601+0000", 2024-09-16T20:49:51.631 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8400000000000005e-07 2024-09-16T20:49:51.631 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.631 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.631 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.631 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277609+0000", 2024-09-16T20:49:51.631 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5719999999999997e-06 2024-09-16T20:49:51.631 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.632 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.632 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.632 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277616+0000", 2024-09-16T20:49:51.632 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.844e-06 2024-09-16T20:49:51.632 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.632 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.632 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.632 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277621+0000", 2024-09-16T20:49:51.632 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3530000000000002e-06 2024-09-16T20:49:51.632 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.632 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.632 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.632 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.632 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.633 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18962 3.10 3.996ee350 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.633 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.283442+0000", 2024-09-16T20:49:51.633 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.010888982, 2024-09-16T20:49:51.633 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.014691564000000001, 2024-09-16T20:49:51.633 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.633 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.633 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.633 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.633 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.633 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.633 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18962 2024-09-16T20:49:51.633 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.633 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.633 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.633 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.634 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283442+0000", 2024-09-16T20:49:51.634 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.634 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.634 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.634 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.634 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283442+0000", 2024-09-16T20:49:51.634 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.634 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.634 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.634 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.634 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283443+0000", 2024-09-16T20:49:51.634 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3099999999999998e-07 2024-09-16T20:49:51.634 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.634 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.635 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.635 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283445+0000", 2024-09-16T20:49:51.635 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0150000000000002e-06 2024-09-16T20:49:51.635 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.635 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.635 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.635 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283451+0000", 2024-09-16T20:49:51.635 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0560000000000001e-06 2024-09-16T20:49:51.635 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.635 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.635 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.635 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283462+0000", 2024-09-16T20:49:51.635 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0998999999999999e-05 2024-09-16T20:49:51.635 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.636 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.636 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.636 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.636 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.636 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18832 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.636 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.265446+0000", 2024-09-16T20:49:51.636 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.028885722999999999, 2024-09-16T20:49:51.636 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.032739694999999999, 2024-09-16T20:49:51.636 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.636 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.636 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.636 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.636 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.637 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.637 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18832 2024-09-16T20:49:51.637 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.637 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.637 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.637 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.637 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265446+0000", 2024-09-16T20:49:51.637 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.637 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.637 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.637 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.637 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265446+0000", 2024-09-16T20:49:51.637 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.637 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.638 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.638 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.638 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265446+0000", 2024-09-16T20:49:51.638 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.17e-07 2024-09-16T20:49:51.638 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.638 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.638 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.638 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265447+0000", 2024-09-16T20:49:51.638 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.9600000000000008e-07 2024-09-16T20:49:51.638 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.638 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.638 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.638 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265450+0000", 2024-09-16T20:49:51.638 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9639999999999999e-06 2024-09-16T20:49:51.639 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.639 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.639 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.639 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265454+0000", 2024-09-16T20:49:51.639 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8489999999999999e-06 2024-09-16T20:49:51.639 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.639 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.639 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.639 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.639 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.639 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18867 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.639 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.271221+0000", 2024-09-16T20:49:51.639 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.02311032, 2024-09-16T20:49:51.639 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.027012655999999999, 2024-09-16T20:49:51.640 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.640 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.640 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.640 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.640 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.640 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.640 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18867 2024-09-16T20:49:51.640 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.640 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.640 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.640 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.640 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271221+0000", 2024-09-16T20:49:51.640 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.640 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.641 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.641 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.641 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271221+0000", 2024-09-16T20:49:51.641 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.641 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.641 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.641 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.641 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271221+0000", 2024-09-16T20:49:51.641 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0100000000000001e-07 2024-09-16T20:49:51.641 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.641 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.641 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.641 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271223+0000", 2024-09-16T20:49:51.641 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.525e-06 2024-09-16T20:49:51.642 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.642 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.642 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.642 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271226+0000", 2024-09-16T20:49:51.642 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4690000000000002e-06 2024-09-16T20:49:51.642 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.642 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.642 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.642 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271229+0000", 2024-09-16T20:49:51.642 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7810000000000001e-06 2024-09-16T20:49:51.642 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.642 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.642 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.643 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.643 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.643 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18931 3.10 3.996ee350 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.643 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.277628+0000", 2024-09-16T20:49:51.643 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.016703300000000001, 2024-09-16T20:49:51.643 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.020653320999999999, 2024-09-16T20:49:51.643 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.643 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.643 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.643 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.643 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.643 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.643 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18931 2024-09-16T20:49:51.643 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.643 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.644 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.644 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.644 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277628+0000", 2024-09-16T20:49:51.644 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.644 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.644 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.644 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.644 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277628+0000", 2024-09-16T20:49:51.644 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.644 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.644 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.644 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.644 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277629+0000", 2024-09-16T20:49:51.644 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5299999999999999e-07 2024-09-16T20:49:51.645 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.645 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.645 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.645 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277631+0000", 2024-09-16T20:49:51.645 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9980000000000002e-06 2024-09-16T20:49:51.645 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.645 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.645 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.645 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277637+0000", 2024-09-16T20:49:51.645 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1800000000000001e-06 2024-09-16T20:49:51.645 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.645 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.645 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.645 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277641+0000", 2024-09-16T20:49:51.646 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6530000000000002e-06 2024-09-16T20:49:51.646 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.646 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.646 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.646 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.646 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.646 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18963 3.10 3.996ee350 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.646 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.283470+0000", 2024-09-16T20:49:51.646 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.010861561, 2024-09-16T20:49:51.646 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.014849002, 2024-09-16T20:49:51.646 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.646 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.646 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.646 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.647 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.647 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.647 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18963 2024-09-16T20:49:51.647 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.647 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.647 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.647 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.647 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283470+0000", 2024-09-16T20:49:51.647 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.647 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.647 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.647 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.647 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283470+0000", 2024-09-16T20:49:51.647 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.647 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.648 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.648 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.648 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283470+0000", 2024-09-16T20:49:51.648 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8699999999999995e-07 2024-09-16T20:49:51.648 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.648 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.648 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.648 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283473+0000", 2024-09-16T20:49:51.648 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.193e-06 2024-09-16T20:49:51.648 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.648 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.648 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.648 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283479+0000", 2024-09-16T20:49:51.649 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7580000000000003e-06 2024-09-16T20:49:51.649 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.649 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.649 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.649 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283485+0000", 2024-09-16T20:49:51.649 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.48e-06 2024-09-16T20:49:51.649 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.649 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.649 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.649 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.649 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.649 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18833 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.649 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.265457+0000", 2024-09-16T20:49:51.649 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.028874612000000001, 2024-09-16T20:49:51.650 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.032886461999999998, 2024-09-16T20:49:51.650 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.650 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.650 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.650 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.650 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.650 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.650 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18833 2024-09-16T20:49:51.650 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.650 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.650 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.650 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.650 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265457+0000", 2024-09-16T20:49:51.650 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.651 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.651 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.651 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.651 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265457+0000", 2024-09-16T20:49:51.651 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.651 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.651 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.651 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.651 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265457+0000", 2024-09-16T20:49:51.651 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2600000000000001e-07 2024-09-16T20:49:51.651 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.651 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.651 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.651 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265458+0000", 2024-09-16T20:49:51.651 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.6500000000000008e-07 2024-09-16T20:49:51.652 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.652 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.652 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.652 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265463+0000", 2024-09-16T20:49:51.652 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6280000000000004e-06 2024-09-16T20:49:51.652 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.652 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.652 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.652 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265465+0000", 2024-09-16T20:49:51.652 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.616e-06 2024-09-16T20:49:51.652 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.652 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.652 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.653 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.653 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.653 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18868 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.653 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.271233+0000", 2024-09-16T20:49:51.653 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.023098629999999998, 2024-09-16T20:49:51.653 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.027134011, 2024-09-16T20:49:51.653 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.653 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.653 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.653 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.653 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.653 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.653 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18868 2024-09-16T20:49:51.653 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.654 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.654 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.654 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.654 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271233+0000", 2024-09-16T20:49:51.654 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.654 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.654 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.654 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.654 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271233+0000", 2024-09-16T20:49:51.654 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.654 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.654 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.654 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.654 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271233+0000", 2024-09-16T20:49:51.654 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4299999999999999e-07 2024-09-16T20:49:51.655 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.655 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.655 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.655 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271235+0000", 2024-09-16T20:49:51.655 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.437e-06 2024-09-16T20:49:51.655 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.655 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.655 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.655 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271238+0000", 2024-09-16T20:49:51.655 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3009999999999998e-06 2024-09-16T20:49:51.655 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.655 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.655 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.655 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271241+0000", 2024-09-16T20:49:51.656 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5820000000000002e-06 2024-09-16T20:49:51.656 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.656 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.656 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.656 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.656 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.656 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18899 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.656 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.271861+0000", 2024-09-16T20:49:51.656 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.022470286999999999, 2024-09-16T20:49:51.656 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.02652939, 2024-09-16T20:49:51.656 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.656 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.656 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.656 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.657 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.657 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.657 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18899 2024-09-16T20:49:51.657 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.657 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.657 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.657 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.657 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271861+0000", 2024-09-16T20:49:51.657 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.657 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.657 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.657 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.657 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271861+0000", 2024-09-16T20:49:51.657 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.658 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.658 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.658 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.658 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274609+0000", 2024-09-16T20:49:51.658 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0027476409999999999 2024-09-16T20:49:51.658 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.658 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.658 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.658 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274613+0000", 2024-09-16T20:49:51.658 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2819999999999998e-06 2024-09-16T20:49:51.658 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.658 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.658 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.658 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274624+0000", 2024-09-16T20:49:51.659 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0728e-05 2024-09-16T20:49:51.659 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.659 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.659 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.659 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274632+0000", 2024-09-16T20:49:51.659 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5920000000000001e-06 2024-09-16T20:49:51.659 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.659 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.659 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.659 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.659 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.659 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18932 3.10 3.996ee350 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.659 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.277647+0000", 2024-09-16T20:49:51.659 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.016684132000000001, 2024-09-16T20:49:51.660 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.020781392999999999, 2024-09-16T20:49:51.660 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.660 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.660 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.660 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.660 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.660 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.660 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18932 2024-09-16T20:49:51.660 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.660 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.660 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.660 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.660 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277647+0000", 2024-09-16T20:49:51.660 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.661 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.661 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.661 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.661 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277647+0000", 2024-09-16T20:49:51.661 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.661 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.661 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.661 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.661 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277648+0000", 2024-09-16T20:49:51.661 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6600000000000002e-07 2024-09-16T20:49:51.661 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.661 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.661 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.661 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277649+0000", 2024-09-16T20:49:51.661 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.778e-06 2024-09-16T20:49:51.662 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.662 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.662 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.662 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277658+0000", 2024-09-16T20:49:51.662 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8079999999999994e-06 2024-09-16T20:49:51.662 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.662 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.662 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.662 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277661+0000", 2024-09-16T20:49:51.662 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.773e-06 2024-09-16T20:49:51.663 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.663 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.663 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.663 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.663 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.663 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18964 3.10 3.996ee350 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.663 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.283492+0000", 2024-09-16T20:49:51.663 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.010839161999999999, 2024-09-16T20:49:51.663 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.014984135000000001, 2024-09-16T20:49:51.663 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.663 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.663 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.663 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.664 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.664 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.664 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18964 2024-09-16T20:49:51.664 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.664 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.664 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.664 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.664 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283492+0000", 2024-09-16T20:49:51.664 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.664 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.664 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.664 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.664 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283492+0000", 2024-09-16T20:49:51.665 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.665 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.665 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.665 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.665 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283493+0000", 2024-09-16T20:49:51.665 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6199999999999998e-07 2024-09-16T20:49:51.665 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.665 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.665 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.665 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283495+0000", 2024-09-16T20:49:51.665 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0470000000000001e-06 2024-09-16T20:49:51.665 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.665 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.666 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.666 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283503+0000", 2024-09-16T20:49:51.666 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5590000000000006e-06 2024-09-16T20:49:51.666 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.666 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.666 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.666 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283509+0000", 2024-09-16T20:49:51.666 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.327e-06 2024-09-16T20:49:51.666 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.666 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.666 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.666 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.666 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.666 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18834 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.667 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.265468+0000", 2024-09-16T20:49:51.667 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.028863026999999999, 2024-09-16T20:49:51.667 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.033055871000000001, 2024-09-16T20:49:51.667 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.667 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.667 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.667 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.667 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.667 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.667 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18834 2024-09-16T20:49:51.667 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.667 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.667 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.667 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.668 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265468+0000", 2024-09-16T20:49:51.668 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.668 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.668 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.668 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.668 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265468+0000", 2024-09-16T20:49:51.668 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.668 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.668 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.668 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.668 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265469+0000", 2024-09-16T20:49:51.668 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2000000000000001e-07 2024-09-16T20:49:51.668 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.669 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.669 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.669 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265470+0000", 2024-09-16T20:49:51.669 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.5000000000000001e-07 2024-09-16T20:49:51.669 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.669 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.669 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.669 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265474+0000", 2024-09-16T20:49:51.669 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9750000000000001e-06 2024-09-16T20:49:51.669 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.669 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.669 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.669 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265476+0000", 2024-09-16T20:49:51.669 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.554e-06 2024-09-16T20:49:51.670 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.670 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.670 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.670 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.670 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.670 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18869 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.670 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.271264+0000", 2024-09-16T20:49:51.670 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.023067859, 2024-09-16T20:49:51.670 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.027309211, 2024-09-16T20:49:51.670 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.670 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.670 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.670 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.670 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.671 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.671 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18869 2024-09-16T20:49:51.671 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.671 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.671 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.671 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.671 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271264+0000", 2024-09-16T20:49:51.671 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.671 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.671 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.671 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.671 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271264+0000", 2024-09-16T20:49:51.671 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.672 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.672 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.672 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.672 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271264+0000", 2024-09-16T20:49:51.672 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2399999999999998e-07 2024-09-16T20:49:51.672 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.672 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.672 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.672 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271271+0000", 2024-09-16T20:49:51.672 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5180000000000001e-06 2024-09-16T20:49:51.672 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.672 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.672 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.672 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271277+0000", 2024-09-16T20:49:51.672 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8329999999999996e-06 2024-09-16T20:49:51.673 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.673 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.673 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.673 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271283+0000", 2024-09-16T20:49:51.673 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7230000000000001e-06 2024-09-16T20:49:51.673 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.673 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.673 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.673 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.673 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.673 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18900 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.673 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.274645+0000", 2024-09-16T20:49:51.673 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.019686535000000002, 2024-09-16T20:49:51.673 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.023972355000000001, 2024-09-16T20:49:51.674 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.674 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.674 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.674 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.674 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.674 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.674 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18900 2024-09-16T20:49:51.674 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.674 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.674 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.674 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.674 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274645+0000", 2024-09-16T20:49:51.674 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.674 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.675 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.675 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.675 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274645+0000", 2024-09-16T20:49:51.675 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.675 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.675 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.675 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.675 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274646+0000", 2024-09-16T20:49:51.675 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3900000000000004e-07 2024-09-16T20:49:51.675 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.675 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.675 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.675 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274648+0000", 2024-09-16T20:49:51.675 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.4210000000000002e-06 2024-09-16T20:49:51.676 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.676 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.676 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.676 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274655+0000", 2024-09-16T20:49:51.676 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6089999999999997e-06 2024-09-16T20:49:51.676 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.676 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.676 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.676 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274658+0000", 2024-09-16T20:49:51.676 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6780000000000002e-06 2024-09-16T20:49:51.676 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.676 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.676 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.676 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.676 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.677 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18933 3.10 3.996ee350 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.677 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.277666+0000", 2024-09-16T20:49:51.677 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.016665902999999999, 2024-09-16T20:49:51.677 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.020977104999999999, 2024-09-16T20:49:51.677 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.677 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.677 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.680 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.680 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.680 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.681 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18933 2024-09-16T20:49:51.681 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.681 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.681 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.681 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.681 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277666+0000", 2024-09-16T20:49:51.681 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.681 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.681 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.681 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.681 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.277666+0000", 2024-09-16T20:49:51.681 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.682 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.682 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.682 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.682 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282575+0000", 2024-09-16T20:49:51.682 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0049091860000000003 2024-09-16T20:49:51.682 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.682 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.682 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.682 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282580+0000", 2024-09-16T20:49:51.682 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9779999999999996e-06 2024-09-16T20:49:51.682 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.682 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.682 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.682 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282593+0000", 2024-09-16T20:49:51.682 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3106e-05 2024-09-16T20:49:51.683 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.683 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.683 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.683 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282601+0000", 2024-09-16T20:49:51.683 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0909999999999993e-06 2024-09-16T20:49:51.683 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.683 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.683 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.683 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.683 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.683 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18965 3.10 3.996ee350 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.683 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.283521+0000", 2024-09-16T20:49:51.683 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.010809914, 2024-09-16T20:49:51.684 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.015144972, 2024-09-16T20:49:51.684 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.684 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.684 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.684 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.684 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.684 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.684 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18965 2024-09-16T20:49:51.684 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.684 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.684 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.684 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.684 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283521+0000", 2024-09-16T20:49:51.684 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.685 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.685 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.685 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.685 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283521+0000", 2024-09-16T20:49:51.685 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.685 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.685 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.685 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.685 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283522+0000", 2024-09-16T20:49:51.685 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2799999999999996e-07 2024-09-16T20:49:51.685 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.685 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.685 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.686 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283524+0000", 2024-09-16T20:49:51.686 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.159e-06 2024-09-16T20:49:51.686 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.686 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.686 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.686 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283531+0000", 2024-09-16T20:49:51.686 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8460000000000002e-06 2024-09-16T20:49:51.686 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.686 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.686 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.686 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283537+0000", 2024-09-16T20:49:51.686 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4940000000000003e-06 2024-09-16T20:49:51.686 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.687 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.687 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.687 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.687 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.687 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18835 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.687 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.265479+0000", 2024-09-16T20:49:51.687 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.028852343999999999, 2024-09-16T20:49:51.687 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.033211314999999998, 2024-09-16T20:49:51.687 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.687 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.687 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.687 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.687 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.688 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.688 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18835 2024-09-16T20:49:51.688 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.688 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.688 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.688 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.688 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265479+0000", 2024-09-16T20:49:51.688 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.688 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.688 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.688 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.688 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265479+0000", 2024-09-16T20:49:51.688 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.688 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.689 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.689 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.689 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265479+0000", 2024-09-16T20:49:51.689 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.4900000000000002e-07 2024-09-16T20:49:51.689 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.689 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.689 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.689 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265481+0000", 2024-09-16T20:49:51.689 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2219999999999999e-06 2024-09-16T20:49:51.689 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.689 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.689 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.689 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265484+0000", 2024-09-16T20:49:51.689 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1719999999999999e-06 2024-09-16T20:49:51.690 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.690 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.690 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.690 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265486+0000", 2024-09-16T20:49:51.690 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7360000000000001e-06 2024-09-16T20:49:51.690 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.690 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.690 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.690 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.690 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.690 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18870 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.690 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.271292+0000", 2024-09-16T20:49:51.690 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.023039649999999998, 2024-09-16T20:49:51.690 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.027435767, 2024-09-16T20:49:51.690 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.691 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.691 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.691 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.691 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.691 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.691 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18870 2024-09-16T20:49:51.691 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.691 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.691 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.691 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.691 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271292+0000", 2024-09-16T20:49:51.691 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.691 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.691 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.691 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.692 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271292+0000", 2024-09-16T20:49:51.692 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.692 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.692 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.692 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.692 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271292+0000", 2024-09-16T20:49:51.692 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6400000000000003e-07 2024-09-16T20:49:51.692 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.692 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.692 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.692 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271295+0000", 2024-09-16T20:49:51.692 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3580000000000001e-06 2024-09-16T20:49:51.692 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.692 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.693 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.693 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271301+0000", 2024-09-16T20:49:51.693 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3069999999999999e-06 2024-09-16T20:49:51.693 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.693 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.693 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.693 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271306+0000", 2024-09-16T20:49:51.693 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4160000000000003e-06 2024-09-16T20:49:51.693 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.693 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.693 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.693 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.693 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.694 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18901 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.694 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.274664+0000", 2024-09-16T20:49:51.694 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.019667750000000001, 2024-09-16T20:49:51.694 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.024110619, 2024-09-16T20:49:51.694 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.694 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.694 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.694 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.694 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.694 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.694 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18901 2024-09-16T20:49:51.694 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.694 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.694 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.694 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.695 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274664+0000", 2024-09-16T20:49:51.695 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.695 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.695 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.695 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.695 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274664+0000", 2024-09-16T20:49:51.695 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.695 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.695 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.695 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.695 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274664+0000", 2024-09-16T20:49:51.695 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8700000000000002e-07 2024-09-16T20:49:51.695 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.695 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.696 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.696 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274665+0000", 2024-09-16T20:49:51.696 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.387e-06 2024-09-16T20:49:51.696 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.696 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.696 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.696 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274669+0000", 2024-09-16T20:49:51.696 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4290000000000001e-06 2024-09-16T20:49:51.696 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.696 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.696 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.696 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274672+0000", 2024-09-16T20:49:51.696 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1939999999999998e-06 2024-09-16T20:49:51.696 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.697 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.697 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.697 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.697 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.697 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18934 3.10 3.996ee350 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.697 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.282616+0000", 2024-09-16T20:49:51.697 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.011715064000000001, 2024-09-16T20:49:51.697 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.016206539999999998, 2024-09-16T20:49:51.697 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.697 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.697 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.697 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.697 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.697 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.698 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18934 2024-09-16T20:49:51.698 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.698 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.698 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.698 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.698 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282616+0000", 2024-09-16T20:49:51.698 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.698 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.698 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.698 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.698 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282616+0000", 2024-09-16T20:49:51.698 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.698 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.699 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.699 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.699 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282617+0000", 2024-09-16T20:49:51.699 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9599999999999999e-07 2024-09-16T20:49:51.699 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.699 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.699 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.699 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282620+0000", 2024-09-16T20:49:51.699 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8700000000000001e-06 2024-09-16T20:49:51.699 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.699 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.699 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.699 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282627+0000", 2024-09-16T20:49:51.699 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5140000000000004e-06 2024-09-16T20:49:51.700 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.700 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.700 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.700 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282634+0000", 2024-09-16T20:49:51.700 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3629999999999999e-06 2024-09-16T20:49:51.700 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.700 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.700 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.700 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.700 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.700 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18966 3.10 3.996ee350 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.700 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.283544+0000", 2024-09-16T20:49:51.700 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.010787784, 2024-09-16T20:49:51.700 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.015327327, 2024-09-16T20:49:51.701 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.701 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.701 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.701 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.701 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.701 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.701 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18966 2024-09-16T20:49:51.701 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.701 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.701 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.701 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.701 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283544+0000", 2024-09-16T20:49:51.701 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.701 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.702 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.702 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.702 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283544+0000", 2024-09-16T20:49:51.702 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.702 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.702 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.702 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.702 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283544+0000", 2024-09-16T20:49:51.702 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8100000000000003e-07 2024-09-16T20:49:51.702 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.702 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.702 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.702 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283552+0000", 2024-09-16T20:49:51.702 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.3899999999999993e-06 2024-09-16T20:49:51.703 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.703 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.703 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.703 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283561+0000", 2024-09-16T20:49:51.703 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.3909999999999998e-06 2024-09-16T20:49:51.703 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.703 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.703 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.703 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283567+0000", 2024-09-16T20:49:51.703 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7570000000000001e-06 2024-09-16T20:49:51.703 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.703 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.703 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.703 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.704 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.704 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18836 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.704 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.265491+0000", 2024-09-16T20:49:51.704 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.028840596, 2024-09-16T20:49:51.704 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.033423969999999997, 2024-09-16T20:49:51.704 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.704 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.704 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.704 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.704 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.704 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.704 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18836 2024-09-16T20:49:51.704 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.705 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.705 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.705 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.705 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265491+0000", 2024-09-16T20:49:51.705 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.705 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.705 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.705 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.705 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265491+0000", 2024-09-16T20:49:51.705 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.705 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.705 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.705 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.705 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265491+0000", 2024-09-16T20:49:51.706 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2100000000000001e-07 2024-09-16T20:49:51.706 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.706 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.706 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.706 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265492+0000", 2024-09-16T20:49:51.706 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0690000000000001e-06 2024-09-16T20:49:51.706 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.706 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.706 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.706 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265497+0000", 2024-09-16T20:49:51.706 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5739999999999997e-06 2024-09-16T20:49:51.706 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.706 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.707 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.707 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265499+0000", 2024-09-16T20:49:51.707 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7300000000000001e-06 2024-09-16T20:49:51.707 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.707 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.707 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.707 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.707 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.707 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18871 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.707 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.271314+0000", 2024-09-16T20:49:51.707 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.023017727000000002, 2024-09-16T20:49:51.707 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.027626291000000001, 2024-09-16T20:49:51.707 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.707 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.707 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.708 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.708 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.708 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.708 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18871 2024-09-16T20:49:51.708 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.708 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.708 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.708 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.708 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271314+0000", 2024-09-16T20:49:51.708 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.708 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.708 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.708 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.709 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271314+0000", 2024-09-16T20:49:51.709 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.709 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.709 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.709 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.709 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271314+0000", 2024-09-16T20:49:51.709 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3700000000000001e-07 2024-09-16T20:49:51.709 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.709 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.709 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.709 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271316+0000", 2024-09-16T20:49:51.709 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9810000000000002e-06 2024-09-16T20:49:51.710 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.710 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.710 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.710 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271322+0000", 2024-09-16T20:49:51.710 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1800000000000001e-06 2024-09-16T20:49:51.710 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.710 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.710 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.710 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271329+0000", 2024-09-16T20:49:51.710 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0589999999999997e-06 2024-09-16T20:49:51.710 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.710 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.710 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.711 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.711 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.711 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18902 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.711 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.274675+0000", 2024-09-16T20:49:51.711 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.019655954, 2024-09-16T20:49:51.711 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.024288358999999999, 2024-09-16T20:49:51.711 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.711 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.711 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.711 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.711 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.711 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.711 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18902 2024-09-16T20:49:51.711 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.712 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.712 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.712 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.712 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274675+0000", 2024-09-16T20:49:51.712 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.712 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.712 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.712 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.712 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274675+0000", 2024-09-16T20:49:51.712 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.712 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.712 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.713 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.713 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274676+0000", 2024-09-16T20:49:51.713 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7500000000000001e-07 2024-09-16T20:49:51.713 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.713 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.713 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.713 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274677+0000", 2024-09-16T20:49:51.713 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3829999999999999e-06 2024-09-16T20:49:51.713 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.713 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.713 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.713 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274680+0000", 2024-09-16T20:49:51.713 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2780000000000002e-06 2024-09-16T20:49:51.713 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.713 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.714 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.714 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274683+0000", 2024-09-16T20:49:51.714 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9019999999999999e-06 2024-09-16T20:49:51.714 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.714 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.714 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.714 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.714 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.714 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18935 3.10 3.996ee350 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.714 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.282643+0000", 2024-09-16T20:49:51.714 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.011688089, 2024-09-16T20:49:51.714 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.016344134, 2024-09-16T20:49:51.714 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.714 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.714 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.715 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.715 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.715 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.715 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18935 2024-09-16T20:49:51.715 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.715 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.715 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.715 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.715 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282643+0000", 2024-09-16T20:49:51.715 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.715 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.715 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.715 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.715 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282643+0000", 2024-09-16T20:49:51.715 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.715 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.716 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.716 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.716 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282644+0000", 2024-09-16T20:49:51.716 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1799999999999995e-07 2024-09-16T20:49:51.716 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.716 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.716 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.716 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282646+0000", 2024-09-16T20:49:51.716 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6110000000000001e-06 2024-09-16T20:49:51.716 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.716 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.716 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.716 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282654+0000", 2024-09-16T20:49:51.716 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2189999999999998e-06 2024-09-16T20:49:51.716 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.717 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.717 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.717 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282659+0000", 2024-09-16T20:49:51.717 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3720000000000004e-06 2024-09-16T20:49:51.717 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.717 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.717 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.717 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.717 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.717 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18967 3.10 3.996ee350 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.717 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.283575+0000", 2024-09-16T20:49:51.717 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.010756594, 2024-09-16T20:49:51.717 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.015447004, 2024-09-16T20:49:51.717 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.717 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.718 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.718 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.718 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.718 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.718 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18967 2024-09-16T20:49:51.718 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.718 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.718 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.718 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.718 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283575+0000", 2024-09-16T20:49:51.718 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.718 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.718 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.719 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.719 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283575+0000", 2024-09-16T20:49:51.719 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.719 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.719 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.719 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.719 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283575+0000", 2024-09-16T20:49:51.719 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7100000000000002e-07 2024-09-16T20:49:51.719 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.719 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.719 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.719 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283578+0000", 2024-09-16T20:49:51.719 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2869999999999999e-06 2024-09-16T20:49:51.719 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.720 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.720 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.720 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283585+0000", 2024-09-16T20:49:51.720 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.939e-06 2024-09-16T20:49:51.720 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.720 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.720 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.720 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283592+0000", 2024-09-16T20:49:51.720 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7589999999999997e-06 2024-09-16T20:49:51.720 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.720 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.720 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.720 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.720 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.720 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18837 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.721 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.265503+0000", 2024-09-16T20:49:51.721 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.028828777, 2024-09-16T20:49:51.721 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.033566612000000003, 2024-09-16T20:49:51.721 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.721 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.721 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.721 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.721 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.721 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.721 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18837 2024-09-16T20:49:51.721 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.721 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.721 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.721 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.721 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265503+0000", 2024-09-16T20:49:51.722 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.722 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.722 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.722 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.722 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265503+0000", 2024-09-16T20:49:51.722 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.722 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.722 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.722 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.722 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265503+0000", 2024-09-16T20:49:51.722 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3999999999999998e-07 2024-09-16T20:49:51.722 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.722 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.722 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.722 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265504+0000", 2024-09-16T20:49:51.722 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0130000000000001e-06 2024-09-16T20:49:51.723 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.723 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.723 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.723 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265507+0000", 2024-09-16T20:49:51.723 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9890000000000001e-06 2024-09-16T20:49:51.723 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.723 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.723 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.723 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265510+0000", 2024-09-16T20:49:51.723 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6869999999999999e-06 2024-09-16T20:49:51.723 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.723 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.723 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.723 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.724 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.724 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18872 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.724 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.271336+0000", 2024-09-16T20:49:51.724 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.022995288999999999, 2024-09-16T20:49:51.724 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.027786109, 2024-09-16T20:49:51.724 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.724 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.724 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.724 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.724 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.724 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.724 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18872 2024-09-16T20:49:51.724 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.724 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.724 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.725 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.725 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271336+0000", 2024-09-16T20:49:51.725 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.725 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.725 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.725 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.725 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271336+0000", 2024-09-16T20:49:51.725 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.725 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.725 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.725 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.725 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271337+0000", 2024-09-16T20:49:51.725 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0999999999999999e-07 2024-09-16T20:49:51.725 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.725 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.726 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.726 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271339+0000", 2024-09-16T20:49:51.726 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9879999999999999e-06 2024-09-16T20:49:51.726 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.726 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.726 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.726 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271345+0000", 2024-09-16T20:49:51.726 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.263e-06 2024-09-16T20:49:51.726 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.726 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.726 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.726 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271350+0000", 2024-09-16T20:49:51.726 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.271e-06 2024-09-16T20:49:51.726 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.726 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.727 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.727 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.727 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.727 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18903 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.727 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.274687+0000", 2024-09-16T20:49:51.727 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.019644838000000001, 2024-09-16T20:49:51.727 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.024483997, 2024-09-16T20:49:51.727 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.727 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.727 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.727 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.727 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.727 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.728 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18903 2024-09-16T20:49:51.728 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.728 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.728 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.728 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.728 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274687+0000", 2024-09-16T20:49:51.728 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.728 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.728 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.728 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.728 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274687+0000", 2024-09-16T20:49:51.728 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.728 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.729 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.729 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.729 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274687+0000", 2024-09-16T20:49:51.729 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5100000000000001e-07 2024-09-16T20:49:51.729 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.729 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.729 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.729 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274688+0000", 2024-09-16T20:49:51.729 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1930000000000001e-06 2024-09-16T20:49:51.729 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.729 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.729 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.729 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274691+0000", 2024-09-16T20:49:51.729 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1810000000000001e-06 2024-09-16T20:49:51.730 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.730 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.730 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.730 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274694+0000", 2024-09-16T20:49:51.730 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6730000000000001e-06 2024-09-16T20:49:51.730 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.730 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.730 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.730 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.730 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.730 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18936 3.10 3.996ee350 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.730 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.282689+0000", 2024-09-16T20:49:51.730 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.011641975000000001, 2024-09-16T20:49:51.730 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.016527607999999999, 2024-09-16T20:49:51.731 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.731 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.731 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.731 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.731 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.731 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.731 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18936 2024-09-16T20:49:51.731 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.731 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.731 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.731 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.731 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282689+0000", 2024-09-16T20:49:51.731 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.731 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.731 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.732 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.732 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282689+0000", 2024-09-16T20:49:51.732 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.732 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.732 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.732 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.732 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282690+0000", 2024-09-16T20:49:51.732 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3499999999999996e-07 2024-09-16T20:49:51.732 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.732 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.732 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.732 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282693+0000", 2024-09-16T20:49:51.732 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1300000000000001e-06 2024-09-16T20:49:51.732 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.732 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.733 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.733 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282701+0000", 2024-09-16T20:49:51.733 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6369999999999999e-06 2024-09-16T20:49:51.733 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.733 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.733 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.733 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282707+0000", 2024-09-16T20:49:51.733 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0519999999999998e-06 2024-09-16T20:49:51.733 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.733 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.733 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.733 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.733 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.733 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18968 3.10 3.996ee350 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.733 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.283599+0000", 2024-09-16T20:49:51.733 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.010732332000000001, 2024-09-16T20:49:51.734 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.015645939000000001, 2024-09-16T20:49:51.734 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.734 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.734 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.734 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.734 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.734 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.734 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18968 2024-09-16T20:49:51.734 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.734 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.734 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.734 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.734 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283599+0000", 2024-09-16T20:49:51.734 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.734 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.735 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.735 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.735 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283599+0000", 2024-09-16T20:49:51.735 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.735 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.735 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.735 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.735 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283600+0000", 2024-09-16T20:49:51.735 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4799999999999999e-07 2024-09-16T20:49:51.735 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.735 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.735 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.735 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283602+0000", 2024-09-16T20:49:51.735 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3479999999999998e-06 2024-09-16T20:49:51.735 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.736 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.736 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.736 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283610+0000", 2024-09-16T20:49:51.736 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0069999999999997e-06 2024-09-16T20:49:51.736 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.736 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.736 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.736 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283615+0000", 2024-09-16T20:49:51.736 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.366e-06 2024-09-16T20:49:51.736 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.736 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.736 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.736 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.736 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.736 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18838 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.736 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.265531+0000", 2024-09-16T20:49:51.737 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.028800586, 2024-09-16T20:49:51.737 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.033738845000000003, 2024-09-16T20:49:51.737 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.737 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.737 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.737 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.737 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.737 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.737 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18838 2024-09-16T20:49:51.737 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.737 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.737 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.737 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.737 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265531+0000", 2024-09-16T20:49:51.738 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.738 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.738 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.738 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.738 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265531+0000", 2024-09-16T20:49:51.738 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.738 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.738 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.738 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.738 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265531+0000", 2024-09-16T20:49:51.738 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1800000000000001e-07 2024-09-16T20:49:51.738 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.738 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.738 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.738 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265533+0000", 2024-09-16T20:49:51.739 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3909999999999999e-06 2024-09-16T20:49:51.739 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.739 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.739 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.739 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265536+0000", 2024-09-16T20:49:51.739 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3500000000000001e-06 2024-09-16T20:49:51.739 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.739 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.739 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.739 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265539+0000", 2024-09-16T20:49:51.739 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8499999999999998e-06 2024-09-16T20:49:51.739 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.739 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.739 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.739 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.740 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.740 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18873 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.740 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.271357+0000", 2024-09-16T20:49:51.740 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.022974771000000001, 2024-09-16T20:49:51.740 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.027936542000000002, 2024-09-16T20:49:51.740 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.740 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.740 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.740 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.740 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.740 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.740 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18873 2024-09-16T20:49:51.740 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.741 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.741 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.741 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.741 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271357+0000", 2024-09-16T20:49:51.741 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.741 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.741 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.741 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.741 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271357+0000", 2024-09-16T20:49:51.741 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.741 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.741 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.741 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.741 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271357+0000", 2024-09-16T20:49:51.741 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.41e-07 2024-09-16T20:49:51.742 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.742 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.742 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.742 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271359+0000", 2024-09-16T20:49:51.742 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0880000000000002e-06 2024-09-16T20:49:51.742 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.742 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.742 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.742 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271365+0000", 2024-09-16T20:49:51.742 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0800000000000002e-06 2024-09-16T20:49:51.742 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.742 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.742 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.742 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271371+0000", 2024-09-16T20:49:51.742 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0050000000000001e-06 2024-09-16T20:49:51.743 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.743 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.743 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.743 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.743 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.743 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18904 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.743 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.274720+0000", 2024-09-16T20:49:51.743 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.019611361000000001, 2024-09-16T20:49:51.743 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.02460611, 2024-09-16T20:49:51.743 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.743 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.743 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.743 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.743 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.744 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.744 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18904 2024-09-16T20:49:51.744 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.744 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.744 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.744 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.744 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274720+0000", 2024-09-16T20:49:51.744 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.744 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.744 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.744 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.744 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274720+0000", 2024-09-16T20:49:51.744 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.745 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.745 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.745 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.745 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274720+0000", 2024-09-16T20:49:51.745 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.72e-07 2024-09-16T20:49:51.745 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.745 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.745 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.745 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274725+0000", 2024-09-16T20:49:51.745 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0089999999999998e-06 2024-09-16T20:49:51.745 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.745 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.745 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.745 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274729+0000", 2024-09-16T20:49:51.745 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9550000000000002e-06 2024-09-16T20:49:51.746 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.746 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.746 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.746 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274733+0000", 2024-09-16T20:49:51.746 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1370000000000002e-06 2024-09-16T20:49:51.746 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.746 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.746 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.746 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.746 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.746 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18937 3.10 3.996ee350 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.746 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.282716+0000", 2024-09-16T20:49:51.746 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.011615851, 2024-09-16T20:49:51.746 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.016657581000000001, 2024-09-16T20:49:51.746 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.747 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.747 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.747 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.747 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.747 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.747 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18937 2024-09-16T20:49:51.747 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.747 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.747 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.747 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.747 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282716+0000", 2024-09-16T20:49:51.747 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.747 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.748 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.748 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.748 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282716+0000", 2024-09-16T20:49:51.748 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.748 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.748 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.748 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.748 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282716+0000", 2024-09-16T20:49:51.748 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9500000000000003e-07 2024-09-16T20:49:51.748 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.748 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.748 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.748 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282718+0000", 2024-09-16T20:49:51.749 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.317e-06 2024-09-16T20:49:51.749 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.749 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.749 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.749 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282725+0000", 2024-09-16T20:49:51.749 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2079999999999997e-06 2024-09-16T20:49:51.749 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.749 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.749 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.749 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282730+0000", 2024-09-16T20:49:51.749 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3480000000000003e-06 2024-09-16T20:49:51.749 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.749 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.749 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.750 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.750 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.750 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18969 3.10 3.996ee350 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.750 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.283628+0000", 2024-09-16T20:49:51.750 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.010702955, 2024-09-16T20:49:51.750 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.015793450000000001, 2024-09-16T20:49:51.750 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.750 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.750 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.750 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.750 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.750 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.750 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18969 2024-09-16T20:49:51.750 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.750 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.751 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.751 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.751 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283628+0000", 2024-09-16T20:49:51.751 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.751 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.751 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.751 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.751 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283628+0000", 2024-09-16T20:49:51.751 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.751 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.751 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.751 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.751 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283629+0000", 2024-09-16T20:49:51.752 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5699999999999998e-07 2024-09-16T20:49:51.752 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.752 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.752 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.752 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283631+0000", 2024-09-16T20:49:51.752 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2520000000000002e-06 2024-09-16T20:49:51.752 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.752 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.752 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.752 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283639+0000", 2024-09-16T20:49:51.752 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4989999999999997e-06 2024-09-16T20:49:51.752 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.752 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.753 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.753 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.283644+0000", 2024-09-16T20:49:51.753 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5990000000000001e-06 2024-09-16T20:49:51.753 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.753 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.753 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.753 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.753 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.753 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18839 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.753 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.265542+0000", 2024-09-16T20:49:51.753 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.028789084, 2024-09-16T20:49:51.753 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.033927820999999997, 2024-09-16T20:49:51.753 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.754 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.754 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.754 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.754 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.754 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.754 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18839 2024-09-16T20:49:51.754 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.754 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.754 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.754 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.754 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265542+0000", 2024-09-16T20:49:51.754 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.754 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.754 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.755 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.755 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265542+0000", 2024-09-16T20:49:51.755 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.755 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.755 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.755 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.755 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265543+0000", 2024-09-16T20:49:51.755 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9900000000000002e-07 2024-09-16T20:49:51.755 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.755 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.755 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.755 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265544+0000", 2024-09-16T20:49:51.755 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.046e-06 2024-09-16T20:49:51.755 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.755 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.756 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.756 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265547+0000", 2024-09-16T20:49:51.756 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9720000000000001e-06 2024-09-16T20:49:51.756 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.756 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.756 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.756 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265550+0000", 2024-09-16T20:49:51.756 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2830000000000002e-06 2024-09-16T20:49:51.756 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.756 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.756 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.756 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.756 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.756 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18874 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.757 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.271378+0000", 2024-09-16T20:49:51.757 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.022953488000000001, 2024-09-16T20:49:51.757 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.028139094, 2024-09-16T20:49:51.757 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.757 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.757 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.757 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.757 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.757 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.757 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18874 2024-09-16T20:49:51.757 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.757 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.757 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.757 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.757 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271378+0000", 2024-09-16T20:49:51.758 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.758 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.758 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.758 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.758 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271378+0000", 2024-09-16T20:49:51.758 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.758 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.758 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.758 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.758 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271378+0000", 2024-09-16T20:49:51.758 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.77e-07 2024-09-16T20:49:51.758 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.758 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.758 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.758 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271380+0000", 2024-09-16T20:49:51.758 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.928e-06 2024-09-16T20:49:51.759 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.759 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.759 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.759 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271386+0000", 2024-09-16T20:49:51.759 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9440000000000002e-06 2024-09-16T20:49:51.759 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.759 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.759 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.759 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271391+0000", 2024-09-16T20:49:51.759 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9459999999999997e-06 2024-09-16T20:49:51.759 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.759 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.759 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.760 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.760 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.760 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18905 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.760 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.274737+0000", 2024-09-16T20:49:51.760 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.019593978000000001, 2024-09-16T20:49:51.760 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.024807543000000001, 2024-09-16T20:49:51.760 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.760 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.760 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.760 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.760 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.760 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.760 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18905 2024-09-16T20:49:51.760 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.760 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.761 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.761 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.761 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274737+0000", 2024-09-16T20:49:51.761 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.761 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.761 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.761 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.761 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274737+0000", 2024-09-16T20:49:51.761 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.761 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.761 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.761 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.761 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274738+0000", 2024-09-16T20:49:51.761 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.77e-07 2024-09-16T20:49:51.761 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.762 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.762 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.762 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274739+0000", 2024-09-16T20:49:51.762 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.167e-06 2024-09-16T20:49:51.762 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.762 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.762 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.762 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274747+0000", 2024-09-16T20:49:51.762 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5089999999999993e-06 2024-09-16T20:49:51.762 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.762 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.762 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.762 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274763+0000", 2024-09-16T20:49:51.762 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5444e-05 2024-09-16T20:49:51.762 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.762 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.763 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.763 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.763 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.763 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18938 3.10 3.996ee350 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.763 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.282737+0000", 2024-09-16T20:49:51.763 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.011594339, 2024-09-16T20:49:51.763 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.016831353, 2024-09-16T20:49:51.763 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.763 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.763 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.763 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.763 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.763 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.763 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18938 2024-09-16T20:49:51.764 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.764 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.764 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.764 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.764 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282737+0000", 2024-09-16T20:49:51.764 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.764 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.764 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.764 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.764 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282737+0000", 2024-09-16T20:49:51.764 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.764 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.764 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.764 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.764 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282738+0000", 2024-09-16T20:49:51.765 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5499999999999998e-07 2024-09-16T20:49:51.765 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.765 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.765 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.765 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282740+0000", 2024-09-16T20:49:51.765 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2189999999999998e-06 2024-09-16T20:49:51.765 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.765 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.765 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.765 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282746+0000", 2024-09-16T20:49:51.765 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3439999999999997e-06 2024-09-16T20:49:51.765 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.765 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.765 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.765 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282751+0000", 2024-09-16T20:49:51.766 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1410000000000004e-06 2024-09-16T20:49:51.766 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.766 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.766 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.766 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.766 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.766 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18840 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.766 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.265553+0000", 2024-09-16T20:49:51.766 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.028778354999999999, 2024-09-16T20:49:51.766 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.034193660000000001, 2024-09-16T20:49:51.766 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.766 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.766 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.766 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.766 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.767 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.767 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18840 2024-09-16T20:49:51.767 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.767 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.767 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.767 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.767 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265553+0000", 2024-09-16T20:49:51.767 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.767 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.767 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.767 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.767 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265553+0000", 2024-09-16T20:49:51.767 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.767 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.768 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.768 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.768 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265553+0000", 2024-09-16T20:49:51.768 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.28e-07 2024-09-16T20:49:51.768 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.768 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.768 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.768 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265554+0000", 2024-09-16T20:49:51.768 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0219999999999999e-06 2024-09-16T20:49:51.768 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.768 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.768 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.768 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265558+0000", 2024-09-16T20:49:51.769 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.63e-06 2024-09-16T20:49:51.769 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.769 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.769 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.769 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265561+0000", 2024-09-16T20:49:51.769 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6199999999999999e-06 2024-09-16T20:49:51.769 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.769 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.769 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.769 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.769 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.769 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18875 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.769 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.271398+0000", 2024-09-16T20:49:51.769 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.022933360999999999, 2024-09-16T20:49:51.769 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.028398850999999999, 2024-09-16T20:49:51.770 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.770 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.770 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.770 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.770 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.770 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.770 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18875 2024-09-16T20:49:51.770 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.770 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.770 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.770 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.770 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271398+0000", 2024-09-16T20:49:51.770 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.771 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.771 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.771 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.771 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271398+0000", 2024-09-16T20:49:51.771 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.771 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.771 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.771 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.771 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271398+0000", 2024-09-16T20:49:51.771 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7800000000000002e-07 2024-09-16T20:49:51.771 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.771 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.771 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.771 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271400+0000", 2024-09-16T20:49:51.771 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0310000000000001e-06 2024-09-16T20:49:51.772 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.772 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.772 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.772 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271406+0000", 2024-09-16T20:49:51.772 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8839999999999999e-06 2024-09-16T20:49:51.772 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.772 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.772 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.772 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271411+0000", 2024-09-16T20:49:51.772 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8860000000000003e-06 2024-09-16T20:49:51.772 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.772 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.772 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.772 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.772 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.773 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18906 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.773 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.274775+0000", 2024-09-16T20:49:51.773 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.019555967, 2024-09-16T20:49:51.773 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.025067558, 2024-09-16T20:49:51.773 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.773 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.773 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.773 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.773 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.773 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.773 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18906 2024-09-16T20:49:51.773 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.773 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.773 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.773 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.774 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274775+0000", 2024-09-16T20:49:51.774 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.774 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.774 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.774 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.774 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274775+0000", 2024-09-16T20:49:51.774 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.774 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.774 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.774 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.774 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274776+0000", 2024-09-16T20:49:51.774 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.58e-07 2024-09-16T20:49:51.774 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.774 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.774 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.774 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274778+0000", 2024-09-16T20:49:51.775 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.911e-06 2024-09-16T20:49:51.775 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.775 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.775 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.775 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274783+0000", 2024-09-16T20:49:51.775 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3970000000000002e-06 2024-09-16T20:49:51.775 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.775 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.775 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.775 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274787+0000", 2024-09-16T20:49:51.775 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4350000000000001e-06 2024-09-16T20:49:51.775 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.775 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.775 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.776 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.776 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.776 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18939 3.10 3.996ee350 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.776 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.282761+0000", 2024-09-16T20:49:51.776 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.011570560000000001, 2024-09-16T20:49:51.776 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.017107503999999999, 2024-09-16T20:49:51.776 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.776 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.776 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.776 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.776 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.776 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.776 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18939 2024-09-16T20:49:51.776 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.777 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.777 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.777 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.777 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282761+0000", 2024-09-16T20:49:51.777 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.777 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.777 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.777 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.777 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282761+0000", 2024-09-16T20:49:51.777 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.777 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.777 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.777 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.777 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282761+0000", 2024-09-16T20:49:51.778 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.34e-07 2024-09-16T20:49:51.778 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.778 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.778 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.778 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282763+0000", 2024-09-16T20:49:51.778 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1090000000000001e-06 2024-09-16T20:49:51.778 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.778 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.778 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.778 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282769+0000", 2024-09-16T20:49:51.778 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9529999999999996e-06 2024-09-16T20:49:51.778 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.778 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.778 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.779 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282774+0000", 2024-09-16T20:49:51.779 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8620000000000002e-06 2024-09-16T20:49:51.779 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.779 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.779 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.779 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.779 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.779 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18841 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.779 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.265564+0000", 2024-09-16T20:49:51.779 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.028767792, 2024-09-16T20:49:51.779 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.034330724, 2024-09-16T20:49:51.779 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.779 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.780 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.780 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.780 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.780 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.780 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18841 2024-09-16T20:49:51.780 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.780 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.780 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.780 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.780 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265564+0000", 2024-09-16T20:49:51.780 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.780 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.780 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.781 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.781 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265564+0000", 2024-09-16T20:49:51.781 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.781 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.781 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.781 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.781 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265564+0000", 2024-09-16T20:49:51.781 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2499999999999999e-07 2024-09-16T20:49:51.781 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.781 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.781 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.781 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265565+0000", 2024-09-16T20:49:51.781 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.051e-06 2024-09-16T20:49:51.781 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.782 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.782 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.782 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265568+0000", 2024-09-16T20:49:51.782 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0070000000000002e-06 2024-09-16T20:49:51.782 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.782 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.782 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.782 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265570+0000", 2024-09-16T20:49:51.782 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5959999999999998e-06 2024-09-16T20:49:51.782 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.782 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.782 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.783 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.783 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.783 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18876 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.783 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.271433+0000", 2024-09-16T20:49:51.783 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.02289832, 2024-09-16T20:49:51.783 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.028485402999999999, 2024-09-16T20:49:51.783 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.783 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.783 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.783 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.783 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.783 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.783 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18876 2024-09-16T20:49:51.783 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.784 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.784 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.784 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.784 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271433+0000", 2024-09-16T20:49:51.784 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.784 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.784 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.784 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.784 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271433+0000", 2024-09-16T20:49:51.784 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.784 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.784 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.784 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.785 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271433+0000", 2024-09-16T20:49:51.785 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3200000000000001e-07 2024-09-16T20:49:51.785 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.785 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.785 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.785 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271435+0000", 2024-09-16T20:49:51.785 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8470000000000001e-06 2024-09-16T20:49:51.785 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.785 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.785 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.785 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271440+0000", 2024-09-16T20:49:51.785 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6419999999999998e-06 2024-09-16T20:49:51.785 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.785 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.786 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.786 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271443+0000", 2024-09-16T20:49:51.786 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3610000000000001e-06 2024-09-16T20:49:51.786 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.786 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.786 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.786 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.786 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.786 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18907 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.786 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.274791+0000", 2024-09-16T20:49:51.786 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.019540491, 2024-09-16T20:49:51.786 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.025173078000000002, 2024-09-16T20:49:51.786 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.786 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.786 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.787 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.787 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.787 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.787 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18907 2024-09-16T20:49:51.787 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.787 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.787 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.787 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.787 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274791+0000", 2024-09-16T20:49:51.787 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.787 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.787 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.787 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.788 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274791+0000", 2024-09-16T20:49:51.788 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.788 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.788 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.788 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.788 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274791+0000", 2024-09-16T20:49:51.788 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.35e-07 2024-09-16T20:49:51.788 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.788 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.788 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.788 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274792+0000", 2024-09-16T20:49:51.788 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.125e-06 2024-09-16T20:49:51.788 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.788 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.789 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.789 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274796+0000", 2024-09-16T20:49:51.789 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2729999999999998e-06 2024-09-16T20:49:51.789 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.789 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.789 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.789 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274798+0000", 2024-09-16T20:49:51.789 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7259999999999998e-06 2024-09-16T20:49:51.789 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.789 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.789 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.789 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.789 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.790 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18940 3.10 3.996ee350 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.790 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.282804+0000", 2024-09-16T20:49:51.790 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.011527824000000001, 2024-09-16T20:49:51.790 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.017208355000000002, 2024-09-16T20:49:51.790 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.790 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.790 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.790 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.790 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.790 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.790 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18940 2024-09-16T20:49:51.790 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.790 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.790 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.791 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.791 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282804+0000", 2024-09-16T20:49:51.791 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.791 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.791 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.791 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.791 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282804+0000", 2024-09-16T20:49:51.791 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.791 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.791 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.791 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.791 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282804+0000", 2024-09-16T20:49:51.791 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8999999999999997e-07 2024-09-16T20:49:51.791 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.792 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.792 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.792 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282806+0000", 2024-09-16T20:49:51.792 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2579999999999998e-06 2024-09-16T20:49:51.792 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.792 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.792 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.792 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282815+0000", 2024-09-16T20:49:51.792 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.3129999999999999e-06 2024-09-16T20:49:51.792 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.792 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.792 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.792 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282821+0000", 2024-09-16T20:49:51.792 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3400000000000003e-06 2024-09-16T20:49:51.792 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.793 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.793 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.793 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.793 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.793 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18842 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.793 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.265574+0000", 2024-09-16T20:49:51.793 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.028757707, 2024-09-16T20:49:51.793 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.034486747999999998, 2024-09-16T20:49:51.793 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.793 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.793 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.793 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.793 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.793 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.794 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18842 2024-09-16T20:49:51.794 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.794 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.794 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.794 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.794 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265574+0000", 2024-09-16T20:49:51.794 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.794 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.794 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.794 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.794 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265574+0000", 2024-09-16T20:49:51.794 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.794 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.795 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.795 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.795 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265574+0000", 2024-09-16T20:49:51.795 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2999999999999999e-07 2024-09-16T20:49:51.795 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.795 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.795 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.795 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265577+0000", 2024-09-16T20:49:51.795 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0529999999999998e-06 2024-09-16T20:49:51.795 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.795 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.795 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.795 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265582+0000", 2024-09-16T20:49:51.795 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5260000000000004e-06 2024-09-16T20:49:51.796 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.796 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.796 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.796 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265584+0000", 2024-09-16T20:49:51.796 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8159999999999998e-06 2024-09-16T20:49:51.796 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.796 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.796 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.796 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.796 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.796 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18877 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.796 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.271449+0000", 2024-09-16T20:49:51.796 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.022882023000000001, 2024-09-16T20:49:51.797 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.028668569000000001, 2024-09-16T20:49:51.797 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.797 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.797 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.797 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.797 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.797 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.797 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18877 2024-09-16T20:49:51.797 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.797 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.797 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.797 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.797 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271449+0000", 2024-09-16T20:49:51.798 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.798 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.798 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.798 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.798 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271449+0000", 2024-09-16T20:49:51.798 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.798 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.798 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.798 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.798 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271450+0000", 2024-09-16T20:49:51.798 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.03e-07 2024-09-16T20:49:51.798 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.798 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.799 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.799 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271451+0000", 2024-09-16T20:49:51.799 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.246e-06 2024-09-16T20:49:51.799 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.799 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.799 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.799 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271454+0000", 2024-09-16T20:49:51.799 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3110000000000001e-06 2024-09-16T20:49:51.799 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.799 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.799 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.799 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271457+0000", 2024-09-16T20:49:51.799 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6019999999999998e-06 2024-09-16T20:49:51.799 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.800 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.800 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.800 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.800 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.800 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18908 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.800 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.274801+0000", 2024-09-16T20:49:51.800 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.019529949000000001, 2024-09-16T20:49:51.800 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.025354747, 2024-09-16T20:49:51.800 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.800 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.800 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.800 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.800 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.801 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.801 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18908 2024-09-16T20:49:51.801 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.801 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.801 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.801 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.801 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274801+0000", 2024-09-16T20:49:51.801 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.801 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.801 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.801 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.801 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274801+0000", 2024-09-16T20:49:51.801 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.801 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.802 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.802 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.802 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274802+0000", 2024-09-16T20:49:51.802 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3900000000000001e-07 2024-09-16T20:49:51.802 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.802 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.802 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.802 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274803+0000", 2024-09-16T20:49:51.802 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.054e-06 2024-09-16T20:49:51.802 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.802 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.802 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.802 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274806+0000", 2024-09-16T20:49:51.802 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1360000000000001e-06 2024-09-16T20:49:51.802 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.803 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.803 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.803 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274808+0000", 2024-09-16T20:49:51.803 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5399999999999998e-06 2024-09-16T20:49:51.803 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.803 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.803 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.803 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.803 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.803 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18941 3.10 3.996ee350 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.803 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.282830+0000", 2024-09-16T20:49:51.803 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.011501332, 2024-09-16T20:49:51.803 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.017353772, 2024-09-16T20:49:51.803 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.803 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.804 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.804 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.804 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.804 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.804 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18941 2024-09-16T20:49:51.804 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.804 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.804 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.804 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.804 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282830+0000", 2024-09-16T20:49:51.804 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.804 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.804 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.804 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.804 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282830+0000", 2024-09-16T20:49:51.805 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.805 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.805 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.805 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.805 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282831+0000", 2024-09-16T20:49:51.805 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8699999999999995e-07 2024-09-16T20:49:51.805 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.805 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.805 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.805 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282833+0000", 2024-09-16T20:49:51.805 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.373e-06 2024-09-16T20:49:51.805 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.805 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.805 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.805 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282841+0000", 2024-09-16T20:49:51.806 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7989999999999994e-06 2024-09-16T20:49:51.806 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.806 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.806 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.806 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282846+0000", 2024-09-16T20:49:51.806 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4960000000000004e-06 2024-09-16T20:49:51.806 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.806 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.806 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.806 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.806 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.806 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18843 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.806 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.265588+0000", 2024-09-16T20:49:51.806 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.028743865, 2024-09-16T20:49:51.806 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.034621159999999998, 2024-09-16T20:49:51.807 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.807 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.807 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.807 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.807 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.807 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.807 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18843 2024-09-16T20:49:51.807 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.807 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.807 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.807 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.807 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265588+0000", 2024-09-16T20:49:51.807 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.807 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.808 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.808 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.808 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265588+0000", 2024-09-16T20:49:51.808 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.808 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.808 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.808 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.808 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265588+0000", 2024-09-16T20:49:51.808 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.22e-07 2024-09-16T20:49:51.808 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.808 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.808 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.809 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265589+0000", 2024-09-16T20:49:51.809 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1799999999999999e-06 2024-09-16T20:49:51.809 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.809 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.809 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.809 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265592+0000", 2024-09-16T20:49:51.809 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0180000000000002e-06 2024-09-16T20:49:51.809 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.809 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.809 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.809 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265595+0000", 2024-09-16T20:49:51.809 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.599e-06 2024-09-16T20:49:51.809 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.809 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.810 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.810 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.810 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.810 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18878 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.810 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.271460+0000", 2024-09-16T20:49:51.810 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.022871203, 2024-09-16T20:49:51.810 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.028779089000000001, 2024-09-16T20:49:51.810 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.810 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.810 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.810 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.810 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.810 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.810 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18878 2024-09-16T20:49:51.810 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.811 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.811 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.811 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.811 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271460+0000", 2024-09-16T20:49:51.811 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.811 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.811 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.811 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.811 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271460+0000", 2024-09-16T20:49:51.811 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.811 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.811 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.811 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.811 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271460+0000", 2024-09-16T20:49:51.811 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.23e-07 2024-09-16T20:49:51.812 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.812 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.812 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.812 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271461+0000", 2024-09-16T20:49:51.812 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0330000000000001e-06 2024-09-16T20:49:51.812 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.812 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.812 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.812 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271464+0000", 2024-09-16T20:49:51.812 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.038e-06 2024-09-16T20:49:51.812 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.812 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.812 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.813 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271467+0000", 2024-09-16T20:49:51.813 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7360000000000001e-06 2024-09-16T20:49:51.813 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.813 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.813 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.813 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.813 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.813 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18909 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.813 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.274811+0000", 2024-09-16T20:49:51.813 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.019519926, 2024-09-16T20:49:51.813 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.025474473000000001, 2024-09-16T20:49:51.813 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.813 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.813 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.813 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.814 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.814 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.814 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18909 2024-09-16T20:49:51.814 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.814 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.814 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.814 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.814 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274811+0000", 2024-09-16T20:49:51.814 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.814 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.814 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.814 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.814 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274811+0000", 2024-09-16T20:49:51.814 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.815 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.815 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.815 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.815 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274812+0000", 2024-09-16T20:49:51.815 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3799999999999999e-07 2024-09-16T20:49:51.815 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.815 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.815 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.815 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274813+0000", 2024-09-16T20:49:51.815 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.6899999999999996e-07 2024-09-16T20:49:51.815 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.815 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.815 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.815 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274817+0000", 2024-09-16T20:49:51.816 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6689999999999997e-06 2024-09-16T20:49:51.816 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.816 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.816 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.816 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274820+0000", 2024-09-16T20:49:51.816 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9119999999999998e-06 2024-09-16T20:49:51.816 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.816 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.816 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.816 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.816 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.816 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18942 3.10 3.996ee350 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.816 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.282854+0000", 2024-09-16T20:49:51.816 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.011477686000000001, 2024-09-16T20:49:51.817 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.017484011000000001, 2024-09-16T20:49:51.817 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.817 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.817 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.817 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.817 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.817 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.817 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18942 2024-09-16T20:49:51.817 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.817 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.817 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.817 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.817 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282854+0000", 2024-09-16T20:49:51.817 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.817 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.817 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.818 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.818 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282854+0000", 2024-09-16T20:49:51.818 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.818 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.818 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.818 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.818 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282854+0000", 2024-09-16T20:49:51.818 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7999999999999996e-07 2024-09-16T20:49:51.818 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.818 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.818 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.818 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282862+0000", 2024-09-16T20:49:51.818 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3220000000000004e-06 2024-09-16T20:49:51.818 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.819 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.819 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.819 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282870+0000", 2024-09-16T20:49:51.819 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5269999999999998e-06 2024-09-16T20:49:51.819 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.819 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.819 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.819 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282877+0000", 2024-09-16T20:49:51.819 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7399999999999998e-06 2024-09-16T20:49:51.819 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.819 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.819 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.820 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.820 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.820 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18844 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.820 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.265598+0000", 2024-09-16T20:49:51.820 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.028732938, 2024-09-16T20:49:51.820 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.034787361000000003, 2024-09-16T20:49:51.820 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.820 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.820 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.820 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.820 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.820 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.820 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18844 2024-09-16T20:49:51.821 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.821 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.821 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.821 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.821 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265598+0000", 2024-09-16T20:49:51.821 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.821 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.821 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.821 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.821 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265598+0000", 2024-09-16T20:49:51.821 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.821 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.821 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.822 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.822 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265599+0000", 2024-09-16T20:49:51.822 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1400000000000001e-07 2024-09-16T20:49:51.822 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.822 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.822 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.822 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265600+0000", 2024-09-16T20:49:51.822 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.7399999999999991e-07 2024-09-16T20:49:51.822 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.822 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.822 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.822 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265603+0000", 2024-09-16T20:49:51.822 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9749999999999999e-06 2024-09-16T20:49:51.822 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.823 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.823 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.823 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.265605+0000", 2024-09-16T20:49:51.823 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6620000000000001e-06 2024-09-16T20:49:51.823 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.823 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.823 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.823 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.823 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.823 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18879 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.823 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.271470+0000", 2024-09-16T20:49:51.823 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.022860953999999999, 2024-09-16T20:49:51.823 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.028965799, 2024-09-16T20:49:51.823 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.824 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.824 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.824 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.824 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.824 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.824 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18879 2024-09-16T20:49:51.824 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.824 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.824 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.824 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.824 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271470+0000", 2024-09-16T20:49:51.824 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.824 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.825 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.825 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.825 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271470+0000", 2024-09-16T20:49:51.825 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.825 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.825 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.825 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.825 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271471+0000", 2024-09-16T20:49:51.825 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2700000000000001e-07 2024-09-16T20:49:51.825 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.825 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.825 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.825 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271472+0000", 2024-09-16T20:49:51.825 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.08e-06 2024-09-16T20:49:51.825 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.826 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.826 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.826 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271475+0000", 2024-09-16T20:49:51.826 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6899999999999998e-06 2024-09-16T20:49:51.826 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.826 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.826 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.826 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.271478+0000", 2024-09-16T20:49:51.826 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5900000000000002e-06 2024-09-16T20:49:51.826 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.826 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.826 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.826 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.826 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.827 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18910 3.10 3.996ee350 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.827 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.274823+0000", 2024-09-16T20:49:51.827 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.019508060000000001, 2024-09-16T20:49:51.827 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.025641341000000002, 2024-09-16T20:49:51.827 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.827 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.827 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.827 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.827 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.827 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.827 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18910 2024-09-16T20:49:51.827 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.827 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.827 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.827 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.828 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274823+0000", 2024-09-16T20:49:51.828 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.828 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.828 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.828 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.828 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274823+0000", 2024-09-16T20:49:51.828 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.828 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.828 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.828 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.828 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274824+0000", 2024-09-16T20:49:51.828 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.72e-07 2024-09-16T20:49:51.828 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.829 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.829 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.829 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274825+0000", 2024-09-16T20:49:51.829 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0699999999999999e-06 2024-09-16T20:49:51.829 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.829 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.829 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.829 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274829+0000", 2024-09-16T20:49:51.829 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8909999999999997e-06 2024-09-16T20:49:51.829 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.829 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.829 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.829 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.274831+0000", 2024-09-16T20:49:51.830 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6579999999999998e-06 2024-09-16T20:49:51.830 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.830 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.830 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.830 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.830 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.830 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:18943 3.10 3.996ee350 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:51.830 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:51.282884+0000", 2024-09-16T20:49:51.830 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.011447084999999999, 2024-09-16T20:49:51.830 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.017608314999999999, 2024-09-16T20:49:51.830 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:51.830 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:51.830 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:51.830 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:51.831 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:51.831 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:51.831 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 18943 2024-09-16T20:49:51.831 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.831 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:51.831 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.831 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:51.831 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282884+0000", 2024-09-16T20:49:51.831 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.831 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.831 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.831 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:51.831 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282884+0000", 2024-09-16T20:49:51.832 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:51.832 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.832 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.832 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:51.832 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282885+0000", 2024-09-16T20:49:51.832 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5400000000000002e-07 2024-09-16T20:49:51.832 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.832 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.832 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:51.832 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282887+0000", 2024-09-16T20:49:51.832 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0700000000000001e-06 2024-09-16T20:49:51.832 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.832 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.832 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:51.832 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282893+0000", 2024-09-16T20:49:51.833 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5440000000000004e-06 2024-09-16T20:49:51.833 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:51.833 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:51.833 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:51.833 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:51.282898+0000", 2024-09-16T20:49:51.833 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0939999999999998e-06 2024-09-16T20:49:51.833 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.833 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:51.833 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.833 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:51.833 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-16T20:49:51.833 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 156 2024-09-16T20:49:51.833 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:51.834 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T20:49:51.834 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T20:49:51.836 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:51.833+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:51.837 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:49:51.840 INFO:teuthology.orchestra.run.smithi139.stdout:4132 write 0x295b600 thru 0x29680dc (0xcadd bytes) 2024-09-16T20:49:51.840 INFO:teuthology.orchestra.run.smithi139.stdout:4134 write 0x9adcc00 thru 0x9ae841f (0xb820 bytes) 2024-09-16T20:49:51.840 INFO:teuthology.orchestra.run.smithi139.stdout:4136 read 0x8276200 thru 0x827a1f3 (0x3ff4 bytes) 2024-09-16T20:49:51.840 INFO:teuthology.orchestra.run.smithi139.stdout:4143 punch from 0x88be000 to 0x88cde00, (0xfe00 bytes) 2024-09-16T20:49:51.840 INFO:teuthology.orchestra.run.smithi139.stdout:4145 write 0x9a4bc00 thru 0x9a4fa76 (0x3e77 bytes) 2024-09-16T20:49:51.840 INFO:teuthology.orchestra.run.smithi139.stdout:4146 trunc from 0xea13a00 to 0x47b1200 2024-09-16T20:49:51.840 INFO:teuthology.orchestra.run.smithi139.stdout:4147 trunc from 0x47b1200 to 0xd2a7600 2024-09-16T20:49:51.840 INFO:teuthology.orchestra.run.smithi139.stdout:4148 write 0x15cb000 thru 0x15d83f9 (0xd3fa bytes) 2024-09-16T20:49:51.840 INFO:teuthology.orchestra.run.smithi139.stdout:4149 trunc from 0xd2a7600 to 0x7a40e00 2024-09-16T20:49:51.840 INFO:teuthology.orchestra.run.smithi139.stdout:4150 trunc from 0x7a40e00 to 0xa590c00 2024-09-16T20:49:51.841 INFO:teuthology.orchestra.run.smithi139.stdout:4151 trunc from 0xa590c00 to 0x49b2000 2024-09-16T20:49:51.841 INFO:teuthology.orchestra.run.smithi139.stdout:4152 trunc from 0x49b2000 to 0xa538e00 2024-09-16T20:49:51.841 INFO:teuthology.orchestra.run.smithi139.stdout:4153 write 0x3420c00 thru 0x342182b (0xc2c bytes) 2024-09-16T20:49:51.841 INFO:teuthology.orchestra.run.smithi139.stdout:4154 read 0xe2b400 thru 0xe37aa9 (0xc6aa bytes) 2024-09-16T20:49:51.841 INFO:teuthology.orchestra.run.smithi139.stdout:4155 write 0x802ae00 thru 0x802bdcd (0xfce bytes) 2024-09-16T20:49:51.841 INFO:teuthology.orchestra.run.smithi139.stdout:4161 write 0x7d4c800 thru 0x7d53c1f (0x7420 bytes) 2024-09-16T20:49:51.841 INFO:teuthology.orchestra.run.smithi139.stdout:4162 punch from 0x8add600 to 0x8ae7c00, (0xa600 bytes) 2024-09-16T20:49:51.841 INFO:teuthology.orchestra.run.smithi139.stdout:4164 trunc from 0xa538e00 to 0xed96c00 2024-09-16T20:49:51.841 INFO:teuthology.orchestra.run.smithi139.stdout:4165 trunc from 0xed96c00 to 0x9b61600 2024-09-16T20:49:51.841 INFO:teuthology.orchestra.run.smithi139.stdout:4168 trunc from 0x9b61600 to 0x2770c00 2024-09-16T20:49:51.841 INFO:teuthology.orchestra.run.smithi139.stdout:4169 read 0xca1c00 thru 0xcafb0f (0xdf10 bytes) 2024-09-16T20:49:51.841 INFO:teuthology.orchestra.run.smithi139.stdout:4170 punch from 0x1b8fe00 to 0x1b98000, (0x8200 bytes) 2024-09-16T20:49:51.841 INFO:teuthology.orchestra.run.smithi139.stdout:4171 punch from 0x1396000 to 0x139dc00, (0x7c00 bytes) 2024-09-16T20:49:51.842 INFO:teuthology.orchestra.run.smithi139.stdout:4172 write 0x82d5000 thru 0x82e208b (0xd08c bytes) 2024-09-16T20:49:51.842 INFO:teuthology.orchestra.run.smithi139.stdout:4174 trunc from 0x82e2200 to 0xeac1800 2024-09-16T20:49:51.842 INFO:teuthology.orchestra.run.smithi139.stdout:4176 read 0x2399200 thru 0x23a21d3 (0x8fd4 bytes) 2024-09-16T20:49:51.842 INFO:teuthology.orchestra.run.smithi139.stdout:4177 trunc from 0xeac1800 to 0xdd1000 2024-09-16T20:49:51.842 INFO:teuthology.orchestra.run.smithi139.stdout:4181 read 0x70000 thru 0x7e4e7 (0xe4e8 bytes) 2024-09-16T20:49:51.842 INFO:teuthology.orchestra.run.smithi139.stdout:4184 trunc from 0xdd1000 to 0x5c82600 2024-09-16T20:49:51.842 INFO:teuthology.orchestra.run.smithi139.stdout:4185 read 0x36c2000 thru 0x36c5ccc (0x3ccd bytes) 2024-09-16T20:49:51.842 INFO:teuthology.orchestra.run.smithi139.stdout:4190 trunc from 0x5c82600 to 0xaa05800 2024-09-16T20:49:51.842 INFO:teuthology.orchestra.run.smithi139.stdout:4193 punch from 0x3852400 to 0x385c800, (0xa400 bytes) 2024-09-16T20:49:51.842 INFO:teuthology.orchestra.run.smithi139.stdout:4195 read 0x5840600 thru 0x584322f (0x2c30 bytes) 2024-09-16T20:49:51.842 INFO:teuthology.orchestra.run.smithi139.stdout:4196 read 0xe06e00 thru 0xe13bb4 (0xcdb5 bytes) 2024-09-16T20:49:51.842 INFO:teuthology.orchestra.run.smithi139.stdout:4199 punch from 0x42ca000 to 0x42cf200, (0x5200 bytes) 2024-09-16T20:49:51.842 INFO:teuthology.orchestra.run.smithi139.stdout:4202 read 0x2eb5200 thru 0x2ec2031 (0xce32 bytes) 2024-09-16T20:49:51.842 INFO:teuthology.orchestra.run.smithi139.stdout:4204 write 0x732f400 thru 0x7338fe4 (0x9be5 bytes) 2024-09-16T20:49:51.842 INFO:teuthology.orchestra.run.smithi139.stdout:4205 write 0x51a2a00 thru 0x51a8b5d (0x615e bytes) 2024-09-16T20:49:51.843 INFO:teuthology.orchestra.run.smithi139.stdout:4206 write 0xea80e00 thru 0xea909f6 (0xfbf7 bytes) 2024-09-16T20:49:51.843 INFO:teuthology.orchestra.run.smithi139.stdout:4210 trunc from 0xea90a00 to 0x1479600 2024-09-16T20:49:51.843 INFO:teuthology.orchestra.run.smithi139.stdout:4212 read 0xa70000 thru 0xa76a8b (0x6a8c bytes) 2024-09-16T20:49:51.843 INFO:teuthology.orchestra.run.smithi139.stdout:4216 read 0x976a00 thru 0x97f133 (0x8734 bytes) 2024-09-16T20:49:51.843 INFO:teuthology.orchestra.run.smithi139.stdout:4218 read 0xd18000 thru 0xd1fb6a (0x7b6b bytes) 2024-09-16T20:49:51.848 INFO:teuthology.orchestra.run.smithi139.stdout:4219 trunc from 0x1479600 to 0xc4fb000 2024-09-16T20:49:51.857 INFO:teuthology.orchestra.run.smithi139.stdout:4223 write 0x6fbce00 thru 0x6fc50d9 (0x82da bytes) 2024-09-16T20:49:51.859 INFO:teuthology.orchestra.run.smithi139.stdout:4226 read 0x55eb800 thru 0x55edfd2 (0x27d3 bytes) 2024-09-16T20:49:51.877 INFO:teuthology.orchestra.run.smithi139.stdout:4227 write 0xcb33e00 thru 0xcb419e9 (0xdbea bytes) 2024-09-16T20:49:51.904 INFO:teuthology.orchestra.run.smithi139.stdout:4228 write 0xcc76000 thru 0xcc7d6eb (0x76ec bytes) 2024-09-16T20:49:51.910 INFO:teuthology.orchestra.run.smithi139.stdout:4229 read 0x8c25600 thru 0x8c30456 (0xae57 bytes) 2024-09-16T20:49:51.917 INFO:teuthology.orchestra.run.smithi139.stdout:4230 trunc from 0xcc7d800 to 0x7164600 2024-09-16T20:49:51.935 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:51.934+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:51.944 INFO:teuthology.orchestra.run.smithi139.stdout:4231 read 0x552c200 thru 0x5538531 (0xc332 bytes) 2024-09-16T20:49:51.951 INFO:teuthology.orchestra.run.smithi139.stdout:4236 punch from 0x60bec00 to 0x60cbe00, (0xd200 bytes) 2024-09-16T20:49:51.952 INFO:teuthology.orchestra.run.smithi139.stdout:4241 read 0x567da00 thru 0x5681646 (0x3c47 bytes) 2024-09-16T20:49:51.952 INFO:teuthology.orchestra.run.smithi139.stdout:4242 read 0x5077c00 thru 0x5079d8c (0x218d bytes) 2024-09-16T20:49:51.952 INFO:teuthology.orchestra.run.smithi139.stdout:4244 read 0x40e7a00 thru 0x40ea967 (0x2f68 bytes) 2024-09-16T20:49:51.952 INFO:teuthology.orchestra.run.smithi139.stdout:4249 write 0x1858600 thru 0x1868432 (0xfe33 bytes) 2024-09-16T20:49:51.957 INFO:teuthology.orchestra.run.smithi139.stdout:4252 trunc from 0x7164600 to 0x2995a00 2024-09-16T20:49:52.000 INFO:teuthology.orchestra.run.smithi139.stdout:4253 write 0x6582e00 thru 0x658c47c (0x967d bytes) 2024-09-16T20:49:52.000 INFO:teuthology.orchestra.run.smithi139.stdout:4254 punch from 0x39d1c00 to 0x39d9e00, (0x8200 bytes) 2024-09-16T20:49:52.014 INFO:teuthology.orchestra.run.smithi139.stdout:4255 trunc from 0x658c600 to 0xe0da800 2024-09-16T20:49:52.026 INFO:teuthology.orchestra.run.smithi139.stdout:4256 read 0x5db5000 thru 0x5dc18ec (0xc8ed bytes) 2024-09-16T20:49:52.034 INFO:teuthology.orchestra.run.smithi139.stdout:4257 punch from 0x2227600 to 0x2229e00, (0x2800 bytes) 2024-09-16T20:49:52.036 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:52.035+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:52.043 INFO:teuthology.orchestra.run.smithi139.stdout:4258 trunc from 0xe0da800 to 0x6597200 2024-09-16T20:49:52.048 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:49:52.048 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:49:52.058 INFO:teuthology.orchestra.run.smithi139.stdout:4260 read 0x2dffa00 thru 0x2e090cc (0x96cd bytes) 2024-09-16T20:49:52.059 INFO:teuthology.orchestra.run.smithi139.stdout:4264 read 0xa3a200 thru 0xa44fcc (0xadcd bytes) 2024-09-16T20:49:52.069 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:49:52.069 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:49:52.070 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:52.071 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:52.077 INFO:teuthology.orchestra.run.smithi139.stdout:4266 write 0xa378200 thru 0xa3859a8 (0xd7a9 bytes) 2024-09-16T20:49:52.078 INFO:teuthology.orchestra.run.smithi139.stdout:4268 trunc from 0xa385a00 to 0x9813400 2024-09-16T20:49:52.081 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:49:52.092 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:49:52.092 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:49:52.104 INFO:teuthology.orchestra.run.smithi139.stdout:4269 punch from 0x4a7dc00 to 0x4a89c00, (0xc000 bytes) 2024-09-16T20:49:52.105 INFO:teuthology.orchestra.run.smithi139.stdout:4270 read 0x6bde200 thru 0x6be0c0e (0x2a0f bytes) 2024-09-16T20:49:52.110 INFO:teuthology.orchestra.run.smithi139.stdout:4273 trunc from 0x9813400 to 0xdd98200 2024-09-16T20:49:52.123 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:49:52.123 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:49:52.125 INFO:teuthology.orchestra.run.smithi139.stdout:4274 punch from 0x44cb000 to 0x44d1c00, (0x6c00 bytes) 2024-09-16T20:49:52.127 INFO:teuthology.orchestra.run.smithi139.stdout:4276 read 0xf3f000 thru 0xf495d2 (0xa5d3 bytes) 2024-09-16T20:49:52.137 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:52.135+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:52.141 INFO:teuthology.orchestra.run.smithi139.stdout:4279 trunc from 0xdd98200 to 0x1b5d000 2024-09-16T20:49:52.146 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:49:52.146 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:49:52.168 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:49:52.168 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:49:52.170 INFO:teuthology.orchestra.run.smithi139.stdout:4281 read 0x173a00 thru 0x179ef7 (0x64f8 bytes) 2024-09-16T20:49:52.171 INFO:teuthology.orchestra.run.smithi139.stdout:4283 punch from 0x1757800 to 0x175d600, (0x5e00 bytes) 2024-09-16T20:49:52.172 INFO:teuthology.orchestra.run.smithi139.stdout:4285 read 0x15e4200 thru 0x15ed6f0 (0x94f1 bytes) 2024-09-16T20:49:52.176 INFO:teuthology.orchestra.run.smithi139.stdout:4286 trunc from 0x1b5d000 to 0x5b09000 2024-09-16T20:49:52.192 INFO:teuthology.orchestra.run.smithi139.stdout:4287 read 0x31a3200 thru 0x31a7d15 (0x4b16 bytes) 2024-09-16T20:49:52.193 INFO:teuthology.orchestra.run.smithi139.stdout:4290 punch from 0x4e3600 to 0x4e9800, (0x6200 bytes) 2024-09-16T20:49:52.197 INFO:teuthology.orchestra.run.smithi139.stdout:4291 trunc from 0x5b09000 to 0x8dc3e00 2024-09-16T20:49:52.211 INFO:teuthology.orchestra.run.smithi139.stdout:4292 read 0x357f600 thru 0x3589de4 (0xa7e5 bytes) 2024-09-16T20:49:52.218 INFO:teuthology.orchestra.run.smithi139.stdout:4294 punch from 0x5853400 to 0x5855600, (0x2200 bytes) 2024-09-16T20:49:52.219 INFO:teuthology.orchestra.run.smithi139.stdout:4295 read 0x64a6400 thru 0x64a99fc (0x35fd bytes) 2024-09-16T20:49:52.224 INFO:teuthology.orchestra.run.smithi139.stdout:4298 trunc from 0x8dc3e00 to 0x4bdae00 2024-09-16T20:49:52.229 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:52.229 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:52.238 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:52.237+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:52.239 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:49:52.245 INFO:teuthology.orchestra.run.smithi139.stdout:4299 read 0x10a4600 thru 0x10ab6fa (0x70fb bytes) 2024-09-16T20:49:52.245 INFO:teuthology.orchestra.run.smithi139.stdout:4300 write 0xf38600 thru 0xf3adc0 (0x27c1 bytes) 2024-09-16T20:49:52.246 INFO:teuthology.orchestra.run.smithi139.stdout:4301 trunc from 0x4bdae00 to 0x545e000 2024-09-16T20:49:52.264 INFO:teuthology.orchestra.run.smithi139.stdout:4302 read 0x245d000 thru 0x2460ef0 (0x3ef1 bytes) 2024-09-16T20:49:52.266 INFO:teuthology.orchestra.run.smithi139.stdout:4303 trunc from 0x545e000 to 0x3a00800 2024-09-16T20:49:52.277 INFO:teuthology.orchestra.run.smithi139.stdout:4304 write 0x2b73400 thru 0x2b81b91 (0xe792 bytes) 2024-09-16T20:49:52.284 INFO:teuthology.orchestra.run.smithi139.stdout:4306 read 0x194800 thru 0x1a3514 (0xed15 bytes) 2024-09-16T20:49:52.303 INFO:teuthology.orchestra.run.smithi139.stdout:4307 write 0x517da00 thru 0x518a54f (0xcb50 bytes) 2024-09-16T20:49:52.304 INFO:teuthology.orchestra.run.smithi139.stdout:4308 punch from 0x2642a00 to 0x2643a00, (0x1000 bytes) 2024-09-16T20:49:52.305 INFO:teuthology.orchestra.run.smithi139.stdout:4309 write 0x11d7e00 thru 0x11e58c7 (0xdac8 bytes) 2024-09-16T20:49:52.305 INFO:teuthology.orchestra.run.smithi139.stdout:4311 read 0x4f96a00 thru 0x4fa0b03 (0xa104 bytes) 2024-09-16T20:49:52.314 INFO:teuthology.orchestra.run.smithi139.stdout:4312 trunc from 0x518a600 to 0xca74600 2024-09-16T20:49:52.335 INFO:teuthology.orchestra.run.smithi139.stdout:4313 write 0x1040200 thru 0x104281e (0x261f bytes) 2024-09-16T20:49:52.335 INFO:teuthology.orchestra.run.smithi139.stdout:4315 write 0x2ad7e00 thru 0x2ade046 (0x6247 bytes) 2024-09-16T20:49:52.337 INFO:teuthology.orchestra.run.smithi139.stdout:4316 write 0x8831a00 thru 0x883fb64 (0xe165 bytes) 2024-09-16T20:49:52.337 INFO:teuthology.orchestra.run.smithi139.stdout:4319 read 0x53e0200 thru 0x53e334a (0x314b bytes) 2024-09-16T20:49:52.337 INFO:teuthology.orchestra.run.smithi139.stdout:4321 read 0x93e7400 thru 0x93e7587 (0x188 bytes) 2024-09-16T20:49:52.337 INFO:teuthology.orchestra.run.smithi139.stdout:4322 read 0xc9ea00 thru 0xcabb64 (0xd165 bytes) 2024-09-16T20:49:52.337 INFO:teuthology.orchestra.run.smithi139.stdout:4324 read 0x93a2800 thru 0x93a6bad (0x43ae bytes) 2024-09-16T20:49:52.337 INFO:teuthology.orchestra.run.smithi139.stdout:4325 read 0x3e38c00 thru 0x3e419ee (0x8def bytes) 2024-09-16T20:49:52.337 INFO:teuthology.orchestra.run.smithi139.stdout:4326 write 0x70f4a00 thru 0x70f8b80 (0x4181 bytes) 2024-09-16T20:49:52.337 INFO:teuthology.orchestra.run.smithi139.stdout:4329 punch from 0xc748400 to 0xc755400, (0xd000 bytes) 2024-09-16T20:49:52.338 INFO:teuthology.orchestra.run.smithi139.stdout:4330 read 0x817f400 thru 0x8188cca (0x98cb bytes) 2024-09-16T20:49:52.338 INFO:teuthology.orchestra.run.smithi139.stdout:4332 read 0x35e3800 thru 0x35e38dd (0xde bytes) 2024-09-16T20:49:52.338 INFO:teuthology.orchestra.run.smithi139.stdout:4333 read 0x7f49400 thru 0x7f58d6e (0xf96f bytes) 2024-09-16T20:49:52.339 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:52.337+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:52.349 INFO:teuthology.orchestra.run.smithi139.stdout:4334 trunc from 0xca74600 to 0x37a0000 2024-09-16T20:49:52.402 INFO:teuthology.orchestra.run.smithi139.stdout:4336 write 0x8ebe600 thru 0x8ecbb8a (0xd58b bytes) 2024-09-16T20:49:52.403 INFO:teuthology.orchestra.run.smithi139.stdout:4337 punch from 0x39e0a00 to 0x39e7c00, (0x7200 bytes) 2024-09-16T20:49:52.404 INFO:teuthology.orchestra.run.smithi139.stdout:4338 read 0x8ae6400 thru 0x8af05a5 (0xa1a6 bytes) 2024-09-16T20:49:52.406 INFO:teuthology.orchestra.run.smithi139.stdout:4341 trunc from 0x8ecbc00 to 0x77b9200 2024-09-16T20:49:52.415 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:52.415 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:52.426 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:49:52.445 INFO:teuthology.orchestra.run.smithi139.stdout:4342 write 0x9258600 thru 0x925f73e (0x713f bytes) 2024-09-16T20:49:52.446 INFO:teuthology.orchestra.run.smithi139.stdout:4346 read 0x22a3800 thru 0x22a9469 (0x5c6a bytes) 2024-09-16T20:49:52.446 INFO:teuthology.orchestra.run.smithi139.stdout:4347 trunc from 0x925f800 to 0x94dae00 2024-09-16T20:49:52.462 INFO:teuthology.orchestra.run.smithi139.stdout:4348 read 0x4bf1000 thru 0x4bfb46e (0xa46f bytes) 2024-09-16T20:49:52.469 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:52.468+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:52.484 INFO:teuthology.orchestra.run.smithi139.stdout:4351 write 0xc6f8200 thru 0xc700f5b (0x8d5c bytes) 2024-09-16T20:49:52.489 INFO:teuthology.orchestra.run.smithi139.stdout:4352 write 0x3377e00 thru 0x33870a8 (0xf2a9 bytes) 2024-09-16T20:49:52.489 INFO:teuthology.orchestra.run.smithi139.stdout:4354 read 0xea0e00 thru 0xea2888 (0x1a89 bytes) 2024-09-16T20:49:52.490 INFO:teuthology.orchestra.run.smithi139.stdout:4356 read 0x48f5a00 thru 0x48f9695 (0x3c96 bytes) 2024-09-16T20:49:52.490 INFO:teuthology.orchestra.run.smithi139.stdout:4359 read 0x62bbe00 thru 0x62c08b0 (0x4ab1 bytes) 2024-09-16T20:49:52.490 INFO:teuthology.orchestra.run.smithi139.stdout:4360 punch from 0x7176200 to 0x717ac00, (0x4a00 bytes) 2024-09-16T20:49:52.491 INFO:teuthology.orchestra.run.smithi139.stdout:4361 write 0x1a11000 thru 0x1a20ae6 (0xfae7 bytes) 2024-09-16T20:49:52.491 INFO:teuthology.orchestra.run.smithi139.stdout:4362 read 0x2e5d200 thru 0x2e69041 (0xbe42 bytes) 2024-09-16T20:49:52.491 INFO:teuthology.orchestra.run.smithi139.stdout:4368 read 0x1c96400 thru 0x1c990a4 (0x2ca5 bytes) 2024-09-16T20:49:52.491 INFO:teuthology.orchestra.run.smithi139.stdout:4375 clone 34 order 25 su 524288 sc 8 2024-09-16T20:49:52.541 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:52.540+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:52.549 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:52.549 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:52.558 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:49:52.623 INFO:teuthology.orchestra.run.smithi029.stderr:nodeep-scrub is unset 2024-09-16T20:49:52.641 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:52.640+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:52.713 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:52.713 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:52.719 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:49:52.763 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:52.762+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:52.830 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:52.830 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:52.839 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:49:52.881 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:52.879+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:52.943 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:52.942+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:52.989 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:52.989 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:52.997 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:49:53.006 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #32, image image_client.0-clone32 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent33 2024-09-16T20:49:53.044 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:53.043+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:53.113 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:53.113 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:53.122 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:49:53.144 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:53.143+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:53.245 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:53.243+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:53.279 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:53.280 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:53.289 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:49:53.346 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:53.345+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:53.436 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:53.436 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:53.444 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:49:53.447 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:53.445+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:53.470 INFO:teuthology.orchestra.run.smithi139.stdout:4377 write 0x6b74600 thru 0x6b7dbd4 (0x95d5 bytes) 2024-09-16T20:49:53.472 INFO:teuthology.orchestra.run.smithi139.stdout:4379 write 0xb29d400 thru 0xb2a31a3 (0x5da4 bytes) 2024-09-16T20:49:53.473 INFO:teuthology.orchestra.run.smithi139.stdout:4380 read 0x2c57200 thru 0x2c598ec (0x26ed bytes) 2024-09-16T20:49:53.473 INFO:teuthology.orchestra.run.smithi139.stdout:4382 write 0x8255c00 thru 0x8261bee (0xbfef bytes) 2024-09-16T20:49:53.475 INFO:teuthology.orchestra.run.smithi139.stdout:4387 trunc from 0xc701000 to 0xcc34a00 2024-09-16T20:49:53.516 INFO:teuthology.orchestra.run.smithi139.stdout:4388 write 0x4fb3e00 thru 0x4fbc949 (0x8b4a bytes) 2024-09-16T20:49:53.517 INFO:teuthology.orchestra.run.smithi139.stdout:4389 read 0x5291e00 thru 0x52a05d6 (0xe7d7 bytes) 2024-09-16T20:49:53.518 INFO:teuthology.orchestra.run.smithi139.stdout:4390 read 0x3a32c00 thru 0x3a3f91a (0xcd1b bytes) 2024-09-16T20:49:53.522 INFO:teuthology.orchestra.run.smithi139.stdout:4391 flatten 2024-09-16T20:49:53.548 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:53.547+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:53.565 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:53.565 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:53.575 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:49:53.579 INFO:teuthology.orchestra.run.smithi139.stdout:4396 punch from 0x2684c00 to 0x2688200, (0x3600 bytes) 2024-09-16T20:49:53.587 INFO:teuthology.orchestra.run.smithi139.stdout:4397 trunc from 0xcc34a00 to 0x680be00 2024-09-16T20:49:53.609 INFO:teuthology.orchestra.run.smithi139.stdout:4399 read 0x4916800 thru 0x4918b2c (0x232d bytes) 2024-09-16T20:49:53.612 INFO:teuthology.orchestra.run.smithi139.stdout:4403 read 0x4759a00 thru 0x476721a (0xd81b bytes) 2024-09-16T20:49:53.616 INFO:teuthology.orchestra.run.smithi139.stdout:4404 trunc from 0x680be00 to 0x3689600 2024-09-16T20:49:53.630 INFO:teuthology.orchestra.run.smithi139.stdout:4405 read 0x274ba00 thru 0x274f87d (0x3e7e bytes) 2024-09-16T20:49:53.631 INFO:teuthology.orchestra.run.smithi139.stdout:4406 punch from 0x3385800 to 0x3392c00, (0xd400 bytes) 2024-09-16T20:49:53.645 INFO:teuthology.orchestra.run.smithi139.stdout:4411 write 0x6746800 thru 0x674cc9c (0x649d bytes) 2024-09-16T20:49:53.648 INFO:teuthology.orchestra.run.smithi139.stdout:4413 read 0x5b7ac00 thru 0x5b8650b (0xb90c bytes) 2024-09-16T20:49:53.649 INFO:teuthology.orchestra.run.smithi139.stdout:4414 trunc from 0x674ce00 to 0x5be8600 2024-09-16T20:49:53.649 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:53.647+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:53.662 INFO:teuthology.orchestra.run.smithi139.stdout:4415 read 0x45e6a00 thru 0x45e7822 (0xe23 bytes) 2024-09-16T20:49:53.673 INFO:teuthology.orchestra.run.smithi139.stdout:4416 trunc from 0x5be8600 to 0x2e25800 2024-09-16T20:49:53.688 INFO:teuthology.orchestra.run.smithi139.stdout:4417 trunc from 0x2e25800 to 0x174ec00 2024-09-16T20:49:53.691 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:53.691 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:53.701 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:49:53.706 INFO:teuthology.orchestra.run.smithi139.stdout:4418 read 0x12f1000 thru 0x12fb2a8 (0xa2a9 bytes) 2024-09-16T20:49:53.713 INFO:teuthology.orchestra.run.smithi139.stdout:4419 read 0xbcf400 thru 0xbdf12b (0xfd2c bytes) 2024-09-16T20:49:53.716 INFO:teuthology.orchestra.run.smithi139.stdout:4421 trunc from 0x174ec00 to 0x360b600 2024-09-16T20:49:53.741 INFO:teuthology.orchestra.run.smithi139.stdout:4423 write 0x834ce00 thru 0x8355606 (0x8807 bytes) 2024-09-16T20:49:53.747 INFO:teuthology.orchestra.run.smithi139.stdout:4424 punch from 0x6dc6c00 to 0x6dcc600, (0x5a00 bytes) 2024-09-16T20:49:53.753 INFO:teuthology.orchestra.run.smithi139.stdout:4426 trunc from 0x8355800 to 0xc83fc00 2024-09-16T20:49:53.779 INFO:teuthology.orchestra.run.smithi139.stdout:4429 write 0xec25e00 thru 0xec2ecb8 (0x8eb9 bytes) 2024-09-16T20:49:53.785 INFO:teuthology.orchestra.run.smithi139.stdout:4432 trunc from 0xec2ee00 to 0xb545000 2024-09-16T20:49:53.786 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:53.785+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:53.805 INFO:teuthology.orchestra.run.smithi139.stdout:4434 trunc from 0xb545000 to 0x6ecb400 2024-09-16T20:49:53.819 INFO:teuthology.orchestra.run.smithi139.stdout:4435 read 0x56d1000 thru 0x56def9b (0xdf9c bytes) 2024-09-16T20:49:53.824 INFO:teuthology.orchestra.run.smithi139.stdout:4439 write 0x48c4800 thru 0x48cb797 (0x6f98 bytes) 2024-09-16T20:49:53.851 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:53.849+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:53.857 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:53.857 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:53.866 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:49:53.951 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:53.951+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:54.013 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:54.013 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:54.023 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:49:54.052 INFO:teuthology.orchestra.run.smithi139.stdout:4440 write 0xa35ae00 thru 0xa35f2f9 (0x44fa bytes) 2024-09-16T20:49:54.052 INFO:teuthology.orchestra.run.smithi139.stdout:4441 write 0x530d600 thru 0x530d66d (0x6e bytes) 2024-09-16T20:49:54.065 INFO:teuthology.orchestra.run.smithi139.stdout:4442 trunc from 0xa35f400 to 0x650400 2024-09-16T20:49:54.067 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:54.066+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:54.101 INFO:teuthology.orchestra.run.smithi139.stdout:4446 trunc from 0x650400 to 0x8e1ea00 2024-09-16T20:49:54.117 INFO:teuthology.orchestra.run.smithi139.stdout:4451 trunc from 0x8e1ea00 to 0x42f1600 2024-09-16T20:49:54.133 INFO:teuthology.orchestra.run.smithi139.stdout:4454 read 0x3701200 thru 0x37048b6 (0x36b7 bytes) 2024-09-16T20:49:54.134 INFO:teuthology.orchestra.run.smithi139.stdout:4457 write 0x214600 thru 0x220228 (0xbc29 bytes) 2024-09-16T20:49:54.134 INFO:teuthology.orchestra.run.smithi139.stdout:4460 write 0xc5cc00 thru 0xc66742 (0x9b43 bytes) 2024-09-16T20:49:54.135 INFO:teuthology.orchestra.run.smithi139.stdout:4461 punch from 0x23ba200 to 0x23c2800, (0x8600 bytes) 2024-09-16T20:49:54.139 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:54.140 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:54.148 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:49:54.153 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:54.151+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:54.159 INFO:teuthology.orchestra.run.smithi139.stdout:4464 write 0xed01c00 thru 0xed0e0fd (0xc4fe bytes) 2024-09-16T20:49:54.169 INFO:teuthology.orchestra.run.smithi139.stdout:4467 trunc from 0xed0e200 to 0x77cf400 2024-09-16T20:49:54.184 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T20:49:54.197 INFO:teuthology.orchestra.run.smithi139.stdout:4468 read 0x2103400 thru 0x2112373 (0xef74 bytes) 2024-09-16T20:49:54.197 INFO:teuthology.orchestra.run.smithi139.stdout:4469 read 0x25bc200 thru 0x25bc8bb (0x6bc bytes) 2024-09-16T20:49:54.220 INFO:teuthology.orchestra.run.smithi139.stdout:4472 write 0xaf47400 thru 0xaf4db57 (0x6758 bytes) 2024-09-16T20:49:54.220 INFO:teuthology.orchestra.run.smithi139.stdout:4476 write 0x69bd200 thru 0x69c146e (0x426f bytes) 2024-09-16T20:49:54.221 INFO:teuthology.orchestra.run.smithi139.stdout:4477 write 0x929da00 thru 0x92a8a6d (0xb06e bytes) 2024-09-16T20:49:54.221 INFO:teuthology.orchestra.run.smithi139.stdout:4478 read 0x265de00 thru 0x2669faa (0xc1ab bytes) 2024-09-16T20:49:54.222 INFO:teuthology.orchestra.run.smithi139.stdout:4487 punch from 0x93cb600 to 0x93d1800, (0x6200 bytes) 2024-09-16T20:49:54.222 INFO:teuthology.orchestra.run.smithi139.stdout:4491 write 0x6ece200 thru 0x6ed97ca (0xb5cb bytes) 2024-09-16T20:49:54.225 INFO:teuthology.orchestra.run.smithi139.stdout:4493 read 0x9da5c00 thru 0x9dac79a (0x6b9b bytes) 2024-09-16T20:49:54.225 INFO:teuthology.orchestra.run.smithi139.stdout:4494 read 0x7973200 thru 0x7974531 (0x1332 bytes) 2024-09-16T20:49:54.231 INFO:teuthology.orchestra.run.smithi139.stdout:4496 trunc from 0xaf4dc00 to 0x62aee00 2024-09-16T20:49:54.249 INFO:teuthology.orchestra.run.smithi139.stdout:4497 read 0x2fa5a00 thru 0x2fb0e1d (0xb41e bytes) 2024-09-16T20:49:54.251 INFO:teuthology.orchestra.run.smithi139.stdout:4498 read 0x4fed600 thru 0x4ff93c4 (0xbdc5 bytes) 2024-09-16T20:49:54.253 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:54.252+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:54.272 INFO:teuthology.orchestra.run.smithi139.stdout:4499 write 0xb290200 thru 0xb293d1c (0x3b1d bytes) 2024-09-16T20:49:54.273 INFO:teuthology.orchestra.run.smithi139.stdout:4501 read 0x1283c00 thru 0x128c191 (0x8592 bytes) 2024-09-16T20:49:54.273 INFO:teuthology.orchestra.run.smithi139.stdout:4502 read 0x68e600 thru 0x69015b (0x1b5c bytes) 2024-09-16T20:49:54.273 INFO:teuthology.orchestra.run.smithi139.stdout:4504 read 0x805600 thru 0x80b875 (0x6276 bytes) 2024-09-16T20:49:54.274 INFO:teuthology.orchestra.run.smithi139.stdout:4505 read 0x6aa000 thru 0x6ac41c (0x241d bytes) 2024-09-16T20:49:54.274 INFO:teuthology.orchestra.run.smithi139.stdout:4506 write 0xa7b0400 thru 0xa7bff5d (0xfb5e bytes) 2024-09-16T20:49:54.275 INFO:teuthology.orchestra.run.smithi139.stdout:4507 write 0x90ec800 thru 0x90fb3a3 (0xeba4 bytes) 2024-09-16T20:49:54.275 INFO:teuthology.orchestra.run.smithi139.stdout:truncating to largest ever: 0xee19200 2024-09-16T20:49:54.280 INFO:teuthology.orchestra.run.smithi139.stdout:4511 trunc from 0xb293e00 to 0xee19200 2024-09-16T20:49:54.292 INFO:teuthology.orchestra.run.smithi139.stdout:4512 punch from 0xc53e00 to 0xc54400, (0x600 bytes) 2024-09-16T20:49:54.294 INFO:teuthology.orchestra.run.smithi139.stdout:4514 read 0xd93f600 thru 0xd944915 (0x5316 bytes) 2024-09-16T20:49:54.294 INFO:teuthology.orchestra.run.smithi139.stdout:4515 write 0x7e2a600 thru 0x7e3647d (0xbe7e bytes) 2024-09-16T20:49:54.298 INFO:teuthology.orchestra.run.smithi139.stdout:4517 write 0xd986200 thru 0xd989685 (0x3486 bytes) 2024-09-16T20:49:54.298 INFO:teuthology.orchestra.run.smithi139.stdout:4521 read 0x98e2600 thru 0x98ed48a (0xae8b bytes) 2024-09-16T20:49:54.298 INFO:teuthology.orchestra.run.smithi139.stdout:4522 write 0x6957200 thru 0x6963a4c (0xc84d bytes) 2024-09-16T20:49:54.298 INFO:teuthology.orchestra.run.smithi139.stdout:4524 punch from 0xd345e00 to 0xd354400, (0xe600 bytes) 2024-09-16T20:49:54.298 INFO:teuthology.orchestra.run.smithi139.stdout:4526 write 0x2440c00 thru 0x244e4f0 (0xd8f1 bytes) 2024-09-16T20:49:54.298 INFO:teuthology.orchestra.run.smithi139.stdout:4528 write 0x5681800 thru 0x568bb11 (0xa312 bytes) 2024-09-16T20:49:54.299 INFO:teuthology.orchestra.run.smithi139.stdout:4529 read 0x7758600 thru 0x7766c1f (0xe620 bytes) 2024-09-16T20:49:54.299 INFO:teuthology.orchestra.run.smithi139.stdout:4531 read 0x6e84200 thru 0x6e84cab (0xaac bytes) 2024-09-16T20:49:54.300 INFO:teuthology.orchestra.run.smithi139.stdout:4532 punch from 0x8624200 to 0x862f400, (0xb200 bytes) 2024-09-16T20:49:54.300 INFO:teuthology.orchestra.run.smithi139.stdout:4533 read 0xd62ae00 thru 0xd62e611 (0x3812 bytes) 2024-09-16T20:49:54.302 INFO:teuthology.orchestra.run.smithi139.stdout:4534 trunc from 0xee19200 to 0xe172400 2024-09-16T20:49:54.312 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:54.312 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:54.315 INFO:teuthology.orchestra.run.smithi139.stdout:4535 read 0xccf9e00 thru 0xcd0956f (0xf770 bytes) 2024-09-16T20:49:54.320 INFO:teuthology.orchestra.run.smithi139.stdout:4536 write 0xa9b3a00 thru 0xa9b62a8 (0x28a9 bytes) 2024-09-16T20:49:54.322 INFO:teuthology.orchestra.run.smithi139.stdout:4537 read 0x702a600 thru 0x703693b (0xc33c bytes) 2024-09-16T20:49:54.322 INFO:teuthology.orchestra.run.smithi139.stdout:4538 write 0x312ec00 thru 0x312f6e7 (0xae8 bytes) 2024-09-16T20:49:54.322 INFO:teuthology.orchestra.run.smithi139.stdout:4539 punch from 0x1e0ea00 to 0x1e1b600, (0xcc00 bytes) 2024-09-16T20:49:54.322 INFO:teuthology.orchestra.run.smithi139.stdout:4541 write 0x477be00 thru 0x478193c (0x5b3d bytes) 2024-09-16T20:49:54.323 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:49:54.324 INFO:teuthology.orchestra.run.smithi139.stdout:4542 read 0x41f0c00 thru 0x41f6a27 (0x5e28 bytes) 2024-09-16T20:49:54.324 INFO:teuthology.orchestra.run.smithi139.stdout:4544 write 0x93e6000 thru 0x93ee786 (0x8787 bytes) 2024-09-16T20:49:54.324 INFO:teuthology.orchestra.run.smithi139.stdout:4548 write 0xa346c00 thru 0xa34c29c (0x569d bytes) 2024-09-16T20:49:54.325 INFO:teuthology.orchestra.run.smithi139.stdout:4549 write 0x77d8600 thru 0x77e5549 (0xcf4a bytes) 2024-09-16T20:49:54.325 INFO:teuthology.orchestra.run.smithi139.stdout:4550 punch from 0x8ec4800 to 0x8ecb000, (0x6800 bytes) 2024-09-16T20:49:54.325 INFO:teuthology.orchestra.run.smithi139.stdout:4551 read 0x78e0000 thru 0x78e69f9 (0x69fa bytes) 2024-09-16T20:49:54.326 INFO:teuthology.orchestra.run.smithi139.stdout:4552 read 0x8f6b000 thru 0x8f73d61 (0x8d62 bytes) 2024-09-16T20:49:54.329 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:49:54.329 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:49:54.335 INFO:teuthology.orchestra.run.smithi139.stdout:4555 trunc from 0xe172400 to 0x882c400 2024-09-16T20:49:54.350 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:49:54.350 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:49:54.354 INFO:teuthology.orchestra.run.smithi139.stdout:4556 punch from 0x5e4cc00 to 0x5e50000, (0x3400 bytes) 2024-09-16T20:49:54.354 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:54.353+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:54.369 INFO:teuthology.orchestra.run.smithi139.stdout:4557 write 0xc368e00 thru 0xc376981 (0xdb82 bytes) 2024-09-16T20:49:54.372 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:49:54.372 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:49:54.375 INFO:teuthology.orchestra.run.smithi139.stdout:4559 read 0xbc59200 thru 0xbc691fd (0xfffe bytes) 2024-09-16T20:49:54.375 INFO:teuthology.orchestra.run.smithi139.stdout:4560 write 0x64200 thru 0x70067 (0xbe68 bytes) 2024-09-16T20:49:54.376 INFO:teuthology.orchestra.run.smithi139.stdout:4564 write 0xa613c00 thru 0xa61d9f9 (0x9dfa bytes) 2024-09-16T20:49:54.376 INFO:teuthology.orchestra.run.smithi139.stdout:4565 read 0x31dc600 thru 0x31ea8e8 (0xe2e9 bytes) 2024-09-16T20:49:54.376 INFO:teuthology.orchestra.run.smithi139.stdout:4568 read 0x5cf3000 thru 0x5cfbf7c (0x8f7d bytes) 2024-09-16T20:49:54.379 INFO:teuthology.orchestra.run.smithi139.stdout:4569 trunc from 0xc376a00 to 0xa9a6c00 2024-09-16T20:49:54.392 INFO:teuthology.orchestra.run.smithi139.stdout:4570 write 0x3c2ac00 thru 0x3c2cb03 (0x1f04 bytes) 2024-09-16T20:49:54.399 INFO:teuthology.orchestra.run.smithi139.stdout:4577 punch from 0x41a2000 to 0x41b0600, (0xe600 bytes) 2024-09-16T20:49:54.400 INFO:teuthology.orchestra.run.smithi139.stdout:4578 write 0x8dce800 thru 0x8ddcf15 (0xe716 bytes) 2024-09-16T20:49:54.402 INFO:teuthology.orchestra.run.smithi139.stdout:4579 read 0x1026400 thru 0x10315be (0xb1bf bytes) 2024-09-16T20:49:54.402 INFO:teuthology.orchestra.run.smithi139.stdout:4580 write 0x4e34600 thru 0x4e40852 (0xc253 bytes) 2024-09-16T20:49:54.404 INFO:teuthology.orchestra.run.smithi139.stdout:4581 trunc from 0xa9a6c00 to 0x9aa2800 2024-09-16T20:49:54.405 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:49:54.405 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:49:54.417 INFO:teuthology.orchestra.run.smithi139.stdout:4583 read 0x2bb3a00 thru 0x2bb660c (0x2c0d bytes) 2024-09-16T20:49:54.419 INFO:teuthology.orchestra.run.smithi139.stdout:4585 read 0x1d9f800 thru 0x1da65ba (0x6dbb bytes) 2024-09-16T20:49:54.433 INFO:teuthology.orchestra.run.smithi139.stdout:4587 trunc from 0x9aa2800 to 0xde5a00 2024-09-16T20:49:54.437 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:49:54.437 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:49:54.449 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 3, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-16T20:49:54.449 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T20:49:54.449 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-16T20:49:54.449 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-16T20:49:54.455 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:54.454+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:54.469 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:49:54.470 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:49:54.478 INFO:teuthology.orchestra.run.smithi139.stdout:4589 write 0xbc1b800 thru 0xbc2b4b1 (0xfcb2 bytes) 2024-09-16T20:49:54.484 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:54.484 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:49:54.485 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:49:54.485 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:49:54.485 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.485 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2206 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1046528~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:49:54.485 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:49:54.485 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 92.146653916000005, 2024-09-16T20:49:54.485 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050147714000000003, 2024-09-16T20:49:54.485 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:54.485 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:54.485 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:54.485 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:54.485 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:54.485 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:54.486 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2206 2024-09-16T20:49:54.486 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.486 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:54.486 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.486 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:54.486 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:49:54.486 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:54.486 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.486 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.486 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:54.486 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:49:54.486 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:54.487 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.487 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.487 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:54.487 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:49:54.487 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8599999999999999e-07 2024-09-16T20:49:54.487 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.487 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.487 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:54.487 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336024+0000", 2024-09-16T20:49:54.487 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.934e-06 2024-09-16T20:49:54.487 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.487 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.488 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:54.488 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336037+0000", 2024-09-16T20:49:54.488 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2496e-05 2024-09-16T20:49:54.488 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.488 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.488 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:54.488 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336042+0000", 2024-09-16T20:49:54.488 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4809999999999997e-06 2024-09-16T20:49:54.488 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.488 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.488 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:54.489 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385980+0000", 2024-09-16T20:49:54.489 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049938376999999999 2024-09-16T20:49:54.489 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.489 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.489 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:54.489 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.386069+0000", 2024-09-16T20:49:54.489 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8368999999999994e-05 2024-09-16T20:49:54.489 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.489 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.489 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:54.489 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.386170+0000", 2024-09-16T20:49:54.489 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000100671 2024-09-16T20:49:54.489 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.490 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:54.490 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.490 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.490 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.490 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5439 3.a 3:5789f5c3:::rbd_data.110ed9ca540.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 1181696~60416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:54.490 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:49:54.490 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 91.617965335999997, 2024-09-16T20:49:54.490 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10102045599999999, 2024-09-16T20:49:54.490 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:54.490 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:54.490 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:54.490 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:54.490 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:54.490 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:54.491 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5439 2024-09-16T20:49:54.491 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.491 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:54.491 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.491 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:54.491 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:49:54.491 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:54.491 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.491 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.491 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:54.491 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:49:54.491 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:54.491 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.491 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.492 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:54.492 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864711+0000", 2024-09-16T20:49:54.492 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7899999999999997e-07 2024-09-16T20:49:54.492 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.492 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.492 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:54.492 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864760+0000", 2024-09-16T20:49:54.492 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8803e-05 2024-09-16T20:49:54.492 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.492 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.492 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:54.492 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864769+0000", 2024-09-16T20:49:54.492 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0999999999999993e-06 2024-09-16T20:49:54.493 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.493 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.493 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:54.493 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864774+0000", 2024-09-16T20:49:54.493 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8919999999999999e-06 2024-09-16T20:49:54.493 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.493 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.493 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:54.493 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864815+0000", 2024-09-16T20:49:54.494 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1307000000000003e-05 2024-09-16T20:49:54.494 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.494 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.494 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:54.494 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864893+0000", 2024-09-16T20:49:54.494 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7581000000000001e-05 2024-09-16T20:49:54.494 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.494 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.494 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 1", 2024-09-16T20:49:54.494 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864996+0000", 2024-09-16T20:49:54.494 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102725 2024-09-16T20:49:54.494 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.494 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.495 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:54.495 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965442+0000", 2024-09-16T20:49:54.495 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100446012 2024-09-16T20:49:54.495 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.495 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.495 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:54.495 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965618+0000", 2024-09-16T20:49:54.495 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00017653100000000001 2024-09-16T20:49:54.495 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.495 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.495 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:54.495 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965682+0000", 2024-09-16T20:49:54.495 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3905999999999994e-05 2024-09-16T20:49:54.496 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.496 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.496 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:54.496 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965731+0000", 2024-09-16T20:49:54.496 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8819999999999997e-05 2024-09-16T20:49:54.496 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.496 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:54.496 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.496 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.496 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.496 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4600.0:5439 3.a e37/23 ondisk, result = 0)", 2024-09-16T20:49:54.496 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:49:54.496 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 91.616969187999999, 2024-09-16T20:49:54.497 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099742634999999996, 2024-09-16T20:49:54.497 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:54.497 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:54.497 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:54.497 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:54.497 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.497 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:54.497 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:49:54.497 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:54.497 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.497 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.497 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:54.497 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:49:54.498 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:54.498 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.498 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.498 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:54.498 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:49:54.498 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0900000000000001e-07 2024-09-16T20:49:54.498 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.498 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.498 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:54.498 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865711+0000", 2024-09-16T20:49:54.498 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3739999999999998e-06 2024-09-16T20:49:54.498 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.498 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.498 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:54.498 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965343+0000", 2024-09-16T20:49:54.499 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099632526999999999 2024-09-16T20:49:54.499 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.499 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.499 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:54.499 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965357+0000", 2024-09-16T20:49:54.499 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3713999999999999e-05 2024-09-16T20:49:54.499 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.499 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.499 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:54.499 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965426+0000", 2024-09-16T20:49:54.499 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8745999999999995e-05 2024-09-16T20:49:54.500 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.500 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.500 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:54.500 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965434+0000", 2024-09-16T20:49:54.500 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6910000000000003e-06 2024-09-16T20:49:54.500 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.500 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.500 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:54.500 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965449+0000", 2024-09-16T20:49:54.500 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4874e-05 2024-09-16T20:49:54.500 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.500 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:54.500 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.501 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.501 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.501 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4882.0:2508 3.12 3:4992771f:::rbd_object_map.110e7c8851c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T20:49:54.501 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:49:54.501 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 68.001165639999996, 2024-09-16T20:49:54.501 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.125007011, 2024-09-16T20:49:54.501 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:54.501 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:54.501 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:54.501 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:54.501 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4882", 2024-09-16T20:49:54.501 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2436923223", 2024-09-16T20:49:54.501 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2508 2024-09-16T20:49:54.502 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.502 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:54.502 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.502 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:54.502 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:49:54.502 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:54.502 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.502 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.502 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:54.502 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:49:54.502 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:54.502 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.503 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.503 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:54.503 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481511+0000", 2024-09-16T20:49:54.503 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0699999999999996e-07 2024-09-16T20:49:54.503 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.503 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.503 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:54.503 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481515+0000", 2024-09-16T20:49:54.503 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0609999999999997e-06 2024-09-16T20:49:54.503 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.503 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.503 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:54.503 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481523+0000", 2024-09-16T20:49:54.504 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1310000000000006e-06 2024-09-16T20:49:54.504 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.504 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.504 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:54.504 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481528+0000", 2024-09-16T20:49:54.504 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4900000000000002e-06 2024-09-16T20:49:54.504 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.504 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.504 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:54.504 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481565+0000", 2024-09-16T20:49:54.504 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7066000000000001e-05 2024-09-16T20:49:54.504 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.504 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.505 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:54.505 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481628+0000", 2024-09-16T20:49:54.505 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3118000000000001e-05 2024-09-16T20:49:54.505 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.505 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.505 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 2", 2024-09-16T20:49:54.505 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481822+0000", 2024-09-16T20:49:54.505 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019446200000000001 2024-09-16T20:49:54.505 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.505 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.505 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:54.505 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605988+0000", 2024-09-16T20:49:54.506 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.124165888 2024-09-16T20:49:54.506 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.506 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.506 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:54.506 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606427+0000", 2024-09-16T20:49:54.506 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00043854899999999999 2024-09-16T20:49:54.506 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.506 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.506 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:54.506 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606476+0000", 2024-09-16T20:49:54.506 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8971999999999998e-05 2024-09-16T20:49:54.506 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.506 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.506 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:54.507 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606517+0000", 2024-09-16T20:49:54.507 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1467000000000001e-05 2024-09-16T20:49:54.507 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.507 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:54.507 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.507 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.507 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.507 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4882.0:2508 3.12 e57/55 ondisk, result = 0)", 2024-09-16T20:49:54.507 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:49:54.507 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 68.000177140000005, 2024-09-16T20:49:54.507 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12349711400000001, 2024-09-16T20:49:54.507 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:54.508 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:54.508 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:54.508 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:54.508 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.508 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:54.508 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:49:54.508 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:54.508 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.508 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.508 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:54.508 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:49:54.509 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:54.509 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.509 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.509 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:54.509 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:49:54.509 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.06e-07 2024-09-16T20:49:54.509 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.509 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.509 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:54.509 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482502+0000", 2024-09-16T20:49:54.509 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.01e-06 2024-09-16T20:49:54.509 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.509 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.509 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:54.510 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605772+0000", 2024-09-16T20:49:54.510 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.123269687 2024-09-16T20:49:54.510 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.510 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.510 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:54.510 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605791+0000", 2024-09-16T20:49:54.510 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8912000000000001e-05 2024-09-16T20:49:54.510 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.510 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.510 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:54.510 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605974+0000", 2024-09-16T20:49:54.510 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000182836 2024-09-16T20:49:54.510 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.510 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.511 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:54.511 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605982+0000", 2024-09-16T20:49:54.511 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1699999999999997e-06 2024-09-16T20:49:54.511 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.511 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.511 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:54.511 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605996+0000", 2024-09-16T20:49:54.511 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3893000000000001e-05 2024-09-16T20:49:54.511 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.511 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:54.511 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.511 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.511 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.511 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4882.0:2512 3.f 3:f4d895b7:::rbd_data.110e7c8851c.0000000000000000:head [read 16384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T20:49:54.512 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:49:54.512 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 67.999740110000005, 2024-09-16T20:49:54.512 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12318483, 2024-09-16T20:49:54.512 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:54.512 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:54.512 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:54.512 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:54.512 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4882", 2024-09-16T20:49:54.512 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2436923223", 2024-09-16T20:49:54.512 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2512 2024-09-16T20:49:54.512 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.512 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:54.512 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.513 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:54.513 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:49:54.513 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:54.513 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.513 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.513 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:54.513 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:49:54.513 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:54.513 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.513 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.513 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:54.513 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:49:54.513 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3499999999999995e-07 2024-09-16T20:49:54.513 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.514 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.514 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:54.514 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482943+0000", 2024-09-16T20:49:54.514 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3450000000000002e-06 2024-09-16T20:49:54.514 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.514 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.514 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:54.514 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605772+0000", 2024-09-16T20:49:54.514 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.122829796 2024-09-16T20:49:54.514 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.514 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.514 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:54.514 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605792+0000", 2024-09-16T20:49:54.514 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9916999999999999e-05 2024-09-16T20:49:54.515 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.515 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.515 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:54.515 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605917+0000", 2024-09-16T20:49:54.515 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00012500799999999999 2024-09-16T20:49:54.515 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.515 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.515 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:54.515 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606004+0000", 2024-09-16T20:49:54.515 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6242999999999994e-05 2024-09-16T20:49:54.515 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.515 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.515 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:54.516 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606120+0000", 2024-09-16T20:49:54.516 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000116786 2024-09-16T20:49:54.516 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.516 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:54.516 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.516 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.516 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.516 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:11702 3.15 3:acf33c8b:::rbd_data.110e747f0779.000000000000001a:head [set-alloc-hint object_size 262144 write_size 262144,write 143360~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:49:54.516 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:49:54.516 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 50.344747687999998, 2024-09-16T20:49:54.516 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.084801043000000006, 2024-09-16T20:49:54.516 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:54.516 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:54.517 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:54.517 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:54.517 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:49:54.517 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:49:54.517 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 11702 2024-09-16T20:49:54.517 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.517 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:54.517 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.517 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:54.517 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:49:54.517 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:54.517 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.518 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.518 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:54.518 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:49:54.518 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:54.518 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.518 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.518 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:54.518 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137929+0000", 2024-09-16T20:49:54.518 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8400000000000002e-07 2024-09-16T20:49:54.518 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.518 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.518 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:54.518 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137937+0000", 2024-09-16T20:49:54.518 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0260000000000007e-06 2024-09-16T20:49:54.519 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.519 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.519 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:54.519 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137952+0000", 2024-09-16T20:49:54.519 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4659e-05 2024-09-16T20:49:54.519 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.519 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.519 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:54.519 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137960+0000", 2024-09-16T20:49:54.519 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0169999999999996e-06 2024-09-16T20:49:54.519 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.519 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.519 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:54.519 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138010+0000", 2024-09-16T20:49:54.520 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0021000000000001e-05 2024-09-16T20:49:54.520 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.520 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.520 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:54.520 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138114+0000", 2024-09-16T20:49:54.520 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000103797 2024-09-16T20:49:54.520 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.520 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.520 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:49:54.520 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138233+0000", 2024-09-16T20:49:54.520 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000118971 2024-09-16T20:49:54.520 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.520 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.521 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:54.521 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138953+0000", 2024-09-16T20:49:54.521 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00072026000000000002 2024-09-16T20:49:54.521 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.521 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.521 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:54.521 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222609+0000", 2024-09-16T20:49:54.521 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.083656650999999999 2024-09-16T20:49:54.521 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.521 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.521 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:54.521 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222684+0000", 2024-09-16T20:49:54.521 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4369999999999994e-05 2024-09-16T20:49:54.522 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.522 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.522 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:54.522 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222729+0000", 2024-09-16T20:49:54.522 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5287000000000002e-05 2024-09-16T20:49:54.522 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.522 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:54.522 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.522 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.522 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.522 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:11704 3.9 3:91220b75:::rbd_data.110e747f0779.000000000000001c:head [set-alloc-hint object_size 262144 write_size 262144,write 139264~6656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:49:54.522 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:49:54.522 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 50.344509686999999, 2024-09-16T20:49:54.523 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.084531133999999994, 2024-09-16T20:49:54.523 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:54.523 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:54.523 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:54.523 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:54.523 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:49:54.523 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:49:54.523 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 11704 2024-09-16T20:49:54.523 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.523 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:54.523 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.523 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:54.523 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:49:54.524 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:54.524 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.524 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.524 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:54.524 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:49:54.524 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:54.524 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.525 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.525 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:54.525 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138167+0000", 2024-09-16T20:49:54.525 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.33e-07 2024-09-16T20:49:54.525 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.525 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.525 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:54.525 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138179+0000", 2024-09-16T20:49:54.525 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2055999999999999e-05 2024-09-16T20:49:54.526 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.526 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.526 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:54.526 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138193+0000", 2024-09-16T20:49:54.526 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3961000000000001e-05 2024-09-16T20:49:54.526 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.526 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.526 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:54.526 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138200+0000", 2024-09-16T20:49:54.526 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4429999999999997e-06 2024-09-16T20:49:54.526 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.526 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.526 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:54.526 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138258+0000", 2024-09-16T20:49:54.527 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7225999999999997e-05 2024-09-16T20:49:54.527 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.527 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.527 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:54.527 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138364+0000", 2024-09-16T20:49:54.527 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000106013 2024-09-16T20:49:54.527 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.527 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.527 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:49:54.527 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138494+0000", 2024-09-16T20:49:54.527 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013059599999999999 2024-09-16T20:49:54.527 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.527 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.528 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:54.528 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.139220+0000", 2024-09-16T20:49:54.528 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00072539800000000004 2024-09-16T20:49:54.528 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.528 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.528 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:54.528 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222608+0000", 2024-09-16T20:49:54.528 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.083388620999999996 2024-09-16T20:49:54.528 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.528 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.528 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:54.528 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222651+0000", 2024-09-16T20:49:54.528 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2228000000000003e-05 2024-09-16T20:49:54.528 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.529 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.529 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:54.529 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222697+0000", 2024-09-16T20:49:54.529 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6659000000000002e-05 2024-09-16T20:49:54.529 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.529 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:54.529 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.529 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.529 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.529 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5138.0:9358 3.c 3.c25699cc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T20:49:54.529 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:49:54.530 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 27.632362382, 2024-09-16T20:49:54.530 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0018657580000001, 2024-09-16T20:49:54.530 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:54.530 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:54.530 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:49:54.530 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:54.530 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5138", 2024-09-16T20:49:54.530 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:49:54.530 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 9358 2024-09-16T20:49:54.530 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.530 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:54.530 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.530 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:54.530 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:49:54.531 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:54.531 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.531 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.531 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:54.531 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:49:54.531 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:54.531 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.531 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.531 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:54.531 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850314+0000", 2024-09-16T20:49:54.531 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4099999999999999e-07 2024-09-16T20:49:54.531 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.531 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.531 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:54.532 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850316+0000", 2024-09-16T20:49:54.532 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2469999999999998e-06 2024-09-16T20:49:54.532 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.532 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.532 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:54.532 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850322+0000", 2024-09-16T20:49:54.532 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3470000000000004e-06 2024-09-16T20:49:54.532 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.532 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.532 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:54.533 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850327+0000", 2024-09-16T20:49:54.533 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9450000000000001e-06 2024-09-16T20:49:54.533 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.533 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.533 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:54.533 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850376+0000", 2024-09-16T20:49:54.533 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8257000000000002e-05 2024-09-16T20:49:54.533 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.533 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.533 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:49:54.533 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850385+0000", 2024-09-16T20:49:54.533 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9800000000000004e-06 2024-09-16T20:49:54.533 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.534 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.534 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:54.534 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852167+0000", 2024-09-16T20:49:54.534 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0017825659999999 2024-09-16T20:49:54.534 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.534 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.534 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:54.534 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852179+0000", 2024-09-16T20:49:54.534 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1975e-05 2024-09-16T20:49:54.534 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.534 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:54.534 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.534 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.534 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.535 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5138.0:9382 3.c 3.c25699cc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T20:49:54.535 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:49:54.535 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 27.624015365999998, 2024-09-16T20:49:54.535 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.99358404700000003, 2024-09-16T20:49:54.535 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:54.535 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:54.535 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:49:54.535 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:54.535 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5138", 2024-09-16T20:49:54.535 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:49:54.535 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 9382 2024-09-16T20:49:54.535 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.535 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:54.535 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.536 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:54.536 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:49:54.536 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:54.536 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.536 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.536 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:54.536 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:49:54.536 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:54.536 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.536 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.536 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:54.536 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858661+0000", 2024-09-16T20:49:54.536 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0200000000000002e-07 2024-09-16T20:49:54.536 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.537 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.537 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:54.537 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858664+0000", 2024-09-16T20:49:54.537 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.038e-06 2024-09-16T20:49:54.537 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.537 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.537 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:54.537 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858671+0000", 2024-09-16T20:49:54.537 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0110000000000003e-06 2024-09-16T20:49:54.537 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.537 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.537 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:54.537 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858677+0000", 2024-09-16T20:49:54.538 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0959999999999997e-06 2024-09-16T20:49:54.538 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.538 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.538 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:54.538 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858732+0000", 2024-09-16T20:49:54.538 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4543000000000003e-05 2024-09-16T20:49:54.538 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.538 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.538 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:49:54.538 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858739+0000", 2024-09-16T20:49:54.538 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5429999999999996e-06 2024-09-16T20:49:54.538 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.538 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.538 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:54.539 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852236+0000", 2024-09-16T20:49:54.539 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.993496453 2024-09-16T20:49:54.539 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.539 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.539 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:54.539 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852244+0000", 2024-09-16T20:49:54.539 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8610000000000004e-06 2024-09-16T20:49:54.539 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.539 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:54.539 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.539 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.539 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.539 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:361 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 372736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:49:54.540 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246714+0000", 2024-09-16T20:49:54.540 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 18.235961218, 2024-09-16T20:49:54.540 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050338229999999998, 2024-09-16T20:49:54.540 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:54.540 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:54.540 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:54.540 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:54.540 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:49:54.540 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:49:54.540 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 361 2024-09-16T20:49:54.540 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.540 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:54.540 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.541 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:54.541 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246714+0000", 2024-09-16T20:49:54.541 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:54.541 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.541 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.541 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:54.541 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246714+0000", 2024-09-16T20:49:54.541 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:54.541 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.541 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.541 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:54.541 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246715+0000", 2024-09-16T20:49:54.542 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1899999999999999e-07 2024-09-16T20:49:54.542 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.542 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.542 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:54.542 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246716+0000", 2024-09-16T20:49:54.542 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1659999999999999e-06 2024-09-16T20:49:54.542 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.542 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.542 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:54.542 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246721+0000", 2024-09-16T20:49:54.542 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9830000000000004e-06 2024-09-16T20:49:54.542 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.542 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.542 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:54.543 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246723+0000", 2024-09-16T20:49:54.543 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5679999999999998e-06 2024-09-16T20:49:54.543 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.543 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.543 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:54.543 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.296875+0000", 2024-09-16T20:49:54.543 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050151683000000002 2024-09-16T20:49:54.543 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.543 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.543 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:54.543 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.296959+0000", 2024-09-16T20:49:54.543 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4079999999999995e-05 2024-09-16T20:49:54.543 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.543 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.544 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:54.544 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297053+0000", 2024-09-16T20:49:54.544 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.3530999999999996e-05 2024-09-16T20:49:54.544 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.544 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:54.544 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.544 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.544 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.544 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:362 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 385024~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:49:54.544 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246727+0000", 2024-09-16T20:49:54.544 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 18.235949109, 2024-09-16T20:49:54.544 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050546528, 2024-09-16T20:49:54.544 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:54.545 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:54.545 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:54.545 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:54.545 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:49:54.545 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:49:54.545 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 362 2024-09-16T20:49:54.545 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.545 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:54.545 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.545 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:54.546 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246727+0000", 2024-09-16T20:49:54.546 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:54.546 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.546 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.546 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:54.546 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246727+0000", 2024-09-16T20:49:54.546 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:54.547 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.547 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.547 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:54.547 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246727+0000", 2024-09-16T20:49:54.547 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.35e-07 2024-09-16T20:49:54.547 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.547 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.548 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:54.548 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246728+0000", 2024-09-16T20:49:54.548 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.175e-06 2024-09-16T20:49:54.548 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.548 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.548 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:54.548 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246734+0000", 2024-09-16T20:49:54.548 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2160000000000003e-06 2024-09-16T20:49:54.548 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.548 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.548 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:54.549 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246737+0000", 2024-09-16T20:49:54.552 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9450000000000002e-06 2024-09-16T20:49:54.552 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.552 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.552 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:54.552 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297103+0000", 2024-09-16T20:49:54.552 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050365518999999997 2024-09-16T20:49:54.552 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.552 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.553 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:54.553 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297183+0000", 2024-09-16T20:49:54.553 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0208000000000002e-05 2024-09-16T20:49:54.553 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.553 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.553 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:54.553 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297273+0000", 2024-09-16T20:49:54.553 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0229999999999995e-05 2024-09-16T20:49:54.553 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.553 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:54.553 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.553 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.553 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.553 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:368 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 327680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:49:54.554 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246746+0000", 2024-09-16T20:49:54.554 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 18.235929829, 2024-09-16T20:49:54.554 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050730747999999999, 2024-09-16T20:49:54.554 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:54.554 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:54.554 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:54.554 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:54.554 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:49:54.554 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:49:54.554 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 368 2024-09-16T20:49:54.554 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.554 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:54.554 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.554 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:54.555 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246746+0000", 2024-09-16T20:49:54.555 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:54.555 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.555 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.555 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:54.555 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246746+0000", 2024-09-16T20:49:54.555 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:54.555 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.555 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.555 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:54.555 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246746+0000", 2024-09-16T20:49:54.555 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8099999999999999e-07 2024-09-16T20:49:54.555 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.555 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.556 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:54.556 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246754+0000", 2024-09-16T20:49:54.556 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4440000000000002e-06 2024-09-16T20:49:54.556 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.556 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.556 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:54.557 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246758+0000", 2024-09-16T20:49:54.557 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3900000000000003e-06 2024-09-16T20:49:54.557 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.557 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.557 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:54.557 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246761+0000", 2024-09-16T20:49:54.558 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.101e-06 2024-09-16T20:49:54.558 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.558 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.558 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:54.558 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297310+0000", 2024-09-16T20:49:54.558 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050549166 2024-09-16T20:49:54.558 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.558 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.558 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:54.558 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297387+0000", 2024-09-16T20:49:54.559 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6658999999999999e-05 2024-09-16T20:49:54.559 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.559 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.559 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:54.559 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297477+0000", 2024-09-16T20:49:54.559 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9707e-05 2024-09-16T20:49:54.559 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.559 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:54.559 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.559 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.559 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.559 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:369 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 339968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:49:54.559 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246766+0000", 2024-09-16T20:49:54.560 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 18.235910103999998, 2024-09-16T20:49:54.560 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050911291999999997, 2024-09-16T20:49:54.560 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:54.560 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:54.560 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:54.560 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:54.560 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:49:54.560 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:49:54.560 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 369 2024-09-16T20:49:54.560 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.560 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:54.560 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.560 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:54.560 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246766+0000", 2024-09-16T20:49:54.561 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:54.561 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.561 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.561 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:54.561 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246766+0000", 2024-09-16T20:49:54.561 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:54.561 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.561 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.561 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:54.561 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246766+0000", 2024-09-16T20:49:54.561 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3200000000000001e-07 2024-09-16T20:49:54.561 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.561 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.562 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:54.562 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246767+0000", 2024-09-16T20:49:54.562 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.252e-06 2024-09-16T20:49:54.562 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.562 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.562 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:54.562 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246770+0000", 2024-09-16T20:49:54.562 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1049999999999999e-06 2024-09-16T20:49:54.562 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.562 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.562 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:54.562 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246773+0000", 2024-09-16T20:49:54.562 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.396e-06 2024-09-16T20:49:54.562 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.563 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.563 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:54.563 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297513+0000", 2024-09-16T20:49:54.563 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050740800000000003 2024-09-16T20:49:54.563 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.563 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.563 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:54.563 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297589+0000", 2024-09-16T20:49:54.563 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5318e-05 2024-09-16T20:49:54.563 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.563 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.563 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:54.564 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297677+0000", 2024-09-16T20:49:54.564 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8189000000000006e-05 2024-09-16T20:49:54.564 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.564 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:54.564 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.564 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.565 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.565 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:370 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 348160~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:49:54.565 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246776+0000", 2024-09-16T20:49:54.565 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 18.235900089000001, 2024-09-16T20:49:54.565 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051096643999999997, 2024-09-16T20:49:54.565 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:54.566 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:54.566 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:54.566 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:54.566 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:49:54.566 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:49:54.566 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 370 2024-09-16T20:49:54.566 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.566 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:54.567 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.567 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:54.567 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246776+0000", 2024-09-16T20:49:54.567 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:54.567 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.567 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.567 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:54.567 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246776+0000", 2024-09-16T20:49:54.567 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:54.567 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.567 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.567 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:54.567 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246776+0000", 2024-09-16T20:49:54.568 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2499999999999999e-07 2024-09-16T20:49:54.568 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.568 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.568 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:54.568 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246777+0000", 2024-09-16T20:49:54.568 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1510000000000001e-06 2024-09-16T20:49:54.568 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.568 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.568 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:54.568 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246780+0000", 2024-09-16T20:49:54.568 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9610000000000001e-06 2024-09-16T20:49:54.568 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.569 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.569 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:54.569 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246783+0000", 2024-09-16T20:49:54.569 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6129999999999998e-06 2024-09-16T20:49:54.569 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.569 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.569 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:54.569 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297713+0000", 2024-09-16T20:49:54.569 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050929950000000002 2024-09-16T20:49:54.569 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.569 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.570 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:54.570 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297786+0000", 2024-09-16T20:49:54.570 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3096000000000006e-05 2024-09-16T20:49:54.570 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.570 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.570 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:54.570 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297872+0000", 2024-09-16T20:49:54.570 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6648000000000001e-05 2024-09-16T20:49:54.570 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.570 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:54.570 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.570 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.571 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.571 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:371 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 380928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:49:54.571 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246791+0000", 2024-09-16T20:49:54.571 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 18.235884453000001, 2024-09-16T20:49:54.571 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051287209, 2024-09-16T20:49:54.571 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:54.571 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:54.571 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:54.571 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:54.571 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:49:54.571 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:49:54.571 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 371 2024-09-16T20:49:54.571 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.571 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:54.572 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.572 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:54.572 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246791+0000", 2024-09-16T20:49:54.572 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:54.572 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.572 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.572 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:54.572 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246791+0000", 2024-09-16T20:49:54.572 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:54.572 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.572 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.572 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:54.573 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246792+0000", 2024-09-16T20:49:54.573 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8200000000000001e-07 2024-09-16T20:49:54.573 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.573 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.573 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:54.573 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246793+0000", 2024-09-16T20:49:54.573 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.28e-06 2024-09-16T20:49:54.573 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.573 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.573 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:54.573 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246796+0000", 2024-09-16T20:49:54.573 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.597e-06 2024-09-16T20:49:54.573 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.574 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.574 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:54.574 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246799+0000", 2024-09-16T20:49:54.574 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7089999999999998e-06 2024-09-16T20:49:54.574 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.574 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.574 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:54.574 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297910+0000", 2024-09-16T20:49:54.574 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051110598 2024-09-16T20:49:54.574 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.574 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.574 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:54.575 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297986+0000", 2024-09-16T20:49:54.575 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6711000000000005e-05 2024-09-16T20:49:54.575 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.575 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.575 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:54.575 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.298078+0000", 2024-09-16T20:49:54.575 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2032000000000002e-05 2024-09-16T20:49:54.575 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.575 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:54.575 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.575 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.575 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.575 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:373 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 376832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:49:54.575 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246904+0000", 2024-09-16T20:49:54.576 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 18.235772031, 2024-09-16T20:49:54.576 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051310617000000003, 2024-09-16T20:49:54.576 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:54.576 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:54.576 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:54.576 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:54.576 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:49:54.576 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:49:54.576 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 373 2024-09-16T20:49:54.576 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.576 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:54.576 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.576 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:54.577 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246904+0000", 2024-09-16T20:49:54.577 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:54.577 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.577 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.577 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:54.577 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246904+0000", 2024-09-16T20:49:54.577 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:54.577 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.577 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.577 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:54.577 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246904+0000", 2024-09-16T20:49:54.577 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8000000000000001e-07 2024-09-16T20:49:54.577 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.578 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.578 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:54.578 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246908+0000", 2024-09-16T20:49:54.578 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4039999999999997e-06 2024-09-16T20:49:54.578 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.578 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.578 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:54.578 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246913+0000", 2024-09-16T20:49:54.578 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0030000000000002e-06 2024-09-16T20:49:54.578 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.578 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.578 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:54.578 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246917+0000", 2024-09-16T20:49:54.578 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.303e-06 2024-09-16T20:49:54.578 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.579 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.579 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:54.579 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.298114+0000", 2024-09-16T20:49:54.579 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051197724 2024-09-16T20:49:54.579 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.579 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.579 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:54.579 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.298159+0000", 2024-09-16T20:49:54.579 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4882999999999997e-05 2024-09-16T20:49:54.579 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.579 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.579 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:54.579 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.298214+0000", 2024-09-16T20:49:54.579 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4920000000000003e-05 2024-09-16T20:49:54.579 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.580 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:54.580 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.580 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.580 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.580 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:374 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 389120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:49:54.580 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246927+0000", 2024-09-16T20:49:54.580 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 18.235748982, 2024-09-16T20:49:54.580 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051498164999999999, 2024-09-16T20:49:54.580 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:54.580 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:54.580 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:54.580 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:54.580 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:49:54.580 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:49:54.581 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 374 2024-09-16T20:49:54.581 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.581 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:54.581 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.581 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:54.581 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246927+0000", 2024-09-16T20:49:54.581 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:54.581 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.581 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.581 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:54.581 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246927+0000", 2024-09-16T20:49:54.581 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:54.581 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.582 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.582 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:54.582 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246927+0000", 2024-09-16T20:49:54.582 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7e-07 2024-09-16T20:49:54.582 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.582 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.582 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:54.582 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246929+0000", 2024-09-16T20:49:54.582 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.607e-06 2024-09-16T20:49:54.582 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.582 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.582 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:54.582 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246932+0000", 2024-09-16T20:49:54.582 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4539999999999999e-06 2024-09-16T20:49:54.583 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.583 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.583 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:54.583 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246935+0000", 2024-09-16T20:49:54.583 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6570000000000001e-06 2024-09-16T20:49:54.583 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.583 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.583 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:54.583 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.298242+0000", 2024-09-16T20:49:54.583 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051306887000000002 2024-09-16T20:49:54.583 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.583 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.583 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:54.584 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.298327+0000", 2024-09-16T20:49:54.584 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5068999999999995e-05 2024-09-16T20:49:54.584 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.584 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.584 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:54.584 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.298425+0000", 2024-09-16T20:49:54.584 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8021000000000002e-05 2024-09-16T20:49:54.584 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.584 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:54.584 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.584 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.584 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.584 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5763.0:4895 3.a 3:5384606c:::rbd_data.110e1b19a241.0000000000000018:head [read 0~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-16T20:49:54.585 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:45.598212+0000", 2024-09-16T20:49:54.585 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 8.8844633250000005, 2024-09-16T20:49:54.585 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0375864829999999, 2024-09-16T20:49:54.585 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:54.585 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:54.585 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:54.585 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:54.585 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5763", 2024-09-16T20:49:54.585 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/623897567", 2024-09-16T20:49:54.585 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4895 2024-09-16T20:49:54.585 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.585 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:54.585 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.586 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:54.586 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598212+0000", 2024-09-16T20:49:54.586 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:54.586 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.586 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.586 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:54.586 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598212+0000", 2024-09-16T20:49:54.586 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:54.586 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.586 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.586 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:54.586 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598214+0000", 2024-09-16T20:49:54.586 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0090000000000002e-06 2024-09-16T20:49:54.586 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.586 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.587 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:54.587 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598222+0000", 2024-09-16T20:49:54.587 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6699999999999994e-06 2024-09-16T20:49:54.587 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.587 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.587 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:54.587 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598241+0000", 2024-09-16T20:49:54.587 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8896999999999999e-05 2024-09-16T20:49:54.587 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.587 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.587 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:54.587 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598250+0000", 2024-09-16T20:49:54.587 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6649999999999992e-06 2024-09-16T20:49:54.588 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.588 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.588 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:54.588 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598325+0000", 2024-09-16T20:49:54.588 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5189000000000001e-05 2024-09-16T20:49:54.588 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.588 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.588 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op must wait for map", 2024-09-16T20:49:54.588 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598333+0000", 2024-09-16T20:49:54.588 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.3170000000000002e-06 2024-09-16T20:49:54.588 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.588 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.588 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:54.589 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601355+0000", 2024-09-16T20:49:54.589 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0030215400000000001 2024-09-16T20:49:54.589 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.589 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.589 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:49:54.589 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601365+0000", 2024-09-16T20:49:54.589 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0215e-05 2024-09-16T20:49:54.589 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.589 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.589 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:54.589 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635506+0000", 2024-09-16T20:49:54.590 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0341408240000001 2024-09-16T20:49:54.590 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.590 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.590 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:54.590 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635655+0000", 2024-09-16T20:49:54.590 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000149412 2024-09-16T20:49:54.590 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.590 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.590 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:54.590 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635799+0000", 2024-09-16T20:49:54.590 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000143745 2024-09-16T20:49:54.590 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.590 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:54.591 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.591 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.591 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.591 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5763.0:4900 3.a 3:5384606c:::rbd_data.110e1b19a241.0000000000000018:head [read 8192~24576] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-16T20:49:54.591 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:49:54.591 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 8.8844048489999992, 2024-09-16T20:49:54.591 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0379340800000001, 2024-09-16T20:49:54.591 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:54.591 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:54.591 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:54.591 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:54.591 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5763", 2024-09-16T20:49:54.591 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/623897567", 2024-09-16T20:49:54.591 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4900 2024-09-16T20:49:54.591 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.592 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:54.592 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.592 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:54.592 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:49:54.592 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:54.592 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.592 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.592 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:54.592 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:49:54.592 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:54.592 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.592 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.592 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:54.593 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:49:54.593 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4300000000000003e-07 2024-09-16T20:49:54.593 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.593 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.593 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:54.593 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598274+0000", 2024-09-16T20:49:54.593 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.976e-06 2024-09-16T20:49:54.593 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.593 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.593 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:54.593 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598282+0000", 2024-09-16T20:49:54.593 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0029999999999994e-06 2024-09-16T20:49:54.594 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.594 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.594 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:54.594 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598289+0000", 2024-09-16T20:49:54.594 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2389999999999999e-06 2024-09-16T20:49:54.594 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.594 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.594 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:54.594 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598430+0000", 2024-09-16T20:49:54.595 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014119600000000001 2024-09-16T20:49:54.595 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.595 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.595 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting_for_map not empty", 2024-09-16T20:49:54.595 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598435+0000", 2024-09-16T20:49:54.595 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3489999999999999e-06 2024-09-16T20:49:54.595 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.595 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.595 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:54.595 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601429+0000", 2024-09-16T20:49:54.595 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0029942720000000001 2024-09-16T20:49:54.595 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.595 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.595 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:49:54.595 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601438+0000", 2024-09-16T20:49:54.596 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4759999999999998e-06 2024-09-16T20:49:54.596 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.596 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.596 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:54.596 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635987+0000", 2024-09-16T20:49:54.596 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0345495010000001 2024-09-16T20:49:54.596 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.596 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.596 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:54.596 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.636056+0000", 2024-09-16T20:49:54.596 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8610999999999998e-05 2024-09-16T20:49:54.596 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.596 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.597 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:54.597 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.636205+0000", 2024-09-16T20:49:54.597 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014891399999999999 2024-09-16T20:49:54.597 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.597 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:54.597 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.597 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.597 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:54.597 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:54.599 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:49:54.602 INFO:teuthology.orchestra.run.smithi139.stdout:4591 punch from 0x450b000 to 0x450b400, (0x400 bytes) 2024-09-16T20:49:54.602 INFO:teuthology.orchestra.run.smithi139.stdout:4592 read 0x5d7ce00 thru 0x5d833a9 (0x65aa bytes) 2024-09-16T20:49:54.602 INFO:teuthology.orchestra.run.smithi139.stdout:4593 read 0x5d4c400 thru 0x5d4de38 (0x1a39 bytes) 2024-09-16T20:49:54.602 INFO:teuthology.orchestra.run.smithi139.stdout:4595 read 0x7f63600 thru 0x7f6f289 (0xbc8a bytes) 2024-09-16T20:49:54.602 INFO:teuthology.orchestra.run.smithi139.stdout:4598 trunc from 0xbc2b600 to 0x8d74800 2024-09-16T20:49:54.603 INFO:teuthology.orchestra.run.smithi139.stdout:4599 read 0x158f000 thru 0x159504b (0x604c bytes) 2024-09-16T20:49:54.603 INFO:teuthology.orchestra.run.smithi139.stdout:4601 read 0x4d0a600 thru 0x4d0b8e5 (0x12e6 bytes) 2024-09-16T20:49:54.603 INFO:teuthology.orchestra.run.smithi139.stdout:4602 read 0x7c3e00 thru 0x7cb109 (0x730a bytes) 2024-09-16T20:49:54.603 INFO:teuthology.orchestra.run.smithi139.stdout:4603 trunc from 0x8d74800 to 0x77f0c00 2024-09-16T20:49:54.603 INFO:teuthology.orchestra.run.smithi139.stdout:4607 trunc from 0x77f0c00 to 0xb96f800 2024-09-16T20:49:54.603 INFO:teuthology.orchestra.run.smithi139.stdout:4610 read 0x14e7a00 thru 0x14e7d18 (0x319 bytes) 2024-09-16T20:49:54.603 INFO:teuthology.orchestra.run.smithi139.stdout:4611 write 0x7d40c00 thru 0x7d4a35f (0x9760 bytes) 2024-09-16T20:49:54.603 INFO:teuthology.orchestra.run.smithi139.stdout:4617 write 0x26fd800 thru 0x27099b2 (0xc1b3 bytes) 2024-09-16T20:49:54.603 INFO:teuthology.orchestra.run.smithi139.stdout:4619 trunc from 0xb96f800 to 0xd9bf400 2024-09-16T20:49:54.603 INFO:teuthology.orchestra.run.smithi139.stdout:4620 write 0x9136400 thru 0x91397ea (0x33eb bytes) 2024-09-16T20:49:54.603 INFO:teuthology.orchestra.run.smithi139.stdout:4621 trunc from 0xd9bf400 to 0xc52d600 2024-09-16T20:49:54.603 INFO:teuthology.orchestra.run.smithi139.stdout:4624 write 0x226ce00 thru 0x2272299 (0x549a bytes) 2024-09-16T20:49:54.603 INFO:teuthology.orchestra.run.smithi139.stdout:4626 write 0x2e72000 thru 0x2e7f814 (0xd815 bytes) 2024-09-16T20:49:54.603 INFO:teuthology.orchestra.run.smithi139.stdout:4629 write 0xaf22000 thru 0xaf31fe1 (0xffe2 bytes) 2024-09-16T20:49:54.604 INFO:teuthology.orchestra.run.smithi139.stdout:4634 read 0x32e3a00 thru 0x32e865b (0x4c5c bytes) 2024-09-16T20:49:54.604 INFO:teuthology.orchestra.run.smithi139.stdout:4635 read 0x59d6600 thru 0x59e428d (0xdc8e bytes) 2024-09-16T20:49:54.604 INFO:teuthology.orchestra.run.smithi139.stdout:4637 punch from 0x70b8600 to 0x70c7400, (0xee00 bytes) 2024-09-16T20:49:54.604 INFO:teuthology.orchestra.run.smithi139.stdout:4639 punch from 0xc37e200 to 0xc38d200, (0xf000 bytes) 2024-09-16T20:49:54.604 INFO:teuthology.orchestra.run.smithi139.stdout:4641 write 0x9e53a00 thru 0x9e57a16 (0x4017 bytes) 2024-09-16T20:49:54.604 INFO:teuthology.orchestra.run.smithi139.stdout:4642 write 0x5fac200 thru 0x5fac3b6 (0x1b7 bytes) 2024-09-16T20:49:54.604 INFO:teuthology.orchestra.run.smithi139.stdout:4644 trunc from 0xc52d600 to 0xbeece00 2024-09-16T20:49:54.604 INFO:teuthology.orchestra.run.smithi139.stdout:4646 write 0xb354600 thru 0xb35ecfa (0xa6fb bytes) 2024-09-16T20:49:54.604 INFO:teuthology.orchestra.run.smithi139.stdout:4647 write 0x5f7bc00 thru 0x5f85163 (0x9564 bytes) 2024-09-16T20:49:54.604 INFO:teuthology.orchestra.run.smithi139.stdout:4649 punch from 0x8adea00 to 0x8aed400, (0xea00 bytes) 2024-09-16T20:49:54.604 INFO:teuthology.orchestra.run.smithi139.stdout:4651 trunc from 0xbeece00 to 0xcd1a400 2024-09-16T20:49:54.611 INFO:teuthology.orchestra.run.smithi139.stdout:4652 read 0x24bb800 thru 0x24cb479 (0xfc7a bytes) 2024-09-16T20:49:54.619 INFO:teuthology.orchestra.run.smithi139.stdout:4653 read 0x2ea9200 thru 0x2eaa5f0 (0x13f1 bytes) 2024-09-16T20:49:54.619 INFO:teuthology.orchestra.run.smithi139.stdout:4654 read 0x5529200 thru 0x5538910 (0xf711 bytes) 2024-09-16T20:49:54.620 INFO:teuthology.orchestra.run.smithi139.stdout:4657 write 0x10ac200 thru 0x10b345b (0x725c bytes) 2024-09-16T20:49:54.620 INFO:teuthology.orchestra.run.smithi139.stdout:4658 read 0x66de200 thru 0x66e2d61 (0x4b62 bytes) 2024-09-16T20:49:54.621 INFO:teuthology.orchestra.run.smithi139.stdout:4660 write 0x91ad200 thru 0x91ba9c0 (0xd7c1 bytes) 2024-09-16T20:49:54.633 INFO:teuthology.orchestra.run.smithi139.stdout:4662 trunc from 0xcd1a400 to 0x3589200 2024-09-16T20:49:54.639 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T20:49:54.640 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:54.638+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:54.658 INFO:teuthology.orchestra.run.smithi139.stdout:4664 read 0x1932e00 thru 0x1942c57 (0xfe58 bytes) 2024-09-16T20:49:54.666 INFO:teuthology.orchestra.run.smithi139.stdout:4665 read 0x2998000 thru 0x299f8fe (0x78ff bytes) 2024-09-16T20:49:54.674 INFO:teuthology.orchestra.run.smithi139.stdout:4666 trunc from 0x3589200 to 0x9888400 2024-09-16T20:49:54.701 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:54.700+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:54.703 INFO:teuthology.orchestra.run.smithi139.stdout:4669 write 0xc5e6200 thru 0xc5e817f (0x1f80 bytes) 2024-09-16T20:49:54.711 INFO:teuthology.orchestra.run.smithi139.stdout:4672 write 0x42e00 thru 0x51a55 (0xec56 bytes) 2024-09-16T20:49:54.712 INFO:teuthology.orchestra.run.smithi139.stdout:4673 write 0x71c9400 thru 0x71d07b4 (0x73b5 bytes) 2024-09-16T20:49:54.712 INFO:teuthology.orchestra.run.smithi139.stdout:4678 read 0x1310600 thru 0x131eb14 (0xe515 bytes) 2024-09-16T20:49:54.713 INFO:teuthology.orchestra.run.smithi139.stdout:4679 punch from 0x5857200 to 0x5866400, (0xf200 bytes) 2024-09-16T20:49:54.713 INFO:teuthology.orchestra.run.smithi139.stdout:4680 write 0x859fe00 thru 0x85aa2ab (0xa4ac bytes) 2024-09-16T20:49:54.717 INFO:teuthology.orchestra.run.smithi139.stdout:4681 clone 35 order 18 su 2048 sc 9 2024-09-16T20:49:54.756 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:54.756 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:49:54.756 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:49:54.756 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:54.766 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:49:54.796 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:49:54.796 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:54.796 INFO:teuthology.orchestra.run.smithi029.stdout: "epoch": 115, 2024-09-16T20:49:54.796 INFO:teuthology.orchestra.run.smithi029.stdout: "fsid": "e28af4a7-5fcd-4de8-8f77-a2b575361bb1", 2024-09-16T20:49:54.796 INFO:teuthology.orchestra.run.smithi029.stdout: "created": "2024-09-16T20:47:39.760609+0000", 2024-09-16T20:49:54.796 INFO:teuthology.orchestra.run.smithi029.stdout: "modified": "2024-09-16T20:49:52.621404+0000", 2024-09-16T20:49:54.796 INFO:teuthology.orchestra.run.smithi029.stdout: "last_up_change": "2024-09-16T20:47:50.238013+0000", 2024-09-16T20:49:54.797 INFO:teuthology.orchestra.run.smithi029.stdout: "last_in_change": "2024-09-16T20:49:08.779548+0000", 2024-09-16T20:49:54.797 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-16T20:49:54.797 INFO:teuthology.orchestra.run.smithi029.stdout: "flags_num": 5799936, 2024-09-16T20:49:54.797 INFO:teuthology.orchestra.run.smithi029.stdout: "flags_set": [ 2024-09-16T20:49:54.797 INFO:teuthology.orchestra.run.smithi029.stdout: "pglog_hardlimit", 2024-09-16T20:49:54.797 INFO:teuthology.orchestra.run.smithi029.stdout: "purged_snapdirs", 2024-09-16T20:49:54.797 INFO:teuthology.orchestra.run.smithi029.stdout: "recovery_deletes", 2024-09-16T20:49:54.797 INFO:teuthology.orchestra.run.smithi029.stdout: "sortbitwise" 2024-09-16T20:49:54.797 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-16T20:49:54.797 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_version": 5, 2024-09-16T20:49:54.797 INFO:teuthology.orchestra.run.smithi029.stdout: "full_ratio": 0.94999998807907104, 2024-09-16T20:49:54.797 INFO:teuthology.orchestra.run.smithi029.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-16T20:49:54.797 INFO:teuthology.orchestra.run.smithi029.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-16T20:49:54.797 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_snapshot": "", 2024-09-16T20:49:54.798 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_max": 3, 2024-09-16T20:49:54.798 INFO:teuthology.orchestra.run.smithi029.stdout: "max_osd": 6, 2024-09-16T20:49:54.798 INFO:teuthology.orchestra.run.smithi029.stdout: "require_min_compat_client": "luminous", 2024-09-16T20:49:54.798 INFO:teuthology.orchestra.run.smithi029.stdout: "min_compat_client": "luminous", 2024-09-16T20:49:54.798 INFO:teuthology.orchestra.run.smithi029.stdout: "require_osd_release": "squid", 2024-09-16T20:49:54.798 INFO:teuthology.orchestra.run.smithi029.stdout: "allow_crimson": false, 2024-09-16T20:49:54.798 INFO:teuthology.orchestra.run.smithi029.stdout: "pools": [ 2024-09-16T20:49:54.798 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.798 INFO:teuthology.orchestra.run.smithi029.stdout: "pool": 1, 2024-09-16T20:49:54.798 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_name": ".mgr", 2024-09-16T20:49:54.798 INFO:teuthology.orchestra.run.smithi029.stdout: "create_time": "2024-09-16T20:47:52.220750+0000", 2024-09-16T20:49:54.798 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-16T20:49:54.798 INFO:teuthology.orchestra.run.smithi029.stdout: "flags_names": "hashpspool", 2024-09-16T20:49:54.798 INFO:teuthology.orchestra.run.smithi029.stdout: "type": 1, 2024-09-16T20:49:54.799 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 2, 2024-09-16T20:49:54.799 INFO:teuthology.orchestra.run.smithi029.stdout: "min_size": 1, 2024-09-16T20:49:54.799 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_rule": 0, 2024-09-16T20:49:54.799 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_count": 0, 2024-09-16T20:49:54.799 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_target": 0, 2024-09-16T20:49:54.799 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T20:49:54.799 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T20:49:54.799 INFO:teuthology.orchestra.run.smithi029.stdout: "is_stretch_pool": false, 2024-09-16T20:49:54.799 INFO:teuthology.orchestra.run.smithi029.stdout: "object_hash": 2, 2024-09-16T20:49:54.799 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_autoscale_mode": "off", 2024-09-16T20:49:54.799 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num": 1, 2024-09-16T20:49:54.799 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num": 1, 2024-09-16T20:49:54.799 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num_target": 1, 2024-09-16T20:49:54.800 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_target": 1, 2024-09-16T20:49:54.800 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_pending": 1, 2024-09-16T20:49:54.800 INFO:teuthology.orchestra.run.smithi029.stdout: "last_pg_merge_meta": { 2024-09-16T20:49:54.800 INFO:teuthology.orchestra.run.smithi029.stdout: "source_pgid": "0.0", 2024-09-16T20:49:54.800 INFO:teuthology.orchestra.run.smithi029.stdout: "ready_epoch": 0, 2024-09-16T20:49:54.800 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_started": 0, 2024-09-16T20:49:54.800 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_clean": 0, 2024-09-16T20:49:54.800 INFO:teuthology.orchestra.run.smithi029.stdout: "source_version": "0'0", 2024-09-16T20:49:54.800 INFO:teuthology.orchestra.run.smithi029.stdout: "target_version": "0'0" 2024-09-16T20:49:54.800 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.800 INFO:teuthology.orchestra.run.smithi029.stdout: "last_change": "17", 2024-09-16T20:49:54.801 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend": "0", 2024-09-16T20:49:54.801 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T20:49:54.801 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T20:49:54.801 INFO:teuthology.orchestra.run.smithi029.stdout: "auid": 0, 2024-09-16T20:49:54.801 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_mode": "selfmanaged", 2024-09-16T20:49:54.801 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_seq": 0, 2024-09-16T20:49:54.801 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_epoch": 0, 2024-09-16T20:49:54.801 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_snaps": [], 2024-09-16T20:49:54.801 INFO:teuthology.orchestra.run.smithi029.stdout: "removed_snaps": "[]", 2024-09-16T20:49:54.801 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_bytes": 0, 2024-09-16T20:49:54.801 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_objects": 0, 2024-09-16T20:49:54.801 INFO:teuthology.orchestra.run.smithi029.stdout: "tiers": [], 2024-09-16T20:49:54.801 INFO:teuthology.orchestra.run.smithi029.stdout: "tier_of": -1, 2024-09-16T20:49:54.802 INFO:teuthology.orchestra.run.smithi029.stdout: "read_tier": -1, 2024-09-16T20:49:54.802 INFO:teuthology.orchestra.run.smithi029.stdout: "write_tier": -1, 2024-09-16T20:49:54.802 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_mode": "none", 2024-09-16T20:49:54.802 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_bytes": 0, 2024-09-16T20:49:54.802 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_objects": 0, 2024-09-16T20:49:54.802 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T20:49:54.802 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T20:49:54.802 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T20:49:54.802 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_flush_age": 0, 2024-09-16T20:49:54.802 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_evict_age": 0, 2024-09-16T20:49:54.802 INFO:teuthology.orchestra.run.smithi029.stdout: "erasure_code_profile": "", 2024-09-16T20:49:54.802 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_params": { 2024-09-16T20:49:54.802 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "none" 2024-09-16T20:49:54.802 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.803 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_period": 0, 2024-09-16T20:49:54.803 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_count": 0, 2024-09-16T20:49:54.803 INFO:teuthology.orchestra.run.smithi029.stdout: "use_gmt_hitset": true, 2024-09-16T20:49:54.803 INFO:teuthology.orchestra.run.smithi029.stdout: "min_read_recency_for_promote": 0, 2024-09-16T20:49:54.803 INFO:teuthology.orchestra.run.smithi029.stdout: "min_write_recency_for_promote": 0, 2024-09-16T20:49:54.803 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T20:49:54.803 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_search_last_n": 0, 2024-09-16T20:49:54.803 INFO:teuthology.orchestra.run.smithi029.stdout: "grade_table": [], 2024-09-16T20:49:54.803 INFO:teuthology.orchestra.run.smithi029.stdout: "stripe_width": 0, 2024-09-16T20:49:54.803 INFO:teuthology.orchestra.run.smithi029.stdout: "expected_num_objects": 0, 2024-09-16T20:49:54.803 INFO:teuthology.orchestra.run.smithi029.stdout: "fast_read": false, 2024-09-16T20:49:54.803 INFO:teuthology.orchestra.run.smithi029.stdout: "options": { 2024-09-16T20:49:54.803 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_max": 32, 2024-09-16T20:49:54.804 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_min": 1 2024-09-16T20:49:54.804 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.804 INFO:teuthology.orchestra.run.smithi029.stdout: "application_metadata": { 2024-09-16T20:49:54.804 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": {} 2024-09-16T20:49:54.804 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.804 INFO:teuthology.orchestra.run.smithi029.stdout: "read_balance": { 2024-09-16T20:49:54.804 INFO:teuthology.orchestra.run.smithi029.stdout: "score_type": "Fair distribution", 2024-09-16T20:49:54.804 INFO:teuthology.orchestra.run.smithi029.stdout: "score_acting": 6.059999942779541, 2024-09-16T20:49:54.804 INFO:teuthology.orchestra.run.smithi029.stdout: "score_stable": 6.059999942779541, 2024-09-16T20:49:54.804 INFO:teuthology.orchestra.run.smithi029.stdout: "optimal_score": 0.33000001311302185, 2024-09-16T20:49:54.804 INFO:teuthology.orchestra.run.smithi029.stdout: "raw_score_acting": 2, 2024-09-16T20:49:54.804 INFO:teuthology.orchestra.run.smithi029.stdout: "raw_score_stable": 2, 2024-09-16T20:49:54.804 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity_weighted": 1, 2024-09-16T20:49:54.805 INFO:teuthology.orchestra.run.smithi029.stdout: "average_primary_affinity": 1, 2024-09-16T20:49:54.805 INFO:teuthology.orchestra.run.smithi029.stdout: "average_primary_affinity_weighted": 1 2024-09-16T20:49:54.805 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.805 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.805 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.805 INFO:teuthology.orchestra.run.smithi029.stdout: "pool": 2, 2024-09-16T20:49:54.805 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_name": "rbd", 2024-09-16T20:49:54.805 INFO:teuthology.orchestra.run.smithi029.stdout: "create_time": "2024-09-16T20:47:56.756666+0000", 2024-09-16T20:49:54.805 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 8193, 2024-09-16T20:49:54.805 INFO:teuthology.orchestra.run.smithi029.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T20:49:54.805 INFO:teuthology.orchestra.run.smithi029.stdout: "type": 1, 2024-09-16T20:49:54.805 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 2, 2024-09-16T20:49:54.805 INFO:teuthology.orchestra.run.smithi029.stdout: "min_size": 1, 2024-09-16T20:49:54.806 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_rule": 0, 2024-09-16T20:49:54.806 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_count": 0, 2024-09-16T20:49:54.806 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_target": 0, 2024-09-16T20:49:54.806 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T20:49:54.806 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T20:49:54.806 INFO:teuthology.orchestra.run.smithi029.stdout: "is_stretch_pool": false, 2024-09-16T20:49:54.806 INFO:teuthology.orchestra.run.smithi029.stdout: "object_hash": 2, 2024-09-16T20:49:54.806 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_autoscale_mode": "off", 2024-09-16T20:49:54.806 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num": 8, 2024-09-16T20:49:54.806 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num": 8, 2024-09-16T20:49:54.806 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num_target": 8, 2024-09-16T20:49:54.806 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_target": 8, 2024-09-16T20:49:54.806 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_pending": 8, 2024-09-16T20:49:54.806 INFO:teuthology.orchestra.run.smithi029.stdout: "last_pg_merge_meta": { 2024-09-16T20:49:54.807 INFO:teuthology.orchestra.run.smithi029.stdout: "source_pgid": "0.0", 2024-09-16T20:49:54.807 INFO:teuthology.orchestra.run.smithi029.stdout: "ready_epoch": 0, 2024-09-16T20:49:54.807 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_started": 0, 2024-09-16T20:49:54.807 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_clean": 0, 2024-09-16T20:49:54.807 INFO:teuthology.orchestra.run.smithi029.stdout: "source_version": "0'0", 2024-09-16T20:49:54.807 INFO:teuthology.orchestra.run.smithi029.stdout: "target_version": "0'0" 2024-09-16T20:49:54.807 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.807 INFO:teuthology.orchestra.run.smithi029.stdout: "last_change": "21", 2024-09-16T20:49:54.807 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend": "0", 2024-09-16T20:49:54.807 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T20:49:54.807 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T20:49:54.807 INFO:teuthology.orchestra.run.smithi029.stdout: "auid": 0, 2024-09-16T20:49:54.808 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_mode": "selfmanaged", 2024-09-16T20:49:54.808 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_seq": 2, 2024-09-16T20:49:54.808 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_epoch": 21, 2024-09-16T20:49:54.808 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_snaps": [], 2024-09-16T20:49:54.808 INFO:teuthology.orchestra.run.smithi029.stdout: "removed_snaps": "[]", 2024-09-16T20:49:54.808 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_bytes": 0, 2024-09-16T20:49:54.808 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_objects": 0, 2024-09-16T20:49:54.808 INFO:teuthology.orchestra.run.smithi029.stdout: "tiers": [], 2024-09-16T20:49:54.808 INFO:teuthology.orchestra.run.smithi029.stdout: "tier_of": -1, 2024-09-16T20:49:54.808 INFO:teuthology.orchestra.run.smithi029.stdout: "read_tier": -1, 2024-09-16T20:49:54.808 INFO:teuthology.orchestra.run.smithi029.stdout: "write_tier": -1, 2024-09-16T20:49:54.808 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_mode": "none", 2024-09-16T20:49:54.808 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_bytes": 0, 2024-09-16T20:49:54.808 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_objects": 0, 2024-09-16T20:49:54.809 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T20:49:54.809 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T20:49:54.809 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T20:49:54.809 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_flush_age": 0, 2024-09-16T20:49:54.809 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_evict_age": 0, 2024-09-16T20:49:54.809 INFO:teuthology.orchestra.run.smithi029.stdout: "erasure_code_profile": "", 2024-09-16T20:49:54.809 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_params": { 2024-09-16T20:49:54.809 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "none" 2024-09-16T20:49:54.809 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.809 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_period": 0, 2024-09-16T20:49:54.809 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_count": 0, 2024-09-16T20:49:54.809 INFO:teuthology.orchestra.run.smithi029.stdout: "use_gmt_hitset": true, 2024-09-16T20:49:54.809 INFO:teuthology.orchestra.run.smithi029.stdout: "min_read_recency_for_promote": 0, 2024-09-16T20:49:54.809 INFO:teuthology.orchestra.run.smithi029.stdout: "min_write_recency_for_promote": 0, 2024-09-16T20:49:54.810 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T20:49:54.810 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_search_last_n": 0, 2024-09-16T20:49:54.810 INFO:teuthology.orchestra.run.smithi029.stdout: "grade_table": [], 2024-09-16T20:49:54.810 INFO:teuthology.orchestra.run.smithi029.stdout: "stripe_width": 0, 2024-09-16T20:49:54.810 INFO:teuthology.orchestra.run.smithi029.stdout: "expected_num_objects": 0, 2024-09-16T20:49:54.810 INFO:teuthology.orchestra.run.smithi029.stdout: "fast_read": false, 2024-09-16T20:49:54.810 INFO:teuthology.orchestra.run.smithi029.stdout: "options": {}, 2024-09-16T20:49:54.810 INFO:teuthology.orchestra.run.smithi029.stdout: "application_metadata": { 2024-09-16T20:49:54.810 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd": {} 2024-09-16T20:49:54.810 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.810 INFO:teuthology.orchestra.run.smithi029.stdout: "read_balance": { 2024-09-16T20:49:54.810 INFO:teuthology.orchestra.run.smithi029.stdout: "score_type": "Fair distribution", 2024-09-16T20:49:54.810 INFO:teuthology.orchestra.run.smithi029.stdout: "score_acting": 2.25, 2024-09-16T20:49:54.810 INFO:teuthology.orchestra.run.smithi029.stdout: "score_stable": 2.25, 2024-09-16T20:49:54.811 INFO:teuthology.orchestra.run.smithi029.stdout: "optimal_score": 1, 2024-09-16T20:49:54.811 INFO:teuthology.orchestra.run.smithi029.stdout: "raw_score_acting": 2.25, 2024-09-16T20:49:54.811 INFO:teuthology.orchestra.run.smithi029.stdout: "raw_score_stable": 2.25, 2024-09-16T20:49:54.811 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity_weighted": 1, 2024-09-16T20:49:54.811 INFO:teuthology.orchestra.run.smithi029.stdout: "average_primary_affinity": 1, 2024-09-16T20:49:54.811 INFO:teuthology.orchestra.run.smithi029.stdout: "average_primary_affinity_weighted": 1 2024-09-16T20:49:54.811 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.811 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.811 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.811 INFO:teuthology.orchestra.run.smithi029.stdout: "pool": 3, 2024-09-16T20:49:54.811 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_name": "pool_client.0", 2024-09-16T20:49:54.811 INFO:teuthology.orchestra.run.smithi029.stdout: "create_time": "2024-09-16T20:48:06.418889+0000", 2024-09-16T20:49:54.811 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 8193, 2024-09-16T20:49:54.811 INFO:teuthology.orchestra.run.smithi029.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T20:49:54.812 INFO:teuthology.orchestra.run.smithi029.stdout: "type": 1, 2024-09-16T20:49:54.812 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 2, 2024-09-16T20:49:54.812 INFO:teuthology.orchestra.run.smithi029.stdout: "min_size": 1, 2024-09-16T20:49:54.812 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_rule": 0, 2024-09-16T20:49:54.812 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_count": 0, 2024-09-16T20:49:54.812 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_target": 0, 2024-09-16T20:49:54.812 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T20:49:54.812 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T20:49:54.812 INFO:teuthology.orchestra.run.smithi029.stdout: "is_stretch_pool": false, 2024-09-16T20:49:54.812 INFO:teuthology.orchestra.run.smithi029.stdout: "object_hash": 2, 2024-09-16T20:49:54.812 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_autoscale_mode": "off", 2024-09-16T20:49:54.812 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num": 32, 2024-09-16T20:49:54.812 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num": 32, 2024-09-16T20:49:54.813 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num_target": 32, 2024-09-16T20:49:54.813 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_target": 32, 2024-09-16T20:49:54.813 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_pending": 32, 2024-09-16T20:49:54.813 INFO:teuthology.orchestra.run.smithi029.stdout: "last_pg_merge_meta": { 2024-09-16T20:49:54.813 INFO:teuthology.orchestra.run.smithi029.stdout: "source_pgid": "0.0", 2024-09-16T20:49:54.813 INFO:teuthology.orchestra.run.smithi029.stdout: "ready_epoch": 0, 2024-09-16T20:49:54.813 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_started": 0, 2024-09-16T20:49:54.813 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_clean": 0, 2024-09-16T20:49:54.813 INFO:teuthology.orchestra.run.smithi029.stdout: "source_version": "0'0", 2024-09-16T20:49:54.813 INFO:teuthology.orchestra.run.smithi029.stdout: "target_version": "0'0" 2024-09-16T20:49:54.813 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.813 INFO:teuthology.orchestra.run.smithi029.stdout: "last_change": "115", 2024-09-16T20:49:54.813 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend": "0", 2024-09-16T20:49:54.814 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T20:49:54.814 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T20:49:54.814 INFO:teuthology.orchestra.run.smithi029.stdout: "auid": 0, 2024-09-16T20:49:54.814 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_mode": "selfmanaged", 2024-09-16T20:49:54.814 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_seq": 36, 2024-09-16T20:49:54.814 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_epoch": 115, 2024-09-16T20:49:54.814 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_snaps": [], 2024-09-16T20:49:54.814 INFO:teuthology.orchestra.run.smithi029.stdout: "removed_snaps": "[]", 2024-09-16T20:49:54.814 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_bytes": 0, 2024-09-16T20:49:54.814 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_objects": 0, 2024-09-16T20:49:54.814 INFO:teuthology.orchestra.run.smithi029.stdout: "tiers": [], 2024-09-16T20:49:54.815 INFO:teuthology.orchestra.run.smithi029.stdout: "tier_of": -1, 2024-09-16T20:49:54.815 INFO:teuthology.orchestra.run.smithi029.stdout: "read_tier": -1, 2024-09-16T20:49:54.815 INFO:teuthology.orchestra.run.smithi029.stdout: "write_tier": -1, 2024-09-16T20:49:54.815 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_mode": "none", 2024-09-16T20:49:54.815 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_bytes": 0, 2024-09-16T20:49:54.815 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_objects": 0, 2024-09-16T20:49:54.815 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T20:49:54.815 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T20:49:54.815 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T20:49:54.815 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_flush_age": 0, 2024-09-16T20:49:54.815 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_evict_age": 0, 2024-09-16T20:49:54.815 INFO:teuthology.orchestra.run.smithi029.stdout: "erasure_code_profile": "", 2024-09-16T20:49:54.815 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_params": { 2024-09-16T20:49:54.815 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "none" 2024-09-16T20:49:54.816 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.816 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_period": 0, 2024-09-16T20:49:54.816 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_count": 0, 2024-09-16T20:49:54.816 INFO:teuthology.orchestra.run.smithi029.stdout: "use_gmt_hitset": true, 2024-09-16T20:49:54.816 INFO:teuthology.orchestra.run.smithi029.stdout: "min_read_recency_for_promote": 0, 2024-09-16T20:49:54.816 INFO:teuthology.orchestra.run.smithi029.stdout: "min_write_recency_for_promote": 0, 2024-09-16T20:49:54.816 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T20:49:54.816 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_search_last_n": 0, 2024-09-16T20:49:54.816 INFO:teuthology.orchestra.run.smithi029.stdout: "grade_table": [], 2024-09-16T20:49:54.816 INFO:teuthology.orchestra.run.smithi029.stdout: "stripe_width": 0, 2024-09-16T20:49:54.816 INFO:teuthology.orchestra.run.smithi029.stdout: "expected_num_objects": 0, 2024-09-16T20:49:54.816 INFO:teuthology.orchestra.run.smithi029.stdout: "fast_read": false, 2024-09-16T20:49:54.817 INFO:teuthology.orchestra.run.smithi029.stdout: "options": {}, 2024-09-16T20:49:54.817 INFO:teuthology.orchestra.run.smithi029.stdout: "application_metadata": { 2024-09-16T20:49:54.817 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd": {} 2024-09-16T20:49:54.817 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.817 INFO:teuthology.orchestra.run.smithi029.stdout: "read_balance": { 2024-09-16T20:49:54.817 INFO:teuthology.orchestra.run.smithi029.stdout: "score_type": "Fair distribution", 2024-09-16T20:49:54.817 INFO:teuthology.orchestra.run.smithi029.stdout: "score_acting": 1.309999942779541, 2024-09-16T20:49:54.817 INFO:teuthology.orchestra.run.smithi029.stdout: "score_stable": 1.309999942779541, 2024-09-16T20:49:54.817 INFO:teuthology.orchestra.run.smithi029.stdout: "optimal_score": 1, 2024-09-16T20:49:54.817 INFO:teuthology.orchestra.run.smithi029.stdout: "raw_score_acting": 1.309999942779541, 2024-09-16T20:49:54.817 INFO:teuthology.orchestra.run.smithi029.stdout: "raw_score_stable": 1.309999942779541, 2024-09-16T20:49:54.817 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity_weighted": 1, 2024-09-16T20:49:54.817 INFO:teuthology.orchestra.run.smithi029.stdout: "average_primary_affinity": 1, 2024-09-16T20:49:54.817 INFO:teuthology.orchestra.run.smithi029.stdout: "average_primary_affinity_weighted": 1 2024-09-16T20:49:54.818 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.818 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.818 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-16T20:49:54.818 INFO:teuthology.orchestra.run.smithi029.stdout: "osds": [ 2024-09-16T20:49:54.818 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.818 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 0, 2024-09-16T20:49:54.818 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "6c3cb2d7-7d23-4566-9ca1-d7b08e12360c", 2024-09-16T20:49:54.818 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-16T20:49:54.818 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-16T20:49:54.818 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-16T20:49:54.818 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-16T20:49:54.818 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-16T20:49:54.819 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-16T20:49:54.819 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 13, 2024-09-16T20:49:54.819 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 111, 2024-09-16T20:49:54.819 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-16T20:49:54.819 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-16T20:49:54.819 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-16T20:49:54.819 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:54.819 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.819 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:54.819 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6816", 2024-09-16T20:49:54.819 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:49:54.819 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.819 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.819 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:54.820 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6817", 2024-09-16T20:49:54.820 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:49:54.820 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.820 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:54.820 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.820 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-16T20:49:54.820 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:54.820 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.820 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:54.820 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6818", 2024-09-16T20:49:54.820 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:49:54.820 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.820 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.821 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:54.821 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6819", 2024-09-16T20:49:54.821 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:49:54.821 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.821 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:54.821 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.821 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-16T20:49:54.821 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:54.821 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.822 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:54.822 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6822", 2024-09-16T20:49:54.822 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:49:54.822 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.822 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.822 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:54.822 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6823", 2024-09-16T20:49:54.822 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:49:54.822 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.822 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:54.822 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.822 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-16T20:49:54.823 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:54.823 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.823 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:54.823 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6820", 2024-09-16T20:49:54.823 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:49:54.823 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.823 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.823 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:54.823 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6821", 2024-09-16T20:49:54.823 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:49:54.823 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.823 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:54.823 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.823 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.29:6817/2744025949", 2024-09-16T20:49:54.824 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.29:6819/2744025949", 2024-09-16T20:49:54.824 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.29:6823/2744025949", 2024-09-16T20:49:54.824 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.29:6821/2744025949", 2024-09-16T20:49:54.824 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-16T20:49:54.824 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-16T20:49:54.824 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-16T20:49:54.824 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:54.824 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.824 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.824 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 1, 2024-09-16T20:49:54.824 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "5cf8c20c-1ed2-42ae-a8b0-c88cc4e36bce", 2024-09-16T20:49:54.824 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-16T20:49:54.825 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-16T20:49:54.825 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-16T20:49:54.825 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-16T20:49:54.825 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-16T20:49:54.825 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-16T20:49:54.825 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 13, 2024-09-16T20:49:54.825 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 81, 2024-09-16T20:49:54.825 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-16T20:49:54.825 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-16T20:49:54.825 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-16T20:49:54.825 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:54.826 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.826 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:54.826 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6800", 2024-09-16T20:49:54.826 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:49:54.826 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.826 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.826 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:54.826 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6801", 2024-09-16T20:49:54.826 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:49:54.826 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.826 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:54.826 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.827 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-16T20:49:54.827 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:54.827 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.827 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:54.827 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6802", 2024-09-16T20:49:54.827 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:49:54.827 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.827 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.827 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:54.828 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6803", 2024-09-16T20:49:54.828 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:49:54.828 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.828 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:54.828 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.828 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-16T20:49:54.828 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:54.828 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.828 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:54.828 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6806", 2024-09-16T20:49:54.828 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:49:54.829 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.829 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.829 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:54.829 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6807", 2024-09-16T20:49:54.829 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:49:54.829 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.829 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:54.829 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.829 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-16T20:49:54.829 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:54.829 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.829 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:54.829 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6804", 2024-09-16T20:49:54.829 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:49:54.830 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.830 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.830 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:54.830 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6805", 2024-09-16T20:49:54.830 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:49:54.830 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.830 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:54.830 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.830 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.29:6801/208628762", 2024-09-16T20:49:54.830 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.29:6803/208628762", 2024-09-16T20:49:54.830 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.29:6807/208628762", 2024-09-16T20:49:54.831 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.29:6805/208628762", 2024-09-16T20:49:54.831 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-16T20:49:54.831 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-16T20:49:54.831 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-16T20:49:54.831 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:54.831 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.831 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.831 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 2, 2024-09-16T20:49:54.831 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "075d16da-82bf-449b-9f6e-542c9e8d810b", 2024-09-16T20:49:54.831 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-16T20:49:54.831 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-16T20:49:54.831 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-16T20:49:54.831 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-16T20:49:54.832 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-16T20:49:54.832 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-16T20:49:54.832 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 13, 2024-09-16T20:49:54.832 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 102, 2024-09-16T20:49:54.832 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-16T20:49:54.832 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-16T20:49:54.832 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-16T20:49:54.832 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:54.832 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.832 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:54.832 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6808", 2024-09-16T20:49:54.832 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:49:54.832 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.832 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.832 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:54.833 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6809", 2024-09-16T20:49:54.833 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:49:54.833 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.833 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:54.833 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.833 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-16T20:49:54.833 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:54.833 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.833 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:54.833 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6810", 2024-09-16T20:49:54.833 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:49:54.833 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.833 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.834 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:54.834 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6811", 2024-09-16T20:49:54.834 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:49:54.834 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.834 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:54.834 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.834 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-16T20:49:54.834 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:54.834 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.834 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:54.834 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6814", 2024-09-16T20:49:54.834 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:49:54.834 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.834 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.835 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:54.835 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6815", 2024-09-16T20:49:54.835 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:49:54.835 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.835 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:54.835 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.835 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-16T20:49:54.835 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:54.835 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.835 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:54.835 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6812", 2024-09-16T20:49:54.835 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:49:54.835 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.835 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.836 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:54.836 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6813", 2024-09-16T20:49:54.836 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:49:54.836 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.836 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:54.836 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.836 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.29:6809/3447426536", 2024-09-16T20:49:54.836 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.29:6811/3447426536", 2024-09-16T20:49:54.836 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.29:6815/3447426536", 2024-09-16T20:49:54.836 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.29:6813/3447426536", 2024-09-16T20:49:54.836 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-16T20:49:54.836 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-16T20:49:54.836 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-16T20:49:54.837 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:54.837 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.837 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.837 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 3, 2024-09-16T20:49:54.837 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "0612be38-3a24-41b6-aba4-5a2fe985165e", 2024-09-16T20:49:54.837 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-16T20:49:54.837 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-16T20:49:54.837 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-16T20:49:54.837 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-16T20:49:54.837 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-16T20:49:54.837 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-16T20:49:54.837 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 13, 2024-09-16T20:49:54.837 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 96, 2024-09-16T20:49:54.837 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-16T20:49:54.838 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-16T20:49:54.838 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-16T20:49:54.838 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:54.838 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.838 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:54.838 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6808", 2024-09-16T20:49:54.838 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:49:54.838 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.838 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.838 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:54.838 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6809", 2024-09-16T20:49:54.838 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:49:54.838 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.839 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:54.839 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.839 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-16T20:49:54.839 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:54.839 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.839 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:54.839 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6810", 2024-09-16T20:49:54.839 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:49:54.839 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.839 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.839 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:54.839 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6811", 2024-09-16T20:49:54.839 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:49:54.840 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.840 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:54.840 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.840 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-16T20:49:54.840 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:54.840 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.840 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:54.840 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6814", 2024-09-16T20:49:54.840 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:49:54.840 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.840 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.840 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:54.840 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6815", 2024-09-16T20:49:54.840 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:49:54.841 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.841 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:54.841 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.841 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-16T20:49:54.841 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:54.841 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.841 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:54.841 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6812", 2024-09-16T20:49:54.841 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:49:54.841 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.841 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.841 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:54.841 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6813", 2024-09-16T20:49:54.842 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:49:54.842 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.842 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:54.842 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.842 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.43:6809/3687583891", 2024-09-16T20:49:54.842 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.43:6811/3687583891", 2024-09-16T20:49:54.842 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.43:6815/3687583891", 2024-09-16T20:49:54.842 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.43:6813/3687583891", 2024-09-16T20:49:54.842 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-16T20:49:54.842 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-16T20:49:54.842 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-16T20:49:54.842 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:54.842 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.842 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.843 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 4, 2024-09-16T20:49:54.843 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "be590bf8-e9fa-44ca-a471-6f6bcb087ca7", 2024-09-16T20:49:54.843 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-16T20:49:54.843 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-16T20:49:54.843 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-16T20:49:54.843 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-16T20:49:54.843 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-16T20:49:54.843 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-16T20:49:54.843 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 11, 2024-09-16T20:49:54.843 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 112, 2024-09-16T20:49:54.843 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-16T20:49:54.844 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-16T20:49:54.844 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-16T20:49:54.844 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:54.844 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.844 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:54.844 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6800", 2024-09-16T20:49:54.844 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:49:54.844 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.844 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.844 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:54.844 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6801", 2024-09-16T20:49:54.844 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:49:54.844 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.845 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:54.845 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.845 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-16T20:49:54.845 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:54.845 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.845 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:54.845 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6802", 2024-09-16T20:49:54.845 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:49:54.845 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.845 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.845 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:54.845 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6803", 2024-09-16T20:49:54.845 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:49:54.846 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.846 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:54.846 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.846 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-16T20:49:54.846 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:54.846 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.846 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:54.846 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6806", 2024-09-16T20:49:54.846 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:49:54.846 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.846 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.846 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:54.846 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6807", 2024-09-16T20:49:54.847 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:49:54.847 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.847 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:54.847 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.847 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-16T20:49:54.847 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:54.847 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.847 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:54.847 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6804", 2024-09-16T20:49:54.847 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:49:54.847 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.847 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.848 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:54.848 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6805", 2024-09-16T20:49:54.848 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:49:54.848 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.848 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:54.848 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.848 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.43:6801/2505220646", 2024-09-16T20:49:54.848 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.43:6803/2505220646", 2024-09-16T20:49:54.848 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.43:6807/2505220646", 2024-09-16T20:49:54.848 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.43:6805/2505220646", 2024-09-16T20:49:54.848 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-16T20:49:54.848 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-16T20:49:54.848 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-16T20:49:54.848 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:54.849 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.849 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.849 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 5, 2024-09-16T20:49:54.849 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "4a208e30-031b-4a9f-af52-e26d4a039ea6", 2024-09-16T20:49:54.849 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-16T20:49:54.849 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-16T20:49:54.849 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-16T20:49:54.849 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-16T20:49:54.849 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-16T20:49:54.849 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-16T20:49:54.849 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 13, 2024-09-16T20:49:54.849 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 103, 2024-09-16T20:49:54.849 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-16T20:49:54.850 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-16T20:49:54.850 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-16T20:49:54.850 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:54.850 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.850 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:54.850 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6816", 2024-09-16T20:49:54.850 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:49:54.850 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.850 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.850 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:54.850 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6817", 2024-09-16T20:49:54.851 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:49:54.851 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.851 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:54.851 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.851 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-16T20:49:54.851 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:54.851 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.851 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:54.851 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6818", 2024-09-16T20:49:54.851 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:49:54.851 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.851 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.851 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:54.851 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6819", 2024-09-16T20:49:54.852 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:49:54.852 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.852 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:54.852 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.852 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-16T20:49:54.852 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:54.852 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.852 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:54.852 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6822", 2024-09-16T20:49:54.852 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:49:54.852 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.852 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.852 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:54.853 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6823", 2024-09-16T20:49:54.853 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:49:54.853 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.853 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:54.853 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.853 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-16T20:49:54.853 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:49:54.853 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.853 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:49:54.853 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6820", 2024-09-16T20:49:54.853 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:49:54.853 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.853 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.854 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:49:54.854 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6821", 2024-09-16T20:49:54.854 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:49:54.854 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.854 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:54.854 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.854 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.43:6817/1517367307", 2024-09-16T20:49:54.854 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.43:6819/1517367307", 2024-09-16T20:49:54.854 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.43:6823/1517367307", 2024-09-16T20:49:54.854 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.43:6821/1517367307", 2024-09-16T20:49:54.854 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-16T20:49:54.854 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-16T20:49:54.854 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-16T20:49:54.854 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:54.855 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.855 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-16T20:49:54.855 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_xinfo": [ 2024-09-16T20:49:54.855 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.855 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 0, 2024-09-16T20:49:54.855 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-16T20:49:54.855 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-16T20:49:54.855 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-16T20:49:54.855 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540719139924082687, 2024-09-16T20:49:54.855 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-16T20:49:54.855 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-16T20:47:47.887412+0000", 2024-09-16T20:49:54.855 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-16T20:49:54.855 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.855 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.856 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 1, 2024-09-16T20:49:54.856 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-16T20:49:54.856 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-16T20:49:54.856 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-16T20:49:54.856 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540719139924082687, 2024-09-16T20:49:54.856 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-16T20:49:54.856 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-16T20:47:47.658782+0000", 2024-09-16T20:49:54.856 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-16T20:49:54.856 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.856 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.856 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 2, 2024-09-16T20:49:54.856 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-16T20:49:54.857 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-16T20:49:54.857 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-16T20:49:54.857 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540719139924082687, 2024-09-16T20:49:54.857 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-16T20:49:54.857 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-16T20:47:47.453752+0000", 2024-09-16T20:49:54.857 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-16T20:49:54.857 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.857 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.857 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 3, 2024-09-16T20:49:54.857 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-16T20:49:54.857 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-16T20:49:54.857 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-16T20:49:54.857 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540719139924082687, 2024-09-16T20:49:54.858 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-16T20:49:54.858 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-16T20:47:47.487992+0000", 2024-09-16T20:49:54.858 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-16T20:49:54.858 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.858 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.858 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 4, 2024-09-16T20:49:54.858 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-16T20:49:54.858 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-16T20:49:54.858 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-16T20:49:54.858 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540719139924082687, 2024-09-16T20:49:54.858 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-16T20:49:54.858 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-16T20:47:46.943344+0000", 2024-09-16T20:49:54.858 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-16T20:49:54.858 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.858 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.859 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 5, 2024-09-16T20:49:54.859 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-16T20:49:54.859 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-16T20:49:54.859 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-16T20:49:54.859 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540719139924082687, 2024-09-16T20:49:54.859 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-16T20:49:54.859 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-16T20:47:47.901547+0000", 2024-09-16T20:49:54.859 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-16T20:49:54.859 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.859 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-16T20:49:54.859 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_upmap": [ 2024-09-16T20:49:54.859 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.859 INFO:teuthology.orchestra.run.smithi029.stdout: "pgid": "3.2", 2024-09-16T20:49:54.859 INFO:teuthology.orchestra.run.smithi029.stdout: "osds": [ 2024-09-16T20:49:54.860 INFO:teuthology.orchestra.run.smithi029.stdout: 5, 2024-09-16T20:49:54.860 INFO:teuthology.orchestra.run.smithi029.stdout: 0 2024-09-16T20:49:54.860 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:54.860 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.860 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-16T20:49:54.860 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_upmap_items": [ 2024-09-16T20:49:54.860 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.860 INFO:teuthology.orchestra.run.smithi029.stdout: "pgid": "3.a", 2024-09-16T20:49:54.860 INFO:teuthology.orchestra.run.smithi029.stdout: "mappings": [ 2024-09-16T20:49:54.860 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:54.860 INFO:teuthology.orchestra.run.smithi029.stdout: "from": 0, 2024-09-16T20:49:54.860 INFO:teuthology.orchestra.run.smithi029.stdout: "to": 4 2024-09-16T20:49:54.860 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.860 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:54.861 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.861 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-16T20:49:54.861 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_upmap_primaries": [], 2024-09-16T20:49:54.861 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_temp": [], 2024-09-16T20:49:54.861 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_temp": [], 2024-09-16T20:49:54.861 INFO:teuthology.orchestra.run.smithi029.stdout: "blocklist": {}, 2024-09-16T20:49:54.861 INFO:teuthology.orchestra.run.smithi029.stdout: "range_blocklist": {}, 2024-09-16T20:49:54.861 INFO:teuthology.orchestra.run.smithi029.stdout: "erasure_code_profiles": { 2024-09-16T20:49:54.861 INFO:teuthology.orchestra.run.smithi029.stdout: "default": { 2024-09-16T20:49:54.861 INFO:teuthology.orchestra.run.smithi029.stdout: "crush-failure-domain": "osd", 2024-09-16T20:49:54.861 INFO:teuthology.orchestra.run.smithi029.stdout: "k": "2", 2024-09-16T20:49:54.861 INFO:teuthology.orchestra.run.smithi029.stdout: "m": "1", 2024-09-16T20:49:54.861 INFO:teuthology.orchestra.run.smithi029.stdout: "plugin": "jerasure", 2024-09-16T20:49:54.861 INFO:teuthology.orchestra.run.smithi029.stdout: "technique": "reed_sol_van" 2024-09-16T20:49:54.862 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.862 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:54.862 INFO:teuthology.orchestra.run.smithi029.stdout: "removed_snaps_queue": [], 2024-09-16T20:49:54.862 INFO:teuthology.orchestra.run.smithi029.stdout: "new_removed_snaps": [], 2024-09-16T20:49:54.862 INFO:teuthology.orchestra.run.smithi029.stdout: "new_purged_snaps": [], 2024-09-16T20:49:54.862 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_node_flags": {}, 2024-09-16T20:49:54.862 INFO:teuthology.orchestra.run.smithi029.stdout: "device_class_flags": {}, 2024-09-16T20:49:54.862 INFO:teuthology.orchestra.run.smithi029.stdout: "stretch_mode": { 2024-09-16T20:49:54.862 INFO:teuthology.orchestra.run.smithi029.stdout: "stretch_mode_enabled": false, 2024-09-16T20:49:54.862 INFO:teuthology.orchestra.run.smithi029.stdout: "stretch_bucket_count": 0, 2024-09-16T20:49:54.862 INFO:teuthology.orchestra.run.smithi029.stdout: "degraded_stretch_mode": 0, 2024-09-16T20:49:54.862 INFO:teuthology.orchestra.run.smithi029.stdout: "recovering_stretch_mode": 0, 2024-09-16T20:49:54.862 INFO:teuthology.orchestra.run.smithi029.stdout: "stretch_mode_bucket": 0 2024-09-16T20:49:54.862 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:54.863 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:54.864 INFO:tasks.thrashosds.thrasher:j is {'epoch': 115, 'fsid': 'e28af4a7-5fcd-4de8-8f77-a2b575361bb1', 'created': '2024-09-16T20:47:39.760609+0000', 'modified': '2024-09-16T20:49:52.621404+0000', 'last_up_change': '2024-09-16T20:47:50.238013+0000', 'last_in_change': '2024-09-16T20:49:08.779548+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T20:47:52.220750+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-16T20:47:56.756666+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-16T20:48:06.418889+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '115', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 36, 'snap_epoch': 115, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.309999942779541, 'score_stable': 1.309999942779541, 'optimal_score': 1, 'raw_score_acting': 1.309999942779541, 'raw_score_stable': 1.309999942779541, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '6c3cb2d7-7d23-4566-9ca1-d7b08e12360c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 111, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6816', 'nonce': 2744025949}, {'type': 'v1', 'addr': '172.21.15.29:6817', 'nonce': 2744025949}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6818', 'nonce': 2744025949}, {'type': 'v1', 'addr': '172.21.15.29:6819', 'nonce': 2744025949}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6822', 'nonce': 2744025949}, {'type': 'v1', 'addr': '172.21.15.29:6823', 'nonce': 2744025949}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6820', 'nonce': 2744025949}, {'type': 'v1', 'addr': '172.21.15.29:6821', 'nonce': 2744025949}]}, 'public_addr': '172.21.15.29:6817/2744025949', 'cluster_addr': '172.21.15.29:6819/2744025949', 'heartbeat_back_addr': '172.21.15.29:6823/2744025949', 'heartbeat_front_addr': '172.21.15.29:6821/2744025949', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '5cf8c20c-1ed2-42ae-a8b0-c88cc4e36bce', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 81, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6800', 'nonce': 208628762}, {'type': 'v1', 'addr': '172.21.15.29:6801', 'nonce': 208628762}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6802', 'nonce': 208628762}, {'type': 'v1', 'addr': '172.21.15.29:6803', 'nonce': 208628762}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6806', 'nonce': 208628762}, {'type': 'v1', 'addr': '172.21.15.29:6807', 'nonce': 208628762}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6804', 'nonce': 208628762}, {'type': 'v1', 'addr': '172.21.15.29:6805', 'nonce': 208628762}]}, 'public_addr': '172.21.15.29:6801/208628762', 'cluster_addr': '172.21.15.29:6803/208628762', 'heartbeat_back_addr': '172.21.15.29:6807/208628762', 'heartbeat_front_addr': '172.21.15.29:6805/208628762', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '075d16da-82bf-449b-9f6e-542c9e8d810b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 102, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6808', 'nonce': 3447426536}, {'type': 'v1', 'addr': '172.21.15.29:6809', 'nonce': 3447426536}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6810', 'nonce': 3447426536}, {'type': 'v1', 'addr': '172.21.15.29:6811', 'nonce': 3447426536}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6814', 'nonce': 3447426536}, {'type': 'v1', 'addr': '172.21.15.29:6815', 'nonce': 3447426536}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6812', 'nonce': 3447426536}, {'type': 'v1', 'addr': '172.21.15.29:6813', 'nonce': 3447426536}]}, 'public_addr': '172.21.15.29:6809/3447426536', 'cluster_addr': '172.21.15.29:6811/3447426536', 'heartbeat_back_addr': '172.21.15.29:6815/3447426536', 'heartbeat_front_addr': '172.21.15.29:6813/3447426536', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '0612be38-3a24-41b6-aba4-5a2fe985165e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 96, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6808', 'nonce': 3687583891}, {'type': 'v1', 'addr': '172.21.15.43:6809', 'nonce': 3687583891}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6810', 'nonce': 3687583891}, {'type': 'v1', 'addr': '172.21.15.43:6811', 'nonce': 3687583891}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6814', 'nonce': 3687583891}, {'type': 'v1', 'addr': '172.21.15.43:6815', 'nonce': 3687583891}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6812', 'nonce': 3687583891}, {'type': 'v1', 'addr': '172.21.15.43:6813', 'nonce': 3687583891}]}, 'public_addr': '172.21.15.43:6809/3687583891', 'cluster_addr': '172.21.15.43:6811/3687583891', 'heartbeat_back_addr': '172.21.15.43:6815/3687583891', 'heartbeat_front_addr': '172.21.15.43:6813/3687583891', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'be590bf8-e9fa-44ca-a471-6f6bcb087ca7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 112, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6800', 'nonce': 2505220646}, {'type': 'v1', 'addr': '172.21.15.43:6801', 'nonce': 2505220646}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6802', 'nonce': 2505220646}, {'type': 'v1', 'addr': '172.21.15.43:6803', 'nonce': 2505220646}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6806', 'nonce': 2505220646}, {'type': 'v1', 'addr': '172.21.15.43:6807', 'nonce': 2505220646}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6804', 'nonce': 2505220646}, {'type': 'v1', 'addr': '172.21.15.43:6805', 'nonce': 2505220646}]}, 'public_addr': '172.21.15.43:6801/2505220646', 'cluster_addr': '172.21.15.43:6803/2505220646', 'heartbeat_back_addr': '172.21.15.43:6807/2505220646', 'heartbeat_front_addr': '172.21.15.43:6805/2505220646', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '4a208e30-031b-4a9f-af52-e26d4a039ea6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 103, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6816', 'nonce': 1517367307}, {'type': 'v1', 'addr': '172.21.15.43:6817', 'nonce': 1517367307}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6818', 'nonce': 1517367307}, {'type': 'v1', 'addr': '172.21.15.43:6819', 'nonce': 1517367307}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6822', 'nonce': 1517367307}, {'type': 'v1', 'addr': '172.21.15.43:6823', 'nonce': 1517367307}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6820', 'nonce': 1517367307}, {'type': 'v1', 'addr': '172.21.15.43:6821', 'nonce': 1517367307}]}, 'public_addr': '172.21.15.43:6817/1517367307', 'cluster_addr': '172.21.15.43:6819/1517367307', 'heartbeat_back_addr': '172.21.15.43:6823/1517367307', 'heartbeat_front_addr': '172.21.15.43:6821/1517367307', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T20:47:47.887412+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T20:47:47.658782+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T20:47:47.453752+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T20:47:47.487992+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T20:47:46.943344+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T20:47:47.901547+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.2', 'osds': [5, 0]}], 'pg_upmap_items': [{'pgid': '3.a', 'mappings': [{'from': 0, 'to': 4}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-16T20:49:54.865 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T20:49:54.865 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:54.863+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:54.965 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:54.963+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:54.970 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:54.970 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:49:54.970 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:49:54.970 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:49:54.971 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:54.980 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:49:55.066 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:55.064+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:55.130 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:55.130 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:49:55.130 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:49:55.130 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:49:55.131 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.131 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:49:55.131 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:49:55.131 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 121.87384492699999, 2024-09-16T20:49:55.131 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67502346999999996, 2024-09-16T20:49:55.131 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:55.131 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:55.131 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:55.131 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:55.131 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:49:55.131 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:49:55.131 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 1 2024-09-16T20:49:55.132 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.132 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:55.132 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.132 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:55.132 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:49:55.132 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.132 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.132 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.132 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:55.132 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:49:55.132 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.132 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.133 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.133 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:55.133 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254128+0000", 2024-09-16T20:49:55.133 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.463e-06 2024-09-16T20:49:55.133 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.133 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.133 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:55.133 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254135+0000", 2024-09-16T20:49:55.133 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1579999999999999e-06 2024-09-16T20:49:55.133 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.133 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.133 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:55.133 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254146+0000", 2024-09-16T20:49:55.133 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1341e-05 2024-09-16T20:49:55.134 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.134 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.134 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:55.134 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254155+0000", 2024-09-16T20:49:55.134 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.1549999999999996e-06 2024-09-16T20:49:55.134 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.134 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.134 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:55.134 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254221+0000", 2024-09-16T20:49:55.134 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5470999999999997e-05 2024-09-16T20:49:55.134 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.134 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.134 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:49:55.134 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254231+0000", 2024-09-16T20:49:55.134 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0592e-05 2024-09-16T20:49:55.135 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.135 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.135 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:55.135 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926711+0000", 2024-09-16T20:49:55.135 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67247946599999997 2024-09-16T20:49:55.135 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.135 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.135 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:55.135 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926910+0000", 2024-09-16T20:49:55.135 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019934 2024-09-16T20:49:55.135 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.135 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.135 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:49:55.135 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.927159+0000", 2024-09-16T20:49:55.135 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00024876599999999998 2024-09-16T20:49:55.136 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.136 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.136 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:55.136 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.928797+0000", 2024-09-16T20:49:55.136 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001638308 2024-09-16T20:49:55.136 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.136 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.136 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:55.136 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929026+0000", 2024-09-16T20:49:55.136 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00022876099999999999 2024-09-16T20:49:55.136 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.136 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.136 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:55.136 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929102+0000", 2024-09-16T20:49:55.137 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6184000000000001e-05 2024-09-16T20:49:55.137 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.137 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.137 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:55.137 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929150+0000", 2024-09-16T20:49:55.137 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7465e-05 2024-09-16T20:49:55.137 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.137 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:55.137 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.137 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.137 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.137 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3574 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:55.137 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:49:55.137 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 65.646681395000002, 2024-09-16T20:49:55.138 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053579441999999998, 2024-09-16T20:49:55.138 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:55.138 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:55.138 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:55.138 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:55.138 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:55.138 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:55.138 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3574 2024-09-16T20:49:55.138 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.138 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:55.138 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.138 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:55.138 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:49:55.138 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.138 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.139 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.139 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:55.139 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:49:55.139 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.139 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.139 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.139 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:55.139 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:49:55.139 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2e-07 2024-09-16T20:49:55.139 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.139 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.139 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:55.139 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481292+0000", 2024-09-16T20:49:55.139 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2630000000000002e-06 2024-09-16T20:49:55.139 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.140 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.140 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:55.140 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481297+0000", 2024-09-16T20:49:55.140 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.296e-06 2024-09-16T20:49:55.140 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.140 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.140 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:55.140 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481300+0000", 2024-09-16T20:49:55.140 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7019999999999999e-06 2024-09-16T20:49:55.140 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.140 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.140 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:55.140 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525533+0000", 2024-09-16T20:49:55.140 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.044232318 2024-09-16T20:49:55.140 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.141 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.141 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:55.141 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525619+0000", 2024-09-16T20:49:55.141 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5892999999999999e-05 2024-09-16T20:49:55.141 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.141 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.141 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:55.141 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525987+0000", 2024-09-16T20:49:55.141 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00036836099999999999 2024-09-16T20:49:55.141 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.141 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.141 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:55.141 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531779+0000", 2024-09-16T20:49:55.141 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0057918240000000001 2024-09-16T20:49:55.142 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.142 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.142 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:55.142 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534747+0000", 2024-09-16T20:49:55.142 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0029681600000000001 2024-09-16T20:49:55.142 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.142 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.142 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:55.142 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534832+0000", 2024-09-16T20:49:55.142 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5180999999999996e-05 2024-09-16T20:49:55.142 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.142 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.142 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:55.142 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534869+0000", 2024-09-16T20:49:55.142 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7024000000000003e-05 2024-09-16T20:49:55.143 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.143 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:55.143 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.143 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.143 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.143 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3575 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:55.143 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:49:55.143 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 65.646666655000004, 2024-09-16T20:49:55.143 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053801463000000001, 2024-09-16T20:49:55.143 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:55.143 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:55.143 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:55.144 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:55.144 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:55.144 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:55.144 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3575 2024-09-16T20:49:55.144 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.144 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:55.144 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.144 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:55.144 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:49:55.144 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.144 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.144 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.144 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:55.145 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:49:55.145 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.145 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.145 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.145 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:55.145 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481305+0000", 2024-09-16T20:49:55.145 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5400000000000002e-07 2024-09-16T20:49:55.145 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.145 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.145 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:55.145 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481307+0000", 2024-09-16T20:49:55.145 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7800000000000001e-06 2024-09-16T20:49:55.145 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.145 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.146 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:55.146 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481312+0000", 2024-09-16T20:49:55.146 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6349999999999997e-06 2024-09-16T20:49:55.146 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.146 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.146 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:55.147 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481315+0000", 2024-09-16T20:49:55.147 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7829999999999999e-06 2024-09-16T20:49:55.147 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.147 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.147 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:55.147 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.526876+0000", 2024-09-16T20:49:55.148 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.045561070000000002 2024-09-16T20:49:55.148 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.148 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.148 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:55.148 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.526954+0000", 2024-09-16T20:49:55.148 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8108000000000005e-05 2024-09-16T20:49:55.149 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.149 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.149 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:55.149 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527282+0000", 2024-09-16T20:49:55.149 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000327483 2024-09-16T20:49:55.149 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.150 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.150 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:55.150 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531784+0000", 2024-09-16T20:49:55.150 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0045020169999999997 2024-09-16T20:49:55.150 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.150 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.151 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:55.151 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534921+0000", 2024-09-16T20:49:55.151 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.003137791 2024-09-16T20:49:55.151 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.151 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.152 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:55.152 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534976+0000", 2024-09-16T20:49:55.152 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4953000000000001e-05 2024-09-16T20:49:55.152 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.152 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.152 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:55.153 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535106+0000", 2024-09-16T20:49:55.153 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000129589 2024-09-16T20:49:55.153 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.153 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:55.153 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.153 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.154 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.154 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3576 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:55.154 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:49:55.154 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 65.646652743000004, 2024-09-16T20:49:55.154 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053955436000000002, 2024-09-16T20:49:55.155 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:55.155 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:55.155 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:55.155 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:55.155 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:55.155 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:55.156 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3576 2024-09-16T20:49:55.156 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.156 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:55.156 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.156 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:55.157 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:49:55.157 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.157 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.157 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.157 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:55.157 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:49:55.158 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.158 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.158 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.158 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:55.158 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481319+0000", 2024-09-16T20:49:55.159 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.53e-07 2024-09-16T20:49:55.159 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.159 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.159 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:55.159 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481320+0000", 2024-09-16T20:49:55.159 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5230000000000001e-06 2024-09-16T20:49:55.160 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.160 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.160 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:55.160 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481330+0000", 2024-09-16T20:49:55.160 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0190000000000001e-05 2024-09-16T20:49:55.160 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.161 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.161 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:55.161 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481333+0000", 2024-09-16T20:49:55.161 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.151e-06 2024-09-16T20:49:55.161 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.161 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.162 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:55.162 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527689+0000", 2024-09-16T20:49:55.162 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046355192000000003 2024-09-16T20:49:55.162 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.162 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.162 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:55.163 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527764+0000", 2024-09-16T20:49:55.163 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5776000000000001e-05 2024-09-16T20:49:55.163 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.163 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.163 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:55.163 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528141+0000", 2024-09-16T20:49:55.164 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00037647599999999999 2024-09-16T20:49:55.164 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.164 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.164 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:55.164 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531788+0000", 2024-09-16T20:49:55.164 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0036470679999999998 2024-09-16T20:49:55.165 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.165 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.165 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:55.165 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535174+0000", 2024-09-16T20:49:55.165 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0033862670000000001 2024-09-16T20:49:55.165 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.166 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.166 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:55.166 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535236+0000", 2024-09-16T20:49:55.166 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1797999999999993e-05 2024-09-16T20:49:55.166 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.166 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.167 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:55.167 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535274+0000", 2024-09-16T20:49:55.167 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7741999999999999e-05 2024-09-16T20:49:55.167 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.167 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:55.167 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.168 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.168 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.168 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3577 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:55.168 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:49:55.168 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 65.646631142999993, 2024-09-16T20:49:55.168 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054073807000000002, 2024-09-16T20:49:55.169 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:55.169 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:55.169 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:55.169 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:55.169 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:55.169 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:55.170 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3577 2024-09-16T20:49:55.170 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.170 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:55.170 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.170 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:55.171 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:49:55.171 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.171 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.171 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.171 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:55.171 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:49:55.172 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.172 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.172 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.172 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:55.172 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:49:55.172 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.96e-07 2024-09-16T20:49:55.173 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.173 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.173 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:55.173 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481342+0000", 2024-09-16T20:49:55.173 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.714e-06 2024-09-16T20:49:55.173 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.174 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.174 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:55.174 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481349+0000", 2024-09-16T20:49:55.174 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4930000000000001e-06 2024-09-16T20:49:55.174 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.174 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.175 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:55.175 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481353+0000", 2024-09-16T20:49:55.175 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4879999999999999e-06 2024-09-16T20:49:55.175 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.175 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.176 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:55.176 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528619+0000", 2024-09-16T20:49:55.176 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047266351999999998 2024-09-16T20:49:55.176 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.176 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.176 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:55.177 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528706+0000", 2024-09-16T20:49:55.177 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6796e-05 2024-09-16T20:49:55.177 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.177 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.177 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:55.177 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529148+0000", 2024-09-16T20:49:55.178 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00044182999999999999 2024-09-16T20:49:55.178 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.178 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.178 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:55.178 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531792+0000", 2024-09-16T20:49:55.178 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0026443119999999998 2024-09-16T20:49:55.179 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.179 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.179 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:55.179 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535319+0000", 2024-09-16T20:49:55.179 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0035266439999999998 2024-09-16T20:49:55.179 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.180 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.180 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:55.180 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535372+0000", 2024-09-16T20:49:55.180 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3207000000000001e-05 2024-09-16T20:49:55.180 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.180 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.181 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:55.181 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535414+0000", 2024-09-16T20:49:55.181 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1675000000000003e-05 2024-09-16T20:49:55.181 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.181 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:55.181 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.182 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.182 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.182 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3578 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:55.182 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:49:55.182 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 65.646609974, 2024-09-16T20:49:55.183 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054189586999999997, 2024-09-16T20:49:55.183 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:55.183 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:55.183 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:55.183 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:55.183 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:55.184 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:55.184 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3578 2024-09-16T20:49:55.184 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.184 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:55.184 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.184 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:55.185 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:49:55.185 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.185 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.185 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.185 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:55.185 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:49:55.186 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.186 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.186 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.186 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:55.186 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:49:55.187 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0800000000000001e-07 2024-09-16T20:49:55.187 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.187 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.187 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:55.187 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481363+0000", 2024-09-16T20:49:55.187 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0339999999999999e-06 2024-09-16T20:49:55.188 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.188 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.188 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:55.188 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481371+0000", 2024-09-16T20:49:55.188 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4440000000000002e-06 2024-09-16T20:49:55.189 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.189 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.189 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:55.189 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481374+0000", 2024-09-16T20:49:55.189 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2049999999999998e-06 2024-09-16T20:49:55.189 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.190 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.190 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:55.190 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529555+0000", 2024-09-16T20:49:55.190 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048181202999999999 2024-09-16T20:49:55.190 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.190 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.191 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:55.191 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529616+0000", 2024-09-16T20:49:55.191 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0912000000000002e-05 2024-09-16T20:49:55.191 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.191 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.191 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:55.191 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529939+0000", 2024-09-16T20:49:55.192 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00032269800000000001 2024-09-16T20:49:55.192 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.192 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.192 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:55.192 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532602+0000", 2024-09-16T20:49:55.193 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0026634940000000002 2024-09-16T20:49:55.193 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.193 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.193 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:55.193 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535465+0000", 2024-09-16T20:49:55.193 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.002862149 2024-09-16T20:49:55.194 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.194 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.194 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:55.194 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535516+0000", 2024-09-16T20:49:55.194 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1696000000000001e-05 2024-09-16T20:49:55.195 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.195 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.195 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:55.195 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535551+0000", 2024-09-16T20:49:55.195 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4443999999999998e-05 2024-09-16T20:49:55.195 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.196 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:55.196 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.196 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.196 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.196 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3579 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:55.197 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:49:55.197 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 65.646591534999999, 2024-09-16T20:49:55.197 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054295151, 2024-09-16T20:49:55.197 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:55.197 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:55.198 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:55.198 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:55.198 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:55.198 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:55.199 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3579 2024-09-16T20:49:55.199 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.199 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:55.199 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.199 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:55.199 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:49:55.200 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.200 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.200 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.200 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:55.200 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:49:55.200 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.201 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.201 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.201 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:55.201 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:49:55.201 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3300000000000001e-07 2024-09-16T20:49:55.202 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.202 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.202 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:55.202 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481381+0000", 2024-09-16T20:49:55.202 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.649e-06 2024-09-16T20:49:55.202 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.203 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.203 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:55.203 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481387+0000", 2024-09-16T20:49:55.203 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4519999999999996e-06 2024-09-16T20:49:55.203 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.203 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.204 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:55.204 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481390+0000", 2024-09-16T20:49:55.204 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0170000000000001e-06 2024-09-16T20:49:55.204 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.204 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.205 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:55.205 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530756+0000", 2024-09-16T20:49:55.205 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049365882999999999 2024-09-16T20:49:55.205 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.205 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.205 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:55.206 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530845+0000", 2024-09-16T20:49:55.206 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9197000000000007e-05 2024-09-16T20:49:55.206 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.206 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.206 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:55.206 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531191+0000", 2024-09-16T20:49:55.207 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00034562500000000001 2024-09-16T20:49:55.207 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.207 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.207 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:55.207 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534078+0000", 2024-09-16T20:49:55.207 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0028878179999999999 2024-09-16T20:49:55.207 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.207 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.207 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:55.208 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535589+0000", 2024-09-16T20:49:55.208 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00151085 2024-09-16T20:49:55.208 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.208 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.208 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:55.208 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535639+0000", 2024-09-16T20:49:55.208 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9697000000000002e-05 2024-09-16T20:49:55.208 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.208 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.208 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:55.208 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535675+0000", 2024-09-16T20:49:55.208 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5729999999999998e-05 2024-09-16T20:49:55.208 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.208 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:55.209 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.209 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.209 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.209 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3580 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:55.209 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:49:55.209 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 65.646577902999994, 2024-09-16T20:49:55.209 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054397361999999998, 2024-09-16T20:49:55.209 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:55.209 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:55.209 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:55.209 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:55.209 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:55.209 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:55.209 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3580 2024-09-16T20:49:55.210 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.210 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:55.210 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.210 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:55.210 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:49:55.210 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.210 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.210 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.210 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:55.210 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:49:55.210 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.210 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.210 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.210 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:55.211 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:49:55.211 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2600000000000001e-07 2024-09-16T20:49:55.211 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.211 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.211 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:55.211 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481395+0000", 2024-09-16T20:49:55.211 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.544e-06 2024-09-16T20:49:55.211 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.211 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.211 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:55.211 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481404+0000", 2024-09-16T20:49:55.211 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8850000000000005e-06 2024-09-16T20:49:55.211 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.211 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.212 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:55.212 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481407+0000", 2024-09-16T20:49:55.212 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1540000000000002e-06 2024-09-16T20:49:55.212 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.212 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.212 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:55.212 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531823+0000", 2024-09-16T20:49:55.212 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050415637999999999 2024-09-16T20:49:55.212 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.212 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.212 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:55.212 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531895+0000", 2024-09-16T20:49:55.212 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2179000000000001e-05 2024-09-16T20:49:55.212 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.212 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.213 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:55.213 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532236+0000", 2024-09-16T20:49:55.213 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00034073799999999999 2024-09-16T20:49:55.213 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.213 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.213 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:55.213 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534083+0000", 2024-09-16T20:49:55.213 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001847298 2024-09-16T20:49:55.213 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.213 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.213 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:55.213 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535714+0000", 2024-09-16T20:49:55.213 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0016316810000000001 2024-09-16T20:49:55.214 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.214 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.214 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:55.214 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535759+0000", 2024-09-16T20:49:55.214 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4478999999999999e-05 2024-09-16T20:49:55.214 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.214 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.214 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:55.214 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535791+0000", 2024-09-16T20:49:55.214 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.154e-05 2024-09-16T20:49:55.214 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.214 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:55.214 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.214 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.214 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.215 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3581 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:49:55.215 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:49:55.215 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 65.646555906000003, 2024-09-16T20:49:55.215 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.057094403000000002, 2024-09-16T20:49:55.215 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:55.215 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:55.215 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:55.215 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:55.215 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:49:55.215 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:55.215 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3581 2024-09-16T20:49:55.215 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.215 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:55.215 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.215 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:55.216 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:49:55.216 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.216 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.216 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.216 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:55.216 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:49:55.216 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.216 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.216 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.216 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:55.216 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:49:55.216 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8799999999999998e-07 2024-09-16T20:49:55.216 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.217 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.217 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:55.217 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481418+0000", 2024-09-16T20:49:55.217 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1859999999999999e-06 2024-09-16T20:49:55.217 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.217 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.217 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:55.217 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481423+0000", 2024-09-16T20:49:55.217 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8279999999999999e-06 2024-09-16T20:49:55.217 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.217 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.217 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:55.217 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481426+0000", 2024-09-16T20:49:55.217 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0110000000000001e-06 2024-09-16T20:49:55.217 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.218 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.218 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:55.218 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532631+0000", 2024-09-16T20:49:55.218 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051204514999999999 2024-09-16T20:49:55.218 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.218 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.218 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:55.218 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.533381+0000", 2024-09-16T20:49:55.218 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00075029000000000003 2024-09-16T20:49:55.218 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.218 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.218 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:55.218 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.533739+0000", 2024-09-16T20:49:55.218 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00035773100000000002 2024-09-16T20:49:55.218 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.219 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.219 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:55.219 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535123+0000", 2024-09-16T20:49:55.219 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0013836160000000001 2024-09-16T20:49:55.219 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.219 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.219 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:55.219 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538413+0000", 2024-09-16T20:49:55.219 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0032902510000000001 2024-09-16T20:49:55.219 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.219 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.219 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:55.219 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538470+0000", 2024-09-16T20:49:55.219 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6697999999999998e-05 2024-09-16T20:49:55.220 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.220 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.220 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:55.220 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538510+0000", 2024-09-16T20:49:55.220 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9988999999999999e-05 2024-09-16T20:49:55.220 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.220 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:55.220 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.220 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.220 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.220 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:12732 3.d 3:b752444a:::rbd_header.110e747f0779:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-16T20:49:55.220 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:49:55.220 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 50.359483328000003, 2024-09-16T20:49:55.220 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.065302467000000003, 2024-09-16T20:49:55.221 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:55.221 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:55.221 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:55.221 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:55.221 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:49:55.221 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:49:55.221 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 12732 2024-09-16T20:49:55.221 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.221 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:55.221 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.221 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:55.221 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:49:55.221 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.222 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.222 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.222 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:55.222 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:49:55.222 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.222 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.222 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.222 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:55.222 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768490+0000", 2024-09-16T20:49:55.222 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2859999999999998e-06 2024-09-16T20:49:55.222 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.222 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.222 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:55.222 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768501+0000", 2024-09-16T20:49:55.223 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1226999999999999e-05 2024-09-16T20:49:55.223 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.223 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.223 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:55.223 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768517+0000", 2024-09-16T20:49:55.223 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.592e-05 2024-09-16T20:49:55.223 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.223 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.223 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:55.223 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768528+0000", 2024-09-16T20:49:55.223 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0689999999999999e-05 2024-09-16T20:49:55.223 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.223 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.223 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:55.224 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768584+0000", 2024-09-16T20:49:55.224 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5889e-05 2024-09-16T20:49:55.224 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.224 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.224 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:55.224 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768720+0000", 2024-09-16T20:49:55.224 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000136718 2024-09-16T20:49:55.224 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.224 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.224 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:55.224 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.769066+0000", 2024-09-16T20:49:55.224 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000346013 2024-09-16T20:49:55.224 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.225 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.225 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:49:55.225 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833659+0000", 2024-09-16T20:49:55.225 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.064592822999999994 2024-09-16T20:49:55.225 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.225 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.225 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:55.225 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833719+0000", 2024-09-16T20:49:55.225 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9441000000000002e-05 2024-09-16T20:49:55.225 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.225 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.225 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:55.225 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833758+0000", 2024-09-16T20:49:55.225 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9122000000000003e-05 2024-09-16T20:49:55.225 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.226 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.226 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:55.226 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833790+0000", 2024-09-16T20:49:55.226 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2338000000000001e-05 2024-09-16T20:49:55.226 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.226 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:55.226 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.226 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.226 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.226 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.5089.0:12732 3.d e73/23 ondisk, result = 0)", 2024-09-16T20:49:55.226 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:49:55.227 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 50.357881157999998, 2024-09-16T20:49:55.227 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.063706456999999994, 2024-09-16T20:49:55.227 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:55.227 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:55.227 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:55.227 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:55.227 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.227 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:55.227 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:49:55.227 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.227 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.227 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.227 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:55.228 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:49:55.228 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.228 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.228 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.228 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:55.228 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770091+0000", 2024-09-16T20:49:55.228 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.53e-06 2024-09-16T20:49:55.228 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.228 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.228 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:55.228 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770104+0000", 2024-09-16T20:49:55.228 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2203e-05 2024-09-16T20:49:55.228 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.228 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.228 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:55.229 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833637+0000", 2024-09-16T20:49:55.229 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.063533201999999997 2024-09-16T20:49:55.229 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.229 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.229 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:55.229 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833655+0000", 2024-09-16T20:49:55.229 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8369e-05 2024-09-16T20:49:55.229 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.229 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.229 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:55.229 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833707+0000", 2024-09-16T20:49:55.229 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2061999999999997e-05 2024-09-16T20:49:55.229 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.229 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.230 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:55.230 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833715+0000", 2024-09-16T20:49:55.230 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5179999999999998e-06 2024-09-16T20:49:55.230 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.230 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.230 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:55.230 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833796+0000", 2024-09-16T20:49:55.230 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1573e-05 2024-09-16T20:49:55.230 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.230 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:55.230 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.230 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.230 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.230 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3281 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 220160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:55.230 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041070+0000", 2024-09-16T20:49:55.231 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 5.0869006890000001, 2024-09-16T20:49:55.231 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051593550000000002, 2024-09-16T20:49:55.231 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:55.231 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:55.231 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:55.231 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:55.231 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:55.231 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:55.231 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3281 2024-09-16T20:49:55.231 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.231 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:55.231 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.231 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:55.231 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041070+0000", 2024-09-16T20:49:55.232 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.232 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.232 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.232 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:55.232 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041070+0000", 2024-09-16T20:49:55.232 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.232 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.232 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.232 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:55.232 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041071+0000", 2024-09-16T20:49:55.232 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1899999999999999e-07 2024-09-16T20:49:55.232 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.232 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.232 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:55.233 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041072+0000", 2024-09-16T20:49:55.233 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.248e-06 2024-09-16T20:49:55.233 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.233 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.233 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:55.233 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041075+0000", 2024-09-16T20:49:55.233 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0029999999999999e-06 2024-09-16T20:49:55.233 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.233 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.233 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:55.233 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041077+0000", 2024-09-16T20:49:55.233 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.621e-06 2024-09-16T20:49:55.233 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.233 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.233 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:55.234 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.092431+0000", 2024-09-16T20:49:55.234 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051353801999999997 2024-09-16T20:49:55.234 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.234 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.234 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:55.234 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.092512+0000", 2024-09-16T20:49:55.234 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1106000000000006e-05 2024-09-16T20:49:55.234 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.234 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.234 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:55.234 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.092664+0000", 2024-09-16T20:49:55.234 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00015155100000000001 2024-09-16T20:49:55.234 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.234 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:55.235 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.235 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.235 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.235 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3282 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 225280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:55.235 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041081+0000", 2024-09-16T20:49:55.235 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 5.0868904109999997, 2024-09-16T20:49:55.235 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051731948, 2024-09-16T20:49:55.235 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:55.235 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:55.235 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:55.235 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:55.235 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:55.235 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:55.235 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3282 2024-09-16T20:49:55.235 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.236 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:55.236 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.236 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:55.236 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041081+0000", 2024-09-16T20:49:55.236 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.236 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.236 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.236 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:55.236 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041081+0000", 2024-09-16T20:49:55.236 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.236 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.236 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.236 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:55.236 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041081+0000", 2024-09-16T20:49:55.237 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3099999999999999e-07 2024-09-16T20:49:55.237 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.237 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.237 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:55.237 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041082+0000", 2024-09-16T20:49:55.237 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1289999999999999e-06 2024-09-16T20:49:55.237 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.237 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.237 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:55.237 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041085+0000", 2024-09-16T20:49:55.237 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2320000000000001e-06 2024-09-16T20:49:55.237 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.237 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.238 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:55.238 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041090+0000", 2024-09-16T20:49:55.238 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3730000000000003e-06 2024-09-16T20:49:55.238 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.238 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.238 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:55.238 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.092678+0000", 2024-09-16T20:49:55.238 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051588235000000003 2024-09-16T20:49:55.238 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.238 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.238 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:55.238 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.092717+0000", 2024-09-16T20:49:55.238 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9057999999999998e-05 2024-09-16T20:49:55.238 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.239 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.239 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:55.239 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.092813+0000", 2024-09-16T20:49:55.239 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.569e-05 2024-09-16T20:49:55.239 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.239 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:55.239 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.239 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.239 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.239 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3283 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 230400~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:55.239 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041102+0000", 2024-09-16T20:49:55.239 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 5.0868687509999999, 2024-09-16T20:49:55.239 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051974723, 2024-09-16T20:49:55.239 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:55.240 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:55.240 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:55.240 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:55.240 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:55.240 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:55.240 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3283 2024-09-16T20:49:55.240 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.240 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:55.240 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.240 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:55.240 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041102+0000", 2024-09-16T20:49:55.240 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.240 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.240 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.241 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:55.241 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041102+0000", 2024-09-16T20:49:55.241 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.241 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.241 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.241 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:55.241 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041103+0000", 2024-09-16T20:49:55.241 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2399999999999999e-07 2024-09-16T20:49:55.241 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.241 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.241 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:55.241 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041110+0000", 2024-09-16T20:49:55.241 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4290000000000004e-06 2024-09-16T20:49:55.241 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.242 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.242 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:55.242 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041118+0000", 2024-09-16T20:49:55.242 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3849999999999996e-06 2024-09-16T20:49:55.242 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.242 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.242 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:55.242 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041123+0000", 2024-09-16T20:49:55.242 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1410000000000004e-06 2024-09-16T20:49:55.242 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.242 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.242 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:55.242 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.092845+0000", 2024-09-16T20:49:55.242 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051722207999999999 2024-09-16T20:49:55.243 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.243 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.243 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:55.243 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.092926+0000", 2024-09-16T20:49:55.243 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0727000000000001e-05 2024-09-16T20:49:55.243 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.243 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.243 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:55.243 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093077+0000", 2024-09-16T20:49:55.243 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000151409 2024-09-16T20:49:55.243 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.243 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:55.243 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.243 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.244 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.244 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3284 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 235520~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:55.244 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041129+0000", 2024-09-16T20:49:55.244 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 5.08684242, 2024-09-16T20:49:55.244 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052120760000000002, 2024-09-16T20:49:55.244 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:55.244 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:55.244 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:55.244 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:55.244 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:55.244 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:55.244 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3284 2024-09-16T20:49:55.244 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.244 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:55.245 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.245 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:55.245 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041129+0000", 2024-09-16T20:49:55.245 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.245 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.245 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.245 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:55.245 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041129+0000", 2024-09-16T20:49:55.245 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.245 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.245 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.245 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:55.245 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041129+0000", 2024-09-16T20:49:55.245 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3700000000000001e-07 2024-09-16T20:49:55.246 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.246 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.246 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:55.246 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041130+0000", 2024-09-16T20:49:55.246 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4050000000000001e-06 2024-09-16T20:49:55.246 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.246 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.246 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:55.246 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041134+0000", 2024-09-16T20:49:55.246 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4290000000000001e-06 2024-09-16T20:49:55.246 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.246 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.246 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:55.246 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041136+0000", 2024-09-16T20:49:55.246 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.678e-06 2024-09-16T20:49:55.247 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.247 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.247 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:55.247 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093109+0000", 2024-09-16T20:49:55.247 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051972879999999999 2024-09-16T20:49:55.247 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.247 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.247 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:55.247 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093149+0000", 2024-09-16T20:49:55.247 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0040000000000003e-05 2024-09-16T20:49:55.247 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.247 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.247 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:55.247 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093249+0000", 2024-09-16T20:49:55.248 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.9990999999999999e-05 2024-09-16T20:49:55.248 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.248 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:55.248 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.248 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.248 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.248 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3285 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 240640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:55.248 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041140+0000", 2024-09-16T20:49:55.248 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 5.0868313799999996, 2024-09-16T20:49:55.248 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052375646999999997, 2024-09-16T20:49:55.248 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:55.248 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:55.248 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:55.248 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:55.249 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:55.249 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:55.249 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3285 2024-09-16T20:49:55.249 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.249 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:55.249 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.249 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:55.249 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041140+0000", 2024-09-16T20:49:55.249 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.249 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.249 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.249 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:55.249 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041140+0000", 2024-09-16T20:49:55.249 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.249 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.250 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.250 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:55.250 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041140+0000", 2024-09-16T20:49:55.250 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2399999999999999e-07 2024-09-16T20:49:55.250 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.250 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.250 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:55.250 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041141+0000", 2024-09-16T20:49:55.250 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2759999999999999e-06 2024-09-16T20:49:55.250 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.250 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.250 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:55.250 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041144+0000", 2024-09-16T20:49:55.250 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0280000000000001e-06 2024-09-16T20:49:55.250 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.251 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.251 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:55.251 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041147+0000", 2024-09-16T20:49:55.251 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5900000000000002e-06 2024-09-16T20:49:55.251 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.251 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.251 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:55.251 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093284+0000", 2024-09-16T20:49:55.251 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052137645000000003 2024-09-16T20:49:55.251 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.251 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.251 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:55.251 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093365+0000", 2024-09-16T20:49:55.251 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0446999999999997e-05 2024-09-16T20:49:55.252 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.252 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.252 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:55.252 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093515+0000", 2024-09-16T20:49:55.252 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00015043700000000001 2024-09-16T20:49:55.252 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.252 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:55.252 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.252 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.252 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.252 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3286 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 245760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:55.252 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:49:55.252 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 5.0868192209999998, 2024-09-16T20:49:55.252 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052523564000000002, 2024-09-16T20:49:55.253 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:55.253 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:55.253 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:55.253 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:55.253 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:55.253 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:55.253 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3286 2024-09-16T20:49:55.253 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.253 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:55.253 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.253 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:55.253 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:49:55.253 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.253 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.254 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.254 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:55.254 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:49:55.254 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.254 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.254 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.254 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:55.254 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:49:55.254 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3900000000000001e-07 2024-09-16T20:49:55.254 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.254 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.254 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:55.254 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041153+0000", 2024-09-16T20:49:55.254 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.119e-06 2024-09-16T20:49:55.255 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.255 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.255 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:55.255 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041156+0000", 2024-09-16T20:49:55.255 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0000000000000001e-06 2024-09-16T20:49:55.255 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.255 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.255 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:55.255 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041159+0000", 2024-09-16T20:49:55.255 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5859999999999999e-06 2024-09-16T20:49:55.255 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.255 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.255 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:55.255 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093529+0000", 2024-09-16T20:49:55.256 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052370591000000001 2024-09-16T20:49:55.256 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.256 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.256 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:55.256 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093570+0000", 2024-09-16T20:49:55.256 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0161999999999999e-05 2024-09-16T20:49:55.256 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.256 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.256 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:55.256 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093675+0000", 2024-09-16T20:49:55.256 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010586699999999999 2024-09-16T20:49:55.256 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.256 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:55.256 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.257 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.257 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.257 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3287 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 250880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:55.257 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:49:55.257 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 5.0868091719999997, 2024-09-16T20:49:55.257 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052781203999999998, 2024-09-16T20:49:55.257 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:55.257 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:55.257 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:55.257 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:55.257 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:55.257 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:55.257 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3287 2024-09-16T20:49:55.258 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.258 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:55.258 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.258 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:55.258 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:49:55.258 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.258 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.258 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.258 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:55.258 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:49:55.258 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.258 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.258 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.258 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:55.259 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:49:55.259 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1e-07 2024-09-16T20:49:55.259 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.259 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.259 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:55.259 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041163+0000", 2024-09-16T20:49:55.259 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0690000000000001e-06 2024-09-16T20:49:55.259 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.259 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.259 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:55.259 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041166+0000", 2024-09-16T20:49:55.259 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.982e-06 2024-09-16T20:49:55.259 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.259 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.260 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:55.260 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041169+0000", 2024-09-16T20:49:55.260 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.593e-06 2024-09-16T20:49:55.260 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.260 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.260 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:55.260 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093709+0000", 2024-09-16T20:49:55.260 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052540492000000001 2024-09-16T20:49:55.260 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.260 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.260 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:55.260 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093792+0000", 2024-09-16T20:49:55.260 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2436999999999997e-05 2024-09-16T20:49:55.260 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.261 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.261 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:55.261 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093943+0000", 2024-09-16T20:49:55.261 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000151421 2024-09-16T20:49:55.261 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.261 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:55.261 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.261 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.261 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.261 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3288 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 256000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:55.261 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:49:55.261 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 5.0867992989999999, 2024-09-16T20:49:55.261 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052928269, 2024-09-16T20:49:55.262 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:55.262 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:55.262 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:55.262 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:55.262 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:55.262 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:55.262 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3288 2024-09-16T20:49:55.262 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.262 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:55.262 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.262 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:55.262 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:49:55.262 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.262 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.263 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.263 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:55.263 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:49:55.263 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.263 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.263 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.263 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:55.263 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:49:55.263 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2700000000000001e-07 2024-09-16T20:49:55.263 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.263 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.263 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:55.263 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041173+0000", 2024-09-16T20:49:55.263 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.113e-06 2024-09-16T20:49:55.263 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.264 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.264 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:55.264 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041176+0000", 2024-09-16T20:49:55.264 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9799999999999998e-06 2024-09-16T20:49:55.264 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.264 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.264 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:55.264 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041179+0000", 2024-09-16T20:49:55.264 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.667e-06 2024-09-16T20:49:55.264 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.264 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.264 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:55.264 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093957+0000", 2024-09-16T20:49:55.264 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052778703000000003 2024-09-16T20:49:55.264 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.265 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.265 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:55.265 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093999+0000", 2024-09-16T20:49:55.265 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1300000000000001e-05 2024-09-16T20:49:55.265 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.265 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.265 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:55.265 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094100+0000", 2024-09-16T20:49:55.265 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101279 2024-09-16T20:49:55.265 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.265 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:55.265 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.265 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.265 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.266 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3289 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 261120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:55.266 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:49:55.266 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 5.0867893029999998, 2024-09-16T20:49:55.266 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053182976, 2024-09-16T20:49:55.266 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:55.266 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:55.266 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:55.266 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:55.266 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:49:55.266 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:55.266 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3289 2024-09-16T20:49:55.266 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.266 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:55.267 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.267 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:55.267 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:49:55.267 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.267 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.267 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.267 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:55.267 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:49:55.267 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.267 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.267 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.267 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:55.267 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:49:55.267 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2999999999999999e-07 2024-09-16T20:49:55.268 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.268 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.268 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:55.268 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041183+0000", 2024-09-16T20:49:55.268 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0720000000000001e-06 2024-09-16T20:49:55.268 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.268 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.268 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:55.268 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041186+0000", 2024-09-16T20:49:55.268 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9869999999999999e-06 2024-09-16T20:49:55.268 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.268 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.268 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:55.268 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041189+0000", 2024-09-16T20:49:55.269 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.587e-06 2024-09-16T20:49:55.269 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.269 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.269 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:55.269 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094136+0000", 2024-09-16T20:49:55.269 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052947270999999997 2024-09-16T20:49:55.269 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.269 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.269 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:55.269 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094213+0000", 2024-09-16T20:49:55.269 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7552000000000006e-05 2024-09-16T20:49:55.269 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.269 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.269 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:55.270 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094365+0000", 2024-09-16T20:49:55.270 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00015127700000000001 2024-09-16T20:49:55.270 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.270 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:55.270 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.270 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.270 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:55.270 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:55.272 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:49:55.273 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:49:55.273 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-09-16T20:49:55.274 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":173,"stamp":"2024-09-16T20:49:54.224288+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1249315356,"num_objects":1460,"num_object_clones":388,"num_object_copies":2920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1460,"num_whiteouts":396,"num_read":273128,"num_read_kb":992511,"num_write":14574,"num_write_kb":264942,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1619,"num_bytes_recovered":64971730,"num_keys_recovered":414,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12239,"ondisk_log_size":12239,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":622352,"kb_used_data":364440,"kb_used_omap":9,"kb_used_meta":257782,"kb_avail":565608688,"statfs":{"total":579820584960,"available":579183296512,"internally_reserved":0,"allocated":373186560,"data_stored":394091391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9567,"internal_metadata":263969441},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,0,13,88],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29427579,"num_objects":136,"num_object_clones":0,"num_object_copies":272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":136,"num_whiteouts":3,"num_read":31697,"num_read_kb":71095,"num_write":755,"num_write_kb":17003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":3988269,"num_keys_recovered":26,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000826"},"pg_stats":[{"pgid":"3.1f","version":"114'223","reported_seq":9019,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-16T20:49:50.756492+0000","last_change":"2024-09-16T20:49:12.802355+0000","last_active":"2024-09-16T20:49:50.756492+0000","last_peered":"2024-09-16T20:49:50.756492+0000","last_clean":"2024-09-16T20:49:50.756492+0000","last_became_active":"2024-09-16T20:49:12.801566+0000","last_became_peered":"2024-09-16T20:49:12.801566+0000","last_unstale":"2024-09-16T20:49:50.756492+0000","last_undegraded":"2024-09-16T20:49:50.756492+0000","last_fullsized":"2024-09-16T20:49:50.756492+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":223,"log_dups_size":0,"ondisk_log_size":223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:08:47.359376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071634499999999996,"stat_sum":{"num_bytes":32666785,"num_objects":44,"num_object_clones":7,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":9,"num_read":8785,"num_read_kb":23646,"num_write":211,"num_write_kb":4057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":2125998,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"114'525","reported_seq":15893,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-16T20:49:50.773155+0000","last_change":"2024-09-16T20:48:53.732932+0000","last_active":"2024-09-16T20:49:50.773155+0000","last_peered":"2024-09-16T20:49:50.773155+0000","last_clean":"2024-09-16T20:49:50.773155+0000","last_became_active":"2024-09-16T20:48:53.732081+0000","last_became_peered":"2024-09-16T20:48:53.732081+0000","last_unstale":"2024-09-16T20:49:50.773155+0000","last_undegraded":"2024-09-16T20:49:50.773155+0000","last_fullsized":"2024-09-16T20:49:50.773155+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:41:11.310839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084864999999999995,"stat_sum":{"num_bytes":47148705,"num_objects":42,"num_object_clones":11,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":9,"num_read":16990,"num_read_kb":48996,"num_write":650,"num_write_kb":6601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":1415412,"num_keys_recovered":59,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"114'452","reported_seq":9695,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-16T20:49:50.456581+0000","last_change":"2024-09-16T20:48:09.564507+0000","last_active":"2024-09-16T20:49:50.456581+0000","last_peered":"2024-09-16T20:49:50.456581+0000","last_clean":"2024-09-16T20:49:50.456581+0000","last_became_active":"2024-09-16T20:48:07.568025+0000","last_became_peered":"2024-09-16T20:48:07.568025+0000","last_unstale":"2024-09-16T20:49:50.456581+0000","last_undegraded":"2024-09-16T20:49:50.456581+0000","last_fullsized":"2024-09-16T20:49:50.456581+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:36:08.746533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068985699999999999,"stat_sum":{"num_bytes":68759106,"num_objects":50,"num_object_clones":15,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":12,"num_read":10582,"num_read_kb":77338,"num_write":567,"num_write_kb":18608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"114'436","reported_seq":2479,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-16T20:49:50.136591+0000","last_change":"2024-09-16T20:48:09.565661+0000","last_active":"2024-09-16T20:49:50.136591+0000","last_peered":"2024-09-16T20:49:50.136591+0000","last_clean":"2024-09-16T20:49:50.136591+0000","last_became_active":"2024-09-16T20:48:07.565241+0000","last_became_peered":"2024-09-16T20:48:07.565241+0000","last_unstale":"2024-09-16T20:49:50.136591+0000","last_undegraded":"2024-09-16T20:49:50.136591+0000","last_fullsized":"2024-09-16T20:49:50.136591+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":436,"log_dups_size":0,"ondisk_log_size":436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:37:48.864804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001045589,"stat_sum":{"num_bytes":19587637,"num_objects":46,"num_object_clones":10,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":14,"num_read":3478,"num_read_kb":20125,"num_write":679,"num_write_kb":12931,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"114'583","reported_seq":3992,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-16T20:49:50.147506+0000","last_change":"2024-09-16T20:48:54.735381+0000","last_active":"2024-09-16T20:49:50.147506+0000","last_peered":"2024-09-16T20:49:50.147506+0000","last_clean":"2024-09-16T20:49:50.147506+0000","last_became_active":"2024-09-16T20:48:54.734278+0000","last_became_peered":"2024-09-16T20:48:54.734278+0000","last_unstale":"2024-09-16T20:49:50.147506+0000","last_undegraded":"2024-09-16T20:49:50.147506+0000","last_fullsized":"2024-09-16T20:49:50.147506+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":583,"log_dups_size":0,"ondisk_log_size":583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:37:18.614009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057432399999999997,"stat_sum":{"num_bytes":24361800,"num_objects":51,"num_object_clones":14,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":14,"num_read":5051,"num_read_kb":20865,"num_write":846,"num_write_kb":10514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":1963751,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"114'754","reported_seq":13959,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-16T20:49:50.567735+0000","last_change":"2024-09-16T20:49:12.803894+0000","last_active":"2024-09-16T20:49:50.567735+0000","last_peered":"2024-09-16T20:49:50.567735+0000","last_clean":"2024-09-16T20:49:50.567735+0000","last_became_active":"2024-09-16T20:49:12.802869+0000","last_became_peered":"2024-09-16T20:49:12.802869+0000","last_unstale":"2024-09-16T20:49:50.567735+0000","last_undegraded":"2024-09-16T20:49:50.567735+0000","last_fullsized":"2024-09-16T20:49:50.567735+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":754,"log_dups_size":0,"ondisk_log_size":754,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:34:19.455643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002037849,"stat_sum":{"num_bytes":13444036,"num_objects":57,"num_object_clones":14,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":16,"num_read":17813,"num_read_kb":39266,"num_write":953,"num_write_kb":5967,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":1051004,"num_keys_recovered":46,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"114'402","reported_seq":6652,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-16T20:49:52.791021+0000","last_change":"2024-09-16T20:48:53.732127+0000","last_active":"2024-09-16T20:49:52.791021+0000","last_peered":"2024-09-16T20:49:52.791021+0000","last_clean":"2024-09-16T20:49:52.791021+0000","last_became_active":"2024-09-16T20:48:53.731485+0000","last_became_peered":"2024-09-16T20:48:53.731485+0000","last_unstale":"2024-09-16T20:49:52.791021+0000","last_undegraded":"2024-09-16T20:49:52.791021+0000","last_fullsized":"2024-09-16T20:49:52.791021+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:33:03.799903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057190699999999997,"stat_sum":{"num_bytes":28130432,"num_objects":51,"num_object_clones":17,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":14,"num_read":6670,"num_read_kb":23224,"num_write":502,"num_write_kb":6572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":5091328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"114'265","reported_seq":6602,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-16T20:49:50.846335+0000","last_change":"2024-09-16T20:48:53.731382+0000","last_active":"2024-09-16T20:49:50.846335+0000","last_peered":"2024-09-16T20:49:50.846335+0000","last_clean":"2024-09-16T20:49:50.846335+0000","last_became_active":"2024-09-16T20:48:53.730714+0000","last_became_peered":"2024-09-16T20:48:53.730714+0000","last_unstale":"2024-09-16T20:49:50.846335+0000","last_undegraded":"2024-09-16T20:49:50.846335+0000","last_fullsized":"2024-09-16T20:49:50.846335+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":265,"log_dups_size":0,"ondisk_log_size":265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:11:53.413358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011061790000000001,"stat_sum":{"num_bytes":19552272,"num_objects":34,"num_object_clones":9,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":9,"num_read":7850,"num_read_kb":27088,"num_write":216,"num_write_kb":6824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2412544,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"114'722","reported_seq":6190,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-16T20:49:50.085203+0000","last_change":"2024-09-16T20:49:12.804399+0000","last_active":"2024-09-16T20:49:50.085203+0000","last_peered":"2024-09-16T20:49:50.085203+0000","last_clean":"2024-09-16T20:49:50.085203+0000","last_became_active":"2024-09-16T20:49:12.803628+0000","last_became_peered":"2024-09-16T20:49:12.803628+0000","last_unstale":"2024-09-16T20:49:50.085203+0000","last_undegraded":"2024-09-16T20:49:50.085203+0000","last_fullsized":"2024-09-16T20:49:50.085203+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":722,"log_dups_size":0,"ondisk_log_size":722,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:31:21.619194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057915500000000003,"stat_sum":{"num_bytes":12105914,"num_objects":32,"num_object_clones":6,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":8,"num_read":10285,"num_read_kb":12456,"num_write":777,"num_write_kb":3306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":331889,"num_keys_recovered":60,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"114'692","reported_seq":8442,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-16T20:49:50.155770+0000","last_change":"2024-09-16T20:48:53.731777+0000","last_active":"2024-09-16T20:49:50.155770+0000","last_peered":"2024-09-16T20:49:50.155770+0000","last_clean":"2024-09-16T20:49:50.155770+0000","last_became_active":"2024-09-16T20:48:53.730899+0000","last_became_peered":"2024-09-16T20:48:53.730899+0000","last_unstale":"2024-09-16T20:49:50.155770+0000","last_undegraded":"2024-09-16T20:49:50.155770+0000","last_fullsized":"2024-09-16T20:49:50.155770+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:48:10.926607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010075049999999999,"stat_sum":{"num_bytes":26578112,"num_objects":50,"num_object_clones":14,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":13,"num_read":12594,"num_read_kb":35475,"num_write":792,"num_write_kb":6180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":6235436,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"114'216","reported_seq":9016,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-16T20:49:50.477248+0000","last_change":"2024-09-16T20:48:53.734382+0000","last_active":"2024-09-16T20:49:50.477248+0000","last_peered":"2024-09-16T20:49:50.477248+0000","last_clean":"2024-09-16T20:49:50.477248+0000","last_became_active":"2024-09-16T20:48:53.733689+0000","last_became_peered":"2024-09-16T20:48:53.733689+0000","last_unstale":"2024-09-16T20:49:50.477248+0000","last_undegraded":"2024-09-16T20:49:50.477248+0000","last_fullsized":"2024-09-16T20:49:50.477248+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:16:09.589101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051100699999999995,"stat_sum":{"num_bytes":42216960,"num_objects":54,"num_object_clones":17,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":18,"num_read":8417,"num_read_kb":26253,"num_write":207,"num_write_kb":5225,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":2749952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"114'221","reported_seq":11766,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-16T20:49:51.227961+0000","last_change":"2024-09-16T20:48:09.566005+0000","last_active":"2024-09-16T20:49:51.227961+0000","last_peered":"2024-09-16T20:49:51.227961+0000","last_clean":"2024-09-16T20:49:51.227961+0000","last_became_active":"2024-09-16T20:48:07.568236+0000","last_became_peered":"2024-09-16T20:48:07.568236+0000","last_unstale":"2024-09-16T20:49:51.227961+0000","last_undegraded":"2024-09-16T20:49:51.227961+0000","last_fullsized":"2024-09-16T20:49:51.227961+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:42:06.738449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001271749,"stat_sum":{"num_bytes":60345058,"num_objects":48,"num_object_clones":15,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":13,"num_read":11697,"num_read_kb":42551,"num_write":370,"num_write_kb":10794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":206,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-16T20:49:49.615957+0000","last_change":"2024-09-16T20:48:00.277670+0000","last_active":"2024-09-16T20:49:49.615957+0000","last_peered":"2024-09-16T20:49:49.615957+0000","last_clean":"2024-09-16T20:49:49.615957+0000","last_became_active":"2024-09-16T20:47:58.274489+0000","last_became_peered":"2024-09-16T20:47:58.274489+0000","last_unstale":"2024-09-16T20:49:49.615957+0000","last_undegraded":"2024-09-16T20:49:49.615957+0000","last_fullsized":"2024-09-16T20:49:49.615957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:27:12.633064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091339500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"114'608","reported_seq":4981,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-16T20:49:53.123637+0000","last_change":"2024-09-16T20:48:50.757942+0000","last_active":"2024-09-16T20:49:53.123637+0000","last_peered":"2024-09-16T20:49:53.123637+0000","last_clean":"2024-09-16T20:49:53.123637+0000","last_became_active":"2024-09-16T20:48:50.719161+0000","last_became_peered":"2024-09-16T20:48:50.719161+0000","last_unstale":"2024-09-16T20:49:53.123637+0000","last_undegraded":"2024-09-16T20:49:53.123637+0000","last_fullsized":"2024-09-16T20:49:53.123637+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":608,"log_dups_size":0,"ondisk_log_size":608,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:06:04.207589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067924999999999995,"stat_sum":{"num_bytes":17545307,"num_objects":36,"num_object_clones":7,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":6,"num_read":6494,"num_read_kb":22820,"num_write":930,"num_write_kb":11546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":466506,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":207,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-16T20:49:52.627214+0000","last_change":"2024-09-16T20:48:00.277182+0000","last_active":"2024-09-16T20:49:52.627214+0000","last_peered":"2024-09-16T20:49:52.627214+0000","last_clean":"2024-09-16T20:49:52.627214+0000","last_became_active":"2024-09-16T20:47:58.274389+0000","last_became_peered":"2024-09-16T20:47:58.274389+0000","last_unstale":"2024-09-16T20:49:52.627214+0000","last_undegraded":"2024-09-16T20:49:52.627214+0000","last_fullsized":"2024-09-16T20:49:52.627214+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:28:02.441830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043659299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"115'391","reported_seq":16008,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-16T20:49:52.951864+0000","last_change":"2024-09-16T20:48:50.791999+0000","last_active":"2024-09-16T20:49:52.951864+0000","last_peered":"2024-09-16T20:49:52.951864+0000","last_clean":"2024-09-16T20:49:52.951864+0000","last_became_active":"2024-09-16T20:48:50.719868+0000","last_became_peered":"2024-09-16T20:48:50.719868+0000","last_unstale":"2024-09-16T20:49:52.951864+0000","last_undegraded":"2024-09-16T20:49:52.951864+0000","last_fullsized":"2024-09-16T20:49:52.951864+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:32:14.546760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012934750000000001,"stat_sum":{"num_bytes":60890719,"num_objects":38,"num_object_clones":11,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":8,"num_read":15393,"num_read_kb":41881,"num_write":292,"num_write_kb":12444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":523776,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":206,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-16T20:49:49.615653+0000","last_change":"2024-09-16T20:48:00.278054+0000","last_active":"2024-09-16T20:49:49.615653+0000","last_peered":"2024-09-16T20:49:49.615653+0000","last_clean":"2024-09-16T20:49:49.615653+0000","last_became_active":"2024-09-16T20:47:58.272861+0000","last_became_peered":"2024-09-16T20:47:58.272861+0000","last_unstale":"2024-09-16T20:49:49.615653+0000","last_undegraded":"2024-09-16T20:49:49.615653+0000","last_fullsized":"2024-09-16T20:49:49.615653+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:13:59.135547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012429489999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"114'314","reported_seq":2635,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-16T20:49:52.627029+0000","last_change":"2024-09-16T20:49:12.802627+0000","last_active":"2024-09-16T20:49:52.627029+0000","last_peered":"2024-09-16T20:49:52.627029+0000","last_clean":"2024-09-16T20:49:52.627029+0000","last_became_active":"2024-09-16T20:49:12.801810+0000","last_became_peered":"2024-09-16T20:49:12.801810+0000","last_unstale":"2024-09-16T20:49:52.627029+0000","last_undegraded":"2024-09-16T20:49:52.627029+0000","last_fullsized":"2024-09-16T20:49:52.627029+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:32:34.775034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076367000000000004,"stat_sum":{"num_bytes":29879333,"num_objects":40,"num_object_clones":13,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":11,"num_read":2389,"num_read_kb":9013,"num_write":392,"num_write_kb":5145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":2071114,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":81,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-16T20:49:49.666365+0000","last_change":"2024-09-16T20:49:09.557800+0000","last_active":"2024-09-16T20:49:49.666365+0000","last_peered":"2024-09-16T20:49:49.666365+0000","last_clean":"2024-09-16T20:49:49.666365+0000","last_became_active":"2024-09-16T20:49:09.557513+0000","last_became_peered":"2024-09-16T20:49:09.557513+0000","last_unstale":"2024-09-16T20:49:49.666365+0000","last_undegraded":"2024-09-16T20:49:49.666365+0000","last_fullsized":"2024-09-16T20:49:49.666365+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:38:59.143723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"114'293","reported_seq":9897,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-16T20:49:51.121933+0000","last_change":"2024-09-16T20:48:53.732314+0000","last_active":"2024-09-16T20:49:51.121933+0000","last_peered":"2024-09-16T20:49:51.121933+0000","last_clean":"2024-09-16T20:49:51.121933+0000","last_became_active":"2024-09-16T20:48:53.731287+0000","last_became_peered":"2024-09-16T20:48:53.731287+0000","last_unstale":"2024-09-16T20:49:51.121933+0000","last_undegraded":"2024-09-16T20:49:51.121933+0000","last_fullsized":"2024-09-16T20:49:51.121933+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:36:07.507817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010471600000000001,"stat_sum":{"num_bytes":67465388,"num_objects":55,"num_object_clones":11,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":12,"num_read":9209,"num_read_kb":36447,"num_write":305,"num_write_kb":10437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":176266,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"114'333","reported_seq":5283,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-16T20:49:50.567742+0000","last_change":"2024-09-16T20:48:09.570254+0000","last_active":"2024-09-16T20:49:50.567742+0000","last_peered":"2024-09-16T20:49:50.567742+0000","last_clean":"2024-09-16T20:49:50.567742+0000","last_became_active":"2024-09-16T20:48:07.568143+0000","last_became_peered":"2024-09-16T20:48:07.568143+0000","last_unstale":"2024-09-16T20:49:50.567742+0000","last_undegraded":"2024-09-16T20:49:50.567742+0000","last_fullsized":"2024-09-16T20:49:50.567742+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:53:29.227864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038800899999999998,"stat_sum":{"num_bytes":48884851,"num_objects":57,"num_object_clones":19,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":22,"num_read":5692,"num_read_kb":12548,"num_write":334,"num_write_kb":2657,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":208,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-16T20:49:49.711625+0000","last_change":"2024-09-16T20:48:00.276970+0000","last_active":"2024-09-16T20:49:49.711625+0000","last_peered":"2024-09-16T20:49:49.711625+0000","last_clean":"2024-09-16T20:49:49.711625+0000","last_became_active":"2024-09-16T20:47:58.274617+0000","last_became_peered":"2024-09-16T20:47:58.274617+0000","last_unstale":"2024-09-16T20:49:49.711625+0000","last_undegraded":"2024-09-16T20:49:49.711625+0000","last_fullsized":"2024-09-16T20:49:49.711625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:28:54.467905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062077499999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"114'296","reported_seq":8186,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-16T20:49:50.719521+0000","last_change":"2024-09-16T20:49:12.801644+0000","last_active":"2024-09-16T20:49:50.719521+0000","last_peered":"2024-09-16T20:49:50.719521+0000","last_clean":"2024-09-16T20:49:50.719521+0000","last_became_active":"2024-09-16T20:49:12.800827+0000","last_became_peered":"2024-09-16T20:49:12.800827+0000","last_unstale":"2024-09-16T20:49:50.719521+0000","last_undegraded":"2024-09-16T20:49:50.719521+0000","last_fullsized":"2024-09-16T20:49:50.719521+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":296,"log_dups_size":0,"ondisk_log_size":296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:36:56.895707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088331900000000003,"stat_sum":{"num_bytes":49126219,"num_objects":41,"num_object_clones":6,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":8,"num_read":9674,"num_read_kb":49358,"num_write":263,"num_write_kb":13399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":1931020,"num_keys_recovered":36,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":121,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-16T20:49:49.711389+0000","last_change":"2024-09-16T20:49:09.559342+0000","last_active":"2024-09-16T20:49:49.711389+0000","last_peered":"2024-09-16T20:49:49.711389+0000","last_clean":"2024-09-16T20:49:49.711389+0000","last_became_active":"2024-09-16T20:49:09.558884+0000","last_became_peered":"2024-09-16T20:49:09.558884+0000","last_unstale":"2024-09-16T20:49:49.711389+0000","last_undegraded":"2024-09-16T20:49:49.711389+0000","last_fullsized":"2024-09-16T20:49:49.711389+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:51:23.264062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"114'213","reported_seq":4559,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-16T20:49:52.626946+0000","last_change":"2024-09-16T20:48:09.565596+0000","last_active":"2024-09-16T20:49:52.626946+0000","last_peered":"2024-09-16T20:49:52.626946+0000","last_clean":"2024-09-16T20:49:52.626946+0000","last_became_active":"2024-09-16T20:48:07.566097+0000","last_became_peered":"2024-09-16T20:48:07.566097+0000","last_unstale":"2024-09-16T20:49:52.626946+0000","last_undegraded":"2024-09-16T20:49:52.626946+0000","last_fullsized":"2024-09-16T20:49:52.626946+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:32:07.842492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012840460000000001,"stat_sum":{"num_bytes":51287552,"num_objects":39,"num_object_clones":11,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":13,"num_read":4191,"num_read_kb":28135,"num_write":212,"num_write_kb":7088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":218,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-16T20:49:49.615222+0000","last_change":"2024-09-16T20:49:09.560350+0000","last_active":"2024-09-16T20:49:49.615222+0000","last_peered":"2024-09-16T20:49:49.615222+0000","last_clean":"2024-09-16T20:49:49.615222+0000","last_became_active":"2024-09-16T20:49:09.559789+0000","last_became_peered":"2024-09-16T20:49:09.559789+0000","last_unstale":"2024-09-16T20:49:49.615222+0000","last_undegraded":"2024-09-16T20:49:49.615222+0000","last_fullsized":"2024-09-16T20:49:49.615222+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:17:54.706074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037636000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"114'393","reported_seq":5201,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-16T20:49:50.174477+0000","last_change":"2024-09-16T20:49:37.570005+0000","last_active":"2024-09-16T20:49:50.174477+0000","last_peered":"2024-09-16T20:49:50.174477+0000","last_clean":"2024-09-16T20:49:50.174477+0000","last_became_active":"2024-09-16T20:49:37.568717+0000","last_became_peered":"2024-09-16T20:49:37.568717+0000","last_unstale":"2024-09-16T20:49:50.174477+0000","last_undegraded":"2024-09-16T20:49:50.174477+0000","last_fullsized":"2024-09-16T20:49:50.174477+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:26:45.686697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063857599999999997,"stat_sum":{"num_bytes":25212483,"num_objects":56,"num_object_clones":16,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":17,"num_read":7747,"num_read_kb":18878,"num_write":302,"num_write_kb":3362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":127,"num_bytes_recovered":2183758,"num_keys_recovered":57,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":207,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-16T20:49:49.615750+0000","last_change":"2024-09-16T20:48:00.277739+0000","last_active":"2024-09-16T20:49:49.615750+0000","last_peered":"2024-09-16T20:49:49.615750+0000","last_clean":"2024-09-16T20:49:49.615750+0000","last_became_active":"2024-09-16T20:47:58.271174+0000","last_became_peered":"2024-09-16T20:47:58.271174+0000","last_unstale":"2024-09-16T20:49:49.615750+0000","last_undegraded":"2024-09-16T20:49:49.615750+0000","last_fullsized":"2024-09-16T20:49:49.615750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:11:35.985075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095771799999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":321,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-16T20:49:49.616336+0000","last_change":"2024-09-16T20:49:09.565188+0000","last_active":"2024-09-16T20:49:49.616336+0000","last_peered":"2024-09-16T20:49:49.616336+0000","last_clean":"2024-09-16T20:49:49.616336+0000","last_became_active":"2024-09-16T20:49:09.559571+0000","last_became_peered":"2024-09-16T20:49:09.559571+0000","last_unstale":"2024-09-16T20:49:49.616336+0000","last_undegraded":"2024-09-16T20:49:49.616336+0000","last_fullsized":"2024-09-16T20:49:49.616336+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:52.241408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:52.241408+0000","last_clean_scrub_stamp":"2024-09-16T20:47:52.241408+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:01:53.651981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"114'487","reported_seq":4392,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-16T20:49:53.095930+0000","last_change":"2024-09-16T20:48:09.565136+0000","last_active":"2024-09-16T20:49:53.095930+0000","last_peered":"2024-09-16T20:49:53.095930+0000","last_clean":"2024-09-16T20:49:53.095930+0000","last_became_active":"2024-09-16T20:48:07.566139+0000","last_became_peered":"2024-09-16T20:48:07.566139+0000","last_unstale":"2024-09-16T20:49:53.095930+0000","last_undegraded":"2024-09-16T20:49:53.095930+0000","last_fullsized":"2024-09-16T20:49:53.095930+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:01:40.042518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001025172,"stat_sum":{"num_bytes":24467034,"num_objects":49,"num_object_clones":10,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":12,"num_read":5270,"num_read_kb":15132,"num_write":715,"num_write_kb":5872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"114'245","reported_seq":6553,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-16T20:49:51.028164+0000","last_change":"2024-09-16T20:48:53.737981+0000","last_active":"2024-09-16T20:49:51.028164+0000","last_peered":"2024-09-16T20:49:51.028164+0000","last_clean":"2024-09-16T20:49:51.028164+0000","last_became_active":"2024-09-16T20:48:53.736558+0000","last_became_peered":"2024-09-16T20:48:53.736558+0000","last_unstale":"2024-09-16T20:49:51.028164+0000","last_undegraded":"2024-09-16T20:49:51.028164+0000","last_fullsized":"2024-09-16T20:49:51.028164+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":245,"log_dups_size":0,"ondisk_log_size":245,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:07:53.898723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044614999999999998,"stat_sum":{"num_bytes":46246432,"num_objects":45,"num_object_clones":16,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":14,"num_read":6887,"num_read_kb":34922,"num_write":254,"num_write_kb":10221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":2635264,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"115'238","reported_seq":6001,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-16T20:49:53.067255+0000","last_change":"2024-09-16T20:49:48.616148+0000","last_active":"2024-09-16T20:49:53.067255+0000","last_peered":"2024-09-16T20:49:53.067255+0000","last_clean":"2024-09-16T20:49:53.067255+0000","last_became_active":"2024-09-16T20:49:48.614776+0000","last_became_peered":"2024-09-16T20:49:48.614776+0000","last_unstale":"2024-09-16T20:49:53.067255+0000","last_undegraded":"2024-09-16T20:49:53.067255+0000","last_fullsized":"2024-09-16T20:49:53.067255+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":238,"log_dups_size":0,"ondisk_log_size":238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:53:14.137079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039392600000000001,"stat_sum":{"num_bytes":40527149,"num_objects":47,"num_object_clones":13,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":11,"num_read":5501,"num_read_kb":30567,"num_write":383,"num_write_kb":7605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":3988269,"num_keys_recovered":26,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"114'257","reported_seq":5443,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-16T20:49:49.868190+0000","last_change":"2024-09-16T20:48:54.735384+0000","last_active":"2024-09-16T20:49:49.868190+0000","last_peered":"2024-09-16T20:49:49.868190+0000","last_clean":"2024-09-16T20:49:49.868190+0000","last_became_active":"2024-09-16T20:48:54.734409+0000","last_became_peered":"2024-09-16T20:48:54.734409+0000","last_unstale":"2024-09-16T20:49:49.868190+0000","last_undegraded":"2024-09-16T20:49:49.868190+0000","last_fullsized":"2024-09-16T20:49:49.868190+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":257,"log_dups_size":0,"ondisk_log_size":257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:42:31.993413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014289540000000001,"stat_sum":{"num_bytes":38590025,"num_objects":36,"num_object_clones":9,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":15,"num_read":5212,"num_read_kb":13335,"num_write":278,"num_write_kb":4713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":971906,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"114'348","reported_seq":13633,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-16T20:49:50.398087+0000","last_change":"2024-09-16T20:49:30.561716+0000","last_active":"2024-09-16T20:49:50.398087+0000","last_peered":"2024-09-16T20:49:50.398087+0000","last_clean":"2024-09-16T20:49:50.398087+0000","last_became_active":"2024-09-16T20:49:30.560372+0000","last_became_peered":"2024-09-16T20:49:30.560372+0000","last_unstale":"2024-09-16T20:49:50.398087+0000","last_undegraded":"2024-09-16T20:49:50.398087+0000","last_fullsized":"2024-09-16T20:49:50.398087+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:36:22.006764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047762999999999999,"stat_sum":{"num_bytes":64263579,"num_objects":37,"num_object_clones":9,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":12,"num_read":14097,"num_read_kb":49977,"num_write":501,"num_write_kb":10787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":163,"num_bytes_recovered":18718820,"num_keys_recovered":48,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"114'535","reported_seq":8118,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-16T20:49:51.087273+0000","last_change":"2024-09-16T20:48:09.565667+0000","last_active":"2024-09-16T20:49:51.087273+0000","last_peered":"2024-09-16T20:49:51.087273+0000","last_clean":"2024-09-16T20:49:51.087273+0000","last_became_active":"2024-09-16T20:48:07.567327+0000","last_became_peered":"2024-09-16T20:48:07.567327+0000","last_unstale":"2024-09-16T20:49:51.087273+0000","last_undegraded":"2024-09-16T20:49:51.087273+0000","last_fullsized":"2024-09-16T20:49:51.087273+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:39:07.550242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012087560000000001,"stat_sum":{"num_bytes":28832071,"num_objects":42,"num_object_clones":12,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":11,"num_read":8970,"num_read_kb":26652,"num_write":662,"num_write_kb":8964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"114'190","reported_seq":6032,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-16T20:49:50.137069+0000","last_change":"2024-09-16T20:48:09.565024+0000","last_active":"2024-09-16T20:49:50.137069+0000","last_peered":"2024-09-16T20:49:50.137069+0000","last_clean":"2024-09-16T20:49:50.137069+0000","last_became_active":"2024-09-16T20:48:07.570704+0000","last_became_peered":"2024-09-16T20:48:07.570704+0000","last_unstale":"2024-09-16T20:49:50.137069+0000","last_undegraded":"2024-09-16T20:49:50.137069+0000","last_fullsized":"2024-09-16T20:49:50.137069+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:28:15.863000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011026149999999999,"stat_sum":{"num_bytes":60378977,"num_objects":47,"num_object_clones":14,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":12,"num_read":5744,"num_read_kb":32785,"num_write":183,"num_write_kb":4318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"114'267","reported_seq":7864,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-16T20:49:50.137541+0000","last_change":"2024-09-16T20:48:09.570786+0000","last_active":"2024-09-16T20:49:50.137541+0000","last_peered":"2024-09-16T20:49:50.137541+0000","last_clean":"2024-09-16T20:49:50.137541+0000","last_became_active":"2024-09-16T20:48:07.560783+0000","last_became_peered":"2024-09-16T20:48:07.560783+0000","last_unstale":"2024-09-16T20:49:50.137541+0000","last_undegraded":"2024-09-16T20:49:50.137541+0000","last_fullsized":"2024-09-16T20:49:50.137541+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":267,"log_dups_size":0,"ondisk_log_size":267,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:41:56.407609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040623999999999999,"stat_sum":{"num_bytes":26457354,"num_objects":56,"num_object_clones":18,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":17,"num_read":8461,"num_read_kb":35905,"num_write":241,"num_write_kb":10069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"114'384","reported_seq":11022,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-16T20:49:51.227709+0000","last_change":"2024-09-16T20:49:12.801692+0000","last_active":"2024-09-16T20:49:51.227709+0000","last_peered":"2024-09-16T20:49:51.227709+0000","last_clean":"2024-09-16T20:49:51.227709+0000","last_became_active":"2024-09-16T20:49:12.801013+0000","last_became_peered":"2024-09-16T20:49:12.801013+0000","last_unstale":"2024-09-16T20:49:51.227709+0000","last_undegraded":"2024-09-16T20:49:51.227709+0000","last_fullsized":"2024-09-16T20:49:51.227709+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:02:32.371670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115576,"stat_sum":{"num_bytes":64014427,"num_objects":56,"num_object_clones":16,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":18,"num_read":12511,"num_read_kb":40830,"num_write":300,"num_write_kb":6978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":99,"num_bytes_recovered":5210742,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"114'234","reported_seq":9174,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-16T20:49:51.075173+0000","last_change":"2024-09-16T20:48:09.564513+0000","last_active":"2024-09-16T20:49:51.075173+0000","last_peered":"2024-09-16T20:49:51.075173+0000","last_clean":"2024-09-16T20:49:51.075173+0000","last_became_active":"2024-09-16T20:48:07.571985+0000","last_became_peered":"2024-09-16T20:48:07.571985+0000","last_unstale":"2024-09-16T20:49:51.075173+0000","last_undegraded":"2024-09-16T20:49:51.075173+0000","last_fullsized":"2024-09-16T20:49:51.075173+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":234,"log_dups_size":0,"ondisk_log_size":234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:11:51.807960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077335299999999995,"stat_sum":{"num_bytes":53990511,"num_objects":44,"num_object_clones":10,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":6,"num_read":9050,"num_read_kb":61422,"num_write":292,"num_write_kb":18975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"114'321","reported_seq":3738,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-16T20:49:50.173671+0000","last_change":"2024-09-16T20:49:12.804699+0000","last_active":"2024-09-16T20:49:50.173671+0000","last_peered":"2024-09-16T20:49:50.173671+0000","last_clean":"2024-09-16T20:49:50.173671+0000","last_became_active":"2024-09-16T20:49:12.803689+0000","last_became_peered":"2024-09-16T20:49:12.803689+0000","last_unstale":"2024-09-16T20:49:50.173671+0000","last_undegraded":"2024-09-16T20:49:50.173671+0000","last_fullsized":"2024-09-16T20:49:50.173671+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:32:52.819989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000840955,"stat_sum":{"num_bytes":20614735,"num_objects":38,"num_object_clones":9,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":10,"num_read":3813,"num_read_kb":13241,"num_write":408,"num_write_kb":2319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":107,"num_bytes_recovered":1536239,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"114'326","reported_seq":6325,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-16T20:49:50.526704+0000","last_change":"2024-09-16T20:48:09.565356+0000","last_active":"2024-09-16T20:49:50.526704+0000","last_peered":"2024-09-16T20:49:50.526704+0000","last_clean":"2024-09-16T20:49:50.526704+0000","last_became_active":"2024-09-16T20:48:07.565717+0000","last_became_peered":"2024-09-16T20:48:07.565717+0000","last_unstale":"2024-09-16T20:49:50.526704+0000","last_undegraded":"2024-09-16T20:49:50.526704+0000","last_fullsized":"2024-09-16T20:49:50.526704+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:45:01.674639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013583239999999999,"stat_sum":{"num_bytes":35154006,"num_objects":38,"num_object_clones":9,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":12,"num_read":6545,"num_read_kb":21325,"num_write":454,"num_write_kb":8834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1248724969,"num_objects":1456,"num_object_clones":388,"num_object_copies":2912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1456,"num_whiteouts":396,"num_read":273062,"num_read_kb":992456,"num_write":14471,"num_write_kb":263312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1615,"num_bytes_recovered":63790994,"num_keys_recovered":414,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":365531136,"data_stored":391182802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12164,"ondisk_log_size":12164,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574876,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61116,"kb_used_data":29924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94310724,"statfs":{"total":96636764160,"available":96574181376,"internally_reserved":0,"allocated":30642176,"data_stored":30228741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574876,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":119536,"kb_used_data":66144,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94252304,"statfs":{"total":96636764160,"available":96514359296,"internally_reserved":0,"allocated":67731456,"data_stored":70742028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574876,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":113096,"kb_used_data":81900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94258744,"statfs":{"total":96636764160,"available":96520953856,"internally_reserved":0,"allocated":83865600,"data_stored":89086594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123916,"kb_used_data":70524,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94247924,"statfs":{"total":96636764160,"available":96509874176,"internally_reserved":0,"allocated":72216576,"data_stored":76989347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,13,88],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574876,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":129328,"kb_used_data":71776,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94242512,"statfs":{"total":96636764160,"available":96504332288,"internally_reserved":0,"allocated":73498624,"data_stored":79397126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640284,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75360,"kb_used_data":44172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94296480,"statfs":{"total":96636764160,"available":96559595520,"internally_reserved":0,"allocated":45232128,"data_stored":47647555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72376320,"data_stored":79062922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":70283264,"data_stored":75855386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":83443712,"data_stored":89431833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":66539520,"data_stored":70338211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":43876352,"data_stored":47098614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":29011968,"data_stored":29395836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T20:49:55.276 INFO:tasks.thrashosds.thrasher:Setting 3.b to [5, 0, 2, 4] 2024-09-16T20:49:55.276 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.b', '5', '0', '2', '4'] 2024-09-16T20:49:55.276 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.b 5 0 2 4 2024-09-16T20:49:55.277 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:55.270+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:55.372 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:55.371+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:55.420 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:55.420 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:49:55.420 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:49:55.420 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:55.430 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:49:55.473 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:55.471+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:55.573 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:55.572+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:55.580 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:55.581 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:49:55.581 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:49:55.581 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:49:55.581 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:55.586 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:49:55.637 INFO:teuthology.orchestra.run.smithi029.stderr:noscrub is set 2024-09-16T20:49:55.637 INFO:teuthology.orchestra.run.smithi029.stderr:set 3.b pg_upmap_items mapping to [5->0,2->4] 2024-09-16T20:49:55.674 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:55.673+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:55.743 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:49:55.743 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:49:55.743 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:49:55.743 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:49:55.743 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.744 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3712 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1472512~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:55.744 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:49:55.744 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 93.102616662000003, 2024-09-16T20:49:55.744 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050004305999999998, 2024-09-16T20:49:55.744 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:55.744 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:55.744 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:55.744 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:55.744 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:55.744 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:55.744 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3712 2024-09-16T20:49:55.744 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.744 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:55.745 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.745 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:55.745 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:49:55.745 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.745 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.745 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.745 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:55.745 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:49:55.745 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.745 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.745 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.745 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:55.745 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638988+0000", 2024-09-16T20:49:55.746 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0199999999999998e-07 2024-09-16T20:49:55.746 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.746 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.746 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:55.746 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638990+0000", 2024-09-16T20:49:55.746 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.618e-06 2024-09-16T20:49:55.746 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.746 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.746 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:55.746 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.638995+0000", 2024-09-16T20:49:55.746 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3940000000000004e-06 2024-09-16T20:49:55.746 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.746 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.747 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:55.747 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639000+0000", 2024-09-16T20:49:55.747 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9309999999999999e-06 2024-09-16T20:49:55.747 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.747 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.747 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:55.747 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688867+0000", 2024-09-16T20:49:55.747 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049866955999999997 2024-09-16T20:49:55.747 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.747 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.747 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:55.747 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688933+0000", 2024-09-16T20:49:55.747 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5359999999999998e-05 2024-09-16T20:49:55.747 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.748 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.748 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:55.748 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.688992+0000", 2024-09-16T20:49:55.748 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9744999999999998e-05 2024-09-16T20:49:55.748 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.748 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:55.748 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.748 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.748 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.748 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3714 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1480704~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:55.748 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:49:55.748 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 93.102599072000004, 2024-09-16T20:49:55.748 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050152552000000003, 2024-09-16T20:49:55.749 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:55.749 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:55.749 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:55.749 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:55.749 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:55.749 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:55.749 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3714 2024-09-16T20:49:55.749 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.749 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:55.749 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.749 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:55.749 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:49:55.749 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.749 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.750 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.750 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:55.750 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:49:55.750 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.750 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.750 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.750 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:55.750 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:49:55.750 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0499999999999999e-07 2024-09-16T20:49:55.750 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.750 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.750 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:55.750 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639007+0000", 2024-09-16T20:49:55.751 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5340000000000001e-06 2024-09-16T20:49:55.751 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.751 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.751 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:55.751 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639013+0000", 2024-09-16T20:49:55.751 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3260000000000003e-06 2024-09-16T20:49:55.751 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.751 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.751 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:55.751 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639018+0000", 2024-09-16T20:49:55.751 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2850000000000002e-06 2024-09-16T20:49:55.751 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.751 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.751 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:55.752 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689026+0000", 2024-09-16T20:49:55.752 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.05000798 2024-09-16T20:49:55.752 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.752 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.752 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:55.752 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689077+0000", 2024-09-16T20:49:55.752 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1063999999999998e-05 2024-09-16T20:49:55.752 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.752 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.752 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:55.752 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689158+0000", 2024-09-16T20:49:55.752 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1057999999999995e-05 2024-09-16T20:49:55.752 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.752 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:55.753 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.753 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.753 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.753 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3716 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1488896~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:55.753 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:49:55.753 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 93.102581262000001, 2024-09-16T20:49:55.753 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050280142, 2024-09-16T20:49:55.753 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:55.753 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:55.753 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:55.753 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:55.753 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:55.753 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:55.753 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3716 2024-09-16T20:49:55.754 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.754 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:55.754 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.754 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:55.754 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:49:55.754 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.754 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.754 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.754 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:55.754 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:49:55.754 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.754 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.754 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.754 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:55.755 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639024+0000", 2024-09-16T20:49:55.755 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9999999999999997e-07 2024-09-16T20:49:55.755 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.755 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.755 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:55.755 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639026+0000", 2024-09-16T20:49:55.755 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.781e-06 2024-09-16T20:49:55.755 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.755 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.755 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:55.755 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639032+0000", 2024-09-16T20:49:55.755 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7780000000000004e-06 2024-09-16T20:49:55.755 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.755 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.756 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:55.756 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639036+0000", 2024-09-16T20:49:55.756 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6909999999999996e-06 2024-09-16T20:49:55.756 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.756 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.756 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:55.756 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689191+0000", 2024-09-16T20:49:55.756 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050155284000000001 2024-09-16T20:49:55.756 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.756 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.756 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:55.756 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689242+0000", 2024-09-16T20:49:55.756 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0662999999999999e-05 2024-09-16T20:49:55.756 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.757 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.757 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:55.757 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689304+0000", 2024-09-16T20:49:55.757 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1345000000000003e-05 2024-09-16T20:49:55.757 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.757 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:55.757 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.757 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.757 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.757 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3719 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1497088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:55.757 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:49:55.757 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 93.102553983999996, 2024-09-16T20:49:55.758 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050399715999999997, 2024-09-16T20:49:55.758 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:55.758 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:55.758 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:55.758 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:55.758 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:55.758 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:55.758 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3719 2024-09-16T20:49:55.758 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.758 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:55.758 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.758 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:55.758 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:49:55.759 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.759 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.759 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.759 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:55.759 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:49:55.759 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.759 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.759 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.759 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:55.759 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:49:55.759 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9700000000000002e-07 2024-09-16T20:49:55.759 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.759 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.760 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:55.760 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639053+0000", 2024-09-16T20:49:55.760 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2900000000000001e-06 2024-09-16T20:49:55.760 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.760 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.760 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:55.760 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639060+0000", 2024-09-16T20:49:55.760 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7750000000000004e-06 2024-09-16T20:49:55.760 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.760 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.760 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:55.760 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639066+0000", 2024-09-16T20:49:55.761 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0639999999999999e-06 2024-09-16T20:49:55.761 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.761 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.761 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:55.761 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689337+0000", 2024-09-16T20:49:55.761 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050270784999999998 2024-09-16T20:49:55.761 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.761 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.761 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:55.761 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689392+0000", 2024-09-16T20:49:55.761 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4920999999999998e-05 2024-09-16T20:49:55.761 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.761 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.761 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:55.762 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689450+0000", 2024-09-16T20:49:55.762 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8483999999999998e-05 2024-09-16T20:49:55.762 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.762 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:55.762 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.762 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.762 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.762 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3721 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1505280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:55.762 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:49:55.762 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 93.102530852000001, 2024-09-16T20:49:55.762 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050513608000000002, 2024-09-16T20:49:55.762 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:55.763 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:55.763 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:55.763 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:55.763 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:55.763 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:55.763 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3721 2024-09-16T20:49:55.763 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.763 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:55.763 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.763 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:55.763 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:49:55.763 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.764 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.764 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.764 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:55.764 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:49:55.764 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.764 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.764 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.764 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:55.764 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:49:55.764 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7e-07 2024-09-16T20:49:55.764 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.764 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.764 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:55.765 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639076+0000", 2024-09-16T20:49:55.765 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8899999999999999e-06 2024-09-16T20:49:55.765 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.765 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.765 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:55.765 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639083+0000", 2024-09-16T20:49:55.765 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1210000000000001e-06 2024-09-16T20:49:55.765 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.765 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.765 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:55.765 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639089+0000", 2024-09-16T20:49:55.765 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.643e-06 2024-09-16T20:49:55.765 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.766 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.766 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:55.766 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689474+0000", 2024-09-16T20:49:55.766 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050385679000000003 2024-09-16T20:49:55.766 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.766 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.766 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:55.766 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689533+0000", 2024-09-16T20:49:55.766 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8303000000000001e-05 2024-09-16T20:49:55.766 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.766 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.766 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:55.766 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689587+0000", 2024-09-16T20:49:55.767 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4601999999999997e-05 2024-09-16T20:49:55.767 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.767 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:55.767 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.767 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.767 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.767 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3723 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1513472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:55.767 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:49:55.767 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 93.102500047999996, 2024-09-16T20:49:55.767 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050699691999999998, 2024-09-16T20:49:55.767 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:55.767 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:55.767 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:55.768 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:55.768 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:55.768 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:55.768 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3723 2024-09-16T20:49:55.768 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.768 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:55.768 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.768 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:55.768 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:49:55.768 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.768 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.768 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.768 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:55.769 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:49:55.769 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.769 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.769 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.769 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:55.769 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:49:55.769 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3399999999999999e-07 2024-09-16T20:49:55.769 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.769 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.769 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:55.769 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639107+0000", 2024-09-16T20:49:55.769 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3860000000000001e-06 2024-09-16T20:49:55.769 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.770 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.770 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:55.770 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639114+0000", 2024-09-16T20:49:55.770 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9800000000000001e-06 2024-09-16T20:49:55.770 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.770 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.770 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:55.770 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639129+0000", 2024-09-16T20:49:55.770 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4887e-05 2024-09-16T20:49:55.770 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.770 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.770 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:55.770 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689625+0000", 2024-09-16T20:49:55.771 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050495848000000003 2024-09-16T20:49:55.771 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.771 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.771 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:55.771 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689720+0000", 2024-09-16T20:49:55.771 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4821000000000006e-05 2024-09-16T20:49:55.771 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.771 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.771 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:55.771 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689804+0000", 2024-09-16T20:49:55.771 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4235999999999998e-05 2024-09-16T20:49:55.771 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.771 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:55.771 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.772 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.772 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.772 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3725 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1521664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:55.772 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:49:55.772 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 93.102466452000002, 2024-09-16T20:49:55.772 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050797660000000001, 2024-09-16T20:49:55.772 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:55.772 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:55.772 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:55.772 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:55.772 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:55.772 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:55.773 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3725 2024-09-16T20:49:55.773 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.773 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:55.773 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.773 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:55.773 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:49:55.773 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.773 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.773 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.773 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:55.773 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:49:55.773 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.774 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.774 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.774 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:55.774 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639139+0000", 2024-09-16T20:49:55.774 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8999999999999997e-07 2024-09-16T20:49:55.774 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.774 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.774 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:55.774 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639141+0000", 2024-09-16T20:49:55.774 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.182e-06 2024-09-16T20:49:55.774 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.774 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.774 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:55.774 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639154+0000", 2024-09-16T20:49:55.775 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3626e-05 2024-09-16T20:49:55.775 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.775 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.775 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:55.775 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639160+0000", 2024-09-16T20:49:55.775 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6810000000000003e-06 2024-09-16T20:49:55.775 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.775 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.775 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:55.775 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689831+0000", 2024-09-16T20:49:55.775 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050670907000000001 2024-09-16T20:49:55.775 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.775 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.775 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:55.776 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689882+0000", 2024-09-16T20:49:55.776 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0445999999999998e-05 2024-09-16T20:49:55.776 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.776 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.776 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:55.776 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689936+0000", 2024-09-16T20:49:55.776 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4327999999999999e-05 2024-09-16T20:49:55.776 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.776 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:55.776 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.776 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.776 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.776 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3727 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1529856~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:55.777 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:49:55.777 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 93.102430874999996, 2024-09-16T20:49:55.777 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051016922999999999, 2024-09-16T20:49:55.777 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:55.777 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:55.777 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:55.777 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:55.777 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:55.777 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:55.777 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3727 2024-09-16T20:49:55.777 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.777 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:55.777 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.778 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:55.778 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:49:55.778 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.778 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.778 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.778 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:55.778 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:49:55.778 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.778 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.778 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.778 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:55.778 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:49:55.778 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9999999999999998e-07 2024-09-16T20:49:55.778 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.779 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.779 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:55.779 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639183+0000", 2024-09-16T20:49:55.779 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2399999999999996e-06 2024-09-16T20:49:55.779 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.779 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.779 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:55.779 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639195+0000", 2024-09-16T20:49:55.779 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1113e-05 2024-09-16T20:49:55.779 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.779 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.779 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:55.779 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639206+0000", 2024-09-16T20:49:55.779 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1503e-05 2024-09-16T20:49:55.780 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.780 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.780 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:55.780 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689969+0000", 2024-09-16T20:49:55.780 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050762676 2024-09-16T20:49:55.780 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.780 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.780 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:55.780 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690071+0000", 2024-09-16T20:49:55.780 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102163 2024-09-16T20:49:55.780 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.780 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.780 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:55.781 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690191+0000", 2024-09-16T20:49:55.781 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000119728 2024-09-16T20:49:55.781 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.781 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:55.781 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.781 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.781 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.781 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3730 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1538048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:55.781 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:49:55.781 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 93.102389312, 2024-09-16T20:49:55.781 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051131467999999999, 2024-09-16T20:49:55.781 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:55.781 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:55.781 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:55.782 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:55.782 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:55.782 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:55.782 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3730 2024-09-16T20:49:55.782 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.782 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:55.782 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.782 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:55.782 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:49:55.782 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.782 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.782 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.782 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:55.782 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:49:55.783 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.783 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.783 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.783 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:55.783 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639216+0000", 2024-09-16T20:49:55.783 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.39e-07 2024-09-16T20:49:55.783 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.783 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.783 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:55.783 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639218+0000", 2024-09-16T20:49:55.783 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.537e-06 2024-09-16T20:49:55.783 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.783 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.783 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:55.784 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639229+0000", 2024-09-16T20:49:55.784 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1122e-05 2024-09-16T20:49:55.784 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.784 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.784 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:55.784 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639237+0000", 2024-09-16T20:49:55.784 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8509999999999999e-06 2024-09-16T20:49:55.784 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.784 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.784 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:55.784 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690227+0000", 2024-09-16T20:49:55.784 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050990147 2024-09-16T20:49:55.785 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.785 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.785 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:55.785 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690278+0000", 2024-09-16T20:49:55.785 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0223999999999999e-05 2024-09-16T20:49:55.785 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.785 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.785 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:55.785 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690347+0000", 2024-09-16T20:49:55.785 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9147999999999996e-05 2024-09-16T20:49:55.785 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.785 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:55.785 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.785 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.786 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.786 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3731 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1546240~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:55.786 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:49:55.786 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 93.102358604000003, 2024-09-16T20:49:55.786 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051276744999999999, 2024-09-16T20:49:55.786 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:55.786 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:55.786 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:55.786 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:55.786 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:55.786 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:55.786 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3731 2024-09-16T20:49:55.786 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.786 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:55.786 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.787 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:55.787 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:49:55.787 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.787 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.787 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.787 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:55.787 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:49:55.787 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.787 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.787 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.787 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:55.787 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:49:55.787 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9900000000000001e-07 2024-09-16T20:49:55.787 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.788 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.788 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:55.788 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639249+0000", 2024-09-16T20:49:55.788 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3630000000000001e-06 2024-09-16T20:49:55.788 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.788 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.788 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:55.788 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639255+0000", 2024-09-16T20:49:55.788 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4960000000000001e-06 2024-09-16T20:49:55.788 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.788 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.788 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:55.788 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639261+0000", 2024-09-16T20:49:55.789 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2839999999999997e-06 2024-09-16T20:49:55.789 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.789 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.789 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:55.789 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690375+0000", 2024-09-16T20:49:55.789 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051114684000000001 2024-09-16T20:49:55.789 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.789 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.789 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:55.789 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690425+0000", 2024-09-16T20:49:55.789 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9379999999999998e-05 2024-09-16T20:49:55.789 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.789 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.789 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:55.790 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690523+0000", 2024-09-16T20:49:55.790 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8139000000000003e-05 2024-09-16T20:49:55.790 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.790 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:55.790 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.790 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.790 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.790 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3732 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1554432~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:55.790 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:49:55.790 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 93.102312822000002, 2024-09-16T20:49:55.790 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051380096, 2024-09-16T20:49:55.790 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:55.790 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:55.791 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:55.791 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:55.791 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:55.791 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:55.791 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3732 2024-09-16T20:49:55.791 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.791 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:55.791 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.791 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:55.791 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:49:55.791 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.791 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.791 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.791 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:55.791 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:49:55.792 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.792 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.792 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.792 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:55.792 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:49:55.792 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6100000000000001e-07 2024-09-16T20:49:55.792 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.792 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.792 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:55.792 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639295+0000", 2024-09-16T20:49:55.792 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.447e-06 2024-09-16T20:49:55.792 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.792 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.793 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:55.793 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639302+0000", 2024-09-16T20:49:55.793 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0570000000000004e-06 2024-09-16T20:49:55.793 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.793 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.793 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:55.793 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639307+0000", 2024-09-16T20:49:55.793 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1429999999999997e-06 2024-09-16T20:49:55.793 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.793 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.793 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:55.793 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690555+0000", 2024-09-16T20:49:55.793 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051247963000000001 2024-09-16T20:49:55.793 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.793 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.794 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:55.794 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690602+0000", 2024-09-16T20:49:55.794 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7602000000000003e-05 2024-09-16T20:49:55.794 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.794 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.794 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:55.794 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690672+0000", 2024-09-16T20:49:55.794 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9423000000000003e-05 2024-09-16T20:49:55.794 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.794 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:55.794 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.794 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.794 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.794 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3734 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1562624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:55.795 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:49:55.795 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 93.102291171999994, 2024-09-16T20:49:55.795 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051490610999999999, 2024-09-16T20:49:55.795 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:55.795 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:55.795 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:55.795 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:55.795 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:55.795 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:55.795 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3734 2024-09-16T20:49:55.795 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.795 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:55.795 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.795 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:55.796 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:49:55.796 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.796 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.796 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.796 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:55.796 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:49:55.796 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.796 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.796 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.796 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:55.796 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639314+0000", 2024-09-16T20:49:55.796 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4400000000000001e-07 2024-09-16T20:49:55.796 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.797 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.797 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:55.797 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639316+0000", 2024-09-16T20:49:55.797 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1459999999999999e-06 2024-09-16T20:49:55.797 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.797 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.797 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:55.797 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639322+0000", 2024-09-16T20:49:55.797 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7610000000000004e-06 2024-09-16T20:49:55.797 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.797 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.797 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:55.797 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639327+0000", 2024-09-16T20:49:55.798 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.186e-06 2024-09-16T20:49:55.798 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.798 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.798 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:55.798 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690698+0000", 2024-09-16T20:49:55.798 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051371223000000001 2024-09-16T20:49:55.798 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.798 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.798 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:55.798 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690748+0000", 2024-09-16T20:49:55.798 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0192999999999999e-05 2024-09-16T20:49:55.798 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.798 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.798 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:55.799 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690804+0000", 2024-09-16T20:49:55.799 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5658000000000001e-05 2024-09-16T20:49:55.799 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.799 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:55.799 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.799 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.799 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.799 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3736 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1570816~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:55.799 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:49:55.799 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 93.102271912999996, 2024-09-16T20:49:55.799 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051596528000000003, 2024-09-16T20:49:55.799 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:55.799 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:55.800 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:55.800 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:55.800 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:55.800 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:55.800 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3736 2024-09-16T20:49:55.800 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.800 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:55.800 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.800 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:55.800 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:49:55.800 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.800 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.800 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.801 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:55.801 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:49:55.801 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.801 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.801 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.801 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:55.801 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:49:55.801 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9999999999999999e-07 2024-09-16T20:49:55.801 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.801 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.801 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:55.801 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639335+0000", 2024-09-16T20:49:55.801 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.635e-06 2024-09-16T20:49:55.802 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.802 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.802 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:55.802 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639340+0000", 2024-09-16T20:49:55.802 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5840000000000002e-06 2024-09-16T20:49:55.802 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.802 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.802 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:55.802 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639346+0000", 2024-09-16T20:49:55.802 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1190000000000002e-06 2024-09-16T20:49:55.802 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.802 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.802 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:55.802 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690817+0000", 2024-09-16T20:49:55.802 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051470721999999997 2024-09-16T20:49:55.803 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.803 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.803 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:55.803 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690875+0000", 2024-09-16T20:49:55.803 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7441000000000001e-05 2024-09-16T20:49:55.803 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.803 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.803 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:55.803 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690929+0000", 2024-09-16T20:49:55.803 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4727e-05 2024-09-16T20:49:55.803 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.803 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:55.803 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.803 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.804 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.804 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5434 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:55.804 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:49:55.804 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 92.877010346999995, 2024-09-16T20:49:55.804 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099755831000000003, 2024-09-16T20:49:55.804 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:55.804 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:55.804 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:55.804 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:55.804 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:55.804 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:55.804 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5434 2024-09-16T20:49:55.804 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.805 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:55.805 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.805 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:55.805 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:49:55.805 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.805 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.805 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.805 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:55.805 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:49:55.805 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.805 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.805 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.805 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:55.806 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864595+0000", 2024-09-16T20:49:55.806 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1099999999999995e-07 2024-09-16T20:49:55.806 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.806 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.806 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:55.806 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864597+0000", 2024-09-16T20:49:55.806 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3989999999999998e-06 2024-09-16T20:49:55.806 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.806 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.806 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:55.806 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864603+0000", 2024-09-16T20:49:55.806 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9309999999999996e-06 2024-09-16T20:49:55.806 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.806 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.807 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:55.807 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864607+0000", 2024-09-16T20:49:55.807 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6200000000000001e-06 2024-09-16T20:49:55.807 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.807 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.807 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:55.807 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964164+0000", 2024-09-16T20:49:55.807 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099557162000000005 2024-09-16T20:49:55.807 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.807 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.807 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:55.807 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964244+0000", 2024-09-16T20:49:55.807 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9597000000000004e-05 2024-09-16T20:49:55.808 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.808 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.808 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:55.808 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964350+0000", 2024-09-16T20:49:55.808 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010641099999999999 2024-09-16T20:49:55.808 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.808 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:55.808 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.808 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.808 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.808 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5435 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 20480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:49:55.808 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:49:55.808 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 92.876980524999993, 2024-09-16T20:49:55.809 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100897102, 2024-09-16T20:49:55.809 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:55.809 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:55.809 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:55.809 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:55.809 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:49:55.809 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:49:55.809 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5435 2024-09-16T20:49:55.809 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.809 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:55.809 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.809 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:55.809 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:49:55.809 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.810 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.810 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.810 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:55.810 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:49:55.810 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.810 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.810 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.810 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:55.810 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:49:55.810 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8000000000000001e-07 2024-09-16T20:49:55.810 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.810 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.810 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:55.810 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864626+0000", 2024-09-16T20:49:55.811 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.745e-06 2024-09-16T20:49:55.811 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.811 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.811 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:55.811 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864633+0000", 2024-09-16T20:49:55.811 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3339999999999998e-06 2024-09-16T20:49:55.811 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.811 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.811 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:55.811 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864636+0000", 2024-09-16T20:49:55.811 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1200000000000002e-06 2024-09-16T20:49:55.811 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.811 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.811 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:55.811 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965299+0000", 2024-09-16T20:49:55.812 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10066352000000001 2024-09-16T20:49:55.812 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.812 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.812 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:55.812 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965390+0000", 2024-09-16T20:49:55.812 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0894000000000001e-05 2024-09-16T20:49:55.812 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.812 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.812 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:55.812 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965521+0000", 2024-09-16T20:49:55.812 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013110899999999999 2024-09-16T20:49:55.812 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.812 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:55.812 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.813 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.813 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.813 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:152 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 327680~32768 out=32768b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:49:55.813 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:49:55.813 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 91.578483582999993, 2024-09-16T20:49:55.813 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11869405800000001, 2024-09-16T20:49:55.813 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:55.813 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:55.813 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:55.813 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:55.813 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:49:55.813 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:49:55.814 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 152 2024-09-16T20:49:55.814 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.814 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:55.814 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.814 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:55.814 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:49:55.814 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.814 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.814 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.814 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:55.814 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:49:55.814 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.814 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.814 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.814 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:55.815 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163122+0000", 2024-09-16T20:49:55.815 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6400000000000003e-07 2024-09-16T20:49:55.815 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.815 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.815 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:55.815 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163124+0000", 2024-09-16T20:49:55.815 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.565e-06 2024-09-16T20:49:55.815 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.815 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.815 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:55.815 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163136+0000", 2024-09-16T20:49:55.815 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1717e-05 2024-09-16T20:49:55.815 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.815 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.816 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:55.816 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163142+0000", 2024-09-16T20:49:55.816 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1750000000000002e-06 2024-09-16T20:49:55.816 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.816 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.816 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:55.816 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163606+0000", 2024-09-16T20:49:55.816 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00046418799999999998 2024-09-16T20:49:55.816 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.816 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.816 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:55.816 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163656+0000", 2024-09-16T20:49:55.816 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9852999999999999e-05 2024-09-16T20:49:55.817 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.817 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.817 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:55.817 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281815+0000", 2024-09-16T20:49:55.817 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11815909600000001 2024-09-16T20:49:55.817 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.817 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:55.817 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.817 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.817 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.817 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:153 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:49:55.817 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:49:55.817 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 91.578450630000006, 2024-09-16T20:49:55.817 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11903462400000001, 2024-09-16T20:49:55.818 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:55.818 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:55.818 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:55.818 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:55.818 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:49:55.818 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:49:55.818 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 153 2024-09-16T20:49:55.818 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.818 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:55.818 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.818 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:55.818 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:49:55.818 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.818 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.818 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.819 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:55.819 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:49:55.819 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.819 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.819 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.819 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:55.819 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163155+0000", 2024-09-16T20:49:55.819 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.37e-07 2024-09-16T20:49:55.819 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.819 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.819 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:55.819 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163157+0000", 2024-09-16T20:49:55.819 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6010000000000002e-06 2024-09-16T20:49:55.819 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.819 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.820 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:55.820 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163165+0000", 2024-09-16T20:49:55.820 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2619999999999998e-06 2024-09-16T20:49:55.820 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.820 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.820 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:55.820 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163171+0000", 2024-09-16T20:49:55.820 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.711e-06 2024-09-16T20:49:55.820 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.820 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.820 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:55.820 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281868+0000", 2024-09-16T20:49:55.820 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118696622 2024-09-16T20:49:55.821 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.821 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.821 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:55.821 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281969+0000", 2024-09-16T20:49:55.821 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101371 2024-09-16T20:49:55.821 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.821 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.821 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:55.821 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282189+0000", 2024-09-16T20:49:55.821 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00021952000000000001 2024-09-16T20:49:55.821 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.821 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:55.821 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.821 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.822 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.822 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:154 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 360448~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:49:55.822 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:49:55.822 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 91.578418858000006, 2024-09-16T20:49:55.822 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11932035100000001, 2024-09-16T20:49:55.822 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:55.822 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:55.822 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:55.822 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:55.822 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:49:55.822 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:49:55.822 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 154 2024-09-16T20:49:55.822 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.822 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:55.823 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.823 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:55.823 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:49:55.823 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.823 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.823 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.823 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:55.823 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:49:55.823 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.823 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.823 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.823 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:55.823 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:49:55.823 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:49:55.824 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.824 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.824 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:55.824 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163189+0000", 2024-09-16T20:49:55.824 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6840000000000001e-06 2024-09-16T20:49:55.824 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.824 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.824 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:55.824 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163196+0000", 2024-09-16T20:49:55.824 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7920000000000004e-06 2024-09-16T20:49:55.824 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.824 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.825 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:55.825 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163201+0000", 2024-09-16T20:49:55.825 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2809999999999999e-06 2024-09-16T20:49:55.825 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.825 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.825 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:55.825 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282219+0000", 2024-09-16T20:49:55.825 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.119018343 2024-09-16T20:49:55.825 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.825 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.825 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:55.825 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282305+0000", 2024-09-16T20:49:55.825 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5865000000000005e-05 2024-09-16T20:49:55.826 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.826 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.826 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:55.826 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282506+0000", 2024-09-16T20:49:55.826 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000200927 2024-09-16T20:49:55.826 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.826 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:55.826 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.826 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.826 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.826 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop(client.4366.0:3088 3.1e e38/23 3:79e5ec01:::rbd_header.110e891a9b2b:head v 38'306, mlcod=36'304)", 2024-09-16T20:49:55.826 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:49:55.826 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 91.577399990000004, 2024-09-16T20:49:55.826 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118102759, 2024-09-16T20:49:55.827 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:55.827 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:55.827 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:55.827 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:55.827 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.827 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:55.827 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:49:55.827 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.827 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.827 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.827 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:55.827 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:49:55.827 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.827 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.827 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.827 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:55.828 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:49:55.828 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.85e-07 2024-09-16T20:49:55.828 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.828 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.828 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:55.828 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164212+0000", 2024-09-16T20:49:55.828 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3999999999999997e-06 2024-09-16T20:49:55.828 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.828 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.828 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:55.828 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281553+0000", 2024-09-16T20:49:55.828 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11734135499999999 2024-09-16T20:49:55.828 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.828 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.829 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:55.829 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281567+0000", 2024-09-16T20:49:55.829 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4188999999999999e-05 2024-09-16T20:49:55.829 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.829 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.829 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:55.829 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281653+0000", 2024-09-16T20:49:55.829 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5385999999999999e-05 2024-09-16T20:49:55.829 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.829 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.829 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:55.829 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281679+0000", 2024-09-16T20:49:55.830 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6689000000000001e-05 2024-09-16T20:49:55.830 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.830 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.830 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:49:55.830 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282277+0000", 2024-09-16T20:49:55.830 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00059798000000000002 2024-09-16T20:49:55.830 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.830 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.830 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:55.830 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282307+0000", 2024-09-16T20:49:55.830 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9975e-05 2024-09-16T20:49:55.830 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.830 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:55.830 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.831 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.831 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.831 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5255.0:44 3.16 3:6f5476df:::rbd_header.110e25db1f60:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:49:55.831 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:49:55.831 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 19.421096935000001, 2024-09-16T20:49:55.831 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.121007509, 2024-09-16T20:49:55.831 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:49:55.831 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:49:55.831 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:49:55.831 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:49:55.831 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5255", 2024-09-16T20:49:55.831 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:49:55.831 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 44 2024-09-16T20:49:55.831 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.832 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:49:55.832 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.832 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:49:55.832 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:49:55.832 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.832 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.832 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.832 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:49:55.832 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:49:55.832 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:49:55.832 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.832 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.832 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:49:55.833 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:49:55.833 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0500000000000003e-07 2024-09-16T20:49:55.833 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.833 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.833 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:49:55.833 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320512+0000", 2024-09-16T20:49:55.833 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2540000000000001e-06 2024-09-16T20:49:55.833 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.833 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.833 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:49:55.833 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320518+0000", 2024-09-16T20:49:55.833 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1689999999999997e-06 2024-09-16T20:49:55.833 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.834 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.834 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:49:55.834 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320521+0000", 2024-09-16T20:49:55.834 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5200000000000002e-06 2024-09-16T20:49:55.834 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.834 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.834 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:49:55.834 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320548+0000", 2024-09-16T20:49:55.834 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6662999999999999e-05 2024-09-16T20:49:55.834 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.834 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.834 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:49:55.834 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320587+0000", 2024-09-16T20:49:55.834 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8866999999999999e-05 2024-09-16T20:49:55.834 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:49:55.835 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:49:55.835 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:49:55.835 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.441515+0000", 2024-09-16T20:49:55.835 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.120928331 2024-09-16T20:49:55.835 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.835 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:55.835 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.835 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:49:55.835 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:49:55.835 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:49:55.837 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:49:55.837 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:55.835+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:55.937 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:55.936+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:55.959 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:55.959 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:49:55.959 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:55.959 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4366.0:19014 3.7 e116/61 3:e63cedc9:::rbd_header.110eaf5b6fa7:head v 116'392, mlcod=115'390)", 2024-09-16T20:49:55.959 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:55.957865+0000", 2024-09-16T20:49:55.959 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.00063733400000000003, 2024-09-16T20:49:55.959 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00064562300000000001, 2024-09-16T20:49:55.959 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:55.959 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:55.959 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:55.960 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:55.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:55.960 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:55.960 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:55.957865+0000", 2024-09-16T20:49:55.960 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:55.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:55.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:55.960 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:55.960 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:55.957865+0000", 2024-09-16T20:49:55.960 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:55.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:55.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:55.961 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:55.961 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:55.957865+0000", 2024-09-16T20:49:55.961 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0299999999999999e-07 2024-09-16T20:49:55.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:55.961 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:55.961 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:55.961 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:55.957871+0000", 2024-09-16T20:49:55.961 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2000000000000002e-06 2024-09-16T20:49:55.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:55.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:55.962 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:55.962 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:55.957880+0000", 2024-09-16T20:49:55.962 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3880000000000006e-06 2024-09-16T20:49:55.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:55.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:55.962 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:55.962 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:55.957885+0000", 2024-09-16T20:49:55.962 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6580000000000001e-06 2024-09-16T20:49:55.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:55.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:55.962 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:55.963 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:55.957932+0000", 2024-09-16T20:49:55.963 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7822999999999999e-05 2024-09-16T20:49:55.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:55.963 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:55.963 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:55.963 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:55.957958+0000", 2024-09-16T20:49:55.963 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5862000000000001e-05 2024-09-16T20:49:55.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:55.963 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:55.963 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:49:55.963 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:55.958477+0000", 2024-09-16T20:49:55.963 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00051834899999999998 2024-09-16T20:49:55.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:55.964 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:55.964 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:55.964 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:55.958510+0000", 2024-09-16T20:49:55.964 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3739999999999999e-05 2024-09-16T20:49:55.964 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:55.964 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:55.964 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:55.964 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:55.964 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-16T20:49:55.964 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 1 2024-09-16T20:49:55.964 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:55.968 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:49:56.036 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #33, image image_client.0-clone33 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent34 2024-09-16T20:49:56.038 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:56.037+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:56.091 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:56.091 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:49:56.092 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:49:56.092 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:49:56.092 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:56.103 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:49:56.139 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:56.138+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:56.239 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:56.238+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:56.282 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:56.282 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:49:56.282 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:49:56.283 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:49:56.283 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.283 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:3088 3.1e 3:79e5ec01:::rbd_header.110e891a9b2b:head [watch watch cookie 140213437171488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:49:56.283 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:49:56.283 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 92.116596212000005, 2024-09-16T20:49:56.283 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.11895186200000001, 2024-09-16T20:49:56.283 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:56.283 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:56.283 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:56.283 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:56.283 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:56.283 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:56.283 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3088 2024-09-16T20:49:56.283 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.284 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:56.284 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.284 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:56.284 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:49:56.284 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:56.284 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.284 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.284 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:56.284 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:49:56.284 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:56.284 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.284 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.284 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:56.284 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163846+0000", 2024-09-16T20:49:56.285 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6799999999999999e-07 2024-09-16T20:49:56.285 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.285 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.285 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:56.285 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163850+0000", 2024-09-16T20:49:56.285 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.929e-06 2024-09-16T20:49:56.285 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.285 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.285 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:56.285 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163862+0000", 2024-09-16T20:49:56.285 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1972e-05 2024-09-16T20:49:56.285 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.286 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.286 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:56.286 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163867+0000", 2024-09-16T20:49:56.286 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.451e-06 2024-09-16T20:49:56.286 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.286 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.286 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:56.286 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163912+0000", 2024-09-16T20:49:56.286 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4456000000000003e-05 2024-09-16T20:49:56.286 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.286 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.286 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:56.286 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163962+0000", 2024-09-16T20:49:56.286 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0578000000000003e-05 2024-09-16T20:49:56.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.287 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.287 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-16T20:49:56.287 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164058+0000", 2024-09-16T20:49:56.287 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5934999999999995e-05 2024-09-16T20:49:56.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.287 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.287 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:49:56.287 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164570+0000", 2024-09-16T20:49:56.287 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00051241100000000005 2024-09-16T20:49:56.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.287 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.287 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:56.287 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282629+0000", 2024-09-16T20:49:56.288 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.118058986 2024-09-16T20:49:56.288 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.288 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.288 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:49:56.288 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282725+0000", 2024-09-16T20:49:56.288 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5063999999999996e-05 2024-09-16T20:49:56.288 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.288 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.288 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:56.288 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282797+0000", 2024-09-16T20:49:56.288 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2212000000000006e-05 2024-09-16T20:49:56.288 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:56.288 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:56.288 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:56.288 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.289 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.289 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:8039 3.1e 3.ac44ff9e (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:49:56.289 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:49:56.289 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 66.572768862999993, 2024-09-16T20:49:56.289 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0053310449999999, 2024-09-16T20:49:56.289 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:56.289 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:56.289 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:49:56.289 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:56.289 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:56.289 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:56.289 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8039 2024-09-16T20:49:56.289 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.290 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:56.290 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.290 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:56.290 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:49:56.290 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:56.290 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.290 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.290 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:56.290 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:49:56.290 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:56.290 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.290 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.290 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:56.290 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707674+0000", 2024-09-16T20:49:56.290 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6530000000000001e-06 2024-09-16T20:49:56.290 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.291 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.291 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:56.291 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707681+0000", 2024-09-16T20:49:56.291 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9639999999999997e-06 2024-09-16T20:49:56.291 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.291 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.291 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:56.291 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707700+0000", 2024-09-16T20:49:56.291 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8674000000000001e-05 2024-09-16T20:49:56.291 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.291 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.291 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:56.291 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707708+0000", 2024-09-16T20:49:56.291 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6810000000000004e-06 2024-09-16T20:49:56.291 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.292 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.292 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:56.292 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711293+0000", 2024-09-16T20:49:56.292 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035848189999999999 2024-09-16T20:49:56.292 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.292 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.292 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:49:56.292 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711301+0000", 2024-09-16T20:49:56.292 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4730000000000003e-06 2024-09-16T20:49:56.292 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.292 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.292 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:56.292 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.712990+0000", 2024-09-16T20:49:56.292 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.001689348 2024-09-16T20:49:56.292 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.293 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.293 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:56.293 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713003+0000", 2024-09-16T20:49:56.293 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3433e-05 2024-09-16T20:49:56.293 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:56.293 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:56.293 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:56.293 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.293 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.293 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:8040 3.1e 3.ac44ff9e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:49:56.293 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:49:56.293 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 66.572708468000002, 2024-09-16T20:49:56.293 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0053410780000001, 2024-09-16T20:49:56.293 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:56.294 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:56.294 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:49:56.294 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:56.294 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:56.294 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:56.294 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8040 2024-09-16T20:49:56.294 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.294 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:56.294 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.294 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:56.294 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:49:56.294 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:56.294 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.294 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.294 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:56.295 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:49:56.295 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:56.295 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.295 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.295 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:56.295 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:49:56.295 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8100000000000003e-07 2024-09-16T20:49:56.295 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.295 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.295 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:56.295 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707737+0000", 2024-09-16T20:49:56.295 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5020000000000001e-06 2024-09-16T20:49:56.295 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.295 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.296 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:56.296 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707745+0000", 2024-09-16T20:49:56.296 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2169999999999994e-06 2024-09-16T20:49:56.296 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.296 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.296 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:56.296 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707751+0000", 2024-09-16T20:49:56.296 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.263e-06 2024-09-16T20:49:56.296 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.296 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.296 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:56.296 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711336+0000", 2024-09-16T20:49:56.296 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035848360000000001 2024-09-16T20:49:56.296 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.296 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.297 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:49:56.297 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711340+0000", 2024-09-16T20:49:56.297 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1309999999999999e-06 2024-09-16T20:49:56.297 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.297 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.297 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:56.297 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713060+0000", 2024-09-16T20:49:56.297 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0017195059999999 2024-09-16T20:49:56.297 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.297 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.297 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:56.297 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713074+0000", 2024-09-16T20:49:56.298 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4042e-05 2024-09-16T20:49:56.298 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:56.298 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:56.298 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:56.298 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.298 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.298 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4927.0:4487 3.1e 3.ac44ff9e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:49:56.298 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:49:56.298 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 66.572188265999998, 2024-09-16T20:49:56.298 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0048840880000001, 2024-09-16T20:49:56.298 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:56.298 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:56.298 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:49:56.299 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:56.299 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4927", 2024-09-16T20:49:56.299 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:56.299 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4487 2024-09-16T20:49:56.299 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.299 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:56.299 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.299 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:56.299 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:49:56.299 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:56.299 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.299 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.299 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:56.300 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:49:56.300 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:56.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.300 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.300 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:56.300 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708254+0000", 2024-09-16T20:49:56.300 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1800000000000005e-07 2024-09-16T20:49:56.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.300 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.300 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:56.300 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708258+0000", 2024-09-16T20:49:56.300 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8689999999999997e-06 2024-09-16T20:49:56.301 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.301 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.301 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:56.301 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708267+0000", 2024-09-16T20:49:56.301 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9590000000000007e-06 2024-09-16T20:49:56.301 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.301 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.301 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:56.301 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708274+0000", 2024-09-16T20:49:56.301 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9970000000000001e-06 2024-09-16T20:49:56.301 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.301 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.301 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:56.301 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711366+0000", 2024-09-16T20:49:56.302 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030919530000000001 2024-09-16T20:49:56.302 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.302 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.302 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:49:56.302 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711372+0000", 2024-09-16T20:49:56.302 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9410000000000004e-06 2024-09-16T20:49:56.302 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.302 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.302 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:56.302 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713126+0000", 2024-09-16T20:49:56.302 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0017537249999999 2024-09-16T20:49:56.302 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.302 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.303 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:56.303 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713137+0000", 2024-09-16T20:49:56.303 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0825999999999999e-05 2024-09-16T20:49:56.303 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:56.303 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:56.303 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:56.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.303 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.303 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5089.0:11703 3.1b 3:dcd4684e:::rbd_data.110e747f0779.000000000000001b:head [set-alloc-hint object_size 262144 write_size 262144,write 139264~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:49:56.303 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:49:56.303 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 52.142452491999997, 2024-09-16T20:49:56.303 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.084231385000000006, 2024-09-16T20:49:56.303 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:56.304 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:56.304 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:56.304 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:56.304 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5089", 2024-09-16T20:49:56.304 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:49:56.304 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 11703 2024-09-16T20:49:56.304 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.304 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:56.304 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.304 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:56.304 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:49:56.304 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:56.304 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.305 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:56.305 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:49:56.305 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:56.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.305 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:56.305 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:49:56.305 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2600000000000001e-07 2024-09-16T20:49:56.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.305 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:56.305 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137998+0000", 2024-09-16T20:49:56.306 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9549999999999998e-06 2024-09-16T20:49:56.306 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.306 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.306 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:56.306 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138008+0000", 2024-09-16T20:49:56.306 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3789999999999995e-06 2024-09-16T20:49:56.306 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.306 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.306 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:56.306 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138013+0000", 2024-09-16T20:49:56.306 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2209999999999996e-06 2024-09-16T20:49:56.306 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.306 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.306 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:56.307 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138054+0000", 2024-09-16T20:49:56.307 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1022999999999997e-05 2024-09-16T20:49:56.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.307 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.307 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:56.307 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138124+0000", 2024-09-16T20:49:56.307 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0234999999999994e-05 2024-09-16T20:49:56.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.307 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.307 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:56.307 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138197+0000", 2024-09-16T20:49:56.307 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2451000000000001e-05 2024-09-16T20:49:56.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.308 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.308 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:56.308 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222010+0000", 2024-09-16T20:49:56.308 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.083813433000000007 2024-09-16T20:49:56.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.308 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.308 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:49:56.308 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222163+0000", 2024-09-16T20:49:56.308 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015274599999999999 2024-09-16T20:49:56.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.308 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.308 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:49:56.308 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222197+0000", 2024-09-16T20:49:56.308 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3683000000000003e-05 2024-09-16T20:49:56.309 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.309 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.309 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:56.309 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222220+0000", 2024-09-16T20:49:56.309 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3433000000000001e-05 2024-09-16T20:49:56.309 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:56.309 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:56.309 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:56.309 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.309 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.309 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.5089.0:11702 3.15 e73/64 3:acf33c8b:::rbd_data.110e747f0779.000000000000001a:head v 73'132, mlcod=73'130)", 2024-09-16T20:49:56.309 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:49:56.309 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 52.142185163000001, 2024-09-16T20:49:56.309 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.084079763000000002, 2024-09-16T20:49:56.310 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:56.310 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:56.310 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:56.310 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:56.310 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.310 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:56.310 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:49:56.310 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:56.310 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.310 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.310 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:56.310 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:49:56.310 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:56.310 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.310 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.311 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:56.311 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:49:56.311 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:49:56.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.311 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.311 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:56.311 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138263+0000", 2024-09-16T20:49:56.311 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1e-06 2024-09-16T20:49:56.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.311 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.311 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:56.311 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221589+0000", 2024-09-16T20:49:56.311 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.083326002999999996 2024-09-16T20:49:56.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.312 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.312 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:56.312 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221614+0000", 2024-09-16T20:49:56.312 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5593999999999999e-05 2024-09-16T20:49:56.312 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.312 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.312 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:56.312 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221714+0000", 2024-09-16T20:49:56.312 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9889000000000005e-05 2024-09-16T20:49:56.312 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.312 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.312 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:56.312 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221727+0000", 2024-09-16T20:49:56.313 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3125999999999999e-05 2024-09-16T20:49:56.313 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.313 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.313 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:49:56.313 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222294+0000", 2024-09-16T20:49:56.313 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00056712199999999998 2024-09-16T20:49:56.313 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.313 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.313 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:56.313 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222336+0000", 2024-09-16T20:49:56.313 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1470000000000001e-05 2024-09-16T20:49:56.313 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:56.313 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:56.313 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:56.314 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.314 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.314 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5138.0:9358 3.c 3:33996a43:::rbd_data.110e6372565d.0000000000000007:head [read 1343488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T20:49:56.314 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:49:56.314 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 28.427909753000002, 2024-09-16T20:49:56.314 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0167493569999999, 2024-09-16T20:49:56.314 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:56.314 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:56.314 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:56.314 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:56.314 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5138", 2024-09-16T20:49:56.314 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:49:56.314 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9358 2024-09-16T20:49:56.315 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.315 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:56.315 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.315 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:56.315 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:49:56.315 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:56.315 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.315 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.315 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:56.315 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:49:56.315 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:56.315 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.315 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.315 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:56.315 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852533+0000", 2024-09-16T20:49:56.315 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.646e-06 2024-09-16T20:49:56.316 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.316 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.316 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:56.316 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852539+0000", 2024-09-16T20:49:56.316 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3389999999999998e-06 2024-09-16T20:49:56.316 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.316 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.316 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:56.316 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852552+0000", 2024-09-16T20:49:56.316 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.271e-05 2024-09-16T20:49:56.316 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.316 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.317 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:56.317 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852563+0000", 2024-09-16T20:49:56.317 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1139e-05 2024-09-16T20:49:56.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.317 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.317 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:56.317 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852883+0000", 2024-09-16T20:49:56.317 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00032010600000000001 2024-09-16T20:49:56.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.317 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.317 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:49:56.317 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852890+0000", 2024-09-16T20:49:56.317 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7669999999999998e-06 2024-09-16T20:49:56.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.318 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.318 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:56.318 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869059+0000", 2024-09-16T20:49:56.318 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.016169353 2024-09-16T20:49:56.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.318 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.318 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:56.318 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869178+0000", 2024-09-16T20:49:56.318 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118193 2024-09-16T20:49:56.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.318 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.318 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:56.318 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869281+0000", 2024-09-16T20:49:56.319 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010310399999999999 2024-09-16T20:49:56.319 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:56.319 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:56.319 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:56.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.319 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.319 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5138.0:9382 3.c 3:33996a43:::rbd_data.110e6372565d.0000000000000007:head [read 0~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T20:49:56.319 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:49:56.319 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 28.427858875999998, 2024-09-16T20:49:56.319 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.017136034, 2024-09-16T20:49:56.319 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:56.319 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:56.319 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:56.319 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:56.320 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5138", 2024-09-16T20:49:56.320 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:49:56.320 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9382 2024-09-16T20:49:56.320 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.320 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:56.320 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.320 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:56.320 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:49:56.320 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:56.320 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.320 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.320 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:56.320 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:49:56.321 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:56.321 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.321 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.321 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:56.321 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852583+0000", 2024-09-16T20:49:56.321 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.51e-07 2024-09-16T20:49:56.321 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.321 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.321 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:56.321 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852586+0000", 2024-09-16T20:49:56.321 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0280000000000001e-06 2024-09-16T20:49:56.321 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.321 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.322 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:56.322 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852595+0000", 2024-09-16T20:49:56.322 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2799999999999992e-06 2024-09-16T20:49:56.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.322 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.322 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:56.322 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852601+0000", 2024-09-16T20:49:56.322 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3810000000000004e-06 2024-09-16T20:49:56.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.322 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.322 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:56.322 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852921+0000", 2024-09-16T20:49:56.322 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00031979699999999997 2024-09-16T20:49:56.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.323 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.323 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:49:56.323 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852930+0000", 2024-09-16T20:49:56.323 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6489999999999997e-06 2024-09-16T20:49:56.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.323 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.323 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:56.323 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869434+0000", 2024-09-16T20:49:56.323 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0165038829999999 2024-09-16T20:49:56.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.323 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.323 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:56.323 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869495+0000", 2024-09-16T20:49:56.323 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1509999999999999e-05 2024-09-16T20:49:56.324 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.324 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.324 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:56.324 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869718+0000", 2024-09-16T20:49:56.324 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000222955 2024-09-16T20:49:56.324 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:56.324 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:56.324 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:56.324 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.324 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.324 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9969 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 632832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:56.324 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642045+0000", 2024-09-16T20:49:56.324 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.6383964190000002, 2024-09-16T20:49:56.325 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076903192999999995, 2024-09-16T20:49:56.325 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:56.325 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:56.325 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:56.325 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:56.325 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:49:56.325 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:56.325 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9969 2024-09-16T20:49:56.325 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.325 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:56.325 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.325 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:56.325 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642045+0000", 2024-09-16T20:49:56.325 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:56.326 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.326 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.326 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:56.326 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642045+0000", 2024-09-16T20:49:56.326 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:56.326 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.326 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.326 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:56.326 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642045+0000", 2024-09-16T20:49:56.326 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5100000000000001e-07 2024-09-16T20:49:56.326 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.326 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.326 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:56.326 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642047+0000", 2024-09-16T20:49:56.327 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8679999999999999e-06 2024-09-16T20:49:56.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.327 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.327 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:56.327 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642053+0000", 2024-09-16T20:49:56.327 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1720000000000004e-06 2024-09-16T20:49:56.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.327 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.327 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:56.327 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642060+0000", 2024-09-16T20:49:56.327 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0339999999999999e-06 2024-09-16T20:49:56.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.327 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.328 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:56.328 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.718562+0000", 2024-09-16T20:49:56.328 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076501969000000003 2024-09-16T20:49:56.328 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.328 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.328 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:56.328 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.718643+0000", 2024-09-16T20:49:56.328 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0539999999999998e-05 2024-09-16T20:49:56.328 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.328 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.328 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:56.328 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.718948+0000", 2024-09-16T20:49:56.328 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000305259 2024-09-16T20:49:56.328 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:56.329 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:56.329 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:56.329 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.329 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.329 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9970 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 637952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:56.329 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642067+0000", 2024-09-16T20:49:56.329 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.6383744389999997, 2024-09-16T20:49:56.329 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077174928000000004, 2024-09-16T20:49:56.329 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:56.329 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:56.329 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:56.329 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:56.329 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:49:56.329 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:56.330 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9970 2024-09-16T20:49:56.330 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.330 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:56.330 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.330 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:56.330 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642067+0000", 2024-09-16T20:49:56.330 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:56.330 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.330 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.330 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:56.330 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642067+0000", 2024-09-16T20:49:56.330 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:56.330 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.331 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.331 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:56.331 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642067+0000", 2024-09-16T20:49:56.331 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2499999999999995e-07 2024-09-16T20:49:56.331 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.331 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.331 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:56.331 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642070+0000", 2024-09-16T20:49:56.331 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3199999999999998e-06 2024-09-16T20:49:56.331 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.331 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.331 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:56.331 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642076+0000", 2024-09-16T20:49:56.331 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.122e-06 2024-09-16T20:49:56.331 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.332 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.332 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:56.332 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642081+0000", 2024-09-16T20:49:56.332 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0869999999999997e-06 2024-09-16T20:49:56.332 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.332 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.332 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:56.332 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.718980+0000", 2024-09-16T20:49:56.332 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076899482000000005 2024-09-16T20:49:56.332 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.332 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.332 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:56.333 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719085+0000", 2024-09-16T20:49:56.333 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010446399999999999 2024-09-16T20:49:56.333 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.333 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.333 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:56.333 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719242+0000", 2024-09-16T20:49:56.333 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000156928 2024-09-16T20:49:56.333 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:56.333 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:56.333 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:56.333 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.333 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.333 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9971 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 643072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:56.333 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642102+0000", 2024-09-16T20:49:56.334 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.6383387320000002, 2024-09-16T20:49:56.334 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077421016999999995, 2024-09-16T20:49:56.334 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:56.334 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:56.334 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:56.334 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:56.334 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:49:56.334 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:56.334 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9971 2024-09-16T20:49:56.334 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.334 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:56.334 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.334 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:56.334 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642102+0000", 2024-09-16T20:49:56.335 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:56.335 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.335 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.335 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:56.335 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642102+0000", 2024-09-16T20:49:56.335 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:56.335 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.335 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.335 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:56.335 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642103+0000", 2024-09-16T20:49:56.335 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7399999999999998e-07 2024-09-16T20:49:56.335 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.335 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.336 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:56.336 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642105+0000", 2024-09-16T20:49:56.336 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2649999999999999e-06 2024-09-16T20:49:56.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.336 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.336 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:56.336 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642112+0000", 2024-09-16T20:49:56.336 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.294e-06 2024-09-16T20:49:56.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.336 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.336 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:56.336 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642118+0000", 2024-09-16T20:49:56.336 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4500000000000003e-06 2024-09-16T20:49:56.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.337 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.337 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:56.337 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719285+0000", 2024-09-16T20:49:56.337 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077166883000000006 2024-09-16T20:49:56.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.337 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.337 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:56.337 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719368+0000", 2024-09-16T20:49:56.337 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2811000000000004e-05 2024-09-16T20:49:56.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.337 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.337 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:56.338 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719523+0000", 2024-09-16T20:49:56.338 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015584 2024-09-16T20:49:56.338 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:56.338 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:56.338 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:56.338 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.338 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.338 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9972 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 648192~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:56.338 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642125+0000", 2024-09-16T20:49:56.338 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.6383157849999996, 2024-09-16T20:49:56.338 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077710324999999997, 2024-09-16T20:49:56.338 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:56.338 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:56.339 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:56.339 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:56.339 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:49:56.339 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:56.339 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9972 2024-09-16T20:49:56.339 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.339 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:56.339 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.339 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:56.339 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642125+0000", 2024-09-16T20:49:56.339 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:56.339 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.339 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.340 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:56.340 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642125+0000", 2024-09-16T20:49:56.340 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:56.340 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.340 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.340 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:56.340 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642126+0000", 2024-09-16T20:49:56.340 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1899999999999998e-07 2024-09-16T20:49:56.340 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.340 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.340 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:56.340 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642128+0000", 2024-09-16T20:49:56.340 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8619999999999999e-06 2024-09-16T20:49:56.341 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.341 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.341 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:56.341 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642134+0000", 2024-09-16T20:49:56.341 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5289999999999997e-06 2024-09-16T20:49:56.341 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.341 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.341 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:56.341 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642139+0000", 2024-09-16T20:49:56.341 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1250000000000001e-06 2024-09-16T20:49:56.341 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.341 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.341 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:56.341 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719570+0000", 2024-09-16T20:49:56.342 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077430516000000005 2024-09-16T20:49:56.342 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.342 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.342 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:56.342 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719655+0000", 2024-09-16T20:49:56.342 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5500000000000005e-05 2024-09-16T20:49:56.342 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.342 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.342 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:56.342 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719836+0000", 2024-09-16T20:49:56.342 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018037399999999999 2024-09-16T20:49:56.342 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:56.342 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:56.343 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:56.343 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.343 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.343 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9973 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 653312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:56.343 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:49:56.343 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.6382951749999997, 2024-09-16T20:49:56.343 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077905262000000003, 2024-09-16T20:49:56.343 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:56.343 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:56.343 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:56.343 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:56.343 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:49:56.343 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:56.343 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9973 2024-09-16T20:49:56.344 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.344 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:56.344 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.344 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:56.344 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:49:56.344 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:56.344 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.344 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.344 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:56.344 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:49:56.344 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:56.344 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.344 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.345 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:56.345 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:49:56.345 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:49:56.345 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.345 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.345 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:56.345 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642148+0000", 2024-09-16T20:49:56.345 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0219999999999999e-06 2024-09-16T20:49:56.345 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.345 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.345 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:56.345 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642154+0000", 2024-09-16T20:49:56.345 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8540000000000002e-06 2024-09-16T20:49:56.346 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.346 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.346 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:56.346 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642159+0000", 2024-09-16T20:49:56.346 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.045e-06 2024-09-16T20:49:56.346 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.346 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.346 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:56.346 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719875+0000", 2024-09-16T20:49:56.346 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077716116000000002 2024-09-16T20:49:56.346 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.346 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.346 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:56.347 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719934+0000", 2024-09-16T20:49:56.347 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8819e-05 2024-09-16T20:49:56.347 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.347 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.347 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:56.347 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720051+0000", 2024-09-16T20:49:56.347 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000116947 2024-09-16T20:49:56.347 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:56.347 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:56.347 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:56.347 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.347 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.347 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9974 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 658432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:56.347 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642166+0000", 2024-09-16T20:49:56.348 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.6382752610000004, 2024-09-16T20:49:56.348 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.078179786000000001, 2024-09-16T20:49:56.348 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:56.348 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:56.348 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:56.348 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:56.348 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:49:56.348 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:56.348 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9974 2024-09-16T20:49:56.348 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.348 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:56.348 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.348 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:56.349 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642166+0000", 2024-09-16T20:49:56.349 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:56.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.349 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.349 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:56.349 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642166+0000", 2024-09-16T20:49:56.349 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:56.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.349 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.349 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:56.349 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647343+0000", 2024-09-16T20:49:56.349 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0051769440000000002 2024-09-16T20:49:56.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.350 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.350 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:56.350 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647348+0000", 2024-09-16T20:49:56.350 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3619999999999997e-06 2024-09-16T20:49:56.350 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.350 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.350 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:56.350 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647359+0000", 2024-09-16T20:49:56.350 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0929e-05 2024-09-16T20:49:56.350 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.350 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.350 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:56.350 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647366+0000", 2024-09-16T20:49:56.350 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0910000000000004e-06 2024-09-16T20:49:56.350 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.351 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.351 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:56.351 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720094+0000", 2024-09-16T20:49:56.351 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.072727403999999995 2024-09-16T20:49:56.351 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.351 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.351 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:56.351 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720173+0000", 2024-09-16T20:49:56.351 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9382000000000007e-05 2024-09-16T20:49:56.351 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.351 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.351 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:56.351 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720346+0000", 2024-09-16T20:49:56.351 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017267399999999999 2024-09-16T20:49:56.352 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:56.352 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:56.352 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:56.352 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.352 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.352 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9988 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 730112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:56.352 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647703+0000", 2024-09-16T20:49:56.352 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.6327382689999999, 2024-09-16T20:49:56.352 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076703473999999994, 2024-09-16T20:49:56.352 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:56.352 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:56.352 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:56.352 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:56.352 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:49:56.352 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:56.353 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9988 2024-09-16T20:49:56.353 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.353 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:56.353 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.353 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:56.353 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647703+0000", 2024-09-16T20:49:56.353 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:56.353 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.353 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.353 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:56.353 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647703+0000", 2024-09-16T20:49:56.353 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:56.353 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.354 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.354 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:56.354 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647703+0000", 2024-09-16T20:49:56.354 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0699999999999998e-07 2024-09-16T20:49:56.354 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.354 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.354 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:56.354 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647705+0000", 2024-09-16T20:49:56.354 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.866e-06 2024-09-16T20:49:56.354 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.354 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.354 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:56.354 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647708+0000", 2024-09-16T20:49:56.354 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3170000000000002e-06 2024-09-16T20:49:56.354 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.355 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.355 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:56.355 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647711+0000", 2024-09-16T20:49:56.355 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6759999999999999e-06 2024-09-16T20:49:56.355 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.355 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.355 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:56.355 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724225+0000", 2024-09-16T20:49:56.355 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.07651434 2024-09-16T20:49:56.355 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.355 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.355 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:56.355 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724291+0000", 2024-09-16T20:49:56.355 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5629000000000005e-05 2024-09-16T20:49:56.356 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.356 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.356 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:56.356 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724406+0000", 2024-09-16T20:49:56.356 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000115239 2024-09-16T20:49:56.356 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:56.356 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:56.356 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:56.356 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.356 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.356 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9989 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 735232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:56.356 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647715+0000", 2024-09-16T20:49:56.356 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.6327265310000003, 2024-09-16T20:49:56.357 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076871693000000005, 2024-09-16T20:49:56.357 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:56.357 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:56.357 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:56.357 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:56.357 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:49:56.357 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:56.357 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9989 2024-09-16T20:49:56.357 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.357 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:56.357 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.357 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:56.357 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647715+0000", 2024-09-16T20:49:56.357 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:56.358 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.358 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.358 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:56.358 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647715+0000", 2024-09-16T20:49:56.358 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:56.358 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.358 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.358 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:56.358 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647715+0000", 2024-09-16T20:49:56.358 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1400000000000001e-07 2024-09-16T20:49:56.358 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.358 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.358 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:56.358 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647716+0000", 2024-09-16T20:49:56.359 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2240000000000001e-06 2024-09-16T20:49:56.359 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.359 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.359 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:56.359 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647719+0000", 2024-09-16T20:49:56.359 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0769999999999999e-06 2024-09-16T20:49:56.359 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.359 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.359 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:56.359 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647722+0000", 2024-09-16T20:49:56.359 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.65e-06 2024-09-16T20:49:56.359 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.359 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.360 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:56.360 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724424+0000", 2024-09-16T20:49:56.360 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076701982000000002 2024-09-16T20:49:56.360 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.360 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.360 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:56.360 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724470+0000", 2024-09-16T20:49:56.360 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6589999999999999e-05 2024-09-16T20:49:56.360 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.360 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.360 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:56.360 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724586+0000", 2024-09-16T20:49:56.360 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000115956 2024-09-16T20:49:56.361 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:56.361 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:56.361 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:56.361 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.361 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.361 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9990 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 740352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:56.361 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647725+0000", 2024-09-16T20:49:56.361 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.6327161769999998, 2024-09-16T20:49:56.361 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077029919000000002, 2024-09-16T20:49:56.361 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:56.361 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:56.361 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:56.361 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:56.361 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:49:56.362 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:56.362 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9990 2024-09-16T20:49:56.362 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.362 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:56.362 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.362 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:56.362 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647725+0000", 2024-09-16T20:49:56.362 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:56.362 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.362 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.362 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:56.362 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647725+0000", 2024-09-16T20:49:56.362 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:56.363 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.363 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.363 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:56.363 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647725+0000", 2024-09-16T20:49:56.363 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2399999999999999e-07 2024-09-16T20:49:56.363 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.363 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.363 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:56.363 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647726+0000", 2024-09-16T20:49:56.363 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.116e-06 2024-09-16T20:49:56.363 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.363 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.363 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:56.363 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647729+0000", 2024-09-16T20:49:56.364 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.937e-06 2024-09-16T20:49:56.364 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.364 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.364 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:56.364 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647732+0000", 2024-09-16T20:49:56.364 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8739999999999999e-06 2024-09-16T20:49:56.364 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.364 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.364 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:56.364 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724612+0000", 2024-09-16T20:49:56.364 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076880102000000006 2024-09-16T20:49:56.364 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.364 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.364 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:56.365 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724649+0000", 2024-09-16T20:49:56.365 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6738e-05 2024-09-16T20:49:56.365 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.365 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.365 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:56.365 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724755+0000", 2024-09-16T20:49:56.365 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010592800000000001 2024-09-16T20:49:56.365 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:56.365 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:56.365 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:56.365 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.365 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.365 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9991 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 745472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:56.366 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:49:56.366 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.6327058680000004, 2024-09-16T20:49:56.366 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077257820000000005, 2024-09-16T20:49:56.366 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:56.366 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:56.366 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:56.366 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:56.366 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:49:56.366 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:56.366 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9991 2024-09-16T20:49:56.366 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.366 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:56.366 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.366 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:56.367 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:49:56.367 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:56.367 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.367 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.367 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:56.367 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:49:56.367 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:56.367 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.367 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.367 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:56.367 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:49:56.367 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1199999999999999e-07 2024-09-16T20:49:56.367 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.367 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.368 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:56.368 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647736+0000", 2024-09-16T20:49:56.368 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3699999999999999e-07 2024-09-16T20:49:56.368 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.368 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.368 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:56.368 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647739+0000", 2024-09-16T20:49:56.368 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9359999999999999e-06 2024-09-16T20:49:56.368 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.368 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.368 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:56.368 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647742+0000", 2024-09-16T20:49:56.368 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5720000000000001e-06 2024-09-16T20:49:56.369 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.369 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.369 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:56.369 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724774+0000", 2024-09-16T20:49:56.369 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077032457999999998 2024-09-16T20:49:56.369 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.369 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.369 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:56.369 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724818+0000", 2024-09-16T20:49:56.369 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3803000000000001e-05 2024-09-16T20:49:56.369 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.369 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.369 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:56.370 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724993+0000", 2024-09-16T20:49:56.370 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000174902 2024-09-16T20:49:56.370 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:56.370 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:56.370 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:56.370 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.370 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.370 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9992 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 750592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:56.370 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:49:56.370 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.6326962790000001, 2024-09-16T20:49:56.370 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077553227000000002, 2024-09-16T20:49:56.370 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:56.370 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:56.370 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:56.371 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:56.371 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:49:56.371 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:56.371 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9992 2024-09-16T20:49:56.371 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.371 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:56.371 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.371 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:56.371 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:49:56.371 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:56.371 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.371 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.371 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:56.372 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:49:56.372 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:56.372 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.372 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.372 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:56.372 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:49:56.372 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0200000000000001e-07 2024-09-16T20:49:56.372 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.372 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.372 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:56.372 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647746+0000", 2024-09-16T20:49:56.372 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.088e-06 2024-09-16T20:49:56.372 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.372 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.373 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:56.373 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647749+0000", 2024-09-16T20:49:56.373 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9170000000000002e-06 2024-09-16T20:49:56.373 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.373 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.373 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:56.373 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647752+0000", 2024-09-16T20:49:56.373 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.774e-06 2024-09-16T20:49:56.373 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.373 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.373 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:56.373 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725042+0000", 2024-09-16T20:49:56.373 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077289952999999995 2024-09-16T20:49:56.373 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.374 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.374 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:56.374 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725113+0000", 2024-09-16T20:49:56.374 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0920999999999994e-05 2024-09-16T20:49:56.374 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.374 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.374 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:56.374 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725298+0000", 2024-09-16T20:49:56.374 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000185372 2024-09-16T20:49:56.374 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:56.374 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:56.374 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:56.374 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.374 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.375 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9993 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 755712~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:56.375 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647755+0000", 2024-09-16T20:49:56.375 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.6326862530000001, 2024-09-16T20:49:56.375 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077725674999999994, 2024-09-16T20:49:56.375 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:56.375 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:56.375 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:56.375 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:56.375 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:49:56.375 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:56.375 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9993 2024-09-16T20:49:56.375 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.375 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:56.376 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.376 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:56.376 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647755+0000", 2024-09-16T20:49:56.376 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:56.376 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.376 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.376 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:56.376 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647755+0000", 2024-09-16T20:49:56.376 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:56.376 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.376 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.376 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:56.376 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653406+0000", 2024-09-16T20:49:56.377 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.005650876 2024-09-16T20:49:56.377 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.377 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.377 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:56.377 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653411+0000", 2024-09-16T20:49:56.377 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8899999999999998e-06 2024-09-16T20:49:56.377 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.377 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.377 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:56.377 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653423+0000", 2024-09-16T20:49:56.377 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2447e-05 2024-09-16T20:49:56.377 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.377 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.377 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:56.378 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653431+0000", 2024-09-16T20:49:56.378 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5730000000000002e-06 2024-09-16T20:49:56.378 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.378 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.378 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:56.378 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725333+0000", 2024-09-16T20:49:56.378 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.071902743000000005 2024-09-16T20:49:56.378 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.378 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.378 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:56.378 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725385+0000", 2024-09-16T20:49:56.378 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1585000000000002e-05 2024-09-16T20:49:56.378 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.378 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.379 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:56.379 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725481+0000", 2024-09-16T20:49:56.379 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5561000000000002e-05 2024-09-16T20:49:56.379 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:56.379 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:56.379 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:56.379 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:56.379 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:56.379 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:56.380 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:56.379+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:56.381 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:49:56.481 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:56.480+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:56.558 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:56.558 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:49:56.558 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.558 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5441.0:8892 3.7 3:e5d5c7f4:::rbd_data.110eaf5b6fa7.0000000000000004:head [read 208896~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116)", 2024-09-16T20:49:56.558 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:56.557579+0000", 2024-09-16T20:49:56.558 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.000105225, 2024-09-16T20:49:56.558 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012365799999999999, 2024-09-16T20:49:56.559 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:56.559 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:56.559 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:49:56.559 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:56.559 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5441", 2024-09-16T20:49:56.559 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3882632001", 2024-09-16T20:49:56.559 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8892 2024-09-16T20:49:56.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.559 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:56.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:56.559 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:56.557579+0000", 2024-09-16T20:49:56.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:56.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.560 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:56.560 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:56.557579+0000", 2024-09-16T20:49:56.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:56.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.560 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:56.560 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:56.557579+0000", 2024-09-16T20:49:56.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6499999999999999e-07 2024-09-16T20:49:56.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.560 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:56.561 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:56.557582+0000", 2024-09-16T20:49:56.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9859999999999999e-06 2024-09-16T20:49:56.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.561 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.561 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:56.561 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:56.557591+0000", 2024-09-16T20:49:56.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5390000000000007e-06 2024-09-16T20:49:56.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.561 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.561 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:56.561 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:56.557596+0000", 2024-09-16T20:49:56.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5639999999999998e-06 2024-09-16T20:49:56.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:56.561 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:56.561 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:56.562 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:56.557649+0000", 2024-09-16T20:49:56.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3831e-05 2024-09-16T20:49:56.562 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:56.562 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:56.562 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:56.562 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:56.562 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-16T20:49:56.562 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 1 2024-09-16T20:49:56.562 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:56.568 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:49:56.582 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:56.581+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:56.600 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T20:49:56.683 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:56.682+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:56.715 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:56.715 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:49:56.715 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:49:56.715 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:49:56.715 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:56.721 INFO:teuthology.orchestra.run.smithi139.stdout:4682 read 0x738a600 thru 0x738da60 (0x3461 bytes) 2024-09-16T20:49:56.724 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:49:56.725 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:49:56.725 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:49:56.725 INFO:teuthology.orchestra.run.smithi139.stdout:4683 write 0x5869000 thru 0x58731c7 (0xa1c8 bytes) 2024-09-16T20:49:56.725 INFO:teuthology.orchestra.run.smithi139.stdout:4687 punch from 0x2ec2000 to 0x2ecc800, (0xa800 bytes) 2024-09-16T20:49:56.741 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:49:56.741 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:49:56.757 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:49:56.757 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:49:56.784 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:56.783+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:56.792 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:49:56.793 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:49:56.810 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:49:56.810 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:49:56.830 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:49:56.830 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:49:56.885 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:56.883+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:56.886 INFO:teuthology.orchestra.run.smithi139.stdout:4688 write 0x6aa3800 thru 0x6aa4a4d (0x124e bytes) 2024-09-16T20:49:56.887 INFO:teuthology.orchestra.run.smithi139.stdout:4689 write 0xb0c8000 thru 0xb0cd1a3 (0x51a4 bytes) 2024-09-16T20:49:56.890 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:56.890 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:56.899 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:49:56.900 INFO:teuthology.orchestra.run.smithi139.stdout:4690 trunc from 0xc5e8200 to 0x1d8ac00 2024-09-16T20:49:56.985 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:56.984+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:57.086 INFO:teuthology.orchestra.run.smithi139.stdout:4691 read 0x1b61c00 thru 0x1b62a13 (0xe14 bytes) 2024-09-16T20:49:57.086 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:57.086 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:57.087 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:57.085+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:57.094 INFO:teuthology.orchestra.run.smithi139.stdout:4692 read 0x9fe200 thru 0xa024df (0x42e0 bytes) 2024-09-16T20:49:57.095 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:49:57.115 INFO:teuthology.orchestra.run.smithi139.stdout:4693 write 0xaacc200 thru 0xaad7193 (0xaf94 bytes) 2024-09-16T20:49:57.122 INFO:teuthology.orchestra.run.smithi139.stdout:4695 write 0x247c200 thru 0x24822c4 (0x60c5 bytes) 2024-09-16T20:49:57.131 INFO:teuthology.orchestra.run.smithi139.stdout:4696 trunc from 0xaad7200 to 0x3c30200 2024-09-16T20:49:57.177 INFO:teuthology.orchestra.run.smithi139.stdout:4697 write 0x38fda00 thru 0x3906bf7 (0x91f8 bytes) 2024-09-16T20:49:57.184 INFO:teuthology.orchestra.run.smithi139.stdout:4698 punch from 0x2ec7e00 to 0x2ed7800, (0xfa00 bytes) 2024-09-16T20:49:57.185 INFO:teuthology.orchestra.run.smithi139.stdout:4700 punch from 0x2e88800 to 0x2e98200, (0xfa00 bytes) 2024-09-16T20:49:57.185 INFO:teuthology.orchestra.run.smithi139.stdout:4701 read 0x1f97400 thru 0x1f9e526 (0x7127 bytes) 2024-09-16T20:49:57.187 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:57.186+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:57.215 INFO:teuthology.orchestra.run.smithi139.stdout:4702 write 0x54e7000 thru 0x54f01eb (0x91ec bytes) 2024-09-16T20:49:57.222 INFO:teuthology.orchestra.run.smithi139.stdout:4703 read 0xd5da00 thru 0xd5f893 (0x1e94 bytes) 2024-09-16T20:49:57.223 INFO:teuthology.orchestra.run.smithi139.stdout:4705 read 0x4104c00 thru 0x4113461 (0xe862 bytes) 2024-09-16T20:49:57.223 INFO:teuthology.orchestra.run.smithi139.stdout:4706 read 0x30fd000 thru 0x3108433 (0xb434 bytes) 2024-09-16T20:49:57.223 INFO:teuthology.orchestra.run.smithi139.stdout:4708 write 0x26e00 thru 0x35a00 (0xec01 bytes) 2024-09-16T20:49:57.224 INFO:teuthology.orchestra.run.smithi139.stdout:4710 read 0x419a200 thru 0x41a4832 (0xa633 bytes) 2024-09-16T20:49:57.289 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:57.288+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:57.291 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:57.291 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:57.304 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:49:57.346 INFO:teuthology.orchestra.run.smithi139.stdout:4712 write 0xc388400 thru 0xc38b41b (0x301c bytes) 2024-09-16T20:49:57.347 INFO:teuthology.orchestra.run.smithi139.stdout:4717 read 0x7477c00 thru 0x747e20b (0x660c bytes) 2024-09-16T20:49:57.347 INFO:teuthology.orchestra.run.smithi139.stdout:4719 read 0xaab7e00 thru 0xaac3c5b (0xbe5c bytes) 2024-09-16T20:49:57.370 INFO:teuthology.orchestra.run.smithi139.stdout:4723 write 0xc55fa00 thru 0xc560a90 (0x1091 bytes) 2024-09-16T20:49:57.371 INFO:teuthology.orchestra.run.smithi139.stdout:4724 write 0x1e23200 thru 0x1e309ad (0xd7ae bytes) 2024-09-16T20:49:57.372 INFO:teuthology.orchestra.run.smithi139.stdout:4725 read 0xae06600 thru 0xae09686 (0x3087 bytes) 2024-09-16T20:49:57.372 INFO:teuthology.orchestra.run.smithi139.stdout:4726 punch from 0x2defc00 to 0x2dfd400, (0xd800 bytes) 2024-09-16T20:49:57.373 INFO:teuthology.orchestra.run.smithi139.stdout:4728 write 0x3e06800 thru 0x3e10f26 (0xa727 bytes) 2024-09-16T20:49:57.384 INFO:teuthology.orchestra.run.smithi139.stdout:4729 trunc from 0xc560c00 to 0x2c11800 2024-09-16T20:49:57.390 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:57.389+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:57.437 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:57.437 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:57.447 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:49:57.474 INFO:teuthology.orchestra.run.smithi139.stdout:4730 write 0xdfa0c00 thru 0xdfa4d37 (0x4138 bytes) 2024-09-16T20:49:57.474 INFO:teuthology.orchestra.run.smithi139.stdout:4735 read 0xc5be400 thru 0xc5c4236 (0x5e37 bytes) 2024-09-16T20:49:57.474 INFO:teuthology.orchestra.run.smithi139.stdout:4737 punch from 0x32c5800 to 0x32cfa00, (0xa200 bytes) 2024-09-16T20:49:57.480 INFO:teuthology.orchestra.run.smithi139.stdout:4739 trunc from 0xdfa4e00 to 0x99d4000 2024-09-16T20:49:57.491 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:57.489+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:57.537 INFO:teuthology.orchestra.run.smithi139.stdout:4743 write 0xc2a7800 thru 0xc2b2781 (0xaf82 bytes) 2024-09-16T20:49:57.537 INFO:teuthology.orchestra.run.smithi139.stdout:4747 write 0x23c4400 thru 0x23d241e (0xe01f bytes) 2024-09-16T20:49:57.543 INFO:teuthology.orchestra.run.smithi139.stdout:4748 trunc from 0xc2b2800 to 0x6f92200 2024-09-16T20:49:57.577 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:57.577 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:57.583 INFO:teuthology.orchestra.run.smithi139.stdout:4750 read 0x641d200 thru 0x64277b1 (0xa5b2 bytes) 2024-09-16T20:49:57.584 INFO:teuthology.orchestra.run.smithi139.stdout:4753 punch from 0x62e9e00 to 0x62f2000, (0x8200 bytes) 2024-09-16T20:49:57.585 INFO:teuthology.orchestra.run.smithi139.stdout:4754 read 0x6e10c00 thru 0x6e143b8 (0x37b9 bytes) 2024-09-16T20:49:57.585 INFO:teuthology.orchestra.run.smithi139.stdout:4756 write 0x6497a00 thru 0x64a4e3d (0xd43e bytes) 2024-09-16T20:49:57.585 INFO:teuthology.orchestra.run.smithi139.stdout:4758 punch from 0x4003a00 to 0x4004800, (0xe00 bytes) 2024-09-16T20:49:57.586 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:49:57.592 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:57.590+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:57.618 INFO:teuthology.orchestra.run.smithi139.stdout:4760 write 0xebd6a00 thru 0xebdaa2c (0x402d bytes) 2024-09-16T20:49:57.619 INFO:teuthology.orchestra.run.smithi139.stdout:4766 write 0x2413e00 thru 0x2416885 (0x2a86 bytes) 2024-09-16T20:49:57.620 INFO:teuthology.orchestra.run.smithi139.stdout:4767 write 0x78b4800 thru 0x78b89f8 (0x41f9 bytes) 2024-09-16T20:49:57.620 INFO:teuthology.orchestra.run.smithi139.stdout:4768 punch from 0x351fa00 to 0x352d800, (0xde00 bytes) 2024-09-16T20:49:57.621 INFO:teuthology.orchestra.run.smithi139.stdout:4769 read 0x17c4400 thru 0x17c770b (0x330c bytes) 2024-09-16T20:49:57.622 INFO:teuthology.orchestra.run.smithi139.stdout:4771 write 0x7eff000 thru 0x7f05d07 (0x6d08 bytes) 2024-09-16T20:49:57.623 INFO:teuthology.orchestra.run.smithi139.stdout:4773 trunc from 0xebdac00 to 0xdf9ae00 2024-09-16T20:49:57.655 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T20:49:57.698 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:57.697+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:57.720 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:57.720 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:57.729 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:49:57.792 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:57.791+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:57.875 INFO:teuthology.orchestra.run.smithi139.stdout:4775 write 0x3d55a00 thru 0x3d55b0d (0x10e bytes) 2024-09-16T20:49:57.875 INFO:teuthology.orchestra.run.smithi139.stdout:4780 write 0x4207400 thru 0x42135fd (0xc1fe bytes) 2024-09-16T20:49:57.875 INFO:teuthology.orchestra.run.smithi139.stdout:4782 punch from 0x25fca00 to 0x25fda00, (0x1000 bytes) 2024-09-16T20:49:57.875 INFO:teuthology.orchestra.run.smithi139.stdout:4783 write 0x204b000 thru 0x204e959 (0x395a bytes) 2024-09-16T20:49:57.876 INFO:teuthology.orchestra.run.smithi139.stdout:4785 write 0x1886e00 thru 0x188a1f8 (0x33f9 bytes) 2024-09-16T20:49:57.877 INFO:teuthology.orchestra.run.smithi139.stdout:4787 write 0xd3a0600 thru 0xd3ac955 (0xc356 bytes) 2024-09-16T20:49:57.877 INFO:teuthology.orchestra.run.smithi139.stdout:4788 write 0x931aa00 thru 0x932815f (0xd760 bytes) 2024-09-16T20:49:57.877 INFO:teuthology.orchestra.run.smithi139.stdout:4791 write 0x76d2600 thru 0x76defac (0xc9ad bytes) 2024-09-16T20:49:57.877 INFO:teuthology.orchestra.run.smithi139.stdout:4794 clone 36 order 18 su 65536 sc 4 2024-09-16T20:49:57.881 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:57.881 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:57.893 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:57.891+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:57.894 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:49:57.994 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:57.992+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:58.019 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:58.019 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:58.028 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:49:58.094 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:58.093+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:58.137 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:58.137 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:58.145 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:49:58.195 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:58.194+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:58.293 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:58.293 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:58.296 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:49:58.294+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:49:58.302 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:49:58.397 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:58.396+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:58.450 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:58.450 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:58.458 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:49:58.498 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:58.497+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:58.598 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:58.597+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:58.605 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:58.606 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:58.615 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:49:58.641 INFO:teuthology.orchestra.run.smithi029.stderr:nodeep-scrub is set 2024-09-16T20:49:58.699 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:58.697+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:58.724 INFO:teuthology.orchestra.run.smithi139.stdout:truncating image image_client.0-clone35 from 0xdf9ae00 (overlap 0x1d8ac00) to 0x5e8800 2024-09-16T20:49:58.730 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:58.730 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:58.746 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:49:58.800 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:58.798+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:58.844 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T20:49:58.901 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:49:58.899+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:49:58.936 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:49:58.936 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:49:58.950 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:49:59.000 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:49:59.000 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:49:59.001 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:59.000+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:59.024 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:49:59.025 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:49:59.043 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:49:59.044 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:49:59.075 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:49:59.075 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:49:59.093 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:49:59.093 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:49:59.102 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:59.101+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:59.113 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:49:59.114 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:49:59.147 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:59.147 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:49:59.147 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:49:59.147 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:49:59.147 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:59.157 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:49:59.204 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:49:59.203+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:49:59.305 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:49:59.303+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:49:59.316 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:59.316 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:49:59.316 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:49:59.316 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:49:59.316 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.316 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:3088 3.1e 3:79e5ec01:::rbd_header.110e891a9b2b:head [watch watch cookie 140213437171488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:49:59.316 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:49:59.316 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 95.149382437, 2024-09-16T20:49:59.316 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.11895186200000001, 2024-09-16T20:49:59.316 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.316 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.317 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:59.317 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.317 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:59.317 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:59.317 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3088 2024-09-16T20:49:59.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.317 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.317 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.317 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.317 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:49:59.317 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.317 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.318 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.318 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:49:59.318 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.318 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.318 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.318 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163846+0000", 2024-09-16T20:49:59.318 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6799999999999999e-07 2024-09-16T20:49:59.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.318 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.318 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.318 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163850+0000", 2024-09-16T20:49:59.318 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.929e-06 2024-09-16T20:49:59.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.318 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.319 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.319 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163862+0000", 2024-09-16T20:49:59.319 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1972e-05 2024-09-16T20:49:59.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.319 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.319 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.319 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163867+0000", 2024-09-16T20:49:59.319 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.451e-06 2024-09-16T20:49:59.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.319 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.319 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:59.319 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163912+0000", 2024-09-16T20:49:59.319 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4456000000000003e-05 2024-09-16T20:49:59.320 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.320 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.320 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:59.320 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163962+0000", 2024-09-16T20:49:59.320 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0578000000000003e-05 2024-09-16T20:49:59.320 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.320 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.320 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-16T20:49:59.320 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164058+0000", 2024-09-16T20:49:59.320 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5934999999999995e-05 2024-09-16T20:49:59.321 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.321 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.321 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:49:59.321 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164570+0000", 2024-09-16T20:49:59.321 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00051241100000000005 2024-09-16T20:49:59.321 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.322 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.322 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:59.322 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282629+0000", 2024-09-16T20:49:59.322 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.118058986 2024-09-16T20:49:59.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.322 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.322 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:49:59.322 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282725+0000", 2024-09-16T20:49:59.322 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5063999999999996e-05 2024-09-16T20:49:59.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.322 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.322 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:59.322 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282797+0000", 2024-09-16T20:49:59.323 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2212000000000006e-05 2024-09-16T20:49:59.323 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.323 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.323 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.323 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.323 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:8039 3.1e 3.ac44ff9e (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:49:59.323 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:49:59.323 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 69.605555088000003, 2024-09-16T20:49:59.323 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0053310449999999, 2024-09-16T20:49:59.323 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.323 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.323 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:49:59.323 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.323 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:59.324 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:59.324 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8039 2024-09-16T20:49:59.324 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.324 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.324 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.324 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.324 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:49:59.324 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.324 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.324 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.324 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.324 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:49:59.324 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.325 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.325 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.325 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.325 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707674+0000", 2024-09-16T20:49:59.325 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6530000000000001e-06 2024-09-16T20:49:59.325 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.325 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.325 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.325 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707681+0000", 2024-09-16T20:49:59.325 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9639999999999997e-06 2024-09-16T20:49:59.325 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.325 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.325 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.326 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707700+0000", 2024-09-16T20:49:59.326 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8674000000000001e-05 2024-09-16T20:49:59.326 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.326 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.326 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.326 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707708+0000", 2024-09-16T20:49:59.326 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6810000000000004e-06 2024-09-16T20:49:59.326 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.326 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.326 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:59.326 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711293+0000", 2024-09-16T20:49:59.326 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035848189999999999 2024-09-16T20:49:59.326 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.326 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.326 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:49:59.327 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711301+0000", 2024-09-16T20:49:59.327 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4730000000000003e-06 2024-09-16T20:49:59.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.327 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.327 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:59.327 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.712990+0000", 2024-09-16T20:49:59.327 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.001689348 2024-09-16T20:49:59.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.327 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.327 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:59.327 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713003+0000", 2024-09-16T20:49:59.327 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3433e-05 2024-09-16T20:49:59.327 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.327 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.328 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.328 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.328 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.328 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:8040 3.1e 3.ac44ff9e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:49:59.328 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:49:59.328 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 69.605494692999997, 2024-09-16T20:49:59.328 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0053410780000001, 2024-09-16T20:49:59.328 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.328 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.328 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:49:59.328 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.328 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:59.328 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:59.328 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8040 2024-09-16T20:49:59.329 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.329 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.329 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.329 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.329 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:49:59.329 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.329 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.329 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.329 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.329 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:49:59.329 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.329 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.329 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.330 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.330 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:49:59.330 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8100000000000003e-07 2024-09-16T20:49:59.330 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.330 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.330 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.330 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707737+0000", 2024-09-16T20:49:59.330 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5020000000000001e-06 2024-09-16T20:49:59.330 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.330 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.330 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.330 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707745+0000", 2024-09-16T20:49:59.330 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2169999999999994e-06 2024-09-16T20:49:59.330 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.330 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.331 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.331 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707751+0000", 2024-09-16T20:49:59.331 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.263e-06 2024-09-16T20:49:59.331 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.331 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.331 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:59.331 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711336+0000", 2024-09-16T20:49:59.331 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035848360000000001 2024-09-16T20:49:59.331 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.331 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.331 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:49:59.331 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711340+0000", 2024-09-16T20:49:59.331 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1309999999999999e-06 2024-09-16T20:49:59.332 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.332 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.332 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:59.332 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713060+0000", 2024-09-16T20:49:59.332 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0017195059999999 2024-09-16T20:49:59.332 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.332 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.332 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:59.332 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713074+0000", 2024-09-16T20:49:59.332 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4042e-05 2024-09-16T20:49:59.332 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.332 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.332 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.333 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.333 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.333 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4927.0:4487 3.1e 3.ac44ff9e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:49:59.333 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:49:59.333 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 69.604974490999993, 2024-09-16T20:49:59.333 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0048840880000001, 2024-09-16T20:49:59.333 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.333 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.333 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:49:59.333 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.333 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4927", 2024-09-16T20:49:59.333 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:49:59.334 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4487 2024-09-16T20:49:59.334 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.334 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.334 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.334 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.334 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:49:59.334 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.334 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.334 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.334 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.334 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:49:59.334 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.334 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.335 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.335 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.335 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708254+0000", 2024-09-16T20:49:59.335 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1800000000000005e-07 2024-09-16T20:49:59.335 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.335 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.335 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.335 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708258+0000", 2024-09-16T20:49:59.335 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8689999999999997e-06 2024-09-16T20:49:59.335 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.335 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.335 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.335 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708267+0000", 2024-09-16T20:49:59.335 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9590000000000007e-06 2024-09-16T20:49:59.335 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.336 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.336 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.336 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708274+0000", 2024-09-16T20:49:59.336 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9970000000000001e-06 2024-09-16T20:49:59.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.336 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.336 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:59.336 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711366+0000", 2024-09-16T20:49:59.336 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030919530000000001 2024-09-16T20:49:59.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.336 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.336 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:49:59.336 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711372+0000", 2024-09-16T20:49:59.337 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9410000000000004e-06 2024-09-16T20:49:59.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.337 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.337 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:59.337 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713126+0000", 2024-09-16T20:49:59.337 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0017537249999999 2024-09-16T20:49:59.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.337 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.337 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:59.337 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713137+0000", 2024-09-16T20:49:59.337 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0825999999999999e-05 2024-09-16T20:49:59.337 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.337 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.337 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.338 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.338 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5089.0:11703 3.1b 3:dcd4684e:::rbd_data.110e747f0779.000000000000001b:head [set-alloc-hint object_size 262144 write_size 262144,write 139264~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:49:59.338 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:49:59.338 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 55.175238716999999, 2024-09-16T20:49:59.338 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.084231385000000006, 2024-09-16T20:49:59.338 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.338 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.338 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:59.338 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.338 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5089", 2024-09-16T20:49:59.338 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:49:59.338 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 11703 2024-09-16T20:49:59.338 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.339 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.339 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.339 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.339 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:49:59.339 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.339 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.339 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.339 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.339 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:49:59.339 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.339 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.339 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.339 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.340 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:49:59.340 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2600000000000001e-07 2024-09-16T20:49:59.340 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.340 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.340 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.340 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137998+0000", 2024-09-16T20:49:59.340 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9549999999999998e-06 2024-09-16T20:49:59.340 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.340 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.340 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.340 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138008+0000", 2024-09-16T20:49:59.340 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3789999999999995e-06 2024-09-16T20:49:59.340 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.341 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.341 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.341 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138013+0000", 2024-09-16T20:49:59.341 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2209999999999996e-06 2024-09-16T20:49:59.341 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.341 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.341 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:59.341 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138054+0000", 2024-09-16T20:49:59.341 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1022999999999997e-05 2024-09-16T20:49:59.341 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.341 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.341 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:59.341 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138124+0000", 2024-09-16T20:49:59.341 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0234999999999994e-05 2024-09-16T20:49:59.342 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.342 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.342 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-16T20:49:59.342 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138197+0000", 2024-09-16T20:49:59.342 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2451000000000001e-05 2024-09-16T20:49:59.342 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.342 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.342 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:49:59.342 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222010+0000", 2024-09-16T20:49:59.342 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.083813433000000007 2024-09-16T20:49:59.342 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.342 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.342 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:49:59.342 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222163+0000", 2024-09-16T20:49:59.342 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015274599999999999 2024-09-16T20:49:59.343 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.343 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.343 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:49:59.343 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222197+0000", 2024-09-16T20:49:59.343 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3683000000000003e-05 2024-09-16T20:49:59.343 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.343 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.343 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:59.343 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222220+0000", 2024-09-16T20:49:59.343 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3433000000000001e-05 2024-09-16T20:49:59.343 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.344 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.344 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.344 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.344 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.344 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.5089.0:11702 3.15 e73/64 3:acf33c8b:::rbd_data.110e747f0779.000000000000001a:head v 73'132, mlcod=73'130)", 2024-09-16T20:49:59.344 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:49:59.344 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 55.174971388000003, 2024-09-16T20:49:59.344 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.084079763000000002, 2024-09-16T20:49:59.344 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.344 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.344 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:49:59.344 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.344 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.345 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.345 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:49:59.345 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.345 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.345 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.345 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.345 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:49:59.345 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.345 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.345 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.346 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.346 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:49:59.346 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:49:59.346 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.346 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.346 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.346 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138263+0000", 2024-09-16T20:49:59.346 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1e-06 2024-09-16T20:49:59.346 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.346 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.346 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.346 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221589+0000", 2024-09-16T20:49:59.346 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.083326002999999996 2024-09-16T20:49:59.346 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.346 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.347 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.347 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221614+0000", 2024-09-16T20:49:59.347 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5593999999999999e-05 2024-09-16T20:49:59.347 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.347 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.347 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:59.347 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221714+0000", 2024-09-16T20:49:59.347 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9889000000000005e-05 2024-09-16T20:49:59.347 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.347 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.347 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:59.347 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221727+0000", 2024-09-16T20:49:59.347 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3125999999999999e-05 2024-09-16T20:49:59.347 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.347 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.348 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:49:59.348 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222294+0000", 2024-09-16T20:49:59.348 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00056712199999999998 2024-09-16T20:49:59.348 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.348 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.348 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:59.348 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222336+0000", 2024-09-16T20:49:59.348 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1470000000000001e-05 2024-09-16T20:49:59.348 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.348 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.348 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.348 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.348 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.348 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5138.0:9358 3.c 3:33996a43:::rbd_data.110e6372565d.0000000000000007:head [read 1343488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T20:49:59.349 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:49:59.349 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 31.460695978, 2024-09-16T20:49:59.349 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0167493569999999, 2024-09-16T20:49:59.349 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.349 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.349 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:59.349 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.349 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5138", 2024-09-16T20:49:59.349 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:49:59.349 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9358 2024-09-16T20:49:59.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.349 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.349 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.349 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.350 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:49:59.350 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.350 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.350 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.350 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.350 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:49:59.350 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.350 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.350 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.350 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.350 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852533+0000", 2024-09-16T20:49:59.350 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.646e-06 2024-09-16T20:49:59.350 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.350 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.350 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.351 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852539+0000", 2024-09-16T20:49:59.351 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3389999999999998e-06 2024-09-16T20:49:59.351 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.351 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.351 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.351 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852552+0000", 2024-09-16T20:49:59.351 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.271e-05 2024-09-16T20:49:59.351 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.351 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.351 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.351 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852563+0000", 2024-09-16T20:49:59.352 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1139e-05 2024-09-16T20:49:59.352 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.352 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.352 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:59.352 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852883+0000", 2024-09-16T20:49:59.352 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00032010600000000001 2024-09-16T20:49:59.352 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.352 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.352 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:49:59.352 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852890+0000", 2024-09-16T20:49:59.352 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7669999999999998e-06 2024-09-16T20:49:59.352 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.352 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.353 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:59.353 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869059+0000", 2024-09-16T20:49:59.353 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.016169353 2024-09-16T20:49:59.353 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.353 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.353 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:59.353 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869178+0000", 2024-09-16T20:49:59.353 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118193 2024-09-16T20:49:59.353 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.353 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.353 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:59.353 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869281+0000", 2024-09-16T20:49:59.353 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010310399999999999 2024-09-16T20:49:59.353 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.354 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.354 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.354 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.354 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.354 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5138.0:9382 3.c 3:33996a43:::rbd_data.110e6372565d.0000000000000007:head [read 0~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T20:49:59.354 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:49:59.354 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 31.460645101000001, 2024-09-16T20:49:59.354 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.017136034, 2024-09-16T20:49:59.354 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.354 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.354 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:59.354 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.354 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5138", 2024-09-16T20:49:59.354 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:49:59.355 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9382 2024-09-16T20:49:59.355 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.355 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.355 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.355 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.355 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:49:59.355 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.355 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.355 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.355 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.355 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:49:59.355 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.355 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.355 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.355 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.356 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852583+0000", 2024-09-16T20:49:59.356 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.51e-07 2024-09-16T20:49:59.356 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.356 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.356 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.356 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852586+0000", 2024-09-16T20:49:59.356 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0280000000000001e-06 2024-09-16T20:49:59.356 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.356 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.356 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.356 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852595+0000", 2024-09-16T20:49:59.356 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2799999999999992e-06 2024-09-16T20:49:59.356 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.356 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.356 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.357 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852601+0000", 2024-09-16T20:49:59.357 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3810000000000004e-06 2024-09-16T20:49:59.357 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.357 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.357 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:59.357 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852921+0000", 2024-09-16T20:49:59.357 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00031979699999999997 2024-09-16T20:49:59.357 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.357 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.357 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:49:59.357 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852930+0000", 2024-09-16T20:49:59.357 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6489999999999997e-06 2024-09-16T20:49:59.357 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.357 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.358 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:59.358 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869434+0000", 2024-09-16T20:49:59.358 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0165038829999999 2024-09-16T20:49:59.358 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.358 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.358 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:59.358 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869495+0000", 2024-09-16T20:49:59.358 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1509999999999999e-05 2024-09-16T20:49:59.358 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.358 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.358 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:59.358 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869718+0000", 2024-09-16T20:49:59.358 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000222955 2024-09-16T20:49:59.358 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.359 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.359 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.359 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.359 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.359 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9969 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 632832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:59.359 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642045+0000", 2024-09-16T20:49:59.359 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.6711826439999999, 2024-09-16T20:49:59.359 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076903192999999995, 2024-09-16T20:49:59.359 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.359 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.359 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:59.359 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.359 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:49:59.359 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:59.360 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9969 2024-09-16T20:49:59.360 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.360 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.360 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.360 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.360 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642045+0000", 2024-09-16T20:49:59.360 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.360 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.360 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.360 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.360 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642045+0000", 2024-09-16T20:49:59.360 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.360 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.360 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.361 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.361 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642045+0000", 2024-09-16T20:49:59.361 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5100000000000001e-07 2024-09-16T20:49:59.361 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.361 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.361 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.361 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642047+0000", 2024-09-16T20:49:59.361 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8679999999999999e-06 2024-09-16T20:49:59.361 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.361 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.361 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.361 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642053+0000", 2024-09-16T20:49:59.362 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1720000000000004e-06 2024-09-16T20:49:59.362 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.362 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.362 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.362 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642060+0000", 2024-09-16T20:49:59.362 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0339999999999999e-06 2024-09-16T20:49:59.362 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.362 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.363 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:59.363 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.718562+0000", 2024-09-16T20:49:59.363 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076501969000000003 2024-09-16T20:49:59.363 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.363 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.363 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:59.363 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.718643+0000", 2024-09-16T20:49:59.363 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0539999999999998e-05 2024-09-16T20:49:59.363 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.363 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.363 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:59.363 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.718948+0000", 2024-09-16T20:49:59.363 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000305259 2024-09-16T20:49:59.363 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.364 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.364 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.364 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.364 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.364 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9970 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 637952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:59.364 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642067+0000", 2024-09-16T20:49:59.364 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.6711606640000003, 2024-09-16T20:49:59.364 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077174928000000004, 2024-09-16T20:49:59.364 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.364 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.364 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:59.364 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.364 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:49:59.364 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:59.364 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9970 2024-09-16T20:49:59.365 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.365 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.365 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.365 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.365 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642067+0000", 2024-09-16T20:49:59.365 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.365 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.365 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.365 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.365 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642067+0000", 2024-09-16T20:49:59.365 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.365 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.365 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.366 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.366 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642067+0000", 2024-09-16T20:49:59.366 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2499999999999995e-07 2024-09-16T20:49:59.366 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.366 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.366 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.366 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642070+0000", 2024-09-16T20:49:59.366 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3199999999999998e-06 2024-09-16T20:49:59.366 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.366 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.366 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.366 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642076+0000", 2024-09-16T20:49:59.366 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.122e-06 2024-09-16T20:49:59.367 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.367 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.367 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.367 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642081+0000", 2024-09-16T20:49:59.367 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0869999999999997e-06 2024-09-16T20:49:59.367 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.367 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.367 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:59.367 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.718980+0000", 2024-09-16T20:49:59.367 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076899482000000005 2024-09-16T20:49:59.367 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.367 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.367 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:59.367 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719085+0000", 2024-09-16T20:49:59.368 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010446399999999999 2024-09-16T20:49:59.368 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.368 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.368 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:59.368 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719242+0000", 2024-09-16T20:49:59.368 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000156928 2024-09-16T20:49:59.368 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.368 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.368 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.368 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.368 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.368 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9971 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 643072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:59.368 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642102+0000", 2024-09-16T20:49:59.368 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.671124957, 2024-09-16T20:49:59.369 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077421016999999995, 2024-09-16T20:49:59.369 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.369 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.369 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:59.369 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.369 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:49:59.369 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:59.369 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9971 2024-09-16T20:49:59.369 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.369 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.369 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.369 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.369 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642102+0000", 2024-09-16T20:49:59.369 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.370 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.370 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.370 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.370 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642102+0000", 2024-09-16T20:49:59.370 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.370 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.370 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.370 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.370 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642103+0000", 2024-09-16T20:49:59.370 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7399999999999998e-07 2024-09-16T20:49:59.370 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.370 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.370 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.370 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642105+0000", 2024-09-16T20:49:59.371 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2649999999999999e-06 2024-09-16T20:49:59.371 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.371 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.371 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.371 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642112+0000", 2024-09-16T20:49:59.371 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.294e-06 2024-09-16T20:49:59.371 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.371 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.371 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.371 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642118+0000", 2024-09-16T20:49:59.371 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4500000000000003e-06 2024-09-16T20:49:59.372 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.372 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.372 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:59.372 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719285+0000", 2024-09-16T20:49:59.372 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077166883000000006 2024-09-16T20:49:59.372 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.372 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.372 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:59.372 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719368+0000", 2024-09-16T20:49:59.372 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2811000000000004e-05 2024-09-16T20:49:59.372 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.372 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.372 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:59.373 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719523+0000", 2024-09-16T20:49:59.373 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015584 2024-09-16T20:49:59.373 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.373 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.373 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.373 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.373 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.373 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9972 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 648192~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:59.373 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642125+0000", 2024-09-16T20:49:59.373 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.6711020100000002, 2024-09-16T20:49:59.373 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077710324999999997, 2024-09-16T20:49:59.373 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.373 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.374 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:59.374 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.374 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:49:59.374 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:59.374 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9972 2024-09-16T20:49:59.374 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.374 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.374 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.374 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.374 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642125+0000", 2024-09-16T20:49:59.374 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.374 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.374 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.374 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.374 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642125+0000", 2024-09-16T20:49:59.375 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.375 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.375 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.375 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.375 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642126+0000", 2024-09-16T20:49:59.375 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1899999999999998e-07 2024-09-16T20:49:59.375 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.375 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.375 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.375 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642128+0000", 2024-09-16T20:49:59.375 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8619999999999999e-06 2024-09-16T20:49:59.375 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.375 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.375 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.376 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642134+0000", 2024-09-16T20:49:59.376 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5289999999999997e-06 2024-09-16T20:49:59.376 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.376 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.376 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.376 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642139+0000", 2024-09-16T20:49:59.376 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1250000000000001e-06 2024-09-16T20:49:59.376 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.376 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.376 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:59.376 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719570+0000", 2024-09-16T20:49:59.376 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077430516000000005 2024-09-16T20:49:59.376 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.376 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.377 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:59.377 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719655+0000", 2024-09-16T20:49:59.377 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5500000000000005e-05 2024-09-16T20:49:59.377 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.377 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.377 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:59.377 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719836+0000", 2024-09-16T20:49:59.377 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018037399999999999 2024-09-16T20:49:59.377 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.377 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.377 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.377 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.377 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.377 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9973 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 653312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:59.378 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:49:59.378 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.6710814000000003, 2024-09-16T20:49:59.378 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077905262000000003, 2024-09-16T20:49:59.378 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.378 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.378 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:59.378 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.378 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:49:59.378 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:59.378 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9973 2024-09-16T20:49:59.378 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.378 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.378 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.378 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.379 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:49:59.379 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.379 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.379 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.379 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.379 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:49:59.379 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.379 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.379 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.379 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.379 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:49:59.379 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:49:59.380 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.380 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.380 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.380 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642148+0000", 2024-09-16T20:49:59.380 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0219999999999999e-06 2024-09-16T20:49:59.380 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.380 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.380 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.380 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642154+0000", 2024-09-16T20:49:59.380 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8540000000000002e-06 2024-09-16T20:49:59.380 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.380 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.380 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.381 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642159+0000", 2024-09-16T20:49:59.381 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.045e-06 2024-09-16T20:49:59.381 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.381 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.381 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:59.381 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719875+0000", 2024-09-16T20:49:59.381 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077716116000000002 2024-09-16T20:49:59.381 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.381 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.381 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:59.381 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719934+0000", 2024-09-16T20:49:59.381 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8819e-05 2024-09-16T20:49:59.381 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.381 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.382 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:59.382 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720051+0000", 2024-09-16T20:49:59.382 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000116947 2024-09-16T20:49:59.382 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.382 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.382 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.382 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.382 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.382 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9974 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 658432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:59.382 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642166+0000", 2024-09-16T20:49:59.382 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.6710614859999993, 2024-09-16T20:49:59.382 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.078179786000000001, 2024-09-16T20:49:59.383 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.383 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.383 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:59.383 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.383 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:49:59.383 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:59.383 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9974 2024-09-16T20:49:59.383 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.383 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.383 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.383 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.383 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642166+0000", 2024-09-16T20:49:59.383 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.383 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.383 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.384 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.384 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642166+0000", 2024-09-16T20:49:59.384 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.384 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.384 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.384 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.384 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647343+0000", 2024-09-16T20:49:59.384 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0051769440000000002 2024-09-16T20:49:59.384 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.384 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.384 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.384 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647348+0000", 2024-09-16T20:49:59.384 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3619999999999997e-06 2024-09-16T20:49:59.385 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.385 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.385 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.385 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647359+0000", 2024-09-16T20:49:59.385 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0929e-05 2024-09-16T20:49:59.385 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.385 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.385 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.385 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647366+0000", 2024-09-16T20:49:59.385 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0910000000000004e-06 2024-09-16T20:49:59.385 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.385 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.385 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:59.385 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720094+0000", 2024-09-16T20:49:59.386 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.072727403999999995 2024-09-16T20:49:59.386 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.386 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.386 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:59.386 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720173+0000", 2024-09-16T20:49:59.386 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9382000000000007e-05 2024-09-16T20:49:59.386 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.386 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.386 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:59.386 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720346+0000", 2024-09-16T20:49:59.386 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017267399999999999 2024-09-16T20:49:59.386 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.386 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.386 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.386 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.387 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.387 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9988 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 730112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:59.387 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647703+0000", 2024-09-16T20:49:59.387 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.6655244939999996, 2024-09-16T20:49:59.387 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076703473999999994, 2024-09-16T20:49:59.387 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.387 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.387 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:59.387 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.387 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:49:59.387 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:59.387 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9988 2024-09-16T20:49:59.387 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.388 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.388 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.388 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.388 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647703+0000", 2024-09-16T20:49:59.388 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.388 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.388 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.388 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.388 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647703+0000", 2024-09-16T20:49:59.388 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.388 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.388 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.388 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.389 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647703+0000", 2024-09-16T20:49:59.389 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0699999999999998e-07 2024-09-16T20:49:59.389 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.389 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.389 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.389 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647705+0000", 2024-09-16T20:49:59.389 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.866e-06 2024-09-16T20:49:59.389 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.389 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.389 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.389 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647708+0000", 2024-09-16T20:49:59.389 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3170000000000002e-06 2024-09-16T20:49:59.389 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.389 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.390 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.390 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647711+0000", 2024-09-16T20:49:59.390 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6759999999999999e-06 2024-09-16T20:49:59.390 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.390 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.390 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:59.390 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724225+0000", 2024-09-16T20:49:59.390 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.07651434 2024-09-16T20:49:59.390 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.390 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.390 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:59.390 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724291+0000", 2024-09-16T20:49:59.390 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5629000000000005e-05 2024-09-16T20:49:59.390 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.391 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.391 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:59.391 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724406+0000", 2024-09-16T20:49:59.391 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000115239 2024-09-16T20:49:59.391 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.391 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.391 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.391 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.391 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.391 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9989 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 735232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:59.391 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647715+0000", 2024-09-16T20:49:59.391 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.665512756, 2024-09-16T20:49:59.391 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076871693000000005, 2024-09-16T20:49:59.391 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.392 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.392 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:59.392 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.392 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:49:59.392 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:59.392 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9989 2024-09-16T20:49:59.392 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.392 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.392 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.392 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.392 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647715+0000", 2024-09-16T20:49:59.392 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.393 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.393 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.393 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.393 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647715+0000", 2024-09-16T20:49:59.393 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.393 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.393 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.393 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.393 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647715+0000", 2024-09-16T20:49:59.393 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1400000000000001e-07 2024-09-16T20:49:59.393 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.393 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.393 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.393 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647716+0000", 2024-09-16T20:49:59.394 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2240000000000001e-06 2024-09-16T20:49:59.394 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.394 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.394 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.394 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647719+0000", 2024-09-16T20:49:59.394 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0769999999999999e-06 2024-09-16T20:49:59.394 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.394 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.394 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.394 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647722+0000", 2024-09-16T20:49:59.394 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.65e-06 2024-09-16T20:49:59.394 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.394 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.394 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:59.394 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724424+0000", 2024-09-16T20:49:59.395 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076701982000000002 2024-09-16T20:49:59.395 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.395 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.395 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:59.395 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724470+0000", 2024-09-16T20:49:59.395 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6589999999999999e-05 2024-09-16T20:49:59.395 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.396 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.396 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:59.396 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724586+0000", 2024-09-16T20:49:59.396 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000115956 2024-09-16T20:49:59.396 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.396 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.397 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.397 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.397 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.397 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9990 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 740352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:59.397 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647725+0000", 2024-09-16T20:49:59.397 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.6655024019999995, 2024-09-16T20:49:59.398 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077029919000000002, 2024-09-16T20:49:59.398 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.398 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.398 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:59.398 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.398 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:49:59.398 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:59.398 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9990 2024-09-16T20:49:59.399 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.399 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.399 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.399 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.399 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647725+0000", 2024-09-16T20:49:59.399 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.399 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.399 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.400 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.400 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647725+0000", 2024-09-16T20:49:59.400 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.400 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.400 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.400 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.400 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647725+0000", 2024-09-16T20:49:59.400 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2399999999999999e-07 2024-09-16T20:49:59.401 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.401 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.401 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.401 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647726+0000", 2024-09-16T20:49:59.401 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.116e-06 2024-09-16T20:49:59.401 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.402 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.402 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.402 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647729+0000", 2024-09-16T20:49:59.402 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.937e-06 2024-09-16T20:49:59.402 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.402 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.403 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.403 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647732+0000", 2024-09-16T20:49:59.403 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8739999999999999e-06 2024-09-16T20:49:59.403 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.403 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.403 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:59.403 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724612+0000", 2024-09-16T20:49:59.404 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076880102000000006 2024-09-16T20:49:59.404 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.404 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.404 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:59.404 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724649+0000", 2024-09-16T20:49:59.404 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6738e-05 2024-09-16T20:49:59.404 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.404 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.404 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:59.404 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724755+0000", 2024-09-16T20:49:59.404 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010592800000000001 2024-09-16T20:49:59.404 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.405 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.405 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.405 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.405 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.405 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9991 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 745472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:59.405 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:49:59.405 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.6654920929999992, 2024-09-16T20:49:59.405 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077257820000000005, 2024-09-16T20:49:59.405 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.405 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.405 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:59.405 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.405 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:49:59.405 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:59.406 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9991 2024-09-16T20:49:59.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.406 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.406 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.406 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.406 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:49:59.406 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.406 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.406 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.406 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:49:59.406 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.406 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.407 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.407 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:49:59.407 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1199999999999999e-07 2024-09-16T20:49:59.407 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.407 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.407 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.407 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647736+0000", 2024-09-16T20:49:59.407 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3699999999999999e-07 2024-09-16T20:49:59.407 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.407 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.407 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.407 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647739+0000", 2024-09-16T20:49:59.407 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9359999999999999e-06 2024-09-16T20:49:59.407 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.408 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.408 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.408 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647742+0000", 2024-09-16T20:49:59.408 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5720000000000001e-06 2024-09-16T20:49:59.408 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.408 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.408 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:59.408 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724774+0000", 2024-09-16T20:49:59.408 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077032457999999998 2024-09-16T20:49:59.408 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.408 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.408 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:59.408 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724818+0000", 2024-09-16T20:49:59.408 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3803000000000001e-05 2024-09-16T20:49:59.409 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.409 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.409 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:59.409 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724993+0000", 2024-09-16T20:49:59.409 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000174902 2024-09-16T20:49:59.409 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.409 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.409 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.409 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.409 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.409 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9992 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 750592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:59.409 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:49:59.409 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.6654825039999999, 2024-09-16T20:49:59.410 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077553227000000002, 2024-09-16T20:49:59.410 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.410 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.410 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:59.410 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.410 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:49:59.410 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:59.410 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9992 2024-09-16T20:49:59.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.410 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.410 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.410 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.410 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:49:59.411 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.411 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.411 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.411 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.411 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:49:59.411 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.411 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.411 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.411 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.411 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:49:59.412 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0200000000000001e-07 2024-09-16T20:49:59.412 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.412 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.412 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.412 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647746+0000", 2024-09-16T20:49:59.412 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.088e-06 2024-09-16T20:49:59.412 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.412 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.412 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.412 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647749+0000", 2024-09-16T20:49:59.412 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9170000000000002e-06 2024-09-16T20:49:59.412 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.413 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.413 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.413 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647752+0000", 2024-09-16T20:49:59.413 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.774e-06 2024-09-16T20:49:59.413 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.413 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.413 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:59.413 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725042+0000", 2024-09-16T20:49:59.413 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077289952999999995 2024-09-16T20:49:59.413 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.413 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.413 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:59.413 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725113+0000", 2024-09-16T20:49:59.414 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0920999999999994e-05 2024-09-16T20:49:59.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.414 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.414 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:59.414 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725298+0000", 2024-09-16T20:49:59.414 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000185372 2024-09-16T20:49:59.414 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.414 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.414 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.414 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.414 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9993 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 755712~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:49:59.414 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647755+0000", 2024-09-16T20:49:59.414 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.6654724779999999, 2024-09-16T20:49:59.414 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077725674999999994, 2024-09-16T20:49:59.415 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.415 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.415 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:49:59.415 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.415 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:49:59.415 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:49:59.415 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9993 2024-09-16T20:49:59.415 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.415 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.415 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.415 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.415 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647755+0000", 2024-09-16T20:49:59.415 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.415 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.415 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.415 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.416 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647755+0000", 2024-09-16T20:49:59.416 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.416 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.416 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.416 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.416 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653406+0000", 2024-09-16T20:49:59.416 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.005650876 2024-09-16T20:49:59.416 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.416 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.416 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.416 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653411+0000", 2024-09-16T20:49:59.416 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8899999999999998e-06 2024-09-16T20:49:59.416 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.416 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.417 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.417 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653423+0000", 2024-09-16T20:49:59.417 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2447e-05 2024-09-16T20:49:59.417 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.417 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.417 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.417 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653431+0000", 2024-09-16T20:49:59.417 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5730000000000002e-06 2024-09-16T20:49:59.417 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.417 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.417 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:49:59.417 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725333+0000", 2024-09-16T20:49:59.417 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.071902743000000005 2024-09-16T20:49:59.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.418 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.418 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:49:59.418 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725385+0000", 2024-09-16T20:49:59.418 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1585000000000002e-05 2024-09-16T20:49:59.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.418 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.418 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:49:59.418 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725481+0000", 2024-09-16T20:49:59.418 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5561000000000002e-05 2024-09-16T20:49:59.418 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.418 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.418 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.418 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.418 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.419 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:49:59.420 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:49:59.420 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:59.418+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:59.520 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:49:59.519+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:49:59.621 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:49:59.619+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:49:59.632 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:49:59.632 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:49:59.632 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.632 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19916 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.632 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.589049+0000", 2024-09-16T20:49:59.632 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.034563763999999997, 2024-09-16T20:49:59.633 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.034579844999999998, 2024-09-16T20:49:59.633 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.633 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.633 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.633 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.633 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:59.633 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:59.633 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19916 2024-09-16T20:49:59.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.633 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.633 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.633 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.633 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.589049+0000", 2024-09-16T20:49:59.633 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.634 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.634 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.589049+0000", 2024-09-16T20:49:59.634 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.634 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.634 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.589050+0000", 2024-09-16T20:49:59.634 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9699999999999996e-07 2024-09-16T20:49:59.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.634 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.634 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.589052+0000", 2024-09-16T20:49:59.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.452e-06 2024-09-16T20:49:59.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.635 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.589057+0000", 2024-09-16T20:49:59.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.198e-06 2024-09-16T20:49:59.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.635 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.589060+0000", 2024-09-16T20:49:59.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1240000000000001e-06 2024-09-16T20:49:59.635 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.635 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.636 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.636 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19932 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.636 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.592455+0000", 2024-09-16T20:49:59.636 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.031157905, 2024-09-16T20:49:59.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.031205248000000001, 2024-09-16T20:49:59.636 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.636 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.636 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.636 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.636 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:59.636 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:59.636 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19932 2024-09-16T20:49:59.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.637 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.637 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.637 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.592455+0000", 2024-09-16T20:49:59.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.637 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.637 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.592455+0000", 2024-09-16T20:49:59.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.638 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.638 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.592455+0000", 2024-09-16T20:49:59.638 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6000000000000004e-07 2024-09-16T20:49:59.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.638 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.638 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.638 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.592459+0000", 2024-09-16T20:49:59.638 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.225e-06 2024-09-16T20:49:59.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.638 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.638 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.638 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.592465+0000", 2024-09-16T20:49:59.638 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7440000000000001e-06 2024-09-16T20:49:59.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.639 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.639 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.592469+0000", 2024-09-16T20:49:59.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6799999999999999e-06 2024-09-16T20:49:59.639 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.639 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.639 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.639 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19948 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.639 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.595794+0000", 2024-09-16T20:49:59.639 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.027818335, 2024-09-16T20:49:59.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.027890020000000001, 2024-09-16T20:49:59.640 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.640 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.640 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.640 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.640 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:59.640 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:59.640 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19948 2024-09-16T20:49:59.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.640 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.640 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.640 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.595794+0000", 2024-09-16T20:49:59.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.595794+0000", 2024-09-16T20:49:59.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.595795+0000", 2024-09-16T20:49:59.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1900000000000003e-07 2024-09-16T20:49:59.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.642 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.642 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.595798+0000", 2024-09-16T20:49:59.642 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.813e-06 2024-09-16T20:49:59.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.642 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.642 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.595804+0000", 2024-09-16T20:49:59.642 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6989999999999997e-06 2024-09-16T20:49:59.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.642 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.642 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.595813+0000", 2024-09-16T20:49:59.642 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3639999999999999e-06 2024-09-16T20:49:59.642 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.643 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.643 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.643 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19964 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.643 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.599310+0000", 2024-09-16T20:49:59.643 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.024303202999999999, 2024-09-16T20:49:59.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.024399346999999998, 2024-09-16T20:49:59.643 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.643 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.643 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.643 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.643 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:59.643 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:59.644 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19964 2024-09-16T20:49:59.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.644 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.644 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.644 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.599310+0000", 2024-09-16T20:49:59.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.644 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.644 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.599310+0000", 2024-09-16T20:49:59.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.599311+0000", 2024-09-16T20:49:59.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7100000000000011e-07 2024-09-16T20:49:59.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.599320+0000", 2024-09-16T20:49:59.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0520000000000007e-06 2024-09-16T20:49:59.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.599329+0000", 2024-09-16T20:49:59.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0000000000000002e-06 2024-09-16T20:49:59.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.646 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.599334+0000", 2024-09-16T20:49:59.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8839999999999999e-06 2024-09-16T20:49:59.646 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.646 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.646 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.646 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24440 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.646 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.602811+0000", 2024-09-16T20:49:59.646 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.020802185000000001, 2024-09-16T20:49:59.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.020921365000000001, 2024-09-16T20:49:59.647 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.647 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.647 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.647 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.647 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:49:59.647 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:49:59.647 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24440 2024-09-16T20:49:59.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.647 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.647 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.647 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602811+0000", 2024-09-16T20:49:59.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.648 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602811+0000", 2024-09-16T20:49:59.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.648 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602811+0000", 2024-09-16T20:49:59.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8100000000000002e-07 2024-09-16T20:49:59.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602814+0000", 2024-09-16T20:49:59.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4700000000000001e-06 2024-09-16T20:49:59.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602821+0000", 2024-09-16T20:49:59.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3379999999999999e-06 2024-09-16T20:49:59.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602826+0000", 2024-09-16T20:49:59.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.464e-06 2024-09-16T20:49:59.650 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.650 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.650 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.650 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24376 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.650 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.589065+0000", 2024-09-16T20:49:59.650 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.034547504999999999, 2024-09-16T20:49:59.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.034691929000000003, 2024-09-16T20:49:59.650 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.650 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.650 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.651 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.651 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:49:59.651 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:49:59.651 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24376 2024-09-16T20:49:59.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.651 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.651 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.651 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.589065+0000", 2024-09-16T20:49:59.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.651 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.652 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.589065+0000", 2024-09-16T20:49:59.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.652 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.652 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.589066+0000", 2024-09-16T20:49:59.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.11e-06 2024-09-16T20:49:59.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.653 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.589071+0000", 2024-09-16T20:49:59.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2640000000000002e-06 2024-09-16T20:49:59.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.653 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.589081+0000", 2024-09-16T20:49:59.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9969999999999993e-06 2024-09-16T20:49:59.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.653 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.589087+0000", 2024-09-16T20:49:59.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3319999999999997e-06 2024-09-16T20:49:59.654 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.654 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.654 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.654 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24392 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.654 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.592479+0000", 2024-09-16T20:49:59.654 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.031133406999999998, 2024-09-16T20:49:59.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.031301487000000003, 2024-09-16T20:49:59.654 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.654 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.654 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.654 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.655 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:49:59.655 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:49:59.655 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24392 2024-09-16T20:49:59.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.655 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.655 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.655 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.655 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.592479+0000", 2024-09-16T20:49:59.655 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.655 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.655 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.655 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.592479+0000", 2024-09-16T20:49:59.655 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.656 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.656 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.592480+0000", 2024-09-16T20:49:59.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3399999999999998e-07 2024-09-16T20:49:59.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.656 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.656 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.592484+0000", 2024-09-16T20:49:59.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.676e-06 2024-09-16T20:49:59.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.656 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.657 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.592493+0000", 2024-09-16T20:49:59.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8289999999999997e-06 2024-09-16T20:49:59.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.657 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.657 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.592499+0000", 2024-09-16T20:49:59.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9479999999999996e-06 2024-09-16T20:49:59.657 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.657 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.657 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.657 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24408 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.658 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.595825+0000", 2024-09-16T20:49:59.658 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.027787783999999999, 2024-09-16T20:49:59.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.027979297, 2024-09-16T20:49:59.658 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.658 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.658 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.658 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.658 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:49:59.658 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:49:59.658 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24408 2024-09-16T20:49:59.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.658 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.659 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.659 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.595825+0000", 2024-09-16T20:49:59.659 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.659 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.659 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.659 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.595825+0000", 2024-09-16T20:49:59.659 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.659 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.659 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.659 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.595826+0000", 2024-09-16T20:49:59.659 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5099999999999998e-07 2024-09-16T20:49:59.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.660 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.660 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.595830+0000", 2024-09-16T20:49:59.660 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9990000000000002e-06 2024-09-16T20:49:59.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.660 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.660 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.595839+0000", 2024-09-16T20:49:59.660 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5900000000000008e-06 2024-09-16T20:49:59.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.660 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.660 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.595846+0000", 2024-09-16T20:49:59.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3390000000000004e-06 2024-09-16T20:49:59.661 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.661 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.661 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.661 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24424 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.661 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.599335+0000", 2024-09-16T20:49:59.661 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.024278132000000001, 2024-09-16T20:49:59.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.024492911999999999, 2024-09-16T20:49:59.661 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.661 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.661 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.661 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.662 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:49:59.662 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:49:59.662 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24424 2024-09-16T20:49:59.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.662 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.662 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.662 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.662 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.599335+0000", 2024-09-16T20:49:59.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.662 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.662 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.662 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.599335+0000", 2024-09-16T20:49:59.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.663 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.663 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.599335+0000", 2024-09-16T20:49:59.663 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.99e-07 2024-09-16T20:49:59.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.663 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.663 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.599338+0000", 2024-09-16T20:49:59.663 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.633e-06 2024-09-16T20:49:59.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.664 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.664 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.599344+0000", 2024-09-16T20:49:59.664 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9719999999999997e-06 2024-09-16T20:49:59.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.664 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.664 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.664 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.599347+0000", 2024-09-16T20:49:59.664 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4060000000000001e-06 2024-09-16T20:49:59.664 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.664 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.664 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.664 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.664 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19980 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.665 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.602805+0000", 2024-09-16T20:49:59.665 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.020807948, 2024-09-16T20:49:59.665 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021059426999999999, 2024-09-16T20:49:59.665 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.665 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.665 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.665 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.665 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:59.665 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:59.665 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19980 2024-09-16T20:49:59.665 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.665 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.665 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.665 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.666 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602805+0000", 2024-09-16T20:49:59.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.666 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.666 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602805+0000", 2024-09-16T20:49:59.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.666 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.666 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602806+0000", 2024-09-16T20:49:59.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0299999999999998e-07 2024-09-16T20:49:59.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.667 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.667 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602810+0000", 2024-09-16T20:49:59.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5369999999999999e-06 2024-09-16T20:49:59.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.667 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.667 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602825+0000", 2024-09-16T20:49:59.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5056e-05 2024-09-16T20:49:59.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.667 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.667 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602832+0000", 2024-09-16T20:49:59.668 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3589999999999996e-06 2024-09-16T20:49:59.668 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.668 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.668 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.668 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.668 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.668 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19917 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.668 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.589284+0000", 2024-09-16T20:49:59.668 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.034328587000000001, 2024-09-16T20:49:59.668 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.034610788000000003, 2024-09-16T20:49:59.668 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.668 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.669 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.669 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.669 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:59.669 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:59.669 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19917 2024-09-16T20:49:59.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.669 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.669 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.669 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.669 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.589284+0000", 2024-09-16T20:49:59.669 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.669 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.669 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.672 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.589284+0000", 2024-09-16T20:49:59.672 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.672 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.672 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.673 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.673 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.589285+0000", 2024-09-16T20:49:59.673 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8699999999999995e-07 2024-09-16T20:49:59.673 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.673 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.673 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.673 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.589287+0000", 2024-09-16T20:49:59.673 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4650000000000001e-06 2024-09-16T20:49:59.673 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.673 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.673 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.673 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.589292+0000", 2024-09-16T20:49:59.673 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9350000000000002e-06 2024-09-16T20:49:59.673 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.674 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.674 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.674 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.589296+0000", 2024-09-16T20:49:59.674 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7869999999999999e-06 2024-09-16T20:49:59.674 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.674 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.674 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.674 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.674 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.674 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19933 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.674 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.592638+0000", 2024-09-16T20:49:59.674 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.030974860999999999, 2024-09-16T20:49:59.674 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.031285017999999998, 2024-09-16T20:49:59.675 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.675 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.675 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.675 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.675 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:59.675 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:59.675 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19933 2024-09-16T20:49:59.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.675 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.675 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.675 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.675 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.592638+0000", 2024-09-16T20:49:59.675 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.675 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.676 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.676 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.592638+0000", 2024-09-16T20:49:59.676 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.676 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.676 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.676 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.676 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.592638+0000", 2024-09-16T20:49:59.676 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6400000000000002e-07 2024-09-16T20:49:59.676 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.676 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.676 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.676 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.592642+0000", 2024-09-16T20:49:59.676 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1870000000000001e-06 2024-09-16T20:49:59.676 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.677 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.677 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.677 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.592648+0000", 2024-09-16T20:49:59.677 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7320000000000001e-06 2024-09-16T20:49:59.677 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.677 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.677 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.677 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.592652+0000", 2024-09-16T20:49:59.677 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8449999999999996e-06 2024-09-16T20:49:59.677 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.677 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.677 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.677 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.677 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.677 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19949 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.678 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.595994+0000", 2024-09-16T20:49:59.678 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.027618838999999999, 2024-09-16T20:49:59.678 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.027951828000000001, 2024-09-16T20:49:59.678 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.678 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.678 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.678 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.678 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:59.678 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:59.678 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19949 2024-09-16T20:49:59.678 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.678 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.678 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.678 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.679 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.595994+0000", 2024-09-16T20:49:59.679 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.679 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.679 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.679 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.595994+0000", 2024-09-16T20:49:59.679 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.679 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.679 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.679 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.595994+0000", 2024-09-16T20:49:59.679 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7599999999999997e-07 2024-09-16T20:49:59.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.679 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.680 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.680 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.595998+0000", 2024-09-16T20:49:59.680 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3340000000000002e-06 2024-09-16T20:49:59.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.680 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.680 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.680 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.596006+0000", 2024-09-16T20:49:59.680 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2470000000000008e-06 2024-09-16T20:49:59.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.680 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.680 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.680 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.596011+0000", 2024-09-16T20:49:59.680 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0669999999999999e-06 2024-09-16T20:49:59.681 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.681 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.681 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.681 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.681 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.681 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19965 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.681 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.599525+0000", 2024-09-16T20:49:59.681 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.02408805, 2024-09-16T20:49:59.681 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.024444074999999999, 2024-09-16T20:49:59.681 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.681 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.681 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.681 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.681 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:59.682 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:59.682 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19965 2024-09-16T20:49:59.682 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.682 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.682 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.682 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.682 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.599525+0000", 2024-09-16T20:49:59.682 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.682 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.682 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.682 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.682 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.599525+0000", 2024-09-16T20:49:59.682 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.682 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.683 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.683 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.683 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.599525+0000", 2024-09-16T20:49:59.683 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1999999999999999e-07 2024-09-16T20:49:59.683 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.683 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.683 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.683 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.599529+0000", 2024-09-16T20:49:59.683 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.642e-06 2024-09-16T20:49:59.683 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.683 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.683 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.683 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.599539+0000", 2024-09-16T20:49:59.684 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7659999999999993e-06 2024-09-16T20:49:59.684 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.684 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.684 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.684 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.599543+0000", 2024-09-16T20:49:59.684 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0980000000000004e-06 2024-09-16T20:49:59.684 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.684 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.684 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.684 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.684 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.684 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24441 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.684 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.602981+0000", 2024-09-16T20:49:59.685 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.020631716000000001, 2024-09-16T20:49:59.685 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021010316000000001, 2024-09-16T20:49:59.685 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.685 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.685 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.685 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.685 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:49:59.685 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:49:59.685 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24441 2024-09-16T20:49:59.685 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.685 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.685 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.685 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.686 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602981+0000", 2024-09-16T20:49:59.686 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.686 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.686 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.686 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.686 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602981+0000", 2024-09-16T20:49:59.686 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.686 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.686 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.686 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.686 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602982+0000", 2024-09-16T20:49:59.686 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6599999999999996e-07 2024-09-16T20:49:59.686 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.686 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.687 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.687 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602984+0000", 2024-09-16T20:49:59.687 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6539999999999999e-06 2024-09-16T20:49:59.687 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.687 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.687 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.687 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602990+0000", 2024-09-16T20:49:59.687 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6729999999999997e-06 2024-09-16T20:49:59.687 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.687 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.687 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.687 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602993+0000", 2024-09-16T20:49:59.687 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2140000000000001e-06 2024-09-16T20:49:59.687 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.687 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.687 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.688 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.688 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.688 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24377 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.688 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.589325+0000", 2024-09-16T20:49:59.688 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.034287459999999999, 2024-09-16T20:49:59.688 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.034689148000000003, 2024-09-16T20:49:59.688 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.688 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.688 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.688 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.688 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:49:59.688 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:49:59.688 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24377 2024-09-16T20:49:59.689 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.689 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.689 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.689 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.689 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.589325+0000", 2024-09-16T20:49:59.689 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.689 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.689 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.689 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.689 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.589325+0000", 2024-09-16T20:49:59.689 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.689 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.689 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.689 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.690 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.589326+0000", 2024-09-16T20:49:59.690 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6099999999999999e-07 2024-09-16T20:49:59.690 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.690 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.690 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.690 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.589330+0000", 2024-09-16T20:49:59.690 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9419999999999997e-06 2024-09-16T20:49:59.690 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.690 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.690 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.690 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.589340+0000", 2024-09-16T20:49:59.690 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3489999999999998e-06 2024-09-16T20:49:59.690 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.690 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.690 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.691 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.589346+0000", 2024-09-16T20:49:59.691 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2419999999999997e-06 2024-09-16T20:49:59.691 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.691 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.691 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.691 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.691 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.691 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24393 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.691 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.592661+0000", 2024-09-16T20:49:59.691 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.030951354, 2024-09-16T20:49:59.691 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.031375905000000003, 2024-09-16T20:49:59.691 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.691 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.691 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.692 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.692 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:49:59.692 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:49:59.692 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24393 2024-09-16T20:49:59.692 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.692 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.692 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.692 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.692 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.592661+0000", 2024-09-16T20:49:59.692 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.692 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.692 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.692 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.692 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.592661+0000", 2024-09-16T20:49:59.692 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.693 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.693 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.693 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.693 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.592663+0000", 2024-09-16T20:49:59.693 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.305e-06 2024-09-16T20:49:59.693 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.693 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.693 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.693 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.592667+0000", 2024-09-16T20:49:59.693 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.651e-06 2024-09-16T20:49:59.693 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.693 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.693 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.693 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.592678+0000", 2024-09-16T20:49:59.693 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0455000000000001e-05 2024-09-16T20:49:59.694 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.694 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.694 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.694 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.592689+0000", 2024-09-16T20:49:59.694 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1080000000000001e-05 2024-09-16T20:49:59.694 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.694 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.694 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.694 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.694 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.694 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24409 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.694 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.596024+0000", 2024-09-16T20:49:59.694 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.027589025999999999, 2024-09-16T20:49:59.695 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.028039541000000001, 2024-09-16T20:49:59.695 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.695 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.695 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.695 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.695 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:49:59.695 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:49:59.695 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24409 2024-09-16T20:49:59.695 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.695 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.695 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.695 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.695 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.596024+0000", 2024-09-16T20:49:59.696 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.696 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.696 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.696 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.696 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.596024+0000", 2024-09-16T20:49:59.696 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.696 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.696 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.696 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.696 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.596025+0000", 2024-09-16T20:49:59.696 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9699999999999994e-07 2024-09-16T20:49:59.696 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.696 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.696 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.696 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.596029+0000", 2024-09-16T20:49:59.697 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0629999999999999e-06 2024-09-16T20:49:59.697 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.697 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.697 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.697 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.596038+0000", 2024-09-16T20:49:59.697 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.414e-06 2024-09-16T20:49:59.697 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.697 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.697 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.697 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.596045+0000", 2024-09-16T20:49:59.697 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3759999999999996e-06 2024-09-16T20:49:59.697 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.697 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.697 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.697 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.698 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.698 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24425 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.698 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.599540+0000", 2024-09-16T20:49:59.698 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.024073161999999999, 2024-09-16T20:49:59.698 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.024546386999999999, 2024-09-16T20:49:59.698 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.698 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.698 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.698 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.698 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:49:59.698 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:49:59.698 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24425 2024-09-16T20:49:59.698 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.699 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.699 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.699 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.699 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.599540+0000", 2024-09-16T20:49:59.699 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.699 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.699 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.699 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.699 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.599540+0000", 2024-09-16T20:49:59.699 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.699 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.699 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.699 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.699 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.599540+0000", 2024-09-16T20:49:59.700 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4000000000000001e-07 2024-09-16T20:49:59.700 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.700 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.700 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.700 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.599543+0000", 2024-09-16T20:49:59.700 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.491e-06 2024-09-16T20:49:59.700 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.700 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.700 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.700 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.599552+0000", 2024-09-16T20:49:59.700 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7490000000000004e-06 2024-09-16T20:49:59.700 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.700 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.701 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.701 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.599555+0000", 2024-09-16T20:49:59.701 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1719999999999999e-06 2024-09-16T20:49:59.701 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.701 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.701 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.701 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.701 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.701 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19981 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.701 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.602985+0000", 2024-09-16T20:49:59.701 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.020628054, 2024-09-16T20:49:59.701 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021126131999999999, 2024-09-16T20:49:59.702 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.702 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.702 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.702 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.702 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:59.702 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:59.702 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19981 2024-09-16T20:49:59.702 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.702 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.702 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.702 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.702 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602985+0000", 2024-09-16T20:49:59.702 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.702 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.703 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.703 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.703 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602985+0000", 2024-09-16T20:49:59.703 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.703 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.703 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.703 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.703 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602986+0000", 2024-09-16T20:49:59.703 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6099999999999999e-07 2024-09-16T20:49:59.703 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.703 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.703 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.703 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602989+0000", 2024-09-16T20:49:59.703 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6940000000000001e-06 2024-09-16T20:49:59.704 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.704 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.704 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.704 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602997+0000", 2024-09-16T20:49:59.704 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9400000000000002e-06 2024-09-16T20:49:59.704 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.704 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.704 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.704 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.603002+0000", 2024-09-16T20:49:59.704 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6489999999999999e-06 2024-09-16T20:49:59.704 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.704 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.704 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.704 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.705 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.705 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19918 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.705 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.589493+0000", 2024-09-16T20:49:59.705 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.034120181999999999, 2024-09-16T20:49:59.705 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.034641390000000001, 2024-09-16T20:49:59.705 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.705 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.705 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.705 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.705 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:59.705 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:59.705 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19918 2024-09-16T20:49:59.705 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.706 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.706 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.706 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.706 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.589493+0000", 2024-09-16T20:49:59.706 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.706 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.706 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.706 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.706 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.589493+0000", 2024-09-16T20:49:59.706 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.706 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.706 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.706 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.706 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.589493+0000", 2024-09-16T20:49:59.706 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4999999999999998e-07 2024-09-16T20:49:59.707 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.707 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.707 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.707 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.589496+0000", 2024-09-16T20:49:59.707 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5229999999999998e-06 2024-09-16T20:49:59.707 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.707 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.707 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.707 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.589501+0000", 2024-09-16T20:49:59.707 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8409999999999996e-06 2024-09-16T20:49:59.707 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.707 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.707 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.707 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.589505+0000", 2024-09-16T20:49:59.707 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6119999999999999e-06 2024-09-16T20:49:59.708 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.708 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.708 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.708 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.708 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.708 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19934 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.708 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.592864+0000", 2024-09-16T20:49:59.708 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.030749087000000001, 2024-09-16T20:49:59.708 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.031294952000000001, 2024-09-16T20:49:59.708 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.708 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.708 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.709 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.709 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:59.709 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:59.709 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19934 2024-09-16T20:49:59.709 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.709 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.709 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.709 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.709 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.592864+0000", 2024-09-16T20:49:59.709 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.709 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.709 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.710 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.710 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.592864+0000", 2024-09-16T20:49:59.710 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.710 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.710 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.710 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.710 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.592864+0000", 2024-09-16T20:49:59.710 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1200000000000003e-07 2024-09-16T20:49:59.710 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.710 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.710 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.710 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.592867+0000", 2024-09-16T20:49:59.710 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4980000000000001e-06 2024-09-16T20:49:59.710 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.711 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.711 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.711 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.592873+0000", 2024-09-16T20:49:59.711 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9039999999999997e-06 2024-09-16T20:49:59.711 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.711 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.711 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.711 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.592876+0000", 2024-09-16T20:49:59.711 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.529e-06 2024-09-16T20:49:59.711 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.711 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.711 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.711 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.712 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.712 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19950 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.712 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.596220+0000", 2024-09-16T20:49:59.712 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.027392768000000001, 2024-09-16T20:49:59.712 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.027973321999999998, 2024-09-16T20:49:59.712 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.712 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.712 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.712 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.712 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:59.712 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:59.712 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19950 2024-09-16T20:49:59.713 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.713 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.713 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.713 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.713 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.596220+0000", 2024-09-16T20:49:59.713 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.713 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.713 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.713 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.713 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.596220+0000", 2024-09-16T20:49:59.713 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.713 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.713 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.714 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.714 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.596220+0000", 2024-09-16T20:49:59.714 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9100000000000004e-07 2024-09-16T20:49:59.714 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.714 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.714 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.714 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.596224+0000", 2024-09-16T20:49:59.714 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.252e-06 2024-09-16T20:49:59.714 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.714 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.714 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.715 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.596234+0000", 2024-09-16T20:49:59.715 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0566e-05 2024-09-16T20:49:59.715 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.715 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.715 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.715 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.596241+0000", 2024-09-16T20:49:59.715 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.754e-06 2024-09-16T20:49:59.715 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.715 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.715 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.715 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.715 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.715 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19966 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.716 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.599780+0000", 2024-09-16T20:49:59.716 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.023832823999999999, 2024-09-16T20:49:59.716 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.024437542, 2024-09-16T20:49:59.716 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.716 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.716 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.716 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.716 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:59.716 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:59.716 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19966 2024-09-16T20:49:59.716 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.717 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.717 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.717 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.717 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.599780+0000", 2024-09-16T20:49:59.717 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.717 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.717 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.717 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.717 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.599780+0000", 2024-09-16T20:49:59.717 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.717 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.717 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.718 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.718 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.599781+0000", 2024-09-16T20:49:59.718 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0999999999999997e-07 2024-09-16T20:49:59.718 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.718 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.718 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.718 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.599784+0000", 2024-09-16T20:49:59.718 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9780000000000001e-06 2024-09-16T20:49:59.718 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.718 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.718 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.719 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.599793+0000", 2024-09-16T20:49:59.719 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1109999999999997e-06 2024-09-16T20:49:59.719 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.719 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.719 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.719 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.599800+0000", 2024-09-16T20:49:59.719 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2409999999999998e-06 2024-09-16T20:49:59.719 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.719 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.719 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.719 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.720 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.720 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24442 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.720 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.603121+0000", 2024-09-16T20:49:59.720 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.020491307, 2024-09-16T20:49:59.720 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021119484000000001, 2024-09-16T20:49:59.720 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.720 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.720 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.720 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.720 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:49:59.720 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:49:59.720 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24442 2024-09-16T20:49:59.720 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.720 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.721 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.721 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.721 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.603121+0000", 2024-09-16T20:49:59.721 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.721 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.721 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.721 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.721 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.603121+0000", 2024-09-16T20:49:59.721 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.721 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.721 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.721 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.721 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.603122+0000", 2024-09-16T20:49:59.721 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0699999999999996e-07 2024-09-16T20:49:59.721 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.722 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.722 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.722 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.603125+0000", 2024-09-16T20:49:59.722 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5000000000000002e-06 2024-09-16T20:49:59.722 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.722 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.722 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.722 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.603131+0000", 2024-09-16T20:49:59.722 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7499999999999997e-06 2024-09-16T20:49:59.722 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.722 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.722 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.722 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.603135+0000", 2024-09-16T20:49:59.722 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3129999999999999e-06 2024-09-16T20:49:59.722 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.723 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.723 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.723 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.723 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.723 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24378 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.723 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.589515+0000", 2024-09-16T20:49:59.723 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.034097308, 2024-09-16T20:49:59.723 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.034748631000000002, 2024-09-16T20:49:59.723 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.723 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.723 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.723 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.723 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:49:59.723 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:49:59.723 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24378 2024-09-16T20:49:59.724 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.724 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.724 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.724 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.724 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.589515+0000", 2024-09-16T20:49:59.724 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.724 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.724 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.724 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.724 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.589515+0000", 2024-09-16T20:49:59.724 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.724 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.724 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.724 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.725 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.589516+0000", 2024-09-16T20:49:59.725 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9699999999999994e-07 2024-09-16T20:49:59.725 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.725 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.725 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.725 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.589521+0000", 2024-09-16T20:49:59.725 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0749999999999999e-06 2024-09-16T20:49:59.725 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.725 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.725 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.725 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.589529+0000", 2024-09-16T20:49:59.725 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9360000000000005e-06 2024-09-16T20:49:59.725 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.725 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.725 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.726 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.589536+0000", 2024-09-16T20:49:59.726 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3289999999999999e-06 2024-09-16T20:49:59.726 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.726 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.726 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.726 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.726 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.726 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24394 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.726 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.592894+0000", 2024-09-16T20:49:59.726 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.030718437000000001, 2024-09-16T20:49:59.726 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.031392849, 2024-09-16T20:49:59.726 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.726 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.726 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.726 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.727 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:49:59.727 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:49:59.727 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24394 2024-09-16T20:49:59.727 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.727 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.727 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.727 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.727 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.592894+0000", 2024-09-16T20:49:59.727 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.727 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.727 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.727 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.727 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.592894+0000", 2024-09-16T20:49:59.727 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.728 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.728 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.728 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.728 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.592895+0000", 2024-09-16T20:49:59.728 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.186e-06 2024-09-16T20:49:59.728 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.728 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.728 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.728 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.592900+0000", 2024-09-16T20:49:59.728 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0339999999999998e-06 2024-09-16T20:49:59.728 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.728 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.728 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.728 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.592910+0000", 2024-09-16T20:49:59.729 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0444e-05 2024-09-16T20:49:59.729 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.729 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.729 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.729 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.592916+0000", 2024-09-16T20:49:59.729 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3489999999999997e-06 2024-09-16T20:49:59.729 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.729 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.729 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.729 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.729 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.729 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24410 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.729 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.596246+0000", 2024-09-16T20:49:59.729 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.027366661, 2024-09-16T20:49:59.730 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.028071058999999999, 2024-09-16T20:49:59.730 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.730 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.730 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.730 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.730 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:49:59.730 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:49:59.730 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24410 2024-09-16T20:49:59.730 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.730 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.730 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.730 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.730 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.596246+0000", 2024-09-16T20:49:59.730 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.730 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.731 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.731 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.731 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.596246+0000", 2024-09-16T20:49:59.731 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.731 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.731 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.731 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.731 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.596247+0000", 2024-09-16T20:49:59.731 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8100000000000001e-07 2024-09-16T20:49:59.731 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.731 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.731 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.731 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.596251+0000", 2024-09-16T20:49:59.732 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0489999999999997e-06 2024-09-16T20:49:59.732 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.732 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.732 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.732 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.596261+0000", 2024-09-16T20:49:59.732 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0232e-05 2024-09-16T20:49:59.732 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.732 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.732 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.732 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.596268+0000", 2024-09-16T20:49:59.732 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3210000000000001e-06 2024-09-16T20:49:59.732 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.732 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.732 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.733 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.733 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.733 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24426 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.733 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.599796+0000", 2024-09-16T20:49:59.733 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.023817181999999999, 2024-09-16T20:49:59.733 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.024548337, 2024-09-16T20:49:59.733 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.733 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.733 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.733 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.733 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:49:59.733 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:49:59.733 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24426 2024-09-16T20:49:59.733 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.733 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.734 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.734 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.734 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.599796+0000", 2024-09-16T20:49:59.734 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.734 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.734 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.734 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.734 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.599796+0000", 2024-09-16T20:49:59.734 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.734 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.734 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.734 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.734 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.599796+0000", 2024-09-16T20:49:59.735 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2699999999999999e-07 2024-09-16T20:49:59.735 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.735 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.735 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.735 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.599799+0000", 2024-09-16T20:49:59.735 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5519999999999999e-06 2024-09-16T20:49:59.735 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.735 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.735 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.735 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.599804+0000", 2024-09-16T20:49:59.735 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0220000000000004e-06 2024-09-16T20:49:59.735 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.735 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.735 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.735 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.599807+0000", 2024-09-16T20:49:59.736 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1729999999999999e-06 2024-09-16T20:49:59.736 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.736 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.736 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.736 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.736 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.736 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19982 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.736 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.603116+0000", 2024-09-16T20:49:59.736 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.020496398999999998, 2024-09-16T20:49:59.736 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021250163999999998, 2024-09-16T20:49:59.736 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.736 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.736 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.737 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.737 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:59.737 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:59.737 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19982 2024-09-16T20:49:59.737 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.737 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.737 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.737 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.737 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.603116+0000", 2024-09-16T20:49:59.737 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.737 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.737 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.737 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.737 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.603116+0000", 2024-09-16T20:49:59.738 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.738 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.738 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.738 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.738 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.603117+0000", 2024-09-16T20:49:59.738 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9699999999999995e-07 2024-09-16T20:49:59.738 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.738 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.738 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.738 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.603122+0000", 2024-09-16T20:49:59.738 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.437e-06 2024-09-16T20:49:59.738 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.738 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.738 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.738 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.603134+0000", 2024-09-16T20:49:59.739 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2333e-05 2024-09-16T20:49:59.739 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.739 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.739 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.739 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.603140+0000", 2024-09-16T20:49:59.739 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0859999999999998e-06 2024-09-16T20:49:59.739 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.739 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.739 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.739 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.739 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.739 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19919 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.739 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.589713+0000", 2024-09-16T20:49:59.739 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.033899563000000001, 2024-09-16T20:49:59.739 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.034675936999999997, 2024-09-16T20:49:59.740 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.740 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.740 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.740 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.740 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:59.740 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:59.740 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19919 2024-09-16T20:49:59.740 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.740 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.740 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.740 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.740 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.589713+0000", 2024-09-16T20:49:59.740 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.740 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.741 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.741 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.741 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.589713+0000", 2024-09-16T20:49:59.741 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.741 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.741 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.741 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.741 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.589714+0000", 2024-09-16T20:49:59.741 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5999999999999999e-07 2024-09-16T20:49:59.741 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.741 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.741 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.741 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.589716+0000", 2024-09-16T20:49:59.741 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4710000000000001e-06 2024-09-16T20:49:59.742 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.742 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.742 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.742 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.589721+0000", 2024-09-16T20:49:59.742 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3009999999999997e-06 2024-09-16T20:49:59.742 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.742 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.742 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.742 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.589725+0000", 2024-09-16T20:49:59.742 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3629999999999998e-06 2024-09-16T20:49:59.742 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.742 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.742 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.742 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.742 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.743 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19935 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.743 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.593036+0000", 2024-09-16T20:49:59.743 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.030576438000000001, 2024-09-16T20:49:59.743 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.03137587, 2024-09-16T20:49:59.743 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.743 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.743 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.743 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.743 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:59.743 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:59.743 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19935 2024-09-16T20:49:59.743 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.743 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.743 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.744 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.744 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.593036+0000", 2024-09-16T20:49:59.744 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.744 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.744 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.744 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.744 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.593036+0000", 2024-09-16T20:49:59.744 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.744 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.744 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.744 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.744 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.593037+0000", 2024-09-16T20:49:59.744 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5400000000000001e-07 2024-09-16T20:49:59.744 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.745 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.745 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.745 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.593039+0000", 2024-09-16T20:49:59.745 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4279999999999999e-06 2024-09-16T20:49:59.745 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.745 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.745 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.745 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.593044+0000", 2024-09-16T20:49:59.745 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0379999999999999e-06 2024-09-16T20:49:59.745 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.745 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.745 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.745 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.593048+0000", 2024-09-16T20:49:59.745 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2799999999999999e-06 2024-09-16T20:49:59.746 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.746 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.746 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.746 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.746 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.746 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19951 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.746 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.596420+0000", 2024-09-16T20:49:59.746 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.027192366999999999, 2024-09-16T20:49:59.746 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.028014853999999999, 2024-09-16T20:49:59.746 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.746 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.746 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.746 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.746 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:59.746 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:59.747 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19951 2024-09-16T20:49:59.747 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.747 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.747 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.747 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.747 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.596420+0000", 2024-09-16T20:49:59.747 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.747 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.747 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.747 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.747 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.596420+0000", 2024-09-16T20:49:59.747 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.747 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.747 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.747 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.748 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.596421+0000", 2024-09-16T20:49:59.748 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8599999999999998e-07 2024-09-16T20:49:59.748 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.748 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.748 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.748 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.596425+0000", 2024-09-16T20:49:59.748 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5489999999999998e-06 2024-09-16T20:49:59.748 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.748 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.748 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.748 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.596431+0000", 2024-09-16T20:49:59.748 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9160000000000004e-06 2024-09-16T20:49:59.748 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.749 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.749 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.749 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.596436+0000", 2024-09-16T20:49:59.749 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2150000000000003e-06 2024-09-16T20:49:59.749 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.749 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.749 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.749 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.749 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.749 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19967 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.749 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.599989+0000", 2024-09-16T20:49:59.749 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.023623495000000001, 2024-09-16T20:49:59.749 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.024469709999999999, 2024-09-16T20:49:59.750 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.750 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.750 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.750 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.750 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:59.750 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:59.750 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19967 2024-09-16T20:49:59.750 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.750 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.750 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.750 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.750 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.599989+0000", 2024-09-16T20:49:59.750 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.750 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.751 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.751 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.751 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.599989+0000", 2024-09-16T20:49:59.751 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.751 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.751 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.751 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.751 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.599990+0000", 2024-09-16T20:49:59.751 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1099999999999995e-07 2024-09-16T20:49:59.751 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.751 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.751 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.751 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.599994+0000", 2024-09-16T20:49:59.751 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6109999999999998e-06 2024-09-16T20:49:59.751 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.752 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.752 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.752 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.600005+0000", 2024-09-16T20:49:59.752 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1234e-05 2024-09-16T20:49:59.752 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.752 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.752 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.752 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.600011+0000", 2024-09-16T20:49:59.752 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4729999999999997e-06 2024-09-16T20:49:59.752 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.752 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.752 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.752 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.753 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.753 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19983 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.753 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.603281+0000", 2024-09-16T20:49:59.753 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.020331561000000001, 2024-09-16T20:49:59.753 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021217119999999999, 2024-09-16T20:49:59.753 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.753 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.753 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.753 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.754 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:59.754 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:59.754 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19983 2024-09-16T20:49:59.754 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.754 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.754 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.754 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.754 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.603281+0000", 2024-09-16T20:49:59.754 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.754 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.754 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.754 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.754 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.603281+0000", 2024-09-16T20:49:59.754 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.754 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.755 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.755 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.755 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.603282+0000", 2024-09-16T20:49:59.755 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3399999999999997e-07 2024-09-16T20:49:59.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.755 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.755 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.755 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.603287+0000", 2024-09-16T20:49:59.755 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4959999999999998e-06 2024-09-16T20:49:59.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.755 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.755 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.755 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.603296+0000", 2024-09-16T20:49:59.756 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6180000000000001e-06 2024-09-16T20:49:59.756 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.756 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.756 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.756 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.603301+0000", 2024-09-16T20:49:59.756 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5800000000000002e-06 2024-09-16T20:49:59.756 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.756 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.756 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.756 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.756 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.756 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24379 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.756 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.589756+0000", 2024-09-16T20:49:59.756 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.033856904, 2024-09-16T20:49:59.757 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.034772178000000001, 2024-09-16T20:49:59.757 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.757 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.757 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.757 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.757 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:49:59.757 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:49:59.757 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24379 2024-09-16T20:49:59.757 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.757 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.757 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.757 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.757 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.589756+0000", 2024-09-16T20:49:59.757 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.758 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.758 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.758 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.758 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.589756+0000", 2024-09-16T20:49:59.758 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.758 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.758 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.758 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.758 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.589757+0000", 2024-09-16T20:49:59.758 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4099999999999997e-07 2024-09-16T20:49:59.758 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.758 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.758 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.759 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.589761+0000", 2024-09-16T20:49:59.759 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0820000000000001e-06 2024-09-16T20:49:59.759 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.759 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.759 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.759 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.589770+0000", 2024-09-16T20:49:59.759 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6980000000000004e-06 2024-09-16T20:49:59.759 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.759 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.759 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.759 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.589776+0000", 2024-09-16T20:49:59.759 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.241e-06 2024-09-16T20:49:59.759 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.759 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.759 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.760 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.760 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.760 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24395 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.760 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.593058+0000", 2024-09-16T20:49:59.760 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.030554907999999999, 2024-09-16T20:49:59.760 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.031493298000000003, 2024-09-16T20:49:59.760 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.760 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.760 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.760 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.760 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:49:59.760 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:49:59.760 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24395 2024-09-16T20:49:59.760 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.760 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.761 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.761 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.761 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.593058+0000", 2024-09-16T20:49:59.761 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.761 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.761 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.761 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.761 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.593058+0000", 2024-09-16T20:49:59.761 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.761 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.761 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.761 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.761 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.593059+0000", 2024-09-16T20:49:59.761 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9399999999999993e-07 2024-09-16T20:49:59.762 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.762 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.762 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.762 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.593063+0000", 2024-09-16T20:49:59.762 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.031e-06 2024-09-16T20:49:59.762 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.762 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.762 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.762 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.593071+0000", 2024-09-16T20:49:59.762 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4470000000000006e-06 2024-09-16T20:49:59.762 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.762 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.762 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.762 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.593077+0000", 2024-09-16T20:49:59.763 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5099999999999998e-06 2024-09-16T20:49:59.763 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.763 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.763 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.763 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.763 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.763 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24411 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.763 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.596448+0000", 2024-09-16T20:49:59.763 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.027164357, 2024-09-16T20:49:59.763 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.028130709, 2024-09-16T20:49:59.763 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.763 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.763 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.763 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.764 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:49:59.764 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:49:59.764 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24411 2024-09-16T20:49:59.764 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.764 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.764 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.764 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.764 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.596448+0000", 2024-09-16T20:49:59.764 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.764 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.764 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.764 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.764 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.596448+0000", 2024-09-16T20:49:59.764 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.765 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.765 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.765 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.765 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.596449+0000", 2024-09-16T20:49:59.765 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7699999999999992e-07 2024-09-16T20:49:59.765 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.765 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.765 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.765 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.596454+0000", 2024-09-16T20:49:59.765 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3869999999999997e-06 2024-09-16T20:49:59.765 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.765 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.765 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.765 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.596463+0000", 2024-09-16T20:49:59.766 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2129999999999997e-06 2024-09-16T20:49:59.766 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.766 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.766 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.766 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.596469+0000", 2024-09-16T20:49:59.766 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2910000000000004e-06 2024-09-16T20:49:59.766 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.766 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.766 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.766 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.766 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.766 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24427 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.766 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.600006+0000", 2024-09-16T20:49:59.766 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.023606477000000001, 2024-09-16T20:49:59.767 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.024597272, 2024-09-16T20:49:59.767 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.767 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.767 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.767 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.767 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:49:59.767 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:49:59.767 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24427 2024-09-16T20:49:59.767 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.767 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.767 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.767 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.767 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.600006+0000", 2024-09-16T20:49:59.767 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.767 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.768 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.768 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.768 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.600006+0000", 2024-09-16T20:49:59.768 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.768 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.768 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.768 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.768 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.600007+0000", 2024-09-16T20:49:59.768 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2900000000000003e-07 2024-09-16T20:49:59.768 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.768 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.768 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.769 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.600010+0000", 2024-09-16T20:49:59.769 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5220000000000002e-06 2024-09-16T20:49:59.769 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.769 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.769 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.769 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.600016+0000", 2024-09-16T20:49:59.769 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2620000000000004e-06 2024-09-16T20:49:59.769 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.769 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.769 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.769 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.600019+0000", 2024-09-16T20:49:59.769 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.388e-06 2024-09-16T20:49:59.770 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.770 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.770 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.770 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.770 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.770 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24443 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.770 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.603288+0000", 2024-09-16T20:49:59.770 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.02032463, 2024-09-16T20:49:59.770 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021340015, 2024-09-16T20:49:59.770 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.770 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.771 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.771 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.771 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:49:59.771 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:49:59.771 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24443 2024-09-16T20:49:59.771 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.771 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.771 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.771 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.771 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.603288+0000", 2024-09-16T20:49:59.771 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.771 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.771 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.771 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.772 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.603288+0000", 2024-09-16T20:49:59.772 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.772 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.772 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.772 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.772 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.603289+0000", 2024-09-16T20:49:59.772 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1699999999999997e-07 2024-09-16T20:49:59.772 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.772 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.772 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.772 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.603291+0000", 2024-09-16T20:49:59.772 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.525e-06 2024-09-16T20:49:59.772 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.772 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.773 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.773 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.603297+0000", 2024-09-16T20:49:59.773 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6210000000000001e-06 2024-09-16T20:49:59.773 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.773 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.773 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.773 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.603301+0000", 2024-09-16T20:49:59.773 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5040000000000004e-06 2024-09-16T20:49:59.773 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.773 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.773 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.773 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.773 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.773 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19920 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.774 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.589985+0000", 2024-09-16T20:49:59.774 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.033627440000000001, 2024-09-16T20:49:59.774 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.034666052000000003, 2024-09-16T20:49:59.774 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.774 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.774 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.774 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.774 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:59.774 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:59.774 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19920 2024-09-16T20:49:59.774 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.774 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.774 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.774 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.775 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.589985+0000", 2024-09-16T20:49:59.775 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.775 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.775 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.775 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.775 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.589985+0000", 2024-09-16T20:49:59.775 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.775 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.775 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.775 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.775 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.589986+0000", 2024-09-16T20:49:59.775 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9699999999999995e-07 2024-09-16T20:49:59.775 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.775 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.776 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.776 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.589989+0000", 2024-09-16T20:49:59.776 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3110000000000001e-06 2024-09-16T20:49:59.776 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.776 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.776 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.776 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.589997+0000", 2024-09-16T20:49:59.776 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2819999999999999e-06 2024-09-16T20:49:59.776 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.776 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.776 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.776 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590002+0000", 2024-09-16T20:49:59.776 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4739999999999996e-06 2024-09-16T20:49:59.776 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.777 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.777 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.777 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.777 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.777 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19936 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.777 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.593254+0000", 2024-09-16T20:49:59.777 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.03035914, 2024-09-16T20:49:59.777 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.031420873000000002, 2024-09-16T20:49:59.777 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.777 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.777 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.777 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.777 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:59.777 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:59.778 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19936 2024-09-16T20:49:59.778 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.778 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.778 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.778 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.778 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.593254+0000", 2024-09-16T20:49:59.778 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.778 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.778 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.778 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.778 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.593254+0000", 2024-09-16T20:49:59.778 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.778 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.779 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.779 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.779 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.593254+0000", 2024-09-16T20:49:59.779 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3500000000000002e-07 2024-09-16T20:49:59.779 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.779 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.779 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.779 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.593257+0000", 2024-09-16T20:49:59.779 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5339999999999998e-06 2024-09-16T20:49:59.779 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.779 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.779 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.779 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.593262+0000", 2024-09-16T20:49:59.780 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7239999999999997e-06 2024-09-16T20:49:59.780 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.780 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.780 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.780 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.593266+0000", 2024-09-16T20:49:59.780 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.348e-06 2024-09-16T20:49:59.780 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.780 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.780 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.780 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.780 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.780 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19952 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.780 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.596647+0000", 2024-09-16T20:49:59.781 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.026966173999999999, 2024-09-16T20:49:59.781 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.028050875999999999, 2024-09-16T20:49:59.781 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.781 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.781 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.781 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.781 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:59.781 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:59.781 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19952 2024-09-16T20:49:59.781 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.781 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.781 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.781 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.781 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.596647+0000", 2024-09-16T20:49:59.781 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.782 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.782 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.782 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.596647+0000", 2024-09-16T20:49:59.782 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.782 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.782 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.782 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.596647+0000", 2024-09-16T20:49:59.782 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4099999999999998e-07 2024-09-16T20:49:59.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.782 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.782 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.782 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.596650+0000", 2024-09-16T20:49:59.783 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2380000000000002e-06 2024-09-16T20:49:59.783 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.783 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.783 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.783 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.596657+0000", 2024-09-16T20:49:59.783 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1419999999999998e-06 2024-09-16T20:49:59.783 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.783 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.783 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.783 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.596663+0000", 2024-09-16T20:49:59.783 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9340000000000003e-06 2024-09-16T20:49:59.784 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.784 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.784 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.784 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.784 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.784 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19968 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.784 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.600233+0000", 2024-09-16T20:49:59.784 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.023379838999999999, 2024-09-16T20:49:59.784 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.024487463000000001, 2024-09-16T20:49:59.784 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.784 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.784 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.785 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.785 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:59.785 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:59.785 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19968 2024-09-16T20:49:59.785 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.785 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.785 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.785 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.785 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.600233+0000", 2024-09-16T20:49:59.785 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.785 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.785 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.785 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.785 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.600233+0000", 2024-09-16T20:49:59.786 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.786 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.786 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.600234+0000", 2024-09-16T20:49:59.786 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7899999999999997e-07 2024-09-16T20:49:59.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.786 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.786 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.600238+0000", 2024-09-16T20:49:59.786 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9310000000000001e-06 2024-09-16T20:49:59.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.786 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.787 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.600247+0000", 2024-09-16T20:49:59.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4809999999999995e-06 2024-09-16T20:49:59.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.787 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.787 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.787 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.600255+0000", 2024-09-16T20:49:59.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1980000000000001e-06 2024-09-16T20:49:59.787 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.787 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.787 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.787 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.787 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19984 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.787 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.603534+0000", 2024-09-16T20:49:59.787 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.020078380999999999, 2024-09-16T20:49:59.788 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021208998999999999, 2024-09-16T20:49:59.788 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.788 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.788 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.788 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.788 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:59.788 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:59.788 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19984 2024-09-16T20:49:59.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.788 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.788 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.788 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.788 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.603534+0000", 2024-09-16T20:49:59.789 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.789 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.789 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.789 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.603534+0000", 2024-09-16T20:49:59.789 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.789 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.789 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.789 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.603535+0000", 2024-09-16T20:49:59.789 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8599999999999996e-07 2024-09-16T20:49:59.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.789 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.789 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.790 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.603540+0000", 2024-09-16T20:49:59.790 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1699999999999999e-06 2024-09-16T20:49:59.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.790 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.790 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.790 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.603560+0000", 2024-09-16T20:49:59.790 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0134999999999999e-05 2024-09-16T20:49:59.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.790 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.790 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.790 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.603567+0000", 2024-09-16T20:49:59.790 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8680000000000001e-06 2024-09-16T20:49:59.790 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.790 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.791 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.791 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.791 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24380 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.791 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.590003+0000", 2024-09-16T20:49:59.791 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.033609361999999997, 2024-09-16T20:49:59.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.034763250000000002, 2024-09-16T20:49:59.791 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.791 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.791 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.791 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.791 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:49:59.792 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:49:59.792 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24380 2024-09-16T20:49:59.792 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.792 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.792 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.792 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.792 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590003+0000", 2024-09-16T20:49:59.792 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.792 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.792 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.792 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.792 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590003+0000", 2024-09-16T20:49:59.792 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.792 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.793 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.793 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.793 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590004+0000", 2024-09-16T20:49:59.793 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.61e-07 2024-09-16T20:49:59.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.793 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.793 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.793 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590008+0000", 2024-09-16T20:49:59.793 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7139999999999999e-06 2024-09-16T20:49:59.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.793 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.793 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.793 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590016+0000", 2024-09-16T20:49:59.794 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5760000000000006e-06 2024-09-16T20:49:59.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.794 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.794 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590022+0000", 2024-09-16T20:49:59.794 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1739999999999999e-06 2024-09-16T20:49:59.794 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.794 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.794 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.794 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24396 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.794 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.593282+0000", 2024-09-16T20:49:59.795 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.030330781000000001, 2024-09-16T20:49:59.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.031510783000000001, 2024-09-16T20:49:59.795 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.795 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.795 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.795 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.795 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:49:59.795 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:49:59.795 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24396 2024-09-16T20:49:59.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.795 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.795 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.795 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.796 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.593282+0000", 2024-09-16T20:49:59.796 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.796 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.796 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.796 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.593282+0000", 2024-09-16T20:49:59.796 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.796 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.796 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.796 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.593283+0000", 2024-09-16T20:49:59.796 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5999999999999991e-07 2024-09-16T20:49:59.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.797 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.797 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.797 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.593287+0000", 2024-09-16T20:49:59.797 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0029999999999996e-06 2024-09-16T20:49:59.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.797 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.797 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.797 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.593296+0000", 2024-09-16T20:49:59.797 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2140000000000002e-06 2024-09-16T20:49:59.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.797 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.797 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.797 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.593303+0000", 2024-09-16T20:49:59.798 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.584e-06 2024-09-16T20:49:59.798 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.798 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.798 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.798 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.798 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24412 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.798 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.596681+0000", 2024-09-16T20:49:59.798 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.02693166, 2024-09-16T20:49:59.798 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.028145743000000001, 2024-09-16T20:49:59.798 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.798 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.798 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.799 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.799 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:49:59.799 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:49:59.799 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24412 2024-09-16T20:49:59.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.799 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.799 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.799 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.799 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.596681+0000", 2024-09-16T20:49:59.799 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.800 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.800 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.596681+0000", 2024-09-16T20:49:59.800 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.800 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.800 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.596682+0000", 2024-09-16T20:49:59.800 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6899999999999996e-07 2024-09-16T20:49:59.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.800 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.800 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.596686+0000", 2024-09-16T20:49:59.801 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1779999999999996e-06 2024-09-16T20:49:59.801 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.801 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.801 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.801 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.596695+0000", 2024-09-16T20:49:59.801 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.217e-06 2024-09-16T20:49:59.801 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.801 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.801 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.801 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.596702+0000", 2024-09-16T20:49:59.801 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2140000000000001e-06 2024-09-16T20:49:59.801 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.802 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.802 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.802 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.802 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.802 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24428 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.802 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.600251+0000", 2024-09-16T20:49:59.802 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.023361478000000001, 2024-09-16T20:49:59.802 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.024599314000000001, 2024-09-16T20:49:59.802 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.802 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.802 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.802 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.803 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:49:59.803 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:49:59.803 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24428 2024-09-16T20:49:59.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.803 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.803 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.803 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.803 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.600251+0000", 2024-09-16T20:49:59.803 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.803 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.803 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.803 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.600251+0000", 2024-09-16T20:49:59.803 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.804 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.804 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.600252+0000", 2024-09-16T20:49:59.804 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3e-07 2024-09-16T20:49:59.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.804 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.804 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.600255+0000", 2024-09-16T20:49:59.804 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7089999999999998e-06 2024-09-16T20:49:59.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.804 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.804 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.600260+0000", 2024-09-16T20:49:59.804 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6829999999999996e-06 2024-09-16T20:49:59.805 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.805 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.805 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.805 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.600264+0000", 2024-09-16T20:49:59.805 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4939999999999999e-06 2024-09-16T20:49:59.805 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.805 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.805 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.805 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.805 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.805 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24444 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.805 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.603557+0000", 2024-09-16T20:49:59.806 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.020055918999999998, 2024-09-16T20:49:59.806 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021316655, 2024-09-16T20:49:59.806 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.806 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.806 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.806 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.806 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:49:59.806 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:49:59.806 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24444 2024-09-16T20:49:59.806 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.806 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.806 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.806 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.806 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.603557+0000", 2024-09-16T20:49:59.807 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.807 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.807 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.807 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.807 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.603557+0000", 2024-09-16T20:49:59.807 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.807 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.807 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.807 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.807 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.603557+0000", 2024-09-16T20:49:59.807 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4700000000000001e-07 2024-09-16T20:49:59.807 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.807 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.807 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.808 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.603560+0000", 2024-09-16T20:49:59.808 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8490000000000002e-06 2024-09-16T20:49:59.808 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.808 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.808 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.808 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.603566+0000", 2024-09-16T20:49:59.808 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3210000000000004e-06 2024-09-16T20:49:59.808 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.808 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.808 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.808 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.603569+0000", 2024-09-16T20:49:59.808 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.608e-06 2024-09-16T20:49:59.808 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.808 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.809 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.809 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.809 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19921 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.809 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.590152+0000", 2024-09-16T20:49:59.809 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.033461142999999999, 2024-09-16T20:49:59.809 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.034745218000000001, 2024-09-16T20:49:59.809 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.809 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.809 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.809 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.809 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:59.809 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:59.810 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19921 2024-09-16T20:49:59.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.810 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.810 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.810 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.810 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590152+0000", 2024-09-16T20:49:59.810 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.810 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.810 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.810 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590152+0000", 2024-09-16T20:49:59.810 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.810 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.810 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.811 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590152+0000", 2024-09-16T20:49:59.811 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1100000000000005e-07 2024-09-16T20:49:59.811 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.811 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.811 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.811 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590157+0000", 2024-09-16T20:49:59.811 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1969999999999998e-06 2024-09-16T20:49:59.811 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.811 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.811 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.811 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590167+0000", 2024-09-16T20:49:59.811 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0427e-05 2024-09-16T20:49:59.811 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.812 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.812 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.812 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590174+0000", 2024-09-16T20:49:59.812 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9709999999999998e-06 2024-09-16T20:49:59.812 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.812 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.812 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.812 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.812 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.812 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19937 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.812 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.593427+0000", 2024-09-16T20:49:59.813 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.030185997999999999, 2024-09-16T20:49:59.813 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.031493210000000001, 2024-09-16T20:49:59.813 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.813 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.813 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.813 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.813 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:59.813 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:59.813 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19937 2024-09-16T20:49:59.813 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.813 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.813 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.814 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.814 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.593427+0000", 2024-09-16T20:49:59.814 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.814 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.814 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.814 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.814 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.593427+0000", 2024-09-16T20:49:59.814 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.814 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.814 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.814 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.814 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.593427+0000", 2024-09-16T20:49:59.815 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2900000000000004e-07 2024-09-16T20:49:59.815 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.815 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.815 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.815 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.593430+0000", 2024-09-16T20:49:59.815 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8439999999999998e-06 2024-09-16T20:49:59.815 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.815 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.815 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.815 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.593438+0000", 2024-09-16T20:49:59.815 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6559999999999992e-06 2024-09-16T20:49:59.815 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.816 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.816 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.816 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.593441+0000", 2024-09-16T20:49:59.816 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6200000000000001e-06 2024-09-16T20:49:59.816 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.816 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.816 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.816 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.816 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.816 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19953 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.816 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.596834+0000", 2024-09-16T20:49:59.816 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.026778765, 2024-09-16T20:49:59.817 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.028109249999999999, 2024-09-16T20:49:59.817 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.817 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.817 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.817 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.817 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:59.817 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:59.817 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19953 2024-09-16T20:49:59.817 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.817 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.817 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.818 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.818 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.596834+0000", 2024-09-16T20:49:59.818 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.818 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.818 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.818 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.818 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.596834+0000", 2024-09-16T20:49:59.818 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.818 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.818 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.818 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.818 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.596835+0000", 2024-09-16T20:49:59.818 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2900000000000003e-07 2024-09-16T20:49:59.819 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.819 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.819 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.819 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.596838+0000", 2024-09-16T20:49:59.819 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2389999999999998e-06 2024-09-16T20:49:59.819 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.819 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.819 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.819 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.596845+0000", 2024-09-16T20:49:59.819 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8709999999999999e-06 2024-09-16T20:49:59.819 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.819 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.819 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.820 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.596849+0000", 2024-09-16T20:49:59.820 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0929999999999996e-06 2024-09-16T20:49:59.820 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.820 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.820 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.820 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.820 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.820 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19969 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.820 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.600441+0000", 2024-09-16T20:49:59.820 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.023171661, 2024-09-16T20:49:59.820 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.024525208, 2024-09-16T20:49:59.820 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.820 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.821 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.821 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.821 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:59.821 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:59.821 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19969 2024-09-16T20:49:59.821 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.821 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.821 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.821 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.821 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.600441+0000", 2024-09-16T20:49:59.821 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.821 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.821 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.822 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.822 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.600441+0000", 2024-09-16T20:49:59.822 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.822 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.822 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.822 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.822 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.600442+0000", 2024-09-16T20:49:59.822 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6199999999999997e-07 2024-09-16T20:49:59.822 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.822 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.822 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.822 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.600445+0000", 2024-09-16T20:49:59.822 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6200000000000001e-06 2024-09-16T20:49:59.823 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.823 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.823 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.823 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.600459+0000", 2024-09-16T20:49:59.823 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3446e-05 2024-09-16T20:49:59.823 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.823 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.823 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.823 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.600465+0000", 2024-09-16T20:49:59.823 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4949999999999996e-06 2024-09-16T20:49:59.823 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.823 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.823 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.824 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.824 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.824 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19985 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.824 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.603745+0000", 2024-09-16T20:49:59.824 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.019867348, 2024-09-16T20:49:59.824 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021245712, 2024-09-16T20:49:59.824 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.824 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.824 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.824 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.824 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:59.824 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:59.825 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19985 2024-09-16T20:49:59.825 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.825 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.825 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.825 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.825 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.603745+0000", 2024-09-16T20:49:59.825 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.825 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.825 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.825 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.825 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.603745+0000", 2024-09-16T20:49:59.825 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.825 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.825 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.826 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.826 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.603746+0000", 2024-09-16T20:49:59.826 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3600000000000002e-07 2024-09-16T20:49:59.826 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.826 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.826 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.826 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.603750+0000", 2024-09-16T20:49:59.826 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.036e-06 2024-09-16T20:49:59.826 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.826 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.826 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.826 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.603758+0000", 2024-09-16T20:49:59.826 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7889999999999995e-06 2024-09-16T20:49:59.827 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.827 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.827 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.827 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.603762+0000", 2024-09-16T20:49:59.827 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1150000000000004e-06 2024-09-16T20:49:59.827 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.827 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.827 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.827 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.827 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.827 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24381 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.827 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.590157+0000", 2024-09-16T20:49:59.827 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.033455812000000001, 2024-09-16T20:49:59.828 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.034857078999999999, 2024-09-16T20:49:59.828 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.828 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.828 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.828 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.828 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:49:59.828 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:49:59.828 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24381 2024-09-16T20:49:59.828 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.828 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.828 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.828 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.828 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590157+0000", 2024-09-16T20:49:59.829 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.829 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.829 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.829 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.829 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590157+0000", 2024-09-16T20:49:59.829 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.829 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.829 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.829 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.829 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590157+0000", 2024-09-16T20:49:59.829 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7700000000000005e-07 2024-09-16T20:49:59.829 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.829 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.830 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.830 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590160+0000", 2024-09-16T20:49:59.830 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8760000000000001e-06 2024-09-16T20:49:59.830 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.830 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.830 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.830 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590167+0000", 2024-09-16T20:49:59.830 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0430000000000002e-06 2024-09-16T20:49:59.830 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.830 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.830 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.830 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590173+0000", 2024-09-16T20:49:59.830 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.818e-06 2024-09-16T20:49:59.831 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.831 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.831 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.831 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.831 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.831 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24397 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.831 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.593450+0000", 2024-09-16T20:49:59.831 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.030162512999999998, 2024-09-16T20:49:59.831 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.031587037999999998, 2024-09-16T20:49:59.831 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.831 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.831 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.831 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.831 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:49:59.832 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:49:59.832 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24397 2024-09-16T20:49:59.832 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.832 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.832 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.832 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.832 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.593450+0000", 2024-09-16T20:49:59.832 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.832 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.832 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.833 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.833 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.593450+0000", 2024-09-16T20:49:59.833 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.833 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.833 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.833 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.833 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.593451+0000", 2024-09-16T20:49:59.833 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.187e-06 2024-09-16T20:49:59.833 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.833 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.833 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.833 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.593455+0000", 2024-09-16T20:49:59.833 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0749999999999999e-06 2024-09-16T20:49:59.834 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.834 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.834 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.834 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.593465+0000", 2024-09-16T20:49:59.834 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0739999999999999e-06 2024-09-16T20:49:59.834 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.834 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.834 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.834 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.593471+0000", 2024-09-16T20:49:59.834 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4169999999999997e-06 2024-09-16T20:49:59.834 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.834 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.834 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.834 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.835 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.835 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24413 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.835 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.596860+0000", 2024-09-16T20:49:59.835 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.026752405999999999, 2024-09-16T20:49:59.835 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.028199874999999999, 2024-09-16T20:49:59.835 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.835 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.835 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.835 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.835 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:49:59.835 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:49:59.835 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24413 2024-09-16T20:49:59.836 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.836 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.836 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.836 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.836 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.596860+0000", 2024-09-16T20:49:59.836 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.836 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.836 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.836 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.836 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.596860+0000", 2024-09-16T20:49:59.836 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.836 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.836 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.837 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.837 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.596861+0000", 2024-09-16T20:49:59.837 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1500000000000003e-07 2024-09-16T20:49:59.837 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.837 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.837 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.837 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.596865+0000", 2024-09-16T20:49:59.837 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.019e-06 2024-09-16T20:49:59.837 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.837 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.837 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.837 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.596875+0000", 2024-09-16T20:49:59.837 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4900000000000006e-06 2024-09-16T20:49:59.838 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.838 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.838 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.838 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.596881+0000", 2024-09-16T20:49:59.838 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1410000000000002e-06 2024-09-16T20:49:59.838 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.838 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.838 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.838 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.838 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.838 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24429 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.838 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.600463+0000", 2024-09-16T20:49:59.839 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.023150184000000001, 2024-09-16T20:49:59.839 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.024620878999999998, 2024-09-16T20:49:59.839 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.839 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.839 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.839 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.839 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:49:59.839 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:49:59.839 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24429 2024-09-16T20:49:59.839 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.839 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.839 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.839 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.840 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.600463+0000", 2024-09-16T20:49:59.840 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.840 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.840 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.840 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.840 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.600463+0000", 2024-09-16T20:49:59.840 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.840 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.840 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.840 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.840 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.600463+0000", 2024-09-16T20:49:59.840 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1200000000000003e-07 2024-09-16T20:49:59.840 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.840 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.841 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.841 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.600466+0000", 2024-09-16T20:49:59.841 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6429999999999999e-06 2024-09-16T20:49:59.841 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.841 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.841 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.841 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.600471+0000", 2024-09-16T20:49:59.841 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3149999999999999e-06 2024-09-16T20:49:59.841 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.841 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.841 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.841 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.600475+0000", 2024-09-16T20:49:59.841 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.394e-06 2024-09-16T20:49:59.841 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.842 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.842 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.842 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.842 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.842 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24445 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.842 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.603750+0000", 2024-09-16T20:49:59.842 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.019862972999999999, 2024-09-16T20:49:59.842 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021368877000000001, 2024-09-16T20:49:59.842 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.842 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.842 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.842 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.842 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:49:59.842 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:49:59.843 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24445 2024-09-16T20:49:59.843 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.843 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.843 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.843 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.843 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.603750+0000", 2024-09-16T20:49:59.843 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.843 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.843 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.843 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.843 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.603750+0000", 2024-09-16T20:49:59.843 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.843 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.843 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.843 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.844 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.603750+0000", 2024-09-16T20:49:59.844 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9699999999999995e-07 2024-09-16T20:49:59.844 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.844 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.844 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.844 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.603753+0000", 2024-09-16T20:49:59.844 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5780000000000001e-06 2024-09-16T20:49:59.844 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.844 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.844 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.844 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.603759+0000", 2024-09-16T20:49:59.844 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5820000000000001e-06 2024-09-16T20:49:59.844 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.844 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.845 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.845 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.603762+0000", 2024-09-16T20:49:59.845 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3280000000000002e-06 2024-09-16T20:49:59.845 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.845 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.845 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.845 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.845 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.845 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19922 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.845 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.590364+0000", 2024-09-16T20:49:59.845 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.033248729999999997, 2024-09-16T20:49:59.845 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.034778668999999998, 2024-09-16T20:49:59.845 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.845 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.846 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.846 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.846 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:59.846 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:59.846 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19922 2024-09-16T20:49:59.846 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.846 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.846 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.846 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.846 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590364+0000", 2024-09-16T20:49:59.846 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.846 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.846 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.846 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.846 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590364+0000", 2024-09-16T20:49:59.846 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.847 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.847 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.847 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.847 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590365+0000", 2024-09-16T20:49:59.847 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8100000000000003e-07 2024-09-16T20:49:59.847 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.847 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.847 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.847 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590368+0000", 2024-09-16T20:49:59.847 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.839e-06 2024-09-16T20:49:59.847 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.847 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.847 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.847 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590382+0000", 2024-09-16T20:49:59.848 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3591000000000001e-05 2024-09-16T20:49:59.848 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.848 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.848 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.848 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590387+0000", 2024-09-16T20:49:59.848 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1599999999999997e-06 2024-09-16T20:49:59.848 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.848 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.848 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.848 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.848 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.848 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19938 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.848 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.593660+0000", 2024-09-16T20:49:59.848 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.029952625, 2024-09-16T20:49:59.849 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.031509429999999998, 2024-09-16T20:49:59.849 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.849 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.849 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.849 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.849 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:59.849 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:59.849 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19938 2024-09-16T20:49:59.849 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.849 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.849 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.849 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.850 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.593660+0000", 2024-09-16T20:49:59.850 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.850 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.850 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.850 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.850 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.593660+0000", 2024-09-16T20:49:59.850 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.850 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.850 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.850 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.850 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.593661+0000", 2024-09-16T20:49:59.850 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7700000000000005e-07 2024-09-16T20:49:59.850 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.851 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.851 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.851 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.593663+0000", 2024-09-16T20:49:59.851 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.869e-06 2024-09-16T20:49:59.851 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.851 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.851 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.851 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.593669+0000", 2024-09-16T20:49:59.851 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9689999999999999e-06 2024-09-16T20:49:59.851 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.851 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.851 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.851 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.593673+0000", 2024-09-16T20:49:59.851 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5049999999999999e-06 2024-09-16T20:49:59.852 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.852 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.852 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.852 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.852 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.852 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19954 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.852 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.597064+0000", 2024-09-16T20:49:59.852 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.026548538999999999, 2024-09-16T20:49:59.852 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.028128667999999999, 2024-09-16T20:49:59.852 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.852 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.852 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.852 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.852 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:59.853 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:59.853 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19954 2024-09-16T20:49:59.853 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.853 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.853 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.853 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.853 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.597064+0000", 2024-09-16T20:49:59.853 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.853 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.853 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.853 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.853 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.597064+0000", 2024-09-16T20:49:59.853 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.854 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.854 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.854 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.854 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.597065+0000", 2024-09-16T20:49:59.854 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9500000000000002e-07 2024-09-16T20:49:59.854 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.854 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.854 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.854 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.597068+0000", 2024-09-16T20:49:59.854 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3299999999999999e-06 2024-09-16T20:49:59.854 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.854 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.854 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.855 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.597079+0000", 2024-09-16T20:49:59.855 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0804e-05 2024-09-16T20:49:59.855 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.855 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.855 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.855 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.597086+0000", 2024-09-16T20:49:59.855 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5549999999999999e-06 2024-09-16T20:49:59.855 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.855 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.855 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.855 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.855 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.855 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19970 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.855 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.600694+0000", 2024-09-16T20:49:59.856 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.022918272999999999, 2024-09-16T20:49:59.856 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.024521567000000001, 2024-09-16T20:49:59.856 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.856 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.856 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.856 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.856 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:59.856 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:59.856 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19970 2024-09-16T20:49:59.856 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.856 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.856 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.857 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.857 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.600694+0000", 2024-09-16T20:49:59.857 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.857 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.857 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.857 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.857 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.600694+0000", 2024-09-16T20:49:59.857 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.857 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.857 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.857 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.857 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.600695+0000", 2024-09-16T20:49:59.857 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8800000000000002e-07 2024-09-16T20:49:59.858 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.858 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.858 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.858 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.600698+0000", 2024-09-16T20:49:59.858 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8990000000000001e-06 2024-09-16T20:49:59.858 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.858 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.858 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.858 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.600707+0000", 2024-09-16T20:49:59.858 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9290000000000004e-06 2024-09-16T20:49:59.858 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.858 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.858 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.859 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.600714+0000", 2024-09-16T20:49:59.859 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2830000000000004e-06 2024-09-16T20:49:59.859 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.859 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.859 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.859 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.859 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.859 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19986 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.859 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.604001+0000", 2024-09-16T20:49:59.859 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.019611775000000001, 2024-09-16T20:49:59.859 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021238713999999999, 2024-09-16T20:49:59.859 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.859 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.859 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.860 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.860 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:59.860 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:59.860 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19986 2024-09-16T20:49:59.860 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.860 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.860 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.860 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.860 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.604001+0000", 2024-09-16T20:49:59.860 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.860 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.860 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.860 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.860 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.604001+0000", 2024-09-16T20:49:59.860 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.861 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.861 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.861 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.861 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.604002+0000", 2024-09-16T20:49:59.861 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3900000000000004e-07 2024-09-16T20:49:59.861 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.861 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.861 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.861 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.604006+0000", 2024-09-16T20:49:59.861 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8500000000000004e-06 2024-09-16T20:49:59.861 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.861 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.861 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.861 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.604014+0000", 2024-09-16T20:49:59.861 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3499999999999997e-06 2024-09-16T20:49:59.862 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.862 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.862 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.862 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.604021+0000", 2024-09-16T20:49:59.862 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1489999999999997e-06 2024-09-16T20:49:59.862 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.862 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.862 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.862 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.862 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.862 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24382 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.862 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.590392+0000", 2024-09-16T20:49:59.862 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.033220415000000003, 2024-09-16T20:49:59.862 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.034871319999999997, 2024-09-16T20:49:59.862 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.863 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.863 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.863 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.863 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:49:59.863 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:49:59.863 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24382 2024-09-16T20:49:59.863 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.863 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.863 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.863 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.863 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590392+0000", 2024-09-16T20:49:59.863 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.863 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.863 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.863 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.864 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590392+0000", 2024-09-16T20:49:59.864 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.864 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.864 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.864 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.864 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590393+0000", 2024-09-16T20:49:59.864 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0599999999999999e-07 2024-09-16T20:49:59.864 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.864 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.864 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.864 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590397+0000", 2024-09-16T20:49:59.864 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.771e-06 2024-09-16T20:49:59.864 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.864 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.864 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.865 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590410+0000", 2024-09-16T20:49:59.865 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3526999999999999e-05 2024-09-16T20:49:59.865 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.865 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.865 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.865 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590417+0000", 2024-09-16T20:49:59.865 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7839999999999998e-06 2024-09-16T20:49:59.865 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.865 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.865 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.865 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.865 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.865 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24398 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.865 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.593687+0000", 2024-09-16T20:49:59.866 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.029926161, 2024-09-16T20:49:59.866 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.031601263999999997, 2024-09-16T20:49:59.866 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.866 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.866 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.866 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.866 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:49:59.866 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:49:59.866 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24398 2024-09-16T20:49:59.866 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.866 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.866 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.866 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.866 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.593687+0000", 2024-09-16T20:49:59.867 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.867 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.867 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.867 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.867 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.593687+0000", 2024-09-16T20:49:59.867 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.867 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.867 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.867 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.867 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.593688+0000", 2024-09-16T20:49:59.867 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0219999999999999e-06 2024-09-16T20:49:59.867 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.867 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.867 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.868 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.593691+0000", 2024-09-16T20:49:59.868 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8650000000000003e-06 2024-09-16T20:49:59.868 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.868 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.868 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.868 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.593700+0000", 2024-09-16T20:49:59.868 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8289999999999997e-06 2024-09-16T20:49:59.868 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.868 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.868 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.868 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.593706+0000", 2024-09-16T20:49:59.868 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1859999999999997e-06 2024-09-16T20:49:59.868 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.868 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.869 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.869 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.869 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.869 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24414 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.869 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.597112+0000", 2024-09-16T20:49:59.869 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.026500339000000001, 2024-09-16T20:49:59.869 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.028198240999999999, 2024-09-16T20:49:59.869 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.869 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.869 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.869 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.869 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:49:59.869 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:49:59.869 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24414 2024-09-16T20:49:59.870 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.870 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.870 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.870 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.870 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.597112+0000", 2024-09-16T20:49:59.870 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.870 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.870 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.870 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.870 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.597112+0000", 2024-09-16T20:49:59.870 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.870 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.870 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.870 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.870 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.597113+0000", 2024-09-16T20:49:59.871 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3300000000000001e-07 2024-09-16T20:49:59.871 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.871 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.871 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.871 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.597117+0000", 2024-09-16T20:49:59.871 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0690000000000003e-06 2024-09-16T20:49:59.871 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.871 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.871 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.871 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.597125+0000", 2024-09-16T20:49:59.871 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1890000000000007e-06 2024-09-16T20:49:59.872 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.872 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.872 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.872 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.597130+0000", 2024-09-16T20:49:59.872 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3579999999999996e-06 2024-09-16T20:49:59.872 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.872 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.872 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.872 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.872 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.872 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24430 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.872 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.600714+0000", 2024-09-16T20:49:59.872 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.022898377000000001, 2024-09-16T20:49:59.873 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.024647621000000002, 2024-09-16T20:49:59.873 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.873 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.873 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.873 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.873 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:49:59.873 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:49:59.873 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24430 2024-09-16T20:49:59.873 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.873 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.873 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.873 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.873 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.600714+0000", 2024-09-16T20:49:59.874 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.874 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.874 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.874 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.874 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.600714+0000", 2024-09-16T20:49:59.874 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.874 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.876 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.876 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.876 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.600715+0000", 2024-09-16T20:49:59.876 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9500000000000002e-07 2024-09-16T20:49:59.876 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.876 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.877 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.877 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.600718+0000", 2024-09-16T20:49:59.877 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4660000000000002e-06 2024-09-16T20:49:59.877 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.877 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.877 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.877 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.600724+0000", 2024-09-16T20:49:59.877 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5030000000000002e-06 2024-09-16T20:49:59.877 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.877 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.877 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.877 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.600728+0000", 2024-09-16T20:49:59.877 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6679999999999999e-06 2024-09-16T20:49:59.878 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.878 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.878 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.878 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.878 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.878 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24446 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.878 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.604024+0000", 2024-09-16T20:49:59.878 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.019588509, 2024-09-16T20:49:59.878 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021363032000000001, 2024-09-16T20:49:59.878 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.878 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.878 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.878 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.879 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:49:59.879 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:49:59.879 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24446 2024-09-16T20:49:59.879 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.879 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.879 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.879 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.879 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.604024+0000", 2024-09-16T20:49:59.879 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.879 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.879 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.879 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.879 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.604024+0000", 2024-09-16T20:49:59.880 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.880 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.880 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.880 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.880 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.605979+0000", 2024-09-16T20:49:59.880 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0019548080000000002 2024-09-16T20:49:59.880 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.880 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.880 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.880 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.605982+0000", 2024-09-16T20:49:59.880 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7949999999999999e-06 2024-09-16T20:49:59.880 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.881 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.881 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.881 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.605988+0000", 2024-09-16T20:49:59.881 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0750000000000003e-06 2024-09-16T20:49:59.881 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.881 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.881 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.881 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.605994+0000", 2024-09-16T20:49:59.881 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4670000000000001e-06 2024-09-16T20:49:59.881 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.881 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.881 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.881 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.881 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.882 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19923 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.882 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.590555+0000", 2024-09-16T20:49:59.882 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.033057259999999998, 2024-09-16T20:49:59.882 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.034868351999999998, 2024-09-16T20:49:59.882 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.882 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.882 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.882 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.882 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:59.882 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:59.882 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19923 2024-09-16T20:49:59.882 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.882 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.883 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.883 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.883 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590555+0000", 2024-09-16T20:49:59.883 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.883 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.883 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.883 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.883 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590555+0000", 2024-09-16T20:49:59.883 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.883 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.883 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.883 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.884 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590556+0000", 2024-09-16T20:49:59.884 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.61e-07 2024-09-16T20:49:59.884 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.884 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.884 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.884 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590559+0000", 2024-09-16T20:49:59.884 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2260000000000001e-06 2024-09-16T20:49:59.884 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.884 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.884 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.884 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590566+0000", 2024-09-16T20:49:59.884 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8630000000000002e-06 2024-09-16T20:49:59.884 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.885 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.885 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.885 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590571+0000", 2024-09-16T20:49:59.885 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5299999999999998e-06 2024-09-16T20:49:59.885 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.885 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.885 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.885 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.885 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.885 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19939 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.885 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.593832+0000", 2024-09-16T20:49:59.885 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.029781076, 2024-09-16T20:49:59.885 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.031616682, 2024-09-16T20:49:59.886 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.886 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.886 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.886 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.886 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:59.886 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:59.886 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19939 2024-09-16T20:49:59.886 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.886 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.886 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.886 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.886 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.593832+0000", 2024-09-16T20:49:59.886 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.886 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.887 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.887 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.887 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.593832+0000", 2024-09-16T20:49:59.887 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.887 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.887 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.887 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.887 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.593832+0000", 2024-09-16T20:49:59.887 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0100000000000005e-07 2024-09-16T20:49:59.887 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.887 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.887 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.887 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.593835+0000", 2024-09-16T20:49:59.887 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5940000000000001e-06 2024-09-16T20:49:59.888 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.888 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.888 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.888 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.593844+0000", 2024-09-16T20:49:59.888 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7499999999999992e-06 2024-09-16T20:49:59.888 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.888 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.888 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.888 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.593847+0000", 2024-09-16T20:49:59.888 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4659999999999999e-06 2024-09-16T20:49:59.888 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.888 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.888 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.889 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.889 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.889 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19955 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.889 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.597259+0000", 2024-09-16T20:49:59.889 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.026354076000000001, 2024-09-16T20:49:59.889 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.028212681999999999, 2024-09-16T20:49:59.889 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.889 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.889 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.889 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.889 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:59.889 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:59.890 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19955 2024-09-16T20:49:59.890 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.890 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.890 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.890 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.890 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.597259+0000", 2024-09-16T20:49:59.890 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.890 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.890 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.890 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.890 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.597259+0000", 2024-09-16T20:49:59.890 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.890 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.890 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.891 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.891 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.597259+0000", 2024-09-16T20:49:59.891 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6000000000000003e-07 2024-09-16T20:49:59.891 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.891 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.891 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.891 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.597263+0000", 2024-09-16T20:49:59.891 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5939999999999998e-06 2024-09-16T20:49:59.891 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.891 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.891 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.891 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.597271+0000", 2024-09-16T20:49:59.891 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8639999999999996e-06 2024-09-16T20:49:59.892 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.892 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.892 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.892 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.597275+0000", 2024-09-16T20:49:59.892 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2930000000000002e-06 2024-09-16T20:49:59.892 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.892 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.892 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.892 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.892 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.892 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19971 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.892 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.600902+0000", 2024-09-16T20:49:59.892 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.022710516, 2024-09-16T20:49:59.893 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.024592005, 2024-09-16T20:49:59.893 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.893 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.893 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.893 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.893 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:59.893 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:59.893 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19971 2024-09-16T20:49:59.893 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.893 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.893 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.893 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.893 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.600902+0000", 2024-09-16T20:49:59.893 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.894 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.894 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.894 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.894 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.600902+0000", 2024-09-16T20:49:59.894 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.894 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.894 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.894 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.894 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.600903+0000", 2024-09-16T20:49:59.894 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2699999999999999e-07 2024-09-16T20:49:59.894 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.894 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.894 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.894 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.600906+0000", 2024-09-16T20:49:59.895 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3069999999999998e-06 2024-09-16T20:49:59.895 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.895 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.895 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.895 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.600921+0000", 2024-09-16T20:49:59.895 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4834000000000001e-05 2024-09-16T20:49:59.895 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.895 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.895 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.895 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.600929+0000", 2024-09-16T20:49:59.895 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6750000000000002e-06 2024-09-16T20:49:59.895 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.895 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.895 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.896 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.896 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.896 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19987 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.896 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.605989+0000", 2024-09-16T20:49:59.896 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.017623857, 2024-09-16T20:49:59.896 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.019528176000000001, 2024-09-16T20:49:59.896 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.896 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.896 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.896 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.896 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:59.896 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:59.896 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19987 2024-09-16T20:49:59.897 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.897 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.897 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.897 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.897 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.605989+0000", 2024-09-16T20:49:59.897 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.897 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.897 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.897 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.897 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.605989+0000", 2024-09-16T20:49:59.897 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.897 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.897 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.897 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.898 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.605990+0000", 2024-09-16T20:49:59.898 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.187e-06 2024-09-16T20:49:59.898 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.898 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.898 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.898 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.605995+0000", 2024-09-16T20:49:59.898 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.454e-06 2024-09-16T20:49:59.898 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.898 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.898 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.898 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.606007+0000", 2024-09-16T20:49:59.898 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2007999999999999e-05 2024-09-16T20:49:59.898 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.898 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.899 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.899 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.606013+0000", 2024-09-16T20:49:59.899 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7329999999999997e-06 2024-09-16T20:49:59.899 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.899 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.899 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.899 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.899 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.899 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24383 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.899 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.590577+0000", 2024-09-16T20:49:59.899 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.033036190999999999, 2024-09-16T20:49:59.899 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.034963631000000002, 2024-09-16T20:49:59.899 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.900 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.900 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.900 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.900 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:49:59.900 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:49:59.900 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24383 2024-09-16T20:49:59.900 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.900 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.900 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.900 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.900 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590577+0000", 2024-09-16T20:49:59.900 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.900 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.901 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.901 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.901 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590577+0000", 2024-09-16T20:49:59.901 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.901 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.901 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.901 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.901 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590577+0000", 2024-09-16T20:49:59.901 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7400000000000003e-07 2024-09-16T20:49:59.901 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.901 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.901 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.901 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590581+0000", 2024-09-16T20:49:59.902 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8480000000000003e-06 2024-09-16T20:49:59.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.902 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.902 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.902 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590590+0000", 2024-09-16T20:49:59.902 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7099999999999996e-06 2024-09-16T20:49:59.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.902 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.902 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.902 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590594+0000", 2024-09-16T20:49:59.902 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4880000000000001e-06 2024-09-16T20:49:59.902 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.902 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.902 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.903 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.903 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.903 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24399 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.903 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.593854+0000", 2024-09-16T20:49:59.903 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.029759157000000001, 2024-09-16T20:49:59.903 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.031709932000000003, 2024-09-16T20:49:59.903 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.903 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.903 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.903 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.903 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:49:59.903 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:49:59.903 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24399 2024-09-16T20:49:59.904 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.904 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.904 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.904 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.904 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.593854+0000", 2024-09-16T20:49:59.904 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.904 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.904 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.904 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.904 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.593854+0000", 2024-09-16T20:49:59.904 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.904 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.904 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.905 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.905 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.593855+0000", 2024-09-16T20:49:59.905 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9999999999999995e-07 2024-09-16T20:49:59.905 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.905 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.905 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.905 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.593859+0000", 2024-09-16T20:49:59.905 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2039999999999999e-06 2024-09-16T20:49:59.905 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.905 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.905 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.905 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.593871+0000", 2024-09-16T20:49:59.905 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1839e-05 2024-09-16T20:49:59.905 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.906 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.906 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.906 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.593878+0000", 2024-09-16T20:49:59.906 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.092e-06 2024-09-16T20:49:59.906 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.906 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.906 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.906 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.906 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24415 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.906 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.597281+0000", 2024-09-16T20:49:59.906 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.026332017999999999, 2024-09-16T20:49:59.906 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.028305988000000001, 2024-09-16T20:49:59.907 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.907 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.907 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.907 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.907 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:49:59.907 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:49:59.907 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24415 2024-09-16T20:49:59.907 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.907 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.907 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.907 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.907 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.597281+0000", 2024-09-16T20:49:59.907 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.907 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.908 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.908 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.908 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.597281+0000", 2024-09-16T20:49:59.908 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.908 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.908 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.908 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.908 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.597281+0000", 2024-09-16T20:49:59.908 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5799999999999998e-07 2024-09-16T20:49:59.908 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.908 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.908 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.908 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.597285+0000", 2024-09-16T20:49:59.909 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.681e-06 2024-09-16T20:49:59.909 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.909 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.909 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.909 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.597291+0000", 2024-09-16T20:49:59.909 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0789999999999997e-06 2024-09-16T20:49:59.909 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.909 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.909 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.909 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.597295+0000", 2024-09-16T20:49:59.909 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7220000000000001e-06 2024-09-16T20:49:59.909 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.909 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.910 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.910 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.910 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.910 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24431 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.910 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.600919+0000", 2024-09-16T20:49:59.910 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.022694188000000001, 2024-09-16T20:49:59.910 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.024691060000000001, 2024-09-16T20:49:59.910 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.910 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.910 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.910 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.910 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:49:59.910 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:49:59.910 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24431 2024-09-16T20:49:59.911 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.911 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.911 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.911 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.911 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.600919+0000", 2024-09-16T20:49:59.911 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.911 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.911 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.911 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.911 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.600919+0000", 2024-09-16T20:49:59.911 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.911 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.911 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.911 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.912 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.600919+0000", 2024-09-16T20:49:59.912 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8299999999999996e-07 2024-09-16T20:49:59.912 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.912 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.912 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.912 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.600922+0000", 2024-09-16T20:49:59.912 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3980000000000002e-06 2024-09-16T20:49:59.912 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.912 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.912 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.912 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.600932+0000", 2024-09-16T20:49:59.912 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0293e-05 2024-09-16T20:49:59.912 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.913 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.913 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.913 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.600936+0000", 2024-09-16T20:49:59.913 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.532e-06 2024-09-16T20:49:59.913 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.913 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.913 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.913 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.913 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.913 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24447 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.913 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.606003+0000", 2024-09-16T20:49:59.913 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.017609599, 2024-09-16T20:49:59.913 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.019629522, 2024-09-16T20:49:59.914 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.914 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.914 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.914 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.914 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:49:59.914 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:49:59.914 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24447 2024-09-16T20:49:59.914 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.914 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.914 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.914 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.914 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.606003+0000", 2024-09-16T20:49:59.914 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.914 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.915 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.915 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.915 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.606003+0000", 2024-09-16T20:49:59.915 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.915 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.915 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.915 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.915 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.606004+0000", 2024-09-16T20:49:59.915 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4000000000000002e-07 2024-09-16T20:49:59.915 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.915 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.915 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.915 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.606005+0000", 2024-09-16T20:49:59.915 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6649999999999999e-06 2024-09-16T20:49:59.916 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.916 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.916 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.916 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.606009+0000", 2024-09-16T20:49:59.916 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.557e-06 2024-09-16T20:49:59.916 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.916 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.916 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.916 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.606012+0000", 2024-09-16T20:49:59.916 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.948e-06 2024-09-16T20:49:59.916 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.916 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.916 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.917 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.917 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.917 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19924 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.917 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.590774+0000", 2024-09-16T20:49:59.917 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.032838440000000003, 2024-09-16T20:49:59.917 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.034881632000000003, 2024-09-16T20:49:59.917 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.917 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.917 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.917 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.917 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:59.917 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:59.917 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19924 2024-09-16T20:49:59.917 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.918 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.918 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.918 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.918 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590774+0000", 2024-09-16T20:49:59.918 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.918 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.918 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.918 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.918 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590774+0000", 2024-09-16T20:49:59.918 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.918 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.918 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.918 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.918 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590775+0000", 2024-09-16T20:49:59.919 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5499999999999998e-07 2024-09-16T20:49:59.919 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.919 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.919 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.919 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590778+0000", 2024-09-16T20:49:59.919 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1259999999999998e-06 2024-09-16T20:49:59.919 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.919 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.919 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.919 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590789+0000", 2024-09-16T20:49:59.919 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0835999999999999e-05 2024-09-16T20:49:59.919 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.919 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.920 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.920 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590796+0000", 2024-09-16T20:49:59.920 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2119999999999997e-06 2024-09-16T20:49:59.920 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.920 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.920 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.920 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.920 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.920 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19940 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.920 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.594089+0000", 2024-09-16T20:49:59.920 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.029523266999999999, 2024-09-16T20:49:59.920 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.031589865000000002, 2024-09-16T20:49:59.920 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.920 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.921 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.921 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.921 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:59.921 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:59.921 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19940 2024-09-16T20:49:59.921 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.921 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.921 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.921 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.921 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.594089+0000", 2024-09-16T20:49:59.921 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.921 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.921 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.921 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.922 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.594089+0000", 2024-09-16T20:49:59.922 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.922 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.922 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.922 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.594090+0000", 2024-09-16T20:49:59.922 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7399999999999991e-07 2024-09-16T20:49:59.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.922 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.922 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.922 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.594095+0000", 2024-09-16T20:49:59.922 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6589999999999998e-06 2024-09-16T20:49:59.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.922 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.923 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.923 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.594111+0000", 2024-09-16T20:49:59.923 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5648e-05 2024-09-16T20:49:59.923 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.923 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.923 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.923 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.594117+0000", 2024-09-16T20:49:59.923 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6699999999999997e-06 2024-09-16T20:49:59.923 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.923 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.923 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.924 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.924 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.924 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19956 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.924 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.597520+0000", 2024-09-16T20:49:59.924 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.026093117999999998, 2024-09-16T20:49:59.924 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.028182955999999999, 2024-09-16T20:49:59.924 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.924 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.924 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.924 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.924 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:59.925 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:59.925 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19956 2024-09-16T20:49:59.925 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.925 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.925 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.925 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.925 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.597520+0000", 2024-09-16T20:49:59.925 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.925 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.925 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.925 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.925 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.597520+0000", 2024-09-16T20:49:59.925 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.926 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.926 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.926 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.926 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.597520+0000", 2024-09-16T20:49:59.926 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6000000000000004e-07 2024-09-16T20:49:59.926 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.926 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.926 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.926 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.597523+0000", 2024-09-16T20:49:59.926 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.948e-06 2024-09-16T20:49:59.926 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.926 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.926 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.926 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.597532+0000", 2024-09-16T20:49:59.926 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9369999999999993e-06 2024-09-16T20:49:59.927 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.927 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.927 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.927 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.597538+0000", 2024-09-16T20:49:59.927 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0090000000000004e-06 2024-09-16T20:49:59.927 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.927 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.927 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.927 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.927 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.927 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24432 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.927 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.601184+0000", 2024-09-16T20:49:59.928 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.022428587, 2024-09-16T20:49:59.928 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.024541568999999999, 2024-09-16T20:49:59.928 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.928 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.928 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.928 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.928 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:49:59.928 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:49:59.928 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24432 2024-09-16T20:49:59.928 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.928 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.928 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.928 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.929 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.601184+0000", 2024-09-16T20:49:59.929 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.929 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.929 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.929 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.929 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.601184+0000", 2024-09-16T20:49:59.929 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.929 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.929 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.929 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.929 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.601185+0000", 2024-09-16T20:49:59.929 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6699999999999999e-07 2024-09-16T20:49:59.929 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.929 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.930 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.930 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.601187+0000", 2024-09-16T20:49:59.930 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4279999999999999e-06 2024-09-16T20:49:59.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.930 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.930 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.930 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.601193+0000", 2024-09-16T20:49:59.930 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2829999999999998e-06 2024-09-16T20:49:59.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.930 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.930 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.930 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.601197+0000", 2024-09-16T20:49:59.930 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2090000000000001e-06 2024-09-16T20:49:59.930 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.931 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.931 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.931 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.931 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.931 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24448 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.931 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.606309+0000", 2024-09-16T20:49:59.931 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.017304053, 2024-09-16T20:49:59.931 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.019439820999999999, 2024-09-16T20:49:59.931 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.931 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.931 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.931 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.931 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:49:59.931 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:49:59.931 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24448 2024-09-16T20:49:59.932 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.932 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.932 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.932 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.932 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.606309+0000", 2024-09-16T20:49:59.932 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.932 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.932 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.932 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.932 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.606309+0000", 2024-09-16T20:49:59.932 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.932 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.932 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.932 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.933 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.606309+0000", 2024-09-16T20:49:59.933 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7300000000000001e-07 2024-09-16T20:49:59.933 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.933 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.933 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.933 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.606317+0000", 2024-09-16T20:49:59.933 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6020000000000002e-06 2024-09-16T20:49:59.933 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.933 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.933 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.933 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.606324+0000", 2024-09-16T20:49:59.933 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0679999999999999e-06 2024-09-16T20:49:59.933 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.933 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.934 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.934 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.606327+0000", 2024-09-16T20:49:59.934 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3550000000000001e-06 2024-09-16T20:49:59.934 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.934 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.934 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.934 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.934 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.934 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24384 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.934 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.590797+0000", 2024-09-16T20:49:59.934 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.032816028999999997, 2024-09-16T20:49:59.935 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.034974719000000001, 2024-09-16T20:49:59.935 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.935 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.935 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.935 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.935 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:49:59.935 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:49:59.935 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24384 2024-09-16T20:49:59.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.935 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.935 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.935 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.935 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590797+0000", 2024-09-16T20:49:59.936 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.936 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.936 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.936 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.936 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590797+0000", 2024-09-16T20:49:59.936 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.936 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.936 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.936 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.936 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590797+0000", 2024-09-16T20:49:59.936 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7999999999999995e-07 2024-09-16T20:49:59.936 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.936 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.937 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.937 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590801+0000", 2024-09-16T20:49:59.937 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6899999999999998e-06 2024-09-16T20:49:59.937 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.937 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.937 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.937 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590809+0000", 2024-09-16T20:49:59.937 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7079999999999997e-06 2024-09-16T20:49:59.937 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.937 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.937 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.937 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590815+0000", 2024-09-16T20:49:59.937 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0610000000000001e-06 2024-09-16T20:49:59.938 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.938 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.938 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.938 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.938 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.938 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24400 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.938 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.594101+0000", 2024-09-16T20:49:59.938 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.029511742000000001, 2024-09-16T20:49:59.938 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.031693172999999998, 2024-09-16T20:49:59.938 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.938 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.938 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.938 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.939 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:49:59.939 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:49:59.939 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24400 2024-09-16T20:49:59.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.939 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.939 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.939 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.594101+0000", 2024-09-16T20:49:59.939 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.939 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.939 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.594101+0000", 2024-09-16T20:49:59.939 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.594102+0000", 2024-09-16T20:49:59.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6799999999999999e-07 2024-09-16T20:49:59.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.594105+0000", 2024-09-16T20:49:59.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2279999999999998e-06 2024-09-16T20:49:59.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.594114+0000", 2024-09-16T20:49:59.941 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7809999999999994e-06 2024-09-16T20:49:59.941 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.941 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.941 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.941 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.594120+0000", 2024-09-16T20:49:59.941 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1619999999999996e-06 2024-09-16T20:49:59.941 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.941 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.941 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.941 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.941 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.941 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24416 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.941 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.597535+0000", 2024-09-16T20:49:59.941 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.026077959000000001, 2024-09-16T20:49:59.942 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.028282087000000001, 2024-09-16T20:49:59.942 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.942 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.942 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.942 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.942 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:49:59.942 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:49:59.942 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24416 2024-09-16T20:49:59.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.942 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.942 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.942 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.942 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.597535+0000", 2024-09-16T20:49:59.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.943 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.943 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.597535+0000", 2024-09-16T20:49:59.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.943 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.943 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.597536+0000", 2024-09-16T20:49:59.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4499999999999996e-07 2024-09-16T20:49:59.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.943 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.944 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.597538+0000", 2024-09-16T20:49:59.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8339999999999999e-06 2024-09-16T20:49:59.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.944 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.944 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.597544+0000", 2024-09-16T20:49:59.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0020000000000003e-06 2024-09-16T20:49:59.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.944 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.944 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.597548+0000", 2024-09-16T20:49:59.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7069999999999998e-06 2024-09-16T20:49:59.944 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.944 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.945 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.945 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19972 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.945 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.601243+0000", 2024-09-16T20:49:59.945 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.022369936, 2024-09-16T20:49:59.945 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.024597293999999999, 2024-09-16T20:49:59.945 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.945 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.945 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.945 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.945 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:59.945 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:59.945 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19972 2024-09-16T20:49:59.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.946 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.946 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.946 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.946 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.601243+0000", 2024-09-16T20:49:59.946 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.946 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.946 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.946 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.601243+0000", 2024-09-16T20:49:59.946 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.946 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.946 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.946 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.601244+0000", 2024-09-16T20:49:59.946 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.569999999999999e-07 2024-09-16T20:49:59.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.947 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.947 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.947 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.601248+0000", 2024-09-16T20:49:59.947 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2799999999999997e-06 2024-09-16T20:49:59.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.947 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.947 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.947 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.601257+0000", 2024-09-16T20:49:59.947 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1919999999999994e-06 2024-09-16T20:49:59.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.947 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.947 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.947 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.601263+0000", 2024-09-16T20:49:59.947 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0839999999999997e-06 2024-09-16T20:49:59.948 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.948 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.948 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.948 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19925 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.948 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.590930+0000", 2024-09-16T20:49:59.948 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.032682389999999999, 2024-09-16T20:49:59.948 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.034933879000000001, 2024-09-16T20:49:59.948 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.948 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.948 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.948 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.948 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:59.949 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:59.949 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19925 2024-09-16T20:49:59.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.949 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.949 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.949 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.949 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590930+0000", 2024-09-16T20:49:59.949 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.949 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.949 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.949 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590930+0000", 2024-09-16T20:49:59.949 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.950 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.950 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.950 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590931+0000", 2024-09-16T20:49:59.950 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3600000000000003e-07 2024-09-16T20:49:59.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.950 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.950 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.950 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590934+0000", 2024-09-16T20:49:59.950 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1300000000000001e-06 2024-09-16T20:49:59.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.950 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.950 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.950 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590943+0000", 2024-09-16T20:49:59.951 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0999999999999993e-06 2024-09-16T20:49:59.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.951 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.951 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.951 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590949+0000", 2024-09-16T20:49:59.951 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.711e-06 2024-09-16T20:49:59.951 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.951 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.951 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.951 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.951 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19941 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.951 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.594292+0000", 2024-09-16T20:49:59.952 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.029320946000000001, 2024-09-16T20:49:59.952 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.031595557000000003, 2024-09-16T20:49:59.952 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.952 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.952 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.952 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.952 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:59.952 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:59.952 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19941 2024-09-16T20:49:59.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.952 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.952 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.952 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.952 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.594292+0000", 2024-09-16T20:49:59.952 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.953 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.953 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.953 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.594292+0000", 2024-09-16T20:49:59.953 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.953 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.953 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.953 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.594293+0000", 2024-09-16T20:49:59.953 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1099999999999995e-07 2024-09-16T20:49:59.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.953 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.953 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.953 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.594296+0000", 2024-09-16T20:49:59.954 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6660000000000001e-06 2024-09-16T20:49:59.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.954 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.954 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.954 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.594304+0000", 2024-09-16T20:49:59.954 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4120000000000004e-06 2024-09-16T20:49:59.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.954 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.954 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.954 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.594310+0000", 2024-09-16T20:49:59.954 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0959999999999997e-06 2024-09-16T20:49:59.954 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.954 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.955 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.955 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19957 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.955 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.597720+0000", 2024-09-16T20:49:59.955 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.025892864000000002, 2024-09-16T20:49:59.955 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.028192618999999999, 2024-09-16T20:49:59.955 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.955 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.955 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.955 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.955 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:59.955 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:59.955 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19957 2024-09-16T20:49:59.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.956 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.956 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.956 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.597720+0000", 2024-09-16T20:49:59.956 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.956 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.956 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.597720+0000", 2024-09-16T20:49:59.956 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.956 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.956 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.597721+0000", 2024-09-16T20:49:59.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4899999999999995e-07 2024-09-16T20:49:59.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.957 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.957 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.597724+0000", 2024-09-16T20:49:59.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6059999999999999e-06 2024-09-16T20:49:59.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.957 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.957 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.597731+0000", 2024-09-16T20:49:59.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1149999999999996e-06 2024-09-16T20:49:59.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.957 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.957 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.597735+0000", 2024-09-16T20:49:59.958 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2490000000000003e-06 2024-09-16T20:49:59.958 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.958 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.958 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.958 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.958 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.958 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19973 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.958 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.601394+0000", 2024-09-16T20:49:59.958 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.022218831000000001, 2024-09-16T20:49:59.958 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.024552481000000001, 2024-09-16T20:49:59.958 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.958 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.958 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.959 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.959 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:59.959 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:59.959 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19973 2024-09-16T20:49:59.959 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.959 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.959 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.959 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.959 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.601394+0000", 2024-09-16T20:49:59.959 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.959 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.959 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.959 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.959 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.601394+0000", 2024-09-16T20:49:59.959 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.960 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.960 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.601394+0000", 2024-09-16T20:49:59.960 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6899999999999997e-07 2024-09-16T20:49:59.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.960 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.960 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.601398+0000", 2024-09-16T20:49:59.960 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4039999999999999e-06 2024-09-16T20:49:59.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.960 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.960 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.601406+0000", 2024-09-16T20:49:59.960 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4060000000000005e-06 2024-09-16T20:49:59.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.961 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.961 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.961 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.601412+0000", 2024-09-16T20:49:59.961 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0599999999999996e-06 2024-09-16T20:49:59.961 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.961 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.961 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.961 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.961 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24385 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.961 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.590944+0000", 2024-09-16T20:49:59.961 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.032669222999999997, 2024-09-16T20:49:59.961 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.035026590000000003, 2024-09-16T20:49:59.961 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.962 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.962 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.962 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.962 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:49:59.962 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:49:59.962 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24385 2024-09-16T20:49:59.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.962 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.962 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.962 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590944+0000", 2024-09-16T20:49:59.962 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.963 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.963 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590944+0000", 2024-09-16T20:49:59.963 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.963 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.963 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.963 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590944+0000", 2024-09-16T20:49:59.963 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0900000000000001e-07 2024-09-16T20:49:59.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.963 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.963 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.963 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590947+0000", 2024-09-16T20:49:59.963 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.269e-06 2024-09-16T20:49:59.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.964 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.964 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.964 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590956+0000", 2024-09-16T20:49:59.964 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9809999999999992e-06 2024-09-16T20:49:59.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.964 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.964 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.964 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.590966+0000", 2024-09-16T20:49:59.964 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4609999999999997e-06 2024-09-16T20:49:59.964 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.964 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.964 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.965 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.965 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24401 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.965 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.594325+0000", 2024-09-16T20:49:59.965 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.029288145000000002, 2024-09-16T20:49:59.965 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.031671513999999998, 2024-09-16T20:49:59.965 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.965 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.965 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.965 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.965 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:49:59.965 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:49:59.965 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24401 2024-09-16T20:49:59.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.965 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.965 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.966 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.966 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.594325+0000", 2024-09-16T20:49:59.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.966 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.966 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.594325+0000", 2024-09-16T20:49:59.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.966 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.966 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.594325+0000", 2024-09-16T20:49:59.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6899999999999996e-07 2024-09-16T20:49:59.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.967 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.967 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.594331+0000", 2024-09-16T20:49:59.967 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2440000000000001e-06 2024-09-16T20:49:59.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.967 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.967 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.594340+0000", 2024-09-16T20:49:59.967 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3370000000000006e-06 2024-09-16T20:49:59.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.967 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.967 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.594346+0000", 2024-09-16T20:49:59.967 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3929999999999996e-06 2024-09-16T20:49:59.968 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.968 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.968 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.968 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.968 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.968 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24417 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.968 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.597734+0000", 2024-09-16T20:49:59.968 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.025878791000000002, 2024-09-16T20:49:59.968 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.028299814999999999, 2024-09-16T20:49:59.968 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.968 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.968 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.968 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.969 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:49:59.969 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:49:59.969 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24417 2024-09-16T20:49:59.969 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.969 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.969 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.969 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.969 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.597734+0000", 2024-09-16T20:49:59.969 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.969 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.969 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.969 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.969 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.597734+0000", 2024-09-16T20:49:59.969 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.969 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.969 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.970 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.970 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.597735+0000", 2024-09-16T20:49:59.970 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2900000000000003e-07 2024-09-16T20:49:59.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.970 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.970 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.970 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.597737+0000", 2024-09-16T20:49:59.970 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7180000000000001e-06 2024-09-16T20:49:59.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.970 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.970 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.970 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.597746+0000", 2024-09-16T20:49:59.970 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0440000000000001e-06 2024-09-16T20:49:59.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.970 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.971 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.971 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.597750+0000", 2024-09-16T20:49:59.971 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3840000000000001e-06 2024-09-16T20:49:59.971 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.971 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.971 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.971 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.971 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24433 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.971 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.601413+0000", 2024-09-16T20:49:59.971 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.022199821000000002, 2024-09-16T20:49:59.971 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.02464473, 2024-09-16T20:49:59.971 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.971 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.971 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.972 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.972 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:49:59.972 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:49:59.972 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24433 2024-09-16T20:49:59.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.972 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.972 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.972 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.972 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.601413+0000", 2024-09-16T20:49:59.972 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.972 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.972 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.973 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.601413+0000", 2024-09-16T20:49:59.973 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.973 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.973 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.973 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.601414+0000", 2024-09-16T20:49:59.973 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6799999999999999e-07 2024-09-16T20:49:59.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.973 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.973 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.973 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.601416+0000", 2024-09-16T20:49:59.973 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7599999999999998e-06 2024-09-16T20:49:59.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.974 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.974 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.601422+0000", 2024-09-16T20:49:59.974 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9399999999999999e-06 2024-09-16T20:49:59.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.974 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.974 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.601426+0000", 2024-09-16T20:49:59.974 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4999999999999999e-06 2024-09-16T20:49:59.974 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.974 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.974 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.974 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19926 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.975 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.591162+0000", 2024-09-16T20:49:59.975 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.032450802000000001, 2024-09-16T20:49:59.975 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.034919126000000002, 2024-09-16T20:49:59.975 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.975 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.975 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.975 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.975 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:59.975 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:59.975 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19926 2024-09-16T20:49:59.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.975 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.975 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.975 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.975 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.591162+0000", 2024-09-16T20:49:59.976 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.976 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.976 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.976 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.591162+0000", 2024-09-16T20:49:59.976 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.976 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.976 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.976 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.591162+0000", 2024-09-16T20:49:59.976 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3399999999999999e-07 2024-09-16T20:49:59.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.976 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.976 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.977 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.591166+0000", 2024-09-16T20:49:59.977 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0539999999999999e-06 2024-09-16T20:49:59.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.977 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.977 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.591172+0000", 2024-09-16T20:49:59.977 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8519999999999998e-06 2024-09-16T20:49:59.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.977 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.977 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.591176+0000", 2024-09-16T20:49:59.977 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6069999999999999e-06 2024-09-16T20:49:59.978 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.978 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.978 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.978 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.978 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19942 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.978 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.594529+0000", 2024-09-16T20:49:59.978 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.029083870000000001, 2024-09-16T20:49:59.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.031575055999999997, 2024-09-16T20:49:59.978 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.978 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.978 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.978 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.979 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:59.979 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:59.979 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19942 2024-09-16T20:49:59.979 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.979 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.979 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.979 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.979 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.594529+0000", 2024-09-16T20:49:59.979 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.979 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.979 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.979 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.980 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.594529+0000", 2024-09-16T20:49:59.980 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.980 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.980 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.980 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.594530+0000", 2024-09-16T20:49:59.980 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7700000000000004e-07 2024-09-16T20:49:59.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.980 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.980 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.980 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.594534+0000", 2024-09-16T20:49:59.980 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0439999999999997e-06 2024-09-16T20:49:59.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.980 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.981 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.981 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.594542+0000", 2024-09-16T20:49:59.981 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1969999999999996e-06 2024-09-16T20:49:59.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.981 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.981 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.981 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.594548+0000", 2024-09-16T20:49:59.981 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1639999999999998e-06 2024-09-16T20:49:59.981 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.981 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.981 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.981 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.982 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19958 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.982 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.597962+0000", 2024-09-16T20:49:59.982 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.025650643000000001, 2024-09-16T20:49:59.982 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.028164707000000001, 2024-09-16T20:49:59.982 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.982 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.982 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.982 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.982 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:59.982 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:59.982 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19958 2024-09-16T20:49:59.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.982 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.982 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.983 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.983 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.597962+0000", 2024-09-16T20:49:59.983 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.983 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.983 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.983 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.597962+0000", 2024-09-16T20:49:59.983 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.983 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.983 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.983 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.597963+0000", 2024-09-16T20:49:59.983 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0500000000000002e-07 2024-09-16T20:49:59.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.984 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.984 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.984 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.597967+0000", 2024-09-16T20:49:59.984 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.126e-06 2024-09-16T20:49:59.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.984 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.984 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.984 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.597975+0000", 2024-09-16T20:49:59.984 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9699999999999999e-06 2024-09-16T20:49:59.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.984 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.984 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.984 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.597981+0000", 2024-09-16T20:49:59.984 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.015e-06 2024-09-16T20:49:59.985 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.985 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.985 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.985 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.985 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19974 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.985 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.601648+0000", 2024-09-16T20:49:59.985 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.021964999999999998, 2024-09-16T20:49:59.985 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.024503391999999999, 2024-09-16T20:49:59.985 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.985 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.985 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.985 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.985 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:49:59.986 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:49:59.986 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19974 2024-09-16T20:49:59.986 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.986 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.986 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.986 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.986 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.601648+0000", 2024-09-16T20:49:59.986 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.986 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.986 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.986 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.986 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.601648+0000", 2024-09-16T20:49:59.986 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.987 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.987 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.987 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.987 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.601649+0000", 2024-09-16T20:49:59.987 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8000000000000004e-07 2024-09-16T20:49:59.987 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.987 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.987 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.987 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.601652+0000", 2024-09-16T20:49:59.987 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3400000000000002e-06 2024-09-16T20:49:59.987 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.987 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.987 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.987 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.601659+0000", 2024-09-16T20:49:59.988 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2980000000000003e-06 2024-09-16T20:49:59.988 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.988 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.988 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.988 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.601667+0000", 2024-09-16T20:49:59.988 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4270000000000002e-06 2024-09-16T20:49:59.988 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.988 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.988 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.988 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.988 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.988 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24386 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.988 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.591185+0000", 2024-09-16T20:49:59.988 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.032427508000000001, 2024-09-16T20:49:59.989 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.034990199999999999, 2024-09-16T20:49:59.989 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.989 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.989 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.989 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.989 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:49:59.989 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:49:59.989 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24386 2024-09-16T20:49:59.989 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.989 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.989 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.989 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.989 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.591185+0000", 2024-09-16T20:49:59.989 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.990 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.990 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.990 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.990 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.591185+0000", 2024-09-16T20:49:59.990 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.990 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.990 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.990 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.990 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.591186+0000", 2024-09-16T20:49:59.990 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9500000000000001e-07 2024-09-16T20:49:59.990 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.990 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.990 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.990 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.591190+0000", 2024-09-16T20:49:59.991 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3510000000000004e-06 2024-09-16T20:49:59.991 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.991 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.991 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.991 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.591199+0000", 2024-09-16T20:49:59.991 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2930000000000001e-06 2024-09-16T20:49:59.991 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.991 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.991 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.991 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.591203+0000", 2024-09-16T20:49:59.991 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4340000000000002e-06 2024-09-16T20:49:59.991 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.991 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.991 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.992 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.992 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.992 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24402 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.992 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.594566+0000", 2024-09-16T20:49:59.992 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.029047073, 2024-09-16T20:49:59.992 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.031632661999999999, 2024-09-16T20:49:59.992 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.992 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.992 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.992 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.992 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:49:59.992 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:49:59.992 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24402 2024-09-16T20:49:59.992 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.993 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.993 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.993 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.993 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.594566+0000", 2024-09-16T20:49:59.993 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.993 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.993 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.993 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.993 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.594566+0000", 2024-09-16T20:49:59.993 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.993 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.993 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.993 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.993 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.594567+0000", 2024-09-16T20:49:59.993 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1489999999999999e-06 2024-09-16T20:49:59.994 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.994 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.994 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.994 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.594571+0000", 2024-09-16T20:49:59.994 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.087e-06 2024-09-16T20:49:59.994 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.994 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.994 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.994 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.594580+0000", 2024-09-16T20:49:59.994 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3079999999999997e-06 2024-09-16T20:49:59.994 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.994 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.994 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.994 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.594586+0000", 2024-09-16T20:49:59.994 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9900000000000002e-06 2024-09-16T20:49:59.995 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.995 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.995 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.995 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.995 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.995 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24418 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.995 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.597981+0000", 2024-09-16T20:49:59.995 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.025631887999999999, 2024-09-16T20:49:59.995 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.028240402000000001, 2024-09-16T20:49:59.995 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.995 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.995 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.995 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.995 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:49:59.996 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:49:59.996 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24418 2024-09-16T20:49:59.996 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.996 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.996 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.996 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.996 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.597981+0000", 2024-09-16T20:49:59.996 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.996 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.996 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.996 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:49:59.996 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.597981+0000", 2024-09-16T20:49:59.996 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:49:59.996 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.997 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.997 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:49:59.997 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.597982+0000", 2024-09-16T20:49:59.997 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3300000000000001e-07 2024-09-16T20:49:59.997 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.997 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.997 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:49:59.997 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.597985+0000", 2024-09-16T20:49:59.997 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9500000000000001e-06 2024-09-16T20:49:59.997 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.997 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.997 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:49:59.997 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.597990+0000", 2024-09-16T20:49:59.998 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7259999999999999e-06 2024-09-16T20:49:59.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.998 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.998 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:49:59.998 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.597994+0000", 2024-09-16T20:49:59.998 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4769999999999999e-06 2024-09-16T20:49:59.998 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.998 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:49:59.998 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:49:59.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.998 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.998 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24434 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:49:59.998 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.601671+0000", 2024-09-16T20:49:59.999 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.021942117000000001, 2024-09-16T20:49:59.999 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.024575509999999998, 2024-09-16T20:49:59.999 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:49:59.999 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:49:59.999 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:49:59.999 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:49:59.999 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:49:59.999 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:49:59.999 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24434 2024-09-16T20:49:59.999 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:49:59.999 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:49:59.999 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:49:59.999 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:49:59.999 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.601671+0000", 2024-09-16T20:50:00.000 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.000 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.000 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.000 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:00.000 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.601671+0000", 2024-09-16T20:50:00.000 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.000 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.000 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.000 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:00.000 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.601671+0000", 2024-09-16T20:50:00.000 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9999999999999997e-07 2024-09-16T20:50:00.000 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.001 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.001 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:00.001 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.601674+0000", 2024-09-16T20:50:00.001 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6019999999999998e-06 2024-09-16T20:50:00.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.001 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.001 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:00.001 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.601679+0000", 2024-09-16T20:50:00.001 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5210000000000002e-06 2024-09-16T20:50:00.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.001 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.001 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:00.001 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.601683+0000", 2024-09-16T20:50:00.001 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3529999999999999e-06 2024-09-16T20:50:00.002 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.002 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.002 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.002 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19927 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:50:00.002 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.591333+0000", 2024-09-16T20:50:00.002 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.032280201000000001, 2024-09-16T20:50:00.002 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.034936513000000002, 2024-09-16T20:50:00.002 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:00.002 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:00.002 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:00.002 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:00.002 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:50:00.003 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:00.003 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19927 2024-09-16T20:50:00.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.003 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:00.003 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.003 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:00.003 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.591333+0000", 2024-09-16T20:50:00.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.003 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.003 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:00.003 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.591333+0000", 2024-09-16T20:50:00.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.004 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.004 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:00.004 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.591333+0000", 2024-09-16T20:50:00.004 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1600000000000001e-07 2024-09-16T20:50:00.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.004 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.004 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:00.004 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.591336+0000", 2024-09-16T20:50:00.004 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3400000000000002e-06 2024-09-16T20:50:00.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.004 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.004 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:00.005 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.591346+0000", 2024-09-16T20:50:00.005 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5959999999999993e-06 2024-09-16T20:50:00.005 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.005 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.005 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:00.005 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.591355+0000", 2024-09-16T20:50:00.005 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2480000000000002e-06 2024-09-16T20:50:00.005 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.005 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.005 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.005 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.005 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.005 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19943 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:50:00.006 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.594715+0000", 2024-09-16T20:50:00.006 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.028897910999999998, 2024-09-16T20:50:00.006 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.031576871999999999, 2024-09-16T20:50:00.006 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:00.006 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:00.006 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:00.006 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:00.006 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:50:00.006 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:00.006 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19943 2024-09-16T20:50:00.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.007 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:00.007 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.007 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:00.007 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.594715+0000", 2024-09-16T20:50:00.007 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.007 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.007 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.007 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:00.007 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.594715+0000", 2024-09-16T20:50:00.007 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.007 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.007 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.007 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:00.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.594716+0000", 2024-09-16T20:50:00.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8500000000000001e-07 2024-09-16T20:50:00.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.008 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:00.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.594719+0000", 2024-09-16T20:50:00.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9770000000000002e-06 2024-09-16T20:50:00.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.008 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:00.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.594727+0000", 2024-09-16T20:50:00.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1790000000000002e-06 2024-09-16T20:50:00.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.009 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.009 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:00.009 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.594732+0000", 2024-09-16T20:50:00.009 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6799999999999998e-06 2024-09-16T20:50:00.009 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.009 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.009 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.009 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.009 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19959 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:50:00.009 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.598161+0000", 2024-09-16T20:50:00.009 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.025452170999999999, 2024-09-16T20:50:00.010 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.028162518000000001, 2024-09-16T20:50:00.010 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:00.010 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:00.010 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:00.010 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:00.010 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:50:00.010 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:00.010 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19959 2024-09-16T20:50:00.010 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.010 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:00.010 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.010 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:00.010 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.598161+0000", 2024-09-16T20:50:00.011 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.011 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.011 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:00.011 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.598161+0000", 2024-09-16T20:50:00.011 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.011 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.011 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:00.011 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.598161+0000", 2024-09-16T20:50:00.011 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3499999999999995e-07 2024-09-16T20:50:00.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.011 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.011 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:00.011 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.598165+0000", 2024-09-16T20:50:00.012 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6200000000000001e-06 2024-09-16T20:50:00.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.012 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.012 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:00.012 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.598175+0000", 2024-09-16T20:50:00.012 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7690000000000008e-06 2024-09-16T20:50:00.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.012 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.012 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:00.012 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.598180+0000", 2024-09-16T20:50:00.012 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1959999999999999e-06 2024-09-16T20:50:00.012 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.012 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.013 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.013 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.013 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19975 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:50:00.013 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.601863+0000", 2024-09-16T20:50:00.013 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.021750127000000001, 2024-09-16T20:50:00.013 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.024494364000000001, 2024-09-16T20:50:00.013 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:00.013 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:00.013 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:00.013 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:00.013 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:50:00.013 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:00.013 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19975 2024-09-16T20:50:00.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.014 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:00.014 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.014 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:00.014 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.601863+0000", 2024-09-16T20:50:00.014 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.014 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.014 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:00.014 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.601863+0000", 2024-09-16T20:50:00.014 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.014 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.015 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:00.015 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.601863+0000", 2024-09-16T20:50:00.015 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.61e-07 2024-09-16T20:50:00.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.015 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.015 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:00.015 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.601867+0000", 2024-09-16T20:50:00.015 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5530000000000001e-06 2024-09-16T20:50:00.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.015 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.015 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:00.015 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.601875+0000", 2024-09-16T20:50:00.015 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6659999999999997e-06 2024-09-16T20:50:00.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.016 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.016 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:00.016 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.601881+0000", 2024-09-16T20:50:00.016 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6409999999999999e-06 2024-09-16T20:50:00.016 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.016 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.016 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.016 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.016 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24387 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:50:00.016 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.591341+0000", 2024-09-16T20:50:00.016 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.032271251000000001, 2024-09-16T20:50:00.017 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.035039832, 2024-09-16T20:50:00.017 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:00.017 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:00.017 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:00.017 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:00.017 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:50:00.017 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:50:00.017 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24387 2024-09-16T20:50:00.017 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.017 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:00.017 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.017 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:00.017 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.591341+0000", 2024-09-16T20:50:00.018 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.018 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.018 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:00.018 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.591341+0000", 2024-09-16T20:50:00.018 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.018 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.018 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:00.018 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.591342+0000", 2024-09-16T20:50:00.018 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8999999999999996e-07 2024-09-16T20:50:00.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.018 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.019 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:00.019 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.591345+0000", 2024-09-16T20:50:00.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2940000000000001e-06 2024-09-16T20:50:00.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.019 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:00.019 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.591358+0000", 2024-09-16T20:50:00.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2183e-05 2024-09-16T20:50:00.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.019 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:00.019 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.591364+0000", 2024-09-16T20:50:00.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.263e-06 2024-09-16T20:50:00.019 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.020 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.020 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.020 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.020 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.020 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24403 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:50:00.020 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.594742+0000", 2024-09-16T20:50:00.020 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.028870285999999998, 2024-09-16T20:50:00.020 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.031661770999999998, 2024-09-16T20:50:00.020 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:00.020 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:00.020 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:00.020 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:00.020 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:50:00.021 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:50:00.021 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24403 2024-09-16T20:50:00.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.021 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:00.021 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.021 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:00.021 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.594742+0000", 2024-09-16T20:50:00.021 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.021 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.021 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:00.021 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.594742+0000", 2024-09-16T20:50:00.021 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.022 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.022 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:00.022 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.594743+0000", 2024-09-16T20:50:00.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5499999999999996e-07 2024-09-16T20:50:00.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.022 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.022 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:00.022 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.594747+0000", 2024-09-16T20:50:00.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8460000000000001e-06 2024-09-16T20:50:00.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.022 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.022 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:00.022 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.594756+0000", 2024-09-16T20:50:00.023 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0450000000000006e-06 2024-09-16T20:50:00.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.023 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.023 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:00.023 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.594763+0000", 2024-09-16T20:50:00.023 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6370000000000002e-06 2024-09-16T20:50:00.023 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.023 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.023 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.023 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.023 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24419 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:50:00.023 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.598177+0000", 2024-09-16T20:50:00.024 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.025435896, 2024-09-16T20:50:00.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.028255690999999999, 2024-09-16T20:50:00.024 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:00.024 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:00.024 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:00.024 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:00.024 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:50:00.024 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:50:00.024 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24419 2024-09-16T20:50:00.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.024 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:00.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.024 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:00.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.598177+0000", 2024-09-16T20:50:00.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:00.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.598177+0000", 2024-09-16T20:50:00.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:00.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.598178+0000", 2024-09-16T20:50:00.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6899999999999996e-07 2024-09-16T20:50:00.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.026 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:00.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.598180+0000", 2024-09-16T20:50:00.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4820000000000001e-06 2024-09-16T20:50:00.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.026 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:00.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.598188+0000", 2024-09-16T20:50:00.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7489999999999998e-06 2024-09-16T20:50:00.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.026 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:00.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.598191+0000", 2024-09-16T20:50:00.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3960000000000002e-06 2024-09-16T20:50:00.027 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.027 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.027 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.027 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.027 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24435 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:50:00.027 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.601881+0000", 2024-09-16T20:50:00.027 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.021732214999999999, 2024-09-16T20:50:00.027 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.024576768999999998, 2024-09-16T20:50:00.027 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:00.027 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:00.027 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:00.027 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:00.027 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:50:00.028 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:50:00.028 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24435 2024-09-16T20:50:00.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.028 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:00.028 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.028 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:00.028 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.601881+0000", 2024-09-16T20:50:00.028 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.028 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.028 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:00.028 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.601881+0000", 2024-09-16T20:50:00.029 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.029 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:00.029 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.601881+0000", 2024-09-16T20:50:00.029 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6599999999999995e-07 2024-09-16T20:50:00.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.029 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:00.029 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.601884+0000", 2024-09-16T20:50:00.029 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.553e-06 2024-09-16T20:50:00.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.030 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:00.030 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.601889+0000", 2024-09-16T20:50:00.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.22e-06 2024-09-16T20:50:00.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.030 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:00.030 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.601892+0000", 2024-09-16T20:50:00.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3110000000000001e-06 2024-09-16T20:50:00.030 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.030 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.030 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.031 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.031 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19928 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:50:00.031 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.591581+0000", 2024-09-16T20:50:00.031 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.032032222999999999, 2024-09-16T20:50:00.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.034900028999999999, 2024-09-16T20:50:00.031 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:00.031 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:00.031 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:00.031 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:00.031 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:50:00.031 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:00.031 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19928 2024-09-16T20:50:00.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.032 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:00.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.032 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:00.032 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.591581+0000", 2024-09-16T20:50:00.032 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.032 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:00.032 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.591581+0000", 2024-09-16T20:50:00.032 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.032 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:00.033 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.591581+0000", 2024-09-16T20:50:00.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2399999999999997e-07 2024-09-16T20:50:00.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.033 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.033 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:00.033 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.591585+0000", 2024-09-16T20:50:00.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7160000000000001e-06 2024-09-16T20:50:00.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.033 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.033 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:00.033 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.591596+0000", 2024-09-16T20:50:00.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1351e-05 2024-09-16T20:50:00.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.034 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.034 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:00.034 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.591603+0000", 2024-09-16T20:50:00.034 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2190000000000001e-06 2024-09-16T20:50:00.034 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.034 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.034 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.034 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.034 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19944 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:50:00.034 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.594938+0000", 2024-09-16T20:50:00.034 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.028674445999999999, 2024-09-16T20:50:00.034 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.031565337999999998, 2024-09-16T20:50:00.034 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:00.035 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:00.035 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:00.035 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:00.035 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:50:00.035 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:00.035 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19944 2024-09-16T20:50:00.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.035 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:00.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.035 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:00.035 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.594938+0000", 2024-09-16T20:50:00.035 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.036 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:00.036 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.594938+0000", 2024-09-16T20:50:00.036 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.036 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:00.036 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.594939+0000", 2024-09-16T20:50:00.036 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4499999999999996e-07 2024-09-16T20:50:00.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.036 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:00.036 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.594943+0000", 2024-09-16T20:50:00.037 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9550000000000002e-06 2024-09-16T20:50:00.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.037 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.037 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:00.037 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.594952+0000", 2024-09-16T20:50:00.037 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8789999999999992e-06 2024-09-16T20:50:00.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.037 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.037 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:00.037 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.594958+0000", 2024-09-16T20:50:00.037 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6489999999999996e-06 2024-09-16T20:50:00.037 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.037 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.038 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.038 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.038 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19960 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:50:00.038 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.598413+0000", 2024-09-16T20:50:00.038 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.025199552, 2024-09-16T20:50:00.038 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.028113457000000001, 2024-09-16T20:50:00.038 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:00.038 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:00.038 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:00.038 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:00.038 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:50:00.038 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:00.039 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19960 2024-09-16T20:50:00.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.039 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:00.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.039 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:00.039 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.598413+0000", 2024-09-16T20:50:00.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.039 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:00.039 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.598413+0000", 2024-09-16T20:50:00.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.040 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.040 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:00.040 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.598414+0000", 2024-09-16T20:50:00.040 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0900000000000001e-07 2024-09-16T20:50:00.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.040 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.040 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:00.040 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.598417+0000", 2024-09-16T20:50:00.040 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0029999999999999e-06 2024-09-16T20:50:00.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.040 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.040 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:00.040 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.598427+0000", 2024-09-16T20:50:00.041 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0067e-05 2024-09-16T20:50:00.041 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.041 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.041 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:00.041 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.598434+0000", 2024-09-16T20:50:00.041 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2749999999999998e-06 2024-09-16T20:50:00.041 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.041 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.041 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.041 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.041 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.041 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19976 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:50:00.041 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.602114+0000", 2024-09-16T20:50:00.042 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.021498769000000001, 2024-09-16T20:50:00.042 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.024435808999999999, 2024-09-16T20:50:00.042 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:00.042 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:00.042 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:00.042 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:00.042 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:50:00.042 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:00.042 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19976 2024-09-16T20:50:00.042 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.042 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:00.042 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.042 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:00.042 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602114+0000", 2024-09-16T20:50:00.043 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.043 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.043 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:00.043 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602114+0000", 2024-09-16T20:50:00.043 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.043 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.043 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:00.043 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602115+0000", 2024-09-16T20:50:00.043 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0699999999999999e-06 2024-09-16T20:50:00.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.043 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.043 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:00.044 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602120+0000", 2024-09-16T20:50:00.044 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3240000000000002e-06 2024-09-16T20:50:00.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.044 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.044 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:00.044 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602130+0000", 2024-09-16T20:50:00.044 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8800000000000003e-06 2024-09-16T20:50:00.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.044 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.044 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:00.044 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602136+0000", 2024-09-16T20:50:00.044 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0750000000000003e-06 2024-09-16T20:50:00.044 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.045 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.045 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.045 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.045 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.045 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24388 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:50:00.045 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.591599+0000", 2024-09-16T20:50:00.045 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.032013523000000002, 2024-09-16T20:50:00.045 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.034973767000000003, 2024-09-16T20:50:00.045 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:00.045 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:00.045 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:00.045 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:00.045 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:50:00.045 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:50:00.046 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24388 2024-09-16T20:50:00.046 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.046 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:00.046 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.046 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:00.046 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.591599+0000", 2024-09-16T20:50:00.046 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.046 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.046 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.046 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:00.046 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.591599+0000", 2024-09-16T20:50:00.046 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.046 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.047 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.047 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:00.047 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.591600+0000", 2024-09-16T20:50:00.047 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2500000000000005e-07 2024-09-16T20:50:00.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.047 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.047 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:00.047 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.591604+0000", 2024-09-16T20:50:00.047 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5559999999999999e-06 2024-09-16T20:50:00.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.047 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.047 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:00.047 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.591611+0000", 2024-09-16T20:50:00.047 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6109999999999996e-06 2024-09-16T20:50:00.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.048 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.048 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:00.048 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.591617+0000", 2024-09-16T20:50:00.048 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6609999999999997e-06 2024-09-16T20:50:00.048 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.048 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.048 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.048 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.048 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24404 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:50:00.048 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.594963+0000", 2024-09-16T20:50:00.048 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.028649621, 2024-09-16T20:50:00.049 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.031633171000000002, 2024-09-16T20:50:00.049 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:00.049 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:00.049 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:00.049 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:00.049 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:50:00.049 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:50:00.049 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24404 2024-09-16T20:50:00.049 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.049 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:00.049 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.049 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:00.049 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.594963+0000", 2024-09-16T20:50:00.049 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.050 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.050 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.050 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:00.050 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.594963+0000", 2024-09-16T20:50:00.050 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.050 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.050 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.050 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:00.050 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.594964+0000", 2024-09-16T20:50:00.050 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.850000000000001e-07 2024-09-16T20:50:00.050 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.050 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.050 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:00.050 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.594968+0000", 2024-09-16T20:50:00.051 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9750000000000001e-06 2024-09-16T20:50:00.051 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.051 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.051 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:00.051 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.594977+0000", 2024-09-16T20:50:00.051 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9230000000000008e-06 2024-09-16T20:50:00.051 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.051 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.051 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:00.051 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.594984+0000", 2024-09-16T20:50:00.051 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5350000000000001e-06 2024-09-16T20:50:00.051 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.051 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.052 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.052 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.052 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24420 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:50:00.052 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.598435+0000", 2024-09-16T20:50:00.052 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.025178072999999999, 2024-09-16T20:50:00.052 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.028185042, 2024-09-16T20:50:00.052 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:00.052 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:00.052 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:00.052 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:00.052 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:50:00.052 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:50:00.052 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24420 2024-09-16T20:50:00.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.053 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:00.053 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.053 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:00.053 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.598435+0000", 2024-09-16T20:50:00.053 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.053 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.053 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.053 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:00.053 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.598435+0000", 2024-09-16T20:50:00.053 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.053 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.053 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.053 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:00.053 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.598435+0000", 2024-09-16T20:50:00.054 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8299999999999996e-07 2024-09-16T20:50:00.054 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.054 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.054 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:00.054 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.598438+0000", 2024-09-16T20:50:00.054 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7030000000000002e-06 2024-09-16T20:50:00.054 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.054 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.054 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:00.054 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.598444+0000", 2024-09-16T20:50:00.054 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.728e-06 2024-09-16T20:50:00.054 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.054 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.054 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:00.055 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.598447+0000", 2024-09-16T20:50:00.055 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.45e-06 2024-09-16T20:50:00.055 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.055 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.055 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.055 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.055 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.055 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24436 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:50:00.055 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.602129+0000", 2024-09-16T20:50:00.055 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.021484092, 2024-09-16T20:50:00.055 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.024526755000000001, 2024-09-16T20:50:00.055 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:00.055 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:00.055 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:00.055 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:00.056 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:50:00.056 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:50:00.056 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24436 2024-09-16T20:50:00.056 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.056 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:00.056 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.056 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:00.056 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602129+0000", 2024-09-16T20:50:00.056 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.056 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.056 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.056 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:00.056 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602129+0000", 2024-09-16T20:50:00.057 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.057 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.057 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:00.057 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602129+0000", 2024-09-16T20:50:00.057 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2699999999999999e-07 2024-09-16T20:50:00.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.057 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.057 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:00.057 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602132+0000", 2024-09-16T20:50:00.057 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6680000000000001e-06 2024-09-16T20:50:00.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.057 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.058 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:00.058 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602138+0000", 2024-09-16T20:50:00.058 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8000000000000004e-06 2024-09-16T20:50:00.058 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.058 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.058 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:00.058 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602141+0000", 2024-09-16T20:50:00.058 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.495e-06 2024-09-16T20:50:00.058 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.058 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.058 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.058 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.058 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.058 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19929 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:50:00.059 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.591779+0000", 2024-09-16T20:50:00.059 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.031834229999999998, 2024-09-16T20:50:00.059 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.034900891000000003, 2024-09-16T20:50:00.059 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:00.059 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:00.059 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:00.059 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:00.059 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:50:00.059 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:00.059 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19929 2024-09-16T20:50:00.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.059 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:00.059 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.060 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:00.060 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.591779+0000", 2024-09-16T20:50:00.060 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.060 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.060 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.060 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:00.060 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.591779+0000", 2024-09-16T20:50:00.060 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.060 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.060 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.060 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:00.060 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.591779+0000", 2024-09-16T20:50:00.060 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.82e-07 2024-09-16T20:50:00.061 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.061 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.061 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:00.061 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.591782+0000", 2024-09-16T20:50:00.061 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1089999999999998e-06 2024-09-16T20:50:00.061 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.061 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.061 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:00.061 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.591792+0000", 2024-09-16T20:50:00.061 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3870000000000001e-06 2024-09-16T20:50:00.061 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.061 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.061 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:00.061 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.591802+0000", 2024-09-16T20:50:00.062 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0236e-05 2024-09-16T20:50:00.062 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.062 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.062 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.062 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.062 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.062 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19945 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:50:00.062 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.595127+0000", 2024-09-16T20:50:00.062 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.028485453000000001, 2024-09-16T20:50:00.062 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.031575239999999997, 2024-09-16T20:50:00.062 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:00.062 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:00.062 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:00.063 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:00.063 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:50:00.063 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:00.063 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19945 2024-09-16T20:50:00.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.063 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:00.063 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.063 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:00.063 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.595127+0000", 2024-09-16T20:50:00.063 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.063 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.063 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:00.063 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.595127+0000", 2024-09-16T20:50:00.063 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.064 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:00.064 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.595128+0000", 2024-09-16T20:50:00.064 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8599999999999998e-07 2024-09-16T20:50:00.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.064 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:00.064 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.595132+0000", 2024-09-16T20:50:00.064 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6009999999999999e-06 2024-09-16T20:50:00.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.064 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:00.065 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.595140+0000", 2024-09-16T20:50:00.065 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7180000000000002e-06 2024-09-16T20:50:00.065 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.065 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.065 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:00.065 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.595145+0000", 2024-09-16T20:50:00.065 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3640000000000001e-06 2024-09-16T20:50:00.065 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.065 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.065 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.065 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.065 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.065 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19961 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:50:00.066 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.598622+0000", 2024-09-16T20:50:00.066 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.024990951000000001, 2024-09-16T20:50:00.066 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.028103512000000001, 2024-09-16T20:50:00.066 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:00.066 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:00.066 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:00.066 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:00.066 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:50:00.066 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:00.066 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19961 2024-09-16T20:50:00.066 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.066 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:00.066 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.066 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:00.066 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.598622+0000", 2024-09-16T20:50:00.067 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.067 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.067 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.067 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:00.067 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.598622+0000", 2024-09-16T20:50:00.067 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.067 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.067 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.067 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:00.067 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.598622+0000", 2024-09-16T20:50:00.067 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.06e-07 2024-09-16T20:50:00.067 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.067 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.067 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:00.067 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.598626+0000", 2024-09-16T20:50:00.068 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7239999999999998e-06 2024-09-16T20:50:00.068 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.068 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.068 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:00.068 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.598635+0000", 2024-09-16T20:50:00.068 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6619999999999994e-06 2024-09-16T20:50:00.068 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.068 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.068 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:00.068 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.598639+0000", 2024-09-16T20:50:00.068 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0999999999999997e-06 2024-09-16T20:50:00.068 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.068 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.069 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.069 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.069 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19977 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:50:00.069 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.602321+0000", 2024-09-16T20:50:00.069 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.021292102, 2024-09-16T20:50:00.069 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.024427535, 2024-09-16T20:50:00.069 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:00.069 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:00.069 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:00.069 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:00.069 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:50:00.069 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:00.070 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19977 2024-09-16T20:50:00.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.070 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:00.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.070 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:00.070 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602321+0000", 2024-09-16T20:50:00.070 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.070 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:00.070 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602321+0000", 2024-09-16T20:50:00.070 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.071 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:00.071 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602322+0000", 2024-09-16T20:50:00.071 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9100000000000002e-07 2024-09-16T20:50:00.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.071 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.071 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:00.071 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602326+0000", 2024-09-16T20:50:00.071 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6090000000000003e-06 2024-09-16T20:50:00.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.071 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.071 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:00.071 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602336+0000", 2024-09-16T20:50:00.071 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3619999999999995e-06 2024-09-16T20:50:00.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.072 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.072 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:00.072 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602340+0000", 2024-09-16T20:50:00.072 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0890000000000002e-06 2024-09-16T20:50:00.072 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.072 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.072 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.072 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.072 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.072 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24389 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:50:00.072 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.591791+0000", 2024-09-16T20:50:00.072 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.031821541000000002, 2024-09-16T20:50:00.072 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.034979728000000002, 2024-09-16T20:50:00.072 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:00.073 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:00.073 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:00.073 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:00.073 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:50:00.073 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:50:00.073 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24389 2024-09-16T20:50:00.073 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.073 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:00.073 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.073 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:00.073 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.591791+0000", 2024-09-16T20:50:00.073 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.073 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.073 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.074 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:00.074 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.591791+0000", 2024-09-16T20:50:00.074 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.074 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.074 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:00.074 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.591792+0000", 2024-09-16T20:50:00.074 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1200000000000003e-07 2024-09-16T20:50:00.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.074 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.074 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:00.074 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.591795+0000", 2024-09-16T20:50:00.074 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6260000000000001e-06 2024-09-16T20:50:00.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.074 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.075 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:00.075 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.591803+0000", 2024-09-16T20:50:00.075 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2520000000000002e-06 2024-09-16T20:50:00.075 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.075 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.075 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:00.075 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.591808+0000", 2024-09-16T20:50:00.075 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0180000000000001e-06 2024-09-16T20:50:00.075 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.075 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.075 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.075 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.075 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.075 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24405 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:50:00.076 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.595153+0000", 2024-09-16T20:50:00.076 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.02845984, 2024-09-16T20:50:00.076 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.031640898000000001, 2024-09-16T20:50:00.076 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:00.076 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:00.076 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:00.076 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:00.076 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:50:00.076 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:50:00.076 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24405 2024-09-16T20:50:00.076 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.076 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:00.076 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.077 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:00.077 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.595153+0000", 2024-09-16T20:50:00.077 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.077 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.077 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:00.077 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.595153+0000", 2024-09-16T20:50:00.077 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.077 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.077 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:00.077 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.595154+0000", 2024-09-16T20:50:00.077 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0249999999999999e-06 2024-09-16T20:50:00.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.078 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.078 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:00.078 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.595158+0000", 2024-09-16T20:50:00.078 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2810000000000002e-06 2024-09-16T20:50:00.078 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.078 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.078 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:00.078 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.595168+0000", 2024-09-16T20:50:00.078 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7899999999999994e-06 2024-09-16T20:50:00.078 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.078 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.078 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:00.078 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.595176+0000", 2024-09-16T20:50:00.079 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6750000000000002e-06 2024-09-16T20:50:00.079 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.079 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.079 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.079 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.079 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.079 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24421 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:50:00.079 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.598634+0000", 2024-09-16T20:50:00.079 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.024979154, 2024-09-16T20:50:00.079 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.028182724999999999, 2024-09-16T20:50:00.079 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:00.079 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:00.079 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:00.079 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:00.079 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:50:00.080 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:50:00.080 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24421 2024-09-16T20:50:00.080 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.080 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:00.080 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.080 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:00.080 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.598634+0000", 2024-09-16T20:50:00.080 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.080 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.080 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.080 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:00.080 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.598634+0000", 2024-09-16T20:50:00.081 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.081 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.081 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.081 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:00.081 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.598634+0000", 2024-09-16T20:50:00.081 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9999999999999996e-07 2024-09-16T20:50:00.081 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.081 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.081 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:00.081 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.598637+0000", 2024-09-16T20:50:00.081 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.915e-06 2024-09-16T20:50:00.081 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.082 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.082 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:00.082 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.598642+0000", 2024-09-16T20:50:00.082 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0590000000000002e-06 2024-09-16T20:50:00.082 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.082 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.082 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:00.082 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.598646+0000", 2024-09-16T20:50:00.082 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3450000000000002e-06 2024-09-16T20:50:00.082 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.082 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.082 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.082 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.083 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.083 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24437 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:50:00.083 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.602331+0000", 2024-09-16T20:50:00.083 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.021282183, 2024-09-16T20:50:00.083 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.024508388999999998, 2024-09-16T20:50:00.083 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:00.083 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:00.083 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:00.083 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:00.083 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:50:00.083 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:50:00.083 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24437 2024-09-16T20:50:00.083 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.083 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:00.084 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.084 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:00.084 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602331+0000", 2024-09-16T20:50:00.084 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.084 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.084 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.084 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:00.084 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602331+0000", 2024-09-16T20:50:00.084 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.084 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.084 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.084 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:00.084 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602331+0000", 2024-09-16T20:50:00.085 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8999999999999995e-07 2024-09-16T20:50:00.085 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.085 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.085 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:00.085 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602334+0000", 2024-09-16T20:50:00.085 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.021e-06 2024-09-16T20:50:00.085 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.085 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.085 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:00.085 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602340+0000", 2024-09-16T20:50:00.085 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0090000000000004e-06 2024-09-16T20:50:00.085 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.086 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.086 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:00.086 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602344+0000", 2024-09-16T20:50:00.086 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7910000000000002e-06 2024-09-16T20:50:00.086 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.086 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.086 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.086 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.086 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19930 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:50:00.086 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.592030+0000", 2024-09-16T20:50:00.086 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.031582514999999999, 2024-09-16T20:50:00.086 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.034831763000000002, 2024-09-16T20:50:00.086 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:00.086 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:00.087 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:00.087 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:00.087 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:50:00.087 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:00.087 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19930 2024-09-16T20:50:00.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.087 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:00.087 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.087 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:00.087 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.592030+0000", 2024-09-16T20:50:00.087 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.087 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.087 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:00.087 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.592030+0000", 2024-09-16T20:50:00.088 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.088 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.088 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.088 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:00.088 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.592031+0000", 2024-09-16T20:50:00.088 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4300000000000003e-07 2024-09-16T20:50:00.088 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.088 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.088 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:00.088 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.592034+0000", 2024-09-16T20:50:00.088 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.061e-06 2024-09-16T20:50:00.088 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.088 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.089 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:00.089 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.592045+0000", 2024-09-16T20:50:00.089 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0886999999999999e-05 2024-09-16T20:50:00.089 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.089 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.089 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:00.089 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.592050+0000", 2024-09-16T20:50:00.089 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6869999999999999e-06 2024-09-16T20:50:00.089 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.089 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.089 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.089 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.089 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.090 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19946 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:50:00.090 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.595357+0000", 2024-09-16T20:50:00.090 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.028255398000000001, 2024-09-16T20:50:00.090 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.031527643000000001, 2024-09-16T20:50:00.090 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:00.090 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:00.090 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:00.090 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:00.090 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:50:00.090 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:00.090 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19946 2024-09-16T20:50:00.090 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.090 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:00.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:00.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.595357+0000", 2024-09-16T20:50:00.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:00.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.595357+0000", 2024-09-16T20:50:00.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:00.092 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.595358+0000", 2024-09-16T20:50:00.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4700000000000001e-07 2024-09-16T20:50:00.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.092 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.092 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:00.092 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.595361+0000", 2024-09-16T20:50:00.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.835e-06 2024-09-16T20:50:00.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.092 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.092 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:00.092 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.595368+0000", 2024-09-16T20:50:00.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1470000000000004e-06 2024-09-16T20:50:00.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.093 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.093 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:00.093 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.595376+0000", 2024-09-16T20:50:00.093 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4109999999999996e-06 2024-09-16T20:50:00.093 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.093 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.093 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.093 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.093 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.093 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19962 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:50:00.093 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.598864+0000", 2024-09-16T20:50:00.093 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.024749205, 2024-09-16T20:50:00.094 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.028044659999999999, 2024-09-16T20:50:00.094 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:00.094 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:00.094 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:00.094 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:00.094 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:50:00.094 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:00.094 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19962 2024-09-16T20:50:00.094 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.094 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:00.094 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.094 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:00.094 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.598864+0000", 2024-09-16T20:50:00.094 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.094 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.095 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:00.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.598864+0000", 2024-09-16T20:50:00.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.095 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:00.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.598864+0000", 2024-09-16T20:50:00.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.23e-07 2024-09-16T20:50:00.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.095 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:00.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.598868+0000", 2024-09-16T20:50:00.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7570000000000002e-06 2024-09-16T20:50:00.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.095 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.096 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:00.096 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.598878+0000", 2024-09-16T20:50:00.096 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8789999999999998e-06 2024-09-16T20:50:00.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.096 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.096 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:00.096 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.598885+0000", 2024-09-16T20:50:00.096 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3810000000000001e-06 2024-09-16T20:50:00.096 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.096 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.096 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.096 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.096 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19978 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:50:00.097 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.602483+0000", 2024-09-16T20:50:00.097 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.021129271000000002, 2024-09-16T20:50:00.097 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.024448025000000002, 2024-09-16T20:50:00.097 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:00.097 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:00.097 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:00.097 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:00.097 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:50:00.097 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:00.097 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19978 2024-09-16T20:50:00.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.097 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:00.097 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.097 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:00.097 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602483+0000", 2024-09-16T20:50:00.098 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.098 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.098 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.098 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:00.098 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602483+0000", 2024-09-16T20:50:00.098 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.098 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.098 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.098 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:00.098 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602484+0000", 2024-09-16T20:50:00.098 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2799999999999995e-07 2024-09-16T20:50:00.098 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.098 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.098 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:00.099 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602489+0000", 2024-09-16T20:50:00.099 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2339999999999996e-06 2024-09-16T20:50:00.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.099 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.099 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:00.099 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602498+0000", 2024-09-16T20:50:00.099 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2990000000000002e-06 2024-09-16T20:50:00.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.099 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.099 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:00.099 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602504+0000", 2024-09-16T20:50:00.099 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9869999999999996e-06 2024-09-16T20:50:00.100 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.100 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.100 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.100 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.100 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.100 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24390 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:50:00.100 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.592054+0000", 2024-09-16T20:50:00.100 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.031559115999999998, 2024-09-16T20:50:00.100 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.034912550000000001, 2024-09-16T20:50:00.100 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:00.100 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:00.100 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:00.100 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:00.100 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:50:00.100 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:50:00.101 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24390 2024-09-16T20:50:00.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.101 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:00.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.101 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:00.101 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.592054+0000", 2024-09-16T20:50:00.101 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.101 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:00.101 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.592054+0000", 2024-09-16T20:50:00.101 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.102 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:00.102 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.592054+0000", 2024-09-16T20:50:00.102 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2200000000000003e-07 2024-09-16T20:50:00.102 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.102 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.102 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:00.102 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.592058+0000", 2024-09-16T20:50:00.102 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7170000000000002e-06 2024-09-16T20:50:00.102 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.102 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.102 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:00.102 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.592066+0000", 2024-09-16T20:50:00.102 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0630000000000006e-06 2024-09-16T20:50:00.102 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.103 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.103 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:00.103 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.592072+0000", 2024-09-16T20:50:00.103 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9089999999999997e-06 2024-09-16T20:50:00.103 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.103 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.103 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.103 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.103 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.103 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24406 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:50:00.103 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.595391+0000", 2024-09-16T20:50:00.103 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.028221929, 2024-09-16T20:50:00.103 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.031602922999999998, 2024-09-16T20:50:00.104 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:00.104 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:00.104 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:00.104 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:00.104 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:50:00.104 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:50:00.104 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24406 2024-09-16T20:50:00.104 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.104 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:00.104 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.104 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:00.104 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.595391+0000", 2024-09-16T20:50:00.104 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.105 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:00.105 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.595391+0000", 2024-09-16T20:50:00.105 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.105 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:00.105 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.595392+0000", 2024-09-16T20:50:00.105 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.78e-07 2024-09-16T20:50:00.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.105 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:00.105 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.595396+0000", 2024-09-16T20:50:00.106 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9149999999999998e-06 2024-09-16T20:50:00.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.106 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:00.106 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.595405+0000", 2024-09-16T20:50:00.106 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2760000000000006e-06 2024-09-16T20:50:00.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.106 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:00.106 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.595411+0000", 2024-09-16T20:50:00.106 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2620000000000004e-06 2024-09-16T20:50:00.106 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.106 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.106 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.107 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24422 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:50:00.107 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.598884+0000", 2024-09-16T20:50:00.107 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.024728446000000001, 2024-09-16T20:50:00.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.028133299000000001, 2024-09-16T20:50:00.107 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:00.107 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:00.107 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:00.107 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:00.107 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:50:00.107 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:50:00.107 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24422 2024-09-16T20:50:00.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.107 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:00.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.107 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:00.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.598884+0000", 2024-09-16T20:50:00.108 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:00.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.598884+0000", 2024-09-16T20:50:00.108 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:00.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.598885+0000", 2024-09-16T20:50:00.108 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8800000000000002e-07 2024-09-16T20:50:00.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:00.109 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.598888+0000", 2024-09-16T20:50:00.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.61e-06 2024-09-16T20:50:00.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.109 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.109 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:00.109 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.598893+0000", 2024-09-16T20:50:00.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1250000000000001e-06 2024-09-16T20:50:00.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.109 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.109 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:00.109 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.598896+0000", 2024-09-16T20:50:00.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.292e-06 2024-09-16T20:50:00.109 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.109 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.110 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.110 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.110 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.110 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24438 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:50:00.110 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.602505+0000", 2024-09-16T20:50:00.110 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.021107330000000001, 2024-09-16T20:50:00.110 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.024560458, 2024-09-16T20:50:00.110 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:00.110 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:00.110 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:00.110 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:00.110 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:50:00.110 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:50:00.110 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24438 2024-09-16T20:50:00.111 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.111 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:00.111 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.111 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:00.111 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602505+0000", 2024-09-16T20:50:00.111 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.111 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.111 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.111 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:00.111 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602505+0000", 2024-09-16T20:50:00.111 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.111 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.111 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.111 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:00.112 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602506+0000", 2024-09-16T20:50:00.112 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4499999999999997e-07 2024-09-16T20:50:00.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.112 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:00.112 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602509+0000", 2024-09-16T20:50:00.112 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.553e-06 2024-09-16T20:50:00.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.112 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:00.112 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602514+0000", 2024-09-16T20:50:00.112 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2820000000000004e-06 2024-09-16T20:50:00.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.112 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:00.113 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602517+0000", 2024-09-16T20:50:00.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2770000000000001e-06 2024-09-16T20:50:00.113 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.113 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.113 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.113 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.113 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19931 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:50:00.113 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.592210+0000", 2024-09-16T20:50:00.113 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.031402260000000001, 2024-09-16T20:50:00.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.034879638999999997, 2024-09-16T20:50:00.113 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:00.113 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:00.113 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:00.113 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:00.114 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:50:00.114 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:00.114 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19931 2024-09-16T20:50:00.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.114 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:00.114 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.114 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:00.114 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.592210+0000", 2024-09-16T20:50:00.114 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.114 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.114 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:00.114 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.592210+0000", 2024-09-16T20:50:00.114 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.115 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.115 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:00.115 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.592211+0000", 2024-09-16T20:50:00.115 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9200000000000001e-07 2024-09-16T20:50:00.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.115 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.115 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:00.115 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.592214+0000", 2024-09-16T20:50:00.115 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0319999999999999e-06 2024-09-16T20:50:00.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.115 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.115 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:00.115 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.592224+0000", 2024-09-16T20:50:00.116 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5519999999999993e-06 2024-09-16T20:50:00.116 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.116 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.116 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:00.116 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.592232+0000", 2024-09-16T20:50:00.116 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2940000000000006e-06 2024-09-16T20:50:00.116 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.116 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.116 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.116 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.116 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.116 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19947 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:50:00.116 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.595552+0000", 2024-09-16T20:50:00.116 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.028060471999999999, 2024-09-16T20:50:00.116 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.031561119999999998, 2024-09-16T20:50:00.117 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:00.117 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:00.117 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:00.117 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:00.117 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:50:00.117 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:00.117 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19947 2024-09-16T20:50:00.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.117 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:00.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.117 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:00.117 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.595552+0000", 2024-09-16T20:50:00.117 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.118 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:00.118 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.595552+0000", 2024-09-16T20:50:00.118 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.118 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.118 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.118 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:00.118 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.595553+0000", 2024-09-16T20:50:00.118 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2399999999999998e-07 2024-09-16T20:50:00.118 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.118 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.118 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:00.118 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.595557+0000", 2024-09-16T20:50:00.118 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6160000000000002e-06 2024-09-16T20:50:00.118 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.118 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.119 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:00.119 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.595567+0000", 2024-09-16T20:50:00.119 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0604e-05 2024-09-16T20:50:00.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.119 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.119 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:00.119 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.595572+0000", 2024-09-16T20:50:00.119 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4680000000000003e-06 2024-09-16T20:50:00.119 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.119 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.119 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.119 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.120 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19963 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:50:00.120 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.599067+0000", 2024-09-16T20:50:00.120 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.024545918, 2024-09-16T20:50:00.120 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.028075637000000001, 2024-09-16T20:50:00.120 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:00.120 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:00.120 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:00.120 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:00.120 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:50:00.120 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:00.120 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19963 2024-09-16T20:50:00.120 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.120 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:00.120 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.121 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:00.121 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.599067+0000", 2024-09-16T20:50:00.121 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.121 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.121 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.121 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:00.121 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.599067+0000", 2024-09-16T20:50:00.121 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.121 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.121 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.121 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:00.121 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.599067+0000", 2024-09-16T20:50:00.121 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5000000000000002e-07 2024-09-16T20:50:00.121 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.121 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.122 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:00.122 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.599071+0000", 2024-09-16T20:50:00.122 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.631e-06 2024-09-16T20:50:00.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.122 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.122 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:00.122 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.599081+0000", 2024-09-16T20:50:00.122 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9289999999999993e-06 2024-09-16T20:50:00.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.122 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.122 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:00.122 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.599086+0000", 2024-09-16T20:50:00.122 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3489999999999999e-06 2024-09-16T20:50:00.123 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.123 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.123 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.123 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.123 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.123 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:19979 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:50:00.123 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.602661+0000", 2024-09-16T20:50:00.123 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.020952156999999999, 2024-09-16T20:50:00.123 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.024504814, 2024-09-16T20:50:00.123 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:00.123 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:00.123 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:00.123 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:00.123 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:50:00.124 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:00.124 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 19979 2024-09-16T20:50:00.124 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.124 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:00.124 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.124 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:00.124 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602661+0000", 2024-09-16T20:50:00.124 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.124 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.124 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.124 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:00.124 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602661+0000", 2024-09-16T20:50:00.125 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.125 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.125 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:00.125 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602661+0000", 2024-09-16T20:50:00.125 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.37e-07 2024-09-16T20:50:00.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.125 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.125 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:00.125 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602665+0000", 2024-09-16T20:50:00.125 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7639999999999999e-06 2024-09-16T20:50:00.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.125 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.125 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:00.125 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602673+0000", 2024-09-16T20:50:00.126 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5329999999999997e-06 2024-09-16T20:50:00.126 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.126 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.126 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:00.126 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602679+0000", 2024-09-16T20:50:00.126 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1279999999999996e-06 2024-09-16T20:50:00.126 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.126 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.126 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.126 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.126 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.126 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24391 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:50:00.126 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.592222+0000", 2024-09-16T20:50:00.126 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.031390942999999998, 2024-09-16T20:50:00.126 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.034966968000000001, 2024-09-16T20:50:00.127 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:00.127 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:00.127 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:00.127 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:00.127 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:50:00.127 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:50:00.127 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24391 2024-09-16T20:50:00.127 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.127 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:00.127 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.127 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:00.127 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.592222+0000", 2024-09-16T20:50:00.127 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.127 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.127 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.128 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:00.128 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.592222+0000", 2024-09-16T20:50:00.128 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.128 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.128 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:00.128 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.592222+0000", 2024-09-16T20:50:00.128 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7400000000000003e-07 2024-09-16T20:50:00.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.128 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.128 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:00.128 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.592226+0000", 2024-09-16T20:50:00.128 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1930000000000002e-06 2024-09-16T20:50:00.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.129 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.129 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:00.129 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.592233+0000", 2024-09-16T20:50:00.129 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3370000000000002e-06 2024-09-16T20:50:00.129 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.129 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.129 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:00.129 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.592240+0000", 2024-09-16T20:50:00.129 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2490000000000004e-06 2024-09-16T20:50:00.129 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.129 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.129 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.129 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.129 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.129 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24407 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:50:00.130 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.595580+0000", 2024-09-16T20:50:00.130 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.028033153000000002, 2024-09-16T20:50:00.130 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.031632436, 2024-09-16T20:50:00.130 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:00.130 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:00.130 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:00.130 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:00.130 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:50:00.130 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:50:00.130 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24407 2024-09-16T20:50:00.130 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.130 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:00.130 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.130 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:00.131 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.595580+0000", 2024-09-16T20:50:00.131 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.131 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.131 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.131 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:00.131 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.595580+0000", 2024-09-16T20:50:00.131 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.131 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.131 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.131 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:00.131 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.595581+0000", 2024-09-16T20:50:00.131 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.291e-06 2024-09-16T20:50:00.131 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.131 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.131 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:00.131 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.595585+0000", 2024-09-16T20:50:00.132 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2130000000000002e-06 2024-09-16T20:50:00.132 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.132 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.132 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:00.132 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.595594+0000", 2024-09-16T20:50:00.132 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1770000000000004e-06 2024-09-16T20:50:00.132 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.132 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.132 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:00.132 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.595601+0000", 2024-09-16T20:50:00.132 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4180000000000002e-06 2024-09-16T20:50:00.132 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.132 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.132 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.132 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.133 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24423 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:50:00.133 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.599085+0000", 2024-09-16T20:50:00.133 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.024527874000000002, 2024-09-16T20:50:00.133 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.028153086000000001, 2024-09-16T20:50:00.133 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:00.133 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:00.133 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:00.133 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:00.133 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:50:00.133 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:50:00.133 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24423 2024-09-16T20:50:00.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.133 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:00.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.134 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:00.134 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.599085+0000", 2024-09-16T20:50:00.134 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.134 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.134 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:00.134 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.599085+0000", 2024-09-16T20:50:00.134 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.134 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.134 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:00.134 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.599086+0000", 2024-09-16T20:50:00.134 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5899999999999995e-07 2024-09-16T20:50:00.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.135 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.135 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:00.135 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.599088+0000", 2024-09-16T20:50:00.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7190000000000001e-06 2024-09-16T20:50:00.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.135 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.135 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:00.135 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.599094+0000", 2024-09-16T20:50:00.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5319999999999997e-06 2024-09-16T20:50:00.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.135 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.135 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:00.135 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.599097+0000", 2024-09-16T20:50:00.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4460000000000001e-06 2024-09-16T20:50:00.136 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.136 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.136 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.136 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.136 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.136 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5889.0:24439 3.19 3.eb34f979 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-16T20:50:00.136 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:59.602680+0000", 2024-09-16T20:50:00.136 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.020932553999999999, 2024-09-16T20:50:00.136 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.024592032, 2024-09-16T20:50:00.136 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:00.136 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:00.136 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:00.136 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:00.136 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5889", 2024-09-16T20:50:00.136 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3609593189", 2024-09-16T20:50:00.136 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 24439 2024-09-16T20:50:00.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.137 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:00.137 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.137 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:00.137 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602680+0000", 2024-09-16T20:50:00.137 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.137 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.137 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:00.137 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602680+0000", 2024-09-16T20:50:00.137 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.137 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.137 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:00.138 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602681+0000", 2024-09-16T20:50:00.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7300000000000005e-07 2024-09-16T20:50:00.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.138 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:00.138 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602684+0000", 2024-09-16T20:50:00.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.605e-06 2024-09-16T20:50:00.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.138 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:00.138 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602689+0000", 2024-09-16T20:50:00.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9609999999999996e-06 2024-09-16T20:50:00.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.139 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:00.139 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:59.602693+0000", 2024-09-16T20:50:00.139 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1649999999999999e-06 2024-09-16T20:50:00.139 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.139 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.139 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.139 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.139 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-16T20:50:00.139 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 145 2024-09-16T20:50:00.139 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:00.140 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #34, image image_client.0-clone34 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent35 2024-09-16T20:50:00.141 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:00.139+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:00.141 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:50:00.241 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:00.239+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:00.267 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:00.268 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:50:00.268 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:50:00.268 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:50:00.268 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:00.277 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:50:00.361 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:00.360+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:00.427 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:00.427 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:50:00.428 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:50:00.428 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:50:00.428 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.428 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12548 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 628736~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:00.428 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757009+0000", 2024-09-16T20:50:00.428 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 9.6682671720000002, 2024-09-16T20:50:00.428 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065743956000000006, 2024-09-16T20:50:00.428 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:00.428 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:00.429 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:00.429 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:00.429 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:00.429 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:00.429 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12548 2024-09-16T20:50:00.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.429 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:00.429 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.429 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:00.429 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757009+0000", 2024-09-16T20:50:00.429 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.429 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.429 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:00.430 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757009+0000", 2024-09-16T20:50:00.430 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.430 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.430 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:00.430 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757010+0000", 2024-09-16T20:50:00.430 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9900000000000001e-07 2024-09-16T20:50:00.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.430 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.430 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:00.430 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757012+0000", 2024-09-16T20:50:00.430 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2689999999999998e-06 2024-09-16T20:50:00.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.430 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.431 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:00.431 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757022+0000", 2024-09-16T20:50:00.431 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9699999999999994e-06 2024-09-16T20:50:00.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.431 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.431 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:00.431 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757028+0000", 2024-09-16T20:50:00.431 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3799999999999999e-06 2024-09-16T20:50:00.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.431 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.431 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:00.431 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822552+0000", 2024-09-16T20:50:00.431 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065524281000000004 2024-09-16T20:50:00.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.432 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.432 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:00.432 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822613+0000", 2024-09-16T20:50:00.433 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0430999999999998e-05 2024-09-16T20:50:00.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.433 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.433 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:00.433 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822753+0000", 2024-09-16T20:50:00.433 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000140226 2024-09-16T20:50:00.433 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.433 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.433 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.434 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.434 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12549 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 633856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:00.434 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:50:00.434 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 9.6682384250000002, 2024-09-16T20:50:00.434 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065978518, 2024-09-16T20:50:00.434 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:00.434 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:00.434 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:00.434 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:00.434 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:00.434 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:00.434 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12549 2024-09-16T20:50:00.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.435 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:00.435 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.435 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:00.435 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:50:00.435 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.435 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.435 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:00.435 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:50:00.435 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.435 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.436 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:00.436 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:50:00.436 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9299999999999999e-07 2024-09-16T20:50:00.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.436 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.436 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:00.436 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757040+0000", 2024-09-16T20:50:00.436 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.677e-06 2024-09-16T20:50:00.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.436 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.436 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:00.436 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757045+0000", 2024-09-16T20:50:00.436 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8149999999999996e-06 2024-09-16T20:50:00.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.437 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:00.437 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757048+0000", 2024-09-16T20:50:00.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1e-06 2024-09-16T20:50:00.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.437 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:00.437 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822796+0000", 2024-09-16T20:50:00.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065748081 2024-09-16T20:50:00.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:00.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822867+0000", 2024-09-16T20:50:00.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0679000000000005e-05 2024-09-16T20:50:00.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:00.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823016+0000", 2024-09-16T20:50:00.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014987299999999999 2024-09-16T20:50:00.438 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.438 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.438 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.439 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12550 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 638976~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:00.439 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757060+0000", 2024-09-16T20:50:00.439 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 9.6682160100000001, 2024-09-16T20:50:00.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066153564999999998, 2024-09-16T20:50:00.439 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:00.439 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:00.439 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:00.439 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:00.439 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:00.439 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:00.439 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12550 2024-09-16T20:50:00.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.439 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:00.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.440 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:00.440 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757060+0000", 2024-09-16T20:50:00.440 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.440 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:00.440 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757060+0000", 2024-09-16T20:50:00.440 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.440 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:00.440 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757061+0000", 2024-09-16T20:50:00.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.15e-07 2024-09-16T20:50:00.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:00.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757062+0000", 2024-09-16T20:50:00.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.663e-06 2024-09-16T20:50:00.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:00.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757066+0000", 2024-09-16T20:50:00.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.817e-06 2024-09-16T20:50:00.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:00.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757070+0000", 2024-09-16T20:50:00.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1509999999999997e-06 2024-09-16T20:50:00.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:00.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823044+0000", 2024-09-16T20:50:00.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065973992999999995 2024-09-16T20:50:00.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:00.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823082+0000", 2024-09-16T20:50:00.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8050999999999999e-05 2024-09-16T20:50:00.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.443 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:00.443 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823214+0000", 2024-09-16T20:50:00.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013157500000000001 2024-09-16T20:50:00.443 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.443 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.443 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.443 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12551 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 644096~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:00.443 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:50:00.443 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 9.668202312, 2024-09-16T20:50:00.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066391378000000001, 2024-09-16T20:50:00.443 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:00.444 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:00.444 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:00.444 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:00.444 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:00.444 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:00.444 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12551 2024-09-16T20:50:00.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.444 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:00.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:00.444 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:50:00.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:00.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:50:00.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:00.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:50:00.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2999999999999999e-07 2024-09-16T20:50:00.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:00.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757075+0000", 2024-09-16T20:50:00.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.097e-06 2024-09-16T20:50:00.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.446 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:00.446 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757082+0000", 2024-09-16T20:50:00.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9500000000000004e-06 2024-09-16T20:50:00.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.446 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:00.446 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757086+0000", 2024-09-16T20:50:00.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2229999999999999e-06 2024-09-16T20:50:00.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.446 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:00.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823262+0000", 2024-09-16T20:50:00.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066176078999999999 2024-09-16T20:50:00.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:00.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823336+0000", 2024-09-16T20:50:00.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3979000000000004e-05 2024-09-16T20:50:00.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:00.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823465+0000", 2024-09-16T20:50:00.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012982 2024-09-16T20:50:00.447 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.448 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.448 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.448 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12552 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 649216~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:00.448 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757089+0000", 2024-09-16T20:50:00.448 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 9.6681871709999996, 2024-09-16T20:50:00.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066655617, 2024-09-16T20:50:00.448 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:00.448 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:00.448 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:00.448 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:00.448 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:00.449 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:00.449 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12552 2024-09-16T20:50:00.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.449 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:00.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.449 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:00.449 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757089+0000", 2024-09-16T20:50:00.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.449 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:00.449 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757089+0000", 2024-09-16T20:50:00.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.450 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:00.450 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759492+0000", 2024-09-16T20:50:00.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0024026109999999998 2024-09-16T20:50:00.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.450 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:00.450 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759497+0000", 2024-09-16T20:50:00.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1120000000000004e-06 2024-09-16T20:50:00.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.450 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:00.450 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759508+0000", 2024-09-16T20:50:00.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0981e-05 2024-09-16T20:50:00.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.451 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:00.451 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759516+0000", 2024-09-16T20:50:00.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7270000000000007e-06 2024-09-16T20:50:00.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.451 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:00.451 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823504+0000", 2024-09-16T20:50:00.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.063987984999999997 2024-09-16T20:50:00.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:00.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823579+0000", 2024-09-16T20:50:00.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5703000000000003e-05 2024-09-16T20:50:00.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:00.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823745+0000", 2024-09-16T20:50:00.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016549800000000001 2024-09-16T20:50:00.452 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.452 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.452 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.453 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12560 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 690176~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:00.453 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.759658+0000", 2024-09-16T20:50:00.453 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 9.6656179689999995, 2024-09-16T20:50:00.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065858517000000005, 2024-09-16T20:50:00.453 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:00.453 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:00.453 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:00.453 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:00.453 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:00.453 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:00.453 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12560 2024-09-16T20:50:00.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.453 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:00.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:00.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759658+0000", 2024-09-16T20:50:00.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:00.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759658+0000", 2024-09-16T20:50:00.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:00.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759659+0000", 2024-09-16T20:50:00.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.17e-07 2024-09-16T20:50:00.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:00.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759660+0000", 2024-09-16T20:50:00.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0189999999999999e-06 2024-09-16T20:50:00.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:00.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759663+0000", 2024-09-16T20:50:00.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0010000000000002e-06 2024-09-16T20:50:00.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:00.456 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759665+0000", 2024-09-16T20:50:00.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5440000000000001e-06 2024-09-16T20:50:00.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.456 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:00.456 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825355+0000", 2024-09-16T20:50:00.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065690224000000005 2024-09-16T20:50:00.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.456 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:00.456 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825394+0000", 2024-09-16T20:50:00.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8810999999999998e-05 2024-09-16T20:50:00.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.457 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:00.457 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825517+0000", 2024-09-16T20:50:00.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012270099999999999 2024-09-16T20:50:00.457 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.457 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.457 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.457 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12561 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 695296~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:00.457 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.759668+0000", 2024-09-16T20:50:00.457 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 9.6656081270000005, 2024-09-16T20:50:00.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066116999999999995, 2024-09-16T20:50:00.457 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:00.457 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:00.458 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:00.458 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:00.458 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:00.458 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:00.458 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12561 2024-09-16T20:50:00.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.458 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:00.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:00.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759668+0000", 2024-09-16T20:50:00.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:00.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759668+0000", 2024-09-16T20:50:00.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:00.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765379+0000", 2024-09-16T20:50:00.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057104410000000001 2024-09-16T20:50:00.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:00.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765383+0000", 2024-09-16T20:50:00.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2980000000000002e-06 2024-09-16T20:50:00.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:00.460 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765395+0000", 2024-09-16T20:50:00.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1671000000000001e-05 2024-09-16T20:50:00.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:00.460 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765403+0000", 2024-09-16T20:50:00.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1210000000000007e-06 2024-09-16T20:50:00.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:00.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825552+0000", 2024-09-16T20:50:00.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.060149498000000003 2024-09-16T20:50:00.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:00.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825625+0000", 2024-09-16T20:50:00.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2540999999999995e-05 2024-09-16T20:50:00.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:00.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825785+0000", 2024-09-16T20:50:00.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016043000000000001 2024-09-16T20:50:00.461 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.462 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.462 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.462 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12585 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 818176~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:00.462 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.765848+0000", 2024-09-16T20:50:00.462 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 9.6594281560000006, 2024-09-16T20:50:00.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065843218999999995, 2024-09-16T20:50:00.462 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:00.462 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:00.462 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:00.462 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:00.462 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:00.463 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:00.463 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12585 2024-09-16T20:50:00.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.463 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:00.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.463 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:00.463 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765848+0000", 2024-09-16T20:50:00.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.463 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:00.463 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765848+0000", 2024-09-16T20:50:00.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.464 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:00.464 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768683+0000", 2024-09-16T20:50:00.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0028344519999999999 2024-09-16T20:50:00.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.464 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:00.464 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768686+0000", 2024-09-16T20:50:00.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8789999999999996e-06 2024-09-16T20:50:00.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.464 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:00.464 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768696+0000", 2024-09-16T20:50:00.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6709999999999994e-06 2024-09-16T20:50:00.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.465 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:00.465 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768702+0000", 2024-09-16T20:50:00.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2550000000000003e-06 2024-09-16T20:50:00.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.465 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:00.465 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.831499+0000", 2024-09-16T20:50:00.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.062797081000000005 2024-09-16T20:50:00.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:00.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.831538+0000", 2024-09-16T20:50:00.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8523999999999999e-05 2024-09-16T20:50:00.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:00.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.831691+0000", 2024-09-16T20:50:00.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015335699999999999 2024-09-16T20:50:00.466 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.466 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.466 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.467 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12597 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 879616~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:00.467 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.768883+0000", 2024-09-16T20:50:00.467 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 9.6563934279999994, 2024-09-16T20:50:00.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065816527, 2024-09-16T20:50:00.467 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:00.467 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:00.467 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:00.467 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:00.467 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:00.467 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:00.467 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12597 2024-09-16T20:50:00.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.468 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:00.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:00.468 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768883+0000", 2024-09-16T20:50:00.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:00.468 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768883+0000", 2024-09-16T20:50:00.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:00.469 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771508+0000", 2024-09-16T20:50:00.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0026247589999999999 2024-09-16T20:50:00.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.469 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:00.469 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771513+0000", 2024-09-16T20:50:00.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9949999999999996e-06 2024-09-16T20:50:00.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.469 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:00.469 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771525+0000", 2024-09-16T20:50:00.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2033e-05 2024-09-16T20:50:00.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:00.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771532+0000", 2024-09-16T20:50:00.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4220000000000003e-06 2024-09-16T20:50:00.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:00.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.834532+0000", 2024-09-16T20:50:00.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.063000159 2024-09-16T20:50:00.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:00.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.834590+0000", 2024-09-16T20:50:00.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8218999999999999e-05 2024-09-16T20:50:00.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:00.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.834699+0000", 2024-09-16T20:50:00.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010894 2024-09-16T20:50:00.471 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.471 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.471 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.471 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12619 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 992256~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:00.471 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.774656+0000", 2024-09-16T20:50:00.471 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 9.6506206540000008, 2024-09-16T20:50:00.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065715859000000001, 2024-09-16T20:50:00.472 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:00.472 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:00.472 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:00.472 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:00.472 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:00.472 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:00.472 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12619 2024-09-16T20:50:00.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.472 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:00.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:00.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774656+0000", 2024-09-16T20:50:00.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:00.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774656+0000", 2024-09-16T20:50:00.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:00.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774875+0000", 2024-09-16T20:50:00.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000218898 2024-09-16T20:50:00.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:00.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774879+0000", 2024-09-16T20:50:00.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0389999999999998e-06 2024-09-16T20:50:00.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:00.474 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774890+0000", 2024-09-16T20:50:00.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1250000000000001e-05 2024-09-16T20:50:00.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:00.474 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774897+0000", 2024-09-16T20:50:00.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7619999999999998e-06 2024-09-16T20:50:00.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:00.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840139+0000", 2024-09-16T20:50:00.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065242438 2024-09-16T20:50:00.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:00.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840186+0000", 2024-09-16T20:50:00.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6894999999999997e-05 2024-09-16T20:50:00.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:00.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840372+0000", 2024-09-16T20:50:00.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000185577 2024-09-16T20:50:00.475 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.476 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.476 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.476 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12620 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 997376~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:00.476 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.774908+0000", 2024-09-16T20:50:00.476 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 9.6503684330000006, 2024-09-16T20:50:00.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065782754999999998, 2024-09-16T20:50:00.476 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:00.476 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:00.476 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:00.476 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:00.476 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:00.477 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:00.477 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12620 2024-09-16T20:50:00.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.477 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:00.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:00.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774908+0000", 2024-09-16T20:50:00.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:00.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774908+0000", 2024-09-16T20:50:00.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:00.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775102+0000", 2024-09-16T20:50:00.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000194098 2024-09-16T20:50:00.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:00.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775105+0000", 2024-09-16T20:50:00.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8739999999999999e-06 2024-09-16T20:50:00.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:00.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775111+0000", 2024-09-16T20:50:00.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9519999999999999e-06 2024-09-16T20:50:00.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:00.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775115+0000", 2024-09-16T20:50:00.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.002e-06 2024-09-16T20:50:00.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:00.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840409+0000", 2024-09-16T20:50:00.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065294195999999999 2024-09-16T20:50:00.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:00.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840507+0000", 2024-09-16T20:50:00.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7662000000000001e-05 2024-09-16T20:50:00.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:00.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840691+0000", 2024-09-16T20:50:00.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000183971 2024-09-16T20:50:00.480 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.480 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.480 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.480 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12621 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1002496~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:00.481 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775123+0000", 2024-09-16T20:50:00.481 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 9.6501537370000001, 2024-09-16T20:50:00.481 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065829586999999995, 2024-09-16T20:50:00.481 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:00.481 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:00.481 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:00.481 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:00.481 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:00.481 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:00.481 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12621 2024-09-16T20:50:00.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.481 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:00.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.482 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:00.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775123+0000", 2024-09-16T20:50:00.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.482 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:00.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775123+0000", 2024-09-16T20:50:00.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.482 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:00.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775348+0000", 2024-09-16T20:50:00.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000225295 2024-09-16T20:50:00.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:00.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775352+0000", 2024-09-16T20:50:00.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0810000000000004e-06 2024-09-16T20:50:00.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:00.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775361+0000", 2024-09-16T20:50:00.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7399999999999993e-06 2024-09-16T20:50:00.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:00.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775367+0000", 2024-09-16T20:50:00.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5200000000000003e-06 2024-09-16T20:50:00.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:00.484 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840730+0000", 2024-09-16T20:50:00.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065362803999999997 2024-09-16T20:50:00.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:00.484 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840801+0000", 2024-09-16T20:50:00.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1006000000000004e-05 2024-09-16T20:50:00.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:00.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840952+0000", 2024-09-16T20:50:00.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015114100000000001 2024-09-16T20:50:00.485 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.485 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.485 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.485 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12622 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1007616~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:00.485 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775381+0000", 2024-09-16T20:50:00.485 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 9.6498949029999999, 2024-09-16T20:50:00.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065822581000000005, 2024-09-16T20:50:00.485 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:00.485 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:00.486 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:00.486 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:00.486 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:00.486 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:00.486 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12622 2024-09-16T20:50:00.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.486 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:00.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.486 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:00.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775381+0000", 2024-09-16T20:50:00.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.486 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:00.487 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775381+0000", 2024-09-16T20:50:00.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:00.487 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775563+0000", 2024-09-16T20:50:00.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018153700000000001 2024-09-16T20:50:00.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:00.487 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775566+0000", 2024-09-16T20:50:00.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9620000000000002e-06 2024-09-16T20:50:00.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.488 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:00.488 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775572+0000", 2024-09-16T20:50:00.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8529999999999997e-06 2024-09-16T20:50:00.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.488 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:00.488 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775576+0000", 2024-09-16T20:50:00.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8920000000000002e-06 2024-09-16T20:50:00.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.488 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:00.488 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840990+0000", 2024-09-16T20:50:00.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065414599000000004 2024-09-16T20:50:00.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:00.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841068+0000", 2024-09-16T20:50:00.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8157000000000004e-05 2024-09-16T20:50:00.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:00.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841204+0000", 2024-09-16T20:50:00.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013558099999999999 2024-09-16T20:50:00.489 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.489 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.489 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.490 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12623 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1012736~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:00.490 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775583+0000", 2024-09-16T20:50:00.490 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 9.6496934650000004, 2024-09-16T20:50:00.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065929087999999997, 2024-09-16T20:50:00.490 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:00.490 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:00.490 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:00.490 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:00.490 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:00.490 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:00.490 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12623 2024-09-16T20:50:00.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.491 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:00.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:00.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775583+0000", 2024-09-16T20:50:00.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:00.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775583+0000", 2024-09-16T20:50:00.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:00.492 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775804+0000", 2024-09-16T20:50:00.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00022116900000000001 2024-09-16T20:50:00.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.492 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:00.492 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775807+0000", 2024-09-16T20:50:00.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9529999999999999e-06 2024-09-16T20:50:00.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.492 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:00.492 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775814+0000", 2024-09-16T20:50:00.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7490000000000001e-06 2024-09-16T20:50:00.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.493 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:00.493 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775818+0000", 2024-09-16T20:50:00.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1999999999999996e-06 2024-09-16T20:50:00.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.493 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:00.493 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841267+0000", 2024-09-16T20:50:00.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065449127999999995 2024-09-16T20:50:00.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.493 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:00.494 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841344+0000", 2024-09-16T20:50:00.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6546999999999997e-05 2024-09-16T20:50:00.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.494 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:00.494 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841512+0000", 2024-09-16T20:50:00.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016834200000000001 2024-09-16T20:50:00.494 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.494 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.494 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.494 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12624 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1017856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:00.495 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:50:00.495 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 9.6494503250000001, 2024-09-16T20:50:00.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065945866000000006, 2024-09-16T20:50:00.495 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:00.495 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:00.495 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:00.495 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:00.495 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:00.495 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:00.495 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12624 2024-09-16T20:50:00.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.495 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:00.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:00.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:50:00.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:00.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:50:00.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:00.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:50:00.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-16T20:50:00.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:00.497 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775828+0000", 2024-09-16T20:50:00.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3999999999999999e-06 2024-09-16T20:50:00.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:00.497 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775831+0000", 2024-09-16T20:50:00.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.478e-06 2024-09-16T20:50:00.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:00.497 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775834+0000", 2024-09-16T20:50:00.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9440000000000001e-06 2024-09-16T20:50:00.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.498 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:00.498 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841547+0000", 2024-09-16T20:50:00.498 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065712801000000001 2024-09-16T20:50:00.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.498 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:00.498 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841618+0000", 2024-09-16T20:50:00.498 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1151999999999999e-05 2024-09-16T20:50:00.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.498 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:00.498 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841772+0000", 2024-09-16T20:50:00.499 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000153721 2024-09-16T20:50:00.499 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.499 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.499 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.499 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.499 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12625 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1022976~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:00.499 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775840+0000", 2024-09-16T20:50:00.499 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 9.6494367160000003, 2024-09-16T20:50:00.499 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066165451, 2024-09-16T20:50:00.499 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:00.499 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:00.499 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:00.500 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:00.500 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:00.500 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:00.500 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12625 2024-09-16T20:50:00.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.500 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:00.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.500 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:00.500 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775840+0000", 2024-09-16T20:50:00.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.500 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:00.500 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775840+0000", 2024-09-16T20:50:00.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:00.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776014+0000", 2024-09-16T20:50:00.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017441099999999999 2024-09-16T20:50:00.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:00.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776018+0000", 2024-09-16T20:50:00.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6509999999999999e-06 2024-09-16T20:50:00.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.502 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:00.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776026+0000", 2024-09-16T20:50:00.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.028e-06 2024-09-16T20:50:00.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.502 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:00.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776031+0000", 2024-09-16T20:50:00.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8400000000000002e-06 2024-09-16T20:50:00.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.502 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:00.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841801+0000", 2024-09-16T20:50:00.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065770472999999996 2024-09-16T20:50:00.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:00.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841847+0000", 2024-09-16T20:50:00.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5558e-05 2024-09-16T20:50:00.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:00.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842005+0000", 2024-09-16T20:50:00.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015849000000000001 2024-09-16T20:50:00.503 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.503 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.503 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.504 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12626 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1028096~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:00.504 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776041+0000", 2024-09-16T20:50:00.504 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 9.6492354169999999, 2024-09-16T20:50:00.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066225260999999994, 2024-09-16T20:50:00.504 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:00.504 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:00.504 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:00.504 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:00.504 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:00.504 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:00.504 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12626 2024-09-16T20:50:00.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.505 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:00.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:00.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776041+0000", 2024-09-16T20:50:00.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:00.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776041+0000", 2024-09-16T20:50:00.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:00.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776263+0000", 2024-09-16T20:50:00.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00022193899999999999 2024-09-16T20:50:00.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:00.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776268+0000", 2024-09-16T20:50:00.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8049999999999997e-06 2024-09-16T20:50:00.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:00.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776280+0000", 2024-09-16T20:50:00.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2635e-05 2024-09-16T20:50:00.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.507 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:00.507 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776288+0000", 2024-09-16T20:50:00.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8199999999999997e-06 2024-09-16T20:50:00.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.507 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.507 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:00.507 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842041+0000", 2024-09-16T20:50:00.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065752506000000002 2024-09-16T20:50:00.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.507 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.507 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:00.507 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842123+0000", 2024-09-16T20:50:00.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2393999999999998e-05 2024-09-16T20:50:00.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.508 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:00.508 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842266+0000", 2024-09-16T20:50:00.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014316200000000001 2024-09-16T20:50:00.508 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.508 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.508 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.508 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12627 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1033216~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:00.508 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776301+0000", 2024-09-16T20:50:00.508 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 9.6489754570000006, 2024-09-16T20:50:00.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066187814999999997, 2024-09-16T20:50:00.508 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:00.509 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:00.509 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:00.509 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:00.509 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:00.509 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:00.509 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12627 2024-09-16T20:50:00.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.509 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:00.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:00.509 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776301+0000", 2024-09-16T20:50:00.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:00.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776301+0000", 2024-09-16T20:50:00.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:00.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776473+0000", 2024-09-16T20:50:00.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017222400000000001 2024-09-16T20:50:00.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:00.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776477+0000", 2024-09-16T20:50:00.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7019999999999999e-06 2024-09-16T20:50:00.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:00.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776486+0000", 2024-09-16T20:50:00.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2119999999999992e-06 2024-09-16T20:50:00.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:00.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776492+0000", 2024-09-16T20:50:00.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8089999999999998e-06 2024-09-16T20:50:00.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:00.512 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842301+0000", 2024-09-16T20:50:00.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065809042999999998 2024-09-16T20:50:00.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.512 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.512 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:00.512 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842362+0000", 2024-09-16T20:50:00.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1260000000000006e-05 2024-09-16T20:50:00.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.512 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.512 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:00.512 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842489+0000", 2024-09-16T20:50:00.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012656500000000001 2024-09-16T20:50:00.512 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.512 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.513 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.513 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12628 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1038336~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:00.513 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776502+0000", 2024-09-16T20:50:00.513 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 9.6487748109999991, 2024-09-16T20:50:00.513 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066283426000000006, 2024-09-16T20:50:00.513 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:00.513 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:00.513 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:00.513 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:00.513 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:00.513 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:00.513 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12628 2024-09-16T20:50:00.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.514 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:00.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:00.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776502+0000", 2024-09-16T20:50:00.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:00.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776502+0000", 2024-09-16T20:50:00.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.515 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:00.515 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776716+0000", 2024-09-16T20:50:00.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000214596 2024-09-16T20:50:00.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.515 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.515 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:00.515 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776723+0000", 2024-09-16T20:50:00.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4000000000000003e-06 2024-09-16T20:50:00.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.515 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.515 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:00.515 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776731+0000", 2024-09-16T20:50:00.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6129999999999998e-06 2024-09-16T20:50:00.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.516 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.516 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:00.516 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776737+0000", 2024-09-16T20:50:00.516 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1970000000000001e-06 2024-09-16T20:50:00.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.516 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.516 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:00.516 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842532+0000", 2024-09-16T20:50:00.516 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065795127999999994 2024-09-16T20:50:00.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.516 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.516 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:00.517 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842599+0000", 2024-09-16T20:50:00.517 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6711000000000006e-05 2024-09-16T20:50:00.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.517 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:00.517 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842785+0000", 2024-09-16T20:50:00.517 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018578099999999999 2024-09-16T20:50:00.517 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.517 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.517 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.517 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12629 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1043456~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:00.518 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776748+0000", 2024-09-16T20:50:00.518 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 9.6485284339999993, 2024-09-16T20:50:00.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066303788000000002, 2024-09-16T20:50:00.518 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:00.518 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:00.518 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:00.518 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:00.518 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:00.518 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:00.518 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12629 2024-09-16T20:50:00.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.518 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:00.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.519 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:00.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776748+0000", 2024-09-16T20:50:00.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.519 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:00.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776748+0000", 2024-09-16T20:50:00.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:00.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.519 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:00.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776934+0000", 2024-09-16T20:50:00.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018598399999999999 2024-09-16T20:50:00.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.520 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:00.520 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776939+0000", 2024-09-16T20:50:00.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1000000000000003e-06 2024-09-16T20:50:00.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.520 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:00.520 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776954+0000", 2024-09-16T20:50:00.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5221e-05 2024-09-16T20:50:00.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.520 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:00.520 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776962+0000", 2024-09-16T20:50:00.521 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.554e-06 2024-09-16T20:50:00.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.521 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.521 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:00.521 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842813+0000", 2024-09-16T20:50:00.521 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065851006000000004 2024-09-16T20:50:00.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.521 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.521 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:00.521 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842886+0000", 2024-09-16T20:50:00.521 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3663000000000003e-05 2024-09-16T20:50:00.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:00.521 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:00.522 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:00.522 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.843052+0000", 2024-09-16T20:50:00.522 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016526 2024-09-16T20:50:00.522 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.522 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.522 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.522 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:00.522 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:00.522 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:00.523 INFO:teuthology.orchestra.run.smithi139.stdout:4795 punch from 0xac84200 to 0xac85400, (0x1200 bytes) 2024-09-16T20:50:00.524 INFO:teuthology.orchestra.run.smithi139.stdout:4796 trunc from 0xdf9ae00 to 0x4c67200 2024-09-16T20:50:00.524 INFO:teuthology.orchestra.run.smithi139.stdout:4798 punch from 0x14c2800 to 0x14cee00, (0xc600 bytes) 2024-09-16T20:50:00.524 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:50:00.525 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:00.522+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:00.525 INFO:teuthology.orchestra.run.smithi139.stdout:4800 write 0x6165e00 thru 0x616dfde (0x81df bytes) 2024-09-16T20:50:00.529 INFO:teuthology.orchestra.run.smithi139.stdout:4801 punch from 0x53c9600 to 0x53ca800, (0x1200 bytes) 2024-09-16T20:50:00.546 INFO:teuthology.orchestra.run.smithi139.stdout:4802 write 0xe2c9400 thru 0xe2cda92 (0x4693 bytes) 2024-09-16T20:50:00.547 INFO:teuthology.orchestra.run.smithi139.stdout:4803 write 0xd856400 thru 0xd8616a4 (0xb2a5 bytes) 2024-09-16T20:50:00.547 INFO:teuthology.orchestra.run.smithi139.stdout:4804 punch from 0x8bbf600 to 0x8bcec00, (0xf600 bytes) 2024-09-16T20:50:00.552 INFO:teuthology.orchestra.run.smithi139.stdout:4805 read 0x6126600 thru 0x61326c9 (0xc0ca bytes) 2024-09-16T20:50:00.552 INFO:teuthology.orchestra.run.smithi139.stdout:4806 read 0x9a1b800 thru 0x9a28a4d (0xd24e bytes) 2024-09-16T20:50:00.552 INFO:teuthology.orchestra.run.smithi139.stdout:4808 write 0x7b63c00 thru 0x7b67683 (0x3a84 bytes) 2024-09-16T20:50:00.552 INFO:teuthology.orchestra.run.smithi139.stdout:4810 write 0xb9f5800 thru 0xb9fae07 (0x5608 bytes) 2024-09-16T20:50:00.552 INFO:teuthology.orchestra.run.smithi139.stdout:4812 write 0x2980600 thru 0x298cf4e (0xc94f bytes) 2024-09-16T20:50:00.552 INFO:teuthology.orchestra.run.smithi139.stdout:4813 write 0x1ae6400 thru 0x1aefb65 (0x9766 bytes) 2024-09-16T20:50:00.552 INFO:teuthology.orchestra.run.smithi139.stdout:4815 punch from 0xbe02c00 to 0xbe0b000, (0x8400 bytes) 2024-09-16T20:50:00.552 INFO:teuthology.orchestra.run.smithi139.stdout:4816 read 0xdaad000 thru 0xdabaf7b (0xdf7c bytes) 2024-09-16T20:50:00.552 INFO:teuthology.orchestra.run.smithi139.stdout:4817 write 0xcbe6e00 thru 0xcbed8a2 (0x6aa3 bytes) 2024-09-16T20:50:00.552 INFO:teuthology.orchestra.run.smithi139.stdout:4818 read 0x791dc00 thru 0x7927878 (0x9c79 bytes) 2024-09-16T20:50:00.553 INFO:teuthology.orchestra.run.smithi139.stdout:4819 punch from 0x84c7c00 to 0x84cb200, (0x3600 bytes) 2024-09-16T20:50:00.553 INFO:teuthology.orchestra.run.smithi139.stdout:4820 punch from 0xb793800 to 0xb793a00, (0x200 bytes) 2024-09-16T20:50:00.553 INFO:teuthology.orchestra.run.smithi139.stdout:4821 write 0x8fb0c00 thru 0x8fbab49 (0x9f4a bytes) 2024-09-16T20:50:00.574 INFO:teuthology.orchestra.run.smithi139.stdout:4822 write 0xeac3c00 thru 0xeacadd6 (0x71d7 bytes) 2024-09-16T20:50:00.579 INFO:teuthology.orchestra.run.smithi139.stdout:4824 write 0x95ee200 thru 0x95f0f06 (0x2d07 bytes) 2024-09-16T20:50:00.584 INFO:teuthology.orchestra.run.smithi139.stdout:4826 trunc from 0xeacae00 to 0x6f77400 2024-09-16T20:50:00.607 INFO:teuthology.orchestra.run.smithi139.stdout:4828 clone 37 order 24 su 1048576 sc 6 2024-09-16T20:50:00.624 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:00.622+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:00.655 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 3, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-16T20:50:00.655 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T20:50:00.655 INFO:tasks.thrashosds.thrasher:inject_pause on osd.2 2024-09-16T20:50:00.655 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T20:50:00.655 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T20:50:00.656 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok config set filestore_inject_stall 3 2024-09-16T20:50:00.657 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T20:50:00.681 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:00.681 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:50:00.681 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:50:00.681 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:00.690 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:50:00.724 INFO:teuthology.orchestra.run.smithi139.stdout:flattening image image_client.0-clone36 2024-09-16T20:50:00.724 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:00.723+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:00.799 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:00.799 INFO:teuthology.orchestra.run.smithi029.stdout: "success": "" 2024-09-16T20:50:00.800 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:00.825 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:00.823+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:00.909 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:00.909 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:50:00.910 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:50:00.910 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:50:00.910 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:00.920 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:50:00.961 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:00.960+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:01.026 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:01.025+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:01.126 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T20:50:01.128 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:01.126+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:01.132 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:01.132 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:50:01.132 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:50:01.132 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:50:01.132 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.132 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5286.0:10643 3.17 3.f6273fb7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-16T20:50:01.132 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:50:01.132 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 52.338913726000001, 2024-09-16T20:50:01.132 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.76672347200000002, 2024-09-16T20:50:01.133 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:01.133 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:01.133 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:01.133 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:01.133 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5286", 2024-09-16T20:50:01.133 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/4135433989", 2024-09-16T20:50:01.134 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 10643 2024-09-16T20:50:01.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.134 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:01.134 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.134 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:01.134 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:50:01.134 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:01.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.134 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.134 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:01.134 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:50:01.134 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:01.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.134 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.135 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:01.135 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790489+0000", 2024-09-16T20:50:01.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5910000000000001e-06 2024-09-16T20:50:01.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.135 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.135 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:01.135 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790493+0000", 2024-09-16T20:50:01.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2669999999999998e-06 2024-09-16T20:50:01.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.135 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.136 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:01.136 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790500+0000", 2024-09-16T20:50:01.136 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3780000000000003e-06 2024-09-16T20:50:01.136 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.136 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.136 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:01.136 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790505+0000", 2024-09-16T20:50:01.136 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1560000000000003e-06 2024-09-16T20:50:01.136 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.136 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.136 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:01.136 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790559+0000", 2024-09-16T20:50:01.136 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3494e-05 2024-09-16T20:50:01.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.137 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.137 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:01.137 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790566+0000", 2024-09-16T20:50:01.137 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7649999999999993e-06 2024-09-16T20:50:01.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.137 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.137 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:01.137 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:09.557198+0000", 2024-09-16T20:50:01.137 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.76663136200000004 2024-09-16T20:50:01.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.137 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.137 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:01.138 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:09.557211+0000", 2024-09-16T20:50:01.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3458999999999999e-05 2024-09-16T20:50:01.138 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:01.138 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:01.138 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:01.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.138 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5512.0:450 3.2 3:44e0cffa:::rbd_header.110ef0d7a535:head [watch reconnect cookie 140263298018912 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:50:01.138 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:50:01.138 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.566890813000001, 2024-09-16T20:50:01.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0084581939999999, 2024-09-16T20:50:01.138 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:01.138 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:01.138 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:01.139 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:01.139 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5512", 2024-09-16T20:50:01.139 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:50:01.139 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 450 2024-09-16T20:50:01.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.139 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:01.139 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.139 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:01.139 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:50:01.139 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:01.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.139 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.140 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:01.140 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:50:01.140 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:01.140 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.140 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.140 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:01.140 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562512+0000", 2024-09-16T20:50:01.140 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.009e-06 2024-09-16T20:50:01.140 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.140 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.140 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:01.140 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562517+0000", 2024-09-16T20:50:01.140 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9339999999999997e-06 2024-09-16T20:50:01.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.141 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.141 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:01.141 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562530+0000", 2024-09-16T20:50:01.141 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3495e-05 2024-09-16T20:50:01.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.142 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.142 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:01.142 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562536+0000", 2024-09-16T20:50:01.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5690000000000004e-06 2024-09-16T20:50:01.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.142 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.143 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:01.143 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562591+0000", 2024-09-16T20:50:01.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5761999999999999e-05 2024-09-16T20:50:01.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.143 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:50:01.144 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562599+0000", 2024-09-16T20:50:01.144 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9179999999999994e-06 2024-09-16T20:50:01.144 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.144 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.144 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:01.144 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.565971+0000", 2024-09-16T20:50:01.144 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003371546 2024-09-16T20:50:01.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.145 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.145 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:01.145 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566019+0000", 2024-09-16T20:50:01.145 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7849000000000002e-05 2024-09-16T20:50:01.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.145 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.145 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:01.145 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.569382+0000", 2024-09-16T20:50:01.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.003362833 2024-09-16T20:50:01.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.146 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:01.146 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.569529+0000", 2024-09-16T20:50:01.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000147823 2024-09-16T20:50:01.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.146 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:01.146 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570969+0000", 2024-09-16T20:50:01.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001439456 2024-09-16T20:50:01.146 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:01.147 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:01.147 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:01.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.147 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.147 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:14996 3.2 3:44e0cffa:::rbd_header.110ef0d7a535:head [watch reconnect cookie 140213436269840 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:50:01.147 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:50:01.147 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.566842400999999, 2024-09-16T20:50:01.147 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.008813943, 2024-09-16T20:50:01.147 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:01.147 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:01.147 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:01.147 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:01.147 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:50:01.148 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:01.148 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 14996 2024-09-16T20:50:01.148 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.148 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:01.148 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.148 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:01.148 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:50:01.148 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:01.148 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.148 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.148 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:01.148 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:50:01.149 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:01.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.149 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.149 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:01.149 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562560+0000", 2024-09-16T20:50:01.149 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.06e-07 2024-09-16T20:50:01.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.149 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.149 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:01.149 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562563+0000", 2024-09-16T20:50:01.149 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.495e-06 2024-09-16T20:50:01.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.149 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.150 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:01.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562571+0000", 2024-09-16T20:50:01.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1950000000000003e-06 2024-09-16T20:50:01.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.150 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.150 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:01.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562579+0000", 2024-09-16T20:50:01.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2649999999999999e-06 2024-09-16T20:50:01.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.150 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:01.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562673+0000", 2024-09-16T20:50:01.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4118999999999998e-05 2024-09-16T20:50:01.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:50:01.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562677+0000", 2024-09-16T20:50:01.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5179999999999998e-06 2024-09-16T20:50:01.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:01.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566461+0000", 2024-09-16T20:50:01.152 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0037837259999999998 2024-09-16T20:50:01.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.152 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.152 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:01.152 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566468+0000", 2024-09-16T20:50:01.152 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3350000000000001e-06 2024-09-16T20:50:01.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.152 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.152 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:01.152 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570306+0000", 2024-09-16T20:50:01.152 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.003837997 2024-09-16T20:50:01.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.152 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.152 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:01.153 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570392+0000", 2024-09-16T20:50:01.153 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5872999999999996e-05 2024-09-16T20:50:01.153 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.153 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.153 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:01.153 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571373+0000", 2024-09-16T20:50:01.153 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00098071399999999993 2024-09-16T20:50:01.153 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:01.153 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:01.153 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:01.153 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.153 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.153 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5255.0:1931 3.2 3:47882819:::rbd_data.110eb5f95afc.0000000000000009:head [read 217088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:50:01.154 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:50:01.154 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.566803651000001, 2024-09-16T20:50:01.154 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0082752880000001, 2024-09-16T20:50:01.154 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:01.154 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:01.154 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:01.154 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:01.154 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5255", 2024-09-16T20:50:01.154 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:50:01.154 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1931 2024-09-16T20:50:01.154 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.154 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:01.155 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.155 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:01.155 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:50:01.155 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:01.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.155 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.155 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:01.155 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:50:01.155 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:01.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.155 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.155 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:01.156 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:50:01.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9699999999999996e-07 2024-09-16T20:50:01.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.156 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.156 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:01.156 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562601+0000", 2024-09-16T20:50:01.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8600000000000001e-06 2024-09-16T20:50:01.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.156 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.156 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:01.156 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562608+0000", 2024-09-16T20:50:01.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9e-06 2024-09-16T20:50:01.157 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.157 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.157 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:01.157 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562615+0000", 2024-09-16T20:50:01.157 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8179999999999998e-06 2024-09-16T20:50:01.157 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.157 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.157 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:01.157 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562703+0000", 2024-09-16T20:50:01.157 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7811000000000003e-05 2024-09-16T20:50:01.157 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.158 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.158 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:50:01.158 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562706+0000", 2024-09-16T20:50:01.158 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4920000000000002e-06 2024-09-16T20:50:01.158 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.158 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.158 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:01.158 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566685+0000", 2024-09-16T20:50:01.158 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0039792780000000002 2024-09-16T20:50:01.158 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.158 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.158 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:01.158 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566693+0000", 2024-09-16T20:50:01.159 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4730000000000003e-06 2024-09-16T20:50:01.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.159 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.159 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:01.159 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570737+0000", 2024-09-16T20:50:01.159 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0040436749999999 2024-09-16T20:50:01.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.159 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.160 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:01.160 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570873+0000", 2024-09-16T20:50:01.160 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000136484 2024-09-16T20:50:01.160 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:01.160 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:01.160 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:01.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.160 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.160 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5255.0:1939 3.2 3:47882819:::rbd_data.110eb5f95afc.0000000000000009:head [read 233472~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:50:01.160 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:50:01.160 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.566648785000002, 2024-09-16T20:50:01.161 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0084657589999999, 2024-09-16T20:50:01.161 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:01.161 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:01.161 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:01.161 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:01.161 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5255", 2024-09-16T20:50:01.161 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:50:01.161 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1939 2024-09-16T20:50:01.161 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.161 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:01.161 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.161 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:01.161 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:50:01.162 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:01.162 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.162 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.162 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:01.162 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:50:01.162 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:01.162 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.162 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.162 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:01.162 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:50:01.162 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5199999999999997e-07 2024-09-16T20:50:01.162 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.162 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.162 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:01.162 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562757+0000", 2024-09-16T20:50:01.163 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4979999999999998e-06 2024-09-16T20:50:01.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.163 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.163 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:01.163 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562764+0000", 2024-09-16T20:50:01.163 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1470000000000004e-06 2024-09-16T20:50:01.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.163 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.163 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:01.163 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562768+0000", 2024-09-16T20:50:01.163 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7929999999999999e-06 2024-09-16T20:50:01.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.163 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.163 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:01.164 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562805+0000", 2024-09-16T20:50:01.164 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7477e-05 2024-09-16T20:50:01.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.164 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.164 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting_for_map not empty", 2024-09-16T20:50:01.164 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562810+0000", 2024-09-16T20:50:01.164 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0819999999999998e-06 2024-09-16T20:50:01.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.164 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.164 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:01.164 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566717+0000", 2024-09-16T20:50:01.165 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0039072600000000001 2024-09-16T20:50:01.165 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.165 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.165 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:01.165 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566724+0000", 2024-09-16T20:50:01.165 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4880000000000004e-06 2024-09-16T20:50:01.165 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.165 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.165 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:01.165 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571009+0000", 2024-09-16T20:50:01.165 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.004284677 2024-09-16T20:50:01.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.166 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.166 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:01.166 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571218+0000", 2024-09-16T20:50:01.166 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00020978500000000001 2024-09-16T20:50:01.166 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:01.166 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:01.166 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:01.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.166 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.166 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:11020 3.1f 3:fee2d1dd:::rbd_data.110ea93691bb.000000000000000e:23 [read 955392~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:01.166 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.693811+0000", 2024-09-16T20:50:01.166 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 10.435590127999999, 2024-09-16T20:50:01.167 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056833005999999998, 2024-09-16T20:50:01.167 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:01.167 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:01.167 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:01.167 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:01.167 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:01.167 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:01.167 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 11020 2024-09-16T20:50:01.167 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.167 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:01.167 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.167 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:01.167 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693811+0000", 2024-09-16T20:50:01.167 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:01.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.168 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.168 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:01.168 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693811+0000", 2024-09-16T20:50:01.168 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:01.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.168 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.168 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:01.168 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693812+0000", 2024-09-16T20:50:01.168 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6100000000000001e-07 2024-09-16T20:50:01.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.168 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.168 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:01.169 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693815+0000", 2024-09-16T20:50:01.169 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7389999999999999e-06 2024-09-16T20:50:01.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.169 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.169 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:01.169 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693823+0000", 2024-09-16T20:50:01.169 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0439999999999996e-06 2024-09-16T20:50:01.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.169 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.169 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:01.169 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693828+0000", 2024-09-16T20:50:01.170 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7440000000000004e-06 2024-09-16T20:50:01.170 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.170 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.170 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:01.170 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750357+0000", 2024-09-16T20:50:01.170 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056529192999999998 2024-09-16T20:50:01.170 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.170 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.170 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:01.170 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750429+0000", 2024-09-16T20:50:01.170 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1557000000000006e-05 2024-09-16T20:50:01.171 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.171 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.171 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:01.171 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750644+0000", 2024-09-16T20:50:01.171 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00021526800000000001 2024-09-16T20:50:01.171 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:01.171 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:01.171 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:01.171 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.171 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.171 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:11021 3.1f 3:fee2d1dd:::rbd_data.110ea93691bb.000000000000000e:23 [read 960512~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:01.171 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.693837+0000", 2024-09-16T20:50:01.171 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 10.43556424, 2024-09-16T20:50:01.171 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057155138000000001, 2024-09-16T20:50:01.172 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:01.172 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:01.172 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:01.172 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:01.172 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:01.172 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:01.172 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 11021 2024-09-16T20:50:01.172 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.172 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:01.172 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.172 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:01.172 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693837+0000", 2024-09-16T20:50:01.172 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:01.173 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.173 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.173 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:01.173 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693837+0000", 2024-09-16T20:50:01.173 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:01.173 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.173 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.173 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:01.173 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699566+0000", 2024-09-16T20:50:01.173 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057283639999999997 2024-09-16T20:50:01.173 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.173 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.173 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:01.174 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699571+0000", 2024-09-16T20:50:01.174 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.604e-06 2024-09-16T20:50:01.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.174 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.174 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:01.174 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699583+0000", 2024-09-16T20:50:01.174 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1372999999999999e-05 2024-09-16T20:50:01.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.174 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.174 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:01.174 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699591+0000", 2024-09-16T20:50:01.174 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5259999999999993e-06 2024-09-16T20:50:01.175 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.175 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.175 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:01.175 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750674+0000", 2024-09-16T20:50:01.175 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051083154999999998 2024-09-16T20:50:01.175 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.175 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.175 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:01.175 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750745+0000", 2024-09-16T20:50:01.175 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0933999999999995e-05 2024-09-16T20:50:01.175 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.176 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.176 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:01.176 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750992+0000", 2024-09-16T20:50:01.176 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00024718199999999998 2024-09-16T20:50:01.176 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:01.176 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:01.176 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:01.176 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.176 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.176 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8789 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2314240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:01.176 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057183+0000", 2024-09-16T20:50:01.176 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.072218322000000001, 2024-09-16T20:50:01.176 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056736416999999997, 2024-09-16T20:50:01.176 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:01.177 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:01.177 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:01.177 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:01.177 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:01.177 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:01.177 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8789 2024-09-16T20:50:01.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.177 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:01.177 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.177 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:01.177 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057183+0000", 2024-09-16T20:50:01.177 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:01.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.178 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.178 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:01.178 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057183+0000", 2024-09-16T20:50:01.178 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:01.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.178 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.178 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:01.178 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057183+0000", 2024-09-16T20:50:01.178 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-16T20:50:01.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.178 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.179 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:01.179 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057186+0000", 2024-09-16T20:50:01.179 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3889999999999999e-06 2024-09-16T20:50:01.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.179 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.179 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:01.179 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057192+0000", 2024-09-16T20:50:01.179 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8950000000000003e-06 2024-09-16T20:50:01.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.179 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.179 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:01.179 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057196+0000", 2024-09-16T20:50:01.179 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5159999999999996e-06 2024-09-16T20:50:01.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.180 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.180 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:01.180 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.113826+0000", 2024-09-16T20:50:01.180 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056629298000000002 2024-09-16T20:50:01.180 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.180 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.180 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:01.180 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.113866+0000", 2024-09-16T20:50:01.180 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0423000000000002e-05 2024-09-16T20:50:01.180 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.180 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.181 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:01.181 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.113920+0000", 2024-09-16T20:50:01.181 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3526000000000002e-05 2024-09-16T20:50:01.181 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:01.181 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:01.181 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:01.181 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.181 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.181 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8790 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2590720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:01.181 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057257+0000", 2024-09-16T20:50:01.181 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.072144920000000001, 2024-09-16T20:50:01.181 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056791169000000002, 2024-09-16T20:50:01.181 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:01.182 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:01.182 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:01.182 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:01.182 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:01.182 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:01.182 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8790 2024-09-16T20:50:01.182 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.182 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:01.182 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.182 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:01.182 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057257+0000", 2024-09-16T20:50:01.182 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:01.183 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.183 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.183 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:01.183 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057257+0000", 2024-09-16T20:50:01.183 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:01.183 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.183 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.183 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:01.183 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057257+0000", 2024-09-16T20:50:01.183 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6600000000000002e-07 2024-09-16T20:50:01.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.184 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.184 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:01.184 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057260+0000", 2024-09-16T20:50:01.184 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7240000000000001e-06 2024-09-16T20:50:01.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.184 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.184 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:01.184 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057267+0000", 2024-09-16T20:50:01.184 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0600000000000002e-06 2024-09-16T20:50:01.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.185 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.185 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:01.185 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057272+0000", 2024-09-16T20:50:01.185 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0390000000000004e-06 2024-09-16T20:50:01.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.185 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.185 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:01.185 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.113950+0000", 2024-09-16T20:50:01.185 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056677989999999998 2024-09-16T20:50:01.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.186 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.186 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:01.186 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.113993+0000", 2024-09-16T20:50:01.186 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3205999999999999e-05 2024-09-16T20:50:01.186 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.186 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.186 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:01.186 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114048+0000", 2024-09-16T20:50:01.186 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4684e-05 2024-09-16T20:50:01.186 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:01.186 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:01.186 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:01.186 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.186 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.187 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8791 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2332672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:01.187 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057279+0000", 2024-09-16T20:50:01.187 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.072122505000000003, 2024-09-16T20:50:01.187 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056890067000000002, 2024-09-16T20:50:01.187 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:01.187 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:01.187 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:01.187 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:01.187 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:01.187 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:01.187 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8791 2024-09-16T20:50:01.187 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.187 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:01.187 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.188 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:01.188 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057279+0000", 2024-09-16T20:50:01.188 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:01.188 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.188 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.188 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:01.189 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057279+0000", 2024-09-16T20:50:01.189 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:01.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.189 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.189 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:01.189 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057279+0000", 2024-09-16T20:50:01.189 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.15e-07 2024-09-16T20:50:01.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.189 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.189 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:01.189 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057281+0000", 2024-09-16T20:50:01.189 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8080000000000001e-06 2024-09-16T20:50:01.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.190 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.190 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:01.190 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057287+0000", 2024-09-16T20:50:01.190 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5999999999999997e-06 2024-09-16T20:50:01.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.190 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.190 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:01.190 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057291+0000", 2024-09-16T20:50:01.190 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5990000000000004e-06 2024-09-16T20:50:01.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.190 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.190 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:01.190 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114078+0000", 2024-09-16T20:50:01.190 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056786246999999998 2024-09-16T20:50:01.191 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.191 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.191 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:01.191 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114118+0000", 2024-09-16T20:50:01.191 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9898999999999999e-05 2024-09-16T20:50:01.191 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.191 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.191 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:01.191 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114169+0000", 2024-09-16T20:50:01.191 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1499000000000003e-05 2024-09-16T20:50:01.191 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:01.191 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:01.191 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:01.191 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.192 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.192 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8792 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2609152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:01.192 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:50:01.192 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.072104523000000004, 2024-09-16T20:50:01.192 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057003524999999999, 2024-09-16T20:50:01.192 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:01.192 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:01.192 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:01.192 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:01.192 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:01.192 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:01.192 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8792 2024-09-16T20:50:01.192 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.193 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:01.193 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.193 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:01.193 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:50:01.193 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:01.193 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.193 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.193 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:01.193 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:50:01.193 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:01.193 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.193 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.193 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:01.193 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:50:01.194 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0400000000000002e-07 2024-09-16T20:50:01.194 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.194 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.194 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:01.194 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057299+0000", 2024-09-16T20:50:01.194 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7460000000000001e-06 2024-09-16T20:50:01.194 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.194 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.194 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:01.194 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057304+0000", 2024-09-16T20:50:01.194 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3210000000000004e-06 2024-09-16T20:50:01.194 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.194 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.194 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:01.195 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057310+0000", 2024-09-16T20:50:01.195 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4360000000000001e-06 2024-09-16T20:50:01.195 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.195 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.195 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:01.195 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114200+0000", 2024-09-16T20:50:01.195 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056889954 2024-09-16T20:50:01.195 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.195 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.195 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:01.195 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114243+0000", 2024-09-16T20:50:01.195 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2998999999999999e-05 2024-09-16T20:50:01.195 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.195 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.196 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:01.196 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114300+0000", 2024-09-16T20:50:01.196 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7765e-05 2024-09-16T20:50:01.196 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:01.196 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:01.196 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:01.196 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.196 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.196 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8794 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2351104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:01.196 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057327+0000", 2024-09-16T20:50:01.196 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.072074199000000005, 2024-09-16T20:50:01.196 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057101130999999999, 2024-09-16T20:50:01.197 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:01.197 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:01.197 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:01.197 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:01.197 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:01.197 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:01.197 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8794 2024-09-16T20:50:01.197 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.197 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:01.197 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.197 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:01.198 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057327+0000", 2024-09-16T20:50:01.198 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:01.198 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.198 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.198 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:01.198 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057327+0000", 2024-09-16T20:50:01.198 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:01.198 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.198 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.198 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:01.198 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057328+0000", 2024-09-16T20:50:01.198 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7399999999999999e-07 2024-09-16T20:50:01.198 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.199 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.199 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:01.199 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057330+0000", 2024-09-16T20:50:01.199 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2199999999999999e-06 2024-09-16T20:50:01.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.199 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.199 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:01.199 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057336+0000", 2024-09-16T20:50:01.199 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1240000000000002e-06 2024-09-16T20:50:01.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.199 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.199 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:01.199 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057341+0000", 2024-09-16T20:50:01.200 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8470000000000003e-06 2024-09-16T20:50:01.200 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.200 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.200 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:01.200 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114334+0000", 2024-09-16T20:50:01.200 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056993267 2024-09-16T20:50:01.200 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.200 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.200 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:01.200 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114376+0000", 2024-09-16T20:50:01.200 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1638999999999999e-05 2024-09-16T20:50:01.200 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.200 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.200 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:01.201 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114428+0000", 2024-09-16T20:50:01.201 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2660000000000001e-05 2024-09-16T20:50:01.201 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:01.201 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:01.201 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:01.201 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.201 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.201 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8798 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2297856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:01.201 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:50:01.201 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.072054471999999994, 2024-09-16T20:50:01.201 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057277041000000001, 2024-09-16T20:50:01.201 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:01.201 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:01.202 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:01.202 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:01.202 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:01.202 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:01.202 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8798 2024-09-16T20:50:01.202 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.202 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:01.202 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.202 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:01.202 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:50:01.202 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:01.202 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.202 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.202 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:01.203 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:50:01.203 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:01.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.203 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.203 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:01.203 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:50:01.203 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2800000000000003e-07 2024-09-16T20:50:01.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.203 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.203 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:01.203 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057354+0000", 2024-09-16T20:50:01.203 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7780000000000002e-06 2024-09-16T20:50:01.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.203 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.204 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:01.204 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057360+0000", 2024-09-16T20:50:01.204 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0419999999999999e-06 2024-09-16T20:50:01.204 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.204 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.204 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:01.204 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057365+0000", 2024-09-16T20:50:01.204 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7929999999999997e-06 2024-09-16T20:50:01.204 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.204 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.204 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:01.204 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114460+0000", 2024-09-16T20:50:01.204 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057094662999999997 2024-09-16T20:50:01.205 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.205 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.205 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:01.205 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114532+0000", 2024-09-16T20:50:01.205 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2881999999999998e-05 2024-09-16T20:50:01.205 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.205 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.205 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:01.205 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114624+0000", 2024-09-16T20:50:01.205 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1555e-05 2024-09-16T20:50:01.205 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:01.205 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:01.205 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:01.205 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.206 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.206 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8800 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2316288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:01.206 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:50:01.206 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.072030532999999994, 2024-09-16T20:50:01.206 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.05746946, 2024-09-16T20:50:01.206 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:01.206 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:01.206 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:01.206 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:01.206 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:01.206 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:01.206 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8800 2024-09-16T20:50:01.206 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.206 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:01.207 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.207 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:01.207 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:50:01.207 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:01.207 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.207 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.207 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:01.207 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:50:01.207 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:01.207 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.207 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.207 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:01.207 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:50:01.207 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5199999999999998e-07 2024-09-16T20:50:01.207 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.208 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.208 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:01.208 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057373+0000", 2024-09-16T20:50:01.208 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.79e-06 2024-09-16T20:50:01.208 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.208 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.208 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:01.208 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057379+0000", 2024-09-16T20:50:01.208 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6699999999999999e-06 2024-09-16T20:50:01.208 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.208 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.208 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:01.208 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057386+0000", 2024-09-16T20:50:01.209 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3170000000000004e-06 2024-09-16T20:50:01.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.209 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.209 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:01.209 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114655+0000", 2024-09-16T20:50:01.209 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057269420000000001 2024-09-16T20:50:01.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.209 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.209 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:01.209 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114734+0000", 2024-09-16T20:50:01.209 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8084000000000007e-05 2024-09-16T20:50:01.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.209 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.209 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:01.210 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114840+0000", 2024-09-16T20:50:01.210 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000106827 2024-09-16T20:50:01.210 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:01.210 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:01.210 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:01.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.210 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.210 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8801 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2334720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:01.210 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:50:01.210 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.072009502000000003, 2024-09-16T20:50:01.210 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057594682000000001, 2024-09-16T20:50:01.210 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:01.210 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:01.210 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:01.211 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:01.211 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:01.211 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:01.211 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8801 2024-09-16T20:50:01.211 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.211 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:01.211 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.211 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:01.211 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:50:01.211 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:01.211 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.211 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.211 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:01.211 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:50:01.212 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:01.212 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.212 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.212 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:01.212 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:50:01.212 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6899999999999998e-07 2024-09-16T20:50:01.212 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.212 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.212 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:01.212 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057394+0000", 2024-09-16T20:50:01.212 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8300000000000001e-06 2024-09-16T20:50:01.212 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.212 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.213 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:01.213 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057400+0000", 2024-09-16T20:50:01.213 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6500000000000001e-06 2024-09-16T20:50:01.213 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.213 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.213 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:01.213 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057404+0000", 2024-09-16T20:50:01.213 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4669999999999998e-06 2024-09-16T20:50:01.213 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.213 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.213 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:01.213 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114874+0000", 2024-09-16T20:50:01.213 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057469602000000002 2024-09-16T20:50:01.213 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.214 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.214 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:01.214 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114933+0000", 2024-09-16T20:50:01.214 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9500000000000003e-05 2024-09-16T20:50:01.214 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.214 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.214 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:01.214 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114987+0000", 2024-09-16T20:50:01.214 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3263999999999997e-05 2024-09-16T20:50:01.214 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:01.214 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:01.214 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:01.214 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.214 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.214 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8802 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2353152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:01.215 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:50:01.215 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.071960816999999996, 2024-09-16T20:50:01.215 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057714079000000001, 2024-09-16T20:50:01.215 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:01.215 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:01.215 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:01.215 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:01.215 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:01.215 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:01.215 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8802 2024-09-16T20:50:01.215 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.215 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:01.215 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.215 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:01.216 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:50:01.216 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:01.216 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.216 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.216 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:01.216 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:50:01.216 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:01.216 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.216 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.216 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:01.216 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:50:01.216 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4999999999999998e-07 2024-09-16T20:50:01.216 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.216 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.217 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:01.217 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057446+0000", 2024-09-16T20:50:01.217 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4240000000000003e-06 2024-09-16T20:50:01.217 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.217 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.217 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:01.217 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057452+0000", 2024-09-16T20:50:01.217 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8379999999999996e-06 2024-09-16T20:50:01.217 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.217 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.217 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:01.217 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057460+0000", 2024-09-16T20:50:01.217 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9629999999999998e-06 2024-09-16T20:50:01.217 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.218 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.218 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:01.218 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115015+0000", 2024-09-16T20:50:01.218 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057554389999999997 2024-09-16T20:50:01.218 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.218 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.218 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:01.218 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115082+0000", 2024-09-16T20:50:01.218 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7733999999999999e-05 2024-09-16T20:50:01.218 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.218 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.218 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:01.218 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115155+0000", 2024-09-16T20:50:01.218 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2280000000000006e-05 2024-09-16T20:50:01.219 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:01.219 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:01.219 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:01.219 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.219 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.219 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8804 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2555904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:01.219 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:50:01.219 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.071934500999999998, 2024-09-16T20:50:01.219 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057833862, 2024-09-16T20:50:01.219 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:01.219 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:01.219 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:01.219 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:01.219 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:01.220 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:01.220 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8804 2024-09-16T20:50:01.220 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.220 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:01.220 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.220 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:01.220 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:50:01.220 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:01.220 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.220 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.220 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:01.220 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:50:01.220 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:01.221 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.221 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.221 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:01.221 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:50:01.221 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7899999999999999e-07 2024-09-16T20:50:01.221 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.221 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.221 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:01.221 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057469+0000", 2024-09-16T20:50:01.221 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.001e-06 2024-09-16T20:50:01.221 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.221 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.221 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:01.221 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057475+0000", 2024-09-16T20:50:01.222 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7629999999999997e-06 2024-09-16T20:50:01.222 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.222 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.222 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:01.222 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057480+0000", 2024-09-16T20:50:01.222 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8069999999999999e-06 2024-09-16T20:50:01.222 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.222 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.222 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:01.222 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115183+0000", 2024-09-16T20:50:01.222 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057703364 2024-09-16T20:50:01.222 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.222 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.222 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:01.222 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115227+0000", 2024-09-16T20:50:01.223 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3903999999999998e-05 2024-09-16T20:50:01.223 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.223 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.223 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:01.223 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115301+0000", 2024-09-16T20:50:01.223 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3543999999999999e-05 2024-09-16T20:50:01.223 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:01.223 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:01.223 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:01.223 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.223 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.223 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8805 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2574336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:01.223 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057487+0000", 2024-09-16T20:50:01.223 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.071914172999999998, 2024-09-16T20:50:01.224 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058021027000000003, 2024-09-16T20:50:01.224 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:01.224 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:01.224 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:01.224 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:01.224 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:01.224 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:01.224 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8805 2024-09-16T20:50:01.224 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.224 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:01.224 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.224 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:01.224 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057487+0000", 2024-09-16T20:50:01.225 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:01.225 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.225 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.225 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:01.225 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057487+0000", 2024-09-16T20:50:01.225 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:01.225 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.225 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.225 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:01.225 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057488+0000", 2024-09-16T20:50:01.225 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5199999999999998e-07 2024-09-16T20:50:01.225 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.225 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.225 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:01.226 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057490+0000", 2024-09-16T20:50:01.226 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.939e-06 2024-09-16T20:50:01.226 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.226 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.226 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:01.226 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057495+0000", 2024-09-16T20:50:01.226 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6439999999999997e-06 2024-09-16T20:50:01.226 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.226 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.226 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:01.226 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057500+0000", 2024-09-16T20:50:01.226 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8119999999999998e-06 2024-09-16T20:50:01.226 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.227 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.227 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:01.227 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115341+0000", 2024-09-16T20:50:01.227 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057841341999999997 2024-09-16T20:50:01.227 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.227 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.227 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:01.227 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115422+0000", 2024-09-16T20:50:01.227 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1003999999999999e-05 2024-09-16T20:50:01.227 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.227 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.227 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:01.227 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115508+0000", 2024-09-16T20:50:01.227 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5933999999999994e-05 2024-09-16T20:50:01.227 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:01.228 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:01.228 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:01.228 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.228 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.228 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8807 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2592768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:01.228 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057505+0000", 2024-09-16T20:50:01.228 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.071895962999999993, 2024-09-16T20:50:01.228 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058186633000000001, 2024-09-16T20:50:01.228 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:01.228 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:01.228 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:01.228 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:01.228 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:01.229 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:01.229 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8807 2024-09-16T20:50:01.229 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.229 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:01.229 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.229 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:01.229 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057505+0000", 2024-09-16T20:50:01.229 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:01.229 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.229 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.229 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:01.229 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057505+0000", 2024-09-16T20:50:01.229 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:01.229 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.230 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.230 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:01.230 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057506+0000", 2024-09-16T20:50:01.230 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.27e-07 2024-09-16T20:50:01.230 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.230 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.230 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:01.230 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057508+0000", 2024-09-16T20:50:01.230 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7400000000000001e-06 2024-09-16T20:50:01.230 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.230 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.230 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:01.230 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057513+0000", 2024-09-16T20:50:01.230 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.468e-06 2024-09-16T20:50:01.230 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.231 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.231 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:01.231 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057517+0000", 2024-09-16T20:50:01.231 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4329999999999997e-06 2024-09-16T20:50:01.231 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.231 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.231 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:01.231 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115538+0000", 2024-09-16T20:50:01.231 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058020355000000003 2024-09-16T20:50:01.231 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.231 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.231 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:01.231 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115616+0000", 2024-09-16T20:50:01.231 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7998999999999996e-05 2024-09-16T20:50:01.232 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.232 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.232 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:01.232 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115692+0000", 2024-09-16T20:50:01.232 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6310999999999995e-05 2024-09-16T20:50:01.232 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:01.232 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:01.232 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:01.232 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.232 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.232 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8808 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2611200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:01.232 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:50:01.232 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.071878702000000003, 2024-09-16T20:50:01.233 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058354046, 2024-09-16T20:50:01.233 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:01.233 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:01.233 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:01.233 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:01.233 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:01.233 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:01.233 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8808 2024-09-16T20:50:01.233 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.233 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:01.233 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.233 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:01.233 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:50:01.233 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:01.233 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.234 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.234 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:01.234 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:50:01.234 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:01.234 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.234 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.234 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:01.234 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:50:01.234 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2500000000000001e-07 2024-09-16T20:50:01.234 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.234 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.234 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:01.234 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057525+0000", 2024-09-16T20:50:01.234 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5549999999999999e-06 2024-09-16T20:50:01.235 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.235 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.235 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:01.235 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057530+0000", 2024-09-16T20:50:01.235 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5489999999999997e-06 2024-09-16T20:50:01.235 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.235 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.235 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:01.235 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057535+0000", 2024-09-16T20:50:01.235 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5709999999999999e-06 2024-09-16T20:50:01.235 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.235 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.235 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:01.235 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115722+0000", 2024-09-16T20:50:01.235 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058187323999999999 2024-09-16T20:50:01.236 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.236 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.236 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:01.236 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115799+0000", 2024-09-16T20:50:01.236 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6445000000000004e-05 2024-09-16T20:50:01.236 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:01.236 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:01.236 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:01.236 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115877+0000", 2024-09-16T20:50:01.236 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8276999999999996e-05 2024-09-16T20:50:01.236 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:01.236 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:01.236 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:01.237 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:01.237 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:01.237 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:01.238 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:01.236+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:01.238 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:50:01.280 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:50:01.280 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:50:01.301 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:50:01.301 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:50:01.330 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:50:01.330 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:50:01.338 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:01.337+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:01.369 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:50:01.369 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:50:01.392 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:01.392 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:01.404 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:50:01.404 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:50:01.407 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:50:01.439 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:50:01.439 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:50:01.439 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:01.438+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:01.540 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:01.539+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:01.548 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:01.548 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:01.557 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:50:01.640 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:01.639+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:01.651 INFO:teuthology.orchestra.run.smithi029.stderr:noscrub is unset 2024-09-16T20:50:01.669 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:01.669 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:01.678 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:50:01.765 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:01.764+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:01.767 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #35, image image_client.0-clone35 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent36 2024-09-16T20:50:01.840 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:01.840 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:01.842 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:01.841+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:01.850 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:50:01.944 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:01.942+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:01.981 INFO:teuthology.orchestra.run.smithi139.stdout:4829 read 0x4164000 thru 0x4168c65 (0x4c66 bytes) 2024-09-16T20:50:01.984 INFO:teuthology.orchestra.run.smithi139.stdout:4830 trunc from 0x6f77400 to 0x8e4f200 2024-09-16T20:50:02.002 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:02.002 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:02.008 INFO:teuthology.orchestra.run.smithi139.stdout:4831 trunc from 0x8e4f200 to 0x3c7fe00 2024-09-16T20:50:02.011 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:50:02.045 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:02.043+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:02.047 INFO:teuthology.orchestra.run.smithi139.stdout:4832 trunc from 0x3c7fe00 to 0x6838800 2024-09-16T20:50:02.061 INFO:teuthology.orchestra.run.smithi139.stdout:4835 write 0x3c08e00 thru 0x3c174fc (0xe6fd bytes) 2024-09-16T20:50:02.064 INFO:teuthology.orchestra.run.smithi139.stdout:4837 trunc from 0x6838800 to 0x87dec00 2024-09-16T20:50:02.098 INFO:teuthology.orchestra.run.smithi139.stdout:4838 write 0xb070a00 thru 0xb07b52f (0xab30 bytes) 2024-09-16T20:50:02.098 INFO:teuthology.orchestra.run.smithi139.stdout:4840 write 0xdae200 thru 0xdb7a89 (0x988a bytes) 2024-09-16T20:50:02.099 INFO:teuthology.orchestra.run.smithi139.stdout:4842 read 0x8c3fa00 thru 0x8c4bd57 (0xc358 bytes) 2024-09-16T20:50:02.122 INFO:teuthology.orchestra.run.smithi139.stdout:4849 write 0xecd7200 thru 0xecdfef9 (0x8cfa bytes) 2024-09-16T20:50:02.122 INFO:teuthology.orchestra.run.smithi139.stdout:4851 write 0xaf16600 thru 0xaf2480e (0xe20f bytes) 2024-09-16T20:50:02.123 INFO:teuthology.orchestra.run.smithi139.stdout:4852 write 0x8894200 thru 0x88a16bb (0xd4bc bytes) 2024-09-16T20:50:02.123 INFO:teuthology.orchestra.run.smithi139.stdout:4854 read 0x234da00 thru 0x23555e2 (0x7be3 bytes) 2024-09-16T20:50:02.123 INFO:teuthology.orchestra.run.smithi139.stdout:4857 punch from 0x7e32000 to 0x7e3da00, (0xba00 bytes) 2024-09-16T20:50:02.123 INFO:teuthology.orchestra.run.smithi139.stdout:4858 write 0x5c1ec00 thru 0x5c24192 (0x5593 bytes) 2024-09-16T20:50:02.127 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:02.128 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:02.128 INFO:teuthology.orchestra.run.smithi139.stdout:4860 trunc from 0xece0000 to 0xb791000 2024-09-16T20:50:02.138 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:50:02.145 INFO:teuthology.orchestra.run.smithi139.stdout:4861 write 0xa93bc00 thru 0xa9433f4 (0x77f5 bytes) 2024-09-16T20:50:02.152 INFO:teuthology.orchestra.run.smithi139.stdout:4862 punch from 0x93b6a00 to 0x93bdc00, (0x7200 bytes) 2024-09-16T20:50:02.153 INFO:teuthology.orchestra.run.smithi139.stdout:4864 write 0x8c99e00 thru 0x8ca5c18 (0xbe19 bytes) 2024-09-16T20:50:02.154 INFO:teuthology.orchestra.run.smithi139.stdout:4865 write 0x5d1d000 thru 0x5d1fee0 (0x2ee1 bytes) 2024-09-16T20:50:02.154 INFO:teuthology.orchestra.run.smithi139.stdout:4866 read 0x5b8a200 thru 0x5b90162 (0x5f63 bytes) 2024-09-16T20:50:02.155 INFO:teuthology.orchestra.run.smithi139.stdout:4867 punch from 0x4ef8c00 to 0x4efcc00, (0x4000 bytes) 2024-09-16T20:50:02.165 INFO:teuthology.orchestra.run.smithi139.stdout:4868 write 0xd37d600 thru 0xd380303 (0x2d04 bytes) 2024-09-16T20:50:02.182 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:02.181+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:02.185 INFO:teuthology.orchestra.run.smithi139.stdout:4870 write 0xd6d6600 thru 0xd6e1a9d (0xb49e bytes) 2024-09-16T20:50:02.185 INFO:teuthology.orchestra.run.smithi139.stdout:4872 punch from 0x7183c00 to 0x718b000, (0x7400 bytes) 2024-09-16T20:50:02.185 INFO:teuthology.orchestra.run.smithi139.stdout:4873 write 0x86c00 thru 0x9117b (0xa57c bytes) 2024-09-16T20:50:02.195 INFO:teuthology.orchestra.run.smithi139.stdout:4875 trunc from 0xd6e1c00 to 0x5422400 2024-09-16T20:50:02.212 INFO:teuthology.orchestra.run.smithi139.stdout:4876 write 0x46acc00 thru 0x46b3af4 (0x6ef5 bytes) 2024-09-16T20:50:02.220 INFO:teuthology.orchestra.run.smithi139.stdout:4878 read 0x51be400 thru 0x51c2d99 (0x499a bytes) 2024-09-16T20:50:02.241 INFO:teuthology.orchestra.run.smithi139.stdout:4880 write 0xdb78200 thru 0xdb7f25c (0x705d bytes) 2024-09-16T20:50:02.242 INFO:teuthology.orchestra.run.smithi139.stdout:4881 punch from 0xa7bbc00 to 0xa7bd000, (0x1400 bytes) 2024-09-16T20:50:02.243 INFO:teuthology.orchestra.run.smithi139.stdout:4882 read 0x1990e00 thru 0x1993af7 (0x2cf8 bytes) 2024-09-16T20:50:02.246 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:02.245+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:02.246 INFO:teuthology.orchestra.run.smithi139.stdout:4884 trunc from 0xdb7f400 to 0xe065800 2024-09-16T20:50:02.256 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:02.256 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:02.262 INFO:teuthology.orchestra.run.smithi139.stdout:4885 clone 38 order 22 su 16384 sc 4 2024-09-16T20:50:02.266 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:50:02.347 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:02.346+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:02.413 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:02.413 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:02.422 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:50:02.466 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:02.466+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:02.529 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:02.530 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:02.538 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:50:02.581 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:02.580+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:02.647 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:02.647 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:02.649 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:02.647+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:02.658 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:50:02.723 INFO:teuthology.orchestra.run.smithi139.stdout:leaving image image_client.0-clone37 intact 2024-09-16T20:50:02.750 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:02.749+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:02.806 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:02.807 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:02.815 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:50:02.856 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:02.854+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:02.952 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:02.951+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:02.965 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:02.965 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:02.974 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:50:03.053 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:03.051+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:03.079 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #36, image image_client.0-clone36 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent37 2024-09-16T20:50:03.083 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:03.083 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:03.092 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:50:03.154 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:03.154+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:03.255 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:03.254+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:03.261 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:03.261 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:03.271 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:50:03.356 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:03.354+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:03.418 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:03.418 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:03.427 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:50:03.438 INFO:teuthology.orchestra.run.smithi139.stdout:4886 read 0x5538e00 thru 0x553e0d2 (0x52d3 bytes) 2024-09-16T20:50:03.439 INFO:teuthology.orchestra.run.smithi139.stdout:4889 write 0x1c41000 thru 0x1c4c634 (0xb635 bytes) 2024-09-16T20:50:03.440 INFO:teuthology.orchestra.run.smithi139.stdout:4892 write 0x2575400 thru 0x25789e8 (0x35e9 bytes) 2024-09-16T20:50:03.442 INFO:teuthology.orchestra.run.smithi139.stdout:4893 write 0xcd0ea00 thru 0xcd19981 (0xaf82 bytes) 2024-09-16T20:50:03.444 INFO:teuthology.orchestra.run.smithi139.stdout:4894 trunc from 0xe065800 to 0xe9bbc00 2024-09-16T20:50:03.452 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T20:50:03.496 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:03.496+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:03.558 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:03.556+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:03.563 INFO:teuthology.orchestra.run.smithi139.stdout:4895 read 0xa8d2200 thru 0xa8dea2f (0xc830 bytes) 2024-09-16T20:50:03.576 INFO:teuthology.orchestra.run.smithi139.stdout:4897 trunc from 0xe9bbc00 to 0x4669e00 2024-09-16T20:50:03.615 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:03.615 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:03.624 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:50:03.624 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:50:03.625 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:50:03.637 INFO:teuthology.orchestra.run.smithi139.stdout:4898 punch from 0x2f25800 to 0x2f30400, (0xac00 bytes) 2024-09-16T20:50:03.643 INFO:teuthology.orchestra.run.smithi139.stdout:4899 trunc from 0x4669e00 to 0x570a800 2024-09-16T20:50:03.644 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:50:03.644 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:50:03.666 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T20:50:03.667 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:03.665+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:03.670 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:50:03.671 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:50:03.692 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:50:03.692 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:50:03.693 INFO:teuthology.orchestra.run.smithi139.stdout:4901 punch from 0x4c0e000 to 0x4c17200, (0x9200 bytes) 2024-09-16T20:50:03.713 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:50:03.713 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:50:03.718 INFO:teuthology.orchestra.run.smithi139.stdout:4904 write 0xd5f4a00 thru 0xd5fda6e (0x906f bytes) 2024-09-16T20:50:03.720 INFO:teuthology.orchestra.run.smithi139.stdout:4905 trunc from 0xd5fdc00 to 0xe9f3c00 2024-09-16T20:50:03.735 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:50:03.735 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:50:03.749 INFO:teuthology.orchestra.run.smithi139.stdout:4907 trunc from 0xe9f3c00 to 0x4398c00 2024-09-16T20:50:03.765 INFO:teuthology.orchestra.run.smithi139.stdout:4908 read 0xdba00 thru 0xe4faf (0x95b0 bytes) 2024-09-16T20:50:03.767 INFO:teuthology.orchestra.run.smithi139.stdout:4911 read 0xc3ec00 thru 0xc4ca8e (0xde8f bytes) 2024-09-16T20:50:03.784 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:03.784 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:50:03.784 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:50:03.784 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:50:03.784 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:03.785 INFO:teuthology.orchestra.run.smithi139.stdout:4912 write 0x9740800 thru 0x9742205 (0x1a06 bytes) 2024-09-16T20:50:03.785 INFO:teuthology.orchestra.run.smithi139.stdout:4914 write 0x5702400 thru 0x570c9b2 (0xa5b3 bytes) 2024-09-16T20:50:03.785 INFO:teuthology.orchestra.run.smithi139.stdout:4915 write 0x5e6e600 thru 0x5e6eaff (0x500 bytes) 2024-09-16T20:50:03.785 INFO:teuthology.orchestra.run.smithi139.stdout:4916 write 0x1f52400 thru 0x1f5b299 (0x8e9a bytes) 2024-09-16T20:50:03.786 INFO:teuthology.orchestra.run.smithi139.stdout:4918 read 0x6060800 thru 0x606ddcb (0xd5cc bytes) 2024-09-16T20:50:03.786 INFO:teuthology.orchestra.run.smithi139.stdout:4921 write 0x5843e00 thru 0x5846fff (0x3200 bytes) 2024-09-16T20:50:03.786 INFO:teuthology.orchestra.run.smithi139.stdout:4922 read 0x759e200 thru 0x75aa393 (0xc194 bytes) 2024-09-16T20:50:03.788 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:03.788+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:03.792 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:50:03.794 INFO:teuthology.orchestra.run.smithi139.stdout:4923 trunc from 0x9742400 to 0x287e800 2024-09-16T20:50:03.835 INFO:teuthology.orchestra.run.smithi139.stdout:4924 write 0x28f6200 thru 0x28fe118 (0x7f19 bytes) 2024-09-16T20:50:03.841 INFO:teuthology.orchestra.run.smithi139.stdout:4927 trunc from 0x28fe200 to 0x3006200 2024-09-16T20:50:03.854 INFO:teuthology.orchestra.run.smithi139.stdout:4931 read 0x294b800 thru 0x294d138 (0x1939 bytes) 2024-09-16T20:50:03.859 INFO:teuthology.orchestra.run.smithi139.stdout:4934 read 0xd98c00 thru 0xda3741 (0xab42 bytes) 2024-09-16T20:50:03.860 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:03.858+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:03.860 INFO:teuthology.orchestra.run.smithi139.stdout:4939 read 0x366200 thru 0x3672c0 (0x10c1 bytes) 2024-09-16T20:50:03.860 INFO:teuthology.orchestra.run.smithi139.stdout:4941 read 0x27e00 thru 0x33784 (0xb985 bytes) 2024-09-16T20:50:03.886 INFO:teuthology.orchestra.run.smithi139.stdout:4944 write 0xe99be00 thru 0xe99f004 (0x3205 bytes) 2024-09-16T20:50:03.893 INFO:teuthology.orchestra.run.smithi139.stdout:4949 trunc from 0xe99f200 to 0xc42ca00 2024-09-16T20:50:03.912 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:03.912 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:50:03.912 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:50:03.912 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:50:03.912 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.912 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5286.0:10643 3.17 3.f6273fb7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-16T20:50:03.912 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:50:03.912 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 55.119377507000003, 2024-09-16T20:50:03.912 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.76672347200000002, 2024-09-16T20:50:03.912 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:03.912 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:03.913 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:03.913 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:03.913 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5286", 2024-09-16T20:50:03.913 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/4135433989", 2024-09-16T20:50:03.913 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 10643 2024-09-16T20:50:03.913 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.914 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:03.914 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.914 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:03.914 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:50:03.914 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:03.915 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.915 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.915 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:03.915 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:50:03.915 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:03.915 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.915 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.916 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:03.916 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790489+0000", 2024-09-16T20:50:03.916 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5910000000000001e-06 2024-09-16T20:50:03.916 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.916 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.916 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:03.916 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790493+0000", 2024-09-16T20:50:03.917 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2669999999999998e-06 2024-09-16T20:50:03.917 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.917 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.917 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:03.917 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790500+0000", 2024-09-16T20:50:03.917 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3780000000000003e-06 2024-09-16T20:50:03.917 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.917 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.917 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:03.917 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790505+0000", 2024-09-16T20:50:03.917 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1560000000000003e-06 2024-09-16T20:50:03.917 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.918 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.918 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:03.918 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790559+0000", 2024-09-16T20:50:03.918 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3494e-05 2024-09-16T20:50:03.918 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.918 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.918 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:03.918 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790566+0000", 2024-09-16T20:50:03.918 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7649999999999993e-06 2024-09-16T20:50:03.918 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.918 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.918 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:03.918 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:09.557198+0000", 2024-09-16T20:50:03.919 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.76663136200000004 2024-09-16T20:50:03.919 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.919 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.919 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:03.919 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:09.557211+0000", 2024-09-16T20:50:03.919 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3458999999999999e-05 2024-09-16T20:50:03.919 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:03.919 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:03.919 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:03.919 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.919 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.919 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5512.0:450 3.2 3:44e0cffa:::rbd_header.110ef0d7a535:head [watch reconnect cookie 140263298018912 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:50:03.919 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:50:03.919 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.347354593999999, 2024-09-16T20:50:03.920 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0084581939999999, 2024-09-16T20:50:03.920 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:03.920 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:03.920 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:03.920 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:03.920 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5512", 2024-09-16T20:50:03.920 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:50:03.920 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 450 2024-09-16T20:50:03.920 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.920 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:03.920 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.920 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:03.920 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:50:03.920 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:03.921 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.921 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.921 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:03.921 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:50:03.921 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:03.921 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.921 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.921 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:03.921 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562512+0000", 2024-09-16T20:50:03.921 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.009e-06 2024-09-16T20:50:03.921 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.921 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.921 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:03.922 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562517+0000", 2024-09-16T20:50:03.922 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9339999999999997e-06 2024-09-16T20:50:03.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.922 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.922 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:03.922 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562530+0000", 2024-09-16T20:50:03.922 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3495e-05 2024-09-16T20:50:03.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.922 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.922 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:03.922 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562536+0000", 2024-09-16T20:50:03.922 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5690000000000004e-06 2024-09-16T20:50:03.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.922 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.923 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:03.923 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562591+0000", 2024-09-16T20:50:03.923 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5761999999999999e-05 2024-09-16T20:50:03.923 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.923 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.923 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:50:03.923 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562599+0000", 2024-09-16T20:50:03.923 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9179999999999994e-06 2024-09-16T20:50:03.923 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.923 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.923 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:03.923 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.565971+0000", 2024-09-16T20:50:03.923 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003371546 2024-09-16T20:50:03.923 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.924 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.924 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:03.924 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566019+0000", 2024-09-16T20:50:03.924 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7849000000000002e-05 2024-09-16T20:50:03.924 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.924 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.924 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:03.924 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.569382+0000", 2024-09-16T20:50:03.924 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.003362833 2024-09-16T20:50:03.924 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.924 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.924 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:03.924 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.569529+0000", 2024-09-16T20:50:03.924 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000147823 2024-09-16T20:50:03.924 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.925 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.925 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:03.925 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570969+0000", 2024-09-16T20:50:03.925 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001439456 2024-09-16T20:50:03.925 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:03.925 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:03.925 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:03.925 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.925 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.925 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:14996 3.2 3:44e0cffa:::rbd_header.110ef0d7a535:head [watch reconnect cookie 140213436269840 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:50:03.925 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:50:03.925 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.347306182000001, 2024-09-16T20:50:03.925 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.008813943, 2024-09-16T20:50:03.925 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:03.926 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:03.926 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:03.926 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:03.926 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:50:03.926 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:03.926 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 14996 2024-09-16T20:50:03.926 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.926 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:03.926 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.926 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:03.926 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:50:03.926 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:03.926 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.926 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.927 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:03.927 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:50:03.927 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:03.927 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.927 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.927 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:03.927 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562560+0000", 2024-09-16T20:50:03.927 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.06e-07 2024-09-16T20:50:03.927 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.927 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.927 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:03.927 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562563+0000", 2024-09-16T20:50:03.927 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.495e-06 2024-09-16T20:50:03.927 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.928 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.928 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:03.928 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562571+0000", 2024-09-16T20:50:03.928 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1950000000000003e-06 2024-09-16T20:50:03.928 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.928 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.928 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:03.928 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562579+0000", 2024-09-16T20:50:03.928 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2649999999999999e-06 2024-09-16T20:50:03.928 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.928 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.928 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:03.929 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562673+0000", 2024-09-16T20:50:03.929 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4118999999999998e-05 2024-09-16T20:50:03.929 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.929 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.929 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:50:03.929 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562677+0000", 2024-09-16T20:50:03.929 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5179999999999998e-06 2024-09-16T20:50:03.929 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.929 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.929 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:03.929 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566461+0000", 2024-09-16T20:50:03.929 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0037837259999999998 2024-09-16T20:50:03.929 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.929 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.930 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:03.930 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566468+0000", 2024-09-16T20:50:03.930 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3350000000000001e-06 2024-09-16T20:50:03.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.930 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.930 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:03.930 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570306+0000", 2024-09-16T20:50:03.930 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.003837997 2024-09-16T20:50:03.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.930 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.930 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:03.930 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570392+0000", 2024-09-16T20:50:03.930 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5872999999999996e-05 2024-09-16T20:50:03.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.931 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.931 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:03.931 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571373+0000", 2024-09-16T20:50:03.931 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00098071399999999993 2024-09-16T20:50:03.931 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:03.931 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:03.931 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:03.931 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.931 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.931 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5255.0:1931 3.2 3:47882819:::rbd_data.110eb5f95afc.0000000000000009:head [read 217088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:50:03.931 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:50:03.932 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.347267431999999, 2024-09-16T20:50:03.932 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0082752880000001, 2024-09-16T20:50:03.932 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:03.932 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:03.932 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:03.932 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:03.932 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5255", 2024-09-16T20:50:03.932 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:50:03.932 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1931 2024-09-16T20:50:03.933 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.933 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:03.933 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.933 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:03.933 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:50:03.933 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:03.933 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.933 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.933 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:03.933 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:50:03.933 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:03.933 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.933 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.934 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:03.934 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:50:03.934 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9699999999999996e-07 2024-09-16T20:50:03.934 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.934 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.934 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:03.934 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562601+0000", 2024-09-16T20:50:03.934 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8600000000000001e-06 2024-09-16T20:50:03.934 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.934 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.934 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:03.934 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562608+0000", 2024-09-16T20:50:03.935 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9e-06 2024-09-16T20:50:03.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.935 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.935 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:03.935 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562615+0000", 2024-09-16T20:50:03.935 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8179999999999998e-06 2024-09-16T20:50:03.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.935 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.935 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:03.935 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562703+0000", 2024-09-16T20:50:03.935 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7811000000000003e-05 2024-09-16T20:50:03.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.935 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.935 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:50:03.936 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562706+0000", 2024-09-16T20:50:03.936 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4920000000000002e-06 2024-09-16T20:50:03.936 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.936 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.936 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:03.936 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566685+0000", 2024-09-16T20:50:03.936 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0039792780000000002 2024-09-16T20:50:03.936 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.936 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.936 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:03.936 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566693+0000", 2024-09-16T20:50:03.936 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4730000000000003e-06 2024-09-16T20:50:03.936 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.937 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.937 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:03.937 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570737+0000", 2024-09-16T20:50:03.937 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0040436749999999 2024-09-16T20:50:03.937 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.937 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.937 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:03.937 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570873+0000", 2024-09-16T20:50:03.937 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000136484 2024-09-16T20:50:03.937 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:03.937 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:03.937 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:03.937 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.938 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.938 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5255.0:1939 3.2 3:47882819:::rbd_data.110eb5f95afc.0000000000000009:head [read 233472~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:50:03.938 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:50:03.938 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.347112566, 2024-09-16T20:50:03.938 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0084657589999999, 2024-09-16T20:50:03.938 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:03.938 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:03.938 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:03.938 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:03.938 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5255", 2024-09-16T20:50:03.938 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:50:03.938 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1939 2024-09-16T20:50:03.938 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.939 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:03.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.939 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:03.939 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:50:03.939 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:03.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.939 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:03.939 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:50:03.939 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:03.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:03.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:50:03.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5199999999999997e-07 2024-09-16T20:50:03.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:03.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562757+0000", 2024-09-16T20:50:03.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4979999999999998e-06 2024-09-16T20:50:03.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:03.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562764+0000", 2024-09-16T20:50:03.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1470000000000004e-06 2024-09-16T20:50:03.941 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.941 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.941 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:03.941 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562768+0000", 2024-09-16T20:50:03.941 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7929999999999999e-06 2024-09-16T20:50:03.941 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.941 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.941 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:03.941 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562805+0000", 2024-09-16T20:50:03.941 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7477e-05 2024-09-16T20:50:03.941 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.941 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.941 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting_for_map not empty", 2024-09-16T20:50:03.941 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562810+0000", 2024-09-16T20:50:03.941 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0819999999999998e-06 2024-09-16T20:50:03.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.942 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.942 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:03.942 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566717+0000", 2024-09-16T20:50:03.942 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0039072600000000001 2024-09-16T20:50:03.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.942 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.942 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:03.942 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566724+0000", 2024-09-16T20:50:03.942 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4880000000000004e-06 2024-09-16T20:50:03.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.942 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.943 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:03.943 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571009+0000", 2024-09-16T20:50:03.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.004284677 2024-09-16T20:50:03.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.943 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:03.943 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571218+0000", 2024-09-16T20:50:03.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00020978500000000001 2024-09-16T20:50:03.943 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:03.943 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:03.943 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:03.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.944 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:11020 3.1f 3:fee2d1dd:::rbd_data.110ea93691bb.000000000000000e:23 [read 955392~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:03.944 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.693811+0000", 2024-09-16T20:50:03.944 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 13.216053908999999, 2024-09-16T20:50:03.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056833005999999998, 2024-09-16T20:50:03.944 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:03.944 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:03.944 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:03.944 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:03.944 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:03.944 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:03.944 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 11020 2024-09-16T20:50:03.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.944 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:03.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.945 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:03.945 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693811+0000", 2024-09-16T20:50:03.945 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:03.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.945 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:03.945 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693811+0000", 2024-09-16T20:50:03.945 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:03.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.945 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:03.945 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693812+0000", 2024-09-16T20:50:03.946 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6100000000000001e-07 2024-09-16T20:50:03.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.946 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.946 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:03.946 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693815+0000", 2024-09-16T20:50:03.946 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7389999999999999e-06 2024-09-16T20:50:03.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.946 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.946 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:03.946 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693823+0000", 2024-09-16T20:50:03.946 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0439999999999996e-06 2024-09-16T20:50:03.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.946 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.947 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:03.947 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693828+0000", 2024-09-16T20:50:03.947 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7440000000000004e-06 2024-09-16T20:50:03.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.947 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.947 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:03.947 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750357+0000", 2024-09-16T20:50:03.947 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056529192999999998 2024-09-16T20:50:03.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.947 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.947 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:03.947 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750429+0000", 2024-09-16T20:50:03.948 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1557000000000006e-05 2024-09-16T20:50:03.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.948 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:03.948 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750644+0000", 2024-09-16T20:50:03.948 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00021526800000000001 2024-09-16T20:50:03.948 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:03.948 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:03.948 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:03.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.948 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:11021 3.1f 3:fee2d1dd:::rbd_data.110ea93691bb.000000000000000e:23 [read 960512~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:03.948 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.693837+0000", 2024-09-16T20:50:03.949 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 13.216028021, 2024-09-16T20:50:03.949 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057155138000000001, 2024-09-16T20:50:03.949 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:03.949 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:03.949 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:03.949 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:03.949 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:03.949 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:03.949 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 11021 2024-09-16T20:50:03.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.949 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:03.949 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.949 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:03.950 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693837+0000", 2024-09-16T20:50:03.950 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:03.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.950 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.950 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:03.950 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693837+0000", 2024-09-16T20:50:03.950 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:03.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.950 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.950 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:03.950 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699566+0000", 2024-09-16T20:50:03.950 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057283639999999997 2024-09-16T20:50:03.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.950 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.951 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:03.951 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699571+0000", 2024-09-16T20:50:03.951 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.604e-06 2024-09-16T20:50:03.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.951 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.951 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:03.951 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699583+0000", 2024-09-16T20:50:03.951 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1372999999999999e-05 2024-09-16T20:50:03.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.951 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.951 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:03.951 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699591+0000", 2024-09-16T20:50:03.951 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5259999999999993e-06 2024-09-16T20:50:03.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.952 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.952 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:03.952 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750674+0000", 2024-09-16T20:50:03.952 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051083154999999998 2024-09-16T20:50:03.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.952 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.952 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:03.952 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750745+0000", 2024-09-16T20:50:03.952 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0933999999999995e-05 2024-09-16T20:50:03.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.952 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.952 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:03.952 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750992+0000", 2024-09-16T20:50:03.953 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00024718199999999998 2024-09-16T20:50:03.953 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:03.953 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:03.953 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:03.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.953 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.953 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8789 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2314240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:03.953 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057183+0000", 2024-09-16T20:50:03.953 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 2.8526821030000002, 2024-09-16T20:50:03.953 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056736416999999997, 2024-09-16T20:50:03.953 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:03.953 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:03.953 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:03.954 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:03.954 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:03.954 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:03.954 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8789 2024-09-16T20:50:03.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.954 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:03.954 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.954 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:03.954 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057183+0000", 2024-09-16T20:50:03.954 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:03.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.954 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.955 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:03.955 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057183+0000", 2024-09-16T20:50:03.955 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:03.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.955 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:03.955 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057183+0000", 2024-09-16T20:50:03.955 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-16T20:50:03.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.955 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:03.955 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057186+0000", 2024-09-16T20:50:03.955 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3889999999999999e-06 2024-09-16T20:50:03.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.956 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:03.956 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057192+0000", 2024-09-16T20:50:03.956 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8950000000000003e-06 2024-09-16T20:50:03.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.956 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:03.956 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057196+0000", 2024-09-16T20:50:03.956 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5159999999999996e-06 2024-09-16T20:50:03.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.956 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:03.956 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.113826+0000", 2024-09-16T20:50:03.956 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056629298000000002 2024-09-16T20:50:03.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.957 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:03.957 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.113866+0000", 2024-09-16T20:50:03.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0423000000000002e-05 2024-09-16T20:50:03.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.957 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:03.957 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.113920+0000", 2024-09-16T20:50:03.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3526000000000002e-05 2024-09-16T20:50:03.957 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:03.957 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:03.957 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:03.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.958 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8790 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2590720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:03.958 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057257+0000", 2024-09-16T20:50:03.958 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 2.8526087009999999, 2024-09-16T20:50:03.958 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056791169000000002, 2024-09-16T20:50:03.958 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:03.958 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:03.958 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:03.958 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:03.958 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:03.958 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:03.958 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8790 2024-09-16T20:50:03.958 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.959 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:03.959 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.959 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:03.959 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057257+0000", 2024-09-16T20:50:03.959 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:03.959 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.959 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.959 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:03.959 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057257+0000", 2024-09-16T20:50:03.959 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:03.959 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.959 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.959 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:03.959 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057257+0000", 2024-09-16T20:50:03.960 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6600000000000002e-07 2024-09-16T20:50:03.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.960 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:03.960 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057260+0000", 2024-09-16T20:50:03.960 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7240000000000001e-06 2024-09-16T20:50:03.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.960 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:03.960 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057267+0000", 2024-09-16T20:50:03.960 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0600000000000002e-06 2024-09-16T20:50:03.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.961 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:03.961 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057272+0000", 2024-09-16T20:50:03.961 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0390000000000004e-06 2024-09-16T20:50:03.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.961 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.961 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:03.961 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.113950+0000", 2024-09-16T20:50:03.961 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056677989999999998 2024-09-16T20:50:03.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.961 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.961 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:03.961 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.113993+0000", 2024-09-16T20:50:03.961 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3205999999999999e-05 2024-09-16T20:50:03.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.962 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:03.962 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114048+0000", 2024-09-16T20:50:03.962 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4684e-05 2024-09-16T20:50:03.962 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:03.962 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:03.962 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:03.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.962 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8791 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2332672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:03.962 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057279+0000", 2024-09-16T20:50:03.962 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 2.8525862860000002, 2024-09-16T20:50:03.962 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056890067000000002, 2024-09-16T20:50:03.962 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:03.963 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:03.963 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:03.963 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:03.963 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:03.963 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:03.963 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8791 2024-09-16T20:50:03.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.963 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:03.963 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.963 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:03.963 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057279+0000", 2024-09-16T20:50:03.963 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:03.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.964 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.964 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:03.964 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057279+0000", 2024-09-16T20:50:03.964 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:03.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.964 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.964 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:03.964 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057279+0000", 2024-09-16T20:50:03.964 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.15e-07 2024-09-16T20:50:03.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.964 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.964 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:03.965 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057281+0000", 2024-09-16T20:50:03.965 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8080000000000001e-06 2024-09-16T20:50:03.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.965 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.965 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:03.965 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057287+0000", 2024-09-16T20:50:03.965 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5999999999999997e-06 2024-09-16T20:50:03.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.965 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.965 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:03.965 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057291+0000", 2024-09-16T20:50:03.965 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5990000000000004e-06 2024-09-16T20:50:03.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.966 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:03.966 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114078+0000", 2024-09-16T20:50:03.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056786246999999998 2024-09-16T20:50:03.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.966 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:03.966 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114118+0000", 2024-09-16T20:50:03.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9898999999999999e-05 2024-09-16T20:50:03.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.966 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:03.966 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114169+0000", 2024-09-16T20:50:03.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1499000000000003e-05 2024-09-16T20:50:03.967 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:03.967 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:03.967 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:03.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.967 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8792 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2609152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:03.967 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:50:03.967 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 2.8525683040000001, 2024-09-16T20:50:03.967 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057003524999999999, 2024-09-16T20:50:03.967 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:03.967 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:03.967 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:03.967 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:03.968 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:03.968 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:03.968 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8792 2024-09-16T20:50:03.968 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.968 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:03.968 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.968 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:03.968 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:50:03.968 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:03.968 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.968 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.968 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:03.968 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:50:03.969 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:03.969 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.969 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.969 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:03.969 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:50:03.969 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0400000000000002e-07 2024-09-16T20:50:03.969 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.969 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.969 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:03.969 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057299+0000", 2024-09-16T20:50:03.969 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7460000000000001e-06 2024-09-16T20:50:03.969 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.969 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.969 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:03.970 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057304+0000", 2024-09-16T20:50:03.970 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3210000000000004e-06 2024-09-16T20:50:03.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.970 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.970 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:03.970 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057310+0000", 2024-09-16T20:50:03.970 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4360000000000001e-06 2024-09-16T20:50:03.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.970 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.970 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:03.970 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114200+0000", 2024-09-16T20:50:03.970 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056889954 2024-09-16T20:50:03.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.970 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.970 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:03.971 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114243+0000", 2024-09-16T20:50:03.971 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2998999999999999e-05 2024-09-16T20:50:03.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.971 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.971 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:03.971 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114300+0000", 2024-09-16T20:50:03.971 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7765e-05 2024-09-16T20:50:03.971 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:03.971 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:03.971 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:03.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.971 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.971 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8794 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2351104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:03.971 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057327+0000", 2024-09-16T20:50:03.972 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 2.8525379800000001, 2024-09-16T20:50:03.972 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057101130999999999, 2024-09-16T20:50:03.972 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:03.972 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:03.972 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:03.972 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:03.972 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:03.972 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:03.972 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8794 2024-09-16T20:50:03.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.972 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:03.972 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.972 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:03.973 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057327+0000", 2024-09-16T20:50:03.973 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:03.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.973 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.973 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:03.973 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057327+0000", 2024-09-16T20:50:03.973 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:03.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.973 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.973 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:03.973 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057328+0000", 2024-09-16T20:50:03.973 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7399999999999999e-07 2024-09-16T20:50:03.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.974 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:03.974 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057330+0000", 2024-09-16T20:50:03.974 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2199999999999999e-06 2024-09-16T20:50:03.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.974 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:03.974 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057336+0000", 2024-09-16T20:50:03.974 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1240000000000002e-06 2024-09-16T20:50:03.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.974 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:03.974 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057341+0000", 2024-09-16T20:50:03.974 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8470000000000003e-06 2024-09-16T20:50:03.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.975 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.975 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:03.975 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114334+0000", 2024-09-16T20:50:03.975 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056993267 2024-09-16T20:50:03.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.975 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.975 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:03.975 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114376+0000", 2024-09-16T20:50:03.975 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1638999999999999e-05 2024-09-16T20:50:03.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.975 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.975 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:03.976 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114428+0000", 2024-09-16T20:50:03.976 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2660000000000001e-05 2024-09-16T20:50:03.976 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:03.976 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:03.976 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:03.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.976 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.976 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8798 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2297856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:03.976 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:50:03.976 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 2.8525182529999999, 2024-09-16T20:50:03.976 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057277041000000001, 2024-09-16T20:50:03.976 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:03.977 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:03.977 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:03.977 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:03.977 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:03.977 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:03.977 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8798 2024-09-16T20:50:03.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.977 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:03.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.977 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:03.977 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:50:03.977 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:03.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.978 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.978 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:03.981 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:50:03.981 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:03.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.981 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.981 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:03.981 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:50:03.981 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2800000000000003e-07 2024-09-16T20:50:03.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.981 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.981 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:03.981 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057354+0000", 2024-09-16T20:50:03.982 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7780000000000002e-06 2024-09-16T20:50:03.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.982 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.982 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:03.982 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057360+0000", 2024-09-16T20:50:03.982 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0419999999999999e-06 2024-09-16T20:50:03.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.982 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.982 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:03.982 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057365+0000", 2024-09-16T20:50:03.982 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7929999999999997e-06 2024-09-16T20:50:03.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.983 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.983 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:03.983 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114460+0000", 2024-09-16T20:50:03.983 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057094662999999997 2024-09-16T20:50:03.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.983 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.983 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:03.983 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114532+0000", 2024-09-16T20:50:03.983 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2881999999999998e-05 2024-09-16T20:50:03.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.983 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.983 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:03.983 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114624+0000", 2024-09-16T20:50:03.983 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1555e-05 2024-09-16T20:50:03.983 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:03.984 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:03.984 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:03.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.984 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.984 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8800 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2316288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:03.984 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:50:03.984 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 2.8524943139999999, 2024-09-16T20:50:03.984 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.05746946, 2024-09-16T20:50:03.984 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:03.984 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:03.984 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:03.984 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:03.984 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:03.984 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:03.985 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8800 2024-09-16T20:50:03.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.985 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:03.985 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.985 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:03.985 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:50:03.985 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:03.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.985 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.985 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:03.985 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:50:03.985 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:03.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.986 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.986 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:03.986 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:50:03.986 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5199999999999998e-07 2024-09-16T20:50:03.986 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.986 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.986 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:03.986 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057373+0000", 2024-09-16T20:50:03.986 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.79e-06 2024-09-16T20:50:03.986 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.986 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.986 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:03.986 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057379+0000", 2024-09-16T20:50:03.986 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6699999999999999e-06 2024-09-16T20:50:03.986 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.987 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.987 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:03.987 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057386+0000", 2024-09-16T20:50:03.987 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3170000000000004e-06 2024-09-16T20:50:03.987 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.987 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.987 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:03.987 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114655+0000", 2024-09-16T20:50:03.987 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057269420000000001 2024-09-16T20:50:03.987 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.987 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.987 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:03.987 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114734+0000", 2024-09-16T20:50:03.987 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8084000000000007e-05 2024-09-16T20:50:03.987 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.988 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.988 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:03.988 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114840+0000", 2024-09-16T20:50:03.988 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000106827 2024-09-16T20:50:03.988 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:03.988 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:03.988 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:03.988 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.988 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.988 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8801 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2334720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:03.988 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:50:03.988 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 2.8524732830000001, 2024-09-16T20:50:03.989 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057594682000000001, 2024-09-16T20:50:03.989 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:03.989 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:03.989 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:03.989 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:03.989 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:03.989 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:03.989 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8801 2024-09-16T20:50:03.989 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.989 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:03.989 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.989 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:03.989 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:50:03.989 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:03.990 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.990 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.990 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:03.990 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:50:03.990 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:03.990 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.990 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.990 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:03.990 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:50:03.990 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6899999999999998e-07 2024-09-16T20:50:03.990 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.990 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.990 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:03.991 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057394+0000", 2024-09-16T20:50:03.991 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8300000000000001e-06 2024-09-16T20:50:03.991 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.991 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.991 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:03.991 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057400+0000", 2024-09-16T20:50:03.991 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6500000000000001e-06 2024-09-16T20:50:03.991 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.991 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.992 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:03.992 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057404+0000", 2024-09-16T20:50:03.992 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4669999999999998e-06 2024-09-16T20:50:03.992 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.992 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.992 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:03.992 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114874+0000", 2024-09-16T20:50:03.992 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057469602000000002 2024-09-16T20:50:03.992 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.992 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.992 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:03.992 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114933+0000", 2024-09-16T20:50:03.992 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9500000000000003e-05 2024-09-16T20:50:03.993 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.993 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.993 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:03.993 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114987+0000", 2024-09-16T20:50:03.993 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3263999999999997e-05 2024-09-16T20:50:03.993 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:03.993 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:03.993 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:03.993 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.993 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.993 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8802 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2353152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:03.993 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:50:03.993 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 2.8524245979999998, 2024-09-16T20:50:03.993 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057714079000000001, 2024-09-16T20:50:03.994 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:03.994 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:03.994 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:03.994 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:03.994 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:03.994 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:03.994 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8802 2024-09-16T20:50:03.994 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.994 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:03.994 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.994 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:03.994 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:50:03.994 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:03.994 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.995 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.995 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:03.995 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:50:03.995 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:03.995 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.995 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.995 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:03.995 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:50:03.995 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4999999999999998e-07 2024-09-16T20:50:03.995 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.995 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.995 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:03.995 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057446+0000", 2024-09-16T20:50:03.995 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4240000000000003e-06 2024-09-16T20:50:03.996 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.996 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.996 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:03.996 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057452+0000", 2024-09-16T20:50:03.996 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8379999999999996e-06 2024-09-16T20:50:03.996 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.996 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.996 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:03.996 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057460+0000", 2024-09-16T20:50:03.996 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9629999999999998e-06 2024-09-16T20:50:03.996 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.996 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.996 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:03.996 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115015+0000", 2024-09-16T20:50:03.997 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057554389999999997 2024-09-16T20:50:03.997 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.997 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.997 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:03.997 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115082+0000", 2024-09-16T20:50:03.997 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7733999999999999e-05 2024-09-16T20:50:03.997 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.997 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.997 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:03.997 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115155+0000", 2024-09-16T20:50:03.997 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2280000000000006e-05 2024-09-16T20:50:03.997 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:03.997 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:03.997 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:03.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.998 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.998 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8804 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2555904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:03.998 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:50:03.998 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 2.8523982820000002, 2024-09-16T20:50:03.998 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057833862, 2024-09-16T20:50:03.998 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:03.998 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:03.998 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:03.998 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:03.998 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:03.998 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:03.998 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8804 2024-09-16T20:50:03.999 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.999 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:03.999 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.999 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:03.999 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:50:03.999 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:03.999 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:03.999 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:03.999 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:03.999 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:50:03.999 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:03.999 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:04.000 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:04.000 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:04.000 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:50:04.000 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7899999999999999e-07 2024-09-16T20:50:04.000 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:04.000 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:04.000 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:04.000 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057469+0000", 2024-09-16T20:50:04.000 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.001e-06 2024-09-16T20:50:04.000 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:04.000 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:04.000 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:04.000 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057475+0000", 2024-09-16T20:50:04.001 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7629999999999997e-06 2024-09-16T20:50:04.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:04.001 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:04.001 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:04.001 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057480+0000", 2024-09-16T20:50:04.001 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8069999999999999e-06 2024-09-16T20:50:04.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:04.001 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:04.001 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:04.001 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115183+0000", 2024-09-16T20:50:04.001 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057703364 2024-09-16T20:50:04.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:04.001 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:04.001 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:04.002 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115227+0000", 2024-09-16T20:50:04.002 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3903999999999998e-05 2024-09-16T20:50:04.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:04.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:04.002 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:04.002 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115301+0000", 2024-09-16T20:50:04.002 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3543999999999999e-05 2024-09-16T20:50:04.002 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:04.002 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:04.002 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:04.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:04.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:04.002 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8805 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2574336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:04.002 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057487+0000", 2024-09-16T20:50:04.003 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 2.852377954, 2024-09-16T20:50:04.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058021027000000003, 2024-09-16T20:50:04.003 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:04.003 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:04.003 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:04.003 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:04.003 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:04.003 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:04.003 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8805 2024-09-16T20:50:04.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:04.003 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:04.003 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:04.003 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:04.003 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057487+0000", 2024-09-16T20:50:04.004 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:04.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:04.004 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:04.004 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:04.004 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057487+0000", 2024-09-16T20:50:04.004 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:04.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:04.004 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:04.004 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:04.004 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057488+0000", 2024-09-16T20:50:04.004 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5199999999999998e-07 2024-09-16T20:50:04.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:04.004 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:04.004 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:04.005 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057490+0000", 2024-09-16T20:50:04.005 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.939e-06 2024-09-16T20:50:04.005 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:04.005 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:04.005 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:04.005 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057495+0000", 2024-09-16T20:50:04.005 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6439999999999997e-06 2024-09-16T20:50:04.005 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:04.005 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:04.005 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:04.005 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057500+0000", 2024-09-16T20:50:04.005 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8119999999999998e-06 2024-09-16T20:50:04.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:04.006 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:04.006 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:04.006 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115341+0000", 2024-09-16T20:50:04.006 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057841341999999997 2024-09-16T20:50:04.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:04.006 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:04.006 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:04.006 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115422+0000", 2024-09-16T20:50:04.006 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1003999999999999e-05 2024-09-16T20:50:04.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:04.006 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:04.006 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:04.007 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115508+0000", 2024-09-16T20:50:04.007 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5933999999999994e-05 2024-09-16T20:50:04.007 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:04.007 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:04.007 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:04.007 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:04.007 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:04.007 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8807 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2592768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:04.007 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057505+0000", 2024-09-16T20:50:04.007 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 2.8523597440000001, 2024-09-16T20:50:04.007 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058186633000000001, 2024-09-16T20:50:04.007 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:04.007 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:04.007 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:04.007 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:04.007 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:04.008 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:04.008 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8807 2024-09-16T20:50:04.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:04.008 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:04.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:04.008 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:04.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057505+0000", 2024-09-16T20:50:04.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:04.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:04.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:04.008 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:04.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057505+0000", 2024-09-16T20:50:04.009 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:04.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:04.009 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:04.009 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:04.009 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057506+0000", 2024-09-16T20:50:04.009 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.27e-07 2024-09-16T20:50:04.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:04.009 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:04.009 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:04.009 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057508+0000", 2024-09-16T20:50:04.009 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7400000000000001e-06 2024-09-16T20:50:04.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:04.009 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:04.010 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:04.010 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057513+0000", 2024-09-16T20:50:04.010 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.468e-06 2024-09-16T20:50:04.010 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:04.010 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:04.010 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:04.010 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057517+0000", 2024-09-16T20:50:04.010 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4329999999999997e-06 2024-09-16T20:50:04.010 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:04.010 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:04.010 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:04.010 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115538+0000", 2024-09-16T20:50:04.010 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058020355000000003 2024-09-16T20:50:04.010 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:04.010 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:04.011 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:04.011 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115616+0000", 2024-09-16T20:50:04.011 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7998999999999996e-05 2024-09-16T20:50:04.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:04.011 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:04.011 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:04.011 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115692+0000", 2024-09-16T20:50:04.011 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6310999999999995e-05 2024-09-16T20:50:04.011 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:04.011 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:04.011 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:04.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:04.011 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:04.011 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8808 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2611200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:04.012 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:50:04.012 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 2.8523424830000002, 2024-09-16T20:50:04.012 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058354046, 2024-09-16T20:50:04.012 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:04.012 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:04.012 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:04.012 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:04.012 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:04.012 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:04.012 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8808 2024-09-16T20:50:04.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:04.012 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:04.012 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:04.013 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:04.013 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:50:04.013 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:04.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:04.013 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:04.013 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:04.013 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:50:04.013 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:04.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:04.013 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:04.013 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:04.013 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:50:04.013 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2500000000000001e-07 2024-09-16T20:50:04.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:04.014 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:04.014 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:04.014 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057525+0000", 2024-09-16T20:50:04.014 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5549999999999999e-06 2024-09-16T20:50:04.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:04.014 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:04.014 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:04.014 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057530+0000", 2024-09-16T20:50:04.014 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5489999999999997e-06 2024-09-16T20:50:04.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:04.014 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:04.014 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:04.014 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057535+0000", 2024-09-16T20:50:04.014 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5709999999999999e-06 2024-09-16T20:50:04.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:04.014 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:04.015 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:04.015 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115722+0000", 2024-09-16T20:50:04.015 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058187323999999999 2024-09-16T20:50:04.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:04.015 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:04.015 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:04.015 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115799+0000", 2024-09-16T20:50:04.015 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6445000000000004e-05 2024-09-16T20:50:04.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:04.015 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:04.015 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:04.015 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115877+0000", 2024-09-16T20:50:04.015 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8276999999999996e-05 2024-09-16T20:50:04.015 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:04.015 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:04.015 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:04.016 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:04.016 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:04.016 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:04.017 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:04.017+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:04.018 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:50:04.019 INFO:teuthology.orchestra.run.smithi139.stdout:4950 trunc from 0xc42ca00 to 0x32ad600 2024-09-16T20:50:04.019 INFO:teuthology.orchestra.run.smithi139.stdout:4951 trunc from 0x32ad600 to 0x760ba00 2024-09-16T20:50:04.019 INFO:teuthology.orchestra.run.smithi139.stdout:4953 read 0x1c92800 thru 0x1c961e1 (0x39e2 bytes) 2024-09-16T20:50:04.019 INFO:teuthology.orchestra.run.smithi139.stdout:4954 punch from 0x6bd8e00 to 0x6bdcc00, (0x3e00 bytes) 2024-09-16T20:50:04.019 INFO:teuthology.orchestra.run.smithi139.stdout:4956 punch from 0x3f7800 to 0x3ff400, (0x7c00 bytes) 2024-09-16T20:50:04.019 INFO:teuthology.orchestra.run.smithi139.stdout:4958 punch from 0x501c200 to 0x5021000, (0x4e00 bytes) 2024-09-16T20:50:04.019 INFO:teuthology.orchestra.run.smithi139.stdout:4960 read 0x526ea00 thru 0x52738c9 (0x4eca bytes) 2024-09-16T20:50:04.019 INFO:teuthology.orchestra.run.smithi139.stdout:4963 write 0x12d6c00 thru 0x12dcaea (0x5eeb bytes) 2024-09-16T20:50:04.117 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:04.116+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:04.125 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:04.126 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:50:04.126 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:50:04.126 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:04.134 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:50:04.218 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:04.218+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:04.224 INFO:teuthology.orchestra.run.smithi139.stdout:4964 write 0xc896600 thru 0xc8a1a72 (0xb473 bytes) 2024-09-16T20:50:04.230 INFO:teuthology.orchestra.run.smithi139.stdout:4966 write 0x767e200 thru 0x767f529 (0x132a bytes) 2024-09-16T20:50:04.230 INFO:teuthology.orchestra.run.smithi139.stdout:4967 read 0x68a0c00 thru 0x68ae813 (0xdc14 bytes) 2024-09-16T20:50:04.230 INFO:teuthology.orchestra.run.smithi139.stdout:4969 punch from 0x3cc1200 to 0x3cd0200, (0xf000 bytes) 2024-09-16T20:50:04.231 INFO:teuthology.orchestra.run.smithi139.stdout:4974 punch from 0x80bbc00 to 0x80bf200, (0x3600 bytes) 2024-09-16T20:50:04.231 INFO:teuthology.orchestra.run.smithi139.stdout:4976 read 0x8a04200 thru 0x8a07a47 (0x3848 bytes) 2024-09-16T20:50:04.236 INFO:teuthology.orchestra.run.smithi139.stdout:4977 trunc from 0xc8a1c00 to 0x84ee600 2024-09-16T20:50:04.252 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:04.253 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:50:04.253 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:50:04.253 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:50:04.253 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:04.258 INFO:teuthology.orchestra.run.smithi139.stdout:4979 read 0x5a88800 thru 0x5a8c4e6 (0x3ce7 bytes) 2024-09-16T20:50:04.262 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:50:04.264 INFO:teuthology.orchestra.run.smithi139.stdout:4980 write 0x6047e00 thru 0x60528ce (0xaacf bytes) 2024-09-16T20:50:04.289 INFO:teuthology.orchestra.run.smithi139.stdout:4981 write 0xcc6c000 thru 0xcc74d09 (0x8d0a bytes) 2024-09-16T20:50:04.289 INFO:teuthology.orchestra.run.smithi139.stdout:4986 read 0x4999400 thru 0x49a4152 (0xad53 bytes) 2024-09-16T20:50:04.289 INFO:teuthology.orchestra.run.smithi139.stdout:4989 read 0x4390200 thru 0x439f507 (0xf308 bytes) 2024-09-16T20:50:04.289 INFO:teuthology.orchestra.run.smithi139.stdout:4991 punch from 0x2537a00 to 0x2539a00, (0x2000 bytes) 2024-09-16T20:50:04.290 INFO:teuthology.orchestra.run.smithi139.stdout:4992 punch from 0x9430c00 to 0x943ba00, (0xae00 bytes) 2024-09-16T20:50:04.290 INFO:teuthology.orchestra.run.smithi139.stdout:4993 punch from 0x6268400 to 0x626f400, (0x7000 bytes) 2024-09-16T20:50:04.290 INFO:teuthology.orchestra.run.smithi139.stdout:4994 read 0x6ab8400 thru 0x6ac0484 (0x8085 bytes) 2024-09-16T20:50:04.291 INFO:teuthology.orchestra.run.smithi139.stdout:4995 read 0x654a800 thru 0x65550f1 (0xa8f2 bytes) 2024-09-16T20:50:04.292 INFO:teuthology.orchestra.run.smithi139.stdout:4996 write 0xa76be00 thru 0xa778409 (0xc60a bytes) 2024-09-16T20:50:04.292 INFO:teuthology.orchestra.run.smithi139.stdout:4997 read 0x7249e00 thru 0x724ed76 (0x4f77 bytes) 2024-09-16T20:50:04.292 INFO:teuthology.orchestra.run.smithi139.stdout:4998 write 0x90d0e00 thru 0x90dcaf2 (0xbcf3 bytes) 2024-09-16T20:50:04.292 INFO:teuthology.orchestra.run.smithi139.stdout:5002 write 0x5d0ea00 thru 0x5d1b977 (0xcf78 bytes) 2024-09-16T20:50:04.294 INFO:teuthology.orchestra.run.smithi139.stdout:5003 read 0x3d0e000 thru 0x3d10b85 (0x2b86 bytes) 2024-09-16T20:50:04.302 INFO:teuthology.orchestra.run.smithi139.stdout:5004 trunc from 0xcc74e00 to 0x62cd400 2024-09-16T20:50:04.319 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:04.317+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:04.336 INFO:teuthology.orchestra.run.smithi139.stdout:5005 punch from 0x5f2ae00 to 0x5f30600, (0x5800 bytes) 2024-09-16T20:50:04.336 INFO:teuthology.orchestra.run.smithi139.stdout:5006 read 0x3dc7000 thru 0x3dd434e (0xd34f bytes) 2024-09-16T20:50:04.337 INFO:teuthology.orchestra.run.smithi139.stdout:5009 punch from 0x1f1f000 to 0x1f23200, (0x4200 bytes) 2024-09-16T20:50:04.343 INFO:teuthology.orchestra.run.smithi139.stdout:5013 trunc from 0x62cd400 to 0xb9e8200 2024-09-16T20:50:04.376 INFO:teuthology.orchestra.run.smithi139.stdout:5015 write 0xe6a5c00 thru 0xe6a62cf (0x6d0 bytes) 2024-09-16T20:50:04.377 INFO:teuthology.orchestra.run.smithi139.stdout:5017 punch from 0x2034c00 to 0x2043400, (0xe800 bytes) 2024-09-16T20:50:04.377 INFO:teuthology.orchestra.run.smithi139.stdout:5018 write 0xa595200 thru 0xa5a1b26 (0xc927 bytes) 2024-09-16T20:50:04.391 INFO:teuthology.orchestra.run.smithi139.stdout:5019 trunc from 0xe6a6400 to 0x363c800 2024-09-16T20:50:04.420 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:04.418+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:04.425 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:04.425 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:50:04.425 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:50:04.426 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:50:04.426 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.426 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2206 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1046528~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:50:04.426 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:50:04.426 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 102.087623286, 2024-09-16T20:50:04.426 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050147714000000003, 2024-09-16T20:50:04.426 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:04.426 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:04.426 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:04.426 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:04.426 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:04.426 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:04.426 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2206 2024-09-16T20:50:04.427 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.427 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:04.427 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.427 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:04.427 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:50:04.427 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:04.427 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.427 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.427 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:04.427 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:50:04.427 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:04.427 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.427 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.428 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:04.428 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:50:04.428 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8599999999999999e-07 2024-09-16T20:50:04.428 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.428 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.428 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:04.428 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336024+0000", 2024-09-16T20:50:04.428 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.934e-06 2024-09-16T20:50:04.428 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.428 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.428 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:04.428 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336037+0000", 2024-09-16T20:50:04.428 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2496e-05 2024-09-16T20:50:04.429 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.429 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.429 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:04.429 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336042+0000", 2024-09-16T20:50:04.429 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4809999999999997e-06 2024-09-16T20:50:04.429 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.429 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.429 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:04.429 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385980+0000", 2024-09-16T20:50:04.429 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049938376999999999 2024-09-16T20:50:04.429 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.429 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.429 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:04.429 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.386069+0000", 2024-09-16T20:50:04.430 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8368999999999994e-05 2024-09-16T20:50:04.430 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.430 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.430 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:04.430 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.386170+0000", 2024-09-16T20:50:04.430 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000100671 2024-09-16T20:50:04.430 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:04.430 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:04.430 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:04.430 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.430 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.430 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5439 3.a 3:5789f5c3:::rbd_data.110ed9ca540.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 1181696~60416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:04.430 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:50:04.430 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 101.558934706, 2024-09-16T20:50:04.431 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10102045599999999, 2024-09-16T20:50:04.431 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:04.431 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:04.431 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:04.431 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:04.431 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:04.431 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:04.431 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5439 2024-09-16T20:50:04.431 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.431 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:04.431 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.431 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:04.431 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:50:04.432 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:04.432 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.432 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.432 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:04.432 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:50:04.432 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:04.432 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.432 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.432 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:04.432 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864711+0000", 2024-09-16T20:50:04.432 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7899999999999997e-07 2024-09-16T20:50:04.432 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.432 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.432 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:04.433 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864760+0000", 2024-09-16T20:50:04.433 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8803e-05 2024-09-16T20:50:04.433 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.433 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.433 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:04.433 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864769+0000", 2024-09-16T20:50:04.433 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0999999999999993e-06 2024-09-16T20:50:04.433 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.433 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.434 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:04.434 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864774+0000", 2024-09-16T20:50:04.435 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8919999999999999e-06 2024-09-16T20:50:04.435 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.435 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.435 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:04.435 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864815+0000", 2024-09-16T20:50:04.435 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1307000000000003e-05 2024-09-16T20:50:04.435 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.435 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.435 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:04.435 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864893+0000", 2024-09-16T20:50:04.436 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7581000000000001e-05 2024-09-16T20:50:04.436 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.436 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.436 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 1", 2024-09-16T20:50:04.436 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864996+0000", 2024-09-16T20:50:04.436 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102725 2024-09-16T20:50:04.436 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.436 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.436 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:04.436 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965442+0000", 2024-09-16T20:50:04.436 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100446012 2024-09-16T20:50:04.436 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.437 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.437 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:04.437 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965618+0000", 2024-09-16T20:50:04.437 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00017653100000000001 2024-09-16T20:50:04.437 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.437 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.437 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:04.437 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965682+0000", 2024-09-16T20:50:04.437 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3905999999999994e-05 2024-09-16T20:50:04.437 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.437 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.437 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:04.437 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965731+0000", 2024-09-16T20:50:04.438 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8819999999999997e-05 2024-09-16T20:50:04.438 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:04.438 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:04.438 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:04.438 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.438 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.438 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4600.0:5439 3.a e37/23 ondisk, result = 0)", 2024-09-16T20:50:04.438 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:50:04.438 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 101.557938558, 2024-09-16T20:50:04.438 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099742634999999996, 2024-09-16T20:50:04.438 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:04.438 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:04.438 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:04.439 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:04.439 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.439 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:04.439 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:50:04.439 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:04.439 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.439 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.439 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:04.439 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:50:04.439 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:04.439 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.439 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.439 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:04.439 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:50:04.440 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0900000000000001e-07 2024-09-16T20:50:04.440 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.440 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.440 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:04.440 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865711+0000", 2024-09-16T20:50:04.440 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3739999999999998e-06 2024-09-16T20:50:04.440 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.440 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.440 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:04.440 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965343+0000", 2024-09-16T20:50:04.440 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099632526999999999 2024-09-16T20:50:04.440 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.440 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.441 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:04.441 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965357+0000", 2024-09-16T20:50:04.441 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3713999999999999e-05 2024-09-16T20:50:04.441 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.441 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.441 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:04.441 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965426+0000", 2024-09-16T20:50:04.441 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8745999999999995e-05 2024-09-16T20:50:04.441 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.441 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.441 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:04.441 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965434+0000", 2024-09-16T20:50:04.441 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6910000000000003e-06 2024-09-16T20:50:04.442 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.442 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.442 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:04.442 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965449+0000", 2024-09-16T20:50:04.442 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4874e-05 2024-09-16T20:50:04.442 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:04.442 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:04.442 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:04.442 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.442 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.442 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4882.0:2508 3.12 3:4992771f:::rbd_object_map.110e7c8851c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T20:50:04.442 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:50:04.442 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 77.942135010000001, 2024-09-16T20:50:04.442 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.125007011, 2024-09-16T20:50:04.443 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:04.443 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:04.443 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:04.443 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:04.443 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4882", 2024-09-16T20:50:04.443 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2436923223", 2024-09-16T20:50:04.443 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2508 2024-09-16T20:50:04.443 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.443 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:04.443 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.443 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:04.443 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:50:04.443 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:04.444 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.444 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.444 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:04.444 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:50:04.444 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:04.444 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.444 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.444 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:04.444 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481511+0000", 2024-09-16T20:50:04.444 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0699999999999996e-07 2024-09-16T20:50:04.444 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.444 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.444 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:04.445 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481515+0000", 2024-09-16T20:50:04.445 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0609999999999997e-06 2024-09-16T20:50:04.445 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.445 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.445 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:04.445 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481523+0000", 2024-09-16T20:50:04.445 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1310000000000006e-06 2024-09-16T20:50:04.445 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.445 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.445 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:04.445 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481528+0000", 2024-09-16T20:50:04.445 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4900000000000002e-06 2024-09-16T20:50:04.445 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.445 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.446 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:04.446 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481565+0000", 2024-09-16T20:50:04.446 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7066000000000001e-05 2024-09-16T20:50:04.446 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.446 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.446 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:04.446 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481628+0000", 2024-09-16T20:50:04.446 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3118000000000001e-05 2024-09-16T20:50:04.446 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.446 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.446 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 2", 2024-09-16T20:50:04.446 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481822+0000", 2024-09-16T20:50:04.446 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019446200000000001 2024-09-16T20:50:04.447 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.447 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.447 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:04.447 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605988+0000", 2024-09-16T20:50:04.447 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.124165888 2024-09-16T20:50:04.447 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.447 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.447 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:04.447 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606427+0000", 2024-09-16T20:50:04.447 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00043854899999999999 2024-09-16T20:50:04.447 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.447 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.447 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:04.448 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606476+0000", 2024-09-16T20:50:04.448 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8971999999999998e-05 2024-09-16T20:50:04.448 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.448 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.448 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:04.448 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606517+0000", 2024-09-16T20:50:04.448 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1467000000000001e-05 2024-09-16T20:50:04.448 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:04.448 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:04.448 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:04.448 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.448 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.448 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4882.0:2508 3.12 e57/55 ondisk, result = 0)", 2024-09-16T20:50:04.449 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:50:04.449 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 77.941146509999996, 2024-09-16T20:50:04.449 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12349711400000001, 2024-09-16T20:50:04.449 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:04.449 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:04.449 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:04.449 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:04.449 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.449 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:04.449 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:50:04.449 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:04.449 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.449 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.449 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:04.450 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:50:04.450 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:04.450 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.450 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.450 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:04.450 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:50:04.450 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.06e-07 2024-09-16T20:50:04.450 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.450 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.450 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:04.450 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482502+0000", 2024-09-16T20:50:04.450 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.01e-06 2024-09-16T20:50:04.450 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.451 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.451 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:04.451 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605772+0000", 2024-09-16T20:50:04.451 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.123269687 2024-09-16T20:50:04.451 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.451 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.451 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:04.451 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605791+0000", 2024-09-16T20:50:04.451 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8912000000000001e-05 2024-09-16T20:50:04.451 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.451 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.451 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:04.451 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605974+0000", 2024-09-16T20:50:04.451 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000182836 2024-09-16T20:50:04.452 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.452 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.452 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:04.452 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605982+0000", 2024-09-16T20:50:04.452 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1699999999999997e-06 2024-09-16T20:50:04.452 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.452 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.452 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:04.452 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605996+0000", 2024-09-16T20:50:04.452 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3893000000000001e-05 2024-09-16T20:50:04.452 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:04.452 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:04.452 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:04.453 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.453 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.453 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4882.0:2512 3.f 3:f4d895b7:::rbd_data.110e7c8851c.0000000000000000:head [read 16384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T20:50:04.453 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:50:04.453 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 77.940709479999995, 2024-09-16T20:50:04.453 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12318483, 2024-09-16T20:50:04.453 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:04.453 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:04.453 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:04.453 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:04.453 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4882", 2024-09-16T20:50:04.453 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2436923223", 2024-09-16T20:50:04.453 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2512 2024-09-16T20:50:04.453 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.454 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:04.454 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.454 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:04.454 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:50:04.454 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:04.454 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.454 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.454 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:04.454 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:50:04.454 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:04.454 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.454 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.454 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:04.455 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:50:04.455 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3499999999999995e-07 2024-09-16T20:50:04.455 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.455 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.455 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:04.455 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482943+0000", 2024-09-16T20:50:04.455 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3450000000000002e-06 2024-09-16T20:50:04.455 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.455 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.455 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:04.455 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605772+0000", 2024-09-16T20:50:04.455 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.122829796 2024-09-16T20:50:04.455 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.456 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.456 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:04.456 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605792+0000", 2024-09-16T20:50:04.456 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9916999999999999e-05 2024-09-16T20:50:04.456 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.456 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.456 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:04.456 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605917+0000", 2024-09-16T20:50:04.456 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00012500799999999999 2024-09-16T20:50:04.456 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.456 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.456 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:04.456 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606004+0000", 2024-09-16T20:50:04.457 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6242999999999994e-05 2024-09-16T20:50:04.457 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.457 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.457 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:04.457 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606120+0000", 2024-09-16T20:50:04.457 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000116786 2024-09-16T20:50:04.457 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:04.457 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:04.457 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:04.457 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.457 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.457 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:11702 3.15 3:acf33c8b:::rbd_data.110e747f0779.000000000000001a:head [set-alloc-hint object_size 262144 write_size 262144,write 143360~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:50:04.457 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:50:04.458 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 60.285717058000003, 2024-09-16T20:50:04.458 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.084801043000000006, 2024-09-16T20:50:04.458 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:04.458 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:04.458 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:04.458 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:04.458 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:50:04.458 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:50:04.458 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 11702 2024-09-16T20:50:04.458 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.458 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:04.458 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.458 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:04.458 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:50:04.459 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:04.459 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.459 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.459 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:04.459 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:50:04.459 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:04.459 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.459 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.459 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:04.459 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137929+0000", 2024-09-16T20:50:04.459 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8400000000000002e-07 2024-09-16T20:50:04.459 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.459 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.460 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:04.460 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137937+0000", 2024-09-16T20:50:04.460 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0260000000000007e-06 2024-09-16T20:50:04.460 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.460 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.460 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:04.460 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137952+0000", 2024-09-16T20:50:04.460 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4659e-05 2024-09-16T20:50:04.460 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.460 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.460 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:04.460 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137960+0000", 2024-09-16T20:50:04.460 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0169999999999996e-06 2024-09-16T20:50:04.461 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.461 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.461 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:04.461 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138010+0000", 2024-09-16T20:50:04.461 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0021000000000001e-05 2024-09-16T20:50:04.461 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.461 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.461 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:04.461 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138114+0000", 2024-09-16T20:50:04.461 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000103797 2024-09-16T20:50:04.461 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.461 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.461 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:50:04.461 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138233+0000", 2024-09-16T20:50:04.462 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000118971 2024-09-16T20:50:04.462 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.462 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.462 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:04.462 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138953+0000", 2024-09-16T20:50:04.462 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00072026000000000002 2024-09-16T20:50:04.462 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.462 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.462 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:04.462 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222609+0000", 2024-09-16T20:50:04.462 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.083656650999999999 2024-09-16T20:50:04.462 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.462 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.463 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:04.463 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222684+0000", 2024-09-16T20:50:04.463 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4369999999999994e-05 2024-09-16T20:50:04.463 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.463 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.463 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:04.463 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222729+0000", 2024-09-16T20:50:04.463 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5287000000000002e-05 2024-09-16T20:50:04.463 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:04.463 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:04.463 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:04.463 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.463 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.464 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:11704 3.9 3:91220b75:::rbd_data.110e747f0779.000000000000001c:head [set-alloc-hint object_size 262144 write_size 262144,write 139264~6656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:50:04.464 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:50:04.464 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 60.285479057000003, 2024-09-16T20:50:04.464 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.084531133999999994, 2024-09-16T20:50:04.464 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:04.464 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:04.464 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:04.464 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:04.464 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:50:04.464 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:50:04.464 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 11704 2024-09-16T20:50:04.464 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.464 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:04.465 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.465 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:04.465 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:50:04.465 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:04.465 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.465 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.465 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:04.465 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:50:04.465 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:04.465 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.465 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.465 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:04.465 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138167+0000", 2024-09-16T20:50:04.466 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.33e-07 2024-09-16T20:50:04.466 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.466 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.466 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:04.466 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138179+0000", 2024-09-16T20:50:04.466 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2055999999999999e-05 2024-09-16T20:50:04.466 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.466 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.466 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:04.466 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138193+0000", 2024-09-16T20:50:04.466 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3961000000000001e-05 2024-09-16T20:50:04.466 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.466 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.466 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:04.467 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138200+0000", 2024-09-16T20:50:04.467 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4429999999999997e-06 2024-09-16T20:50:04.467 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.467 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.467 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:04.467 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138258+0000", 2024-09-16T20:50:04.467 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7225999999999997e-05 2024-09-16T20:50:04.467 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.467 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.467 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:04.467 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138364+0000", 2024-09-16T20:50:04.467 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000106013 2024-09-16T20:50:04.467 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.467 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.468 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:50:04.468 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138494+0000", 2024-09-16T20:50:04.468 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013059599999999999 2024-09-16T20:50:04.468 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.468 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.468 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:04.468 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.139220+0000", 2024-09-16T20:50:04.468 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00072539800000000004 2024-09-16T20:50:04.468 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.468 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.468 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:04.468 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222608+0000", 2024-09-16T20:50:04.468 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.083388620999999996 2024-09-16T20:50:04.469 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.469 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.469 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:04.469 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222651+0000", 2024-09-16T20:50:04.469 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2228000000000003e-05 2024-09-16T20:50:04.469 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.469 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.469 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:04.469 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222697+0000", 2024-09-16T20:50:04.469 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6659000000000002e-05 2024-09-16T20:50:04.469 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:04.469 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:04.469 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:04.470 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.470 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.470 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5138.0:9358 3.c 3.c25699cc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T20:50:04.470 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:50:04.470 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 37.573331752000001, 2024-09-16T20:50:04.470 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0018657580000001, 2024-09-16T20:50:04.470 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:04.470 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:04.470 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:50:04.470 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:04.470 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5138", 2024-09-16T20:50:04.470 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:50:04.470 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 9358 2024-09-16T20:50:04.471 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.471 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:04.471 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.471 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:04.471 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:50:04.471 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:04.471 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.471 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.471 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:04.471 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:50:04.471 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:04.471 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.471 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.471 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:04.472 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850314+0000", 2024-09-16T20:50:04.472 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4099999999999999e-07 2024-09-16T20:50:04.472 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.472 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.472 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:04.472 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850316+0000", 2024-09-16T20:50:04.472 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2469999999999998e-06 2024-09-16T20:50:04.472 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.472 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.472 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:04.472 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850322+0000", 2024-09-16T20:50:04.472 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3470000000000004e-06 2024-09-16T20:50:04.472 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.472 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.473 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:04.473 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850327+0000", 2024-09-16T20:50:04.473 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9450000000000001e-06 2024-09-16T20:50:04.473 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.473 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.473 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:04.473 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850376+0000", 2024-09-16T20:50:04.473 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8257000000000002e-05 2024-09-16T20:50:04.473 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.473 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.473 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:04.473 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850385+0000", 2024-09-16T20:50:04.473 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9800000000000004e-06 2024-09-16T20:50:04.474 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.474 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.474 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:04.474 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852167+0000", 2024-09-16T20:50:04.474 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0017825659999999 2024-09-16T20:50:04.474 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.474 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.474 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:04.474 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852179+0000", 2024-09-16T20:50:04.474 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1975e-05 2024-09-16T20:50:04.474 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:04.474 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:04.474 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:04.474 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.475 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.475 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5138.0:9382 3.c 3.c25699cc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T20:50:04.475 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:50:04.475 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 37.564984736, 2024-09-16T20:50:04.475 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.99358404700000003, 2024-09-16T20:50:04.475 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:04.475 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:04.475 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:50:04.475 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:04.475 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5138", 2024-09-16T20:50:04.475 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:50:04.475 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 9382 2024-09-16T20:50:04.475 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.476 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:04.476 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.476 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:04.476 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:50:04.476 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:04.476 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.476 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.476 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:04.476 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:50:04.476 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:04.476 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.476 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.476 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:04.477 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858661+0000", 2024-09-16T20:50:04.477 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0200000000000002e-07 2024-09-16T20:50:04.477 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.477 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.477 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:04.477 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858664+0000", 2024-09-16T20:50:04.477 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.038e-06 2024-09-16T20:50:04.477 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.477 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.477 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:04.477 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858671+0000", 2024-09-16T20:50:04.477 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0110000000000003e-06 2024-09-16T20:50:04.477 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.477 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.478 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:04.555 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858677+0000", 2024-09-16T20:50:04.555 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0959999999999997e-06 2024-09-16T20:50:04.555 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.555 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.555 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:04.555 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858732+0000", 2024-09-16T20:50:04.555 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4543000000000003e-05 2024-09-16T20:50:04.555 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.555 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.556 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:04.556 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858739+0000", 2024-09-16T20:50:04.556 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5429999999999996e-06 2024-09-16T20:50:04.556 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.556 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.556 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:04.556 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852236+0000", 2024-09-16T20:50:04.556 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.993496453 2024-09-16T20:50:04.556 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.556 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.556 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:04.556 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852244+0000", 2024-09-16T20:50:04.556 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8610000000000004e-06 2024-09-16T20:50:04.557 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:04.557 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:04.557 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:04.557 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.557 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.557 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:361 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 372736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:50:04.557 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246714+0000", 2024-09-16T20:50:04.557 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 28.176930588000001, 2024-09-16T20:50:04.557 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050338229999999998, 2024-09-16T20:50:04.557 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:04.557 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:04.557 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:04.557 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:04.557 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:50:04.558 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:50:04.558 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 361 2024-09-16T20:50:04.558 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.558 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:04.558 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.558 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:04.558 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246714+0000", 2024-09-16T20:50:04.558 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:04.558 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.558 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.558 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:04.558 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246714+0000", 2024-09-16T20:50:04.558 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:04.559 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.559 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.559 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:04.559 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246715+0000", 2024-09-16T20:50:04.559 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1899999999999999e-07 2024-09-16T20:50:04.559 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.559 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.559 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:04.559 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246716+0000", 2024-09-16T20:50:04.559 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1659999999999999e-06 2024-09-16T20:50:04.559 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.559 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.559 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:04.559 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246721+0000", 2024-09-16T20:50:04.560 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9830000000000004e-06 2024-09-16T20:50:04.560 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.560 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.560 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:04.560 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246723+0000", 2024-09-16T20:50:04.560 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5679999999999998e-06 2024-09-16T20:50:04.560 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.560 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.560 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:04.560 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.296875+0000", 2024-09-16T20:50:04.560 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050151683000000002 2024-09-16T20:50:04.560 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.560 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.561 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:04.561 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.296959+0000", 2024-09-16T20:50:04.561 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4079999999999995e-05 2024-09-16T20:50:04.561 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.561 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.561 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:04.561 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297053+0000", 2024-09-16T20:50:04.561 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.3530999999999996e-05 2024-09-16T20:50:04.561 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:04.561 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:04.561 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:04.561 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.561 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.561 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:362 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 385024~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:50:04.562 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246727+0000", 2024-09-16T20:50:04.562 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 28.176918479000001, 2024-09-16T20:50:04.562 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050546528, 2024-09-16T20:50:04.562 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:04.562 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:04.562 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:04.562 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:04.562 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:50:04.562 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:50:04.562 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 362 2024-09-16T20:50:04.562 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.562 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:04.562 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.563 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:04.563 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246727+0000", 2024-09-16T20:50:04.563 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:04.563 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.563 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.563 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:04.563 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246727+0000", 2024-09-16T20:50:04.563 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:04.563 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.563 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.563 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:04.563 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246727+0000", 2024-09-16T20:50:04.563 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.35e-07 2024-09-16T20:50:04.563 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.564 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.564 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:04.564 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246728+0000", 2024-09-16T20:50:04.564 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.175e-06 2024-09-16T20:50:04.564 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.564 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.564 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:04.564 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246734+0000", 2024-09-16T20:50:04.564 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2160000000000003e-06 2024-09-16T20:50:04.564 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.564 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.564 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:04.564 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246737+0000", 2024-09-16T20:50:04.564 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9450000000000002e-06 2024-09-16T20:50:04.565 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.565 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.565 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:04.565 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297103+0000", 2024-09-16T20:50:04.565 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050365518999999997 2024-09-16T20:50:04.565 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.565 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.565 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:04.565 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297183+0000", 2024-09-16T20:50:04.565 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0208000000000002e-05 2024-09-16T20:50:04.565 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.565 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.565 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:04.566 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297273+0000", 2024-09-16T20:50:04.566 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0229999999999995e-05 2024-09-16T20:50:04.566 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:04.566 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:04.566 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:04.566 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.566 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.566 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:368 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 327680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:50:04.566 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246746+0000", 2024-09-16T20:50:04.566 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 28.176899199000001, 2024-09-16T20:50:04.566 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050730747999999999, 2024-09-16T20:50:04.566 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:04.566 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:04.566 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:04.566 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:04.567 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:50:04.567 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:50:04.567 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 368 2024-09-16T20:50:04.567 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.567 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:04.567 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.567 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:04.567 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246746+0000", 2024-09-16T20:50:04.567 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:04.567 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.567 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.567 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:04.567 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246746+0000", 2024-09-16T20:50:04.567 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:04.568 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.568 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.568 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:04.568 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246746+0000", 2024-09-16T20:50:04.568 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8099999999999999e-07 2024-09-16T20:50:04.568 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.568 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.568 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:04.568 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246754+0000", 2024-09-16T20:50:04.568 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4440000000000002e-06 2024-09-16T20:50:04.568 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.568 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.568 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:04.568 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246758+0000", 2024-09-16T20:50:04.569 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3900000000000003e-06 2024-09-16T20:50:04.569 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.569 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.569 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:04.569 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246761+0000", 2024-09-16T20:50:04.569 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.101e-06 2024-09-16T20:50:04.569 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.569 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.569 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:04.569 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297310+0000", 2024-09-16T20:50:04.569 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050549166 2024-09-16T20:50:04.569 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.569 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.570 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:04.570 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297387+0000", 2024-09-16T20:50:04.570 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6658999999999999e-05 2024-09-16T20:50:04.570 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.570 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.570 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:04.570 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297477+0000", 2024-09-16T20:50:04.570 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9707e-05 2024-09-16T20:50:04.570 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:04.570 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:04.570 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:04.570 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.570 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.570 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:369 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 339968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:50:04.571 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246766+0000", 2024-09-16T20:50:04.571 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 28.176879474, 2024-09-16T20:50:04.571 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050911291999999997, 2024-09-16T20:50:04.571 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:04.571 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:04.571 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:04.571 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:04.571 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:50:04.571 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:50:04.571 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 369 2024-09-16T20:50:04.571 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.571 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:04.571 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.571 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:04.572 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246766+0000", 2024-09-16T20:50:04.572 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:04.572 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.572 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.572 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:04.572 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246766+0000", 2024-09-16T20:50:04.572 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:04.572 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.572 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.572 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:04.572 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246766+0000", 2024-09-16T20:50:04.572 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3200000000000001e-07 2024-09-16T20:50:04.572 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.572 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.573 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:04.573 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246767+0000", 2024-09-16T20:50:04.573 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.252e-06 2024-09-16T20:50:04.573 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.573 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.573 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:04.573 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246770+0000", 2024-09-16T20:50:04.573 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1049999999999999e-06 2024-09-16T20:50:04.573 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.573 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.573 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:04.573 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246773+0000", 2024-09-16T20:50:04.573 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.396e-06 2024-09-16T20:50:04.573 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.574 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.574 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:04.574 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297513+0000", 2024-09-16T20:50:04.574 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050740800000000003 2024-09-16T20:50:04.574 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.574 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.574 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:04.574 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297589+0000", 2024-09-16T20:50:04.574 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5318e-05 2024-09-16T20:50:04.574 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.574 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.574 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:04.574 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297677+0000", 2024-09-16T20:50:04.574 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8189000000000006e-05 2024-09-16T20:50:04.574 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:04.575 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:04.575 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:04.575 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.575 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.575 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:370 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 348160~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:50:04.575 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246776+0000", 2024-09-16T20:50:04.575 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 28.176869458999999, 2024-09-16T20:50:04.575 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051096643999999997, 2024-09-16T20:50:04.575 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:04.575 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:04.575 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:04.575 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:04.575 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:50:04.575 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:50:04.576 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 370 2024-09-16T20:50:04.576 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.576 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:04.576 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.576 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:04.576 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246776+0000", 2024-09-16T20:50:04.576 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:04.576 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.576 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.576 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:04.576 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246776+0000", 2024-09-16T20:50:04.576 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:04.576 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.577 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.577 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:04.577 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246776+0000", 2024-09-16T20:50:04.577 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2499999999999999e-07 2024-09-16T20:50:04.577 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.577 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.577 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:04.577 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246777+0000", 2024-09-16T20:50:04.577 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1510000000000001e-06 2024-09-16T20:50:04.577 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.577 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.577 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:04.577 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246780+0000", 2024-09-16T20:50:04.577 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9610000000000001e-06 2024-09-16T20:50:04.578 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.578 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.578 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:04.578 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246783+0000", 2024-09-16T20:50:04.578 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6129999999999998e-06 2024-09-16T20:50:04.578 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.578 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.578 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:04.578 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297713+0000", 2024-09-16T20:50:04.578 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050929950000000002 2024-09-16T20:50:04.578 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.578 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.578 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:04.578 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297786+0000", 2024-09-16T20:50:04.578 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3096000000000006e-05 2024-09-16T20:50:04.579 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.579 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.579 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:04.579 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297872+0000", 2024-09-16T20:50:04.579 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6648000000000001e-05 2024-09-16T20:50:04.579 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:04.579 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:04.579 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:04.579 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.579 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.579 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:371 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 380928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:50:04.579 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246791+0000", 2024-09-16T20:50:04.579 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 28.176853822999998, 2024-09-16T20:50:04.579 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051287209, 2024-09-16T20:50:04.580 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:04.580 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:04.580 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:04.580 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:04.580 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:50:04.580 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:50:04.580 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 371 2024-09-16T20:50:04.580 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.580 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:04.580 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.580 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:04.580 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246791+0000", 2024-09-16T20:50:04.580 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:04.580 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.581 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.581 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:04.581 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246791+0000", 2024-09-16T20:50:04.581 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:04.581 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.581 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.581 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:04.581 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246792+0000", 2024-09-16T20:50:04.581 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8200000000000001e-07 2024-09-16T20:50:04.581 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.581 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.581 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:04.581 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246793+0000", 2024-09-16T20:50:04.581 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.28e-06 2024-09-16T20:50:04.582 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.582 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.582 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:04.582 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246796+0000", 2024-09-16T20:50:04.582 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.597e-06 2024-09-16T20:50:04.582 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.582 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.582 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:04.582 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246799+0000", 2024-09-16T20:50:04.582 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7089999999999998e-06 2024-09-16T20:50:04.582 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.582 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.582 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:04.582 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297910+0000", 2024-09-16T20:50:04.582 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051110598 2024-09-16T20:50:04.583 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.583 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.583 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:04.583 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297986+0000", 2024-09-16T20:50:04.583 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6711000000000005e-05 2024-09-16T20:50:04.583 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.583 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.583 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:04.583 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.298078+0000", 2024-09-16T20:50:04.583 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2032000000000002e-05 2024-09-16T20:50:04.583 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:04.583 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:04.583 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:04.583 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.584 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.584 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:373 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 376832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:50:04.584 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246904+0000", 2024-09-16T20:50:04.584 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 28.176741401000001, 2024-09-16T20:50:04.584 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051310617000000003, 2024-09-16T20:50:04.584 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:04.584 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:04.584 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:04.584 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:04.584 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:50:04.584 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:50:04.584 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 373 2024-09-16T20:50:04.584 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.584 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:04.585 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.585 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:04.585 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246904+0000", 2024-09-16T20:50:04.585 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:04.585 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.585 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.585 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:04.585 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246904+0000", 2024-09-16T20:50:04.585 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:04.585 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.585 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.585 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:04.585 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246904+0000", 2024-09-16T20:50:04.586 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8000000000000001e-07 2024-09-16T20:50:04.586 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.586 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.586 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:04.586 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246908+0000", 2024-09-16T20:50:04.586 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4039999999999997e-06 2024-09-16T20:50:04.586 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.586 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.586 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:04.586 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246913+0000", 2024-09-16T20:50:04.586 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0030000000000002e-06 2024-09-16T20:50:04.586 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.586 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.586 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:04.586 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246917+0000", 2024-09-16T20:50:04.587 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.303e-06 2024-09-16T20:50:04.587 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.587 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.587 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:04.587 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.298114+0000", 2024-09-16T20:50:04.587 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051197724 2024-09-16T20:50:04.587 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.587 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.587 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:04.587 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.298159+0000", 2024-09-16T20:50:04.587 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4882999999999997e-05 2024-09-16T20:50:04.587 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.587 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.587 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:04.588 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.298214+0000", 2024-09-16T20:50:04.588 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4920000000000003e-05 2024-09-16T20:50:04.588 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:04.588 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:04.588 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:04.588 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.588 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.588 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:374 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 389120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:50:04.588 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246927+0000", 2024-09-16T20:50:04.588 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 28.176718352000002, 2024-09-16T20:50:04.588 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051498164999999999, 2024-09-16T20:50:04.588 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:04.588 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:04.588 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:04.589 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:04.589 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:50:04.589 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:50:04.589 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 374 2024-09-16T20:50:04.589 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.589 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:04.589 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.589 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:04.589 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246927+0000", 2024-09-16T20:50:04.589 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:04.589 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.594 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.594 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:04.594 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246927+0000", 2024-09-16T20:50:04.594 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:04.594 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.594 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.594 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:04.594 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246927+0000", 2024-09-16T20:50:04.594 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7e-07 2024-09-16T20:50:04.594 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.594 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.594 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:04.595 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246929+0000", 2024-09-16T20:50:04.595 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.607e-06 2024-09-16T20:50:04.595 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.595 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.595 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:04.595 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246932+0000", 2024-09-16T20:50:04.595 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4539999999999999e-06 2024-09-16T20:50:04.595 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.595 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.595 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:04.595 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246935+0000", 2024-09-16T20:50:04.595 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6570000000000001e-06 2024-09-16T20:50:04.595 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.595 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.596 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:04.596 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.298242+0000", 2024-09-16T20:50:04.596 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051306887000000002 2024-09-16T20:50:04.596 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.596 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.596 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:04.596 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.298327+0000", 2024-09-16T20:50:04.596 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5068999999999995e-05 2024-09-16T20:50:04.596 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.596 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.596 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:04.596 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.298425+0000", 2024-09-16T20:50:04.596 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8021000000000002e-05 2024-09-16T20:50:04.597 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:04.597 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:04.597 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:04.597 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.597 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.597 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5763.0:4895 3.a 3:5384606c:::rbd_data.110e1b19a241.0000000000000018:head [read 0~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-16T20:50:04.597 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:45.598212+0000", 2024-09-16T20:50:04.597 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 18.825432695, 2024-09-16T20:50:04.597 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0375864829999999, 2024-09-16T20:50:04.597 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:04.597 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:04.597 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:04.597 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:04.597 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5763", 2024-09-16T20:50:04.597 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/623897567", 2024-09-16T20:50:04.598 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4895 2024-09-16T20:50:04.598 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.598 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:04.598 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.598 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:04.598 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598212+0000", 2024-09-16T20:50:04.598 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:04.598 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.598 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.598 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:04.598 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598212+0000", 2024-09-16T20:50:04.598 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:04.598 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.598 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.599 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:04.599 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598214+0000", 2024-09-16T20:50:04.599 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0090000000000002e-06 2024-09-16T20:50:04.599 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.599 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.599 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:04.599 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598222+0000", 2024-09-16T20:50:04.599 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6699999999999994e-06 2024-09-16T20:50:04.599 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.599 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.599 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:04.599 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598241+0000", 2024-09-16T20:50:04.599 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8896999999999999e-05 2024-09-16T20:50:04.599 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.600 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.600 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:04.600 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598250+0000", 2024-09-16T20:50:04.600 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6649999999999992e-06 2024-09-16T20:50:04.600 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.600 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.600 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:04.600 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598325+0000", 2024-09-16T20:50:04.600 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5189000000000001e-05 2024-09-16T20:50:04.600 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.600 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.600 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op must wait for map", 2024-09-16T20:50:04.600 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598333+0000", 2024-09-16T20:50:04.601 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.3170000000000002e-06 2024-09-16T20:50:04.601 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.601 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.601 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:04.601 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601355+0000", 2024-09-16T20:50:04.601 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0030215400000000001 2024-09-16T20:50:04.601 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.601 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.601 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:04.601 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601365+0000", 2024-09-16T20:50:04.601 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0215e-05 2024-09-16T20:50:04.601 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.601 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.601 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:04.602 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635506+0000", 2024-09-16T20:50:04.602 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0341408240000001 2024-09-16T20:50:04.602 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.602 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.602 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:04.602 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635655+0000", 2024-09-16T20:50:04.602 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000149412 2024-09-16T20:50:04.602 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.602 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.602 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:04.602 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635799+0000", 2024-09-16T20:50:04.602 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000143745 2024-09-16T20:50:04.602 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:04.602 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:04.603 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:04.603 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.603 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.603 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5763.0:4900 3.a 3:5384606c:::rbd_data.110e1b19a241.0000000000000018:head [read 8192~24576] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-16T20:50:04.603 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:50:04.603 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 18.825374219, 2024-09-16T20:50:04.603 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0379340800000001, 2024-09-16T20:50:04.603 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:04.603 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:04.603 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:04.603 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:04.603 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5763", 2024-09-16T20:50:04.603 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/623897567", 2024-09-16T20:50:04.603 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4900 2024-09-16T20:50:04.604 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.604 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:04.604 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.604 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:04.604 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:50:04.604 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:04.604 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.604 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.604 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:04.604 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:50:04.604 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:04.604 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.604 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.604 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:04.605 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:50:04.605 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4300000000000003e-07 2024-09-16T20:50:04.605 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.605 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.605 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:04.605 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598274+0000", 2024-09-16T20:50:04.605 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.976e-06 2024-09-16T20:50:04.605 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.605 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.605 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:04.605 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598282+0000", 2024-09-16T20:50:04.605 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0029999999999994e-06 2024-09-16T20:50:04.605 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.605 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.606 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:04.606 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598289+0000", 2024-09-16T20:50:04.606 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2389999999999999e-06 2024-09-16T20:50:04.606 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.606 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.606 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:04.606 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598430+0000", 2024-09-16T20:50:04.606 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014119600000000001 2024-09-16T20:50:04.606 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.606 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.606 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting_for_map not empty", 2024-09-16T20:50:04.606 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598435+0000", 2024-09-16T20:50:04.606 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3489999999999999e-06 2024-09-16T20:50:04.606 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.606 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.607 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:04.607 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601429+0000", 2024-09-16T20:50:04.607 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0029942720000000001 2024-09-16T20:50:04.607 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.607 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.607 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:04.607 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601438+0000", 2024-09-16T20:50:04.607 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4759999999999998e-06 2024-09-16T20:50:04.607 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.607 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.607 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:04.607 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635987+0000", 2024-09-16T20:50:04.607 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0345495010000001 2024-09-16T20:50:04.607 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.608 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.608 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:04.608 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.636056+0000", 2024-09-16T20:50:04.608 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8610999999999998e-05 2024-09-16T20:50:04.608 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:04.608 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:04.608 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:04.608 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.636205+0000", 2024-09-16T20:50:04.608 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014891399999999999 2024-09-16T20:50:04.608 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:04.608 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:04.608 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:04.608 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:04.608 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:04.609 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:04.610 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:04.608+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:04.611 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:50:04.623 INFO:teuthology.orchestra.run.smithi139.stdout:5020 read 0xcdda00 thru 0xce3615 (0x5c16 bytes) 2024-09-16T20:50:04.624 INFO:teuthology.orchestra.run.smithi139.stdout:5022 punch from 0x2584c00 to 0x2589c00, (0x5000 bytes) 2024-09-16T20:50:04.644 INFO:teuthology.orchestra.run.smithi139.stdout:5023 write 0x6cf8400 thru 0x6cf91bf (0xdc0 bytes) 2024-09-16T20:50:04.658 INFO:teuthology.orchestra.run.smithi029.stderr:nodeep-scrub is unset 2024-09-16T20:50:04.672 INFO:teuthology.orchestra.run.smithi139.stdout:5024 write 0xd838600 thru 0xd8474a2 (0xeea3 bytes) 2024-09-16T20:50:04.673 INFO:teuthology.orchestra.run.smithi139.stdout:5025 write 0x6bc3a00 thru 0x6bc5d97 (0x2398 bytes) 2024-09-16T20:50:04.673 INFO:teuthology.orchestra.run.smithi139.stdout:5026 write 0x623e800 thru 0x6244d48 (0x6549 bytes) 2024-09-16T20:50:04.673 INFO:teuthology.orchestra.run.smithi139.stdout:5027 read 0x22dc800 thru 0x22e75c7 (0xadc8 bytes) 2024-09-16T20:50:04.685 INFO:teuthology.orchestra.run.smithi139.stdout:5030 trunc from 0xd847600 to 0x3c49600 2024-09-16T20:50:04.711 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:04.709+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:04.717 INFO:teuthology.orchestra.run.smithi139.stdout:5031 punch from 0x24b0000 to 0x24b5a00, (0x5a00 bytes) 2024-09-16T20:50:04.717 INFO:teuthology.orchestra.run.smithi139.stdout:5034 punch from 0x25e3a00 to 0x25ecc00, (0x9200 bytes) 2024-09-16T20:50:04.735 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:04.735 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:50:04.735 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:50:04.735 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:04.736 INFO:teuthology.orchestra.run.smithi139.stdout:5035 write 0x774f000 thru 0x775262c (0x362d bytes) 2024-09-16T20:50:04.740 INFO:teuthology.orchestra.run.smithi139.stdout:5036 trunc from 0x7752800 to 0xb88ca00 2024-09-16T20:50:04.745 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:50:04.758 INFO:teuthology.orchestra.run.smithi139.stdout:5037 read 0x1dd0800 thru 0x1dd97a7 (0x8fa8 bytes) 2024-09-16T20:50:04.759 INFO:teuthology.orchestra.run.smithi139.stdout:5044 read 0x4469200 thru 0x446e4a4 (0x52a5 bytes) 2024-09-16T20:50:04.759 INFO:teuthology.orchestra.run.smithi139.stdout:5049 read 0x3a24c00 thru 0x3a3440f (0xf810 bytes) 2024-09-16T20:50:04.760 INFO:teuthology.orchestra.run.smithi139.stdout:5051 punch from 0xaecd000 to 0xaed9600, (0xc600 bytes) 2024-09-16T20:50:04.760 INFO:teuthology.orchestra.run.smithi139.stdout:5052 read 0x2dd6a00 thru 0x2de1aeb (0xb0ec bytes) 2024-09-16T20:50:04.761 INFO:teuthology.orchestra.run.smithi139.stdout:5053 read 0x3d25c00 thru 0x3d2bcac (0x60ad bytes) 2024-09-16T20:50:04.761 INFO:teuthology.orchestra.run.smithi139.stdout:5055 write 0xa966600 thru 0xa96a398 (0x3d99 bytes) 2024-09-16T20:50:04.762 INFO:teuthology.orchestra.run.smithi139.stdout:5058 write 0x757400 thru 0x758cd4 (0x18d5 bytes) 2024-09-16T20:50:04.763 INFO:teuthology.orchestra.run.smithi139.stdout:5064 punch from 0x1305000 to 0x1312a00, (0xda00 bytes) 2024-09-16T20:50:04.763 INFO:teuthology.orchestra.run.smithi139.stdout:5065 clone 39 order 23 su 4194304 sc 15 2024-09-16T20:50:04.812 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:04.812+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:04.855 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:04.856 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:50:04.856 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:50:04.856 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:50:04.856 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:04.864 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:50:04.948 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:04.948+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:05.013 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:05.013 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:50:05.013 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:50:05.013 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:50:05.013 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.013 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:50:05.013 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:50:05.013 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 131.75690679100001, 2024-09-16T20:50:05.014 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67502346999999996, 2024-09-16T20:50:05.014 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:05.014 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:05.014 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:05.014 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:05.014 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:50:05.014 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:50:05.014 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 1 2024-09-16T20:50:05.014 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.014 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:05.014 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.014 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:05.014 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:50:05.015 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.015 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.015 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.015 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:05.015 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:50:05.015 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.015 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.015 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.015 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:05.015 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254128+0000", 2024-09-16T20:50:05.015 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.463e-06 2024-09-16T20:50:05.015 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.015 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.015 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:05.016 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254135+0000", 2024-09-16T20:50:05.016 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1579999999999999e-06 2024-09-16T20:50:05.016 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.016 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.016 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:05.016 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254146+0000", 2024-09-16T20:50:05.016 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1341e-05 2024-09-16T20:50:05.016 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.016 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.016 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:05.016 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254155+0000", 2024-09-16T20:50:05.016 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.1549999999999996e-06 2024-09-16T20:50:05.016 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.017 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.017 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:05.017 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254221+0000", 2024-09-16T20:50:05.017 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5470999999999997e-05 2024-09-16T20:50:05.017 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.017 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.017 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:05.017 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254231+0000", 2024-09-16T20:50:05.017 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0592e-05 2024-09-16T20:50:05.017 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.017 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.017 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:05.017 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926711+0000", 2024-09-16T20:50:05.017 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67247946599999997 2024-09-16T20:50:05.018 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.018 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.018 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:05.018 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926910+0000", 2024-09-16T20:50:05.018 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019934 2024-09-16T20:50:05.018 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.018 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.018 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:50:05.018 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.927159+0000", 2024-09-16T20:50:05.018 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00024876599999999998 2024-09-16T20:50:05.018 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.018 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.018 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:05.018 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.928797+0000", 2024-09-16T20:50:05.019 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001638308 2024-09-16T20:50:05.019 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.019 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.019 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:05.019 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929026+0000", 2024-09-16T20:50:05.019 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00022876099999999999 2024-09-16T20:50:05.019 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.019 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.019 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:05.019 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929102+0000", 2024-09-16T20:50:05.019 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6184000000000001e-05 2024-09-16T20:50:05.019 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.019 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.019 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:05.020 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929150+0000", 2024-09-16T20:50:05.020 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7465e-05 2024-09-16T20:50:05.020 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.020 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:05.020 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.020 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.020 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.020 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3574 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:05.020 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:50:05.020 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 75.529743259, 2024-09-16T20:50:05.020 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053579441999999998, 2024-09-16T20:50:05.020 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:05.020 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:05.020 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:05.021 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:05.021 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:05.021 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:05.021 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3574 2024-09-16T20:50:05.021 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.021 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:05.021 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.021 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:05.021 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:50:05.021 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.021 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.021 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.021 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:05.021 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:50:05.022 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.022 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.022 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.022 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:05.022 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:50:05.022 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2e-07 2024-09-16T20:50:05.022 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.022 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.022 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:05.022 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481292+0000", 2024-09-16T20:50:05.022 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2630000000000002e-06 2024-09-16T20:50:05.022 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.022 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.022 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:05.023 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481297+0000", 2024-09-16T20:50:05.023 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.296e-06 2024-09-16T20:50:05.023 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.023 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.023 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:05.023 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481300+0000", 2024-09-16T20:50:05.023 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7019999999999999e-06 2024-09-16T20:50:05.023 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.023 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.023 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:05.023 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525533+0000", 2024-09-16T20:50:05.023 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.044232318 2024-09-16T20:50:05.023 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.023 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.024 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:05.024 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525619+0000", 2024-09-16T20:50:05.024 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5892999999999999e-05 2024-09-16T20:50:05.024 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.024 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.024 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:05.024 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525987+0000", 2024-09-16T20:50:05.024 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00036836099999999999 2024-09-16T20:50:05.024 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.024 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.024 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:05.024 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531779+0000", 2024-09-16T20:50:05.024 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0057918240000000001 2024-09-16T20:50:05.024 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.025 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.025 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:05.025 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534747+0000", 2024-09-16T20:50:05.025 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0029681600000000001 2024-09-16T20:50:05.025 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.025 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.025 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:05.025 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534832+0000", 2024-09-16T20:50:05.025 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5180999999999996e-05 2024-09-16T20:50:05.025 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.025 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.025 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:05.025 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534869+0000", 2024-09-16T20:50:05.025 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7024000000000003e-05 2024-09-16T20:50:05.026 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.026 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:05.026 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.026 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.026 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.026 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3575 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:05.026 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:50:05.026 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 75.529728519000003, 2024-09-16T20:50:05.026 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053801463000000001, 2024-09-16T20:50:05.026 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:05.026 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:05.026 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:05.026 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:05.027 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:05.027 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:05.027 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3575 2024-09-16T20:50:05.027 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.027 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:05.027 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.027 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:05.027 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:50:05.027 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.027 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.027 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.027 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:05.027 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:50:05.028 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.028 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.028 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.028 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:05.028 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481305+0000", 2024-09-16T20:50:05.028 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5400000000000002e-07 2024-09-16T20:50:05.028 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.028 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.028 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:05.028 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481307+0000", 2024-09-16T20:50:05.028 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7800000000000001e-06 2024-09-16T20:50:05.028 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.028 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.028 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:05.029 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481312+0000", 2024-09-16T20:50:05.029 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6349999999999997e-06 2024-09-16T20:50:05.029 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.029 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.029 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:05.029 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481315+0000", 2024-09-16T20:50:05.029 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7829999999999999e-06 2024-09-16T20:50:05.029 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.029 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.029 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:05.029 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.526876+0000", 2024-09-16T20:50:05.029 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.045561070000000002 2024-09-16T20:50:05.029 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.030 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.030 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:05.030 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.526954+0000", 2024-09-16T20:50:05.030 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8108000000000005e-05 2024-09-16T20:50:05.030 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.030 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.030 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:05.030 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527282+0000", 2024-09-16T20:50:05.030 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000327483 2024-09-16T20:50:05.030 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.030 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.030 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:05.030 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531784+0000", 2024-09-16T20:50:05.030 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0045020169999999997 2024-09-16T20:50:05.031 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.031 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.031 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:05.031 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534921+0000", 2024-09-16T20:50:05.031 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.003137791 2024-09-16T20:50:05.031 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.031 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.031 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:05.031 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534976+0000", 2024-09-16T20:50:05.031 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4953000000000001e-05 2024-09-16T20:50:05.031 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.031 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.031 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:05.032 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535106+0000", 2024-09-16T20:50:05.032 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000129589 2024-09-16T20:50:05.032 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.032 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:05.032 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.032 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.032 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.032 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3576 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:05.032 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:50:05.032 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 75.529714607000003, 2024-09-16T20:50:05.032 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053955436000000002, 2024-09-16T20:50:05.032 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:05.032 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:05.033 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:05.033 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:05.033 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:05.033 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:05.033 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3576 2024-09-16T20:50:05.033 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.033 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:05.033 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.033 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:05.033 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:50:05.033 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.033 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.033 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.033 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:05.034 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:50:05.034 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.034 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.034 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.034 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:05.034 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481319+0000", 2024-09-16T20:50:05.034 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.53e-07 2024-09-16T20:50:05.034 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.034 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.034 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:05.034 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481320+0000", 2024-09-16T20:50:05.034 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5230000000000001e-06 2024-09-16T20:50:05.034 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.034 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.035 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:05.035 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481330+0000", 2024-09-16T20:50:05.035 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0190000000000001e-05 2024-09-16T20:50:05.035 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.035 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.035 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:05.035 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481333+0000", 2024-09-16T20:50:05.035 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.151e-06 2024-09-16T20:50:05.035 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.035 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.036 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:05.036 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527689+0000", 2024-09-16T20:50:05.036 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046355192000000003 2024-09-16T20:50:05.036 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.036 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.037 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:05.037 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527764+0000", 2024-09-16T20:50:05.037 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5776000000000001e-05 2024-09-16T20:50:05.037 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.037 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.037 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:05.037 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528141+0000", 2024-09-16T20:50:05.037 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00037647599999999999 2024-09-16T20:50:05.037 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.037 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.037 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:05.037 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531788+0000", 2024-09-16T20:50:05.037 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0036470679999999998 2024-09-16T20:50:05.038 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.038 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.038 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:05.038 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535174+0000", 2024-09-16T20:50:05.038 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0033862670000000001 2024-09-16T20:50:05.038 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.038 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.038 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:05.038 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535236+0000", 2024-09-16T20:50:05.038 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1797999999999993e-05 2024-09-16T20:50:05.038 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.038 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.038 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:05.038 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535274+0000", 2024-09-16T20:50:05.039 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7741999999999999e-05 2024-09-16T20:50:05.039 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.039 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:05.039 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.039 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.039 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.039 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3577 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:05.039 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:50:05.039 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 75.529693007000006, 2024-09-16T20:50:05.039 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054073807000000002, 2024-09-16T20:50:05.039 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:05.039 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:05.039 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:05.039 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:05.040 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:05.040 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:05.040 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3577 2024-09-16T20:50:05.040 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.040 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:05.040 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.040 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:05.040 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:50:05.040 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.040 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.040 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.040 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:05.040 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:50:05.041 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.041 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.041 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.041 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:05.041 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:50:05.041 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.96e-07 2024-09-16T20:50:05.041 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.041 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.041 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:05.041 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481342+0000", 2024-09-16T20:50:05.041 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.714e-06 2024-09-16T20:50:05.041 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.041 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.042 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:05.042 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481349+0000", 2024-09-16T20:50:05.042 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4930000000000001e-06 2024-09-16T20:50:05.042 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.042 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.042 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:05.042 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481353+0000", 2024-09-16T20:50:05.042 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4879999999999999e-06 2024-09-16T20:50:05.042 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.042 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.042 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:05.042 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528619+0000", 2024-09-16T20:50:05.042 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047266351999999998 2024-09-16T20:50:05.042 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.043 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.043 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:05.043 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528706+0000", 2024-09-16T20:50:05.043 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6796e-05 2024-09-16T20:50:05.043 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.043 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.043 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:05.043 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529148+0000", 2024-09-16T20:50:05.043 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00044182999999999999 2024-09-16T20:50:05.043 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.043 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.043 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:05.043 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531792+0000", 2024-09-16T20:50:05.044 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0026443119999999998 2024-09-16T20:50:05.044 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.044 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.044 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:05.044 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535319+0000", 2024-09-16T20:50:05.044 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0035266439999999998 2024-09-16T20:50:05.044 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.044 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.044 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:05.044 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535372+0000", 2024-09-16T20:50:05.044 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3207000000000001e-05 2024-09-16T20:50:05.044 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.044 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.044 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:05.045 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535414+0000", 2024-09-16T20:50:05.045 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1675000000000003e-05 2024-09-16T20:50:05.045 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.045 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:05.045 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.045 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.045 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.045 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3578 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:05.045 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:50:05.045 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 75.529671837999999, 2024-09-16T20:50:05.045 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054189586999999997, 2024-09-16T20:50:05.045 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:05.045 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:05.046 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:05.046 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:05.046 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:05.046 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:05.046 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3578 2024-09-16T20:50:05.046 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.046 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:05.046 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.046 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:05.046 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:50:05.046 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.046 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.046 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.046 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:05.047 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:50:05.047 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.047 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.047 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.047 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:05.047 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:50:05.047 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0800000000000001e-07 2024-09-16T20:50:05.047 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.047 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.048 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:05.048 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481363+0000", 2024-09-16T20:50:05.048 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0339999999999999e-06 2024-09-16T20:50:05.048 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.048 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.048 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:05.048 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481371+0000", 2024-09-16T20:50:05.048 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4440000000000002e-06 2024-09-16T20:50:05.048 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.048 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.048 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:05.048 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481374+0000", 2024-09-16T20:50:05.048 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2049999999999998e-06 2024-09-16T20:50:05.049 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.049 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.049 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:05.049 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529555+0000", 2024-09-16T20:50:05.049 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048181202999999999 2024-09-16T20:50:05.049 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.049 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.049 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:05.049 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529616+0000", 2024-09-16T20:50:05.049 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0912000000000002e-05 2024-09-16T20:50:05.049 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.049 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.049 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:05.050 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529939+0000", 2024-09-16T20:50:05.050 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00032269800000000001 2024-09-16T20:50:05.050 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.050 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.050 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:05.050 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532602+0000", 2024-09-16T20:50:05.050 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0026634940000000002 2024-09-16T20:50:05.050 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.050 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.050 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:05.050 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535465+0000", 2024-09-16T20:50:05.050 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.002862149 2024-09-16T20:50:05.050 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.050 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.051 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:05.051 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535516+0000", 2024-09-16T20:50:05.051 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1696000000000001e-05 2024-09-16T20:50:05.051 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.051 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.051 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:05.051 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535551+0000", 2024-09-16T20:50:05.051 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4443999999999998e-05 2024-09-16T20:50:05.051 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.051 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:05.051 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.051 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.051 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.051 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3579 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:05.052 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:50:05.052 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 75.529653398999997, 2024-09-16T20:50:05.052 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054295151, 2024-09-16T20:50:05.052 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:05.052 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:05.052 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:05.052 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:05.052 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:05.052 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:05.052 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3579 2024-09-16T20:50:05.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.052 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:05.052 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.053 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:05.053 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:50:05.053 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.053 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.053 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.053 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:05.053 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:50:05.053 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.053 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.053 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.053 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:05.053 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:50:05.053 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3300000000000001e-07 2024-09-16T20:50:05.053 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.054 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.054 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:05.054 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481381+0000", 2024-09-16T20:50:05.054 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.649e-06 2024-09-16T20:50:05.054 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.054 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.054 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:05.054 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481387+0000", 2024-09-16T20:50:05.054 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4519999999999996e-06 2024-09-16T20:50:05.054 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.054 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.054 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:05.054 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481390+0000", 2024-09-16T20:50:05.054 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0170000000000001e-06 2024-09-16T20:50:05.055 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.055 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.055 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:05.055 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530756+0000", 2024-09-16T20:50:05.055 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049365882999999999 2024-09-16T20:50:05.055 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.055 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.055 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:05.055 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530845+0000", 2024-09-16T20:50:05.055 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9197000000000007e-05 2024-09-16T20:50:05.055 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.055 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.055 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:05.055 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531191+0000", 2024-09-16T20:50:05.056 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00034562500000000001 2024-09-16T20:50:05.056 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.056 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.056 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:05.056 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534078+0000", 2024-09-16T20:50:05.056 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0028878179999999999 2024-09-16T20:50:05.056 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.056 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.056 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:05.056 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535589+0000", 2024-09-16T20:50:05.056 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00151085 2024-09-16T20:50:05.056 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.057 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.057 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:05.057 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535639+0000", 2024-09-16T20:50:05.057 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9697000000000002e-05 2024-09-16T20:50:05.057 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.057 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.057 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:05.057 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535675+0000", 2024-09-16T20:50:05.057 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5729999999999998e-05 2024-09-16T20:50:05.057 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.057 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:05.057 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.057 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.057 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.058 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3580 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:05.058 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:50:05.058 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 75.529639767000006, 2024-09-16T20:50:05.058 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054397361999999998, 2024-09-16T20:50:05.058 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:05.058 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:05.058 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:05.058 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:05.058 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:05.058 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:05.058 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3580 2024-09-16T20:50:05.058 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.058 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:05.059 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.059 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:05.059 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:50:05.059 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.059 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.059 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.059 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:05.059 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:50:05.059 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.059 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.059 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.059 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:05.059 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:50:05.060 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2600000000000001e-07 2024-09-16T20:50:05.060 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.060 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.060 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:05.060 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481395+0000", 2024-09-16T20:50:05.060 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.544e-06 2024-09-16T20:50:05.060 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.060 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.060 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:05.060 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481404+0000", 2024-09-16T20:50:05.060 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8850000000000005e-06 2024-09-16T20:50:05.060 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.060 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.061 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:05.061 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481407+0000", 2024-09-16T20:50:05.061 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1540000000000002e-06 2024-09-16T20:50:05.061 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.061 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.061 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:05.061 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531823+0000", 2024-09-16T20:50:05.061 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050415637999999999 2024-09-16T20:50:05.061 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.061 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.061 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:05.061 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531895+0000", 2024-09-16T20:50:05.061 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2179000000000001e-05 2024-09-16T20:50:05.062 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.062 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.062 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:05.062 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532236+0000", 2024-09-16T20:50:05.062 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00034073799999999999 2024-09-16T20:50:05.062 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.062 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.062 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:05.062 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534083+0000", 2024-09-16T20:50:05.062 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001847298 2024-09-16T20:50:05.062 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.062 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.062 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:05.063 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535714+0000", 2024-09-16T20:50:05.063 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0016316810000000001 2024-09-16T20:50:05.063 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.063 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.063 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:05.063 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535759+0000", 2024-09-16T20:50:05.063 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4478999999999999e-05 2024-09-16T20:50:05.063 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.063 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.063 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:05.063 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535791+0000", 2024-09-16T20:50:05.063 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.154e-05 2024-09-16T20:50:05.063 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.064 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:05.064 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.064 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.064 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.064 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3581 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:05.064 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:50:05.064 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 75.529617770000002, 2024-09-16T20:50:05.064 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.057094403000000002, 2024-09-16T20:50:05.064 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:05.064 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:05.064 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:05.065 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:05.065 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:05.065 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:05.065 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3581 2024-09-16T20:50:05.065 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.065 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:05.065 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.065 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:05.065 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:50:05.065 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.065 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.065 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.065 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:05.066 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:50:05.066 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.066 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.066 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.066 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:05.066 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:50:05.066 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8799999999999998e-07 2024-09-16T20:50:05.066 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.066 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.066 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:05.066 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481418+0000", 2024-09-16T20:50:05.066 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1859999999999999e-06 2024-09-16T20:50:05.066 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.066 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.067 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:05.067 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481423+0000", 2024-09-16T20:50:05.067 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8279999999999999e-06 2024-09-16T20:50:05.067 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.067 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.067 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:05.067 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481426+0000", 2024-09-16T20:50:05.067 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0110000000000001e-06 2024-09-16T20:50:05.067 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.067 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.067 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:05.067 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532631+0000", 2024-09-16T20:50:05.067 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051204514999999999 2024-09-16T20:50:05.067 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.068 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.068 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:05.068 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.533381+0000", 2024-09-16T20:50:05.068 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00075029000000000003 2024-09-16T20:50:05.068 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.068 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.068 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:05.068 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.533739+0000", 2024-09-16T20:50:05.068 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00035773100000000002 2024-09-16T20:50:05.068 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.068 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.068 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:05.068 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535123+0000", 2024-09-16T20:50:05.068 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0013836160000000001 2024-09-16T20:50:05.069 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.069 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.069 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:05.069 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538413+0000", 2024-09-16T20:50:05.069 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0032902510000000001 2024-09-16T20:50:05.069 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.069 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.069 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:05.069 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538470+0000", 2024-09-16T20:50:05.069 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6697999999999998e-05 2024-09-16T20:50:05.069 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.069 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.069 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:05.069 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538510+0000", 2024-09-16T20:50:05.070 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9988999999999999e-05 2024-09-16T20:50:05.070 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.070 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:05.070 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.070 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.070 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.070 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:12732 3.d 3:b752444a:::rbd_header.110e747f0779:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-16T20:50:05.070 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:50:05.070 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 60.242545192000001, 2024-09-16T20:50:05.070 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.065302467000000003, 2024-09-16T20:50:05.070 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:05.070 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:05.070 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:05.071 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:05.071 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:50:05.071 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:50:05.071 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 12732 2024-09-16T20:50:05.071 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.071 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:05.071 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.071 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:05.071 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:50:05.071 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.071 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.071 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.071 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:05.071 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:50:05.072 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.072 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.072 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.072 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:05.072 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768490+0000", 2024-09-16T20:50:05.072 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2859999999999998e-06 2024-09-16T20:50:05.072 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.072 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.072 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:05.073 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768501+0000", 2024-09-16T20:50:05.073 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1226999999999999e-05 2024-09-16T20:50:05.073 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.073 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.073 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:05.073 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768517+0000", 2024-09-16T20:50:05.073 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.592e-05 2024-09-16T20:50:05.073 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.073 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.073 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:05.073 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768528+0000", 2024-09-16T20:50:05.073 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0689999999999999e-05 2024-09-16T20:50:05.073 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.073 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.074 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:05.074 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768584+0000", 2024-09-16T20:50:05.074 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5889e-05 2024-09-16T20:50:05.074 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.074 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.074 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:05.074 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768720+0000", 2024-09-16T20:50:05.074 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000136718 2024-09-16T20:50:05.074 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.074 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.074 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:05.074 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.769066+0000", 2024-09-16T20:50:05.074 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000346013 2024-09-16T20:50:05.075 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.075 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.075 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:05.075 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833659+0000", 2024-09-16T20:50:05.075 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.064592822999999994 2024-09-16T20:50:05.075 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.075 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.075 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:05.075 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833719+0000", 2024-09-16T20:50:05.075 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9441000000000002e-05 2024-09-16T20:50:05.075 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.075 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.076 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:05.076 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833758+0000", 2024-09-16T20:50:05.076 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9122000000000003e-05 2024-09-16T20:50:05.076 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.076 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.076 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:05.076 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833790+0000", 2024-09-16T20:50:05.076 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2338000000000001e-05 2024-09-16T20:50:05.076 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.076 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:05.076 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.076 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.076 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.077 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.5089.0:12732 3.d e73/23 ondisk, result = 0)", 2024-09-16T20:50:05.077 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:50:05.077 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 60.240943022000003, 2024-09-16T20:50:05.077 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.063706456999999994, 2024-09-16T20:50:05.077 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:05.077 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:05.077 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:05.077 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:05.077 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.077 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:05.077 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:50:05.077 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.077 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.077 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.078 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:05.078 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:50:05.078 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.078 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.078 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.078 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:05.078 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770091+0000", 2024-09-16T20:50:05.078 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.53e-06 2024-09-16T20:50:05.078 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.078 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.078 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:05.078 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770104+0000", 2024-09-16T20:50:05.078 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2203e-05 2024-09-16T20:50:05.079 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.079 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.079 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:05.079 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833637+0000", 2024-09-16T20:50:05.079 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.063533201999999997 2024-09-16T20:50:05.079 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.079 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.079 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:05.079 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833655+0000", 2024-09-16T20:50:05.079 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8369e-05 2024-09-16T20:50:05.079 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.079 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.079 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:05.079 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833707+0000", 2024-09-16T20:50:05.080 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2061999999999997e-05 2024-09-16T20:50:05.080 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.080 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.080 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:05.080 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833715+0000", 2024-09-16T20:50:05.080 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5179999999999998e-06 2024-09-16T20:50:05.080 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.080 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.080 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:05.080 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833796+0000", 2024-09-16T20:50:05.080 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1573e-05 2024-09-16T20:50:05.080 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.080 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:05.080 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.081 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.081 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.081 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3281 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 220160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:05.081 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041070+0000", 2024-09-16T20:50:05.081 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 14.969962553, 2024-09-16T20:50:05.081 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051593550000000002, 2024-09-16T20:50:05.081 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:05.081 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:05.081 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:05.081 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:05.081 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:05.081 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:05.081 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3281 2024-09-16T20:50:05.081 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.082 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:05.082 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.082 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:05.082 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041070+0000", 2024-09-16T20:50:05.082 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.082 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.082 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.082 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:05.082 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041070+0000", 2024-09-16T20:50:05.082 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.082 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.082 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.082 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:05.082 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041071+0000", 2024-09-16T20:50:05.082 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1899999999999999e-07 2024-09-16T20:50:05.083 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.083 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.083 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:05.083 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041072+0000", 2024-09-16T20:50:05.083 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.248e-06 2024-09-16T20:50:05.083 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.083 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.083 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:05.083 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041075+0000", 2024-09-16T20:50:05.083 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0029999999999999e-06 2024-09-16T20:50:05.083 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.083 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.083 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:05.083 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041077+0000", 2024-09-16T20:50:05.084 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.621e-06 2024-09-16T20:50:05.084 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.084 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.084 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:05.084 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.092431+0000", 2024-09-16T20:50:05.084 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051353801999999997 2024-09-16T20:50:05.084 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.084 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.084 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:05.084 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.092512+0000", 2024-09-16T20:50:05.084 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1106000000000006e-05 2024-09-16T20:50:05.084 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.084 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.085 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:05.085 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.092664+0000", 2024-09-16T20:50:05.085 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00015155100000000001 2024-09-16T20:50:05.085 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.085 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:05.085 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.085 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.085 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.085 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3282 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 225280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:05.085 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041081+0000", 2024-09-16T20:50:05.085 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 14.969952275000001, 2024-09-16T20:50:05.085 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051731948, 2024-09-16T20:50:05.085 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:05.085 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:05.086 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:05.086 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:05.086 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:05.086 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:05.086 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3282 2024-09-16T20:50:05.086 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.086 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:05.086 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.086 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:05.086 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041081+0000", 2024-09-16T20:50:05.086 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.086 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.086 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.086 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:05.087 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041081+0000", 2024-09-16T20:50:05.087 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.087 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.087 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.087 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:05.087 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041081+0000", 2024-09-16T20:50:05.087 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3099999999999999e-07 2024-09-16T20:50:05.087 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.087 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.087 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:05.087 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041082+0000", 2024-09-16T20:50:05.087 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1289999999999999e-06 2024-09-16T20:50:05.087 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.087 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.088 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:05.088 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041085+0000", 2024-09-16T20:50:05.088 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2320000000000001e-06 2024-09-16T20:50:05.088 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.088 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.088 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:05.088 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041090+0000", 2024-09-16T20:50:05.088 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3730000000000003e-06 2024-09-16T20:50:05.088 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.088 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.088 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:05.088 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.092678+0000", 2024-09-16T20:50:05.088 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051588235000000003 2024-09-16T20:50:05.088 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.089 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.089 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:05.089 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.092717+0000", 2024-09-16T20:50:05.089 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9057999999999998e-05 2024-09-16T20:50:05.089 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.089 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.089 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:05.089 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.092813+0000", 2024-09-16T20:50:05.089 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.569e-05 2024-09-16T20:50:05.089 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.089 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:05.089 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.089 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.089 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.090 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3283 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 230400~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:05.090 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041102+0000", 2024-09-16T20:50:05.090 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 14.969930614999999, 2024-09-16T20:50:05.090 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051974723, 2024-09-16T20:50:05.090 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:05.090 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:05.090 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:05.090 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:05.090 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:05.090 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:05.090 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3283 2024-09-16T20:50:05.090 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.090 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:05.090 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.091 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:05.091 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041102+0000", 2024-09-16T20:50:05.091 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.091 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.091 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.091 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:05.091 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041102+0000", 2024-09-16T20:50:05.091 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.091 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.091 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.091 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:05.091 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041103+0000", 2024-09-16T20:50:05.091 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2399999999999999e-07 2024-09-16T20:50:05.092 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.092 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.092 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:05.092 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041110+0000", 2024-09-16T20:50:05.092 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4290000000000004e-06 2024-09-16T20:50:05.092 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.092 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.092 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:05.092 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041118+0000", 2024-09-16T20:50:05.092 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3849999999999996e-06 2024-09-16T20:50:05.092 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.092 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.092 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:05.093 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041123+0000", 2024-09-16T20:50:05.093 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1410000000000004e-06 2024-09-16T20:50:05.093 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.093 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.093 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:05.093 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.092845+0000", 2024-09-16T20:50:05.093 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051722207999999999 2024-09-16T20:50:05.093 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.093 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.093 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:05.093 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.092926+0000", 2024-09-16T20:50:05.093 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0727000000000001e-05 2024-09-16T20:50:05.093 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.093 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.094 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:05.094 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093077+0000", 2024-09-16T20:50:05.094 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000151409 2024-09-16T20:50:05.094 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.094 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:05.094 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.094 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.094 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.094 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3284 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 235520~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:05.094 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041129+0000", 2024-09-16T20:50:05.094 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 14.969904284, 2024-09-16T20:50:05.094 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052120760000000002, 2024-09-16T20:50:05.094 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:05.094 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:05.095 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:05.095 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:05.095 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:05.095 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:05.095 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3284 2024-09-16T20:50:05.095 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.095 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:05.095 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.095 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:05.095 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041129+0000", 2024-09-16T20:50:05.095 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.095 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.095 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.095 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:05.095 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041129+0000", 2024-09-16T20:50:05.096 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.096 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.096 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.096 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:05.096 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041129+0000", 2024-09-16T20:50:05.096 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3700000000000001e-07 2024-09-16T20:50:05.096 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.096 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.096 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:05.096 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041130+0000", 2024-09-16T20:50:05.096 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4050000000000001e-06 2024-09-16T20:50:05.096 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.096 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.097 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:05.097 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041134+0000", 2024-09-16T20:50:05.097 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4290000000000001e-06 2024-09-16T20:50:05.097 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.097 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.097 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:05.097 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041136+0000", 2024-09-16T20:50:05.097 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.678e-06 2024-09-16T20:50:05.097 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.097 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.097 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:05.097 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093109+0000", 2024-09-16T20:50:05.097 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051972879999999999 2024-09-16T20:50:05.097 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.098 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.098 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:05.098 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093149+0000", 2024-09-16T20:50:05.098 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0040000000000003e-05 2024-09-16T20:50:05.098 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.098 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.098 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:05.098 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093249+0000", 2024-09-16T20:50:05.098 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.9990999999999999e-05 2024-09-16T20:50:05.098 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.098 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:05.098 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.098 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.098 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.099 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3285 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 240640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:05.099 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041140+0000", 2024-09-16T20:50:05.099 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 14.969893244, 2024-09-16T20:50:05.099 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052375646999999997, 2024-09-16T20:50:05.099 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:05.099 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:05.099 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:05.099 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:05.099 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:05.099 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:05.099 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3285 2024-09-16T20:50:05.099 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.099 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:05.100 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.100 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:05.100 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041140+0000", 2024-09-16T20:50:05.100 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.100 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.100 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.100 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:05.100 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041140+0000", 2024-09-16T20:50:05.100 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.100 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.100 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.100 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:05.100 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041140+0000", 2024-09-16T20:50:05.100 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2399999999999999e-07 2024-09-16T20:50:05.101 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.101 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.101 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:05.101 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041141+0000", 2024-09-16T20:50:05.101 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2759999999999999e-06 2024-09-16T20:50:05.101 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.101 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.101 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:05.101 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041144+0000", 2024-09-16T20:50:05.101 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0280000000000001e-06 2024-09-16T20:50:05.101 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.101 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.101 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:05.101 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041147+0000", 2024-09-16T20:50:05.102 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5900000000000002e-06 2024-09-16T20:50:05.102 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.102 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.102 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:05.102 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093284+0000", 2024-09-16T20:50:05.102 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052137645000000003 2024-09-16T20:50:05.102 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.102 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.102 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:05.102 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093365+0000", 2024-09-16T20:50:05.102 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0446999999999997e-05 2024-09-16T20:50:05.102 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.102 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.102 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:05.103 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093515+0000", 2024-09-16T20:50:05.103 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00015043700000000001 2024-09-16T20:50:05.103 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.103 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:05.103 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.103 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.103 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.103 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3286 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 245760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:05.103 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:50:05.103 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 14.969881085000001, 2024-09-16T20:50:05.103 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052523564000000002, 2024-09-16T20:50:05.103 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:05.103 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:05.103 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:05.104 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:05.104 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:05.104 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:05.104 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3286 2024-09-16T20:50:05.104 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.104 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:05.104 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.104 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:05.104 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:50:05.104 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.104 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.104 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.104 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:05.105 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:50:05.105 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.105 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.105 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.105 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:05.105 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:50:05.105 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3900000000000001e-07 2024-09-16T20:50:05.105 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.105 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.105 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:05.105 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041153+0000", 2024-09-16T20:50:05.105 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.119e-06 2024-09-16T20:50:05.105 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.105 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.106 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:05.106 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041156+0000", 2024-09-16T20:50:05.106 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0000000000000001e-06 2024-09-16T20:50:05.106 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.106 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.106 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:05.106 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041159+0000", 2024-09-16T20:50:05.106 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5859999999999999e-06 2024-09-16T20:50:05.106 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.106 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.106 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:05.106 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093529+0000", 2024-09-16T20:50:05.106 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052370591000000001 2024-09-16T20:50:05.106 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.107 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.107 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:05.107 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093570+0000", 2024-09-16T20:50:05.107 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0161999999999999e-05 2024-09-16T20:50:05.107 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.107 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.107 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:05.107 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093675+0000", 2024-09-16T20:50:05.107 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010586699999999999 2024-09-16T20:50:05.107 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.107 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:05.107 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.107 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.107 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.108 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3287 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 250880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:05.108 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:50:05.108 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 14.969871036000001, 2024-09-16T20:50:05.108 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052781203999999998, 2024-09-16T20:50:05.108 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:05.108 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:05.108 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:05.108 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:05.108 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:05.108 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:05.108 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3287 2024-09-16T20:50:05.108 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.108 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:05.109 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.109 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:05.109 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:50:05.109 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.109 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.109 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.109 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:05.109 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:50:05.109 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.109 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.109 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.109 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:05.109 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:50:05.109 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1e-07 2024-09-16T20:50:05.110 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.110 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.110 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:05.110 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041163+0000", 2024-09-16T20:50:05.110 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0690000000000001e-06 2024-09-16T20:50:05.110 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.110 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.110 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:05.110 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041166+0000", 2024-09-16T20:50:05.110 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.982e-06 2024-09-16T20:50:05.110 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.110 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.110 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:05.110 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041169+0000", 2024-09-16T20:50:05.111 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.593e-06 2024-09-16T20:50:05.111 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.111 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.111 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:05.111 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093709+0000", 2024-09-16T20:50:05.111 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052540492000000001 2024-09-16T20:50:05.111 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.111 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.111 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:05.111 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093792+0000", 2024-09-16T20:50:05.111 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2436999999999997e-05 2024-09-16T20:50:05.111 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.111 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.111 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:05.112 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093943+0000", 2024-09-16T20:50:05.112 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000151421 2024-09-16T20:50:05.112 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.112 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:05.112 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.112 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.112 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.112 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3288 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 256000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:05.112 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:50:05.112 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 14.969861163000001, 2024-09-16T20:50:05.112 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052928269, 2024-09-16T20:50:05.112 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:05.112 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:05.112 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:05.113 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:05.113 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:05.113 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:05.113 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3288 2024-09-16T20:50:05.113 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.113 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:05.113 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.113 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:05.113 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:50:05.113 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.113 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.113 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.113 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:05.114 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:50:05.114 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.114 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.114 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.114 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:05.114 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:50:05.114 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2700000000000001e-07 2024-09-16T20:50:05.114 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.114 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.114 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:05.114 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041173+0000", 2024-09-16T20:50:05.114 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.113e-06 2024-09-16T20:50:05.114 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.114 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.114 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:05.115 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041176+0000", 2024-09-16T20:50:05.115 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9799999999999998e-06 2024-09-16T20:50:05.115 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.115 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.115 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:05.115 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041179+0000", 2024-09-16T20:50:05.115 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.667e-06 2024-09-16T20:50:05.115 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.115 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.115 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:05.115 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093957+0000", 2024-09-16T20:50:05.115 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052778703000000003 2024-09-16T20:50:05.115 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.115 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.116 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:05.116 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093999+0000", 2024-09-16T20:50:05.116 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1300000000000001e-05 2024-09-16T20:50:05.116 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.116 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.116 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:05.116 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094100+0000", 2024-09-16T20:50:05.116 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101279 2024-09-16T20:50:05.116 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.116 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:05.116 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.116 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.116 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.116 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3289 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 261120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:05.117 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:50:05.117 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 14.969851167, 2024-09-16T20:50:05.117 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053182976, 2024-09-16T20:50:05.117 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:05.117 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:05.117 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:05.117 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:05.117 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:05.117 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:05.117 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3289 2024-09-16T20:50:05.117 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.117 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:05.117 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.117 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:05.118 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:50:05.118 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.118 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.118 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.118 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:05.118 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:50:05.118 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.118 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.118 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.118 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:05.118 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:50:05.118 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2999999999999999e-07 2024-09-16T20:50:05.118 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.119 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.119 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:05.119 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041183+0000", 2024-09-16T20:50:05.119 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0720000000000001e-06 2024-09-16T20:50:05.119 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.119 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.119 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:05.119 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041186+0000", 2024-09-16T20:50:05.119 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9869999999999999e-06 2024-09-16T20:50:05.119 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.119 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.119 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:05.119 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041189+0000", 2024-09-16T20:50:05.119 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.587e-06 2024-09-16T20:50:05.120 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.120 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.120 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:05.120 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094136+0000", 2024-09-16T20:50:05.120 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052947270999999997 2024-09-16T20:50:05.120 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.120 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.120 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:05.120 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094213+0000", 2024-09-16T20:50:05.120 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7552000000000006e-05 2024-09-16T20:50:05.120 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.120 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.120 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:05.121 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094365+0000", 2024-09-16T20:50:05.121 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00015127700000000001 2024-09-16T20:50:05.121 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.121 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:05.121 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.121 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.121 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:05.121 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:05.122 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:50:05.123 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:05.121+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:05.223 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:05.221+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:05.271 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:05.271 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:50:05.271 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:50:05.271 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:05.279 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:50:05.324 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:05.322+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:05.387 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:05.387 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:50:05.387 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:50:05.387 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:50:05.387 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:05.396 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:50:05.424 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:05.423+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:05.503 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:05.504 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:50:05.504 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:50:05.504 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:50:05.504 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.504 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3714 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1480704~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:05.504 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:50:05.504 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 102.86308323999999, 2024-09-16T20:50:05.504 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050152552000000003, 2024-09-16T20:50:05.504 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:05.505 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:05.505 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:05.505 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:05.505 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:05.505 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:05.505 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3714 2024-09-16T20:50:05.505 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.506 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:05.506 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.506 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:05.506 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:50:05.506 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.506 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.506 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.506 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:05.506 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:50:05.506 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.506 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.506 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.507 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:05.507 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:50:05.507 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0499999999999999e-07 2024-09-16T20:50:05.507 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.507 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.507 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:05.507 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639007+0000", 2024-09-16T20:50:05.507 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5340000000000001e-06 2024-09-16T20:50:05.507 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.507 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.507 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:05.507 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639013+0000", 2024-09-16T20:50:05.507 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3260000000000003e-06 2024-09-16T20:50:05.507 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.508 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.508 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:05.508 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639018+0000", 2024-09-16T20:50:05.508 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2850000000000002e-06 2024-09-16T20:50:05.508 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.508 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.508 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:05.508 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689026+0000", 2024-09-16T20:50:05.508 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.05000798 2024-09-16T20:50:05.508 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.508 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.508 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:05.508 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689077+0000", 2024-09-16T20:50:05.509 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1063999999999998e-05 2024-09-16T20:50:05.509 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.509 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.509 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:05.509 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689158+0000", 2024-09-16T20:50:05.509 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1057999999999995e-05 2024-09-16T20:50:05.509 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.509 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:05.509 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.509 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.509 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.509 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3716 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1488896~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:05.509 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:50:05.510 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 102.86306543000001, 2024-09-16T20:50:05.510 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050280142, 2024-09-16T20:50:05.510 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:05.510 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:05.510 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:05.510 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:05.510 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:05.510 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:05.510 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3716 2024-09-16T20:50:05.510 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.510 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:05.510 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.510 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:05.510 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:50:05.511 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.511 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.511 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.511 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:05.511 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:50:05.511 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.511 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.511 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.511 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:05.511 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639024+0000", 2024-09-16T20:50:05.511 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9999999999999997e-07 2024-09-16T20:50:05.511 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.511 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.511 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:05.512 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639026+0000", 2024-09-16T20:50:05.512 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.781e-06 2024-09-16T20:50:05.512 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.512 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.512 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:05.512 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639032+0000", 2024-09-16T20:50:05.512 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7780000000000004e-06 2024-09-16T20:50:05.512 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.512 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.512 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:05.512 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639036+0000", 2024-09-16T20:50:05.512 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6909999999999996e-06 2024-09-16T20:50:05.512 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.513 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.513 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:05.513 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689191+0000", 2024-09-16T20:50:05.513 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050155284000000001 2024-09-16T20:50:05.513 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.513 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.513 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:05.513 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689242+0000", 2024-09-16T20:50:05.513 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0662999999999999e-05 2024-09-16T20:50:05.513 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.513 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.513 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:05.513 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689304+0000", 2024-09-16T20:50:05.514 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1345000000000003e-05 2024-09-16T20:50:05.514 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.514 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:05.514 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.514 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.514 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.514 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3719 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1497088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:05.514 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:50:05.514 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 102.863038152, 2024-09-16T20:50:05.514 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050399715999999997, 2024-09-16T20:50:05.514 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:05.514 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:05.514 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:05.515 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:05.515 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:05.515 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:05.515 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3719 2024-09-16T20:50:05.515 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.515 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:05.515 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.515 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:05.515 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:50:05.515 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.515 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.515 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.515 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:05.515 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:50:05.516 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.516 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.516 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.516 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:05.516 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:50:05.516 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9700000000000002e-07 2024-09-16T20:50:05.516 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.516 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.516 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:05.516 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639053+0000", 2024-09-16T20:50:05.516 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2900000000000001e-06 2024-09-16T20:50:05.516 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.516 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.517 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:05.517 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639060+0000", 2024-09-16T20:50:05.517 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7750000000000004e-06 2024-09-16T20:50:05.517 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.517 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.517 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:05.517 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639066+0000", 2024-09-16T20:50:05.517 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0639999999999999e-06 2024-09-16T20:50:05.517 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.517 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.517 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:05.517 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689337+0000", 2024-09-16T20:50:05.517 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050270784999999998 2024-09-16T20:50:05.518 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.518 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.518 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:05.518 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689392+0000", 2024-09-16T20:50:05.518 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4920999999999998e-05 2024-09-16T20:50:05.518 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.518 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.518 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:05.518 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689450+0000", 2024-09-16T20:50:05.519 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8483999999999998e-05 2024-09-16T20:50:05.519 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.519 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:05.519 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.519 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.519 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.519 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3721 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1505280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:05.519 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:50:05.520 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 102.86301502000001, 2024-09-16T20:50:05.520 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050513608000000002, 2024-09-16T20:50:05.520 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:05.520 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:05.520 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:05.520 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:05.520 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:05.520 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:05.520 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3721 2024-09-16T20:50:05.521 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.521 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:05.521 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.521 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:05.521 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:50:05.521 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.521 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.522 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.522 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:05.522 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:50:05.522 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.522 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.522 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.522 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:05.522 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:50:05.522 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7e-07 2024-09-16T20:50:05.523 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.523 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.523 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:05.523 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639076+0000", 2024-09-16T20:50:05.523 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8899999999999999e-06 2024-09-16T20:50:05.523 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.523 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.523 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:05.523 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639083+0000", 2024-09-16T20:50:05.524 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1210000000000001e-06 2024-09-16T20:50:05.524 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.524 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.524 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:05.524 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639089+0000", 2024-09-16T20:50:05.524 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.643e-06 2024-09-16T20:50:05.524 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.524 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.525 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:05.525 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689474+0000", 2024-09-16T20:50:05.525 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050385679000000003 2024-09-16T20:50:05.525 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.525 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.525 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:05.525 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689533+0000", 2024-09-16T20:50:05.525 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8303000000000001e-05 2024-09-16T20:50:05.526 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.526 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.526 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:05.526 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689587+0000", 2024-09-16T20:50:05.526 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4601999999999997e-05 2024-09-16T20:50:05.526 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.526 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:05.526 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.527 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.527 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.527 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3723 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1513472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:05.527 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:50:05.527 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 102.862984216, 2024-09-16T20:50:05.527 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050699691999999998, 2024-09-16T20:50:05.527 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:05.527 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:05.528 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:05.528 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:05.528 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:05.528 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:05.528 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3723 2024-09-16T20:50:05.528 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.528 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:05.528 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.528 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:05.529 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:50:05.529 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.529 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.529 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.529 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:05.529 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:50:05.529 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.529 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.530 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.530 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:05.530 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:50:05.530 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3399999999999999e-07 2024-09-16T20:50:05.530 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.530 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.530 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:05.530 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639107+0000", 2024-09-16T20:50:05.530 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3860000000000001e-06 2024-09-16T20:50:05.530 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.531 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.531 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:05.531 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639114+0000", 2024-09-16T20:50:05.531 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9800000000000001e-06 2024-09-16T20:50:05.531 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.531 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.531 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:05.531 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639129+0000", 2024-09-16T20:50:05.531 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4887e-05 2024-09-16T20:50:05.531 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.531 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.531 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:05.531 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689625+0000", 2024-09-16T20:50:05.531 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050495848000000003 2024-09-16T20:50:05.532 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.532 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.532 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:05.532 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689720+0000", 2024-09-16T20:50:05.532 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4821000000000006e-05 2024-09-16T20:50:05.532 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.532 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.532 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:05.532 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689804+0000", 2024-09-16T20:50:05.532 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4235999999999998e-05 2024-09-16T20:50:05.532 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.532 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:05.532 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.532 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.533 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.533 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3725 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1521664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:05.533 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:50:05.533 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 102.86295062000001, 2024-09-16T20:50:05.533 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050797660000000001, 2024-09-16T20:50:05.533 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:05.533 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:05.533 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:05.533 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:05.533 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:05.533 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:05.533 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3725 2024-09-16T20:50:05.533 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.533 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:05.534 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.534 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:05.534 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:50:05.534 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.534 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.534 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.534 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:05.534 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:50:05.534 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.534 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.534 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.534 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:05.534 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639139+0000", 2024-09-16T20:50:05.535 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8999999999999997e-07 2024-09-16T20:50:05.535 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.535 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.535 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:05.535 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639141+0000", 2024-09-16T20:50:05.535 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.182e-06 2024-09-16T20:50:05.535 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.535 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.535 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:05.535 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639154+0000", 2024-09-16T20:50:05.536 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3626e-05 2024-09-16T20:50:05.536 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.536 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.536 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:05.536 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639160+0000", 2024-09-16T20:50:05.536 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6810000000000003e-06 2024-09-16T20:50:05.536 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.536 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.536 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:05.536 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689831+0000", 2024-09-16T20:50:05.536 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050670907000000001 2024-09-16T20:50:05.536 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.536 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.536 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:05.537 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689882+0000", 2024-09-16T20:50:05.537 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0445999999999998e-05 2024-09-16T20:50:05.537 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.537 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.537 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:05.537 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689936+0000", 2024-09-16T20:50:05.537 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4327999999999999e-05 2024-09-16T20:50:05.537 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.537 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:05.537 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.537 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.538 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.538 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3727 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1529856~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:05.538 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:50:05.538 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 102.862915043, 2024-09-16T20:50:05.538 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051016922999999999, 2024-09-16T20:50:05.538 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:05.539 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:05.539 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:05.539 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:05.539 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:05.539 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:05.539 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3727 2024-09-16T20:50:05.539 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.539 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:05.539 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.539 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:05.539 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:50:05.539 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.539 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.540 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.540 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:05.540 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:50:05.540 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.540 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.540 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.540 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:05.540 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:50:05.540 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9999999999999998e-07 2024-09-16T20:50:05.540 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.540 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.540 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:05.540 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639183+0000", 2024-09-16T20:50:05.541 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2399999999999996e-06 2024-09-16T20:50:05.541 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.541 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.541 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:05.541 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639195+0000", 2024-09-16T20:50:05.541 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1113e-05 2024-09-16T20:50:05.541 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.541 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.541 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:05.541 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639206+0000", 2024-09-16T20:50:05.541 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1503e-05 2024-09-16T20:50:05.541 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.541 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.541 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:05.542 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689969+0000", 2024-09-16T20:50:05.542 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050762676 2024-09-16T20:50:05.542 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.542 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.542 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:05.542 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690071+0000", 2024-09-16T20:50:05.542 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102163 2024-09-16T20:50:05.542 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.542 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.542 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:05.542 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690191+0000", 2024-09-16T20:50:05.542 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000119728 2024-09-16T20:50:05.542 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.542 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:05.542 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.543 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.543 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.543 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3730 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1538048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:05.543 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:50:05.543 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 102.86287348, 2024-09-16T20:50:05.543 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051131467999999999, 2024-09-16T20:50:05.543 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:05.543 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:05.543 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:05.543 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:05.543 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:05.543 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:05.543 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3730 2024-09-16T20:50:05.543 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.544 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:05.544 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.544 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:05.544 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:50:05.544 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.544 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.544 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.544 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:05.544 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:50:05.544 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.544 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.544 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.544 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:05.545 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639216+0000", 2024-09-16T20:50:05.545 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.39e-07 2024-09-16T20:50:05.545 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.545 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.545 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:05.545 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639218+0000", 2024-09-16T20:50:05.545 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.537e-06 2024-09-16T20:50:05.545 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.545 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.545 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:05.545 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639229+0000", 2024-09-16T20:50:05.545 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1122e-05 2024-09-16T20:50:05.545 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.546 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.546 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:05.546 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639237+0000", 2024-09-16T20:50:05.546 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8509999999999999e-06 2024-09-16T20:50:05.546 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.546 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.546 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:05.546 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690227+0000", 2024-09-16T20:50:05.546 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050990147 2024-09-16T20:50:05.546 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.546 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.546 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:05.546 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690278+0000", 2024-09-16T20:50:05.546 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0223999999999999e-05 2024-09-16T20:50:05.546 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.547 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.547 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:05.547 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690347+0000", 2024-09-16T20:50:05.547 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9147999999999996e-05 2024-09-16T20:50:05.547 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.547 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:05.547 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.547 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.547 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.547 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3731 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1546240~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:05.547 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:50:05.547 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 102.86284277199999, 2024-09-16T20:50:05.547 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051276744999999999, 2024-09-16T20:50:05.548 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:05.548 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:05.548 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:05.548 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:05.548 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:05.548 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:05.548 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3731 2024-09-16T20:50:05.548 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.548 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:05.548 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.548 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:05.548 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:50:05.548 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.548 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.549 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.549 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:05.549 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:50:05.549 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.549 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.549 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.549 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:05.549 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:50:05.549 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9900000000000001e-07 2024-09-16T20:50:05.549 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.549 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.549 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:05.549 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639249+0000", 2024-09-16T20:50:05.549 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3630000000000001e-06 2024-09-16T20:50:05.550 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.550 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.550 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:05.550 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639255+0000", 2024-09-16T20:50:05.550 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4960000000000001e-06 2024-09-16T20:50:05.550 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.550 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.550 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:05.550 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639261+0000", 2024-09-16T20:50:05.550 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2839999999999997e-06 2024-09-16T20:50:05.550 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.550 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.550 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:05.550 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690375+0000", 2024-09-16T20:50:05.550 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051114684000000001 2024-09-16T20:50:05.551 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.551 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.551 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:05.551 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690425+0000", 2024-09-16T20:50:05.551 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9379999999999998e-05 2024-09-16T20:50:05.551 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.551 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.551 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:05.551 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690523+0000", 2024-09-16T20:50:05.551 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8139000000000003e-05 2024-09-16T20:50:05.551 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.551 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:05.551 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.551 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.552 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.552 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3732 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1554432~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:05.552 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:50:05.552 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 102.86279699000001, 2024-09-16T20:50:05.552 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051380096, 2024-09-16T20:50:05.552 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:05.552 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:05.552 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:05.552 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:05.552 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:05.552 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:05.552 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3732 2024-09-16T20:50:05.552 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.552 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:05.553 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.553 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:05.553 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:50:05.553 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.553 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.553 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.553 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:05.553 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:50:05.553 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.553 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.553 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.553 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:05.553 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:50:05.554 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6100000000000001e-07 2024-09-16T20:50:05.554 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.554 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.554 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:05.554 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639295+0000", 2024-09-16T20:50:05.554 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.447e-06 2024-09-16T20:50:05.554 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.554 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.554 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:05.554 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639302+0000", 2024-09-16T20:50:05.554 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0570000000000004e-06 2024-09-16T20:50:05.554 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.554 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.554 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:05.555 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639307+0000", 2024-09-16T20:50:05.555 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1429999999999997e-06 2024-09-16T20:50:05.555 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.555 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.555 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:05.555 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690555+0000", 2024-09-16T20:50:05.555 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051247963000000001 2024-09-16T20:50:05.555 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.555 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.555 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:05.555 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690602+0000", 2024-09-16T20:50:05.555 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7602000000000003e-05 2024-09-16T20:50:05.555 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.555 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.556 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:05.556 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690672+0000", 2024-09-16T20:50:05.556 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9423000000000003e-05 2024-09-16T20:50:05.556 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.556 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:05.556 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.556 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.556 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.556 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3734 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1562624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:05.556 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:50:05.556 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 102.86277534, 2024-09-16T20:50:05.556 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051490610999999999, 2024-09-16T20:50:05.556 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:05.557 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:05.557 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:05.557 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:05.557 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:05.557 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:05.557 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3734 2024-09-16T20:50:05.557 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.557 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:05.557 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.557 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:05.557 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:50:05.557 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.557 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.557 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.558 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:05.558 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:50:05.558 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.558 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.558 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.558 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:05.558 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639314+0000", 2024-09-16T20:50:05.558 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4400000000000001e-07 2024-09-16T20:50:05.558 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.558 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.558 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:05.558 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639316+0000", 2024-09-16T20:50:05.558 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1459999999999999e-06 2024-09-16T20:50:05.559 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.559 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.559 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:05.559 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639322+0000", 2024-09-16T20:50:05.559 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7610000000000004e-06 2024-09-16T20:50:05.559 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.559 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.559 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:05.559 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639327+0000", 2024-09-16T20:50:05.559 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.186e-06 2024-09-16T20:50:05.559 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.559 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.559 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:05.559 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690698+0000", 2024-09-16T20:50:05.559 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051371223000000001 2024-09-16T20:50:05.560 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.560 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.560 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:05.560 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690748+0000", 2024-09-16T20:50:05.560 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0192999999999999e-05 2024-09-16T20:50:05.560 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.560 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.560 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:05.560 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690804+0000", 2024-09-16T20:50:05.560 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5658000000000001e-05 2024-09-16T20:50:05.560 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.560 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:05.560 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.561 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.561 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.561 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3736 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1570816~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:05.561 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:50:05.561 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 102.862756081, 2024-09-16T20:50:05.561 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051596528000000003, 2024-09-16T20:50:05.561 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:05.561 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:05.561 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:05.561 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:05.561 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:05.561 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:05.561 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3736 2024-09-16T20:50:05.561 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.562 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:05.562 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.562 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:05.562 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:50:05.562 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.562 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.562 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.562 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:05.562 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:50:05.562 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.562 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.562 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.562 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:05.562 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:50:05.563 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9999999999999999e-07 2024-09-16T20:50:05.563 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.563 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.563 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:05.563 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639335+0000", 2024-09-16T20:50:05.563 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.635e-06 2024-09-16T20:50:05.563 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.563 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.563 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:05.563 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639340+0000", 2024-09-16T20:50:05.563 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5840000000000002e-06 2024-09-16T20:50:05.563 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.563 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.563 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:05.564 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639346+0000", 2024-09-16T20:50:05.564 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1190000000000002e-06 2024-09-16T20:50:05.564 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.564 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.564 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:05.564 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690817+0000", 2024-09-16T20:50:05.564 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051470721999999997 2024-09-16T20:50:05.564 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.564 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.564 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:05.564 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690875+0000", 2024-09-16T20:50:05.564 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7441000000000001e-05 2024-09-16T20:50:05.564 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.564 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.565 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:05.565 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690929+0000", 2024-09-16T20:50:05.565 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4727e-05 2024-09-16T20:50:05.565 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.565 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:05.565 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.565 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.565 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.565 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5434 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:05.565 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:50:05.565 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 102.637494515, 2024-09-16T20:50:05.565 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099755831000000003, 2024-09-16T20:50:05.565 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:05.565 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:05.566 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:05.566 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:05.566 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:05.566 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:05.566 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5434 2024-09-16T20:50:05.566 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.566 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:05.566 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.566 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:05.566 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:50:05.566 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.566 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.566 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.566 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:05.566 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:50:05.567 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.567 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.567 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.567 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:05.567 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864595+0000", 2024-09-16T20:50:05.567 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1099999999999995e-07 2024-09-16T20:50:05.567 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.567 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.567 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:05.567 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864597+0000", 2024-09-16T20:50:05.567 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3989999999999998e-06 2024-09-16T20:50:05.567 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.567 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.568 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:05.568 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864603+0000", 2024-09-16T20:50:05.568 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9309999999999996e-06 2024-09-16T20:50:05.568 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.568 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.568 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:05.568 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864607+0000", 2024-09-16T20:50:05.568 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6200000000000001e-06 2024-09-16T20:50:05.568 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.568 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.568 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:05.568 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964164+0000", 2024-09-16T20:50:05.568 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099557162000000005 2024-09-16T20:50:05.569 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.569 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.569 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:05.569 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964244+0000", 2024-09-16T20:50:05.569 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9597000000000004e-05 2024-09-16T20:50:05.569 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.569 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.569 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:05.569 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964350+0000", 2024-09-16T20:50:05.569 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010641099999999999 2024-09-16T20:50:05.569 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.569 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:05.569 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.569 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.570 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.570 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5435 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 20480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:05.570 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:50:05.570 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 102.637464693, 2024-09-16T20:50:05.570 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100897102, 2024-09-16T20:50:05.570 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:05.570 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:05.570 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:05.570 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:05.570 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:05.570 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:05.570 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5435 2024-09-16T20:50:05.570 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.571 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:05.571 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.571 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:05.571 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:50:05.571 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.571 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.571 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.571 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:05.571 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:50:05.571 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.571 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.571 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.571 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:05.571 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:50:05.572 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8000000000000001e-07 2024-09-16T20:50:05.572 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.572 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.572 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:05.572 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864626+0000", 2024-09-16T20:50:05.572 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.745e-06 2024-09-16T20:50:05.572 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.572 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.572 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:05.572 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864633+0000", 2024-09-16T20:50:05.572 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3339999999999998e-06 2024-09-16T20:50:05.572 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.572 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.573 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:05.573 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864636+0000", 2024-09-16T20:50:05.573 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1200000000000002e-06 2024-09-16T20:50:05.573 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.573 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.573 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:05.573 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965299+0000", 2024-09-16T20:50:05.573 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10066352000000001 2024-09-16T20:50:05.573 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.573 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.574 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:05.574 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965390+0000", 2024-09-16T20:50:05.574 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0894000000000001e-05 2024-09-16T20:50:05.574 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.574 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.574 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:05.574 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965521+0000", 2024-09-16T20:50:05.574 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013110899999999999 2024-09-16T20:50:05.574 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.574 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:05.574 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.575 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.575 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.575 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:152 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 327680~32768 out=32768b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:50:05.575 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:50:05.575 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 101.338967751, 2024-09-16T20:50:05.575 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11869405800000001, 2024-09-16T20:50:05.575 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:05.575 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:05.575 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:05.575 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:05.575 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:50:05.575 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:50:05.575 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 152 2024-09-16T20:50:05.576 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.576 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:05.576 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.576 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:05.576 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:50:05.576 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.576 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.576 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.576 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:05.576 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:50:05.576 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.576 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.576 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.576 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:05.577 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163122+0000", 2024-09-16T20:50:05.577 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6400000000000003e-07 2024-09-16T20:50:05.577 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.577 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.577 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:05.577 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163124+0000", 2024-09-16T20:50:05.577 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.565e-06 2024-09-16T20:50:05.577 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.577 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.577 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:05.577 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163136+0000", 2024-09-16T20:50:05.577 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1717e-05 2024-09-16T20:50:05.577 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.578 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.578 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:05.578 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163142+0000", 2024-09-16T20:50:05.578 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1750000000000002e-06 2024-09-16T20:50:05.578 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.578 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.578 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:05.578 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163606+0000", 2024-09-16T20:50:05.578 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00046418799999999998 2024-09-16T20:50:05.578 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.578 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.578 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:05.578 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163656+0000", 2024-09-16T20:50:05.578 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9852999999999999e-05 2024-09-16T20:50:05.579 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.579 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.579 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:05.579 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281815+0000", 2024-09-16T20:50:05.579 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11815909600000001 2024-09-16T20:50:05.579 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.579 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:05.579 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.579 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.579 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.579 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:153 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:50:05.579 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:50:05.579 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 101.338934798, 2024-09-16T20:50:05.580 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11903462400000001, 2024-09-16T20:50:05.580 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:05.580 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:05.580 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:05.580 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:05.580 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:50:05.580 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:50:05.580 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 153 2024-09-16T20:50:05.580 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.580 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:05.580 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.580 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:05.580 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:50:05.581 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.581 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.581 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.581 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:05.581 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:50:05.581 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.581 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.581 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.581 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:05.581 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163155+0000", 2024-09-16T20:50:05.581 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.37e-07 2024-09-16T20:50:05.581 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.581 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.581 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:05.582 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163157+0000", 2024-09-16T20:50:05.582 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6010000000000002e-06 2024-09-16T20:50:05.582 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.582 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.582 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:05.582 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163165+0000", 2024-09-16T20:50:05.582 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2619999999999998e-06 2024-09-16T20:50:05.582 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.582 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.582 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:05.582 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163171+0000", 2024-09-16T20:50:05.582 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.711e-06 2024-09-16T20:50:05.582 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.582 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.583 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:05.583 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281868+0000", 2024-09-16T20:50:05.583 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118696622 2024-09-16T20:50:05.583 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.583 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.583 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:05.583 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281969+0000", 2024-09-16T20:50:05.583 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101371 2024-09-16T20:50:05.583 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.583 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.583 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:05.583 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282189+0000", 2024-09-16T20:50:05.583 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00021952000000000001 2024-09-16T20:50:05.583 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.584 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:05.584 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.584 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.584 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.584 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:154 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 360448~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:50:05.584 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:50:05.584 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 101.338903026, 2024-09-16T20:50:05.584 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11932035100000001, 2024-09-16T20:50:05.584 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:05.584 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:05.584 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:05.584 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:05.584 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:50:05.584 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:50:05.585 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 154 2024-09-16T20:50:05.585 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.585 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:05.585 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.585 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:05.585 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:50:05.585 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.585 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.585 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.585 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:05.585 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:50:05.585 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.585 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.585 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.586 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:05.586 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:50:05.586 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:50:05.586 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.586 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.586 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:05.586 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163189+0000", 2024-09-16T20:50:05.586 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6840000000000001e-06 2024-09-16T20:50:05.586 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.586 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.586 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:05.586 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163196+0000", 2024-09-16T20:50:05.586 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7920000000000004e-06 2024-09-16T20:50:05.586 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.587 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.587 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:05.587 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163201+0000", 2024-09-16T20:50:05.587 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2809999999999999e-06 2024-09-16T20:50:05.587 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.587 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.587 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:05.587 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282219+0000", 2024-09-16T20:50:05.587 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.119018343 2024-09-16T20:50:05.587 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.587 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.587 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:05.587 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282305+0000", 2024-09-16T20:50:05.587 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5865000000000005e-05 2024-09-16T20:50:05.588 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.588 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.588 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:05.588 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282506+0000", 2024-09-16T20:50:05.588 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000200927 2024-09-16T20:50:05.588 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.588 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:05.588 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.588 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.588 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.588 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop(client.4366.0:3088 3.1e e38/23 3:79e5ec01:::rbd_header.110e891a9b2b:head v 38'306, mlcod=36'304)", 2024-09-16T20:50:05.588 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:50:05.588 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 101.33788415799999, 2024-09-16T20:50:05.588 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118102759, 2024-09-16T20:50:05.589 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:05.589 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:05.589 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:05.589 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:05.589 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.589 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:05.589 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:50:05.589 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.589 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.589 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.589 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:05.589 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:50:05.589 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.590 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.590 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.590 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:05.590 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:50:05.590 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.85e-07 2024-09-16T20:50:05.590 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.590 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.590 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:05.590 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164212+0000", 2024-09-16T20:50:05.590 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3999999999999997e-06 2024-09-16T20:50:05.590 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.590 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.590 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:05.590 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281553+0000", 2024-09-16T20:50:05.590 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11734135499999999 2024-09-16T20:50:05.591 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.591 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.591 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:05.591 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281567+0000", 2024-09-16T20:50:05.591 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4188999999999999e-05 2024-09-16T20:50:05.591 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.591 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.591 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:05.591 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281653+0000", 2024-09-16T20:50:05.591 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5385999999999999e-05 2024-09-16T20:50:05.591 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.591 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.591 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:05.592 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281679+0000", 2024-09-16T20:50:05.592 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6689000000000001e-05 2024-09-16T20:50:05.592 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.592 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.592 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:05.592 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282277+0000", 2024-09-16T20:50:05.592 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00059798000000000002 2024-09-16T20:50:05.592 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.592 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.592 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:05.592 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282307+0000", 2024-09-16T20:50:05.592 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9975e-05 2024-09-16T20:50:05.592 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.592 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:05.593 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.593 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.593 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.593 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5255.0:44 3.16 3:6f5476df:::rbd_header.110e25db1f60:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:50:05.593 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:50:05.593 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 29.181581102999999, 2024-09-16T20:50:05.593 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.121007509, 2024-09-16T20:50:05.593 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:05.593 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:05.593 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:05.593 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:05.593 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5255", 2024-09-16T20:50:05.593 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:50:05.593 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 44 2024-09-16T20:50:05.594 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.594 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:05.594 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.594 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:05.594 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:50:05.594 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.594 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.594 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.594 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:05.594 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:50:05.594 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.594 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.594 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.594 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:05.595 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:50:05.595 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0500000000000003e-07 2024-09-16T20:50:05.595 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.595 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.595 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:05.595 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320512+0000", 2024-09-16T20:50:05.595 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2540000000000001e-06 2024-09-16T20:50:05.595 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.595 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.595 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:05.595 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320518+0000", 2024-09-16T20:50:05.595 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1689999999999997e-06 2024-09-16T20:50:05.595 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.595 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.596 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:05.596 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320521+0000", 2024-09-16T20:50:05.596 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5200000000000002e-06 2024-09-16T20:50:05.596 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.596 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.596 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:05.596 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320548+0000", 2024-09-16T20:50:05.596 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6662999999999999e-05 2024-09-16T20:50:05.596 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.596 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.596 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:05.596 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320587+0000", 2024-09-16T20:50:05.596 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8866999999999999e-05 2024-09-16T20:50:05.596 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.597 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.597 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:05.597 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.441515+0000", 2024-09-16T20:50:05.597 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.120928331 2024-09-16T20:50:05.597 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.597 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:05.597 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.597 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.597 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.597 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:20329 3.5 3:a4ea64b7:::rbd_header.110ede5ef857:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:05.597 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:01.061694+0000", 2024-09-16T20:50:05.597 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.4403945350000003, 2024-09-16T20:50:05.597 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.071234077000000007, 2024-09-16T20:50:05.597 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:05.598 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:05.598 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:05.598 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:05.598 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:50:05.598 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:05.598 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 20329 2024-09-16T20:50:05.598 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.598 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:05.598 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.598 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:05.598 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.061694+0000", 2024-09-16T20:50:05.599 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.599 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.599 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.599 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:05.599 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.061694+0000", 2024-09-16T20:50:05.599 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:05.600 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.600 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.600 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:05.600 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089232+0000", 2024-09-16T20:50:05.600 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.027537275 2024-09-16T20:50:05.600 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.601 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.601 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:05.601 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089237+0000", 2024-09-16T20:50:05.601 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4709999999999998e-06 2024-09-16T20:50:05.601 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.601 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.601 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:05.602 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089246+0000", 2024-09-16T20:50:05.602 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8230000000000001e-06 2024-09-16T20:50:05.602 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.602 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.602 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:05.602 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089252+0000", 2024-09-16T20:50:05.602 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3060000000000003e-06 2024-09-16T20:50:05.603 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.603 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.603 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:05.603 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.132756+0000", 2024-09-16T20:50:05.603 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.043504334999999998 2024-09-16T20:50:05.603 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.603 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.603 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:05.603 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.132829+0000", 2024-09-16T20:50:05.603 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2849999999999995e-05 2024-09-16T20:50:05.603 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:05.604 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:05.604 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:05.604 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.132928+0000", 2024-09-16T20:50:05.604 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.9017000000000004e-05 2024-09-16T20:50:05.604 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.604 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:05.604 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.604 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:05.604 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:05.604 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:05.606 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:50:05.606 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:05.604+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:05.707 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:05.707+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:05.730 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:05.730 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:50:05.730 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:50:05.730 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:05.740 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:50:05.746 INFO:teuthology.orchestra.run.smithi139.stdout:truncating image image_client.0-clone38 from 0xb88ca00 (overlap 0x287e800) to 0x0 2024-09-16T20:50:05.747 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T20:50:05.808 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:05.808+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:05.813 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 3, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-16T20:50:05.813 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T20:50:05.814 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-16T20:50:05.814 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-16T20:50:05.859 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:05.860 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:50:05.860 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:50:05.860 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:50:05.860 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:05.869 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:50:05.883 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:50:05.883 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:50:05.902 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:50:05.902 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:50:05.912 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:05.911+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:05.918 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:50:05.919 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:50:05.938 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:50:05.938 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:50:05.955 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:50:05.955 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:50:05.972 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:50:05.972 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:50:05.981 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:05.981 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:05.990 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:50:06.009 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:06.009+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:06.068 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #37, image image_client.0-clone37 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent38 2024-09-16T20:50:06.110 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:06.110+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:06.127 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:50:06.127 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:06.127 INFO:teuthology.orchestra.run.smithi029.stdout: "epoch": 122, 2024-09-16T20:50:06.127 INFO:teuthology.orchestra.run.smithi029.stdout: "fsid": "e28af4a7-5fcd-4de8-8f77-a2b575361bb1", 2024-09-16T20:50:06.128 INFO:teuthology.orchestra.run.smithi029.stdout: "created": "2024-09-16T20:47:39.760609+0000", 2024-09-16T20:50:06.128 INFO:teuthology.orchestra.run.smithi029.stdout: "modified": "2024-09-16T20:50:05.658012+0000", 2024-09-16T20:50:06.128 INFO:teuthology.orchestra.run.smithi029.stdout: "last_up_change": "2024-09-16T20:47:50.238013+0000", 2024-09-16T20:50:06.128 INFO:teuthology.orchestra.run.smithi029.stdout: "last_in_change": "2024-09-16T20:49:08.779548+0000", 2024-09-16T20:50:06.128 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-16T20:50:06.128 INFO:teuthology.orchestra.run.smithi029.stdout: "flags_num": 5799936, 2024-09-16T20:50:06.128 INFO:teuthology.orchestra.run.smithi029.stdout: "flags_set": [ 2024-09-16T20:50:06.128 INFO:teuthology.orchestra.run.smithi029.stdout: "pglog_hardlimit", 2024-09-16T20:50:06.128 INFO:teuthology.orchestra.run.smithi029.stdout: "purged_snapdirs", 2024-09-16T20:50:06.128 INFO:teuthology.orchestra.run.smithi029.stdout: "recovery_deletes", 2024-09-16T20:50:06.129 INFO:teuthology.orchestra.run.smithi029.stdout: "sortbitwise" 2024-09-16T20:50:06.129 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-16T20:50:06.129 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_version": 5, 2024-09-16T20:50:06.129 INFO:teuthology.orchestra.run.smithi029.stdout: "full_ratio": 0.94999998807907104, 2024-09-16T20:50:06.129 INFO:teuthology.orchestra.run.smithi029.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-16T20:50:06.129 INFO:teuthology.orchestra.run.smithi029.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-16T20:50:06.129 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_snapshot": "", 2024-09-16T20:50:06.129 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_max": 3, 2024-09-16T20:50:06.129 INFO:teuthology.orchestra.run.smithi029.stdout: "max_osd": 6, 2024-09-16T20:50:06.129 INFO:teuthology.orchestra.run.smithi029.stdout: "require_min_compat_client": "luminous", 2024-09-16T20:50:06.130 INFO:teuthology.orchestra.run.smithi029.stdout: "min_compat_client": "luminous", 2024-09-16T20:50:06.130 INFO:teuthology.orchestra.run.smithi029.stdout: "require_osd_release": "squid", 2024-09-16T20:50:06.130 INFO:teuthology.orchestra.run.smithi029.stdout: "allow_crimson": false, 2024-09-16T20:50:06.130 INFO:teuthology.orchestra.run.smithi029.stdout: "pools": [ 2024-09-16T20:50:06.130 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.130 INFO:teuthology.orchestra.run.smithi029.stdout: "pool": 1, 2024-09-16T20:50:06.130 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_name": ".mgr", 2024-09-16T20:50:06.130 INFO:teuthology.orchestra.run.smithi029.stdout: "create_time": "2024-09-16T20:47:52.220750+0000", 2024-09-16T20:50:06.130 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-16T20:50:06.130 INFO:teuthology.orchestra.run.smithi029.stdout: "flags_names": "hashpspool", 2024-09-16T20:50:06.131 INFO:teuthology.orchestra.run.smithi029.stdout: "type": 1, 2024-09-16T20:50:06.131 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 2, 2024-09-16T20:50:06.131 INFO:teuthology.orchestra.run.smithi029.stdout: "min_size": 1, 2024-09-16T20:50:06.131 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_rule": 0, 2024-09-16T20:50:06.131 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_count": 0, 2024-09-16T20:50:06.131 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_target": 0, 2024-09-16T20:50:06.131 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T20:50:06.131 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T20:50:06.131 INFO:teuthology.orchestra.run.smithi029.stdout: "is_stretch_pool": false, 2024-09-16T20:50:06.131 INFO:teuthology.orchestra.run.smithi029.stdout: "object_hash": 2, 2024-09-16T20:50:06.132 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_autoscale_mode": "off", 2024-09-16T20:50:06.132 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num": 1, 2024-09-16T20:50:06.132 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num": 1, 2024-09-16T20:50:06.132 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num_target": 1, 2024-09-16T20:50:06.132 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_target": 1, 2024-09-16T20:50:06.132 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_pending": 1, 2024-09-16T20:50:06.132 INFO:teuthology.orchestra.run.smithi029.stdout: "last_pg_merge_meta": { 2024-09-16T20:50:06.132 INFO:teuthology.orchestra.run.smithi029.stdout: "source_pgid": "0.0", 2024-09-16T20:50:06.132 INFO:teuthology.orchestra.run.smithi029.stdout: "ready_epoch": 0, 2024-09-16T20:50:06.132 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_started": 0, 2024-09-16T20:50:06.132 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_clean": 0, 2024-09-16T20:50:06.132 INFO:teuthology.orchestra.run.smithi029.stdout: "source_version": "0'0", 2024-09-16T20:50:06.133 INFO:teuthology.orchestra.run.smithi029.stdout: "target_version": "0'0" 2024-09-16T20:50:06.133 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.133 INFO:teuthology.orchestra.run.smithi029.stdout: "last_change": "17", 2024-09-16T20:50:06.133 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend": "0", 2024-09-16T20:50:06.133 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T20:50:06.133 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T20:50:06.133 INFO:teuthology.orchestra.run.smithi029.stdout: "auid": 0, 2024-09-16T20:50:06.133 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_mode": "selfmanaged", 2024-09-16T20:50:06.133 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_seq": 0, 2024-09-16T20:50:06.133 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_epoch": 0, 2024-09-16T20:50:06.134 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_snaps": [], 2024-09-16T20:50:06.134 INFO:teuthology.orchestra.run.smithi029.stdout: "removed_snaps": "[]", 2024-09-16T20:50:06.134 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_bytes": 0, 2024-09-16T20:50:06.134 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_objects": 0, 2024-09-16T20:50:06.134 INFO:teuthology.orchestra.run.smithi029.stdout: "tiers": [], 2024-09-16T20:50:06.134 INFO:teuthology.orchestra.run.smithi029.stdout: "tier_of": -1, 2024-09-16T20:50:06.134 INFO:teuthology.orchestra.run.smithi029.stdout: "read_tier": -1, 2024-09-16T20:50:06.134 INFO:teuthology.orchestra.run.smithi029.stdout: "write_tier": -1, 2024-09-16T20:50:06.134 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_mode": "none", 2024-09-16T20:50:06.134 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_bytes": 0, 2024-09-16T20:50:06.134 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_objects": 0, 2024-09-16T20:50:06.135 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T20:50:06.135 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T20:50:06.135 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T20:50:06.135 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_flush_age": 0, 2024-09-16T20:50:06.135 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_evict_age": 0, 2024-09-16T20:50:06.135 INFO:teuthology.orchestra.run.smithi029.stdout: "erasure_code_profile": "", 2024-09-16T20:50:06.135 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_params": { 2024-09-16T20:50:06.135 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "none" 2024-09-16T20:50:06.135 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.135 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_period": 0, 2024-09-16T20:50:06.135 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_count": 0, 2024-09-16T20:50:06.135 INFO:teuthology.orchestra.run.smithi029.stdout: "use_gmt_hitset": true, 2024-09-16T20:50:06.136 INFO:teuthology.orchestra.run.smithi029.stdout: "min_read_recency_for_promote": 0, 2024-09-16T20:50:06.136 INFO:teuthology.orchestra.run.smithi029.stdout: "min_write_recency_for_promote": 0, 2024-09-16T20:50:06.136 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T20:50:06.136 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_search_last_n": 0, 2024-09-16T20:50:06.136 INFO:teuthology.orchestra.run.smithi029.stdout: "grade_table": [], 2024-09-16T20:50:06.136 INFO:teuthology.orchestra.run.smithi029.stdout: "stripe_width": 0, 2024-09-16T20:50:06.136 INFO:teuthology.orchestra.run.smithi029.stdout: "expected_num_objects": 0, 2024-09-16T20:50:06.136 INFO:teuthology.orchestra.run.smithi029.stdout: "fast_read": false, 2024-09-16T20:50:06.136 INFO:teuthology.orchestra.run.smithi029.stdout: "options": { 2024-09-16T20:50:06.136 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_max": 32, 2024-09-16T20:50:06.136 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_min": 1 2024-09-16T20:50:06.137 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.137 INFO:teuthology.orchestra.run.smithi029.stdout: "application_metadata": { 2024-09-16T20:50:06.137 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": {} 2024-09-16T20:50:06.137 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.137 INFO:teuthology.orchestra.run.smithi029.stdout: "read_balance": { 2024-09-16T20:50:06.137 INFO:teuthology.orchestra.run.smithi029.stdout: "score_type": "Fair distribution", 2024-09-16T20:50:06.137 INFO:teuthology.orchestra.run.smithi029.stdout: "score_acting": 6.059999942779541, 2024-09-16T20:50:06.137 INFO:teuthology.orchestra.run.smithi029.stdout: "score_stable": 6.059999942779541, 2024-09-16T20:50:06.137 INFO:teuthology.orchestra.run.smithi029.stdout: "optimal_score": 0.33000001311302185, 2024-09-16T20:50:06.137 INFO:teuthology.orchestra.run.smithi029.stdout: "raw_score_acting": 2, 2024-09-16T20:50:06.137 INFO:teuthology.orchestra.run.smithi029.stdout: "raw_score_stable": 2, 2024-09-16T20:50:06.138 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity_weighted": 1, 2024-09-16T20:50:06.138 INFO:teuthology.orchestra.run.smithi029.stdout: "average_primary_affinity": 1, 2024-09-16T20:50:06.138 INFO:teuthology.orchestra.run.smithi029.stdout: "average_primary_affinity_weighted": 1 2024-09-16T20:50:06.138 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:06.138 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.138 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.138 INFO:teuthology.orchestra.run.smithi029.stdout: "pool": 2, 2024-09-16T20:50:06.138 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_name": "rbd", 2024-09-16T20:50:06.138 INFO:teuthology.orchestra.run.smithi029.stdout: "create_time": "2024-09-16T20:47:56.756666+0000", 2024-09-16T20:50:06.138 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 8193, 2024-09-16T20:50:06.139 INFO:teuthology.orchestra.run.smithi029.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T20:50:06.139 INFO:teuthology.orchestra.run.smithi029.stdout: "type": 1, 2024-09-16T20:50:06.139 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 2, 2024-09-16T20:50:06.139 INFO:teuthology.orchestra.run.smithi029.stdout: "min_size": 1, 2024-09-16T20:50:06.139 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_rule": 0, 2024-09-16T20:50:06.139 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_count": 0, 2024-09-16T20:50:06.139 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_target": 0, 2024-09-16T20:50:06.139 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T20:50:06.139 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T20:50:06.140 INFO:teuthology.orchestra.run.smithi029.stdout: "is_stretch_pool": false, 2024-09-16T20:50:06.140 INFO:teuthology.orchestra.run.smithi029.stdout: "object_hash": 2, 2024-09-16T20:50:06.140 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_autoscale_mode": "off", 2024-09-16T20:50:06.140 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num": 8, 2024-09-16T20:50:06.140 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num": 8, 2024-09-16T20:50:06.141 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num_target": 8, 2024-09-16T20:50:06.141 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_target": 8, 2024-09-16T20:50:06.141 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_pending": 8, 2024-09-16T20:50:06.141 INFO:teuthology.orchestra.run.smithi029.stdout: "last_pg_merge_meta": { 2024-09-16T20:50:06.141 INFO:teuthology.orchestra.run.smithi029.stdout: "source_pgid": "0.0", 2024-09-16T20:50:06.141 INFO:teuthology.orchestra.run.smithi029.stdout: "ready_epoch": 0, 2024-09-16T20:50:06.141 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_started": 0, 2024-09-16T20:50:06.141 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_clean": 0, 2024-09-16T20:50:06.141 INFO:teuthology.orchestra.run.smithi029.stdout: "source_version": "0'0", 2024-09-16T20:50:06.141 INFO:teuthology.orchestra.run.smithi029.stdout: "target_version": "0'0" 2024-09-16T20:50:06.142 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.142 INFO:teuthology.orchestra.run.smithi029.stdout: "last_change": "21", 2024-09-16T20:50:06.142 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend": "0", 2024-09-16T20:50:06.142 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T20:50:06.142 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T20:50:06.142 INFO:teuthology.orchestra.run.smithi029.stdout: "auid": 0, 2024-09-16T20:50:06.142 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_mode": "selfmanaged", 2024-09-16T20:50:06.142 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_seq": 2, 2024-09-16T20:50:06.142 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_epoch": 21, 2024-09-16T20:50:06.142 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_snaps": [], 2024-09-16T20:50:06.143 INFO:teuthology.orchestra.run.smithi029.stdout: "removed_snaps": "[]", 2024-09-16T20:50:06.143 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_bytes": 0, 2024-09-16T20:50:06.143 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_objects": 0, 2024-09-16T20:50:06.143 INFO:teuthology.orchestra.run.smithi029.stdout: "tiers": [], 2024-09-16T20:50:06.143 INFO:teuthology.orchestra.run.smithi029.stdout: "tier_of": -1, 2024-09-16T20:50:06.143 INFO:teuthology.orchestra.run.smithi029.stdout: "read_tier": -1, 2024-09-16T20:50:06.143 INFO:teuthology.orchestra.run.smithi029.stdout: "write_tier": -1, 2024-09-16T20:50:06.143 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_mode": "none", 2024-09-16T20:50:06.143 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_bytes": 0, 2024-09-16T20:50:06.143 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_objects": 0, 2024-09-16T20:50:06.144 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T20:50:06.144 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T20:50:06.144 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T20:50:06.144 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_flush_age": 0, 2024-09-16T20:50:06.144 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_evict_age": 0, 2024-09-16T20:50:06.144 INFO:teuthology.orchestra.run.smithi029.stdout: "erasure_code_profile": "", 2024-09-16T20:50:06.144 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_params": { 2024-09-16T20:50:06.144 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "none" 2024-09-16T20:50:06.144 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.144 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_period": 0, 2024-09-16T20:50:06.144 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_count": 0, 2024-09-16T20:50:06.145 INFO:teuthology.orchestra.run.smithi029.stdout: "use_gmt_hitset": true, 2024-09-16T20:50:06.145 INFO:teuthology.orchestra.run.smithi029.stdout: "min_read_recency_for_promote": 0, 2024-09-16T20:50:06.145 INFO:teuthology.orchestra.run.smithi029.stdout: "min_write_recency_for_promote": 0, 2024-09-16T20:50:06.145 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T20:50:06.145 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_search_last_n": 0, 2024-09-16T20:50:06.145 INFO:teuthology.orchestra.run.smithi029.stdout: "grade_table": [], 2024-09-16T20:50:06.145 INFO:teuthology.orchestra.run.smithi029.stdout: "stripe_width": 0, 2024-09-16T20:50:06.145 INFO:teuthology.orchestra.run.smithi029.stdout: "expected_num_objects": 0, 2024-09-16T20:50:06.145 INFO:teuthology.orchestra.run.smithi029.stdout: "fast_read": false, 2024-09-16T20:50:06.145 INFO:teuthology.orchestra.run.smithi029.stdout: "options": {}, 2024-09-16T20:50:06.146 INFO:teuthology.orchestra.run.smithi029.stdout: "application_metadata": { 2024-09-16T20:50:06.146 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd": {} 2024-09-16T20:50:06.146 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.146 INFO:teuthology.orchestra.run.smithi029.stdout: "read_balance": { 2024-09-16T20:50:06.146 INFO:teuthology.orchestra.run.smithi029.stdout: "score_type": "Fair distribution", 2024-09-16T20:50:06.146 INFO:teuthology.orchestra.run.smithi029.stdout: "score_acting": 2.25, 2024-09-16T20:50:06.146 INFO:teuthology.orchestra.run.smithi029.stdout: "score_stable": 2.25, 2024-09-16T20:50:06.146 INFO:teuthology.orchestra.run.smithi029.stdout: "optimal_score": 1, 2024-09-16T20:50:06.146 INFO:teuthology.orchestra.run.smithi029.stdout: "raw_score_acting": 2.25, 2024-09-16T20:50:06.146 INFO:teuthology.orchestra.run.smithi029.stdout: "raw_score_stable": 2.25, 2024-09-16T20:50:06.146 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity_weighted": 1, 2024-09-16T20:50:06.147 INFO:teuthology.orchestra.run.smithi029.stdout: "average_primary_affinity": 1, 2024-09-16T20:50:06.147 INFO:teuthology.orchestra.run.smithi029.stdout: "average_primary_affinity_weighted": 1 2024-09-16T20:50:06.147 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:06.147 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.147 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.147 INFO:teuthology.orchestra.run.smithi029.stdout: "pool": 3, 2024-09-16T20:50:06.147 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_name": "pool_client.0", 2024-09-16T20:50:06.147 INFO:teuthology.orchestra.run.smithi029.stdout: "create_time": "2024-09-16T20:48:06.418889+0000", 2024-09-16T20:50:06.147 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 8193, 2024-09-16T20:50:06.147 INFO:teuthology.orchestra.run.smithi029.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T20:50:06.148 INFO:teuthology.orchestra.run.smithi029.stdout: "type": 1, 2024-09-16T20:50:06.148 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 2, 2024-09-16T20:50:06.148 INFO:teuthology.orchestra.run.smithi029.stdout: "min_size": 1, 2024-09-16T20:50:06.148 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_rule": 0, 2024-09-16T20:50:06.148 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_count": 0, 2024-09-16T20:50:06.148 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_target": 0, 2024-09-16T20:50:06.148 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T20:50:06.148 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T20:50:06.148 INFO:teuthology.orchestra.run.smithi029.stdout: "is_stretch_pool": false, 2024-09-16T20:50:06.148 INFO:teuthology.orchestra.run.smithi029.stdout: "object_hash": 2, 2024-09-16T20:50:06.148 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_autoscale_mode": "off", 2024-09-16T20:50:06.149 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num": 32, 2024-09-16T20:50:06.149 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num": 32, 2024-09-16T20:50:06.149 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num_target": 32, 2024-09-16T20:50:06.149 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_target": 32, 2024-09-16T20:50:06.149 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_pending": 32, 2024-09-16T20:50:06.149 INFO:teuthology.orchestra.run.smithi029.stdout: "last_pg_merge_meta": { 2024-09-16T20:50:06.149 INFO:teuthology.orchestra.run.smithi029.stdout: "source_pgid": "0.0", 2024-09-16T20:50:06.149 INFO:teuthology.orchestra.run.smithi029.stdout: "ready_epoch": 0, 2024-09-16T20:50:06.149 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_started": 0, 2024-09-16T20:50:06.149 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_clean": 0, 2024-09-16T20:50:06.149 INFO:teuthology.orchestra.run.smithi029.stdout: "source_version": "0'0", 2024-09-16T20:50:06.150 INFO:teuthology.orchestra.run.smithi029.stdout: "target_version": "0'0" 2024-09-16T20:50:06.150 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.150 INFO:teuthology.orchestra.run.smithi029.stdout: "last_change": "122", 2024-09-16T20:50:06.150 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend": "0", 2024-09-16T20:50:06.150 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T20:50:06.150 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T20:50:06.150 INFO:teuthology.orchestra.run.smithi029.stdout: "auid": 0, 2024-09-16T20:50:06.150 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_mode": "selfmanaged", 2024-09-16T20:50:06.150 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_seq": 41, 2024-09-16T20:50:06.150 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_epoch": 122, 2024-09-16T20:50:06.150 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_snaps": [], 2024-09-16T20:50:06.151 INFO:teuthology.orchestra.run.smithi029.stdout: "removed_snaps": "[]", 2024-09-16T20:50:06.151 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_bytes": 0, 2024-09-16T20:50:06.151 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_objects": 0, 2024-09-16T20:50:06.151 INFO:teuthology.orchestra.run.smithi029.stdout: "tiers": [], 2024-09-16T20:50:06.151 INFO:teuthology.orchestra.run.smithi029.stdout: "tier_of": -1, 2024-09-16T20:50:06.151 INFO:teuthology.orchestra.run.smithi029.stdout: "read_tier": -1, 2024-09-16T20:50:06.151 INFO:teuthology.orchestra.run.smithi029.stdout: "write_tier": -1, 2024-09-16T20:50:06.151 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_mode": "none", 2024-09-16T20:50:06.151 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_bytes": 0, 2024-09-16T20:50:06.151 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_objects": 0, 2024-09-16T20:50:06.152 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T20:50:06.152 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T20:50:06.152 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T20:50:06.152 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_flush_age": 0, 2024-09-16T20:50:06.152 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_evict_age": 0, 2024-09-16T20:50:06.152 INFO:teuthology.orchestra.run.smithi029.stdout: "erasure_code_profile": "", 2024-09-16T20:50:06.152 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_params": { 2024-09-16T20:50:06.152 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "none" 2024-09-16T20:50:06.152 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.152 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_period": 0, 2024-09-16T20:50:06.152 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_count": 0, 2024-09-16T20:50:06.153 INFO:teuthology.orchestra.run.smithi029.stdout: "use_gmt_hitset": true, 2024-09-16T20:50:06.153 INFO:teuthology.orchestra.run.smithi029.stdout: "min_read_recency_for_promote": 0, 2024-09-16T20:50:06.153 INFO:teuthology.orchestra.run.smithi029.stdout: "min_write_recency_for_promote": 0, 2024-09-16T20:50:06.153 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T20:50:06.153 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_search_last_n": 0, 2024-09-16T20:50:06.153 INFO:teuthology.orchestra.run.smithi029.stdout: "grade_table": [], 2024-09-16T20:50:06.153 INFO:teuthology.orchestra.run.smithi029.stdout: "stripe_width": 0, 2024-09-16T20:50:06.153 INFO:teuthology.orchestra.run.smithi029.stdout: "expected_num_objects": 0, 2024-09-16T20:50:06.153 INFO:teuthology.orchestra.run.smithi029.stdout: "fast_read": false, 2024-09-16T20:50:06.153 INFO:teuthology.orchestra.run.smithi029.stdout: "options": {}, 2024-09-16T20:50:06.153 INFO:teuthology.orchestra.run.smithi029.stdout: "application_metadata": { 2024-09-16T20:50:06.153 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd": {} 2024-09-16T20:50:06.154 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.154 INFO:teuthology.orchestra.run.smithi029.stdout: "read_balance": { 2024-09-16T20:50:06.154 INFO:teuthology.orchestra.run.smithi029.stdout: "score_type": "Fair distribution", 2024-09-16T20:50:06.154 INFO:teuthology.orchestra.run.smithi029.stdout: "score_acting": 1.309999942779541, 2024-09-16T20:50:06.154 INFO:teuthology.orchestra.run.smithi029.stdout: "score_stable": 1.309999942779541, 2024-09-16T20:50:06.154 INFO:teuthology.orchestra.run.smithi029.stdout: "optimal_score": 1, 2024-09-16T20:50:06.154 INFO:teuthology.orchestra.run.smithi029.stdout: "raw_score_acting": 1.309999942779541, 2024-09-16T20:50:06.154 INFO:teuthology.orchestra.run.smithi029.stdout: "raw_score_stable": 1.309999942779541, 2024-09-16T20:50:06.154 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity_weighted": 1, 2024-09-16T20:50:06.154 INFO:teuthology.orchestra.run.smithi029.stdout: "average_primary_affinity": 1, 2024-09-16T20:50:06.154 INFO:teuthology.orchestra.run.smithi029.stdout: "average_primary_affinity_weighted": 1 2024-09-16T20:50:06.154 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:06.155 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:06.155 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-16T20:50:06.155 INFO:teuthology.orchestra.run.smithi029.stdout: "osds": [ 2024-09-16T20:50:06.155 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.155 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 0, 2024-09-16T20:50:06.155 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "6c3cb2d7-7d23-4566-9ca1-d7b08e12360c", 2024-09-16T20:50:06.155 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-16T20:50:06.155 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-16T20:50:06.155 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-16T20:50:06.155 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-16T20:50:06.155 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-16T20:50:06.155 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-16T20:50:06.155 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 13, 2024-09-16T20:50:06.156 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 111, 2024-09-16T20:50:06.156 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-16T20:50:06.156 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-16T20:50:06.156 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-16T20:50:06.156 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:50:06.156 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.156 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:50:06.156 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6816", 2024-09-16T20:50:06.156 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:50:06.156 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.156 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.157 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:50:06.157 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6817", 2024-09-16T20:50:06.157 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:50:06.157 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:06.157 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:06.157 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.157 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-16T20:50:06.157 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:50:06.157 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.157 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:50:06.157 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6818", 2024-09-16T20:50:06.157 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:50:06.158 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.158 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.158 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:50:06.158 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6819", 2024-09-16T20:50:06.158 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:50:06.158 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:06.158 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:06.158 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.158 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-16T20:50:06.158 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:50:06.158 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.159 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:50:06.159 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6822", 2024-09-16T20:50:06.159 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:50:06.159 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.159 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.159 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:50:06.159 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6823", 2024-09-16T20:50:06.159 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:50:06.159 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:06.159 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:06.160 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.160 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-16T20:50:06.160 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:50:06.160 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.160 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:50:06.160 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6820", 2024-09-16T20:50:06.160 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:50:06.160 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.160 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.160 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:50:06.160 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6821", 2024-09-16T20:50:06.161 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:50:06.161 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:06.161 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:06.161 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.161 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.29:6817/2744025949", 2024-09-16T20:50:06.161 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.29:6819/2744025949", 2024-09-16T20:50:06.161 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.29:6823/2744025949", 2024-09-16T20:50:06.161 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.29:6821/2744025949", 2024-09-16T20:50:06.161 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-16T20:50:06.161 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-16T20:50:06.161 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-16T20:50:06.162 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:06.162 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.162 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.162 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 1, 2024-09-16T20:50:06.162 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "5cf8c20c-1ed2-42ae-a8b0-c88cc4e36bce", 2024-09-16T20:50:06.162 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-16T20:50:06.162 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-16T20:50:06.162 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-16T20:50:06.162 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-16T20:50:06.162 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-16T20:50:06.162 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-16T20:50:06.162 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 13, 2024-09-16T20:50:06.163 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 81, 2024-09-16T20:50:06.163 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-16T20:50:06.163 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-16T20:50:06.163 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-16T20:50:06.163 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:50:06.163 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.163 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:50:06.163 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6800", 2024-09-16T20:50:06.163 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:50:06.163 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.163 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.163 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:50:06.164 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6801", 2024-09-16T20:50:06.164 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:50:06.164 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:06.164 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:06.164 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.164 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-16T20:50:06.164 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:50:06.164 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.164 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:50:06.164 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6802", 2024-09-16T20:50:06.164 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:50:06.164 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.165 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.165 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:50:06.165 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6803", 2024-09-16T20:50:06.165 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:50:06.165 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:06.165 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:06.165 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.165 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-16T20:50:06.165 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:50:06.165 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.165 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:50:06.165 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6806", 2024-09-16T20:50:06.166 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:50:06.166 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.166 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.166 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:50:06.166 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6807", 2024-09-16T20:50:06.166 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:50:06.166 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:06.166 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:06.166 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.166 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-16T20:50:06.166 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:50:06.166 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.167 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:50:06.167 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6804", 2024-09-16T20:50:06.167 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:50:06.167 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.167 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.167 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:50:06.167 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6805", 2024-09-16T20:50:06.167 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:50:06.167 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:06.167 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:06.167 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.167 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.29:6801/208628762", 2024-09-16T20:50:06.167 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.29:6803/208628762", 2024-09-16T20:50:06.168 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.29:6807/208628762", 2024-09-16T20:50:06.168 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.29:6805/208628762", 2024-09-16T20:50:06.168 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-16T20:50:06.168 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-16T20:50:06.168 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-16T20:50:06.168 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:06.168 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.168 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.168 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 2, 2024-09-16T20:50:06.168 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "075d16da-82bf-449b-9f6e-542c9e8d810b", 2024-09-16T20:50:06.168 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-16T20:50:06.169 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-16T20:50:06.169 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-16T20:50:06.169 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-16T20:50:06.169 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-16T20:50:06.169 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-16T20:50:06.169 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 13, 2024-09-16T20:50:06.169 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 102, 2024-09-16T20:50:06.169 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-16T20:50:06.169 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-16T20:50:06.169 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-16T20:50:06.169 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:50:06.170 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.170 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:50:06.170 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6808", 2024-09-16T20:50:06.170 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:50:06.170 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.170 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.170 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:50:06.170 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6809", 2024-09-16T20:50:06.170 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:50:06.170 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:06.170 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:06.171 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.171 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-16T20:50:06.171 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:50:06.171 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.171 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:50:06.171 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6810", 2024-09-16T20:50:06.171 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:50:06.171 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.171 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.171 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:50:06.172 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6811", 2024-09-16T20:50:06.172 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:50:06.172 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:06.172 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:06.172 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.172 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-16T20:50:06.172 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:50:06.172 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.172 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:50:06.172 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6814", 2024-09-16T20:50:06.172 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:50:06.173 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.173 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.173 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:50:06.173 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6815", 2024-09-16T20:50:06.173 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:50:06.173 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:06.173 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:06.173 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.173 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-16T20:50:06.173 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:50:06.173 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.174 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:50:06.174 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6812", 2024-09-16T20:50:06.174 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:50:06.174 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.174 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.174 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:50:06.174 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6813", 2024-09-16T20:50:06.174 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:50:06.174 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:06.174 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:06.175 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.175 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.29:6809/3447426536", 2024-09-16T20:50:06.175 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.29:6811/3447426536", 2024-09-16T20:50:06.175 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.29:6815/3447426536", 2024-09-16T20:50:06.175 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.29:6813/3447426536", 2024-09-16T20:50:06.175 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-16T20:50:06.175 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-16T20:50:06.175 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-16T20:50:06.175 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:06.176 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.176 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.176 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 3, 2024-09-16T20:50:06.176 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "0612be38-3a24-41b6-aba4-5a2fe985165e", 2024-09-16T20:50:06.176 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-16T20:50:06.176 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-16T20:50:06.176 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-16T20:50:06.176 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-16T20:50:06.176 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-16T20:50:06.176 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-16T20:50:06.176 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 13, 2024-09-16T20:50:06.176 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 96, 2024-09-16T20:50:06.177 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-16T20:50:06.177 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-16T20:50:06.177 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-16T20:50:06.177 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:50:06.177 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.177 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:50:06.177 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6808", 2024-09-16T20:50:06.177 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:50:06.177 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.177 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.177 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:50:06.178 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6809", 2024-09-16T20:50:06.178 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:50:06.178 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:06.178 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:06.178 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.178 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-16T20:50:06.178 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:50:06.178 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.178 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:50:06.178 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6810", 2024-09-16T20:50:06.178 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:50:06.178 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.179 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.179 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:50:06.179 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6811", 2024-09-16T20:50:06.179 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:50:06.179 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:06.179 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:06.179 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.179 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-16T20:50:06.179 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:50:06.179 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.179 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:50:06.180 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6814", 2024-09-16T20:50:06.180 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:50:06.180 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.180 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.180 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:50:06.180 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6815", 2024-09-16T20:50:06.180 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:50:06.180 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:06.180 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:06.180 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.180 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-16T20:50:06.181 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:50:06.181 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.181 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:50:06.181 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6812", 2024-09-16T20:50:06.181 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:50:06.181 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.181 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.181 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:50:06.181 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6813", 2024-09-16T20:50:06.181 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:50:06.181 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:06.181 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:06.182 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.182 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.43:6809/3687583891", 2024-09-16T20:50:06.182 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.43:6811/3687583891", 2024-09-16T20:50:06.182 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.43:6815/3687583891", 2024-09-16T20:50:06.182 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.43:6813/3687583891", 2024-09-16T20:50:06.182 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-16T20:50:06.182 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-16T20:50:06.183 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-16T20:50:06.183 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:06.183 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.183 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.183 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 4, 2024-09-16T20:50:06.183 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "be590bf8-e9fa-44ca-a471-6f6bcb087ca7", 2024-09-16T20:50:06.183 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-16T20:50:06.183 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-16T20:50:06.183 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-16T20:50:06.183 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-16T20:50:06.184 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-16T20:50:06.184 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-16T20:50:06.184 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 11, 2024-09-16T20:50:06.184 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 112, 2024-09-16T20:50:06.184 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-16T20:50:06.184 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-16T20:50:06.184 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-16T20:50:06.184 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:50:06.184 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.184 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:50:06.184 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6800", 2024-09-16T20:50:06.184 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:50:06.185 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.185 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.185 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:50:06.185 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6801", 2024-09-16T20:50:06.185 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:50:06.185 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:06.185 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:06.185 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.185 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-16T20:50:06.185 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:50:06.185 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.186 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:50:06.186 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6802", 2024-09-16T20:50:06.186 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:50:06.186 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.186 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.186 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:50:06.186 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6803", 2024-09-16T20:50:06.186 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:50:06.186 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:06.186 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:06.187 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.187 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-16T20:50:06.187 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:50:06.187 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.187 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:50:06.187 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6806", 2024-09-16T20:50:06.187 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:50:06.187 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.187 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.187 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:50:06.188 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6807", 2024-09-16T20:50:06.188 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:50:06.188 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:06.188 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:06.188 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.188 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-16T20:50:06.188 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:50:06.188 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.188 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:50:06.188 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6804", 2024-09-16T20:50:06.189 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:50:06.189 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.189 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.189 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:50:06.189 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6805", 2024-09-16T20:50:06.189 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:50:06.189 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:06.189 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:06.189 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.189 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.43:6801/2505220646", 2024-09-16T20:50:06.190 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.43:6803/2505220646", 2024-09-16T20:50:06.190 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.43:6807/2505220646", 2024-09-16T20:50:06.190 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.43:6805/2505220646", 2024-09-16T20:50:06.190 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-16T20:50:06.190 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-16T20:50:06.190 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-16T20:50:06.190 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:06.190 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.190 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.190 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 5, 2024-09-16T20:50:06.190 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "4a208e30-031b-4a9f-af52-e26d4a039ea6", 2024-09-16T20:50:06.190 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-16T20:50:06.191 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-16T20:50:06.191 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-16T20:50:06.191 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-16T20:50:06.191 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-16T20:50:06.191 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-16T20:50:06.191 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 13, 2024-09-16T20:50:06.191 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 103, 2024-09-16T20:50:06.191 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-16T20:50:06.191 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-16T20:50:06.191 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-16T20:50:06.191 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:50:06.191 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.192 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:50:06.192 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6816", 2024-09-16T20:50:06.192 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:50:06.192 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.192 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.192 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:50:06.192 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6817", 2024-09-16T20:50:06.192 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:50:06.192 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:06.192 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:06.193 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.193 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-16T20:50:06.193 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:50:06.193 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.193 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:50:06.193 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6818", 2024-09-16T20:50:06.193 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:50:06.193 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.193 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.193 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:50:06.193 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6819", 2024-09-16T20:50:06.193 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:50:06.194 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:06.194 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:06.194 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.194 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-16T20:50:06.194 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:50:06.194 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.194 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:50:06.194 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6822", 2024-09-16T20:50:06.194 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:50:06.194 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.194 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.194 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:50:06.195 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6823", 2024-09-16T20:50:06.195 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:50:06.195 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:06.195 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:06.195 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.195 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-16T20:50:06.195 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:50:06.195 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.195 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:50:06.195 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6820", 2024-09-16T20:50:06.195 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:50:06.196 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.196 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.196 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:50:06.196 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6821", 2024-09-16T20:50:06.196 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:50:06.196 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:06.196 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:06.196 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.196 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.43:6817/1517367307", 2024-09-16T20:50:06.196 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.43:6819/1517367307", 2024-09-16T20:50:06.196 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.43:6823/1517367307", 2024-09-16T20:50:06.197 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.43:6821/1517367307", 2024-09-16T20:50:06.197 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-16T20:50:06.197 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-16T20:50:06.197 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-16T20:50:06.197 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:06.197 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:06.197 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-16T20:50:06.197 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_xinfo": [ 2024-09-16T20:50:06.197 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.197 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 0, 2024-09-16T20:50:06.197 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-16T20:50:06.198 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-16T20:50:06.198 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-16T20:50:06.198 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540719139924082687, 2024-09-16T20:50:06.198 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-16T20:50:06.198 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-16T20:47:47.887412+0000", 2024-09-16T20:50:06.198 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-16T20:50:06.198 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.198 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.198 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 1, 2024-09-16T20:50:06.198 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-16T20:50:06.199 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-16T20:50:06.199 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-16T20:50:06.199 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540719139924082687, 2024-09-16T20:50:06.199 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-16T20:50:06.199 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-16T20:47:47.658782+0000", 2024-09-16T20:50:06.199 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-16T20:50:06.199 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.199 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.199 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 2, 2024-09-16T20:50:06.199 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-16T20:50:06.199 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-16T20:50:06.199 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-16T20:50:06.200 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540719139924082687, 2024-09-16T20:50:06.200 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-16T20:50:06.200 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-16T20:47:47.453752+0000", 2024-09-16T20:50:06.200 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-16T20:50:06.200 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.200 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.200 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 3, 2024-09-16T20:50:06.200 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-16T20:50:06.200 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-16T20:50:06.200 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-16T20:50:06.200 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540719139924082687, 2024-09-16T20:50:06.201 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-16T20:50:06.201 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-16T20:47:47.487992+0000", 2024-09-16T20:50:06.201 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-16T20:50:06.201 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.201 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.201 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 4, 2024-09-16T20:50:06.201 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-16T20:50:06.201 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-16T20:50:06.201 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-16T20:50:06.201 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540719139924082687, 2024-09-16T20:50:06.202 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-16T20:50:06.202 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-16T20:47:46.943344+0000", 2024-09-16T20:50:06.202 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-16T20:50:06.202 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.202 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.202 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 5, 2024-09-16T20:50:06.202 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-16T20:50:06.202 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-16T20:50:06.202 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-16T20:50:06.202 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540719139924082687, 2024-09-16T20:50:06.202 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-16T20:50:06.202 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-16T20:47:47.901547+0000", 2024-09-16T20:50:06.203 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-16T20:50:06.203 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:06.203 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-16T20:50:06.203 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_upmap": [ 2024-09-16T20:50:06.203 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.203 INFO:teuthology.orchestra.run.smithi029.stdout: "pgid": "3.2", 2024-09-16T20:50:06.203 INFO:teuthology.orchestra.run.smithi029.stdout: "osds": [ 2024-09-16T20:50:06.203 INFO:teuthology.orchestra.run.smithi029.stdout: 5, 2024-09-16T20:50:06.203 INFO:teuthology.orchestra.run.smithi029.stdout: 0 2024-09-16T20:50:06.203 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:06.203 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:06.203 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-16T20:50:06.203 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_upmap_items": [ 2024-09-16T20:50:06.204 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.204 INFO:teuthology.orchestra.run.smithi029.stdout: "pgid": "3.a", 2024-09-16T20:50:06.204 INFO:teuthology.orchestra.run.smithi029.stdout: "mappings": [ 2024-09-16T20:50:06.204 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:06.204 INFO:teuthology.orchestra.run.smithi029.stdout: "from": 0, 2024-09-16T20:50:06.204 INFO:teuthology.orchestra.run.smithi029.stdout: "to": 4 2024-09-16T20:50:06.204 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:06.204 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:06.204 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:06.204 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-16T20:50:06.205 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_upmap_primaries": [], 2024-09-16T20:50:06.205 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_temp": [], 2024-09-16T20:50:06.205 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_temp": [], 2024-09-16T20:50:06.205 INFO:teuthology.orchestra.run.smithi029.stdout: "blocklist": {}, 2024-09-16T20:50:06.205 INFO:teuthology.orchestra.run.smithi029.stdout: "range_blocklist": {}, 2024-09-16T20:50:06.205 INFO:teuthology.orchestra.run.smithi029.stdout: "erasure_code_profiles": { 2024-09-16T20:50:06.205 INFO:teuthology.orchestra.run.smithi029.stdout: "default": { 2024-09-16T20:50:06.205 INFO:teuthology.orchestra.run.smithi029.stdout: "crush-failure-domain": "osd", 2024-09-16T20:50:06.205 INFO:teuthology.orchestra.run.smithi029.stdout: "k": "2", 2024-09-16T20:50:06.205 INFO:teuthology.orchestra.run.smithi029.stdout: "m": "1", 2024-09-16T20:50:06.205 INFO:teuthology.orchestra.run.smithi029.stdout: "plugin": "jerasure", 2024-09-16T20:50:06.206 INFO:teuthology.orchestra.run.smithi029.stdout: "technique": "reed_sol_van" 2024-09-16T20:50:06.206 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:06.206 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:06.206 INFO:teuthology.orchestra.run.smithi029.stdout: "removed_snaps_queue": [], 2024-09-16T20:50:06.206 INFO:teuthology.orchestra.run.smithi029.stdout: "new_removed_snaps": [], 2024-09-16T20:50:06.206 INFO:teuthology.orchestra.run.smithi029.stdout: "new_purged_snaps": [], 2024-09-16T20:50:06.206 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_node_flags": {}, 2024-09-16T20:50:06.206 INFO:teuthology.orchestra.run.smithi029.stdout: "device_class_flags": {}, 2024-09-16T20:50:06.206 INFO:teuthology.orchestra.run.smithi029.stdout: "stretch_mode": { 2024-09-16T20:50:06.206 INFO:teuthology.orchestra.run.smithi029.stdout: "stretch_mode_enabled": false, 2024-09-16T20:50:06.207 INFO:teuthology.orchestra.run.smithi029.stdout: "stretch_bucket_count": 0, 2024-09-16T20:50:06.207 INFO:teuthology.orchestra.run.smithi029.stdout: "degraded_stretch_mode": 0, 2024-09-16T20:50:06.207 INFO:teuthology.orchestra.run.smithi029.stdout: "recovering_stretch_mode": 0, 2024-09-16T20:50:06.207 INFO:teuthology.orchestra.run.smithi029.stdout: "stretch_mode_bucket": 0 2024-09-16T20:50:06.207 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:06.207 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:06.209 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:06.209 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:06.210 INFO:tasks.thrashosds.thrasher:j is {'epoch': 122, 'fsid': 'e28af4a7-5fcd-4de8-8f77-a2b575361bb1', 'created': '2024-09-16T20:47:39.760609+0000', 'modified': '2024-09-16T20:50:05.658012+0000', 'last_up_change': '2024-09-16T20:47:50.238013+0000', 'last_in_change': '2024-09-16T20:49:08.779548+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T20:47:52.220750+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-16T20:47:56.756666+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-16T20:48:06.418889+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '122', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 41, 'snap_epoch': 122, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.309999942779541, 'score_stable': 1.309999942779541, 'optimal_score': 1, 'raw_score_acting': 1.309999942779541, 'raw_score_stable': 1.309999942779541, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '6c3cb2d7-7d23-4566-9ca1-d7b08e12360c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 111, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6816', 'nonce': 2744025949}, {'type': 'v1', 'addr': '172.21.15.29:6817', 'nonce': 2744025949}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6818', 'nonce': 2744025949}, {'type': 'v1', 'addr': '172.21.15.29:6819', 'nonce': 2744025949}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6822', 'nonce': 2744025949}, {'type': 'v1', 'addr': '172.21.15.29:6823', 'nonce': 2744025949}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6820', 'nonce': 2744025949}, {'type': 'v1', 'addr': '172.21.15.29:6821', 'nonce': 2744025949}]}, 'public_addr': '172.21.15.29:6817/2744025949', 'cluster_addr': '172.21.15.29:6819/2744025949', 'heartbeat_back_addr': '172.21.15.29:6823/2744025949', 'heartbeat_front_addr': '172.21.15.29:6821/2744025949', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '5cf8c20c-1ed2-42ae-a8b0-c88cc4e36bce', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 81, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6800', 'nonce': 208628762}, {'type': 'v1', 'addr': '172.21.15.29:6801', 'nonce': 208628762}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6802', 'nonce': 208628762}, {'type': 'v1', 'addr': '172.21.15.29:6803', 'nonce': 208628762}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6806', 'nonce': 208628762}, {'type': 'v1', 'addr': '172.21.15.29:6807', 'nonce': 208628762}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6804', 'nonce': 208628762}, {'type': 'v1', 'addr': '172.21.15.29:6805', 'nonce': 208628762}]}, 'public_addr': '172.21.15.29:6801/208628762', 'cluster_addr': '172.21.15.29:6803/208628762', 'heartbeat_back_addr': '172.21.15.29:6807/208628762', 'heartbeat_front_addr': '172.21.15.29:6805/208628762', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '075d16da-82bf-449b-9f6e-542c9e8d810b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 102, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6808', 'nonce': 3447426536}, {'type': 'v1', 'addr': '172.21.15.29:6809', 'nonce': 3447426536}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6810', 'nonce': 3447426536}, {'type': 'v1', 'addr': '172.21.15.29:6811', 'nonce': 3447426536}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6814', 'nonce': 3447426536}, {'type': 'v1', 'addr': '172.21.15.29:6815', 'nonce': 3447426536}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6812', 'nonce': 3447426536}, {'type': 'v1', 'addr': '172.21.15.29:6813', 'nonce': 3447426536}]}, 'public_addr': '172.21.15.29:6809/3447426536', 'cluster_addr': '172.21.15.29:6811/3447426536', 'heartbeat_back_addr': '172.21.15.29:6815/3447426536', 'heartbeat_front_addr': '172.21.15.29:6813/3447426536', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '0612be38-3a24-41b6-aba4-5a2fe985165e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 96, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6808', 'nonce': 3687583891}, {'type': 'v1', 'addr': '172.21.15.43:6809', 'nonce': 3687583891}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6810', 'nonce': 3687583891}, {'type': 'v1', 'addr': '172.21.15.43:6811', 'nonce': 3687583891}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6814', 'nonce': 3687583891}, {'type': 'v1', 'addr': '172.21.15.43:6815', 'nonce': 3687583891}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6812', 'nonce': 3687583891}, {'type': 'v1', 'addr': '172.21.15.43:6813', 'nonce': 3687583891}]}, 'public_addr': '172.21.15.43:6809/3687583891', 'cluster_addr': '172.21.15.43:6811/3687583891', 'heartbeat_back_addr': '172.21.15.43:6815/3687583891', 'heartbeat_front_addr': '172.21.15.43:6813/3687583891', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'be590bf8-e9fa-44ca-a471-6f6bcb087ca7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 112, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6800', 'nonce': 2505220646}, {'type': 'v1', 'addr': '172.21.15.43:6801', 'nonce': 2505220646}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6802', 'nonce': 2505220646}, {'type': 'v1', 'addr': '172.21.15.43:6803', 'nonce': 2505220646}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6806', 'nonce': 2505220646}, {'type': 'v1', 'addr': '172.21.15.43:6807', 'nonce': 2505220646}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6804', 'nonce': 2505220646}, {'type': 'v1', 'addr': '172.21.15.43:6805', 'nonce': 2505220646}]}, 'public_addr': '172.21.15.43:6801/2505220646', 'cluster_addr': '172.21.15.43:6803/2505220646', 'heartbeat_back_addr': '172.21.15.43:6807/2505220646', 'heartbeat_front_addr': '172.21.15.43:6805/2505220646', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '4a208e30-031b-4a9f-af52-e26d4a039ea6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 103, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6816', 'nonce': 1517367307}, {'type': 'v1', 'addr': '172.21.15.43:6817', 'nonce': 1517367307}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6818', 'nonce': 1517367307}, {'type': 'v1', 'addr': '172.21.15.43:6819', 'nonce': 1517367307}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6822', 'nonce': 1517367307}, {'type': 'v1', 'addr': '172.21.15.43:6823', 'nonce': 1517367307}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6820', 'nonce': 1517367307}, {'type': 'v1', 'addr': '172.21.15.43:6821', 'nonce': 1517367307}]}, 'public_addr': '172.21.15.43:6817/1517367307', 'cluster_addr': '172.21.15.43:6819/1517367307', 'heartbeat_back_addr': '172.21.15.43:6823/1517367307', 'heartbeat_front_addr': '172.21.15.43:6821/1517367307', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T20:47:47.887412+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T20:47:47.658782+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T20:47:47.453752+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T20:47:47.487992+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T20:47:46.943344+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T20:47:47.901547+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.2', 'osds': [5, 0]}], 'pg_upmap_items': [{'pgid': '3.a', 'mappings': [{'from': 0, 'to': 4}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-16T20:50:06.210 INFO:tasks.thrashosds.thrasher:Clearing pg_upmap on 3.2 2024-09-16T20:50:06.210 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd rm-pg-upmap 3.2 2024-09-16T20:50:06.211 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:50:06.212 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:06.210+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:06.313 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:06.313+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:06.329 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:06.330 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:06.340 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:50:06.414 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:06.414+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:06.463 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:06.463 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:06.473 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:50:06.517 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:06.516+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:06.580 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:06.580 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:06.588 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:50:06.630 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:06.628+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:06.654 INFO:teuthology.orchestra.run.smithi139.stdout:5066 read 0x9189000 thru 0x9190cb2 (0x7cb3 bytes) 2024-09-16T20:50:06.655 INFO:teuthology.orchestra.run.smithi139.stdout:5070 read 0x935da00 thru 0x936d653 (0xfc54 bytes) 2024-09-16T20:50:06.656 INFO:teuthology.orchestra.run.smithi139.stdout:5073 read 0x7d99c00 thru 0x7d9dd66 (0x4167 bytes) 2024-09-16T20:50:06.664 INFO:teuthology.orchestra.run.smithi029.stderr:clear 3.2 pg_upmap mapping 2024-09-16T20:50:06.665 INFO:teuthology.orchestra.run.smithi139.stdout:5074 trunc from 0xb88ca00 to 0x4103600 2024-09-16T20:50:06.674 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T20:50:06.717 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:06.715+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:06.749 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:06.749 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:06.759 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:50:06.846 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:06.841+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:06.906 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:06.906 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:06.914 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:50:06.917 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:06.917+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:07.018 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:07.017+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:07.023 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:07.023 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:07.031 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:50:07.119 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:07.119+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:07.139 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:07.139 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:07.148 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:50:07.220 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:07.219+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:07.295 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:07.295 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:07.304 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:50:07.349 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:07.349+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:07.411 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:07.411 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:07.420 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:50:07.461 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:07.461+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:07.522 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:07.522+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:07.568 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:07.568 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:07.577 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:50:07.661 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:07.661+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:07.667 INFO:teuthology.orchestra.run.smithi029.stderr:noscrub is set 2024-09-16T20:50:07.724 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:07.723+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:07.729 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:07.729 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:07.737 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:50:07.824 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:07.824+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:07.845 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:07.845 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:07.853 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:50:07.937 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:07.937+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:07.985 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T20:50:08.001 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:08.001 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:08.010 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:50:08.051 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:08.051+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:08.109 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:50:08.109 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:50:08.127 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:50:08.127 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:50:08.127 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:08.126+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:08.142 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:50:08.142 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:50:08.160 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:50:08.161 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:50:08.162 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:08.162 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:50:08.162 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:50:08.162 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:50:08.162 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.162 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3714 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1480704~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:08.162 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:50:08.162 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 105.520140337, 2024-09-16T20:50:08.163 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050152552000000003, 2024-09-16T20:50:08.163 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:08.163 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:08.163 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:08.163 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:08.163 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:08.163 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:08.163 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3714 2024-09-16T20:50:08.164 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.164 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:08.164 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.164 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:08.164 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:50:08.164 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:08.164 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.165 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.165 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:08.165 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:50:08.165 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:08.165 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.165 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.165 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:08.165 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:50:08.165 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0499999999999999e-07 2024-09-16T20:50:08.165 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.165 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.166 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:08.166 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639007+0000", 2024-09-16T20:50:08.166 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5340000000000001e-06 2024-09-16T20:50:08.166 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.166 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.166 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:08.166 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639013+0000", 2024-09-16T20:50:08.166 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3260000000000003e-06 2024-09-16T20:50:08.166 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.166 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.166 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:08.166 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639018+0000", 2024-09-16T20:50:08.166 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2850000000000002e-06 2024-09-16T20:50:08.166 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.167 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.167 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:08.167 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689026+0000", 2024-09-16T20:50:08.167 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.05000798 2024-09-16T20:50:08.167 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.167 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.167 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:08.167 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689077+0000", 2024-09-16T20:50:08.167 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1063999999999998e-05 2024-09-16T20:50:08.167 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.167 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.167 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:08.167 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689158+0000", 2024-09-16T20:50:08.168 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1057999999999995e-05 2024-09-16T20:50:08.168 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:08.168 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:08.168 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:08.168 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.168 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.168 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3716 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1488896~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:08.168 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:50:08.168 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 105.520122527, 2024-09-16T20:50:08.168 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050280142, 2024-09-16T20:50:08.168 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:08.168 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:08.168 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:08.168 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:08.169 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:08.169 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:08.169 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3716 2024-09-16T20:50:08.169 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.169 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:08.169 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.169 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:08.169 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:50:08.169 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:08.169 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.169 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.169 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:08.169 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:50:08.170 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:08.170 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.170 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.170 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:08.170 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639024+0000", 2024-09-16T20:50:08.170 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9999999999999997e-07 2024-09-16T20:50:08.170 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.170 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.170 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:08.170 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639026+0000", 2024-09-16T20:50:08.170 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.781e-06 2024-09-16T20:50:08.170 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.170 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.170 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:08.170 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639032+0000", 2024-09-16T20:50:08.171 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7780000000000004e-06 2024-09-16T20:50:08.171 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.171 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.171 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:08.171 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639036+0000", 2024-09-16T20:50:08.171 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6909999999999996e-06 2024-09-16T20:50:08.171 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.171 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.171 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:08.171 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689191+0000", 2024-09-16T20:50:08.171 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050155284000000001 2024-09-16T20:50:08.171 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.171 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.172 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:08.172 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689242+0000", 2024-09-16T20:50:08.172 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0662999999999999e-05 2024-09-16T20:50:08.172 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.172 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.172 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:08.172 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689304+0000", 2024-09-16T20:50:08.172 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1345000000000003e-05 2024-09-16T20:50:08.172 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:08.172 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:08.172 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:08.172 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.173 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.173 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3719 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1497088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:08.173 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:50:08.173 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 105.52009524899999, 2024-09-16T20:50:08.173 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050399715999999997, 2024-09-16T20:50:08.173 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:08.173 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:08.173 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:08.173 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:08.173 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:08.173 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:08.173 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3719 2024-09-16T20:50:08.173 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.174 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:08.174 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.174 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:08.174 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:50:08.174 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:08.174 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.174 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.174 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:08.174 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:50:08.174 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:08.174 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.174 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.174 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:08.175 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:50:08.175 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9700000000000002e-07 2024-09-16T20:50:08.175 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.175 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.175 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:08.175 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639053+0000", 2024-09-16T20:50:08.175 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2900000000000001e-06 2024-09-16T20:50:08.175 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.175 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.175 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:08.175 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639060+0000", 2024-09-16T20:50:08.175 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7750000000000004e-06 2024-09-16T20:50:08.175 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.175 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.176 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:08.176 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639066+0000", 2024-09-16T20:50:08.176 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0639999999999999e-06 2024-09-16T20:50:08.176 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.176 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.176 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:08.176 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689337+0000", 2024-09-16T20:50:08.176 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050270784999999998 2024-09-16T20:50:08.176 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.176 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.176 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:08.176 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689392+0000", 2024-09-16T20:50:08.176 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4920999999999998e-05 2024-09-16T20:50:08.176 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.177 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.177 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:08.177 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689450+0000", 2024-09-16T20:50:08.177 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8483999999999998e-05 2024-09-16T20:50:08.177 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:08.177 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:08.177 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:08.177 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.177 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.177 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3721 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1505280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:08.177 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:50:08.177 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 105.520072117, 2024-09-16T20:50:08.177 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050513608000000002, 2024-09-16T20:50:08.177 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:08.178 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:08.178 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:08.178 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:08.178 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:08.178 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:08.178 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3721 2024-09-16T20:50:08.178 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.178 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:08.178 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.179 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:08.179 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:50:08.179 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:08.179 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.179 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.179 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:08.179 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:50:08.179 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:08.179 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.179 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.179 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:08.179 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:50:08.179 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7e-07 2024-09-16T20:50:08.180 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.180 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.180 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:08.180 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639076+0000", 2024-09-16T20:50:08.180 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8899999999999999e-06 2024-09-16T20:50:08.180 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.180 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.180 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:08.180 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639083+0000", 2024-09-16T20:50:08.180 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1210000000000001e-06 2024-09-16T20:50:08.180 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.180 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.180 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:08.180 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639089+0000", 2024-09-16T20:50:08.181 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.643e-06 2024-09-16T20:50:08.181 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.181 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.181 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:08.181 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689474+0000", 2024-09-16T20:50:08.181 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050385679000000003 2024-09-16T20:50:08.181 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.181 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.181 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:08.181 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689533+0000", 2024-09-16T20:50:08.181 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8303000000000001e-05 2024-09-16T20:50:08.182 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.182 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.182 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:08.182 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689587+0000", 2024-09-16T20:50:08.182 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4601999999999997e-05 2024-09-16T20:50:08.182 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:08.182 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:08.182 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:08.182 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.182 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.182 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3723 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1513472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:08.183 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:50:08.183 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 105.52004131299999, 2024-09-16T20:50:08.183 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050699691999999998, 2024-09-16T20:50:08.183 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:08.183 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:08.183 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:08.183 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:08.183 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:08.183 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:08.183 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3723 2024-09-16T20:50:08.183 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.183 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:08.183 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.183 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:08.184 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:50:08.184 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:08.184 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.184 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.184 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:08.184 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:50:08.184 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:08.184 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.184 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.184 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:08.184 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:50:08.184 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3399999999999999e-07 2024-09-16T20:50:08.185 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.185 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.185 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:08.185 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639107+0000", 2024-09-16T20:50:08.185 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3860000000000001e-06 2024-09-16T20:50:08.185 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.185 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.185 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:08.185 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639114+0000", 2024-09-16T20:50:08.185 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9800000000000001e-06 2024-09-16T20:50:08.185 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.186 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.186 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:08.186 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639129+0000", 2024-09-16T20:50:08.186 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4887e-05 2024-09-16T20:50:08.186 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.186 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.186 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:08.186 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689625+0000", 2024-09-16T20:50:08.186 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050495848000000003 2024-09-16T20:50:08.186 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.186 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.186 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:08.186 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689720+0000", 2024-09-16T20:50:08.187 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4821000000000006e-05 2024-09-16T20:50:08.187 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.187 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.187 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:08.187 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689804+0000", 2024-09-16T20:50:08.187 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4235999999999998e-05 2024-09-16T20:50:08.187 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:08.187 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:08.187 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:08.187 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.187 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.187 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3725 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1521664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:08.187 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:50:08.187 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 105.520007717, 2024-09-16T20:50:08.188 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050797660000000001, 2024-09-16T20:50:08.188 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:08.188 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:08.188 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:08.188 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:08.188 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:08.188 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:08.188 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3725 2024-09-16T20:50:08.188 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.188 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:08.188 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.188 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:08.188 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:50:08.189 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:08.189 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.189 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.189 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:08.189 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:50:08.189 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:08.189 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.189 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.189 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:08.189 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639139+0000", 2024-09-16T20:50:08.190 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8999999999999997e-07 2024-09-16T20:50:08.190 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.190 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.190 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:08.190 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639141+0000", 2024-09-16T20:50:08.190 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.182e-06 2024-09-16T20:50:08.190 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.190 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.190 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:08.190 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639154+0000", 2024-09-16T20:50:08.190 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3626e-05 2024-09-16T20:50:08.190 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.190 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.190 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:08.191 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639160+0000", 2024-09-16T20:50:08.191 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6810000000000003e-06 2024-09-16T20:50:08.191 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.191 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.191 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:08.191 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689831+0000", 2024-09-16T20:50:08.191 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050670907000000001 2024-09-16T20:50:08.191 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.191 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.191 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:08.191 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689882+0000", 2024-09-16T20:50:08.191 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0445999999999998e-05 2024-09-16T20:50:08.191 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.191 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.192 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:08.192 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689936+0000", 2024-09-16T20:50:08.192 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4327999999999999e-05 2024-09-16T20:50:08.192 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:08.192 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:08.192 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:08.192 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.192 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.192 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3727 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1529856~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:08.192 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:50:08.192 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 105.51997213999999, 2024-09-16T20:50:08.192 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051016922999999999, 2024-09-16T20:50:08.192 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:08.193 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:08.193 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:08.193 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:08.193 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:08.193 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:08.193 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3727 2024-09-16T20:50:08.193 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.193 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:08.193 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.193 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:08.193 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:50:08.193 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:08.193 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.193 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.194 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:08.194 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:50:08.194 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:08.194 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.194 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.194 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:08.194 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:50:08.194 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9999999999999998e-07 2024-09-16T20:50:08.195 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.195 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.195 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:08.195 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639183+0000", 2024-09-16T20:50:08.195 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2399999999999996e-06 2024-09-16T20:50:08.195 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.195 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.195 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:08.195 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639195+0000", 2024-09-16T20:50:08.195 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1113e-05 2024-09-16T20:50:08.195 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.195 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.195 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:08.196 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639206+0000", 2024-09-16T20:50:08.196 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1503e-05 2024-09-16T20:50:08.196 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.196 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.196 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:08.196 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689969+0000", 2024-09-16T20:50:08.196 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050762676 2024-09-16T20:50:08.196 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.196 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.196 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:08.196 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690071+0000", 2024-09-16T20:50:08.196 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102163 2024-09-16T20:50:08.196 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.196 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.197 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:08.197 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690191+0000", 2024-09-16T20:50:08.197 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000119728 2024-09-16T20:50:08.197 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:08.197 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:08.197 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:08.197 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.197 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.197 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3730 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1538048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:08.197 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:50:08.197 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 105.519930577, 2024-09-16T20:50:08.197 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051131467999999999, 2024-09-16T20:50:08.197 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:08.198 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:08.198 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:08.198 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:08.198 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:08.198 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:08.198 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3730 2024-09-16T20:50:08.198 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.198 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:08.198 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.198 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:08.198 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:50:08.198 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:08.199 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.199 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.199 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:08.199 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:50:08.199 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:08.199 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.199 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.199 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:08.199 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639216+0000", 2024-09-16T20:50:08.199 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.39e-07 2024-09-16T20:50:08.200 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.200 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.200 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:08.200 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639218+0000", 2024-09-16T20:50:08.200 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.537e-06 2024-09-16T20:50:08.200 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.200 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.200 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:08.200 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639229+0000", 2024-09-16T20:50:08.200 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1122e-05 2024-09-16T20:50:08.200 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.200 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.200 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:08.201 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639237+0000", 2024-09-16T20:50:08.201 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8509999999999999e-06 2024-09-16T20:50:08.201 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.201 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.201 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:08.201 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690227+0000", 2024-09-16T20:50:08.201 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050990147 2024-09-16T20:50:08.201 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.201 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.201 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:08.201 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690278+0000", 2024-09-16T20:50:08.201 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0223999999999999e-05 2024-09-16T20:50:08.201 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.201 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.202 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:08.202 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690347+0000", 2024-09-16T20:50:08.202 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9147999999999996e-05 2024-09-16T20:50:08.202 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:08.202 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:08.202 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:08.202 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.202 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.202 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3731 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1546240~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:08.202 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:50:08.202 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 105.519899869, 2024-09-16T20:50:08.202 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051276744999999999, 2024-09-16T20:50:08.202 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:08.203 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:08.203 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:08.203 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:08.203 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:08.203 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:08.203 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3731 2024-09-16T20:50:08.203 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.203 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:08.203 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.203 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:08.203 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:50:08.203 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:08.203 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.203 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.203 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:08.204 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:50:08.204 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:08.204 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.204 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.204 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:08.204 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:50:08.204 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9900000000000001e-07 2024-09-16T20:50:08.204 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.204 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.204 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:08.204 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639249+0000", 2024-09-16T20:50:08.204 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3630000000000001e-06 2024-09-16T20:50:08.204 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.204 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.205 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:08.205 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639255+0000", 2024-09-16T20:50:08.205 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4960000000000001e-06 2024-09-16T20:50:08.205 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.205 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.205 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:08.205 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639261+0000", 2024-09-16T20:50:08.205 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2839999999999997e-06 2024-09-16T20:50:08.205 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.205 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.205 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:08.205 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690375+0000", 2024-09-16T20:50:08.205 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051114684000000001 2024-09-16T20:50:08.205 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.206 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.206 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:08.206 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690425+0000", 2024-09-16T20:50:08.206 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9379999999999998e-05 2024-09-16T20:50:08.206 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.206 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.206 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:08.206 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690523+0000", 2024-09-16T20:50:08.206 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8139000000000003e-05 2024-09-16T20:50:08.206 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:08.206 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:08.206 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:08.206 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.206 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.207 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3732 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1554432~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:08.207 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:50:08.207 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 105.519854087, 2024-09-16T20:50:08.207 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051380096, 2024-09-16T20:50:08.207 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:08.207 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:08.207 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:08.207 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:08.207 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:08.207 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:08.207 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3732 2024-09-16T20:50:08.207 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.207 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:08.207 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.208 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:08.208 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:50:08.208 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:08.208 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.208 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.208 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:08.208 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:50:08.208 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:08.208 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.208 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.208 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:08.208 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:50:08.208 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6100000000000001e-07 2024-09-16T20:50:08.209 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.209 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.209 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:08.209 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639295+0000", 2024-09-16T20:50:08.209 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.447e-06 2024-09-16T20:50:08.209 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.209 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.209 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:08.209 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639302+0000", 2024-09-16T20:50:08.209 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0570000000000004e-06 2024-09-16T20:50:08.209 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.209 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.209 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:08.210 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639307+0000", 2024-09-16T20:50:08.210 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1429999999999997e-06 2024-09-16T20:50:08.210 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.210 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.210 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:08.210 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690555+0000", 2024-09-16T20:50:08.210 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051247963000000001 2024-09-16T20:50:08.210 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.210 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.210 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:08.211 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690602+0000", 2024-09-16T20:50:08.211 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7602000000000003e-05 2024-09-16T20:50:08.211 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.211 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.211 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:08.211 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690672+0000", 2024-09-16T20:50:08.211 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9423000000000003e-05 2024-09-16T20:50:08.211 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:08.211 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:08.211 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:08.211 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.211 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.211 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3734 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1562624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:08.212 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:50:08.212 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 105.51983243700001, 2024-09-16T20:50:08.212 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051490610999999999, 2024-09-16T20:50:08.212 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:08.212 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:08.212 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:08.212 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:08.212 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:08.212 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:08.212 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3734 2024-09-16T20:50:08.212 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.212 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:08.212 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.212 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:08.213 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:50:08.213 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:08.213 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.213 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.213 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:08.213 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:50:08.213 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:08.213 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.213 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.213 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:08.213 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639314+0000", 2024-09-16T20:50:08.213 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4400000000000001e-07 2024-09-16T20:50:08.213 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.214 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.214 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:08.214 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639316+0000", 2024-09-16T20:50:08.214 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1459999999999999e-06 2024-09-16T20:50:08.214 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.214 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.214 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:08.214 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639322+0000", 2024-09-16T20:50:08.214 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7610000000000004e-06 2024-09-16T20:50:08.214 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.214 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.214 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:08.214 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639327+0000", 2024-09-16T20:50:08.215 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.186e-06 2024-09-16T20:50:08.215 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.215 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.215 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:08.215 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690698+0000", 2024-09-16T20:50:08.215 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051371223000000001 2024-09-16T20:50:08.215 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.215 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.215 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:08.215 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690748+0000", 2024-09-16T20:50:08.215 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0192999999999999e-05 2024-09-16T20:50:08.215 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.215 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.216 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:08.216 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690804+0000", 2024-09-16T20:50:08.216 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5658000000000001e-05 2024-09-16T20:50:08.216 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:08.216 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:08.217 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:08.217 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.217 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.217 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3736 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1570816~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:08.217 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:50:08.218 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 105.51981317800001, 2024-09-16T20:50:08.218 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051596528000000003, 2024-09-16T20:50:08.218 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:08.218 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:08.218 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:08.218 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:08.218 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:08.218 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:08.219 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3736 2024-09-16T20:50:08.219 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.219 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:08.219 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.219 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:08.219 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:50:08.219 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:08.220 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.220 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.220 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:08.220 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:50:08.220 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:08.220 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.220 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.220 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:08.221 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:50:08.221 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9999999999999999e-07 2024-09-16T20:50:08.221 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.221 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.221 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:08.221 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639335+0000", 2024-09-16T20:50:08.221 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.635e-06 2024-09-16T20:50:08.222 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.222 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.222 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:08.222 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639340+0000", 2024-09-16T20:50:08.222 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5840000000000002e-06 2024-09-16T20:50:08.222 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.223 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.223 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:08.223 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639346+0000", 2024-09-16T20:50:08.223 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1190000000000002e-06 2024-09-16T20:50:08.223 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.223 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.223 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:08.223 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690817+0000", 2024-09-16T20:50:08.223 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051470721999999997 2024-09-16T20:50:08.224 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.224 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.224 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:08.224 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690875+0000", 2024-09-16T20:50:08.224 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7441000000000001e-05 2024-09-16T20:50:08.224 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.224 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.224 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:08.224 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690929+0000", 2024-09-16T20:50:08.224 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4727e-05 2024-09-16T20:50:08.224 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:08.225 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:08.225 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:08.225 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.225 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.225 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5434 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:08.225 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:50:08.225 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 105.29455161200001, 2024-09-16T20:50:08.225 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099755831000000003, 2024-09-16T20:50:08.225 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:08.225 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:08.225 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:08.226 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:08.226 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:08.226 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:08.226 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5434 2024-09-16T20:50:08.226 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.226 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:08.226 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.226 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:08.226 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:50:08.226 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:08.226 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.227 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.227 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:08.227 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:50:08.227 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:08.227 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.227 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.227 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:08.227 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864595+0000", 2024-09-16T20:50:08.227 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1099999999999995e-07 2024-09-16T20:50:08.227 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.227 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.227 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:08.227 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864597+0000", 2024-09-16T20:50:08.227 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3989999999999998e-06 2024-09-16T20:50:08.228 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.228 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.228 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:08.228 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864603+0000", 2024-09-16T20:50:08.228 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9309999999999996e-06 2024-09-16T20:50:08.228 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.228 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.228 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:08.228 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864607+0000", 2024-09-16T20:50:08.228 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6200000000000001e-06 2024-09-16T20:50:08.228 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.228 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.228 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:08.229 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964164+0000", 2024-09-16T20:50:08.229 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099557162000000005 2024-09-16T20:50:08.229 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.229 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.229 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:08.229 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964244+0000", 2024-09-16T20:50:08.229 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9597000000000004e-05 2024-09-16T20:50:08.229 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.229 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.229 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:08.229 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964350+0000", 2024-09-16T20:50:08.229 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010641099999999999 2024-09-16T20:50:08.229 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:08.229 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:08.229 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:08.230 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.230 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.230 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5435 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 20480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:08.230 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:50:08.230 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 105.29452179, 2024-09-16T20:50:08.230 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100897102, 2024-09-16T20:50:08.230 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:08.230 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:08.230 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:08.230 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:08.230 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:08.230 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:08.230 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5435 2024-09-16T20:50:08.230 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.231 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:08.231 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.231 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:08.231 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:50:08.231 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:08.231 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.231 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.231 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:08.231 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:50:08.231 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:08.231 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.231 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.231 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:08.232 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:50:08.232 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8000000000000001e-07 2024-09-16T20:50:08.232 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.232 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.232 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:08.232 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864626+0000", 2024-09-16T20:50:08.232 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.745e-06 2024-09-16T20:50:08.232 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.232 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.232 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:08.232 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864633+0000", 2024-09-16T20:50:08.232 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3339999999999998e-06 2024-09-16T20:50:08.232 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.232 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.233 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:08.233 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864636+0000", 2024-09-16T20:50:08.233 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1200000000000002e-06 2024-09-16T20:50:08.233 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.233 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.233 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:08.233 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965299+0000", 2024-09-16T20:50:08.233 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10066352000000001 2024-09-16T20:50:08.233 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.233 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.233 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:08.233 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965390+0000", 2024-09-16T20:50:08.233 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0894000000000001e-05 2024-09-16T20:50:08.233 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.234 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.234 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:08.234 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965521+0000", 2024-09-16T20:50:08.234 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013110899999999999 2024-09-16T20:50:08.234 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:08.234 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:08.234 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:08.234 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.234 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.234 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:152 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 327680~32768 out=32768b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:50:08.234 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:50:08.234 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 103.996024848, 2024-09-16T20:50:08.234 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11869405800000001, 2024-09-16T20:50:08.234 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:08.234 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:08.235 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:08.235 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:08.235 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:50:08.235 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:50:08.235 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 152 2024-09-16T20:50:08.235 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.235 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:08.235 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.235 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:08.235 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:50:08.235 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:08.235 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.235 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.235 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:08.236 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:50:08.236 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:08.236 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.236 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.236 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:08.236 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163122+0000", 2024-09-16T20:50:08.236 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6400000000000003e-07 2024-09-16T20:50:08.236 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.236 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.236 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:08.236 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163124+0000", 2024-09-16T20:50:08.236 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.565e-06 2024-09-16T20:50:08.236 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.236 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.237 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:08.237 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163136+0000", 2024-09-16T20:50:08.237 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1717e-05 2024-09-16T20:50:08.237 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.237 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.237 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:08.237 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163142+0000", 2024-09-16T20:50:08.237 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1750000000000002e-06 2024-09-16T20:50:08.237 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.237 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.237 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:08.237 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163606+0000", 2024-09-16T20:50:08.237 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00046418799999999998 2024-09-16T20:50:08.237 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.238 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.238 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:08.238 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163656+0000", 2024-09-16T20:50:08.238 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9852999999999999e-05 2024-09-16T20:50:08.238 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.238 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.238 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:08.238 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281815+0000", 2024-09-16T20:50:08.238 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11815909600000001 2024-09-16T20:50:08.238 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:08.238 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:08.238 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:08.238 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.238 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.239 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:153 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:50:08.239 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:50:08.239 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 103.995991895, 2024-09-16T20:50:08.239 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11903462400000001, 2024-09-16T20:50:08.239 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:08.239 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:08.239 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:08.239 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:08.239 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:50:08.239 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:50:08.239 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 153 2024-09-16T20:50:08.239 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.239 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:08.239 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.239 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:08.240 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:50:08.240 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:08.240 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.240 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.240 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:08.240 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:50:08.240 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:08.240 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.240 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.240 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:08.240 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163155+0000", 2024-09-16T20:50:08.240 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.37e-07 2024-09-16T20:50:08.240 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.240 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.241 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:08.241 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163157+0000", 2024-09-16T20:50:08.241 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6010000000000002e-06 2024-09-16T20:50:08.241 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.241 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.241 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:08.241 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163165+0000", 2024-09-16T20:50:08.241 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2619999999999998e-06 2024-09-16T20:50:08.241 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.241 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.241 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:08.241 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163171+0000", 2024-09-16T20:50:08.241 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.711e-06 2024-09-16T20:50:08.241 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.242 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.242 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:08.242 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281868+0000", 2024-09-16T20:50:08.242 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118696622 2024-09-16T20:50:08.242 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.242 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.242 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:08.242 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281969+0000", 2024-09-16T20:50:08.242 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101371 2024-09-16T20:50:08.242 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.242 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.242 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:08.242 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282189+0000", 2024-09-16T20:50:08.242 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00021952000000000001 2024-09-16T20:50:08.243 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:08.243 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:08.243 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:08.243 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.243 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.243 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:154 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 360448~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:50:08.243 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:50:08.243 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 103.995960123, 2024-09-16T20:50:08.243 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11932035100000001, 2024-09-16T20:50:08.243 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:08.243 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:08.243 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:08.243 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:08.243 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:50:08.244 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:50:08.244 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 154 2024-09-16T20:50:08.244 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.244 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:08.244 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.244 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:08.244 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:50:08.244 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:08.244 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.244 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.244 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:08.244 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:50:08.244 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:08.244 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.245 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.245 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:08.245 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:50:08.245 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:50:08.245 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.245 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.245 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:08.245 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163189+0000", 2024-09-16T20:50:08.245 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6840000000000001e-06 2024-09-16T20:50:08.245 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.245 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.245 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:08.245 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163196+0000", 2024-09-16T20:50:08.245 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7920000000000004e-06 2024-09-16T20:50:08.246 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.246 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.246 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:08.246 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163201+0000", 2024-09-16T20:50:08.246 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2809999999999999e-06 2024-09-16T20:50:08.246 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.246 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.246 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:08.246 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282219+0000", 2024-09-16T20:50:08.246 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.119018343 2024-09-16T20:50:08.246 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.246 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.246 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:08.246 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282305+0000", 2024-09-16T20:50:08.247 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5865000000000005e-05 2024-09-16T20:50:08.247 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.247 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.247 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:08.247 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282506+0000", 2024-09-16T20:50:08.247 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000200927 2024-09-16T20:50:08.247 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:08.247 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:08.247 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:08.247 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.247 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.247 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop(client.4366.0:3088 3.1e e38/23 3:79e5ec01:::rbd_header.110e891a9b2b:head v 38'306, mlcod=36'304)", 2024-09-16T20:50:08.247 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:50:08.247 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 103.994941255, 2024-09-16T20:50:08.248 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118102759, 2024-09-16T20:50:08.248 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:08.248 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:08.248 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:08.248 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:08.248 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.248 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:08.248 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:50:08.248 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:08.248 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.248 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.248 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:08.248 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:50:08.248 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:08.249 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.249 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.249 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:08.249 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:50:08.249 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.85e-07 2024-09-16T20:50:08.249 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.249 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.249 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:08.249 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164212+0000", 2024-09-16T20:50:08.249 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3999999999999997e-06 2024-09-16T20:50:08.249 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.249 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.249 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:08.249 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281553+0000", 2024-09-16T20:50:08.250 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11734135499999999 2024-09-16T20:50:08.250 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.250 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.250 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:08.250 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281567+0000", 2024-09-16T20:50:08.250 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4188999999999999e-05 2024-09-16T20:50:08.250 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.250 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.250 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:08.250 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281653+0000", 2024-09-16T20:50:08.250 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5385999999999999e-05 2024-09-16T20:50:08.250 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.250 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.251 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:08.251 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281679+0000", 2024-09-16T20:50:08.251 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6689000000000001e-05 2024-09-16T20:50:08.251 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.251 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.252 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:08.252 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282277+0000", 2024-09-16T20:50:08.252 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00059798000000000002 2024-09-16T20:50:08.252 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.252 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.252 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:08.252 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282307+0000", 2024-09-16T20:50:08.252 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9975e-05 2024-09-16T20:50:08.252 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:08.252 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:08.252 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:08.252 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.252 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.253 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5255.0:44 3.16 3:6f5476df:::rbd_header.110e25db1f60:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:50:08.253 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:50:08.253 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 31.838638199999998, 2024-09-16T20:50:08.253 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.121007509, 2024-09-16T20:50:08.253 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:08.253 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:08.253 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:08.253 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:08.253 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5255", 2024-09-16T20:50:08.253 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:50:08.253 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 44 2024-09-16T20:50:08.253 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.253 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:08.254 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.254 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:08.254 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:50:08.254 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:08.254 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.254 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.254 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:08.254 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:50:08.254 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:08.254 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.254 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.254 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:08.254 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:50:08.254 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0500000000000003e-07 2024-09-16T20:50:08.255 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.255 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.255 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:08.255 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320512+0000", 2024-09-16T20:50:08.255 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2540000000000001e-06 2024-09-16T20:50:08.255 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.255 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.255 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:08.255 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320518+0000", 2024-09-16T20:50:08.255 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1689999999999997e-06 2024-09-16T20:50:08.255 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.255 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.255 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:08.255 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320521+0000", 2024-09-16T20:50:08.256 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5200000000000002e-06 2024-09-16T20:50:08.256 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.256 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.256 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:08.256 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320548+0000", 2024-09-16T20:50:08.256 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6662999999999999e-05 2024-09-16T20:50:08.256 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.256 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.256 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:08.256 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320587+0000", 2024-09-16T20:50:08.256 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8866999999999999e-05 2024-09-16T20:50:08.256 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.256 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.257 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:08.257 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.441515+0000", 2024-09-16T20:50:08.257 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.120928331 2024-09-16T20:50:08.257 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:08.257 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:08.257 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:08.257 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.257 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.257 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:20329 3.5 3:a4ea64b7:::rbd_header.110ede5ef857:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:08.257 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:01.061694+0000", 2024-09-16T20:50:08.257 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 7.0974516320000003, 2024-09-16T20:50:08.257 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.071234077000000007, 2024-09-16T20:50:08.257 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:08.257 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:08.258 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:08.258 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:08.258 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:50:08.258 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:08.258 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 20329 2024-09-16T20:50:08.258 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.258 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:08.258 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.258 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:08.258 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.061694+0000", 2024-09-16T20:50:08.258 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:08.258 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.258 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.258 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:08.259 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.061694+0000", 2024-09-16T20:50:08.259 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:08.259 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.259 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.259 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:08.259 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089232+0000", 2024-09-16T20:50:08.259 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.027537275 2024-09-16T20:50:08.259 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.259 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.259 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:08.259 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089237+0000", 2024-09-16T20:50:08.259 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4709999999999998e-06 2024-09-16T20:50:08.259 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.259 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.259 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:08.260 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089246+0000", 2024-09-16T20:50:08.260 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8230000000000001e-06 2024-09-16T20:50:08.260 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.260 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.260 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:08.260 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089252+0000", 2024-09-16T20:50:08.260 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3060000000000003e-06 2024-09-16T20:50:08.260 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.260 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.260 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:08.260 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.132756+0000", 2024-09-16T20:50:08.260 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.043504334999999998 2024-09-16T20:50:08.261 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.261 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.261 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:08.261 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.132829+0000", 2024-09-16T20:50:08.261 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2849999999999995e-05 2024-09-16T20:50:08.261 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:08.261 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:08.261 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:08.261 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.132928+0000", 2024-09-16T20:50:08.261 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.9017000000000004e-05 2024-09-16T20:50:08.261 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:08.261 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:08.261 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:08.262 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:08.262 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:08.262 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:08.263 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:50:08.264 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:50:08.264 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:50:08.265 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:50:08.266 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:50:08.266 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:08.262+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:08.364 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:08.364+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:08.385 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:08.385 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:50:08.385 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:50:08.386 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:08.394 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:50:08.465 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:08.465+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:08.502 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:08.502 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:50:08.502 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:50:08.503 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:50:08.503 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:08.511 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:50:08.567 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:08.566+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:08.620 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:08.620 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:50:08.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:50:08.620 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:50:08.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.620 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:3088 3.1e 3:79e5ec01:::rbd_header.110e891a9b2b:head [watch watch cookie 140213437171488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:50:08.620 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:50:08.621 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 104.454345254, 2024-09-16T20:50:08.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.11895186200000001, 2024-09-16T20:50:08.621 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:08.621 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:08.621 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:08.621 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:08.621 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:50:08.621 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:08.621 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3088 2024-09-16T20:50:08.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.621 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:08.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.622 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:08.622 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:50:08.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:08.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.622 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:08.622 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:50:08.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:08.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.622 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:08.622 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163846+0000", 2024-09-16T20:50:08.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6799999999999999e-07 2024-09-16T20:50:08.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:08.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163850+0000", 2024-09-16T20:50:08.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.929e-06 2024-09-16T20:50:08.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:08.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163862+0000", 2024-09-16T20:50:08.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1972e-05 2024-09-16T20:50:08.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:08.624 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163867+0000", 2024-09-16T20:50:08.624 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.451e-06 2024-09-16T20:50:08.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.624 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.624 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:08.624 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163912+0000", 2024-09-16T20:50:08.624 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4456000000000003e-05 2024-09-16T20:50:08.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.624 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.624 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:08.624 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163962+0000", 2024-09-16T20:50:08.624 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0578000000000003e-05 2024-09-16T20:50:08.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.624 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.625 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-16T20:50:08.625 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164058+0000", 2024-09-16T20:50:08.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5934999999999995e-05 2024-09-16T20:50:08.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.625 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:50:08.625 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164570+0000", 2024-09-16T20:50:08.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00051241100000000005 2024-09-16T20:50:08.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.625 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:08.625 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282629+0000", 2024-09-16T20:50:08.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.118058986 2024-09-16T20:50:08.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.626 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:50:08.626 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282725+0000", 2024-09-16T20:50:08.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5063999999999996e-05 2024-09-16T20:50:08.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.626 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:08.626 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282797+0000", 2024-09-16T20:50:08.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2212000000000006e-05 2024-09-16T20:50:08.626 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:08.626 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:08.626 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:08.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.627 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:8039 3.1e 3.ac44ff9e (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:50:08.627 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:50:08.627 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 78.910517905000006, 2024-09-16T20:50:08.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0053310449999999, 2024-09-16T20:50:08.627 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:08.627 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:08.627 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:08.627 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:08.627 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:50:08.627 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:08.627 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8039 2024-09-16T20:50:08.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.628 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:08.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:08.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:50:08.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:08.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:08.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:50:08.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:08.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.629 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.629 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:08.629 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707674+0000", 2024-09-16T20:50:08.629 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6530000000000001e-06 2024-09-16T20:50:08.629 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.629 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.629 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:08.629 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707681+0000", 2024-09-16T20:50:08.629 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9639999999999997e-06 2024-09-16T20:50:08.629 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.629 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.629 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:08.629 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707700+0000", 2024-09-16T20:50:08.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8674000000000001e-05 2024-09-16T20:50:08.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:08.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707708+0000", 2024-09-16T20:50:08.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6810000000000004e-06 2024-09-16T20:50:08.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:08.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711293+0000", 2024-09-16T20:50:08.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035848189999999999 2024-09-16T20:50:08.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:08.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711301+0000", 2024-09-16T20:50:08.631 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4730000000000003e-06 2024-09-16T20:50:08.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.631 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.631 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:08.631 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.712990+0000", 2024-09-16T20:50:08.631 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.001689348 2024-09-16T20:50:08.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.631 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.631 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:08.631 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713003+0000", 2024-09-16T20:50:08.631 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3433e-05 2024-09-16T20:50:08.631 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:08.631 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:08.631 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:08.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.632 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.632 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:8040 3.1e 3.ac44ff9e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:50:08.632 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:50:08.632 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 78.910457510000001, 2024-09-16T20:50:08.632 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0053410780000001, 2024-09-16T20:50:08.632 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:08.632 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:08.632 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:08.632 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:08.632 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:50:08.632 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:08.632 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8040 2024-09-16T20:50:08.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.633 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:08.633 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.633 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:08.633 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:50:08.633 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:08.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.633 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.633 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:08.633 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:50:08.633 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:08.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.634 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:08.634 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:50:08.634 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8100000000000003e-07 2024-09-16T20:50:08.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.634 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:08.634 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707737+0000", 2024-09-16T20:50:08.634 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5020000000000001e-06 2024-09-16T20:50:08.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.635 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:08.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707745+0000", 2024-09-16T20:50:08.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2169999999999994e-06 2024-09-16T20:50:08.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.635 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:08.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707751+0000", 2024-09-16T20:50:08.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.263e-06 2024-09-16T20:50:08.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.635 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:08.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711336+0000", 2024-09-16T20:50:08.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035848360000000001 2024-09-16T20:50:08.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.636 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:08.636 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711340+0000", 2024-09-16T20:50:08.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1309999999999999e-06 2024-09-16T20:50:08.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.636 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:08.636 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713060+0000", 2024-09-16T20:50:08.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0017195059999999 2024-09-16T20:50:08.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.636 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:08.636 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713074+0000", 2024-09-16T20:50:08.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4042e-05 2024-09-16T20:50:08.636 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:08.637 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:08.637 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:08.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.637 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4927.0:4487 3.1e 3.ac44ff9e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:50:08.637 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:50:08.637 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 78.909937307999996, 2024-09-16T20:50:08.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0048840880000001, 2024-09-16T20:50:08.637 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:08.637 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:08.637 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:08.637 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:08.637 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4927", 2024-09-16T20:50:08.637 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:08.638 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4487 2024-09-16T20:50:08.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.638 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:08.638 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.638 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:08.638 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:50:08.638 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:08.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.638 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.638 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:08.638 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:50:08.638 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:08.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.638 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.638 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:08.639 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708254+0000", 2024-09-16T20:50:08.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1800000000000005e-07 2024-09-16T20:50:08.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.639 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:08.639 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708258+0000", 2024-09-16T20:50:08.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8689999999999997e-06 2024-09-16T20:50:08.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.639 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:08.639 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708267+0000", 2024-09-16T20:50:08.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9590000000000007e-06 2024-09-16T20:50:08.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.639 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:08.640 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708274+0000", 2024-09-16T20:50:08.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9970000000000001e-06 2024-09-16T20:50:08.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.640 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:08.640 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711366+0000", 2024-09-16T20:50:08.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030919530000000001 2024-09-16T20:50:08.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.640 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:08.640 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711372+0000", 2024-09-16T20:50:08.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9410000000000004e-06 2024-09-16T20:50:08.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:08.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713126+0000", 2024-09-16T20:50:08.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0017537249999999 2024-09-16T20:50:08.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:08.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713137+0000", 2024-09-16T20:50:08.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0825999999999999e-05 2024-09-16T20:50:08.641 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:08.641 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:08.641 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:08.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.642 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5089.0:11703 3.1b 3:dcd4684e:::rbd_data.110e747f0779.000000000000001b:head [set-alloc-hint object_size 262144 write_size 262144,write 139264~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:50:08.642 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:50:08.642 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 64.480201534000003, 2024-09-16T20:50:08.642 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.084231385000000006, 2024-09-16T20:50:08.642 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:08.642 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:08.642 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:08.642 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:08.642 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5089", 2024-09-16T20:50:08.642 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:50:08.642 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 11703 2024-09-16T20:50:08.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.642 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:08.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.643 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:08.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:50:08.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:08.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.643 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:08.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:50:08.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:08.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.643 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:08.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:50:08.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2600000000000001e-07 2024-09-16T20:50:08.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.644 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:08.644 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137998+0000", 2024-09-16T20:50:08.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9549999999999998e-06 2024-09-16T20:50:08.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.644 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:08.644 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138008+0000", 2024-09-16T20:50:08.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3789999999999995e-06 2024-09-16T20:50:08.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.644 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:08.644 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138013+0000", 2024-09-16T20:50:08.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2209999999999996e-06 2024-09-16T20:50:08.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:08.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138054+0000", 2024-09-16T20:50:08.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1022999999999997e-05 2024-09-16T20:50:08.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:08.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138124+0000", 2024-09-16T20:50:08.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0234999999999994e-05 2024-09-16T20:50:08.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:08.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138197+0000", 2024-09-16T20:50:08.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2451000000000001e-05 2024-09-16T20:50:08.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.646 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:08.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222010+0000", 2024-09-16T20:50:08.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.083813433000000007 2024-09-16T20:50:08.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.646 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:50:08.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222163+0000", 2024-09-16T20:50:08.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015274599999999999 2024-09-16T20:50:08.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.647 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:50:08.647 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222197+0000", 2024-09-16T20:50:08.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3683000000000003e-05 2024-09-16T20:50:08.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.647 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:08.647 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222220+0000", 2024-09-16T20:50:08.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3433000000000001e-05 2024-09-16T20:50:08.647 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:08.647 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:08.647 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:08.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.647 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.5089.0:11702 3.15 e73/64 3:acf33c8b:::rbd_data.110e747f0779.000000000000001a:head v 73'132, mlcod=73'130)", 2024-09-16T20:50:08.648 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:50:08.648 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 64.479934205000006, 2024-09-16T20:50:08.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.084079763000000002, 2024-09-16T20:50:08.648 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:08.648 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:08.648 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:08.648 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:08.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:08.648 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:50:08.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:08.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:08.648 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:50:08.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:08.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:08.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:50:08.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:50:08.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:08.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138263+0000", 2024-09-16T20:50:08.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1e-06 2024-09-16T20:50:08.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.650 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:08.650 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221589+0000", 2024-09-16T20:50:08.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.083326002999999996 2024-09-16T20:50:08.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.650 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:08.650 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221614+0000", 2024-09-16T20:50:08.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5593999999999999e-05 2024-09-16T20:50:08.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.650 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:08.650 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221714+0000", 2024-09-16T20:50:08.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9889000000000005e-05 2024-09-16T20:50:08.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.651 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:08.651 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221727+0000", 2024-09-16T20:50:08.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3125999999999999e-05 2024-09-16T20:50:08.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.651 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:50:08.651 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222294+0000", 2024-09-16T20:50:08.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00056712199999999998 2024-09-16T20:50:08.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.651 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:08.651 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222336+0000", 2024-09-16T20:50:08.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1470000000000001e-05 2024-09-16T20:50:08.652 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:08.652 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:08.652 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:08.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.652 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5138.0:9358 3.c 3:33996a43:::rbd_data.110e6372565d.0000000000000007:head [read 1343488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T20:50:08.652 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:50:08.653 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 40.765658795, 2024-09-16T20:50:08.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0167493569999999, 2024-09-16T20:50:08.653 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:08.653 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:08.653 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:08.653 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:08.653 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5138", 2024-09-16T20:50:08.653 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:50:08.654 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9358 2024-09-16T20:50:08.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.654 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:08.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:08.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:50:08.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:08.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:08.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:50:08.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:08.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.655 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.655 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:08.655 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852533+0000", 2024-09-16T20:50:08.655 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.646e-06 2024-09-16T20:50:08.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.655 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.655 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:08.655 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852539+0000", 2024-09-16T20:50:08.655 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3389999999999998e-06 2024-09-16T20:50:08.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.655 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.655 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:08.655 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852552+0000", 2024-09-16T20:50:08.655 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.271e-05 2024-09-16T20:50:08.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.656 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:08.656 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852563+0000", 2024-09-16T20:50:08.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1139e-05 2024-09-16T20:50:08.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.656 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:08.656 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852883+0000", 2024-09-16T20:50:08.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00032010600000000001 2024-09-16T20:50:08.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.657 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:08.657 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852890+0000", 2024-09-16T20:50:08.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7669999999999998e-06 2024-09-16T20:50:08.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.657 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:08.657 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869059+0000", 2024-09-16T20:50:08.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.016169353 2024-09-16T20:50:08.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.657 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:08.657 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869178+0000", 2024-09-16T20:50:08.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118193 2024-09-16T20:50:08.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.658 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:08.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869281+0000", 2024-09-16T20:50:08.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010310399999999999 2024-09-16T20:50:08.658 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:08.658 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:08.658 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:08.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.658 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5138.0:9382 3.c 3:33996a43:::rbd_data.110e6372565d.0000000000000007:head [read 0~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T20:50:08.658 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:50:08.658 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 40.765607918000001, 2024-09-16T20:50:08.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.017136034, 2024-09-16T20:50:08.659 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:08.659 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:08.659 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:08.659 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:08.659 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5138", 2024-09-16T20:50:08.659 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:50:08.659 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9382 2024-09-16T20:50:08.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.659 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:08.659 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.659 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:08.659 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:50:08.659 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:08.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.659 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.659 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:08.660 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:50:08.660 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:08.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.660 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:08.660 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852583+0000", 2024-09-16T20:50:08.660 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.51e-07 2024-09-16T20:50:08.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.660 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:08.660 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852586+0000", 2024-09-16T20:50:08.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0280000000000001e-06 2024-09-16T20:50:08.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.661 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:08.661 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852595+0000", 2024-09-16T20:50:08.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2799999999999992e-06 2024-09-16T20:50:08.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.661 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:08.661 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852601+0000", 2024-09-16T20:50:08.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3810000000000004e-06 2024-09-16T20:50:08.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.662 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.662 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:08.662 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852921+0000", 2024-09-16T20:50:08.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00031979699999999997 2024-09-16T20:50:08.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.662 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.662 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:08.662 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852930+0000", 2024-09-16T20:50:08.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6489999999999997e-06 2024-09-16T20:50:08.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.662 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.662 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:08.662 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869434+0000", 2024-09-16T20:50:08.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0165038829999999 2024-09-16T20:50:08.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.663 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:08.663 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869495+0000", 2024-09-16T20:50:08.663 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1509999999999999e-05 2024-09-16T20:50:08.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.663 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:08.663 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869718+0000", 2024-09-16T20:50:08.663 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000222955 2024-09-16T20:50:08.663 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:08.663 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:08.663 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:08.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.664 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.664 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9969 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 632832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:08.664 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642045+0000", 2024-09-16T20:50:08.664 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 17.976145461000002, 2024-09-16T20:50:08.664 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076903192999999995, 2024-09-16T20:50:08.664 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:08.664 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:08.664 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:08.664 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:08.664 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:08.664 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:08.664 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9969 2024-09-16T20:50:08.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.664 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:08.665 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.665 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:08.665 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642045+0000", 2024-09-16T20:50:08.665 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:08.665 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.665 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.665 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:08.665 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642045+0000", 2024-09-16T20:50:08.665 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:08.665 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.665 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.665 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:08.665 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642045+0000", 2024-09-16T20:50:08.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5100000000000001e-07 2024-09-16T20:50:08.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.666 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:08.666 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642047+0000", 2024-09-16T20:50:08.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8679999999999999e-06 2024-09-16T20:50:08.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.667 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:08.667 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642053+0000", 2024-09-16T20:50:08.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1720000000000004e-06 2024-09-16T20:50:08.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.667 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:08.667 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642060+0000", 2024-09-16T20:50:08.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0339999999999999e-06 2024-09-16T20:50:08.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.667 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:08.667 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.718562+0000", 2024-09-16T20:50:08.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076501969000000003 2024-09-16T20:50:08.668 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.668 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.668 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:08.668 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.718643+0000", 2024-09-16T20:50:08.668 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0539999999999998e-05 2024-09-16T20:50:08.668 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.668 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.668 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:08.668 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.718948+0000", 2024-09-16T20:50:08.668 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000305259 2024-09-16T20:50:08.668 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:08.668 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:08.668 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:08.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.669 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.669 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9970 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 637952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:08.669 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642067+0000", 2024-09-16T20:50:08.669 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 17.976123480999998, 2024-09-16T20:50:08.669 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077174928000000004, 2024-09-16T20:50:08.669 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:08.669 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:08.669 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:08.669 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:08.669 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:08.669 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:08.669 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9970 2024-09-16T20:50:08.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.670 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:08.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.670 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:08.670 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642067+0000", 2024-09-16T20:50:08.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:08.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.670 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:08.670 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642067+0000", 2024-09-16T20:50:08.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:08.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.670 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:08.671 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642067+0000", 2024-09-16T20:50:08.671 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2499999999999995e-07 2024-09-16T20:50:08.671 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.671 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.671 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:08.671 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642070+0000", 2024-09-16T20:50:08.671 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3199999999999998e-06 2024-09-16T20:50:08.671 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.671 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.671 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:08.671 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642076+0000", 2024-09-16T20:50:08.671 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.122e-06 2024-09-16T20:50:08.671 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.672 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.672 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:08.672 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642081+0000", 2024-09-16T20:50:08.672 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0869999999999997e-06 2024-09-16T20:50:08.672 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.672 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.672 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:08.672 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.718980+0000", 2024-09-16T20:50:08.672 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076899482000000005 2024-09-16T20:50:08.672 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.672 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.672 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:08.672 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719085+0000", 2024-09-16T20:50:08.673 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010446399999999999 2024-09-16T20:50:08.673 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.673 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.673 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:08.673 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719242+0000", 2024-09-16T20:50:08.673 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000156928 2024-09-16T20:50:08.673 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:08.673 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:08.673 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:08.673 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.673 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.673 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9971 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 643072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:08.673 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642102+0000", 2024-09-16T20:50:08.674 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 17.976087774, 2024-09-16T20:50:08.674 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077421016999999995, 2024-09-16T20:50:08.674 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:08.674 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:08.674 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:08.674 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:08.674 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:08.674 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:08.674 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9971 2024-09-16T20:50:08.674 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.674 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:08.674 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.674 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:08.675 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642102+0000", 2024-09-16T20:50:08.675 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:08.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.675 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.675 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:08.675 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642102+0000", 2024-09-16T20:50:08.675 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:08.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.675 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.675 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:08.675 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642103+0000", 2024-09-16T20:50:08.676 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7399999999999998e-07 2024-09-16T20:50:08.676 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.676 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.676 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:08.676 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642105+0000", 2024-09-16T20:50:08.676 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2649999999999999e-06 2024-09-16T20:50:08.676 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.676 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.676 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:08.676 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642112+0000", 2024-09-16T20:50:08.676 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.294e-06 2024-09-16T20:50:08.677 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.677 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.677 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:08.677 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642118+0000", 2024-09-16T20:50:08.677 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4500000000000003e-06 2024-09-16T20:50:08.677 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.677 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.677 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:08.677 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719285+0000", 2024-09-16T20:50:08.678 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077166883000000006 2024-09-16T20:50:08.678 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.678 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.678 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:08.678 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719368+0000", 2024-09-16T20:50:08.678 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2811000000000004e-05 2024-09-16T20:50:08.678 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.678 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.678 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:08.678 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719523+0000", 2024-09-16T20:50:08.678 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015584 2024-09-16T20:50:08.678 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:08.678 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:08.678 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:08.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.679 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.679 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9972 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 648192~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:08.679 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642125+0000", 2024-09-16T20:50:08.679 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 17.976064826999998, 2024-09-16T20:50:08.679 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077710324999999997, 2024-09-16T20:50:08.679 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:08.679 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:08.679 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:08.679 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:08.679 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:08.679 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:08.679 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9972 2024-09-16T20:50:08.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.679 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:08.680 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.680 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:08.680 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642125+0000", 2024-09-16T20:50:08.680 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:08.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.680 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.680 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:08.680 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642125+0000", 2024-09-16T20:50:08.680 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:08.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.680 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.680 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:08.680 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642126+0000", 2024-09-16T20:50:08.681 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1899999999999998e-07 2024-09-16T20:50:08.681 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.681 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.681 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:08.681 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642128+0000", 2024-09-16T20:50:08.681 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8619999999999999e-06 2024-09-16T20:50:08.681 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.681 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.681 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:08.681 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642134+0000", 2024-09-16T20:50:08.681 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5289999999999997e-06 2024-09-16T20:50:08.681 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.681 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.682 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:08.682 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642139+0000", 2024-09-16T20:50:08.682 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1250000000000001e-06 2024-09-16T20:50:08.684 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.685 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.685 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:08.685 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719570+0000", 2024-09-16T20:50:08.685 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077430516000000005 2024-09-16T20:50:08.685 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.685 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.685 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:08.685 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719655+0000", 2024-09-16T20:50:08.685 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5500000000000005e-05 2024-09-16T20:50:08.685 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.685 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.685 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:08.686 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719836+0000", 2024-09-16T20:50:08.686 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018037399999999999 2024-09-16T20:50:08.686 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:08.686 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:08.686 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:08.686 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.686 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.686 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9973 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 653312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:08.686 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:50:08.686 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 17.976044216999998, 2024-09-16T20:50:08.686 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077905262000000003, 2024-09-16T20:50:08.686 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:08.687 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:08.687 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:08.687 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:08.687 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:08.687 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:08.687 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9973 2024-09-16T20:50:08.687 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.688 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:08.688 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.688 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:08.688 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:50:08.688 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:08.688 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.688 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.688 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:08.688 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:50:08.688 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:08.688 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.688 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.689 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:08.689 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:50:08.689 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:50:08.689 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.689 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.689 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:08.689 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642148+0000", 2024-09-16T20:50:08.689 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0219999999999999e-06 2024-09-16T20:50:08.689 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.690 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.690 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:08.690 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642154+0000", 2024-09-16T20:50:08.690 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8540000000000002e-06 2024-09-16T20:50:08.690 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.690 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.690 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:08.690 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642159+0000", 2024-09-16T20:50:08.690 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.045e-06 2024-09-16T20:50:08.691 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.691 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.691 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:08.691 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719875+0000", 2024-09-16T20:50:08.691 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077716116000000002 2024-09-16T20:50:08.691 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.691 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.691 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:08.691 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719934+0000", 2024-09-16T20:50:08.691 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8819e-05 2024-09-16T20:50:08.691 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.691 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.691 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:08.692 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720051+0000", 2024-09-16T20:50:08.692 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000116947 2024-09-16T20:50:08.692 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:08.692 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:08.692 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:08.692 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.692 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.692 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9974 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 658432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:08.692 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642166+0000", 2024-09-16T20:50:08.692 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 17.976024302999999, 2024-09-16T20:50:08.692 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.078179786000000001, 2024-09-16T20:50:08.692 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:08.692 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:08.693 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:08.693 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:08.693 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:08.693 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:08.693 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9974 2024-09-16T20:50:08.693 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.693 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:08.693 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.693 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:08.694 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642166+0000", 2024-09-16T20:50:08.694 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:08.694 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.694 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.694 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:08.694 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642166+0000", 2024-09-16T20:50:08.694 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:08.694 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.694 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.694 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:08.694 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647343+0000", 2024-09-16T20:50:08.694 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0051769440000000002 2024-09-16T20:50:08.694 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.695 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.695 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:08.695 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647348+0000", 2024-09-16T20:50:08.695 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3619999999999997e-06 2024-09-16T20:50:08.695 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.695 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.695 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:08.695 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647359+0000", 2024-09-16T20:50:08.695 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0929e-05 2024-09-16T20:50:08.695 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.696 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.696 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:08.696 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647366+0000", 2024-09-16T20:50:08.696 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0910000000000004e-06 2024-09-16T20:50:08.696 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.696 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.696 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:08.696 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720094+0000", 2024-09-16T20:50:08.696 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.072727403999999995 2024-09-16T20:50:08.696 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.696 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.697 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:08.697 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720173+0000", 2024-09-16T20:50:08.697 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9382000000000007e-05 2024-09-16T20:50:08.697 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.697 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.697 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:08.697 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720346+0000", 2024-09-16T20:50:08.697 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017267399999999999 2024-09-16T20:50:08.697 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:08.697 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:08.697 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:08.697 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.697 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.697 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9988 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 730112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:08.698 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647703+0000", 2024-09-16T20:50:08.698 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 17.970487310999999, 2024-09-16T20:50:08.698 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076703473999999994, 2024-09-16T20:50:08.698 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:08.698 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:08.698 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:08.698 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:08.698 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:08.698 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:08.698 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9988 2024-09-16T20:50:08.699 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.699 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:08.699 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.699 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:08.699 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647703+0000", 2024-09-16T20:50:08.699 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:08.699 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.699 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.699 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:08.699 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647703+0000", 2024-09-16T20:50:08.699 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:08.699 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.699 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.700 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:08.700 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647703+0000", 2024-09-16T20:50:08.700 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0699999999999998e-07 2024-09-16T20:50:08.700 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.700 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.700 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:08.700 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647705+0000", 2024-09-16T20:50:08.700 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.866e-06 2024-09-16T20:50:08.700 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.700 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.700 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:08.700 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647708+0000", 2024-09-16T20:50:08.700 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3170000000000002e-06 2024-09-16T20:50:08.700 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.701 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.701 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:08.701 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647711+0000", 2024-09-16T20:50:08.701 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6759999999999999e-06 2024-09-16T20:50:08.701 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.701 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.701 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:08.701 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724225+0000", 2024-09-16T20:50:08.701 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.07651434 2024-09-16T20:50:08.701 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.701 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.701 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:08.701 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724291+0000", 2024-09-16T20:50:08.702 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5629000000000005e-05 2024-09-16T20:50:08.702 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.702 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.702 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:08.702 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724406+0000", 2024-09-16T20:50:08.702 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000115239 2024-09-16T20:50:08.702 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:08.702 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:08.702 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:08.702 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.702 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.702 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9989 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 735232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:08.702 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647715+0000", 2024-09-16T20:50:08.703 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 17.970475573000002, 2024-09-16T20:50:08.703 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076871693000000005, 2024-09-16T20:50:08.703 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:08.703 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:08.703 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:08.703 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:08.703 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:08.703 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:08.703 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9989 2024-09-16T20:50:08.703 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.703 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:08.703 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.703 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:08.704 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647715+0000", 2024-09-16T20:50:08.704 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:08.704 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.704 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.704 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:08.704 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647715+0000", 2024-09-16T20:50:08.704 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:08.704 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.704 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.704 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:08.704 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647715+0000", 2024-09-16T20:50:08.704 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1400000000000001e-07 2024-09-16T20:50:08.705 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.705 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.705 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:08.705 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647716+0000", 2024-09-16T20:50:08.705 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2240000000000001e-06 2024-09-16T20:50:08.705 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.705 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.705 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:08.705 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647719+0000", 2024-09-16T20:50:08.705 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0769999999999999e-06 2024-09-16T20:50:08.705 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.705 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.705 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:08.706 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647722+0000", 2024-09-16T20:50:08.706 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.65e-06 2024-09-16T20:50:08.706 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.706 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.706 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:08.706 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724424+0000", 2024-09-16T20:50:08.706 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076701982000000002 2024-09-16T20:50:08.706 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.706 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.706 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:08.706 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724470+0000", 2024-09-16T20:50:08.706 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6589999999999999e-05 2024-09-16T20:50:08.706 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.706 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.707 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:08.707 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724586+0000", 2024-09-16T20:50:08.707 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000115956 2024-09-16T20:50:08.707 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:08.707 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:08.707 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:08.707 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.707 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.707 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9990 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 740352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:08.707 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647725+0000", 2024-09-16T20:50:08.707 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 17.970465219000001, 2024-09-16T20:50:08.707 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077029919000000002, 2024-09-16T20:50:08.707 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:08.707 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:08.708 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:08.708 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:08.708 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:08.708 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:08.708 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9990 2024-09-16T20:50:08.708 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.708 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:08.708 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.708 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:08.709 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647725+0000", 2024-09-16T20:50:08.709 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:08.709 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.709 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.709 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:08.709 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647725+0000", 2024-09-16T20:50:08.709 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:08.709 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.709 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.709 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:08.709 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647725+0000", 2024-09-16T20:50:08.709 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2399999999999999e-07 2024-09-16T20:50:08.709 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.710 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.710 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:08.710 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647726+0000", 2024-09-16T20:50:08.710 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.116e-06 2024-09-16T20:50:08.710 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.710 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.710 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:08.710 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647729+0000", 2024-09-16T20:50:08.710 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.937e-06 2024-09-16T20:50:08.710 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.710 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.710 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:08.710 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647732+0000", 2024-09-16T20:50:08.710 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8739999999999999e-06 2024-09-16T20:50:08.711 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.711 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.711 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:08.711 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724612+0000", 2024-09-16T20:50:08.711 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076880102000000006 2024-09-16T20:50:08.711 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.711 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.711 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:08.711 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724649+0000", 2024-09-16T20:50:08.711 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6738e-05 2024-09-16T20:50:08.711 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.711 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.711 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:08.711 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724755+0000", 2024-09-16T20:50:08.711 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010592800000000001 2024-09-16T20:50:08.712 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:08.712 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:08.712 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:08.712 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.712 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.712 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9991 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 745472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:08.712 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:50:08.712 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 17.970454910000001, 2024-09-16T20:50:08.712 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077257820000000005, 2024-09-16T20:50:08.713 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:08.713 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:08.713 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:08.713 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:08.713 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:08.713 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:08.713 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9991 2024-09-16T20:50:08.713 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.713 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:08.713 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.713 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:08.713 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:50:08.713 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:08.714 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.714 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.714 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:08.714 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:50:08.714 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:08.714 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.714 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.714 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:08.714 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:50:08.714 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1199999999999999e-07 2024-09-16T20:50:08.714 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.714 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.714 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:08.715 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647736+0000", 2024-09-16T20:50:08.715 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3699999999999999e-07 2024-09-16T20:50:08.715 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.715 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.715 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:08.715 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647739+0000", 2024-09-16T20:50:08.715 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9359999999999999e-06 2024-09-16T20:50:08.715 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.715 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.715 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:08.715 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647742+0000", 2024-09-16T20:50:08.716 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5720000000000001e-06 2024-09-16T20:50:08.716 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.716 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.716 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:08.716 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724774+0000", 2024-09-16T20:50:08.716 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077032457999999998 2024-09-16T20:50:08.716 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.716 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.716 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:08.716 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724818+0000", 2024-09-16T20:50:08.716 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3803000000000001e-05 2024-09-16T20:50:08.716 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.716 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.717 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:08.717 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724993+0000", 2024-09-16T20:50:08.717 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000174902 2024-09-16T20:50:08.717 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:08.717 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:08.717 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:08.717 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.717 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.717 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9992 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 750592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:08.717 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:50:08.717 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 17.970445321, 2024-09-16T20:50:08.717 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077553227000000002, 2024-09-16T20:50:08.717 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:08.718 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:08.718 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:08.718 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:08.718 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:08.718 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:08.718 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9992 2024-09-16T20:50:08.718 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.718 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:08.718 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.718 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:08.718 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:50:08.718 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:08.718 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.718 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.719 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:08.719 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:50:08.719 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:08.719 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.719 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.719 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:08.719 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:50:08.719 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0200000000000001e-07 2024-09-16T20:50:08.719 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.720 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.720 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:08.720 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647746+0000", 2024-09-16T20:50:08.720 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.088e-06 2024-09-16T20:50:08.720 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.720 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.720 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:08.720 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647749+0000", 2024-09-16T20:50:08.720 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9170000000000002e-06 2024-09-16T20:50:08.720 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.720 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.720 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:08.720 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647752+0000", 2024-09-16T20:50:08.721 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.774e-06 2024-09-16T20:50:08.721 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.721 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.721 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:08.721 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725042+0000", 2024-09-16T20:50:08.721 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077289952999999995 2024-09-16T20:50:08.721 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.721 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.721 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:08.721 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725113+0000", 2024-09-16T20:50:08.721 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0920999999999994e-05 2024-09-16T20:50:08.721 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.721 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.722 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:08.722 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725298+0000", 2024-09-16T20:50:08.722 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000185372 2024-09-16T20:50:08.722 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:08.722 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:08.722 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:08.722 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.722 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.722 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9993 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 755712~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:08.722 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647755+0000", 2024-09-16T20:50:08.722 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 17.970435295000001, 2024-09-16T20:50:08.722 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077725674999999994, 2024-09-16T20:50:08.723 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:08.723 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:08.723 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:08.723 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:08.723 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:08.723 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:08.723 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9993 2024-09-16T20:50:08.723 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.723 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:08.723 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.723 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:08.723 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647755+0000", 2024-09-16T20:50:08.723 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:08.724 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.724 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.724 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:08.724 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647755+0000", 2024-09-16T20:50:08.724 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:08.724 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.724 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.724 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:08.724 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653406+0000", 2024-09-16T20:50:08.724 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.005650876 2024-09-16T20:50:08.724 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.724 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.724 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:08.724 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653411+0000", 2024-09-16T20:50:08.725 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8899999999999998e-06 2024-09-16T20:50:08.725 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.725 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.725 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:08.725 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653423+0000", 2024-09-16T20:50:08.725 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2447e-05 2024-09-16T20:50:08.725 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.725 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.725 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:08.725 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653431+0000", 2024-09-16T20:50:08.725 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5730000000000002e-06 2024-09-16T20:50:08.725 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.726 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.726 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:08.726 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725333+0000", 2024-09-16T20:50:08.726 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.071902743000000005 2024-09-16T20:50:08.726 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.726 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.726 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:08.726 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725385+0000", 2024-09-16T20:50:08.726 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1585000000000002e-05 2024-09-16T20:50:08.726 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:08.726 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:08.726 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:08.726 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725481+0000", 2024-09-16T20:50:08.726 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5561000000000002e-05 2024-09-16T20:50:08.726 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:08.727 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:08.727 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:08.727 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:08.727 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:08.727 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:08.728 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:50:08.729 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:08.728+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:08.737 INFO:teuthology.orchestra.run.smithi139.stdout:5075 write 0x7646e00 thru 0x764f574 (0x8775 bytes) 2024-09-16T20:50:08.746 INFO:teuthology.orchestra.run.smithi139.stdout:5076 trunc from 0x764f600 to 0x58ce800 2024-09-16T20:50:08.786 INFO:teuthology.orchestra.run.smithi139.stdout:5077 write 0xe917800 thru 0xe91fe3f (0x8640 bytes) 2024-09-16T20:50:08.790 INFO:teuthology.orchestra.run.smithi139.stdout:5078 trunc from 0xe920000 to 0xc8bfa00 2024-09-16T20:50:08.805 INFO:teuthology.orchestra.run.smithi139.stdout:5082 read 0x74ba800 thru 0x74be40f (0x3c10 bytes) 2024-09-16T20:50:08.805 INFO:teuthology.orchestra.run.smithi139.stdout:5083 punch from 0xc726c00 to 0xc735400, (0xe800 bytes) 2024-09-16T20:50:08.808 INFO:teuthology.orchestra.run.smithi139.stdout:5084 trunc from 0xc8bfa00 to 0xd7f1800 2024-09-16T20:50:08.819 INFO:teuthology.orchestra.run.smithi139.stdout:5085 read 0x5c27200 thru 0x5c2720c (0xd bytes) 2024-09-16T20:50:08.824 INFO:teuthology.orchestra.run.smithi139.stdout:5086 read 0x3da4600 thru 0x3db308b (0xea8c bytes) 2024-09-16T20:50:08.825 INFO:teuthology.orchestra.run.smithi139.stdout:5087 clone 40 order 24 su 131072 sc 11 2024-09-16T20:50:08.829 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:08.827+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:08.856 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:08.857 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:50:08.857 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:50:08.857 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:08.866 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:50:08.930 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:08.929+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:09.014 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:09.014 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:50:09.014 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:50:09.014 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:50:09.014 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:09.022 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:50:09.066 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:09.065+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:09.131 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:09.131 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:50:09.131 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:50:09.131 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:50:09.131 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.132 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12548 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 628736~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:09.132 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757009+0000", 2024-09-16T20:50:09.132 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.372524459000001, 2024-09-16T20:50:09.132 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065743956000000006, 2024-09-16T20:50:09.132 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:09.132 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:09.132 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:09.132 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:09.132 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:09.132 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:09.132 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12548 2024-09-16T20:50:09.132 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.133 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:09.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.133 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:09.133 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757009+0000", 2024-09-16T20:50:09.133 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.133 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:09.133 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757009+0000", 2024-09-16T20:50:09.133 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.133 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:09.133 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757010+0000", 2024-09-16T20:50:09.134 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9900000000000001e-07 2024-09-16T20:50:09.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.134 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.134 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:09.134 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757012+0000", 2024-09-16T20:50:09.134 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2689999999999998e-06 2024-09-16T20:50:09.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.134 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.134 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:09.134 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757022+0000", 2024-09-16T20:50:09.134 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9699999999999994e-06 2024-09-16T20:50:09.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.134 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.134 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:09.135 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757028+0000", 2024-09-16T20:50:09.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3799999999999999e-06 2024-09-16T20:50:09.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.135 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.135 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:09.135 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822552+0000", 2024-09-16T20:50:09.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065524281000000004 2024-09-16T20:50:09.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.135 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.135 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:09.135 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822613+0000", 2024-09-16T20:50:09.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0430999999999998e-05 2024-09-16T20:50:09.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.136 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.136 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:09.136 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822753+0000", 2024-09-16T20:50:09.136 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000140226 2024-09-16T20:50:09.136 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.136 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:09.136 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.136 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.137 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.137 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12549 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 633856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:09.137 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:50:09.137 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.372495711999999, 2024-09-16T20:50:09.137 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065978518, 2024-09-16T20:50:09.137 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:09.137 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:09.137 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:09.137 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:09.137 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:09.137 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:09.137 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12549 2024-09-16T20:50:09.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.137 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:09.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.138 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:09.138 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:50:09.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.138 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:09.138 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:50:09.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.138 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:09.138 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:50:09.139 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9299999999999999e-07 2024-09-16T20:50:09.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.139 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.139 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:09.139 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757040+0000", 2024-09-16T20:50:09.139 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.677e-06 2024-09-16T20:50:09.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.139 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.139 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:09.139 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757045+0000", 2024-09-16T20:50:09.139 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8149999999999996e-06 2024-09-16T20:50:09.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.139 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.139 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:09.140 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757048+0000", 2024-09-16T20:50:09.140 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1e-06 2024-09-16T20:50:09.140 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.140 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.140 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:09.140 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822796+0000", 2024-09-16T20:50:09.140 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065748081 2024-09-16T20:50:09.140 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.140 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.140 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:09.140 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822867+0000", 2024-09-16T20:50:09.140 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0679000000000005e-05 2024-09-16T20:50:09.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.141 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.141 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:09.141 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823016+0000", 2024-09-16T20:50:09.141 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014987299999999999 2024-09-16T20:50:09.141 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.141 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:09.141 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.141 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.141 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12550 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 638976~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:09.141 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757060+0000", 2024-09-16T20:50:09.141 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.372473296999999, 2024-09-16T20:50:09.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066153564999999998, 2024-09-16T20:50:09.142 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:09.142 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:09.142 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:09.142 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:09.142 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:09.142 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:09.142 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12550 2024-09-16T20:50:09.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.142 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:09.142 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.142 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:09.143 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757060+0000", 2024-09-16T20:50:09.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.143 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:09.143 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757060+0000", 2024-09-16T20:50:09.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.143 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:09.143 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757061+0000", 2024-09-16T20:50:09.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.15e-07 2024-09-16T20:50:09.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.144 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:09.144 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757062+0000", 2024-09-16T20:50:09.144 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.663e-06 2024-09-16T20:50:09.144 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.144 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.144 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:09.144 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757066+0000", 2024-09-16T20:50:09.144 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.817e-06 2024-09-16T20:50:09.144 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.144 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.144 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:09.144 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757070+0000", 2024-09-16T20:50:09.145 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1509999999999997e-06 2024-09-16T20:50:09.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.145 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.145 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:09.145 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823044+0000", 2024-09-16T20:50:09.145 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065973992999999995 2024-09-16T20:50:09.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.145 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.145 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:09.145 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823082+0000", 2024-09-16T20:50:09.145 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8050999999999999e-05 2024-09-16T20:50:09.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.146 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:09.146 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823214+0000", 2024-09-16T20:50:09.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013157500000000001 2024-09-16T20:50:09.146 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.146 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:09.146 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.146 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12551 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 644096~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:09.146 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:50:09.146 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.372459598999999, 2024-09-16T20:50:09.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066391378000000001, 2024-09-16T20:50:09.147 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:09.147 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:09.147 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:09.147 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:09.147 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:09.147 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:09.147 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12551 2024-09-16T20:50:09.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.147 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:09.147 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.147 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:09.147 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:50:09.147 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.148 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.148 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:09.148 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:50:09.148 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.148 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.148 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.148 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:09.148 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:50:09.148 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2999999999999999e-07 2024-09-16T20:50:09.148 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.148 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.148 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:09.149 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757075+0000", 2024-09-16T20:50:09.149 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.097e-06 2024-09-16T20:50:09.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.149 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.149 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:09.149 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757082+0000", 2024-09-16T20:50:09.149 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9500000000000004e-06 2024-09-16T20:50:09.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.149 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.149 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:09.149 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757086+0000", 2024-09-16T20:50:09.149 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2229999999999999e-06 2024-09-16T20:50:09.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.150 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.150 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:09.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823262+0000", 2024-09-16T20:50:09.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066176078999999999 2024-09-16T20:50:09.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.150 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.150 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:09.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823336+0000", 2024-09-16T20:50:09.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3979000000000004e-05 2024-09-16T20:50:09.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.150 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.150 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:09.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823465+0000", 2024-09-16T20:50:09.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012982 2024-09-16T20:50:09.151 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.151 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:09.151 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.151 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12552 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 649216~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:09.151 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757089+0000", 2024-09-16T20:50:09.151 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.372444458, 2024-09-16T20:50:09.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066655617, 2024-09-16T20:50:09.151 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:09.151 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:09.151 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:09.151 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:09.151 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:09.152 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:09.152 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12552 2024-09-16T20:50:09.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.152 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:09.152 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.152 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:09.152 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757089+0000", 2024-09-16T20:50:09.152 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.153 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.153 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:09.153 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757089+0000", 2024-09-16T20:50:09.153 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.153 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.153 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.154 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:09.154 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759492+0000", 2024-09-16T20:50:09.154 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0024026109999999998 2024-09-16T20:50:09.154 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.154 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.154 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:09.155 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759497+0000", 2024-09-16T20:50:09.155 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1120000000000004e-06 2024-09-16T20:50:09.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.155 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.155 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:09.155 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759508+0000", 2024-09-16T20:50:09.155 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0981e-05 2024-09-16T20:50:09.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.156 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.156 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:09.156 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759516+0000", 2024-09-16T20:50:09.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7270000000000007e-06 2024-09-16T20:50:09.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.156 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.156 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:09.156 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823504+0000", 2024-09-16T20:50:09.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.063987984999999997 2024-09-16T20:50:09.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.156 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.156 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:09.157 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823579+0000", 2024-09-16T20:50:09.157 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5703000000000003e-05 2024-09-16T20:50:09.157 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.157 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.157 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:09.157 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823745+0000", 2024-09-16T20:50:09.157 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016549800000000001 2024-09-16T20:50:09.157 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.158 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:09.158 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.158 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.158 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.158 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12560 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 690176~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:09.158 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.759658+0000", 2024-09-16T20:50:09.158 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.369875256, 2024-09-16T20:50:09.158 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065858517000000005, 2024-09-16T20:50:09.158 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:09.159 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:09.159 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:09.159 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:09.159 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:09.159 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:09.159 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12560 2024-09-16T20:50:09.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.159 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:09.160 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.160 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:09.160 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759658+0000", 2024-09-16T20:50:09.160 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.160 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.160 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:09.160 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759658+0000", 2024-09-16T20:50:09.161 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.161 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.161 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.161 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:09.161 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759659+0000", 2024-09-16T20:50:09.161 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.17e-07 2024-09-16T20:50:09.161 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.161 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.162 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:09.162 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759660+0000", 2024-09-16T20:50:09.162 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0189999999999999e-06 2024-09-16T20:50:09.162 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.162 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.162 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:09.162 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759663+0000", 2024-09-16T20:50:09.162 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0010000000000002e-06 2024-09-16T20:50:09.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.163 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.163 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:09.163 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759665+0000", 2024-09-16T20:50:09.163 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5440000000000001e-06 2024-09-16T20:50:09.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.163 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.163 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:09.164 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825355+0000", 2024-09-16T20:50:09.164 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065690224000000005 2024-09-16T20:50:09.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.164 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.164 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:09.164 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825394+0000", 2024-09-16T20:50:09.164 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8810999999999998e-05 2024-09-16T20:50:09.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.164 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.165 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:09.165 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825517+0000", 2024-09-16T20:50:09.165 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012270099999999999 2024-09-16T20:50:09.165 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.165 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:09.165 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.165 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.165 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.166 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12561 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 695296~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:09.166 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.759668+0000", 2024-09-16T20:50:09.166 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.369865414, 2024-09-16T20:50:09.166 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066116999999999995, 2024-09-16T20:50:09.166 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:09.166 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:09.166 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:09.166 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:09.167 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:09.167 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:09.167 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12561 2024-09-16T20:50:09.167 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.167 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:09.167 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.167 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:09.168 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759668+0000", 2024-09-16T20:50:09.168 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.168 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.168 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:09.168 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759668+0000", 2024-09-16T20:50:09.168 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.168 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.169 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:09.169 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765379+0000", 2024-09-16T20:50:09.169 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057104410000000001 2024-09-16T20:50:09.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.169 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.169 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:09.169 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765383+0000", 2024-09-16T20:50:09.169 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2980000000000002e-06 2024-09-16T20:50:09.170 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.170 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.170 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:09.170 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765395+0000", 2024-09-16T20:50:09.170 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1671000000000001e-05 2024-09-16T20:50:09.170 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.170 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.170 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:09.171 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765403+0000", 2024-09-16T20:50:09.171 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1210000000000007e-06 2024-09-16T20:50:09.171 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.171 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.171 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:09.171 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825552+0000", 2024-09-16T20:50:09.171 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.060149498000000003 2024-09-16T20:50:09.171 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.171 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.171 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:09.171 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825625+0000", 2024-09-16T20:50:09.171 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2540999999999995e-05 2024-09-16T20:50:09.171 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.172 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.172 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:09.172 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825785+0000", 2024-09-16T20:50:09.172 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016043000000000001 2024-09-16T20:50:09.172 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.172 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:09.172 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.172 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.172 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.172 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12585 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 818176~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:09.172 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.765848+0000", 2024-09-16T20:50:09.172 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.363685443000001, 2024-09-16T20:50:09.173 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065843218999999995, 2024-09-16T20:50:09.173 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:09.173 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:09.173 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:09.173 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:09.173 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:09.173 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:09.173 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12585 2024-09-16T20:50:09.173 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.173 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:09.173 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.173 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:09.174 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765848+0000", 2024-09-16T20:50:09.174 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.174 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.174 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:09.174 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765848+0000", 2024-09-16T20:50:09.174 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.174 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.174 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:09.174 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768683+0000", 2024-09-16T20:50:09.174 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0028344519999999999 2024-09-16T20:50:09.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.174 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.175 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:09.175 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768686+0000", 2024-09-16T20:50:09.175 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8789999999999996e-06 2024-09-16T20:50:09.175 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.175 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.175 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:09.175 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768696+0000", 2024-09-16T20:50:09.175 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6709999999999994e-06 2024-09-16T20:50:09.175 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.175 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.175 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:09.175 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768702+0000", 2024-09-16T20:50:09.175 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2550000000000003e-06 2024-09-16T20:50:09.176 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.176 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.176 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:09.176 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.831499+0000", 2024-09-16T20:50:09.176 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.062797081000000005 2024-09-16T20:50:09.176 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.176 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.176 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:09.176 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.831538+0000", 2024-09-16T20:50:09.176 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8523999999999999e-05 2024-09-16T20:50:09.176 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.176 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.176 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:09.176 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.831691+0000", 2024-09-16T20:50:09.177 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015335699999999999 2024-09-16T20:50:09.177 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.177 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:09.177 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.177 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.177 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12597 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 879616~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:09.177 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.768883+0000", 2024-09-16T20:50:09.177 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.360650714999998, 2024-09-16T20:50:09.177 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065816527, 2024-09-16T20:50:09.177 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:09.177 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:09.177 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:09.178 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:09.178 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:09.178 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:09.178 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12597 2024-09-16T20:50:09.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.178 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:09.178 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.178 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:09.178 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768883+0000", 2024-09-16T20:50:09.178 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.178 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.178 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:09.179 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768883+0000", 2024-09-16T20:50:09.179 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.179 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.179 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:09.179 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771508+0000", 2024-09-16T20:50:09.179 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0026247589999999999 2024-09-16T20:50:09.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.179 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.179 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:09.179 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771513+0000", 2024-09-16T20:50:09.179 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9949999999999996e-06 2024-09-16T20:50:09.180 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.180 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.180 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:09.180 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771525+0000", 2024-09-16T20:50:09.180 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2033e-05 2024-09-16T20:50:09.180 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.180 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.180 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:09.180 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771532+0000", 2024-09-16T20:50:09.180 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4220000000000003e-06 2024-09-16T20:50:09.180 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.180 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.180 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:09.180 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.834532+0000", 2024-09-16T20:50:09.181 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.063000159 2024-09-16T20:50:09.181 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.181 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.181 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:09.181 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.834590+0000", 2024-09-16T20:50:09.181 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8218999999999999e-05 2024-09-16T20:50:09.181 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.181 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.181 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:09.181 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.834699+0000", 2024-09-16T20:50:09.181 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010894 2024-09-16T20:50:09.181 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.181 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:09.182 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.182 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.182 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.182 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12619 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 992256~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:09.182 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.774656+0000", 2024-09-16T20:50:09.182 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.354877941000002, 2024-09-16T20:50:09.182 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065715859000000001, 2024-09-16T20:50:09.182 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:09.182 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:09.182 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:09.182 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:09.182 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:09.182 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:09.183 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12619 2024-09-16T20:50:09.183 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.183 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:09.183 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.183 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:09.183 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774656+0000", 2024-09-16T20:50:09.183 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.183 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.183 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.183 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:09.183 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774656+0000", 2024-09-16T20:50:09.184 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.184 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.184 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:09.184 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774875+0000", 2024-09-16T20:50:09.184 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000218898 2024-09-16T20:50:09.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.184 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.184 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:09.184 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774879+0000", 2024-09-16T20:50:09.184 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0389999999999998e-06 2024-09-16T20:50:09.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.184 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.185 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:09.185 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774890+0000", 2024-09-16T20:50:09.185 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1250000000000001e-05 2024-09-16T20:50:09.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.185 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.185 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:09.185 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774897+0000", 2024-09-16T20:50:09.185 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7619999999999998e-06 2024-09-16T20:50:09.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.185 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.185 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:09.185 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840139+0000", 2024-09-16T20:50:09.185 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065242438 2024-09-16T20:50:09.186 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.186 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.186 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:09.186 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840186+0000", 2024-09-16T20:50:09.186 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6894999999999997e-05 2024-09-16T20:50:09.186 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.186 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.186 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:09.186 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840372+0000", 2024-09-16T20:50:09.186 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000185577 2024-09-16T20:50:09.186 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.186 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:09.186 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.186 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.187 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.187 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12620 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 997376~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:09.187 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.774908+0000", 2024-09-16T20:50:09.187 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.354625720000001, 2024-09-16T20:50:09.187 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065782754999999998, 2024-09-16T20:50:09.187 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:09.187 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:09.187 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:09.187 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:09.187 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:09.187 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:09.187 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12620 2024-09-16T20:50:09.187 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.188 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:09.188 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.188 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:09.188 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774908+0000", 2024-09-16T20:50:09.188 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.188 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.188 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.188 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:09.188 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774908+0000", 2024-09-16T20:50:09.188 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.188 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.188 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.188 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:09.188 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775102+0000", 2024-09-16T20:50:09.189 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000194098 2024-09-16T20:50:09.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.189 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.189 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:09.189 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775105+0000", 2024-09-16T20:50:09.189 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8739999999999999e-06 2024-09-16T20:50:09.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.189 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.189 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:09.189 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775111+0000", 2024-09-16T20:50:09.189 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9519999999999999e-06 2024-09-16T20:50:09.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.189 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.190 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:09.190 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775115+0000", 2024-09-16T20:50:09.190 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.002e-06 2024-09-16T20:50:09.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.190 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.190 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:09.191 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840409+0000", 2024-09-16T20:50:09.191 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065294195999999999 2024-09-16T20:50:09.191 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.191 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.191 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:09.191 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840507+0000", 2024-09-16T20:50:09.191 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7662000000000001e-05 2024-09-16T20:50:09.191 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.191 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.191 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:09.191 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840691+0000", 2024-09-16T20:50:09.191 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000183971 2024-09-16T20:50:09.191 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.191 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:09.192 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.192 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.192 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.192 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12621 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1002496~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:09.192 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775123+0000", 2024-09-16T20:50:09.192 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.354411024000001, 2024-09-16T20:50:09.192 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065829586999999995, 2024-09-16T20:50:09.192 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:09.192 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:09.192 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:09.192 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:09.192 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:09.192 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:09.193 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12621 2024-09-16T20:50:09.193 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.193 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:09.193 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.193 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:09.193 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775123+0000", 2024-09-16T20:50:09.193 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.193 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.193 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.193 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:09.193 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775123+0000", 2024-09-16T20:50:09.193 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.193 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.194 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.194 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:09.194 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775348+0000", 2024-09-16T20:50:09.194 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000225295 2024-09-16T20:50:09.194 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.194 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.194 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:09.194 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775352+0000", 2024-09-16T20:50:09.194 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0810000000000004e-06 2024-09-16T20:50:09.194 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.194 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.194 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:09.195 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775361+0000", 2024-09-16T20:50:09.195 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7399999999999993e-06 2024-09-16T20:50:09.195 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.195 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.195 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:09.195 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775367+0000", 2024-09-16T20:50:09.195 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5200000000000003e-06 2024-09-16T20:50:09.195 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.195 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.195 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:09.195 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840730+0000", 2024-09-16T20:50:09.196 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065362803999999997 2024-09-16T20:50:09.196 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.196 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.196 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:09.196 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840801+0000", 2024-09-16T20:50:09.196 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1006000000000004e-05 2024-09-16T20:50:09.196 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.196 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.196 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:09.196 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840952+0000", 2024-09-16T20:50:09.196 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015114100000000001 2024-09-16T20:50:09.196 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.196 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:09.196 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.197 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.197 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.197 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12622 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1007616~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:09.197 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775381+0000", 2024-09-16T20:50:09.197 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.354152190000001, 2024-09-16T20:50:09.197 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065822581000000005, 2024-09-16T20:50:09.197 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:09.197 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:09.197 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:09.197 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:09.197 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:09.197 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:09.197 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12622 2024-09-16T20:50:09.198 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.198 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:09.198 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.198 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:09.198 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775381+0000", 2024-09-16T20:50:09.198 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.198 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.198 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.198 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:09.198 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775381+0000", 2024-09-16T20:50:09.198 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.198 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.198 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.198 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:09.199 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775563+0000", 2024-09-16T20:50:09.199 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018153700000000001 2024-09-16T20:50:09.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.199 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.199 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:09.199 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775566+0000", 2024-09-16T20:50:09.199 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9620000000000002e-06 2024-09-16T20:50:09.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.199 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.199 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:09.199 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775572+0000", 2024-09-16T20:50:09.199 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8529999999999997e-06 2024-09-16T20:50:09.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.200 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.200 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:09.200 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775576+0000", 2024-09-16T20:50:09.200 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8920000000000002e-06 2024-09-16T20:50:09.200 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.200 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.200 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:09.200 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840990+0000", 2024-09-16T20:50:09.200 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065414599000000004 2024-09-16T20:50:09.200 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.200 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.200 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:09.200 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841068+0000", 2024-09-16T20:50:09.201 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8157000000000004e-05 2024-09-16T20:50:09.201 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.201 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.201 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:09.201 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841204+0000", 2024-09-16T20:50:09.201 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013558099999999999 2024-09-16T20:50:09.201 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.201 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:09.201 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.201 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.201 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.201 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12623 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1012736~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:09.201 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775583+0000", 2024-09-16T20:50:09.201 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.353950751999999, 2024-09-16T20:50:09.202 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065929087999999997, 2024-09-16T20:50:09.202 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:09.202 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:09.202 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:09.202 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:09.202 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:09.202 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:09.202 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12623 2024-09-16T20:50:09.202 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.202 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:09.202 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.202 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:09.202 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775583+0000", 2024-09-16T20:50:09.202 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.203 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.203 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:09.203 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775583+0000", 2024-09-16T20:50:09.203 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.203 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.203 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:09.203 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775804+0000", 2024-09-16T20:50:09.203 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00022116900000000001 2024-09-16T20:50:09.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.204 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.204 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:09.204 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775807+0000", 2024-09-16T20:50:09.204 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9529999999999999e-06 2024-09-16T20:50:09.204 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.204 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.204 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:09.204 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775814+0000", 2024-09-16T20:50:09.205 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7490000000000001e-06 2024-09-16T20:50:09.205 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.205 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.205 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:09.205 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775818+0000", 2024-09-16T20:50:09.205 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1999999999999996e-06 2024-09-16T20:50:09.205 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.205 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.205 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:09.205 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841267+0000", 2024-09-16T20:50:09.205 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065449127999999995 2024-09-16T20:50:09.205 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.206 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.206 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:09.206 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841344+0000", 2024-09-16T20:50:09.206 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6546999999999997e-05 2024-09-16T20:50:09.206 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.206 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.206 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:09.206 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841512+0000", 2024-09-16T20:50:09.206 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016834200000000001 2024-09-16T20:50:09.206 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.206 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:09.207 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.207 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.207 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.207 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12624 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1017856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:09.207 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:50:09.207 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.353707612000001, 2024-09-16T20:50:09.207 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065945866000000006, 2024-09-16T20:50:09.207 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:09.207 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:09.207 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:09.208 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:09.208 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:09.208 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:09.208 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12624 2024-09-16T20:50:09.208 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.208 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:09.208 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.208 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:09.208 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:50:09.208 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.208 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.209 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.209 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:09.209 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:50:09.209 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.209 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.209 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:09.209 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:50:09.209 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-16T20:50:09.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.209 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.210 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:09.210 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775828+0000", 2024-09-16T20:50:09.210 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3999999999999999e-06 2024-09-16T20:50:09.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.210 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.210 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:09.210 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775831+0000", 2024-09-16T20:50:09.210 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.478e-06 2024-09-16T20:50:09.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.210 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.211 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:09.211 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775834+0000", 2024-09-16T20:50:09.211 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9440000000000001e-06 2024-09-16T20:50:09.211 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.211 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.211 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:09.211 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841547+0000", 2024-09-16T20:50:09.211 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065712801000000001 2024-09-16T20:50:09.211 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.211 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.212 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:09.212 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841618+0000", 2024-09-16T20:50:09.212 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1151999999999999e-05 2024-09-16T20:50:09.212 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.212 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.212 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:09.212 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841772+0000", 2024-09-16T20:50:09.212 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000153721 2024-09-16T20:50:09.212 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.212 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:09.212 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.212 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.212 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.213 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12625 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1022976~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:09.213 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775840+0000", 2024-09-16T20:50:09.213 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.353694003000001, 2024-09-16T20:50:09.213 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066165451, 2024-09-16T20:50:09.213 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:09.213 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:09.213 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:09.213 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:09.213 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:09.213 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:09.213 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12625 2024-09-16T20:50:09.213 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.213 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:09.214 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.214 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:09.214 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775840+0000", 2024-09-16T20:50:09.214 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.214 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.214 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.214 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:09.214 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775840+0000", 2024-09-16T20:50:09.214 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.214 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.214 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.214 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:09.215 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776014+0000", 2024-09-16T20:50:09.215 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017441099999999999 2024-09-16T20:50:09.215 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.215 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.215 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:09.215 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776018+0000", 2024-09-16T20:50:09.215 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6509999999999999e-06 2024-09-16T20:50:09.215 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.215 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.215 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:09.215 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776026+0000", 2024-09-16T20:50:09.215 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.028e-06 2024-09-16T20:50:09.216 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.216 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.216 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:09.216 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776031+0000", 2024-09-16T20:50:09.216 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8400000000000002e-06 2024-09-16T20:50:09.216 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.216 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.216 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:09.216 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841801+0000", 2024-09-16T20:50:09.216 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065770472999999996 2024-09-16T20:50:09.216 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.217 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.217 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:09.217 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841847+0000", 2024-09-16T20:50:09.217 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5558e-05 2024-09-16T20:50:09.217 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.217 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.217 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:09.217 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842005+0000", 2024-09-16T20:50:09.217 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015849000000000001 2024-09-16T20:50:09.217 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.217 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:09.217 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.218 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.218 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.218 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12626 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1028096~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:09.218 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776041+0000", 2024-09-16T20:50:09.218 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.353492704000001, 2024-09-16T20:50:09.218 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066225260999999994, 2024-09-16T20:50:09.218 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:09.218 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:09.218 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:09.218 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:09.219 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:09.219 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:09.219 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12626 2024-09-16T20:50:09.219 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.219 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:09.219 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.219 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:09.219 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776041+0000", 2024-09-16T20:50:09.219 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.219 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.219 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.219 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:09.220 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776041+0000", 2024-09-16T20:50:09.220 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.220 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.220 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.220 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:09.220 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776263+0000", 2024-09-16T20:50:09.220 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00022193899999999999 2024-09-16T20:50:09.220 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.220 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.220 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:09.220 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776268+0000", 2024-09-16T20:50:09.221 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8049999999999997e-06 2024-09-16T20:50:09.221 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.221 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.221 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:09.221 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776280+0000", 2024-09-16T20:50:09.221 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2635e-05 2024-09-16T20:50:09.221 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.221 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.222 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:09.222 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776288+0000", 2024-09-16T20:50:09.222 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8199999999999997e-06 2024-09-16T20:50:09.222 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.222 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.222 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:09.222 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842041+0000", 2024-09-16T20:50:09.222 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065752506000000002 2024-09-16T20:50:09.222 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.222 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.222 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:09.223 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842123+0000", 2024-09-16T20:50:09.223 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2393999999999998e-05 2024-09-16T20:50:09.223 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.223 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.223 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:09.223 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842266+0000", 2024-09-16T20:50:09.223 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014316200000000001 2024-09-16T20:50:09.223 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.223 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:09.223 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.224 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.224 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.224 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12627 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1033216~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:09.224 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776301+0000", 2024-09-16T20:50:09.224 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.353232744, 2024-09-16T20:50:09.224 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066187814999999997, 2024-09-16T20:50:09.224 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:09.224 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:09.224 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:09.224 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:09.224 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:09.225 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:09.225 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12627 2024-09-16T20:50:09.225 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.225 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:09.225 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.225 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:09.225 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776301+0000", 2024-09-16T20:50:09.225 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.225 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.225 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.225 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:09.225 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776301+0000", 2024-09-16T20:50:09.226 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.226 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.226 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.226 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:09.226 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776473+0000", 2024-09-16T20:50:09.226 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017222400000000001 2024-09-16T20:50:09.226 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.226 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.226 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:09.226 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776477+0000", 2024-09-16T20:50:09.226 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7019999999999999e-06 2024-09-16T20:50:09.227 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.227 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.227 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:09.227 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776486+0000", 2024-09-16T20:50:09.227 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2119999999999992e-06 2024-09-16T20:50:09.227 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.227 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.227 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:09.227 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776492+0000", 2024-09-16T20:50:09.227 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8089999999999998e-06 2024-09-16T20:50:09.227 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.227 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.228 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:09.228 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842301+0000", 2024-09-16T20:50:09.228 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065809042999999998 2024-09-16T20:50:09.228 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.228 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.228 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:09.228 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842362+0000", 2024-09-16T20:50:09.228 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1260000000000006e-05 2024-09-16T20:50:09.228 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.228 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.228 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:09.228 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842489+0000", 2024-09-16T20:50:09.228 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012656500000000001 2024-09-16T20:50:09.228 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.229 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:09.229 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.229 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.229 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.229 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12628 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1038336~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:09.229 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776502+0000", 2024-09-16T20:50:09.229 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.353032098, 2024-09-16T20:50:09.229 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066283426000000006, 2024-09-16T20:50:09.229 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:09.229 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:09.229 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:09.229 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:09.229 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:09.229 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:09.230 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12628 2024-09-16T20:50:09.230 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.230 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:09.230 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.230 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:09.230 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776502+0000", 2024-09-16T20:50:09.230 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.230 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.230 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.230 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:09.230 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776502+0000", 2024-09-16T20:50:09.231 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.231 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.231 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.231 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:09.231 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776716+0000", 2024-09-16T20:50:09.231 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000214596 2024-09-16T20:50:09.231 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.231 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.231 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:09.231 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776723+0000", 2024-09-16T20:50:09.232 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4000000000000003e-06 2024-09-16T20:50:09.232 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.232 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.232 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:09.232 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776731+0000", 2024-09-16T20:50:09.232 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6129999999999998e-06 2024-09-16T20:50:09.232 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.232 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.232 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:09.232 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776737+0000", 2024-09-16T20:50:09.232 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1970000000000001e-06 2024-09-16T20:50:09.233 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.233 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.233 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:09.233 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842532+0000", 2024-09-16T20:50:09.233 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065795127999999994 2024-09-16T20:50:09.233 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.233 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.233 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:09.233 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842599+0000", 2024-09-16T20:50:09.233 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6711000000000006e-05 2024-09-16T20:50:09.233 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.233 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.234 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:09.234 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842785+0000", 2024-09-16T20:50:09.234 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018578099999999999 2024-09-16T20:50:09.234 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.234 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:09.234 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.234 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.234 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.234 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12629 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1043456~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:09.234 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776748+0000", 2024-09-16T20:50:09.234 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.352785721, 2024-09-16T20:50:09.234 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066303788000000002, 2024-09-16T20:50:09.234 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:09.234 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:09.234 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:09.235 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:09.235 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:09.235 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:09.235 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12629 2024-09-16T20:50:09.235 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.235 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:09.235 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.235 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:09.235 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776748+0000", 2024-09-16T20:50:09.235 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.236 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.236 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.236 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:09.236 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776748+0000", 2024-09-16T20:50:09.236 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.236 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.236 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.236 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:09.236 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776934+0000", 2024-09-16T20:50:09.236 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018598399999999999 2024-09-16T20:50:09.236 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.236 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.236 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:09.237 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776939+0000", 2024-09-16T20:50:09.237 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1000000000000003e-06 2024-09-16T20:50:09.237 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.237 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.237 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:09.237 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776954+0000", 2024-09-16T20:50:09.237 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5221e-05 2024-09-16T20:50:09.237 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.237 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.237 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:09.237 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776962+0000", 2024-09-16T20:50:09.237 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.554e-06 2024-09-16T20:50:09.237 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.237 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.238 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:09.238 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842813+0000", 2024-09-16T20:50:09.238 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065851006000000004 2024-09-16T20:50:09.238 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.238 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.238 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:09.238 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842886+0000", 2024-09-16T20:50:09.238 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3663000000000003e-05 2024-09-16T20:50:09.238 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.238 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.238 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:09.238 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.843052+0000", 2024-09-16T20:50:09.239 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016526 2024-09-16T20:50:09.239 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.239 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:09.239 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.239 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.239 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:09.239 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:09.241 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:50:09.241 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:09.239+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:09.341 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:09.340+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:09.389 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:09.390 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:50:09.390 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:50:09.390 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:09.398 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:50:09.482 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:09.481+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:09.543 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:09.541+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:09.546 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:09.546 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:50:09.546 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:50:09.546 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:50:09.546 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:09.555 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:50:09.643 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:09.642+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:09.655 INFO:teuthology.orchestra.run.smithi139.stdout:truncating image image_client.0-clone39 from 0xd7f1800 (overlap 0x4103600) to 0x1a01400 2024-09-16T20:50:09.683 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T20:50:09.711 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:09.711 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:50:09.711 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:50:09.711 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:50:09.711 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.711 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5286.0:10643 3.17 3.f6273fb7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-16T20:50:09.711 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:50:09.712 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 60.918913035999999, 2024-09-16T20:50:09.712 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.76672347200000002, 2024-09-16T20:50:09.712 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:09.712 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:09.712 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:09.712 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:09.712 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5286", 2024-09-16T20:50:09.712 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/4135433989", 2024-09-16T20:50:09.712 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 10643 2024-09-16T20:50:09.712 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.712 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:09.712 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.712 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:09.713 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:50:09.713 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.713 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.713 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.713 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:09.713 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:50:09.713 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.713 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.713 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.713 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:09.713 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790489+0000", 2024-09-16T20:50:09.713 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5910000000000001e-06 2024-09-16T20:50:09.713 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.714 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.714 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:09.714 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790493+0000", 2024-09-16T20:50:09.714 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2669999999999998e-06 2024-09-16T20:50:09.717 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.717 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.717 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:09.717 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790500+0000", 2024-09-16T20:50:09.717 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3780000000000003e-06 2024-09-16T20:50:09.717 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.717 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.718 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:09.718 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790505+0000", 2024-09-16T20:50:09.718 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1560000000000003e-06 2024-09-16T20:50:09.718 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.718 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.718 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:09.718 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790559+0000", 2024-09-16T20:50:09.718 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3494e-05 2024-09-16T20:50:09.718 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.718 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.718 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:09.718 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790566+0000", 2024-09-16T20:50:09.718 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7649999999999993e-06 2024-09-16T20:50:09.719 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.719 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.719 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:09.719 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:09.557198+0000", 2024-09-16T20:50:09.719 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.76663136200000004 2024-09-16T20:50:09.719 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.719 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.719 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:09.719 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:09.557211+0000", 2024-09-16T20:50:09.719 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3458999999999999e-05 2024-09-16T20:50:09.719 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.719 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:09.719 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.720 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.720 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.720 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5512.0:450 3.2 3:44e0cffa:::rbd_header.110ef0d7a535:head [watch reconnect cookie 140263298018912 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:50:09.720 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:50:09.720 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 33.146890122999999, 2024-09-16T20:50:09.720 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0084581939999999, 2024-09-16T20:50:09.720 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:09.720 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:09.720 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:09.720 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:09.720 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5512", 2024-09-16T20:50:09.720 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:50:09.720 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 450 2024-09-16T20:50:09.721 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.721 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:09.721 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.721 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:09.721 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:50:09.721 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.721 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.721 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.721 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:09.721 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:50:09.721 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.721 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.721 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.721 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:09.722 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562512+0000", 2024-09-16T20:50:09.722 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.009e-06 2024-09-16T20:50:09.722 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.722 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.722 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:09.722 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562517+0000", 2024-09-16T20:50:09.722 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9339999999999997e-06 2024-09-16T20:50:09.722 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.722 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.722 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:09.722 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562530+0000", 2024-09-16T20:50:09.722 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3495e-05 2024-09-16T20:50:09.722 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.722 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.722 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:09.723 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562536+0000", 2024-09-16T20:50:09.723 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5690000000000004e-06 2024-09-16T20:50:09.723 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.723 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.723 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:09.723 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562591+0000", 2024-09-16T20:50:09.723 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5761999999999999e-05 2024-09-16T20:50:09.723 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.723 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.723 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:50:09.723 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562599+0000", 2024-09-16T20:50:09.723 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9179999999999994e-06 2024-09-16T20:50:09.723 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.723 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.723 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:09.724 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.565971+0000", 2024-09-16T20:50:09.724 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003371546 2024-09-16T20:50:09.724 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.724 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.724 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:09.724 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566019+0000", 2024-09-16T20:50:09.724 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7849000000000002e-05 2024-09-16T20:50:09.724 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.724 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.724 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:09.724 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.569382+0000", 2024-09-16T20:50:09.724 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.003362833 2024-09-16T20:50:09.724 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.724 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.725 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:09.725 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.569529+0000", 2024-09-16T20:50:09.725 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000147823 2024-09-16T20:50:09.725 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.725 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.725 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:09.725 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570969+0000", 2024-09-16T20:50:09.725 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001439456 2024-09-16T20:50:09.725 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.725 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:09.725 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.725 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.725 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.726 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:14996 3.2 3:44e0cffa:::rbd_header.110ef0d7a535:head [watch reconnect cookie 140213436269840 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:50:09.726 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:50:09.726 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 33.146841711, 2024-09-16T20:50:09.726 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.008813943, 2024-09-16T20:50:09.726 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:09.726 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:09.726 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:09.726 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:09.726 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:50:09.726 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:09.726 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 14996 2024-09-16T20:50:09.726 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.726 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:09.726 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.727 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:09.727 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:50:09.727 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.727 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.727 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.727 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:09.727 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:50:09.727 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.727 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.727 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.727 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:09.727 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562560+0000", 2024-09-16T20:50:09.727 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.06e-07 2024-09-16T20:50:09.727 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.728 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.728 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:09.728 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562563+0000", 2024-09-16T20:50:09.728 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.495e-06 2024-09-16T20:50:09.728 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.728 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.728 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:09.728 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562571+0000", 2024-09-16T20:50:09.728 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1950000000000003e-06 2024-09-16T20:50:09.728 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.728 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.728 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:09.729 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562579+0000", 2024-09-16T20:50:09.729 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2649999999999999e-06 2024-09-16T20:50:09.729 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.729 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.729 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:09.729 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562673+0000", 2024-09-16T20:50:09.729 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4118999999999998e-05 2024-09-16T20:50:09.729 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.729 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.729 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:50:09.729 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562677+0000", 2024-09-16T20:50:09.729 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5179999999999998e-06 2024-09-16T20:50:09.729 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.729 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.730 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:09.730 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566461+0000", 2024-09-16T20:50:09.730 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0037837259999999998 2024-09-16T20:50:09.730 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.730 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.730 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:09.730 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566468+0000", 2024-09-16T20:50:09.730 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3350000000000001e-06 2024-09-16T20:50:09.730 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.730 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.730 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:09.730 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570306+0000", 2024-09-16T20:50:09.730 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.003837997 2024-09-16T20:50:09.730 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.731 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.731 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:09.731 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570392+0000", 2024-09-16T20:50:09.731 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5872999999999996e-05 2024-09-16T20:50:09.731 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.731 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.731 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:09.731 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571373+0000", 2024-09-16T20:50:09.731 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00098071399999999993 2024-09-16T20:50:09.731 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.731 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:09.731 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.731 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.732 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.732 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5255.0:1931 3.2 3:47882819:::rbd_data.110eb5f95afc.0000000000000009:head [read 217088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:50:09.732 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:50:09.732 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 33.146802960999999, 2024-09-16T20:50:09.732 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0082752880000001, 2024-09-16T20:50:09.732 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:09.732 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:09.732 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:09.732 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:09.732 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5255", 2024-09-16T20:50:09.732 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:50:09.732 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1931 2024-09-16T20:50:09.733 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.733 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:09.733 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.733 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:09.733 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:50:09.733 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.733 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.733 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.733 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:09.733 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:50:09.733 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.733 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.733 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.734 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:09.734 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:50:09.734 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9699999999999996e-07 2024-09-16T20:50:09.734 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.734 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.734 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:09.734 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562601+0000", 2024-09-16T20:50:09.734 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8600000000000001e-06 2024-09-16T20:50:09.734 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.734 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.734 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:09.734 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562608+0000", 2024-09-16T20:50:09.734 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9e-06 2024-09-16T20:50:09.734 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.735 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.735 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:09.735 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562615+0000", 2024-09-16T20:50:09.735 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8179999999999998e-06 2024-09-16T20:50:09.735 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.735 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.735 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:09.735 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562703+0000", 2024-09-16T20:50:09.735 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7811000000000003e-05 2024-09-16T20:50:09.735 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.735 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.735 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:50:09.735 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562706+0000", 2024-09-16T20:50:09.735 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4920000000000002e-06 2024-09-16T20:50:09.736 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.736 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.736 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:09.736 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566685+0000", 2024-09-16T20:50:09.736 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0039792780000000002 2024-09-16T20:50:09.736 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.736 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.736 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:09.736 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566693+0000", 2024-09-16T20:50:09.736 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4730000000000003e-06 2024-09-16T20:50:09.736 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.736 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.736 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:09.737 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570737+0000", 2024-09-16T20:50:09.737 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0040436749999999 2024-09-16T20:50:09.737 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.737 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.737 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:09.737 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570873+0000", 2024-09-16T20:50:09.737 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000136484 2024-09-16T20:50:09.737 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.737 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:09.737 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.737 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.737 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.737 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5255.0:1939 3.2 3:47882819:::rbd_data.110eb5f95afc.0000000000000009:head [read 233472~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:50:09.737 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:50:09.738 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 33.146648095000003, 2024-09-16T20:50:09.738 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0084657589999999, 2024-09-16T20:50:09.738 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:09.738 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:09.738 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:09.738 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:09.738 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5255", 2024-09-16T20:50:09.738 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:50:09.738 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1939 2024-09-16T20:50:09.738 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.738 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:09.738 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.738 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:09.739 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:50:09.739 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.739 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.739 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.739 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:09.739 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:50:09.739 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.739 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.739 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.739 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:09.739 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:50:09.739 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5199999999999997e-07 2024-09-16T20:50:09.739 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.739 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.740 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:09.740 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562757+0000", 2024-09-16T20:50:09.740 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4979999999999998e-06 2024-09-16T20:50:09.740 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.740 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.740 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:09.740 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562764+0000", 2024-09-16T20:50:09.740 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1470000000000004e-06 2024-09-16T20:50:09.740 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.740 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.740 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:09.740 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562768+0000", 2024-09-16T20:50:09.740 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7929999999999999e-06 2024-09-16T20:50:09.741 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.741 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.741 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:09.741 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562805+0000", 2024-09-16T20:50:09.741 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7477e-05 2024-09-16T20:50:09.741 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.741 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.741 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting_for_map not empty", 2024-09-16T20:50:09.741 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562810+0000", 2024-09-16T20:50:09.741 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0819999999999998e-06 2024-09-16T20:50:09.741 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.741 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.741 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:09.741 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566717+0000", 2024-09-16T20:50:09.742 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0039072600000000001 2024-09-16T20:50:09.742 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.742 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.742 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:09.742 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566724+0000", 2024-09-16T20:50:09.742 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4880000000000004e-06 2024-09-16T20:50:09.742 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.742 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.742 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:09.742 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571009+0000", 2024-09-16T20:50:09.742 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.004284677 2024-09-16T20:50:09.742 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.742 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.742 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:09.743 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571218+0000", 2024-09-16T20:50:09.743 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00020978500000000001 2024-09-16T20:50:09.743 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.743 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:09.743 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.743 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.743 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.743 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:11020 3.1f 3:fee2d1dd:::rbd_data.110ea93691bb.000000000000000e:23 [read 955392~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:09.743 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.693811+0000", 2024-09-16T20:50:09.743 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.015589437999999, 2024-09-16T20:50:09.743 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056833005999999998, 2024-09-16T20:50:09.743 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:09.743 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:09.744 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:09.744 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:09.744 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:09.744 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:09.744 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 11020 2024-09-16T20:50:09.744 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.744 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:09.744 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.744 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:09.744 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693811+0000", 2024-09-16T20:50:09.744 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.744 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.744 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.744 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:09.745 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693811+0000", 2024-09-16T20:50:09.745 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.745 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.745 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.745 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:09.745 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693812+0000", 2024-09-16T20:50:09.745 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6100000000000001e-07 2024-09-16T20:50:09.745 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.745 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.745 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:09.745 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693815+0000", 2024-09-16T20:50:09.745 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7389999999999999e-06 2024-09-16T20:50:09.745 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.745 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.746 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:09.746 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693823+0000", 2024-09-16T20:50:09.746 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0439999999999996e-06 2024-09-16T20:50:09.746 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.746 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.746 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:09.746 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693828+0000", 2024-09-16T20:50:09.746 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7440000000000004e-06 2024-09-16T20:50:09.746 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.746 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.746 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:09.746 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750357+0000", 2024-09-16T20:50:09.746 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056529192999999998 2024-09-16T20:50:09.746 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.746 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.747 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:09.747 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750429+0000", 2024-09-16T20:50:09.747 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1557000000000006e-05 2024-09-16T20:50:09.747 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.747 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.747 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:09.747 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750644+0000", 2024-09-16T20:50:09.747 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00021526800000000001 2024-09-16T20:50:09.747 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.747 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:09.747 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.747 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.747 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.747 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:11021 3.1f 3:fee2d1dd:::rbd_data.110ea93691bb.000000000000000e:23 [read 960512~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:09.747 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.693837+0000", 2024-09-16T20:50:09.748 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.01556355, 2024-09-16T20:50:09.748 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057155138000000001, 2024-09-16T20:50:09.748 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:09.748 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:09.748 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:09.748 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:09.748 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:09.748 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:09.748 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 11021 2024-09-16T20:50:09.748 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.748 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:09.748 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.748 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:09.748 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693837+0000", 2024-09-16T20:50:09.749 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.749 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.749 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.749 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:09.749 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693837+0000", 2024-09-16T20:50:09.749 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.749 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.749 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.749 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:09.749 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699566+0000", 2024-09-16T20:50:09.749 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057283639999999997 2024-09-16T20:50:09.749 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.749 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.749 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:09.750 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699571+0000", 2024-09-16T20:50:09.750 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.604e-06 2024-09-16T20:50:09.750 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.750 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.750 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:09.750 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699583+0000", 2024-09-16T20:50:09.750 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1372999999999999e-05 2024-09-16T20:50:09.750 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.750 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.750 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:09.750 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699591+0000", 2024-09-16T20:50:09.750 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5259999999999993e-06 2024-09-16T20:50:09.750 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.751 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.751 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:09.751 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750674+0000", 2024-09-16T20:50:09.751 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051083154999999998 2024-09-16T20:50:09.751 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.751 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.751 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:09.751 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750745+0000", 2024-09-16T20:50:09.751 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0933999999999995e-05 2024-09-16T20:50:09.751 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.751 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.751 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:09.751 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750992+0000", 2024-09-16T20:50:09.751 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00024718199999999998 2024-09-16T20:50:09.751 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.751 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:09.752 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.752 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.752 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.752 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8789 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2314240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:09.752 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057183+0000", 2024-09-16T20:50:09.752 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.6522176319999993, 2024-09-16T20:50:09.752 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056736416999999997, 2024-09-16T20:50:09.752 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:09.752 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:09.752 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:09.752 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:09.752 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:09.752 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:09.752 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8789 2024-09-16T20:50:09.753 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.753 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:09.753 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.753 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:09.753 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057183+0000", 2024-09-16T20:50:09.753 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.753 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.753 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.753 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:09.753 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057183+0000", 2024-09-16T20:50:09.753 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.753 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.753 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.753 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:09.753 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057183+0000", 2024-09-16T20:50:09.754 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-16T20:50:09.754 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.754 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.754 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:09.754 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057186+0000", 2024-09-16T20:50:09.754 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3889999999999999e-06 2024-09-16T20:50:09.754 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.754 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.754 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:09.754 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057192+0000", 2024-09-16T20:50:09.754 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8950000000000003e-06 2024-09-16T20:50:09.754 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.754 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.754 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:09.754 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057196+0000", 2024-09-16T20:50:09.755 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5159999999999996e-06 2024-09-16T20:50:09.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.755 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.755 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:09.755 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.113826+0000", 2024-09-16T20:50:09.755 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056629298000000002 2024-09-16T20:50:09.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.755 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.755 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:09.755 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.113866+0000", 2024-09-16T20:50:09.755 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0423000000000002e-05 2024-09-16T20:50:09.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.755 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.755 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:09.755 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.113920+0000", 2024-09-16T20:50:09.756 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3526000000000002e-05 2024-09-16T20:50:09.756 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.756 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:09.756 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.756 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.756 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.756 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8790 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2590720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:09.756 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057257+0000", 2024-09-16T20:50:09.756 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.6521442299999993, 2024-09-16T20:50:09.756 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056791169000000002, 2024-09-16T20:50:09.756 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:09.756 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:09.757 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:09.757 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:09.757 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:09.757 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:09.757 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8790 2024-09-16T20:50:09.757 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.757 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:09.757 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.757 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:09.758 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057257+0000", 2024-09-16T20:50:09.758 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.758 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.758 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.758 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:09.758 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057257+0000", 2024-09-16T20:50:09.758 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.758 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.758 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.758 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:09.758 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057257+0000", 2024-09-16T20:50:09.758 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6600000000000002e-07 2024-09-16T20:50:09.758 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.759 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.759 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:09.759 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057260+0000", 2024-09-16T20:50:09.759 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7240000000000001e-06 2024-09-16T20:50:09.759 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.759 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.759 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:09.759 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057267+0000", 2024-09-16T20:50:09.759 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0600000000000002e-06 2024-09-16T20:50:09.759 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.759 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.759 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:09.759 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057272+0000", 2024-09-16T20:50:09.759 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0390000000000004e-06 2024-09-16T20:50:09.760 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.760 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.760 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:09.760 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.113950+0000", 2024-09-16T20:50:09.760 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056677989999999998 2024-09-16T20:50:09.760 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.760 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.760 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:09.760 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.113993+0000", 2024-09-16T20:50:09.760 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3205999999999999e-05 2024-09-16T20:50:09.760 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.760 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.761 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:09.761 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114048+0000", 2024-09-16T20:50:09.761 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4684e-05 2024-09-16T20:50:09.761 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.761 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:09.761 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.761 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.761 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.761 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8791 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2332672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:09.761 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057279+0000", 2024-09-16T20:50:09.761 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.6521218149999992, 2024-09-16T20:50:09.761 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056890067000000002, 2024-09-16T20:50:09.761 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:09.762 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:09.762 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:09.762 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:09.762 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:09.762 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:09.762 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8791 2024-09-16T20:50:09.762 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.762 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:09.762 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.762 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:09.762 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057279+0000", 2024-09-16T20:50:09.762 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.762 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.763 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.763 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:09.763 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057279+0000", 2024-09-16T20:50:09.763 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.763 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.763 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.763 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:09.763 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057279+0000", 2024-09-16T20:50:09.763 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.15e-07 2024-09-16T20:50:09.763 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.763 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.763 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:09.763 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057281+0000", 2024-09-16T20:50:09.764 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8080000000000001e-06 2024-09-16T20:50:09.764 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.764 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.764 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:09.764 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057287+0000", 2024-09-16T20:50:09.764 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5999999999999997e-06 2024-09-16T20:50:09.764 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.764 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.764 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:09.764 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057291+0000", 2024-09-16T20:50:09.764 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5990000000000004e-06 2024-09-16T20:50:09.765 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.765 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.765 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:09.765 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114078+0000", 2024-09-16T20:50:09.765 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056786246999999998 2024-09-16T20:50:09.765 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.765 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.765 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:09.765 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114118+0000", 2024-09-16T20:50:09.765 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9898999999999999e-05 2024-09-16T20:50:09.765 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.765 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.766 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:09.766 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114169+0000", 2024-09-16T20:50:09.766 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1499000000000003e-05 2024-09-16T20:50:09.766 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.766 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:09.766 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.766 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.766 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.766 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8792 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2609152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:09.766 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:50:09.766 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.652103833, 2024-09-16T20:50:09.766 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057003524999999999, 2024-09-16T20:50:09.767 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:09.767 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:09.767 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:09.767 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:09.767 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:09.767 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:09.767 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8792 2024-09-16T20:50:09.767 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.767 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:09.767 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.767 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:09.767 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:50:09.767 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.768 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.768 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.768 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:09.768 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:50:09.768 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.768 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.768 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.768 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:09.768 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:50:09.768 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0400000000000002e-07 2024-09-16T20:50:09.768 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.769 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.769 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:09.769 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057299+0000", 2024-09-16T20:50:09.769 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7460000000000001e-06 2024-09-16T20:50:09.769 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.769 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.769 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:09.769 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057304+0000", 2024-09-16T20:50:09.769 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3210000000000004e-06 2024-09-16T20:50:09.769 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.769 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.770 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:09.770 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057310+0000", 2024-09-16T20:50:09.770 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4360000000000001e-06 2024-09-16T20:50:09.770 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.770 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.770 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:09.770 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114200+0000", 2024-09-16T20:50:09.770 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056889954 2024-09-16T20:50:09.770 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.770 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.770 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:09.770 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114243+0000", 2024-09-16T20:50:09.770 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2998999999999999e-05 2024-09-16T20:50:09.771 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.771 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.771 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:09.771 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114300+0000", 2024-09-16T20:50:09.771 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7765e-05 2024-09-16T20:50:09.771 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.771 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:09.771 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.771 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.771 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.771 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8794 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2351104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:09.771 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057327+0000", 2024-09-16T20:50:09.771 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.6520735090000009, 2024-09-16T20:50:09.771 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057101130999999999, 2024-09-16T20:50:09.771 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:09.772 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:09.772 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:09.772 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:09.772 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:09.772 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:09.772 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8794 2024-09-16T20:50:09.772 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.772 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:09.772 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.772 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:09.772 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057327+0000", 2024-09-16T20:50:09.772 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.772 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.773 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.773 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:09.773 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057327+0000", 2024-09-16T20:50:09.773 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.773 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.773 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.773 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:09.773 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057328+0000", 2024-09-16T20:50:09.773 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7399999999999999e-07 2024-09-16T20:50:09.773 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.773 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.773 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:09.773 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057330+0000", 2024-09-16T20:50:09.773 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2199999999999999e-06 2024-09-16T20:50:09.774 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.774 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.774 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:09.774 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057336+0000", 2024-09-16T20:50:09.774 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1240000000000002e-06 2024-09-16T20:50:09.774 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.774 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.774 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:09.774 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057341+0000", 2024-09-16T20:50:09.774 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8470000000000003e-06 2024-09-16T20:50:09.774 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.774 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.774 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:09.774 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114334+0000", 2024-09-16T20:50:09.774 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056993267 2024-09-16T20:50:09.775 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.775 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.775 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:09.775 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114376+0000", 2024-09-16T20:50:09.775 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1638999999999999e-05 2024-09-16T20:50:09.775 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.775 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.775 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:09.775 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114428+0000", 2024-09-16T20:50:09.775 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2660000000000001e-05 2024-09-16T20:50:09.775 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.775 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:09.775 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.775 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.775 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.776 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8798 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2297856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:09.776 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:50:09.776 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.6520537819999994, 2024-09-16T20:50:09.776 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057277041000000001, 2024-09-16T20:50:09.776 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:09.776 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:09.776 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:09.776 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:09.776 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:09.776 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:09.776 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8798 2024-09-16T20:50:09.776 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.777 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:09.777 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.777 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:09.777 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:50:09.777 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.777 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.777 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.777 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:09.777 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:50:09.777 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.777 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.778 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.778 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:09.778 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:50:09.778 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2800000000000003e-07 2024-09-16T20:50:09.778 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.778 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.778 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:09.778 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057354+0000", 2024-09-16T20:50:09.778 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7780000000000002e-06 2024-09-16T20:50:09.778 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.778 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.778 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:09.778 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057360+0000", 2024-09-16T20:50:09.778 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0419999999999999e-06 2024-09-16T20:50:09.779 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.779 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.779 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:09.779 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057365+0000", 2024-09-16T20:50:09.779 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7929999999999997e-06 2024-09-16T20:50:09.779 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.779 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.779 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:09.779 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114460+0000", 2024-09-16T20:50:09.779 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057094662999999997 2024-09-16T20:50:09.779 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.779 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.779 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:09.779 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114532+0000", 2024-09-16T20:50:09.780 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2881999999999998e-05 2024-09-16T20:50:09.780 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.780 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.780 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:09.780 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114624+0000", 2024-09-16T20:50:09.780 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1555e-05 2024-09-16T20:50:09.780 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.780 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:09.780 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.780 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.780 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.780 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8800 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2316288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:09.780 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:50:09.780 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.6520298429999993, 2024-09-16T20:50:09.781 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.05746946, 2024-09-16T20:50:09.781 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:09.781 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:09.781 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:09.781 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:09.781 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:09.781 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:09.781 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8800 2024-09-16T20:50:09.781 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.781 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:09.781 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.781 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:09.781 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:50:09.782 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.782 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.782 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:09.782 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:50:09.782 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.782 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.782 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:09.782 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:50:09.782 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5199999999999998e-07 2024-09-16T20:50:09.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.782 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.783 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:09.783 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057373+0000", 2024-09-16T20:50:09.783 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.79e-06 2024-09-16T20:50:09.783 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.783 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.783 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:09.783 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057379+0000", 2024-09-16T20:50:09.783 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6699999999999999e-06 2024-09-16T20:50:09.783 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.783 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.783 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:09.783 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057386+0000", 2024-09-16T20:50:09.784 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3170000000000004e-06 2024-09-16T20:50:09.784 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.784 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.784 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:09.784 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114655+0000", 2024-09-16T20:50:09.784 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057269420000000001 2024-09-16T20:50:09.784 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.784 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.784 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:09.784 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114734+0000", 2024-09-16T20:50:09.784 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8084000000000007e-05 2024-09-16T20:50:09.784 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.784 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.784 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:09.785 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114840+0000", 2024-09-16T20:50:09.785 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000106827 2024-09-16T20:50:09.785 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.785 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:09.785 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.785 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.785 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.785 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8801 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2334720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:09.785 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:50:09.785 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.652008812, 2024-09-16T20:50:09.785 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057594682000000001, 2024-09-16T20:50:09.785 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:09.785 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:09.785 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:09.786 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:09.786 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:09.786 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:09.786 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8801 2024-09-16T20:50:09.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.786 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:09.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.786 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:09.786 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:50:09.786 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.786 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:09.786 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:50:09.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.787 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.787 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:09.787 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:50:09.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6899999999999998e-07 2024-09-16T20:50:09.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.787 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.787 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:09.787 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057394+0000", 2024-09-16T20:50:09.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8300000000000001e-06 2024-09-16T20:50:09.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.787 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.787 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:09.787 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057400+0000", 2024-09-16T20:50:09.788 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6500000000000001e-06 2024-09-16T20:50:09.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.788 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.788 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:09.788 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057404+0000", 2024-09-16T20:50:09.788 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4669999999999998e-06 2024-09-16T20:50:09.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.788 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.788 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:09.788 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114874+0000", 2024-09-16T20:50:09.788 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057469602000000002 2024-09-16T20:50:09.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.788 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.789 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:09.789 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114933+0000", 2024-09-16T20:50:09.789 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9500000000000003e-05 2024-09-16T20:50:09.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.789 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.789 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:09.789 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114987+0000", 2024-09-16T20:50:09.789 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3263999999999997e-05 2024-09-16T20:50:09.789 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.789 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:09.789 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.789 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.790 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8802 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2353152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:09.790 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:50:09.790 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.6519601270000006, 2024-09-16T20:50:09.790 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057714079000000001, 2024-09-16T20:50:09.790 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:09.790 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:09.790 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:09.790 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:09.790 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:09.790 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:09.790 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8802 2024-09-16T20:50:09.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.790 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:09.790 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.790 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:09.791 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:50:09.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.791 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.791 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:09.791 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:50:09.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.791 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.791 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:09.791 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:50:09.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4999999999999998e-07 2024-09-16T20:50:09.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.791 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.791 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:09.792 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057446+0000", 2024-09-16T20:50:09.792 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4240000000000003e-06 2024-09-16T20:50:09.792 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.792 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.792 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:09.792 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057452+0000", 2024-09-16T20:50:09.792 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8379999999999996e-06 2024-09-16T20:50:09.792 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.793 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.793 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:09.793 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057460+0000", 2024-09-16T20:50:09.793 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9629999999999998e-06 2024-09-16T20:50:09.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.793 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.793 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:09.793 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115015+0000", 2024-09-16T20:50:09.793 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057554389999999997 2024-09-16T20:50:09.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.793 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.793 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:09.793 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115082+0000", 2024-09-16T20:50:09.793 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7733999999999999e-05 2024-09-16T20:50:09.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.794 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:09.794 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115155+0000", 2024-09-16T20:50:09.794 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2280000000000006e-05 2024-09-16T20:50:09.794 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.794 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:09.794 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.794 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8804 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2555904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:09.794 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:50:09.794 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.6519338109999993, 2024-09-16T20:50:09.794 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057833862, 2024-09-16T20:50:09.795 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:09.795 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:09.795 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:09.795 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:09.795 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:09.795 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:09.795 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8804 2024-09-16T20:50:09.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.795 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:09.795 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.795 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:09.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:50:09.796 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.796 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.796 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:09.796 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:50:09.796 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.796 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.796 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:09.796 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:50:09.796 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7899999999999999e-07 2024-09-16T20:50:09.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.797 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.797 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:09.797 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057469+0000", 2024-09-16T20:50:09.797 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.001e-06 2024-09-16T20:50:09.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.797 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.797 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:09.797 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057475+0000", 2024-09-16T20:50:09.797 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7629999999999997e-06 2024-09-16T20:50:09.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.797 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.797 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:09.797 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057480+0000", 2024-09-16T20:50:09.798 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8069999999999999e-06 2024-09-16T20:50:09.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.798 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.798 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:09.798 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115183+0000", 2024-09-16T20:50:09.798 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057703364 2024-09-16T20:50:09.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.798 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.798 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:09.798 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115227+0000", 2024-09-16T20:50:09.798 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3903999999999998e-05 2024-09-16T20:50:09.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.798 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.799 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:09.799 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115301+0000", 2024-09-16T20:50:09.799 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3543999999999999e-05 2024-09-16T20:50:09.799 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.799 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:09.799 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.799 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.799 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8805 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2574336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:09.799 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057487+0000", 2024-09-16T20:50:09.799 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.6519134829999995, 2024-09-16T20:50:09.799 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058021027000000003, 2024-09-16T20:50:09.799 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:09.800 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:09.800 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:09.800 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:09.800 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:09.800 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:09.800 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8805 2024-09-16T20:50:09.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.800 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:09.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.800 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:09.800 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057487+0000", 2024-09-16T20:50:09.800 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.801 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:09.801 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057487+0000", 2024-09-16T20:50:09.801 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.801 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.801 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.801 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:09.801 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057488+0000", 2024-09-16T20:50:09.801 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5199999999999998e-07 2024-09-16T20:50:09.801 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.801 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.801 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:09.801 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057490+0000", 2024-09-16T20:50:09.801 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.939e-06 2024-09-16T20:50:09.802 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.802 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.802 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:09.802 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057495+0000", 2024-09-16T20:50:09.802 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6439999999999997e-06 2024-09-16T20:50:09.802 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.802 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.802 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:09.802 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057500+0000", 2024-09-16T20:50:09.802 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8119999999999998e-06 2024-09-16T20:50:09.802 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.802 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.803 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:09.803 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115341+0000", 2024-09-16T20:50:09.803 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057841341999999997 2024-09-16T20:50:09.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.803 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.803 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:09.803 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115422+0000", 2024-09-16T20:50:09.803 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1003999999999999e-05 2024-09-16T20:50:09.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.803 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.803 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:09.803 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115508+0000", 2024-09-16T20:50:09.803 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5933999999999994e-05 2024-09-16T20:50:09.804 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.804 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:09.804 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.804 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8807 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2592768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:09.804 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057505+0000", 2024-09-16T20:50:09.804 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.6518952730000009, 2024-09-16T20:50:09.804 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058186633000000001, 2024-09-16T20:50:09.804 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:09.804 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:09.804 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:09.804 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:09.805 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:09.805 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:09.805 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8807 2024-09-16T20:50:09.805 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.805 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:09.805 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.805 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:09.805 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057505+0000", 2024-09-16T20:50:09.805 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.805 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.805 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.805 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:09.806 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057505+0000", 2024-09-16T20:50:09.806 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.806 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.806 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.806 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:09.806 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057506+0000", 2024-09-16T20:50:09.806 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.27e-07 2024-09-16T20:50:09.806 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.806 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.806 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:09.806 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057508+0000", 2024-09-16T20:50:09.806 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7400000000000001e-06 2024-09-16T20:50:09.806 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.807 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.807 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:09.807 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057513+0000", 2024-09-16T20:50:09.807 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.468e-06 2024-09-16T20:50:09.807 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.807 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.807 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:09.807 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057517+0000", 2024-09-16T20:50:09.807 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4329999999999997e-06 2024-09-16T20:50:09.807 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.807 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.807 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:09.808 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115538+0000", 2024-09-16T20:50:09.808 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058020355000000003 2024-09-16T20:50:09.808 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.808 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.808 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:09.808 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115616+0000", 2024-09-16T20:50:09.808 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7998999999999996e-05 2024-09-16T20:50:09.808 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.808 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.808 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:09.808 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115692+0000", 2024-09-16T20:50:09.808 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6310999999999995e-05 2024-09-16T20:50:09.808 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.809 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:09.809 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.809 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.809 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8808 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2611200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:09.809 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:50:09.809 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.6518780119999992, 2024-09-16T20:50:09.809 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058354046, 2024-09-16T20:50:09.809 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:09.809 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:09.809 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:09.809 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:09.809 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:09.810 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:09.810 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8808 2024-09-16T20:50:09.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.810 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:09.810 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.810 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:09.810 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:50:09.810 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.810 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.810 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:09.810 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:50:09.810 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:09.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.811 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.811 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:09.811 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:50:09.811 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2500000000000001e-07 2024-09-16T20:50:09.811 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.811 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.811 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:09.811 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057525+0000", 2024-09-16T20:50:09.811 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5549999999999999e-06 2024-09-16T20:50:09.811 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.811 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.811 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:09.811 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057530+0000", 2024-09-16T20:50:09.811 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5489999999999997e-06 2024-09-16T20:50:09.811 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.812 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.812 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:09.812 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057535+0000", 2024-09-16T20:50:09.812 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5709999999999999e-06 2024-09-16T20:50:09.812 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.812 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.812 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:09.812 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115722+0000", 2024-09-16T20:50:09.812 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058187323999999999 2024-09-16T20:50:09.812 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.812 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.812 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:09.812 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115799+0000", 2024-09-16T20:50:09.814 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6445000000000004e-05 2024-09-16T20:50:09.814 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:09.814 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:09.814 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:09.814 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115877+0000", 2024-09-16T20:50:09.814 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8276999999999996e-05 2024-09-16T20:50:09.814 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.814 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:09.814 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.815 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:09.815 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:09.815 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:09.816 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:09.815+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:09.817 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:50:09.917 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:09.916+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:09.924 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:09.924 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:50:09.924 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:50:09.924 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:09.933 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:50:10.018 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:10.017+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:10.080 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:10.080 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:50:10.080 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:50:10.080 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:50:10.080 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:10.089 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:50:10.130 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:10.129+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:10.220 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:10.219+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:10.238 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:10.238 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:50:10.238 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:50:10.238 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:50:10.238 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.238 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2206 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1046528~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:50:10.238 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:50:10.238 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 107.900353794, 2024-09-16T20:50:10.238 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050147714000000003, 2024-09-16T20:50:10.239 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:10.239 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:10.239 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:10.239 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:10.239 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:10.239 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:10.239 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2206 2024-09-16T20:50:10.239 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.239 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:10.239 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.239 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:10.239 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:50:10.239 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:10.240 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.240 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.240 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:10.240 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:50:10.240 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:10.240 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.240 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.240 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:10.240 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:50:10.240 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8599999999999999e-07 2024-09-16T20:50:10.240 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.240 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.240 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:10.240 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336024+0000", 2024-09-16T20:50:10.241 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.934e-06 2024-09-16T20:50:10.241 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.241 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.241 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:10.241 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336037+0000", 2024-09-16T20:50:10.241 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2496e-05 2024-09-16T20:50:10.241 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.241 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.241 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:10.241 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336042+0000", 2024-09-16T20:50:10.241 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4809999999999997e-06 2024-09-16T20:50:10.241 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.241 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.242 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:10.242 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385980+0000", 2024-09-16T20:50:10.242 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049938376999999999 2024-09-16T20:50:10.242 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.242 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.242 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:10.242 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.386069+0000", 2024-09-16T20:50:10.242 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8368999999999994e-05 2024-09-16T20:50:10.242 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.242 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.242 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:10.242 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.386170+0000", 2024-09-16T20:50:10.242 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000100671 2024-09-16T20:50:10.242 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:10.243 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:10.243 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:10.243 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.243 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.243 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5439 3.a 3:5789f5c3:::rbd_data.110ed9ca540.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 1181696~60416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:10.243 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:50:10.243 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 107.371665214, 2024-09-16T20:50:10.243 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10102045599999999, 2024-09-16T20:50:10.243 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:10.243 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:10.243 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:10.243 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:10.243 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:10.244 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:10.244 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5439 2024-09-16T20:50:10.244 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.244 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:10.244 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.244 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:10.244 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:50:10.244 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:10.244 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.244 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.244 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:10.244 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:50:10.244 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:10.245 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.245 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.245 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:10.245 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864711+0000", 2024-09-16T20:50:10.245 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7899999999999997e-07 2024-09-16T20:50:10.245 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.245 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.245 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:10.245 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864760+0000", 2024-09-16T20:50:10.245 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8803e-05 2024-09-16T20:50:10.245 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.245 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.245 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:10.245 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864769+0000", 2024-09-16T20:50:10.246 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0999999999999993e-06 2024-09-16T20:50:10.246 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.246 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.246 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:10.246 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864774+0000", 2024-09-16T20:50:10.246 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8919999999999999e-06 2024-09-16T20:50:10.246 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.246 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.246 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:10.246 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864815+0000", 2024-09-16T20:50:10.246 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1307000000000003e-05 2024-09-16T20:50:10.246 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.246 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.246 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:10.247 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864893+0000", 2024-09-16T20:50:10.247 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7581000000000001e-05 2024-09-16T20:50:10.247 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.247 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.247 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 1", 2024-09-16T20:50:10.247 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864996+0000", 2024-09-16T20:50:10.247 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102725 2024-09-16T20:50:10.247 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.247 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.247 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:10.247 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965442+0000", 2024-09-16T20:50:10.247 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100446012 2024-09-16T20:50:10.247 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.247 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.248 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:10.248 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965618+0000", 2024-09-16T20:50:10.248 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00017653100000000001 2024-09-16T20:50:10.248 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.248 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.248 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:10.248 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965682+0000", 2024-09-16T20:50:10.248 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3905999999999994e-05 2024-09-16T20:50:10.248 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.248 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.248 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:10.248 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965731+0000", 2024-09-16T20:50:10.248 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8819999999999997e-05 2024-09-16T20:50:10.249 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:10.249 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:10.249 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:10.249 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.249 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.249 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4600.0:5439 3.a e37/23 ondisk, result = 0)", 2024-09-16T20:50:10.249 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:50:10.249 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 107.370669066, 2024-09-16T20:50:10.249 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099742634999999996, 2024-09-16T20:50:10.249 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:10.249 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:10.249 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:10.249 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:10.249 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.250 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:10.250 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:50:10.250 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:10.250 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.250 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.250 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:10.250 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:50:10.250 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:10.250 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.250 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.250 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:10.250 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:50:10.250 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0900000000000001e-07 2024-09-16T20:50:10.251 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.251 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.251 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:10.251 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865711+0000", 2024-09-16T20:50:10.251 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3739999999999998e-06 2024-09-16T20:50:10.251 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.251 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.251 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:10.251 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965343+0000", 2024-09-16T20:50:10.251 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099632526999999999 2024-09-16T20:50:10.251 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.251 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.251 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:10.251 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965357+0000", 2024-09-16T20:50:10.252 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3713999999999999e-05 2024-09-16T20:50:10.252 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.252 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.252 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:10.252 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965426+0000", 2024-09-16T20:50:10.252 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8745999999999995e-05 2024-09-16T20:50:10.252 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.252 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.252 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:10.252 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965434+0000", 2024-09-16T20:50:10.252 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6910000000000003e-06 2024-09-16T20:50:10.252 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.252 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.253 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:10.253 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965449+0000", 2024-09-16T20:50:10.253 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4874e-05 2024-09-16T20:50:10.253 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:10.253 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:10.253 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:10.253 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.253 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.253 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4882.0:2508 3.12 3:4992771f:::rbd_object_map.110e7c8851c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T20:50:10.253 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:50:10.253 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 83.754865518000003, 2024-09-16T20:50:10.253 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.125007011, 2024-09-16T20:50:10.253 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:10.253 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:10.254 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:10.254 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:10.254 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4882", 2024-09-16T20:50:10.254 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2436923223", 2024-09-16T20:50:10.254 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2508 2024-09-16T20:50:10.254 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.254 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:10.254 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.254 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:10.254 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:50:10.254 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:10.254 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.254 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.254 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:10.255 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:50:10.255 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:10.255 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.255 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.255 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:10.255 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481511+0000", 2024-09-16T20:50:10.255 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0699999999999996e-07 2024-09-16T20:50:10.255 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.255 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.255 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:10.255 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481515+0000", 2024-09-16T20:50:10.255 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0609999999999997e-06 2024-09-16T20:50:10.255 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.255 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.256 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:10.256 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481523+0000", 2024-09-16T20:50:10.256 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1310000000000006e-06 2024-09-16T20:50:10.256 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.256 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.256 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:10.256 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481528+0000", 2024-09-16T20:50:10.256 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4900000000000002e-06 2024-09-16T20:50:10.256 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.256 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.256 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:10.256 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481565+0000", 2024-09-16T20:50:10.256 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7066000000000001e-05 2024-09-16T20:50:10.257 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.257 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.257 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:10.257 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481628+0000", 2024-09-16T20:50:10.257 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3118000000000001e-05 2024-09-16T20:50:10.257 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.257 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.257 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 2", 2024-09-16T20:50:10.257 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481822+0000", 2024-09-16T20:50:10.257 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019446200000000001 2024-09-16T20:50:10.257 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.257 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.257 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:10.258 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605988+0000", 2024-09-16T20:50:10.258 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.124165888 2024-09-16T20:50:10.258 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.258 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.258 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:10.258 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606427+0000", 2024-09-16T20:50:10.258 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00043854899999999999 2024-09-16T20:50:10.258 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.258 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.258 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:10.258 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606476+0000", 2024-09-16T20:50:10.259 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8971999999999998e-05 2024-09-16T20:50:10.259 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.259 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.259 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:10.259 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606517+0000", 2024-09-16T20:50:10.259 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1467000000000001e-05 2024-09-16T20:50:10.259 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:10.259 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:10.259 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:10.259 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.259 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.259 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4882.0:2508 3.12 e57/55 ondisk, result = 0)", 2024-09-16T20:50:10.260 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:50:10.260 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 83.753877017999997, 2024-09-16T20:50:10.260 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12349711400000001, 2024-09-16T20:50:10.260 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:10.260 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:10.260 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:10.260 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:10.260 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.260 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:10.260 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:50:10.260 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:10.260 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.260 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.260 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:10.261 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:50:10.261 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:10.261 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.261 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.261 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:10.261 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:50:10.261 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.06e-07 2024-09-16T20:50:10.261 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.261 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.261 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:10.261 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482502+0000", 2024-09-16T20:50:10.261 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.01e-06 2024-09-16T20:50:10.261 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.261 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.262 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:10.262 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605772+0000", 2024-09-16T20:50:10.262 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.123269687 2024-09-16T20:50:10.262 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.262 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.262 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:10.262 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605791+0000", 2024-09-16T20:50:10.262 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8912000000000001e-05 2024-09-16T20:50:10.262 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.262 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.262 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:10.262 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605974+0000", 2024-09-16T20:50:10.262 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000182836 2024-09-16T20:50:10.262 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.263 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.263 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:10.263 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605982+0000", 2024-09-16T20:50:10.263 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1699999999999997e-06 2024-09-16T20:50:10.263 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.263 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.263 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:10.263 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605996+0000", 2024-09-16T20:50:10.263 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3893000000000001e-05 2024-09-16T20:50:10.263 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:10.263 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:10.263 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:10.263 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.264 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.264 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4882.0:2512 3.f 3:f4d895b7:::rbd_data.110e7c8851c.0000000000000000:head [read 16384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T20:50:10.264 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:50:10.264 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 83.753439987999997, 2024-09-16T20:50:10.264 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12318483, 2024-09-16T20:50:10.264 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:10.264 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:10.264 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:10.264 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:10.264 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4882", 2024-09-16T20:50:10.264 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2436923223", 2024-09-16T20:50:10.264 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2512 2024-09-16T20:50:10.264 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.264 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:10.265 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.265 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:10.265 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:50:10.265 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:10.265 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.265 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.265 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:10.265 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:50:10.265 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:10.265 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.265 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.265 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:10.265 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:50:10.265 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3499999999999995e-07 2024-09-16T20:50:10.266 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.266 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.266 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:10.266 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482943+0000", 2024-09-16T20:50:10.266 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3450000000000002e-06 2024-09-16T20:50:10.266 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.266 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.266 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:10.266 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605772+0000", 2024-09-16T20:50:10.266 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.122829796 2024-09-16T20:50:10.266 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.266 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.266 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:10.267 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605792+0000", 2024-09-16T20:50:10.267 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9916999999999999e-05 2024-09-16T20:50:10.267 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.267 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.267 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:10.267 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605917+0000", 2024-09-16T20:50:10.267 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00012500799999999999 2024-09-16T20:50:10.267 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.267 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.267 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:10.267 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606004+0000", 2024-09-16T20:50:10.267 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6242999999999994e-05 2024-09-16T20:50:10.267 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.267 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.268 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:10.268 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606120+0000", 2024-09-16T20:50:10.268 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000116786 2024-09-16T20:50:10.268 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:10.268 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:10.268 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:10.268 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.268 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.268 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:11702 3.15 3:acf33c8b:::rbd_data.110e747f0779.000000000000001a:head [set-alloc-hint object_size 262144 write_size 262144,write 143360~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:50:10.268 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:50:10.268 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 66.098447566000004, 2024-09-16T20:50:10.268 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.084801043000000006, 2024-09-16T20:50:10.268 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:10.269 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:10.269 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:10.269 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:10.269 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:50:10.269 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:50:10.269 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 11702 2024-09-16T20:50:10.269 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.269 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:10.269 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.269 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:10.269 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:50:10.269 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:10.269 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.270 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.270 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:10.270 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:50:10.270 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:10.270 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.270 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.270 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:10.270 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137929+0000", 2024-09-16T20:50:10.270 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8400000000000002e-07 2024-09-16T20:50:10.270 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.270 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.270 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:10.270 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137937+0000", 2024-09-16T20:50:10.270 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0260000000000007e-06 2024-09-16T20:50:10.271 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.271 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.271 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:10.271 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137952+0000", 2024-09-16T20:50:10.271 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4659e-05 2024-09-16T20:50:10.271 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.271 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.271 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:10.271 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137960+0000", 2024-09-16T20:50:10.271 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0169999999999996e-06 2024-09-16T20:50:10.271 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.271 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.271 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:10.272 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138010+0000", 2024-09-16T20:50:10.272 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0021000000000001e-05 2024-09-16T20:50:10.272 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.272 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.272 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:10.272 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138114+0000", 2024-09-16T20:50:10.272 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000103797 2024-09-16T20:50:10.272 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.272 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.272 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:50:10.272 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138233+0000", 2024-09-16T20:50:10.272 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000118971 2024-09-16T20:50:10.272 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.273 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.273 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:10.273 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138953+0000", 2024-09-16T20:50:10.273 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00072026000000000002 2024-09-16T20:50:10.273 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.273 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.273 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:10.273 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222609+0000", 2024-09-16T20:50:10.273 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.083656650999999999 2024-09-16T20:50:10.273 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.273 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.273 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:10.273 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222684+0000", 2024-09-16T20:50:10.274 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4369999999999994e-05 2024-09-16T20:50:10.274 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.274 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.274 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:10.274 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222729+0000", 2024-09-16T20:50:10.274 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5287000000000002e-05 2024-09-16T20:50:10.274 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:10.274 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:10.274 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:10.274 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.274 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.274 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:11704 3.9 3:91220b75:::rbd_data.110e747f0779.000000000000001c:head [set-alloc-hint object_size 262144 write_size 262144,write 139264~6656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:50:10.274 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:50:10.274 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 66.098209565000005, 2024-09-16T20:50:10.275 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.084531133999999994, 2024-09-16T20:50:10.275 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:10.275 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:10.275 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:10.275 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:10.275 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:50:10.275 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:50:10.275 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 11704 2024-09-16T20:50:10.275 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.275 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:10.275 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.275 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:10.275 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:50:10.275 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:10.275 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.275 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.276 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:10.276 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:50:10.276 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:10.276 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.276 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.276 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:10.276 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138167+0000", 2024-09-16T20:50:10.276 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.33e-07 2024-09-16T20:50:10.276 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.276 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.276 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:10.276 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138179+0000", 2024-09-16T20:50:10.276 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2055999999999999e-05 2024-09-16T20:50:10.276 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.277 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.277 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:10.277 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138193+0000", 2024-09-16T20:50:10.277 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3961000000000001e-05 2024-09-16T20:50:10.277 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.277 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.277 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:10.277 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138200+0000", 2024-09-16T20:50:10.277 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4429999999999997e-06 2024-09-16T20:50:10.277 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.277 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.277 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:10.277 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138258+0000", 2024-09-16T20:50:10.277 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7225999999999997e-05 2024-09-16T20:50:10.278 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.278 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.278 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:10.278 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138364+0000", 2024-09-16T20:50:10.278 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000106013 2024-09-16T20:50:10.278 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.278 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.278 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:50:10.278 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138494+0000", 2024-09-16T20:50:10.278 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013059599999999999 2024-09-16T20:50:10.278 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.278 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.278 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:10.278 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.139220+0000", 2024-09-16T20:50:10.278 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00072539800000000004 2024-09-16T20:50:10.279 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.279 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.279 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:10.279 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222608+0000", 2024-09-16T20:50:10.279 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.083388620999999996 2024-09-16T20:50:10.279 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.279 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.279 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:10.279 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222651+0000", 2024-09-16T20:50:10.279 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2228000000000003e-05 2024-09-16T20:50:10.279 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.279 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.279 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:10.279 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222697+0000", 2024-09-16T20:50:10.279 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6659000000000002e-05 2024-09-16T20:50:10.280 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:10.280 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:10.280 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:10.280 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.280 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.280 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5138.0:9358 3.c 3.c25699cc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T20:50:10.280 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:50:10.280 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 43.386062260000003, 2024-09-16T20:50:10.280 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0018657580000001, 2024-09-16T20:50:10.280 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:10.280 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:10.280 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:50:10.280 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:10.280 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5138", 2024-09-16T20:50:10.280 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:50:10.281 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 9358 2024-09-16T20:50:10.281 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.281 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:10.281 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.281 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:10.281 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:50:10.281 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:10.281 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.281 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.281 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:10.281 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:50:10.281 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:10.281 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.282 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.282 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:10.282 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850314+0000", 2024-09-16T20:50:10.282 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4099999999999999e-07 2024-09-16T20:50:10.282 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.282 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.282 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:10.282 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850316+0000", 2024-09-16T20:50:10.282 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2469999999999998e-06 2024-09-16T20:50:10.282 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.282 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.282 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:10.282 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850322+0000", 2024-09-16T20:50:10.282 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3470000000000004e-06 2024-09-16T20:50:10.282 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.283 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.283 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:10.283 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850327+0000", 2024-09-16T20:50:10.283 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9450000000000001e-06 2024-09-16T20:50:10.283 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.283 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.283 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:10.283 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850376+0000", 2024-09-16T20:50:10.283 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8257000000000002e-05 2024-09-16T20:50:10.283 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.283 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.283 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:10.283 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850385+0000", 2024-09-16T20:50:10.283 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9800000000000004e-06 2024-09-16T20:50:10.283 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.284 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.284 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:10.284 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852167+0000", 2024-09-16T20:50:10.284 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0017825659999999 2024-09-16T20:50:10.284 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.284 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.284 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:10.284 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852179+0000", 2024-09-16T20:50:10.284 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1975e-05 2024-09-16T20:50:10.284 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:10.284 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:10.284 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:10.284 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.284 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.284 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5138.0:9382 3.c 3.c25699cc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T20:50:10.285 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:50:10.285 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 43.377715244000001, 2024-09-16T20:50:10.285 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.99358404700000003, 2024-09-16T20:50:10.285 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:10.285 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:10.285 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:50:10.285 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:10.285 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5138", 2024-09-16T20:50:10.285 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:50:10.285 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 9382 2024-09-16T20:50:10.285 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.285 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:10.285 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.285 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:10.285 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:50:10.286 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:10.286 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.286 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.286 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:10.286 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:50:10.286 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:10.286 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.286 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.286 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:10.286 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858661+0000", 2024-09-16T20:50:10.286 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0200000000000002e-07 2024-09-16T20:50:10.286 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.286 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.286 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:10.287 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858664+0000", 2024-09-16T20:50:10.287 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.038e-06 2024-09-16T20:50:10.287 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.287 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.287 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:10.287 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858671+0000", 2024-09-16T20:50:10.287 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0110000000000003e-06 2024-09-16T20:50:10.287 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.287 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.287 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:10.287 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858677+0000", 2024-09-16T20:50:10.287 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0959999999999997e-06 2024-09-16T20:50:10.287 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.287 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.287 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:10.288 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858732+0000", 2024-09-16T20:50:10.288 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4543000000000003e-05 2024-09-16T20:50:10.288 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.288 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.288 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:10.288 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858739+0000", 2024-09-16T20:50:10.288 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5429999999999996e-06 2024-09-16T20:50:10.288 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.288 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.288 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:10.288 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852236+0000", 2024-09-16T20:50:10.288 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.993496453 2024-09-16T20:50:10.288 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.288 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.289 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:10.289 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852244+0000", 2024-09-16T20:50:10.289 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8610000000000004e-06 2024-09-16T20:50:10.289 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:10.289 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:10.289 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:10.289 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.289 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.289 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:361 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 372736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:50:10.289 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246714+0000", 2024-09-16T20:50:10.289 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 33.989661095999999, 2024-09-16T20:50:10.289 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050338229999999998, 2024-09-16T20:50:10.289 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:10.289 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:10.289 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:10.290 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:10.290 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:50:10.290 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:50:10.290 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 361 2024-09-16T20:50:10.290 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.290 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:10.290 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.290 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:10.290 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246714+0000", 2024-09-16T20:50:10.290 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:10.290 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.290 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.290 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:10.290 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246714+0000", 2024-09-16T20:50:10.290 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:10.291 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.291 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.291 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:10.291 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246715+0000", 2024-09-16T20:50:10.291 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1899999999999999e-07 2024-09-16T20:50:10.291 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.291 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.291 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:10.291 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246716+0000", 2024-09-16T20:50:10.291 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1659999999999999e-06 2024-09-16T20:50:10.291 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.291 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.291 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:10.291 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246721+0000", 2024-09-16T20:50:10.292 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9830000000000004e-06 2024-09-16T20:50:10.292 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.292 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.292 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:10.292 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246723+0000", 2024-09-16T20:50:10.292 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5679999999999998e-06 2024-09-16T20:50:10.292 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.292 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.292 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:10.292 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.296875+0000", 2024-09-16T20:50:10.292 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050151683000000002 2024-09-16T20:50:10.292 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.292 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.293 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:10.293 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.296959+0000", 2024-09-16T20:50:10.293 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4079999999999995e-05 2024-09-16T20:50:10.293 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.293 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.293 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:10.293 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297053+0000", 2024-09-16T20:50:10.293 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.3530999999999996e-05 2024-09-16T20:50:10.293 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:10.293 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:10.293 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:10.293 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.293 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.293 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:362 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 385024~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:50:10.294 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246727+0000", 2024-09-16T20:50:10.294 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 33.989648987000002, 2024-09-16T20:50:10.294 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050546528, 2024-09-16T20:50:10.294 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:10.294 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:10.294 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:10.294 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:10.294 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:50:10.294 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:50:10.294 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 362 2024-09-16T20:50:10.294 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.294 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:10.294 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.295 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:10.295 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246727+0000", 2024-09-16T20:50:10.295 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:10.295 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.295 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.295 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:10.295 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246727+0000", 2024-09-16T20:50:10.295 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:10.295 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.295 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.295 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:10.295 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246727+0000", 2024-09-16T20:50:10.295 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.35e-07 2024-09-16T20:50:10.296 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.296 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.296 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:10.296 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246728+0000", 2024-09-16T20:50:10.296 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.175e-06 2024-09-16T20:50:10.296 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.296 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.296 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:10.296 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246734+0000", 2024-09-16T20:50:10.296 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2160000000000003e-06 2024-09-16T20:50:10.296 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.296 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.296 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:10.297 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246737+0000", 2024-09-16T20:50:10.297 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9450000000000002e-06 2024-09-16T20:50:10.297 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.297 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.297 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:10.297 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297103+0000", 2024-09-16T20:50:10.297 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050365518999999997 2024-09-16T20:50:10.297 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.297 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.297 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:10.297 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297183+0000", 2024-09-16T20:50:10.297 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0208000000000002e-05 2024-09-16T20:50:10.297 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.297 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.298 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:10.298 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297273+0000", 2024-09-16T20:50:10.298 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0229999999999995e-05 2024-09-16T20:50:10.298 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:10.298 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:10.298 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:10.298 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.298 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.298 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:368 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 327680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:50:10.298 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246746+0000", 2024-09-16T20:50:10.298 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 33.989629706999999, 2024-09-16T20:50:10.298 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050730747999999999, 2024-09-16T20:50:10.298 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:10.298 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:10.299 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:10.299 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:10.299 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:50:10.299 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:50:10.299 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 368 2024-09-16T20:50:10.299 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.299 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:10.299 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.299 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:10.299 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246746+0000", 2024-09-16T20:50:10.299 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:10.299 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.299 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.300 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:10.300 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246746+0000", 2024-09-16T20:50:10.300 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:10.300 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.300 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.300 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:10.300 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246746+0000", 2024-09-16T20:50:10.300 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8099999999999999e-07 2024-09-16T20:50:10.300 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.300 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.300 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:10.300 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246754+0000", 2024-09-16T20:50:10.300 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4440000000000002e-06 2024-09-16T20:50:10.301 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.301 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.301 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:10.301 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246758+0000", 2024-09-16T20:50:10.301 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3900000000000003e-06 2024-09-16T20:50:10.301 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.301 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.301 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:10.301 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246761+0000", 2024-09-16T20:50:10.301 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.101e-06 2024-09-16T20:50:10.301 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.301 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.301 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:10.301 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297310+0000", 2024-09-16T20:50:10.301 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050549166 2024-09-16T20:50:10.302 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.302 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.302 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:10.302 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297387+0000", 2024-09-16T20:50:10.302 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6658999999999999e-05 2024-09-16T20:50:10.302 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.302 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.302 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:10.302 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297477+0000", 2024-09-16T20:50:10.302 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9707e-05 2024-09-16T20:50:10.302 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:10.302 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:10.302 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:10.303 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.303 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.303 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:369 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 339968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:50:10.303 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246766+0000", 2024-09-16T20:50:10.303 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 33.989609981999998, 2024-09-16T20:50:10.303 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050911291999999997, 2024-09-16T20:50:10.303 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:10.303 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:10.303 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:10.303 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:10.303 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:50:10.303 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:50:10.303 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 369 2024-09-16T20:50:10.303 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.304 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:10.304 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.304 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:10.304 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246766+0000", 2024-09-16T20:50:10.304 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:10.304 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.304 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.304 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:10.304 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246766+0000", 2024-09-16T20:50:10.304 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:10.304 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.304 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.304 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:10.305 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246766+0000", 2024-09-16T20:50:10.305 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3200000000000001e-07 2024-09-16T20:50:10.305 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.305 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.305 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:10.305 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246767+0000", 2024-09-16T20:50:10.305 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.252e-06 2024-09-16T20:50:10.305 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.305 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.305 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:10.305 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246770+0000", 2024-09-16T20:50:10.305 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1049999999999999e-06 2024-09-16T20:50:10.305 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.305 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.306 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:10.306 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246773+0000", 2024-09-16T20:50:10.306 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.396e-06 2024-09-16T20:50:10.306 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.306 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.306 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:10.306 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297513+0000", 2024-09-16T20:50:10.306 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050740800000000003 2024-09-16T20:50:10.306 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.306 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.306 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:10.306 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297589+0000", 2024-09-16T20:50:10.306 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5318e-05 2024-09-16T20:50:10.306 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.307 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.307 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:10.307 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297677+0000", 2024-09-16T20:50:10.307 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8189000000000006e-05 2024-09-16T20:50:10.307 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:10.307 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:10.307 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:10.307 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.307 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.307 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:370 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 348160~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:50:10.307 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246776+0000", 2024-09-16T20:50:10.307 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 33.989599966999997, 2024-09-16T20:50:10.307 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051096643999999997, 2024-09-16T20:50:10.307 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:10.307 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:10.308 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:10.308 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:10.308 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:50:10.308 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:50:10.308 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 370 2024-09-16T20:50:10.308 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.308 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:10.308 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.308 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:10.308 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246776+0000", 2024-09-16T20:50:10.308 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:10.308 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.308 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.309 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:10.309 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246776+0000", 2024-09-16T20:50:10.309 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:10.309 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.309 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.309 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:10.309 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246776+0000", 2024-09-16T20:50:10.309 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2499999999999999e-07 2024-09-16T20:50:10.309 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.309 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.309 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:10.309 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246777+0000", 2024-09-16T20:50:10.309 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1510000000000001e-06 2024-09-16T20:50:10.309 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.310 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.310 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:10.310 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246780+0000", 2024-09-16T20:50:10.310 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9610000000000001e-06 2024-09-16T20:50:10.310 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.310 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.310 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:10.310 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246783+0000", 2024-09-16T20:50:10.310 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6129999999999998e-06 2024-09-16T20:50:10.310 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.310 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.310 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:10.310 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297713+0000", 2024-09-16T20:50:10.310 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050929950000000002 2024-09-16T20:50:10.310 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.311 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.311 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:10.311 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297786+0000", 2024-09-16T20:50:10.311 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3096000000000006e-05 2024-09-16T20:50:10.311 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.311 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.311 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:10.311 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297872+0000", 2024-09-16T20:50:10.311 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6648000000000001e-05 2024-09-16T20:50:10.311 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:10.311 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:10.311 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:10.311 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.311 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.311 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:371 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 380928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:50:10.312 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246791+0000", 2024-09-16T20:50:10.312 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 33.989584331000003, 2024-09-16T20:50:10.312 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051287209, 2024-09-16T20:50:10.312 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:10.312 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:10.312 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:10.312 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:10.312 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:50:10.312 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:50:10.312 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 371 2024-09-16T20:50:10.312 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.312 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:10.312 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.312 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:10.312 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246791+0000", 2024-09-16T20:50:10.313 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:10.313 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.313 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.313 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:10.313 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246791+0000", 2024-09-16T20:50:10.313 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:10.313 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.313 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.313 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:10.313 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246792+0000", 2024-09-16T20:50:10.313 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8200000000000001e-07 2024-09-16T20:50:10.314 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.314 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.314 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:10.314 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246793+0000", 2024-09-16T20:50:10.314 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.28e-06 2024-09-16T20:50:10.314 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.314 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.314 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:10.314 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246796+0000", 2024-09-16T20:50:10.314 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.597e-06 2024-09-16T20:50:10.314 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.314 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.314 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:10.314 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246799+0000", 2024-09-16T20:50:10.314 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7089999999999998e-06 2024-09-16T20:50:10.315 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.315 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.315 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:10.315 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297910+0000", 2024-09-16T20:50:10.315 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051110598 2024-09-16T20:50:10.315 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.315 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.315 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:10.315 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297986+0000", 2024-09-16T20:50:10.315 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6711000000000005e-05 2024-09-16T20:50:10.315 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.315 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.315 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:10.315 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.298078+0000", 2024-09-16T20:50:10.315 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2032000000000002e-05 2024-09-16T20:50:10.315 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:10.316 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:10.316 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:10.316 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.316 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.316 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:373 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 376832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:50:10.316 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246904+0000", 2024-09-16T20:50:10.316 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 33.989471909000002, 2024-09-16T20:50:10.316 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051310617000000003, 2024-09-16T20:50:10.316 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:10.316 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:10.316 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:10.316 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:10.316 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:50:10.316 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:50:10.317 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 373 2024-09-16T20:50:10.317 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.317 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:10.317 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.317 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:10.317 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246904+0000", 2024-09-16T20:50:10.317 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:10.317 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.317 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.317 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:10.317 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246904+0000", 2024-09-16T20:50:10.317 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:10.317 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.317 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.317 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:10.318 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246904+0000", 2024-09-16T20:50:10.318 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8000000000000001e-07 2024-09-16T20:50:10.318 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.318 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.318 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:10.318 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246908+0000", 2024-09-16T20:50:10.318 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4039999999999997e-06 2024-09-16T20:50:10.318 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.318 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.318 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:10.318 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246913+0000", 2024-09-16T20:50:10.318 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0030000000000002e-06 2024-09-16T20:50:10.318 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.318 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.318 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:10.319 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246917+0000", 2024-09-16T20:50:10.319 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.303e-06 2024-09-16T20:50:10.319 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.319 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.319 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:10.319 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.298114+0000", 2024-09-16T20:50:10.319 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051197724 2024-09-16T20:50:10.319 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.319 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.319 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:10.319 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.298159+0000", 2024-09-16T20:50:10.319 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4882999999999997e-05 2024-09-16T20:50:10.319 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.319 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.320 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:10.320 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.298214+0000", 2024-09-16T20:50:10.320 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4920000000000003e-05 2024-09-16T20:50:10.320 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:10.320 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:10.320 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:10.320 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.320 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.320 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:374 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 389120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:50:10.320 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246927+0000", 2024-09-16T20:50:10.320 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 33.989448860000003, 2024-09-16T20:50:10.320 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051498164999999999, 2024-09-16T20:50:10.320 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:10.320 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:10.321 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:10.321 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:10.321 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:50:10.321 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:50:10.321 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 374 2024-09-16T20:50:10.321 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.321 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:10.321 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.321 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:10.321 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246927+0000", 2024-09-16T20:50:10.321 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:10.321 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.321 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.321 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:10.321 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246927+0000", 2024-09-16T20:50:10.322 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:10.322 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.322 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.322 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:10.322 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246927+0000", 2024-09-16T20:50:10.322 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7e-07 2024-09-16T20:50:10.322 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.322 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.322 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:10.322 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246929+0000", 2024-09-16T20:50:10.322 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.607e-06 2024-09-16T20:50:10.322 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.322 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.322 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:10.323 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246932+0000", 2024-09-16T20:50:10.323 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4539999999999999e-06 2024-09-16T20:50:10.323 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.323 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.323 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:10.323 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246935+0000", 2024-09-16T20:50:10.323 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6570000000000001e-06 2024-09-16T20:50:10.323 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.323 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.323 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:10.323 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.298242+0000", 2024-09-16T20:50:10.323 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051306887000000002 2024-09-16T20:50:10.323 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.323 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.323 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:10.323 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.298327+0000", 2024-09-16T20:50:10.324 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5068999999999995e-05 2024-09-16T20:50:10.324 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.324 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.324 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:10.324 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.298425+0000", 2024-09-16T20:50:10.324 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8021000000000002e-05 2024-09-16T20:50:10.324 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:10.324 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:10.324 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:10.324 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.324 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.324 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5763.0:4895 3.a 3:5384606c:::rbd_data.110e1b19a241.0000000000000018:head [read 0~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-16T20:50:10.324 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:45.598212+0000", 2024-09-16T20:50:10.324 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 24.638163203000001, 2024-09-16T20:50:10.324 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0375864829999999, 2024-09-16T20:50:10.325 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:10.325 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:10.325 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:10.325 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:10.325 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5763", 2024-09-16T20:50:10.325 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/623897567", 2024-09-16T20:50:10.325 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4895 2024-09-16T20:50:10.325 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.325 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:10.325 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.325 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:10.325 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598212+0000", 2024-09-16T20:50:10.325 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:10.326 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.326 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.326 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:10.326 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598212+0000", 2024-09-16T20:50:10.326 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:10.326 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.326 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.326 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:10.326 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598214+0000", 2024-09-16T20:50:10.326 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0090000000000002e-06 2024-09-16T20:50:10.326 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.326 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.326 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:10.326 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598222+0000", 2024-09-16T20:50:10.327 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6699999999999994e-06 2024-09-16T20:50:10.327 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.327 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.327 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:10.327 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598241+0000", 2024-09-16T20:50:10.327 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8896999999999999e-05 2024-09-16T20:50:10.327 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.327 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.327 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:10.327 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598250+0000", 2024-09-16T20:50:10.327 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6649999999999992e-06 2024-09-16T20:50:10.327 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.327 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.327 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:10.328 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598325+0000", 2024-09-16T20:50:10.328 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5189000000000001e-05 2024-09-16T20:50:10.328 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.328 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.328 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op must wait for map", 2024-09-16T20:50:10.328 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598333+0000", 2024-09-16T20:50:10.328 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.3170000000000002e-06 2024-09-16T20:50:10.328 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.328 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.328 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:10.328 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601355+0000", 2024-09-16T20:50:10.328 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0030215400000000001 2024-09-16T20:50:10.328 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.328 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.329 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:10.329 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601365+0000", 2024-09-16T20:50:10.329 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0215e-05 2024-09-16T20:50:10.329 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.329 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.329 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:10.329 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635506+0000", 2024-09-16T20:50:10.329 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0341408240000001 2024-09-16T20:50:10.329 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.329 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.329 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:10.329 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635655+0000", 2024-09-16T20:50:10.329 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000149412 2024-09-16T20:50:10.329 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.330 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.330 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:10.330 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635799+0000", 2024-09-16T20:50:10.330 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000143745 2024-09-16T20:50:10.330 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:10.330 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:10.332 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:10.332 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.332 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.333 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5763.0:4900 3.a 3:5384606c:::rbd_data.110e1b19a241.0000000000000018:head [read 8192~24576] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-16T20:50:10.333 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:50:10.333 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 24.638104726999998, 2024-09-16T20:50:10.333 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0379340800000001, 2024-09-16T20:50:10.333 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:10.333 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:10.333 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:10.333 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:10.333 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5763", 2024-09-16T20:50:10.333 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/623897567", 2024-09-16T20:50:10.333 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4900 2024-09-16T20:50:10.333 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.333 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:10.334 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.334 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:10.334 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:50:10.334 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:10.334 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.334 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.334 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:10.334 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:50:10.334 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:10.334 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.334 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.334 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:10.334 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:50:10.334 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4300000000000003e-07 2024-09-16T20:50:10.335 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.335 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.335 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:10.335 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598274+0000", 2024-09-16T20:50:10.335 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.976e-06 2024-09-16T20:50:10.335 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.335 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.335 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:10.335 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598282+0000", 2024-09-16T20:50:10.335 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0029999999999994e-06 2024-09-16T20:50:10.335 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.335 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.335 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:10.335 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598289+0000", 2024-09-16T20:50:10.335 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2389999999999999e-06 2024-09-16T20:50:10.336 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.336 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.336 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:10.336 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598430+0000", 2024-09-16T20:50:10.336 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014119600000000001 2024-09-16T20:50:10.336 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.336 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.336 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting_for_map not empty", 2024-09-16T20:50:10.336 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598435+0000", 2024-09-16T20:50:10.336 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3489999999999999e-06 2024-09-16T20:50:10.336 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.336 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.336 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:10.337 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601429+0000", 2024-09-16T20:50:10.337 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0029942720000000001 2024-09-16T20:50:10.337 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.337 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.337 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:10.337 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601438+0000", 2024-09-16T20:50:10.337 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4759999999999998e-06 2024-09-16T20:50:10.337 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.337 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.337 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:10.337 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635987+0000", 2024-09-16T20:50:10.337 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0345495010000001 2024-09-16T20:50:10.337 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.337 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.338 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:10.338 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.636056+0000", 2024-09-16T20:50:10.338 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8610999999999998e-05 2024-09-16T20:50:10.338 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:10.338 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:10.338 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:10.338 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.636205+0000", 2024-09-16T20:50:10.338 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014891399999999999 2024-09-16T20:50:10.338 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:10.338 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:10.338 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:10.338 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:10.338 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:10.338 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:10.339 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T20:50:10.341 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:50:10.341 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:10.339+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:10.441 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:10.439+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:10.492 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:10.492 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:50:10.492 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:50:10.492 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:10.500 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:50:10.505 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:50:10.505 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:50:10.523 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:50:10.523 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:50:10.541 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:50:10.541 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:50:10.542 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:10.540+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:10.557 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:50:10.557 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:50:10.560 INFO:teuthology.orchestra.run.smithi029.stderr:nodeep-scrub is set 2024-09-16T20:50:10.578 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:50:10.579 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:50:10.597 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:50:10.597 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:50:10.624 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:10.624 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:10.633 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:50:10.642 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:10.641+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:10.744 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:10.742+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:10.783 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:10.783 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:10.792 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:50:10.844 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:10.842+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:10.901 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #38, image image_client.0-clone38 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent39 2024-09-16T20:50:10.904 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:10.904 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:10.912 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:50:10.946 INFO:teuthology.orchestra.run.smithi139.stdout:5089 read 0x39dd000 thru 0x39e1b46 (0x4b47 bytes) 2024-09-16T20:50:10.947 INFO:teuthology.orchestra.run.smithi139.stdout:5090 write 0xa3b5800 thru 0xa3be88e (0x908f bytes) 2024-09-16T20:50:10.947 INFO:teuthology.orchestra.run.smithi139.stdout:5091 punch from 0x3774a00 to 0x377fc00, (0xb200 bytes) 2024-09-16T20:50:10.953 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:10.952+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:10.985 INFO:teuthology.orchestra.run.smithi139.stdout:5094 punch from 0x3d6bc00 to 0x3d6f800, (0x3c00 bytes) 2024-09-16T20:50:11.002 INFO:teuthology.orchestra.run.smithi139.stdout:5096 write 0x1a98c00 thru 0x1a9f804 (0x6c05 bytes) 2024-09-16T20:50:11.003 INFO:teuthology.orchestra.run.smithi139.stdout:5100 write 0xbbda200 thru 0xbbe370c (0x950d bytes) 2024-09-16T20:50:11.003 INFO:teuthology.orchestra.run.smithi139.stdout:5101 write 0xb0a0000 thru 0xb0a2466 (0x2467 bytes) 2024-09-16T20:50:11.014 INFO:teuthology.orchestra.run.smithi139.stdout:5103 trunc from 0xd7f1800 to 0x3f3a800 2024-09-16T20:50:11.047 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:11.045+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:11.085 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:11.085 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:11.094 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:50:11.131 INFO:teuthology.orchestra.run.smithi139.stdout:5110 write 0x1f57800 thru 0x1f5bb86 (0x4387 bytes) 2024-09-16T20:50:11.138 INFO:teuthology.orchestra.run.smithi139.stdout:5111 read 0x1766e00 thru 0x176b8b0 (0x4ab1 bytes) 2024-09-16T20:50:11.138 INFO:teuthology.orchestra.run.smithi139.stdout:5112 punch from 0x650e00 to 0x654000, (0x3200 bytes) 2024-09-16T20:50:11.138 INFO:teuthology.orchestra.run.smithi139.stdout:5115 punch from 0x11c7600 to 0x11cfa00, (0x8400 bytes) 2024-09-16T20:50:11.138 INFO:teuthology.orchestra.run.smithi139.stdout:5117 read 0x507c00 thru 0x5155e4 (0xd9e5 bytes) 2024-09-16T20:50:11.139 INFO:teuthology.orchestra.run.smithi139.stdout:5119 read 0x455e00 thru 0x45a246 (0x4447 bytes) 2024-09-16T20:50:11.140 INFO:teuthology.orchestra.run.smithi139.stdout:5120 punch from 0x2009600 to 0x2010200, (0x6c00 bytes) 2024-09-16T20:50:11.140 INFO:teuthology.orchestra.run.smithi139.stdout:5123 read 0x3a0d400 thru 0x3a0dac9 (0x6ca bytes) 2024-09-16T20:50:11.140 INFO:teuthology.orchestra.run.smithi139.stdout:5125 read 0x1529c00 thru 0x152f5a8 (0x59a9 bytes) 2024-09-16T20:50:11.141 INFO:teuthology.orchestra.run.smithi139.stdout:5127 read 0x3606400 thru 0x36074eb (0x10ec bytes) 2024-09-16T20:50:11.149 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:11.148+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:11.157 INFO:teuthology.orchestra.run.smithi139.stdout:5129 write 0xa4b6800 thru 0xa4b6fb0 (0x7b1 bytes) 2024-09-16T20:50:11.164 INFO:teuthology.orchestra.run.smithi139.stdout:5130 write 0x8aa4c00 thru 0x8aaffda (0xb3db bytes) 2024-09-16T20:50:11.175 INFO:teuthology.orchestra.run.smithi139.stdout:5132 trunc from 0xa4b7000 to 0xcdc400 2024-09-16T20:50:11.216 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:11.216 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:11.220 INFO:teuthology.orchestra.run.smithi139.stdout:5134 write 0x5123c00 thru 0x512e33c (0xa73d bytes) 2024-09-16T20:50:11.226 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:50:11.247 INFO:teuthology.orchestra.run.smithi139.stdout:5136 write 0xcc13400 thru 0xcc1f5a3 (0xc1a4 bytes) 2024-09-16T20:50:11.250 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:11.249+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:11.254 INFO:teuthology.orchestra.run.smithi139.stdout:5138 write 0x6db0e00 thru 0x6db808f (0x7290 bytes) 2024-09-16T20:50:11.255 INFO:teuthology.orchestra.run.smithi139.stdout:5139 write 0x9b1ec00 thru 0x9b1eea9 (0x2aa bytes) 2024-09-16T20:50:11.255 INFO:teuthology.orchestra.run.smithi139.stdout:5141 write 0xb380400 thru 0xb383091 (0x2c92 bytes) 2024-09-16T20:50:11.255 INFO:teuthology.orchestra.run.smithi139.stdout:5142 punch from 0x69b1e00 to 0x69b8400, (0x6600 bytes) 2024-09-16T20:50:11.256 INFO:teuthology.orchestra.run.smithi139.stdout:5146 punch from 0x3dd3000 to 0x3dd7000, (0x4000 bytes) 2024-09-16T20:50:11.256 INFO:teuthology.orchestra.run.smithi139.stdout:5147 punch from 0x91a9600 to 0x91b4800, (0xb200 bytes) 2024-09-16T20:50:11.256 INFO:teuthology.orchestra.run.smithi139.stdout:5148 read 0x3e01a00 thru 0x3e117d4 (0xfdd5 bytes) 2024-09-16T20:50:11.260 INFO:teuthology.orchestra.run.smithi139.stdout:5151 trunc from 0xcc1f600 to 0x9a77e00 2024-09-16T20:50:11.276 INFO:teuthology.orchestra.run.smithi139.stdout:5152 trunc from 0x9a77e00 to 0x9360200 2024-09-16T20:50:11.289 INFO:teuthology.orchestra.run.smithi139.stdout:5155 read 0x48de600 thru 0x48e1e46 (0x3847 bytes) 2024-09-16T20:50:11.291 INFO:teuthology.orchestra.run.smithi139.stdout:5156 read 0x6012200 thru 0x60173ae (0x51af bytes) 2024-09-16T20:50:11.291 INFO:teuthology.orchestra.run.smithi139.stdout:5157 read 0x1e3b800 thru 0x1e425f6 (0x6df7 bytes) 2024-09-16T20:50:11.293 INFO:teuthology.orchestra.run.smithi139.stdout:5159 trunc from 0x9360200 to 0x7c28e00 2024-09-16T20:50:11.304 INFO:teuthology.orchestra.run.smithi139.stdout:5160 write 0x51d8600 thru 0x51dbcbb (0x36bc bytes) 2024-09-16T20:50:11.306 INFO:teuthology.orchestra.run.smithi139.stdout:5161 punch from 0x6927e00 to 0x6930200, (0x8400 bytes) 2024-09-16T20:50:11.313 INFO:teuthology.orchestra.run.smithi139.stdout:5166 trunc from 0x7c28e00 to 0x22c3e00 2024-09-16T20:50:11.343 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:11.343 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:11.354 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:50:11.354 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:11.352+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:11.355 INFO:teuthology.orchestra.run.smithi139.stdout:5167 write 0x8114000 thru 0x811be50 (0x7e51 bytes) 2024-09-16T20:50:11.356 INFO:teuthology.orchestra.run.smithi139.stdout:5168 write 0x27c3000 thru 0x27d0f1d (0xdf1e bytes) 2024-09-16T20:50:11.356 INFO:teuthology.orchestra.run.smithi139.stdout:5170 read 0x38a1800 thru 0x38b169d (0xfe9e bytes) 2024-09-16T20:50:11.356 INFO:teuthology.orchestra.run.smithi139.stdout:5171 punch from 0x2528600 to 0x2530600, (0x8000 bytes) 2024-09-16T20:50:11.356 INFO:teuthology.orchestra.run.smithi139.stdout:5172 write 0x5cff000 thru 0x5d063e9 (0x73ea bytes) 2024-09-16T20:50:11.362 INFO:teuthology.orchestra.run.smithi139.stdout:5174 trunc from 0x811c000 to 0x34a6a00 2024-09-16T20:50:11.377 INFO:teuthology.orchestra.run.smithi139.stdout:5176 read 0x2e37c00 thru 0x2e3de67 (0x6268 bytes) 2024-09-16T20:50:11.378 INFO:teuthology.orchestra.run.smithi139.stdout:5177 read 0x2e0aa00 thru 0x2e19406 (0xea07 bytes) 2024-09-16T20:50:11.386 INFO:teuthology.orchestra.run.smithi139.stdout:5179 write 0x36d4600 thru 0x36e0bad (0xc5ae bytes) 2024-09-16T20:50:11.412 INFO:teuthology.orchestra.run.smithi139.stdout:5183 write 0xb8f3a00 thru 0xb8fc44b (0x8a4c bytes) 2024-09-16T20:50:11.417 INFO:teuthology.orchestra.run.smithi139.stdout:5184 write 0x46ba200 thru 0x46bd641 (0x3442 bytes) 2024-09-16T20:50:11.418 INFO:teuthology.orchestra.run.smithi139.stdout:5185 write 0xb3cc000 thru 0xb3cd402 (0x1403 bytes) 2024-09-16T20:50:11.418 INFO:teuthology.orchestra.run.smithi139.stdout:5188 read 0x9839600 thru 0x983f518 (0x5f19 bytes) 2024-09-16T20:50:11.418 INFO:teuthology.orchestra.run.smithi139.stdout:5189 write 0xb398400 thru 0xb3a3de0 (0xb9e1 bytes) 2024-09-16T20:50:11.432 INFO:teuthology.orchestra.run.smithi139.stdout:5191 trunc from 0xb8fc600 to 0x3f6a00 2024-09-16T20:50:11.454 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:11.453+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:11.465 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:11.465 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:11.474 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:50:11.498 INFO:teuthology.orchestra.run.smithi139.stdout:5192 write 0xd1cac00 thru 0xd1d0ea6 (0x62a7 bytes) 2024-09-16T20:50:11.499 INFO:teuthology.orchestra.run.smithi139.stdout:5195 read 0x21e1600 thru 0x21e8452 (0x6e53 bytes) 2024-09-16T20:50:11.499 INFO:teuthology.orchestra.run.smithi139.stdout:5196 write 0x30b9c00 thru 0x30c7211 (0xd612 bytes) 2024-09-16T20:50:11.515 INFO:teuthology.orchestra.run.smithi139.stdout:5199 trunc from 0xd1d1000 to 0x48a600 2024-09-16T20:50:11.557 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:11.556+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:11.570 INFO:teuthology.orchestra.run.smithi139.stdout:5201 write 0x3904400 thru 0x390c86c (0x846d bytes) 2024-09-16T20:50:11.575 INFO:teuthology.orchestra.run.smithi139.stdout:5202 trunc from 0x390ca00 to 0x594e00 2024-09-16T20:50:11.602 INFO:teuthology.orchestra.run.smithi139.stdout:5208 trunc from 0x594e00 to 0xa4a3400 2024-09-16T20:50:11.611 INFO:teuthology.orchestra.run.smithi139.stdout:5213 read 0xa370800 thru 0xa376114 (0x5915 bytes) 2024-09-16T20:50:11.611 INFO:teuthology.orchestra.run.smithi139.stdout:5214 read 0x46dee00 thru 0x46e0822 (0x1a23 bytes) 2024-09-16T20:50:11.612 INFO:teuthology.orchestra.run.smithi139.stdout:5215 punch from 0x2657600 to 0x2660800, (0x9200 bytes) 2024-09-16T20:50:11.618 INFO:teuthology.orchestra.run.smithi139.stdout:5216 trunc from 0xa4a3400 to 0x589d400 2024-09-16T20:50:11.629 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:11.629 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:11.638 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:50:11.646 INFO:teuthology.orchestra.run.smithi139.stdout:5218 write 0xabd4200 thru 0xabd61c7 (0x1fc8 bytes) 2024-09-16T20:50:11.649 INFO:teuthology.orchestra.run.smithi139.stdout:5219 trunc from 0xabd6200 to 0x8e3ce00 2024-09-16T20:50:11.658 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:11.657+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:11.673 INFO:teuthology.orchestra.run.smithi139.stdout:5222 write 0xb422a00 thru 0xb4265b0 (0x3bb1 bytes) 2024-09-16T20:50:11.681 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 3, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-16T20:50:11.681 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T20:50:11.682 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-16T20:50:11.682 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-16T20:50:11.689 INFO:teuthology.orchestra.run.smithi139.stdout:5225 write 0xe9bb800 thru 0xe9c7f7f (0xc780 bytes) 2024-09-16T20:50:11.690 INFO:teuthology.orchestra.run.smithi139.stdout:5226 read 0x2aeda00 thru 0x2af06c6 (0x2cc7 bytes) 2024-09-16T20:50:11.691 INFO:teuthology.orchestra.run.smithi139.stdout:5229 write 0x591be00 thru 0x592a2b1 (0xe4b2 bytes) 2024-09-16T20:50:11.691 INFO:teuthology.orchestra.run.smithi139.stdout:5230 read 0xc8c9200 thru 0xc8cc62e (0x342f bytes) 2024-09-16T20:50:11.692 INFO:teuthology.orchestra.run.smithi139.stdout:5231 read 0xa1eb000 thru 0xa1f5122 (0xa123 bytes) 2024-09-16T20:50:11.692 INFO:teuthology.orchestra.run.smithi139.stdout:5232 write 0x1f6800 thru 0x2008a5 (0xa0a6 bytes) 2024-09-16T20:50:11.692 INFO:teuthology.orchestra.run.smithi139.stdout:5233 read 0x7f11a00 thru 0x7f1934b (0x794c bytes) 2024-09-16T20:50:11.693 INFO:teuthology.orchestra.run.smithi139.stdout:5234 read 0x3105e00 thru 0x3115d9c (0xff9d bytes) 2024-09-16T20:50:11.693 INFO:teuthology.orchestra.run.smithi139.stdout:5235 punch from 0x579b600 to 0x57a9000, (0xda00 bytes) 2024-09-16T20:50:11.709 INFO:teuthology.orchestra.run.smithi139.stdout:5239 trunc from 0xe9c8000 to 0x17d6400 2024-09-16T20:50:11.738 INFO:teuthology.orchestra.run.smithi139.stdout:5240 punch from 0x17ca000 to 0x17d0400, (0x6400 bytes) 2024-09-16T20:50:11.753 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:11.753 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:11.754 INFO:teuthology.orchestra.run.smithi139.stdout:5241 write 0x573a800 thru 0x574a06e (0xf86f bytes) 2024-09-16T20:50:11.759 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:11.758+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:11.766 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:50:11.776 INFO:teuthology.orchestra.run.smithi139.stdout:5242 write 0xe780a00 thru 0xe788e2e (0x842f bytes) 2024-09-16T20:50:11.777 INFO:teuthology.orchestra.run.smithi139.stdout:5245 punch from 0xe086e00 to 0xe089a00, (0x2c00 bytes) 2024-09-16T20:50:11.778 INFO:teuthology.orchestra.run.smithi139.stdout:5246 punch from 0xb631400 to 0xb636e00, (0x5a00 bytes) 2024-09-16T20:50:11.778 INFO:teuthology.orchestra.run.smithi139.stdout:5247 read 0x76dfa00 thru 0x76ed39e (0xd99f bytes) 2024-09-16T20:50:11.779 INFO:teuthology.orchestra.run.smithi139.stdout:5248 read 0xc96d400 thru 0xc97b2ea (0xdeeb bytes) 2024-09-16T20:50:11.795 INFO:teuthology.orchestra.run.smithi139.stdout:5250 trunc from 0xe789000 to 0xafa200 2024-09-16T20:50:11.825 INFO:teuthology.orchestra.run.smithi139.stdout:5252 read 0x76e000 thru 0x77b472 (0xd473 bytes) 2024-09-16T20:50:11.827 INFO:teuthology.orchestra.run.smithi139.stdout:5254 punch from 0x6ea200 to 0x6f4200, (0xa000 bytes) 2024-09-16T20:50:11.844 INFO:teuthology.orchestra.run.smithi139.stdout:5255 write 0x7222a00 thru 0x722b381 (0x8982 bytes) 2024-09-16T20:50:11.846 INFO:teuthology.orchestra.run.smithi139.stdout:5257 read 0xa6ec00 thru 0xa75668 (0x6a69 bytes) 2024-09-16T20:50:11.852 INFO:teuthology.orchestra.run.smithi139.stdout:5259 trunc from 0x722b400 to 0x2571000 2024-09-16T20:50:11.860 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:11.859+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:11.870 INFO:teuthology.orchestra.run.smithi139.stdout:5261 punch from 0x2069200 to 0x2074e00, (0xbc00 bytes) 2024-09-16T20:50:11.878 INFO:teuthology.orchestra.run.smithi139.stdout:5263 trunc from 0x2571000 to 0x7818400 2024-09-16T20:50:11.903 INFO:teuthology.orchestra.run.smithi139.stdout:5266 write 0xd6be200 thru 0xd6c87e2 (0xa5e3 bytes) 2024-09-16T20:50:11.903 INFO:teuthology.orchestra.run.smithi139.stdout:5268 punch from 0xd187400 to 0xd191a00, (0xa600 bytes) 2024-09-16T20:50:11.904 INFO:teuthology.orchestra.run.smithi139.stdout:5269 read 0xa91b800 thru 0xa91d18f (0x1990 bytes) 2024-09-16T20:50:11.905 INFO:teuthology.orchestra.run.smithi139.stdout:5272 punch from 0x1b28000 to 0x1b37c00, (0xfc00 bytes) 2024-09-16T20:50:11.918 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:11.919 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:11.920 INFO:teuthology.orchestra.run.smithi139.stdout:5273 trunc from 0xd6c8800 to 0x637000 2024-09-16T20:50:11.928 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:50:11.955 INFO:teuthology.orchestra.run.smithi139.stdout:5276 punch from 0x415200 to 0x423400, (0xe200 bytes) 2024-09-16T20:50:11.955 INFO:teuthology.orchestra.run.smithi139.stdout:5278 punch from 0x616c00 to 0x624a00, (0xde00 bytes) 2024-09-16T20:50:11.955 INFO:teuthology.orchestra.run.smithi139.stdout:5279 punch from 0x607000 to 0x60d000, (0x6000 bytes) 2024-09-16T20:50:11.955 INFO:teuthology.orchestra.run.smithi139.stdout:5281 read 0x456200 thru 0x462896 (0xc697 bytes) 2024-09-16T20:50:11.956 INFO:teuthology.orchestra.run.smithi139.stdout:5284 punch from 0x336000 to 0x33dc00, (0x7c00 bytes) 2024-09-16T20:50:11.972 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:11.970+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:11.973 INFO:teuthology.orchestra.run.smithi139.stdout:5285 write 0x90f400 thru 0x916f3d (0x7b3e bytes) 2024-09-16T20:50:11.974 INFO:teuthology.orchestra.run.smithi139.stdout:5286 read 0x53b400 thru 0x546102 (0xad03 bytes) 2024-09-16T20:50:11.975 INFO:teuthology.orchestra.run.smithi139.stdout:5289 punch from 0x4a4c00 to 0x4ada00, (0x8e00 bytes) 2024-09-16T20:50:11.985 INFO:teuthology.orchestra.run.smithi139.stdout:5293 trunc from 0x917000 to 0x9847e00 2024-09-16T20:50:11.994 INFO:teuthology.orchestra.run.smithi139.stdout:5294 read 0x26f9e00 thru 0x2701345 (0x7546 bytes) 2024-09-16T20:50:11.995 INFO:teuthology.orchestra.run.smithi139.stdout:5295 punch from 0x69dbe00 to 0x69e5c00, (0x9e00 bytes) 2024-09-16T20:50:11.996 INFO:teuthology.orchestra.run.smithi139.stdout:5297 read 0x2e77a00 thru 0x2e7c5a0 (0x4ba1 bytes) 2024-09-16T20:50:11.996 INFO:teuthology.orchestra.run.smithi139.stdout:5302 trunc from 0x9847e00 to 0x8ebbc00 2024-09-16T20:50:12.023 INFO:teuthology.orchestra.run.smithi139.stdout:5303 write 0xd28ac00 thru 0xd296fee (0xc3ef bytes) 2024-09-16T20:50:12.029 INFO:teuthology.orchestra.run.smithi139.stdout:5306 punch from 0xb2d8a00 to 0xb2d9200, (0x800 bytes) 2024-09-16T20:50:12.030 INFO:teuthology.orchestra.run.smithi139.stdout:5308 read 0xcec0600 thru 0xcec9aad (0x94ae bytes) 2024-09-16T20:50:12.030 INFO:teuthology.orchestra.run.smithi139.stdout:5310 write 0xc3c3200 thru 0xc3c92af (0x60b0 bytes) 2024-09-16T20:50:12.030 INFO:teuthology.orchestra.run.smithi139.stdout:5311 write 0x3089800 thru 0x308fa28 (0x6229 bytes) 2024-09-16T20:50:12.033 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:50:12.034 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:12.034 INFO:teuthology.orchestra.run.smithi029.stdout: "epoch": 127, 2024-09-16T20:50:12.034 INFO:teuthology.orchestra.run.smithi029.stdout: "fsid": "e28af4a7-5fcd-4de8-8f77-a2b575361bb1", 2024-09-16T20:50:12.034 INFO:teuthology.orchestra.run.smithi029.stdout: "created": "2024-09-16T20:47:39.760609+0000", 2024-09-16T20:50:12.034 INFO:teuthology.orchestra.run.smithi029.stdout: "modified": "2024-09-16T20:50:10.559128+0000", 2024-09-16T20:50:12.034 INFO:teuthology.orchestra.run.smithi029.stdout: "last_up_change": "2024-09-16T20:47:50.238013+0000", 2024-09-16T20:50:12.034 INFO:teuthology.orchestra.run.smithi029.stdout: "last_in_change": "2024-09-16T20:49:08.779548+0000", 2024-09-16T20:50:12.034 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-16T20:50:12.034 INFO:teuthology.orchestra.run.smithi029.stdout: "flags_num": 5806080, 2024-09-16T20:50:12.034 INFO:teuthology.orchestra.run.smithi029.stdout: "flags_set": [ 2024-09-16T20:50:12.034 INFO:teuthology.orchestra.run.smithi029.stdout: "nodeep-scrub", 2024-09-16T20:50:12.034 INFO:teuthology.orchestra.run.smithi029.stdout: "noscrub", 2024-09-16T20:50:12.034 INFO:teuthology.orchestra.run.smithi029.stdout: "pglog_hardlimit", 2024-09-16T20:50:12.034 INFO:teuthology.orchestra.run.smithi029.stdout: "purged_snapdirs", 2024-09-16T20:50:12.035 INFO:teuthology.orchestra.run.smithi029.stdout: "recovery_deletes", 2024-09-16T20:50:12.035 INFO:teuthology.orchestra.run.smithi029.stdout: "sortbitwise" 2024-09-16T20:50:12.035 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-16T20:50:12.035 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_version": 5, 2024-09-16T20:50:12.035 INFO:teuthology.orchestra.run.smithi029.stdout: "full_ratio": 0.94999998807907104, 2024-09-16T20:50:12.035 INFO:teuthology.orchestra.run.smithi029.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-16T20:50:12.035 INFO:teuthology.orchestra.run.smithi029.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-16T20:50:12.035 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_snapshot": "", 2024-09-16T20:50:12.035 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_max": 3, 2024-09-16T20:50:12.035 INFO:teuthology.orchestra.run.smithi029.stdout: "max_osd": 6, 2024-09-16T20:50:12.035 INFO:teuthology.orchestra.run.smithi029.stdout: "require_min_compat_client": "luminous", 2024-09-16T20:50:12.035 INFO:teuthology.orchestra.run.smithi029.stdout: "min_compat_client": "luminous", 2024-09-16T20:50:12.035 INFO:teuthology.orchestra.run.smithi029.stdout: "require_osd_release": "squid", 2024-09-16T20:50:12.035 INFO:teuthology.orchestra.run.smithi029.stdout: "allow_crimson": false, 2024-09-16T20:50:12.035 INFO:teuthology.orchestra.run.smithi029.stdout: "pools": [ 2024-09-16T20:50:12.035 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.036 INFO:teuthology.orchestra.run.smithi029.stdout: "pool": 1, 2024-09-16T20:50:12.036 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_name": ".mgr", 2024-09-16T20:50:12.036 INFO:teuthology.orchestra.run.smithi029.stdout: "create_time": "2024-09-16T20:47:52.220750+0000", 2024-09-16T20:50:12.036 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-16T20:50:12.036 INFO:teuthology.orchestra.run.smithi029.stdout: "flags_names": "hashpspool", 2024-09-16T20:50:12.036 INFO:teuthology.orchestra.run.smithi029.stdout: "type": 1, 2024-09-16T20:50:12.036 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 2, 2024-09-16T20:50:12.036 INFO:teuthology.orchestra.run.smithi029.stdout: "min_size": 1, 2024-09-16T20:50:12.036 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_rule": 0, 2024-09-16T20:50:12.036 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_count": 0, 2024-09-16T20:50:12.036 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_target": 0, 2024-09-16T20:50:12.036 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T20:50:12.036 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T20:50:12.036 INFO:teuthology.orchestra.run.smithi029.stdout: "is_stretch_pool": false, 2024-09-16T20:50:12.037 INFO:teuthology.orchestra.run.smithi029.stdout: "object_hash": 2, 2024-09-16T20:50:12.037 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_autoscale_mode": "off", 2024-09-16T20:50:12.037 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num": 1, 2024-09-16T20:50:12.037 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num": 1, 2024-09-16T20:50:12.037 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num_target": 1, 2024-09-16T20:50:12.037 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_target": 1, 2024-09-16T20:50:12.037 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_pending": 1, 2024-09-16T20:50:12.037 INFO:teuthology.orchestra.run.smithi029.stdout: "last_pg_merge_meta": { 2024-09-16T20:50:12.037 INFO:teuthology.orchestra.run.smithi029.stdout: "source_pgid": "0.0", 2024-09-16T20:50:12.037 INFO:teuthology.orchestra.run.smithi029.stdout: "ready_epoch": 0, 2024-09-16T20:50:12.037 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_started": 0, 2024-09-16T20:50:12.037 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_clean": 0, 2024-09-16T20:50:12.038 INFO:teuthology.orchestra.run.smithi029.stdout: "source_version": "0'0", 2024-09-16T20:50:12.038 INFO:teuthology.orchestra.run.smithi029.stdout: "target_version": "0'0" 2024-09-16T20:50:12.038 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.038 INFO:teuthology.orchestra.run.smithi029.stdout: "last_change": "17", 2024-09-16T20:50:12.038 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend": "0", 2024-09-16T20:50:12.038 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T20:50:12.038 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T20:50:12.038 INFO:teuthology.orchestra.run.smithi029.stdout: "auid": 0, 2024-09-16T20:50:12.038 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_mode": "selfmanaged", 2024-09-16T20:50:12.038 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_seq": 0, 2024-09-16T20:50:12.038 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_epoch": 0, 2024-09-16T20:50:12.038 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_snaps": [], 2024-09-16T20:50:12.038 INFO:teuthology.orchestra.run.smithi029.stdout: "removed_snaps": "[]", 2024-09-16T20:50:12.038 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_bytes": 0, 2024-09-16T20:50:12.038 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_objects": 0, 2024-09-16T20:50:12.039 INFO:teuthology.orchestra.run.smithi029.stdout: "tiers": [], 2024-09-16T20:50:12.039 INFO:teuthology.orchestra.run.smithi029.stdout: "tier_of": -1, 2024-09-16T20:50:12.039 INFO:teuthology.orchestra.run.smithi029.stdout: "read_tier": -1, 2024-09-16T20:50:12.039 INFO:teuthology.orchestra.run.smithi029.stdout: "write_tier": -1, 2024-09-16T20:50:12.039 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_mode": "none", 2024-09-16T20:50:12.039 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_bytes": 0, 2024-09-16T20:50:12.039 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_objects": 0, 2024-09-16T20:50:12.039 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T20:50:12.039 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T20:50:12.039 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T20:50:12.039 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_flush_age": 0, 2024-09-16T20:50:12.039 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_evict_age": 0, 2024-09-16T20:50:12.039 INFO:teuthology.orchestra.run.smithi029.stdout: "erasure_code_profile": "", 2024-09-16T20:50:12.039 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_params": { 2024-09-16T20:50:12.039 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "none" 2024-09-16T20:50:12.040 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.040 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_period": 0, 2024-09-16T20:50:12.040 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_count": 0, 2024-09-16T20:50:12.040 INFO:teuthology.orchestra.run.smithi029.stdout: "use_gmt_hitset": true, 2024-09-16T20:50:12.040 INFO:teuthology.orchestra.run.smithi029.stdout: "min_read_recency_for_promote": 0, 2024-09-16T20:50:12.040 INFO:teuthology.orchestra.run.smithi029.stdout: "min_write_recency_for_promote": 0, 2024-09-16T20:50:12.040 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T20:50:12.040 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_search_last_n": 0, 2024-09-16T20:50:12.040 INFO:teuthology.orchestra.run.smithi029.stdout: "grade_table": [], 2024-09-16T20:50:12.040 INFO:teuthology.orchestra.run.smithi029.stdout: "stripe_width": 0, 2024-09-16T20:50:12.040 INFO:teuthology.orchestra.run.smithi029.stdout: "expected_num_objects": 0, 2024-09-16T20:50:12.040 INFO:teuthology.orchestra.run.smithi029.stdout: "fast_read": false, 2024-09-16T20:50:12.040 INFO:teuthology.orchestra.run.smithi029.stdout: "options": { 2024-09-16T20:50:12.040 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_max": 32, 2024-09-16T20:50:12.040 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_min": 1 2024-09-16T20:50:12.041 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.041 INFO:teuthology.orchestra.run.smithi029.stdout: "application_metadata": { 2024-09-16T20:50:12.041 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": {} 2024-09-16T20:50:12.041 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.041 INFO:teuthology.orchestra.run.smithi029.stdout: "read_balance": { 2024-09-16T20:50:12.041 INFO:teuthology.orchestra.run.smithi029.stdout: "score_type": "Fair distribution", 2024-09-16T20:50:12.041 INFO:teuthology.orchestra.run.smithi029.stdout: "score_acting": 6.059999942779541, 2024-09-16T20:50:12.041 INFO:teuthology.orchestra.run.smithi029.stdout: "score_stable": 6.059999942779541, 2024-09-16T20:50:12.041 INFO:teuthology.orchestra.run.smithi029.stdout: "optimal_score": 0.33000001311302185, 2024-09-16T20:50:12.041 INFO:teuthology.orchestra.run.smithi029.stdout: "raw_score_acting": 2, 2024-09-16T20:50:12.041 INFO:teuthology.orchestra.run.smithi029.stdout: "raw_score_stable": 2, 2024-09-16T20:50:12.041 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity_weighted": 1, 2024-09-16T20:50:12.041 INFO:teuthology.orchestra.run.smithi029.stdout: "average_primary_affinity": 1, 2024-09-16T20:50:12.041 INFO:teuthology.orchestra.run.smithi029.stdout: "average_primary_affinity_weighted": 1 2024-09-16T20:50:12.042 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:12.042 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.042 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.042 INFO:teuthology.orchestra.run.smithi029.stdout: "pool": 2, 2024-09-16T20:50:12.042 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_name": "rbd", 2024-09-16T20:50:12.042 INFO:teuthology.orchestra.run.smithi029.stdout: "create_time": "2024-09-16T20:47:56.756666+0000", 2024-09-16T20:50:12.042 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 8193, 2024-09-16T20:50:12.042 INFO:teuthology.orchestra.run.smithi029.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T20:50:12.042 INFO:teuthology.orchestra.run.smithi029.stdout: "type": 1, 2024-09-16T20:50:12.042 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 2, 2024-09-16T20:50:12.042 INFO:teuthology.orchestra.run.smithi029.stdout: "min_size": 1, 2024-09-16T20:50:12.042 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_rule": 0, 2024-09-16T20:50:12.042 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_count": 0, 2024-09-16T20:50:12.042 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_target": 0, 2024-09-16T20:50:12.043 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T20:50:12.043 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T20:50:12.043 INFO:teuthology.orchestra.run.smithi029.stdout: "is_stretch_pool": false, 2024-09-16T20:50:12.043 INFO:teuthology.orchestra.run.smithi029.stdout: "object_hash": 2, 2024-09-16T20:50:12.043 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_autoscale_mode": "off", 2024-09-16T20:50:12.043 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num": 8, 2024-09-16T20:50:12.043 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num": 8, 2024-09-16T20:50:12.043 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num_target": 8, 2024-09-16T20:50:12.043 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_target": 8, 2024-09-16T20:50:12.043 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_pending": 8, 2024-09-16T20:50:12.043 INFO:teuthology.orchestra.run.smithi029.stdout: "last_pg_merge_meta": { 2024-09-16T20:50:12.043 INFO:teuthology.orchestra.run.smithi029.stdout: "source_pgid": "0.0", 2024-09-16T20:50:12.043 INFO:teuthology.orchestra.run.smithi029.stdout: "ready_epoch": 0, 2024-09-16T20:50:12.043 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_started": 0, 2024-09-16T20:50:12.043 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_clean": 0, 2024-09-16T20:50:12.044 INFO:teuthology.orchestra.run.smithi029.stdout: "source_version": "0'0", 2024-09-16T20:50:12.044 INFO:teuthology.orchestra.run.smithi029.stdout: "target_version": "0'0" 2024-09-16T20:50:12.044 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.044 INFO:teuthology.orchestra.run.smithi029.stdout: "last_change": "21", 2024-09-16T20:50:12.044 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend": "0", 2024-09-16T20:50:12.044 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T20:50:12.044 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T20:50:12.044 INFO:teuthology.orchestra.run.smithi029.stdout: "auid": 0, 2024-09-16T20:50:12.044 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_mode": "selfmanaged", 2024-09-16T20:50:12.044 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_seq": 2, 2024-09-16T20:50:12.044 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_epoch": 21, 2024-09-16T20:50:12.044 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_snaps": [], 2024-09-16T20:50:12.044 INFO:teuthology.orchestra.run.smithi029.stdout: "removed_snaps": "[]", 2024-09-16T20:50:12.044 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_bytes": 0, 2024-09-16T20:50:12.045 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_objects": 0, 2024-09-16T20:50:12.045 INFO:teuthology.orchestra.run.smithi029.stdout: "tiers": [], 2024-09-16T20:50:12.045 INFO:teuthology.orchestra.run.smithi029.stdout: "tier_of": -1, 2024-09-16T20:50:12.045 INFO:teuthology.orchestra.run.smithi029.stdout: "read_tier": -1, 2024-09-16T20:50:12.045 INFO:teuthology.orchestra.run.smithi029.stdout: "write_tier": -1, 2024-09-16T20:50:12.045 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_mode": "none", 2024-09-16T20:50:12.045 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_bytes": 0, 2024-09-16T20:50:12.045 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_objects": 0, 2024-09-16T20:50:12.045 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T20:50:12.045 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T20:50:12.045 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T20:50:12.045 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_flush_age": 0, 2024-09-16T20:50:12.045 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_evict_age": 0, 2024-09-16T20:50:12.045 INFO:teuthology.orchestra.run.smithi029.stdout: "erasure_code_profile": "", 2024-09-16T20:50:12.046 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_params": { 2024-09-16T20:50:12.046 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "none" 2024-09-16T20:50:12.046 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.046 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_period": 0, 2024-09-16T20:50:12.046 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_count": 0, 2024-09-16T20:50:12.046 INFO:teuthology.orchestra.run.smithi029.stdout: "use_gmt_hitset": true, 2024-09-16T20:50:12.046 INFO:teuthology.orchestra.run.smithi029.stdout: "min_read_recency_for_promote": 0, 2024-09-16T20:50:12.046 INFO:teuthology.orchestra.run.smithi029.stdout: "min_write_recency_for_promote": 0, 2024-09-16T20:50:12.046 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T20:50:12.046 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_search_last_n": 0, 2024-09-16T20:50:12.046 INFO:teuthology.orchestra.run.smithi029.stdout: "grade_table": [], 2024-09-16T20:50:12.046 INFO:teuthology.orchestra.run.smithi029.stdout: "stripe_width": 0, 2024-09-16T20:50:12.046 INFO:teuthology.orchestra.run.smithi029.stdout: "expected_num_objects": 0, 2024-09-16T20:50:12.046 INFO:teuthology.orchestra.run.smithi029.stdout: "fast_read": false, 2024-09-16T20:50:12.046 INFO:teuthology.orchestra.run.smithi029.stdout: "options": {}, 2024-09-16T20:50:12.047 INFO:teuthology.orchestra.run.smithi029.stdout: "application_metadata": { 2024-09-16T20:50:12.047 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd": {} 2024-09-16T20:50:12.047 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.047 INFO:teuthology.orchestra.run.smithi029.stdout: "read_balance": { 2024-09-16T20:50:12.047 INFO:teuthology.orchestra.run.smithi029.stdout: "score_type": "Fair distribution", 2024-09-16T20:50:12.047 INFO:teuthology.orchestra.run.smithi029.stdout: "score_acting": 2.25, 2024-09-16T20:50:12.047 INFO:teuthology.orchestra.run.smithi029.stdout: "score_stable": 2.25, 2024-09-16T20:50:12.047 INFO:teuthology.orchestra.run.smithi029.stdout: "optimal_score": 1, 2024-09-16T20:50:12.047 INFO:teuthology.orchestra.run.smithi029.stdout: "raw_score_acting": 2.25, 2024-09-16T20:50:12.047 INFO:teuthology.orchestra.run.smithi029.stdout: "raw_score_stable": 2.25, 2024-09-16T20:50:12.047 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity_weighted": 1, 2024-09-16T20:50:12.047 INFO:teuthology.orchestra.run.smithi029.stdout: "average_primary_affinity": 1, 2024-09-16T20:50:12.047 INFO:teuthology.orchestra.run.smithi029.stdout: "average_primary_affinity_weighted": 1 2024-09-16T20:50:12.047 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:12.047 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.048 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.048 INFO:teuthology.orchestra.run.smithi029.stdout: "pool": 3, 2024-09-16T20:50:12.048 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_name": "pool_client.0", 2024-09-16T20:50:12.048 INFO:teuthology.orchestra.run.smithi029.stdout: "create_time": "2024-09-16T20:48:06.418889+0000", 2024-09-16T20:50:12.048 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 8193, 2024-09-16T20:50:12.048 INFO:teuthology.orchestra.run.smithi029.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T20:50:12.048 INFO:teuthology.orchestra.run.smithi029.stdout: "type": 1, 2024-09-16T20:50:12.048 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 2, 2024-09-16T20:50:12.048 INFO:teuthology.orchestra.run.smithi029.stdout: "min_size": 1, 2024-09-16T20:50:12.048 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_rule": 0, 2024-09-16T20:50:12.048 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_count": 0, 2024-09-16T20:50:12.048 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_target": 0, 2024-09-16T20:50:12.048 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T20:50:12.049 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T20:50:12.049 INFO:teuthology.orchestra.run.smithi029.stdout: "is_stretch_pool": false, 2024-09-16T20:50:12.049 INFO:teuthology.orchestra.run.smithi029.stdout: "object_hash": 2, 2024-09-16T20:50:12.049 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_autoscale_mode": "off", 2024-09-16T20:50:12.049 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num": 32, 2024-09-16T20:50:12.049 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num": 32, 2024-09-16T20:50:12.049 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num_target": 32, 2024-09-16T20:50:12.049 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_target": 32, 2024-09-16T20:50:12.049 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_pending": 32, 2024-09-16T20:50:12.049 INFO:teuthology.orchestra.run.smithi029.stdout: "last_pg_merge_meta": { 2024-09-16T20:50:12.049 INFO:teuthology.orchestra.run.smithi029.stdout: "source_pgid": "0.0", 2024-09-16T20:50:12.049 INFO:teuthology.orchestra.run.smithi029.stdout: "ready_epoch": 0, 2024-09-16T20:50:12.050 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_started": 0, 2024-09-16T20:50:12.050 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_clean": 0, 2024-09-16T20:50:12.050 INFO:teuthology.orchestra.run.smithi029.stdout: "source_version": "0'0", 2024-09-16T20:50:12.050 INFO:teuthology.orchestra.run.smithi029.stdout: "target_version": "0'0" 2024-09-16T20:50:12.050 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.050 INFO:teuthology.orchestra.run.smithi029.stdout: "last_change": "126", 2024-09-16T20:50:12.050 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend": "0", 2024-09-16T20:50:12.050 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T20:50:12.051 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T20:50:12.051 INFO:teuthology.orchestra.run.smithi029.stdout: "auid": 0, 2024-09-16T20:50:12.051 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_mode": "selfmanaged", 2024-09-16T20:50:12.051 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_seq": 42, 2024-09-16T20:50:12.051 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_epoch": 126, 2024-09-16T20:50:12.051 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_snaps": [], 2024-09-16T20:50:12.051 INFO:teuthology.orchestra.run.smithi029.stdout: "removed_snaps": "[]", 2024-09-16T20:50:12.051 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_bytes": 0, 2024-09-16T20:50:12.051 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_objects": 0, 2024-09-16T20:50:12.051 INFO:teuthology.orchestra.run.smithi029.stdout: "tiers": [], 2024-09-16T20:50:12.051 INFO:teuthology.orchestra.run.smithi029.stdout: "tier_of": -1, 2024-09-16T20:50:12.051 INFO:teuthology.orchestra.run.smithi029.stdout: "read_tier": -1, 2024-09-16T20:50:12.051 INFO:teuthology.orchestra.run.smithi029.stdout: "write_tier": -1, 2024-09-16T20:50:12.051 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_mode": "none", 2024-09-16T20:50:12.052 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_bytes": 0, 2024-09-16T20:50:12.052 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_objects": 0, 2024-09-16T20:50:12.052 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T20:50:12.052 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T20:50:12.052 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T20:50:12.052 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_flush_age": 0, 2024-09-16T20:50:12.052 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_evict_age": 0, 2024-09-16T20:50:12.052 INFO:teuthology.orchestra.run.smithi029.stdout: "erasure_code_profile": "", 2024-09-16T20:50:12.052 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_params": { 2024-09-16T20:50:12.052 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "none" 2024-09-16T20:50:12.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.052 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_period": 0, 2024-09-16T20:50:12.052 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_count": 0, 2024-09-16T20:50:12.052 INFO:teuthology.orchestra.run.smithi029.stdout: "use_gmt_hitset": true, 2024-09-16T20:50:12.053 INFO:teuthology.orchestra.run.smithi029.stdout: "min_read_recency_for_promote": 0, 2024-09-16T20:50:12.053 INFO:teuthology.orchestra.run.smithi029.stdout: "min_write_recency_for_promote": 0, 2024-09-16T20:50:12.053 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T20:50:12.053 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_search_last_n": 0, 2024-09-16T20:50:12.053 INFO:teuthology.orchestra.run.smithi029.stdout: "grade_table": [], 2024-09-16T20:50:12.053 INFO:teuthology.orchestra.run.smithi029.stdout: "stripe_width": 0, 2024-09-16T20:50:12.053 INFO:teuthology.orchestra.run.smithi029.stdout: "expected_num_objects": 0, 2024-09-16T20:50:12.053 INFO:teuthology.orchestra.run.smithi029.stdout: "fast_read": false, 2024-09-16T20:50:12.053 INFO:teuthology.orchestra.run.smithi029.stdout: "options": {}, 2024-09-16T20:50:12.053 INFO:teuthology.orchestra.run.smithi029.stdout: "application_metadata": { 2024-09-16T20:50:12.053 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd": {} 2024-09-16T20:50:12.053 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.053 INFO:teuthology.orchestra.run.smithi029.stdout: "read_balance": { 2024-09-16T20:50:12.053 INFO:teuthology.orchestra.run.smithi029.stdout: "score_type": "Fair distribution", 2024-09-16T20:50:12.054 INFO:teuthology.orchestra.run.smithi029.stdout: "score_acting": 1.309999942779541, 2024-09-16T20:50:12.054 INFO:teuthology.orchestra.run.smithi029.stdout: "score_stable": 1.309999942779541, 2024-09-16T20:50:12.054 INFO:teuthology.orchestra.run.smithi029.stdout: "optimal_score": 1, 2024-09-16T20:50:12.054 INFO:teuthology.orchestra.run.smithi029.stdout: "raw_score_acting": 1.309999942779541, 2024-09-16T20:50:12.054 INFO:teuthology.orchestra.run.smithi029.stdout: "raw_score_stable": 1.309999942779541, 2024-09-16T20:50:12.054 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity_weighted": 1, 2024-09-16T20:50:12.054 INFO:teuthology.orchestra.run.smithi029.stdout: "average_primary_affinity": 1, 2024-09-16T20:50:12.054 INFO:teuthology.orchestra.run.smithi029.stdout: "average_primary_affinity_weighted": 1 2024-09-16T20:50:12.054 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:12.054 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:12.054 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-16T20:50:12.054 INFO:teuthology.orchestra.run.smithi029.stdout: "osds": [ 2024-09-16T20:50:12.054 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.054 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 0, 2024-09-16T20:50:12.055 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "6c3cb2d7-7d23-4566-9ca1-d7b08e12360c", 2024-09-16T20:50:12.055 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-16T20:50:12.055 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-16T20:50:12.055 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-16T20:50:12.055 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-16T20:50:12.055 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-16T20:50:12.055 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-16T20:50:12.055 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 13, 2024-09-16T20:50:12.055 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 125, 2024-09-16T20:50:12.055 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-16T20:50:12.055 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-16T20:50:12.055 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-16T20:50:12.055 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:50:12.055 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.056 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:50:12.056 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6816", 2024-09-16T20:50:12.056 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:50:12.056 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.056 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.056 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:50:12.056 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6817", 2024-09-16T20:50:12.056 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:50:12.056 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:12.056 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:12.056 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.056 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-16T20:50:12.057 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:50:12.057 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.057 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:50:12.057 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6818", 2024-09-16T20:50:12.057 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:50:12.057 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.057 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.057 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:50:12.057 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6819", 2024-09-16T20:50:12.057 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:50:12.057 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:12.057 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:12.057 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.057 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-16T20:50:12.058 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:50:12.058 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.058 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:50:12.058 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6822", 2024-09-16T20:50:12.058 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:50:12.058 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.058 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.058 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:50:12.058 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6823", 2024-09-16T20:50:12.058 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:50:12.058 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:12.058 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:12.059 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.059 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-16T20:50:12.059 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:50:12.059 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.059 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:50:12.059 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6820", 2024-09-16T20:50:12.059 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:50:12.059 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.059 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.059 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:50:12.060 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6821", 2024-09-16T20:50:12.060 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:50:12.060 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:12.060 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:12.060 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.060 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.29:6817/2744025949", 2024-09-16T20:50:12.060 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.29:6819/2744025949", 2024-09-16T20:50:12.060 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.29:6823/2744025949", 2024-09-16T20:50:12.060 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.29:6821/2744025949", 2024-09-16T20:50:12.060 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-16T20:50:12.061 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-16T20:50:12.061 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-16T20:50:12.061 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:12.061 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.061 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.061 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 1, 2024-09-16T20:50:12.061 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "5cf8c20c-1ed2-42ae-a8b0-c88cc4e36bce", 2024-09-16T20:50:12.061 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-16T20:50:12.061 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-16T20:50:12.061 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-16T20:50:12.061 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-16T20:50:12.061 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-16T20:50:12.061 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-16T20:50:12.061 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 13, 2024-09-16T20:50:12.062 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 81, 2024-09-16T20:50:12.062 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-16T20:50:12.062 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-16T20:50:12.062 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-16T20:50:12.062 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:50:12.062 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.062 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:50:12.062 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6800", 2024-09-16T20:50:12.062 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:50:12.062 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.062 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.062 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:50:12.062 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6801", 2024-09-16T20:50:12.062 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:50:12.063 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:12.063 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:12.063 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.063 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-16T20:50:12.063 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:50:12.063 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.063 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:50:12.063 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6802", 2024-09-16T20:50:12.063 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:50:12.063 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.063 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.063 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:50:12.063 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6803", 2024-09-16T20:50:12.063 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:50:12.064 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:12.064 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:12.064 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.064 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-16T20:50:12.064 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:50:12.064 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.064 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:50:12.064 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6806", 2024-09-16T20:50:12.064 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:50:12.064 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.064 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.064 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:50:12.064 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6807", 2024-09-16T20:50:12.064 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:50:12.065 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:12.065 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:12.065 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.065 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-16T20:50:12.065 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:50:12.065 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.065 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:50:12.065 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6804", 2024-09-16T20:50:12.065 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:50:12.065 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.065 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.065 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:50:12.066 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6805", 2024-09-16T20:50:12.066 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:50:12.066 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:12.066 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:12.066 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.066 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.29:6801/208628762", 2024-09-16T20:50:12.066 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.29:6803/208628762", 2024-09-16T20:50:12.066 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.29:6807/208628762", 2024-09-16T20:50:12.066 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.29:6805/208628762", 2024-09-16T20:50:12.066 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-16T20:50:12.066 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-16T20:50:12.066 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-16T20:50:12.066 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:12.066 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.066 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.067 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 2, 2024-09-16T20:50:12.067 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "075d16da-82bf-449b-9f6e-542c9e8d810b", 2024-09-16T20:50:12.067 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-16T20:50:12.067 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-16T20:50:12.067 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-16T20:50:12.067 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-16T20:50:12.067 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-16T20:50:12.067 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-16T20:50:12.067 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 13, 2024-09-16T20:50:12.067 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 102, 2024-09-16T20:50:12.067 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-16T20:50:12.067 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-16T20:50:12.067 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-16T20:50:12.067 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:50:12.067 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.068 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:50:12.068 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6808", 2024-09-16T20:50:12.068 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:50:12.068 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.068 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.068 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:50:12.068 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6809", 2024-09-16T20:50:12.068 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:50:12.068 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:12.068 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:12.068 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.068 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-16T20:50:12.068 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:50:12.068 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.069 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:50:12.069 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6810", 2024-09-16T20:50:12.069 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:50:12.069 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.069 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.069 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:50:12.069 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6811", 2024-09-16T20:50:12.069 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:50:12.070 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:12.070 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:12.070 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.070 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-16T20:50:12.070 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:50:12.070 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.070 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:50:12.070 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6814", 2024-09-16T20:50:12.070 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:50:12.071 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.071 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.071 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:50:12.071 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6815", 2024-09-16T20:50:12.071 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:50:12.071 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:12.071 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:12.071 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.071 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-16T20:50:12.071 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:50:12.071 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.071 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:50:12.071 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6812", 2024-09-16T20:50:12.071 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:50:12.072 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.072 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.072 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:50:12.072 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6813", 2024-09-16T20:50:12.072 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:50:12.072 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:12.072 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:12.072 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.072 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.29:6809/3447426536", 2024-09-16T20:50:12.072 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.29:6811/3447426536", 2024-09-16T20:50:12.072 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.29:6815/3447426536", 2024-09-16T20:50:12.072 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.29:6813/3447426536", 2024-09-16T20:50:12.072 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-16T20:50:12.072 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-16T20:50:12.073 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-16T20:50:12.073 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:12.073 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.073 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.073 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 3, 2024-09-16T20:50:12.073 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "0612be38-3a24-41b6-aba4-5a2fe985165e", 2024-09-16T20:50:12.073 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-16T20:50:12.073 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-16T20:50:12.073 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-16T20:50:12.073 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-16T20:50:12.073 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-16T20:50:12.073 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-16T20:50:12.073 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 13, 2024-09-16T20:50:12.074 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 126, 2024-09-16T20:50:12.074 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-16T20:50:12.074 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-16T20:50:12.074 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-16T20:50:12.074 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:50:12.074 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.074 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:50:12.074 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6808", 2024-09-16T20:50:12.074 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:50:12.074 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.074 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.074 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:50:12.074 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6809", 2024-09-16T20:50:12.074 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:50:12.075 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:12.075 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:12.075 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.075 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-16T20:50:12.075 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:50:12.075 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.075 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:50:12.075 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6810", 2024-09-16T20:50:12.075 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:50:12.075 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.075 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.075 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:50:12.075 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6811", 2024-09-16T20:50:12.075 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:50:12.076 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:12.076 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:12.076 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.076 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-16T20:50:12.076 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:50:12.076 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.076 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:50:12.076 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6814", 2024-09-16T20:50:12.076 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:50:12.076 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.076 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.076 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:50:12.076 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6815", 2024-09-16T20:50:12.076 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:50:12.077 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:12.077 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:12.077 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.077 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-16T20:50:12.077 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:50:12.077 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.077 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:50:12.077 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6812", 2024-09-16T20:50:12.077 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:50:12.077 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.077 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.077 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:50:12.078 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6813", 2024-09-16T20:50:12.078 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:50:12.078 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:12.078 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:12.078 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.078 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.43:6809/3687583891", 2024-09-16T20:50:12.078 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.43:6811/3687583891", 2024-09-16T20:50:12.078 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.43:6815/3687583891", 2024-09-16T20:50:12.078 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.43:6813/3687583891", 2024-09-16T20:50:12.078 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-16T20:50:12.078 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-16T20:50:12.078 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-16T20:50:12.078 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:12.078 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.079 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.079 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 4, 2024-09-16T20:50:12.079 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "be590bf8-e9fa-44ca-a471-6f6bcb087ca7", 2024-09-16T20:50:12.079 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-16T20:50:12.079 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-16T20:50:12.079 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-16T20:50:12.079 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-16T20:50:12.079 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-16T20:50:12.079 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-16T20:50:12.079 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 11, 2024-09-16T20:50:12.079 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 112, 2024-09-16T20:50:12.079 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-16T20:50:12.079 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-16T20:50:12.079 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-16T20:50:12.080 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:50:12.080 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.080 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:50:12.080 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6800", 2024-09-16T20:50:12.080 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:50:12.080 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.080 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.080 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:50:12.080 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6801", 2024-09-16T20:50:12.080 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:50:12.080 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:12.080 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:12.080 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.080 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-16T20:50:12.081 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:50:12.081 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.081 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:50:12.081 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6802", 2024-09-16T20:50:12.081 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:50:12.081 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.081 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.081 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:50:12.081 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6803", 2024-09-16T20:50:12.081 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:50:12.081 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:12.081 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:12.081 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.081 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-16T20:50:12.081 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:50:12.082 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.082 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:50:12.082 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6806", 2024-09-16T20:50:12.082 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:50:12.082 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.082 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.082 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:50:12.082 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6807", 2024-09-16T20:50:12.082 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:50:12.082 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:12.082 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:12.082 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.082 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-16T20:50:12.082 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:50:12.083 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.083 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:50:12.083 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6804", 2024-09-16T20:50:12.083 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:50:12.083 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.083 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.083 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:50:12.083 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6805", 2024-09-16T20:50:12.083 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:50:12.083 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:12.083 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:12.083 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.083 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.43:6801/2505220646", 2024-09-16T20:50:12.083 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.43:6803/2505220646", 2024-09-16T20:50:12.083 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.43:6807/2505220646", 2024-09-16T20:50:12.083 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.43:6805/2505220646", 2024-09-16T20:50:12.084 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-16T20:50:12.084 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-16T20:50:12.084 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-16T20:50:12.084 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:12.084 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.084 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.084 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 5, 2024-09-16T20:50:12.084 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "4a208e30-031b-4a9f-af52-e26d4a039ea6", 2024-09-16T20:50:12.084 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-16T20:50:12.084 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-16T20:50:12.084 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-16T20:50:12.085 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-16T20:50:12.085 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-16T20:50:12.085 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-16T20:50:12.085 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 13, 2024-09-16T20:50:12.085 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 103, 2024-09-16T20:50:12.085 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-16T20:50:12.085 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-16T20:50:12.085 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-16T20:50:12.085 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:50:12.085 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.085 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:50:12.085 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6816", 2024-09-16T20:50:12.086 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:50:12.086 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.086 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.086 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:50:12.086 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6817", 2024-09-16T20:50:12.086 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:50:12.086 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:12.086 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:12.086 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.086 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-16T20:50:12.086 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:50:12.086 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.086 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:50:12.086 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6818", 2024-09-16T20:50:12.086 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:50:12.087 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.087 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.087 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:50:12.087 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6819", 2024-09-16T20:50:12.087 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:50:12.087 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:12.087 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:12.087 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.087 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-16T20:50:12.087 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:50:12.087 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.087 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:50:12.088 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6822", 2024-09-16T20:50:12.088 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:50:12.088 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.088 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.088 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:50:12.088 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6823", 2024-09-16T20:50:12.088 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:50:12.088 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:12.088 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:12.088 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.088 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-16T20:50:12.088 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:50:12.088 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.088 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:50:12.089 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6820", 2024-09-16T20:50:12.089 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:50:12.089 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.089 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.089 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:50:12.089 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6821", 2024-09-16T20:50:12.089 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:50:12.089 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:12.089 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:12.089 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.089 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.43:6817/1517367307", 2024-09-16T20:50:12.089 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.43:6819/1517367307", 2024-09-16T20:50:12.089 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.43:6823/1517367307", 2024-09-16T20:50:12.090 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.43:6821/1517367307", 2024-09-16T20:50:12.090 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-16T20:50:12.090 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-16T20:50:12.090 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-16T20:50:12.090 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:12.090 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:12.090 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-16T20:50:12.090 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_xinfo": [ 2024-09-16T20:50:12.090 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.090 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 0, 2024-09-16T20:50:12.090 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-16T20:50:12.090 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-16T20:50:12.090 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-16T20:50:12.090 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540719139924082687, 2024-09-16T20:50:12.091 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-16T20:50:12.091 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-16T20:47:47.887412+0000", 2024-09-16T20:50:12.091 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-16T20:50:12.091 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.091 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.091 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 1, 2024-09-16T20:50:12.091 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-16T20:50:12.091 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-16T20:50:12.091 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-16T20:50:12.091 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540719139924082687, 2024-09-16T20:50:12.092 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-16T20:50:12.092 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-16T20:47:47.658782+0000", 2024-09-16T20:50:12.092 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-16T20:50:12.092 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.092 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.092 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 2, 2024-09-16T20:50:12.093 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-16T20:50:12.093 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-16T20:50:12.093 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-16T20:50:12.093 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540719139924082687, 2024-09-16T20:50:12.093 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-16T20:50:12.093 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-16T20:47:47.453752+0000", 2024-09-16T20:50:12.093 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-16T20:50:12.093 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.093 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.093 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 3, 2024-09-16T20:50:12.094 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-16T20:50:12.094 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-16T20:50:12.094 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-16T20:50:12.094 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540719139924082687, 2024-09-16T20:50:12.094 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-16T20:50:12.094 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-16T20:47:47.487992+0000", 2024-09-16T20:50:12.094 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-16T20:50:12.094 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.094 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.094 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 4, 2024-09-16T20:50:12.094 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-16T20:50:12.094 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-16T20:50:12.094 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-16T20:50:12.094 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540719139924082687, 2024-09-16T20:50:12.094 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-16T20:50:12.095 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-16T20:47:46.943344+0000", 2024-09-16T20:50:12.095 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-16T20:50:12.095 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.095 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.095 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 5, 2024-09-16T20:50:12.095 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-16T20:50:12.095 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-16T20:50:12.095 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-16T20:50:12.095 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540719139924082687, 2024-09-16T20:50:12.095 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-16T20:50:12.095 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-16T20:47:47.901547+0000", 2024-09-16T20:50:12.095 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-16T20:50:12.095 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:12.095 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-16T20:50:12.096 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_upmap": [], 2024-09-16T20:50:12.096 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_upmap_items": [ 2024-09-16T20:50:12.096 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.096 INFO:teuthology.orchestra.run.smithi029.stdout: "pgid": "3.a", 2024-09-16T20:50:12.096 INFO:teuthology.orchestra.run.smithi029.stdout: "mappings": [ 2024-09-16T20:50:12.096 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:12.096 INFO:teuthology.orchestra.run.smithi029.stdout: "from": 0, 2024-09-16T20:50:12.096 INFO:teuthology.orchestra.run.smithi029.stdout: "to": 4 2024-09-16T20:50:12.097 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:12.097 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:12.097 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:12.097 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-16T20:50:12.097 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_upmap_primaries": [], 2024-09-16T20:50:12.097 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_temp": [], 2024-09-16T20:50:12.097 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_temp": [], 2024-09-16T20:50:12.097 INFO:teuthology.orchestra.run.smithi029.stdout: "blocklist": {}, 2024-09-16T20:50:12.097 INFO:teuthology.orchestra.run.smithi029.stdout: "range_blocklist": {}, 2024-09-16T20:50:12.097 INFO:teuthology.orchestra.run.smithi029.stdout: "erasure_code_profiles": { 2024-09-16T20:50:12.098 INFO:teuthology.orchestra.run.smithi029.stdout: "default": { 2024-09-16T20:50:12.098 INFO:teuthology.orchestra.run.smithi029.stdout: "crush-failure-domain": "osd", 2024-09-16T20:50:12.098 INFO:teuthology.orchestra.run.smithi029.stdout: "k": "2", 2024-09-16T20:50:12.098 INFO:teuthology.orchestra.run.smithi029.stdout: "m": "1", 2024-09-16T20:50:12.098 INFO:teuthology.orchestra.run.smithi029.stdout: "plugin": "jerasure", 2024-09-16T20:50:12.098 INFO:teuthology.orchestra.run.smithi029.stdout: "technique": "reed_sol_van" 2024-09-16T20:50:12.098 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:12.098 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:12.098 INFO:teuthology.orchestra.run.smithi029.stdout: "removed_snaps_queue": [], 2024-09-16T20:50:12.098 INFO:teuthology.orchestra.run.smithi029.stdout: "new_removed_snaps": [], 2024-09-16T20:50:12.098 INFO:teuthology.orchestra.run.smithi029.stdout: "new_purged_snaps": [], 2024-09-16T20:50:12.098 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_node_flags": {}, 2024-09-16T20:50:12.098 INFO:teuthology.orchestra.run.smithi029.stdout: "device_class_flags": {}, 2024-09-16T20:50:12.098 INFO:teuthology.orchestra.run.smithi029.stdout: "stretch_mode": { 2024-09-16T20:50:12.099 INFO:teuthology.orchestra.run.smithi029.stdout: "stretch_mode_enabled": false, 2024-09-16T20:50:12.099 INFO:teuthology.orchestra.run.smithi029.stdout: "stretch_bucket_count": 0, 2024-09-16T20:50:12.099 INFO:teuthology.orchestra.run.smithi029.stdout: "degraded_stretch_mode": 0, 2024-09-16T20:50:12.099 INFO:teuthology.orchestra.run.smithi029.stdout: "recovering_stretch_mode": 0, 2024-09-16T20:50:12.099 INFO:teuthology.orchestra.run.smithi029.stdout: "stretch_mode_bucket": 0 2024-09-16T20:50:12.099 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:12.099 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:12.101 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:12.102 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:12.102 INFO:tasks.thrashosds.thrasher:j is {'epoch': 127, 'fsid': 'e28af4a7-5fcd-4de8-8f77-a2b575361bb1', 'created': '2024-09-16T20:47:39.760609+0000', 'modified': '2024-09-16T20:50:10.559128+0000', 'last_up_change': '2024-09-16T20:47:50.238013+0000', 'last_in_change': '2024-09-16T20:49:08.779548+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T20:47:52.220750+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-16T20:47:56.756666+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-16T20:48:06.418889+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 42, 'snap_epoch': 126, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.309999942779541, 'score_stable': 1.309999942779541, 'optimal_score': 1, 'raw_score_acting': 1.309999942779541, 'raw_score_stable': 1.309999942779541, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '6c3cb2d7-7d23-4566-9ca1-d7b08e12360c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 125, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6816', 'nonce': 2744025949}, {'type': 'v1', 'addr': '172.21.15.29:6817', 'nonce': 2744025949}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6818', 'nonce': 2744025949}, {'type': 'v1', 'addr': '172.21.15.29:6819', 'nonce': 2744025949}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6822', 'nonce': 2744025949}, {'type': 'v1', 'addr': '172.21.15.29:6823', 'nonce': 2744025949}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6820', 'nonce': 2744025949}, {'type': 'v1', 'addr': '172.21.15.29:6821', 'nonce': 2744025949}]}, 'public_addr': '172.21.15.29:6817/2744025949', 'cluster_addr': '172.21.15.29:6819/2744025949', 'heartbeat_back_addr': '172.21.15.29:6823/2744025949', 'heartbeat_front_addr': '172.21.15.29:6821/2744025949', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '5cf8c20c-1ed2-42ae-a8b0-c88cc4e36bce', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 81, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6800', 'nonce': 208628762}, {'type': 'v1', 'addr': '172.21.15.29:6801', 'nonce': 208628762}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6802', 'nonce': 208628762}, {'type': 'v1', 'addr': '172.21.15.29:6803', 'nonce': 208628762}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6806', 'nonce': 208628762}, {'type': 'v1', 'addr': '172.21.15.29:6807', 'nonce': 208628762}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6804', 'nonce': 208628762}, {'type': 'v1', 'addr': '172.21.15.29:6805', 'nonce': 208628762}]}, 'public_addr': '172.21.15.29:6801/208628762', 'cluster_addr': '172.21.15.29:6803/208628762', 'heartbeat_back_addr': '172.21.15.29:6807/208628762', 'heartbeat_front_addr': '172.21.15.29:6805/208628762', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '075d16da-82bf-449b-9f6e-542c9e8d810b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 102, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6808', 'nonce': 3447426536}, {'type': 'v1', 'addr': '172.21.15.29:6809', 'nonce': 3447426536}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6810', 'nonce': 3447426536}, {'type': 'v1', 'addr': '172.21.15.29:6811', 'nonce': 3447426536}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6814', 'nonce': 3447426536}, {'type': 'v1', 'addr': '172.21.15.29:6815', 'nonce': 3447426536}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6812', 'nonce': 3447426536}, {'type': 'v1', 'addr': '172.21.15.29:6813', 'nonce': 3447426536}]}, 'public_addr': '172.21.15.29:6809/3447426536', 'cluster_addr': '172.21.15.29:6811/3447426536', 'heartbeat_back_addr': '172.21.15.29:6815/3447426536', 'heartbeat_front_addr': '172.21.15.29:6813/3447426536', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '0612be38-3a24-41b6-aba4-5a2fe985165e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 126, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6808', 'nonce': 3687583891}, {'type': 'v1', 'addr': '172.21.15.43:6809', 'nonce': 3687583891}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6810', 'nonce': 3687583891}, {'type': 'v1', 'addr': '172.21.15.43:6811', 'nonce': 3687583891}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6814', 'nonce': 3687583891}, {'type': 'v1', 'addr': '172.21.15.43:6815', 'nonce': 3687583891}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6812', 'nonce': 3687583891}, {'type': 'v1', 'addr': '172.21.15.43:6813', 'nonce': 3687583891}]}, 'public_addr': '172.21.15.43:6809/3687583891', 'cluster_addr': '172.21.15.43:6811/3687583891', 'heartbeat_back_addr': '172.21.15.43:6815/3687583891', 'heartbeat_front_addr': '172.21.15.43:6813/3687583891', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'be590bf8-e9fa-44ca-a471-6f6bcb087ca7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 112, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6800', 'nonce': 2505220646}, {'type': 'v1', 'addr': '172.21.15.43:6801', 'nonce': 2505220646}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6802', 'nonce': 2505220646}, {'type': 'v1', 'addr': '172.21.15.43:6803', 'nonce': 2505220646}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6806', 'nonce': 2505220646}, {'type': 'v1', 'addr': '172.21.15.43:6807', 'nonce': 2505220646}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6804', 'nonce': 2505220646}, {'type': 'v1', 'addr': '172.21.15.43:6805', 'nonce': 2505220646}]}, 'public_addr': '172.21.15.43:6801/2505220646', 'cluster_addr': '172.21.15.43:6803/2505220646', 'heartbeat_back_addr': '172.21.15.43:6807/2505220646', 'heartbeat_front_addr': '172.21.15.43:6805/2505220646', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '4a208e30-031b-4a9f-af52-e26d4a039ea6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 103, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6816', 'nonce': 1517367307}, {'type': 'v1', 'addr': '172.21.15.43:6817', 'nonce': 1517367307}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6818', 'nonce': 1517367307}, {'type': 'v1', 'addr': '172.21.15.43:6819', 'nonce': 1517367307}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6822', 'nonce': 1517367307}, {'type': 'v1', 'addr': '172.21.15.43:6823', 'nonce': 1517367307}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6820', 'nonce': 1517367307}, {'type': 'v1', 'addr': '172.21.15.43:6821', 'nonce': 1517367307}]}, 'public_addr': '172.21.15.43:6817/1517367307', 'cluster_addr': '172.21.15.43:6819/1517367307', 'heartbeat_back_addr': '172.21.15.43:6823/1517367307', 'heartbeat_front_addr': '172.21.15.43:6821/1517367307', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T20:47:47.887412+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T20:47:47.658782+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T20:47:47.453752+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T20:47:47.487992+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T20:47:46.943344+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T20:47:47.901547+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.a', 'mappings': [{'from': 0, 'to': 4}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-16T20:50:12.103 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T20:50:12.103 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:12.099+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:12.103 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:50:12.104 INFO:teuthology.orchestra.run.smithi139.stdout:5312 trunc from 0xd297000 to 0x4dc7600 2024-09-16T20:50:12.104 INFO:teuthology.orchestra.run.smithi139.stdout:5313 punch from 0x4bb6800 to 0x4bb8400, (0x1c00 bytes) 2024-09-16T20:50:12.104 INFO:teuthology.orchestra.run.smithi139.stdout:5314 read 0x6c3c00 thru 0x6cb799 (0x7b9a bytes) 2024-09-16T20:50:12.105 INFO:teuthology.orchestra.run.smithi139.stdout:5315 read 0x1625000 thru 0x162d98b (0x898c bytes) 2024-09-16T20:50:12.105 INFO:teuthology.orchestra.run.smithi139.stdout:5316 punch from 0xc0b400 to 0xc11a00, (0x6600 bytes) 2024-09-16T20:50:12.105 INFO:teuthology.orchestra.run.smithi139.stdout:5319 write 0xd93d600 thru 0xd94a0ea (0xcaeb bytes) 2024-09-16T20:50:12.105 INFO:teuthology.orchestra.run.smithi139.stdout:5320 write 0x8729600 thru 0x872b6f2 (0x20f3 bytes) 2024-09-16T20:50:12.105 INFO:teuthology.orchestra.run.smithi139.stdout:5321 punch from 0xbc47800 to 0xbc4a600, (0x2e00 bytes) 2024-09-16T20:50:12.105 INFO:teuthology.orchestra.run.smithi139.stdout:5322 read 0x4b1d400 thru 0x4b25804 (0x8405 bytes) 2024-09-16T20:50:12.105 INFO:teuthology.orchestra.run.smithi139.stdout:5326 write 0x1e5600 thru 0x1e788a (0x228b bytes) 2024-09-16T20:50:12.105 INFO:teuthology.orchestra.run.smithi139.stdout:5328 read 0x6b7b400 thru 0x6b85d8c (0xa98d bytes) 2024-09-16T20:50:12.105 INFO:teuthology.orchestra.run.smithi139.stdout:5329 trunc from 0xd94a200 to 0x64f7e00 2024-09-16T20:50:12.134 INFO:teuthology.orchestra.run.smithi139.stdout:5331 write 0xbea9a00 thru 0xbeaa73c (0xd3d bytes) 2024-09-16T20:50:12.134 INFO:teuthology.orchestra.run.smithi139.stdout:5333 read 0x8c6a200 thru 0x8c6da03 (0x3804 bytes) 2024-09-16T20:50:12.148 INFO:teuthology.orchestra.run.smithi139.stdout:5337 write 0xd302200 thru 0xd3081ed (0x5fee bytes) 2024-09-16T20:50:12.149 INFO:teuthology.orchestra.run.smithi139.stdout:5338 write 0x2a9b000 thru 0x2aa0d63 (0x5d64 bytes) 2024-09-16T20:50:12.162 INFO:teuthology.orchestra.run.smithi139.stdout:5340 trunc from 0xd308200 to 0x261c400 2024-09-16T20:50:12.201 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:12.200+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:12.202 INFO:teuthology.orchestra.run.smithi139.stdout:5341 trunc from 0x261c400 to 0x8085e00 2024-09-16T20:50:12.210 INFO:teuthology.orchestra.run.smithi139.stdout:5342 write 0x8e7e00 thru 0x8f6488 (0xe689 bytes) 2024-09-16T20:50:12.211 INFO:teuthology.orchestra.run.smithi139.stdout:5344 read 0x7b42c00 thru 0x7b466ba (0x3abb bytes) 2024-09-16T20:50:12.212 INFO:teuthology.orchestra.run.smithi139.stdout:5346 write 0x6ecec00 thru 0x6ed30bb (0x44bc bytes) 2024-09-16T20:50:12.212 INFO:teuthology.orchestra.run.smithi139.stdout:5347 write 0x79eac00 thru 0x79ef85b (0x4c5c bytes) 2024-09-16T20:50:12.213 INFO:teuthology.orchestra.run.smithi139.stdout:5352 read 0x2aa3000 thru 0x2aad606 (0xa607 bytes) 2024-09-16T20:50:12.219 INFO:teuthology.orchestra.run.smithi139.stdout:5353 trunc from 0x8085e00 to 0xc8bdc00 2024-09-16T20:50:12.244 INFO:teuthology.orchestra.run.smithi139.stdout:5354 trunc from 0xc8bdc00 to 0xa9fa00 2024-09-16T20:50:12.255 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:12.255 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:12.265 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:50:12.270 INFO:teuthology.orchestra.run.smithi139.stdout:5355 read 0x467c00 thru 0x472db8 (0xb1b9 bytes) 2024-09-16T20:50:12.299 INFO:teuthology.orchestra.run.smithi139.stdout:5356 write 0xdc64000 thru 0xdc6f49e (0xb49f bytes) 2024-09-16T20:50:12.305 INFO:teuthology.orchestra.run.smithi139.stdout:5357 write 0x2ee600 thru 0x2f9074 (0xaa75 bytes) 2024-09-16T20:50:12.306 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:12.305+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:12.311 INFO:teuthology.orchestra.run.smithi139.stdout:5359 trunc from 0xdc6f600 to 0x9c60600 2024-09-16T20:50:12.336 INFO:teuthology.orchestra.run.smithi139.stdout:5360 trunc from 0x9c60600 to 0xe946800 2024-09-16T20:50:12.347 INFO:teuthology.orchestra.run.smithi139.stdout:5362 trunc from 0xe946800 to 0xdf4d200 2024-09-16T20:50:12.361 INFO:teuthology.orchestra.run.smithi139.stdout:5363 read 0xde99e00 thru 0xdea2ec2 (0x90c3 bytes) 2024-09-16T20:50:12.361 INFO:teuthology.orchestra.run.smithi139.stdout:5365 write 0x7a94a00 thru 0x7a99479 (0x4a7a bytes) 2024-09-16T20:50:12.362 INFO:teuthology.orchestra.run.smithi139.stdout:5369 trunc from 0xdf4d200 to 0xdbb0600 2024-09-16T20:50:12.387 INFO:teuthology.orchestra.run.smithi139.stdout:5371 trunc from 0xdbb0600 to 0x3d27000 2024-09-16T20:50:12.403 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:12.402+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:12.405 INFO:teuthology.orchestra.run.smithi139.stdout:5373 read 0x4cf400 thru 0x4d792f (0x8530 bytes) 2024-09-16T20:50:12.406 INFO:teuthology.orchestra.run.smithi139.stdout:5377 punch from 0xf35600 to 0xf36400, (0xe00 bytes) 2024-09-16T20:50:12.419 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:12.419 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:12.429 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:50:12.429 INFO:teuthology.orchestra.run.smithi139.stdout:5379 write 0xc559200 thru 0xc55fffe (0x6dff bytes) 2024-09-16T20:50:12.441 INFO:teuthology.orchestra.run.smithi139.stdout:5380 trunc from 0xc560000 to 0x2812000 2024-09-16T20:50:12.481 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:50:12.482 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-09-16T20:50:12.490 INFO:teuthology.orchestra.run.smithi139.stdout:5382 write 0x7e60200 thru 0x7e64f17 (0x4d18 bytes) 2024-09-16T20:50:12.494 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":194,"stamp":"2024-09-16T20:50:12.228055+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1629893368,"num_objects":2030,"num_object_clones":606,"num_object_copies":4060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2030,"num_whiteouts":643,"num_read":345471,"num_read_kb":1357054,"num_write":18560,"num_write_kb":402636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1730,"num_bytes_recovered":71460885,"num_keys_recovered":449,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15611,"ondisk_log_size":15611,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":84,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":854944,"kb_used_data":560868,"kb_used_omap":9,"kb_used_meta":293878,"kb_avail":565376096,"statfs":{"total":579820584960,"available":578945122304,"internally_reserved":0,"allocated":574328832,"data_stored":637166641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9567,"internal_metadata":300931745},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":107503209,"num_objects":56,"num_object_clones":20,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":13,"num_read":5650,"num_read_kb":96245,"num_write":328,"num_write_kb":66568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":6489155,"num_keys_recovered":35,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.560805"},"pg_stats":[{"pgid":"3.1f","version":"120'268","reported_seq":10159,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-16T20:50:10.562473+0000","last_change":"2024-09-16T20:49:12.802355+0000","last_active":"2024-09-16T20:50:10.562473+0000","last_peered":"2024-09-16T20:50:10.562473+0000","last_clean":"2024-09-16T20:50:10.562473+0000","last_became_active":"2024-09-16T20:49:12.801566+0000","last_became_peered":"2024-09-16T20:49:12.801566+0000","last_unstale":"2024-09-16T20:50:10.562473+0000","last_undegraded":"2024-09-16T20:50:10.562473+0000","last_fullsized":"2024-09-16T20:50:10.562473+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:43:48.625875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071634499999999996,"stat_sum":{"num_bytes":45307275,"num_objects":52,"num_object_clones":9,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":12,"num_read":9857,"num_read_kb":30801,"num_write":252,"num_write_kb":4479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":2125998,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"121'643","reported_seq":25968,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-16T20:50:10.563958+0000","last_change":"2024-09-16T20:48:53.732932+0000","last_active":"2024-09-16T20:50:10.563958+0000","last_peered":"2024-09-16T20:50:10.563958+0000","last_clean":"2024-09-16T20:50:10.563958+0000","last_became_active":"2024-09-16T20:48:53.732081+0000","last_became_peered":"2024-09-16T20:48:53.732081+0000","last_unstale":"2024-09-16T20:50:10.563958+0000","last_undegraded":"2024-09-16T20:50:10.563958+0000","last_fullsized":"2024-09-16T20:50:10.563958+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":643,"log_dups_size":0,"ondisk_log_size":643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:48:29.474984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084864999999999995,"stat_sum":{"num_bytes":95968417,"num_objects":58,"num_object_clones":16,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":14,"num_read":26929,"num_read_kb":81297,"num_write":874,"num_write_kb":9741,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":1415412,"num_keys_recovered":59,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"127'509","reported_seq":10258,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-16T20:50:11.175306+0000","last_change":"2024-09-16T20:48:09.564507+0000","last_active":"2024-09-16T20:50:11.175306+0000","last_peered":"2024-09-16T20:50:11.175306+0000","last_clean":"2024-09-16T20:50:11.175306+0000","last_became_active":"2024-09-16T20:48:07.568025+0000","last_became_peered":"2024-09-16T20:48:07.568025+0000","last_unstale":"2024-09-16T20:50:11.175306+0000","last_undegraded":"2024-09-16T20:50:11.175306+0000","last_fullsized":"2024-09-16T20:50:11.175306+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":509,"log_dups_size":0,"ondisk_log_size":509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:31:47.765202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068985699999999999,"stat_sum":{"num_bytes":74860114,"num_objects":69,"num_object_clones":21,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":20,"num_read":11078,"num_read_kb":84500,"num_write":620,"num_write_kb":26161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"127'546","reported_seq":2900,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-16T20:50:11.102442+0000","last_change":"2024-09-16T20:48:09.565661+0000","last_active":"2024-09-16T20:50:11.102442+0000","last_peered":"2024-09-16T20:50:11.102442+0000","last_clean":"2024-09-16T20:50:11.102442+0000","last_became_active":"2024-09-16T20:48:07.565241+0000","last_became_peered":"2024-09-16T20:48:07.565241+0000","last_unstale":"2024-09-16T20:50:11.102442+0000","last_undegraded":"2024-09-16T20:50:11.102442+0000","last_fullsized":"2024-09-16T20:50:11.102442+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:40:36.360154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001045589,"stat_sum":{"num_bytes":31298137,"num_objects":61,"num_object_clones":18,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":21,"num_read":4057,"num_read_kb":22088,"num_write":821,"num_write_kb":13694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"126'642","reported_seq":4289,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-16T20:50:11.103275+0000","last_change":"2024-09-16T20:48:54.735381+0000","last_active":"2024-09-16T20:50:11.103275+0000","last_peered":"2024-09-16T20:50:11.103275+0000","last_clean":"2024-09-16T20:50:11.103275+0000","last_became_active":"2024-09-16T20:48:54.734278+0000","last_became_peered":"2024-09-16T20:48:54.734278+0000","last_unstale":"2024-09-16T20:50:11.103275+0000","last_undegraded":"2024-09-16T20:50:11.103275+0000","last_fullsized":"2024-09-16T20:50:11.103275+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":642,"log_dups_size":0,"ondisk_log_size":642,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:54:19.713452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057432399999999997,"stat_sum":{"num_bytes":29757804,"num_objects":65,"num_object_clones":17,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":20,"num_read":5284,"num_read_kb":26386,"num_write":905,"num_write_kb":15051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":1963751,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"127'925","reported_seq":16250,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-16T20:50:11.220449+0000","last_change":"2024-09-16T20:49:12.803894+0000","last_active":"2024-09-16T20:50:11.220449+0000","last_peered":"2024-09-16T20:50:11.220449+0000","last_clean":"2024-09-16T20:50:11.220449+0000","last_became_active":"2024-09-16T20:49:12.802869+0000","last_became_peered":"2024-09-16T20:49:12.802869+0000","last_unstale":"2024-09-16T20:50:11.220449+0000","last_undegraded":"2024-09-16T20:50:11.220449+0000","last_fullsized":"2024-09-16T20:50:11.220449+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":925,"log_dups_size":0,"ondisk_log_size":925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:53:34.645767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002037849,"stat_sum":{"num_bytes":20120004,"num_objects":79,"num_object_clones":23,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":28,"num_read":21511,"num_read_kb":48019,"num_write":1061,"num_write_kb":13521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":1051004,"num_keys_recovered":46,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"122'543","reported_seq":10116,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-16T20:50:07.936648+0000","last_change":"2024-09-16T20:48:53.732127+0000","last_active":"2024-09-16T20:50:07.936648+0000","last_peered":"2024-09-16T20:50:07.936648+0000","last_clean":"2024-09-16T20:50:07.936648+0000","last_became_active":"2024-09-16T20:48:53.731485+0000","last_became_peered":"2024-09-16T20:48:53.731485+0000","last_unstale":"2024-09-16T20:50:07.936648+0000","last_undegraded":"2024-09-16T20:50:07.936648+0000","last_fullsized":"2024-09-16T20:50:07.936648+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":543,"log_dups_size":0,"ondisk_log_size":543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:09:40.310973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057190699999999997,"stat_sum":{"num_bytes":33923728,"num_objects":80,"num_object_clones":28,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":26,"num_read":8199,"num_read_kb":28188,"num_write":600,"num_write_kb":7725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":5091328,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"120'337","reported_seq":12578,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-16T20:50:10.563707+0000","last_change":"2024-09-16T20:48:53.731382+0000","last_active":"2024-09-16T20:50:10.563707+0000","last_peered":"2024-09-16T20:50:10.563707+0000","last_clean":"2024-09-16T20:50:10.563707+0000","last_became_active":"2024-09-16T20:48:53.730714+0000","last_became_peered":"2024-09-16T20:48:53.730714+0000","last_unstale":"2024-09-16T20:50:10.563707+0000","last_undegraded":"2024-09-16T20:50:10.563707+0000","last_fullsized":"2024-09-16T20:50:10.563707+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":337,"log_dups_size":0,"ondisk_log_size":337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:41:18.460472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011061790000000001,"stat_sum":{"num_bytes":43136016,"num_objects":50,"num_object_clones":15,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":16,"num_read":13739,"num_read_kb":51819,"num_write":272,"num_write_kb":8685,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2412544,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"117'772","reported_seq":6561,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-16T20:50:10.562521+0000","last_change":"2024-09-16T20:49:12.804399+0000","last_active":"2024-09-16T20:50:10.562521+0000","last_peered":"2024-09-16T20:50:10.562521+0000","last_clean":"2024-09-16T20:50:10.562521+0000","last_became_active":"2024-09-16T20:49:12.803628+0000","last_became_peered":"2024-09-16T20:49:12.803628+0000","last_unstale":"2024-09-16T20:50:10.562521+0000","last_undegraded":"2024-09-16T20:50:10.562521+0000","last_fullsized":"2024-09-16T20:50:10.562521+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":772,"log_dups_size":0,"ondisk_log_size":772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:04:37.759377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057915500000000003,"stat_sum":{"num_bytes":13646010,"num_objects":50,"num_object_clones":12,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":15,"num_read":10586,"num_read_kb":14830,"num_write":819,"num_write_kb":3679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":331889,"num_keys_recovered":60,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"120'1037","reported_seq":8976,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-16T20:50:10.562309+0000","last_change":"2024-09-16T20:48:53.731777+0000","last_active":"2024-09-16T20:50:10.562309+0000","last_peered":"2024-09-16T20:50:10.562309+0000","last_clean":"2024-09-16T20:50:10.562309+0000","last_became_active":"2024-09-16T20:48:53.730899+0000","last_became_peered":"2024-09-16T20:48:53.730899+0000","last_unstale":"2024-09-16T20:50:10.562309+0000","last_undegraded":"2024-09-16T20:50:10.562309+0000","last_fullsized":"2024-09-16T20:50:10.562309+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":1037,"log_dups_size":0,"ondisk_log_size":1037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:46:37.478362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010075049999999999,"stat_sum":{"num_bytes":26836201,"num_objects":59,"num_object_clones":16,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":19,"num_read":14537,"num_read_kb":37777,"num_write":1408,"num_write_kb":6931,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":6235436,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"127'268","reported_seq":11043,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-16T20:50:11.217132+0000","last_change":"2024-09-16T20:48:53.734382+0000","last_active":"2024-09-16T20:50:11.217132+0000","last_peered":"2024-09-16T20:50:11.217132+0000","last_clean":"2024-09-16T20:50:11.217132+0000","last_became_active":"2024-09-16T20:48:53.733689+0000","last_became_peered":"2024-09-16T20:48:53.733689+0000","last_unstale":"2024-09-16T20:50:11.217132+0000","last_undegraded":"2024-09-16T20:50:11.217132+0000","last_fullsized":"2024-09-16T20:50:11.217132+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:57:48.641484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051100699999999995,"stat_sum":{"num_bytes":62686208,"num_objects":65,"num_object_clones":21,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":24,"num_read":10376,"num_read_kb":50892,"num_write":267,"num_write_kb":13152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":2749952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"118'261","reported_seq":12241,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-16T20:50:10.562599+0000","last_change":"2024-09-16T20:48:09.566005+0000","last_active":"2024-09-16T20:50:10.562599+0000","last_peered":"2024-09-16T20:50:10.562599+0000","last_clean":"2024-09-16T20:50:10.562599+0000","last_became_active":"2024-09-16T20:48:07.568236+0000","last_became_peered":"2024-09-16T20:48:07.568236+0000","last_unstale":"2024-09-16T20:50:10.562599+0000","last_undegraded":"2024-09-16T20:50:10.562599+0000","last_fullsized":"2024-09-16T20:50:10.562599+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":261,"log_dups_size":0,"ondisk_log_size":261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:07:44.251070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001271749,"stat_sum":{"num_bytes":60837602,"num_objects":59,"num_object_clones":19,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":16,"num_read":12112,"num_read_kb":45193,"num_write":404,"num_write_kb":11634,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":233,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-16T20:50:10.562367+0000","last_change":"2024-09-16T20:48:00.277670+0000","last_active":"2024-09-16T20:50:10.562367+0000","last_peered":"2024-09-16T20:50:10.562367+0000","last_clean":"2024-09-16T20:50:10.562367+0000","last_became_active":"2024-09-16T20:47:58.274489+0000","last_became_peered":"2024-09-16T20:47:58.274489+0000","last_unstale":"2024-09-16T20:50:10.562367+0000","last_undegraded":"2024-09-16T20:50:10.562367+0000","last_fullsized":"2024-09-16T20:50:10.562367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:27:12.633064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091339500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"122'731","reported_seq":10549,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-16T20:50:07.936709+0000","last_change":"2024-09-16T20:48:50.757942+0000","last_active":"2024-09-16T20:50:07.936709+0000","last_peered":"2024-09-16T20:50:07.936709+0000","last_clean":"2024-09-16T20:50:07.936709+0000","last_became_active":"2024-09-16T20:48:50.719161+0000","last_became_peered":"2024-09-16T20:48:50.719161+0000","last_unstale":"2024-09-16T20:50:07.936709+0000","last_undegraded":"2024-09-16T20:50:07.936709+0000","last_fullsized":"2024-09-16T20:50:07.936709+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:10:37.189167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067924999999999995,"stat_sum":{"num_bytes":34599003,"num_objects":56,"num_object_clones":15,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":15,"num_read":12563,"num_read_kb":42377,"num_write":1014,"num_write_kb":21368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":466506,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":226,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-16T20:50:07.936891+0000","last_change":"2024-09-16T20:48:00.277182+0000","last_active":"2024-09-16T20:50:07.936891+0000","last_peered":"2024-09-16T20:50:07.936891+0000","last_clean":"2024-09-16T20:50:07.936891+0000","last_became_active":"2024-09-16T20:47:58.274389+0000","last_became_peered":"2024-09-16T20:47:58.274389+0000","last_unstale":"2024-09-16T20:50:07.936891+0000","last_undegraded":"2024-09-16T20:50:07.936891+0000","last_fullsized":"2024-09-16T20:50:07.936891+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:28:02.441830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043659299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"123'421","reported_seq":16570,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-16T20:50:07.936854+0000","last_change":"2024-09-16T20:48:50.791999+0000","last_active":"2024-09-16T20:50:07.936854+0000","last_peered":"2024-09-16T20:50:07.936854+0000","last_clean":"2024-09-16T20:50:07.936854+0000","last_became_active":"2024-09-16T20:48:50.719868+0000","last_became_peered":"2024-09-16T20:48:50.719868+0000","last_unstale":"2024-09-16T20:50:07.936854+0000","last_undegraded":"2024-09-16T20:50:07.936854+0000","last_fullsized":"2024-09-16T20:50:07.936854+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:45:12.769028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012934750000000001,"stat_sum":{"num_bytes":65974367,"num_objects":48,"num_object_clones":14,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":12,"num_read":15959,"num_read_kb":45369,"num_write":310,"num_write_kb":12985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":523776,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":233,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-16T20:50:10.562216+0000","last_change":"2024-09-16T20:48:00.278054+0000","last_active":"2024-09-16T20:50:10.562216+0000","last_peered":"2024-09-16T20:50:10.562216+0000","last_clean":"2024-09-16T20:50:10.562216+0000","last_became_active":"2024-09-16T20:47:58.272861+0000","last_became_peered":"2024-09-16T20:47:58.272861+0000","last_unstale":"2024-09-16T20:50:10.562216+0000","last_undegraded":"2024-09-16T20:50:10.562216+0000","last_fullsized":"2024-09-16T20:50:10.562216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:13:59.135547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012429489999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"123'353","reported_seq":2848,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-16T20:50:07.936727+0000","last_change":"2024-09-16T20:49:12.802627+0000","last_active":"2024-09-16T20:50:07.936727+0000","last_peered":"2024-09-16T20:50:07.936727+0000","last_clean":"2024-09-16T20:50:07.936727+0000","last_became_active":"2024-09-16T20:49:12.801810+0000","last_became_peered":"2024-09-16T20:49:12.801810+0000","last_unstale":"2024-09-16T20:50:07.936727+0000","last_undegraded":"2024-09-16T20:50:07.936727+0000","last_fullsized":"2024-09-16T20:50:07.936727+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:51:23.572817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076367000000000004,"stat_sum":{"num_bytes":30561845,"num_objects":51,"num_object_clones":17,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":15,"num_read":2606,"num_read_kb":10306,"num_write":427,"num_write_kb":5738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":2071114,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":107,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-16T20:50:10.562393+0000","last_change":"2024-09-16T20:49:09.557800+0000","last_active":"2024-09-16T20:50:10.562393+0000","last_peered":"2024-09-16T20:50:10.562393+0000","last_clean":"2024-09-16T20:50:10.562393+0000","last_became_active":"2024-09-16T20:49:09.557513+0000","last_became_peered":"2024-09-16T20:49:09.557513+0000","last_unstale":"2024-09-16T20:50:10.562393+0000","last_undegraded":"2024-09-16T20:50:10.562393+0000","last_fullsized":"2024-09-16T20:50:10.562393+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:38:59.143723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"120'499","reported_seq":17960,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-16T20:50:10.803193+0000","last_change":"2024-09-16T20:48:53.732314+0000","last_active":"2024-09-16T20:50:10.803193+0000","last_peered":"2024-09-16T20:50:10.803193+0000","last_clean":"2024-09-16T20:50:10.803193+0000","last_became_active":"2024-09-16T20:48:53.731287+0000","last_became_peered":"2024-09-16T20:48:53.731287+0000","last_unstale":"2024-09-16T20:50:10.803193+0000","last_undegraded":"2024-09-16T20:50:10.803193+0000","last_fullsized":"2024-09-16T20:50:10.803193+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":499,"log_dups_size":0,"ondisk_log_size":499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:03:29.853677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010471600000000001,"stat_sum":{"num_bytes":74738935,"num_objects":64,"num_object_clones":15,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":17,"num_read":16821,"num_read_kb":56548,"num_write":506,"num_write_kb":11756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":176266,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"122'400","reported_seq":6681,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-16T20:50:10.571546+0000","last_change":"2024-09-16T20:48:09.570254+0000","last_active":"2024-09-16T20:50:10.571546+0000","last_peered":"2024-09-16T20:50:10.571546+0000","last_clean":"2024-09-16T20:50:10.571546+0000","last_became_active":"2024-09-16T20:48:07.568143+0000","last_became_peered":"2024-09-16T20:48:07.568143+0000","last_unstale":"2024-09-16T20:50:10.571546+0000","last_undegraded":"2024-09-16T20:50:10.571546+0000","last_fullsized":"2024-09-16T20:50:10.571546+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:12:21.435205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038800899999999998,"stat_sum":{"num_bytes":71567491,"num_objects":78,"num_object_clones":27,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":32,"num_read":7029,"num_read_kb":34438,"num_write":400,"num_write_kb":12095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":235,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-16T20:50:10.572178+0000","last_change":"2024-09-16T20:48:00.276970+0000","last_active":"2024-09-16T20:50:10.572178+0000","last_peered":"2024-09-16T20:50:10.572178+0000","last_clean":"2024-09-16T20:50:10.572178+0000","last_became_active":"2024-09-16T20:47:58.274617+0000","last_became_peered":"2024-09-16T20:47:58.274617+0000","last_unstale":"2024-09-16T20:50:10.572178+0000","last_undegraded":"2024-09-16T20:50:10.572178+0000","last_fullsized":"2024-09-16T20:50:10.572178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:28:54.467905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062077499999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"122'383","reported_seq":13318,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-16T20:50:10.716772+0000","last_change":"2024-09-16T20:49:12.801644+0000","last_active":"2024-09-16T20:50:10.716772+0000","last_peered":"2024-09-16T20:50:10.716772+0000","last_clean":"2024-09-16T20:50:10.716772+0000","last_became_active":"2024-09-16T20:49:12.800827+0000","last_became_peered":"2024-09-16T20:49:12.800827+0000","last_unstale":"2024-09-16T20:50:10.716772+0000","last_undegraded":"2024-09-16T20:50:10.716772+0000","last_fullsized":"2024-09-16T20:50:10.716772+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:39:00.991202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088331900000000003,"stat_sum":{"num_bytes":60996978,"num_objects":63,"num_object_clones":14,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":19,"num_read":14712,"num_read_kb":61354,"num_write":379,"num_write_kb":15159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":1931020,"num_keys_recovered":36,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":147,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-16T20:50:10.563904+0000","last_change":"2024-09-16T20:49:09.559342+0000","last_active":"2024-09-16T20:50:10.563904+0000","last_peered":"2024-09-16T20:50:10.563904+0000","last_clean":"2024-09-16T20:50:10.563904+0000","last_became_active":"2024-09-16T20:49:09.558884+0000","last_became_peered":"2024-09-16T20:49:09.558884+0000","last_unstale":"2024-09-16T20:50:10.563904+0000","last_undegraded":"2024-09-16T20:50:10.563904+0000","last_fullsized":"2024-09-16T20:50:10.563904+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:51:23.264062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"122'245","reported_seq":4674,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-16T20:50:07.936648+0000","last_change":"2024-09-16T20:48:09.565596+0000","last_active":"2024-09-16T20:50:07.936648+0000","last_peered":"2024-09-16T20:50:07.936648+0000","last_clean":"2024-09-16T20:50:07.936648+0000","last_became_active":"2024-09-16T20:48:07.566097+0000","last_became_peered":"2024-09-16T20:48:07.566097+0000","last_unstale":"2024-09-16T20:50:07.936648+0000","last_undegraded":"2024-09-16T20:50:07.936648+0000","last_fullsized":"2024-09-16T20:50:07.936648+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":245,"log_dups_size":0,"ondisk_log_size":245,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:23:19.524493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012840460000000001,"stat_sum":{"num_bytes":56475136,"num_objects":52,"num_object_clones":16,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":20,"num_read":4266,"num_read_kb":32661,"num_write":240,"num_write_kb":11780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":245,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-16T20:50:10.562677+0000","last_change":"2024-09-16T20:49:09.560350+0000","last_active":"2024-09-16T20:50:10.562677+0000","last_peered":"2024-09-16T20:50:10.562677+0000","last_clean":"2024-09-16T20:50:10.562677+0000","last_became_active":"2024-09-16T20:49:09.559789+0000","last_became_peered":"2024-09-16T20:49:09.559789+0000","last_unstale":"2024-09-16T20:50:10.562677+0000","last_undegraded":"2024-09-16T20:50:10.562677+0000","last_fullsized":"2024-09-16T20:50:10.562677+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:17:54.706074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037636000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"127'443","reported_seq":5624,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-16T20:50:11.102889+0000","last_change":"2024-09-16T20:50:10.567395+0000","last_active":"2024-09-16T20:50:11.102889+0000","last_peered":"2024-09-16T20:50:11.102889+0000","last_clean":"2024-09-16T20:50:11.102889+0000","last_became_active":"2024-09-16T20:50:10.565111+0000","last_became_peered":"2024-09-16T20:50:10.565111+0000","last_unstale":"2024-09-16T20:50:11.102889+0000","last_undegraded":"2024-09-16T20:50:11.102889+0000","last_fullsized":"2024-09-16T20:50:11.102889+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":443,"log_dups_size":0,"ondisk_log_size":443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:14:03.597293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063857599999999997,"stat_sum":{"num_bytes":34713667,"num_objects":69,"num_object_clones":23,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":23,"num_read":8127,"num_read_kb":31404,"num_write":346,"num_write_kb":11944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":238,"num_bytes_recovered":8672913,"num_keys_recovered":92,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":234,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-16T20:50:10.562405+0000","last_change":"2024-09-16T20:48:00.277739+0000","last_active":"2024-09-16T20:50:10.562405+0000","last_peered":"2024-09-16T20:50:10.562405+0000","last_clean":"2024-09-16T20:50:10.562405+0000","last_became_active":"2024-09-16T20:47:58.271174+0000","last_became_peered":"2024-09-16T20:47:58.271174+0000","last_unstale":"2024-09-16T20:50:10.562405+0000","last_undegraded":"2024-09-16T20:50:10.562405+0000","last_fullsized":"2024-09-16T20:50:10.562405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:11:35.985075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095771799999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":348,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-16T20:50:10.562078+0000","last_change":"2024-09-16T20:49:09.565188+0000","last_active":"2024-09-16T20:50:10.562078+0000","last_peered":"2024-09-16T20:50:10.562078+0000","last_clean":"2024-09-16T20:50:10.562078+0000","last_became_active":"2024-09-16T20:49:09.559571+0000","last_became_peered":"2024-09-16T20:49:09.559571+0000","last_unstale":"2024-09-16T20:50:10.562078+0000","last_undegraded":"2024-09-16T20:50:10.562078+0000","last_fullsized":"2024-09-16T20:50:10.562078+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:52.241408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:52.241408+0000","last_clean_scrub_stamp":"2024-09-16T20:47:52.241408+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:01:53.651981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"123'537","reported_seq":5860,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-16T20:50:07.936641+0000","last_change":"2024-09-16T20:48:09.565136+0000","last_active":"2024-09-16T20:50:07.936641+0000","last_peered":"2024-09-16T20:50:07.936641+0000","last_clean":"2024-09-16T20:50:07.936641+0000","last_became_active":"2024-09-16T20:48:07.566139+0000","last_became_peered":"2024-09-16T20:48:07.566139+0000","last_unstale":"2024-09-16T20:50:07.936641+0000","last_undegraded":"2024-09-16T20:50:07.936641+0000","last_fullsized":"2024-09-16T20:50:07.936641+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":537,"log_dups_size":0,"ondisk_log_size":537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:51:49.512367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001025172,"stat_sum":{"num_bytes":59083354,"num_objects":70,"num_object_clones":17,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":20,"num_read":6685,"num_read_kb":32874,"num_write":762,"num_write_kb":7041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"127'364","reported_seq":7090,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-16T20:50:11.216208+0000","last_change":"2024-09-16T20:48:53.737981+0000","last_active":"2024-09-16T20:50:11.216208+0000","last_peered":"2024-09-16T20:50:11.216208+0000","last_clean":"2024-09-16T20:50:11.216208+0000","last_became_active":"2024-09-16T20:48:53.736558+0000","last_became_peered":"2024-09-16T20:48:53.736558+0000","last_unstale":"2024-09-16T20:50:11.216208+0000","last_undegraded":"2024-09-16T20:50:11.216208+0000","last_fullsized":"2024-09-16T20:50:11.216208+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":364,"log_dups_size":0,"ondisk_log_size":364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:48:38.155187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044614999999999998,"stat_sum":{"num_bytes":69381469,"num_objects":82,"num_object_clones":33,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":28,"num_read":7432,"num_read_kb":43697,"num_write":387,"num_write_kb":25262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":2635264,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"122'333","reported_seq":7329,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-16T20:50:07.936637+0000","last_change":"2024-09-16T20:49:48.616148+0000","last_active":"2024-09-16T20:50:07.936637+0000","last_peered":"2024-09-16T20:50:07.936637+0000","last_clean":"2024-09-16T20:50:07.936637+0000","last_became_active":"2024-09-16T20:49:48.614776+0000","last_became_peered":"2024-09-16T20:49:48.614776+0000","last_unstale":"2024-09-16T20:50:07.936637+0000","last_undegraded":"2024-09-16T20:50:07.936637+0000","last_fullsized":"2024-09-16T20:50:07.936637+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:10:12.895775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039392600000000001,"stat_sum":{"num_bytes":41170221,"num_objects":56,"num_object_clones":16,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":15,"num_read":7462,"num_read_kb":33689,"num_write":444,"num_write_kb":7927,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":3988269,"num_keys_recovered":26,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"127'313","reported_seq":5765,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-16T20:50:11.214722+0000","last_change":"2024-09-16T20:48:54.735384+0000","last_active":"2024-09-16T20:50:11.214722+0000","last_peered":"2024-09-16T20:50:11.214722+0000","last_clean":"2024-09-16T20:50:11.214722+0000","last_became_active":"2024-09-16T20:48:54.734409+0000","last_became_peered":"2024-09-16T20:48:54.734409+0000","last_unstale":"2024-09-16T20:50:11.214722+0000","last_undegraded":"2024-09-16T20:50:11.214722+0000","last_fullsized":"2024-09-16T20:50:11.214722+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:54:06.450000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014289540000000001,"stat_sum":{"num_bytes":47541321,"num_objects":57,"num_object_clones":18,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":26,"num_read":5473,"num_read_kb":16355,"num_write":327,"num_write_kb":12235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":971906,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"126'468","reported_seq":16080,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-16T20:50:11.105131+0000","last_change":"2024-09-16T20:49:30.561716+0000","last_active":"2024-09-16T20:50:11.105131+0000","last_peered":"2024-09-16T20:50:11.105131+0000","last_clean":"2024-09-16T20:50:11.105131+0000","last_became_active":"2024-09-16T20:49:30.560372+0000","last_became_peered":"2024-09-16T20:49:30.560372+0000","last_unstale":"2024-09-16T20:50:11.105131+0000","last_undegraded":"2024-09-16T20:50:11.105131+0000","last_fullsized":"2024-09-16T20:50:11.105131+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:01:32.598839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047762999999999999,"stat_sum":{"num_bytes":88212525,"num_objects":54,"num_object_clones":15,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":16,"num_read":16774,"num_read_kb":84112,"num_write":707,"num_write_kb":17903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":163,"num_bytes_recovered":18718820,"num_keys_recovered":48,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"126'915","reported_seq":8937,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-16T20:50:11.104392+0000","last_change":"2024-09-16T20:48:09.565667+0000","last_active":"2024-09-16T20:50:11.104392+0000","last_peered":"2024-09-16T20:50:11.104392+0000","last_clean":"2024-09-16T20:50:11.104392+0000","last_became_active":"2024-09-16T20:48:07.567327+0000","last_became_peered":"2024-09-16T20:48:07.567327+0000","last_unstale":"2024-09-16T20:50:11.104392+0000","last_undegraded":"2024-09-16T20:50:11.104392+0000","last_fullsized":"2024-09-16T20:50:11.104392+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":915,"log_dups_size":0,"ondisk_log_size":915,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:13:45.042220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012087560000000001,"stat_sum":{"num_bytes":37456180,"num_objects":76,"num_object_clones":26,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":28,"num_read":11027,"num_read_kb":33309,"num_write":1275,"num_write_kb":14770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"127'404","reported_seq":6648,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-16T20:50:11.177489+0000","last_change":"2024-09-16T20:48:09.565024+0000","last_active":"2024-09-16T20:50:11.177489+0000","last_peered":"2024-09-16T20:50:11.177489+0000","last_clean":"2024-09-16T20:50:11.177489+0000","last_became_active":"2024-09-16T20:48:07.570704+0000","last_became_peered":"2024-09-16T20:48:07.570704+0000","last_unstale":"2024-09-16T20:50:11.177489+0000","last_undegraded":"2024-09-16T20:50:11.177489+0000","last_fullsized":"2024-09-16T20:50:11.177489+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:29:13.218459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011026149999999999,"stat_sum":{"num_bytes":62708753,"num_objects":60,"num_object_clones":18,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":16,"num_read":6945,"num_read_kb":37004,"num_write":524,"num_write_kb":9013,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"121'389","reported_seq":12684,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-16T20:50:10.571792+0000","last_change":"2024-09-16T20:48:09.570786+0000","last_active":"2024-09-16T20:50:10.571792+0000","last_peered":"2024-09-16T20:50:10.571792+0000","last_clean":"2024-09-16T20:50:10.571792+0000","last_became_active":"2024-09-16T20:48:07.560783+0000","last_became_peered":"2024-09-16T20:48:07.560783+0000","last_unstale":"2024-09-16T20:50:10.571792+0000","last_undegraded":"2024-09-16T20:50:10.571792+0000","last_fullsized":"2024-09-16T20:50:10.571792+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:00:22.954211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040623999999999999,"stat_sum":{"num_bytes":35680538,"num_objects":85,"num_object_clones":30,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":30,"num_read":13157,"num_read_kb":47747,"num_write":352,"num_write_kb":12945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"126'454","reported_seq":12174,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-16T20:50:11.103602+0000","last_change":"2024-09-16T20:49:12.801692+0000","last_active":"2024-09-16T20:50:11.103602+0000","last_peered":"2024-09-16T20:50:11.103602+0000","last_clean":"2024-09-16T20:50:11.103602+0000","last_became_active":"2024-09-16T20:49:12.801013+0000","last_became_peered":"2024-09-16T20:49:12.801013+0000","last_unstale":"2024-09-16T20:50:11.103602+0000","last_undegraded":"2024-09-16T20:50:11.103602+0000","last_fullsized":"2024-09-16T20:50:11.103602+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:22:38.731871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115576,"stat_sum":{"num_bytes":71433353,"num_objects":82,"num_object_clones":26,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":27,"num_read":13596,"num_read_kb":50903,"num_write":362,"num_write_kb":11732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":99,"num_bytes_recovered":5210742,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"127'342","reported_seq":9772,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-16T20:50:11.226608+0000","last_change":"2024-09-16T20:48:09.564513+0000","last_active":"2024-09-16T20:50:11.226608+0000","last_peered":"2024-09-16T20:50:11.226608+0000","last_clean":"2024-09-16T20:50:11.226608+0000","last_became_active":"2024-09-16T20:48:07.571985+0000","last_became_peered":"2024-09-16T20:48:07.571985+0000","last_unstale":"2024-09-16T20:50:11.226608+0000","last_undegraded":"2024-09-16T20:50:11.226608+0000","last_fullsized":"2024-09-16T20:50:11.226608+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:05:28.179732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077335299999999995,"stat_sum":{"num_bytes":66253972,"num_objects":70,"num_object_clones":21,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":19,"num_read":9803,"num_read_kb":67328,"num_write":404,"num_write_kb":23750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"126'422","reported_seq":9875,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-16T20:50:11.103000+0000","last_change":"2024-09-16T20:49:12.804699+0000","last_active":"2024-09-16T20:50:11.103000+0000","last_peered":"2024-09-16T20:50:11.103000+0000","last_clean":"2024-09-16T20:50:11.103000+0000","last_became_active":"2024-09-16T20:49:12.803689+0000","last_became_peered":"2024-09-16T20:49:12.803689+0000","last_unstale":"2024-09-16T20:50:11.103000+0000","last_undegraded":"2024-09-16T20:50:11.103000+0000","last_fullsized":"2024-09-16T20:50:11.103000+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":422,"log_dups_size":0,"ondisk_log_size":422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:18:37.410003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000840955,"stat_sum":{"num_bytes":37889615,"num_objects":57,"num_object_clones":16,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":18,"num_read":9833,"num_read_kb":44994,"num_write":490,"num_write_kb":7865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":107,"num_bytes_recovered":1536239,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"127'369","reported_seq":6716,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-16T20:50:11.102941+0000","last_change":"2024-09-16T20:48:09.565356+0000","last_active":"2024-09-16T20:50:11.102941+0000","last_peered":"2024-09-16T20:50:11.102941+0000","last_clean":"2024-09-16T20:50:11.102941+0000","last_became_active":"2024-09-16T20:48:07.565717+0000","last_became_peered":"2024-09-16T20:48:07.565717+0000","last_unstale":"2024-09-16T20:50:11.102941+0000","last_undegraded":"2024-09-16T20:50:11.102941+0000","last_fullsized":"2024-09-16T20:50:11.102941+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:59:03.002896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013583239999999999,"stat_sum":{"num_bytes":44486742,"num_objects":49,"num_object_clones":14,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":16,"num_read":6870,"num_read_kb":28740,"num_write":498,"num_write_kb":13285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1629302981,"num_objects":2026,"num_object_clones":606,"num_object_copies":4052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2026,"num_whiteouts":643,"num_read":345405,"num_read_kb":1356999,"num_write":18457,"num_write_kb":401006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1726,"num_bytes_recovered":70280149,"num_keys_recovered":449,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":574529536,"data_stored":646864266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15536,"ondisk_log_size":15536,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574880,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81732,"kb_used_data":46380,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94290108,"statfs":{"total":96636764160,"available":96553070592,"internally_reserved":0,"allocated":47493120,"data_stored":56013523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574880,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":160964,"kb_used_data":115884,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94210876,"statfs":{"total":96636764160,"available":96471937024,"internally_reserved":0,"allocated":118665216,"data_stored":131309960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574880,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":172928,"kb_used_data":115372,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94198912,"statfs":{"total":96636764160,"available":96459685888,"internally_reserved":0,"allocated":118140928,"data_stored":126743984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574879,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":157232,"kb_used_data":99668,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94214608,"statfs":{"total":96636764160,"available":96475758592,"internally_reserved":0,"allocated":102060032,"data_stored":112971715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574880,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":184172,"kb_used_data":121052,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94187668,"statfs":{"total":96636764160,"available":96448172032,"internally_reserved":0,"allocated":123957248,"data_stored":142708841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640287,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":97916,"kb_used_data":62512,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94273924,"statfs":{"total":96636764160,"available":96536498176,"internally_reserved":0,"allocated":64012288,"data_stored":67418618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":127553536,"data_stored":147194411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":95510528,"data_stored":107314040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":122421248,"data_stored":136099564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":123314176,"data_stored":136844125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":62558208,"data_stored":66828054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":43171840,"data_stored":52584072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T20:50:12.496 INFO:tasks.thrashosds.thrasher:Setting 3.1b to [0, 1] 2024-09-16T20:50:12.496 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.1b', '0', '1'] 2024-09-16T20:50:12.496 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.1b 0 1 2024-09-16T20:50:12.497 INFO:teuthology.orchestra.run.smithi139.stdout:5384 trunc from 0x7e65000 to 0x4b1d200 2024-09-16T20:50:12.510 INFO:teuthology.orchestra.run.smithi139.stdout:5385 read 0x3aa4000 thru 0x3ab29e3 (0xe9e4 bytes) 2024-09-16T20:50:12.517 INFO:teuthology.orchestra.run.smithi139.stdout:5386 trunc from 0x4b1d200 to 0x1c99200 2024-09-16T20:50:12.536 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:12.535+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:12.543 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:12.543 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:12.544 INFO:teuthology.orchestra.run.smithi139.stdout:5389 write 0x4e9b000 thru 0x4ea26f8 (0x76f9 bytes) 2024-09-16T20:50:12.549 INFO:teuthology.orchestra.run.smithi139.stdout:5391 trunc from 0x4ea2800 to 0x485c400 2024-09-16T20:50:12.552 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:50:12.563 INFO:teuthology.orchestra.run.smithi139.stdout:5392 write 0x1ef9600 thru 0x1f07202 (0xdc03 bytes) 2024-09-16T20:50:12.582 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T20:50:12.586 INFO:teuthology.orchestra.run.smithi139.stdout:5393 write 0xc838000 thru 0xc844542 (0xc543 bytes) 2024-09-16T20:50:12.587 INFO:teuthology.orchestra.run.smithi139.stdout:5394 read 0x8520200 thru 0x852a9f2 (0xa7f3 bytes) 2024-09-16T20:50:12.587 INFO:teuthology.orchestra.run.smithi139.stdout:5399 write 0x1721000 thru 0x172cb97 (0xbb98 bytes) 2024-09-16T20:50:12.607 INFO:teuthology.orchestra.run.smithi139.stdout:5400 write 0xe69fe00 thru 0xe6ad4f5 (0xd6f6 bytes) 2024-09-16T20:50:12.608 INFO:teuthology.orchestra.run.smithi139.stdout:5403 write 0x3417200 thru 0x341d45f (0x6260 bytes) 2024-09-16T20:50:12.609 INFO:teuthology.orchestra.run.smithi139.stdout:5404 write 0x4689400 thru 0x4698e9b (0xfa9c bytes) 2024-09-16T20:50:12.609 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T20:50:12.609 INFO:teuthology.orchestra.run.smithi139.stdout:5406 write 0x7091600 thru 0x709bb4f (0xa550 bytes) 2024-09-16T20:50:12.610 INFO:teuthology.orchestra.run.smithi139.stdout:5411 read 0x4185c00 thru 0x419447f (0xe880 bytes) 2024-09-16T20:50:12.620 INFO:teuthology.orchestra.run.smithi139.stdout:5412 trunc from 0xe6ad600 to 0x6c1b800 2024-09-16T20:50:12.626 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:12.625+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:12.676 INFO:teuthology.orchestra.run.smithi139.stdout:5413 write 0x8815200 thru 0x8816c63 (0x1a64 bytes) 2024-09-16T20:50:12.677 INFO:teuthology.orchestra.run.smithi139.stdout:5414 read 0x17b7000 thru 0x17bf232 (0x8233 bytes) 2024-09-16T20:50:12.677 INFO:teuthology.orchestra.run.smithi139.stdout:5416 read 0x7a8c600 thru 0x7a9094b (0x434c bytes) 2024-09-16T20:50:12.678 INFO:teuthology.orchestra.run.smithi139.stdout:5417 write 0x6a69000 thru 0x6a710d3 (0x80d4 bytes) 2024-09-16T20:50:12.678 INFO:teuthology.orchestra.run.smithi139.stdout:5418 punch from 0x17ea200 to 0x17f5a00, (0xb800 bytes) 2024-09-16T20:50:12.679 INFO:teuthology.orchestra.run.smithi139.stdout:5419 read 0x8233800 thru 0x82394e3 (0x5ce4 bytes) 2024-09-16T20:50:12.679 INFO:teuthology.orchestra.run.smithi139.stdout:5421 read 0x3e48400 thru 0x3e51ba7 (0x97a8 bytes) 2024-09-16T20:50:12.680 INFO:teuthology.orchestra.run.smithi139.stdout:5423 punch from 0x5a81000 to 0x5a85400, (0x4400 bytes) 2024-09-16T20:50:12.680 INFO:teuthology.orchestra.run.smithi139.stdout:5424 read 0xd56a00 thru 0xd57600 (0xc01 bytes) 2024-09-16T20:50:12.703 INFO:teuthology.orchestra.run.smithi139.stdout:5426 write 0xa246e00 thru 0xa24943e (0x263f bytes) 2024-09-16T20:50:12.705 INFO:teuthology.orchestra.run.smithi139.stdout:5430 trunc from 0xa249600 to 0xac5d200 2024-09-16T20:50:12.705 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:12.704+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:12.717 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:12.717 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:12.717 INFO:teuthology.orchestra.run.smithi139.stdout:5432 read 0x5af1800 thru 0x5afff8a (0xe78b bytes) 2024-09-16T20:50:12.718 INFO:teuthology.orchestra.run.smithi139.stdout:5435 write 0x1583a00 thru 0x1585502 (0x1b03 bytes) 2024-09-16T20:50:12.718 INFO:teuthology.orchestra.run.smithi139.stdout:5437 write 0x514a600 thru 0x514f932 (0x5333 bytes) 2024-09-16T20:50:12.719 INFO:teuthology.orchestra.run.smithi139.stdout:5438 read 0x4fc9e00 thru 0x4fd2bda (0x8ddb bytes) 2024-09-16T20:50:12.720 INFO:teuthology.orchestra.run.smithi139.stdout:5439 write 0x8411a00 thru 0x8416076 (0x4677 bytes) 2024-09-16T20:50:12.720 INFO:teuthology.orchestra.run.smithi139.stdout:5440 punch from 0x62a2c00 to 0x62ae200, (0xb600 bytes) 2024-09-16T20:50:12.726 INFO:teuthology.orchestra.run.smithi139.stdout:5445 trunc from 0xac5d200 to 0xeac9800 2024-09-16T20:50:12.727 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:50:12.738 INFO:teuthology.orchestra.run.smithi139.stdout:5448 punch from 0xb0e5c00 to 0xb0e8a00, (0x2e00 bytes) 2024-09-16T20:50:12.739 INFO:teuthology.orchestra.run.smithi139.stdout:5449 write 0x9a32800 thru 0x9a33b26 (0x1327 bytes) 2024-09-16T20:50:12.740 INFO:teuthology.orchestra.run.smithi139.stdout:5450 read 0x2a55400 thru 0x2a5eec8 (0x9ac9 bytes) 2024-09-16T20:50:12.740 INFO:teuthology.orchestra.run.smithi139.stdout:5452 punch from 0x9121000 to 0x912ca00, (0xba00 bytes) 2024-09-16T20:50:12.740 INFO:teuthology.orchestra.run.smithi139.stdout:5456 write 0x19e4800 thru 0x19f262f (0xde30 bytes) 2024-09-16T20:50:12.741 INFO:teuthology.orchestra.run.smithi139.stdout:5457 punch from 0xccb2a00 to 0xccbf200, (0xc800 bytes) 2024-09-16T20:50:12.741 INFO:teuthology.orchestra.run.smithi139.stdout:5462 read 0x8a53c00 thru 0x8a5c1d9 (0x85da bytes) 2024-09-16T20:50:12.744 INFO:teuthology.orchestra.run.smithi139.stdout:5465 read 0xad83400 thru 0xad8a55d (0x715e bytes) 2024-09-16T20:50:12.745 INFO:teuthology.orchestra.run.smithi139.stdout:5467 punch from 0x67c3e00 to 0x67d2c00, (0xee00 bytes) 2024-09-16T20:50:12.745 INFO:teuthology.orchestra.run.smithi139.stdout:5469 read 0x382de00 thru 0x38339e0 (0x5be1 bytes) 2024-09-16T20:50:12.746 INFO:teuthology.orchestra.run.smithi139.stdout:5470 read 0x8360a00 thru 0x8364c51 (0x4252 bytes) 2024-09-16T20:50:12.746 INFO:teuthology.orchestra.run.smithi139.stdout:5472 punch from 0x8857c00 to 0x8863e00, (0xc200 bytes) 2024-09-16T20:50:12.746 INFO:teuthology.orchestra.run.smithi139.stdout:5475 read 0xa51f600 thru 0xa51fba8 (0x5a9 bytes) 2024-09-16T20:50:12.747 INFO:teuthology.orchestra.run.smithi139.stdout:5477 punch from 0x18b9e00 to 0x18ba600, (0x800 bytes) 2024-09-16T20:50:12.747 INFO:teuthology.orchestra.run.smithi139.stdout:5478 read 0xcbb7600 thru 0xcbc4cf9 (0xd6fa bytes) 2024-09-16T20:50:12.761 INFO:teuthology.orchestra.run.smithi139.stdout:5479 trunc from 0xeac9800 to 0x2206800 2024-09-16T20:50:12.802 INFO:teuthology.orchestra.run.smithi139.stdout:5480 read 0x12f2c00 thru 0x12f89d7 (0x5dd8 bytes) 2024-09-16T20:50:12.806 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:12.805+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:12.824 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:50:12.824 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:50:12.827 INFO:teuthology.orchestra.run.smithi139.stdout:5481 write 0xbfc9600 thru 0xbfc992e (0x32f bytes) 2024-09-16T20:50:12.829 INFO:teuthology.orchestra.run.smithi139.stdout:5482 trunc from 0xbfc9a00 to 0xb46e800 2024-09-16T20:50:12.845 INFO:teuthology.orchestra.run.smithi139.stdout:5483 read 0x650a600 thru 0x650d7c3 (0x31c4 bytes) 2024-09-16T20:50:12.859 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:50:12.859 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:50:12.864 INFO:teuthology.orchestra.run.smithi139.stdout:5485 write 0xcb61200 thru 0xcb6f7da (0xe5db bytes) 2024-09-16T20:50:12.866 INFO:teuthology.orchestra.run.smithi139.stdout:5487 punch from 0x154ec00 to 0x1558200, (0x9600 bytes) 2024-09-16T20:50:12.866 INFO:teuthology.orchestra.run.smithi139.stdout:5488 punch from 0x961ce00 to 0x962a400, (0xd600 bytes) 2024-09-16T20:50:12.868 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:12.869 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:50:12.869 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:50:12.869 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:50:12.869 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:12.875 INFO:teuthology.orchestra.run.smithi139.stdout:5491 trunc from 0xcb6f800 to 0x6222e00 2024-09-16T20:50:12.882 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:50:12.889 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:50:12.890 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:50:12.911 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:50:12.911 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:50:12.911 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:12.910+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:12.919 INFO:teuthology.orchestra.run.smithi139.stdout:5497 write 0xafbd800 thru 0xafc9310 (0xbb11 bytes) 2024-09-16T20:50:12.924 INFO:teuthology.orchestra.run.smithi139.stdout:5498 trunc from 0xafc9400 to 0xe446800 2024-09-16T20:50:12.937 INFO:teuthology.orchestra.run.smithi139.stdout:5501 write 0xad0c00 thru 0xadea1a (0xde1b bytes) 2024-09-16T20:50:12.943 INFO:teuthology.orchestra.run.smithi139.stdout:5502 write 0x6e7e200 thru 0x6e81a3d (0x383e bytes) 2024-09-16T20:50:12.943 INFO:teuthology.orchestra.run.smithi139.stdout:5503 punch from 0x833d000 to 0x834a600, (0xd600 bytes) 2024-09-16T20:50:12.944 INFO:teuthology.orchestra.run.smithi139.stdout:5505 write 0x6ba000 thru 0x6be6e6 (0x46e7 bytes) 2024-09-16T20:50:12.962 INFO:teuthology.orchestra.run.smithi139.stdout:5506 trunc from 0xe446800 to 0x1a22400 2024-09-16T20:50:13.009 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:13.008+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:13.014 INFO:teuthology.orchestra.run.smithi139.stdout:5507 write 0xc184c00 thru 0xc18995d (0x4d5e bytes) 2024-09-16T20:50:13.017 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:13.017 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:50:13.017 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:50:13.017 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:50:13.017 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.018 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2206 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1046528~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:50:13.018 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:50:13.018 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 110.679547566, 2024-09-16T20:50:13.018 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050147714000000003, 2024-09-16T20:50:13.018 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:13.018 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:13.018 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:13.018 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:13.018 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:13.018 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:13.018 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2206 2024-09-16T20:50:13.018 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.018 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:13.018 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.019 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:13.019 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:50:13.019 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.019 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.019 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.019 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:13.019 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:50:13.019 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.019 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.019 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.019 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:13.019 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:50:13.020 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8599999999999999e-07 2024-09-16T20:50:13.020 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.020 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.020 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:13.020 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336024+0000", 2024-09-16T20:50:13.020 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.934e-06 2024-09-16T20:50:13.020 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.020 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.020 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:13.020 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336037+0000", 2024-09-16T20:50:13.020 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2496e-05 2024-09-16T20:50:13.020 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.020 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.021 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:13.021 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336042+0000", 2024-09-16T20:50:13.021 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4809999999999997e-06 2024-09-16T20:50:13.021 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.021 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.021 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:13.021 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385980+0000", 2024-09-16T20:50:13.021 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049938376999999999 2024-09-16T20:50:13.021 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.021 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.021 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:13.021 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.386069+0000", 2024-09-16T20:50:13.022 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8368999999999994e-05 2024-09-16T20:50:13.022 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.022 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.022 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:13.022 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.386170+0000", 2024-09-16T20:50:13.022 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000100671 2024-09-16T20:50:13.022 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.022 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:13.022 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.022 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.022 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.022 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5439 3.a 3:5789f5c3:::rbd_data.110ed9ca540.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 1181696~60416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:13.022 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:50:13.023 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 110.150858986, 2024-09-16T20:50:13.023 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10102045599999999, 2024-09-16T20:50:13.023 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:13.023 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:13.023 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:13.023 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:13.023 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:13.023 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:13.023 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5439 2024-09-16T20:50:13.023 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.023 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:13.023 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.023 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:13.024 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:50:13.024 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.024 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.024 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.024 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:13.024 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:50:13.024 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.024 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.024 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.024 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:13.024 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864711+0000", 2024-09-16T20:50:13.025 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7899999999999997e-07 2024-09-16T20:50:13.025 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.025 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.025 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:13.025 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864760+0000", 2024-09-16T20:50:13.025 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8803e-05 2024-09-16T20:50:13.025 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.025 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.025 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:13.025 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864769+0000", 2024-09-16T20:50:13.025 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0999999999999993e-06 2024-09-16T20:50:13.025 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.026 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.026 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:13.026 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864774+0000", 2024-09-16T20:50:13.026 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8919999999999999e-06 2024-09-16T20:50:13.026 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.026 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.026 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:13.026 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864815+0000", 2024-09-16T20:50:13.026 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1307000000000003e-05 2024-09-16T20:50:13.026 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.026 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.026 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:13.026 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864893+0000", 2024-09-16T20:50:13.026 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7581000000000001e-05 2024-09-16T20:50:13.026 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.026 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.027 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 1", 2024-09-16T20:50:13.027 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864996+0000", 2024-09-16T20:50:13.027 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102725 2024-09-16T20:50:13.027 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.027 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.027 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:13.027 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965442+0000", 2024-09-16T20:50:13.027 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100446012 2024-09-16T20:50:13.027 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.027 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.027 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:13.027 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965618+0000", 2024-09-16T20:50:13.027 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00017653100000000001 2024-09-16T20:50:13.027 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.028 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.028 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:13.028 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965682+0000", 2024-09-16T20:50:13.028 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3905999999999994e-05 2024-09-16T20:50:13.028 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.028 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.028 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:13.028 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965731+0000", 2024-09-16T20:50:13.028 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8819999999999997e-05 2024-09-16T20:50:13.028 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.028 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:13.028 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.028 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.029 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.029 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4600.0:5439 3.a e37/23 ondisk, result = 0)", 2024-09-16T20:50:13.029 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:50:13.029 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 110.149862838, 2024-09-16T20:50:13.029 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099742634999999996, 2024-09-16T20:50:13.029 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:13.029 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:13.029 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:13.029 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:13.029 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.029 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:13.029 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:50:13.029 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.030 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.030 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.030 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:13.030 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:50:13.030 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.030 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.030 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.030 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:13.030 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:50:13.030 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0900000000000001e-07 2024-09-16T20:50:13.030 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.031 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.031 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:13.031 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865711+0000", 2024-09-16T20:50:13.031 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3739999999999998e-06 2024-09-16T20:50:13.031 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.031 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.031 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:13.031 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965343+0000", 2024-09-16T20:50:13.031 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099632526999999999 2024-09-16T20:50:13.031 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.031 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.031 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:13.031 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965357+0000", 2024-09-16T20:50:13.032 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3713999999999999e-05 2024-09-16T20:50:13.032 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.032 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.032 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:13.032 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965426+0000", 2024-09-16T20:50:13.032 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8745999999999995e-05 2024-09-16T20:50:13.032 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.032 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.032 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:13.032 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965434+0000", 2024-09-16T20:50:13.032 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6910000000000003e-06 2024-09-16T20:50:13.032 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.032 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.033 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:13.033 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965449+0000", 2024-09-16T20:50:13.033 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4874e-05 2024-09-16T20:50:13.033 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.033 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:13.033 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.033 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.033 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.033 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4882.0:2508 3.12 3:4992771f:::rbd_object_map.110e7c8851c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T20:50:13.033 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:50:13.033 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 86.534059290000002, 2024-09-16T20:50:13.033 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.125007011, 2024-09-16T20:50:13.033 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:13.034 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:13.034 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:13.034 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:13.034 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4882", 2024-09-16T20:50:13.034 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2436923223", 2024-09-16T20:50:13.034 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2508 2024-09-16T20:50:13.034 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.034 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:13.034 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.034 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:13.034 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:50:13.035 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.035 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.035 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.035 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:13.035 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:50:13.035 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.035 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.035 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.035 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:13.035 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481511+0000", 2024-09-16T20:50:13.035 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0699999999999996e-07 2024-09-16T20:50:13.035 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.035 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.036 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:13.036 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481515+0000", 2024-09-16T20:50:13.036 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0609999999999997e-06 2024-09-16T20:50:13.036 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.036 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.036 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:13.036 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481523+0000", 2024-09-16T20:50:13.036 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1310000000000006e-06 2024-09-16T20:50:13.036 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.036 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.036 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:13.036 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481528+0000", 2024-09-16T20:50:13.036 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4900000000000002e-06 2024-09-16T20:50:13.036 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.037 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.037 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:13.037 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481565+0000", 2024-09-16T20:50:13.037 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7066000000000001e-05 2024-09-16T20:50:13.037 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.037 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.037 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:13.037 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481628+0000", 2024-09-16T20:50:13.037 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3118000000000001e-05 2024-09-16T20:50:13.037 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.037 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.037 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 2", 2024-09-16T20:50:13.037 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481822+0000", 2024-09-16T20:50:13.038 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019446200000000001 2024-09-16T20:50:13.038 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.038 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.038 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:13.038 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605988+0000", 2024-09-16T20:50:13.038 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.124165888 2024-09-16T20:50:13.038 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.038 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.038 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:13.038 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606427+0000", 2024-09-16T20:50:13.038 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00043854899999999999 2024-09-16T20:50:13.038 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.039 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.039 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:13.039 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606476+0000", 2024-09-16T20:50:13.039 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8971999999999998e-05 2024-09-16T20:50:13.039 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.039 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.039 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:13.039 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606517+0000", 2024-09-16T20:50:13.039 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1467000000000001e-05 2024-09-16T20:50:13.039 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.039 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:13.039 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.040 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.040 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.040 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4882.0:2508 3.12 e57/55 ondisk, result = 0)", 2024-09-16T20:50:13.040 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:50:13.040 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 86.533070789999996, 2024-09-16T20:50:13.040 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12349711400000001, 2024-09-16T20:50:13.040 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:13.040 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:13.040 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:13.040 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:13.040 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.040 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:13.040 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:50:13.041 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.041 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.041 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.041 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:13.041 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:50:13.041 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.041 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.041 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.041 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:13.041 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:50:13.041 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.06e-07 2024-09-16T20:50:13.041 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.042 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.042 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:13.042 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482502+0000", 2024-09-16T20:50:13.042 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.01e-06 2024-09-16T20:50:13.042 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.042 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.042 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:13.042 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605772+0000", 2024-09-16T20:50:13.042 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.123269687 2024-09-16T20:50:13.042 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.042 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.042 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:13.042 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605791+0000", 2024-09-16T20:50:13.043 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8912000000000001e-05 2024-09-16T20:50:13.043 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.043 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.043 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:13.043 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605974+0000", 2024-09-16T20:50:13.043 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000182836 2024-09-16T20:50:13.043 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.043 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.043 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:13.043 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605982+0000", 2024-09-16T20:50:13.043 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1699999999999997e-06 2024-09-16T20:50:13.043 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.043 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.043 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:13.043 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605996+0000", 2024-09-16T20:50:13.044 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3893000000000001e-05 2024-09-16T20:50:13.044 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.044 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:13.044 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.044 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.044 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.044 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4882.0:2512 3.f 3:f4d895b7:::rbd_data.110e7c8851c.0000000000000000:head [read 16384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T20:50:13.044 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:50:13.044 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 86.532633759999996, 2024-09-16T20:50:13.044 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12318483, 2024-09-16T20:50:13.044 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:13.044 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:13.044 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:13.044 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:13.044 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4882", 2024-09-16T20:50:13.045 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2436923223", 2024-09-16T20:50:13.045 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2512 2024-09-16T20:50:13.045 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.045 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:13.045 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.045 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:13.045 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:50:13.045 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.045 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.045 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.045 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:13.045 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:50:13.045 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.045 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.045 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.046 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:13.046 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:50:13.046 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3499999999999995e-07 2024-09-16T20:50:13.046 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.046 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.046 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:13.046 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482943+0000", 2024-09-16T20:50:13.046 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3450000000000002e-06 2024-09-16T20:50:13.046 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.046 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.046 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:13.046 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605772+0000", 2024-09-16T20:50:13.046 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.122829796 2024-09-16T20:50:13.046 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.046 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.047 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:13.047 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605792+0000", 2024-09-16T20:50:13.047 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9916999999999999e-05 2024-09-16T20:50:13.047 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.047 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.047 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:13.047 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605917+0000", 2024-09-16T20:50:13.047 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00012500799999999999 2024-09-16T20:50:13.047 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.047 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.047 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:13.047 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606004+0000", 2024-09-16T20:50:13.047 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6242999999999994e-05 2024-09-16T20:50:13.047 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.047 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.048 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:13.048 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606120+0000", 2024-09-16T20:50:13.048 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000116786 2024-09-16T20:50:13.048 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.048 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:13.048 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.048 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.048 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.048 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:11702 3.15 3:acf33c8b:::rbd_data.110e747f0779.000000000000001a:head [set-alloc-hint object_size 262144 write_size 262144,write 143360~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:50:13.048 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:50:13.048 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 68.877641338000004, 2024-09-16T20:50:13.048 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.084801043000000006, 2024-09-16T20:50:13.048 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:13.048 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:13.049 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:13.049 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:13.049 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:50:13.049 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:50:13.049 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 11702 2024-09-16T20:50:13.049 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.049 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:13.049 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.049 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:13.049 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:50:13.049 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.049 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.049 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.049 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:13.049 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:50:13.050 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.050 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.050 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.050 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:13.050 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137929+0000", 2024-09-16T20:50:13.050 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8400000000000002e-07 2024-09-16T20:50:13.050 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.050 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.050 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:13.050 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137937+0000", 2024-09-16T20:50:13.050 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0260000000000007e-06 2024-09-16T20:50:13.050 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.050 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.050 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:13.050 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137952+0000", 2024-09-16T20:50:13.051 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4659e-05 2024-09-16T20:50:13.051 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.051 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.051 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:13.051 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137960+0000", 2024-09-16T20:50:13.051 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0169999999999996e-06 2024-09-16T20:50:13.051 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.051 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.051 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:13.051 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138010+0000", 2024-09-16T20:50:13.051 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0021000000000001e-05 2024-09-16T20:50:13.051 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.051 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.051 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:13.051 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138114+0000", 2024-09-16T20:50:13.052 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000103797 2024-09-16T20:50:13.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.052 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.052 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:50:13.052 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138233+0000", 2024-09-16T20:50:13.052 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000118971 2024-09-16T20:50:13.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.052 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.052 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:13.052 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138953+0000", 2024-09-16T20:50:13.052 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00072026000000000002 2024-09-16T20:50:13.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.052 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.052 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:13.053 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222609+0000", 2024-09-16T20:50:13.053 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.083656650999999999 2024-09-16T20:50:13.053 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.053 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.053 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:13.053 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222684+0000", 2024-09-16T20:50:13.053 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4369999999999994e-05 2024-09-16T20:50:13.053 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.053 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.053 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:13.053 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222729+0000", 2024-09-16T20:50:13.053 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5287000000000002e-05 2024-09-16T20:50:13.053 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.053 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:13.054 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.054 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.054 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.054 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:11704 3.9 3:91220b75:::rbd_data.110e747f0779.000000000000001c:head [set-alloc-hint object_size 262144 write_size 262144,write 139264~6656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:50:13.054 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:50:13.054 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 68.877403337000004, 2024-09-16T20:50:13.054 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.084531133999999994, 2024-09-16T20:50:13.054 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:13.054 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:13.054 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:13.054 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:13.054 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:50:13.054 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:50:13.054 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 11704 2024-09-16T20:50:13.054 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.055 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:13.055 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.055 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:13.055 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:50:13.055 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.055 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.055 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.055 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:13.055 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:50:13.055 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.055 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.055 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.055 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:13.056 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138167+0000", 2024-09-16T20:50:13.056 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.33e-07 2024-09-16T20:50:13.056 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.056 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.056 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:13.056 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138179+0000", 2024-09-16T20:50:13.056 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2055999999999999e-05 2024-09-16T20:50:13.056 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.056 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.056 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:13.056 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138193+0000", 2024-09-16T20:50:13.056 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3961000000000001e-05 2024-09-16T20:50:13.056 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.056 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.057 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:13.057 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138200+0000", 2024-09-16T20:50:13.057 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4429999999999997e-06 2024-09-16T20:50:13.057 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.057 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.057 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:13.057 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138258+0000", 2024-09-16T20:50:13.057 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7225999999999997e-05 2024-09-16T20:50:13.057 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.057 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.057 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:13.057 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138364+0000", 2024-09-16T20:50:13.057 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000106013 2024-09-16T20:50:13.057 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.057 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.058 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:50:13.058 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138494+0000", 2024-09-16T20:50:13.058 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013059599999999999 2024-09-16T20:50:13.058 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.058 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.058 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:13.058 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.139220+0000", 2024-09-16T20:50:13.058 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00072539800000000004 2024-09-16T20:50:13.058 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.058 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.058 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:13.058 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222608+0000", 2024-09-16T20:50:13.058 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.083388620999999996 2024-09-16T20:50:13.058 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.059 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.059 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:13.059 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222651+0000", 2024-09-16T20:50:13.059 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2228000000000003e-05 2024-09-16T20:50:13.059 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.059 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.059 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:13.059 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222697+0000", 2024-09-16T20:50:13.059 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6659000000000002e-05 2024-09-16T20:50:13.059 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.059 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:13.059 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.059 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.059 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.059 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5138.0:9358 3.c 3.c25699cc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T20:50:13.060 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:50:13.060 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 46.165256032000002, 2024-09-16T20:50:13.060 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0018657580000001, 2024-09-16T20:50:13.060 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:13.060 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:13.060 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:50:13.060 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:13.060 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5138", 2024-09-16T20:50:13.060 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:50:13.060 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 9358 2024-09-16T20:50:13.060 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.060 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:13.060 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.061 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:13.061 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:50:13.061 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.061 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.061 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.061 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:13.061 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:50:13.061 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.061 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.061 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.061 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:13.061 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850314+0000", 2024-09-16T20:50:13.061 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4099999999999999e-07 2024-09-16T20:50:13.061 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.061 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.062 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:13.062 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850316+0000", 2024-09-16T20:50:13.062 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2469999999999998e-06 2024-09-16T20:50:13.062 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.062 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.062 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:13.062 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850322+0000", 2024-09-16T20:50:13.062 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3470000000000004e-06 2024-09-16T20:50:13.062 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.062 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.062 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:13.062 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850327+0000", 2024-09-16T20:50:13.062 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9450000000000001e-06 2024-09-16T20:50:13.063 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.063 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.063 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:13.063 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850376+0000", 2024-09-16T20:50:13.063 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8257000000000002e-05 2024-09-16T20:50:13.063 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.063 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.063 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:13.063 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850385+0000", 2024-09-16T20:50:13.063 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9800000000000004e-06 2024-09-16T20:50:13.063 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.063 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.064 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:13.064 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852167+0000", 2024-09-16T20:50:13.064 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0017825659999999 2024-09-16T20:50:13.064 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.064 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.064 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:13.064 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852179+0000", 2024-09-16T20:50:13.064 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1975e-05 2024-09-16T20:50:13.064 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.064 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:13.064 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.064 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.064 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.065 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5138.0:9382 3.c 3.c25699cc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T20:50:13.065 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:50:13.065 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 46.156909016, 2024-09-16T20:50:13.065 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.99358404700000003, 2024-09-16T20:50:13.065 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:13.065 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:13.065 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:50:13.065 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:13.065 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5138", 2024-09-16T20:50:13.065 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:50:13.065 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 9382 2024-09-16T20:50:13.065 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.065 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:13.066 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.066 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:13.066 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:50:13.066 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.066 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.066 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.066 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:13.066 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:50:13.066 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.066 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.066 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.066 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:13.066 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858661+0000", 2024-09-16T20:50:13.066 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0200000000000002e-07 2024-09-16T20:50:13.067 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.067 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.067 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:13.067 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858664+0000", 2024-09-16T20:50:13.067 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.038e-06 2024-09-16T20:50:13.067 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.067 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.067 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:13.067 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858671+0000", 2024-09-16T20:50:13.067 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0110000000000003e-06 2024-09-16T20:50:13.067 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.067 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.067 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:13.067 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858677+0000", 2024-09-16T20:50:13.067 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0959999999999997e-06 2024-09-16T20:50:13.068 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.068 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.068 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:13.068 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858732+0000", 2024-09-16T20:50:13.068 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4543000000000003e-05 2024-09-16T20:50:13.068 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.068 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.068 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:13.068 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858739+0000", 2024-09-16T20:50:13.068 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5429999999999996e-06 2024-09-16T20:50:13.068 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.068 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.068 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:13.069 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852236+0000", 2024-09-16T20:50:13.069 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.993496453 2024-09-16T20:50:13.069 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.069 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.069 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:13.069 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852244+0000", 2024-09-16T20:50:13.069 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8610000000000004e-06 2024-09-16T20:50:13.069 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.069 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:13.069 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.069 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.069 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.069 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:361 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 372736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:50:13.070 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246714+0000", 2024-09-16T20:50:13.070 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 36.768854867999998, 2024-09-16T20:50:13.070 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050338229999999998, 2024-09-16T20:50:13.070 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:13.070 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:13.070 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:13.070 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:13.070 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:50:13.070 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:50:13.070 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 361 2024-09-16T20:50:13.070 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.070 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:13.070 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.070 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:13.071 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246714+0000", 2024-09-16T20:50:13.071 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.071 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.071 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.071 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:13.071 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246714+0000", 2024-09-16T20:50:13.071 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.071 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.071 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.071 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:13.071 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246715+0000", 2024-09-16T20:50:13.071 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1899999999999999e-07 2024-09-16T20:50:13.071 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.071 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.072 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:13.072 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246716+0000", 2024-09-16T20:50:13.072 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1659999999999999e-06 2024-09-16T20:50:13.072 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.072 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.072 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:13.072 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246721+0000", 2024-09-16T20:50:13.072 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9830000000000004e-06 2024-09-16T20:50:13.072 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.072 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.072 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:13.072 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246723+0000", 2024-09-16T20:50:13.072 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5679999999999998e-06 2024-09-16T20:50:13.073 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.073 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.073 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:13.073 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.296875+0000", 2024-09-16T20:50:13.073 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050151683000000002 2024-09-16T20:50:13.073 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.073 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.073 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:13.073 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.296959+0000", 2024-09-16T20:50:13.073 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4079999999999995e-05 2024-09-16T20:50:13.073 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.073 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.073 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:13.073 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297053+0000", 2024-09-16T20:50:13.073 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.3530999999999996e-05 2024-09-16T20:50:13.074 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.074 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:13.074 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.074 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.074 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.074 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:362 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 385024~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:50:13.074 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246727+0000", 2024-09-16T20:50:13.074 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 36.768842759000002, 2024-09-16T20:50:13.074 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050546528, 2024-09-16T20:50:13.074 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:13.074 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:13.074 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:13.074 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:13.074 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:50:13.074 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:50:13.075 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 362 2024-09-16T20:50:13.075 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.075 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:13.075 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.075 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:13.075 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246727+0000", 2024-09-16T20:50:13.075 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.075 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.075 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.075 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:13.075 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246727+0000", 2024-09-16T20:50:13.075 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.075 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.075 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.076 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:13.076 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246727+0000", 2024-09-16T20:50:13.076 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.35e-07 2024-09-16T20:50:13.076 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.076 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.076 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:13.076 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246728+0000", 2024-09-16T20:50:13.076 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.175e-06 2024-09-16T20:50:13.076 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.076 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.076 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:13.076 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246734+0000", 2024-09-16T20:50:13.076 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2160000000000003e-06 2024-09-16T20:50:13.077 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.077 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.077 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:13.077 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246737+0000", 2024-09-16T20:50:13.077 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9450000000000002e-06 2024-09-16T20:50:13.077 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.077 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.077 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:13.077 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297103+0000", 2024-09-16T20:50:13.077 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050365518999999997 2024-09-16T20:50:13.077 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.077 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.077 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:13.077 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297183+0000", 2024-09-16T20:50:13.078 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0208000000000002e-05 2024-09-16T20:50:13.078 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.078 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.078 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:13.078 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297273+0000", 2024-09-16T20:50:13.078 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0229999999999995e-05 2024-09-16T20:50:13.078 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.078 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:13.078 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.078 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.078 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.078 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:368 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 327680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:50:13.078 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246746+0000", 2024-09-16T20:50:13.079 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 36.768823478999998, 2024-09-16T20:50:13.079 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050730747999999999, 2024-09-16T20:50:13.079 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:13.079 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:13.079 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:13.079 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:13.079 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:50:13.079 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:50:13.079 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 368 2024-09-16T20:50:13.079 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.079 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:13.079 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.079 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:13.080 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246746+0000", 2024-09-16T20:50:13.080 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.080 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.080 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.080 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:13.080 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246746+0000", 2024-09-16T20:50:13.080 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.080 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.080 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.080 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:13.080 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246746+0000", 2024-09-16T20:50:13.080 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8099999999999999e-07 2024-09-16T20:50:13.080 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.080 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.081 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:13.081 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246754+0000", 2024-09-16T20:50:13.081 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4440000000000002e-06 2024-09-16T20:50:13.081 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.081 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.081 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:13.081 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246758+0000", 2024-09-16T20:50:13.081 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3900000000000003e-06 2024-09-16T20:50:13.081 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.081 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.081 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:13.081 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246761+0000", 2024-09-16T20:50:13.081 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.101e-06 2024-09-16T20:50:13.081 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.082 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.082 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:13.082 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297310+0000", 2024-09-16T20:50:13.082 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050549166 2024-09-16T20:50:13.082 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.082 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.082 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:13.082 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297387+0000", 2024-09-16T20:50:13.082 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6658999999999999e-05 2024-09-16T20:50:13.082 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.082 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.082 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:13.082 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297477+0000", 2024-09-16T20:50:13.082 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9707e-05 2024-09-16T20:50:13.083 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.083 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:13.083 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.083 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.083 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.083 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:369 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 339968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:50:13.083 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246766+0000", 2024-09-16T20:50:13.083 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 36.768803753999997, 2024-09-16T20:50:13.083 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050911291999999997, 2024-09-16T20:50:13.083 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:13.083 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:13.083 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:13.083 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:13.084 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:50:13.084 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:50:13.084 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 369 2024-09-16T20:50:13.084 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.084 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:13.084 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.084 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:13.084 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246766+0000", 2024-09-16T20:50:13.084 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.084 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.084 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.084 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:13.084 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246766+0000", 2024-09-16T20:50:13.085 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.085 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.085 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.085 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:13.085 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246766+0000", 2024-09-16T20:50:13.085 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3200000000000001e-07 2024-09-16T20:50:13.085 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.085 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.085 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:13.085 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246767+0000", 2024-09-16T20:50:13.085 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.252e-06 2024-09-16T20:50:13.085 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.085 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.085 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:13.086 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246770+0000", 2024-09-16T20:50:13.086 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1049999999999999e-06 2024-09-16T20:50:13.086 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.086 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.086 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:13.086 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246773+0000", 2024-09-16T20:50:13.086 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.396e-06 2024-09-16T20:50:13.086 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.086 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.086 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:13.086 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297513+0000", 2024-09-16T20:50:13.086 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050740800000000003 2024-09-16T20:50:13.086 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.086 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.087 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:13.087 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297589+0000", 2024-09-16T20:50:13.087 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5318e-05 2024-09-16T20:50:13.087 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.087 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.087 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:13.087 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297677+0000", 2024-09-16T20:50:13.087 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8189000000000006e-05 2024-09-16T20:50:13.087 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.087 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:13.087 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.087 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.087 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.088 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:370 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 348160~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:50:13.088 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246776+0000", 2024-09-16T20:50:13.088 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 36.768793739000003, 2024-09-16T20:50:13.088 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051096643999999997, 2024-09-16T20:50:13.088 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:13.088 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:13.088 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:13.088 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:13.088 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:50:13.088 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:50:13.088 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 370 2024-09-16T20:50:13.088 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.088 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:13.088 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.089 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:13.089 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246776+0000", 2024-09-16T20:50:13.089 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.089 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.089 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.089 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:13.089 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246776+0000", 2024-09-16T20:50:13.089 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.089 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.089 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.089 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:13.089 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246776+0000", 2024-09-16T20:50:13.090 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2499999999999999e-07 2024-09-16T20:50:13.090 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.090 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.090 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:13.090 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246777+0000", 2024-09-16T20:50:13.090 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1510000000000001e-06 2024-09-16T20:50:13.091 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.091 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.091 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:13.091 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246780+0000", 2024-09-16T20:50:13.091 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9610000000000001e-06 2024-09-16T20:50:13.091 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.091 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.091 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:13.091 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246783+0000", 2024-09-16T20:50:13.091 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6129999999999998e-06 2024-09-16T20:50:13.091 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.091 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.091 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:13.092 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297713+0000", 2024-09-16T20:50:13.092 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050929950000000002 2024-09-16T20:50:13.092 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.092 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.092 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:13.092 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297786+0000", 2024-09-16T20:50:13.092 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3096000000000006e-05 2024-09-16T20:50:13.092 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.092 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.092 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:13.092 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297872+0000", 2024-09-16T20:50:13.092 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6648000000000001e-05 2024-09-16T20:50:13.092 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.093 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:13.093 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.093 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.093 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.093 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:371 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 380928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:50:13.093 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246791+0000", 2024-09-16T20:50:13.093 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 36.768778103000002, 2024-09-16T20:50:13.093 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051287209, 2024-09-16T20:50:13.093 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:13.093 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:13.093 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:13.093 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:13.093 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:50:13.093 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:50:13.094 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 371 2024-09-16T20:50:13.094 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.094 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:13.094 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.094 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:13.094 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246791+0000", 2024-09-16T20:50:13.094 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.094 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.094 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.094 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:13.094 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246791+0000", 2024-09-16T20:50:13.094 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.094 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.095 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.095 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:13.095 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246792+0000", 2024-09-16T20:50:13.095 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8200000000000001e-07 2024-09-16T20:50:13.095 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.095 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.095 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:13.095 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246793+0000", 2024-09-16T20:50:13.095 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.28e-06 2024-09-16T20:50:13.095 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.095 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.095 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:13.095 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246796+0000", 2024-09-16T20:50:13.095 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.597e-06 2024-09-16T20:50:13.096 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.096 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.096 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:13.096 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246799+0000", 2024-09-16T20:50:13.096 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7089999999999998e-06 2024-09-16T20:50:13.096 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.096 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.096 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:13.096 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297910+0000", 2024-09-16T20:50:13.096 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051110598 2024-09-16T20:50:13.096 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.096 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.097 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:13.097 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297986+0000", 2024-09-16T20:50:13.097 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6711000000000005e-05 2024-09-16T20:50:13.097 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.097 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.097 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:13.097 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.298078+0000", 2024-09-16T20:50:13.097 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2032000000000002e-05 2024-09-16T20:50:13.097 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.097 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:13.097 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.097 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.097 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.097 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:373 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 376832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:50:13.098 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246904+0000", 2024-09-16T20:50:13.098 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 36.768665681000002, 2024-09-16T20:50:13.098 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051310617000000003, 2024-09-16T20:50:13.098 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:13.098 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:13.098 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:13.098 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:13.098 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:50:13.098 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:50:13.098 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 373 2024-09-16T20:50:13.098 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.098 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:13.098 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.098 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:13.098 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246904+0000", 2024-09-16T20:50:13.099 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.099 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.099 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.099 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:13.099 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246904+0000", 2024-09-16T20:50:13.099 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.099 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.099 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.099 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:13.099 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246904+0000", 2024-09-16T20:50:13.099 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8000000000000001e-07 2024-09-16T20:50:13.099 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.099 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.099 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:13.100 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246908+0000", 2024-09-16T20:50:13.100 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4039999999999997e-06 2024-09-16T20:50:13.100 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.100 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.100 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:13.100 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246913+0000", 2024-09-16T20:50:13.100 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0030000000000002e-06 2024-09-16T20:50:13.100 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.100 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.100 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:13.100 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246917+0000", 2024-09-16T20:50:13.100 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.303e-06 2024-09-16T20:50:13.101 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.101 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.101 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:13.101 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.298114+0000", 2024-09-16T20:50:13.101 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051197724 2024-09-16T20:50:13.101 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.101 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.101 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:13.101 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.298159+0000", 2024-09-16T20:50:13.101 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4882999999999997e-05 2024-09-16T20:50:13.101 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.101 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.101 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:13.101 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.298214+0000", 2024-09-16T20:50:13.102 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4920000000000003e-05 2024-09-16T20:50:13.102 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.102 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:13.102 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.102 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.102 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.102 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:374 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 389120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:50:13.102 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246927+0000", 2024-09-16T20:50:13.102 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 36.768642632000002, 2024-09-16T20:50:13.102 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051498164999999999, 2024-09-16T20:50:13.102 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:13.102 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:13.102 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:13.102 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:13.102 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:50:13.103 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:50:13.103 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 374 2024-09-16T20:50:13.103 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.103 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:13.103 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.103 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:13.103 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246927+0000", 2024-09-16T20:50:13.103 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.103 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.103 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.103 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:13.103 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246927+0000", 2024-09-16T20:50:13.103 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.103 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.103 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.103 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:13.104 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246927+0000", 2024-09-16T20:50:13.104 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7e-07 2024-09-16T20:50:13.104 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.104 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.104 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:13.104 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246929+0000", 2024-09-16T20:50:13.104 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.607e-06 2024-09-16T20:50:13.104 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.104 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.104 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:13.104 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246932+0000", 2024-09-16T20:50:13.104 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4539999999999999e-06 2024-09-16T20:50:13.104 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.104 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.105 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:13.105 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246935+0000", 2024-09-16T20:50:13.105 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6570000000000001e-06 2024-09-16T20:50:13.105 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.105 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.105 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:13.105 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.298242+0000", 2024-09-16T20:50:13.105 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051306887000000002 2024-09-16T20:50:13.105 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.105 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.105 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:13.105 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.298327+0000", 2024-09-16T20:50:13.105 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5068999999999995e-05 2024-09-16T20:50:13.106 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.106 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.106 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:13.106 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.298425+0000", 2024-09-16T20:50:13.106 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8021000000000002e-05 2024-09-16T20:50:13.106 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.106 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:13.106 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.106 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.106 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.106 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5763.0:4895 3.a 3:5384606c:::rbd_data.110e1b19a241.0000000000000018:head [read 0~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-16T20:50:13.106 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:45.598212+0000", 2024-09-16T20:50:13.106 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 27.417356975000001, 2024-09-16T20:50:13.107 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0375864829999999, 2024-09-16T20:50:13.107 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:13.107 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:13.107 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:13.107 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:13.107 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5763", 2024-09-16T20:50:13.107 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/623897567", 2024-09-16T20:50:13.107 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4895 2024-09-16T20:50:13.107 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.107 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:13.107 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.107 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:13.108 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598212+0000", 2024-09-16T20:50:13.108 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.108 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.108 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.108 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:13.108 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598212+0000", 2024-09-16T20:50:13.108 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.108 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.108 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.108 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:13.108 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598214+0000", 2024-09-16T20:50:13.108 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0090000000000002e-06 2024-09-16T20:50:13.109 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.109 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.109 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:13.109 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598222+0000", 2024-09-16T20:50:13.109 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6699999999999994e-06 2024-09-16T20:50:13.109 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.109 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.109 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:13.109 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598241+0000", 2024-09-16T20:50:13.109 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8896999999999999e-05 2024-09-16T20:50:13.109 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.109 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.109 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:13.109 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598250+0000", 2024-09-16T20:50:13.110 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6649999999999992e-06 2024-09-16T20:50:13.110 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.110 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.110 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:13.110 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598325+0000", 2024-09-16T20:50:13.110 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5189000000000001e-05 2024-09-16T20:50:13.110 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.110 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.110 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op must wait for map", 2024-09-16T20:50:13.110 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598333+0000", 2024-09-16T20:50:13.110 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.3170000000000002e-06 2024-09-16T20:50:13.110 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.110 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.110 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:13.111 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601355+0000", 2024-09-16T20:50:13.111 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0030215400000000001 2024-09-16T20:50:13.111 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.111 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.111 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:13.111 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601365+0000", 2024-09-16T20:50:13.111 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0215e-05 2024-09-16T20:50:13.111 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.111 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.111 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:13.111 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635506+0000", 2024-09-16T20:50:13.111 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0341408240000001 2024-09-16T20:50:13.111 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.112 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.112 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:13.112 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635655+0000", 2024-09-16T20:50:13.112 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000149412 2024-09-16T20:50:13.112 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.112 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.112 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:13.112 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635799+0000", 2024-09-16T20:50:13.112 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000143745 2024-09-16T20:50:13.112 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.112 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:13.112 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.112 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.112 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.113 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5763.0:4900 3.a 3:5384606c:::rbd_data.110e1b19a241.0000000000000018:head [read 8192~24576] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-16T20:50:13.113 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:50:13.113 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 27.417298499000001, 2024-09-16T20:50:13.113 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0379340800000001, 2024-09-16T20:50:13.113 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:13.113 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:13.113 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:13.113 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:13.113 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5763", 2024-09-16T20:50:13.113 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/623897567", 2024-09-16T20:50:13.113 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4900 2024-09-16T20:50:13.113 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.113 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:13.113 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.114 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:13.114 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:50:13.114 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.114 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.114 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.114 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:13.114 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:50:13.114 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.114 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.114 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.114 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:13.114 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:50:13.114 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4300000000000003e-07 2024-09-16T20:50:13.114 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.114 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.115 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:13.115 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598274+0000", 2024-09-16T20:50:13.115 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.976e-06 2024-09-16T20:50:13.115 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.115 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.115 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:13.115 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598282+0000", 2024-09-16T20:50:13.115 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0029999999999994e-06 2024-09-16T20:50:13.115 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.115 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.115 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:13.115 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598289+0000", 2024-09-16T20:50:13.115 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2389999999999999e-06 2024-09-16T20:50:13.116 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.116 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.116 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:13.116 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598430+0000", 2024-09-16T20:50:13.116 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014119600000000001 2024-09-16T20:50:13.116 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.116 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.116 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting_for_map not empty", 2024-09-16T20:50:13.116 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598435+0000", 2024-09-16T20:50:13.116 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3489999999999999e-06 2024-09-16T20:50:13.116 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.116 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.116 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:13.116 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601429+0000", 2024-09-16T20:50:13.117 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0029942720000000001 2024-09-16T20:50:13.117 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.117 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.117 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:13.117 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601438+0000", 2024-09-16T20:50:13.117 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4759999999999998e-06 2024-09-16T20:50:13.117 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.117 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.117 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:13.117 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635987+0000", 2024-09-16T20:50:13.117 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0345495010000001 2024-09-16T20:50:13.117 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.117 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.117 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:13.118 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.636056+0000", 2024-09-16T20:50:13.118 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8610999999999998e-05 2024-09-16T20:50:13.118 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.118 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.118 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:13.118 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.636205+0000", 2024-09-16T20:50:13.118 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014891399999999999 2024-09-16T20:50:13.118 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.118 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:13.118 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.118 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.118 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:13.118 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:13.121 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:50:13.121 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:13.118+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:13.123 INFO:teuthology.orchestra.run.smithi139.stdout:5508 write 0x9edf800 thru 0x9eea924 (0xb125 bytes) 2024-09-16T20:50:13.123 INFO:teuthology.orchestra.run.smithi139.stdout:5509 punch from 0xccc00 to 0xdb200, (0xe600 bytes) 2024-09-16T20:50:13.123 INFO:teuthology.orchestra.run.smithi139.stdout:5510 write 0xadc9600 thru 0xadce2b5 (0x4cb6 bytes) 2024-09-16T20:50:13.123 INFO:teuthology.orchestra.run.smithi139.stdout:5511 read 0x65ce000 thru 0x65d8f1d (0xaf1e bytes) 2024-09-16T20:50:13.123 INFO:teuthology.orchestra.run.smithi139.stdout:5512 write 0xbe4f000 thru 0xbe5b5fc (0xc5fd bytes) 2024-09-16T20:50:13.123 INFO:teuthology.orchestra.run.smithi139.stdout:5513 read 0x3ecc600 thru 0x3eda361 (0xdd62 bytes) 2024-09-16T20:50:13.123 INFO:teuthology.orchestra.run.smithi139.stdout:5514 write 0x286ca00 thru 0x286d321 (0x922 bytes) 2024-09-16T20:50:13.123 INFO:teuthology.orchestra.run.smithi139.stdout:5520 write 0x4f98600 thru 0x4f9b507 (0x2f08 bytes) 2024-09-16T20:50:13.123 INFO:teuthology.orchestra.run.smithi139.stdout:5521 write 0x9322600 thru 0x9323eef (0x18f0 bytes) 2024-09-16T20:50:13.123 INFO:teuthology.orchestra.run.smithi139.stdout:5523 write 0x55c4600 thru 0x55cd808 (0x9209 bytes) 2024-09-16T20:50:13.123 INFO:teuthology.orchestra.run.smithi139.stdout:5524 write 0x8451800 thru 0x845c6c7 (0xaec8 bytes) 2024-09-16T20:50:13.123 INFO:teuthology.orchestra.run.smithi139.stdout:5526 write 0x5fd6200 thru 0x5fe55a3 (0xf3a4 bytes) 2024-09-16T20:50:13.124 INFO:teuthology.orchestra.run.smithi139.stdout:5527 write 0x189ec00 thru 0x18a86fe (0x9aff bytes) 2024-09-16T20:50:13.124 INFO:teuthology.orchestra.run.smithi139.stdout:5529 read 0x3ffe600 thru 0x4007f6a (0x996b bytes) 2024-09-16T20:50:13.124 INFO:teuthology.orchestra.run.smithi139.stdout:5531 punch from 0x3bd6400 to 0x3bde200, (0x7e00 bytes) 2024-09-16T20:50:13.124 INFO:teuthology.orchestra.run.smithi139.stdout:5532 punch from 0x8edba00 to 0x8ee3e00, (0x8400 bytes) 2024-09-16T20:50:13.124 INFO:teuthology.orchestra.run.smithi139.stdout:5533 trunc from 0xc189a00 to 0x2dd3800 2024-09-16T20:50:13.124 INFO:teuthology.orchestra.run.smithi139.stdout:5534 read 0x2109e00 thru 0x210a6c6 (0x8c7 bytes) 2024-09-16T20:50:13.124 INFO:teuthology.orchestra.run.smithi139.stdout:5535 trunc from 0x2dd3800 to 0x88fbe00 2024-09-16T20:50:13.124 INFO:teuthology.orchestra.run.smithi139.stdout:5537 read 0x621f800 thru 0x62234c0 (0x3cc1 bytes) 2024-09-16T20:50:13.124 INFO:teuthology.orchestra.run.smithi139.stdout:5538 read 0x29fa800 thru 0x2a06e12 (0xc613 bytes) 2024-09-16T20:50:13.124 INFO:teuthology.orchestra.run.smithi139.stdout:5541 punch from 0x2ba600 to 0x2c0000, (0x5a00 bytes) 2024-09-16T20:50:13.124 INFO:teuthology.orchestra.run.smithi139.stdout:5544 write 0xadc4a00 thru 0xadd123c (0xc83d bytes) 2024-09-16T20:50:13.124 INFO:teuthology.orchestra.run.smithi139.stdout:5546 write 0x352ac00 thru 0x353141a (0x681b bytes) 2024-09-16T20:50:13.124 INFO:teuthology.orchestra.run.smithi139.stdout:5548 read 0x84c1c00 thru 0x84ce893 (0xcc94 bytes) 2024-09-16T20:50:13.124 INFO:teuthology.orchestra.run.smithi139.stdout:5550 write 0x6567800 thru 0x6570010 (0x8811 bytes) 2024-09-16T20:50:13.124 INFO:teuthology.orchestra.run.smithi139.stdout:5551 trunc from 0xadd1400 to 0xd980400 2024-09-16T20:50:13.134 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:50:13.135 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:50:13.135 INFO:teuthology.orchestra.run.smithi139.stdout:5552 trunc from 0xd980400 to 0x9c52e00 2024-09-16T20:50:13.150 INFO:teuthology.orchestra.run.smithi139.stdout:5554 read 0x8a8bc00 thru 0x8a9609f (0xa4a0 bytes) 2024-09-16T20:50:13.152 INFO:teuthology.orchestra.run.smithi139.stdout:5555 write 0x5c39600 thru 0x5c39927 (0x328 bytes) 2024-09-16T20:50:13.153 INFO:teuthology.orchestra.run.smithi139.stdout:5556 read 0x65d2e00 thru 0x65db1fb (0x83fc bytes) 2024-09-16T20:50:13.156 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:50:13.157 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:50:13.160 INFO:teuthology.orchestra.run.smithi139.stdout:5560 trunc from 0x9c52e00 to 0x47bca00 2024-09-16T20:50:13.196 INFO:teuthology.orchestra.run.smithi139.stdout:5561 write 0xe753c00 thru 0xe75a7bd (0x6bbe bytes) 2024-09-16T20:50:13.198 INFO:teuthology.orchestra.run.smithi139.stdout:5563 write 0x9677600 thru 0x967d97e (0x637f bytes) 2024-09-16T20:50:13.198 INFO:teuthology.orchestra.run.smithi139.stdout:5564 read 0x9527200 thru 0x95327ca (0xb5cb bytes) 2024-09-16T20:50:13.206 INFO:teuthology.orchestra.run.smithi139.stdout:5568 trunc from 0xe75a800 to 0x8197a00 2024-09-16T20:50:13.220 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:13.219+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:13.230 INFO:teuthology.orchestra.run.smithi139.stdout:5570 trunc from 0x8197a00 to 0x4a78600 2024-09-16T20:50:13.246 INFO:teuthology.orchestra.run.smithi139.stdout:5572 trunc from 0x4a78600 to 0x48d0a00 2024-09-16T20:50:13.262 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:13.262 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:50:13.262 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:50:13.262 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:13.263 INFO:teuthology.orchestra.run.smithi139.stdout:5573 write 0x1371000 thru 0x137f460 (0xe461 bytes) 2024-09-16T20:50:13.269 INFO:teuthology.orchestra.run.smithi139.stdout:5574 read 0x3a57200 thru 0x3a61270 (0xa071 bytes) 2024-09-16T20:50:13.269 INFO:teuthology.orchestra.run.smithi139.stdout:5575 read 0x21a0c00 thru 0x21ac62d (0xba2e bytes) 2024-09-16T20:50:13.270 INFO:teuthology.orchestra.run.smithi139.stdout:5576 punch from 0x7efc00 to 0x7fec00, (0xf000 bytes) 2024-09-16T20:50:13.271 INFO:teuthology.orchestra.run.smithi139.stdout:5577 write 0x2099800 thru 0x20a927c (0xfa7d bytes) 2024-09-16T20:50:13.271 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:50:13.274 INFO:teuthology.orchestra.run.smithi139.stdout:5580 trunc from 0x48d0a00 to 0x30d0200 2024-09-16T20:50:13.287 INFO:teuthology.orchestra.run.smithi139.stdout:5581 punch from 0x28c1400 to 0x28c5800, (0x4400 bytes) 2024-09-16T20:50:13.294 INFO:teuthology.orchestra.run.smithi139.stdout:5582 read 0x2139400 thru 0x2143339 (0x9f3a bytes) 2024-09-16T20:50:13.294 INFO:teuthology.orchestra.run.smithi139.stdout:5587 punch from 0x1a88c00 to 0x1a8ca00, (0x3e00 bytes) 2024-09-16T20:50:13.294 INFO:teuthology.orchestra.run.smithi139.stdout:5588 read 0x18d3600 thru 0x18dac6a (0x766b bytes) 2024-09-16T20:50:13.295 INFO:teuthology.orchestra.run.smithi139.stdout:5590 read 0x7b2200 thru 0x7bc44d (0xa24e bytes) 2024-09-16T20:50:13.295 INFO:teuthology.orchestra.run.smithi139.stdout:5592 write 0x22b1800 thru 0x22be2de (0xcadf bytes) 2024-09-16T20:50:13.302 INFO:teuthology.orchestra.run.smithi139.stdout:5597 trunc from 0x30d0200 to 0x8649e00 2024-09-16T20:50:13.312 INFO:teuthology.orchestra.run.smithi139.stdout:5599 read 0x9ca00 thru 0xa675b (0x9d5c bytes) 2024-09-16T20:50:13.318 INFO:teuthology.orchestra.run.smithi139.stdout:5600 write 0x16e4600 thru 0x16eeee2 (0xa8e3 bytes) 2024-09-16T20:50:13.319 INFO:teuthology.orchestra.run.smithi139.stdout:5601 punch from 0x6e98e00 to 0x6ea2a00, (0x9c00 bytes) 2024-09-16T20:50:13.319 INFO:teuthology.orchestra.run.smithi139.stdout:5602 read 0x3723600 thru 0x3732c91 (0xf692 bytes) 2024-09-16T20:50:13.319 INFO:teuthology.orchestra.run.smithi139.stdout:5603 write 0x78be600 thru 0x78c31ed (0x4bee bytes) 2024-09-16T20:50:13.320 INFO:teuthology.orchestra.run.smithi139.stdout:5605 read 0x286c00 thru 0x28fda4 (0x91a5 bytes) 2024-09-16T20:50:13.320 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:13.319+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:13.322 INFO:teuthology.orchestra.run.smithi139.stdout:5606 trunc from 0x8649e00 to 0x9932200 2024-09-16T20:50:13.334 INFO:teuthology.orchestra.run.smithi139.stdout:5607 write 0x4f3e200 thru 0x4f4413a (0x5f3b bytes) 2024-09-16T20:50:13.340 INFO:teuthology.orchestra.run.smithi139.stdout:5609 punch from 0x3a98a00 to 0x3a9fc00, (0x7200 bytes) 2024-09-16T20:50:13.340 INFO:teuthology.orchestra.run.smithi139.stdout:5610 write 0x3ce8000 thru 0x3ce9700 (0x1701 bytes) 2024-09-16T20:50:13.340 INFO:teuthology.orchestra.run.smithi139.stdout:5611 trunc from 0x9932200 to 0x955ea00 2024-09-16T20:50:13.353 INFO:teuthology.orchestra.run.smithi139.stdout:5612 read 0x2f19600 thru 0x2f19827 (0x228 bytes) 2024-09-16T20:50:13.356 INFO:teuthology.orchestra.run.smithi139.stdout:5614 trunc from 0x955ea00 to 0xae49600 2024-09-16T20:50:13.366 INFO:teuthology.orchestra.run.smithi139.stdout:5616 read 0x3d8fc00 thru 0x3d9e165 (0xe566 bytes) 2024-09-16T20:50:13.368 INFO:teuthology.orchestra.run.smithi139.stdout:5619 punch from 0x5078200 to 0x5087c00, (0xfa00 bytes) 2024-09-16T20:50:13.369 INFO:teuthology.orchestra.run.smithi139.stdout:5620 write 0x6d07800 thru 0x6d14b75 (0xd376 bytes) 2024-09-16T20:50:13.369 INFO:teuthology.orchestra.run.smithi139.stdout:5627 read 0x1b7ba00 thru 0x1b87a08 (0xc009 bytes) 2024-09-16T20:50:13.377 INFO:teuthology.orchestra.run.smithi139.stdout:5628 trunc from 0xae49600 to 0x41f6000 2024-09-16T20:50:13.395 INFO:teuthology.orchestra.run.smithi139.stdout:5629 read 0x3737c00 thru 0x3742dbf (0xb1c0 bytes) 2024-09-16T20:50:13.419 INFO:teuthology.orchestra.run.smithi139.stdout:5630 write 0x907f200 thru 0x90828ff (0x3700 bytes) 2024-09-16T20:50:13.420 INFO:teuthology.orchestra.run.smithi139.stdout:5631 punch from 0x53fc000 to 0x5404a00, (0x8a00 bytes) 2024-09-16T20:50:13.421 INFO:teuthology.orchestra.run.smithi139.stdout:5632 write 0x88a5a00 thru 0x88a8bfe (0x31ff bytes) 2024-09-16T20:50:13.421 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:13.420+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:13.426 INFO:teuthology.orchestra.run.smithi139.stdout:5634 trunc from 0x9082a00 to 0x45c2200 2024-09-16T20:50:13.435 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:13.435 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:50:13.436 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:50:13.436 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:50:13.436 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:13.445 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:50:13.448 INFO:teuthology.orchestra.run.smithi139.stdout:5638 trunc from 0x45c2200 to 0x8b10e00 2024-09-16T20:50:13.457 INFO:teuthology.orchestra.run.smithi139.stdout:5639 punch from 0x3102e00 to 0x3111200, (0xe400 bytes) 2024-09-16T20:50:13.462 INFO:teuthology.orchestra.run.smithi139.stdout:5640 read 0x6475600 thru 0x6476f65 (0x1966 bytes) 2024-09-16T20:50:13.465 INFO:teuthology.orchestra.run.smithi139.stdout:5649 trunc from 0x8b10e00 to 0x35ee000 2024-09-16T20:50:13.480 INFO:teuthology.orchestra.run.smithi139.stdout:5650 punch from 0x2099c00 to 0x209f600, (0x5a00 bytes) 2024-09-16T20:50:13.482 INFO:teuthology.orchestra.run.smithi139.stdout:5655 read 0x107da00 thru 0x107ec9e (0x129f bytes) 2024-09-16T20:50:13.482 INFO:teuthology.orchestra.run.smithi139.stdout:5657 write 0x71d400 thru 0x71f07c (0x1c7d bytes) 2024-09-16T20:50:13.502 INFO:teuthology.orchestra.run.smithi139.stdout:5662 write 0xa14a200 thru 0xa14ab53 (0x954 bytes) 2024-09-16T20:50:13.505 INFO:teuthology.orchestra.run.smithi139.stdout:5665 write 0x29c7400 thru 0x29c7a1d (0x61e bytes) 2024-09-16T20:50:13.507 INFO:teuthology.orchestra.run.smithi139.stdout:5666 trunc from 0xa14ac00 to 0xe01ac00 2024-09-16T20:50:13.519 INFO:teuthology.orchestra.run.smithi139.stdout:5667 write 0x7e45600 thru 0x7e4c5e0 (0x6fe1 bytes) 2024-09-16T20:50:13.520 INFO:teuthology.orchestra.run.smithi139.stdout:5668 read 0xd01d600 thru 0xd01ea72 (0x1473 bytes) 2024-09-16T20:50:13.521 INFO:teuthology.orchestra.run.smithi139.stdout:5672 punch from 0x6534600 to 0x653e800, (0xa200 bytes) 2024-09-16T20:50:13.524 INFO:teuthology.orchestra.run.smithi139.stdout:5673 write 0x6f26800 thru 0x6f34a4e (0xe24f bytes) 2024-09-16T20:50:13.524 INFO:teuthology.orchestra.run.smithi139.stdout:5674 punch from 0xb377a00 to 0xb379e00, (0x2400 bytes) 2024-09-16T20:50:13.524 INFO:teuthology.orchestra.run.smithi139.stdout:5677 read 0xa05a800 thru 0xa05c50f (0x1d10 bytes) 2024-09-16T20:50:13.527 INFO:teuthology.orchestra.run.smithi139.stdout:5679 trunc from 0xe01ac00 to 0xac1f800 2024-09-16T20:50:13.531 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:13.530+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:13.543 INFO:teuthology.orchestra.run.smithi139.stdout:5682 read 0x901e400 thru 0x902d4c5 (0xf0c6 bytes) 2024-09-16T20:50:13.545 INFO:teuthology.orchestra.run.smithi139.stdout:5683 read 0x10b5000 thru 0x10b9662 (0x4663 bytes) 2024-09-16T20:50:13.545 INFO:teuthology.orchestra.run.smithi139.stdout:5685 punch from 0x35dd000 to 0x35df400, (0x2400 bytes) 2024-09-16T20:50:13.548 INFO:teuthology.orchestra.run.smithi139.stdout:5688 write 0x63af600 thru 0x63bc153 (0xcb54 bytes) 2024-09-16T20:50:13.548 INFO:teuthology.orchestra.run.smithi139.stdout:5689 punch from 0xab88a00 to 0xab94800, (0xbe00 bytes) 2024-09-16T20:50:13.548 INFO:teuthology.orchestra.run.smithi139.stdout:5691 write 0x90a7e00 thru 0x90b7ccc (0xfecd bytes) 2024-09-16T20:50:13.548 INFO:teuthology.orchestra.run.smithi139.stdout:5695 read 0x3574000 thru 0x357853b (0x453c bytes) 2024-09-16T20:50:13.548 INFO:teuthology.orchestra.run.smithi139.stdout:5696 read 0xa12fa00 thru 0xa138811 (0x8e12 bytes) 2024-09-16T20:50:13.548 INFO:teuthology.orchestra.run.smithi139.stdout:5697 read 0x37c800 thru 0x38633c (0x9b3d bytes) 2024-09-16T20:50:13.549 INFO:teuthology.orchestra.run.smithi139.stdout:5698 trunc from 0xac1f800 to 0xa2b4400 2024-09-16T20:50:13.580 INFO:teuthology.orchestra.run.smithi139.stdout:5701 write 0xc81fc00 thru 0xc826dfc (0x71fd bytes) 2024-09-16T20:50:13.585 INFO:teuthology.orchestra.run.smithi139.stdout:5705 read 0x24ffe00 thru 0x2506a94 (0x6c95 bytes) 2024-09-16T20:50:13.585 INFO:teuthology.orchestra.run.smithi139.stdout:5708 read 0xa57be00 thru 0xa57d4d9 (0x16da bytes) 2024-09-16T20:50:13.586 INFO:teuthology.orchestra.run.smithi139.stdout:5710 read 0x1c4a600 thru 0x1c5963e (0xf03f bytes) 2024-09-16T20:50:13.586 INFO:teuthology.orchestra.run.smithi139.stdout:5712 write 0x74b4c00 thru 0x74bb297 (0x6698 bytes) 2024-09-16T20:50:13.587 INFO:teuthology.orchestra.run.smithi139.stdout:5714 punch from 0x127e800 to 0x1285600, (0x6e00 bytes) 2024-09-16T20:50:13.587 INFO:teuthology.orchestra.run.smithi139.stdout:5719 clone 41 order 18 su 16384 sc 6 2024-09-16T20:50:13.612 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:13.612 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:50:13.612 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:50:13.612 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:50:13.612 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.612 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:50:13.612 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:50:13.612 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 140.35608503, 2024-09-16T20:50:13.612 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67502346999999996, 2024-09-16T20:50:13.612 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:13.613 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:13.613 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:13.613 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:13.613 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:50:13.613 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:50:13.613 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 1 2024-09-16T20:50:13.613 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.613 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:13.613 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.613 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:13.613 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:50:13.613 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.614 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.614 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.614 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:13.614 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:50:13.614 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.614 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.614 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.614 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:13.614 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254128+0000", 2024-09-16T20:50:13.614 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.463e-06 2024-09-16T20:50:13.614 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.614 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.614 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:13.614 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254135+0000", 2024-09-16T20:50:13.615 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1579999999999999e-06 2024-09-16T20:50:13.615 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.615 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.615 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:13.615 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254146+0000", 2024-09-16T20:50:13.615 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1341e-05 2024-09-16T20:50:13.615 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.615 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.615 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:13.615 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254155+0000", 2024-09-16T20:50:13.615 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.1549999999999996e-06 2024-09-16T20:50:13.615 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.615 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.616 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:13.616 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254221+0000", 2024-09-16T20:50:13.616 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5470999999999997e-05 2024-09-16T20:50:13.616 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.616 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.616 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:13.616 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254231+0000", 2024-09-16T20:50:13.616 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0592e-05 2024-09-16T20:50:13.616 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.616 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.616 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:13.616 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926711+0000", 2024-09-16T20:50:13.617 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67247946599999997 2024-09-16T20:50:13.617 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.617 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.617 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:13.617 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926910+0000", 2024-09-16T20:50:13.617 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019934 2024-09-16T20:50:13.617 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.617 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.617 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:50:13.617 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.927159+0000", 2024-09-16T20:50:13.617 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00024876599999999998 2024-09-16T20:50:13.617 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.617 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.618 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:13.618 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.928797+0000", 2024-09-16T20:50:13.618 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001638308 2024-09-16T20:50:13.618 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.618 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.618 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:13.618 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929026+0000", 2024-09-16T20:50:13.618 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00022876099999999999 2024-09-16T20:50:13.618 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.618 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.618 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:13.618 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929102+0000", 2024-09-16T20:50:13.618 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6184000000000001e-05 2024-09-16T20:50:13.618 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.619 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.619 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:13.619 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929150+0000", 2024-09-16T20:50:13.619 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7465e-05 2024-09-16T20:50:13.619 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.619 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:13.619 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.619 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.619 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.619 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3574 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:13.619 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:50:13.619 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 84.128921497999997, 2024-09-16T20:50:13.619 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053579441999999998, 2024-09-16T20:50:13.620 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:13.620 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:13.620 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:13.620 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:13.620 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:13.620 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:13.620 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3574 2024-09-16T20:50:13.620 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.620 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:13.620 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.620 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:13.620 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:50:13.620 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.621 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.621 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.621 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:13.621 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:50:13.621 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.621 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.621 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.621 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:13.621 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:50:13.621 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2e-07 2024-09-16T20:50:13.621 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.621 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.621 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:13.622 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481292+0000", 2024-09-16T20:50:13.622 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2630000000000002e-06 2024-09-16T20:50:13.622 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.622 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.622 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:13.622 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481297+0000", 2024-09-16T20:50:13.622 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.296e-06 2024-09-16T20:50:13.622 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.622 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.622 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:13.622 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481300+0000", 2024-09-16T20:50:13.622 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7019999999999999e-06 2024-09-16T20:50:13.622 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.623 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.623 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:13.623 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525533+0000", 2024-09-16T20:50:13.623 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.044232318 2024-09-16T20:50:13.623 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.623 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.623 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:13.623 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525619+0000", 2024-09-16T20:50:13.623 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5892999999999999e-05 2024-09-16T20:50:13.623 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.623 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.623 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:13.623 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525987+0000", 2024-09-16T20:50:13.623 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00036836099999999999 2024-09-16T20:50:13.624 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.624 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.624 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:13.624 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531779+0000", 2024-09-16T20:50:13.624 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0057918240000000001 2024-09-16T20:50:13.624 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.624 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.624 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:13.624 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534747+0000", 2024-09-16T20:50:13.624 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0029681600000000001 2024-09-16T20:50:13.624 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.624 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.624 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:13.625 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534832+0000", 2024-09-16T20:50:13.625 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5180999999999996e-05 2024-09-16T20:50:13.625 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.625 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.625 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:13.625 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534869+0000", 2024-09-16T20:50:13.625 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7024000000000003e-05 2024-09-16T20:50:13.625 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.625 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:13.625 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.625 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.625 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.625 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3575 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:13.626 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:50:13.626 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 84.128906757999999, 2024-09-16T20:50:13.626 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053801463000000001, 2024-09-16T20:50:13.626 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:13.626 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:13.626 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:13.626 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:13.626 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:13.626 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:13.626 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3575 2024-09-16T20:50:13.626 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.626 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:13.626 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.627 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:13.627 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:50:13.627 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.627 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.627 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.627 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:13.627 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:50:13.627 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.627 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.627 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.627 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:13.627 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481305+0000", 2024-09-16T20:50:13.627 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5400000000000002e-07 2024-09-16T20:50:13.628 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.628 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.628 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:13.628 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481307+0000", 2024-09-16T20:50:13.628 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7800000000000001e-06 2024-09-16T20:50:13.628 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.628 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.628 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:13.628 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481312+0000", 2024-09-16T20:50:13.628 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6349999999999997e-06 2024-09-16T20:50:13.628 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.628 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.628 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:13.629 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481315+0000", 2024-09-16T20:50:13.629 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7829999999999999e-06 2024-09-16T20:50:13.629 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.629 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.629 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:13.629 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.526876+0000", 2024-09-16T20:50:13.629 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.045561070000000002 2024-09-16T20:50:13.629 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.629 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.629 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:13.629 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.526954+0000", 2024-09-16T20:50:13.629 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8108000000000005e-05 2024-09-16T20:50:13.630 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.630 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.630 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:13.630 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527282+0000", 2024-09-16T20:50:13.630 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000327483 2024-09-16T20:50:13.630 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.630 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.630 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:13.630 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531784+0000", 2024-09-16T20:50:13.630 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0045020169999999997 2024-09-16T20:50:13.630 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.630 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.630 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:13.630 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534921+0000", 2024-09-16T20:50:13.631 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.003137791 2024-09-16T20:50:13.631 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.631 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.631 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:13.631 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534976+0000", 2024-09-16T20:50:13.631 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4953000000000001e-05 2024-09-16T20:50:13.631 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.631 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.631 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:13.631 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535106+0000", 2024-09-16T20:50:13.631 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000129589 2024-09-16T20:50:13.631 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.631 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:13.632 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.632 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.632 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.632 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3576 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:13.632 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:50:13.632 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 84.128892845999999, 2024-09-16T20:50:13.632 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053955436000000002, 2024-09-16T20:50:13.632 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:13.632 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:13.632 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:13.632 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:13.632 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:13.632 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:13.633 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3576 2024-09-16T20:50:13.633 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.633 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:13.633 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.633 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:13.633 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:50:13.633 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.633 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.633 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.633 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:13.633 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:50:13.633 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.633 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.634 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.634 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:13.634 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481319+0000", 2024-09-16T20:50:13.634 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.53e-07 2024-09-16T20:50:13.634 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.634 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.634 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:13.634 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481320+0000", 2024-09-16T20:50:13.634 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5230000000000001e-06 2024-09-16T20:50:13.634 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.634 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.634 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:13.634 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481330+0000", 2024-09-16T20:50:13.634 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0190000000000001e-05 2024-09-16T20:50:13.635 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.635 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.635 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:13.635 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481333+0000", 2024-09-16T20:50:13.635 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.151e-06 2024-09-16T20:50:13.635 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.635 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.635 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:13.635 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527689+0000", 2024-09-16T20:50:13.635 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046355192000000003 2024-09-16T20:50:13.635 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.635 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.635 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:13.635 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527764+0000", 2024-09-16T20:50:13.636 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5776000000000001e-05 2024-09-16T20:50:13.636 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.636 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.636 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:13.636 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528141+0000", 2024-09-16T20:50:13.636 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00037647599999999999 2024-09-16T20:50:13.636 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.636 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.636 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:13.636 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531788+0000", 2024-09-16T20:50:13.636 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0036470679999999998 2024-09-16T20:50:13.636 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.636 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.637 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:13.637 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535174+0000", 2024-09-16T20:50:13.637 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0033862670000000001 2024-09-16T20:50:13.637 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.637 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.637 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:13.637 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535236+0000", 2024-09-16T20:50:13.637 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1797999999999993e-05 2024-09-16T20:50:13.637 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.637 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.637 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:13.637 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535274+0000", 2024-09-16T20:50:13.637 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7741999999999999e-05 2024-09-16T20:50:13.638 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.638 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:13.638 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.638 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.638 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.638 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3577 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:13.638 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:50:13.638 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 84.128871246000003, 2024-09-16T20:50:13.638 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054073807000000002, 2024-09-16T20:50:13.639 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:13.639 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:13.639 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:13.639 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:13.639 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:13.639 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:13.639 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3577 2024-09-16T20:50:13.639 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.639 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:13.640 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.640 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:13.640 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:50:13.640 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.640 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.640 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.640 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:13.640 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:50:13.640 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.640 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.640 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.640 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:13.641 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:50:13.641 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.96e-07 2024-09-16T20:50:13.641 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.641 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.641 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:13.641 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481342+0000", 2024-09-16T20:50:13.641 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.714e-06 2024-09-16T20:50:13.641 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.641 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.641 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:13.641 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481349+0000", 2024-09-16T20:50:13.641 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4930000000000001e-06 2024-09-16T20:50:13.641 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.642 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.642 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:13.642 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481353+0000", 2024-09-16T20:50:13.642 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4879999999999999e-06 2024-09-16T20:50:13.642 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.642 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.642 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:13.642 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528619+0000", 2024-09-16T20:50:13.642 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047266351999999998 2024-09-16T20:50:13.642 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.642 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.642 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:13.642 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528706+0000", 2024-09-16T20:50:13.642 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6796e-05 2024-09-16T20:50:13.643 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.643 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.643 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:13.643 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529148+0000", 2024-09-16T20:50:13.643 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00044182999999999999 2024-09-16T20:50:13.643 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.643 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.643 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:13.643 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531792+0000", 2024-09-16T20:50:13.643 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0026443119999999998 2024-09-16T20:50:13.643 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.643 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.643 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:13.643 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535319+0000", 2024-09-16T20:50:13.643 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0035266439999999998 2024-09-16T20:50:13.644 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.644 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.644 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:13.644 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535372+0000", 2024-09-16T20:50:13.644 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3207000000000001e-05 2024-09-16T20:50:13.644 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.644 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.644 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:13.644 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535414+0000", 2024-09-16T20:50:13.644 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1675000000000003e-05 2024-09-16T20:50:13.644 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.644 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:13.644 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.644 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.644 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.645 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3578 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:13.645 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:50:13.645 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 84.128850076999996, 2024-09-16T20:50:13.645 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054189586999999997, 2024-09-16T20:50:13.645 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:13.645 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:13.645 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:13.645 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:13.645 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:13.645 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:13.645 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3578 2024-09-16T20:50:13.645 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.646 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:13.646 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.646 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:13.646 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:50:13.646 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.646 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.646 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.646 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:13.646 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:50:13.646 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.646 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.646 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.646 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:13.646 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:50:13.647 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0800000000000001e-07 2024-09-16T20:50:13.647 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.647 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.647 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:13.647 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481363+0000", 2024-09-16T20:50:13.647 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0339999999999999e-06 2024-09-16T20:50:13.647 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.647 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.647 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:13.647 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481371+0000", 2024-09-16T20:50:13.647 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4440000000000002e-06 2024-09-16T20:50:13.647 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.647 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.648 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:13.648 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481374+0000", 2024-09-16T20:50:13.648 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2049999999999998e-06 2024-09-16T20:50:13.648 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.648 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.648 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:13.648 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529555+0000", 2024-09-16T20:50:13.648 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048181202999999999 2024-09-16T20:50:13.648 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.648 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.648 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:13.648 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529616+0000", 2024-09-16T20:50:13.648 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0912000000000002e-05 2024-09-16T20:50:13.649 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.649 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.649 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:13.649 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529939+0000", 2024-09-16T20:50:13.649 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00032269800000000001 2024-09-16T20:50:13.649 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.649 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.649 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:13.649 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532602+0000", 2024-09-16T20:50:13.649 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0026634940000000002 2024-09-16T20:50:13.649 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.649 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.650 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:13.650 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535465+0000", 2024-09-16T20:50:13.650 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.002862149 2024-09-16T20:50:13.650 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.650 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.650 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:13.650 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535516+0000", 2024-09-16T20:50:13.650 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1696000000000001e-05 2024-09-16T20:50:13.650 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.650 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.650 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:13.650 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535551+0000", 2024-09-16T20:50:13.650 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4443999999999998e-05 2024-09-16T20:50:13.650 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.651 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:13.651 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.651 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.651 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.651 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3579 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:13.651 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:50:13.651 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 84.128831637999994, 2024-09-16T20:50:13.651 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054295151, 2024-09-16T20:50:13.651 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:13.651 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:13.651 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:13.651 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:13.652 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:13.652 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:13.652 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3579 2024-09-16T20:50:13.652 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.652 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:13.652 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.652 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:13.652 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:50:13.652 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.652 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.652 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.652 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:13.653 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:50:13.653 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.653 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.653 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.653 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:13.653 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:50:13.653 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3300000000000001e-07 2024-09-16T20:50:13.653 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.653 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.653 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:13.653 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481381+0000", 2024-09-16T20:50:13.653 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.649e-06 2024-09-16T20:50:13.653 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.654 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.654 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:13.654 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481387+0000", 2024-09-16T20:50:13.654 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4519999999999996e-06 2024-09-16T20:50:13.654 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.654 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.654 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:13.654 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481390+0000", 2024-09-16T20:50:13.654 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0170000000000001e-06 2024-09-16T20:50:13.654 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.654 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.654 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:13.654 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530756+0000", 2024-09-16T20:50:13.655 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049365882999999999 2024-09-16T20:50:13.655 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.655 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.655 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:13.655 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530845+0000", 2024-09-16T20:50:13.655 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9197000000000007e-05 2024-09-16T20:50:13.655 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.655 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.655 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:13.655 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531191+0000", 2024-09-16T20:50:13.655 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00034562500000000001 2024-09-16T20:50:13.655 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.655 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.656 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:13.656 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534078+0000", 2024-09-16T20:50:13.656 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0028878179999999999 2024-09-16T20:50:13.656 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.656 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.656 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:13.656 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535589+0000", 2024-09-16T20:50:13.656 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00151085 2024-09-16T20:50:13.656 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.656 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.656 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:13.656 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535639+0000", 2024-09-16T20:50:13.656 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9697000000000002e-05 2024-09-16T20:50:13.656 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.657 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.657 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:13.657 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535675+0000", 2024-09-16T20:50:13.657 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5729999999999998e-05 2024-09-16T20:50:13.657 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.657 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:13.657 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.657 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.657 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.657 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3580 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:13.657 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:50:13.657 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 84.128818006000003, 2024-09-16T20:50:13.657 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054397361999999998, 2024-09-16T20:50:13.658 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:13.658 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:13.658 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:13.658 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:13.658 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:13.658 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:13.658 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3580 2024-09-16T20:50:13.658 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.658 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:13.658 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.658 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:13.658 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:50:13.658 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.658 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.659 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.659 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:13.659 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:50:13.659 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.659 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.659 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.659 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:13.659 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:50:13.659 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2600000000000001e-07 2024-09-16T20:50:13.659 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.659 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.659 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:13.659 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481395+0000", 2024-09-16T20:50:13.659 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.544e-06 2024-09-16T20:50:13.660 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.660 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.660 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:13.660 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481404+0000", 2024-09-16T20:50:13.660 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8850000000000005e-06 2024-09-16T20:50:13.660 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.660 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.660 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:13.660 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481407+0000", 2024-09-16T20:50:13.660 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1540000000000002e-06 2024-09-16T20:50:13.660 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.660 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.660 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:13.660 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531823+0000", 2024-09-16T20:50:13.661 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050415637999999999 2024-09-16T20:50:13.661 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.661 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.661 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:13.661 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531895+0000", 2024-09-16T20:50:13.661 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2179000000000001e-05 2024-09-16T20:50:13.661 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.661 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.661 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:13.661 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532236+0000", 2024-09-16T20:50:13.661 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00034073799999999999 2024-09-16T20:50:13.661 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.661 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.662 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:13.662 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534083+0000", 2024-09-16T20:50:13.662 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001847298 2024-09-16T20:50:13.662 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.662 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.662 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:13.662 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535714+0000", 2024-09-16T20:50:13.662 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0016316810000000001 2024-09-16T20:50:13.662 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.662 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.662 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:13.662 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535759+0000", 2024-09-16T20:50:13.662 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4478999999999999e-05 2024-09-16T20:50:13.663 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.663 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.663 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:13.663 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535791+0000", 2024-09-16T20:50:13.663 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.154e-05 2024-09-16T20:50:13.663 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.663 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:13.663 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.663 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.663 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.663 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3581 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:13.663 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:50:13.663 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 84.128796008999998, 2024-09-16T20:50:13.664 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.057094403000000002, 2024-09-16T20:50:13.664 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:13.664 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:13.664 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:13.664 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:13.664 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:13.664 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:13.664 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3581 2024-09-16T20:50:13.664 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.664 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:13.664 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.664 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:13.664 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:50:13.665 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.665 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.665 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.665 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:13.665 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:50:13.665 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.665 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.665 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.665 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:13.665 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:50:13.665 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8799999999999998e-07 2024-09-16T20:50:13.665 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.665 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.665 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:13.666 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481418+0000", 2024-09-16T20:50:13.666 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1859999999999999e-06 2024-09-16T20:50:13.666 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.666 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.666 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:13.666 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481423+0000", 2024-09-16T20:50:13.666 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8279999999999999e-06 2024-09-16T20:50:13.666 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.666 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.666 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:13.666 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481426+0000", 2024-09-16T20:50:13.666 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0110000000000001e-06 2024-09-16T20:50:13.666 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.666 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.667 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:13.667 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532631+0000", 2024-09-16T20:50:13.667 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051204514999999999 2024-09-16T20:50:13.667 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.667 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.667 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:13.667 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.533381+0000", 2024-09-16T20:50:13.667 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00075029000000000003 2024-09-16T20:50:13.667 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.667 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.667 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:13.667 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.533739+0000", 2024-09-16T20:50:13.667 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00035773100000000002 2024-09-16T20:50:13.667 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.667 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.668 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:13.668 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535123+0000", 2024-09-16T20:50:13.668 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0013836160000000001 2024-09-16T20:50:13.668 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.668 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.668 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:13.668 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538413+0000", 2024-09-16T20:50:13.668 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0032902510000000001 2024-09-16T20:50:13.668 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.668 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.668 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:13.668 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538470+0000", 2024-09-16T20:50:13.669 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6697999999999998e-05 2024-09-16T20:50:13.669 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.669 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.669 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:13.669 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538510+0000", 2024-09-16T20:50:13.669 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9988999999999999e-05 2024-09-16T20:50:13.669 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.669 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:13.669 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.669 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.669 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.669 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:12732 3.d 3:b752444a:::rbd_header.110e747f0779:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-16T20:50:13.669 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:50:13.669 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 68.841723431000005, 2024-09-16T20:50:13.670 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.065302467000000003, 2024-09-16T20:50:13.670 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:13.670 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:13.670 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:13.670 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:13.670 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:50:13.670 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:50:13.670 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 12732 2024-09-16T20:50:13.670 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.670 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:13.670 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.670 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:13.670 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:50:13.670 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.671 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.671 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.671 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:13.671 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:50:13.671 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.671 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.671 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.671 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:13.671 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768490+0000", 2024-09-16T20:50:13.671 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2859999999999998e-06 2024-09-16T20:50:13.671 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.671 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.671 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:13.671 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768501+0000", 2024-09-16T20:50:13.672 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1226999999999999e-05 2024-09-16T20:50:13.672 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.672 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.672 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:13.672 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768517+0000", 2024-09-16T20:50:13.672 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.592e-05 2024-09-16T20:50:13.672 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.672 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.672 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:13.672 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768528+0000", 2024-09-16T20:50:13.672 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0689999999999999e-05 2024-09-16T20:50:13.672 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.672 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.673 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:13.673 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768584+0000", 2024-09-16T20:50:13.673 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5889e-05 2024-09-16T20:50:13.673 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.673 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.673 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:13.673 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768720+0000", 2024-09-16T20:50:13.673 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000136718 2024-09-16T20:50:13.673 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.673 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.673 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:13.673 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.769066+0000", 2024-09-16T20:50:13.673 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000346013 2024-09-16T20:50:13.674 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.674 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.674 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:13.674 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833659+0000", 2024-09-16T20:50:13.674 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.064592822999999994 2024-09-16T20:50:13.674 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.674 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.674 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:13.674 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833719+0000", 2024-09-16T20:50:13.674 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9441000000000002e-05 2024-09-16T20:50:13.674 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.674 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.674 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:13.674 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833758+0000", 2024-09-16T20:50:13.674 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9122000000000003e-05 2024-09-16T20:50:13.675 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.675 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.675 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:13.675 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833790+0000", 2024-09-16T20:50:13.675 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2338000000000001e-05 2024-09-16T20:50:13.675 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.675 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:13.675 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.675 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.675 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.675 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.5089.0:12732 3.d e73/23 ondisk, result = 0)", 2024-09-16T20:50:13.675 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:50:13.675 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 68.840121260999993, 2024-09-16T20:50:13.675 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.063706456999999994, 2024-09-16T20:50:13.675 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:13.676 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:13.676 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:13.676 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:13.676 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.676 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:13.676 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:50:13.676 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.676 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.676 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.676 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:13.676 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:50:13.676 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.677 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.677 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.677 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:13.677 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770091+0000", 2024-09-16T20:50:13.677 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.53e-06 2024-09-16T20:50:13.677 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.677 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.677 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:13.677 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770104+0000", 2024-09-16T20:50:13.677 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2203e-05 2024-09-16T20:50:13.677 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.677 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.677 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:13.677 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833637+0000", 2024-09-16T20:50:13.678 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.063533201999999997 2024-09-16T20:50:13.678 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.678 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.678 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:13.678 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833655+0000", 2024-09-16T20:50:13.678 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8369e-05 2024-09-16T20:50:13.678 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.678 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.678 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:13.678 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833707+0000", 2024-09-16T20:50:13.678 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2061999999999997e-05 2024-09-16T20:50:13.679 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.679 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.679 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:13.679 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833715+0000", 2024-09-16T20:50:13.679 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5179999999999998e-06 2024-09-16T20:50:13.679 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.679 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.679 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:13.679 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833796+0000", 2024-09-16T20:50:13.679 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1573e-05 2024-09-16T20:50:13.679 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.679 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:13.679 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.680 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.680 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.680 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3281 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 220160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:13.680 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041070+0000", 2024-09-16T20:50:13.680 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.569140791999999, 2024-09-16T20:50:13.680 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051593550000000002, 2024-09-16T20:50:13.680 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:13.680 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:13.680 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:13.680 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:13.680 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:13.680 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:13.680 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3281 2024-09-16T20:50:13.680 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.681 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:13.681 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.681 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:13.681 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041070+0000", 2024-09-16T20:50:13.681 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.681 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.681 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.681 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:13.681 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041070+0000", 2024-09-16T20:50:13.681 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.681 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.681 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.681 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:13.682 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041071+0000", 2024-09-16T20:50:13.682 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1899999999999999e-07 2024-09-16T20:50:13.682 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.682 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.682 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:13.682 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041072+0000", 2024-09-16T20:50:13.682 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.248e-06 2024-09-16T20:50:13.682 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.682 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.682 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:13.682 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041075+0000", 2024-09-16T20:50:13.682 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0029999999999999e-06 2024-09-16T20:50:13.682 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.682 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.683 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:13.683 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041077+0000", 2024-09-16T20:50:13.683 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.621e-06 2024-09-16T20:50:13.683 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.683 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.683 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:13.683 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.092431+0000", 2024-09-16T20:50:13.683 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051353801999999997 2024-09-16T20:50:13.683 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.683 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.683 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:13.683 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.092512+0000", 2024-09-16T20:50:13.683 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1106000000000006e-05 2024-09-16T20:50:13.683 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.684 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.684 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:13.684 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.092664+0000", 2024-09-16T20:50:13.684 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00015155100000000001 2024-09-16T20:50:13.684 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.684 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:13.684 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.684 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.684 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.684 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3282 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 225280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:13.684 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041081+0000", 2024-09-16T20:50:13.684 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.569130514000001, 2024-09-16T20:50:13.684 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051731948, 2024-09-16T20:50:13.685 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:13.685 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:13.685 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:13.685 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:13.685 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:13.685 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:13.685 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3282 2024-09-16T20:50:13.685 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.685 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:13.685 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.685 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:13.685 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041081+0000", 2024-09-16T20:50:13.685 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.685 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.686 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.686 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:13.686 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041081+0000", 2024-09-16T20:50:13.686 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.686 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.686 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.686 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:13.686 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041081+0000", 2024-09-16T20:50:13.686 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3099999999999999e-07 2024-09-16T20:50:13.686 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.686 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.686 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:13.686 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041082+0000", 2024-09-16T20:50:13.686 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1289999999999999e-06 2024-09-16T20:50:13.686 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.687 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.687 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:13.687 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041085+0000", 2024-09-16T20:50:13.687 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2320000000000001e-06 2024-09-16T20:50:13.687 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.687 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.687 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:13.687 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041090+0000", 2024-09-16T20:50:13.687 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3730000000000003e-06 2024-09-16T20:50:13.687 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.687 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.687 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:13.687 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.092678+0000", 2024-09-16T20:50:13.687 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051588235000000003 2024-09-16T20:50:13.687 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.687 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.688 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:13.688 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.092717+0000", 2024-09-16T20:50:13.688 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9057999999999998e-05 2024-09-16T20:50:13.688 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.688 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.688 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:13.688 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.092813+0000", 2024-09-16T20:50:13.688 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.569e-05 2024-09-16T20:50:13.688 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.688 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:13.688 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.688 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.688 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.689 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3283 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 230400~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:13.689 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041102+0000", 2024-09-16T20:50:13.689 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.569108854, 2024-09-16T20:50:13.689 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051974723, 2024-09-16T20:50:13.689 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:13.689 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:13.689 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:13.689 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:13.689 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:13.689 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:13.689 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3283 2024-09-16T20:50:13.689 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.689 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:13.689 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.690 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:13.690 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041102+0000", 2024-09-16T20:50:13.690 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.690 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.690 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.690 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:13.690 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041102+0000", 2024-09-16T20:50:13.690 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.690 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.690 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.690 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:13.690 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041103+0000", 2024-09-16T20:50:13.690 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2399999999999999e-07 2024-09-16T20:50:13.691 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.691 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.691 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:13.691 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041110+0000", 2024-09-16T20:50:13.691 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4290000000000004e-06 2024-09-16T20:50:13.691 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.691 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.691 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:13.691 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041118+0000", 2024-09-16T20:50:13.691 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3849999999999996e-06 2024-09-16T20:50:13.691 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.691 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.691 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:13.691 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041123+0000", 2024-09-16T20:50:13.692 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1410000000000004e-06 2024-09-16T20:50:13.692 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.692 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.692 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:13.692 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.092845+0000", 2024-09-16T20:50:13.692 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051722207999999999 2024-09-16T20:50:13.692 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.692 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.692 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:13.692 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.092926+0000", 2024-09-16T20:50:13.692 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0727000000000001e-05 2024-09-16T20:50:13.692 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.692 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.693 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:13.693 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093077+0000", 2024-09-16T20:50:13.693 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000151409 2024-09-16T20:50:13.693 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.693 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:13.693 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.693 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.693 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.693 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3284 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 235520~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:13.693 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041129+0000", 2024-09-16T20:50:13.693 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.569082522999999, 2024-09-16T20:50:13.693 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052120760000000002, 2024-09-16T20:50:13.693 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:13.693 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:13.694 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:13.694 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:13.694 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:13.694 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:13.694 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3284 2024-09-16T20:50:13.694 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.694 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:13.694 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.694 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:13.694 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041129+0000", 2024-09-16T20:50:13.694 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.694 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.694 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.695 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:13.695 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041129+0000", 2024-09-16T20:50:13.695 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.695 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.695 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.695 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:13.695 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041129+0000", 2024-09-16T20:50:13.695 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3700000000000001e-07 2024-09-16T20:50:13.695 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.696 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.696 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:13.696 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041130+0000", 2024-09-16T20:50:13.696 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4050000000000001e-06 2024-09-16T20:50:13.696 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.696 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.696 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:13.696 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041134+0000", 2024-09-16T20:50:13.696 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4290000000000001e-06 2024-09-16T20:50:13.696 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.696 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.696 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:13.696 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041136+0000", 2024-09-16T20:50:13.697 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.678e-06 2024-09-16T20:50:13.697 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.697 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.697 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:13.697 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093109+0000", 2024-09-16T20:50:13.697 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051972879999999999 2024-09-16T20:50:13.697 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.697 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.697 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:13.697 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093149+0000", 2024-09-16T20:50:13.697 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0040000000000003e-05 2024-09-16T20:50:13.697 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.698 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.698 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:13.698 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093249+0000", 2024-09-16T20:50:13.698 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.9990999999999999e-05 2024-09-16T20:50:13.698 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.698 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:13.698 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.698 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.698 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.698 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3285 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 240640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:13.698 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041140+0000", 2024-09-16T20:50:13.698 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.569071482999998, 2024-09-16T20:50:13.698 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052375646999999997, 2024-09-16T20:50:13.699 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:13.699 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:13.699 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:13.699 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:13.699 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:13.699 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:13.699 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3285 2024-09-16T20:50:13.699 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.699 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:13.699 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.699 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:13.699 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041140+0000", 2024-09-16T20:50:13.699 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.700 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.700 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.700 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:13.700 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041140+0000", 2024-09-16T20:50:13.700 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.700 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.700 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.700 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:13.700 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041140+0000", 2024-09-16T20:50:13.700 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2399999999999999e-07 2024-09-16T20:50:13.700 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.700 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.701 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:13.701 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041141+0000", 2024-09-16T20:50:13.701 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2759999999999999e-06 2024-09-16T20:50:13.701 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.701 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.701 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:13.701 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041144+0000", 2024-09-16T20:50:13.701 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0280000000000001e-06 2024-09-16T20:50:13.701 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.701 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.701 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:13.701 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041147+0000", 2024-09-16T20:50:13.701 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5900000000000002e-06 2024-09-16T20:50:13.701 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.702 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.702 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:13.702 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093284+0000", 2024-09-16T20:50:13.702 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052137645000000003 2024-09-16T20:50:13.702 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.702 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.702 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:13.702 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093365+0000", 2024-09-16T20:50:13.702 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0446999999999997e-05 2024-09-16T20:50:13.702 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.702 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.702 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:13.702 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093515+0000", 2024-09-16T20:50:13.703 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00015043700000000001 2024-09-16T20:50:13.703 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.703 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:13.703 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.703 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.703 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.703 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3286 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 245760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:13.703 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:50:13.703 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.569059324000001, 2024-09-16T20:50:13.703 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052523564000000002, 2024-09-16T20:50:13.703 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:13.703 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:13.703 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:13.703 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:13.704 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:13.704 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:13.704 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3286 2024-09-16T20:50:13.704 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.704 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:13.704 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.704 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:13.704 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:50:13.704 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.704 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.704 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.704 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:13.705 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:50:13.705 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.705 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.705 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.705 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:13.705 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:50:13.705 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3900000000000001e-07 2024-09-16T20:50:13.705 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.705 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.705 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:13.705 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041153+0000", 2024-09-16T20:50:13.705 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.119e-06 2024-09-16T20:50:13.705 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.706 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.706 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:13.706 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041156+0000", 2024-09-16T20:50:13.706 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0000000000000001e-06 2024-09-16T20:50:13.706 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.706 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.706 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:13.706 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041159+0000", 2024-09-16T20:50:13.706 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5859999999999999e-06 2024-09-16T20:50:13.706 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.706 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.706 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:13.707 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093529+0000", 2024-09-16T20:50:13.707 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052370591000000001 2024-09-16T20:50:13.707 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.707 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.707 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:13.707 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093570+0000", 2024-09-16T20:50:13.707 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0161999999999999e-05 2024-09-16T20:50:13.707 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.707 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.707 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:13.707 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093675+0000", 2024-09-16T20:50:13.707 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010586699999999999 2024-09-16T20:50:13.707 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.708 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:13.708 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.708 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.708 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.708 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3287 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 250880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:13.708 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:50:13.708 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.569049275000001, 2024-09-16T20:50:13.708 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052781203999999998, 2024-09-16T20:50:13.708 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:13.708 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:13.708 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:13.708 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:13.708 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:13.709 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:13.709 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3287 2024-09-16T20:50:13.709 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.709 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:13.709 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.709 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:13.709 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:50:13.709 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.709 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.709 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.709 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:13.709 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:50:13.709 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.710 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.710 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.710 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:13.710 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:50:13.710 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1e-07 2024-09-16T20:50:13.710 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.710 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.710 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:13.710 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041163+0000", 2024-09-16T20:50:13.710 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0690000000000001e-06 2024-09-16T20:50:13.710 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.710 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.710 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:13.710 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041166+0000", 2024-09-16T20:50:13.711 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.982e-06 2024-09-16T20:50:13.711 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.711 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.711 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:13.711 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041169+0000", 2024-09-16T20:50:13.711 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.593e-06 2024-09-16T20:50:13.711 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.711 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.711 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:13.711 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093709+0000", 2024-09-16T20:50:13.711 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052540492000000001 2024-09-16T20:50:13.711 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.711 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.711 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:13.711 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093792+0000", 2024-09-16T20:50:13.712 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2436999999999997e-05 2024-09-16T20:50:13.712 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.712 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.712 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:13.712 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093943+0000", 2024-09-16T20:50:13.712 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000151421 2024-09-16T20:50:13.712 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.712 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:13.712 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.712 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.712 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.712 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3288 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 256000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:13.712 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:50:13.713 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.569039402000001, 2024-09-16T20:50:13.713 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052928269, 2024-09-16T20:50:13.713 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:13.713 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:13.713 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:13.713 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:13.713 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:13.713 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:13.713 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3288 2024-09-16T20:50:13.713 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.713 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:13.713 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.713 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:13.714 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:50:13.714 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.714 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.714 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.714 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:13.714 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:50:13.714 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.714 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.714 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.714 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:13.714 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:50:13.714 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2700000000000001e-07 2024-09-16T20:50:13.714 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.714 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.714 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:13.714 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041173+0000", 2024-09-16T20:50:13.715 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.113e-06 2024-09-16T20:50:13.715 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.715 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.715 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:13.715 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041176+0000", 2024-09-16T20:50:13.715 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9799999999999998e-06 2024-09-16T20:50:13.715 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.715 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.715 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:13.715 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041179+0000", 2024-09-16T20:50:13.715 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.667e-06 2024-09-16T20:50:13.715 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.715 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.715 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:13.716 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093957+0000", 2024-09-16T20:50:13.716 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052778703000000003 2024-09-16T20:50:13.716 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.716 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.716 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:13.716 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093999+0000", 2024-09-16T20:50:13.716 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1300000000000001e-05 2024-09-16T20:50:13.716 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.716 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.716 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:13.716 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094100+0000", 2024-09-16T20:50:13.716 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101279 2024-09-16T20:50:13.716 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.716 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:13.717 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.717 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.717 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.717 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3289 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 261120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:13.717 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:50:13.717 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.569029405999999, 2024-09-16T20:50:13.717 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053182976, 2024-09-16T20:50:13.717 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:13.717 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:13.717 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:13.717 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:13.717 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:13.717 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:13.718 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3289 2024-09-16T20:50:13.718 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.718 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:13.718 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.718 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:13.718 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:50:13.718 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.718 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.718 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.718 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:13.718 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:50:13.718 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:13.718 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.719 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.719 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:13.719 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:50:13.719 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2999999999999999e-07 2024-09-16T20:50:13.719 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.719 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.719 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:13.719 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041183+0000", 2024-09-16T20:50:13.719 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0720000000000001e-06 2024-09-16T20:50:13.719 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.719 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.719 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:13.719 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041186+0000", 2024-09-16T20:50:13.720 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9869999999999999e-06 2024-09-16T20:50:13.720 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.720 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.720 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:13.720 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041189+0000", 2024-09-16T20:50:13.720 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.587e-06 2024-09-16T20:50:13.720 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.720 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.720 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:13.720 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094136+0000", 2024-09-16T20:50:13.720 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052947270999999997 2024-09-16T20:50:13.720 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.720 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.720 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:13.721 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094213+0000", 2024-09-16T20:50:13.721 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7552000000000006e-05 2024-09-16T20:50:13.721 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:13.721 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:13.721 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:13.721 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094365+0000", 2024-09-16T20:50:13.721 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00015127700000000001 2024-09-16T20:50:13.721 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.721 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:13.721 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.721 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:13.721 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:13.721 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:13.723 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:50:13.723 INFO:teuthology.orchestra.run.smithi029.stderr:set 3.1b pg_upmap mapping to [0,1] 2024-09-16T20:50:13.724 INFO:teuthology.orchestra.run.smithi029.stderr:noscrub is unset 2024-09-16T20:50:13.725 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:13.721+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:13.776 INFO:teuthology.orchestra.run.smithi139.stdout:truncating image image_client.0-clone40 from 0xc826e00 (overlap 0x3f6a00) to 0x65600 2024-09-16T20:50:13.824 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:13.822+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:13.880 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:13.880 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:50:13.880 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:50:13.880 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:13.889 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:50:13.933 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:13.932+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:13.997 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:13.997 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:50:13.997 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:50:13.997 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:50:13.997 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:14.006 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:50:14.047 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:14.046+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:14.127 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:14.126+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:14.156 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:14.156 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:50:14.156 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:50:14.156 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:50:14.156 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.156 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3714 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1480704~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:14.157 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:50:14.157 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 111.515621161, 2024-09-16T20:50:14.157 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050152552000000003, 2024-09-16T20:50:14.157 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:14.157 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:14.157 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:14.157 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:14.157 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:14.157 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:14.157 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3714 2024-09-16T20:50:14.157 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.157 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:14.157 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.158 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:14.158 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:50:14.158 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:14.158 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.158 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.158 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:14.158 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:50:14.158 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:14.158 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.158 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.158 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:14.158 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:50:14.159 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0499999999999999e-07 2024-09-16T20:50:14.159 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.159 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.159 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:14.159 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639007+0000", 2024-09-16T20:50:14.159 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5340000000000001e-06 2024-09-16T20:50:14.159 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.159 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.159 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:14.159 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639013+0000", 2024-09-16T20:50:14.159 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3260000000000003e-06 2024-09-16T20:50:14.159 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.159 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.159 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:14.160 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639018+0000", 2024-09-16T20:50:14.160 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2850000000000002e-06 2024-09-16T20:50:14.160 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.160 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.160 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:14.160 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689026+0000", 2024-09-16T20:50:14.160 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.05000798 2024-09-16T20:50:14.160 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.160 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.160 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:14.160 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689077+0000", 2024-09-16T20:50:14.160 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1063999999999998e-05 2024-09-16T20:50:14.161 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.161 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.161 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:14.161 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689158+0000", 2024-09-16T20:50:14.161 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1057999999999995e-05 2024-09-16T20:50:14.161 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:14.161 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:14.161 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:14.161 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.161 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.161 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3716 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1488896~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:14.161 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:50:14.161 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 111.515603351, 2024-09-16T20:50:14.161 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050280142, 2024-09-16T20:50:14.162 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:14.162 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:14.162 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:14.162 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:14.162 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:14.162 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:14.162 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3716 2024-09-16T20:50:14.162 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.162 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:14.162 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.162 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:14.162 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:50:14.162 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:14.162 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.163 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.163 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:14.163 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:50:14.163 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:14.163 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.163 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.163 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:14.163 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639024+0000", 2024-09-16T20:50:14.163 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9999999999999997e-07 2024-09-16T20:50:14.163 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.163 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.163 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:14.163 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639026+0000", 2024-09-16T20:50:14.163 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.781e-06 2024-09-16T20:50:14.164 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.164 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.164 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:14.164 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639032+0000", 2024-09-16T20:50:14.164 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7780000000000004e-06 2024-09-16T20:50:14.164 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.164 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.164 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:14.164 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639036+0000", 2024-09-16T20:50:14.164 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6909999999999996e-06 2024-09-16T20:50:14.164 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.164 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.165 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:14.165 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689191+0000", 2024-09-16T20:50:14.165 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050155284000000001 2024-09-16T20:50:14.165 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.165 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.165 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:14.165 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689242+0000", 2024-09-16T20:50:14.165 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0662999999999999e-05 2024-09-16T20:50:14.165 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.165 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.165 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:14.165 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689304+0000", 2024-09-16T20:50:14.165 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1345000000000003e-05 2024-09-16T20:50:14.165 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:14.166 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:14.166 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:14.166 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.166 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.166 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3719 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1497088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:14.166 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:50:14.166 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 111.51557607300001, 2024-09-16T20:50:14.166 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050399715999999997, 2024-09-16T20:50:14.166 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:14.166 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:14.166 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:14.166 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:14.166 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:14.166 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:14.167 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3719 2024-09-16T20:50:14.167 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.167 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:14.167 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.167 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:14.167 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:50:14.167 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:14.167 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.167 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.167 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:14.167 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:50:14.167 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:14.167 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.168 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.168 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:14.168 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:50:14.168 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9700000000000002e-07 2024-09-16T20:50:14.168 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.168 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.168 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:14.168 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639053+0000", 2024-09-16T20:50:14.168 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2900000000000001e-06 2024-09-16T20:50:14.168 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.168 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.168 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:14.168 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639060+0000", 2024-09-16T20:50:14.168 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7750000000000004e-06 2024-09-16T20:50:14.169 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.169 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.169 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:14.169 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639066+0000", 2024-09-16T20:50:14.169 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0639999999999999e-06 2024-09-16T20:50:14.169 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.169 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.169 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:14.169 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689337+0000", 2024-09-16T20:50:14.169 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050270784999999998 2024-09-16T20:50:14.169 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.169 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.169 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:14.169 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689392+0000", 2024-09-16T20:50:14.170 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4920999999999998e-05 2024-09-16T20:50:14.170 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.170 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.170 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:14.170 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689450+0000", 2024-09-16T20:50:14.170 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8483999999999998e-05 2024-09-16T20:50:14.170 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:14.170 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:14.170 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:14.170 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.170 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.170 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3721 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1505280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:14.170 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:50:14.170 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 111.515552941, 2024-09-16T20:50:14.171 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050513608000000002, 2024-09-16T20:50:14.171 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:14.171 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:14.171 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:14.171 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:14.171 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:14.171 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:14.171 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3721 2024-09-16T20:50:14.171 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.171 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:14.171 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.171 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:14.171 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:50:14.172 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:14.172 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.172 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.172 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:14.172 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:50:14.172 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:14.172 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.172 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.172 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:14.172 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:50:14.172 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7e-07 2024-09-16T20:50:14.172 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.172 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.173 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:14.173 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639076+0000", 2024-09-16T20:50:14.173 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8899999999999999e-06 2024-09-16T20:50:14.173 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.173 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.173 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:14.173 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639083+0000", 2024-09-16T20:50:14.173 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1210000000000001e-06 2024-09-16T20:50:14.173 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.173 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.173 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:14.173 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639089+0000", 2024-09-16T20:50:14.173 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.643e-06 2024-09-16T20:50:14.173 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.174 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.174 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:14.174 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689474+0000", 2024-09-16T20:50:14.174 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050385679000000003 2024-09-16T20:50:14.174 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.174 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.174 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:14.174 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689533+0000", 2024-09-16T20:50:14.174 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8303000000000001e-05 2024-09-16T20:50:14.174 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.174 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.174 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:14.174 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689587+0000", 2024-09-16T20:50:14.174 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4601999999999997e-05 2024-09-16T20:50:14.175 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:14.175 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:14.175 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:14.175 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.175 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.175 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3723 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1513472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:14.175 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:50:14.175 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 111.515522137, 2024-09-16T20:50:14.175 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050699691999999998, 2024-09-16T20:50:14.175 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:14.175 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:14.175 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:14.176 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:14.176 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:14.176 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:14.176 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3723 2024-09-16T20:50:14.176 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.176 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:14.176 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.176 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:14.176 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:50:14.176 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:14.176 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.176 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.176 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:14.176 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:50:14.177 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:14.177 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.177 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.177 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:14.177 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:50:14.177 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3399999999999999e-07 2024-09-16T20:50:14.177 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.177 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.177 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:14.177 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639107+0000", 2024-09-16T20:50:14.177 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3860000000000001e-06 2024-09-16T20:50:14.177 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.177 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.177 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:14.178 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639114+0000", 2024-09-16T20:50:14.178 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9800000000000001e-06 2024-09-16T20:50:14.178 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.178 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.178 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:14.178 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639129+0000", 2024-09-16T20:50:14.178 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4887e-05 2024-09-16T20:50:14.178 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.178 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.178 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:14.178 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689625+0000", 2024-09-16T20:50:14.178 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050495848000000003 2024-09-16T20:50:14.178 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.178 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.179 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:14.179 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689720+0000", 2024-09-16T20:50:14.179 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4821000000000006e-05 2024-09-16T20:50:14.179 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.179 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.179 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:14.179 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689804+0000", 2024-09-16T20:50:14.179 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4235999999999998e-05 2024-09-16T20:50:14.179 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:14.179 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:14.179 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:14.179 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.179 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.179 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3725 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1521664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:14.180 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:50:14.180 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 111.515488541, 2024-09-16T20:50:14.180 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050797660000000001, 2024-09-16T20:50:14.180 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:14.180 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:14.180 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:14.180 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:14.180 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:14.180 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:14.180 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3725 2024-09-16T20:50:14.180 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.180 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:14.180 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.181 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:14.181 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:50:14.181 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:14.181 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.181 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.181 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:14.181 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:50:14.181 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:14.181 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.181 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.181 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:14.181 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639139+0000", 2024-09-16T20:50:14.181 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8999999999999997e-07 2024-09-16T20:50:14.182 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.182 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.182 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:14.182 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639141+0000", 2024-09-16T20:50:14.182 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.182e-06 2024-09-16T20:50:14.182 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.182 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.182 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:14.182 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639154+0000", 2024-09-16T20:50:14.182 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3626e-05 2024-09-16T20:50:14.182 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.182 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.182 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:14.182 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639160+0000", 2024-09-16T20:50:14.183 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6810000000000003e-06 2024-09-16T20:50:14.183 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.183 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.183 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:14.183 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689831+0000", 2024-09-16T20:50:14.183 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050670907000000001 2024-09-16T20:50:14.183 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.183 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.183 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:14.183 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689882+0000", 2024-09-16T20:50:14.183 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0445999999999998e-05 2024-09-16T20:50:14.183 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.183 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.183 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:14.183 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689936+0000", 2024-09-16T20:50:14.184 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4327999999999999e-05 2024-09-16T20:50:14.184 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:14.184 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:14.184 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:14.184 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.184 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.184 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3727 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1529856~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:14.184 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:50:14.184 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 111.515452964, 2024-09-16T20:50:14.184 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051016922999999999, 2024-09-16T20:50:14.184 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:14.184 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:14.184 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:14.185 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:14.185 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:14.185 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:14.185 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3727 2024-09-16T20:50:14.185 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.185 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:14.185 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.185 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:14.185 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:50:14.186 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:14.186 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.186 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.186 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:14.186 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:50:14.186 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:14.186 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.186 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.186 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:14.186 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:50:14.187 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9999999999999998e-07 2024-09-16T20:50:14.187 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.187 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.187 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:14.187 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639183+0000", 2024-09-16T20:50:14.187 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2399999999999996e-06 2024-09-16T20:50:14.187 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.187 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.187 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:14.188 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639195+0000", 2024-09-16T20:50:14.188 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1113e-05 2024-09-16T20:50:14.188 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.188 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.188 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:14.188 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639206+0000", 2024-09-16T20:50:14.188 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1503e-05 2024-09-16T20:50:14.188 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.188 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.188 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:14.188 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689969+0000", 2024-09-16T20:50:14.189 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050762676 2024-09-16T20:50:14.189 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.189 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.189 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:14.189 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690071+0000", 2024-09-16T20:50:14.189 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102163 2024-09-16T20:50:14.189 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.189 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.189 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:14.189 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690191+0000", 2024-09-16T20:50:14.189 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000119728 2024-09-16T20:50:14.189 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:14.189 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:14.189 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:14.190 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.190 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.190 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3730 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1538048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:14.190 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:50:14.190 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 111.51541140099999, 2024-09-16T20:50:14.190 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051131467999999999, 2024-09-16T20:50:14.190 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:14.190 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:14.190 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:14.190 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:14.190 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:14.190 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:14.190 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3730 2024-09-16T20:50:14.190 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.190 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:14.191 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.191 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:14.191 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:50:14.191 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:14.191 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.191 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.191 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:14.191 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:50:14.191 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:14.191 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.191 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.191 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:14.191 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639216+0000", 2024-09-16T20:50:14.192 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.39e-07 2024-09-16T20:50:14.192 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.192 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.192 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:14.192 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639218+0000", 2024-09-16T20:50:14.192 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.537e-06 2024-09-16T20:50:14.192 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.192 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.192 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:14.192 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639229+0000", 2024-09-16T20:50:14.192 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1122e-05 2024-09-16T20:50:14.193 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.193 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.193 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:14.193 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639237+0000", 2024-09-16T20:50:14.193 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8509999999999999e-06 2024-09-16T20:50:14.193 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.193 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.193 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:14.193 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690227+0000", 2024-09-16T20:50:14.193 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050990147 2024-09-16T20:50:14.193 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.193 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.193 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:14.193 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690278+0000", 2024-09-16T20:50:14.194 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0223999999999999e-05 2024-09-16T20:50:14.194 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.194 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.194 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:14.194 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690347+0000", 2024-09-16T20:50:14.194 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9147999999999996e-05 2024-09-16T20:50:14.194 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:14.194 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:14.194 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:14.194 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.194 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.194 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3731 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1546240~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:14.194 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:50:14.194 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 111.515380693, 2024-09-16T20:50:14.194 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051276744999999999, 2024-09-16T20:50:14.195 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:14.195 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:14.195 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:14.195 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:14.195 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:14.195 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:14.195 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3731 2024-09-16T20:50:14.195 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.195 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:14.195 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.195 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:14.195 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:50:14.195 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:14.196 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.196 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.196 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:14.196 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:50:14.196 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:14.196 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.196 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.196 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:14.196 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:50:14.196 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9900000000000001e-07 2024-09-16T20:50:14.196 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.197 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.197 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:14.197 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639249+0000", 2024-09-16T20:50:14.197 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3630000000000001e-06 2024-09-16T20:50:14.197 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.197 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.197 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:14.197 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639255+0000", 2024-09-16T20:50:14.197 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4960000000000001e-06 2024-09-16T20:50:14.197 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.198 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.198 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:14.198 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639261+0000", 2024-09-16T20:50:14.198 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2839999999999997e-06 2024-09-16T20:50:14.198 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.198 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.198 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:14.198 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690375+0000", 2024-09-16T20:50:14.198 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051114684000000001 2024-09-16T20:50:14.198 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.198 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.198 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:14.198 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690425+0000", 2024-09-16T20:50:14.199 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9379999999999998e-05 2024-09-16T20:50:14.199 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.199 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.199 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:14.199 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690523+0000", 2024-09-16T20:50:14.199 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8139000000000003e-05 2024-09-16T20:50:14.199 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:14.199 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:14.199 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:14.199 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.199 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.199 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3732 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1554432~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:14.199 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:50:14.200 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 111.515334911, 2024-09-16T20:50:14.200 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051380096, 2024-09-16T20:50:14.200 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:14.200 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:14.200 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:14.200 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:14.200 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:14.200 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:14.200 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3732 2024-09-16T20:50:14.200 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.200 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:14.200 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.200 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:14.200 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:50:14.201 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:14.201 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.201 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.201 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:14.201 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:50:14.201 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:14.201 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.201 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.201 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:14.201 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:50:14.202 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6100000000000001e-07 2024-09-16T20:50:14.202 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.202 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.202 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:14.202 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639295+0000", 2024-09-16T20:50:14.202 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.447e-06 2024-09-16T20:50:14.202 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.202 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.202 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:14.203 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639302+0000", 2024-09-16T20:50:14.203 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0570000000000004e-06 2024-09-16T20:50:14.203 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.203 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.203 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:14.203 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639307+0000", 2024-09-16T20:50:14.203 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1429999999999997e-06 2024-09-16T20:50:14.203 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.203 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.203 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:14.204 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690555+0000", 2024-09-16T20:50:14.204 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051247963000000001 2024-09-16T20:50:14.204 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.204 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.204 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:14.204 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690602+0000", 2024-09-16T20:50:14.204 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7602000000000003e-05 2024-09-16T20:50:14.205 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.205 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.205 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:14.205 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690672+0000", 2024-09-16T20:50:14.205 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9423000000000003e-05 2024-09-16T20:50:14.205 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:14.205 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:14.205 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:14.205 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.206 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.206 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3734 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1562624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:14.206 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:50:14.206 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 111.515313261, 2024-09-16T20:50:14.206 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051490610999999999, 2024-09-16T20:50:14.206 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:14.206 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:14.206 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:14.206 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:14.206 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:14.206 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:14.206 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3734 2024-09-16T20:50:14.207 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.207 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:14.207 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.207 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:14.207 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:50:14.207 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:14.207 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.207 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.207 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:14.207 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:50:14.207 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:14.207 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.207 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.207 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:14.208 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639314+0000", 2024-09-16T20:50:14.208 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4400000000000001e-07 2024-09-16T20:50:14.208 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.208 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.208 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:14.208 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639316+0000", 2024-09-16T20:50:14.208 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1459999999999999e-06 2024-09-16T20:50:14.208 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.208 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.208 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:14.208 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639322+0000", 2024-09-16T20:50:14.208 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7610000000000004e-06 2024-09-16T20:50:14.209 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.209 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.209 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:14.209 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639327+0000", 2024-09-16T20:50:14.209 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.186e-06 2024-09-16T20:50:14.209 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.209 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.209 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:14.209 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690698+0000", 2024-09-16T20:50:14.209 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051371223000000001 2024-09-16T20:50:14.209 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.209 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.209 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:14.209 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690748+0000", 2024-09-16T20:50:14.210 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0192999999999999e-05 2024-09-16T20:50:14.210 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.210 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.210 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:14.210 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690804+0000", 2024-09-16T20:50:14.210 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5658000000000001e-05 2024-09-16T20:50:14.210 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:14.210 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:14.210 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:14.210 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.210 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.210 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3736 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1570816~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:14.210 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:50:14.210 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 111.515294002, 2024-09-16T20:50:14.211 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051596528000000003, 2024-09-16T20:50:14.211 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:14.211 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:14.211 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:14.211 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:14.211 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:14.211 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:14.211 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3736 2024-09-16T20:50:14.211 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.211 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:14.211 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.211 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:14.211 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:50:14.211 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:14.212 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.212 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.212 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:14.212 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:50:14.212 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:14.212 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.212 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.212 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:14.212 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:50:14.212 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9999999999999999e-07 2024-09-16T20:50:14.212 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.212 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.212 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:14.212 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639335+0000", 2024-09-16T20:50:14.213 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.635e-06 2024-09-16T20:50:14.213 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.213 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.213 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:14.213 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639340+0000", 2024-09-16T20:50:14.213 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5840000000000002e-06 2024-09-16T20:50:14.213 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.213 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.213 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:14.213 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639346+0000", 2024-09-16T20:50:14.213 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1190000000000002e-06 2024-09-16T20:50:14.213 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.213 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.213 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:14.214 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690817+0000", 2024-09-16T20:50:14.214 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051470721999999997 2024-09-16T20:50:14.214 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.214 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.214 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:14.214 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690875+0000", 2024-09-16T20:50:14.214 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7441000000000001e-05 2024-09-16T20:50:14.214 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.214 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.214 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:14.214 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690929+0000", 2024-09-16T20:50:14.214 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4727e-05 2024-09-16T20:50:14.214 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:14.214 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:14.214 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:14.215 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.215 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.215 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5434 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:14.215 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:50:14.215 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 111.290032436, 2024-09-16T20:50:14.215 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099755831000000003, 2024-09-16T20:50:14.215 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:14.215 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:14.215 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:14.215 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:14.215 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:14.215 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:14.215 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5434 2024-09-16T20:50:14.215 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.216 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:14.216 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.216 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:14.216 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:50:14.216 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:14.216 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.216 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.216 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:14.216 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:50:14.216 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:14.216 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.216 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.216 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:14.216 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864595+0000", 2024-09-16T20:50:14.217 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1099999999999995e-07 2024-09-16T20:50:14.217 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.217 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.217 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:14.217 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864597+0000", 2024-09-16T20:50:14.217 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3989999999999998e-06 2024-09-16T20:50:14.217 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.217 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.217 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:14.217 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864603+0000", 2024-09-16T20:50:14.217 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9309999999999996e-06 2024-09-16T20:50:14.218 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.218 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.218 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:14.218 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864607+0000", 2024-09-16T20:50:14.218 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6200000000000001e-06 2024-09-16T20:50:14.218 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.218 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.218 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:14.218 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964164+0000", 2024-09-16T20:50:14.218 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099557162000000005 2024-09-16T20:50:14.218 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.218 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.219 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:14.219 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964244+0000", 2024-09-16T20:50:14.219 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9597000000000004e-05 2024-09-16T20:50:14.219 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.219 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.219 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:14.219 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964350+0000", 2024-09-16T20:50:14.219 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010641099999999999 2024-09-16T20:50:14.219 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:14.219 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:14.219 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:14.219 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.219 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.219 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5435 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 20480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:14.220 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:50:14.220 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 111.290002614, 2024-09-16T20:50:14.220 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100897102, 2024-09-16T20:50:14.220 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:14.220 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:14.220 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:14.220 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:14.220 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:14.220 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:14.220 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5435 2024-09-16T20:50:14.220 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.220 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:14.220 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.221 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:14.221 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:50:14.221 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:14.221 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.221 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.221 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:14.221 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:50:14.221 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:14.221 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.221 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.221 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:14.221 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:50:14.221 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8000000000000001e-07 2024-09-16T20:50:14.221 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.222 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.222 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:14.222 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864626+0000", 2024-09-16T20:50:14.222 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.745e-06 2024-09-16T20:50:14.222 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.222 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.222 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:14.222 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864633+0000", 2024-09-16T20:50:14.222 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3339999999999998e-06 2024-09-16T20:50:14.222 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.222 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.222 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:14.222 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864636+0000", 2024-09-16T20:50:14.223 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1200000000000002e-06 2024-09-16T20:50:14.223 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.223 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.223 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:14.223 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965299+0000", 2024-09-16T20:50:14.223 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10066352000000001 2024-09-16T20:50:14.223 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.223 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.223 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:14.223 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965390+0000", 2024-09-16T20:50:14.223 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0894000000000001e-05 2024-09-16T20:50:14.223 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.223 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.223 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:14.224 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965521+0000", 2024-09-16T20:50:14.224 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013110899999999999 2024-09-16T20:50:14.224 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:14.224 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:14.224 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:14.224 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.224 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.224 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:152 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 327680~32768 out=32768b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:50:14.224 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:50:14.224 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 109.991505672, 2024-09-16T20:50:14.224 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11869405800000001, 2024-09-16T20:50:14.224 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:14.224 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:14.225 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:14.225 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:14.225 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:50:14.225 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:50:14.225 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 152 2024-09-16T20:50:14.225 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.225 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:14.225 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.225 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:14.225 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:50:14.225 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:14.226 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.226 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.226 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:14.226 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:50:14.226 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:14.226 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.226 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.226 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:14.226 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163122+0000", 2024-09-16T20:50:14.227 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6400000000000003e-07 2024-09-16T20:50:14.227 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.227 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.227 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:14.227 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163124+0000", 2024-09-16T20:50:14.227 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.565e-06 2024-09-16T20:50:14.227 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.227 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.227 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:14.228 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163136+0000", 2024-09-16T20:50:14.228 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1717e-05 2024-09-16T20:50:14.228 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.228 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.228 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:14.228 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163142+0000", 2024-09-16T20:50:14.228 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1750000000000002e-06 2024-09-16T20:50:14.228 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.228 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.229 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:14.229 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163606+0000", 2024-09-16T20:50:14.229 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00046418799999999998 2024-09-16T20:50:14.229 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.229 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.229 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:14.229 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163656+0000", 2024-09-16T20:50:14.229 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9852999999999999e-05 2024-09-16T20:50:14.230 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.230 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.230 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:14.230 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281815+0000", 2024-09-16T20:50:14.230 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11815909600000001 2024-09-16T20:50:14.230 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:14.230 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:14.230 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:14.230 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.231 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.231 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:153 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:50:14.231 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:50:14.231 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 109.991472719, 2024-09-16T20:50:14.231 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11903462400000001, 2024-09-16T20:50:14.231 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:14.231 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:14.231 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:14.231 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:14.231 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:50:14.231 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:50:14.231 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 153 2024-09-16T20:50:14.231 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.231 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:14.232 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.232 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:14.232 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:50:14.232 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:14.232 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.232 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.232 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:14.232 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:50:14.232 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:14.232 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.232 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.232 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:14.232 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163155+0000", 2024-09-16T20:50:14.232 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.37e-07 2024-09-16T20:50:14.233 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.233 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.233 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:14.233 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163157+0000", 2024-09-16T20:50:14.233 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6010000000000002e-06 2024-09-16T20:50:14.233 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.233 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.233 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:14.233 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163165+0000", 2024-09-16T20:50:14.233 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2619999999999998e-06 2024-09-16T20:50:14.233 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.233 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.233 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:14.233 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163171+0000", 2024-09-16T20:50:14.234 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.711e-06 2024-09-16T20:50:14.234 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.234 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.234 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:14.234 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281868+0000", 2024-09-16T20:50:14.234 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118696622 2024-09-16T20:50:14.234 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.234 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.234 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:14.234 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281969+0000", 2024-09-16T20:50:14.234 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101371 2024-09-16T20:50:14.234 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.234 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.234 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:14.235 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282189+0000", 2024-09-16T20:50:14.235 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00021952000000000001 2024-09-16T20:50:14.235 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:14.235 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:14.235 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:14.235 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.235 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.235 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:154 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 360448~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:50:14.235 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:50:14.235 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 109.991440947, 2024-09-16T20:50:14.235 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11932035100000001, 2024-09-16T20:50:14.235 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:14.235 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:14.235 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:14.236 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:14.236 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:50:14.236 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:50:14.236 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 154 2024-09-16T20:50:14.236 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.236 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:14.236 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.236 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:14.236 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:50:14.236 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:14.236 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.236 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.236 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:14.236 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:50:14.237 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:14.237 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.237 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.237 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:14.237 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:50:14.237 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:50:14.237 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.237 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.237 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:14.237 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163189+0000", 2024-09-16T20:50:14.237 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6840000000000001e-06 2024-09-16T20:50:14.237 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.237 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.237 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:14.238 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163196+0000", 2024-09-16T20:50:14.238 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7920000000000004e-06 2024-09-16T20:50:14.238 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.238 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.238 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:14.238 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163201+0000", 2024-09-16T20:50:14.238 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2809999999999999e-06 2024-09-16T20:50:14.238 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.238 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.238 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:14.238 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282219+0000", 2024-09-16T20:50:14.238 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.119018343 2024-09-16T20:50:14.238 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.238 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.239 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:14.239 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282305+0000", 2024-09-16T20:50:14.239 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5865000000000005e-05 2024-09-16T20:50:14.239 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.239 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.239 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:14.239 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282506+0000", 2024-09-16T20:50:14.239 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000200927 2024-09-16T20:50:14.239 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:14.239 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:14.239 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:14.239 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.240 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.240 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop(client.4366.0:3088 3.1e e38/23 3:79e5ec01:::rbd_header.110e891a9b2b:head v 38'306, mlcod=36'304)", 2024-09-16T20:50:14.240 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:50:14.240 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 109.990422079, 2024-09-16T20:50:14.240 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118102759, 2024-09-16T20:50:14.240 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:14.240 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:14.240 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:14.240 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:14.240 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.240 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:14.240 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:50:14.240 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:14.241 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.241 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.241 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:14.241 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:50:14.241 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:14.241 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.241 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.241 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:14.241 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:50:14.241 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.85e-07 2024-09-16T20:50:14.241 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.241 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.241 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:14.241 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164212+0000", 2024-09-16T20:50:14.242 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3999999999999997e-06 2024-09-16T20:50:14.242 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.242 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.242 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:14.242 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281553+0000", 2024-09-16T20:50:14.242 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11734135499999999 2024-09-16T20:50:14.242 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.242 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.242 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:14.242 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281567+0000", 2024-09-16T20:50:14.242 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4188999999999999e-05 2024-09-16T20:50:14.242 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.242 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.243 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:14.243 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281653+0000", 2024-09-16T20:50:14.243 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5385999999999999e-05 2024-09-16T20:50:14.243 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.243 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.243 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:14.243 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281679+0000", 2024-09-16T20:50:14.244 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6689000000000001e-05 2024-09-16T20:50:14.244 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.244 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.244 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:14.244 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282277+0000", 2024-09-16T20:50:14.244 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00059798000000000002 2024-09-16T20:50:14.244 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.244 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.244 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:14.245 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282307+0000", 2024-09-16T20:50:14.245 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9975e-05 2024-09-16T20:50:14.245 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:14.245 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:14.245 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:14.245 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.245 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.245 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5255.0:44 3.16 3:6f5476df:::rbd_header.110e25db1f60:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:50:14.245 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:50:14.246 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 37.834119024000003, 2024-09-16T20:50:14.246 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.121007509, 2024-09-16T20:50:14.246 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:14.246 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:14.246 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:14.246 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:14.246 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5255", 2024-09-16T20:50:14.246 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:50:14.246 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 44 2024-09-16T20:50:14.247 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.247 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:14.247 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.247 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:14.247 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:50:14.247 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:14.247 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.247 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.248 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:14.248 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:50:14.248 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:14.248 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.248 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.248 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:14.249 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:50:14.249 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0500000000000003e-07 2024-09-16T20:50:14.249 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.249 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.249 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:14.249 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320512+0000", 2024-09-16T20:50:14.249 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2540000000000001e-06 2024-09-16T20:50:14.250 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.250 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.250 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:14.250 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320518+0000", 2024-09-16T20:50:14.250 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1689999999999997e-06 2024-09-16T20:50:14.250 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.250 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.250 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:14.250 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320521+0000", 2024-09-16T20:50:14.250 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5200000000000002e-06 2024-09-16T20:50:14.250 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.251 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.251 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:14.251 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320548+0000", 2024-09-16T20:50:14.251 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6662999999999999e-05 2024-09-16T20:50:14.251 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.251 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.251 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:14.251 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320587+0000", 2024-09-16T20:50:14.251 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8866999999999999e-05 2024-09-16T20:50:14.252 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.252 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.252 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:14.252 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.441515+0000", 2024-09-16T20:50:14.252 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.120928331 2024-09-16T20:50:14.252 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:14.252 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:14.252 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:14.252 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.253 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.253 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:20329 3.5 3:a4ea64b7:::rbd_header.110ede5ef857:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:14.253 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:01.061694+0000", 2024-09-16T20:50:14.253 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 13.092932456, 2024-09-16T20:50:14.253 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.071234077000000007, 2024-09-16T20:50:14.253 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:14.253 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:14.253 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:14.253 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:14.254 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:50:14.254 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:14.254 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 20329 2024-09-16T20:50:14.254 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.254 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:14.254 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.254 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:14.255 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.061694+0000", 2024-09-16T20:50:14.255 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:14.255 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.255 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.255 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:14.255 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.061694+0000", 2024-09-16T20:50:14.255 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:14.255 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.256 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.256 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:14.256 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089232+0000", 2024-09-16T20:50:14.256 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.027537275 2024-09-16T20:50:14.256 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.256 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.256 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:14.256 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089237+0000", 2024-09-16T20:50:14.257 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4709999999999998e-06 2024-09-16T20:50:14.257 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.257 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.257 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:14.257 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089246+0000", 2024-09-16T20:50:14.257 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8230000000000001e-06 2024-09-16T20:50:14.257 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.257 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.257 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:14.257 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089252+0000", 2024-09-16T20:50:14.257 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3060000000000003e-06 2024-09-16T20:50:14.258 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.258 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.258 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:14.258 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.132756+0000", 2024-09-16T20:50:14.258 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.043504334999999998 2024-09-16T20:50:14.258 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.258 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.258 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:14.258 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.132829+0000", 2024-09-16T20:50:14.258 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2849999999999995e-05 2024-09-16T20:50:14.258 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:14.258 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:14.258 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:14.258 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.132928+0000", 2024-09-16T20:50:14.258 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.9017000000000004e-05 2024-09-16T20:50:14.259 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:14.259 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:14.259 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:14.259 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:14.259 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:14.259 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:14.260 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #39, image image_client.0-clone39 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent40 2024-09-16T20:50:14.260 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:14.259+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:14.261 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:50:14.361 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:14.359+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:14.374 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:14.374 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:50:14.374 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:50:14.374 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:14.383 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:50:14.467 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:14.466+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:14.531 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:14.531 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:50:14.531 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:50:14.531 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:50:14.531 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:14.539 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:50:14.581 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:14.579+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:14.664 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:14.663+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:14.689 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:14.689 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:50:14.689 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:50:14.689 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:50:14.689 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.689 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:3088 3.1e 3:79e5ec01:::rbd_header.110e891a9b2b:head [watch watch cookie 140213437171488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:50:14.689 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:50:14.689 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 110.52311609100001, 2024-09-16T20:50:14.689 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.11895186200000001, 2024-09-16T20:50:14.689 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:14.690 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:14.690 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:14.690 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:14.690 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:50:14.690 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:14.690 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3088 2024-09-16T20:50:14.690 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.690 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:14.690 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.690 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:14.690 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:50:14.690 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:14.690 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.690 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.691 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:14.691 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:50:14.691 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:14.691 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.691 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.691 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:14.691 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163846+0000", 2024-09-16T20:50:14.691 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6799999999999999e-07 2024-09-16T20:50:14.691 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.691 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.691 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:14.691 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163850+0000", 2024-09-16T20:50:14.691 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.929e-06 2024-09-16T20:50:14.692 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.692 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.692 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:14.692 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163862+0000", 2024-09-16T20:50:14.692 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1972e-05 2024-09-16T20:50:14.692 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.692 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.692 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:14.692 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163867+0000", 2024-09-16T20:50:14.692 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.451e-06 2024-09-16T20:50:14.692 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.692 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.693 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:14.693 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163912+0000", 2024-09-16T20:50:14.693 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4456000000000003e-05 2024-09-16T20:50:14.693 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.693 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.693 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:14.693 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163962+0000", 2024-09-16T20:50:14.693 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0578000000000003e-05 2024-09-16T20:50:14.693 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.694 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.694 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-16T20:50:14.694 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164058+0000", 2024-09-16T20:50:14.694 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5934999999999995e-05 2024-09-16T20:50:14.694 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.694 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.694 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:50:14.694 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164570+0000", 2024-09-16T20:50:14.694 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00051241100000000005 2024-09-16T20:50:14.695 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.695 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.695 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:14.695 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282629+0000", 2024-09-16T20:50:14.695 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.118058986 2024-09-16T20:50:14.695 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.695 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.695 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:50:14.696 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282725+0000", 2024-09-16T20:50:14.696 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5063999999999996e-05 2024-09-16T20:50:14.696 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.696 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.696 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:14.696 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282797+0000", 2024-09-16T20:50:14.696 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2212000000000006e-05 2024-09-16T20:50:14.696 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:14.697 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:14.697 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:14.697 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.697 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.697 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:8039 3.1e 3.ac44ff9e (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:50:14.697 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:50:14.697 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 84.979288741999994, 2024-09-16T20:50:14.697 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0053310449999999, 2024-09-16T20:50:14.698 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:14.698 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:14.698 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:14.698 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:14.698 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:50:14.698 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:14.698 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8039 2024-09-16T20:50:14.698 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.698 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:14.699 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.699 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:14.699 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:50:14.699 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:14.699 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.699 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.699 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:14.699 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:50:14.699 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:14.699 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.699 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.700 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:14.700 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707674+0000", 2024-09-16T20:50:14.700 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6530000000000001e-06 2024-09-16T20:50:14.700 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.700 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.700 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:14.700 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707681+0000", 2024-09-16T20:50:14.700 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9639999999999997e-06 2024-09-16T20:50:14.700 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.700 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.700 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:14.700 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707700+0000", 2024-09-16T20:50:14.701 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8674000000000001e-05 2024-09-16T20:50:14.701 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.701 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.701 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:14.701 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707708+0000", 2024-09-16T20:50:14.701 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6810000000000004e-06 2024-09-16T20:50:14.701 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.701 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.701 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:14.701 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711293+0000", 2024-09-16T20:50:14.701 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035848189999999999 2024-09-16T20:50:14.701 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.701 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.701 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:14.702 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711301+0000", 2024-09-16T20:50:14.702 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4730000000000003e-06 2024-09-16T20:50:14.702 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.702 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.702 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:14.702 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.712990+0000", 2024-09-16T20:50:14.702 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.001689348 2024-09-16T20:50:14.702 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.702 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.702 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:14.702 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713003+0000", 2024-09-16T20:50:14.702 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3433e-05 2024-09-16T20:50:14.702 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:14.702 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:14.703 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:14.703 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.703 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.703 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:8040 3.1e 3.ac44ff9e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:50:14.703 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:50:14.703 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 84.979228347000003, 2024-09-16T20:50:14.703 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0053410780000001, 2024-09-16T20:50:14.703 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:14.703 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:14.703 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:14.703 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:14.703 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:50:14.703 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:14.704 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8040 2024-09-16T20:50:14.704 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.704 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:14.704 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.704 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:14.704 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:50:14.704 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:14.704 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.704 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.704 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:14.704 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:50:14.704 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:14.704 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.704 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.705 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:14.705 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:50:14.705 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8100000000000003e-07 2024-09-16T20:50:14.705 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.705 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.705 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:14.705 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707737+0000", 2024-09-16T20:50:14.705 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5020000000000001e-06 2024-09-16T20:50:14.705 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.705 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.705 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:14.705 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707745+0000", 2024-09-16T20:50:14.705 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2169999999999994e-06 2024-09-16T20:50:14.706 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.706 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.706 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:14.706 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707751+0000", 2024-09-16T20:50:14.706 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.263e-06 2024-09-16T20:50:14.706 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.706 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.706 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:14.706 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711336+0000", 2024-09-16T20:50:14.706 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035848360000000001 2024-09-16T20:50:14.706 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.706 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.706 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:14.707 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711340+0000", 2024-09-16T20:50:14.707 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1309999999999999e-06 2024-09-16T20:50:14.707 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.707 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.707 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:14.707 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713060+0000", 2024-09-16T20:50:14.707 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0017195059999999 2024-09-16T20:50:14.707 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.707 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.707 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:14.707 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713074+0000", 2024-09-16T20:50:14.707 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4042e-05 2024-09-16T20:50:14.707 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:14.708 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:14.708 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:14.708 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.708 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.708 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4927.0:4487 3.1e 3.ac44ff9e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:50:14.708 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:50:14.708 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 84.978708144999999, 2024-09-16T20:50:14.708 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0048840880000001, 2024-09-16T20:50:14.708 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:14.708 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:14.708 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:14.708 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:14.708 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4927", 2024-09-16T20:50:14.709 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:14.709 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4487 2024-09-16T20:50:14.709 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.709 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:14.709 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.709 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:14.709 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:50:14.709 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:14.709 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.709 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.709 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:14.709 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:50:14.710 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:14.710 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.710 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.710 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:14.710 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708254+0000", 2024-09-16T20:50:14.710 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1800000000000005e-07 2024-09-16T20:50:14.710 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.710 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.710 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:14.710 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708258+0000", 2024-09-16T20:50:14.710 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8689999999999997e-06 2024-09-16T20:50:14.710 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.710 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.710 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:14.711 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708267+0000", 2024-09-16T20:50:14.711 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9590000000000007e-06 2024-09-16T20:50:14.711 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.711 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.711 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:14.711 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708274+0000", 2024-09-16T20:50:14.711 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9970000000000001e-06 2024-09-16T20:50:14.711 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.711 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.711 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:14.711 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711366+0000", 2024-09-16T20:50:14.711 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030919530000000001 2024-09-16T20:50:14.712 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.712 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.712 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:14.712 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711372+0000", 2024-09-16T20:50:14.712 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9410000000000004e-06 2024-09-16T20:50:14.712 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.712 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.712 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:14.712 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713126+0000", 2024-09-16T20:50:14.712 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0017537249999999 2024-09-16T20:50:14.712 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.712 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.712 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:14.712 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713137+0000", 2024-09-16T20:50:14.713 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0825999999999999e-05 2024-09-16T20:50:14.713 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:14.713 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:14.713 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:14.713 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.713 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.713 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5089.0:11703 3.1b 3:dcd4684e:::rbd_data.110e747f0779.000000000000001b:head [set-alloc-hint object_size 262144 write_size 262144,write 139264~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:50:14.713 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:50:14.713 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 70.548972371000005, 2024-09-16T20:50:14.713 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.084231385000000006, 2024-09-16T20:50:14.713 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:14.713 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:14.713 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:14.713 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:14.714 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5089", 2024-09-16T20:50:14.714 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:50:14.714 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 11703 2024-09-16T20:50:14.714 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.714 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:14.714 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.714 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:14.714 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:50:14.714 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:14.714 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.714 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.714 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:14.714 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:50:14.714 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:14.715 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.715 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.715 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:14.715 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:50:14.715 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2600000000000001e-07 2024-09-16T20:50:14.715 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.715 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.715 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:14.715 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137998+0000", 2024-09-16T20:50:14.715 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9549999999999998e-06 2024-09-16T20:50:14.715 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.715 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.715 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:14.715 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138008+0000", 2024-09-16T20:50:14.716 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3789999999999995e-06 2024-09-16T20:50:14.716 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.716 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.716 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:14.716 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138013+0000", 2024-09-16T20:50:14.716 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2209999999999996e-06 2024-09-16T20:50:14.716 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.716 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.716 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:14.716 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138054+0000", 2024-09-16T20:50:14.716 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1022999999999997e-05 2024-09-16T20:50:14.716 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.716 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.717 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:14.717 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138124+0000", 2024-09-16T20:50:14.717 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0234999999999994e-05 2024-09-16T20:50:14.717 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.717 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.717 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:14.717 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138197+0000", 2024-09-16T20:50:14.717 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2451000000000001e-05 2024-09-16T20:50:14.717 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.717 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.717 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:14.717 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222010+0000", 2024-09-16T20:50:14.717 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.083813433000000007 2024-09-16T20:50:14.718 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.718 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.718 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:50:14.718 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222163+0000", 2024-09-16T20:50:14.718 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015274599999999999 2024-09-16T20:50:14.718 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.718 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.718 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:50:14.718 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222197+0000", 2024-09-16T20:50:14.718 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3683000000000003e-05 2024-09-16T20:50:14.718 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.718 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.718 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:14.718 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222220+0000", 2024-09-16T20:50:14.719 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3433000000000001e-05 2024-09-16T20:50:14.719 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:14.719 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:14.719 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:14.719 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.719 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.719 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.5089.0:11702 3.15 e73/64 3:acf33c8b:::rbd_data.110e747f0779.000000000000001a:head v 73'132, mlcod=73'130)", 2024-09-16T20:50:14.719 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:50:14.719 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 70.548705041999995, 2024-09-16T20:50:14.719 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.084079763000000002, 2024-09-16T20:50:14.719 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:14.719 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:14.719 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:14.719 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:14.720 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.720 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:14.720 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:50:14.720 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:14.720 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.720 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.720 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:14.720 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:50:14.720 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:14.720 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.720 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.720 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:14.721 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:50:14.721 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:50:14.721 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.721 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.721 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:14.721 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138263+0000", 2024-09-16T20:50:14.721 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1e-06 2024-09-16T20:50:14.721 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.721 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.721 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:14.721 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221589+0000", 2024-09-16T20:50:14.721 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.083326002999999996 2024-09-16T20:50:14.722 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.722 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.722 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:14.722 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221614+0000", 2024-09-16T20:50:14.722 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5593999999999999e-05 2024-09-16T20:50:14.722 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.722 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.722 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:14.722 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221714+0000", 2024-09-16T20:50:14.722 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9889000000000005e-05 2024-09-16T20:50:14.722 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.722 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.722 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:14.722 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221727+0000", 2024-09-16T20:50:14.723 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3125999999999999e-05 2024-09-16T20:50:14.723 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.723 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.723 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:50:14.723 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222294+0000", 2024-09-16T20:50:14.723 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00056712199999999998 2024-09-16T20:50:14.723 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.723 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.723 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:14.723 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222336+0000", 2024-09-16T20:50:14.723 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1470000000000001e-05 2024-09-16T20:50:14.723 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:14.723 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:14.723 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:14.724 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.724 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.724 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5138.0:9358 3.c 3:33996a43:::rbd_data.110e6372565d.0000000000000007:head [read 1343488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T20:50:14.724 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:50:14.724 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 46.834429632000003, 2024-09-16T20:50:14.724 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0167493569999999, 2024-09-16T20:50:14.724 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:14.724 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:14.724 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:14.724 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:14.724 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5138", 2024-09-16T20:50:14.725 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:50:14.725 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9358 2024-09-16T20:50:14.725 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.725 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:14.725 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.725 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:14.725 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:50:14.725 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:14.725 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.725 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.725 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:14.725 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:50:14.725 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:14.725 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.726 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.726 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:14.726 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852533+0000", 2024-09-16T20:50:14.726 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.646e-06 2024-09-16T20:50:14.726 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.726 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.726 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:14.726 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852539+0000", 2024-09-16T20:50:14.726 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3389999999999998e-06 2024-09-16T20:50:14.726 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.726 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.726 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:14.726 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852552+0000", 2024-09-16T20:50:14.726 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.271e-05 2024-09-16T20:50:14.727 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.727 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.727 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:14.727 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852563+0000", 2024-09-16T20:50:14.727 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1139e-05 2024-09-16T20:50:14.727 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.727 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.727 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:14.727 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852883+0000", 2024-09-16T20:50:14.727 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00032010600000000001 2024-09-16T20:50:14.727 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.727 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.727 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:14.728 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852890+0000", 2024-09-16T20:50:14.728 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7669999999999998e-06 2024-09-16T20:50:14.728 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.728 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.728 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:14.728 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869059+0000", 2024-09-16T20:50:14.728 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.016169353 2024-09-16T20:50:14.728 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.729 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.729 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:14.729 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869178+0000", 2024-09-16T20:50:14.729 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118193 2024-09-16T20:50:14.729 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.729 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.729 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:14.729 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869281+0000", 2024-09-16T20:50:14.729 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010310399999999999 2024-09-16T20:50:14.729 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:14.729 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:14.729 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:14.729 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.729 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.730 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5138.0:9382 3.c 3:33996a43:::rbd_data.110e6372565d.0000000000000007:head [read 0~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T20:50:14.730 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:50:14.730 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 46.834378755000003, 2024-09-16T20:50:14.730 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.017136034, 2024-09-16T20:50:14.730 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:14.730 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:14.730 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:14.730 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:14.730 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5138", 2024-09-16T20:50:14.730 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:50:14.730 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9382 2024-09-16T20:50:14.730 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.731 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:14.731 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.731 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:14.731 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:50:14.731 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:14.731 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.731 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.731 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:14.731 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:50:14.731 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:14.731 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.731 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.731 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:14.732 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852583+0000", 2024-09-16T20:50:14.732 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.51e-07 2024-09-16T20:50:14.732 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.732 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.732 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:14.732 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852586+0000", 2024-09-16T20:50:14.732 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0280000000000001e-06 2024-09-16T20:50:14.732 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.732 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.732 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:14.732 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852595+0000", 2024-09-16T20:50:14.732 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2799999999999992e-06 2024-09-16T20:50:14.732 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.733 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.733 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:14.733 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852601+0000", 2024-09-16T20:50:14.733 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3810000000000004e-06 2024-09-16T20:50:14.733 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.733 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.733 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:14.733 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852921+0000", 2024-09-16T20:50:14.733 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00031979699999999997 2024-09-16T20:50:14.733 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.733 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.733 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:14.733 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852930+0000", 2024-09-16T20:50:14.733 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6489999999999997e-06 2024-09-16T20:50:14.734 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.734 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.734 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:14.734 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869434+0000", 2024-09-16T20:50:14.734 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0165038829999999 2024-09-16T20:50:14.734 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.734 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.734 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:14.734 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869495+0000", 2024-09-16T20:50:14.734 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1509999999999999e-05 2024-09-16T20:50:14.734 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.734 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.734 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:14.734 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869718+0000", 2024-09-16T20:50:14.735 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000222955 2024-09-16T20:50:14.735 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:14.735 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:14.735 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:14.735 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.735 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.735 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9969 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 632832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:14.735 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642045+0000", 2024-09-16T20:50:14.735 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.044916298, 2024-09-16T20:50:14.735 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076903192999999995, 2024-09-16T20:50:14.735 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:14.735 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:14.735 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:14.735 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:14.736 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:14.736 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:14.736 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9969 2024-09-16T20:50:14.736 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.736 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:14.736 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.736 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:14.736 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642045+0000", 2024-09-16T20:50:14.736 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:14.736 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.736 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.736 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:14.736 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642045+0000", 2024-09-16T20:50:14.737 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:14.737 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.737 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.737 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:14.737 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642045+0000", 2024-09-16T20:50:14.737 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5100000000000001e-07 2024-09-16T20:50:14.737 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.737 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.737 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:14.737 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642047+0000", 2024-09-16T20:50:14.737 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8679999999999999e-06 2024-09-16T20:50:14.737 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.737 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.738 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:14.738 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642053+0000", 2024-09-16T20:50:14.738 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1720000000000004e-06 2024-09-16T20:50:14.738 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.738 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.738 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:14.738 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642060+0000", 2024-09-16T20:50:14.738 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0339999999999999e-06 2024-09-16T20:50:14.738 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.738 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.738 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:14.738 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.718562+0000", 2024-09-16T20:50:14.738 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076501969000000003 2024-09-16T20:50:14.738 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.739 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.739 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:14.739 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.718643+0000", 2024-09-16T20:50:14.740 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0539999999999998e-05 2024-09-16T20:50:14.740 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.740 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.740 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:14.740 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.718948+0000", 2024-09-16T20:50:14.740 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000305259 2024-09-16T20:50:14.740 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:14.740 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:14.740 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:14.741 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.741 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.741 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9970 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 637952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:14.741 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642067+0000", 2024-09-16T20:50:14.741 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.044894318000001, 2024-09-16T20:50:14.741 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077174928000000004, 2024-09-16T20:50:14.741 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:14.741 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:14.741 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:14.741 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:14.741 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:14.741 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:14.741 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9970 2024-09-16T20:50:14.742 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.742 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:14.742 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.742 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:14.742 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642067+0000", 2024-09-16T20:50:14.742 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:14.742 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.742 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.742 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:14.742 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642067+0000", 2024-09-16T20:50:14.742 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:14.743 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.743 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.743 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:14.743 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642067+0000", 2024-09-16T20:50:14.743 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2499999999999995e-07 2024-09-16T20:50:14.743 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.743 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.743 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:14.743 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642070+0000", 2024-09-16T20:50:14.743 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3199999999999998e-06 2024-09-16T20:50:14.743 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.743 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.743 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:14.744 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642076+0000", 2024-09-16T20:50:14.744 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.122e-06 2024-09-16T20:50:14.744 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.744 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.744 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:14.744 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642081+0000", 2024-09-16T20:50:14.744 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0869999999999997e-06 2024-09-16T20:50:14.744 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.744 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.744 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:14.744 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.718980+0000", 2024-09-16T20:50:14.744 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076899482000000005 2024-09-16T20:50:14.744 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.745 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.745 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:14.745 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719085+0000", 2024-09-16T20:50:14.745 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010446399999999999 2024-09-16T20:50:14.745 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.745 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.745 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:14.745 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719242+0000", 2024-09-16T20:50:14.745 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000156928 2024-09-16T20:50:14.745 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:14.745 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:14.745 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:14.745 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.745 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.746 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9971 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 643072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:14.746 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642102+0000", 2024-09-16T20:50:14.746 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.044858610999999, 2024-09-16T20:50:14.746 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077421016999999995, 2024-09-16T20:50:14.746 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:14.746 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:14.746 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:14.746 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:14.746 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:14.746 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:14.746 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9971 2024-09-16T20:50:14.746 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.746 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:14.746 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.747 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:14.747 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642102+0000", 2024-09-16T20:50:14.747 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:14.747 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.747 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.747 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:14.747 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642102+0000", 2024-09-16T20:50:14.747 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:14.747 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.747 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.747 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:14.747 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642103+0000", 2024-09-16T20:50:14.747 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7399999999999998e-07 2024-09-16T20:50:14.748 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.748 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.748 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:14.748 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642105+0000", 2024-09-16T20:50:14.748 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2649999999999999e-06 2024-09-16T20:50:14.748 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.748 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.748 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:14.748 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642112+0000", 2024-09-16T20:50:14.748 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.294e-06 2024-09-16T20:50:14.748 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.748 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.748 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:14.748 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642118+0000", 2024-09-16T20:50:14.749 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4500000000000003e-06 2024-09-16T20:50:14.749 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.749 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.749 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:14.749 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719285+0000", 2024-09-16T20:50:14.749 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077166883000000006 2024-09-16T20:50:14.749 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.749 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.749 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:14.749 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719368+0000", 2024-09-16T20:50:14.749 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2811000000000004e-05 2024-09-16T20:50:14.749 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.749 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.749 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:14.750 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719523+0000", 2024-09-16T20:50:14.750 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015584 2024-09-16T20:50:14.750 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:14.750 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:14.750 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:14.750 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.750 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.750 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9972 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 648192~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:14.750 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642125+0000", 2024-09-16T20:50:14.750 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.044835664000001, 2024-09-16T20:50:14.750 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077710324999999997, 2024-09-16T20:50:14.750 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:14.750 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:14.750 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:14.751 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:14.751 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:14.751 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:14.751 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9972 2024-09-16T20:50:14.751 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.751 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:14.751 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.751 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:14.751 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642125+0000", 2024-09-16T20:50:14.751 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:14.751 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.751 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.751 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:14.751 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642125+0000", 2024-09-16T20:50:14.752 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:14.752 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.752 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.752 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:14.752 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642126+0000", 2024-09-16T20:50:14.752 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1899999999999998e-07 2024-09-16T20:50:14.752 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.752 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.752 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:14.752 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642128+0000", 2024-09-16T20:50:14.753 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8619999999999999e-06 2024-09-16T20:50:14.753 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.753 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.753 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:14.753 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642134+0000", 2024-09-16T20:50:14.753 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5289999999999997e-06 2024-09-16T20:50:14.754 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.754 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.754 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:14.754 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642139+0000", 2024-09-16T20:50:14.754 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1250000000000001e-06 2024-09-16T20:50:14.754 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.755 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.755 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:14.755 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719570+0000", 2024-09-16T20:50:14.755 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077430516000000005 2024-09-16T20:50:14.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.755 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.755 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:14.755 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719655+0000", 2024-09-16T20:50:14.756 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5500000000000005e-05 2024-09-16T20:50:14.756 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.756 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.756 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:14.756 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719836+0000", 2024-09-16T20:50:14.756 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018037399999999999 2024-09-16T20:50:14.756 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:14.757 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:14.757 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:14.757 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.757 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.757 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9973 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 653312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:14.757 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:50:14.757 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.044815054000001, 2024-09-16T20:50:14.757 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077905262000000003, 2024-09-16T20:50:14.758 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:14.758 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:14.758 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:14.758 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:14.758 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:14.758 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:14.758 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9973 2024-09-16T20:50:14.759 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.759 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:14.759 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.759 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:14.759 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:50:14.759 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:14.760 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.760 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.760 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:14.760 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:50:14.760 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:14.760 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.760 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.761 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:14.761 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:50:14.761 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:50:14.761 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.761 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.761 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:14.761 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642148+0000", 2024-09-16T20:50:14.761 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0219999999999999e-06 2024-09-16T20:50:14.761 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.761 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.761 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:14.761 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642154+0000", 2024-09-16T20:50:14.762 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8540000000000002e-06 2024-09-16T20:50:14.762 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.762 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.762 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:14.762 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642159+0000", 2024-09-16T20:50:14.762 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.045e-06 2024-09-16T20:50:14.762 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.762 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.762 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:14.762 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719875+0000", 2024-09-16T20:50:14.762 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077716116000000002 2024-09-16T20:50:14.762 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.762 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.762 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:14.763 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719934+0000", 2024-09-16T20:50:14.763 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8819e-05 2024-09-16T20:50:14.763 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.763 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.763 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:14.763 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720051+0000", 2024-09-16T20:50:14.763 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000116947 2024-09-16T20:50:14.763 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:14.763 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:14.763 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:14.763 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.763 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.763 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9974 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 658432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:14.763 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642166+0000", 2024-09-16T20:50:14.764 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.044795140000002, 2024-09-16T20:50:14.764 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.078179786000000001, 2024-09-16T20:50:14.764 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:14.764 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:14.764 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:14.764 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:14.764 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:14.764 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:14.764 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9974 2024-09-16T20:50:14.764 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.764 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:14.764 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.764 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:14.764 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642166+0000", 2024-09-16T20:50:14.765 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:14.765 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.765 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.765 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:14.765 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642166+0000", 2024-09-16T20:50:14.765 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:14.765 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.765 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.765 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:14.765 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647343+0000", 2024-09-16T20:50:14.765 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0051769440000000002 2024-09-16T20:50:14.765 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.766 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.766 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:14.766 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647348+0000", 2024-09-16T20:50:14.766 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3619999999999997e-06 2024-09-16T20:50:14.766 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.766 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.766 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:14.766 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647359+0000", 2024-09-16T20:50:14.766 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0929e-05 2024-09-16T20:50:14.766 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.766 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.766 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:14.766 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647366+0000", 2024-09-16T20:50:14.766 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0910000000000004e-06 2024-09-16T20:50:14.767 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.767 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.767 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:14.767 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720094+0000", 2024-09-16T20:50:14.767 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.072727403999999995 2024-09-16T20:50:14.767 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.767 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.767 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:14.767 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720173+0000", 2024-09-16T20:50:14.767 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9382000000000007e-05 2024-09-16T20:50:14.767 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.767 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.767 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:14.767 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720346+0000", 2024-09-16T20:50:14.768 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017267399999999999 2024-09-16T20:50:14.768 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:14.768 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:14.768 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:14.768 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.768 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.768 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9988 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 730112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:14.768 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647703+0000", 2024-09-16T20:50:14.768 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.039258147999998, 2024-09-16T20:50:14.768 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076703473999999994, 2024-09-16T20:50:14.768 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:14.768 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:14.768 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:14.769 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:14.769 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:14.769 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:14.769 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9988 2024-09-16T20:50:14.769 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.769 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:14.769 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.769 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:14.769 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647703+0000", 2024-09-16T20:50:14.769 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:14.769 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.770 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.770 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:14.770 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647703+0000", 2024-09-16T20:50:14.770 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:14.770 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.770 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.770 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:14.770 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647703+0000", 2024-09-16T20:50:14.770 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0699999999999998e-07 2024-09-16T20:50:14.770 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.770 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.770 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:14.770 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647705+0000", 2024-09-16T20:50:14.771 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.866e-06 2024-09-16T20:50:14.771 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.771 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.771 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:14.771 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647708+0000", 2024-09-16T20:50:14.771 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3170000000000002e-06 2024-09-16T20:50:14.771 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.771 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.772 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:14.772 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647711+0000", 2024-09-16T20:50:14.772 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6759999999999999e-06 2024-09-16T20:50:14.772 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.772 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.772 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:14.772 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724225+0000", 2024-09-16T20:50:14.772 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.07651434 2024-09-16T20:50:14.772 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.772 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.772 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:14.772 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724291+0000", 2024-09-16T20:50:14.773 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5629000000000005e-05 2024-09-16T20:50:14.773 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.773 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.773 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:14.773 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724406+0000", 2024-09-16T20:50:14.773 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000115239 2024-09-16T20:50:14.773 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:14.773 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:14.773 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:14.773 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.773 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.773 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9989 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 735232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:14.773 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647715+0000", 2024-09-16T20:50:14.773 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.039246410000001, 2024-09-16T20:50:14.774 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076871693000000005, 2024-09-16T20:50:14.774 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:14.774 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:14.774 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:14.774 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:14.774 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:14.774 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:14.774 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9989 2024-09-16T20:50:14.774 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.774 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:14.774 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.774 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:14.774 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647715+0000", 2024-09-16T20:50:14.775 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:14.775 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.775 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.775 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:14.775 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647715+0000", 2024-09-16T20:50:14.775 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:14.775 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.775 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.775 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:14.775 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647715+0000", 2024-09-16T20:50:14.775 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1400000000000001e-07 2024-09-16T20:50:14.775 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.776 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.776 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:14.776 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647716+0000", 2024-09-16T20:50:14.776 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2240000000000001e-06 2024-09-16T20:50:14.776 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.776 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.776 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:14.776 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647719+0000", 2024-09-16T20:50:14.776 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0769999999999999e-06 2024-09-16T20:50:14.776 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.776 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.776 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:14.776 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647722+0000", 2024-09-16T20:50:14.776 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.65e-06 2024-09-16T20:50:14.777 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.777 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.777 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:14.777 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724424+0000", 2024-09-16T20:50:14.777 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076701982000000002 2024-09-16T20:50:14.777 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.777 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.777 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:14.777 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724470+0000", 2024-09-16T20:50:14.777 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6589999999999999e-05 2024-09-16T20:50:14.777 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.777 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.777 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:14.778 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724586+0000", 2024-09-16T20:50:14.778 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000115956 2024-09-16T20:50:14.778 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:14.778 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:14.778 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:14.778 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.778 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.778 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9990 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 740352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:14.778 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647725+0000", 2024-09-16T20:50:14.778 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.039236056, 2024-09-16T20:50:14.778 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077029919000000002, 2024-09-16T20:50:14.778 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:14.778 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:14.778 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:14.779 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:14.779 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:14.779 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:14.779 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9990 2024-09-16T20:50:14.779 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.779 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:14.779 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.779 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:14.779 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647725+0000", 2024-09-16T20:50:14.779 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:14.779 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.779 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.779 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:14.779 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647725+0000", 2024-09-16T20:50:14.780 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:14.780 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.780 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.780 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:14.780 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647725+0000", 2024-09-16T20:50:14.780 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2399999999999999e-07 2024-09-16T20:50:14.780 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.780 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.780 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:14.780 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647726+0000", 2024-09-16T20:50:14.780 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.116e-06 2024-09-16T20:50:14.780 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.780 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.780 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:14.781 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647729+0000", 2024-09-16T20:50:14.781 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.937e-06 2024-09-16T20:50:14.781 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.781 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.781 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:14.781 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647732+0000", 2024-09-16T20:50:14.781 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8739999999999999e-06 2024-09-16T20:50:14.781 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.781 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.781 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:14.781 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724612+0000", 2024-09-16T20:50:14.781 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076880102000000006 2024-09-16T20:50:14.781 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.781 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.782 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:14.782 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724649+0000", 2024-09-16T20:50:14.782 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6738e-05 2024-09-16T20:50:14.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.782 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.782 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:14.782 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724755+0000", 2024-09-16T20:50:14.782 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010592800000000001 2024-09-16T20:50:14.782 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:14.782 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:14.782 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:14.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.782 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.782 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9991 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 745472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:14.783 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:50:14.783 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.039225747, 2024-09-16T20:50:14.783 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077257820000000005, 2024-09-16T20:50:14.783 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:14.783 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:14.783 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:14.783 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:14.783 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:14.783 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:14.783 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9991 2024-09-16T20:50:14.783 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.783 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:14.783 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.783 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:14.783 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:50:14.784 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:14.784 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.784 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.784 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:14.784 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:50:14.784 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:14.784 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.784 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.784 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:14.784 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:50:14.784 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1199999999999999e-07 2024-09-16T20:50:14.784 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.784 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.784 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:14.785 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647736+0000", 2024-09-16T20:50:14.785 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3699999999999999e-07 2024-09-16T20:50:14.785 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.785 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.785 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:14.785 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647739+0000", 2024-09-16T20:50:14.785 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9359999999999999e-06 2024-09-16T20:50:14.785 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.785 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.785 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:14.785 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647742+0000", 2024-09-16T20:50:14.785 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5720000000000001e-06 2024-09-16T20:50:14.785 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.786 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:14.786 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724774+0000", 2024-09-16T20:50:14.786 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077032457999999998 2024-09-16T20:50:14.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.786 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:14.786 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724818+0000", 2024-09-16T20:50:14.786 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3803000000000001e-05 2024-09-16T20:50:14.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.786 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:14.786 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724993+0000", 2024-09-16T20:50:14.786 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000174902 2024-09-16T20:50:14.786 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:14.787 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:14.787 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:14.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.787 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.787 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9992 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 750592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:14.787 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:50:14.787 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.039216157999999, 2024-09-16T20:50:14.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077553227000000002, 2024-09-16T20:50:14.787 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:14.787 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:14.787 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:14.787 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:14.787 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:14.787 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:14.788 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9992 2024-09-16T20:50:14.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.788 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:14.788 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.788 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:14.788 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:50:14.788 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:14.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.788 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.788 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:14.788 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:50:14.788 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:14.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.789 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.789 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:14.789 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:50:14.789 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0200000000000001e-07 2024-09-16T20:50:14.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.789 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.789 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:14.789 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647746+0000", 2024-09-16T20:50:14.789 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.088e-06 2024-09-16T20:50:14.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.789 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.789 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:14.789 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647749+0000", 2024-09-16T20:50:14.789 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9170000000000002e-06 2024-09-16T20:50:14.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.790 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.790 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:14.790 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647752+0000", 2024-09-16T20:50:14.790 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.774e-06 2024-09-16T20:50:14.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.790 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.790 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:14.790 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725042+0000", 2024-09-16T20:50:14.790 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077289952999999995 2024-09-16T20:50:14.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.790 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.790 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:14.790 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725113+0000", 2024-09-16T20:50:14.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0920999999999994e-05 2024-09-16T20:50:14.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.791 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.791 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:14.791 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725298+0000", 2024-09-16T20:50:14.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000185372 2024-09-16T20:50:14.791 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:14.791 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:14.791 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:14.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.791 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.791 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9993 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 755712~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:14.791 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647755+0000", 2024-09-16T20:50:14.791 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.039206132, 2024-09-16T20:50:14.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077725674999999994, 2024-09-16T20:50:14.792 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:14.792 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:14.792 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:14.792 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:14.792 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:14.792 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:14.792 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9993 2024-09-16T20:50:14.792 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.792 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:14.792 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.792 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:14.792 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647755+0000", 2024-09-16T20:50:14.792 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:14.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.793 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.793 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:14.793 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647755+0000", 2024-09-16T20:50:14.793 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:14.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.793 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.793 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:14.793 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653406+0000", 2024-09-16T20:50:14.793 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.005650876 2024-09-16T20:50:14.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.793 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.794 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:14.794 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653411+0000", 2024-09-16T20:50:14.794 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8899999999999998e-06 2024-09-16T20:50:14.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.794 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:14.794 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653423+0000", 2024-09-16T20:50:14.794 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2447e-05 2024-09-16T20:50:14.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.794 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:14.794 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653431+0000", 2024-09-16T20:50:14.794 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5730000000000002e-06 2024-09-16T20:50:14.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.795 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.795 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:14.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725333+0000", 2024-09-16T20:50:14.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.071902743000000005 2024-09-16T20:50:14.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.795 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.795 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:14.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725385+0000", 2024-09-16T20:50:14.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1585000000000002e-05 2024-09-16T20:50:14.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:14.795 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:14.795 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:14.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725481+0000", 2024-09-16T20:50:14.796 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5561000000000002e-05 2024-09-16T20:50:14.796 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:14.796 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:14.796 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:14.796 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:14.796 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:14.796 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:14.798 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:14.796+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:14.798 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:50:14.898 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:14.897+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:14.946 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:14.946 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:50:14.946 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:50:14.946 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:14.955 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:50:14.999 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:14.997+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:15.061 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:15.061 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:50:15.062 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:50:15.062 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:50:15.062 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:15.070 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:50:15.111 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:15.109+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:15.170 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T20:50:15.201 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:15.199+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:15.218 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:15.218 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:50:15.219 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:50:15.219 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:50:15.219 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.219 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12548 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 628736~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:15.219 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757009+0000", 2024-09-16T20:50:15.219 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.459849939000001, 2024-09-16T20:50:15.219 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065743956000000006, 2024-09-16T20:50:15.219 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:15.219 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:15.219 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:15.219 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:15.219 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:15.219 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:15.220 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12548 2024-09-16T20:50:15.220 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.220 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:15.220 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.220 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:15.220 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757009+0000", 2024-09-16T20:50:15.220 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:15.220 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.220 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.220 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:15.220 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757009+0000", 2024-09-16T20:50:15.220 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:15.220 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.221 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.221 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:15.221 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757010+0000", 2024-09-16T20:50:15.221 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9900000000000001e-07 2024-09-16T20:50:15.221 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.221 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.221 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:15.221 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757012+0000", 2024-09-16T20:50:15.221 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2689999999999998e-06 2024-09-16T20:50:15.221 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.221 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.221 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:15.221 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757022+0000", 2024-09-16T20:50:15.221 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9699999999999994e-06 2024-09-16T20:50:15.222 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.222 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.222 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:15.222 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757028+0000", 2024-09-16T20:50:15.222 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3799999999999999e-06 2024-09-16T20:50:15.222 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.222 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.222 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:15.222 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822552+0000", 2024-09-16T20:50:15.222 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065524281000000004 2024-09-16T20:50:15.222 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.222 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.222 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:15.222 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822613+0000", 2024-09-16T20:50:15.223 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0430999999999998e-05 2024-09-16T20:50:15.223 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.223 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.223 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:15.223 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822753+0000", 2024-09-16T20:50:15.223 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000140226 2024-09-16T20:50:15.223 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:15.223 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:15.223 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:15.223 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.223 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.223 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12549 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 633856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:15.223 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:50:15.223 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.459821192, 2024-09-16T20:50:15.224 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065978518, 2024-09-16T20:50:15.224 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:15.224 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:15.224 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:15.224 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:15.224 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:15.224 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:15.224 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12549 2024-09-16T20:50:15.224 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.224 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:15.224 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.224 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:15.224 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:50:15.225 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:15.225 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.225 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.225 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:15.225 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:50:15.225 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:15.225 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.225 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.225 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:15.225 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:50:15.225 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9299999999999999e-07 2024-09-16T20:50:15.225 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.225 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.225 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:15.226 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757040+0000", 2024-09-16T20:50:15.226 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.677e-06 2024-09-16T20:50:15.226 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.226 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.226 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:15.226 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757045+0000", 2024-09-16T20:50:15.226 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8149999999999996e-06 2024-09-16T20:50:15.226 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.226 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.226 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:15.226 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757048+0000", 2024-09-16T20:50:15.226 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1e-06 2024-09-16T20:50:15.226 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.227 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.227 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:15.227 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822796+0000", 2024-09-16T20:50:15.227 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065748081 2024-09-16T20:50:15.227 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.227 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.227 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:15.227 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822867+0000", 2024-09-16T20:50:15.227 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0679000000000005e-05 2024-09-16T20:50:15.227 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.227 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.227 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:15.227 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823016+0000", 2024-09-16T20:50:15.227 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014987299999999999 2024-09-16T20:50:15.228 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:15.228 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:15.228 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:15.228 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.228 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.228 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12550 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 638976~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:15.228 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757060+0000", 2024-09-16T20:50:15.228 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.459798777, 2024-09-16T20:50:15.228 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066153564999999998, 2024-09-16T20:50:15.228 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:15.228 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:15.228 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:15.228 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:15.228 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:15.229 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:15.229 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12550 2024-09-16T20:50:15.229 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.229 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:15.229 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.229 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:15.229 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757060+0000", 2024-09-16T20:50:15.229 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:15.229 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.229 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.229 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:15.229 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757060+0000", 2024-09-16T20:50:15.229 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:15.229 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.230 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.230 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:15.230 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757061+0000", 2024-09-16T20:50:15.230 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.15e-07 2024-09-16T20:50:15.230 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.230 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.230 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:15.230 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757062+0000", 2024-09-16T20:50:15.230 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.663e-06 2024-09-16T20:50:15.230 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.230 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.230 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:15.230 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757066+0000", 2024-09-16T20:50:15.230 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.817e-06 2024-09-16T20:50:15.231 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.231 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.231 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:15.231 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757070+0000", 2024-09-16T20:50:15.231 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1509999999999997e-06 2024-09-16T20:50:15.231 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.231 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.231 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:15.231 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823044+0000", 2024-09-16T20:50:15.231 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065973992999999995 2024-09-16T20:50:15.231 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.231 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.231 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:15.231 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823082+0000", 2024-09-16T20:50:15.231 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8050999999999999e-05 2024-09-16T20:50:15.232 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.232 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.232 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:15.232 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823214+0000", 2024-09-16T20:50:15.232 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013157500000000001 2024-09-16T20:50:15.232 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:15.232 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:15.232 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:15.232 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.232 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.232 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12551 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 644096~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:15.232 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:50:15.232 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.459785079, 2024-09-16T20:50:15.233 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066391378000000001, 2024-09-16T20:50:15.233 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:15.233 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:15.233 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:15.233 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:15.233 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:15.233 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:15.233 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12551 2024-09-16T20:50:15.233 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.233 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:15.233 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.233 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:15.233 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:50:15.234 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:15.234 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.234 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.234 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:15.234 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:50:15.234 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:15.234 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.234 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.234 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:15.234 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:50:15.234 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2999999999999999e-07 2024-09-16T20:50:15.234 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.234 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.234 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:15.235 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757075+0000", 2024-09-16T20:50:15.235 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.097e-06 2024-09-16T20:50:15.235 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.235 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.235 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:15.235 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757082+0000", 2024-09-16T20:50:15.235 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9500000000000004e-06 2024-09-16T20:50:15.235 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.235 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.235 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:15.235 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757086+0000", 2024-09-16T20:50:15.235 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2229999999999999e-06 2024-09-16T20:50:15.235 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.235 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.236 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:15.236 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823262+0000", 2024-09-16T20:50:15.236 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066176078999999999 2024-09-16T20:50:15.236 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.236 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.236 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:15.236 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823336+0000", 2024-09-16T20:50:15.236 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3979000000000004e-05 2024-09-16T20:50:15.236 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.236 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.236 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:15.236 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823465+0000", 2024-09-16T20:50:15.236 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012982 2024-09-16T20:50:15.236 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:15.237 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:15.237 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:15.237 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.237 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.237 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12552 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 649216~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:15.237 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757089+0000", 2024-09-16T20:50:15.237 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.459769938000001, 2024-09-16T20:50:15.237 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066655617, 2024-09-16T20:50:15.237 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:15.237 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:15.237 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:15.237 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:15.237 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:15.237 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:15.238 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12552 2024-09-16T20:50:15.238 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.238 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:15.238 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.238 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:15.238 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757089+0000", 2024-09-16T20:50:15.238 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:15.238 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.239 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.239 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:15.240 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757089+0000", 2024-09-16T20:50:15.240 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:15.240 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.240 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.240 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:15.240 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759492+0000", 2024-09-16T20:50:15.240 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0024026109999999998 2024-09-16T20:50:15.240 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.240 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.240 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:15.240 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759497+0000", 2024-09-16T20:50:15.240 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1120000000000004e-06 2024-09-16T20:50:15.240 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.241 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.241 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:15.241 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759508+0000", 2024-09-16T20:50:15.241 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0981e-05 2024-09-16T20:50:15.241 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.241 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.241 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:15.241 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759516+0000", 2024-09-16T20:50:15.241 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7270000000000007e-06 2024-09-16T20:50:15.241 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.241 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.241 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:15.241 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823504+0000", 2024-09-16T20:50:15.241 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.063987984999999997 2024-09-16T20:50:15.242 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.242 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.242 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:15.242 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823579+0000", 2024-09-16T20:50:15.242 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5703000000000003e-05 2024-09-16T20:50:15.242 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.242 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.242 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:15.242 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823745+0000", 2024-09-16T20:50:15.242 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016549800000000001 2024-09-16T20:50:15.242 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:15.242 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:15.242 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:15.243 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.243 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.243 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12560 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 690176~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:15.243 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.759658+0000", 2024-09-16T20:50:15.243 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.457200736000001, 2024-09-16T20:50:15.243 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065858517000000005, 2024-09-16T20:50:15.243 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:15.243 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:15.243 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:15.243 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:15.243 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:15.243 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:15.243 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12560 2024-09-16T20:50:15.244 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.244 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:15.244 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.244 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:15.244 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759658+0000", 2024-09-16T20:50:15.244 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:15.244 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.244 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.244 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:15.244 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759658+0000", 2024-09-16T20:50:15.244 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:15.244 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.244 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.244 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:15.245 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759659+0000", 2024-09-16T20:50:15.245 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.17e-07 2024-09-16T20:50:15.245 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.245 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.245 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:15.245 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759660+0000", 2024-09-16T20:50:15.245 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0189999999999999e-06 2024-09-16T20:50:15.245 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.245 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.245 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:15.245 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759663+0000", 2024-09-16T20:50:15.245 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0010000000000002e-06 2024-09-16T20:50:15.245 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.245 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.246 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:15.246 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759665+0000", 2024-09-16T20:50:15.246 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5440000000000001e-06 2024-09-16T20:50:15.246 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.246 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.246 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:15.246 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825355+0000", 2024-09-16T20:50:15.246 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065690224000000005 2024-09-16T20:50:15.246 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.246 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.246 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:15.246 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825394+0000", 2024-09-16T20:50:15.246 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8810999999999998e-05 2024-09-16T20:50:15.246 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.247 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.247 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:15.247 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825517+0000", 2024-09-16T20:50:15.247 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012270099999999999 2024-09-16T20:50:15.247 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:15.247 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:15.247 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:15.247 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.247 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.247 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12561 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 695296~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:15.247 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.759668+0000", 2024-09-16T20:50:15.247 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.457190894, 2024-09-16T20:50:15.247 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066116999999999995, 2024-09-16T20:50:15.247 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:15.248 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:15.248 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:15.248 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:15.248 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:15.248 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:15.248 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12561 2024-09-16T20:50:15.248 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.248 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:15.248 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.248 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:15.248 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759668+0000", 2024-09-16T20:50:15.248 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:15.248 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.249 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.249 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:15.249 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759668+0000", 2024-09-16T20:50:15.249 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:15.249 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.249 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.249 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:15.249 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765379+0000", 2024-09-16T20:50:15.249 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057104410000000001 2024-09-16T20:50:15.249 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.249 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.249 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:15.249 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765383+0000", 2024-09-16T20:50:15.249 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2980000000000002e-06 2024-09-16T20:50:15.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.250 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.250 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:15.250 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765395+0000", 2024-09-16T20:50:15.250 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1671000000000001e-05 2024-09-16T20:50:15.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.250 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.250 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:15.250 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765403+0000", 2024-09-16T20:50:15.250 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1210000000000007e-06 2024-09-16T20:50:15.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.250 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.250 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:15.250 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825552+0000", 2024-09-16T20:50:15.251 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.060149498000000003 2024-09-16T20:50:15.251 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.251 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.251 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:15.251 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825625+0000", 2024-09-16T20:50:15.251 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2540999999999995e-05 2024-09-16T20:50:15.251 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.251 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.251 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:15.251 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825785+0000", 2024-09-16T20:50:15.251 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016043000000000001 2024-09-16T20:50:15.251 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:15.251 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:15.251 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:15.252 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.252 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.252 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12585 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 818176~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:15.252 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.765848+0000", 2024-09-16T20:50:15.252 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.451010922999998, 2024-09-16T20:50:15.252 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065843218999999995, 2024-09-16T20:50:15.252 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:15.252 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:15.252 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:15.252 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:15.252 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:15.252 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:15.252 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12585 2024-09-16T20:50:15.252 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.253 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:15.253 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.253 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:15.253 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765848+0000", 2024-09-16T20:50:15.253 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:15.253 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.253 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.253 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:15.253 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765848+0000", 2024-09-16T20:50:15.253 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:15.253 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.253 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.253 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:15.253 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768683+0000", 2024-09-16T20:50:15.254 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0028344519999999999 2024-09-16T20:50:15.254 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.254 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.254 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:15.254 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768686+0000", 2024-09-16T20:50:15.254 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8789999999999996e-06 2024-09-16T20:50:15.254 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.254 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.254 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:15.254 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768696+0000", 2024-09-16T20:50:15.254 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6709999999999994e-06 2024-09-16T20:50:15.254 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.254 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.254 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:15.255 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768702+0000", 2024-09-16T20:50:15.255 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2550000000000003e-06 2024-09-16T20:50:15.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.255 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.255 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:15.255 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.831499+0000", 2024-09-16T20:50:15.255 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.062797081000000005 2024-09-16T20:50:15.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.255 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.255 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:15.255 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.831538+0000", 2024-09-16T20:50:15.255 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8523999999999999e-05 2024-09-16T20:50:15.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.255 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.256 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:15.256 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.831691+0000", 2024-09-16T20:50:15.256 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015335699999999999 2024-09-16T20:50:15.256 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:15.256 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:15.256 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:15.256 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.256 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.256 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12597 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 879616~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:15.256 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.768883+0000", 2024-09-16T20:50:15.256 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.447976194999999, 2024-09-16T20:50:15.256 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065816527, 2024-09-16T20:50:15.256 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:15.257 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:15.257 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:15.257 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:15.257 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:15.257 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:15.257 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12597 2024-09-16T20:50:15.257 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.257 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:15.257 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.257 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:15.257 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768883+0000", 2024-09-16T20:50:15.257 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:15.257 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.257 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.258 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:15.258 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768883+0000", 2024-09-16T20:50:15.258 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:15.258 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.258 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.258 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:15.258 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771508+0000", 2024-09-16T20:50:15.258 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0026247589999999999 2024-09-16T20:50:15.258 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.258 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.258 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:15.258 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771513+0000", 2024-09-16T20:50:15.258 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9949999999999996e-06 2024-09-16T20:50:15.258 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.259 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.259 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:15.259 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771525+0000", 2024-09-16T20:50:15.259 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2033e-05 2024-09-16T20:50:15.259 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.259 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.259 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:15.259 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771532+0000", 2024-09-16T20:50:15.259 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4220000000000003e-06 2024-09-16T20:50:15.259 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.259 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.259 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:15.259 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.834532+0000", 2024-09-16T20:50:15.259 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.063000159 2024-09-16T20:50:15.260 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.260 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.260 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:15.260 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.834590+0000", 2024-09-16T20:50:15.260 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8218999999999999e-05 2024-09-16T20:50:15.260 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.260 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.260 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:15.260 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.834699+0000", 2024-09-16T20:50:15.260 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010894 2024-09-16T20:50:15.260 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:15.260 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:15.260 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:15.260 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.261 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.261 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12619 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 992256~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:15.261 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.774656+0000", 2024-09-16T20:50:15.261 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.442203420999999, 2024-09-16T20:50:15.261 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065715859000000001, 2024-09-16T20:50:15.261 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:15.261 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:15.261 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:15.261 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:15.261 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:15.261 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:15.261 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12619 2024-09-16T20:50:15.261 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.261 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:15.262 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.262 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:15.262 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774656+0000", 2024-09-16T20:50:15.262 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:15.262 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.262 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.262 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:15.262 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774656+0000", 2024-09-16T20:50:15.262 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:15.262 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.262 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.262 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:15.262 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774875+0000", 2024-09-16T20:50:15.262 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000218898 2024-09-16T20:50:15.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.263 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.263 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:15.263 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774879+0000", 2024-09-16T20:50:15.263 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0389999999999998e-06 2024-09-16T20:50:15.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.263 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.263 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:15.263 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774890+0000", 2024-09-16T20:50:15.263 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1250000000000001e-05 2024-09-16T20:50:15.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.263 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.263 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:15.263 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774897+0000", 2024-09-16T20:50:15.264 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7619999999999998e-06 2024-09-16T20:50:15.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.264 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.264 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:15.264 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840139+0000", 2024-09-16T20:50:15.264 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065242438 2024-09-16T20:50:15.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.264 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.264 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:15.264 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840186+0000", 2024-09-16T20:50:15.264 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6894999999999997e-05 2024-09-16T20:50:15.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.264 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.265 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:15.265 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840372+0000", 2024-09-16T20:50:15.265 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000185577 2024-09-16T20:50:15.265 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:15.265 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:15.265 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:15.265 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.265 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.265 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12620 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 997376~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:15.265 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.774908+0000", 2024-09-16T20:50:15.265 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.441951199999998, 2024-09-16T20:50:15.265 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065782754999999998, 2024-09-16T20:50:15.265 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:15.265 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:15.266 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:15.266 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:15.266 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:15.266 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:15.266 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12620 2024-09-16T20:50:15.266 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.266 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:15.266 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.266 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:15.266 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774908+0000", 2024-09-16T20:50:15.266 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:15.266 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.266 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.266 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:15.267 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774908+0000", 2024-09-16T20:50:15.267 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:15.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.267 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.267 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:15.267 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775102+0000", 2024-09-16T20:50:15.267 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000194098 2024-09-16T20:50:15.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.267 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.267 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:15.267 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775105+0000", 2024-09-16T20:50:15.267 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8739999999999999e-06 2024-09-16T20:50:15.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.267 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.268 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:15.268 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775111+0000", 2024-09-16T20:50:15.268 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9519999999999999e-06 2024-09-16T20:50:15.268 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.268 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.268 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:15.268 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775115+0000", 2024-09-16T20:50:15.268 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.002e-06 2024-09-16T20:50:15.268 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.268 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.268 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:15.268 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840409+0000", 2024-09-16T20:50:15.268 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065294195999999999 2024-09-16T20:50:15.268 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.269 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.269 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:15.269 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840507+0000", 2024-09-16T20:50:15.269 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7662000000000001e-05 2024-09-16T20:50:15.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.269 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.269 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:15.269 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840691+0000", 2024-09-16T20:50:15.269 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000183971 2024-09-16T20:50:15.269 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:15.269 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:15.269 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:15.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.269 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.270 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12621 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1002496~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:15.270 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775123+0000", 2024-09-16T20:50:15.270 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.441736504000001, 2024-09-16T20:50:15.270 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065829586999999995, 2024-09-16T20:50:15.270 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:15.270 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:15.270 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:15.270 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:15.270 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:15.270 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:15.270 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12621 2024-09-16T20:50:15.270 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.270 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:15.270 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.271 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:15.271 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775123+0000", 2024-09-16T20:50:15.271 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:15.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.271 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.271 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:15.271 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775123+0000", 2024-09-16T20:50:15.271 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:15.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.271 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.271 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:15.271 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775348+0000", 2024-09-16T20:50:15.271 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000225295 2024-09-16T20:50:15.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.272 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.272 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:15.272 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775352+0000", 2024-09-16T20:50:15.272 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0810000000000004e-06 2024-09-16T20:50:15.272 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.272 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.272 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:15.272 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775361+0000", 2024-09-16T20:50:15.272 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7399999999999993e-06 2024-09-16T20:50:15.272 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.272 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.272 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:15.272 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775367+0000", 2024-09-16T20:50:15.272 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5200000000000003e-06 2024-09-16T20:50:15.273 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.273 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.273 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:15.273 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840730+0000", 2024-09-16T20:50:15.273 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065362803999999997 2024-09-16T20:50:15.273 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.273 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.273 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:15.273 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840801+0000", 2024-09-16T20:50:15.273 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1006000000000004e-05 2024-09-16T20:50:15.273 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.273 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.273 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:15.274 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840952+0000", 2024-09-16T20:50:15.274 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015114100000000001 2024-09-16T20:50:15.274 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:15.274 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:15.274 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:15.274 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.274 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.274 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12622 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1007616~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:15.274 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775381+0000", 2024-09-16T20:50:15.274 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.441477670000001, 2024-09-16T20:50:15.274 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065822581000000005, 2024-09-16T20:50:15.274 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:15.274 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:15.274 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:15.275 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:15.275 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:15.275 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:15.275 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12622 2024-09-16T20:50:15.275 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.275 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:15.275 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.275 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:15.275 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775381+0000", 2024-09-16T20:50:15.275 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:15.275 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.275 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.275 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:15.275 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775381+0000", 2024-09-16T20:50:15.276 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:15.276 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.276 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.276 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:15.276 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775563+0000", 2024-09-16T20:50:15.276 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018153700000000001 2024-09-16T20:50:15.276 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.276 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.276 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:15.276 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775566+0000", 2024-09-16T20:50:15.276 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9620000000000002e-06 2024-09-16T20:50:15.276 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.276 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.277 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:15.277 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775572+0000", 2024-09-16T20:50:15.277 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8529999999999997e-06 2024-09-16T20:50:15.277 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.277 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.277 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:15.277 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775576+0000", 2024-09-16T20:50:15.277 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8920000000000002e-06 2024-09-16T20:50:15.277 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.277 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.277 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:15.277 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840990+0000", 2024-09-16T20:50:15.277 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065414599000000004 2024-09-16T20:50:15.277 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.278 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.278 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:15.278 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841068+0000", 2024-09-16T20:50:15.278 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8157000000000004e-05 2024-09-16T20:50:15.278 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.278 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.278 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:15.278 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841204+0000", 2024-09-16T20:50:15.278 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013558099999999999 2024-09-16T20:50:15.278 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:15.278 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:15.278 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:15.278 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.278 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.279 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12623 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1012736~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:15.279 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775583+0000", 2024-09-16T20:50:15.279 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.441276232, 2024-09-16T20:50:15.279 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065929087999999997, 2024-09-16T20:50:15.279 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:15.279 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:15.279 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:15.279 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:15.279 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:15.279 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:15.279 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12623 2024-09-16T20:50:15.279 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.279 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:15.279 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.280 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:15.280 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775583+0000", 2024-09-16T20:50:15.280 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:15.280 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.280 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.280 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:15.280 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775583+0000", 2024-09-16T20:50:15.280 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:15.280 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.280 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.280 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:15.280 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775804+0000", 2024-09-16T20:50:15.280 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00022116900000000001 2024-09-16T20:50:15.281 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.281 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.281 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:15.281 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775807+0000", 2024-09-16T20:50:15.281 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9529999999999999e-06 2024-09-16T20:50:15.281 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.281 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.281 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:15.281 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775814+0000", 2024-09-16T20:50:15.281 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7490000000000001e-06 2024-09-16T20:50:15.281 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.281 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.281 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:15.281 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775818+0000", 2024-09-16T20:50:15.282 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1999999999999996e-06 2024-09-16T20:50:15.282 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.282 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.282 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:15.282 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841267+0000", 2024-09-16T20:50:15.282 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065449127999999995 2024-09-16T20:50:15.282 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.282 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.282 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:15.282 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841344+0000", 2024-09-16T20:50:15.282 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6546999999999997e-05 2024-09-16T20:50:15.282 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.282 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.283 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:15.283 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841512+0000", 2024-09-16T20:50:15.283 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016834200000000001 2024-09-16T20:50:15.283 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:15.283 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:15.283 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:15.283 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.283 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.283 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12624 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1017856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:15.283 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:50:15.283 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.441033092000001, 2024-09-16T20:50:15.283 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065945866000000006, 2024-09-16T20:50:15.283 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:15.283 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:15.284 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:15.284 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:15.284 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:15.284 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:15.284 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12624 2024-09-16T20:50:15.284 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.284 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:15.284 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.284 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:15.284 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:50:15.284 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:15.284 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.284 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.285 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:15.285 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:50:15.285 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:15.285 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.285 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.285 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:15.285 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:50:15.285 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-16T20:50:15.285 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.285 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.285 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:15.285 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775828+0000", 2024-09-16T20:50:15.285 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3999999999999999e-06 2024-09-16T20:50:15.285 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.286 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.286 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:15.286 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775831+0000", 2024-09-16T20:50:15.286 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.478e-06 2024-09-16T20:50:15.286 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.286 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.286 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:15.286 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775834+0000", 2024-09-16T20:50:15.286 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9440000000000001e-06 2024-09-16T20:50:15.286 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.286 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.286 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:15.286 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841547+0000", 2024-09-16T20:50:15.286 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065712801000000001 2024-09-16T20:50:15.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.287 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.287 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:15.287 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841618+0000", 2024-09-16T20:50:15.287 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1151999999999999e-05 2024-09-16T20:50:15.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.287 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.287 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:15.287 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841772+0000", 2024-09-16T20:50:15.287 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000153721 2024-09-16T20:50:15.287 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:15.287 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:15.287 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:15.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.288 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.288 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12625 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1022976~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:15.288 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775840+0000", 2024-09-16T20:50:15.288 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.441019483000002, 2024-09-16T20:50:15.288 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066165451, 2024-09-16T20:50:15.288 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:15.288 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:15.288 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:15.288 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:15.288 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:15.288 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:15.288 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12625 2024-09-16T20:50:15.288 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.288 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:15.289 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.289 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:15.289 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775840+0000", 2024-09-16T20:50:15.289 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:15.289 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.289 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.289 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:15.289 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775840+0000", 2024-09-16T20:50:15.289 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:15.289 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.289 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.289 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:15.289 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776014+0000", 2024-09-16T20:50:15.290 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017441099999999999 2024-09-16T20:50:15.290 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.290 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.290 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:15.290 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776018+0000", 2024-09-16T20:50:15.290 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6509999999999999e-06 2024-09-16T20:50:15.290 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.290 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.290 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:15.290 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776026+0000", 2024-09-16T20:50:15.290 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.028e-06 2024-09-16T20:50:15.290 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.290 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.290 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:15.291 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776031+0000", 2024-09-16T20:50:15.291 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8400000000000002e-06 2024-09-16T20:50:15.291 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.291 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.291 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:15.291 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841801+0000", 2024-09-16T20:50:15.291 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065770472999999996 2024-09-16T20:50:15.291 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.291 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.291 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:15.291 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841847+0000", 2024-09-16T20:50:15.291 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5558e-05 2024-09-16T20:50:15.291 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.291 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.292 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:15.292 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842005+0000", 2024-09-16T20:50:15.292 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015849000000000001 2024-09-16T20:50:15.292 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:15.292 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:15.292 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:15.292 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.292 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.292 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12626 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1028096~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:15.292 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776041+0000", 2024-09-16T20:50:15.292 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.440818184000001, 2024-09-16T20:50:15.292 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066225260999999994, 2024-09-16T20:50:15.292 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:15.293 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:15.293 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:15.293 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:15.293 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:15.293 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:15.293 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12626 2024-09-16T20:50:15.293 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.293 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:15.293 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.293 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:15.293 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776041+0000", 2024-09-16T20:50:15.293 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:15.293 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.293 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.294 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:15.294 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776041+0000", 2024-09-16T20:50:15.294 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:15.294 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.294 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.294 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:15.294 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776263+0000", 2024-09-16T20:50:15.294 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00022193899999999999 2024-09-16T20:50:15.294 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.294 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.294 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:15.294 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776268+0000", 2024-09-16T20:50:15.294 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8049999999999997e-06 2024-09-16T20:50:15.294 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.295 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.295 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:15.295 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776280+0000", 2024-09-16T20:50:15.295 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2635e-05 2024-09-16T20:50:15.295 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.295 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.295 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:15.295 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776288+0000", 2024-09-16T20:50:15.295 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8199999999999997e-06 2024-09-16T20:50:15.295 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.295 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.295 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:15.295 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842041+0000", 2024-09-16T20:50:15.295 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065752506000000002 2024-09-16T20:50:15.296 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.296 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.296 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:15.296 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842123+0000", 2024-09-16T20:50:15.296 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2393999999999998e-05 2024-09-16T20:50:15.296 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.296 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.296 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:15.296 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842266+0000", 2024-09-16T20:50:15.296 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014316200000000001 2024-09-16T20:50:15.296 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:15.296 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:15.296 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:15.296 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.297 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.297 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12627 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1033216~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:15.297 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776301+0000", 2024-09-16T20:50:15.297 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.440558224, 2024-09-16T20:50:15.297 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066187814999999997, 2024-09-16T20:50:15.297 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:15.297 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:15.297 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:15.297 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:15.297 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:15.297 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:15.297 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12627 2024-09-16T20:50:15.297 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.297 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:15.298 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.298 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:15.298 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776301+0000", 2024-09-16T20:50:15.298 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:15.298 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.298 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.298 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:15.298 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776301+0000", 2024-09-16T20:50:15.298 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:15.298 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.298 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.298 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:15.298 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776473+0000", 2024-09-16T20:50:15.298 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017222400000000001 2024-09-16T20:50:15.299 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.299 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.299 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:15.299 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776477+0000", 2024-09-16T20:50:15.299 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7019999999999999e-06 2024-09-16T20:50:15.299 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.299 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.299 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:15.299 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776486+0000", 2024-09-16T20:50:15.299 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2119999999999992e-06 2024-09-16T20:50:15.299 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.299 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.299 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:15.299 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776492+0000", 2024-09-16T20:50:15.300 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8089999999999998e-06 2024-09-16T20:50:15.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.300 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.300 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:15.300 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842301+0000", 2024-09-16T20:50:15.300 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065809042999999998 2024-09-16T20:50:15.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.300 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.300 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:15.300 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842362+0000", 2024-09-16T20:50:15.300 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1260000000000006e-05 2024-09-16T20:50:15.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.301 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.301 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:15.301 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842489+0000", 2024-09-16T20:50:15.301 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012656500000000001 2024-09-16T20:50:15.301 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:15.301 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:15.301 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:15.301 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.302 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.302 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12628 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1038336~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:15.302 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776502+0000", 2024-09-16T20:50:15.302 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.440357578, 2024-09-16T20:50:15.302 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066283426000000006, 2024-09-16T20:50:15.302 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:15.302 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:15.302 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:15.302 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:15.302 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:15.302 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:15.302 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12628 2024-09-16T20:50:15.302 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.302 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:15.303 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.303 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:15.303 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776502+0000", 2024-09-16T20:50:15.303 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:15.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.303 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.303 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:15.303 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776502+0000", 2024-09-16T20:50:15.303 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:15.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.303 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.303 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:15.303 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776716+0000", 2024-09-16T20:50:15.304 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000214596 2024-09-16T20:50:15.304 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.304 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.304 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:15.304 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776723+0000", 2024-09-16T20:50:15.304 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4000000000000003e-06 2024-09-16T20:50:15.304 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.304 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.304 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:15.304 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776731+0000", 2024-09-16T20:50:15.304 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6129999999999998e-06 2024-09-16T20:50:15.304 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.304 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.304 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:15.305 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776737+0000", 2024-09-16T20:50:15.305 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1970000000000001e-06 2024-09-16T20:50:15.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.305 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:15.305 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842532+0000", 2024-09-16T20:50:15.305 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065795127999999994 2024-09-16T20:50:15.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.305 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:15.305 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842599+0000", 2024-09-16T20:50:15.305 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6711000000000006e-05 2024-09-16T20:50:15.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.306 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:15.306 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842785+0000", 2024-09-16T20:50:15.306 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018578099999999999 2024-09-16T20:50:15.306 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:15.306 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:15.306 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:15.306 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.306 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.306 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12629 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1043456~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:15.306 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776748+0000", 2024-09-16T20:50:15.306 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.440111201000001, 2024-09-16T20:50:15.306 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066303788000000002, 2024-09-16T20:50:15.307 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:15.307 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:15.307 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:15.307 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:15.307 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:15.307 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:15.307 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12629 2024-09-16T20:50:15.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.307 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:15.307 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.307 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:15.307 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776748+0000", 2024-09-16T20:50:15.307 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:15.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.308 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.308 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:15.308 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776748+0000", 2024-09-16T20:50:15.308 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:15.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.308 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.308 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:15.308 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776934+0000", 2024-09-16T20:50:15.308 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018598399999999999 2024-09-16T20:50:15.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.308 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.308 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:15.308 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776939+0000", 2024-09-16T20:50:15.308 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1000000000000003e-06 2024-09-16T20:50:15.309 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.309 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.309 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:15.309 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776954+0000", 2024-09-16T20:50:15.309 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5221e-05 2024-09-16T20:50:15.309 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.309 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.309 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:15.309 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776962+0000", 2024-09-16T20:50:15.309 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.554e-06 2024-09-16T20:50:15.309 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.309 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.309 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:15.309 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842813+0000", 2024-09-16T20:50:15.310 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065851006000000004 2024-09-16T20:50:15.310 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.310 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.310 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:15.310 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842886+0000", 2024-09-16T20:50:15.310 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3663000000000003e-05 2024-09-16T20:50:15.310 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:15.310 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:15.310 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:15.310 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.843052+0000", 2024-09-16T20:50:15.310 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016526 2024-09-16T20:50:15.310 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:15.310 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:15.310 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:15.311 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:15.311 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:15.311 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:15.312 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:50:15.312 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:50:15.313 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:50:15.313 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:15.310+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:15.324 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:50:15.324 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:50:15.341 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:50:15.341 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:50:15.357 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:50:15.358 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:50:15.374 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:50:15.374 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:50:15.391 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:50:15.391 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:50:15.412 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:15.411+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:15.460 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:15.460 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:15.469 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:50:15.513 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:15.512+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:15.576 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:15.577 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:15.586 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:50:15.590 INFO:teuthology.orchestra.run.smithi139.stdout:5720 write 0xdd89400 thru 0xdd8dc22 (0x4823 bytes) 2024-09-16T20:50:15.597 INFO:teuthology.orchestra.run.smithi139.stdout:5722 read 0xad54800 thru 0xad575a5 (0x2da6 bytes) 2024-09-16T20:50:15.599 INFO:teuthology.orchestra.run.smithi139.stdout:5723 read 0x41afc00 thru 0x41bbdfb (0xc1fc bytes) 2024-09-16T20:50:15.599 INFO:teuthology.orchestra.run.smithi139.stdout:5724 read 0x5b1b000 thru 0x5b1c930 (0x1931 bytes) 2024-09-16T20:50:15.601 INFO:teuthology.orchestra.run.smithi139.stdout:5725 flatten 2024-09-16T20:50:15.628 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:15.626+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:15.716 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:15.715+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:15.725 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T20:50:15.805 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:15.805 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:15.816 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:15.815+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:15.817 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:50:15.917 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:15.916+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:15.993 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:15.993 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:16.008 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:50:16.050 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:16.049+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:16.118 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:16.117+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:16.200 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:16.200 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:16.203 INFO:teuthology.orchestra.run.smithi139.stdout:5729 trunc from 0xdd8de00 to 0xd4dda00 2024-09-16T20:50:16.209 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:50:16.218 INFO:teuthology.orchestra.run.smithi139.stdout:5730 read 0x2f95e00 thru 0x2fa2ec0 (0xd0c1 bytes) 2024-09-16T20:50:16.219 INFO:teuthology.orchestra.run.smithi139.stdout:5731 read 0x91bec00 thru 0x91cd0c0 (0xe4c1 bytes) 2024-09-16T20:50:16.219 INFO:teuthology.orchestra.run.smithi139.stdout:5735 read 0xd2f7600 thru 0xd302b27 (0xb528 bytes) 2024-09-16T20:50:16.219 INFO:teuthology.orchestra.run.smithi139.stdout:5737 read 0x581b400 thru 0x581d462 (0x2063 bytes) 2024-09-16T20:50:16.219 INFO:teuthology.orchestra.run.smithi139.stdout:5743 punch from 0x8ff2600 to 0x9001200, (0xec00 bytes) 2024-09-16T20:50:16.221 INFO:teuthology.orchestra.run.smithi139.stdout:5744 write 0x5dd1200 thru 0x5de05e8 (0xf3e9 bytes) 2024-09-16T20:50:16.221 INFO:teuthology.orchestra.run.smithi139.stdout:5745 write 0x3687200 thru 0x368ba2d (0x482e bytes) 2024-09-16T20:50:16.223 INFO:teuthology.orchestra.run.smithi139.stdout:5746 trunc from 0xd4dda00 to 0xac8e800 2024-09-16T20:50:16.246 INFO:teuthology.orchestra.run.smithi139.stdout:5747 write 0x2b41000 thru 0x2b4dec6 (0xcec7 bytes) 2024-09-16T20:50:16.251 INFO:teuthology.orchestra.run.smithi139.stdout:5748 punch from 0x6be1600 to 0x6beda00, (0xc400 bytes) 2024-09-16T20:50:16.252 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:16.251+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:16.260 INFO:teuthology.orchestra.run.smithi139.stdout:5750 trunc from 0xac8e800 to 0x49dda00 2024-09-16T20:50:16.299 INFO:teuthology.orchestra.run.smithi139.stdout:5751 punch from 0x1420c00 to 0x1422600, (0x1a00 bytes) 2024-09-16T20:50:16.308 INFO:teuthology.orchestra.run.smithi139.stdout:5752 trunc from 0x49dda00 to 0x31cbe00 2024-09-16T20:50:16.320 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:16.318+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:16.329 INFO:teuthology.orchestra.run.smithi139.stdout:5754 read 0x7c2400 thru 0x7d1c64 (0xf865 bytes) 2024-09-16T20:50:16.329 INFO:teuthology.orchestra.run.smithi139.stdout:5755 punch from 0x518e00 to 0x51b400, (0x2600 bytes) 2024-09-16T20:50:16.331 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:16.331 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:16.332 INFO:teuthology.orchestra.run.smithi139.stdout:5758 trunc from 0x31cbe00 to 0x55da00 2024-09-16T20:50:16.342 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:50:16.392 INFO:teuthology.orchestra.run.smithi139.stdout:5760 write 0x9d4d400 thru 0x9d579a4 (0xa5a5 bytes) 2024-09-16T20:50:16.397 INFO:teuthology.orchestra.run.smithi139.stdout:5761 write 0x1f1800 thru 0x1ff13f (0xd940 bytes) 2024-09-16T20:50:16.398 INFO:teuthology.orchestra.run.smithi139.stdout:5763 read 0x858fa00 thru 0x858fff4 (0x5f5 bytes) 2024-09-16T20:50:16.419 INFO:teuthology.orchestra.run.smithi139.stdout:5765 write 0xde47600 thru 0xde484ce (0xecf bytes) 2024-09-16T20:50:16.420 INFO:teuthology.orchestra.run.smithi139.stdout:5767 punch from 0x75e2600 to 0x75e4600, (0x2000 bytes) 2024-09-16T20:50:16.421 INFO:teuthology.orchestra.run.smithi139.stdout:5768 punch from 0xc152600 to 0xc153800, (0x1200 bytes) 2024-09-16T20:50:16.421 INFO:teuthology.orchestra.run.smithi139.stdout:5770 write 0xc0c2400 thru 0xc0ceb58 (0xc759 bytes) 2024-09-16T20:50:16.422 INFO:teuthology.orchestra.run.smithi139.stdout:5772 trunc from 0xde48600 to 0xd132000 2024-09-16T20:50:16.426 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:16.425+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:16.442 INFO:teuthology.orchestra.run.smithi139.stdout:5776 write 0x66a9a00 thru 0x66b3d32 (0xa333 bytes) 2024-09-16T20:50:16.444 INFO:teuthology.orchestra.run.smithi139.stdout:5777 write 0x6248400 thru 0x624b511 (0x3112 bytes) 2024-09-16T20:50:16.444 INFO:teuthology.orchestra.run.smithi139.stdout:5778 read 0x77c7e00 thru 0x77cb7b0 (0x39b1 bytes) 2024-09-16T20:50:16.444 INFO:teuthology.orchestra.run.smithi139.stdout:5780 punch from 0x5de8e00 to 0x5df5c00, (0xce00 bytes) 2024-09-16T20:50:16.447 INFO:teuthology.orchestra.run.smithi139.stdout:5784 write 0x7b39800 thru 0x7b42cc1 (0x94c2 bytes) 2024-09-16T20:50:16.447 INFO:teuthology.orchestra.run.smithi139.stdout:5785 write 0x96da400 thru 0x96e0eb1 (0x6ab2 bytes) 2024-09-16T20:50:16.447 INFO:teuthology.orchestra.run.smithi139.stdout:5787 write 0x568b600 thru 0x569af40 (0xf941 bytes) 2024-09-16T20:50:16.448 INFO:teuthology.orchestra.run.smithi139.stdout:5788 write 0x21eda00 thru 0x21ee994 (0xf95 bytes) 2024-09-16T20:50:16.448 INFO:teuthology.orchestra.run.smithi139.stdout:5792 write 0x424da00 thru 0x4253b0a (0x610b bytes) 2024-09-16T20:50:16.448 INFO:teuthology.orchestra.run.smithi139.stdout:5797 write 0xac9a200 thru 0xaca1e26 (0x7c27 bytes) 2024-09-16T20:50:16.448 INFO:teuthology.orchestra.run.smithi139.stdout:5801 write 0x4c83c00 thru 0x4c8c796 (0x8b97 bytes) 2024-09-16T20:50:16.448 INFO:teuthology.orchestra.run.smithi139.stdout:5802 read 0xd115000 thru 0xd11d3d0 (0x83d1 bytes) 2024-09-16T20:50:16.448 INFO:teuthology.orchestra.run.smithi139.stdout:5803 read 0xcb23e00 thru 0xcb30d69 (0xcf6a bytes) 2024-09-16T20:50:16.448 INFO:teuthology.orchestra.run.smithi139.stdout:5805 clone 42 order 25 su 4194304 sc 10 2024-09-16T20:50:16.504 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:16.504 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:16.512 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:50:16.553 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:16.552+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:16.622 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:16.620+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:16.658 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:16.659 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:16.667 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:50:16.694 INFO:teuthology.orchestra.run.smithi029.stderr:nodeep-scrub is unset 2024-09-16T20:50:16.722 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:16.720+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:16.805 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:16.805 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:16.814 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:50:16.823 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:16.822+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:16.921 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:16.921 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:16.924 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:16.923+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:16.930 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:50:17.026 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:17.024+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:17.077 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:17.077 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:17.086 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:50:17.129 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:17.128+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:17.193 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:17.193 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:17.202 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:50:17.243 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:17.242+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:17.314 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:17.315 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:17.323 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:50:17.366 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:17.365+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:17.403 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T20:50:17.429 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:17.427+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:17.429 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:17.429 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:17.438 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:50:17.529 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:50:17.529 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:50:17.530 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:17.529+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:17.545 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:50:17.545 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:50:17.561 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:50:17.562 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:50:17.578 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:50:17.578 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:50:17.587 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:17.587 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:50:17.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:50:17.587 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:50:17.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.587 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:3088 3.1e 3:79e5ec01:::rbd_header.110e891a9b2b:head [watch watch cookie 140213437171488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:50:17.587 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:50:17.588 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 113.42179515300001, 2024-09-16T20:50:17.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.11895186200000001, 2024-09-16T20:50:17.588 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:17.588 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:17.588 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:17.588 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:17.588 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:50:17.588 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:17.588 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3088 2024-09-16T20:50:17.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.588 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:17.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:17.589 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:50:17.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:17.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:17.589 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:50:17.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:17.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:17.589 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163846+0000", 2024-09-16T20:50:17.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6799999999999999e-07 2024-09-16T20:50:17.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.590 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:17.590 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163850+0000", 2024-09-16T20:50:17.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.929e-06 2024-09-16T20:50:17.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.590 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:17.590 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163862+0000", 2024-09-16T20:50:17.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1972e-05 2024-09-16T20:50:17.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.590 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:17.590 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163867+0000", 2024-09-16T20:50:17.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.451e-06 2024-09-16T20:50:17.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.591 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:17.591 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163912+0000", 2024-09-16T20:50:17.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4456000000000003e-05 2024-09-16T20:50:17.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.591 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:17.591 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163962+0000", 2024-09-16T20:50:17.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0578000000000003e-05 2024-09-16T20:50:17.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.591 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-16T20:50:17.591 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164058+0000", 2024-09-16T20:50:17.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5934999999999995e-05 2024-09-16T20:50:17.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.592 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:50:17.592 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164570+0000", 2024-09-16T20:50:17.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00051241100000000005 2024-09-16T20:50:17.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.592 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:17.592 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282629+0000", 2024-09-16T20:50:17.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.118058986 2024-09-16T20:50:17.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.592 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:50:17.592 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282725+0000", 2024-09-16T20:50:17.593 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5063999999999996e-05 2024-09-16T20:50:17.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.593 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:17.593 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282797+0000", 2024-09-16T20:50:17.593 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2212000000000006e-05 2024-09-16T20:50:17.593 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:17.593 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:17.593 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:17.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.593 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:8039 3.1e 3.ac44ff9e (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:50:17.594 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:50:17.594 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 87.877967803999994, 2024-09-16T20:50:17.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0053310449999999, 2024-09-16T20:50:17.594 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:17.594 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:17.594 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:17.594 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:17.594 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:50:17.594 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:17.594 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8039 2024-09-16T20:50:17.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.594 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:17.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.594 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:17.594 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:50:17.595 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:17.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.595 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:17.595 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:50:17.595 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:17.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.595 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:17.595 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707674+0000", 2024-09-16T20:50:17.595 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6530000000000001e-06 2024-09-16T20:50:17.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.596 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:17.596 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707681+0000", 2024-09-16T20:50:17.596 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9639999999999997e-06 2024-09-16T20:50:17.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.596 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.596 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:17.596 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707700+0000", 2024-09-16T20:50:17.596 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8674000000000001e-05 2024-09-16T20:50:17.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.596 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.596 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:17.596 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707708+0000", 2024-09-16T20:50:17.596 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6810000000000004e-06 2024-09-16T20:50:17.597 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.597 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.597 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:17.597 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711293+0000", 2024-09-16T20:50:17.597 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035848189999999999 2024-09-16T20:50:17.597 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.597 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.597 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:17.597 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711301+0000", 2024-09-16T20:50:17.597 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4730000000000003e-06 2024-09-16T20:50:17.597 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.597 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.598 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:17.598 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.712990+0000", 2024-09-16T20:50:17.598 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.001689348 2024-09-16T20:50:17.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.598 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:17.598 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713003+0000", 2024-09-16T20:50:17.598 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3433e-05 2024-09-16T20:50:17.598 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:17.598 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:17.598 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:17.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.599 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:8040 3.1e 3.ac44ff9e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:50:17.599 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:50:17.599 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 87.877907409000002, 2024-09-16T20:50:17.599 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0053410780000001, 2024-09-16T20:50:17.599 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:17.599 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:17.599 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:17.599 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:17.599 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:50:17.599 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:17.599 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8040 2024-09-16T20:50:17.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.600 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:17.600 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.600 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:17.600 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:50:17.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:17.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.600 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.600 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:17.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:50:17.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:17.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.601 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:17.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:50:17.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8100000000000003e-07 2024-09-16T20:50:17.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.601 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:17.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707737+0000", 2024-09-16T20:50:17.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5020000000000001e-06 2024-09-16T20:50:17.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.602 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:17.602 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707745+0000", 2024-09-16T20:50:17.602 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2169999999999994e-06 2024-09-16T20:50:17.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.602 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:17.602 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707751+0000", 2024-09-16T20:50:17.602 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.263e-06 2024-09-16T20:50:17.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.602 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:17.602 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711336+0000", 2024-09-16T20:50:17.602 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035848360000000001 2024-09-16T20:50:17.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.603 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:17.603 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711340+0000", 2024-09-16T20:50:17.603 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1309999999999999e-06 2024-09-16T20:50:17.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.603 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:17.603 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713060+0000", 2024-09-16T20:50:17.603 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0017195059999999 2024-09-16T20:50:17.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.603 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:17.603 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713074+0000", 2024-09-16T20:50:17.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4042e-05 2024-09-16T20:50:17.604 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:17.604 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:17.604 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:17.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.604 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4927.0:4487 3.1e 3.ac44ff9e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:50:17.604 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:50:17.604 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 87.877387206999998, 2024-09-16T20:50:17.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0048840880000001, 2024-09-16T20:50:17.604 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:17.604 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:17.604 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:17.605 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:17.605 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4927", 2024-09-16T20:50:17.605 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:17.605 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4487 2024-09-16T20:50:17.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.605 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:17.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:17.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:50:17.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:17.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:17.606 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:50:17.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:17.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.606 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:17.606 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708254+0000", 2024-09-16T20:50:17.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1800000000000005e-07 2024-09-16T20:50:17.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.606 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:17.606 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708258+0000", 2024-09-16T20:50:17.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8689999999999997e-06 2024-09-16T20:50:17.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.607 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:17.607 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708267+0000", 2024-09-16T20:50:17.607 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9590000000000007e-06 2024-09-16T20:50:17.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.607 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:17.607 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708274+0000", 2024-09-16T20:50:17.607 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9970000000000001e-06 2024-09-16T20:50:17.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.607 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:17.607 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711366+0000", 2024-09-16T20:50:17.607 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030919530000000001 2024-09-16T20:50:17.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.608 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:17.608 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711372+0000", 2024-09-16T20:50:17.608 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9410000000000004e-06 2024-09-16T20:50:17.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.608 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:17.608 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713126+0000", 2024-09-16T20:50:17.608 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0017537249999999 2024-09-16T20:50:17.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.608 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:17.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713137+0000", 2024-09-16T20:50:17.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0825999999999999e-05 2024-09-16T20:50:17.609 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:17.609 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:17.609 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:17.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.610 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5089.0:11703 3.1b 3:dcd4684e:::rbd_data.110e747f0779.000000000000001b:head [set-alloc-hint object_size 262144 write_size 262144,write 139264~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:50:17.610 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:50:17.610 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 73.447651433000004, 2024-09-16T20:50:17.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.084231385000000006, 2024-09-16T20:50:17.610 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:17.610 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:17.610 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:17.610 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:17.610 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5089", 2024-09-16T20:50:17.611 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:50:17.611 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 11703 2024-09-16T20:50:17.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.611 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:17.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.611 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:17.611 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:50:17.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:17.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.611 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:17.611 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:50:17.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:17.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.612 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:17.612 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:50:17.612 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2600000000000001e-07 2024-09-16T20:50:17.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.612 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:17.612 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137998+0000", 2024-09-16T20:50:17.612 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9549999999999998e-06 2024-09-16T20:50:17.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.612 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:17.612 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138008+0000", 2024-09-16T20:50:17.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3789999999999995e-06 2024-09-16T20:50:17.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.613 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:17.613 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138013+0000", 2024-09-16T20:50:17.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2209999999999996e-06 2024-09-16T20:50:17.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.614 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:17.614 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138054+0000", 2024-09-16T20:50:17.614 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1022999999999997e-05 2024-09-16T20:50:17.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.615 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.615 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:17.615 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138124+0000", 2024-09-16T20:50:17.615 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0234999999999994e-05 2024-09-16T20:50:17.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.616 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.616 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:17.616 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138197+0000", 2024-09-16T20:50:17.616 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2451000000000001e-05 2024-09-16T20:50:17.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.616 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.616 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:17.616 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222010+0000", 2024-09-16T20:50:17.617 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.083813433000000007 2024-09-16T20:50:17.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.617 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:50:17.617 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222163+0000", 2024-09-16T20:50:17.617 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015274599999999999 2024-09-16T20:50:17.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.617 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:50:17.617 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222197+0000", 2024-09-16T20:50:17.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3683000000000003e-05 2024-09-16T20:50:17.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:17.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222220+0000", 2024-09-16T20:50:17.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3433000000000001e-05 2024-09-16T20:50:17.618 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:17.619 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:17.619 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:17.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.619 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.5089.0:11702 3.15 e73/64 3:acf33c8b:::rbd_data.110e747f0779.000000000000001a:head v 73'132, mlcod=73'130)", 2024-09-16T20:50:17.619 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:50:17.619 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 73.447384103999994, 2024-09-16T20:50:17.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.084079763000000002, 2024-09-16T20:50:17.619 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:17.619 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:17.619 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:17.619 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:17.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.619 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:17.620 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:50:17.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:17.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.620 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:17.620 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:50:17.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:17.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.620 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:17.620 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:50:17.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:50:17.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.621 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:17.621 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138263+0000", 2024-09-16T20:50:17.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1e-06 2024-09-16T20:50:17.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.621 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:17.621 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221589+0000", 2024-09-16T20:50:17.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.083326002999999996 2024-09-16T20:50:17.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.621 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:17.621 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221614+0000", 2024-09-16T20:50:17.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5593999999999999e-05 2024-09-16T20:50:17.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.622 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:17.622 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221714+0000", 2024-09-16T20:50:17.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9889000000000005e-05 2024-09-16T20:50:17.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.622 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:17.622 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221727+0000", 2024-09-16T20:50:17.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3125999999999999e-05 2024-09-16T20:50:17.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:50:17.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222294+0000", 2024-09-16T20:50:17.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00056712199999999998 2024-09-16T20:50:17.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:17.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222336+0000", 2024-09-16T20:50:17.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1470000000000001e-05 2024-09-16T20:50:17.623 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:17.623 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:17.623 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:17.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.623 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5138.0:9358 3.c 3:33996a43:::rbd_data.110e6372565d.0000000000000007:head [read 1343488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T20:50:17.624 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:50:17.624 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 49.733108694000002, 2024-09-16T20:50:17.624 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0167493569999999, 2024-09-16T20:50:17.624 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:17.624 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:17.624 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:17.624 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:17.624 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5138", 2024-09-16T20:50:17.624 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:50:17.624 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9358 2024-09-16T20:50:17.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.624 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:17.624 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.624 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:17.625 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:50:17.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:17.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.625 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:17.625 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:50:17.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:17.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.625 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:17.625 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852533+0000", 2024-09-16T20:50:17.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.646e-06 2024-09-16T20:50:17.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.625 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:17.626 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852539+0000", 2024-09-16T20:50:17.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3389999999999998e-06 2024-09-16T20:50:17.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.626 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:17.626 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852552+0000", 2024-09-16T20:50:17.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.271e-05 2024-09-16T20:50:17.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.626 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:17.626 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852563+0000", 2024-09-16T20:50:17.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1139e-05 2024-09-16T20:50:17.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.627 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:17.627 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852883+0000", 2024-09-16T20:50:17.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00032010600000000001 2024-09-16T20:50:17.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.627 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:17.627 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852890+0000", 2024-09-16T20:50:17.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7669999999999998e-06 2024-09-16T20:50:17.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.627 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:17.627 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869059+0000", 2024-09-16T20:50:17.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.016169353 2024-09-16T20:50:17.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:17.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869178+0000", 2024-09-16T20:50:17.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118193 2024-09-16T20:50:17.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:17.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869281+0000", 2024-09-16T20:50:17.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010310399999999999 2024-09-16T20:50:17.628 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:17.628 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:17.628 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:17.629 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.629 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.629 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5138.0:9382 3.c 3:33996a43:::rbd_data.110e6372565d.0000000000000007:head [read 0~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T20:50:17.629 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:50:17.629 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 49.733057817000002, 2024-09-16T20:50:17.629 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.017136034, 2024-09-16T20:50:17.629 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:17.629 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:17.629 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:17.629 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:17.629 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5138", 2024-09-16T20:50:17.629 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:50:17.629 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9382 2024-09-16T20:50:17.629 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.630 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:17.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:17.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:50:17.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:17.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:17.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:50:17.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:17.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:17.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852583+0000", 2024-09-16T20:50:17.631 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.51e-07 2024-09-16T20:50:17.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.631 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.631 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:17.631 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852586+0000", 2024-09-16T20:50:17.631 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0280000000000001e-06 2024-09-16T20:50:17.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.631 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.631 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:17.631 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852595+0000", 2024-09-16T20:50:17.631 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2799999999999992e-06 2024-09-16T20:50:17.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.631 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.631 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:17.631 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852601+0000", 2024-09-16T20:50:17.632 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3810000000000004e-06 2024-09-16T20:50:17.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.632 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.632 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:17.632 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852921+0000", 2024-09-16T20:50:17.632 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00031979699999999997 2024-09-16T20:50:17.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.632 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.632 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:17.632 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852930+0000", 2024-09-16T20:50:17.632 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6489999999999997e-06 2024-09-16T20:50:17.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.632 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.633 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:17.633 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869434+0000", 2024-09-16T20:50:17.633 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0165038829999999 2024-09-16T20:50:17.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.633 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.633 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:17.633 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869495+0000", 2024-09-16T20:50:17.633 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1509999999999999e-05 2024-09-16T20:50:17.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.633 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.633 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:17.633 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869718+0000", 2024-09-16T20:50:17.633 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000222955 2024-09-16T20:50:17.633 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:17.634 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:17.634 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:17.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.634 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9969 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 632832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:17.634 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642045+0000", 2024-09-16T20:50:17.634 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 26.94359536, 2024-09-16T20:50:17.634 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076903192999999995, 2024-09-16T20:50:17.634 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:17.634 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:17.634 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:17.634 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:17.634 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:17.634 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:17.635 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9969 2024-09-16T20:50:17.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.635 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:17.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.635 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:17.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642045+0000", 2024-09-16T20:50:17.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:17.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.635 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:17.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642045+0000", 2024-09-16T20:50:17.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:17.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.635 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:17.636 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642045+0000", 2024-09-16T20:50:17.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5100000000000001e-07 2024-09-16T20:50:17.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.636 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:17.636 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642047+0000", 2024-09-16T20:50:17.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8679999999999999e-06 2024-09-16T20:50:17.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.636 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:17.636 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642053+0000", 2024-09-16T20:50:17.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1720000000000004e-06 2024-09-16T20:50:17.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.637 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:17.637 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642060+0000", 2024-09-16T20:50:17.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0339999999999999e-06 2024-09-16T20:50:17.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.637 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:17.637 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.718562+0000", 2024-09-16T20:50:17.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076501969000000003 2024-09-16T20:50:17.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.637 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:17.638 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.718643+0000", 2024-09-16T20:50:17.638 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0539999999999998e-05 2024-09-16T20:50:17.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.638 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.638 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:17.638 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.718948+0000", 2024-09-16T20:50:17.638 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000305259 2024-09-16T20:50:17.638 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:17.638 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:17.638 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:17.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.638 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.639 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9970 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 637952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:17.639 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642067+0000", 2024-09-16T20:50:17.639 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 26.94357338, 2024-09-16T20:50:17.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077174928000000004, 2024-09-16T20:50:17.639 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:17.639 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:17.639 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:17.639 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:17.639 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:17.639 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:17.639 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9970 2024-09-16T20:50:17.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.639 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:17.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.640 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:17.640 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642067+0000", 2024-09-16T20:50:17.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:17.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.640 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:17.640 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642067+0000", 2024-09-16T20:50:17.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:17.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.640 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:17.640 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642067+0000", 2024-09-16T20:50:17.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2499999999999995e-07 2024-09-16T20:50:17.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:17.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642070+0000", 2024-09-16T20:50:17.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3199999999999998e-06 2024-09-16T20:50:17.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:17.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642076+0000", 2024-09-16T20:50:17.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.122e-06 2024-09-16T20:50:17.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:17.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642081+0000", 2024-09-16T20:50:17.642 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0869999999999997e-06 2024-09-16T20:50:17.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.642 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:17.642 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.718980+0000", 2024-09-16T20:50:17.642 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076899482000000005 2024-09-16T20:50:17.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.642 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:17.642 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719085+0000", 2024-09-16T20:50:17.642 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010446399999999999 2024-09-16T20:50:17.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.642 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:17.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719242+0000", 2024-09-16T20:50:17.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000156928 2024-09-16T20:50:17.643 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:17.643 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:17.643 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:17.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.643 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9971 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 643072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:17.643 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642102+0000", 2024-09-16T20:50:17.643 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 26.943537673000002, 2024-09-16T20:50:17.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077421016999999995, 2024-09-16T20:50:17.644 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:17.644 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:17.644 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:17.644 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:17.644 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:17.645 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:17.645 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9971 2024-09-16T20:50:17.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.645 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:17.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:17.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642102+0000", 2024-09-16T20:50:17.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:17.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:17.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642102+0000", 2024-09-16T20:50:17.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:17.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.646 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:17.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642103+0000", 2024-09-16T20:50:17.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7399999999999998e-07 2024-09-16T20:50:17.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.647 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:17.647 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642105+0000", 2024-09-16T20:50:17.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2649999999999999e-06 2024-09-16T20:50:17.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.647 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:17.647 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642112+0000", 2024-09-16T20:50:17.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.294e-06 2024-09-16T20:50:17.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:17.648 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642118+0000", 2024-09-16T20:50:17.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4500000000000003e-06 2024-09-16T20:50:17.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:17.648 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719285+0000", 2024-09-16T20:50:17.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077166883000000006 2024-09-16T20:50:17.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:17.648 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719368+0000", 2024-09-16T20:50:17.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2811000000000004e-05 2024-09-16T20:50:17.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:17.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719523+0000", 2024-09-16T20:50:17.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015584 2024-09-16T20:50:17.649 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:17.649 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:17.649 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:17.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.649 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9972 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 648192~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:17.649 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642125+0000", 2024-09-16T20:50:17.649 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 26.943514726, 2024-09-16T20:50:17.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077710324999999997, 2024-09-16T20:50:17.650 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:17.650 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:17.650 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:17.650 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:17.650 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:17.650 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:17.650 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9972 2024-09-16T20:50:17.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.650 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:17.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.650 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:17.650 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642125+0000", 2024-09-16T20:50:17.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:17.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.651 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:17.651 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642125+0000", 2024-09-16T20:50:17.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:17.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.651 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:17.652 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642126+0000", 2024-09-16T20:50:17.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1899999999999998e-07 2024-09-16T20:50:17.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.652 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:17.652 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642128+0000", 2024-09-16T20:50:17.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8619999999999999e-06 2024-09-16T20:50:17.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.652 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:17.652 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642134+0000", 2024-09-16T20:50:17.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5289999999999997e-06 2024-09-16T20:50:17.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:17.653 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642139+0000", 2024-09-16T20:50:17.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1250000000000001e-06 2024-09-16T20:50:17.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:17.653 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719570+0000", 2024-09-16T20:50:17.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077430516000000005 2024-09-16T20:50:17.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:17.653 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719655+0000", 2024-09-16T20:50:17.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5500000000000005e-05 2024-09-16T20:50:17.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:17.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719836+0000", 2024-09-16T20:50:17.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018037399999999999 2024-09-16T20:50:17.654 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:17.654 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:17.654 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:17.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.654 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9973 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 653312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:17.654 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:50:17.654 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 26.943494116, 2024-09-16T20:50:17.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077905262000000003, 2024-09-16T20:50:17.654 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:17.655 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:17.655 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:17.655 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:17.655 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:17.655 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:17.655 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9973 2024-09-16T20:50:17.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.655 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:17.655 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.655 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:17.655 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:50:17.655 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:17.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.656 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:17.656 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:50:17.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:17.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.656 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:17.656 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:50:17.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:50:17.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.656 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:17.656 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642148+0000", 2024-09-16T20:50:17.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0219999999999999e-06 2024-09-16T20:50:17.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.657 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:17.657 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642154+0000", 2024-09-16T20:50:17.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8540000000000002e-06 2024-09-16T20:50:17.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.657 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:17.657 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642159+0000", 2024-09-16T20:50:17.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.045e-06 2024-09-16T20:50:17.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.657 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:17.657 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719875+0000", 2024-09-16T20:50:17.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077716116000000002 2024-09-16T20:50:17.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.658 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:17.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719934+0000", 2024-09-16T20:50:17.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8819e-05 2024-09-16T20:50:17.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.658 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:17.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720051+0000", 2024-09-16T20:50:17.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000116947 2024-09-16T20:50:17.658 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:17.658 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:17.658 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:17.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.659 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9974 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 658432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:17.659 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642166+0000", 2024-09-16T20:50:17.659 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 26.943474202000001, 2024-09-16T20:50:17.659 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.078179786000000001, 2024-09-16T20:50:17.659 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:17.659 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:17.659 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:17.659 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:17.659 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:17.659 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:17.659 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9974 2024-09-16T20:50:17.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.659 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:17.659 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.659 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:17.660 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642166+0000", 2024-09-16T20:50:17.660 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:17.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.660 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:17.660 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642166+0000", 2024-09-16T20:50:17.660 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:17.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.660 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:17.660 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647343+0000", 2024-09-16T20:50:17.660 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0051769440000000002 2024-09-16T20:50:17.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.660 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:17.661 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647348+0000", 2024-09-16T20:50:17.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3619999999999997e-06 2024-09-16T20:50:17.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.661 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:17.661 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647359+0000", 2024-09-16T20:50:17.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0929e-05 2024-09-16T20:50:17.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.661 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:17.661 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647366+0000", 2024-09-16T20:50:17.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0910000000000004e-06 2024-09-16T20:50:17.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.662 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.662 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:17.662 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720094+0000", 2024-09-16T20:50:17.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.072727403999999995 2024-09-16T20:50:17.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.662 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.662 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:17.662 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720173+0000", 2024-09-16T20:50:17.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9382000000000007e-05 2024-09-16T20:50:17.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.663 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:17.663 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720346+0000", 2024-09-16T20:50:17.663 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017267399999999999 2024-09-16T20:50:17.663 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:17.663 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:17.663 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:17.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.663 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9988 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 730112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:17.663 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647703+0000", 2024-09-16T20:50:17.663 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 26.937937210000001, 2024-09-16T20:50:17.663 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076703473999999994, 2024-09-16T20:50:17.664 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:17.664 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:17.664 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:17.664 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:17.664 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:17.664 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:17.664 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9988 2024-09-16T20:50:17.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.664 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:17.664 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.664 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:17.664 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647703+0000", 2024-09-16T20:50:17.664 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:17.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.665 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.665 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:17.665 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647703+0000", 2024-09-16T20:50:17.665 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:17.665 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.665 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.665 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:17.665 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647703+0000", 2024-09-16T20:50:17.665 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0699999999999998e-07 2024-09-16T20:50:17.665 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.665 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.665 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:17.665 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647705+0000", 2024-09-16T20:50:17.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.866e-06 2024-09-16T20:50:17.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.666 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:17.666 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647708+0000", 2024-09-16T20:50:17.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3170000000000002e-06 2024-09-16T20:50:17.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.666 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:17.666 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647711+0000", 2024-09-16T20:50:17.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6759999999999999e-06 2024-09-16T20:50:17.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.667 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:17.667 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724225+0000", 2024-09-16T20:50:17.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.07651434 2024-09-16T20:50:17.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.667 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:17.667 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724291+0000", 2024-09-16T20:50:17.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5629000000000005e-05 2024-09-16T20:50:17.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.667 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:17.667 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724406+0000", 2024-09-16T20:50:17.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000115239 2024-09-16T20:50:17.667 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:17.667 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:17.667 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:17.668 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.668 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.668 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9989 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 735232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:17.668 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647715+0000", 2024-09-16T20:50:17.668 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 26.937925472, 2024-09-16T20:50:17.668 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076871693000000005, 2024-09-16T20:50:17.668 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:17.668 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:17.668 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:17.668 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:17.668 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:17.668 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:17.668 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9989 2024-09-16T20:50:17.668 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.669 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:17.669 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.669 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:17.669 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647715+0000", 2024-09-16T20:50:17.669 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:17.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.669 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.669 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:17.669 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647715+0000", 2024-09-16T20:50:17.669 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:17.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.669 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.669 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:17.670 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647715+0000", 2024-09-16T20:50:17.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1400000000000001e-07 2024-09-16T20:50:17.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.670 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:17.670 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647716+0000", 2024-09-16T20:50:17.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2240000000000001e-06 2024-09-16T20:50:17.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.670 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:17.670 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647719+0000", 2024-09-16T20:50:17.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0769999999999999e-06 2024-09-16T20:50:17.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.671 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:17.671 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647722+0000", 2024-09-16T20:50:17.671 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.65e-06 2024-09-16T20:50:17.671 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.671 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.671 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:17.671 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724424+0000", 2024-09-16T20:50:17.671 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076701982000000002 2024-09-16T20:50:17.671 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.671 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.671 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:17.671 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724470+0000", 2024-09-16T20:50:17.671 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6589999999999999e-05 2024-09-16T20:50:17.671 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.672 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.672 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:17.672 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724586+0000", 2024-09-16T20:50:17.672 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000115956 2024-09-16T20:50:17.672 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:17.672 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:17.672 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:17.672 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.673 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.673 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9990 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 740352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:17.673 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647725+0000", 2024-09-16T20:50:17.673 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 26.937915117999999, 2024-09-16T20:50:17.673 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077029919000000002, 2024-09-16T20:50:17.673 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:17.673 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:17.673 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:17.673 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:17.673 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:17.673 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:17.673 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9990 2024-09-16T20:50:17.673 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.673 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:17.674 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.674 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:17.674 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647725+0000", 2024-09-16T20:50:17.674 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:17.674 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.674 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.674 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:17.674 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647725+0000", 2024-09-16T20:50:17.674 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:17.674 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.674 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.674 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:17.674 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647725+0000", 2024-09-16T20:50:17.674 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2399999999999999e-07 2024-09-16T20:50:17.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.675 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.675 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:17.675 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647726+0000", 2024-09-16T20:50:17.675 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.116e-06 2024-09-16T20:50:17.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.675 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.675 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:17.675 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647729+0000", 2024-09-16T20:50:17.675 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.937e-06 2024-09-16T20:50:17.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.675 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.675 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:17.675 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647732+0000", 2024-09-16T20:50:17.675 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8739999999999999e-06 2024-09-16T20:50:17.676 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.676 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.676 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:17.676 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724612+0000", 2024-09-16T20:50:17.676 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076880102000000006 2024-09-16T20:50:17.676 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.676 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.676 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:17.676 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724649+0000", 2024-09-16T20:50:17.676 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6738e-05 2024-09-16T20:50:17.676 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.676 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.676 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:17.676 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724755+0000", 2024-09-16T20:50:17.676 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010592800000000001 2024-09-16T20:50:17.677 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:17.677 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:17.677 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:17.677 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.677 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.677 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9991 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 745472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:17.677 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:50:17.677 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 26.937904808999999, 2024-09-16T20:50:17.677 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077257820000000005, 2024-09-16T20:50:17.677 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:17.677 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:17.678 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:17.678 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:17.678 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:17.678 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:17.678 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9991 2024-09-16T20:50:17.678 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.678 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:17.678 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.678 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:17.678 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:50:17.678 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:17.678 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.678 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.678 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:17.679 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:50:17.679 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:17.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.679 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.679 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:17.679 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:50:17.679 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1199999999999999e-07 2024-09-16T20:50:17.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.679 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.679 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:17.679 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647736+0000", 2024-09-16T20:50:17.679 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3699999999999999e-07 2024-09-16T20:50:17.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.679 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.680 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:17.680 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647739+0000", 2024-09-16T20:50:17.680 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9359999999999999e-06 2024-09-16T20:50:17.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.680 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.680 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:17.680 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647742+0000", 2024-09-16T20:50:17.680 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5720000000000001e-06 2024-09-16T20:50:17.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.680 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.680 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:17.680 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724774+0000", 2024-09-16T20:50:17.680 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077032457999999998 2024-09-16T20:50:17.681 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.681 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.681 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:17.681 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724818+0000", 2024-09-16T20:50:17.681 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3803000000000001e-05 2024-09-16T20:50:17.681 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.681 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.681 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:17.681 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724993+0000", 2024-09-16T20:50:17.681 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000174902 2024-09-16T20:50:17.681 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:17.681 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:17.681 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:17.682 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.682 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.682 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9992 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 750592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:17.682 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:50:17.682 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 26.937895220000001, 2024-09-16T20:50:17.682 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077553227000000002, 2024-09-16T20:50:17.682 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:17.682 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:17.682 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:17.682 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:17.683 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:17.683 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:17.683 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9992 2024-09-16T20:50:17.683 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.683 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:17.683 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.683 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:17.683 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:50:17.683 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:17.683 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.683 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.683 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:17.684 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:50:17.684 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:17.684 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.684 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.684 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:17.684 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:50:17.684 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0200000000000001e-07 2024-09-16T20:50:17.684 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.684 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.684 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:17.684 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647746+0000", 2024-09-16T20:50:17.684 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.088e-06 2024-09-16T20:50:17.684 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.685 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.685 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:17.685 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647749+0000", 2024-09-16T20:50:17.685 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9170000000000002e-06 2024-09-16T20:50:17.685 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.685 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.685 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:17.685 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647752+0000", 2024-09-16T20:50:17.685 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.774e-06 2024-09-16T20:50:17.685 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.685 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.685 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:17.685 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725042+0000", 2024-09-16T20:50:17.686 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077289952999999995 2024-09-16T20:50:17.686 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.686 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.686 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:17.686 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725113+0000", 2024-09-16T20:50:17.686 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0920999999999994e-05 2024-09-16T20:50:17.686 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.686 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.686 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:17.686 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725298+0000", 2024-09-16T20:50:17.686 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000185372 2024-09-16T20:50:17.686 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:17.686 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:17.686 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:17.686 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.687 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.687 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9993 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 755712~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:17.687 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647755+0000", 2024-09-16T20:50:17.687 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 26.937885194, 2024-09-16T20:50:17.687 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077725674999999994, 2024-09-16T20:50:17.687 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:17.687 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:17.687 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:17.687 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:17.687 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:17.687 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:17.687 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9993 2024-09-16T20:50:17.687 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.688 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:17.688 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.688 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:17.688 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647755+0000", 2024-09-16T20:50:17.688 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:17.688 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.689 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.689 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:17.689 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647755+0000", 2024-09-16T20:50:17.689 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:17.689 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.689 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.689 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:17.689 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653406+0000", 2024-09-16T20:50:17.689 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.005650876 2024-09-16T20:50:17.689 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.689 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.689 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:17.690 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653411+0000", 2024-09-16T20:50:17.690 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8899999999999998e-06 2024-09-16T20:50:17.690 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.690 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.690 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:17.690 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653423+0000", 2024-09-16T20:50:17.690 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2447e-05 2024-09-16T20:50:17.690 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.690 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.690 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:17.690 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653431+0000", 2024-09-16T20:50:17.690 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5730000000000002e-06 2024-09-16T20:50:17.690 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.690 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.690 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:17.690 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725333+0000", 2024-09-16T20:50:17.691 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.071902743000000005 2024-09-16T20:50:17.691 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.691 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.691 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:17.691 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725385+0000", 2024-09-16T20:50:17.691 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1585000000000002e-05 2024-09-16T20:50:17.691 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:17.691 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:17.691 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:17.691 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725481+0000", 2024-09-16T20:50:17.691 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5561000000000002e-05 2024-09-16T20:50:17.691 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:17.691 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:17.691 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:17.691 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:17.691 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:17.692 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:17.692 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:50:17.693 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:50:17.694 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:50:17.696 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:50:17.697 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:50:17.697 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:17.691+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:17.793 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:17.792+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:17.809 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:17.810 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:50:17.810 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:50:17.810 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:17.818 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:50:17.894 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:17.893+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:17.928 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:17.928 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:50:17.928 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:50:17.928 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:50:17.928 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:17.937 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:50:17.957 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #40, image image_client.0-clone40 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent41 2024-09-16T20:50:17.995 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:17.994+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:18.011 INFO:teuthology.orchestra.run.smithi139.stdout:5806 trunc from 0xd132000 to 0xb06bc00 2024-09-16T20:50:18.056 INFO:teuthology.orchestra.run.smithi139.stdout:5807 clone 43 order 20 su 4096 sc 5 2024-09-16T20:50:18.093 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:18.094 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:50:18.094 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:50:18.094 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:50:18.094 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.094 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12548 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 628736~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:18.094 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757009+0000", 2024-09-16T20:50:18.094 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.335048772, 2024-09-16T20:50:18.094 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065743956000000006, 2024-09-16T20:50:18.094 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:18.094 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:18.094 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:18.094 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:18.094 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:18.095 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:18.095 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12548 2024-09-16T20:50:18.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.095 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:18.095 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:18.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757009+0000", 2024-09-16T20:50:18.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.095 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:18.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757009+0000", 2024-09-16T20:50:18.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.095 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.096 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:18.096 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757010+0000", 2024-09-16T20:50:18.096 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9900000000000001e-07 2024-09-16T20:50:18.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.096 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.096 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:18.096 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757012+0000", 2024-09-16T20:50:18.096 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2689999999999998e-06 2024-09-16T20:50:18.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.096 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.096 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:18.096 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757022+0000", 2024-09-16T20:50:18.096 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9699999999999994e-06 2024-09-16T20:50:18.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.096 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.097 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:18.097 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757028+0000", 2024-09-16T20:50:18.097 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3799999999999999e-06 2024-09-16T20:50:18.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.097 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.097 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:18.097 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822552+0000", 2024-09-16T20:50:18.097 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065524281000000004 2024-09-16T20:50:18.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.097 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.097 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:18.097 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822613+0000", 2024-09-16T20:50:18.097 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0430999999999998e-05 2024-09-16T20:50:18.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.098 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.098 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:18.098 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822753+0000", 2024-09-16T20:50:18.098 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000140226 2024-09-16T20:50:18.098 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.098 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:18.098 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.098 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.098 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.098 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12549 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 633856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:18.098 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:50:18.098 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.335020024999999, 2024-09-16T20:50:18.098 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065978518, 2024-09-16T20:50:18.098 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:18.099 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:18.099 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:18.099 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:18.099 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:18.099 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:18.099 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12549 2024-09-16T20:50:18.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.099 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:18.099 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.099 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:18.099 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:50:18.099 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.100 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.100 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.100 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:18.100 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:50:18.100 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.100 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.100 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.100 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:18.100 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:50:18.100 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9299999999999999e-07 2024-09-16T20:50:18.100 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.100 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.101 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:18.101 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757040+0000", 2024-09-16T20:50:18.101 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.677e-06 2024-09-16T20:50:18.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.101 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:18.101 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757045+0000", 2024-09-16T20:50:18.101 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8149999999999996e-06 2024-09-16T20:50:18.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.101 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:18.101 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757048+0000", 2024-09-16T20:50:18.101 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1e-06 2024-09-16T20:50:18.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.102 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.102 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:18.102 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822796+0000", 2024-09-16T20:50:18.102 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065748081 2024-09-16T20:50:18.102 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.102 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.102 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:18.102 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822867+0000", 2024-09-16T20:50:18.102 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0679000000000005e-05 2024-09-16T20:50:18.102 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.102 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.103 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:18.104 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823016+0000", 2024-09-16T20:50:18.104 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014987299999999999 2024-09-16T20:50:18.105 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.105 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:18.105 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.105 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12550 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 638976~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:18.105 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757060+0000", 2024-09-16T20:50:18.105 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.334997609999999, 2024-09-16T20:50:18.105 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066153564999999998, 2024-09-16T20:50:18.105 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:18.105 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:18.105 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:18.105 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:18.105 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:18.105 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:18.106 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12550 2024-09-16T20:50:18.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.106 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:18.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.106 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:18.106 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757060+0000", 2024-09-16T20:50:18.106 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.106 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:18.106 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757060+0000", 2024-09-16T20:50:18.106 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.107 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:18.107 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757061+0000", 2024-09-16T20:50:18.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.15e-07 2024-09-16T20:50:18.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.107 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:18.107 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757062+0000", 2024-09-16T20:50:18.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.663e-06 2024-09-16T20:50:18.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.107 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:18.107 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757066+0000", 2024-09-16T20:50:18.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.817e-06 2024-09-16T20:50:18.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:18.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757070+0000", 2024-09-16T20:50:18.108 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1509999999999997e-06 2024-09-16T20:50:18.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:18.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823044+0000", 2024-09-16T20:50:18.108 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065973992999999995 2024-09-16T20:50:18.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:18.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823082+0000", 2024-09-16T20:50:18.108 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8050999999999999e-05 2024-09-16T20:50:18.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.109 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.109 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:18.109 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823214+0000", 2024-09-16T20:50:18.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013157500000000001 2024-09-16T20:50:18.109 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.109 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:18.109 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.109 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.109 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12551 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 644096~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:18.109 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:50:18.109 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.334983911999998, 2024-09-16T20:50:18.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066391378000000001, 2024-09-16T20:50:18.110 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:18.110 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:18.110 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:18.110 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:18.110 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:18.110 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:18.110 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12551 2024-09-16T20:50:18.110 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.110 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:18.110 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.110 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:18.111 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:50:18.111 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.111 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.111 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.111 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:18.111 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:50:18.111 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.111 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.111 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.111 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:18.112 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:50:18.112 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2999999999999999e-07 2024-09-16T20:50:18.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.112 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:18.112 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757075+0000", 2024-09-16T20:50:18.112 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.097e-06 2024-09-16T20:50:18.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.112 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:18.112 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757082+0000", 2024-09-16T20:50:18.112 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9500000000000004e-06 2024-09-16T20:50:18.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.112 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.113 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:18.113 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757086+0000", 2024-09-16T20:50:18.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2229999999999999e-06 2024-09-16T20:50:18.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.113 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.113 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:18.113 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823262+0000", 2024-09-16T20:50:18.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066176078999999999 2024-09-16T20:50:18.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.113 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.113 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:18.113 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823336+0000", 2024-09-16T20:50:18.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3979000000000004e-05 2024-09-16T20:50:18.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.113 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.113 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:18.114 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823465+0000", 2024-09-16T20:50:18.114 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012982 2024-09-16T20:50:18.114 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.114 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:18.114 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.114 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.114 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12552 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 649216~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:18.114 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757089+0000", 2024-09-16T20:50:18.114 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.334968771, 2024-09-16T20:50:18.114 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066655617, 2024-09-16T20:50:18.114 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:18.114 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:18.114 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:18.114 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:18.115 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:18.115 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:18.115 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12552 2024-09-16T20:50:18.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.115 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:18.115 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.115 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:18.115 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757089+0000", 2024-09-16T20:50:18.115 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.115 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.115 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:18.115 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757089+0000", 2024-09-16T20:50:18.115 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.116 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.116 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:18.116 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759492+0000", 2024-09-16T20:50:18.116 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0024026109999999998 2024-09-16T20:50:18.116 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.116 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.116 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:18.116 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759497+0000", 2024-09-16T20:50:18.116 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1120000000000004e-06 2024-09-16T20:50:18.116 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.116 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.116 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:18.116 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759508+0000", 2024-09-16T20:50:18.116 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0981e-05 2024-09-16T20:50:18.116 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.117 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:18.117 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759516+0000", 2024-09-16T20:50:18.117 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7270000000000007e-06 2024-09-16T20:50:18.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.117 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:18.117 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823504+0000", 2024-09-16T20:50:18.117 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.063987984999999997 2024-09-16T20:50:18.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.117 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:18.118 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823579+0000", 2024-09-16T20:50:18.118 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5703000000000003e-05 2024-09-16T20:50:18.118 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.118 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.118 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:18.118 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823745+0000", 2024-09-16T20:50:18.118 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016549800000000001 2024-09-16T20:50:18.118 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.118 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:18.118 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.118 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.118 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.118 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12560 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 690176~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:18.118 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.759658+0000", 2024-09-16T20:50:18.119 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.332399569, 2024-09-16T20:50:18.119 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065858517000000005, 2024-09-16T20:50:18.119 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:18.119 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:18.119 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:18.119 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:18.119 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:18.119 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:18.119 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12560 2024-09-16T20:50:18.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.119 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:18.119 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.119 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:18.119 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759658+0000", 2024-09-16T20:50:18.119 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.120 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.120 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.120 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:18.120 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759658+0000", 2024-09-16T20:50:18.120 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.120 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.120 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.120 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:18.120 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759659+0000", 2024-09-16T20:50:18.120 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.17e-07 2024-09-16T20:50:18.120 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.120 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.120 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:18.120 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759660+0000", 2024-09-16T20:50:18.121 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0189999999999999e-06 2024-09-16T20:50:18.121 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.121 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.121 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:18.121 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759663+0000", 2024-09-16T20:50:18.121 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0010000000000002e-06 2024-09-16T20:50:18.121 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.121 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.121 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:18.121 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759665+0000", 2024-09-16T20:50:18.121 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5440000000000001e-06 2024-09-16T20:50:18.121 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.121 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.121 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:18.121 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825355+0000", 2024-09-16T20:50:18.121 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065690224000000005 2024-09-16T20:50:18.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.122 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.122 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:18.122 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825394+0000", 2024-09-16T20:50:18.122 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8810999999999998e-05 2024-09-16T20:50:18.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.122 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.122 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:18.122 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825517+0000", 2024-09-16T20:50:18.122 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012270099999999999 2024-09-16T20:50:18.122 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.122 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:18.122 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.122 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.123 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12561 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 695296~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:18.123 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.759668+0000", 2024-09-16T20:50:18.123 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.332389726999999, 2024-09-16T20:50:18.123 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066116999999999995, 2024-09-16T20:50:18.123 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:18.123 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:18.123 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:18.123 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:18.123 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:18.123 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:18.123 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12561 2024-09-16T20:50:18.123 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.123 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:18.123 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.124 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:18.124 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759668+0000", 2024-09-16T20:50:18.124 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.124 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.124 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.124 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:18.124 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759668+0000", 2024-09-16T20:50:18.124 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.124 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.124 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.124 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:18.124 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765379+0000", 2024-09-16T20:50:18.124 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057104410000000001 2024-09-16T20:50:18.124 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.125 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.125 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:18.125 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765383+0000", 2024-09-16T20:50:18.125 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2980000000000002e-06 2024-09-16T20:50:18.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.125 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.125 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:18.125 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765395+0000", 2024-09-16T20:50:18.125 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1671000000000001e-05 2024-09-16T20:50:18.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.125 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.125 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:18.125 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765403+0000", 2024-09-16T20:50:18.125 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1210000000000007e-06 2024-09-16T20:50:18.126 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.126 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.126 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:18.126 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825552+0000", 2024-09-16T20:50:18.126 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.060149498000000003 2024-09-16T20:50:18.126 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.126 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.126 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:18.126 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825625+0000", 2024-09-16T20:50:18.126 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2540999999999995e-05 2024-09-16T20:50:18.126 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.126 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.126 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:18.126 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825785+0000", 2024-09-16T20:50:18.126 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016043000000000001 2024-09-16T20:50:18.127 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.127 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:18.127 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.127 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.127 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.127 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12585 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 818176~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:18.127 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.765848+0000", 2024-09-16T20:50:18.127 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.326209756000001, 2024-09-16T20:50:18.127 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065843218999999995, 2024-09-16T20:50:18.127 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:18.127 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:18.127 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:18.127 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:18.127 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:18.127 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:18.128 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12585 2024-09-16T20:50:18.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.128 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:18.128 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.128 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:18.128 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765848+0000", 2024-09-16T20:50:18.128 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.128 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.128 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:18.128 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765848+0000", 2024-09-16T20:50:18.128 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.129 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.129 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:18.129 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768683+0000", 2024-09-16T20:50:18.129 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0028344519999999999 2024-09-16T20:50:18.129 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.129 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.129 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:18.129 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768686+0000", 2024-09-16T20:50:18.129 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8789999999999996e-06 2024-09-16T20:50:18.129 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.129 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.129 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:18.129 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768696+0000", 2024-09-16T20:50:18.129 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6709999999999994e-06 2024-09-16T20:50:18.130 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.130 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.130 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:18.130 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768702+0000", 2024-09-16T20:50:18.130 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2550000000000003e-06 2024-09-16T20:50:18.130 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.130 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.130 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:18.130 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.831499+0000", 2024-09-16T20:50:18.130 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.062797081000000005 2024-09-16T20:50:18.130 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.130 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.130 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:18.130 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.831538+0000", 2024-09-16T20:50:18.130 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8523999999999999e-05 2024-09-16T20:50:18.131 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.131 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.131 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:18.131 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.831691+0000", 2024-09-16T20:50:18.131 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015335699999999999 2024-09-16T20:50:18.131 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.131 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:18.131 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.131 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.131 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.131 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12597 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 879616~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:18.131 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.768883+0000", 2024-09-16T20:50:18.131 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.323175028000001, 2024-09-16T20:50:18.131 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065816527, 2024-09-16T20:50:18.132 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:18.132 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:18.132 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:18.132 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:18.132 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:18.132 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:18.132 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12597 2024-09-16T20:50:18.132 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.132 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:18.132 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.132 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:18.132 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768883+0000", 2024-09-16T20:50:18.132 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.132 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.133 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:18.133 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768883+0000", 2024-09-16T20:50:18.133 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.133 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:18.133 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771508+0000", 2024-09-16T20:50:18.133 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0026247589999999999 2024-09-16T20:50:18.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.133 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:18.133 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771513+0000", 2024-09-16T20:50:18.133 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9949999999999996e-06 2024-09-16T20:50:18.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.134 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.134 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:18.134 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771525+0000", 2024-09-16T20:50:18.134 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2033e-05 2024-09-16T20:50:18.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.134 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.134 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:18.134 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771532+0000", 2024-09-16T20:50:18.134 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4220000000000003e-06 2024-09-16T20:50:18.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.134 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.134 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:18.134 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.834532+0000", 2024-09-16T20:50:18.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.063000159 2024-09-16T20:50:18.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.135 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.135 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:18.135 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.834590+0000", 2024-09-16T20:50:18.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8218999999999999e-05 2024-09-16T20:50:18.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.135 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.135 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:18.135 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.834699+0000", 2024-09-16T20:50:18.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010894 2024-09-16T20:50:18.135 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.135 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:18.135 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.136 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.136 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12619 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 992256~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:18.136 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.774656+0000", 2024-09-16T20:50:18.136 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.317402254000001, 2024-09-16T20:50:18.136 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065715859000000001, 2024-09-16T20:50:18.136 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:18.136 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:18.136 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:18.136 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:18.136 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:18.136 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:18.136 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12619 2024-09-16T20:50:18.136 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.137 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:18.137 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.137 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:18.137 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774656+0000", 2024-09-16T20:50:18.137 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.137 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.137 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:18.137 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774656+0000", 2024-09-16T20:50:18.137 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.137 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.137 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:18.137 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774875+0000", 2024-09-16T20:50:18.137 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000218898 2024-09-16T20:50:18.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.138 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:18.138 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774879+0000", 2024-09-16T20:50:18.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0389999999999998e-06 2024-09-16T20:50:18.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.138 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:18.138 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774890+0000", 2024-09-16T20:50:18.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1250000000000001e-05 2024-09-16T20:50:18.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.138 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:18.138 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774897+0000", 2024-09-16T20:50:18.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7619999999999998e-06 2024-09-16T20:50:18.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.139 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.139 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:18.139 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840139+0000", 2024-09-16T20:50:18.139 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065242438 2024-09-16T20:50:18.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.139 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.139 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:18.139 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840186+0000", 2024-09-16T20:50:18.139 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6894999999999997e-05 2024-09-16T20:50:18.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.139 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.139 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:18.139 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840372+0000", 2024-09-16T20:50:18.139 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000185577 2024-09-16T20:50:18.140 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.140 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:18.140 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.140 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.140 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.140 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12620 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 997376~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:18.140 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.774908+0000", 2024-09-16T20:50:18.140 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.317150033000001, 2024-09-16T20:50:18.140 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065782754999999998, 2024-09-16T20:50:18.140 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:18.140 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:18.140 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:18.140 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:18.140 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:18.140 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:18.141 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12620 2024-09-16T20:50:18.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.141 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:18.141 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.141 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:18.141 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774908+0000", 2024-09-16T20:50:18.141 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.141 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.141 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:18.141 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774908+0000", 2024-09-16T20:50:18.141 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.141 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.141 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:18.142 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775102+0000", 2024-09-16T20:50:18.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000194098 2024-09-16T20:50:18.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.142 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.142 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:18.142 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775105+0000", 2024-09-16T20:50:18.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8739999999999999e-06 2024-09-16T20:50:18.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.142 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.142 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:18.142 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775111+0000", 2024-09-16T20:50:18.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9519999999999999e-06 2024-09-16T20:50:18.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.142 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.143 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:18.143 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775115+0000", 2024-09-16T20:50:18.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.002e-06 2024-09-16T20:50:18.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.143 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:18.143 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840409+0000", 2024-09-16T20:50:18.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065294195999999999 2024-09-16T20:50:18.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.143 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:18.143 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840507+0000", 2024-09-16T20:50:18.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7662000000000001e-05 2024-09-16T20:50:18.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.144 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:18.144 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840691+0000", 2024-09-16T20:50:18.144 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000183971 2024-09-16T20:50:18.144 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.144 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:18.144 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.144 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.144 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.144 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12621 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1002496~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:18.144 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775123+0000", 2024-09-16T20:50:18.144 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.316935337, 2024-09-16T20:50:18.144 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065829586999999995, 2024-09-16T20:50:18.144 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:18.144 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:18.145 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:18.145 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:18.145 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:18.145 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:18.145 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12621 2024-09-16T20:50:18.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.145 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:18.145 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.145 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:18.145 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775123+0000", 2024-09-16T20:50:18.145 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.145 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.146 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:18.146 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775123+0000", 2024-09-16T20:50:18.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.146 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:18.146 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775348+0000", 2024-09-16T20:50:18.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000225295 2024-09-16T20:50:18.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.146 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:18.146 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775352+0000", 2024-09-16T20:50:18.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0810000000000004e-06 2024-09-16T20:50:18.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.147 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.147 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:18.147 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775361+0000", 2024-09-16T20:50:18.147 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7399999999999993e-06 2024-09-16T20:50:18.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.147 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.147 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:18.147 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775367+0000", 2024-09-16T20:50:18.147 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5200000000000003e-06 2024-09-16T20:50:18.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.147 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.147 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:18.147 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840730+0000", 2024-09-16T20:50:18.147 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065362803999999997 2024-09-16T20:50:18.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.148 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.148 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:18.148 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840801+0000", 2024-09-16T20:50:18.148 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1006000000000004e-05 2024-09-16T20:50:18.148 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.148 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.148 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:18.148 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840952+0000", 2024-09-16T20:50:18.148 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015114100000000001 2024-09-16T20:50:18.148 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.148 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:18.148 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.148 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.148 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.149 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12622 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1007616~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:18.149 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775381+0000", 2024-09-16T20:50:18.149 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.316676503, 2024-09-16T20:50:18.149 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065822581000000005, 2024-09-16T20:50:18.149 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:18.149 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:18.149 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:18.149 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:18.149 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:18.149 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:18.149 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12622 2024-09-16T20:50:18.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.149 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:18.149 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.149 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:18.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775381+0000", 2024-09-16T20:50:18.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.150 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.150 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:18.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775381+0000", 2024-09-16T20:50:18.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.150 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.150 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:18.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775563+0000", 2024-09-16T20:50:18.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018153700000000001 2024-09-16T20:50:18.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.150 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:18.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775566+0000", 2024-09-16T20:50:18.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9620000000000002e-06 2024-09-16T20:50:18.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:18.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775572+0000", 2024-09-16T20:50:18.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8529999999999997e-06 2024-09-16T20:50:18.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:18.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775576+0000", 2024-09-16T20:50:18.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8920000000000002e-06 2024-09-16T20:50:18.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:18.152 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840990+0000", 2024-09-16T20:50:18.152 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065414599000000004 2024-09-16T20:50:18.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.152 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.152 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:18.152 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841068+0000", 2024-09-16T20:50:18.152 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8157000000000004e-05 2024-09-16T20:50:18.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.152 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.152 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:18.152 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841204+0000", 2024-09-16T20:50:18.152 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013558099999999999 2024-09-16T20:50:18.152 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.152 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:18.152 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.153 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.153 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.153 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12623 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1012736~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:18.153 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775583+0000", 2024-09-16T20:50:18.153 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.316475064999999, 2024-09-16T20:50:18.153 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065929087999999997, 2024-09-16T20:50:18.153 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:18.153 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:18.153 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:18.153 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:18.153 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:18.153 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:18.153 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12623 2024-09-16T20:50:18.154 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.154 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:18.154 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.154 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:18.154 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775583+0000", 2024-09-16T20:50:18.154 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.154 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.154 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.154 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:18.154 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775583+0000", 2024-09-16T20:50:18.154 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.154 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.154 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.154 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:18.154 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775804+0000", 2024-09-16T20:50:18.155 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00022116900000000001 2024-09-16T20:50:18.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.155 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.155 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:18.155 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775807+0000", 2024-09-16T20:50:18.155 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9529999999999999e-06 2024-09-16T20:50:18.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.155 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.155 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:18.155 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775814+0000", 2024-09-16T20:50:18.155 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7490000000000001e-06 2024-09-16T20:50:18.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.155 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.155 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:18.156 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775818+0000", 2024-09-16T20:50:18.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1999999999999996e-06 2024-09-16T20:50:18.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.156 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.156 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:18.156 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841267+0000", 2024-09-16T20:50:18.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065449127999999995 2024-09-16T20:50:18.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.156 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.156 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:18.156 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841344+0000", 2024-09-16T20:50:18.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6546999999999997e-05 2024-09-16T20:50:18.157 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.157 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.157 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:18.157 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841512+0000", 2024-09-16T20:50:18.157 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016834200000000001 2024-09-16T20:50:18.157 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.157 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:18.157 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.157 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.157 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.157 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12624 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1017856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:18.157 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:50:18.157 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.316231925, 2024-09-16T20:50:18.158 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065945866000000006, 2024-09-16T20:50:18.158 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:18.158 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:18.158 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:18.158 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:18.158 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:18.158 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:18.158 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12624 2024-09-16T20:50:18.158 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.158 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:18.158 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.158 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:18.158 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:50:18.158 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.159 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.159 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:18.159 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:50:18.159 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.159 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.159 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:18.159 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:50:18.159 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-16T20:50:18.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.159 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.159 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:18.159 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775828+0000", 2024-09-16T20:50:18.159 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3999999999999999e-06 2024-09-16T20:50:18.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.160 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.160 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:18.160 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775831+0000", 2024-09-16T20:50:18.160 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.478e-06 2024-09-16T20:50:18.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.160 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.160 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:18.160 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775834+0000", 2024-09-16T20:50:18.160 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9440000000000001e-06 2024-09-16T20:50:18.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.160 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.160 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:18.160 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841547+0000", 2024-09-16T20:50:18.160 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065712801000000001 2024-09-16T20:50:18.161 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.161 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.161 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:18.161 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841618+0000", 2024-09-16T20:50:18.161 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1151999999999999e-05 2024-09-16T20:50:18.161 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.161 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.161 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:18.161 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841772+0000", 2024-09-16T20:50:18.161 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000153721 2024-09-16T20:50:18.161 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.161 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:18.161 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.161 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.161 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.162 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12625 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1022976~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:18.162 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775840+0000", 2024-09-16T20:50:18.162 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.316218316, 2024-09-16T20:50:18.162 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066165451, 2024-09-16T20:50:18.162 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:18.162 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:18.162 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:18.162 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:18.162 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:18.162 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:18.162 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12625 2024-09-16T20:50:18.162 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.162 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:18.162 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.162 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:18.162 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775840+0000", 2024-09-16T20:50:18.163 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.163 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.163 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:18.163 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775840+0000", 2024-09-16T20:50:18.163 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.163 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.163 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:18.163 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776014+0000", 2024-09-16T20:50:18.163 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017441099999999999 2024-09-16T20:50:18.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.163 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.163 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:18.164 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776018+0000", 2024-09-16T20:50:18.164 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6509999999999999e-06 2024-09-16T20:50:18.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.164 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.164 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:18.164 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776026+0000", 2024-09-16T20:50:18.164 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.028e-06 2024-09-16T20:50:18.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.164 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.164 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:18.164 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776031+0000", 2024-09-16T20:50:18.164 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8400000000000002e-06 2024-09-16T20:50:18.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.164 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.165 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:18.165 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841801+0000", 2024-09-16T20:50:18.165 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065770472999999996 2024-09-16T20:50:18.165 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.165 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.165 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:18.165 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841847+0000", 2024-09-16T20:50:18.165 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5558e-05 2024-09-16T20:50:18.165 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.165 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.165 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:18.165 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842005+0000", 2024-09-16T20:50:18.165 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015849000000000001 2024-09-16T20:50:18.165 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.165 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:18.166 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.166 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.166 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12626 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1028096~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:18.166 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776041+0000", 2024-09-16T20:50:18.166 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.316017017, 2024-09-16T20:50:18.166 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066225260999999994, 2024-09-16T20:50:18.166 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:18.166 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:18.166 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:18.166 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:18.166 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:18.166 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:18.166 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12626 2024-09-16T20:50:18.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.167 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:18.167 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.167 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:18.167 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776041+0000", 2024-09-16T20:50:18.167 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.167 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.167 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.167 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:18.167 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776041+0000", 2024-09-16T20:50:18.167 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.167 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.167 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.167 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:18.167 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776263+0000", 2024-09-16T20:50:18.167 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00022193899999999999 2024-09-16T20:50:18.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.168 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.168 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:18.168 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776268+0000", 2024-09-16T20:50:18.168 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8049999999999997e-06 2024-09-16T20:50:18.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.168 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.168 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:18.168 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776280+0000", 2024-09-16T20:50:18.168 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2635e-05 2024-09-16T20:50:18.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.168 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.168 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:18.168 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776288+0000", 2024-09-16T20:50:18.168 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8199999999999997e-06 2024-09-16T20:50:18.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.169 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.169 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:18.169 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842041+0000", 2024-09-16T20:50:18.169 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065752506000000002 2024-09-16T20:50:18.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.169 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.169 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:18.169 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842123+0000", 2024-09-16T20:50:18.169 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2393999999999998e-05 2024-09-16T20:50:18.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.169 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.169 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:18.169 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842266+0000", 2024-09-16T20:50:18.169 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014316200000000001 2024-09-16T20:50:18.170 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.170 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:18.170 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.170 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.170 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.170 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12627 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1033216~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:18.170 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776301+0000", 2024-09-16T20:50:18.170 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.315757056999999, 2024-09-16T20:50:18.170 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066187814999999997, 2024-09-16T20:50:18.170 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:18.170 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:18.170 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:18.170 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:18.170 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:18.170 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:18.171 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12627 2024-09-16T20:50:18.171 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.171 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:18.171 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.171 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:18.171 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776301+0000", 2024-09-16T20:50:18.171 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.171 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.171 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.171 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:18.171 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776301+0000", 2024-09-16T20:50:18.171 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.171 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.171 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.172 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:18.172 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776473+0000", 2024-09-16T20:50:18.172 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017222400000000001 2024-09-16T20:50:18.172 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.172 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.172 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:18.172 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776477+0000", 2024-09-16T20:50:18.172 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7019999999999999e-06 2024-09-16T20:50:18.172 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.172 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.172 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:18.172 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776486+0000", 2024-09-16T20:50:18.172 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2119999999999992e-06 2024-09-16T20:50:18.172 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.172 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.173 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:18.173 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776492+0000", 2024-09-16T20:50:18.173 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8089999999999998e-06 2024-09-16T20:50:18.173 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.173 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.173 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:18.173 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842301+0000", 2024-09-16T20:50:18.173 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065809042999999998 2024-09-16T20:50:18.173 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.173 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.173 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:18.173 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842362+0000", 2024-09-16T20:50:18.173 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1260000000000006e-05 2024-09-16T20:50:18.173 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.173 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.174 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:18.174 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842489+0000", 2024-09-16T20:50:18.174 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012656500000000001 2024-09-16T20:50:18.174 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.174 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:18.174 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.174 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.174 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12628 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1038336~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:18.174 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776502+0000", 2024-09-16T20:50:18.174 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.315556410999999, 2024-09-16T20:50:18.174 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066283426000000006, 2024-09-16T20:50:18.174 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:18.174 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:18.174 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:18.175 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:18.175 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:18.175 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:18.175 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12628 2024-09-16T20:50:18.175 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.175 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:18.175 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.175 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:18.175 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776502+0000", 2024-09-16T20:50:18.175 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.175 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.175 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.175 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:18.175 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776502+0000", 2024-09-16T20:50:18.175 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.175 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.176 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.176 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:18.176 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776716+0000", 2024-09-16T20:50:18.176 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000214596 2024-09-16T20:50:18.176 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.176 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.176 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:18.176 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776723+0000", 2024-09-16T20:50:18.176 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4000000000000003e-06 2024-09-16T20:50:18.176 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.176 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.176 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:18.176 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776731+0000", 2024-09-16T20:50:18.176 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6129999999999998e-06 2024-09-16T20:50:18.176 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.177 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.177 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:18.177 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776737+0000", 2024-09-16T20:50:18.177 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1970000000000001e-06 2024-09-16T20:50:18.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.177 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.177 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:18.177 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842532+0000", 2024-09-16T20:50:18.177 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065795127999999994 2024-09-16T20:50:18.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.177 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.177 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:18.177 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842599+0000", 2024-09-16T20:50:18.177 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6711000000000006e-05 2024-09-16T20:50:18.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.178 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.178 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:18.178 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842785+0000", 2024-09-16T20:50:18.178 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018578099999999999 2024-09-16T20:50:18.178 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.178 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:18.178 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.178 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.178 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12629 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1043456~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:18.178 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776748+0000", 2024-09-16T20:50:18.178 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.315310033999999, 2024-09-16T20:50:18.179 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066303788000000002, 2024-09-16T20:50:18.179 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:18.179 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:18.179 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:18.179 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:18.179 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:18.179 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:18.179 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12629 2024-09-16T20:50:18.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.179 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:18.179 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.179 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:18.179 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776748+0000", 2024-09-16T20:50:18.179 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.180 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.180 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.180 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:18.180 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776748+0000", 2024-09-16T20:50:18.180 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.180 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.180 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.180 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:18.180 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776934+0000", 2024-09-16T20:50:18.180 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018598399999999999 2024-09-16T20:50:18.180 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.180 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.180 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:18.181 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776939+0000", 2024-09-16T20:50:18.181 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1000000000000003e-06 2024-09-16T20:50:18.181 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.181 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.181 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:18.181 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776954+0000", 2024-09-16T20:50:18.181 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5221e-05 2024-09-16T20:50:18.181 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.181 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.181 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:18.181 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776962+0000", 2024-09-16T20:50:18.181 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.554e-06 2024-09-16T20:50:18.181 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.181 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.182 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:18.182 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842813+0000", 2024-09-16T20:50:18.182 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065851006000000004 2024-09-16T20:50:18.182 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.182 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.182 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:18.182 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842886+0000", 2024-09-16T20:50:18.182 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3663000000000003e-05 2024-09-16T20:50:18.182 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.182 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.182 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:18.182 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.843052+0000", 2024-09-16T20:50:18.182 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016526 2024-09-16T20:50:18.182 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.183 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:18.183 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.183 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.183 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:18.183 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:18.184 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:50:18.184 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:18.183+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:18.285 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:18.284+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:18.333 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:18.333 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:50:18.333 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:50:18.333 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:18.341 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:50:18.386 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:18.385+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:18.448 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:18.448 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:50:18.448 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:50:18.448 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:50:18.448 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:18.456 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:50:18.487 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:18.486+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:18.565 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:18.565 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:50:18.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:50:18.565 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:50:18.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.565 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5286.0:10643 3.17 3.f6273fb7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-16T20:50:18.565 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:50:18.565 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 69.772875560000003, 2024-09-16T20:50:18.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.76672347200000002, 2024-09-16T20:50:18.565 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:18.565 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:18.565 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:18.566 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:18.566 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5286", 2024-09-16T20:50:18.566 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/4135433989", 2024-09-16T20:50:18.566 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 10643 2024-09-16T20:50:18.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.566 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:18.566 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.566 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:18.566 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:50:18.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.566 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.566 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:18.566 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:50:18.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.567 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:18.567 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790489+0000", 2024-09-16T20:50:18.567 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5910000000000001e-06 2024-09-16T20:50:18.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.567 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:18.567 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790493+0000", 2024-09-16T20:50:18.567 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2669999999999998e-06 2024-09-16T20:50:18.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.567 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:18.567 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790500+0000", 2024-09-16T20:50:18.567 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3780000000000003e-06 2024-09-16T20:50:18.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.568 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:18.568 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790505+0000", 2024-09-16T20:50:18.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1560000000000003e-06 2024-09-16T20:50:18.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.568 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:18.568 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790559+0000", 2024-09-16T20:50:18.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3494e-05 2024-09-16T20:50:18.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.568 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:18.568 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790566+0000", 2024-09-16T20:50:18.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7649999999999993e-06 2024-09-16T20:50:18.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.569 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:18.569 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:09.557198+0000", 2024-09-16T20:50:18.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.76663136200000004 2024-09-16T20:50:18.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.569 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:18.569 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:09.557211+0000", 2024-09-16T20:50:18.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3458999999999999e-05 2024-09-16T20:50:18.569 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.569 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:18.569 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.570 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5512.0:450 3.2 3:44e0cffa:::rbd_header.110ef0d7a535:head [watch reconnect cookie 140263298018912 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:50:18.570 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:50:18.570 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 42.000852647000002, 2024-09-16T20:50:18.570 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0084581939999999, 2024-09-16T20:50:18.570 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:18.570 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:18.570 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:18.570 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:18.570 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5512", 2024-09-16T20:50:18.570 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:50:18.570 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 450 2024-09-16T20:50:18.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.570 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:18.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.571 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:18.571 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:50:18.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.571 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.571 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:18.571 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:50:18.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.571 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.571 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:18.571 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562512+0000", 2024-09-16T20:50:18.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.009e-06 2024-09-16T20:50:18.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.571 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.572 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:18.572 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562517+0000", 2024-09-16T20:50:18.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9339999999999997e-06 2024-09-16T20:50:18.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.572 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:18.572 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562530+0000", 2024-09-16T20:50:18.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3495e-05 2024-09-16T20:50:18.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.572 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:18.572 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562536+0000", 2024-09-16T20:50:18.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5690000000000004e-06 2024-09-16T20:50:18.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.573 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:18.573 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562591+0000", 2024-09-16T20:50:18.573 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5761999999999999e-05 2024-09-16T20:50:18.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.573 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.573 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:50:18.573 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562599+0000", 2024-09-16T20:50:18.573 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9179999999999994e-06 2024-09-16T20:50:18.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.573 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.573 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:18.573 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.565971+0000", 2024-09-16T20:50:18.573 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003371546 2024-09-16T20:50:18.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.574 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:18.574 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566019+0000", 2024-09-16T20:50:18.574 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7849000000000002e-05 2024-09-16T20:50:18.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.574 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:18.574 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.569382+0000", 2024-09-16T20:50:18.574 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.003362833 2024-09-16T20:50:18.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.574 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:18.574 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.569529+0000", 2024-09-16T20:50:18.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000147823 2024-09-16T20:50:18.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.575 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:18.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570969+0000", 2024-09-16T20:50:18.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001439456 2024-09-16T20:50:18.575 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.575 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:18.575 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.575 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:14996 3.2 3:44e0cffa:::rbd_header.110ef0d7a535:head [watch reconnect cookie 140213436269840 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:50:18.575 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:50:18.575 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 42.000804234999997, 2024-09-16T20:50:18.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.008813943, 2024-09-16T20:50:18.576 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:18.576 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:18.576 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:18.576 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:18.576 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:50:18.576 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:18.576 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 14996 2024-09-16T20:50:18.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.576 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:18.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:18.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:50:18.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.577 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:18.577 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:50:18.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.577 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:18.577 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562560+0000", 2024-09-16T20:50:18.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.06e-07 2024-09-16T20:50:18.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.577 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:18.578 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562563+0000", 2024-09-16T20:50:18.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.495e-06 2024-09-16T20:50:18.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.578 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:18.578 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562571+0000", 2024-09-16T20:50:18.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1950000000000003e-06 2024-09-16T20:50:18.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.578 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:18.578 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562579+0000", 2024-09-16T20:50:18.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2649999999999999e-06 2024-09-16T20:50:18.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:18.579 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562673+0000", 2024-09-16T20:50:18.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4118999999999998e-05 2024-09-16T20:50:18.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:50:18.579 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562677+0000", 2024-09-16T20:50:18.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5179999999999998e-06 2024-09-16T20:50:18.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:18.579 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566461+0000", 2024-09-16T20:50:18.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0037837259999999998 2024-09-16T20:50:18.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:18.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566468+0000", 2024-09-16T20:50:18.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3350000000000001e-06 2024-09-16T20:50:18.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:18.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570306+0000", 2024-09-16T20:50:18.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.003837997 2024-09-16T20:50:18.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:18.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570392+0000", 2024-09-16T20:50:18.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5872999999999996e-05 2024-09-16T20:50:18.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:18.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571373+0000", 2024-09-16T20:50:18.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00098071399999999993 2024-09-16T20:50:18.581 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.581 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:18.581 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.581 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5255.0:1931 3.2 3:47882819:::rbd_data.110eb5f95afc.0000000000000009:head [read 217088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:50:18.581 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:50:18.581 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 42.000765485000002, 2024-09-16T20:50:18.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0082752880000001, 2024-09-16T20:50:18.582 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:18.582 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:18.582 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:18.582 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:18.582 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5255", 2024-09-16T20:50:18.582 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:50:18.582 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1931 2024-09-16T20:50:18.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.582 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:18.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:18.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:50:18.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:18.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:50:18.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:18.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:50:18.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9699999999999996e-07 2024-09-16T20:50:18.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:18.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562601+0000", 2024-09-16T20:50:18.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8600000000000001e-06 2024-09-16T20:50:18.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.584 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:18.584 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562608+0000", 2024-09-16T20:50:18.584 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9e-06 2024-09-16T20:50:18.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.584 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.584 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:18.584 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562615+0000", 2024-09-16T20:50:18.584 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8179999999999998e-06 2024-09-16T20:50:18.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.584 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.584 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:18.584 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562703+0000", 2024-09-16T20:50:18.584 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7811000000000003e-05 2024-09-16T20:50:18.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:50:18.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562706+0000", 2024-09-16T20:50:18.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4920000000000002e-06 2024-09-16T20:50:18.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:18.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566685+0000", 2024-09-16T20:50:18.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0039792780000000002 2024-09-16T20:50:18.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:18.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566693+0000", 2024-09-16T20:50:18.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4730000000000003e-06 2024-09-16T20:50:18.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:18.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570737+0000", 2024-09-16T20:50:18.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0040436749999999 2024-09-16T20:50:18.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:18.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570873+0000", 2024-09-16T20:50:18.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000136484 2024-09-16T20:50:18.586 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.586 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:18.586 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.586 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5255.0:1939 3.2 3:47882819:::rbd_data.110eb5f95afc.0000000000000009:head [read 233472~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:50:18.586 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:50:18.586 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 42.000610619, 2024-09-16T20:50:18.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0084657589999999, 2024-09-16T20:50:18.587 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:18.587 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:18.587 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:18.587 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:18.587 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5255", 2024-09-16T20:50:18.587 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:50:18.587 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1939 2024-09-16T20:50:18.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.587 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:18.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:18.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:50:18.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:18.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:50:18.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:18.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:50:18.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5199999999999997e-07 2024-09-16T20:50:18.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:18.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562757+0000", 2024-09-16T20:50:18.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4979999999999998e-06 2024-09-16T20:50:18.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:18.589 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562764+0000", 2024-09-16T20:50:18.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1470000000000004e-06 2024-09-16T20:50:18.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:18.589 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562768+0000", 2024-09-16T20:50:18.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7929999999999999e-06 2024-09-16T20:50:18.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:18.589 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562805+0000", 2024-09-16T20:50:18.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7477e-05 2024-09-16T20:50:18.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.590 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting_for_map not empty", 2024-09-16T20:50:18.590 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562810+0000", 2024-09-16T20:50:18.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0819999999999998e-06 2024-09-16T20:50:18.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.590 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:18.590 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566717+0000", 2024-09-16T20:50:18.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0039072600000000001 2024-09-16T20:50:18.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.590 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:18.590 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566724+0000", 2024-09-16T20:50:18.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4880000000000004e-06 2024-09-16T20:50:18.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.590 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:18.591 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571009+0000", 2024-09-16T20:50:18.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.004284677 2024-09-16T20:50:18.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.591 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:18.591 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571218+0000", 2024-09-16T20:50:18.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00020978500000000001 2024-09-16T20:50:18.591 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.591 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:18.591 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.591 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:11020 3.1f 3:fee2d1dd:::rbd_data.110ea93691bb.000000000000000e:23 [read 955392~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:18.591 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.693811+0000", 2024-09-16T20:50:18.591 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.869551961999999, 2024-09-16T20:50:18.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056833005999999998, 2024-09-16T20:50:18.592 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:18.592 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:18.592 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:18.592 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:18.592 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:18.592 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:18.592 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 11020 2024-09-16T20:50:18.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.592 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:18.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.592 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:18.592 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693811+0000", 2024-09-16T20:50:18.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.593 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:18.593 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693811+0000", 2024-09-16T20:50:18.593 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.593 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:18.593 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693812+0000", 2024-09-16T20:50:18.593 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6100000000000001e-07 2024-09-16T20:50:18.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.593 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:18.593 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693815+0000", 2024-09-16T20:50:18.593 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7389999999999999e-06 2024-09-16T20:50:18.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.594 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:18.594 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693823+0000", 2024-09-16T20:50:18.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0439999999999996e-06 2024-09-16T20:50:18.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.594 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:18.594 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693828+0000", 2024-09-16T20:50:18.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7440000000000004e-06 2024-09-16T20:50:18.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.594 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:18.594 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750357+0000", 2024-09-16T20:50:18.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056529192999999998 2024-09-16T20:50:18.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.595 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:18.595 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750429+0000", 2024-09-16T20:50:18.595 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1557000000000006e-05 2024-09-16T20:50:18.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.595 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:18.595 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750644+0000", 2024-09-16T20:50:18.595 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00021526800000000001 2024-09-16T20:50:18.595 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.595 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:18.595 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.595 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:11021 3.1f 3:fee2d1dd:::rbd_data.110ea93691bb.000000000000000e:23 [read 960512~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:18.595 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.693837+0000", 2024-09-16T20:50:18.595 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.869526073999999, 2024-09-16T20:50:18.596 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057155138000000001, 2024-09-16T20:50:18.596 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:18.596 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:18.596 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:18.596 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:18.596 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:18.596 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:18.596 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 11021 2024-09-16T20:50:18.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.596 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:18.596 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.596 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:18.596 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693837+0000", 2024-09-16T20:50:18.596 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.597 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.597 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:18.597 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693837+0000", 2024-09-16T20:50:18.597 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.597 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.597 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.597 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:18.597 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699566+0000", 2024-09-16T20:50:18.597 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057283639999999997 2024-09-16T20:50:18.597 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.597 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.597 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:18.597 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699571+0000", 2024-09-16T20:50:18.597 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.604e-06 2024-09-16T20:50:18.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.598 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:18.598 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699583+0000", 2024-09-16T20:50:18.598 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1372999999999999e-05 2024-09-16T20:50:18.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.598 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:18.598 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699591+0000", 2024-09-16T20:50:18.598 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5259999999999993e-06 2024-09-16T20:50:18.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.598 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:18.598 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750674+0000", 2024-09-16T20:50:18.598 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051083154999999998 2024-09-16T20:50:18.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.599 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:18.599 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750745+0000", 2024-09-16T20:50:18.599 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0933999999999995e-05 2024-09-16T20:50:18.599 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.599 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:18.599 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750992+0000", 2024-09-16T20:50:18.599 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00024718199999999998 2024-09-16T20:50:18.599 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.599 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:18.599 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.599 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.599 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8789 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2314240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:18.600 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057183+0000", 2024-09-16T20:50:18.600 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 17.506180155999999, 2024-09-16T20:50:18.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056736416999999997, 2024-09-16T20:50:18.600 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:18.600 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:18.600 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:18.600 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:18.600 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:18.600 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:18.600 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8789 2024-09-16T20:50:18.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.600 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:18.600 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.600 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:18.600 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057183+0000", 2024-09-16T20:50:18.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.601 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:18.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057183+0000", 2024-09-16T20:50:18.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.601 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:18.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057183+0000", 2024-09-16T20:50:18.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-16T20:50:18.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.601 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:18.602 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057186+0000", 2024-09-16T20:50:18.602 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3889999999999999e-06 2024-09-16T20:50:18.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.602 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:18.602 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057192+0000", 2024-09-16T20:50:18.602 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8950000000000003e-06 2024-09-16T20:50:18.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.602 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:18.602 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057196+0000", 2024-09-16T20:50:18.602 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5159999999999996e-06 2024-09-16T20:50:18.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.602 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:18.602 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.113826+0000", 2024-09-16T20:50:18.603 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056629298000000002 2024-09-16T20:50:18.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.603 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:18.603 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.113866+0000", 2024-09-16T20:50:18.603 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0423000000000002e-05 2024-09-16T20:50:18.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.603 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:18.603 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.113920+0000", 2024-09-16T20:50:18.603 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3526000000000002e-05 2024-09-16T20:50:18.603 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.603 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:18.603 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.604 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8790 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2590720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:18.604 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057257+0000", 2024-09-16T20:50:18.604 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 17.506106754000001, 2024-09-16T20:50:18.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056791169000000002, 2024-09-16T20:50:18.604 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:18.604 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:18.604 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:18.604 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:18.604 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:18.604 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:18.604 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8790 2024-09-16T20:50:18.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.604 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:18.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:18.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057257+0000", 2024-09-16T20:50:18.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:18.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057257+0000", 2024-09-16T20:50:18.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:18.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057257+0000", 2024-09-16T20:50:18.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6600000000000002e-07 2024-09-16T20:50:18.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:18.606 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057260+0000", 2024-09-16T20:50:18.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7240000000000001e-06 2024-09-16T20:50:18.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.606 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:18.606 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057267+0000", 2024-09-16T20:50:18.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0600000000000002e-06 2024-09-16T20:50:18.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.606 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:18.606 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057272+0000", 2024-09-16T20:50:18.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0390000000000004e-06 2024-09-16T20:50:18.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.606 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:18.607 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.113950+0000", 2024-09-16T20:50:18.607 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056677989999999998 2024-09-16T20:50:18.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.607 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:18.607 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.113993+0000", 2024-09-16T20:50:18.607 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3205999999999999e-05 2024-09-16T20:50:18.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.607 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:18.607 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114048+0000", 2024-09-16T20:50:18.607 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4684e-05 2024-09-16T20:50:18.607 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.607 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:18.607 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.608 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8791 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2332672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:18.608 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057279+0000", 2024-09-16T20:50:18.608 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 17.506084339000001, 2024-09-16T20:50:18.608 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056890067000000002, 2024-09-16T20:50:18.608 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:18.608 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:18.608 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:18.608 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:18.608 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:18.608 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:18.608 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8791 2024-09-16T20:50:18.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.608 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:18.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:18.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057279+0000", 2024-09-16T20:50:18.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:18.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057279+0000", 2024-09-16T20:50:18.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:18.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057279+0000", 2024-09-16T20:50:18.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.15e-07 2024-09-16T20:50:18.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.610 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:18.610 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057281+0000", 2024-09-16T20:50:18.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8080000000000001e-06 2024-09-16T20:50:18.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.610 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:18.610 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057287+0000", 2024-09-16T20:50:18.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5999999999999997e-06 2024-09-16T20:50:18.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.610 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:18.610 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057291+0000", 2024-09-16T20:50:18.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5990000000000004e-06 2024-09-16T20:50:18.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.611 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:18.611 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114078+0000", 2024-09-16T20:50:18.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056786246999999998 2024-09-16T20:50:18.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.611 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:18.611 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114118+0000", 2024-09-16T20:50:18.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9898999999999999e-05 2024-09-16T20:50:18.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.611 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:18.612 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114169+0000", 2024-09-16T20:50:18.612 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1499000000000003e-05 2024-09-16T20:50:18.612 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.612 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:18.612 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.612 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8792 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2609152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:18.613 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:50:18.613 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 17.506066357000002, 2024-09-16T20:50:18.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057003524999999999, 2024-09-16T20:50:18.613 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:18.613 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:18.613 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:18.613 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:18.613 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:18.613 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:18.613 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8792 2024-09-16T20:50:18.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.613 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:18.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.613 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:18.614 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:50:18.614 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.614 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:18.614 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:50:18.614 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.614 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:18.614 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:50:18.614 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0400000000000002e-07 2024-09-16T20:50:18.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.614 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:18.614 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057299+0000", 2024-09-16T20:50:18.615 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7460000000000001e-06 2024-09-16T20:50:18.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.615 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.615 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:18.615 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057304+0000", 2024-09-16T20:50:18.615 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3210000000000004e-06 2024-09-16T20:50:18.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.615 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.615 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:18.615 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057310+0000", 2024-09-16T20:50:18.615 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4360000000000001e-06 2024-09-16T20:50:18.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.615 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.615 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:18.615 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114200+0000", 2024-09-16T20:50:18.615 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056889954 2024-09-16T20:50:18.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.616 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.616 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:18.616 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114243+0000", 2024-09-16T20:50:18.616 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2998999999999999e-05 2024-09-16T20:50:18.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.616 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.616 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:18.616 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114300+0000", 2024-09-16T20:50:18.616 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7765e-05 2024-09-16T20:50:18.616 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.616 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:18.616 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.616 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.616 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8794 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2351104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:18.617 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057327+0000", 2024-09-16T20:50:18.617 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 17.506036033000001, 2024-09-16T20:50:18.617 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057101130999999999, 2024-09-16T20:50:18.617 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:18.617 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:18.617 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:18.617 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:18.617 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:18.617 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:18.617 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8794 2024-09-16T20:50:18.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.617 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:18.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.617 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:18.617 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057327+0000", 2024-09-16T20:50:18.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:18.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057327+0000", 2024-09-16T20:50:18.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:18.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057328+0000", 2024-09-16T20:50:18.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7399999999999999e-07 2024-09-16T20:50:18.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:18.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057330+0000", 2024-09-16T20:50:18.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2199999999999999e-06 2024-09-16T20:50:18.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.619 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:18.619 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057336+0000", 2024-09-16T20:50:18.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1240000000000002e-06 2024-09-16T20:50:18.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.619 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:18.619 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057341+0000", 2024-09-16T20:50:18.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8470000000000003e-06 2024-09-16T20:50:18.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.620 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:18.620 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114334+0000", 2024-09-16T20:50:18.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056993267 2024-09-16T20:50:18.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.620 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:18.620 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114376+0000", 2024-09-16T20:50:18.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1638999999999999e-05 2024-09-16T20:50:18.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.620 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:18.620 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114428+0000", 2024-09-16T20:50:18.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2660000000000001e-05 2024-09-16T20:50:18.620 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.620 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:18.621 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.621 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8798 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2297856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:18.621 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:50:18.621 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 17.506016305999999, 2024-09-16T20:50:18.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057277041000000001, 2024-09-16T20:50:18.621 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:18.621 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:18.621 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:18.621 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:18.621 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:18.621 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:18.622 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8798 2024-09-16T20:50:18.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.622 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:18.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.622 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:18.622 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:50:18.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.622 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:18.622 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:50:18.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.622 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:18.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:50:18.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2800000000000003e-07 2024-09-16T20:50:18.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:18.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057354+0000", 2024-09-16T20:50:18.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7780000000000002e-06 2024-09-16T20:50:18.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:18.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057360+0000", 2024-09-16T20:50:18.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0419999999999999e-06 2024-09-16T20:50:18.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:18.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057365+0000", 2024-09-16T20:50:18.624 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7929999999999997e-06 2024-09-16T20:50:18.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.624 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.624 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:18.624 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114460+0000", 2024-09-16T20:50:18.624 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057094662999999997 2024-09-16T20:50:18.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.624 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.624 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:18.624 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114532+0000", 2024-09-16T20:50:18.624 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2881999999999998e-05 2024-09-16T20:50:18.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.624 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.624 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:18.624 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114624+0000", 2024-09-16T20:50:18.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1555e-05 2024-09-16T20:50:18.625 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.625 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:18.625 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.625 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8800 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2316288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:18.625 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:50:18.625 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 17.505992367000001, 2024-09-16T20:50:18.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.05746946, 2024-09-16T20:50:18.625 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:18.625 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:18.625 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:18.625 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:18.625 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:18.625 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:18.626 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8800 2024-09-16T20:50:18.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.626 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:18.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.626 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:18.626 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:50:18.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.626 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:18.626 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:50:18.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.627 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:18.627 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:50:18.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5199999999999998e-07 2024-09-16T20:50:18.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.627 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:18.627 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057373+0000", 2024-09-16T20:50:18.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.79e-06 2024-09-16T20:50:18.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.627 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:18.627 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057379+0000", 2024-09-16T20:50:18.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6699999999999999e-06 2024-09-16T20:50:18.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.627 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:18.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057386+0000", 2024-09-16T20:50:18.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3170000000000004e-06 2024-09-16T20:50:18.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:18.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114655+0000", 2024-09-16T20:50:18.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057269420000000001 2024-09-16T20:50:18.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:18.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114734+0000", 2024-09-16T20:50:18.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8084000000000007e-05 2024-09-16T20:50:18.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:18.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114840+0000", 2024-09-16T20:50:18.629 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000106827 2024-09-16T20:50:18.629 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.629 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:18.629 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.629 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.629 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.629 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8801 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2334720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:18.629 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:50:18.629 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 17.505971336000002, 2024-09-16T20:50:18.629 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057594682000000001, 2024-09-16T20:50:18.629 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:18.629 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:18.629 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:18.629 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:18.629 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:18.630 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:18.630 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8801 2024-09-16T20:50:18.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.630 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:18.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:18.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:50:18.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:18.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:50:18.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:18.631 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:50:18.631 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6899999999999998e-07 2024-09-16T20:50:18.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.631 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.631 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:18.631 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057394+0000", 2024-09-16T20:50:18.631 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8300000000000001e-06 2024-09-16T20:50:18.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.631 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.631 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:18.631 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057400+0000", 2024-09-16T20:50:18.631 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6500000000000001e-06 2024-09-16T20:50:18.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.631 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.631 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:18.631 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057404+0000", 2024-09-16T20:50:18.632 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4669999999999998e-06 2024-09-16T20:50:18.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.632 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.632 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:18.632 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114874+0000", 2024-09-16T20:50:18.632 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057469602000000002 2024-09-16T20:50:18.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.632 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.632 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:18.632 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114933+0000", 2024-09-16T20:50:18.632 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9500000000000003e-05 2024-09-16T20:50:18.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.632 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.632 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:18.632 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114987+0000", 2024-09-16T20:50:18.633 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3263999999999997e-05 2024-09-16T20:50:18.633 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.633 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:18.633 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.633 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.633 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8802 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2353152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:18.633 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:50:18.633 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 17.505922650999999, 2024-09-16T20:50:18.633 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057714079000000001, 2024-09-16T20:50:18.633 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:18.633 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:18.633 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:18.633 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:18.633 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:18.634 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:18.634 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8802 2024-09-16T20:50:18.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.634 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:18.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.634 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:18.634 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:50:18.634 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.634 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:18.634 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:50:18.634 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.634 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:18.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:50:18.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4999999999999998e-07 2024-09-16T20:50:18.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.635 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:18.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057446+0000", 2024-09-16T20:50:18.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4240000000000003e-06 2024-09-16T20:50:18.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.635 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:18.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057452+0000", 2024-09-16T20:50:18.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8379999999999996e-06 2024-09-16T20:50:18.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.635 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:18.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057460+0000", 2024-09-16T20:50:18.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9629999999999998e-06 2024-09-16T20:50:18.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.636 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:18.636 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115015+0000", 2024-09-16T20:50:18.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057554389999999997 2024-09-16T20:50:18.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.636 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:18.636 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115082+0000", 2024-09-16T20:50:18.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7733999999999999e-05 2024-09-16T20:50:18.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.636 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:18.636 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115155+0000", 2024-09-16T20:50:18.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2280000000000006e-05 2024-09-16T20:50:18.637 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.637 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:18.637 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.637 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8804 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2555904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:18.637 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:50:18.637 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 17.505896334999999, 2024-09-16T20:50:18.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057833862, 2024-09-16T20:50:18.637 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:18.637 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:18.637 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:18.637 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:18.637 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:18.637 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:18.638 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8804 2024-09-16T20:50:18.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.638 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:18.638 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.638 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:18.638 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:50:18.638 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.638 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.638 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:18.638 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:50:18.638 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.638 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.638 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:18.639 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:50:18.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7899999999999999e-07 2024-09-16T20:50:18.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.639 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:18.639 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057469+0000", 2024-09-16T20:50:18.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.001e-06 2024-09-16T20:50:18.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.639 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:18.639 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057475+0000", 2024-09-16T20:50:18.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7629999999999997e-06 2024-09-16T20:50:18.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.639 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:18.640 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057480+0000", 2024-09-16T20:50:18.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8069999999999999e-06 2024-09-16T20:50:18.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.640 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:18.640 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115183+0000", 2024-09-16T20:50:18.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057703364 2024-09-16T20:50:18.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.640 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:18.640 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115227+0000", 2024-09-16T20:50:18.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3903999999999998e-05 2024-09-16T20:50:18.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.640 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:18.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115301+0000", 2024-09-16T20:50:18.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3543999999999999e-05 2024-09-16T20:50:18.641 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.641 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:18.641 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.641 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8805 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2574336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:18.641 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057487+0000", 2024-09-16T20:50:18.641 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 17.505876007000001, 2024-09-16T20:50:18.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058021027000000003, 2024-09-16T20:50:18.641 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:18.641 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:18.641 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:18.641 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:18.641 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:18.642 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:18.642 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8805 2024-09-16T20:50:18.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.642 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:18.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.642 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:18.642 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057487+0000", 2024-09-16T20:50:18.642 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.642 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:18.642 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057487+0000", 2024-09-16T20:50:18.642 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.642 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:18.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057488+0000", 2024-09-16T20:50:18.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5199999999999998e-07 2024-09-16T20:50:18.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.643 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:18.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057490+0000", 2024-09-16T20:50:18.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.939e-06 2024-09-16T20:50:18.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.643 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:18.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057495+0000", 2024-09-16T20:50:18.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6439999999999997e-06 2024-09-16T20:50:18.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.643 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:18.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057500+0000", 2024-09-16T20:50:18.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8119999999999998e-06 2024-09-16T20:50:18.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.644 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:18.644 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115341+0000", 2024-09-16T20:50:18.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057841341999999997 2024-09-16T20:50:18.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.644 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:18.644 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115422+0000", 2024-09-16T20:50:18.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1003999999999999e-05 2024-09-16T20:50:18.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.644 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:18.644 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115508+0000", 2024-09-16T20:50:18.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5933999999999994e-05 2024-09-16T20:50:18.645 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.645 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:18.645 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.645 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8807 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2592768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:18.645 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057505+0000", 2024-09-16T20:50:18.645 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 17.505857797000001, 2024-09-16T20:50:18.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058186633000000001, 2024-09-16T20:50:18.645 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:18.645 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:18.645 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:18.645 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:18.645 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:18.645 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:18.646 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8807 2024-09-16T20:50:18.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.646 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:18.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.646 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:18.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057505+0000", 2024-09-16T20:50:18.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.646 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:18.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057505+0000", 2024-09-16T20:50:18.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.646 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:18.647 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057506+0000", 2024-09-16T20:50:18.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.27e-07 2024-09-16T20:50:18.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.647 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:18.647 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057508+0000", 2024-09-16T20:50:18.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7400000000000001e-06 2024-09-16T20:50:18.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.647 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:18.647 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057513+0000", 2024-09-16T20:50:18.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.468e-06 2024-09-16T20:50:18.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.647 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:18.648 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057517+0000", 2024-09-16T20:50:18.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4329999999999997e-06 2024-09-16T20:50:18.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:18.648 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115538+0000", 2024-09-16T20:50:18.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058020355000000003 2024-09-16T20:50:18.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:18.648 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115616+0000", 2024-09-16T20:50:18.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7998999999999996e-05 2024-09-16T20:50:18.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:18.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115692+0000", 2024-09-16T20:50:18.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6310999999999995e-05 2024-09-16T20:50:18.649 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.649 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:18.649 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.649 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8808 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2611200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:18.649 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:50:18.649 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 17.505840536000001, 2024-09-16T20:50:18.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058354046, 2024-09-16T20:50:18.649 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:18.649 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:18.649 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:18.649 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:18.650 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:18.650 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:18.650 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8808 2024-09-16T20:50:18.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.650 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:18.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.650 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:18.650 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:50:18.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.650 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:18.650 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:50:18.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:18.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.651 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:18.651 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:50:18.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2500000000000001e-07 2024-09-16T20:50:18.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.651 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:18.651 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057525+0000", 2024-09-16T20:50:18.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5549999999999999e-06 2024-09-16T20:50:18.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.651 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:18.651 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057530+0000", 2024-09-16T20:50:18.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5489999999999997e-06 2024-09-16T20:50:18.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.651 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:18.652 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057535+0000", 2024-09-16T20:50:18.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5709999999999999e-06 2024-09-16T20:50:18.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.652 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:18.652 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115722+0000", 2024-09-16T20:50:18.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058187323999999999 2024-09-16T20:50:18.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.652 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:18.652 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115799+0000", 2024-09-16T20:50:18.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6445000000000004e-05 2024-09-16T20:50:18.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:18.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:18.652 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:18.652 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115877+0000", 2024-09-16T20:50:18.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8276999999999996e-05 2024-09-16T20:50:18.653 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.653 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:18.653 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.653 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:18.653 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:18.653 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:18.654 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:50:18.655 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:18.653+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:18.708 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T20:50:18.726 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 3, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-16T20:50:18.726 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T20:50:18.726 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-16T20:50:18.727 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T20:50:18.727 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T20:50:18.727 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2024-09-16T20:50:18.770 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:18.769+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:18.772 INFO:teuthology.orchestra.run.smithi139.stdout:truncating image image_client.0-clone42 from 0xb06bc00 (overlap 0xb06bc00) to 0x9ec7600 2024-09-16T20:50:18.791 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:18.791 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:50:18.791 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:50:18.791 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:18.803 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:50:18.837 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:18.837 INFO:teuthology.orchestra.run.smithi043.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-16T20:50:18.837 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:18.855 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:18.854+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:18.913 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:18.914 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:50:18.914 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:50:18.914 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:50:18.914 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:18.923 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:50:18.955 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:18.955+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:19.056 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:19.055+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:19.082 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:19.082 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:50:19.082 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:50:19.082 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:50:19.082 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.082 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:2206 3.c 3:373d06ad:::rbd_data.110e3f481dff.000000000000000a:7 [read 1046528~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T20:50:19.082 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:50:19.082 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 116.744053029, 2024-09-16T20:50:19.082 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050147714000000003, 2024-09-16T20:50:19.082 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:19.082 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:19.082 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:19.082 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:19.083 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:19.083 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:19.083 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2206 2024-09-16T20:50:19.083 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.083 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:19.083 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.083 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:19.083 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:50:19.083 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.083 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.083 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.083 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:19.083 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:50:19.083 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.083 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.083 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.084 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:19.084 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336022+0000", 2024-09-16T20:50:19.084 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8599999999999999e-07 2024-09-16T20:50:19.084 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.084 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.084 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:19.084 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336024+0000", 2024-09-16T20:50:19.084 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.934e-06 2024-09-16T20:50:19.084 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.084 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.084 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:19.084 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336037+0000", 2024-09-16T20:50:19.084 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2496e-05 2024-09-16T20:50:19.084 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.084 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.085 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:19.085 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.336042+0000", 2024-09-16T20:50:19.085 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4809999999999997e-06 2024-09-16T20:50:19.085 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.085 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.085 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:19.085 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.385980+0000", 2024-09-16T20:50:19.085 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049938376999999999 2024-09-16T20:50:19.085 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.085 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.085 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:19.085 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.386069+0000", 2024-09-16T20:50:19.086 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8368999999999994e-05 2024-09-16T20:50:19.086 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.086 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.086 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:19.086 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.386170+0000", 2024-09-16T20:50:19.086 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000100671 2024-09-16T20:50:19.086 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.086 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:19.086 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.086 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.086 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.086 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5439 3.a 3:5789f5c3:::rbd_data.110ed9ca540.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 1181696~60416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:19.086 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:50:19.086 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 116.21536444900001, 2024-09-16T20:50:19.087 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10102045599999999, 2024-09-16T20:50:19.087 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:19.087 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:19.087 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:19.087 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:19.087 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:19.087 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:19.087 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5439 2024-09-16T20:50:19.087 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.087 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:19.087 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.087 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:19.087 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:50:19.087 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.088 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.088 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.088 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:19.088 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:50:19.088 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.088 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.088 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.088 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:19.088 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864711+0000", 2024-09-16T20:50:19.088 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7899999999999997e-07 2024-09-16T20:50:19.088 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.088 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.088 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:19.088 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864760+0000", 2024-09-16T20:50:19.089 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8803e-05 2024-09-16T20:50:19.089 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.089 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.089 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:19.089 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864769+0000", 2024-09-16T20:50:19.089 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0999999999999993e-06 2024-09-16T20:50:19.089 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.089 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.089 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:19.089 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864774+0000", 2024-09-16T20:50:19.089 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8919999999999999e-06 2024-09-16T20:50:19.089 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.089 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.089 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:19.089 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864815+0000", 2024-09-16T20:50:19.090 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1307000000000003e-05 2024-09-16T20:50:19.090 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.090 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.090 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:19.090 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864893+0000", 2024-09-16T20:50:19.090 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7581000000000001e-05 2024-09-16T20:50:19.090 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.090 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.090 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 1", 2024-09-16T20:50:19.090 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864996+0000", 2024-09-16T20:50:19.090 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102725 2024-09-16T20:50:19.090 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.090 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.090 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:19.090 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965442+0000", 2024-09-16T20:50:19.090 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100446012 2024-09-16T20:50:19.091 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.091 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.091 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:19.091 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965618+0000", 2024-09-16T20:50:19.091 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00017653100000000001 2024-09-16T20:50:19.091 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.091 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.091 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:19.091 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965682+0000", 2024-09-16T20:50:19.091 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3905999999999994e-05 2024-09-16T20:50:19.091 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.091 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.091 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:19.091 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965731+0000", 2024-09-16T20:50:19.091 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8819999999999997e-05 2024-09-16T20:50:19.092 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.092 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:19.092 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.092 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.092 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.092 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4600.0:5439 3.a e37/23 ondisk, result = 0)", 2024-09-16T20:50:19.092 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:50:19.092 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 116.21436830099999, 2024-09-16T20:50:19.092 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099742634999999996, 2024-09-16T20:50:19.092 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:19.092 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:19.092 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:19.092 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:19.092 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.092 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:19.092 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:50:19.093 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.093 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.093 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.093 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:19.093 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:50:19.093 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.093 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.093 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.093 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:19.093 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:50:19.093 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0900000000000001e-07 2024-09-16T20:50:19.093 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.093 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.093 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:19.094 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865711+0000", 2024-09-16T20:50:19.094 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3739999999999998e-06 2024-09-16T20:50:19.094 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.094 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.094 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:19.094 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965343+0000", 2024-09-16T20:50:19.094 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099632526999999999 2024-09-16T20:50:19.094 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.094 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.094 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:19.094 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965357+0000", 2024-09-16T20:50:19.094 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3713999999999999e-05 2024-09-16T20:50:19.094 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.094 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.094 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:19.094 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965426+0000", 2024-09-16T20:50:19.095 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8745999999999995e-05 2024-09-16T20:50:19.095 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.095 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.095 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:19.095 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965434+0000", 2024-09-16T20:50:19.095 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6910000000000003e-06 2024-09-16T20:50:19.095 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.095 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.095 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:19.095 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965449+0000", 2024-09-16T20:50:19.095 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4874e-05 2024-09-16T20:50:19.095 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.095 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:19.095 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.095 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.096 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.096 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4882.0:2508 3.12 3:4992771f:::rbd_object_map.110e7c8851c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T20:50:19.096 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:50:19.096 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 92.598564753000005, 2024-09-16T20:50:19.096 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.125007011, 2024-09-16T20:50:19.096 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:19.096 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:19.096 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:19.096 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:19.096 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4882", 2024-09-16T20:50:19.096 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2436923223", 2024-09-16T20:50:19.096 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2508 2024-09-16T20:50:19.096 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.096 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:19.096 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.097 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:19.097 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:50:19.097 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.097 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.097 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.097 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:19.097 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:50:19.097 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.097 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.097 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.097 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:19.097 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481511+0000", 2024-09-16T20:50:19.097 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0699999999999996e-07 2024-09-16T20:50:19.097 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.098 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.098 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:19.098 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481515+0000", 2024-09-16T20:50:19.098 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0609999999999997e-06 2024-09-16T20:50:19.098 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.098 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.098 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:19.098 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481523+0000", 2024-09-16T20:50:19.098 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1310000000000006e-06 2024-09-16T20:50:19.098 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.098 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.099 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:19.099 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481528+0000", 2024-09-16T20:50:19.099 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4900000000000002e-06 2024-09-16T20:50:19.099 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.099 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.099 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:19.099 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481565+0000", 2024-09-16T20:50:19.099 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7066000000000001e-05 2024-09-16T20:50:19.099 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.099 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.099 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:19.099 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481628+0000", 2024-09-16T20:50:19.100 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3118000000000001e-05 2024-09-16T20:50:19.100 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.100 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.100 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 2", 2024-09-16T20:50:19.100 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481822+0000", 2024-09-16T20:50:19.100 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019446200000000001 2024-09-16T20:50:19.100 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.100 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.100 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:19.100 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605988+0000", 2024-09-16T20:50:19.100 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.124165888 2024-09-16T20:50:19.100 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.100 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.100 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:19.100 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606427+0000", 2024-09-16T20:50:19.101 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00043854899999999999 2024-09-16T20:50:19.101 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.101 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.101 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:19.101 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606476+0000", 2024-09-16T20:50:19.101 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8971999999999998e-05 2024-09-16T20:50:19.101 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.101 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.101 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:19.101 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606517+0000", 2024-09-16T20:50:19.101 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1467000000000001e-05 2024-09-16T20:50:19.101 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.101 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:19.101 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.102 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.102 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.102 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4882.0:2508 3.12 e57/55 ondisk, result = 0)", 2024-09-16T20:50:19.102 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:50:19.102 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 92.597576253, 2024-09-16T20:50:19.102 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12349711400000001, 2024-09-16T20:50:19.102 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:19.102 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:19.102 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:19.102 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:19.102 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.102 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:19.102 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:50:19.102 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.103 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.103 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.103 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:19.103 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:50:19.103 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.103 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.103 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.103 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:19.103 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:50:19.103 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.06e-07 2024-09-16T20:50:19.103 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.103 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.103 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:19.103 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482502+0000", 2024-09-16T20:50:19.104 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.01e-06 2024-09-16T20:50:19.104 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.104 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.104 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:19.104 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605772+0000", 2024-09-16T20:50:19.104 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.123269687 2024-09-16T20:50:19.104 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.104 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.104 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:19.104 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605791+0000", 2024-09-16T20:50:19.104 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8912000000000001e-05 2024-09-16T20:50:19.104 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.104 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.104 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:19.105 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605974+0000", 2024-09-16T20:50:19.105 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000182836 2024-09-16T20:50:19.105 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.105 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.105 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:19.105 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605982+0000", 2024-09-16T20:50:19.105 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1699999999999997e-06 2024-09-16T20:50:19.105 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.105 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.105 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:19.105 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605996+0000", 2024-09-16T20:50:19.105 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3893000000000001e-05 2024-09-16T20:50:19.105 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.105 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:19.105 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.106 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.106 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.106 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4882.0:2512 3.f 3:f4d895b7:::rbd_data.110e7c8851c.0000000000000000:head [read 16384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T20:50:19.106 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:50:19.106 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 92.597139222999999, 2024-09-16T20:50:19.106 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12318483, 2024-09-16T20:50:19.106 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:19.106 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:19.106 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:19.106 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:19.106 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4882", 2024-09-16T20:50:19.106 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2436923223", 2024-09-16T20:50:19.106 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2512 2024-09-16T20:50:19.106 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.106 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:19.107 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.107 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:19.107 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:50:19.107 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.107 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.107 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.107 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:19.107 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:50:19.107 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.107 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.107 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.107 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:19.107 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:50:19.107 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3499999999999995e-07 2024-09-16T20:50:19.107 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.108 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.108 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:19.108 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482943+0000", 2024-09-16T20:50:19.108 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3450000000000002e-06 2024-09-16T20:50:19.108 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.108 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.108 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:19.108 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605772+0000", 2024-09-16T20:50:19.108 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.122829796 2024-09-16T20:50:19.108 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.108 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.108 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:19.108 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605792+0000", 2024-09-16T20:50:19.108 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9916999999999999e-05 2024-09-16T20:50:19.109 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.109 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.109 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:19.109 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605917+0000", 2024-09-16T20:50:19.109 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00012500799999999999 2024-09-16T20:50:19.109 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.109 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.109 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:19.109 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606004+0000", 2024-09-16T20:50:19.109 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6242999999999994e-05 2024-09-16T20:50:19.109 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.109 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.109 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:19.110 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606120+0000", 2024-09-16T20:50:19.110 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000116786 2024-09-16T20:50:19.110 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.110 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:19.110 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.110 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.110 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.110 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:11702 3.15 3:acf33c8b:::rbd_data.110e747f0779.000000000000001a:head [set-alloc-hint object_size 262144 write_size 262144,write 143360~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:50:19.110 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:50:19.110 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 74.942146801000007, 2024-09-16T20:50:19.110 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.084801043000000006, 2024-09-16T20:50:19.110 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:19.110 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:19.110 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:19.111 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:19.111 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:50:19.111 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:50:19.111 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 11702 2024-09-16T20:50:19.111 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.111 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:19.111 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.111 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:19.111 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:50:19.111 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.111 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.111 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.111 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:19.112 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:50:19.112 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.112 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.112 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.112 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:19.112 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137929+0000", 2024-09-16T20:50:19.112 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8400000000000002e-07 2024-09-16T20:50:19.112 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.112 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.112 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:19.112 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137937+0000", 2024-09-16T20:50:19.112 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0260000000000007e-06 2024-09-16T20:50:19.112 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.112 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.112 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:19.113 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137952+0000", 2024-09-16T20:50:19.113 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4659e-05 2024-09-16T20:50:19.113 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.113 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.113 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:19.113 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137960+0000", 2024-09-16T20:50:19.113 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0169999999999996e-06 2024-09-16T20:50:19.113 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.113 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.113 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:19.113 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138010+0000", 2024-09-16T20:50:19.113 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0021000000000001e-05 2024-09-16T20:50:19.113 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.113 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.114 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:19.114 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138114+0000", 2024-09-16T20:50:19.114 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000103797 2024-09-16T20:50:19.114 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.114 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.114 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:50:19.114 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138233+0000", 2024-09-16T20:50:19.115 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000118971 2024-09-16T20:50:19.115 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.115 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.115 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:19.115 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138953+0000", 2024-09-16T20:50:19.115 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00072026000000000002 2024-09-16T20:50:19.115 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.115 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.115 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:19.115 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222609+0000", 2024-09-16T20:50:19.115 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.083656650999999999 2024-09-16T20:50:19.115 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.115 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.115 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:19.116 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222684+0000", 2024-09-16T20:50:19.116 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4369999999999994e-05 2024-09-16T20:50:19.116 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.116 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.116 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:19.116 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222729+0000", 2024-09-16T20:50:19.116 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5287000000000002e-05 2024-09-16T20:50:19.116 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.116 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:19.116 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.116 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.116 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.116 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:11704 3.9 3:91220b75:::rbd_data.110e747f0779.000000000000001c:head [set-alloc-hint object_size 262144 write_size 262144,write 139264~6656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:50:19.116 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:50:19.117 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 74.941908799999993, 2024-09-16T20:50:19.117 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.084531133999999994, 2024-09-16T20:50:19.117 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:19.117 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:19.117 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:19.117 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:19.117 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:50:19.117 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:50:19.117 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 11704 2024-09-16T20:50:19.117 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.117 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:19.117 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.118 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:19.118 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:50:19.118 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.118 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.118 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.118 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:19.118 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:50:19.118 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.118 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.118 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.118 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:19.118 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138167+0000", 2024-09-16T20:50:19.118 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.33e-07 2024-09-16T20:50:19.118 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.118 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.119 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:19.119 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138179+0000", 2024-09-16T20:50:19.119 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2055999999999999e-05 2024-09-16T20:50:19.119 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.119 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.119 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:19.119 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138193+0000", 2024-09-16T20:50:19.119 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3961000000000001e-05 2024-09-16T20:50:19.119 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.119 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.119 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:19.119 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138200+0000", 2024-09-16T20:50:19.119 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4429999999999997e-06 2024-09-16T20:50:19.119 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.119 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.119 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:19.120 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138258+0000", 2024-09-16T20:50:19.120 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7225999999999997e-05 2024-09-16T20:50:19.120 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.120 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.120 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:19.120 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138364+0000", 2024-09-16T20:50:19.120 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000106013 2024-09-16T20:50:19.120 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.120 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.120 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:50:19.120 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138494+0000", 2024-09-16T20:50:19.120 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013059599999999999 2024-09-16T20:50:19.120 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.120 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.120 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:19.120 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.139220+0000", 2024-09-16T20:50:19.121 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00072539800000000004 2024-09-16T20:50:19.121 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.121 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.121 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:19.121 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222608+0000", 2024-09-16T20:50:19.121 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.083388620999999996 2024-09-16T20:50:19.121 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.121 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.121 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:19.121 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222651+0000", 2024-09-16T20:50:19.121 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2228000000000003e-05 2024-09-16T20:50:19.121 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.121 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.121 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:19.121 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222697+0000", 2024-09-16T20:50:19.122 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6659000000000002e-05 2024-09-16T20:50:19.122 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.122 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:19.122 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.122 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.122 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.122 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5138.0:9358 3.c 3.c25699cc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T20:50:19.122 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:50:19.122 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 52.229761494999998, 2024-09-16T20:50:19.122 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0018657580000001, 2024-09-16T20:50:19.122 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:19.122 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:19.122 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:50:19.122 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:19.122 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5138", 2024-09-16T20:50:19.122 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:50:19.123 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 9358 2024-09-16T20:50:19.123 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.123 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:19.123 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.123 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:19.123 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:50:19.123 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.123 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.123 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.123 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:19.123 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:50:19.123 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.123 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.123 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.123 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:19.124 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850314+0000", 2024-09-16T20:50:19.124 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4099999999999999e-07 2024-09-16T20:50:19.124 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.124 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.124 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:19.124 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850316+0000", 2024-09-16T20:50:19.124 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2469999999999998e-06 2024-09-16T20:50:19.124 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.124 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.124 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:19.124 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850322+0000", 2024-09-16T20:50:19.124 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3470000000000004e-06 2024-09-16T20:50:19.124 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.124 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.124 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:19.125 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850327+0000", 2024-09-16T20:50:19.125 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9450000000000001e-06 2024-09-16T20:50:19.125 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.125 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.125 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:19.125 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850376+0000", 2024-09-16T20:50:19.125 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8257000000000002e-05 2024-09-16T20:50:19.125 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.125 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.125 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:19.125 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850385+0000", 2024-09-16T20:50:19.125 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9800000000000004e-06 2024-09-16T20:50:19.125 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.125 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.125 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:19.126 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852167+0000", 2024-09-16T20:50:19.126 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0017825659999999 2024-09-16T20:50:19.126 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.126 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.126 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:19.126 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852179+0000", 2024-09-16T20:50:19.126 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1975e-05 2024-09-16T20:50:19.126 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.126 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:19.126 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.126 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.126 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.126 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5138.0:9382 3.c 3.c25699cc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T20:50:19.126 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:50:19.126 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 52.221414479000003, 2024-09-16T20:50:19.127 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.99358404700000003, 2024-09-16T20:50:19.127 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:19.127 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:19.127 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:50:19.127 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:19.127 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5138", 2024-09-16T20:50:19.127 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:50:19.127 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 9382 2024-09-16T20:50:19.127 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.127 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:19.127 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.127 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:19.127 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:50:19.127 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.127 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.128 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.128 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:19.128 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:50:19.128 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.128 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.128 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.128 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:19.128 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858661+0000", 2024-09-16T20:50:19.128 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0200000000000002e-07 2024-09-16T20:50:19.128 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.128 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.128 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:19.128 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858664+0000", 2024-09-16T20:50:19.128 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.038e-06 2024-09-16T20:50:19.129 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.129 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.129 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:19.129 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858671+0000", 2024-09-16T20:50:19.129 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0110000000000003e-06 2024-09-16T20:50:19.129 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.129 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.129 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:19.129 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858677+0000", 2024-09-16T20:50:19.129 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0959999999999997e-06 2024-09-16T20:50:19.129 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.129 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.129 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:19.129 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858732+0000", 2024-09-16T20:50:19.130 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4543000000000003e-05 2024-09-16T20:50:19.130 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.130 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.130 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:19.130 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858739+0000", 2024-09-16T20:50:19.130 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5429999999999996e-06 2024-09-16T20:50:19.130 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.130 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.130 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:19.130 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852236+0000", 2024-09-16T20:50:19.130 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.993496453 2024-09-16T20:50:19.130 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.130 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.130 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:19.131 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852244+0000", 2024-09-16T20:50:19.131 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8610000000000004e-06 2024-09-16T20:50:19.131 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.131 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:19.131 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.131 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.131 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.131 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:361 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 372736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:50:19.131 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246714+0000", 2024-09-16T20:50:19.131 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 42.833360331000002, 2024-09-16T20:50:19.131 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050338229999999998, 2024-09-16T20:50:19.131 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:19.131 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:19.131 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:19.131 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:19.131 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:50:19.132 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:50:19.132 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 361 2024-09-16T20:50:19.132 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.132 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:19.132 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.132 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:19.132 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246714+0000", 2024-09-16T20:50:19.132 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.132 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.132 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.132 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:19.132 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246714+0000", 2024-09-16T20:50:19.132 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.132 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.132 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.132 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:19.133 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246715+0000", 2024-09-16T20:50:19.133 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1899999999999999e-07 2024-09-16T20:50:19.133 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.133 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.133 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:19.133 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246716+0000", 2024-09-16T20:50:19.133 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1659999999999999e-06 2024-09-16T20:50:19.133 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.133 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.133 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:19.133 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246721+0000", 2024-09-16T20:50:19.133 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9830000000000004e-06 2024-09-16T20:50:19.133 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.133 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.134 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:19.134 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246723+0000", 2024-09-16T20:50:19.134 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5679999999999998e-06 2024-09-16T20:50:19.134 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.134 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.134 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:19.134 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.296875+0000", 2024-09-16T20:50:19.134 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050151683000000002 2024-09-16T20:50:19.134 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.134 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.134 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:19.134 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.296959+0000", 2024-09-16T20:50:19.134 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4079999999999995e-05 2024-09-16T20:50:19.134 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.134 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.134 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:19.135 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297053+0000", 2024-09-16T20:50:19.135 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.3530999999999996e-05 2024-09-16T20:50:19.135 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.135 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:19.135 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.135 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.135 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.135 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:362 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 385024~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:50:19.135 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246727+0000", 2024-09-16T20:50:19.135 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 42.833348221999998, 2024-09-16T20:50:19.135 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050546528, 2024-09-16T20:50:19.135 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:19.135 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:19.135 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:19.135 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:19.135 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:50:19.136 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:50:19.136 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 362 2024-09-16T20:50:19.136 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.136 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:19.136 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.136 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:19.136 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246727+0000", 2024-09-16T20:50:19.136 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.136 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.136 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.136 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:19.136 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246727+0000", 2024-09-16T20:50:19.136 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.136 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.136 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.137 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:19.137 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246727+0000", 2024-09-16T20:50:19.137 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.35e-07 2024-09-16T20:50:19.137 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.137 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.137 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:19.137 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246728+0000", 2024-09-16T20:50:19.137 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.175e-06 2024-09-16T20:50:19.137 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.137 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.137 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:19.137 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246734+0000", 2024-09-16T20:50:19.137 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2160000000000003e-06 2024-09-16T20:50:19.137 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.138 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.138 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:19.138 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246737+0000", 2024-09-16T20:50:19.138 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9450000000000002e-06 2024-09-16T20:50:19.138 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.138 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.138 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:19.138 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297103+0000", 2024-09-16T20:50:19.138 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050365518999999997 2024-09-16T20:50:19.138 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.138 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.138 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:19.138 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297183+0000", 2024-09-16T20:50:19.138 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0208000000000002e-05 2024-09-16T20:50:19.139 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.139 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.139 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:19.139 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297273+0000", 2024-09-16T20:50:19.139 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0229999999999995e-05 2024-09-16T20:50:19.139 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.139 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:19.139 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.139 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.139 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.139 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:368 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 327680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:50:19.139 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246746+0000", 2024-09-16T20:50:19.139 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 42.833328942000001, 2024-09-16T20:50:19.139 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050730747999999999, 2024-09-16T20:50:19.139 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:19.140 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:19.140 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:19.140 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:19.140 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:50:19.140 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:50:19.140 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 368 2024-09-16T20:50:19.140 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.140 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:19.140 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.140 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:19.140 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246746+0000", 2024-09-16T20:50:19.140 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.140 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.140 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.140 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:19.140 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246746+0000", 2024-09-16T20:50:19.141 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.141 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.141 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.141 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:19.141 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246746+0000", 2024-09-16T20:50:19.141 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8099999999999999e-07 2024-09-16T20:50:19.141 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.141 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.141 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:19.141 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246754+0000", 2024-09-16T20:50:19.141 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4440000000000002e-06 2024-09-16T20:50:19.141 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.141 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.141 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:19.141 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246758+0000", 2024-09-16T20:50:19.142 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3900000000000003e-06 2024-09-16T20:50:19.142 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.142 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.142 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:19.142 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246761+0000", 2024-09-16T20:50:19.142 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.101e-06 2024-09-16T20:50:19.142 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.142 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.142 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:19.142 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297310+0000", 2024-09-16T20:50:19.142 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050549166 2024-09-16T20:50:19.142 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.142 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.143 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:19.143 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297387+0000", 2024-09-16T20:50:19.143 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6658999999999999e-05 2024-09-16T20:50:19.143 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.143 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.143 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:19.143 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297477+0000", 2024-09-16T20:50:19.143 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9707e-05 2024-09-16T20:50:19.143 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.143 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:19.143 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.143 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.143 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.144 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:369 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 339968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:50:19.144 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246766+0000", 2024-09-16T20:50:19.144 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 42.833309217, 2024-09-16T20:50:19.144 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050911291999999997, 2024-09-16T20:50:19.144 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:19.144 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:19.144 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:19.144 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:19.144 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:50:19.144 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:50:19.144 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 369 2024-09-16T20:50:19.144 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.144 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:19.144 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.145 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:19.145 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246766+0000", 2024-09-16T20:50:19.145 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.145 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.145 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.145 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:19.145 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246766+0000", 2024-09-16T20:50:19.145 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.145 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.145 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.145 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:19.145 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246766+0000", 2024-09-16T20:50:19.145 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3200000000000001e-07 2024-09-16T20:50:19.145 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.146 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.146 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:19.146 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246767+0000", 2024-09-16T20:50:19.146 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.252e-06 2024-09-16T20:50:19.146 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.146 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.146 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:19.146 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246770+0000", 2024-09-16T20:50:19.146 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1049999999999999e-06 2024-09-16T20:50:19.146 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.146 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.146 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:19.146 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246773+0000", 2024-09-16T20:50:19.146 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.396e-06 2024-09-16T20:50:19.147 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.147 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.147 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:19.147 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297513+0000", 2024-09-16T20:50:19.147 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050740800000000003 2024-09-16T20:50:19.147 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.147 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.147 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:19.147 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297589+0000", 2024-09-16T20:50:19.147 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5318e-05 2024-09-16T20:50:19.147 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.147 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.147 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:19.147 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297677+0000", 2024-09-16T20:50:19.147 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8189000000000006e-05 2024-09-16T20:50:19.148 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.148 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:19.148 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.148 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.148 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.148 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:370 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 348160~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:50:19.148 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246776+0000", 2024-09-16T20:50:19.148 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 42.833299201999999, 2024-09-16T20:50:19.148 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051096643999999997, 2024-09-16T20:50:19.148 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:19.148 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:19.148 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:19.148 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:19.149 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:50:19.149 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:50:19.149 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 370 2024-09-16T20:50:19.149 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.149 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:19.149 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.149 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:19.149 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246776+0000", 2024-09-16T20:50:19.149 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.149 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.149 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.149 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:19.149 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246776+0000", 2024-09-16T20:50:19.150 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.150 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.150 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.150 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:19.150 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246776+0000", 2024-09-16T20:50:19.150 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2499999999999999e-07 2024-09-16T20:50:19.150 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.150 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.150 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:19.150 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246777+0000", 2024-09-16T20:50:19.150 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1510000000000001e-06 2024-09-16T20:50:19.150 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.150 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.150 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:19.151 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246780+0000", 2024-09-16T20:50:19.151 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9610000000000001e-06 2024-09-16T20:50:19.151 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.151 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.151 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:19.151 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246783+0000", 2024-09-16T20:50:19.151 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6129999999999998e-06 2024-09-16T20:50:19.151 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.151 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.151 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:19.151 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297713+0000", 2024-09-16T20:50:19.151 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050929950000000002 2024-09-16T20:50:19.151 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.151 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.152 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:19.152 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297786+0000", 2024-09-16T20:50:19.152 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3096000000000006e-05 2024-09-16T20:50:19.152 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.152 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.152 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:19.152 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297872+0000", 2024-09-16T20:50:19.152 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6648000000000001e-05 2024-09-16T20:50:19.152 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.152 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:19.152 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.152 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.152 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.152 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:371 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 380928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:50:19.152 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246791+0000", 2024-09-16T20:50:19.153 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 42.833283565999999, 2024-09-16T20:50:19.153 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051287209, 2024-09-16T20:50:19.153 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:19.153 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:19.153 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:19.153 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:19.153 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:50:19.153 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:50:19.153 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 371 2024-09-16T20:50:19.153 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.153 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:19.153 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.153 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:19.153 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246791+0000", 2024-09-16T20:50:19.154 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.154 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.154 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.154 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:19.154 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246791+0000", 2024-09-16T20:50:19.154 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.154 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.154 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.154 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:19.154 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246792+0000", 2024-09-16T20:50:19.154 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8200000000000001e-07 2024-09-16T20:50:19.154 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.154 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.154 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:19.154 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246793+0000", 2024-09-16T20:50:19.154 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.28e-06 2024-09-16T20:50:19.155 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.155 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.155 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:19.155 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246796+0000", 2024-09-16T20:50:19.155 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.597e-06 2024-09-16T20:50:19.155 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.155 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.155 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:19.155 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246799+0000", 2024-09-16T20:50:19.155 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7089999999999998e-06 2024-09-16T20:50:19.155 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.155 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.155 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:19.155 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297910+0000", 2024-09-16T20:50:19.155 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051110598 2024-09-16T20:50:19.156 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.156 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.156 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:19.156 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.297986+0000", 2024-09-16T20:50:19.156 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6711000000000005e-05 2024-09-16T20:50:19.156 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.156 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.156 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:19.156 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.298078+0000", 2024-09-16T20:50:19.156 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2032000000000002e-05 2024-09-16T20:50:19.156 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.156 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:19.156 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.156 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.156 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.156 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:373 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 376832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:50:19.157 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246904+0000", 2024-09-16T20:50:19.157 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 42.833171143999998, 2024-09-16T20:50:19.157 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051310617000000003, 2024-09-16T20:50:19.157 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:19.157 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:19.157 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:19.157 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:19.157 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:50:19.157 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:50:19.157 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 373 2024-09-16T20:50:19.157 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.157 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:19.157 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.157 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:19.157 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246904+0000", 2024-09-16T20:50:19.158 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.158 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.158 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.158 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:19.158 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246904+0000", 2024-09-16T20:50:19.158 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.158 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.158 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.158 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:19.158 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246904+0000", 2024-09-16T20:50:19.158 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8000000000000001e-07 2024-09-16T20:50:19.158 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.158 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.159 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:19.159 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246908+0000", 2024-09-16T20:50:19.159 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4039999999999997e-06 2024-09-16T20:50:19.159 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.159 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.159 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:19.159 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246913+0000", 2024-09-16T20:50:19.159 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0030000000000002e-06 2024-09-16T20:50:19.159 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.159 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.159 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:19.159 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246917+0000", 2024-09-16T20:50:19.159 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.303e-06 2024-09-16T20:50:19.159 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.159 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.160 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:19.160 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.298114+0000", 2024-09-16T20:50:19.160 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051197724 2024-09-16T20:50:19.160 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.160 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.160 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:19.160 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.298159+0000", 2024-09-16T20:50:19.160 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4882999999999997e-05 2024-09-16T20:50:19.160 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.160 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.160 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:19.160 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.298214+0000", 2024-09-16T20:50:19.160 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4920000000000003e-05 2024-09-16T20:50:19.160 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.161 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:19.161 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.161 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.161 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.161 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5512.0:374 3.f 3:f2c945a5:::rbd_data.110e39315da6.00000000000000cd:1f [read 389120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:50:19.161 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.246927+0000", 2024-09-16T20:50:19.161 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 42.833148094999999, 2024-09-16T20:50:19.161 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051498164999999999, 2024-09-16T20:50:19.161 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:19.161 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:19.161 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:19.161 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:19.161 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5512", 2024-09-16T20:50:19.161 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:50:19.161 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 374 2024-09-16T20:50:19.161 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.162 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:19.162 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.162 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:19.162 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246927+0000", 2024-09-16T20:50:19.162 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.162 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.162 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.162 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:19.162 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246927+0000", 2024-09-16T20:50:19.162 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.162 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.162 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.162 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:19.162 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246927+0000", 2024-09-16T20:50:19.162 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7e-07 2024-09-16T20:50:19.162 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.163 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.163 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:19.163 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246929+0000", 2024-09-16T20:50:19.163 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.607e-06 2024-09-16T20:50:19.163 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.163 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.163 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:19.163 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246932+0000", 2024-09-16T20:50:19.163 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4539999999999999e-06 2024-09-16T20:50:19.163 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.163 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.163 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:19.163 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.246935+0000", 2024-09-16T20:50:19.163 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6570000000000001e-06 2024-09-16T20:50:19.163 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.163 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.164 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:19.164 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.298242+0000", 2024-09-16T20:50:19.164 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051306887000000002 2024-09-16T20:50:19.164 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.164 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.164 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:19.164 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.298327+0000", 2024-09-16T20:50:19.164 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5068999999999995e-05 2024-09-16T20:50:19.164 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.164 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.164 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:19.164 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.298425+0000", 2024-09-16T20:50:19.164 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8021000000000002e-05 2024-09-16T20:50:19.164 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.164 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:19.164 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.165 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.165 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.165 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5763.0:4895 3.a 3:5384606c:::rbd_data.110e1b19a241.0000000000000018:head [read 0~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-16T20:50:19.165 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:45.598212+0000", 2024-09-16T20:50:19.165 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 33.481862438, 2024-09-16T20:50:19.165 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0375864829999999, 2024-09-16T20:50:19.165 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:19.165 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:19.165 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:19.165 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:19.165 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5763", 2024-09-16T20:50:19.165 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/623897567", 2024-09-16T20:50:19.165 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4895 2024-09-16T20:50:19.165 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.165 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:19.166 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.166 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:19.166 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598212+0000", 2024-09-16T20:50:19.166 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.166 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.166 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.166 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:19.166 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598212+0000", 2024-09-16T20:50:19.166 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.166 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.166 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.166 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:19.166 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598214+0000", 2024-09-16T20:50:19.166 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0090000000000002e-06 2024-09-16T20:50:19.166 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.166 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.167 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:19.167 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598222+0000", 2024-09-16T20:50:19.167 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6699999999999994e-06 2024-09-16T20:50:19.167 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.167 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.167 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:19.167 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598241+0000", 2024-09-16T20:50:19.167 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8896999999999999e-05 2024-09-16T20:50:19.167 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.167 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.167 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:19.167 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598250+0000", 2024-09-16T20:50:19.167 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6649999999999992e-06 2024-09-16T20:50:19.167 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.167 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.168 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:19.168 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598325+0000", 2024-09-16T20:50:19.168 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5189000000000001e-05 2024-09-16T20:50:19.168 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.168 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.168 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op must wait for map", 2024-09-16T20:50:19.168 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598333+0000", 2024-09-16T20:50:19.168 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.3170000000000002e-06 2024-09-16T20:50:19.168 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.168 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.168 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:19.168 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601355+0000", 2024-09-16T20:50:19.168 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0030215400000000001 2024-09-16T20:50:19.169 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.169 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.169 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:19.169 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601365+0000", 2024-09-16T20:50:19.169 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0215e-05 2024-09-16T20:50:19.169 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.169 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.169 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:19.169 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635506+0000", 2024-09-16T20:50:19.169 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0341408240000001 2024-09-16T20:50:19.169 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.169 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.169 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:19.169 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635655+0000", 2024-09-16T20:50:19.169 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000149412 2024-09-16T20:50:19.170 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.170 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.170 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:19.170 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635799+0000", 2024-09-16T20:50:19.170 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000143745 2024-09-16T20:50:19.170 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.170 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:19.170 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.170 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.170 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.170 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5763.0:4900 3.a 3:5384606c:::rbd_data.110e1b19a241.0000000000000018:head [read 8192~24576] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-16T20:50:19.170 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:50:19.170 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 33.481803962000001, 2024-09-16T20:50:19.170 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0379340800000001, 2024-09-16T20:50:19.171 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:19.171 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:19.171 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:19.171 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:19.171 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5763", 2024-09-16T20:50:19.171 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/623897567", 2024-09-16T20:50:19.171 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4900 2024-09-16T20:50:19.171 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.171 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:19.171 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.171 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:19.171 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:50:19.171 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.171 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.172 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.172 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:19.172 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:50:19.172 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.172 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.172 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.172 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:19.172 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:50:19.172 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4300000000000003e-07 2024-09-16T20:50:19.172 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.172 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.172 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:19.172 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598274+0000", 2024-09-16T20:50:19.172 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.976e-06 2024-09-16T20:50:19.173 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.173 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.173 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:19.173 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598282+0000", 2024-09-16T20:50:19.173 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0029999999999994e-06 2024-09-16T20:50:19.173 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.173 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.173 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:19.173 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598289+0000", 2024-09-16T20:50:19.173 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2389999999999999e-06 2024-09-16T20:50:19.173 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.173 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.173 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:19.173 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598430+0000", 2024-09-16T20:50:19.174 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014119600000000001 2024-09-16T20:50:19.174 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.174 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.174 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting_for_map not empty", 2024-09-16T20:50:19.174 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598435+0000", 2024-09-16T20:50:19.174 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3489999999999999e-06 2024-09-16T20:50:19.174 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.174 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.174 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:19.174 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601429+0000", 2024-09-16T20:50:19.174 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0029942720000000001 2024-09-16T20:50:19.174 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.174 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.174 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:19.175 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601438+0000", 2024-09-16T20:50:19.175 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4759999999999998e-06 2024-09-16T20:50:19.175 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.175 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.175 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:19.175 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635987+0000", 2024-09-16T20:50:19.175 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0345495010000001 2024-09-16T20:50:19.175 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.175 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.175 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:19.175 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.636056+0000", 2024-09-16T20:50:19.175 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8610999999999998e-05 2024-09-16T20:50:19.175 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.175 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.175 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:19.175 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.636205+0000", 2024-09-16T20:50:19.176 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014891399999999999 2024-09-16T20:50:19.176 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.176 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:19.176 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.176 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.176 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:19.176 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:19.177 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #41, image image_client.0-clone41 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent42 2024-09-16T20:50:19.177 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:50:19.178 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:19.175+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:19.278 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:19.276+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:19.325 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:19.325 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:50:19.325 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.325 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6213.0:3658 3.1a 3:5a74f802:::rbd_data.110e9f405c74.0000000000000123:head [read 245760~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-16T20:50:19.326 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:19.324909+0000", 2024-09-16T20:50:19.326 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.000104961, 2024-09-16T20:50:19.326 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014142000000000001, 2024-09-16T20:50:19.326 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:19.326 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:19.326 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:50:19.326 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:19.326 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6213", 2024-09-16T20:50:19.326 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3488506189", 2024-09-16T20:50:19.326 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3658 2024-09-16T20:50:19.326 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.326 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:19.326 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.326 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:19.327 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:19.324909+0000", 2024-09-16T20:50:19.327 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.327 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.327 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.327 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:19.327 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:19.324909+0000", 2024-09-16T20:50:19.327 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.327 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.327 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.327 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:19.327 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:19.324909+0000", 2024-09-16T20:50:19.327 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7400000000000002e-07 2024-09-16T20:50:19.327 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.328 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.328 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:19.328 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:19.324913+0000", 2024-09-16T20:50:19.328 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5269999999999999e-06 2024-09-16T20:50:19.328 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.328 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.328 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:19.328 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:19.324922+0000", 2024-09-16T20:50:19.328 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.5249999999999994e-06 2024-09-16T20:50:19.328 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.328 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.328 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:19.328 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:19.324929+0000", 2024-09-16T20:50:19.329 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0570000000000004e-06 2024-09-16T20:50:19.329 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.329 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.329 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:19.329 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:19.325009+0000", 2024-09-16T20:50:19.332 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9079999999999995e-05 2024-09-16T20:50:19.332 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.332 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:19.332 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.332 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.332 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.332 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6213.0:3661 3.14 3:2cd97b00:::rbd_data.110e9f405c74.0000000000000126:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-16T20:50:19.332 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:19.324968+0000", 2024-09-16T20:50:19.332 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.5651e-05, 2024-09-16T20:50:19.332 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000154809, 2024-09-16T20:50:19.332 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:19.332 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:19.333 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:50:19.333 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:19.333 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6213", 2024-09-16T20:50:19.333 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3488506189", 2024-09-16T20:50:19.333 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3661 2024-09-16T20:50:19.333 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.333 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:19.333 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.333 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:19.333 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:19.324968+0000", 2024-09-16T20:50:19.333 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.333 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.333 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.334 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:19.334 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:19.324968+0000", 2024-09-16T20:50:19.334 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.334 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.334 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.334 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:19.334 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:19.324969+0000", 2024-09-16T20:50:19.334 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3499999999999996e-07 2024-09-16T20:50:19.334 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.334 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.334 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:19.334 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:19.324972+0000", 2024-09-16T20:50:19.334 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.388e-06 2024-09-16T20:50:19.334 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.335 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.335 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:19.335 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:19.324980+0000", 2024-09-16T20:50:19.335 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.551e-06 2024-09-16T20:50:19.335 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.335 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.335 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:19.335 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:19.324986+0000", 2024-09-16T20:50:19.335 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9109999999999998e-06 2024-09-16T20:50:19.335 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.335 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.335 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:19.335 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:19.325036+0000", 2024-09-16T20:50:19.336 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0040000000000002e-05 2024-09-16T20:50:19.336 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.336 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.336 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:19.336 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:19.325127+0000", 2024-09-16T20:50:19.336 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0989000000000006e-05 2024-09-16T20:50:19.336 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.336 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:19.336 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.336 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.336 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-16T20:50:19.336 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 2 2024-09-16T20:50:19.336 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:19.338 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:50:19.379 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:19.377+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:19.479 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:19.478+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:19.487 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:19.487 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:50:19.487 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:50:19.487 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:50:19.487 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:19.497 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:50:19.580 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:19.578+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:19.671 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:19.671 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:50:19.671 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:50:19.671 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:50:19.671 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.671 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:50:19.671 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:50:19.671 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 146.41505980100001, 2024-09-16T20:50:19.672 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67502346999999996, 2024-09-16T20:50:19.672 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:19.672 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:19.672 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:19.672 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:19.672 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:50:19.672 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:50:19.672 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 1 2024-09-16T20:50:19.672 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.672 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:19.672 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.672 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:19.672 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:50:19.673 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.673 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.673 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.673 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:19.673 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:50:19.673 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.673 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.673 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.673 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:19.673 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254128+0000", 2024-09-16T20:50:19.673 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.463e-06 2024-09-16T20:50:19.673 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.673 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.673 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:19.674 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254135+0000", 2024-09-16T20:50:19.674 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1579999999999999e-06 2024-09-16T20:50:19.674 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.674 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.674 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:19.674 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254146+0000", 2024-09-16T20:50:19.674 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1341e-05 2024-09-16T20:50:19.674 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.674 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.674 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:19.674 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254155+0000", 2024-09-16T20:50:19.674 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.1549999999999996e-06 2024-09-16T20:50:19.674 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.675 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.675 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:19.675 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254221+0000", 2024-09-16T20:50:19.675 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5470999999999997e-05 2024-09-16T20:50:19.675 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.675 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.675 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:19.675 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254231+0000", 2024-09-16T20:50:19.675 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0592e-05 2024-09-16T20:50:19.675 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.675 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.675 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:19.675 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926711+0000", 2024-09-16T20:50:19.675 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67247946599999997 2024-09-16T20:50:19.676 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.676 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.676 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:19.676 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926910+0000", 2024-09-16T20:50:19.676 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019934 2024-09-16T20:50:19.676 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.676 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.676 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:50:19.676 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.927159+0000", 2024-09-16T20:50:19.676 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00024876599999999998 2024-09-16T20:50:19.676 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.676 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.676 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:19.677 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.928797+0000", 2024-09-16T20:50:19.677 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001638308 2024-09-16T20:50:19.677 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.677 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.677 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:19.677 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929026+0000", 2024-09-16T20:50:19.677 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00022876099999999999 2024-09-16T20:50:19.677 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.677 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.677 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:19.677 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929102+0000", 2024-09-16T20:50:19.677 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6184000000000001e-05 2024-09-16T20:50:19.677 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.677 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.678 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:19.678 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929150+0000", 2024-09-16T20:50:19.678 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7465e-05 2024-09-16T20:50:19.678 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.678 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:19.678 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.678 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.678 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.678 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3574 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:19.678 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:50:19.678 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 90.187896269000007, 2024-09-16T20:50:19.678 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053579441999999998, 2024-09-16T20:50:19.678 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:19.679 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:19.679 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:19.679 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:19.679 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:19.679 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:19.679 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3574 2024-09-16T20:50:19.679 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.679 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:19.679 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.679 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:19.679 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:50:19.679 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.679 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.680 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.680 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:19.680 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:50:19.680 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.680 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.680 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.680 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:19.680 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:50:19.680 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2e-07 2024-09-16T20:50:19.680 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.680 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.680 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:19.680 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481292+0000", 2024-09-16T20:50:19.680 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2630000000000002e-06 2024-09-16T20:50:19.681 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.681 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.681 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:19.681 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481297+0000", 2024-09-16T20:50:19.681 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.296e-06 2024-09-16T20:50:19.681 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.681 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.681 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:19.681 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481300+0000", 2024-09-16T20:50:19.681 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7019999999999999e-06 2024-09-16T20:50:19.681 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.681 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.681 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:19.682 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525533+0000", 2024-09-16T20:50:19.682 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.044232318 2024-09-16T20:50:19.682 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.682 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.682 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:19.682 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525619+0000", 2024-09-16T20:50:19.682 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5892999999999999e-05 2024-09-16T20:50:19.682 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.682 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.682 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:19.682 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525987+0000", 2024-09-16T20:50:19.682 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00036836099999999999 2024-09-16T20:50:19.682 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.683 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.683 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:19.683 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531779+0000", 2024-09-16T20:50:19.683 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0057918240000000001 2024-09-16T20:50:19.683 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.683 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.683 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:19.683 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534747+0000", 2024-09-16T20:50:19.683 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0029681600000000001 2024-09-16T20:50:19.683 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.683 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.683 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:19.683 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534832+0000", 2024-09-16T20:50:19.683 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5180999999999996e-05 2024-09-16T20:50:19.683 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.684 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.684 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:19.684 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534869+0000", 2024-09-16T20:50:19.684 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7024000000000003e-05 2024-09-16T20:50:19.684 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.684 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:19.684 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.684 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.684 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.684 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3575 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:19.684 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:50:19.684 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 90.187881528999995, 2024-09-16T20:50:19.684 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053801463000000001, 2024-09-16T20:50:19.685 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:19.685 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:19.685 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:19.685 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:19.685 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:19.685 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:19.685 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3575 2024-09-16T20:50:19.685 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.685 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:19.685 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.685 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:19.685 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:50:19.685 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.686 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.686 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.686 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:19.686 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:50:19.686 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.686 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.686 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.686 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:19.686 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481305+0000", 2024-09-16T20:50:19.686 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5400000000000002e-07 2024-09-16T20:50:19.686 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.686 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.686 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:19.686 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481307+0000", 2024-09-16T20:50:19.687 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7800000000000001e-06 2024-09-16T20:50:19.687 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.687 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.687 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:19.687 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481312+0000", 2024-09-16T20:50:19.687 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6349999999999997e-06 2024-09-16T20:50:19.687 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.687 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.687 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:19.687 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481315+0000", 2024-09-16T20:50:19.687 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7829999999999999e-06 2024-09-16T20:50:19.687 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.687 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.687 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:19.688 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.526876+0000", 2024-09-16T20:50:19.688 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.045561070000000002 2024-09-16T20:50:19.688 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.688 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.688 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:19.688 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.526954+0000", 2024-09-16T20:50:19.688 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8108000000000005e-05 2024-09-16T20:50:19.688 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.688 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.688 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:19.688 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527282+0000", 2024-09-16T20:50:19.688 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000327483 2024-09-16T20:50:19.688 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.689 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.689 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:19.689 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531784+0000", 2024-09-16T20:50:19.689 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0045020169999999997 2024-09-16T20:50:19.689 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.689 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.689 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:19.689 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534921+0000", 2024-09-16T20:50:19.689 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.003137791 2024-09-16T20:50:19.689 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.689 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.689 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:19.689 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534976+0000", 2024-09-16T20:50:19.689 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4953000000000001e-05 2024-09-16T20:50:19.690 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.690 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.690 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:19.690 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535106+0000", 2024-09-16T20:50:19.690 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000129589 2024-09-16T20:50:19.690 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.690 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:19.690 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.690 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.690 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.690 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3576 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:19.690 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:50:19.690 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 90.187867616999995, 2024-09-16T20:50:19.691 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053955436000000002, 2024-09-16T20:50:19.691 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:19.691 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:19.691 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:19.691 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:19.691 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:19.691 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:19.691 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3576 2024-09-16T20:50:19.691 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.691 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:19.691 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.691 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:19.691 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:50:19.691 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.692 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.692 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.692 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:19.692 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:50:19.692 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.692 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.692 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.692 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:19.692 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481319+0000", 2024-09-16T20:50:19.692 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.53e-07 2024-09-16T20:50:19.692 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.692 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.692 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:19.692 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481320+0000", 2024-09-16T20:50:19.693 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5230000000000001e-06 2024-09-16T20:50:19.693 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.693 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.693 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:19.693 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481330+0000", 2024-09-16T20:50:19.693 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0190000000000001e-05 2024-09-16T20:50:19.693 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.693 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.693 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:19.693 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481333+0000", 2024-09-16T20:50:19.693 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.151e-06 2024-09-16T20:50:19.693 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.693 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.694 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:19.694 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527689+0000", 2024-09-16T20:50:19.694 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046355192000000003 2024-09-16T20:50:19.694 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.694 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.694 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:19.694 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527764+0000", 2024-09-16T20:50:19.694 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5776000000000001e-05 2024-09-16T20:50:19.694 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.694 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.694 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:19.694 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528141+0000", 2024-09-16T20:50:19.694 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00037647599999999999 2024-09-16T20:50:19.694 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.694 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.695 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:19.695 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531788+0000", 2024-09-16T20:50:19.695 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0036470679999999998 2024-09-16T20:50:19.695 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.695 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.695 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:19.695 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535174+0000", 2024-09-16T20:50:19.695 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0033862670000000001 2024-09-16T20:50:19.695 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.695 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.695 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:19.695 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535236+0000", 2024-09-16T20:50:19.695 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1797999999999993e-05 2024-09-16T20:50:19.696 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.696 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.696 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:19.696 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535274+0000", 2024-09-16T20:50:19.696 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7741999999999999e-05 2024-09-16T20:50:19.696 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.696 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:19.696 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.696 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.696 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.696 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3577 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:19.696 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:50:19.696 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 90.187846016999998, 2024-09-16T20:50:19.696 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054073807000000002, 2024-09-16T20:50:19.697 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:19.697 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:19.697 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:19.697 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:19.697 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:19.697 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:19.697 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3577 2024-09-16T20:50:19.697 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.697 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:19.697 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.697 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:19.697 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:50:19.697 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.698 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.698 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.698 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:19.698 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:50:19.698 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.698 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.698 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.698 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:19.698 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:50:19.698 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.96e-07 2024-09-16T20:50:19.698 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.698 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.698 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:19.698 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481342+0000", 2024-09-16T20:50:19.699 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.714e-06 2024-09-16T20:50:19.699 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.699 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.699 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:19.699 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481349+0000", 2024-09-16T20:50:19.699 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4930000000000001e-06 2024-09-16T20:50:19.699 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.699 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.699 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:19.699 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481353+0000", 2024-09-16T20:50:19.699 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4879999999999999e-06 2024-09-16T20:50:19.699 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.699 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.700 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:19.700 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528619+0000", 2024-09-16T20:50:19.700 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047266351999999998 2024-09-16T20:50:19.700 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.700 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.700 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:19.700 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528706+0000", 2024-09-16T20:50:19.700 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6796e-05 2024-09-16T20:50:19.700 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.700 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.700 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:19.700 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529148+0000", 2024-09-16T20:50:19.700 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00044182999999999999 2024-09-16T20:50:19.700 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.701 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.701 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:19.701 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531792+0000", 2024-09-16T20:50:19.701 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0026443119999999998 2024-09-16T20:50:19.701 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.701 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.701 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:19.701 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535319+0000", 2024-09-16T20:50:19.701 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0035266439999999998 2024-09-16T20:50:19.701 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.701 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.701 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:19.701 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535372+0000", 2024-09-16T20:50:19.702 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3207000000000001e-05 2024-09-16T20:50:19.702 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.702 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.702 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:19.702 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535414+0000", 2024-09-16T20:50:19.702 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1675000000000003e-05 2024-09-16T20:50:19.702 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.702 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:19.702 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.702 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.702 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.702 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3578 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:19.702 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:50:19.702 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 90.187824848000005, 2024-09-16T20:50:19.703 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054189586999999997, 2024-09-16T20:50:19.703 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:19.703 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:19.703 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:19.703 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:19.703 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:19.703 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:19.703 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3578 2024-09-16T20:50:19.703 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.703 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:19.703 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.703 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:19.703 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:50:19.703 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.704 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.704 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.704 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:19.704 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:50:19.704 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.704 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.704 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.704 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:19.704 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:50:19.704 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0800000000000001e-07 2024-09-16T20:50:19.704 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.704 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.704 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:19.705 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481363+0000", 2024-09-16T20:50:19.705 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0339999999999999e-06 2024-09-16T20:50:19.705 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.705 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.705 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:19.705 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481371+0000", 2024-09-16T20:50:19.705 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4440000000000002e-06 2024-09-16T20:50:19.705 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.705 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.705 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:19.705 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481374+0000", 2024-09-16T20:50:19.705 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2049999999999998e-06 2024-09-16T20:50:19.705 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.705 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.706 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:19.706 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529555+0000", 2024-09-16T20:50:19.706 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048181202999999999 2024-09-16T20:50:19.706 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.706 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.706 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:19.706 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529616+0000", 2024-09-16T20:50:19.706 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0912000000000002e-05 2024-09-16T20:50:19.706 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.706 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.706 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:19.706 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529939+0000", 2024-09-16T20:50:19.706 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00032269800000000001 2024-09-16T20:50:19.706 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.707 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.707 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:19.707 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532602+0000", 2024-09-16T20:50:19.707 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0026634940000000002 2024-09-16T20:50:19.707 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.707 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.707 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:19.707 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535465+0000", 2024-09-16T20:50:19.707 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.002862149 2024-09-16T20:50:19.707 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.707 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.707 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:19.707 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535516+0000", 2024-09-16T20:50:19.708 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1696000000000001e-05 2024-09-16T20:50:19.708 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.708 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.708 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:19.708 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535551+0000", 2024-09-16T20:50:19.708 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4443999999999998e-05 2024-09-16T20:50:19.708 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.708 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:19.708 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.708 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.708 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.708 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3579 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:19.708 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:50:19.709 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 90.187806409000004, 2024-09-16T20:50:19.709 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054295151, 2024-09-16T20:50:19.709 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:19.709 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:19.709 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:19.709 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:19.709 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:19.709 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:19.709 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3579 2024-09-16T20:50:19.709 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.709 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:19.709 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.709 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:19.709 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:50:19.709 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.710 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.710 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.710 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:19.710 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:50:19.710 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.710 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.710 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.710 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:19.710 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:50:19.710 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3300000000000001e-07 2024-09-16T20:50:19.710 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.710 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.710 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:19.710 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481381+0000", 2024-09-16T20:50:19.711 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.649e-06 2024-09-16T20:50:19.711 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.711 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.711 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:19.711 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481387+0000", 2024-09-16T20:50:19.711 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4519999999999996e-06 2024-09-16T20:50:19.711 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.711 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.711 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:19.711 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481390+0000", 2024-09-16T20:50:19.711 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0170000000000001e-06 2024-09-16T20:50:19.711 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.711 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.711 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:19.712 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530756+0000", 2024-09-16T20:50:19.712 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049365882999999999 2024-09-16T20:50:19.712 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.712 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.712 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:19.712 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530845+0000", 2024-09-16T20:50:19.712 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9197000000000007e-05 2024-09-16T20:50:19.712 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.712 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.712 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:19.712 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531191+0000", 2024-09-16T20:50:19.712 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00034562500000000001 2024-09-16T20:50:19.712 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.713 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.713 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:19.713 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534078+0000", 2024-09-16T20:50:19.713 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0028878179999999999 2024-09-16T20:50:19.713 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.713 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.713 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:19.713 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535589+0000", 2024-09-16T20:50:19.713 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00151085 2024-09-16T20:50:19.713 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.713 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.713 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:19.713 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535639+0000", 2024-09-16T20:50:19.714 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9697000000000002e-05 2024-09-16T20:50:19.714 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.714 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.714 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:19.714 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535675+0000", 2024-09-16T20:50:19.714 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5729999999999998e-05 2024-09-16T20:50:19.714 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.714 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:19.714 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.714 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.714 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.714 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3580 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:19.714 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:50:19.714 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 90.187792776999999, 2024-09-16T20:50:19.715 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054397361999999998, 2024-09-16T20:50:19.715 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:19.715 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:19.715 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:19.715 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:19.715 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:19.715 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:19.715 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3580 2024-09-16T20:50:19.715 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.715 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:19.715 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.715 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:19.715 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:50:19.716 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.716 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.716 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.716 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:19.716 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:50:19.716 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.716 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.716 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.716 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:19.716 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:50:19.716 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2600000000000001e-07 2024-09-16T20:50:19.716 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.717 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.717 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:19.717 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481395+0000", 2024-09-16T20:50:19.717 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.544e-06 2024-09-16T20:50:19.717 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.717 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.717 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:19.718 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481404+0000", 2024-09-16T20:50:19.718 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8850000000000005e-06 2024-09-16T20:50:19.718 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.718 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.718 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:19.718 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481407+0000", 2024-09-16T20:50:19.718 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1540000000000002e-06 2024-09-16T20:50:19.718 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.718 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.718 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:19.718 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531823+0000", 2024-09-16T20:50:19.718 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050415637999999999 2024-09-16T20:50:19.718 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.719 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.719 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:19.719 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531895+0000", 2024-09-16T20:50:19.719 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2179000000000001e-05 2024-09-16T20:50:19.719 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.719 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.719 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:19.719 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532236+0000", 2024-09-16T20:50:19.719 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00034073799999999999 2024-09-16T20:50:19.719 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.719 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.719 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:19.719 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534083+0000", 2024-09-16T20:50:19.720 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001847298 2024-09-16T20:50:19.720 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.720 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.720 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:19.720 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535714+0000", 2024-09-16T20:50:19.720 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0016316810000000001 2024-09-16T20:50:19.720 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.720 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.720 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:19.720 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535759+0000", 2024-09-16T20:50:19.720 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4478999999999999e-05 2024-09-16T20:50:19.720 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.721 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.721 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:19.721 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535791+0000", 2024-09-16T20:50:19.721 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.154e-05 2024-09-16T20:50:19.721 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.721 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:19.721 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.721 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.721 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.721 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3581 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:19.721 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:50:19.721 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 90.187770779999994, 2024-09-16T20:50:19.721 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.057094403000000002, 2024-09-16T20:50:19.721 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:19.722 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:19.722 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:19.722 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:19.722 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:19.722 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:19.722 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3581 2024-09-16T20:50:19.722 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.722 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:19.722 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.722 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:19.722 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:50:19.722 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.723 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.723 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.723 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:19.723 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:50:19.723 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.723 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.723 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.723 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:19.723 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:50:19.723 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8799999999999998e-07 2024-09-16T20:50:19.723 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.723 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.723 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:19.723 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481418+0000", 2024-09-16T20:50:19.723 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1859999999999999e-06 2024-09-16T20:50:19.724 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.724 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.724 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:19.724 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481423+0000", 2024-09-16T20:50:19.724 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8279999999999999e-06 2024-09-16T20:50:19.724 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.724 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.724 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:19.724 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481426+0000", 2024-09-16T20:50:19.724 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0110000000000001e-06 2024-09-16T20:50:19.724 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.724 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.724 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:19.724 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532631+0000", 2024-09-16T20:50:19.724 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051204514999999999 2024-09-16T20:50:19.725 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.725 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.725 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:19.725 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.533381+0000", 2024-09-16T20:50:19.725 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00075029000000000003 2024-09-16T20:50:19.725 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.725 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.725 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:19.725 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.533739+0000", 2024-09-16T20:50:19.725 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00035773100000000002 2024-09-16T20:50:19.725 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.725 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.725 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:19.726 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535123+0000", 2024-09-16T20:50:19.726 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0013836160000000001 2024-09-16T20:50:19.726 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.726 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.726 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:19.726 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538413+0000", 2024-09-16T20:50:19.726 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0032902510000000001 2024-09-16T20:50:19.726 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.726 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.726 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:19.726 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538470+0000", 2024-09-16T20:50:19.726 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6697999999999998e-05 2024-09-16T20:50:19.726 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.726 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.726 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:19.727 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538510+0000", 2024-09-16T20:50:19.727 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9988999999999999e-05 2024-09-16T20:50:19.727 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.727 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:19.727 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.727 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.727 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.727 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:12732 3.d 3:b752444a:::rbd_header.110e747f0779:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-16T20:50:19.727 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:50:19.727 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 74.900698202000001, 2024-09-16T20:50:19.727 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.065302467000000003, 2024-09-16T20:50:19.727 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:19.727 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:19.728 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:19.728 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:19.728 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:50:19.728 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:50:19.728 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 12732 2024-09-16T20:50:19.728 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.728 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:19.728 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.728 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:19.728 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:50:19.728 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.728 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.728 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.729 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:19.729 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:50:19.729 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.729 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.729 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.729 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:19.729 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768490+0000", 2024-09-16T20:50:19.729 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2859999999999998e-06 2024-09-16T20:50:19.729 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.729 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.729 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:19.729 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768501+0000", 2024-09-16T20:50:19.729 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1226999999999999e-05 2024-09-16T20:50:19.730 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.730 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.730 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:19.730 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768517+0000", 2024-09-16T20:50:19.730 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.592e-05 2024-09-16T20:50:19.730 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.730 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.730 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:19.730 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768528+0000", 2024-09-16T20:50:19.730 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0689999999999999e-05 2024-09-16T20:50:19.730 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.730 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.730 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:19.730 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768584+0000", 2024-09-16T20:50:19.731 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5889e-05 2024-09-16T20:50:19.731 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.731 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.731 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:19.731 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768720+0000", 2024-09-16T20:50:19.731 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000136718 2024-09-16T20:50:19.731 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.731 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.731 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:19.731 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.769066+0000", 2024-09-16T20:50:19.731 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000346013 2024-09-16T20:50:19.731 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.731 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.732 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:19.732 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833659+0000", 2024-09-16T20:50:19.732 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.064592822999999994 2024-09-16T20:50:19.732 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.732 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.732 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:19.732 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833719+0000", 2024-09-16T20:50:19.732 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9441000000000002e-05 2024-09-16T20:50:19.732 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.732 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.732 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:19.732 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833758+0000", 2024-09-16T20:50:19.732 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9122000000000003e-05 2024-09-16T20:50:19.732 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.733 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.733 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:19.733 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833790+0000", 2024-09-16T20:50:19.733 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2338000000000001e-05 2024-09-16T20:50:19.733 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.733 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:19.733 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.733 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.733 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.733 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.5089.0:12732 3.d e73/23 ondisk, result = 0)", 2024-09-16T20:50:19.733 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:50:19.733 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 74.899096032000003, 2024-09-16T20:50:19.733 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.063706456999999994, 2024-09-16T20:50:19.734 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:19.734 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:19.734 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:19.734 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:19.734 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.734 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:19.734 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:50:19.734 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.734 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.734 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.734 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:19.734 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:50:19.734 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.734 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.735 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.735 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:19.735 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770091+0000", 2024-09-16T20:50:19.735 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.53e-06 2024-09-16T20:50:19.735 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.735 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.735 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:19.735 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770104+0000", 2024-09-16T20:50:19.735 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2203e-05 2024-09-16T20:50:19.735 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.735 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.735 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:19.735 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833637+0000", 2024-09-16T20:50:19.736 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.063533201999999997 2024-09-16T20:50:19.736 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.736 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.736 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:19.736 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833655+0000", 2024-09-16T20:50:19.736 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8369e-05 2024-09-16T20:50:19.736 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.736 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.736 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:19.736 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833707+0000", 2024-09-16T20:50:19.736 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2061999999999997e-05 2024-09-16T20:50:19.736 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.736 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.736 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:19.737 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833715+0000", 2024-09-16T20:50:19.737 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5179999999999998e-06 2024-09-16T20:50:19.737 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.737 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.737 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:19.737 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833796+0000", 2024-09-16T20:50:19.737 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1573e-05 2024-09-16T20:50:19.737 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.739 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:19.740 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.740 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.740 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.740 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3281 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 220160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:19.740 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041070+0000", 2024-09-16T20:50:19.740 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 29.628115563000001, 2024-09-16T20:50:19.740 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051593550000000002, 2024-09-16T20:50:19.740 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:19.740 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:19.740 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:19.740 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:19.740 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:19.740 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:19.741 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3281 2024-09-16T20:50:19.741 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.741 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:19.741 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.741 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:19.741 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041070+0000", 2024-09-16T20:50:19.741 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.741 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.741 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.741 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:19.741 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041070+0000", 2024-09-16T20:50:19.741 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.741 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.741 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.742 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:19.742 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041071+0000", 2024-09-16T20:50:19.742 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1899999999999999e-07 2024-09-16T20:50:19.742 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.742 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.742 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:19.742 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041072+0000", 2024-09-16T20:50:19.742 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.248e-06 2024-09-16T20:50:19.742 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.742 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.742 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:19.742 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041075+0000", 2024-09-16T20:50:19.742 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0029999999999999e-06 2024-09-16T20:50:19.743 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.743 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.743 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:19.743 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041077+0000", 2024-09-16T20:50:19.743 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.621e-06 2024-09-16T20:50:19.743 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.743 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.743 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:19.743 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.092431+0000", 2024-09-16T20:50:19.743 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051353801999999997 2024-09-16T20:50:19.743 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.743 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.743 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:19.743 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.092512+0000", 2024-09-16T20:50:19.744 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1106000000000006e-05 2024-09-16T20:50:19.744 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.744 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.744 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:19.744 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.092664+0000", 2024-09-16T20:50:19.744 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00015155100000000001 2024-09-16T20:50:19.744 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.744 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:19.744 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.744 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.744 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.744 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3282 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 225280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:19.745 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041081+0000", 2024-09-16T20:50:19.745 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 29.628105285, 2024-09-16T20:50:19.745 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051731948, 2024-09-16T20:50:19.745 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:19.745 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:19.745 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:19.745 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:19.745 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:19.745 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:19.745 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3282 2024-09-16T20:50:19.745 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.745 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:19.745 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.745 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:19.746 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041081+0000", 2024-09-16T20:50:19.746 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.746 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.746 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.746 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:19.746 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041081+0000", 2024-09-16T20:50:19.746 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.746 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.746 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.746 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:19.746 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041081+0000", 2024-09-16T20:50:19.746 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3099999999999999e-07 2024-09-16T20:50:19.746 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.746 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.747 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:19.747 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041082+0000", 2024-09-16T20:50:19.747 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1289999999999999e-06 2024-09-16T20:50:19.747 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.747 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.747 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:19.747 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041085+0000", 2024-09-16T20:50:19.747 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2320000000000001e-06 2024-09-16T20:50:19.747 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.747 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.747 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:19.747 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041090+0000", 2024-09-16T20:50:19.747 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3730000000000003e-06 2024-09-16T20:50:19.748 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.748 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.748 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:19.748 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.092678+0000", 2024-09-16T20:50:19.748 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051588235000000003 2024-09-16T20:50:19.748 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.748 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.748 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:19.748 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.092717+0000", 2024-09-16T20:50:19.748 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9057999999999998e-05 2024-09-16T20:50:19.748 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.748 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.748 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:19.749 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.092813+0000", 2024-09-16T20:50:19.749 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.569e-05 2024-09-16T20:50:19.749 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.749 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:19.749 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.749 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.749 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.749 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3283 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 230400~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:19.749 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041102+0000", 2024-09-16T20:50:19.749 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 29.628083624999999, 2024-09-16T20:50:19.749 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051974723, 2024-09-16T20:50:19.749 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:19.749 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:19.749 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:19.749 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:19.750 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:19.750 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:19.750 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3283 2024-09-16T20:50:19.750 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.750 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:19.750 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.750 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:19.750 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041102+0000", 2024-09-16T20:50:19.750 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.750 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.750 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.750 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:19.750 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041102+0000", 2024-09-16T20:50:19.750 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.751 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.751 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.751 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:19.751 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041103+0000", 2024-09-16T20:50:19.751 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2399999999999999e-07 2024-09-16T20:50:19.751 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.751 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.751 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:19.751 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041110+0000", 2024-09-16T20:50:19.751 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4290000000000004e-06 2024-09-16T20:50:19.751 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.751 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.751 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:19.752 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041118+0000", 2024-09-16T20:50:19.752 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3849999999999996e-06 2024-09-16T20:50:19.752 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.752 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.752 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:19.752 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041123+0000", 2024-09-16T20:50:19.752 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1410000000000004e-06 2024-09-16T20:50:19.752 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.752 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.752 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:19.752 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.092845+0000", 2024-09-16T20:50:19.752 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051722207999999999 2024-09-16T20:50:19.752 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.752 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.752 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:19.753 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.092926+0000", 2024-09-16T20:50:19.753 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0727000000000001e-05 2024-09-16T20:50:19.753 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.753 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.753 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:19.753 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093077+0000", 2024-09-16T20:50:19.753 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000151409 2024-09-16T20:50:19.753 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.753 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:19.753 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.753 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.753 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.753 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3284 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 235520~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:19.754 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041129+0000", 2024-09-16T20:50:19.754 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 29.628057294000001, 2024-09-16T20:50:19.754 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052120760000000002, 2024-09-16T20:50:19.754 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:19.754 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:19.754 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:19.754 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:19.754 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:19.754 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:19.754 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3284 2024-09-16T20:50:19.754 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.754 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:19.754 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.754 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:19.755 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041129+0000", 2024-09-16T20:50:19.755 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.755 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.755 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.755 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:19.755 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041129+0000", 2024-09-16T20:50:19.755 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.755 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.755 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.755 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:19.755 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041129+0000", 2024-09-16T20:50:19.755 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3700000000000001e-07 2024-09-16T20:50:19.755 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.755 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.755 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:19.755 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041130+0000", 2024-09-16T20:50:19.756 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4050000000000001e-06 2024-09-16T20:50:19.756 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.756 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.756 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:19.756 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041134+0000", 2024-09-16T20:50:19.756 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4290000000000001e-06 2024-09-16T20:50:19.756 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.756 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.756 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:19.756 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041136+0000", 2024-09-16T20:50:19.756 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.678e-06 2024-09-16T20:50:19.756 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.756 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.756 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:19.756 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093109+0000", 2024-09-16T20:50:19.757 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051972879999999999 2024-09-16T20:50:19.757 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.757 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.757 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:19.757 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093149+0000", 2024-09-16T20:50:19.757 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0040000000000003e-05 2024-09-16T20:50:19.757 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.757 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.757 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:19.757 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093249+0000", 2024-09-16T20:50:19.757 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.9990999999999999e-05 2024-09-16T20:50:19.757 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.757 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:19.757 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.758 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.758 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.758 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3285 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 240640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:19.758 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041140+0000", 2024-09-16T20:50:19.758 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 29.628046254000001, 2024-09-16T20:50:19.758 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052375646999999997, 2024-09-16T20:50:19.758 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:19.758 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:19.758 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:19.758 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:19.758 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:19.758 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:19.758 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3285 2024-09-16T20:50:19.759 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.759 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:19.759 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.759 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:19.759 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041140+0000", 2024-09-16T20:50:19.759 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.759 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.759 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.759 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:19.759 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041140+0000", 2024-09-16T20:50:19.759 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.759 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.759 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.760 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:19.760 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041140+0000", 2024-09-16T20:50:19.760 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2399999999999999e-07 2024-09-16T20:50:19.760 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.760 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.760 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:19.760 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041141+0000", 2024-09-16T20:50:19.760 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2759999999999999e-06 2024-09-16T20:50:19.760 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.760 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.760 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:19.760 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041144+0000", 2024-09-16T20:50:19.760 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0280000000000001e-06 2024-09-16T20:50:19.761 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.761 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.761 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:19.761 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041147+0000", 2024-09-16T20:50:19.761 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5900000000000002e-06 2024-09-16T20:50:19.761 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.761 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.761 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:19.761 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093284+0000", 2024-09-16T20:50:19.761 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052137645000000003 2024-09-16T20:50:19.761 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.761 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.761 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:19.762 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093365+0000", 2024-09-16T20:50:19.762 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0446999999999997e-05 2024-09-16T20:50:19.762 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.762 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.762 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:19.762 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093515+0000", 2024-09-16T20:50:19.762 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00015043700000000001 2024-09-16T20:50:19.762 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.762 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:19.762 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.762 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.762 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.762 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3286 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 245760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:19.763 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:50:19.763 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 29.628034095, 2024-09-16T20:50:19.763 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052523564000000002, 2024-09-16T20:50:19.763 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:19.763 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:19.763 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:19.763 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:19.763 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:19.763 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:19.763 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3286 2024-09-16T20:50:19.763 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.763 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:19.763 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.763 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:19.764 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:50:19.764 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.764 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.764 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.764 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:19.764 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:50:19.764 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.764 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.764 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.764 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:19.764 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:50:19.764 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3900000000000001e-07 2024-09-16T20:50:19.764 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.765 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.765 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:19.765 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041153+0000", 2024-09-16T20:50:19.765 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.119e-06 2024-09-16T20:50:19.765 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.765 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.765 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:19.765 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041156+0000", 2024-09-16T20:50:19.765 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0000000000000001e-06 2024-09-16T20:50:19.765 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.765 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.765 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:19.765 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041159+0000", 2024-09-16T20:50:19.765 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5859999999999999e-06 2024-09-16T20:50:19.765 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.766 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.766 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:19.766 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093529+0000", 2024-09-16T20:50:19.766 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052370591000000001 2024-09-16T20:50:19.766 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.766 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.766 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:19.766 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093570+0000", 2024-09-16T20:50:19.766 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0161999999999999e-05 2024-09-16T20:50:19.766 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.766 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.766 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:19.766 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093675+0000", 2024-09-16T20:50:19.766 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010586699999999999 2024-09-16T20:50:19.766 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.767 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:19.767 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.767 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.767 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.767 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3287 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 250880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:19.767 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:50:19.767 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 29.628024046, 2024-09-16T20:50:19.767 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052781203999999998, 2024-09-16T20:50:19.767 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:19.767 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:19.767 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:19.767 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:19.767 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:19.767 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:19.768 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3287 2024-09-16T20:50:19.768 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.768 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:19.768 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.768 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:19.768 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:50:19.768 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.768 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.768 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.768 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:19.768 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:50:19.768 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.769 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.769 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.769 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:19.769 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:50:19.769 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1e-07 2024-09-16T20:50:19.769 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.769 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.769 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:19.769 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041163+0000", 2024-09-16T20:50:19.769 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0690000000000001e-06 2024-09-16T20:50:19.769 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.769 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.769 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:19.770 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041166+0000", 2024-09-16T20:50:19.770 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.982e-06 2024-09-16T20:50:19.770 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.770 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.770 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:19.770 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041169+0000", 2024-09-16T20:50:19.770 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.593e-06 2024-09-16T20:50:19.770 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.770 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.770 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:19.770 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093709+0000", 2024-09-16T20:50:19.770 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052540492000000001 2024-09-16T20:50:19.770 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.770 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.770 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:19.771 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093792+0000", 2024-09-16T20:50:19.771 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2436999999999997e-05 2024-09-16T20:50:19.771 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.771 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.771 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:19.771 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093943+0000", 2024-09-16T20:50:19.771 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000151421 2024-09-16T20:50:19.771 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.771 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:19.771 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.771 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.771 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.771 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3288 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 256000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:19.772 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:50:19.772 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 29.628014173, 2024-09-16T20:50:19.772 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052928269, 2024-09-16T20:50:19.772 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:19.772 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:19.772 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:19.772 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:19.772 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:19.772 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:19.772 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3288 2024-09-16T20:50:19.772 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.772 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:19.772 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.773 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:19.773 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:50:19.773 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.773 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.773 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.773 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:19.773 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:50:19.773 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.773 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.773 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.773 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:19.773 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:50:19.773 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2700000000000001e-07 2024-09-16T20:50:19.773 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.774 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.774 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:19.774 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041173+0000", 2024-09-16T20:50:19.774 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.113e-06 2024-09-16T20:50:19.774 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.774 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.774 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:19.774 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041176+0000", 2024-09-16T20:50:19.774 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9799999999999998e-06 2024-09-16T20:50:19.774 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.774 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.774 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:19.774 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041179+0000", 2024-09-16T20:50:19.774 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.667e-06 2024-09-16T20:50:19.774 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.775 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.775 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:19.775 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093957+0000", 2024-09-16T20:50:19.775 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052778703000000003 2024-09-16T20:50:19.775 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.775 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.775 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:19.775 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093999+0000", 2024-09-16T20:50:19.775 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1300000000000001e-05 2024-09-16T20:50:19.775 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.775 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.775 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:19.775 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094100+0000", 2024-09-16T20:50:19.775 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101279 2024-09-16T20:50:19.776 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.776 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:19.776 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.776 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.776 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.776 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3289 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 261120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:19.776 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:50:19.776 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 29.628004177000001, 2024-09-16T20:50:19.776 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053182976, 2024-09-16T20:50:19.776 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:19.776 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:19.776 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:19.776 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:19.777 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:19.777 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:19.777 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3289 2024-09-16T20:50:19.777 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.777 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:19.777 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.777 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:19.777 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:50:19.777 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.777 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.777 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.777 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:19.777 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:50:19.778 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:19.778 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.778 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.778 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:19.778 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:50:19.778 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2999999999999999e-07 2024-09-16T20:50:19.778 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.778 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.778 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:19.778 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041183+0000", 2024-09-16T20:50:19.778 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0720000000000001e-06 2024-09-16T20:50:19.778 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.778 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.779 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:19.779 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041186+0000", 2024-09-16T20:50:19.779 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9869999999999999e-06 2024-09-16T20:50:19.779 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.779 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.779 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:19.779 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041189+0000", 2024-09-16T20:50:19.779 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.587e-06 2024-09-16T20:50:19.779 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.779 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.779 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:19.779 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094136+0000", 2024-09-16T20:50:19.779 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052947270999999997 2024-09-16T20:50:19.779 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.779 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.780 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:19.780 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094213+0000", 2024-09-16T20:50:19.780 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7552000000000006e-05 2024-09-16T20:50:19.780 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:19.780 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:19.780 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:19.780 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094365+0000", 2024-09-16T20:50:19.780 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00015127700000000001 2024-09-16T20:50:19.780 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.780 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:19.780 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.780 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:19.780 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:19.781 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:19.781 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T20:50:19.783 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:50:19.783 INFO:teuthology.orchestra.run.smithi029.stderr:noscrub is set 2024-09-16T20:50:19.784 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:19.780+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:19.829 INFO:teuthology.orchestra.run.smithi139.stdout:5808 punch from 0x5773400 to 0x5782e00, (0xfa00 bytes) 2024-09-16T20:50:19.882 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:19.881+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:19.941 INFO:teuthology.orchestra.run.smithi139.stdout:5809 punch from 0x7911400 to 0x7921200, (0xfe00 bytes) 2024-09-16T20:50:19.964 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:19.964 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:50:19.964 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:50:19.964 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:19.977 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:50:19.982 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:50:19.983 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:50:19.983 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:19.981+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:20.005 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:50:20.006 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:50:20.028 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:50:20.028 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:50:20.051 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:50:20.051 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:50:20.080 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:50:20.081 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:50:20.084 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:20.082+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:20.100 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:50:20.101 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:50:20.118 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:20.118 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:20.130 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:50:20.131 INFO:teuthology.orchestra.run.smithi139.stdout:5810 read 0x8711a00 thru 0x871d8fb (0xbefc bytes) 2024-09-16T20:50:20.133 INFO:teuthology.orchestra.run.smithi139.stdout:5812 read 0x668d800 thru 0x669bb9f (0xe3a0 bytes) 2024-09-16T20:50:20.137 INFO:teuthology.orchestra.run.smithi139.stdout:5813 read 0x4bea800 thru 0x4bf8f3d (0xe73e bytes) 2024-09-16T20:50:20.139 INFO:teuthology.orchestra.run.smithi139.stdout:5814 write 0x6c93600 thru 0x6c9c98b (0x938c bytes) 2024-09-16T20:50:20.142 INFO:teuthology.orchestra.run.smithi139.stdout:5818 trunc from 0xb06bc00 to 0x9fa4800 2024-09-16T20:50:20.184 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:20.183+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:20.285 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:20.284+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:20.306 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:20.306 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:20.315 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:50:20.386 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:20.385+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:20.424 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:20.424 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:20.433 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:50:20.433 INFO:teuthology.orchestra.run.smithi139.stdout:5819 punch from 0x599f200 to 0x59ada00, (0xe800 bytes) 2024-09-16T20:50:20.440 INFO:teuthology.orchestra.run.smithi139.stdout:5821 write 0x9103a00 thru 0x910b5e2 (0x7be3 bytes) 2024-09-16T20:50:20.441 INFO:teuthology.orchestra.run.smithi139.stdout:5822 clone 44 order 18 su 65536 sc 12 2024-09-16T20:50:20.487 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:20.486+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:20.554 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:20.554 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:20.563 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:50:20.607 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:20.606+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:20.670 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:20.670 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:20.679 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:50:20.721 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:20.719+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:20.782 INFO:teuthology.orchestra.run.smithi139.stdout:truncating image image_client.0-clone43 from 0x9fa4800 (overlap 0x9fa4800) to 0x5fc9000 2024-09-16T20:50:20.790 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:20.789+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:20.839 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:20.839 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:20.848 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:50:20.891 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:20.890+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:20.993 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:20.992+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:21.047 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:21.047 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:21.060 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:50:21.103 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:21.102+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:21.193 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:21.192+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:21.200 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:21.201 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:21.209 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:50:21.293 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:21.292+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:21.370 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:21.370 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:21.381 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:50:21.394 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:21.393+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:21.495 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:21.494+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:21.526 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:21.527 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:21.536 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:50:21.620 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:21.619+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:21.697 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:21.695+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:21.704 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:21.704 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:21.713 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:50:21.784 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T20:50:21.828 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:21.827+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:21.830 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:21.831 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:21.840 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:50:21.924 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:21.923+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:21.996 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:21.996 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:21.999 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:21.998+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:22.008 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:50:22.061 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #42, image image_client.0-clone42 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent43 2024-09-16T20:50:22.100 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:22.099+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:22.112 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T20:50:22.170 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:22.170 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:22.180 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:50:22.200 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:22.199+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:22.255 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:50:22.255 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:50:22.274 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:50:22.275 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:50:22.295 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:50:22.295 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:50:22.301 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:22.300+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:22.302 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:22.303 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:50:22.303 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.303 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6028.0:1100 3.d 3:b1fe01a0:::rbd_data.110e7ef3004.0000000000000000:head [sparse-read 15863808~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-16T20:50:22.303 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:22.302506+0000", 2024-09-16T20:50:22.303 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 8.0937000000000001e-05, 2024-09-16T20:50:22.303 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010092, 2024-09-16T20:50:22.303 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:22.303 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:22.303 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:22.303 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:22.303 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6028", 2024-09-16T20:50:22.303 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3995274260", 2024-09-16T20:50:22.303 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 1100 2024-09-16T20:50:22.304 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.304 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:22.304 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.304 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:22.304 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.302506+0000", 2024-09-16T20:50:22.304 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:22.304 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.304 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.304 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:22.304 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.302506+0000", 2024-09-16T20:50:22.304 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:22.305 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.305 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.305 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:22.305 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.302507+0000", 2024-09-16T20:50:22.305 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9200000000000001e-07 2024-09-16T20:50:22.305 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.305 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.305 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:22.305 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.302509+0000", 2024-09-16T20:50:22.305 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.4650000000000001e-06 2024-09-16T20:50:22.305 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.305 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.305 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:22.305 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.302516+0000", 2024-09-16T20:50:22.306 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4359999999999999e-06 2024-09-16T20:50:22.306 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.306 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.306 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:22.306 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.302521+0000", 2024-09-16T20:50:22.306 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1150000000000002e-06 2024-09-16T20:50:22.306 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.306 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.306 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:22.306 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.302558+0000", 2024-09-16T20:50:22.306 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7489e-05 2024-09-16T20:50:22.306 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.306 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.306 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:22.306 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.302591+0000", 2024-09-16T20:50:22.307 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2449000000000001e-05 2024-09-16T20:50:22.307 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:22.307 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:22.307 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:22.307 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:22.307 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-16T20:50:22.307 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 1 2024-09-16T20:50:22.307 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:22.312 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:50:22.317 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:50:22.317 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:50:22.335 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:50:22.336 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:50:22.355 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:50:22.355 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:50:22.407 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:22.406+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:22.428 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:22.428 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:50:22.428 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:50:22.428 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:50:22.429 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:22.436 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:50:22.521 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:22.520+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:22.539 INFO:teuthology.orchestra.run.smithi139.stdout:5824 write 0xd69f400 thru 0xd6a38c2 (0x44c3 bytes) 2024-09-16T20:50:22.545 INFO:teuthology.orchestra.run.smithi139.stdout:5825 read 0x6dc2400 thru 0x6dd1f5c (0xfb5d bytes) 2024-09-16T20:50:22.545 INFO:teuthology.orchestra.run.smithi139.stdout:5827 write 0xd33f200 thru 0xd344313 (0x5114 bytes) 2024-09-16T20:50:22.545 INFO:teuthology.orchestra.run.smithi139.stdout:5828 read 0x8479200 thru 0x847f114 (0x5f15 bytes) 2024-09-16T20:50:22.547 INFO:teuthology.orchestra.run.smithi139.stdout:5829 punch from 0x37b7e00 to 0x37c6400, (0xe600 bytes) 2024-09-16T20:50:22.548 INFO:teuthology.orchestra.run.smithi139.stdout:5830 write 0xdeb400 thru 0xdf731d (0xbf1e bytes) 2024-09-16T20:50:22.573 INFO:teuthology.orchestra.run.smithi139.stdout:5831 read 0x8706600 thru 0x8713e80 (0xd881 bytes) 2024-09-16T20:50:22.576 INFO:teuthology.orchestra.run.smithi139.stdout:5832 write 0x25f0e00 thru 0x25f56cc (0x48cd bytes) 2024-09-16T20:50:22.577 INFO:teuthology.orchestra.run.smithi139.stdout:5833 write 0x2fd2000 thru 0x2fe0bd3 (0xebd4 bytes) 2024-09-16T20:50:22.577 INFO:teuthology.orchestra.run.smithi139.stdout:5834 write 0x1b05c00 thru 0x1b074f1 (0x18f2 bytes) 2024-09-16T20:50:22.578 INFO:teuthology.orchestra.run.smithi139.stdout:5835 read 0xd297000 thru 0xd2a642a (0xf42b bytes) 2024-09-16T20:50:22.578 INFO:teuthology.orchestra.run.smithi139.stdout:5837 read 0xc255e00 thru 0xc261419 (0xb61a bytes) 2024-09-16T20:50:22.593 INFO:teuthology.orchestra.run.smithi139.stdout:5839 trunc from 0xd6a3a00 to 0x100be00 2024-09-16T20:50:22.601 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:22.601 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:50:22.601 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:50:22.601 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:50:22.601 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.601 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:50:22.601 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:50:22.602 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 149.34459532, 2024-09-16T20:50:22.602 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67502346999999996, 2024-09-16T20:50:22.602 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:22.602 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:22.602 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:22.602 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:22.602 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:50:22.602 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:50:22.602 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 1 2024-09-16T20:50:22.602 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.602 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:22.602 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.602 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:22.602 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:50:22.603 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:22.603 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.603 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.603 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:22.603 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:50:22.603 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:22.603 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.603 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.603 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:22.603 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254128+0000", 2024-09-16T20:50:22.603 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.463e-06 2024-09-16T20:50:22.603 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.603 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.603 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:22.604 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254135+0000", 2024-09-16T20:50:22.604 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1579999999999999e-06 2024-09-16T20:50:22.604 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.604 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.604 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:22.604 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254146+0000", 2024-09-16T20:50:22.604 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1341e-05 2024-09-16T20:50:22.604 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.604 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.604 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:22.604 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254155+0000", 2024-09-16T20:50:22.604 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.1549999999999996e-06 2024-09-16T20:50:22.604 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.605 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.605 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:22.605 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254221+0000", 2024-09-16T20:50:22.605 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5470999999999997e-05 2024-09-16T20:50:22.605 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.605 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.605 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:22.605 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254231+0000", 2024-09-16T20:50:22.605 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0592e-05 2024-09-16T20:50:22.605 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.605 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.605 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:22.605 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926711+0000", 2024-09-16T20:50:22.605 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67247946599999997 2024-09-16T20:50:22.606 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.606 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.606 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:22.606 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926910+0000", 2024-09-16T20:50:22.606 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019934 2024-09-16T20:50:22.606 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.606 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.606 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:50:22.606 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.927159+0000", 2024-09-16T20:50:22.606 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00024876599999999998 2024-09-16T20:50:22.606 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.606 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.606 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:22.606 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.928797+0000", 2024-09-16T20:50:22.607 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001638308 2024-09-16T20:50:22.607 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.607 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.607 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:22.607 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929026+0000", 2024-09-16T20:50:22.607 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00022876099999999999 2024-09-16T20:50:22.607 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.607 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.607 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:22.607 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929102+0000", 2024-09-16T20:50:22.607 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6184000000000001e-05 2024-09-16T20:50:22.607 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.607 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.607 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:22.608 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929150+0000", 2024-09-16T20:50:22.608 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7465e-05 2024-09-16T20:50:22.608 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:22.608 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:22.608 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:22.608 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.608 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.608 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3574 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:22.608 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:50:22.608 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 93.117431788000005, 2024-09-16T20:50:22.608 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053579441999999998, 2024-09-16T20:50:22.608 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:22.608 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:22.609 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:22.609 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:22.609 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:22.609 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:22.609 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3574 2024-09-16T20:50:22.609 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.609 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:22.609 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.609 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:22.609 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:50:22.609 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:22.609 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.609 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.609 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:22.610 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:50:22.610 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:22.610 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.610 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.610 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:22.610 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:50:22.610 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2e-07 2024-09-16T20:50:22.610 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.610 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.610 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:22.610 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481292+0000", 2024-09-16T20:50:22.610 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2630000000000002e-06 2024-09-16T20:50:22.610 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.610 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.611 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:22.611 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481297+0000", 2024-09-16T20:50:22.611 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.296e-06 2024-09-16T20:50:22.611 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.611 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.611 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:22.611 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481300+0000", 2024-09-16T20:50:22.611 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7019999999999999e-06 2024-09-16T20:50:22.611 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.611 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.611 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:22.611 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525533+0000", 2024-09-16T20:50:22.611 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.044232318 2024-09-16T20:50:22.611 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.612 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.612 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:22.612 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525619+0000", 2024-09-16T20:50:22.612 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5892999999999999e-05 2024-09-16T20:50:22.612 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.612 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.612 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:22.612 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525987+0000", 2024-09-16T20:50:22.612 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00036836099999999999 2024-09-16T20:50:22.612 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.612 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.612 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:22.613 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531779+0000", 2024-09-16T20:50:22.613 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0057918240000000001 2024-09-16T20:50:22.613 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.613 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.613 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:22.613 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534747+0000", 2024-09-16T20:50:22.613 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0029681600000000001 2024-09-16T20:50:22.613 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.613 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.613 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:22.613 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534832+0000", 2024-09-16T20:50:22.613 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5180999999999996e-05 2024-09-16T20:50:22.613 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.613 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.614 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:22.614 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534869+0000", 2024-09-16T20:50:22.614 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7024000000000003e-05 2024-09-16T20:50:22.614 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:22.614 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:22.614 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:22.614 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.614 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.614 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3575 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:22.614 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:50:22.614 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 93.117417047999993, 2024-09-16T20:50:22.614 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053801463000000001, 2024-09-16T20:50:22.614 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:22.614 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:22.615 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:22.615 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:22.615 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:22.615 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:22.615 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3575 2024-09-16T20:50:22.615 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.615 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:22.615 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.615 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:22.615 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:50:22.615 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:22.615 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.615 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.616 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:22.616 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:50:22.616 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:22.616 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.616 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.616 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:22.616 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481305+0000", 2024-09-16T20:50:22.616 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5400000000000002e-07 2024-09-16T20:50:22.616 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.616 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.616 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:22.616 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481307+0000", 2024-09-16T20:50:22.616 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7800000000000001e-06 2024-09-16T20:50:22.617 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.617 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.617 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:22.617 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481312+0000", 2024-09-16T20:50:22.617 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6349999999999997e-06 2024-09-16T20:50:22.617 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.617 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.617 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:22.617 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481315+0000", 2024-09-16T20:50:22.617 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7829999999999999e-06 2024-09-16T20:50:22.617 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.617 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.617 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:22.617 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.526876+0000", 2024-09-16T20:50:22.618 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.045561070000000002 2024-09-16T20:50:22.618 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.618 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.618 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:22.618 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.526954+0000", 2024-09-16T20:50:22.618 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8108000000000005e-05 2024-09-16T20:50:22.618 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.618 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.618 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:22.618 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527282+0000", 2024-09-16T20:50:22.618 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000327483 2024-09-16T20:50:22.618 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.618 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.619 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:22.619 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531784+0000", 2024-09-16T20:50:22.619 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0045020169999999997 2024-09-16T20:50:22.619 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.619 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.619 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:22.619 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534921+0000", 2024-09-16T20:50:22.619 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.003137791 2024-09-16T20:50:22.619 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.619 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.619 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:22.619 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534976+0000", 2024-09-16T20:50:22.619 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4953000000000001e-05 2024-09-16T20:50:22.619 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.620 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.620 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:22.620 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535106+0000", 2024-09-16T20:50:22.620 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000129589 2024-09-16T20:50:22.620 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:22.620 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:22.620 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:22.620 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.620 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.620 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3576 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:22.620 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:50:22.620 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 93.117403135999993, 2024-09-16T20:50:22.620 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053955436000000002, 2024-09-16T20:50:22.620 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:22.621 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:22.621 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:22.621 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:22.621 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:22.621 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:22.621 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3576 2024-09-16T20:50:22.621 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.621 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:22.621 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.621 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:22.621 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:50:22.621 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:22.621 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.622 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.622 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:22.622 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:50:22.622 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:22.622 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.622 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.622 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:22.622 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481319+0000", 2024-09-16T20:50:22.622 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.53e-07 2024-09-16T20:50:22.622 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.622 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.622 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:22.622 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481320+0000", 2024-09-16T20:50:22.622 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5230000000000001e-06 2024-09-16T20:50:22.623 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.623 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.623 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:22.623 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481330+0000", 2024-09-16T20:50:22.623 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0190000000000001e-05 2024-09-16T20:50:22.623 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.623 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.623 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:22.623 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481333+0000", 2024-09-16T20:50:22.623 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.151e-06 2024-09-16T20:50:22.623 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.623 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.623 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:22.623 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527689+0000", 2024-09-16T20:50:22.624 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046355192000000003 2024-09-16T20:50:22.624 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.624 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.624 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:22.624 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527764+0000", 2024-09-16T20:50:22.624 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5776000000000001e-05 2024-09-16T20:50:22.624 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.624 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.624 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:22.624 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528141+0000", 2024-09-16T20:50:22.624 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00037647599999999999 2024-09-16T20:50:22.624 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.624 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.624 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:22.625 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531788+0000", 2024-09-16T20:50:22.625 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0036470679999999998 2024-09-16T20:50:22.625 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.625 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.625 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:22.625 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535174+0000", 2024-09-16T20:50:22.625 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0033862670000000001 2024-09-16T20:50:22.625 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.625 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.625 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:22.625 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535236+0000", 2024-09-16T20:50:22.625 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1797999999999993e-05 2024-09-16T20:50:22.625 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.626 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.626 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:22.626 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535274+0000", 2024-09-16T20:50:22.626 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7741999999999999e-05 2024-09-16T20:50:22.626 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:22.626 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:22.626 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:22.626 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.626 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.626 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3577 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:22.626 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:50:22.626 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 93.117381535999996, 2024-09-16T20:50:22.626 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054073807000000002, 2024-09-16T20:50:22.627 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:22.627 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:22.627 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:22.627 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:22.627 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:22.627 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:22.627 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3577 2024-09-16T20:50:22.627 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.627 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:22.627 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.628 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:22.628 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:50:22.628 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:22.628 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.628 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.628 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:22.628 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:50:22.628 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:22.628 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.628 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.628 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:22.628 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:50:22.628 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.96e-07 2024-09-16T20:50:22.629 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.629 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.629 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:22.629 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481342+0000", 2024-09-16T20:50:22.629 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.714e-06 2024-09-16T20:50:22.629 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.629 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.629 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:22.629 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481349+0000", 2024-09-16T20:50:22.629 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4930000000000001e-06 2024-09-16T20:50:22.629 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.629 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.629 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:22.630 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481353+0000", 2024-09-16T20:50:22.630 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4879999999999999e-06 2024-09-16T20:50:22.630 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.630 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.630 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:22.630 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528619+0000", 2024-09-16T20:50:22.630 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047266351999999998 2024-09-16T20:50:22.630 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.630 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.630 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:22.630 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528706+0000", 2024-09-16T20:50:22.630 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6796e-05 2024-09-16T20:50:22.630 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.630 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.631 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:22.631 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529148+0000", 2024-09-16T20:50:22.631 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00044182999999999999 2024-09-16T20:50:22.631 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.631 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.631 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:22.631 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531792+0000", 2024-09-16T20:50:22.631 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0026443119999999998 2024-09-16T20:50:22.631 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.631 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.631 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:22.631 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535319+0000", 2024-09-16T20:50:22.631 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0035266439999999998 2024-09-16T20:50:22.631 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.632 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.632 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:22.632 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535372+0000", 2024-09-16T20:50:22.632 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3207000000000001e-05 2024-09-16T20:50:22.632 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.632 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.632 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:22.632 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535414+0000", 2024-09-16T20:50:22.632 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1675000000000003e-05 2024-09-16T20:50:22.632 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:22.632 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:22.632 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:22.632 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.633 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.633 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3578 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:22.633 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:50:22.633 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 93.117360367000003, 2024-09-16T20:50:22.633 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054189586999999997, 2024-09-16T20:50:22.633 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:22.633 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:22.633 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:22.633 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:22.633 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:22.633 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:22.633 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3578 2024-09-16T20:50:22.633 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.633 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:22.634 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.634 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:22.634 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:50:22.634 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:22.634 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.634 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.634 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:22.634 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:50:22.634 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:22.634 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.634 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.634 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:22.634 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:50:22.634 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0800000000000001e-07 2024-09-16T20:50:22.635 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.635 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.635 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:22.635 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481363+0000", 2024-09-16T20:50:22.635 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0339999999999999e-06 2024-09-16T20:50:22.635 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.635 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.635 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:22.635 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481371+0000", 2024-09-16T20:50:22.635 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4440000000000002e-06 2024-09-16T20:50:22.635 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.635 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.635 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:22.635 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481374+0000", 2024-09-16T20:50:22.636 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2049999999999998e-06 2024-09-16T20:50:22.636 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.636 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.636 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:22.636 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529555+0000", 2024-09-16T20:50:22.636 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048181202999999999 2024-09-16T20:50:22.636 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.636 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.636 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:22.636 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529616+0000", 2024-09-16T20:50:22.636 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0912000000000002e-05 2024-09-16T20:50:22.636 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.636 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.637 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:22.637 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529939+0000", 2024-09-16T20:50:22.637 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00032269800000000001 2024-09-16T20:50:22.637 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.637 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.637 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:22.637 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532602+0000", 2024-09-16T20:50:22.637 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0026634940000000002 2024-09-16T20:50:22.637 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.637 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.637 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:22.637 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535465+0000", 2024-09-16T20:50:22.637 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.002862149 2024-09-16T20:50:22.638 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.638 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.638 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:22.638 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535516+0000", 2024-09-16T20:50:22.638 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1696000000000001e-05 2024-09-16T20:50:22.638 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.638 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.638 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:22.638 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535551+0000", 2024-09-16T20:50:22.638 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4443999999999998e-05 2024-09-16T20:50:22.638 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:22.638 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:22.638 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:22.638 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.639 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.639 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3579 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:22.639 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:50:22.639 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 93.117341928000002, 2024-09-16T20:50:22.639 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054295151, 2024-09-16T20:50:22.639 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:22.639 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:22.639 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:22.639 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:22.639 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:22.639 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:22.639 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3579 2024-09-16T20:50:22.639 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.639 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:22.640 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.640 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:22.640 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:50:22.640 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:22.640 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.640 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.640 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:22.640 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:50:22.640 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:22.640 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.640 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.640 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:22.640 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:50:22.640 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3300000000000001e-07 2024-09-16T20:50:22.641 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.641 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.641 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:22.641 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481381+0000", 2024-09-16T20:50:22.641 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.649e-06 2024-09-16T20:50:22.641 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.641 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.641 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:22.641 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481387+0000", 2024-09-16T20:50:22.641 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4519999999999996e-06 2024-09-16T20:50:22.641 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.641 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.641 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:22.641 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481390+0000", 2024-09-16T20:50:22.642 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0170000000000001e-06 2024-09-16T20:50:22.642 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.642 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.642 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:22.642 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530756+0000", 2024-09-16T20:50:22.642 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049365882999999999 2024-09-16T20:50:22.642 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.642 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.642 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:22.642 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530845+0000", 2024-09-16T20:50:22.642 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9197000000000007e-05 2024-09-16T20:50:22.642 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.642 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.642 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:22.643 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531191+0000", 2024-09-16T20:50:22.643 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00034562500000000001 2024-09-16T20:50:22.643 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.643 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.643 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:22.643 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534078+0000", 2024-09-16T20:50:22.643 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0028878179999999999 2024-09-16T20:50:22.643 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.643 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.643 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:22.643 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535589+0000", 2024-09-16T20:50:22.643 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00151085 2024-09-16T20:50:22.644 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.644 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.644 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:22.644 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535639+0000", 2024-09-16T20:50:22.644 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9697000000000002e-05 2024-09-16T20:50:22.644 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.644 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.644 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:22.644 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535675+0000", 2024-09-16T20:50:22.644 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5729999999999998e-05 2024-09-16T20:50:22.644 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:22.644 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:22.644 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:22.644 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.645 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.645 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3580 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:22.645 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:50:22.645 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 93.117328295999997, 2024-09-16T20:50:22.645 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054397361999999998, 2024-09-16T20:50:22.645 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:22.645 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:22.645 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:22.645 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:22.645 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:22.645 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:22.645 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3580 2024-09-16T20:50:22.645 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.645 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:22.646 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.646 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:22.646 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:50:22.646 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:22.646 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.646 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.646 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:22.646 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:50:22.646 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:22.646 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.646 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.646 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:22.646 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:50:22.646 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2600000000000001e-07 2024-09-16T20:50:22.647 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.647 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.647 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:22.647 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481395+0000", 2024-09-16T20:50:22.647 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.544e-06 2024-09-16T20:50:22.647 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.647 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.647 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:22.647 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481404+0000", 2024-09-16T20:50:22.647 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8850000000000005e-06 2024-09-16T20:50:22.647 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.647 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.647 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:22.647 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481407+0000", 2024-09-16T20:50:22.648 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1540000000000002e-06 2024-09-16T20:50:22.648 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.648 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.648 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:22.648 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531823+0000", 2024-09-16T20:50:22.648 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050415637999999999 2024-09-16T20:50:22.648 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.648 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.648 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:22.648 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531895+0000", 2024-09-16T20:50:22.648 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2179000000000001e-05 2024-09-16T20:50:22.648 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.648 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.649 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:22.649 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532236+0000", 2024-09-16T20:50:22.649 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00034073799999999999 2024-09-16T20:50:22.649 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.649 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.649 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:22.649 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534083+0000", 2024-09-16T20:50:22.649 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001847298 2024-09-16T20:50:22.649 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.649 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.649 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:22.649 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535714+0000", 2024-09-16T20:50:22.649 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0016316810000000001 2024-09-16T20:50:22.650 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.650 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.650 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:22.650 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535759+0000", 2024-09-16T20:50:22.650 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4478999999999999e-05 2024-09-16T20:50:22.650 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.650 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.650 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:22.650 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535791+0000", 2024-09-16T20:50:22.650 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.154e-05 2024-09-16T20:50:22.650 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:22.650 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:22.650 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:22.650 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.651 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.651 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3581 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:22.651 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:50:22.651 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 93.117306299000006, 2024-09-16T20:50:22.651 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.057094403000000002, 2024-09-16T20:50:22.651 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:22.651 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:22.651 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:22.651 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:22.651 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:22.651 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:22.651 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3581 2024-09-16T20:50:22.651 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.651 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:22.652 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.652 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:22.652 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:50:22.652 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:22.652 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.652 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.652 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:22.652 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:50:22.652 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:22.652 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.652 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.652 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:22.652 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:50:22.653 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8799999999999998e-07 2024-09-16T20:50:22.653 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.653 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.653 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:22.653 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481418+0000", 2024-09-16T20:50:22.653 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1859999999999999e-06 2024-09-16T20:50:22.653 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.653 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.653 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:22.653 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481423+0000", 2024-09-16T20:50:22.653 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8279999999999999e-06 2024-09-16T20:50:22.653 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.653 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.653 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:22.654 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481426+0000", 2024-09-16T20:50:22.654 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0110000000000001e-06 2024-09-16T20:50:22.654 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.654 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.654 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:22.654 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532631+0000", 2024-09-16T20:50:22.654 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051204514999999999 2024-09-16T20:50:22.654 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.654 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.654 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:22.654 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.533381+0000", 2024-09-16T20:50:22.654 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00075029000000000003 2024-09-16T20:50:22.654 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.654 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.655 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:22.655 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.533739+0000", 2024-09-16T20:50:22.655 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00035773100000000002 2024-09-16T20:50:22.655 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.655 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.655 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:22.655 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535123+0000", 2024-09-16T20:50:22.655 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0013836160000000001 2024-09-16T20:50:22.655 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.655 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.655 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:22.655 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538413+0000", 2024-09-16T20:50:22.655 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0032902510000000001 2024-09-16T20:50:22.655 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.656 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.656 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:22.656 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538470+0000", 2024-09-16T20:50:22.656 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6697999999999998e-05 2024-09-16T20:50:22.656 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.656 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.656 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:22.656 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538510+0000", 2024-09-16T20:50:22.656 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9988999999999999e-05 2024-09-16T20:50:22.656 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:22.656 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:22.656 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:22.656 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.657 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.657 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:12732 3.d 3:b752444a:::rbd_header.110e747f0779:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-16T20:50:22.657 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:50:22.657 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 77.830233720999999, 2024-09-16T20:50:22.657 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.065302467000000003, 2024-09-16T20:50:22.657 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:22.657 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:22.657 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:22.657 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:22.657 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:50:22.657 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:50:22.657 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 12732 2024-09-16T20:50:22.657 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.658 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:22.658 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.658 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:22.658 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:50:22.658 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:22.658 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.658 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.658 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:22.658 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:50:22.658 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:22.658 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.658 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.658 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:22.658 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768490+0000", 2024-09-16T20:50:22.659 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2859999999999998e-06 2024-09-16T20:50:22.659 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.659 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.659 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:22.659 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768501+0000", 2024-09-16T20:50:22.659 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1226999999999999e-05 2024-09-16T20:50:22.659 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.659 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.659 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:22.659 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768517+0000", 2024-09-16T20:50:22.659 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.592e-05 2024-09-16T20:50:22.659 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.659 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.659 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:22.660 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768528+0000", 2024-09-16T20:50:22.660 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0689999999999999e-05 2024-09-16T20:50:22.660 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.660 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.660 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:22.660 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768584+0000", 2024-09-16T20:50:22.660 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5889e-05 2024-09-16T20:50:22.660 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.660 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.660 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:22.660 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768720+0000", 2024-09-16T20:50:22.660 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000136718 2024-09-16T20:50:22.660 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.660 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.661 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:22.661 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.769066+0000", 2024-09-16T20:50:22.661 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000346013 2024-09-16T20:50:22.661 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.661 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.661 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:22.661 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833659+0000", 2024-09-16T20:50:22.661 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.064592822999999994 2024-09-16T20:50:22.661 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.661 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.661 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:22.661 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833719+0000", 2024-09-16T20:50:22.661 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9441000000000002e-05 2024-09-16T20:50:22.662 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.662 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.662 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:22.662 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833758+0000", 2024-09-16T20:50:22.662 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9122000000000003e-05 2024-09-16T20:50:22.662 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.662 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.662 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:22.662 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833790+0000", 2024-09-16T20:50:22.662 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2338000000000001e-05 2024-09-16T20:50:22.662 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:22.662 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:22.662 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:22.662 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.663 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.663 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.5089.0:12732 3.d e73/23 ondisk, result = 0)", 2024-09-16T20:50:22.663 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:50:22.663 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 77.828631551000001, 2024-09-16T20:50:22.663 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.063706456999999994, 2024-09-16T20:50:22.663 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:22.663 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:22.663 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:22.663 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:22.663 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.663 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:22.663 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:50:22.663 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:22.663 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.664 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.664 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:22.664 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:50:22.664 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:22.664 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.664 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.664 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:22.664 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770091+0000", 2024-09-16T20:50:22.664 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.53e-06 2024-09-16T20:50:22.664 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.664 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.664 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:22.664 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770104+0000", 2024-09-16T20:50:22.665 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2203e-05 2024-09-16T20:50:22.665 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.665 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.665 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:22.665 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833637+0000", 2024-09-16T20:50:22.665 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.063533201999999997 2024-09-16T20:50:22.665 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.665 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.665 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:22.665 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833655+0000", 2024-09-16T20:50:22.665 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8369e-05 2024-09-16T20:50:22.665 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.665 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.665 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:22.666 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833707+0000", 2024-09-16T20:50:22.666 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2061999999999997e-05 2024-09-16T20:50:22.666 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.666 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.666 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:22.666 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833715+0000", 2024-09-16T20:50:22.666 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5179999999999998e-06 2024-09-16T20:50:22.666 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.666 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.666 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:22.666 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833796+0000", 2024-09-16T20:50:22.666 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1573e-05 2024-09-16T20:50:22.666 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:22.666 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:22.667 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:22.667 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.667 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.667 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3281 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 220160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:22.667 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041070+0000", 2024-09-16T20:50:22.667 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.557651082, 2024-09-16T20:50:22.667 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051593550000000002, 2024-09-16T20:50:22.667 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:22.667 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:22.667 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:22.667 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:22.667 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:22.667 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:22.667 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3281 2024-09-16T20:50:22.668 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.668 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:22.668 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.668 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:22.668 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041070+0000", 2024-09-16T20:50:22.668 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:22.668 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.668 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.668 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:22.668 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041070+0000", 2024-09-16T20:50:22.668 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:22.668 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.668 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.669 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:22.669 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041071+0000", 2024-09-16T20:50:22.669 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1899999999999999e-07 2024-09-16T20:50:22.669 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.669 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.669 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:22.669 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041072+0000", 2024-09-16T20:50:22.669 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.248e-06 2024-09-16T20:50:22.669 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.669 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.669 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:22.669 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041075+0000", 2024-09-16T20:50:22.669 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0029999999999999e-06 2024-09-16T20:50:22.669 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.670 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.670 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:22.670 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041077+0000", 2024-09-16T20:50:22.670 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.621e-06 2024-09-16T20:50:22.670 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.670 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.670 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:22.670 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.092431+0000", 2024-09-16T20:50:22.670 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051353801999999997 2024-09-16T20:50:22.670 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.670 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.670 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:22.670 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.092512+0000", 2024-09-16T20:50:22.670 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1106000000000006e-05 2024-09-16T20:50:22.671 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.671 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.671 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:22.671 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.092664+0000", 2024-09-16T20:50:22.671 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00015155100000000001 2024-09-16T20:50:22.671 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:22.671 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:22.671 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:22.671 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.671 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.671 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3282 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 225280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:22.671 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041081+0000", 2024-09-16T20:50:22.671 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.557640804000002, 2024-09-16T20:50:22.671 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051731948, 2024-09-16T20:50:22.672 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:22.672 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:22.672 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:22.672 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:22.672 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:22.672 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:22.672 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3282 2024-09-16T20:50:22.672 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.672 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:22.672 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.672 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:22.672 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041081+0000", 2024-09-16T20:50:22.672 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:22.673 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.673 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.673 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:22.673 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041081+0000", 2024-09-16T20:50:22.673 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:22.673 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.673 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.673 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:22.673 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041081+0000", 2024-09-16T20:50:22.673 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3099999999999999e-07 2024-09-16T20:50:22.673 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.673 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.673 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:22.673 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041082+0000", 2024-09-16T20:50:22.674 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1289999999999999e-06 2024-09-16T20:50:22.674 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.674 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.674 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:22.674 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041085+0000", 2024-09-16T20:50:22.674 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2320000000000001e-06 2024-09-16T20:50:22.674 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.674 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.674 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:22.674 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041090+0000", 2024-09-16T20:50:22.674 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3730000000000003e-06 2024-09-16T20:50:22.674 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.674 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.674 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:22.675 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.092678+0000", 2024-09-16T20:50:22.675 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051588235000000003 2024-09-16T20:50:22.675 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.675 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.675 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:22.675 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.092717+0000", 2024-09-16T20:50:22.675 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9057999999999998e-05 2024-09-16T20:50:22.675 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.675 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.675 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:22.675 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.092813+0000", 2024-09-16T20:50:22.675 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.569e-05 2024-09-16T20:50:22.675 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:22.675 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:22.676 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:22.676 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.676 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.676 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3283 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 230400~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:22.676 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041102+0000", 2024-09-16T20:50:22.676 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.557619144, 2024-09-16T20:50:22.676 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051974723, 2024-09-16T20:50:22.676 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:22.676 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:22.676 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:22.676 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:22.676 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:22.676 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:22.677 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3283 2024-09-16T20:50:22.677 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.677 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:22.677 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.677 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:22.677 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041102+0000", 2024-09-16T20:50:22.677 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:22.677 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.677 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.677 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:22.677 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041102+0000", 2024-09-16T20:50:22.677 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:22.677 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.677 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.678 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:22.678 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041103+0000", 2024-09-16T20:50:22.678 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2399999999999999e-07 2024-09-16T20:50:22.678 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.678 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.678 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:22.678 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041110+0000", 2024-09-16T20:50:22.678 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4290000000000004e-06 2024-09-16T20:50:22.678 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.678 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.678 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:22.678 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041118+0000", 2024-09-16T20:50:22.678 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3849999999999996e-06 2024-09-16T20:50:22.678 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.679 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.679 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:22.679 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041123+0000", 2024-09-16T20:50:22.679 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1410000000000004e-06 2024-09-16T20:50:22.679 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.679 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.679 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:22.679 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.092845+0000", 2024-09-16T20:50:22.679 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051722207999999999 2024-09-16T20:50:22.679 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.679 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.679 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:22.679 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.092926+0000", 2024-09-16T20:50:22.679 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0727000000000001e-05 2024-09-16T20:50:22.680 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.680 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.680 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:22.680 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093077+0000", 2024-09-16T20:50:22.680 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000151409 2024-09-16T20:50:22.680 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:22.680 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:22.680 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:22.680 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.680 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.680 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3284 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 235520~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:22.680 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041129+0000", 2024-09-16T20:50:22.680 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.557592812999999, 2024-09-16T20:50:22.681 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052120760000000002, 2024-09-16T20:50:22.681 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:22.681 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:22.681 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:22.681 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:22.681 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:22.681 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:22.681 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3284 2024-09-16T20:50:22.681 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.681 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:22.681 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.681 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:22.681 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041129+0000", 2024-09-16T20:50:22.681 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:22.682 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.682 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.682 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:22.682 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041129+0000", 2024-09-16T20:50:22.682 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:22.682 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.682 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.682 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:22.682 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041129+0000", 2024-09-16T20:50:22.682 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3700000000000001e-07 2024-09-16T20:50:22.682 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.682 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.682 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:22.682 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041130+0000", 2024-09-16T20:50:22.683 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4050000000000001e-06 2024-09-16T20:50:22.683 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.683 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.683 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:22.683 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041134+0000", 2024-09-16T20:50:22.683 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4290000000000001e-06 2024-09-16T20:50:22.683 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.683 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.683 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:22.683 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041136+0000", 2024-09-16T20:50:22.683 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.678e-06 2024-09-16T20:50:22.683 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.683 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.683 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:22.683 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093109+0000", 2024-09-16T20:50:22.684 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051972879999999999 2024-09-16T20:50:22.684 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.684 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.684 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:22.684 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093149+0000", 2024-09-16T20:50:22.684 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0040000000000003e-05 2024-09-16T20:50:22.684 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.684 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.684 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:22.684 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093249+0000", 2024-09-16T20:50:22.684 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.9990999999999999e-05 2024-09-16T20:50:22.684 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:22.684 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:22.684 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:22.685 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.685 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.685 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3285 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 240640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:22.685 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041140+0000", 2024-09-16T20:50:22.685 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.557581773000003, 2024-09-16T20:50:22.685 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052375646999999997, 2024-09-16T20:50:22.685 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:22.685 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:22.685 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:22.685 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:22.685 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:22.685 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:22.685 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3285 2024-09-16T20:50:22.686 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.686 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:22.686 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.686 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:22.686 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041140+0000", 2024-09-16T20:50:22.686 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:22.686 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.686 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.686 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:22.686 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041140+0000", 2024-09-16T20:50:22.686 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:22.686 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.686 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.686 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:22.687 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041140+0000", 2024-09-16T20:50:22.687 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2399999999999999e-07 2024-09-16T20:50:22.687 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.687 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.687 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:22.687 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041141+0000", 2024-09-16T20:50:22.687 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2759999999999999e-06 2024-09-16T20:50:22.687 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.687 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.687 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:22.687 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041144+0000", 2024-09-16T20:50:22.687 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0280000000000001e-06 2024-09-16T20:50:22.687 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.687 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.688 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:22.688 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041147+0000", 2024-09-16T20:50:22.688 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5900000000000002e-06 2024-09-16T20:50:22.688 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.688 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.688 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:22.688 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093284+0000", 2024-09-16T20:50:22.688 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052137645000000003 2024-09-16T20:50:22.688 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.688 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.688 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:22.688 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093365+0000", 2024-09-16T20:50:22.688 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0446999999999997e-05 2024-09-16T20:50:22.689 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.689 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.689 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:22.689 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093515+0000", 2024-09-16T20:50:22.689 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00015043700000000001 2024-09-16T20:50:22.689 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:22.689 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:22.689 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:22.689 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.689 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.689 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3286 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 245760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:22.689 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:50:22.689 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.557569614000002, 2024-09-16T20:50:22.689 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052523564000000002, 2024-09-16T20:50:22.690 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:22.690 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:22.690 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:22.690 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:22.690 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:22.690 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:22.690 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3286 2024-09-16T20:50:22.690 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.690 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:22.690 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.690 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:22.690 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:50:22.690 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:22.690 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.691 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.691 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:22.691 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:50:22.691 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:22.691 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.691 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.691 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:22.691 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:50:22.691 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3900000000000001e-07 2024-09-16T20:50:22.691 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.691 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.691 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:22.691 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041153+0000", 2024-09-16T20:50:22.691 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.119e-06 2024-09-16T20:50:22.692 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.692 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.692 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:22.692 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041156+0000", 2024-09-16T20:50:22.692 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0000000000000001e-06 2024-09-16T20:50:22.692 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.692 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.692 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:22.692 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041159+0000", 2024-09-16T20:50:22.692 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5859999999999999e-06 2024-09-16T20:50:22.692 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.692 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.692 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:22.692 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093529+0000", 2024-09-16T20:50:22.693 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052370591000000001 2024-09-16T20:50:22.693 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.693 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.693 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:22.693 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093570+0000", 2024-09-16T20:50:22.693 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0161999999999999e-05 2024-09-16T20:50:22.693 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.693 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.693 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:22.693 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093675+0000", 2024-09-16T20:50:22.693 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010586699999999999 2024-09-16T20:50:22.693 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:22.693 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:22.693 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:22.694 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.694 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.694 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3287 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 250880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:22.694 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:50:22.694 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.557559564999998, 2024-09-16T20:50:22.694 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052781203999999998, 2024-09-16T20:50:22.694 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:22.694 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:22.694 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:22.694 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:22.694 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:22.694 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:22.694 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3287 2024-09-16T20:50:22.694 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.695 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:22.695 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.695 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:22.695 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:50:22.695 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:22.695 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.695 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.695 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:22.695 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:50:22.695 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:22.695 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.695 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.695 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:22.695 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:50:22.696 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1e-07 2024-09-16T20:50:22.696 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.696 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.696 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:22.696 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041163+0000", 2024-09-16T20:50:22.696 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0690000000000001e-06 2024-09-16T20:50:22.696 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.696 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.696 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:22.696 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041166+0000", 2024-09-16T20:50:22.696 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.982e-06 2024-09-16T20:50:22.696 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.696 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.696 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:22.697 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041169+0000", 2024-09-16T20:50:22.697 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.593e-06 2024-09-16T20:50:22.697 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.697 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.697 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:22.697 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093709+0000", 2024-09-16T20:50:22.697 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052540492000000001 2024-09-16T20:50:22.697 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.697 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.697 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:22.697 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093792+0000", 2024-09-16T20:50:22.697 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2436999999999997e-05 2024-09-16T20:50:22.697 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.697 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.698 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:22.698 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093943+0000", 2024-09-16T20:50:22.698 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000151421 2024-09-16T20:50:22.698 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:22.698 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:22.698 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:22.698 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.698 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.698 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3288 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 256000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:22.698 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:50:22.698 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.557549692000002, 2024-09-16T20:50:22.698 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052928269, 2024-09-16T20:50:22.698 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:22.698 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:22.699 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:22.699 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:22.699 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:22.699 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:22.699 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3288 2024-09-16T20:50:22.699 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.699 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:22.699 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.699 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:22.699 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:50:22.699 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:22.699 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.699 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.700 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:22.700 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:50:22.700 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:22.700 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.700 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.700 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:22.700 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:50:22.700 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2700000000000001e-07 2024-09-16T20:50:22.700 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.700 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.700 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:22.700 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041173+0000", 2024-09-16T20:50:22.700 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.113e-06 2024-09-16T20:50:22.701 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.701 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.701 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:22.701 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041176+0000", 2024-09-16T20:50:22.701 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9799999999999998e-06 2024-09-16T20:50:22.701 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.701 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.701 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:22.701 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041179+0000", 2024-09-16T20:50:22.701 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.667e-06 2024-09-16T20:50:22.701 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.701 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.701 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:22.701 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093957+0000", 2024-09-16T20:50:22.702 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052778703000000003 2024-09-16T20:50:22.702 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.702 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.702 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:22.702 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093999+0000", 2024-09-16T20:50:22.702 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1300000000000001e-05 2024-09-16T20:50:22.702 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.702 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.702 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:22.702 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094100+0000", 2024-09-16T20:50:22.702 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101279 2024-09-16T20:50:22.702 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:22.702 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:22.702 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:22.703 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.703 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.703 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3289 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 261120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:22.703 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:50:22.703 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.557539695999999, 2024-09-16T20:50:22.703 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053182976, 2024-09-16T20:50:22.703 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:22.703 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:22.703 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:22.703 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:22.703 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:22.703 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:22.703 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3289 2024-09-16T20:50:22.703 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.704 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:22.704 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.704 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:22.704 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:50:22.704 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:22.704 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.704 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.704 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:22.704 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:50:22.704 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:22.704 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.704 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.704 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:22.704 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:50:22.705 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2999999999999999e-07 2024-09-16T20:50:22.705 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.705 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.705 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:22.705 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041183+0000", 2024-09-16T20:50:22.705 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0720000000000001e-06 2024-09-16T20:50:22.705 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.705 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.705 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:22.705 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041186+0000", 2024-09-16T20:50:22.705 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9869999999999999e-06 2024-09-16T20:50:22.705 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.705 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.705 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:22.706 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041189+0000", 2024-09-16T20:50:22.706 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.587e-06 2024-09-16T20:50:22.706 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.706 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.706 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:22.706 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094136+0000", 2024-09-16T20:50:22.706 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052947270999999997 2024-09-16T20:50:22.706 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.706 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.706 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:22.706 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094213+0000", 2024-09-16T20:50:22.706 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7552000000000006e-05 2024-09-16T20:50:22.706 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:22.706 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:22.707 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:22.707 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094365+0000", 2024-09-16T20:50:22.707 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00015127700000000001 2024-09-16T20:50:22.707 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:22.707 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:22.707 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:22.707 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:22.707 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:22.707 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:22.709 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:50:22.709 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:22.707+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:22.710 INFO:teuthology.orchestra.run.smithi139.stdout:5841 punch from 0xd50e00 to 0xd57200, (0x6400 bytes) 2024-09-16T20:50:22.712 INFO:teuthology.orchestra.run.smithi139.stdout:5845 punch from 0x37a600 to 0x381400, (0x6e00 bytes) 2024-09-16T20:50:22.713 INFO:teuthology.orchestra.run.smithi139.stdout:5846 read 0x3b400 thru 0x413d5 (0x5fd6 bytes) 2024-09-16T20:50:22.716 INFO:teuthology.orchestra.run.smithi029.stderr:nodeep-scrub is set 2024-09-16T20:50:22.721 INFO:teuthology.orchestra.run.smithi139.stdout:5847 read 0xbe0200 thru 0xbeede6 (0xebe7 bytes) 2024-09-16T20:50:22.725 INFO:teuthology.orchestra.run.smithi139.stdout:5848 read 0x2c0e00 thru 0x2c576b (0x496c bytes) 2024-09-16T20:50:22.829 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:22.829 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:50:22.829 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:50:22.829 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:22.837 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:50:22.859 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:22.811+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:22.874 INFO:teuthology.orchestra.run.smithi139.stdout:5852 write 0xcc90200 thru 0xcc906b5 (0x4b6 bytes) 2024-09-16T20:50:22.892 INFO:teuthology.orchestra.run.smithi139.stdout:5855 write 0xdd36600 thru 0xdd3fc07 (0x9608 bytes) 2024-09-16T20:50:22.893 INFO:teuthology.orchestra.run.smithi139.stdout:5856 read 0xb85400 thru 0xb8b89f (0x64a0 bytes) 2024-09-16T20:50:22.897 INFO:teuthology.orchestra.run.smithi139.stdout:5857 punch from 0x5e6fc00 to 0x5e71000, (0x1400 bytes) 2024-09-16T20:50:22.903 INFO:teuthology.orchestra.run.smithi139.stdout:5858 trunc from 0xdd3fe00 to 0x989de00 2024-09-16T20:50:22.913 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:22.912+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:22.930 INFO:teuthology.orchestra.run.smithi139.stdout:5859 trunc from 0x989de00 to 0x3ce8200 2024-09-16T20:50:22.958 INFO:teuthology.orchestra.run.smithi139.stdout:5860 trunc from 0x3ce8200 to 0x6616600 2024-09-16T20:50:22.985 INFO:teuthology.orchestra.run.smithi139.stdout:5862 write 0x6fe0a00 thru 0x6fecc10 (0xc211 bytes) 2024-09-16T20:50:22.986 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:22.986 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:50:22.986 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:50:22.986 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:50:22.986 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:22.986 INFO:teuthology.orchestra.run.smithi139.stdout:5863 punch from 0x160b000 to 0x1619200, (0xe200 bytes) 2024-09-16T20:50:22.991 INFO:teuthology.orchestra.run.smithi139.stdout:5867 trunc from 0x6fece00 to 0xaec4a00 2024-09-16T20:50:22.997 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:50:23.008 INFO:teuthology.orchestra.run.smithi139.stdout:5868 read 0x7cb4c00 thru 0x7cbd6f1 (0x8af2 bytes) 2024-09-16T20:50:23.009 INFO:teuthology.orchestra.run.smithi139.stdout:5869 read 0x11dce00 thru 0x11e6e8c (0xa08d bytes) 2024-09-16T20:50:23.011 INFO:teuthology.orchestra.run.smithi139.stdout:5871 read 0x97e1600 thru 0x97e1b60 (0x561 bytes) 2024-09-16T20:50:23.011 INFO:teuthology.orchestra.run.smithi139.stdout:5874 read 0x1325600 thru 0x133181c (0xc21d bytes) 2024-09-16T20:50:23.011 INFO:teuthology.orchestra.run.smithi139.stdout:5876 read 0x2b20c00 thru 0x2b21da2 (0x11a3 bytes) 2024-09-16T20:50:23.011 INFO:teuthology.orchestra.run.smithi139.stdout:5878 write 0x554fa00 thru 0x5557c2c (0x822d bytes) 2024-09-16T20:50:23.011 INFO:teuthology.orchestra.run.smithi139.stdout:5881 clone 45 order 25 su 2097152 sc 9 2024-09-16T20:50:23.038 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:23.037+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:23.115 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:23.114+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:23.145 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:23.146 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:50:23.146 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:50:23.146 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:50:23.146 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.146 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3714 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1480704~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:23.146 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:50:23.146 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 120.50500900999999, 2024-09-16T20:50:23.146 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050152552000000003, 2024-09-16T20:50:23.146 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:23.146 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:23.147 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:23.147 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:23.147 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:23.147 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:23.147 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3714 2024-09-16T20:50:23.147 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.147 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:23.147 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.147 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:23.147 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:50:23.147 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:23.147 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.147 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.148 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:23.148 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:50:23.148 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:23.148 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.148 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.148 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:23.148 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:50:23.148 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0499999999999999e-07 2024-09-16T20:50:23.148 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.148 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.148 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:23.148 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639007+0000", 2024-09-16T20:50:23.149 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5340000000000001e-06 2024-09-16T20:50:23.149 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.149 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.149 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:23.149 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639013+0000", 2024-09-16T20:50:23.149 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3260000000000003e-06 2024-09-16T20:50:23.149 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.149 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.149 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.149 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639018+0000", 2024-09-16T20:50:23.149 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2850000000000002e-06 2024-09-16T20:50:23.149 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.149 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.150 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:23.150 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689026+0000", 2024-09-16T20:50:23.150 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.05000798 2024-09-16T20:50:23.150 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.150 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.150 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:23.150 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689077+0000", 2024-09-16T20:50:23.150 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1063999999999998e-05 2024-09-16T20:50:23.150 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.150 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.150 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:23.150 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689158+0000", 2024-09-16T20:50:23.150 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1057999999999995e-05 2024-09-16T20:50:23.150 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:23.151 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:23.151 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:23.151 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.151 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.151 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3716 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1488896~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:23.151 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:50:23.151 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 120.50499120000001, 2024-09-16T20:50:23.151 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050280142, 2024-09-16T20:50:23.151 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:23.151 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:23.151 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:23.151 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:23.151 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:23.152 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:23.152 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3716 2024-09-16T20:50:23.152 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.152 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:23.152 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.152 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:23.152 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:50:23.152 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:23.152 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.152 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.152 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:23.152 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:50:23.152 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:23.153 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.153 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.153 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:23.153 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639024+0000", 2024-09-16T20:50:23.153 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9999999999999997e-07 2024-09-16T20:50:23.153 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.153 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.153 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:23.153 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639026+0000", 2024-09-16T20:50:23.153 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.781e-06 2024-09-16T20:50:23.153 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.153 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.153 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:23.153 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639032+0000", 2024-09-16T20:50:23.154 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7780000000000004e-06 2024-09-16T20:50:23.154 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.154 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.154 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.154 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639036+0000", 2024-09-16T20:50:23.154 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6909999999999996e-06 2024-09-16T20:50:23.154 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.154 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.154 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:23.154 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689191+0000", 2024-09-16T20:50:23.154 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050155284000000001 2024-09-16T20:50:23.154 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.154 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.155 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:23.155 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689242+0000", 2024-09-16T20:50:23.155 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0662999999999999e-05 2024-09-16T20:50:23.155 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.155 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.155 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:23.155 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689304+0000", 2024-09-16T20:50:23.155 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1345000000000003e-05 2024-09-16T20:50:23.155 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:23.155 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:23.155 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:23.155 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.155 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.155 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3719 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1497088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:23.156 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:50:23.156 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 120.504963922, 2024-09-16T20:50:23.156 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050399715999999997, 2024-09-16T20:50:23.156 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:23.156 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:23.156 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:23.156 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:23.156 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:23.156 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:23.156 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3719 2024-09-16T20:50:23.156 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.156 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:23.156 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.156 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:23.156 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:50:23.157 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:23.157 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.157 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.157 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:23.157 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:50:23.157 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:23.157 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.157 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.157 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:23.157 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:50:23.157 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9700000000000002e-07 2024-09-16T20:50:23.157 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.157 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.158 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:23.158 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639053+0000", 2024-09-16T20:50:23.158 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2900000000000001e-06 2024-09-16T20:50:23.158 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.158 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.158 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:23.158 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639060+0000", 2024-09-16T20:50:23.158 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7750000000000004e-06 2024-09-16T20:50:23.158 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.158 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.158 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.158 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639066+0000", 2024-09-16T20:50:23.158 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0639999999999999e-06 2024-09-16T20:50:23.158 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.159 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.159 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:23.159 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689337+0000", 2024-09-16T20:50:23.159 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050270784999999998 2024-09-16T20:50:23.159 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.159 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.159 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:23.159 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689392+0000", 2024-09-16T20:50:23.159 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4920999999999998e-05 2024-09-16T20:50:23.159 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.159 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.159 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:23.159 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689450+0000", 2024-09-16T20:50:23.160 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8483999999999998e-05 2024-09-16T20:50:23.160 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:23.160 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:23.160 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:23.160 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.160 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.160 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3721 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1505280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:23.160 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:50:23.160 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 120.50494079000001, 2024-09-16T20:50:23.160 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050513608000000002, 2024-09-16T20:50:23.160 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:23.160 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:23.160 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:23.160 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:23.161 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:23.161 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:23.161 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3721 2024-09-16T20:50:23.161 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.161 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:23.161 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.161 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:23.161 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:50:23.161 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:23.161 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.161 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.161 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:23.161 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:50:23.161 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:23.162 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.162 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.162 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:23.162 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:50:23.162 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7e-07 2024-09-16T20:50:23.162 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.162 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.162 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:23.162 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639076+0000", 2024-09-16T20:50:23.162 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8899999999999999e-06 2024-09-16T20:50:23.162 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.162 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.162 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:23.162 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639083+0000", 2024-09-16T20:50:23.163 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1210000000000001e-06 2024-09-16T20:50:23.163 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.163 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.163 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.163 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639089+0000", 2024-09-16T20:50:23.163 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.643e-06 2024-09-16T20:50:23.163 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.163 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.163 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:23.163 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689474+0000", 2024-09-16T20:50:23.163 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050385679000000003 2024-09-16T20:50:23.163 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.163 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.164 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:23.164 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689533+0000", 2024-09-16T20:50:23.164 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8303000000000001e-05 2024-09-16T20:50:23.164 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.164 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.164 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:23.164 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689587+0000", 2024-09-16T20:50:23.164 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4601999999999997e-05 2024-09-16T20:50:23.164 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:23.164 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:23.164 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:23.164 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.164 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.165 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3723 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1513472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:23.165 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:50:23.165 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 120.504909986, 2024-09-16T20:50:23.165 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050699691999999998, 2024-09-16T20:50:23.165 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:23.165 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:23.165 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:23.165 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:23.165 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:23.165 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:23.165 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3723 2024-09-16T20:50:23.165 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.165 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:23.166 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.166 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:23.166 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:50:23.166 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:23.166 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.166 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.166 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:23.166 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:50:23.166 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:23.166 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.166 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.166 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:23.166 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:50:23.166 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3399999999999999e-07 2024-09-16T20:50:23.167 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.167 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.167 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:23.167 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639107+0000", 2024-09-16T20:50:23.167 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3860000000000001e-06 2024-09-16T20:50:23.167 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.167 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.167 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:23.167 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639114+0000", 2024-09-16T20:50:23.167 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9800000000000001e-06 2024-09-16T20:50:23.167 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.167 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.167 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.168 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639129+0000", 2024-09-16T20:50:23.168 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4887e-05 2024-09-16T20:50:23.168 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.168 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.168 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:23.168 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689625+0000", 2024-09-16T20:50:23.168 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050495848000000003 2024-09-16T20:50:23.168 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.168 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.168 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:23.168 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689720+0000", 2024-09-16T20:50:23.168 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4821000000000006e-05 2024-09-16T20:50:23.168 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.169 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.169 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:23.169 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689804+0000", 2024-09-16T20:50:23.169 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4235999999999998e-05 2024-09-16T20:50:23.169 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:23.169 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:23.169 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:23.169 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.169 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.169 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3725 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1521664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:23.169 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:50:23.169 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 120.50487639000001, 2024-09-16T20:50:23.169 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050797660000000001, 2024-09-16T20:50:23.170 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:23.170 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:23.170 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:23.170 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:23.170 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:23.170 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:23.170 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3725 2024-09-16T20:50:23.170 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.170 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:23.170 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.170 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:23.170 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:50:23.170 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:23.170 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.170 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.171 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:23.171 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:50:23.171 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:23.171 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.171 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.171 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:23.171 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639139+0000", 2024-09-16T20:50:23.171 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8999999999999997e-07 2024-09-16T20:50:23.171 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.171 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.171 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:23.171 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639141+0000", 2024-09-16T20:50:23.171 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.182e-06 2024-09-16T20:50:23.171 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.172 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.172 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:23.172 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639154+0000", 2024-09-16T20:50:23.172 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3626e-05 2024-09-16T20:50:23.172 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.172 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.172 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.172 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639160+0000", 2024-09-16T20:50:23.172 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6810000000000003e-06 2024-09-16T20:50:23.172 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.172 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.172 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:23.173 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689831+0000", 2024-09-16T20:50:23.173 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050670907000000001 2024-09-16T20:50:23.173 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.173 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.173 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:23.173 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689882+0000", 2024-09-16T20:50:23.173 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0445999999999998e-05 2024-09-16T20:50:23.173 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.173 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.174 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:23.174 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689936+0000", 2024-09-16T20:50:23.174 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4327999999999999e-05 2024-09-16T20:50:23.174 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:23.174 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:23.174 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:23.174 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.174 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.175 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3727 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1529856~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:23.175 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:50:23.175 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 120.504840813, 2024-09-16T20:50:23.175 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051016922999999999, 2024-09-16T20:50:23.175 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:23.175 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:23.175 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:23.175 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:23.175 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:23.175 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:23.175 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3727 2024-09-16T20:50:23.175 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.175 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:23.175 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.176 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:23.176 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:50:23.176 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:23.176 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.176 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.176 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:23.176 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:50:23.176 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:23.176 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.176 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.176 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:23.176 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:50:23.176 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9999999999999998e-07 2024-09-16T20:50:23.177 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.177 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.177 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:23.177 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639183+0000", 2024-09-16T20:50:23.177 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2399999999999996e-06 2024-09-16T20:50:23.177 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.177 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.177 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:23.177 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639195+0000", 2024-09-16T20:50:23.177 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1113e-05 2024-09-16T20:50:23.177 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.177 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.177 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.177 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639206+0000", 2024-09-16T20:50:23.178 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1503e-05 2024-09-16T20:50:23.178 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.178 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.178 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:23.178 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689969+0000", 2024-09-16T20:50:23.178 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050762676 2024-09-16T20:50:23.178 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.178 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.178 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:23.178 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690071+0000", 2024-09-16T20:50:23.178 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102163 2024-09-16T20:50:23.178 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.178 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.178 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:23.179 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690191+0000", 2024-09-16T20:50:23.179 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000119728 2024-09-16T20:50:23.179 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:23.179 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:23.179 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:23.179 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.179 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.179 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3730 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1538048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:23.179 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:50:23.179 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 120.50479925, 2024-09-16T20:50:23.179 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051131467999999999, 2024-09-16T20:50:23.179 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:23.179 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:23.179 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:23.180 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:23.180 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:23.180 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:23.180 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3730 2024-09-16T20:50:23.180 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.180 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:23.180 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.180 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:23.180 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:50:23.180 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:23.180 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.180 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.180 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:23.181 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:50:23.181 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:23.181 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.181 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.181 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:23.181 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639216+0000", 2024-09-16T20:50:23.181 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.39e-07 2024-09-16T20:50:23.181 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.181 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.181 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:23.181 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639218+0000", 2024-09-16T20:50:23.181 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.537e-06 2024-09-16T20:50:23.181 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.181 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.182 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:23.182 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639229+0000", 2024-09-16T20:50:23.182 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1122e-05 2024-09-16T20:50:23.182 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.182 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.182 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.182 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639237+0000", 2024-09-16T20:50:23.182 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8509999999999999e-06 2024-09-16T20:50:23.182 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.182 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.182 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:23.182 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690227+0000", 2024-09-16T20:50:23.182 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050990147 2024-09-16T20:50:23.182 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.183 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.183 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:23.183 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690278+0000", 2024-09-16T20:50:23.183 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0223999999999999e-05 2024-09-16T20:50:23.183 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.183 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.183 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:23.183 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690347+0000", 2024-09-16T20:50:23.183 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9147999999999996e-05 2024-09-16T20:50:23.183 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:23.183 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:23.183 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:23.183 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.183 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.184 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3731 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1546240~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:23.184 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:50:23.184 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 120.50476854199999, 2024-09-16T20:50:23.184 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051276744999999999, 2024-09-16T20:50:23.184 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:23.184 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:23.184 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:23.184 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:23.184 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:23.184 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:23.184 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3731 2024-09-16T20:50:23.184 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.184 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:23.185 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.185 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:23.185 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:50:23.185 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:23.185 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.185 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.185 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:23.185 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:50:23.185 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:23.185 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.185 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.185 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:23.185 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:50:23.185 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9900000000000001e-07 2024-09-16T20:50:23.186 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.186 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.186 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:23.186 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639249+0000", 2024-09-16T20:50:23.186 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3630000000000001e-06 2024-09-16T20:50:23.186 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.186 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.186 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:23.186 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639255+0000", 2024-09-16T20:50:23.186 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4960000000000001e-06 2024-09-16T20:50:23.186 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.186 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.186 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.186 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639261+0000", 2024-09-16T20:50:23.186 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2839999999999997e-06 2024-09-16T20:50:23.187 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.187 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.187 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:23.187 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690375+0000", 2024-09-16T20:50:23.187 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051114684000000001 2024-09-16T20:50:23.187 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.187 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.187 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:23.187 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690425+0000", 2024-09-16T20:50:23.187 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9379999999999998e-05 2024-09-16T20:50:23.187 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.187 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.187 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:23.187 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690523+0000", 2024-09-16T20:50:23.188 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8139000000000003e-05 2024-09-16T20:50:23.188 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:23.188 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:23.188 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:23.188 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.188 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.188 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3732 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1554432~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:23.188 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:50:23.188 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 120.50472276000001, 2024-09-16T20:50:23.188 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051380096, 2024-09-16T20:50:23.188 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:23.188 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:23.188 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:23.188 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:23.189 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:23.189 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:23.189 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3732 2024-09-16T20:50:23.189 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.189 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:23.189 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.189 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:23.189 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:50:23.189 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:23.189 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.189 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.189 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:23.189 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:50:23.190 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:23.190 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.190 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.190 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:23.190 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:50:23.190 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6100000000000001e-07 2024-09-16T20:50:23.190 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.190 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.190 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:23.190 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639295+0000", 2024-09-16T20:50:23.190 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.447e-06 2024-09-16T20:50:23.190 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.190 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.190 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:23.191 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639302+0000", 2024-09-16T20:50:23.191 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0570000000000004e-06 2024-09-16T20:50:23.191 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.191 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.191 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.191 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639307+0000", 2024-09-16T20:50:23.191 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1429999999999997e-06 2024-09-16T20:50:23.191 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.191 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.191 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:23.191 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690555+0000", 2024-09-16T20:50:23.191 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051247963000000001 2024-09-16T20:50:23.191 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.191 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.192 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:23.192 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690602+0000", 2024-09-16T20:50:23.192 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7602000000000003e-05 2024-09-16T20:50:23.192 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.192 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.192 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:23.192 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690672+0000", 2024-09-16T20:50:23.192 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9423000000000003e-05 2024-09-16T20:50:23.192 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:23.192 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:23.192 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:23.192 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.192 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.193 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3734 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1562624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:23.193 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:50:23.193 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 120.50470111, 2024-09-16T20:50:23.193 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051490610999999999, 2024-09-16T20:50:23.193 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:23.193 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:23.193 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:23.193 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:23.193 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:23.193 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:23.193 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3734 2024-09-16T20:50:23.193 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.193 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:23.194 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.194 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:23.194 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:50:23.194 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:23.194 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.194 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.194 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:23.194 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:50:23.194 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:23.194 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.194 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.194 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:23.194 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639314+0000", 2024-09-16T20:50:23.194 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4400000000000001e-07 2024-09-16T20:50:23.195 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.195 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.195 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:23.195 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639316+0000", 2024-09-16T20:50:23.195 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1459999999999999e-06 2024-09-16T20:50:23.195 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.195 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.195 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:23.195 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639322+0000", 2024-09-16T20:50:23.195 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7610000000000004e-06 2024-09-16T20:50:23.195 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.195 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.195 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.196 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639327+0000", 2024-09-16T20:50:23.196 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.186e-06 2024-09-16T20:50:23.196 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.196 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.196 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:23.196 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690698+0000", 2024-09-16T20:50:23.196 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051371223000000001 2024-09-16T20:50:23.196 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.196 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.196 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:23.196 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690748+0000", 2024-09-16T20:50:23.196 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0192999999999999e-05 2024-09-16T20:50:23.197 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.197 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.197 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:23.197 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690804+0000", 2024-09-16T20:50:23.197 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5658000000000001e-05 2024-09-16T20:50:23.197 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:23.197 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:23.197 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:23.197 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.197 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.197 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3736 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1570816~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:23.197 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:50:23.197 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 120.504681851, 2024-09-16T20:50:23.198 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051596528000000003, 2024-09-16T20:50:23.198 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:23.198 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:23.198 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:23.198 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:23.198 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:23.198 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:23.198 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3736 2024-09-16T20:50:23.198 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.198 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:23.198 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.198 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:23.198 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:50:23.198 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:23.199 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.199 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.199 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:23.199 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:50:23.199 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:23.199 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.199 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.199 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:23.199 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:50:23.199 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9999999999999999e-07 2024-09-16T20:50:23.199 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.199 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.199 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:23.200 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639335+0000", 2024-09-16T20:50:23.200 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.635e-06 2024-09-16T20:50:23.200 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.200 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.200 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:23.200 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639340+0000", 2024-09-16T20:50:23.200 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5840000000000002e-06 2024-09-16T20:50:23.200 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.200 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.200 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.200 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639346+0000", 2024-09-16T20:50:23.200 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1190000000000002e-06 2024-09-16T20:50:23.201 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.201 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.201 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:23.201 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690817+0000", 2024-09-16T20:50:23.201 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051470721999999997 2024-09-16T20:50:23.201 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.201 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.201 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:23.201 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690875+0000", 2024-09-16T20:50:23.201 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7441000000000001e-05 2024-09-16T20:50:23.201 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.201 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.201 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:23.201 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690929+0000", 2024-09-16T20:50:23.202 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4727e-05 2024-09-16T20:50:23.202 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:23.202 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:23.202 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:23.202 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.202 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.202 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5434 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:23.202 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:50:23.202 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 120.279420285, 2024-09-16T20:50:23.202 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099755831000000003, 2024-09-16T20:50:23.203 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:23.203 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:23.203 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:23.203 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:23.203 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:23.203 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:23.203 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5434 2024-09-16T20:50:23.203 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.203 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:23.203 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.203 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:23.203 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:50:23.203 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:23.203 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.204 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.204 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:23.204 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:50:23.204 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:23.204 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.204 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.204 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:23.204 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864595+0000", 2024-09-16T20:50:23.204 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1099999999999995e-07 2024-09-16T20:50:23.204 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.204 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.204 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:23.204 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864597+0000", 2024-09-16T20:50:23.204 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3989999999999998e-06 2024-09-16T20:50:23.205 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.205 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.205 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:23.205 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864603+0000", 2024-09-16T20:50:23.205 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9309999999999996e-06 2024-09-16T20:50:23.205 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.205 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.205 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.205 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864607+0000", 2024-09-16T20:50:23.205 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6200000000000001e-06 2024-09-16T20:50:23.205 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.205 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.205 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:23.206 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964164+0000", 2024-09-16T20:50:23.206 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099557162000000005 2024-09-16T20:50:23.206 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.206 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.206 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:23.206 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964244+0000", 2024-09-16T20:50:23.206 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9597000000000004e-05 2024-09-16T20:50:23.206 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.206 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.206 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:23.206 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964350+0000", 2024-09-16T20:50:23.206 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010641099999999999 2024-09-16T20:50:23.206 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:23.206 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:23.207 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:23.207 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.207 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.207 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5435 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 20480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:23.207 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:50:23.207 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 120.279390463, 2024-09-16T20:50:23.207 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100897102, 2024-09-16T20:50:23.207 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:23.207 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:23.207 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:23.207 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:23.207 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:23.207 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:23.207 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5435 2024-09-16T20:50:23.208 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.208 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:23.208 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.208 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:23.208 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:50:23.208 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:23.208 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.208 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.208 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:23.208 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:50:23.208 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:23.208 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.208 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.208 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:23.209 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:50:23.209 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8000000000000001e-07 2024-09-16T20:50:23.209 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.209 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.209 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:23.209 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864626+0000", 2024-09-16T20:50:23.209 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.745e-06 2024-09-16T20:50:23.209 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.209 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.209 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:23.209 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864633+0000", 2024-09-16T20:50:23.209 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3339999999999998e-06 2024-09-16T20:50:23.209 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.209 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.210 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.210 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864636+0000", 2024-09-16T20:50:23.210 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1200000000000002e-06 2024-09-16T20:50:23.210 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.210 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.210 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:23.210 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965299+0000", 2024-09-16T20:50:23.210 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10066352000000001 2024-09-16T20:50:23.210 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.210 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.210 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:23.210 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965390+0000", 2024-09-16T20:50:23.210 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0894000000000001e-05 2024-09-16T20:50:23.210 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.211 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.211 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:23.211 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965521+0000", 2024-09-16T20:50:23.211 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013110899999999999 2024-09-16T20:50:23.211 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:23.211 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:23.211 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:23.211 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.211 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.211 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:152 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 327680~32768 out=32768b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:50:23.211 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:50:23.211 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 118.980893521, 2024-09-16T20:50:23.211 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11869405800000001, 2024-09-16T20:50:23.212 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:23.212 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:23.212 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:23.212 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:23.212 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:50:23.212 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:50:23.212 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 152 2024-09-16T20:50:23.212 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.212 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:23.212 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.212 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:23.212 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:50:23.212 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:23.213 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.213 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.213 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:23.213 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:50:23.213 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:23.213 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.213 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.213 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:23.213 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163122+0000", 2024-09-16T20:50:23.213 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6400000000000003e-07 2024-09-16T20:50:23.213 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.213 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.213 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:23.213 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163124+0000", 2024-09-16T20:50:23.214 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.565e-06 2024-09-16T20:50:23.214 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.214 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.214 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:23.214 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163136+0000", 2024-09-16T20:50:23.214 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1717e-05 2024-09-16T20:50:23.214 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.215 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.216 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.216 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163142+0000", 2024-09-16T20:50:23.216 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1750000000000002e-06 2024-09-16T20:50:23.216 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.216 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.216 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:23.216 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163606+0000", 2024-09-16T20:50:23.216 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00046418799999999998 2024-09-16T20:50:23.216 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.216 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.216 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:23.216 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163656+0000", 2024-09-16T20:50:23.216 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9852999999999999e-05 2024-09-16T20:50:23.216 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.217 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.217 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:23.217 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281815+0000", 2024-09-16T20:50:23.217 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11815909600000001 2024-09-16T20:50:23.217 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:23.217 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:23.217 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:23.217 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.217 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.217 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:153 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:50:23.217 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:50:23.217 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 118.980860568, 2024-09-16T20:50:23.217 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11903462400000001, 2024-09-16T20:50:23.218 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:23.218 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:23.218 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:23.218 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:23.218 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:50:23.218 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:50:23.218 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 153 2024-09-16T20:50:23.218 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.218 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:23.218 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.218 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:23.218 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:50:23.218 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:23.218 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.219 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.219 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:23.219 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:50:23.219 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:23.219 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.219 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.219 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:23.219 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163155+0000", 2024-09-16T20:50:23.219 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.37e-07 2024-09-16T20:50:23.219 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.219 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.219 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:23.219 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163157+0000", 2024-09-16T20:50:23.219 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6010000000000002e-06 2024-09-16T20:50:23.219 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.220 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.220 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:23.220 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163165+0000", 2024-09-16T20:50:23.220 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2619999999999998e-06 2024-09-16T20:50:23.220 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.220 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.220 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.220 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163171+0000", 2024-09-16T20:50:23.220 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.711e-06 2024-09-16T20:50:23.220 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.220 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.220 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:23.220 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281868+0000", 2024-09-16T20:50:23.221 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118696622 2024-09-16T20:50:23.221 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.221 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.221 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:23.221 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281969+0000", 2024-09-16T20:50:23.221 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101371 2024-09-16T20:50:23.221 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.221 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.221 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:23.221 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282189+0000", 2024-09-16T20:50:23.221 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00021952000000000001 2024-09-16T20:50:23.221 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:23.221 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:23.222 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:23.222 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.222 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.222 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:154 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 360448~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:50:23.222 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:50:23.222 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 118.980828796, 2024-09-16T20:50:23.222 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11932035100000001, 2024-09-16T20:50:23.222 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:23.222 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:23.222 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:23.222 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:23.222 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:50:23.222 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:50:23.222 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 154 2024-09-16T20:50:23.223 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.223 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:23.223 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.223 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:23.223 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:50:23.223 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:23.223 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.223 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.223 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:23.223 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:50:23.223 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:23.223 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.223 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.223 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:23.224 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:50:23.224 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:50:23.224 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.224 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.224 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:23.224 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163189+0000", 2024-09-16T20:50:23.224 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6840000000000001e-06 2024-09-16T20:50:23.224 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.224 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.224 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:23.224 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163196+0000", 2024-09-16T20:50:23.224 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7920000000000004e-06 2024-09-16T20:50:23.224 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.225 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.225 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.225 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163201+0000", 2024-09-16T20:50:23.225 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2809999999999999e-06 2024-09-16T20:50:23.225 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.225 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.225 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:23.225 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282219+0000", 2024-09-16T20:50:23.225 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.119018343 2024-09-16T20:50:23.225 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.225 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.225 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:23.225 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282305+0000", 2024-09-16T20:50:23.225 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5865000000000005e-05 2024-09-16T20:50:23.226 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.226 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.226 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:23.226 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282506+0000", 2024-09-16T20:50:23.226 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000200927 2024-09-16T20:50:23.226 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:23.226 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:23.226 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:23.226 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.226 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.226 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop(client.4366.0:3088 3.1e e38/23 3:79e5ec01:::rbd_header.110e891a9b2b:head v 38'306, mlcod=36'304)", 2024-09-16T20:50:23.226 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:50:23.226 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 118.97980992799999, 2024-09-16T20:50:23.226 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118102759, 2024-09-16T20:50:23.227 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:23.227 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:23.227 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:23.227 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:23.227 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.227 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:23.227 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:50:23.227 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:23.227 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.227 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.227 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:23.227 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:50:23.227 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:23.227 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.228 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.228 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:23.228 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:50:23.228 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.85e-07 2024-09-16T20:50:23.228 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.228 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.228 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:23.228 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164212+0000", 2024-09-16T20:50:23.228 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3999999999999997e-06 2024-09-16T20:50:23.228 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.228 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.228 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:23.229 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281553+0000", 2024-09-16T20:50:23.229 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11734135499999999 2024-09-16T20:50:23.229 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.230 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.230 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.230 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281567+0000", 2024-09-16T20:50:23.230 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4188999999999999e-05 2024-09-16T20:50:23.230 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.230 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.230 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:23.230 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281653+0000", 2024-09-16T20:50:23.230 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5385999999999999e-05 2024-09-16T20:50:23.230 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.230 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.230 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:23.230 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281679+0000", 2024-09-16T20:50:23.231 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6689000000000001e-05 2024-09-16T20:50:23.231 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.231 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.231 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:23.231 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282277+0000", 2024-09-16T20:50:23.231 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00059798000000000002 2024-09-16T20:50:23.231 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.231 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.231 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:23.231 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282307+0000", 2024-09-16T20:50:23.231 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9975e-05 2024-09-16T20:50:23.231 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:23.231 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:23.231 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:23.232 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.232 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.232 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5255.0:44 3.16 3:6f5476df:::rbd_header.110e25db1f60:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:50:23.232 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:50:23.232 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 46.823506872999999, 2024-09-16T20:50:23.232 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.121007509, 2024-09-16T20:50:23.232 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:23.232 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:23.232 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:23.232 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:23.232 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5255", 2024-09-16T20:50:23.232 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:50:23.232 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 44 2024-09-16T20:50:23.233 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.233 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:23.233 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.233 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:23.233 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:50:23.233 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:23.233 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.233 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.233 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:23.233 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:50:23.233 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:23.233 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.233 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.233 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:23.234 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:50:23.234 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0500000000000003e-07 2024-09-16T20:50:23.234 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.234 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.234 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:23.234 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320512+0000", 2024-09-16T20:50:23.234 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2540000000000001e-06 2024-09-16T20:50:23.234 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.234 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.234 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:23.234 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320518+0000", 2024-09-16T20:50:23.234 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1689999999999997e-06 2024-09-16T20:50:23.234 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.235 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.235 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.235 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320521+0000", 2024-09-16T20:50:23.235 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5200000000000002e-06 2024-09-16T20:50:23.235 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.235 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.235 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:23.235 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320548+0000", 2024-09-16T20:50:23.235 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6662999999999999e-05 2024-09-16T20:50:23.235 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.235 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.235 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:23.235 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320587+0000", 2024-09-16T20:50:23.235 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8866999999999999e-05 2024-09-16T20:50:23.236 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.236 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.236 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:23.236 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.441515+0000", 2024-09-16T20:50:23.236 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.120928331 2024-09-16T20:50:23.236 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:23.236 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:23.236 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:23.236 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.236 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.236 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:20329 3.5 3:a4ea64b7:::rbd_header.110ede5ef857:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:23.236 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:01.061694+0000", 2024-09-16T20:50:23.236 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 22.082320305, 2024-09-16T20:50:23.237 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.071234077000000007, 2024-09-16T20:50:23.237 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:23.237 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:23.237 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:23.237 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:23.237 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:50:23.237 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:23.237 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 20329 2024-09-16T20:50:23.237 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.237 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:23.237 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.237 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:23.237 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.061694+0000", 2024-09-16T20:50:23.237 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:23.238 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.238 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.238 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:23.238 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.061694+0000", 2024-09-16T20:50:23.238 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:23.238 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.238 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.238 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:23.238 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089232+0000", 2024-09-16T20:50:23.238 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.027537275 2024-09-16T20:50:23.238 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.238 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.238 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:23.238 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089237+0000", 2024-09-16T20:50:23.239 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4709999999999998e-06 2024-09-16T20:50:23.239 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.239 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.239 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:23.239 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089246+0000", 2024-09-16T20:50:23.239 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8230000000000001e-06 2024-09-16T20:50:23.239 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.239 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.239 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.239 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089252+0000", 2024-09-16T20:50:23.239 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3060000000000003e-06 2024-09-16T20:50:23.239 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.239 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.239 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:23.239 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.132756+0000", 2024-09-16T20:50:23.240 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.043504334999999998 2024-09-16T20:50:23.240 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.240 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.240 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:23.240 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.132829+0000", 2024-09-16T20:50:23.240 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2849999999999995e-05 2024-09-16T20:50:23.240 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:23.240 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:23.240 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:23.240 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.132928+0000", 2024-09-16T20:50:23.240 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.9017000000000004e-05 2024-09-16T20:50:23.240 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:23.240 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:23.240 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:23.241 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:23.241 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:23.241 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:23.242 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:50:23.243 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:23.241+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:23.343 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:23.342+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:23.351 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:23.351 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:50:23.351 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:50:23.351 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:23.360 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:50:23.444 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:23.443+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:23.507 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:23.507 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:50:23.507 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:50:23.507 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:50:23.508 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:23.516 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:50:23.559 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:23.558+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:23.624 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:23.624 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:50:23.624 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:50:23.624 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:50:23.624 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.624 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:3088 3.1e 3:79e5ec01:::rbd_header.110e891a9b2b:head [watch watch cookie 140213437171488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:50:23.624 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:50:23.624 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 119.458292552, 2024-09-16T20:50:23.624 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.11895186200000001, 2024-09-16T20:50:23.625 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:23.625 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:23.625 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:23.625 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:23.625 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:50:23.625 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:23.625 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3088 2024-09-16T20:50:23.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.625 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:23.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.625 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:23.625 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:50:23.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.626 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:23.626 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:50:23.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.626 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:23.626 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163846+0000", 2024-09-16T20:50:23.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6799999999999999e-07 2024-09-16T20:50:23.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.626 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:23.627 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163850+0000", 2024-09-16T20:50:23.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.929e-06 2024-09-16T20:50:23.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.627 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:23.627 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163862+0000", 2024-09-16T20:50:23.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1972e-05 2024-09-16T20:50:23.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.627 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.627 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163867+0000", 2024-09-16T20:50:23.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.451e-06 2024-09-16T20:50:23.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:23.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163912+0000", 2024-09-16T20:50:23.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4456000000000003e-05 2024-09-16T20:50:23.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:23.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163962+0000", 2024-09-16T20:50:23.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0578000000000003e-05 2024-09-16T20:50:23.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-16T20:50:23.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164058+0000", 2024-09-16T20:50:23.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5934999999999995e-05 2024-09-16T20:50:23.629 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.629 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.629 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:50:23.629 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164570+0000", 2024-09-16T20:50:23.629 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00051241100000000005 2024-09-16T20:50:23.629 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.629 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.629 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:23.629 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282629+0000", 2024-09-16T20:50:23.629 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.118058986 2024-09-16T20:50:23.629 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.629 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.629 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:50:23.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282725+0000", 2024-09-16T20:50:23.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5063999999999996e-05 2024-09-16T20:50:23.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:23.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282797+0000", 2024-09-16T20:50:23.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2212000000000006e-05 2024-09-16T20:50:23.631 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.631 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:23.631 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.631 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.631 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:8039 3.1e 3.ac44ff9e (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:50:23.631 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:50:23.631 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 93.914465203000006, 2024-09-16T20:50:23.631 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0053310449999999, 2024-09-16T20:50:23.631 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:23.631 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:23.632 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:23.632 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:23.632 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:50:23.632 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:23.632 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8039 2024-09-16T20:50:23.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.632 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:23.632 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.632 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:23.632 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:50:23.632 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.632 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.633 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:23.633 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:50:23.633 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.633 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.633 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:23.633 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707674+0000", 2024-09-16T20:50:23.633 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6530000000000001e-06 2024-09-16T20:50:23.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.633 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.633 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:23.633 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707681+0000", 2024-09-16T20:50:23.633 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9639999999999997e-06 2024-09-16T20:50:23.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.634 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:23.634 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707700+0000", 2024-09-16T20:50:23.634 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8674000000000001e-05 2024-09-16T20:50:23.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.634 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.634 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707708+0000", 2024-09-16T20:50:23.634 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6810000000000004e-06 2024-09-16T20:50:23.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.634 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:23.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711293+0000", 2024-09-16T20:50:23.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035848189999999999 2024-09-16T20:50:23.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.635 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:23.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711301+0000", 2024-09-16T20:50:23.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4730000000000003e-06 2024-09-16T20:50:23.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.635 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:23.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.712990+0000", 2024-09-16T20:50:23.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.001689348 2024-09-16T20:50:23.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.635 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:23.636 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713003+0000", 2024-09-16T20:50:23.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3433e-05 2024-09-16T20:50:23.636 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.636 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:23.636 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.636 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:8040 3.1e 3.ac44ff9e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:50:23.636 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:50:23.636 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 93.914404808, 2024-09-16T20:50:23.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0053410780000001, 2024-09-16T20:50:23.636 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:23.636 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:23.637 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:23.637 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:23.637 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:50:23.637 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:23.637 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8040 2024-09-16T20:50:23.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.637 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:23.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.637 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:23.637 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:50:23.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.637 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:23.637 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:50:23.638 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.638 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.638 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:23.638 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:50:23.638 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8100000000000003e-07 2024-09-16T20:50:23.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.638 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.638 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:23.638 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707737+0000", 2024-09-16T20:50:23.638 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5020000000000001e-06 2024-09-16T20:50:23.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.638 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.638 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:23.639 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707745+0000", 2024-09-16T20:50:23.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2169999999999994e-06 2024-09-16T20:50:23.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.639 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.639 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707751+0000", 2024-09-16T20:50:23.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.263e-06 2024-09-16T20:50:23.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.639 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:23.639 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711336+0000", 2024-09-16T20:50:23.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035848360000000001 2024-09-16T20:50:23.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.640 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:23.640 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711340+0000", 2024-09-16T20:50:23.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1309999999999999e-06 2024-09-16T20:50:23.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.640 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:23.640 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713060+0000", 2024-09-16T20:50:23.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0017195059999999 2024-09-16T20:50:23.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.640 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:23.640 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713074+0000", 2024-09-16T20:50:23.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4042e-05 2024-09-16T20:50:23.641 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.641 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:23.641 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.641 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4927.0:4487 3.1e 3.ac44ff9e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:50:23.641 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:50:23.641 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 93.913884605999996, 2024-09-16T20:50:23.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0048840880000001, 2024-09-16T20:50:23.641 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:23.641 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:23.641 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:23.641 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:23.641 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4927", 2024-09-16T20:50:23.641 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:23.642 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4487 2024-09-16T20:50:23.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.642 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:23.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.642 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:23.642 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:50:23.642 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.642 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:23.642 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:50:23.642 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.643 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:23.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708254+0000", 2024-09-16T20:50:23.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1800000000000005e-07 2024-09-16T20:50:23.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.643 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:23.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708258+0000", 2024-09-16T20:50:23.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8689999999999997e-06 2024-09-16T20:50:23.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.643 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:23.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708267+0000", 2024-09-16T20:50:23.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9590000000000007e-06 2024-09-16T20:50:23.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.644 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.644 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708274+0000", 2024-09-16T20:50:23.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9970000000000001e-06 2024-09-16T20:50:23.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.644 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:23.644 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711366+0000", 2024-09-16T20:50:23.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030919530000000001 2024-09-16T20:50:23.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:23.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711372+0000", 2024-09-16T20:50:23.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9410000000000004e-06 2024-09-16T20:50:23.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:23.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713126+0000", 2024-09-16T20:50:23.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0017537249999999 2024-09-16T20:50:23.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:23.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713137+0000", 2024-09-16T20:50:23.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0825999999999999e-05 2024-09-16T20:50:23.646 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.646 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:23.646 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.646 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5089.0:11703 3.1b 3:dcd4684e:::rbd_data.110e747f0779.000000000000001b:head [set-alloc-hint object_size 262144 write_size 262144,write 139264~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:50:23.646 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:50:23.646 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 79.484148832000002, 2024-09-16T20:50:23.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.084231385000000006, 2024-09-16T20:50:23.646 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:23.646 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:23.646 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:23.647 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:23.647 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5089", 2024-09-16T20:50:23.647 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:50:23.647 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 11703 2024-09-16T20:50:23.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.647 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:23.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.647 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:23.647 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:50:23.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.647 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:23.647 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:50:23.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:23.648 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:50:23.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2600000000000001e-07 2024-09-16T20:50:23.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:23.648 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137998+0000", 2024-09-16T20:50:23.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9549999999999998e-06 2024-09-16T20:50:23.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:23.648 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138008+0000", 2024-09-16T20:50:23.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3789999999999995e-06 2024-09-16T20:50:23.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138013+0000", 2024-09-16T20:50:23.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2209999999999996e-06 2024-09-16T20:50:23.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:23.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138054+0000", 2024-09-16T20:50:23.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1022999999999997e-05 2024-09-16T20:50:23.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:23.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138124+0000", 2024-09-16T20:50:23.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0234999999999994e-05 2024-09-16T20:50:23.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.650 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:23.650 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138197+0000", 2024-09-16T20:50:23.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2451000000000001e-05 2024-09-16T20:50:23.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.650 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:23.650 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222010+0000", 2024-09-16T20:50:23.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.083813433000000007 2024-09-16T20:50:23.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.650 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:50:23.650 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222163+0000", 2024-09-16T20:50:23.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015274599999999999 2024-09-16T20:50:23.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.651 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:50:23.651 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222197+0000", 2024-09-16T20:50:23.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3683000000000003e-05 2024-09-16T20:50:23.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.651 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:23.651 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222220+0000", 2024-09-16T20:50:23.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3433000000000001e-05 2024-09-16T20:50:23.651 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.651 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:23.651 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.652 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.5089.0:11702 3.15 e73/64 3:acf33c8b:::rbd_data.110e747f0779.000000000000001a:head v 73'132, mlcod=73'130)", 2024-09-16T20:50:23.652 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:50:23.652 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 79.483881503000006, 2024-09-16T20:50:23.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.084079763000000002, 2024-09-16T20:50:23.652 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:23.652 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:23.652 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:23.652 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:23.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.652 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:23.652 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:50:23.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:23.653 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:50:23.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:23.653 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:50:23.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:50:23.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:23.653 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138263+0000", 2024-09-16T20:50:23.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1e-06 2024-09-16T20:50:23.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:23.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221589+0000", 2024-09-16T20:50:23.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.083326002999999996 2024-09-16T20:50:23.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221614+0000", 2024-09-16T20:50:23.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5593999999999999e-05 2024-09-16T20:50:23.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:23.655 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221714+0000", 2024-09-16T20:50:23.655 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9889000000000005e-05 2024-09-16T20:50:23.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.655 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.655 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:23.655 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221727+0000", 2024-09-16T20:50:23.655 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3125999999999999e-05 2024-09-16T20:50:23.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.655 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.655 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:50:23.655 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222294+0000", 2024-09-16T20:50:23.655 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00056712199999999998 2024-09-16T20:50:23.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.655 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.656 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:23.656 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222336+0000", 2024-09-16T20:50:23.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1470000000000001e-05 2024-09-16T20:50:23.656 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.656 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:23.656 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.656 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5138.0:9358 3.c 3:33996a43:::rbd_data.110e6372565d.0000000000000007:head [read 1343488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T20:50:23.656 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:50:23.656 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 55.769606093, 2024-09-16T20:50:23.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0167493569999999, 2024-09-16T20:50:23.656 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:23.656 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:23.657 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:23.657 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:23.657 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5138", 2024-09-16T20:50:23.657 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:50:23.657 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9358 2024-09-16T20:50:23.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.657 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:23.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.657 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:23.657 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:50:23.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.657 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:23.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:50:23.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.658 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:23.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852533+0000", 2024-09-16T20:50:23.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.646e-06 2024-09-16T20:50:23.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.658 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:23.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852539+0000", 2024-09-16T20:50:23.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3389999999999998e-06 2024-09-16T20:50:23.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.659 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.659 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:23.659 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852552+0000", 2024-09-16T20:50:23.659 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.271e-05 2024-09-16T20:50:23.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.659 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.659 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.659 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852563+0000", 2024-09-16T20:50:23.659 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1139e-05 2024-09-16T20:50:23.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.659 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.659 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:23.659 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852883+0000", 2024-09-16T20:50:23.659 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00032010600000000001 2024-09-16T20:50:23.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.660 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:23.660 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852890+0000", 2024-09-16T20:50:23.660 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7669999999999998e-06 2024-09-16T20:50:23.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.660 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:23.660 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869059+0000", 2024-09-16T20:50:23.660 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.016169353 2024-09-16T20:50:23.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.660 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:23.660 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869178+0000", 2024-09-16T20:50:23.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118193 2024-09-16T20:50:23.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.661 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:23.661 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869281+0000", 2024-09-16T20:50:23.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010310399999999999 2024-09-16T20:50:23.661 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.661 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:23.661 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.661 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5138.0:9382 3.c 3:33996a43:::rbd_data.110e6372565d.0000000000000007:head [read 0~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T20:50:23.661 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:50:23.661 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 55.769555216000001, 2024-09-16T20:50:23.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.017136034, 2024-09-16T20:50:23.662 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:23.662 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:23.662 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:23.662 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:23.662 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5138", 2024-09-16T20:50:23.662 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:50:23.662 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9382 2024-09-16T20:50:23.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.662 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:23.662 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.662 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:23.662 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:50:23.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.663 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:23.663 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:50:23.663 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.663 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:23.663 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852583+0000", 2024-09-16T20:50:23.663 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.51e-07 2024-09-16T20:50:23.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.663 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:23.663 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852586+0000", 2024-09-16T20:50:23.664 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0280000000000001e-06 2024-09-16T20:50:23.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.664 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.664 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:23.664 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852595+0000", 2024-09-16T20:50:23.664 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2799999999999992e-06 2024-09-16T20:50:23.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.664 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.664 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.664 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852601+0000", 2024-09-16T20:50:23.664 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3810000000000004e-06 2024-09-16T20:50:23.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.664 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.664 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:23.665 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852921+0000", 2024-09-16T20:50:23.665 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00031979699999999997 2024-09-16T20:50:23.665 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.665 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.665 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:23.665 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852930+0000", 2024-09-16T20:50:23.665 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6489999999999997e-06 2024-09-16T20:50:23.665 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.665 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.665 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:23.665 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869434+0000", 2024-09-16T20:50:23.665 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0165038829999999 2024-09-16T20:50:23.665 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.666 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:23.666 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869495+0000", 2024-09-16T20:50:23.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1509999999999999e-05 2024-09-16T20:50:23.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.666 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:23.666 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869718+0000", 2024-09-16T20:50:23.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000222955 2024-09-16T20:50:23.666 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.666 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:23.666 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.667 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9969 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 632832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:23.667 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642045+0000", 2024-09-16T20:50:23.667 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 32.980092759000001, 2024-09-16T20:50:23.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076903192999999995, 2024-09-16T20:50:23.667 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:23.667 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:23.667 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:23.667 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:23.667 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:23.667 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:23.667 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9969 2024-09-16T20:50:23.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.667 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:23.668 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.668 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:23.668 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642045+0000", 2024-09-16T20:50:23.668 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.668 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.668 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.668 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:23.668 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642045+0000", 2024-09-16T20:50:23.668 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.668 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.668 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.668 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:23.668 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642045+0000", 2024-09-16T20:50:23.668 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5100000000000001e-07 2024-09-16T20:50:23.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.669 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.669 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:23.669 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642047+0000", 2024-09-16T20:50:23.669 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8679999999999999e-06 2024-09-16T20:50:23.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.669 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.669 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:23.669 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642053+0000", 2024-09-16T20:50:23.669 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1720000000000004e-06 2024-09-16T20:50:23.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.669 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.670 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.670 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642060+0000", 2024-09-16T20:50:23.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0339999999999999e-06 2024-09-16T20:50:23.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.670 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:23.670 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.718562+0000", 2024-09-16T20:50:23.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076501969000000003 2024-09-16T20:50:23.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.670 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:23.670 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.718643+0000", 2024-09-16T20:50:23.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0539999999999998e-05 2024-09-16T20:50:23.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.671 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:23.671 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.718948+0000", 2024-09-16T20:50:23.671 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000305259 2024-09-16T20:50:23.671 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.671 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:23.671 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.671 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.671 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.671 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9970 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 637952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:23.671 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642067+0000", 2024-09-16T20:50:23.671 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 32.980070779000002, 2024-09-16T20:50:23.671 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077174928000000004, 2024-09-16T20:50:23.671 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:23.671 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:23.672 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:23.672 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:23.672 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:23.672 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:23.672 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9970 2024-09-16T20:50:23.672 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.672 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:23.672 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.672 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:23.672 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642067+0000", 2024-09-16T20:50:23.672 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.672 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.672 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.673 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:23.673 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642067+0000", 2024-09-16T20:50:23.673 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.673 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.673 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.673 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:23.673 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642067+0000", 2024-09-16T20:50:23.673 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2499999999999995e-07 2024-09-16T20:50:23.673 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.673 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.673 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:23.673 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642070+0000", 2024-09-16T20:50:23.673 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3199999999999998e-06 2024-09-16T20:50:23.673 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.674 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.674 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:23.674 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642076+0000", 2024-09-16T20:50:23.674 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.122e-06 2024-09-16T20:50:23.674 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.674 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.674 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.674 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642081+0000", 2024-09-16T20:50:23.674 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0869999999999997e-06 2024-09-16T20:50:23.674 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.674 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.674 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:23.674 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.718980+0000", 2024-09-16T20:50:23.674 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076899482000000005 2024-09-16T20:50:23.674 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.675 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.675 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:23.675 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719085+0000", 2024-09-16T20:50:23.675 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010446399999999999 2024-09-16T20:50:23.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.675 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.675 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:23.675 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719242+0000", 2024-09-16T20:50:23.675 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000156928 2024-09-16T20:50:23.675 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.675 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:23.675 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.676 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.676 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9971 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 643072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:23.676 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642102+0000", 2024-09-16T20:50:23.676 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 32.980035072, 2024-09-16T20:50:23.676 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077421016999999995, 2024-09-16T20:50:23.676 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:23.676 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:23.676 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:23.676 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:23.676 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:23.676 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:23.676 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9971 2024-09-16T20:50:23.676 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.676 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:23.677 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.677 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:23.677 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642102+0000", 2024-09-16T20:50:23.677 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.677 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.677 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.677 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:23.677 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642102+0000", 2024-09-16T20:50:23.677 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.677 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.677 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.677 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:23.677 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642103+0000", 2024-09-16T20:50:23.678 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7399999999999998e-07 2024-09-16T20:50:23.678 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.678 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.678 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:23.678 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642105+0000", 2024-09-16T20:50:23.678 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2649999999999999e-06 2024-09-16T20:50:23.678 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.678 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.678 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:23.678 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642112+0000", 2024-09-16T20:50:23.678 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.294e-06 2024-09-16T20:50:23.678 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.678 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.678 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.678 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642118+0000", 2024-09-16T20:50:23.679 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4500000000000003e-06 2024-09-16T20:50:23.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.679 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.679 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:23.679 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719285+0000", 2024-09-16T20:50:23.679 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077166883000000006 2024-09-16T20:50:23.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.679 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.679 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:23.679 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719368+0000", 2024-09-16T20:50:23.679 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2811000000000004e-05 2024-09-16T20:50:23.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.679 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.679 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:23.680 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719523+0000", 2024-09-16T20:50:23.680 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015584 2024-09-16T20:50:23.680 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.680 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:23.680 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.680 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.680 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9972 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 648192~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:23.680 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642125+0000", 2024-09-16T20:50:23.680 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 32.980012125000002, 2024-09-16T20:50:23.680 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077710324999999997, 2024-09-16T20:50:23.680 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:23.680 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:23.680 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:23.681 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:23.681 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:23.681 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:23.681 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9972 2024-09-16T20:50:23.681 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.681 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:23.681 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.681 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:23.681 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642125+0000", 2024-09-16T20:50:23.681 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.681 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.681 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.681 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:23.682 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642125+0000", 2024-09-16T20:50:23.682 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.682 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.682 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.682 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:23.682 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642126+0000", 2024-09-16T20:50:23.682 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1899999999999998e-07 2024-09-16T20:50:23.682 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.682 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.682 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:23.682 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642128+0000", 2024-09-16T20:50:23.682 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8619999999999999e-06 2024-09-16T20:50:23.682 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.682 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.683 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:23.683 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642134+0000", 2024-09-16T20:50:23.683 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5289999999999997e-06 2024-09-16T20:50:23.683 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.683 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.683 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.683 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642139+0000", 2024-09-16T20:50:23.683 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1250000000000001e-06 2024-09-16T20:50:23.683 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.683 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.683 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:23.683 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719570+0000", 2024-09-16T20:50:23.683 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077430516000000005 2024-09-16T20:50:23.683 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.684 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.684 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:23.684 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719655+0000", 2024-09-16T20:50:23.684 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5500000000000005e-05 2024-09-16T20:50:23.684 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.684 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.684 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:23.684 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719836+0000", 2024-09-16T20:50:23.684 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018037399999999999 2024-09-16T20:50:23.684 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.684 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:23.684 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.684 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.684 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.685 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9973 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 653312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:23.685 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:50:23.685 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 32.979991515000002, 2024-09-16T20:50:23.685 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077905262000000003, 2024-09-16T20:50:23.685 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:23.685 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:23.685 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:23.685 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:23.685 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:23.685 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:23.685 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9973 2024-09-16T20:50:23.685 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.685 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:23.685 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.686 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:23.686 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:50:23.686 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.686 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.686 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.686 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:23.686 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:50:23.686 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.686 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.686 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.686 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:23.686 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:50:23.686 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:50:23.686 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.687 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.687 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:23.687 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642148+0000", 2024-09-16T20:50:23.687 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0219999999999999e-06 2024-09-16T20:50:23.687 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.687 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.687 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:23.687 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642154+0000", 2024-09-16T20:50:23.687 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8540000000000002e-06 2024-09-16T20:50:23.687 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.687 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.687 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.687 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642159+0000", 2024-09-16T20:50:23.687 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.045e-06 2024-09-16T20:50:23.688 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.688 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.688 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:23.688 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719875+0000", 2024-09-16T20:50:23.688 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077716116000000002 2024-09-16T20:50:23.688 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.688 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.688 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:23.688 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719934+0000", 2024-09-16T20:50:23.688 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8819e-05 2024-09-16T20:50:23.688 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.688 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.688 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:23.689 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720051+0000", 2024-09-16T20:50:23.689 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000116947 2024-09-16T20:50:23.689 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.689 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:23.689 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.689 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.689 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.689 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9974 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 658432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:23.689 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642166+0000", 2024-09-16T20:50:23.689 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 32.979971601000003, 2024-09-16T20:50:23.689 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.078179786000000001, 2024-09-16T20:50:23.689 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:23.689 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:23.689 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:23.690 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:23.690 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:23.690 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:23.690 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9974 2024-09-16T20:50:23.690 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.690 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:23.690 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.690 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:23.690 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642166+0000", 2024-09-16T20:50:23.690 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.690 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.690 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.690 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:23.690 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642166+0000", 2024-09-16T20:50:23.691 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.691 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.691 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.691 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:23.691 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647343+0000", 2024-09-16T20:50:23.691 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0051769440000000002 2024-09-16T20:50:23.691 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.691 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.691 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:23.691 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647348+0000", 2024-09-16T20:50:23.691 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3619999999999997e-06 2024-09-16T20:50:23.691 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.691 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.691 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:23.691 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647359+0000", 2024-09-16T20:50:23.692 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0929e-05 2024-09-16T20:50:23.692 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.692 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.692 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.692 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647366+0000", 2024-09-16T20:50:23.692 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0910000000000004e-06 2024-09-16T20:50:23.692 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.692 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.692 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:23.692 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720094+0000", 2024-09-16T20:50:23.692 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.072727403999999995 2024-09-16T20:50:23.692 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.692 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.692 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:23.693 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720173+0000", 2024-09-16T20:50:23.693 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9382000000000007e-05 2024-09-16T20:50:23.693 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.693 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.693 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:23.693 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720346+0000", 2024-09-16T20:50:23.693 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017267399999999999 2024-09-16T20:50:23.693 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.693 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:23.693 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.693 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.693 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.693 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9988 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 730112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:23.693 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647703+0000", 2024-09-16T20:50:23.694 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 32.974434608999999, 2024-09-16T20:50:23.694 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076703473999999994, 2024-09-16T20:50:23.694 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:23.694 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:23.694 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:23.694 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:23.694 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:23.694 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:23.694 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9988 2024-09-16T20:50:23.694 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.694 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:23.694 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.694 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:23.694 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647703+0000", 2024-09-16T20:50:23.694 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.695 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.695 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.695 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:23.695 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647703+0000", 2024-09-16T20:50:23.695 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.695 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.695 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.695 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:23.695 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647703+0000", 2024-09-16T20:50:23.695 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0699999999999998e-07 2024-09-16T20:50:23.695 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.695 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.695 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:23.695 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647705+0000", 2024-09-16T20:50:23.696 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.866e-06 2024-09-16T20:50:23.696 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.696 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.696 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:23.696 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647708+0000", 2024-09-16T20:50:23.696 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3170000000000002e-06 2024-09-16T20:50:23.696 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.696 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.696 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.696 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647711+0000", 2024-09-16T20:50:23.696 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6759999999999999e-06 2024-09-16T20:50:23.696 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.696 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.697 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:23.697 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724225+0000", 2024-09-16T20:50:23.697 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.07651434 2024-09-16T20:50:23.697 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.697 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.697 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:23.697 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724291+0000", 2024-09-16T20:50:23.697 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5629000000000005e-05 2024-09-16T20:50:23.697 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.697 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.697 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:23.697 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724406+0000", 2024-09-16T20:50:23.697 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000115239 2024-09-16T20:50:23.697 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.698 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:23.698 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.698 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.698 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.698 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9989 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 735232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:23.698 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647715+0000", 2024-09-16T20:50:23.698 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 32.974422871000002, 2024-09-16T20:50:23.698 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076871693000000005, 2024-09-16T20:50:23.698 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:23.698 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:23.698 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:23.698 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:23.698 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:23.698 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:23.698 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9989 2024-09-16T20:50:23.699 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.699 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:23.699 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.699 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:23.699 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647715+0000", 2024-09-16T20:50:23.699 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.699 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.699 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.699 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:23.699 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647715+0000", 2024-09-16T20:50:23.699 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.699 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.699 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.699 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:23.699 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647715+0000", 2024-09-16T20:50:23.700 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1400000000000001e-07 2024-09-16T20:50:23.700 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.700 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.700 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:23.700 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647716+0000", 2024-09-16T20:50:23.700 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2240000000000001e-06 2024-09-16T20:50:23.700 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.700 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.700 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:23.700 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647719+0000", 2024-09-16T20:50:23.700 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0769999999999999e-06 2024-09-16T20:50:23.700 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.700 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.700 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.701 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647722+0000", 2024-09-16T20:50:23.701 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.65e-06 2024-09-16T20:50:23.701 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.701 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.701 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:23.701 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724424+0000", 2024-09-16T20:50:23.701 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076701982000000002 2024-09-16T20:50:23.701 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.701 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.701 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:23.701 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724470+0000", 2024-09-16T20:50:23.701 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6589999999999999e-05 2024-09-16T20:50:23.701 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.701 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.702 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:23.702 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724586+0000", 2024-09-16T20:50:23.702 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000115956 2024-09-16T20:50:23.702 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.702 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:23.702 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.702 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.702 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.702 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9990 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 740352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:23.702 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647725+0000", 2024-09-16T20:50:23.702 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 32.974412516999998, 2024-09-16T20:50:23.702 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077029919000000002, 2024-09-16T20:50:23.702 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:23.702 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:23.702 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:23.703 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:23.703 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:23.703 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:23.703 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9990 2024-09-16T20:50:23.703 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.703 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:23.703 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.703 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:23.703 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647725+0000", 2024-09-16T20:50:23.703 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.703 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.703 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.703 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:23.703 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647725+0000", 2024-09-16T20:50:23.704 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.704 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.704 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.704 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:23.704 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647725+0000", 2024-09-16T20:50:23.704 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2399999999999999e-07 2024-09-16T20:50:23.704 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.704 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.704 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:23.704 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647726+0000", 2024-09-16T20:50:23.704 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.116e-06 2024-09-16T20:50:23.704 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.704 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.705 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:23.705 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647729+0000", 2024-09-16T20:50:23.705 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.937e-06 2024-09-16T20:50:23.705 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.705 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.705 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.705 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647732+0000", 2024-09-16T20:50:23.705 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8739999999999999e-06 2024-09-16T20:50:23.705 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.705 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.705 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:23.705 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724612+0000", 2024-09-16T20:50:23.705 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076880102000000006 2024-09-16T20:50:23.705 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.706 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.706 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:23.706 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724649+0000", 2024-09-16T20:50:23.706 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6738e-05 2024-09-16T20:50:23.706 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.706 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.706 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:23.706 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724755+0000", 2024-09-16T20:50:23.706 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010592800000000001 2024-09-16T20:50:23.706 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.706 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:23.706 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.706 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.706 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.706 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9991 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 745472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:23.707 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:50:23.707 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 32.974402208000001, 2024-09-16T20:50:23.707 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077257820000000005, 2024-09-16T20:50:23.707 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:23.707 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:23.707 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:23.707 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:23.707 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:23.707 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:23.707 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9991 2024-09-16T20:50:23.707 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.707 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:23.707 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.707 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:23.707 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:50:23.708 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.708 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.708 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.708 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:23.708 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:50:23.708 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.708 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.708 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.708 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:23.708 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:50:23.708 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1199999999999999e-07 2024-09-16T20:50:23.708 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.708 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.708 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:23.708 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647736+0000", 2024-09-16T20:50:23.709 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3699999999999999e-07 2024-09-16T20:50:23.709 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.709 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.709 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:23.709 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647739+0000", 2024-09-16T20:50:23.709 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9359999999999999e-06 2024-09-16T20:50:23.709 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.709 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.709 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.709 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647742+0000", 2024-09-16T20:50:23.709 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5720000000000001e-06 2024-09-16T20:50:23.709 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.709 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.710 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:23.710 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724774+0000", 2024-09-16T20:50:23.710 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077032457999999998 2024-09-16T20:50:23.710 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.710 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.710 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:23.710 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724818+0000", 2024-09-16T20:50:23.710 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3803000000000001e-05 2024-09-16T20:50:23.710 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.710 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.710 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:23.710 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724993+0000", 2024-09-16T20:50:23.710 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000174902 2024-09-16T20:50:23.710 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.710 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:23.711 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.711 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.711 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.711 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9992 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 750592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:23.711 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:50:23.711 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 32.974392619, 2024-09-16T20:50:23.711 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077553227000000002, 2024-09-16T20:50:23.711 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:23.711 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:23.711 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:23.711 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:23.711 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:23.711 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:23.711 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9992 2024-09-16T20:50:23.711 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.712 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:23.712 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.712 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:23.712 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:50:23.712 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.712 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.712 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.712 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:23.712 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:50:23.712 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.712 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.712 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.712 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:23.712 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:50:23.713 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0200000000000001e-07 2024-09-16T20:50:23.713 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.713 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.713 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:23.713 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647746+0000", 2024-09-16T20:50:23.713 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.088e-06 2024-09-16T20:50:23.713 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.713 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.713 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:23.713 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647749+0000", 2024-09-16T20:50:23.714 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9170000000000002e-06 2024-09-16T20:50:23.714 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.714 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.714 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.714 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647752+0000", 2024-09-16T20:50:23.714 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.774e-06 2024-09-16T20:50:23.714 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.714 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.714 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:23.714 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725042+0000", 2024-09-16T20:50:23.714 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077289952999999995 2024-09-16T20:50:23.714 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.714 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.714 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:23.714 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725113+0000", 2024-09-16T20:50:23.715 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0920999999999994e-05 2024-09-16T20:50:23.715 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.715 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.715 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:23.715 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725298+0000", 2024-09-16T20:50:23.715 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000185372 2024-09-16T20:50:23.715 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.715 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:23.715 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.715 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.715 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.715 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9993 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 755712~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:23.715 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647755+0000", 2024-09-16T20:50:23.716 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 32.974382593000001, 2024-09-16T20:50:23.716 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077725674999999994, 2024-09-16T20:50:23.716 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:23.716 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:23.716 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:23.716 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:23.716 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:23.716 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:23.716 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9993 2024-09-16T20:50:23.716 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.716 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:23.716 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.716 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:23.716 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647755+0000", 2024-09-16T20:50:23.717 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.717 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.717 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.717 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:23.717 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647755+0000", 2024-09-16T20:50:23.717 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.717 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.717 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.717 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:23.717 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653406+0000", 2024-09-16T20:50:23.717 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.005650876 2024-09-16T20:50:23.717 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.717 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.717 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:23.718 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653411+0000", 2024-09-16T20:50:23.718 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8899999999999998e-06 2024-09-16T20:50:23.718 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.718 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.718 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:23.718 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653423+0000", 2024-09-16T20:50:23.718 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2447e-05 2024-09-16T20:50:23.718 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.718 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.718 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.718 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653431+0000", 2024-09-16T20:50:23.718 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5730000000000002e-06 2024-09-16T20:50:23.718 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.718 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.718 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:23.719 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725333+0000", 2024-09-16T20:50:23.719 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.071902743000000005 2024-09-16T20:50:23.719 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.719 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.719 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:23.719 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725385+0000", 2024-09-16T20:50:23.719 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1585000000000002e-05 2024-09-16T20:50:23.719 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.719 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.719 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:23.719 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725481+0000", 2024-09-16T20:50:23.719 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5561000000000002e-05 2024-09-16T20:50:23.719 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.719 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:23.720 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.720 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.720 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:23.720 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:23.721 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:50:23.722 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:23.719+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:23.813 INFO:teuthology.orchestra.run.smithi139.stdout:truncating image image_client.0-clone44 from 0xaec4a00 (overlap 0x100be00) to 0x4d0400 2024-09-16T20:50:23.824 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:23.821+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:23.846 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 3, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-16T20:50:23.846 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T20:50:23.847 INFO:tasks.thrashosds.thrasher:Removing osd 0, in_osds are: [0, 1, 2, 4, 3, 5] 2024-09-16T20:50:23.847 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 0 2024-09-16T20:50:23.894 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:23.894 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:50:23.894 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.894 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:4613 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:23.894 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.854140+0000", 2024-09-16T20:50:23.895 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.034317464999999998, 2024-09-16T20:50:23.895 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.034334759999999999, 2024-09-16T20:50:23.895 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:23.895 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:23.895 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:23.895 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:23.895 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:23.895 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:23.895 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4613 2024-09-16T20:50:23.895 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.895 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:23.895 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.895 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:23.895 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.854140+0000", 2024-09-16T20:50:23.896 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.896 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.896 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.896 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:23.896 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.854140+0000", 2024-09-16T20:50:23.896 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.896 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.896 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.896 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:23.896 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.854403+0000", 2024-09-16T20:50:23.896 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00026276400000000002 2024-09-16T20:50:23.896 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.896 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.896 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:23.897 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.854407+0000", 2024-09-16T20:50:23.897 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.69e-06 2024-09-16T20:50:23.897 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.897 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.897 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:23.897 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.854422+0000", 2024-09-16T20:50:23.897 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4524e-05 2024-09-16T20:50:23.897 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.897 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.897 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.897 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.854430+0000", 2024-09-16T20:50:23.897 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3219999999999993e-06 2024-09-16T20:50:23.897 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.898 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:23.898 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.898 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.898 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.898 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5062 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:23.898 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.875200+0000", 2024-09-16T20:50:23.898 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.01325755, 2024-09-16T20:50:23.898 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.013331946000000001, 2024-09-16T20:50:23.898 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:23.898 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:23.898 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:23.898 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:23.898 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:23.898 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:23.898 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5062 2024-09-16T20:50:23.898 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.899 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:23.899 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.899 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:23.899 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.875200+0000", 2024-09-16T20:50:23.899 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.899 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.899 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.899 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:23.899 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.875200+0000", 2024-09-16T20:50:23.899 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.899 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.899 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.899 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:23.899 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.875200+0000", 2024-09-16T20:50:23.899 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.51e-07 2024-09-16T20:50:23.900 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.900 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.900 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:23.900 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.875202+0000", 2024-09-16T20:50:23.900 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1500000000000002e-06 2024-09-16T20:50:23.900 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.900 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.900 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:23.900 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.875212+0000", 2024-09-16T20:50:23.900 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5319999999999995e-06 2024-09-16T20:50:23.900 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.900 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.900 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.900 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.875218+0000", 2024-09-16T20:50:23.901 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5150000000000003e-06 2024-09-16T20:50:23.901 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.901 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:23.901 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.901 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.901 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.901 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5254 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:23.901 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.886944+0000", 2024-09-16T20:50:23.901 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.001512907, 2024-09-16T20:50:23.901 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016380120000000001, 2024-09-16T20:50:23.901 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:23.901 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:23.901 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:23.901 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:23.902 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:23.902 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:23.902 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5254 2024-09-16T20:50:23.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.902 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:23.902 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.902 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:23.902 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.886944+0000", 2024-09-16T20:50:23.902 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.902 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.902 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:23.902 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.886944+0000", 2024-09-16T20:50:23.902 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.903 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.903 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:23.903 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.886945+0000", 2024-09-16T20:50:23.903 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5300000000000003e-07 2024-09-16T20:50:23.903 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.903 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.903 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:23.903 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.886949+0000", 2024-09-16T20:50:23.903 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.856e-06 2024-09-16T20:50:23.903 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.903 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.903 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:23.903 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.886962+0000", 2024-09-16T20:50:23.904 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2320000000000001e-05 2024-09-16T20:50:23.904 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.904 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.904 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.904 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.886969+0000", 2024-09-16T20:50:23.904 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4390000000000003e-06 2024-09-16T20:50:23.904 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.904 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:23.904 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.904 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.904 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.904 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:4614 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:23.904 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.854446+0000", 2024-09-16T20:50:23.904 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.034011514999999999, 2024-09-16T20:50:23.904 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.034164089000000002, 2024-09-16T20:50:23.905 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:23.905 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:23.905 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:23.905 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:23.905 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:23.905 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:23.905 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4614 2024-09-16T20:50:23.905 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.905 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:23.905 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.905 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:23.905 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.854446+0000", 2024-09-16T20:50:23.905 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.905 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.905 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.906 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:23.906 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.854446+0000", 2024-09-16T20:50:23.906 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.906 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.906 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:23.906 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.854446+0000", 2024-09-16T20:50:23.906 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6700000000000003e-07 2024-09-16T20:50:23.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.906 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.906 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:23.906 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.854449+0000", 2024-09-16T20:50:23.906 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7609999999999999e-06 2024-09-16T20:50:23.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.906 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.907 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:23.907 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.854457+0000", 2024-09-16T20:50:23.907 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9230000000000002e-06 2024-09-16T20:50:23.907 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.907 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.907 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.907 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.854463+0000", 2024-09-16T20:50:23.907 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0560000000000001e-06 2024-09-16T20:50:23.907 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.907 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:23.907 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.907 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.907 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.907 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5063 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:23.908 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.875286+0000", 2024-09-16T20:50:23.908 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.01317172, 2024-09-16T20:50:23.908 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.013347615, 2024-09-16T20:50:23.908 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:23.908 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:23.908 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:23.908 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:23.908 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:23.908 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:23.908 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5063 2024-09-16T20:50:23.908 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.908 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:23.908 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.908 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:23.908 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.875286+0000", 2024-09-16T20:50:23.909 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.909 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.909 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.909 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:23.909 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.875286+0000", 2024-09-16T20:50:23.909 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.909 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.909 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.909 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:23.909 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.875286+0000", 2024-09-16T20:50:23.909 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7100000000000001e-07 2024-09-16T20:50:23.909 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.909 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.909 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:23.910 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.875290+0000", 2024-09-16T20:50:23.910 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4460000000000001e-06 2024-09-16T20:50:23.910 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.910 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.910 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:23.910 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.875300+0000", 2024-09-16T20:50:23.910 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0225e-05 2024-09-16T20:50:23.910 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.910 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.910 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.910 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.875306+0000", 2024-09-16T20:50:23.910 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.049e-06 2024-09-16T20:50:23.910 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.910 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:23.910 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.911 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.911 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.911 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5255 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:23.911 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.886981+0000", 2024-09-16T20:50:23.911 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0014766530000000001, 2024-09-16T20:50:23.911 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016971849999999999, 2024-09-16T20:50:23.911 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:23.911 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:23.911 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:23.911 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:23.911 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:23.911 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:23.911 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5255 2024-09-16T20:50:23.911 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.911 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:23.912 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.912 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:23.912 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.886981+0000", 2024-09-16T20:50:23.912 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.912 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.912 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.912 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:23.912 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.886981+0000", 2024-09-16T20:50:23.912 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.912 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.912 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.912 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:23.912 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.886981+0000", 2024-09-16T20:50:23.912 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5300000000000004e-07 2024-09-16T20:50:23.912 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.913 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.913 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:23.913 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.886984+0000", 2024-09-16T20:50:23.913 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.655e-06 2024-09-16T20:50:23.913 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.913 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.913 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:23.913 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887001+0000", 2024-09-16T20:50:23.913 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6585000000000001e-05 2024-09-16T20:50:23.913 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.913 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.913 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.913 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887007+0000", 2024-09-16T20:50:23.913 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3369999999999996e-06 2024-09-16T20:50:23.913 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.914 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:23.914 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.914 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.914 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.914 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:4616 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:23.914 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.854472+0000", 2024-09-16T20:50:23.914 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.033985334999999998, 2024-09-16T20:50:23.914 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.034239428000000002, 2024-09-16T20:50:23.914 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:23.914 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:23.914 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:23.914 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:23.914 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:23.914 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:23.914 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4616 2024-09-16T20:50:23.915 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.915 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:23.915 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.915 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:23.915 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.854472+0000", 2024-09-16T20:50:23.915 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.915 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.915 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.915 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:23.915 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.854472+0000", 2024-09-16T20:50:23.915 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.915 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.915 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.915 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:23.915 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.854473+0000", 2024-09-16T20:50:23.916 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7800000000000002e-07 2024-09-16T20:50:23.916 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.916 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.916 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:23.916 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.854475+0000", 2024-09-16T20:50:23.916 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2759999999999999e-06 2024-09-16T20:50:23.916 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.916 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.916 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:23.916 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.854482+0000", 2024-09-16T20:50:23.916 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.922e-06 2024-09-16T20:50:23.916 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.916 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.916 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.916 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.854487+0000", 2024-09-16T20:50:23.917 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4210000000000003e-06 2024-09-16T20:50:23.917 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.917 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:23.917 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.917 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.917 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.917 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5064 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:23.917 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.875421+0000", 2024-09-16T20:50:23.917 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.013036514000000001, 2024-09-16T20:50:23.917 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.013318284999999999, 2024-09-16T20:50:23.917 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:23.917 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:23.917 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:23.917 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:23.917 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:23.918 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:23.918 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5064 2024-09-16T20:50:23.918 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.918 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:23.918 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.918 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:23.918 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.875421+0000", 2024-09-16T20:50:23.918 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.918 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.918 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.918 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:23.918 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.875421+0000", 2024-09-16T20:50:23.918 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.918 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.918 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.919 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:23.919 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.875422+0000", 2024-09-16T20:50:23.919 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3399999999999998e-07 2024-09-16T20:50:23.919 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.919 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.919 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:23.919 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.875425+0000", 2024-09-16T20:50:23.919 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7639999999999999e-06 2024-09-16T20:50:23.919 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.919 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.919 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:23.919 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.875436+0000", 2024-09-16T20:50:23.919 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0251000000000001e-05 2024-09-16T20:50:23.920 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.920 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.920 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.920 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.875442+0000", 2024-09-16T20:50:23.920 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4690000000000002e-06 2024-09-16T20:50:23.920 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.920 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:23.920 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.920 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.920 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.920 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5256 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:23.920 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.887016+0000", 2024-09-16T20:50:23.920 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0014418499999999999, 2024-09-16T20:50:23.920 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001762949, 2024-09-16T20:50:23.921 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:23.921 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:23.921 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:23.921 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:23.921 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:23.921 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:23.921 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5256 2024-09-16T20:50:23.921 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.921 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:23.921 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.921 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:23.921 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887016+0000", 2024-09-16T20:50:23.921 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.921 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.921 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.922 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:23.922 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887016+0000", 2024-09-16T20:50:23.922 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.922 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.922 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:23.922 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887016+0000", 2024-09-16T20:50:23.922 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0500000000000003e-07 2024-09-16T20:50:23.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.922 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.922 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:23.922 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887019+0000", 2024-09-16T20:50:23.922 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.57e-06 2024-09-16T20:50:23.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.923 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.923 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:23.923 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887026+0000", 2024-09-16T20:50:23.923 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3300000000000001e-06 2024-09-16T20:50:23.923 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.923 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.923 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.923 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887031+0000", 2024-09-16T20:50:23.923 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4229999999999996e-06 2024-09-16T20:50:23.923 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.923 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:23.923 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.923 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.923 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.924 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:4618 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:23.924 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.854495+0000", 2024-09-16T20:50:23.924 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.033962340000000001, 2024-09-16T20:50:23.924 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.034309056999999997, 2024-09-16T20:50:23.924 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:23.924 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:23.924 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:23.924 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:23.924 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:23.924 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:23.924 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4618 2024-09-16T20:50:23.924 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.924 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:23.924 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.924 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:23.925 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.854495+0000", 2024-09-16T20:50:23.925 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.925 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.925 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.925 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:23.925 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.854495+0000", 2024-09-16T20:50:23.925 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.925 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.925 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.925 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:23.925 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860379+0000", 2024-09-16T20:50:23.925 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0058839879999999997 2024-09-16T20:50:23.925 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.925 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.925 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:23.926 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860385+0000", 2024-09-16T20:50:23.926 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5550000000000002e-06 2024-09-16T20:50:23.926 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.926 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.926 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:23.926 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860397+0000", 2024-09-16T20:50:23.926 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.269e-05 2024-09-16T20:50:23.926 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.926 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.926 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.926 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860407+0000", 2024-09-16T20:50:23.926 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3500000000000003e-06 2024-09-16T20:50:23.926 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.926 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:23.926 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.927 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.927 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.927 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5065 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:23.927 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.875453+0000", 2024-09-16T20:50:23.927 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.013004104000000001, 2024-09-16T20:50:23.927 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.013373576, 2024-09-16T20:50:23.927 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:23.927 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:23.927 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:23.927 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:23.927 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:23.927 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:23.927 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5065 2024-09-16T20:50:23.927 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.927 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:23.928 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.928 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:23.928 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.875453+0000", 2024-09-16T20:50:23.928 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.928 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.928 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.928 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:23.928 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.875453+0000", 2024-09-16T20:50:23.928 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.928 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.928 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.928 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:23.928 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.875454+0000", 2024-09-16T20:50:23.928 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3799999999999998e-07 2024-09-16T20:50:23.929 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.929 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.929 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:23.929 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.875456+0000", 2024-09-16T20:50:23.929 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3300000000000001e-06 2024-09-16T20:50:23.929 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.929 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.929 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:23.929 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.875463+0000", 2024-09-16T20:50:23.929 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9160000000000004e-06 2024-09-16T20:50:23.929 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.929 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.929 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.929 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.875468+0000", 2024-09-16T20:50:23.929 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3159999999999996e-06 2024-09-16T20:50:23.929 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.930 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:23.930 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.930 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.930 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5257 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:23.930 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.887039+0000", 2024-09-16T20:50:23.930 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.001418758, 2024-09-16T20:50:23.930 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018114590000000001, 2024-09-16T20:50:23.930 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:23.930 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:23.930 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:23.930 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:23.930 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:23.930 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:23.931 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5257 2024-09-16T20:50:23.931 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.931 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:23.931 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.931 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:23.931 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887039+0000", 2024-09-16T20:50:23.931 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.931 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.931 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.932 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:23.932 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887039+0000", 2024-09-16T20:50:23.932 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.932 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.932 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.932 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:23.932 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887039+0000", 2024-09-16T20:50:23.932 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1799999999999995e-07 2024-09-16T20:50:23.933 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.933 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.933 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:23.933 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887041+0000", 2024-09-16T20:50:23.933 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0660000000000002e-06 2024-09-16T20:50:23.933 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.933 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.933 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:23.933 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887048+0000", 2024-09-16T20:50:23.933 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.81e-06 2024-09-16T20:50:23.933 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.933 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.933 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.933 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887053+0000", 2024-09-16T20:50:23.934 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2800000000000003e-06 2024-09-16T20:50:23.934 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.934 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:23.934 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.934 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.934 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.934 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:4623 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:23.936 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.860420+0000", 2024-09-16T20:50:23.936 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.028036978000000001, 2024-09-16T20:50:23.936 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.028473376000000002, 2024-09-16T20:50:23.936 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:23.936 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:23.936 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:23.936 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:23.936 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:23.936 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:23.936 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4623 2024-09-16T20:50:23.936 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.936 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:23.936 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.937 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:23.937 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860420+0000", 2024-09-16T20:50:23.937 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.937 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.937 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.937 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:23.937 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860420+0000", 2024-09-16T20:50:23.937 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.937 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.937 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.937 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:23.937 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860421+0000", 2024-09-16T20:50:23.937 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4000000000000001e-07 2024-09-16T20:50:23.937 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.937 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.937 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:23.938 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860424+0000", 2024-09-16T20:50:23.938 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9399999999999998e-06 2024-09-16T20:50:23.938 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.938 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.938 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:23.938 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860432+0000", 2024-09-16T20:50:23.938 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2020000000000004e-06 2024-09-16T20:50:23.938 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.938 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.938 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.938 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860439+0000", 2024-09-16T20:50:23.938 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3840000000000002e-06 2024-09-16T20:50:23.938 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.938 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:23.939 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.939 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5066 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:23.939 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.875478+0000", 2024-09-16T20:50:23.939 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.012979707, 2024-09-16T20:50:23.939 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.013449205000000001, 2024-09-16T20:50:23.939 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:23.939 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:23.939 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:23.939 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:23.939 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:23.939 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:23.939 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5066 2024-09-16T20:50:23.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.940 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:23.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:23.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.875478+0000", 2024-09-16T20:50:23.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:23.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.875478+0000", 2024-09-16T20:50:23.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:23.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.875478+0000", 2024-09-16T20:50:23.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9500000000000003e-07 2024-09-16T20:50:23.941 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.941 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.941 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:23.941 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.875480+0000", 2024-09-16T20:50:23.941 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2699999999999999e-06 2024-09-16T20:50:23.941 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.941 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.941 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:23.941 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.875487+0000", 2024-09-16T20:50:23.941 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.72e-06 2024-09-16T20:50:23.941 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.941 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.941 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.941 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.875492+0000", 2024-09-16T20:50:23.941 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2249999999999999e-06 2024-09-16T20:50:23.942 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.942 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:23.942 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.942 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.942 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5258 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:23.942 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.887064+0000", 2024-09-16T20:50:23.942 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0013936689999999999, 2024-09-16T20:50:23.942 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001887838, 2024-09-16T20:50:23.942 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:23.942 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:23.942 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:23.942 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:23.942 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:23.942 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:23.943 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5258 2024-09-16T20:50:23.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.943 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:23.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.943 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:23.943 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887064+0000", 2024-09-16T20:50:23.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.943 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:23.943 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887064+0000", 2024-09-16T20:50:23.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.943 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:23.944 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887064+0000", 2024-09-16T20:50:23.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6400000000000003e-07 2024-09-16T20:50:23.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.944 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:23.944 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887066+0000", 2024-09-16T20:50:23.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2000000000000001e-06 2024-09-16T20:50:23.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.944 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:23.944 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887073+0000", 2024-09-16T20:50:23.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3949999999999998e-06 2024-09-16T20:50:23.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.945 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.945 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887078+0000", 2024-09-16T20:50:23.945 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0880000000000002e-06 2024-09-16T20:50:23.945 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.945 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:23.945 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.945 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:4625 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:23.945 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.860448+0000", 2024-09-16T20:50:23.945 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.028009769, 2024-09-16T20:50:23.945 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.028549241, 2024-09-16T20:50:23.946 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:23.946 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:23.946 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:23.946 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:23.946 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:23.946 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:23.946 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4625 2024-09-16T20:50:23.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.946 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:23.946 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.946 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:23.946 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860448+0000", 2024-09-16T20:50:23.946 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.947 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.947 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:23.947 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860448+0000", 2024-09-16T20:50:23.947 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.947 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.947 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:23.947 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860448+0000", 2024-09-16T20:50:23.947 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0999999999999999e-07 2024-09-16T20:50:23.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.947 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.947 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:23.947 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860451+0000", 2024-09-16T20:50:23.947 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4260000000000002e-06 2024-09-16T20:50:23.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.948 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:23.948 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860458+0000", 2024-09-16T20:50:23.948 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9249999999999995e-06 2024-09-16T20:50:23.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.948 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.948 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860464+0000", 2024-09-16T20:50:23.948 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7010000000000001e-06 2024-09-16T20:50:23.948 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.948 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:23.948 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.949 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5073 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:23.949 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.875807+0000", 2024-09-16T20:50:23.949 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.012650730000000001, 2024-09-16T20:50:23.949 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.013219639, 2024-09-16T20:50:23.949 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:23.949 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:23.949 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:23.949 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:23.949 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:23.949 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:23.949 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5073 2024-09-16T20:50:23.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.949 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:23.950 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.950 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:23.950 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.875807+0000", 2024-09-16T20:50:23.950 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.950 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.950 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:23.950 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.875807+0000", 2024-09-16T20:50:23.950 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.950 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.950 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:23.950 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.875808+0000", 2024-09-16T20:50:23.950 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6099999999999999e-07 2024-09-16T20:50:23.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.951 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.951 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:23.951 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.875811+0000", 2024-09-16T20:50:23.951 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8909999999999997e-06 2024-09-16T20:50:23.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.951 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.951 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:23.951 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.875822+0000", 2024-09-16T20:50:23.951 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0702e-05 2024-09-16T20:50:23.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.951 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.951 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.951 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.875829+0000", 2024-09-16T20:50:23.951 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9460000000000001e-06 2024-09-16T20:50:23.951 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.951 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:23.952 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.952 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.952 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5259 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:23.952 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.887202+0000", 2024-09-16T20:50:23.952 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0012550910000000001, 2024-09-16T20:50:23.952 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001855338, 2024-09-16T20:50:23.952 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:23.952 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:23.952 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:23.952 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:23.952 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:23.952 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:23.952 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5259 2024-09-16T20:50:23.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.953 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:23.953 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.953 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:23.953 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887202+0000", 2024-09-16T20:50:23.953 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.953 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.953 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:23.953 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887202+0000", 2024-09-16T20:50:23.953 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.953 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.953 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:23.954 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887203+0000", 2024-09-16T20:50:23.954 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5099999999999998e-07 2024-09-16T20:50:23.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.954 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.954 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:23.954 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887207+0000", 2024-09-16T20:50:23.954 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.614e-06 2024-09-16T20:50:23.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.954 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.954 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:23.954 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887217+0000", 2024-09-16T20:50:23.954 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0491e-05 2024-09-16T20:50:23.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.954 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.954 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.955 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887224+0000", 2024-09-16T20:50:23.955 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4350000000000002e-06 2024-09-16T20:50:23.955 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.955 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:23.955 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.955 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:4626 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:23.955 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.860472+0000", 2024-09-16T20:50:23.955 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.027985652999999999, 2024-09-16T20:50:23.955 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.028632642999999999, 2024-09-16T20:50:23.955 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:23.955 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:23.955 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:23.956 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:23.956 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:23.956 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:23.956 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4626 2024-09-16T20:50:23.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.956 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:23.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.956 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:23.956 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860472+0000", 2024-09-16T20:50:23.956 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.956 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:23.956 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860472+0000", 2024-09-16T20:50:23.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.957 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:23.957 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860472+0000", 2024-09-16T20:50:23.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9499999999999998e-07 2024-09-16T20:50:23.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.957 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:23.957 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860474+0000", 2024-09-16T20:50:23.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3099999999999999e-06 2024-09-16T20:50:23.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.957 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:23.958 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860482+0000", 2024-09-16T20:50:23.958 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5290000000000003e-06 2024-09-16T20:50:23.958 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.958 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.958 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.958 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860494+0000", 2024-09-16T20:50:23.958 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2368000000000001e-05 2024-09-16T20:50:23.958 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.958 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:23.958 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.958 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.958 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.958 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5074 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:23.958 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.875935+0000", 2024-09-16T20:50:23.959 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.012521991, 2024-09-16T20:50:23.959 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.013205138999999999, 2024-09-16T20:50:23.959 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:23.959 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:23.959 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:23.959 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:23.959 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:23.959 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:23.959 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5074 2024-09-16T20:50:23.959 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.959 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:23.959 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.959 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:23.959 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.875935+0000", 2024-09-16T20:50:23.960 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.960 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:23.960 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.875935+0000", 2024-09-16T20:50:23.960 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.960 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:23.960 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.875936+0000", 2024-09-16T20:50:23.960 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7499999999999999e-07 2024-09-16T20:50:23.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.960 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:23.960 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.875940+0000", 2024-09-16T20:50:23.961 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9600000000000002e-06 2024-09-16T20:50:23.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.961 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.961 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:23.961 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.875952+0000", 2024-09-16T20:50:23.961 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1627000000000001e-05 2024-09-16T20:50:23.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.961 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.961 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.961 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.875958+0000", 2024-09-16T20:50:23.961 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5910000000000001e-06 2024-09-16T20:50:23.961 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.961 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:23.961 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.962 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5260 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:23.962 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.887243+0000", 2024-09-16T20:50:23.962 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.001214204, 2024-09-16T20:50:23.962 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001933401, 2024-09-16T20:50:23.962 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:23.962 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:23.962 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:23.962 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:23.962 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:23.962 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:23.962 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5260 2024-09-16T20:50:23.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.963 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:23.963 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.963 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:23.963 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887243+0000", 2024-09-16T20:50:23.963 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.963 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.963 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:23.963 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887243+0000", 2024-09-16T20:50:23.963 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.963 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.963 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:23.963 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887244+0000", 2024-09-16T20:50:23.964 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8100000000000003e-07 2024-09-16T20:50:23.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.964 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.964 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:23.964 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887247+0000", 2024-09-16T20:50:23.964 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.129e-06 2024-09-16T20:50:23.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.964 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.964 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:23.964 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887256+0000", 2024-09-16T20:50:23.964 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4690000000000003e-06 2024-09-16T20:50:23.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.964 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.964 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.964 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887262+0000", 2024-09-16T20:50:23.965 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7139999999999998e-06 2024-09-16T20:50:23.965 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.965 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:23.965 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.965 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.965 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:4627 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:23.965 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.860511+0000", 2024-09-16T20:50:23.965 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.027946019999999999, 2024-09-16T20:50:23.965 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.028701137000000002, 2024-09-16T20:50:23.965 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:23.965 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:23.965 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:23.965 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:23.966 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:23.966 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:23.966 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4627 2024-09-16T20:50:23.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.966 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:23.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.966 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:23.966 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860511+0000", 2024-09-16T20:50:23.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.966 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:23.966 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860511+0000", 2024-09-16T20:50:23.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.967 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:23.967 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860512+0000", 2024-09-16T20:50:23.967 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9400000000000005e-07 2024-09-16T20:50:23.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.967 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:23.967 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860514+0000", 2024-09-16T20:50:23.967 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3159999999999999e-06 2024-09-16T20:50:23.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.967 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:23.967 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860521+0000", 2024-09-16T20:50:23.967 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0829999999999998e-06 2024-09-16T20:50:23.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.968 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.968 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.968 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860527+0000", 2024-09-16T20:50:23.968 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.327e-06 2024-09-16T20:50:23.968 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.968 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:23.968 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.968 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.968 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.968 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5075 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:23.968 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.875969+0000", 2024-09-16T20:50:23.968 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.012488681, 2024-09-16T20:50:23.968 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.013268558999999999, 2024-09-16T20:50:23.968 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:23.969 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:23.969 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:23.969 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:23.969 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:23.969 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:23.969 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5075 2024-09-16T20:50:23.969 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.969 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:23.969 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.969 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:23.969 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.875969+0000", 2024-09-16T20:50:23.969 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.969 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.969 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.970 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:23.970 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.875969+0000", 2024-09-16T20:50:23.970 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.970 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.970 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:23.970 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.875969+0000", 2024-09-16T20:50:23.970 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8100000000000003e-07 2024-09-16T20:50:23.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.970 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.970 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:23.970 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.875971+0000", 2024-09-16T20:50:23.970 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1500000000000002e-06 2024-09-16T20:50:23.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.971 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.971 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:23.971 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.875978+0000", 2024-09-16T20:50:23.971 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5200000000000003e-06 2024-09-16T20:50:23.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.971 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.971 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.971 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.875983+0000", 2024-09-16T20:50:23.971 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1919999999999996e-06 2024-09-16T20:50:23.971 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.971 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:23.971 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.971 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.972 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5262 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:23.972 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.887271+0000", 2024-09-16T20:50:23.972 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0011863570000000001, 2024-09-16T20:50:23.972 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0019888309999999999, 2024-09-16T20:50:23.972 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:23.972 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:23.972 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:23.972 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:23.972 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:23.972 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:23.972 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5262 2024-09-16T20:50:23.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.972 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:23.973 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.973 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:23.973 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887271+0000", 2024-09-16T20:50:23.973 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.973 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.973 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:23.973 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887271+0000", 2024-09-16T20:50:23.973 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.973 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.973 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:23.973 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887272+0000", 2024-09-16T20:50:23.973 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0399999999999996e-07 2024-09-16T20:50:23.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.974 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:23.974 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887274+0000", 2024-09-16T20:50:23.974 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3369999999999998e-06 2024-09-16T20:50:23.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.974 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:23.974 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887281+0000", 2024-09-16T20:50:23.974 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1289999999999998e-06 2024-09-16T20:50:23.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.974 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.974 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887287+0000", 2024-09-16T20:50:23.974 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2049999999999999e-06 2024-09-16T20:50:23.975 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.975 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:23.975 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.975 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.975 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:4628 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:23.975 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.860534+0000", 2024-09-16T20:50:23.975 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.027923147999999998, 2024-09-16T20:50:23.975 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.028762251999999999, 2024-09-16T20:50:23.975 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:23.975 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:23.975 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:23.975 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:23.975 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:23.976 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:23.976 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4628 2024-09-16T20:50:23.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.976 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:23.976 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.976 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:23.976 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860534+0000", 2024-09-16T20:50:23.976 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.976 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.976 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:23.976 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860534+0000", 2024-09-16T20:50:23.976 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.977 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:23.977 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860535+0000", 2024-09-16T20:50:23.977 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9100000000000004e-07 2024-09-16T20:50:23.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.977 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:23.977 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860537+0000", 2024-09-16T20:50:23.977 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.984e-06 2024-09-16T20:50:23.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.977 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:23.977 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860543+0000", 2024-09-16T20:50:23.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0889999999999996e-06 2024-09-16T20:50:23.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.978 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.978 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.978 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860550+0000", 2024-09-16T20:50:23.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.046e-06 2024-09-16T20:50:23.978 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.978 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:23.978 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.978 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.978 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5076 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:23.978 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.876065+0000", 2024-09-16T20:50:23.978 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.012392261999999999, 2024-09-16T20:50:23.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.013285214, 2024-09-16T20:50:23.979 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:23.979 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:23.979 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:23.979 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:23.979 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:23.979 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:23.979 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5076 2024-09-16T20:50:23.979 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.979 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:23.979 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.979 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:23.979 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.876065+0000", 2024-09-16T20:50:23.979 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.979 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.980 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.980 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:23.980 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.876065+0000", 2024-09-16T20:50:23.980 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.980 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.980 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:23.980 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.876066+0000", 2024-09-16T20:50:23.980 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2399999999999997e-07 2024-09-16T20:50:23.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.980 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.980 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:23.980 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.876069+0000", 2024-09-16T20:50:23.980 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5719999999999999e-06 2024-09-16T20:50:23.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.981 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.981 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:23.981 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.876079+0000", 2024-09-16T20:50:23.981 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4760000000000004e-06 2024-09-16T20:50:23.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.981 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.981 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.981 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.876085+0000", 2024-09-16T20:50:23.981 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2530000000000001e-06 2024-09-16T20:50:23.981 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.981 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:23.981 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.982 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.982 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5264 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:23.982 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.887342+0000", 2024-09-16T20:50:23.982 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.001115188, 2024-09-16T20:50:23.982 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0020482289999999999, 2024-09-16T20:50:23.982 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:23.982 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:23.982 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:23.982 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:23.982 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:23.982 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:23.982 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5264 2024-09-16T20:50:23.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.982 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:23.983 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.983 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:23.983 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887342+0000", 2024-09-16T20:50:23.983 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.983 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.983 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:23.983 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887342+0000", 2024-09-16T20:50:23.983 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.983 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.983 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:23.983 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887343+0000", 2024-09-16T20:50:23.983 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.61e-07 2024-09-16T20:50:23.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.984 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.984 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:23.984 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887346+0000", 2024-09-16T20:50:23.984 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1999999999999999e-06 2024-09-16T20:50:23.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.984 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.984 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:23.984 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887355+0000", 2024-09-16T20:50:23.984 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.585e-06 2024-09-16T20:50:23.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.984 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.984 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.985 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887361+0000", 2024-09-16T20:50:23.985 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4849999999999997e-06 2024-09-16T20:50:23.985 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.985 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:23.985 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.985 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.985 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:4629 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:23.985 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.860556+0000", 2024-09-16T20:50:23.985 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.027900997, 2024-09-16T20:50:23.985 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.028858623999999999, 2024-09-16T20:50:23.985 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:23.985 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:23.985 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:23.986 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:23.986 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:23.986 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:23.986 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4629 2024-09-16T20:50:23.986 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.986 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:23.986 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.986 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:23.986 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860556+0000", 2024-09-16T20:50:23.986 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.986 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.986 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.986 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:23.986 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860556+0000", 2024-09-16T20:50:23.986 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.987 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.987 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.987 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:23.987 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860557+0000", 2024-09-16T20:50:23.987 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8700000000000001e-07 2024-09-16T20:50:23.987 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.987 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.987 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:23.987 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860559+0000", 2024-09-16T20:50:23.987 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8700000000000001e-06 2024-09-16T20:50:23.987 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.987 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.987 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:23.988 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860565+0000", 2024-09-16T20:50:23.988 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.37e-06 2024-09-16T20:50:23.988 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.988 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.988 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.988 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860570+0000", 2024-09-16T20:50:23.988 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0699999999999997e-06 2024-09-16T20:50:23.988 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.988 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:23.988 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.988 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.988 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.988 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5077 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:23.989 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.876095+0000", 2024-09-16T20:50:23.989 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.012362493, 2024-09-16T20:50:23.989 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.013349366999999999, 2024-09-16T20:50:23.989 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:23.989 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:23.989 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:23.989 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:23.989 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:23.989 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:23.989 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5077 2024-09-16T20:50:23.989 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.989 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:23.989 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.990 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:23.990 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.876095+0000", 2024-09-16T20:50:23.990 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.990 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.990 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.990 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:23.990 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.876095+0000", 2024-09-16T20:50:23.990 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.990 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.990 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.990 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:23.990 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.876095+0000", 2024-09-16T20:50:23.990 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.84e-07 2024-09-16T20:50:23.990 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.991 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.991 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:23.991 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.876097+0000", 2024-09-16T20:50:23.991 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.092e-06 2024-09-16T20:50:23.991 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.991 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.991 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:23.991 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.876110+0000", 2024-09-16T20:50:23.991 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2738000000000001e-05 2024-09-16T20:50:23.991 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.991 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.991 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.991 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.876116+0000", 2024-09-16T20:50:23.991 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5929999999999996e-06 2024-09-16T20:50:23.992 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.992 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:23.992 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.992 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.992 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.992 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5266 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:23.992 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.887416+0000", 2024-09-16T20:50:23.992 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.001041766, 2024-09-16T20:50:23.992 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0020732210000000001, 2024-09-16T20:50:23.992 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:23.992 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:23.992 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:23.992 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:23.992 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:23.993 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:23.993 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5266 2024-09-16T20:50:23.993 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.993 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:23.993 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.993 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:23.993 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887416+0000", 2024-09-16T20:50:23.993 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.993 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.993 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.993 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:23.993 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887416+0000", 2024-09-16T20:50:23.993 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.994 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.994 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.994 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:23.994 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887416+0000", 2024-09-16T20:50:23.994 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4900000000000005e-07 2024-09-16T20:50:23.994 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.994 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.994 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:23.994 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887420+0000", 2024-09-16T20:50:23.994 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6200000000000001e-06 2024-09-16T20:50:23.994 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.994 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.994 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:23.994 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887429+0000", 2024-09-16T20:50:23.995 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4879999999999996e-06 2024-09-16T20:50:23.995 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.995 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.995 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.995 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887436+0000", 2024-09-16T20:50:23.995 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4849999999999997e-06 2024-09-16T20:50:23.995 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.995 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:23.995 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.995 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.995 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.995 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:4630 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:23.995 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.860576+0000", 2024-09-16T20:50:23.995 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.027881152999999999, 2024-09-16T20:50:23.995 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.028948432999999999, 2024-09-16T20:50:23.996 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:23.996 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:23.996 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:23.996 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:23.996 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:23.996 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:23.996 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4630 2024-09-16T20:50:23.996 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.996 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:23.996 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.996 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:23.996 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860576+0000", 2024-09-16T20:50:23.996 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.996 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.997 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.997 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:23.997 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860576+0000", 2024-09-16T20:50:23.997 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:23.997 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.997 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.997 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:23.997 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860577+0000", 2024-09-16T20:50:23.997 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6600000000000002e-07 2024-09-16T20:50:23.997 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.997 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.997 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:23.997 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860585+0000", 2024-09-16T20:50:23.997 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6580000000000008e-06 2024-09-16T20:50:23.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.998 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.998 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:23.998 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860594+0000", 2024-09-16T20:50:23.998 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6179999999999995e-06 2024-09-16T20:50:23.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.998 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.998 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:23.998 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860600+0000", 2024-09-16T20:50:23.998 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.579e-06 2024-09-16T20:50:23.998 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.998 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:23.998 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:23.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.999 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:23.999 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5078 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:23.999 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.876188+0000", 2024-09-16T20:50:23.999 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.012269428000000001, 2024-09-16T20:50:23.999 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.013366266999999999, 2024-09-16T20:50:23.999 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:23.999 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:23.999 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:23.999 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:23.999 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:23.999 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:23.999 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5078 2024-09-16T20:50:23.999 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:23.999 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:23.999 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.000 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.000 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.876188+0000", 2024-09-16T20:50:24.000 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.000 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.000 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.000 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.000 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.876188+0000", 2024-09-16T20:50:24.000 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.000 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.000 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.000 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.000 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.876189+0000", 2024-09-16T20:50:24.000 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0100000000000004e-07 2024-09-16T20:50:24.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.001 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.001 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.001 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.876192+0000", 2024-09-16T20:50:24.001 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3100000000000001e-06 2024-09-16T20:50:24.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.001 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.001 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.001 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.876201+0000", 2024-09-16T20:50:24.001 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8179999999999993e-06 2024-09-16T20:50:24.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.001 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.001 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.001 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.876207+0000", 2024-09-16T20:50:24.002 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0560000000000001e-06 2024-09-16T20:50:24.002 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.002 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.002 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.002 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5267 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.002 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.887500+0000", 2024-09-16T20:50:24.002 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.00095745699999999999, 2024-09-16T20:50:24.002 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0020813369999999999, 2024-09-16T20:50:24.002 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.002 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.002 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.002 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.003 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.003 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.003 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5267 2024-09-16T20:50:24.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.003 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.003 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.003 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.003 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887500+0000", 2024-09-16T20:50:24.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.003 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.003 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.003 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887500+0000", 2024-09-16T20:50:24.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.004 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.004 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.004 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887501+0000", 2024-09-16T20:50:24.004 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5099999999999999e-07 2024-09-16T20:50:24.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.004 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.004 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.004 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887504+0000", 2024-09-16T20:50:24.004 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7560000000000001e-06 2024-09-16T20:50:24.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.004 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.004 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.004 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887513+0000", 2024-09-16T20:50:24.005 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5120000000000008e-06 2024-09-16T20:50:24.005 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.005 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.005 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.005 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887519+0000", 2024-09-16T20:50:24.005 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2020000000000001e-06 2024-09-16T20:50:24.005 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.005 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.005 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.005 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.005 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.005 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:4636 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.005 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.860618+0000", 2024-09-16T20:50:24.005 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.027838964000000001, 2024-09-16T20:50:24.006 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.028985595999999999, 2024-09-16T20:50:24.006 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.006 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.006 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.006 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.006 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.006 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.006 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4636 2024-09-16T20:50:24.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.006 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.006 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.006 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.006 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860618+0000", 2024-09-16T20:50:24.006 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.007 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.007 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.007 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.007 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860618+0000", 2024-09-16T20:50:24.007 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.007 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.007 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.007 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.007 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860619+0000", 2024-09-16T20:50:24.007 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3200000000000005e-07 2024-09-16T20:50:24.007 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.007 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.007 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.007 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860621+0000", 2024-09-16T20:50:24.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3240000000000001e-06 2024-09-16T20:50:24.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.008 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860629+0000", 2024-09-16T20:50:24.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6140000000000003e-06 2024-09-16T20:50:24.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.008 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860634+0000", 2024-09-16T20:50:24.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3820000000000003e-06 2024-09-16T20:50:24.008 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.008 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.008 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.009 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.009 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5085 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.009 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.876598+0000", 2024-09-16T20:50:24.009 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.011859155, 2024-09-16T20:50:24.009 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.013081539, 2024-09-16T20:50:24.009 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.009 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.009 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.009 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.009 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.009 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.009 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5085 2024-09-16T20:50:24.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.010 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.010 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.010 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.010 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.876598+0000", 2024-09-16T20:50:24.010 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.010 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.010 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.010 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.010 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.876598+0000", 2024-09-16T20:50:24.010 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.010 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.010 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.010 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.011 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.876599+0000", 2024-09-16T20:50:24.011 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8000000000000004e-07 2024-09-16T20:50:24.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.011 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.011 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.011 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.876603+0000", 2024-09-16T20:50:24.011 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1439999999999996e-06 2024-09-16T20:50:24.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.011 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.011 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.011 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.876615+0000", 2024-09-16T20:50:24.011 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1673e-05 2024-09-16T20:50:24.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.011 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.012 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.012 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.876622+0000", 2024-09-16T20:50:24.012 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3289999999999996e-06 2024-09-16T20:50:24.012 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.012 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.012 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.012 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.012 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5268 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.012 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.887536+0000", 2024-09-16T20:50:24.012 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.00092186299999999998, 2024-09-16T20:50:24.012 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0021988760000000002, 2024-09-16T20:50:24.012 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.012 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.013 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.013 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.013 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.013 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.013 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5268 2024-09-16T20:50:24.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.013 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.013 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.013 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.013 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887536+0000", 2024-09-16T20:50:24.013 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.013 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.014 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.014 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887536+0000", 2024-09-16T20:50:24.014 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.014 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.014 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.014 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887536+0000", 2024-09-16T20:50:24.014 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7899999999999998e-07 2024-09-16T20:50:24.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.014 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.014 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.014 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887539+0000", 2024-09-16T20:50:24.014 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6800000000000002e-06 2024-09-16T20:50:24.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.015 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.015 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.015 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887546+0000", 2024-09-16T20:50:24.015 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5839999999999997e-06 2024-09-16T20:50:24.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.015 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.015 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.015 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887552+0000", 2024-09-16T20:50:24.015 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9939999999999997e-06 2024-09-16T20:50:24.015 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.015 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.015 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.015 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.015 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:4638 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.016 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.860642+0000", 2024-09-16T20:50:24.016 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.027815849, 2024-09-16T20:50:24.016 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.029140851999999998, 2024-09-16T20:50:24.016 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.016 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.016 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.016 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.016 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.016 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.016 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4638 2024-09-16T20:50:24.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.016 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.016 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.017 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.017 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860642+0000", 2024-09-16T20:50:24.017 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.017 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.017 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.017 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.017 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860642+0000", 2024-09-16T20:50:24.017 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.017 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.017 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.017 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.017 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860642+0000", 2024-09-16T20:50:24.017 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1500000000000005e-07 2024-09-16T20:50:24.017 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.018 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.018 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.018 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860644+0000", 2024-09-16T20:50:24.018 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0430000000000002e-06 2024-09-16T20:50:24.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.018 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.018 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.018 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860651+0000", 2024-09-16T20:50:24.018 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5910000000000001e-06 2024-09-16T20:50:24.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.018 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.018 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.018 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860656+0000", 2024-09-16T20:50:24.018 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2120000000000002e-06 2024-09-16T20:50:24.019 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.019 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.019 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.019 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5087 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.019 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.876641+0000", 2024-09-16T20:50:24.019 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.011816042000000001, 2024-09-16T20:50:24.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.013190248, 2024-09-16T20:50:24.019 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.019 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.019 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.019 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.019 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.020 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.020 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5087 2024-09-16T20:50:24.020 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.020 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.020 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.020 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.020 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.876641+0000", 2024-09-16T20:50:24.020 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.020 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.020 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.020 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.020 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.876641+0000", 2024-09-16T20:50:24.020 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.021 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.021 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.021 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.876642+0000", 2024-09-16T20:50:24.021 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3399999999999999e-07 2024-09-16T20:50:24.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.021 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.021 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.021 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.876645+0000", 2024-09-16T20:50:24.021 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6850000000000001e-06 2024-09-16T20:50:24.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.021 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.021 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.021 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.876652+0000", 2024-09-16T20:50:24.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8939999999999993e-06 2024-09-16T20:50:24.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.022 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.022 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.022 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.876659+0000", 2024-09-16T20:50:24.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4999999999999996e-06 2024-09-16T20:50:24.022 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.022 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.022 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.022 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.022 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5269 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.022 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.887564+0000", 2024-09-16T20:50:24.022 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.00089329600000000002, 2024-09-16T20:50:24.023 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0022918539999999999, 2024-09-16T20:50:24.023 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.023 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.023 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.023 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.023 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.023 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.023 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5269 2024-09-16T20:50:24.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.023 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.023 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.023 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.023 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887564+0000", 2024-09-16T20:50:24.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.024 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.024 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887564+0000", 2024-09-16T20:50:24.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.024 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.024 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887565+0000", 2024-09-16T20:50:24.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9500000000000003e-07 2024-09-16T20:50:24.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887567+0000", 2024-09-16T20:50:24.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3499999999999999e-06 2024-09-16T20:50:24.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887574+0000", 2024-09-16T20:50:24.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0609999999999998e-06 2024-09-16T20:50:24.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887580+0000", 2024-09-16T20:50:24.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6110000000000001e-06 2024-09-16T20:50:24.026 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.026 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.026 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.026 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:4639 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.026 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.860663+0000", 2024-09-16T20:50:24.026 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.027794743, 2024-09-16T20:50:24.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.029231244999999999, 2024-09-16T20:50:24.026 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.026 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.026 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.026 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.026 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.027 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.027 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4639 2024-09-16T20:50:24.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.027 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.027 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.027 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.027 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860663+0000", 2024-09-16T20:50:24.027 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.027 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.027 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.027 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860663+0000", 2024-09-16T20:50:24.027 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.028 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.028 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.028 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860663+0000", 2024-09-16T20:50:24.028 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7999999999999996e-07 2024-09-16T20:50:24.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.028 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.028 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.028 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860665+0000", 2024-09-16T20:50:24.028 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.153e-06 2024-09-16T20:50:24.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.028 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.028 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.028 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860672+0000", 2024-09-16T20:50:24.028 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9820000000000002e-06 2024-09-16T20:50:24.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.029 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.029 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860686+0000", 2024-09-16T20:50:24.029 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3743e-05 2024-09-16T20:50:24.029 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.029 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.029 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.029 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5088 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.029 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.876731+0000", 2024-09-16T20:50:24.029 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.011726075000000001, 2024-09-16T20:50:24.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.013199779999999999, 2024-09-16T20:50:24.030 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.030 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.030 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.030 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.030 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.030 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.030 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5088 2024-09-16T20:50:24.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.030 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.030 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.030 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.876731+0000", 2024-09-16T20:50:24.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.031 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.031 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.031 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.876731+0000", 2024-09-16T20:50:24.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.031 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.031 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.031 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.876732+0000", 2024-09-16T20:50:24.032 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1699999999999997e-07 2024-09-16T20:50:24.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.032 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.032 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.876736+0000", 2024-09-16T20:50:24.032 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.658e-06 2024-09-16T20:50:24.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.033 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.033 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.033 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.876745+0000", 2024-09-16T20:50:24.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7170000000000003e-06 2024-09-16T20:50:24.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.033 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.033 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.033 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.876752+0000", 2024-09-16T20:50:24.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6560000000000003e-06 2024-09-16T20:50:24.033 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.033 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.033 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.034 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.034 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5270 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.034 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.887625+0000", 2024-09-16T20:50:24.034 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.00083257800000000005, 2024-09-16T20:50:24.034 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0023516129999999998, 2024-09-16T20:50:24.034 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.034 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.034 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.034 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.034 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.034 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.034 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5270 2024-09-16T20:50:24.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.035 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.035 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.035 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887625+0000", 2024-09-16T20:50:24.035 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.035 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.035 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887625+0000", 2024-09-16T20:50:24.035 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.035 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.036 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887625+0000", 2024-09-16T20:50:24.036 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7999999999999995e-07 2024-09-16T20:50:24.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.036 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.036 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887629+0000", 2024-09-16T20:50:24.036 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.388e-06 2024-09-16T20:50:24.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.036 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.036 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887639+0000", 2024-09-16T20:50:24.036 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8910000000000007e-06 2024-09-16T20:50:24.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.037 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.037 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887645+0000", 2024-09-16T20:50:24.037 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2899999999999999e-06 2024-09-16T20:50:24.037 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.037 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.037 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.037 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.037 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:4640 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.037 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.860694+0000", 2024-09-16T20:50:24.037 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.027763519, 2024-09-16T20:50:24.037 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.029324146999999998, 2024-09-16T20:50:24.037 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.038 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.038 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.038 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.038 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.038 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.038 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4640 2024-09-16T20:50:24.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.038 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.038 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.038 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.038 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860694+0000", 2024-09-16T20:50:24.038 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.039 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.039 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860694+0000", 2024-09-16T20:50:24.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.039 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.039 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860694+0000", 2024-09-16T20:50:24.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2500000000000001e-07 2024-09-16T20:50:24.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.039 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.039 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860697+0000", 2024-09-16T20:50:24.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3590000000000002e-06 2024-09-16T20:50:24.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.040 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.040 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.040 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860703+0000", 2024-09-16T20:50:24.040 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5509999999999996e-06 2024-09-16T20:50:24.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.040 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.040 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.040 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860710+0000", 2024-09-16T20:50:24.040 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9920000000000001e-06 2024-09-16T20:50:24.040 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.040 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.040 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.040 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.041 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5089 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.041 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.876786+0000", 2024-09-16T20:50:24.041 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.011671333000000001, 2024-09-16T20:50:24.041 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.013276247, 2024-09-16T20:50:24.041 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.041 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.041 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.041 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.041 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.041 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.041 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5089 2024-09-16T20:50:24.041 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.041 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.042 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.042 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.042 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.876786+0000", 2024-09-16T20:50:24.042 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.042 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.042 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.042 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.042 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.876786+0000", 2024-09-16T20:50:24.042 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.042 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.042 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.042 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.042 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.876787+0000", 2024-09-16T20:50:24.042 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6499999999999999e-07 2024-09-16T20:50:24.042 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.043 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.043 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.043 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.876789+0000", 2024-09-16T20:50:24.043 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7199999999999998e-06 2024-09-16T20:50:24.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.043 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.043 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.043 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.876798+0000", 2024-09-16T20:50:24.043 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5609999999999999e-06 2024-09-16T20:50:24.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.043 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.043 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.043 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.876807+0000", 2024-09-16T20:50:24.044 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5780000000000005e-06 2024-09-16T20:50:24.044 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.044 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.044 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.044 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.044 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5271 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.044 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.887708+0000", 2024-09-16T20:50:24.044 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.00074945700000000003, 2024-09-16T20:50:24.044 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0023785540000000002, 2024-09-16T20:50:24.044 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.044 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.044 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.045 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.045 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.045 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.045 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5271 2024-09-16T20:50:24.045 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.045 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.045 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.045 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.045 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887708+0000", 2024-09-16T20:50:24.046 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.046 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.046 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.046 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.046 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887708+0000", 2024-09-16T20:50:24.046 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.046 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.046 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.046 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.046 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887709+0000", 2024-09-16T20:50:24.046 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3e-07 2024-09-16T20:50:24.046 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.046 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.047 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.047 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887712+0000", 2024-09-16T20:50:24.047 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7859999999999998e-06 2024-09-16T20:50:24.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.047 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.047 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.047 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887722+0000", 2024-09-16T20:50:24.047 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4949999999999997e-06 2024-09-16T20:50:24.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.047 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.047 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.047 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887729+0000", 2024-09-16T20:50:24.047 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5649999999999998e-06 2024-09-16T20:50:24.047 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.048 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.048 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.048 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.048 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:4642 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.048 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.860738+0000", 2024-09-16T20:50:24.048 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.027719486000000002, 2024-09-16T20:50:24.048 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.029373949, 2024-09-16T20:50:24.048 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.048 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.048 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.048 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.048 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.048 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.049 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4642 2024-09-16T20:50:24.049 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.049 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.049 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.049 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.049 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860738+0000", 2024-09-16T20:50:24.049 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.049 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.049 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.049 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.049 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860738+0000", 2024-09-16T20:50:24.049 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.049 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.050 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.050 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.050 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860739+0000", 2024-09-16T20:50:24.050 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.13e-07 2024-09-16T20:50:24.050 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.050 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.050 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.050 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860741+0000", 2024-09-16T20:50:24.050 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6630000000000002e-06 2024-09-16T20:50:24.050 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.050 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.050 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.050 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860750+0000", 2024-09-16T20:50:24.051 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8990000000000007e-06 2024-09-16T20:50:24.051 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.051 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.051 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.051 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860756+0000", 2024-09-16T20:50:24.051 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7710000000000003e-06 2024-09-16T20:50:24.051 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.051 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.051 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.051 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.051 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.051 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5090 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.051 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.876820+0000", 2024-09-16T20:50:24.051 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.011637366999999999, 2024-09-16T20:50:24.052 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.01331452, 2024-09-16T20:50:24.052 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.052 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.052 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.052 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.052 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.052 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.052 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5090 2024-09-16T20:50:24.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.052 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.052 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.052 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.052 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.876820+0000", 2024-09-16T20:50:24.053 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.053 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.053 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.053 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.053 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.876820+0000", 2024-09-16T20:50:24.053 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.053 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.053 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.053 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.053 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.876821+0000", 2024-09-16T20:50:24.053 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1200000000000003e-07 2024-09-16T20:50:24.053 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.053 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.054 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.054 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.876823+0000", 2024-09-16T20:50:24.054 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5100000000000001e-06 2024-09-16T20:50:24.054 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.054 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.054 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.054 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.876830+0000", 2024-09-16T20:50:24.054 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4410000000000004e-06 2024-09-16T20:50:24.054 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.054 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.054 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.054 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.876836+0000", 2024-09-16T20:50:24.055 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8089999999999998e-06 2024-09-16T20:50:24.055 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.055 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.055 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.055 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.055 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.055 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5272 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.055 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.887754+0000", 2024-09-16T20:50:24.055 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.00070357899999999997, 2024-09-16T20:50:24.055 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0024046470000000002, 2024-09-16T20:50:24.055 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.055 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.055 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.055 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.056 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.056 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.056 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5272 2024-09-16T20:50:24.056 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.056 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.056 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.056 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.056 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887754+0000", 2024-09-16T20:50:24.056 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.056 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.056 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.056 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.056 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887754+0000", 2024-09-16T20:50:24.056 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.057 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.057 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.057 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887754+0000", 2024-09-16T20:50:24.057 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5099999999999999e-07 2024-09-16T20:50:24.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.057 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.057 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.057 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887757+0000", 2024-09-16T20:50:24.057 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8459999999999999e-06 2024-09-16T20:50:24.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.057 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.057 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.058 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887766+0000", 2024-09-16T20:50:24.058 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3520000000000007e-06 2024-09-16T20:50:24.058 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.058 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.058 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.058 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887771+0000", 2024-09-16T20:50:24.058 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8100000000000003e-06 2024-09-16T20:50:24.058 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.058 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.058 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.058 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.058 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.058 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:4644 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.058 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.860764+0000", 2024-09-16T20:50:24.059 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.027692985, 2024-09-16T20:50:24.059 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.029422639, 2024-09-16T20:50:24.059 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.059 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.059 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.059 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.059 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.059 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.059 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4644 2024-09-16T20:50:24.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.059 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.059 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.059 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.060 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860764+0000", 2024-09-16T20:50:24.060 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.060 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.060 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.060 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.060 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860764+0000", 2024-09-16T20:50:24.060 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.060 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.060 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.060 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.060 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860765+0000", 2024-09-16T20:50:24.060 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4300000000000003e-07 2024-09-16T20:50:24.060 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.061 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.061 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.061 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860767+0000", 2024-09-16T20:50:24.061 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0279999999999999e-06 2024-09-16T20:50:24.061 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.061 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.061 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.061 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860775+0000", 2024-09-16T20:50:24.061 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1280000000000008e-06 2024-09-16T20:50:24.061 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.061 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.061 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.062 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860781+0000", 2024-09-16T20:50:24.062 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5369999999999997e-06 2024-09-16T20:50:24.062 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.062 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.062 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.062 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.062 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.062 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5092 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.062 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.876848+0000", 2024-09-16T20:50:24.062 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.011609029999999999, 2024-09-16T20:50:24.062 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.013370656, 2024-09-16T20:50:24.062 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.062 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.062 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.063 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.063 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.063 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.063 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5092 2024-09-16T20:50:24.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.063 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.063 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.063 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.063 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.876848+0000", 2024-09-16T20:50:24.063 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.063 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.063 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.064 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.876848+0000", 2024-09-16T20:50:24.064 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.064 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.064 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.876849+0000", 2024-09-16T20:50:24.064 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8299999999999997e-07 2024-09-16T20:50:24.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.064 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.064 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.876851+0000", 2024-09-16T20:50:24.064 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2579999999999998e-06 2024-09-16T20:50:24.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.065 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.065 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.876864+0000", 2024-09-16T20:50:24.065 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3165e-05 2024-09-16T20:50:24.065 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.065 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.065 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.065 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.876875+0000", 2024-09-16T20:50:24.065 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0927999999999999e-05 2024-09-16T20:50:24.065 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.065 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.065 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.065 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.065 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.066 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5273 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.066 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.887801+0000", 2024-09-16T20:50:24.066 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.000656406, 2024-09-16T20:50:24.066 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0024411490000000001, 2024-09-16T20:50:24.066 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.066 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.066 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.066 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.066 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.066 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.066 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5273 2024-09-16T20:50:24.066 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.066 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.067 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.067 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.067 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887801+0000", 2024-09-16T20:50:24.067 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.067 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.067 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.067 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.067 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887801+0000", 2024-09-16T20:50:24.067 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.067 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.067 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.067 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.067 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887802+0000", 2024-09-16T20:50:24.067 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9400000000000005e-07 2024-09-16T20:50:24.068 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.068 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.068 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.068 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887804+0000", 2024-09-16T20:50:24.068 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6510000000000001e-06 2024-09-16T20:50:24.068 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.068 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.068 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.068 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887812+0000", 2024-09-16T20:50:24.068 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7589999999999997e-06 2024-09-16T20:50:24.068 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.068 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.068 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.068 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887818+0000", 2024-09-16T20:50:24.069 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9560000000000002e-06 2024-09-16T20:50:24.069 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.069 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.069 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.069 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.069 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:4647 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.069 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.860789+0000", 2024-09-16T20:50:24.069 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.027668853, 2024-09-16T20:50:24.069 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.029476491, 2024-09-16T20:50:24.069 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.069 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.069 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.070 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.070 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.070 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.070 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4647 2024-09-16T20:50:24.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.070 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.070 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.070 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860789+0000", 2024-09-16T20:50:24.070 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.070 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.070 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860789+0000", 2024-09-16T20:50:24.071 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.071 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.071 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.071 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860789+0000", 2024-09-16T20:50:24.071 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4099999999999999e-07 2024-09-16T20:50:24.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.071 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.071 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.071 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860791+0000", 2024-09-16T20:50:24.071 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0820000000000001e-06 2024-09-16T20:50:24.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.071 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.071 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.071 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860798+0000", 2024-09-16T20:50:24.072 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5760000000000002e-06 2024-09-16T20:50:24.072 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.072 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.072 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.072 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860803+0000", 2024-09-16T20:50:24.072 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9389999999999996e-06 2024-09-16T20:50:24.072 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.072 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.072 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.072 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.072 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.072 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5094 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.072 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.877047+0000", 2024-09-16T20:50:24.073 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.011410683, 2024-09-16T20:50:24.073 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.013255406000000001, 2024-09-16T20:50:24.073 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.073 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.073 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.073 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.073 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.073 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.073 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5094 2024-09-16T20:50:24.073 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.073 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.073 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.073 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.074 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.877047+0000", 2024-09-16T20:50:24.074 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.074 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.074 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.074 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.877047+0000", 2024-09-16T20:50:24.074 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.074 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.074 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.074 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.877048+0000", 2024-09-16T20:50:24.074 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1999999999999998e-07 2024-09-16T20:50:24.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.075 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.075 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.075 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.877051+0000", 2024-09-16T20:50:24.075 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8870000000000002e-06 2024-09-16T20:50:24.075 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.075 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.075 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.075 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.877065+0000", 2024-09-16T20:50:24.075 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3138e-05 2024-09-16T20:50:24.075 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.075 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.075 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.075 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.877071+0000", 2024-09-16T20:50:24.076 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.528e-06 2024-09-16T20:50:24.076 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.076 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.076 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.076 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.076 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.076 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5275 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.076 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.887898+0000", 2024-09-16T20:50:24.076 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.00055897200000000003, 2024-09-16T20:50:24.076 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.002438506, 2024-09-16T20:50:24.076 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.076 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.076 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.076 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.077 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.077 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.077 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5275 2024-09-16T20:50:24.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.077 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.077 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.077 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.077 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887898+0000", 2024-09-16T20:50:24.077 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.077 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.077 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.077 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887898+0000", 2024-09-16T20:50:24.078 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.078 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.078 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.078 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.078 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887899+0000", 2024-09-16T20:50:24.078 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2200000000000003e-07 2024-09-16T20:50:24.078 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.078 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.078 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.078 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887903+0000", 2024-09-16T20:50:24.078 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5190000000000001e-06 2024-09-16T20:50:24.078 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.078 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.078 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.079 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887912+0000", 2024-09-16T20:50:24.079 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4830000000000005e-06 2024-09-16T20:50:24.079 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.079 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.079 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.079 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.887919+0000", 2024-09-16T20:50:24.079 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5289999999999997e-06 2024-09-16T20:50:24.079 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.079 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.079 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.079 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.079 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.079 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:4649 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.079 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.860810+0000", 2024-09-16T20:50:24.080 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.027647794999999999, 2024-09-16T20:50:24.080 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.029553388, 2024-09-16T20:50:24.080 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.080 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.080 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.080 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.080 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.080 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.080 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4649 2024-09-16T20:50:24.080 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.080 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.080 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.080 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.080 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860810+0000", 2024-09-16T20:50:24.081 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.081 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.081 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.081 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.081 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860810+0000", 2024-09-16T20:50:24.081 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.081 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.081 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.081 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.081 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860810+0000", 2024-09-16T20:50:24.081 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2500000000000001e-07 2024-09-16T20:50:24.081 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.081 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.082 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.082 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860812+0000", 2024-09-16T20:50:24.082 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.249e-06 2024-09-16T20:50:24.082 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.082 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.082 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.082 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860819+0000", 2024-09-16T20:50:24.082 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8510000000000001e-06 2024-09-16T20:50:24.082 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.082 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.082 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.082 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860824+0000", 2024-09-16T20:50:24.082 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1070000000000004e-06 2024-09-16T20:50:24.082 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.083 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.083 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.083 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.083 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.083 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5096 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.083 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.877104+0000", 2024-09-16T20:50:24.083 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.011353716999999999, 2024-09-16T20:50:24.083 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.013282054, 2024-09-16T20:50:24.083 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.083 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.083 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.083 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.083 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.083 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.084 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5096 2024-09-16T20:50:24.084 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.084 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.084 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.084 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.084 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.877104+0000", 2024-09-16T20:50:24.084 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.084 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.084 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.084 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.084 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.877104+0000", 2024-09-16T20:50:24.084 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.085 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.085 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.085 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.085 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.877104+0000", 2024-09-16T20:50:24.085 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5899999999999996e-07 2024-09-16T20:50:24.085 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.085 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.085 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.085 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.877107+0000", 2024-09-16T20:50:24.085 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0960000000000001e-06 2024-09-16T20:50:24.085 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.085 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.085 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.086 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.877115+0000", 2024-09-16T20:50:24.086 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1499999999999999e-06 2024-09-16T20:50:24.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.086 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.086 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.086 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.877122+0000", 2024-09-16T20:50:24.086 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2600000000000002e-06 2024-09-16T20:50:24.086 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.086 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.086 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.086 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.086 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5277 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.086 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.888060+0000", 2024-09-16T20:50:24.087 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.00039758099999999999, 2024-09-16T20:50:24.087 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0023509749999999999, 2024-09-16T20:50:24.087 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.087 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.087 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.087 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.087 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.087 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.087 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5277 2024-09-16T20:50:24.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.087 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.087 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.087 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.087 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888060+0000", 2024-09-16T20:50:24.088 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.088 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.088 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.088 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.088 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888060+0000", 2024-09-16T20:50:24.088 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.088 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.088 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.088 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.088 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888061+0000", 2024-09-16T20:50:24.088 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7700000000000004e-07 2024-09-16T20:50:24.088 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.088 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.089 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.089 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888065+0000", 2024-09-16T20:50:24.089 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0040000000000001e-06 2024-09-16T20:50:24.089 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.089 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.089 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.089 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888075+0000", 2024-09-16T20:50:24.089 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0305e-05 2024-09-16T20:50:24.089 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.089 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.089 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.089 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888082+0000", 2024-09-16T20:50:24.089 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9700000000000002e-06 2024-09-16T20:50:24.089 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.090 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.090 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.090 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.090 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.090 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:4651 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.090 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.860843+0000", 2024-09-16T20:50:24.090 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.027614062000000002, 2024-09-16T20:50:24.090 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.029590274, 2024-09-16T20:50:24.090 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.090 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.090 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.090 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.090 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.090 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.091 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4651 2024-09-16T20:50:24.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.091 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860843+0000", 2024-09-16T20:50:24.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860843+0000", 2024-09-16T20:50:24.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.092 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860844+0000", 2024-09-16T20:50:24.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9699999999999995e-07 2024-09-16T20:50:24.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.092 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.092 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.092 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860847+0000", 2024-09-16T20:50:24.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5320000000000001e-06 2024-09-16T20:50:24.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.092 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.092 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.092 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860855+0000", 2024-09-16T20:50:24.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3510000000000002e-06 2024-09-16T20:50:24.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.092 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.093 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.093 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860861+0000", 2024-09-16T20:50:24.093 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7200000000000003e-06 2024-09-16T20:50:24.093 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.093 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.093 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.093 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.093 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.093 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5098 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.093 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.877245+0000", 2024-09-16T20:50:24.093 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.011211942000000001, 2024-09-16T20:50:24.093 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.013212142, 2024-09-16T20:50:24.094 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.094 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.094 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.094 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.094 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.094 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.094 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5098 2024-09-16T20:50:24.094 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.094 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.094 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.094 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.094 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.877245+0000", 2024-09-16T20:50:24.094 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.094 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.095 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.877245+0000", 2024-09-16T20:50:24.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.095 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.877246+0000", 2024-09-16T20:50:24.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2699999999999998e-07 2024-09-16T20:50:24.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.095 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.877250+0000", 2024-09-16T20:50:24.096 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5370000000000002e-06 2024-09-16T20:50:24.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.096 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.096 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.096 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.877260+0000", 2024-09-16T20:50:24.096 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0015e-05 2024-09-16T20:50:24.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.096 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.096 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.096 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.877266+0000", 2024-09-16T20:50:24.096 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4999999999999996e-06 2024-09-16T20:50:24.096 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.096 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.096 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.097 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.097 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5279 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.097 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.888117+0000", 2024-09-16T20:50:24.097 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.00034004199999999998, 2024-09-16T20:50:24.097 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0023626749999999998, 2024-09-16T20:50:24.097 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.097 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.097 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.097 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.097 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.097 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.097 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5279 2024-09-16T20:50:24.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.098 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.098 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.098 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.098 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888117+0000", 2024-09-16T20:50:24.098 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.098 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.098 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.098 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.098 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888117+0000", 2024-09-16T20:50:24.098 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.098 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.098 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.098 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.098 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888118+0000", 2024-09-16T20:50:24.099 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9699999999999995e-07 2024-09-16T20:50:24.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.099 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.099 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.099 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888121+0000", 2024-09-16T20:50:24.099 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3639999999999999e-06 2024-09-16T20:50:24.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.099 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.099 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.099 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888130+0000", 2024-09-16T20:50:24.099 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.799e-06 2024-09-16T20:50:24.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.099 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.100 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.100 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888136+0000", 2024-09-16T20:50:24.100 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1959999999999996e-06 2024-09-16T20:50:24.100 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.100 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.100 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.100 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.100 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.100 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:4653 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.100 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.860869+0000", 2024-09-16T20:50:24.100 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.027588056, 2024-09-16T20:50:24.100 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.029638958999999999, 2024-09-16T20:50:24.100 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.100 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.101 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.101 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.101 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.101 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.101 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4653 2024-09-16T20:50:24.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.101 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.101 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.101 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860869+0000", 2024-09-16T20:50:24.101 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.101 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.102 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860869+0000", 2024-09-16T20:50:24.102 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.102 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.102 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.102 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.102 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860870+0000", 2024-09-16T20:50:24.102 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0200000000000002e-07 2024-09-16T20:50:24.102 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.102 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.102 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.102 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860872+0000", 2024-09-16T20:50:24.102 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2479999999999999e-06 2024-09-16T20:50:24.102 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.102 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.103 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.103 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860885+0000", 2024-09-16T20:50:24.103 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3104e-05 2024-09-16T20:50:24.103 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.103 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.103 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.103 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860892+0000", 2024-09-16T20:50:24.103 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0400000000000004e-06 2024-09-16T20:50:24.103 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.103 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.103 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.103 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.103 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.104 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5099 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.104 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.877298+0000", 2024-09-16T20:50:24.104 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.011159469, 2024-09-16T20:50:24.104 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.013242611, 2024-09-16T20:50:24.104 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.104 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.104 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.104 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.104 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.104 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.104 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5099 2024-09-16T20:50:24.104 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.104 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.104 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.105 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.105 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.877298+0000", 2024-09-16T20:50:24.105 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.105 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.105 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.877298+0000", 2024-09-16T20:50:24.105 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.105 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.105 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.877299+0000", 2024-09-16T20:50:24.105 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8899999999999999e-07 2024-09-16T20:50:24.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.106 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.106 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.877302+0000", 2024-09-16T20:50:24.106 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2140000000000001e-06 2024-09-16T20:50:24.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.106 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.106 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.877310+0000", 2024-09-16T20:50:24.106 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6249999999999996e-06 2024-09-16T20:50:24.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.106 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.106 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.877322+0000", 2024-09-16T20:50:24.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1708999999999999e-05 2024-09-16T20:50:24.107 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.107 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.107 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.107 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5280 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.107 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.888179+0000", 2024-09-16T20:50:24.107 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.00027882500000000001, 2024-09-16T20:50:24.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0023888749999999999, 2024-09-16T20:50:24.107 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.107 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.107 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.107 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.108 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.108 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.108 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5280 2024-09-16T20:50:24.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.108 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888179+0000", 2024-09-16T20:50:24.108 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888179+0000", 2024-09-16T20:50:24.108 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.109 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.109 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.109 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888179+0000", 2024-09-16T20:50:24.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3399999999999998e-07 2024-09-16T20:50:24.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.109 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.109 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.109 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888182+0000", 2024-09-16T20:50:24.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1370000000000002e-06 2024-09-16T20:50:24.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.109 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.109 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.109 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888191+0000", 2024-09-16T20:50:24.110 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3240000000000003e-06 2024-09-16T20:50:24.110 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.110 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.110 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.110 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888197+0000", 2024-09-16T20:50:24.110 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8170000000000004e-06 2024-09-16T20:50:24.110 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.110 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.110 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.110 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.110 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.110 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:4655 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.110 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.860900+0000", 2024-09-16T20:50:24.110 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.027557249999999998, 2024-09-16T20:50:24.111 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.029690639000000001, 2024-09-16T20:50:24.111 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.111 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.111 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.111 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.111 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.111 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.111 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4655 2024-09-16T20:50:24.111 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.111 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.111 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.111 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.111 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860900+0000", 2024-09-16T20:50:24.112 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.112 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.112 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860900+0000", 2024-09-16T20:50:24.112 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.112 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.112 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860901+0000", 2024-09-16T20:50:24.112 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6499999999999999e-07 2024-09-16T20:50:24.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.112 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.113 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860903+0000", 2024-09-16T20:50:24.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3290000000000001e-06 2024-09-16T20:50:24.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.113 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.113 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.113 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860910+0000", 2024-09-16T20:50:24.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7719999999999997e-06 2024-09-16T20:50:24.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.113 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.113 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.113 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860917+0000", 2024-09-16T20:50:24.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9179999999999997e-06 2024-09-16T20:50:24.113 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.113 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.114 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.114 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.114 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5101 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.114 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.877482+0000", 2024-09-16T20:50:24.114 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.010975665000000001, 2024-09-16T20:50:24.114 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.013144650000000001, 2024-09-16T20:50:24.114 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.114 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.114 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.114 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.114 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.114 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.115 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5101 2024-09-16T20:50:24.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.115 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.115 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.115 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.115 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.877482+0000", 2024-09-16T20:50:24.115 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.115 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.115 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.115 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.877482+0000", 2024-09-16T20:50:24.115 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.115 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.116 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.116 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.877483+0000", 2024-09-16T20:50:24.116 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8299999999999996e-07 2024-09-16T20:50:24.116 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.116 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.116 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.116 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.877486+0000", 2024-09-16T20:50:24.116 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8750000000000002e-06 2024-09-16T20:50:24.116 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.116 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.116 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.116 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.877497+0000", 2024-09-16T20:50:24.116 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1011e-05 2024-09-16T20:50:24.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.117 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.117 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.877507+0000", 2024-09-16T20:50:24.117 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2029999999999998e-06 2024-09-16T20:50:24.117 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.117 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.117 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.117 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5281 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.117 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.888213+0000", 2024-09-16T20:50:24.117 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.00024418600000000002, 2024-09-16T20:50:24.118 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0024371649999999998, 2024-09-16T20:50:24.118 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.118 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.118 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.118 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.118 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.118 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.118 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5281 2024-09-16T20:50:24.118 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.118 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.118 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.118 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.118 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888213+0000", 2024-09-16T20:50:24.119 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.119 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.119 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.119 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888213+0000", 2024-09-16T20:50:24.119 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.119 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.119 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.119 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888214+0000", 2024-09-16T20:50:24.119 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8100000000000003e-07 2024-09-16T20:50:24.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.119 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.120 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.120 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888216+0000", 2024-09-16T20:50:24.120 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6309999999999999e-06 2024-09-16T20:50:24.120 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.120 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.120 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.120 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888224+0000", 2024-09-16T20:50:24.120 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7519999999999996e-06 2024-09-16T20:50:24.120 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.120 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.120 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.120 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888232+0000", 2024-09-16T20:50:24.121 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6909999999999997e-06 2024-09-16T20:50:24.121 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.121 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.121 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.121 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.121 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.121 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:4657 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.121 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.860924+0000", 2024-09-16T20:50:24.121 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.027533294999999999, 2024-09-16T20:50:24.121 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.029751229000000001, 2024-09-16T20:50:24.121 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.121 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.121 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.122 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.122 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.122 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.122 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4657 2024-09-16T20:50:24.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.122 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.122 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.122 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.122 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860924+0000", 2024-09-16T20:50:24.122 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.122 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.122 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.123 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860924+0000", 2024-09-16T20:50:24.123 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.123 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.123 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.123 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.123 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860925+0000", 2024-09-16T20:50:24.123 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1800000000000001e-07 2024-09-16T20:50:24.123 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.123 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.123 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.123 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860935+0000", 2024-09-16T20:50:24.123 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0302000000000001e-05 2024-09-16T20:50:24.123 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.124 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.124 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.124 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860942+0000", 2024-09-16T20:50:24.124 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4529999999999996e-06 2024-09-16T20:50:24.124 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.124 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.124 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.124 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860948+0000", 2024-09-16T20:50:24.124 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4840000000000003e-06 2024-09-16T20:50:24.124 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.124 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.124 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.125 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.125 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5103 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.125 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.877643+0000", 2024-09-16T20:50:24.125 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.010814007, 2024-09-16T20:50:24.125 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.013054815000000001, 2024-09-16T20:50:24.125 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.125 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.125 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.125 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.125 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.125 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.125 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5103 2024-09-16T20:50:24.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.126 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.126 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.126 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.126 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.877643+0000", 2024-09-16T20:50:24.126 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.126 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.126 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.126 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.126 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.877643+0000", 2024-09-16T20:50:24.126 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.126 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.126 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.126 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.127 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.877644+0000", 2024-09-16T20:50:24.127 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2399999999999997e-07 2024-09-16T20:50:24.127 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.127 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.127 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.127 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.877648+0000", 2024-09-16T20:50:24.127 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7060000000000002e-06 2024-09-16T20:50:24.127 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.127 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.127 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.127 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.877661+0000", 2024-09-16T20:50:24.127 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2870999999999999e-05 2024-09-16T20:50:24.127 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.128 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.128 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.128 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.877668+0000", 2024-09-16T20:50:24.128 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.922e-06 2024-09-16T20:50:24.128 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.128 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.128 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.128 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.128 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5282 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.128 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.888329+0000", 2024-09-16T20:50:24.128 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.000128183, 2024-09-16T20:50:24.128 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.002391691, 2024-09-16T20:50:24.129 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.129 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.129 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.129 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.129 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.129 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.129 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5282 2024-09-16T20:50:24.129 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.129 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.129 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.129 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.129 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888329+0000", 2024-09-16T20:50:24.129 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.130 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.130 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.130 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.130 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888329+0000", 2024-09-16T20:50:24.130 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.130 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.130 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.130 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.130 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888330+0000", 2024-09-16T20:50:24.130 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2799999999999995e-07 2024-09-16T20:50:24.130 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.130 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.130 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.131 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888334+0000", 2024-09-16T20:50:24.131 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.89e-06 2024-09-16T20:50:24.131 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.131 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.131 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.131 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888347+0000", 2024-09-16T20:50:24.131 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.258e-05 2024-09-16T20:50:24.131 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.131 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.131 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.131 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888355+0000", 2024-09-16T20:50:24.131 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8999999999999995e-06 2024-09-16T20:50:24.132 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.132 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.132 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.132 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.132 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.132 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:4659 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.132 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.860956+0000", 2024-09-16T20:50:24.132 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.027501634, 2024-09-16T20:50:24.132 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.029844612999999999, 2024-09-16T20:50:24.133 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.133 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.133 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.133 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.133 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.133 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.133 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4659 2024-09-16T20:50:24.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.133 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.133 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.133 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860956+0000", 2024-09-16T20:50:24.133 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.134 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.134 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.134 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860956+0000", 2024-09-16T20:50:24.134 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.134 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.134 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.134 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860956+0000", 2024-09-16T20:50:24.134 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6100000000000001e-07 2024-09-16T20:50:24.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.134 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.134 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.135 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860958+0000", 2024-09-16T20:50:24.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.119e-06 2024-09-16T20:50:24.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.135 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.135 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.135 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860967+0000", 2024-09-16T20:50:24.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.422e-06 2024-09-16T20:50:24.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.135 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.135 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.135 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860972+0000", 2024-09-16T20:50:24.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2689999999999999e-06 2024-09-16T20:50:24.135 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.136 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.136 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.136 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.136 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.136 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5105 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.136 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.877829+0000", 2024-09-16T20:50:24.136 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.010628525, 2024-09-16T20:50:24.136 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.013003143, 2024-09-16T20:50:24.136 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.136 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.136 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.136 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.137 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.137 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.137 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5105 2024-09-16T20:50:24.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.137 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.137 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.137 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.137 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.877829+0000", 2024-09-16T20:50:24.137 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.137 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.137 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.137 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.877829+0000", 2024-09-16T20:50:24.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.138 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.138 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.877830+0000", 2024-09-16T20:50:24.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9100000000000003e-07 2024-09-16T20:50:24.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.138 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.138 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.877833+0000", 2024-09-16T20:50:24.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7170000000000002e-06 2024-09-16T20:50:24.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.139 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.139 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.877843+0000", 2024-09-16T20:50:24.139 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.005e-05 2024-09-16T20:50:24.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.139 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.139 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.139 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.877850+0000", 2024-09-16T20:50:24.139 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0199999999999997e-06 2024-09-16T20:50:24.139 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.139 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.139 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.140 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.140 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5283 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.140 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.888377+0000", 2024-09-16T20:50:24.140 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.0486000000000002e-05, 2024-09-16T20:50:24.140 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0024849109999999998, 2024-09-16T20:50:24.140 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.140 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.140 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.140 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.140 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.140 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.140 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5283 2024-09-16T20:50:24.140 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.141 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.141 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.141 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.141 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888377+0000", 2024-09-16T20:50:24.141 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.141 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.141 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.141 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888377+0000", 2024-09-16T20:50:24.141 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.141 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.141 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.142 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888377+0000", 2024-09-16T20:50:24.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5400000000000002e-07 2024-09-16T20:50:24.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.142 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.142 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.142 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888380+0000", 2024-09-16T20:50:24.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3080000000000002e-06 2024-09-16T20:50:24.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.142 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.142 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.142 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888387+0000", 2024-09-16T20:50:24.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2080000000000003e-06 2024-09-16T20:50:24.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.143 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.143 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888392+0000", 2024-09-16T20:50:24.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.502e-06 2024-09-16T20:50:24.143 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.143 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.143 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.143 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:4667 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.143 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.860979+0000", 2024-09-16T20:50:24.143 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.027478255, 2024-09-16T20:50:24.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.029905971999999999, 2024-09-16T20:50:24.144 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.144 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.144 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.144 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.144 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.144 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.144 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4667 2024-09-16T20:50:24.144 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.144 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.144 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.144 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.144 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860979+0000", 2024-09-16T20:50:24.145 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.145 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.145 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.145 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860979+0000", 2024-09-16T20:50:24.145 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.145 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.145 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.145 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860980+0000", 2024-09-16T20:50:24.145 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3099999999999998e-07 2024-09-16T20:50:24.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.145 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.146 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.146 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860982+0000", 2024-09-16T20:50:24.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2170000000000001e-06 2024-09-16T20:50:24.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.146 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.146 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860988+0000", 2024-09-16T20:50:24.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1249999999999998e-06 2024-09-16T20:50:24.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.146 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.146 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.860993+0000", 2024-09-16T20:50:24.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8949999999999997e-06 2024-09-16T20:50:24.147 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.147 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.147 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.147 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.147 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5111 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.147 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.878157+0000", 2024-09-16T20:50:24.147 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.010300429, 2024-09-16T20:50:24.147 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.012752553999999999, 2024-09-16T20:50:24.147 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.147 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.147 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.147 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.147 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.148 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.148 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5111 2024-09-16T20:50:24.148 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.148 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.148 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.148 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.148 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.878157+0000", 2024-09-16T20:50:24.148 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.148 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.148 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.148 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.148 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.878157+0000", 2024-09-16T20:50:24.148 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.149 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.149 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.149 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.878158+0000", 2024-09-16T20:50:24.149 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2600000000000001e-07 2024-09-16T20:50:24.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.149 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.149 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.149 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.878162+0000", 2024-09-16T20:50:24.149 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.89e-06 2024-09-16T20:50:24.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.149 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.149 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.878173+0000", 2024-09-16T20:50:24.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1163e-05 2024-09-16T20:50:24.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.150 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.150 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.878180+0000", 2024-09-16T20:50:24.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9789999999999996e-06 2024-09-16T20:50:24.150 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.150 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.150 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.150 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.150 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5284 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.150 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.888403+0000", 2024-09-16T20:50:24.151 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.4728000000000002e-05, 2024-09-16T20:50:24.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0025404870000000001, 2024-09-16T20:50:24.151 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.151 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.151 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.151 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.151 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.151 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.151 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5284 2024-09-16T20:50:24.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.151 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888403+0000", 2024-09-16T20:50:24.152 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.152 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.152 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.152 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888403+0000", 2024-09-16T20:50:24.152 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.152 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.152 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.152 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888403+0000", 2024-09-16T20:50:24.152 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1300000000000001e-07 2024-09-16T20:50:24.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.152 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.153 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.153 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888406+0000", 2024-09-16T20:50:24.153 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6960000000000001e-06 2024-09-16T20:50:24.153 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.153 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.153 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.153 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888418+0000", 2024-09-16T20:50:24.153 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2585999999999999e-05 2024-09-16T20:50:24.153 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.153 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.153 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.153 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888425+0000", 2024-09-16T20:50:24.153 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1500000000000004e-06 2024-09-16T20:50:24.154 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.154 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.154 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.154 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.154 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.154 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:4669 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.154 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.861000+0000", 2024-09-16T20:50:24.154 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.027457221, 2024-09-16T20:50:24.154 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.029970501, 2024-09-16T20:50:24.154 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.154 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.154 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.154 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.155 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.155 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.155 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4669 2024-09-16T20:50:24.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.155 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.155 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.155 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.155 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.861000+0000", 2024-09-16T20:50:24.155 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.155 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.155 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.155 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.861000+0000", 2024-09-16T20:50:24.155 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.156 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.156 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.156 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.861001+0000", 2024-09-16T20:50:24.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1699999999999999e-07 2024-09-16T20:50:24.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.156 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.156 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.156 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.861003+0000", 2024-09-16T20:50:24.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2290000000000002e-06 2024-09-16T20:50:24.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.156 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.156 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.157 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.861010+0000", 2024-09-16T20:50:24.157 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3069999999999997e-06 2024-09-16T20:50:24.157 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.157 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.157 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.157 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.861016+0000", 2024-09-16T20:50:24.157 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7620000000000001e-06 2024-09-16T20:50:24.157 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.157 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.157 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.157 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.157 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.157 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5112 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.157 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.878190+0000", 2024-09-16T20:50:24.158 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.010266979000000001, 2024-09-16T20:50:24.158 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.012803045000000001, 2024-09-16T20:50:24.158 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.158 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.158 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.158 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.158 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.158 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.158 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5112 2024-09-16T20:50:24.158 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.158 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.158 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.158 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.159 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.878190+0000", 2024-09-16T20:50:24.159 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.159 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.159 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.159 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.878190+0000", 2024-09-16T20:50:24.159 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.159 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.159 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.159 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.878191+0000", 2024-09-16T20:50:24.159 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2399999999999998e-07 2024-09-16T20:50:24.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.159 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.160 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.160 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.878193+0000", 2024-09-16T20:50:24.160 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3659999999999999e-06 2024-09-16T20:50:24.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.160 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.160 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.160 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.878201+0000", 2024-09-16T20:50:24.160 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3989999999999998e-06 2024-09-16T20:50:24.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.160 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.160 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.160 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.878206+0000", 2024-09-16T20:50:24.160 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5960000000000003e-06 2024-09-16T20:50:24.161 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.161 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.161 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.161 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.161 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.161 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5285 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.161 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.888481+0000", 2024-09-16T20:50:24.161 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4294967295.9999762, 2024-09-16T20:50:24.161 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0025372760000000002, 2024-09-16T20:50:24.161 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.161 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.161 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.161 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.162 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.162 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.162 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5285 2024-09-16T20:50:24.162 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.162 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.162 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.162 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.162 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888481+0000", 2024-09-16T20:50:24.162 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.162 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.162 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.162 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.162 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888481+0000", 2024-09-16T20:50:24.162 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.163 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.163 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.163 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888482+0000", 2024-09-16T20:50:24.163 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5600000000000005e-07 2024-09-16T20:50:24.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.163 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.163 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.163 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888486+0000", 2024-09-16T20:50:24.163 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5090000000000002e-06 2024-09-16T20:50:24.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.163 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.163 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.163 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888496+0000", 2024-09-16T20:50:24.164 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0522e-05 2024-09-16T20:50:24.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.164 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.164 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.164 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888503+0000", 2024-09-16T20:50:24.164 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4339999999999997e-06 2024-09-16T20:50:24.164 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.164 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.164 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.164 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.164 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:4671 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.164 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.861024+0000", 2024-09-16T20:50:24.165 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.027433723, 2024-09-16T20:50:24.165 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.030019191000000001, 2024-09-16T20:50:24.165 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.165 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.165 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.165 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.165 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.165 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.165 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4671 2024-09-16T20:50:24.165 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.165 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.165 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.165 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.166 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.861024+0000", 2024-09-16T20:50:24.166 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.166 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.166 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.166 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.861024+0000", 2024-09-16T20:50:24.166 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.166 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.166 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.166 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.861024+0000", 2024-09-16T20:50:24.166 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2599999999999998e-07 2024-09-16T20:50:24.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.166 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.167 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.167 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.861026+0000", 2024-09-16T20:50:24.167 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2050000000000001e-06 2024-09-16T20:50:24.167 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.167 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.167 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.167 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.861033+0000", 2024-09-16T20:50:24.167 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4749999999999998e-06 2024-09-16T20:50:24.167 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.167 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.167 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.167 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.861038+0000", 2024-09-16T20:50:24.167 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8550000000000001e-06 2024-09-16T20:50:24.167 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.168 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.168 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.168 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.168 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5113 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.168 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.878214+0000", 2024-09-16T20:50:24.168 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.010243808, 2024-09-16T20:50:24.168 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.012854835, 2024-09-16T20:50:24.168 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.168 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.168 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.168 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.168 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.168 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.169 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5113 2024-09-16T20:50:24.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.169 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.169 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.169 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.169 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.878214+0000", 2024-09-16T20:50:24.169 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.169 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.169 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.169 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.878214+0000", 2024-09-16T20:50:24.169 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.169 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.170 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.170 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.878214+0000", 2024-09-16T20:50:24.170 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8800000000000003e-07 2024-09-16T20:50:24.170 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.170 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.170 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.170 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.878216+0000", 2024-09-16T20:50:24.170 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0810000000000001e-06 2024-09-16T20:50:24.170 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.170 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.170 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.170 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.878223+0000", 2024-09-16T20:50:24.170 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6259999999999997e-06 2024-09-16T20:50:24.170 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.171 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.171 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.171 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.878228+0000", 2024-09-16T20:50:24.171 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.322e-06 2024-09-16T20:50:24.171 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.171 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.171 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.171 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.171 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.171 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5287 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.171 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.888547+0000", 2024-09-16T20:50:24.171 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4294967295.9999104, 2024-09-16T20:50:24.171 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0025450239999999999, 2024-09-16T20:50:24.171 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.172 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.172 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.172 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.172 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.172 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.172 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5287 2024-09-16T20:50:24.172 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.172 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.172 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.172 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.172 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888547+0000", 2024-09-16T20:50:24.172 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.172 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.173 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.173 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.173 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888547+0000", 2024-09-16T20:50:24.173 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.173 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.173 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.173 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.173 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888548+0000", 2024-09-16T20:50:24.173 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0699999999999997e-07 2024-09-16T20:50:24.173 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.173 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.173 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.173 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888551+0000", 2024-09-16T20:50:24.173 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1389999999999999e-06 2024-09-16T20:50:24.173 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.174 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.174 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.174 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888559+0000", 2024-09-16T20:50:24.174 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4120000000000001e-06 2024-09-16T20:50:24.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.174 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.174 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.174 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888565+0000", 2024-09-16T20:50:24.174 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2430000000000002e-06 2024-09-16T20:50:24.174 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.174 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.174 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.175 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.175 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:4674 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.175 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.861046+0000", 2024-09-16T20:50:24.175 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.027411715999999999, 2024-09-16T20:50:24.175 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.030070880000000001, 2024-09-16T20:50:24.175 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.175 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.175 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.175 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.175 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.175 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.175 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4674 2024-09-16T20:50:24.175 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.175 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.176 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.176 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.176 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.861046+0000", 2024-09-16T20:50:24.176 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.176 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.176 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.176 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.176 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.861046+0000", 2024-09-16T20:50:24.176 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.176 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.176 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.176 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.176 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.861046+0000", 2024-09-16T20:50:24.176 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.13e-07 2024-09-16T20:50:24.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.177 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.177 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.177 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.861048+0000", 2024-09-16T20:50:24.177 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0360000000000001e-06 2024-09-16T20:50:24.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.177 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.177 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.177 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.861055+0000", 2024-09-16T20:50:24.177 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2809999999999997e-06 2024-09-16T20:50:24.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.177 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.177 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.177 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.861060+0000", 2024-09-16T20:50:24.178 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3310000000000003e-06 2024-09-16T20:50:24.178 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.178 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.178 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.178 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.178 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5114 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.178 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.878235+0000", 2024-09-16T20:50:24.178 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.010222325000000001, 2024-09-16T20:50:24.178 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.01291632, 2024-09-16T20:50:24.178 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.178 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.178 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.178 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.179 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.179 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.179 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5114 2024-09-16T20:50:24.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.179 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.179 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.179 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.179 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.878235+0000", 2024-09-16T20:50:24.179 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.179 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.179 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.179 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.878235+0000", 2024-09-16T20:50:24.180 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.180 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.180 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.180 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.180 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.878235+0000", 2024-09-16T20:50:24.180 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2199999999999999e-07 2024-09-16T20:50:24.180 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.180 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.180 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.180 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.878238+0000", 2024-09-16T20:50:24.180 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1289999999999999e-06 2024-09-16T20:50:24.180 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.180 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.180 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.181 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.878246+0000", 2024-09-16T20:50:24.181 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5639999999999997e-06 2024-09-16T20:50:24.181 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.181 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.181 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.181 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.878252+0000", 2024-09-16T20:50:24.181 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3469999999999998e-06 2024-09-16T20:50:24.181 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.181 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.181 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.181 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.181 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.181 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5289 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.182 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.888660+0000", 2024-09-16T20:50:24.182 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4294967295.9997973, 2024-09-16T20:50:24.182 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0025186499999999999, 2024-09-16T20:50:24.182 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.182 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.182 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.182 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.182 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.182 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.182 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5289 2024-09-16T20:50:24.182 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.182 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.182 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.182 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.183 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888660+0000", 2024-09-16T20:50:24.183 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.183 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.183 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.183 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.183 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888660+0000", 2024-09-16T20:50:24.183 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.183 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.183 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.183 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.183 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888661+0000", 2024-09-16T20:50:24.183 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0999999999999997e-07 2024-09-16T20:50:24.183 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.183 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.184 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.184 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888664+0000", 2024-09-16T20:50:24.184 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1549999999999999e-06 2024-09-16T20:50:24.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.184 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.184 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.184 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888673+0000", 2024-09-16T20:50:24.184 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5890000000000003e-06 2024-09-16T20:50:24.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.184 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.184 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.184 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888679+0000", 2024-09-16T20:50:24.184 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2149999999999998e-06 2024-09-16T20:50:24.184 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.185 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.185 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.185 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.185 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:4676 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.185 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.861066+0000", 2024-09-16T20:50:24.185 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.027390984, 2024-09-16T20:50:24.185 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.030136109000000001, 2024-09-16T20:50:24.185 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.185 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.185 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.185 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.185 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.186 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.186 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4676 2024-09-16T20:50:24.186 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.186 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.186 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.186 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.186 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.861066+0000", 2024-09-16T20:50:24.186 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.186 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.186 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.186 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.186 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.861066+0000", 2024-09-16T20:50:24.186 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.186 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.187 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.187 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.187 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.861067+0000", 2024-09-16T20:50:24.187 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.39e-07 2024-09-16T20:50:24.187 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.187 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.187 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.187 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.861069+0000", 2024-09-16T20:50:24.187 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2409999999999998e-06 2024-09-16T20:50:24.187 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.187 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.187 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.187 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.861076+0000", 2024-09-16T20:50:24.187 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4359999999999999e-06 2024-09-16T20:50:24.188 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.188 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.188 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.188 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.861081+0000", 2024-09-16T20:50:24.188 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8839999999999999e-06 2024-09-16T20:50:24.188 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.188 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.188 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.188 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.188 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.188 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5115 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.188 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.878261+0000", 2024-09-16T20:50:24.188 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.010195938999999999, 2024-09-16T20:50:24.189 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.012965598, 2024-09-16T20:50:24.189 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.189 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.189 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.189 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.189 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.189 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.189 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5115 2024-09-16T20:50:24.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.189 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.189 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.189 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.189 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.878261+0000", 2024-09-16T20:50:24.190 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.190 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.190 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.190 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.878261+0000", 2024-09-16T20:50:24.190 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.190 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.190 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.190 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.878262+0000", 2024-09-16T20:50:24.190 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3399999999999999e-07 2024-09-16T20:50:24.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.190 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.190 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.191 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.878264+0000", 2024-09-16T20:50:24.191 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1919999999999999e-06 2024-09-16T20:50:24.191 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.191 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.191 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.191 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.878271+0000", 2024-09-16T20:50:24.191 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8299999999999998e-06 2024-09-16T20:50:24.191 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.191 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.191 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.191 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.878276+0000", 2024-09-16T20:50:24.191 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.383e-06 2024-09-16T20:50:24.191 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.192 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.192 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.192 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.192 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.192 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5291 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.192 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.888738+0000", 2024-09-16T20:50:24.192 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4294967295.9997196, 2024-09-16T20:50:24.192 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0025233399999999998, 2024-09-16T20:50:24.192 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.192 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.192 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.192 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.192 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.193 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.193 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5291 2024-09-16T20:50:24.193 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.193 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.193 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.193 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.193 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888738+0000", 2024-09-16T20:50:24.193 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.193 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.193 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.193 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.193 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888738+0000", 2024-09-16T20:50:24.193 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.193 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.194 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.194 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.194 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888738+0000", 2024-09-16T20:50:24.194 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9599999999999999e-07 2024-09-16T20:50:24.194 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.194 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.194 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.194 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888741+0000", 2024-09-16T20:50:24.194 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0410000000000002e-06 2024-09-16T20:50:24.194 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.194 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.194 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.194 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888749+0000", 2024-09-16T20:50:24.194 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.435e-06 2024-09-16T20:50:24.195 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.195 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.195 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.195 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888755+0000", 2024-09-16T20:50:24.195 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7459999999999997e-06 2024-09-16T20:50:24.195 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.195 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.195 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.195 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.195 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.195 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:4679 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.195 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.861088+0000", 2024-09-16T20:50:24.195 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.027369289000000002, 2024-09-16T20:50:24.196 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.030196466000000002, 2024-09-16T20:50:24.196 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.196 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.196 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.196 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.196 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.196 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.196 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4679 2024-09-16T20:50:24.196 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.196 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.196 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.196 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.196 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.861088+0000", 2024-09-16T20:50:24.196 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.197 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.197 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.197 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.197 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.861088+0000", 2024-09-16T20:50:24.197 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.197 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.197 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.197 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.197 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.861088+0000", 2024-09-16T20:50:24.197 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9299999999999999e-07 2024-09-16T20:50:24.197 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.197 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.197 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.198 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.861091+0000", 2024-09-16T20:50:24.198 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1780000000000002e-06 2024-09-16T20:50:24.198 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.198 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.198 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.198 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.861097+0000", 2024-09-16T20:50:24.198 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.28e-06 2024-09-16T20:50:24.198 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.198 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.198 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.198 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.861102+0000", 2024-09-16T20:50:24.198 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9389999999999996e-06 2024-09-16T20:50:24.198 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.198 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.199 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.199 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.199 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5117 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.199 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.878321+0000", 2024-09-16T20:50:24.199 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.010136783999999999, 2024-09-16T20:50:24.199 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.012988842, 2024-09-16T20:50:24.199 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.199 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.199 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.199 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.199 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.199 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.199 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5117 2024-09-16T20:50:24.200 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.200 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.200 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.200 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.200 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.878321+0000", 2024-09-16T20:50:24.200 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.200 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.200 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.200 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.200 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.878321+0000", 2024-09-16T20:50:24.200 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.200 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.200 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.200 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.201 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.878321+0000", 2024-09-16T20:50:24.201 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2500000000000005e-07 2024-09-16T20:50:24.201 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.201 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.201 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.201 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.878325+0000", 2024-09-16T20:50:24.201 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.765e-06 2024-09-16T20:50:24.201 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.201 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.201 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.201 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.878335+0000", 2024-09-16T20:50:24.201 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0159e-05 2024-09-16T20:50:24.201 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.201 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.202 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.202 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.878341+0000", 2024-09-16T20:50:24.202 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2199999999999997e-06 2024-09-16T20:50:24.202 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.202 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.202 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.202 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.202 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.202 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5293 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.202 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.888877+0000", 2024-09-16T20:50:24.202 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4294967295.9995809, 2024-09-16T20:50:24.202 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0024620499999999999, 2024-09-16T20:50:24.202 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.202 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.203 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.203 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.203 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.203 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.203 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5293 2024-09-16T20:50:24.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.203 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.203 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.203 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.203 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888877+0000", 2024-09-16T20:50:24.203 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.203 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.203 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.204 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888877+0000", 2024-09-16T20:50:24.204 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.204 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.204 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.204 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.204 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888877+0000", 2024-09-16T20:50:24.204 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5799999999999998e-07 2024-09-16T20:50:24.204 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.204 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.204 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.204 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888880+0000", 2024-09-16T20:50:24.204 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.118e-06 2024-09-16T20:50:24.204 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.205 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.205 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.205 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888889+0000", 2024-09-16T20:50:24.205 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0600000000000008e-06 2024-09-16T20:50:24.205 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.205 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.205 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.205 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888894+0000", 2024-09-16T20:50:24.205 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9089999999999997e-06 2024-09-16T20:50:24.205 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.205 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.205 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.205 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.205 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.206 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5059 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.206 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.875040+0000", 2024-09-16T20:50:24.206 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.013417597, 2024-09-16T20:50:24.206 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.016323312999999999, 2024-09-16T20:50:24.206 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.206 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.206 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.206 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.206 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.206 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.206 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5059 2024-09-16T20:50:24.206 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.206 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.206 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.207 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.207 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.875040+0000", 2024-09-16T20:50:24.207 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.207 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.207 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.207 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.207 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.875040+0000", 2024-09-16T20:50:24.207 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.207 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.207 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.207 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.207 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.875041+0000", 2024-09-16T20:50:24.207 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4160000000000001e-06 2024-09-16T20:50:24.207 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.208 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.208 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.208 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.875047+0000", 2024-09-16T20:50:24.208 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0809999999999999e-06 2024-09-16T20:50:24.208 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.208 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.208 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.208 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.875061+0000", 2024-09-16T20:50:24.208 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3383999999999999e-05 2024-09-16T20:50:24.208 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.208 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.208 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.208 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.875070+0000", 2024-09-16T20:50:24.208 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1339999999999993e-06 2024-09-16T20:50:24.209 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.209 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.209 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.209 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.209 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5250 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.209 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.886555+0000", 2024-09-16T20:50:24.209 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.001902118, 2024-09-16T20:50:24.209 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0048307410000000004, 2024-09-16T20:50:24.209 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.209 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.209 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.209 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.210 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.210 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.210 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5250 2024-09-16T20:50:24.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.210 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.210 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.210 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.210 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.886555+0000", 2024-09-16T20:50:24.210 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.210 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.210 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.210 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.886555+0000", 2024-09-16T20:50:24.210 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.211 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.211 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.211 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.211 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.886556+0000", 2024-09-16T20:50:24.211 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8899999999999998e-07 2024-09-16T20:50:24.211 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.211 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.211 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.211 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.886561+0000", 2024-09-16T20:50:24.211 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4599999999999996e-06 2024-09-16T20:50:24.211 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.211 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.211 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.211 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.886575+0000", 2024-09-16T20:50:24.212 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4759e-05 2024-09-16T20:50:24.212 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.212 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.212 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.212 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.886584+0000", 2024-09-16T20:50:24.212 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0239999999999997e-06 2024-09-16T20:50:24.212 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.212 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.212 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.212 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.212 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.212 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5294 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.212 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.888972+0000", 2024-09-16T20:50:24.213 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4294967295.9994855, 2024-09-16T20:50:24.213 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0024385510000000002, 2024-09-16T20:50:24.213 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.213 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.213 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.213 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.213 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.213 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.213 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5294 2024-09-16T20:50:24.213 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.213 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.213 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.213 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.213 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888972+0000", 2024-09-16T20:50:24.214 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.214 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.214 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.214 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.214 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888972+0000", 2024-09-16T20:50:24.214 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.214 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.214 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.214 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.214 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888973+0000", 2024-09-16T20:50:24.214 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5499999999999997e-07 2024-09-16T20:50:24.214 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.214 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.214 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.215 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888976+0000", 2024-09-16T20:50:24.215 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0350000000000002e-06 2024-09-16T20:50:24.215 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.215 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.215 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.215 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888984+0000", 2024-09-16T20:50:24.215 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0739999999999993e-06 2024-09-16T20:50:24.215 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.215 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.215 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.215 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.888990+0000", 2024-09-16T20:50:24.215 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8830000000000003e-06 2024-09-16T20:50:24.215 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.215 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.216 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.216 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.216 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.216 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5061 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.216 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.875166+0000", 2024-09-16T20:50:24.216 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.013291707999999999, 2024-09-16T20:50:24.216 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.016267818999999999, 2024-09-16T20:50:24.216 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.216 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.216 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.216 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.216 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.216 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.216 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5061 2024-09-16T20:50:24.217 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.217 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.217 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.217 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.217 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.875166+0000", 2024-09-16T20:50:24.217 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.217 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.217 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.217 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.217 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.875166+0000", 2024-09-16T20:50:24.217 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.217 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.217 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.217 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.218 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.875166+0000", 2024-09-16T20:50:24.218 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0999999999999997e-07 2024-09-16T20:50:24.218 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.218 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.218 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.218 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.875170+0000", 2024-09-16T20:50:24.218 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6880000000000001e-06 2024-09-16T20:50:24.218 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.218 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.218 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.218 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.875180+0000", 2024-09-16T20:50:24.218 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0030999999999999e-05 2024-09-16T20:50:24.218 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.219 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.219 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.219 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.875189+0000", 2024-09-16T20:50:24.219 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7709999999999995e-06 2024-09-16T20:50:24.219 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.219 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.219 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.219 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.219 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.219 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5252 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.219 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.886687+0000", 2024-09-16T20:50:24.219 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.001770277, 2024-09-16T20:50:24.219 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0047826980000000002, 2024-09-16T20:50:24.219 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.220 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.220 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.220 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.220 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.220 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.220 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5252 2024-09-16T20:50:24.220 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.220 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.220 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.220 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.220 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.886687+0000", 2024-09-16T20:50:24.220 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.220 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.220 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.221 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.221 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.886687+0000", 2024-09-16T20:50:24.221 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.221 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.221 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.221 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.221 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.886688+0000", 2024-09-16T20:50:24.221 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8899999999999998e-07 2024-09-16T20:50:24.221 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.221 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.221 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.221 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.886692+0000", 2024-09-16T20:50:24.221 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8070000000000001e-06 2024-09-16T20:50:24.221 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.222 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.222 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.222 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.886702+0000", 2024-09-16T20:50:24.222 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8600000000000005e-06 2024-09-16T20:50:24.222 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.222 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.222 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.222 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.886708+0000", 2024-09-16T20:50:24.222 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2639999999999997e-06 2024-09-16T20:50:24.222 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.222 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.222 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.222 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.222 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.223 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:5295 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:24.223 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:23.889056+0000", 2024-09-16T20:50:24.223 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4294967295.9994011, 2024-09-16T20:50:24.223 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0024384350000000001, 2024-09-16T20:50:24.223 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.223 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.223 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:24.223 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.223 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:24.223 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:24.223 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5295 2024-09-16T20:50:24.223 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.223 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.223 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.223 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.224 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.889056+0000", 2024-09-16T20:50:24.224 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.224 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.224 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.224 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.224 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.889056+0000", 2024-09-16T20:50:24.224 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.224 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.224 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.224 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.224 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.889057+0000", 2024-09-16T20:50:24.224 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.06e-07 2024-09-16T20:50:24.224 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.224 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.225 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.225 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.889060+0000", 2024-09-16T20:50:24.225 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.066e-06 2024-09-16T20:50:24.225 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.225 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.225 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.225 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.889068+0000", 2024-09-16T20:50:24.225 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7109999999999995e-06 2024-09-16T20:50:24.225 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.225 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.225 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.225 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:23.889073+0000", 2024-09-16T20:50:24.225 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7089999999999999e-06 2024-09-16T20:50:24.226 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.226 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.226 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.226 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.226 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-16T20:50:24.226 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 96 2024-09-16T20:50:24.226 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:24.227 INFO:tasks.ceph.mon.a.smithi029.stderr:2024-09-16T20:50:24.223+0000 7fe0ce96d640 -1 mon.a@0(leader).osd e137 definitely_dead 0 2024-09-16T20:50:24.228 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:50:24.228 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:24.226+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:24.328 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:24.327+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:24.367 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T20:50:24.375 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:24.375 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:50:24.375 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:50:24.375 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:50:24.375 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:24.383 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:50:24.429 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:24.428+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:24.501 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:50:24.502 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:50:24.520 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:50:24.520 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:50:24.529 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:24.528+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:24.533 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:24.533 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:50:24.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:50:24.534 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:50:24.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.534 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12548 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 628736~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:24.534 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757009+0000", 2024-09-16T20:50:24.534 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 33.774425026999999, 2024-09-16T20:50:24.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065743956000000006, 2024-09-16T20:50:24.534 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.534 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.534 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:24.534 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.534 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:24.534 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:24.535 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12548 2024-09-16T20:50:24.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.535 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757009+0000", 2024-09-16T20:50:24.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757009+0000", 2024-09-16T20:50:24.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.536 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757010+0000", 2024-09-16T20:50:24.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9900000000000001e-07 2024-09-16T20:50:24.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.536 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757012+0000", 2024-09-16T20:50:24.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2689999999999998e-06 2024-09-16T20:50:24.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.536 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757022+0000", 2024-09-16T20:50:24.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9699999999999994e-06 2024-09-16T20:50:24.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757028+0000", 2024-09-16T20:50:24.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3799999999999999e-06 2024-09-16T20:50:24.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:24.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822552+0000", 2024-09-16T20:50:24.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065524281000000004 2024-09-16T20:50:24.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.538 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:24.538 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822613+0000", 2024-09-16T20:50:24.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0430999999999998e-05 2024-09-16T20:50:24.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.538 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:24.538 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822753+0000", 2024-09-16T20:50:24.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000140226 2024-09-16T20:50:24.538 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.538 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.538 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.539 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12549 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 633856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:24.539 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:50:24.539 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 33.774396279999998, 2024-09-16T20:50:24.539 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065978518, 2024-09-16T20:50:24.539 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.539 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.539 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:24.539 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.539 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:24.539 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:24.539 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12549 2024-09-16T20:50:24.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.539 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:50:24.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:50:24.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.541 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:50:24.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9299999999999999e-07 2024-09-16T20:50:24.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.541 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757040+0000", 2024-09-16T20:50:24.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.677e-06 2024-09-16T20:50:24.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.541 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757045+0000", 2024-09-16T20:50:24.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8149999999999996e-06 2024-09-16T20:50:24.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.542 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757048+0000", 2024-09-16T20:50:24.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1e-06 2024-09-16T20:50:24.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:24.542 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822796+0000", 2024-09-16T20:50:24.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065748081 2024-09-16T20:50:24.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:24.542 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822867+0000", 2024-09-16T20:50:24.543 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0679000000000005e-05 2024-09-16T20:50:24.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.543 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.543 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:24.543 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823016+0000", 2024-09-16T20:50:24.543 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014987299999999999 2024-09-16T20:50:24.543 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.543 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.543 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.543 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.543 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12550 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 638976~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:24.543 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757060+0000", 2024-09-16T20:50:24.543 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 33.774373865000001, 2024-09-16T20:50:24.543 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066153564999999998, 2024-09-16T20:50:24.544 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.544 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.544 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:24.544 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.544 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:24.544 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:24.544 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12550 2024-09-16T20:50:24.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.544 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.544 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.544 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757060+0000", 2024-09-16T20:50:24.544 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.545 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.545 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757060+0000", 2024-09-16T20:50:24.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.545 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.545 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757061+0000", 2024-09-16T20:50:24.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.15e-07 2024-09-16T20:50:24.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.545 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.545 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757062+0000", 2024-09-16T20:50:24.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.663e-06 2024-09-16T20:50:24.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757066+0000", 2024-09-16T20:50:24.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.817e-06 2024-09-16T20:50:24.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757070+0000", 2024-09-16T20:50:24.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1509999999999997e-06 2024-09-16T20:50:24.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.547 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:24.547 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823044+0000", 2024-09-16T20:50:24.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065973992999999995 2024-09-16T20:50:24.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.547 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:24.547 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823082+0000", 2024-09-16T20:50:24.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8050999999999999e-05 2024-09-16T20:50:24.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.547 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:24.547 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823214+0000", 2024-09-16T20:50:24.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013157500000000001 2024-09-16T20:50:24.548 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.548 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.548 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.548 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.548 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12551 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 644096~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:24.548 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:50:24.548 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 33.774360166999998, 2024-09-16T20:50:24.548 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066391378000000001, 2024-09-16T20:50:24.548 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.548 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.548 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:24.549 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.549 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:24.549 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:24.549 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12551 2024-09-16T20:50:24.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.549 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.549 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.549 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:50:24.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.549 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.550 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:50:24.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.550 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.550 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:50:24.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2999999999999999e-07 2024-09-16T20:50:24.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.550 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.550 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757075+0000", 2024-09-16T20:50:24.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.097e-06 2024-09-16T20:50:24.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.551 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.551 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757082+0000", 2024-09-16T20:50:24.551 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9500000000000004e-06 2024-09-16T20:50:24.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.551 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.551 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757086+0000", 2024-09-16T20:50:24.551 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2229999999999999e-06 2024-09-16T20:50:24.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.551 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:24.551 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823262+0000", 2024-09-16T20:50:24.552 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066176078999999999 2024-09-16T20:50:24.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.552 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.552 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:24.552 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823336+0000", 2024-09-16T20:50:24.552 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3979000000000004e-05 2024-09-16T20:50:24.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.552 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.552 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:24.552 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823465+0000", 2024-09-16T20:50:24.552 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012982 2024-09-16T20:50:24.552 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.553 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.553 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.553 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12552 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 649216~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:24.553 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757089+0000", 2024-09-16T20:50:24.553 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 33.774345025999999, 2024-09-16T20:50:24.553 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066655617, 2024-09-16T20:50:24.553 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.553 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.553 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:24.553 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.553 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:24.554 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:24.554 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12552 2024-09-16T20:50:24.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.554 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.554 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.554 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.554 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757089+0000", 2024-09-16T20:50:24.554 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.554 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.554 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.554 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757089+0000", 2024-09-16T20:50:24.554 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759492+0000", 2024-09-16T20:50:24.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0024026109999999998 2024-09-16T20:50:24.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759497+0000", 2024-09-16T20:50:24.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1120000000000004e-06 2024-09-16T20:50:24.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759508+0000", 2024-09-16T20:50:24.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0981e-05 2024-09-16T20:50:24.556 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.556 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.556 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.556 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759516+0000", 2024-09-16T20:50:24.556 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7270000000000007e-06 2024-09-16T20:50:24.556 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.556 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.556 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:24.556 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823504+0000", 2024-09-16T20:50:24.556 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.063987984999999997 2024-09-16T20:50:24.556 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.556 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.556 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:24.557 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823579+0000", 2024-09-16T20:50:24.557 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5703000000000003e-05 2024-09-16T20:50:24.557 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.557 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.557 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:24.557 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823745+0000", 2024-09-16T20:50:24.557 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016549800000000001 2024-09-16T20:50:24.557 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.557 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.557 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.557 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.557 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.557 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12560 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 690176~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:24.558 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.759658+0000", 2024-09-16T20:50:24.558 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 33.771775824000002, 2024-09-16T20:50:24.558 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065858517000000005, 2024-09-16T20:50:24.558 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.558 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.558 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:24.558 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.558 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:24.558 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:24.558 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12560 2024-09-16T20:50:24.558 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.558 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.558 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.559 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759658+0000", 2024-09-16T20:50:24.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.559 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759658+0000", 2024-09-16T20:50:24.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.559 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759659+0000", 2024-09-16T20:50:24.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.17e-07 2024-09-16T20:50:24.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.560 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.560 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759660+0000", 2024-09-16T20:50:24.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0189999999999999e-06 2024-09-16T20:50:24.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.560 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.560 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759663+0000", 2024-09-16T20:50:24.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0010000000000002e-06 2024-09-16T20:50:24.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.560 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.561 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759665+0000", 2024-09-16T20:50:24.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5440000000000001e-06 2024-09-16T20:50:24.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.561 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.561 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:24.561 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825355+0000", 2024-09-16T20:50:24.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065690224000000005 2024-09-16T20:50:24.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.561 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.561 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:24.561 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825394+0000", 2024-09-16T20:50:24.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8810999999999998e-05 2024-09-16T20:50:24.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.561 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.562 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:24.562 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825517+0000", 2024-09-16T20:50:24.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012270099999999999 2024-09-16T20:50:24.562 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.562 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.562 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.562 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12561 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 695296~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:24.562 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.759668+0000", 2024-09-16T20:50:24.562 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 33.771765981999998, 2024-09-16T20:50:24.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066116999999999995, 2024-09-16T20:50:24.562 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.563 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.563 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:24.563 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.563 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:24.563 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:24.563 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12561 2024-09-16T20:50:24.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.563 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.563 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.563 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759668+0000", 2024-09-16T20:50:24.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759668+0000", 2024-09-16T20:50:24.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765379+0000", 2024-09-16T20:50:24.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057104410000000001 2024-09-16T20:50:24.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.565 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.565 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765383+0000", 2024-09-16T20:50:24.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2980000000000002e-06 2024-09-16T20:50:24.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.565 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.565 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765395+0000", 2024-09-16T20:50:24.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1671000000000001e-05 2024-09-16T20:50:24.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.565 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.565 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765403+0000", 2024-09-16T20:50:24.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1210000000000007e-06 2024-09-16T20:50:24.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.566 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.566 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:24.566 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825552+0000", 2024-09-16T20:50:24.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.060149498000000003 2024-09-16T20:50:24.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.566 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.566 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:24.566 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825625+0000", 2024-09-16T20:50:24.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2540999999999995e-05 2024-09-16T20:50:24.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.566 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.567 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:24.567 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825785+0000", 2024-09-16T20:50:24.567 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016043000000000001 2024-09-16T20:50:24.567 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.567 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.567 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.567 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12585 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 818176~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:24.567 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.765848+0000", 2024-09-16T20:50:24.567 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 33.765586011000003, 2024-09-16T20:50:24.567 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065843218999999995, 2024-09-16T20:50:24.567 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.568 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.568 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:24.568 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.568 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:24.568 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:24.568 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12585 2024-09-16T20:50:24.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.568 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.568 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.568 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765848+0000", 2024-09-16T20:50:24.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.569 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.569 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765848+0000", 2024-09-16T20:50:24.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.569 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.569 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768683+0000", 2024-09-16T20:50:24.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0028344519999999999 2024-09-16T20:50:24.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.569 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.570 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768686+0000", 2024-09-16T20:50:24.570 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8789999999999996e-06 2024-09-16T20:50:24.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.570 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.570 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768696+0000", 2024-09-16T20:50:24.570 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6709999999999994e-06 2024-09-16T20:50:24.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.570 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.570 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768702+0000", 2024-09-16T20:50:24.570 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2550000000000003e-06 2024-09-16T20:50:24.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.571 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.571 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:24.571 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.831499+0000", 2024-09-16T20:50:24.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.062797081000000005 2024-09-16T20:50:24.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.571 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.571 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:24.571 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.831538+0000", 2024-09-16T20:50:24.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8523999999999999e-05 2024-09-16T20:50:24.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.571 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.571 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:24.571 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.831691+0000", 2024-09-16T20:50:24.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015335699999999999 2024-09-16T20:50:24.572 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.572 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.572 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.572 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12597 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 879616~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:24.572 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.768883+0000", 2024-09-16T20:50:24.572 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 33.762551283000001, 2024-09-16T20:50:24.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065816527, 2024-09-16T20:50:24.572 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.572 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.572 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:24.573 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.573 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:24.573 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:24.573 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12597 2024-09-16T20:50:24.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.573 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.573 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.573 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.573 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768883+0000", 2024-09-16T20:50:24.573 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.573 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.573 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.573 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768883+0000", 2024-09-16T20:50:24.574 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.574 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.574 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771508+0000", 2024-09-16T20:50:24.574 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0026247589999999999 2024-09-16T20:50:24.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.574 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.574 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771513+0000", 2024-09-16T20:50:24.574 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9949999999999996e-06 2024-09-16T20:50:24.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.574 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.574 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771525+0000", 2024-09-16T20:50:24.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2033e-05 2024-09-16T20:50:24.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.575 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771532+0000", 2024-09-16T20:50:24.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4220000000000003e-06 2024-09-16T20:50:24.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.575 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:24.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.834532+0000", 2024-09-16T20:50:24.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.063000159 2024-09-16T20:50:24.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:24.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.834590+0000", 2024-09-16T20:50:24.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8218999999999999e-05 2024-09-16T20:50:24.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:24.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.834699+0000", 2024-09-16T20:50:24.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010894 2024-09-16T20:50:24.576 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.576 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.576 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.577 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12619 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 992256~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:24.577 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.774656+0000", 2024-09-16T20:50:24.577 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 33.756778509, 2024-09-16T20:50:24.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065715859000000001, 2024-09-16T20:50:24.577 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.577 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.577 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:24.577 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.577 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:24.577 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:24.577 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12619 2024-09-16T20:50:24.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.577 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.578 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.578 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774656+0000", 2024-09-16T20:50:24.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.578 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.578 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774656+0000", 2024-09-16T20:50:24.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.578 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.578 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774875+0000", 2024-09-16T20:50:24.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000218898 2024-09-16T20:50:24.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.579 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774879+0000", 2024-09-16T20:50:24.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0389999999999998e-06 2024-09-16T20:50:24.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.579 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774890+0000", 2024-09-16T20:50:24.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1250000000000001e-05 2024-09-16T20:50:24.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774897+0000", 2024-09-16T20:50:24.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7619999999999998e-06 2024-09-16T20:50:24.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:24.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840139+0000", 2024-09-16T20:50:24.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065242438 2024-09-16T20:50:24.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:24.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840186+0000", 2024-09-16T20:50:24.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6894999999999997e-05 2024-09-16T20:50:24.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:24.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840372+0000", 2024-09-16T20:50:24.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000185577 2024-09-16T20:50:24.581 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.581 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.581 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.581 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12620 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 997376~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:24.581 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.774908+0000", 2024-09-16T20:50:24.581 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 33.756526288000003, 2024-09-16T20:50:24.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065782754999999998, 2024-09-16T20:50:24.581 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.582 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.582 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:24.582 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.582 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:24.582 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:24.582 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12620 2024-09-16T20:50:24.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.582 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774908+0000", 2024-09-16T20:50:24.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774908+0000", 2024-09-16T20:50:24.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775102+0000", 2024-09-16T20:50:24.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000194098 2024-09-16T20:50:24.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775105+0000", 2024-09-16T20:50:24.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8739999999999999e-06 2024-09-16T20:50:24.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.584 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.584 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.584 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775111+0000", 2024-09-16T20:50:24.584 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9519999999999999e-06 2024-09-16T20:50:24.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.584 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.584 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.584 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775115+0000", 2024-09-16T20:50:24.584 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.002e-06 2024-09-16T20:50:24.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.584 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.584 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:24.584 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840409+0000", 2024-09-16T20:50:24.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065294195999999999 2024-09-16T20:50:24.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:24.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840507+0000", 2024-09-16T20:50:24.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7662000000000001e-05 2024-09-16T20:50:24.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:24.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840691+0000", 2024-09-16T20:50:24.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000183971 2024-09-16T20:50:24.585 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.585 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.586 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.586 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12621 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1002496~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:24.586 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775123+0000", 2024-09-16T20:50:24.586 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 33.756311592000003, 2024-09-16T20:50:24.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065829586999999995, 2024-09-16T20:50:24.586 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.586 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.586 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:24.586 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.586 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:24.586 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:24.586 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12621 2024-09-16T20:50:24.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.587 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775123+0000", 2024-09-16T20:50:24.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775123+0000", 2024-09-16T20:50:24.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775348+0000", 2024-09-16T20:50:24.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000225295 2024-09-16T20:50:24.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775352+0000", 2024-09-16T20:50:24.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0810000000000004e-06 2024-09-16T20:50:24.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775361+0000", 2024-09-16T20:50:24.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7399999999999993e-06 2024-09-16T20:50:24.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.589 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775367+0000", 2024-09-16T20:50:24.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5200000000000003e-06 2024-09-16T20:50:24.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:24.589 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840730+0000", 2024-09-16T20:50:24.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065362803999999997 2024-09-16T20:50:24.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:24.589 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840801+0000", 2024-09-16T20:50:24.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1006000000000004e-05 2024-09-16T20:50:24.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.590 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:24.590 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840952+0000", 2024-09-16T20:50:24.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015114100000000001 2024-09-16T20:50:24.590 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.590 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.590 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.590 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12622 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1007616~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:24.590 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775381+0000", 2024-09-16T20:50:24.590 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 33.756052758000003, 2024-09-16T20:50:24.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065822581000000005, 2024-09-16T20:50:24.591 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.591 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.591 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:24.591 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.591 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:24.591 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:24.591 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12622 2024-09-16T20:50:24.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.591 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.591 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.591 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775381+0000", 2024-09-16T20:50:24.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.592 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.592 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775381+0000", 2024-09-16T20:50:24.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.592 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.592 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775563+0000", 2024-09-16T20:50:24.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018153700000000001 2024-09-16T20:50:24.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.592 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.593 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775566+0000", 2024-09-16T20:50:24.593 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9620000000000002e-06 2024-09-16T20:50:24.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.593 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.593 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775572+0000", 2024-09-16T20:50:24.593 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8529999999999997e-06 2024-09-16T20:50:24.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.593 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.593 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775576+0000", 2024-09-16T20:50:24.593 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8920000000000002e-06 2024-09-16T20:50:24.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.594 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:24.594 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840990+0000", 2024-09-16T20:50:24.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065414599000000004 2024-09-16T20:50:24.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.594 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:24.594 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841068+0000", 2024-09-16T20:50:24.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8157000000000004e-05 2024-09-16T20:50:24.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.594 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:24.594 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841204+0000", 2024-09-16T20:50:24.595 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013558099999999999 2024-09-16T20:50:24.595 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.595 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.595 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.595 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12623 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1012736~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:24.595 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775583+0000", 2024-09-16T20:50:24.595 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 33.755851319999998, 2024-09-16T20:50:24.595 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065929087999999997, 2024-09-16T20:50:24.595 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.595 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.596 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:24.596 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.596 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:24.596 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:24.596 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12623 2024-09-16T20:50:24.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.596 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.596 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.596 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.596 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775583+0000", 2024-09-16T20:50:24.596 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.596 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.597 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.597 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775583+0000", 2024-09-16T20:50:24.597 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.597 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.597 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.597 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.597 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775804+0000", 2024-09-16T20:50:24.597 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00022116900000000001 2024-09-16T20:50:24.597 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.597 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.597 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.597 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775807+0000", 2024-09-16T20:50:24.597 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9529999999999999e-06 2024-09-16T20:50:24.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.598 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.598 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775814+0000", 2024-09-16T20:50:24.598 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7490000000000001e-06 2024-09-16T20:50:24.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.598 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.598 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775818+0000", 2024-09-16T20:50:24.598 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1999999999999996e-06 2024-09-16T20:50:24.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.598 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:24.598 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841267+0000", 2024-09-16T20:50:24.599 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065449127999999995 2024-09-16T20:50:24.599 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.599 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:24.599 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841344+0000", 2024-09-16T20:50:24.599 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6546999999999997e-05 2024-09-16T20:50:24.599 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.599 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:24.599 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841512+0000", 2024-09-16T20:50:24.599 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016834200000000001 2024-09-16T20:50:24.599 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.599 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.600 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.600 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.600 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12624 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1017856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:24.600 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:50:24.600 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 33.755608180000003, 2024-09-16T20:50:24.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065945866000000006, 2024-09-16T20:50:24.600 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.600 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.600 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:24.600 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.600 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:24.601 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:24.601 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12624 2024-09-16T20:50:24.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.601 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.601 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:50:24.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.601 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:50:24.602 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.602 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.602 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:50:24.602 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-16T20:50:24.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.602 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.602 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775828+0000", 2024-09-16T20:50:24.602 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3999999999999999e-06 2024-09-16T20:50:24.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.603 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.603 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775831+0000", 2024-09-16T20:50:24.603 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.478e-06 2024-09-16T20:50:24.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.603 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.603 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775834+0000", 2024-09-16T20:50:24.603 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9440000000000001e-06 2024-09-16T20:50:24.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.603 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:24.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841547+0000", 2024-09-16T20:50:24.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065712801000000001 2024-09-16T20:50:24.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.604 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:24.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841618+0000", 2024-09-16T20:50:24.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1151999999999999e-05 2024-09-16T20:50:24.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.604 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:24.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841772+0000", 2024-09-16T20:50:24.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000153721 2024-09-16T20:50:24.604 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.605 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.605 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.605 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12625 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1022976~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:24.605 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775840+0000", 2024-09-16T20:50:24.605 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 33.755594571000003, 2024-09-16T20:50:24.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066165451, 2024-09-16T20:50:24.605 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.605 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.605 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:24.605 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.606 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:24.606 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:24.606 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12625 2024-09-16T20:50:24.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.606 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.606 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.606 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775840+0000", 2024-09-16T20:50:24.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.606 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.606 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775840+0000", 2024-09-16T20:50:24.607 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.607 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.607 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776014+0000", 2024-09-16T20:50:24.607 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017441099999999999 2024-09-16T20:50:24.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.607 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.607 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776018+0000", 2024-09-16T20:50:24.607 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6509999999999999e-06 2024-09-16T20:50:24.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.608 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.608 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776026+0000", 2024-09-16T20:50:24.608 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.028e-06 2024-09-16T20:50:24.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.608 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.608 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776031+0000", 2024-09-16T20:50:24.608 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8400000000000002e-06 2024-09-16T20:50:24.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.608 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:24.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841801+0000", 2024-09-16T20:50:24.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065770472999999996 2024-09-16T20:50:24.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:24.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841847+0000", 2024-09-16T20:50:24.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5558e-05 2024-09-16T20:50:24.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:24.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842005+0000", 2024-09-16T20:50:24.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015849000000000001 2024-09-16T20:50:24.609 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.610 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.610 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.610 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12626 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1028096~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:24.610 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776041+0000", 2024-09-16T20:50:24.610 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 33.755393271999999, 2024-09-16T20:50:24.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066225260999999994, 2024-09-16T20:50:24.610 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.610 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.610 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:24.610 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.610 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:24.610 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:24.611 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12626 2024-09-16T20:50:24.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.611 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.611 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.611 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776041+0000", 2024-09-16T20:50:24.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.611 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.611 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776041+0000", 2024-09-16T20:50:24.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.612 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.612 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776263+0000", 2024-09-16T20:50:24.612 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00022193899999999999 2024-09-16T20:50:24.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.612 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.612 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776268+0000", 2024-09-16T20:50:24.612 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8049999999999997e-06 2024-09-16T20:50:24.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.612 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.612 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776280+0000", 2024-09-16T20:50:24.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2635e-05 2024-09-16T20:50:24.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.613 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.613 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776288+0000", 2024-09-16T20:50:24.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8199999999999997e-06 2024-09-16T20:50:24.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.613 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:24.613 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842041+0000", 2024-09-16T20:50:24.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065752506000000002 2024-09-16T20:50:24.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.613 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:24.614 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842123+0000", 2024-09-16T20:50:24.614 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2393999999999998e-05 2024-09-16T20:50:24.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.614 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:24.614 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842266+0000", 2024-09-16T20:50:24.614 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014316200000000001 2024-09-16T20:50:24.614 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.614 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.614 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.614 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12627 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1033216~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:24.615 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776301+0000", 2024-09-16T20:50:24.615 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 33.755133311999998, 2024-09-16T20:50:24.615 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066187814999999997, 2024-09-16T20:50:24.615 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.615 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.615 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:24.615 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.615 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:24.615 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:24.615 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12627 2024-09-16T20:50:24.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.615 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.615 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.615 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.616 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776301+0000", 2024-09-16T20:50:24.616 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.616 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.616 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.616 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776301+0000", 2024-09-16T20:50:24.616 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.616 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.616 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.616 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776473+0000", 2024-09-16T20:50:24.616 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017222400000000001 2024-09-16T20:50:24.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.617 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.617 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776477+0000", 2024-09-16T20:50:24.617 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7019999999999999e-06 2024-09-16T20:50:24.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.617 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.617 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776486+0000", 2024-09-16T20:50:24.617 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2119999999999992e-06 2024-09-16T20:50:24.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.617 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776492+0000", 2024-09-16T20:50:24.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8089999999999998e-06 2024-09-16T20:50:24.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:24.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842301+0000", 2024-09-16T20:50:24.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065809042999999998 2024-09-16T20:50:24.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:24.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842362+0000", 2024-09-16T20:50:24.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1260000000000006e-05 2024-09-16T20:50:24.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.619 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:24.619 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842489+0000", 2024-09-16T20:50:24.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012656500000000001 2024-09-16T20:50:24.619 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.619 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.619 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.619 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12628 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1038336~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:24.619 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776502+0000", 2024-09-16T20:50:24.619 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 33.754932666000002, 2024-09-16T20:50:24.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066283426000000006, 2024-09-16T20:50:24.619 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.620 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.620 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:24.620 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.620 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:24.620 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:24.620 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12628 2024-09-16T20:50:24.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.620 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.620 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.620 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776502+0000", 2024-09-16T20:50:24.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.621 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.621 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776502+0000", 2024-09-16T20:50:24.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.621 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.621 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776716+0000", 2024-09-16T20:50:24.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000214596 2024-09-16T20:50:24.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.621 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.621 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776723+0000", 2024-09-16T20:50:24.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4000000000000003e-06 2024-09-16T20:50:24.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.622 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.622 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776731+0000", 2024-09-16T20:50:24.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6129999999999998e-06 2024-09-16T20:50:24.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.622 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.622 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776737+0000", 2024-09-16T20:50:24.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1970000000000001e-06 2024-09-16T20:50:24.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.622 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:24.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842532+0000", 2024-09-16T20:50:24.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065795127999999994 2024-09-16T20:50:24.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:24.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842599+0000", 2024-09-16T20:50:24.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6711000000000006e-05 2024-09-16T20:50:24.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:24.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842785+0000", 2024-09-16T20:50:24.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018578099999999999 2024-09-16T20:50:24.623 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.624 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.624 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.624 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.624 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12629 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1043456~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:24.624 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776748+0000", 2024-09-16T20:50:24.624 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 33.754686288999999, 2024-09-16T20:50:24.624 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066303788000000002, 2024-09-16T20:50:24.624 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:24.624 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:24.624 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:24.624 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:24.624 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:24.624 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:24.625 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12629 2024-09-16T20:50:24.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.625 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:24.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.625 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:24.625 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776748+0000", 2024-09-16T20:50:24.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.625 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:24.625 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776748+0000", 2024-09-16T20:50:24.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:24.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.626 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:24.626 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776934+0000", 2024-09-16T20:50:24.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018598399999999999 2024-09-16T20:50:24.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.626 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:24.626 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776939+0000", 2024-09-16T20:50:24.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1000000000000003e-06 2024-09-16T20:50:24.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.626 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:24.626 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776954+0000", 2024-09-16T20:50:24.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5221e-05 2024-09-16T20:50:24.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.627 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:24.627 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776962+0000", 2024-09-16T20:50:24.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.554e-06 2024-09-16T20:50:24.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.627 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:24.627 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842813+0000", 2024-09-16T20:50:24.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065851006000000004 2024-09-16T20:50:24.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.627 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:24.627 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842886+0000", 2024-09-16T20:50:24.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3663000000000003e-05 2024-09-16T20:50:24.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:24.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:24.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:24.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.843052+0000", 2024-09-16T20:50:24.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016526 2024-09-16T20:50:24.628 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.628 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.628 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.628 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:24.628 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:24.628 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:24.629 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:50:24.629 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:50:24.630 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:50:24.630 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:50:24.631 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:50:24.631 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:50:24.631 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:50:24.632 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:50:24.632 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:50:24.674 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:24.673+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:24.726 INFO:teuthology.orchestra.run.smithi029.stderr:marked out osd.0. 2024-09-16T20:50:24.735 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T20:50:24.736 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:24.734+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:24.743 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:24.744 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:24.753 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:50:24.836 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:24.835+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:24.860 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:24.860 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:24.869 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:50:24.953 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:24.952+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:25.016 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:25.017 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:25.025 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:50:25.067 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:25.066+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:25.132 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:25.132 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:25.139 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:25.138+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:25.141 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:50:25.241 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:25.240+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:25.288 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:25.288 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:25.297 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:50:25.342 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:25.341+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:25.442 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:25.441+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:25.445 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:25.445 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:25.453 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:50:25.543 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:25.542+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:25.601 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:25.601 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:25.609 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:50:25.650 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:25.649+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:25.723 INFO:teuthology.orchestra.run.smithi029.stderr:noscrub is unset 2024-09-16T20:50:25.745 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:25.744+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:25.763 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:25.763 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:25.772 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:50:25.856 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:25.855+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:25.919 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:25.919 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:25.928 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:50:25.969 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:25.968+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:26.048 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:26.047+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:26.076 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:26.076 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:26.084 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:50:26.168 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:26.167+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:26.232 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:26.232 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:26.241 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:50:26.282 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:26.281+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:26.351 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:26.350+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:26.388 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:26.388 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:26.396 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:50:26.452 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:26.451+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:26.517 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:26.518 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:26.527 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:50:26.553 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:26.552+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:26.634 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T20:50:26.675 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:26.676 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:26.678 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:26.677+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:26.684 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:50:26.757 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:26.757+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:26.774 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #43, image image_client.0-clone43 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent44 2024-09-16T20:50:26.786 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:50:26.786 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:50:26.813 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:50:26.813 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:50:26.824 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:26.824 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:26.836 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:50:26.837 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:50:26.837 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:50:26.861 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:50:26.861 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:50:26.877 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:26.877+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:26.886 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:50:26.887 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:50:26.918 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:50:26.918 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:50:26.973 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:26.972+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:27.018 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:27.018 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:50:27.019 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:50:27.019 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:27.030 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:50:27.057 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:27.056+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:27.158 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:27.158+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:27.170 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:27.170 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:50:27.170 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:50:27.170 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:50:27.170 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:27.181 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:50:27.265 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:27.265+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:27.360 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:27.359+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:27.376 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:27.376 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:50:27.376 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:50:27.376 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:50:27.376 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.376 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12548 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 628736~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:27.377 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757009+0000", 2024-09-16T20:50:27.377 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.617418110000003, 2024-09-16T20:50:27.377 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065743956000000006, 2024-09-16T20:50:27.377 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:27.377 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:27.377 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:27.377 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:27.377 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:27.377 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:27.377 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12548 2024-09-16T20:50:27.377 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.377 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:27.377 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.378 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:27.378 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757009+0000", 2024-09-16T20:50:27.378 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:27.378 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.378 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.378 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:27.378 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757009+0000", 2024-09-16T20:50:27.378 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:27.378 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.378 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.378 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:27.378 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757010+0000", 2024-09-16T20:50:27.378 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9900000000000001e-07 2024-09-16T20:50:27.378 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.379 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.379 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:27.379 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757012+0000", 2024-09-16T20:50:27.379 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2689999999999998e-06 2024-09-16T20:50:27.379 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.379 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.379 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:27.379 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757022+0000", 2024-09-16T20:50:27.379 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9699999999999994e-06 2024-09-16T20:50:27.379 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.379 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.379 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:27.379 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757028+0000", 2024-09-16T20:50:27.379 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3799999999999999e-06 2024-09-16T20:50:27.380 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.380 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.380 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:27.380 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822552+0000", 2024-09-16T20:50:27.380 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065524281000000004 2024-09-16T20:50:27.380 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.380 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.380 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:27.380 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822613+0000", 2024-09-16T20:50:27.380 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0430999999999998e-05 2024-09-16T20:50:27.380 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.380 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.380 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:27.380 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822753+0000", 2024-09-16T20:50:27.381 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000140226 2024-09-16T20:50:27.381 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:27.381 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:27.381 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:27.381 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.381 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.381 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12549 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 633856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:27.381 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:50:27.381 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.617389363000001, 2024-09-16T20:50:27.381 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065978518, 2024-09-16T20:50:27.381 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:27.381 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:27.381 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:27.382 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:27.382 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:27.382 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:27.382 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12549 2024-09-16T20:50:27.382 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.382 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:27.382 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.382 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:27.382 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:50:27.382 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:27.382 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.382 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.382 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:27.383 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:50:27.383 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:27.383 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.383 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.383 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:27.383 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:50:27.383 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9299999999999999e-07 2024-09-16T20:50:27.383 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.383 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.383 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:27.383 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757040+0000", 2024-09-16T20:50:27.383 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.677e-06 2024-09-16T20:50:27.383 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.383 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.384 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:27.384 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757045+0000", 2024-09-16T20:50:27.384 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8149999999999996e-06 2024-09-16T20:50:27.385 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.385 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.385 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:27.385 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757048+0000", 2024-09-16T20:50:27.385 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1e-06 2024-09-16T20:50:27.385 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.385 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.385 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:27.385 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822796+0000", 2024-09-16T20:50:27.385 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065748081 2024-09-16T20:50:27.386 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.386 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.386 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:27.386 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822867+0000", 2024-09-16T20:50:27.386 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0679000000000005e-05 2024-09-16T20:50:27.386 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.386 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.386 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:27.386 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823016+0000", 2024-09-16T20:50:27.386 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014987299999999999 2024-09-16T20:50:27.386 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:27.386 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:27.386 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:27.387 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.387 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.387 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12550 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 638976~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:27.387 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757060+0000", 2024-09-16T20:50:27.387 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.617366947999997, 2024-09-16T20:50:27.387 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066153564999999998, 2024-09-16T20:50:27.387 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:27.387 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:27.387 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:27.387 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:27.387 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:27.387 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:27.388 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12550 2024-09-16T20:50:27.388 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.388 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:27.388 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.388 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:27.388 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757060+0000", 2024-09-16T20:50:27.388 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:27.388 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.388 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.388 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:27.388 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757060+0000", 2024-09-16T20:50:27.388 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:27.389 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.389 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.389 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:27.389 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757061+0000", 2024-09-16T20:50:27.389 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.15e-07 2024-09-16T20:50:27.389 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.389 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.389 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:27.389 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757062+0000", 2024-09-16T20:50:27.389 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.663e-06 2024-09-16T20:50:27.389 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.389 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.390 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:27.390 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757066+0000", 2024-09-16T20:50:27.390 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.817e-06 2024-09-16T20:50:27.390 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.390 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.390 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:27.390 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757070+0000", 2024-09-16T20:50:27.390 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1509999999999997e-06 2024-09-16T20:50:27.390 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.390 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.390 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:27.390 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823044+0000", 2024-09-16T20:50:27.391 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065973992999999995 2024-09-16T20:50:27.391 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.391 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.391 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:27.391 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823082+0000", 2024-09-16T20:50:27.391 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8050999999999999e-05 2024-09-16T20:50:27.391 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.391 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.391 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:27.391 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823214+0000", 2024-09-16T20:50:27.391 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013157500000000001 2024-09-16T20:50:27.391 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:27.391 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:27.391 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:27.392 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.392 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.392 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12551 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 644096~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:27.392 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:50:27.392 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.617353250000001, 2024-09-16T20:50:27.392 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066391378000000001, 2024-09-16T20:50:27.392 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:27.392 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:27.392 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:27.392 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:27.392 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:27.392 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:27.392 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12551 2024-09-16T20:50:27.393 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.393 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:27.393 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.393 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:27.393 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:50:27.393 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:27.393 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.393 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.393 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:27.393 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:50:27.393 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:27.393 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.394 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.394 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:27.394 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:50:27.394 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2999999999999999e-07 2024-09-16T20:50:27.394 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.394 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.394 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:27.394 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757075+0000", 2024-09-16T20:50:27.394 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.097e-06 2024-09-16T20:50:27.394 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.395 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.395 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:27.395 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757082+0000", 2024-09-16T20:50:27.395 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9500000000000004e-06 2024-09-16T20:50:27.395 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.395 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.395 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:27.395 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757086+0000", 2024-09-16T20:50:27.395 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2229999999999999e-06 2024-09-16T20:50:27.395 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.395 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.396 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:27.396 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823262+0000", 2024-09-16T20:50:27.396 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066176078999999999 2024-09-16T20:50:27.396 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.396 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.396 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:27.396 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823336+0000", 2024-09-16T20:50:27.396 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3979000000000004e-05 2024-09-16T20:50:27.396 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.396 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.396 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:27.396 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823465+0000", 2024-09-16T20:50:27.396 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012982 2024-09-16T20:50:27.397 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:27.397 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:27.397 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:27.397 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.397 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.397 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12552 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 649216~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:27.397 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757089+0000", 2024-09-16T20:50:27.397 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.617338109000002, 2024-09-16T20:50:27.397 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066655617, 2024-09-16T20:50:27.398 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:27.398 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:27.398 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:27.398 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:27.398 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:27.398 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:27.398 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12552 2024-09-16T20:50:27.398 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.398 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:27.398 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.398 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:27.398 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757089+0000", 2024-09-16T20:50:27.398 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:27.399 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.399 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.399 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:27.399 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757089+0000", 2024-09-16T20:50:27.399 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:27.399 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.399 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.399 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:27.399 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759492+0000", 2024-09-16T20:50:27.399 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0024026109999999998 2024-09-16T20:50:27.399 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.399 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.399 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:27.400 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759497+0000", 2024-09-16T20:50:27.400 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1120000000000004e-06 2024-09-16T20:50:27.400 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.400 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.400 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:27.400 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759508+0000", 2024-09-16T20:50:27.400 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0981e-05 2024-09-16T20:50:27.400 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.400 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.400 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:27.400 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759516+0000", 2024-09-16T20:50:27.400 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7270000000000007e-06 2024-09-16T20:50:27.400 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.401 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.401 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:27.401 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823504+0000", 2024-09-16T20:50:27.401 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.063987984999999997 2024-09-16T20:50:27.401 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.401 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.401 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:27.401 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823579+0000", 2024-09-16T20:50:27.402 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5703000000000003e-05 2024-09-16T20:50:27.402 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.402 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.402 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:27.402 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823745+0000", 2024-09-16T20:50:27.402 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016549800000000001 2024-09-16T20:50:27.402 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:27.402 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:27.402 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:27.402 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.402 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.402 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12560 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 690176~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:27.402 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.759658+0000", 2024-09-16T20:50:27.402 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.614768906999998, 2024-09-16T20:50:27.403 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065858517000000005, 2024-09-16T20:50:27.403 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:27.403 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:27.403 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:27.403 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:27.403 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:27.403 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:27.403 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12560 2024-09-16T20:50:27.403 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.403 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:27.403 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.403 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:27.404 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759658+0000", 2024-09-16T20:50:27.404 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:27.404 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.404 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.404 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:27.404 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759658+0000", 2024-09-16T20:50:27.404 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:27.404 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.404 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.404 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:27.404 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759659+0000", 2024-09-16T20:50:27.404 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.17e-07 2024-09-16T20:50:27.404 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.405 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.405 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:27.405 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759660+0000", 2024-09-16T20:50:27.405 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0189999999999999e-06 2024-09-16T20:50:27.405 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.405 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.405 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:27.405 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759663+0000", 2024-09-16T20:50:27.405 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0010000000000002e-06 2024-09-16T20:50:27.405 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.405 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.405 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:27.406 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759665+0000", 2024-09-16T20:50:27.406 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5440000000000001e-06 2024-09-16T20:50:27.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.406 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.406 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:27.406 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825355+0000", 2024-09-16T20:50:27.406 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065690224000000005 2024-09-16T20:50:27.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.406 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.407 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:27.407 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825394+0000", 2024-09-16T20:50:27.407 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8810999999999998e-05 2024-09-16T20:50:27.407 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.407 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.407 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:27.407 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825517+0000", 2024-09-16T20:50:27.407 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012270099999999999 2024-09-16T20:50:27.407 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:27.407 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:27.407 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:27.407 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.407 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.407 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12561 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 695296~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:27.408 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.759668+0000", 2024-09-16T20:50:27.408 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.614759065000001, 2024-09-16T20:50:27.408 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066116999999999995, 2024-09-16T20:50:27.408 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:27.408 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:27.408 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:27.408 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:27.408 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:27.408 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:27.408 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12561 2024-09-16T20:50:27.408 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.408 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:27.408 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.409 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:27.409 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759668+0000", 2024-09-16T20:50:27.409 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:27.409 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.409 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.409 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:27.409 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759668+0000", 2024-09-16T20:50:27.409 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:27.409 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.409 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.409 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:27.409 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765379+0000", 2024-09-16T20:50:27.409 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057104410000000001 2024-09-16T20:50:27.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.410 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.410 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:27.410 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765383+0000", 2024-09-16T20:50:27.410 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2980000000000002e-06 2024-09-16T20:50:27.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.410 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.410 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:27.410 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765395+0000", 2024-09-16T20:50:27.410 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1671000000000001e-05 2024-09-16T20:50:27.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.410 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.410 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:27.410 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765403+0000", 2024-09-16T20:50:27.410 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1210000000000007e-06 2024-09-16T20:50:27.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.411 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.411 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:27.411 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825552+0000", 2024-09-16T20:50:27.411 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.060149498000000003 2024-09-16T20:50:27.411 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.411 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.411 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:27.411 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825625+0000", 2024-09-16T20:50:27.411 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2540999999999995e-05 2024-09-16T20:50:27.411 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.411 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.411 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:27.411 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825785+0000", 2024-09-16T20:50:27.411 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016043000000000001 2024-09-16T20:50:27.412 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:27.412 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:27.412 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:27.412 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.412 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.412 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12585 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 818176~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:27.412 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.765848+0000", 2024-09-16T20:50:27.412 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.608579094, 2024-09-16T20:50:27.412 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065843218999999995, 2024-09-16T20:50:27.412 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:27.412 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:27.412 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:27.412 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:27.412 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:27.413 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:27.413 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12585 2024-09-16T20:50:27.413 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.413 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:27.413 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.413 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:27.413 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765848+0000", 2024-09-16T20:50:27.413 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:27.413 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.413 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.413 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:27.413 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765848+0000", 2024-09-16T20:50:27.413 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:27.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.414 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.414 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:27.414 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768683+0000", 2024-09-16T20:50:27.414 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0028344519999999999 2024-09-16T20:50:27.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.414 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.414 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:27.414 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768686+0000", 2024-09-16T20:50:27.414 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8789999999999996e-06 2024-09-16T20:50:27.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.414 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.415 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:27.415 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768696+0000", 2024-09-16T20:50:27.415 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6709999999999994e-06 2024-09-16T20:50:27.415 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.415 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.415 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:27.415 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768702+0000", 2024-09-16T20:50:27.415 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2550000000000003e-06 2024-09-16T20:50:27.415 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.415 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.415 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:27.415 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.831499+0000", 2024-09-16T20:50:27.415 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.062797081000000005 2024-09-16T20:50:27.416 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.416 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.416 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:27.416 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.831538+0000", 2024-09-16T20:50:27.416 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8523999999999999e-05 2024-09-16T20:50:27.416 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.416 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.416 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:27.416 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.831691+0000", 2024-09-16T20:50:27.416 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015335699999999999 2024-09-16T20:50:27.416 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:27.416 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:27.416 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:27.416 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.417 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.417 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12597 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 879616~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:27.417 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.768883+0000", 2024-09-16T20:50:27.417 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.605544365999997, 2024-09-16T20:50:27.417 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065816527, 2024-09-16T20:50:27.417 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:27.417 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:27.417 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:27.417 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:27.417 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:27.418 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:27.418 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12597 2024-09-16T20:50:27.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.418 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:27.418 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.418 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:27.418 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768883+0000", 2024-09-16T20:50:27.418 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:27.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.418 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.418 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:27.418 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768883+0000", 2024-09-16T20:50:27.418 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:27.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.419 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.419 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:27.419 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771508+0000", 2024-09-16T20:50:27.419 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0026247589999999999 2024-09-16T20:50:27.419 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.419 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.419 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:27.419 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771513+0000", 2024-09-16T20:50:27.419 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9949999999999996e-06 2024-09-16T20:50:27.419 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.419 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.419 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:27.419 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771525+0000", 2024-09-16T20:50:27.419 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2033e-05 2024-09-16T20:50:27.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.420 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.420 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:27.420 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771532+0000", 2024-09-16T20:50:27.420 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4220000000000003e-06 2024-09-16T20:50:27.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.420 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.420 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:27.420 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.834532+0000", 2024-09-16T20:50:27.421 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.063000159 2024-09-16T20:50:27.421 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.421 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.421 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:27.421 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.834590+0000", 2024-09-16T20:50:27.421 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8218999999999999e-05 2024-09-16T20:50:27.421 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.421 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.421 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:27.421 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.834699+0000", 2024-09-16T20:50:27.421 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010894 2024-09-16T20:50:27.421 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:27.421 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:27.421 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:27.421 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.422 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.422 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12619 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 992256~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:27.422 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.774656+0000", 2024-09-16T20:50:27.422 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.599771592000003, 2024-09-16T20:50:27.422 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065715859000000001, 2024-09-16T20:50:27.422 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:27.422 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:27.422 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:27.422 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:27.422 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:27.422 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:27.422 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12619 2024-09-16T20:50:27.422 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.422 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:27.422 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.423 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:27.423 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774656+0000", 2024-09-16T20:50:27.423 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:27.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.423 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.423 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:27.423 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774656+0000", 2024-09-16T20:50:27.423 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:27.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.423 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.423 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:27.423 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774875+0000", 2024-09-16T20:50:27.423 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000218898 2024-09-16T20:50:27.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.424 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:27.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774879+0000", 2024-09-16T20:50:27.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0389999999999998e-06 2024-09-16T20:50:27.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.424 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:27.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774890+0000", 2024-09-16T20:50:27.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1250000000000001e-05 2024-09-16T20:50:27.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.424 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:27.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774897+0000", 2024-09-16T20:50:27.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7619999999999998e-06 2024-09-16T20:50:27.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.425 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:27.425 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840139+0000", 2024-09-16T20:50:27.425 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065242438 2024-09-16T20:50:27.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.425 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:27.425 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840186+0000", 2024-09-16T20:50:27.425 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6894999999999997e-05 2024-09-16T20:50:27.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.425 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:27.426 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840372+0000", 2024-09-16T20:50:27.426 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000185577 2024-09-16T20:50:27.426 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:27.426 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:27.426 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:27.426 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.426 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.426 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12620 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 997376~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:27.426 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.774908+0000", 2024-09-16T20:50:27.426 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.599519371, 2024-09-16T20:50:27.426 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065782754999999998, 2024-09-16T20:50:27.426 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:27.427 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:27.427 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:27.427 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:27.427 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:27.427 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:27.427 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12620 2024-09-16T20:50:27.427 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.427 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:27.427 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.427 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:27.427 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774908+0000", 2024-09-16T20:50:27.427 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:27.427 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.427 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.428 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:27.428 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774908+0000", 2024-09-16T20:50:27.428 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:27.428 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.428 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.428 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:27.428 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775102+0000", 2024-09-16T20:50:27.428 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000194098 2024-09-16T20:50:27.428 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.428 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.428 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:27.428 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775105+0000", 2024-09-16T20:50:27.429 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8739999999999999e-06 2024-09-16T20:50:27.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.429 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.429 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:27.429 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775111+0000", 2024-09-16T20:50:27.429 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9519999999999999e-06 2024-09-16T20:50:27.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.429 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.429 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:27.429 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775115+0000", 2024-09-16T20:50:27.429 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.002e-06 2024-09-16T20:50:27.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.429 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.430 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:27.430 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840409+0000", 2024-09-16T20:50:27.430 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065294195999999999 2024-09-16T20:50:27.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.430 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.430 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:27.430 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840507+0000", 2024-09-16T20:50:27.430 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7662000000000001e-05 2024-09-16T20:50:27.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.430 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.430 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:27.430 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840691+0000", 2024-09-16T20:50:27.430 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000183971 2024-09-16T20:50:27.430 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:27.430 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:27.430 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:27.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.431 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.431 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12621 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1002496~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:27.431 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775123+0000", 2024-09-16T20:50:27.431 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.599304674999999, 2024-09-16T20:50:27.431 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065829586999999995, 2024-09-16T20:50:27.431 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:27.431 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:27.431 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:27.431 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:27.431 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:27.431 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:27.431 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12621 2024-09-16T20:50:27.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.432 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:27.432 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.432 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:27.432 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775123+0000", 2024-09-16T20:50:27.432 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:27.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.432 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.432 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:27.432 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775123+0000", 2024-09-16T20:50:27.432 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:27.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.432 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.432 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:27.433 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775348+0000", 2024-09-16T20:50:27.433 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000225295 2024-09-16T20:50:27.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.433 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.433 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:27.433 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775352+0000", 2024-09-16T20:50:27.433 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0810000000000004e-06 2024-09-16T20:50:27.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.433 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.433 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:27.434 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775361+0000", 2024-09-16T20:50:27.434 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7399999999999993e-06 2024-09-16T20:50:27.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.434 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.434 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:27.434 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775367+0000", 2024-09-16T20:50:27.434 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5200000000000003e-06 2024-09-16T20:50:27.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.434 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.434 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:27.434 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840730+0000", 2024-09-16T20:50:27.434 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065362803999999997 2024-09-16T20:50:27.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.435 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.435 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:27.435 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840801+0000", 2024-09-16T20:50:27.435 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1006000000000004e-05 2024-09-16T20:50:27.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.435 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.435 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:27.435 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840952+0000", 2024-09-16T20:50:27.435 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015114100000000001 2024-09-16T20:50:27.435 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:27.435 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:27.435 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:27.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.436 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.436 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12622 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1007616~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:27.436 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775381+0000", 2024-09-16T20:50:27.436 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.599045840999999, 2024-09-16T20:50:27.436 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065822581000000005, 2024-09-16T20:50:27.436 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:27.436 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:27.436 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:27.436 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:27.436 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:27.436 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:27.436 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12622 2024-09-16T20:50:27.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.437 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:27.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.437 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:27.437 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775381+0000", 2024-09-16T20:50:27.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:27.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.437 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:27.437 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775381+0000", 2024-09-16T20:50:27.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:27.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:27.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775563+0000", 2024-09-16T20:50:27.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018153700000000001 2024-09-16T20:50:27.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:27.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775566+0000", 2024-09-16T20:50:27.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9620000000000002e-06 2024-09-16T20:50:27.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:27.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775572+0000", 2024-09-16T20:50:27.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8529999999999997e-06 2024-09-16T20:50:27.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.439 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.439 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:27.439 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775576+0000", 2024-09-16T20:50:27.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8920000000000002e-06 2024-09-16T20:50:27.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.439 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.439 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:27.439 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840990+0000", 2024-09-16T20:50:27.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065414599000000004 2024-09-16T20:50:27.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.439 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.440 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:27.440 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841068+0000", 2024-09-16T20:50:27.440 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8157000000000004e-05 2024-09-16T20:50:27.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.440 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:27.440 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841204+0000", 2024-09-16T20:50:27.440 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013558099999999999 2024-09-16T20:50:27.440 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:27.440 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:27.440 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:27.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.441 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12623 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1012736~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:27.441 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775583+0000", 2024-09-16T20:50:27.441 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.598844403000001, 2024-09-16T20:50:27.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065929087999999997, 2024-09-16T20:50:27.441 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:27.441 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:27.441 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:27.441 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:27.441 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:27.441 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:27.441 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12623 2024-09-16T20:50:27.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.442 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:27.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:27.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775583+0000", 2024-09-16T20:50:27.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:27.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:27.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775583+0000", 2024-09-16T20:50:27.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:27.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:27.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775804+0000", 2024-09-16T20:50:27.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00022116900000000001 2024-09-16T20:50:27.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.443 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:27.443 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775807+0000", 2024-09-16T20:50:27.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9529999999999999e-06 2024-09-16T20:50:27.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.443 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:27.443 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775814+0000", 2024-09-16T20:50:27.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7490000000000001e-06 2024-09-16T20:50:27.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:27.444 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775818+0000", 2024-09-16T20:50:27.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1999999999999996e-06 2024-09-16T20:50:27.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:27.444 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841267+0000", 2024-09-16T20:50:27.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065449127999999995 2024-09-16T20:50:27.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:27.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841344+0000", 2024-09-16T20:50:27.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6546999999999997e-05 2024-09-16T20:50:27.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:27.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841512+0000", 2024-09-16T20:50:27.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016834200000000001 2024-09-16T20:50:27.445 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:27.446 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:27.446 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:27.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.446 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12624 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1017856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:27.446 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:50:27.446 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.598601262999999, 2024-09-16T20:50:27.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065945866000000006, 2024-09-16T20:50:27.446 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:27.446 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:27.446 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:27.446 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:27.446 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:27.446 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:27.447 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12624 2024-09-16T20:50:27.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.447 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:27.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:27.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:50:27.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:27.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:27.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:50:27.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:27.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:27.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:50:27.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-16T20:50:27.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:27.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775828+0000", 2024-09-16T20:50:27.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3999999999999999e-06 2024-09-16T20:50:27.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:27.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775831+0000", 2024-09-16T20:50:27.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.478e-06 2024-09-16T20:50:27.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.449 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:27.449 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775834+0000", 2024-09-16T20:50:27.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9440000000000001e-06 2024-09-16T20:50:27.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.449 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:27.449 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841547+0000", 2024-09-16T20:50:27.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065712801000000001 2024-09-16T20:50:27.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.449 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:27.449 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841618+0000", 2024-09-16T20:50:27.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1151999999999999e-05 2024-09-16T20:50:27.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.450 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:27.450 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841772+0000", 2024-09-16T20:50:27.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000153721 2024-09-16T20:50:27.450 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:27.450 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:27.450 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:27.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.450 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12625 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1022976~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:27.450 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775840+0000", 2024-09-16T20:50:27.450 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.598587653999999, 2024-09-16T20:50:27.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066165451, 2024-09-16T20:50:27.451 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:27.451 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:27.451 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:27.451 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:27.451 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:27.451 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:27.451 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12625 2024-09-16T20:50:27.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.451 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:27.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.451 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:27.451 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775840+0000", 2024-09-16T20:50:27.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:27.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:27.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775840+0000", 2024-09-16T20:50:27.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:27.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:27.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776014+0000", 2024-09-16T20:50:27.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017441099999999999 2024-09-16T20:50:27.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:27.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776018+0000", 2024-09-16T20:50:27.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6509999999999999e-06 2024-09-16T20:50:27.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:27.453 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776026+0000", 2024-09-16T20:50:27.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.028e-06 2024-09-16T20:50:27.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:27.453 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776031+0000", 2024-09-16T20:50:27.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8400000000000002e-06 2024-09-16T20:50:27.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:27.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841801+0000", 2024-09-16T20:50:27.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065770472999999996 2024-09-16T20:50:27.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:27.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841847+0000", 2024-09-16T20:50:27.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5558e-05 2024-09-16T20:50:27.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:27.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842005+0000", 2024-09-16T20:50:27.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015849000000000001 2024-09-16T20:50:27.455 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:27.455 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:27.455 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:27.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.455 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12626 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1028096~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:27.455 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776041+0000", 2024-09-16T20:50:27.455 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.598386355000002, 2024-09-16T20:50:27.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066225260999999994, 2024-09-16T20:50:27.455 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:27.455 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:27.455 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:27.455 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:27.455 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:27.456 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:27.456 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12626 2024-09-16T20:50:27.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.456 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:27.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.456 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:27.456 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776041+0000", 2024-09-16T20:50:27.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:27.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.456 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:27.456 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776041+0000", 2024-09-16T20:50:27.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:27.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.457 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:27.457 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776263+0000", 2024-09-16T20:50:27.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00022193899999999999 2024-09-16T20:50:27.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.457 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:27.457 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776268+0000", 2024-09-16T20:50:27.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8049999999999997e-06 2024-09-16T20:50:27.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.457 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:27.457 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776280+0000", 2024-09-16T20:50:27.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2635e-05 2024-09-16T20:50:27.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:27.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776288+0000", 2024-09-16T20:50:27.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8199999999999997e-06 2024-09-16T20:50:27.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:27.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842041+0000", 2024-09-16T20:50:27.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065752506000000002 2024-09-16T20:50:27.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:27.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842123+0000", 2024-09-16T20:50:27.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2393999999999998e-05 2024-09-16T20:50:27.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:27.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842266+0000", 2024-09-16T20:50:27.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014316200000000001 2024-09-16T20:50:27.459 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:27.459 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:27.460 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:27.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.460 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12627 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1033216~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:27.460 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776301+0000", 2024-09-16T20:50:27.460 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.598126395000001, 2024-09-16T20:50:27.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066187814999999997, 2024-09-16T20:50:27.460 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:27.460 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:27.460 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:27.460 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:27.460 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:27.460 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:27.461 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12627 2024-09-16T20:50:27.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.461 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:27.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:27.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776301+0000", 2024-09-16T20:50:27.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:27.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:27.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776301+0000", 2024-09-16T20:50:27.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:27.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.462 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:27.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776473+0000", 2024-09-16T20:50:27.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017222400000000001 2024-09-16T20:50:27.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.462 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:27.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776477+0000", 2024-09-16T20:50:27.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7019999999999999e-06 2024-09-16T20:50:27.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.462 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:27.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776486+0000", 2024-09-16T20:50:27.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2119999999999992e-06 2024-09-16T20:50:27.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.463 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:27.463 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776492+0000", 2024-09-16T20:50:27.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8089999999999998e-06 2024-09-16T20:50:27.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.463 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:27.463 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842301+0000", 2024-09-16T20:50:27.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065809042999999998 2024-09-16T20:50:27.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.463 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:27.463 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842362+0000", 2024-09-16T20:50:27.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1260000000000006e-05 2024-09-16T20:50:27.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.464 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:27.464 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842489+0000", 2024-09-16T20:50:27.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012656500000000001 2024-09-16T20:50:27.464 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:27.464 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:27.464 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:27.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.464 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12628 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1038336~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:27.464 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776502+0000", 2024-09-16T20:50:27.464 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.597925748999998, 2024-09-16T20:50:27.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066283426000000006, 2024-09-16T20:50:27.465 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:27.465 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:27.465 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:27.465 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:27.465 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:27.465 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:27.465 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12628 2024-09-16T20:50:27.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.465 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:27.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.465 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:27.465 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776502+0000", 2024-09-16T20:50:27.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:27.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:27.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776502+0000", 2024-09-16T20:50:27.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:27.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:27.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776716+0000", 2024-09-16T20:50:27.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000214596 2024-09-16T20:50:27.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:27.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776723+0000", 2024-09-16T20:50:27.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4000000000000003e-06 2024-09-16T20:50:27.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.467 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:27.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776731+0000", 2024-09-16T20:50:27.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6129999999999998e-06 2024-09-16T20:50:27.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.467 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:27.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776737+0000", 2024-09-16T20:50:27.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1970000000000001e-06 2024-09-16T20:50:27.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:27.468 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842532+0000", 2024-09-16T20:50:27.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065795127999999994 2024-09-16T20:50:27.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:27.468 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842599+0000", 2024-09-16T20:50:27.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6711000000000006e-05 2024-09-16T20:50:27.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:27.468 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842785+0000", 2024-09-16T20:50:27.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018578099999999999 2024-09-16T20:50:27.469 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:27.469 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:27.469 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:27.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.469 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12629 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1043456~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:27.469 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776748+0000", 2024-09-16T20:50:27.469 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.597679372000002, 2024-09-16T20:50:27.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066303788000000002, 2024-09-16T20:50:27.469 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:27.469 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:27.469 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:27.470 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:27.470 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:27.470 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:27.470 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12629 2024-09-16T20:50:27.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.470 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:27.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:27.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776748+0000", 2024-09-16T20:50:27.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:27.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:27.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776748+0000", 2024-09-16T20:50:27.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:27.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:27.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776934+0000", 2024-09-16T20:50:27.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018598399999999999 2024-09-16T20:50:27.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:27.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776939+0000", 2024-09-16T20:50:27.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1000000000000003e-06 2024-09-16T20:50:27.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:27.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776954+0000", 2024-09-16T20:50:27.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5221e-05 2024-09-16T20:50:27.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:27.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776962+0000", 2024-09-16T20:50:27.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.554e-06 2024-09-16T20:50:27.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:27.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842813+0000", 2024-09-16T20:50:27.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065851006000000004 2024-09-16T20:50:27.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:27.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842886+0000", 2024-09-16T20:50:27.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3663000000000003e-05 2024-09-16T20:50:27.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:27.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.843052+0000", 2024-09-16T20:50:27.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016526 2024-09-16T20:50:27.473 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:27.473 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:27.473 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:27.473 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:27.473 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:27.474 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:27.475 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:50:27.475 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:27.473+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:27.575 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:27.575+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:27.634 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:27.635 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:50:27.635 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:50:27.635 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:27.645 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:50:27.677 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:27.675+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:27.746 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T20:50:27.791 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:27.790+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:27.827 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:27.827 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:50:27.827 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:50:27.827 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:50:27.827 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:27.837 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:50:27.879 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:27.878+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:27.950 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:27.950 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:50:27.951 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:50:27.951 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:50:27.951 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.951 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5286.0:10643 3.17 3.f6273fb7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-16T20:50:27.951 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:50:27.951 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 79.157764396999994, 2024-09-16T20:50:27.951 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.76672347200000002, 2024-09-16T20:50:27.951 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:27.951 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:27.951 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:27.951 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:27.951 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5286", 2024-09-16T20:50:27.951 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/4135433989", 2024-09-16T20:50:27.951 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 10643 2024-09-16T20:50:27.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.952 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:27.952 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.952 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:27.952 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:50:27.952 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:27.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.952 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.952 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:27.952 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:50:27.952 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:27.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.952 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.952 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:27.952 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790489+0000", 2024-09-16T20:50:27.953 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5910000000000001e-06 2024-09-16T20:50:27.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.953 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.953 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:27.953 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790493+0000", 2024-09-16T20:50:27.953 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2669999999999998e-06 2024-09-16T20:50:27.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.953 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.953 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:27.953 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790500+0000", 2024-09-16T20:50:27.953 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3780000000000003e-06 2024-09-16T20:50:27.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.953 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.954 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:27.954 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790505+0000", 2024-09-16T20:50:27.954 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1560000000000003e-06 2024-09-16T20:50:27.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.954 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.954 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:27.954 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790559+0000", 2024-09-16T20:50:27.954 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3494e-05 2024-09-16T20:50:27.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.954 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.954 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:27.954 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790566+0000", 2024-09-16T20:50:27.955 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7649999999999993e-06 2024-09-16T20:50:27.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.955 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:27.955 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:09.557198+0000", 2024-09-16T20:50:27.955 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.76663136200000004 2024-09-16T20:50:27.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.955 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:27.955 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:09.557211+0000", 2024-09-16T20:50:27.955 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3458999999999999e-05 2024-09-16T20:50:27.955 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:27.955 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:27.955 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:27.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.956 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5512.0:450 3.2 3:44e0cffa:::rbd_header.110ef0d7a535:head [watch reconnect cookie 140263298018912 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:50:27.956 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:50:27.956 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.385741484, 2024-09-16T20:50:27.956 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0084581939999999, 2024-09-16T20:50:27.956 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:27.956 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:27.956 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:27.956 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:27.956 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5512", 2024-09-16T20:50:27.956 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:50:27.956 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 450 2024-09-16T20:50:27.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.957 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:27.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.957 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:27.957 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:50:27.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:27.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.957 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:27.957 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:50:27.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:27.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.957 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:27.958 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562512+0000", 2024-09-16T20:50:27.958 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.009e-06 2024-09-16T20:50:27.958 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.958 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.958 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:27.958 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562517+0000", 2024-09-16T20:50:27.958 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9339999999999997e-06 2024-09-16T20:50:27.959 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.959 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.959 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:27.959 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562530+0000", 2024-09-16T20:50:27.959 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3495e-05 2024-09-16T20:50:27.959 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.959 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.959 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:27.959 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562536+0000", 2024-09-16T20:50:27.959 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5690000000000004e-06 2024-09-16T20:50:27.959 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.959 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.959 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:27.960 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562591+0000", 2024-09-16T20:50:27.960 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5761999999999999e-05 2024-09-16T20:50:27.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.960 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:50:27.960 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562599+0000", 2024-09-16T20:50:27.960 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9179999999999994e-06 2024-09-16T20:50:27.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.961 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.961 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:27.961 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.565971+0000", 2024-09-16T20:50:27.961 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003371546 2024-09-16T20:50:27.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.961 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.961 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:27.961 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566019+0000", 2024-09-16T20:50:27.962 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7849000000000002e-05 2024-09-16T20:50:27.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.962 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:27.962 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.569382+0000", 2024-09-16T20:50:27.962 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.003362833 2024-09-16T20:50:27.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.962 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:27.962 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.569529+0000", 2024-09-16T20:50:27.962 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000147823 2024-09-16T20:50:27.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.963 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:27.963 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570969+0000", 2024-09-16T20:50:27.963 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001439456 2024-09-16T20:50:27.963 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:27.963 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:27.963 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:27.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.963 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.963 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:14996 3.2 3:44e0cffa:::rbd_header.110ef0d7a535:head [watch reconnect cookie 140213436269840 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:50:27.963 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:50:27.963 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.385693072000002, 2024-09-16T20:50:27.963 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.008813943, 2024-09-16T20:50:27.963 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:27.963 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:27.964 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:27.964 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:27.964 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:50:27.964 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:27.964 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 14996 2024-09-16T20:50:27.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.964 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:27.964 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.964 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:27.964 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:50:27.964 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:27.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.964 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.965 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:27.965 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:50:27.965 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:27.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.965 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.965 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:27.965 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562560+0000", 2024-09-16T20:50:27.965 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.06e-07 2024-09-16T20:50:27.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.965 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.965 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:27.965 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562563+0000", 2024-09-16T20:50:27.965 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.495e-06 2024-09-16T20:50:27.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.966 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:27.966 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562571+0000", 2024-09-16T20:50:27.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1950000000000003e-06 2024-09-16T20:50:27.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.966 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:27.966 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562579+0000", 2024-09-16T20:50:27.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2649999999999999e-06 2024-09-16T20:50:27.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.966 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:27.967 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562673+0000", 2024-09-16T20:50:27.967 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4118999999999998e-05 2024-09-16T20:50:27.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.967 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:50:27.967 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562677+0000", 2024-09-16T20:50:27.967 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5179999999999998e-06 2024-09-16T20:50:27.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.967 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:27.967 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566461+0000", 2024-09-16T20:50:27.967 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0037837259999999998 2024-09-16T20:50:27.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.968 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:27.968 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566468+0000", 2024-09-16T20:50:27.968 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3350000000000001e-06 2024-09-16T20:50:27.968 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.968 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.968 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:27.968 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570306+0000", 2024-09-16T20:50:27.968 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.003837997 2024-09-16T20:50:27.968 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.968 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.968 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:27.970 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570392+0000", 2024-09-16T20:50:27.970 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5872999999999996e-05 2024-09-16T20:50:27.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.970 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.970 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:27.970 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571373+0000", 2024-09-16T20:50:27.970 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00098071399999999993 2024-09-16T20:50:27.970 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:27.970 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:27.970 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:27.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.970 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.971 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5255.0:1931 3.2 3:47882819:::rbd_data.110eb5f95afc.0000000000000009:head [read 217088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:50:27.971 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:50:27.971 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.385654322000001, 2024-09-16T20:50:27.971 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0082752880000001, 2024-09-16T20:50:27.971 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:27.971 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:27.971 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:27.971 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:27.971 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5255", 2024-09-16T20:50:27.971 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:50:27.971 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1931 2024-09-16T20:50:27.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.971 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:27.971 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.972 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:27.972 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:50:27.972 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:27.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.972 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.972 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:27.972 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:50:27.972 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:27.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.972 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.972 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:27.972 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:50:27.972 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9699999999999996e-07 2024-09-16T20:50:27.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.973 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.973 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:27.973 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562601+0000", 2024-09-16T20:50:27.973 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8600000000000001e-06 2024-09-16T20:50:27.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.973 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.973 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:27.973 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562608+0000", 2024-09-16T20:50:27.973 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9e-06 2024-09-16T20:50:27.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.973 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.973 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:27.973 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562615+0000", 2024-09-16T20:50:27.974 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8179999999999998e-06 2024-09-16T20:50:27.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.974 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:27.974 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562703+0000", 2024-09-16T20:50:27.974 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7811000000000003e-05 2024-09-16T20:50:27.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.974 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:50:27.974 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562706+0000", 2024-09-16T20:50:27.974 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4920000000000002e-06 2024-09-16T20:50:27.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.975 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.975 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:27.975 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566685+0000", 2024-09-16T20:50:27.975 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0039792780000000002 2024-09-16T20:50:27.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.975 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.975 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:27.975 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566693+0000", 2024-09-16T20:50:27.975 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4730000000000003e-06 2024-09-16T20:50:27.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.975 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.975 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:27.975 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570737+0000", 2024-09-16T20:50:27.975 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0040436749999999 2024-09-16T20:50:27.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.976 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.976 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:27.976 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570873+0000", 2024-09-16T20:50:27.976 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000136484 2024-09-16T20:50:27.976 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:27.976 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:27.976 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:27.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.976 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.976 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5255.0:1939 3.2 3:47882819:::rbd_data.110eb5f95afc.0000000000000009:head [read 233472~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:50:27.976 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:50:27.976 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.385499455999998, 2024-09-16T20:50:27.976 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0084657589999999, 2024-09-16T20:50:27.976 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:27.977 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:27.977 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:27.977 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:27.977 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5255", 2024-09-16T20:50:27.977 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:50:27.977 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1939 2024-09-16T20:50:27.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.977 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:27.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.977 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:27.977 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:50:27.977 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:27.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.977 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:27.978 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:50:27.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:27.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.978 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.978 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:27.978 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:50:27.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5199999999999997e-07 2024-09-16T20:50:27.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.978 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.978 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:27.978 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562757+0000", 2024-09-16T20:50:27.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4979999999999998e-06 2024-09-16T20:50:27.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.978 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.979 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:27.979 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562764+0000", 2024-09-16T20:50:27.979 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1470000000000004e-06 2024-09-16T20:50:27.979 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.979 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.979 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:27.979 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562768+0000", 2024-09-16T20:50:27.979 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7929999999999999e-06 2024-09-16T20:50:27.979 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.979 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.979 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:27.979 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562805+0000", 2024-09-16T20:50:27.979 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7477e-05 2024-09-16T20:50:27.979 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.980 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.980 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting_for_map not empty", 2024-09-16T20:50:27.980 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562810+0000", 2024-09-16T20:50:27.980 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0819999999999998e-06 2024-09-16T20:50:27.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.980 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.980 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:27.980 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566717+0000", 2024-09-16T20:50:27.980 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0039072600000000001 2024-09-16T20:50:27.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.980 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.980 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:27.980 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566724+0000", 2024-09-16T20:50:27.980 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4880000000000004e-06 2024-09-16T20:50:27.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.981 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.981 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:27.981 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571009+0000", 2024-09-16T20:50:27.981 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.004284677 2024-09-16T20:50:27.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.981 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.981 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:27.981 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571218+0000", 2024-09-16T20:50:27.981 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00020978500000000001 2024-09-16T20:50:27.981 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:27.981 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:27.981 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:27.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.981 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.982 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:11020 3.1f 3:fee2d1dd:::rbd_data.110ea93691bb.000000000000000e:23 [read 955392~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:27.982 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.693811+0000", 2024-09-16T20:50:27.982 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 37.254440799000001, 2024-09-16T20:50:27.982 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056833005999999998, 2024-09-16T20:50:27.982 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:27.982 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:27.982 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:27.982 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:27.982 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:27.982 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:27.982 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 11020 2024-09-16T20:50:27.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.982 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:27.982 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.983 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:27.983 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693811+0000", 2024-09-16T20:50:27.983 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:27.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.983 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.983 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:27.983 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693811+0000", 2024-09-16T20:50:27.983 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:27.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.983 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.983 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:27.983 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693812+0000", 2024-09-16T20:50:27.983 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6100000000000001e-07 2024-09-16T20:50:27.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.983 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.983 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:27.984 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693815+0000", 2024-09-16T20:50:27.984 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7389999999999999e-06 2024-09-16T20:50:27.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.984 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.984 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:27.984 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693823+0000", 2024-09-16T20:50:27.984 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0439999999999996e-06 2024-09-16T20:50:27.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.984 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.984 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:27.985 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693828+0000", 2024-09-16T20:50:27.985 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7440000000000004e-06 2024-09-16T20:50:27.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.985 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.985 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:27.985 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750357+0000", 2024-09-16T20:50:27.985 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056529192999999998 2024-09-16T20:50:27.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.985 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.985 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:27.985 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750429+0000", 2024-09-16T20:50:27.985 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1557000000000006e-05 2024-09-16T20:50:27.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.986 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.986 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:27.986 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750644+0000", 2024-09-16T20:50:27.986 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00021526800000000001 2024-09-16T20:50:27.986 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:27.986 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:27.986 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:27.986 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.986 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.986 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:11021 3.1f 3:fee2d1dd:::rbd_data.110ea93691bb.000000000000000e:23 [read 960512~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:27.986 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.693837+0000", 2024-09-16T20:50:27.986 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 37.254414910999998, 2024-09-16T20:50:27.986 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057155138000000001, 2024-09-16T20:50:27.986 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:27.986 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:27.987 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:27.987 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:27.987 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:27.987 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:27.987 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 11021 2024-09-16T20:50:27.987 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.987 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:27.987 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.987 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:27.987 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693837+0000", 2024-09-16T20:50:27.987 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:27.987 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.987 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.987 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:27.987 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693837+0000", 2024-09-16T20:50:27.988 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:27.988 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.988 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.988 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:27.988 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699566+0000", 2024-09-16T20:50:27.988 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057283639999999997 2024-09-16T20:50:27.988 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.988 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.988 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:27.988 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699571+0000", 2024-09-16T20:50:27.988 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.604e-06 2024-09-16T20:50:27.988 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.988 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.988 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:27.989 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699583+0000", 2024-09-16T20:50:27.989 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1372999999999999e-05 2024-09-16T20:50:27.989 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.989 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.989 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:27.989 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699591+0000", 2024-09-16T20:50:27.989 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5259999999999993e-06 2024-09-16T20:50:27.989 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.989 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.989 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:27.989 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750674+0000", 2024-09-16T20:50:27.989 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051083154999999998 2024-09-16T20:50:27.989 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.989 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.990 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:27.990 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750745+0000", 2024-09-16T20:50:27.990 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0933999999999995e-05 2024-09-16T20:50:27.990 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.990 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.990 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:27.990 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750992+0000", 2024-09-16T20:50:27.990 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00024718199999999998 2024-09-16T20:50:27.990 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:27.990 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:27.990 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:27.990 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.990 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.990 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8790 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2590720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:27.990 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057257+0000", 2024-09-16T20:50:27.991 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 26.890995590999999, 2024-09-16T20:50:27.991 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056791169000000002, 2024-09-16T20:50:27.991 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:27.991 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:27.991 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:27.991 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:27.991 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:27.991 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:27.991 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8790 2024-09-16T20:50:27.991 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.991 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:27.991 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.991 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:27.991 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057257+0000", 2024-09-16T20:50:27.991 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:27.992 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.992 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.992 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:27.992 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057257+0000", 2024-09-16T20:50:27.992 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:27.992 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.992 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.992 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:27.992 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057257+0000", 2024-09-16T20:50:27.992 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6600000000000002e-07 2024-09-16T20:50:27.992 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.992 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.992 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:27.992 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057260+0000", 2024-09-16T20:50:27.993 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7240000000000001e-06 2024-09-16T20:50:27.993 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.993 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.993 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:27.993 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057267+0000", 2024-09-16T20:50:27.993 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0600000000000002e-06 2024-09-16T20:50:27.993 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.993 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.993 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:27.993 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057272+0000", 2024-09-16T20:50:27.993 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0390000000000004e-06 2024-09-16T20:50:27.993 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.993 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.993 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:27.993 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.113950+0000", 2024-09-16T20:50:27.994 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056677989999999998 2024-09-16T20:50:27.994 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.994 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.994 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:27.994 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.113993+0000", 2024-09-16T20:50:27.994 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3205999999999999e-05 2024-09-16T20:50:27.994 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.994 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.994 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:27.994 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114048+0000", 2024-09-16T20:50:27.994 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4684e-05 2024-09-16T20:50:27.994 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:27.995 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:27.995 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:27.995 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.995 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.995 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8791 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2332672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:27.995 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057279+0000", 2024-09-16T20:50:27.995 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 26.890973175999999, 2024-09-16T20:50:27.995 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056890067000000002, 2024-09-16T20:50:27.995 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:27.995 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:27.995 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:27.995 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:27.995 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:27.996 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:27.996 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8791 2024-09-16T20:50:27.996 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.996 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:27.996 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.996 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:27.996 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057279+0000", 2024-09-16T20:50:27.996 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:27.996 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.996 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.996 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:27.996 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057279+0000", 2024-09-16T20:50:27.996 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:27.996 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.997 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.997 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:27.997 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057279+0000", 2024-09-16T20:50:27.997 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.15e-07 2024-09-16T20:50:27.997 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.997 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.997 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:27.997 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057281+0000", 2024-09-16T20:50:27.997 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8080000000000001e-06 2024-09-16T20:50:27.997 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.997 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.997 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:27.997 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057287+0000", 2024-09-16T20:50:27.997 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5999999999999997e-06 2024-09-16T20:50:27.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.998 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.998 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:27.998 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057291+0000", 2024-09-16T20:50:27.998 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5990000000000004e-06 2024-09-16T20:50:27.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.998 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.998 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:27.998 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114078+0000", 2024-09-16T20:50:27.998 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056786246999999998 2024-09-16T20:50:27.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.998 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.998 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:27.999 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114118+0000", 2024-09-16T20:50:27.999 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9898999999999999e-05 2024-09-16T20:50:27.999 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.999 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.999 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:27.999 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114169+0000", 2024-09-16T20:50:27.999 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1499000000000003e-05 2024-09-16T20:50:27.999 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:27.999 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:27.999 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:27.999 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:27.999 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:27.999 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8792 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2609152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:27.999 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:50:28.000 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 26.890955194, 2024-09-16T20:50:28.000 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057003524999999999, 2024-09-16T20:50:28.000 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:28.000 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:28.000 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:28.000 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:28.000 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:28.000 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:28.000 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8792 2024-09-16T20:50:28.000 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.000 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:28.000 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.000 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:28.000 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:50:28.001 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:28.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.001 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.001 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:28.001 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:50:28.001 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:28.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.001 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.001 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:28.001 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:50:28.001 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0400000000000002e-07 2024-09-16T20:50:28.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.001 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.001 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:28.002 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057299+0000", 2024-09-16T20:50:28.002 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7460000000000001e-06 2024-09-16T20:50:28.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.002 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:28.002 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057304+0000", 2024-09-16T20:50:28.002 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3210000000000004e-06 2024-09-16T20:50:28.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.002 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:28.002 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057310+0000", 2024-09-16T20:50:28.002 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4360000000000001e-06 2024-09-16T20:50:28.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.002 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:28.003 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114200+0000", 2024-09-16T20:50:28.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056889954 2024-09-16T20:50:28.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.003 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.003 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:28.003 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114243+0000", 2024-09-16T20:50:28.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2998999999999999e-05 2024-09-16T20:50:28.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.003 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.003 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:28.003 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114300+0000", 2024-09-16T20:50:28.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7765e-05 2024-09-16T20:50:28.003 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:28.003 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:28.004 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:28.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.004 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.004 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8794 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2351104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:28.004 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057327+0000", 2024-09-16T20:50:28.004 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 26.890924869999999, 2024-09-16T20:50:28.004 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057101130999999999, 2024-09-16T20:50:28.004 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:28.004 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:28.004 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:28.004 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:28.004 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:28.004 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:28.005 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8794 2024-09-16T20:50:28.005 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.005 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:28.005 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.005 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:28.005 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057327+0000", 2024-09-16T20:50:28.005 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:28.005 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.005 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.006 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:28.006 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057327+0000", 2024-09-16T20:50:28.006 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:28.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.006 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.006 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:28.006 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057328+0000", 2024-09-16T20:50:28.006 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7399999999999999e-07 2024-09-16T20:50:28.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.006 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.006 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:28.006 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057330+0000", 2024-09-16T20:50:28.006 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2199999999999999e-06 2024-09-16T20:50:28.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.007 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.007 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:28.007 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057336+0000", 2024-09-16T20:50:28.007 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1240000000000002e-06 2024-09-16T20:50:28.007 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.007 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.007 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:28.007 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057341+0000", 2024-09-16T20:50:28.007 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8470000000000003e-06 2024-09-16T20:50:28.007 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.007 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.007 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:28.007 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114334+0000", 2024-09-16T20:50:28.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056993267 2024-09-16T20:50:28.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.008 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:28.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114376+0000", 2024-09-16T20:50:28.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1638999999999999e-05 2024-09-16T20:50:28.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.008 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:28.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114428+0000", 2024-09-16T20:50:28.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2660000000000001e-05 2024-09-16T20:50:28.008 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:28.008 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:28.008 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:28.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.009 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.009 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8798 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2297856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:28.009 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:50:28.009 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 26.890905143000001, 2024-09-16T20:50:28.009 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057277041000000001, 2024-09-16T20:50:28.009 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:28.009 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:28.009 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:28.009 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:28.009 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:28.009 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:28.009 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8798 2024-09-16T20:50:28.010 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.010 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:28.010 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.010 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:28.010 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:50:28.010 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:28.010 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.010 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.010 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:28.010 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:50:28.010 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:28.010 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.010 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.011 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:28.011 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:50:28.011 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2800000000000003e-07 2024-09-16T20:50:28.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.011 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.011 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:28.011 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057354+0000", 2024-09-16T20:50:28.011 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7780000000000002e-06 2024-09-16T20:50:28.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.011 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.011 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:28.011 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057360+0000", 2024-09-16T20:50:28.011 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0419999999999999e-06 2024-09-16T20:50:28.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.012 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.012 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:28.012 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057365+0000", 2024-09-16T20:50:28.012 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7929999999999997e-06 2024-09-16T20:50:28.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.012 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.012 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:28.012 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114460+0000", 2024-09-16T20:50:28.012 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057094662999999997 2024-09-16T20:50:28.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.012 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.012 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:28.012 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114532+0000", 2024-09-16T20:50:28.013 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2881999999999998e-05 2024-09-16T20:50:28.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.013 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.013 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:28.013 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114624+0000", 2024-09-16T20:50:28.013 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1555e-05 2024-09-16T20:50:28.013 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:28.013 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:28.013 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:28.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.013 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.013 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8800 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2316288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:28.013 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:50:28.013 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 26.890881203999999, 2024-09-16T20:50:28.014 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.05746946, 2024-09-16T20:50:28.014 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:28.014 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:28.014 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:28.014 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:28.014 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:28.014 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:28.014 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8800 2024-09-16T20:50:28.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.014 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:28.014 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.014 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:28.014 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:50:28.014 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:28.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.015 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.015 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:28.015 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:50:28.015 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:28.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.015 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.015 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:28.015 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:50:28.015 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5199999999999998e-07 2024-09-16T20:50:28.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.015 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.015 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:28.016 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057373+0000", 2024-09-16T20:50:28.016 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.79e-06 2024-09-16T20:50:28.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.016 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.016 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:28.016 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057379+0000", 2024-09-16T20:50:28.016 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6699999999999999e-06 2024-09-16T20:50:28.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.016 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.016 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:28.016 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057386+0000", 2024-09-16T20:50:28.016 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3170000000000004e-06 2024-09-16T20:50:28.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.016 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.017 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:28.017 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114655+0000", 2024-09-16T20:50:28.017 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057269420000000001 2024-09-16T20:50:28.017 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.017 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.017 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:28.017 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114734+0000", 2024-09-16T20:50:28.017 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8084000000000007e-05 2024-09-16T20:50:28.017 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.017 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.017 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:28.017 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114840+0000", 2024-09-16T20:50:28.017 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000106827 2024-09-16T20:50:28.018 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:28.018 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:28.018 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:28.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.018 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.018 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8801 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2334720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:28.018 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:50:28.018 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 26.890860173, 2024-09-16T20:50:28.018 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057594682000000001, 2024-09-16T20:50:28.018 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:28.018 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:28.018 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:28.018 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:28.018 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:28.018 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:28.019 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8801 2024-09-16T20:50:28.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.019 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:28.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.019 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:28.019 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:50:28.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:28.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.019 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:28.019 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:50:28.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:28.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.020 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:28.020 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:50:28.020 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6899999999999998e-07 2024-09-16T20:50:28.020 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.020 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.020 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:28.020 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057394+0000", 2024-09-16T20:50:28.020 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8300000000000001e-06 2024-09-16T20:50:28.020 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.020 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.020 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:28.020 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057400+0000", 2024-09-16T20:50:28.020 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6500000000000001e-06 2024-09-16T20:50:28.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.021 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.021 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:28.021 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057404+0000", 2024-09-16T20:50:28.021 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4669999999999998e-06 2024-09-16T20:50:28.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.021 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.021 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:28.021 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114874+0000", 2024-09-16T20:50:28.021 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057469602000000002 2024-09-16T20:50:28.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.021 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.021 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:28.021 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114933+0000", 2024-09-16T20:50:28.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9500000000000003e-05 2024-09-16T20:50:28.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.022 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.022 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:28.022 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114987+0000", 2024-09-16T20:50:28.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3263999999999997e-05 2024-09-16T20:50:28.022 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:28.022 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:28.022 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:28.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.022 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.022 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8802 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2353152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:28.022 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:50:28.022 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 26.890811488000001, 2024-09-16T20:50:28.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057714079000000001, 2024-09-16T20:50:28.023 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:28.023 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:28.023 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:28.023 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:28.023 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:28.023 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:28.023 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8802 2024-09-16T20:50:28.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.023 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:28.023 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.023 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:28.023 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:50:28.023 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:28.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.024 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:28.024 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:50:28.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:28.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.024 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:28.024 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:50:28.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4999999999999998e-07 2024-09-16T20:50:28.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.024 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:28.024 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057446+0000", 2024-09-16T20:50:28.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4240000000000003e-06 2024-09-16T20:50:28.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:28.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057452+0000", 2024-09-16T20:50:28.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8379999999999996e-06 2024-09-16T20:50:28.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:28.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057460+0000", 2024-09-16T20:50:28.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9629999999999998e-06 2024-09-16T20:50:28.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:28.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115015+0000", 2024-09-16T20:50:28.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057554389999999997 2024-09-16T20:50:28.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.026 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:28.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115082+0000", 2024-09-16T20:50:28.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7733999999999999e-05 2024-09-16T20:50:28.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.026 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:28.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115155+0000", 2024-09-16T20:50:28.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2280000000000006e-05 2024-09-16T20:50:28.026 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:28.027 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:28.027 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:28.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.027 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.027 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8804 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2555904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:28.027 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:50:28.027 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 26.890785172000001, 2024-09-16T20:50:28.027 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057833862, 2024-09-16T20:50:28.027 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:28.027 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:28.027 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:28.027 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:28.027 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:28.027 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:28.028 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8804 2024-09-16T20:50:28.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.028 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:28.028 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.028 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:28.028 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:50:28.028 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:28.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.028 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.028 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:28.028 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:50:28.028 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:28.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.029 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:28.029 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:50:28.029 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7899999999999999e-07 2024-09-16T20:50:28.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.029 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:28.029 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057469+0000", 2024-09-16T20:50:28.029 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.001e-06 2024-09-16T20:50:28.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.029 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:28.029 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057475+0000", 2024-09-16T20:50:28.029 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7629999999999997e-06 2024-09-16T20:50:28.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.030 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:28.030 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057480+0000", 2024-09-16T20:50:28.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8069999999999999e-06 2024-09-16T20:50:28.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.030 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:28.030 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115183+0000", 2024-09-16T20:50:28.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057703364 2024-09-16T20:50:28.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.030 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:28.030 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115227+0000", 2024-09-16T20:50:28.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3903999999999998e-05 2024-09-16T20:50:28.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.031 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.031 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:28.031 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115301+0000", 2024-09-16T20:50:28.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3543999999999999e-05 2024-09-16T20:50:28.031 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:28.031 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:28.031 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:28.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.031 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.031 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8805 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2574336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:28.031 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057487+0000", 2024-09-16T20:50:28.031 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 26.890764844, 2024-09-16T20:50:28.032 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058021027000000003, 2024-09-16T20:50:28.032 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:28.032 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:28.032 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:28.032 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:28.032 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:28.032 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:28.032 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8805 2024-09-16T20:50:28.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.032 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:28.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.032 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:28.032 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057487+0000", 2024-09-16T20:50:28.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:28.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.033 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.033 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:28.033 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057487+0000", 2024-09-16T20:50:28.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:28.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.033 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.033 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:28.033 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057488+0000", 2024-09-16T20:50:28.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5199999999999998e-07 2024-09-16T20:50:28.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.033 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.033 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:28.034 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057490+0000", 2024-09-16T20:50:28.034 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.939e-06 2024-09-16T20:50:28.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.034 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.034 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:28.034 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057495+0000", 2024-09-16T20:50:28.034 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6439999999999997e-06 2024-09-16T20:50:28.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.034 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.034 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:28.034 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057500+0000", 2024-09-16T20:50:28.034 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8119999999999998e-06 2024-09-16T20:50:28.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.034 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.035 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:28.035 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115341+0000", 2024-09-16T20:50:28.035 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057841341999999997 2024-09-16T20:50:28.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.035 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:28.035 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115422+0000", 2024-09-16T20:50:28.035 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1003999999999999e-05 2024-09-16T20:50:28.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.035 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:28.035 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115508+0000", 2024-09-16T20:50:28.035 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5933999999999994e-05 2024-09-16T20:50:28.035 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:28.036 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:28.036 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:28.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.036 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8807 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2592768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:28.036 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057505+0000", 2024-09-16T20:50:28.036 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 26.890746633999999, 2024-09-16T20:50:28.036 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058186633000000001, 2024-09-16T20:50:28.036 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:28.036 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:28.036 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:28.036 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:28.036 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:28.036 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:28.037 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8807 2024-09-16T20:50:28.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.037 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:28.037 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.037 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:28.037 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057505+0000", 2024-09-16T20:50:28.037 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:28.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.037 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.037 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:28.037 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057505+0000", 2024-09-16T20:50:28.037 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:28.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.037 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.038 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:28.038 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057506+0000", 2024-09-16T20:50:28.038 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.27e-07 2024-09-16T20:50:28.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.038 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.038 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:28.038 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057508+0000", 2024-09-16T20:50:28.038 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7400000000000001e-06 2024-09-16T20:50:28.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.038 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.038 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:28.038 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057513+0000", 2024-09-16T20:50:28.038 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.468e-06 2024-09-16T20:50:28.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.039 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:28.039 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057517+0000", 2024-09-16T20:50:28.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4329999999999997e-06 2024-09-16T20:50:28.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.039 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:28.039 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115538+0000", 2024-09-16T20:50:28.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058020355000000003 2024-09-16T20:50:28.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.039 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:28.039 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115616+0000", 2024-09-16T20:50:28.040 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7998999999999996e-05 2024-09-16T20:50:28.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.040 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.040 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:28.040 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115692+0000", 2024-09-16T20:50:28.040 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6310999999999995e-05 2024-09-16T20:50:28.040 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:28.040 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:28.040 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:28.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.040 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.040 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8808 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2611200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:28.040 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:50:28.040 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 26.890729372999999, 2024-09-16T20:50:28.041 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058354046, 2024-09-16T20:50:28.041 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:28.041 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:28.041 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:28.041 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:28.041 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:28.041 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:28.041 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8808 2024-09-16T20:50:28.041 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.041 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:28.041 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.041 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:28.041 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:50:28.041 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:28.042 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.042 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.042 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:28.042 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:50:28.042 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:28.042 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.042 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.042 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:28.042 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:50:28.042 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2500000000000001e-07 2024-09-16T20:50:28.042 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.042 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.042 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:28.042 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057525+0000", 2024-09-16T20:50:28.043 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5549999999999999e-06 2024-09-16T20:50:28.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.043 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.043 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:28.043 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057530+0000", 2024-09-16T20:50:28.043 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5489999999999997e-06 2024-09-16T20:50:28.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.043 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.043 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:28.043 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057535+0000", 2024-09-16T20:50:28.043 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5709999999999999e-06 2024-09-16T20:50:28.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.043 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.043 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:28.044 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115722+0000", 2024-09-16T20:50:28.044 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058187323999999999 2024-09-16T20:50:28.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.044 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.044 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:28.044 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115799+0000", 2024-09-16T20:50:28.044 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6445000000000004e-05 2024-09-16T20:50:28.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.044 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.044 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:28.044 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115877+0000", 2024-09-16T20:50:28.044 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8276999999999996e-05 2024-09-16T20:50:28.044 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:28.044 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:28.045 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:28.045 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.045 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.045 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:2182 3.17 3:edb5e5cd:::rbd_data.110e61479aee.0000000000000014:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write 0~5120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-16T20:50:28.045 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:22.734609+0000", 2024-09-16T20:50:28.045 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.213643126, 2024-09-16T20:50:28.045 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.124890894, 2024-09-16T20:50:28.045 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:28.045 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:28.045 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:28.045 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:28.045 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:28.046 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:28.046 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 2182 2024-09-16T20:50:28.046 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.046 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:28.046 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.047 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:28.047 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734609+0000", 2024-09-16T20:50:28.047 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:28.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.047 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.047 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:28.047 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734609+0000", 2024-09-16T20:50:28.047 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:28.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.047 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.047 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:28.048 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734611+0000", 2024-09-16T20:50:28.048 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9599999999999999e-06 2024-09-16T20:50:28.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.048 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.048 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:28.048 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734628+0000", 2024-09-16T20:50:28.048 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.715e-05 2024-09-16T20:50:28.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.048 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.048 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:28.048 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734656+0000", 2024-09-16T20:50:28.048 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7759999999999998e-05 2024-09-16T20:50:28.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.049 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.049 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:28.049 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734669+0000", 2024-09-16T20:50:28.049 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3315999999999999e-05 2024-09-16T20:50:28.049 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.049 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.049 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:28.049 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734826+0000", 2024-09-16T20:50:28.049 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015668499999999999 2024-09-16T20:50:28.049 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.049 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.049 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:28.049 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.735048+0000", 2024-09-16T20:50:28.049 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00022179 2024-09-16T20:50:28.050 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.050 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.050 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:28.050 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.735248+0000", 2024-09-16T20:50:28.050 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00020010399999999999 2024-09-16T20:50:28.050 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.050 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.050 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:50:28.050 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.736252+0000", 2024-09-16T20:50:28.050 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010039739999999999 2024-09-16T20:50:28.050 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.050 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.050 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:28.051 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.859340+0000", 2024-09-16T20:50:28.051 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.12308846599999999 2024-09-16T20:50:28.051 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.051 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.051 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:50:28.051 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.859431+0000", 2024-09-16T20:50:28.051 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1099000000000003e-05 2024-09-16T20:50:28.051 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:28.051 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:28.051 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:28.051 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.859500+0000", 2024-09-16T20:50:28.051 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8590000000000006e-05 2024-09-16T20:50:28.051 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:28.051 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:28.052 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:28.052 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:28.052 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:28.052 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:28.053 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:50:28.054 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:28.052+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:28.154 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:28.154+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:28.164 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:28.164 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:50:28.164 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:50:28.164 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:28.172 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:50:28.255 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:28.254+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:28.279 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:28.279 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:50:28.279 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:50:28.280 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:50:28.280 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:28.288 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:50:28.356 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:28.356+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:28.396 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:28.396 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:50:28.396 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:50:28.396 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:50:28.396 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.397 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5439 3.a 3:5789f5c3:::rbd_data.110ed9ca540.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 1181696~60416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:28.398 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:50:28.398 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 125.52957690300001, 2024-09-16T20:50:28.398 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10102045599999999, 2024-09-16T20:50:28.398 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:28.398 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:28.398 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:28.398 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:28.398 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:28.398 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:28.399 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5439 2024-09-16T20:50:28.399 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.399 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:28.399 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.399 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:28.399 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:50:28.399 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.399 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.399 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.399 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:28.399 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:50:28.399 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.399 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.399 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.399 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:28.400 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864711+0000", 2024-09-16T20:50:28.400 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7899999999999997e-07 2024-09-16T20:50:28.400 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.400 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.400 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:28.400 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864760+0000", 2024-09-16T20:50:28.400 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8803e-05 2024-09-16T20:50:28.400 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.400 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.400 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:28.400 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864769+0000", 2024-09-16T20:50:28.400 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0999999999999993e-06 2024-09-16T20:50:28.400 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.400 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.401 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:28.401 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864774+0000", 2024-09-16T20:50:28.401 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8919999999999999e-06 2024-09-16T20:50:28.401 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.401 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.401 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:28.401 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864815+0000", 2024-09-16T20:50:28.401 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1307000000000003e-05 2024-09-16T20:50:28.401 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.401 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.401 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:28.401 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864893+0000", 2024-09-16T20:50:28.401 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7581000000000001e-05 2024-09-16T20:50:28.402 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.402 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.402 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 1", 2024-09-16T20:50:28.402 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864996+0000", 2024-09-16T20:50:28.402 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102725 2024-09-16T20:50:28.402 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.402 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.402 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:28.402 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965442+0000", 2024-09-16T20:50:28.402 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100446012 2024-09-16T20:50:28.402 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.402 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.402 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:28.402 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965618+0000", 2024-09-16T20:50:28.403 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00017653100000000001 2024-09-16T20:50:28.403 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.403 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.403 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:28.403 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965682+0000", 2024-09-16T20:50:28.403 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3905999999999994e-05 2024-09-16T20:50:28.403 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.403 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.403 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:28.403 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965731+0000", 2024-09-16T20:50:28.403 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8819999999999997e-05 2024-09-16T20:50:28.403 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.404 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:28.404 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.404 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.404 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.404 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4600.0:5439 3.a e37/23 ondisk, result = 0)", 2024-09-16T20:50:28.404 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:50:28.404 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 125.52858075499999, 2024-09-16T20:50:28.404 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099742634999999996, 2024-09-16T20:50:28.404 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:28.404 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:28.404 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:28.404 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:28.404 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.405 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:28.405 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:50:28.405 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.405 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.405 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.405 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:28.405 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:50:28.405 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.405 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.405 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.405 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:28.405 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:50:28.405 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0900000000000001e-07 2024-09-16T20:50:28.406 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.406 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.406 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:28.406 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865711+0000", 2024-09-16T20:50:28.406 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3739999999999998e-06 2024-09-16T20:50:28.406 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.406 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.406 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:28.406 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965343+0000", 2024-09-16T20:50:28.406 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099632526999999999 2024-09-16T20:50:28.406 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.406 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.406 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:28.406 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965357+0000", 2024-09-16T20:50:28.407 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3713999999999999e-05 2024-09-16T20:50:28.407 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.407 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.407 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:28.407 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965426+0000", 2024-09-16T20:50:28.407 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8745999999999995e-05 2024-09-16T20:50:28.407 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.407 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.407 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:28.407 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965434+0000", 2024-09-16T20:50:28.407 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6910000000000003e-06 2024-09-16T20:50:28.407 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.407 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.407 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:28.407 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965449+0000", 2024-09-16T20:50:28.408 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4874e-05 2024-09-16T20:50:28.408 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.408 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:28.408 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.408 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.408 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.408 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4882.0:2508 3.12 3:4992771f:::rbd_object_map.110e7c8851c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T20:50:28.408 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:50:28.408 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 101.912777207, 2024-09-16T20:50:28.408 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.125007011, 2024-09-16T20:50:28.408 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:28.408 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:28.408 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:28.409 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:28.409 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4882", 2024-09-16T20:50:28.409 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2436923223", 2024-09-16T20:50:28.409 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2508 2024-09-16T20:50:28.409 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.409 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:28.409 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.409 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:28.409 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:50:28.409 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.409 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.409 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.409 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:28.410 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:50:28.410 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.410 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.410 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.410 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:28.410 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481511+0000", 2024-09-16T20:50:28.410 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0699999999999996e-07 2024-09-16T20:50:28.410 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.410 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.410 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:28.410 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481515+0000", 2024-09-16T20:50:28.410 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0609999999999997e-06 2024-09-16T20:50:28.410 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.410 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.410 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:28.411 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481523+0000", 2024-09-16T20:50:28.411 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1310000000000006e-06 2024-09-16T20:50:28.411 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.411 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.411 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:28.411 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481528+0000", 2024-09-16T20:50:28.411 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4900000000000002e-06 2024-09-16T20:50:28.411 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.411 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.411 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:28.411 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481565+0000", 2024-09-16T20:50:28.411 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7066000000000001e-05 2024-09-16T20:50:28.411 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.411 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.411 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:28.412 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481628+0000", 2024-09-16T20:50:28.412 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3118000000000001e-05 2024-09-16T20:50:28.412 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.412 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.412 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 2", 2024-09-16T20:50:28.412 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481822+0000", 2024-09-16T20:50:28.412 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019446200000000001 2024-09-16T20:50:28.412 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.412 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.412 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:28.412 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605988+0000", 2024-09-16T20:50:28.412 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.124165888 2024-09-16T20:50:28.412 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.413 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.413 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:28.413 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606427+0000", 2024-09-16T20:50:28.413 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00043854899999999999 2024-09-16T20:50:28.413 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.413 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.413 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:28.413 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606476+0000", 2024-09-16T20:50:28.413 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8971999999999998e-05 2024-09-16T20:50:28.413 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.413 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.413 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:28.414 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606517+0000", 2024-09-16T20:50:28.414 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1467000000000001e-05 2024-09-16T20:50:28.414 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.414 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:28.414 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.414 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.414 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.414 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4882.0:2508 3.12 e57/55 ondisk, result = 0)", 2024-09-16T20:50:28.414 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:50:28.414 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 101.911788707, 2024-09-16T20:50:28.414 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12349711400000001, 2024-09-16T20:50:28.414 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:28.414 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:28.414 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:28.415 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:28.415 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.415 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:28.415 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:50:28.415 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.415 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.415 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.415 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:28.415 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:50:28.415 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.415 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.415 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.415 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:28.415 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:50:28.415 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.06e-07 2024-09-16T20:50:28.415 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.416 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.416 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:28.416 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482502+0000", 2024-09-16T20:50:28.416 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.01e-06 2024-09-16T20:50:28.416 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.416 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.416 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:28.416 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605772+0000", 2024-09-16T20:50:28.416 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.123269687 2024-09-16T20:50:28.416 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.416 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.416 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:28.416 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605791+0000", 2024-09-16T20:50:28.416 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8912000000000001e-05 2024-09-16T20:50:28.417 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.417 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.417 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:28.417 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605974+0000", 2024-09-16T20:50:28.417 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000182836 2024-09-16T20:50:28.417 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.417 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.417 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:28.417 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605982+0000", 2024-09-16T20:50:28.417 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1699999999999997e-06 2024-09-16T20:50:28.418 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.418 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.418 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:28.418 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605996+0000", 2024-09-16T20:50:28.418 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3893000000000001e-05 2024-09-16T20:50:28.418 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.418 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:28.418 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.418 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.418 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.418 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4882.0:2512 3.f 3:f4d895b7:::rbd_data.110e7c8851c.0000000000000000:head [read 16384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T20:50:28.418 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:50:28.418 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 101.911351677, 2024-09-16T20:50:28.418 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12318483, 2024-09-16T20:50:28.418 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:28.419 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:28.419 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:28.419 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:28.419 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4882", 2024-09-16T20:50:28.419 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2436923223", 2024-09-16T20:50:28.419 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2512 2024-09-16T20:50:28.419 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.419 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:28.419 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.419 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:28.419 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:50:28.419 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.419 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.419 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.420 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:28.420 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:50:28.420 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.420 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.420 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.420 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:28.420 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:50:28.420 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3499999999999995e-07 2024-09-16T20:50:28.420 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.420 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.420 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:28.420 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482943+0000", 2024-09-16T20:50:28.420 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3450000000000002e-06 2024-09-16T20:50:28.421 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.421 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.421 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:28.421 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605772+0000", 2024-09-16T20:50:28.421 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.122829796 2024-09-16T20:50:28.421 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.421 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.421 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:28.421 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605792+0000", 2024-09-16T20:50:28.421 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9916999999999999e-05 2024-09-16T20:50:28.421 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.421 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.421 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:28.422 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605917+0000", 2024-09-16T20:50:28.422 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00012500799999999999 2024-09-16T20:50:28.422 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.422 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.422 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:28.422 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606004+0000", 2024-09-16T20:50:28.422 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6242999999999994e-05 2024-09-16T20:50:28.422 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.422 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.422 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:28.422 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606120+0000", 2024-09-16T20:50:28.422 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000116786 2024-09-16T20:50:28.423 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.423 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:28.423 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.423 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.423 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.423 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:11702 3.15 3:acf33c8b:::rbd_data.110e747f0779.000000000000001a:head [set-alloc-hint object_size 262144 write_size 262144,write 143360~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:50:28.423 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:50:28.423 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 84.256359255000007, 2024-09-16T20:50:28.423 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.084801043000000006, 2024-09-16T20:50:28.423 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:28.423 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:28.423 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:28.423 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:28.423 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:50:28.424 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:50:28.424 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 11702 2024-09-16T20:50:28.424 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.424 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:28.424 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.424 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:28.424 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:50:28.424 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.424 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.424 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.424 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:28.424 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:50:28.425 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.425 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.425 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.425 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:28.425 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137929+0000", 2024-09-16T20:50:28.425 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8400000000000002e-07 2024-09-16T20:50:28.425 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.425 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.425 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:28.425 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137937+0000", 2024-09-16T20:50:28.425 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0260000000000007e-06 2024-09-16T20:50:28.425 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.425 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.426 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:28.426 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137952+0000", 2024-09-16T20:50:28.426 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4659e-05 2024-09-16T20:50:28.426 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.426 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.426 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:28.426 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137960+0000", 2024-09-16T20:50:28.426 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0169999999999996e-06 2024-09-16T20:50:28.426 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.426 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.426 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:28.426 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138010+0000", 2024-09-16T20:50:28.426 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0021000000000001e-05 2024-09-16T20:50:28.427 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.427 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.427 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:28.427 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138114+0000", 2024-09-16T20:50:28.427 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000103797 2024-09-16T20:50:28.427 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.427 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.427 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:50:28.427 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138233+0000", 2024-09-16T20:50:28.427 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000118971 2024-09-16T20:50:28.427 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.427 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.427 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:28.427 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138953+0000", 2024-09-16T20:50:28.428 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00072026000000000002 2024-09-16T20:50:28.428 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.428 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.428 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:28.428 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222609+0000", 2024-09-16T20:50:28.428 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.083656650999999999 2024-09-16T20:50:28.428 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.428 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.428 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:28.428 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222684+0000", 2024-09-16T20:50:28.428 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4369999999999994e-05 2024-09-16T20:50:28.428 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.429 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.429 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:28.429 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222729+0000", 2024-09-16T20:50:28.429 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5287000000000002e-05 2024-09-16T20:50:28.429 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.429 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:28.429 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.429 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.429 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.429 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:11704 3.9 3:91220b75:::rbd_data.110e747f0779.000000000000001c:head [set-alloc-hint object_size 262144 write_size 262144,write 139264~6656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:50:28.429 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:50:28.429 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 84.256121254000007, 2024-09-16T20:50:28.429 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.084531133999999994, 2024-09-16T20:50:28.429 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:28.430 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:28.430 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:28.430 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:28.430 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:50:28.430 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:50:28.430 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 11704 2024-09-16T20:50:28.430 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.430 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:28.430 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.430 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:28.430 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:50:28.430 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.430 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.431 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.431 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:28.431 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:50:28.431 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.431 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.431 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.431 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:28.431 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138167+0000", 2024-09-16T20:50:28.431 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.33e-07 2024-09-16T20:50:28.431 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.431 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.431 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:28.431 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138179+0000", 2024-09-16T20:50:28.432 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2055999999999999e-05 2024-09-16T20:50:28.432 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.432 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.432 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:28.432 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138193+0000", 2024-09-16T20:50:28.432 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3961000000000001e-05 2024-09-16T20:50:28.432 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.432 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.432 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:28.432 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138200+0000", 2024-09-16T20:50:28.432 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4429999999999997e-06 2024-09-16T20:50:28.432 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.432 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.432 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:28.433 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138258+0000", 2024-09-16T20:50:28.433 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7225999999999997e-05 2024-09-16T20:50:28.433 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.433 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.433 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:28.433 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138364+0000", 2024-09-16T20:50:28.433 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000106013 2024-09-16T20:50:28.433 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.433 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.433 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:50:28.433 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138494+0000", 2024-09-16T20:50:28.433 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013059599999999999 2024-09-16T20:50:28.434 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.434 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.434 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:28.434 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.139220+0000", 2024-09-16T20:50:28.434 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00072539800000000004 2024-09-16T20:50:28.434 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.434 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.434 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:28.434 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222608+0000", 2024-09-16T20:50:28.434 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.083388620999999996 2024-09-16T20:50:28.434 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.434 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.434 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:28.434 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222651+0000", 2024-09-16T20:50:28.435 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2228000000000003e-05 2024-09-16T20:50:28.435 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.435 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.435 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:28.435 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222697+0000", 2024-09-16T20:50:28.435 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6659000000000002e-05 2024-09-16T20:50:28.435 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.435 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:28.435 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.435 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.435 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.435 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5138.0:9358 3.c 3.c25699cc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T20:50:28.435 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:50:28.436 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 61.543973948999998, 2024-09-16T20:50:28.436 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0018657580000001, 2024-09-16T20:50:28.436 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:28.436 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:28.436 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:50:28.436 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:28.436 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5138", 2024-09-16T20:50:28.436 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:50:28.436 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 9358 2024-09-16T20:50:28.436 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.436 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:28.436 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.436 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:28.437 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:50:28.437 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.437 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.437 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.437 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:28.437 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:50:28.437 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.437 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.437 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.437 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:28.437 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850314+0000", 2024-09-16T20:50:28.437 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4099999999999999e-07 2024-09-16T20:50:28.437 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.437 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.438 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:28.438 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850316+0000", 2024-09-16T20:50:28.438 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2469999999999998e-06 2024-09-16T20:50:28.438 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.438 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.438 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:28.438 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850322+0000", 2024-09-16T20:50:28.438 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3470000000000004e-06 2024-09-16T20:50:28.438 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.438 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.438 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:28.438 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850327+0000", 2024-09-16T20:50:28.438 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9450000000000001e-06 2024-09-16T20:50:28.438 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.439 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.439 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:28.439 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850376+0000", 2024-09-16T20:50:28.439 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8257000000000002e-05 2024-09-16T20:50:28.439 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.439 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.439 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:28.439 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850385+0000", 2024-09-16T20:50:28.439 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9800000000000004e-06 2024-09-16T20:50:28.439 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.439 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.439 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:28.439 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852167+0000", 2024-09-16T20:50:28.439 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0017825659999999 2024-09-16T20:50:28.440 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.440 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.440 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:28.440 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852179+0000", 2024-09-16T20:50:28.440 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1975e-05 2024-09-16T20:50:28.440 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.440 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:28.440 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.440 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.440 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.440 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5138.0:9382 3.c 3.c25699cc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T20:50:28.440 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:50:28.440 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 61.535626933000003, 2024-09-16T20:50:28.441 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.99358404700000003, 2024-09-16T20:50:28.441 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:28.441 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:28.441 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:50:28.441 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:28.441 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5138", 2024-09-16T20:50:28.441 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:50:28.441 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 9382 2024-09-16T20:50:28.441 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.441 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:28.441 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.441 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:28.441 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:50:28.442 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.442 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.442 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.442 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:28.442 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:50:28.442 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.442 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.442 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.442 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:28.442 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858661+0000", 2024-09-16T20:50:28.442 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0200000000000002e-07 2024-09-16T20:50:28.442 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.442 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.442 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:28.443 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858664+0000", 2024-09-16T20:50:28.443 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.038e-06 2024-09-16T20:50:28.443 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.443 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.443 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:28.443 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858671+0000", 2024-09-16T20:50:28.443 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0110000000000003e-06 2024-09-16T20:50:28.443 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.443 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.443 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:28.443 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858677+0000", 2024-09-16T20:50:28.443 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0959999999999997e-06 2024-09-16T20:50:28.443 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.443 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.444 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:28.444 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858732+0000", 2024-09-16T20:50:28.444 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4543000000000003e-05 2024-09-16T20:50:28.444 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.444 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.444 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:28.444 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858739+0000", 2024-09-16T20:50:28.444 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5429999999999996e-06 2024-09-16T20:50:28.444 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.444 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.444 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:28.444 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852236+0000", 2024-09-16T20:50:28.445 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.993496453 2024-09-16T20:50:28.445 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.445 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.445 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:28.445 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852244+0000", 2024-09-16T20:50:28.445 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8610000000000004e-06 2024-09-16T20:50:28.445 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.445 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:28.445 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.445 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.445 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.445 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5763.0:4895 3.a 3:5384606c:::rbd_data.110e1b19a241.0000000000000018:head [read 0~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-16T20:50:28.446 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:45.598212+0000", 2024-09-16T20:50:28.446 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 42.796074892, 2024-09-16T20:50:28.446 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0375864829999999, 2024-09-16T20:50:28.446 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:28.446 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:28.446 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:28.446 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:28.446 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5763", 2024-09-16T20:50:28.446 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/623897567", 2024-09-16T20:50:28.446 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4895 2024-09-16T20:50:28.446 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.446 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:28.446 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.447 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:28.447 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598212+0000", 2024-09-16T20:50:28.447 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.447 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.447 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.447 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:28.447 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598212+0000", 2024-09-16T20:50:28.447 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.447 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.447 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.447 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:28.447 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598214+0000", 2024-09-16T20:50:28.448 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0090000000000002e-06 2024-09-16T20:50:28.448 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.448 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.448 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:28.448 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598222+0000", 2024-09-16T20:50:28.448 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6699999999999994e-06 2024-09-16T20:50:28.448 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.449 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.449 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:28.449 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598241+0000", 2024-09-16T20:50:28.449 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8896999999999999e-05 2024-09-16T20:50:28.449 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.449 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.449 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:28.449 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598250+0000", 2024-09-16T20:50:28.449 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6649999999999992e-06 2024-09-16T20:50:28.449 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.449 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.449 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:28.449 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598325+0000", 2024-09-16T20:50:28.450 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5189000000000001e-05 2024-09-16T20:50:28.450 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.450 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.450 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op must wait for map", 2024-09-16T20:50:28.450 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598333+0000", 2024-09-16T20:50:28.450 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.3170000000000002e-06 2024-09-16T20:50:28.450 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.450 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.450 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:28.450 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601355+0000", 2024-09-16T20:50:28.450 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0030215400000000001 2024-09-16T20:50:28.450 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.450 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.450 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:28.451 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601365+0000", 2024-09-16T20:50:28.451 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0215e-05 2024-09-16T20:50:28.451 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.451 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.451 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:28.451 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635506+0000", 2024-09-16T20:50:28.451 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0341408240000001 2024-09-16T20:50:28.451 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.451 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.451 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:28.451 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635655+0000", 2024-09-16T20:50:28.451 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000149412 2024-09-16T20:50:28.451 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.452 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.452 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:28.452 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635799+0000", 2024-09-16T20:50:28.452 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000143745 2024-09-16T20:50:28.452 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.452 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:28.452 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.452 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.452 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.452 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5763.0:4900 3.a 3:5384606c:::rbd_data.110e1b19a241.0000000000000018:head [read 8192~24576] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-16T20:50:28.452 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:50:28.452 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 42.796016416000001, 2024-09-16T20:50:28.452 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0379340800000001, 2024-09-16T20:50:28.452 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:28.453 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:28.453 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:28.453 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:28.453 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5763", 2024-09-16T20:50:28.453 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/623897567", 2024-09-16T20:50:28.453 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4900 2024-09-16T20:50:28.453 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.453 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:28.453 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.453 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:28.453 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:50:28.453 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.453 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.454 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.454 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:28.454 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:50:28.454 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.454 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.454 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.454 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:28.454 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:50:28.454 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4300000000000003e-07 2024-09-16T20:50:28.454 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.454 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.454 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:28.454 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598274+0000", 2024-09-16T20:50:28.454 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.976e-06 2024-09-16T20:50:28.455 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.455 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.455 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:28.455 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598282+0000", 2024-09-16T20:50:28.455 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0029999999999994e-06 2024-09-16T20:50:28.455 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.455 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.455 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:28.455 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598289+0000", 2024-09-16T20:50:28.455 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2389999999999999e-06 2024-09-16T20:50:28.455 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.455 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.455 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:28.455 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598430+0000", 2024-09-16T20:50:28.456 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014119600000000001 2024-09-16T20:50:28.456 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.456 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.456 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting_for_map not empty", 2024-09-16T20:50:28.456 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598435+0000", 2024-09-16T20:50:28.456 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3489999999999999e-06 2024-09-16T20:50:28.456 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.456 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.456 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:28.456 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601429+0000", 2024-09-16T20:50:28.456 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0029942720000000001 2024-09-16T20:50:28.456 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.456 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.457 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:28.457 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601438+0000", 2024-09-16T20:50:28.457 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4759999999999998e-06 2024-09-16T20:50:28.457 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.457 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.457 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:28.457 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635987+0000", 2024-09-16T20:50:28.457 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0345495010000001 2024-09-16T20:50:28.457 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.457 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.457 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:28.457 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.636056+0000", 2024-09-16T20:50:28.457 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8610999999999998e-05 2024-09-16T20:50:28.458 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.458 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.458 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:28.458 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.636205+0000", 2024-09-16T20:50:28.458 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014891399999999999 2024-09-16T20:50:28.458 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.458 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:28.458 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.459 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.459 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.459 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop(client.6231.0:2182 3.17 e136/81 3:edb5e5cd:::rbd_data.110e61479aee.0000000000000014:head v 136'791, mlcod=135'789)", 2024-09-16T20:50:28.459 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:22.735499+0000", 2024-09-16T20:50:28.459 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 5.6587884749999997, 2024-09-16T20:50:28.459 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12369608, 2024-09-16T20:50:28.459 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:28.459 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:28.459 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:28.459 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:28.459 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.459 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:28.459 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735499+0000", 2024-09-16T20:50:28.459 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.460 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.460 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.460 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:28.460 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735499+0000", 2024-09-16T20:50:28.460 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.460 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.460 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.460 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:28.460 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735499+0000", 2024-09-16T20:50:28.460 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8099999999999999e-07 2024-09-16T20:50:28.460 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.460 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.460 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:28.461 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735506+0000", 2024-09-16T20:50:28.461 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3579999999999997e-06 2024-09-16T20:50:28.461 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.461 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.461 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:28.461 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735517+0000", 2024-09-16T20:50:28.461 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0869999999999999e-05 2024-09-16T20:50:28.461 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.461 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.461 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:28.461 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735533+0000", 2024-09-16T20:50:28.461 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5858e-05 2024-09-16T20:50:28.461 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.461 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.462 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:28.462 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735584+0000", 2024-09-16T20:50:28.462 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0470999999999998e-05 2024-09-16T20:50:28.462 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.462 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.462 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:28.462 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735607+0000", 2024-09-16T20:50:28.462 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2875999999999999e-05 2024-09-16T20:50:28.462 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.462 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.462 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:28.462 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.859144+0000", 2024-09-16T20:50:28.462 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.123537701 2024-09-16T20:50:28.463 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.463 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.463 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:28.463 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.859195+0000", 2024-09-16T20:50:28.463 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0664999999999997e-05 2024-09-16T20:50:28.463 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.463 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:28.463 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.463 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.463 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.463 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5204 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 2523136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:28.463 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.883660+0000", 2024-09-16T20:50:28.463 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.5106269450000003, 2024-09-16T20:50:28.463 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052517638999999998, 2024-09-16T20:50:28.464 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:28.464 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:28.464 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:28.464 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:28.464 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:50:28.464 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:28.464 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5204 2024-09-16T20:50:28.464 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.464 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:28.464 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.464 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:28.464 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883660+0000", 2024-09-16T20:50:28.464 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.465 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.465 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.465 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:28.465 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883660+0000", 2024-09-16T20:50:28.465 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.465 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.465 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.465 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:28.465 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883661+0000", 2024-09-16T20:50:28.465 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9999999999999998e-07 2024-09-16T20:50:28.465 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.465 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.465 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:28.465 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883663+0000", 2024-09-16T20:50:28.466 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.4480000000000001e-06 2024-09-16T20:50:28.466 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.466 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.466 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:28.466 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883669+0000", 2024-09-16T20:50:28.466 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.071e-06 2024-09-16T20:50:28.466 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.466 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.466 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:28.466 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883675+0000", 2024-09-16T20:50:28.466 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.366e-06 2024-09-16T20:50:28.466 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.466 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.467 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:28.467 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.935970+0000", 2024-09-16T20:50:28.467 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052295346999999999 2024-09-16T20:50:28.467 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.467 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.467 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:28.467 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936064+0000", 2024-09-16T20:50:28.467 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4438000000000006e-05 2024-09-16T20:50:28.467 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.467 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.467 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:28.467 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936178+0000", 2024-09-16T20:50:28.467 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000113569 2024-09-16T20:50:28.467 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.468 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:28.468 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.468 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.468 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.468 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5205 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 2543616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:28.468 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.883745+0000", 2024-09-16T20:50:28.468 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.5105421870000004, 2024-09-16T20:50:28.468 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052692112999999999, 2024-09-16T20:50:28.468 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:28.468 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:28.468 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:28.468 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:28.468 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:50:28.468 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:28.469 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5205 2024-09-16T20:50:28.469 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.469 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:28.469 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.469 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:28.469 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883745+0000", 2024-09-16T20:50:28.469 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.469 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.469 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.469 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:28.469 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883745+0000", 2024-09-16T20:50:28.469 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.469 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.470 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.470 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:28.470 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883746+0000", 2024-09-16T20:50:28.470 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2699999999999999e-07 2024-09-16T20:50:28.470 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.470 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.470 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:28.470 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883750+0000", 2024-09-16T20:50:28.470 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7050000000000001e-06 2024-09-16T20:50:28.470 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.470 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.470 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:28.470 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883769+0000", 2024-09-16T20:50:28.471 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9165e-05 2024-09-16T20:50:28.471 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.471 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.471 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:28.471 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883778+0000", 2024-09-16T20:50:28.471 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.6690000000000001e-06 2024-09-16T20:50:28.471 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.471 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.471 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:28.471 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936228+0000", 2024-09-16T20:50:28.471 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052449587999999998 2024-09-16T20:50:28.471 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.471 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.472 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:28.472 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936333+0000", 2024-09-16T20:50:28.472 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010511800000000001 2024-09-16T20:50:28.472 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.472 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.472 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:28.472 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936437+0000", 2024-09-16T20:50:28.472 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000104141 2024-09-16T20:50:28.472 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.472 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:28.472 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.472 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.472 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.472 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5206 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 1167360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:28.473 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.883802+0000", 2024-09-16T20:50:28.473 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.5104856929999997, 2024-09-16T20:50:28.473 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052800780999999998, 2024-09-16T20:50:28.473 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:28.473 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:28.473 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:28.473 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:28.473 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:50:28.473 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:28.473 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5206 2024-09-16T20:50:28.473 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.473 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:28.473 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.474 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:28.474 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883802+0000", 2024-09-16T20:50:28.474 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.474 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.474 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.474 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:28.474 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883802+0000", 2024-09-16T20:50:28.474 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.474 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.474 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.474 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:28.474 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883933+0000", 2024-09-16T20:50:28.474 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013182999999999999 2024-09-16T20:50:28.475 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.475 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.475 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:28.475 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883937+0000", 2024-09-16T20:50:28.475 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.771e-06 2024-09-16T20:50:28.475 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.475 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.475 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:28.475 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883953+0000", 2024-09-16T20:50:28.475 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5409000000000001e-05 2024-09-16T20:50:28.475 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.475 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.475 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:28.475 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883959+0000", 2024-09-16T20:50:28.476 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0700000000000003e-06 2024-09-16T20:50:28.476 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.476 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.476 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:28.476 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936476+0000", 2024-09-16T20:50:28.476 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052517434000000002 2024-09-16T20:50:28.476 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.476 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.476 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:28.476 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936529+0000", 2024-09-16T20:50:28.476 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2969000000000001e-05 2024-09-16T20:50:28.476 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.476 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.477 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:28.477 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936602+0000", 2024-09-16T20:50:28.477 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3298000000000002e-05 2024-09-16T20:50:28.477 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.477 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:28.477 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.477 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.477 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.477 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5207 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 1904640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:28.477 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.883980+0000", 2024-09-16T20:50:28.477 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.5103074840000001, 2024-09-16T20:50:28.477 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052759922000000001, 2024-09-16T20:50:28.477 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:28.477 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:28.478 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:28.478 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:28.478 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:50:28.478 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:28.478 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5207 2024-09-16T20:50:28.478 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.478 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:28.478 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.478 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:28.478 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883980+0000", 2024-09-16T20:50:28.478 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.478 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.478 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.479 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:28.479 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883980+0000", 2024-09-16T20:50:28.479 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.479 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.479 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.479 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:28.479 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884126+0000", 2024-09-16T20:50:28.479 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014577399999999999 2024-09-16T20:50:28.479 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.479 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.479 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:28.479 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884130+0000", 2024-09-16T20:50:28.479 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9729999999999999e-06 2024-09-16T20:50:28.479 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.480 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.480 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:28.480 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884139+0000", 2024-09-16T20:50:28.480 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8660000000000001e-06 2024-09-16T20:50:28.480 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.480 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.480 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:28.480 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884144+0000", 2024-09-16T20:50:28.480 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3669999999999999e-06 2024-09-16T20:50:28.480 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.480 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.480 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:28.480 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936635+0000", 2024-09-16T20:50:28.481 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052491705 2024-09-16T20:50:28.481 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.481 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.481 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:28.481 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936679+0000", 2024-09-16T20:50:28.481 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3105999999999997e-05 2024-09-16T20:50:28.481 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.481 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.481 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:28.481 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936740+0000", 2024-09-16T20:50:28.481 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1130999999999995e-05 2024-09-16T20:50:28.481 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.481 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:28.481 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.482 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.482 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.482 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5208 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 131072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:28.482 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.884165+0000", 2024-09-16T20:50:28.482 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.5101221880000004, 2024-09-16T20:50:28.482 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052746603000000003, 2024-09-16T20:50:28.482 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:28.482 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:28.482 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:28.482 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:28.482 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:50:28.482 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:28.482 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5208 2024-09-16T20:50:28.483 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.483 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:28.483 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.483 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:28.483 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884165+0000", 2024-09-16T20:50:28.483 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.483 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.483 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.483 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:28.483 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884165+0000", 2024-09-16T20:50:28.483 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.483 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.483 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.483 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:28.484 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884463+0000", 2024-09-16T20:50:28.484 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00029801699999999998 2024-09-16T20:50:28.484 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.484 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.484 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:28.484 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884467+0000", 2024-09-16T20:50:28.484 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8210000000000003e-06 2024-09-16T20:50:28.484 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.484 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.484 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:28.484 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884477+0000", 2024-09-16T20:50:28.484 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.7030000000000001e-06 2024-09-16T20:50:28.484 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.484 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.485 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:28.485 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884485+0000", 2024-09-16T20:50:28.485 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9710000000000004e-06 2024-09-16T20:50:28.485 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.485 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.485 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:28.485 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936785+0000", 2024-09-16T20:50:28.485 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052300665000000003 2024-09-16T20:50:28.485 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.485 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.485 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:28.485 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936851+0000", 2024-09-16T20:50:28.485 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6159000000000001e-05 2024-09-16T20:50:28.486 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.486 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.486 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:28.486 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936912+0000", 2024-09-16T20:50:28.486 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0266999999999997e-05 2024-09-16T20:50:28.486 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.486 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:28.486 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.486 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.486 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.486 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5209 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 1925120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:28.486 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.884504+0000", 2024-09-16T20:50:28.486 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.5097833029999999, 2024-09-16T20:50:28.487 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052575445999999998, 2024-09-16T20:50:28.487 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:28.487 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:28.487 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:28.487 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:28.487 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:50:28.487 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:28.487 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5209 2024-09-16T20:50:28.487 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.487 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:28.487 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.487 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:28.487 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884504+0000", 2024-09-16T20:50:28.487 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.488 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.488 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.488 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:28.488 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884504+0000", 2024-09-16T20:50:28.488 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.488 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.488 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.488 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:28.488 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884741+0000", 2024-09-16T20:50:28.488 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00023679899999999999 2024-09-16T20:50:28.488 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.488 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.488 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:28.489 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884747+0000", 2024-09-16T20:50:28.489 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0009999999999998e-06 2024-09-16T20:50:28.489 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.489 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.489 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:28.489 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884764+0000", 2024-09-16T20:50:28.489 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7371999999999999e-05 2024-09-16T20:50:28.489 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.489 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.489 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:28.489 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884772+0000", 2024-09-16T20:50:28.489 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1349999999999992e-06 2024-09-16T20:50:28.489 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.489 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.490 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:28.490 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936956+0000", 2024-09-16T20:50:28.490 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052183754999999998 2024-09-16T20:50:28.490 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.490 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.490 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:28.490 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937003+0000", 2024-09-16T20:50:28.490 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6925000000000002e-05 2024-09-16T20:50:28.490 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.490 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.490 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:28.490 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937079+0000", 2024-09-16T20:50:28.490 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6458999999999994e-05 2024-09-16T20:50:28.491 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.491 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:28.491 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.491 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.491 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.491 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5210 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 151552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:28.491 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.884799+0000", 2024-09-16T20:50:28.491 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.5094880340000003, 2024-09-16T20:50:28.491 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052462483999999997, 2024-09-16T20:50:28.491 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:28.491 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:28.491 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:28.491 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:28.491 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:50:28.492 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:28.492 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5210 2024-09-16T20:50:28.492 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.492 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:28.492 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.492 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:28.492 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884799+0000", 2024-09-16T20:50:28.492 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.492 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.492 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.492 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:28.492 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884799+0000", 2024-09-16T20:50:28.492 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.493 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.493 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.493 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:28.493 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884990+0000", 2024-09-16T20:50:28.493 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019051800000000001 2024-09-16T20:50:28.493 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.493 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.493 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:28.493 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884996+0000", 2024-09-16T20:50:28.493 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0689999999999998e-06 2024-09-16T20:50:28.493 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.493 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.493 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:28.494 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.885008+0000", 2024-09-16T20:50:28.494 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2153999999999999e-05 2024-09-16T20:50:28.494 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.494 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.494 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:28.494 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.885016+0000", 2024-09-16T20:50:28.494 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2770000000000005e-06 2024-09-16T20:50:28.494 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.494 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.494 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:28.494 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937128+0000", 2024-09-16T20:50:28.494 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052111377 2024-09-16T20:50:28.494 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.494 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.495 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:28.495 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937188+0000", 2024-09-16T20:50:28.495 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9933999999999999e-05 2024-09-16T20:50:28.495 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.495 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.495 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:28.495 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937262+0000", 2024-09-16T20:50:28.495 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4154999999999997e-05 2024-09-16T20:50:28.495 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.495 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:28.495 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.495 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.495 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.495 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5211 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 1945600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:28.496 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.885030+0000", 2024-09-16T20:50:28.496 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.5092569490000001, 2024-09-16T20:50:28.496 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052369470000000001, 2024-09-16T20:50:28.496 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:28.496 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:28.496 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:28.496 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:28.496 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:50:28.496 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:28.496 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5211 2024-09-16T20:50:28.496 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.496 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:28.496 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.497 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:28.497 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.885030+0000", 2024-09-16T20:50:28.497 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.497 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.497 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.497 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:28.497 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.885030+0000", 2024-09-16T20:50:28.497 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.497 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.497 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.497 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:28.497 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.885219+0000", 2024-09-16T20:50:28.497 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000188278 2024-09-16T20:50:28.497 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.498 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.498 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:28.498 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.885223+0000", 2024-09-16T20:50:28.498 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7849999999999999e-06 2024-09-16T20:50:28.498 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.498 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.498 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:28.498 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.885233+0000", 2024-09-16T20:50:28.498 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.5060000000000001e-06 2024-09-16T20:50:28.498 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.498 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.498 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:28.498 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.885240+0000", 2024-09-16T20:50:28.498 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9630000000000001e-06 2024-09-16T20:50:28.499 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.499 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.499 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:28.499 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937304+0000", 2024-09-16T20:50:28.499 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052064619999999999 2024-09-16T20:50:28.499 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.499 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.499 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:28.499 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937347+0000", 2024-09-16T20:50:28.499 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2407000000000002e-05 2024-09-16T20:50:28.499 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.499 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.499 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:28.500 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937400+0000", 2024-09-16T20:50:28.500 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2911000000000002e-05 2024-09-16T20:50:28.500 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.500 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:28.500 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.500 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.500 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:28.500 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:28.501 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:50:28.502 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:28.501+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:28.602 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:28.602+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:28.609 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:28.609 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:50:28.609 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.610 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6061.0:5425 3.13 3.12c2aa13 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-16T20:50:28.610 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:27.767049+0000", 2024-09-16T20:50:28.610 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.84233714599999998, 2024-09-16T20:50:28.610 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.84235786899999998, 2024-09-16T20:50:28.610 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:28.610 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:28.610 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "waiting for peered", 2024-09-16T20:50:28.610 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:28.610 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6061", 2024-09-16T20:50:28.611 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3930128440", 2024-09-16T20:50:28.611 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5425 2024-09-16T20:50:28.611 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.611 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:28.611 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.611 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:28.611 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767049+0000", 2024-09-16T20:50:28.611 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.611 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.611 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.611 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:28.611 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767049+0000", 2024-09-16T20:50:28.611 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.612 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.612 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.612 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:28.612 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767050+0000", 2024-09-16T20:50:28.612 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6499999999999999e-07 2024-09-16T20:50:28.612 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.612 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.612 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:28.612 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767052+0000", 2024-09-16T20:50:28.612 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7350000000000001e-06 2024-09-16T20:50:28.612 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.612 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.613 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:28.613 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767060+0000", 2024-09-16T20:50:28.613 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5109999999999997e-06 2024-09-16T20:50:28.613 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.613 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.613 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:28.613 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767066+0000", 2024-09-16T20:50:28.613 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8139999999999997e-06 2024-09-16T20:50:28.613 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.613 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.613 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:28.613 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767128+0000", 2024-09-16T20:50:28.613 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2174999999999993e-05 2024-09-16T20:50:28.614 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.614 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.614 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:28.614 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767133+0000", 2024-09-16T20:50:28.614 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2140000000000004e-06 2024-09-16T20:50:28.614 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.614 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:28.614 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.614 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.614 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.614 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6061.0:5401 3.13 3.12c2aa13 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-16T20:50:28.614 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:27.761536+0000", 2024-09-16T20:50:28.614 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.84784980499999996, 2024-09-16T20:50:28.615 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.84790850100000004, 2024-09-16T20:50:28.615 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:28.615 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:28.615 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "waiting for peered", 2024-09-16T20:50:28.615 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:28.615 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6061", 2024-09-16T20:50:28.615 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3930128440", 2024-09-16T20:50:28.615 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5401 2024-09-16T20:50:28.615 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.615 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:28.615 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.615 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:28.615 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761536+0000", 2024-09-16T20:50:28.615 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.615 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.616 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.616 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:28.616 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761536+0000", 2024-09-16T20:50:28.616 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.616 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.616 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.616 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:28.616 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761537+0000", 2024-09-16T20:50:28.616 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2900000000000003e-07 2024-09-16T20:50:28.616 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.616 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.616 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:28.616 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761540+0000", 2024-09-16T20:50:28.617 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2320000000000001e-06 2024-09-16T20:50:28.617 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.617 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.617 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:28.617 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761548+0000", 2024-09-16T20:50:28.617 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1289999999999996e-06 2024-09-16T20:50:28.617 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.617 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.617 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:28.617 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761554+0000", 2024-09-16T20:50:28.617 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0820000000000004e-06 2024-09-16T20:50:28.618 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.618 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.618 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:28.618 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761621+0000", 2024-09-16T20:50:28.618 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6266000000000006e-05 2024-09-16T20:50:28.618 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.618 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.618 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:28.618 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761626+0000", 2024-09-16T20:50:28.618 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.614e-06 2024-09-16T20:50:28.618 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.619 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:28.619 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.619 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.619 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-16T20:50:28.619 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 2 2024-09-16T20:50:28.619 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:28.620 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:50:28.704 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:28.703+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:28.727 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:28.727 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:50:28.727 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:50:28.727 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:50:28.727 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:28.735 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:50:28.759 INFO:teuthology.orchestra.run.smithi029.stderr:nodeep-scrub is unset 2024-09-16T20:50:28.804 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:28.804+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:28.862 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:28.862 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:50:28.862 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:50:28.863 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:50:28.863 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.863 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:50:28.863 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:50:28.863 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 155.60657035899999, 2024-09-16T20:50:28.863 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67502346999999996, 2024-09-16T20:50:28.863 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:28.863 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:28.863 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:28.863 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:28.863 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:50:28.863 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:50:28.864 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 1 2024-09-16T20:50:28.864 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.864 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:28.864 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.864 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:28.864 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:50:28.864 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.864 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.864 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.864 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:28.864 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:50:28.864 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.864 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.865 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.865 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:28.865 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254128+0000", 2024-09-16T20:50:28.865 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.463e-06 2024-09-16T20:50:28.865 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.865 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.865 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:28.865 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254135+0000", 2024-09-16T20:50:28.865 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1579999999999999e-06 2024-09-16T20:50:28.865 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.865 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.865 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:28.865 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254146+0000", 2024-09-16T20:50:28.866 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1341e-05 2024-09-16T20:50:28.866 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.866 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.866 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:28.866 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254155+0000", 2024-09-16T20:50:28.866 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.1549999999999996e-06 2024-09-16T20:50:28.866 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.866 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.866 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:28.866 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254221+0000", 2024-09-16T20:50:28.866 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5470999999999997e-05 2024-09-16T20:50:28.866 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.866 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.867 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:28.867 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254231+0000", 2024-09-16T20:50:28.867 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0592e-05 2024-09-16T20:50:28.867 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.867 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.867 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:28.867 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926711+0000", 2024-09-16T20:50:28.867 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67247946599999997 2024-09-16T20:50:28.867 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.867 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.867 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:28.867 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926910+0000", 2024-09-16T20:50:28.867 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019934 2024-09-16T20:50:28.867 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.868 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.868 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:50:28.868 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.927159+0000", 2024-09-16T20:50:28.868 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00024876599999999998 2024-09-16T20:50:28.868 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.868 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.868 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:28.868 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.928797+0000", 2024-09-16T20:50:28.868 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001638308 2024-09-16T20:50:28.868 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.868 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.868 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:28.869 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929026+0000", 2024-09-16T20:50:28.869 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00022876099999999999 2024-09-16T20:50:28.869 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.869 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.869 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:28.869 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929102+0000", 2024-09-16T20:50:28.869 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6184000000000001e-05 2024-09-16T20:50:28.869 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.869 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.869 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:28.869 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929150+0000", 2024-09-16T20:50:28.869 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7465e-05 2024-09-16T20:50:28.869 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.870 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:28.870 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.870 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.870 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.870 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3574 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:28.870 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:50:28.870 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 99.379406826999997, 2024-09-16T20:50:28.870 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053579441999999998, 2024-09-16T20:50:28.870 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:28.870 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:28.870 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:28.870 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:28.870 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:28.871 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:28.871 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3574 2024-09-16T20:50:28.871 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.871 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:28.871 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.871 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:28.871 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:50:28.871 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.871 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.871 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.871 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:28.871 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:50:28.872 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.872 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.872 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.872 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:28.872 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:50:28.872 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2e-07 2024-09-16T20:50:28.872 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.872 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.872 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:28.872 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481292+0000", 2024-09-16T20:50:28.872 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2630000000000002e-06 2024-09-16T20:50:28.872 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.872 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.873 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:28.873 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481297+0000", 2024-09-16T20:50:28.873 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.296e-06 2024-09-16T20:50:28.873 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.873 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.873 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:28.873 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481300+0000", 2024-09-16T20:50:28.873 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7019999999999999e-06 2024-09-16T20:50:28.873 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.873 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.873 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:28.873 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525533+0000", 2024-09-16T20:50:28.873 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.044232318 2024-09-16T20:50:28.873 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.874 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.874 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:28.874 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525619+0000", 2024-09-16T20:50:28.874 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5892999999999999e-05 2024-09-16T20:50:28.874 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.874 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.874 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:28.874 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525987+0000", 2024-09-16T20:50:28.874 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00036836099999999999 2024-09-16T20:50:28.874 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.874 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.874 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:28.875 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531779+0000", 2024-09-16T20:50:28.875 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0057918240000000001 2024-09-16T20:50:28.875 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.875 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.875 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:28.875 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534747+0000", 2024-09-16T20:50:28.875 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0029681600000000001 2024-09-16T20:50:28.875 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.875 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.875 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:28.875 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534832+0000", 2024-09-16T20:50:28.875 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5180999999999996e-05 2024-09-16T20:50:28.876 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.876 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.876 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:28.876 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534869+0000", 2024-09-16T20:50:28.876 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7024000000000003e-05 2024-09-16T20:50:28.876 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.876 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:28.876 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.876 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.876 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.876 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3575 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:28.877 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:50:28.877 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 99.379392086999999, 2024-09-16T20:50:28.877 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053801463000000001, 2024-09-16T20:50:28.877 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:28.877 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:28.877 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:28.877 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:28.877 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:28.877 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:28.877 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3575 2024-09-16T20:50:28.877 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.878 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:28.878 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.878 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:28.878 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:50:28.878 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.878 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.878 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.878 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:28.878 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:50:28.878 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.878 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.878 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.878 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:28.879 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481305+0000", 2024-09-16T20:50:28.879 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5400000000000002e-07 2024-09-16T20:50:28.879 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.879 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.879 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:28.879 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481307+0000", 2024-09-16T20:50:28.879 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7800000000000001e-06 2024-09-16T20:50:28.879 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.879 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.879 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:28.879 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481312+0000", 2024-09-16T20:50:28.879 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6349999999999997e-06 2024-09-16T20:50:28.880 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.880 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.880 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:28.880 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481315+0000", 2024-09-16T20:50:28.880 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7829999999999999e-06 2024-09-16T20:50:28.880 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.880 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.880 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:28.880 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.526876+0000", 2024-09-16T20:50:28.880 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.045561070000000002 2024-09-16T20:50:28.880 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.880 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.881 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:28.881 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.526954+0000", 2024-09-16T20:50:28.881 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8108000000000005e-05 2024-09-16T20:50:28.881 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.881 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.881 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:28.881 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527282+0000", 2024-09-16T20:50:28.881 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000327483 2024-09-16T20:50:28.881 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.881 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.881 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:28.882 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531784+0000", 2024-09-16T20:50:28.882 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0045020169999999997 2024-09-16T20:50:28.882 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.882 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.882 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:28.882 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534921+0000", 2024-09-16T20:50:28.882 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.003137791 2024-09-16T20:50:28.882 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.882 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.882 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:28.882 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534976+0000", 2024-09-16T20:50:28.882 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4953000000000001e-05 2024-09-16T20:50:28.882 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.882 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.883 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:28.883 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535106+0000", 2024-09-16T20:50:28.883 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000129589 2024-09-16T20:50:28.883 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.883 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:28.883 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.883 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.883 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.883 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3576 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:28.883 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:50:28.883 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 99.379378174999999, 2024-09-16T20:50:28.883 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053955436000000002, 2024-09-16T20:50:28.883 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:28.883 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:28.884 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:28.884 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:28.884 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:28.884 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:28.884 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3576 2024-09-16T20:50:28.884 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.884 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:28.884 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.884 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:28.884 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:50:28.884 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.884 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.884 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.884 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:28.885 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:50:28.885 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.885 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.885 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.885 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:28.885 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481319+0000", 2024-09-16T20:50:28.885 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.53e-07 2024-09-16T20:50:28.885 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.885 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.885 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:28.885 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481320+0000", 2024-09-16T20:50:28.885 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5230000000000001e-06 2024-09-16T20:50:28.885 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.885 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.886 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:28.886 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481330+0000", 2024-09-16T20:50:28.886 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0190000000000001e-05 2024-09-16T20:50:28.886 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.886 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.886 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:28.886 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481333+0000", 2024-09-16T20:50:28.886 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.151e-06 2024-09-16T20:50:28.886 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.886 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.886 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:28.886 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527689+0000", 2024-09-16T20:50:28.886 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046355192000000003 2024-09-16T20:50:28.886 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.886 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.887 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:28.887 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527764+0000", 2024-09-16T20:50:28.887 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5776000000000001e-05 2024-09-16T20:50:28.887 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.887 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.887 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:28.887 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528141+0000", 2024-09-16T20:50:28.887 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00037647599999999999 2024-09-16T20:50:28.887 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.887 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.887 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:28.887 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531788+0000", 2024-09-16T20:50:28.887 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0036470679999999998 2024-09-16T20:50:28.887 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.887 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.888 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:28.888 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535174+0000", 2024-09-16T20:50:28.888 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0033862670000000001 2024-09-16T20:50:28.888 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.888 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.888 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:28.888 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535236+0000", 2024-09-16T20:50:28.888 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1797999999999993e-05 2024-09-16T20:50:28.888 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.888 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.888 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:28.888 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535274+0000", 2024-09-16T20:50:28.888 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7741999999999999e-05 2024-09-16T20:50:28.889 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.889 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:28.889 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.889 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.889 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.889 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3577 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:28.889 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:50:28.889 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 99.379356575000003, 2024-09-16T20:50:28.889 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054073807000000002, 2024-09-16T20:50:28.889 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:28.889 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:28.889 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:28.889 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:28.890 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:28.890 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:28.890 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3577 2024-09-16T20:50:28.890 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.890 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:28.890 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.890 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:28.890 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:50:28.890 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.890 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.890 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.890 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:28.890 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:50:28.891 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.891 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.891 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.891 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:28.891 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:50:28.891 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.96e-07 2024-09-16T20:50:28.891 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.891 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.891 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:28.891 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481342+0000", 2024-09-16T20:50:28.891 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.714e-06 2024-09-16T20:50:28.891 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.891 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.891 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:28.891 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481349+0000", 2024-09-16T20:50:28.892 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4930000000000001e-06 2024-09-16T20:50:28.892 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.892 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.892 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:28.892 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481353+0000", 2024-09-16T20:50:28.892 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4879999999999999e-06 2024-09-16T20:50:28.892 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.892 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.892 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:28.892 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528619+0000", 2024-09-16T20:50:28.892 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047266351999999998 2024-09-16T20:50:28.892 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.892 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.892 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:28.893 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528706+0000", 2024-09-16T20:50:28.893 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6796e-05 2024-09-16T20:50:28.893 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.893 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.893 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:28.893 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529148+0000", 2024-09-16T20:50:28.893 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00044182999999999999 2024-09-16T20:50:28.893 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.893 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.893 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:28.893 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531792+0000", 2024-09-16T20:50:28.893 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0026443119999999998 2024-09-16T20:50:28.893 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.894 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.894 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:28.894 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535319+0000", 2024-09-16T20:50:28.894 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0035266439999999998 2024-09-16T20:50:28.894 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.894 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.894 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:28.894 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535372+0000", 2024-09-16T20:50:28.894 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3207000000000001e-05 2024-09-16T20:50:28.894 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.894 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.894 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:28.894 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535414+0000", 2024-09-16T20:50:28.895 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1675000000000003e-05 2024-09-16T20:50:28.895 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.895 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:28.895 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.895 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.895 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.895 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3578 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:28.895 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:50:28.895 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 99.379335405999996, 2024-09-16T20:50:28.895 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054189586999999997, 2024-09-16T20:50:28.895 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:28.895 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:28.896 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:28.896 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:28.896 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:28.896 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:28.896 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3578 2024-09-16T20:50:28.896 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.896 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:28.896 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.896 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:28.896 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:50:28.896 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.896 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.896 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.897 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:28.897 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:50:28.897 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.897 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.897 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.897 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:28.897 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:50:28.897 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0800000000000001e-07 2024-09-16T20:50:28.897 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.897 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.897 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:28.897 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481363+0000", 2024-09-16T20:50:28.898 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0339999999999999e-06 2024-09-16T20:50:28.898 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.898 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.898 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:28.898 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481371+0000", 2024-09-16T20:50:28.898 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4440000000000002e-06 2024-09-16T20:50:28.898 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.898 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.898 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:28.898 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481374+0000", 2024-09-16T20:50:28.898 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2049999999999998e-06 2024-09-16T20:50:28.898 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.899 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.899 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:28.899 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529555+0000", 2024-09-16T20:50:28.899 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048181202999999999 2024-09-16T20:50:28.899 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.899 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.899 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:28.899 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529616+0000", 2024-09-16T20:50:28.899 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0912000000000002e-05 2024-09-16T20:50:28.899 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.899 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.899 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:28.899 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529939+0000", 2024-09-16T20:50:28.900 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00032269800000000001 2024-09-16T20:50:28.900 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.900 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.900 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:28.900 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532602+0000", 2024-09-16T20:50:28.900 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0026634940000000002 2024-09-16T20:50:28.900 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.900 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.900 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:28.900 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535465+0000", 2024-09-16T20:50:28.900 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.002862149 2024-09-16T20:50:28.900 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.900 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.900 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:28.901 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535516+0000", 2024-09-16T20:50:28.901 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1696000000000001e-05 2024-09-16T20:50:28.901 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.901 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.901 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:28.901 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535551+0000", 2024-09-16T20:50:28.901 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4443999999999998e-05 2024-09-16T20:50:28.901 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.901 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:28.901 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.901 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.901 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.901 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3579 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:28.902 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:50:28.902 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 99.379316966999994, 2024-09-16T20:50:28.902 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054295151, 2024-09-16T20:50:28.902 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:28.902 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:28.902 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:28.902 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:28.902 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:28.902 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:28.902 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3579 2024-09-16T20:50:28.902 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.902 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:28.902 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.902 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:28.903 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:50:28.903 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.903 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.903 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.903 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:28.903 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:50:28.903 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.903 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.903 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.903 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:28.903 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:50:28.903 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3300000000000001e-07 2024-09-16T20:50:28.903 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.903 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.904 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:28.904 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481381+0000", 2024-09-16T20:50:28.904 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.649e-06 2024-09-16T20:50:28.904 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.904 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.904 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:28.904 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481387+0000", 2024-09-16T20:50:28.904 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4519999999999996e-06 2024-09-16T20:50:28.904 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.904 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.904 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:28.904 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481390+0000", 2024-09-16T20:50:28.904 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0170000000000001e-06 2024-09-16T20:50:28.905 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.905 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.905 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:28.905 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530756+0000", 2024-09-16T20:50:28.905 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049365882999999999 2024-09-16T20:50:28.905 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.905 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.905 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:28.905 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530845+0000", 2024-09-16T20:50:28.905 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9197000000000007e-05 2024-09-16T20:50:28.905 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.905 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.905 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:28.906 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531191+0000", 2024-09-16T20:50:28.906 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00034562500000000001 2024-09-16T20:50:28.906 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.906 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.906 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:28.906 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534078+0000", 2024-09-16T20:50:28.906 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0028878179999999999 2024-09-16T20:50:28.906 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.906 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.906 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:28.906 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535589+0000", 2024-09-16T20:50:28.906 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00151085 2024-09-16T20:50:28.907 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.907 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.907 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:28.907 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535639+0000", 2024-09-16T20:50:28.907 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9697000000000002e-05 2024-09-16T20:50:28.907 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.907 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.907 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:28.907 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535675+0000", 2024-09-16T20:50:28.907 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5729999999999998e-05 2024-09-16T20:50:28.907 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.907 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:28.907 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.907 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.908 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.908 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3580 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:28.908 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:50:28.908 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 99.379303335000003, 2024-09-16T20:50:28.908 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054397361999999998, 2024-09-16T20:50:28.908 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:28.908 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:28.908 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:28.908 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:28.908 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:28.908 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:28.908 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3580 2024-09-16T20:50:28.908 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.909 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:28.909 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.909 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:28.909 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:50:28.909 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.909 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.909 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.909 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:28.909 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:50:28.909 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.909 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.909 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.909 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:28.910 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:50:28.910 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2600000000000001e-07 2024-09-16T20:50:28.910 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.910 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.910 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:28.910 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481395+0000", 2024-09-16T20:50:28.910 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.544e-06 2024-09-16T20:50:28.910 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.910 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.910 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:28.910 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481404+0000", 2024-09-16T20:50:28.910 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8850000000000005e-06 2024-09-16T20:50:28.910 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.911 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.911 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:28.911 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481407+0000", 2024-09-16T20:50:28.911 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1540000000000002e-06 2024-09-16T20:50:28.911 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.911 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.911 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:28.911 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531823+0000", 2024-09-16T20:50:28.911 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050415637999999999 2024-09-16T20:50:28.911 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.911 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.911 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:28.911 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531895+0000", 2024-09-16T20:50:28.912 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2179000000000001e-05 2024-09-16T20:50:28.912 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.912 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.912 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:28.912 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532236+0000", 2024-09-16T20:50:28.912 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00034073799999999999 2024-09-16T20:50:28.912 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.912 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.912 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:28.912 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534083+0000", 2024-09-16T20:50:28.912 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001847298 2024-09-16T20:50:28.912 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.912 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.913 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:28.913 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535714+0000", 2024-09-16T20:50:28.913 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0016316810000000001 2024-09-16T20:50:28.913 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.913 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.913 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:28.913 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535759+0000", 2024-09-16T20:50:28.913 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4478999999999999e-05 2024-09-16T20:50:28.913 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.913 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.914 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:28.914 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535791+0000", 2024-09-16T20:50:28.914 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.154e-05 2024-09-16T20:50:28.914 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.914 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:28.914 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.914 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.914 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.914 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3581 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:28.914 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:50:28.914 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 99.379281337999998, 2024-09-16T20:50:28.914 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.057094403000000002, 2024-09-16T20:50:28.914 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:28.915 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:28.915 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:28.915 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:28.915 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:28.915 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:28.915 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3581 2024-09-16T20:50:28.915 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.915 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:28.915 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.915 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:28.915 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:50:28.915 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.915 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.915 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.916 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:28.916 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:50:28.916 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.916 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.916 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.916 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:28.916 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:50:28.916 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8799999999999998e-07 2024-09-16T20:50:28.916 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.916 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.916 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:28.916 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481418+0000", 2024-09-16T20:50:28.916 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1859999999999999e-06 2024-09-16T20:50:28.917 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.917 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.917 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:28.917 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481423+0000", 2024-09-16T20:50:28.917 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8279999999999999e-06 2024-09-16T20:50:28.917 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.917 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.917 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:28.917 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481426+0000", 2024-09-16T20:50:28.917 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0110000000000001e-06 2024-09-16T20:50:28.917 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.917 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.917 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:28.918 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532631+0000", 2024-09-16T20:50:28.918 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051204514999999999 2024-09-16T20:50:28.918 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.918 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.918 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:28.918 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.533381+0000", 2024-09-16T20:50:28.918 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00075029000000000003 2024-09-16T20:50:28.918 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.918 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.918 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:28.918 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.533739+0000", 2024-09-16T20:50:28.918 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00035773100000000002 2024-09-16T20:50:28.918 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.919 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.919 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:28.919 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535123+0000", 2024-09-16T20:50:28.919 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0013836160000000001 2024-09-16T20:50:28.919 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.919 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.919 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:28.919 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538413+0000", 2024-09-16T20:50:28.919 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0032902510000000001 2024-09-16T20:50:28.919 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.919 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.919 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:28.919 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538470+0000", 2024-09-16T20:50:28.919 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6697999999999998e-05 2024-09-16T20:50:28.920 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.920 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.920 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:28.920 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538510+0000", 2024-09-16T20:50:28.920 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9988999999999999e-05 2024-09-16T20:50:28.920 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.920 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:28.920 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.920 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.920 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.920 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:12732 3.d 3:b752444a:::rbd_header.110e747f0779:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-16T20:50:28.920 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:50:28.921 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 84.092208760000005, 2024-09-16T20:50:28.921 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.065302467000000003, 2024-09-16T20:50:28.921 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:28.921 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:28.921 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:28.921 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:28.921 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:50:28.921 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:50:28.921 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 12732 2024-09-16T20:50:28.921 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.921 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:28.921 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.921 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:28.922 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:50:28.922 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.922 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.922 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.922 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:28.922 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:50:28.922 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.922 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.922 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.922 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:28.922 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768490+0000", 2024-09-16T20:50:28.922 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2859999999999998e-06 2024-09-16T20:50:28.923 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.923 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.923 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:28.923 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768501+0000", 2024-09-16T20:50:28.923 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1226999999999999e-05 2024-09-16T20:50:28.923 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.923 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.923 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:28.923 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768517+0000", 2024-09-16T20:50:28.923 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.592e-05 2024-09-16T20:50:28.923 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.923 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.923 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:28.924 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768528+0000", 2024-09-16T20:50:28.924 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0689999999999999e-05 2024-09-16T20:50:28.924 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.924 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.924 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:28.924 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768584+0000", 2024-09-16T20:50:28.924 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5889e-05 2024-09-16T20:50:28.924 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.924 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.924 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:28.924 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768720+0000", 2024-09-16T20:50:28.924 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000136718 2024-09-16T20:50:28.924 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.925 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.925 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:28.925 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.769066+0000", 2024-09-16T20:50:28.925 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000346013 2024-09-16T20:50:28.925 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.925 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.925 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:28.925 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833659+0000", 2024-09-16T20:50:28.925 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.064592822999999994 2024-09-16T20:50:28.925 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.925 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.925 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:28.925 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833719+0000", 2024-09-16T20:50:28.926 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9441000000000002e-05 2024-09-16T20:50:28.926 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.926 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.926 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:28.926 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833758+0000", 2024-09-16T20:50:28.926 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9122000000000003e-05 2024-09-16T20:50:28.926 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.926 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.926 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:28.926 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833790+0000", 2024-09-16T20:50:28.926 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2338000000000001e-05 2024-09-16T20:50:28.926 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.926 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:28.926 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.927 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.927 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.927 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.5089.0:12732 3.d e73/23 ondisk, result = 0)", 2024-09-16T20:50:28.927 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:50:28.927 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 84.090606589999993, 2024-09-16T20:50:28.927 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.063706456999999994, 2024-09-16T20:50:28.927 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:28.927 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:28.927 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:28.927 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:28.927 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.927 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:28.927 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:50:28.928 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.928 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.928 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.928 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:28.928 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:50:28.928 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.928 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.928 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.928 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:28.928 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770091+0000", 2024-09-16T20:50:28.928 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.53e-06 2024-09-16T20:50:28.928 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.928 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.928 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:28.929 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770104+0000", 2024-09-16T20:50:28.929 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2203e-05 2024-09-16T20:50:28.929 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.929 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.929 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:28.929 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833637+0000", 2024-09-16T20:50:28.929 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.063533201999999997 2024-09-16T20:50:28.929 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.929 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.929 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:28.929 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833655+0000", 2024-09-16T20:50:28.929 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8369e-05 2024-09-16T20:50:28.929 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.930 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.930 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:28.930 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833707+0000", 2024-09-16T20:50:28.930 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2061999999999997e-05 2024-09-16T20:50:28.930 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.930 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.930 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:28.930 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833715+0000", 2024-09-16T20:50:28.930 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5179999999999998e-06 2024-09-16T20:50:28.930 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.930 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.930 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:28.930 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833796+0000", 2024-09-16T20:50:28.930 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1573e-05 2024-09-16T20:50:28.931 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.931 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:28.931 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.931 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.931 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.931 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3282 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 225280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:28.931 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041081+0000", 2024-09-16T20:50:28.931 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 38.819615843000001, 2024-09-16T20:50:28.931 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051731948, 2024-09-16T20:50:28.931 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:28.931 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:28.931 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:28.931 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:28.931 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:28.932 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:28.932 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3282 2024-09-16T20:50:28.932 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.932 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:28.932 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.932 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:28.932 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041081+0000", 2024-09-16T20:50:28.932 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.932 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.932 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.932 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:28.932 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041081+0000", 2024-09-16T20:50:28.932 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.933 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.933 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.933 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:28.933 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041081+0000", 2024-09-16T20:50:28.933 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3099999999999999e-07 2024-09-16T20:50:28.933 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.933 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.933 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:28.933 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041082+0000", 2024-09-16T20:50:28.933 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1289999999999999e-06 2024-09-16T20:50:28.933 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.933 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.933 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:28.933 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041085+0000", 2024-09-16T20:50:28.934 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2320000000000001e-06 2024-09-16T20:50:28.934 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.934 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.934 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:28.934 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041090+0000", 2024-09-16T20:50:28.934 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3730000000000003e-06 2024-09-16T20:50:28.934 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.934 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.934 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:28.934 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.092678+0000", 2024-09-16T20:50:28.934 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051588235000000003 2024-09-16T20:50:28.934 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.934 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.934 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:28.934 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.092717+0000", 2024-09-16T20:50:28.935 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9057999999999998e-05 2024-09-16T20:50:28.935 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.935 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.935 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:28.935 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.092813+0000", 2024-09-16T20:50:28.935 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.569e-05 2024-09-16T20:50:28.935 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.935 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:28.935 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.935 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.935 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.935 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3283 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 230400~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:28.935 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041102+0000", 2024-09-16T20:50:28.935 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 38.819594183, 2024-09-16T20:50:28.936 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051974723, 2024-09-16T20:50:28.936 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:28.936 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:28.936 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:28.936 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:28.936 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:28.936 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:28.936 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3283 2024-09-16T20:50:28.936 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.936 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:28.936 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.936 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:28.937 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041102+0000", 2024-09-16T20:50:28.937 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.937 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.937 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.937 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:28.937 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041102+0000", 2024-09-16T20:50:28.937 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.937 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.937 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.937 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:28.937 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041103+0000", 2024-09-16T20:50:28.937 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2399999999999999e-07 2024-09-16T20:50:28.937 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.937 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.938 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:28.938 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041110+0000", 2024-09-16T20:50:28.938 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4290000000000004e-06 2024-09-16T20:50:28.938 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.938 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.938 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:28.938 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041118+0000", 2024-09-16T20:50:28.938 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3849999999999996e-06 2024-09-16T20:50:28.938 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.938 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.938 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:28.938 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041123+0000", 2024-09-16T20:50:28.939 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1410000000000004e-06 2024-09-16T20:50:28.939 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.939 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.939 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:28.939 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.092845+0000", 2024-09-16T20:50:28.939 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051722207999999999 2024-09-16T20:50:28.939 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.939 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.939 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:28.939 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.092926+0000", 2024-09-16T20:50:28.939 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0727000000000001e-05 2024-09-16T20:50:28.939 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.940 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.940 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:28.940 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093077+0000", 2024-09-16T20:50:28.940 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000151409 2024-09-16T20:50:28.940 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.940 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:28.940 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.940 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.940 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.940 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3284 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 235520~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:28.940 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041129+0000", 2024-09-16T20:50:28.940 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 38.819567851999999, 2024-09-16T20:50:28.941 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052120760000000002, 2024-09-16T20:50:28.941 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:28.941 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:28.941 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:28.941 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:28.941 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:28.941 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:28.941 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3284 2024-09-16T20:50:28.941 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.941 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:28.941 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.941 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:28.941 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041129+0000", 2024-09-16T20:50:28.942 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.942 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.942 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.942 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:28.943 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041129+0000", 2024-09-16T20:50:28.943 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.944 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.944 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.944 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:28.944 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041129+0000", 2024-09-16T20:50:28.944 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3700000000000001e-07 2024-09-16T20:50:28.944 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.944 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.944 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:28.944 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041130+0000", 2024-09-16T20:50:28.944 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4050000000000001e-06 2024-09-16T20:50:28.944 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.944 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.944 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:28.945 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041134+0000", 2024-09-16T20:50:28.945 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4290000000000001e-06 2024-09-16T20:50:28.945 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.945 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.945 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:28.945 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041136+0000", 2024-09-16T20:50:28.945 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.678e-06 2024-09-16T20:50:28.945 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.945 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.945 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:28.945 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093109+0000", 2024-09-16T20:50:28.945 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051972879999999999 2024-09-16T20:50:28.945 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.945 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.946 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:28.946 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093149+0000", 2024-09-16T20:50:28.946 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0040000000000003e-05 2024-09-16T20:50:28.946 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.946 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.946 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:28.946 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093249+0000", 2024-09-16T20:50:28.946 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.9990999999999999e-05 2024-09-16T20:50:28.946 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.946 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:28.946 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.946 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.946 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.946 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3285 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 240640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:28.947 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041140+0000", 2024-09-16T20:50:28.947 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 38.819556812000002, 2024-09-16T20:50:28.947 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052375646999999997, 2024-09-16T20:50:28.947 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:28.947 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:28.947 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:28.947 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:28.947 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:28.947 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:28.947 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3285 2024-09-16T20:50:28.947 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.947 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:28.947 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.947 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:28.948 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041140+0000", 2024-09-16T20:50:28.948 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.948 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.948 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.948 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:28.948 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041140+0000", 2024-09-16T20:50:28.948 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.948 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.948 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.948 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:28.948 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041140+0000", 2024-09-16T20:50:28.948 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2399999999999999e-07 2024-09-16T20:50:28.948 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.949 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.949 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:28.949 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041141+0000", 2024-09-16T20:50:28.949 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2759999999999999e-06 2024-09-16T20:50:28.949 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.949 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.950 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:28.950 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041144+0000", 2024-09-16T20:50:28.950 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0280000000000001e-06 2024-09-16T20:50:28.950 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.950 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.950 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:28.950 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041147+0000", 2024-09-16T20:50:28.950 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5900000000000002e-06 2024-09-16T20:50:28.950 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.950 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.950 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:28.950 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093284+0000", 2024-09-16T20:50:28.951 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052137645000000003 2024-09-16T20:50:28.951 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.951 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.951 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:28.951 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093365+0000", 2024-09-16T20:50:28.951 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0446999999999997e-05 2024-09-16T20:50:28.951 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.951 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.951 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:28.951 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093515+0000", 2024-09-16T20:50:28.951 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00015043700000000001 2024-09-16T20:50:28.951 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.951 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:28.951 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.952 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.952 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.952 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3286 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 245760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:28.952 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:50:28.952 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 38.819544653000001, 2024-09-16T20:50:28.952 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052523564000000002, 2024-09-16T20:50:28.952 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:28.952 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:28.952 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:28.952 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:28.952 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:28.952 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:28.953 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3286 2024-09-16T20:50:28.953 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.953 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:28.953 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.953 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:28.953 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:50:28.953 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.953 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.953 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.953 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:28.953 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:50:28.953 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.953 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.954 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.954 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:28.954 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:50:28.954 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3900000000000001e-07 2024-09-16T20:50:28.954 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.954 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.954 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:28.954 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041153+0000", 2024-09-16T20:50:28.954 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.119e-06 2024-09-16T20:50:28.954 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.954 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.954 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:28.954 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041156+0000", 2024-09-16T20:50:28.955 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0000000000000001e-06 2024-09-16T20:50:28.955 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.955 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.955 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:28.955 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041159+0000", 2024-09-16T20:50:28.955 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5859999999999999e-06 2024-09-16T20:50:28.955 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.955 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.955 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:28.955 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093529+0000", 2024-09-16T20:50:28.955 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052370591000000001 2024-09-16T20:50:28.955 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.955 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.956 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:28.956 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093570+0000", 2024-09-16T20:50:28.956 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0161999999999999e-05 2024-09-16T20:50:28.956 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.956 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.956 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:28.956 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093675+0000", 2024-09-16T20:50:28.956 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010586699999999999 2024-09-16T20:50:28.956 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.956 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:28.956 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.956 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.956 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.957 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3287 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 250880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:28.957 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:50:28.957 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 38.819534603999998, 2024-09-16T20:50:28.957 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052781203999999998, 2024-09-16T20:50:28.957 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:28.957 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:28.957 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:28.957 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:28.957 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:28.957 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:28.957 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3287 2024-09-16T20:50:28.957 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.958 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:28.958 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.958 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:28.958 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:50:28.958 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.958 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.958 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.958 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:28.958 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:50:28.958 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.958 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.958 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.958 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:28.958 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:50:28.959 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1e-07 2024-09-16T20:50:28.959 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.959 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.959 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:28.959 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041163+0000", 2024-09-16T20:50:28.959 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0690000000000001e-06 2024-09-16T20:50:28.959 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.959 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.959 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:28.959 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041166+0000", 2024-09-16T20:50:28.959 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.982e-06 2024-09-16T20:50:28.959 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.959 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.960 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:28.960 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041169+0000", 2024-09-16T20:50:28.960 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.593e-06 2024-09-16T20:50:28.960 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.960 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.960 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:28.960 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093709+0000", 2024-09-16T20:50:28.960 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052540492000000001 2024-09-16T20:50:28.960 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.960 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.960 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:28.960 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093792+0000", 2024-09-16T20:50:28.960 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2436999999999997e-05 2024-09-16T20:50:28.961 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.961 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.961 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:28.961 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093943+0000", 2024-09-16T20:50:28.961 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000151421 2024-09-16T20:50:28.961 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.961 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:28.961 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.961 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.961 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.961 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3288 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 256000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:28.961 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:50:28.961 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 38.819524731000001, 2024-09-16T20:50:28.961 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052928269, 2024-09-16T20:50:28.962 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:28.962 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:28.962 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:28.962 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:28.962 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:28.962 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:28.962 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3288 2024-09-16T20:50:28.962 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.962 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:28.962 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.962 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:28.962 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:50:28.962 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.963 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.963 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.963 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:28.963 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:50:28.963 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.963 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.963 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.963 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:28.963 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:50:28.963 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2700000000000001e-07 2024-09-16T20:50:28.963 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.963 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.963 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:28.964 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041173+0000", 2024-09-16T20:50:28.964 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.113e-06 2024-09-16T20:50:28.964 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.964 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.964 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:28.964 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041176+0000", 2024-09-16T20:50:28.964 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9799999999999998e-06 2024-09-16T20:50:28.964 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.964 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.964 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:28.964 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041179+0000", 2024-09-16T20:50:28.964 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.667e-06 2024-09-16T20:50:28.964 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.964 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.965 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:28.965 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093957+0000", 2024-09-16T20:50:28.965 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052778703000000003 2024-09-16T20:50:28.965 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.965 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.965 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:28.965 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093999+0000", 2024-09-16T20:50:28.965 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1300000000000001e-05 2024-09-16T20:50:28.965 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.965 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.965 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:28.965 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094100+0000", 2024-09-16T20:50:28.966 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101279 2024-09-16T20:50:28.966 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.966 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:28.966 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.966 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.966 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.966 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3289 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 261120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:28.966 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:50:28.966 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 38.819514734999998, 2024-09-16T20:50:28.966 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053182976, 2024-09-16T20:50:28.966 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:28.966 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:28.966 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:28.967 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:28.967 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:28.967 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:28.967 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3289 2024-09-16T20:50:28.967 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.967 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:28.967 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.967 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:28.967 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:50:28.967 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.967 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.967 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.968 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:28.968 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:50:28.968 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.968 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.968 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.968 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:28.968 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:50:28.968 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2999999999999999e-07 2024-09-16T20:50:28.968 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.968 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.968 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:28.968 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041183+0000", 2024-09-16T20:50:28.969 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0720000000000001e-06 2024-09-16T20:50:28.969 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.969 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.969 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:28.969 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041186+0000", 2024-09-16T20:50:28.969 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9869999999999999e-06 2024-09-16T20:50:28.969 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.969 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.969 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:28.969 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041189+0000", 2024-09-16T20:50:28.969 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.587e-06 2024-09-16T20:50:28.969 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.969 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.970 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:28.970 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094136+0000", 2024-09-16T20:50:28.970 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052947270999999997 2024-09-16T20:50:28.970 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.970 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.970 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:28.970 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094213+0000", 2024-09-16T20:50:28.970 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7552000000000006e-05 2024-09-16T20:50:28.970 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.970 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.970 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:28.970 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094365+0000", 2024-09-16T20:50:28.970 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00015127700000000001 2024-09-16T20:50:28.970 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.971 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:28.971 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.971 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.971 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.971 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:2196 3.d 3:b1fe01a0:::rbd_data.110e7ef3004.0000000000000000:2d [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e136)", 2024-09-16T20:50:28.971 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:22.739057+0000", 2024-09-16T20:50:28.971 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 6.1216399360000002, 2024-09-16T20:50:28.971 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11999464899999999, 2024-09-16T20:50:28.971 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:28.971 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:28.971 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:28.971 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:28.971 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:50:28.972 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:28.972 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2196 2024-09-16T20:50:28.972 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.972 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:28.972 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.972 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:28.972 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.739057+0000", 2024-09-16T20:50:28.972 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.972 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.972 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.972 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:28.972 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.739057+0000", 2024-09-16T20:50:28.972 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:28.973 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.973 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.973 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:28.973 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.739057+0000", 2024-09-16T20:50:28.973 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1399999999999997e-07 2024-09-16T20:50:28.973 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.973 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.973 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:28.973 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.739060+0000", 2024-09-16T20:50:28.973 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6019999999999998e-06 2024-09-16T20:50:28.973 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.973 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.973 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:28.974 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.858755+0000", 2024-09-16T20:50:28.974 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.119695151 2024-09-16T20:50:28.974 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.974 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.974 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:28.974 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.858778+0000", 2024-09-16T20:50:28.974 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3184999999999999e-05 2024-09-16T20:50:28.974 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.974 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.974 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:28.974 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.858865+0000", 2024-09-16T20:50:28.974 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.7330000000000006e-05 2024-09-16T20:50:28.974 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.974 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.975 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:28.975 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.858952+0000", 2024-09-16T20:50:28.975 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6803000000000002e-05 2024-09-16T20:50:28.975 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:28.975 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:28.975 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:28.975 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.859051+0000", 2024-09-16T20:50:28.975 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8963999999999996e-05 2024-09-16T20:50:28.975 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.975 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:28.975 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.975 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:28.975 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:28.975 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:28.976 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T20:50:28.978 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:50:28.978 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:28.975+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:29.078 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:29.077+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:29.130 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:29.131 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:50:29.131 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:50:29.131 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:29.143 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:50:29.150 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:50:29.150 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:50:29.168 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:50:29.168 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:50:29.178 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:29.177+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:29.185 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:50:29.185 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:50:29.202 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:50:29.202 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:50:29.219 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:50:29.219 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:50:29.236 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:50:29.237 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:50:29.251 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:29.251 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:29.260 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:50:29.301 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:29.300+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:29.380 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:29.380+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:29.407 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:29.407 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:29.416 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:50:29.481 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:29.480+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:29.535 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:29.536 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:29.544 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:50:29.588 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:29.588+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:29.652 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:29.652 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:29.660 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:50:29.683 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:29.682+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:29.741 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 4, 3, 5] out_osds: [0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-16T20:50:29.741 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T20:50:29.742 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-16T20:50:29.742 INFO:tasks.thrashosds.thrasher:Setting osd 2 primary_affinity to 1.000000 2024-09-16T20:50:29.742 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-16T20:50:29.786 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:29.785+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:29.828 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:29.828 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:29.838 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:50:29.884 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:29.884+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:29.983 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:29.983 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:29.985 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:29.984+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:29.992 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:50:30.086 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:30.085+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:30.141 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:30.141 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:30.149 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:50:30.187 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:30.186+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:30.276 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:30.276 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:30.286 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:50:30.287 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:30.286+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:30.388 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:30.387+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:30.422 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:30.422 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:30.431 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:50:30.489 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:30.488+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:30.571 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:30.571 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:30.582 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:50:30.590 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:30.589+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:30.691 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:30.690+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:30.731 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:30.731 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:30.740 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:50:30.769 INFO:teuthology.orchestra.run.smithi029.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-16T20:50:30.782 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T20:50:30.792 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:30.791+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:30.857 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:30.858 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:30.867 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:50:30.908 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:30.907+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:30.994 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:30.993+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:31.038 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:31.039 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:31.050 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:50:31.094 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:31.093+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:31.163 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:31.164 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:31.172 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:50:31.196 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:31.195+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:31.248 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T20:50:31.301 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:31.300+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:31.323 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:31.323 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:31.332 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:50:31.384 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:50:31.384 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:50:31.397 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:31.397+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:31.402 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:50:31.402 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:50:31.419 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:50:31.419 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:50:31.436 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:50:31.436 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:50:31.454 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:50:31.454 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:50:31.471 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:50:31.471 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:50:31.484 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:31.485 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:50:31.485 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:50:31.485 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:50:31.485 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:31.493 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:50:31.497 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:31.497+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:31.601 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:31.600+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:31.643 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:31.643 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:50:31.643 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:50:31.643 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:50:31.643 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.643 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:50:31.643 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:50:31.643 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 158.38687986299999, 2024-09-16T20:50:31.643 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67502346999999996, 2024-09-16T20:50:31.643 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:31.643 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:31.644 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:31.644 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:31.644 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:50:31.644 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:50:31.644 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 1 2024-09-16T20:50:31.644 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.644 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:31.644 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.644 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:31.644 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:50:31.644 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:31.644 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.644 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.645 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:31.645 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:50:31.645 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:31.645 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.645 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.645 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:31.645 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254128+0000", 2024-09-16T20:50:31.645 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.463e-06 2024-09-16T20:50:31.645 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.645 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.645 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:31.645 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254135+0000", 2024-09-16T20:50:31.645 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1579999999999999e-06 2024-09-16T20:50:31.645 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.646 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.646 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:31.646 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254146+0000", 2024-09-16T20:50:31.646 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1341e-05 2024-09-16T20:50:31.646 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.646 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.646 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:31.646 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254155+0000", 2024-09-16T20:50:31.646 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.1549999999999996e-06 2024-09-16T20:50:31.646 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.646 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.646 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:31.646 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254221+0000", 2024-09-16T20:50:31.646 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5470999999999997e-05 2024-09-16T20:50:31.647 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.647 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.647 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:31.647 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254231+0000", 2024-09-16T20:50:31.647 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0592e-05 2024-09-16T20:50:31.647 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.647 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.647 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:31.647 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926711+0000", 2024-09-16T20:50:31.647 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67247946599999997 2024-09-16T20:50:31.647 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.647 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.647 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:31.647 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926910+0000", 2024-09-16T20:50:31.648 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019934 2024-09-16T20:50:31.648 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.648 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.648 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:50:31.648 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.927159+0000", 2024-09-16T20:50:31.648 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00024876599999999998 2024-09-16T20:50:31.648 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.648 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.648 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:31.648 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.928797+0000", 2024-09-16T20:50:31.648 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001638308 2024-09-16T20:50:31.648 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.648 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.649 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:31.649 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929026+0000", 2024-09-16T20:50:31.649 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00022876099999999999 2024-09-16T20:50:31.649 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.649 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.649 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:31.649 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929102+0000", 2024-09-16T20:50:31.649 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6184000000000001e-05 2024-09-16T20:50:31.649 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.649 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.649 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:31.649 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929150+0000", 2024-09-16T20:50:31.649 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7465e-05 2024-09-16T20:50:31.650 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:31.650 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:31.650 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:31.650 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.650 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.650 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3574 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:31.650 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:50:31.650 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 102.159716331, 2024-09-16T20:50:31.650 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053579441999999998, 2024-09-16T20:50:31.650 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:31.650 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:31.650 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:31.650 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:31.651 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:31.651 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:31.651 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3574 2024-09-16T20:50:31.651 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.651 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:31.651 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.651 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:31.651 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:50:31.651 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:31.651 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.651 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.651 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:31.651 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:50:31.651 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:31.652 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.652 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.652 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:31.652 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:50:31.652 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2e-07 2024-09-16T20:50:31.652 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.652 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.652 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:31.652 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481292+0000", 2024-09-16T20:50:31.652 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2630000000000002e-06 2024-09-16T20:50:31.652 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.652 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.652 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:31.652 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481297+0000", 2024-09-16T20:50:31.652 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.296e-06 2024-09-16T20:50:31.653 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.653 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.653 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:31.653 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481300+0000", 2024-09-16T20:50:31.653 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7019999999999999e-06 2024-09-16T20:50:31.653 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.653 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.653 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:31.653 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525533+0000", 2024-09-16T20:50:31.653 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.044232318 2024-09-16T20:50:31.653 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.653 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.653 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:31.654 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525619+0000", 2024-09-16T20:50:31.654 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5892999999999999e-05 2024-09-16T20:50:31.654 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.654 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.654 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:31.654 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525987+0000", 2024-09-16T20:50:31.654 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00036836099999999999 2024-09-16T20:50:31.654 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.654 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.654 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:31.654 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531779+0000", 2024-09-16T20:50:31.654 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0057918240000000001 2024-09-16T20:50:31.654 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.655 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.655 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:31.655 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534747+0000", 2024-09-16T20:50:31.655 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0029681600000000001 2024-09-16T20:50:31.655 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.655 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.655 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:31.655 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534832+0000", 2024-09-16T20:50:31.655 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5180999999999996e-05 2024-09-16T20:50:31.655 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.655 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.655 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:31.655 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534869+0000", 2024-09-16T20:50:31.655 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7024000000000003e-05 2024-09-16T20:50:31.655 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:31.656 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:31.656 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:31.656 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.656 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.656 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3575 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:31.656 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:50:31.656 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 102.159701591, 2024-09-16T20:50:31.656 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053801463000000001, 2024-09-16T20:50:31.656 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:31.656 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:31.656 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:31.656 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:31.656 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:31.656 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:31.657 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3575 2024-09-16T20:50:31.657 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.657 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:31.657 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.657 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:31.657 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:50:31.657 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:31.657 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.657 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.657 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:31.657 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:50:31.657 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:31.657 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.657 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.658 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:31.658 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481305+0000", 2024-09-16T20:50:31.658 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5400000000000002e-07 2024-09-16T20:50:31.658 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.658 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.658 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:31.658 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481307+0000", 2024-09-16T20:50:31.658 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7800000000000001e-06 2024-09-16T20:50:31.658 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.658 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.658 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:31.658 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481312+0000", 2024-09-16T20:50:31.658 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6349999999999997e-06 2024-09-16T20:50:31.659 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.659 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.659 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:31.659 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481315+0000", 2024-09-16T20:50:31.659 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7829999999999999e-06 2024-09-16T20:50:31.659 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.659 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.659 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:31.659 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.526876+0000", 2024-09-16T20:50:31.659 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.045561070000000002 2024-09-16T20:50:31.659 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.659 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.659 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:31.660 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.526954+0000", 2024-09-16T20:50:31.660 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8108000000000005e-05 2024-09-16T20:50:31.660 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.660 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.660 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:31.660 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527282+0000", 2024-09-16T20:50:31.660 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000327483 2024-09-16T20:50:31.660 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.660 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.660 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:31.660 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531784+0000", 2024-09-16T20:50:31.660 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0045020169999999997 2024-09-16T20:50:31.660 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.661 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.661 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:31.661 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534921+0000", 2024-09-16T20:50:31.661 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.003137791 2024-09-16T20:50:31.661 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.661 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.661 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:31.661 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534976+0000", 2024-09-16T20:50:31.661 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4953000000000001e-05 2024-09-16T20:50:31.661 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.661 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.661 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:31.661 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535106+0000", 2024-09-16T20:50:31.662 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000129589 2024-09-16T20:50:31.662 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:31.662 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:31.662 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:31.662 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.662 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.662 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3576 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:31.662 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:50:31.662 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 102.159687679, 2024-09-16T20:50:31.662 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053955436000000002, 2024-09-16T20:50:31.662 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:31.662 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:31.662 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:31.662 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:31.663 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:31.663 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:31.663 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3576 2024-09-16T20:50:31.663 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.663 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:31.663 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.663 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:31.663 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:50:31.663 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:31.663 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.663 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.663 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:31.663 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:50:31.663 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:31.663 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.664 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.664 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:31.664 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481319+0000", 2024-09-16T20:50:31.664 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.53e-07 2024-09-16T20:50:31.664 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.664 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.664 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:31.664 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481320+0000", 2024-09-16T20:50:31.664 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5230000000000001e-06 2024-09-16T20:50:31.664 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.664 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.664 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:31.664 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481330+0000", 2024-09-16T20:50:31.664 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0190000000000001e-05 2024-09-16T20:50:31.665 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.665 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.665 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:31.665 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481333+0000", 2024-09-16T20:50:31.665 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.151e-06 2024-09-16T20:50:31.665 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.665 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.665 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:31.665 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527689+0000", 2024-09-16T20:50:31.665 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046355192000000003 2024-09-16T20:50:31.665 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.665 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.665 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:31.665 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527764+0000", 2024-09-16T20:50:31.666 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5776000000000001e-05 2024-09-16T20:50:31.666 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.666 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.666 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:31.666 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528141+0000", 2024-09-16T20:50:31.666 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00037647599999999999 2024-09-16T20:50:31.666 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.666 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.666 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:31.666 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531788+0000", 2024-09-16T20:50:31.666 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0036470679999999998 2024-09-16T20:50:31.666 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.666 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.666 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:31.667 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535174+0000", 2024-09-16T20:50:31.667 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0033862670000000001 2024-09-16T20:50:31.667 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.667 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.667 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:31.667 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535236+0000", 2024-09-16T20:50:31.667 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1797999999999993e-05 2024-09-16T20:50:31.667 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.667 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.667 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:31.667 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535274+0000", 2024-09-16T20:50:31.667 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7741999999999999e-05 2024-09-16T20:50:31.667 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:31.667 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:31.667 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:31.668 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.668 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.668 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3577 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:31.668 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:50:31.668 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 102.159666079, 2024-09-16T20:50:31.668 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054073807000000002, 2024-09-16T20:50:31.668 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:31.668 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:31.668 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:31.668 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:31.668 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:31.668 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:31.668 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3577 2024-09-16T20:50:31.668 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.669 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:31.669 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.669 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:31.669 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:50:31.669 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:31.669 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.669 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.669 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:31.669 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:50:31.669 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:31.669 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.669 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.669 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:31.670 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:50:31.670 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.96e-07 2024-09-16T20:50:31.670 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.670 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.670 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:31.670 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481342+0000", 2024-09-16T20:50:31.670 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.714e-06 2024-09-16T20:50:31.670 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.670 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.670 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:31.670 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481349+0000", 2024-09-16T20:50:31.670 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4930000000000001e-06 2024-09-16T20:50:31.670 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.670 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.670 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:31.671 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481353+0000", 2024-09-16T20:50:31.671 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4879999999999999e-06 2024-09-16T20:50:31.671 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.671 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.671 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:31.671 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528619+0000", 2024-09-16T20:50:31.671 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047266351999999998 2024-09-16T20:50:31.671 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.671 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.671 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:31.671 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528706+0000", 2024-09-16T20:50:31.671 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6796e-05 2024-09-16T20:50:31.671 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.672 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.672 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:31.672 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529148+0000", 2024-09-16T20:50:31.672 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00044182999999999999 2024-09-16T20:50:31.672 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.672 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.672 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:31.672 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531792+0000", 2024-09-16T20:50:31.672 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0026443119999999998 2024-09-16T20:50:31.672 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.672 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.672 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:31.672 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535319+0000", 2024-09-16T20:50:31.672 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0035266439999999998 2024-09-16T20:50:31.673 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.673 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.673 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:31.673 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535372+0000", 2024-09-16T20:50:31.673 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3207000000000001e-05 2024-09-16T20:50:31.673 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.673 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.673 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:31.673 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535414+0000", 2024-09-16T20:50:31.673 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1675000000000003e-05 2024-09-16T20:50:31.673 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:31.673 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:31.673 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:31.673 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.674 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.674 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3578 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:31.674 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:50:31.674 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 102.15964491, 2024-09-16T20:50:31.674 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054189586999999997, 2024-09-16T20:50:31.674 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:31.674 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:31.674 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:31.674 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:31.674 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:31.674 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:31.674 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3578 2024-09-16T20:50:31.674 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.674 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:31.675 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.675 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:31.675 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:50:31.675 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:31.675 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.675 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.675 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:31.675 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:50:31.675 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:31.675 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.675 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.675 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:31.675 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:50:31.675 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0800000000000001e-07 2024-09-16T20:50:31.676 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.676 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.676 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:31.676 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481363+0000", 2024-09-16T20:50:31.676 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0339999999999999e-06 2024-09-16T20:50:31.676 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.676 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.676 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:31.676 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481371+0000", 2024-09-16T20:50:31.676 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4440000000000002e-06 2024-09-16T20:50:31.676 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.676 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.676 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:31.677 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481374+0000", 2024-09-16T20:50:31.677 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2049999999999998e-06 2024-09-16T20:50:31.677 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.677 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.677 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:31.677 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529555+0000", 2024-09-16T20:50:31.677 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048181202999999999 2024-09-16T20:50:31.677 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.677 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.677 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:31.677 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529616+0000", 2024-09-16T20:50:31.677 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0912000000000002e-05 2024-09-16T20:50:31.677 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.677 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.678 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:31.678 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529939+0000", 2024-09-16T20:50:31.678 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00032269800000000001 2024-09-16T20:50:31.678 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.678 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.678 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:31.678 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532602+0000", 2024-09-16T20:50:31.678 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0026634940000000002 2024-09-16T20:50:31.678 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.678 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.678 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:31.678 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535465+0000", 2024-09-16T20:50:31.679 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.002862149 2024-09-16T20:50:31.679 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.679 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.679 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:31.679 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535516+0000", 2024-09-16T20:50:31.679 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1696000000000001e-05 2024-09-16T20:50:31.679 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.679 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.679 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:31.679 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535551+0000", 2024-09-16T20:50:31.679 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4443999999999998e-05 2024-09-16T20:50:31.679 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:31.679 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:31.679 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:31.680 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.680 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.680 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3579 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:31.680 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:50:31.680 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 102.159626471, 2024-09-16T20:50:31.680 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054295151, 2024-09-16T20:50:31.680 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:31.680 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:31.680 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:31.680 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:31.680 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:31.680 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:31.680 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3579 2024-09-16T20:50:31.681 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.681 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:31.681 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.681 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:31.681 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:50:31.681 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:31.681 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.681 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.681 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:31.681 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:50:31.681 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:31.681 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.681 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.681 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:31.682 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:50:31.682 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3300000000000001e-07 2024-09-16T20:50:31.682 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.682 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.682 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:31.682 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481381+0000", 2024-09-16T20:50:31.682 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.649e-06 2024-09-16T20:50:31.682 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.682 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.682 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:31.682 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481387+0000", 2024-09-16T20:50:31.682 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4519999999999996e-06 2024-09-16T20:50:31.682 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.683 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.683 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:31.683 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481390+0000", 2024-09-16T20:50:31.683 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0170000000000001e-06 2024-09-16T20:50:31.683 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.683 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.683 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:31.683 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530756+0000", 2024-09-16T20:50:31.683 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049365882999999999 2024-09-16T20:50:31.683 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.683 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.683 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:31.683 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530845+0000", 2024-09-16T20:50:31.683 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9197000000000007e-05 2024-09-16T20:50:31.684 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.684 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.684 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:31.684 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531191+0000", 2024-09-16T20:50:31.684 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00034562500000000001 2024-09-16T20:50:31.684 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.684 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.684 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:31.684 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534078+0000", 2024-09-16T20:50:31.684 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0028878179999999999 2024-09-16T20:50:31.684 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.684 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.684 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:31.684 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535589+0000", 2024-09-16T20:50:31.685 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00151085 2024-09-16T20:50:31.685 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.685 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.685 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:31.685 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535639+0000", 2024-09-16T20:50:31.685 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9697000000000002e-05 2024-09-16T20:50:31.685 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.685 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.685 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:31.685 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535675+0000", 2024-09-16T20:50:31.685 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5729999999999998e-05 2024-09-16T20:50:31.685 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:31.685 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:31.685 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:31.685 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.686 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.686 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3580 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:31.686 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:50:31.686 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 102.159612839, 2024-09-16T20:50:31.686 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054397361999999998, 2024-09-16T20:50:31.686 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:31.686 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:31.686 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:31.686 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:31.686 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:31.686 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:31.686 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3580 2024-09-16T20:50:31.686 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.686 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:31.687 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.687 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:31.687 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:50:31.687 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:31.687 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.687 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.687 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:31.687 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:50:31.687 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:31.687 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.687 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.687 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:31.687 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:50:31.687 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2600000000000001e-07 2024-09-16T20:50:31.687 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.688 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.688 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:31.688 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481395+0000", 2024-09-16T20:50:31.688 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.544e-06 2024-09-16T20:50:31.688 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.688 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.688 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:31.688 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481404+0000", 2024-09-16T20:50:31.688 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8850000000000005e-06 2024-09-16T20:50:31.688 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.688 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.688 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:31.688 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481407+0000", 2024-09-16T20:50:31.688 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1540000000000002e-06 2024-09-16T20:50:31.689 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.689 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.689 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:31.689 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531823+0000", 2024-09-16T20:50:31.689 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050415637999999999 2024-09-16T20:50:31.689 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.689 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.689 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:31.689 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531895+0000", 2024-09-16T20:50:31.689 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2179000000000001e-05 2024-09-16T20:50:31.689 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.689 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.689 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:31.689 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532236+0000", 2024-09-16T20:50:31.690 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00034073799999999999 2024-09-16T20:50:31.690 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.690 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.690 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:31.690 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534083+0000", 2024-09-16T20:50:31.690 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001847298 2024-09-16T20:50:31.690 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.690 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.690 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:31.690 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535714+0000", 2024-09-16T20:50:31.690 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0016316810000000001 2024-09-16T20:50:31.690 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.690 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.690 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:31.690 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535759+0000", 2024-09-16T20:50:31.691 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4478999999999999e-05 2024-09-16T20:50:31.691 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.691 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.691 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:31.691 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535791+0000", 2024-09-16T20:50:31.691 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.154e-05 2024-09-16T20:50:31.691 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:31.691 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:31.691 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:31.691 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.691 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.691 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3581 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:31.691 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:50:31.691 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 102.159590842, 2024-09-16T20:50:31.691 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.057094403000000002, 2024-09-16T20:50:31.691 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:31.692 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:31.692 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:31.692 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:31.692 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:31.692 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:31.692 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3581 2024-09-16T20:50:31.692 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.692 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:31.692 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.692 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:31.692 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:50:31.692 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:31.692 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.692 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.693 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:31.693 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:50:31.693 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:31.693 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.693 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.693 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:31.693 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:50:31.693 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8799999999999998e-07 2024-09-16T20:50:31.693 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.693 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.693 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:31.693 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481418+0000", 2024-09-16T20:50:31.693 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1859999999999999e-06 2024-09-16T20:50:31.693 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.694 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.694 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:31.694 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481423+0000", 2024-09-16T20:50:31.694 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8279999999999999e-06 2024-09-16T20:50:31.694 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.694 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.694 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:31.694 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481426+0000", 2024-09-16T20:50:31.694 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0110000000000001e-06 2024-09-16T20:50:31.694 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.694 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.694 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:31.694 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532631+0000", 2024-09-16T20:50:31.694 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051204514999999999 2024-09-16T20:50:31.694 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.695 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.695 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:31.695 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.533381+0000", 2024-09-16T20:50:31.695 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00075029000000000003 2024-09-16T20:50:31.695 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.695 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.695 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:31.695 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.533739+0000", 2024-09-16T20:50:31.695 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00035773100000000002 2024-09-16T20:50:31.695 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.695 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.695 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:31.695 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535123+0000", 2024-09-16T20:50:31.695 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0013836160000000001 2024-09-16T20:50:31.696 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.696 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.696 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:31.696 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538413+0000", 2024-09-16T20:50:31.696 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0032902510000000001 2024-09-16T20:50:31.696 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.696 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.696 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:31.696 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538470+0000", 2024-09-16T20:50:31.696 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6697999999999998e-05 2024-09-16T20:50:31.696 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.696 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.696 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:31.697 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538510+0000", 2024-09-16T20:50:31.697 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9988999999999999e-05 2024-09-16T20:50:31.697 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:31.697 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:31.697 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:31.697 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.697 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.697 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:12732 3.d 3:b752444a:::rbd_header.110e747f0779:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-16T20:50:31.697 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:50:31.697 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 86.872518264000007, 2024-09-16T20:50:31.697 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.065302467000000003, 2024-09-16T20:50:31.697 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:31.697 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:31.698 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:31.698 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:31.698 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:50:31.698 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:50:31.698 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 12732 2024-09-16T20:50:31.698 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.698 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:31.698 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.698 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:31.698 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:50:31.698 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:31.698 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.699 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.699 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:31.699 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:50:31.699 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:31.699 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.699 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.699 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:31.699 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768490+0000", 2024-09-16T20:50:31.699 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2859999999999998e-06 2024-09-16T20:50:31.699 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.699 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.699 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:31.699 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768501+0000", 2024-09-16T20:50:31.700 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1226999999999999e-05 2024-09-16T20:50:31.700 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.700 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.700 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:31.700 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768517+0000", 2024-09-16T20:50:31.700 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.592e-05 2024-09-16T20:50:31.700 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.700 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.700 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:31.700 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768528+0000", 2024-09-16T20:50:31.700 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0689999999999999e-05 2024-09-16T20:50:31.700 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.700 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.700 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:31.701 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768584+0000", 2024-09-16T20:50:31.701 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5889e-05 2024-09-16T20:50:31.701 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.701 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.701 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:31.701 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768720+0000", 2024-09-16T20:50:31.701 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000136718 2024-09-16T20:50:31.701 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.701 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.701 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:31.701 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.769066+0000", 2024-09-16T20:50:31.701 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000346013 2024-09-16T20:50:31.701 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.701 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.702 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:31.702 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833659+0000", 2024-09-16T20:50:31.702 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.064592822999999994 2024-09-16T20:50:31.702 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.702 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.702 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:31.702 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833719+0000", 2024-09-16T20:50:31.702 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9441000000000002e-05 2024-09-16T20:50:31.702 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.702 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.702 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:31.702 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833758+0000", 2024-09-16T20:50:31.702 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9122000000000003e-05 2024-09-16T20:50:31.702 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.703 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.703 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:31.703 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833790+0000", 2024-09-16T20:50:31.703 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2338000000000001e-05 2024-09-16T20:50:31.703 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:31.703 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:31.703 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:31.703 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.703 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.703 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.5089.0:12732 3.d e73/23 ondisk, result = 0)", 2024-09-16T20:50:31.703 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:50:31.703 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 86.870916093999995, 2024-09-16T20:50:31.703 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.063706456999999994, 2024-09-16T20:50:31.703 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:31.703 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:31.704 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:31.704 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:31.704 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.704 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:31.704 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:50:31.704 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:31.704 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.704 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.704 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:31.704 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:50:31.704 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:31.704 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.705 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.705 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:31.705 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770091+0000", 2024-09-16T20:50:31.705 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.53e-06 2024-09-16T20:50:31.705 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.705 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.705 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:31.705 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770104+0000", 2024-09-16T20:50:31.705 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2203e-05 2024-09-16T20:50:31.705 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.705 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.705 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:31.705 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833637+0000", 2024-09-16T20:50:31.705 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.063533201999999997 2024-09-16T20:50:31.706 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.706 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.706 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:31.706 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833655+0000", 2024-09-16T20:50:31.706 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8369e-05 2024-09-16T20:50:31.706 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.706 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.706 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:31.706 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833707+0000", 2024-09-16T20:50:31.706 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2061999999999997e-05 2024-09-16T20:50:31.706 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.706 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.707 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:31.707 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833715+0000", 2024-09-16T20:50:31.707 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5179999999999998e-06 2024-09-16T20:50:31.707 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.707 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.707 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:31.707 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833796+0000", 2024-09-16T20:50:31.707 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1573e-05 2024-09-16T20:50:31.707 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:31.707 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:31.707 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:31.707 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.708 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.708 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3284 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 235520~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:31.708 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041129+0000", 2024-09-16T20:50:31.708 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 41.599877356, 2024-09-16T20:50:31.708 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052120760000000002, 2024-09-16T20:50:31.708 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:31.708 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:31.708 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:31.708 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:31.708 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:31.708 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:31.708 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3284 2024-09-16T20:50:31.709 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.709 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:31.709 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.709 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:31.709 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041129+0000", 2024-09-16T20:50:31.709 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:31.709 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.709 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.709 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:31.709 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041129+0000", 2024-09-16T20:50:31.709 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:31.709 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.710 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.710 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:31.710 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041129+0000", 2024-09-16T20:50:31.710 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3700000000000001e-07 2024-09-16T20:50:31.710 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.710 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.710 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:31.710 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041130+0000", 2024-09-16T20:50:31.710 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4050000000000001e-06 2024-09-16T20:50:31.710 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.710 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.710 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:31.710 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041134+0000", 2024-09-16T20:50:31.710 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4290000000000001e-06 2024-09-16T20:50:31.711 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.711 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.711 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:31.711 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041136+0000", 2024-09-16T20:50:31.711 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.678e-06 2024-09-16T20:50:31.711 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.711 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.711 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:31.711 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093109+0000", 2024-09-16T20:50:31.711 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051972879999999999 2024-09-16T20:50:31.711 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.711 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.711 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:31.711 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093149+0000", 2024-09-16T20:50:31.711 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0040000000000003e-05 2024-09-16T20:50:31.711 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.712 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.712 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:31.712 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093249+0000", 2024-09-16T20:50:31.712 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.9990999999999999e-05 2024-09-16T20:50:31.712 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:31.712 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:31.712 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:31.712 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.712 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.712 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3285 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 240640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:31.712 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041140+0000", 2024-09-16T20:50:31.712 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 41.599866316000004, 2024-09-16T20:50:31.712 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052375646999999997, 2024-09-16T20:50:31.712 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:31.713 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:31.713 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:31.713 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:31.713 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:31.713 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:31.713 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3285 2024-09-16T20:50:31.713 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.713 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:31.713 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.713 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:31.713 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041140+0000", 2024-09-16T20:50:31.713 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:31.713 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.713 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.714 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:31.714 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041140+0000", 2024-09-16T20:50:31.714 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:31.714 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.714 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.714 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:31.714 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041140+0000", 2024-09-16T20:50:31.714 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2399999999999999e-07 2024-09-16T20:50:31.714 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.714 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.714 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:31.714 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041141+0000", 2024-09-16T20:50:31.714 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2759999999999999e-06 2024-09-16T20:50:31.714 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.714 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.715 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:31.715 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041144+0000", 2024-09-16T20:50:31.715 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0280000000000001e-06 2024-09-16T20:50:31.715 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.715 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.715 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:31.715 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041147+0000", 2024-09-16T20:50:31.715 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5900000000000002e-06 2024-09-16T20:50:31.715 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.715 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.715 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:31.715 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093284+0000", 2024-09-16T20:50:31.715 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052137645000000003 2024-09-16T20:50:31.715 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.716 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.716 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:31.716 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093365+0000", 2024-09-16T20:50:31.716 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0446999999999997e-05 2024-09-16T20:50:31.716 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.716 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.716 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:31.716 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093515+0000", 2024-09-16T20:50:31.716 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00015043700000000001 2024-09-16T20:50:31.716 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:31.716 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:31.716 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:31.716 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.716 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.717 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3286 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 245760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:31.717 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:50:31.717 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 41.599854157000003, 2024-09-16T20:50:31.717 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052523564000000002, 2024-09-16T20:50:31.717 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:31.717 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:31.717 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:31.717 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:31.717 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:31.717 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:31.717 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3286 2024-09-16T20:50:31.717 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.717 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:31.718 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.718 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:31.718 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:50:31.718 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:31.718 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.718 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.718 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:31.718 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:50:31.718 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:31.718 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.718 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.718 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:31.718 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:50:31.718 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3900000000000001e-07 2024-09-16T20:50:31.718 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.719 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.719 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:31.719 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041153+0000", 2024-09-16T20:50:31.719 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.119e-06 2024-09-16T20:50:31.719 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.719 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.719 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:31.719 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041156+0000", 2024-09-16T20:50:31.719 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0000000000000001e-06 2024-09-16T20:50:31.719 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.719 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.719 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:31.719 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041159+0000", 2024-09-16T20:50:31.720 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5859999999999999e-06 2024-09-16T20:50:31.720 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.720 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.720 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:31.720 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093529+0000", 2024-09-16T20:50:31.720 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052370591000000001 2024-09-16T20:50:31.720 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.720 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.720 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:31.720 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093570+0000", 2024-09-16T20:50:31.720 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0161999999999999e-05 2024-09-16T20:50:31.720 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.720 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.720 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:31.720 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093675+0000", 2024-09-16T20:50:31.721 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010586699999999999 2024-09-16T20:50:31.721 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:31.721 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:31.721 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:31.721 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.721 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.721 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3287 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 250880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:31.721 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:50:31.721 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 41.599844107999999, 2024-09-16T20:50:31.721 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052781203999999998, 2024-09-16T20:50:31.721 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:31.721 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:31.721 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:31.722 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:31.722 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:31.722 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:31.722 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3287 2024-09-16T20:50:31.722 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.722 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:31.722 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.722 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:31.722 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:50:31.722 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:31.722 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.722 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.722 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:31.722 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:50:31.722 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:31.722 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.723 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.723 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:31.723 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:50:31.723 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1e-07 2024-09-16T20:50:31.723 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.723 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.723 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:31.723 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041163+0000", 2024-09-16T20:50:31.723 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0690000000000001e-06 2024-09-16T20:50:31.723 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.723 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.723 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:31.723 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041166+0000", 2024-09-16T20:50:31.723 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.982e-06 2024-09-16T20:50:31.723 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.724 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.724 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:31.724 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041169+0000", 2024-09-16T20:50:31.724 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.593e-06 2024-09-16T20:50:31.724 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.724 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.724 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:31.724 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093709+0000", 2024-09-16T20:50:31.724 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052540492000000001 2024-09-16T20:50:31.724 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.724 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.724 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:31.724 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093792+0000", 2024-09-16T20:50:31.724 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2436999999999997e-05 2024-09-16T20:50:31.724 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.725 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.725 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:31.725 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093943+0000", 2024-09-16T20:50:31.725 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000151421 2024-09-16T20:50:31.725 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:31.725 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:31.725 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:31.725 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.725 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.725 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3288 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 256000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:31.725 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:50:31.725 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 41.599834235000003, 2024-09-16T20:50:31.725 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052928269, 2024-09-16T20:50:31.725 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:31.726 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:31.726 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:31.726 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:31.726 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:31.726 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:31.726 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3288 2024-09-16T20:50:31.726 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.726 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:31.726 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.726 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:31.726 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:50:31.726 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:31.726 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.726 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.726 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:31.726 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:50:31.727 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:31.727 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.727 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.727 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:31.727 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:50:31.727 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2700000000000001e-07 2024-09-16T20:50:31.727 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.727 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.727 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:31.727 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041173+0000", 2024-09-16T20:50:31.727 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.113e-06 2024-09-16T20:50:31.727 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.727 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.727 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:31.728 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041176+0000", 2024-09-16T20:50:31.728 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9799999999999998e-06 2024-09-16T20:50:31.728 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.728 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.728 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:31.728 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041179+0000", 2024-09-16T20:50:31.728 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.667e-06 2024-09-16T20:50:31.728 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.728 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.728 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:31.728 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093957+0000", 2024-09-16T20:50:31.728 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052778703000000003 2024-09-16T20:50:31.728 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.728 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.728 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:31.729 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093999+0000", 2024-09-16T20:50:31.729 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1300000000000001e-05 2024-09-16T20:50:31.729 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.729 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.729 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:31.729 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094100+0000", 2024-09-16T20:50:31.729 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101279 2024-09-16T20:50:31.729 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:31.729 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:31.729 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:31.729 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.729 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.729 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3289 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 261120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:31.730 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:50:31.730 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 41.599824239, 2024-09-16T20:50:31.730 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053182976, 2024-09-16T20:50:31.730 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:31.730 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:31.730 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:31.730 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:31.730 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:31.730 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:31.730 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3289 2024-09-16T20:50:31.730 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.730 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:31.730 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.730 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:31.731 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:50:31.731 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:31.731 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.731 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.731 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:31.731 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:50:31.731 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:31.731 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.731 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.731 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:31.731 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:50:31.731 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2999999999999999e-07 2024-09-16T20:50:31.731 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.731 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.731 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:31.732 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041183+0000", 2024-09-16T20:50:31.732 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0720000000000001e-06 2024-09-16T20:50:31.732 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.732 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.732 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:31.732 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041186+0000", 2024-09-16T20:50:31.732 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9869999999999999e-06 2024-09-16T20:50:31.732 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.732 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.732 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:31.732 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041189+0000", 2024-09-16T20:50:31.732 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.587e-06 2024-09-16T20:50:31.732 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.732 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.733 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:31.733 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094136+0000", 2024-09-16T20:50:31.733 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052947270999999997 2024-09-16T20:50:31.733 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.733 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.733 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:31.733 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094213+0000", 2024-09-16T20:50:31.733 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7552000000000006e-05 2024-09-16T20:50:31.733 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.733 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.733 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:31.733 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094365+0000", 2024-09-16T20:50:31.733 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00015127700000000001 2024-09-16T20:50:31.734 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:31.734 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:31.734 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:31.734 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.734 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.734 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:2196 3.d 3:b1fe01a0:::rbd_data.110e7ef3004.0000000000000000:2d [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e136)", 2024-09-16T20:50:31.734 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:22.739057+0000", 2024-09-16T20:50:31.734 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 8.9019494399999992, 2024-09-16T20:50:31.734 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11999464899999999, 2024-09-16T20:50:31.734 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:31.734 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:31.734 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:31.734 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:31.734 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:50:31.734 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:31.735 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2196 2024-09-16T20:50:31.735 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.735 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:31.735 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.735 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:31.735 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.739057+0000", 2024-09-16T20:50:31.735 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:31.735 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.735 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.735 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:31.735 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.739057+0000", 2024-09-16T20:50:31.735 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:31.735 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.735 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.735 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:31.736 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.739057+0000", 2024-09-16T20:50:31.736 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1399999999999997e-07 2024-09-16T20:50:31.736 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.736 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.736 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:31.736 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.739060+0000", 2024-09-16T20:50:31.736 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6019999999999998e-06 2024-09-16T20:50:31.736 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.736 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.736 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:31.736 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.858755+0000", 2024-09-16T20:50:31.736 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.119695151 2024-09-16T20:50:31.736 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.736 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.736 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:31.737 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.858778+0000", 2024-09-16T20:50:31.737 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3184999999999999e-05 2024-09-16T20:50:31.737 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.737 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.737 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:31.737 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.858865+0000", 2024-09-16T20:50:31.737 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.7330000000000006e-05 2024-09-16T20:50:31.737 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.737 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.737 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:31.737 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.858952+0000", 2024-09-16T20:50:31.737 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6803000000000002e-05 2024-09-16T20:50:31.737 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.737 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.738 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:31.738 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.859051+0000", 2024-09-16T20:50:31.738 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8963999999999996e-05 2024-09-16T20:50:31.738 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:31.738 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:31.738 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:31.738 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.738 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.738 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6061.0:5401 3.13 3:c8554348:::rbd_data.110e16eafbd.0000000000000011:head [read 577536~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-16T20:50:31.738 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:27.761536+0000", 2024-09-16T20:50:31.738 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 3.8794696269999998, 2024-09-16T20:50:31.738 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.003742712, 2024-09-16T20:50:31.738 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:31.738 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:31.738 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:50:31.739 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:31.739 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6061", 2024-09-16T20:50:31.739 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3930128440", 2024-09-16T20:50:31.739 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5401 2024-09-16T20:50:31.739 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.739 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:31.739 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.739 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:31.739 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761536+0000", 2024-09-16T20:50:31.739 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:31.739 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.739 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.739 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:31.739 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761536+0000", 2024-09-16T20:50:31.740 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:31.740 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.740 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.740 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:31.740 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761537+0000", 2024-09-16T20:50:31.740 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2900000000000003e-07 2024-09-16T20:50:31.740 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.740 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.740 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:31.740 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761540+0000", 2024-09-16T20:50:31.740 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2320000000000001e-06 2024-09-16T20:50:31.740 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.740 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.740 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:31.740 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761548+0000", 2024-09-16T20:50:31.741 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1289999999999996e-06 2024-09-16T20:50:31.741 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.741 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.741 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:31.741 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761554+0000", 2024-09-16T20:50:31.741 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0820000000000004e-06 2024-09-16T20:50:31.741 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.741 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.741 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:31.741 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761621+0000", 2024-09-16T20:50:31.741 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6266000000000006e-05 2024-09-16T20:50:31.741 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.741 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.741 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:31.742 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761626+0000", 2024-09-16T20:50:31.742 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.614e-06 2024-09-16T20:50:31.742 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.742 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.742 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:31.742 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:28.765163+0000", 2024-09-16T20:50:31.742 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.003536416 2024-09-16T20:50:31.742 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.742 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.742 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:31.742 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:28.765279+0000", 2024-09-16T20:50:31.742 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000116344 2024-09-16T20:50:31.742 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:31.742 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:31.743 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:31.743 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.743 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.743 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6061.0:5425 3.13 3:c8554348:::rbd_data.110e16eafbd.0000000000000011:head [read 602112~28672] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-16T20:50:31.743 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:27.767049+0000", 2024-09-16T20:50:31.743 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 3.8739569679999999, 2024-09-16T20:50:31.743 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.99847897900000004, 2024-09-16T20:50:31.743 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:31.743 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:31.743 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:50:31.743 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:31.743 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6061", 2024-09-16T20:50:31.743 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3930128440", 2024-09-16T20:50:31.743 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5425 2024-09-16T20:50:31.743 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.744 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:31.744 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.744 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:31.744 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767049+0000", 2024-09-16T20:50:31.744 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:31.744 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.744 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.744 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:31.744 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767049+0000", 2024-09-16T20:50:31.744 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:31.744 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.744 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.744 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:31.745 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767050+0000", 2024-09-16T20:50:31.745 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6499999999999999e-07 2024-09-16T20:50:31.745 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.745 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.745 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:31.745 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767052+0000", 2024-09-16T20:50:31.745 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7350000000000001e-06 2024-09-16T20:50:31.745 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.745 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.745 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:31.745 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767060+0000", 2024-09-16T20:50:31.745 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5109999999999997e-06 2024-09-16T20:50:31.745 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.745 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.746 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:31.746 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767066+0000", 2024-09-16T20:50:31.746 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8139999999999997e-06 2024-09-16T20:50:31.746 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.746 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.746 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:31.746 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767128+0000", 2024-09-16T20:50:31.746 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2174999999999993e-05 2024-09-16T20:50:31.746 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.746 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.746 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:31.746 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767133+0000", 2024-09-16T20:50:31.746 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2140000000000004e-06 2024-09-16T20:50:31.746 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.746 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.747 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:31.747 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:28.765441+0000", 2024-09-16T20:50:31.747 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.99830808599999998 2024-09-16T20:50:31.747 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:31.747 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:31.747 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:31.747 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:28.765528+0000", 2024-09-16T20:50:31.747 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6879000000000006e-05 2024-09-16T20:50:31.747 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:31.747 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:31.747 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:31.747 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:31.747 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:31.747 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:31.749 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:50:31.749 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:31.747+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:31.772 INFO:teuthology.orchestra.run.smithi029.stderr:noscrub is set 2024-09-16T20:50:31.849 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:31.849+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:31.888 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:31.888 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:50:31.888 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:50:31.888 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:31.897 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:50:31.950 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:31.950+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:32.006 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:32.006 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:50:32.006 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:50:32.006 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:50:32.006 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:32.015 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:50:32.059 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:32.058+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:32.126 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:32.126 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:50:32.127 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:50:32.127 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:50:32.127 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.127 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3714 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1480704~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:32.127 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:50:32.127 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 129.48593672999999, 2024-09-16T20:50:32.127 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050152552000000003, 2024-09-16T20:50:32.127 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:32.127 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:32.127 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:32.127 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:32.127 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:32.127 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:32.128 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3714 2024-09-16T20:50:32.128 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.128 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:32.128 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.128 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:32.128 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:50:32.128 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:32.128 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.128 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.128 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:32.128 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:50:32.128 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:32.128 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.128 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.129 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:32.129 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:50:32.129 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0499999999999999e-07 2024-09-16T20:50:32.129 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.129 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.129 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:32.129 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639007+0000", 2024-09-16T20:50:32.129 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5340000000000001e-06 2024-09-16T20:50:32.129 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.129 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.129 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:32.130 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639013+0000", 2024-09-16T20:50:32.130 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3260000000000003e-06 2024-09-16T20:50:32.130 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.130 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.130 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.130 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639018+0000", 2024-09-16T20:50:32.130 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2850000000000002e-06 2024-09-16T20:50:32.130 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.130 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.130 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:32.130 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689026+0000", 2024-09-16T20:50:32.130 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.05000798 2024-09-16T20:50:32.130 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.130 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.131 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:32.131 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689077+0000", 2024-09-16T20:50:32.131 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1063999999999998e-05 2024-09-16T20:50:32.131 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.131 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.131 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:32.131 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689158+0000", 2024-09-16T20:50:32.131 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1057999999999995e-05 2024-09-16T20:50:32.131 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:32.132 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:32.132 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:32.132 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.132 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.132 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3716 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1488896~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:32.132 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:50:32.132 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 129.48591891999999, 2024-09-16T20:50:32.132 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050280142, 2024-09-16T20:50:32.132 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:32.132 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:32.132 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:32.132 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:32.132 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:32.132 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:32.133 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3716 2024-09-16T20:50:32.133 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.133 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:32.133 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.133 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:32.133 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:50:32.133 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:32.133 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.133 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.133 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:32.133 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:50:32.133 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:32.133 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.134 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.134 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:32.134 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639024+0000", 2024-09-16T20:50:32.134 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9999999999999997e-07 2024-09-16T20:50:32.134 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.134 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.134 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:32.134 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639026+0000", 2024-09-16T20:50:32.134 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.781e-06 2024-09-16T20:50:32.134 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.134 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.134 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:32.134 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639032+0000", 2024-09-16T20:50:32.134 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7780000000000004e-06 2024-09-16T20:50:32.135 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.135 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.135 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.135 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639036+0000", 2024-09-16T20:50:32.135 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6909999999999996e-06 2024-09-16T20:50:32.135 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.135 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.135 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:32.135 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689191+0000", 2024-09-16T20:50:32.135 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050155284000000001 2024-09-16T20:50:32.135 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.135 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.135 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:32.135 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689242+0000", 2024-09-16T20:50:32.136 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0662999999999999e-05 2024-09-16T20:50:32.136 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.136 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.136 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:32.136 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689304+0000", 2024-09-16T20:50:32.136 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1345000000000003e-05 2024-09-16T20:50:32.136 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:32.136 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:32.136 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:32.136 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.136 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.136 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3719 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1497088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:32.136 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:50:32.136 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 129.48589164200001, 2024-09-16T20:50:32.137 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050399715999999997, 2024-09-16T20:50:32.137 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:32.137 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:32.137 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:32.137 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:32.137 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:32.137 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:32.137 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3719 2024-09-16T20:50:32.137 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.137 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:32.137 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.137 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:32.137 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:50:32.137 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:32.138 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.138 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.138 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:32.138 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:50:32.138 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:32.138 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.138 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.138 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:32.138 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:50:32.138 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9700000000000002e-07 2024-09-16T20:50:32.138 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.138 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.138 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:32.139 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639053+0000", 2024-09-16T20:50:32.139 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2900000000000001e-06 2024-09-16T20:50:32.139 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.139 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.139 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:32.139 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639060+0000", 2024-09-16T20:50:32.139 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7750000000000004e-06 2024-09-16T20:50:32.139 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.139 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.139 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.139 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639066+0000", 2024-09-16T20:50:32.139 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0639999999999999e-06 2024-09-16T20:50:32.139 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.139 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.139 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:32.140 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689337+0000", 2024-09-16T20:50:32.140 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050270784999999998 2024-09-16T20:50:32.140 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.140 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.140 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:32.140 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689392+0000", 2024-09-16T20:50:32.140 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4920999999999998e-05 2024-09-16T20:50:32.140 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.140 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.140 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:32.140 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689450+0000", 2024-09-16T20:50:32.140 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8483999999999998e-05 2024-09-16T20:50:32.140 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:32.140 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:32.140 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:32.141 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.141 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.141 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3721 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1505280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:32.141 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:50:32.141 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 129.48586850999999, 2024-09-16T20:50:32.141 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050513608000000002, 2024-09-16T20:50:32.141 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:32.141 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:32.141 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:32.141 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:32.141 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:32.141 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:32.141 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3721 2024-09-16T20:50:32.141 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.141 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:32.142 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.142 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:32.142 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:50:32.142 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:32.142 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.142 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.142 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:32.142 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:50:32.142 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:32.142 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.142 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.142 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:32.142 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:50:32.142 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7e-07 2024-09-16T20:50:32.142 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.143 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.143 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:32.143 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639076+0000", 2024-09-16T20:50:32.143 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8899999999999999e-06 2024-09-16T20:50:32.143 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.143 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.143 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:32.143 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639083+0000", 2024-09-16T20:50:32.143 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1210000000000001e-06 2024-09-16T20:50:32.143 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.143 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.143 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.143 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639089+0000", 2024-09-16T20:50:32.143 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.643e-06 2024-09-16T20:50:32.143 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.144 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.144 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:32.144 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689474+0000", 2024-09-16T20:50:32.144 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050385679000000003 2024-09-16T20:50:32.144 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.144 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.144 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:32.144 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689533+0000", 2024-09-16T20:50:32.144 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8303000000000001e-05 2024-09-16T20:50:32.144 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.144 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.144 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:32.144 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689587+0000", 2024-09-16T20:50:32.144 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4601999999999997e-05 2024-09-16T20:50:32.145 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:32.145 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:32.145 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:32.145 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.145 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.145 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3723 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1513472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:32.145 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:50:32.145 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 129.48583770600001, 2024-09-16T20:50:32.145 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050699691999999998, 2024-09-16T20:50:32.145 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:32.145 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:32.145 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:32.145 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:32.145 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:32.146 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:32.146 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3723 2024-09-16T20:50:32.146 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.146 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:32.146 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.146 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:32.146 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:50:32.146 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:32.146 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.146 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.146 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:32.146 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:50:32.146 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:32.146 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.147 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.147 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:32.147 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:50:32.147 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3399999999999999e-07 2024-09-16T20:50:32.147 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.147 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.147 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:32.147 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639107+0000", 2024-09-16T20:50:32.147 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3860000000000001e-06 2024-09-16T20:50:32.147 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.147 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.147 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:32.147 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639114+0000", 2024-09-16T20:50:32.147 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9800000000000001e-06 2024-09-16T20:50:32.147 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.148 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.148 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.148 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639129+0000", 2024-09-16T20:50:32.148 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4887e-05 2024-09-16T20:50:32.148 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.148 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.148 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:32.148 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689625+0000", 2024-09-16T20:50:32.148 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050495848000000003 2024-09-16T20:50:32.148 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.148 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.148 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:32.148 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689720+0000", 2024-09-16T20:50:32.148 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4821000000000006e-05 2024-09-16T20:50:32.149 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.149 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.149 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:32.149 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689804+0000", 2024-09-16T20:50:32.149 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4235999999999998e-05 2024-09-16T20:50:32.149 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:32.149 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:32.149 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:32.149 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.149 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.149 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3725 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1521664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:32.149 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:50:32.149 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 129.48580411, 2024-09-16T20:50:32.150 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050797660000000001, 2024-09-16T20:50:32.150 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:32.150 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:32.150 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:32.150 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:32.150 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:32.150 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:32.150 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3725 2024-09-16T20:50:32.150 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.150 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:32.150 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.150 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:32.150 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:50:32.150 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:32.150 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.151 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.151 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:32.151 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:50:32.151 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:32.151 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.151 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.151 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:32.151 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639139+0000", 2024-09-16T20:50:32.151 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8999999999999997e-07 2024-09-16T20:50:32.151 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.151 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.151 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:32.151 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639141+0000", 2024-09-16T20:50:32.152 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.182e-06 2024-09-16T20:50:32.152 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.152 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.152 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:32.152 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639154+0000", 2024-09-16T20:50:32.152 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3626e-05 2024-09-16T20:50:32.152 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.152 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.152 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.152 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639160+0000", 2024-09-16T20:50:32.152 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6810000000000003e-06 2024-09-16T20:50:32.152 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.152 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.152 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:32.153 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689831+0000", 2024-09-16T20:50:32.153 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050670907000000001 2024-09-16T20:50:32.153 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.153 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.153 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:32.153 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689882+0000", 2024-09-16T20:50:32.153 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0445999999999998e-05 2024-09-16T20:50:32.153 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.153 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.153 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:32.153 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689936+0000", 2024-09-16T20:50:32.153 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4327999999999999e-05 2024-09-16T20:50:32.153 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:32.154 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:32.154 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:32.154 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.154 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.154 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3727 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1529856~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:32.154 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:50:32.154 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 129.485768533, 2024-09-16T20:50:32.154 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051016922999999999, 2024-09-16T20:50:32.154 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:32.154 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:32.154 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:32.154 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:32.155 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:32.155 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:32.155 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3727 2024-09-16T20:50:32.155 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.155 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:32.155 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.155 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:32.155 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:50:32.155 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:32.155 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.155 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.155 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:32.155 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:50:32.156 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:32.156 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.156 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.156 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:32.156 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:50:32.156 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9999999999999998e-07 2024-09-16T20:50:32.156 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.156 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.156 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:32.156 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639183+0000", 2024-09-16T20:50:32.156 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2399999999999996e-06 2024-09-16T20:50:32.156 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.156 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.156 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:32.157 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639195+0000", 2024-09-16T20:50:32.157 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1113e-05 2024-09-16T20:50:32.157 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.157 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.157 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.157 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639206+0000", 2024-09-16T20:50:32.157 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1503e-05 2024-09-16T20:50:32.157 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.157 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.157 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:32.157 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689969+0000", 2024-09-16T20:50:32.157 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050762676 2024-09-16T20:50:32.158 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.158 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.158 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:32.158 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690071+0000", 2024-09-16T20:50:32.158 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102163 2024-09-16T20:50:32.158 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.158 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.158 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:32.158 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690191+0000", 2024-09-16T20:50:32.158 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000119728 2024-09-16T20:50:32.158 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:32.158 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:32.158 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:32.158 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.159 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.159 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3730 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1538048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:32.159 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:50:32.159 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 129.48572697, 2024-09-16T20:50:32.159 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051131467999999999, 2024-09-16T20:50:32.159 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:32.159 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:32.159 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:32.159 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:32.159 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:32.159 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:32.159 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3730 2024-09-16T20:50:32.159 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.160 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:32.160 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.160 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:32.160 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:50:32.160 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:32.160 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.160 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.160 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:32.160 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:50:32.160 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:32.160 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.160 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.160 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:32.161 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639216+0000", 2024-09-16T20:50:32.161 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.39e-07 2024-09-16T20:50:32.161 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.161 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.161 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:32.161 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639218+0000", 2024-09-16T20:50:32.161 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.537e-06 2024-09-16T20:50:32.161 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.161 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.161 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:32.161 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639229+0000", 2024-09-16T20:50:32.162 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1122e-05 2024-09-16T20:50:32.162 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.162 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.162 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.162 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639237+0000", 2024-09-16T20:50:32.162 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8509999999999999e-06 2024-09-16T20:50:32.162 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.162 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.162 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:32.162 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690227+0000", 2024-09-16T20:50:32.162 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050990147 2024-09-16T20:50:32.162 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.162 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.163 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:32.163 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690278+0000", 2024-09-16T20:50:32.163 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0223999999999999e-05 2024-09-16T20:50:32.163 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.163 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.163 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:32.163 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690347+0000", 2024-09-16T20:50:32.163 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9147999999999996e-05 2024-09-16T20:50:32.163 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:32.163 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:32.163 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:32.163 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.163 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.163 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3731 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1546240~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:32.163 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:50:32.164 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 129.485696262, 2024-09-16T20:50:32.164 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051276744999999999, 2024-09-16T20:50:32.164 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:32.164 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:32.164 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:32.164 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:32.164 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:32.164 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:32.164 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3731 2024-09-16T20:50:32.164 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.164 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:32.164 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.164 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:32.164 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:50:32.165 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:32.165 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.165 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.165 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:32.165 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:50:32.165 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:32.165 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.165 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.165 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:32.165 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:50:32.165 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9900000000000001e-07 2024-09-16T20:50:32.165 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.165 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.165 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:32.165 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639249+0000", 2024-09-16T20:50:32.166 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3630000000000001e-06 2024-09-16T20:50:32.166 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.166 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.166 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:32.166 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639255+0000", 2024-09-16T20:50:32.166 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4960000000000001e-06 2024-09-16T20:50:32.166 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.166 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.166 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.166 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639261+0000", 2024-09-16T20:50:32.166 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2839999999999997e-06 2024-09-16T20:50:32.166 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.166 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.166 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:32.166 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690375+0000", 2024-09-16T20:50:32.166 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051114684000000001 2024-09-16T20:50:32.167 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.167 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.167 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:32.167 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690425+0000", 2024-09-16T20:50:32.167 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9379999999999998e-05 2024-09-16T20:50:32.167 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.167 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.167 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:32.167 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690523+0000", 2024-09-16T20:50:32.167 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8139000000000003e-05 2024-09-16T20:50:32.167 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:32.167 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:32.167 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:32.167 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.167 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.167 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3732 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1554432~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:32.168 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:50:32.168 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 129.48565048, 2024-09-16T20:50:32.168 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051380096, 2024-09-16T20:50:32.168 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:32.168 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:32.168 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:32.168 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:32.168 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:32.168 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:32.168 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3732 2024-09-16T20:50:32.168 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.168 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:32.168 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.169 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:32.169 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:50:32.169 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:32.169 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.169 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.169 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:32.169 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:50:32.169 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:32.169 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.169 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.169 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:32.169 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:50:32.169 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6100000000000001e-07 2024-09-16T20:50:32.169 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.170 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.170 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:32.170 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639295+0000", 2024-09-16T20:50:32.170 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.447e-06 2024-09-16T20:50:32.170 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.170 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.170 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:32.170 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639302+0000", 2024-09-16T20:50:32.170 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0570000000000004e-06 2024-09-16T20:50:32.170 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.170 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.170 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.170 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639307+0000", 2024-09-16T20:50:32.170 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1429999999999997e-06 2024-09-16T20:50:32.171 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.171 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.171 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:32.171 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690555+0000", 2024-09-16T20:50:32.171 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051247963000000001 2024-09-16T20:50:32.171 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.171 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.171 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:32.171 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690602+0000", 2024-09-16T20:50:32.171 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7602000000000003e-05 2024-09-16T20:50:32.171 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.171 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.171 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:32.171 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690672+0000", 2024-09-16T20:50:32.171 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9423000000000003e-05 2024-09-16T20:50:32.172 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:32.172 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:32.172 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:32.172 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.172 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.172 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3734 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1562624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:32.172 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:50:32.172 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 129.48562883, 2024-09-16T20:50:32.172 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051490610999999999, 2024-09-16T20:50:32.172 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:32.172 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:32.172 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:32.172 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:32.173 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:32.173 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:32.173 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3734 2024-09-16T20:50:32.173 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.173 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:32.173 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.173 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:32.173 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:50:32.173 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:32.174 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.174 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.174 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:32.174 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:50:32.174 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:32.174 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.174 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.174 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:32.174 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639314+0000", 2024-09-16T20:50:32.174 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4400000000000001e-07 2024-09-16T20:50:32.174 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.174 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.174 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:32.174 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639316+0000", 2024-09-16T20:50:32.174 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1459999999999999e-06 2024-09-16T20:50:32.175 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.175 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.175 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:32.175 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639322+0000", 2024-09-16T20:50:32.175 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7610000000000004e-06 2024-09-16T20:50:32.175 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.175 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.175 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.175 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639327+0000", 2024-09-16T20:50:32.175 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.186e-06 2024-09-16T20:50:32.175 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.175 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.175 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:32.175 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690698+0000", 2024-09-16T20:50:32.175 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051371223000000001 2024-09-16T20:50:32.176 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.176 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.176 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:32.176 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690748+0000", 2024-09-16T20:50:32.176 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0192999999999999e-05 2024-09-16T20:50:32.176 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.176 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.176 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:32.176 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690804+0000", 2024-09-16T20:50:32.176 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5658000000000001e-05 2024-09-16T20:50:32.176 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:32.176 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:32.176 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:32.176 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.176 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.177 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3736 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1570816~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:32.177 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:50:32.177 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 129.485609571, 2024-09-16T20:50:32.177 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051596528000000003, 2024-09-16T20:50:32.177 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:32.177 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:32.177 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:32.177 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:32.177 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:32.177 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:32.177 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3736 2024-09-16T20:50:32.177 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.177 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:32.177 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.178 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:32.178 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:50:32.178 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:32.178 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.178 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.178 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:32.178 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:50:32.178 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:32.178 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.178 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.178 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:32.178 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:50:32.178 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9999999999999999e-07 2024-09-16T20:50:32.178 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.179 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.179 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:32.179 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639335+0000", 2024-09-16T20:50:32.179 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.635e-06 2024-09-16T20:50:32.179 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.179 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.179 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:32.179 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639340+0000", 2024-09-16T20:50:32.179 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5840000000000002e-06 2024-09-16T20:50:32.179 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.179 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.179 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.179 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639346+0000", 2024-09-16T20:50:32.179 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1190000000000002e-06 2024-09-16T20:50:32.179 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.180 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.180 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:32.180 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690817+0000", 2024-09-16T20:50:32.180 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051470721999999997 2024-09-16T20:50:32.180 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.180 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.180 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:32.180 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690875+0000", 2024-09-16T20:50:32.180 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7441000000000001e-05 2024-09-16T20:50:32.180 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.180 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.180 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:32.181 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690929+0000", 2024-09-16T20:50:32.181 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4727e-05 2024-09-16T20:50:32.181 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:32.181 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:32.181 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:32.181 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.181 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.181 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5434 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:32.181 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:50:32.181 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 129.260348005, 2024-09-16T20:50:32.181 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099755831000000003, 2024-09-16T20:50:32.181 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:32.181 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:32.181 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:32.182 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:32.182 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:32.182 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:32.182 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5434 2024-09-16T20:50:32.182 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.182 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:32.182 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.182 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:32.182 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:50:32.182 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:32.182 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.182 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.182 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:32.182 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:50:32.182 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:32.182 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.183 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.183 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:32.183 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864595+0000", 2024-09-16T20:50:32.183 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1099999999999995e-07 2024-09-16T20:50:32.183 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.183 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.183 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:32.183 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864597+0000", 2024-09-16T20:50:32.183 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3989999999999998e-06 2024-09-16T20:50:32.183 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.183 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.183 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:32.183 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864603+0000", 2024-09-16T20:50:32.183 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9309999999999996e-06 2024-09-16T20:50:32.183 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.184 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.184 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.184 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864607+0000", 2024-09-16T20:50:32.184 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6200000000000001e-06 2024-09-16T20:50:32.184 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.184 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.184 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:32.184 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964164+0000", 2024-09-16T20:50:32.184 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099557162000000005 2024-09-16T20:50:32.184 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.184 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.184 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:32.184 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964244+0000", 2024-09-16T20:50:32.185 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9597000000000004e-05 2024-09-16T20:50:32.185 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.185 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.185 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:32.185 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964350+0000", 2024-09-16T20:50:32.185 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010641099999999999 2024-09-16T20:50:32.185 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:32.185 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:32.185 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:32.185 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.185 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.185 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5435 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 20480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:32.185 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:50:32.185 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 129.26031818300001, 2024-09-16T20:50:32.186 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100897102, 2024-09-16T20:50:32.186 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:32.186 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:32.186 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:32.186 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:32.186 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:32.186 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:32.186 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5435 2024-09-16T20:50:32.186 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.186 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:32.186 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.186 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:32.186 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:50:32.186 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:32.186 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.187 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.187 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:32.187 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:50:32.187 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:32.187 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.187 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.187 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:32.187 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:50:32.187 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8000000000000001e-07 2024-09-16T20:50:32.187 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.187 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.187 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:32.187 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864626+0000", 2024-09-16T20:50:32.187 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.745e-06 2024-09-16T20:50:32.187 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.188 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.188 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:32.188 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864633+0000", 2024-09-16T20:50:32.188 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3339999999999998e-06 2024-09-16T20:50:32.188 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.188 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.188 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.188 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864636+0000", 2024-09-16T20:50:32.188 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1200000000000002e-06 2024-09-16T20:50:32.188 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.188 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.188 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:32.188 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965299+0000", 2024-09-16T20:50:32.188 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10066352000000001 2024-09-16T20:50:32.189 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.189 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.189 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:32.189 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965390+0000", 2024-09-16T20:50:32.189 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0894000000000001e-05 2024-09-16T20:50:32.189 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.189 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.189 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:32.189 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965521+0000", 2024-09-16T20:50:32.189 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013110899999999999 2024-09-16T20:50:32.189 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:32.189 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:32.189 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:32.190 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.190 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.190 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:152 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 327680~32768 out=32768b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:50:32.190 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:50:32.190 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 127.961821241, 2024-09-16T20:50:32.190 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11869405800000001, 2024-09-16T20:50:32.190 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:32.190 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:32.190 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:32.190 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:32.190 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:50:32.190 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:50:32.190 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 152 2024-09-16T20:50:32.190 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.190 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:32.191 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.191 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:32.191 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:50:32.191 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:32.191 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.191 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.191 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:32.191 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:50:32.191 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:32.191 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.191 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.191 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:32.191 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163122+0000", 2024-09-16T20:50:32.191 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6400000000000003e-07 2024-09-16T20:50:32.191 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.192 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.192 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:32.192 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163124+0000", 2024-09-16T20:50:32.192 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.565e-06 2024-09-16T20:50:32.192 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.192 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.192 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:32.192 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163136+0000", 2024-09-16T20:50:32.192 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1717e-05 2024-09-16T20:50:32.192 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.192 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.192 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.192 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163142+0000", 2024-09-16T20:50:32.193 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1750000000000002e-06 2024-09-16T20:50:32.193 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.193 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.193 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:32.193 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163606+0000", 2024-09-16T20:50:32.193 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00046418799999999998 2024-09-16T20:50:32.193 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.193 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.193 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:32.193 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163656+0000", 2024-09-16T20:50:32.193 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9852999999999999e-05 2024-09-16T20:50:32.193 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.193 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.194 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:32.194 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281815+0000", 2024-09-16T20:50:32.194 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11815909600000001 2024-09-16T20:50:32.194 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:32.194 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:32.194 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:32.194 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.194 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.194 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:153 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:50:32.194 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:50:32.194 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 127.96178828799999, 2024-09-16T20:50:32.194 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11903462400000001, 2024-09-16T20:50:32.194 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:32.194 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:32.194 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:32.194 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:32.195 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:50:32.195 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:50:32.195 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 153 2024-09-16T20:50:32.195 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.195 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:32.195 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.195 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:32.195 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:50:32.195 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:32.195 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.195 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.195 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:32.195 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:50:32.195 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:32.195 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.196 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.196 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:32.196 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163155+0000", 2024-09-16T20:50:32.196 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.37e-07 2024-09-16T20:50:32.196 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.196 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.196 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:32.196 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163157+0000", 2024-09-16T20:50:32.196 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6010000000000002e-06 2024-09-16T20:50:32.196 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.196 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.196 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:32.196 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163165+0000", 2024-09-16T20:50:32.196 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2619999999999998e-06 2024-09-16T20:50:32.197 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.197 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.197 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.197 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163171+0000", 2024-09-16T20:50:32.197 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.711e-06 2024-09-16T20:50:32.197 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.197 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.197 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:32.197 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281868+0000", 2024-09-16T20:50:32.197 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118696622 2024-09-16T20:50:32.197 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.197 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.197 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:32.198 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281969+0000", 2024-09-16T20:50:32.198 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101371 2024-09-16T20:50:32.198 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.198 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.198 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:32.198 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282189+0000", 2024-09-16T20:50:32.198 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00021952000000000001 2024-09-16T20:50:32.198 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:32.198 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:32.198 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:32.198 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.198 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.199 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:154 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 360448~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:50:32.199 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:50:32.199 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 127.96175651599999, 2024-09-16T20:50:32.199 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11932035100000001, 2024-09-16T20:50:32.199 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:32.199 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:32.199 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:32.199 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:32.199 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:50:32.199 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:50:32.199 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 154 2024-09-16T20:50:32.199 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.199 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:32.199 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.200 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:32.200 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:50:32.200 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:32.200 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.200 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.200 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:32.200 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:50:32.200 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:32.200 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.200 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.200 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:32.200 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:50:32.200 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:50:32.201 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.201 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.201 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:32.201 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163189+0000", 2024-09-16T20:50:32.201 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6840000000000001e-06 2024-09-16T20:50:32.201 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.201 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.201 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:32.201 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163196+0000", 2024-09-16T20:50:32.201 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7920000000000004e-06 2024-09-16T20:50:32.201 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.201 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.201 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.202 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163201+0000", 2024-09-16T20:50:32.202 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2809999999999999e-06 2024-09-16T20:50:32.202 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.202 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.202 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:32.202 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282219+0000", 2024-09-16T20:50:32.202 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.119018343 2024-09-16T20:50:32.202 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.202 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.202 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:32.202 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282305+0000", 2024-09-16T20:50:32.202 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5865000000000005e-05 2024-09-16T20:50:32.202 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.203 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.203 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:32.203 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282506+0000", 2024-09-16T20:50:32.203 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000200927 2024-09-16T20:50:32.203 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:32.203 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:32.203 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:32.203 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.203 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.203 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop(client.4366.0:3088 3.1e e38/23 3:79e5ec01:::rbd_header.110e891a9b2b:head v 38'306, mlcod=36'304)", 2024-09-16T20:50:32.203 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:50:32.203 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 127.96073764800001, 2024-09-16T20:50:32.203 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118102759, 2024-09-16T20:50:32.203 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:32.204 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:32.204 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:32.204 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:32.204 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.204 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:32.204 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:50:32.204 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:32.204 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.204 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.204 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:32.204 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:50:32.204 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:32.204 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.204 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.205 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:32.205 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:50:32.205 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.85e-07 2024-09-16T20:50:32.205 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.205 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.205 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:32.205 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164212+0000", 2024-09-16T20:50:32.205 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3999999999999997e-06 2024-09-16T20:50:32.205 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.205 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.205 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:32.205 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281553+0000", 2024-09-16T20:50:32.205 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11734135499999999 2024-09-16T20:50:32.205 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.206 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.206 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.206 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281567+0000", 2024-09-16T20:50:32.206 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4188999999999999e-05 2024-09-16T20:50:32.206 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.206 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.206 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:32.206 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281653+0000", 2024-09-16T20:50:32.206 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5385999999999999e-05 2024-09-16T20:50:32.206 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.206 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.206 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:32.206 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281679+0000", 2024-09-16T20:50:32.206 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6689000000000001e-05 2024-09-16T20:50:32.207 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.207 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.207 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:32.207 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282277+0000", 2024-09-16T20:50:32.207 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00059798000000000002 2024-09-16T20:50:32.207 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.207 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.207 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:32.207 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282307+0000", 2024-09-16T20:50:32.207 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9975e-05 2024-09-16T20:50:32.207 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:32.207 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:32.207 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:32.207 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.207 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.208 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5255.0:44 3.16 3:6f5476df:::rbd_header.110e25db1f60:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:50:32.208 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:50:32.208 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 55.804434593000003, 2024-09-16T20:50:32.208 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.121007509, 2024-09-16T20:50:32.208 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:32.208 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:32.208 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:32.208 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:32.208 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5255", 2024-09-16T20:50:32.208 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:50:32.208 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 44 2024-09-16T20:50:32.208 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.209 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:32.209 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.209 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:32.209 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:50:32.209 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:32.209 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.209 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.209 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:32.209 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:50:32.209 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:32.209 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.209 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.209 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:32.209 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:50:32.210 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0500000000000003e-07 2024-09-16T20:50:32.210 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.210 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.210 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:32.210 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320512+0000", 2024-09-16T20:50:32.210 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2540000000000001e-06 2024-09-16T20:50:32.210 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.210 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.210 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:32.210 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320518+0000", 2024-09-16T20:50:32.210 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1689999999999997e-06 2024-09-16T20:50:32.210 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.210 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.210 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.211 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320521+0000", 2024-09-16T20:50:32.211 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5200000000000002e-06 2024-09-16T20:50:32.211 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.211 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.211 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:32.211 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320548+0000", 2024-09-16T20:50:32.211 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6662999999999999e-05 2024-09-16T20:50:32.211 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.211 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.211 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:32.211 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320587+0000", 2024-09-16T20:50:32.211 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8866999999999999e-05 2024-09-16T20:50:32.211 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.211 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.212 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:32.212 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.441515+0000", 2024-09-16T20:50:32.212 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.120928331 2024-09-16T20:50:32.212 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:32.212 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:32.212 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:32.212 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.212 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.212 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:20329 3.5 3:a4ea64b7:::rbd_header.110ede5ef857:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:32.212 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:01.061694+0000", 2024-09-16T20:50:32.212 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 31.063248025, 2024-09-16T20:50:32.212 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.071234077000000007, 2024-09-16T20:50:32.212 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:32.212 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:32.212 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:32.213 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:32.213 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:50:32.213 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:32.213 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 20329 2024-09-16T20:50:32.213 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.213 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:32.213 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.213 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:32.213 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.061694+0000", 2024-09-16T20:50:32.213 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:32.213 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.213 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.213 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:32.213 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.061694+0000", 2024-09-16T20:50:32.213 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:32.214 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.214 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.214 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:32.214 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089232+0000", 2024-09-16T20:50:32.214 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.027537275 2024-09-16T20:50:32.214 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.214 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.214 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:32.214 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089237+0000", 2024-09-16T20:50:32.214 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4709999999999998e-06 2024-09-16T20:50:32.214 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.214 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.214 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:32.214 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089246+0000", 2024-09-16T20:50:32.214 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8230000000000001e-06 2024-09-16T20:50:32.215 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.215 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.215 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.215 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089252+0000", 2024-09-16T20:50:32.215 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3060000000000003e-06 2024-09-16T20:50:32.215 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.215 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.215 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:32.215 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.132756+0000", 2024-09-16T20:50:32.215 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.043504334999999998 2024-09-16T20:50:32.215 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.215 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.215 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:32.215 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.132829+0000", 2024-09-16T20:50:32.216 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2849999999999995e-05 2024-09-16T20:50:32.216 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:32.216 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:32.216 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:32.216 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.132928+0000", 2024-09-16T20:50:32.216 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.9017000000000004e-05 2024-09-16T20:50:32.216 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:32.216 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:32.216 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:32.216 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:32.216 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:32.216 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:32.218 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:32.217+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:32.218 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:50:32.319 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:32.318+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:32.332 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:32.333 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:50:32.333 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:50:32.333 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:32.342 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:50:32.421 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:32.419+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:32.502 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:32.502 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:50:32.502 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:50:32.502 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:50:32.502 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:32.511 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:50:32.555 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:32.555+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:32.625 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:32.621+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:32.626 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:32.627 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:50:32.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:50:32.627 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:50:32.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.627 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:3088 3.1e 3:79e5ec01:::rbd_header.110e891a9b2b:head [watch watch cookie 140213437171488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:50:32.627 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:50:32.627 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 128.460897969, 2024-09-16T20:50:32.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.11895186200000001, 2024-09-16T20:50:32.627 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:32.627 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:32.627 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:32.627 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:32.627 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:50:32.627 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:32.628 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3088 2024-09-16T20:50:32.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.628 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:32.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:32.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:50:32.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:32.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:50:32.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.629 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.629 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:32.629 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163846+0000", 2024-09-16T20:50:32.629 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6799999999999999e-07 2024-09-16T20:50:32.629 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.629 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.629 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:32.629 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163850+0000", 2024-09-16T20:50:32.629 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.929e-06 2024-09-16T20:50:32.629 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.629 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.629 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:32.629 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163862+0000", 2024-09-16T20:50:32.629 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1972e-05 2024-09-16T20:50:32.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163867+0000", 2024-09-16T20:50:32.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.451e-06 2024-09-16T20:50:32.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:32.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163912+0000", 2024-09-16T20:50:32.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4456000000000003e-05 2024-09-16T20:50:32.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:32.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163962+0000", 2024-09-16T20:50:32.631 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0578000000000003e-05 2024-09-16T20:50:32.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.631 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.631 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-16T20:50:32.631 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164058+0000", 2024-09-16T20:50:32.631 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5934999999999995e-05 2024-09-16T20:50:32.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.631 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.631 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:50:32.631 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164570+0000", 2024-09-16T20:50:32.631 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00051241100000000005 2024-09-16T20:50:32.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.631 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.631 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:32.631 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282629+0000", 2024-09-16T20:50:32.632 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.118058986 2024-09-16T20:50:32.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.632 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.632 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:50:32.632 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282725+0000", 2024-09-16T20:50:32.632 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5063999999999996e-05 2024-09-16T20:50:32.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.632 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.632 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:32.632 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282797+0000", 2024-09-16T20:50:32.632 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2212000000000006e-05 2024-09-16T20:50:32.632 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.632 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:32.632 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.633 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.633 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:8039 3.1e 3.ac44ff9e (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:50:32.633 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:50:32.633 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 102.91707062, 2024-09-16T20:50:32.633 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0053310449999999, 2024-09-16T20:50:32.633 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:32.633 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:32.633 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:32.633 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:32.633 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:50:32.633 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:32.633 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8039 2024-09-16T20:50:32.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.634 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:32.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.634 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:32.634 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:50:32.634 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.634 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:32.634 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:50:32.634 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.634 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:32.634 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707674+0000", 2024-09-16T20:50:32.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6530000000000001e-06 2024-09-16T20:50:32.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.635 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:32.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707681+0000", 2024-09-16T20:50:32.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9639999999999997e-06 2024-09-16T20:50:32.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.635 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:32.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707700+0000", 2024-09-16T20:50:32.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8674000000000001e-05 2024-09-16T20:50:32.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.636 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.636 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707708+0000", 2024-09-16T20:50:32.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6810000000000004e-06 2024-09-16T20:50:32.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.636 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:32.636 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711293+0000", 2024-09-16T20:50:32.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035848189999999999 2024-09-16T20:50:32.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.636 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:32.637 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711301+0000", 2024-09-16T20:50:32.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4730000000000003e-06 2024-09-16T20:50:32.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.637 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:32.637 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.712990+0000", 2024-09-16T20:50:32.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.001689348 2024-09-16T20:50:32.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.637 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:32.637 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713003+0000", 2024-09-16T20:50:32.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3433e-05 2024-09-16T20:50:32.637 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.637 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:32.637 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.638 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.638 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:8040 3.1e 3.ac44ff9e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:50:32.638 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:50:32.638 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 102.917010225, 2024-09-16T20:50:32.638 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0053410780000001, 2024-09-16T20:50:32.638 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:32.638 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:32.638 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:32.638 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:32.638 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:50:32.638 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:32.638 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8040 2024-09-16T20:50:32.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.639 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:32.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.639 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:32.639 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:50:32.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.639 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:32.639 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:50:32.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.639 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:32.640 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:50:32.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8100000000000003e-07 2024-09-16T20:50:32.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.640 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:32.640 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707737+0000", 2024-09-16T20:50:32.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5020000000000001e-06 2024-09-16T20:50:32.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.640 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:32.640 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707745+0000", 2024-09-16T20:50:32.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2169999999999994e-06 2024-09-16T20:50:32.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707751+0000", 2024-09-16T20:50:32.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.263e-06 2024-09-16T20:50:32.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:32.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711336+0000", 2024-09-16T20:50:32.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035848360000000001 2024-09-16T20:50:32.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:32.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711340+0000", 2024-09-16T20:50:32.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1309999999999999e-06 2024-09-16T20:50:32.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.642 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:32.642 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713060+0000", 2024-09-16T20:50:32.642 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0017195059999999 2024-09-16T20:50:32.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.642 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:32.642 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713074+0000", 2024-09-16T20:50:32.642 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4042e-05 2024-09-16T20:50:32.642 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.642 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:32.642 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.642 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4927.0:4487 3.1e 3.ac44ff9e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:50:32.642 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:50:32.642 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 102.91649002299999, 2024-09-16T20:50:32.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0048840880000001, 2024-09-16T20:50:32.643 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:32.643 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:32.643 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:32.643 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:32.643 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4927", 2024-09-16T20:50:32.643 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:32.643 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4487 2024-09-16T20:50:32.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.643 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:32.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.643 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:32.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:50:32.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.644 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:32.644 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:50:32.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.644 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:32.644 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708254+0000", 2024-09-16T20:50:32.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1800000000000005e-07 2024-09-16T20:50:32.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.644 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:32.644 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708258+0000", 2024-09-16T20:50:32.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8689999999999997e-06 2024-09-16T20:50:32.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:32.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708267+0000", 2024-09-16T20:50:32.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9590000000000007e-06 2024-09-16T20:50:32.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708274+0000", 2024-09-16T20:50:32.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9970000000000001e-06 2024-09-16T20:50:32.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:32.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711366+0000", 2024-09-16T20:50:32.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030919530000000001 2024-09-16T20:50:32.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.646 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:32.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711372+0000", 2024-09-16T20:50:32.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9410000000000004e-06 2024-09-16T20:50:32.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.646 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:32.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713126+0000", 2024-09-16T20:50:32.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0017537249999999 2024-09-16T20:50:32.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.646 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:32.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713137+0000", 2024-09-16T20:50:32.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0825999999999999e-05 2024-09-16T20:50:32.646 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.646 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:32.647 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.647 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5089.0:11703 3.1b 3:dcd4684e:::rbd_data.110e747f0779.000000000000001b:head [set-alloc-hint object_size 262144 write_size 262144,write 139264~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:50:32.647 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:50:32.647 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 88.486754249000001, 2024-09-16T20:50:32.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.084231385000000006, 2024-09-16T20:50:32.647 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:32.647 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:32.647 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:32.647 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:32.647 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5089", 2024-09-16T20:50:32.647 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:50:32.647 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 11703 2024-09-16T20:50:32.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.647 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:32.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:32.648 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:50:32.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:32.648 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:50:32.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:32.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:50:32.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2600000000000001e-07 2024-09-16T20:50:32.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:32.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137998+0000", 2024-09-16T20:50:32.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9549999999999998e-06 2024-09-16T20:50:32.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:32.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138008+0000", 2024-09-16T20:50:32.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3789999999999995e-06 2024-09-16T20:50:32.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.650 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.650 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138013+0000", 2024-09-16T20:50:32.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2209999999999996e-06 2024-09-16T20:50:32.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.650 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:32.650 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138054+0000", 2024-09-16T20:50:32.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1022999999999997e-05 2024-09-16T20:50:32.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.650 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:32.650 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138124+0000", 2024-09-16T20:50:32.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0234999999999994e-05 2024-09-16T20:50:32.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.650 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:32.651 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138197+0000", 2024-09-16T20:50:32.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2451000000000001e-05 2024-09-16T20:50:32.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.651 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:32.651 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222010+0000", 2024-09-16T20:50:32.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.083813433000000007 2024-09-16T20:50:32.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.651 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:50:32.651 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222163+0000", 2024-09-16T20:50:32.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015274599999999999 2024-09-16T20:50:32.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.651 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:50:32.652 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222197+0000", 2024-09-16T20:50:32.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3683000000000003e-05 2024-09-16T20:50:32.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.652 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:32.652 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222220+0000", 2024-09-16T20:50:32.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3433000000000001e-05 2024-09-16T20:50:32.652 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.652 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:32.652 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.652 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.5089.0:11702 3.15 e73/64 3:acf33c8b:::rbd_data.110e747f0779.000000000000001a:head v 73'132, mlcod=73'130)", 2024-09-16T20:50:32.653 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:50:32.653 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 88.486486920000004, 2024-09-16T20:50:32.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.084079763000000002, 2024-09-16T20:50:32.653 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:32.653 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:32.653 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:32.653 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:32.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:32.653 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:50:32.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:32.653 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:50:32.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:32.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:50:32.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:50:32.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:32.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138263+0000", 2024-09-16T20:50:32.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1e-06 2024-09-16T20:50:32.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.655 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:32.655 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221589+0000", 2024-09-16T20:50:32.655 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.083326002999999996 2024-09-16T20:50:32.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.655 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.655 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.655 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221614+0000", 2024-09-16T20:50:32.655 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5593999999999999e-05 2024-09-16T20:50:32.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.655 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.655 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:32.655 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221714+0000", 2024-09-16T20:50:32.655 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9889000000000005e-05 2024-09-16T20:50:32.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.656 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:32.656 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221727+0000", 2024-09-16T20:50:32.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3125999999999999e-05 2024-09-16T20:50:32.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.656 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:50:32.656 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222294+0000", 2024-09-16T20:50:32.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00056712199999999998 2024-09-16T20:50:32.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.656 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:32.656 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222336+0000", 2024-09-16T20:50:32.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1470000000000001e-05 2024-09-16T20:50:32.656 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.657 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:32.657 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.657 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5138.0:9358 3.c 3:33996a43:::rbd_data.110e6372565d.0000000000000007:head [read 1343488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T20:50:32.657 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:50:32.657 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 64.772211510000005, 2024-09-16T20:50:32.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0167493569999999, 2024-09-16T20:50:32.657 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:32.657 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:32.657 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:32.657 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:32.657 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5138", 2024-09-16T20:50:32.658 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:50:32.658 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9358 2024-09-16T20:50:32.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.658 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:32.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.658 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:32.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:50:32.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.658 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:32.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:50:32.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.659 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.659 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:32.659 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852533+0000", 2024-09-16T20:50:32.659 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.646e-06 2024-09-16T20:50:32.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.659 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.659 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:32.659 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852539+0000", 2024-09-16T20:50:32.659 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3389999999999998e-06 2024-09-16T20:50:32.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.659 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.659 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:32.659 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852552+0000", 2024-09-16T20:50:32.660 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.271e-05 2024-09-16T20:50:32.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.660 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.660 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852563+0000", 2024-09-16T20:50:32.660 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1139e-05 2024-09-16T20:50:32.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.660 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:32.660 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852883+0000", 2024-09-16T20:50:32.660 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00032010600000000001 2024-09-16T20:50:32.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.660 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:32.661 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852890+0000", 2024-09-16T20:50:32.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7669999999999998e-06 2024-09-16T20:50:32.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.661 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:32.661 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869059+0000", 2024-09-16T20:50:32.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.016169353 2024-09-16T20:50:32.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.661 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:32.661 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869178+0000", 2024-09-16T20:50:32.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118193 2024-09-16T20:50:32.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.662 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:32.662 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869281+0000", 2024-09-16T20:50:32.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010310399999999999 2024-09-16T20:50:32.662 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.662 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:32.662 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.662 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.662 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5138.0:9382 3.c 3:33996a43:::rbd_data.110e6372565d.0000000000000007:head [read 0~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T20:50:32.662 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:50:32.662 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 64.772160632999999, 2024-09-16T20:50:32.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.017136034, 2024-09-16T20:50:32.662 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:32.663 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:32.663 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:32.663 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:32.663 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5138", 2024-09-16T20:50:32.663 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:50:32.663 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9382 2024-09-16T20:50:32.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.663 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:32.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.663 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:32.663 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:50:32.663 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.664 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.664 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:32.664 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:50:32.664 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.664 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.664 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:32.664 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852583+0000", 2024-09-16T20:50:32.664 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.51e-07 2024-09-16T20:50:32.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.664 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.664 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:32.664 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852586+0000", 2024-09-16T20:50:32.664 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0280000000000001e-06 2024-09-16T20:50:32.665 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.665 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.665 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:32.665 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852595+0000", 2024-09-16T20:50:32.665 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2799999999999992e-06 2024-09-16T20:50:32.665 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.665 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.665 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.665 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852601+0000", 2024-09-16T20:50:32.665 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3810000000000004e-06 2024-09-16T20:50:32.665 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.665 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.665 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:32.665 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852921+0000", 2024-09-16T20:50:32.665 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00031979699999999997 2024-09-16T20:50:32.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.666 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:32.666 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852930+0000", 2024-09-16T20:50:32.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6489999999999997e-06 2024-09-16T20:50:32.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.666 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:32.666 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869434+0000", 2024-09-16T20:50:32.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0165038829999999 2024-09-16T20:50:32.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.666 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:32.666 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869495+0000", 2024-09-16T20:50:32.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1509999999999999e-05 2024-09-16T20:50:32.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.667 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:32.667 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869718+0000", 2024-09-16T20:50:32.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000222955 2024-09-16T20:50:32.667 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.667 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:32.667 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.667 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9969 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 632832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:32.667 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642045+0000", 2024-09-16T20:50:32.667 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 41.982698176, 2024-09-16T20:50:32.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076903192999999995, 2024-09-16T20:50:32.667 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:32.668 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:32.668 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:32.668 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:32.668 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:32.668 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:32.668 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9969 2024-09-16T20:50:32.668 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.668 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:32.668 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.668 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:32.668 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642045+0000", 2024-09-16T20:50:32.668 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.669 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.669 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:32.669 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642045+0000", 2024-09-16T20:50:32.669 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.669 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.669 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:32.669 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642045+0000", 2024-09-16T20:50:32.669 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5100000000000001e-07 2024-09-16T20:50:32.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.669 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.669 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:32.670 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642047+0000", 2024-09-16T20:50:32.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8679999999999999e-06 2024-09-16T20:50:32.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.670 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:32.670 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642053+0000", 2024-09-16T20:50:32.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1720000000000004e-06 2024-09-16T20:50:32.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.670 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.670 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642060+0000", 2024-09-16T20:50:32.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0339999999999999e-06 2024-09-16T20:50:32.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.671 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:32.671 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.718562+0000", 2024-09-16T20:50:32.671 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076501969000000003 2024-09-16T20:50:32.671 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.671 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.671 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:32.671 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.718643+0000", 2024-09-16T20:50:32.671 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0539999999999998e-05 2024-09-16T20:50:32.671 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.671 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.671 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:32.671 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.718948+0000", 2024-09-16T20:50:32.671 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000305259 2024-09-16T20:50:32.671 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.672 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:32.672 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.672 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.672 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.672 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9970 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 637952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:32.672 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642067+0000", 2024-09-16T20:50:32.672 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 41.982676196, 2024-09-16T20:50:32.672 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077174928000000004, 2024-09-16T20:50:32.672 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:32.672 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:32.672 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:32.672 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:32.672 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:32.672 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:32.673 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9970 2024-09-16T20:50:32.673 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.673 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:32.673 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.673 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:32.673 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642067+0000", 2024-09-16T20:50:32.673 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.673 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.673 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.673 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:32.673 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642067+0000", 2024-09-16T20:50:32.673 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.673 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.674 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.674 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:32.674 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642067+0000", 2024-09-16T20:50:32.674 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2499999999999995e-07 2024-09-16T20:50:32.674 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.674 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.674 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:32.674 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642070+0000", 2024-09-16T20:50:32.674 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3199999999999998e-06 2024-09-16T20:50:32.674 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.674 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.674 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:32.674 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642076+0000", 2024-09-16T20:50:32.675 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.122e-06 2024-09-16T20:50:32.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.675 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.675 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.675 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642081+0000", 2024-09-16T20:50:32.675 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0869999999999997e-06 2024-09-16T20:50:32.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.675 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.675 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:32.675 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.718980+0000", 2024-09-16T20:50:32.675 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076899482000000005 2024-09-16T20:50:32.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.675 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.675 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:32.675 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719085+0000", 2024-09-16T20:50:32.676 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010446399999999999 2024-09-16T20:50:32.676 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.676 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.676 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:32.676 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719242+0000", 2024-09-16T20:50:32.676 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000156928 2024-09-16T20:50:32.676 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.676 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:32.676 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.676 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.676 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.676 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9971 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 643072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:32.677 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642102+0000", 2024-09-16T20:50:32.677 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 41.982640488999998, 2024-09-16T20:50:32.677 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077421016999999995, 2024-09-16T20:50:32.677 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:32.677 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:32.677 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:32.677 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:32.677 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:32.677 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:32.677 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9971 2024-09-16T20:50:32.677 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.677 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:32.677 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.677 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:32.678 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642102+0000", 2024-09-16T20:50:32.678 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.678 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.678 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.678 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:32.678 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642102+0000", 2024-09-16T20:50:32.678 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.678 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.678 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.678 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:32.678 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642103+0000", 2024-09-16T20:50:32.678 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7399999999999998e-07 2024-09-16T20:50:32.678 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.678 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.678 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:32.679 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642105+0000", 2024-09-16T20:50:32.679 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2649999999999999e-06 2024-09-16T20:50:32.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.679 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.679 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:32.679 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642112+0000", 2024-09-16T20:50:32.679 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.294e-06 2024-09-16T20:50:32.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.679 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.679 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.679 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642118+0000", 2024-09-16T20:50:32.679 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4500000000000003e-06 2024-09-16T20:50:32.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.679 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.679 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:32.680 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719285+0000", 2024-09-16T20:50:32.680 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077166883000000006 2024-09-16T20:50:32.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.680 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.680 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:32.680 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719368+0000", 2024-09-16T20:50:32.680 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2811000000000004e-05 2024-09-16T20:50:32.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.680 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.680 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:32.680 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719523+0000", 2024-09-16T20:50:32.680 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015584 2024-09-16T20:50:32.680 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.680 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:32.680 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.681 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.681 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.681 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9972 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 648192~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:32.681 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642125+0000", 2024-09-16T20:50:32.681 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 41.982617542, 2024-09-16T20:50:32.681 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077710324999999997, 2024-09-16T20:50:32.681 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:32.681 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:32.681 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:32.681 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:32.681 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:32.681 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:32.681 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9972 2024-09-16T20:50:32.681 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.682 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:32.682 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.682 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:32.682 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642125+0000", 2024-09-16T20:50:32.682 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.682 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.682 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.682 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:32.682 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642125+0000", 2024-09-16T20:50:32.682 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.682 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.683 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.683 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:32.683 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642126+0000", 2024-09-16T20:50:32.683 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1899999999999998e-07 2024-09-16T20:50:32.683 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.683 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.683 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:32.683 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642128+0000", 2024-09-16T20:50:32.683 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8619999999999999e-06 2024-09-16T20:50:32.683 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.683 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.683 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:32.684 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642134+0000", 2024-09-16T20:50:32.684 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5289999999999997e-06 2024-09-16T20:50:32.684 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.684 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.684 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.684 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642139+0000", 2024-09-16T20:50:32.684 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1250000000000001e-06 2024-09-16T20:50:32.684 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.684 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.684 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:32.684 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719570+0000", 2024-09-16T20:50:32.684 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077430516000000005 2024-09-16T20:50:32.685 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.685 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.685 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:32.685 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719655+0000", 2024-09-16T20:50:32.685 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5500000000000005e-05 2024-09-16T20:50:32.685 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.685 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.685 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:32.685 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719836+0000", 2024-09-16T20:50:32.685 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018037399999999999 2024-09-16T20:50:32.685 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.685 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:32.685 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.686 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.686 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.686 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9973 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 653312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:32.686 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:50:32.686 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 41.982596932, 2024-09-16T20:50:32.686 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077905262000000003, 2024-09-16T20:50:32.686 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:32.686 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:32.686 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:32.686 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:32.686 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:32.686 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:32.686 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9973 2024-09-16T20:50:32.686 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.686 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:32.687 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.687 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:32.687 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:50:32.687 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.687 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.687 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.687 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:32.687 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:50:32.687 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.687 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.687 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.687 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:32.687 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:50:32.687 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:50:32.687 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.688 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.688 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:32.688 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642148+0000", 2024-09-16T20:50:32.688 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0219999999999999e-06 2024-09-16T20:50:32.688 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.688 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.688 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:32.688 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642154+0000", 2024-09-16T20:50:32.688 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8540000000000002e-06 2024-09-16T20:50:32.688 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.688 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.688 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.688 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642159+0000", 2024-09-16T20:50:32.689 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.045e-06 2024-09-16T20:50:32.689 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.689 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.689 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:32.689 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719875+0000", 2024-09-16T20:50:32.689 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077716116000000002 2024-09-16T20:50:32.689 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.689 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.689 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:32.689 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719934+0000", 2024-09-16T20:50:32.689 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8819e-05 2024-09-16T20:50:32.689 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.689 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.689 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:32.690 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720051+0000", 2024-09-16T20:50:32.690 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000116947 2024-09-16T20:50:32.690 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.690 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:32.690 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.690 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.690 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.690 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9974 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 658432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:32.690 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642166+0000", 2024-09-16T20:50:32.690 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 41.982577018000001, 2024-09-16T20:50:32.690 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.078179786000000001, 2024-09-16T20:50:32.690 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:32.690 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:32.690 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:32.690 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:32.691 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:32.691 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:32.691 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9974 2024-09-16T20:50:32.691 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.691 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:32.691 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.691 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:32.691 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642166+0000", 2024-09-16T20:50:32.691 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.691 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.691 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.691 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:32.691 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642166+0000", 2024-09-16T20:50:32.691 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.692 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.692 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.692 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:32.692 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647343+0000", 2024-09-16T20:50:32.692 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0051769440000000002 2024-09-16T20:50:32.692 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.692 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.692 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:32.692 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647348+0000", 2024-09-16T20:50:32.692 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3619999999999997e-06 2024-09-16T20:50:32.692 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.692 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.692 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:32.693 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647359+0000", 2024-09-16T20:50:32.693 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0929e-05 2024-09-16T20:50:32.693 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.693 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.693 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.693 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647366+0000", 2024-09-16T20:50:32.693 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0910000000000004e-06 2024-09-16T20:50:32.693 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.693 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.693 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:32.693 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720094+0000", 2024-09-16T20:50:32.693 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.072727403999999995 2024-09-16T20:50:32.693 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.693 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.694 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:32.694 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720173+0000", 2024-09-16T20:50:32.694 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9382000000000007e-05 2024-09-16T20:50:32.694 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.694 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.694 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:32.694 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720346+0000", 2024-09-16T20:50:32.694 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017267399999999999 2024-09-16T20:50:32.694 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.694 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:32.694 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.694 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.694 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.694 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9990 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 740352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:32.695 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647725+0000", 2024-09-16T20:50:32.695 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 41.977017934000003, 2024-09-16T20:50:32.695 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077029919000000002, 2024-09-16T20:50:32.695 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:32.695 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:32.695 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:32.695 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:32.695 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:32.695 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:32.695 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9990 2024-09-16T20:50:32.695 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.695 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:32.695 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.695 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:32.696 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647725+0000", 2024-09-16T20:50:32.696 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.696 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.696 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.696 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:32.696 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647725+0000", 2024-09-16T20:50:32.696 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.696 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.696 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.696 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:32.696 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647725+0000", 2024-09-16T20:50:32.696 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2399999999999999e-07 2024-09-16T20:50:32.696 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.696 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.697 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:32.697 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647726+0000", 2024-09-16T20:50:32.697 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.116e-06 2024-09-16T20:50:32.697 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.697 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.697 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:32.697 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647729+0000", 2024-09-16T20:50:32.697 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.937e-06 2024-09-16T20:50:32.697 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.697 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.697 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.697 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647732+0000", 2024-09-16T20:50:32.697 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8739999999999999e-06 2024-09-16T20:50:32.698 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.698 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.698 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:32.698 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724612+0000", 2024-09-16T20:50:32.698 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076880102000000006 2024-09-16T20:50:32.698 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.698 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.698 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:32.698 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724649+0000", 2024-09-16T20:50:32.698 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6738e-05 2024-09-16T20:50:32.698 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.698 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.698 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:32.699 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724755+0000", 2024-09-16T20:50:32.699 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010592800000000001 2024-09-16T20:50:32.699 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.699 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:32.699 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.699 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.699 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.699 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9991 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 745472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:32.699 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:50:32.699 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 41.977007624999999, 2024-09-16T20:50:32.699 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077257820000000005, 2024-09-16T20:50:32.699 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:32.699 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:32.700 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:32.700 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:32.700 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:32.700 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:32.700 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9991 2024-09-16T20:50:32.700 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.700 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:32.700 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.700 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:32.700 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:50:32.700 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.700 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.700 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.700 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:32.701 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:50:32.701 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.701 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.701 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.701 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:32.701 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:50:32.701 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1199999999999999e-07 2024-09-16T20:50:32.701 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.701 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.701 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:32.701 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647736+0000", 2024-09-16T20:50:32.701 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3699999999999999e-07 2024-09-16T20:50:32.701 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.701 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.701 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:32.702 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647739+0000", 2024-09-16T20:50:32.702 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9359999999999999e-06 2024-09-16T20:50:32.702 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.702 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.702 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.702 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647742+0000", 2024-09-16T20:50:32.702 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5720000000000001e-06 2024-09-16T20:50:32.702 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.702 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.702 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:32.702 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724774+0000", 2024-09-16T20:50:32.702 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077032457999999998 2024-09-16T20:50:32.702 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.703 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.703 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:32.703 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724818+0000", 2024-09-16T20:50:32.703 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3803000000000001e-05 2024-09-16T20:50:32.703 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.703 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.703 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:32.703 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724993+0000", 2024-09-16T20:50:32.703 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000174902 2024-09-16T20:50:32.703 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.703 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:32.703 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.703 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.703 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.704 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9992 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 750592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:32.704 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:50:32.704 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 41.976998035999998, 2024-09-16T20:50:32.704 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077553227000000002, 2024-09-16T20:50:32.704 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:32.704 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:32.704 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:32.704 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:32.704 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:32.704 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:32.704 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9992 2024-09-16T20:50:32.704 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.704 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:32.705 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.705 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:32.705 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:50:32.705 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.705 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.705 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.705 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:32.705 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:50:32.705 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.705 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.705 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.705 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:32.705 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:50:32.706 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0200000000000001e-07 2024-09-16T20:50:32.706 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.706 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.706 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:32.706 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647746+0000", 2024-09-16T20:50:32.706 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.088e-06 2024-09-16T20:50:32.706 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.706 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.706 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:32.706 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647749+0000", 2024-09-16T20:50:32.706 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9170000000000002e-06 2024-09-16T20:50:32.706 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.706 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.706 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.706 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647752+0000", 2024-09-16T20:50:32.707 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.774e-06 2024-09-16T20:50:32.707 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.707 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.707 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:32.707 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725042+0000", 2024-09-16T20:50:32.707 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077289952999999995 2024-09-16T20:50:32.707 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.707 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.707 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:32.707 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725113+0000", 2024-09-16T20:50:32.707 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0920999999999994e-05 2024-09-16T20:50:32.707 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.707 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.707 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:32.708 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725298+0000", 2024-09-16T20:50:32.708 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000185372 2024-09-16T20:50:32.708 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.708 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:32.708 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.708 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.708 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.708 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9993 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 755712~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:32.708 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647755+0000", 2024-09-16T20:50:32.708 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 41.976988009999999, 2024-09-16T20:50:32.708 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077725674999999994, 2024-09-16T20:50:32.708 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:32.708 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:32.708 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:32.709 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:32.709 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:32.709 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:32.709 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9993 2024-09-16T20:50:32.709 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.709 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:32.709 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.709 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:32.709 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647755+0000", 2024-09-16T20:50:32.709 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.709 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.709 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.709 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:32.709 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647755+0000", 2024-09-16T20:50:32.710 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.710 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.710 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.710 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:32.710 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653406+0000", 2024-09-16T20:50:32.710 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.005650876 2024-09-16T20:50:32.710 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.710 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.710 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:32.710 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653411+0000", 2024-09-16T20:50:32.710 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8899999999999998e-06 2024-09-16T20:50:32.710 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.711 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.711 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:32.711 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653423+0000", 2024-09-16T20:50:32.711 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2447e-05 2024-09-16T20:50:32.711 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.711 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.711 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.711 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653431+0000", 2024-09-16T20:50:32.711 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5730000000000002e-06 2024-09-16T20:50:32.711 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.711 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.711 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:32.711 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725333+0000", 2024-09-16T20:50:32.711 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.071902743000000005 2024-09-16T20:50:32.711 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.712 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.712 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:32.712 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725385+0000", 2024-09-16T20:50:32.712 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1585000000000002e-05 2024-09-16T20:50:32.712 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.712 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.712 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:32.712 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725481+0000", 2024-09-16T20:50:32.712 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5561000000000002e-05 2024-09-16T20:50:32.712 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.712 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:32.712 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.712 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.712 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.712 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6061.0:5400 3.b 3:d353d652:::rbd_data.110e16eafbd.0000000000000010:head [read 577536~28672] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-16T20:50:32.713 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:27.761367+0000", 2024-09-16T20:50:32.713 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.8633760639999997, 2024-09-16T20:50:32.713 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0029066630000001, 2024-09-16T20:50:32.713 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:32.713 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:32.713 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:32.713 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:32.713 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6061", 2024-09-16T20:50:32.713 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3930128440", 2024-09-16T20:50:32.713 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5400 2024-09-16T20:50:32.713 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.713 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:32.713 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.713 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:32.714 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761367+0000", 2024-09-16T20:50:32.714 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.714 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.714 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.714 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:32.714 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761367+0000", 2024-09-16T20:50:32.714 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.714 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.714 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.714 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:32.714 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761369+0000", 2024-09-16T20:50:32.714 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.934e-06 2024-09-16T20:50:32.714 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.714 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.714 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:32.715 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761374+0000", 2024-09-16T20:50:32.715 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7139999999999998e-06 2024-09-16T20:50:32.715 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.715 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.715 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:32.715 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761387+0000", 2024-09-16T20:50:32.715 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2349e-05 2024-09-16T20:50:32.715 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.715 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.715 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.715 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761393+0000", 2024-09-16T20:50:32.715 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.652e-06 2024-09-16T20:50:32.715 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.715 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.715 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:32.716 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761442+0000", 2024-09-16T20:50:32.716 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9011000000000002e-05 2024-09-16T20:50:32.716 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.716 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.716 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:32.716 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761451+0000", 2024-09-16T20:50:32.716 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1920000000000005e-06 2024-09-16T20:50:32.716 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.716 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.716 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:32.716 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:28.764169+0000", 2024-09-16T20:50:32.716 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.002718161 2024-09-16T20:50:32.716 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.717 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.717 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:32.717 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:28.764273+0000", 2024-09-16T20:50:32.717 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010465 2024-09-16T20:50:32.717 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.717 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:32.717 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.717 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.717 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.717 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6061.0:5424 3.b 3:d353d652:::rbd_data.110e16eafbd.0000000000000010:head [read 606208~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-16T20:50:32.717 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:27.766882+0000", 2024-09-16T20:50:32.717 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.8578607009999999, 2024-09-16T20:50:32.717 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99753650500000002, 2024-09-16T20:50:32.717 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:32.718 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:32.718 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:32.718 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:32.718 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6061", 2024-09-16T20:50:32.718 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3930128440", 2024-09-16T20:50:32.718 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5424 2024-09-16T20:50:32.718 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.718 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:32.718 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.719 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:32.719 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766882+0000", 2024-09-16T20:50:32.719 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.719 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.719 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.719 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:32.719 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766882+0000", 2024-09-16T20:50:32.719 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.719 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.719 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.719 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:32.719 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766883+0000", 2024-09-16T20:50:32.719 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3289999999999999e-06 2024-09-16T20:50:32.719 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.720 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.720 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:32.720 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766888+0000", 2024-09-16T20:50:32.720 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5800000000000002e-06 2024-09-16T20:50:32.720 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.720 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.720 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:32.720 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766900+0000", 2024-09-16T20:50:32.720 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1861e-05 2024-09-16T20:50:32.720 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.720 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.720 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.720 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766909+0000", 2024-09-16T20:50:32.720 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5210000000000008e-06 2024-09-16T20:50:32.720 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.721 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.721 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:32.721 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766958+0000", 2024-09-16T20:50:32.721 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8718999999999999e-05 2024-09-16T20:50:32.721 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.721 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.721 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:32.721 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766975+0000", 2024-09-16T20:50:32.721 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7231000000000002e-05 2024-09-16T20:50:32.721 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.721 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.721 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:32.721 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:28.764327+0000", 2024-09-16T20:50:32.721 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99735112500000001 2024-09-16T20:50:32.722 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.722 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.722 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:32.722 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:28.764419+0000", 2024-09-16T20:50:32.722 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2139000000000007e-05 2024-09-16T20:50:32.722 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.722 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:32.722 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.722 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.722 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:32.722 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:32.724 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:50:32.724 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:32.722+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:32.750 INFO:teuthology.orchestra.run.smithi139.stdout:5883 write 0x5c10c00 thru 0x5c13834 (0x2c35 bytes) 2024-09-16T20:50:32.751 INFO:teuthology.orchestra.run.smithi139.stdout:5885 read 0x62f4600 thru 0x62f8e24 (0x4825 bytes) 2024-09-16T20:50:32.752 INFO:teuthology.orchestra.run.smithi139.stdout:5886 punch from 0x4694800 to 0x469f200, (0xaa00 bytes) 2024-09-16T20:50:32.824 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:32.823+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:32.852 INFO:teuthology.orchestra.run.smithi139.stdout:5887 flatten 2024-09-16T20:50:32.907 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:32.907 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:50:32.907 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.907 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1569 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:32.907 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.863274+0000", 2024-09-16T20:50:32.907 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.025732530999999999, 2024-09-16T20:50:32.907 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.025755449, 2024-09-16T20:50:32.907 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:32.907 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:32.907 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:32.908 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:32.908 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:32.908 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:32.908 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1569 2024-09-16T20:50:32.908 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.908 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:32.908 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.908 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:32.908 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.863274+0000", 2024-09-16T20:50:32.908 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.908 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.908 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.909 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:32.909 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.863274+0000", 2024-09-16T20:50:32.909 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.909 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.909 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.909 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:32.909 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.863275+0000", 2024-09-16T20:50:32.909 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1199999999999999e-07 2024-09-16T20:50:32.909 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.909 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.909 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:32.909 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.863276+0000", 2024-09-16T20:50:32.910 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.088e-06 2024-09-16T20:50:32.910 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.910 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.910 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:32.910 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.863279+0000", 2024-09-16T20:50:32.910 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0529999999999998e-06 2024-09-16T20:50:32.910 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.910 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.910 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.910 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.863281+0000", 2024-09-16T20:50:32.910 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5110000000000002e-06 2024-09-16T20:50:32.910 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.911 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:32.911 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.911 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.911 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.911 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1601 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:32.911 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.866288+0000", 2024-09-16T20:50:32.911 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.022718957000000001, 2024-09-16T20:50:32.911 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.022801772000000001, 2024-09-16T20:50:32.911 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:32.911 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:32.912 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:32.912 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:32.912 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:32.912 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:32.912 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1601 2024-09-16T20:50:32.912 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.912 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:32.912 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.912 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:32.912 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.866288+0000", 2024-09-16T20:50:32.912 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.912 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.912 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.913 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:32.913 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.866288+0000", 2024-09-16T20:50:32.913 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.913 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.913 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.913 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:32.913 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.866382+0000", 2024-09-16T20:50:32.913 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3482999999999999e-05 2024-09-16T20:50:32.913 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.913 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.913 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:32.913 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.866384+0000", 2024-09-16T20:50:32.913 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5459999999999998e-06 2024-09-16T20:50:32.913 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.914 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.914 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:32.914 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.866390+0000", 2024-09-16T20:50:32.914 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9429999999999997e-06 2024-09-16T20:50:32.914 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.914 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.914 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.914 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.866394+0000", 2024-09-16T20:50:32.914 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5829999999999998e-06 2024-09-16T20:50:32.914 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.914 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:32.914 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.914 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.914 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.915 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1633 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:32.915 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.869550+0000", 2024-09-16T20:50:32.915 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.019456773, 2024-09-16T20:50:32.915 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.019584028, 2024-09-16T20:50:32.915 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:32.915 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:32.915 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:32.915 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:32.915 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:32.915 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:32.915 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1633 2024-09-16T20:50:32.915 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.915 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:32.915 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.916 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:32.916 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.869550+0000", 2024-09-16T20:50:32.916 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.916 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.916 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.916 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:32.916 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.869550+0000", 2024-09-16T20:50:32.916 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.916 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.916 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.916 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:32.916 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.869661+0000", 2024-09-16T20:50:32.916 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011096599999999999 2024-09-16T20:50:32.917 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.917 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.917 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:32.917 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.869664+0000", 2024-09-16T20:50:32.917 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.004e-06 2024-09-16T20:50:32.917 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.917 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.917 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:32.917 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.869669+0000", 2024-09-16T20:50:32.917 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0370000000000002e-06 2024-09-16T20:50:32.917 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.917 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.917 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.917 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.869673+0000", 2024-09-16T20:50:32.918 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.382e-06 2024-09-16T20:50:32.918 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.918 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:32.918 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.918 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.918 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.918 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1665 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:32.918 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.872778+0000", 2024-09-16T20:50:32.918 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.016229252999999999, 2024-09-16T20:50:32.918 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.016381086999999999, 2024-09-16T20:50:32.918 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:32.918 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:32.918 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:32.919 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:32.919 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:32.919 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:32.919 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1665 2024-09-16T20:50:32.919 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.919 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:32.919 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.919 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:32.919 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872778+0000", 2024-09-16T20:50:32.919 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.919 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.920 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.920 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:32.920 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872778+0000", 2024-09-16T20:50:32.920 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.920 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.920 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.920 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:32.920 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872887+0000", 2024-09-16T20:50:32.920 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000108838 2024-09-16T20:50:32.920 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.920 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.920 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:32.920 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872890+0000", 2024-09-16T20:50:32.921 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3299999999999999e-06 2024-09-16T20:50:32.921 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.921 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.921 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:32.921 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872895+0000", 2024-09-16T20:50:32.921 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1250000000000001e-06 2024-09-16T20:50:32.921 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.921 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.921 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.921 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872899+0000", 2024-09-16T20:50:32.921 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2019999999999997e-06 2024-09-16T20:50:32.921 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.921 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:32.921 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.922 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.922 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1697 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:32.922 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.880524+0000", 2024-09-16T20:50:32.922 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0084826420000000003, 2024-09-16T20:50:32.922 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0086572669999999997, 2024-09-16T20:50:32.922 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:32.922 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:32.922 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:32.922 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:32.922 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:32.922 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:32.923 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1697 2024-09-16T20:50:32.923 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.923 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:32.923 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.923 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:32.923 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880524+0000", 2024-09-16T20:50:32.923 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.923 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.923 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.923 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:32.923 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880524+0000", 2024-09-16T20:50:32.923 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.923 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.923 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.923 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:32.924 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880525+0000", 2024-09-16T20:50:32.924 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1799999999999999e-07 2024-09-16T20:50:32.924 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.924 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.924 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:32.924 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880526+0000", 2024-09-16T20:50:32.924 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0300000000000001e-06 2024-09-16T20:50:32.924 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.924 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.925 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:32.925 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880529+0000", 2024-09-16T20:50:32.925 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9100000000000001e-06 2024-09-16T20:50:32.925 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.925 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.925 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.925 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880532+0000", 2024-09-16T20:50:32.925 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1690000000000001e-06 2024-09-16T20:50:32.925 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.925 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:32.925 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.925 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.925 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.925 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1729 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:32.926 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.880901+0000", 2024-09-16T20:50:32.926 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0081059789999999993, 2024-09-16T20:50:32.926 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0083071140000000009, 2024-09-16T20:50:32.926 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:32.926 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:32.926 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:32.926 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:32.926 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:32.926 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:32.926 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1729 2024-09-16T20:50:32.926 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.926 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:32.926 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.926 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:32.927 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880901+0000", 2024-09-16T20:50:32.927 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.927 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.927 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.927 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:32.927 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880901+0000", 2024-09-16T20:50:32.927 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.927 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.927 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.927 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:32.927 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880901+0000", 2024-09-16T20:50:32.927 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0800000000000001e-07 2024-09-16T20:50:32.927 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.928 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.928 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:32.928 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880902+0000", 2024-09-16T20:50:32.928 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.95e-07 2024-09-16T20:50:32.928 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.928 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.928 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:32.928 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880905+0000", 2024-09-16T20:50:32.928 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9229999999999998e-06 2024-09-16T20:50:32.928 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.928 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.928 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.928 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880908+0000", 2024-09-16T20:50:32.928 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5459999999999998e-06 2024-09-16T20:50:32.929 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.929 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:32.929 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.929 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.929 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.929 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1761 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:32.929 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.886508+0000", 2024-09-16T20:50:32.929 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0024990809999999998, 2024-09-16T20:50:32.929 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0027452029999999999, 2024-09-16T20:50:32.929 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:32.929 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:32.929 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:32.929 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:32.930 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:32.930 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:32.930 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1761 2024-09-16T20:50:32.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.930 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:32.930 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.930 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:32.930 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886508+0000", 2024-09-16T20:50:32.930 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.930 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.930 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:32.930 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886508+0000", 2024-09-16T20:50:32.930 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.931 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.931 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.931 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:32.931 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886508+0000", 2024-09-16T20:50:32.931 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0599999999999999e-07 2024-09-16T20:50:32.931 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.931 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.931 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:32.931 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886509+0000", 2024-09-16T20:50:32.931 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9600000000000008e-07 2024-09-16T20:50:32.931 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.931 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.931 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:32.931 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886512+0000", 2024-09-16T20:50:32.932 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8830000000000002e-06 2024-09-16T20:50:32.932 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.932 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.932 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.932 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886515+0000", 2024-09-16T20:50:32.932 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6910000000000002e-06 2024-09-16T20:50:32.932 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.932 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:32.932 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.932 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.932 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.932 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1793 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:32.932 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.886885+0000", 2024-09-16T20:50:32.932 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0021217190000000002, 2024-09-16T20:50:32.933 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0024157499999999999, 2024-09-16T20:50:32.933 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:32.933 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:32.933 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:32.933 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:32.933 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:32.933 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:32.933 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1793 2024-09-16T20:50:32.933 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.933 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:32.933 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.933 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:32.933 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886885+0000", 2024-09-16T20:50:32.934 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.934 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.934 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.934 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:32.934 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886885+0000", 2024-09-16T20:50:32.934 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.934 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.934 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.934 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:32.934 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886886+0000", 2024-09-16T20:50:32.934 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2499999999999999e-07 2024-09-16T20:50:32.934 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.934 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.934 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:32.934 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886886+0000", 2024-09-16T20:50:32.935 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7600000000000006e-07 2024-09-16T20:50:32.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.935 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.935 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:32.935 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886890+0000", 2024-09-16T20:50:32.935 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5250000000000001e-06 2024-09-16T20:50:32.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.935 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.935 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.935 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886893+0000", 2024-09-16T20:50:32.935 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7939999999999998e-06 2024-09-16T20:50:32.935 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.935 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:32.935 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.936 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.936 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.936 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1570 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:32.936 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.863284+0000", 2024-09-16T20:50:32.936 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.025722827, 2024-09-16T20:50:32.936 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.026061339999999999, 2024-09-16T20:50:32.936 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:32.936 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:32.936 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:32.936 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:32.936 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:32.936 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:32.936 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1570 2024-09-16T20:50:32.937 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.937 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:32.937 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.937 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:32.937 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.863284+0000", 2024-09-16T20:50:32.937 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.937 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.937 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.937 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:32.937 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.863284+0000", 2024-09-16T20:50:32.937 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.937 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.938 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.938 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:32.938 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.863284+0000", 2024-09-16T20:50:32.938 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.04e-07 2024-09-16T20:50:32.938 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.938 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.938 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:32.938 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.863285+0000", 2024-09-16T20:50:32.938 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.052e-06 2024-09-16T20:50:32.938 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.938 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.938 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:32.938 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.863288+0000", 2024-09-16T20:50:32.939 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.038e-06 2024-09-16T20:50:32.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.939 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.939 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.863291+0000", 2024-09-16T20:50:32.939 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6450000000000001e-06 2024-09-16T20:50:32.939 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.939 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:32.939 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.939 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1602 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:32.939 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.866399+0000", 2024-09-16T20:50:32.939 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.022607518, 2024-09-16T20:50:32.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.022972932000000001, 2024-09-16T20:50:32.940 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:32.940 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:32.940 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:32.940 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:32.940 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:32.940 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:32.940 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1602 2024-09-16T20:50:32.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.940 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:32.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:32.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.866399+0000", 2024-09-16T20:50:32.941 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.941 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.941 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.941 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:32.941 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.866399+0000", 2024-09-16T20:50:32.941 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.941 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.941 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.941 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:32.941 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.866493+0000", 2024-09-16T20:50:32.941 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3676000000000002e-05 2024-09-16T20:50:32.941 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.941 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.942 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:32.942 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.866496+0000", 2024-09-16T20:50:32.942 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5320000000000001e-06 2024-09-16T20:50:32.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.942 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.942 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:32.942 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.866501+0000", 2024-09-16T20:50:32.942 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1259999999999997e-06 2024-09-16T20:50:32.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.942 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.942 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.942 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.866504+0000", 2024-09-16T20:50:32.942 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5939999999999998e-06 2024-09-16T20:50:32.942 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.943 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:32.943 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.943 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1634 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:32.943 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.869679+0000", 2024-09-16T20:50:32.943 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.019328391, 2024-09-16T20:50:32.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.019717017, 2024-09-16T20:50:32.943 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:32.943 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:32.943 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:32.943 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:32.943 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:32.944 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:32.944 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1634 2024-09-16T20:50:32.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.944 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:32.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.944 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:32.944 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.869679+0000", 2024-09-16T20:50:32.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.944 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:32.944 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.869679+0000", 2024-09-16T20:50:32.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.945 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:32.945 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.869693+0000", 2024-09-16T20:50:32.945 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4608e-05 2024-09-16T20:50:32.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.945 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:32.945 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.869695+0000", 2024-09-16T20:50:32.945 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7719999999999999e-06 2024-09-16T20:50:32.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.945 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:32.946 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.869699+0000", 2024-09-16T20:50:32.946 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8430000000000003e-06 2024-09-16T20:50:32.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.946 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.946 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.946 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.869702+0000", 2024-09-16T20:50:32.946 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4130000000000002e-06 2024-09-16T20:50:32.946 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.946 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:32.946 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.946 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.946 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1666 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:32.946 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.872911+0000", 2024-09-16T20:50:32.946 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.016095627000000001, 2024-09-16T20:50:32.947 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.016516776, 2024-09-16T20:50:32.947 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:32.947 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:32.947 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:32.947 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:32.947 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:32.947 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:32.947 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1666 2024-09-16T20:50:32.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.947 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:32.947 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.947 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:32.947 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872911+0000", 2024-09-16T20:50:32.947 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.948 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:32.948 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872911+0000", 2024-09-16T20:50:32.948 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.948 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:32.948 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872932+0000", 2024-09-16T20:50:32.948 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0758e-05 2024-09-16T20:50:32.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.948 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:32.949 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872934+0000", 2024-09-16T20:50:32.949 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1890000000000001e-06 2024-09-16T20:50:32.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.949 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.949 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:32.949 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872939+0000", 2024-09-16T20:50:32.949 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.454e-06 2024-09-16T20:50:32.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.949 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.949 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.949 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872942+0000", 2024-09-16T20:50:32.949 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3859999999999999e-06 2024-09-16T20:50:32.949 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.950 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:32.950 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.950 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.950 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1698 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:32.950 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.880535+0000", 2024-09-16T20:50:32.950 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0084723469999999999, 2024-09-16T20:50:32.950 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0089403369999999996, 2024-09-16T20:50:32.950 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:32.950 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:32.950 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:32.950 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:32.950 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:32.950 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:32.950 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1698 2024-09-16T20:50:32.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.951 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:32.951 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.951 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:32.951 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880535+0000", 2024-09-16T20:50:32.951 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.951 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.951 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:32.951 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880535+0000", 2024-09-16T20:50:32.951 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.951 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.951 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:32.951 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880535+0000", 2024-09-16T20:50:32.952 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1400000000000001e-07 2024-09-16T20:50:32.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.952 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.952 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:32.952 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880536+0000", 2024-09-16T20:50:32.952 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.074e-06 2024-09-16T20:50:32.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.952 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.952 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:32.952 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880540+0000", 2024-09-16T20:50:32.952 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1330000000000001e-06 2024-09-16T20:50:32.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.952 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.953 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.953 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880543+0000", 2024-09-16T20:50:32.953 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.638e-06 2024-09-16T20:50:32.953 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.953 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:32.953 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.953 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.953 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1730 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:32.953 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.880911+0000", 2024-09-16T20:50:32.953 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0080962259999999998, 2024-09-16T20:50:32.953 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0086120199999999997, 2024-09-16T20:50:32.953 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:32.953 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:32.954 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:32.954 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:32.954 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:32.954 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:32.954 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1730 2024-09-16T20:50:32.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.954 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:32.954 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.954 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:32.954 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880911+0000", 2024-09-16T20:50:32.954 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.954 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.954 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:32.955 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880911+0000", 2024-09-16T20:50:32.955 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.955 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:32.955 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880911+0000", 2024-09-16T20:50:32.955 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1799999999999999e-07 2024-09-16T20:50:32.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.955 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:32.955 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880915+0000", 2024-09-16T20:50:32.955 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7280000000000001e-06 2024-09-16T20:50:32.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.956 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:32.956 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880918+0000", 2024-09-16T20:50:32.956 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3670000000000001e-06 2024-09-16T20:50:32.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.956 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.956 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880921+0000", 2024-09-16T20:50:32.956 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5069999999999999e-06 2024-09-16T20:50:32.956 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.956 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:32.956 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.957 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1762 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:32.957 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.886518+0000", 2024-09-16T20:50:32.957 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.002489308, 2024-09-16T20:50:32.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003052337, 2024-09-16T20:50:32.957 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:32.957 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:32.957 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:32.957 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:32.957 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:32.957 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:32.957 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1762 2024-09-16T20:50:32.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.957 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:32.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.958 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:32.958 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886518+0000", 2024-09-16T20:50:32.958 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.958 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.958 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.958 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:32.958 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886518+0000", 2024-09-16T20:50:32.958 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.958 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.958 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.958 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:32.958 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886518+0000", 2024-09-16T20:50:32.958 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0699999999999999e-07 2024-09-16T20:50:32.958 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.959 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.959 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:32.959 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886519+0000", 2024-09-16T20:50:32.959 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8200000000000008e-07 2024-09-16T20:50:32.959 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.959 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.959 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:32.959 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886522+0000", 2024-09-16T20:50:32.959 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.864e-06 2024-09-16T20:50:32.959 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.959 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.959 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.959 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886524+0000", 2024-09-16T20:50:32.960 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.458e-06 2024-09-16T20:50:32.960 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.960 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:32.960 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.960 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1794 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:32.960 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.886896+0000", 2024-09-16T20:50:32.960 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0021113490000000002, 2024-09-16T20:50:32.960 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.002720971, 2024-09-16T20:50:32.960 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:32.960 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:32.960 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:32.961 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:32.961 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:32.961 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:32.961 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1794 2024-09-16T20:50:32.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.961 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:32.961 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.961 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:32.961 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886896+0000", 2024-09-16T20:50:32.961 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.961 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.962 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:32.962 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886896+0000", 2024-09-16T20:50:32.962 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.962 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:32.962 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886896+0000", 2024-09-16T20:50:32.962 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0699999999999999e-07 2024-09-16T20:50:32.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.962 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:32.962 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886897+0000", 2024-09-16T20:50:32.962 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9199999999999999e-07 2024-09-16T20:50:32.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.963 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.963 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:32.963 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886900+0000", 2024-09-16T20:50:32.963 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.886e-06 2024-09-16T20:50:32.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.963 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.963 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.963 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886902+0000", 2024-09-16T20:50:32.963 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.605e-06 2024-09-16T20:50:32.963 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.963 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:32.963 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.964 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.964 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1571 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:32.964 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.863294+0000", 2024-09-16T20:50:32.964 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.02571304, 2024-09-16T20:50:32.964 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.026367043999999999, 2024-09-16T20:50:32.965 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:32.965 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:32.965 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:32.965 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:32.965 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:32.965 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:32.965 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1571 2024-09-16T20:50:32.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.965 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:32.965 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.965 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:32.965 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.863294+0000", 2024-09-16T20:50:32.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.966 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:32.966 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.863294+0000", 2024-09-16T20:50:32.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.966 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:32.966 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.863499+0000", 2024-09-16T20:50:32.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00020540200000000001 2024-09-16T20:50:32.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.966 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:32.967 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.863502+0000", 2024-09-16T20:50:32.967 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.588e-06 2024-09-16T20:50:32.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.967 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:32.967 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.863510+0000", 2024-09-16T20:50:32.967 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4880000000000007e-06 2024-09-16T20:50:32.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.967 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.967 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.863519+0000", 2024-09-16T20:50:32.967 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1089999999999998e-06 2024-09-16T20:50:32.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.968 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:32.968 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.889647+0000", 2024-09-16T20:50:32.968 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.026128327 2024-09-16T20:50:32.968 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.968 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:32.968 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.968 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.968 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.968 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1603 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:32.968 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.866511+0000", 2024-09-16T20:50:32.968 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.022495973999999998, 2024-09-16T20:50:32.968 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.023194991000000002, 2024-09-16T20:50:32.969 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:32.969 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:32.969 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:32.969 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:32.969 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:32.969 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:32.969 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1603 2024-09-16T20:50:32.969 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.969 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:32.969 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.969 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:32.969 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.866511+0000", 2024-09-16T20:50:32.970 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.970 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.970 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:32.970 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.866511+0000", 2024-09-16T20:50:32.970 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.970 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.970 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:32.970 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.866608+0000", 2024-09-16T20:50:32.970 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7252999999999999e-05 2024-09-16T20:50:32.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.970 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.971 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:32.971 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.866611+0000", 2024-09-16T20:50:32.971 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2960000000000001e-06 2024-09-16T20:50:32.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.971 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.971 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:32.971 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.866615+0000", 2024-09-16T20:50:32.971 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9219999999999996e-06 2024-09-16T20:50:32.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.971 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.971 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.971 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.866620+0000", 2024-09-16T20:50:32.971 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2969999999999997e-06 2024-09-16T20:50:32.971 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.972 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:32.972 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.972 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.972 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1635 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:32.972 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.869707+0000", 2024-09-16T20:50:32.972 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.019300258000000001, 2024-09-16T20:50:32.972 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.020022699000000001, 2024-09-16T20:50:32.972 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:32.972 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:32.972 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:32.972 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:32.973 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:32.973 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:32.973 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1635 2024-09-16T20:50:32.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.973 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:32.973 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.973 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:32.973 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.869707+0000", 2024-09-16T20:50:32.973 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.973 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.974 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:32.974 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.869707+0000", 2024-09-16T20:50:32.974 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.974 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:32.974 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.869796+0000", 2024-09-16T20:50:32.974 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9146000000000003e-05 2024-09-16T20:50:32.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.974 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:32.974 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.869801+0000", 2024-09-16T20:50:32.974 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1089999999999997e-06 2024-09-16T20:50:32.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.975 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.975 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:32.975 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.869807+0000", 2024-09-16T20:50:32.975 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6160000000000001e-06 2024-09-16T20:50:32.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.975 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.975 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.975 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.869810+0000", 2024-09-16T20:50:32.975 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4230000000000001e-06 2024-09-16T20:50:32.975 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.975 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:32.975 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.975 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.975 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1667 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:32.976 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.872948+0000", 2024-09-16T20:50:32.976 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.016059426000000002, 2024-09-16T20:50:32.976 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.016804481999999999, 2024-09-16T20:50:32.976 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:32.976 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:32.976 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:32.976 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:32.976 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:32.976 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:32.976 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1667 2024-09-16T20:50:32.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.976 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:32.976 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.976 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:32.976 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872948+0000", 2024-09-16T20:50:32.977 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.977 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:32.977 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872948+0000", 2024-09-16T20:50:32.977 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.977 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:32.977 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.873144+0000", 2024-09-16T20:50:32.977 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00019648199999999999 2024-09-16T20:50:32.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.978 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:32.978 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.873147+0000", 2024-09-16T20:50:32.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0759999999999999e-06 2024-09-16T20:50:32.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.978 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.978 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:32.978 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.873153+0000", 2024-09-16T20:50:32.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2190000000000001e-06 2024-09-16T20:50:32.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.978 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.978 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.978 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.873158+0000", 2024-09-16T20:50:32.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.391e-06 2024-09-16T20:50:32.978 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.979 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:32.979 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.979 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.979 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.979 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1699 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:32.979 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.880546+0000", 2024-09-16T20:50:32.979 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0084613180000000007, 2024-09-16T20:50:32.979 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.009244347, 2024-09-16T20:50:32.979 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:32.980 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:32.980 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:32.980 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:32.980 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:32.980 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:32.980 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1699 2024-09-16T20:50:32.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.980 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:32.980 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.980 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:32.980 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880546+0000", 2024-09-16T20:50:32.980 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.980 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.981 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:32.981 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880546+0000", 2024-09-16T20:50:32.981 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.981 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.981 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:32.981 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880546+0000", 2024-09-16T20:50:32.981 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0900000000000001e-07 2024-09-16T20:50:32.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.981 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.981 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:32.981 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880547+0000", 2024-09-16T20:50:32.981 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.004e-06 2024-09-16T20:50:32.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.982 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.982 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:32.982 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880550+0000", 2024-09-16T20:50:32.982 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.039e-06 2024-09-16T20:50:32.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.982 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.982 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.982 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880553+0000", 2024-09-16T20:50:32.982 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.219e-06 2024-09-16T20:50:32.982 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.982 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:32.982 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.983 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.983 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1731 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:32.983 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.880924+0000", 2024-09-16T20:50:32.983 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0080833599999999995, 2024-09-16T20:50:32.983 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0089152150000000006, 2024-09-16T20:50:32.983 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:32.983 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:32.983 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:32.983 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:32.983 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:32.983 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:32.983 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1731 2024-09-16T20:50:32.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.984 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:32.984 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.984 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:32.984 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880924+0000", 2024-09-16T20:50:32.984 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.984 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.984 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:32.984 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880924+0000", 2024-09-16T20:50:32.984 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.984 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.984 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:32.984 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880924+0000", 2024-09-16T20:50:32.985 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3799999999999999e-07 2024-09-16T20:50:32.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.985 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.985 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:32.985 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880925+0000", 2024-09-16T20:50:32.985 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1399999999999995e-07 2024-09-16T20:50:32.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.985 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.985 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:32.985 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880928+0000", 2024-09-16T20:50:32.985 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9270000000000001e-06 2024-09-16T20:50:32.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.985 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.985 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.986 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880930+0000", 2024-09-16T20:50:32.986 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4399999999999999e-06 2024-09-16T20:50:32.986 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.986 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:32.986 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.986 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.986 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.986 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1763 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:32.986 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.886527+0000", 2024-09-16T20:50:32.986 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0024798860000000002, 2024-09-16T20:50:32.986 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0033599229999999999, 2024-09-16T20:50:32.986 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:32.986 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:32.986 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:32.987 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:32.987 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:32.987 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:32.987 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1763 2024-09-16T20:50:32.987 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.987 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:32.987 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.987 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:32.987 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886527+0000", 2024-09-16T20:50:32.987 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.987 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.987 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.987 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:32.987 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886527+0000", 2024-09-16T20:50:32.988 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.988 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.988 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.988 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:32.988 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886527+0000", 2024-09-16T20:50:32.988 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0800000000000001e-07 2024-09-16T20:50:32.988 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.988 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.988 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:32.988 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886528+0000", 2024-09-16T20:50:32.988 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9199999999999999e-07 2024-09-16T20:50:32.988 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.988 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.989 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:32.989 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886531+0000", 2024-09-16T20:50:32.989 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9780000000000001e-06 2024-09-16T20:50:32.989 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.989 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.989 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.989 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886534+0000", 2024-09-16T20:50:32.989 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5320000000000001e-06 2024-09-16T20:50:32.989 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.989 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:32.989 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.989 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.989 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.989 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1795 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:32.990 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.886905+0000", 2024-09-16T20:50:32.990 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0021018790000000001, 2024-09-16T20:50:32.990 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030285099999999999, 2024-09-16T20:50:32.990 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:32.990 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:32.990 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:32.990 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:32.990 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:32.990 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:32.990 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1795 2024-09-16T20:50:32.990 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.990 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:32.990 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.991 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:32.991 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886905+0000", 2024-09-16T20:50:32.991 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.991 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.991 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.991 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:32.991 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886905+0000", 2024-09-16T20:50:32.991 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.991 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.991 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.991 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:32.991 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886905+0000", 2024-09-16T20:50:32.991 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0800000000000001e-07 2024-09-16T20:50:32.991 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.992 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.992 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:32.992 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886906+0000", 2024-09-16T20:50:32.992 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0860000000000001e-06 2024-09-16T20:50:32.992 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.992 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.992 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:32.992 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886909+0000", 2024-09-16T20:50:32.992 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8650000000000001e-06 2024-09-16T20:50:32.992 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.992 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.992 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.992 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886912+0000", 2024-09-16T20:50:32.993 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5670000000000002e-06 2024-09-16T20:50:32.993 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.993 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:32.993 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.993 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.993 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.993 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1604 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:32.993 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.866634+0000", 2024-09-16T20:50:32.993 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.022373291999999999, 2024-09-16T20:50:32.993 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.023348526000000001, 2024-09-16T20:50:32.993 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:32.993 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:32.993 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:32.993 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:32.994 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:32.994 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:32.994 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1604 2024-09-16T20:50:32.994 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.994 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:32.994 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.994 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:32.994 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.866634+0000", 2024-09-16T20:50:32.994 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.994 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.994 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.994 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:32.994 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.866634+0000", 2024-09-16T20:50:32.995 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.995 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.995 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.995 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:32.995 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.866634+0000", 2024-09-16T20:50:32.995 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3000000000000001e-07 2024-09-16T20:50:32.995 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.995 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.995 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:32.995 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.866636+0000", 2024-09-16T20:50:32.995 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.655e-06 2024-09-16T20:50:32.995 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.995 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.995 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:32.995 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.866640+0000", 2024-09-16T20:50:32.996 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7670000000000001e-06 2024-09-16T20:50:32.996 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.996 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.996 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.996 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.866642+0000", 2024-09-16T20:50:32.996 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8760000000000001e-06 2024-09-16T20:50:32.996 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.996 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:32.996 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.996 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.996 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.996 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1636 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:32.996 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.869816+0000", 2024-09-16T20:50:32.996 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.019191094999999998, 2024-09-16T20:50:32.996 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.020213743999999999, 2024-09-16T20:50:32.997 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:32.997 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:32.997 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:32.997 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:32.997 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:32.997 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:32.997 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1636 2024-09-16T20:50:32.997 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.997 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:32.997 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.997 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:32.997 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.869816+0000", 2024-09-16T20:50:32.997 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.997 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.998 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.998 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:32.998 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.869816+0000", 2024-09-16T20:50:32.998 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:32.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.998 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.998 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:32.998 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.869920+0000", 2024-09-16T20:50:32.998 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010435899999999999 2024-09-16T20:50:32.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.998 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.998 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:32.998 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.869923+0000", 2024-09-16T20:50:32.998 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2279999999999998e-06 2024-09-16T20:50:32.999 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.999 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.999 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:32.999 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.869930+0000", 2024-09-16T20:50:32.999 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2400000000000004e-06 2024-09-16T20:50:32.999 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:32.999 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:32.999 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:32.999 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.869934+0000", 2024-09-16T20:50:32.999 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2869999999999998e-06 2024-09-16T20:50:32.999 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.999 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:32.999 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:32.999 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.000 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.000 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1668 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.000 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.873164+0000", 2024-09-16T20:50:33.000 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.015842992, 2024-09-16T20:50:33.000 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.016913695999999999, 2024-09-16T20:50:33.000 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.000 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.000 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.000 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.000 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.000 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.000 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1668 2024-09-16T20:50:33.000 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.000 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.000 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.001 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.001 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.873164+0000", 2024-09-16T20:50:33.001 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.001 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.001 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.001 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.873164+0000", 2024-09-16T20:50:33.001 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.001 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.001 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.001 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.873164+0000", 2024-09-16T20:50:33.001 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9000000000000002e-07 2024-09-16T20:50:33.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.002 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.002 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.873166+0000", 2024-09-16T20:50:33.002 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.243e-06 2024-09-16T20:50:33.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.002 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.002 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.873169+0000", 2024-09-16T20:50:33.002 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7500000000000001e-06 2024-09-16T20:50:33.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.002 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.003 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.873172+0000", 2024-09-16T20:50:33.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.937e-06 2024-09-16T20:50:33.003 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.003 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.003 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.003 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.003 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1700 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.003 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.880556+0000", 2024-09-16T20:50:33.003 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0084508489999999999, 2024-09-16T20:50:33.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0095688319999999993, 2024-09-16T20:50:33.003 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.003 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.003 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.004 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.004 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.004 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.004 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1700 2024-09-16T20:50:33.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.004 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.004 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.004 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.004 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880556+0000", 2024-09-16T20:50:33.004 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.004 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.004 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.004 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880556+0000", 2024-09-16T20:50:33.005 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.005 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.005 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.005 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.005 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880556+0000", 2024-09-16T20:50:33.005 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.04e-07 2024-09-16T20:50:33.005 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.005 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.005 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.005 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880558+0000", 2024-09-16T20:50:33.005 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.26e-06 2024-09-16T20:50:33.005 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.005 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.006 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.006 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880562+0000", 2024-09-16T20:50:33.006 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9960000000000004e-06 2024-09-16T20:50:33.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.006 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.006 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.006 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880564+0000", 2024-09-16T20:50:33.006 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6340000000000001e-06 2024-09-16T20:50:33.006 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.006 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.006 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.006 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.006 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1732 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.007 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.880933+0000", 2024-09-16T20:50:33.007 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.00807393, 2024-09-16T20:50:33.007 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0092386680000000002, 2024-09-16T20:50:33.007 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.007 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.007 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.007 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.007 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.007 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.007 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1732 2024-09-16T20:50:33.007 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.007 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.007 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.007 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880933+0000", 2024-09-16T20:50:33.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.008 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880933+0000", 2024-09-16T20:50:33.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.008 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880933+0000", 2024-09-16T20:50:33.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0800000000000001e-07 2024-09-16T20:50:33.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.009 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.009 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880934+0000", 2024-09-16T20:50:33.009 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0440000000000001e-06 2024-09-16T20:50:33.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.009 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.009 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.009 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880937+0000", 2024-09-16T20:50:33.009 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9450000000000002e-06 2024-09-16T20:50:33.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.009 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.009 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.009 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880940+0000", 2024-09-16T20:50:33.009 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6029999999999999e-06 2024-09-16T20:50:33.010 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.010 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.010 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.010 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.010 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.010 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1764 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.010 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.886537+0000", 2024-09-16T20:50:33.010 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0024703770000000002, 2024-09-16T20:50:33.010 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0036821940000000002, 2024-09-16T20:50:33.010 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.010 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.010 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.010 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.010 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.011 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.011 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1764 2024-09-16T20:50:33.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.011 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.011 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.011 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.011 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886537+0000", 2024-09-16T20:50:33.011 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.011 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.011 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.011 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886537+0000", 2024-09-16T20:50:33.011 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.012 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.012 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.012 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886537+0000", 2024-09-16T20:50:33.012 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2999999999999999e-07 2024-09-16T20:50:33.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.012 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.012 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.012 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886538+0000", 2024-09-16T20:50:33.012 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1599999999999999e-06 2024-09-16T20:50:33.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.012 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.012 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.012 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886541+0000", 2024-09-16T20:50:33.013 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.886e-06 2024-09-16T20:50:33.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.013 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.013 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.013 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886544+0000", 2024-09-16T20:50:33.013 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3510000000000002e-06 2024-09-16T20:50:33.013 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.013 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.013 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.013 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.013 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1796 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.013 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.886915+0000", 2024-09-16T20:50:33.014 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0020922509999999998, 2024-09-16T20:50:33.014 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0033335679999999999, 2024-09-16T20:50:33.014 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.014 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.014 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.014 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.014 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.014 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.014 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1796 2024-09-16T20:50:33.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.014 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.014 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.014 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.015 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886915+0000", 2024-09-16T20:50:33.015 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.015 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.015 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.015 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886915+0000", 2024-09-16T20:50:33.015 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.015 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.015 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.015 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886915+0000", 2024-09-16T20:50:33.015 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2600000000000001e-07 2024-09-16T20:50:33.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.015 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.016 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.016 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886916+0000", 2024-09-16T20:50:33.016 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4e-07 2024-09-16T20:50:33.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.016 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.016 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.016 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886919+0000", 2024-09-16T20:50:33.016 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.892e-06 2024-09-16T20:50:33.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.016 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.016 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.016 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886921+0000", 2024-09-16T20:50:33.016 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5390000000000002e-06 2024-09-16T20:50:33.017 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.017 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.017 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.017 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.017 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.017 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1605 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.017 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.866646+0000", 2024-09-16T20:50:33.017 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.022360875999999998, 2024-09-16T20:50:33.017 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.023626372999999999, 2024-09-16T20:50:33.017 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.017 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.017 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.017 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.017 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.018 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.018 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1605 2024-09-16T20:50:33.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.018 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.018 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.018 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.018 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.866646+0000", 2024-09-16T20:50:33.018 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.018 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.018 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.019 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.866646+0000", 2024-09-16T20:50:33.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.019 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.019 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.866729+0000", 2024-09-16T20:50:33.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2513000000000001e-05 2024-09-16T20:50:33.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.019 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.019 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.866731+0000", 2024-09-16T20:50:33.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2089999999999999e-06 2024-09-16T20:50:33.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.019 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.020 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.866736+0000", 2024-09-16T20:50:33.020 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1189999999999996e-06 2024-09-16T20:50:33.020 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.020 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.020 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.020 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.866740+0000", 2024-09-16T20:50:33.020 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.551e-06 2024-09-16T20:50:33.020 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.020 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.020 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.020 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.020 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.020 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1637 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.020 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.869940+0000", 2024-09-16T20:50:33.021 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.019066787000000002, 2024-09-16T20:50:33.021 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.020355390000000001, 2024-09-16T20:50:33.021 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.021 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.021 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.021 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.021 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.021 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.021 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1637 2024-09-16T20:50:33.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.021 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.021 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.022 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.022 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.869940+0000", 2024-09-16T20:50:33.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.022 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.022 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.022 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.869940+0000", 2024-09-16T20:50:33.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.022 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.022 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.022 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.870047+0000", 2024-09-16T20:50:33.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000107182 2024-09-16T20:50:33.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.023 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.023 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.023 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.870050+0000", 2024-09-16T20:50:33.023 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3530000000000002e-06 2024-09-16T20:50:33.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.023 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.023 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.023 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.870055+0000", 2024-09-16T20:50:33.023 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9740000000000001e-06 2024-09-16T20:50:33.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.023 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.023 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.023 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.870058+0000", 2024-09-16T20:50:33.023 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3119999999999998e-06 2024-09-16T20:50:33.023 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.024 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.024 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.024 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1669 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.024 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.873175+0000", 2024-09-16T20:50:33.024 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.01583159, 2024-09-16T20:50:33.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.017163570999999999, 2024-09-16T20:50:33.024 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.024 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.024 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.024 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.024 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.024 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.025 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1669 2024-09-16T20:50:33.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.025 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.873175+0000", 2024-09-16T20:50:33.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.873175+0000", 2024-09-16T20:50:33.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.026 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.873274+0000", 2024-09-16T20:50:33.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8306000000000004e-05 2024-09-16T20:50:33.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.026 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.873277+0000", 2024-09-16T20:50:33.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8629999999999999e-06 2024-09-16T20:50:33.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.026 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.873282+0000", 2024-09-16T20:50:33.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9230000000000001e-06 2024-09-16T20:50:33.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.027 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.027 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.873285+0000", 2024-09-16T20:50:33.027 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4489999999999999e-06 2024-09-16T20:50:33.027 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.027 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.027 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.027 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.027 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1701 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.027 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.880567+0000", 2024-09-16T20:50:33.027 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0084398049999999999, 2024-09-16T20:50:33.027 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0098677850000000004, 2024-09-16T20:50:33.027 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.028 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.028 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.028 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.028 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.028 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.028 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1701 2024-09-16T20:50:33.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.028 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.028 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.028 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.028 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880567+0000", 2024-09-16T20:50:33.028 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.029 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.029 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880567+0000", 2024-09-16T20:50:33.029 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.029 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.029 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880568+0000", 2024-09-16T20:50:33.029 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4999999999999998e-07 2024-09-16T20:50:33.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.029 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.029 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880569+0000", 2024-09-16T20:50:33.029 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0610000000000001e-06 2024-09-16T20:50:33.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.030 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.030 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880572+0000", 2024-09-16T20:50:33.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9739999999999998e-06 2024-09-16T20:50:33.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.030 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.030 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880574+0000", 2024-09-16T20:50:33.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5720000000000001e-06 2024-09-16T20:50:33.030 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.030 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.030 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.031 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.031 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1733 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.031 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.880943+0000", 2024-09-16T20:50:33.031 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0080641670000000006, 2024-09-16T20:50:33.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0095423460000000002, 2024-09-16T20:50:33.031 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.031 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.031 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.031 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.031 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.031 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.031 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1733 2024-09-16T20:50:33.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.032 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.032 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.032 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880943+0000", 2024-09-16T20:50:33.032 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.032 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.032 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880943+0000", 2024-09-16T20:50:33.032 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.032 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.032 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880943+0000", 2024-09-16T20:50:33.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0900000000000001e-07 2024-09-16T20:50:33.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.033 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.033 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.033 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880944+0000", 2024-09-16T20:50:33.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3600000000000002e-07 2024-09-16T20:50:33.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.033 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.033 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.033 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880948+0000", 2024-09-16T20:50:33.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0799999999999999e-06 2024-09-16T20:50:33.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.033 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.034 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.034 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880951+0000", 2024-09-16T20:50:33.034 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.571e-06 2024-09-16T20:50:33.034 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.034 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.034 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.034 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.034 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1765 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.034 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.886547+0000", 2024-09-16T20:50:33.034 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.002459691, 2024-09-16T20:50:33.034 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0039849849999999999, 2024-09-16T20:50:33.034 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.034 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.034 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.035 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.035 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.035 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.035 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1765 2024-09-16T20:50:33.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.035 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.035 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.035 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886547+0000", 2024-09-16T20:50:33.035 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.035 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.035 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886547+0000", 2024-09-16T20:50:33.035 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.036 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.036 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886548+0000", 2024-09-16T20:50:33.036 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6100000000000002e-07 2024-09-16T20:50:33.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.036 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.036 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886549+0000", 2024-09-16T20:50:33.036 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.031e-06 2024-09-16T20:50:33.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.036 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.037 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886553+0000", 2024-09-16T20:50:33.037 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.031e-06 2024-09-16T20:50:33.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.037 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.037 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.037 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886555+0000", 2024-09-16T20:50:33.037 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6630000000000002e-06 2024-09-16T20:50:33.037 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.037 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.037 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.037 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.037 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1797 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.038 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.886924+0000", 2024-09-16T20:50:33.038 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0020827649999999999, 2024-09-16T20:50:33.038 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0036544759999999998, 2024-09-16T20:50:33.038 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.038 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.038 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.038 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.038 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.038 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.038 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1797 2024-09-16T20:50:33.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.038 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.038 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.038 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.039 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886924+0000", 2024-09-16T20:50:33.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.039 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.039 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886924+0000", 2024-09-16T20:50:33.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.039 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.039 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886925+0000", 2024-09-16T20:50:33.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7099999999999998e-07 2024-09-16T20:50:33.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.040 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.040 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.040 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886925+0000", 2024-09-16T20:50:33.040 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6599999999999994e-07 2024-09-16T20:50:33.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.040 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.040 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.040 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886928+0000", 2024-09-16T20:50:33.040 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.897e-06 2024-09-16T20:50:33.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.040 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.040 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.040 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886931+0000", 2024-09-16T20:50:33.040 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4820000000000001e-06 2024-09-16T20:50:33.041 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.041 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.041 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.041 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.041 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.041 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1606 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.041 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.866745+0000", 2024-09-16T20:50:33.041 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.022261620999999999, 2024-09-16T20:50:33.041 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.023862329000000002, 2024-09-16T20:50:33.041 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.041 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.041 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.041 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.042 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.042 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.042 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1606 2024-09-16T20:50:33.042 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.042 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.042 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.042 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.042 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.866745+0000", 2024-09-16T20:50:33.042 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.042 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.042 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.042 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.042 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.866745+0000", 2024-09-16T20:50:33.043 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.043 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.043 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.043 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.866843+0000", 2024-09-16T20:50:33.043 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7505999999999998e-05 2024-09-16T20:50:33.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.043 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.043 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.043 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.866846+0000", 2024-09-16T20:50:33.043 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.129e-06 2024-09-16T20:50:33.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.043 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.043 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.044 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.866852+0000", 2024-09-16T20:50:33.044 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3389999999999998e-06 2024-09-16T20:50:33.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.044 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.044 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.044 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.866856+0000", 2024-09-16T20:50:33.044 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.534e-06 2024-09-16T20:50:33.044 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.044 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.044 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.044 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.044 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1638 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.044 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.870064+0000", 2024-09-16T20:50:33.045 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.018942877, 2024-09-16T20:50:33.045 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.020570734, 2024-09-16T20:50:33.045 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.045 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.045 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.045 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.045 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.045 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.045 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1638 2024-09-16T20:50:33.045 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.045 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.045 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.045 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.046 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.870064+0000", 2024-09-16T20:50:33.046 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.046 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.046 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.046 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.046 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.870064+0000", 2024-09-16T20:50:33.046 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.046 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.046 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.046 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.046 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.870177+0000", 2024-09-16T20:50:33.046 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000112954 2024-09-16T20:50:33.046 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.046 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.046 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.047 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.870180+0000", 2024-09-16T20:50:33.047 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0429999999999999e-06 2024-09-16T20:50:33.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.047 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.047 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.047 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.870185+0000", 2024-09-16T20:50:33.047 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8779999999999997e-06 2024-09-16T20:50:33.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.047 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.047 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.047 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.870188+0000", 2024-09-16T20:50:33.047 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4630000000000001e-06 2024-09-16T20:50:33.047 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.047 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.048 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.048 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.048 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1670 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.048 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.873291+0000", 2024-09-16T20:50:33.048 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.015715953000000001, 2024-09-16T20:50:33.048 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.017367047, 2024-09-16T20:50:33.048 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.048 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.048 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.048 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.048 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.048 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.048 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1670 2024-09-16T20:50:33.049 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.049 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.049 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.049 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.049 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.873291+0000", 2024-09-16T20:50:33.049 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.049 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.049 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.049 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.049 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.873291+0000", 2024-09-16T20:50:33.049 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.049 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.049 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.049 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.050 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.873404+0000", 2024-09-16T20:50:33.050 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000112568 2024-09-16T20:50:33.050 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.050 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.050 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.050 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.873407+0000", 2024-09-16T20:50:33.050 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0730000000000001e-06 2024-09-16T20:50:33.050 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.050 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.050 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.050 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.873412+0000", 2024-09-16T20:50:33.050 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3489999999999999e-06 2024-09-16T20:50:33.050 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.050 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.050 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.051 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.873416+0000", 2024-09-16T20:50:33.051 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7890000000000001e-06 2024-09-16T20:50:33.051 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.051 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.051 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.051 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.051 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.051 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1702 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.051 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.880585+0000", 2024-09-16T20:50:33.051 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0084221829999999998, 2024-09-16T20:50:33.051 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.010117954, 2024-09-16T20:50:33.051 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.051 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.051 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.052 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.052 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.052 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.052 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1702 2024-09-16T20:50:33.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.052 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.052 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.052 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.052 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880585+0000", 2024-09-16T20:50:33.052 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.052 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.052 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.052 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880585+0000", 2024-09-16T20:50:33.052 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.053 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.053 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.053 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.053 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880585+0000", 2024-09-16T20:50:33.053 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.53e-07 2024-09-16T20:50:33.053 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.053 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.053 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.053 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880586+0000", 2024-09-16T20:50:33.053 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2449999999999999e-06 2024-09-16T20:50:33.053 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.053 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.053 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.054 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880590+0000", 2024-09-16T20:50:33.054 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.529e-06 2024-09-16T20:50:33.054 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.054 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.054 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.054 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880593+0000", 2024-09-16T20:50:33.054 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8420000000000001e-06 2024-09-16T20:50:33.054 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.054 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.054 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.054 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.054 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.054 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1734 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.054 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.880954+0000", 2024-09-16T20:50:33.055 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0080533489999999996, 2024-09-16T20:50:33.055 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0098004609999999999, 2024-09-16T20:50:33.055 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.055 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.055 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.055 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.055 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.055 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.055 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1734 2024-09-16T20:50:33.055 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.055 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.055 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.055 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.055 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880954+0000", 2024-09-16T20:50:33.056 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.056 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.056 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.056 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.056 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880954+0000", 2024-09-16T20:50:33.056 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.056 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.056 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.056 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.056 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880954+0000", 2024-09-16T20:50:33.056 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0599999999999999e-07 2024-09-16T20:50:33.056 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.056 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.057 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.057 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880955+0000", 2024-09-16T20:50:33.057 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0020000000000001e-06 2024-09-16T20:50:33.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.057 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.057 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.057 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880958+0000", 2024-09-16T20:50:33.057 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9289999999999998e-06 2024-09-16T20:50:33.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.057 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.057 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.057 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880960+0000", 2024-09-16T20:50:33.057 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5950000000000001e-06 2024-09-16T20:50:33.058 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.058 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.058 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.058 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.058 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.058 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1766 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.058 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.886558+0000", 2024-09-16T20:50:33.058 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0024486600000000001, 2024-09-16T20:50:33.058 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0042473349999999997, 2024-09-16T20:50:33.058 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.058 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.058 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.058 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.058 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.058 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.059 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1766 2024-09-16T20:50:33.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.059 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.059 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.059 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.059 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886558+0000", 2024-09-16T20:50:33.059 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.059 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.059 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.059 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886558+0000", 2024-09-16T20:50:33.059 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.059 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.060 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.060 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886559+0000", 2024-09-16T20:50:33.060 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.04e-07 2024-09-16T20:50:33.060 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.060 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.060 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.060 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886562+0000", 2024-09-16T20:50:33.060 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8850000000000001e-06 2024-09-16T20:50:33.060 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.060 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.060 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.060 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886566+0000", 2024-09-16T20:50:33.060 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1109999999999999e-06 2024-09-16T20:50:33.061 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.061 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.061 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.061 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886568+0000", 2024-09-16T20:50:33.061 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6299999999999998e-06 2024-09-16T20:50:33.061 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.061 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.061 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.061 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.061 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.061 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1798 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.061 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.886947+0000", 2024-09-16T20:50:33.062 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0020603349999999999, 2024-09-16T20:50:33.062 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0039061669999999999, 2024-09-16T20:50:33.062 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.062 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.062 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.062 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.062 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.062 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.062 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1798 2024-09-16T20:50:33.062 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.062 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.062 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.062 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.062 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886947+0000", 2024-09-16T20:50:33.063 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.063 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.063 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.063 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886947+0000", 2024-09-16T20:50:33.063 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.063 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.063 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.063 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886947+0000", 2024-09-16T20:50:33.063 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1199999999999998e-07 2024-09-16T20:50:33.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.063 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.063 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.064 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886949+0000", 2024-09-16T20:50:33.064 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4219999999999999e-06 2024-09-16T20:50:33.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.064 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.064 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886952+0000", 2024-09-16T20:50:33.064 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4970000000000001e-06 2024-09-16T20:50:33.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.065 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.065 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886955+0000", 2024-09-16T20:50:33.065 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7329999999999999e-06 2024-09-16T20:50:33.065 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.065 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.065 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.065 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.065 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.065 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1607 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.065 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.866862+0000", 2024-09-16T20:50:33.066 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.022145084999999998, 2024-09-16T20:50:33.066 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.024041729000000001, 2024-09-16T20:50:33.066 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.066 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.066 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.066 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.066 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.066 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.066 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1607 2024-09-16T20:50:33.066 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.066 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.066 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.066 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.067 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.866862+0000", 2024-09-16T20:50:33.067 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.067 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.067 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.067 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.067 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.866862+0000", 2024-09-16T20:50:33.067 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.067 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.067 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.067 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.067 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.866963+0000", 2024-09-16T20:50:33.067 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101535 2024-09-16T20:50:33.067 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.067 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.068 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.068 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.866967+0000", 2024-09-16T20:50:33.068 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0869999999999998e-06 2024-09-16T20:50:33.068 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.068 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.068 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.068 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.866972+0000", 2024-09-16T20:50:33.068 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2170000000000002e-06 2024-09-16T20:50:33.068 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.068 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.068 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.068 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.866975+0000", 2024-09-16T20:50:33.068 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.456e-06 2024-09-16T20:50:33.068 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.069 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.069 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.069 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.069 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1639 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.069 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.870194+0000", 2024-09-16T20:50:33.069 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.018812618, 2024-09-16T20:50:33.069 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.020744358000000001, 2024-09-16T20:50:33.069 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.069 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.069 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.069 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.069 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.070 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.070 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1639 2024-09-16T20:50:33.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.070 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.070 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.070 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.870194+0000", 2024-09-16T20:50:33.070 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.070 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.070 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.870194+0000", 2024-09-16T20:50:33.070 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.071 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.071 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.071 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.870195+0000", 2024-09-16T20:50:33.071 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7599999999999998e-07 2024-09-16T20:50:33.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.071 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.071 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.071 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.870196+0000", 2024-09-16T20:50:33.071 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.13e-06 2024-09-16T20:50:33.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.071 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.071 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.071 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.870199+0000", 2024-09-16T20:50:33.071 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.123e-06 2024-09-16T20:50:33.072 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.072 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.072 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.072 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.870202+0000", 2024-09-16T20:50:33.072 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.83e-06 2024-09-16T20:50:33.072 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.072 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.072 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.072 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.072 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.072 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1671 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.072 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.873422+0000", 2024-09-16T20:50:33.072 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.015584983, 2024-09-16T20:50:33.073 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.017544943, 2024-09-16T20:50:33.073 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.073 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.073 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.073 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.073 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.073 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.073 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1671 2024-09-16T20:50:33.073 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.073 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.073 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.073 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.073 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.873422+0000", 2024-09-16T20:50:33.074 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.074 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.074 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.074 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.873422+0000", 2024-09-16T20:50:33.074 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.074 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.074 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.074 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.873438+0000", 2024-09-16T20:50:33.074 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5597999999999999e-05 2024-09-16T20:50:33.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.074 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.074 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.074 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.873443+0000", 2024-09-16T20:50:33.075 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8629999999999996e-06 2024-09-16T20:50:33.075 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.075 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.075 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.075 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.873448+0000", 2024-09-16T20:50:33.075 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2699999999999998e-06 2024-09-16T20:50:33.075 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.075 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.075 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.075 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.873451+0000", 2024-09-16T20:50:33.075 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.162e-06 2024-09-16T20:50:33.075 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.075 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.075 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.076 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.076 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.076 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1703 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.076 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.880596+0000", 2024-09-16T20:50:33.076 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0084109779999999995, 2024-09-16T20:50:33.076 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.010393757999999999, 2024-09-16T20:50:33.076 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.076 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.076 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.076 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.076 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.076 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.076 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1703 2024-09-16T20:50:33.076 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.077 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.077 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.077 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.077 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880596+0000", 2024-09-16T20:50:33.077 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.077 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.077 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.077 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880596+0000", 2024-09-16T20:50:33.077 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.077 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.077 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.078 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880596+0000", 2024-09-16T20:50:33.078 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0199999999999998e-07 2024-09-16T20:50:33.078 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.078 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.078 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.078 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880597+0000", 2024-09-16T20:50:33.078 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0419999999999999e-06 2024-09-16T20:50:33.078 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.078 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.078 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.078 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880600+0000", 2024-09-16T20:50:33.078 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.016e-06 2024-09-16T20:50:33.078 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.078 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.079 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.079 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880603+0000", 2024-09-16T20:50:33.079 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5189999999999999e-06 2024-09-16T20:50:33.079 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.079 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.079 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.079 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.079 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.079 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1735 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.079 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.880963+0000", 2024-09-16T20:50:33.079 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0080437219999999997, 2024-09-16T20:50:33.079 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.010054347, 2024-09-16T20:50:33.080 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.080 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.080 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.080 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.080 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.080 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.080 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1735 2024-09-16T20:50:33.080 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.080 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.080 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.080 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.081 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880963+0000", 2024-09-16T20:50:33.081 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.081 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.081 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.081 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.081 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880963+0000", 2024-09-16T20:50:33.081 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.081 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.081 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.081 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.081 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880963+0000", 2024-09-16T20:50:33.081 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2000000000000001e-07 2024-09-16T20:50:33.081 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.082 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.082 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.082 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880965+0000", 2024-09-16T20:50:33.082 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0640000000000001e-06 2024-09-16T20:50:33.082 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.082 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.082 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.082 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880968+0000", 2024-09-16T20:50:33.082 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.948e-06 2024-09-16T20:50:33.082 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.082 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.082 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.082 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880970+0000", 2024-09-16T20:50:33.082 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4870000000000001e-06 2024-09-16T20:50:33.083 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.083 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.083 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.083 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.083 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.083 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1767 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.083 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.886571+0000", 2024-09-16T20:50:33.083 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0024357269999999999, 2024-09-16T20:50:33.083 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0044925800000000004, 2024-09-16T20:50:33.083 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.083 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.083 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.083 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.083 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.084 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.084 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1767 2024-09-16T20:50:33.084 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.084 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.084 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.084 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.084 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886571+0000", 2024-09-16T20:50:33.084 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.084 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.084 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.084 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.084 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886571+0000", 2024-09-16T20:50:33.084 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.084 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.085 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.085 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.085 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886572+0000", 2024-09-16T20:50:33.085 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2800000000000003e-07 2024-09-16T20:50:33.085 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.085 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.085 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.085 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886573+0000", 2024-09-16T20:50:33.085 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0139999999999999e-06 2024-09-16T20:50:33.085 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.085 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.085 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.085 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886575+0000", 2024-09-16T20:50:33.086 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8540000000000001e-06 2024-09-16T20:50:33.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.086 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.086 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.086 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886578+0000", 2024-09-16T20:50:33.086 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5129999999999999e-06 2024-09-16T20:50:33.086 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.086 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.086 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.086 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.086 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1799 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.086 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.886958+0000", 2024-09-16T20:50:33.086 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0020489050000000002, 2024-09-16T20:50:33.087 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.004156529, 2024-09-16T20:50:33.087 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.087 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.087 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.087 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.087 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.087 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.087 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1799 2024-09-16T20:50:33.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.087 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.087 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.087 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.087 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886958+0000", 2024-09-16T20:50:33.088 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.088 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.088 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.088 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.088 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886958+0000", 2024-09-16T20:50:33.088 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.088 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.088 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.088 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.088 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886958+0000", 2024-09-16T20:50:33.088 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0199999999999998e-07 2024-09-16T20:50:33.088 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.088 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.088 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.089 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886959+0000", 2024-09-16T20:50:33.089 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8200000000000008e-07 2024-09-16T20:50:33.089 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.089 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.089 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.089 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886962+0000", 2024-09-16T20:50:33.089 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8959999999999999e-06 2024-09-16T20:50:33.089 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.089 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.089 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.089 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886965+0000", 2024-09-16T20:50:33.089 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5809999999999999e-06 2024-09-16T20:50:33.089 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.090 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.090 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.090 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.090 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.090 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1608 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.090 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.866981+0000", 2024-09-16T20:50:33.090 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.022025759999999998, 2024-09-16T20:50:33.090 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.024181602, 2024-09-16T20:50:33.090 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.090 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.090 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.090 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.090 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.090 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.091 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1608 2024-09-16T20:50:33.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.091 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.866981+0000", 2024-09-16T20:50:33.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.866981+0000", 2024-09-16T20:50:33.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.092 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.867087+0000", 2024-09-16T20:50:33.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000106234 2024-09-16T20:50:33.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.092 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.092 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.092 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.867090+0000", 2024-09-16T20:50:33.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5940000000000001e-06 2024-09-16T20:50:33.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.092 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.092 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.092 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.867096+0000", 2024-09-16T20:50:33.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9630000000000003e-06 2024-09-16T20:50:33.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.093 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.093 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.093 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.867099+0000", 2024-09-16T20:50:33.093 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3680000000000002e-06 2024-09-16T20:50:33.093 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.093 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.093 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.093 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.093 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.093 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1640 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.093 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.870205+0000", 2024-09-16T20:50:33.093 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.018802142000000001, 2024-09-16T20:50:33.093 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021005253000000002, 2024-09-16T20:50:33.093 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.094 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.094 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.094 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.094 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.094 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.094 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1640 2024-09-16T20:50:33.094 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.094 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.094 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.094 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.094 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.870205+0000", 2024-09-16T20:50:33.094 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.094 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.094 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.870205+0000", 2024-09-16T20:50:33.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.095 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.870307+0000", 2024-09-16T20:50:33.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101811 2024-09-16T20:50:33.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.095 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.870310+0000", 2024-09-16T20:50:33.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2969999999999999e-06 2024-09-16T20:50:33.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.096 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.096 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.096 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.870320+0000", 2024-09-16T20:50:33.096 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0433e-05 2024-09-16T20:50:33.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.096 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.096 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.096 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.870324+0000", 2024-09-16T20:50:33.096 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.439e-06 2024-09-16T20:50:33.096 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.096 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.096 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.097 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.097 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1672 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.097 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.873456+0000", 2024-09-16T20:50:33.097 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.015551196999999999, 2024-09-16T20:50:33.097 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.017805148, 2024-09-16T20:50:33.097 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.097 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.097 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.097 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.097 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.097 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.097 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1672 2024-09-16T20:50:33.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.097 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.097 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.098 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.098 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.873456+0000", 2024-09-16T20:50:33.098 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.098 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.098 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.098 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.098 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.873456+0000", 2024-09-16T20:50:33.098 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.098 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.098 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.098 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.098 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.873545+0000", 2024-09-16T20:50:33.098 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8784999999999998e-05 2024-09-16T20:50:33.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.099 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.099 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.099 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.873547+0000", 2024-09-16T20:50:33.099 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.261e-06 2024-09-16T20:50:33.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.099 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.099 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.099 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.873552+0000", 2024-09-16T20:50:33.099 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9169999999999997e-06 2024-09-16T20:50:33.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.099 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.099 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.099 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.873555+0000", 2024-09-16T20:50:33.099 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.551e-06 2024-09-16T20:50:33.100 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.100 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.100 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.100 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.100 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.100 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1704 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.100 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.880606+0000", 2024-09-16T20:50:33.100 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.008401347, 2024-09-16T20:50:33.100 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.010684242, 2024-09-16T20:50:33.100 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.100 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.100 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.100 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.101 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.101 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.101 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1704 2024-09-16T20:50:33.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.101 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.101 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.101 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880606+0000", 2024-09-16T20:50:33.101 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.101 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.101 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880606+0000", 2024-09-16T20:50:33.102 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.102 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.102 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.102 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.102 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880606+0000", 2024-09-16T20:50:33.102 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0900000000000001e-07 2024-09-16T20:50:33.102 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.102 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.102 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.102 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880607+0000", 2024-09-16T20:50:33.102 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.052e-06 2024-09-16T20:50:33.102 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.102 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.102 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.103 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880611+0000", 2024-09-16T20:50:33.103 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6030000000000001e-06 2024-09-16T20:50:33.103 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.103 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.103 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.103 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880613+0000", 2024-09-16T20:50:33.103 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4930000000000001e-06 2024-09-16T20:50:33.103 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.103 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.103 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.103 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.103 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.103 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1736 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.104 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.880986+0000", 2024-09-16T20:50:33.104 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0080206610000000001, 2024-09-16T20:50:33.104 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.010338814, 2024-09-16T20:50:33.104 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.104 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.104 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.104 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.104 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.104 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.104 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1736 2024-09-16T20:50:33.104 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.104 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.104 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.104 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.105 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880986+0000", 2024-09-16T20:50:33.105 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.105 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.105 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880986+0000", 2024-09-16T20:50:33.105 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.105 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.105 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880987+0000", 2024-09-16T20:50:33.105 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2e-07 2024-09-16T20:50:33.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.106 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.106 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880988+0000", 2024-09-16T20:50:33.106 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4079999999999999e-06 2024-09-16T20:50:33.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.106 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.106 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880992+0000", 2024-09-16T20:50:33.106 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4319999999999999e-06 2024-09-16T20:50:33.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.106 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.106 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880994+0000", 2024-09-16T20:50:33.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.672e-06 2024-09-16T20:50:33.107 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.107 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.107 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.107 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1768 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.107 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.886587+0000", 2024-09-16T20:50:33.107 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0024195549999999999, 2024-09-16T20:50:33.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0047737489999999999, 2024-09-16T20:50:33.107 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.107 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.107 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.107 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.107 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.108 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.108 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1768 2024-09-16T20:50:33.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.108 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886587+0000", 2024-09-16T20:50:33.108 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886587+0000", 2024-09-16T20:50:33.108 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.109 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.109 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.109 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886588+0000", 2024-09-16T20:50:33.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8900000000000001e-07 2024-09-16T20:50:33.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.109 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.109 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.109 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886589+0000", 2024-09-16T20:50:33.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.15e-06 2024-09-16T20:50:33.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.109 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.109 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.109 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886592+0000", 2024-09-16T20:50:33.110 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3450000000000002e-06 2024-09-16T20:50:33.110 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.110 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.110 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.110 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886595+0000", 2024-09-16T20:50:33.110 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6630000000000002e-06 2024-09-16T20:50:33.110 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.110 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.110 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.110 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.110 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.110 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1800 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.110 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.886968+0000", 2024-09-16T20:50:33.110 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0020392909999999999, 2024-09-16T20:50:33.111 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0044434629999999999, 2024-09-16T20:50:33.111 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.111 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.111 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.111 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.111 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.111 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.111 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1800 2024-09-16T20:50:33.111 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.111 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.111 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.111 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.111 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886968+0000", 2024-09-16T20:50:33.111 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.112 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.112 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886968+0000", 2024-09-16T20:50:33.112 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.112 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.112 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886968+0000", 2024-09-16T20:50:33.112 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.17e-07 2024-09-16T20:50:33.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.112 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.113 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886969+0000", 2024-09-16T20:50:33.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2100000000000001e-06 2024-09-16T20:50:33.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.113 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.113 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.113 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886972+0000", 2024-09-16T20:50:33.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8899999999999999e-06 2024-09-16T20:50:33.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.113 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.113 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.113 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886975+0000", 2024-09-16T20:50:33.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6199999999999999e-06 2024-09-16T20:50:33.113 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.113 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.114 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.114 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.114 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1609 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.114 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.867105+0000", 2024-09-16T20:50:33.114 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.021901777000000001, 2024-09-16T20:50:33.114 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.02435493, 2024-09-16T20:50:33.114 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.114 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.114 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.114 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.114 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.114 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.115 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1609 2024-09-16T20:50:33.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.115 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.115 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.115 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.115 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.867105+0000", 2024-09-16T20:50:33.115 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.115 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.115 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.115 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.867105+0000", 2024-09-16T20:50:33.115 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.115 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.116 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.116 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.867212+0000", 2024-09-16T20:50:33.116 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000107019 2024-09-16T20:50:33.116 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.116 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.116 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.116 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.867215+0000", 2024-09-16T20:50:33.116 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4169999999999999e-06 2024-09-16T20:50:33.116 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.116 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.116 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.116 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.867220+0000", 2024-09-16T20:50:33.116 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9749999999999998e-06 2024-09-16T20:50:33.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.117 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.117 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.867223+0000", 2024-09-16T20:50:33.117 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.275e-06 2024-09-16T20:50:33.117 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.117 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.117 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.117 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1641 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.117 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.870330+0000", 2024-09-16T20:50:33.117 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.018676852000000001, 2024-09-16T20:50:33.118 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021181499999999999, 2024-09-16T20:50:33.118 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.118 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.118 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.118 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.118 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.118 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.118 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1641 2024-09-16T20:50:33.118 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.118 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.118 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.118 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.118 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.870330+0000", 2024-09-16T20:50:33.119 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.119 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.119 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.119 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.870330+0000", 2024-09-16T20:50:33.119 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.119 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.119 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.119 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.870437+0000", 2024-09-16T20:50:33.119 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000107007 2024-09-16T20:50:33.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.119 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.119 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.120 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.870440+0000", 2024-09-16T20:50:33.120 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0110000000000001e-06 2024-09-16T20:50:33.120 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.120 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.120 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.120 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.870445+0000", 2024-09-16T20:50:33.120 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0810000000000001e-06 2024-09-16T20:50:33.120 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.120 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.120 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.120 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.870449+0000", 2024-09-16T20:50:33.120 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5080000000000001e-06 2024-09-16T20:50:33.120 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.121 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.121 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.121 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.121 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.121 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1673 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.121 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.873561+0000", 2024-09-16T20:50:33.121 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.015445792999999999, 2024-09-16T20:50:33.121 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.017997741000000001, 2024-09-16T20:50:33.121 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.121 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.121 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.121 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.121 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.121 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.122 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1673 2024-09-16T20:50:33.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.122 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.122 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.122 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.122 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.873561+0000", 2024-09-16T20:50:33.122 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.122 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.122 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.122 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.873561+0000", 2024-09-16T20:50:33.122 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.122 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.123 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.123 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.873667+0000", 2024-09-16T20:50:33.123 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000105431 2024-09-16T20:50:33.123 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.123 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.123 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.123 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.873669+0000", 2024-09-16T20:50:33.123 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4200000000000001e-06 2024-09-16T20:50:33.123 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.123 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.123 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.123 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.873674+0000", 2024-09-16T20:50:33.123 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1970000000000004e-06 2024-09-16T20:50:33.123 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.124 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.124 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.124 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.873678+0000", 2024-09-16T20:50:33.124 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5130000000000001e-06 2024-09-16T20:50:33.124 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.124 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.124 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.124 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.124 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.124 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1705 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.124 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.880616+0000", 2024-09-16T20:50:33.124 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0083911530000000002, 2024-09-16T20:50:33.124 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.010987653999999999, 2024-09-16T20:50:33.124 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.125 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.125 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.125 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.125 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.125 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.125 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1705 2024-09-16T20:50:33.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.125 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.125 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.125 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.125 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880616+0000", 2024-09-16T20:50:33.125 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.126 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.126 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.126 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880616+0000", 2024-09-16T20:50:33.126 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.126 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.126 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.126 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.126 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880616+0000", 2024-09-16T20:50:33.126 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1899999999999999e-07 2024-09-16T20:50:33.126 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.126 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.126 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.126 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880617+0000", 2024-09-16T20:50:33.126 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0109999999999999e-06 2024-09-16T20:50:33.127 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.127 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.127 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.127 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880620+0000", 2024-09-16T20:50:33.127 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9849999999999998e-06 2024-09-16T20:50:33.127 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.127 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.127 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.127 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880623+0000", 2024-09-16T20:50:33.127 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4540000000000001e-06 2024-09-16T20:50:33.127 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.127 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.127 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.127 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.128 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.128 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1737 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.128 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.880998+0000", 2024-09-16T20:50:33.128 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0080094629999999997, 2024-09-16T20:50:33.128 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.010633633999999999, 2024-09-16T20:50:33.128 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.128 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.128 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.128 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.128 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.128 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.128 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1737 2024-09-16T20:50:33.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.128 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.129 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.129 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.129 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880998+0000", 2024-09-16T20:50:33.129 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.129 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.129 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.129 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.129 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880998+0000", 2024-09-16T20:50:33.129 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.129 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.129 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.129 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.129 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880998+0000", 2024-09-16T20:50:33.129 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1199999999999999e-07 2024-09-16T20:50:33.130 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.130 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.130 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.130 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880999+0000", 2024-09-16T20:50:33.130 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0109999999999999e-06 2024-09-16T20:50:33.130 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.130 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.130 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.130 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.881002+0000", 2024-09-16T20:50:33.130 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9579999999999999e-06 2024-09-16T20:50:33.130 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.130 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.130 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.130 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.881004+0000", 2024-09-16T20:50:33.131 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5749999999999999e-06 2024-09-16T20:50:33.131 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.131 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.131 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.131 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.131 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.131 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1769 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.131 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.886599+0000", 2024-09-16T20:50:33.131 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0024077289999999999, 2024-09-16T20:50:33.131 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0050580110000000003, 2024-09-16T20:50:33.131 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.131 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.131 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.132 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.132 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.132 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.132 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1769 2024-09-16T20:50:33.132 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.132 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.132 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.132 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.132 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886599+0000", 2024-09-16T20:50:33.132 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.132 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.132 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.132 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.132 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886599+0000", 2024-09-16T20:50:33.133 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.133 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.133 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886600+0000", 2024-09-16T20:50:33.133 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.29e-07 2024-09-16T20:50:33.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.133 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.133 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886601+0000", 2024-09-16T20:50:33.133 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0499999999999999e-06 2024-09-16T20:50:33.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.134 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.134 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886603+0000", 2024-09-16T20:50:33.134 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.881e-06 2024-09-16T20:50:33.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.134 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.134 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.134 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886606+0000", 2024-09-16T20:50:33.134 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5919999999999999e-06 2024-09-16T20:50:33.134 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.134 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.134 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.134 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.135 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1801 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.135 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.886977+0000", 2024-09-16T20:50:33.135 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0020295809999999999, 2024-09-16T20:50:33.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0047112630000000003, 2024-09-16T20:50:33.135 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.135 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.135 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.135 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.135 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.135 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.135 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1801 2024-09-16T20:50:33.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.136 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.136 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.136 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.136 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886977+0000", 2024-09-16T20:50:33.136 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.136 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.136 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.136 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.136 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886977+0000", 2024-09-16T20:50:33.136 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.136 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.136 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.137 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.137 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886978+0000", 2024-09-16T20:50:33.137 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0599999999999999e-07 2024-09-16T20:50:33.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.137 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.137 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.137 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886979+0000", 2024-09-16T20:50:33.137 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0499999999999999e-06 2024-09-16T20:50:33.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.137 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.137 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.137 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886981+0000", 2024-09-16T20:50:33.137 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8059999999999999e-06 2024-09-16T20:50:33.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.138 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.138 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886984+0000", 2024-09-16T20:50:33.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5189999999999999e-06 2024-09-16T20:50:33.138 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.138 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.138 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.138 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1610 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.138 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.867244+0000", 2024-09-16T20:50:33.138 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.021762782000000001, 2024-09-16T20:50:33.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.024494472999999999, 2024-09-16T20:50:33.139 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.139 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.139 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.139 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.139 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.139 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.139 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1610 2024-09-16T20:50:33.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.139 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.139 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.139 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.139 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.867244+0000", 2024-09-16T20:50:33.139 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.140 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.140 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.140 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.867244+0000", 2024-09-16T20:50:33.140 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.140 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.140 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.140 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.140 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.867245+0000", 2024-09-16T20:50:33.140 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2500000000000001e-07 2024-09-16T20:50:33.140 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.140 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.140 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.140 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.867246+0000", 2024-09-16T20:50:33.141 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7e-06 2024-09-16T20:50:33.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.141 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.141 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.141 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.867250+0000", 2024-09-16T20:50:33.141 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6480000000000001e-06 2024-09-16T20:50:33.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.141 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.141 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.141 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.867254+0000", 2024-09-16T20:50:33.141 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9879999999999998e-06 2024-09-16T20:50:33.141 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.141 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.141 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.142 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.142 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1642 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.142 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.870455+0000", 2024-09-16T20:50:33.142 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.018552099999999998, 2024-09-16T20:50:33.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021332799999999999, 2024-09-16T20:50:33.142 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.142 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.142 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.142 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.142 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.142 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.142 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1642 2024-09-16T20:50:33.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.143 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.143 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.143 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.870455+0000", 2024-09-16T20:50:33.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.143 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.143 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.870455+0000", 2024-09-16T20:50:33.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.143 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.144 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.870564+0000", 2024-09-16T20:50:33.144 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000109405 2024-09-16T20:50:33.144 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.144 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.144 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.144 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.870567+0000", 2024-09-16T20:50:33.144 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9849999999999998e-06 2024-09-16T20:50:33.144 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.144 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.144 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.144 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.870572+0000", 2024-09-16T20:50:33.144 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8600000000000001e-06 2024-09-16T20:50:33.144 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.145 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.145 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.145 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.870576+0000", 2024-09-16T20:50:33.145 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8689999999999997e-06 2024-09-16T20:50:33.145 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.145 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.145 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.145 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.145 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1674 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.145 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.873684+0000", 2024-09-16T20:50:33.145 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.015323369, 2024-09-16T20:50:33.145 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.018152325, 2024-09-16T20:50:33.145 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.146 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.146 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.146 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.146 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.146 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.146 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1674 2024-09-16T20:50:33.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.146 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.146 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.146 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.873684+0000", 2024-09-16T20:50:33.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.147 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.147 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.147 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.873684+0000", 2024-09-16T20:50:33.147 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.147 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.147 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.147 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.873791+0000", 2024-09-16T20:50:33.147 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000107555 2024-09-16T20:50:33.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.147 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.147 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.147 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.873794+0000", 2024-09-16T20:50:33.148 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3360000000000002e-06 2024-09-16T20:50:33.148 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.148 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.148 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.148 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.873799+0000", 2024-09-16T20:50:33.148 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1449999999999999e-06 2024-09-16T20:50:33.148 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.148 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.148 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.148 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.873802+0000", 2024-09-16T20:50:33.148 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5750000000000001e-06 2024-09-16T20:50:33.148 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.148 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.148 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.149 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.149 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1706 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.149 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.880625+0000", 2024-09-16T20:50:33.149 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0083816700000000008, 2024-09-16T20:50:33.149 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.011320788, 2024-09-16T20:50:33.149 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.149 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.149 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.149 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.149 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.149 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.149 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1706 2024-09-16T20:50:33.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.150 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.150 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.150 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880625+0000", 2024-09-16T20:50:33.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.150 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.150 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880625+0000", 2024-09-16T20:50:33.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.150 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880626+0000", 2024-09-16T20:50:33.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.16e-07 2024-09-16T20:50:33.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880630+0000", 2024-09-16T20:50:33.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3200000000000001e-06 2024-09-16T20:50:33.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880633+0000", 2024-09-16T20:50:33.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1949999999999999e-06 2024-09-16T20:50:33.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.152 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.152 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.152 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880636+0000", 2024-09-16T20:50:33.152 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7250000000000002e-06 2024-09-16T20:50:33.152 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.152 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.152 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.152 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.152 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1738 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.152 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.881007+0000", 2024-09-16T20:50:33.152 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0079995169999999994, 2024-09-16T20:50:33.152 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.010964911000000001, 2024-09-16T20:50:33.152 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.152 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.153 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.153 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.153 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.153 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.153 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1738 2024-09-16T20:50:33.153 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.153 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.153 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.153 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.153 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.881007+0000", 2024-09-16T20:50:33.153 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.153 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.153 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.153 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.154 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.881007+0000", 2024-09-16T20:50:33.154 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.154 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.154 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.154 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.154 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.881008+0000", 2024-09-16T20:50:33.154 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0900000000000001e-07 2024-09-16T20:50:33.154 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.154 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.154 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.154 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.881009+0000", 2024-09-16T20:50:33.154 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.001e-06 2024-09-16T20:50:33.154 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.154 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.155 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.155 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.881012+0000", 2024-09-16T20:50:33.155 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9280000000000002e-06 2024-09-16T20:50:33.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.155 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.155 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.155 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.881014+0000", 2024-09-16T20:50:33.155 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5950000000000001e-06 2024-09-16T20:50:33.155 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.155 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.155 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.155 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.156 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1770 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.156 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.886609+0000", 2024-09-16T20:50:33.156 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0023980199999999998, 2024-09-16T20:50:33.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0053918170000000001, 2024-09-16T20:50:33.156 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.156 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.156 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.156 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.156 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.156 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.156 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1770 2024-09-16T20:50:33.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.156 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.156 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.157 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.157 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886609+0000", 2024-09-16T20:50:33.157 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.157 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.157 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.157 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.157 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886609+0000", 2024-09-16T20:50:33.157 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.157 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.157 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.157 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.157 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886609+0000", 2024-09-16T20:50:33.157 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1400000000000001e-07 2024-09-16T20:50:33.158 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.158 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.158 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.158 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886610+0000", 2024-09-16T20:50:33.158 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.189e-06 2024-09-16T20:50:33.158 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.158 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.158 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.158 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886613+0000", 2024-09-16T20:50:33.158 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8480000000000001e-06 2024-09-16T20:50:33.158 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.158 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.158 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.158 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886616+0000", 2024-09-16T20:50:33.159 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.802e-06 2024-09-16T20:50:33.159 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.159 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.159 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.159 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.159 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1802 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.159 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.886988+0000", 2024-09-16T20:50:33.159 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0020185950000000002, 2024-09-16T20:50:33.159 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.005047892, 2024-09-16T20:50:33.159 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.159 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.159 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.159 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.160 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.160 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.160 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1802 2024-09-16T20:50:33.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.160 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.160 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.160 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.160 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886988+0000", 2024-09-16T20:50:33.160 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.160 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.160 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.160 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886988+0000", 2024-09-16T20:50:33.160 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.161 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.161 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.161 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.161 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886989+0000", 2024-09-16T20:50:33.161 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.05e-07 2024-09-16T20:50:33.161 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.161 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.161 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.161 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886992+0000", 2024-09-16T20:50:33.161 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6009999999999999e-06 2024-09-16T20:50:33.161 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.161 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.161 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.162 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886996+0000", 2024-09-16T20:50:33.162 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.349e-06 2024-09-16T20:50:33.162 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.162 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.162 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.162 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886998+0000", 2024-09-16T20:50:33.162 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7180000000000001e-06 2024-09-16T20:50:33.162 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.162 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.162 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.162 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.162 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.162 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1611 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.162 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.867259+0000", 2024-09-16T20:50:33.163 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.021747981999999999, 2024-09-16T20:50:33.163 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.024827370000000001, 2024-09-16T20:50:33.163 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.163 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.163 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.163 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.163 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.163 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.163 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1611 2024-09-16T20:50:33.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.163 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.163 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.163 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.163 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.867259+0000", 2024-09-16T20:50:33.164 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.164 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.164 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.164 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.867259+0000", 2024-09-16T20:50:33.164 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.164 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.164 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.164 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.867350+0000", 2024-09-16T20:50:33.164 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1182999999999998e-05 2024-09-16T20:50:33.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.164 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.165 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.165 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.867355+0000", 2024-09-16T20:50:33.165 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2259999999999996e-06 2024-09-16T20:50:33.165 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.165 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.165 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.166 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.867361+0000", 2024-09-16T20:50:33.166 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6300000000000003e-06 2024-09-16T20:50:33.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.166 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.166 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.166 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.867365+0000", 2024-09-16T20:50:33.166 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4520000000000002e-06 2024-09-16T20:50:33.166 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.166 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.166 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.166 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.166 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1643 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.167 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.870582+0000", 2024-09-16T20:50:33.167 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.018424904999999998, 2024-09-16T20:50:33.167 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021552717999999998, 2024-09-16T20:50:33.167 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.167 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.167 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.167 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.167 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.167 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.167 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1643 2024-09-16T20:50:33.167 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.167 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.167 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.167 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.168 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.870582+0000", 2024-09-16T20:50:33.168 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.168 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.168 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.168 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.870582+0000", 2024-09-16T20:50:33.168 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.168 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.168 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.168 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.870597+0000", 2024-09-16T20:50:33.168 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4756999999999999e-05 2024-09-16T20:50:33.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.168 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.169 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.169 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.870599+0000", 2024-09-16T20:50:33.169 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.733e-06 2024-09-16T20:50:33.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.169 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.169 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.169 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.870602+0000", 2024-09-16T20:50:33.169 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7699999999999999e-06 2024-09-16T20:50:33.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.169 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.169 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.169 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.870605+0000", 2024-09-16T20:50:33.169 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.061e-06 2024-09-16T20:50:33.170 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.170 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.170 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.170 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.170 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.170 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1675 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.170 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.873809+0000", 2024-09-16T20:50:33.170 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.015197784000000001, 2024-09-16T20:50:33.170 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.018377299999999999, 2024-09-16T20:50:33.170 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.170 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.170 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.170 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.170 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.171 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.171 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1675 2024-09-16T20:50:33.171 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.171 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.171 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.171 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.171 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.873809+0000", 2024-09-16T20:50:33.171 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.171 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.171 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.171 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.171 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.873809+0000", 2024-09-16T20:50:33.171 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.171 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.172 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.172 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.172 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.873920+0000", 2024-09-16T20:50:33.172 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000111202 2024-09-16T20:50:33.172 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.172 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.172 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.172 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.873923+0000", 2024-09-16T20:50:33.172 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4760000000000001e-06 2024-09-16T20:50:33.172 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.172 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.172 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.172 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.873928+0000", 2024-09-16T20:50:33.172 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9599999999999999e-06 2024-09-16T20:50:33.173 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.173 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.173 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.173 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.873932+0000", 2024-09-16T20:50:33.173 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1710000000000004e-06 2024-09-16T20:50:33.173 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.173 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.173 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.173 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.173 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.173 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1707 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.173 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.880640+0000", 2024-09-16T20:50:33.173 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0083672450000000006, 2024-09-16T20:50:33.173 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.011593897000000001, 2024-09-16T20:50:33.174 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.174 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.174 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.174 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.174 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.174 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.174 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1707 2024-09-16T20:50:33.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.174 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.174 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.174 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.174 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880640+0000", 2024-09-16T20:50:33.174 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.174 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.175 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.175 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880640+0000", 2024-09-16T20:50:33.175 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.175 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.175 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.175 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.175 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880640+0000", 2024-09-16T20:50:33.175 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4499999999999998e-07 2024-09-16T20:50:33.175 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.175 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.175 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.175 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880641+0000", 2024-09-16T20:50:33.175 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0249999999999999e-06 2024-09-16T20:50:33.175 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.175 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.176 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.176 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880644+0000", 2024-09-16T20:50:33.176 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0589999999999998e-06 2024-09-16T20:50:33.176 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.176 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.176 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.176 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880647+0000", 2024-09-16T20:50:33.176 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6869999999999999e-06 2024-09-16T20:50:33.176 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.176 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.176 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.176 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.176 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.176 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1739 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.177 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.881017+0000", 2024-09-16T20:50:33.177 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0079899180000000004, 2024-09-16T20:50:33.177 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.011261312000000001, 2024-09-16T20:50:33.177 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.177 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.177 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.177 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.177 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.177 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.177 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1739 2024-09-16T20:50:33.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.177 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.177 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.177 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.177 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.881017+0000", 2024-09-16T20:50:33.178 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.178 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.178 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.178 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.881017+0000", 2024-09-16T20:50:33.178 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.178 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.178 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.178 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.881017+0000", 2024-09-16T20:50:33.178 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3799999999999999e-07 2024-09-16T20:50:33.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.178 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.178 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.178 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.881018+0000", 2024-09-16T20:50:33.179 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8899999999999998e-07 2024-09-16T20:50:33.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.179 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.179 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.179 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.881021+0000", 2024-09-16T20:50:33.179 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8820000000000001e-06 2024-09-16T20:50:33.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.179 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.179 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.179 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.881025+0000", 2024-09-16T20:50:33.179 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.349e-06 2024-09-16T20:50:33.179 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.179 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.179 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.180 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.180 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.180 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1771 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.180 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.886619+0000", 2024-09-16T20:50:33.180 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.002388121, 2024-09-16T20:50:33.180 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0056837659999999998, 2024-09-16T20:50:33.180 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.180 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.180 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.180 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.180 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.180 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.180 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1771 2024-09-16T20:50:33.181 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.181 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.181 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.181 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.181 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886619+0000", 2024-09-16T20:50:33.181 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.181 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.181 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.181 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.181 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886619+0000", 2024-09-16T20:50:33.181 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.181 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.181 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.181 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.182 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886619+0000", 2024-09-16T20:50:33.182 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1899999999999999e-07 2024-09-16T20:50:33.182 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.182 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.182 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.182 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886620+0000", 2024-09-16T20:50:33.182 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.003e-06 2024-09-16T20:50:33.182 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.182 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.182 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.182 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886623+0000", 2024-09-16T20:50:33.182 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8710000000000001e-06 2024-09-16T20:50:33.182 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.182 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.183 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.183 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886626+0000", 2024-09-16T20:50:33.183 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.531e-06 2024-09-16T20:50:33.183 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.183 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.183 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.183 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.183 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.183 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1803 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.183 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.887001+0000", 2024-09-16T20:50:33.183 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0020055530000000002, 2024-09-16T20:50:33.183 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.005334905, 2024-09-16T20:50:33.183 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.183 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.184 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.184 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.184 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.184 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.184 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1803 2024-09-16T20:50:33.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.184 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.184 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.184 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.184 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.887001+0000", 2024-09-16T20:50:33.184 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.184 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.184 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.185 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.887001+0000", 2024-09-16T20:50:33.185 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.185 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.185 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.185 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.887002+0000", 2024-09-16T20:50:33.185 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1799999999999999e-07 2024-09-16T20:50:33.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.185 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.185 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.185 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.887003+0000", 2024-09-16T20:50:33.185 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.006e-06 2024-09-16T20:50:33.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.185 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.186 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.186 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.887006+0000", 2024-09-16T20:50:33.186 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8930000000000001e-06 2024-09-16T20:50:33.186 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.186 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.186 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.186 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.887008+0000", 2024-09-16T20:50:33.186 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6730000000000001e-06 2024-09-16T20:50:33.186 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.186 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.186 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.186 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.186 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.186 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1612 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.186 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.867371+0000", 2024-09-16T20:50:33.187 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.021636430000000002, 2024-09-16T20:50:33.187 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.025002124000000001, 2024-09-16T20:50:33.187 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.187 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.187 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.187 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.187 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.187 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.187 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1612 2024-09-16T20:50:33.187 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.187 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.187 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.187 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.187 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.867371+0000", 2024-09-16T20:50:33.188 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.188 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.188 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.188 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.188 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.867371+0000", 2024-09-16T20:50:33.188 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.188 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.188 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.188 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.188 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.867472+0000", 2024-09-16T20:50:33.188 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010186100000000001 2024-09-16T20:50:33.188 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.188 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.188 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.188 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.867475+0000", 2024-09-16T20:50:33.189 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0089999999999999e-06 2024-09-16T20:50:33.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.189 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.189 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.189 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.867481+0000", 2024-09-16T20:50:33.189 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2390000000000002e-06 2024-09-16T20:50:33.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.189 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.189 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.189 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.867484+0000", 2024-09-16T20:50:33.189 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.366e-06 2024-09-16T20:50:33.189 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.189 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.190 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.190 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.190 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1644 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.190 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.870612+0000", 2024-09-16T20:50:33.190 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.018395247, 2024-09-16T20:50:33.190 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021813788000000001, 2024-09-16T20:50:33.190 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.190 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.190 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.190 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.190 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.190 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.190 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1644 2024-09-16T20:50:33.191 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.191 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.191 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.191 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.191 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.870612+0000", 2024-09-16T20:50:33.191 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.191 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.191 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.191 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.191 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.870612+0000", 2024-09-16T20:50:33.191 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.191 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.191 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.191 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.192 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.870702+0000", 2024-09-16T20:50:33.192 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0538999999999995e-05 2024-09-16T20:50:33.192 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.192 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.192 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.192 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.870705+0000", 2024-09-16T20:50:33.192 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2350000000000002e-06 2024-09-16T20:50:33.192 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.192 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.192 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.192 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.870709+0000", 2024-09-16T20:50:33.192 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.899e-06 2024-09-16T20:50:33.192 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.193 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.193 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.193 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.870713+0000", 2024-09-16T20:50:33.193 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3720000000000001e-06 2024-09-16T20:50:33.193 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.193 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.193 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.193 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.193 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.193 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1676 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.193 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.873951+0000", 2024-09-16T20:50:33.193 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.015056316, 2024-09-16T20:50:33.193 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.018522535, 2024-09-16T20:50:33.193 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.194 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.194 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.194 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.194 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.194 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.194 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1676 2024-09-16T20:50:33.194 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.194 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.194 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.194 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.194 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.873951+0000", 2024-09-16T20:50:33.194 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.194 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.194 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.195 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.195 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.873951+0000", 2024-09-16T20:50:33.195 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.195 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.195 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.195 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.195 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.873951+0000", 2024-09-16T20:50:33.195 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3700000000000001e-07 2024-09-16T20:50:33.195 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.195 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.195 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.195 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.873953+0000", 2024-09-16T20:50:33.195 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6610000000000001e-06 2024-09-16T20:50:33.195 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.196 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.196 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.196 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.873956+0000", 2024-09-16T20:50:33.196 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.681e-06 2024-09-16T20:50:33.196 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.196 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.196 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.196 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.873960+0000", 2024-09-16T20:50:33.196 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.788e-06 2024-09-16T20:50:33.196 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.196 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.196 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.196 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.196 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.197 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1708 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.197 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.880650+0000", 2024-09-16T20:50:33.197 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0083572839999999995, 2024-09-16T20:50:33.197 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.011871395999999999, 2024-09-16T20:50:33.197 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.197 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.197 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.197 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.197 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.197 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.197 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1708 2024-09-16T20:50:33.197 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.197 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.198 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.198 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.198 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880650+0000", 2024-09-16T20:50:33.198 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.198 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.198 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.198 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.198 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880650+0000", 2024-09-16T20:50:33.198 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.198 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.198 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.198 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.198 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880650+0000", 2024-09-16T20:50:33.198 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0900000000000001e-07 2024-09-16T20:50:33.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.199 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.199 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.199 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880651+0000", 2024-09-16T20:50:33.199 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.139e-06 2024-09-16T20:50:33.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.199 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.199 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.199 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880654+0000", 2024-09-16T20:50:33.199 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9579999999999999e-06 2024-09-16T20:50:33.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.199 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.199 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.199 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880657+0000", 2024-09-16T20:50:33.200 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5409999999999999e-06 2024-09-16T20:50:33.200 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.200 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.200 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.200 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.200 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.200 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1740 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.200 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.881027+0000", 2024-09-16T20:50:33.200 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0079795879999999993, 2024-09-16T20:50:33.200 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.011544926, 2024-09-16T20:50:33.200 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.200 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.200 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.200 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.201 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.201 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.201 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1740 2024-09-16T20:50:33.201 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.201 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.201 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.201 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.201 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.881027+0000", 2024-09-16T20:50:33.201 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.201 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.201 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.201 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.201 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.881027+0000", 2024-09-16T20:50:33.201 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.202 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.202 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.202 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.202 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.881028+0000", 2024-09-16T20:50:33.202 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.17e-07 2024-09-16T20:50:33.202 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.202 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.202 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.202 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.881029+0000", 2024-09-16T20:50:33.202 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6299999999999993e-07 2024-09-16T20:50:33.202 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.202 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.202 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.202 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.881032+0000", 2024-09-16T20:50:33.203 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.914e-06 2024-09-16T20:50:33.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.203 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.203 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.203 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.881034+0000", 2024-09-16T20:50:33.203 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6709999999999999e-06 2024-09-16T20:50:33.203 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.203 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.203 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.203 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.203 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1772 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.203 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.886628+0000", 2024-09-16T20:50:33.203 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0023786110000000001, 2024-09-16T20:50:33.204 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0059697099999999996, 2024-09-16T20:50:33.204 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.204 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.204 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.204 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.204 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.204 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.204 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1772 2024-09-16T20:50:33.204 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.204 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.204 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.204 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.204 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886628+0000", 2024-09-16T20:50:33.205 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.205 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.205 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.205 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.205 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886628+0000", 2024-09-16T20:50:33.205 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.205 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.205 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.205 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.205 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886629+0000", 2024-09-16T20:50:33.205 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.11e-07 2024-09-16T20:50:33.205 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.205 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.205 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.206 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886630+0000", 2024-09-16T20:50:33.206 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0139999999999999e-06 2024-09-16T20:50:33.206 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.206 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.206 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.206 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886632+0000", 2024-09-16T20:50:33.206 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8760000000000001e-06 2024-09-16T20:50:33.206 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.206 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.206 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.206 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886635+0000", 2024-09-16T20:50:33.206 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6220000000000001e-06 2024-09-16T20:50:33.206 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.207 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.207 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.207 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.207 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.207 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1804 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.207 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.887011+0000", 2024-09-16T20:50:33.207 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0019957619999999999, 2024-09-16T20:50:33.207 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0056125639999999996, 2024-09-16T20:50:33.207 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.207 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.207 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.207 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.207 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.207 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.208 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1804 2024-09-16T20:50:33.208 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.208 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.208 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.208 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.208 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.887011+0000", 2024-09-16T20:50:33.208 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.208 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.208 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.208 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.208 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.887011+0000", 2024-09-16T20:50:33.208 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.208 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.208 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.209 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.209 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892346+0000", 2024-09-16T20:50:33.209 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0053350130000000004 2024-09-16T20:50:33.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.209 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.209 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.209 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892350+0000", 2024-09-16T20:50:33.209 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9920000000000001e-06 2024-09-16T20:50:33.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.209 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.209 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.209 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892357+0000", 2024-09-16T20:50:33.209 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2520000000000002e-06 2024-09-16T20:50:33.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.210 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.210 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.210 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892364+0000", 2024-09-16T20:50:33.210 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2940000000000002e-06 2024-09-16T20:50:33.210 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.210 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.210 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.210 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.210 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1613 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.210 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.867490+0000", 2024-09-16T20:50:33.210 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.021517242999999998, 2024-09-16T20:50:33.210 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.025160249999999999, 2024-09-16T20:50:33.210 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.211 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.211 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.211 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.211 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.211 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.211 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1613 2024-09-16T20:50:33.211 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.211 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.211 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.211 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.211 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.867490+0000", 2024-09-16T20:50:33.211 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.211 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.211 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.212 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.212 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.867490+0000", 2024-09-16T20:50:33.212 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.212 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.212 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.212 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.212 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.867596+0000", 2024-09-16T20:50:33.212 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010665799999999999 2024-09-16T20:50:33.212 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.212 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.212 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.212 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.867600+0000", 2024-09-16T20:50:33.212 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2499999999999998e-06 2024-09-16T20:50:33.213 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.213 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.213 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.213 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.867606+0000", 2024-09-16T20:50:33.213 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8619999999999999e-06 2024-09-16T20:50:33.213 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.213 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.213 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.213 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.867609+0000", 2024-09-16T20:50:33.213 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2799999999999999e-06 2024-09-16T20:50:33.213 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.213 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.213 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.214 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.214 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.214 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1645 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.214 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.870718+0000", 2024-09-16T20:50:33.214 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.018288525999999999, 2024-09-16T20:50:33.214 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021980264999999999, 2024-09-16T20:50:33.214 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.214 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.214 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.214 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.214 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.214 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.214 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1645 2024-09-16T20:50:33.214 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.215 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.215 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.215 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.215 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.870718+0000", 2024-09-16T20:50:33.215 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.215 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.215 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.215 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.215 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.870718+0000", 2024-09-16T20:50:33.215 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.215 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.215 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.215 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.215 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.870823+0000", 2024-09-16T20:50:33.216 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010412 2024-09-16T20:50:33.216 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.216 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.216 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.216 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.870826+0000", 2024-09-16T20:50:33.216 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.112e-06 2024-09-16T20:50:33.216 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.216 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.216 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.216 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.870832+0000", 2024-09-16T20:50:33.216 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6270000000000002e-06 2024-09-16T20:50:33.216 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.216 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.216 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.217 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.870836+0000", 2024-09-16T20:50:33.217 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4470000000000002e-06 2024-09-16T20:50:33.217 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.217 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.217 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.217 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.217 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.217 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1677 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.217 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.873964+0000", 2024-09-16T20:50:33.217 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.015043099000000001, 2024-09-16T20:50:33.217 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.018783500000000002, 2024-09-16T20:50:33.217 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.217 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.217 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.218 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.218 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.218 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.218 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1677 2024-09-16T20:50:33.218 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.218 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.218 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.218 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.218 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.873964+0000", 2024-09-16T20:50:33.218 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.218 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.218 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.218 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.218 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.873964+0000", 2024-09-16T20:50:33.219 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.219 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.219 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.219 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.219 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.874048+0000", 2024-09-16T20:50:33.219 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3980999999999994e-05 2024-09-16T20:50:33.219 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.219 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.219 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.219 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.874051+0000", 2024-09-16T20:50:33.219 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.039e-06 2024-09-16T20:50:33.219 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.219 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.220 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.220 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.874057+0000", 2024-09-16T20:50:33.220 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5660000000000003e-06 2024-09-16T20:50:33.220 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.220 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.220 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.220 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.874062+0000", 2024-09-16T20:50:33.220 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2749999999999997e-06 2024-09-16T20:50:33.220 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.220 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.220 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.220 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.220 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.220 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1709 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.220 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.880659+0000", 2024-09-16T20:50:33.221 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0083475589999999992, 2024-09-16T20:50:33.221 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.012138594000000001, 2024-09-16T20:50:33.221 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.221 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.221 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.221 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.221 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.221 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.221 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1709 2024-09-16T20:50:33.221 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.221 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.221 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.221 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.222 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880659+0000", 2024-09-16T20:50:33.222 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.222 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.222 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.222 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.222 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880659+0000", 2024-09-16T20:50:33.222 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.222 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.222 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.222 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.222 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880660+0000", 2024-09-16T20:50:33.222 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.05e-07 2024-09-16T20:50:33.222 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.222 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.223 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.223 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880661+0000", 2024-09-16T20:50:33.223 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0440000000000001e-06 2024-09-16T20:50:33.223 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.223 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.223 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.223 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880664+0000", 2024-09-16T20:50:33.223 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9739999999999998e-06 2024-09-16T20:50:33.223 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.223 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.223 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.223 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880669+0000", 2024-09-16T20:50:33.223 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0089999999999998e-06 2024-09-16T20:50:33.224 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.224 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.224 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.224 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.224 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.224 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1741 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.224 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.881037+0000", 2024-09-16T20:50:33.224 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0079699739999999995, 2024-09-16T20:50:33.224 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.011807615, 2024-09-16T20:50:33.224 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.224 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.224 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.224 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.224 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.225 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.225 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1741 2024-09-16T20:50:33.225 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.225 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.225 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.225 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.225 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.881037+0000", 2024-09-16T20:50:33.225 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.225 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.225 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.225 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.225 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.881037+0000", 2024-09-16T20:50:33.225 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.225 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.226 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.226 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.226 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.881037+0000", 2024-09-16T20:50:33.226 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2000000000000001e-07 2024-09-16T20:50:33.226 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.226 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.226 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.226 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.881038+0000", 2024-09-16T20:50:33.226 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8599999999999996e-07 2024-09-16T20:50:33.226 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.226 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.226 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.226 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.881041+0000", 2024-09-16T20:50:33.226 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.909e-06 2024-09-16T20:50:33.227 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.227 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.227 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.227 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.881044+0000", 2024-09-16T20:50:33.227 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6699999999999998e-06 2024-09-16T20:50:33.227 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.227 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.227 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.227 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.227 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.227 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1773 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.227 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.886648+0000", 2024-09-16T20:50:33.227 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0023590769999999998, 2024-09-16T20:50:33.227 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0062366360000000003, 2024-09-16T20:50:33.228 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.228 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.228 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.228 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.228 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.228 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.228 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1773 2024-09-16T20:50:33.228 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.228 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.228 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.228 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.228 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886648+0000", 2024-09-16T20:50:33.228 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.228 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.229 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.229 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.229 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886648+0000", 2024-09-16T20:50:33.229 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.229 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.229 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.229 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.229 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886648+0000", 2024-09-16T20:50:33.229 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-16T20:50:33.229 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.229 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.229 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.229 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886649+0000", 2024-09-16T20:50:33.230 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1790000000000001e-06 2024-09-16T20:50:33.230 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.230 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.230 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.230 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886653+0000", 2024-09-16T20:50:33.230 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.41e-06 2024-09-16T20:50:33.230 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.230 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.230 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.230 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886656+0000", 2024-09-16T20:50:33.230 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7590000000000002e-06 2024-09-16T20:50:33.230 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.230 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.230 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.231 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.231 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.231 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1805 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.231 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.892374+0000", 2024-09-16T20:50:33.231 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4294967295.9966326, 2024-09-16T20:50:33.231 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00053561199999999998, 2024-09-16T20:50:33.231 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.231 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.231 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.231 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.231 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.231 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.231 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1805 2024-09-16T20:50:33.231 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.232 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.232 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.232 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.232 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892374+0000", 2024-09-16T20:50:33.232 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.232 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.232 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.232 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.232 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892374+0000", 2024-09-16T20:50:33.232 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.232 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.232 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.232 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.232 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892375+0000", 2024-09-16T20:50:33.233 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9299999999999999e-07 2024-09-16T20:50:33.233 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.233 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.233 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.233 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892376+0000", 2024-09-16T20:50:33.233 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5030000000000001e-06 2024-09-16T20:50:33.233 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.233 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.233 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.233 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892380+0000", 2024-09-16T20:50:33.233 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7249999999999999e-06 2024-09-16T20:50:33.233 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.233 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.233 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.234 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892383+0000", 2024-09-16T20:50:33.234 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9979999999999999e-06 2024-09-16T20:50:33.234 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.234 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.234 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.234 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.234 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.234 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1614 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.234 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.867615+0000", 2024-09-16T20:50:33.234 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.021392162999999999, 2024-09-16T20:50:33.234 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.025325186999999999, 2024-09-16T20:50:33.234 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.234 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.234 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.235 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.235 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.235 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.235 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1614 2024-09-16T20:50:33.235 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.235 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.235 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.235 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.235 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.867615+0000", 2024-09-16T20:50:33.235 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.235 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.235 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.235 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.235 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.867615+0000", 2024-09-16T20:50:33.236 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.236 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.236 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.236 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.236 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.867726+0000", 2024-09-16T20:50:33.236 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000110709 2024-09-16T20:50:33.236 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.236 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.236 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.236 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.867728+0000", 2024-09-16T20:50:33.236 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4320000000000002e-06 2024-09-16T20:50:33.236 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.237 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.237 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.237 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.867733+0000", 2024-09-16T20:50:33.237 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8600000000000001e-06 2024-09-16T20:50:33.237 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.237 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.237 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.237 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.867736+0000", 2024-09-16T20:50:33.237 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.281e-06 2024-09-16T20:50:33.237 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.237 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.237 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.237 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.238 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.238 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1646 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.238 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.870842+0000", 2024-09-16T20:50:33.238 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.018165225, 2024-09-16T20:50:33.238 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.022139131999999999, 2024-09-16T20:50:33.238 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.238 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.238 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.238 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.238 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.238 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.238 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1646 2024-09-16T20:50:33.238 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.238 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.238 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.239 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.239 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.870842+0000", 2024-09-16T20:50:33.239 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.239 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.239 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.239 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.239 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.870842+0000", 2024-09-16T20:50:33.239 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.239 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.239 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.239 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.239 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.870951+0000", 2024-09-16T20:50:33.239 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000109378 2024-09-16T20:50:33.240 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.240 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.240 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.240 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.870954+0000", 2024-09-16T20:50:33.240 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3810000000000002e-06 2024-09-16T20:50:33.240 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.240 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.240 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.240 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.870959+0000", 2024-09-16T20:50:33.240 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7960000000000001e-06 2024-09-16T20:50:33.240 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.240 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.240 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.240 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.870963+0000", 2024-09-16T20:50:33.241 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3560000000000001e-06 2024-09-16T20:50:33.241 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.241 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.241 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.241 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.241 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.241 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1678 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.241 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.874068+0000", 2024-09-16T20:50:33.241 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.014939015999999999, 2024-09-16T20:50:33.241 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.018963819999999999, 2024-09-16T20:50:33.241 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.241 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.241 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.241 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.242 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.242 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.242 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1678 2024-09-16T20:50:33.242 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.242 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.242 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.242 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.242 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.874068+0000", 2024-09-16T20:50:33.242 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.242 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.242 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.242 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.242 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.874068+0000", 2024-09-16T20:50:33.242 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.243 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.243 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.243 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.243 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.874177+0000", 2024-09-16T20:50:33.243 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000109207 2024-09-16T20:50:33.243 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.243 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.243 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.243 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.874180+0000", 2024-09-16T20:50:33.243 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1580000000000001e-06 2024-09-16T20:50:33.243 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.243 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.243 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.243 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.874185+0000", 2024-09-16T20:50:33.244 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0180000000000001e-06 2024-09-16T20:50:33.244 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.244 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.244 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.244 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.874189+0000", 2024-09-16T20:50:33.244 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.484e-06 2024-09-16T20:50:33.244 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.244 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.244 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.244 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.244 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.244 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1710 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.244 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.880672+0000", 2024-09-16T20:50:33.244 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0083353260000000005, 2024-09-16T20:50:33.245 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.012407611000000001, 2024-09-16T20:50:33.245 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.245 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.245 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.245 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.245 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.245 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.245 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1710 2024-09-16T20:50:33.245 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.245 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.245 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.245 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.245 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880672+0000", 2024-09-16T20:50:33.245 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.246 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.246 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.246 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.246 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880672+0000", 2024-09-16T20:50:33.246 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.246 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.246 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.246 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.246 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880672+0000", 2024-09-16T20:50:33.246 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5100000000000001e-07 2024-09-16T20:50:33.246 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.246 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.246 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.246 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880673+0000", 2024-09-16T20:50:33.247 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0839999999999999e-06 2024-09-16T20:50:33.247 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.247 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.247 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.247 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880676+0000", 2024-09-16T20:50:33.247 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0340000000000001e-06 2024-09-16T20:50:33.247 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.247 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.247 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.247 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880679+0000", 2024-09-16T20:50:33.247 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5799999999999999e-06 2024-09-16T20:50:33.247 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.247 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.247 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.247 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.248 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.248 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1742 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.248 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.881047+0000", 2024-09-16T20:50:33.248 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0079603829999999997, 2024-09-16T20:50:33.248 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.012082529, 2024-09-16T20:50:33.248 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.248 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.248 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.248 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.248 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.248 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.248 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1742 2024-09-16T20:50:33.249 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.249 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.249 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.249 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.249 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.881047+0000", 2024-09-16T20:50:33.249 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.249 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.249 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.249 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.249 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.881047+0000", 2024-09-16T20:50:33.249 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.249 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.249 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.249 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.250 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.881047+0000", 2024-09-16T20:50:33.250 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0800000000000001e-07 2024-09-16T20:50:33.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.250 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.250 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.250 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.881051+0000", 2024-09-16T20:50:33.250 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0300000000000004e-06 2024-09-16T20:50:33.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.250 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.250 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.250 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.881054+0000", 2024-09-16T20:50:33.250 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.095e-06 2024-09-16T20:50:33.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.251 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.251 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.251 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.881057+0000", 2024-09-16T20:50:33.251 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6290000000000002e-06 2024-09-16T20:50:33.251 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.251 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.251 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.251 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.251 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.251 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1774 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.251 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.886659+0000", 2024-09-16T20:50:33.251 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0023480670000000001, 2024-09-16T20:50:33.251 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0065164589999999996, 2024-09-16T20:50:33.251 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.252 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.252 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.252 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.252 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.252 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.252 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1774 2024-09-16T20:50:33.252 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.252 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.252 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.252 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.252 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886659+0000", 2024-09-16T20:50:33.252 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.252 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.252 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.253 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.253 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886659+0000", 2024-09-16T20:50:33.253 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.253 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.253 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.253 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.253 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886659+0000", 2024-09-16T20:50:33.253 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2499999999999999e-07 2024-09-16T20:50:33.253 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.253 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.253 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.253 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886660+0000", 2024-09-16T20:50:33.253 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5199999999999995e-07 2024-09-16T20:50:33.253 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.254 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.254 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.254 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886663+0000", 2024-09-16T20:50:33.254 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0019999999999998e-06 2024-09-16T20:50:33.254 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.254 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.254 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.254 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886666+0000", 2024-09-16T20:50:33.254 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4899999999999999e-06 2024-09-16T20:50:33.254 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.254 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.254 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.254 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.254 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.255 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1806 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.255 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.892387+0000", 2024-09-16T20:50:33.255 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4294967295.9966202, 2024-09-16T20:50:33.255 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00081641099999999996, 2024-09-16T20:50:33.255 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.255 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.255 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.255 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.255 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.255 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.255 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1806 2024-09-16T20:50:33.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.255 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.255 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.256 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.256 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892387+0000", 2024-09-16T20:50:33.256 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.256 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.256 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.256 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.256 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892387+0000", 2024-09-16T20:50:33.256 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.256 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.256 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.256 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.256 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892387+0000", 2024-09-16T20:50:33.256 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4400000000000001e-07 2024-09-16T20:50:33.257 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.257 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.257 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.257 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892388+0000", 2024-09-16T20:50:33.257 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.096e-06 2024-09-16T20:50:33.257 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.257 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.257 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.257 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892391+0000", 2024-09-16T20:50:33.257 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1599999999999998e-06 2024-09-16T20:50:33.257 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.257 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.257 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.257 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892394+0000", 2024-09-16T20:50:33.258 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7429999999999998e-06 2024-09-16T20:50:33.258 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.258 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.258 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.258 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.258 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.258 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1615 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.258 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.867742+0000", 2024-09-16T20:50:33.258 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.021264946, 2024-09-16T20:50:33.258 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.025489392999999999, 2024-09-16T20:50:33.258 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.258 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.258 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.258 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.259 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.259 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.259 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1615 2024-09-16T20:50:33.259 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.259 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.259 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.259 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.259 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.867742+0000", 2024-09-16T20:50:33.259 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.259 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.259 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.259 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.259 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.867742+0000", 2024-09-16T20:50:33.260 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.260 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.260 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.260 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.260 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.867742+0000", 2024-09-16T20:50:33.260 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.84e-07 2024-09-16T20:50:33.260 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.260 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.260 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.260 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.867744+0000", 2024-09-16T20:50:33.260 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.207e-06 2024-09-16T20:50:33.260 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.260 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.260 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.261 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.867748+0000", 2024-09-16T20:50:33.261 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9519999999999996e-06 2024-09-16T20:50:33.261 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.261 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.261 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.261 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.867750+0000", 2024-09-16T20:50:33.261 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7700000000000002e-06 2024-09-16T20:50:33.261 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.261 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.261 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.261 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.261 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.261 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1647 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.262 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.870968+0000", 2024-09-16T20:50:33.262 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.018038604999999999, 2024-09-16T20:50:33.262 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.022286250000000001, 2024-09-16T20:50:33.262 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.262 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.262 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.262 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.262 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.262 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.262 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1647 2024-09-16T20:50:33.262 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.262 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.262 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.263 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.263 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.870968+0000", 2024-09-16T20:50:33.263 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.263 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.263 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.263 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.870968+0000", 2024-09-16T20:50:33.263 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.263 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.263 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.263 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.871080+0000", 2024-09-16T20:50:33.263 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000111928 2024-09-16T20:50:33.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.264 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.264 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.264 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.871086+0000", 2024-09-16T20:50:33.264 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7250000000000002e-06 2024-09-16T20:50:33.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.264 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.264 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.264 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.871093+0000", 2024-09-16T20:50:33.264 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3649999999999998e-06 2024-09-16T20:50:33.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.264 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.264 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.265 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.871097+0000", 2024-09-16T20:50:33.265 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6600000000000001e-06 2024-09-16T20:50:33.265 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.265 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.265 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.265 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.265 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.265 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1679 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.265 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.874195+0000", 2024-09-16T20:50:33.265 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.014812178, 2024-09-16T20:50:33.265 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.019093252000000002, 2024-09-16T20:50:33.265 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.265 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.266 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.266 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.266 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.266 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.266 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1679 2024-09-16T20:50:33.266 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.266 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.266 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.266 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.266 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.874195+0000", 2024-09-16T20:50:33.266 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.266 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.266 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.266 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.267 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.874195+0000", 2024-09-16T20:50:33.267 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.267 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.267 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.267 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.874307+0000", 2024-09-16T20:50:33.267 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000112184 2024-09-16T20:50:33.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.267 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.267 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.267 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.874310+0000", 2024-09-16T20:50:33.267 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.509e-06 2024-09-16T20:50:33.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.267 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.268 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.268 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.874318+0000", 2024-09-16T20:50:33.268 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8109999999999992e-06 2024-09-16T20:50:33.268 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.268 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.268 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.268 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.874322+0000", 2024-09-16T20:50:33.268 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.315e-06 2024-09-16T20:50:33.268 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.268 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.268 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.268 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.268 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.269 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1711 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.269 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.880688+0000", 2024-09-16T20:50:33.269 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0083192460000000006, 2024-09-16T20:50:33.269 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.012652180000000001, 2024-09-16T20:50:33.269 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.269 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.269 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.269 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.269 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.269 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.269 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1711 2024-09-16T20:50:33.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.269 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.269 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.270 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.270 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880688+0000", 2024-09-16T20:50:33.270 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.270 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.270 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.270 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.270 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880688+0000", 2024-09-16T20:50:33.270 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.270 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.270 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.270 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.270 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880688+0000", 2024-09-16T20:50:33.270 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3999999999999998e-07 2024-09-16T20:50:33.270 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.271 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.271 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.271 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880689+0000", 2024-09-16T20:50:33.271 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1790000000000001e-06 2024-09-16T20:50:33.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.271 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.271 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.271 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880693+0000", 2024-09-16T20:50:33.271 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4369999999999999e-06 2024-09-16T20:50:33.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.271 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.271 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.271 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880695+0000", 2024-09-16T20:50:33.271 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7240000000000001e-06 2024-09-16T20:50:33.272 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.272 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.272 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.272 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.272 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.272 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1743 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.272 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.881060+0000", 2024-09-16T20:50:33.272 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0079474799999999998, 2024-09-16T20:50:33.272 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.012331302000000001, 2024-09-16T20:50:33.272 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.272 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.272 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.272 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.272 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.273 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.273 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1743 2024-09-16T20:50:33.273 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.273 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.273 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.273 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.273 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.881060+0000", 2024-09-16T20:50:33.273 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.273 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.273 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.273 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.273 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.881060+0000", 2024-09-16T20:50:33.273 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.274 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.274 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.274 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.274 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.881060+0000", 2024-09-16T20:50:33.274 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3900000000000001e-07 2024-09-16T20:50:33.274 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.274 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.274 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.274 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.881061+0000", 2024-09-16T20:50:33.274 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.031e-06 2024-09-16T20:50:33.274 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.274 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.274 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.274 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.881064+0000", 2024-09-16T20:50:33.275 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9409999999999999e-06 2024-09-16T20:50:33.275 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.275 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.275 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.275 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.881066+0000", 2024-09-16T20:50:33.275 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6910000000000002e-06 2024-09-16T20:50:33.275 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.275 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.275 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.275 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.275 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.275 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1775 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.275 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.886668+0000", 2024-09-16T20:50:33.275 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0023385509999999999, 2024-09-16T20:50:33.276 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0067700939999999999, 2024-09-16T20:50:33.276 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.276 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.276 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.276 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.276 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.276 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.276 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1775 2024-09-16T20:50:33.276 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.276 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.276 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.276 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.276 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886668+0000", 2024-09-16T20:50:33.276 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.277 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.277 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.277 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.277 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886668+0000", 2024-09-16T20:50:33.277 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.277 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.277 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.277 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.277 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886669+0000", 2024-09-16T20:50:33.277 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1199999999999999e-07 2024-09-16T20:50:33.278 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.278 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.278 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.278 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886670+0000", 2024-09-16T20:50:33.278 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0070000000000001e-06 2024-09-16T20:50:33.278 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.278 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.278 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.278 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886673+0000", 2024-09-16T20:50:33.278 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9639999999999999e-06 2024-09-16T20:50:33.278 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.278 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.278 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.279 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886675+0000", 2024-09-16T20:50:33.279 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4200000000000001e-06 2024-09-16T20:50:33.279 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.279 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.279 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.279 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.279 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.279 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1807 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.279 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.892397+0000", 2024-09-16T20:50:33.279 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4294967295.9966097, 2024-09-16T20:50:33.279 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010924750000000001, 2024-09-16T20:50:33.279 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.279 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.279 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.280 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.280 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.280 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.280 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1807 2024-09-16T20:50:33.280 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.280 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.280 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.280 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.280 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892397+0000", 2024-09-16T20:50:33.280 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.280 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.280 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.280 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.280 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892397+0000", 2024-09-16T20:50:33.281 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.281 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.281 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.281 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.281 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892398+0000", 2024-09-16T20:50:33.281 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.15e-07 2024-09-16T20:50:33.281 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.281 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.281 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.281 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892399+0000", 2024-09-16T20:50:33.281 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.313e-06 2024-09-16T20:50:33.281 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.281 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.282 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.282 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892402+0000", 2024-09-16T20:50:33.282 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2059999999999999e-06 2024-09-16T20:50:33.282 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.282 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.282 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.282 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892405+0000", 2024-09-16T20:50:33.282 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6000000000000001e-06 2024-09-16T20:50:33.282 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.282 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.282 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.282 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.282 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.282 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1616 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.283 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.867759+0000", 2024-09-16T20:50:33.283 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.021247972, 2024-09-16T20:50:33.283 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.02576589, 2024-09-16T20:50:33.283 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.283 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.283 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.283 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.283 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.283 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.283 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1616 2024-09-16T20:50:33.283 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.283 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.283 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.284 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.284 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.867759+0000", 2024-09-16T20:50:33.284 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.284 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.284 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.284 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.284 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.867759+0000", 2024-09-16T20:50:33.284 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.284 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.284 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.284 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.284 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.867856+0000", 2024-09-16T20:50:33.284 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7057000000000003e-05 2024-09-16T20:50:33.285 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.285 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.285 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.285 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.867859+0000", 2024-09-16T20:50:33.285 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.242e-06 2024-09-16T20:50:33.285 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.285 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.285 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.285 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.867866+0000", 2024-09-16T20:50:33.285 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0079999999999996e-06 2024-09-16T20:50:33.285 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.285 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.285 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.286 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.867870+0000", 2024-09-16T20:50:33.286 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5300000000000001e-06 2024-09-16T20:50:33.286 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.286 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.286 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.286 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.286 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.286 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1648 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.286 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.871107+0000", 2024-09-16T20:50:33.286 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.017899693000000001, 2024-09-16T20:50:33.286 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.022444136, 2024-09-16T20:50:33.286 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.286 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.287 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.287 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.287 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.287 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.287 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1648 2024-09-16T20:50:33.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.287 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.287 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.287 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.287 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.871107+0000", 2024-09-16T20:50:33.287 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.287 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.287 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.288 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.871107+0000", 2024-09-16T20:50:33.288 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.288 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.288 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.288 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.288 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.871108+0000", 2024-09-16T20:50:33.288 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7500000000000001e-07 2024-09-16T20:50:33.288 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.288 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.288 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.288 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.871109+0000", 2024-09-16T20:50:33.288 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.279e-06 2024-09-16T20:50:33.288 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.288 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.289 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.289 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.871112+0000", 2024-09-16T20:50:33.289 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3210000000000001e-06 2024-09-16T20:50:33.289 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.289 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.289 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.289 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.871115+0000", 2024-09-16T20:50:33.289 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8459999999999999e-06 2024-09-16T20:50:33.289 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.289 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.289 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.289 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.289 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.289 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1680 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.290 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.874328+0000", 2024-09-16T20:50:33.290 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.014679183, 2024-09-16T20:50:33.290 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.019249457000000001, 2024-09-16T20:50:33.290 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.290 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.290 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.290 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.290 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.290 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.290 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1680 2024-09-16T20:50:33.290 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.290 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.290 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.290 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.291 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.874328+0000", 2024-09-16T20:50:33.291 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.291 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.291 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.291 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.291 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.874328+0000", 2024-09-16T20:50:33.291 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.291 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.291 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.291 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.291 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.874435+0000", 2024-09-16T20:50:33.291 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000107423 2024-09-16T20:50:33.291 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.291 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.292 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.292 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.874438+0000", 2024-09-16T20:50:33.292 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.914e-06 2024-09-16T20:50:33.292 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.292 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.292 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.292 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.874443+0000", 2024-09-16T20:50:33.292 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0259999999999998e-06 2024-09-16T20:50:33.292 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.292 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.292 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.292 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.874447+0000", 2024-09-16T20:50:33.293 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.326e-06 2024-09-16T20:50:33.293 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.293 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.293 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.293 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.293 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.293 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1712 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.293 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.880700+0000", 2024-09-16T20:50:33.293 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0083071699999999991, 2024-09-16T20:50:33.293 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.012901759, 2024-09-16T20:50:33.293 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.293 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.293 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.293 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.294 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.294 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.294 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1712 2024-09-16T20:50:33.294 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.294 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.294 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.294 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.294 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880700+0000", 2024-09-16T20:50:33.294 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.294 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.294 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.294 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.294 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880700+0000", 2024-09-16T20:50:33.294 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.295 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.295 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.295 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.295 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880700+0000", 2024-09-16T20:50:33.295 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1199999999999999e-07 2024-09-16T20:50:33.295 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.295 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.295 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.295 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880701+0000", 2024-09-16T20:50:33.295 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0139999999999999e-06 2024-09-16T20:50:33.295 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.295 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.295 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.296 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880704+0000", 2024-09-16T20:50:33.296 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0130000000000002e-06 2024-09-16T20:50:33.296 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.296 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.296 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.296 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880707+0000", 2024-09-16T20:50:33.296 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.553e-06 2024-09-16T20:50:33.296 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.296 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.296 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.296 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.296 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.296 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1744 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.296 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.881083+0000", 2024-09-16T20:50:33.297 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0079241999999999993, 2024-09-16T20:50:33.297 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.012562386, 2024-09-16T20:50:33.297 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.297 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.297 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.297 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.297 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.297 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.297 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1744 2024-09-16T20:50:33.297 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.297 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.297 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.297 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.298 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.881083+0000", 2024-09-16T20:50:33.298 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.298 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.298 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.298 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.298 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.881083+0000", 2024-09-16T20:50:33.298 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.298 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.298 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.298 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.298 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.881083+0000", 2024-09-16T20:50:33.298 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0999999999999999e-07 2024-09-16T20:50:33.298 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.298 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.299 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.299 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.881085+0000", 2024-09-16T20:50:33.299 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3510000000000001e-06 2024-09-16T20:50:33.299 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.299 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.299 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.299 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.881088+0000", 2024-09-16T20:50:33.299 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6200000000000001e-06 2024-09-16T20:50:33.299 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.299 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.299 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.299 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.881091+0000", 2024-09-16T20:50:33.299 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5689999999999999e-06 2024-09-16T20:50:33.299 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.300 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.300 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.300 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.300 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1776 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.300 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.886678+0000", 2024-09-16T20:50:33.300 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0023290160000000002, 2024-09-16T20:50:33.300 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0070195939999999997, 2024-09-16T20:50:33.300 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.300 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.300 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.300 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.300 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.301 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.301 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1776 2024-09-16T20:50:33.301 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.301 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.301 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.301 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.301 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886678+0000", 2024-09-16T20:50:33.301 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.301 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.301 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.301 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.301 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886678+0000", 2024-09-16T20:50:33.301 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.301 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.302 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.302 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.302 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886678+0000", 2024-09-16T20:50:33.302 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0699999999999999e-07 2024-09-16T20:50:33.302 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.302 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.302 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.302 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886679+0000", 2024-09-16T20:50:33.302 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.78e-07 2024-09-16T20:50:33.302 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.302 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.302 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.302 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886682+0000", 2024-09-16T20:50:33.302 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8760000000000001e-06 2024-09-16T20:50:33.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.303 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.303 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.303 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886685+0000", 2024-09-16T20:50:33.303 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4760000000000001e-06 2024-09-16T20:50:33.303 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.303 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.303 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.303 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.303 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1808 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.303 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.892408+0000", 2024-09-16T20:50:33.303 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4294967295.9965992, 2024-09-16T20:50:33.304 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0013383519999999999, 2024-09-16T20:50:33.304 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.304 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.304 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.304 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.304 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.304 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.304 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1808 2024-09-16T20:50:33.304 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.304 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.304 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.304 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.304 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892408+0000", 2024-09-16T20:50:33.304 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.305 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.305 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892408+0000", 2024-09-16T20:50:33.305 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.305 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.305 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892408+0000", 2024-09-16T20:50:33.305 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3300000000000001e-07 2024-09-16T20:50:33.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.305 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.305 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892409+0000", 2024-09-16T20:50:33.306 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1209999999999999e-06 2024-09-16T20:50:33.306 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.306 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.306 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.306 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892412+0000", 2024-09-16T20:50:33.306 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.982e-06 2024-09-16T20:50:33.306 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.306 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.306 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.306 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892415+0000", 2024-09-16T20:50:33.306 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5900000000000002e-06 2024-09-16T20:50:33.306 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.306 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.306 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.307 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.307 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1617 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.307 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.867877+0000", 2024-09-16T20:50:33.307 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.021129933, 2024-09-16T20:50:33.307 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.025918983999999999, 2024-09-16T20:50:33.307 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.307 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.307 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.307 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.307 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.307 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.307 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1617 2024-09-16T20:50:33.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.308 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.308 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.308 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.308 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.867877+0000", 2024-09-16T20:50:33.308 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.308 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.308 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.308 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.867877+0000", 2024-09-16T20:50:33.308 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.308 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.309 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.309 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.867985+0000", 2024-09-16T20:50:33.309 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000108103 2024-09-16T20:50:33.309 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.309 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.309 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.309 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.867988+0000", 2024-09-16T20:50:33.309 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4540000000000001e-06 2024-09-16T20:50:33.309 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.309 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.309 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.309 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.867993+0000", 2024-09-16T20:50:33.309 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0769999999999998e-06 2024-09-16T20:50:33.309 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.310 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.310 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.310 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.867996+0000", 2024-09-16T20:50:33.310 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3610000000000001e-06 2024-09-16T20:50:33.310 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.310 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.310 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.310 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.310 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.310 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1649 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.310 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.871118+0000", 2024-09-16T20:50:33.310 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.01788851, 2024-09-16T20:50:33.310 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.022723841000000002, 2024-09-16T20:50:33.310 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.311 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.311 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.311 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.311 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.311 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.311 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1649 2024-09-16T20:50:33.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.311 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.311 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.311 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.311 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.871118+0000", 2024-09-16T20:50:33.311 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.311 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.312 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.312 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.871118+0000", 2024-09-16T20:50:33.312 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.312 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.312 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.312 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.312 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.871215+0000", 2024-09-16T20:50:33.312 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6463000000000001e-05 2024-09-16T20:50:33.312 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.312 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.312 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.312 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.871218+0000", 2024-09-16T20:50:33.312 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.667e-06 2024-09-16T20:50:33.312 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.313 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.313 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.313 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.871223+0000", 2024-09-16T20:50:33.313 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2429999999999996e-06 2024-09-16T20:50:33.313 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.313 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.313 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.313 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.871226+0000", 2024-09-16T20:50:33.313 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4699999999999998e-06 2024-09-16T20:50:33.313 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.313 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.313 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.313 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.313 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.314 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1681 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.314 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.874453+0000", 2024-09-16T20:50:33.314 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.014554433, 2024-09-16T20:50:33.314 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.019413858999999999, 2024-09-16T20:50:33.314 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.314 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.314 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.314 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.314 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.314 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.314 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1681 2024-09-16T20:50:33.314 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.314 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.314 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.315 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.315 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.874453+0000", 2024-09-16T20:50:33.315 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.315 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.315 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.315 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.315 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.874453+0000", 2024-09-16T20:50:33.315 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.315 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.315 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.315 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.315 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.874567+0000", 2024-09-16T20:50:33.315 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011440800000000001 2024-09-16T20:50:33.315 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.316 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.316 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.316 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.874570+0000", 2024-09-16T20:50:33.316 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.807e-06 2024-09-16T20:50:33.316 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.316 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.316 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.316 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.874575+0000", 2024-09-16T20:50:33.316 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4110000000000004e-06 2024-09-16T20:50:33.316 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.316 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.316 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.316 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.874579+0000", 2024-09-16T20:50:33.317 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5030000000000002e-06 2024-09-16T20:50:33.317 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.317 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.317 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.317 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.317 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1713 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.317 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.880710+0000", 2024-09-16T20:50:33.317 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0082965469999999996, 2024-09-16T20:50:33.317 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.013181495, 2024-09-16T20:50:33.317 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.317 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.317 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.318 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.318 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.318 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.318 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1713 2024-09-16T20:50:33.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.318 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.318 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.318 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.318 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880710+0000", 2024-09-16T20:50:33.318 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.318 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.318 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.319 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880710+0000", 2024-09-16T20:50:33.319 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.319 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.319 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.319 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880711+0000", 2024-09-16T20:50:33.319 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0800000000000001e-07 2024-09-16T20:50:33.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.319 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.319 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.319 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880712+0000", 2024-09-16T20:50:33.319 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9999999999999995e-07 2024-09-16T20:50:33.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.319 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.320 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.320 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880715+0000", 2024-09-16T20:50:33.320 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9629999999999998e-06 2024-09-16T20:50:33.320 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.320 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.320 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.320 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880717+0000", 2024-09-16T20:50:33.320 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5170000000000002e-06 2024-09-16T20:50:33.320 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.320 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.320 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.320 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.320 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.320 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1745 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.321 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.881094+0000", 2024-09-16T20:50:33.321 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0079131800000000006, 2024-09-16T20:50:33.321 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.012821447, 2024-09-16T20:50:33.321 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.321 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.321 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.321 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.321 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.321 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.321 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1745 2024-09-16T20:50:33.321 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.321 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.321 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.322 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.322 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.881094+0000", 2024-09-16T20:50:33.322 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.322 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.322 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.322 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.881094+0000", 2024-09-16T20:50:33.322 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.322 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.322 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.322 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.881094+0000", 2024-09-16T20:50:33.322 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0699999999999998e-07 2024-09-16T20:50:33.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.323 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.323 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.323 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.881095+0000", 2024-09-16T20:50:33.323 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0699999999999999e-06 2024-09-16T20:50:33.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.323 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.323 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.323 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.881098+0000", 2024-09-16T20:50:33.323 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9579999999999999e-06 2024-09-16T20:50:33.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.323 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.323 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.324 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.881101+0000", 2024-09-16T20:50:33.324 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5610000000000001e-06 2024-09-16T20:50:33.324 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.324 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.324 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.324 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.324 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.324 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1777 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.324 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.886687+0000", 2024-09-16T20:50:33.324 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.002319607, 2024-09-16T20:50:33.324 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.007259677, 2024-09-16T20:50:33.324 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.324 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.324 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.325 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.325 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.325 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.325 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1777 2024-09-16T20:50:33.325 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.325 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.325 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.325 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.325 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886687+0000", 2024-09-16T20:50:33.325 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.325 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.325 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.325 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.326 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886687+0000", 2024-09-16T20:50:33.326 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.326 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.326 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.326 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.326 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886688+0000", 2024-09-16T20:50:33.326 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1799999999999999e-07 2024-09-16T20:50:33.326 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.326 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.326 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.326 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886689+0000", 2024-09-16T20:50:33.326 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1699999999999997e-07 2024-09-16T20:50:33.326 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.326 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.327 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.327 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886691+0000", 2024-09-16T20:50:33.327 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8650000000000001e-06 2024-09-16T20:50:33.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.327 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.327 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.327 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886694+0000", 2024-09-16T20:50:33.327 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.525e-06 2024-09-16T20:50:33.327 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.327 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.327 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.327 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.327 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1809 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.328 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.892418+0000", 2024-09-16T20:50:33.328 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4294967295.9965892, 2024-09-16T20:50:33.328 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0015768990000000001, 2024-09-16T20:50:33.328 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.328 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.328 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.328 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.328 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.328 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.328 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1809 2024-09-16T20:50:33.328 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.328 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.328 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.329 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.329 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892418+0000", 2024-09-16T20:50:33.329 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.329 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.329 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.329 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.329 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892418+0000", 2024-09-16T20:50:33.329 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.329 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.329 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.329 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.329 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892418+0000", 2024-09-16T20:50:33.329 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1799999999999999e-07 2024-09-16T20:50:33.330 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.330 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.330 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.330 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892419+0000", 2024-09-16T20:50:33.330 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.096e-06 2024-09-16T20:50:33.330 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.330 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.330 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.330 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892422+0000", 2024-09-16T20:50:33.330 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0189999999999998e-06 2024-09-16T20:50:33.330 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.330 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.330 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.330 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892425+0000", 2024-09-16T20:50:33.331 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6369999999999999e-06 2024-09-16T20:50:33.331 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.331 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.331 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.331 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.331 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.331 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1618 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.331 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.868002+0000", 2024-09-16T20:50:33.331 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.021004886, 2024-09-16T20:50:33.331 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.026045499999999999, 2024-09-16T20:50:33.331 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.331 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.331 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.332 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.332 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.332 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.332 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1618 2024-09-16T20:50:33.332 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.332 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.332 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.332 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.332 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.868002+0000", 2024-09-16T20:50:33.332 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.332 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.332 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.332 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.333 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.868002+0000", 2024-09-16T20:50:33.333 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.333 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.333 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.333 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.333 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.868114+0000", 2024-09-16T20:50:33.333 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000111557 2024-09-16T20:50:33.333 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.333 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.333 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.333 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.868116+0000", 2024-09-16T20:50:33.333 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3740000000000001e-06 2024-09-16T20:50:33.333 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.334 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.334 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.334 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.868121+0000", 2024-09-16T20:50:33.334 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.95e-06 2024-09-16T20:50:33.334 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.334 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.334 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.334 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.868124+0000", 2024-09-16T20:50:33.334 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3160000000000001e-06 2024-09-16T20:50:33.334 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.334 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.334 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.334 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.335 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.335 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1650 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.335 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.871232+0000", 2024-09-16T20:50:33.335 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.017774636999999999, 2024-09-16T20:50:33.335 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.022863083999999999, 2024-09-16T20:50:33.335 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.335 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.335 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.335 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.335 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.335 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.335 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1650 2024-09-16T20:50:33.335 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.336 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.336 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.336 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.336 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.871232+0000", 2024-09-16T20:50:33.336 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.336 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.336 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.336 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.871232+0000", 2024-09-16T20:50:33.336 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.336 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.336 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.336 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.871339+0000", 2024-09-16T20:50:33.337 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000106502 2024-09-16T20:50:33.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.337 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.337 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.337 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.871342+0000", 2024-09-16T20:50:33.337 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9610000000000001e-06 2024-09-16T20:50:33.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.337 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.337 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.337 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.871347+0000", 2024-09-16T20:50:33.337 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2599999999999996e-06 2024-09-16T20:50:33.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.338 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.338 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.338 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.871350+0000", 2024-09-16T20:50:33.338 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3699999999999999e-06 2024-09-16T20:50:33.338 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.338 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.338 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.338 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.338 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.338 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1682 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.338 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.874585+0000", 2024-09-16T20:50:33.338 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.014422394999999999, 2024-09-16T20:50:33.338 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.019560827999999999, 2024-09-16T20:50:33.339 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.339 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.339 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.339 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.339 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.339 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.339 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1682 2024-09-16T20:50:33.339 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.339 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.339 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.339 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.339 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.874585+0000", 2024-09-16T20:50:33.339 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.339 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.340 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.340 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.340 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.874585+0000", 2024-09-16T20:50:33.340 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.340 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.340 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.340 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.340 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.874585+0000", 2024-09-16T20:50:33.340 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0899999999999997e-07 2024-09-16T20:50:33.340 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.340 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.340 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.340 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.874586+0000", 2024-09-16T20:50:33.341 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.144e-06 2024-09-16T20:50:33.341 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.341 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.341 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.341 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.874589+0000", 2024-09-16T20:50:33.341 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.174e-06 2024-09-16T20:50:33.341 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.341 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.341 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.341 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.874592+0000", 2024-09-16T20:50:33.341 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7539999999999998e-06 2024-09-16T20:50:33.341 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.341 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.342 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.342 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.342 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.342 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1714 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.342 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.880720+0000", 2024-09-16T20:50:33.342 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0082868179999999996, 2024-09-16T20:50:33.342 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.013460468999999999, 2024-09-16T20:50:33.342 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.342 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.342 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.342 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.342 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.342 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.342 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1714 2024-09-16T20:50:33.343 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.343 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.343 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.343 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.343 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880720+0000", 2024-09-16T20:50:33.343 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.343 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.343 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.343 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.343 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880720+0000", 2024-09-16T20:50:33.343 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.343 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.343 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.344 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.344 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880720+0000", 2024-09-16T20:50:33.344 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1e-07 2024-09-16T20:50:33.344 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.344 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.344 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.344 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880721+0000", 2024-09-16T20:50:33.344 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.071e-06 2024-09-16T20:50:33.344 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.344 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.344 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.344 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880724+0000", 2024-09-16T20:50:33.344 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9160000000000001e-06 2024-09-16T20:50:33.344 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.345 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.345 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.345 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880727+0000", 2024-09-16T20:50:33.345 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.491e-06 2024-09-16T20:50:33.345 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.345 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.345 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.345 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.345 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.345 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1746 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.345 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.881104+0000", 2024-09-16T20:50:33.345 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0079034910000000003, 2024-09-16T20:50:33.345 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.013103294999999999, 2024-09-16T20:50:33.345 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.346 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.346 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.346 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.346 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.346 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.346 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1746 2024-09-16T20:50:33.346 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.346 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.346 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.346 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.346 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.881104+0000", 2024-09-16T20:50:33.346 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.346 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.346 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.347 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.347 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.881104+0000", 2024-09-16T20:50:33.347 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.347 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.347 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.347 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.347 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.881104+0000", 2024-09-16T20:50:33.347 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0599999999999999e-07 2024-09-16T20:50:33.347 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.347 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.347 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.347 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.881105+0000", 2024-09-16T20:50:33.347 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5099999999999998e-07 2024-09-16T20:50:33.348 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.348 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.348 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.348 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.881108+0000", 2024-09-16T20:50:33.348 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8890000000000002e-06 2024-09-16T20:50:33.348 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.348 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.348 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.348 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.881110+0000", 2024-09-16T20:50:33.348 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4839999999999998e-06 2024-09-16T20:50:33.348 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.348 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.348 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.349 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.349 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1778 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.349 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.886697+0000", 2024-09-16T20:50:33.349 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0023103210000000002, 2024-09-16T20:50:33.349 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0075329810000000002, 2024-09-16T20:50:33.349 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.349 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.349 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.349 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.349 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.349 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.349 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1778 2024-09-16T20:50:33.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.350 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.350 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.350 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.350 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886697+0000", 2024-09-16T20:50:33.350 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.350 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.350 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.350 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.350 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886697+0000", 2024-09-16T20:50:33.350 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.350 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.350 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.350 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.351 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886697+0000", 2024-09-16T20:50:33.351 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.05e-07 2024-09-16T20:50:33.351 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.351 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.351 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.351 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886701+0000", 2024-09-16T20:50:33.351 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7740000000000002e-06 2024-09-16T20:50:33.351 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.351 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.351 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.351 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886704+0000", 2024-09-16T20:50:33.351 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7000000000000002e-06 2024-09-16T20:50:33.351 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.351 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.351 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.352 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886707+0000", 2024-09-16T20:50:33.352 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5749999999999999e-06 2024-09-16T20:50:33.352 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.352 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.352 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.352 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.352 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.352 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1810 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.352 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.892428+0000", 2024-09-16T20:50:33.352 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4294967295.9965792, 2024-09-16T20:50:33.352 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018279119999999999, 2024-09-16T20:50:33.352 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.352 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.353 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.353 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.353 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.353 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.353 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1810 2024-09-16T20:50:33.353 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.353 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.353 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.353 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.353 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892428+0000", 2024-09-16T20:50:33.353 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.353 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.353 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.354 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.354 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892428+0000", 2024-09-16T20:50:33.354 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.354 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.354 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.354 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.354 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892428+0000", 2024-09-16T20:50:33.354 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1500000000000001e-07 2024-09-16T20:50:33.354 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.354 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.354 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.354 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892429+0000", 2024-09-16T20:50:33.354 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.054e-06 2024-09-16T20:50:33.355 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.355 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.355 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.355 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892433+0000", 2024-09-16T20:50:33.355 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0389999999999998e-06 2024-09-16T20:50:33.355 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.355 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.355 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.355 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892436+0000", 2024-09-16T20:50:33.355 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.712e-06 2024-09-16T20:50:33.355 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.355 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.355 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.355 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.356 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.356 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1619 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.356 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.868130+0000", 2024-09-16T20:50:33.356 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.020876784999999998, 2024-09-16T20:50:33.356 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.026173595000000001, 2024-09-16T20:50:33.356 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.356 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.356 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.356 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.356 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.356 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.356 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1619 2024-09-16T20:50:33.356 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.357 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.357 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.357 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.357 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.868130+0000", 2024-09-16T20:50:33.357 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.357 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.357 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.357 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.357 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.868130+0000", 2024-09-16T20:50:33.357 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.357 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.357 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.357 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.358 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.868241+0000", 2024-09-16T20:50:33.358 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000111018 2024-09-16T20:50:33.358 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.358 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.358 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.358 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.868244+0000", 2024-09-16T20:50:33.358 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6769999999999999e-06 2024-09-16T20:50:33.358 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.358 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.358 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.358 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.868249+0000", 2024-09-16T20:50:33.358 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0200000000000002e-06 2024-09-16T20:50:33.358 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.358 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.359 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.359 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.868253+0000", 2024-09-16T20:50:33.359 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5930000000000002e-06 2024-09-16T20:50:33.359 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.359 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.359 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.359 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.359 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.359 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1651 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.359 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.871356+0000", 2024-09-16T20:50:33.359 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.017650724, 2024-09-16T20:50:33.359 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.022993573999999999, 2024-09-16T20:50:33.359 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.360 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.360 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.360 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.360 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.360 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.360 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1651 2024-09-16T20:50:33.360 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.360 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.360 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.360 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.360 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.871356+0000", 2024-09-16T20:50:33.360 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.360 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.360 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.361 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.361 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.871356+0000", 2024-09-16T20:50:33.361 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.361 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.361 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.361 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.361 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.871469+0000", 2024-09-16T20:50:33.361 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000112524 2024-09-16T20:50:33.361 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.361 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.361 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.361 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.871471+0000", 2024-09-16T20:50:33.361 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3180000000000001e-06 2024-09-16T20:50:33.362 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.362 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.362 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.362 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.871476+0000", 2024-09-16T20:50:33.362 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9869999999999999e-06 2024-09-16T20:50:33.362 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.362 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.362 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.362 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.871479+0000", 2024-09-16T20:50:33.362 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2430000000000001e-06 2024-09-16T20:50:33.362 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.362 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.362 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.362 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.363 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.363 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1683 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.363 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.874598+0000", 2024-09-16T20:50:33.363 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.014408804000000001, 2024-09-16T20:50:33.363 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.019777955999999999, 2024-09-16T20:50:33.363 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.363 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.363 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.363 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.363 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.363 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.363 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1683 2024-09-16T20:50:33.363 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.363 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.364 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.364 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.364 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.874598+0000", 2024-09-16T20:50:33.364 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.364 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.364 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.364 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.364 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.874598+0000", 2024-09-16T20:50:33.364 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.364 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.364 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.364 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.364 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880361+0000", 2024-09-16T20:50:33.364 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057630390000000002 2024-09-16T20:50:33.365 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.365 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.365 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.365 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880365+0000", 2024-09-16T20:50:33.365 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9890000000000003e-06 2024-09-16T20:50:33.365 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.365 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.365 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.365 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880374+0000", 2024-09-16T20:50:33.365 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4239999999999993e-06 2024-09-16T20:50:33.365 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.365 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.365 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.365 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880378+0000", 2024-09-16T20:50:33.366 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2239999999999997e-06 2024-09-16T20:50:33.366 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.366 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.366 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.366 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.366 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.366 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1715 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.366 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.880730+0000", 2024-09-16T20:50:33.366 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0082771110000000002, 2024-09-16T20:50:33.366 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.013669005999999999, 2024-09-16T20:50:33.366 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.366 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.366 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.366 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.367 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.367 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.367 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1715 2024-09-16T20:50:33.367 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.367 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.367 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.367 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.367 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880730+0000", 2024-09-16T20:50:33.367 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.367 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.367 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.367 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.368 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880730+0000", 2024-09-16T20:50:33.368 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.368 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.368 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.368 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.368 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880730+0000", 2024-09-16T20:50:33.368 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0699999999999999e-07 2024-09-16T20:50:33.368 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.368 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.368 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.368 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880731+0000", 2024-09-16T20:50:33.368 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0189999999999999e-06 2024-09-16T20:50:33.368 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.369 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.369 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.369 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880734+0000", 2024-09-16T20:50:33.369 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.914e-06 2024-09-16T20:50:33.369 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.369 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.369 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.369 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880737+0000", 2024-09-16T20:50:33.369 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5220000000000002e-06 2024-09-16T20:50:33.369 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.369 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.369 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.369 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.369 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.370 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1747 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.370 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.881113+0000", 2024-09-16T20:50:33.370 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0078942189999999992, 2024-09-16T20:50:33.370 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.013311211999999999, 2024-09-16T20:50:33.370 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.370 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.370 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.370 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.370 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.370 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.370 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1747 2024-09-16T20:50:33.370 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.370 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.371 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.371 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.371 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.881113+0000", 2024-09-16T20:50:33.371 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.371 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.371 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.371 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.371 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.881113+0000", 2024-09-16T20:50:33.371 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.371 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.371 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.371 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.371 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886333+0000", 2024-09-16T20:50:33.371 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0052205120000000001 2024-09-16T20:50:33.372 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.372 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.372 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.372 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886337+0000", 2024-09-16T20:50:33.372 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2799999999999999e-06 2024-09-16T20:50:33.372 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.372 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.372 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.372 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886344+0000", 2024-09-16T20:50:33.372 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5299999999999999e-06 2024-09-16T20:50:33.372 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.372 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.372 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.372 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886348+0000", 2024-09-16T20:50:33.373 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9779999999999999e-06 2024-09-16T20:50:33.373 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.373 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.373 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.373 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.373 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.373 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1779 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.373 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.886710+0000", 2024-09-16T20:50:33.373 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0022969930000000002, 2024-09-16T20:50:33.373 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.007737346, 2024-09-16T20:50:33.373 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.373 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.373 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.374 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.374 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.374 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.374 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1779 2024-09-16T20:50:33.374 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.374 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.374 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.374 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.374 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886710+0000", 2024-09-16T20:50:33.374 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.374 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.374 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.374 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.375 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886710+0000", 2024-09-16T20:50:33.375 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.375 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.375 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.375 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.375 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886710+0000", 2024-09-16T20:50:33.375 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3999999999999998e-07 2024-09-16T20:50:33.375 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.375 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.375 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.375 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886711+0000", 2024-09-16T20:50:33.375 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5300000000000002e-07 2024-09-16T20:50:33.375 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.376 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.376 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.376 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886714+0000", 2024-09-16T20:50:33.376 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9840000000000001e-06 2024-09-16T20:50:33.376 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.376 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.376 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.376 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886717+0000", 2024-09-16T20:50:33.376 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.576e-06 2024-09-16T20:50:33.376 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.376 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.376 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.376 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.377 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.377 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1811 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.377 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.892439+0000", 2024-09-16T20:50:33.377 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4294967295.9965682, 2024-09-16T20:50:33.377 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0020336239999999999, 2024-09-16T20:50:33.377 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.377 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.377 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.377 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.377 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.377 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.377 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1811 2024-09-16T20:50:33.377 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.378 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.378 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.378 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.378 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892439+0000", 2024-09-16T20:50:33.378 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.378 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.378 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.378 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.378 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892439+0000", 2024-09-16T20:50:33.378 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.378 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.378 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.378 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.379 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892439+0000", 2024-09-16T20:50:33.379 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1500000000000001e-07 2024-09-16T20:50:33.379 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.379 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.379 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.379 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892440+0000", 2024-09-16T20:50:33.379 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0920000000000001e-06 2024-09-16T20:50:33.379 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.379 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.379 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.379 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892443+0000", 2024-09-16T20:50:33.379 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9720000000000001e-06 2024-09-16T20:50:33.379 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.379 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.380 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.380 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892446+0000", 2024-09-16T20:50:33.380 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6180000000000002e-06 2024-09-16T20:50:33.380 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.380 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.380 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.380 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.380 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.380 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1620 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.380 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.868266+0000", 2024-09-16T20:50:33.380 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.020741169, 2024-09-16T20:50:33.380 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.026231346999999999, 2024-09-16T20:50:33.380 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.380 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.381 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.381 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.381 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.381 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.381 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1620 2024-09-16T20:50:33.381 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.381 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.381 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.381 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.381 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.868266+0000", 2024-09-16T20:50:33.381 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.381 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.381 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.381 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.382 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.868266+0000", 2024-09-16T20:50:33.382 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.382 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.382 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.382 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.382 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.868266+0000", 2024-09-16T20:50:33.382 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4999999999999998e-07 2024-09-16T20:50:33.382 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.382 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.382 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.382 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.868268+0000", 2024-09-16T20:50:33.382 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.657e-06 2024-09-16T20:50:33.382 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.383 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.383 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.383 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.868272+0000", 2024-09-16T20:50:33.383 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7929999999999999e-06 2024-09-16T20:50:33.383 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.383 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.383 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.383 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.868276+0000", 2024-09-16T20:50:33.383 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5460000000000002e-06 2024-09-16T20:50:33.383 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.383 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.383 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.383 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.383 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.384 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1652 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.384 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.871485+0000", 2024-09-16T20:50:33.384 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.017521914, 2024-09-16T20:50:33.384 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.023140594, 2024-09-16T20:50:33.384 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.384 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.384 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.384 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.384 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.384 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.384 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1652 2024-09-16T20:50:33.384 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.384 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.385 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.385 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.385 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.871485+0000", 2024-09-16T20:50:33.385 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.385 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.385 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.385 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.385 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.871485+0000", 2024-09-16T20:50:33.385 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.385 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.385 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.385 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.385 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.871596+0000", 2024-09-16T20:50:33.386 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000111069 2024-09-16T20:50:33.386 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.386 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.386 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.386 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.871599+0000", 2024-09-16T20:50:33.386 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8310000000000001e-06 2024-09-16T20:50:33.386 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.386 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.386 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.386 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.871605+0000", 2024-09-16T20:50:33.386 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.699e-06 2024-09-16T20:50:33.386 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.386 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.386 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.387 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.871608+0000", 2024-09-16T20:50:33.387 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6339999999999999e-06 2024-09-16T20:50:33.387 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.387 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.387 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.387 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.387 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.387 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1684 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.387 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.880386+0000", 2024-09-16T20:50:33.387 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0086212249999999997, 2024-09-16T20:50:33.387 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.014263876999999999, 2024-09-16T20:50:33.387 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.387 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.387 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.388 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.388 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.388 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.388 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1684 2024-09-16T20:50:33.388 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.388 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.388 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.388 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.388 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880386+0000", 2024-09-16T20:50:33.388 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.388 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.388 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.388 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.389 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880386+0000", 2024-09-16T20:50:33.389 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.389 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.389 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.389 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.389 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880386+0000", 2024-09-16T20:50:33.389 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9099999999999999e-07 2024-09-16T20:50:33.389 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.389 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.389 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.389 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880388+0000", 2024-09-16T20:50:33.389 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.482e-06 2024-09-16T20:50:33.389 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.389 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.390 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.390 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880391+0000", 2024-09-16T20:50:33.390 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4379999999999999e-06 2024-09-16T20:50:33.390 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.390 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.390 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.390 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880395+0000", 2024-09-16T20:50:33.390 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9960000000000004e-06 2024-09-16T20:50:33.390 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.390 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.390 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.390 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.390 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.391 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1716 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.391 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.880739+0000", 2024-09-16T20:50:33.391 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0082675270000000002, 2024-09-16T20:50:33.391 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.013935637000000001, 2024-09-16T20:50:33.391 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.391 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.391 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.391 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.391 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.391 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.391 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1716 2024-09-16T20:50:33.391 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.391 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.391 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.392 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.392 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880739+0000", 2024-09-16T20:50:33.392 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.392 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.392 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.392 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.392 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880739+0000", 2024-09-16T20:50:33.392 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.392 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.392 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.392 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.392 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880740+0000", 2024-09-16T20:50:33.392 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.17e-07 2024-09-16T20:50:33.393 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.393 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.393 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.393 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880741+0000", 2024-09-16T20:50:33.393 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0550000000000001e-06 2024-09-16T20:50:33.393 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.393 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.393 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.393 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880744+0000", 2024-09-16T20:50:33.393 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9059999999999998e-06 2024-09-16T20:50:33.393 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.393 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.393 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.393 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880746+0000", 2024-09-16T20:50:33.394 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8080000000000001e-06 2024-09-16T20:50:33.394 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.394 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.394 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.394 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.394 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.394 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1748 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.394 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.886355+0000", 2024-09-16T20:50:33.394 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0026516180000000001, 2024-09-16T20:50:33.394 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0083431890000000009, 2024-09-16T20:50:33.394 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.394 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.394 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.394 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.395 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.395 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.395 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1748 2024-09-16T20:50:33.395 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.395 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.395 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.395 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.395 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886355+0000", 2024-09-16T20:50:33.395 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.395 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.395 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.395 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.395 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886355+0000", 2024-09-16T20:50:33.395 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.396 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.396 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.396 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.396 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886356+0000", 2024-09-16T20:50:33.396 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8500000000000002e-07 2024-09-16T20:50:33.396 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.396 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.396 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.396 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886357+0000", 2024-09-16T20:50:33.396 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.358e-06 2024-09-16T20:50:33.396 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.396 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.396 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.397 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886360+0000", 2024-09-16T20:50:33.397 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1559999999999999e-06 2024-09-16T20:50:33.397 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.397 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.397 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.397 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886363+0000", 2024-09-16T20:50:33.397 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.01e-06 2024-09-16T20:50:33.397 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.397 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.397 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.397 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.397 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.397 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1780 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.398 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.886720+0000", 2024-09-16T20:50:33.398 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0022872819999999999, 2024-09-16T20:50:33.398 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0080034400000000006, 2024-09-16T20:50:33.398 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.398 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.398 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.398 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.398 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.398 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.398 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1780 2024-09-16T20:50:33.398 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.398 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.398 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.398 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.399 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886720+0000", 2024-09-16T20:50:33.399 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.399 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.399 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.399 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.399 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886720+0000", 2024-09-16T20:50:33.399 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.399 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.399 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.399 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.399 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886720+0000", 2024-09-16T20:50:33.399 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0900000000000001e-07 2024-09-16T20:50:33.399 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.399 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.400 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.400 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886721+0000", 2024-09-16T20:50:33.400 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0240000000000001e-06 2024-09-16T20:50:33.400 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.400 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.400 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.400 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886724+0000", 2024-09-16T20:50:33.400 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8609999999999998e-06 2024-09-16T20:50:33.400 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.400 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.400 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.400 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886727+0000", 2024-09-16T20:50:33.400 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6900000000000001e-06 2024-09-16T20:50:33.401 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.401 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.401 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.401 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.401 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.401 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1812 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.401 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.892448+0000", 2024-09-16T20:50:33.401 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4294967295.9965587, 2024-09-16T20:50:33.401 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.002298062, 2024-09-16T20:50:33.401 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.401 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.401 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.401 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.402 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.402 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.402 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1812 2024-09-16T20:50:33.402 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.402 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.402 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.402 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.402 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892448+0000", 2024-09-16T20:50:33.402 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.402 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.402 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.402 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.402 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892448+0000", 2024-09-16T20:50:33.402 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.403 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.403 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.403 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.403 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892449+0000", 2024-09-16T20:50:33.403 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1400000000000001e-07 2024-09-16T20:50:33.403 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.403 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.403 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.403 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892450+0000", 2024-09-16T20:50:33.403 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0330000000000001e-06 2024-09-16T20:50:33.403 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.403 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.403 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.404 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892463+0000", 2024-09-16T20:50:33.404 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3647e-05 2024-09-16T20:50:33.404 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.404 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.404 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.404 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892472+0000", 2024-09-16T20:50:33.404 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4109999999999996e-06 2024-09-16T20:50:33.404 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.404 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.404 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.404 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.404 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.404 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1621 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.405 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.868281+0000", 2024-09-16T20:50:33.405 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.020726449000000001, 2024-09-16T20:50:33.405 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.026492471, 2024-09-16T20:50:33.405 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.405 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.405 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.405 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.405 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.405 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.405 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1621 2024-09-16T20:50:33.405 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.405 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.405 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.405 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.406 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.868281+0000", 2024-09-16T20:50:33.406 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.406 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.406 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.406 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.868281+0000", 2024-09-16T20:50:33.406 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.406 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.406 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.406 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.868376+0000", 2024-09-16T20:50:33.406 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5425000000000002e-05 2024-09-16T20:50:33.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.407 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.407 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.407 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.868379+0000", 2024-09-16T20:50:33.407 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9299999999999999e-06 2024-09-16T20:50:33.407 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.407 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.407 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.407 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.868384+0000", 2024-09-16T20:50:33.407 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9840000000000001e-06 2024-09-16T20:50:33.407 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.407 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.407 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.407 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.868387+0000", 2024-09-16T20:50:33.408 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6009999999999999e-06 2024-09-16T20:50:33.408 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.408 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.408 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.408 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.408 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.408 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1653 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.408 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.871622+0000", 2024-09-16T20:50:33.408 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.017385346999999999, 2024-09-16T20:50:33.408 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.023175346999999999, 2024-09-16T20:50:33.408 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.408 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.408 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.409 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.409 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.409 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.409 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1653 2024-09-16T20:50:33.409 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.409 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.409 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.409 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.409 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.871622+0000", 2024-09-16T20:50:33.409 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.409 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.409 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.409 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.409 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.871622+0000", 2024-09-16T20:50:33.410 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.410 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.410 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.410 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.871622+0000", 2024-09-16T20:50:33.410 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6100000000000001e-07 2024-09-16T20:50:33.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.410 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.410 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.410 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.871624+0000", 2024-09-16T20:50:33.410 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.604e-06 2024-09-16T20:50:33.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.410 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.411 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.411 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.871627+0000", 2024-09-16T20:50:33.411 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.67e-06 2024-09-16T20:50:33.411 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.411 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.411 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.411 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.871630+0000", 2024-09-16T20:50:33.411 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9270000000000001e-06 2024-09-16T20:50:33.411 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.411 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.411 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.411 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.411 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.412 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1685 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.412 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.880399+0000", 2024-09-16T20:50:33.412 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0086082439999999993, 2024-09-16T20:50:33.412 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.014422492, 2024-09-16T20:50:33.412 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.412 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.412 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.412 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.412 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.412 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.412 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1685 2024-09-16T20:50:33.412 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.412 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.413 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.413 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.413 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880399+0000", 2024-09-16T20:50:33.413 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.413 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.413 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.413 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.413 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880399+0000", 2024-09-16T20:50:33.413 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.413 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.413 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.413 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.413 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880399+0000", 2024-09-16T20:50:33.413 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1300000000000001e-07 2024-09-16T20:50:33.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.414 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.414 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.414 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880400+0000", 2024-09-16T20:50:33.414 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.235e-06 2024-09-16T20:50:33.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.414 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.414 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.414 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880403+0000", 2024-09-16T20:50:33.414 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1049999999999999e-06 2024-09-16T20:50:33.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.414 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.414 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.414 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880406+0000", 2024-09-16T20:50:33.415 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7439999999999999e-06 2024-09-16T20:50:33.415 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.415 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.415 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.415 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.415 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.415 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1717 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.415 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.880749+0000", 2024-09-16T20:50:33.415 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.008257538, 2024-09-16T20:50:33.415 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.014094305999999999, 2024-09-16T20:50:33.415 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.415 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.415 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.415 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.416 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.416 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.416 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1717 2024-09-16T20:50:33.416 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.416 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.416 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.416 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.416 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880749+0000", 2024-09-16T20:50:33.416 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.416 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.416 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.416 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.416 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880749+0000", 2024-09-16T20:50:33.417 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.417 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.417 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.417 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.417 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880750+0000", 2024-09-16T20:50:33.417 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.28e-07 2024-09-16T20:50:33.417 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.417 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.417 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.417 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880751+0000", 2024-09-16T20:50:33.417 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0330000000000001e-06 2024-09-16T20:50:33.417 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.417 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.418 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.418 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880754+0000", 2024-09-16T20:50:33.418 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.681e-06 2024-09-16T20:50:33.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.418 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.418 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.418 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880757+0000", 2024-09-16T20:50:33.418 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4549999999999998e-06 2024-09-16T20:50:33.418 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.418 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.418 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.418 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.418 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1749 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.419 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.886367+0000", 2024-09-16T20:50:33.419 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0026403120000000001, 2024-09-16T20:50:33.419 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0084996110000000007, 2024-09-16T20:50:33.419 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.419 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.419 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.419 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.419 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.419 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.419 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1749 2024-09-16T20:50:33.419 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.419 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.419 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.419 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.420 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886367+0000", 2024-09-16T20:50:33.420 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.420 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.420 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.420 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886367+0000", 2024-09-16T20:50:33.420 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.420 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.420 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.420 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886367+0000", 2024-09-16T20:50:33.420 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0499999999999999e-07 2024-09-16T20:50:33.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.421 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.421 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.421 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886368+0000", 2024-09-16T20:50:33.421 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.068e-06 2024-09-16T20:50:33.421 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.421 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.421 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.421 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886371+0000", 2024-09-16T20:50:33.421 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.004e-06 2024-09-16T20:50:33.421 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.421 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.421 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.421 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886374+0000", 2024-09-16T20:50:33.421 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7180000000000001e-06 2024-09-16T20:50:33.422 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.422 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.422 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.422 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.422 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.422 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1781 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.422 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.886730+0000", 2024-09-16T20:50:33.422 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0022769579999999999, 2024-09-16T20:50:33.422 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0081604030000000001, 2024-09-16T20:50:33.422 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.422 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.422 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.422 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.422 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.423 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.423 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1781 2024-09-16T20:50:33.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.423 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.423 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.423 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.423 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886730+0000", 2024-09-16T20:50:33.423 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.423 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.423 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.423 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886730+0000", 2024-09-16T20:50:33.423 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.424 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886730+0000", 2024-09-16T20:50:33.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0900000000000001e-07 2024-09-16T20:50:33.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.424 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886731+0000", 2024-09-16T20:50:33.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0729999999999999e-06 2024-09-16T20:50:33.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.424 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886734+0000", 2024-09-16T20:50:33.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.976e-06 2024-09-16T20:50:33.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.425 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.425 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886737+0000", 2024-09-16T20:50:33.425 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8629999999999999e-06 2024-09-16T20:50:33.425 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.425 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.425 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.425 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1813 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.425 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.892476+0000", 2024-09-16T20:50:33.425 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4294967295.9965305, 2024-09-16T20:50:33.426 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0024371929999999998, 2024-09-16T20:50:33.426 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.426 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.426 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.426 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.426 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.426 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.426 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1813 2024-09-16T20:50:33.426 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.426 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.426 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.426 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.426 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892476+0000", 2024-09-16T20:50:33.426 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.427 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.427 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.427 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.427 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892476+0000", 2024-09-16T20:50:33.427 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.427 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.427 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.427 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.427 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892477+0000", 2024-09-16T20:50:33.427 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5899999999999998e-07 2024-09-16T20:50:33.427 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.427 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.427 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.427 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892478+0000", 2024-09-16T20:50:33.428 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2410000000000001e-06 2024-09-16T20:50:33.428 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.428 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.428 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.428 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892481+0000", 2024-09-16T20:50:33.428 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3220000000000001e-06 2024-09-16T20:50:33.428 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.428 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.428 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.428 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892484+0000", 2024-09-16T20:50:33.428 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7779999999999999e-06 2024-09-16T20:50:33.428 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.428 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.429 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.429 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.429 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1622 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.429 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.868394+0000", 2024-09-16T20:50:33.429 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.020613374, 2024-09-16T20:50:33.429 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.026545261000000001, 2024-09-16T20:50:33.429 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.429 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.429 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.429 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.429 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.429 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.430 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1622 2024-09-16T20:50:33.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.430 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.430 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.430 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.430 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.868394+0000", 2024-09-16T20:50:33.430 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.430 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.430 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.430 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.868394+0000", 2024-09-16T20:50:33.430 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.430 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.431 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.431 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.868500+0000", 2024-09-16T20:50:33.431 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010676899999999999 2024-09-16T20:50:33.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.431 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.431 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.431 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.868504+0000", 2024-09-16T20:50:33.431 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1709999999999998e-06 2024-09-16T20:50:33.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.431 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.431 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.431 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.868510+0000", 2024-09-16T20:50:33.431 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5740000000000001e-06 2024-09-16T20:50:33.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.432 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.432 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.432 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.868514+0000", 2024-09-16T20:50:33.432 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5370000000000002e-06 2024-09-16T20:50:33.432 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.432 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.432 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.432 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.432 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1654 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.432 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.871634+0000", 2024-09-16T20:50:33.432 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.017373137, 2024-09-16T20:50:33.432 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.023327931, 2024-09-16T20:50:33.433 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.433 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.433 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.433 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.433 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.433 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.433 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1654 2024-09-16T20:50:33.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.433 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.433 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.433 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.433 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.871634+0000", 2024-09-16T20:50:33.433 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.434 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.434 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.434 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.871634+0000", 2024-09-16T20:50:33.434 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.434 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.434 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.434 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.871730+0000", 2024-09-16T20:50:33.434 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6345e-05 2024-09-16T20:50:33.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.434 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.434 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.435 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.871733+0000", 2024-09-16T20:50:33.435 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.464e-06 2024-09-16T20:50:33.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.435 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.435 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.435 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.871738+0000", 2024-09-16T20:50:33.435 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.989e-06 2024-09-16T20:50:33.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.435 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.435 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.435 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.871741+0000", 2024-09-16T20:50:33.435 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4910000000000001e-06 2024-09-16T20:50:33.435 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.435 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.436 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.436 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.436 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1686 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.436 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.880409+0000", 2024-09-16T20:50:33.436 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0085977669999999992, 2024-09-16T20:50:33.436 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.014576676, 2024-09-16T20:50:33.436 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.436 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.436 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.436 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.436 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.436 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.437 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1686 2024-09-16T20:50:33.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.437 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.437 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.437 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880409+0000", 2024-09-16T20:50:33.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.437 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.437 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880409+0000", 2024-09-16T20:50:33.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880409+0000", 2024-09-16T20:50:33.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2399999999999999e-07 2024-09-16T20:50:33.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880411+0000", 2024-09-16T20:50:33.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.141e-06 2024-09-16T20:50:33.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880414+0000", 2024-09-16T20:50:33.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.954e-06 2024-09-16T20:50:33.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.439 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.439 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.439 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880416+0000", 2024-09-16T20:50:33.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.61e-06 2024-09-16T20:50:33.439 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.439 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.439 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.439 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.439 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1718 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.439 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.880760+0000", 2024-09-16T20:50:33.439 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0082472169999999994, 2024-09-16T20:50:33.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.014248902000000001, 2024-09-16T20:50:33.439 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.440 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.440 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.440 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.440 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.440 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.440 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1718 2024-09-16T20:50:33.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.440 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.440 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.440 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880760+0000", 2024-09-16T20:50:33.440 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880760+0000", 2024-09-16T20:50:33.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880760+0000", 2024-09-16T20:50:33.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.22e-07 2024-09-16T20:50:33.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880764+0000", 2024-09-16T20:50:33.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6739999999999999e-06 2024-09-16T20:50:33.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880767+0000", 2024-09-16T20:50:33.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4350000000000001e-06 2024-09-16T20:50:33.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880770+0000", 2024-09-16T20:50:33.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.926e-06 2024-09-16T20:50:33.442 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.442 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.442 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.443 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1750 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.443 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.886377+0000", 2024-09-16T20:50:33.443 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.002630009, 2024-09-16T20:50:33.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0086557360000000007, 2024-09-16T20:50:33.443 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.443 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.443 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.443 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.443 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.443 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.443 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1750 2024-09-16T20:50:33.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.444 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.444 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886377+0000", 2024-09-16T20:50:33.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.444 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886377+0000", 2024-09-16T20:50:33.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886377+0000", 2024-09-16T20:50:33.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.11e-07 2024-09-16T20:50:33.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886378+0000", 2024-09-16T20:50:33.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7900000000000007e-07 2024-09-16T20:50:33.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886381+0000", 2024-09-16T20:50:33.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0750000000000002e-06 2024-09-16T20:50:33.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.446 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.446 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886384+0000", 2024-09-16T20:50:33.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5859999999999999e-06 2024-09-16T20:50:33.446 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.446 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.446 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.446 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1782 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.446 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.886756+0000", 2024-09-16T20:50:33.446 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0022513680000000001, 2024-09-16T20:50:33.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0082996509999999999, 2024-09-16T20:50:33.446 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.446 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.447 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.447 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.447 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.447 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.447 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1782 2024-09-16T20:50:33.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.447 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886756+0000", 2024-09-16T20:50:33.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886756+0000", 2024-09-16T20:50:33.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886756+0000", 2024-09-16T20:50:33.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2899999999999999e-07 2024-09-16T20:50:33.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886757+0000", 2024-09-16T20:50:33.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4279999999999999e-06 2024-09-16T20:50:33.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.449 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.449 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886761+0000", 2024-09-16T20:50:33.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4429999999999999e-06 2024-09-16T20:50:33.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.449 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.449 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886764+0000", 2024-09-16T20:50:33.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6680000000000001e-06 2024-09-16T20:50:33.449 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.449 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.449 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.450 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1814 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.450 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.892487+0000", 2024-09-16T20:50:33.450 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4294967295.99652, 2024-09-16T20:50:33.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0025929339999999999, 2024-09-16T20:50:33.450 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.450 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.450 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.450 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.450 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.450 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.450 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1814 2024-09-16T20:50:33.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.450 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.451 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.451 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892487+0000", 2024-09-16T20:50:33.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.451 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.451 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892487+0000", 2024-09-16T20:50:33.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.451 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.451 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892487+0000", 2024-09-16T20:50:33.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1e-07 2024-09-16T20:50:33.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892492+0000", 2024-09-16T20:50:33.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.831e-06 2024-09-16T20:50:33.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892500+0000", 2024-09-16T20:50:33.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4100000000000008e-06 2024-09-16T20:50:33.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892503+0000", 2024-09-16T20:50:33.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0259999999999999e-06 2024-09-16T20:50:33.453 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.453 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.453 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.453 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1623 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.453 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.868520+0000", 2024-09-16T20:50:33.453 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.020487215, 2024-09-16T20:50:33.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.026584050000000001, 2024-09-16T20:50:33.453 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.453 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.453 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.453 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.454 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.454 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.454 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1623 2024-09-16T20:50:33.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.454 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.868520+0000", 2024-09-16T20:50:33.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.868520+0000", 2024-09-16T20:50:33.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.868631+0000", 2024-09-16T20:50:33.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000111049 2024-09-16T20:50:33.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.868637+0000", 2024-09-16T20:50:33.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5440000000000004e-06 2024-09-16T20:50:33.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.456 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.456 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.868643+0000", 2024-09-16T20:50:33.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6890000000000001e-06 2024-09-16T20:50:33.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.456 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.456 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.868647+0000", 2024-09-16T20:50:33.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4279999999999998e-06 2024-09-16T20:50:33.456 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.456 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.456 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.456 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1655 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.457 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.871748+0000", 2024-09-16T20:50:33.457 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.017258895, 2024-09-16T20:50:33.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.023380622, 2024-09-16T20:50:33.457 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.457 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.457 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.457 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.457 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.457 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.457 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1655 2024-09-16T20:50:33.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.457 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.871748+0000", 2024-09-16T20:50:33.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.871748+0000", 2024-09-16T20:50:33.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.871857+0000", 2024-09-16T20:50:33.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000108806 2024-09-16T20:50:33.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.871860+0000", 2024-09-16T20:50:33.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1540000000000002e-06 2024-09-16T20:50:33.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.871865+0000", 2024-09-16T20:50:33.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1560000000000003e-06 2024-09-16T20:50:33.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.460 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.871869+0000", 2024-09-16T20:50:33.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3830000000000001e-06 2024-09-16T20:50:33.460 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.460 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.460 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.460 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1687 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.460 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.880419+0000", 2024-09-16T20:50:33.460 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0085878429999999995, 2024-09-16T20:50:33.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.014733777, 2024-09-16T20:50:33.460 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.460 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.461 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.461 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.461 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.461 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.461 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1687 2024-09-16T20:50:33.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.461 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880419+0000", 2024-09-16T20:50:33.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.462 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880419+0000", 2024-09-16T20:50:33.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.462 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880419+0000", 2024-09-16T20:50:33.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0800000000000001e-07 2024-09-16T20:50:33.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.462 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880420+0000", 2024-09-16T20:50:33.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.119e-06 2024-09-16T20:50:33.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.463 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.463 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880424+0000", 2024-09-16T20:50:33.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0620000000000001e-06 2024-09-16T20:50:33.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.463 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.463 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880426+0000", 2024-09-16T20:50:33.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.441e-06 2024-09-16T20:50:33.463 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.463 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.463 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.464 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1719 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.464 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.880788+0000", 2024-09-16T20:50:33.464 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0082192719999999997, 2024-09-16T20:50:33.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.014389328999999999, 2024-09-16T20:50:33.464 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.464 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.464 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.464 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.464 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.464 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.464 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1719 2024-09-16T20:50:33.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.465 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.465 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.465 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880788+0000", 2024-09-16T20:50:33.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.465 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.465 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880788+0000", 2024-09-16T20:50:33.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.465 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.465 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880788+0000", 2024-09-16T20:50:33.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2399999999999999e-07 2024-09-16T20:50:33.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880790+0000", 2024-09-16T20:50:33.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.469e-06 2024-09-16T20:50:33.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880793+0000", 2024-09-16T20:50:33.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4690000000000002e-06 2024-09-16T20:50:33.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.467 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880796+0000", 2024-09-16T20:50:33.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.711e-06 2024-09-16T20:50:33.467 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.467 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.467 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.467 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1751 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.467 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.886387+0000", 2024-09-16T20:50:33.467 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0026202349999999998, 2024-09-16T20:50:33.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0088128300000000007, 2024-09-16T20:50:33.467 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.467 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.468 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.468 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.468 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.468 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.468 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1751 2024-09-16T20:50:33.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.468 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.468 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886387+0000", 2024-09-16T20:50:33.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.469 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.469 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886387+0000", 2024-09-16T20:50:33.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.469 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.469 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886387+0000", 2024-09-16T20:50:33.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.17e-07 2024-09-16T20:50:33.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.469 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.469 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886388+0000", 2024-09-16T20:50:33.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.04e-06 2024-09-16T20:50:33.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886391+0000", 2024-09-16T20:50:33.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9000000000000002e-06 2024-09-16T20:50:33.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886394+0000", 2024-09-16T20:50:33.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6690000000000002e-06 2024-09-16T20:50:33.470 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.470 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.470 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.471 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1783 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.471 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.886767+0000", 2024-09-16T20:50:33.471 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0022399989999999999, 2024-09-16T20:50:33.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0084555419999999999, 2024-09-16T20:50:33.471 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.471 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.471 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.471 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.471 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.471 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.471 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1783 2024-09-16T20:50:33.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.471 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886767+0000", 2024-09-16T20:50:33.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886767+0000", 2024-09-16T20:50:33.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886767+0000", 2024-09-16T20:50:33.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0800000000000001e-07 2024-09-16T20:50:33.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886768+0000", 2024-09-16T20:50:33.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.009e-06 2024-09-16T20:50:33.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886771+0000", 2024-09-16T20:50:33.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9560000000000002e-06 2024-09-16T20:50:33.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.474 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886774+0000", 2024-09-16T20:50:33.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5950000000000001e-06 2024-09-16T20:50:33.474 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.474 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.474 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.474 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1815 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.474 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.892507+0000", 2024-09-16T20:50:33.474 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4294967295.9964995, 2024-09-16T20:50:33.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0027399939999999999, 2024-09-16T20:50:33.474 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.474 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.475 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.475 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.475 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.475 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.475 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1815 2024-09-16T20:50:33.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.475 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892507+0000", 2024-09-16T20:50:33.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.476 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892507+0000", 2024-09-16T20:50:33.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.476 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892507+0000", 2024-09-16T20:50:33.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6100000000000002e-07 2024-09-16T20:50:33.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.476 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892509+0000", 2024-09-16T20:50:33.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.288e-06 2024-09-16T20:50:33.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892512+0000", 2024-09-16T20:50:33.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2380000000000002e-06 2024-09-16T20:50:33.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892514+0000", 2024-09-16T20:50:33.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4710000000000001e-06 2024-09-16T20:50:33.477 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.477 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.477 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.478 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1624 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.478 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.868654+0000", 2024-09-16T20:50:33.478 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.020353156000000001, 2024-09-16T20:50:33.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.026618102000000001, 2024-09-16T20:50:33.478 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.478 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.478 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.478 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.478 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.478 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.478 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1624 2024-09-16T20:50:33.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.479 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.868654+0000", 2024-09-16T20:50:33.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.868654+0000", 2024-09-16T20:50:33.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.868760+0000", 2024-09-16T20:50:33.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010576700000000001 2024-09-16T20:50:33.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.868762+0000", 2024-09-16T20:50:33.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.492e-06 2024-09-16T20:50:33.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.868767+0000", 2024-09-16T20:50:33.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.079e-06 2024-09-16T20:50:33.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.481 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.481 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.868771+0000", 2024-09-16T20:50:33.481 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4060000000000001e-06 2024-09-16T20:50:33.481 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.481 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.481 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.481 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1656 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.481 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.871874+0000", 2024-09-16T20:50:33.481 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.017132564999999999, 2024-09-16T20:50:33.481 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.023422136999999999, 2024-09-16T20:50:33.481 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.482 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.482 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.482 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.482 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.482 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.482 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1656 2024-09-16T20:50:33.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.482 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.482 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.871874+0000", 2024-09-16T20:50:33.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.871874+0000", 2024-09-16T20:50:33.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.871984+0000", 2024-09-16T20:50:33.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010936 2024-09-16T20:50:33.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.871987+0000", 2024-09-16T20:50:33.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8700000000000001e-06 2024-09-16T20:50:33.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.484 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.871992+0000", 2024-09-16T20:50:33.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9459999999999997e-06 2024-09-16T20:50:33.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.484 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.871995+0000", 2024-09-16T20:50:33.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.523e-06 2024-09-16T20:50:33.484 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.484 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.484 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.485 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1688 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.485 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.880429+0000", 2024-09-16T20:50:33.485 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0085780230000000006, 2024-09-16T20:50:33.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.014897169, 2024-09-16T20:50:33.485 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.485 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.485 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.485 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.485 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.485 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.485 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1688 2024-09-16T20:50:33.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.486 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.486 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880429+0000", 2024-09-16T20:50:33.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.486 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880429+0000", 2024-09-16T20:50:33.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.486 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880429+0000", 2024-09-16T20:50:33.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0900000000000001e-07 2024-09-16T20:50:33.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.487 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880430+0000", 2024-09-16T20:50:33.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.102e-06 2024-09-16T20:50:33.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.487 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880433+0000", 2024-09-16T20:50:33.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0580000000000002e-06 2024-09-16T20:50:33.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.488 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.488 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880436+0000", 2024-09-16T20:50:33.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5189999999999999e-06 2024-09-16T20:50:33.488 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.488 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.488 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.488 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1720 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.488 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.880799+0000", 2024-09-16T20:50:33.488 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0082078350000000001, 2024-09-16T20:50:33.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.014552127999999999, 2024-09-16T20:50:33.488 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.489 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.489 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.489 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.489 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.489 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.489 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1720 2024-09-16T20:50:33.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.489 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880799+0000", 2024-09-16T20:50:33.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880799+0000", 2024-09-16T20:50:33.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880799+0000", 2024-09-16T20:50:33.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1300000000000001e-07 2024-09-16T20:50:33.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880801+0000", 2024-09-16T20:50:33.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0669999999999999e-06 2024-09-16T20:50:33.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880804+0000", 2024-09-16T20:50:33.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9799999999999998e-06 2024-09-16T20:50:33.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880806+0000", 2024-09-16T20:50:33.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5289999999999998e-06 2024-09-16T20:50:33.491 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.491 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.491 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.492 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1752 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.492 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.886397+0000", 2024-09-16T20:50:33.492 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0026104090000000002, 2024-09-16T20:50:33.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0089783840000000007, 2024-09-16T20:50:33.492 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.492 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.492 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.492 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.492 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.492 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.492 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1752 2024-09-16T20:50:33.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.493 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.493 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.493 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886397+0000", 2024-09-16T20:50:33.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.493 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.493 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886397+0000", 2024-09-16T20:50:33.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.494 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.494 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886397+0000", 2024-09-16T20:50:33.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1e-07 2024-09-16T20:50:33.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.494 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.494 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886398+0000", 2024-09-16T20:50:33.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0100000000000001e-06 2024-09-16T20:50:33.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.494 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.494 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886402+0000", 2024-09-16T20:50:33.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6909999999999999e-06 2024-09-16T20:50:33.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.495 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886404+0000", 2024-09-16T20:50:33.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7049999999999999e-06 2024-09-16T20:50:33.495 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.495 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.495 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.495 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1784 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.495 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.886777+0000", 2024-09-16T20:50:33.495 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0022301769999999999, 2024-09-16T20:50:33.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0086234020000000005, 2024-09-16T20:50:33.495 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.496 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.496 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.496 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.496 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.496 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.496 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1784 2024-09-16T20:50:33.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.496 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886777+0000", 2024-09-16T20:50:33.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.497 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886777+0000", 2024-09-16T20:50:33.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.497 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886777+0000", 2024-09-16T20:50:33.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1500000000000001e-07 2024-09-16T20:50:33.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.497 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886778+0000", 2024-09-16T20:50:33.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0160000000000001e-06 2024-09-16T20:50:33.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.498 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.498 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886781+0000", 2024-09-16T20:50:33.498 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9330000000000001e-06 2024-09-16T20:50:33.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.498 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.498 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886784+0000", 2024-09-16T20:50:33.498 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7e-06 2024-09-16T20:50:33.498 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.498 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.498 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.499 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.499 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1816 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.499 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.892517+0000", 2024-09-16T20:50:33.499 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4294967295.9964895, 2024-09-16T20:50:33.499 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0029070379999999998, 2024-09-16T20:50:33.499 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.499 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.499 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.499 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.499 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.499 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.499 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1816 2024-09-16T20:50:33.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.499 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.499 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.500 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.500 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892517+0000", 2024-09-16T20:50:33.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.500 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.500 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892517+0000", 2024-09-16T20:50:33.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.500 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.500 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892518+0000", 2024-09-16T20:50:33.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1e-07 2024-09-16T20:50:33.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892519+0000", 2024-09-16T20:50:33.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.083e-06 2024-09-16T20:50:33.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892522+0000", 2024-09-16T20:50:33.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9280000000000002e-06 2024-09-16T20:50:33.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892525+0000", 2024-09-16T20:50:33.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8710000000000001e-06 2024-09-16T20:50:33.502 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.502 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.502 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.502 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1625 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.502 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.868777+0000", 2024-09-16T20:50:33.502 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.020230346, 2024-09-16T20:50:33.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.026673641000000001, 2024-09-16T20:50:33.502 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.502 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.502 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.502 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.503 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.503 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.503 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1625 2024-09-16T20:50:33.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.503 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.868777+0000", 2024-09-16T20:50:33.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.868777+0000", 2024-09-16T20:50:33.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.504 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.504 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.868777+0000", 2024-09-16T20:50:33.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6300000000000001e-07 2024-09-16T20:50:33.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.504 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.504 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.868778+0000", 2024-09-16T20:50:33.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.178e-06 2024-09-16T20:50:33.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.504 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.868781+0000", 2024-09-16T20:50:33.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2370000000000001e-06 2024-09-16T20:50:33.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.868784+0000", 2024-09-16T20:50:33.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7149999999999998e-06 2024-09-16T20:50:33.505 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.505 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.505 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.505 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1657 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.505 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.872002+0000", 2024-09-16T20:50:33.506 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.017004866, 2024-09-16T20:50:33.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.023471369999999998, 2024-09-16T20:50:33.506 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.506 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.506 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.506 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.506 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.506 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.506 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1657 2024-09-16T20:50:33.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.506 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872002+0000", 2024-09-16T20:50:33.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.507 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.507 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.507 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872002+0000", 2024-09-16T20:50:33.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.507 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.507 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.507 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872003+0000", 2024-09-16T20:50:33.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6699999999999999e-07 2024-09-16T20:50:33.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.507 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.507 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.508 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872004+0000", 2024-09-16T20:50:33.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.158e-06 2024-09-16T20:50:33.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.508 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.508 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872007+0000", 2024-09-16T20:50:33.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.151e-06 2024-09-16T20:50:33.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.508 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.508 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872010+0000", 2024-09-16T20:50:33.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.773e-06 2024-09-16T20:50:33.508 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.508 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.509 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.509 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1689 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.509 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.880439+0000", 2024-09-16T20:50:33.509 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0085682060000000001, 2024-09-16T20:50:33.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015059176000000001, 2024-09-16T20:50:33.509 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.509 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.509 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.509 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.509 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.509 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.510 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1689 2024-09-16T20:50:33.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.510 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880439+0000", 2024-09-16T20:50:33.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880439+0000", 2024-09-16T20:50:33.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880439+0000", 2024-09-16T20:50:33.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1199999999999999e-07 2024-09-16T20:50:33.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880440+0000", 2024-09-16T20:50:33.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0589999999999999e-06 2024-09-16T20:50:33.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880443+0000", 2024-09-16T20:50:33.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.942e-06 2024-09-16T20:50:33.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.512 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.512 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880446+0000", 2024-09-16T20:50:33.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5150000000000001e-06 2024-09-16T20:50:33.512 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.512 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.512 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.512 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.512 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1721 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.512 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.880809+0000", 2024-09-16T20:50:33.512 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0081980190000000008, 2024-09-16T20:50:33.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.014713189999999999, 2024-09-16T20:50:33.512 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.513 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.513 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.513 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.513 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.513 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.513 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1721 2024-09-16T20:50:33.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.513 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.513 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.513 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880809+0000", 2024-09-16T20:50:33.513 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880809+0000", 2024-09-16T20:50:33.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880809+0000", 2024-09-16T20:50:33.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1899999999999999e-07 2024-09-16T20:50:33.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880810+0000", 2024-09-16T20:50:33.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6200000000000006e-07 2024-09-16T20:50:33.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.515 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.515 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.515 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880813+0000", 2024-09-16T20:50:33.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9079999999999999e-06 2024-09-16T20:50:33.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.515 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.515 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.515 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880816+0000", 2024-09-16T20:50:33.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.571e-06 2024-09-16T20:50:33.515 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.515 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.515 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.516 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.516 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1753 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.516 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.886407+0000", 2024-09-16T20:50:33.516 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0025998940000000002, 2024-09-16T20:50:33.516 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0091391979999999994, 2024-09-16T20:50:33.516 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.516 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.516 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.516 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.516 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.516 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.516 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1753 2024-09-16T20:50:33.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.517 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.517 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.517 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886407+0000", 2024-09-16T20:50:33.517 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.517 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.517 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886407+0000", 2024-09-16T20:50:33.517 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.517 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.517 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886407+0000", 2024-09-16T20:50:33.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2999999999999999e-07 2024-09-16T20:50:33.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.518 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.518 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886408+0000", 2024-09-16T20:50:33.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1289999999999999e-06 2024-09-16T20:50:33.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.518 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.518 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886411+0000", 2024-09-16T20:50:33.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8820000000000001e-06 2024-09-16T20:50:33.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.518 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.518 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886414+0000", 2024-09-16T20:50:33.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6680000000000001e-06 2024-09-16T20:50:33.519 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.519 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.519 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.519 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1785 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.519 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.886787+0000", 2024-09-16T20:50:33.519 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0022204730000000002, 2024-09-16T20:50:33.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0087826850000000001, 2024-09-16T20:50:33.519 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.519 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.519 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.519 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.520 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.520 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.520 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1785 2024-09-16T20:50:33.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.520 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.520 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.520 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886787+0000", 2024-09-16T20:50:33.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.520 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.520 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886787+0000", 2024-09-16T20:50:33.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.521 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.521 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.521 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886787+0000", 2024-09-16T20:50:33.521 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1400000000000001e-07 2024-09-16T20:50:33.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.521 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.521 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.521 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886788+0000", 2024-09-16T20:50:33.521 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7199999999999997e-07 2024-09-16T20:50:33.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.521 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.521 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.521 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886791+0000", 2024-09-16T20:50:33.522 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.807e-06 2024-09-16T20:50:33.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.522 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.522 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886793+0000", 2024-09-16T20:50:33.522 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4880000000000001e-06 2024-09-16T20:50:33.522 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.522 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.522 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.522 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1817 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.522 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.892527+0000", 2024-09-16T20:50:33.522 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4294967295.9964795, 2024-09-16T20:50:33.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030670680000000001, 2024-09-16T20:50:33.523 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.523 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.523 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.523 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.523 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.523 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.523 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1817 2024-09-16T20:50:33.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.523 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.523 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892527+0000", 2024-09-16T20:50:33.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.524 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892527+0000", 2024-09-16T20:50:33.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.524 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892528+0000", 2024-09-16T20:50:33.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1500000000000001e-07 2024-09-16T20:50:33.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.524 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892529+0000", 2024-09-16T20:50:33.525 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8899999999999998e-07 2024-09-16T20:50:33.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.525 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.525 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.525 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892533+0000", 2024-09-16T20:50:33.525 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4309999999999996e-06 2024-09-16T20:50:33.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.525 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.525 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.525 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892536+0000", 2024-09-16T20:50:33.525 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5050000000000002e-06 2024-09-16T20:50:33.525 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.525 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.525 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.526 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.526 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1626 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.526 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.868787+0000", 2024-09-16T20:50:33.526 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.020219582, 2024-09-16T20:50:33.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.026831469, 2024-09-16T20:50:33.526 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.526 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.526 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.526 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.526 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.526 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.526 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1626 2024-09-16T20:50:33.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.527 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.527 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.527 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.868787+0000", 2024-09-16T20:50:33.527 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.527 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.527 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.868787+0000", 2024-09-16T20:50:33.527 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.527 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.527 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.868887+0000", 2024-09-16T20:50:33.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9605999999999995e-05 2024-09-16T20:50:33.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.868890+0000", 2024-09-16T20:50:33.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1489999999999998e-06 2024-09-16T20:50:33.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.868897+0000", 2024-09-16T20:50:33.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3770000000000001e-06 2024-09-16T20:50:33.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.529 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.868900+0000", 2024-09-16T20:50:33.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5140000000000002e-06 2024-09-16T20:50:33.529 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.529 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.529 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.529 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1658 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.529 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.872018+0000", 2024-09-16T20:50:33.529 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.016989015, 2024-09-16T20:50:33.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.023630066000000002, 2024-09-16T20:50:33.529 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.529 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.530 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.530 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.530 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.530 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.530 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1658 2024-09-16T20:50:33.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.530 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.530 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.530 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872018+0000", 2024-09-16T20:50:33.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.530 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872018+0000", 2024-09-16T20:50:33.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872114+0000", 2024-09-16T20:50:33.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5684000000000001e-05 2024-09-16T20:50:33.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872117+0000", 2024-09-16T20:50:33.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.118e-06 2024-09-16T20:50:33.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.532 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.532 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872123+0000", 2024-09-16T20:50:33.532 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4570000000000002e-06 2024-09-16T20:50:33.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.532 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.532 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872127+0000", 2024-09-16T20:50:33.532 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5140000000000002e-06 2024-09-16T20:50:33.532 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.532 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.532 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.533 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.533 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1690 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.533 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.880449+0000", 2024-09-16T20:50:33.533 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0085584819999999992, 2024-09-16T20:50:33.533 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015248538000000001, 2024-09-16T20:50:33.533 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.533 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.533 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.533 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.533 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.533 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.533 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1690 2024-09-16T20:50:33.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.534 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.534 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.534 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880449+0000", 2024-09-16T20:50:33.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.534 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.534 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880449+0000", 2024-09-16T20:50:33.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.534 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.534 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880449+0000", 2024-09-16T20:50:33.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.04e-07 2024-09-16T20:50:33.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880450+0000", 2024-09-16T20:50:33.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1260000000000001e-06 2024-09-16T20:50:33.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880453+0000", 2024-09-16T20:50:33.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0070000000000002e-06 2024-09-16T20:50:33.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.536 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880455+0000", 2024-09-16T20:50:33.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6029999999999999e-06 2024-09-16T20:50:33.536 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.536 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.536 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.536 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1722 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.536 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.880819+0000", 2024-09-16T20:50:33.536 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.008188351, 2024-09-16T20:50:33.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.014926358000000001, 2024-09-16T20:50:33.536 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.537 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.537 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.537 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.537 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.537 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.537 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1722 2024-09-16T20:50:33.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.537 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880819+0000", 2024-09-16T20:50:33.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.538 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.538 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880819+0000", 2024-09-16T20:50:33.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.538 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.538 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880819+0000", 2024-09-16T20:50:33.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.22e-07 2024-09-16T20:50:33.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.538 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.538 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880820+0000", 2024-09-16T20:50:33.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.018e-06 2024-09-16T20:50:33.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.539 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.539 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880823+0000", 2024-09-16T20:50:33.539 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.886e-06 2024-09-16T20:50:33.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.539 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.539 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880827+0000", 2024-09-16T20:50:33.539 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.76e-06 2024-09-16T20:50:33.539 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.539 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.539 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.540 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1754 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.540 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.886417+0000", 2024-09-16T20:50:33.540 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.002590106, 2024-09-16T20:50:33.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0093793239999999996, 2024-09-16T20:50:33.540 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.540 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.540 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.540 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.540 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.540 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.540 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1754 2024-09-16T20:50:33.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.540 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.541 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886417+0000", 2024-09-16T20:50:33.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.541 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886417+0000", 2024-09-16T20:50:33.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.541 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886417+0000", 2024-09-16T20:50:33.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.11e-07 2024-09-16T20:50:33.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.542 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886422+0000", 2024-09-16T20:50:33.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3369999999999999e-06 2024-09-16T20:50:33.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.542 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886426+0000", 2024-09-16T20:50:33.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3050000000000001e-06 2024-09-16T20:50:33.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.542 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886429+0000", 2024-09-16T20:50:33.543 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3629999999999998e-06 2024-09-16T20:50:33.543 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.543 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.543 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.543 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.543 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1786 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.543 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.886797+0000", 2024-09-16T20:50:33.543 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0022104500000000001, 2024-09-16T20:50:33.543 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0090475480000000007, 2024-09-16T20:50:33.543 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.543 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.543 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.543 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.544 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.544 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.544 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1786 2024-09-16T20:50:33.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.544 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.544 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.544 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886797+0000", 2024-09-16T20:50:33.544 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.544 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.544 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886797+0000", 2024-09-16T20:50:33.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.545 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.545 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886797+0000", 2024-09-16T20:50:33.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1199999999999999e-07 2024-09-16T20:50:33.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.545 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.545 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886798+0000", 2024-09-16T20:50:33.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2699999999999999e-06 2024-09-16T20:50:33.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886801+0000", 2024-09-16T20:50:33.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9299999999999999e-06 2024-09-16T20:50:33.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886804+0000", 2024-09-16T20:50:33.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5679999999999998e-06 2024-09-16T20:50:33.546 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.546 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.546 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.546 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1818 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.547 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.892539+0000", 2024-09-16T20:50:33.547 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4294967295.9964685, 2024-09-16T20:50:33.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003351812, 2024-09-16T20:50:33.547 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.547 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.547 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.547 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.547 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.547 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.547 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1818 2024-09-16T20:50:33.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.547 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.547 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.548 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892539+0000", 2024-09-16T20:50:33.548 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.548 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.548 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.548 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892539+0000", 2024-09-16T20:50:33.548 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.548 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.548 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.548 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892539+0000", 2024-09-16T20:50:33.548 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.22e-07 2024-09-16T20:50:33.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.549 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.549 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892540+0000", 2024-09-16T20:50:33.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.054e-06 2024-09-16T20:50:33.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.549 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.549 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892543+0000", 2024-09-16T20:50:33.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9699999999999999e-06 2024-09-16T20:50:33.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.549 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.549 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892545+0000", 2024-09-16T20:50:33.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5840000000000002e-06 2024-09-16T20:50:33.550 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.550 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.550 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.550 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1627 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.550 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.868906+0000", 2024-09-16T20:50:33.550 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.020100654999999999, 2024-09-16T20:50:33.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.027010521999999999, 2024-09-16T20:50:33.550 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.550 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.550 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.550 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.551 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.551 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.551 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1627 2024-09-16T20:50:33.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.551 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.551 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.551 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.868906+0000", 2024-09-16T20:50:33.551 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.551 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.551 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.868906+0000", 2024-09-16T20:50:33.552 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.552 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.552 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.552 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.869016+0000", 2024-09-16T20:50:33.552 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000109565 2024-09-16T20:50:33.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.552 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.552 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.552 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.869018+0000", 2024-09-16T20:50:33.552 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.503e-06 2024-09-16T20:50:33.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.552 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.552 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.553 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.869024+0000", 2024-09-16T20:50:33.553 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2859999999999999e-06 2024-09-16T20:50:33.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.553 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.553 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.869027+0000", 2024-09-16T20:50:33.553 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3349999999999998e-06 2024-09-16T20:50:33.553 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.553 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.553 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.553 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1659 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.553 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.872133+0000", 2024-09-16T20:50:33.554 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.016874263, 2024-09-16T20:50:33.554 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.023810003, 2024-09-16T20:50:33.554 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.554 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.554 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.554 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.554 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.554 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.554 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1659 2024-09-16T20:50:33.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.554 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.554 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.554 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.554 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872133+0000", 2024-09-16T20:50:33.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872133+0000", 2024-09-16T20:50:33.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872241+0000", 2024-09-16T20:50:33.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010850500000000001 2024-09-16T20:50:33.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.556 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872248+0000", 2024-09-16T20:50:33.556 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8560000000000001e-06 2024-09-16T20:50:33.556 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.556 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.556 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.556 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872256+0000", 2024-09-16T20:50:33.556 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4229999999999999e-06 2024-09-16T20:50:33.556 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.556 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.556 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.556 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872259+0000", 2024-09-16T20:50:33.556 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.489e-06 2024-09-16T20:50:33.556 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.557 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.557 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.557 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.557 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.557 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1691 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.557 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.880458+0000", 2024-09-16T20:50:33.557 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0085486339999999994, 2024-09-16T20:50:33.557 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015521371000000001, 2024-09-16T20:50:33.557 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.557 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.557 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.557 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.557 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.557 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.558 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1691 2024-09-16T20:50:33.558 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.558 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.558 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.558 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.558 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880458+0000", 2024-09-16T20:50:33.558 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.558 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.558 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.558 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.558 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880458+0000", 2024-09-16T20:50:33.558 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.558 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.559 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880459+0000", 2024-09-16T20:50:33.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3999999999999998e-07 2024-09-16T20:50:33.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.559 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880460+0000", 2024-09-16T20:50:33.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.006e-06 2024-09-16T20:50:33.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.559 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880463+0000", 2024-09-16T20:50:33.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.993e-06 2024-09-16T20:50:33.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.560 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.560 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880465+0000", 2024-09-16T20:50:33.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.621e-06 2024-09-16T20:50:33.560 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.560 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.560 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.560 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1723 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.560 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.880831+0000", 2024-09-16T20:50:33.560 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0081764709999999994, 2024-09-16T20:50:33.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015201694, 2024-09-16T20:50:33.561 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.561 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.561 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.561 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.561 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.561 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.561 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1723 2024-09-16T20:50:33.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.561 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.561 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.561 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.561 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880831+0000", 2024-09-16T20:50:33.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.562 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.562 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880831+0000", 2024-09-16T20:50:33.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.562 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.562 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880831+0000", 2024-09-16T20:50:33.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1e-07 2024-09-16T20:50:33.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.562 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.563 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880832+0000", 2024-09-16T20:50:33.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4099999999999997e-07 2024-09-16T20:50:33.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.563 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.563 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880835+0000", 2024-09-16T20:50:33.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9299999999999999e-06 2024-09-16T20:50:33.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.563 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.563 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880837+0000", 2024-09-16T20:50:33.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4880000000000001e-06 2024-09-16T20:50:33.563 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.563 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.564 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.564 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1755 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.564 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.886432+0000", 2024-09-16T20:50:33.564 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0025745960000000002, 2024-09-16T20:50:33.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0096489130000000003, 2024-09-16T20:50:33.564 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.564 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.564 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.564 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.564 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.564 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.564 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1755 2024-09-16T20:50:33.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.565 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.565 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.565 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886432+0000", 2024-09-16T20:50:33.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.565 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.565 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886432+0000", 2024-09-16T20:50:33.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.566 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.566 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886433+0000", 2024-09-16T20:50:33.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3999999999999998e-07 2024-09-16T20:50:33.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.566 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.566 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.566 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886434+0000", 2024-09-16T20:50:33.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7300000000000004e-07 2024-09-16T20:50:33.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.566 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.566 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.566 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886437+0000", 2024-09-16T20:50:33.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.948e-06 2024-09-16T20:50:33.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.567 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.567 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886439+0000", 2024-09-16T20:50:33.567 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.667e-06 2024-09-16T20:50:33.567 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.567 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.567 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.567 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1787 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.567 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.886806+0000", 2024-09-16T20:50:33.567 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0022005710000000001, 2024-09-16T20:50:33.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0093261430000000003, 2024-09-16T20:50:33.568 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.568 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.568 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.568 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.568 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.568 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.568 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1787 2024-09-16T20:50:33.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.568 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.568 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.568 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886806+0000", 2024-09-16T20:50:33.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.569 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.569 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886806+0000", 2024-09-16T20:50:33.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.569 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.569 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886807+0000", 2024-09-16T20:50:33.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1199999999999999e-07 2024-09-16T20:50:33.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.569 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.570 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886808+0000", 2024-09-16T20:50:33.570 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4e-07 2024-09-16T20:50:33.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.570 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.570 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886811+0000", 2024-09-16T20:50:33.570 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9660000000000001e-06 2024-09-16T20:50:33.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.570 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.570 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886813+0000", 2024-09-16T20:50:33.570 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.604e-06 2024-09-16T20:50:33.570 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.570 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.571 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.571 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.571 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1819 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.571 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.892548+0000", 2024-09-16T20:50:33.571 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4294967295.9964585, 2024-09-16T20:50:33.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003632358, 2024-09-16T20:50:33.571 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.571 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.571 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.571 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.571 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.571 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.572 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1819 2024-09-16T20:50:33.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.572 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.572 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.572 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892548+0000", 2024-09-16T20:50:33.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.572 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.572 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892548+0000", 2024-09-16T20:50:33.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.573 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.573 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.573 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892548+0000", 2024-09-16T20:50:33.573 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3300000000000001e-07 2024-09-16T20:50:33.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.573 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.573 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.573 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892549+0000", 2024-09-16T20:50:33.573 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9600000000000008e-07 2024-09-16T20:50:33.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.573 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.573 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.573 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892552+0000", 2024-09-16T20:50:33.574 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9500000000000001e-06 2024-09-16T20:50:33.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.574 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.574 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892555+0000", 2024-09-16T20:50:33.574 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6139999999999999e-06 2024-09-16T20:50:33.574 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.574 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.574 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.574 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1628 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.574 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.869033+0000", 2024-09-16T20:50:33.575 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.019974090999999999, 2024-09-16T20:50:33.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.027179244000000002, 2024-09-16T20:50:33.575 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.575 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.575 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.575 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.575 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.575 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.575 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1628 2024-09-16T20:50:33.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.575 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.575 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.869033+0000", 2024-09-16T20:50:33.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.869033+0000", 2024-09-16T20:50:33.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.869146+0000", 2024-09-16T20:50:33.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000112911 2024-09-16T20:50:33.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.577 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.869149+0000", 2024-09-16T20:50:33.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7530000000000002e-06 2024-09-16T20:50:33.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.577 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.577 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.869154+0000", 2024-09-16T20:50:33.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1610000000000002e-06 2024-09-16T20:50:33.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.577 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.577 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.869157+0000", 2024-09-16T20:50:33.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6289999999999999e-06 2024-09-16T20:50:33.577 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.577 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.578 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.578 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1660 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.578 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.872265+0000", 2024-09-16T20:50:33.578 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.016741889999999999, 2024-09-16T20:50:33.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.023970789999999999, 2024-09-16T20:50:33.578 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.578 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.578 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.578 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.578 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.578 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.578 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1660 2024-09-16T20:50:33.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.579 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.579 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872265+0000", 2024-09-16T20:50:33.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.579 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872265+0000", 2024-09-16T20:50:33.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872372+0000", 2024-09-16T20:50:33.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000107366 2024-09-16T20:50:33.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872375+0000", 2024-09-16T20:50:33.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9390000000000002e-06 2024-09-16T20:50:33.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872381+0000", 2024-09-16T20:50:33.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4049999999999999e-06 2024-09-16T20:50:33.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872384+0000", 2024-09-16T20:50:33.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5279999999999999e-06 2024-09-16T20:50:33.581 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.581 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.581 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.581 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1692 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.581 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.880470+0000", 2024-09-16T20:50:33.581 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0085372690000000001, 2024-09-16T20:50:33.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015789243000000001, 2024-09-16T20:50:33.581 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.582 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.582 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.582 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.582 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.582 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.582 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1692 2024-09-16T20:50:33.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.582 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880470+0000", 2024-09-16T20:50:33.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880470+0000", 2024-09-16T20:50:33.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880470+0000", 2024-09-16T20:50:33.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.65e-07 2024-09-16T20:50:33.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880471+0000", 2024-09-16T20:50:33.584 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1850000000000001e-06 2024-09-16T20:50:33.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.584 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.584 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.584 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880474+0000", 2024-09-16T20:50:33.584 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.072e-06 2024-09-16T20:50:33.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.584 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.584 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.584 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880477+0000", 2024-09-16T20:50:33.584 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4710000000000001e-06 2024-09-16T20:50:33.584 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.584 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.585 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.585 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1724 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.585 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.880840+0000", 2024-09-16T20:50:33.585 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0081669900000000007, 2024-09-16T20:50:33.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015449592, 2024-09-16T20:50:33.585 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.585 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.585 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.585 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.585 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.585 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.585 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1724 2024-09-16T20:50:33.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.586 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880840+0000", 2024-09-16T20:50:33.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880840+0000", 2024-09-16T20:50:33.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880840+0000", 2024-09-16T20:50:33.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0900000000000001e-07 2024-09-16T20:50:33.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880841+0000", 2024-09-16T20:50:33.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.119e-06 2024-09-16T20:50:33.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880844+0000", 2024-09-16T20:50:33.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8949999999999998e-06 2024-09-16T20:50:33.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880847+0000", 2024-09-16T20:50:33.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.554e-06 2024-09-16T20:50:33.588 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.588 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.588 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.588 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1756 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.588 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.886456+0000", 2024-09-16T20:50:33.588 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0025513369999999999, 2024-09-16T20:50:33.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0098851110000000002, 2024-09-16T20:50:33.588 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.589 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.589 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.589 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.589 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.589 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.589 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1756 2024-09-16T20:50:33.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.589 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.589 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886456+0000", 2024-09-16T20:50:33.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.590 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.590 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886456+0000", 2024-09-16T20:50:33.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.590 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.590 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886456+0000", 2024-09-16T20:50:33.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8000000000000001e-07 2024-09-16T20:50:33.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.590 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.590 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886457+0000", 2024-09-16T20:50:33.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3480000000000001e-06 2024-09-16T20:50:33.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.591 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.591 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886461+0000", 2024-09-16T20:50:33.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.551e-06 2024-09-16T20:50:33.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.591 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.591 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886464+0000", 2024-09-16T20:50:33.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8040000000000002e-06 2024-09-16T20:50:33.591 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.591 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.591 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.592 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1788 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.592 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.886816+0000", 2024-09-16T20:50:33.592 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0021910369999999998, 2024-09-16T20:50:33.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0095764070000000003, 2024-09-16T20:50:33.592 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.592 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.592 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.592 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.592 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.592 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.592 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1788 2024-09-16T20:50:33.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.593 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.593 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.593 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886816+0000", 2024-09-16T20:50:33.593 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.593 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.593 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886816+0000", 2024-09-16T20:50:33.593 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.593 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.593 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886816+0000", 2024-09-16T20:50:33.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3699999999999999e-07 2024-09-16T20:50:33.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.594 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.594 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886817+0000", 2024-09-16T20:50:33.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1510000000000001e-06 2024-09-16T20:50:33.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.594 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.594 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886820+0000", 2024-09-16T20:50:33.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.819e-06 2024-09-16T20:50:33.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.595 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.595 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886823+0000", 2024-09-16T20:50:33.595 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7259999999999998e-06 2024-09-16T20:50:33.595 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.595 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.595 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.595 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1820 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.595 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.892558+0000", 2024-09-16T20:50:33.595 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4294967295.996449, 2024-09-16T20:50:33.595 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0038823709999999999, 2024-09-16T20:50:33.595 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.595 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.596 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.596 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.596 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.596 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.596 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1820 2024-09-16T20:50:33.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.596 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.596 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.596 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.596 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892558+0000", 2024-09-16T20:50:33.596 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.596 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.597 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.597 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892558+0000", 2024-09-16T20:50:33.597 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.597 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.597 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.597 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.597 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892558+0000", 2024-09-16T20:50:33.597 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5199999999999998e-07 2024-09-16T20:50:33.597 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.597 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.597 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.597 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892560+0000", 2024-09-16T20:50:33.597 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1939999999999999e-06 2024-09-16T20:50:33.597 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.598 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.598 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892564+0000", 2024-09-16T20:50:33.598 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.267e-06 2024-09-16T20:50:33.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.598 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.598 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892566+0000", 2024-09-16T20:50:33.598 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5890000000000001e-06 2024-09-16T20:50:33.598 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.598 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.598 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.599 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1629 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.599 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.869179+0000", 2024-09-16T20:50:33.599 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.019827576999999999, 2024-09-16T20:50:33.599 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.027313090000000002, 2024-09-16T20:50:33.599 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.599 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.599 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.599 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.599 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.599 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.599 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1629 2024-09-16T20:50:33.599 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.599 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.599 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.600 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.869179+0000", 2024-09-16T20:50:33.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.600 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.600 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.600 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.869179+0000", 2024-09-16T20:50:33.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.600 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.600 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.600 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.869180+0000", 2024-09-16T20:50:33.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:50:33.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.601 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.869182+0000", 2024-09-16T20:50:33.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7770000000000001e-06 2024-09-16T20:50:33.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.601 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.869185+0000", 2024-09-16T20:50:33.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5429999999999998e-06 2024-09-16T20:50:33.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.601 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.869188+0000", 2024-09-16T20:50:33.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.005e-06 2024-09-16T20:50:33.602 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.602 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.602 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.602 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1661 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.602 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.872390+0000", 2024-09-16T20:50:33.602 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.01661665, 2024-09-16T20:50:33.602 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.024142469, 2024-09-16T20:50:33.602 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.602 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.602 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.602 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.602 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.602 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.603 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1661 2024-09-16T20:50:33.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.603 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.603 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.603 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872390+0000", 2024-09-16T20:50:33.603 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.603 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.603 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872390+0000", 2024-09-16T20:50:33.603 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.604 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872500+0000", 2024-09-16T20:50:33.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000109543 2024-09-16T20:50:33.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.604 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872503+0000", 2024-09-16T20:50:33.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1030000000000002e-06 2024-09-16T20:50:33.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.604 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872508+0000", 2024-09-16T20:50:33.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0010000000000001e-06 2024-09-16T20:50:33.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872512+0000", 2024-09-16T20:50:33.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1620000000000001e-06 2024-09-16T20:50:33.605 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.605 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.605 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.605 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1693 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.605 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.880480+0000", 2024-09-16T20:50:33.605 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0085271740000000002, 2024-09-16T20:50:33.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.016081142999999999, 2024-09-16T20:50:33.606 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.606 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.606 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.606 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.606 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.606 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.606 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1693 2024-09-16T20:50:33.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.606 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.606 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.606 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880480+0000", 2024-09-16T20:50:33.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.607 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.607 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880480+0000", 2024-09-16T20:50:33.607 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.607 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.607 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880480+0000", 2024-09-16T20:50:33.607 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0599999999999999e-07 2024-09-16T20:50:33.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.607 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.607 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880481+0000", 2024-09-16T20:50:33.608 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1340000000000001e-06 2024-09-16T20:50:33.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.608 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.608 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880484+0000", 2024-09-16T20:50:33.608 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9560000000000002e-06 2024-09-16T20:50:33.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.608 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.608 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880487+0000", 2024-09-16T20:50:33.608 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5189999999999999e-06 2024-09-16T20:50:33.608 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.608 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.608 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.609 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1725 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.609 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.880850+0000", 2024-09-16T20:50:33.609 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0081572970000000009, 2024-09-16T20:50:33.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015734438999999999, 2024-09-16T20:50:33.609 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.609 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.609 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.609 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.609 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.609 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.609 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1725 2024-09-16T20:50:33.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.610 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.610 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.610 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880850+0000", 2024-09-16T20:50:33.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.610 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.610 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880850+0000", 2024-09-16T20:50:33.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.610 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.610 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880850+0000", 2024-09-16T20:50:33.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0699999999999999e-07 2024-09-16T20:50:33.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.611 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.611 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880851+0000", 2024-09-16T20:50:33.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.189e-06 2024-09-16T20:50:33.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.611 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.611 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880854+0000", 2024-09-16T20:50:33.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.915e-06 2024-09-16T20:50:33.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.611 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.611 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880857+0000", 2024-09-16T20:50:33.612 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5289999999999998e-06 2024-09-16T20:50:33.612 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.612 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.612 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.612 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1757 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.612 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.886467+0000", 2024-09-16T20:50:33.612 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0025399340000000002, 2024-09-16T20:50:33.612 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.010145471, 2024-09-16T20:50:33.612 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.612 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.612 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.612 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.613 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.613 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.613 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1757 2024-09-16T20:50:33.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.613 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.613 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.613 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886467+0000", 2024-09-16T20:50:33.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.613 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.613 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886467+0000", 2024-09-16T20:50:33.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.614 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.614 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886467+0000", 2024-09-16T20:50:33.614 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1800000000000002e-07 2024-09-16T20:50:33.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.614 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.614 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886469+0000", 2024-09-16T20:50:33.614 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2190000000000001e-06 2024-09-16T20:50:33.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.614 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.615 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886472+0000", 2024-09-16T20:50:33.615 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9620000000000002e-06 2024-09-16T20:50:33.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.615 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.615 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.615 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886474+0000", 2024-09-16T20:50:33.615 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6180000000000002e-06 2024-09-16T20:50:33.615 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.615 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.615 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.615 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.615 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1789 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.615 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.886826+0000", 2024-09-16T20:50:33.615 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.002181157, 2024-09-16T20:50:33.616 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0098306239999999996, 2024-09-16T20:50:33.616 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.616 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.616 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.616 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.616 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.616 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.616 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1789 2024-09-16T20:50:33.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.616 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.616 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.616 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.616 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886826+0000", 2024-09-16T20:50:33.616 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.617 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.617 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886826+0000", 2024-09-16T20:50:33.617 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.617 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.617 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886826+0000", 2024-09-16T20:50:33.617 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1799999999999999e-07 2024-09-16T20:50:33.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.617 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.617 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886827+0000", 2024-09-16T20:50:33.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.136e-06 2024-09-16T20:50:33.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886830+0000", 2024-09-16T20:50:33.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8650000000000001e-06 2024-09-16T20:50:33.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886833+0000", 2024-09-16T20:50:33.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5469999999999999e-06 2024-09-16T20:50:33.618 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.618 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.618 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.619 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1821 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.619 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.892570+0000", 2024-09-16T20:50:33.619 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4294967295.9964375, 2024-09-16T20:50:33.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.004138821, 2024-09-16T20:50:33.619 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.619 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.619 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.619 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.619 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.619 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.619 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1821 2024-09-16T20:50:33.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.619 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.620 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.620 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892570+0000", 2024-09-16T20:50:33.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.620 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.620 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892570+0000", 2024-09-16T20:50:33.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.620 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.620 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892570+0000", 2024-09-16T20:50:33.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1100000000000002e-07 2024-09-16T20:50:33.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.621 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.621 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892571+0000", 2024-09-16T20:50:33.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2100000000000001e-06 2024-09-16T20:50:33.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.621 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.621 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892574+0000", 2024-09-16T20:50:33.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9529999999999999e-06 2024-09-16T20:50:33.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.621 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.622 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892577+0000", 2024-09-16T20:50:33.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.576e-06 2024-09-16T20:50:33.622 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.622 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.622 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.622 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1630 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.622 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.869192+0000", 2024-09-16T20:50:33.622 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.019815121000000002, 2024-09-16T20:50:33.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.027566839999999999, 2024-09-16T20:50:33.622 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.622 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.622 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.622 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.623 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.623 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.623 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1630 2024-09-16T20:50:33.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.623 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.869192+0000", 2024-09-16T20:50:33.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.869192+0000", 2024-09-16T20:50:33.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.624 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.624 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.624 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.869279+0000", 2024-09-16T20:50:33.624 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7267000000000003e-05 2024-09-16T20:50:33.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.624 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.624 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.624 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.869282+0000", 2024-09-16T20:50:33.624 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8729999999999999e-06 2024-09-16T20:50:33.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.624 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.624 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.624 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.869287+0000", 2024-09-16T20:50:33.624 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2939999999999996e-06 2024-09-16T20:50:33.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.625 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.625 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.869291+0000", 2024-09-16T20:50:33.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.523e-06 2024-09-16T20:50:33.625 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.625 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.625 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.625 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1662 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.625 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.872528+0000", 2024-09-16T20:50:33.625 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.016479056999999998, 2024-09-16T20:50:33.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.024281636999999998, 2024-09-16T20:50:33.626 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.626 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.626 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.626 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.626 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.626 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.626 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1662 2024-09-16T20:50:33.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.626 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.626 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.626 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872528+0000", 2024-09-16T20:50:33.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.627 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.627 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872528+0000", 2024-09-16T20:50:33.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.627 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.627 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872528+0000", 2024-09-16T20:50:33.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.34e-07 2024-09-16T20:50:33.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.627 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.627 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872530+0000", 2024-09-16T20:50:33.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.668e-06 2024-09-16T20:50:33.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872534+0000", 2024-09-16T20:50:33.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8410000000000002e-06 2024-09-16T20:50:33.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872537+0000", 2024-09-16T20:50:33.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9469999999999999e-06 2024-09-16T20:50:33.628 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.628 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.628 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.629 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1694 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.629 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.880490+0000", 2024-09-16T20:50:33.629 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0085174580000000003, 2024-09-16T20:50:33.629 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.016359314999999999, 2024-09-16T20:50:33.629 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.629 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.629 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.629 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.629 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.629 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.629 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1694 2024-09-16T20:50:33.629 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.629 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.629 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880490+0000", 2024-09-16T20:50:33.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880490+0000", 2024-09-16T20:50:33.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880490+0000", 2024-09-16T20:50:33.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0100000000000001e-07 2024-09-16T20:50:33.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.631 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.631 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.631 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880491+0000", 2024-09-16T20:50:33.631 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0809999999999999e-06 2024-09-16T20:50:33.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.631 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.631 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.631 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880494+0000", 2024-09-16T20:50:33.631 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.853e-06 2024-09-16T20:50:33.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.631 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.631 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.631 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880497+0000", 2024-09-16T20:50:33.631 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1839999999999999e-06 2024-09-16T20:50:33.632 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.632 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.632 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.632 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.632 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1726 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.632 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.880860+0000", 2024-09-16T20:50:33.632 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0081474630000000006, 2024-09-16T20:50:33.632 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.016013146999999998, 2024-09-16T20:50:33.632 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.632 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.632 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.632 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.632 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.633 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.633 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1726 2024-09-16T20:50:33.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.633 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.633 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.633 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.633 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880860+0000", 2024-09-16T20:50:33.633 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.633 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.633 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.633 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880860+0000", 2024-09-16T20:50:33.633 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.634 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.634 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880860+0000", 2024-09-16T20:50:33.634 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.05e-07 2024-09-16T20:50:33.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.634 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.634 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880861+0000", 2024-09-16T20:50:33.634 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6200000000000006e-07 2024-09-16T20:50:33.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.634 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.634 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880864+0000", 2024-09-16T20:50:33.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9730000000000002e-06 2024-09-16T20:50:33.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.635 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880866+0000", 2024-09-16T20:50:33.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4949999999999998e-06 2024-09-16T20:50:33.635 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.635 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.635 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.635 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1758 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.635 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.886478+0000", 2024-09-16T20:50:33.635 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0025291570000000002, 2024-09-16T20:50:33.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.010420716999999999, 2024-09-16T20:50:33.636 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.636 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.636 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.636 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.636 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.636 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.636 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1758 2024-09-16T20:50:33.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.636 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.636 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.636 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886478+0000", 2024-09-16T20:50:33.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.637 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.637 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886478+0000", 2024-09-16T20:50:33.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.637 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.637 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886478+0000", 2024-09-16T20:50:33.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1899999999999999e-07 2024-09-16T20:50:33.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.637 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.637 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886479+0000", 2024-09-16T20:50:33.638 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0440000000000001e-06 2024-09-16T20:50:33.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.638 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.638 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.638 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886482+0000", 2024-09-16T20:50:33.638 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8880000000000001e-06 2024-09-16T20:50:33.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.638 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.638 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.638 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886484+0000", 2024-09-16T20:50:33.638 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.497e-06 2024-09-16T20:50:33.638 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.638 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.638 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.639 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1790 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.639 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.886851+0000", 2024-09-16T20:50:33.639 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0021555820000000001, 2024-09-16T20:50:33.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.010069605000000001, 2024-09-16T20:50:33.639 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.639 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.639 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.639 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.639 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.639 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.639 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1790 2024-09-16T20:50:33.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.640 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.640 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.640 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886851+0000", 2024-09-16T20:50:33.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.640 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.640 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886851+0000", 2024-09-16T20:50:33.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.640 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.640 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886852+0000", 2024-09-16T20:50:33.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1100000000000001e-07 2024-09-16T20:50:33.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886856+0000", 2024-09-16T20:50:33.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3200000000000001e-06 2024-09-16T20:50:33.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886860+0000", 2024-09-16T20:50:33.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4520000000000002e-06 2024-09-16T20:50:33.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.642 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.642 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886862+0000", 2024-09-16T20:50:33.642 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7609999999999999e-06 2024-09-16T20:50:33.642 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.642 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.642 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.642 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1822 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.642 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.892580+0000", 2024-09-16T20:50:33.642 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4294967295.9964275, 2024-09-16T20:50:33.642 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0043800239999999997, 2024-09-16T20:50:33.642 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.642 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.643 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.643 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.643 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.643 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.643 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1822 2024-09-16T20:50:33.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.643 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.643 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892580+0000", 2024-09-16T20:50:33.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.643 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892580+0000", 2024-09-16T20:50:33.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.644 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.644 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892580+0000", 2024-09-16T20:50:33.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.29e-07 2024-09-16T20:50:33.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.644 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.644 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892581+0000", 2024-09-16T20:50:33.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0610000000000001e-06 2024-09-16T20:50:33.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.644 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892584+0000", 2024-09-16T20:50:33.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9790000000000002e-06 2024-09-16T20:50:33.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892586+0000", 2024-09-16T20:50:33.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5689999999999999e-06 2024-09-16T20:50:33.645 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.645 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.645 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.645 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1631 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.645 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.869298+0000", 2024-09-16T20:50:33.646 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.019709357, 2024-09-16T20:50:33.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.027711929999999999, 2024-09-16T20:50:33.646 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.646 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.646 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.646 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.646 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.646 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.646 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1631 2024-09-16T20:50:33.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.646 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.646 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.869298+0000", 2024-09-16T20:50:33.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.647 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.647 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.869298+0000", 2024-09-16T20:50:33.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.647 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.647 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.869404+0000", 2024-09-16T20:50:33.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000106047 2024-09-16T20:50:33.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.647 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.647 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.869407+0000", 2024-09-16T20:50:33.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9969999999999999e-06 2024-09-16T20:50:33.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.648 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.869412+0000", 2024-09-16T20:50:33.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2800000000000003e-06 2024-09-16T20:50:33.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.648 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.869416+0000", 2024-09-16T20:50:33.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.534e-06 2024-09-16T20:50:33.648 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.648 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.648 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.649 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1663 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.649 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.872540+0000", 2024-09-16T20:50:33.649 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.016466527000000002, 2024-09-16T20:50:33.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.024520310999999999, 2024-09-16T20:50:33.649 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.649 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.649 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.649 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.649 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.649 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.649 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1663 2024-09-16T20:50:33.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.650 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.650 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.650 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872540+0000", 2024-09-16T20:50:33.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.650 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.650 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872540+0000", 2024-09-16T20:50:33.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.650 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.651 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872633+0000", 2024-09-16T20:50:33.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2663000000000004e-05 2024-09-16T20:50:33.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.651 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.651 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872636+0000", 2024-09-16T20:50:33.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7920000000000001e-06 2024-09-16T20:50:33.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.651 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.651 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872641+0000", 2024-09-16T20:50:33.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3759999999999999e-06 2024-09-16T20:50:33.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.652 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.652 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872645+0000", 2024-09-16T20:50:33.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.32e-06 2024-09-16T20:50:33.652 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.652 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.652 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.652 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1695 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.652 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.880504+0000", 2024-09-16T20:50:33.652 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0085033929999999997, 2024-09-16T20:50:33.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.016604908000000002, 2024-09-16T20:50:33.652 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.652 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.653 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.653 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.653 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.653 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.653 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1695 2024-09-16T20:50:33.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.653 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.653 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880504+0000", 2024-09-16T20:50:33.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.653 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880504+0000", 2024-09-16T20:50:33.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880504+0000", 2024-09-16T20:50:33.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2100000000000001e-07 2024-09-16T20:50:33.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880505+0000", 2024-09-16T20:50:33.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.356e-06 2024-09-16T20:50:33.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.655 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880509+0000", 2024-09-16T20:50:33.655 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4029999999999999e-06 2024-09-16T20:50:33.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.655 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.655 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.655 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880511+0000", 2024-09-16T20:50:33.655 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5770000000000001e-06 2024-09-16T20:50:33.655 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.655 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.655 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.655 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.655 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1727 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.655 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.880879+0000", 2024-09-16T20:50:33.655 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0081284319999999997, 2024-09-16T20:50:33.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.016280171999999999, 2024-09-16T20:50:33.656 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.656 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.656 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.656 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.656 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.656 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.656 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1727 2024-09-16T20:50:33.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.656 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.656 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.656 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880879+0000", 2024-09-16T20:50:33.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.657 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.657 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880879+0000", 2024-09-16T20:50:33.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.657 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.657 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880879+0000", 2024-09-16T20:50:33.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2e-07 2024-09-16T20:50:33.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.657 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880880+0000", 2024-09-16T20:50:33.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.412e-06 2024-09-16T20:50:33.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.658 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880885+0000", 2024-09-16T20:50:33.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6229999999999996e-06 2024-09-16T20:50:33.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.658 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880888+0000", 2024-09-16T20:50:33.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.791e-06 2024-09-16T20:50:33.658 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.658 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.659 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.659 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.659 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1759 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.659 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.886487+0000", 2024-09-16T20:50:33.659 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0025195360000000002, 2024-09-16T20:50:33.659 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.010697651000000001, 2024-09-16T20:50:33.659 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.659 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.659 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.659 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.659 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.659 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.659 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1759 2024-09-16T20:50:33.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.660 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.660 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.660 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886487+0000", 2024-09-16T20:50:33.660 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.660 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.660 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886487+0000", 2024-09-16T20:50:33.660 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.660 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.660 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886488+0000", 2024-09-16T20:50:33.660 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1799999999999999e-07 2024-09-16T20:50:33.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.661 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.661 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886489+0000", 2024-09-16T20:50:33.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.012e-06 2024-09-16T20:50:33.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.661 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.661 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886492+0000", 2024-09-16T20:50:33.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.852e-06 2024-09-16T20:50:33.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.662 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.662 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.662 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886494+0000", 2024-09-16T20:50:33.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5560000000000002e-06 2024-09-16T20:50:33.662 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.662 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.662 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.662 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.662 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1791 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.662 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.886866+0000", 2024-09-16T20:50:33.662 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0021410999999999999, 2024-09-16T20:50:33.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.010343984000000001, 2024-09-16T20:50:33.662 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.663 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.663 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.663 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.663 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.663 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.663 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1791 2024-09-16T20:50:33.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.663 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.663 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.663 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886866+0000", 2024-09-16T20:50:33.663 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.664 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.664 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886866+0000", 2024-09-16T20:50:33.664 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.664 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.664 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.664 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886866+0000", 2024-09-16T20:50:33.664 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0800000000000001e-07 2024-09-16T20:50:33.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.664 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.664 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.664 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886867+0000", 2024-09-16T20:50:33.664 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3900000000000003e-07 2024-09-16T20:50:33.665 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.665 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.665 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.665 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886870+0000", 2024-09-16T20:50:33.665 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8660000000000002e-06 2024-09-16T20:50:33.665 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.665 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.665 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.665 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886873+0000", 2024-09-16T20:50:33.665 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.576e-06 2024-09-16T20:50:33.665 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.665 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.665 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.665 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.666 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1823 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.666 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.892590+0000", 2024-09-16T20:50:33.666 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4294967295.996417, 2024-09-16T20:50:33.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0046446589999999998, 2024-09-16T20:50:33.666 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.666 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.666 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.666 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.666 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.666 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.666 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1823 2024-09-16T20:50:33.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.667 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.667 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.667 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892590+0000", 2024-09-16T20:50:33.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.667 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.667 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892590+0000", 2024-09-16T20:50:33.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.667 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.668 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892590+0000", 2024-09-16T20:50:33.668 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1e-07 2024-09-16T20:50:33.668 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.668 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.668 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.668 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892591+0000", 2024-09-16T20:50:33.668 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.08e-06 2024-09-16T20:50:33.668 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.668 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.668 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.668 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892594+0000", 2024-09-16T20:50:33.668 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9330000000000001e-06 2024-09-16T20:50:33.668 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.668 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.669 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.669 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892597+0000", 2024-09-16T20:50:33.669 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5100000000000001e-06 2024-09-16T20:50:33.669 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.669 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.669 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.669 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.669 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1632 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.669 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.869421+0000", 2024-09-16T20:50:33.669 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.019585512999999999, 2024-09-16T20:50:33.669 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.027862155, 2024-09-16T20:50:33.669 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.669 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.670 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.670 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.670 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.670 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.670 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1632 2024-09-16T20:50:33.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.670 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.670 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.670 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.869421+0000", 2024-09-16T20:50:33.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.670 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.671 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.869421+0000", 2024-09-16T20:50:33.671 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.671 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.671 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.671 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.671 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.869532+0000", 2024-09-16T20:50:33.671 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000110192 2024-09-16T20:50:33.671 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.671 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.671 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.671 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.869535+0000", 2024-09-16T20:50:33.671 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.168e-06 2024-09-16T20:50:33.671 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.671 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.672 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.672 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.869541+0000", 2024-09-16T20:50:33.672 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.942e-06 2024-09-16T20:50:33.672 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.672 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.672 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.672 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.869544+0000", 2024-09-16T20:50:33.672 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4199999999999999e-06 2024-09-16T20:50:33.672 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.672 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.672 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.672 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.672 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.672 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1664 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.673 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.872650+0000", 2024-09-16T20:50:33.673 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.016356582000000001, 2024-09-16T20:50:33.673 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.024686657000000001, 2024-09-16T20:50:33.673 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.673 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.673 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.673 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.673 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.673 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.673 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1664 2024-09-16T20:50:33.673 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.673 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.673 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.673 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.674 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872650+0000", 2024-09-16T20:50:33.674 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.674 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.674 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.674 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.674 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872650+0000", 2024-09-16T20:50:33.674 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.674 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.674 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.674 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.674 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872758+0000", 2024-09-16T20:50:33.674 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010739900000000001 2024-09-16T20:50:33.674 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.674 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.674 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.675 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872761+0000", 2024-09-16T20:50:33.675 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0970000000000002e-06 2024-09-16T20:50:33.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.675 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.675 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.675 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872768+0000", 2024-09-16T20:50:33.675 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6579999999999996e-06 2024-09-16T20:50:33.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.675 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.675 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.675 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.872771+0000", 2024-09-16T20:50:33.675 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3720000000000001e-06 2024-09-16T20:50:33.675 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.675 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.676 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.676 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.676 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.676 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1696 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.676 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.880515+0000", 2024-09-16T20:50:33.676 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0084923749999999999, 2024-09-16T20:50:33.676 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.016873719999999998, 2024-09-16T20:50:33.676 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.676 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.676 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.676 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.676 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.676 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.676 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1696 2024-09-16T20:50:33.677 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.677 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.677 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.677 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.677 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880515+0000", 2024-09-16T20:50:33.677 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.677 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.677 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.677 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.677 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880515+0000", 2024-09-16T20:50:33.677 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.677 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.677 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.677 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.677 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880515+0000", 2024-09-16T20:50:33.678 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.11e-07 2024-09-16T20:50:33.678 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.678 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.678 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.678 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880516+0000", 2024-09-16T20:50:33.678 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.850000000000001e-07 2024-09-16T20:50:33.678 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.678 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.678 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.678 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880519+0000", 2024-09-16T20:50:33.678 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0630000000000002e-06 2024-09-16T20:50:33.678 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.678 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.678 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.679 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880521+0000", 2024-09-16T20:50:33.679 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5790000000000002e-06 2024-09-16T20:50:33.679 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.679 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.679 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.679 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.679 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1728 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.679 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.880891+0000", 2024-09-16T20:50:33.679 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0081157629999999998, 2024-09-16T20:50:33.679 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.016545043999999998, 2024-09-16T20:50:33.679 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.679 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.679 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.680 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.680 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.680 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.680 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1728 2024-09-16T20:50:33.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.680 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.680 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.680 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.680 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880891+0000", 2024-09-16T20:50:33.680 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.680 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.680 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.680 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880891+0000", 2024-09-16T20:50:33.681 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.681 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.681 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.681 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.681 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880891+0000", 2024-09-16T20:50:33.681 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3099999999999999e-07 2024-09-16T20:50:33.681 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.681 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.681 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.681 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880893+0000", 2024-09-16T20:50:33.681 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0330000000000001e-06 2024-09-16T20:50:33.681 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.681 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.682 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.682 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880896+0000", 2024-09-16T20:50:33.682 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0089999999999999e-06 2024-09-16T20:50:33.682 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.682 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.682 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.682 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.880898+0000", 2024-09-16T20:50:33.682 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5900000000000002e-06 2024-09-16T20:50:33.682 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.682 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.682 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.682 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.682 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.682 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1760 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.682 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.886497+0000", 2024-09-16T20:50:33.683 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0025099879999999999, 2024-09-16T20:50:33.683 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.010980080999999999, 2024-09-16T20:50:33.683 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.683 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.683 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.683 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.683 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.683 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.683 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1760 2024-09-16T20:50:33.683 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.683 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.683 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.683 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.683 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886497+0000", 2024-09-16T20:50:33.684 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.684 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.684 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.684 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.684 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886497+0000", 2024-09-16T20:50:33.684 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.684 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.684 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.684 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.684 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886497+0000", 2024-09-16T20:50:33.684 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.23e-07 2024-09-16T20:50:33.684 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.684 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.684 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.685 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886498+0000", 2024-09-16T20:50:33.685 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7499999999999998e-07 2024-09-16T20:50:33.685 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.685 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.685 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.685 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886501+0000", 2024-09-16T20:50:33.685 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9000000000000002e-06 2024-09-16T20:50:33.685 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.685 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.685 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.685 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886504+0000", 2024-09-16T20:50:33.685 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5909999999999998e-06 2024-09-16T20:50:33.685 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.685 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.686 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.686 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.686 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.686 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1792 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.686 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.886876+0000", 2024-09-16T20:50:33.686 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0021312169999999999, 2024-09-16T20:50:33.686 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.010628039000000001, 2024-09-16T20:50:33.686 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.686 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.686 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.686 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.686 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.686 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.686 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1792 2024-09-16T20:50:33.687 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.687 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.687 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.687 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.687 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886876+0000", 2024-09-16T20:50:33.687 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.687 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.687 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.687 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.687 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886876+0000", 2024-09-16T20:50:33.687 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.687 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.687 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.687 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.688 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886876+0000", 2024-09-16T20:50:33.688 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1400000000000001e-07 2024-09-16T20:50:33.688 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.688 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.688 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.688 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886877+0000", 2024-09-16T20:50:33.688 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7000000000000003e-07 2024-09-16T20:50:33.688 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.688 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.688 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.688 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886880+0000", 2024-09-16T20:50:33.688 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8949999999999998e-06 2024-09-16T20:50:33.688 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.688 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.689 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.689 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.886882+0000", 2024-09-16T20:50:33.689 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5849999999999998e-06 2024-09-16T20:50:33.689 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.689 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.689 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.689 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.689 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.689 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5720.0:1824 3.7 3.3d4bf6c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:33.689 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:32.892600+0000", 2024-09-16T20:50:33.689 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4294967295.9964075, 2024-09-16T20:50:33.689 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0049275270000000001, 2024-09-16T20:50:33.689 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:33.689 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:33.690 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-16T20:50:33.690 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:33.690 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5720", 2024-09-16T20:50:33.690 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2936975192", 2024-09-16T20:50:33.690 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1824 2024-09-16T20:50:33.690 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.690 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:33.690 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.690 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:33.690 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892600+0000", 2024-09-16T20:50:33.690 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.690 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.690 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.690 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:33.691 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892600+0000", 2024-09-16T20:50:33.691 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:33.691 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.691 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.691 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:33.691 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892600+0000", 2024-09-16T20:50:33.691 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1e-07 2024-09-16T20:50:33.691 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.691 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.691 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:33.691 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892601+0000", 2024-09-16T20:50:33.691 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0529999999999999e-06 2024-09-16T20:50:33.691 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.691 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.692 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:33.692 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892605+0000", 2024-09-16T20:50:33.692 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1559999999999997e-06 2024-09-16T20:50:33.692 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:33.692 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:33.692 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:33.692 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:32.892608+0000", 2024-09-16T20:50:33.692 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4940000000000002e-06 2024-09-16T20:50:33.692 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.692 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:33.692 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.692 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:33.692 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-16T20:50:33.692 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 227 2024-09-16T20:50:33.693 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:33.693 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T20:50:33.694 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:33.693+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:33.695 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:50:33.698 INFO:teuthology.orchestra.run.smithi139.stdout:5888 write 0x6e08e00 thru 0x6e0e967 (0x5b68 bytes) 2024-09-16T20:50:33.698 INFO:teuthology.orchestra.run.smithi139.stdout:5889 read 0x3e1000 thru 0x3e1d83 (0xd84 bytes) 2024-09-16T20:50:33.698 INFO:teuthology.orchestra.run.smithi139.stdout:5890 write 0xcf39800 thru 0xcf43375 (0x9b76 bytes) 2024-09-16T20:50:33.698 INFO:teuthology.orchestra.run.smithi139.stdout:5892 write 0xed5a800 thru 0xed65d33 (0xb534 bytes) 2024-09-16T20:50:33.698 INFO:teuthology.orchestra.run.smithi139.stdout:5893 punch from 0xa5d5400 to 0xa5dce00, (0x7a00 bytes) 2024-09-16T20:50:33.698 INFO:teuthology.orchestra.run.smithi139.stdout:5895 punch from 0x542be00 to 0x5437c00, (0xbe00 bytes) 2024-09-16T20:50:33.698 INFO:teuthology.orchestra.run.smithi139.stdout:5896 write 0x2709800 thru 0x270b49b (0x1c9c bytes) 2024-09-16T20:50:33.698 INFO:teuthology.orchestra.run.smithi139.stdout:5897 punch from 0x6826a00 to 0x6827600, (0xc00 bytes) 2024-09-16T20:50:33.698 INFO:teuthology.orchestra.run.smithi139.stdout:5898 trunc from 0xed65e00 to 0xaaa5e00 2024-09-16T20:50:33.698 INFO:teuthology.orchestra.run.smithi139.stdout:5899 punch from 0x5d7ea00 to 0x5d8d800, (0xee00 bytes) 2024-09-16T20:50:33.698 INFO:teuthology.orchestra.run.smithi139.stdout:5903 read 0x49d2000 thru 0x49d34fd (0x14fe bytes) 2024-09-16T20:50:33.698 INFO:teuthology.orchestra.run.smithi139.stdout:5904 write 0x71e7200 thru 0x71e806c (0xe6d bytes) 2024-09-16T20:50:33.698 INFO:teuthology.orchestra.run.smithi139.stdout:5906 read 0x78aa200 thru 0x78b152b (0x732c bytes) 2024-09-16T20:50:33.699 INFO:teuthology.orchestra.run.smithi139.stdout:5907 trunc from 0xaaa5e00 to 0x14bea00 2024-09-16T20:50:33.699 INFO:teuthology.orchestra.run.smithi139.stdout:5911 trunc from 0x14bea00 to 0x4b98400 2024-09-16T20:50:33.699 INFO:teuthology.orchestra.run.smithi139.stdout:5912 write 0x666c400 thru 0x667606e (0x9c6f bytes) 2024-09-16T20:50:33.699 INFO:teuthology.orchestra.run.smithi139.stdout:5913 read 0x5dac400 thru 0x5dbab4e (0xe74f bytes) 2024-09-16T20:50:33.699 INFO:teuthology.orchestra.run.smithi139.stdout:5916 trunc from 0x6676200 to 0x1e5fc00 2024-09-16T20:50:33.699 INFO:teuthology.orchestra.run.smithi139.stdout:5919 read 0x1e00c00 thru 0x1e0a2be (0x96bf bytes) 2024-09-16T20:50:33.699 INFO:teuthology.orchestra.run.smithi139.stdout:5921 write 0x231600 thru 0x23dab4 (0xc4b5 bytes) 2024-09-16T20:50:33.699 INFO:teuthology.orchestra.run.smithi139.stdout:5923 trunc from 0x1e5fc00 to 0x9329200 2024-09-16T20:50:33.699 INFO:teuthology.orchestra.run.smithi139.stdout:5924 write 0x2f7a200 thru 0x2f8779d (0xd59e bytes) 2024-09-16T20:50:33.699 INFO:teuthology.orchestra.run.smithi139.stdout:5926 read 0x847ae00 thru 0x8483718 (0x8919 bytes) 2024-09-16T20:50:33.699 INFO:teuthology.orchestra.run.smithi139.stdout:5933 trunc from 0x9329200 to 0x2b20e00 2024-09-16T20:50:33.699 INFO:teuthology.orchestra.run.smithi139.stdout:5939 trunc from 0x2b20e00 to 0xb292c00 2024-09-16T20:50:33.699 INFO:teuthology.orchestra.run.smithi139.stdout:5941 punch from 0x7c72400 to 0x7c80c00, (0xe800 bytes) 2024-09-16T20:50:33.699 INFO:teuthology.orchestra.run.smithi139.stdout:5942 read 0x6379000 thru 0x6382984 (0x9985 bytes) 2024-09-16T20:50:33.700 INFO:teuthology.orchestra.run.smithi139.stdout:5943 clone 46 order 25 su 131072 sc 10 2024-09-16T20:50:33.790 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T20:50:33.812 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:33.812 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:50:33.812 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:50:33.812 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:50:33.812 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:33.821 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:50:33.828 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:50:33.828 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:50:33.828 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:33.827+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:33.849 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:50:33.849 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:50:33.865 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:50:33.865 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:50:33.882 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:50:33.883 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:50:33.896 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:33.895+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:33.900 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:50:33.901 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:50:33.918 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:50:33.918 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:50:33.929 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:33.929 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:33.938 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:50:33.997 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:33.995+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:34.050 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:34.050 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:34.059 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:50:34.098 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:34.097+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:34.169 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:34.169 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:34.178 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:50:34.190 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #44, image image_client.0-clone44 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent45 2024-09-16T20:50:34.221 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:34.221+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:34.286 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:34.286 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:34.294 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:50:34.335 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:34.335+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:34.382 INFO:teuthology.orchestra.run.smithi139.stdout:5944 read 0x61a8200 thru 0x61a9fd8 (0x1dd9 bytes) 2024-09-16T20:50:34.383 INFO:teuthology.orchestra.run.smithi139.stdout:5945 trunc from 0xb292c00 to 0xad84200 2024-09-16T20:50:34.399 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:34.398+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:34.419 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:34.420 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:34.430 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:50:34.489 INFO:teuthology.orchestra.run.smithi139.stdout:5947 write 0x997d400 thru 0x997d661 (0x262 bytes) 2024-09-16T20:50:34.490 INFO:teuthology.orchestra.run.smithi139.stdout:5948 trunc from 0xad84200 to 0xa1a0c00 2024-09-16T20:50:34.513 INFO:teuthology.orchestra.run.smithi139.stdout:5949 trunc from 0xa1a0c00 to 0x6558000 2024-09-16T20:50:34.516 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:34.515+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:34.533 INFO:teuthology.orchestra.run.smithi139.stdout:5954 read 0x3affe00 thru 0x3b0ba7c (0xbc7d bytes) 2024-09-16T20:50:34.533 INFO:teuthology.orchestra.run.smithi139.stdout:5955 write 0x2cfa200 thru 0x2cfde6e (0x3c6f bytes) 2024-09-16T20:50:34.534 INFO:teuthology.orchestra.run.smithi139.stdout:5958 read 0x5d37000 thru 0x5d46062 (0xf063 bytes) 2024-09-16T20:50:34.535 INFO:teuthology.orchestra.run.smithi139.stdout:5960 read 0x1491a00 thru 0x14a12ce (0xf8cf bytes) 2024-09-16T20:50:34.539 INFO:teuthology.orchestra.run.smithi139.stdout:5961 read 0x1ce3400 thru 0x1ce8608 (0x5209 bytes) 2024-09-16T20:50:34.539 INFO:teuthology.orchestra.run.smithi139.stdout:5962 read 0x11e2c00 thru 0x11ea066 (0x7467 bytes) 2024-09-16T20:50:34.541 INFO:teuthology.orchestra.run.smithi139.stdout:5963 read 0x4e53800 thru 0x4e58720 (0x4f21 bytes) 2024-09-16T20:50:34.542 INFO:teuthology.orchestra.run.smithi139.stdout:5965 read 0x3e0800 thru 0x3ee1c9 (0xd9ca bytes) 2024-09-16T20:50:34.564 INFO:teuthology.orchestra.run.smithi139.stdout:5966 write 0x8646200 thru 0x8651782 (0xb583 bytes) 2024-09-16T20:50:34.564 INFO:teuthology.orchestra.run.smithi139.stdout:5969 punch from 0x3a49800 to 0x3a4ba00, (0x2200 bytes) 2024-09-16T20:50:34.581 INFO:teuthology.orchestra.run.smithi139.stdout:5973 write 0xa654400 thru 0xa65ad1d (0x691e bytes) 2024-09-16T20:50:34.587 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:34.587 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:34.596 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:50:34.599 INFO:teuthology.orchestra.run.smithi139.stdout:5976 write 0xba47c00 thru 0xba4a151 (0x2552 bytes) 2024-09-16T20:50:34.601 INFO:teuthology.orchestra.run.smithi139.stdout:5978 trunc from 0xba4a200 to 0xa002800 2024-09-16T20:50:34.620 INFO:teuthology.orchestra.run.smithi139.stdout:5981 write 0x852e600 thru 0x8536881 (0x8282 bytes) 2024-09-16T20:50:34.625 INFO:teuthology.orchestra.run.smithi139.stdout:5982 trunc from 0xa002800 to 0xd586c00 2024-09-16T20:50:34.637 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:34.636+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:34.641 INFO:teuthology.orchestra.run.smithi139.stdout:5983 write 0x16a1a00 thru 0x16a9d60 (0x8361 bytes) 2024-09-16T20:50:34.642 INFO:teuthology.orchestra.run.smithi139.stdout:5987 punch from 0x48b3400 to 0x48be800, (0xb400 bytes) 2024-09-16T20:50:34.646 INFO:teuthology.orchestra.run.smithi139.stdout:5990 trunc from 0xd586c00 to 0xa1a3200 2024-09-16T20:50:34.665 INFO:teuthology.orchestra.run.smithi139.stdout:5992 read 0x3f68400 thru 0x3f6b0cb (0x2ccc bytes) 2024-09-16T20:50:34.674 INFO:teuthology.orchestra.run.smithi139.stdout:5994 trunc from 0xa1a3200 to 0x2ec6600 2024-09-16T20:50:34.702 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:34.701+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:34.711 INFO:teuthology.orchestra.run.smithi139.stdout:6000 write 0x31bba00 thru 0x31c1577 (0x5b78 bytes) 2024-09-16T20:50:34.750 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:34.750 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:34.759 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:50:34.780 INFO:teuthology.orchestra.run.smithi029.stderr:nodeep-scrub is set 2024-09-16T20:50:34.836 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:34.835+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:34.870 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:34.870 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:34.880 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:50:34.880 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #45, image image_client.0-clone45 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent46 2024-09-16T20:50:34.904 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:34.903+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:35.005 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:35.004+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:35.035 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:35.035 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:35.045 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:50:35.129 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:35.128+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:35.207 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:35.207+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:35.258 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:35.258 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:35.258 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:50:35.343 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:35.342+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:35.406 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:35.406 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:35.408 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:35.408+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:35.415 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:50:35.509 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:35.508+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:35.570 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:35.571 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:35.579 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:50:35.610 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:35.609+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:35.685 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:35.685 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:35.694 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:50:35.711 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:35.709+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:35.787 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 4, 3, 5] out_osds: [0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-16T20:50:35.787 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T20:50:35.787 INFO:tasks.thrashosds.thrasher:Removing osd 3, in_osds are: [1, 2, 4, 3, 5] 2024-09-16T20:50:35.787 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 3 2024-09-16T20:50:35.805 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:35.805 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:35.811 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:35.811+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:35.815 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:50:35.912 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:35.912+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:35.930 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T20:50:35.966 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:35.967 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:35.975 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:50:36.016 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:36.016+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:36.052 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:50:36.052 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:50:36.069 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:50:36.070 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:50:36.086 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:50:36.086 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:50:36.102 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:50:36.103 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:50:36.107 INFO:tasks.ceph.mon.a.smithi029.stderr:2024-09-16T20:50:36.106+0000 7fe0ce96d640 -1 mon.a@0(leader).osd e148 definitely_dead 0 2024-09-16T20:50:36.114 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:36.113+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:36.119 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:50:36.119 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:50:36.124 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:36.125 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:50:36.125 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:50:36.125 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:50:36.125 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.125 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:3088 3.1e 3:79e5ec01:::rbd_header.110e891a9b2b:head [watch watch cookie 140213437171488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:50:36.125 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:50:36.125 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 131.958982929, 2024-09-16T20:50:36.125 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.11895186200000001, 2024-09-16T20:50:36.125 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:36.125 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:36.126 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:36.126 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:36.126 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:50:36.126 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:36.126 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3088 2024-09-16T20:50:36.126 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.126 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:36.126 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.126 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:36.126 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:50:36.126 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.126 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.127 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.127 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:36.127 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:50:36.127 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.127 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.127 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.127 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:36.127 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163846+0000", 2024-09-16T20:50:36.127 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6799999999999999e-07 2024-09-16T20:50:36.127 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.127 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.127 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:36.127 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163850+0000", 2024-09-16T20:50:36.128 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.929e-06 2024-09-16T20:50:36.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.128 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.128 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:36.128 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163862+0000", 2024-09-16T20:50:36.128 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1972e-05 2024-09-16T20:50:36.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.128 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.128 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:36.128 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163867+0000", 2024-09-16T20:50:36.128 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.451e-06 2024-09-16T20:50:36.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.129 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.129 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:36.129 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163912+0000", 2024-09-16T20:50:36.129 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4456000000000003e-05 2024-09-16T20:50:36.129 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.129 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.129 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:36.129 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163962+0000", 2024-09-16T20:50:36.129 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0578000000000003e-05 2024-09-16T20:50:36.129 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.129 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.129 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-16T20:50:36.129 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164058+0000", 2024-09-16T20:50:36.130 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5934999999999995e-05 2024-09-16T20:50:36.130 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.130 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.130 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:50:36.130 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164570+0000", 2024-09-16T20:50:36.130 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00051241100000000005 2024-09-16T20:50:36.130 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.130 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.130 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:36.130 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282629+0000", 2024-09-16T20:50:36.130 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.118058986 2024-09-16T20:50:36.130 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.130 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.130 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:50:36.131 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282725+0000", 2024-09-16T20:50:36.131 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5063999999999996e-05 2024-09-16T20:50:36.131 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.131 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.131 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:36.131 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282797+0000", 2024-09-16T20:50:36.131 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2212000000000006e-05 2024-09-16T20:50:36.131 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.131 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:36.131 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.131 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.131 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.132 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:8039 3.1e 3.ac44ff9e (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:50:36.132 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:50:36.132 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 106.41515558, 2024-09-16T20:50:36.132 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0053310449999999, 2024-09-16T20:50:36.132 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:36.132 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:36.132 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:36.132 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:36.132 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:50:36.132 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:36.132 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8039 2024-09-16T20:50:36.132 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.132 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:36.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.133 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:36.133 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:50:36.133 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.133 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:36.133 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:50:36.133 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.133 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:36.133 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707674+0000", 2024-09-16T20:50:36.133 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6530000000000001e-06 2024-09-16T20:50:36.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.134 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.134 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:36.134 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707681+0000", 2024-09-16T20:50:36.134 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9639999999999997e-06 2024-09-16T20:50:36.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.134 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.134 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:36.134 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707700+0000", 2024-09-16T20:50:36.134 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8674000000000001e-05 2024-09-16T20:50:36.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.134 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.134 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:36.134 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707708+0000", 2024-09-16T20:50:36.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6810000000000004e-06 2024-09-16T20:50:36.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.135 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.135 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:36.135 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711293+0000", 2024-09-16T20:50:36.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035848189999999999 2024-09-16T20:50:36.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.135 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.135 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:36.135 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711301+0000", 2024-09-16T20:50:36.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4730000000000003e-06 2024-09-16T20:50:36.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.135 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.135 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:36.135 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.712990+0000", 2024-09-16T20:50:36.136 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.001689348 2024-09-16T20:50:36.136 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.136 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.136 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:36.136 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713003+0000", 2024-09-16T20:50:36.136 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3433e-05 2024-09-16T20:50:36.136 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.136 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:36.136 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.136 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.136 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.136 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:8040 3.1e 3.ac44ff9e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:50:36.136 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:50:36.136 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 106.415095185, 2024-09-16T20:50:36.137 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0053410780000001, 2024-09-16T20:50:36.137 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:36.137 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:36.137 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:36.137 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:36.137 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:50:36.137 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:36.137 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8040 2024-09-16T20:50:36.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.137 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:36.137 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.137 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:36.137 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:50:36.137 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.138 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:36.138 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:50:36.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.138 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:36.138 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:50:36.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8100000000000003e-07 2024-09-16T20:50:36.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.138 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:36.138 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707737+0000", 2024-09-16T20:50:36.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5020000000000001e-06 2024-09-16T20:50:36.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.139 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.139 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:36.139 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707745+0000", 2024-09-16T20:50:36.139 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2169999999999994e-06 2024-09-16T20:50:36.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.139 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.139 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:36.139 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707751+0000", 2024-09-16T20:50:36.139 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.263e-06 2024-09-16T20:50:36.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.139 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.139 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:36.139 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711336+0000", 2024-09-16T20:50:36.139 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035848360000000001 2024-09-16T20:50:36.140 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.140 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.140 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:36.140 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711340+0000", 2024-09-16T20:50:36.140 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1309999999999999e-06 2024-09-16T20:50:36.140 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.140 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.140 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:36.140 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713060+0000", 2024-09-16T20:50:36.140 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0017195059999999 2024-09-16T20:50:36.140 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.140 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.140 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:36.141 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713074+0000", 2024-09-16T20:50:36.141 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4042e-05 2024-09-16T20:50:36.141 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.141 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:36.141 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.141 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.141 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4927.0:4487 3.1e 3.ac44ff9e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:50:36.141 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:50:36.141 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 106.41457498299999, 2024-09-16T20:50:36.141 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0048840880000001, 2024-09-16T20:50:36.141 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:36.141 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:36.141 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:36.142 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:36.142 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4927", 2024-09-16T20:50:36.142 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:36.142 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4487 2024-09-16T20:50:36.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.142 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:36.142 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.142 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:36.142 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:50:36.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.142 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.142 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:36.142 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:50:36.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.143 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:36.143 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708254+0000", 2024-09-16T20:50:36.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1800000000000005e-07 2024-09-16T20:50:36.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.143 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:36.143 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708258+0000", 2024-09-16T20:50:36.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8689999999999997e-06 2024-09-16T20:50:36.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.143 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:36.144 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708267+0000", 2024-09-16T20:50:36.144 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9590000000000007e-06 2024-09-16T20:50:36.144 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.144 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.144 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:36.144 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708274+0000", 2024-09-16T20:50:36.144 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9970000000000001e-06 2024-09-16T20:50:36.144 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.144 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.144 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:36.144 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711366+0000", 2024-09-16T20:50:36.144 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030919530000000001 2024-09-16T20:50:36.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.145 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.145 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:36.145 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711372+0000", 2024-09-16T20:50:36.145 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9410000000000004e-06 2024-09-16T20:50:36.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.145 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.145 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:36.145 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713126+0000", 2024-09-16T20:50:36.145 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0017537249999999 2024-09-16T20:50:36.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.145 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.145 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:36.146 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713137+0000", 2024-09-16T20:50:36.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0825999999999999e-05 2024-09-16T20:50:36.146 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.146 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:36.146 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.146 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5089.0:11703 3.1b 3:dcd4684e:::rbd_data.110e747f0779.000000000000001b:head [set-alloc-hint object_size 262144 write_size 262144,write 139264~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:50:36.146 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:50:36.146 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 91.984839209, 2024-09-16T20:50:36.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.084231385000000006, 2024-09-16T20:50:36.146 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:36.147 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:36.147 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:36.147 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:36.147 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5089", 2024-09-16T20:50:36.147 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:50:36.147 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 11703 2024-09-16T20:50:36.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.147 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:36.147 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.147 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:36.147 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:50:36.147 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.147 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.148 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:36.148 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:50:36.148 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.148 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.148 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.148 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:36.148 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:50:36.148 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2600000000000001e-07 2024-09-16T20:50:36.148 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.148 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.148 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:36.148 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137998+0000", 2024-09-16T20:50:36.149 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9549999999999998e-06 2024-09-16T20:50:36.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.149 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.149 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:36.149 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138008+0000", 2024-09-16T20:50:36.149 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3789999999999995e-06 2024-09-16T20:50:36.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.149 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.149 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:36.149 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138013+0000", 2024-09-16T20:50:36.149 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2209999999999996e-06 2024-09-16T20:50:36.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.149 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.149 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:36.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138054+0000", 2024-09-16T20:50:36.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1022999999999997e-05 2024-09-16T20:50:36.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.150 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.150 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:36.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138124+0000", 2024-09-16T20:50:36.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0234999999999994e-05 2024-09-16T20:50:36.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.150 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.150 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:36.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138197+0000", 2024-09-16T20:50:36.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2451000000000001e-05 2024-09-16T20:50:36.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:36.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222010+0000", 2024-09-16T20:50:36.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.083813433000000007 2024-09-16T20:50:36.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:50:36.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222163+0000", 2024-09-16T20:50:36.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015274599999999999 2024-09-16T20:50:36.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:50:36.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222197+0000", 2024-09-16T20:50:36.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3683000000000003e-05 2024-09-16T20:50:36.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.152 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.152 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:36.152 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222220+0000", 2024-09-16T20:50:36.152 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3433000000000001e-05 2024-09-16T20:50:36.152 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.152 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:36.152 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.152 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.152 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.5089.0:11702 3.15 e73/64 3:acf33c8b:::rbd_data.110e747f0779.000000000000001a:head v 73'132, mlcod=73'130)", 2024-09-16T20:50:36.152 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:50:36.152 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 91.984571880000004, 2024-09-16T20:50:36.152 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.084079763000000002, 2024-09-16T20:50:36.153 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:36.153 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:36.153 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:36.153 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:36.153 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.153 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:36.153 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:50:36.153 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.153 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.153 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.153 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:36.153 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:50:36.153 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.154 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.154 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.154 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:36.154 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:50:36.154 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:50:36.154 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.154 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.154 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:36.154 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138263+0000", 2024-09-16T20:50:36.154 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1e-06 2024-09-16T20:50:36.154 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.154 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.154 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:36.155 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221589+0000", 2024-09-16T20:50:36.155 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.083326002999999996 2024-09-16T20:50:36.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.155 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.155 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:36.155 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221614+0000", 2024-09-16T20:50:36.155 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5593999999999999e-05 2024-09-16T20:50:36.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.155 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.155 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:36.155 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221714+0000", 2024-09-16T20:50:36.155 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9889000000000005e-05 2024-09-16T20:50:36.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.155 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.155 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:36.156 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221727+0000", 2024-09-16T20:50:36.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3125999999999999e-05 2024-09-16T20:50:36.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.156 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.156 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:50:36.156 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222294+0000", 2024-09-16T20:50:36.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00056712199999999998 2024-09-16T20:50:36.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.156 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.156 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:36.156 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222336+0000", 2024-09-16T20:50:36.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1470000000000001e-05 2024-09-16T20:50:36.157 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.157 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:36.157 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.157 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.157 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.157 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5138.0:9358 3.c 3:33996a43:::rbd_data.110e6372565d.0000000000000007:head [read 1343488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T20:50:36.157 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:50:36.157 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 68.270296470000005, 2024-09-16T20:50:36.157 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0167493569999999, 2024-09-16T20:50:36.157 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:36.157 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:36.157 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:36.157 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:36.158 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5138", 2024-09-16T20:50:36.158 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:50:36.158 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9358 2024-09-16T20:50:36.158 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.158 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:36.158 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.158 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:36.158 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:50:36.158 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.158 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.158 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.158 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:36.158 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:50:36.158 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.159 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.159 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:36.159 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852533+0000", 2024-09-16T20:50:36.159 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.646e-06 2024-09-16T20:50:36.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.159 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.159 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:36.159 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852539+0000", 2024-09-16T20:50:36.159 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3389999999999998e-06 2024-09-16T20:50:36.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.159 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.159 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:36.159 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852552+0000", 2024-09-16T20:50:36.160 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.271e-05 2024-09-16T20:50:36.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.160 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.160 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:36.160 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852563+0000", 2024-09-16T20:50:36.160 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1139e-05 2024-09-16T20:50:36.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.160 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.160 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:36.160 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852883+0000", 2024-09-16T20:50:36.160 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00032010600000000001 2024-09-16T20:50:36.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.160 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.160 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:36.161 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852890+0000", 2024-09-16T20:50:36.161 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7669999999999998e-06 2024-09-16T20:50:36.161 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.161 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.161 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:36.161 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869059+0000", 2024-09-16T20:50:36.161 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.016169353 2024-09-16T20:50:36.161 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.161 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.161 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:36.161 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869178+0000", 2024-09-16T20:50:36.161 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118193 2024-09-16T20:50:36.161 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.161 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.162 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:36.162 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869281+0000", 2024-09-16T20:50:36.162 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010310399999999999 2024-09-16T20:50:36.162 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.162 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:36.162 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.162 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.162 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.162 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5138.0:9382 3.c 3:33996a43:::rbd_data.110e6372565d.0000000000000007:head [read 0~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T20:50:36.162 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:50:36.162 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 68.270245592999999, 2024-09-16T20:50:36.162 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.017136034, 2024-09-16T20:50:36.162 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:36.162 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:36.162 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:36.163 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:36.163 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5138", 2024-09-16T20:50:36.163 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:50:36.163 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9382 2024-09-16T20:50:36.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.163 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:36.163 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.163 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:36.163 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:50:36.163 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.163 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.163 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:36.163 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:50:36.164 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.164 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.164 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:36.164 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852583+0000", 2024-09-16T20:50:36.164 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.51e-07 2024-09-16T20:50:36.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.164 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.164 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:36.164 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852586+0000", 2024-09-16T20:50:36.164 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0280000000000001e-06 2024-09-16T20:50:36.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.164 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.164 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:36.165 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852595+0000", 2024-09-16T20:50:36.165 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2799999999999992e-06 2024-09-16T20:50:36.165 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.165 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.165 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:36.165 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852601+0000", 2024-09-16T20:50:36.165 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3810000000000004e-06 2024-09-16T20:50:36.165 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.165 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.165 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:36.165 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852921+0000", 2024-09-16T20:50:36.165 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00031979699999999997 2024-09-16T20:50:36.165 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.165 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.166 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:36.166 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852930+0000", 2024-09-16T20:50:36.166 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6489999999999997e-06 2024-09-16T20:50:36.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.166 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.166 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:36.166 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869434+0000", 2024-09-16T20:50:36.166 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0165038829999999 2024-09-16T20:50:36.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.166 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.166 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:36.166 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869495+0000", 2024-09-16T20:50:36.166 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1509999999999999e-05 2024-09-16T20:50:36.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.166 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.166 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:36.167 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869718+0000", 2024-09-16T20:50:36.167 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000222955 2024-09-16T20:50:36.167 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.167 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:36.167 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.167 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.167 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.167 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9969 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 632832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:36.167 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642045+0000", 2024-09-16T20:50:36.167 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 45.480783135999999, 2024-09-16T20:50:36.167 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076903192999999995, 2024-09-16T20:50:36.167 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:36.167 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:36.167 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:36.167 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:36.168 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:36.168 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:36.168 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9969 2024-09-16T20:50:36.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.168 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:36.168 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.168 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:36.168 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642045+0000", 2024-09-16T20:50:36.169 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.169 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.169 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:36.169 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642045+0000", 2024-09-16T20:50:36.169 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.169 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.169 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:36.169 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642045+0000", 2024-09-16T20:50:36.169 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5100000000000001e-07 2024-09-16T20:50:36.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.169 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.170 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:36.170 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642047+0000", 2024-09-16T20:50:36.170 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8679999999999999e-06 2024-09-16T20:50:36.170 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.170 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.170 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:36.170 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642053+0000", 2024-09-16T20:50:36.170 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1720000000000004e-06 2024-09-16T20:50:36.170 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.170 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.170 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:36.170 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642060+0000", 2024-09-16T20:50:36.170 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0339999999999999e-06 2024-09-16T20:50:36.170 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.171 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.171 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:36.171 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.718562+0000", 2024-09-16T20:50:36.171 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076501969000000003 2024-09-16T20:50:36.171 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.171 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.171 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:36.171 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.718643+0000", 2024-09-16T20:50:36.171 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0539999999999998e-05 2024-09-16T20:50:36.171 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.171 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.171 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:36.171 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.718948+0000", 2024-09-16T20:50:36.171 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000305259 2024-09-16T20:50:36.171 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.172 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:36.172 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.172 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.172 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.172 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9970 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 637952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:36.172 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642067+0000", 2024-09-16T20:50:36.172 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 45.480761156, 2024-09-16T20:50:36.172 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077174928000000004, 2024-09-16T20:50:36.172 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:36.172 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:36.172 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:36.172 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:36.173 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:36.173 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:36.173 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9970 2024-09-16T20:50:36.173 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.173 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:36.173 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.173 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:36.173 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642067+0000", 2024-09-16T20:50:36.173 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.173 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.173 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.174 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:36.174 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642067+0000", 2024-09-16T20:50:36.174 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.174 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.174 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:36.174 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642067+0000", 2024-09-16T20:50:36.174 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2499999999999995e-07 2024-09-16T20:50:36.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.174 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.174 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:36.174 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642070+0000", 2024-09-16T20:50:36.174 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3199999999999998e-06 2024-09-16T20:50:36.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.175 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.175 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:36.175 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642076+0000", 2024-09-16T20:50:36.175 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.122e-06 2024-09-16T20:50:36.175 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.175 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.176 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:36.176 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642081+0000", 2024-09-16T20:50:36.176 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0869999999999997e-06 2024-09-16T20:50:36.176 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.177 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.177 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:36.177 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.718980+0000", 2024-09-16T20:50:36.177 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076899482000000005 2024-09-16T20:50:36.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.177 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.178 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:36.178 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719085+0000", 2024-09-16T20:50:36.178 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010446399999999999 2024-09-16T20:50:36.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.178 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.178 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:36.178 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719242+0000", 2024-09-16T20:50:36.179 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000156928 2024-09-16T20:50:36.179 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.179 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:36.179 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.179 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.179 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9971 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 643072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:36.180 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642102+0000", 2024-09-16T20:50:36.180 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 45.480725448999998, 2024-09-16T20:50:36.180 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077421016999999995, 2024-09-16T20:50:36.180 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:36.180 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:36.180 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:36.180 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:36.180 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:36.181 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:36.181 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9971 2024-09-16T20:50:36.181 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.181 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:36.181 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.181 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:36.182 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642102+0000", 2024-09-16T20:50:36.182 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.182 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.182 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.182 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:36.182 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642102+0000", 2024-09-16T20:50:36.182 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.183 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.183 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.183 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:36.183 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642103+0000", 2024-09-16T20:50:36.183 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7399999999999998e-07 2024-09-16T20:50:36.183 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.184 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.184 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:36.184 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642105+0000", 2024-09-16T20:50:36.184 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2649999999999999e-06 2024-09-16T20:50:36.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.184 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.184 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:36.184 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642112+0000", 2024-09-16T20:50:36.184 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.294e-06 2024-09-16T20:50:36.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.184 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.185 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:36.185 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642118+0000", 2024-09-16T20:50:36.185 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4500000000000003e-06 2024-09-16T20:50:36.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.185 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.185 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:36.185 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719285+0000", 2024-09-16T20:50:36.185 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077166883000000006 2024-09-16T20:50:36.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.185 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.185 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:36.185 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719368+0000", 2024-09-16T20:50:36.186 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2811000000000004e-05 2024-09-16T20:50:36.186 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.186 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.186 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:36.186 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719523+0000", 2024-09-16T20:50:36.186 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015584 2024-09-16T20:50:36.186 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.186 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:36.186 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.186 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.186 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.186 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9972 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 648192~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:36.186 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642125+0000", 2024-09-16T20:50:36.186 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 45.480702502, 2024-09-16T20:50:36.186 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077710324999999997, 2024-09-16T20:50:36.187 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:36.187 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:36.187 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:36.187 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:36.187 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:36.187 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:36.187 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9972 2024-09-16T20:50:36.187 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.187 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:36.187 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.187 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:36.187 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642125+0000", 2024-09-16T20:50:36.187 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.187 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.187 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.188 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:36.188 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642125+0000", 2024-09-16T20:50:36.188 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.188 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.188 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.188 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:36.188 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642126+0000", 2024-09-16T20:50:36.188 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1899999999999998e-07 2024-09-16T20:50:36.188 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.188 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.188 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:36.188 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642128+0000", 2024-09-16T20:50:36.188 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8619999999999999e-06 2024-09-16T20:50:36.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.189 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.189 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:36.189 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642134+0000", 2024-09-16T20:50:36.189 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5289999999999997e-06 2024-09-16T20:50:36.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.189 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.189 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:36.189 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642139+0000", 2024-09-16T20:50:36.189 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1250000000000001e-06 2024-09-16T20:50:36.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.189 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.189 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:36.190 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719570+0000", 2024-09-16T20:50:36.190 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077430516000000005 2024-09-16T20:50:36.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.190 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.190 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:36.190 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719655+0000", 2024-09-16T20:50:36.190 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5500000000000005e-05 2024-09-16T20:50:36.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.190 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.190 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:36.190 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719836+0000", 2024-09-16T20:50:36.190 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018037399999999999 2024-09-16T20:50:36.190 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.190 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:36.191 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.191 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.191 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.191 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9973 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 653312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:36.191 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:50:36.191 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 45.480681892, 2024-09-16T20:50:36.191 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077905262000000003, 2024-09-16T20:50:36.191 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:36.191 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:36.191 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:36.191 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:36.191 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:36.191 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:36.191 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9973 2024-09-16T20:50:36.192 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.192 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:36.192 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.192 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:36.192 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:50:36.192 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.192 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.192 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.192 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:36.192 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:50:36.192 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.192 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.192 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.193 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:36.193 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:50:36.193 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:50:36.193 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.193 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.193 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:36.193 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642148+0000", 2024-09-16T20:50:36.193 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0219999999999999e-06 2024-09-16T20:50:36.193 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.193 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.193 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:36.193 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642154+0000", 2024-09-16T20:50:36.193 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8540000000000002e-06 2024-09-16T20:50:36.193 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.194 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.194 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:36.194 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642159+0000", 2024-09-16T20:50:36.194 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.045e-06 2024-09-16T20:50:36.194 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.194 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.194 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:36.194 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719875+0000", 2024-09-16T20:50:36.194 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077716116000000002 2024-09-16T20:50:36.194 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.194 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.194 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:36.194 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719934+0000", 2024-09-16T20:50:36.194 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8819e-05 2024-09-16T20:50:36.194 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.195 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.195 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:36.195 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720051+0000", 2024-09-16T20:50:36.195 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000116947 2024-09-16T20:50:36.195 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.195 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:36.195 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.195 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.195 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.195 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9974 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 658432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:36.195 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642166+0000", 2024-09-16T20:50:36.195 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 45.480661978000001, 2024-09-16T20:50:36.195 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.078179786000000001, 2024-09-16T20:50:36.196 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:36.196 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:36.196 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:36.196 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:36.196 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:36.196 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:36.196 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9974 2024-09-16T20:50:36.196 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.196 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:36.196 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.196 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:36.196 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642166+0000", 2024-09-16T20:50:36.196 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.196 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.197 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.197 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:36.197 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642166+0000", 2024-09-16T20:50:36.197 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.197 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.197 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.197 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:36.197 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647343+0000", 2024-09-16T20:50:36.197 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0051769440000000002 2024-09-16T20:50:36.197 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.197 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.197 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:36.197 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647348+0000", 2024-09-16T20:50:36.197 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3619999999999997e-06 2024-09-16T20:50:36.198 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.198 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.198 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:36.198 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647359+0000", 2024-09-16T20:50:36.198 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0929e-05 2024-09-16T20:50:36.198 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.198 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.198 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:36.198 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647366+0000", 2024-09-16T20:50:36.198 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0910000000000004e-06 2024-09-16T20:50:36.198 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.198 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.198 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:36.198 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720094+0000", 2024-09-16T20:50:36.198 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.072727403999999995 2024-09-16T20:50:36.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.199 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.199 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:36.199 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720173+0000", 2024-09-16T20:50:36.199 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9382000000000007e-05 2024-09-16T20:50:36.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.199 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.199 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:36.199 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720346+0000", 2024-09-16T20:50:36.199 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017267399999999999 2024-09-16T20:50:36.199 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.199 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:36.199 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.199 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.200 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9990 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 740352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:36.200 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647725+0000", 2024-09-16T20:50:36.200 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 45.475102894000003, 2024-09-16T20:50:36.200 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077029919000000002, 2024-09-16T20:50:36.200 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:36.200 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:36.200 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:36.200 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:36.200 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:36.200 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:36.200 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9990 2024-09-16T20:50:36.200 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.200 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:36.200 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.201 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:36.201 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647725+0000", 2024-09-16T20:50:36.201 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.201 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.201 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.201 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:36.201 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647725+0000", 2024-09-16T20:50:36.201 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.201 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.201 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.201 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:36.201 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647725+0000", 2024-09-16T20:50:36.201 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2399999999999999e-07 2024-09-16T20:50:36.201 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.202 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.202 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:36.202 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647726+0000", 2024-09-16T20:50:36.202 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.116e-06 2024-09-16T20:50:36.202 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.202 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.202 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:36.202 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647729+0000", 2024-09-16T20:50:36.202 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.937e-06 2024-09-16T20:50:36.202 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.202 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.202 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:36.203 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647732+0000", 2024-09-16T20:50:36.203 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8739999999999999e-06 2024-09-16T20:50:36.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.203 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.203 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:36.203 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724612+0000", 2024-09-16T20:50:36.203 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076880102000000006 2024-09-16T20:50:36.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.204 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.204 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:36.204 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724649+0000", 2024-09-16T20:50:36.204 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6738e-05 2024-09-16T20:50:36.204 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.204 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.204 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:36.204 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724755+0000", 2024-09-16T20:50:36.204 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010592800000000001 2024-09-16T20:50:36.204 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.204 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:36.204 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.204 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.205 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.205 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9991 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 745472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:36.205 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:50:36.205 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 45.475092584999999, 2024-09-16T20:50:36.205 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077257820000000005, 2024-09-16T20:50:36.205 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:36.205 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:36.205 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:36.205 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:36.205 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:36.205 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:36.205 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9991 2024-09-16T20:50:36.206 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.206 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:36.206 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.206 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:36.206 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:50:36.206 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.206 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.206 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.206 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:36.206 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:50:36.206 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.206 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.206 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.207 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:36.207 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:50:36.207 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1199999999999999e-07 2024-09-16T20:50:36.207 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.207 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.207 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:36.207 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647736+0000", 2024-09-16T20:50:36.207 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3699999999999999e-07 2024-09-16T20:50:36.207 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.207 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.207 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:36.207 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647739+0000", 2024-09-16T20:50:36.207 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9359999999999999e-06 2024-09-16T20:50:36.207 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.208 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.208 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:36.208 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647742+0000", 2024-09-16T20:50:36.208 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5720000000000001e-06 2024-09-16T20:50:36.208 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.208 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.208 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:36.208 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724774+0000", 2024-09-16T20:50:36.208 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077032457999999998 2024-09-16T20:50:36.208 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.208 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.208 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:36.208 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724818+0000", 2024-09-16T20:50:36.209 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3803000000000001e-05 2024-09-16T20:50:36.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.209 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.209 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:36.209 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724993+0000", 2024-09-16T20:50:36.209 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000174902 2024-09-16T20:50:36.209 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.209 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:36.209 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.209 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.209 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9992 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 750592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:36.209 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:50:36.210 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 45.475082995999998, 2024-09-16T20:50:36.210 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077553227000000002, 2024-09-16T20:50:36.210 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:36.210 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:36.210 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:36.210 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:36.210 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:36.210 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:36.210 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9992 2024-09-16T20:50:36.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.210 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:36.210 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.210 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:36.210 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:50:36.211 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.211 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.211 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.211 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:36.211 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:50:36.211 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.211 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.211 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.211 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:36.211 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:50:36.212 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0200000000000001e-07 2024-09-16T20:50:36.212 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.212 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.212 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:36.212 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647746+0000", 2024-09-16T20:50:36.212 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.088e-06 2024-09-16T20:50:36.212 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.212 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.212 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:36.212 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647749+0000", 2024-09-16T20:50:36.212 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9170000000000002e-06 2024-09-16T20:50:36.212 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.212 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.212 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:36.213 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647752+0000", 2024-09-16T20:50:36.213 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.774e-06 2024-09-16T20:50:36.213 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.213 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.213 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:36.213 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725042+0000", 2024-09-16T20:50:36.213 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077289952999999995 2024-09-16T20:50:36.213 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.213 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.213 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:36.213 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725113+0000", 2024-09-16T20:50:36.213 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0920999999999994e-05 2024-09-16T20:50:36.213 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.213 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.214 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:36.214 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725298+0000", 2024-09-16T20:50:36.214 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000185372 2024-09-16T20:50:36.214 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.214 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:36.214 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.214 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.214 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.214 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9993 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 755712~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:36.214 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647755+0000", 2024-09-16T20:50:36.214 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 45.475072969999999, 2024-09-16T20:50:36.214 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077725674999999994, 2024-09-16T20:50:36.214 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:36.214 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:36.214 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:36.215 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:36.215 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:36.215 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:36.215 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9993 2024-09-16T20:50:36.215 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.215 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:36.215 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.215 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:36.215 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647755+0000", 2024-09-16T20:50:36.215 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.215 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.215 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.215 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:36.216 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647755+0000", 2024-09-16T20:50:36.216 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.216 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.216 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.216 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:36.216 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653406+0000", 2024-09-16T20:50:36.216 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.005650876 2024-09-16T20:50:36.216 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.216 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.216 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:36.216 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653411+0000", 2024-09-16T20:50:36.216 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8899999999999998e-06 2024-09-16T20:50:36.216 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.217 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.217 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:36.217 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653423+0000", 2024-09-16T20:50:36.217 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2447e-05 2024-09-16T20:50:36.217 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.217 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.217 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:36.217 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653431+0000", 2024-09-16T20:50:36.217 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5730000000000002e-06 2024-09-16T20:50:36.217 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.217 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.217 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:36.217 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725333+0000", 2024-09-16T20:50:36.218 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.071902743000000005 2024-09-16T20:50:36.218 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.218 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.218 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:36.218 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725385+0000", 2024-09-16T20:50:36.218 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1585000000000002e-05 2024-09-16T20:50:36.218 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.218 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.218 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:36.218 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725481+0000", 2024-09-16T20:50:36.218 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5561000000000002e-05 2024-09-16T20:50:36.218 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.218 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:36.218 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.219 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.219 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.219 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6061.0:5400 3.b 3:d353d652:::rbd_data.110e16eafbd.0000000000000010:head [read 577536~28672] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-16T20:50:36.219 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:27.761367+0000", 2024-09-16T20:50:36.219 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.3614610240000005, 2024-09-16T20:50:36.219 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0029066630000001, 2024-09-16T20:50:36.219 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:36.219 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:36.219 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:36.219 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:36.219 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6061", 2024-09-16T20:50:36.219 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3930128440", 2024-09-16T20:50:36.219 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5400 2024-09-16T20:50:36.220 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.220 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:36.220 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.220 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:36.220 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761367+0000", 2024-09-16T20:50:36.220 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.220 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.220 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.220 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:36.220 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761367+0000", 2024-09-16T20:50:36.220 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.220 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.220 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.220 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:36.221 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761369+0000", 2024-09-16T20:50:36.221 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.934e-06 2024-09-16T20:50:36.221 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.221 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.221 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:36.221 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761374+0000", 2024-09-16T20:50:36.221 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7139999999999998e-06 2024-09-16T20:50:36.221 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.221 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.221 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:36.221 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761387+0000", 2024-09-16T20:50:36.221 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2349e-05 2024-09-16T20:50:36.221 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.221 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.222 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:36.222 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761393+0000", 2024-09-16T20:50:36.222 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.652e-06 2024-09-16T20:50:36.222 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.222 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.222 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:36.222 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761442+0000", 2024-09-16T20:50:36.222 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9011000000000002e-05 2024-09-16T20:50:36.222 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.222 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.222 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:36.222 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761451+0000", 2024-09-16T20:50:36.222 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1920000000000005e-06 2024-09-16T20:50:36.222 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.223 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.223 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:36.223 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:28.764169+0000", 2024-09-16T20:50:36.223 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.002718161 2024-09-16T20:50:36.223 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.223 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.223 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:36.223 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:28.764273+0000", 2024-09-16T20:50:36.223 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010465 2024-09-16T20:50:36.223 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.223 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:36.223 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.223 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.223 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.223 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6061.0:5424 3.b 3:d353d652:::rbd_data.110e16eafbd.0000000000000010:head [read 606208~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-16T20:50:36.224 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:27.766882+0000", 2024-09-16T20:50:36.224 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.3559456609999998, 2024-09-16T20:50:36.224 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99753650500000002, 2024-09-16T20:50:36.224 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:36.224 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:36.224 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:36.224 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:36.224 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6061", 2024-09-16T20:50:36.224 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3930128440", 2024-09-16T20:50:36.224 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5424 2024-09-16T20:50:36.224 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.224 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:36.224 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.225 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:36.225 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766882+0000", 2024-09-16T20:50:36.225 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.225 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.225 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.225 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:36.225 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766882+0000", 2024-09-16T20:50:36.225 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.225 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.225 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.225 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:36.225 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766883+0000", 2024-09-16T20:50:36.226 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3289999999999999e-06 2024-09-16T20:50:36.226 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.226 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.226 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:36.226 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766888+0000", 2024-09-16T20:50:36.226 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5800000000000002e-06 2024-09-16T20:50:36.226 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.226 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.226 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:36.226 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766900+0000", 2024-09-16T20:50:36.226 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1861e-05 2024-09-16T20:50:36.226 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.226 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.226 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:36.227 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766909+0000", 2024-09-16T20:50:36.227 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5210000000000008e-06 2024-09-16T20:50:36.227 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.227 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.227 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:36.227 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766958+0000", 2024-09-16T20:50:36.227 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8718999999999999e-05 2024-09-16T20:50:36.227 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.227 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.227 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:36.227 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766975+0000", 2024-09-16T20:50:36.227 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7231000000000002e-05 2024-09-16T20:50:36.227 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.228 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.228 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:36.228 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:28.764327+0000", 2024-09-16T20:50:36.228 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99735112500000001 2024-09-16T20:50:36.228 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.228 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.228 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:36.228 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:28.764419+0000", 2024-09-16T20:50:36.228 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2139000000000007e-05 2024-09-16T20:50:36.228 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.228 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:36.228 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.229 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.229 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:36.229 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:36.230 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:50:36.231 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:50:36.231 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:50:36.232 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:36.229+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:36.331 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:36.330+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:36.378 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:36.378 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:50:36.378 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:50:36.378 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:36.387 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:50:36.432 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:36.432+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:36.493 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:36.493 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:50:36.493 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:50:36.493 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:50:36.493 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:36.501 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:50:36.542 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:36.542+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:36.634 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:36.633+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:36.650 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:36.650 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:50:36.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:50:36.650 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:50:36.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.650 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12548 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 628736~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:36.650 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757009+0000", 2024-09-16T20:50:36.651 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 45.891461685000003, 2024-09-16T20:50:36.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065743956000000006, 2024-09-16T20:50:36.651 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:36.651 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:36.651 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:36.651 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:36.651 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:36.651 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:36.651 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12548 2024-09-16T20:50:36.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.651 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:36.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.651 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:36.652 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757009+0000", 2024-09-16T20:50:36.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.652 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:36.652 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757009+0000", 2024-09-16T20:50:36.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.652 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:36.652 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757010+0000", 2024-09-16T20:50:36.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9900000000000001e-07 2024-09-16T20:50:36.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:36.653 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757012+0000", 2024-09-16T20:50:36.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2689999999999998e-06 2024-09-16T20:50:36.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:36.653 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757022+0000", 2024-09-16T20:50:36.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9699999999999994e-06 2024-09-16T20:50:36.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:36.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757028+0000", 2024-09-16T20:50:36.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3799999999999999e-06 2024-09-16T20:50:36.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:36.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822552+0000", 2024-09-16T20:50:36.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065524281000000004 2024-09-16T20:50:36.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:36.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822613+0000", 2024-09-16T20:50:36.655 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0430999999999998e-05 2024-09-16T20:50:36.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.655 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.655 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:36.655 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822753+0000", 2024-09-16T20:50:36.655 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000140226 2024-09-16T20:50:36.655 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.655 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:36.655 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.655 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.655 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12549 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 633856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:36.655 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:50:36.656 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 45.891432938000001, 2024-09-16T20:50:36.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065978518, 2024-09-16T20:50:36.656 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:36.656 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:36.656 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:36.656 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:36.656 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:36.656 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:36.656 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12549 2024-09-16T20:50:36.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.656 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:36.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.656 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:36.657 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:50:36.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.657 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:36.657 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:50:36.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.657 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:36.657 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:50:36.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9299999999999999e-07 2024-09-16T20:50:36.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.658 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:36.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757040+0000", 2024-09-16T20:50:36.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.677e-06 2024-09-16T20:50:36.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.658 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:36.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757045+0000", 2024-09-16T20:50:36.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8149999999999996e-06 2024-09-16T20:50:36.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.658 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:36.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757048+0000", 2024-09-16T20:50:36.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1e-06 2024-09-16T20:50:36.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.659 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:36.659 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822796+0000", 2024-09-16T20:50:36.659 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065748081 2024-09-16T20:50:36.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.659 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.659 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:36.659 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822867+0000", 2024-09-16T20:50:36.659 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0679000000000005e-05 2024-09-16T20:50:36.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.659 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.659 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:36.659 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823016+0000", 2024-09-16T20:50:36.659 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014987299999999999 2024-09-16T20:50:36.659 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.659 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:36.660 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.660 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12550 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 638976~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:36.660 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757060+0000", 2024-09-16T20:50:36.660 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 45.891410522999998, 2024-09-16T20:50:36.660 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066153564999999998, 2024-09-16T20:50:36.660 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:36.660 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:36.660 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:36.660 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:36.660 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:36.660 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:36.661 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12550 2024-09-16T20:50:36.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.661 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:36.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.661 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:36.661 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757060+0000", 2024-09-16T20:50:36.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.661 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:36.661 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757060+0000", 2024-09-16T20:50:36.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.662 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:36.662 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757061+0000", 2024-09-16T20:50:36.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.15e-07 2024-09-16T20:50:36.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.662 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.662 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:36.662 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757062+0000", 2024-09-16T20:50:36.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.663e-06 2024-09-16T20:50:36.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.662 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.662 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:36.662 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757066+0000", 2024-09-16T20:50:36.663 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.817e-06 2024-09-16T20:50:36.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.663 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:36.663 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757070+0000", 2024-09-16T20:50:36.663 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1509999999999997e-06 2024-09-16T20:50:36.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.663 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:36.663 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823044+0000", 2024-09-16T20:50:36.663 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065973992999999995 2024-09-16T20:50:36.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.664 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:36.664 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823082+0000", 2024-09-16T20:50:36.664 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8050999999999999e-05 2024-09-16T20:50:36.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.664 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.664 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:36.664 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823214+0000", 2024-09-16T20:50:36.664 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013157500000000001 2024-09-16T20:50:36.664 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.664 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:36.664 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.664 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.665 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12551 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 644096~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:36.665 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:50:36.665 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 45.891396825000001, 2024-09-16T20:50:36.665 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066391378000000001, 2024-09-16T20:50:36.665 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:36.665 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:36.665 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:36.665 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:36.665 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:36.665 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:36.665 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12551 2024-09-16T20:50:36.665 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.666 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:36.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.666 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:36.666 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:50:36.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.666 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:36.666 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:50:36.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.666 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:36.666 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:50:36.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2999999999999999e-07 2024-09-16T20:50:36.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.667 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:36.667 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757075+0000", 2024-09-16T20:50:36.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.097e-06 2024-09-16T20:50:36.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.667 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:36.667 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757082+0000", 2024-09-16T20:50:36.668 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9500000000000004e-06 2024-09-16T20:50:36.668 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.668 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.668 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:36.668 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757086+0000", 2024-09-16T20:50:36.668 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2229999999999999e-06 2024-09-16T20:50:36.668 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.668 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.668 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:36.669 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823262+0000", 2024-09-16T20:50:36.669 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066176078999999999 2024-09-16T20:50:36.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.669 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.669 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:36.669 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823336+0000", 2024-09-16T20:50:36.669 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3979000000000004e-05 2024-09-16T20:50:36.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.669 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.669 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:36.670 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823465+0000", 2024-09-16T20:50:36.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012982 2024-09-16T20:50:36.670 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.670 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:36.670 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.670 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12552 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 649216~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:36.671 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757089+0000", 2024-09-16T20:50:36.671 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 45.891381684000002, 2024-09-16T20:50:36.671 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066655617, 2024-09-16T20:50:36.671 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:36.671 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:36.671 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:36.671 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:36.671 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:36.671 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:36.671 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12552 2024-09-16T20:50:36.671 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.672 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:36.672 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.672 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:36.672 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757089+0000", 2024-09-16T20:50:36.672 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.672 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.672 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.672 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:36.672 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757089+0000", 2024-09-16T20:50:36.672 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.672 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.672 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.672 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:36.672 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759492+0000", 2024-09-16T20:50:36.673 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0024026109999999998 2024-09-16T20:50:36.673 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.673 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.673 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:36.673 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759497+0000", 2024-09-16T20:50:36.673 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1120000000000004e-06 2024-09-16T20:50:36.673 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.673 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.673 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:36.673 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759508+0000", 2024-09-16T20:50:36.673 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0981e-05 2024-09-16T20:50:36.673 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.673 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.674 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:36.674 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759516+0000", 2024-09-16T20:50:36.674 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7270000000000007e-06 2024-09-16T20:50:36.674 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.674 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.674 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:36.674 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823504+0000", 2024-09-16T20:50:36.674 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.063987984999999997 2024-09-16T20:50:36.674 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.674 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.674 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:36.674 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823579+0000", 2024-09-16T20:50:36.674 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5703000000000003e-05 2024-09-16T20:50:36.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.675 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.675 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:36.675 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823745+0000", 2024-09-16T20:50:36.675 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016549800000000001 2024-09-16T20:50:36.675 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.675 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:36.675 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.676 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.676 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.676 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12560 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 690176~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:36.676 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.759658+0000", 2024-09-16T20:50:36.676 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 45.888812481999999, 2024-09-16T20:50:36.677 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065858517000000005, 2024-09-16T20:50:36.677 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:36.677 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:36.677 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:36.677 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:36.677 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:36.678 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:36.678 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12560 2024-09-16T20:50:36.678 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.679 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:36.679 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.679 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:36.679 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759658+0000", 2024-09-16T20:50:36.679 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.679 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.679 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:36.679 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759658+0000", 2024-09-16T20:50:36.679 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.680 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.680 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:36.680 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759659+0000", 2024-09-16T20:50:36.680 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.17e-07 2024-09-16T20:50:36.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.680 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.681 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:36.681 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759660+0000", 2024-09-16T20:50:36.681 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0189999999999999e-06 2024-09-16T20:50:36.681 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.681 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.681 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:36.681 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759663+0000", 2024-09-16T20:50:36.681 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0010000000000002e-06 2024-09-16T20:50:36.681 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.681 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.681 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:36.682 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759665+0000", 2024-09-16T20:50:36.682 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5440000000000001e-06 2024-09-16T20:50:36.682 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.682 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.682 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:36.682 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825355+0000", 2024-09-16T20:50:36.682 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065690224000000005 2024-09-16T20:50:36.682 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.682 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.682 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:36.683 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825394+0000", 2024-09-16T20:50:36.683 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8810999999999998e-05 2024-09-16T20:50:36.683 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.683 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.683 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:36.683 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825517+0000", 2024-09-16T20:50:36.683 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012270099999999999 2024-09-16T20:50:36.683 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.683 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:36.684 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.684 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.684 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.684 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12561 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 695296~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:36.684 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.759668+0000", 2024-09-16T20:50:36.684 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 45.888802640000002, 2024-09-16T20:50:36.684 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066116999999999995, 2024-09-16T20:50:36.684 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:36.684 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:36.684 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:36.684 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:36.684 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:36.684 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:36.685 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12561 2024-09-16T20:50:36.685 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.685 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:36.685 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.685 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:36.685 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759668+0000", 2024-09-16T20:50:36.685 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.685 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.685 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.685 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:36.685 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759668+0000", 2024-09-16T20:50:36.685 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.685 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.686 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.686 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:36.686 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765379+0000", 2024-09-16T20:50:36.686 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057104410000000001 2024-09-16T20:50:36.686 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.686 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.686 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:36.686 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765383+0000", 2024-09-16T20:50:36.686 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2980000000000002e-06 2024-09-16T20:50:36.686 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.686 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.686 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:36.687 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765395+0000", 2024-09-16T20:50:36.687 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1671000000000001e-05 2024-09-16T20:50:36.687 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.687 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.687 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:36.687 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765403+0000", 2024-09-16T20:50:36.687 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1210000000000007e-06 2024-09-16T20:50:36.687 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.687 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.687 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:36.687 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825552+0000", 2024-09-16T20:50:36.687 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.060149498000000003 2024-09-16T20:50:36.687 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.687 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.688 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:36.688 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825625+0000", 2024-09-16T20:50:36.688 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2540999999999995e-05 2024-09-16T20:50:36.688 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.688 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.688 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:36.688 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825785+0000", 2024-09-16T20:50:36.688 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016043000000000001 2024-09-16T20:50:36.688 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.688 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:36.688 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.688 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.688 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.689 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12585 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 818176~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:36.689 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.765848+0000", 2024-09-16T20:50:36.689 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 45.882622669, 2024-09-16T20:50:36.689 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065843218999999995, 2024-09-16T20:50:36.689 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:36.689 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:36.689 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:36.689 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:36.689 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:36.689 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:36.689 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12585 2024-09-16T20:50:36.689 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.689 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:36.689 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.689 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:36.690 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765848+0000", 2024-09-16T20:50:36.690 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.690 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.690 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.690 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:36.690 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765848+0000", 2024-09-16T20:50:36.690 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.690 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.690 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.690 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:36.690 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768683+0000", 2024-09-16T20:50:36.690 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0028344519999999999 2024-09-16T20:50:36.690 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.690 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.690 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:36.691 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768686+0000", 2024-09-16T20:50:36.691 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8789999999999996e-06 2024-09-16T20:50:36.691 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.691 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.691 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:36.691 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768696+0000", 2024-09-16T20:50:36.691 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6709999999999994e-06 2024-09-16T20:50:36.691 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.691 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.691 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:36.691 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768702+0000", 2024-09-16T20:50:36.691 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2550000000000003e-06 2024-09-16T20:50:36.691 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.691 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.692 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:36.692 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.831499+0000", 2024-09-16T20:50:36.692 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.062797081000000005 2024-09-16T20:50:36.692 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.692 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.692 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:36.692 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.831538+0000", 2024-09-16T20:50:36.692 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8523999999999999e-05 2024-09-16T20:50:36.692 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.692 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.692 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:36.692 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.831691+0000", 2024-09-16T20:50:36.692 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015335699999999999 2024-09-16T20:50:36.692 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.693 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:36.693 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.693 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.693 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.693 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12597 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 879616~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:36.693 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.768883+0000", 2024-09-16T20:50:36.693 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 45.879587940999997, 2024-09-16T20:50:36.693 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065816527, 2024-09-16T20:50:36.693 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:36.693 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:36.694 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:36.694 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:36.694 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:36.694 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:36.694 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12597 2024-09-16T20:50:36.694 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.694 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:36.694 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.694 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:36.694 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768883+0000", 2024-09-16T20:50:36.694 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.694 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.694 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.694 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:36.695 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768883+0000", 2024-09-16T20:50:36.695 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.695 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.695 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.695 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:36.695 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771508+0000", 2024-09-16T20:50:36.695 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0026247589999999999 2024-09-16T20:50:36.695 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.695 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.695 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:36.695 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771513+0000", 2024-09-16T20:50:36.695 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9949999999999996e-06 2024-09-16T20:50:36.695 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.695 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.696 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:36.696 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771525+0000", 2024-09-16T20:50:36.696 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2033e-05 2024-09-16T20:50:36.696 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.696 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.696 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:36.696 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771532+0000", 2024-09-16T20:50:36.696 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4220000000000003e-06 2024-09-16T20:50:36.696 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.696 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.696 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:36.696 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.834532+0000", 2024-09-16T20:50:36.696 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.063000159 2024-09-16T20:50:36.696 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.697 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.697 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:36.697 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.834590+0000", 2024-09-16T20:50:36.697 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8218999999999999e-05 2024-09-16T20:50:36.697 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.697 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.697 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:36.697 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.834699+0000", 2024-09-16T20:50:36.697 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010894 2024-09-16T20:50:36.697 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.697 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:36.697 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.697 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.697 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.697 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12619 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 992256~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:36.698 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.774656+0000", 2024-09-16T20:50:36.698 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 45.873815166999997, 2024-09-16T20:50:36.698 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065715859000000001, 2024-09-16T20:50:36.698 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:36.698 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:36.698 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:36.698 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:36.698 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:36.698 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:36.698 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12619 2024-09-16T20:50:36.698 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.698 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:36.699 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.699 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:36.699 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774656+0000", 2024-09-16T20:50:36.699 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.699 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.699 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.699 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:36.699 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774656+0000", 2024-09-16T20:50:36.699 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.699 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.699 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.699 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:36.699 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774875+0000", 2024-09-16T20:50:36.700 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000218898 2024-09-16T20:50:36.700 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.700 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.700 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:36.700 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774879+0000", 2024-09-16T20:50:36.700 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0389999999999998e-06 2024-09-16T20:50:36.700 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.700 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.700 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:36.700 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774890+0000", 2024-09-16T20:50:36.700 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1250000000000001e-05 2024-09-16T20:50:36.700 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.700 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.700 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:36.701 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774897+0000", 2024-09-16T20:50:36.701 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7619999999999998e-06 2024-09-16T20:50:36.701 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.701 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.701 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:36.701 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840139+0000", 2024-09-16T20:50:36.701 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065242438 2024-09-16T20:50:36.701 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.701 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.701 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:36.701 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840186+0000", 2024-09-16T20:50:36.701 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6894999999999997e-05 2024-09-16T20:50:36.701 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.702 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.702 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:36.702 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840372+0000", 2024-09-16T20:50:36.702 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000185577 2024-09-16T20:50:36.702 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.702 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:36.702 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.702 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.702 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.702 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12620 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 997376~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:36.702 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.774908+0000", 2024-09-16T20:50:36.702 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 45.873562946, 2024-09-16T20:50:36.702 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065782754999999998, 2024-09-16T20:50:36.702 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:36.703 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:36.703 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:36.703 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:36.703 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:36.703 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:36.703 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12620 2024-09-16T20:50:36.703 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.703 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:36.703 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.703 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:36.703 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774908+0000", 2024-09-16T20:50:36.703 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.703 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.703 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.703 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:36.704 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774908+0000", 2024-09-16T20:50:36.704 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.704 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.704 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.704 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:36.704 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775102+0000", 2024-09-16T20:50:36.704 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000194098 2024-09-16T20:50:36.704 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.704 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.704 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:36.704 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775105+0000", 2024-09-16T20:50:36.704 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8739999999999999e-06 2024-09-16T20:50:36.704 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.705 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.705 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:36.705 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775111+0000", 2024-09-16T20:50:36.705 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9519999999999999e-06 2024-09-16T20:50:36.705 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.705 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.705 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:36.705 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775115+0000", 2024-09-16T20:50:36.705 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.002e-06 2024-09-16T20:50:36.705 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.705 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.705 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:36.705 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840409+0000", 2024-09-16T20:50:36.705 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065294195999999999 2024-09-16T20:50:36.706 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.706 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.706 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:36.706 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840507+0000", 2024-09-16T20:50:36.706 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7662000000000001e-05 2024-09-16T20:50:36.706 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.706 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.706 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:36.706 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840691+0000", 2024-09-16T20:50:36.706 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000183971 2024-09-16T20:50:36.706 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.706 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:36.706 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.707 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.707 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.707 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12621 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1002496~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:36.707 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775123+0000", 2024-09-16T20:50:36.707 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 45.873348249999999, 2024-09-16T20:50:36.707 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065829586999999995, 2024-09-16T20:50:36.707 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:36.707 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:36.707 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:36.707 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:36.707 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:36.707 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:36.707 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12621 2024-09-16T20:50:36.708 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.708 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:36.708 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.708 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:36.708 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775123+0000", 2024-09-16T20:50:36.708 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.708 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.708 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.708 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:36.708 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775123+0000", 2024-09-16T20:50:36.708 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.708 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.709 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.709 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:36.709 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775348+0000", 2024-09-16T20:50:36.709 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000225295 2024-09-16T20:50:36.709 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.709 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.709 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:36.709 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775352+0000", 2024-09-16T20:50:36.709 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0810000000000004e-06 2024-09-16T20:50:36.710 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.710 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.710 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:36.710 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775361+0000", 2024-09-16T20:50:36.710 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7399999999999993e-06 2024-09-16T20:50:36.710 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.710 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.710 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:36.710 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775367+0000", 2024-09-16T20:50:36.710 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5200000000000003e-06 2024-09-16T20:50:36.710 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.710 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.710 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:36.710 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840730+0000", 2024-09-16T20:50:36.711 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065362803999999997 2024-09-16T20:50:36.711 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.711 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.711 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:36.711 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840801+0000", 2024-09-16T20:50:36.711 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1006000000000004e-05 2024-09-16T20:50:36.711 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.711 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.711 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:36.711 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840952+0000", 2024-09-16T20:50:36.711 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015114100000000001 2024-09-16T20:50:36.711 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.711 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:36.711 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.712 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.712 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.712 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12622 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1007616~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:36.712 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775381+0000", 2024-09-16T20:50:36.712 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 45.873089415999999, 2024-09-16T20:50:36.712 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065822581000000005, 2024-09-16T20:50:36.712 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:36.712 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:36.712 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:36.712 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:36.712 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:36.712 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:36.712 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12622 2024-09-16T20:50:36.713 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.713 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:36.713 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.713 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:36.713 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775381+0000", 2024-09-16T20:50:36.713 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.713 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.713 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.713 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:36.713 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775381+0000", 2024-09-16T20:50:36.713 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.713 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.713 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.714 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:36.714 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775563+0000", 2024-09-16T20:50:36.714 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018153700000000001 2024-09-16T20:50:36.714 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.714 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.714 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:36.714 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775566+0000", 2024-09-16T20:50:36.714 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9620000000000002e-06 2024-09-16T20:50:36.714 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.714 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.714 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:36.714 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775572+0000", 2024-09-16T20:50:36.714 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8529999999999997e-06 2024-09-16T20:50:36.715 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.715 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.715 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:36.715 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775576+0000", 2024-09-16T20:50:36.715 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8920000000000002e-06 2024-09-16T20:50:36.715 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.715 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.715 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:36.715 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840990+0000", 2024-09-16T20:50:36.715 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065414599000000004 2024-09-16T20:50:36.715 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.715 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.715 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:36.716 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841068+0000", 2024-09-16T20:50:36.716 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8157000000000004e-05 2024-09-16T20:50:36.716 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.716 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.716 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:36.716 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841204+0000", 2024-09-16T20:50:36.716 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013558099999999999 2024-09-16T20:50:36.716 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.716 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:36.716 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.716 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.716 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.716 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12623 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1012736~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:36.716 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775583+0000", 2024-09-16T20:50:36.717 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 45.872887978000001, 2024-09-16T20:50:36.717 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065929087999999997, 2024-09-16T20:50:36.717 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:36.717 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:36.717 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:36.717 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:36.717 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:36.717 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:36.717 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12623 2024-09-16T20:50:36.717 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.717 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:36.717 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.717 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:36.718 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775583+0000", 2024-09-16T20:50:36.718 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.718 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.718 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.718 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:36.718 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775583+0000", 2024-09-16T20:50:36.718 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.718 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.718 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.718 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:36.718 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775804+0000", 2024-09-16T20:50:36.718 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00022116900000000001 2024-09-16T20:50:36.718 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.719 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.719 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:36.719 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775807+0000", 2024-09-16T20:50:36.719 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9529999999999999e-06 2024-09-16T20:50:36.719 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.719 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.719 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:36.719 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775814+0000", 2024-09-16T20:50:36.719 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7490000000000001e-06 2024-09-16T20:50:36.719 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.719 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.719 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:36.719 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775818+0000", 2024-09-16T20:50:36.719 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1999999999999996e-06 2024-09-16T20:50:36.720 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.720 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.720 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:36.720 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841267+0000", 2024-09-16T20:50:36.720 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065449127999999995 2024-09-16T20:50:36.720 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.720 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.720 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:36.720 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841344+0000", 2024-09-16T20:50:36.720 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6546999999999997e-05 2024-09-16T20:50:36.720 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.720 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.721 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:36.721 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841512+0000", 2024-09-16T20:50:36.721 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016834200000000001 2024-09-16T20:50:36.721 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.721 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:36.721 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.721 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.721 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.721 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12624 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1017856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:36.721 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:50:36.721 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 45.872644837999999, 2024-09-16T20:50:36.721 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065945866000000006, 2024-09-16T20:50:36.721 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:36.722 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:36.722 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:36.722 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:36.722 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:36.722 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:36.722 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12624 2024-09-16T20:50:36.722 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.722 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:36.722 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.722 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:36.722 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:50:36.722 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.722 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.722 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.723 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:36.723 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:50:36.723 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.723 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.723 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.723 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:36.723 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:50:36.723 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-16T20:50:36.723 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.723 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.723 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:36.723 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775828+0000", 2024-09-16T20:50:36.723 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3999999999999999e-06 2024-09-16T20:50:36.724 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.724 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.724 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:36.724 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775831+0000", 2024-09-16T20:50:36.724 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.478e-06 2024-09-16T20:50:36.724 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.724 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.724 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:36.724 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775834+0000", 2024-09-16T20:50:36.724 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9440000000000001e-06 2024-09-16T20:50:36.724 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.724 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.725 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:36.725 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841547+0000", 2024-09-16T20:50:36.725 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065712801000000001 2024-09-16T20:50:36.725 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.725 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.725 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:36.725 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841618+0000", 2024-09-16T20:50:36.725 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1151999999999999e-05 2024-09-16T20:50:36.725 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.725 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.725 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:36.725 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841772+0000", 2024-09-16T20:50:36.726 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000153721 2024-09-16T20:50:36.726 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.726 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:36.726 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.726 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.726 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.726 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12625 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1022976~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:36.726 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775840+0000", 2024-09-16T20:50:36.726 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 45.872631229, 2024-09-16T20:50:36.726 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066165451, 2024-09-16T20:50:36.726 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:36.726 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:36.726 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:36.727 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:36.727 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:36.727 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:36.727 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12625 2024-09-16T20:50:36.727 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.727 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:36.727 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.727 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:36.727 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775840+0000", 2024-09-16T20:50:36.727 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.727 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.727 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.728 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:36.728 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775840+0000", 2024-09-16T20:50:36.728 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.728 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.728 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.728 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:36.728 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776014+0000", 2024-09-16T20:50:36.728 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017441099999999999 2024-09-16T20:50:36.728 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.728 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.728 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:36.728 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776018+0000", 2024-09-16T20:50:36.728 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6509999999999999e-06 2024-09-16T20:50:36.729 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.729 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.729 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:36.729 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776026+0000", 2024-09-16T20:50:36.729 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.028e-06 2024-09-16T20:50:36.729 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.729 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.729 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:36.729 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776031+0000", 2024-09-16T20:50:36.729 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8400000000000002e-06 2024-09-16T20:50:36.729 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.729 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.729 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:36.730 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841801+0000", 2024-09-16T20:50:36.730 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065770472999999996 2024-09-16T20:50:36.730 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.730 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.730 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:36.730 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841847+0000", 2024-09-16T20:50:36.730 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5558e-05 2024-09-16T20:50:36.730 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.730 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.730 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:36.730 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842005+0000", 2024-09-16T20:50:36.730 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015849000000000001 2024-09-16T20:50:36.730 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.731 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:36.731 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.731 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.731 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.731 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12626 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1028096~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:36.731 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776041+0000", 2024-09-16T20:50:36.731 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 45.872429930000003, 2024-09-16T20:50:36.731 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066225260999999994, 2024-09-16T20:50:36.731 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:36.731 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:36.731 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:36.731 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:36.731 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:36.732 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:36.732 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12626 2024-09-16T20:50:36.732 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.732 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:36.732 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.732 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:36.732 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776041+0000", 2024-09-16T20:50:36.732 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.732 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.732 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.732 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:36.732 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776041+0000", 2024-09-16T20:50:36.732 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.733 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.733 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.733 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:36.733 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776263+0000", 2024-09-16T20:50:36.733 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00022193899999999999 2024-09-16T20:50:36.733 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.733 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.733 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:36.733 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776268+0000", 2024-09-16T20:50:36.733 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8049999999999997e-06 2024-09-16T20:50:36.733 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.733 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.733 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:36.733 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776280+0000", 2024-09-16T20:50:36.734 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2635e-05 2024-09-16T20:50:36.734 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.734 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.734 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:36.734 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776288+0000", 2024-09-16T20:50:36.734 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8199999999999997e-06 2024-09-16T20:50:36.734 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.734 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.734 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:36.734 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842041+0000", 2024-09-16T20:50:36.734 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065752506000000002 2024-09-16T20:50:36.735 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.735 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.735 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:36.735 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842123+0000", 2024-09-16T20:50:36.735 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2393999999999998e-05 2024-09-16T20:50:36.735 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.735 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.735 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:36.735 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842266+0000", 2024-09-16T20:50:36.735 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014316200000000001 2024-09-16T20:50:36.736 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.736 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:36.736 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.736 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.736 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.736 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12627 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1033216~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:36.736 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776301+0000", 2024-09-16T20:50:36.736 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 45.872169970000002, 2024-09-16T20:50:36.736 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066187814999999997, 2024-09-16T20:50:36.736 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:36.736 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:36.736 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:36.736 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:36.737 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:36.737 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:36.737 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12627 2024-09-16T20:50:36.737 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.737 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:36.737 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.737 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:36.737 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776301+0000", 2024-09-16T20:50:36.737 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.737 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.737 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.737 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:36.737 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776301+0000", 2024-09-16T20:50:36.738 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.738 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.738 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.738 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:36.738 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776473+0000", 2024-09-16T20:50:36.738 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017222400000000001 2024-09-16T20:50:36.738 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.738 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.738 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:36.738 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776477+0000", 2024-09-16T20:50:36.738 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7019999999999999e-06 2024-09-16T20:50:36.738 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.738 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.739 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:36.739 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776486+0000", 2024-09-16T20:50:36.739 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2119999999999992e-06 2024-09-16T20:50:36.739 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.739 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.739 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:36.739 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776492+0000", 2024-09-16T20:50:36.739 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8089999999999998e-06 2024-09-16T20:50:36.739 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.740 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.740 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:36.740 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842301+0000", 2024-09-16T20:50:36.740 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065809042999999998 2024-09-16T20:50:36.740 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.740 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.740 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:36.740 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842362+0000", 2024-09-16T20:50:36.740 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1260000000000006e-05 2024-09-16T20:50:36.740 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.740 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.740 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:36.740 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842489+0000", 2024-09-16T20:50:36.741 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012656500000000001 2024-09-16T20:50:36.741 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.741 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:36.741 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.741 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.741 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.741 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12628 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1038336~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:36.741 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776502+0000", 2024-09-16T20:50:36.741 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 45.871969323999998, 2024-09-16T20:50:36.741 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066283426000000006, 2024-09-16T20:50:36.741 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:36.741 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:36.741 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:36.742 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:36.742 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:36.742 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:36.742 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12628 2024-09-16T20:50:36.742 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.742 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:36.742 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.742 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:36.742 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776502+0000", 2024-09-16T20:50:36.742 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.742 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.742 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.742 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:36.743 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776502+0000", 2024-09-16T20:50:36.743 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.743 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.743 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.743 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:36.743 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776716+0000", 2024-09-16T20:50:36.743 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000214596 2024-09-16T20:50:36.743 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.743 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.743 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:36.743 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776723+0000", 2024-09-16T20:50:36.743 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4000000000000003e-06 2024-09-16T20:50:36.743 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.744 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.744 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:36.744 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776731+0000", 2024-09-16T20:50:36.744 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6129999999999998e-06 2024-09-16T20:50:36.744 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.744 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.744 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:36.744 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776737+0000", 2024-09-16T20:50:36.744 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1970000000000001e-06 2024-09-16T20:50:36.744 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.744 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.744 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:36.744 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842532+0000", 2024-09-16T20:50:36.745 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065795127999999994 2024-09-16T20:50:36.745 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.745 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.745 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:36.745 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842599+0000", 2024-09-16T20:50:36.745 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6711000000000006e-05 2024-09-16T20:50:36.745 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.745 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.745 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:36.745 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842785+0000", 2024-09-16T20:50:36.745 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018578099999999999 2024-09-16T20:50:36.745 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.746 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:36.746 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.746 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.746 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.746 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12629 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1043456~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:36.746 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776748+0000", 2024-09-16T20:50:36.746 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 45.871722947000002, 2024-09-16T20:50:36.746 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066303788000000002, 2024-09-16T20:50:36.746 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:36.746 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:36.746 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:36.746 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:36.746 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:36.747 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:36.747 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12629 2024-09-16T20:50:36.747 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.747 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:36.747 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.747 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:36.747 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776748+0000", 2024-09-16T20:50:36.747 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.747 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.747 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.747 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:36.747 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776748+0000", 2024-09-16T20:50:36.747 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:36.747 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.748 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.748 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:36.748 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776934+0000", 2024-09-16T20:50:36.748 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018598399999999999 2024-09-16T20:50:36.748 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.748 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.748 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:36.748 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776939+0000", 2024-09-16T20:50:36.748 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1000000000000003e-06 2024-09-16T20:50:36.748 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.748 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.748 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:36.748 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776954+0000", 2024-09-16T20:50:36.749 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5221e-05 2024-09-16T20:50:36.749 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.749 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.749 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:36.749 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776962+0000", 2024-09-16T20:50:36.749 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.554e-06 2024-09-16T20:50:36.749 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.749 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.749 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:36.749 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842813+0000", 2024-09-16T20:50:36.749 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065851006000000004 2024-09-16T20:50:36.749 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.750 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.750 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:36.750 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842886+0000", 2024-09-16T20:50:36.750 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3663000000000003e-05 2024-09-16T20:50:36.750 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:36.750 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:36.750 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:36.750 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.843052+0000", 2024-09-16T20:50:36.750 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016526 2024-09-16T20:50:36.750 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.750 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:36.750 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.750 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:36.751 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:36.751 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:36.752 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:50:36.752 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:36.751+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:36.790 INFO:teuthology.orchestra.run.smithi029.stderr:marked out osd.3. 2024-09-16T20:50:36.795 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T20:50:36.852 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:36.852+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:36.901 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:36.901 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:50:36.901 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:50:36.901 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:36.909 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:50:36.953 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:36.952+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:37.053 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:37.052+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:37.057 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:37.057 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:50:37.057 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:50:37.057 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:50:37.057 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:37.066 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:50:37.154 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:37.153+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:37.174 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:37.174 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:50:37.174 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:50:37.174 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:50:37.174 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.175 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5286.0:10643 3.17 3.f6273fb7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-16T20:50:37.175 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:50:37.175 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 88.382088703999997, 2024-09-16T20:50:37.175 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.76672347200000002, 2024-09-16T20:50:37.175 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:37.175 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:37.175 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:37.175 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:37.175 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5286", 2024-09-16T20:50:37.175 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/4135433989", 2024-09-16T20:50:37.175 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 10643 2024-09-16T20:50:37.175 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.175 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:37.175 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.176 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:37.176 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:50:37.176 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:37.176 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.176 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.176 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:37.176 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:50:37.176 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:37.176 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.176 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.176 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:37.176 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790489+0000", 2024-09-16T20:50:37.176 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5910000000000001e-06 2024-09-16T20:50:37.176 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.176 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.177 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:37.177 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790493+0000", 2024-09-16T20:50:37.177 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2669999999999998e-06 2024-09-16T20:50:37.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.177 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.177 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:37.177 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790500+0000", 2024-09-16T20:50:37.177 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3780000000000003e-06 2024-09-16T20:50:37.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.177 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.178 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:37.178 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790505+0000", 2024-09-16T20:50:37.178 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1560000000000003e-06 2024-09-16T20:50:37.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.178 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.178 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:37.178 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790559+0000", 2024-09-16T20:50:37.178 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3494e-05 2024-09-16T20:50:37.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.178 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.178 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:37.178 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790566+0000", 2024-09-16T20:50:37.179 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7649999999999993e-06 2024-09-16T20:50:37.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.179 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.179 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:37.179 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:09.557198+0000", 2024-09-16T20:50:37.179 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.76663136200000004 2024-09-16T20:50:37.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.179 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.179 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:37.179 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:09.557211+0000", 2024-09-16T20:50:37.179 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3458999999999999e-05 2024-09-16T20:50:37.179 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:37.179 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:37.180 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:37.180 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.180 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.180 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5512.0:450 3.2 3:44e0cffa:::rbd_header.110ef0d7a535:head [watch reconnect cookie 140263298018912 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:50:37.180 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:50:37.180 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 60.610065790999997, 2024-09-16T20:50:37.180 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0084581939999999, 2024-09-16T20:50:37.181 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:37.181 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:37.181 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:37.181 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:37.181 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5512", 2024-09-16T20:50:37.181 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:50:37.181 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 450 2024-09-16T20:50:37.181 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.181 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:37.181 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.181 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:37.181 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:50:37.181 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:37.182 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.182 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.182 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:37.182 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:50:37.182 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:37.182 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.182 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.182 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:37.182 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562512+0000", 2024-09-16T20:50:37.182 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.009e-06 2024-09-16T20:50:37.182 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.182 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.182 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:37.182 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562517+0000", 2024-09-16T20:50:37.183 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9339999999999997e-06 2024-09-16T20:50:37.183 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.183 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.183 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:37.183 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562530+0000", 2024-09-16T20:50:37.183 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3495e-05 2024-09-16T20:50:37.183 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.183 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.183 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:37.183 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562536+0000", 2024-09-16T20:50:37.183 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5690000000000004e-06 2024-09-16T20:50:37.183 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.183 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.184 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:37.184 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562591+0000", 2024-09-16T20:50:37.184 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5761999999999999e-05 2024-09-16T20:50:37.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.184 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.184 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:50:37.184 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562599+0000", 2024-09-16T20:50:37.184 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9179999999999994e-06 2024-09-16T20:50:37.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.184 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.184 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:37.184 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.565971+0000", 2024-09-16T20:50:37.184 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003371546 2024-09-16T20:50:37.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.185 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.185 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:37.185 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566019+0000", 2024-09-16T20:50:37.185 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7849000000000002e-05 2024-09-16T20:50:37.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.185 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.185 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:37.185 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.569382+0000", 2024-09-16T20:50:37.185 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.003362833 2024-09-16T20:50:37.186 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.186 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.186 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:37.186 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.569529+0000", 2024-09-16T20:50:37.186 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000147823 2024-09-16T20:50:37.186 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.186 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.186 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:37.186 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570969+0000", 2024-09-16T20:50:37.186 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001439456 2024-09-16T20:50:37.186 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:37.187 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:37.187 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:37.187 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.187 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.187 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:14996 3.2 3:44e0cffa:::rbd_header.110ef0d7a535:head [watch reconnect cookie 140213436269840 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:50:37.187 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:50:37.187 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 60.610017378999999, 2024-09-16T20:50:37.187 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.008813943, 2024-09-16T20:50:37.187 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:37.187 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:37.187 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:37.187 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:37.187 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:50:37.187 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:37.188 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 14996 2024-09-16T20:50:37.188 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.188 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:37.188 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.188 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:37.188 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:50:37.188 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:37.188 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.188 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.188 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:37.188 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:50:37.188 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:37.188 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.188 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.189 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:37.189 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562560+0000", 2024-09-16T20:50:37.189 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.06e-07 2024-09-16T20:50:37.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.189 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.189 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:37.189 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562563+0000", 2024-09-16T20:50:37.189 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.495e-06 2024-09-16T20:50:37.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.189 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.189 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:37.189 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562571+0000", 2024-09-16T20:50:37.189 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1950000000000003e-06 2024-09-16T20:50:37.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.189 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.189 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:37.190 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562579+0000", 2024-09-16T20:50:37.190 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2649999999999999e-06 2024-09-16T20:50:37.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.190 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.190 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:37.190 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562673+0000", 2024-09-16T20:50:37.190 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4118999999999998e-05 2024-09-16T20:50:37.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.190 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.190 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:50:37.190 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562677+0000", 2024-09-16T20:50:37.190 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5179999999999998e-06 2024-09-16T20:50:37.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.190 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.191 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:37.191 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566461+0000", 2024-09-16T20:50:37.191 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0037837259999999998 2024-09-16T20:50:37.191 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.191 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.191 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:37.191 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566468+0000", 2024-09-16T20:50:37.191 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3350000000000001e-06 2024-09-16T20:50:37.191 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.191 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.191 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:37.191 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570306+0000", 2024-09-16T20:50:37.191 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.003837997 2024-09-16T20:50:37.191 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.191 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.192 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:37.192 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570392+0000", 2024-09-16T20:50:37.192 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5872999999999996e-05 2024-09-16T20:50:37.192 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.192 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.192 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:37.192 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571373+0000", 2024-09-16T20:50:37.192 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00098071399999999993 2024-09-16T20:50:37.192 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:37.192 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:37.192 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:37.192 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.192 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.192 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5255.0:1931 3.2 3:47882819:::rbd_data.110eb5f95afc.0000000000000009:head [read 217088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:50:37.192 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:50:37.193 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 60.609978628999997, 2024-09-16T20:50:37.193 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0082752880000001, 2024-09-16T20:50:37.193 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:37.193 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:37.193 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:37.193 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:37.193 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5255", 2024-09-16T20:50:37.193 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:50:37.193 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1931 2024-09-16T20:50:37.193 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.193 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:37.193 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.193 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:37.193 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:50:37.194 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:37.194 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.194 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.194 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:37.194 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:50:37.194 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:37.194 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.194 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.194 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:37.194 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:50:37.194 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9699999999999996e-07 2024-09-16T20:50:37.194 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.194 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.194 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:37.195 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562601+0000", 2024-09-16T20:50:37.195 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8600000000000001e-06 2024-09-16T20:50:37.195 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.195 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.195 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:37.195 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562608+0000", 2024-09-16T20:50:37.195 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9e-06 2024-09-16T20:50:37.195 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.195 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.195 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:37.195 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562615+0000", 2024-09-16T20:50:37.196 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8179999999999998e-06 2024-09-16T20:50:37.196 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.196 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.196 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:37.196 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562703+0000", 2024-09-16T20:50:37.196 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7811000000000003e-05 2024-09-16T20:50:37.196 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.196 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.196 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:50:37.196 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562706+0000", 2024-09-16T20:50:37.196 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4920000000000002e-06 2024-09-16T20:50:37.196 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.196 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.196 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:37.196 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566685+0000", 2024-09-16T20:50:37.197 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0039792780000000002 2024-09-16T20:50:37.197 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.197 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.197 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:37.197 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566693+0000", 2024-09-16T20:50:37.197 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4730000000000003e-06 2024-09-16T20:50:37.197 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.197 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.197 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:37.197 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570737+0000", 2024-09-16T20:50:37.197 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0040436749999999 2024-09-16T20:50:37.197 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.197 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.197 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:37.197 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570873+0000", 2024-09-16T20:50:37.198 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000136484 2024-09-16T20:50:37.198 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:37.198 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:37.198 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:37.198 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.198 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.198 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5255.0:1939 3.2 3:47882819:::rbd_data.110eb5f95afc.0000000000000009:head [read 233472~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:50:37.198 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:50:37.198 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 60.609823763000001, 2024-09-16T20:50:37.198 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0084657589999999, 2024-09-16T20:50:37.198 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:37.198 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:37.198 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:37.198 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:37.199 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5255", 2024-09-16T20:50:37.199 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:50:37.199 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1939 2024-09-16T20:50:37.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.199 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:37.199 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.199 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:37.199 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:50:37.199 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:37.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.199 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.199 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:37.199 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:50:37.199 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:37.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.200 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.200 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:37.200 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:50:37.200 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5199999999999997e-07 2024-09-16T20:50:37.200 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.200 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.200 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:37.200 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562757+0000", 2024-09-16T20:50:37.200 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4979999999999998e-06 2024-09-16T20:50:37.200 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.200 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.200 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:37.200 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562764+0000", 2024-09-16T20:50:37.200 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1470000000000004e-06 2024-09-16T20:50:37.200 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.200 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.201 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:37.201 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562768+0000", 2024-09-16T20:50:37.201 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7929999999999999e-06 2024-09-16T20:50:37.201 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.201 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.201 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:37.201 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562805+0000", 2024-09-16T20:50:37.201 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7477e-05 2024-09-16T20:50:37.201 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.201 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.202 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting_for_map not empty", 2024-09-16T20:50:37.202 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562810+0000", 2024-09-16T20:50:37.202 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0819999999999998e-06 2024-09-16T20:50:37.202 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.202 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.202 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:37.202 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566717+0000", 2024-09-16T20:50:37.202 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0039072600000000001 2024-09-16T20:50:37.202 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.202 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.202 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:37.202 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566724+0000", 2024-09-16T20:50:37.202 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4880000000000004e-06 2024-09-16T20:50:37.202 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.202 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.203 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:37.203 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571009+0000", 2024-09-16T20:50:37.203 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.004284677 2024-09-16T20:50:37.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.203 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.203 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:37.203 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571218+0000", 2024-09-16T20:50:37.203 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00020978500000000001 2024-09-16T20:50:37.203 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:37.203 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:37.203 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:37.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.203 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.203 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:11020 3.1f 3:fee2d1dd:::rbd_data.110ea93691bb.000000000000000e:23 [read 955392~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:37.203 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.693811+0000", 2024-09-16T20:50:37.204 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 46.478765105999997, 2024-09-16T20:50:37.204 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056833005999999998, 2024-09-16T20:50:37.204 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:37.204 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:37.204 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:37.204 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:37.204 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:37.204 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:37.204 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 11020 2024-09-16T20:50:37.204 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.204 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:37.204 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.204 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:37.204 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693811+0000", 2024-09-16T20:50:37.205 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:37.205 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.205 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.205 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:37.205 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693811+0000", 2024-09-16T20:50:37.205 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:37.205 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.205 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.205 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:37.205 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693812+0000", 2024-09-16T20:50:37.205 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6100000000000001e-07 2024-09-16T20:50:37.205 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.205 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.205 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:37.206 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693815+0000", 2024-09-16T20:50:37.206 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7389999999999999e-06 2024-09-16T20:50:37.206 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.206 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.206 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:37.206 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693823+0000", 2024-09-16T20:50:37.206 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0439999999999996e-06 2024-09-16T20:50:37.206 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.206 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.206 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:37.206 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693828+0000", 2024-09-16T20:50:37.206 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7440000000000004e-06 2024-09-16T20:50:37.206 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.206 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.207 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:37.207 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750357+0000", 2024-09-16T20:50:37.207 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056529192999999998 2024-09-16T20:50:37.207 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.207 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.207 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:37.207 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750429+0000", 2024-09-16T20:50:37.207 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1557000000000006e-05 2024-09-16T20:50:37.207 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.207 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.207 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:37.207 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750644+0000", 2024-09-16T20:50:37.207 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00021526800000000001 2024-09-16T20:50:37.207 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:37.207 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:37.208 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:37.208 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.208 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.208 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:11021 3.1f 3:fee2d1dd:::rbd_data.110ea93691bb.000000000000000e:23 [read 960512~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:37.208 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.693837+0000", 2024-09-16T20:50:37.208 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 46.478739218000001, 2024-09-16T20:50:37.208 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057155138000000001, 2024-09-16T20:50:37.208 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:37.208 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:37.208 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:37.208 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:37.208 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:37.208 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:37.208 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 11021 2024-09-16T20:50:37.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.209 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:37.209 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.209 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:37.209 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693837+0000", 2024-09-16T20:50:37.209 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:37.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.209 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.209 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:37.209 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693837+0000", 2024-09-16T20:50:37.209 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:37.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.209 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.209 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:37.209 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699566+0000", 2024-09-16T20:50:37.210 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057283639999999997 2024-09-16T20:50:37.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.210 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.210 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:37.210 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699571+0000", 2024-09-16T20:50:37.210 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.604e-06 2024-09-16T20:50:37.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.210 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.210 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:37.210 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699583+0000", 2024-09-16T20:50:37.210 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1372999999999999e-05 2024-09-16T20:50:37.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.210 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.211 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:37.211 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699591+0000", 2024-09-16T20:50:37.211 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5259999999999993e-06 2024-09-16T20:50:37.211 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.212 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.212 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:37.212 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750674+0000", 2024-09-16T20:50:37.212 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051083154999999998 2024-09-16T20:50:37.212 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.212 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.212 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:37.212 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750745+0000", 2024-09-16T20:50:37.212 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0933999999999995e-05 2024-09-16T20:50:37.212 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.212 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.212 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:37.212 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750992+0000", 2024-09-16T20:50:37.213 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00024718199999999998 2024-09-16T20:50:37.213 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:37.213 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:37.213 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:37.213 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.213 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.213 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8790 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2590720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:37.213 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057257+0000", 2024-09-16T20:50:37.213 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.115319898000003, 2024-09-16T20:50:37.213 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056791169000000002, 2024-09-16T20:50:37.213 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:37.213 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:37.213 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:37.214 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:37.214 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:37.214 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:37.214 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8790 2024-09-16T20:50:37.214 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.214 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:37.214 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.214 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:37.214 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057257+0000", 2024-09-16T20:50:37.214 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:37.214 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.214 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.214 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:37.214 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057257+0000", 2024-09-16T20:50:37.214 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:37.215 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.215 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.215 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:37.215 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057257+0000", 2024-09-16T20:50:37.215 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6600000000000002e-07 2024-09-16T20:50:37.215 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.215 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.215 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:37.215 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057260+0000", 2024-09-16T20:50:37.215 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7240000000000001e-06 2024-09-16T20:50:37.215 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.215 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.215 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:37.215 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057267+0000", 2024-09-16T20:50:37.215 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0600000000000002e-06 2024-09-16T20:50:37.216 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.216 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.216 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:37.216 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057272+0000", 2024-09-16T20:50:37.216 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0390000000000004e-06 2024-09-16T20:50:37.216 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.216 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.216 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:37.216 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.113950+0000", 2024-09-16T20:50:37.216 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056677989999999998 2024-09-16T20:50:37.216 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.216 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.216 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:37.216 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.113993+0000", 2024-09-16T20:50:37.217 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3205999999999999e-05 2024-09-16T20:50:37.217 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.217 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.217 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:37.217 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114048+0000", 2024-09-16T20:50:37.217 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4684e-05 2024-09-16T20:50:37.217 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:37.217 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:37.217 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:37.217 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.217 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.217 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8791 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2332672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:37.217 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057279+0000", 2024-09-16T20:50:37.217 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.115297482999999, 2024-09-16T20:50:37.218 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056890067000000002, 2024-09-16T20:50:37.218 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:37.218 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:37.218 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:37.218 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:37.218 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:37.218 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:37.218 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8791 2024-09-16T20:50:37.218 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.218 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:37.218 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.218 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:37.218 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057279+0000", 2024-09-16T20:50:37.218 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:37.219 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.219 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.219 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:37.219 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057279+0000", 2024-09-16T20:50:37.219 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:37.219 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.219 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.219 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:37.219 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057279+0000", 2024-09-16T20:50:37.219 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.15e-07 2024-09-16T20:50:37.219 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.219 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.219 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:37.219 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057281+0000", 2024-09-16T20:50:37.220 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8080000000000001e-06 2024-09-16T20:50:37.220 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.220 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.220 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:37.220 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057287+0000", 2024-09-16T20:50:37.220 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5999999999999997e-06 2024-09-16T20:50:37.220 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.220 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.220 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:37.220 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057291+0000", 2024-09-16T20:50:37.220 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5990000000000004e-06 2024-09-16T20:50:37.220 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.220 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.220 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:37.221 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114078+0000", 2024-09-16T20:50:37.221 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056786246999999998 2024-09-16T20:50:37.221 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.221 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.221 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:37.221 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114118+0000", 2024-09-16T20:50:37.221 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9898999999999999e-05 2024-09-16T20:50:37.221 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.221 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.221 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:37.221 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114169+0000", 2024-09-16T20:50:37.221 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1499000000000003e-05 2024-09-16T20:50:37.221 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:37.221 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:37.222 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:37.222 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.222 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.222 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8792 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2609152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:37.222 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:50:37.222 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.115279501000003, 2024-09-16T20:50:37.222 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057003524999999999, 2024-09-16T20:50:37.222 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:37.222 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:37.222 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:37.222 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:37.222 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:37.222 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:37.222 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8792 2024-09-16T20:50:37.222 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.223 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:37.223 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.223 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:37.223 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:50:37.223 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:37.223 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.223 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.223 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:37.223 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:50:37.223 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:37.223 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.223 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.223 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:37.223 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:50:37.223 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0400000000000002e-07 2024-09-16T20:50:37.224 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.224 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.224 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:37.224 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057299+0000", 2024-09-16T20:50:37.224 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7460000000000001e-06 2024-09-16T20:50:37.224 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.224 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.224 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:37.224 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057304+0000", 2024-09-16T20:50:37.224 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3210000000000004e-06 2024-09-16T20:50:37.224 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.224 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.224 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:37.224 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057310+0000", 2024-09-16T20:50:37.224 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4360000000000001e-06 2024-09-16T20:50:37.225 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.225 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.225 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:37.225 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114200+0000", 2024-09-16T20:50:37.225 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056889954 2024-09-16T20:50:37.225 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.225 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.225 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:37.225 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114243+0000", 2024-09-16T20:50:37.225 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2998999999999999e-05 2024-09-16T20:50:37.225 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.225 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.225 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:37.225 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114300+0000", 2024-09-16T20:50:37.226 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7765e-05 2024-09-16T20:50:37.226 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:37.226 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:37.226 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:37.226 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.226 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.226 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8794 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2351104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:37.226 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057327+0000", 2024-09-16T20:50:37.226 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.115249177000003, 2024-09-16T20:50:37.226 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057101130999999999, 2024-09-16T20:50:37.226 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:37.226 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:37.226 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:37.226 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:37.227 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:37.227 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:37.227 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8794 2024-09-16T20:50:37.227 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.227 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:37.227 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.227 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:37.227 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057327+0000", 2024-09-16T20:50:37.227 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:37.227 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.227 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.227 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:37.227 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057327+0000", 2024-09-16T20:50:37.227 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:37.227 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.228 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.228 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:37.228 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057328+0000", 2024-09-16T20:50:37.228 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7399999999999999e-07 2024-09-16T20:50:37.228 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.228 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.228 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:37.228 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057330+0000", 2024-09-16T20:50:37.229 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2199999999999999e-06 2024-09-16T20:50:37.229 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.229 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.229 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:37.229 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057336+0000", 2024-09-16T20:50:37.229 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1240000000000002e-06 2024-09-16T20:50:37.229 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.229 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.229 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:37.229 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057341+0000", 2024-09-16T20:50:37.229 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8470000000000003e-06 2024-09-16T20:50:37.229 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.229 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.229 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:37.230 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114334+0000", 2024-09-16T20:50:37.230 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056993267 2024-09-16T20:50:37.230 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.230 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.230 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:37.230 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114376+0000", 2024-09-16T20:50:37.230 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1638999999999999e-05 2024-09-16T20:50:37.230 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.230 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.230 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:37.230 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114428+0000", 2024-09-16T20:50:37.230 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2660000000000001e-05 2024-09-16T20:50:37.230 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:37.230 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:37.230 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:37.231 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.231 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.231 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8798 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2297856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:37.231 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:50:37.231 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.115229450000001, 2024-09-16T20:50:37.231 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057277041000000001, 2024-09-16T20:50:37.231 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:37.231 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:37.231 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:37.231 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:37.231 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:37.231 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:37.231 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8798 2024-09-16T20:50:37.231 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.231 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:37.231 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.232 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:37.232 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:50:37.232 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:37.232 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.232 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.232 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:37.232 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:50:37.232 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:37.232 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.232 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.232 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:37.232 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:50:37.232 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2800000000000003e-07 2024-09-16T20:50:37.232 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.233 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.233 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:37.233 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057354+0000", 2024-09-16T20:50:37.233 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7780000000000002e-06 2024-09-16T20:50:37.233 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.233 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.233 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:37.233 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057360+0000", 2024-09-16T20:50:37.233 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0419999999999999e-06 2024-09-16T20:50:37.233 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.233 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.233 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:37.233 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057365+0000", 2024-09-16T20:50:37.233 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7929999999999997e-06 2024-09-16T20:50:37.233 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.233 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.234 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:37.234 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114460+0000", 2024-09-16T20:50:37.234 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057094662999999997 2024-09-16T20:50:37.234 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.234 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.234 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:37.234 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114532+0000", 2024-09-16T20:50:37.234 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2881999999999998e-05 2024-09-16T20:50:37.234 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.234 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.234 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:37.234 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114624+0000", 2024-09-16T20:50:37.234 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1555e-05 2024-09-16T20:50:37.234 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:37.234 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:37.235 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:37.235 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.235 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.235 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8800 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2316288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:37.235 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:50:37.235 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.115205510999999, 2024-09-16T20:50:37.235 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.05746946, 2024-09-16T20:50:37.235 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:37.235 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:37.235 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:37.235 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:37.235 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:37.235 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:37.235 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8800 2024-09-16T20:50:37.235 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.235 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:37.236 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.236 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:37.236 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:50:37.236 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:37.236 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.236 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.236 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:37.236 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:50:37.236 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:37.236 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.236 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.236 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:37.236 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:50:37.236 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5199999999999998e-07 2024-09-16T20:50:37.236 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.237 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.237 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:37.237 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057373+0000", 2024-09-16T20:50:37.237 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.79e-06 2024-09-16T20:50:37.237 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.237 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.237 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:37.237 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057379+0000", 2024-09-16T20:50:37.237 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6699999999999999e-06 2024-09-16T20:50:37.237 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.238 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.238 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:37.238 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057386+0000", 2024-09-16T20:50:37.238 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3170000000000004e-06 2024-09-16T20:50:37.238 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.238 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.238 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:37.238 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114655+0000", 2024-09-16T20:50:37.238 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057269420000000001 2024-09-16T20:50:37.238 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.238 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.238 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:37.238 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114734+0000", 2024-09-16T20:50:37.238 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8084000000000007e-05 2024-09-16T20:50:37.239 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.239 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.239 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:37.239 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114840+0000", 2024-09-16T20:50:37.239 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000106827 2024-09-16T20:50:37.239 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:37.239 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:37.239 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:37.239 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.239 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.239 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8801 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2334720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:37.239 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:50:37.239 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.115184480000003, 2024-09-16T20:50:37.239 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057594682000000001, 2024-09-16T20:50:37.239 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:37.240 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:37.240 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:37.240 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:37.240 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:37.240 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:37.240 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8801 2024-09-16T20:50:37.240 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.240 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:37.240 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.240 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:37.240 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:50:37.240 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:37.240 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.240 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.240 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:37.241 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:50:37.241 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:37.241 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.241 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.241 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:37.241 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:50:37.241 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6899999999999998e-07 2024-09-16T20:50:37.241 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.241 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.241 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:37.241 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057394+0000", 2024-09-16T20:50:37.241 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8300000000000001e-06 2024-09-16T20:50:37.242 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.242 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.242 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:37.242 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057400+0000", 2024-09-16T20:50:37.242 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6500000000000001e-06 2024-09-16T20:50:37.242 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.242 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.242 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:37.242 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057404+0000", 2024-09-16T20:50:37.242 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4669999999999998e-06 2024-09-16T20:50:37.242 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.242 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.242 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:37.242 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114874+0000", 2024-09-16T20:50:37.243 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057469602000000002 2024-09-16T20:50:37.243 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.243 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.243 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:37.243 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114933+0000", 2024-09-16T20:50:37.243 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9500000000000003e-05 2024-09-16T20:50:37.243 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.243 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.243 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:37.243 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114987+0000", 2024-09-16T20:50:37.243 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3263999999999997e-05 2024-09-16T20:50:37.243 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:37.243 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:37.243 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:37.244 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.244 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.244 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8802 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2353152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:37.244 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:50:37.244 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.115135795, 2024-09-16T20:50:37.244 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057714079000000001, 2024-09-16T20:50:37.244 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:37.244 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:37.244 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:37.244 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:37.244 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:37.244 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:37.244 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8802 2024-09-16T20:50:37.244 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.244 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:37.245 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.245 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:37.245 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:50:37.245 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:37.245 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.245 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.245 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:37.245 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:50:37.245 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:37.245 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.245 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.245 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:37.245 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:50:37.245 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4999999999999998e-07 2024-09-16T20:50:37.246 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.246 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.246 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:37.246 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057446+0000", 2024-09-16T20:50:37.246 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4240000000000003e-06 2024-09-16T20:50:37.246 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.246 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.246 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:37.246 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057452+0000", 2024-09-16T20:50:37.246 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8379999999999996e-06 2024-09-16T20:50:37.246 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.246 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.246 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:37.246 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057460+0000", 2024-09-16T20:50:37.247 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9629999999999998e-06 2024-09-16T20:50:37.247 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.247 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.247 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:37.247 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115015+0000", 2024-09-16T20:50:37.247 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057554389999999997 2024-09-16T20:50:37.247 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.247 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.247 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:37.247 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115082+0000", 2024-09-16T20:50:37.247 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7733999999999999e-05 2024-09-16T20:50:37.247 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.247 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.248 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:37.248 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115155+0000", 2024-09-16T20:50:37.248 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2280000000000006e-05 2024-09-16T20:50:37.248 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:37.248 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:37.248 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:37.248 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.248 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.248 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8804 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2555904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:37.248 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:50:37.249 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.115109478999997, 2024-09-16T20:50:37.249 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057833862, 2024-09-16T20:50:37.249 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:37.249 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:37.249 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:37.249 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:37.249 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:37.249 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:37.249 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8804 2024-09-16T20:50:37.249 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.249 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:37.249 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.249 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:37.250 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:50:37.250 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:37.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.250 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.250 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:37.250 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:50:37.250 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:37.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.250 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.250 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:37.250 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:50:37.250 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7899999999999999e-07 2024-09-16T20:50:37.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.251 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.251 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:37.251 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057469+0000", 2024-09-16T20:50:37.251 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.001e-06 2024-09-16T20:50:37.251 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.251 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.251 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:37.251 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057475+0000", 2024-09-16T20:50:37.251 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7629999999999997e-06 2024-09-16T20:50:37.251 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.251 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.251 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:37.251 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057480+0000", 2024-09-16T20:50:37.251 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8069999999999999e-06 2024-09-16T20:50:37.251 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.252 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.252 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:37.252 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115183+0000", 2024-09-16T20:50:37.252 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057703364 2024-09-16T20:50:37.252 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.252 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.252 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:37.252 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115227+0000", 2024-09-16T20:50:37.252 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3903999999999998e-05 2024-09-16T20:50:37.252 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.252 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.252 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:37.252 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115301+0000", 2024-09-16T20:50:37.252 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3543999999999999e-05 2024-09-16T20:50:37.253 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:37.253 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:37.253 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:37.253 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.253 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.253 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8805 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2574336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:37.253 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057487+0000", 2024-09-16T20:50:37.253 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.115089150999999, 2024-09-16T20:50:37.253 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058021027000000003, 2024-09-16T20:50:37.253 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:37.253 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:37.253 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:37.253 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:37.253 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:37.253 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:37.254 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8805 2024-09-16T20:50:37.254 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.254 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:37.254 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.254 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:37.254 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057487+0000", 2024-09-16T20:50:37.254 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:37.254 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.254 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.254 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:37.254 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057487+0000", 2024-09-16T20:50:37.254 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:37.254 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.254 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.254 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:37.255 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057488+0000", 2024-09-16T20:50:37.255 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5199999999999998e-07 2024-09-16T20:50:37.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.255 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.255 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:37.255 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057490+0000", 2024-09-16T20:50:37.255 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.939e-06 2024-09-16T20:50:37.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.255 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.255 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:37.255 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057495+0000", 2024-09-16T20:50:37.255 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6439999999999997e-06 2024-09-16T20:50:37.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.255 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.255 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:37.255 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057500+0000", 2024-09-16T20:50:37.256 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8119999999999998e-06 2024-09-16T20:50:37.256 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.256 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.256 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:37.256 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115341+0000", 2024-09-16T20:50:37.256 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057841341999999997 2024-09-16T20:50:37.256 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.256 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.256 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:37.256 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115422+0000", 2024-09-16T20:50:37.256 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1003999999999999e-05 2024-09-16T20:50:37.256 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.256 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.256 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:37.257 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115508+0000", 2024-09-16T20:50:37.257 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5933999999999994e-05 2024-09-16T20:50:37.257 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:37.257 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:37.257 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:37.257 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.257 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.257 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8807 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2592768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:37.257 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057505+0000", 2024-09-16T20:50:37.257 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.115070940999999, 2024-09-16T20:50:37.257 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058186633000000001, 2024-09-16T20:50:37.257 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:37.257 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:37.257 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:37.257 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:37.258 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:37.258 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:37.258 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8807 2024-09-16T20:50:37.258 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.258 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:37.258 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.258 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:37.258 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057505+0000", 2024-09-16T20:50:37.258 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:37.258 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.258 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.258 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:37.258 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057505+0000", 2024-09-16T20:50:37.258 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:37.259 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.259 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.259 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:37.259 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057506+0000", 2024-09-16T20:50:37.259 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.27e-07 2024-09-16T20:50:37.259 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.259 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.259 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:37.259 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057508+0000", 2024-09-16T20:50:37.259 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7400000000000001e-06 2024-09-16T20:50:37.259 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.259 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.259 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:37.259 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057513+0000", 2024-09-16T20:50:37.260 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.468e-06 2024-09-16T20:50:37.260 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.260 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.260 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:37.260 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057517+0000", 2024-09-16T20:50:37.260 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4329999999999997e-06 2024-09-16T20:50:37.260 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.260 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.260 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:37.260 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115538+0000", 2024-09-16T20:50:37.260 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058020355000000003 2024-09-16T20:50:37.260 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.260 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.260 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:37.261 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115616+0000", 2024-09-16T20:50:37.261 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7998999999999996e-05 2024-09-16T20:50:37.261 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.261 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.261 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:37.261 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115692+0000", 2024-09-16T20:50:37.261 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6310999999999995e-05 2024-09-16T20:50:37.261 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:37.261 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:37.261 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:37.261 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.261 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.261 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8808 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2611200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:37.261 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:50:37.262 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.115053680000003, 2024-09-16T20:50:37.262 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058354046, 2024-09-16T20:50:37.262 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:37.262 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:37.262 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:37.262 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:37.262 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:37.262 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:37.262 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8808 2024-09-16T20:50:37.262 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.262 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:37.262 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.262 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:37.262 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:50:37.262 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:37.262 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.263 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.263 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:37.263 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:50:37.263 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:37.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.263 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.263 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:37.263 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:50:37.263 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2500000000000001e-07 2024-09-16T20:50:37.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.263 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.263 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:37.263 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057525+0000", 2024-09-16T20:50:37.263 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5549999999999999e-06 2024-09-16T20:50:37.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.264 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.264 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:37.264 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057530+0000", 2024-09-16T20:50:37.264 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5489999999999997e-06 2024-09-16T20:50:37.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.264 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.264 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:37.264 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057535+0000", 2024-09-16T20:50:37.264 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5709999999999999e-06 2024-09-16T20:50:37.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.264 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.264 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:37.264 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115722+0000", 2024-09-16T20:50:37.264 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058187323999999999 2024-09-16T20:50:37.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.265 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.265 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:37.265 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115799+0000", 2024-09-16T20:50:37.265 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6445000000000004e-05 2024-09-16T20:50:37.265 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.265 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.265 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:37.265 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115877+0000", 2024-09-16T20:50:37.265 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8276999999999996e-05 2024-09-16T20:50:37.265 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:37.265 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:37.265 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:37.265 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.266 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.266 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:2182 3.17 3:edb5e5cd:::rbd_data.110e61479aee.0000000000000014:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write 0~5120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-16T20:50:37.266 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:22.734609+0000", 2024-09-16T20:50:37.266 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 14.437967433000001, 2024-09-16T20:50:37.266 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.124890894, 2024-09-16T20:50:37.266 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:37.266 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:37.266 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:37.266 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:37.266 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:37.266 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:37.267 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 2182 2024-09-16T20:50:37.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.267 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:37.267 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.267 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:37.267 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734609+0000", 2024-09-16T20:50:37.267 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:37.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.267 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.267 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:37.267 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734609+0000", 2024-09-16T20:50:37.267 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:37.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.267 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.267 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:37.268 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734611+0000", 2024-09-16T20:50:37.268 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9599999999999999e-06 2024-09-16T20:50:37.268 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.268 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.268 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:37.268 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734628+0000", 2024-09-16T20:50:37.268 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.715e-05 2024-09-16T20:50:37.268 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.268 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.268 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:37.268 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734656+0000", 2024-09-16T20:50:37.268 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7759999999999998e-05 2024-09-16T20:50:37.268 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.268 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.268 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:37.269 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734669+0000", 2024-09-16T20:50:37.269 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3315999999999999e-05 2024-09-16T20:50:37.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.269 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.269 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:37.269 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734826+0000", 2024-09-16T20:50:37.269 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015668499999999999 2024-09-16T20:50:37.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.269 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.269 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:37.269 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.735048+0000", 2024-09-16T20:50:37.269 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00022179 2024-09-16T20:50:37.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.269 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.269 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:37.270 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.735248+0000", 2024-09-16T20:50:37.270 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00020010399999999999 2024-09-16T20:50:37.270 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.270 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.270 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:50:37.270 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.736252+0000", 2024-09-16T20:50:37.270 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010039739999999999 2024-09-16T20:50:37.270 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.270 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.270 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:37.270 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.859340+0000", 2024-09-16T20:50:37.270 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.12308846599999999 2024-09-16T20:50:37.270 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.270 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.270 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:50:37.270 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.859431+0000", 2024-09-16T20:50:37.271 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1099000000000003e-05 2024-09-16T20:50:37.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:37.271 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:37.271 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:37.271 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.859500+0000", 2024-09-16T20:50:37.271 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8590000000000006e-05 2024-09-16T20:50:37.271 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:37.271 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:37.271 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:37.271 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:37.271 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:37.271 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:37.273 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:37.272+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:37.273 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:50:37.373 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:37.372+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:37.382 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:37.382 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:50:37.382 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:50:37.382 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:37.391 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:50:37.474 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:37.474+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:37.497 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:37.497 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:50:37.497 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:50:37.497 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:50:37.497 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:37.505 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:50:37.575 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:37.574+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:37.614 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:37.614 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:50:37.614 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:50:37.614 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:50:37.614 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.614 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5439 3.a 3:5789f5c3:::rbd_data.110ed9ca540.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 1181696~60416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:37.614 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:50:37.614 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 134.74710556599999, 2024-09-16T20:50:37.615 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10102045599999999, 2024-09-16T20:50:37.615 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:37.615 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:37.615 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:37.615 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:37.615 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:37.615 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:37.615 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5439 2024-09-16T20:50:37.615 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.615 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:37.615 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.615 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:37.615 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:50:37.615 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:37.615 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.616 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.616 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:37.616 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:50:37.616 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:37.616 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.616 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.616 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:37.616 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864711+0000", 2024-09-16T20:50:37.616 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7899999999999997e-07 2024-09-16T20:50:37.616 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.616 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.616 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:37.616 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864760+0000", 2024-09-16T20:50:37.616 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8803e-05 2024-09-16T20:50:37.617 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.617 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.617 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:37.617 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864769+0000", 2024-09-16T20:50:37.617 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0999999999999993e-06 2024-09-16T20:50:37.617 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.617 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.617 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:37.617 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864774+0000", 2024-09-16T20:50:37.617 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8919999999999999e-06 2024-09-16T20:50:37.617 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.617 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.617 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:37.617 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864815+0000", 2024-09-16T20:50:37.618 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1307000000000003e-05 2024-09-16T20:50:37.618 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.618 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.618 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:37.618 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864893+0000", 2024-09-16T20:50:37.618 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7581000000000001e-05 2024-09-16T20:50:37.618 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.618 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.618 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 1", 2024-09-16T20:50:37.618 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864996+0000", 2024-09-16T20:50:37.618 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102725 2024-09-16T20:50:37.618 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.618 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.618 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:37.618 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965442+0000", 2024-09-16T20:50:37.619 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100446012 2024-09-16T20:50:37.619 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.619 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.619 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:37.619 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965618+0000", 2024-09-16T20:50:37.619 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00017653100000000001 2024-09-16T20:50:37.619 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.619 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.619 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:37.619 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965682+0000", 2024-09-16T20:50:37.619 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3905999999999994e-05 2024-09-16T20:50:37.619 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.619 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.619 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:37.620 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965731+0000", 2024-09-16T20:50:37.620 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8819999999999997e-05 2024-09-16T20:50:37.620 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:37.620 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:37.620 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:37.620 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.620 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.620 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4600.0:5439 3.a e37/23 ondisk, result = 0)", 2024-09-16T20:50:37.620 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:50:37.620 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 134.746109418, 2024-09-16T20:50:37.620 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099742634999999996, 2024-09-16T20:50:37.620 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:37.621 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:37.621 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:37.621 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:37.621 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.621 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:37.622 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:50:37.622 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:37.622 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.622 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.622 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:37.622 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:50:37.623 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:37.623 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.623 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.623 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:37.623 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:50:37.623 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0900000000000001e-07 2024-09-16T20:50:37.623 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.623 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.623 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:37.623 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865711+0000", 2024-09-16T20:50:37.623 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3739999999999998e-06 2024-09-16T20:50:37.623 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.624 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.624 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:37.624 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965343+0000", 2024-09-16T20:50:37.624 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099632526999999999 2024-09-16T20:50:37.624 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.624 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.624 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:37.625 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965357+0000", 2024-09-16T20:50:37.625 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3713999999999999e-05 2024-09-16T20:50:37.625 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.625 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.625 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:37.625 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965426+0000", 2024-09-16T20:50:37.625 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8745999999999995e-05 2024-09-16T20:50:37.625 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.625 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.625 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:37.625 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965434+0000", 2024-09-16T20:50:37.625 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6910000000000003e-06 2024-09-16T20:50:37.625 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.626 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.626 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:37.626 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965449+0000", 2024-09-16T20:50:37.626 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4874e-05 2024-09-16T20:50:37.626 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:37.626 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:37.626 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:37.626 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.626 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.626 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4882.0:2508 3.12 3:4992771f:::rbd_object_map.110e7c8851c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T20:50:37.626 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:50:37.626 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 111.13030587, 2024-09-16T20:50:37.626 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.125007011, 2024-09-16T20:50:37.626 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:37.626 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:37.627 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:37.627 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:37.627 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4882", 2024-09-16T20:50:37.627 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2436923223", 2024-09-16T20:50:37.627 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2508 2024-09-16T20:50:37.627 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.627 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:37.627 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.627 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:37.627 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:50:37.627 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:37.627 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.627 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.627 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:37.627 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:50:37.627 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:37.628 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.628 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.628 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:37.628 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481511+0000", 2024-09-16T20:50:37.628 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0699999999999996e-07 2024-09-16T20:50:37.628 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.628 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.628 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:37.628 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481515+0000", 2024-09-16T20:50:37.628 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0609999999999997e-06 2024-09-16T20:50:37.628 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.628 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.628 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:37.628 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481523+0000", 2024-09-16T20:50:37.629 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1310000000000006e-06 2024-09-16T20:50:37.629 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.629 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.629 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:37.629 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481528+0000", 2024-09-16T20:50:37.629 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4900000000000002e-06 2024-09-16T20:50:37.629 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.629 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.629 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:37.629 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481565+0000", 2024-09-16T20:50:37.629 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7066000000000001e-05 2024-09-16T20:50:37.630 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.630 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.630 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:37.630 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481628+0000", 2024-09-16T20:50:37.630 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3118000000000001e-05 2024-09-16T20:50:37.630 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.630 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.630 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 2", 2024-09-16T20:50:37.630 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481822+0000", 2024-09-16T20:50:37.630 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019446200000000001 2024-09-16T20:50:37.630 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.630 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.630 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:37.630 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605988+0000", 2024-09-16T20:50:37.630 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.124165888 2024-09-16T20:50:37.631 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.631 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.631 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:37.631 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606427+0000", 2024-09-16T20:50:37.631 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00043854899999999999 2024-09-16T20:50:37.631 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.631 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.631 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:37.631 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606476+0000", 2024-09-16T20:50:37.631 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8971999999999998e-05 2024-09-16T20:50:37.631 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.631 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.631 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:37.631 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606517+0000", 2024-09-16T20:50:37.631 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1467000000000001e-05 2024-09-16T20:50:37.632 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:37.632 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:37.632 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:37.632 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.632 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.632 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4882.0:2508 3.12 e57/55 ondisk, result = 0)", 2024-09-16T20:50:37.632 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:50:37.632 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 111.12931737, 2024-09-16T20:50:37.632 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12349711400000001, 2024-09-16T20:50:37.632 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:37.632 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:37.633 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:37.633 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:37.633 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.633 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:37.633 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:50:37.633 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:37.633 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.633 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.633 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:37.633 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:50:37.633 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:37.633 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.633 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.633 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:37.633 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:50:37.633 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.06e-07 2024-09-16T20:50:37.634 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.634 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.634 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:37.634 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482502+0000", 2024-09-16T20:50:37.634 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.01e-06 2024-09-16T20:50:37.634 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.634 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.634 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:37.634 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605772+0000", 2024-09-16T20:50:37.634 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.123269687 2024-09-16T20:50:37.634 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.634 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.634 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:37.634 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605791+0000", 2024-09-16T20:50:37.634 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8912000000000001e-05 2024-09-16T20:50:37.635 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.635 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.635 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:37.635 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605974+0000", 2024-09-16T20:50:37.635 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000182836 2024-09-16T20:50:37.635 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.635 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.635 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:37.635 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605982+0000", 2024-09-16T20:50:37.635 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1699999999999997e-06 2024-09-16T20:50:37.635 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.635 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.635 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:37.635 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605996+0000", 2024-09-16T20:50:37.635 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3893000000000001e-05 2024-09-16T20:50:37.636 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:37.636 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:37.636 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:37.636 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.636 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.636 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4882.0:2512 3.f 3:f4d895b7:::rbd_data.110e7c8851c.0000000000000000:head [read 16384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T20:50:37.636 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:50:37.636 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 111.12888033999999, 2024-09-16T20:50:37.636 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12318483, 2024-09-16T20:50:37.636 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:37.636 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:37.636 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:37.637 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:37.637 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4882", 2024-09-16T20:50:37.637 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2436923223", 2024-09-16T20:50:37.637 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2512 2024-09-16T20:50:37.637 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.637 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:37.637 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.637 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:37.637 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:50:37.637 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:37.637 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.637 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.637 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:37.637 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:50:37.637 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:37.638 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.638 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.638 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:37.638 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:50:37.638 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3499999999999995e-07 2024-09-16T20:50:37.638 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.638 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.638 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:37.638 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482943+0000", 2024-09-16T20:50:37.638 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3450000000000002e-06 2024-09-16T20:50:37.638 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.638 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.638 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:37.639 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605772+0000", 2024-09-16T20:50:37.639 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.122829796 2024-09-16T20:50:37.639 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.639 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.639 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:37.639 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605792+0000", 2024-09-16T20:50:37.639 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9916999999999999e-05 2024-09-16T20:50:37.639 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.639 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.639 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:37.639 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605917+0000", 2024-09-16T20:50:37.639 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00012500799999999999 2024-09-16T20:50:37.639 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.640 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.640 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:37.640 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606004+0000", 2024-09-16T20:50:37.640 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6242999999999994e-05 2024-09-16T20:50:37.640 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.640 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.640 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:37.640 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606120+0000", 2024-09-16T20:50:37.640 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000116786 2024-09-16T20:50:37.640 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:37.640 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:37.640 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:37.640 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.641 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.641 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:11702 3.15 3:acf33c8b:::rbd_data.110e747f0779.000000000000001a:head [set-alloc-hint object_size 262144 write_size 262144,write 143360~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:50:37.641 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:50:37.641 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 93.473887918000003, 2024-09-16T20:50:37.641 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.084801043000000006, 2024-09-16T20:50:37.641 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:37.641 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:37.641 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:37.641 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:37.641 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:50:37.641 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:50:37.641 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 11702 2024-09-16T20:50:37.641 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.641 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:37.641 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.641 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:37.642 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:50:37.642 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:37.642 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.642 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.642 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:37.642 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:50:37.642 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:37.642 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.642 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.642 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:37.642 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137929+0000", 2024-09-16T20:50:37.642 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8400000000000002e-07 2024-09-16T20:50:37.642 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.642 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.642 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:37.643 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137937+0000", 2024-09-16T20:50:37.643 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0260000000000007e-06 2024-09-16T20:50:37.643 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.643 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.643 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:37.643 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137952+0000", 2024-09-16T20:50:37.643 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4659e-05 2024-09-16T20:50:37.643 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.643 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.643 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:37.643 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137960+0000", 2024-09-16T20:50:37.643 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0169999999999996e-06 2024-09-16T20:50:37.644 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.644 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.644 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:37.644 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138010+0000", 2024-09-16T20:50:37.644 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0021000000000001e-05 2024-09-16T20:50:37.644 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.644 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.644 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:37.644 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138114+0000", 2024-09-16T20:50:37.644 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000103797 2024-09-16T20:50:37.644 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.644 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.644 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:50:37.644 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138233+0000", 2024-09-16T20:50:37.645 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000118971 2024-09-16T20:50:37.645 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.645 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.645 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:37.645 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138953+0000", 2024-09-16T20:50:37.645 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00072026000000000002 2024-09-16T20:50:37.645 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.645 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.645 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:37.645 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222609+0000", 2024-09-16T20:50:37.645 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.083656650999999999 2024-09-16T20:50:37.645 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.645 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.645 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:37.645 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222684+0000", 2024-09-16T20:50:37.646 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4369999999999994e-05 2024-09-16T20:50:37.646 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.646 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.646 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:37.646 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222729+0000", 2024-09-16T20:50:37.646 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5287000000000002e-05 2024-09-16T20:50:37.646 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:37.646 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:37.646 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:37.646 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.646 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.646 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:11704 3.9 3:91220b75:::rbd_data.110e747f0779.000000000000001c:head [set-alloc-hint object_size 262144 write_size 262144,write 139264~6656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:50:37.646 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:50:37.646 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 93.473649917000003, 2024-09-16T20:50:37.646 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.084531133999999994, 2024-09-16T20:50:37.647 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:37.647 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:37.647 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:37.647 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:37.647 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:50:37.647 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:50:37.647 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 11704 2024-09-16T20:50:37.647 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.647 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:37.647 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.647 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:37.647 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:50:37.647 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:37.647 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.647 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.648 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:37.648 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:50:37.648 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:37.648 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.648 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.648 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:37.648 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138167+0000", 2024-09-16T20:50:37.648 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.33e-07 2024-09-16T20:50:37.648 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.648 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.648 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:37.649 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138179+0000", 2024-09-16T20:50:37.649 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2055999999999999e-05 2024-09-16T20:50:37.649 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.649 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.649 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:37.649 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138193+0000", 2024-09-16T20:50:37.649 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3961000000000001e-05 2024-09-16T20:50:37.649 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.649 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.649 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:37.649 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138200+0000", 2024-09-16T20:50:37.649 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4429999999999997e-06 2024-09-16T20:50:37.649 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.649 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.649 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:37.650 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138258+0000", 2024-09-16T20:50:37.650 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7225999999999997e-05 2024-09-16T20:50:37.650 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.650 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.650 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:37.650 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138364+0000", 2024-09-16T20:50:37.650 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000106013 2024-09-16T20:50:37.650 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.650 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.650 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:50:37.650 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138494+0000", 2024-09-16T20:50:37.650 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013059599999999999 2024-09-16T20:50:37.650 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.650 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.651 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:37.651 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.139220+0000", 2024-09-16T20:50:37.651 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00072539800000000004 2024-09-16T20:50:37.651 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.651 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.651 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:37.651 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222608+0000", 2024-09-16T20:50:37.651 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.083388620999999996 2024-09-16T20:50:37.651 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.651 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.651 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:37.651 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222651+0000", 2024-09-16T20:50:37.652 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2228000000000003e-05 2024-09-16T20:50:37.652 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.652 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.652 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:37.652 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222697+0000", 2024-09-16T20:50:37.652 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6659000000000002e-05 2024-09-16T20:50:37.652 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:37.652 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:37.652 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:37.652 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.652 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.652 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5138.0:9358 3.c 3.c25699cc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T20:50:37.652 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:50:37.652 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 70.761502612000001, 2024-09-16T20:50:37.652 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0018657580000001, 2024-09-16T20:50:37.653 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:37.653 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:37.653 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:50:37.653 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:37.653 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5138", 2024-09-16T20:50:37.653 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:50:37.653 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 9358 2024-09-16T20:50:37.653 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.653 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:37.653 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.653 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:37.653 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:50:37.653 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:37.653 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.653 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.653 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:37.654 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:50:37.654 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:37.654 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.654 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.654 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:37.654 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850314+0000", 2024-09-16T20:50:37.654 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4099999999999999e-07 2024-09-16T20:50:37.654 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.654 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.654 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:37.654 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850316+0000", 2024-09-16T20:50:37.654 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2469999999999998e-06 2024-09-16T20:50:37.654 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.654 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.654 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:37.655 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850322+0000", 2024-09-16T20:50:37.655 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3470000000000004e-06 2024-09-16T20:50:37.655 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.655 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.655 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:37.655 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850327+0000", 2024-09-16T20:50:37.655 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9450000000000001e-06 2024-09-16T20:50:37.655 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.655 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.655 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:37.655 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850376+0000", 2024-09-16T20:50:37.655 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8257000000000002e-05 2024-09-16T20:50:37.655 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.655 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.655 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:37.656 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850385+0000", 2024-09-16T20:50:37.656 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9800000000000004e-06 2024-09-16T20:50:37.656 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.656 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.656 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:37.656 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852167+0000", 2024-09-16T20:50:37.656 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0017825659999999 2024-09-16T20:50:37.656 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.656 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.656 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:37.656 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852179+0000", 2024-09-16T20:50:37.656 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1975e-05 2024-09-16T20:50:37.656 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:37.656 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:37.656 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:37.657 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.657 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.657 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5138.0:9382 3.c 3.c25699cc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T20:50:37.657 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:50:37.657 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 70.753155595999999, 2024-09-16T20:50:37.657 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.99358404700000003, 2024-09-16T20:50:37.657 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:37.657 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:37.657 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:50:37.657 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:37.657 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5138", 2024-09-16T20:50:37.657 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:50:37.657 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 9382 2024-09-16T20:50:37.657 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.657 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:37.658 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.658 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:37.658 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:50:37.658 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:37.658 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.658 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.658 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:37.658 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:50:37.658 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:37.658 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.658 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.658 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:37.658 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858661+0000", 2024-09-16T20:50:37.658 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0200000000000002e-07 2024-09-16T20:50:37.658 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.659 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.659 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:37.659 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858664+0000", 2024-09-16T20:50:37.659 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.038e-06 2024-09-16T20:50:37.659 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.659 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.659 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:37.659 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858671+0000", 2024-09-16T20:50:37.659 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0110000000000003e-06 2024-09-16T20:50:37.659 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.659 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.659 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:37.659 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858677+0000", 2024-09-16T20:50:37.659 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0959999999999997e-06 2024-09-16T20:50:37.659 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.660 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.660 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:37.660 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858732+0000", 2024-09-16T20:50:37.660 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4543000000000003e-05 2024-09-16T20:50:37.660 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.660 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.660 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:37.660 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858739+0000", 2024-09-16T20:50:37.660 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5429999999999996e-06 2024-09-16T20:50:37.660 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.660 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.660 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:37.660 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852236+0000", 2024-09-16T20:50:37.660 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.993496453 2024-09-16T20:50:37.660 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.661 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.661 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:37.661 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852244+0000", 2024-09-16T20:50:37.661 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8610000000000004e-06 2024-09-16T20:50:37.661 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:37.661 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:37.661 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:37.661 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.661 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.661 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5763.0:4895 3.a 3:5384606c:::rbd_data.110e1b19a241.0000000000000018:head [read 0~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-16T20:50:37.661 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:45.598212+0000", 2024-09-16T20:50:37.661 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 52.013603555000003, 2024-09-16T20:50:37.661 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0375864829999999, 2024-09-16T20:50:37.661 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:37.661 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:37.662 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:37.662 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:37.662 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5763", 2024-09-16T20:50:37.662 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/623897567", 2024-09-16T20:50:37.662 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4895 2024-09-16T20:50:37.662 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.662 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:37.662 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.662 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:37.662 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598212+0000", 2024-09-16T20:50:37.662 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:37.662 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.662 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.662 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:37.663 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598212+0000", 2024-09-16T20:50:37.663 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:37.663 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.663 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.663 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:37.663 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598214+0000", 2024-09-16T20:50:37.663 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0090000000000002e-06 2024-09-16T20:50:37.663 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.663 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.663 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:37.663 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598222+0000", 2024-09-16T20:50:37.663 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6699999999999994e-06 2024-09-16T20:50:37.663 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.663 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.663 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:37.663 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598241+0000", 2024-09-16T20:50:37.664 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8896999999999999e-05 2024-09-16T20:50:37.664 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.664 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.664 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:37.664 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598250+0000", 2024-09-16T20:50:37.664 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6649999999999992e-06 2024-09-16T20:50:37.664 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.664 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.664 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:37.664 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598325+0000", 2024-09-16T20:50:37.664 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5189000000000001e-05 2024-09-16T20:50:37.664 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.664 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.664 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op must wait for map", 2024-09-16T20:50:37.665 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598333+0000", 2024-09-16T20:50:37.665 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.3170000000000002e-06 2024-09-16T20:50:37.665 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.665 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.665 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:37.665 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601355+0000", 2024-09-16T20:50:37.665 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0030215400000000001 2024-09-16T20:50:37.665 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.665 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.665 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:37.665 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601365+0000", 2024-09-16T20:50:37.665 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0215e-05 2024-09-16T20:50:37.665 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.665 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.666 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:37.666 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635506+0000", 2024-09-16T20:50:37.666 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0341408240000001 2024-09-16T20:50:37.666 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.666 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.666 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:37.666 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635655+0000", 2024-09-16T20:50:37.666 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000149412 2024-09-16T20:50:37.666 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.666 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.666 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:37.666 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635799+0000", 2024-09-16T20:50:37.666 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000143745 2024-09-16T20:50:37.666 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:37.666 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:37.667 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:37.667 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.667 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.667 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5763.0:4900 3.a 3:5384606c:::rbd_data.110e1b19a241.0000000000000018:head [read 8192~24576] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-16T20:50:37.667 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:50:37.667 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 52.013545078999996, 2024-09-16T20:50:37.667 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0379340800000001, 2024-09-16T20:50:37.667 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:37.667 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:37.667 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:37.667 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:37.667 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5763", 2024-09-16T20:50:37.667 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/623897567", 2024-09-16T20:50:37.667 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4900 2024-09-16T20:50:37.668 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.668 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:37.668 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.668 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:37.668 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:50:37.668 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:37.668 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.668 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.668 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:37.668 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:50:37.668 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:37.668 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.668 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.669 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:37.669 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:50:37.669 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4300000000000003e-07 2024-09-16T20:50:37.669 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.669 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.669 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:37.669 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598274+0000", 2024-09-16T20:50:37.669 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.976e-06 2024-09-16T20:50:37.669 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.669 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.669 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:37.669 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598282+0000", 2024-09-16T20:50:37.670 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0029999999999994e-06 2024-09-16T20:50:37.670 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.670 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.670 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:37.670 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598289+0000", 2024-09-16T20:50:37.670 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2389999999999999e-06 2024-09-16T20:50:37.670 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.670 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.670 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:37.670 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598430+0000", 2024-09-16T20:50:37.670 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014119600000000001 2024-09-16T20:50:37.670 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.670 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.670 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting_for_map not empty", 2024-09-16T20:50:37.670 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598435+0000", 2024-09-16T20:50:37.671 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3489999999999999e-06 2024-09-16T20:50:37.671 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.671 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.671 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:37.671 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601429+0000", 2024-09-16T20:50:37.671 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0029942720000000001 2024-09-16T20:50:37.671 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.671 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.671 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:37.671 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601438+0000", 2024-09-16T20:50:37.671 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4759999999999998e-06 2024-09-16T20:50:37.671 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.671 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.671 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:37.672 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635987+0000", 2024-09-16T20:50:37.672 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0345495010000001 2024-09-16T20:50:37.672 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.672 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.672 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:37.672 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.636056+0000", 2024-09-16T20:50:37.672 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8610999999999998e-05 2024-09-16T20:50:37.672 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.672 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.672 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:37.672 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.636205+0000", 2024-09-16T20:50:37.672 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014891399999999999 2024-09-16T20:50:37.672 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:37.672 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:37.673 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:37.673 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.673 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.673 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop(client.6231.0:2182 3.17 e136/81 3:edb5e5cd:::rbd_data.110e61479aee.0000000000000014:head v 136'791, mlcod=135'789)", 2024-09-16T20:50:37.673 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:22.735499+0000", 2024-09-16T20:50:37.673 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 14.876317137999999, 2024-09-16T20:50:37.673 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12369608, 2024-09-16T20:50:37.673 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:37.673 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:37.673 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:37.673 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:37.673 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.673 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:37.673 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735499+0000", 2024-09-16T20:50:37.673 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:37.674 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.674 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.674 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:37.674 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735499+0000", 2024-09-16T20:50:37.674 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:37.674 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.674 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.674 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:37.674 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735499+0000", 2024-09-16T20:50:37.674 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8099999999999999e-07 2024-09-16T20:50:37.674 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.674 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.674 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:37.674 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735506+0000", 2024-09-16T20:50:37.674 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3579999999999997e-06 2024-09-16T20:50:37.674 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.675 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.675 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:37.675 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735517+0000", 2024-09-16T20:50:37.675 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0869999999999999e-05 2024-09-16T20:50:37.675 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.675 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.675 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:37.675 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735533+0000", 2024-09-16T20:50:37.675 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5858e-05 2024-09-16T20:50:37.675 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.675 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.675 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:37.675 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735584+0000", 2024-09-16T20:50:37.675 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0470999999999998e-05 2024-09-16T20:50:37.676 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.676 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.676 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:37.676 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735607+0000", 2024-09-16T20:50:37.676 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2875999999999999e-05 2024-09-16T20:50:37.676 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.676 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.676 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:37.676 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.859144+0000", 2024-09-16T20:50:37.676 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.123537701 2024-09-16T20:50:37.676 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.676 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.676 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:37.677 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.859195+0000", 2024-09-16T20:50:37.677 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0664999999999997e-05 2024-09-16T20:50:37.677 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:37.677 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:37.677 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:37.677 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.677 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.677 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5204 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 2523136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:37.677 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.883660+0000", 2024-09-16T20:50:37.677 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 13.728155608, 2024-09-16T20:50:37.677 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052517638999999998, 2024-09-16T20:50:37.677 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:37.677 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:37.677 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:37.677 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:37.678 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:50:37.678 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:37.678 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5204 2024-09-16T20:50:37.678 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.678 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:37.678 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.678 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:37.678 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883660+0000", 2024-09-16T20:50:37.678 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:37.678 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.678 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.678 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:37.678 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883660+0000", 2024-09-16T20:50:37.678 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:37.678 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.679 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.679 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:37.679 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883661+0000", 2024-09-16T20:50:37.679 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9999999999999998e-07 2024-09-16T20:50:37.679 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.679 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.680 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:37.680 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883663+0000", 2024-09-16T20:50:37.680 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.4480000000000001e-06 2024-09-16T20:50:37.680 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.680 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.680 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:37.680 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883669+0000", 2024-09-16T20:50:37.680 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.071e-06 2024-09-16T20:50:37.680 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.680 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.681 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:37.681 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883675+0000", 2024-09-16T20:50:37.681 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.366e-06 2024-09-16T20:50:37.681 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.681 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.681 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:37.681 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.935970+0000", 2024-09-16T20:50:37.681 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052295346999999999 2024-09-16T20:50:37.681 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.681 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.681 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:37.681 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936064+0000", 2024-09-16T20:50:37.681 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4438000000000006e-05 2024-09-16T20:50:37.681 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.682 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.682 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:37.682 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936178+0000", 2024-09-16T20:50:37.682 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000113569 2024-09-16T20:50:37.682 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:37.682 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:37.682 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:37.682 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.682 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.683 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5205 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 2543616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:37.683 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.883745+0000", 2024-09-16T20:50:37.683 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 13.72807085, 2024-09-16T20:50:37.683 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052692112999999999, 2024-09-16T20:50:37.683 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:37.683 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:37.683 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:37.683 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:37.683 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:50:37.683 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:37.683 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5205 2024-09-16T20:50:37.683 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.683 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:37.683 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.683 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:37.683 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883745+0000", 2024-09-16T20:50:37.684 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:37.684 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.684 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.684 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:37.684 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883745+0000", 2024-09-16T20:50:37.684 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:37.684 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.684 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.684 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:37.684 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883746+0000", 2024-09-16T20:50:37.684 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2699999999999999e-07 2024-09-16T20:50:37.684 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.684 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.684 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:37.684 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883750+0000", 2024-09-16T20:50:37.685 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7050000000000001e-06 2024-09-16T20:50:37.685 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.685 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.685 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:37.685 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883769+0000", 2024-09-16T20:50:37.685 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9165e-05 2024-09-16T20:50:37.685 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.685 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.685 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:37.685 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883778+0000", 2024-09-16T20:50:37.686 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.6690000000000001e-06 2024-09-16T20:50:37.686 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.686 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.686 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:37.686 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936228+0000", 2024-09-16T20:50:37.686 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052449587999999998 2024-09-16T20:50:37.686 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.686 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.686 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:37.686 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936333+0000", 2024-09-16T20:50:37.686 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010511800000000001 2024-09-16T20:50:37.686 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.686 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.687 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:37.687 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936437+0000", 2024-09-16T20:50:37.687 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000104141 2024-09-16T20:50:37.687 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:37.687 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:37.687 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:37.687 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.687 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.687 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5206 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 1167360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:37.687 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.883802+0000", 2024-09-16T20:50:37.687 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 13.728014355999999, 2024-09-16T20:50:37.687 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052800780999999998, 2024-09-16T20:50:37.687 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:37.687 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:37.688 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:37.688 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:37.688 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:50:37.688 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:37.688 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5206 2024-09-16T20:50:37.688 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.688 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:37.688 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.688 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:37.689 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883802+0000", 2024-09-16T20:50:37.689 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:37.689 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.689 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.689 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:37.689 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883802+0000", 2024-09-16T20:50:37.689 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:37.689 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.689 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.689 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:37.689 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883933+0000", 2024-09-16T20:50:37.689 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013182999999999999 2024-09-16T20:50:37.690 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.690 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.690 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:37.690 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883937+0000", 2024-09-16T20:50:37.690 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.771e-06 2024-09-16T20:50:37.690 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.690 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.690 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:37.690 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883953+0000", 2024-09-16T20:50:37.690 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5409000000000001e-05 2024-09-16T20:50:37.690 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.690 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.690 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:37.691 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883959+0000", 2024-09-16T20:50:37.691 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0700000000000003e-06 2024-09-16T20:50:37.691 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.691 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.691 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:37.691 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936476+0000", 2024-09-16T20:50:37.691 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052517434000000002 2024-09-16T20:50:37.691 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.691 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.691 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:37.691 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936529+0000", 2024-09-16T20:50:37.691 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2969000000000001e-05 2024-09-16T20:50:37.691 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.691 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.691 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:37.691 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936602+0000", 2024-09-16T20:50:37.692 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3298000000000002e-05 2024-09-16T20:50:37.692 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:37.692 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:37.692 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:37.692 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.692 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.692 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5207 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 1904640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:37.692 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.883980+0000", 2024-09-16T20:50:37.692 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 13.727836147, 2024-09-16T20:50:37.692 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052759922000000001, 2024-09-16T20:50:37.692 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:37.692 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:37.692 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:37.692 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:37.693 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:50:37.693 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:37.693 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5207 2024-09-16T20:50:37.693 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.693 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:37.693 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.693 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:37.693 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883980+0000", 2024-09-16T20:50:37.693 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:37.693 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.693 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.693 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:37.693 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883980+0000", 2024-09-16T20:50:37.693 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:37.693 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.693 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.694 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:37.694 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884126+0000", 2024-09-16T20:50:37.694 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014577399999999999 2024-09-16T20:50:37.694 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.694 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.694 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:37.694 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884130+0000", 2024-09-16T20:50:37.694 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9729999999999999e-06 2024-09-16T20:50:37.694 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.694 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.694 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:37.694 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884139+0000", 2024-09-16T20:50:37.694 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8660000000000001e-06 2024-09-16T20:50:37.695 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.695 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.695 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:37.695 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884144+0000", 2024-09-16T20:50:37.695 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3669999999999999e-06 2024-09-16T20:50:37.695 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.695 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.696 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:37.696 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936635+0000", 2024-09-16T20:50:37.696 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052491705 2024-09-16T20:50:37.696 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.696 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.696 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:37.697 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936679+0000", 2024-09-16T20:50:37.697 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3105999999999997e-05 2024-09-16T20:50:37.697 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.697 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.697 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:37.697 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936740+0000", 2024-09-16T20:50:37.698 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1130999999999995e-05 2024-09-16T20:50:37.698 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:37.698 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:37.698 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:37.698 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.698 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.698 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5208 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 131072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:37.698 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.884165+0000", 2024-09-16T20:50:37.699 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 13.727650851, 2024-09-16T20:50:37.699 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052746603000000003, 2024-09-16T20:50:37.699 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:37.699 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:37.699 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:37.699 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:37.699 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:50:37.699 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:37.699 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5208 2024-09-16T20:50:37.699 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.699 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:37.699 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.699 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:37.699 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884165+0000", 2024-09-16T20:50:37.699 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:37.700 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.700 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.700 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:37.700 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884165+0000", 2024-09-16T20:50:37.700 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:37.700 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.700 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.700 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:37.700 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884463+0000", 2024-09-16T20:50:37.700 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00029801699999999998 2024-09-16T20:50:37.700 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.700 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.700 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:37.700 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884467+0000", 2024-09-16T20:50:37.700 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8210000000000003e-06 2024-09-16T20:50:37.701 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.701 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.701 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:37.701 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884477+0000", 2024-09-16T20:50:37.701 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.7030000000000001e-06 2024-09-16T20:50:37.701 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.701 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.701 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:37.701 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884485+0000", 2024-09-16T20:50:37.701 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9710000000000004e-06 2024-09-16T20:50:37.701 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.701 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.701 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:37.701 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936785+0000", 2024-09-16T20:50:37.701 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052300665000000003 2024-09-16T20:50:37.702 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.702 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.702 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:37.702 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936851+0000", 2024-09-16T20:50:37.702 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6159000000000001e-05 2024-09-16T20:50:37.702 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.702 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.702 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:37.702 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936912+0000", 2024-09-16T20:50:37.702 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0266999999999997e-05 2024-09-16T20:50:37.702 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:37.702 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:37.702 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:37.702 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.702 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.702 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5209 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 1925120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:37.703 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.884504+0000", 2024-09-16T20:50:37.703 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 13.727311966, 2024-09-16T20:50:37.703 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052575445999999998, 2024-09-16T20:50:37.703 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:37.703 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:37.703 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:37.703 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:37.703 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:50:37.703 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:37.703 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5209 2024-09-16T20:50:37.703 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.703 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:37.703 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.703 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:37.703 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884504+0000", 2024-09-16T20:50:37.704 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:37.704 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.704 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.704 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:37.704 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884504+0000", 2024-09-16T20:50:37.704 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:37.704 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.704 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.704 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:37.704 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884741+0000", 2024-09-16T20:50:37.704 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00023679899999999999 2024-09-16T20:50:37.704 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.704 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.705 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:37.705 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884747+0000", 2024-09-16T20:50:37.705 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0009999999999998e-06 2024-09-16T20:50:37.705 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.705 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.705 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:37.705 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884764+0000", 2024-09-16T20:50:37.705 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7371999999999999e-05 2024-09-16T20:50:37.705 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.705 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.705 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:37.705 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884772+0000", 2024-09-16T20:50:37.705 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1349999999999992e-06 2024-09-16T20:50:37.706 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.706 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.706 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:37.706 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936956+0000", 2024-09-16T20:50:37.706 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052183754999999998 2024-09-16T20:50:37.706 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.706 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.706 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:37.706 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937003+0000", 2024-09-16T20:50:37.707 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6925000000000002e-05 2024-09-16T20:50:37.707 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.707 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.707 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:37.707 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937079+0000", 2024-09-16T20:50:37.707 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6458999999999994e-05 2024-09-16T20:50:37.707 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:37.707 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:37.707 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:37.708 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.708 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.708 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5210 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 151552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:37.708 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.884799+0000", 2024-09-16T20:50:37.708 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 13.727016697, 2024-09-16T20:50:37.708 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052462483999999997, 2024-09-16T20:50:37.708 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:37.708 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:37.708 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:37.708 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:37.708 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:50:37.708 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:37.708 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5210 2024-09-16T20:50:37.708 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.709 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:37.709 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.709 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:37.709 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884799+0000", 2024-09-16T20:50:37.709 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:37.709 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.709 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.709 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:37.709 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884799+0000", 2024-09-16T20:50:37.709 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:37.709 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.709 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.709 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:37.709 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884990+0000", 2024-09-16T20:50:37.709 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019051800000000001 2024-09-16T20:50:37.710 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.710 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.710 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:37.710 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884996+0000", 2024-09-16T20:50:37.710 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0689999999999998e-06 2024-09-16T20:50:37.710 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.710 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.710 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:37.710 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.885008+0000", 2024-09-16T20:50:37.710 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2153999999999999e-05 2024-09-16T20:50:37.710 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.710 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.710 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:37.710 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.885016+0000", 2024-09-16T20:50:37.710 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2770000000000005e-06 2024-09-16T20:50:37.711 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.711 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.711 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:37.711 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937128+0000", 2024-09-16T20:50:37.711 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052111377 2024-09-16T20:50:37.711 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.711 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.711 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:37.711 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937188+0000", 2024-09-16T20:50:37.711 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9933999999999999e-05 2024-09-16T20:50:37.711 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.711 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.711 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:37.711 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937262+0000", 2024-09-16T20:50:37.711 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4154999999999997e-05 2024-09-16T20:50:37.711 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:37.712 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:37.712 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:37.712 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.712 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.712 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5211 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 1945600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:37.712 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.885030+0000", 2024-09-16T20:50:37.712 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 13.726785612, 2024-09-16T20:50:37.712 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052369470000000001, 2024-09-16T20:50:37.712 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:37.712 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:37.712 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:37.712 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:37.712 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:50:37.712 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:37.713 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5211 2024-09-16T20:50:37.713 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.713 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:37.713 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.713 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:37.713 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.885030+0000", 2024-09-16T20:50:37.713 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:37.713 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.713 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.713 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:37.713 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.885030+0000", 2024-09-16T20:50:37.714 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:37.714 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.714 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.714 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:37.714 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.885219+0000", 2024-09-16T20:50:37.714 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000188278 2024-09-16T20:50:37.714 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.714 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.714 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:37.714 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.885223+0000", 2024-09-16T20:50:37.715 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7849999999999999e-06 2024-09-16T20:50:37.715 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.715 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.715 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:37.715 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.885233+0000", 2024-09-16T20:50:37.715 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.5060000000000001e-06 2024-09-16T20:50:37.715 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.715 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.715 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:37.715 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.885240+0000", 2024-09-16T20:50:37.715 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9630000000000001e-06 2024-09-16T20:50:37.715 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.715 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.715 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:37.716 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937304+0000", 2024-09-16T20:50:37.716 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052064619999999999 2024-09-16T20:50:37.716 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.716 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.716 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:37.716 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937347+0000", 2024-09-16T20:50:37.716 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2407000000000002e-05 2024-09-16T20:50:37.716 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:37.716 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:37.716 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:37.716 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937400+0000", 2024-09-16T20:50:37.716 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2911000000000002e-05 2024-09-16T20:50:37.717 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:37.717 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:37.717 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:37.717 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:37.717 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:37.717 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:37.718 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:50:37.719 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:37.717+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:37.786 INFO:teuthology.orchestra.run.smithi029.stderr:noscrub is unset 2024-09-16T20:50:37.819 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:37.819+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:37.872 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:37.872 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:50:37.872 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:50:37.872 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:37.881 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:50:37.920 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:37.919+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:37.987 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:37.987 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:50:37.987 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:50:37.987 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:50:37.988 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:37.996 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:50:38.039 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:38.038+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:38.104 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:38.104 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:50:38.104 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:50:38.104 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:50:38.104 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.105 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:50:38.105 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:50:38.105 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 164.84824154899999, 2024-09-16T20:50:38.105 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67502346999999996, 2024-09-16T20:50:38.105 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:38.105 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:38.105 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:38.105 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:38.105 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:50:38.105 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:50:38.105 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 1 2024-09-16T20:50:38.105 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.105 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:38.105 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.106 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:38.106 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:50:38.106 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:38.106 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.106 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.106 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:38.106 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:50:38.106 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:38.106 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.106 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.106 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:38.106 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254128+0000", 2024-09-16T20:50:38.106 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.463e-06 2024-09-16T20:50:38.106 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.106 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.107 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:38.107 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254135+0000", 2024-09-16T20:50:38.107 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1579999999999999e-06 2024-09-16T20:50:38.107 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.107 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.107 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:38.107 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254146+0000", 2024-09-16T20:50:38.107 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1341e-05 2024-09-16T20:50:38.107 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.107 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.107 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:38.107 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254155+0000", 2024-09-16T20:50:38.107 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.1549999999999996e-06 2024-09-16T20:50:38.107 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.108 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.108 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:38.108 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254221+0000", 2024-09-16T20:50:38.108 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5470999999999997e-05 2024-09-16T20:50:38.108 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.108 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.108 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:38.108 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254231+0000", 2024-09-16T20:50:38.108 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0592e-05 2024-09-16T20:50:38.108 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.108 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.108 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:38.108 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926711+0000", 2024-09-16T20:50:38.108 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67247946599999997 2024-09-16T20:50:38.109 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.109 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.109 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:38.109 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926910+0000", 2024-09-16T20:50:38.109 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019934 2024-09-16T20:50:38.109 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.109 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.109 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:50:38.109 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.927159+0000", 2024-09-16T20:50:38.109 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00024876599999999998 2024-09-16T20:50:38.109 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.109 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.109 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:38.109 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.928797+0000", 2024-09-16T20:50:38.110 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001638308 2024-09-16T20:50:38.110 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.110 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.110 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:38.110 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929026+0000", 2024-09-16T20:50:38.110 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00022876099999999999 2024-09-16T20:50:38.110 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.110 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.110 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:38.110 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929102+0000", 2024-09-16T20:50:38.110 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6184000000000001e-05 2024-09-16T20:50:38.110 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.110 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.111 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:38.111 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929150+0000", 2024-09-16T20:50:38.111 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7465e-05 2024-09-16T20:50:38.111 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:38.111 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:38.111 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:38.111 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.111 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.111 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3574 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:38.111 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:50:38.111 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 108.621078017, 2024-09-16T20:50:38.111 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053579441999999998, 2024-09-16T20:50:38.111 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:38.112 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:38.112 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:38.112 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:38.112 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:38.112 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:38.112 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3574 2024-09-16T20:50:38.112 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.112 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:38.112 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.112 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:38.112 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:50:38.112 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:38.112 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.112 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.113 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:38.113 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:50:38.113 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:38.113 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.113 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.113 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:38.113 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:50:38.113 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2e-07 2024-09-16T20:50:38.113 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.113 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.113 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:38.113 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481292+0000", 2024-09-16T20:50:38.113 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2630000000000002e-06 2024-09-16T20:50:38.113 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.114 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.114 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:38.114 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481297+0000", 2024-09-16T20:50:38.114 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.296e-06 2024-09-16T20:50:38.114 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.114 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.114 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:38.114 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481300+0000", 2024-09-16T20:50:38.114 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7019999999999999e-06 2024-09-16T20:50:38.114 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.114 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.114 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:38.114 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525533+0000", 2024-09-16T20:50:38.114 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.044232318 2024-09-16T20:50:38.115 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.115 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.115 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:38.115 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525619+0000", 2024-09-16T20:50:38.115 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5892999999999999e-05 2024-09-16T20:50:38.115 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.115 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.115 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:38.115 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525987+0000", 2024-09-16T20:50:38.115 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00036836099999999999 2024-09-16T20:50:38.115 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.115 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.115 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:38.115 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531779+0000", 2024-09-16T20:50:38.116 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0057918240000000001 2024-09-16T20:50:38.116 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.116 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.116 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:38.116 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534747+0000", 2024-09-16T20:50:38.116 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0029681600000000001 2024-09-16T20:50:38.116 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.116 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.116 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:38.116 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534832+0000", 2024-09-16T20:50:38.116 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5180999999999996e-05 2024-09-16T20:50:38.116 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.116 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.116 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:38.116 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534869+0000", 2024-09-16T20:50:38.117 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7024000000000003e-05 2024-09-16T20:50:38.117 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:38.117 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:38.117 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:38.117 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.117 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.117 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3575 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:38.117 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:50:38.117 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 108.621063277, 2024-09-16T20:50:38.117 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053801463000000001, 2024-09-16T20:50:38.117 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:38.117 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:38.117 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:38.118 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:38.118 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:38.118 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:38.118 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3575 2024-09-16T20:50:38.118 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.118 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:38.118 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.118 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:38.118 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:50:38.118 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:38.118 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.118 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.118 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:38.118 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:50:38.119 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:38.119 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.119 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.119 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:38.119 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481305+0000", 2024-09-16T20:50:38.119 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5400000000000002e-07 2024-09-16T20:50:38.119 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.119 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.119 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:38.119 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481307+0000", 2024-09-16T20:50:38.119 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7800000000000001e-06 2024-09-16T20:50:38.119 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.119 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.119 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:38.120 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481312+0000", 2024-09-16T20:50:38.120 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6349999999999997e-06 2024-09-16T20:50:38.120 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.120 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.120 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:38.120 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481315+0000", 2024-09-16T20:50:38.120 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7829999999999999e-06 2024-09-16T20:50:38.120 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.120 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.120 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:38.120 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.526876+0000", 2024-09-16T20:50:38.120 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.045561070000000002 2024-09-16T20:50:38.120 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.120 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.121 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:38.121 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.526954+0000", 2024-09-16T20:50:38.121 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8108000000000005e-05 2024-09-16T20:50:38.121 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.121 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.121 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:38.121 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527282+0000", 2024-09-16T20:50:38.121 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000327483 2024-09-16T20:50:38.121 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.121 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.121 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:38.121 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531784+0000", 2024-09-16T20:50:38.121 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0045020169999999997 2024-09-16T20:50:38.121 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.121 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.122 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:38.122 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534921+0000", 2024-09-16T20:50:38.122 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.003137791 2024-09-16T20:50:38.122 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.122 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.122 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:38.122 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534976+0000", 2024-09-16T20:50:38.122 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4953000000000001e-05 2024-09-16T20:50:38.122 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.122 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.122 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:38.122 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535106+0000", 2024-09-16T20:50:38.123 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000129589 2024-09-16T20:50:38.123 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:38.123 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:38.123 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:38.123 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.123 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.123 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3576 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:38.123 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:50:38.123 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 108.621049365, 2024-09-16T20:50:38.123 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053955436000000002, 2024-09-16T20:50:38.123 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:38.124 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:38.124 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:38.124 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:38.124 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:38.124 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:38.124 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3576 2024-09-16T20:50:38.124 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.124 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:38.124 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.124 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:38.124 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:50:38.124 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:38.124 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.125 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.125 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:38.125 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:50:38.125 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:38.125 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.125 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.125 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:38.125 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481319+0000", 2024-09-16T20:50:38.125 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.53e-07 2024-09-16T20:50:38.125 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.125 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.125 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:38.125 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481320+0000", 2024-09-16T20:50:38.125 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5230000000000001e-06 2024-09-16T20:50:38.125 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.126 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.126 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:38.126 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481330+0000", 2024-09-16T20:50:38.126 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0190000000000001e-05 2024-09-16T20:50:38.126 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.126 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.126 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:38.126 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481333+0000", 2024-09-16T20:50:38.126 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.151e-06 2024-09-16T20:50:38.126 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.126 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.126 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:38.126 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527689+0000", 2024-09-16T20:50:38.126 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046355192000000003 2024-09-16T20:50:38.126 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.127 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.127 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:38.127 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527764+0000", 2024-09-16T20:50:38.127 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5776000000000001e-05 2024-09-16T20:50:38.127 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.127 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.127 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:38.127 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528141+0000", 2024-09-16T20:50:38.127 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00037647599999999999 2024-09-16T20:50:38.127 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.127 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.127 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:38.127 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531788+0000", 2024-09-16T20:50:38.127 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0036470679999999998 2024-09-16T20:50:38.128 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.128 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.128 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:38.128 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535174+0000", 2024-09-16T20:50:38.128 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0033862670000000001 2024-09-16T20:50:38.128 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.128 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.128 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:38.128 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535236+0000", 2024-09-16T20:50:38.128 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1797999999999993e-05 2024-09-16T20:50:38.128 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.128 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.128 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:38.128 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535274+0000", 2024-09-16T20:50:38.129 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7741999999999999e-05 2024-09-16T20:50:38.129 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:38.129 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:38.129 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:38.129 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.129 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.129 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3577 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:38.129 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:50:38.129 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 108.62102776499999, 2024-09-16T20:50:38.129 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054073807000000002, 2024-09-16T20:50:38.129 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:38.129 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:38.129 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:38.129 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:38.130 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:38.130 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:38.130 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3577 2024-09-16T20:50:38.130 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.130 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:38.130 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.130 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:38.130 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:50:38.130 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:38.130 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.130 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.130 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:38.130 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:50:38.130 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:38.130 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.131 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.131 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:38.131 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:50:38.131 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.96e-07 2024-09-16T20:50:38.131 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.131 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.131 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:38.131 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481342+0000", 2024-09-16T20:50:38.131 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.714e-06 2024-09-16T20:50:38.131 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.131 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.131 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:38.131 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481349+0000", 2024-09-16T20:50:38.131 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4930000000000001e-06 2024-09-16T20:50:38.132 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.132 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.132 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:38.132 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481353+0000", 2024-09-16T20:50:38.132 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4879999999999999e-06 2024-09-16T20:50:38.132 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.132 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.132 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:38.132 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528619+0000", 2024-09-16T20:50:38.132 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047266351999999998 2024-09-16T20:50:38.132 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.132 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.132 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:38.133 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528706+0000", 2024-09-16T20:50:38.133 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6796e-05 2024-09-16T20:50:38.133 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.133 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.133 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:38.133 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529148+0000", 2024-09-16T20:50:38.133 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00044182999999999999 2024-09-16T20:50:38.133 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.133 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.133 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:38.133 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531792+0000", 2024-09-16T20:50:38.133 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0026443119999999998 2024-09-16T20:50:38.133 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.133 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.134 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:38.134 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535319+0000", 2024-09-16T20:50:38.134 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0035266439999999998 2024-09-16T20:50:38.134 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.134 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.134 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:38.134 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535372+0000", 2024-09-16T20:50:38.134 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3207000000000001e-05 2024-09-16T20:50:38.134 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.134 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.134 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:38.134 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535414+0000", 2024-09-16T20:50:38.134 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1675000000000003e-05 2024-09-16T20:50:38.134 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:38.135 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:38.135 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:38.135 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.135 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.135 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3578 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:38.135 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:50:38.135 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 108.621006596, 2024-09-16T20:50:38.135 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054189586999999997, 2024-09-16T20:50:38.135 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:38.135 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:38.135 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:38.135 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:38.135 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:38.135 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:38.136 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3578 2024-09-16T20:50:38.136 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.136 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:38.136 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.136 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:38.136 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:50:38.136 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:38.136 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.136 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.136 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:38.136 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:50:38.136 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:38.136 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.136 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.137 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:38.137 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:50:38.137 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0800000000000001e-07 2024-09-16T20:50:38.137 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.137 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.137 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:38.137 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481363+0000", 2024-09-16T20:50:38.137 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0339999999999999e-06 2024-09-16T20:50:38.137 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.137 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.137 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:38.137 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481371+0000", 2024-09-16T20:50:38.137 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4440000000000002e-06 2024-09-16T20:50:38.137 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.138 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.138 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:38.138 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481374+0000", 2024-09-16T20:50:38.138 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2049999999999998e-06 2024-09-16T20:50:38.138 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.138 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.138 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:38.138 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529555+0000", 2024-09-16T20:50:38.138 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048181202999999999 2024-09-16T20:50:38.138 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.138 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.138 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:38.138 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529616+0000", 2024-09-16T20:50:38.138 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0912000000000002e-05 2024-09-16T20:50:38.138 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.139 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.139 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:38.139 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529939+0000", 2024-09-16T20:50:38.139 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00032269800000000001 2024-09-16T20:50:38.139 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.139 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.139 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:38.139 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532602+0000", 2024-09-16T20:50:38.139 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0026634940000000002 2024-09-16T20:50:38.139 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.139 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.139 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:38.139 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535465+0000", 2024-09-16T20:50:38.139 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.002862149 2024-09-16T20:50:38.140 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.140 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.140 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:38.140 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535516+0000", 2024-09-16T20:50:38.140 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1696000000000001e-05 2024-09-16T20:50:38.140 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.140 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.140 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:38.140 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535551+0000", 2024-09-16T20:50:38.140 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4443999999999998e-05 2024-09-16T20:50:38.140 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:38.140 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:38.140 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:38.140 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.141 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.141 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3579 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:38.141 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:50:38.141 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 108.620988157, 2024-09-16T20:50:38.141 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054295151, 2024-09-16T20:50:38.141 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:38.141 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:38.141 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:38.141 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:38.141 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:38.141 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:38.141 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3579 2024-09-16T20:50:38.141 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.142 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:38.142 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.142 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:38.142 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:50:38.142 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:38.142 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.142 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.142 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:38.142 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:50:38.142 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:38.142 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.142 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.142 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:38.142 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:50:38.143 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3300000000000001e-07 2024-09-16T20:50:38.143 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.143 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.143 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:38.143 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481381+0000", 2024-09-16T20:50:38.143 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.649e-06 2024-09-16T20:50:38.143 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.143 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.143 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:38.143 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481387+0000", 2024-09-16T20:50:38.143 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4519999999999996e-06 2024-09-16T20:50:38.143 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.143 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.143 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:38.144 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481390+0000", 2024-09-16T20:50:38.144 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0170000000000001e-06 2024-09-16T20:50:38.144 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.144 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.144 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:38.144 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530756+0000", 2024-09-16T20:50:38.144 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049365882999999999 2024-09-16T20:50:38.144 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.144 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.144 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:38.144 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530845+0000", 2024-09-16T20:50:38.144 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9197000000000007e-05 2024-09-16T20:50:38.144 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.144 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.145 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:38.145 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531191+0000", 2024-09-16T20:50:38.145 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00034562500000000001 2024-09-16T20:50:38.145 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.145 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.145 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:38.145 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534078+0000", 2024-09-16T20:50:38.145 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0028878179999999999 2024-09-16T20:50:38.145 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.145 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.145 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:38.145 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535589+0000", 2024-09-16T20:50:38.145 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00151085 2024-09-16T20:50:38.145 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.146 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.146 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:38.146 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535639+0000", 2024-09-16T20:50:38.146 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9697000000000002e-05 2024-09-16T20:50:38.146 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.146 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.146 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:38.146 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535675+0000", 2024-09-16T20:50:38.146 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5729999999999998e-05 2024-09-16T20:50:38.146 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:38.146 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:38.146 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:38.146 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.146 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.146 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3580 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:38.147 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:50:38.147 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 108.62097452499999, 2024-09-16T20:50:38.147 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054397361999999998, 2024-09-16T20:50:38.147 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:38.147 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:38.147 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:38.147 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:38.147 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:38.147 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:38.147 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3580 2024-09-16T20:50:38.147 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.147 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:38.147 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.148 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:38.148 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:50:38.148 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:38.148 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.148 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.148 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:38.148 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:50:38.148 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:38.148 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.148 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.148 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:38.148 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:50:38.148 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2600000000000001e-07 2024-09-16T20:50:38.149 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.149 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.149 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:38.149 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481395+0000", 2024-09-16T20:50:38.149 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.544e-06 2024-09-16T20:50:38.149 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.149 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.149 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:38.149 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481404+0000", 2024-09-16T20:50:38.149 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8850000000000005e-06 2024-09-16T20:50:38.149 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.149 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.149 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:38.149 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481407+0000", 2024-09-16T20:50:38.150 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1540000000000002e-06 2024-09-16T20:50:38.150 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.150 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.150 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:38.150 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531823+0000", 2024-09-16T20:50:38.150 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050415637999999999 2024-09-16T20:50:38.150 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.150 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.150 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:38.150 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531895+0000", 2024-09-16T20:50:38.150 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2179000000000001e-05 2024-09-16T20:50:38.150 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.150 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.151 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:38.151 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532236+0000", 2024-09-16T20:50:38.151 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00034073799999999999 2024-09-16T20:50:38.151 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.151 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.151 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:38.151 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534083+0000", 2024-09-16T20:50:38.151 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001847298 2024-09-16T20:50:38.151 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.151 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.151 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:38.151 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535714+0000", 2024-09-16T20:50:38.151 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0016316810000000001 2024-09-16T20:50:38.151 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.152 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.152 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:38.152 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535759+0000", 2024-09-16T20:50:38.152 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4478999999999999e-05 2024-09-16T20:50:38.152 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.152 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.152 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:38.152 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535791+0000", 2024-09-16T20:50:38.152 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.154e-05 2024-09-16T20:50:38.152 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:38.152 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:38.152 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:38.152 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.152 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.153 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3581 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:38.153 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:50:38.153 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 108.620952528, 2024-09-16T20:50:38.153 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.057094403000000002, 2024-09-16T20:50:38.153 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:38.153 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:38.153 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:38.153 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:38.153 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:38.153 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:38.153 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3581 2024-09-16T20:50:38.153 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.153 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:38.154 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.154 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:38.154 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:50:38.154 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:38.154 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.154 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.154 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:38.154 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:50:38.154 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:38.154 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.154 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.154 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:38.154 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:50:38.154 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8799999999999998e-07 2024-09-16T20:50:38.155 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.155 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.155 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:38.155 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481418+0000", 2024-09-16T20:50:38.155 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1859999999999999e-06 2024-09-16T20:50:38.155 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.155 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.155 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:38.155 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481423+0000", 2024-09-16T20:50:38.155 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8279999999999999e-06 2024-09-16T20:50:38.155 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.155 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.155 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:38.155 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481426+0000", 2024-09-16T20:50:38.156 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0110000000000001e-06 2024-09-16T20:50:38.156 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.156 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.156 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:38.156 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532631+0000", 2024-09-16T20:50:38.156 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051204514999999999 2024-09-16T20:50:38.156 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.156 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.156 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:38.156 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.533381+0000", 2024-09-16T20:50:38.156 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00075029000000000003 2024-09-16T20:50:38.156 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.156 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.156 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:38.156 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.533739+0000", 2024-09-16T20:50:38.157 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00035773100000000002 2024-09-16T20:50:38.157 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.157 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.157 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:38.157 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535123+0000", 2024-09-16T20:50:38.157 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0013836160000000001 2024-09-16T20:50:38.157 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.157 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.157 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:38.157 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538413+0000", 2024-09-16T20:50:38.157 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0032902510000000001 2024-09-16T20:50:38.157 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.157 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.157 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:38.157 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538470+0000", 2024-09-16T20:50:38.158 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6697999999999998e-05 2024-09-16T20:50:38.158 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.158 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.158 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:38.158 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538510+0000", 2024-09-16T20:50:38.158 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9988999999999999e-05 2024-09-16T20:50:38.158 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:38.158 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:38.158 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:38.158 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.158 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.158 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:12732 3.d 3:b752444a:::rbd_header.110e747f0779:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-16T20:50:38.158 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:50:38.158 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 93.333879949999996, 2024-09-16T20:50:38.159 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.065302467000000003, 2024-09-16T20:50:38.159 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:38.159 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:38.159 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:38.159 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:38.159 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:50:38.159 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:50:38.159 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 12732 2024-09-16T20:50:38.159 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.159 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:38.159 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.159 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:38.159 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:50:38.160 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:38.160 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.160 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.160 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:38.160 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:50:38.160 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:38.160 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.160 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.160 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:38.160 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768490+0000", 2024-09-16T20:50:38.160 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2859999999999998e-06 2024-09-16T20:50:38.160 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.160 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.160 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:38.161 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768501+0000", 2024-09-16T20:50:38.161 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1226999999999999e-05 2024-09-16T20:50:38.161 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.161 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.161 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:38.161 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768517+0000", 2024-09-16T20:50:38.161 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.592e-05 2024-09-16T20:50:38.161 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.161 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.161 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:38.161 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768528+0000", 2024-09-16T20:50:38.161 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0689999999999999e-05 2024-09-16T20:50:38.161 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.161 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.162 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:38.162 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768584+0000", 2024-09-16T20:50:38.162 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5889e-05 2024-09-16T20:50:38.162 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.162 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.162 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:38.162 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768720+0000", 2024-09-16T20:50:38.162 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000136718 2024-09-16T20:50:38.162 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.162 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.162 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:38.162 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.769066+0000", 2024-09-16T20:50:38.162 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000346013 2024-09-16T20:50:38.162 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.162 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.163 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:38.163 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833659+0000", 2024-09-16T20:50:38.163 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.064592822999999994 2024-09-16T20:50:38.163 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.163 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.163 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:38.163 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833719+0000", 2024-09-16T20:50:38.163 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9441000000000002e-05 2024-09-16T20:50:38.163 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.163 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.163 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:38.163 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833758+0000", 2024-09-16T20:50:38.163 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9122000000000003e-05 2024-09-16T20:50:38.163 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.164 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.164 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:38.164 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833790+0000", 2024-09-16T20:50:38.164 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2338000000000001e-05 2024-09-16T20:50:38.164 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:38.164 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:38.164 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:38.164 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.164 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.164 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.5089.0:12732 3.d e73/23 ondisk, result = 0)", 2024-09-16T20:50:38.164 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:50:38.164 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 93.332277779999998, 2024-09-16T20:50:38.164 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.063706456999999994, 2024-09-16T20:50:38.165 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:38.165 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:38.165 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:38.165 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:38.165 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.165 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:38.165 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:50:38.165 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:38.165 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.165 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.165 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:38.165 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:50:38.165 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:38.165 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.166 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.166 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:38.166 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770091+0000", 2024-09-16T20:50:38.166 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.53e-06 2024-09-16T20:50:38.166 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.166 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.166 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:38.166 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770104+0000", 2024-09-16T20:50:38.166 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2203e-05 2024-09-16T20:50:38.166 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.166 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.166 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:38.166 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833637+0000", 2024-09-16T20:50:38.166 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.063533201999999997 2024-09-16T20:50:38.167 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.167 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.167 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:38.167 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833655+0000", 2024-09-16T20:50:38.167 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8369e-05 2024-09-16T20:50:38.167 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.167 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.167 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:38.167 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833707+0000", 2024-09-16T20:50:38.167 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2061999999999997e-05 2024-09-16T20:50:38.167 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.167 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.167 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:38.168 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833715+0000", 2024-09-16T20:50:38.168 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5179999999999998e-06 2024-09-16T20:50:38.168 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.168 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.168 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:38.168 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833796+0000", 2024-09-16T20:50:38.168 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1573e-05 2024-09-16T20:50:38.168 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:38.168 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:38.168 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:38.168 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.168 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.168 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3285 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 240640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:38.169 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041140+0000", 2024-09-16T20:50:38.169 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 48.061228002, 2024-09-16T20:50:38.169 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052375646999999997, 2024-09-16T20:50:38.169 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:38.169 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:38.169 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:38.169 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:38.169 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:38.169 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:38.169 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3285 2024-09-16T20:50:38.169 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.169 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:38.169 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.169 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:38.169 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041140+0000", 2024-09-16T20:50:38.170 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:38.170 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.170 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.170 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:38.170 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041140+0000", 2024-09-16T20:50:38.170 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:38.170 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.170 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.170 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:38.170 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041140+0000", 2024-09-16T20:50:38.170 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2399999999999999e-07 2024-09-16T20:50:38.170 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.170 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.170 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:38.170 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041141+0000", 2024-09-16T20:50:38.171 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2759999999999999e-06 2024-09-16T20:50:38.171 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.171 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.171 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:38.171 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041144+0000", 2024-09-16T20:50:38.171 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0280000000000001e-06 2024-09-16T20:50:38.171 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.171 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.171 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:38.171 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041147+0000", 2024-09-16T20:50:38.171 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5900000000000002e-06 2024-09-16T20:50:38.171 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.171 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.171 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:38.172 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093284+0000", 2024-09-16T20:50:38.172 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052137645000000003 2024-09-16T20:50:38.172 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.172 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.172 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:38.172 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093365+0000", 2024-09-16T20:50:38.172 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0446999999999997e-05 2024-09-16T20:50:38.172 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.172 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.172 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:38.172 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093515+0000", 2024-09-16T20:50:38.172 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00015043700000000001 2024-09-16T20:50:38.172 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:38.172 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:38.173 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:38.173 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.173 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.173 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3286 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 245760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:38.173 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:50:38.173 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 48.061215842999999, 2024-09-16T20:50:38.173 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052523564000000002, 2024-09-16T20:50:38.173 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:38.173 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:38.173 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:38.173 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:38.173 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:38.173 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:38.173 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3286 2024-09-16T20:50:38.174 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.174 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:38.174 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.174 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:38.174 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:50:38.174 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:38.174 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.174 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.174 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:38.174 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:50:38.174 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:38.174 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.174 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.174 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:38.175 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:50:38.175 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3900000000000001e-07 2024-09-16T20:50:38.175 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.175 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.175 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:38.175 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041153+0000", 2024-09-16T20:50:38.175 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.119e-06 2024-09-16T20:50:38.175 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.175 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.175 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:38.175 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041156+0000", 2024-09-16T20:50:38.175 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0000000000000001e-06 2024-09-16T20:50:38.175 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.175 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.176 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:38.176 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041159+0000", 2024-09-16T20:50:38.176 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5859999999999999e-06 2024-09-16T20:50:38.176 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.176 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.176 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:38.176 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093529+0000", 2024-09-16T20:50:38.176 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052370591000000001 2024-09-16T20:50:38.176 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.176 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.176 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:38.176 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093570+0000", 2024-09-16T20:50:38.176 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0161999999999999e-05 2024-09-16T20:50:38.177 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.177 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.177 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:38.177 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093675+0000", 2024-09-16T20:50:38.177 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010586699999999999 2024-09-16T20:50:38.177 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:38.177 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:38.177 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:38.177 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.177 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.177 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3287 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 250880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:38.177 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:50:38.177 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 48.061205794000003, 2024-09-16T20:50:38.177 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052781203999999998, 2024-09-16T20:50:38.178 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:38.178 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:38.178 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:38.178 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:38.178 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:38.178 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:38.178 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3287 2024-09-16T20:50:38.178 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.178 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:38.178 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.178 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:38.178 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:50:38.178 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:38.178 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.179 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.179 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:38.179 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:50:38.179 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:38.179 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.179 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.179 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:38.179 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:50:38.179 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1e-07 2024-09-16T20:50:38.179 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.179 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.179 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:38.179 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041163+0000", 2024-09-16T20:50:38.180 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0690000000000001e-06 2024-09-16T20:50:38.180 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.180 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.180 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:38.180 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041166+0000", 2024-09-16T20:50:38.180 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.982e-06 2024-09-16T20:50:38.180 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.180 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.180 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:38.180 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041169+0000", 2024-09-16T20:50:38.180 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.593e-06 2024-09-16T20:50:38.181 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.181 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.181 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:38.181 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093709+0000", 2024-09-16T20:50:38.181 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052540492000000001 2024-09-16T20:50:38.181 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.181 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.181 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:38.181 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093792+0000", 2024-09-16T20:50:38.181 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2436999999999997e-05 2024-09-16T20:50:38.181 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.181 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.182 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:38.182 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093943+0000", 2024-09-16T20:50:38.182 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000151421 2024-09-16T20:50:38.182 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:38.182 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:38.182 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:38.182 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.182 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.182 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3288 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 256000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:38.182 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:50:38.182 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 48.061195920999999, 2024-09-16T20:50:38.182 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052928269, 2024-09-16T20:50:38.183 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:38.183 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:38.183 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:38.183 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:38.183 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:38.183 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:38.183 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3288 2024-09-16T20:50:38.183 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.184 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:38.184 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.184 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:38.184 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:50:38.184 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:38.184 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.184 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.184 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:38.184 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:50:38.184 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:38.184 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.184 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.184 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:38.185 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:50:38.185 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2700000000000001e-07 2024-09-16T20:50:38.185 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.185 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.185 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:38.185 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041173+0000", 2024-09-16T20:50:38.185 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.113e-06 2024-09-16T20:50:38.185 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.185 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.185 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:38.185 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041176+0000", 2024-09-16T20:50:38.185 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9799999999999998e-06 2024-09-16T20:50:38.185 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.186 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.186 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:38.186 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041179+0000", 2024-09-16T20:50:38.186 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.667e-06 2024-09-16T20:50:38.186 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.186 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.186 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:38.186 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093957+0000", 2024-09-16T20:50:38.186 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052778703000000003 2024-09-16T20:50:38.186 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.186 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.186 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:38.186 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093999+0000", 2024-09-16T20:50:38.186 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1300000000000001e-05 2024-09-16T20:50:38.187 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.187 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.187 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:38.187 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094100+0000", 2024-09-16T20:50:38.187 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101279 2024-09-16T20:50:38.187 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:38.187 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:38.187 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:38.187 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.187 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.187 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3289 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 261120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:38.187 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:50:38.187 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 48.061185924999997, 2024-09-16T20:50:38.188 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053182976, 2024-09-16T20:50:38.188 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:38.188 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:38.188 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:38.188 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:38.188 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:38.188 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:38.188 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3289 2024-09-16T20:50:38.188 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.188 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:38.188 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.188 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:38.188 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:50:38.189 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:38.189 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.189 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.189 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:38.189 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:50:38.189 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:38.189 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.189 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.189 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:38.189 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:50:38.189 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2999999999999999e-07 2024-09-16T20:50:38.189 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.189 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.190 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:38.190 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041183+0000", 2024-09-16T20:50:38.190 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0720000000000001e-06 2024-09-16T20:50:38.190 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.190 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.190 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:38.190 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041186+0000", 2024-09-16T20:50:38.190 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9869999999999999e-06 2024-09-16T20:50:38.190 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.190 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.190 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:38.190 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041189+0000", 2024-09-16T20:50:38.190 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.587e-06 2024-09-16T20:50:38.190 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.191 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.191 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:38.191 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094136+0000", 2024-09-16T20:50:38.191 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052947270999999997 2024-09-16T20:50:38.191 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.191 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.191 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:38.191 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094213+0000", 2024-09-16T20:50:38.191 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7552000000000006e-05 2024-09-16T20:50:38.191 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.191 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.191 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:38.191 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094365+0000", 2024-09-16T20:50:38.191 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00015127700000000001 2024-09-16T20:50:38.192 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:38.192 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:38.192 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:38.192 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.192 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.192 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:2196 3.d 3:b1fe01a0:::rbd_data.110e7ef3004.0000000000000000:2d [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e136)", 2024-09-16T20:50:38.192 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:22.739057+0000", 2024-09-16T20:50:38.192 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.363311125999999, 2024-09-16T20:50:38.192 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11999464899999999, 2024-09-16T20:50:38.192 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:38.192 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:38.192 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:38.192 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:38.192 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:50:38.193 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:38.193 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2196 2024-09-16T20:50:38.193 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.193 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:38.193 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.193 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:38.193 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.739057+0000", 2024-09-16T20:50:38.193 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:38.193 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.193 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.193 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:38.193 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.739057+0000", 2024-09-16T20:50:38.193 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:38.193 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.194 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.194 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:38.194 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.739057+0000", 2024-09-16T20:50:38.194 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1399999999999997e-07 2024-09-16T20:50:38.194 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.194 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.194 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:38.194 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.739060+0000", 2024-09-16T20:50:38.194 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6019999999999998e-06 2024-09-16T20:50:38.194 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.194 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.194 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:38.194 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.858755+0000", 2024-09-16T20:50:38.194 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.119695151 2024-09-16T20:50:38.195 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.195 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.195 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:38.195 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.858778+0000", 2024-09-16T20:50:38.195 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3184999999999999e-05 2024-09-16T20:50:38.195 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.195 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.195 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:38.195 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.858865+0000", 2024-09-16T20:50:38.195 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.7330000000000006e-05 2024-09-16T20:50:38.195 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.195 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.195 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:38.196 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.858952+0000", 2024-09-16T20:50:38.196 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6803000000000002e-05 2024-09-16T20:50:38.196 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.196 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.196 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:38.196 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.859051+0000", 2024-09-16T20:50:38.196 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8963999999999996e-05 2024-09-16T20:50:38.196 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:38.196 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:38.196 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:38.196 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.196 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.196 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6061.0:5401 3.13 3:c8554348:::rbd_data.110e16eafbd.0000000000000011:head [read 577536~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-16T20:50:38.196 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:27.761536+0000", 2024-09-16T20:50:38.196 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 10.340831313000001, 2024-09-16T20:50:38.197 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.003742712, 2024-09-16T20:50:38.197 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:38.197 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:38.197 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:50:38.197 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:38.197 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6061", 2024-09-16T20:50:38.197 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3930128440", 2024-09-16T20:50:38.197 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5401 2024-09-16T20:50:38.197 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.197 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:38.197 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.197 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:38.197 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761536+0000", 2024-09-16T20:50:38.198 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:38.198 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.198 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.198 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:38.198 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761536+0000", 2024-09-16T20:50:38.198 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:38.198 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.198 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.198 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:38.198 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761537+0000", 2024-09-16T20:50:38.198 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2900000000000003e-07 2024-09-16T20:50:38.198 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.198 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.199 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:38.199 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761540+0000", 2024-09-16T20:50:38.199 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2320000000000001e-06 2024-09-16T20:50:38.199 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.199 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.199 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:38.199 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761548+0000", 2024-09-16T20:50:38.199 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1289999999999996e-06 2024-09-16T20:50:38.199 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.199 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.199 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:38.199 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761554+0000", 2024-09-16T20:50:38.199 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0820000000000004e-06 2024-09-16T20:50:38.199 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.200 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.200 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:38.200 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761621+0000", 2024-09-16T20:50:38.200 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6266000000000006e-05 2024-09-16T20:50:38.200 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.200 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.200 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:38.200 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761626+0000", 2024-09-16T20:50:38.200 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.614e-06 2024-09-16T20:50:38.200 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.200 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.200 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:38.200 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:28.765163+0000", 2024-09-16T20:50:38.200 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.003536416 2024-09-16T20:50:38.201 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.201 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.201 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:38.201 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:28.765279+0000", 2024-09-16T20:50:38.201 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000116344 2024-09-16T20:50:38.201 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:38.201 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:38.201 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:38.201 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.201 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.201 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6061.0:5425 3.13 3:c8554348:::rbd_data.110e16eafbd.0000000000000011:head [read 602112~28672] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-16T20:50:38.201 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:27.767049+0000", 2024-09-16T20:50:38.201 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 10.335318654, 2024-09-16T20:50:38.201 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.99847897900000004, 2024-09-16T20:50:38.202 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:38.202 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:38.202 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:50:38.202 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:38.202 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6061", 2024-09-16T20:50:38.202 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3930128440", 2024-09-16T20:50:38.202 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5425 2024-09-16T20:50:38.202 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.202 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:38.202 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.202 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:38.202 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767049+0000", 2024-09-16T20:50:38.202 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:38.203 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.203 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.203 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:38.203 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767049+0000", 2024-09-16T20:50:38.203 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:38.203 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.203 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.203 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:38.203 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767050+0000", 2024-09-16T20:50:38.203 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6499999999999999e-07 2024-09-16T20:50:38.203 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.203 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.203 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:38.203 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767052+0000", 2024-09-16T20:50:38.204 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7350000000000001e-06 2024-09-16T20:50:38.204 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.204 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.204 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:38.204 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767060+0000", 2024-09-16T20:50:38.204 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5109999999999997e-06 2024-09-16T20:50:38.204 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.204 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.204 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:38.204 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767066+0000", 2024-09-16T20:50:38.204 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8139999999999997e-06 2024-09-16T20:50:38.204 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.205 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.205 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:38.205 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767128+0000", 2024-09-16T20:50:38.205 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2174999999999993e-05 2024-09-16T20:50:38.205 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.205 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.205 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:38.205 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767133+0000", 2024-09-16T20:50:38.205 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2140000000000004e-06 2024-09-16T20:50:38.205 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.205 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.205 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:38.205 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:28.765441+0000", 2024-09-16T20:50:38.205 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.99830808599999998 2024-09-16T20:50:38.206 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.206 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.206 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:38.206 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:28.765528+0000", 2024-09-16T20:50:38.206 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6879000000000006e-05 2024-09-16T20:50:38.206 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:38.206 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:38.206 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:38.206 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.206 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.206 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:25585 3.10 3:0c7c5b23:::rbd_header.110ebf0678ad:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:38.206 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:32.877132+0000", 2024-09-16T20:50:38.206 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 5.2252361049999996, 2024-09-16T20:50:38.206 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054096699999999998, 2024-09-16T20:50:38.206 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:38.207 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:38.207 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:38.207 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:38.207 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:50:38.207 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:38.207 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 25585 2024-09-16T20:50:38.207 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.207 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:38.207 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.207 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:38.207 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.877132+0000", 2024-09-16T20:50:38.207 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:38.207 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.208 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.208 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:38.208 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.877132+0000", 2024-09-16T20:50:38.208 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:38.208 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.208 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.208 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:38.208 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931072+0000", 2024-09-16T20:50:38.208 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053939931000000003 2024-09-16T20:50:38.208 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.208 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.208 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:38.208 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931076+0000", 2024-09-16T20:50:38.208 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2860000000000001e-06 2024-09-16T20:50:38.209 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.209 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.209 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:38.209 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931085+0000", 2024-09-16T20:50:38.209 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.3370000000000006e-06 2024-09-16T20:50:38.209 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.209 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.209 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:38.209 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931092+0000", 2024-09-16T20:50:38.209 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9970000000000001e-06 2024-09-16T20:50:38.209 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.209 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.209 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:38.210 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931137+0000", 2024-09-16T20:50:38.210 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5160000000000001e-05 2024-09-16T20:50:38.210 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.210 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.210 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:38.210 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931178+0000", 2024-09-16T20:50:38.210 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0379000000000001e-05 2024-09-16T20:50:38.210 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.210 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.210 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:38.210 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931228+0000", 2024-09-16T20:50:38.210 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0609999999999998e-05 2024-09-16T20:50:38.210 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:38.210 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:38.211 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:38.211 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:38.211 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:38.211 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:38.212 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:50:38.213 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:38.211+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:38.232 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T20:50:38.312 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:38.311+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:38.364 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:38.364 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:50:38.364 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.364 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:26393 3.1e 3.562e3dde (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-16T20:50:38.365 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:36.786061+0000", 2024-09-16T20:50:38.365 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 1.578027155, 2024-09-16T20:50:38.365 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.578053828, 2024-09-16T20:50:38.365 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:38.365 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:38.365 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "waiting for peered", 2024-09-16T20:50:38.365 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:38.365 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:50:38.365 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:38.365 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 26393 2024-09-16T20:50:38.365 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.365 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:38.366 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.366 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:38.366 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:36.786061+0000", 2024-09-16T20:50:38.366 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:38.366 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.366 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.366 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:38.366 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:36.786061+0000", 2024-09-16T20:50:38.366 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:38.366 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.366 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.366 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:38.366 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:36.786063+0000", 2024-09-16T20:50:38.367 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.52e-06 2024-09-16T20:50:38.367 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.367 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.367 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:38.367 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:36.786071+0000", 2024-09-16T20:50:38.367 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1589999999999993e-06 2024-09-16T20:50:38.367 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.367 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.367 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:38.367 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:36.786087+0000", 2024-09-16T20:50:38.367 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5588000000000001e-05 2024-09-16T20:50:38.367 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.367 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.368 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:38.368 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:36.786109+0000", 2024-09-16T20:50:38.368 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1994999999999999e-05 2024-09-16T20:50:38.368 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.368 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.368 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:38.368 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:36.786205+0000", 2024-09-16T20:50:38.368 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.6198000000000003e-05 2024-09-16T20:50:38.368 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.368 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.368 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op must wait for map", 2024-09-16T20:50:38.368 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:36.786215+0000", 2024-09-16T20:50:38.369 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8980000000000008e-06 2024-09-16T20:50:38.369 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.369 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.369 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:38.369 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:36.791802+0000", 2024-09-16T20:50:38.369 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0055869170000000003 2024-09-16T20:50:38.369 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.369 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.369 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:38.369 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:36.791815+0000", 2024-09-16T20:50:38.369 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3198e-05 2024-09-16T20:50:38.369 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.369 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.369 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:38.370 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:37.792151+0000", 2024-09-16T20:50:38.370 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.000336047 2024-09-16T20:50:38.370 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.370 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.370 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:38.370 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:37.792165+0000", 2024-09-16T20:50:38.370 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4042e-05 2024-09-16T20:50:38.370 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:38.370 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:38.370 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:38.370 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.370 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.370 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:26394 3.1e 3.562e3dde (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-16T20:50:38.371 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:36.790792+0000", 2024-09-16T20:50:38.371 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 1.573296096, 2024-09-16T20:50:38.371 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5733704450000001, 2024-09-16T20:50:38.371 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:38.371 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:38.371 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "waiting for peered", 2024-09-16T20:50:38.371 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:38.371 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:50:38.371 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:38.371 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 26394 2024-09-16T20:50:38.371 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.371 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:38.371 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.371 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:38.371 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:36.790792+0000", 2024-09-16T20:50:38.372 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:38.372 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.372 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.372 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:38.372 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:36.790792+0000", 2024-09-16T20:50:38.372 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:38.372 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.372 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.372 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:38.372 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:36.790794+0000", 2024-09-16T20:50:38.372 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7590000000000002e-06 2024-09-16T20:50:38.372 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.372 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.372 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:38.373 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:36.790805+0000", 2024-09-16T20:50:38.373 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0341e-05 2024-09-16T20:50:38.373 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.373 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.373 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:38.373 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:36.790819+0000", 2024-09-16T20:50:38.373 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4382999999999999e-05 2024-09-16T20:50:38.373 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.373 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.373 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:38.373 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:36.790830+0000", 2024-09-16T20:50:38.373 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1187e-05 2024-09-16T20:50:38.374 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.374 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.374 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:38.374 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:36.793108+0000", 2024-09-16T20:50:38.374 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0022777159999999999 2024-09-16T20:50:38.374 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.374 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.374 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:38.374 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:36.793119+0000", 2024-09-16T20:50:38.374 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1375999999999999e-05 2024-09-16T20:50:38.374 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.374 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.374 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:38.374 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:37.792193+0000", 2024-09-16T20:50:38.375 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.99907357399999996 2024-09-16T20:50:38.375 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.375 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.375 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:38.375 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:37.792206+0000", 2024-09-16T20:50:38.375 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3135e-05 2024-09-16T20:50:38.375 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:38.375 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:38.375 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:38.375 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.375 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.375 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:26395 3.1e 3.562e3dde (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e150)", 2024-09-16T20:50:38.375 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:37.792589+0000", 2024-09-16T20:50:38.375 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.57149867799999998, 2024-09-16T20:50:38.375 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.57160797799999996, 2024-09-16T20:50:38.376 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:38.376 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:38.376 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "waiting for peered", 2024-09-16T20:50:38.376 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:38.376 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:50:38.376 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:38.376 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 26395 2024-09-16T20:50:38.376 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.376 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:38.376 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.376 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:38.376 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:37.792589+0000", 2024-09-16T20:50:38.376 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:38.376 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.377 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.377 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:38.377 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:37.792589+0000", 2024-09-16T20:50:38.377 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:38.377 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.377 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.377 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:38.377 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:37.792591+0000", 2024-09-16T20:50:38.377 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5999999999999999e-06 2024-09-16T20:50:38.377 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.377 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.377 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:38.377 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:37.792595+0000", 2024-09-16T20:50:38.377 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2440000000000004e-06 2024-09-16T20:50:38.378 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.378 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.378 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:38.378 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:37.792604+0000", 2024-09-16T20:50:38.378 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2699999999999993e-06 2024-09-16T20:50:38.378 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.378 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.378 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:38.378 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:37.792615+0000", 2024-09-16T20:50:38.378 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0644e-05 2024-09-16T20:50:38.378 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.378 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.378 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:38.379 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:37.793143+0000", 2024-09-16T20:50:38.379 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00052851099999999995 2024-09-16T20:50:38.379 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:38.379 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:38.379 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:38.379 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:37.793153+0000", 2024-09-16T20:50:38.379 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8730000000000002e-06 2024-09-16T20:50:38.379 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:38.379 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:38.379 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:38.379 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:38.379 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-16T20:50:38.379 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 3 2024-09-16T20:50:38.379 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:38.381 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:50:38.381 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:50:38.381 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:50:38.396 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:50:38.396 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:50:38.413 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:50:38.413 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:50:38.413 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:38.412+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:38.431 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:50:38.431 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:50:38.448 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:50:38.448 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:50:38.465 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:50:38.466 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:50:38.504 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:38.504 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:38.513 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:50:38.554 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:38.553+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:38.614 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:38.613+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:38.660 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:38.660 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:38.669 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:50:38.716 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:38.715+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:38.791 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:38.791 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:38.808 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:50:38.849 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:38.849+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:38.918 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:38.917+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:38.995 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:38.995 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:39.006 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:50:39.018 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:39.017+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:39.119 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:39.118+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:39.156 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:39.156 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:39.165 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:50:39.220 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:39.220+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:39.286 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:39.286 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:39.296 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:50:39.338 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:39.338+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:39.422 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:39.421+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:39.465 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:39.465 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:39.476 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:50:39.524 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:39.523+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:39.603 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:39.604 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:39.614 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:50:39.655 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:39.655+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:39.725 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:39.724+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:39.781 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:39.781 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:39.791 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:50:39.806 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T20:50:39.834 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:39.834+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:39.899 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:39.900 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:39.908 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:50:39.949 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:39.949+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:40.028 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:40.027+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:40.074 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:40.074 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:40.084 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:50:40.129 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:40.128+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:40.203 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:40.203 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:40.211 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:50:40.229 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:40.229+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:40.318 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:40.318 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:40.327 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:50:40.330 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:40.329+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:40.430 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:40.429+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:40.434 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:40.434 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:40.442 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:50:40.478 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T20:50:40.531 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:40.530+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:40.563 INFO:teuthology.orchestra.run.smithi029.stderr:nodeep-scrub is unset 2024-09-16T20:50:40.604 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:40.604 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:40.613 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:50:40.617 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:50:40.617 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:50:40.632 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:40.631+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:40.635 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:50:40.635 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:50:40.652 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:50:40.652 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:50:40.669 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:50:40.669 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:50:40.687 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:50:40.687 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:50:40.705 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:50:40.705 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:50:40.725 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:40.725 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:50:40.725 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:50:40.725 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:50:40.725 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:40.734 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:40.733+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:40.735 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:50:40.834 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:40.833+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:40.885 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:40.885 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:50:40.885 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:50:40.885 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:50:40.885 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.885 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:50:40.885 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:50:40.885 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 167.62890883599999, 2024-09-16T20:50:40.885 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67502346999999996, 2024-09-16T20:50:40.885 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:40.885 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:40.885 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:40.886 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:40.886 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:50:40.886 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:50:40.886 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 1 2024-09-16T20:50:40.886 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.886 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:40.886 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.886 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:40.886 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:50:40.886 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:40.886 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.886 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.886 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:40.887 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:50:40.887 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:40.887 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.887 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.887 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:40.887 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254128+0000", 2024-09-16T20:50:40.887 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.463e-06 2024-09-16T20:50:40.887 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.887 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.887 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:40.887 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254135+0000", 2024-09-16T20:50:40.887 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1579999999999999e-06 2024-09-16T20:50:40.887 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.887 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.887 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:40.888 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254146+0000", 2024-09-16T20:50:40.888 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1341e-05 2024-09-16T20:50:40.888 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.888 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.888 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:40.888 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254155+0000", 2024-09-16T20:50:40.888 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.1549999999999996e-06 2024-09-16T20:50:40.888 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.888 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.888 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:40.888 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254221+0000", 2024-09-16T20:50:40.888 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5470999999999997e-05 2024-09-16T20:50:40.888 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.888 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.889 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:40.889 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254231+0000", 2024-09-16T20:50:40.889 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0592e-05 2024-09-16T20:50:40.889 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.889 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.889 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:40.889 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926711+0000", 2024-09-16T20:50:40.889 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67247946599999997 2024-09-16T20:50:40.889 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.889 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.889 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:40.889 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926910+0000", 2024-09-16T20:50:40.889 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019934 2024-09-16T20:50:40.890 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.890 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.890 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:50:40.890 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.927159+0000", 2024-09-16T20:50:40.890 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00024876599999999998 2024-09-16T20:50:40.890 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.890 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.890 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:40.890 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.928797+0000", 2024-09-16T20:50:40.890 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001638308 2024-09-16T20:50:40.890 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.890 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.890 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:40.890 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929026+0000", 2024-09-16T20:50:40.891 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00022876099999999999 2024-09-16T20:50:40.891 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.891 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.891 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:40.891 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929102+0000", 2024-09-16T20:50:40.891 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6184000000000001e-05 2024-09-16T20:50:40.891 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.891 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.891 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:40.891 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929150+0000", 2024-09-16T20:50:40.891 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7465e-05 2024-09-16T20:50:40.891 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:40.891 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:40.891 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:40.892 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.892 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.892 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3574 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:40.892 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:50:40.892 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 111.401745304, 2024-09-16T20:50:40.892 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053579441999999998, 2024-09-16T20:50:40.892 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:40.892 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:40.892 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:40.892 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:40.892 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:40.893 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:40.893 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3574 2024-09-16T20:50:40.893 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.893 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:40.893 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.893 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:40.893 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:50:40.893 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:40.893 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.893 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.893 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:40.893 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:50:40.893 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:40.894 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.894 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.894 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:40.894 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:50:40.894 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2e-07 2024-09-16T20:50:40.894 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.894 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.894 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:40.894 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481292+0000", 2024-09-16T20:50:40.894 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2630000000000002e-06 2024-09-16T20:50:40.894 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.894 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.894 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:40.894 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481297+0000", 2024-09-16T20:50:40.894 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.296e-06 2024-09-16T20:50:40.895 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.895 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.895 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:40.895 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481300+0000", 2024-09-16T20:50:40.895 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7019999999999999e-06 2024-09-16T20:50:40.895 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.895 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.895 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:40.895 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525533+0000", 2024-09-16T20:50:40.895 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.044232318 2024-09-16T20:50:40.895 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.895 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.896 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:40.896 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525619+0000", 2024-09-16T20:50:40.896 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5892999999999999e-05 2024-09-16T20:50:40.896 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.896 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.896 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:40.896 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525987+0000", 2024-09-16T20:50:40.896 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00036836099999999999 2024-09-16T20:50:40.896 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.896 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.896 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:40.896 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531779+0000", 2024-09-16T20:50:40.896 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0057918240000000001 2024-09-16T20:50:40.896 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.897 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.897 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:40.897 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534747+0000", 2024-09-16T20:50:40.897 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0029681600000000001 2024-09-16T20:50:40.897 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.897 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.897 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:40.897 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534832+0000", 2024-09-16T20:50:40.897 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5180999999999996e-05 2024-09-16T20:50:40.897 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.897 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.897 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:40.897 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534869+0000", 2024-09-16T20:50:40.897 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7024000000000003e-05 2024-09-16T20:50:40.897 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:40.898 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:40.898 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:40.898 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.898 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.898 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3575 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:40.898 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:50:40.898 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 111.401730564, 2024-09-16T20:50:40.898 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053801463000000001, 2024-09-16T20:50:40.898 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:40.898 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:40.898 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:40.898 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:40.898 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:40.899 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:40.899 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3575 2024-09-16T20:50:40.899 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.899 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:40.899 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.899 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:40.899 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:50:40.899 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:40.899 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.899 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.899 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:40.899 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:50:40.899 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:40.899 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.900 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.900 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:40.900 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481305+0000", 2024-09-16T20:50:40.900 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5400000000000002e-07 2024-09-16T20:50:40.900 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.900 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.900 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:40.900 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481307+0000", 2024-09-16T20:50:40.900 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7800000000000001e-06 2024-09-16T20:50:40.900 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.900 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.900 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:40.900 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481312+0000", 2024-09-16T20:50:40.900 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6349999999999997e-06 2024-09-16T20:50:40.901 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.901 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.901 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:40.901 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481315+0000", 2024-09-16T20:50:40.901 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7829999999999999e-06 2024-09-16T20:50:40.901 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.901 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.901 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:40.901 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.526876+0000", 2024-09-16T20:50:40.901 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.045561070000000002 2024-09-16T20:50:40.901 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.901 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.901 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:40.901 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.526954+0000", 2024-09-16T20:50:40.901 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8108000000000005e-05 2024-09-16T20:50:40.902 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.902 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.902 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:40.902 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527282+0000", 2024-09-16T20:50:40.902 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000327483 2024-09-16T20:50:40.902 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.902 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.902 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:40.902 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531784+0000", 2024-09-16T20:50:40.902 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0045020169999999997 2024-09-16T20:50:40.902 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.902 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.902 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:40.903 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534921+0000", 2024-09-16T20:50:40.903 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.003137791 2024-09-16T20:50:40.903 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.903 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.903 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:40.903 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534976+0000", 2024-09-16T20:50:40.903 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4953000000000001e-05 2024-09-16T20:50:40.903 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.903 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.903 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:40.903 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535106+0000", 2024-09-16T20:50:40.903 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000129589 2024-09-16T20:50:40.903 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:40.904 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:40.904 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:40.904 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.904 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.904 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3576 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:40.904 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:50:40.904 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 111.401716652, 2024-09-16T20:50:40.904 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053955436000000002, 2024-09-16T20:50:40.904 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:40.904 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:40.904 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:40.904 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:40.904 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:40.904 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:40.905 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3576 2024-09-16T20:50:40.905 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.905 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:40.905 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.905 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:40.905 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:50:40.905 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:40.905 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.905 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.905 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:40.905 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:50:40.905 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:40.905 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.905 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.905 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:40.905 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481319+0000", 2024-09-16T20:50:40.906 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.53e-07 2024-09-16T20:50:40.906 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.906 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.906 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:40.906 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481320+0000", 2024-09-16T20:50:40.906 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5230000000000001e-06 2024-09-16T20:50:40.906 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.906 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.906 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:40.906 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481330+0000", 2024-09-16T20:50:40.906 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0190000000000001e-05 2024-09-16T20:50:40.906 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.906 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.906 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:40.907 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481333+0000", 2024-09-16T20:50:40.907 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.151e-06 2024-09-16T20:50:40.907 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.907 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.907 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:40.907 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527689+0000", 2024-09-16T20:50:40.907 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046355192000000003 2024-09-16T20:50:40.907 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.907 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.907 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:40.907 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527764+0000", 2024-09-16T20:50:40.907 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5776000000000001e-05 2024-09-16T20:50:40.907 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.907 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.907 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:40.908 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528141+0000", 2024-09-16T20:50:40.908 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00037647599999999999 2024-09-16T20:50:40.908 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.908 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.908 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:40.908 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531788+0000", 2024-09-16T20:50:40.908 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0036470679999999998 2024-09-16T20:50:40.908 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.908 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.908 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:40.908 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535174+0000", 2024-09-16T20:50:40.908 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0033862670000000001 2024-09-16T20:50:40.908 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.908 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.909 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:40.909 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535236+0000", 2024-09-16T20:50:40.909 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1797999999999993e-05 2024-09-16T20:50:40.909 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.909 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.909 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:40.909 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535274+0000", 2024-09-16T20:50:40.909 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7741999999999999e-05 2024-09-16T20:50:40.909 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:40.909 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:40.909 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:40.909 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.909 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.909 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3577 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:40.910 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:50:40.910 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 111.40169505199999, 2024-09-16T20:50:40.910 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054073807000000002, 2024-09-16T20:50:40.910 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:40.910 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:40.910 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:40.910 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:40.910 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:40.910 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:40.910 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3577 2024-09-16T20:50:40.910 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.910 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:40.910 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.910 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:40.911 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:50:40.911 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:40.911 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.911 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.911 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:40.911 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:50:40.911 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:40.911 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.911 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.911 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:40.911 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:50:40.911 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.96e-07 2024-09-16T20:50:40.912 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.912 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.912 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:40.912 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481342+0000", 2024-09-16T20:50:40.912 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.714e-06 2024-09-16T20:50:40.912 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.912 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.912 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:40.912 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481349+0000", 2024-09-16T20:50:40.912 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4930000000000001e-06 2024-09-16T20:50:40.912 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.912 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.912 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:40.912 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481353+0000", 2024-09-16T20:50:40.913 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4879999999999999e-06 2024-09-16T20:50:40.913 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.913 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.913 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:40.913 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528619+0000", 2024-09-16T20:50:40.913 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047266351999999998 2024-09-16T20:50:40.913 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.913 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.913 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:40.913 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528706+0000", 2024-09-16T20:50:40.913 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6796e-05 2024-09-16T20:50:40.913 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.913 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.914 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:40.914 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529148+0000", 2024-09-16T20:50:40.914 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00044182999999999999 2024-09-16T20:50:40.914 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.914 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.914 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:40.914 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531792+0000", 2024-09-16T20:50:40.914 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0026443119999999998 2024-09-16T20:50:40.914 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.914 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.914 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:40.914 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535319+0000", 2024-09-16T20:50:40.914 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0035266439999999998 2024-09-16T20:50:40.914 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.914 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.914 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:40.915 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535372+0000", 2024-09-16T20:50:40.915 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3207000000000001e-05 2024-09-16T20:50:40.915 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.915 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.915 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:40.915 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535414+0000", 2024-09-16T20:50:40.915 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1675000000000003e-05 2024-09-16T20:50:40.915 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:40.915 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:40.915 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:40.915 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.915 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.915 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3578 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:40.915 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:50:40.915 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 111.401673883, 2024-09-16T20:50:40.916 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054189586999999997, 2024-09-16T20:50:40.916 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:40.916 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:40.916 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:40.916 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:40.916 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:40.916 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:40.916 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3578 2024-09-16T20:50:40.916 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.916 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:40.916 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.916 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:40.916 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:50:40.916 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:40.916 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.916 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.917 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:40.917 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:50:40.917 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:40.917 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.917 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.917 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:40.917 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:50:40.917 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0800000000000001e-07 2024-09-16T20:50:40.917 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.917 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.917 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:40.917 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481363+0000", 2024-09-16T20:50:40.918 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0339999999999999e-06 2024-09-16T20:50:40.918 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.918 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.918 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:40.918 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481371+0000", 2024-09-16T20:50:40.918 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4440000000000002e-06 2024-09-16T20:50:40.918 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.918 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.918 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:40.918 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481374+0000", 2024-09-16T20:50:40.918 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2049999999999998e-06 2024-09-16T20:50:40.918 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.918 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.918 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:40.919 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529555+0000", 2024-09-16T20:50:40.919 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048181202999999999 2024-09-16T20:50:40.919 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.919 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.919 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:40.919 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529616+0000", 2024-09-16T20:50:40.919 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0912000000000002e-05 2024-09-16T20:50:40.919 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.919 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.919 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:40.919 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529939+0000", 2024-09-16T20:50:40.919 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00032269800000000001 2024-09-16T20:50:40.919 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.919 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.919 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:40.920 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532602+0000", 2024-09-16T20:50:40.920 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0026634940000000002 2024-09-16T20:50:40.920 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.920 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.920 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:40.920 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535465+0000", 2024-09-16T20:50:40.920 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.002862149 2024-09-16T20:50:40.920 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.920 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.920 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:40.920 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535516+0000", 2024-09-16T20:50:40.920 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1696000000000001e-05 2024-09-16T20:50:40.920 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.920 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.920 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:40.921 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535551+0000", 2024-09-16T20:50:40.921 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4443999999999998e-05 2024-09-16T20:50:40.921 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:40.921 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:40.921 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:40.921 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.921 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.921 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3579 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:40.921 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:50:40.921 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 111.401655444, 2024-09-16T20:50:40.921 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054295151, 2024-09-16T20:50:40.921 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:40.921 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:40.921 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:40.922 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:40.922 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:40.922 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:40.922 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3579 2024-09-16T20:50:40.922 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.922 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:40.922 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.922 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:40.922 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:50:40.922 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:40.922 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.922 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.922 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:40.922 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:50:40.922 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:40.923 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.923 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.923 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:40.923 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:50:40.923 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3300000000000001e-07 2024-09-16T20:50:40.923 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.923 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.923 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:40.923 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481381+0000", 2024-09-16T20:50:40.923 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.649e-06 2024-09-16T20:50:40.923 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.923 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.923 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:40.923 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481387+0000", 2024-09-16T20:50:40.923 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4519999999999996e-06 2024-09-16T20:50:40.924 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.924 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.924 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:40.924 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481390+0000", 2024-09-16T20:50:40.924 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0170000000000001e-06 2024-09-16T20:50:40.924 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.924 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.924 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:40.924 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530756+0000", 2024-09-16T20:50:40.924 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049365882999999999 2024-09-16T20:50:40.924 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.924 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.924 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:40.924 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530845+0000", 2024-09-16T20:50:40.924 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9197000000000007e-05 2024-09-16T20:50:40.924 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.925 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.925 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:40.925 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531191+0000", 2024-09-16T20:50:40.925 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00034562500000000001 2024-09-16T20:50:40.925 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.925 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.925 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:40.925 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534078+0000", 2024-09-16T20:50:40.925 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0028878179999999999 2024-09-16T20:50:40.925 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.925 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.925 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:40.925 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535589+0000", 2024-09-16T20:50:40.925 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00151085 2024-09-16T20:50:40.926 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.926 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.926 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:40.926 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535639+0000", 2024-09-16T20:50:40.926 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9697000000000002e-05 2024-09-16T20:50:40.926 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.926 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.926 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:40.926 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535675+0000", 2024-09-16T20:50:40.926 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5729999999999998e-05 2024-09-16T20:50:40.926 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:40.926 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:40.926 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:40.926 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.926 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.927 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3580 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:40.927 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:50:40.927 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 111.40164181199999, 2024-09-16T20:50:40.927 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054397361999999998, 2024-09-16T20:50:40.927 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:40.927 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:40.927 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:40.927 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:40.927 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:40.927 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:40.927 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3580 2024-09-16T20:50:40.927 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.927 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:40.927 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.927 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:40.928 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:50:40.928 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:40.928 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.928 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.928 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:40.928 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:50:40.928 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:40.928 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.928 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.928 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:40.928 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:50:40.928 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2600000000000001e-07 2024-09-16T20:50:40.928 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.928 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.930 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:40.930 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481395+0000", 2024-09-16T20:50:40.930 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.544e-06 2024-09-16T20:50:40.930 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.930 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.930 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:40.930 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481404+0000", 2024-09-16T20:50:40.930 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8850000000000005e-06 2024-09-16T20:50:40.930 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.930 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.931 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:40.931 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481407+0000", 2024-09-16T20:50:40.931 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1540000000000002e-06 2024-09-16T20:50:40.931 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.931 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.931 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:40.931 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531823+0000", 2024-09-16T20:50:40.931 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050415637999999999 2024-09-16T20:50:40.931 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.931 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.931 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:40.931 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531895+0000", 2024-09-16T20:50:40.931 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2179000000000001e-05 2024-09-16T20:50:40.931 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.932 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.932 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:40.932 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532236+0000", 2024-09-16T20:50:40.932 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00034073799999999999 2024-09-16T20:50:40.932 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.932 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.932 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:40.932 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534083+0000", 2024-09-16T20:50:40.932 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001847298 2024-09-16T20:50:40.932 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.932 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.932 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:40.932 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535714+0000", 2024-09-16T20:50:40.933 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0016316810000000001 2024-09-16T20:50:40.933 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.933 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.933 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:40.933 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535759+0000", 2024-09-16T20:50:40.933 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4478999999999999e-05 2024-09-16T20:50:40.933 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.933 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.933 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:40.933 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535791+0000", 2024-09-16T20:50:40.933 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.154e-05 2024-09-16T20:50:40.933 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:40.933 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:40.933 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:40.934 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.934 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.934 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3581 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:40.934 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:50:40.934 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 111.401619815, 2024-09-16T20:50:40.934 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.057094403000000002, 2024-09-16T20:50:40.934 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:40.934 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:40.934 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:40.934 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:40.934 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:40.934 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:40.934 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3581 2024-09-16T20:50:40.934 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.934 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:40.935 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.935 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:40.935 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:50:40.935 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:40.935 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.935 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.935 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:40.935 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:50:40.935 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:40.935 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.935 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.935 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:40.935 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:50:40.936 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8799999999999998e-07 2024-09-16T20:50:40.936 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.936 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.936 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:40.936 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481418+0000", 2024-09-16T20:50:40.936 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1859999999999999e-06 2024-09-16T20:50:40.936 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.936 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.936 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:40.936 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481423+0000", 2024-09-16T20:50:40.936 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8279999999999999e-06 2024-09-16T20:50:40.936 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.936 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.937 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:40.937 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481426+0000", 2024-09-16T20:50:40.937 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0110000000000001e-06 2024-09-16T20:50:40.937 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.937 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.937 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:40.937 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532631+0000", 2024-09-16T20:50:40.937 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051204514999999999 2024-09-16T20:50:40.937 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.937 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.937 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:40.937 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.533381+0000", 2024-09-16T20:50:40.937 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00075029000000000003 2024-09-16T20:50:40.937 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.938 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.938 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:40.938 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.533739+0000", 2024-09-16T20:50:40.938 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00035773100000000002 2024-09-16T20:50:40.938 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.938 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.938 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:40.938 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535123+0000", 2024-09-16T20:50:40.938 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0013836160000000001 2024-09-16T20:50:40.938 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.938 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.938 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:40.938 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538413+0000", 2024-09-16T20:50:40.939 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0032902510000000001 2024-09-16T20:50:40.939 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.939 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.939 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:40.939 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538470+0000", 2024-09-16T20:50:40.939 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6697999999999998e-05 2024-09-16T20:50:40.939 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.939 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.939 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:40.939 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538510+0000", 2024-09-16T20:50:40.939 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9988999999999999e-05 2024-09-16T20:50:40.939 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:40.939 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:40.939 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:40.939 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.940 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.940 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:12732 3.d 3:b752444a:::rbd_header.110e747f0779:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-16T20:50:40.940 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:50:40.940 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 96.114547236999996, 2024-09-16T20:50:40.940 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.065302467000000003, 2024-09-16T20:50:40.940 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:40.940 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:40.940 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:40.940 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:40.940 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:50:40.940 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:50:40.940 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 12732 2024-09-16T20:50:40.940 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.941 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:40.941 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.941 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:40.941 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:50:40.941 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:40.941 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.941 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.941 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:40.941 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:50:40.941 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:40.941 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.941 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.941 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:40.941 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768490+0000", 2024-09-16T20:50:40.942 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2859999999999998e-06 2024-09-16T20:50:40.942 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.942 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.942 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:40.942 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768501+0000", 2024-09-16T20:50:40.942 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1226999999999999e-05 2024-09-16T20:50:40.942 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.942 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.942 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:40.942 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768517+0000", 2024-09-16T20:50:40.942 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.592e-05 2024-09-16T20:50:40.942 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.942 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.942 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:40.943 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768528+0000", 2024-09-16T20:50:40.943 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0689999999999999e-05 2024-09-16T20:50:40.943 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.943 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.943 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:40.943 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768584+0000", 2024-09-16T20:50:40.943 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5889e-05 2024-09-16T20:50:40.943 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.943 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.943 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:40.943 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768720+0000", 2024-09-16T20:50:40.943 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000136718 2024-09-16T20:50:40.943 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.943 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.944 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:40.944 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.769066+0000", 2024-09-16T20:50:40.944 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000346013 2024-09-16T20:50:40.944 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.944 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.944 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:40.944 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833659+0000", 2024-09-16T20:50:40.944 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.064592822999999994 2024-09-16T20:50:40.944 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.944 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.944 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:40.944 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833719+0000", 2024-09-16T20:50:40.944 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9441000000000002e-05 2024-09-16T20:50:40.944 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.945 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.945 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:40.945 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833758+0000", 2024-09-16T20:50:40.945 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9122000000000003e-05 2024-09-16T20:50:40.945 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.945 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.945 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:40.945 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833790+0000", 2024-09-16T20:50:40.945 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2338000000000001e-05 2024-09-16T20:50:40.945 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:40.945 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:40.945 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:40.945 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.945 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.946 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.5089.0:12732 3.d e73/23 ondisk, result = 0)", 2024-09-16T20:50:40.946 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:50:40.946 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 96.112945066999998, 2024-09-16T20:50:40.946 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.063706456999999994, 2024-09-16T20:50:40.946 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:40.946 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:40.946 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:40.946 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:40.946 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.946 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:40.946 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:50:40.946 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:40.946 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.947 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.947 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:40.947 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:50:40.947 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:40.947 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.947 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.947 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:40.947 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770091+0000", 2024-09-16T20:50:40.947 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.53e-06 2024-09-16T20:50:40.948 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.948 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.948 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:40.948 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770104+0000", 2024-09-16T20:50:40.948 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2203e-05 2024-09-16T20:50:40.948 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.948 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.948 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:40.948 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833637+0000", 2024-09-16T20:50:40.948 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.063533201999999997 2024-09-16T20:50:40.948 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.948 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.948 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:40.948 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833655+0000", 2024-09-16T20:50:40.949 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8369e-05 2024-09-16T20:50:40.949 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.949 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.949 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:40.949 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833707+0000", 2024-09-16T20:50:40.949 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2061999999999997e-05 2024-09-16T20:50:40.949 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.949 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.949 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:40.949 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833715+0000", 2024-09-16T20:50:40.949 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5179999999999998e-06 2024-09-16T20:50:40.949 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.949 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.949 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:40.950 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833796+0000", 2024-09-16T20:50:40.950 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1573e-05 2024-09-16T20:50:40.950 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:40.950 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:40.950 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:40.950 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.950 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.950 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3285 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 240640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:40.950 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041140+0000", 2024-09-16T20:50:40.950 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 50.841895289, 2024-09-16T20:50:40.950 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052375646999999997, 2024-09-16T20:50:40.950 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:40.950 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:40.950 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:40.951 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:40.951 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:40.951 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:40.951 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3285 2024-09-16T20:50:40.951 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.951 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:40.951 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.951 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:40.951 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041140+0000", 2024-09-16T20:50:40.951 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:40.951 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.951 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.951 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:40.951 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041140+0000", 2024-09-16T20:50:40.952 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:40.952 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.952 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.952 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:40.952 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041140+0000", 2024-09-16T20:50:40.952 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2399999999999999e-07 2024-09-16T20:50:40.952 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.952 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.952 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:40.952 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041141+0000", 2024-09-16T20:50:40.952 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2759999999999999e-06 2024-09-16T20:50:40.952 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.952 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.952 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:40.953 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041144+0000", 2024-09-16T20:50:40.953 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0280000000000001e-06 2024-09-16T20:50:40.953 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.953 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.953 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:40.953 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041147+0000", 2024-09-16T20:50:40.953 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5900000000000002e-06 2024-09-16T20:50:40.953 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.953 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.953 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:40.953 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093284+0000", 2024-09-16T20:50:40.953 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052137645000000003 2024-09-16T20:50:40.953 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.953 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.954 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:40.954 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093365+0000", 2024-09-16T20:50:40.954 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0446999999999997e-05 2024-09-16T20:50:40.954 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.954 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.954 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:40.954 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093515+0000", 2024-09-16T20:50:40.954 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00015043700000000001 2024-09-16T20:50:40.954 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:40.954 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:40.954 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:40.954 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.954 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.954 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3286 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 245760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:40.955 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:50:40.955 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 50.841883129999999, 2024-09-16T20:50:40.955 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052523564000000002, 2024-09-16T20:50:40.955 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:40.955 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:40.955 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:40.955 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:40.955 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:40.955 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:40.955 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3286 2024-09-16T20:50:40.955 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.955 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:40.955 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.955 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:40.956 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:50:40.956 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:40.956 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.956 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.956 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:40.956 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:50:40.956 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:40.956 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.956 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.956 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:40.956 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:50:40.956 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3900000000000001e-07 2024-09-16T20:50:40.956 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.957 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.957 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:40.957 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041153+0000", 2024-09-16T20:50:40.957 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.119e-06 2024-09-16T20:50:40.957 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.957 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.957 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:40.957 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041156+0000", 2024-09-16T20:50:40.957 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0000000000000001e-06 2024-09-16T20:50:40.957 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.957 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.957 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:40.957 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041159+0000", 2024-09-16T20:50:40.957 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5859999999999999e-06 2024-09-16T20:50:40.957 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.958 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.958 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:40.958 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093529+0000", 2024-09-16T20:50:40.958 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052370591000000001 2024-09-16T20:50:40.958 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.958 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.958 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:40.958 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093570+0000", 2024-09-16T20:50:40.958 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0161999999999999e-05 2024-09-16T20:50:40.958 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.958 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.958 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:40.958 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093675+0000", 2024-09-16T20:50:40.959 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010586699999999999 2024-09-16T20:50:40.959 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:40.959 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:40.959 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:40.959 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.959 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.959 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3287 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 250880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:40.959 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:50:40.959 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 50.841873081000003, 2024-09-16T20:50:40.959 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052781203999999998, 2024-09-16T20:50:40.959 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:40.959 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:40.959 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:40.960 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:40.960 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:40.960 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:40.960 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3287 2024-09-16T20:50:40.960 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.960 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:40.960 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.960 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:40.960 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:50:40.960 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:40.960 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.960 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.960 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:40.961 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:50:40.961 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:40.961 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.961 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.961 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:40.961 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:50:40.961 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1e-07 2024-09-16T20:50:40.961 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.961 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.961 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:40.961 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041163+0000", 2024-09-16T20:50:40.961 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0690000000000001e-06 2024-09-16T20:50:40.961 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.961 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.962 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:40.962 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041166+0000", 2024-09-16T20:50:40.962 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.982e-06 2024-09-16T20:50:40.962 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.962 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.962 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:40.962 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041169+0000", 2024-09-16T20:50:40.962 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.593e-06 2024-09-16T20:50:40.962 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.962 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.962 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:40.962 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093709+0000", 2024-09-16T20:50:40.962 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052540492000000001 2024-09-16T20:50:40.963 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.963 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.963 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:40.963 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093792+0000", 2024-09-16T20:50:40.963 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2436999999999997e-05 2024-09-16T20:50:40.963 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.963 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.963 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:40.963 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093943+0000", 2024-09-16T20:50:40.963 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000151421 2024-09-16T20:50:40.963 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:40.964 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:40.964 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:40.964 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.964 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.964 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3288 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 256000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:40.964 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:50:40.964 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 50.841863207999999, 2024-09-16T20:50:40.964 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052928269, 2024-09-16T20:50:40.964 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:40.964 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:40.964 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:40.964 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:40.964 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:40.965 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:40.965 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3288 2024-09-16T20:50:40.965 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.965 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:40.965 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.965 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:40.965 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:50:40.965 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:40.965 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.965 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.965 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:40.965 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:50:40.965 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:40.965 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.965 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.966 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:40.966 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:50:40.966 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2700000000000001e-07 2024-09-16T20:50:40.966 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.966 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.966 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:40.966 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041173+0000", 2024-09-16T20:50:40.966 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.113e-06 2024-09-16T20:50:40.966 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.966 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.966 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:40.966 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041176+0000", 2024-09-16T20:50:40.966 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9799999999999998e-06 2024-09-16T20:50:40.966 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.967 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.967 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:40.967 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041179+0000", 2024-09-16T20:50:40.967 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.667e-06 2024-09-16T20:50:40.967 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.967 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.967 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:40.967 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093957+0000", 2024-09-16T20:50:40.967 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052778703000000003 2024-09-16T20:50:40.967 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.967 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.967 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:40.967 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093999+0000", 2024-09-16T20:50:40.967 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1300000000000001e-05 2024-09-16T20:50:40.968 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.968 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.968 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:40.968 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094100+0000", 2024-09-16T20:50:40.968 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101279 2024-09-16T20:50:40.968 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:40.968 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:40.968 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:40.968 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.968 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.968 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3289 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 261120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:40.968 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:50:40.968 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 50.841853211999997, 2024-09-16T20:50:40.968 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053182976, 2024-09-16T20:50:40.969 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:40.969 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:40.969 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:40.969 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:40.969 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:40.969 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:40.969 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3289 2024-09-16T20:50:40.969 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.969 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:40.969 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.969 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:40.969 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:50:40.969 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:40.970 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.970 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.970 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:40.970 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:50:40.970 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:40.970 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.970 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.970 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:40.970 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:50:40.970 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2999999999999999e-07 2024-09-16T20:50:40.970 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.970 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.970 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:40.970 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041183+0000", 2024-09-16T20:50:40.970 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0720000000000001e-06 2024-09-16T20:50:40.971 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.971 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.971 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:40.971 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041186+0000", 2024-09-16T20:50:40.971 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9869999999999999e-06 2024-09-16T20:50:40.971 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.971 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.971 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:40.971 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041189+0000", 2024-09-16T20:50:40.971 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.587e-06 2024-09-16T20:50:40.971 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.971 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.971 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:40.972 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094136+0000", 2024-09-16T20:50:40.972 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052947270999999997 2024-09-16T20:50:40.972 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.972 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.972 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:40.972 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094213+0000", 2024-09-16T20:50:40.972 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7552000000000006e-05 2024-09-16T20:50:40.972 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.972 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.972 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:40.972 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094365+0000", 2024-09-16T20:50:40.972 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00015127700000000001 2024-09-16T20:50:40.972 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:40.972 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:40.973 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:40.973 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.973 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.973 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:2196 3.d 3:b1fe01a0:::rbd_data.110e7ef3004.0000000000000000:2d [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e136)", 2024-09-16T20:50:40.973 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:22.739057+0000", 2024-09-16T20:50:40.973 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 18.143978412999999, 2024-09-16T20:50:40.973 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11999464899999999, 2024-09-16T20:50:40.973 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:40.973 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:40.973 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:40.973 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:40.973 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:50:40.973 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:40.973 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2196 2024-09-16T20:50:40.974 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.974 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:40.974 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.974 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:40.974 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.739057+0000", 2024-09-16T20:50:40.974 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:40.974 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.974 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.974 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:40.974 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.739057+0000", 2024-09-16T20:50:40.974 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:40.974 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.974 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.974 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:40.975 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.739057+0000", 2024-09-16T20:50:40.975 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1399999999999997e-07 2024-09-16T20:50:40.975 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.975 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.975 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:40.975 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.739060+0000", 2024-09-16T20:50:40.975 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6019999999999998e-06 2024-09-16T20:50:40.975 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.975 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.975 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:40.975 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.858755+0000", 2024-09-16T20:50:40.975 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.119695151 2024-09-16T20:50:40.975 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.976 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.976 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:40.976 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.858778+0000", 2024-09-16T20:50:40.976 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3184999999999999e-05 2024-09-16T20:50:40.976 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.976 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.976 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:40.976 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.858865+0000", 2024-09-16T20:50:40.976 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.7330000000000006e-05 2024-09-16T20:50:40.976 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.976 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.976 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:40.976 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.858952+0000", 2024-09-16T20:50:40.977 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6803000000000002e-05 2024-09-16T20:50:40.977 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.977 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.977 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:40.977 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.859051+0000", 2024-09-16T20:50:40.977 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8963999999999996e-05 2024-09-16T20:50:40.977 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:40.977 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:40.977 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:40.977 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.977 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.977 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6061.0:5401 3.13 3:c8554348:::rbd_data.110e16eafbd.0000000000000011:head [read 577536~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-16T20:50:40.977 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:27.761536+0000", 2024-09-16T20:50:40.978 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 13.121498600000001, 2024-09-16T20:50:40.978 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.003742712, 2024-09-16T20:50:40.978 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:40.978 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:40.978 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:50:40.978 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:40.978 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6061", 2024-09-16T20:50:40.978 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3930128440", 2024-09-16T20:50:40.978 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5401 2024-09-16T20:50:40.978 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.978 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:40.978 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.978 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:40.978 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761536+0000", 2024-09-16T20:50:40.979 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:40.979 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.979 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.979 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:40.979 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761536+0000", 2024-09-16T20:50:40.979 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:40.979 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.979 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.979 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:40.979 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761537+0000", 2024-09-16T20:50:40.979 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2900000000000003e-07 2024-09-16T20:50:40.979 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.979 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.979 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:40.980 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761540+0000", 2024-09-16T20:50:40.980 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2320000000000001e-06 2024-09-16T20:50:40.980 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.980 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.980 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:40.980 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761548+0000", 2024-09-16T20:50:40.980 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1289999999999996e-06 2024-09-16T20:50:40.980 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.980 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.980 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:40.980 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761554+0000", 2024-09-16T20:50:40.980 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0820000000000004e-06 2024-09-16T20:50:40.980 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.980 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.980 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:40.981 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761621+0000", 2024-09-16T20:50:40.981 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6266000000000006e-05 2024-09-16T20:50:40.981 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.981 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.981 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:40.981 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761626+0000", 2024-09-16T20:50:40.981 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.614e-06 2024-09-16T20:50:40.981 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.981 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.981 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:40.981 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:28.765163+0000", 2024-09-16T20:50:40.981 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.003536416 2024-09-16T20:50:40.981 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.981 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.982 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:40.982 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:28.765279+0000", 2024-09-16T20:50:40.982 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000116344 2024-09-16T20:50:40.982 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:40.982 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:40.982 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:40.982 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.982 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.982 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6061.0:5425 3.13 3:c8554348:::rbd_data.110e16eafbd.0000000000000011:head [read 602112~28672] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-16T20:50:40.982 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:27.767049+0000", 2024-09-16T20:50:40.982 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 13.115985941, 2024-09-16T20:50:40.982 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.99847897900000004, 2024-09-16T20:50:40.982 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:40.982 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:40.983 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:50:40.983 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:40.983 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6061", 2024-09-16T20:50:40.983 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3930128440", 2024-09-16T20:50:40.983 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5425 2024-09-16T20:50:40.983 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.983 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:40.983 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.983 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:40.983 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767049+0000", 2024-09-16T20:50:40.983 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:40.983 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.983 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.983 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:40.984 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767049+0000", 2024-09-16T20:50:40.984 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:40.984 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.984 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.984 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:40.984 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767050+0000", 2024-09-16T20:50:40.984 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6499999999999999e-07 2024-09-16T20:50:40.984 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.984 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.984 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:40.984 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767052+0000", 2024-09-16T20:50:40.984 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7350000000000001e-06 2024-09-16T20:50:40.984 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.984 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.984 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:40.985 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767060+0000", 2024-09-16T20:50:40.985 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5109999999999997e-06 2024-09-16T20:50:40.985 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.985 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.985 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:40.985 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767066+0000", 2024-09-16T20:50:40.985 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8139999999999997e-06 2024-09-16T20:50:40.985 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.985 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.985 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:40.985 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767128+0000", 2024-09-16T20:50:40.985 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2174999999999993e-05 2024-09-16T20:50:40.985 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.985 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.986 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:40.986 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767133+0000", 2024-09-16T20:50:40.986 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2140000000000004e-06 2024-09-16T20:50:40.986 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.986 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.986 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:40.986 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:28.765441+0000", 2024-09-16T20:50:40.986 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.99830808599999998 2024-09-16T20:50:40.986 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.986 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.986 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:40.986 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:28.765528+0000", 2024-09-16T20:50:40.986 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6879000000000006e-05 2024-09-16T20:50:40.986 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:40.986 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:40.987 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:40.987 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.987 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.987 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:25585 3.10 3:0c7c5b23:::rbd_header.110ebf0678ad:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:40.987 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:32.877132+0000", 2024-09-16T20:50:40.987 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 8.0059033920000005, 2024-09-16T20:50:40.987 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054096699999999998, 2024-09-16T20:50:40.987 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:40.987 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:40.987 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:40.987 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:40.987 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:50:40.987 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:40.987 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 25585 2024-09-16T20:50:40.987 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.988 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:40.988 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.988 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:40.988 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.877132+0000", 2024-09-16T20:50:40.988 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:40.988 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.988 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.988 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:40.988 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.877132+0000", 2024-09-16T20:50:40.988 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:40.988 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.988 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.988 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:40.989 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931072+0000", 2024-09-16T20:50:40.989 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053939931000000003 2024-09-16T20:50:40.989 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.989 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.989 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:40.989 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931076+0000", 2024-09-16T20:50:40.989 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2860000000000001e-06 2024-09-16T20:50:40.989 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.989 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.989 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:40.989 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931085+0000", 2024-09-16T20:50:40.989 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.3370000000000006e-06 2024-09-16T20:50:40.990 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.990 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.990 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:40.990 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931092+0000", 2024-09-16T20:50:40.990 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9970000000000001e-06 2024-09-16T20:50:40.990 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.990 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.990 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:40.990 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931137+0000", 2024-09-16T20:50:40.990 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5160000000000001e-05 2024-09-16T20:50:40.990 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.990 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.990 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:40.990 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931178+0000", 2024-09-16T20:50:40.991 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0379000000000001e-05 2024-09-16T20:50:40.991 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:40.991 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:40.991 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:40.991 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931228+0000", 2024-09-16T20:50:40.991 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0609999999999998e-05 2024-09-16T20:50:40.991 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:40.991 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:40.991 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:40.991 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:40.991 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:40.991 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:40.993 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:50:40.993 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:40.992+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:41.093 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:41.093+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:41.140 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:41.140 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:50:41.140 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:50:41.140 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:41.149 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:50:41.195 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:41.194+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:41.255 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:41.256 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:50:41.256 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:50:41.256 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:50:41.256 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:41.264 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:50:41.296 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:41.295+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:41.372 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:41.372 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:50:41.372 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:50:41.372 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:50:41.372 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.372 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3714 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1480704~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:41.373 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:50:41.373 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 138.731795831, 2024-09-16T20:50:41.373 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050152552000000003, 2024-09-16T20:50:41.373 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:41.373 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:41.373 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:41.373 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:41.373 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:41.373 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:41.373 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3714 2024-09-16T20:50:41.373 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.373 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:41.373 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.374 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:41.374 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:50:41.374 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:41.374 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.374 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.374 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:41.374 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:50:41.374 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:41.374 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.374 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.374 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:41.374 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:50:41.374 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0499999999999999e-07 2024-09-16T20:50:41.375 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.375 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.375 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:41.375 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639007+0000", 2024-09-16T20:50:41.375 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5340000000000001e-06 2024-09-16T20:50:41.375 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.375 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.375 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:41.375 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639013+0000", 2024-09-16T20:50:41.375 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3260000000000003e-06 2024-09-16T20:50:41.375 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.375 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.375 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:41.375 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639018+0000", 2024-09-16T20:50:41.376 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2850000000000002e-06 2024-09-16T20:50:41.376 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.376 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.376 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:41.376 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689026+0000", 2024-09-16T20:50:41.376 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.05000798 2024-09-16T20:50:41.376 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.376 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.376 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:41.376 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689077+0000", 2024-09-16T20:50:41.376 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1063999999999998e-05 2024-09-16T20:50:41.376 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.377 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.377 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:41.377 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689158+0000", 2024-09-16T20:50:41.377 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1057999999999995e-05 2024-09-16T20:50:41.377 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:41.377 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:41.377 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:41.377 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.377 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.377 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3716 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1488896~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:41.377 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:50:41.377 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 138.731778021, 2024-09-16T20:50:41.377 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050280142, 2024-09-16T20:50:41.377 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:41.378 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:41.378 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:41.378 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:41.378 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:41.378 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:41.378 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3716 2024-09-16T20:50:41.378 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.378 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:41.378 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.378 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:41.378 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:50:41.378 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:41.378 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.378 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.379 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:41.379 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:50:41.379 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:41.379 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.379 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.379 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:41.379 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639024+0000", 2024-09-16T20:50:41.379 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9999999999999997e-07 2024-09-16T20:50:41.379 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.379 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.379 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:41.379 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639026+0000", 2024-09-16T20:50:41.379 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.781e-06 2024-09-16T20:50:41.380 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.380 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.380 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:41.380 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639032+0000", 2024-09-16T20:50:41.380 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7780000000000004e-06 2024-09-16T20:50:41.380 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.380 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.380 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:41.380 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639036+0000", 2024-09-16T20:50:41.380 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6909999999999996e-06 2024-09-16T20:50:41.380 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.380 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.380 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:41.380 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689191+0000", 2024-09-16T20:50:41.380 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050155284000000001 2024-09-16T20:50:41.381 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.381 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.381 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:41.381 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689242+0000", 2024-09-16T20:50:41.381 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0662999999999999e-05 2024-09-16T20:50:41.381 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.381 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.381 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:41.381 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689304+0000", 2024-09-16T20:50:41.381 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1345000000000003e-05 2024-09-16T20:50:41.381 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:41.381 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:41.381 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:41.381 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.382 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.382 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3719 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1497088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:41.382 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:50:41.382 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 138.73175074299999, 2024-09-16T20:50:41.382 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050399715999999997, 2024-09-16T20:50:41.382 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:41.382 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:41.382 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:41.382 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:41.382 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:41.382 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:41.382 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3719 2024-09-16T20:50:41.382 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.382 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:41.382 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.383 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:41.383 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:50:41.383 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:41.383 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.383 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.383 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:41.383 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:50:41.383 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:41.383 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.383 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.383 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:41.383 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:50:41.383 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9700000000000002e-07 2024-09-16T20:50:41.384 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.384 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.384 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:41.384 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639053+0000", 2024-09-16T20:50:41.384 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2900000000000001e-06 2024-09-16T20:50:41.384 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.384 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.384 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:41.384 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639060+0000", 2024-09-16T20:50:41.384 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7750000000000004e-06 2024-09-16T20:50:41.384 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.384 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.384 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:41.385 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639066+0000", 2024-09-16T20:50:41.385 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0639999999999999e-06 2024-09-16T20:50:41.385 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.385 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.385 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:41.385 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689337+0000", 2024-09-16T20:50:41.385 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050270784999999998 2024-09-16T20:50:41.385 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.385 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.385 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:41.385 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689392+0000", 2024-09-16T20:50:41.385 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4920999999999998e-05 2024-09-16T20:50:41.385 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.385 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.385 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:41.386 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689450+0000", 2024-09-16T20:50:41.386 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8483999999999998e-05 2024-09-16T20:50:41.386 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:41.386 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:41.386 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:41.386 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.386 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.386 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3721 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1505280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:41.386 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:50:41.386 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 138.731727611, 2024-09-16T20:50:41.386 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050513608000000002, 2024-09-16T20:50:41.386 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:41.386 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:41.386 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:41.386 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:41.387 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:41.387 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:41.387 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3721 2024-09-16T20:50:41.387 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.387 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:41.387 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.387 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:41.387 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:50:41.387 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:41.387 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.387 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.387 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:41.387 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:50:41.387 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:41.388 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.388 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.388 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:41.388 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:50:41.388 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7e-07 2024-09-16T20:50:41.388 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.388 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.388 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:41.388 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639076+0000", 2024-09-16T20:50:41.388 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8899999999999999e-06 2024-09-16T20:50:41.388 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.388 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.388 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:41.389 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639083+0000", 2024-09-16T20:50:41.389 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1210000000000001e-06 2024-09-16T20:50:41.389 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.389 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.389 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:41.389 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639089+0000", 2024-09-16T20:50:41.389 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.643e-06 2024-09-16T20:50:41.389 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.389 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.389 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:41.389 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689474+0000", 2024-09-16T20:50:41.389 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050385679000000003 2024-09-16T20:50:41.389 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.389 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.390 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:41.390 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689533+0000", 2024-09-16T20:50:41.390 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8303000000000001e-05 2024-09-16T20:50:41.390 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.390 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.390 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:41.390 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689587+0000", 2024-09-16T20:50:41.390 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4601999999999997e-05 2024-09-16T20:50:41.390 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:41.390 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:41.390 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:41.390 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.390 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.391 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3723 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1513472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:41.391 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:50:41.391 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 138.73169680699999, 2024-09-16T20:50:41.391 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050699691999999998, 2024-09-16T20:50:41.391 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:41.391 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:41.391 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:41.391 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:41.391 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:41.391 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:41.391 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3723 2024-09-16T20:50:41.391 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.391 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:41.391 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.392 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:41.392 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:50:41.392 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:41.392 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.392 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.392 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:41.392 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:50:41.392 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:41.392 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.392 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.392 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:41.392 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:50:41.392 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3399999999999999e-07 2024-09-16T20:50:41.392 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.392 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.393 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:41.393 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639107+0000", 2024-09-16T20:50:41.393 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3860000000000001e-06 2024-09-16T20:50:41.393 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.393 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.393 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:41.393 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639114+0000", 2024-09-16T20:50:41.393 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9800000000000001e-06 2024-09-16T20:50:41.393 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.393 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.393 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:41.393 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639129+0000", 2024-09-16T20:50:41.393 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4887e-05 2024-09-16T20:50:41.393 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.394 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.394 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:41.394 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689625+0000", 2024-09-16T20:50:41.394 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050495848000000003 2024-09-16T20:50:41.394 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.394 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.394 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:41.394 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689720+0000", 2024-09-16T20:50:41.394 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4821000000000006e-05 2024-09-16T20:50:41.394 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.394 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.394 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:41.394 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689804+0000", 2024-09-16T20:50:41.395 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4235999999999998e-05 2024-09-16T20:50:41.395 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:41.395 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:41.395 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:41.395 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.395 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.395 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3725 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1521664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:41.395 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:50:41.395 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 138.73166321100001, 2024-09-16T20:50:41.395 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050797660000000001, 2024-09-16T20:50:41.396 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:41.396 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:41.396 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:41.396 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:41.396 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:41.396 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:41.396 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3725 2024-09-16T20:50:41.396 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.396 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:41.396 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.396 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:41.396 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:50:41.396 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:41.396 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.397 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.397 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:41.397 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:50:41.397 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:41.397 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.397 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.397 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:41.397 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639139+0000", 2024-09-16T20:50:41.397 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8999999999999997e-07 2024-09-16T20:50:41.397 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.397 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.397 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:41.397 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639141+0000", 2024-09-16T20:50:41.397 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.182e-06 2024-09-16T20:50:41.397 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.398 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.398 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:41.398 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639154+0000", 2024-09-16T20:50:41.398 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3626e-05 2024-09-16T20:50:41.398 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.398 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.398 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:41.398 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639160+0000", 2024-09-16T20:50:41.398 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6810000000000003e-06 2024-09-16T20:50:41.398 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.398 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.398 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:41.399 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689831+0000", 2024-09-16T20:50:41.399 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050670907000000001 2024-09-16T20:50:41.399 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.399 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.399 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:41.399 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689882+0000", 2024-09-16T20:50:41.399 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0445999999999998e-05 2024-09-16T20:50:41.399 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.399 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.399 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:41.399 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689936+0000", 2024-09-16T20:50:41.399 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4327999999999999e-05 2024-09-16T20:50:41.399 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:41.400 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:41.400 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:41.400 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.400 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.400 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3727 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1529856~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:41.400 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:50:41.400 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 138.73162763400001, 2024-09-16T20:50:41.400 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051016922999999999, 2024-09-16T20:50:41.400 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:41.400 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:41.400 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:41.400 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:41.400 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:41.400 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:41.401 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3727 2024-09-16T20:50:41.401 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.401 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:41.401 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.401 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:41.401 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:50:41.401 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:41.401 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.401 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.401 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:41.401 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:50:41.401 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:41.401 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.401 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.402 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:41.402 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:50:41.402 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9999999999999998e-07 2024-09-16T20:50:41.402 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.402 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.402 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:41.402 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639183+0000", 2024-09-16T20:50:41.402 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2399999999999996e-06 2024-09-16T20:50:41.402 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.402 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.402 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:41.402 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639195+0000", 2024-09-16T20:50:41.402 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1113e-05 2024-09-16T20:50:41.402 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.403 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.403 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:41.403 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639206+0000", 2024-09-16T20:50:41.403 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1503e-05 2024-09-16T20:50:41.403 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.403 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.403 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:41.403 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689969+0000", 2024-09-16T20:50:41.403 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050762676 2024-09-16T20:50:41.403 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.403 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.403 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:41.403 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690071+0000", 2024-09-16T20:50:41.403 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102163 2024-09-16T20:50:41.403 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.404 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.404 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:41.404 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690191+0000", 2024-09-16T20:50:41.404 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000119728 2024-09-16T20:50:41.404 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:41.404 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:41.404 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:41.404 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.404 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.404 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3730 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1538048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:41.404 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:50:41.404 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 138.73158607100001, 2024-09-16T20:50:41.404 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051131467999999999, 2024-09-16T20:50:41.404 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:41.404 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:41.405 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:41.405 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:41.405 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:41.405 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:41.405 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3730 2024-09-16T20:50:41.405 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.405 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:41.405 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.405 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:41.405 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:50:41.405 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:41.405 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.405 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.405 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:41.406 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:50:41.406 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:41.406 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.406 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.406 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:41.406 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639216+0000", 2024-09-16T20:50:41.406 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.39e-07 2024-09-16T20:50:41.406 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.406 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.406 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:41.406 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639218+0000", 2024-09-16T20:50:41.406 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.537e-06 2024-09-16T20:50:41.406 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.406 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.406 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:41.407 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639229+0000", 2024-09-16T20:50:41.407 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1122e-05 2024-09-16T20:50:41.407 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.407 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.407 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:41.407 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639237+0000", 2024-09-16T20:50:41.407 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8509999999999999e-06 2024-09-16T20:50:41.407 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.407 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.407 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:41.407 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690227+0000", 2024-09-16T20:50:41.407 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050990147 2024-09-16T20:50:41.407 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.407 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.408 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:41.408 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690278+0000", 2024-09-16T20:50:41.408 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0223999999999999e-05 2024-09-16T20:50:41.408 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.408 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.408 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:41.408 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690347+0000", 2024-09-16T20:50:41.408 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9147999999999996e-05 2024-09-16T20:50:41.408 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:41.408 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:41.408 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:41.408 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.408 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.409 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3731 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1546240~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:41.409 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:50:41.409 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 138.73155536300001, 2024-09-16T20:50:41.409 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051276744999999999, 2024-09-16T20:50:41.409 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:41.409 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:41.409 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:41.409 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:41.409 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:41.409 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:41.409 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3731 2024-09-16T20:50:41.409 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.409 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:41.409 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.409 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:41.410 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:50:41.410 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:41.410 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.410 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.410 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:41.410 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:50:41.410 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:41.410 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.410 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.410 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:41.410 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:50:41.410 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9900000000000001e-07 2024-09-16T20:50:41.410 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.410 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.411 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:41.411 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639249+0000", 2024-09-16T20:50:41.411 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3630000000000001e-06 2024-09-16T20:50:41.411 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.411 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.411 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:41.411 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639255+0000", 2024-09-16T20:50:41.411 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4960000000000001e-06 2024-09-16T20:50:41.411 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.411 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.411 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:41.411 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639261+0000", 2024-09-16T20:50:41.411 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2839999999999997e-06 2024-09-16T20:50:41.411 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.412 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.412 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:41.412 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690375+0000", 2024-09-16T20:50:41.412 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051114684000000001 2024-09-16T20:50:41.412 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.412 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.412 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:41.412 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690425+0000", 2024-09-16T20:50:41.412 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9379999999999998e-05 2024-09-16T20:50:41.412 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.412 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.412 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:41.412 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690523+0000", 2024-09-16T20:50:41.412 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8139000000000003e-05 2024-09-16T20:50:41.412 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:41.413 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:41.413 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:41.413 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.413 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.413 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3732 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1554432~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:41.413 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:50:41.413 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 138.73150958100001, 2024-09-16T20:50:41.413 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051380096, 2024-09-16T20:50:41.413 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:41.414 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:41.414 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:41.414 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:41.414 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:41.414 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:41.414 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3732 2024-09-16T20:50:41.414 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.414 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:41.414 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.414 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:41.414 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:50:41.414 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:41.414 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.414 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.414 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:41.414 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:50:41.415 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:41.415 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.415 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.415 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:41.415 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:50:41.415 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6100000000000001e-07 2024-09-16T20:50:41.415 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.415 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.415 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:41.415 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639295+0000", 2024-09-16T20:50:41.415 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.447e-06 2024-09-16T20:50:41.415 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.415 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.415 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:41.415 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639302+0000", 2024-09-16T20:50:41.416 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0570000000000004e-06 2024-09-16T20:50:41.416 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.416 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.416 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:41.416 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639307+0000", 2024-09-16T20:50:41.416 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1429999999999997e-06 2024-09-16T20:50:41.416 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.416 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.416 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:41.416 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690555+0000", 2024-09-16T20:50:41.416 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051247963000000001 2024-09-16T20:50:41.416 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.416 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.416 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:41.416 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690602+0000", 2024-09-16T20:50:41.417 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7602000000000003e-05 2024-09-16T20:50:41.417 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.417 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.417 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:41.417 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690672+0000", 2024-09-16T20:50:41.417 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9423000000000003e-05 2024-09-16T20:50:41.417 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:41.417 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:41.417 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:41.417 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.417 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.417 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3734 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1562624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:41.417 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:50:41.417 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 138.731487931, 2024-09-16T20:50:41.418 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051490610999999999, 2024-09-16T20:50:41.418 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:41.418 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:41.418 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:41.418 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:41.418 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:41.418 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:41.418 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3734 2024-09-16T20:50:41.418 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.418 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:41.418 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.418 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:41.418 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:50:41.418 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:41.419 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.419 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.419 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:41.419 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:50:41.419 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:41.419 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.419 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.419 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:41.419 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639314+0000", 2024-09-16T20:50:41.419 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4400000000000001e-07 2024-09-16T20:50:41.419 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.419 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.419 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:41.420 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639316+0000", 2024-09-16T20:50:41.420 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1459999999999999e-06 2024-09-16T20:50:41.420 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.420 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.420 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:41.420 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639322+0000", 2024-09-16T20:50:41.420 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7610000000000004e-06 2024-09-16T20:50:41.420 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.420 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.420 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:41.420 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639327+0000", 2024-09-16T20:50:41.420 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.186e-06 2024-09-16T20:50:41.420 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.420 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.421 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:41.421 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690698+0000", 2024-09-16T20:50:41.421 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051371223000000001 2024-09-16T20:50:41.421 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.421 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.421 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:41.421 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690748+0000", 2024-09-16T20:50:41.421 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0192999999999999e-05 2024-09-16T20:50:41.421 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.421 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.421 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:41.421 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690804+0000", 2024-09-16T20:50:41.421 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5658000000000001e-05 2024-09-16T20:50:41.421 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:41.422 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:41.422 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:41.422 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.422 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.422 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3736 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1570816~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:41.422 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:50:41.422 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 138.73146867200001, 2024-09-16T20:50:41.422 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051596528000000003, 2024-09-16T20:50:41.422 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:41.422 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:41.422 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:41.422 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:41.422 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:41.423 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:41.423 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3736 2024-09-16T20:50:41.423 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.423 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:41.423 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.423 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:41.423 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:50:41.423 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:41.423 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.423 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.423 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:41.423 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:50:41.423 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:41.423 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.424 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.424 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:41.424 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:50:41.424 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9999999999999999e-07 2024-09-16T20:50:41.424 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.424 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.424 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:41.424 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639335+0000", 2024-09-16T20:50:41.424 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.635e-06 2024-09-16T20:50:41.424 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.424 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.424 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:41.424 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639340+0000", 2024-09-16T20:50:41.424 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5840000000000002e-06 2024-09-16T20:50:41.425 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.425 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.425 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:41.425 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639346+0000", 2024-09-16T20:50:41.425 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1190000000000002e-06 2024-09-16T20:50:41.425 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.425 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.425 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:41.425 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690817+0000", 2024-09-16T20:50:41.425 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051470721999999997 2024-09-16T20:50:41.425 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.425 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.425 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:41.425 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690875+0000", 2024-09-16T20:50:41.426 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7441000000000001e-05 2024-09-16T20:50:41.426 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.426 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.426 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:41.426 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690929+0000", 2024-09-16T20:50:41.426 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4727e-05 2024-09-16T20:50:41.426 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:41.426 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:41.426 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:41.426 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.426 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.426 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5434 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:41.426 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:50:41.426 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 138.50620710600001, 2024-09-16T20:50:41.427 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099755831000000003, 2024-09-16T20:50:41.427 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:41.427 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:41.427 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:41.427 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:41.427 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:41.427 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:41.427 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5434 2024-09-16T20:50:41.427 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.427 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:41.427 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.427 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:41.427 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:50:41.427 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:41.428 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.428 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.428 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:41.428 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:50:41.428 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:41.428 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.428 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.428 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:41.428 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864595+0000", 2024-09-16T20:50:41.428 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1099999999999995e-07 2024-09-16T20:50:41.428 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.428 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.428 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:41.429 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864597+0000", 2024-09-16T20:50:41.429 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3989999999999998e-06 2024-09-16T20:50:41.429 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.429 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.429 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:41.429 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864603+0000", 2024-09-16T20:50:41.429 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9309999999999996e-06 2024-09-16T20:50:41.429 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.429 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.429 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:41.429 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864607+0000", 2024-09-16T20:50:41.429 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6200000000000001e-06 2024-09-16T20:50:41.429 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.429 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.430 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:41.430 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964164+0000", 2024-09-16T20:50:41.430 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099557162000000005 2024-09-16T20:50:41.430 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.430 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.430 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:41.430 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964244+0000", 2024-09-16T20:50:41.430 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9597000000000004e-05 2024-09-16T20:50:41.430 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.430 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.430 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:41.430 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964350+0000", 2024-09-16T20:50:41.430 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010641099999999999 2024-09-16T20:50:41.430 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:41.431 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:41.431 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:41.431 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.431 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.431 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5435 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 20480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:41.431 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:50:41.431 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 138.50617728399999, 2024-09-16T20:50:41.431 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100897102, 2024-09-16T20:50:41.431 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:41.431 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:41.431 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:41.431 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:41.431 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:41.431 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:41.432 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5435 2024-09-16T20:50:41.432 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.432 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:41.432 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.432 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:41.432 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:50:41.432 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:41.432 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.432 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.432 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:41.432 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:50:41.432 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:41.432 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.433 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.433 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:41.433 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:50:41.433 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8000000000000001e-07 2024-09-16T20:50:41.433 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.433 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.433 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:41.433 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864626+0000", 2024-09-16T20:50:41.433 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.745e-06 2024-09-16T20:50:41.433 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.433 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.433 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:41.433 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864633+0000", 2024-09-16T20:50:41.434 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3339999999999998e-06 2024-09-16T20:50:41.434 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.434 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.434 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:41.434 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864636+0000", 2024-09-16T20:50:41.434 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1200000000000002e-06 2024-09-16T20:50:41.434 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.434 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.434 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:41.435 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965299+0000", 2024-09-16T20:50:41.435 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10066352000000001 2024-09-16T20:50:41.435 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.435 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.435 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:41.435 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965390+0000", 2024-09-16T20:50:41.435 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0894000000000001e-05 2024-09-16T20:50:41.435 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.435 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.435 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:41.435 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965521+0000", 2024-09-16T20:50:41.435 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013110899999999999 2024-09-16T20:50:41.435 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:41.435 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:41.436 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:41.436 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.436 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.436 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:152 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 327680~32768 out=32768b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:50:41.436 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:50:41.436 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 137.207680342, 2024-09-16T20:50:41.436 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11869405800000001, 2024-09-16T20:50:41.436 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:41.436 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:41.436 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:41.436 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:41.436 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:50:41.436 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:50:41.437 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 152 2024-09-16T20:50:41.437 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.437 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:41.437 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.437 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:41.437 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:50:41.437 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:41.437 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.437 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.437 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:41.437 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:50:41.437 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:41.437 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.438 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.438 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:41.438 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163122+0000", 2024-09-16T20:50:41.438 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6400000000000003e-07 2024-09-16T20:50:41.438 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.438 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.438 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:41.438 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163124+0000", 2024-09-16T20:50:41.438 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.565e-06 2024-09-16T20:50:41.438 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.438 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.438 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:41.438 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163136+0000", 2024-09-16T20:50:41.439 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1717e-05 2024-09-16T20:50:41.439 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.439 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.439 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:41.439 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163142+0000", 2024-09-16T20:50:41.439 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1750000000000002e-06 2024-09-16T20:50:41.439 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.439 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.439 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:41.439 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163606+0000", 2024-09-16T20:50:41.439 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00046418799999999998 2024-09-16T20:50:41.439 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.439 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.439 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:41.440 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163656+0000", 2024-09-16T20:50:41.440 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9852999999999999e-05 2024-09-16T20:50:41.440 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.440 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.440 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:41.440 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281815+0000", 2024-09-16T20:50:41.440 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11815909600000001 2024-09-16T20:50:41.440 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:41.440 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:41.440 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:41.440 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.440 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.440 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:153 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:50:41.440 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:50:41.441 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 137.20764738899999, 2024-09-16T20:50:41.441 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11903462400000001, 2024-09-16T20:50:41.441 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:41.441 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:41.441 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:41.441 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:41.441 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:50:41.441 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:50:41.441 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 153 2024-09-16T20:50:41.441 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.441 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:41.441 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.441 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:41.441 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:50:41.442 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:41.442 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.442 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.442 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:41.442 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:50:41.442 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:41.442 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.442 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.442 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:41.442 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163155+0000", 2024-09-16T20:50:41.442 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.37e-07 2024-09-16T20:50:41.442 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.442 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.442 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:41.443 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163157+0000", 2024-09-16T20:50:41.443 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6010000000000002e-06 2024-09-16T20:50:41.443 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.443 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.443 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:41.443 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163165+0000", 2024-09-16T20:50:41.443 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2619999999999998e-06 2024-09-16T20:50:41.443 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.443 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.443 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:41.443 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163171+0000", 2024-09-16T20:50:41.443 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.711e-06 2024-09-16T20:50:41.443 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.443 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.444 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:41.444 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281868+0000", 2024-09-16T20:50:41.444 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118696622 2024-09-16T20:50:41.444 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.444 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.444 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:41.444 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281969+0000", 2024-09-16T20:50:41.444 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101371 2024-09-16T20:50:41.444 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.444 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.444 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:41.444 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282189+0000", 2024-09-16T20:50:41.444 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00021952000000000001 2024-09-16T20:50:41.444 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:41.445 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:41.445 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:41.445 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.445 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.445 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:154 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 360448~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:50:41.445 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:50:41.445 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 137.20761561699999, 2024-09-16T20:50:41.445 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11932035100000001, 2024-09-16T20:50:41.445 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:41.445 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:41.445 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:41.445 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:41.445 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:50:41.445 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:50:41.446 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 154 2024-09-16T20:50:41.446 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.446 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:41.446 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.446 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:41.446 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:50:41.446 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:41.446 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.446 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.446 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:41.446 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:50:41.446 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:41.446 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.446 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.446 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:41.447 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:50:41.447 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:50:41.447 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.447 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.447 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:41.447 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163189+0000", 2024-09-16T20:50:41.447 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6840000000000001e-06 2024-09-16T20:50:41.447 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.447 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.447 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:41.447 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163196+0000", 2024-09-16T20:50:41.447 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7920000000000004e-06 2024-09-16T20:50:41.447 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.447 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.448 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:41.448 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163201+0000", 2024-09-16T20:50:41.448 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2809999999999999e-06 2024-09-16T20:50:41.448 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.448 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.448 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:41.448 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282219+0000", 2024-09-16T20:50:41.448 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.119018343 2024-09-16T20:50:41.448 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.448 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.448 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:41.448 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282305+0000", 2024-09-16T20:50:41.449 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5865000000000005e-05 2024-09-16T20:50:41.449 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.449 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.449 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:41.449 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282506+0000", 2024-09-16T20:50:41.449 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000200927 2024-09-16T20:50:41.449 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:41.449 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:41.449 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:41.449 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.449 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.449 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop(client.4366.0:3088 3.1e e38/23 3:79e5ec01:::rbd_header.110e891a9b2b:head v 38'306, mlcod=36'304)", 2024-09-16T20:50:41.450 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:50:41.450 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 137.206596749, 2024-09-16T20:50:41.450 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118102759, 2024-09-16T20:50:41.450 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:41.450 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:41.450 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:41.450 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:41.450 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.450 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:41.450 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:50:41.450 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:41.450 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.450 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.451 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:41.451 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:50:41.451 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:41.451 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.451 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.451 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:41.451 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:50:41.451 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.85e-07 2024-09-16T20:50:41.451 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.451 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.451 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:41.451 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164212+0000", 2024-09-16T20:50:41.451 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3999999999999997e-06 2024-09-16T20:50:41.452 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.452 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.452 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:41.452 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281553+0000", 2024-09-16T20:50:41.452 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11734135499999999 2024-09-16T20:50:41.452 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.452 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.452 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:41.452 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281567+0000", 2024-09-16T20:50:41.452 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4188999999999999e-05 2024-09-16T20:50:41.452 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.452 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.452 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:41.453 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281653+0000", 2024-09-16T20:50:41.453 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5385999999999999e-05 2024-09-16T20:50:41.453 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.453 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.453 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:41.453 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281679+0000", 2024-09-16T20:50:41.453 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6689000000000001e-05 2024-09-16T20:50:41.453 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.453 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.453 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:41.453 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282277+0000", 2024-09-16T20:50:41.453 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00059798000000000002 2024-09-16T20:50:41.453 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.454 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.454 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:41.454 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282307+0000", 2024-09-16T20:50:41.454 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9975e-05 2024-09-16T20:50:41.454 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:41.454 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:41.454 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:41.454 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.454 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.454 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5255.0:44 3.16 3:6f5476df:::rbd_header.110e25db1f60:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:50:41.454 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:50:41.454 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 65.050293694000004, 2024-09-16T20:50:41.454 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.121007509, 2024-09-16T20:50:41.455 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:41.455 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:41.455 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:41.455 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:41.455 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5255", 2024-09-16T20:50:41.455 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:50:41.455 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 44 2024-09-16T20:50:41.455 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.455 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:41.455 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.455 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:41.455 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:50:41.455 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:41.455 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.456 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.456 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:41.456 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:50:41.456 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:41.456 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.456 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.456 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:41.456 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:50:41.456 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0500000000000003e-07 2024-09-16T20:50:41.456 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.456 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.456 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:41.456 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320512+0000", 2024-09-16T20:50:41.457 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2540000000000001e-06 2024-09-16T20:50:41.457 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.457 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.457 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:41.457 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320518+0000", 2024-09-16T20:50:41.457 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1689999999999997e-06 2024-09-16T20:50:41.457 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.457 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.457 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:41.457 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320521+0000", 2024-09-16T20:50:41.457 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5200000000000002e-06 2024-09-16T20:50:41.457 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.457 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.458 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:41.458 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320548+0000", 2024-09-16T20:50:41.458 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6662999999999999e-05 2024-09-16T20:50:41.458 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.458 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.458 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:41.458 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320587+0000", 2024-09-16T20:50:41.458 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8866999999999999e-05 2024-09-16T20:50:41.458 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.458 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.458 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:41.458 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.441515+0000", 2024-09-16T20:50:41.458 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.120928331 2024-09-16T20:50:41.459 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:41.459 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:41.459 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:41.459 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.459 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.459 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:20329 3.5 3:a4ea64b7:::rbd_header.110ede5ef857:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:41.459 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:01.061694+0000", 2024-09-16T20:50:41.459 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 40.309107126000001, 2024-09-16T20:50:41.459 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.071234077000000007, 2024-09-16T20:50:41.459 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:41.459 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:41.459 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:41.459 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:41.459 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:50:41.460 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:41.460 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 20329 2024-09-16T20:50:41.460 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.460 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:41.460 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.460 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:41.460 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.061694+0000", 2024-09-16T20:50:41.460 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:41.460 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.460 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.460 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:41.460 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.061694+0000", 2024-09-16T20:50:41.460 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:41.460 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.461 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.461 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:41.461 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089232+0000", 2024-09-16T20:50:41.461 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.027537275 2024-09-16T20:50:41.461 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.461 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.461 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:41.461 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089237+0000", 2024-09-16T20:50:41.461 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4709999999999998e-06 2024-09-16T20:50:41.461 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.461 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.461 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:41.461 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089246+0000", 2024-09-16T20:50:41.461 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8230000000000001e-06 2024-09-16T20:50:41.461 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.462 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.462 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:41.462 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089252+0000", 2024-09-16T20:50:41.462 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3060000000000003e-06 2024-09-16T20:50:41.462 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.462 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.462 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:41.462 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.132756+0000", 2024-09-16T20:50:41.462 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.043504334999999998 2024-09-16T20:50:41.462 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.462 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.462 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:41.462 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.132829+0000", 2024-09-16T20:50:41.462 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2849999999999995e-05 2024-09-16T20:50:41.462 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:41.462 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:41.463 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:41.463 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.132928+0000", 2024-09-16T20:50:41.463 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.9017000000000004e-05 2024-09-16T20:50:41.463 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:41.463 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:41.463 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:41.463 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:41.463 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:41.463 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:41.464 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:50:41.465 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:41.464+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:41.565 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:41.563+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:41.602 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:41.602 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:50:41.602 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:50:41.602 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:41.613 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:50:41.666 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:41.665+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:41.722 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:41.722 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:50:41.722 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:50:41.722 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:50:41.722 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:41.730 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:50:41.774 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:41.774+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:41.812 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 4, 5] out_osds: [0, 3] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-16T20:50:41.812 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T20:50:41.812 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-09-16T20:50:41.812 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T20:50:41.812 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T20:50:41.812 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2024-09-16T20:50:41.839 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:41.839 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:50:41.839 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:50:41.839 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:50:41.839 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.839 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:3088 3.1e 3:79e5ec01:::rbd_header.110e891a9b2b:head [watch watch cookie 140213437171488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:50:41.840 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:50:41.840 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 137.673673729, 2024-09-16T20:50:41.840 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.11895186200000001, 2024-09-16T20:50:41.840 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:41.840 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:41.840 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:41.840 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:41.840 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:50:41.840 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:41.840 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3088 2024-09-16T20:50:41.840 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.840 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:41.840 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.841 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:41.841 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:50:41.841 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:41.841 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.841 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.841 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:41.841 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:50:41.841 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:41.841 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.841 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.841 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:41.841 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163846+0000", 2024-09-16T20:50:41.841 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6799999999999999e-07 2024-09-16T20:50:41.842 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.842 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.842 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:41.842 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163850+0000", 2024-09-16T20:50:41.842 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.929e-06 2024-09-16T20:50:41.842 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.842 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.842 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:41.842 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163862+0000", 2024-09-16T20:50:41.842 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1972e-05 2024-09-16T20:50:41.842 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.842 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.842 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:41.842 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163867+0000", 2024-09-16T20:50:41.843 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.451e-06 2024-09-16T20:50:41.843 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.843 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.843 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:41.843 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163912+0000", 2024-09-16T20:50:41.843 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4456000000000003e-05 2024-09-16T20:50:41.843 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.843 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.843 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:41.843 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163962+0000", 2024-09-16T20:50:41.843 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0578000000000003e-05 2024-09-16T20:50:41.843 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.843 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.844 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-16T20:50:41.844 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164058+0000", 2024-09-16T20:50:41.844 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5934999999999995e-05 2024-09-16T20:50:41.844 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.844 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.844 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:50:41.844 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164570+0000", 2024-09-16T20:50:41.844 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00051241100000000005 2024-09-16T20:50:41.844 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.844 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.844 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:41.844 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282629+0000", 2024-09-16T20:50:41.845 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.118058986 2024-09-16T20:50:41.845 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.845 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.845 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:50:41.845 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282725+0000", 2024-09-16T20:50:41.845 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5063999999999996e-05 2024-09-16T20:50:41.845 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.845 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.845 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:41.845 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282797+0000", 2024-09-16T20:50:41.845 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2212000000000006e-05 2024-09-16T20:50:41.845 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:41.845 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:41.846 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:41.846 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.846 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.846 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:8039 3.1e 3.ac44ff9e (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:50:41.846 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:50:41.846 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 112.12984638, 2024-09-16T20:50:41.846 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0053310449999999, 2024-09-16T20:50:41.846 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:41.846 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:41.846 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:41.846 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:41.846 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:50:41.846 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:41.847 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8039 2024-09-16T20:50:41.847 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.847 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:41.847 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.847 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:41.847 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:50:41.847 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:41.847 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.847 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.847 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:41.847 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:50:41.847 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:41.847 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.847 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.847 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:41.848 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707674+0000", 2024-09-16T20:50:41.848 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6530000000000001e-06 2024-09-16T20:50:41.848 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.848 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.848 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:41.848 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707681+0000", 2024-09-16T20:50:41.848 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9639999999999997e-06 2024-09-16T20:50:41.848 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.848 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.848 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:41.848 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707700+0000", 2024-09-16T20:50:41.848 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8674000000000001e-05 2024-09-16T20:50:41.848 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.849 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.849 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:41.849 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707708+0000", 2024-09-16T20:50:41.849 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6810000000000004e-06 2024-09-16T20:50:41.849 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.849 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.849 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:41.849 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711293+0000", 2024-09-16T20:50:41.849 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035848189999999999 2024-09-16T20:50:41.849 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.849 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.849 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:41.849 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711301+0000", 2024-09-16T20:50:41.849 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4730000000000003e-06 2024-09-16T20:50:41.849 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.850 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.850 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:41.850 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.712990+0000", 2024-09-16T20:50:41.850 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.001689348 2024-09-16T20:50:41.850 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.850 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.850 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:41.850 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713003+0000", 2024-09-16T20:50:41.850 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3433e-05 2024-09-16T20:50:41.850 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:41.850 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:41.850 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:41.850 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.850 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.851 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:8040 3.1e 3.ac44ff9e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:50:41.851 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:50:41.851 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 112.129785985, 2024-09-16T20:50:41.851 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0053410780000001, 2024-09-16T20:50:41.851 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:41.851 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:41.851 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:41.851 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:41.851 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:50:41.851 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:41.851 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8040 2024-09-16T20:50:41.851 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.851 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:41.851 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.852 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:41.852 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:50:41.852 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:41.852 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.852 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.852 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:41.852 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:50:41.852 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:41.852 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.852 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.852 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:41.852 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:50:41.852 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8100000000000003e-07 2024-09-16T20:50:41.852 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.853 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.853 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:41.853 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707737+0000", 2024-09-16T20:50:41.853 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5020000000000001e-06 2024-09-16T20:50:41.853 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.853 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.853 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:41.853 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707745+0000", 2024-09-16T20:50:41.853 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2169999999999994e-06 2024-09-16T20:50:41.853 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.853 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.853 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:41.853 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707751+0000", 2024-09-16T20:50:41.853 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.263e-06 2024-09-16T20:50:41.853 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.854 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.854 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:41.854 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711336+0000", 2024-09-16T20:50:41.854 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035848360000000001 2024-09-16T20:50:41.854 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.854 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.854 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:41.854 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711340+0000", 2024-09-16T20:50:41.854 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1309999999999999e-06 2024-09-16T20:50:41.854 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.854 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.854 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:41.854 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713060+0000", 2024-09-16T20:50:41.855 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0017195059999999 2024-09-16T20:50:41.855 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.855 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.855 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:41.855 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713074+0000", 2024-09-16T20:50:41.855 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4042e-05 2024-09-16T20:50:41.855 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:41.855 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:41.855 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:41.855 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.855 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.855 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4927.0:4487 3.1e 3.ac44ff9e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:50:41.855 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:50:41.855 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 112.12926578299999, 2024-09-16T20:50:41.856 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0048840880000001, 2024-09-16T20:50:41.856 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:41.856 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:41.856 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:41.856 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:41.856 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4927", 2024-09-16T20:50:41.856 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:41.856 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4487 2024-09-16T20:50:41.856 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.856 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:41.856 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.856 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:41.856 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:50:41.856 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:41.857 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.857 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.857 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:41.857 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:50:41.857 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:41.857 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.857 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.857 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:41.857 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708254+0000", 2024-09-16T20:50:41.857 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1800000000000005e-07 2024-09-16T20:50:41.857 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.857 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.857 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:41.857 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708258+0000", 2024-09-16T20:50:41.858 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8689999999999997e-06 2024-09-16T20:50:41.858 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.858 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.858 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:41.858 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708267+0000", 2024-09-16T20:50:41.858 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9590000000000007e-06 2024-09-16T20:50:41.858 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.858 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.858 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:41.858 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708274+0000", 2024-09-16T20:50:41.858 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9970000000000001e-06 2024-09-16T20:50:41.858 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.858 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.858 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:41.859 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711366+0000", 2024-09-16T20:50:41.859 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030919530000000001 2024-09-16T20:50:41.859 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.859 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.859 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:41.859 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711372+0000", 2024-09-16T20:50:41.859 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9410000000000004e-06 2024-09-16T20:50:41.859 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.859 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.859 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:41.859 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713126+0000", 2024-09-16T20:50:41.859 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0017537249999999 2024-09-16T20:50:41.859 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.859 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.860 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:41.860 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713137+0000", 2024-09-16T20:50:41.860 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0825999999999999e-05 2024-09-16T20:50:41.860 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:41.860 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:41.860 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:41.860 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.860 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.860 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5089.0:11703 3.1b 3:dcd4684e:::rbd_data.110e747f0779.000000000000001b:head [set-alloc-hint object_size 262144 write_size 262144,write 139264~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:50:41.860 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:50:41.860 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 97.699530009, 2024-09-16T20:50:41.860 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.084231385000000006, 2024-09-16T20:50:41.860 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:41.860 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:41.861 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:41.861 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:41.861 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5089", 2024-09-16T20:50:41.861 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:50:41.861 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 11703 2024-09-16T20:50:41.861 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.861 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:41.861 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.861 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:41.861 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:50:41.861 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:41.861 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.861 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.862 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:41.862 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:50:41.862 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:41.862 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.862 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.862 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:41.862 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:50:41.862 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2600000000000001e-07 2024-09-16T20:50:41.862 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.862 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.862 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:41.862 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137998+0000", 2024-09-16T20:50:41.862 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9549999999999998e-06 2024-09-16T20:50:41.862 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.863 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.863 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:41.863 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138008+0000", 2024-09-16T20:50:41.863 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3789999999999995e-06 2024-09-16T20:50:41.863 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.863 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.863 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:41.863 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138013+0000", 2024-09-16T20:50:41.863 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2209999999999996e-06 2024-09-16T20:50:41.863 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.863 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.863 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:41.864 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138054+0000", 2024-09-16T20:50:41.864 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1022999999999997e-05 2024-09-16T20:50:41.864 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.864 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.864 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:41.864 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138124+0000", 2024-09-16T20:50:41.864 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0234999999999994e-05 2024-09-16T20:50:41.864 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.864 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.864 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:41.864 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138197+0000", 2024-09-16T20:50:41.864 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2451000000000001e-05 2024-09-16T20:50:41.864 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.865 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.865 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:41.865 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222010+0000", 2024-09-16T20:50:41.865 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.083813433000000007 2024-09-16T20:50:41.865 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.865 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.865 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:50:41.865 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222163+0000", 2024-09-16T20:50:41.865 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015274599999999999 2024-09-16T20:50:41.865 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.865 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.865 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:50:41.865 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222197+0000", 2024-09-16T20:50:41.865 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3683000000000003e-05 2024-09-16T20:50:41.866 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.866 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.866 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:41.866 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222220+0000", 2024-09-16T20:50:41.866 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3433000000000001e-05 2024-09-16T20:50:41.866 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:41.866 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:41.866 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:41.866 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.866 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.866 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.5089.0:11702 3.15 e73/64 3:acf33c8b:::rbd_data.110e747f0779.000000000000001a:head v 73'132, mlcod=73'130)", 2024-09-16T20:50:41.866 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:50:41.866 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 97.699262680000004, 2024-09-16T20:50:41.866 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.084079763000000002, 2024-09-16T20:50:41.867 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:41.867 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:41.867 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:41.867 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:41.867 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.867 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:41.867 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:50:41.867 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:41.867 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.867 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.867 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:41.867 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:50:41.867 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:41.867 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.868 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.868 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:41.868 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:50:41.868 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:50:41.868 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.868 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.868 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:41.868 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138263+0000", 2024-09-16T20:50:41.868 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1e-06 2024-09-16T20:50:41.868 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.868 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.868 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:41.868 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221589+0000", 2024-09-16T20:50:41.869 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.083326002999999996 2024-09-16T20:50:41.869 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.869 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.869 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:41.869 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221614+0000", 2024-09-16T20:50:41.869 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5593999999999999e-05 2024-09-16T20:50:41.869 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.869 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.869 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:41.869 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221714+0000", 2024-09-16T20:50:41.869 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9889000000000005e-05 2024-09-16T20:50:41.869 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.869 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.869 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:41.870 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221727+0000", 2024-09-16T20:50:41.870 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3125999999999999e-05 2024-09-16T20:50:41.870 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.870 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.870 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:50:41.870 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222294+0000", 2024-09-16T20:50:41.870 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00056712199999999998 2024-09-16T20:50:41.870 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.870 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.870 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:41.870 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222336+0000", 2024-09-16T20:50:41.870 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1470000000000001e-05 2024-09-16T20:50:41.870 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:41.871 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:41.871 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:41.871 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.871 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.871 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5138.0:9358 3.c 3:33996a43:::rbd_data.110e6372565d.0000000000000007:head [read 1343488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T20:50:41.871 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:50:41.871 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 73.984987270000005, 2024-09-16T20:50:41.871 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0167493569999999, 2024-09-16T20:50:41.871 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:41.871 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:41.871 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:41.871 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:41.871 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5138", 2024-09-16T20:50:41.871 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:50:41.872 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9358 2024-09-16T20:50:41.872 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.872 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:41.872 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.872 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:41.872 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:50:41.872 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:41.872 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.872 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.872 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:41.872 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:50:41.872 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:41.872 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.873 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.873 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:41.873 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852533+0000", 2024-09-16T20:50:41.873 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.646e-06 2024-09-16T20:50:41.873 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.873 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.873 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:41.873 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852539+0000", 2024-09-16T20:50:41.873 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3389999999999998e-06 2024-09-16T20:50:41.873 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.873 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.873 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:41.873 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852552+0000", 2024-09-16T20:50:41.873 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.271e-05 2024-09-16T20:50:41.874 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.874 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.874 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:41.874 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852563+0000", 2024-09-16T20:50:41.874 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1139e-05 2024-09-16T20:50:41.874 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.874 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.874 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:41.874 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852883+0000", 2024-09-16T20:50:41.874 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00032010600000000001 2024-09-16T20:50:41.874 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.874 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.874 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:41.874 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852890+0000", 2024-09-16T20:50:41.875 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7669999999999998e-06 2024-09-16T20:50:41.875 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.875 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.875 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:41.875 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869059+0000", 2024-09-16T20:50:41.875 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.016169353 2024-09-16T20:50:41.875 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.875 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.875 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:41.875 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869178+0000", 2024-09-16T20:50:41.875 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118193 2024-09-16T20:50:41.875 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.875 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.875 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:41.876 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869281+0000", 2024-09-16T20:50:41.876 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010310399999999999 2024-09-16T20:50:41.876 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:41.876 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:41.876 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:41.876 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.876 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.876 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5138.0:9382 3.c 3:33996a43:::rbd_data.110e6372565d.0000000000000007:head [read 0~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T20:50:41.876 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:50:41.876 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 73.984936392999998, 2024-09-16T20:50:41.876 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.017136034, 2024-09-16T20:50:41.876 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:41.876 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:41.877 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:41.877 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:41.877 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5138", 2024-09-16T20:50:41.877 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:50:41.877 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9382 2024-09-16T20:50:41.877 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.877 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:41.877 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.877 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:41.877 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:50:41.877 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:41.877 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.877 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.878 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:41.878 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:50:41.878 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:41.878 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.878 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.878 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:41.878 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852583+0000", 2024-09-16T20:50:41.878 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.51e-07 2024-09-16T20:50:41.878 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.878 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.878 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:41.879 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852586+0000", 2024-09-16T20:50:41.879 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0280000000000001e-06 2024-09-16T20:50:41.879 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.879 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.879 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:41.879 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852595+0000", 2024-09-16T20:50:41.879 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2799999999999992e-06 2024-09-16T20:50:41.879 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.879 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.879 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:41.879 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852601+0000", 2024-09-16T20:50:41.879 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3810000000000004e-06 2024-09-16T20:50:41.880 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.880 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.880 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:41.880 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852921+0000", 2024-09-16T20:50:41.880 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00031979699999999997 2024-09-16T20:50:41.880 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.880 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.880 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:41.880 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852930+0000", 2024-09-16T20:50:41.880 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6489999999999997e-06 2024-09-16T20:50:41.880 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.880 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.880 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:41.881 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869434+0000", 2024-09-16T20:50:41.881 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0165038829999999 2024-09-16T20:50:41.881 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.881 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.881 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:41.881 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869495+0000", 2024-09-16T20:50:41.881 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1509999999999999e-05 2024-09-16T20:50:41.881 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.881 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.881 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:41.881 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869718+0000", 2024-09-16T20:50:41.881 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000222955 2024-09-16T20:50:41.881 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:41.881 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:41.882 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:41.882 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.882 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.882 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9969 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 632832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:41.882 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642045+0000", 2024-09-16T20:50:41.882 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.195473935999999, 2024-09-16T20:50:41.882 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076903192999999995, 2024-09-16T20:50:41.882 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:41.882 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:41.882 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:41.882 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:41.882 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:41.882 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:41.882 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9969 2024-09-16T20:50:41.883 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.883 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:41.883 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.883 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:41.883 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642045+0000", 2024-09-16T20:50:41.883 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:41.883 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.883 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.883 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:41.883 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642045+0000", 2024-09-16T20:50:41.883 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:41.883 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.883 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.884 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:41.884 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642045+0000", 2024-09-16T20:50:41.884 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5100000000000001e-07 2024-09-16T20:50:41.884 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.884 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.884 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:41.884 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642047+0000", 2024-09-16T20:50:41.884 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8679999999999999e-06 2024-09-16T20:50:41.884 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.884 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.884 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:41.884 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642053+0000", 2024-09-16T20:50:41.884 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1720000000000004e-06 2024-09-16T20:50:41.884 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.884 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.885 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:41.885 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642060+0000", 2024-09-16T20:50:41.885 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0339999999999999e-06 2024-09-16T20:50:41.885 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.885 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.885 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:41.885 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.718562+0000", 2024-09-16T20:50:41.885 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076501969000000003 2024-09-16T20:50:41.885 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.885 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.885 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:41.885 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.718643+0000", 2024-09-16T20:50:41.885 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0539999999999998e-05 2024-09-16T20:50:41.885 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.886 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.886 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:41.886 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.718948+0000", 2024-09-16T20:50:41.886 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000305259 2024-09-16T20:50:41.886 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:41.886 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:41.886 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:41.886 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.886 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.886 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9970 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 637952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:41.886 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642067+0000", 2024-09-16T20:50:41.886 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.195451955999999, 2024-09-16T20:50:41.886 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077174928000000004, 2024-09-16T20:50:41.886 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:41.887 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:41.887 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:41.887 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:41.887 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:41.887 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:41.887 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9970 2024-09-16T20:50:41.887 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.887 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:41.887 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.887 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:41.887 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642067+0000", 2024-09-16T20:50:41.887 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:41.887 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.887 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.887 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:41.888 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642067+0000", 2024-09-16T20:50:41.888 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:41.888 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.888 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.888 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:41.888 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642067+0000", 2024-09-16T20:50:41.888 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2499999999999995e-07 2024-09-16T20:50:41.888 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.888 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.888 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:41.888 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642070+0000", 2024-09-16T20:50:41.888 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3199999999999998e-06 2024-09-16T20:50:41.888 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.888 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.889 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:41.889 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642076+0000", 2024-09-16T20:50:41.889 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.122e-06 2024-09-16T20:50:41.889 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.889 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.889 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:41.889 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642081+0000", 2024-09-16T20:50:41.889 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0869999999999997e-06 2024-09-16T20:50:41.889 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.889 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.889 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:41.889 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.718980+0000", 2024-09-16T20:50:41.889 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076899482000000005 2024-09-16T20:50:41.889 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.890 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.890 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:41.890 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719085+0000", 2024-09-16T20:50:41.890 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010446399999999999 2024-09-16T20:50:41.890 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.890 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.890 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:41.890 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719242+0000", 2024-09-16T20:50:41.890 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000156928 2024-09-16T20:50:41.890 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:41.890 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:41.890 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:41.890 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.890 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.890 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9971 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 643072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:41.890 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642102+0000", 2024-09-16T20:50:41.891 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.195416248999997, 2024-09-16T20:50:41.891 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077421016999999995, 2024-09-16T20:50:41.891 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:41.891 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:41.891 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:41.891 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:41.891 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:41.891 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:41.891 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9971 2024-09-16T20:50:41.891 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.891 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:41.891 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.891 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:41.891 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642102+0000", 2024-09-16T20:50:41.891 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:41.891 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.892 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.892 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:41.892 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642102+0000", 2024-09-16T20:50:41.892 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:41.892 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.892 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.892 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:41.892 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642103+0000", 2024-09-16T20:50:41.892 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7399999999999998e-07 2024-09-16T20:50:41.892 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.892 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.892 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:41.892 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642105+0000", 2024-09-16T20:50:41.892 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2649999999999999e-06 2024-09-16T20:50:41.893 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.893 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.893 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:41.893 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642112+0000", 2024-09-16T20:50:41.893 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.294e-06 2024-09-16T20:50:41.893 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.893 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.893 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:41.893 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642118+0000", 2024-09-16T20:50:41.893 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4500000000000003e-06 2024-09-16T20:50:41.893 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.893 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.893 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:41.893 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719285+0000", 2024-09-16T20:50:41.893 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077166883000000006 2024-09-16T20:50:41.894 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.894 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.894 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:41.894 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719368+0000", 2024-09-16T20:50:41.894 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2811000000000004e-05 2024-09-16T20:50:41.894 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.894 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.894 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:41.894 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719523+0000", 2024-09-16T20:50:41.894 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015584 2024-09-16T20:50:41.894 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:41.894 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:41.894 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:41.894 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.894 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.894 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9972 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 648192~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:41.895 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642125+0000", 2024-09-16T20:50:41.895 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.195393301999999, 2024-09-16T20:50:41.895 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077710324999999997, 2024-09-16T20:50:41.895 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:41.895 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:41.895 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:41.895 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:41.895 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:41.895 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:41.895 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9972 2024-09-16T20:50:41.895 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.895 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:41.895 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.895 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:41.896 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642125+0000", 2024-09-16T20:50:41.896 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:41.896 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.896 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.896 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:41.896 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642125+0000", 2024-09-16T20:50:41.896 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:41.896 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.896 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.896 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:41.896 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642126+0000", 2024-09-16T20:50:41.896 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1899999999999998e-07 2024-09-16T20:50:41.896 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.897 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.897 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:41.897 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642128+0000", 2024-09-16T20:50:41.897 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8619999999999999e-06 2024-09-16T20:50:41.897 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.897 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.897 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:41.897 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642134+0000", 2024-09-16T20:50:41.898 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5289999999999997e-06 2024-09-16T20:50:41.898 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.898 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.898 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:41.898 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642139+0000", 2024-09-16T20:50:41.898 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1250000000000001e-06 2024-09-16T20:50:41.898 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.898 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.898 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:41.898 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719570+0000", 2024-09-16T20:50:41.898 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077430516000000005 2024-09-16T20:50:41.898 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.898 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.899 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:41.899 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719655+0000", 2024-09-16T20:50:41.899 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5500000000000005e-05 2024-09-16T20:50:41.899 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.899 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.899 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:41.899 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719836+0000", 2024-09-16T20:50:41.899 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018037399999999999 2024-09-16T20:50:41.899 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:41.899 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:41.899 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:41.899 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.899 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.899 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9973 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 653312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:41.900 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:50:41.900 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.195372691999999, 2024-09-16T20:50:41.900 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077905262000000003, 2024-09-16T20:50:41.900 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:41.900 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:41.900 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:41.900 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:41.900 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:41.900 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:41.900 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9973 2024-09-16T20:50:41.900 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.900 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:41.900 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.901 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:41.901 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:50:41.901 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:41.901 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.901 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.901 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:41.901 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:50:41.901 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:41.901 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.901 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.901 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:41.901 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:50:41.901 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:50:41.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.902 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.902 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:41.902 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642148+0000", 2024-09-16T20:50:41.902 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0219999999999999e-06 2024-09-16T20:50:41.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.902 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.902 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:41.902 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642154+0000", 2024-09-16T20:50:41.902 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8540000000000002e-06 2024-09-16T20:50:41.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.902 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.902 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:41.902 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642159+0000", 2024-09-16T20:50:41.903 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.045e-06 2024-09-16T20:50:41.903 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.903 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.903 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:41.903 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719875+0000", 2024-09-16T20:50:41.903 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077716116000000002 2024-09-16T20:50:41.903 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.903 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.903 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:41.903 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719934+0000", 2024-09-16T20:50:41.903 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8819e-05 2024-09-16T20:50:41.903 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.904 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.904 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:41.904 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720051+0000", 2024-09-16T20:50:41.904 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000116947 2024-09-16T20:50:41.904 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:41.904 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:41.904 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:41.904 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.904 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.904 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9974 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 658432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:41.904 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642166+0000", 2024-09-16T20:50:41.904 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.195352778, 2024-09-16T20:50:41.904 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.078179786000000001, 2024-09-16T20:50:41.905 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:41.905 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:41.905 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:41.905 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:41.905 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:41.905 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:41.905 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9974 2024-09-16T20:50:41.905 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.905 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:41.905 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.905 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:41.905 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642166+0000", 2024-09-16T20:50:41.905 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:41.905 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.906 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.906 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:41.906 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642166+0000", 2024-09-16T20:50:41.906 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:41.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.906 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.906 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:41.906 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647343+0000", 2024-09-16T20:50:41.906 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0051769440000000002 2024-09-16T20:50:41.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.906 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.906 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:41.906 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647348+0000", 2024-09-16T20:50:41.906 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3619999999999997e-06 2024-09-16T20:50:41.907 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.907 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.907 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:41.907 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647359+0000", 2024-09-16T20:50:41.907 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0929e-05 2024-09-16T20:50:41.907 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.907 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.907 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:41.907 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647366+0000", 2024-09-16T20:50:41.907 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0910000000000004e-06 2024-09-16T20:50:41.907 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.907 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.907 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:41.907 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720094+0000", 2024-09-16T20:50:41.908 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.072727403999999995 2024-09-16T20:50:41.908 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.908 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.908 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:41.908 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720173+0000", 2024-09-16T20:50:41.908 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9382000000000007e-05 2024-09-16T20:50:41.908 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.908 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.908 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:41.908 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720346+0000", 2024-09-16T20:50:41.908 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017267399999999999 2024-09-16T20:50:41.908 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:41.908 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:41.908 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:41.909 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.909 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.909 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9990 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 740352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:41.909 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647725+0000", 2024-09-16T20:50:41.909 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.189793694000002, 2024-09-16T20:50:41.909 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077029919000000002, 2024-09-16T20:50:41.909 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:41.909 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:41.909 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:41.909 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:41.909 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:41.909 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:41.909 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9990 2024-09-16T20:50:41.909 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.910 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:41.910 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.910 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:41.910 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647725+0000", 2024-09-16T20:50:41.910 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:41.910 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.910 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.910 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:41.910 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647725+0000", 2024-09-16T20:50:41.910 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:41.910 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.910 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.910 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:41.910 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647725+0000", 2024-09-16T20:50:41.910 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2399999999999999e-07 2024-09-16T20:50:41.911 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.911 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.911 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:41.911 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647726+0000", 2024-09-16T20:50:41.911 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.116e-06 2024-09-16T20:50:41.911 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.911 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.911 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:41.911 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647729+0000", 2024-09-16T20:50:41.911 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.937e-06 2024-09-16T20:50:41.911 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.911 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.911 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:41.911 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647732+0000", 2024-09-16T20:50:41.911 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8739999999999999e-06 2024-09-16T20:50:41.912 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.912 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.912 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:41.912 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724612+0000", 2024-09-16T20:50:41.912 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076880102000000006 2024-09-16T20:50:41.912 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.912 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.912 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:41.912 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724649+0000", 2024-09-16T20:50:41.912 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6738e-05 2024-09-16T20:50:41.912 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.912 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.912 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:41.912 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724755+0000", 2024-09-16T20:50:41.913 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010592800000000001 2024-09-16T20:50:41.913 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:41.913 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:41.913 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:41.913 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.913 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.913 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9991 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 745472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:41.913 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:50:41.913 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.189783384999998, 2024-09-16T20:50:41.913 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077257820000000005, 2024-09-16T20:50:41.913 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:41.913 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:41.913 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:41.913 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:41.914 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:41.914 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:41.914 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9991 2024-09-16T20:50:41.914 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.914 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:41.914 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.914 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:41.914 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:50:41.914 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:41.914 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.914 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.914 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:41.914 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:50:41.914 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:41.914 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.914 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.915 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:41.915 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:50:41.915 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1199999999999999e-07 2024-09-16T20:50:41.915 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.915 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.915 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:41.915 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647736+0000", 2024-09-16T20:50:41.915 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3699999999999999e-07 2024-09-16T20:50:41.915 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.915 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.915 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:41.915 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647739+0000", 2024-09-16T20:50:41.915 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9359999999999999e-06 2024-09-16T20:50:41.915 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.915 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.916 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:41.916 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647742+0000", 2024-09-16T20:50:41.916 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5720000000000001e-06 2024-09-16T20:50:41.916 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.916 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.916 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:41.916 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724774+0000", 2024-09-16T20:50:41.916 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077032457999999998 2024-09-16T20:50:41.916 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.916 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.916 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:41.916 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724818+0000", 2024-09-16T20:50:41.916 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3803000000000001e-05 2024-09-16T20:50:41.917 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.917 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.917 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:41.917 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724993+0000", 2024-09-16T20:50:41.917 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000174902 2024-09-16T20:50:41.917 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:41.917 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:41.917 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:41.917 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.917 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.917 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9992 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 750592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:41.917 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:50:41.917 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.189773795999997, 2024-09-16T20:50:41.918 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077553227000000002, 2024-09-16T20:50:41.918 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:41.918 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:41.918 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:41.918 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:41.918 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:41.918 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:41.918 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9992 2024-09-16T20:50:41.918 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.918 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:41.918 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.918 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:41.918 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:50:41.919 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:41.919 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.919 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.919 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:41.919 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:50:41.919 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:41.919 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.919 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.919 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:41.919 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:50:41.919 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0200000000000001e-07 2024-09-16T20:50:41.919 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.919 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.920 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:41.920 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647746+0000", 2024-09-16T20:50:41.920 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.088e-06 2024-09-16T20:50:41.920 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.920 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.920 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:41.920 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647749+0000", 2024-09-16T20:50:41.920 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9170000000000002e-06 2024-09-16T20:50:41.920 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.920 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.920 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:41.920 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647752+0000", 2024-09-16T20:50:41.920 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.774e-06 2024-09-16T20:50:41.920 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.921 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.921 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:41.921 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725042+0000", 2024-09-16T20:50:41.921 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077289952999999995 2024-09-16T20:50:41.921 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.921 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.921 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:41.921 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725113+0000", 2024-09-16T20:50:41.921 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0920999999999994e-05 2024-09-16T20:50:41.921 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.921 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.921 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:41.921 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725298+0000", 2024-09-16T20:50:41.922 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000185372 2024-09-16T20:50:41.922 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:41.922 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:41.922 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:41.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.922 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.922 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9993 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 755712~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:41.922 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647755+0000", 2024-09-16T20:50:41.922 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.189763769999999, 2024-09-16T20:50:41.922 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077725674999999994, 2024-09-16T20:50:41.922 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:41.922 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:41.922 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:41.922 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:41.923 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:41.923 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:41.923 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9993 2024-09-16T20:50:41.923 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.923 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:41.923 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.923 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:41.923 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647755+0000", 2024-09-16T20:50:41.923 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:41.923 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.923 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.923 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:41.923 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647755+0000", 2024-09-16T20:50:41.923 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:41.924 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.924 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.924 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:41.924 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653406+0000", 2024-09-16T20:50:41.924 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.005650876 2024-09-16T20:50:41.924 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.924 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.924 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:41.924 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653411+0000", 2024-09-16T20:50:41.924 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8899999999999998e-06 2024-09-16T20:50:41.924 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.924 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.924 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:41.924 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653423+0000", 2024-09-16T20:50:41.925 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2447e-05 2024-09-16T20:50:41.925 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.925 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.925 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:41.925 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653431+0000", 2024-09-16T20:50:41.925 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5730000000000002e-06 2024-09-16T20:50:41.925 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.925 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.925 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:41.925 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725333+0000", 2024-09-16T20:50:41.925 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.071902743000000005 2024-09-16T20:50:41.925 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.925 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.926 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:41.926 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725385+0000", 2024-09-16T20:50:41.926 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1585000000000002e-05 2024-09-16T20:50:41.926 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.926 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.926 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:41.926 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725481+0000", 2024-09-16T20:50:41.926 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5561000000000002e-05 2024-09-16T20:50:41.926 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:41.926 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:41.926 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:41.926 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.926 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.926 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6061.0:5400 3.b 3:d353d652:::rbd_data.110e16eafbd.0000000000000010:head [read 577536~28672] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-16T20:50:41.926 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:27.761367+0000", 2024-09-16T20:50:41.927 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 14.076151824, 2024-09-16T20:50:41.927 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0029066630000001, 2024-09-16T20:50:41.927 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:41.927 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:41.927 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:41.927 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:41.927 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6061", 2024-09-16T20:50:41.927 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3930128440", 2024-09-16T20:50:41.927 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5400 2024-09-16T20:50:41.927 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.927 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:41.927 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.927 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:41.927 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761367+0000", 2024-09-16T20:50:41.928 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:41.928 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.928 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.928 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:41.928 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761367+0000", 2024-09-16T20:50:41.928 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:41.928 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.928 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.928 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:41.928 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761369+0000", 2024-09-16T20:50:41.928 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.934e-06 2024-09-16T20:50:41.928 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.928 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.929 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:41.929 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761374+0000", 2024-09-16T20:50:41.929 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7139999999999998e-06 2024-09-16T20:50:41.929 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.929 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.929 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:41.929 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761387+0000", 2024-09-16T20:50:41.929 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2349e-05 2024-09-16T20:50:41.929 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.929 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.929 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:41.929 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761393+0000", 2024-09-16T20:50:41.929 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.652e-06 2024-09-16T20:50:41.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.930 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.930 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:41.930 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761442+0000", 2024-09-16T20:50:41.930 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9011000000000002e-05 2024-09-16T20:50:41.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.930 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.930 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:41.930 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761451+0000", 2024-09-16T20:50:41.930 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1920000000000005e-06 2024-09-16T20:50:41.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.930 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.930 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:41.930 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:28.764169+0000", 2024-09-16T20:50:41.930 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.002718161 2024-09-16T20:50:41.931 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.931 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.931 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:41.931 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:28.764273+0000", 2024-09-16T20:50:41.931 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010465 2024-09-16T20:50:41.931 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:41.931 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:41.931 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:41.931 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.931 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.931 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6061.0:5424 3.b 3:d353d652:::rbd_data.110e16eafbd.0000000000000010:head [read 606208~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-16T20:50:41.931 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:27.766882+0000", 2024-09-16T20:50:41.931 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 14.070636460999999, 2024-09-16T20:50:41.931 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99753650500000002, 2024-09-16T20:50:41.931 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:41.932 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:41.932 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:41.932 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:41.932 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6061", 2024-09-16T20:50:41.932 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3930128440", 2024-09-16T20:50:41.932 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5424 2024-09-16T20:50:41.932 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.932 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:41.932 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.932 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:41.932 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766882+0000", 2024-09-16T20:50:41.932 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:41.932 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.932 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.933 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:41.933 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766882+0000", 2024-09-16T20:50:41.933 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:41.933 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.933 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.933 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:41.933 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766883+0000", 2024-09-16T20:50:41.933 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3289999999999999e-06 2024-09-16T20:50:41.933 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.933 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.933 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:41.933 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766888+0000", 2024-09-16T20:50:41.933 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5800000000000002e-06 2024-09-16T20:50:41.933 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.933 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.934 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:41.934 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766900+0000", 2024-09-16T20:50:41.934 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1861e-05 2024-09-16T20:50:41.934 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.934 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.934 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:41.934 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766909+0000", 2024-09-16T20:50:41.934 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5210000000000008e-06 2024-09-16T20:50:41.934 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.934 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.934 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:41.934 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766958+0000", 2024-09-16T20:50:41.934 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8718999999999999e-05 2024-09-16T20:50:41.934 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.934 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.935 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:41.935 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766975+0000", 2024-09-16T20:50:41.935 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7231000000000002e-05 2024-09-16T20:50:41.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.935 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.935 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:41.935 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:28.764327+0000", 2024-09-16T20:50:41.935 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99735112500000001 2024-09-16T20:50:41.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:41.935 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:41.935 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:41.935 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:28.764419+0000", 2024-09-16T20:50:41.935 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2139000000000007e-05 2024-09-16T20:50:41.935 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:41.935 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:41.935 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:41.936 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:41.936 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:41.936 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:41.938 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:41.938 INFO:teuthology.orchestra.run.smithi029.stdout: "success": "" 2024-09-16T20:50:41.938 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:41.938 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:50:41.938 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:41.936+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:42.038 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:42.037+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:42.085 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:42.085 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:50:42.086 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:50:42.086 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:42.095 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:50:42.138 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:42.138+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:42.239 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:42.238+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:42.243 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:42.243 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:50:42.243 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:50:42.243 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:50:42.243 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:42.252 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:50:42.340 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:42.339+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:42.400 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:42.400 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:50:42.400 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:50:42.400 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:50:42.400 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.400 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12548 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 628736~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:42.401 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757009+0000", 2024-09-16T20:50:42.401 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.641676930999999, 2024-09-16T20:50:42.401 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065743956000000006, 2024-09-16T20:50:42.401 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:42.401 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:42.401 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:42.401 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:42.401 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:42.401 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:42.401 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12548 2024-09-16T20:50:42.401 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.401 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:42.401 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.401 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:42.402 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757009+0000", 2024-09-16T20:50:42.402 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.402 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.402 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.402 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:42.402 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757009+0000", 2024-09-16T20:50:42.402 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.402 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.402 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.402 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:42.402 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757010+0000", 2024-09-16T20:50:42.402 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9900000000000001e-07 2024-09-16T20:50:42.402 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.402 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.402 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:42.403 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757012+0000", 2024-09-16T20:50:42.403 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2689999999999998e-06 2024-09-16T20:50:42.403 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.403 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.403 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:42.403 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757022+0000", 2024-09-16T20:50:42.403 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9699999999999994e-06 2024-09-16T20:50:42.403 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.403 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.403 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:42.403 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757028+0000", 2024-09-16T20:50:42.403 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3799999999999999e-06 2024-09-16T20:50:42.403 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.403 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.403 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:42.404 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822552+0000", 2024-09-16T20:50:42.404 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065524281000000004 2024-09-16T20:50:42.404 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.404 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.404 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:42.404 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822613+0000", 2024-09-16T20:50:42.404 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0430999999999998e-05 2024-09-16T20:50:42.404 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.404 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.404 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:42.404 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822753+0000", 2024-09-16T20:50:42.404 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000140226 2024-09-16T20:50:42.404 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.404 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:42.404 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.405 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.405 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.405 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12549 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 633856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:42.405 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:50:42.405 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.641648183999997, 2024-09-16T20:50:42.405 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065978518, 2024-09-16T20:50:42.405 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:42.405 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:42.405 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:42.405 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:42.405 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:42.405 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:42.405 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12549 2024-09-16T20:50:42.405 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.406 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:42.406 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.406 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:42.406 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:50:42.406 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.406 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.406 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:42.406 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:50:42.406 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.406 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.406 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:42.406 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:50:42.406 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9299999999999999e-07 2024-09-16T20:50:42.407 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.407 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.407 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:42.407 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757040+0000", 2024-09-16T20:50:42.407 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.677e-06 2024-09-16T20:50:42.407 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.407 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.407 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:42.407 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757045+0000", 2024-09-16T20:50:42.407 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8149999999999996e-06 2024-09-16T20:50:42.407 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.407 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.407 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:42.407 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757048+0000", 2024-09-16T20:50:42.408 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1e-06 2024-09-16T20:50:42.408 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.408 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.408 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:42.408 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822796+0000", 2024-09-16T20:50:42.408 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065748081 2024-09-16T20:50:42.408 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.408 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.408 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:42.408 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822867+0000", 2024-09-16T20:50:42.408 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0679000000000005e-05 2024-09-16T20:50:42.408 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.408 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.408 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:42.409 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823016+0000", 2024-09-16T20:50:42.409 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014987299999999999 2024-09-16T20:50:42.409 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.409 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:42.409 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.409 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.409 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.409 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12550 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 638976~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:42.409 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757060+0000", 2024-09-16T20:50:42.409 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.641625769000001, 2024-09-16T20:50:42.409 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066153564999999998, 2024-09-16T20:50:42.409 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:42.409 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:42.409 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:42.410 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:42.410 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:42.410 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:42.410 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12550 2024-09-16T20:50:42.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.410 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:42.410 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.410 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:42.410 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757060+0000", 2024-09-16T20:50:42.410 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.410 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.411 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:42.411 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757060+0000", 2024-09-16T20:50:42.411 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.411 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.411 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.411 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:42.411 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757061+0000", 2024-09-16T20:50:42.411 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.15e-07 2024-09-16T20:50:42.411 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.411 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.411 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:42.411 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757062+0000", 2024-09-16T20:50:42.411 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.663e-06 2024-09-16T20:50:42.411 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.412 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.412 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:42.412 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757066+0000", 2024-09-16T20:50:42.412 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.817e-06 2024-09-16T20:50:42.412 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.412 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.412 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:42.412 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757070+0000", 2024-09-16T20:50:42.412 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1509999999999997e-06 2024-09-16T20:50:42.412 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.412 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.412 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:42.412 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823044+0000", 2024-09-16T20:50:42.412 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065973992999999995 2024-09-16T20:50:42.413 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.413 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.413 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:42.413 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823082+0000", 2024-09-16T20:50:42.413 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8050999999999999e-05 2024-09-16T20:50:42.413 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.413 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.413 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:42.413 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823214+0000", 2024-09-16T20:50:42.413 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013157500000000001 2024-09-16T20:50:42.413 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.413 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:42.413 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.413 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.413 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.414 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12551 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 644096~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:42.414 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:50:42.414 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.641612070999997, 2024-09-16T20:50:42.414 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066391378000000001, 2024-09-16T20:50:42.414 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:42.414 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:42.414 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:42.414 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:42.414 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:42.414 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:42.414 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12551 2024-09-16T20:50:42.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.414 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:42.414 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.414 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:42.415 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:50:42.415 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.415 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.415 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.415 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:42.415 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:50:42.415 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.415 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.415 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.415 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:42.415 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:50:42.415 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2999999999999999e-07 2024-09-16T20:50:42.415 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.415 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.416 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:42.416 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757075+0000", 2024-09-16T20:50:42.416 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.097e-06 2024-09-16T20:50:42.416 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.416 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.416 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:42.416 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757082+0000", 2024-09-16T20:50:42.416 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9500000000000004e-06 2024-09-16T20:50:42.416 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.416 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.416 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:42.416 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757086+0000", 2024-09-16T20:50:42.417 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2229999999999999e-06 2024-09-16T20:50:42.417 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.417 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.417 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:42.417 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823262+0000", 2024-09-16T20:50:42.417 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066176078999999999 2024-09-16T20:50:42.417 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.417 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.417 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:42.417 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823336+0000", 2024-09-16T20:50:42.417 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3979000000000004e-05 2024-09-16T20:50:42.417 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.418 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.418 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:42.418 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823465+0000", 2024-09-16T20:50:42.418 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012982 2024-09-16T20:50:42.418 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.418 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:42.418 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.418 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.418 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12552 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 649216~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:42.418 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757089+0000", 2024-09-16T20:50:42.418 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.641596929999999, 2024-09-16T20:50:42.418 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066655617, 2024-09-16T20:50:42.419 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:42.419 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:42.419 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:42.419 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:42.419 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:42.419 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:42.419 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12552 2024-09-16T20:50:42.419 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.419 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:42.419 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.419 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:42.419 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757089+0000", 2024-09-16T20:50:42.420 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.420 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.420 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:42.420 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757089+0000", 2024-09-16T20:50:42.420 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.420 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.420 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:42.420 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759492+0000", 2024-09-16T20:50:42.420 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0024026109999999998 2024-09-16T20:50:42.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.420 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.420 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:42.421 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759497+0000", 2024-09-16T20:50:42.421 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1120000000000004e-06 2024-09-16T20:50:42.421 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.421 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.421 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:42.421 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759508+0000", 2024-09-16T20:50:42.421 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0981e-05 2024-09-16T20:50:42.421 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.421 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.421 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:42.421 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759516+0000", 2024-09-16T20:50:42.421 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7270000000000007e-06 2024-09-16T20:50:42.421 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.422 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.422 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:42.422 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823504+0000", 2024-09-16T20:50:42.422 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.063987984999999997 2024-09-16T20:50:42.422 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.422 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.422 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:42.422 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823579+0000", 2024-09-16T20:50:42.422 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5703000000000003e-05 2024-09-16T20:50:42.422 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.422 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.422 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:42.422 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823745+0000", 2024-09-16T20:50:42.423 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016549800000000001 2024-09-16T20:50:42.423 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.423 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:42.423 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.423 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.423 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12560 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 690176~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:42.423 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.759658+0000", 2024-09-16T20:50:42.423 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.639027728000002, 2024-09-16T20:50:42.423 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065858517000000005, 2024-09-16T20:50:42.423 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:42.424 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:42.424 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:42.424 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:42.424 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:42.424 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:42.424 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12560 2024-09-16T20:50:42.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.424 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:42.424 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:42.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759658+0000", 2024-09-16T20:50:42.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.425 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:42.425 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759658+0000", 2024-09-16T20:50:42.425 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.425 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:42.425 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759659+0000", 2024-09-16T20:50:42.425 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.17e-07 2024-09-16T20:50:42.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.425 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:42.425 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759660+0000", 2024-09-16T20:50:42.426 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0189999999999999e-06 2024-09-16T20:50:42.426 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.426 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.426 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:42.426 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759663+0000", 2024-09-16T20:50:42.426 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0010000000000002e-06 2024-09-16T20:50:42.426 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.426 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.426 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:42.426 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759665+0000", 2024-09-16T20:50:42.426 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5440000000000001e-06 2024-09-16T20:50:42.426 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.426 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.426 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:42.427 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825355+0000", 2024-09-16T20:50:42.427 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065690224000000005 2024-09-16T20:50:42.427 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.427 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.427 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:42.427 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825394+0000", 2024-09-16T20:50:42.427 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8810999999999998e-05 2024-09-16T20:50:42.427 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.427 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.427 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:42.427 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825517+0000", 2024-09-16T20:50:42.427 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012270099999999999 2024-09-16T20:50:42.427 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.427 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:42.427 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.428 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.428 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.428 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12561 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 695296~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:42.428 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.759668+0000", 2024-09-16T20:50:42.428 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.639017885999998, 2024-09-16T20:50:42.428 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066116999999999995, 2024-09-16T20:50:42.428 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:42.428 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:42.428 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:42.428 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:42.428 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:42.428 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:42.428 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12561 2024-09-16T20:50:42.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.429 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:42.429 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.429 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:42.429 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759668+0000", 2024-09-16T20:50:42.429 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.429 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.429 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:42.429 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759668+0000", 2024-09-16T20:50:42.429 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.429 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.430 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:42.430 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765379+0000", 2024-09-16T20:50:42.430 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057104410000000001 2024-09-16T20:50:42.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.430 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.430 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:42.430 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765383+0000", 2024-09-16T20:50:42.430 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2980000000000002e-06 2024-09-16T20:50:42.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.430 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.430 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:42.430 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765395+0000", 2024-09-16T20:50:42.430 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1671000000000001e-05 2024-09-16T20:50:42.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.430 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.431 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:42.431 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765403+0000", 2024-09-16T20:50:42.431 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1210000000000007e-06 2024-09-16T20:50:42.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.431 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.431 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:42.431 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825552+0000", 2024-09-16T20:50:42.431 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.060149498000000003 2024-09-16T20:50:42.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.431 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.431 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:42.431 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825625+0000", 2024-09-16T20:50:42.431 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2540999999999995e-05 2024-09-16T20:50:42.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.432 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.432 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:42.432 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825785+0000", 2024-09-16T20:50:42.432 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016043000000000001 2024-09-16T20:50:42.432 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.432 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:42.432 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.432 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.432 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12585 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 818176~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:42.432 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.765848+0000", 2024-09-16T20:50:42.432 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.632837915000003, 2024-09-16T20:50:42.432 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065843218999999995, 2024-09-16T20:50:42.433 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:42.433 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:42.433 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:42.433 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:42.433 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:42.433 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:42.433 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12585 2024-09-16T20:50:42.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.433 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:42.433 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.433 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:42.433 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765848+0000", 2024-09-16T20:50:42.433 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.434 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.434 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:42.434 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765848+0000", 2024-09-16T20:50:42.434 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.434 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.434 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:42.434 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768683+0000", 2024-09-16T20:50:42.434 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0028344519999999999 2024-09-16T20:50:42.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.434 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.434 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:42.434 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768686+0000", 2024-09-16T20:50:42.434 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8789999999999996e-06 2024-09-16T20:50:42.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.435 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.435 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:42.435 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768696+0000", 2024-09-16T20:50:42.435 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6709999999999994e-06 2024-09-16T20:50:42.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.435 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.435 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:42.435 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768702+0000", 2024-09-16T20:50:42.435 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2550000000000003e-06 2024-09-16T20:50:42.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.435 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.435 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:42.435 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.831499+0000", 2024-09-16T20:50:42.436 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.062797081000000005 2024-09-16T20:50:42.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.436 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.436 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:42.436 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.831538+0000", 2024-09-16T20:50:42.436 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8523999999999999e-05 2024-09-16T20:50:42.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.436 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.436 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:42.436 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.831691+0000", 2024-09-16T20:50:42.436 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015335699999999999 2024-09-16T20:50:42.436 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.436 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:42.437 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.437 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12597 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 879616~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:42.437 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.768883+0000", 2024-09-16T20:50:42.437 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.629803187, 2024-09-16T20:50:42.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065816527, 2024-09-16T20:50:42.437 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:42.437 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:42.437 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:42.437 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:42.437 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:42.437 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:42.438 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12597 2024-09-16T20:50:42.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.438 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:42.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:42.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768883+0000", 2024-09-16T20:50:42.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:42.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768883+0000", 2024-09-16T20:50:42.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.439 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.439 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:42.439 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771508+0000", 2024-09-16T20:50:42.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0026247589999999999 2024-09-16T20:50:42.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.439 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.439 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:42.439 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771513+0000", 2024-09-16T20:50:42.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9949999999999996e-06 2024-09-16T20:50:42.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.439 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.439 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:42.439 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771525+0000", 2024-09-16T20:50:42.440 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2033e-05 2024-09-16T20:50:42.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.440 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:42.440 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771532+0000", 2024-09-16T20:50:42.440 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4220000000000003e-06 2024-09-16T20:50:42.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.440 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:42.440 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.834532+0000", 2024-09-16T20:50:42.440 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.063000159 2024-09-16T20:50:42.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.440 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:42.440 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.834590+0000", 2024-09-16T20:50:42.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8218999999999999e-05 2024-09-16T20:50:42.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:42.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.834699+0000", 2024-09-16T20:50:42.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010894 2024-09-16T20:50:42.441 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.441 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:42.441 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.441 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12619 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 992256~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:42.441 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.774656+0000", 2024-09-16T20:50:42.441 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.624030413, 2024-09-16T20:50:42.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065715859000000001, 2024-09-16T20:50:42.442 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:42.442 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:42.442 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:42.442 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:42.442 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:42.442 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:42.442 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12619 2024-09-16T20:50:42.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.442 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:42.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:42.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774656+0000", 2024-09-16T20:50:42.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.443 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:42.443 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774656+0000", 2024-09-16T20:50:42.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.443 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:42.443 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774875+0000", 2024-09-16T20:50:42.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000218898 2024-09-16T20:50:42.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.443 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:42.443 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774879+0000", 2024-09-16T20:50:42.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0389999999999998e-06 2024-09-16T20:50:42.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:42.444 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774890+0000", 2024-09-16T20:50:42.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1250000000000001e-05 2024-09-16T20:50:42.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:42.444 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774897+0000", 2024-09-16T20:50:42.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7619999999999998e-06 2024-09-16T20:50:42.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:42.444 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840139+0000", 2024-09-16T20:50:42.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065242438 2024-09-16T20:50:42.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:42.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840186+0000", 2024-09-16T20:50:42.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6894999999999997e-05 2024-09-16T20:50:42.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:42.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840372+0000", 2024-09-16T20:50:42.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000185577 2024-09-16T20:50:42.445 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.445 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:42.445 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.446 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12620 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 997376~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:42.446 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.774908+0000", 2024-09-16T20:50:42.446 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.623778192000003, 2024-09-16T20:50:42.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065782754999999998, 2024-09-16T20:50:42.446 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:42.446 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:42.446 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:42.446 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:42.446 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:42.446 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:42.446 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12620 2024-09-16T20:50:42.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.446 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:42.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:42.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774908+0000", 2024-09-16T20:50:42.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:42.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774908+0000", 2024-09-16T20:50:42.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:42.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775102+0000", 2024-09-16T20:50:42.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000194098 2024-09-16T20:50:42.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:42.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775105+0000", 2024-09-16T20:50:42.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8739999999999999e-06 2024-09-16T20:50:42.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:42.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775111+0000", 2024-09-16T20:50:42.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9519999999999999e-06 2024-09-16T20:50:42.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:42.449 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775115+0000", 2024-09-16T20:50:42.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.002e-06 2024-09-16T20:50:42.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.449 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:42.449 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840409+0000", 2024-09-16T20:50:42.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065294195999999999 2024-09-16T20:50:42.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.449 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:42.449 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840507+0000", 2024-09-16T20:50:42.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7662000000000001e-05 2024-09-16T20:50:42.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.450 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:42.450 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840691+0000", 2024-09-16T20:50:42.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000183971 2024-09-16T20:50:42.450 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.450 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:42.450 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.450 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12621 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1002496~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:42.450 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775123+0000", 2024-09-16T20:50:42.451 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.623563496000003, 2024-09-16T20:50:42.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065829586999999995, 2024-09-16T20:50:42.451 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:42.451 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:42.451 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:42.451 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:42.451 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:42.451 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:42.451 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12621 2024-09-16T20:50:42.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.451 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:42.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.451 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:42.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775123+0000", 2024-09-16T20:50:42.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:42.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775123+0000", 2024-09-16T20:50:42.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:42.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775348+0000", 2024-09-16T20:50:42.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000225295 2024-09-16T20:50:42.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:42.453 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775352+0000", 2024-09-16T20:50:42.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0810000000000004e-06 2024-09-16T20:50:42.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:42.453 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775361+0000", 2024-09-16T20:50:42.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7399999999999993e-06 2024-09-16T20:50:42.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:42.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775367+0000", 2024-09-16T20:50:42.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5200000000000003e-06 2024-09-16T20:50:42.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:42.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840730+0000", 2024-09-16T20:50:42.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065362803999999997 2024-09-16T20:50:42.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:42.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840801+0000", 2024-09-16T20:50:42.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1006000000000004e-05 2024-09-16T20:50:42.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:42.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840952+0000", 2024-09-16T20:50:42.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015114100000000001 2024-09-16T20:50:42.455 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.455 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:42.455 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.455 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12622 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1007616~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:42.456 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775381+0000", 2024-09-16T20:50:42.456 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.623304662000002, 2024-09-16T20:50:42.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065822581000000005, 2024-09-16T20:50:42.456 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:42.456 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:42.456 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:42.456 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:42.456 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:42.456 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:42.456 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12622 2024-09-16T20:50:42.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.456 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:42.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.457 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:42.457 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775381+0000", 2024-09-16T20:50:42.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.457 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:42.457 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775381+0000", 2024-09-16T20:50:42.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.457 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:42.457 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775563+0000", 2024-09-16T20:50:42.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018153700000000001 2024-09-16T20:50:42.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:42.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775566+0000", 2024-09-16T20:50:42.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9620000000000002e-06 2024-09-16T20:50:42.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:42.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775572+0000", 2024-09-16T20:50:42.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8529999999999997e-06 2024-09-16T20:50:42.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:42.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775576+0000", 2024-09-16T20:50:42.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8920000000000002e-06 2024-09-16T20:50:42.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:42.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840990+0000", 2024-09-16T20:50:42.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065414599000000004 2024-09-16T20:50:42.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:42.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841068+0000", 2024-09-16T20:50:42.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8157000000000004e-05 2024-09-16T20:50:42.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:42.460 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841204+0000", 2024-09-16T20:50:42.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013558099999999999 2024-09-16T20:50:42.460 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.460 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:42.460 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.460 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12623 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1012736~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:42.460 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775583+0000", 2024-09-16T20:50:42.460 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.623103223999998, 2024-09-16T20:50:42.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065929087999999997, 2024-09-16T20:50:42.460 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:42.461 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:42.461 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:42.461 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:42.461 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:42.461 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:42.461 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12623 2024-09-16T20:50:42.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.461 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:42.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:42.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775583+0000", 2024-09-16T20:50:42.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.462 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:42.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775583+0000", 2024-09-16T20:50:42.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.462 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:42.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775804+0000", 2024-09-16T20:50:42.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00022116900000000001 2024-09-16T20:50:42.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.462 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:42.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775807+0000", 2024-09-16T20:50:42.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9529999999999999e-06 2024-09-16T20:50:42.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.463 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:42.463 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775814+0000", 2024-09-16T20:50:42.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7490000000000001e-06 2024-09-16T20:50:42.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.463 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:42.463 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775818+0000", 2024-09-16T20:50:42.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1999999999999996e-06 2024-09-16T20:50:42.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.463 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:42.463 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841267+0000", 2024-09-16T20:50:42.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065449127999999995 2024-09-16T20:50:42.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.464 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:42.464 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841344+0000", 2024-09-16T20:50:42.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6546999999999997e-05 2024-09-16T20:50:42.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.464 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:42.464 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841512+0000", 2024-09-16T20:50:42.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016834200000000001 2024-09-16T20:50:42.464 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.464 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:42.464 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.465 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12624 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1017856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:42.465 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:50:42.465 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.622860084000003, 2024-09-16T20:50:42.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065945866000000006, 2024-09-16T20:50:42.465 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:42.465 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:42.465 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:42.465 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:42.465 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:42.465 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:42.465 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12624 2024-09-16T20:50:42.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.465 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:42.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:42.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:50:42.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:42.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:50:42.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:42.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:50:42.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-16T20:50:42.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.467 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:42.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775828+0000", 2024-09-16T20:50:42.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3999999999999999e-06 2024-09-16T20:50:42.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.467 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:42.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775831+0000", 2024-09-16T20:50:42.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.478e-06 2024-09-16T20:50:42.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.467 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:42.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775834+0000", 2024-09-16T20:50:42.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9440000000000001e-06 2024-09-16T20:50:42.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:42.468 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841547+0000", 2024-09-16T20:50:42.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065712801000000001 2024-09-16T20:50:42.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:42.468 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841618+0000", 2024-09-16T20:50:42.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1151999999999999e-05 2024-09-16T20:50:42.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:42.468 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841772+0000", 2024-09-16T20:50:42.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000153721 2024-09-16T20:50:42.469 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.469 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:42.469 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.469 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12625 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1022976~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:42.469 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775840+0000", 2024-09-16T20:50:42.469 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.622846475000003, 2024-09-16T20:50:42.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066165451, 2024-09-16T20:50:42.469 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:42.469 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:42.469 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:42.470 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:42.470 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:42.470 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:42.470 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12625 2024-09-16T20:50:42.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.470 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:42.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:42.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775840+0000", 2024-09-16T20:50:42.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:42.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775840+0000", 2024-09-16T20:50:42.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:42.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776014+0000", 2024-09-16T20:50:42.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017441099999999999 2024-09-16T20:50:42.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:42.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776018+0000", 2024-09-16T20:50:42.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6509999999999999e-06 2024-09-16T20:50:42.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:42.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776026+0000", 2024-09-16T20:50:42.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.028e-06 2024-09-16T20:50:42.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:42.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776031+0000", 2024-09-16T20:50:42.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8400000000000002e-06 2024-09-16T20:50:42.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:42.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841801+0000", 2024-09-16T20:50:42.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065770472999999996 2024-09-16T20:50:42.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:42.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841847+0000", 2024-09-16T20:50:42.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5558e-05 2024-09-16T20:50:42.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:42.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842005+0000", 2024-09-16T20:50:42.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015849000000000001 2024-09-16T20:50:42.473 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.473 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:42.473 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.474 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12626 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1028096~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:42.474 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776041+0000", 2024-09-16T20:50:42.474 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.622645175999999, 2024-09-16T20:50:42.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066225260999999994, 2024-09-16T20:50:42.474 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:42.474 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:42.474 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:42.474 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:42.474 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:42.474 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:42.474 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12626 2024-09-16T20:50:42.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.474 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:42.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:42.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776041+0000", 2024-09-16T20:50:42.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:42.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776041+0000", 2024-09-16T20:50:42.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:42.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776263+0000", 2024-09-16T20:50:42.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00022193899999999999 2024-09-16T20:50:42.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.476 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:42.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776268+0000", 2024-09-16T20:50:42.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8049999999999997e-06 2024-09-16T20:50:42.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.476 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:42.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776280+0000", 2024-09-16T20:50:42.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2635e-05 2024-09-16T20:50:42.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:42.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776288+0000", 2024-09-16T20:50:42.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8199999999999997e-06 2024-09-16T20:50:42.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:42.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842041+0000", 2024-09-16T20:50:42.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065752506000000002 2024-09-16T20:50:42.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:42.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842123+0000", 2024-09-16T20:50:42.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2393999999999998e-05 2024-09-16T20:50:42.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:42.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842266+0000", 2024-09-16T20:50:42.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014316200000000001 2024-09-16T20:50:42.478 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.478 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:42.478 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.478 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12627 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1033216~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:42.478 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776301+0000", 2024-09-16T20:50:42.479 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.622385215999998, 2024-09-16T20:50:42.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066187814999999997, 2024-09-16T20:50:42.479 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:42.479 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:42.479 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:42.479 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:42.479 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:42.479 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:42.479 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12627 2024-09-16T20:50:42.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.479 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:42.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:42.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776301+0000", 2024-09-16T20:50:42.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:42.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776301+0000", 2024-09-16T20:50:42.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:42.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776473+0000", 2024-09-16T20:50:42.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017222400000000001 2024-09-16T20:50:42.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.481 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:42.481 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776477+0000", 2024-09-16T20:50:42.481 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7019999999999999e-06 2024-09-16T20:50:42.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.481 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:42.481 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776486+0000", 2024-09-16T20:50:42.481 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2119999999999992e-06 2024-09-16T20:50:42.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.481 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:42.481 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776492+0000", 2024-09-16T20:50:42.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8089999999999998e-06 2024-09-16T20:50:42.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.482 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:42.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842301+0000", 2024-09-16T20:50:42.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065809042999999998 2024-09-16T20:50:42.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.482 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:42.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842362+0000", 2024-09-16T20:50:42.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1260000000000006e-05 2024-09-16T20:50:42.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.482 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:42.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842489+0000", 2024-09-16T20:50:42.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012656500000000001 2024-09-16T20:50:42.483 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.483 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:42.483 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.483 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12628 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1038336~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:42.483 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776502+0000", 2024-09-16T20:50:42.483 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.622184570000002, 2024-09-16T20:50:42.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066283426000000006, 2024-09-16T20:50:42.483 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:42.483 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:42.483 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:42.484 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:42.484 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:42.484 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:42.484 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12628 2024-09-16T20:50:42.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.484 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:42.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:42.484 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776502+0000", 2024-09-16T20:50:42.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:42.484 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776502+0000", 2024-09-16T20:50:42.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.485 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:42.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776716+0000", 2024-09-16T20:50:42.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000214596 2024-09-16T20:50:42.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.485 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:42.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776723+0000", 2024-09-16T20:50:42.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4000000000000003e-06 2024-09-16T20:50:42.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.485 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:42.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776731+0000", 2024-09-16T20:50:42.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6129999999999998e-06 2024-09-16T20:50:42.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.486 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:42.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776737+0000", 2024-09-16T20:50:42.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1970000000000001e-06 2024-09-16T20:50:42.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.486 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:42.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842532+0000", 2024-09-16T20:50:42.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065795127999999994 2024-09-16T20:50:42.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:42.487 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842599+0000", 2024-09-16T20:50:42.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6711000000000006e-05 2024-09-16T20:50:42.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:42.487 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842785+0000", 2024-09-16T20:50:42.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018578099999999999 2024-09-16T20:50:42.487 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.487 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:42.487 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.487 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12629 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1043456~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:42.487 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776748+0000", 2024-09-16T20:50:42.488 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.621938192999998, 2024-09-16T20:50:42.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066303788000000002, 2024-09-16T20:50:42.488 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:42.488 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:42.488 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:42.488 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:42.488 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:42.488 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:42.488 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12629 2024-09-16T20:50:42.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.488 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:42.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.488 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:42.488 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776748+0000", 2024-09-16T20:50:42.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:42.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776748+0000", 2024-09-16T20:50:42.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:42.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776934+0000", 2024-09-16T20:50:42.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018598399999999999 2024-09-16T20:50:42.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:42.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776939+0000", 2024-09-16T20:50:42.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1000000000000003e-06 2024-09-16T20:50:42.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:42.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776954+0000", 2024-09-16T20:50:42.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5221e-05 2024-09-16T20:50:42.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:42.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776962+0000", 2024-09-16T20:50:42.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.554e-06 2024-09-16T20:50:42.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:42.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842813+0000", 2024-09-16T20:50:42.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065851006000000004 2024-09-16T20:50:42.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:42.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842886+0000", 2024-09-16T20:50:42.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3663000000000003e-05 2024-09-16T20:50:42.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:42.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.843052+0000", 2024-09-16T20:50:42.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016526 2024-09-16T20:50:42.491 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.491 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:42.491 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.491 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.492 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:42.492 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:42.493 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:50:42.494 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:42.492+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:42.589 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T20:50:42.594 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:42.593+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:42.646 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:42.646 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:50:42.646 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:50:42.646 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:42.655 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:50:42.694 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:42.693+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:42.717 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T20:50:42.795 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:42.794+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:42.803 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:42.803 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:50:42.803 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:50:42.803 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:50:42.803 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:42.812 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:50:42.844 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:50:42.844 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:50:42.860 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:50:42.860 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:50:42.878 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:50:42.878 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:50:42.895 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:50:42.895 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:50:42.896 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:42.896+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:42.911 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:50:42.912 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:50:42.920 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:42.920 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:50:42.920 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:50:42.920 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:50:42.921 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.921 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5286.0:10643 3.17 3.f6273fb7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-16T20:50:42.921 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:50:42.921 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 94.128014917000002, 2024-09-16T20:50:42.921 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.76672347200000002, 2024-09-16T20:50:42.921 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:42.921 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:42.921 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:42.921 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:42.921 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5286", 2024-09-16T20:50:42.921 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/4135433989", 2024-09-16T20:50:42.922 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 10643 2024-09-16T20:50:42.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.922 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:42.922 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.922 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:42.922 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:50:42.922 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.922 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.922 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:42.923 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:50:42.923 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.923 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.923 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.923 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:42.923 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790489+0000", 2024-09-16T20:50:42.923 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5910000000000001e-06 2024-09-16T20:50:42.923 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.923 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.923 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:42.923 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790493+0000", 2024-09-16T20:50:42.924 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2669999999999998e-06 2024-09-16T20:50:42.924 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.924 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.924 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:42.924 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790500+0000", 2024-09-16T20:50:42.924 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3780000000000003e-06 2024-09-16T20:50:42.924 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.924 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.924 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:42.924 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790505+0000", 2024-09-16T20:50:42.924 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1560000000000003e-06 2024-09-16T20:50:42.924 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.925 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.925 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:42.925 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790559+0000", 2024-09-16T20:50:42.925 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3494e-05 2024-09-16T20:50:42.925 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.925 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.925 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:42.925 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790566+0000", 2024-09-16T20:50:42.925 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7649999999999993e-06 2024-09-16T20:50:42.925 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.926 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.926 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:42.926 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:09.557198+0000", 2024-09-16T20:50:42.926 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.76663136200000004 2024-09-16T20:50:42.926 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.926 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.926 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:42.926 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:09.557211+0000", 2024-09-16T20:50:42.926 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3458999999999999e-05 2024-09-16T20:50:42.926 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.926 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:42.926 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.927 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.927 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.927 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5512.0:450 3.2 3:44e0cffa:::rbd_header.110ef0d7a535:head [watch reconnect cookie 140263298018912 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:50:42.927 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:50:42.927 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 66.355992004000001, 2024-09-16T20:50:42.927 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0084581939999999, 2024-09-16T20:50:42.927 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:42.927 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:42.927 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:42.927 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:42.928 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5512", 2024-09-16T20:50:42.928 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:50:42.928 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 450 2024-09-16T20:50:42.928 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.928 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:42.928 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.928 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:42.928 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:50:42.928 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.928 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.928 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.929 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:42.929 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:50:42.929 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.929 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.929 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.929 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:42.929 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562512+0000", 2024-09-16T20:50:42.929 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.009e-06 2024-09-16T20:50:42.929 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.929 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.929 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:42.930 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562517+0000", 2024-09-16T20:50:42.930 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9339999999999997e-06 2024-09-16T20:50:42.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.930 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.930 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:42.930 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562530+0000", 2024-09-16T20:50:42.930 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3495e-05 2024-09-16T20:50:42.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.930 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.930 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:42.930 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562536+0000", 2024-09-16T20:50:42.931 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5690000000000004e-06 2024-09-16T20:50:42.931 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.931 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.931 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:42.931 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562591+0000", 2024-09-16T20:50:42.931 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5761999999999999e-05 2024-09-16T20:50:42.931 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.931 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.931 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:50:42.931 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562599+0000", 2024-09-16T20:50:42.931 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9179999999999994e-06 2024-09-16T20:50:42.931 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.931 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.932 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:42.932 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.565971+0000", 2024-09-16T20:50:42.932 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003371546 2024-09-16T20:50:42.932 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.932 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.932 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:42.932 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566019+0000", 2024-09-16T20:50:42.932 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7849000000000002e-05 2024-09-16T20:50:42.932 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.932 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.932 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:42.932 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.569382+0000", 2024-09-16T20:50:42.933 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.003362833 2024-09-16T20:50:42.933 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.933 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.933 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:42.933 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.569529+0000", 2024-09-16T20:50:42.933 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000147823 2024-09-16T20:50:42.933 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.933 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.933 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:42.933 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570969+0000", 2024-09-16T20:50:42.933 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001439456 2024-09-16T20:50:42.934 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.934 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:42.934 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.934 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.934 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.934 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:14996 3.2 3:44e0cffa:::rbd_header.110ef0d7a535:head [watch reconnect cookie 140213436269840 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:50:42.934 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:50:42.934 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 66.355943592000003, 2024-09-16T20:50:42.934 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.008813943, 2024-09-16T20:50:42.934 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:42.934 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:42.934 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:42.935 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:42.935 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:50:42.935 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:42.935 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 14996 2024-09-16T20:50:42.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.935 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:42.935 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.935 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:42.935 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:50:42.935 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.936 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.936 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:42.936 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:50:42.936 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.936 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.936 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.936 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:42.936 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562560+0000", 2024-09-16T20:50:42.936 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.06e-07 2024-09-16T20:50:42.937 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.937 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.937 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:42.937 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562563+0000", 2024-09-16T20:50:42.937 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.495e-06 2024-09-16T20:50:42.937 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.937 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.937 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:42.937 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562571+0000", 2024-09-16T20:50:42.937 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1950000000000003e-06 2024-09-16T20:50:42.937 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.938 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.938 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:42.938 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562579+0000", 2024-09-16T20:50:42.938 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2649999999999999e-06 2024-09-16T20:50:42.938 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.938 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.938 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:42.938 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562673+0000", 2024-09-16T20:50:42.938 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4118999999999998e-05 2024-09-16T20:50:42.938 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.938 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.939 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:50:42.939 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562677+0000", 2024-09-16T20:50:42.939 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5179999999999998e-06 2024-09-16T20:50:42.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.939 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:42.939 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566461+0000", 2024-09-16T20:50:42.939 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0037837259999999998 2024-09-16T20:50:42.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.939 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:42.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566468+0000", 2024-09-16T20:50:42.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3350000000000001e-06 2024-09-16T20:50:42.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:42.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570306+0000", 2024-09-16T20:50:42.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.003837997 2024-09-16T20:50:42.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:42.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570392+0000", 2024-09-16T20:50:42.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5872999999999996e-05 2024-09-16T20:50:42.941 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.941 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.941 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:42.941 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571373+0000", 2024-09-16T20:50:42.941 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00098071399999999993 2024-09-16T20:50:42.941 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.941 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:42.941 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.941 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.941 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.941 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5255.0:1931 3.2 3:47882819:::rbd_data.110eb5f95afc.0000000000000009:head [read 217088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:50:42.941 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:50:42.942 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 66.355904842000001, 2024-09-16T20:50:42.942 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0082752880000001, 2024-09-16T20:50:42.942 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:42.942 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:42.942 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:42.942 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:42.942 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5255", 2024-09-16T20:50:42.942 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:50:42.942 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1931 2024-09-16T20:50:42.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.942 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:42.942 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.943 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:42.943 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:50:42.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.943 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:42.943 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:50:42.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.943 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:42.943 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:50:42.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9699999999999996e-07 2024-09-16T20:50:42.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.944 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:42.944 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562601+0000", 2024-09-16T20:50:42.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8600000000000001e-06 2024-09-16T20:50:42.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.944 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:42.944 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562608+0000", 2024-09-16T20:50:42.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9e-06 2024-09-16T20:50:42.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.945 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:42.945 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562615+0000", 2024-09-16T20:50:42.945 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8179999999999998e-06 2024-09-16T20:50:42.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.945 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:42.945 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562703+0000", 2024-09-16T20:50:42.945 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7811000000000003e-05 2024-09-16T20:50:42.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.945 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:50:42.946 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562706+0000", 2024-09-16T20:50:42.946 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4920000000000002e-06 2024-09-16T20:50:42.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.946 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.946 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:42.946 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566685+0000", 2024-09-16T20:50:42.946 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0039792780000000002 2024-09-16T20:50:42.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.946 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.946 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:42.946 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566693+0000", 2024-09-16T20:50:42.947 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4730000000000003e-06 2024-09-16T20:50:42.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.947 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.947 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:42.947 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570737+0000", 2024-09-16T20:50:42.947 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0040436749999999 2024-09-16T20:50:42.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.947 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.947 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:42.947 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570873+0000", 2024-09-16T20:50:42.947 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000136484 2024-09-16T20:50:42.947 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.947 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:42.948 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.948 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5255.0:1939 3.2 3:47882819:::rbd_data.110eb5f95afc.0000000000000009:head [read 233472~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:50:42.948 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:50:42.948 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 66.355749975999998, 2024-09-16T20:50:42.948 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0084657589999999, 2024-09-16T20:50:42.948 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:42.948 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:42.948 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:42.948 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:42.949 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5255", 2024-09-16T20:50:42.949 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:50:42.949 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1939 2024-09-16T20:50:42.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.949 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:42.949 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.949 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:42.949 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:50:42.949 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.949 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.949 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:42.950 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:50:42.950 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.950 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.950 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:42.950 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:50:42.950 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5199999999999997e-07 2024-09-16T20:50:42.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.950 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.950 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:42.950 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562757+0000", 2024-09-16T20:50:42.950 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4979999999999998e-06 2024-09-16T20:50:42.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.951 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.951 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:42.951 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562764+0000", 2024-09-16T20:50:42.951 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1470000000000004e-06 2024-09-16T20:50:42.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.951 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.951 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:42.951 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562768+0000", 2024-09-16T20:50:42.951 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7929999999999999e-06 2024-09-16T20:50:42.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.951 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.951 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:42.951 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562805+0000", 2024-09-16T20:50:42.952 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7477e-05 2024-09-16T20:50:42.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.952 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.952 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting_for_map not empty", 2024-09-16T20:50:42.952 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562810+0000", 2024-09-16T20:50:42.952 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0819999999999998e-06 2024-09-16T20:50:42.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.952 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.952 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:42.952 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566717+0000", 2024-09-16T20:50:42.952 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0039072600000000001 2024-09-16T20:50:42.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.952 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.953 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:42.953 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566724+0000", 2024-09-16T20:50:42.953 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4880000000000004e-06 2024-09-16T20:50:42.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.953 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.953 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:42.953 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571009+0000", 2024-09-16T20:50:42.953 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.004284677 2024-09-16T20:50:42.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.953 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.953 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:42.953 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571218+0000", 2024-09-16T20:50:42.953 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00020978500000000001 2024-09-16T20:50:42.953 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.954 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:42.954 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.954 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.954 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:11020 3.1f 3:fee2d1dd:::rbd_data.110ea93691bb.000000000000000e:23 [read 955392~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:42.954 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.693811+0000", 2024-09-16T20:50:42.954 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 52.224691319000001, 2024-09-16T20:50:42.954 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056833005999999998, 2024-09-16T20:50:42.954 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:42.954 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:42.954 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:42.954 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:42.954 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:42.955 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:42.955 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 11020 2024-09-16T20:50:42.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.955 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:42.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.955 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:42.955 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693811+0000", 2024-09-16T20:50:42.955 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.955 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:42.955 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693811+0000", 2024-09-16T20:50:42.956 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.956 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:42.956 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693812+0000", 2024-09-16T20:50:42.956 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6100000000000001e-07 2024-09-16T20:50:42.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.956 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:42.956 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693815+0000", 2024-09-16T20:50:42.956 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7389999999999999e-06 2024-09-16T20:50:42.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.957 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:42.957 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693823+0000", 2024-09-16T20:50:42.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0439999999999996e-06 2024-09-16T20:50:42.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.957 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:42.957 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693828+0000", 2024-09-16T20:50:42.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7440000000000004e-06 2024-09-16T20:50:42.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.957 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:42.958 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750357+0000", 2024-09-16T20:50:42.958 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056529192999999998 2024-09-16T20:50:42.958 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.958 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.958 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:42.958 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750429+0000", 2024-09-16T20:50:42.958 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1557000000000006e-05 2024-09-16T20:50:42.958 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.958 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.958 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:42.959 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750644+0000", 2024-09-16T20:50:42.959 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00021526800000000001 2024-09-16T20:50:42.959 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.959 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:42.959 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.959 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.959 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.959 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:11021 3.1f 3:fee2d1dd:::rbd_data.110ea93691bb.000000000000000e:23 [read 960512~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:42.959 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.693837+0000", 2024-09-16T20:50:42.959 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 52.224665430999998, 2024-09-16T20:50:42.959 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057155138000000001, 2024-09-16T20:50:42.959 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:42.960 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:42.960 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:42.960 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:42.960 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:42.960 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:42.960 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 11021 2024-09-16T20:50:42.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.960 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:42.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.960 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:42.960 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693837+0000", 2024-09-16T20:50:42.961 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.961 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.961 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:42.961 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693837+0000", 2024-09-16T20:50:42.961 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.961 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.961 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:42.961 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699566+0000", 2024-09-16T20:50:42.961 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057283639999999997 2024-09-16T20:50:42.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.962 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:42.962 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699571+0000", 2024-09-16T20:50:42.962 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.604e-06 2024-09-16T20:50:42.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.962 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:42.962 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699583+0000", 2024-09-16T20:50:42.962 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1372999999999999e-05 2024-09-16T20:50:42.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.962 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:42.963 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699591+0000", 2024-09-16T20:50:42.963 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5259999999999993e-06 2024-09-16T20:50:42.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.963 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.963 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:42.963 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750674+0000", 2024-09-16T20:50:42.963 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051083154999999998 2024-09-16T20:50:42.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.963 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.963 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:42.963 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750745+0000", 2024-09-16T20:50:42.963 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0933999999999995e-05 2024-09-16T20:50:42.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.964 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.964 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:42.964 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750992+0000", 2024-09-16T20:50:42.964 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00024718199999999998 2024-09-16T20:50:42.964 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.964 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:42.964 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.964 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.964 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8790 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2590720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:42.964 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057257+0000", 2024-09-16T20:50:42.965 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 41.861246111, 2024-09-16T20:50:42.965 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056791169000000002, 2024-09-16T20:50:42.965 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:42.965 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:42.965 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:42.965 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:42.965 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:42.965 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:42.965 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8790 2024-09-16T20:50:42.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.965 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:42.965 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.965 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:42.966 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057257+0000", 2024-09-16T20:50:42.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.966 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:42.966 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057257+0000", 2024-09-16T20:50:42.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.966 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:42.966 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057257+0000", 2024-09-16T20:50:42.967 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6600000000000002e-07 2024-09-16T20:50:42.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.967 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:42.967 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057260+0000", 2024-09-16T20:50:42.967 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7240000000000001e-06 2024-09-16T20:50:42.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.967 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:42.967 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057267+0000", 2024-09-16T20:50:42.967 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0600000000000002e-06 2024-09-16T20:50:42.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.967 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:42.968 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057272+0000", 2024-09-16T20:50:42.968 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0390000000000004e-06 2024-09-16T20:50:42.968 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.968 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.968 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:42.968 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.113950+0000", 2024-09-16T20:50:42.968 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056677989999999998 2024-09-16T20:50:42.968 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.968 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.968 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:42.968 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.113993+0000", 2024-09-16T20:50:42.968 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3205999999999999e-05 2024-09-16T20:50:42.969 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.969 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.969 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:42.969 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114048+0000", 2024-09-16T20:50:42.969 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4684e-05 2024-09-16T20:50:42.969 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.969 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:42.969 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.969 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.969 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.969 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8791 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2332672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:42.969 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057279+0000", 2024-09-16T20:50:42.970 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 41.861223696000003, 2024-09-16T20:50:42.970 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056890067000000002, 2024-09-16T20:50:42.970 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:42.970 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:42.970 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:42.970 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:42.970 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:42.970 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:42.970 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8791 2024-09-16T20:50:42.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.970 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:42.970 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.971 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:42.971 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057279+0000", 2024-09-16T20:50:42.971 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.971 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.971 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:42.971 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057279+0000", 2024-09-16T20:50:42.971 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.971 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.971 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:42.971 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057279+0000", 2024-09-16T20:50:42.971 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.15e-07 2024-09-16T20:50:42.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.972 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.972 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:42.972 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057281+0000", 2024-09-16T20:50:42.972 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8080000000000001e-06 2024-09-16T20:50:42.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.972 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.972 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:42.972 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057287+0000", 2024-09-16T20:50:42.972 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5999999999999997e-06 2024-09-16T20:50:42.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.973 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.973 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:42.973 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057291+0000", 2024-09-16T20:50:42.973 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5990000000000004e-06 2024-09-16T20:50:42.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.973 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.973 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:42.973 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114078+0000", 2024-09-16T20:50:42.973 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056786246999999998 2024-09-16T20:50:42.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.973 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.973 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:42.974 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114118+0000", 2024-09-16T20:50:42.974 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9898999999999999e-05 2024-09-16T20:50:42.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.974 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:42.974 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114169+0000", 2024-09-16T20:50:42.974 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1499000000000003e-05 2024-09-16T20:50:42.974 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.974 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:42.974 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.975 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8792 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2609152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:42.975 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:50:42.975 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 41.861205714, 2024-09-16T20:50:42.975 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057003524999999999, 2024-09-16T20:50:42.975 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:42.975 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:42.975 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:42.975 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:42.975 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:42.975 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:42.975 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8792 2024-09-16T20:50:42.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.976 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:42.976 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.976 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:42.976 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:50:42.976 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.976 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.976 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:42.976 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:50:42.976 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.976 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.976 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:42.977 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:50:42.977 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0400000000000002e-07 2024-09-16T20:50:42.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.977 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:42.977 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057299+0000", 2024-09-16T20:50:42.977 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7460000000000001e-06 2024-09-16T20:50:42.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.977 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:42.977 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057304+0000", 2024-09-16T20:50:42.977 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3210000000000004e-06 2024-09-16T20:50:42.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.978 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.978 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:42.978 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057310+0000", 2024-09-16T20:50:42.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4360000000000001e-06 2024-09-16T20:50:42.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.978 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.978 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:42.978 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114200+0000", 2024-09-16T20:50:42.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056889954 2024-09-16T20:50:42.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.979 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.979 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:42.979 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114243+0000", 2024-09-16T20:50:42.979 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2998999999999999e-05 2024-09-16T20:50:42.979 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.979 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.979 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:42.979 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114300+0000", 2024-09-16T20:50:42.979 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7765e-05 2024-09-16T20:50:42.979 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.979 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:42.979 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.979 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.980 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.980 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8794 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2351104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:42.980 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057327+0000", 2024-09-16T20:50:42.980 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 41.86117539, 2024-09-16T20:50:42.980 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057101130999999999, 2024-09-16T20:50:42.980 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:42.980 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:42.980 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:42.980 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:42.980 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:42.980 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:42.980 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8794 2024-09-16T20:50:42.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.981 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:42.981 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.981 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:42.981 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057327+0000", 2024-09-16T20:50:42.981 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.981 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.981 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:42.981 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057327+0000", 2024-09-16T20:50:42.981 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.982 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.982 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:42.982 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057328+0000", 2024-09-16T20:50:42.982 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7399999999999999e-07 2024-09-16T20:50:42.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.982 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.982 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:42.982 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057330+0000", 2024-09-16T20:50:42.982 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2199999999999999e-06 2024-09-16T20:50:42.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.982 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.982 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:42.983 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057336+0000", 2024-09-16T20:50:42.983 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1240000000000002e-06 2024-09-16T20:50:42.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.983 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.983 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:42.983 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057341+0000", 2024-09-16T20:50:42.983 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8470000000000003e-06 2024-09-16T20:50:42.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.983 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.983 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:42.983 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114334+0000", 2024-09-16T20:50:42.983 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056993267 2024-09-16T20:50:42.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.984 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.984 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:42.984 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114376+0000", 2024-09-16T20:50:42.984 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1638999999999999e-05 2024-09-16T20:50:42.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.984 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.984 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:42.984 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114428+0000", 2024-09-16T20:50:42.984 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2660000000000001e-05 2024-09-16T20:50:42.984 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.984 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:42.984 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.985 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.985 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8798 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2297856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:42.985 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:50:42.985 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 41.861155662999998, 2024-09-16T20:50:42.985 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057277041000000001, 2024-09-16T20:50:42.985 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:42.985 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:42.985 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:42.985 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:42.985 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:42.985 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:42.985 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8798 2024-09-16T20:50:42.986 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.986 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:42.986 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.986 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:42.986 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:50:42.986 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.986 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.986 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.986 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:42.986 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:50:42.986 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.986 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.987 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.987 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:42.987 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:50:42.987 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2800000000000003e-07 2024-09-16T20:50:42.987 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.987 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.987 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:42.987 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057354+0000", 2024-09-16T20:50:42.987 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7780000000000002e-06 2024-09-16T20:50:42.987 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.987 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.987 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:42.987 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057360+0000", 2024-09-16T20:50:42.988 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0419999999999999e-06 2024-09-16T20:50:42.988 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.988 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.988 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:42.988 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057365+0000", 2024-09-16T20:50:42.988 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7929999999999997e-06 2024-09-16T20:50:42.988 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.988 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.988 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:42.988 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114460+0000", 2024-09-16T20:50:42.988 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057094662999999997 2024-09-16T20:50:42.988 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.988 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.989 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:42.989 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114532+0000", 2024-09-16T20:50:42.989 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2881999999999998e-05 2024-09-16T20:50:42.989 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.989 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.989 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:42.989 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114624+0000", 2024-09-16T20:50:42.989 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1555e-05 2024-09-16T20:50:42.989 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.989 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:42.989 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.989 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.989 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.990 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8800 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2316288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:42.990 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:50:42.990 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 41.861131724000003, 2024-09-16T20:50:42.990 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.05746946, 2024-09-16T20:50:42.990 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:42.990 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:42.990 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:42.990 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:42.990 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:42.990 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:42.990 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8800 2024-09-16T20:50:42.990 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.990 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:42.991 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.991 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:42.991 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:50:42.991 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.991 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.991 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.991 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:42.991 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:50:42.991 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.991 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.991 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.991 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:42.992 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:50:42.992 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5199999999999998e-07 2024-09-16T20:50:42.992 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.992 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.992 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:42.992 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057373+0000", 2024-09-16T20:50:42.992 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.79e-06 2024-09-16T20:50:42.992 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.992 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.992 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:42.992 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057379+0000", 2024-09-16T20:50:42.992 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6699999999999999e-06 2024-09-16T20:50:42.992 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.993 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.993 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:42.993 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057386+0000", 2024-09-16T20:50:42.993 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3170000000000004e-06 2024-09-16T20:50:42.993 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.993 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.993 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:42.993 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114655+0000", 2024-09-16T20:50:42.993 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057269420000000001 2024-09-16T20:50:42.993 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.993 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.993 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:42.993 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114734+0000", 2024-09-16T20:50:42.994 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8084000000000007e-05 2024-09-16T20:50:42.994 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.994 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.994 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:42.994 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114840+0000", 2024-09-16T20:50:42.994 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000106827 2024-09-16T20:50:42.994 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.994 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:42.994 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.994 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.994 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.994 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8801 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2334720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:42.994 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:50:42.995 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 41.861110693000001, 2024-09-16T20:50:42.995 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057594682000000001, 2024-09-16T20:50:42.995 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:42.995 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:42.995 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:42.995 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:42.995 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:42.995 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:42.995 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8801 2024-09-16T20:50:42.995 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.995 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:42.995 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.996 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:42.996 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:50:42.996 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.996 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.996 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.996 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:42.996 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:50:42.996 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:42.996 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.996 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.996 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:42.996 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:50:42.996 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6899999999999998e-07 2024-09-16T20:50:42.997 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.997 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.997 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:42.997 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057394+0000", 2024-09-16T20:50:42.997 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8300000000000001e-06 2024-09-16T20:50:42.997 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.997 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.997 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:42.997 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057400+0000", 2024-09-16T20:50:42.997 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6500000000000001e-06 2024-09-16T20:50:42.997 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.997 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.998 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:42.998 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057404+0000", 2024-09-16T20:50:42.998 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4669999999999998e-06 2024-09-16T20:50:42.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.998 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.998 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:42.998 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114874+0000", 2024-09-16T20:50:42.998 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057469602000000002 2024-09-16T20:50:42.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.998 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.998 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:42.998 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114933+0000", 2024-09-16T20:50:42.999 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9500000000000003e-05 2024-09-16T20:50:42.999 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.999 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.999 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:42.999 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114987+0000", 2024-09-16T20:50:42.999 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3263999999999997e-05 2024-09-16T20:50:42.999 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.999 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:42.999 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:42.999 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:42.999 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:42.999 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8802 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2353152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:43.000 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:50:43.000 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 41.861062007999998, 2024-09-16T20:50:43.000 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057714079000000001, 2024-09-16T20:50:43.000 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:43.000 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:43.001 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:43.001 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:43.001 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:43.001 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:43.001 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8802 2024-09-16T20:50:43.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.001 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:43.001 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.001 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:43.001 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:50:43.001 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:43.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.002 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:43.002 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:50:43.002 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:43.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.002 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:43.002 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:50:43.002 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4999999999999998e-07 2024-09-16T20:50:43.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.002 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:43.002 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057446+0000", 2024-09-16T20:50:43.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4240000000000003e-06 2024-09-16T20:50:43.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.003 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.003 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:43.003 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057452+0000", 2024-09-16T20:50:43.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8379999999999996e-06 2024-09-16T20:50:43.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.003 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.003 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:43.003 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057460+0000", 2024-09-16T20:50:43.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9629999999999998e-06 2024-09-16T20:50:43.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.003 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.004 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:43.004 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115015+0000", 2024-09-16T20:50:43.004 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057554389999999997 2024-09-16T20:50:43.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.004 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.004 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:43.004 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115082+0000", 2024-09-16T20:50:43.004 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7733999999999999e-05 2024-09-16T20:50:43.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.004 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.004 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:43.004 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115155+0000", 2024-09-16T20:50:43.005 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2280000000000006e-05 2024-09-16T20:50:43.005 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:43.005 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:43.005 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:43.005 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.005 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.005 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8804 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2555904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:43.005 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:50:43.005 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 41.861035692000002, 2024-09-16T20:50:43.005 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057833862, 2024-09-16T20:50:43.005 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:43.005 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:43.006 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:43.006 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:43.006 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:43.006 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:43.006 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8804 2024-09-16T20:50:43.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.006 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:43.006 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.006 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:43.006 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:50:43.006 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:43.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.006 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.007 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:43.007 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:50:43.007 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:43.007 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.007 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.007 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:43.007 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:50:43.007 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7899999999999999e-07 2024-09-16T20:50:43.007 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.007 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.007 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:43.007 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057469+0000", 2024-09-16T20:50:43.007 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.001e-06 2024-09-16T20:50:43.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.008 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:43.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057475+0000", 2024-09-16T20:50:43.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7629999999999997e-06 2024-09-16T20:50:43.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.008 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:43.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057480+0000", 2024-09-16T20:50:43.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8069999999999999e-06 2024-09-16T20:50:43.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.008 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:43.009 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115183+0000", 2024-09-16T20:50:43.009 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057703364 2024-09-16T20:50:43.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.009 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.009 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:43.009 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115227+0000", 2024-09-16T20:50:43.009 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3903999999999998e-05 2024-09-16T20:50:43.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.009 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.009 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:43.009 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115301+0000", 2024-09-16T20:50:43.009 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3543999999999999e-05 2024-09-16T20:50:43.010 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:43.010 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:43.010 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:43.010 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.010 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.010 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8805 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2574336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:43.010 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057487+0000", 2024-09-16T20:50:43.010 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 41.861015363999996, 2024-09-16T20:50:43.010 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058021027000000003, 2024-09-16T20:50:43.010 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:43.010 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:43.010 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:43.010 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:43.011 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:43.011 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:43.011 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8805 2024-09-16T20:50:43.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.011 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:43.011 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.011 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:43.011 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057487+0000", 2024-09-16T20:50:43.011 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:43.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.011 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.011 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:43.011 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057487+0000", 2024-09-16T20:50:43.012 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:43.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.012 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.012 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:43.012 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057488+0000", 2024-09-16T20:50:43.012 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5199999999999998e-07 2024-09-16T20:50:43.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.012 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.012 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:43.012 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057490+0000", 2024-09-16T20:50:43.012 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.939e-06 2024-09-16T20:50:43.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.012 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.013 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:43.013 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057495+0000", 2024-09-16T20:50:43.013 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6439999999999997e-06 2024-09-16T20:50:43.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.013 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.013 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:43.013 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057500+0000", 2024-09-16T20:50:43.013 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8119999999999998e-06 2024-09-16T20:50:43.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.013 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.013 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:43.013 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115341+0000", 2024-09-16T20:50:43.013 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057841341999999997 2024-09-16T20:50:43.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.014 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.014 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:43.014 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115422+0000", 2024-09-16T20:50:43.014 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1003999999999999e-05 2024-09-16T20:50:43.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.014 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.014 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:43.014 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115508+0000", 2024-09-16T20:50:43.014 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5933999999999994e-05 2024-09-16T20:50:43.014 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:43.014 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:43.014 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:43.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.015 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.015 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8807 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2592768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:43.015 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057505+0000", 2024-09-16T20:50:43.015 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 41.860997154000003, 2024-09-16T20:50:43.015 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058186633000000001, 2024-09-16T20:50:43.015 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:43.015 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:43.015 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:43.015 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:43.015 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:43.015 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:43.015 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8807 2024-09-16T20:50:43.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.016 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:43.016 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.016 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:43.016 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057505+0000", 2024-09-16T20:50:43.016 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:43.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.016 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.016 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:43.016 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057505+0000", 2024-09-16T20:50:43.016 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:43.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.016 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.017 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:43.017 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057506+0000", 2024-09-16T20:50:43.017 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.27e-07 2024-09-16T20:50:43.017 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.017 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.017 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:43.017 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057508+0000", 2024-09-16T20:50:43.017 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7400000000000001e-06 2024-09-16T20:50:43.017 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.017 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.017 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:43.017 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057513+0000", 2024-09-16T20:50:43.018 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.468e-06 2024-09-16T20:50:43.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.018 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.018 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:43.018 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057517+0000", 2024-09-16T20:50:43.018 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4329999999999997e-06 2024-09-16T20:50:43.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.018 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.018 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:43.018 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115538+0000", 2024-09-16T20:50:43.018 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058020355000000003 2024-09-16T20:50:43.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.018 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.019 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:43.019 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115616+0000", 2024-09-16T20:50:43.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7998999999999996e-05 2024-09-16T20:50:43.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.019 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:43.019 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115692+0000", 2024-09-16T20:50:43.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6310999999999995e-05 2024-09-16T20:50:43.019 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:43.019 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:43.019 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:43.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.020 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.020 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8808 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2611200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:43.020 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:50:43.020 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 41.860979893, 2024-09-16T20:50:43.020 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058354046, 2024-09-16T20:50:43.020 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:43.020 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:43.020 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:43.020 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:43.020 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:43.020 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:43.021 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8808 2024-09-16T20:50:43.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.021 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:43.021 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.021 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:43.021 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:50:43.021 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:43.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.021 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.021 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:43.021 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:50:43.021 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:43.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.022 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.022 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:43.022 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:50:43.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2500000000000001e-07 2024-09-16T20:50:43.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.022 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.022 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:43.022 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057525+0000", 2024-09-16T20:50:43.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5549999999999999e-06 2024-09-16T20:50:43.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.022 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.022 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:43.023 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057530+0000", 2024-09-16T20:50:43.023 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5489999999999997e-06 2024-09-16T20:50:43.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.023 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.023 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:43.023 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057535+0000", 2024-09-16T20:50:43.023 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5709999999999999e-06 2024-09-16T20:50:43.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.023 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.023 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:43.023 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115722+0000", 2024-09-16T20:50:43.023 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058187323999999999 2024-09-16T20:50:43.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.024 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:43.024 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115799+0000", 2024-09-16T20:50:43.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6445000000000004e-05 2024-09-16T20:50:43.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.024 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:43.024 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115877+0000", 2024-09-16T20:50:43.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8276999999999996e-05 2024-09-16T20:50:43.024 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:43.024 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:43.024 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:43.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.025 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:2182 3.17 3:edb5e5cd:::rbd_data.110e61479aee.0000000000000014:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write 0~5120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-16T20:50:43.025 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:22.734609+0000", 2024-09-16T20:50:43.025 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 20.183893646000001, 2024-09-16T20:50:43.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.124890894, 2024-09-16T20:50:43.025 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:43.025 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:43.025 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:43.025 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:43.025 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:43.025 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:43.025 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 2182 2024-09-16T20:50:43.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.026 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:43.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.026 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:43.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734609+0000", 2024-09-16T20:50:43.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:43.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.026 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:43.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734609+0000", 2024-09-16T20:50:43.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:43.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.027 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:43.027 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734611+0000", 2024-09-16T20:50:43.027 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9599999999999999e-06 2024-09-16T20:50:43.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.027 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.027 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:43.027 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734628+0000", 2024-09-16T20:50:43.027 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.715e-05 2024-09-16T20:50:43.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.027 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.027 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:43.027 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734656+0000", 2024-09-16T20:50:43.027 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7759999999999998e-05 2024-09-16T20:50:43.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.028 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.028 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:43.028 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734669+0000", 2024-09-16T20:50:43.028 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3315999999999999e-05 2024-09-16T20:50:43.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.028 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.028 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:43.028 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734826+0000", 2024-09-16T20:50:43.028 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015668499999999999 2024-09-16T20:50:43.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.029 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:43.029 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.735048+0000", 2024-09-16T20:50:43.029 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00022179 2024-09-16T20:50:43.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.029 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:43.029 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.735248+0000", 2024-09-16T20:50:43.029 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00020010399999999999 2024-09-16T20:50:43.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.029 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:50:43.029 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.736252+0000", 2024-09-16T20:50:43.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010039739999999999 2024-09-16T20:50:43.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.030 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:43.030 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.859340+0000", 2024-09-16T20:50:43.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.12308846599999999 2024-09-16T20:50:43.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.030 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:50:43.030 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.859431+0000", 2024-09-16T20:50:43.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1099000000000003e-05 2024-09-16T20:50:43.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:43.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:43.031 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:43.031 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.859500+0000", 2024-09-16T20:50:43.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8590000000000006e-05 2024-09-16T20:50:43.031 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:43.031 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:43.031 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:43.031 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:43.031 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:43.031 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:43.032 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:50:43.034 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:43.032+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:43.034 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:50:43.035 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:50:43.133 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:43.133+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:43.182 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:43.182 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:43.191 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:50:43.235 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:43.234+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:43.298 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:43.299 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:43.307 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:50:43.351 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:43.350+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:43.413 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:43.414 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:43.422 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:50:43.436 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:43.436+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:43.537 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:43.536+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:43.569 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:43.569 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:43.575 INFO:teuthology.orchestra.run.smithi029.stderr:noscrub is set 2024-09-16T20:50:43.583 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:50:43.638 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:43.637+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:43.739 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:43.738+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:43.747 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:43.747 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:43.757 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:50:43.839 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:43.838+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:43.864 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:43.864 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:43.871 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:50:43.956 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:43.955+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:44.019 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:44.020 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:44.028 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:50:44.041 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:44.040+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:44.141 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:44.140+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:44.175 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:44.175 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:44.184 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:50:44.268 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:44.267+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:44.332 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:44.332 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:44.341 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:50:44.342 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:44.342+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:44.443 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:44.443+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:44.461 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:44.461 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:44.470 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:50:44.544 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:44.543+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:44.626 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:44.627 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:44.635 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:50:44.681 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:44.681+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:44.745 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:44.746 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:44.746 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:44.745+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:44.753 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:50:44.847 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:44.846+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:44.901 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:44.901 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:44.910 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:50:44.948 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:44.947+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:45.016 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:45.016 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:45.025 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:50:45.036 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T20:50:45.081 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:45.080+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:45.132 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:45.133 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:45.141 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:50:45.163 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:50:45.163 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:50:45.180 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:50:45.180 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:50:45.185 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:45.185+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:45.196 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:50:45.196 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:50:45.214 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:50:45.215 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:50:45.231 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:50:45.231 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:50:45.248 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:50:45.248 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:50:45.250 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:45.250 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:50:45.250 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:50:45.250 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:45.250 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:45.249+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:45.259 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:50:45.351 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:45.350+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:45.406 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:45.407 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:50:45.407 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:50:45.407 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:50:45.407 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:45.415 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:50:45.451 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:45.450+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:45.553 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:45.552+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:45.566 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:45.566 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:50:45.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:50:45.566 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:50:45.566 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.566 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5286.0:10643 3.17 3.f6273fb7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-16T20:50:45.566 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:50:45.566 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 96.773723415999996, 2024-09-16T20:50:45.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.76672347200000002, 2024-09-16T20:50:45.567 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:45.567 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:45.567 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:45.567 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:45.567 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5286", 2024-09-16T20:50:45.567 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/4135433989", 2024-09-16T20:50:45.567 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 10643 2024-09-16T20:50:45.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.567 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:45.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.567 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:45.567 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:50:45.567 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:45.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.568 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:45.568 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:50:45.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:45.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.568 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:45.568 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790489+0000", 2024-09-16T20:50:45.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5910000000000001e-06 2024-09-16T20:50:45.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.568 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:45.569 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790493+0000", 2024-09-16T20:50:45.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2669999999999998e-06 2024-09-16T20:50:45.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.569 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:45.569 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790500+0000", 2024-09-16T20:50:45.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3780000000000003e-06 2024-09-16T20:50:45.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.569 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:45.569 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790505+0000", 2024-09-16T20:50:45.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1560000000000003e-06 2024-09-16T20:50:45.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.570 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:45.570 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790559+0000", 2024-09-16T20:50:45.570 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3494e-05 2024-09-16T20:50:45.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.570 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:45.570 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790566+0000", 2024-09-16T20:50:45.570 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7649999999999993e-06 2024-09-16T20:50:45.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.570 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:45.570 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:09.557198+0000", 2024-09-16T20:50:45.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.76663136200000004 2024-09-16T20:50:45.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.571 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.571 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:45.571 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:09.557211+0000", 2024-09-16T20:50:45.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3458999999999999e-05 2024-09-16T20:50:45.571 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:45.571 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:45.571 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:45.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.571 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.571 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5512.0:450 3.2 3:44e0cffa:::rbd_header.110ef0d7a535:head [watch reconnect cookie 140263298018912 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:50:45.571 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:50:45.571 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 69.001700502999995, 2024-09-16T20:50:45.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0084581939999999, 2024-09-16T20:50:45.572 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:45.572 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:45.572 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:45.572 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:45.572 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5512", 2024-09-16T20:50:45.572 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:50:45.572 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 450 2024-09-16T20:50:45.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.572 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:45.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.572 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:45.572 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:50:45.573 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:45.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.573 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.573 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:45.573 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:50:45.573 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:45.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.573 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.573 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:45.573 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562512+0000", 2024-09-16T20:50:45.573 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.009e-06 2024-09-16T20:50:45.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.573 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.573 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:45.574 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562517+0000", 2024-09-16T20:50:45.574 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9339999999999997e-06 2024-09-16T20:50:45.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.574 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:45.574 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562530+0000", 2024-09-16T20:50:45.574 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3495e-05 2024-09-16T20:50:45.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.574 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:45.574 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562536+0000", 2024-09-16T20:50:45.574 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5690000000000004e-06 2024-09-16T20:50:45.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.575 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:45.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562591+0000", 2024-09-16T20:50:45.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5761999999999999e-05 2024-09-16T20:50:45.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.575 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:50:45.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562599+0000", 2024-09-16T20:50:45.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9179999999999994e-06 2024-09-16T20:50:45.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.575 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:45.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.565971+0000", 2024-09-16T20:50:45.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003371546 2024-09-16T20:50:45.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:45.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566019+0000", 2024-09-16T20:50:45.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7849000000000002e-05 2024-09-16T20:50:45.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:45.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.569382+0000", 2024-09-16T20:50:45.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.003362833 2024-09-16T20:50:45.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:45.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.569529+0000", 2024-09-16T20:50:45.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000147823 2024-09-16T20:50:45.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.577 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:45.577 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570969+0000", 2024-09-16T20:50:45.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001439456 2024-09-16T20:50:45.577 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:45.577 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:45.577 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:45.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.577 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:14996 3.2 3:44e0cffa:::rbd_header.110ef0d7a535:head [watch reconnect cookie 140213436269840 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:50:45.577 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:50:45.577 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 69.001652090999997, 2024-09-16T20:50:45.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.008813943, 2024-09-16T20:50:45.578 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:45.578 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:45.578 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:45.578 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:45.578 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:50:45.578 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:45.578 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 14996 2024-09-16T20:50:45.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.578 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:45.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.578 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:45.578 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:50:45.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:45.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:45.579 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:50:45.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:45.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:45.579 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562560+0000", 2024-09-16T20:50:45.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.06e-07 2024-09-16T20:50:45.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:45.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562563+0000", 2024-09-16T20:50:45.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.495e-06 2024-09-16T20:50:45.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:45.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562571+0000", 2024-09-16T20:50:45.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1950000000000003e-06 2024-09-16T20:50:45.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:45.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562579+0000", 2024-09-16T20:50:45.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2649999999999999e-06 2024-09-16T20:50:45.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:45.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562673+0000", 2024-09-16T20:50:45.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4118999999999998e-05 2024-09-16T20:50:45.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:50:45.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562677+0000", 2024-09-16T20:50:45.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5179999999999998e-06 2024-09-16T20:50:45.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:45.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566461+0000", 2024-09-16T20:50:45.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0037837259999999998 2024-09-16T20:50:45.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:45.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566468+0000", 2024-09-16T20:50:45.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3350000000000001e-06 2024-09-16T20:50:45.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:45.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570306+0000", 2024-09-16T20:50:45.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.003837997 2024-09-16T20:50:45.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:45.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570392+0000", 2024-09-16T20:50:45.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5872999999999996e-05 2024-09-16T20:50:45.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:45.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571373+0000", 2024-09-16T20:50:45.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00098071399999999993 2024-09-16T20:50:45.583 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:45.583 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:45.583 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:45.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.583 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5255.0:1931 3.2 3:47882819:::rbd_data.110eb5f95afc.0000000000000009:head [read 217088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:50:45.583 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:50:45.583 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 69.001613340999995, 2024-09-16T20:50:45.584 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0082752880000001, 2024-09-16T20:50:45.584 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:45.584 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:45.584 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:45.584 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:45.584 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5255", 2024-09-16T20:50:45.584 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:50:45.584 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1931 2024-09-16T20:50:45.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.584 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:45.584 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.584 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:45.584 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:50:45.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:45.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:45.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:50:45.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:45.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:45.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:50:45.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9699999999999996e-07 2024-09-16T20:50:45.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:45.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562601+0000", 2024-09-16T20:50:45.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8600000000000001e-06 2024-09-16T20:50:45.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:45.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562608+0000", 2024-09-16T20:50:45.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9e-06 2024-09-16T20:50:45.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:45.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562615+0000", 2024-09-16T20:50:45.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8179999999999998e-06 2024-09-16T20:50:45.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:45.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562703+0000", 2024-09-16T20:50:45.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7811000000000003e-05 2024-09-16T20:50:45.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:50:45.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562706+0000", 2024-09-16T20:50:45.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4920000000000002e-06 2024-09-16T20:50:45.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:45.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566685+0000", 2024-09-16T20:50:45.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0039792780000000002 2024-09-16T20:50:45.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:45.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566693+0000", 2024-09-16T20:50:45.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4730000000000003e-06 2024-09-16T20:50:45.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:45.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570737+0000", 2024-09-16T20:50:45.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0040436749999999 2024-09-16T20:50:45.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:45.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570873+0000", 2024-09-16T20:50:45.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000136484 2024-09-16T20:50:45.589 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:45.589 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:45.589 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:45.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.589 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5255.0:1939 3.2 3:47882819:::rbd_data.110eb5f95afc.0000000000000009:head [read 233472~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:50:45.589 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:50:45.589 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 69.001458475000007, 2024-09-16T20:50:45.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0084657589999999, 2024-09-16T20:50:45.589 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:45.589 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:45.589 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:45.589 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:45.589 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5255", 2024-09-16T20:50:45.589 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:50:45.590 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1939 2024-09-16T20:50:45.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.590 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:45.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.590 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:45.590 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:50:45.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:45.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.590 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:45.590 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:50:45.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:45.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.590 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:45.591 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:50:45.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5199999999999997e-07 2024-09-16T20:50:45.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.591 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:45.591 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562757+0000", 2024-09-16T20:50:45.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4979999999999998e-06 2024-09-16T20:50:45.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.591 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:45.591 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562764+0000", 2024-09-16T20:50:45.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1470000000000004e-06 2024-09-16T20:50:45.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.592 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:45.592 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562768+0000", 2024-09-16T20:50:45.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7929999999999999e-06 2024-09-16T20:50:45.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.592 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:45.592 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562805+0000", 2024-09-16T20:50:45.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7477e-05 2024-09-16T20:50:45.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.592 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting_for_map not empty", 2024-09-16T20:50:45.592 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562810+0000", 2024-09-16T20:50:45.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0819999999999998e-06 2024-09-16T20:50:45.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.593 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:45.593 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566717+0000", 2024-09-16T20:50:45.593 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0039072600000000001 2024-09-16T20:50:45.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.593 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:45.593 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566724+0000", 2024-09-16T20:50:45.593 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4880000000000004e-06 2024-09-16T20:50:45.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.593 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:45.593 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571009+0000", 2024-09-16T20:50:45.593 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.004284677 2024-09-16T20:50:45.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.594 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:45.594 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571218+0000", 2024-09-16T20:50:45.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00020978500000000001 2024-09-16T20:50:45.594 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:45.594 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:45.594 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:45.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.594 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:11020 3.1f 3:fee2d1dd:::rbd_data.110ea93691bb.000000000000000e:23 [read 955392~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:45.594 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.693811+0000", 2024-09-16T20:50:45.594 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 54.870399818000003, 2024-09-16T20:50:45.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056833005999999998, 2024-09-16T20:50:45.594 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:45.594 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:45.594 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:45.595 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:45.595 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:45.595 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:45.595 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 11020 2024-09-16T20:50:45.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.595 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:45.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.595 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:45.595 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693811+0000", 2024-09-16T20:50:45.595 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:45.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.595 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:45.595 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693811+0000", 2024-09-16T20:50:45.595 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:45.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.596 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.596 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:45.596 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693812+0000", 2024-09-16T20:50:45.596 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6100000000000001e-07 2024-09-16T20:50:45.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.596 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.596 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:45.596 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693815+0000", 2024-09-16T20:50:45.596 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7389999999999999e-06 2024-09-16T20:50:45.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.596 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.596 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:45.596 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693823+0000", 2024-09-16T20:50:45.597 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0439999999999996e-06 2024-09-16T20:50:45.597 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.597 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.597 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:45.597 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693828+0000", 2024-09-16T20:50:45.597 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7440000000000004e-06 2024-09-16T20:50:45.597 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.597 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.597 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:45.597 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750357+0000", 2024-09-16T20:50:45.597 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056529192999999998 2024-09-16T20:50:45.597 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.597 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.598 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:45.598 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750429+0000", 2024-09-16T20:50:45.598 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1557000000000006e-05 2024-09-16T20:50:45.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.598 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:45.598 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750644+0000", 2024-09-16T20:50:45.598 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00021526800000000001 2024-09-16T20:50:45.598 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:45.598 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:45.598 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:45.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.598 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:11021 3.1f 3:fee2d1dd:::rbd_data.110ea93691bb.000000000000000e:23 [read 960512~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:45.599 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.693837+0000", 2024-09-16T20:50:45.599 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 54.87037393, 2024-09-16T20:50:45.599 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057155138000000001, 2024-09-16T20:50:45.599 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:45.599 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:45.599 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:45.599 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:45.599 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:45.599 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:45.599 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 11021 2024-09-16T20:50:45.599 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.599 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:45.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.599 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:45.600 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693837+0000", 2024-09-16T20:50:45.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:45.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.600 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.600 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:45.600 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693837+0000", 2024-09-16T20:50:45.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:45.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.600 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.600 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:45.600 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699566+0000", 2024-09-16T20:50:45.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057283639999999997 2024-09-16T20:50:45.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.600 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.600 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:45.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699571+0000", 2024-09-16T20:50:45.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.604e-06 2024-09-16T20:50:45.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.601 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:45.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699583+0000", 2024-09-16T20:50:45.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1372999999999999e-05 2024-09-16T20:50:45.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.601 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:45.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699591+0000", 2024-09-16T20:50:45.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5259999999999993e-06 2024-09-16T20:50:45.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.602 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:45.602 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750674+0000", 2024-09-16T20:50:45.602 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051083154999999998 2024-09-16T20:50:45.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.602 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:45.602 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750745+0000", 2024-09-16T20:50:45.602 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0933999999999995e-05 2024-09-16T20:50:45.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.602 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:45.602 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750992+0000", 2024-09-16T20:50:45.602 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00024718199999999998 2024-09-16T20:50:45.603 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:45.603 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:45.603 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:45.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.603 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8790 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2590720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:45.603 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057257+0000", 2024-09-16T20:50:45.603 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 44.506954610000001, 2024-09-16T20:50:45.603 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056791169000000002, 2024-09-16T20:50:45.603 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:45.603 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:45.603 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:45.603 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:45.604 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:45.604 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:45.604 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8790 2024-09-16T20:50:45.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.604 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:45.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.604 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:45.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057257+0000", 2024-09-16T20:50:45.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:45.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.604 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:45.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057257+0000", 2024-09-16T20:50:45.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:45.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:45.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057257+0000", 2024-09-16T20:50:45.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6600000000000002e-07 2024-09-16T20:50:45.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:45.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057260+0000", 2024-09-16T20:50:45.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7240000000000001e-06 2024-09-16T20:50:45.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.606 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:45.606 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057267+0000", 2024-09-16T20:50:45.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0600000000000002e-06 2024-09-16T20:50:45.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.606 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:45.606 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057272+0000", 2024-09-16T20:50:45.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0390000000000004e-06 2024-09-16T20:50:45.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.606 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:45.606 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.113950+0000", 2024-09-16T20:50:45.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056677989999999998 2024-09-16T20:50:45.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.607 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:45.607 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.113993+0000", 2024-09-16T20:50:45.607 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3205999999999999e-05 2024-09-16T20:50:45.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.607 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:45.607 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114048+0000", 2024-09-16T20:50:45.607 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4684e-05 2024-09-16T20:50:45.607 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:45.607 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:45.607 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:45.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.608 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8791 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2332672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:45.608 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057279+0000", 2024-09-16T20:50:45.608 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 44.506932194999997, 2024-09-16T20:50:45.608 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056890067000000002, 2024-09-16T20:50:45.608 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:45.608 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:45.608 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:45.608 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:45.608 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:45.608 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:45.608 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8791 2024-09-16T20:50:45.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.609 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:45.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:45.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057279+0000", 2024-09-16T20:50:45.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:45.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:45.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057279+0000", 2024-09-16T20:50:45.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:45.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:45.610 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057279+0000", 2024-09-16T20:50:45.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.15e-07 2024-09-16T20:50:45.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.610 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:45.610 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057281+0000", 2024-09-16T20:50:45.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8080000000000001e-06 2024-09-16T20:50:45.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.610 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:45.611 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057287+0000", 2024-09-16T20:50:45.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5999999999999997e-06 2024-09-16T20:50:45.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.611 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:45.611 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057291+0000", 2024-09-16T20:50:45.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5990000000000004e-06 2024-09-16T20:50:45.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.611 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:45.611 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114078+0000", 2024-09-16T20:50:45.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056786246999999998 2024-09-16T20:50:45.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.612 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:45.612 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114118+0000", 2024-09-16T20:50:45.612 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9898999999999999e-05 2024-09-16T20:50:45.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.612 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:45.612 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114169+0000", 2024-09-16T20:50:45.612 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1499000000000003e-05 2024-09-16T20:50:45.612 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:45.612 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:45.612 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:45.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.613 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8792 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2609152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:45.613 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:50:45.613 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 44.506914213000002, 2024-09-16T20:50:45.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057003524999999999, 2024-09-16T20:50:45.613 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:45.613 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:45.613 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:45.613 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:45.613 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:45.613 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:45.613 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8792 2024-09-16T20:50:45.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.613 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:45.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.614 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:45.614 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:50:45.614 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:45.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.614 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:45.614 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:50:45.614 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:45.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.614 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:45.614 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:50:45.615 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0400000000000002e-07 2024-09-16T20:50:45.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.615 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.615 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:45.615 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057299+0000", 2024-09-16T20:50:45.615 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7460000000000001e-06 2024-09-16T20:50:45.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.615 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.615 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:45.615 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057304+0000", 2024-09-16T20:50:45.615 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3210000000000004e-06 2024-09-16T20:50:45.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.615 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.615 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:45.616 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057310+0000", 2024-09-16T20:50:45.616 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4360000000000001e-06 2024-09-16T20:50:45.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.616 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.616 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:45.616 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114200+0000", 2024-09-16T20:50:45.616 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056889954 2024-09-16T20:50:45.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.616 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.616 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:45.616 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114243+0000", 2024-09-16T20:50:45.616 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2998999999999999e-05 2024-09-16T20:50:45.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.617 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:45.617 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114300+0000", 2024-09-16T20:50:45.617 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7765e-05 2024-09-16T20:50:45.617 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:45.617 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:45.617 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:45.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.617 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8794 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2351104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:45.617 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057327+0000", 2024-09-16T20:50:45.617 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 44.506883889000001, 2024-09-16T20:50:45.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057101130999999999, 2024-09-16T20:50:45.618 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:45.618 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:45.618 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:45.618 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:45.618 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:45.618 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:45.618 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8794 2024-09-16T20:50:45.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.618 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:45.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:45.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057327+0000", 2024-09-16T20:50:45.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:45.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.619 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:45.619 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057327+0000", 2024-09-16T20:50:45.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:45.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.619 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:45.619 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057328+0000", 2024-09-16T20:50:45.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7399999999999999e-07 2024-09-16T20:50:45.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.619 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:45.619 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057330+0000", 2024-09-16T20:50:45.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2199999999999999e-06 2024-09-16T20:50:45.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.620 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:45.620 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057336+0000", 2024-09-16T20:50:45.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1240000000000002e-06 2024-09-16T20:50:45.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.620 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:45.620 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057341+0000", 2024-09-16T20:50:45.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8470000000000003e-06 2024-09-16T20:50:45.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.620 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:45.620 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114334+0000", 2024-09-16T20:50:45.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056993267 2024-09-16T20:50:45.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.621 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:45.621 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114376+0000", 2024-09-16T20:50:45.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1638999999999999e-05 2024-09-16T20:50:45.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.621 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:45.621 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114428+0000", 2024-09-16T20:50:45.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2660000000000001e-05 2024-09-16T20:50:45.621 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:45.621 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:45.621 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:45.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.622 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8798 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2297856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:45.622 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:50:45.622 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 44.506864161999999, 2024-09-16T20:50:45.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057277041000000001, 2024-09-16T20:50:45.622 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:45.622 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:45.622 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:45.622 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:45.622 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:45.622 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:45.622 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8798 2024-09-16T20:50:45.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.623 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:45.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:45.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:50:45.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:45.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:45.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:50:45.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:45.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:45.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:50:45.624 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2800000000000003e-07 2024-09-16T20:50:45.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.624 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.624 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:45.624 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057354+0000", 2024-09-16T20:50:45.624 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7780000000000002e-06 2024-09-16T20:50:45.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.624 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.624 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:45.624 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057360+0000", 2024-09-16T20:50:45.624 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0419999999999999e-06 2024-09-16T20:50:45.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.624 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.624 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:45.625 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057365+0000", 2024-09-16T20:50:45.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7929999999999997e-06 2024-09-16T20:50:45.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.625 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:45.625 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114460+0000", 2024-09-16T20:50:45.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057094662999999997 2024-09-16T20:50:45.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.625 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:45.625 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114532+0000", 2024-09-16T20:50:45.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2881999999999998e-05 2024-09-16T20:50:45.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.626 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:45.626 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114624+0000", 2024-09-16T20:50:45.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1555e-05 2024-09-16T20:50:45.626 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:45.626 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:45.626 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:45.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.626 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8800 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2316288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:45.626 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:50:45.626 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 44.506840222999998, 2024-09-16T20:50:45.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.05746946, 2024-09-16T20:50:45.626 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:45.627 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:45.627 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:45.627 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:45.627 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:45.627 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:45.627 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8800 2024-09-16T20:50:45.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.627 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:45.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.627 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:45.627 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:50:45.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:45.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:45.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:50:45.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:45.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:45.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:50:45.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5199999999999998e-07 2024-09-16T20:50:45.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:45.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057373+0000", 2024-09-16T20:50:45.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.79e-06 2024-09-16T20:50:45.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.629 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.629 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:45.629 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057379+0000", 2024-09-16T20:50:45.629 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6699999999999999e-06 2024-09-16T20:50:45.629 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.629 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.629 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:45.629 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057386+0000", 2024-09-16T20:50:45.629 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3170000000000004e-06 2024-09-16T20:50:45.629 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.629 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.629 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:45.629 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114655+0000", 2024-09-16T20:50:45.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057269420000000001 2024-09-16T20:50:45.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:45.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114734+0000", 2024-09-16T20:50:45.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8084000000000007e-05 2024-09-16T20:50:45.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:45.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114840+0000", 2024-09-16T20:50:45.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000106827 2024-09-16T20:50:45.630 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:45.630 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:45.630 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:45.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.631 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.631 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8801 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2334720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:45.631 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:50:45.631 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 44.506819192000002, 2024-09-16T20:50:45.631 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057594682000000001, 2024-09-16T20:50:45.631 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:45.631 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:45.631 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:45.631 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:45.631 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:45.631 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:45.631 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8801 2024-09-16T20:50:45.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.631 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:45.632 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.632 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:45.632 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:50:45.632 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:45.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.632 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.632 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:45.632 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:50:45.632 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:45.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.632 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.632 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:45.632 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:50:45.633 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6899999999999998e-07 2024-09-16T20:50:45.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.633 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.633 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:45.633 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057394+0000", 2024-09-16T20:50:45.633 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8300000000000001e-06 2024-09-16T20:50:45.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.633 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.633 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:45.633 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057400+0000", 2024-09-16T20:50:45.633 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6500000000000001e-06 2024-09-16T20:50:45.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.633 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.634 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:45.634 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057404+0000", 2024-09-16T20:50:45.634 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4669999999999998e-06 2024-09-16T20:50:45.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.634 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:45.634 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114874+0000", 2024-09-16T20:50:45.634 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057469602000000002 2024-09-16T20:50:45.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.634 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:45.634 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114933+0000", 2024-09-16T20:50:45.634 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9500000000000003e-05 2024-09-16T20:50:45.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.635 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:45.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114987+0000", 2024-09-16T20:50:45.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3263999999999997e-05 2024-09-16T20:50:45.635 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:45.635 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:45.635 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:45.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.635 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8802 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2353152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:45.635 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:50:45.635 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 44.506770506999999, 2024-09-16T20:50:45.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057714079000000001, 2024-09-16T20:50:45.636 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:45.636 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:45.636 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:45.636 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:45.636 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:45.636 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:45.636 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8802 2024-09-16T20:50:45.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.636 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:45.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.636 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:45.636 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:50:45.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:45.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.637 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:45.637 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:50:45.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:45.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.637 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:45.637 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:50:45.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4999999999999998e-07 2024-09-16T20:50:45.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.637 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:45.637 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057446+0000", 2024-09-16T20:50:45.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4240000000000003e-06 2024-09-16T20:50:45.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.638 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.638 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:45.638 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057452+0000", 2024-09-16T20:50:45.638 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8379999999999996e-06 2024-09-16T20:50:45.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.638 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.638 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:45.638 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057460+0000", 2024-09-16T20:50:45.638 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9629999999999998e-06 2024-09-16T20:50:45.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.638 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.638 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:45.639 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115015+0000", 2024-09-16T20:50:45.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057554389999999997 2024-09-16T20:50:45.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.639 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:45.639 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115082+0000", 2024-09-16T20:50:45.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7733999999999999e-05 2024-09-16T20:50:45.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.639 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:45.639 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115155+0000", 2024-09-16T20:50:45.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2280000000000006e-05 2024-09-16T20:50:45.639 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:45.639 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:45.640 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:45.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.640 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8804 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2555904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:45.640 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:50:45.640 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 44.506744191000003, 2024-09-16T20:50:45.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057833862, 2024-09-16T20:50:45.640 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:45.640 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:45.640 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:45.640 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:45.640 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:45.640 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:45.641 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8804 2024-09-16T20:50:45.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.641 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:45.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:45.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:50:45.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:45.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:45.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:50:45.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:45.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.642 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:45.642 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:50:45.642 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7899999999999999e-07 2024-09-16T20:50:45.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.642 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:45.642 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057469+0000", 2024-09-16T20:50:45.642 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.001e-06 2024-09-16T20:50:45.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.642 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:45.642 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057475+0000", 2024-09-16T20:50:45.642 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7629999999999997e-06 2024-09-16T20:50:45.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.643 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:45.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057480+0000", 2024-09-16T20:50:45.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8069999999999999e-06 2024-09-16T20:50:45.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.643 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:45.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115183+0000", 2024-09-16T20:50:45.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057703364 2024-09-16T20:50:45.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.643 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:45.644 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115227+0000", 2024-09-16T20:50:45.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3903999999999998e-05 2024-09-16T20:50:45.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.644 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:45.644 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115301+0000", 2024-09-16T20:50:45.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3543999999999999e-05 2024-09-16T20:50:45.644 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:45.644 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:45.644 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:45.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.644 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8805 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2574336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:45.645 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057487+0000", 2024-09-16T20:50:45.645 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 44.506723862999998, 2024-09-16T20:50:45.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058021027000000003, 2024-09-16T20:50:45.645 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:45.645 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:45.645 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:45.645 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:45.645 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:45.645 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:45.645 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8805 2024-09-16T20:50:45.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.645 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:45.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.646 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:45.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057487+0000", 2024-09-16T20:50:45.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:45.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.646 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:45.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057487+0000", 2024-09-16T20:50:45.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:45.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.646 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:45.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057488+0000", 2024-09-16T20:50:45.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5199999999999998e-07 2024-09-16T20:50:45.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.647 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:45.647 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057490+0000", 2024-09-16T20:50:45.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.939e-06 2024-09-16T20:50:45.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.647 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:45.647 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057495+0000", 2024-09-16T20:50:45.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6439999999999997e-06 2024-09-16T20:50:45.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.647 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:45.648 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057500+0000", 2024-09-16T20:50:45.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8119999999999998e-06 2024-09-16T20:50:45.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:45.648 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115341+0000", 2024-09-16T20:50:45.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057841341999999997 2024-09-16T20:50:45.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:45.648 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115422+0000", 2024-09-16T20:50:45.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1003999999999999e-05 2024-09-16T20:50:45.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:45.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115508+0000", 2024-09-16T20:50:45.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5933999999999994e-05 2024-09-16T20:50:45.649 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:45.649 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:45.649 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:45.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.649 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8807 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2592768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:45.650 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057505+0000", 2024-09-16T20:50:45.650 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 44.506705652999997, 2024-09-16T20:50:45.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058186633000000001, 2024-09-16T20:50:45.650 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:45.650 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:45.650 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:45.650 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:45.650 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:45.650 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:45.650 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8807 2024-09-16T20:50:45.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.650 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:45.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.650 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:45.651 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057505+0000", 2024-09-16T20:50:45.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:45.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.651 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:45.651 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057505+0000", 2024-09-16T20:50:45.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:45.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.651 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:45.651 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057506+0000", 2024-09-16T20:50:45.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.27e-07 2024-09-16T20:50:45.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.652 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:45.652 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057508+0000", 2024-09-16T20:50:45.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7400000000000001e-06 2024-09-16T20:50:45.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.652 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:45.652 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057513+0000", 2024-09-16T20:50:45.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.468e-06 2024-09-16T20:50:45.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.652 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:45.652 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057517+0000", 2024-09-16T20:50:45.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4329999999999997e-06 2024-09-16T20:50:45.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:45.653 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115538+0000", 2024-09-16T20:50:45.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058020355000000003 2024-09-16T20:50:45.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:45.653 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115616+0000", 2024-09-16T20:50:45.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7998999999999996e-05 2024-09-16T20:50:45.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:45.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115692+0000", 2024-09-16T20:50:45.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6310999999999995e-05 2024-09-16T20:50:45.654 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:45.654 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:45.654 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:45.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.654 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8808 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2611200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:45.654 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:50:45.654 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 44.506688392000001, 2024-09-16T20:50:45.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058354046, 2024-09-16T20:50:45.654 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:45.654 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:45.655 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:45.655 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:45.655 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:45.655 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:45.655 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8808 2024-09-16T20:50:45.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.655 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:45.655 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.655 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:45.655 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:50:45.655 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:45.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.655 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.655 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:45.656 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:50:45.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:45.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.656 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:45.656 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:50:45.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2500000000000001e-07 2024-09-16T20:50:45.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.656 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:45.656 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057525+0000", 2024-09-16T20:50:45.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5549999999999999e-06 2024-09-16T20:50:45.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.657 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:45.657 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057530+0000", 2024-09-16T20:50:45.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5489999999999997e-06 2024-09-16T20:50:45.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.657 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:45.657 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057535+0000", 2024-09-16T20:50:45.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5709999999999999e-06 2024-09-16T20:50:45.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.657 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:45.657 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115722+0000", 2024-09-16T20:50:45.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058187323999999999 2024-09-16T20:50:45.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.658 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:45.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115799+0000", 2024-09-16T20:50:45.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6445000000000004e-05 2024-09-16T20:50:45.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.658 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:45.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115877+0000", 2024-09-16T20:50:45.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8276999999999996e-05 2024-09-16T20:50:45.658 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:45.658 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:45.658 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:45.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.659 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.659 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:2182 3.17 3:edb5e5cd:::rbd_data.110e61479aee.0000000000000014:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write 0~5120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-16T20:50:45.659 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:22.734609+0000", 2024-09-16T20:50:45.659 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.829602144999999, 2024-09-16T20:50:45.659 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.124890894, 2024-09-16T20:50:45.659 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:45.659 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:45.659 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:45.659 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:45.659 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:45.659 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:45.659 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 2182 2024-09-16T20:50:45.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.660 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:45.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.660 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:45.660 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734609+0000", 2024-09-16T20:50:45.660 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:45.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.660 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:45.660 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734609+0000", 2024-09-16T20:50:45.660 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:45.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.660 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:45.661 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734611+0000", 2024-09-16T20:50:45.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9599999999999999e-06 2024-09-16T20:50:45.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.661 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:45.661 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734628+0000", 2024-09-16T20:50:45.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.715e-05 2024-09-16T20:50:45.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.661 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:45.661 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734656+0000", 2024-09-16T20:50:45.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7759999999999998e-05 2024-09-16T20:50:45.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.662 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.662 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:45.662 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734669+0000", 2024-09-16T20:50:45.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3315999999999999e-05 2024-09-16T20:50:45.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.662 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.662 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:45.662 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734826+0000", 2024-09-16T20:50:45.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015668499999999999 2024-09-16T20:50:45.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.662 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.662 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:45.662 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.735048+0000", 2024-09-16T20:50:45.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00022179 2024-09-16T20:50:45.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.663 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:45.663 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.735248+0000", 2024-09-16T20:50:45.663 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00020010399999999999 2024-09-16T20:50:45.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.663 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:50:45.663 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.736252+0000", 2024-09-16T20:50:45.663 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010039739999999999 2024-09-16T20:50:45.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.663 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:45.663 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.859340+0000", 2024-09-16T20:50:45.664 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.12308846599999999 2024-09-16T20:50:45.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.664 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.664 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:50:45.664 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.859431+0000", 2024-09-16T20:50:45.664 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1099000000000003e-05 2024-09-16T20:50:45.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:45.664 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:45.664 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:45.664 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.859500+0000", 2024-09-16T20:50:45.664 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8590000000000006e-05 2024-09-16T20:50:45.664 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:45.664 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:45.665 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:45.665 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:45.665 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:45.665 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:45.665 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T20:50:45.667 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:45.666+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:45.667 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:50:45.767 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:45.766+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:45.782 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:45.782 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:50:45.782 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:50:45.782 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:45.791 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:50:45.875 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:45.874+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:45.938 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:45.938 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:50:45.938 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:50:45.938 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:50:45.939 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:45.947 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:50:45.988 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:45.987+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:46.071 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:46.070+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:46.096 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:46.096 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:50:46.096 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:50:46.096 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:50:46.096 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.096 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5439 3.a 3:5789f5c3:::rbd_data.110ed9ca540.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 1181696~60416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:46.096 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:50:46.097 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 143.22998687800001, 2024-09-16T20:50:46.097 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10102045599999999, 2024-09-16T20:50:46.097 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:46.097 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:46.097 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:46.097 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:46.097 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:46.097 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:46.097 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5439 2024-09-16T20:50:46.097 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.097 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:46.097 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.097 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:46.098 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:50:46.098 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.098 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.098 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.098 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:46.098 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:50:46.098 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.098 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.098 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.098 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:46.098 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864711+0000", 2024-09-16T20:50:46.098 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7899999999999997e-07 2024-09-16T20:50:46.098 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.098 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.098 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:46.099 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864760+0000", 2024-09-16T20:50:46.099 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8803e-05 2024-09-16T20:50:46.099 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.099 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.099 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:46.099 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864769+0000", 2024-09-16T20:50:46.099 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0999999999999993e-06 2024-09-16T20:50:46.099 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.099 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.099 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:46.099 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864774+0000", 2024-09-16T20:50:46.099 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8919999999999999e-06 2024-09-16T20:50:46.099 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.099 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.099 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:46.099 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864815+0000", 2024-09-16T20:50:46.100 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1307000000000003e-05 2024-09-16T20:50:46.100 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.100 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.100 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:46.100 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864893+0000", 2024-09-16T20:50:46.100 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7581000000000001e-05 2024-09-16T20:50:46.100 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.100 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.100 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 1", 2024-09-16T20:50:46.100 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864996+0000", 2024-09-16T20:50:46.100 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102725 2024-09-16T20:50:46.100 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.100 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.100 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:46.101 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965442+0000", 2024-09-16T20:50:46.101 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100446012 2024-09-16T20:50:46.101 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.101 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.101 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:46.101 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965618+0000", 2024-09-16T20:50:46.101 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00017653100000000001 2024-09-16T20:50:46.101 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.101 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.101 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:46.101 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965682+0000", 2024-09-16T20:50:46.101 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3905999999999994e-05 2024-09-16T20:50:46.101 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.101 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.101 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:46.102 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965731+0000", 2024-09-16T20:50:46.102 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8819999999999997e-05 2024-09-16T20:50:46.102 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.102 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:46.102 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.102 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.102 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.102 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4600.0:5439 3.a e37/23 ondisk, result = 0)", 2024-09-16T20:50:46.102 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:50:46.102 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 143.22899072999999, 2024-09-16T20:50:46.102 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099742634999999996, 2024-09-16T20:50:46.102 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:46.102 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:46.103 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:46.103 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:46.103 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.103 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:46.103 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:50:46.103 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.103 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.103 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.103 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:46.103 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:50:46.103 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.103 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.103 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.103 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:46.104 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:50:46.104 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0900000000000001e-07 2024-09-16T20:50:46.104 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.104 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.104 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:46.104 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865711+0000", 2024-09-16T20:50:46.104 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3739999999999998e-06 2024-09-16T20:50:46.104 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.104 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.104 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:46.104 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965343+0000", 2024-09-16T20:50:46.104 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099632526999999999 2024-09-16T20:50:46.104 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.104 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.105 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:46.105 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965357+0000", 2024-09-16T20:50:46.105 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3713999999999999e-05 2024-09-16T20:50:46.105 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.105 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.105 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:46.105 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965426+0000", 2024-09-16T20:50:46.105 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8745999999999995e-05 2024-09-16T20:50:46.105 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.105 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.105 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:46.105 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965434+0000", 2024-09-16T20:50:46.105 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6910000000000003e-06 2024-09-16T20:50:46.105 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.106 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.106 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:46.106 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965449+0000", 2024-09-16T20:50:46.106 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4874e-05 2024-09-16T20:50:46.106 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.106 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:46.106 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.106 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.106 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.106 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4882.0:2508 3.12 3:4992771f:::rbd_object_map.110e7c8851c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T20:50:46.106 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:50:46.106 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 119.613187182, 2024-09-16T20:50:46.106 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.125007011, 2024-09-16T20:50:46.106 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4882", 2024-09-16T20:50:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2436923223", 2024-09-16T20:50:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2508 2024-09-16T20:50:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:50:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:46.108 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:50:46.108 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.108 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.108 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.108 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:46.108 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481511+0000", 2024-09-16T20:50:46.108 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0699999999999996e-07 2024-09-16T20:50:46.108 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.108 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.108 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:46.108 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481515+0000", 2024-09-16T20:50:46.108 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0609999999999997e-06 2024-09-16T20:50:46.108 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.108 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.109 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:46.109 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481523+0000", 2024-09-16T20:50:46.109 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1310000000000006e-06 2024-09-16T20:50:46.109 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.109 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.109 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:46.109 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481528+0000", 2024-09-16T20:50:46.109 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4900000000000002e-06 2024-09-16T20:50:46.109 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.109 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.112 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:46.112 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481565+0000", 2024-09-16T20:50:46.112 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7066000000000001e-05 2024-09-16T20:50:46.112 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.112 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.113 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:46.113 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481628+0000", 2024-09-16T20:50:46.113 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3118000000000001e-05 2024-09-16T20:50:46.113 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.113 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.113 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 2", 2024-09-16T20:50:46.113 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481822+0000", 2024-09-16T20:50:46.113 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019446200000000001 2024-09-16T20:50:46.113 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.113 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.113 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:46.113 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605988+0000", 2024-09-16T20:50:46.113 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.124165888 2024-09-16T20:50:46.113 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.114 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.114 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:46.114 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606427+0000", 2024-09-16T20:50:46.114 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00043854899999999999 2024-09-16T20:50:46.114 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.114 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.114 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:46.114 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606476+0000", 2024-09-16T20:50:46.114 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8971999999999998e-05 2024-09-16T20:50:46.114 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.114 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.114 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:46.114 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606517+0000", 2024-09-16T20:50:46.115 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1467000000000001e-05 2024-09-16T20:50:46.115 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.115 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:46.115 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.115 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.115 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.115 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4882.0:2508 3.12 e57/55 ondisk, result = 0)", 2024-09-16T20:50:46.115 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:50:46.115 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 119.612198682, 2024-09-16T20:50:46.115 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12349711400000001, 2024-09-16T20:50:46.115 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:46.115 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:46.115 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:46.115 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:46.116 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.116 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:46.116 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:50:46.116 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.116 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.116 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.116 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:46.116 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:50:46.116 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.116 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.116 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.116 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:46.117 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:50:46.117 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.06e-07 2024-09-16T20:50:46.117 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.117 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.117 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:46.117 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482502+0000", 2024-09-16T20:50:46.117 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.01e-06 2024-09-16T20:50:46.117 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.117 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.117 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:46.117 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605772+0000", 2024-09-16T20:50:46.117 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.123269687 2024-09-16T20:50:46.117 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.118 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.118 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:46.118 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605791+0000", 2024-09-16T20:50:46.118 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8912000000000001e-05 2024-09-16T20:50:46.118 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.118 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.118 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:46.118 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605974+0000", 2024-09-16T20:50:46.118 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000182836 2024-09-16T20:50:46.118 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.118 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.118 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:46.118 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605982+0000", 2024-09-16T20:50:46.119 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1699999999999997e-06 2024-09-16T20:50:46.119 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.119 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.119 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:46.119 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605996+0000", 2024-09-16T20:50:46.119 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3893000000000001e-05 2024-09-16T20:50:46.119 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.119 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:46.119 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.119 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.119 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.119 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4882.0:2512 3.f 3:f4d895b7:::rbd_data.110e7c8851c.0000000000000000:head [read 16384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T20:50:46.119 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:50:46.119 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 119.611761652, 2024-09-16T20:50:46.120 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12318483, 2024-09-16T20:50:46.120 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:46.120 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:46.120 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:46.120 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:46.120 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4882", 2024-09-16T20:50:46.120 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2436923223", 2024-09-16T20:50:46.120 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2512 2024-09-16T20:50:46.120 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.120 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:46.120 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.120 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:46.120 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:50:46.121 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.121 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.121 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.121 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:46.121 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:50:46.121 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.121 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.121 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.121 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:46.121 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:50:46.121 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3499999999999995e-07 2024-09-16T20:50:46.121 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.121 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.122 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:46.122 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482943+0000", 2024-09-16T20:50:46.122 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3450000000000002e-06 2024-09-16T20:50:46.122 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.122 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.122 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:46.122 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605772+0000", 2024-09-16T20:50:46.122 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.122829796 2024-09-16T20:50:46.122 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.122 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.122 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:46.122 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605792+0000", 2024-09-16T20:50:46.122 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9916999999999999e-05 2024-09-16T20:50:46.122 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.122 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.123 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:46.123 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605917+0000", 2024-09-16T20:50:46.123 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00012500799999999999 2024-09-16T20:50:46.123 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.123 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.123 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:46.123 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606004+0000", 2024-09-16T20:50:46.123 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6242999999999994e-05 2024-09-16T20:50:46.123 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.123 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.123 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:46.123 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606120+0000", 2024-09-16T20:50:46.123 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000116786 2024-09-16T20:50:46.123 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.123 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:46.124 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.124 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.124 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.124 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:11702 3.15 3:acf33c8b:::rbd_data.110e747f0779.000000000000001a:head [set-alloc-hint object_size 262144 write_size 262144,write 143360~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:50:46.124 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:50:46.124 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 101.95676923000001, 2024-09-16T20:50:46.124 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.084801043000000006, 2024-09-16T20:50:46.124 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:46.124 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:46.124 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:46.124 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:46.124 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:50:46.124 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:50:46.124 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 11702 2024-09-16T20:50:46.125 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.125 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:46.125 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.125 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:46.125 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:50:46.125 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.125 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.125 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.125 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:46.125 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:50:46.125 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.125 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.125 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.125 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:46.126 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137929+0000", 2024-09-16T20:50:46.126 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8400000000000002e-07 2024-09-16T20:50:46.126 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.126 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.126 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:46.126 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137937+0000", 2024-09-16T20:50:46.126 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0260000000000007e-06 2024-09-16T20:50:46.126 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.126 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.126 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:46.126 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137952+0000", 2024-09-16T20:50:46.126 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4659e-05 2024-09-16T20:50:46.126 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.126 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.126 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:46.127 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137960+0000", 2024-09-16T20:50:46.127 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0169999999999996e-06 2024-09-16T20:50:46.127 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.127 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.127 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:46.127 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138010+0000", 2024-09-16T20:50:46.127 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0021000000000001e-05 2024-09-16T20:50:46.127 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.127 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.127 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:46.127 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138114+0000", 2024-09-16T20:50:46.127 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000103797 2024-09-16T20:50:46.128 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.128 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.128 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:50:46.128 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138233+0000", 2024-09-16T20:50:46.128 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000118971 2024-09-16T20:50:46.128 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.128 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.128 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:46.128 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138953+0000", 2024-09-16T20:50:46.128 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00072026000000000002 2024-09-16T20:50:46.128 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.128 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.128 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:46.128 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222609+0000", 2024-09-16T20:50:46.129 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.083656650999999999 2024-09-16T20:50:46.129 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.129 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.129 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:46.129 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222684+0000", 2024-09-16T20:50:46.129 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4369999999999994e-05 2024-09-16T20:50:46.129 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.129 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.129 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:46.129 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222729+0000", 2024-09-16T20:50:46.129 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5287000000000002e-05 2024-09-16T20:50:46.129 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.129 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:46.129 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.130 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.130 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.130 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:11704 3.9 3:91220b75:::rbd_data.110e747f0779.000000000000001c:head [set-alloc-hint object_size 262144 write_size 262144,write 139264~6656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:50:46.130 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:50:46.130 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 101.95653122900001, 2024-09-16T20:50:46.130 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.084531133999999994, 2024-09-16T20:50:46.130 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:46.130 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:46.130 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:46.130 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:46.130 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:50:46.130 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:50:46.130 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 11704 2024-09-16T20:50:46.130 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.131 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:46.131 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.131 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:46.131 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:50:46.131 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.131 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.131 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.131 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:46.131 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:50:46.131 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.131 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.131 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.131 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:46.131 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138167+0000", 2024-09-16T20:50:46.132 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.33e-07 2024-09-16T20:50:46.132 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.132 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.132 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:46.132 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138179+0000", 2024-09-16T20:50:46.132 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2055999999999999e-05 2024-09-16T20:50:46.132 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.132 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.132 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:46.132 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138193+0000", 2024-09-16T20:50:46.132 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3961000000000001e-05 2024-09-16T20:50:46.132 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.132 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.132 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:46.133 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138200+0000", 2024-09-16T20:50:46.133 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4429999999999997e-06 2024-09-16T20:50:46.133 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.133 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.133 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:46.133 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138258+0000", 2024-09-16T20:50:46.133 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7225999999999997e-05 2024-09-16T20:50:46.133 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.133 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.133 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:46.133 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138364+0000", 2024-09-16T20:50:46.133 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000106013 2024-09-16T20:50:46.133 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.133 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.133 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:50:46.134 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138494+0000", 2024-09-16T20:50:46.134 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013059599999999999 2024-09-16T20:50:46.134 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.134 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.134 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:46.134 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.139220+0000", 2024-09-16T20:50:46.134 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00072539800000000004 2024-09-16T20:50:46.134 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.134 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.134 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:46.134 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222608+0000", 2024-09-16T20:50:46.134 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.083388620999999996 2024-09-16T20:50:46.134 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.134 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.135 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:46.135 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222651+0000", 2024-09-16T20:50:46.135 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2228000000000003e-05 2024-09-16T20:50:46.135 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.135 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.135 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:46.135 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222697+0000", 2024-09-16T20:50:46.135 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6659000000000002e-05 2024-09-16T20:50:46.135 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.135 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:46.135 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.135 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.135 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.135 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5138.0:9358 3.c 3.c25699cc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T20:50:46.135 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:50:46.136 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 79.244383924000005, 2024-09-16T20:50:46.136 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0018657580000001, 2024-09-16T20:50:46.136 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:46.136 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:46.136 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:50:46.136 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:46.136 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5138", 2024-09-16T20:50:46.136 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:50:46.136 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 9358 2024-09-16T20:50:46.136 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.136 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:46.136 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.136 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:46.136 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:50:46.137 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.137 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.137 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.137 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:46.137 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:50:46.137 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.137 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.137 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.137 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:46.137 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850314+0000", 2024-09-16T20:50:46.137 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4099999999999999e-07 2024-09-16T20:50:46.137 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.137 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.137 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:46.138 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850316+0000", 2024-09-16T20:50:46.138 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2469999999999998e-06 2024-09-16T20:50:46.138 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.138 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.138 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:46.138 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850322+0000", 2024-09-16T20:50:46.138 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3470000000000004e-06 2024-09-16T20:50:46.138 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.138 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.138 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:46.138 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850327+0000", 2024-09-16T20:50:46.138 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9450000000000001e-06 2024-09-16T20:50:46.138 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.138 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.139 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:46.139 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850376+0000", 2024-09-16T20:50:46.139 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8257000000000002e-05 2024-09-16T20:50:46.139 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.139 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.139 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:46.139 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850385+0000", 2024-09-16T20:50:46.139 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9800000000000004e-06 2024-09-16T20:50:46.139 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.139 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.139 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:46.139 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852167+0000", 2024-09-16T20:50:46.139 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0017825659999999 2024-09-16T20:50:46.139 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.140 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.140 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:46.140 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852179+0000", 2024-09-16T20:50:46.140 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1975e-05 2024-09-16T20:50:46.140 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.140 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:46.140 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.140 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.140 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.140 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5138.0:9382 3.c 3.c25699cc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T20:50:46.140 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:50:46.140 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 79.236036908000003, 2024-09-16T20:50:46.140 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.99358404700000003, 2024-09-16T20:50:46.141 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:46.141 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:46.141 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:50:46.141 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:46.141 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5138", 2024-09-16T20:50:46.141 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:50:46.141 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 9382 2024-09-16T20:50:46.141 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.141 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:46.141 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.141 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:46.141 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:50:46.141 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.141 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.142 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.142 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:46.142 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:50:46.142 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.142 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.142 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.142 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:46.142 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858661+0000", 2024-09-16T20:50:46.142 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0200000000000002e-07 2024-09-16T20:50:46.142 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.142 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.142 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:46.142 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858664+0000", 2024-09-16T20:50:46.142 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.038e-06 2024-09-16T20:50:46.143 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.143 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.143 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:46.143 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858671+0000", 2024-09-16T20:50:46.143 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0110000000000003e-06 2024-09-16T20:50:46.143 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.143 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.143 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:46.143 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858677+0000", 2024-09-16T20:50:46.143 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0959999999999997e-06 2024-09-16T20:50:46.143 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.143 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.143 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:46.143 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858732+0000", 2024-09-16T20:50:46.144 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4543000000000003e-05 2024-09-16T20:50:46.144 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.144 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.144 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:46.144 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858739+0000", 2024-09-16T20:50:46.144 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5429999999999996e-06 2024-09-16T20:50:46.144 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.144 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.144 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:46.144 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852236+0000", 2024-09-16T20:50:46.144 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.993496453 2024-09-16T20:50:46.144 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.144 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.144 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:46.145 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852244+0000", 2024-09-16T20:50:46.145 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8610000000000004e-06 2024-09-16T20:50:46.145 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.145 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:46.145 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.145 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.145 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.145 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5763.0:4895 3.a 3:5384606c:::rbd_data.110e1b19a241.0000000000000018:head [read 0~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-16T20:50:46.145 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:45.598212+0000", 2024-09-16T20:50:46.145 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 60.496484866999999, 2024-09-16T20:50:46.145 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0375864829999999, 2024-09-16T20:50:46.145 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:46.145 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:46.145 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:46.146 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:46.146 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5763", 2024-09-16T20:50:46.146 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/623897567", 2024-09-16T20:50:46.146 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4895 2024-09-16T20:50:46.146 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.146 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:46.146 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.146 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:46.146 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598212+0000", 2024-09-16T20:50:46.146 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.146 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.146 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.146 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:46.146 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598212+0000", 2024-09-16T20:50:46.147 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.147 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.147 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.147 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:46.147 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598214+0000", 2024-09-16T20:50:46.147 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0090000000000002e-06 2024-09-16T20:50:46.147 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.147 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.147 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:46.147 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598222+0000", 2024-09-16T20:50:46.147 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6699999999999994e-06 2024-09-16T20:50:46.147 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.147 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.147 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:46.148 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598241+0000", 2024-09-16T20:50:46.148 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8896999999999999e-05 2024-09-16T20:50:46.148 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.148 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.148 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:46.148 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598250+0000", 2024-09-16T20:50:46.148 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6649999999999992e-06 2024-09-16T20:50:46.148 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.148 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.148 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:46.148 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598325+0000", 2024-09-16T20:50:46.148 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5189000000000001e-05 2024-09-16T20:50:46.148 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.148 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.148 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op must wait for map", 2024-09-16T20:50:46.149 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598333+0000", 2024-09-16T20:50:46.149 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.3170000000000002e-06 2024-09-16T20:50:46.149 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.149 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.149 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:46.149 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601355+0000", 2024-09-16T20:50:46.149 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0030215400000000001 2024-09-16T20:50:46.149 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.149 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.149 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:46.149 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601365+0000", 2024-09-16T20:50:46.149 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0215e-05 2024-09-16T20:50:46.150 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.150 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.150 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:46.150 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635506+0000", 2024-09-16T20:50:46.150 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0341408240000001 2024-09-16T20:50:46.150 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.150 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.150 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:46.150 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635655+0000", 2024-09-16T20:50:46.150 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000149412 2024-09-16T20:50:46.150 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.150 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.150 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:46.151 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635799+0000", 2024-09-16T20:50:46.151 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000143745 2024-09-16T20:50:46.151 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.151 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:46.151 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.151 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.151 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.151 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5763.0:4900 3.a 3:5384606c:::rbd_data.110e1b19a241.0000000000000018:head [read 8192~24576] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-16T20:50:46.151 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:50:46.151 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 60.496426391, 2024-09-16T20:50:46.151 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0379340800000001, 2024-09-16T20:50:46.151 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:46.151 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:46.152 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:46.152 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:46.152 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5763", 2024-09-16T20:50:46.152 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/623897567", 2024-09-16T20:50:46.152 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4900 2024-09-16T20:50:46.152 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.152 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:46.152 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.152 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:46.152 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:50:46.152 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.152 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.152 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.152 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:46.153 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:50:46.153 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.153 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.153 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.153 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:46.153 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:50:46.153 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4300000000000003e-07 2024-09-16T20:50:46.153 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.153 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.153 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:46.153 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598274+0000", 2024-09-16T20:50:46.153 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.976e-06 2024-09-16T20:50:46.153 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.153 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.154 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:46.154 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598282+0000", 2024-09-16T20:50:46.154 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0029999999999994e-06 2024-09-16T20:50:46.154 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.154 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.154 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:46.154 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598289+0000", 2024-09-16T20:50:46.154 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2389999999999999e-06 2024-09-16T20:50:46.154 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.154 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.154 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:46.154 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598430+0000", 2024-09-16T20:50:46.154 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014119600000000001 2024-09-16T20:50:46.154 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.155 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.155 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting_for_map not empty", 2024-09-16T20:50:46.155 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598435+0000", 2024-09-16T20:50:46.155 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3489999999999999e-06 2024-09-16T20:50:46.155 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.155 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.155 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:46.155 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601429+0000", 2024-09-16T20:50:46.155 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0029942720000000001 2024-09-16T20:50:46.155 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.155 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.155 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:46.155 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601438+0000", 2024-09-16T20:50:46.155 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4759999999999998e-06 2024-09-16T20:50:46.156 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.156 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.156 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:46.156 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635987+0000", 2024-09-16T20:50:46.156 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0345495010000001 2024-09-16T20:50:46.156 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.156 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.156 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:46.156 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.636056+0000", 2024-09-16T20:50:46.156 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8610999999999998e-05 2024-09-16T20:50:46.156 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.156 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.156 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:46.156 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.636205+0000", 2024-09-16T20:50:46.157 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014891399999999999 2024-09-16T20:50:46.157 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.157 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:46.157 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.157 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.157 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.157 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop(client.6231.0:2182 3.17 e136/81 3:edb5e5cd:::rbd_data.110e61479aee.0000000000000014:head v 136'791, mlcod=135'789)", 2024-09-16T20:50:46.157 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:22.735499+0000", 2024-09-16T20:50:46.157 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.359198450000001, 2024-09-16T20:50:46.157 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12369608, 2024-09-16T20:50:46.157 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:46.157 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:46.157 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:46.157 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:46.158 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.158 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:46.158 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735499+0000", 2024-09-16T20:50:46.158 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.158 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.158 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.158 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:46.158 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735499+0000", 2024-09-16T20:50:46.158 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.158 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.158 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.158 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:46.158 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735499+0000", 2024-09-16T20:50:46.158 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8099999999999999e-07 2024-09-16T20:50:46.158 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.159 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.159 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:46.159 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735506+0000", 2024-09-16T20:50:46.159 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3579999999999997e-06 2024-09-16T20:50:46.159 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.159 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.159 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:46.159 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735517+0000", 2024-09-16T20:50:46.159 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0869999999999999e-05 2024-09-16T20:50:46.159 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.159 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.159 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:46.159 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735533+0000", 2024-09-16T20:50:46.160 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5858e-05 2024-09-16T20:50:46.160 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.160 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.160 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:46.160 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735584+0000", 2024-09-16T20:50:46.160 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0470999999999998e-05 2024-09-16T20:50:46.160 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.160 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.160 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:46.160 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735607+0000", 2024-09-16T20:50:46.160 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2875999999999999e-05 2024-09-16T20:50:46.160 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.160 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.161 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:46.161 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.859144+0000", 2024-09-16T20:50:46.161 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.123537701 2024-09-16T20:50:46.161 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.161 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.161 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:46.161 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.859195+0000", 2024-09-16T20:50:46.161 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0664999999999997e-05 2024-09-16T20:50:46.161 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.161 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:46.161 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.161 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.161 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.161 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5204 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 2523136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:46.162 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.883660+0000", 2024-09-16T20:50:46.162 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 22.211036920000002, 2024-09-16T20:50:46.162 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052517638999999998, 2024-09-16T20:50:46.162 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:46.162 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:46.162 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:46.162 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:46.162 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:50:46.162 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:46.162 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5204 2024-09-16T20:50:46.162 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.162 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:46.162 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.162 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:46.163 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883660+0000", 2024-09-16T20:50:46.163 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.163 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.163 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.163 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:46.163 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883660+0000", 2024-09-16T20:50:46.163 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.163 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.163 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.163 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:46.163 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883661+0000", 2024-09-16T20:50:46.163 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9999999999999998e-07 2024-09-16T20:50:46.163 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.163 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.164 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:46.164 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883663+0000", 2024-09-16T20:50:46.164 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.4480000000000001e-06 2024-09-16T20:50:46.164 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.164 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.164 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:46.164 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883669+0000", 2024-09-16T20:50:46.164 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.071e-06 2024-09-16T20:50:46.164 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.164 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.164 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:46.164 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883675+0000", 2024-09-16T20:50:46.164 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.366e-06 2024-09-16T20:50:46.164 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.165 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.165 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:46.165 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.935970+0000", 2024-09-16T20:50:46.165 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052295346999999999 2024-09-16T20:50:46.165 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.165 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.165 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:46.165 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936064+0000", 2024-09-16T20:50:46.165 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4438000000000006e-05 2024-09-16T20:50:46.165 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.165 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.165 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:46.165 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936178+0000", 2024-09-16T20:50:46.165 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000113569 2024-09-16T20:50:46.166 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.166 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:46.166 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.166 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.166 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.166 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5205 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 2543616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:46.166 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.883745+0000", 2024-09-16T20:50:46.166 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 22.210952162000002, 2024-09-16T20:50:46.166 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052692112999999999, 2024-09-16T20:50:46.166 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:46.166 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:46.166 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:46.166 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:46.167 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:50:46.167 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:46.167 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5205 2024-09-16T20:50:46.167 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.167 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:46.167 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.167 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:46.167 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883745+0000", 2024-09-16T20:50:46.167 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.167 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.167 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.167 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:46.167 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883745+0000", 2024-09-16T20:50:46.167 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.168 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.168 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.168 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:46.168 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883746+0000", 2024-09-16T20:50:46.168 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2699999999999999e-07 2024-09-16T20:50:46.168 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.168 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.168 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:46.168 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883750+0000", 2024-09-16T20:50:46.168 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7050000000000001e-06 2024-09-16T20:50:46.168 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.168 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.168 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:46.168 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883769+0000", 2024-09-16T20:50:46.169 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9165e-05 2024-09-16T20:50:46.169 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.169 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.169 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:46.169 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883778+0000", 2024-09-16T20:50:46.169 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.6690000000000001e-06 2024-09-16T20:50:46.169 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.169 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.169 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:46.169 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936228+0000", 2024-09-16T20:50:46.169 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052449587999999998 2024-09-16T20:50:46.169 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.169 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.170 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:46.170 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936333+0000", 2024-09-16T20:50:46.170 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010511800000000001 2024-09-16T20:50:46.170 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.170 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.170 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:46.170 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936437+0000", 2024-09-16T20:50:46.170 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000104141 2024-09-16T20:50:46.170 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.170 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:46.170 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.170 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.170 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.171 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5206 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 1167360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:46.171 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.883802+0000", 2024-09-16T20:50:46.171 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 22.210895667999999, 2024-09-16T20:50:46.171 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052800780999999998, 2024-09-16T20:50:46.171 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:46.171 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:46.171 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:46.171 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:46.171 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:50:46.171 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:46.171 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5206 2024-09-16T20:50:46.171 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.171 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:46.171 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.172 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:46.172 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883802+0000", 2024-09-16T20:50:46.172 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.172 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.172 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.172 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:46.172 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883802+0000", 2024-09-16T20:50:46.172 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.172 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.172 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.172 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:46.172 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883933+0000", 2024-09-16T20:50:46.172 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013182999999999999 2024-09-16T20:50:46.173 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.173 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.173 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:46.173 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883937+0000", 2024-09-16T20:50:46.173 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.771e-06 2024-09-16T20:50:46.173 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.173 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.173 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:46.173 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883953+0000", 2024-09-16T20:50:46.173 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5409000000000001e-05 2024-09-16T20:50:46.173 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.173 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.173 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:46.173 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883959+0000", 2024-09-16T20:50:46.174 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0700000000000003e-06 2024-09-16T20:50:46.174 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.174 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.174 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:46.174 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936476+0000", 2024-09-16T20:50:46.174 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052517434000000002 2024-09-16T20:50:46.174 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.174 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.174 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:46.174 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936529+0000", 2024-09-16T20:50:46.174 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2969000000000001e-05 2024-09-16T20:50:46.174 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.174 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.174 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:46.175 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936602+0000", 2024-09-16T20:50:46.175 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3298000000000002e-05 2024-09-16T20:50:46.175 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.175 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:46.175 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.175 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.175 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.175 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5207 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 1904640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:46.175 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.883980+0000", 2024-09-16T20:50:46.175 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 22.210717459000001, 2024-09-16T20:50:46.175 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052759922000000001, 2024-09-16T20:50:46.175 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:46.175 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:46.175 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:46.176 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:46.176 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:50:46.176 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:46.176 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5207 2024-09-16T20:50:46.176 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.176 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:46.176 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.176 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:46.176 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883980+0000", 2024-09-16T20:50:46.176 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.176 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.176 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.176 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:46.177 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883980+0000", 2024-09-16T20:50:46.177 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.177 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.177 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.177 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:46.177 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884126+0000", 2024-09-16T20:50:46.177 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014577399999999999 2024-09-16T20:50:46.177 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.177 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.177 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:46.177 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884130+0000", 2024-09-16T20:50:46.177 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9729999999999999e-06 2024-09-16T20:50:46.177 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.177 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.177 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:46.178 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884139+0000", 2024-09-16T20:50:46.178 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8660000000000001e-06 2024-09-16T20:50:46.178 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.178 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.178 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:46.178 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884144+0000", 2024-09-16T20:50:46.178 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3669999999999999e-06 2024-09-16T20:50:46.178 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.178 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.178 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:46.178 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936635+0000", 2024-09-16T20:50:46.178 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052491705 2024-09-16T20:50:46.178 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.178 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.179 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:46.179 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936679+0000", 2024-09-16T20:50:46.179 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3105999999999997e-05 2024-09-16T20:50:46.179 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.179 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.179 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:46.179 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936740+0000", 2024-09-16T20:50:46.179 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1130999999999995e-05 2024-09-16T20:50:46.179 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.179 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:46.179 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.179 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.179 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.179 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5208 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 131072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:46.180 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.884165+0000", 2024-09-16T20:50:46.180 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 22.210532163, 2024-09-16T20:50:46.180 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052746603000000003, 2024-09-16T20:50:46.180 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:46.180 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:46.180 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:46.180 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:46.180 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:50:46.180 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:46.180 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5208 2024-09-16T20:50:46.180 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.180 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:46.180 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.180 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:46.180 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884165+0000", 2024-09-16T20:50:46.181 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.181 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.181 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.181 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:46.181 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884165+0000", 2024-09-16T20:50:46.181 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.181 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.181 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.181 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:46.181 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884463+0000", 2024-09-16T20:50:46.181 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00029801699999999998 2024-09-16T20:50:46.181 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.181 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.181 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:46.182 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884467+0000", 2024-09-16T20:50:46.182 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8210000000000003e-06 2024-09-16T20:50:46.182 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.182 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.182 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:46.182 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884477+0000", 2024-09-16T20:50:46.182 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.7030000000000001e-06 2024-09-16T20:50:46.182 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.182 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.182 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:46.182 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884485+0000", 2024-09-16T20:50:46.182 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9710000000000004e-06 2024-09-16T20:50:46.182 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.183 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.183 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:46.183 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936785+0000", 2024-09-16T20:50:46.183 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052300665000000003 2024-09-16T20:50:46.183 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.183 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.183 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:46.183 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936851+0000", 2024-09-16T20:50:46.183 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6159000000000001e-05 2024-09-16T20:50:46.183 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.183 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.183 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:46.183 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936912+0000", 2024-09-16T20:50:46.184 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0266999999999997e-05 2024-09-16T20:50:46.184 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.184 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:46.184 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.184 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.184 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.184 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5209 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 1925120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:46.184 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.884504+0000", 2024-09-16T20:50:46.184 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 22.210193277999998, 2024-09-16T20:50:46.184 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052575445999999998, 2024-09-16T20:50:46.184 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:46.184 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:46.184 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:46.184 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:46.185 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:50:46.185 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:46.185 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5209 2024-09-16T20:50:46.185 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.185 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:46.185 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.185 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:46.185 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884504+0000", 2024-09-16T20:50:46.185 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.185 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.185 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.185 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:46.185 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884504+0000", 2024-09-16T20:50:46.185 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.186 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.186 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.186 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:46.186 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884741+0000", 2024-09-16T20:50:46.186 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00023679899999999999 2024-09-16T20:50:46.186 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.186 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.186 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:46.186 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884747+0000", 2024-09-16T20:50:46.186 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0009999999999998e-06 2024-09-16T20:50:46.186 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.186 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.186 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:46.186 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884764+0000", 2024-09-16T20:50:46.187 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7371999999999999e-05 2024-09-16T20:50:46.187 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.187 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.187 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:46.187 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884772+0000", 2024-09-16T20:50:46.187 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1349999999999992e-06 2024-09-16T20:50:46.187 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.187 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.187 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:46.187 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936956+0000", 2024-09-16T20:50:46.187 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052183754999999998 2024-09-16T20:50:46.187 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.187 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.187 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:46.188 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937003+0000", 2024-09-16T20:50:46.188 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6925000000000002e-05 2024-09-16T20:50:46.188 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.188 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.188 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:46.188 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937079+0000", 2024-09-16T20:50:46.188 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6458999999999994e-05 2024-09-16T20:50:46.188 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.188 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:46.188 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.188 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.188 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.188 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5210 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 151552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:46.188 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.884799+0000", 2024-09-16T20:50:46.189 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 22.209898009, 2024-09-16T20:50:46.189 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052462483999999997, 2024-09-16T20:50:46.189 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:46.189 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:46.189 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:46.189 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:46.189 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:50:46.189 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:46.189 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5210 2024-09-16T20:50:46.189 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.189 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:46.189 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.189 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:46.190 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884799+0000", 2024-09-16T20:50:46.190 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.190 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.190 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.190 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:46.190 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884799+0000", 2024-09-16T20:50:46.190 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.190 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.190 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.190 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:46.190 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884990+0000", 2024-09-16T20:50:46.190 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019051800000000001 2024-09-16T20:50:46.190 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.190 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.190 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:46.191 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884996+0000", 2024-09-16T20:50:46.191 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0689999999999998e-06 2024-09-16T20:50:46.191 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.191 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.191 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:46.191 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.885008+0000", 2024-09-16T20:50:46.191 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2153999999999999e-05 2024-09-16T20:50:46.191 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.191 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.191 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:46.191 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.885016+0000", 2024-09-16T20:50:46.191 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2770000000000005e-06 2024-09-16T20:50:46.191 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.192 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.192 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:46.192 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937128+0000", 2024-09-16T20:50:46.192 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052111377 2024-09-16T20:50:46.192 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.192 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.192 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:46.192 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937188+0000", 2024-09-16T20:50:46.192 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9933999999999999e-05 2024-09-16T20:50:46.192 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.192 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.192 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:46.192 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937262+0000", 2024-09-16T20:50:46.192 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4154999999999997e-05 2024-09-16T20:50:46.193 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.193 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:46.193 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.193 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.193 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.193 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5211 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 1945600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:46.193 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.885030+0000", 2024-09-16T20:50:46.193 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 22.209666924, 2024-09-16T20:50:46.193 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052369470000000001, 2024-09-16T20:50:46.193 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:46.193 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:46.193 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:46.193 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:46.193 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:50:46.194 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:46.194 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5211 2024-09-16T20:50:46.194 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.194 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:46.194 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.194 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:46.194 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.885030+0000", 2024-09-16T20:50:46.194 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.194 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.194 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.194 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:46.194 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.885030+0000", 2024-09-16T20:50:46.194 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.194 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.195 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.195 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:46.195 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.885219+0000", 2024-09-16T20:50:46.195 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000188278 2024-09-16T20:50:46.195 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.195 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.195 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:46.195 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.885223+0000", 2024-09-16T20:50:46.195 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7849999999999999e-06 2024-09-16T20:50:46.195 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.195 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.195 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:46.195 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.885233+0000", 2024-09-16T20:50:46.195 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.5060000000000001e-06 2024-09-16T20:50:46.196 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.196 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.196 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:46.196 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.885240+0000", 2024-09-16T20:50:46.196 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9630000000000001e-06 2024-09-16T20:50:46.196 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.196 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.196 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:46.196 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937304+0000", 2024-09-16T20:50:46.196 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052064619999999999 2024-09-16T20:50:46.196 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.196 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.196 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:46.196 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937347+0000", 2024-09-16T20:50:46.197 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2407000000000002e-05 2024-09-16T20:50:46.197 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.197 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.197 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:46.197 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937400+0000", 2024-09-16T20:50:46.197 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2911000000000002e-05 2024-09-16T20:50:46.197 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.197 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:46.197 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.197 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.197 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:46.197 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:46.199 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:50:46.199 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:46.197+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:46.299 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:46.298+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:46.348 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:46.348 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:50:46.348 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:50:46.348 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:46.356 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:50:46.440 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:46.439+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:46.501 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:46.500+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:46.503 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:46.503 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:50:46.503 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:50:46.503 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:50:46.503 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:46.512 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:50:46.596 INFO:teuthology.orchestra.run.smithi029.stderr:nodeep-scrub is set 2024-09-16T20:50:46.602 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:46.601+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:46.679 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:46.679 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:50:46.679 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:50:46.679 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:50:46.679 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.679 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:50:46.679 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:50:46.679 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 173.42323304999999, 2024-09-16T20:50:46.679 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67502346999999996, 2024-09-16T20:50:46.679 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:46.680 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:46.680 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:46.680 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:46.680 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:50:46.680 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:50:46.680 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 1 2024-09-16T20:50:46.680 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.680 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:46.680 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.680 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:46.680 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:50:46.680 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.680 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.680 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.680 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:46.681 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:50:46.681 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.681 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.681 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.681 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:46.681 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254128+0000", 2024-09-16T20:50:46.681 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.463e-06 2024-09-16T20:50:46.681 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.681 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.681 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:46.681 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254135+0000", 2024-09-16T20:50:46.681 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1579999999999999e-06 2024-09-16T20:50:46.681 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.681 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.681 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:46.682 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254146+0000", 2024-09-16T20:50:46.682 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1341e-05 2024-09-16T20:50:46.682 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.682 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.682 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:46.682 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254155+0000", 2024-09-16T20:50:46.682 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.1549999999999996e-06 2024-09-16T20:50:46.682 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.682 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.682 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:46.682 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254221+0000", 2024-09-16T20:50:46.682 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5470999999999997e-05 2024-09-16T20:50:46.682 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.682 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.683 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:46.683 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254231+0000", 2024-09-16T20:50:46.683 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0592e-05 2024-09-16T20:50:46.683 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.683 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.683 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:46.683 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926711+0000", 2024-09-16T20:50:46.683 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67247946599999997 2024-09-16T20:50:46.683 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.683 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.683 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:46.683 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926910+0000", 2024-09-16T20:50:46.683 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019934 2024-09-16T20:50:46.683 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.684 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.684 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:50:46.684 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.927159+0000", 2024-09-16T20:50:46.684 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00024876599999999998 2024-09-16T20:50:46.684 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.684 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.684 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:46.684 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.928797+0000", 2024-09-16T20:50:46.684 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001638308 2024-09-16T20:50:46.684 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.684 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.684 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:46.684 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929026+0000", 2024-09-16T20:50:46.684 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00022876099999999999 2024-09-16T20:50:46.685 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.685 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.685 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:46.685 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929102+0000", 2024-09-16T20:50:46.685 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6184000000000001e-05 2024-09-16T20:50:46.685 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.685 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.685 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:46.685 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929150+0000", 2024-09-16T20:50:46.685 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7465e-05 2024-09-16T20:50:46.685 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.685 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:46.685 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.685 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.685 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.686 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3574 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:46.686 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:50:46.686 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 117.196069518, 2024-09-16T20:50:46.686 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053579441999999998, 2024-09-16T20:50:46.686 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:46.686 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:46.686 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:46.686 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:46.686 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:46.686 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:46.686 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3574 2024-09-16T20:50:46.686 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.686 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:46.686 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.686 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:46.687 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:50:46.687 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.687 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.687 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.687 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:46.687 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:50:46.687 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.687 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.687 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.687 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:46.687 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:50:46.687 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2e-07 2024-09-16T20:50:46.687 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.687 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.687 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:46.688 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481292+0000", 2024-09-16T20:50:46.688 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2630000000000002e-06 2024-09-16T20:50:46.688 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.688 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.688 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:46.688 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481297+0000", 2024-09-16T20:50:46.688 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.296e-06 2024-09-16T20:50:46.688 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.688 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.688 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:46.688 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481300+0000", 2024-09-16T20:50:46.688 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7019999999999999e-06 2024-09-16T20:50:46.688 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.689 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.689 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:46.689 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525533+0000", 2024-09-16T20:50:46.689 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.044232318 2024-09-16T20:50:46.689 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.689 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.689 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:46.689 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525619+0000", 2024-09-16T20:50:46.689 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5892999999999999e-05 2024-09-16T20:50:46.689 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.689 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.689 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:46.689 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525987+0000", 2024-09-16T20:50:46.690 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00036836099999999999 2024-09-16T20:50:46.690 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.690 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.690 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:46.690 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531779+0000", 2024-09-16T20:50:46.690 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0057918240000000001 2024-09-16T20:50:46.690 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.690 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.690 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:46.690 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534747+0000", 2024-09-16T20:50:46.690 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0029681600000000001 2024-09-16T20:50:46.690 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.690 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.691 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:46.691 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534832+0000", 2024-09-16T20:50:46.691 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5180999999999996e-05 2024-09-16T20:50:46.691 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.691 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.691 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:46.691 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534869+0000", 2024-09-16T20:50:46.691 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7024000000000003e-05 2024-09-16T20:50:46.691 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.691 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:46.691 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.691 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.691 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.691 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3575 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:46.691 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:50:46.692 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 117.196054778, 2024-09-16T20:50:46.692 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053801463000000001, 2024-09-16T20:50:46.692 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:46.692 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:46.692 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:46.692 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:46.692 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:46.692 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:46.692 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3575 2024-09-16T20:50:46.692 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.692 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:46.692 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.692 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:46.693 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:50:46.693 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.693 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.693 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.693 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:46.693 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:50:46.693 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.693 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.693 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.693 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:46.693 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481305+0000", 2024-09-16T20:50:46.693 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5400000000000002e-07 2024-09-16T20:50:46.693 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.693 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.693 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:46.694 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481307+0000", 2024-09-16T20:50:46.694 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7800000000000001e-06 2024-09-16T20:50:46.694 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.694 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.694 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:46.694 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481312+0000", 2024-09-16T20:50:46.694 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6349999999999997e-06 2024-09-16T20:50:46.694 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.694 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.694 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:46.694 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481315+0000", 2024-09-16T20:50:46.694 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7829999999999999e-06 2024-09-16T20:50:46.694 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.694 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.694 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:46.695 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.526876+0000", 2024-09-16T20:50:46.695 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.045561070000000002 2024-09-16T20:50:46.695 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.695 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.695 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:46.695 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.526954+0000", 2024-09-16T20:50:46.695 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8108000000000005e-05 2024-09-16T20:50:46.695 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.695 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.695 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:46.695 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527282+0000", 2024-09-16T20:50:46.695 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000327483 2024-09-16T20:50:46.695 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.695 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.695 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:46.696 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531784+0000", 2024-09-16T20:50:46.696 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0045020169999999997 2024-09-16T20:50:46.696 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.696 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.696 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:46.696 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534921+0000", 2024-09-16T20:50:46.696 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.003137791 2024-09-16T20:50:46.696 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.696 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.696 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:46.696 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534976+0000", 2024-09-16T20:50:46.696 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4953000000000001e-05 2024-09-16T20:50:46.696 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.696 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.697 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:46.697 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535106+0000", 2024-09-16T20:50:46.697 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000129589 2024-09-16T20:50:46.697 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.697 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:46.697 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.697 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.697 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.697 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3576 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:46.697 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:50:46.697 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 117.196040866, 2024-09-16T20:50:46.697 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053955436000000002, 2024-09-16T20:50:46.697 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:46.697 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:46.698 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:46.698 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:46.698 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:46.698 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:46.698 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3576 2024-09-16T20:50:46.698 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.698 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:46.698 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.698 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:46.698 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:50:46.698 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.698 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.698 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.698 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:46.699 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:50:46.699 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.699 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.699 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.699 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:46.699 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481319+0000", 2024-09-16T20:50:46.699 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.53e-07 2024-09-16T20:50:46.699 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.699 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.699 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:46.699 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481320+0000", 2024-09-16T20:50:46.699 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5230000000000001e-06 2024-09-16T20:50:46.699 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.699 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.700 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:46.700 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481330+0000", 2024-09-16T20:50:46.700 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0190000000000001e-05 2024-09-16T20:50:46.700 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.700 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.700 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:46.700 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481333+0000", 2024-09-16T20:50:46.700 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.151e-06 2024-09-16T20:50:46.700 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.700 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.700 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:46.700 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527689+0000", 2024-09-16T20:50:46.700 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046355192000000003 2024-09-16T20:50:46.700 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.701 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.701 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:46.701 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527764+0000", 2024-09-16T20:50:46.701 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5776000000000001e-05 2024-09-16T20:50:46.701 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.701 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.701 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:46.701 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528141+0000", 2024-09-16T20:50:46.701 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00037647599999999999 2024-09-16T20:50:46.701 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.701 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.701 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:46.701 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531788+0000", 2024-09-16T20:50:46.701 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0036470679999999998 2024-09-16T20:50:46.701 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.702 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.702 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:46.702 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535174+0000", 2024-09-16T20:50:46.702 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0033862670000000001 2024-09-16T20:50:46.702 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.702 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.702 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:46.702 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535236+0000", 2024-09-16T20:50:46.702 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1797999999999993e-05 2024-09-16T20:50:46.702 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.702 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.702 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:46.702 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535274+0000", 2024-09-16T20:50:46.702 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7741999999999999e-05 2024-09-16T20:50:46.703 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.703 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:46.703 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.703 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.703 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.703 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3577 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:46.703 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:50:46.703 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 117.19601926599999, 2024-09-16T20:50:46.703 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054073807000000002, 2024-09-16T20:50:46.703 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:46.703 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:46.703 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:46.703 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:46.703 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:46.704 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:46.704 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3577 2024-09-16T20:50:46.704 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.704 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:46.704 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.704 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:46.704 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:50:46.704 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.704 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.704 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.704 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:46.704 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:50:46.704 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.704 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.705 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.705 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:46.705 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:50:46.705 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.96e-07 2024-09-16T20:50:46.705 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.705 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.705 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:46.705 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481342+0000", 2024-09-16T20:50:46.705 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.714e-06 2024-09-16T20:50:46.705 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.705 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.705 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:46.705 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481349+0000", 2024-09-16T20:50:46.706 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4930000000000001e-06 2024-09-16T20:50:46.706 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.706 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.706 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:46.706 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481353+0000", 2024-09-16T20:50:46.706 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4879999999999999e-06 2024-09-16T20:50:46.706 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.706 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.706 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:46.706 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528619+0000", 2024-09-16T20:50:46.706 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047266351999999998 2024-09-16T20:50:46.706 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.706 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.706 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:46.706 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528706+0000", 2024-09-16T20:50:46.707 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6796e-05 2024-09-16T20:50:46.707 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.707 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.707 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:46.707 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529148+0000", 2024-09-16T20:50:46.707 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00044182999999999999 2024-09-16T20:50:46.707 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.707 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.707 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:46.707 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531792+0000", 2024-09-16T20:50:46.707 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0026443119999999998 2024-09-16T20:50:46.707 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.707 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.707 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:46.708 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535319+0000", 2024-09-16T20:50:46.708 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0035266439999999998 2024-09-16T20:50:46.708 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.708 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.708 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:46.708 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535372+0000", 2024-09-16T20:50:46.708 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3207000000000001e-05 2024-09-16T20:50:46.708 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.708 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.708 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:46.708 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535414+0000", 2024-09-16T20:50:46.708 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1675000000000003e-05 2024-09-16T20:50:46.708 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.709 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:46.709 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.709 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.709 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.709 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3578 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:46.709 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:50:46.709 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 117.195998097, 2024-09-16T20:50:46.709 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054189586999999997, 2024-09-16T20:50:46.709 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:46.709 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:46.709 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:46.709 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:46.709 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:46.710 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:46.710 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3578 2024-09-16T20:50:46.710 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.710 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:46.710 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.710 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:46.710 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:50:46.710 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.710 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.710 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.710 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:46.710 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:50:46.710 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.711 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.711 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.711 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:46.711 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:50:46.711 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0800000000000001e-07 2024-09-16T20:50:46.711 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.711 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.711 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:46.711 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481363+0000", 2024-09-16T20:50:46.711 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0339999999999999e-06 2024-09-16T20:50:46.711 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.711 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.712 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:46.712 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481371+0000", 2024-09-16T20:50:46.712 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4440000000000002e-06 2024-09-16T20:50:46.712 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.712 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.712 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:46.712 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481374+0000", 2024-09-16T20:50:46.712 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2049999999999998e-06 2024-09-16T20:50:46.712 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.712 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.712 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:46.712 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529555+0000", 2024-09-16T20:50:46.712 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048181202999999999 2024-09-16T20:50:46.712 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.713 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.713 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:46.713 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529616+0000", 2024-09-16T20:50:46.713 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0912000000000002e-05 2024-09-16T20:50:46.713 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.713 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.713 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:46.713 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529939+0000", 2024-09-16T20:50:46.713 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00032269800000000001 2024-09-16T20:50:46.713 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.713 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.713 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:46.713 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532602+0000", 2024-09-16T20:50:46.713 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0026634940000000002 2024-09-16T20:50:46.714 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.714 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.714 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:46.714 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535465+0000", 2024-09-16T20:50:46.714 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.002862149 2024-09-16T20:50:46.714 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.714 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.714 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:46.714 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535516+0000", 2024-09-16T20:50:46.714 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1696000000000001e-05 2024-09-16T20:50:46.714 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.715 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.715 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:46.715 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535551+0000", 2024-09-16T20:50:46.715 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4443999999999998e-05 2024-09-16T20:50:46.715 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.715 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:46.715 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.715 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.715 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.715 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3579 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:46.715 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:50:46.715 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 117.195979658, 2024-09-16T20:50:46.715 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054295151, 2024-09-16T20:50:46.716 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:46.716 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:46.716 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:46.716 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:46.716 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:46.716 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:46.716 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3579 2024-09-16T20:50:46.716 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.716 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:46.716 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.716 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:46.716 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:50:46.716 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.716 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.717 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.717 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:46.717 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:50:46.717 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.717 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.717 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.717 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:46.717 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:50:46.717 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3300000000000001e-07 2024-09-16T20:50:46.717 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.717 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.717 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:46.718 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481381+0000", 2024-09-16T20:50:46.718 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.649e-06 2024-09-16T20:50:46.718 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.718 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.718 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:46.718 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481387+0000", 2024-09-16T20:50:46.718 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4519999999999996e-06 2024-09-16T20:50:46.718 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.718 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.718 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:46.718 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481390+0000", 2024-09-16T20:50:46.718 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0170000000000001e-06 2024-09-16T20:50:46.718 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.718 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.719 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:46.719 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530756+0000", 2024-09-16T20:50:46.719 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049365882999999999 2024-09-16T20:50:46.719 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.719 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.719 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:46.719 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530845+0000", 2024-09-16T20:50:46.719 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9197000000000007e-05 2024-09-16T20:50:46.719 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.719 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.719 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:46.719 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531191+0000", 2024-09-16T20:50:46.719 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00034562500000000001 2024-09-16T20:50:46.719 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.720 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.720 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:46.720 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534078+0000", 2024-09-16T20:50:46.720 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0028878179999999999 2024-09-16T20:50:46.720 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.720 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.720 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:46.720 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535589+0000", 2024-09-16T20:50:46.720 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00151085 2024-09-16T20:50:46.720 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.720 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.720 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:46.720 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535639+0000", 2024-09-16T20:50:46.720 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9697000000000002e-05 2024-09-16T20:50:46.721 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.721 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.721 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:46.721 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535675+0000", 2024-09-16T20:50:46.721 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5729999999999998e-05 2024-09-16T20:50:46.721 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.721 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:46.721 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.721 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.721 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.721 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3580 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:46.721 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:50:46.721 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 117.19596602599999, 2024-09-16T20:50:46.722 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054397361999999998, 2024-09-16T20:50:46.722 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:46.722 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:46.722 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:46.722 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:46.722 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:46.722 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:46.722 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3580 2024-09-16T20:50:46.722 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.722 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:46.722 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.722 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:46.722 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:50:46.722 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.723 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.723 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.723 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:46.723 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:50:46.723 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.723 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.723 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.723 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:46.723 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:50:46.723 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2600000000000001e-07 2024-09-16T20:50:46.723 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.723 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.723 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:46.723 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481395+0000", 2024-09-16T20:50:46.724 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.544e-06 2024-09-16T20:50:46.724 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.724 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.724 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:46.724 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481404+0000", 2024-09-16T20:50:46.724 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8850000000000005e-06 2024-09-16T20:50:46.724 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.724 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.724 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:46.724 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481407+0000", 2024-09-16T20:50:46.724 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1540000000000002e-06 2024-09-16T20:50:46.724 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.724 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.725 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:46.725 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531823+0000", 2024-09-16T20:50:46.725 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050415637999999999 2024-09-16T20:50:46.725 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.725 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.725 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:46.725 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531895+0000", 2024-09-16T20:50:46.725 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2179000000000001e-05 2024-09-16T20:50:46.725 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.725 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.725 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:46.725 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532236+0000", 2024-09-16T20:50:46.725 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00034073799999999999 2024-09-16T20:50:46.725 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.726 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.726 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:46.726 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534083+0000", 2024-09-16T20:50:46.726 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001847298 2024-09-16T20:50:46.726 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.726 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.726 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:46.726 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535714+0000", 2024-09-16T20:50:46.726 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0016316810000000001 2024-09-16T20:50:46.726 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.726 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.726 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:46.726 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535759+0000", 2024-09-16T20:50:46.726 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4478999999999999e-05 2024-09-16T20:50:46.727 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.727 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.727 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:46.727 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535791+0000", 2024-09-16T20:50:46.727 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.154e-05 2024-09-16T20:50:46.727 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.727 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:46.727 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.727 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.727 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.727 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3581 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:46.727 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:50:46.727 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 117.195944029, 2024-09-16T20:50:46.727 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.057094403000000002, 2024-09-16T20:50:46.727 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:46.728 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:46.728 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:46.728 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:46.728 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:46.728 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:46.728 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3581 2024-09-16T20:50:46.728 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.728 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:46.728 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.728 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:46.728 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:50:46.728 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.728 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.728 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.729 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:46.729 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:50:46.729 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.729 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.729 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.729 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:46.729 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:50:46.729 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8799999999999998e-07 2024-09-16T20:50:46.729 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.729 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.729 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:46.729 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481418+0000", 2024-09-16T20:50:46.729 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1859999999999999e-06 2024-09-16T20:50:46.730 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.730 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.730 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:46.730 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481423+0000", 2024-09-16T20:50:46.730 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8279999999999999e-06 2024-09-16T20:50:46.730 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.730 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.730 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:46.730 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481426+0000", 2024-09-16T20:50:46.730 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0110000000000001e-06 2024-09-16T20:50:46.730 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.730 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.730 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:46.731 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532631+0000", 2024-09-16T20:50:46.731 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051204514999999999 2024-09-16T20:50:46.731 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.731 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.731 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:46.731 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.533381+0000", 2024-09-16T20:50:46.731 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00075029000000000003 2024-09-16T20:50:46.731 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.731 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.731 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:46.731 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.533739+0000", 2024-09-16T20:50:46.731 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00035773100000000002 2024-09-16T20:50:46.731 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.731 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.732 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:46.732 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535123+0000", 2024-09-16T20:50:46.732 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0013836160000000001 2024-09-16T20:50:46.732 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.732 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.732 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:46.732 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538413+0000", 2024-09-16T20:50:46.732 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0032902510000000001 2024-09-16T20:50:46.732 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.732 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.732 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:46.732 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538470+0000", 2024-09-16T20:50:46.732 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6697999999999998e-05 2024-09-16T20:50:46.733 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.733 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.733 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:46.733 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538510+0000", 2024-09-16T20:50:46.733 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9988999999999999e-05 2024-09-16T20:50:46.733 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.733 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:46.733 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.733 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.733 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.733 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:12732 3.d 3:b752444a:::rbd_header.110e747f0779:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-16T20:50:46.733 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:50:46.733 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 101.908871451, 2024-09-16T20:50:46.733 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.065302467000000003, 2024-09-16T20:50:46.734 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:46.734 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:46.734 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:46.734 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:46.734 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:50:46.734 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:50:46.734 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 12732 2024-09-16T20:50:46.734 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.734 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:46.734 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.734 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:46.734 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:50:46.734 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.734 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.734 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.735 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:46.735 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:50:46.735 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.735 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.735 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.735 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:46.735 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768490+0000", 2024-09-16T20:50:46.735 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2859999999999998e-06 2024-09-16T20:50:46.735 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.735 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.735 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:46.735 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768501+0000", 2024-09-16T20:50:46.735 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1226999999999999e-05 2024-09-16T20:50:46.735 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.735 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.735 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:46.736 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768517+0000", 2024-09-16T20:50:46.736 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.592e-05 2024-09-16T20:50:46.736 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.736 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.736 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:46.736 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768528+0000", 2024-09-16T20:50:46.736 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0689999999999999e-05 2024-09-16T20:50:46.736 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.736 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.736 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:46.736 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768584+0000", 2024-09-16T20:50:46.736 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5889e-05 2024-09-16T20:50:46.736 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.736 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.736 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:46.737 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768720+0000", 2024-09-16T20:50:46.737 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000136718 2024-09-16T20:50:46.737 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.737 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.737 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:46.737 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.769066+0000", 2024-09-16T20:50:46.737 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000346013 2024-09-16T20:50:46.737 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.737 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.737 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:46.737 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833659+0000", 2024-09-16T20:50:46.737 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.064592822999999994 2024-09-16T20:50:46.737 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.737 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.737 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:46.738 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833719+0000", 2024-09-16T20:50:46.738 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9441000000000002e-05 2024-09-16T20:50:46.738 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.738 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.738 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:46.738 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833758+0000", 2024-09-16T20:50:46.738 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9122000000000003e-05 2024-09-16T20:50:46.738 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.738 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.738 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:46.738 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833790+0000", 2024-09-16T20:50:46.738 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2338000000000001e-05 2024-09-16T20:50:46.738 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.738 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:46.738 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.739 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.739 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.739 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.5089.0:12732 3.d e73/23 ondisk, result = 0)", 2024-09-16T20:50:46.739 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:50:46.739 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 101.907269281, 2024-09-16T20:50:46.739 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.063706456999999994, 2024-09-16T20:50:46.739 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:46.739 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:46.739 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:46.739 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:46.739 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.739 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:46.739 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:50:46.739 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.740 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.740 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.740 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:46.740 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:50:46.740 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.740 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.740 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.740 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:46.740 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770091+0000", 2024-09-16T20:50:46.740 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.53e-06 2024-09-16T20:50:46.740 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.740 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.740 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:46.740 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770104+0000", 2024-09-16T20:50:46.741 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2203e-05 2024-09-16T20:50:46.741 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.741 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.741 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:46.741 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833637+0000", 2024-09-16T20:50:46.741 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.063533201999999997 2024-09-16T20:50:46.741 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.741 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.741 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:46.741 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833655+0000", 2024-09-16T20:50:46.741 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8369e-05 2024-09-16T20:50:46.741 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.741 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.741 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:46.742 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833707+0000", 2024-09-16T20:50:46.742 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2061999999999997e-05 2024-09-16T20:50:46.742 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.742 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.742 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:46.742 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833715+0000", 2024-09-16T20:50:46.742 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5179999999999998e-06 2024-09-16T20:50:46.742 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.742 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.742 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:46.742 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833796+0000", 2024-09-16T20:50:46.742 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1573e-05 2024-09-16T20:50:46.742 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.742 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:46.742 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.743 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.743 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.743 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3286 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 245760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:46.743 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:50:46.743 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 56.636207343999999, 2024-09-16T20:50:46.743 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052523564000000002, 2024-09-16T20:50:46.743 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:46.743 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:46.743 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:46.743 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:46.743 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:46.743 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:46.743 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3286 2024-09-16T20:50:46.743 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.743 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:46.744 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.744 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:46.744 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:50:46.744 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.744 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.744 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.744 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:46.744 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:50:46.744 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.744 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.744 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.744 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:46.744 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:50:46.744 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3900000000000001e-07 2024-09-16T20:50:46.744 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.745 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.745 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:46.745 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041153+0000", 2024-09-16T20:50:46.745 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.119e-06 2024-09-16T20:50:46.745 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.745 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.745 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:46.745 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041156+0000", 2024-09-16T20:50:46.745 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0000000000000001e-06 2024-09-16T20:50:46.745 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.745 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.745 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:46.745 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041159+0000", 2024-09-16T20:50:46.745 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5859999999999999e-06 2024-09-16T20:50:46.745 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.746 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.746 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:46.746 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093529+0000", 2024-09-16T20:50:46.746 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052370591000000001 2024-09-16T20:50:46.746 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.746 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.746 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:46.746 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093570+0000", 2024-09-16T20:50:46.746 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0161999999999999e-05 2024-09-16T20:50:46.746 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.746 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.746 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:46.746 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093675+0000", 2024-09-16T20:50:46.746 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010586699999999999 2024-09-16T20:50:46.746 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.746 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:46.747 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.747 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.747 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.747 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3287 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 250880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:46.747 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:50:46.747 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 56.636197295000002, 2024-09-16T20:50:46.747 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052781203999999998, 2024-09-16T20:50:46.747 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:46.747 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:46.747 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:46.747 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:46.747 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:46.747 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:46.748 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3287 2024-09-16T20:50:46.748 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.748 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:46.748 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.748 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:46.748 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:50:46.748 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.748 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.748 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.748 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:46.748 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:50:46.748 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.748 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.748 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.748 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:46.749 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:50:46.749 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1e-07 2024-09-16T20:50:46.749 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.749 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.749 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:46.749 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041163+0000", 2024-09-16T20:50:46.749 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0690000000000001e-06 2024-09-16T20:50:46.749 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.749 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.749 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:46.749 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041166+0000", 2024-09-16T20:50:46.749 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.982e-06 2024-09-16T20:50:46.749 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.749 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.750 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:46.750 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041169+0000", 2024-09-16T20:50:46.750 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.593e-06 2024-09-16T20:50:46.750 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.750 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.750 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:46.750 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093709+0000", 2024-09-16T20:50:46.750 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052540492000000001 2024-09-16T20:50:46.750 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.750 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.750 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:46.750 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093792+0000", 2024-09-16T20:50:46.750 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2436999999999997e-05 2024-09-16T20:50:46.750 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.750 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.751 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:46.751 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093943+0000", 2024-09-16T20:50:46.751 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000151421 2024-09-16T20:50:46.751 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.751 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:46.751 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.751 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.751 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.751 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3288 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 256000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:46.751 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:50:46.751 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 56.636187421999999, 2024-09-16T20:50:46.751 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052928269, 2024-09-16T20:50:46.751 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:46.751 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:46.752 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:46.752 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:46.752 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:46.752 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:46.752 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3288 2024-09-16T20:50:46.752 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.752 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:46.752 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.752 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:46.752 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:50:46.752 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.752 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.752 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.752 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:46.753 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:50:46.753 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.753 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.753 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.753 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:46.753 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:50:46.753 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2700000000000001e-07 2024-09-16T20:50:46.753 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.753 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.753 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:46.753 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041173+0000", 2024-09-16T20:50:46.753 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.113e-06 2024-09-16T20:50:46.753 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.753 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.753 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:46.754 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041176+0000", 2024-09-16T20:50:46.754 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9799999999999998e-06 2024-09-16T20:50:46.754 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.754 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.754 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:46.754 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041179+0000", 2024-09-16T20:50:46.754 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.667e-06 2024-09-16T20:50:46.754 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.754 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.754 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:46.754 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093957+0000", 2024-09-16T20:50:46.754 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052778703000000003 2024-09-16T20:50:46.754 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.754 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.754 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:46.755 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093999+0000", 2024-09-16T20:50:46.755 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1300000000000001e-05 2024-09-16T20:50:46.755 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.755 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.755 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:46.755 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094100+0000", 2024-09-16T20:50:46.755 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101279 2024-09-16T20:50:46.755 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.755 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:46.755 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.755 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.755 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.755 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3289 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 261120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:46.755 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:50:46.755 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 56.636177426000003, 2024-09-16T20:50:46.755 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053182976, 2024-09-16T20:50:46.756 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:46.756 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:46.756 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:46.756 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:46.756 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:46.756 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:46.756 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3289 2024-09-16T20:50:46.756 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.756 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:46.756 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.756 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:46.756 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:50:46.756 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.756 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.757 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.757 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:46.757 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:50:46.757 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.757 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.757 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.757 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:46.757 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:50:46.757 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2999999999999999e-07 2024-09-16T20:50:46.757 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.757 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.757 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:46.757 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041183+0000", 2024-09-16T20:50:46.757 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0720000000000001e-06 2024-09-16T20:50:46.758 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.758 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.758 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:46.758 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041186+0000", 2024-09-16T20:50:46.758 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9869999999999999e-06 2024-09-16T20:50:46.758 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.758 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.758 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:46.758 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041189+0000", 2024-09-16T20:50:46.758 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.587e-06 2024-09-16T20:50:46.758 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.758 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.758 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:46.758 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094136+0000", 2024-09-16T20:50:46.759 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052947270999999997 2024-09-16T20:50:46.759 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.759 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.759 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:46.759 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094213+0000", 2024-09-16T20:50:46.759 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7552000000000006e-05 2024-09-16T20:50:46.759 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.759 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.759 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:46.759 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094365+0000", 2024-09-16T20:50:46.759 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00015127700000000001 2024-09-16T20:50:46.759 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.759 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:46.759 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.759 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.760 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.760 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:2196 3.d 3:b1fe01a0:::rbd_data.110e7ef3004.0000000000000000:2d [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e136)", 2024-09-16T20:50:46.760 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:22.739057+0000", 2024-09-16T20:50:46.760 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.938302626999999, 2024-09-16T20:50:46.760 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11999464899999999, 2024-09-16T20:50:46.760 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:46.760 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:46.760 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:46.760 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:46.760 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:50:46.760 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:46.760 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2196 2024-09-16T20:50:46.760 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.760 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:46.761 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.761 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:46.761 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.739057+0000", 2024-09-16T20:50:46.761 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.761 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.761 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.761 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:46.761 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.739057+0000", 2024-09-16T20:50:46.761 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.761 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.761 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.761 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:46.761 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.739057+0000", 2024-09-16T20:50:46.761 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1399999999999997e-07 2024-09-16T20:50:46.762 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.762 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.762 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:46.762 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.739060+0000", 2024-09-16T20:50:46.762 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6019999999999998e-06 2024-09-16T20:50:46.762 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.762 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.762 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:46.762 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.858755+0000", 2024-09-16T20:50:46.762 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.119695151 2024-09-16T20:50:46.762 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.762 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.762 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:46.762 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.858778+0000", 2024-09-16T20:50:46.763 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3184999999999999e-05 2024-09-16T20:50:46.763 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.763 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.763 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:46.763 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.858865+0000", 2024-09-16T20:50:46.763 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.7330000000000006e-05 2024-09-16T20:50:46.763 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.763 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.763 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:46.763 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.858952+0000", 2024-09-16T20:50:46.763 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6803000000000002e-05 2024-09-16T20:50:46.763 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.763 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.764 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:46.764 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.859051+0000", 2024-09-16T20:50:46.764 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8963999999999996e-05 2024-09-16T20:50:46.764 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.764 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:46.764 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.764 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.764 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.764 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6061.0:5401 3.13 3:c8554348:::rbd_data.110e16eafbd.0000000000000011:head [read 577536~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-16T20:50:46.764 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:27.761536+0000", 2024-09-16T20:50:46.764 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 18.915822813999998, 2024-09-16T20:50:46.764 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.003742712, 2024-09-16T20:50:46.764 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:46.764 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:46.765 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:50:46.765 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:46.765 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6061", 2024-09-16T20:50:46.765 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3930128440", 2024-09-16T20:50:46.765 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5401 2024-09-16T20:50:46.765 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.765 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:46.765 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.765 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:46.765 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761536+0000", 2024-09-16T20:50:46.765 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.765 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.765 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.765 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:46.766 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761536+0000", 2024-09-16T20:50:46.766 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.766 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.766 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.766 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:46.766 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761537+0000", 2024-09-16T20:50:46.766 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2900000000000003e-07 2024-09-16T20:50:46.766 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.766 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.766 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:46.766 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761540+0000", 2024-09-16T20:50:46.766 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2320000000000001e-06 2024-09-16T20:50:46.766 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.767 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.767 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:46.767 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761548+0000", 2024-09-16T20:50:46.767 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1289999999999996e-06 2024-09-16T20:50:46.767 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.767 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.767 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:46.767 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761554+0000", 2024-09-16T20:50:46.767 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0820000000000004e-06 2024-09-16T20:50:46.767 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.767 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.767 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:46.767 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761621+0000", 2024-09-16T20:50:46.768 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6266000000000006e-05 2024-09-16T20:50:46.768 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.768 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.768 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:46.768 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761626+0000", 2024-09-16T20:50:46.768 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.614e-06 2024-09-16T20:50:46.768 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.768 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.768 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:46.768 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:28.765163+0000", 2024-09-16T20:50:46.768 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.003536416 2024-09-16T20:50:46.768 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.768 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.769 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:46.769 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:28.765279+0000", 2024-09-16T20:50:46.769 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000116344 2024-09-16T20:50:46.769 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.769 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:46.769 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.769 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.769 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.769 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6061.0:5425 3.13 3:c8554348:::rbd_data.110e16eafbd.0000000000000011:head [read 602112~28672] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-16T20:50:46.769 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:27.767049+0000", 2024-09-16T20:50:46.769 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 18.910310155000001, 2024-09-16T20:50:46.769 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.99847897900000004, 2024-09-16T20:50:46.769 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:46.769 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:46.770 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:50:46.770 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:46.770 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6061", 2024-09-16T20:50:46.770 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3930128440", 2024-09-16T20:50:46.770 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5425 2024-09-16T20:50:46.770 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.770 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:46.770 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.770 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:46.770 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767049+0000", 2024-09-16T20:50:46.770 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.770 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.770 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.770 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:46.771 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767049+0000", 2024-09-16T20:50:46.771 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.771 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.771 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.771 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:46.771 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767050+0000", 2024-09-16T20:50:46.771 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6499999999999999e-07 2024-09-16T20:50:46.771 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.771 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.771 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:46.771 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767052+0000", 2024-09-16T20:50:46.771 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7350000000000001e-06 2024-09-16T20:50:46.771 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.771 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.772 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:46.772 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767060+0000", 2024-09-16T20:50:46.772 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5109999999999997e-06 2024-09-16T20:50:46.772 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.772 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.772 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:46.772 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767066+0000", 2024-09-16T20:50:46.772 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8139999999999997e-06 2024-09-16T20:50:46.772 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.772 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.772 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:46.772 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767128+0000", 2024-09-16T20:50:46.772 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2174999999999993e-05 2024-09-16T20:50:46.773 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.773 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.773 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:46.773 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767133+0000", 2024-09-16T20:50:46.773 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2140000000000004e-06 2024-09-16T20:50:46.773 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.773 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.773 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:46.773 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:28.765441+0000", 2024-09-16T20:50:46.773 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.99830808599999998 2024-09-16T20:50:46.773 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.773 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.774 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:46.774 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:28.765528+0000", 2024-09-16T20:50:46.774 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6879000000000006e-05 2024-09-16T20:50:46.774 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.774 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:46.774 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.774 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.774 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.774 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:25585 3.10 3:0c7c5b23:::rbd_header.110ebf0678ad:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:46.774 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:32.877132+0000", 2024-09-16T20:50:46.774 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 13.800227606, 2024-09-16T20:50:46.774 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054096699999999998, 2024-09-16T20:50:46.774 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:46.774 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:46.775 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:46.775 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:46.775 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:50:46.775 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:46.775 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 25585 2024-09-16T20:50:46.775 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.775 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:46.775 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.775 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:46.775 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.877132+0000", 2024-09-16T20:50:46.775 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.775 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.775 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.776 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:46.776 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.877132+0000", 2024-09-16T20:50:46.776 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.776 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.776 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.776 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:46.776 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931072+0000", 2024-09-16T20:50:46.776 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053939931000000003 2024-09-16T20:50:46.776 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.776 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.776 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:46.776 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931076+0000", 2024-09-16T20:50:46.776 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2860000000000001e-06 2024-09-16T20:50:46.776 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.777 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.777 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:46.777 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931085+0000", 2024-09-16T20:50:46.777 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.3370000000000006e-06 2024-09-16T20:50:46.777 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.777 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.777 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:46.777 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931092+0000", 2024-09-16T20:50:46.777 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9970000000000001e-06 2024-09-16T20:50:46.777 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.777 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.777 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:46.777 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931137+0000", 2024-09-16T20:50:46.778 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5160000000000001e-05 2024-09-16T20:50:46.778 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.778 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.778 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:46.778 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931178+0000", 2024-09-16T20:50:46.778 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0379000000000001e-05 2024-09-16T20:50:46.778 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.778 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.778 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:46.778 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931228+0000", 2024-09-16T20:50:46.778 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0609999999999998e-05 2024-09-16T20:50:46.778 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.778 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:46.778 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.779 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.779 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.779 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:26679 3.c 3:33e2f78f:::rbd_header.110e16eafbd:head [watch ping cookie 140213436013408 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e153)", 2024-09-16T20:50:46.779 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:41.440287+0000", 2024-09-16T20:50:46.779 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 5.2370717310000003, 2024-09-16T20:50:46.779 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.13295633600000001, 2024-09-16T20:50:46.779 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:46.779 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:46.779 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:46.779 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:46.779 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:50:46.779 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:46.779 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 26679 2024-09-16T20:50:46.779 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.780 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:46.780 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.780 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:46.780 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440287+0000", 2024-09-16T20:50:46.780 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.780 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.780 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.780 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:46.780 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440287+0000", 2024-09-16T20:50:46.780 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:46.780 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.780 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.780 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:46.780 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440288+0000", 2024-09-16T20:50:46.781 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0900000000000001e-07 2024-09-16T20:50:46.781 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.781 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.781 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:46.781 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440292+0000", 2024-09-16T20:50:46.781 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3749999999999996e-06 2024-09-16T20:50:46.781 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.781 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.781 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:46.781 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440299+0000", 2024-09-16T20:50:46.781 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8249999999999999e-06 2024-09-16T20:50:46.781 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.781 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.781 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:46.781 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440307+0000", 2024-09-16T20:50:46.782 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4800000000000004e-06 2024-09-16T20:50:46.782 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.782 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.782 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:46.782 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440451+0000", 2024-09-16T20:50:46.782 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014414500000000001 2024-09-16T20:50:46.782 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.782 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.782 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:46.782 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440465+0000", 2024-09-16T20:50:46.782 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.402e-05 2024-09-16T20:50:46.782 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.782 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.782 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:46.783 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.573043+0000", 2024-09-16T20:50:46.783 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.13257855399999999 2024-09-16T20:50:46.783 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.783 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.783 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:46.783 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.573130+0000", 2024-09-16T20:50:46.783 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6168999999999994e-05 2024-09-16T20:50:46.783 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:46.783 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:46.783 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:46.783 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.573244+0000", 2024-09-16T20:50:46.783 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000114159 2024-09-16T20:50:46.783 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.783 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:46.784 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.784 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:46.784 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:46.784 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:46.785 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:50:46.786 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:46.784+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:46.886 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:46.885+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:46.932 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:46.933 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:50:46.933 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:50:46.933 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:46.939 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 4, 5] out_osds: [0, 3] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-16T20:50:46.939 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T20:50:46.939 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-16T20:50:46.939 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-16T20:50:46.941 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:50:46.986 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:46.986+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:47.075 INFO:teuthology.orchestra.run.smithi029.stderr:no valid command found; 10 closest matches: 2024-09-16T20:50:47.075 INFO:teuthology.orchestra.run.smithi029.stderr:0 2024-09-16T20:50:47.075 INFO:teuthology.orchestra.run.smithi029.stderr:1 2024-09-16T20:50:47.075 INFO:teuthology.orchestra.run.smithi029.stderr:2 2024-09-16T20:50:47.075 INFO:teuthology.orchestra.run.smithi029.stderr:abort 2024-09-16T20:50:47.076 INFO:teuthology.orchestra.run.smithi029.stderr:assert 2024-09-16T20:50:47.076 INFO:teuthology.orchestra.run.smithi029.stderr:bench [] [] [] [] 2024-09-16T20:50:47.076 INFO:teuthology.orchestra.run.smithi029.stderr:bluefs debug_inject_read_zeros 2024-09-16T20:50:47.076 INFO:teuthology.orchestra.run.smithi029.stderr:bluefs files list 2024-09-16T20:50:47.076 INFO:teuthology.orchestra.run.smithi029.stderr:bluefs stats 2024-09-16T20:50:47.076 INFO:teuthology.orchestra.run.smithi029.stderr:bluestore allocator dump block 2024-09-16T20:50:47.076 INFO:teuthology.orchestra.run.smithi029.stderr:Error EINVAL: invalid command 2024-09-16T20:50:47.079 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T20:50:47.079 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-16T20:50:47.079 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-16T20:50:47.090 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:47.091 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:50:47.091 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:50:47.091 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:50:47.091 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:47.091 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:47.089+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:47.099 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:50:47.188 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:47.188+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:47.243 INFO:teuthology.orchestra.run.smithi029.stderr:no valid command found; 10 closest matches: 2024-09-16T20:50:47.243 INFO:teuthology.orchestra.run.smithi029.stderr:0 2024-09-16T20:50:47.243 INFO:teuthology.orchestra.run.smithi029.stderr:1 2024-09-16T20:50:47.243 INFO:teuthology.orchestra.run.smithi029.stderr:2 2024-09-16T20:50:47.243 INFO:teuthology.orchestra.run.smithi029.stderr:abort 2024-09-16T20:50:47.243 INFO:teuthology.orchestra.run.smithi029.stderr:assert 2024-09-16T20:50:47.243 INFO:teuthology.orchestra.run.smithi029.stderr:bench [] [] [] [] 2024-09-16T20:50:47.244 INFO:teuthology.orchestra.run.smithi029.stderr:bluefs debug_inject_read_zeros 2024-09-16T20:50:47.244 INFO:teuthology.orchestra.run.smithi029.stderr:bluefs files list 2024-09-16T20:50:47.244 INFO:teuthology.orchestra.run.smithi029.stderr:bluefs stats 2024-09-16T20:50:47.244 INFO:teuthology.orchestra.run.smithi029.stderr:bluestore allocator dump block 2024-09-16T20:50:47.244 INFO:teuthology.orchestra.run.smithi029.stderr:Error EINVAL: invalid command 2024-09-16T20:50:47.246 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T20:50:47.247 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-16T20:50:47.247 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 'trim stale osdmaps' 2024-09-16T20:50:47.259 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T20:50:47.270 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:47.270 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:50:47.270 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:50:47.270 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:50:47.270 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.270 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3714 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1480704~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:47.270 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:50:47.271 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 144.629568702, 2024-09-16T20:50:47.271 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050152552000000003, 2024-09-16T20:50:47.271 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:47.271 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:47.271 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:47.271 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:47.271 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:47.271 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:47.271 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3714 2024-09-16T20:50:47.271 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.271 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:47.271 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.271 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:47.271 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:50:47.271 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:47.272 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.272 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.272 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:47.272 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:50:47.272 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:47.272 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.272 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.272 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:47.272 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:50:47.272 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0499999999999999e-07 2024-09-16T20:50:47.272 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.272 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.272 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:47.272 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639007+0000", 2024-09-16T20:50:47.273 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5340000000000001e-06 2024-09-16T20:50:47.273 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.273 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.273 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:47.273 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639013+0000", 2024-09-16T20:50:47.273 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3260000000000003e-06 2024-09-16T20:50:47.273 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.273 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.273 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:47.273 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639018+0000", 2024-09-16T20:50:47.273 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2850000000000002e-06 2024-09-16T20:50:47.273 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.273 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.273 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:47.274 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689026+0000", 2024-09-16T20:50:47.274 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.05000798 2024-09-16T20:50:47.274 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.274 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.274 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:47.274 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689077+0000", 2024-09-16T20:50:47.274 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1063999999999998e-05 2024-09-16T20:50:47.274 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.274 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.274 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:47.274 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689158+0000", 2024-09-16T20:50:47.274 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1057999999999995e-05 2024-09-16T20:50:47.274 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:47.274 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:47.275 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:47.275 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.275 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.275 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3716 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1488896~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:47.275 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:50:47.275 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 144.629550892, 2024-09-16T20:50:47.275 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050280142, 2024-09-16T20:50:47.275 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:47.275 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:47.275 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:47.275 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:47.275 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:47.275 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:47.275 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3716 2024-09-16T20:50:47.276 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.276 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:47.276 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.276 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:47.276 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:50:47.276 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:47.276 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.276 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.276 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:47.276 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:50:47.276 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:47.276 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.276 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.276 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:47.276 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639024+0000", 2024-09-16T20:50:47.277 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9999999999999997e-07 2024-09-16T20:50:47.277 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.277 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.277 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:47.277 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639026+0000", 2024-09-16T20:50:47.277 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.781e-06 2024-09-16T20:50:47.277 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.277 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.277 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:47.277 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639032+0000", 2024-09-16T20:50:47.277 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7780000000000004e-06 2024-09-16T20:50:47.277 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.277 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.277 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:47.277 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639036+0000", 2024-09-16T20:50:47.278 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6909999999999996e-06 2024-09-16T20:50:47.278 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.278 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.278 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:47.278 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689191+0000", 2024-09-16T20:50:47.278 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050155284000000001 2024-09-16T20:50:47.278 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.278 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.278 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:47.278 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689242+0000", 2024-09-16T20:50:47.278 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0662999999999999e-05 2024-09-16T20:50:47.278 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.278 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.278 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:47.278 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689304+0000", 2024-09-16T20:50:47.279 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1345000000000003e-05 2024-09-16T20:50:47.279 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:47.279 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:47.279 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:47.279 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.279 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.279 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3719 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1497088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:47.279 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:50:47.279 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 144.62952361399999, 2024-09-16T20:50:47.279 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050399715999999997, 2024-09-16T20:50:47.279 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:47.279 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:47.279 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:47.279 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:47.279 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:47.280 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:47.280 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3719 2024-09-16T20:50:47.280 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.280 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:47.280 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.280 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:47.280 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:50:47.280 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:47.280 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.280 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.280 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:47.280 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:50:47.280 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:47.281 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.281 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.281 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:47.281 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:50:47.281 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9700000000000002e-07 2024-09-16T20:50:47.281 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.281 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.281 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:47.281 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639053+0000", 2024-09-16T20:50:47.281 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2900000000000001e-06 2024-09-16T20:50:47.281 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.281 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.281 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:47.281 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639060+0000", 2024-09-16T20:50:47.281 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7750000000000004e-06 2024-09-16T20:50:47.282 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.282 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.282 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:47.282 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639066+0000", 2024-09-16T20:50:47.282 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0639999999999999e-06 2024-09-16T20:50:47.282 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.282 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.282 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:47.282 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689337+0000", 2024-09-16T20:50:47.282 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050270784999999998 2024-09-16T20:50:47.282 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.282 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.282 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:47.282 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689392+0000", 2024-09-16T20:50:47.282 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4920999999999998e-05 2024-09-16T20:50:47.282 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.283 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.283 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:47.283 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689450+0000", 2024-09-16T20:50:47.283 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8483999999999998e-05 2024-09-16T20:50:47.283 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:47.283 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:47.283 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:47.283 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.283 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.283 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3721 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1505280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:47.283 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:50:47.283 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 144.629500482, 2024-09-16T20:50:47.283 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050513608000000002, 2024-09-16T20:50:47.283 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:47.283 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:47.284 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:47.284 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:47.284 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:47.284 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:47.284 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3721 2024-09-16T20:50:47.284 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.284 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:47.284 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.284 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:47.284 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:50:47.284 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:47.284 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.284 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.284 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:47.284 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:50:47.285 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:47.285 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.285 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.285 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:47.285 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:50:47.285 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7e-07 2024-09-16T20:50:47.285 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.285 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.285 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:47.285 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639076+0000", 2024-09-16T20:50:47.285 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8899999999999999e-06 2024-09-16T20:50:47.285 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.285 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.285 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:47.285 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639083+0000", 2024-09-16T20:50:47.286 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1210000000000001e-06 2024-09-16T20:50:47.286 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.286 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.286 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:47.286 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639089+0000", 2024-09-16T20:50:47.286 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.643e-06 2024-09-16T20:50:47.286 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.286 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.286 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:47.286 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689474+0000", 2024-09-16T20:50:47.286 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050385679000000003 2024-09-16T20:50:47.286 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.286 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.286 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:47.286 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689533+0000", 2024-09-16T20:50:47.286 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8303000000000001e-05 2024-09-16T20:50:47.287 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.287 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.287 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:47.287 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689587+0000", 2024-09-16T20:50:47.287 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4601999999999997e-05 2024-09-16T20:50:47.287 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:47.287 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:47.287 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:47.287 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.287 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.287 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3723 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1513472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:47.287 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:50:47.287 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 144.62946967799999, 2024-09-16T20:50:47.287 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050699691999999998, 2024-09-16T20:50:47.287 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:47.287 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:47.288 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:47.288 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:47.288 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:47.288 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:47.288 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3723 2024-09-16T20:50:47.288 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.288 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:47.288 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.288 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:47.288 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:50:47.288 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:47.288 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.288 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.288 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:47.289 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:50:47.289 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:47.289 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.289 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.289 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:47.289 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:50:47.289 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3399999999999999e-07 2024-09-16T20:50:47.289 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.289 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.289 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:47.289 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639107+0000", 2024-09-16T20:50:47.289 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3860000000000001e-06 2024-09-16T20:50:47.289 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.289 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.289 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:47.290 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639114+0000", 2024-09-16T20:50:47.290 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9800000000000001e-06 2024-09-16T20:50:47.290 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.290 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.290 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:47.290 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639129+0000", 2024-09-16T20:50:47.290 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4887e-05 2024-09-16T20:50:47.290 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.290 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.290 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:47.290 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689625+0000", 2024-09-16T20:50:47.290 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050495848000000003 2024-09-16T20:50:47.290 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.291 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.291 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:47.291 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689720+0000", 2024-09-16T20:50:47.291 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4821000000000006e-05 2024-09-16T20:50:47.291 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.291 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.291 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:47.291 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689804+0000", 2024-09-16T20:50:47.291 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4235999999999998e-05 2024-09-16T20:50:47.291 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:47.291 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:47.291 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:47.291 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.291 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.291 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3725 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1521664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:47.292 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:50:47.292 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 144.62943608200001, 2024-09-16T20:50:47.292 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050797660000000001, 2024-09-16T20:50:47.292 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:47.292 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:47.292 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:47.292 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:47.292 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:47.292 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:47.292 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3725 2024-09-16T20:50:47.292 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.292 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:47.292 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.292 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:47.292 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:50:47.292 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:47.293 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.293 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.293 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:47.293 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:50:47.293 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:47.293 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.293 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.293 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:47.293 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639139+0000", 2024-09-16T20:50:47.293 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8999999999999997e-07 2024-09-16T20:50:47.293 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.293 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.293 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:47.293 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639141+0000", 2024-09-16T20:50:47.293 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.182e-06 2024-09-16T20:50:47.294 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.294 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.294 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:47.294 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639154+0000", 2024-09-16T20:50:47.294 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3626e-05 2024-09-16T20:50:47.294 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.294 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.294 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:47.294 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639160+0000", 2024-09-16T20:50:47.294 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6810000000000003e-06 2024-09-16T20:50:47.294 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.294 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.294 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:47.294 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689831+0000", 2024-09-16T20:50:47.294 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050670907000000001 2024-09-16T20:50:47.295 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.295 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.295 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:47.295 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689882+0000", 2024-09-16T20:50:47.295 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0445999999999998e-05 2024-09-16T20:50:47.295 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.295 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.295 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:47.295 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689936+0000", 2024-09-16T20:50:47.295 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4327999999999999e-05 2024-09-16T20:50:47.295 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:47.295 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:47.295 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:47.295 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.295 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.296 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3727 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1529856~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:47.296 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:50:47.296 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 144.62940050500001, 2024-09-16T20:50:47.296 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051016922999999999, 2024-09-16T20:50:47.296 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:47.296 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:47.296 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:47.296 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:47.296 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:47.296 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:47.296 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3727 2024-09-16T20:50:47.296 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.296 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:47.296 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.296 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:47.297 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:50:47.297 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:47.297 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.297 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.297 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:47.297 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:50:47.297 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:47.297 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.297 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.297 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:47.297 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:50:47.297 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9999999999999998e-07 2024-09-16T20:50:47.297 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.297 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.298 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:47.298 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639183+0000", 2024-09-16T20:50:47.298 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2399999999999996e-06 2024-09-16T20:50:47.298 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.298 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.298 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:47.298 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639195+0000", 2024-09-16T20:50:47.298 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1113e-05 2024-09-16T20:50:47.298 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.298 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.298 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:47.298 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639206+0000", 2024-09-16T20:50:47.298 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1503e-05 2024-09-16T20:50:47.298 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.299 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.299 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:47.299 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689969+0000", 2024-09-16T20:50:47.299 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050762676 2024-09-16T20:50:47.299 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.299 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.299 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:47.299 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690071+0000", 2024-09-16T20:50:47.299 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102163 2024-09-16T20:50:47.299 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.299 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.299 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:47.299 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690191+0000", 2024-09-16T20:50:47.300 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000119728 2024-09-16T20:50:47.300 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:47.300 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:47.300 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:47.300 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.300 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.300 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3730 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1538048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:47.300 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:50:47.300 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 144.62935894200001, 2024-09-16T20:50:47.300 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051131467999999999, 2024-09-16T20:50:47.300 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:47.300 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:47.300 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:47.301 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:47.301 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:47.301 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:47.301 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3730 2024-09-16T20:50:47.301 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.301 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:47.301 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.301 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:47.301 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:50:47.301 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:47.301 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.301 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.301 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:47.301 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:50:47.302 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:47.302 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.302 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.302 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:47.302 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639216+0000", 2024-09-16T20:50:47.302 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.39e-07 2024-09-16T20:50:47.302 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.302 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.302 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:47.302 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639218+0000", 2024-09-16T20:50:47.302 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.537e-06 2024-09-16T20:50:47.302 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.302 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.302 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:47.302 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639229+0000", 2024-09-16T20:50:47.303 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1122e-05 2024-09-16T20:50:47.303 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.303 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.303 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:47.303 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639237+0000", 2024-09-16T20:50:47.303 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8509999999999999e-06 2024-09-16T20:50:47.303 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.303 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.303 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:47.303 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690227+0000", 2024-09-16T20:50:47.303 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050990147 2024-09-16T20:50:47.303 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.303 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.303 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:47.304 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690278+0000", 2024-09-16T20:50:47.304 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0223999999999999e-05 2024-09-16T20:50:47.304 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.304 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.304 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:47.304 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690347+0000", 2024-09-16T20:50:47.304 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9147999999999996e-05 2024-09-16T20:50:47.304 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:47.304 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:47.304 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:47.304 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.304 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.304 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3731 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1546240~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:47.305 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:50:47.305 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 144.62932823400001, 2024-09-16T20:50:47.305 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051276744999999999, 2024-09-16T20:50:47.305 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:47.305 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:47.305 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:47.305 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:47.305 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:47.305 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:47.305 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3731 2024-09-16T20:50:47.305 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.305 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:47.305 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.305 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:47.306 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:50:47.306 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:47.306 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.306 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.306 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:47.306 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:50:47.306 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:47.306 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.306 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.306 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:47.306 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:50:47.306 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9900000000000001e-07 2024-09-16T20:50:47.306 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.306 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.306 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:47.307 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639249+0000", 2024-09-16T20:50:47.307 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3630000000000001e-06 2024-09-16T20:50:47.307 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.307 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.307 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:47.307 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639255+0000", 2024-09-16T20:50:47.307 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4960000000000001e-06 2024-09-16T20:50:47.307 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.307 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.307 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:47.307 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639261+0000", 2024-09-16T20:50:47.307 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2839999999999997e-06 2024-09-16T20:50:47.307 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.307 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.308 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:47.308 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690375+0000", 2024-09-16T20:50:47.308 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051114684000000001 2024-09-16T20:50:47.308 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.308 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.308 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:47.308 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690425+0000", 2024-09-16T20:50:47.308 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9379999999999998e-05 2024-09-16T20:50:47.308 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.308 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.308 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:47.308 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690523+0000", 2024-09-16T20:50:47.308 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8139000000000003e-05 2024-09-16T20:50:47.308 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:47.308 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:47.309 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:47.309 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.309 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.309 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3732 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1554432~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:47.309 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:50:47.309 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 144.62928245200001, 2024-09-16T20:50:47.309 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051380096, 2024-09-16T20:50:47.309 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:47.309 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:47.309 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:47.309 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:47.309 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:47.309 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:47.309 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3732 2024-09-16T20:50:47.310 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.310 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:47.310 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.310 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:47.310 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:50:47.310 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:47.310 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.310 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.310 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:47.310 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:50:47.310 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:47.310 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.310 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.310 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:47.311 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:50:47.311 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6100000000000001e-07 2024-09-16T20:50:47.311 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.311 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.311 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:47.311 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639295+0000", 2024-09-16T20:50:47.311 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.447e-06 2024-09-16T20:50:47.311 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.311 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.311 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:47.311 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639302+0000", 2024-09-16T20:50:47.311 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0570000000000004e-06 2024-09-16T20:50:47.311 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.311 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.312 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:47.312 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639307+0000", 2024-09-16T20:50:47.312 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1429999999999997e-06 2024-09-16T20:50:47.312 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.312 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.312 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:47.312 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690555+0000", 2024-09-16T20:50:47.312 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051247963000000001 2024-09-16T20:50:47.312 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.312 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.312 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:47.312 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690602+0000", 2024-09-16T20:50:47.312 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7602000000000003e-05 2024-09-16T20:50:47.312 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.313 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.313 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:47.313 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690672+0000", 2024-09-16T20:50:47.313 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9423000000000003e-05 2024-09-16T20:50:47.313 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:47.313 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:47.313 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:47.313 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.313 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.313 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3734 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1562624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:47.313 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:50:47.313 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 144.629260802, 2024-09-16T20:50:47.313 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051490610999999999, 2024-09-16T20:50:47.314 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:47.314 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:47.314 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:47.314 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:47.314 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:47.314 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:47.314 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3734 2024-09-16T20:50:47.314 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.314 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:47.314 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.314 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:47.314 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:50:47.314 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:47.314 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.315 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.315 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:47.315 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:50:47.315 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:47.315 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.315 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.315 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:47.315 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639314+0000", 2024-09-16T20:50:47.315 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4400000000000001e-07 2024-09-16T20:50:47.315 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.315 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.315 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:47.315 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639316+0000", 2024-09-16T20:50:47.315 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1459999999999999e-06 2024-09-16T20:50:47.316 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.316 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.316 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:47.316 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639322+0000", 2024-09-16T20:50:47.316 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7610000000000004e-06 2024-09-16T20:50:47.316 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.316 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.316 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:47.316 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639327+0000", 2024-09-16T20:50:47.316 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.186e-06 2024-09-16T20:50:47.316 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.316 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.316 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:47.316 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690698+0000", 2024-09-16T20:50:47.317 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051371223000000001 2024-09-16T20:50:47.317 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.317 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.317 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:47.317 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690748+0000", 2024-09-16T20:50:47.317 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0192999999999999e-05 2024-09-16T20:50:47.317 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.317 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.317 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:47.317 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690804+0000", 2024-09-16T20:50:47.317 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5658000000000001e-05 2024-09-16T20:50:47.317 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:47.317 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:47.317 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:47.317 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.318 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.318 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3736 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1570816~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:47.318 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:50:47.318 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 144.62924154300001, 2024-09-16T20:50:47.318 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051596528000000003, 2024-09-16T20:50:47.318 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:47.318 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:47.318 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:47.318 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:47.318 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:47.318 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:47.318 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3736 2024-09-16T20:50:47.318 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.318 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:47.319 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.319 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:47.319 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:50:47.319 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:47.319 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.319 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.319 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:47.319 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:50:47.319 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:47.319 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.319 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.319 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:47.319 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:50:47.319 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9999999999999999e-07 2024-09-16T20:50:47.320 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.320 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.320 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:47.320 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639335+0000", 2024-09-16T20:50:47.320 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.635e-06 2024-09-16T20:50:47.320 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.320 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.320 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:47.320 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639340+0000", 2024-09-16T20:50:47.320 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5840000000000002e-06 2024-09-16T20:50:47.320 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.320 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.320 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:47.320 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639346+0000", 2024-09-16T20:50:47.321 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1190000000000002e-06 2024-09-16T20:50:47.321 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.321 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.321 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:47.321 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690817+0000", 2024-09-16T20:50:47.321 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051470721999999997 2024-09-16T20:50:47.321 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.321 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.321 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:47.321 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690875+0000", 2024-09-16T20:50:47.321 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7441000000000001e-05 2024-09-16T20:50:47.321 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.321 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.321 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:47.322 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690929+0000", 2024-09-16T20:50:47.322 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4727e-05 2024-09-16T20:50:47.322 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:47.322 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:47.322 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:47.322 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.322 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.322 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5434 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:47.322 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:50:47.322 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 144.40397997700001, 2024-09-16T20:50:47.322 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099755831000000003, 2024-09-16T20:50:47.322 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:47.322 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:47.322 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:47.323 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:47.323 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:47.323 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:47.323 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5434 2024-09-16T20:50:47.323 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.323 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:47.323 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.323 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:47.323 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:50:47.323 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:47.323 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.323 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.323 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:47.323 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:50:47.323 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:47.323 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.324 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.324 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:47.324 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864595+0000", 2024-09-16T20:50:47.324 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1099999999999995e-07 2024-09-16T20:50:47.324 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.324 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.324 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:47.324 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864597+0000", 2024-09-16T20:50:47.324 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3989999999999998e-06 2024-09-16T20:50:47.324 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.324 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.324 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:47.324 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864603+0000", 2024-09-16T20:50:47.324 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9309999999999996e-06 2024-09-16T20:50:47.325 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.325 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.325 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:47.325 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864607+0000", 2024-09-16T20:50:47.325 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6200000000000001e-06 2024-09-16T20:50:47.325 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.325 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.325 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:47.325 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964164+0000", 2024-09-16T20:50:47.325 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099557162000000005 2024-09-16T20:50:47.325 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.325 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.325 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:47.325 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964244+0000", 2024-09-16T20:50:47.326 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9597000000000004e-05 2024-09-16T20:50:47.326 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.326 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.326 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:47.326 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964350+0000", 2024-09-16T20:50:47.326 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010641099999999999 2024-09-16T20:50:47.326 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:47.326 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:47.326 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:47.326 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.326 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.326 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5435 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 20480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:47.326 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:50:47.326 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 144.40395015499999, 2024-09-16T20:50:47.326 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100897102, 2024-09-16T20:50:47.326 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:47.327 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:47.327 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:47.327 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:47.327 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:47.327 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:47.327 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5435 2024-09-16T20:50:47.327 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.327 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:47.327 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.327 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:47.327 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:50:47.327 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:47.327 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.327 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.327 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:47.328 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:50:47.328 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:47.328 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.328 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.328 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:47.328 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:50:47.328 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8000000000000001e-07 2024-09-16T20:50:47.328 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.328 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.328 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:47.328 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864626+0000", 2024-09-16T20:50:47.328 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.745e-06 2024-09-16T20:50:47.328 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.328 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.328 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:47.329 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864633+0000", 2024-09-16T20:50:47.329 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3339999999999998e-06 2024-09-16T20:50:47.329 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.329 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.329 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:47.329 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864636+0000", 2024-09-16T20:50:47.329 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1200000000000002e-06 2024-09-16T20:50:47.329 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.329 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.329 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:47.329 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965299+0000", 2024-09-16T20:50:47.329 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10066352000000001 2024-09-16T20:50:47.329 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.329 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.329 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:47.329 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965390+0000", 2024-09-16T20:50:47.330 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0894000000000001e-05 2024-09-16T20:50:47.330 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.330 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.330 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:47.330 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965521+0000", 2024-09-16T20:50:47.330 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013110899999999999 2024-09-16T20:50:47.330 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:47.330 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:47.330 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:47.330 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.330 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.330 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:152 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 327680~32768 out=32768b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:50:47.330 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:50:47.330 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 143.105453213, 2024-09-16T20:50:47.331 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11869405800000001, 2024-09-16T20:50:47.331 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:47.331 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:47.331 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:47.331 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:47.331 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:50:47.331 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:50:47.331 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 152 2024-09-16T20:50:47.331 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.331 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:47.331 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.331 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:47.331 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:50:47.331 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:47.332 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.332 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.332 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:47.332 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:50:47.332 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:47.332 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.332 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.332 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:47.332 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163122+0000", 2024-09-16T20:50:47.332 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6400000000000003e-07 2024-09-16T20:50:47.332 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.332 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.332 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:47.332 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163124+0000", 2024-09-16T20:50:47.333 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.565e-06 2024-09-16T20:50:47.333 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.333 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.333 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:47.333 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163136+0000", 2024-09-16T20:50:47.333 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1717e-05 2024-09-16T20:50:47.333 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.333 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.333 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:47.333 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163142+0000", 2024-09-16T20:50:47.333 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1750000000000002e-06 2024-09-16T20:50:47.333 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.333 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.334 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:47.334 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163606+0000", 2024-09-16T20:50:47.334 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00046418799999999998 2024-09-16T20:50:47.334 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.334 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.334 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:47.334 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163656+0000", 2024-09-16T20:50:47.334 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9852999999999999e-05 2024-09-16T20:50:47.334 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.334 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.334 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:47.334 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281815+0000", 2024-09-16T20:50:47.334 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11815909600000001 2024-09-16T20:50:47.335 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:47.335 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:47.335 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:47.335 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.335 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.335 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:153 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:50:47.335 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:50:47.335 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 143.10542025999999, 2024-09-16T20:50:47.335 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11903462400000001, 2024-09-16T20:50:47.335 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:47.335 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:47.335 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:47.335 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:47.335 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:50:47.336 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:50:47.336 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 153 2024-09-16T20:50:47.336 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.336 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:47.336 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.336 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:47.336 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:50:47.336 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:47.336 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.336 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.336 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:47.336 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:50:47.336 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:47.336 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.337 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.337 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:47.337 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163155+0000", 2024-09-16T20:50:47.337 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.37e-07 2024-09-16T20:50:47.337 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.337 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.337 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:47.337 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163157+0000", 2024-09-16T20:50:47.337 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6010000000000002e-06 2024-09-16T20:50:47.337 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.337 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.337 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:47.337 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163165+0000", 2024-09-16T20:50:47.338 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2619999999999998e-06 2024-09-16T20:50:47.338 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.338 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.338 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:47.338 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163171+0000", 2024-09-16T20:50:47.338 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.711e-06 2024-09-16T20:50:47.338 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.338 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.338 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:47.338 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281868+0000", 2024-09-16T20:50:47.338 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118696622 2024-09-16T20:50:47.338 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.338 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.338 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:47.339 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281969+0000", 2024-09-16T20:50:47.339 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101371 2024-09-16T20:50:47.339 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.339 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.339 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:47.339 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282189+0000", 2024-09-16T20:50:47.339 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00021952000000000001 2024-09-16T20:50:47.339 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:47.339 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:47.339 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:47.339 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.339 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.339 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:154 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 360448~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:50:47.339 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:50:47.339 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 143.10538848799999, 2024-09-16T20:50:47.340 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11932035100000001, 2024-09-16T20:50:47.340 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:47.340 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:47.340 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:47.340 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:47.340 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:50:47.340 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:50:47.340 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 154 2024-09-16T20:50:47.340 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.340 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:47.340 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.340 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:47.340 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:50:47.340 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:47.340 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.340 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.341 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:47.341 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:50:47.341 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:47.341 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.341 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.341 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:47.341 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:50:47.341 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:50:47.341 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.341 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.341 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:47.341 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163189+0000", 2024-09-16T20:50:47.341 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6840000000000001e-06 2024-09-16T20:50:47.341 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.341 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.342 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:47.342 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163196+0000", 2024-09-16T20:50:47.342 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7920000000000004e-06 2024-09-16T20:50:47.342 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.342 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.342 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:47.342 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163201+0000", 2024-09-16T20:50:47.342 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2809999999999999e-06 2024-09-16T20:50:47.342 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.342 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.342 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:47.342 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282219+0000", 2024-09-16T20:50:47.342 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.119018343 2024-09-16T20:50:47.342 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.343 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.343 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:47.343 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282305+0000", 2024-09-16T20:50:47.343 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5865000000000005e-05 2024-09-16T20:50:47.343 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.343 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.343 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:47.343 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282506+0000", 2024-09-16T20:50:47.343 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000200927 2024-09-16T20:50:47.343 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:47.343 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:47.343 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:47.343 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.343 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.344 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop(client.4366.0:3088 3.1e e38/23 3:79e5ec01:::rbd_header.110e891a9b2b:head v 38'306, mlcod=36'304)", 2024-09-16T20:50:47.344 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:50:47.344 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 143.10436962, 2024-09-16T20:50:47.344 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118102759, 2024-09-16T20:50:47.344 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:47.344 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:47.344 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:47.344 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:47.344 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.344 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:47.344 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:50:47.344 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:47.344 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.344 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.344 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:47.345 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:50:47.345 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:47.345 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.345 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.345 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:47.345 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:50:47.345 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.85e-07 2024-09-16T20:50:47.345 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.345 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.345 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:47.345 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164212+0000", 2024-09-16T20:50:47.345 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3999999999999997e-06 2024-09-16T20:50:47.345 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.345 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.346 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:47.346 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281553+0000", 2024-09-16T20:50:47.346 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11734135499999999 2024-09-16T20:50:47.346 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.346 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.346 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:47.346 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281567+0000", 2024-09-16T20:50:47.346 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4188999999999999e-05 2024-09-16T20:50:47.346 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.346 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.346 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:47.346 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281653+0000", 2024-09-16T20:50:47.346 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5385999999999999e-05 2024-09-16T20:50:47.346 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.346 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.347 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:47.347 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281679+0000", 2024-09-16T20:50:47.347 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6689000000000001e-05 2024-09-16T20:50:47.347 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.347 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.347 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:47.347 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282277+0000", 2024-09-16T20:50:47.347 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00059798000000000002 2024-09-16T20:50:47.347 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.347 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.347 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:47.347 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282307+0000", 2024-09-16T20:50:47.347 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9975e-05 2024-09-16T20:50:47.347 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:47.348 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:47.348 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:47.348 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.348 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.348 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5255.0:44 3.16 3:6f5476df:::rbd_header.110e25db1f60:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:50:47.348 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:50:47.348 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 70.948066565000005, 2024-09-16T20:50:47.348 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.121007509, 2024-09-16T20:50:47.348 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:47.348 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:47.348 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:47.348 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:47.348 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5255", 2024-09-16T20:50:47.348 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:50:47.348 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 44 2024-09-16T20:50:47.349 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.349 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:47.349 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.349 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:47.349 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:50:47.349 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:47.349 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.349 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.349 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:47.349 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:50:47.349 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:47.349 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.349 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.349 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:47.349 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:50:47.350 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0500000000000003e-07 2024-09-16T20:50:47.350 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.350 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.350 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:47.350 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320512+0000", 2024-09-16T20:50:47.350 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2540000000000001e-06 2024-09-16T20:50:47.350 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.350 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.350 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:47.350 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320518+0000", 2024-09-16T20:50:47.350 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1689999999999997e-06 2024-09-16T20:50:47.350 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.350 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.350 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:47.350 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320521+0000", 2024-09-16T20:50:47.350 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5200000000000002e-06 2024-09-16T20:50:47.351 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.351 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.351 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:47.351 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320548+0000", 2024-09-16T20:50:47.351 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6662999999999999e-05 2024-09-16T20:50:47.351 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.351 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.351 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:47.351 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320587+0000", 2024-09-16T20:50:47.351 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8866999999999999e-05 2024-09-16T20:50:47.351 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.351 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.351 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:47.351 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.441515+0000", 2024-09-16T20:50:47.352 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.120928331 2024-09-16T20:50:47.352 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:47.352 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:47.352 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:47.352 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.352 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.352 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:20329 3.5 3:a4ea64b7:::rbd_header.110ede5ef857:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:47.352 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:01.061694+0000", 2024-09-16T20:50:47.352 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 46.206879997000001, 2024-09-16T20:50:47.352 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.071234077000000007, 2024-09-16T20:50:47.352 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:47.352 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:47.352 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:47.352 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:47.352 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:50:47.353 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:47.353 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 20329 2024-09-16T20:50:47.353 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.353 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:47.353 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.353 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:47.353 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.061694+0000", 2024-09-16T20:50:47.353 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:47.353 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.353 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.353 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:47.353 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.061694+0000", 2024-09-16T20:50:47.353 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:47.353 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.353 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.353 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:47.354 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089232+0000", 2024-09-16T20:50:47.354 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.027537275 2024-09-16T20:50:47.354 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.354 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.354 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:47.354 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089237+0000", 2024-09-16T20:50:47.354 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4709999999999998e-06 2024-09-16T20:50:47.354 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.354 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.354 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:47.354 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089246+0000", 2024-09-16T20:50:47.354 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8230000000000001e-06 2024-09-16T20:50:47.354 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.355 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.355 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:47.355 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089252+0000", 2024-09-16T20:50:47.355 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3060000000000003e-06 2024-09-16T20:50:47.355 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.355 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.355 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:47.355 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.132756+0000", 2024-09-16T20:50:47.355 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.043504334999999998 2024-09-16T20:50:47.355 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.355 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.355 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:47.355 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.132829+0000", 2024-09-16T20:50:47.355 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2849999999999995e-05 2024-09-16T20:50:47.355 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:47.356 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:47.356 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:47.356 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.132928+0000", 2024-09-16T20:50:47.356 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.9017000000000004e-05 2024-09-16T20:50:47.356 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:47.356 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:47.356 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:47.356 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:47.356 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:47.356 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:47.358 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:50:47.359 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:47.357+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:47.381 INFO:teuthology.orchestra.run.smithi029.stderr:no valid command found; 10 closest matches: 2024-09-16T20:50:47.381 INFO:teuthology.orchestra.run.smithi029.stderr:0 2024-09-16T20:50:47.381 INFO:teuthology.orchestra.run.smithi029.stderr:1 2024-09-16T20:50:47.381 INFO:teuthology.orchestra.run.smithi029.stderr:2 2024-09-16T20:50:47.381 INFO:teuthology.orchestra.run.smithi029.stderr:abort 2024-09-16T20:50:47.381 INFO:teuthology.orchestra.run.smithi029.stderr:assert 2024-09-16T20:50:47.381 INFO:teuthology.orchestra.run.smithi029.stderr:bench [] [] [] [] 2024-09-16T20:50:47.381 INFO:teuthology.orchestra.run.smithi029.stderr:bluefs debug_inject_read_zeros 2024-09-16T20:50:47.381 INFO:teuthology.orchestra.run.smithi029.stderr:bluefs files list 2024-09-16T20:50:47.382 INFO:teuthology.orchestra.run.smithi029.stderr:bluefs stats 2024-09-16T20:50:47.382 INFO:teuthology.orchestra.run.smithi029.stderr:bluestore allocator dump block 2024-09-16T20:50:47.382 INFO:teuthology.orchestra.run.smithi029.stderr:Error EINVAL: invalid command 2024-09-16T20:50:47.384 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T20:50:47.384 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-16T20:50:47.384 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 'trim stale osdmaps' 2024-09-16T20:50:47.469 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:47.468+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:47.477 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:47.477 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:50:47.477 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:50:47.477 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:47.486 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:50:47.495 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:50:47.496 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:50:47.529 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:50:47.529 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:50:47.557 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:50:47.557 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:50:47.559 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:47.559+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:47.576 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:50:47.576 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:50:47.593 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:50:47.594 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:50:47.596 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:47.596 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:47.597 INFO:teuthology.orchestra.run.smithi029.stderr:no valid command found; 10 closest matches: 2024-09-16T20:50:47.597 INFO:teuthology.orchestra.run.smithi029.stderr:0 2024-09-16T20:50:47.597 INFO:teuthology.orchestra.run.smithi029.stderr:1 2024-09-16T20:50:47.597 INFO:teuthology.orchestra.run.smithi029.stderr:2 2024-09-16T20:50:47.597 INFO:teuthology.orchestra.run.smithi029.stderr:abort 2024-09-16T20:50:47.597 INFO:teuthology.orchestra.run.smithi029.stderr:assert 2024-09-16T20:50:47.598 INFO:teuthology.orchestra.run.smithi029.stderr:bench [] [] [] [] 2024-09-16T20:50:47.598 INFO:teuthology.orchestra.run.smithi029.stderr:bluefs debug_inject_read_zeros 2024-09-16T20:50:47.598 INFO:teuthology.orchestra.run.smithi029.stderr:bluefs files list 2024-09-16T20:50:47.598 INFO:teuthology.orchestra.run.smithi029.stderr:bluefs stats 2024-09-16T20:50:47.598 INFO:teuthology.orchestra.run.smithi029.stderr:bluestore allocator dump block 2024-09-16T20:50:47.598 INFO:teuthology.orchestra.run.smithi029.stderr:Error EINVAL: invalid command 2024-09-16T20:50:47.600 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T20:50:47.601 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-16T20:50:47.607 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:50:47.659 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:47.658+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:47.759 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:47.759 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:47.761 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:47.761+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:47.767 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:50:47.813 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:50:47.813 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:50:47.864 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:47.863+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:47.915 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:47.915 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:47.924 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:50:47.967 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:47.967+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:48.031 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:48.031 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:48.040 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:50:48.081 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:48.081+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:48.164 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:48.163+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:48.188 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:48.188 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:48.197 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:50:48.281 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:48.281+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:48.344 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:48.345 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:48.353 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:50:48.394 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:48.394+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:48.467 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:48.466+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:48.501 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:48.501 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:48.509 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:50:48.593 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:48.593+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:48.617 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T20:50:48.664 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:48.664 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:48.668 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:48.668+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:48.673 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:50:48.770 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:48.769+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:48.795 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:48.795 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:48.805 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:50:48.871 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:48.870+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:48.952 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:48.953 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:48.961 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:50:48.972 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:48.972+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:49.068 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:49.068 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:49.073 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:49.072+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:49.076 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:50:49.173 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:49.172+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:49.223 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:49.224 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:49.233 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:50:49.274 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:49.273+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:49.376 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:49.376+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:49.380 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:49.380 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:49.389 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:50:49.477 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:49.476+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:49.496 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:49.496 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:49.504 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:50:49.569 INFO:teuthology.orchestra.run.smithi029.stderr:noscrub is unset 2024-09-16T20:50:49.578 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:49.578+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:49.679 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:49.678+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:49.682 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:49.682 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:49.691 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:50:49.779 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:49.779+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:49.824 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T20:50:49.838 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:49.838 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:49.847 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:50:49.888 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:49.887+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:49.947 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:50:49.947 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:50:49.966 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:50:49.966 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:50:49.981 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:49.980+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:49.983 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:50:49.983 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:50:49.999 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:49.999 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:50:49.999 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:50:49.999 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:50:49.999 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:49.999 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3714 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1480704~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:49.999 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:50:49.999 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 147.358540046, 2024-09-16T20:50:50.000 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050152552000000003, 2024-09-16T20:50:50.000 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:50.000 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:50.000 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:50.000 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:50.000 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:50.000 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:50.000 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3714 2024-09-16T20:50:50.000 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.000 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:50.000 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.000 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:50.000 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:50:50.001 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:50.001 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.001 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.001 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:50.001 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:50:50.001 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:50.001 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.001 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.001 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:50.001 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:50:50.001 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0499999999999999e-07 2024-09-16T20:50:50.001 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.001 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.001 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:50.002 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639007+0000", 2024-09-16T20:50:50.002 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5340000000000001e-06 2024-09-16T20:50:50.002 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.002 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.002 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:50.002 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639013+0000", 2024-09-16T20:50:50.002 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3260000000000003e-06 2024-09-16T20:50:50.002 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.002 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.002 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:50.002 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639018+0000", 2024-09-16T20:50:50.002 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2850000000000002e-06 2024-09-16T20:50:50.002 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.002 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.003 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:50.003 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689026+0000", 2024-09-16T20:50:50.003 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.05000798 2024-09-16T20:50:50.003 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.003 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.003 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:50.003 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689077+0000", 2024-09-16T20:50:50.003 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1063999999999998e-05 2024-09-16T20:50:50.003 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.003 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.003 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:50.003 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689158+0000", 2024-09-16T20:50:50.003 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1057999999999995e-05 2024-09-16T20:50:50.003 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:50.004 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:50.004 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:50.004 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.004 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.004 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3716 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1488896~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:50.004 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:50:50.004 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 147.358522236, 2024-09-16T20:50:50.004 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050280142, 2024-09-16T20:50:50.004 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:50.004 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:50.004 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:50.004 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:50.004 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:50.005 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:50.005 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3716 2024-09-16T20:50:50.005 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.005 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:50.005 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.005 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:50.005 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:50:50.005 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:50.005 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.005 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.005 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:50.005 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:50:50.005 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:50.005 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.006 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.006 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:50.006 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639024+0000", 2024-09-16T20:50:50.006 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9999999999999997e-07 2024-09-16T20:50:50.006 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.006 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.006 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:50.006 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639026+0000", 2024-09-16T20:50:50.006 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.781e-06 2024-09-16T20:50:50.006 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.006 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.006 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:50.006 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639032+0000", 2024-09-16T20:50:50.006 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7780000000000004e-06 2024-09-16T20:50:50.007 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.007 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.007 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:50.007 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639036+0000", 2024-09-16T20:50:50.007 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6909999999999996e-06 2024-09-16T20:50:50.007 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.007 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.007 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:50.007 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689191+0000", 2024-09-16T20:50:50.007 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050155284000000001 2024-09-16T20:50:50.007 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.007 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.008 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:50.008 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689242+0000", 2024-09-16T20:50:50.008 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0662999999999999e-05 2024-09-16T20:50:50.008 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.008 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.008 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:50.008 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689304+0000", 2024-09-16T20:50:50.008 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1345000000000003e-05 2024-09-16T20:50:50.008 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:50.008 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:50.008 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:50.008 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.008 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.009 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3719 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1497088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:50.009 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:50:50.009 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 147.35849495799999, 2024-09-16T20:50:50.009 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050399715999999997, 2024-09-16T20:50:50.009 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:50.009 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:50.009 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:50.009 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:50.009 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:50.009 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:50.009 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3719 2024-09-16T20:50:50.009 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.010 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:50.010 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.010 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:50.010 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:50:50.010 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:50.010 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.010 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.010 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:50.010 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:50:50.010 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:50.010 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.010 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.010 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:50.010 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:50:50.011 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9700000000000002e-07 2024-09-16T20:50:50.011 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.011 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.011 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:50.011 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639053+0000", 2024-09-16T20:50:50.011 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2900000000000001e-06 2024-09-16T20:50:50.011 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.011 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.011 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:50.011 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639060+0000", 2024-09-16T20:50:50.011 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7750000000000004e-06 2024-09-16T20:50:50.011 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.011 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.011 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:50.011 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639066+0000", 2024-09-16T20:50:50.012 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0639999999999999e-06 2024-09-16T20:50:50.012 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.012 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.012 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:50.012 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689337+0000", 2024-09-16T20:50:50.012 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050270784999999998 2024-09-16T20:50:50.012 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.012 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.012 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:50.012 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689392+0000", 2024-09-16T20:50:50.012 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4920999999999998e-05 2024-09-16T20:50:50.012 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.012 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.013 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:50.013 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689450+0000", 2024-09-16T20:50:50.013 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8483999999999998e-05 2024-09-16T20:50:50.013 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:50.013 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:50.013 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:50.013 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.013 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.013 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3721 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1505280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:50.013 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:50:50.013 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 147.358471826, 2024-09-16T20:50:50.014 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050513608000000002, 2024-09-16T20:50:50.014 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:50.014 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:50.014 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:50.014 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:50.014 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:50.014 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:50.014 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3721 2024-09-16T20:50:50.014 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.014 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:50.014 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.014 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:50.014 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:50:50.014 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:50.014 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.015 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.015 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:50.015 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:50:50.015 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:50.015 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.015 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.015 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:50.015 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:50:50.015 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7e-07 2024-09-16T20:50:50.015 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.015 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.015 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:50.015 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639076+0000", 2024-09-16T20:50:50.015 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8899999999999999e-06 2024-09-16T20:50:50.016 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.016 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.016 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:50.016 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639083+0000", 2024-09-16T20:50:50.016 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1210000000000001e-06 2024-09-16T20:50:50.016 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.016 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.016 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:50.016 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639089+0000", 2024-09-16T20:50:50.016 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.643e-06 2024-09-16T20:50:50.016 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.016 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.017 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:50.017 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689474+0000", 2024-09-16T20:50:50.017 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050385679000000003 2024-09-16T20:50:50.017 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.017 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.017 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:50.017 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689533+0000", 2024-09-16T20:50:50.017 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8303000000000001e-05 2024-09-16T20:50:50.017 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.017 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.017 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:50.017 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689587+0000", 2024-09-16T20:50:50.018 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4601999999999997e-05 2024-09-16T20:50:50.018 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:50.018 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:50.018 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:50.018 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.018 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.018 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3723 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1513472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:50.018 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:50:50.018 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 147.35844102199999, 2024-09-16T20:50:50.018 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050699691999999998, 2024-09-16T20:50:50.018 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:50.018 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:50.019 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:50.019 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:50.019 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:50.019 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:50.019 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3723 2024-09-16T20:50:50.019 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.019 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:50.019 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.019 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:50.019 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:50:50.019 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:50.019 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.019 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.020 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:50.020 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:50:50.020 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:50.020 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.020 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.020 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:50.020 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:50:50.020 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3399999999999999e-07 2024-09-16T20:50:50.020 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.020 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.020 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:50.020 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639107+0000", 2024-09-16T20:50:50.020 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3860000000000001e-06 2024-09-16T20:50:50.021 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.021 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.021 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:50.021 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639114+0000", 2024-09-16T20:50:50.021 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9800000000000001e-06 2024-09-16T20:50:50.021 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.021 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.021 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:50.021 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639129+0000", 2024-09-16T20:50:50.021 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4887e-05 2024-09-16T20:50:50.021 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.022 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.022 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:50.022 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689625+0000", 2024-09-16T20:50:50.022 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050495848000000003 2024-09-16T20:50:50.022 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.022 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.022 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:50.022 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689720+0000", 2024-09-16T20:50:50.022 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4821000000000006e-05 2024-09-16T20:50:50.022 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.022 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.022 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:50.022 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689804+0000", 2024-09-16T20:50:50.023 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4235999999999998e-05 2024-09-16T20:50:50.023 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:50.023 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:50.023 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:50.023 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.023 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.023 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3725 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1521664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:50.023 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:50:50.023 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 147.35840742600001, 2024-09-16T20:50:50.023 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050797660000000001, 2024-09-16T20:50:50.023 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:50.023 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:50.023 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:50.024 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:50.024 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:50.024 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:50.024 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3725 2024-09-16T20:50:50.024 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.024 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:50.024 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.024 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:50.024 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:50:50.024 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:50.024 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.024 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.024 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:50.024 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:50:50.025 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:50.025 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.025 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.025 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:50.025 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639139+0000", 2024-09-16T20:50:50.025 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8999999999999997e-07 2024-09-16T20:50:50.025 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.025 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.025 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:50.025 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639141+0000", 2024-09-16T20:50:50.025 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.182e-06 2024-09-16T20:50:50.025 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.025 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.026 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:50.026 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639154+0000", 2024-09-16T20:50:50.026 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3626e-05 2024-09-16T20:50:50.026 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.026 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.026 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:50.026 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639160+0000", 2024-09-16T20:50:50.026 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6810000000000003e-06 2024-09-16T20:50:50.026 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.026 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.026 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:50.026 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689831+0000", 2024-09-16T20:50:50.026 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050670907000000001 2024-09-16T20:50:50.026 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.027 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.027 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:50.027 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689882+0000", 2024-09-16T20:50:50.027 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0445999999999998e-05 2024-09-16T20:50:50.027 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.027 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.027 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:50.027 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689936+0000", 2024-09-16T20:50:50.027 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4327999999999999e-05 2024-09-16T20:50:50.027 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:50.027 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:50.027 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:50.027 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.027 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.028 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3727 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1529856~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:50.028 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:50:50.028 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 147.35837184900001, 2024-09-16T20:50:50.028 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051016922999999999, 2024-09-16T20:50:50.028 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:50.028 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:50.028 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:50.028 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:50.028 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:50.028 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:50.028 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3727 2024-09-16T20:50:50.028 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.028 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:50.028 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.029 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:50.029 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:50:50.029 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:50.029 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.029 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.029 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:50.029 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:50:50.029 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:50.029 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.029 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.029 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:50.029 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:50:50.029 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9999999999999998e-07 2024-09-16T20:50:50.029 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.030 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.030 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:50.030 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639183+0000", 2024-09-16T20:50:50.030 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2399999999999996e-06 2024-09-16T20:50:50.030 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.030 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.030 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:50.030 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639195+0000", 2024-09-16T20:50:50.030 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1113e-05 2024-09-16T20:50:50.030 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.030 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.030 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:50.030 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639206+0000", 2024-09-16T20:50:50.030 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1503e-05 2024-09-16T20:50:50.030 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.031 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.031 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:50.031 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689969+0000", 2024-09-16T20:50:50.031 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050762676 2024-09-16T20:50:50.031 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.031 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.031 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:50.031 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690071+0000", 2024-09-16T20:50:50.031 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102163 2024-09-16T20:50:50.031 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.031 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.031 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:50.031 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690191+0000", 2024-09-16T20:50:50.031 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000119728 2024-09-16T20:50:50.031 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:50.032 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:50.032 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:50.032 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.032 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.032 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3730 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1538048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:50.032 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:50:50.032 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 147.35833028600001, 2024-09-16T20:50:50.032 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051131467999999999, 2024-09-16T20:50:50.032 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:50.032 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:50.032 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:50.032 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:50.032 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:50.032 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:50.033 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3730 2024-09-16T20:50:50.033 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.033 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:50.033 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.033 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:50.033 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:50:50.033 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:50.033 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.033 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.033 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:50.033 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:50:50.033 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:50.033 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.033 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.033 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:50.034 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639216+0000", 2024-09-16T20:50:50.034 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.39e-07 2024-09-16T20:50:50.034 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.034 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.034 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:50.034 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639218+0000", 2024-09-16T20:50:50.034 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.537e-06 2024-09-16T20:50:50.034 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.034 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.034 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:50.034 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639229+0000", 2024-09-16T20:50:50.034 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1122e-05 2024-09-16T20:50:50.034 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.034 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.035 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:50.035 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639237+0000", 2024-09-16T20:50:50.035 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8509999999999999e-06 2024-09-16T20:50:50.035 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.035 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.035 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:50.035 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690227+0000", 2024-09-16T20:50:50.035 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050990147 2024-09-16T20:50:50.035 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.035 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.035 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:50.035 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690278+0000", 2024-09-16T20:50:50.035 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0223999999999999e-05 2024-09-16T20:50:50.035 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.036 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.036 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:50.036 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690347+0000", 2024-09-16T20:50:50.036 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9147999999999996e-05 2024-09-16T20:50:50.036 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:50.036 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:50.036 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:50.036 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.036 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.036 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3731 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1546240~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:50.036 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:50:50.036 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 147.35829957799999, 2024-09-16T20:50:50.036 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051276744999999999, 2024-09-16T20:50:50.037 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:50.037 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:50.037 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:50.037 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:50.037 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:50.037 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:50.037 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3731 2024-09-16T20:50:50.037 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.037 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:50.037 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.037 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:50.037 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:50:50.037 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:50.037 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.037 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.038 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:50.038 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:50:50.038 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:50.038 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.038 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.038 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:50.038 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:50:50.038 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9900000000000001e-07 2024-09-16T20:50:50.038 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.038 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.038 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:50.038 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639249+0000", 2024-09-16T20:50:50.038 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3630000000000001e-06 2024-09-16T20:50:50.038 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.038 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.039 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:50.039 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639255+0000", 2024-09-16T20:50:50.039 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4960000000000001e-06 2024-09-16T20:50:50.039 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.039 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.039 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:50.039 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639261+0000", 2024-09-16T20:50:50.039 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2839999999999997e-06 2024-09-16T20:50:50.039 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.039 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.039 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:50.039 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690375+0000", 2024-09-16T20:50:50.039 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051114684000000001 2024-09-16T20:50:50.039 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.040 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.040 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:50.040 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690425+0000", 2024-09-16T20:50:50.040 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9379999999999998e-05 2024-09-16T20:50:50.040 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.040 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.040 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:50.040 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690523+0000", 2024-09-16T20:50:50.040 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8139000000000003e-05 2024-09-16T20:50:50.040 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:50.040 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:50.040 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:50.040 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.040 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.040 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3732 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1554432~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:50.041 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:50:50.041 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 147.35825379600001, 2024-09-16T20:50:50.041 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051380096, 2024-09-16T20:50:50.041 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:50.041 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:50.041 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:50.041 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:50.041 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:50.041 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:50.041 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3732 2024-09-16T20:50:50.041 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.041 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:50.041 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.041 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:50.042 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:50:50.042 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:50.042 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.042 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.042 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:50.042 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:50:50.042 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:50.042 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.042 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.042 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:50.042 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:50:50.042 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6100000000000001e-07 2024-09-16T20:50:50.042 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.042 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.042 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:50.043 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639295+0000", 2024-09-16T20:50:50.043 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.447e-06 2024-09-16T20:50:50.043 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.043 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.043 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:50.043 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639302+0000", 2024-09-16T20:50:50.043 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0570000000000004e-06 2024-09-16T20:50:50.043 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.043 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.043 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:50.043 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639307+0000", 2024-09-16T20:50:50.043 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1429999999999997e-06 2024-09-16T20:50:50.043 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.043 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.043 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:50.043 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690555+0000", 2024-09-16T20:50:50.044 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051247963000000001 2024-09-16T20:50:50.044 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.044 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.044 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:50.044 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690602+0000", 2024-09-16T20:50:50.044 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7602000000000003e-05 2024-09-16T20:50:50.044 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.044 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.044 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:50.044 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690672+0000", 2024-09-16T20:50:50.044 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9423000000000003e-05 2024-09-16T20:50:50.044 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:50.044 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:50.044 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:50.044 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.045 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.045 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3734 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1562624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:50.045 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:50:50.045 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 147.35823214600001, 2024-09-16T20:50:50.045 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051490610999999999, 2024-09-16T20:50:50.045 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:50.045 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:50.045 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:50.045 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:50.045 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:50.045 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:50.045 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3734 2024-09-16T20:50:50.045 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.045 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:50.046 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.046 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:50.046 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:50:50.046 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:50.046 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.046 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.046 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:50.046 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:50:50.046 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:50.046 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.046 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.046 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:50.046 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639314+0000", 2024-09-16T20:50:50.046 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4400000000000001e-07 2024-09-16T20:50:50.046 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.047 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.047 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:50.047 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639316+0000", 2024-09-16T20:50:50.047 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1459999999999999e-06 2024-09-16T20:50:50.047 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.047 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.047 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:50.047 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639322+0000", 2024-09-16T20:50:50.047 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7610000000000004e-06 2024-09-16T20:50:50.047 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.047 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.047 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:50.047 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639327+0000", 2024-09-16T20:50:50.047 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.186e-06 2024-09-16T20:50:50.047 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.047 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.048 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:50.048 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690698+0000", 2024-09-16T20:50:50.048 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051371223000000001 2024-09-16T20:50:50.048 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.048 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.048 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:50.048 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690748+0000", 2024-09-16T20:50:50.048 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0192999999999999e-05 2024-09-16T20:50:50.048 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.048 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.048 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:50.048 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690804+0000", 2024-09-16T20:50:50.048 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5658000000000001e-05 2024-09-16T20:50:50.049 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:50.049 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:50.049 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:50.049 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.049 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.049 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3736 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1570816~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:50.049 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:50:50.049 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 147.35821288700001, 2024-09-16T20:50:50.049 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051596528000000003, 2024-09-16T20:50:50.049 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:50.049 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:50.049 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:50.049 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:50.049 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:50.049 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:50.050 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3736 2024-09-16T20:50:50.050 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.050 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:50.050 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.050 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:50.050 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:50:50.050 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:50.050 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.050 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.050 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:50.050 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:50:50.050 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:50.050 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.050 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.050 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:50.051 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:50:50.051 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9999999999999999e-07 2024-09-16T20:50:50.051 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.051 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.051 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:50.051 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639335+0000", 2024-09-16T20:50:50.051 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.635e-06 2024-09-16T20:50:50.051 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.051 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.051 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:50.051 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639340+0000", 2024-09-16T20:50:50.051 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5840000000000002e-06 2024-09-16T20:50:50.051 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.051 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.051 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:50.051 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639346+0000", 2024-09-16T20:50:50.052 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1190000000000002e-06 2024-09-16T20:50:50.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.052 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.052 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:50.052 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690817+0000", 2024-09-16T20:50:50.052 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051470721999999997 2024-09-16T20:50:50.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.052 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.052 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:50.052 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690875+0000", 2024-09-16T20:50:50.052 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7441000000000001e-05 2024-09-16T20:50:50.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.052 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.052 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:50.052 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690929+0000", 2024-09-16T20:50:50.052 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4727e-05 2024-09-16T20:50:50.053 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:50.053 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:50.053 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:50.053 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.053 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.053 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5434 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:50.053 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:50:50.053 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 147.13295132100001, 2024-09-16T20:50:50.053 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099755831000000003, 2024-09-16T20:50:50.053 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:50.053 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:50.053 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:50.053 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:50.053 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:50.053 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:50.054 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5434 2024-09-16T20:50:50.054 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.054 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:50.054 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.054 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:50.054 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:50:50.054 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:50.054 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.054 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.054 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:50.054 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:50:50.054 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:50.054 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.054 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.054 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:50.054 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864595+0000", 2024-09-16T20:50:50.055 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1099999999999995e-07 2024-09-16T20:50:50.055 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.055 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.055 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:50.055 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864597+0000", 2024-09-16T20:50:50.055 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3989999999999998e-06 2024-09-16T20:50:50.055 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.055 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.055 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:50.055 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864603+0000", 2024-09-16T20:50:50.055 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9309999999999996e-06 2024-09-16T20:50:50.055 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.055 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.055 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:50.055 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864607+0000", 2024-09-16T20:50:50.056 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6200000000000001e-06 2024-09-16T20:50:50.056 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.056 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.056 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:50.056 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964164+0000", 2024-09-16T20:50:50.056 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099557162000000005 2024-09-16T20:50:50.056 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.056 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.056 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:50.056 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964244+0000", 2024-09-16T20:50:50.056 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9597000000000004e-05 2024-09-16T20:50:50.056 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.056 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.056 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:50.056 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964350+0000", 2024-09-16T20:50:50.057 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010641099999999999 2024-09-16T20:50:50.057 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:50.057 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:50.057 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:50.057 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.057 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.057 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5435 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 20480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:50.057 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:50:50.057 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 147.13292149899999, 2024-09-16T20:50:50.057 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100897102, 2024-09-16T20:50:50.057 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:50.057 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:50.057 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:50.057 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:50.058 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:50.058 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:50.058 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5435 2024-09-16T20:50:50.058 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.058 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:50.058 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.058 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:50.058 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:50:50.058 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:50.058 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.058 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.058 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:50.058 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:50:50.058 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:50.058 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.059 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.059 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:50.059 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:50:50.059 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8000000000000001e-07 2024-09-16T20:50:50.059 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.059 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.059 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:50.059 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864626+0000", 2024-09-16T20:50:50.059 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.745e-06 2024-09-16T20:50:50.059 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.059 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.059 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:50.059 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864633+0000", 2024-09-16T20:50:50.059 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3339999999999998e-06 2024-09-16T20:50:50.059 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.059 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.060 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:50.060 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864636+0000", 2024-09-16T20:50:50.060 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1200000000000002e-06 2024-09-16T20:50:50.060 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.060 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.060 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:50.060 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965299+0000", 2024-09-16T20:50:50.060 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10066352000000001 2024-09-16T20:50:50.060 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.060 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.060 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:50.060 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965390+0000", 2024-09-16T20:50:50.060 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0894000000000001e-05 2024-09-16T20:50:50.060 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.060 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.061 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:50.061 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965521+0000", 2024-09-16T20:50:50.061 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013110899999999999 2024-09-16T20:50:50.061 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:50.061 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:50.061 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:50.061 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.061 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.061 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:152 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 327680~32768 out=32768b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:50:50.061 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:50:50.061 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 145.83442455700001, 2024-09-16T20:50:50.061 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11869405800000001, 2024-09-16T20:50:50.061 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:50.061 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:50.061 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:50.062 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:50.062 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:50:50.062 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:50:50.062 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 152 2024-09-16T20:50:50.062 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.062 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:50.062 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.062 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:50.062 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:50:50.062 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:50.062 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.062 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.062 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:50.062 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:50:50.062 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:50.063 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.063 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.063 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:50.063 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163122+0000", 2024-09-16T20:50:50.063 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6400000000000003e-07 2024-09-16T20:50:50.063 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.063 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.063 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:50.063 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163124+0000", 2024-09-16T20:50:50.063 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.565e-06 2024-09-16T20:50:50.063 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.063 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.063 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:50.063 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163136+0000", 2024-09-16T20:50:50.064 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1717e-05 2024-09-16T20:50:50.064 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.064 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.064 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:50.064 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163142+0000", 2024-09-16T20:50:50.064 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1750000000000002e-06 2024-09-16T20:50:50.064 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.064 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.064 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:50.064 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163606+0000", 2024-09-16T20:50:50.064 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00046418799999999998 2024-09-16T20:50:50.064 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.064 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.065 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:50.065 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163656+0000", 2024-09-16T20:50:50.065 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9852999999999999e-05 2024-09-16T20:50:50.065 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.065 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.065 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:50.065 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281815+0000", 2024-09-16T20:50:50.065 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11815909600000001 2024-09-16T20:50:50.065 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:50.065 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:50.065 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:50.065 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.065 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.066 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:153 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:50:50.066 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:50:50.066 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 145.83439160399999, 2024-09-16T20:50:50.066 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11903462400000001, 2024-09-16T20:50:50.066 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:50.066 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:50.066 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:50.066 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:50.066 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:50:50.066 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:50:50.066 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 153 2024-09-16T20:50:50.066 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.066 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:50.066 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.067 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:50.067 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:50:50.067 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:50.067 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.067 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.067 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:50.067 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:50:50.067 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:50.067 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.067 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.067 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:50.067 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163155+0000", 2024-09-16T20:50:50.067 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.37e-07 2024-09-16T20:50:50.068 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.068 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.068 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:50.068 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163157+0000", 2024-09-16T20:50:50.068 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6010000000000002e-06 2024-09-16T20:50:50.068 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.068 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.068 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:50.068 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163165+0000", 2024-09-16T20:50:50.068 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2619999999999998e-06 2024-09-16T20:50:50.068 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.068 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.068 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:50.069 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163171+0000", 2024-09-16T20:50:50.069 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.711e-06 2024-09-16T20:50:50.069 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.069 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.069 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:50.069 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281868+0000", 2024-09-16T20:50:50.069 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118696622 2024-09-16T20:50:50.069 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.069 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.069 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:50.069 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281969+0000", 2024-09-16T20:50:50.069 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101371 2024-09-16T20:50:50.069 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.069 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.070 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:50.070 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282189+0000", 2024-09-16T20:50:50.070 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00021952000000000001 2024-09-16T20:50:50.070 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:50.070 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:50.070 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:50.070 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.070 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.070 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:154 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 360448~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:50:50.070 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:50:50.070 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 145.83435983199999, 2024-09-16T20:50:50.070 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11932035100000001, 2024-09-16T20:50:50.070 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:50.070 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:50.070 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:50.070 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:50.071 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:50:50.071 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:50:50.071 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 154 2024-09-16T20:50:50.071 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.071 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:50.071 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.071 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:50.071 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:50:50.071 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:50.071 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.071 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.071 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:50.071 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:50:50.071 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:50.071 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.072 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.072 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:50.072 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:50:50.072 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:50:50.072 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.072 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.072 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:50.072 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163189+0000", 2024-09-16T20:50:50.072 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6840000000000001e-06 2024-09-16T20:50:50.072 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.072 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.072 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:50.072 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163196+0000", 2024-09-16T20:50:50.072 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7920000000000004e-06 2024-09-16T20:50:50.072 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.073 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.073 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:50.073 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163201+0000", 2024-09-16T20:50:50.073 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2809999999999999e-06 2024-09-16T20:50:50.073 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.073 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.073 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:50.073 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282219+0000", 2024-09-16T20:50:50.073 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.119018343 2024-09-16T20:50:50.073 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.073 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.073 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:50.073 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282305+0000", 2024-09-16T20:50:50.073 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5865000000000005e-05 2024-09-16T20:50:50.074 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.074 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.074 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:50.074 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282506+0000", 2024-09-16T20:50:50.074 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000200927 2024-09-16T20:50:50.074 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:50.074 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:50.074 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:50.074 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.074 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.074 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop(client.4366.0:3088 3.1e e38/23 3:79e5ec01:::rbd_header.110e891a9b2b:head v 38'306, mlcod=36'304)", 2024-09-16T20:50:50.074 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:50:50.074 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 145.833340964, 2024-09-16T20:50:50.075 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118102759, 2024-09-16T20:50:50.075 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:50.075 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:50.075 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:50.075 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:50.075 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.075 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:50.075 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:50:50.075 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:50.075 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.075 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.075 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:50.075 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:50:50.075 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:50.076 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.076 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.076 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:50.076 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:50:50.076 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.85e-07 2024-09-16T20:50:50.076 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.076 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.076 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:50.076 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164212+0000", 2024-09-16T20:50:50.076 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3999999999999997e-06 2024-09-16T20:50:50.076 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.076 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.076 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:50.076 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281553+0000", 2024-09-16T20:50:50.076 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11734135499999999 2024-09-16T20:50:50.077 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.077 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.077 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:50.077 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281567+0000", 2024-09-16T20:50:50.077 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4188999999999999e-05 2024-09-16T20:50:50.077 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.077 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.077 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:50.077 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281653+0000", 2024-09-16T20:50:50.077 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5385999999999999e-05 2024-09-16T20:50:50.077 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.077 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.077 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:50.077 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281679+0000", 2024-09-16T20:50:50.077 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6689000000000001e-05 2024-09-16T20:50:50.077 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.078 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.078 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:50.078 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282277+0000", 2024-09-16T20:50:50.078 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00059798000000000002 2024-09-16T20:50:50.078 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.078 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.078 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:50.078 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282307+0000", 2024-09-16T20:50:50.078 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9975e-05 2024-09-16T20:50:50.078 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:50.078 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:50.078 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:50.078 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.079 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.079 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5255.0:44 3.16 3:6f5476df:::rbd_header.110e25db1f60:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:50:50.079 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:50:50.079 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 73.677037909000006, 2024-09-16T20:50:50.079 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.121007509, 2024-09-16T20:50:50.079 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:50.079 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:50.079 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:50.079 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:50.079 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5255", 2024-09-16T20:50:50.079 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:50:50.079 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 44 2024-09-16T20:50:50.080 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.080 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:50.080 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.080 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:50.080 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:50:50.080 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:50.080 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.080 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.080 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:50.080 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:50:50.080 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:50.080 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.080 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.081 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:50.081 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:50:50.081 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0500000000000003e-07 2024-09-16T20:50:50.081 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.081 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.081 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:50.081 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320512+0000", 2024-09-16T20:50:50.081 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2540000000000001e-06 2024-09-16T20:50:50.081 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.081 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.081 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:50.081 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320518+0000", 2024-09-16T20:50:50.081 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1689999999999997e-06 2024-09-16T20:50:50.081 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.081 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.081 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:50.082 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320521+0000", 2024-09-16T20:50:50.082 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5200000000000002e-06 2024-09-16T20:50:50.082 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.082 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.082 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:50.082 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320548+0000", 2024-09-16T20:50:50.082 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6662999999999999e-05 2024-09-16T20:50:50.082 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.082 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.082 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:50.082 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320587+0000", 2024-09-16T20:50:50.082 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8866999999999999e-05 2024-09-16T20:50:50.082 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.082 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.082 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:50.083 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.441515+0000", 2024-09-16T20:50:50.083 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.120928331 2024-09-16T20:50:50.083 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:50.083 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:50.083 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:50.083 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.083 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.083 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:20329 3.5 3:a4ea64b7:::rbd_header.110ede5ef857:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:50.083 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:01.061694+0000", 2024-09-16T20:50:50.083 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 48.935851341000003, 2024-09-16T20:50:50.083 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.071234077000000007, 2024-09-16T20:50:50.083 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:50.083 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:50.083 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:50.083 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:50.083 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:50:50.084 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:50.084 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 20329 2024-09-16T20:50:50.084 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.084 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:50.084 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.084 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:50.084 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.061694+0000", 2024-09-16T20:50:50.084 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:50.084 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.084 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.084 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:50.084 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.061694+0000", 2024-09-16T20:50:50.084 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:50.084 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.084 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.085 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:50.085 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089232+0000", 2024-09-16T20:50:50.085 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.027537275 2024-09-16T20:50:50.085 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.085 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.085 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:50.085 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089237+0000", 2024-09-16T20:50:50.085 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4709999999999998e-06 2024-09-16T20:50:50.085 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.085 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.085 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:50.085 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089246+0000", 2024-09-16T20:50:50.085 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8230000000000001e-06 2024-09-16T20:50:50.085 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.085 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.085 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:50.086 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089252+0000", 2024-09-16T20:50:50.086 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3060000000000003e-06 2024-09-16T20:50:50.086 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.086 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.086 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:50.086 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.132756+0000", 2024-09-16T20:50:50.086 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.043504334999999998 2024-09-16T20:50:50.086 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.086 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.086 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:50.086 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.132829+0000", 2024-09-16T20:50:50.086 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2849999999999995e-05 2024-09-16T20:50:50.086 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:50.086 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:50.086 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:50.086 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.132928+0000", 2024-09-16T20:50:50.087 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.9017000000000004e-05 2024-09-16T20:50:50.087 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:50.087 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:50.087 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:50.087 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:50.087 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:50.087 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:50.088 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:50:50.088 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:50:50.089 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:50:50.089 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:50:50.089 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:50:50.090 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:50:50.090 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:50:50.091 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:50.088+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:50.188 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:50.187+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:50.210 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:50.210 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:50:50.210 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:50:50.210 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:50.219 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:50:50.289 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:50.288+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:50.327 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:50.328 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:50:50.328 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:50:50.328 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:50:50.328 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:50.336 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:50:50.390 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:50.390+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:50.485 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:50.485 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:50:50.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:50:50.485 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:50:50.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.486 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:3088 3.1e 3:79e5ec01:::rbd_header.110e891a9b2b:head [watch watch cookie 140213437171488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:50:50.486 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:50:50.486 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 146.31969290399999, 2024-09-16T20:50:50.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.11895186200000001, 2024-09-16T20:50:50.486 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:50.486 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:50.486 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:50.486 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:50.486 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:50:50.486 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:50.486 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3088 2024-09-16T20:50:50.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.486 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:50.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:50.487 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:50:50.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:50.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:50.487 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:50:50.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:50.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:50.487 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163846+0000", 2024-09-16T20:50:50.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6799999999999999e-07 2024-09-16T20:50:50.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.488 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:50.488 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163850+0000", 2024-09-16T20:50:50.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.929e-06 2024-09-16T20:50:50.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.488 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:50.488 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163862+0000", 2024-09-16T20:50:50.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1972e-05 2024-09-16T20:50:50.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:50.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163867+0000", 2024-09-16T20:50:50.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.451e-06 2024-09-16T20:50:50.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:50.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163912+0000", 2024-09-16T20:50:50.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4456000000000003e-05 2024-09-16T20:50:50.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:50.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163962+0000", 2024-09-16T20:50:50.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0578000000000003e-05 2024-09-16T20:50:50.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-16T20:50:50.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164058+0000", 2024-09-16T20:50:50.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5934999999999995e-05 2024-09-16T20:50:50.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:50:50.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164570+0000", 2024-09-16T20:50:50.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00051241100000000005 2024-09-16T20:50:50.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:50.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282629+0000", 2024-09-16T20:50:50.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.118058986 2024-09-16T20:50:50.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:50:50.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282725+0000", 2024-09-16T20:50:50.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5063999999999996e-05 2024-09-16T20:50:50.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:50.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282797+0000", 2024-09-16T20:50:50.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2212000000000006e-05 2024-09-16T20:50:50.491 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:50.492 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:50.492 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:50.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.492 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:8039 3.1e 3.ac44ff9e (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:50:50.492 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:50:50.492 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 120.775865555, 2024-09-16T20:50:50.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0053310449999999, 2024-09-16T20:50:50.492 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:50.492 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:50.492 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:50.492 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:50.493 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:50:50.493 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:50.493 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8039 2024-09-16T20:50:50.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.493 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:50.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.493 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:50.493 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:50:50.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:50.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.493 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:50.493 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:50:50.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:50.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.494 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:50.494 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707674+0000", 2024-09-16T20:50:50.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6530000000000001e-06 2024-09-16T20:50:50.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.494 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:50.494 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707681+0000", 2024-09-16T20:50:50.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9639999999999997e-06 2024-09-16T20:50:50.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:50.495 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707700+0000", 2024-09-16T20:50:50.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8674000000000001e-05 2024-09-16T20:50:50.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:50.495 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707708+0000", 2024-09-16T20:50:50.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6810000000000004e-06 2024-09-16T20:50:50.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:50.495 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711293+0000", 2024-09-16T20:50:50.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035848189999999999 2024-09-16T20:50:50.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:50.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711301+0000", 2024-09-16T20:50:50.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4730000000000003e-06 2024-09-16T20:50:50.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:50.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.712990+0000", 2024-09-16T20:50:50.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.001689348 2024-09-16T20:50:50.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:50.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713003+0000", 2024-09-16T20:50:50.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3433e-05 2024-09-16T20:50:50.497 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:50.497 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:50.497 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:50.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.497 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:8040 3.1e 3.ac44ff9e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:50:50.497 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:50:50.497 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 120.77580516, 2024-09-16T20:50:50.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0053410780000001, 2024-09-16T20:50:50.497 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:50.497 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:50.497 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:50.497 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:50.497 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:50:50.498 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:50.498 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8040 2024-09-16T20:50:50.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.498 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:50.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.498 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:50.498 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:50:50.498 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:50.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.498 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:50.498 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:50:50.498 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:50.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.499 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.499 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:50.499 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:50:50.499 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8100000000000003e-07 2024-09-16T20:50:50.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.499 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.499 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:50.499 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707737+0000", 2024-09-16T20:50:50.499 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5020000000000001e-06 2024-09-16T20:50:50.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.499 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.499 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:50.499 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707745+0000", 2024-09-16T20:50:50.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2169999999999994e-06 2024-09-16T20:50:50.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.500 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:50.500 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707751+0000", 2024-09-16T20:50:50.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.263e-06 2024-09-16T20:50:50.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.500 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:50.500 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711336+0000", 2024-09-16T20:50:50.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035848360000000001 2024-09-16T20:50:50.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:50.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711340+0000", 2024-09-16T20:50:50.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1309999999999999e-06 2024-09-16T20:50:50.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:50.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713060+0000", 2024-09-16T20:50:50.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0017195059999999 2024-09-16T20:50:50.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:50.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713074+0000", 2024-09-16T20:50:50.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4042e-05 2024-09-16T20:50:50.502 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:50.502 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:50.502 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:50.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.502 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4927.0:4487 3.1e 3.ac44ff9e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:50:50.502 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:50:50.502 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 120.775284958, 2024-09-16T20:50:50.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0048840880000001, 2024-09-16T20:50:50.502 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:50.502 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:50.502 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:50.502 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:50.502 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4927", 2024-09-16T20:50:50.503 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:50.503 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4487 2024-09-16T20:50:50.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.503 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:50.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:50.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:50:50.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:50.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:50.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:50:50.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:50.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.504 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:50.504 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708254+0000", 2024-09-16T20:50:50.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1800000000000005e-07 2024-09-16T20:50:50.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.504 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:50.504 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708258+0000", 2024-09-16T20:50:50.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8689999999999997e-06 2024-09-16T20:50:50.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.504 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:50.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708267+0000", 2024-09-16T20:50:50.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9590000000000007e-06 2024-09-16T20:50:50.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:50.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708274+0000", 2024-09-16T20:50:50.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9970000000000001e-06 2024-09-16T20:50:50.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:50.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711366+0000", 2024-09-16T20:50:50.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030919530000000001 2024-09-16T20:50:50.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:50.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711372+0000", 2024-09-16T20:50:50.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9410000000000004e-06 2024-09-16T20:50:50.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:50.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713126+0000", 2024-09-16T20:50:50.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0017537249999999 2024-09-16T20:50:50.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:50.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713137+0000", 2024-09-16T20:50:50.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0825999999999999e-05 2024-09-16T20:50:50.507 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:50.507 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:50.507 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:50.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.507 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.507 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5089.0:11703 3.1b 3:dcd4684e:::rbd_data.110e747f0779.000000000000001b:head [set-alloc-hint object_size 262144 write_size 262144,write 139264~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:50:50.507 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:50:50.507 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 106.34554918400001, 2024-09-16T20:50:50.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.084231385000000006, 2024-09-16T20:50:50.507 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:50.507 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:50.507 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:50.507 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:50.507 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5089", 2024-09-16T20:50:50.508 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:50:50.508 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 11703 2024-09-16T20:50:50.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.508 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:50.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.508 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:50.508 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:50:50.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:50.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.508 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:50.508 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:50:50.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:50.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:50.509 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:50:50.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2600000000000001e-07 2024-09-16T20:50:50.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:50.509 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137998+0000", 2024-09-16T20:50:50.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9549999999999998e-06 2024-09-16T20:50:50.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:50.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138008+0000", 2024-09-16T20:50:50.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3789999999999995e-06 2024-09-16T20:50:50.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:50.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138013+0000", 2024-09-16T20:50:50.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2209999999999996e-06 2024-09-16T20:50:50.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:50.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138054+0000", 2024-09-16T20:50:50.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1022999999999997e-05 2024-09-16T20:50:50.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:50.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138124+0000", 2024-09-16T20:50:50.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0234999999999994e-05 2024-09-16T20:50:50.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:50.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138197+0000", 2024-09-16T20:50:50.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2451000000000001e-05 2024-09-16T20:50:50.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:50.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222010+0000", 2024-09-16T20:50:50.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.083813433000000007 2024-09-16T20:50:50.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.512 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.512 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:50:50.512 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222163+0000", 2024-09-16T20:50:50.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015274599999999999 2024-09-16T20:50:50.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.512 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.512 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:50:50.512 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222197+0000", 2024-09-16T20:50:50.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3683000000000003e-05 2024-09-16T20:50:50.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.512 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.512 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:50.513 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222220+0000", 2024-09-16T20:50:50.513 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3433000000000001e-05 2024-09-16T20:50:50.513 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:50.513 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:50.513 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:50.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.513 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.5089.0:11702 3.15 e73/64 3:acf33c8b:::rbd_data.110e747f0779.000000000000001a:head v 73'132, mlcod=73'130)", 2024-09-16T20:50:50.513 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:50:50.513 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 106.345281855, 2024-09-16T20:50:50.513 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.084079763000000002, 2024-09-16T20:50:50.513 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:50.513 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:50.513 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:50.514 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:50.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:50.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:50:50.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:50.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:50.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:50:50.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:50.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:50.515 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:50:50.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:50:50.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.515 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.515 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:50.515 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138263+0000", 2024-09-16T20:50:50.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1e-06 2024-09-16T20:50:50.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.515 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.515 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:50.515 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221589+0000", 2024-09-16T20:50:50.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.083326002999999996 2024-09-16T20:50:50.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.515 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.515 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:50.516 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221614+0000", 2024-09-16T20:50:50.516 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5593999999999999e-05 2024-09-16T20:50:50.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.516 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.516 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:50.516 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221714+0000", 2024-09-16T20:50:50.516 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9889000000000005e-05 2024-09-16T20:50:50.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.516 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.516 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:50.516 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221727+0000", 2024-09-16T20:50:50.516 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3125999999999999e-05 2024-09-16T20:50:50.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.517 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:50:50.517 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222294+0000", 2024-09-16T20:50:50.517 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00056712199999999998 2024-09-16T20:50:50.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.517 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:50.517 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222336+0000", 2024-09-16T20:50:50.517 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1470000000000001e-05 2024-09-16T20:50:50.517 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:50.517 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:50.517 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:50.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.518 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5138.0:9358 3.c 3:33996a43:::rbd_data.110e6372565d.0000000000000007:head [read 1343488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T20:50:50.518 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:50:50.518 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 82.631006444999997, 2024-09-16T20:50:50.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0167493569999999, 2024-09-16T20:50:50.518 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:50.518 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:50.518 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:50.518 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:50.518 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5138", 2024-09-16T20:50:50.518 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:50:50.518 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9358 2024-09-16T20:50:50.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.518 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:50.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.519 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:50.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:50:50.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:50.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.519 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:50.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:50:50.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:50.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.519 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:50.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852533+0000", 2024-09-16T20:50:50.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.646e-06 2024-09-16T20:50:50.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.520 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:50.520 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852539+0000", 2024-09-16T20:50:50.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3389999999999998e-06 2024-09-16T20:50:50.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.520 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:50.520 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852552+0000", 2024-09-16T20:50:50.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.271e-05 2024-09-16T20:50:50.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.520 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:50.521 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852563+0000", 2024-09-16T20:50:50.521 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1139e-05 2024-09-16T20:50:50.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.521 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.521 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:50.521 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852883+0000", 2024-09-16T20:50:50.521 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00032010600000000001 2024-09-16T20:50:50.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.521 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.521 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:50.521 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852890+0000", 2024-09-16T20:50:50.521 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7669999999999998e-06 2024-09-16T20:50:50.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.521 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.522 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:50.522 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869059+0000", 2024-09-16T20:50:50.522 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.016169353 2024-09-16T20:50:50.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.522 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:50.522 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869178+0000", 2024-09-16T20:50:50.522 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118193 2024-09-16T20:50:50.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.522 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:50.522 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869281+0000", 2024-09-16T20:50:50.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010310399999999999 2024-09-16T20:50:50.523 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:50.523 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:50.523 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:50.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.523 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5138.0:9382 3.c 3:33996a43:::rbd_data.110e6372565d.0000000000000007:head [read 0~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T20:50:50.523 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:50:50.523 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 82.630955568000005, 2024-09-16T20:50:50.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.017136034, 2024-09-16T20:50:50.523 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:50.523 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:50.523 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:50.523 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:50.524 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5138", 2024-09-16T20:50:50.524 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:50:50.524 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9382 2024-09-16T20:50:50.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.524 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:50.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.524 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:50.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:50:50.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:50.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.524 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:50.525 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:50:50.525 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:50.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.525 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.525 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:50.525 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852583+0000", 2024-09-16T20:50:50.525 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.51e-07 2024-09-16T20:50:50.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.525 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.525 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:50.525 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852586+0000", 2024-09-16T20:50:50.525 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0280000000000001e-06 2024-09-16T20:50:50.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.525 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.526 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:50.526 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852595+0000", 2024-09-16T20:50:50.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2799999999999992e-06 2024-09-16T20:50:50.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.526 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.526 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:50.526 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852601+0000", 2024-09-16T20:50:50.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3810000000000004e-06 2024-09-16T20:50:50.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.526 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.526 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:50.526 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852921+0000", 2024-09-16T20:50:50.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00031979699999999997 2024-09-16T20:50:50.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.527 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:50.527 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852930+0000", 2024-09-16T20:50:50.527 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6489999999999997e-06 2024-09-16T20:50:50.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.527 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:50.527 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869434+0000", 2024-09-16T20:50:50.527 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0165038829999999 2024-09-16T20:50:50.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.527 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:50.527 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869495+0000", 2024-09-16T20:50:50.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1509999999999999e-05 2024-09-16T20:50:50.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:50.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869718+0000", 2024-09-16T20:50:50.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000222955 2024-09-16T20:50:50.528 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:50.528 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:50.528 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:50.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.528 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9969 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 632832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:50.528 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642045+0000", 2024-09-16T20:50:50.529 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 59.841493110999998, 2024-09-16T20:50:50.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076903192999999995, 2024-09-16T20:50:50.529 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:50.529 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:50.529 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:50.529 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:50.529 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:50.529 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:50.529 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9969 2024-09-16T20:50:50.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.529 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:50.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.530 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:50.530 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642045+0000", 2024-09-16T20:50:50.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:50.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.530 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:50.530 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642045+0000", 2024-09-16T20:50:50.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:50.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.530 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:50.530 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642045+0000", 2024-09-16T20:50:50.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5100000000000001e-07 2024-09-16T20:50:50.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:50.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642047+0000", 2024-09-16T20:50:50.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8679999999999999e-06 2024-09-16T20:50:50.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:50.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642053+0000", 2024-09-16T20:50:50.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1720000000000004e-06 2024-09-16T20:50:50.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:50.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642060+0000", 2024-09-16T20:50:50.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0339999999999999e-06 2024-09-16T20:50:50.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.532 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:50.532 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.718562+0000", 2024-09-16T20:50:50.532 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076501969000000003 2024-09-16T20:50:50.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.532 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:50.532 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.718643+0000", 2024-09-16T20:50:50.532 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0539999999999998e-05 2024-09-16T20:50:50.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.532 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:50.533 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.718948+0000", 2024-09-16T20:50:50.533 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000305259 2024-09-16T20:50:50.533 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:50.533 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:50.533 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:50.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.533 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.533 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9970 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 637952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:50.533 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642067+0000", 2024-09-16T20:50:50.533 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 59.841471130999999, 2024-09-16T20:50:50.533 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077174928000000004, 2024-09-16T20:50:50.533 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:50.533 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:50.534 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:50.534 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:50.534 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:50.534 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:50.534 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9970 2024-09-16T20:50:50.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.534 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:50.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.534 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:50.534 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642067+0000", 2024-09-16T20:50:50.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:50.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.534 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:50.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642067+0000", 2024-09-16T20:50:50.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:50.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:50.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642067+0000", 2024-09-16T20:50:50.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2499999999999995e-07 2024-09-16T20:50:50.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:50.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642070+0000", 2024-09-16T20:50:50.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3199999999999998e-06 2024-09-16T20:50:50.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.536 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:50.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642076+0000", 2024-09-16T20:50:50.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.122e-06 2024-09-16T20:50:50.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.536 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:50.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642081+0000", 2024-09-16T20:50:50.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0869999999999997e-06 2024-09-16T20:50:50.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.536 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:50.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.718980+0000", 2024-09-16T20:50:50.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076899482000000005 2024-09-16T20:50:50.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:50.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719085+0000", 2024-09-16T20:50:50.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010446399999999999 2024-09-16T20:50:50.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:50.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719242+0000", 2024-09-16T20:50:50.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000156928 2024-09-16T20:50:50.537 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:50.537 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:50.537 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:50.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.538 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9971 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 643072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:50.538 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642102+0000", 2024-09-16T20:50:50.538 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 59.841435423999997, 2024-09-16T20:50:50.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077421016999999995, 2024-09-16T20:50:50.538 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:50.538 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:50.538 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:50.538 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:50.538 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:50.538 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:50.538 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9971 2024-09-16T20:50:50.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.539 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:50.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.539 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:50.539 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642102+0000", 2024-09-16T20:50:50.539 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:50.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.539 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:50.539 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642102+0000", 2024-09-16T20:50:50.539 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:50.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:50.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642103+0000", 2024-09-16T20:50:50.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7399999999999998e-07 2024-09-16T20:50:50.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:50.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642105+0000", 2024-09-16T20:50:50.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2649999999999999e-06 2024-09-16T20:50:50.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:50.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642112+0000", 2024-09-16T20:50:50.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.294e-06 2024-09-16T20:50:50.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.541 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:50.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642118+0000", 2024-09-16T20:50:50.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4500000000000003e-06 2024-09-16T20:50:50.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.541 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:50.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719285+0000", 2024-09-16T20:50:50.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077166883000000006 2024-09-16T20:50:50.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.541 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:50.542 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719368+0000", 2024-09-16T20:50:50.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2811000000000004e-05 2024-09-16T20:50:50.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:50.542 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719523+0000", 2024-09-16T20:50:50.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015584 2024-09-16T20:50:50.542 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:50.542 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:50.542 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:50.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.542 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9972 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 648192~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:50.543 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642125+0000", 2024-09-16T20:50:50.543 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 59.841412476999999, 2024-09-16T20:50:50.543 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077710324999999997, 2024-09-16T20:50:50.543 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:50.543 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:50.543 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:50.543 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:50.543 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:50.543 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:50.543 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9972 2024-09-16T20:50:50.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.543 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:50.543 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.544 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:50.544 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642125+0000", 2024-09-16T20:50:50.544 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:50.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.544 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:50.544 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642125+0000", 2024-09-16T20:50:50.544 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:50.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.544 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:50.544 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642126+0000", 2024-09-16T20:50:50.544 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1899999999999998e-07 2024-09-16T20:50:50.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.545 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:50.545 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642128+0000", 2024-09-16T20:50:50.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8619999999999999e-06 2024-09-16T20:50:50.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.545 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:50.545 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642134+0000", 2024-09-16T20:50:50.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5289999999999997e-06 2024-09-16T20:50:50.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.545 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:50.545 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642139+0000", 2024-09-16T20:50:50.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1250000000000001e-06 2024-09-16T20:50:50.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:50.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719570+0000", 2024-09-16T20:50:50.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077430516000000005 2024-09-16T20:50:50.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:50.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719655+0000", 2024-09-16T20:50:50.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5500000000000005e-05 2024-09-16T20:50:50.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:50.547 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719836+0000", 2024-09-16T20:50:50.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018037399999999999 2024-09-16T20:50:50.547 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:50.547 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:50.547 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:50.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.547 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9973 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 653312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:50.547 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:50:50.547 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 59.841391866999999, 2024-09-16T20:50:50.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077905262000000003, 2024-09-16T20:50:50.547 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:50.547 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:50.548 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:50.548 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:50.548 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:50.548 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:50.548 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9973 2024-09-16T20:50:50.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.548 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:50.548 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.548 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:50.548 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:50:50.548 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:50.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.548 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.548 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:50.549 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:50:50.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:50.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.549 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:50.549 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:50:50.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:50:50.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.549 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:50.549 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642148+0000", 2024-09-16T20:50:50.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0219999999999999e-06 2024-09-16T20:50:50.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.549 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:50.550 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642154+0000", 2024-09-16T20:50:50.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8540000000000002e-06 2024-09-16T20:50:50.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.550 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:50.550 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642159+0000", 2024-09-16T20:50:50.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.045e-06 2024-09-16T20:50:50.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.550 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:50.550 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719875+0000", 2024-09-16T20:50:50.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077716116000000002 2024-09-16T20:50:50.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.551 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:50.551 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719934+0000", 2024-09-16T20:50:50.551 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8819e-05 2024-09-16T20:50:50.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.551 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:50.551 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720051+0000", 2024-09-16T20:50:50.551 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000116947 2024-09-16T20:50:50.551 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:50.551 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:50.551 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:50.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.552 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.552 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9974 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 658432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:50.552 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642166+0000", 2024-09-16T20:50:50.552 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 59.841371952999999, 2024-09-16T20:50:50.552 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.078179786000000001, 2024-09-16T20:50:50.552 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:50.552 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:50.552 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:50.552 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:50.552 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:50.552 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:50.552 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9974 2024-09-16T20:50:50.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.553 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:50.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.553 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:50.553 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642166+0000", 2024-09-16T20:50:50.553 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:50.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.553 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:50.553 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642166+0000", 2024-09-16T20:50:50.553 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:50.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.553 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:50.553 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647343+0000", 2024-09-16T20:50:50.554 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0051769440000000002 2024-09-16T20:50:50.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.554 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.554 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:50.554 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647348+0000", 2024-09-16T20:50:50.554 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3619999999999997e-06 2024-09-16T20:50:50.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.554 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.554 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:50.554 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647359+0000", 2024-09-16T20:50:50.554 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0929e-05 2024-09-16T20:50:50.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.554 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:50.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647366+0000", 2024-09-16T20:50:50.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0910000000000004e-06 2024-09-16T20:50:50.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:50.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720094+0000", 2024-09-16T20:50:50.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.072727403999999995 2024-09-16T20:50:50.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:50.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720173+0000", 2024-09-16T20:50:50.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9382000000000007e-05 2024-09-16T20:50:50.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.556 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:50.556 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720346+0000", 2024-09-16T20:50:50.556 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017267399999999999 2024-09-16T20:50:50.556 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:50.556 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:50.556 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:50.556 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.556 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.556 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9990 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 740352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:50.556 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647725+0000", 2024-09-16T20:50:50.556 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 59.835812869000002, 2024-09-16T20:50:50.556 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077029919000000002, 2024-09-16T20:50:50.556 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:50.556 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:50.557 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:50.557 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:50.557 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:50.557 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:50.557 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9990 2024-09-16T20:50:50.557 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.557 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:50.557 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.557 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:50.557 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647725+0000", 2024-09-16T20:50:50.557 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:50.557 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.557 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.557 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:50.558 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647725+0000", 2024-09-16T20:50:50.558 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:50.558 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.558 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.558 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:50.558 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647725+0000", 2024-09-16T20:50:50.558 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2399999999999999e-07 2024-09-16T20:50:50.558 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.558 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.558 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:50.558 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647726+0000", 2024-09-16T20:50:50.558 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.116e-06 2024-09-16T20:50:50.558 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:50.559 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647729+0000", 2024-09-16T20:50:50.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.937e-06 2024-09-16T20:50:50.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:50.559 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647732+0000", 2024-09-16T20:50:50.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8739999999999999e-06 2024-09-16T20:50:50.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:50.559 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724612+0000", 2024-09-16T20:50:50.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076880102000000006 2024-09-16T20:50:50.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.560 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:50.560 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724649+0000", 2024-09-16T20:50:50.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6738e-05 2024-09-16T20:50:50.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.560 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:50.560 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724755+0000", 2024-09-16T20:50:50.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010592800000000001 2024-09-16T20:50:50.560 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:50.560 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:50.560 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:50.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.561 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.561 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9991 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 745472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:50.561 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:50:50.561 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 59.835802559999998, 2024-09-16T20:50:50.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077257820000000005, 2024-09-16T20:50:50.561 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:50.561 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:50.561 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:50.561 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:50.561 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:50.561 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:50.561 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9991 2024-09-16T20:50:50.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.562 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:50.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.562 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:50.562 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:50:50.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:50.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.562 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:50.562 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:50:50.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:50.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.562 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:50.562 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:50:50.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1199999999999999e-07 2024-09-16T20:50:50.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.563 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:50.563 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647736+0000", 2024-09-16T20:50:50.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3699999999999999e-07 2024-09-16T20:50:50.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.563 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:50.563 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647739+0000", 2024-09-16T20:50:50.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9359999999999999e-06 2024-09-16T20:50:50.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:50.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647742+0000", 2024-09-16T20:50:50.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5720000000000001e-06 2024-09-16T20:50:50.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:50.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724774+0000", 2024-09-16T20:50:50.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077032457999999998 2024-09-16T20:50:50.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:50.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724818+0000", 2024-09-16T20:50:50.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3803000000000001e-05 2024-09-16T20:50:50.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.565 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:50.565 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724993+0000", 2024-09-16T20:50:50.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000174902 2024-09-16T20:50:50.565 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:50.565 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:50.565 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:50.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.565 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9992 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 750592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:50.565 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:50:50.565 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 59.835792971000004, 2024-09-16T20:50:50.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077553227000000002, 2024-09-16T20:50:50.565 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:50.566 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:50.566 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:50.566 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:50.566 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:50.566 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:50.566 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9992 2024-09-16T20:50:50.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.566 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:50.566 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.566 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:50.566 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:50:50.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:50.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.567 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:50.567 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:50:50.567 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:50.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.567 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:50.567 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:50:50.567 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0200000000000001e-07 2024-09-16T20:50:50.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.567 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:50.567 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647746+0000", 2024-09-16T20:50:50.567 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.088e-06 2024-09-16T20:50:50.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.568 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:50.568 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647749+0000", 2024-09-16T20:50:50.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9170000000000002e-06 2024-09-16T20:50:50.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.568 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:50.568 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647752+0000", 2024-09-16T20:50:50.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.774e-06 2024-09-16T20:50:50.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.568 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:50.568 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725042+0000", 2024-09-16T20:50:50.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077289952999999995 2024-09-16T20:50:50.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.569 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:50.569 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725113+0000", 2024-09-16T20:50:50.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0920999999999994e-05 2024-09-16T20:50:50.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.569 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:50.569 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725298+0000", 2024-09-16T20:50:50.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000185372 2024-09-16T20:50:50.569 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:50.569 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:50.570 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:50.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.570 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9993 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 755712~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:50.570 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647755+0000", 2024-09-16T20:50:50.570 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 59.835782944999998, 2024-09-16T20:50:50.570 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077725674999999994, 2024-09-16T20:50:50.570 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:50.570 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:50.570 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:50.570 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:50.570 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:50.570 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:50.570 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9993 2024-09-16T20:50:50.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.571 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:50.571 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.571 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:50.571 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647755+0000", 2024-09-16T20:50:50.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:50.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.571 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.571 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:50.571 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647755+0000", 2024-09-16T20:50:50.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:50.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.571 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.571 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:50.571 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653406+0000", 2024-09-16T20:50:50.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.005650876 2024-09-16T20:50:50.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.572 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:50.572 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653411+0000", 2024-09-16T20:50:50.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8899999999999998e-06 2024-09-16T20:50:50.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.572 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:50.572 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653423+0000", 2024-09-16T20:50:50.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2447e-05 2024-09-16T20:50:50.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.572 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:50.572 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653431+0000", 2024-09-16T20:50:50.573 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5730000000000002e-06 2024-09-16T20:50:50.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.573 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.573 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:50.573 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725333+0000", 2024-09-16T20:50:50.573 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.071902743000000005 2024-09-16T20:50:50.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.573 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.573 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:50.573 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725385+0000", 2024-09-16T20:50:50.573 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1585000000000002e-05 2024-09-16T20:50:50.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.574 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:50.574 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725481+0000", 2024-09-16T20:50:50.574 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5561000000000002e-05 2024-09-16T20:50:50.574 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:50.574 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:50.574 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:50.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.574 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6061.0:5400 3.b 3:d353d652:::rbd_data.110e16eafbd.0000000000000010:head [read 577536~28672] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-16T20:50:50.574 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:27.761367+0000", 2024-09-16T20:50:50.574 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.722170998999999, 2024-09-16T20:50:50.574 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0029066630000001, 2024-09-16T20:50:50.575 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:50.575 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:50.575 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:50.575 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:50.575 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6061", 2024-09-16T20:50:50.575 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3930128440", 2024-09-16T20:50:50.575 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5400 2024-09-16T20:50:50.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.575 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:50.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.575 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:50.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761367+0000", 2024-09-16T20:50:50.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:50.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:50.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761367+0000", 2024-09-16T20:50:50.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:50.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:50.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761369+0000", 2024-09-16T20:50:50.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.934e-06 2024-09-16T20:50:50.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:50.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761374+0000", 2024-09-16T20:50:50.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7139999999999998e-06 2024-09-16T20:50:50.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.577 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:50.577 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761387+0000", 2024-09-16T20:50:50.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2349e-05 2024-09-16T20:50:50.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.577 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:50.577 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761393+0000", 2024-09-16T20:50:50.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.652e-06 2024-09-16T20:50:50.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.577 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:50.577 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761442+0000", 2024-09-16T20:50:50.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9011000000000002e-05 2024-09-16T20:50:50.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.578 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:50.578 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761451+0000", 2024-09-16T20:50:50.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1920000000000005e-06 2024-09-16T20:50:50.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.578 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:50.578 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:28.764169+0000", 2024-09-16T20:50:50.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.002718161 2024-09-16T20:50:50.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.578 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:50.578 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:28.764273+0000", 2024-09-16T20:50:50.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010465 2024-09-16T20:50:50.579 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:50.579 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:50.579 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:50.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.579 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6061.0:5424 3.b 3:d353d652:::rbd_data.110e16eafbd.0000000000000010:head [read 606208~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-16T20:50:50.579 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:27.766882+0000", 2024-09-16T20:50:50.579 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.716655635999999, 2024-09-16T20:50:50.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99753650500000002, 2024-09-16T20:50:50.579 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:50.579 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:50.579 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:50.579 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:50.580 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6061", 2024-09-16T20:50:50.580 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3930128440", 2024-09-16T20:50:50.580 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5424 2024-09-16T20:50:50.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.580 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:50.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:50.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766882+0000", 2024-09-16T20:50:50.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:50.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:50.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766882+0000", 2024-09-16T20:50:50.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:50.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:50.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766883+0000", 2024-09-16T20:50:50.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3289999999999999e-06 2024-09-16T20:50:50.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:50.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766888+0000", 2024-09-16T20:50:50.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5800000000000002e-06 2024-09-16T20:50:50.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:50.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766900+0000", 2024-09-16T20:50:50.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1861e-05 2024-09-16T20:50:50.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:50.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766909+0000", 2024-09-16T20:50:50.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5210000000000008e-06 2024-09-16T20:50:50.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:50.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766958+0000", 2024-09-16T20:50:50.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8718999999999999e-05 2024-09-16T20:50:50.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:50.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766975+0000", 2024-09-16T20:50:50.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7231000000000002e-05 2024-09-16T20:50:50.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:50.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:28.764327+0000", 2024-09-16T20:50:50.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99735112500000001 2024-09-16T20:50:50.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:50.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:28.764419+0000", 2024-09-16T20:50:50.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2139000000000007e-05 2024-09-16T20:50:50.583 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:50.584 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:50.584 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:50.584 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:50.584 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:50.584 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:50.585 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:50:50.586 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:50.584+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:50.686 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:50.685+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:50.698 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:50.698 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:50:50.699 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:50:50.699 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:50.708 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:50:50.787 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:50.786+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:50.856 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:50.856 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:50:50.856 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:50:50.856 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:50:50.856 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:50.864 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:50:50.888 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:50.887+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:50.972 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:50.973 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:50:50.973 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:50:50.973 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:50:50.973 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.973 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12548 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 628736~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:50.973 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757009+0000", 2024-09-16T20:50:50.973 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 60.213960479999997, 2024-09-16T20:50:50.973 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065743956000000006, 2024-09-16T20:50:50.973 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:50.973 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:50.973 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:50.973 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:50.973 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:50.974 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:50.974 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12548 2024-09-16T20:50:50.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.974 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:50.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.974 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:50.974 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757009+0000", 2024-09-16T20:50:50.974 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:50.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.974 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:50.974 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757009+0000", 2024-09-16T20:50:50.974 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:50.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.975 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.975 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:50.975 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757010+0000", 2024-09-16T20:50:50.975 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9900000000000001e-07 2024-09-16T20:50:50.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.975 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.975 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:50.975 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757012+0000", 2024-09-16T20:50:50.975 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2689999999999998e-06 2024-09-16T20:50:50.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.975 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.975 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:50.975 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757022+0000", 2024-09-16T20:50:50.975 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9699999999999994e-06 2024-09-16T20:50:50.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.976 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.976 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:50.976 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757028+0000", 2024-09-16T20:50:50.976 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3799999999999999e-06 2024-09-16T20:50:50.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.976 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.976 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:50.976 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822552+0000", 2024-09-16T20:50:50.976 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065524281000000004 2024-09-16T20:50:50.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.976 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.976 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:50.976 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822613+0000", 2024-09-16T20:50:50.977 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0430999999999998e-05 2024-09-16T20:50:50.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.977 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:50.977 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822753+0000", 2024-09-16T20:50:50.977 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000140226 2024-09-16T20:50:50.977 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:50.977 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:50.977 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:50.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.977 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12549 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 633856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:50.977 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:50:50.978 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 60.213931733000003, 2024-09-16T20:50:50.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065978518, 2024-09-16T20:50:50.978 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:50.978 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:50.978 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:50.978 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:50.978 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:50.978 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:50.978 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12549 2024-09-16T20:50:50.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.978 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:50.978 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.978 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:50.978 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:50:50.979 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:50.979 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.979 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.979 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:50.979 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:50:50.979 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:50.979 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.979 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.979 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:50.979 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:50:50.979 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9299999999999999e-07 2024-09-16T20:50:50.979 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.979 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.979 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:50.979 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757040+0000", 2024-09-16T20:50:50.980 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.677e-06 2024-09-16T20:50:50.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.980 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.980 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:50.980 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757045+0000", 2024-09-16T20:50:50.980 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8149999999999996e-06 2024-09-16T20:50:50.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.980 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.980 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:50.980 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757048+0000", 2024-09-16T20:50:50.980 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1e-06 2024-09-16T20:50:50.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.981 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.981 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:50.981 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822796+0000", 2024-09-16T20:50:50.981 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065748081 2024-09-16T20:50:50.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.981 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.981 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:50.981 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822867+0000", 2024-09-16T20:50:50.981 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0679000000000005e-05 2024-09-16T20:50:50.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.982 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.982 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:50.982 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823016+0000", 2024-09-16T20:50:50.982 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014987299999999999 2024-09-16T20:50:50.982 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:50.982 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:50.982 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:50.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.982 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.982 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12550 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 638976~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:50.982 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757060+0000", 2024-09-16T20:50:50.982 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 60.213909317999999, 2024-09-16T20:50:50.982 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066153564999999998, 2024-09-16T20:50:50.982 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:50.983 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:50.983 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:50.983 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:50.983 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:50.983 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:50.983 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12550 2024-09-16T20:50:50.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.983 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:50.983 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.983 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:50.983 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757060+0000", 2024-09-16T20:50:50.983 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:50.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.983 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.984 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:50.984 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757060+0000", 2024-09-16T20:50:50.984 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:50.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.984 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.984 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:50.984 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757061+0000", 2024-09-16T20:50:50.984 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.15e-07 2024-09-16T20:50:50.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.984 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.984 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:50.984 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757062+0000", 2024-09-16T20:50:50.984 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.663e-06 2024-09-16T20:50:50.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.985 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.985 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:50.985 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757066+0000", 2024-09-16T20:50:50.985 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.817e-06 2024-09-16T20:50:50.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.985 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.985 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:50.985 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757070+0000", 2024-09-16T20:50:50.985 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1509999999999997e-06 2024-09-16T20:50:50.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.985 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.985 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:50.985 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823044+0000", 2024-09-16T20:50:50.986 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065973992999999995 2024-09-16T20:50:50.986 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.986 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.986 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:50.986 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823082+0000", 2024-09-16T20:50:50.986 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8050999999999999e-05 2024-09-16T20:50:50.986 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.986 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.986 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:50.986 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823214+0000", 2024-09-16T20:50:50.986 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013157500000000001 2024-09-16T20:50:50.986 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:50.986 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:50.986 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:50.987 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.987 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.987 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12551 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 644096~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:50.987 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:50:50.987 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 60.213895620000002, 2024-09-16T20:50:50.987 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066391378000000001, 2024-09-16T20:50:50.987 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:50.987 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:50.987 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:50.987 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:50.987 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:50.987 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:50.987 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12551 2024-09-16T20:50:50.987 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.988 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:50.988 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.988 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:50.988 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:50:50.988 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:50.988 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.988 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.988 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:50.988 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:50:50.988 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:50.988 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.988 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.988 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:50.988 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:50:50.989 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2999999999999999e-07 2024-09-16T20:50:50.989 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.989 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.989 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:50.989 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757075+0000", 2024-09-16T20:50:50.989 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.097e-06 2024-09-16T20:50:50.989 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.989 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.989 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:50.989 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757082+0000", 2024-09-16T20:50:50.989 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9500000000000004e-06 2024-09-16T20:50:50.989 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.989 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.990 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:50.990 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757086+0000", 2024-09-16T20:50:50.990 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2229999999999999e-06 2024-09-16T20:50:50.990 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.990 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.990 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:50.990 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823262+0000", 2024-09-16T20:50:50.990 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066176078999999999 2024-09-16T20:50:50.990 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.990 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.990 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:50.990 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823336+0000", 2024-09-16T20:50:50.990 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3979000000000004e-05 2024-09-16T20:50:50.990 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.991 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.991 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:50.991 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823465+0000", 2024-09-16T20:50:50.991 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012982 2024-09-16T20:50:50.991 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:50.991 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:50.991 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:50.991 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.991 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.991 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12552 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 649216~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:50.991 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757089+0000", 2024-09-16T20:50:50.991 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 60.213880478999997, 2024-09-16T20:50:50.991 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066655617, 2024-09-16T20:50:50.992 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:50.992 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:50.992 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:50.992 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:50.992 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:50.992 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:50.992 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12552 2024-09-16T20:50:50.992 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.992 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:50.992 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.992 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:50.993 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757089+0000", 2024-09-16T20:50:50.993 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:50.993 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.993 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.993 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:50.993 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757089+0000", 2024-09-16T20:50:50.993 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:50.993 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.993 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.993 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:50.993 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759492+0000", 2024-09-16T20:50:50.993 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0024026109999999998 2024-09-16T20:50:50.993 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.994 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.994 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:50.994 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759497+0000", 2024-09-16T20:50:50.994 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1120000000000004e-06 2024-09-16T20:50:50.994 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.994 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.994 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:50.994 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759508+0000", 2024-09-16T20:50:50.994 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0981e-05 2024-09-16T20:50:50.994 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.994 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.994 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:50.995 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759516+0000", 2024-09-16T20:50:50.995 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7270000000000007e-06 2024-09-16T20:50:50.995 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.995 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.995 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:50.995 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823504+0000", 2024-09-16T20:50:50.995 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.063987984999999997 2024-09-16T20:50:50.995 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.995 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.995 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:50.995 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823579+0000", 2024-09-16T20:50:50.995 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5703000000000003e-05 2024-09-16T20:50:50.995 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.995 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.996 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:50.996 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823745+0000", 2024-09-16T20:50:50.996 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016549800000000001 2024-09-16T20:50:50.996 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:50.996 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:50.996 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:50.996 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.996 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.996 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12560 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 690176~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:50.996 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.759658+0000", 2024-09-16T20:50:50.996 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 60.211311277, 2024-09-16T20:50:50.996 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065858517000000005, 2024-09-16T20:50:50.996 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:50.997 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:50.997 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:50.997 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:50.997 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:50.997 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:50.997 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12560 2024-09-16T20:50:50.997 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.997 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:50.997 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.997 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:50.997 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759658+0000", 2024-09-16T20:50:50.997 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:50.997 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.998 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.998 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:50.998 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759658+0000", 2024-09-16T20:50:50.998 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:50.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.998 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.998 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:50.998 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759659+0000", 2024-09-16T20:50:50.998 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.17e-07 2024-09-16T20:50:50.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.998 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.998 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:50.998 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759660+0000", 2024-09-16T20:50:50.998 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0189999999999999e-06 2024-09-16T20:50:50.999 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.999 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.999 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:50.999 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759663+0000", 2024-09-16T20:50:50.999 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0010000000000002e-06 2024-09-16T20:50:50.999 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.999 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.999 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:50.999 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759665+0000", 2024-09-16T20:50:50.999 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5440000000000001e-06 2024-09-16T20:50:50.999 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:50.999 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:50.999 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:50.999 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825355+0000", 2024-09-16T20:50:51.000 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065690224000000005 2024-09-16T20:50:51.000 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.000 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.000 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:51.000 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825394+0000", 2024-09-16T20:50:51.000 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8810999999999998e-05 2024-09-16T20:50:51.000 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.000 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.000 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:51.000 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825517+0000", 2024-09-16T20:50:51.000 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012270099999999999 2024-09-16T20:50:51.000 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.000 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:51.001 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.001 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.001 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12561 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 695296~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:51.001 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.759668+0000", 2024-09-16T20:50:51.001 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 60.211301435000003, 2024-09-16T20:50:51.001 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066116999999999995, 2024-09-16T20:50:51.001 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:51.001 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:51.001 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:51.001 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:51.001 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:51.001 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:51.002 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12561 2024-09-16T20:50:51.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.002 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:51.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.002 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:51.002 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759668+0000", 2024-09-16T20:50:51.002 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.002 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:51.002 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759668+0000", 2024-09-16T20:50:51.002 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.003 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:51.003 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765379+0000", 2024-09-16T20:50:51.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057104410000000001 2024-09-16T20:50:51.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.003 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.003 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:51.003 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765383+0000", 2024-09-16T20:50:51.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2980000000000002e-06 2024-09-16T20:50:51.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.003 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.003 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:51.003 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765395+0000", 2024-09-16T20:50:51.004 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1671000000000001e-05 2024-09-16T20:50:51.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.004 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.004 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:51.004 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765403+0000", 2024-09-16T20:50:51.004 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1210000000000007e-06 2024-09-16T20:50:51.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.004 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.004 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:51.004 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825552+0000", 2024-09-16T20:50:51.004 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.060149498000000003 2024-09-16T20:50:51.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.004 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.005 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:51.005 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825625+0000", 2024-09-16T20:50:51.005 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2540999999999995e-05 2024-09-16T20:50:51.005 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.005 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.005 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:51.005 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825785+0000", 2024-09-16T20:50:51.005 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016043000000000001 2024-09-16T20:50:51.005 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.005 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:51.005 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.005 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.005 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.006 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12585 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 818176~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:51.006 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.765848+0000", 2024-09-16T20:50:51.006 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 60.205121464000001, 2024-09-16T20:50:51.006 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065843218999999995, 2024-09-16T20:50:51.006 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:51.006 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:51.006 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:51.006 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:51.006 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:51.006 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:51.006 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12585 2024-09-16T20:50:51.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.006 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:51.006 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.007 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:51.007 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765848+0000", 2024-09-16T20:50:51.007 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.007 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.007 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.007 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:51.007 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765848+0000", 2024-09-16T20:50:51.007 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.007 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.007 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.007 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:51.007 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768683+0000", 2024-09-16T20:50:51.007 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0028344519999999999 2024-09-16T20:50:51.007 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.008 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:51.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768686+0000", 2024-09-16T20:50:51.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8789999999999996e-06 2024-09-16T20:50:51.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.008 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:51.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768696+0000", 2024-09-16T20:50:51.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6709999999999994e-06 2024-09-16T20:50:51.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.008 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:51.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768702+0000", 2024-09-16T20:50:51.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2550000000000003e-06 2024-09-16T20:50:51.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.009 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.009 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:51.009 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.831499+0000", 2024-09-16T20:50:51.009 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.062797081000000005 2024-09-16T20:50:51.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.009 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.009 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:51.009 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.831538+0000", 2024-09-16T20:50:51.009 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8523999999999999e-05 2024-09-16T20:50:51.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.009 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.009 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:51.010 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.831691+0000", 2024-09-16T20:50:51.010 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015335699999999999 2024-09-16T20:50:51.010 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.010 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:51.010 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.010 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.010 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.010 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12597 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 879616~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:51.010 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.768883+0000", 2024-09-16T20:50:51.010 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 60.202086735999998, 2024-09-16T20:50:51.010 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065816527, 2024-09-16T20:50:51.010 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:51.010 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:51.010 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:51.010 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:51.011 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:51.011 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:51.011 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12597 2024-09-16T20:50:51.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.011 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:51.011 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.011 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:51.011 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768883+0000", 2024-09-16T20:50:51.011 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.011 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.011 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:51.011 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768883+0000", 2024-09-16T20:50:51.011 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.012 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.012 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:51.012 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771508+0000", 2024-09-16T20:50:51.012 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0026247589999999999 2024-09-16T20:50:51.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.012 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.012 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:51.012 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771513+0000", 2024-09-16T20:50:51.012 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9949999999999996e-06 2024-09-16T20:50:51.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.012 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.012 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:51.013 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771525+0000", 2024-09-16T20:50:51.013 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2033e-05 2024-09-16T20:50:51.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.013 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.013 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:51.013 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771532+0000", 2024-09-16T20:50:51.013 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4220000000000003e-06 2024-09-16T20:50:51.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.013 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.013 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:51.013 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.834532+0000", 2024-09-16T20:50:51.013 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.063000159 2024-09-16T20:50:51.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.014 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.014 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:51.014 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.834590+0000", 2024-09-16T20:50:51.014 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8218999999999999e-05 2024-09-16T20:50:51.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.014 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.014 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:51.014 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.834699+0000", 2024-09-16T20:50:51.014 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010894 2024-09-16T20:50:51.014 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.014 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:51.014 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.014 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.015 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12619 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 992256~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:51.015 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.774656+0000", 2024-09-16T20:50:51.015 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 60.196313961999998, 2024-09-16T20:50:51.015 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065715859000000001, 2024-09-16T20:50:51.015 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:51.015 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:51.015 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:51.015 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:51.015 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:51.015 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:51.015 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12619 2024-09-16T20:50:51.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.015 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:51.015 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.015 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:51.016 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774656+0000", 2024-09-16T20:50:51.016 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.016 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.016 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:51.016 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774656+0000", 2024-09-16T20:50:51.016 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.016 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.016 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:51.016 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774875+0000", 2024-09-16T20:50:51.016 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000218898 2024-09-16T20:50:51.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.017 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.017 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:51.017 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774879+0000", 2024-09-16T20:50:51.017 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0389999999999998e-06 2024-09-16T20:50:51.017 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.017 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.017 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:51.017 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774890+0000", 2024-09-16T20:50:51.017 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1250000000000001e-05 2024-09-16T20:50:51.017 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.017 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.017 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:51.018 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774897+0000", 2024-09-16T20:50:51.018 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7619999999999998e-06 2024-09-16T20:50:51.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.018 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.018 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:51.018 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840139+0000", 2024-09-16T20:50:51.018 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065242438 2024-09-16T20:50:51.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.018 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.018 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:51.018 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840186+0000", 2024-09-16T20:50:51.018 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6894999999999997e-05 2024-09-16T20:50:51.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.018 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.019 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:51.019 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840372+0000", 2024-09-16T20:50:51.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000185577 2024-09-16T20:50:51.019 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.019 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:51.019 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.019 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12620 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 997376~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:51.019 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.774908+0000", 2024-09-16T20:50:51.019 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 60.196061741000001, 2024-09-16T20:50:51.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065782754999999998, 2024-09-16T20:50:51.019 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:51.019 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:51.020 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:51.020 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:51.020 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:51.020 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:51.020 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12620 2024-09-16T20:50:51.020 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.020 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:51.020 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.020 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:51.020 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774908+0000", 2024-09-16T20:50:51.020 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.020 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.020 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.021 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:51.021 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774908+0000", 2024-09-16T20:50:51.021 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.021 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.021 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:51.021 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775102+0000", 2024-09-16T20:50:51.021 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000194098 2024-09-16T20:50:51.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.021 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.021 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:51.021 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775105+0000", 2024-09-16T20:50:51.021 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8739999999999999e-06 2024-09-16T20:50:51.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.022 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.022 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:51.022 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775111+0000", 2024-09-16T20:50:51.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9519999999999999e-06 2024-09-16T20:50:51.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.022 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.022 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:51.022 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775115+0000", 2024-09-16T20:50:51.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.002e-06 2024-09-16T20:50:51.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.022 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.022 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:51.023 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840409+0000", 2024-09-16T20:50:51.023 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065294195999999999 2024-09-16T20:50:51.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.023 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.023 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:51.023 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840507+0000", 2024-09-16T20:50:51.023 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7662000000000001e-05 2024-09-16T20:50:51.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.023 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.023 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:51.023 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840691+0000", 2024-09-16T20:50:51.023 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000183971 2024-09-16T20:50:51.023 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.023 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:51.024 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.024 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12621 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1002496~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:51.024 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775123+0000", 2024-09-16T20:50:51.024 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 60.195847045000001, 2024-09-16T20:50:51.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065829586999999995, 2024-09-16T20:50:51.024 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:51.024 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:51.024 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:51.024 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:51.024 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:51.024 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:51.024 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12621 2024-09-16T20:50:51.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.025 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:51.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:51.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775123+0000", 2024-09-16T20:50:51.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:51.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775123+0000", 2024-09-16T20:50:51.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.026 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:51.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775348+0000", 2024-09-16T20:50:51.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000225295 2024-09-16T20:50:51.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.026 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:51.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775352+0000", 2024-09-16T20:50:51.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0810000000000004e-06 2024-09-16T20:50:51.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.026 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:51.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775361+0000", 2024-09-16T20:50:51.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7399999999999993e-06 2024-09-16T20:50:51.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.027 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.027 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:51.027 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775367+0000", 2024-09-16T20:50:51.027 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5200000000000003e-06 2024-09-16T20:50:51.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.027 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.027 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:51.027 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840730+0000", 2024-09-16T20:50:51.027 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065362803999999997 2024-09-16T20:50:51.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.027 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.027 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:51.027 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840801+0000", 2024-09-16T20:50:51.028 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1006000000000004e-05 2024-09-16T20:50:51.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.028 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.028 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:51.028 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840952+0000", 2024-09-16T20:50:51.028 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015114100000000001 2024-09-16T20:50:51.028 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.028 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:51.028 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.028 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.028 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12622 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1007616~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:51.028 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775381+0000", 2024-09-16T20:50:51.029 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 60.195588211, 2024-09-16T20:50:51.029 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065822581000000005, 2024-09-16T20:50:51.029 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:51.029 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:51.029 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:51.029 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:51.029 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:51.029 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:51.029 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12622 2024-09-16T20:50:51.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.029 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:51.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.029 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:51.030 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775381+0000", 2024-09-16T20:50:51.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.030 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:51.030 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775381+0000", 2024-09-16T20:50:51.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.030 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:51.030 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775563+0000", 2024-09-16T20:50:51.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018153700000000001 2024-09-16T20:50:51.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.031 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.031 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:51.031 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775566+0000", 2024-09-16T20:50:51.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9620000000000002e-06 2024-09-16T20:50:51.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.031 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.031 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:51.031 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775572+0000", 2024-09-16T20:50:51.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8529999999999997e-06 2024-09-16T20:50:51.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.032 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:51.032 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775576+0000", 2024-09-16T20:50:51.032 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8920000000000002e-06 2024-09-16T20:50:51.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.032 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:51.032 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840990+0000", 2024-09-16T20:50:51.032 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065414599000000004 2024-09-16T20:50:51.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.032 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:51.032 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841068+0000", 2024-09-16T20:50:51.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8157000000000004e-05 2024-09-16T20:50:51.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.033 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.033 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:51.033 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841204+0000", 2024-09-16T20:50:51.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013558099999999999 2024-09-16T20:50:51.033 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.033 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:51.033 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.033 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.033 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12623 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1012736~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:51.033 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775583+0000", 2024-09-16T20:50:51.033 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 60.195386773000003, 2024-09-16T20:50:51.034 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065929087999999997, 2024-09-16T20:50:51.034 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:51.034 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:51.034 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:51.034 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:51.034 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:51.034 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:51.034 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12623 2024-09-16T20:50:51.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.034 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:51.034 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.034 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:51.034 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775583+0000", 2024-09-16T20:50:51.034 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.035 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:51.035 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775583+0000", 2024-09-16T20:50:51.035 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.035 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:51.035 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775804+0000", 2024-09-16T20:50:51.035 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00022116900000000001 2024-09-16T20:50:51.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.035 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:51.036 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775807+0000", 2024-09-16T20:50:51.036 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9529999999999999e-06 2024-09-16T20:50:51.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.036 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:51.036 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775814+0000", 2024-09-16T20:50:51.036 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7490000000000001e-06 2024-09-16T20:50:51.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.036 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:51.036 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775818+0000", 2024-09-16T20:50:51.036 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1999999999999996e-06 2024-09-16T20:50:51.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.037 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.037 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:51.037 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841267+0000", 2024-09-16T20:50:51.037 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065449127999999995 2024-09-16T20:50:51.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.037 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.037 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:51.037 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841344+0000", 2024-09-16T20:50:51.037 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6546999999999997e-05 2024-09-16T20:50:51.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.037 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.037 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:51.037 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841512+0000", 2024-09-16T20:50:51.038 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016834200000000001 2024-09-16T20:50:51.038 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.038 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:51.038 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.038 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.038 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12624 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1017856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:51.038 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:50:51.038 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 60.195143633000001, 2024-09-16T20:50:51.038 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065945866000000006, 2024-09-16T20:50:51.038 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:51.038 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:51.038 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:51.038 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:51.039 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:51.039 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:51.039 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12624 2024-09-16T20:50:51.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.039 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:51.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.039 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:51.039 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:50:51.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.039 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:51.039 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:50:51.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.040 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.040 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:51.040 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:50:51.040 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-16T20:50:51.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.040 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.040 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:51.040 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775828+0000", 2024-09-16T20:50:51.040 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3999999999999999e-06 2024-09-16T20:50:51.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.040 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.040 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:51.040 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775831+0000", 2024-09-16T20:50:51.041 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.478e-06 2024-09-16T20:50:51.041 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.041 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.041 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:51.041 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775834+0000", 2024-09-16T20:50:51.041 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9440000000000001e-06 2024-09-16T20:50:51.041 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.041 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.041 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:51.041 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841547+0000", 2024-09-16T20:50:51.041 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065712801000000001 2024-09-16T20:50:51.041 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.041 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.042 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:51.042 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841618+0000", 2024-09-16T20:50:51.042 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1151999999999999e-05 2024-09-16T20:50:51.042 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.042 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.042 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:51.042 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841772+0000", 2024-09-16T20:50:51.042 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000153721 2024-09-16T20:50:51.042 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.042 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:51.042 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.042 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.042 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.043 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12625 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1022976~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:51.043 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775840+0000", 2024-09-16T20:50:51.043 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 60.195130024000001, 2024-09-16T20:50:51.043 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066165451, 2024-09-16T20:50:51.043 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:51.043 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:51.043 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:51.043 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:51.043 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:51.043 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:51.043 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12625 2024-09-16T20:50:51.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.043 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:51.043 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.044 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:51.044 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775840+0000", 2024-09-16T20:50:51.044 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.044 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.044 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:51.044 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775840+0000", 2024-09-16T20:50:51.044 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.044 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.044 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:51.044 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776014+0000", 2024-09-16T20:50:51.044 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017441099999999999 2024-09-16T20:50:51.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.045 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.045 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:51.045 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776018+0000", 2024-09-16T20:50:51.045 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6509999999999999e-06 2024-09-16T20:50:51.045 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.045 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.045 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:51.045 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776026+0000", 2024-09-16T20:50:51.045 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.028e-06 2024-09-16T20:50:51.045 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.045 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.045 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:51.045 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776031+0000", 2024-09-16T20:50:51.046 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8400000000000002e-06 2024-09-16T20:50:51.046 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.046 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.046 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:51.046 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841801+0000", 2024-09-16T20:50:51.046 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065770472999999996 2024-09-16T20:50:51.046 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.046 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.046 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:51.046 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841847+0000", 2024-09-16T20:50:51.046 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5558e-05 2024-09-16T20:50:51.046 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.046 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.046 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:51.047 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842005+0000", 2024-09-16T20:50:51.047 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015849000000000001 2024-09-16T20:50:51.047 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.047 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:51.047 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.047 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.047 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12626 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1028096~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:51.047 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776041+0000", 2024-09-16T20:50:51.047 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 60.194928724999997, 2024-09-16T20:50:51.047 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066225260999999994, 2024-09-16T20:50:51.047 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:51.047 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:51.048 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:51.048 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:51.048 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:51.048 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:51.048 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12626 2024-09-16T20:50:51.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.048 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:51.048 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.048 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:51.048 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776041+0000", 2024-09-16T20:50:51.048 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.048 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.049 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:51.049 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776041+0000", 2024-09-16T20:50:51.049 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.049 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.049 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.049 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:51.049 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776263+0000", 2024-09-16T20:50:51.049 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00022193899999999999 2024-09-16T20:50:51.049 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.049 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.049 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:51.049 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776268+0000", 2024-09-16T20:50:51.049 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8049999999999997e-06 2024-09-16T20:50:51.050 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.050 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.050 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:51.050 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776280+0000", 2024-09-16T20:50:51.050 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2635e-05 2024-09-16T20:50:51.050 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.050 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.050 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:51.050 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776288+0000", 2024-09-16T20:50:51.050 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8199999999999997e-06 2024-09-16T20:50:51.050 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.050 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.050 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:51.050 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842041+0000", 2024-09-16T20:50:51.051 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065752506000000002 2024-09-16T20:50:51.051 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.051 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.051 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:51.051 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842123+0000", 2024-09-16T20:50:51.051 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2393999999999998e-05 2024-09-16T20:50:51.051 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.051 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.051 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:51.051 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842266+0000", 2024-09-16T20:50:51.051 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014316200000000001 2024-09-16T20:50:51.051 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.051 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:51.051 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.052 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.052 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12627 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1033216~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:51.052 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776301+0000", 2024-09-16T20:50:51.052 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 60.194668765000003, 2024-09-16T20:50:51.052 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066187814999999997, 2024-09-16T20:50:51.052 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:51.052 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:51.052 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:51.052 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:51.052 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:51.052 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:51.052 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12627 2024-09-16T20:50:51.053 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.053 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:51.053 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.053 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:51.053 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776301+0000", 2024-09-16T20:50:51.053 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.053 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.053 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.053 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:51.053 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776301+0000", 2024-09-16T20:50:51.053 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.053 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.053 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.054 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:51.054 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776473+0000", 2024-09-16T20:50:51.054 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017222400000000001 2024-09-16T20:50:51.054 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.054 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.054 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:51.054 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776477+0000", 2024-09-16T20:50:51.054 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7019999999999999e-06 2024-09-16T20:50:51.054 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.054 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.054 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:51.054 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776486+0000", 2024-09-16T20:50:51.054 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2119999999999992e-06 2024-09-16T20:50:51.055 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.055 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.055 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:51.055 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776492+0000", 2024-09-16T20:50:51.055 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8089999999999998e-06 2024-09-16T20:50:51.055 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.055 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.055 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:51.055 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842301+0000", 2024-09-16T20:50:51.055 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065809042999999998 2024-09-16T20:50:51.055 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.055 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.055 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:51.055 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842362+0000", 2024-09-16T20:50:51.056 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1260000000000006e-05 2024-09-16T20:50:51.056 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.056 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.056 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:51.056 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842489+0000", 2024-09-16T20:50:51.056 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012656500000000001 2024-09-16T20:50:51.056 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.056 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:51.056 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.056 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.056 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.056 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12628 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1038336~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:51.056 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776502+0000", 2024-09-16T20:50:51.057 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 60.194468119, 2024-09-16T20:50:51.057 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066283426000000006, 2024-09-16T20:50:51.057 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:51.057 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:51.057 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:51.057 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:51.057 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:51.057 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:51.057 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12628 2024-09-16T20:50:51.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.057 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:51.057 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.057 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:51.058 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776502+0000", 2024-09-16T20:50:51.058 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.058 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.058 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.058 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:51.058 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776502+0000", 2024-09-16T20:50:51.058 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.058 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.058 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.058 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:51.058 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776716+0000", 2024-09-16T20:50:51.058 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000214596 2024-09-16T20:50:51.058 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.058 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.059 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:51.059 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776723+0000", 2024-09-16T20:50:51.059 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4000000000000003e-06 2024-09-16T20:50:51.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.059 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.059 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:51.059 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776731+0000", 2024-09-16T20:50:51.059 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6129999999999998e-06 2024-09-16T20:50:51.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.059 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.059 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:51.059 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776737+0000", 2024-09-16T20:50:51.059 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1970000000000001e-06 2024-09-16T20:50:51.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.060 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.060 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:51.060 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842532+0000", 2024-09-16T20:50:51.060 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065795127999999994 2024-09-16T20:50:51.060 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.060 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.060 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:51.060 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842599+0000", 2024-09-16T20:50:51.060 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6711000000000006e-05 2024-09-16T20:50:51.060 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.060 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.060 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:51.060 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842785+0000", 2024-09-16T20:50:51.060 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018578099999999999 2024-09-16T20:50:51.061 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.061 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:51.061 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.061 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.061 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.061 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12629 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1043456~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:51.061 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776748+0000", 2024-09-16T20:50:51.061 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 60.194221742000003, 2024-09-16T20:50:51.061 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066303788000000002, 2024-09-16T20:50:51.061 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:51.061 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:51.061 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:51.061 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:51.062 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:51.062 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:51.062 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12629 2024-09-16T20:50:51.062 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.062 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:51.062 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.062 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:51.062 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776748+0000", 2024-09-16T20:50:51.062 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.062 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.062 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.062 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:51.062 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776748+0000", 2024-09-16T20:50:51.062 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.063 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.063 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:51.063 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776934+0000", 2024-09-16T20:50:51.063 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018598399999999999 2024-09-16T20:50:51.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.063 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.063 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:51.063 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776939+0000", 2024-09-16T20:50:51.063 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1000000000000003e-06 2024-09-16T20:50:51.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.063 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.063 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:51.064 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776954+0000", 2024-09-16T20:50:51.064 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5221e-05 2024-09-16T20:50:51.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.064 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:51.064 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776962+0000", 2024-09-16T20:50:51.064 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.554e-06 2024-09-16T20:50:51.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.064 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:51.064 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842813+0000", 2024-09-16T20:50:51.064 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065851006000000004 2024-09-16T20:50:51.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.065 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:51.065 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842886+0000", 2024-09-16T20:50:51.065 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3663000000000003e-05 2024-09-16T20:50:51.065 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.065 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.065 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:51.065 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.843052+0000", 2024-09-16T20:50:51.065 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016526 2024-09-16T20:50:51.065 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.065 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:51.065 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.065 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.065 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:51.065 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:51.067 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:50:51.067 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:51.065+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:51.168 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:51.167+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:51.174 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:51.174 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:50:51.175 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:50:51.175 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:51.183 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:50:51.268 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:51.268+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:51.330 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:51.330 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:50:51.330 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:50:51.330 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:50:51.330 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:51.339 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:50:51.369 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:51.368+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:51.470 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:51.470+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:51.488 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:51.488 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:50:51.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:50:51.488 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:50:51.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.488 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5286.0:10643 3.17 3.f6273fb7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-16T20:50:51.488 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:50:51.488 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 102.695457038, 2024-09-16T20:50:51.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.76672347200000002, 2024-09-16T20:50:51.488 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:51.488 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:51.489 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:51.489 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:51.489 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5286", 2024-09-16T20:50:51.489 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/4135433989", 2024-09-16T20:50:51.489 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 10643 2024-09-16T20:50:51.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.489 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:51.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:51.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:50:51.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:51.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:50:51.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:51.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790489+0000", 2024-09-16T20:50:51.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5910000000000001e-06 2024-09-16T20:50:51.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:51.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790493+0000", 2024-09-16T20:50:51.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2669999999999998e-06 2024-09-16T20:50:51.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:51.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790500+0000", 2024-09-16T20:50:51.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3780000000000003e-06 2024-09-16T20:50:51.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:51.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790505+0000", 2024-09-16T20:50:51.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1560000000000003e-06 2024-09-16T20:50:51.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:51.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790559+0000", 2024-09-16T20:50:51.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3494e-05 2024-09-16T20:50:51.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.492 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:51.492 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790566+0000", 2024-09-16T20:50:51.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7649999999999993e-06 2024-09-16T20:50:51.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.492 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:51.492 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:09.557198+0000", 2024-09-16T20:50:51.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.76663136200000004 2024-09-16T20:50:51.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.493 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:51.493 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:09.557211+0000", 2024-09-16T20:50:51.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3458999999999999e-05 2024-09-16T20:50:51.493 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.493 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:51.493 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.493 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5512.0:450 3.2 3:44e0cffa:::rbd_header.110ef0d7a535:head [watch reconnect cookie 140263298018912 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:50:51.493 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:50:51.493 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 74.923434125, 2024-09-16T20:50:51.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0084581939999999, 2024-09-16T20:50:51.493 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:51.494 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:51.494 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:51.494 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:51.494 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5512", 2024-09-16T20:50:51.494 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:50:51.494 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 450 2024-09-16T20:50:51.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.494 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:51.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.494 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:51.494 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:50:51.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:51.495 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:50:51.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:51.495 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562512+0000", 2024-09-16T20:50:51.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.009e-06 2024-09-16T20:50:51.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:51.495 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562517+0000", 2024-09-16T20:50:51.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9339999999999997e-06 2024-09-16T20:50:51.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:51.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562530+0000", 2024-09-16T20:50:51.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3495e-05 2024-09-16T20:50:51.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:51.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562536+0000", 2024-09-16T20:50:51.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5690000000000004e-06 2024-09-16T20:50:51.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:51.497 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562591+0000", 2024-09-16T20:50:51.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5761999999999999e-05 2024-09-16T20:50:51.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:50:51.497 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562599+0000", 2024-09-16T20:50:51.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9179999999999994e-06 2024-09-16T20:50:51.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:51.497 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.565971+0000", 2024-09-16T20:50:51.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003371546 2024-09-16T20:50:51.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.498 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:51.498 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566019+0000", 2024-09-16T20:50:51.498 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7849000000000002e-05 2024-09-16T20:50:51.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.498 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:51.498 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.569382+0000", 2024-09-16T20:50:51.498 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.003362833 2024-09-16T20:50:51.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.498 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:51.498 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.569529+0000", 2024-09-16T20:50:51.499 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000147823 2024-09-16T20:50:51.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.499 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.499 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:51.499 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570969+0000", 2024-09-16T20:50:51.499 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001439456 2024-09-16T20:50:51.499 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.499 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:51.499 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.499 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.499 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:14996 3.2 3:44e0cffa:::rbd_header.110ef0d7a535:head [watch reconnect cookie 140213436269840 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:50:51.499 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:50:51.499 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 74.923385713000002, 2024-09-16T20:50:51.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.008813943, 2024-09-16T20:50:51.500 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:51.500 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:51.500 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:51.500 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:51.500 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:50:51.500 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:51.500 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 14996 2024-09-16T20:50:51.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.500 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:51.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.500 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:51.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:50:51.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:51.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:50:51.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:51.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562560+0000", 2024-09-16T20:50:51.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.06e-07 2024-09-16T20:50:51.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.502 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:51.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562563+0000", 2024-09-16T20:50:51.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.495e-06 2024-09-16T20:50:51.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.502 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:51.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562571+0000", 2024-09-16T20:50:51.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1950000000000003e-06 2024-09-16T20:50:51.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.502 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:51.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562579+0000", 2024-09-16T20:50:51.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2649999999999999e-06 2024-09-16T20:50:51.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:51.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562673+0000", 2024-09-16T20:50:51.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4118999999999998e-05 2024-09-16T20:50:51.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:50:51.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562677+0000", 2024-09-16T20:50:51.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5179999999999998e-06 2024-09-16T20:50:51.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:51.504 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566461+0000", 2024-09-16T20:50:51.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0037837259999999998 2024-09-16T20:50:51.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.504 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:51.504 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566468+0000", 2024-09-16T20:50:51.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3350000000000001e-06 2024-09-16T20:50:51.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.504 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:51.504 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570306+0000", 2024-09-16T20:50:51.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.003837997 2024-09-16T20:50:51.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:51.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570392+0000", 2024-09-16T20:50:51.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5872999999999996e-05 2024-09-16T20:50:51.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:51.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571373+0000", 2024-09-16T20:50:51.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00098071399999999993 2024-09-16T20:50:51.505 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.505 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:51.505 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.506 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5255.0:1931 3.2 3:47882819:::rbd_data.110eb5f95afc.0000000000000009:head [read 217088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:50:51.506 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:50:51.506 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 74.923346963, 2024-09-16T20:50:51.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0082752880000001, 2024-09-16T20:50:51.506 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:51.506 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:51.506 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:51.506 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:51.506 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5255", 2024-09-16T20:50:51.506 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:50:51.506 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1931 2024-09-16T20:50:51.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.506 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:51.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.507 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:51.507 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:50:51.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.507 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.507 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:51.507 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:50:51.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.507 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.507 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:51.507 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:50:51.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9699999999999996e-07 2024-09-16T20:50:51.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.508 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:51.508 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562601+0000", 2024-09-16T20:50:51.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8600000000000001e-06 2024-09-16T20:50:51.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.508 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:51.508 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562608+0000", 2024-09-16T20:50:51.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9e-06 2024-09-16T20:50:51.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.508 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:51.508 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562615+0000", 2024-09-16T20:50:51.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8179999999999998e-06 2024-09-16T20:50:51.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:51.509 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562703+0000", 2024-09-16T20:50:51.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7811000000000003e-05 2024-09-16T20:50:51.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:50:51.509 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562706+0000", 2024-09-16T20:50:51.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4920000000000002e-06 2024-09-16T20:50:51.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:51.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566685+0000", 2024-09-16T20:50:51.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0039792780000000002 2024-09-16T20:50:51.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:51.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566693+0000", 2024-09-16T20:50:51.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4730000000000003e-06 2024-09-16T20:50:51.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:51.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570737+0000", 2024-09-16T20:50:51.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0040436749999999 2024-09-16T20:50:51.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:51.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570873+0000", 2024-09-16T20:50:51.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000136484 2024-09-16T20:50:51.511 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.511 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:51.511 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.511 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5255.0:1939 3.2 3:47882819:::rbd_data.110eb5f95afc.0000000000000009:head [read 233472~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:50:51.511 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:50:51.511 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 74.923192096999998, 2024-09-16T20:50:51.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0084657589999999, 2024-09-16T20:50:51.511 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:51.512 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:51.512 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:51.512 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:51.512 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5255", 2024-09-16T20:50:51.512 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:50:51.512 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1939 2024-09-16T20:50:51.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.512 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:51.512 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.512 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:51.512 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:50:51.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.513 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:51.513 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:50:51.513 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.513 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:51.513 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:50:51.513 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5199999999999997e-07 2024-09-16T20:50:51.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.513 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:51.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562757+0000", 2024-09-16T20:50:51.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4979999999999998e-06 2024-09-16T20:50:51.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:51.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562764+0000", 2024-09-16T20:50:51.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1470000000000004e-06 2024-09-16T20:50:51.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:51.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562768+0000", 2024-09-16T20:50:51.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7929999999999999e-06 2024-09-16T20:50:51.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.515 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:51.515 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562805+0000", 2024-09-16T20:50:51.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7477e-05 2024-09-16T20:50:51.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.515 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.515 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting_for_map not empty", 2024-09-16T20:50:51.515 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562810+0000", 2024-09-16T20:50:51.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0819999999999998e-06 2024-09-16T20:50:51.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.515 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.515 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:51.515 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566717+0000", 2024-09-16T20:50:51.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0039072600000000001 2024-09-16T20:50:51.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.516 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.516 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:51.516 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566724+0000", 2024-09-16T20:50:51.516 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4880000000000004e-06 2024-09-16T20:50:51.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.516 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.516 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:51.516 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571009+0000", 2024-09-16T20:50:51.516 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.004284677 2024-09-16T20:50:51.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.516 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.516 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:51.516 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571218+0000", 2024-09-16T20:50:51.516 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00020978500000000001 2024-09-16T20:50:51.517 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.517 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:51.517 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.517 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:11020 3.1f 3:fee2d1dd:::rbd_data.110ea93691bb.000000000000000e:23 [read 955392~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:51.517 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.693811+0000", 2024-09-16T20:50:51.517 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 60.792133440000001, 2024-09-16T20:50:51.517 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056833005999999998, 2024-09-16T20:50:51.517 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:51.517 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:51.517 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:51.517 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:51.518 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:51.518 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:51.518 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 11020 2024-09-16T20:50:51.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.518 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:51.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.518 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:51.518 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693811+0000", 2024-09-16T20:50:51.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.518 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:51.518 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693811+0000", 2024-09-16T20:50:51.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.519 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:51.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693812+0000", 2024-09-16T20:50:51.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6100000000000001e-07 2024-09-16T20:50:51.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.519 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:51.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693815+0000", 2024-09-16T20:50:51.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7389999999999999e-06 2024-09-16T20:50:51.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.520 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:51.520 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693823+0000", 2024-09-16T20:50:51.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0439999999999996e-06 2024-09-16T20:50:51.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.520 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:51.520 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693828+0000", 2024-09-16T20:50:51.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7440000000000004e-06 2024-09-16T20:50:51.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.520 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:51.520 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750357+0000", 2024-09-16T20:50:51.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056529192999999998 2024-09-16T20:50:51.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.521 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.521 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:51.521 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750429+0000", 2024-09-16T20:50:51.521 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1557000000000006e-05 2024-09-16T20:50:51.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.521 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.521 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:51.521 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750644+0000", 2024-09-16T20:50:51.521 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00021526800000000001 2024-09-16T20:50:51.521 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.521 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:51.521 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.522 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:11021 3.1f 3:fee2d1dd:::rbd_data.110ea93691bb.000000000000000e:23 [read 960512~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:51.522 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.693837+0000", 2024-09-16T20:50:51.522 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 60.792107551999997, 2024-09-16T20:50:51.522 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057155138000000001, 2024-09-16T20:50:51.522 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:51.522 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:51.522 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:51.522 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:51.522 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:51.522 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:51.522 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 11021 2024-09-16T20:50:51.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.522 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:51.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:51.523 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693837+0000", 2024-09-16T20:50:51.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:51.523 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693837+0000", 2024-09-16T20:50:51.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:51.523 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699566+0000", 2024-09-16T20:50:51.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057283639999999997 2024-09-16T20:50:51.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.524 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:51.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699571+0000", 2024-09-16T20:50:51.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.604e-06 2024-09-16T20:50:51.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.524 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:51.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699583+0000", 2024-09-16T20:50:51.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1372999999999999e-05 2024-09-16T20:50:51.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.524 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:51.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699591+0000", 2024-09-16T20:50:51.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5259999999999993e-06 2024-09-16T20:50:51.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.525 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.525 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:51.525 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750674+0000", 2024-09-16T20:50:51.525 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051083154999999998 2024-09-16T20:50:51.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.525 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.525 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:51.525 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750745+0000", 2024-09-16T20:50:51.525 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0933999999999995e-05 2024-09-16T20:50:51.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.525 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.525 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:51.526 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750992+0000", 2024-09-16T20:50:51.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00024718199999999998 2024-09-16T20:50:51.526 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.526 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:51.526 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.526 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.526 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8790 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2590720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:51.526 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057257+0000", 2024-09-16T20:50:51.526 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 50.428688231999999, 2024-09-16T20:50:51.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056791169000000002, 2024-09-16T20:50:51.526 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:51.526 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:51.526 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:51.527 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:51.527 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:51.527 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:51.527 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8790 2024-09-16T20:50:51.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.527 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:51.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.527 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:51.527 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057257+0000", 2024-09-16T20:50:51.527 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.527 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:51.527 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057257+0000", 2024-09-16T20:50:51.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:51.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057257+0000", 2024-09-16T20:50:51.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6600000000000002e-07 2024-09-16T20:50:51.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:51.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057260+0000", 2024-09-16T20:50:51.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7240000000000001e-06 2024-09-16T20:50:51.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.529 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:51.529 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057267+0000", 2024-09-16T20:50:51.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0600000000000002e-06 2024-09-16T20:50:51.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.529 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:51.529 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057272+0000", 2024-09-16T20:50:51.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0390000000000004e-06 2024-09-16T20:50:51.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.529 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:51.529 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.113950+0000", 2024-09-16T20:50:51.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056677989999999998 2024-09-16T20:50:51.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.530 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:51.530 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.113993+0000", 2024-09-16T20:50:51.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3205999999999999e-05 2024-09-16T20:50:51.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.530 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:51.530 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114048+0000", 2024-09-16T20:50:51.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4684e-05 2024-09-16T20:50:51.530 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.530 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:51.530 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.531 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8791 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2332672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:51.531 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057279+0000", 2024-09-16T20:50:51.531 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 50.428665817000002, 2024-09-16T20:50:51.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056890067000000002, 2024-09-16T20:50:51.531 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:51.531 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:51.531 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:51.531 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:51.531 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:51.531 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:51.531 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8791 2024-09-16T20:50:51.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.532 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:51.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.532 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:51.532 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057279+0000", 2024-09-16T20:50:51.532 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.532 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:51.532 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057279+0000", 2024-09-16T20:50:51.532 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.533 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:51.533 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057279+0000", 2024-09-16T20:50:51.533 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.15e-07 2024-09-16T20:50:51.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.533 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.533 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:51.533 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057281+0000", 2024-09-16T20:50:51.533 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8080000000000001e-06 2024-09-16T20:50:51.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.533 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.533 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:51.533 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057287+0000", 2024-09-16T20:50:51.533 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5999999999999997e-06 2024-09-16T20:50:51.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.534 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:51.534 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057291+0000", 2024-09-16T20:50:51.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5990000000000004e-06 2024-09-16T20:50:51.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.534 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:51.534 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114078+0000", 2024-09-16T20:50:51.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056786246999999998 2024-09-16T20:50:51.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.534 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:51.534 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114118+0000", 2024-09-16T20:50:51.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9898999999999999e-05 2024-09-16T20:50:51.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:51.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114169+0000", 2024-09-16T20:50:51.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1499000000000003e-05 2024-09-16T20:50:51.535 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.535 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:51.535 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.535 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8792 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2609152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:51.535 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:50:51.536 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 50.428647835, 2024-09-16T20:50:51.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057003524999999999, 2024-09-16T20:50:51.536 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:51.536 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:51.536 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:51.536 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:51.536 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:51.536 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:51.536 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8792 2024-09-16T20:50:51.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.536 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:51.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.536 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:51.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:50:51.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:51.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:50:51.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:51.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:50:51.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0400000000000002e-07 2024-09-16T20:50:51.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.538 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:51.538 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057299+0000", 2024-09-16T20:50:51.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7460000000000001e-06 2024-09-16T20:50:51.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.538 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:51.538 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057304+0000", 2024-09-16T20:50:51.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3210000000000004e-06 2024-09-16T20:50:51.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.538 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:51.538 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057310+0000", 2024-09-16T20:50:51.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4360000000000001e-06 2024-09-16T20:50:51.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.539 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:51.539 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114200+0000", 2024-09-16T20:50:51.539 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056889954 2024-09-16T20:50:51.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.539 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:51.539 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114243+0000", 2024-09-16T20:50:51.539 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2998999999999999e-05 2024-09-16T20:50:51.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.539 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:51.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114300+0000", 2024-09-16T20:50:51.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7765e-05 2024-09-16T20:50:51.540 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.540 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:51.540 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.540 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8794 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2351104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:51.540 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057327+0000", 2024-09-16T20:50:51.540 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 50.428617510999999, 2024-09-16T20:50:51.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057101130999999999, 2024-09-16T20:50:51.540 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:51.540 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:51.540 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:51.541 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:51.541 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:51.541 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:51.541 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8794 2024-09-16T20:50:51.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.541 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:51.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.541 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:51.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057327+0000", 2024-09-16T20:50:51.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.541 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:51.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057327+0000", 2024-09-16T20:50:51.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:51.542 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057328+0000", 2024-09-16T20:50:51.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7399999999999999e-07 2024-09-16T20:50:51.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:51.542 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057330+0000", 2024-09-16T20:50:51.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2199999999999999e-06 2024-09-16T20:50:51.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:51.543 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057336+0000", 2024-09-16T20:50:51.543 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1240000000000002e-06 2024-09-16T20:50:51.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.543 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.543 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:51.543 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057341+0000", 2024-09-16T20:50:51.543 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8470000000000003e-06 2024-09-16T20:50:51.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.543 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.543 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:51.543 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114334+0000", 2024-09-16T20:50:51.543 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056993267 2024-09-16T20:50:51.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.544 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:51.544 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114376+0000", 2024-09-16T20:50:51.544 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1638999999999999e-05 2024-09-16T20:50:51.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.544 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:51.544 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114428+0000", 2024-09-16T20:50:51.544 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2660000000000001e-05 2024-09-16T20:50:51.544 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.544 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:51.544 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.545 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8798 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2297856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:51.545 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:50:51.545 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 50.428597783999997, 2024-09-16T20:50:51.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057277041000000001, 2024-09-16T20:50:51.545 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:51.545 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:51.545 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:51.545 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:51.545 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:51.545 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:51.545 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8798 2024-09-16T20:50:51.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.545 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:51.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:51.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:50:51.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:51.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:50:51.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:51.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:50:51.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2800000000000003e-07 2024-09-16T20:50:51.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.547 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:51.547 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057354+0000", 2024-09-16T20:50:51.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7780000000000002e-06 2024-09-16T20:50:51.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.547 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:51.547 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057360+0000", 2024-09-16T20:50:51.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0419999999999999e-06 2024-09-16T20:50:51.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.547 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:51.548 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057365+0000", 2024-09-16T20:50:51.548 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7929999999999997e-06 2024-09-16T20:50:51.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.548 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.548 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:51.548 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114460+0000", 2024-09-16T20:50:51.548 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057094662999999997 2024-09-16T20:50:51.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.548 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.548 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:51.548 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114532+0000", 2024-09-16T20:50:51.548 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2881999999999998e-05 2024-09-16T20:50:51.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.549 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:51.549 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114624+0000", 2024-09-16T20:50:51.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1555e-05 2024-09-16T20:50:51.549 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.549 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:51.549 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.549 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8800 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2316288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:51.549 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:50:51.549 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 50.428573845000003, 2024-09-16T20:50:51.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.05746946, 2024-09-16T20:50:51.550 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:51.550 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:51.550 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:51.550 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:51.550 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:51.550 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:51.550 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8800 2024-09-16T20:50:51.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.550 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:51.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.550 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:51.550 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:50:51.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.551 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:51.551 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:50:51.551 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.551 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:51.551 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:50:51.551 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5199999999999998e-07 2024-09-16T20:50:51.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.551 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:51.552 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057373+0000", 2024-09-16T20:50:51.552 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.79e-06 2024-09-16T20:50:51.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.552 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.552 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:51.552 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057379+0000", 2024-09-16T20:50:51.552 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6699999999999999e-06 2024-09-16T20:50:51.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.552 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.552 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:51.552 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057386+0000", 2024-09-16T20:50:51.552 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3170000000000004e-06 2024-09-16T20:50:51.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.553 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:51.553 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114655+0000", 2024-09-16T20:50:51.553 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057269420000000001 2024-09-16T20:50:51.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.553 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:51.553 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114734+0000", 2024-09-16T20:50:51.553 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8084000000000007e-05 2024-09-16T20:50:51.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.553 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:51.554 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114840+0000", 2024-09-16T20:50:51.554 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000106827 2024-09-16T20:50:51.554 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.554 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:51.554 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.554 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.554 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8801 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2334720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:51.554 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:50:51.554 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 50.428552814, 2024-09-16T20:50:51.554 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057594682000000001, 2024-09-16T20:50:51.554 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:51.554 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:51.554 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:51.555 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:51.555 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:51.555 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:51.555 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8801 2024-09-16T20:50:51.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.555 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:51.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:51.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:50:51.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:51.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:50:51.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.556 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.556 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.556 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:51.556 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:50:51.556 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6899999999999998e-07 2024-09-16T20:50:51.556 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.556 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.556 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:51.556 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057394+0000", 2024-09-16T20:50:51.556 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8300000000000001e-06 2024-09-16T20:50:51.556 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.556 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.556 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:51.557 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057400+0000", 2024-09-16T20:50:51.557 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6500000000000001e-06 2024-09-16T20:50:51.557 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.557 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.557 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:51.557 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057404+0000", 2024-09-16T20:50:51.557 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4669999999999998e-06 2024-09-16T20:50:51.557 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.557 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.557 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:51.557 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114874+0000", 2024-09-16T20:50:51.557 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057469602000000002 2024-09-16T20:50:51.557 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.557 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.558 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:51.558 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114933+0000", 2024-09-16T20:50:51.558 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9500000000000003e-05 2024-09-16T20:50:51.558 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.558 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.558 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:51.558 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114987+0000", 2024-09-16T20:50:51.558 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3263999999999997e-05 2024-09-16T20:50:51.558 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.558 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:51.558 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.558 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.558 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.558 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8802 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2353152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:51.558 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:50:51.559 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 50.428504128999997, 2024-09-16T20:50:51.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057714079000000001, 2024-09-16T20:50:51.559 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:51.559 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:51.559 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:51.559 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:51.559 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:51.559 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:51.559 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8802 2024-09-16T20:50:51.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.559 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:51.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:51.559 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:50:51.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.560 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:51.560 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:50:51.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.560 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:51.560 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:50:51.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4999999999999998e-07 2024-09-16T20:50:51.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.560 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:51.561 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057446+0000", 2024-09-16T20:50:51.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4240000000000003e-06 2024-09-16T20:50:51.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.561 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.561 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:51.561 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057452+0000", 2024-09-16T20:50:51.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8379999999999996e-06 2024-09-16T20:50:51.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.561 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.561 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:51.561 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057460+0000", 2024-09-16T20:50:51.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9629999999999998e-06 2024-09-16T20:50:51.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.562 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:51.562 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115015+0000", 2024-09-16T20:50:51.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057554389999999997 2024-09-16T20:50:51.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.562 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:51.562 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115082+0000", 2024-09-16T20:50:51.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7733999999999999e-05 2024-09-16T20:50:51.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.562 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:51.562 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115155+0000", 2024-09-16T20:50:51.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2280000000000006e-05 2024-09-16T20:50:51.563 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.563 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:51.563 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.563 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8804 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2555904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:51.563 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:50:51.563 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 50.428477813000001, 2024-09-16T20:50:51.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057833862, 2024-09-16T20:50:51.563 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:51.563 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:51.563 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:51.563 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:51.563 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:51.564 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:51.564 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8804 2024-09-16T20:50:51.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.564 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:51.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:51.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:50:51.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:51.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:50:51.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.565 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:51.565 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:50:51.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7899999999999999e-07 2024-09-16T20:50:51.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.565 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:51.565 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057469+0000", 2024-09-16T20:50:51.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.001e-06 2024-09-16T20:50:51.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.565 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:51.566 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057475+0000", 2024-09-16T20:50:51.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7629999999999997e-06 2024-09-16T20:50:51.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.566 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.566 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:51.566 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057480+0000", 2024-09-16T20:50:51.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8069999999999999e-06 2024-09-16T20:50:51.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.566 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.566 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:51.566 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115183+0000", 2024-09-16T20:50:51.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057703364 2024-09-16T20:50:51.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.566 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.566 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:51.567 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115227+0000", 2024-09-16T20:50:51.567 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3903999999999998e-05 2024-09-16T20:50:51.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.567 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:51.567 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115301+0000", 2024-09-16T20:50:51.567 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3543999999999999e-05 2024-09-16T20:50:51.567 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.567 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:51.567 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.567 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8805 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2574336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:51.567 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057487+0000", 2024-09-16T20:50:51.568 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 50.428457485000003, 2024-09-16T20:50:51.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058021027000000003, 2024-09-16T20:50:51.568 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:51.568 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:51.568 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:51.568 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:51.568 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:51.568 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:51.568 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8805 2024-09-16T20:50:51.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.568 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:51.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.569 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:51.569 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057487+0000", 2024-09-16T20:50:51.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.569 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:51.569 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057487+0000", 2024-09-16T20:50:51.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.569 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:51.569 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057488+0000", 2024-09-16T20:50:51.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5199999999999998e-07 2024-09-16T20:50:51.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.570 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:51.570 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057490+0000", 2024-09-16T20:50:51.570 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.939e-06 2024-09-16T20:50:51.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.570 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:51.570 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057495+0000", 2024-09-16T20:50:51.570 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6439999999999997e-06 2024-09-16T20:50:51.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.570 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:51.571 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057500+0000", 2024-09-16T20:50:51.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8119999999999998e-06 2024-09-16T20:50:51.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.571 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.571 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:51.571 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115341+0000", 2024-09-16T20:50:51.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057841341999999997 2024-09-16T20:50:51.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.571 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.571 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:51.571 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115422+0000", 2024-09-16T20:50:51.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1003999999999999e-05 2024-09-16T20:50:51.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.572 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:51.572 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115508+0000", 2024-09-16T20:50:51.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5933999999999994e-05 2024-09-16T20:50:51.572 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.572 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:51.572 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.572 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8807 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2592768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:51.572 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057505+0000", 2024-09-16T20:50:51.572 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 50.428439275000002, 2024-09-16T20:50:51.573 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058186633000000001, 2024-09-16T20:50:51.573 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:51.573 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:51.573 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:51.573 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:51.573 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:51.573 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:51.573 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8807 2024-09-16T20:50:51.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.573 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:51.573 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.573 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:51.573 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057505+0000", 2024-09-16T20:50:51.573 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.574 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:51.574 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057505+0000", 2024-09-16T20:50:51.574 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.574 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:51.574 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057506+0000", 2024-09-16T20:50:51.574 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.27e-07 2024-09-16T20:50:51.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.574 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:51.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057508+0000", 2024-09-16T20:50:51.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7400000000000001e-06 2024-09-16T20:50:51.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.575 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:51.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057513+0000", 2024-09-16T20:50:51.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.468e-06 2024-09-16T20:50:51.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.575 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:51.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057517+0000", 2024-09-16T20:50:51.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4329999999999997e-06 2024-09-16T20:50:51.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:51.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115538+0000", 2024-09-16T20:50:51.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058020355000000003 2024-09-16T20:50:51.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:51.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115616+0000", 2024-09-16T20:50:51.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7998999999999996e-05 2024-09-16T20:50:51.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:51.577 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115692+0000", 2024-09-16T20:50:51.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6310999999999995e-05 2024-09-16T20:50:51.577 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.577 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:51.577 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.577 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8808 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2611200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:51.577 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:50:51.577 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 50.428422013999999, 2024-09-16T20:50:51.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058354046, 2024-09-16T20:50:51.577 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:51.577 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:51.578 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:51.578 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:51.578 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:51.578 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:51.578 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8808 2024-09-16T20:50:51.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.578 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:51.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.578 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:51.578 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:50:51.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:51.579 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:50:51.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:51.579 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:50:51.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2500000000000001e-07 2024-09-16T20:50:51.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:51.579 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057525+0000", 2024-09-16T20:50:51.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5549999999999999e-06 2024-09-16T20:50:51.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:51.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057530+0000", 2024-09-16T20:50:51.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5489999999999997e-06 2024-09-16T20:50:51.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:51.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057535+0000", 2024-09-16T20:50:51.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5709999999999999e-06 2024-09-16T20:50:51.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:51.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115722+0000", 2024-09-16T20:50:51.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058187323999999999 2024-09-16T20:50:51.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:51.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115799+0000", 2024-09-16T20:50:51.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6445000000000004e-05 2024-09-16T20:50:51.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:51.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115877+0000", 2024-09-16T20:50:51.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8276999999999996e-05 2024-09-16T20:50:51.581 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.582 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:51.582 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.582 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:2182 3.17 3:edb5e5cd:::rbd_data.110e61479aee.0000000000000014:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write 0~5120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-16T20:50:51.582 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:22.734609+0000", 2024-09-16T20:50:51.582 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 28.751335767, 2024-09-16T20:50:51.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.124890894, 2024-09-16T20:50:51.582 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:51.582 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:51.582 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:51.582 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:51.582 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:51.583 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:51.583 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 2182 2024-09-16T20:50:51.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.583 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:51.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:51.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734609+0000", 2024-09-16T20:50:51.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:51.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734609+0000", 2024-09-16T20:50:51.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:51.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.584 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.584 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:51.584 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734611+0000", 2024-09-16T20:50:51.584 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9599999999999999e-06 2024-09-16T20:50:51.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.584 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.584 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:51.584 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734628+0000", 2024-09-16T20:50:51.584 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.715e-05 2024-09-16T20:50:51.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.584 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.584 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:51.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734656+0000", 2024-09-16T20:50:51.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7759999999999998e-05 2024-09-16T20:50:51.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:51.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734669+0000", 2024-09-16T20:50:51.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3315999999999999e-05 2024-09-16T20:50:51.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:51.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734826+0000", 2024-09-16T20:50:51.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015668499999999999 2024-09-16T20:50:51.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:51.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.735048+0000", 2024-09-16T20:50:51.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00022179 2024-09-16T20:50:51.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:51.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.735248+0000", 2024-09-16T20:50:51.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00020010399999999999 2024-09-16T20:50:51.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:50:51.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.736252+0000", 2024-09-16T20:50:51.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010039739999999999 2024-09-16T20:50:51.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:51.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.859340+0000", 2024-09-16T20:50:51.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.12308846599999999 2024-09-16T20:50:51.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:50:51.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.859431+0000", 2024-09-16T20:50:51.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1099000000000003e-05 2024-09-16T20:50:51.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:51.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:51.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:51.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.859500+0000", 2024-09-16T20:50:51.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8590000000000006e-05 2024-09-16T20:50:51.588 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.588 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:51.588 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.588 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:51.588 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:51.588 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:51.590 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:50:51.590 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:51.588+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:51.592 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T20:50:51.690 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:51.689+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:51.736 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:51.736 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:50:51.736 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:50:51.736 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:51.746 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:50:51.792 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:51.791+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:51.893 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:51.892+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:51.893 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:51.894 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:50:51.894 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:50:51.894 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:50:51.894 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:51.902 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:50:51.994 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:51.993+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:52.011 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:52.011 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:50:52.011 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:50:52.011 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:50:52.011 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.011 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5439 3.a 3:5789f5c3:::rbd_data.110ed9ca540.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 1181696~60416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:52.011 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:50:52.012 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 149.14464883700001, 2024-09-16T20:50:52.012 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10102045599999999, 2024-09-16T20:50:52.012 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:52.012 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:52.012 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:52.012 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:52.012 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:52.012 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:52.012 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5439 2024-09-16T20:50:52.012 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.012 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:52.012 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.013 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:52.013 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:50:52.013 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:52.013 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.013 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.013 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:52.013 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:50:52.013 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:52.013 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.013 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.013 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:52.014 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864711+0000", 2024-09-16T20:50:52.014 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7899999999999997e-07 2024-09-16T20:50:52.014 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.014 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.014 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:52.014 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864760+0000", 2024-09-16T20:50:52.014 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8803e-05 2024-09-16T20:50:52.014 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.014 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.014 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:52.014 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864769+0000", 2024-09-16T20:50:52.014 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0999999999999993e-06 2024-09-16T20:50:52.015 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.015 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.015 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:52.015 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864774+0000", 2024-09-16T20:50:52.015 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8919999999999999e-06 2024-09-16T20:50:52.015 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.015 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.015 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:52.015 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864815+0000", 2024-09-16T20:50:52.015 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1307000000000003e-05 2024-09-16T20:50:52.015 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.016 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.016 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:52.016 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864893+0000", 2024-09-16T20:50:52.016 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7581000000000001e-05 2024-09-16T20:50:52.016 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.016 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.016 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 1", 2024-09-16T20:50:52.016 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864996+0000", 2024-09-16T20:50:52.016 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102725 2024-09-16T20:50:52.016 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.017 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.017 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:52.017 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965442+0000", 2024-09-16T20:50:52.017 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100446012 2024-09-16T20:50:52.017 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.017 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.017 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:52.017 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965618+0000", 2024-09-16T20:50:52.017 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00017653100000000001 2024-09-16T20:50:52.017 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.017 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.017 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:52.018 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965682+0000", 2024-09-16T20:50:52.018 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3905999999999994e-05 2024-09-16T20:50:52.018 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.018 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.018 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:52.018 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965731+0000", 2024-09-16T20:50:52.018 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8819999999999997e-05 2024-09-16T20:50:52.018 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:52.018 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:52.018 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:52.018 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.019 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.019 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4600.0:5439 3.a e37/23 ondisk, result = 0)", 2024-09-16T20:50:52.019 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:50:52.019 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 149.14365268899999, 2024-09-16T20:50:52.019 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099742634999999996, 2024-09-16T20:50:52.019 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:52.019 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:52.019 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:52.019 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:52.019 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.019 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:52.020 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:50:52.020 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:52.020 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.020 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.020 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:52.020 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:50:52.020 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:52.020 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.020 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.020 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:52.020 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:50:52.021 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0900000000000001e-07 2024-09-16T20:50:52.021 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.021 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.021 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:52.021 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865711+0000", 2024-09-16T20:50:52.021 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3739999999999998e-06 2024-09-16T20:50:52.021 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.021 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.021 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:52.021 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965343+0000", 2024-09-16T20:50:52.021 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099632526999999999 2024-09-16T20:50:52.021 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.022 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.022 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:52.022 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965357+0000", 2024-09-16T20:50:52.022 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3713999999999999e-05 2024-09-16T20:50:52.022 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.022 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.022 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:52.022 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965426+0000", 2024-09-16T20:50:52.022 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8745999999999995e-05 2024-09-16T20:50:52.022 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.022 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.023 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:52.023 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965434+0000", 2024-09-16T20:50:52.023 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6910000000000003e-06 2024-09-16T20:50:52.023 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.023 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.023 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:52.023 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965449+0000", 2024-09-16T20:50:52.023 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4874e-05 2024-09-16T20:50:52.023 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:52.023 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:52.023 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:52.023 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.024 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.024 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4882.0:2508 3.12 3:4992771f:::rbd_object_map.110e7c8851c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T20:50:52.024 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:50:52.024 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 125.527849141, 2024-09-16T20:50:52.024 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.125007011, 2024-09-16T20:50:52.024 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:52.024 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:52.024 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:52.024 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:52.024 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4882", 2024-09-16T20:50:52.025 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2436923223", 2024-09-16T20:50:52.025 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2508 2024-09-16T20:50:52.025 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.025 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:52.025 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.025 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:52.025 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:50:52.025 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:52.025 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.025 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.025 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:52.025 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:50:52.025 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:52.026 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.026 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.026 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:52.026 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481511+0000", 2024-09-16T20:50:52.026 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0699999999999996e-07 2024-09-16T20:50:52.026 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.026 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.026 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:52.026 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481515+0000", 2024-09-16T20:50:52.026 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0609999999999997e-06 2024-09-16T20:50:52.026 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.026 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.027 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:52.027 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481523+0000", 2024-09-16T20:50:52.027 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1310000000000006e-06 2024-09-16T20:50:52.027 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.027 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.027 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:52.027 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481528+0000", 2024-09-16T20:50:52.027 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4900000000000002e-06 2024-09-16T20:50:52.027 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.027 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.027 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:52.027 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481565+0000", 2024-09-16T20:50:52.027 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7066000000000001e-05 2024-09-16T20:50:52.028 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.028 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.028 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:52.028 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481628+0000", 2024-09-16T20:50:52.028 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3118000000000001e-05 2024-09-16T20:50:52.028 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.028 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.028 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 2", 2024-09-16T20:50:52.028 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481822+0000", 2024-09-16T20:50:52.028 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019446200000000001 2024-09-16T20:50:52.029 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.029 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.029 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:52.029 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605988+0000", 2024-09-16T20:50:52.029 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.124165888 2024-09-16T20:50:52.029 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.029 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.029 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:52.029 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606427+0000", 2024-09-16T20:50:52.029 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00043854899999999999 2024-09-16T20:50:52.029 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.029 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.030 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:52.030 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606476+0000", 2024-09-16T20:50:52.030 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8971999999999998e-05 2024-09-16T20:50:52.030 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.030 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.030 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:52.030 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606517+0000", 2024-09-16T20:50:52.030 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1467000000000001e-05 2024-09-16T20:50:52.030 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:52.030 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:52.030 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:52.030 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.031 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.031 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4882.0:2508 3.12 e57/55 ondisk, result = 0)", 2024-09-16T20:50:52.031 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:50:52.031 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 125.526860641, 2024-09-16T20:50:52.031 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12349711400000001, 2024-09-16T20:50:52.031 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:52.031 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:52.031 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:52.031 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:52.031 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.031 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:52.031 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:50:52.032 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:52.032 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.032 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.032 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:52.032 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:50:52.032 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:52.032 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.032 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.032 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:52.032 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:50:52.032 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.06e-07 2024-09-16T20:50:52.033 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.033 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.033 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:52.033 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482502+0000", 2024-09-16T20:50:52.033 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.01e-06 2024-09-16T20:50:52.033 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.033 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.033 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:52.033 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605772+0000", 2024-09-16T20:50:52.033 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.123269687 2024-09-16T20:50:52.033 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.033 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.033 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:52.034 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605791+0000", 2024-09-16T20:50:52.034 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8912000000000001e-05 2024-09-16T20:50:52.034 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.034 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.034 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:52.034 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605974+0000", 2024-09-16T20:50:52.034 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000182836 2024-09-16T20:50:52.034 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.034 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.034 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:52.034 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605982+0000", 2024-09-16T20:50:52.034 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1699999999999997e-06 2024-09-16T20:50:52.035 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.035 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.035 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:52.035 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605996+0000", 2024-09-16T20:50:52.035 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3893000000000001e-05 2024-09-16T20:50:52.035 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:52.035 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:52.035 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:52.035 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.035 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.035 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4882.0:2512 3.f 3:f4d895b7:::rbd_data.110e7c8851c.0000000000000000:head [read 16384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T20:50:52.036 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:50:52.036 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 125.526423611, 2024-09-16T20:50:52.036 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12318483, 2024-09-16T20:50:52.036 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:52.036 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:52.036 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:52.036 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:52.036 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4882", 2024-09-16T20:50:52.036 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2436923223", 2024-09-16T20:50:52.036 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2512 2024-09-16T20:50:52.037 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.037 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:52.037 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.037 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:52.037 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:50:52.037 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:52.037 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.037 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.039 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:52.040 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:50:52.040 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:52.040 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.040 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.040 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:52.040 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:50:52.040 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3499999999999995e-07 2024-09-16T20:50:52.040 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.040 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.040 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:52.041 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482943+0000", 2024-09-16T20:50:52.041 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3450000000000002e-06 2024-09-16T20:50:52.041 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.041 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.041 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:52.041 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605772+0000", 2024-09-16T20:50:52.041 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.122829796 2024-09-16T20:50:52.041 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.041 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.041 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:52.041 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605792+0000", 2024-09-16T20:50:52.042 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9916999999999999e-05 2024-09-16T20:50:52.042 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.042 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.042 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:52.042 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605917+0000", 2024-09-16T20:50:52.042 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00012500799999999999 2024-09-16T20:50:52.042 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.042 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.042 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:52.042 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606004+0000", 2024-09-16T20:50:52.042 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6242999999999994e-05 2024-09-16T20:50:52.042 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.043 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.043 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:52.043 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606120+0000", 2024-09-16T20:50:52.043 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000116786 2024-09-16T20:50:52.043 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:52.043 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:52.043 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:52.043 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.043 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.043 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:11702 3.15 3:acf33c8b:::rbd_data.110e747f0779.000000000000001a:head [set-alloc-hint object_size 262144 write_size 262144,write 143360~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:50:52.043 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:50:52.043 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 107.87143118900001, 2024-09-16T20:50:52.044 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.084801043000000006, 2024-09-16T20:50:52.044 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:52.044 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:52.044 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:52.044 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:52.044 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:50:52.044 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:50:52.044 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 11702 2024-09-16T20:50:52.044 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.044 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:52.044 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.045 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:52.045 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:50:52.045 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:52.045 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.045 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.045 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:52.045 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:50:52.045 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:52.045 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.045 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.045 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:52.045 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137929+0000", 2024-09-16T20:50:52.046 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8400000000000002e-07 2024-09-16T20:50:52.046 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.046 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.046 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:52.046 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137937+0000", 2024-09-16T20:50:52.046 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0260000000000007e-06 2024-09-16T20:50:52.046 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.046 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.046 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:52.046 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137952+0000", 2024-09-16T20:50:52.046 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4659e-05 2024-09-16T20:50:52.046 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.047 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.047 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:52.047 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137960+0000", 2024-09-16T20:50:52.047 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0169999999999996e-06 2024-09-16T20:50:52.047 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.047 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.047 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:52.047 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138010+0000", 2024-09-16T20:50:52.047 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0021000000000001e-05 2024-09-16T20:50:52.047 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.047 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.048 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:52.048 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138114+0000", 2024-09-16T20:50:52.048 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000103797 2024-09-16T20:50:52.048 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.048 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.048 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:50:52.048 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138233+0000", 2024-09-16T20:50:52.048 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000118971 2024-09-16T20:50:52.048 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.048 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.049 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:52.049 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138953+0000", 2024-09-16T20:50:52.049 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00072026000000000002 2024-09-16T20:50:52.049 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.049 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.049 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:52.049 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222609+0000", 2024-09-16T20:50:52.049 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.083656650999999999 2024-09-16T20:50:52.049 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.049 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.049 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:52.049 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222684+0000", 2024-09-16T20:50:52.050 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4369999999999994e-05 2024-09-16T20:50:52.050 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.050 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.050 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:52.050 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222729+0000", 2024-09-16T20:50:52.050 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5287000000000002e-05 2024-09-16T20:50:52.050 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:52.050 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:52.050 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:52.050 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.050 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.050 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:11704 3.9 3:91220b75:::rbd_data.110e747f0779.000000000000001c:head [set-alloc-hint object_size 262144 write_size 262144,write 139264~6656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:50:52.051 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:50:52.051 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 107.87119318800001, 2024-09-16T20:50:52.051 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.084531133999999994, 2024-09-16T20:50:52.051 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:52.051 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:52.051 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:52.051 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:52.051 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:50:52.051 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:50:52.051 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 11704 2024-09-16T20:50:52.051 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.051 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:52.051 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.052 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:52.052 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:50:52.052 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:52.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.052 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.052 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:52.052 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:50:52.052 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:52.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.052 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.052 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:52.052 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138167+0000", 2024-09-16T20:50:52.052 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.33e-07 2024-09-16T20:50:52.053 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.053 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.053 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:52.053 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138179+0000", 2024-09-16T20:50:52.053 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2055999999999999e-05 2024-09-16T20:50:52.053 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.053 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.053 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:52.053 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138193+0000", 2024-09-16T20:50:52.053 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3961000000000001e-05 2024-09-16T20:50:52.053 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.053 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.054 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:52.054 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138200+0000", 2024-09-16T20:50:52.054 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4429999999999997e-06 2024-09-16T20:50:52.054 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.054 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.054 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:52.054 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138258+0000", 2024-09-16T20:50:52.054 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7225999999999997e-05 2024-09-16T20:50:52.054 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.054 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.054 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:52.054 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138364+0000", 2024-09-16T20:50:52.055 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000106013 2024-09-16T20:50:52.055 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.055 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.055 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:50:52.055 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138494+0000", 2024-09-16T20:50:52.055 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013059599999999999 2024-09-16T20:50:52.055 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.055 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.055 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:52.055 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.139220+0000", 2024-09-16T20:50:52.055 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00072539800000000004 2024-09-16T20:50:52.055 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.056 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.056 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:52.056 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222608+0000", 2024-09-16T20:50:52.056 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.083388620999999996 2024-09-16T20:50:52.056 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.056 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.056 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:52.056 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222651+0000", 2024-09-16T20:50:52.056 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2228000000000003e-05 2024-09-16T20:50:52.056 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.056 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.056 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:52.057 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222697+0000", 2024-09-16T20:50:52.057 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6659000000000002e-05 2024-09-16T20:50:52.057 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:52.057 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:52.057 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:52.057 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.057 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.057 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5138.0:9358 3.c 3.c25699cc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T20:50:52.057 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:50:52.057 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 85.159045883000005, 2024-09-16T20:50:52.057 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0018657580000001, 2024-09-16T20:50:52.058 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:52.058 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:52.058 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:50:52.058 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:52.058 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5138", 2024-09-16T20:50:52.058 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:50:52.058 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 9358 2024-09-16T20:50:52.058 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.058 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:52.058 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.058 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:52.058 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:50:52.059 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:52.059 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.059 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.059 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:52.059 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:50:52.059 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:52.059 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.059 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.059 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:52.059 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850314+0000", 2024-09-16T20:50:52.059 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4099999999999999e-07 2024-09-16T20:50:52.059 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.060 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.060 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:52.060 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850316+0000", 2024-09-16T20:50:52.060 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2469999999999998e-06 2024-09-16T20:50:52.060 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.060 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.060 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:52.060 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850322+0000", 2024-09-16T20:50:52.060 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3470000000000004e-06 2024-09-16T20:50:52.060 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.060 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.060 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:52.061 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850327+0000", 2024-09-16T20:50:52.061 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9450000000000001e-06 2024-09-16T20:50:52.061 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.061 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.061 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:52.061 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850376+0000", 2024-09-16T20:50:52.061 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8257000000000002e-05 2024-09-16T20:50:52.061 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.061 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.061 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:52.061 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850385+0000", 2024-09-16T20:50:52.062 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9800000000000004e-06 2024-09-16T20:50:52.062 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.062 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.062 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:52.062 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852167+0000", 2024-09-16T20:50:52.062 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0017825659999999 2024-09-16T20:50:52.062 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.062 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.062 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:52.062 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852179+0000", 2024-09-16T20:50:52.062 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1975e-05 2024-09-16T20:50:52.062 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:52.063 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:52.063 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:52.063 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.063 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.063 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5138.0:9382 3.c 3.c25699cc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T20:50:52.063 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:50:52.063 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 85.150698867000003, 2024-09-16T20:50:52.063 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.99358404700000003, 2024-09-16T20:50:52.063 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:52.063 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:52.063 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:50:52.063 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:52.063 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5138", 2024-09-16T20:50:52.064 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:50:52.064 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 9382 2024-09-16T20:50:52.064 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.064 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:52.064 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.064 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:52.064 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:50:52.064 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:52.064 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.064 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.064 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:52.064 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:50:52.065 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:52.065 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.065 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.065 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:52.065 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858661+0000", 2024-09-16T20:50:52.065 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0200000000000002e-07 2024-09-16T20:50:52.065 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.065 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.065 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:52.065 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858664+0000", 2024-09-16T20:50:52.065 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.038e-06 2024-09-16T20:50:52.065 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.066 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.066 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:52.066 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858671+0000", 2024-09-16T20:50:52.066 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0110000000000003e-06 2024-09-16T20:50:52.066 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.066 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.066 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:52.066 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858677+0000", 2024-09-16T20:50:52.066 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0959999999999997e-06 2024-09-16T20:50:52.066 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.067 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.067 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:52.067 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858732+0000", 2024-09-16T20:50:52.067 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4543000000000003e-05 2024-09-16T20:50:52.067 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.067 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.067 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:52.067 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858739+0000", 2024-09-16T20:50:52.067 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5429999999999996e-06 2024-09-16T20:50:52.067 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.067 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.068 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:52.068 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852236+0000", 2024-09-16T20:50:52.068 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.993496453 2024-09-16T20:50:52.068 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.068 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.068 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:52.068 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852244+0000", 2024-09-16T20:50:52.068 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8610000000000004e-06 2024-09-16T20:50:52.068 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:52.068 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:52.068 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:52.068 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.069 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.069 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5763.0:4895 3.a 3:5384606c:::rbd_data.110e1b19a241.0000000000000018:head [read 0~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-16T20:50:52.069 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:45.598212+0000", 2024-09-16T20:50:52.069 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 66.411146826000007, 2024-09-16T20:50:52.069 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0375864829999999, 2024-09-16T20:50:52.069 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:52.069 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:52.069 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:52.069 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:52.069 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5763", 2024-09-16T20:50:52.069 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/623897567", 2024-09-16T20:50:52.070 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4895 2024-09-16T20:50:52.070 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.070 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:52.070 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.070 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:52.070 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598212+0000", 2024-09-16T20:50:52.070 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:52.070 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.070 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.070 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:52.070 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598212+0000", 2024-09-16T20:50:52.070 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:52.071 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.071 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.071 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:52.071 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598214+0000", 2024-09-16T20:50:52.071 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0090000000000002e-06 2024-09-16T20:50:52.071 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.071 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.071 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:52.071 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598222+0000", 2024-09-16T20:50:52.071 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6699999999999994e-06 2024-09-16T20:50:52.071 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.072 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.072 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:52.072 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598241+0000", 2024-09-16T20:50:52.072 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8896999999999999e-05 2024-09-16T20:50:52.072 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.072 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.072 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:52.072 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598250+0000", 2024-09-16T20:50:52.072 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6649999999999992e-06 2024-09-16T20:50:52.072 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.072 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.073 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:52.073 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598325+0000", 2024-09-16T20:50:52.073 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5189000000000001e-05 2024-09-16T20:50:52.073 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.073 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.073 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op must wait for map", 2024-09-16T20:50:52.073 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598333+0000", 2024-09-16T20:50:52.073 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.3170000000000002e-06 2024-09-16T20:50:52.073 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.073 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.073 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:52.073 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601355+0000", 2024-09-16T20:50:52.074 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0030215400000000001 2024-09-16T20:50:52.074 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.074 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.074 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:52.074 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601365+0000", 2024-09-16T20:50:52.074 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0215e-05 2024-09-16T20:50:52.074 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.074 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.074 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:52.074 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635506+0000", 2024-09-16T20:50:52.074 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0341408240000001 2024-09-16T20:50:52.075 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.075 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.075 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:52.075 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635655+0000", 2024-09-16T20:50:52.075 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000149412 2024-09-16T20:50:52.075 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.075 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.075 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:52.075 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635799+0000", 2024-09-16T20:50:52.075 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000143745 2024-09-16T20:50:52.075 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:52.075 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:52.076 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:52.076 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.076 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.076 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5763.0:4900 3.a 3:5384606c:::rbd_data.110e1b19a241.0000000000000018:head [read 8192~24576] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-16T20:50:52.076 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:50:52.076 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 66.41108835, 2024-09-16T20:50:52.076 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0379340800000001, 2024-09-16T20:50:52.076 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:52.076 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:52.076 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:52.076 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:52.076 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5763", 2024-09-16T20:50:52.077 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/623897567", 2024-09-16T20:50:52.077 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4900 2024-09-16T20:50:52.077 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.077 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:52.077 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.077 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:52.077 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:50:52.077 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:52.077 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.077 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.077 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:52.077 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:50:52.078 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:52.078 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.078 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.078 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:52.078 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:50:52.078 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4300000000000003e-07 2024-09-16T20:50:52.078 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.078 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.078 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:52.078 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598274+0000", 2024-09-16T20:50:52.078 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.976e-06 2024-09-16T20:50:52.078 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.079 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.079 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:52.079 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598282+0000", 2024-09-16T20:50:52.079 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0029999999999994e-06 2024-09-16T20:50:52.079 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.079 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.079 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:52.079 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598289+0000", 2024-09-16T20:50:52.079 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2389999999999999e-06 2024-09-16T20:50:52.079 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.079 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.079 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:52.080 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598430+0000", 2024-09-16T20:50:52.080 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014119600000000001 2024-09-16T20:50:52.080 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.080 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.080 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting_for_map not empty", 2024-09-16T20:50:52.080 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598435+0000", 2024-09-16T20:50:52.080 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3489999999999999e-06 2024-09-16T20:50:52.080 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.080 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.080 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:52.081 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601429+0000", 2024-09-16T20:50:52.081 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0029942720000000001 2024-09-16T20:50:52.081 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.081 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.081 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:52.081 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601438+0000", 2024-09-16T20:50:52.081 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4759999999999998e-06 2024-09-16T20:50:52.081 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.081 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.081 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:52.081 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635987+0000", 2024-09-16T20:50:52.082 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0345495010000001 2024-09-16T20:50:52.082 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.082 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.082 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:52.082 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.636056+0000", 2024-09-16T20:50:52.082 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8610999999999998e-05 2024-09-16T20:50:52.082 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.082 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.082 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:52.082 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.636205+0000", 2024-09-16T20:50:52.082 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014891399999999999 2024-09-16T20:50:52.083 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:52.083 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:52.083 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:52.083 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.083 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.083 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop(client.6231.0:2182 3.17 e136/81 3:edb5e5cd:::rbd_data.110e61479aee.0000000000000014:head v 136'791, mlcod=135'789)", 2024-09-16T20:50:52.083 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:22.735499+0000", 2024-09-16T20:50:52.083 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 29.273860409000001, 2024-09-16T20:50:52.083 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12369608, 2024-09-16T20:50:52.083 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:52.083 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:52.083 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:52.084 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:52.084 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.084 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:52.084 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735499+0000", 2024-09-16T20:50:52.084 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:52.084 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.084 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.084 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:52.084 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735499+0000", 2024-09-16T20:50:52.084 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:52.084 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.084 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.085 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:52.085 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735499+0000", 2024-09-16T20:50:52.085 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8099999999999999e-07 2024-09-16T20:50:52.085 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.085 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.085 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:52.085 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735506+0000", 2024-09-16T20:50:52.085 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3579999999999997e-06 2024-09-16T20:50:52.085 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.085 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.085 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:52.085 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735517+0000", 2024-09-16T20:50:52.085 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0869999999999999e-05 2024-09-16T20:50:52.086 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.086 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.086 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:52.086 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735533+0000", 2024-09-16T20:50:52.086 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5858e-05 2024-09-16T20:50:52.086 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.086 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.086 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:52.086 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735584+0000", 2024-09-16T20:50:52.086 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0470999999999998e-05 2024-09-16T20:50:52.086 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.086 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.087 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:52.087 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735607+0000", 2024-09-16T20:50:52.087 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2875999999999999e-05 2024-09-16T20:50:52.087 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.087 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.087 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:52.087 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.859144+0000", 2024-09-16T20:50:52.087 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.123537701 2024-09-16T20:50:52.087 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.087 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.087 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:52.087 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.859195+0000", 2024-09-16T20:50:52.088 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0664999999999997e-05 2024-09-16T20:50:52.088 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:52.088 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:52.088 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:52.088 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.088 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.088 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5204 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 2523136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:52.088 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.883660+0000", 2024-09-16T20:50:52.088 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 28.125698879000002, 2024-09-16T20:50:52.088 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052517638999999998, 2024-09-16T20:50:52.088 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:52.089 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:52.089 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:52.089 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:52.089 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:50:52.089 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:52.089 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5204 2024-09-16T20:50:52.089 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.089 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:52.089 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.089 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:52.089 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883660+0000", 2024-09-16T20:50:52.090 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:52.090 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.090 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.090 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:52.090 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883660+0000", 2024-09-16T20:50:52.090 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:52.090 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.090 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.090 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:52.090 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883661+0000", 2024-09-16T20:50:52.090 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9999999999999998e-07 2024-09-16T20:50:52.090 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.091 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.091 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:52.091 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883663+0000", 2024-09-16T20:50:52.091 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.4480000000000001e-06 2024-09-16T20:50:52.091 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.091 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.091 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:52.091 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883669+0000", 2024-09-16T20:50:52.091 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.071e-06 2024-09-16T20:50:52.091 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.091 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.091 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:52.092 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883675+0000", 2024-09-16T20:50:52.092 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.366e-06 2024-09-16T20:50:52.092 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.092 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.092 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:52.092 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.935970+0000", 2024-09-16T20:50:52.092 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052295346999999999 2024-09-16T20:50:52.092 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.092 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.092 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:52.092 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936064+0000", 2024-09-16T20:50:52.092 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4438000000000006e-05 2024-09-16T20:50:52.093 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.093 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.093 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:52.093 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936178+0000", 2024-09-16T20:50:52.093 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000113569 2024-09-16T20:50:52.093 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:52.093 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:52.093 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:52.093 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.093 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.094 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5205 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 2543616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:52.094 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.883745+0000", 2024-09-16T20:50:52.094 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 28.125614121000002, 2024-09-16T20:50:52.094 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052692112999999999, 2024-09-16T20:50:52.094 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:52.094 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:52.094 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:52.094 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:52.094 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:50:52.094 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:52.094 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5205 2024-09-16T20:50:52.095 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.095 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:52.095 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.095 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:52.095 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883745+0000", 2024-09-16T20:50:52.095 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:52.095 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.095 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.095 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:52.095 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883745+0000", 2024-09-16T20:50:52.095 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:52.095 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.096 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.096 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:52.096 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883746+0000", 2024-09-16T20:50:52.096 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2699999999999999e-07 2024-09-16T20:50:52.096 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.096 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.096 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:52.096 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883750+0000", 2024-09-16T20:50:52.096 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7050000000000001e-06 2024-09-16T20:50:52.096 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.096 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.097 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:52.097 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883769+0000", 2024-09-16T20:50:52.097 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9165e-05 2024-09-16T20:50:52.097 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.097 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.097 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:52.097 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883778+0000", 2024-09-16T20:50:52.097 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.6690000000000001e-06 2024-09-16T20:50:52.097 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.097 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.098 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:52.098 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936228+0000", 2024-09-16T20:50:52.098 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052449587999999998 2024-09-16T20:50:52.098 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.098 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.098 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:52.098 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936333+0000", 2024-09-16T20:50:52.098 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010511800000000001 2024-09-16T20:50:52.098 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.098 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.098 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:52.098 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936437+0000", 2024-09-16T20:50:52.099 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000104141 2024-09-16T20:50:52.099 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:52.099 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:52.099 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:52.099 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.099 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.099 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5206 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 1167360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:52.099 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.883802+0000", 2024-09-16T20:50:52.099 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 28.125557626999999, 2024-09-16T20:50:52.099 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052800780999999998, 2024-09-16T20:50:52.099 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:52.100 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:52.100 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:52.100 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:52.100 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:50:52.100 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:52.100 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5206 2024-09-16T20:50:52.100 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.100 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:52.100 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.100 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:52.100 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883802+0000", 2024-09-16T20:50:52.100 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:52.101 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.101 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.101 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:52.101 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883802+0000", 2024-09-16T20:50:52.101 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:52.101 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.101 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.101 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:52.101 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883933+0000", 2024-09-16T20:50:52.101 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013182999999999999 2024-09-16T20:50:52.101 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.102 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.102 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:52.102 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883937+0000", 2024-09-16T20:50:52.102 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.771e-06 2024-09-16T20:50:52.102 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.102 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.102 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:52.102 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883953+0000", 2024-09-16T20:50:52.102 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5409000000000001e-05 2024-09-16T20:50:52.102 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.102 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.103 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:52.103 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883959+0000", 2024-09-16T20:50:52.103 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0700000000000003e-06 2024-09-16T20:50:52.103 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.103 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.103 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:52.103 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936476+0000", 2024-09-16T20:50:52.103 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052517434000000002 2024-09-16T20:50:52.103 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.103 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.103 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:52.103 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936529+0000", 2024-09-16T20:50:52.104 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2969000000000001e-05 2024-09-16T20:50:52.104 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.104 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.104 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:52.104 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936602+0000", 2024-09-16T20:50:52.104 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3298000000000002e-05 2024-09-16T20:50:52.104 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:52.104 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:52.104 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:52.104 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.104 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.105 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5207 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 1904640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:52.105 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.883980+0000", 2024-09-16T20:50:52.105 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 28.125379418000001, 2024-09-16T20:50:52.105 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052759922000000001, 2024-09-16T20:50:52.105 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:52.105 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:52.105 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:52.105 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:52.105 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:50:52.105 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:52.106 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5207 2024-09-16T20:50:52.106 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.106 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:52.106 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.106 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:52.106 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883980+0000", 2024-09-16T20:50:52.106 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:52.106 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.106 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.106 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:52.106 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883980+0000", 2024-09-16T20:50:52.106 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:52.107 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.107 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.107 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:52.107 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884126+0000", 2024-09-16T20:50:52.107 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014577399999999999 2024-09-16T20:50:52.107 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.107 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.107 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:52.107 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884130+0000", 2024-09-16T20:50:52.107 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9729999999999999e-06 2024-09-16T20:50:52.107 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.108 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.108 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:52.108 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884139+0000", 2024-09-16T20:50:52.108 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8660000000000001e-06 2024-09-16T20:50:52.108 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.108 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.108 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:52.108 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884144+0000", 2024-09-16T20:50:52.108 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3669999999999999e-06 2024-09-16T20:50:52.108 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.108 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.109 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:52.109 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936635+0000", 2024-09-16T20:50:52.109 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052491705 2024-09-16T20:50:52.109 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.109 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.109 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:52.109 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936679+0000", 2024-09-16T20:50:52.109 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3105999999999997e-05 2024-09-16T20:50:52.109 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.109 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.109 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:52.109 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936740+0000", 2024-09-16T20:50:52.110 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1130999999999995e-05 2024-09-16T20:50:52.110 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:52.110 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:52.110 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:52.110 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.110 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.110 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5208 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 131072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:52.110 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.884165+0000", 2024-09-16T20:50:52.110 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 28.125194122, 2024-09-16T20:50:52.110 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052746603000000003, 2024-09-16T20:50:52.110 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:52.110 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:52.111 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:52.111 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:52.111 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:50:52.111 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:52.111 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5208 2024-09-16T20:50:52.111 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.111 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:52.111 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.111 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:52.111 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884165+0000", 2024-09-16T20:50:52.111 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:52.112 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.112 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.112 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:52.112 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884165+0000", 2024-09-16T20:50:52.112 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:52.112 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.112 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.112 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:52.112 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884463+0000", 2024-09-16T20:50:52.112 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00029801699999999998 2024-09-16T20:50:52.112 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.113 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.113 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:52.113 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884467+0000", 2024-09-16T20:50:52.113 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8210000000000003e-06 2024-09-16T20:50:52.113 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.113 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.113 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:52.113 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884477+0000", 2024-09-16T20:50:52.113 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.7030000000000001e-06 2024-09-16T20:50:52.113 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.113 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.114 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:52.114 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884485+0000", 2024-09-16T20:50:52.114 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9710000000000004e-06 2024-09-16T20:50:52.114 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.114 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.114 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:52.114 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936785+0000", 2024-09-16T20:50:52.114 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052300665000000003 2024-09-16T20:50:52.114 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.114 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.114 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:52.115 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936851+0000", 2024-09-16T20:50:52.115 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6159000000000001e-05 2024-09-16T20:50:52.115 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.115 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.115 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:52.115 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936912+0000", 2024-09-16T20:50:52.115 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0266999999999997e-05 2024-09-16T20:50:52.115 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:52.115 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:52.115 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:52.115 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.115 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.115 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5209 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 1925120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:52.116 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.884504+0000", 2024-09-16T20:50:52.116 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 28.124855236999998, 2024-09-16T20:50:52.116 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052575445999999998, 2024-09-16T20:50:52.116 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:52.116 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:52.116 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:52.116 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:52.116 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:50:52.116 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:52.116 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5209 2024-09-16T20:50:52.116 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.117 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:52.117 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.117 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:52.117 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884504+0000", 2024-09-16T20:50:52.117 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:52.117 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.117 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.117 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:52.117 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884504+0000", 2024-09-16T20:50:52.117 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:52.117 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.117 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.118 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:52.118 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884741+0000", 2024-09-16T20:50:52.118 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00023679899999999999 2024-09-16T20:50:52.118 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.118 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.118 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:52.118 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884747+0000", 2024-09-16T20:50:52.118 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0009999999999998e-06 2024-09-16T20:50:52.118 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.118 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.118 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:52.119 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884764+0000", 2024-09-16T20:50:52.119 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7371999999999999e-05 2024-09-16T20:50:52.119 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.119 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.119 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:52.119 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884772+0000", 2024-09-16T20:50:52.119 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1349999999999992e-06 2024-09-16T20:50:52.119 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.119 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.119 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:52.119 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936956+0000", 2024-09-16T20:50:52.119 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052183754999999998 2024-09-16T20:50:52.120 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.120 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.120 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:52.120 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937003+0000", 2024-09-16T20:50:52.120 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6925000000000002e-05 2024-09-16T20:50:52.120 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.120 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.120 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:52.120 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937079+0000", 2024-09-16T20:50:52.120 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6458999999999994e-05 2024-09-16T20:50:52.120 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:52.120 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:52.121 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:52.121 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.121 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.121 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5210 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 151552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:52.121 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.884799+0000", 2024-09-16T20:50:52.121 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 28.124559968, 2024-09-16T20:50:52.121 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052462483999999997, 2024-09-16T20:50:52.121 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:52.121 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:52.121 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:52.121 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:52.121 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:50:52.122 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:52.122 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5210 2024-09-16T20:50:52.122 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.122 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:52.122 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.122 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:52.122 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884799+0000", 2024-09-16T20:50:52.122 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:52.122 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.122 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.122 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:52.122 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884799+0000", 2024-09-16T20:50:52.123 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:52.123 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.123 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.123 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:52.123 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884990+0000", 2024-09-16T20:50:52.123 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019051800000000001 2024-09-16T20:50:52.123 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.123 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.123 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:52.123 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884996+0000", 2024-09-16T20:50:52.123 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0689999999999998e-06 2024-09-16T20:50:52.123 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.124 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.124 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:52.124 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.885008+0000", 2024-09-16T20:50:52.124 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2153999999999999e-05 2024-09-16T20:50:52.124 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.124 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.124 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:52.124 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.885016+0000", 2024-09-16T20:50:52.124 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2770000000000005e-06 2024-09-16T20:50:52.124 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.124 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.124 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:52.125 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937128+0000", 2024-09-16T20:50:52.125 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052111377 2024-09-16T20:50:52.125 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.125 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.125 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:52.125 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937188+0000", 2024-09-16T20:50:52.125 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9933999999999999e-05 2024-09-16T20:50:52.125 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.125 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.125 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:52.125 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937262+0000", 2024-09-16T20:50:52.125 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4154999999999997e-05 2024-09-16T20:50:52.126 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:52.126 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:52.126 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:52.126 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.126 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.126 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5211 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 1945600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:52.126 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.885030+0000", 2024-09-16T20:50:52.126 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 28.124328883, 2024-09-16T20:50:52.126 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052369470000000001, 2024-09-16T20:50:52.126 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:52.126 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:52.126 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:52.127 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:52.127 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:50:52.127 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:52.127 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5211 2024-09-16T20:50:52.127 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.127 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:52.127 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.127 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:52.127 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.885030+0000", 2024-09-16T20:50:52.127 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:52.127 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.127 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.127 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:52.128 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.885030+0000", 2024-09-16T20:50:52.128 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:52.128 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.128 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.128 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:52.128 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.885219+0000", 2024-09-16T20:50:52.128 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000188278 2024-09-16T20:50:52.128 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.128 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.128 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:52.128 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.885223+0000", 2024-09-16T20:50:52.128 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7849999999999999e-06 2024-09-16T20:50:52.129 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.129 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.129 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:52.129 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.885233+0000", 2024-09-16T20:50:52.129 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.5060000000000001e-06 2024-09-16T20:50:52.129 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.129 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.129 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:52.129 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.885240+0000", 2024-09-16T20:50:52.129 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9630000000000001e-06 2024-09-16T20:50:52.129 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.130 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.130 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:52.130 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937304+0000", 2024-09-16T20:50:52.130 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052064619999999999 2024-09-16T20:50:52.130 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.130 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.130 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:52.130 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937347+0000", 2024-09-16T20:50:52.130 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2407000000000002e-05 2024-09-16T20:50:52.130 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:52.130 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:52.131 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:52.131 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937400+0000", 2024-09-16T20:50:52.131 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2911000000000002e-05 2024-09-16T20:50:52.131 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:52.131 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:52.131 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:52.131 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:52.131 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:52.131 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:52.132 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T20:50:52.134 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:50:52.134 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:52.132+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:52.234 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:52.234+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:52.285 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:52.285 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:50:52.285 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:50:52.285 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:52.294 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:50:52.298 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:50:52.299 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:50:52.316 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:50:52.316 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:50:52.332 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:50:52.333 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:50:52.335 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:52.334+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:52.350 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:50:52.350 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:50:52.367 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:50:52.367 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:50:52.384 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:50:52.384 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:50:52.401 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:52.401 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:52.410 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:50:52.451 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:52.450+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:52.537 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:52.536+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:52.557 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:52.557 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:52.566 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:50:52.602 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 4, 5] out_osds: [0, 3] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-16T20:50:52.602 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T20:50:52.602 INFO:tasks.thrashosds.thrasher:Reweighting osd 1 to 0.8807241527955407 2024-09-16T20:50:52.602 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 0.8807241527955407 2024-09-16T20:50:52.618 INFO:teuthology.orchestra.run.smithi029.stderr:nodeep-scrub is unset 2024-09-16T20:50:52.676 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:52.675+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:52.730 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:52.730 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:52.739 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:52.738+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:52.740 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:50:52.840 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:52.839+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:52.893 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:52.894 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:52.902 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:50:52.946 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:52.945+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:53.010 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:53.010 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:53.018 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:50:53.063 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:53.062+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:53.135 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:53.135 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:53.142 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:53.142+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:53.144 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:50:53.244 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:53.243+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:53.291 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:53.291 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:53.301 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:50:53.344 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:53.344+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:53.408 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:53.408 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:53.416 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:50:53.445 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:53.445+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:53.546 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:53.546+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:53.564 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:53.564 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:53.573 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:50:53.623 INFO:teuthology.orchestra.run.smithi029.stderr:reweighted osd.1 to 0.880724 (e177) 2024-09-16T20:50:53.683 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:53.682+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:53.725 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:53.725 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:53.735 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:50:53.748 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:53.747+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:53.843 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:53.843 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:53.848 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:53.848+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:53.852 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:50:53.949 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:53.949+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:54.000 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:54.000 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:54.008 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:50:54.050 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:54.049+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:54.151 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:54.149+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:54.156 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:54.156 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:54.164 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:50:54.251 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:54.250+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:54.313 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:54.313 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:54.321 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:50:54.365 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:54.364+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:54.396 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T20:50:54.435 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:54.436 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:54.445 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:50:54.453 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:54.451+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:54.522 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:50:54.522 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:50:54.539 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:50:54.539 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:50:54.554 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:54.552+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:54.557 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:50:54.558 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:50:54.583 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:50:54.584 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:50:54.604 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:54.605 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:50:54.605 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:50:54.605 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:50:54.605 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:54.605 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:50:54.605 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:50:54.614 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:50:54.626 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:50:54.626 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:50:54.635 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T20:50:54.680 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:54.679+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:54.738 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:54.739 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:50:54.739 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:50:54.739 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:50:54.739 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.739 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5439 3.a 3:5789f5c3:::rbd_data.110ed9ca540.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 1181696~60416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:54.739 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:50:54.739 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 151.87218793700001, 2024-09-16T20:50:54.739 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10102045599999999, 2024-09-16T20:50:54.739 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:54.739 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:54.739 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:54.740 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:54.740 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:54.740 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:54.740 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5439 2024-09-16T20:50:54.740 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.740 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:54.740 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.740 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:54.740 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:50:54.740 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:54.740 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.741 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.741 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:54.741 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:50:54.741 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:54.741 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.741 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.741 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:54.741 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864711+0000", 2024-09-16T20:50:54.741 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7899999999999997e-07 2024-09-16T20:50:54.741 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.741 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.741 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:54.741 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864760+0000", 2024-09-16T20:50:54.742 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8803e-05 2024-09-16T20:50:54.742 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.742 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.742 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:54.742 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864769+0000", 2024-09-16T20:50:54.742 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0999999999999993e-06 2024-09-16T20:50:54.742 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.742 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.742 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:54.742 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864774+0000", 2024-09-16T20:50:54.742 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8919999999999999e-06 2024-09-16T20:50:54.742 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.742 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.743 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:54.743 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864815+0000", 2024-09-16T20:50:54.743 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1307000000000003e-05 2024-09-16T20:50:54.743 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.743 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.743 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:54.743 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864893+0000", 2024-09-16T20:50:54.744 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7581000000000001e-05 2024-09-16T20:50:54.744 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.744 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.744 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 1", 2024-09-16T20:50:54.744 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864996+0000", 2024-09-16T20:50:54.744 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102725 2024-09-16T20:50:54.744 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.744 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.744 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:54.744 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965442+0000", 2024-09-16T20:50:54.744 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100446012 2024-09-16T20:50:54.744 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.745 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.745 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:54.745 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965618+0000", 2024-09-16T20:50:54.745 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00017653100000000001 2024-09-16T20:50:54.745 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.745 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.745 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:54.745 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965682+0000", 2024-09-16T20:50:54.745 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3905999999999994e-05 2024-09-16T20:50:54.745 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.745 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.745 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:54.746 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965731+0000", 2024-09-16T20:50:54.746 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8819999999999997e-05 2024-09-16T20:50:54.746 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:54.746 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:54.746 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:54.746 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.746 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.746 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4600.0:5439 3.a e37/23 ondisk, result = 0)", 2024-09-16T20:50:54.746 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:50:54.746 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 151.87119178899999, 2024-09-16T20:50:54.746 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099742634999999996, 2024-09-16T20:50:54.746 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:54.746 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:54.746 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:54.747 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:54.747 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.747 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:54.747 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:50:54.747 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:54.747 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.747 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.747 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:54.747 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:50:54.747 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:54.747 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.747 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.748 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:54.748 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:50:54.748 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0900000000000001e-07 2024-09-16T20:50:54.748 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.748 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.748 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:54.748 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865711+0000", 2024-09-16T20:50:54.748 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3739999999999998e-06 2024-09-16T20:50:54.748 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.748 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.748 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:54.748 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965343+0000", 2024-09-16T20:50:54.748 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099632526999999999 2024-09-16T20:50:54.749 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.749 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.749 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:54.749 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965357+0000", 2024-09-16T20:50:54.749 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3713999999999999e-05 2024-09-16T20:50:54.749 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.749 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.749 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:54.749 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965426+0000", 2024-09-16T20:50:54.749 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8745999999999995e-05 2024-09-16T20:50:54.749 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.749 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.749 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:54.750 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965434+0000", 2024-09-16T20:50:54.750 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6910000000000003e-06 2024-09-16T20:50:54.750 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.750 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.750 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:54.750 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965449+0000", 2024-09-16T20:50:54.750 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4874e-05 2024-09-16T20:50:54.750 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:54.750 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:54.750 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:54.750 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.750 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.750 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4882.0:2508 3.12 3:4992771f:::rbd_object_map.110e7c8851c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T20:50:54.750 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:50:54.751 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 128.25538824099999, 2024-09-16T20:50:54.751 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.125007011, 2024-09-16T20:50:54.751 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:54.751 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:54.751 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:54.751 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:54.751 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4882", 2024-09-16T20:50:54.751 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2436923223", 2024-09-16T20:50:54.751 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2508 2024-09-16T20:50:54.751 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.751 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:54.751 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.752 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:54.752 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:50:54.752 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:54.752 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.752 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.752 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:54.752 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:50:54.752 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:54.752 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.752 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.752 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:54.752 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481511+0000", 2024-09-16T20:50:54.752 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0699999999999996e-07 2024-09-16T20:50:54.753 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.753 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.753 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:54.753 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481515+0000", 2024-09-16T20:50:54.753 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0609999999999997e-06 2024-09-16T20:50:54.753 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.753 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.753 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:54.753 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481523+0000", 2024-09-16T20:50:54.753 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1310000000000006e-06 2024-09-16T20:50:54.753 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.753 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.753 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:54.753 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481528+0000", 2024-09-16T20:50:54.754 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4900000000000002e-06 2024-09-16T20:50:54.754 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.754 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.754 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:54.754 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481565+0000", 2024-09-16T20:50:54.754 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7066000000000001e-05 2024-09-16T20:50:54.754 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.754 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.754 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:54.754 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481628+0000", 2024-09-16T20:50:54.754 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3118000000000001e-05 2024-09-16T20:50:54.754 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.754 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.755 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 2", 2024-09-16T20:50:54.755 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481822+0000", 2024-09-16T20:50:54.755 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019446200000000001 2024-09-16T20:50:54.755 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.755 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.755 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:54.755 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605988+0000", 2024-09-16T20:50:54.755 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.124165888 2024-09-16T20:50:54.755 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.755 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.755 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:54.755 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606427+0000", 2024-09-16T20:50:54.755 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00043854899999999999 2024-09-16T20:50:54.755 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.756 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.756 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:54.756 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606476+0000", 2024-09-16T20:50:54.756 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8971999999999998e-05 2024-09-16T20:50:54.756 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.756 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.756 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:54.756 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606517+0000", 2024-09-16T20:50:54.756 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1467000000000001e-05 2024-09-16T20:50:54.756 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:54.756 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:54.756 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:54.756 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.757 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.757 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4882.0:2508 3.12 e57/55 ondisk, result = 0)", 2024-09-16T20:50:54.757 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:50:54.757 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 128.25439974099999, 2024-09-16T20:50:54.757 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12349711400000001, 2024-09-16T20:50:54.757 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:54.757 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:54.757 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:54.757 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:54.757 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.757 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:54.757 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:50:54.757 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:54.757 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.758 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.758 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:54.758 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:50:54.758 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:54.758 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.758 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.758 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:54.758 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:50:54.758 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.06e-07 2024-09-16T20:50:54.758 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.758 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.758 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:54.758 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482502+0000", 2024-09-16T20:50:54.758 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.01e-06 2024-09-16T20:50:54.759 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.759 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.759 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:54.759 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605772+0000", 2024-09-16T20:50:54.759 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.123269687 2024-09-16T20:50:54.759 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.759 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.759 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:54.759 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605791+0000", 2024-09-16T20:50:54.759 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8912000000000001e-05 2024-09-16T20:50:54.759 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.759 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.759 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:54.760 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605974+0000", 2024-09-16T20:50:54.760 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000182836 2024-09-16T20:50:54.760 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.760 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.760 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:54.760 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605982+0000", 2024-09-16T20:50:54.760 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1699999999999997e-06 2024-09-16T20:50:54.760 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.760 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.760 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:54.760 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605996+0000", 2024-09-16T20:50:54.760 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3893000000000001e-05 2024-09-16T20:50:54.760 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:54.761 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:54.761 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:54.761 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.761 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.761 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4882.0:2512 3.f 3:f4d895b7:::rbd_data.110e7c8851c.0000000000000000:head [read 16384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T20:50:54.761 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:50:54.761 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 128.25396271100001, 2024-09-16T20:50:54.761 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12318483, 2024-09-16T20:50:54.761 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:54.761 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:54.761 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:54.761 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:54.761 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4882", 2024-09-16T20:50:54.762 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2436923223", 2024-09-16T20:50:54.762 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2512 2024-09-16T20:50:54.762 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.762 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:54.762 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.762 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:54.762 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:50:54.762 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:54.762 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.762 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.762 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:54.762 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:50:54.762 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:54.762 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.762 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.763 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:54.763 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:50:54.763 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3499999999999995e-07 2024-09-16T20:50:54.763 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.763 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.763 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:54.763 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482943+0000", 2024-09-16T20:50:54.763 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3450000000000002e-06 2024-09-16T20:50:54.763 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.763 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.763 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:54.763 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605772+0000", 2024-09-16T20:50:54.763 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.122829796 2024-09-16T20:50:54.764 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.764 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.764 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:54.764 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605792+0000", 2024-09-16T20:50:54.764 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9916999999999999e-05 2024-09-16T20:50:54.764 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.764 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.764 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:54.764 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605917+0000", 2024-09-16T20:50:54.764 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00012500799999999999 2024-09-16T20:50:54.764 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.764 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.764 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:54.765 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606004+0000", 2024-09-16T20:50:54.765 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6242999999999994e-05 2024-09-16T20:50:54.765 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.765 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.765 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:54.765 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606120+0000", 2024-09-16T20:50:54.765 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000116786 2024-09-16T20:50:54.765 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:54.765 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:54.765 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:54.765 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.765 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.765 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:11702 3.15 3:acf33c8b:::rbd_data.110e747f0779.000000000000001a:head [set-alloc-hint object_size 262144 write_size 262144,write 143360~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:50:54.766 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:50:54.766 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 110.59897028899999, 2024-09-16T20:50:54.766 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.084801043000000006, 2024-09-16T20:50:54.766 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:54.766 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:54.766 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:54.766 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:54.766 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:50:54.766 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:50:54.766 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 11702 2024-09-16T20:50:54.766 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.766 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:54.766 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.767 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:54.767 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:50:54.767 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:54.767 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.767 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.767 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:54.767 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:50:54.767 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:54.767 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.767 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.767 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:54.767 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137929+0000", 2024-09-16T20:50:54.767 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8400000000000002e-07 2024-09-16T20:50:54.767 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.768 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.768 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:54.768 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137937+0000", 2024-09-16T20:50:54.768 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0260000000000007e-06 2024-09-16T20:50:54.768 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.768 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.768 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:54.768 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137952+0000", 2024-09-16T20:50:54.768 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4659e-05 2024-09-16T20:50:54.768 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.768 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.768 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:54.768 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137960+0000", 2024-09-16T20:50:54.769 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0169999999999996e-06 2024-09-16T20:50:54.769 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.769 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.769 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:54.769 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138010+0000", 2024-09-16T20:50:54.769 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0021000000000001e-05 2024-09-16T20:50:54.769 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.769 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.769 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:54.769 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138114+0000", 2024-09-16T20:50:54.769 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000103797 2024-09-16T20:50:54.769 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.770 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.770 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:50:54.770 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138233+0000", 2024-09-16T20:50:54.770 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000118971 2024-09-16T20:50:54.770 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.770 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.770 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:54.770 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138953+0000", 2024-09-16T20:50:54.770 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00072026000000000002 2024-09-16T20:50:54.770 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.770 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.770 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:54.770 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222609+0000", 2024-09-16T20:50:54.771 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.083656650999999999 2024-09-16T20:50:54.771 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.771 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.771 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:54.771 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222684+0000", 2024-09-16T20:50:54.771 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4369999999999994e-05 2024-09-16T20:50:54.771 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.771 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.771 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:54.771 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222729+0000", 2024-09-16T20:50:54.771 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5287000000000002e-05 2024-09-16T20:50:54.771 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:54.771 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:54.772 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:54.772 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.772 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.772 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:11704 3.9 3:91220b75:::rbd_data.110e747f0779.000000000000001c:head [set-alloc-hint object_size 262144 write_size 262144,write 139264~6656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:50:54.772 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:50:54.772 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 110.59873228799999, 2024-09-16T20:50:54.772 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.084531133999999994, 2024-09-16T20:50:54.772 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:54.772 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:54.772 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:54.772 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:54.772 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:50:54.772 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:50:54.773 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 11704 2024-09-16T20:50:54.773 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.773 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:54.773 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.773 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:54.773 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:50:54.773 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:54.773 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.773 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.773 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:54.773 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:50:54.773 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:54.773 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.774 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.774 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:54.774 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138167+0000", 2024-09-16T20:50:54.774 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.33e-07 2024-09-16T20:50:54.774 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.774 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.774 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:54.774 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138179+0000", 2024-09-16T20:50:54.774 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2055999999999999e-05 2024-09-16T20:50:54.774 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.774 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.774 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:54.774 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138193+0000", 2024-09-16T20:50:54.775 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3961000000000001e-05 2024-09-16T20:50:54.775 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.775 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.775 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:54.775 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138200+0000", 2024-09-16T20:50:54.775 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4429999999999997e-06 2024-09-16T20:50:54.775 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.775 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.775 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:54.775 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138258+0000", 2024-09-16T20:50:54.775 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7225999999999997e-05 2024-09-16T20:50:54.775 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.775 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.775 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:54.776 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138364+0000", 2024-09-16T20:50:54.776 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000106013 2024-09-16T20:50:54.776 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.776 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.776 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:50:54.776 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138494+0000", 2024-09-16T20:50:54.776 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013059599999999999 2024-09-16T20:50:54.776 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.776 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.776 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:54.776 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.139220+0000", 2024-09-16T20:50:54.776 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00072539800000000004 2024-09-16T20:50:54.776 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.776 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.776 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:54.777 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222608+0000", 2024-09-16T20:50:54.777 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.083388620999999996 2024-09-16T20:50:54.777 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.777 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.777 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:54.777 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222651+0000", 2024-09-16T20:50:54.777 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2228000000000003e-05 2024-09-16T20:50:54.777 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.777 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.777 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:54.777 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222697+0000", 2024-09-16T20:50:54.777 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6659000000000002e-05 2024-09-16T20:50:54.777 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:54.777 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:54.778 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:54.778 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.778 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.778 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5138.0:9358 3.c 3.c25699cc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T20:50:54.778 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:50:54.778 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 87.886584983000006, 2024-09-16T20:50:54.778 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0018657580000001, 2024-09-16T20:50:54.778 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:54.778 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:54.778 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:50:54.778 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:54.778 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5138", 2024-09-16T20:50:54.778 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:50:54.778 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 9358 2024-09-16T20:50:54.779 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.779 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:54.779 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.779 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:54.779 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:50:54.779 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:54.779 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.779 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.779 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:54.779 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:50:54.779 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:54.779 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.779 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.779 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:54.780 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850314+0000", 2024-09-16T20:50:54.780 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4099999999999999e-07 2024-09-16T20:50:54.780 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.780 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.780 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:54.780 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850316+0000", 2024-09-16T20:50:54.780 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2469999999999998e-06 2024-09-16T20:50:54.780 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.780 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.780 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:54.780 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850322+0000", 2024-09-16T20:50:54.780 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3470000000000004e-06 2024-09-16T20:50:54.780 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.780 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.781 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:54.781 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850327+0000", 2024-09-16T20:50:54.781 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9450000000000001e-06 2024-09-16T20:50:54.781 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.781 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.781 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:54.781 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850376+0000", 2024-09-16T20:50:54.781 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8257000000000002e-05 2024-09-16T20:50:54.781 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.781 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.781 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:54.781 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850385+0000", 2024-09-16T20:50:54.781 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9800000000000004e-06 2024-09-16T20:50:54.782 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.782 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.782 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:54.782 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852167+0000", 2024-09-16T20:50:54.782 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0017825659999999 2024-09-16T20:50:54.782 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.782 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.782 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:54.782 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852179+0000", 2024-09-16T20:50:54.782 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1975e-05 2024-09-16T20:50:54.782 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:54.782 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:54.782 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:54.782 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.782 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.783 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5138.0:9382 3.c 3.c25699cc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T20:50:54.783 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:50:54.783 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 87.878237967000004, 2024-09-16T20:50:54.783 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.99358404700000003, 2024-09-16T20:50:54.783 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:54.783 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:54.783 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:50:54.783 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:54.783 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5138", 2024-09-16T20:50:54.783 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:50:54.783 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 9382 2024-09-16T20:50:54.783 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.783 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:54.783 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.784 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:54.784 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:50:54.784 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:54.784 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.784 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.784 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:54.784 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:50:54.784 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:54.784 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.784 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.784 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:54.784 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858661+0000", 2024-09-16T20:50:54.784 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0200000000000002e-07 2024-09-16T20:50:54.784 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.785 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.785 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:54.785 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858664+0000", 2024-09-16T20:50:54.785 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.038e-06 2024-09-16T20:50:54.785 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.785 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.785 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:54.785 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858671+0000", 2024-09-16T20:50:54.785 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0110000000000003e-06 2024-09-16T20:50:54.785 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.785 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.785 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:54.785 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858677+0000", 2024-09-16T20:50:54.785 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0959999999999997e-06 2024-09-16T20:50:54.786 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.786 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.786 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:54.786 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858732+0000", 2024-09-16T20:50:54.786 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4543000000000003e-05 2024-09-16T20:50:54.786 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.786 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.786 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:54.786 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858739+0000", 2024-09-16T20:50:54.786 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5429999999999996e-06 2024-09-16T20:50:54.786 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.786 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.786 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:54.786 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852236+0000", 2024-09-16T20:50:54.787 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.993496453 2024-09-16T20:50:54.787 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.787 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.787 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:54.787 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852244+0000", 2024-09-16T20:50:54.787 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8610000000000004e-06 2024-09-16T20:50:54.787 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:54.787 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:54.787 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:54.787 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.787 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.787 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5763.0:4895 3.a 3:5384606c:::rbd_data.110e1b19a241.0000000000000018:head [read 0~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-16T20:50:54.787 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:45.598212+0000", 2024-09-16T20:50:54.788 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 69.138685925999994, 2024-09-16T20:50:54.788 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0375864829999999, 2024-09-16T20:50:54.788 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:54.788 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:54.788 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:54.788 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:54.788 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5763", 2024-09-16T20:50:54.788 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/623897567", 2024-09-16T20:50:54.788 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4895 2024-09-16T20:50:54.788 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.788 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:54.788 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.789 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:54.789 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598212+0000", 2024-09-16T20:50:54.789 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:54.789 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.789 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.789 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:54.789 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598212+0000", 2024-09-16T20:50:54.789 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:54.789 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.789 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.789 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:54.789 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598214+0000", 2024-09-16T20:50:54.789 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0090000000000002e-06 2024-09-16T20:50:54.790 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.790 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.790 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:54.790 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598222+0000", 2024-09-16T20:50:54.790 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6699999999999994e-06 2024-09-16T20:50:54.790 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.790 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.790 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:54.790 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598241+0000", 2024-09-16T20:50:54.790 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8896999999999999e-05 2024-09-16T20:50:54.790 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.790 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.790 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:54.791 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598250+0000", 2024-09-16T20:50:54.791 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6649999999999992e-06 2024-09-16T20:50:54.791 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.791 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.791 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:54.791 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598325+0000", 2024-09-16T20:50:54.791 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5189000000000001e-05 2024-09-16T20:50:54.791 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.791 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.791 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op must wait for map", 2024-09-16T20:50:54.791 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598333+0000", 2024-09-16T20:50:54.791 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.3170000000000002e-06 2024-09-16T20:50:54.791 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.791 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.792 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:54.792 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601355+0000", 2024-09-16T20:50:54.792 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0030215400000000001 2024-09-16T20:50:54.792 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.792 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.792 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:54.792 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601365+0000", 2024-09-16T20:50:54.792 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0215e-05 2024-09-16T20:50:54.792 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.792 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.792 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:54.792 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635506+0000", 2024-09-16T20:50:54.793 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0341408240000001 2024-09-16T20:50:54.793 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.793 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.793 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:54.793 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635655+0000", 2024-09-16T20:50:54.793 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000149412 2024-09-16T20:50:54.793 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.793 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.793 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:54.793 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635799+0000", 2024-09-16T20:50:54.793 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000143745 2024-09-16T20:50:54.793 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:54.793 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:54.794 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:54.794 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.794 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.794 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5763.0:4900 3.a 3:5384606c:::rbd_data.110e1b19a241.0000000000000018:head [read 8192~24576] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-16T20:50:54.794 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:50:54.794 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 69.138627450000001, 2024-09-16T20:50:54.794 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0379340800000001, 2024-09-16T20:50:54.794 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:54.794 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:54.794 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:54.794 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:54.794 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5763", 2024-09-16T20:50:54.794 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/623897567", 2024-09-16T20:50:54.794 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4900 2024-09-16T20:50:54.795 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.795 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:54.795 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.795 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:54.795 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:50:54.795 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:54.795 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.795 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.795 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:54.795 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:50:54.795 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:54.795 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.795 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.795 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:54.795 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:50:54.796 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4300000000000003e-07 2024-09-16T20:50:54.796 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.796 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.796 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:54.796 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598274+0000", 2024-09-16T20:50:54.796 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.976e-06 2024-09-16T20:50:54.796 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.796 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.796 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:54.796 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598282+0000", 2024-09-16T20:50:54.796 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0029999999999994e-06 2024-09-16T20:50:54.796 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.796 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.797 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:54.797 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598289+0000", 2024-09-16T20:50:54.797 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2389999999999999e-06 2024-09-16T20:50:54.797 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.797 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.797 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:54.797 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598430+0000", 2024-09-16T20:50:54.797 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014119600000000001 2024-09-16T20:50:54.797 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.797 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.797 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting_for_map not empty", 2024-09-16T20:50:54.797 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598435+0000", 2024-09-16T20:50:54.797 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3489999999999999e-06 2024-09-16T20:50:54.798 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.798 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.798 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:54.798 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601429+0000", 2024-09-16T20:50:54.798 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0029942720000000001 2024-09-16T20:50:54.798 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.798 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.798 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:54.798 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601438+0000", 2024-09-16T20:50:54.798 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4759999999999998e-06 2024-09-16T20:50:54.798 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.798 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.798 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:54.799 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635987+0000", 2024-09-16T20:50:54.799 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0345495010000001 2024-09-16T20:50:54.799 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.799 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.799 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:54.799 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.636056+0000", 2024-09-16T20:50:54.799 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8610999999999998e-05 2024-09-16T20:50:54.799 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.799 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.799 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:54.799 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.636205+0000", 2024-09-16T20:50:54.799 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014891399999999999 2024-09-16T20:50:54.799 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:54.800 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:54.800 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:54.800 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.800 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.800 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop(client.6231.0:2182 3.17 e136/81 3:edb5e5cd:::rbd_data.110e61479aee.0000000000000014:head v 136'791, mlcod=135'789)", 2024-09-16T20:50:54.800 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:22.735499+0000", 2024-09-16T20:50:54.800 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.001399509000002, 2024-09-16T20:50:54.800 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12369608, 2024-09-16T20:50:54.800 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:54.800 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:54.800 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:54.800 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:54.800 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.800 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:54.801 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735499+0000", 2024-09-16T20:50:54.801 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:54.801 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.801 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.801 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:54.801 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735499+0000", 2024-09-16T20:50:54.801 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:54.801 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.801 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.801 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:54.801 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735499+0000", 2024-09-16T20:50:54.801 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8099999999999999e-07 2024-09-16T20:50:54.801 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.801 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.802 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:54.802 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735506+0000", 2024-09-16T20:50:54.802 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3579999999999997e-06 2024-09-16T20:50:54.802 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.802 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.802 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:54.802 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735517+0000", 2024-09-16T20:50:54.802 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0869999999999999e-05 2024-09-16T20:50:54.802 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.802 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.802 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:54.802 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735533+0000", 2024-09-16T20:50:54.802 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5858e-05 2024-09-16T20:50:54.803 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.803 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.803 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:54.803 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735584+0000", 2024-09-16T20:50:54.803 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0470999999999998e-05 2024-09-16T20:50:54.803 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.803 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.803 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:54.803 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735607+0000", 2024-09-16T20:50:54.803 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2875999999999999e-05 2024-09-16T20:50:54.803 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.803 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.803 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:54.804 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.859144+0000", 2024-09-16T20:50:54.804 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.123537701 2024-09-16T20:50:54.804 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.804 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.804 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:54.804 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.859195+0000", 2024-09-16T20:50:54.804 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0664999999999997e-05 2024-09-16T20:50:54.804 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:54.804 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:54.804 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:54.804 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.804 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.804 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5204 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 2523136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:54.805 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.883660+0000", 2024-09-16T20:50:54.805 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 30.853237978999999, 2024-09-16T20:50:54.805 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052517638999999998, 2024-09-16T20:50:54.805 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:54.805 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:54.805 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:54.805 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:54.805 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:50:54.805 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:54.806 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5204 2024-09-16T20:50:54.806 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.806 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:54.806 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.806 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:54.806 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883660+0000", 2024-09-16T20:50:54.806 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:54.806 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.806 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.806 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:54.806 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883660+0000", 2024-09-16T20:50:54.806 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:54.806 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.807 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.807 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:54.807 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883661+0000", 2024-09-16T20:50:54.807 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9999999999999998e-07 2024-09-16T20:50:54.807 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.807 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.807 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:54.807 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883663+0000", 2024-09-16T20:50:54.807 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.4480000000000001e-06 2024-09-16T20:50:54.807 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.807 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.807 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:54.807 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883669+0000", 2024-09-16T20:50:54.808 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.071e-06 2024-09-16T20:50:54.808 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.808 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.808 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:54.808 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883675+0000", 2024-09-16T20:50:54.808 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.366e-06 2024-09-16T20:50:54.808 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.808 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.808 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:54.808 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.935970+0000", 2024-09-16T20:50:54.808 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052295346999999999 2024-09-16T20:50:54.808 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.809 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.809 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:54.809 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936064+0000", 2024-09-16T20:50:54.809 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4438000000000006e-05 2024-09-16T20:50:54.809 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.809 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.809 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:54.809 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936178+0000", 2024-09-16T20:50:54.809 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000113569 2024-09-16T20:50:54.809 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:54.809 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:54.809 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:54.809 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.810 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.810 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5205 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 2543616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:54.810 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.883745+0000", 2024-09-16T20:50:54.810 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 30.853153220999999, 2024-09-16T20:50:54.810 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052692112999999999, 2024-09-16T20:50:54.810 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:54.810 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:54.810 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:54.810 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:54.810 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:50:54.810 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:54.810 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5205 2024-09-16T20:50:54.811 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.811 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:54.811 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.811 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:54.811 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883745+0000", 2024-09-16T20:50:54.811 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:54.811 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.811 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.811 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:54.811 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883745+0000", 2024-09-16T20:50:54.811 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:54.811 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.811 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.811 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:54.812 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883746+0000", 2024-09-16T20:50:54.812 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2699999999999999e-07 2024-09-16T20:50:54.812 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.812 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.812 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:54.812 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883750+0000", 2024-09-16T20:50:54.812 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7050000000000001e-06 2024-09-16T20:50:54.812 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.812 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.812 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:54.812 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883769+0000", 2024-09-16T20:50:54.812 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9165e-05 2024-09-16T20:50:54.813 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.813 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.813 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:54.813 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883778+0000", 2024-09-16T20:50:54.813 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.6690000000000001e-06 2024-09-16T20:50:54.813 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.813 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.813 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:54.813 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936228+0000", 2024-09-16T20:50:54.813 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052449587999999998 2024-09-16T20:50:54.813 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.813 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.813 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:54.814 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936333+0000", 2024-09-16T20:50:54.814 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010511800000000001 2024-09-16T20:50:54.814 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.814 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.814 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:54.814 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936437+0000", 2024-09-16T20:50:54.814 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000104141 2024-09-16T20:50:54.814 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:54.814 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:54.814 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:54.814 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.814 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.814 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5206 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 1167360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:54.815 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.883802+0000", 2024-09-16T20:50:54.815 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 30.853096727, 2024-09-16T20:50:54.815 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052800780999999998, 2024-09-16T20:50:54.815 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:54.815 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:54.815 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:54.815 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:54.815 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:50:54.815 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:54.815 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5206 2024-09-16T20:50:54.815 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.815 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:54.815 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.815 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:54.816 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883802+0000", 2024-09-16T20:50:54.816 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:54.816 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.816 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.816 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:54.816 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883802+0000", 2024-09-16T20:50:54.816 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:54.816 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.816 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.816 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:54.816 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883933+0000", 2024-09-16T20:50:54.816 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013182999999999999 2024-09-16T20:50:54.816 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.817 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.817 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:54.817 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883937+0000", 2024-09-16T20:50:54.817 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.771e-06 2024-09-16T20:50:54.817 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.817 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.817 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:54.817 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883953+0000", 2024-09-16T20:50:54.817 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5409000000000001e-05 2024-09-16T20:50:54.817 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.817 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.817 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:54.817 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883959+0000", 2024-09-16T20:50:54.818 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0700000000000003e-06 2024-09-16T20:50:54.818 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.818 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.818 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:54.818 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936476+0000", 2024-09-16T20:50:54.818 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052517434000000002 2024-09-16T20:50:54.818 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.818 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.818 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:54.818 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936529+0000", 2024-09-16T20:50:54.818 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2969000000000001e-05 2024-09-16T20:50:54.818 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.818 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.818 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:54.818 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936602+0000", 2024-09-16T20:50:54.819 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3298000000000002e-05 2024-09-16T20:50:54.819 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:54.819 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:54.819 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:54.819 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.819 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.819 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5207 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 1904640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:54.819 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.883980+0000", 2024-09-16T20:50:54.819 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 30.852918517999999, 2024-09-16T20:50:54.819 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052759922000000001, 2024-09-16T20:50:54.819 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:54.819 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:54.819 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:54.819 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:54.819 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:50:54.820 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:54.820 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5207 2024-09-16T20:50:54.820 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.820 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:54.820 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.820 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:54.820 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883980+0000", 2024-09-16T20:50:54.820 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:54.820 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.820 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.820 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:54.820 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883980+0000", 2024-09-16T20:50:54.820 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:54.821 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.821 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.821 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:54.821 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884126+0000", 2024-09-16T20:50:54.821 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014577399999999999 2024-09-16T20:50:54.821 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.821 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.821 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:54.821 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884130+0000", 2024-09-16T20:50:54.821 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9729999999999999e-06 2024-09-16T20:50:54.821 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.821 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.821 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:54.821 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884139+0000", 2024-09-16T20:50:54.822 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8660000000000001e-06 2024-09-16T20:50:54.822 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.822 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.822 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:54.822 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884144+0000", 2024-09-16T20:50:54.822 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3669999999999999e-06 2024-09-16T20:50:54.822 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.822 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.822 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:54.822 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936635+0000", 2024-09-16T20:50:54.822 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052491705 2024-09-16T20:50:54.822 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.822 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.822 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:54.822 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936679+0000", 2024-09-16T20:50:54.822 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3105999999999997e-05 2024-09-16T20:50:54.823 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.823 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.823 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:54.823 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936740+0000", 2024-09-16T20:50:54.823 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1130999999999995e-05 2024-09-16T20:50:54.823 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:54.823 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:54.823 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:54.823 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.823 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.823 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5208 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 131072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:54.823 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.884165+0000", 2024-09-16T20:50:54.823 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 30.852733222000001, 2024-09-16T20:50:54.823 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052746603000000003, 2024-09-16T20:50:54.823 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:54.824 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:54.824 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:54.824 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:54.824 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:50:54.824 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:54.824 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5208 2024-09-16T20:50:54.824 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.824 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:54.824 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.824 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:54.824 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884165+0000", 2024-09-16T20:50:54.824 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:54.824 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.824 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.824 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:54.825 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884165+0000", 2024-09-16T20:50:54.825 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:54.825 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.825 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.825 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:54.825 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884463+0000", 2024-09-16T20:50:54.825 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00029801699999999998 2024-09-16T20:50:54.825 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.825 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.825 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:54.825 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884467+0000", 2024-09-16T20:50:54.825 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8210000000000003e-06 2024-09-16T20:50:54.825 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.825 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.826 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:54.826 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884477+0000", 2024-09-16T20:50:54.826 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.7030000000000001e-06 2024-09-16T20:50:54.826 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.826 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.826 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:54.826 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884485+0000", 2024-09-16T20:50:54.826 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9710000000000004e-06 2024-09-16T20:50:54.826 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.826 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.826 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:54.826 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936785+0000", 2024-09-16T20:50:54.826 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052300665000000003 2024-09-16T20:50:54.826 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.826 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.827 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:54.827 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936851+0000", 2024-09-16T20:50:54.827 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6159000000000001e-05 2024-09-16T20:50:54.827 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.827 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.827 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:54.827 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936912+0000", 2024-09-16T20:50:54.827 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0266999999999997e-05 2024-09-16T20:50:54.827 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:54.827 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:54.827 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:54.827 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.827 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.827 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5209 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 1925120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:54.828 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.884504+0000", 2024-09-16T20:50:54.828 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 30.852394337, 2024-09-16T20:50:54.828 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052575445999999998, 2024-09-16T20:50:54.828 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:54.828 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:54.828 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:54.828 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:54.828 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:50:54.828 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:54.828 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5209 2024-09-16T20:50:54.828 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.828 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:54.828 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.828 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:54.828 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884504+0000", 2024-09-16T20:50:54.829 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:54.829 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.829 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.829 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:54.829 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884504+0000", 2024-09-16T20:50:54.829 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:54.829 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.829 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.829 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:54.829 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884741+0000", 2024-09-16T20:50:54.829 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00023679899999999999 2024-09-16T20:50:54.829 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.829 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.829 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:54.830 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884747+0000", 2024-09-16T20:50:54.830 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0009999999999998e-06 2024-09-16T20:50:54.830 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.830 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.830 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:54.830 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884764+0000", 2024-09-16T20:50:54.830 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7371999999999999e-05 2024-09-16T20:50:54.830 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.830 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.830 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:54.830 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884772+0000", 2024-09-16T20:50:54.830 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1349999999999992e-06 2024-09-16T20:50:54.830 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.830 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.830 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:54.831 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936956+0000", 2024-09-16T20:50:54.831 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052183754999999998 2024-09-16T20:50:54.831 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.831 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.831 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:54.831 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937003+0000", 2024-09-16T20:50:54.831 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6925000000000002e-05 2024-09-16T20:50:54.831 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.831 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.831 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:54.831 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937079+0000", 2024-09-16T20:50:54.831 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6458999999999994e-05 2024-09-16T20:50:54.831 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:54.831 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:54.831 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:54.832 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.832 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.832 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5210 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 151552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:54.832 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.884799+0000", 2024-09-16T20:50:54.832 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 30.852099068000001, 2024-09-16T20:50:54.832 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052462483999999997, 2024-09-16T20:50:54.832 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:54.832 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:54.832 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:54.832 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:54.832 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:50:54.832 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:54.832 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5210 2024-09-16T20:50:54.832 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.832 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:54.833 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.833 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:54.833 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884799+0000", 2024-09-16T20:50:54.833 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:54.833 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.833 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.833 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:54.833 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884799+0000", 2024-09-16T20:50:54.833 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:54.833 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.833 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.833 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:54.833 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884990+0000", 2024-09-16T20:50:54.833 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019051800000000001 2024-09-16T20:50:54.834 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.834 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.834 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:54.834 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884996+0000", 2024-09-16T20:50:54.834 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0689999999999998e-06 2024-09-16T20:50:54.834 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.834 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.834 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:54.834 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.885008+0000", 2024-09-16T20:50:54.834 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2153999999999999e-05 2024-09-16T20:50:54.834 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.834 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.834 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:54.834 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.885016+0000", 2024-09-16T20:50:54.834 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2770000000000005e-06 2024-09-16T20:50:54.835 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.835 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.835 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:54.835 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937128+0000", 2024-09-16T20:50:54.835 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052111377 2024-09-16T20:50:54.835 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.835 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.835 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:54.835 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937188+0000", 2024-09-16T20:50:54.835 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9933999999999999e-05 2024-09-16T20:50:54.835 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.835 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.835 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:54.835 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937262+0000", 2024-09-16T20:50:54.835 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4154999999999997e-05 2024-09-16T20:50:54.836 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:54.836 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:54.836 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:54.836 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.836 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.836 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5211 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 1945600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:50:54.836 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.885030+0000", 2024-09-16T20:50:54.836 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 30.851867982999998, 2024-09-16T20:50:54.836 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052369470000000001, 2024-09-16T20:50:54.836 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:54.836 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:54.836 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:54.836 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:54.836 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:50:54.837 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:54.837 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5211 2024-09-16T20:50:54.837 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.837 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:54.837 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.837 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:54.837 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.885030+0000", 2024-09-16T20:50:54.837 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:54.837 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.837 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.837 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:54.837 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.885030+0000", 2024-09-16T20:50:54.837 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:54.837 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.838 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.838 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:54.838 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.885219+0000", 2024-09-16T20:50:54.838 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000188278 2024-09-16T20:50:54.838 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.838 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.838 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:54.838 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.885223+0000", 2024-09-16T20:50:54.838 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7849999999999999e-06 2024-09-16T20:50:54.838 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.838 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.838 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:54.838 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.885233+0000", 2024-09-16T20:50:54.838 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.5060000000000001e-06 2024-09-16T20:50:54.839 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.839 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.839 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:54.839 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.885240+0000", 2024-09-16T20:50:54.839 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9630000000000001e-06 2024-09-16T20:50:54.839 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.839 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.839 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:54.839 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937304+0000", 2024-09-16T20:50:54.839 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052064619999999999 2024-09-16T20:50:54.839 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.839 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.839 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:54.839 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937347+0000", 2024-09-16T20:50:54.840 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2407000000000002e-05 2024-09-16T20:50:54.840 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:54.840 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:54.840 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:54.840 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937400+0000", 2024-09-16T20:50:54.840 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2911000000000002e-05 2024-09-16T20:50:54.840 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:54.840 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:54.840 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:54.840 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:54.840 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:54.840 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:54.842 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:50:54.842 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:54.840+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:54.942 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:54.941+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:54.949 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:54.949 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:50:54.949 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:50:54.949 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:54.958 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:50:55.043 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:55.042+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:55.065 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:55.065 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:50:55.065 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:50:55.065 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:50:55.065 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:55.073 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:50:55.145 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:55.143+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:55.223 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:55.223 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:50:55.223 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:50:55.223 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:50:55.223 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.223 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:50:55.223 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:50:55.223 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 181.96687128900001, 2024-09-16T20:50:55.223 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67502346999999996, 2024-09-16T20:50:55.223 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:55.223 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:55.223 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:55.224 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:55.224 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:50:55.224 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:50:55.224 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 1 2024-09-16T20:50:55.224 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.224 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:55.224 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.224 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:55.224 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:50:55.224 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.224 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.224 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.224 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:55.224 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:50:55.225 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.225 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.225 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.225 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:55.225 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254128+0000", 2024-09-16T20:50:55.225 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.463e-06 2024-09-16T20:50:55.225 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.225 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.225 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:55.225 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254135+0000", 2024-09-16T20:50:55.225 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1579999999999999e-06 2024-09-16T20:50:55.225 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.225 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.225 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:55.226 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254146+0000", 2024-09-16T20:50:55.226 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1341e-05 2024-09-16T20:50:55.226 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.226 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.226 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:55.226 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254155+0000", 2024-09-16T20:50:55.226 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.1549999999999996e-06 2024-09-16T20:50:55.226 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.226 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.226 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:55.226 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254221+0000", 2024-09-16T20:50:55.226 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5470999999999997e-05 2024-09-16T20:50:55.226 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.226 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.226 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:55.227 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254231+0000", 2024-09-16T20:50:55.227 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0592e-05 2024-09-16T20:50:55.227 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.227 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.227 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:55.227 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926711+0000", 2024-09-16T20:50:55.227 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67247946599999997 2024-09-16T20:50:55.227 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.227 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.227 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:55.227 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926910+0000", 2024-09-16T20:50:55.227 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019934 2024-09-16T20:50:55.227 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.228 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.228 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:50:55.228 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.927159+0000", 2024-09-16T20:50:55.228 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00024876599999999998 2024-09-16T20:50:55.228 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.228 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.228 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:55.228 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.928797+0000", 2024-09-16T20:50:55.228 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001638308 2024-09-16T20:50:55.228 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.228 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.228 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:55.228 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929026+0000", 2024-09-16T20:50:55.229 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00022876099999999999 2024-09-16T20:50:55.229 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.229 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.229 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:55.229 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929102+0000", 2024-09-16T20:50:55.229 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6184000000000001e-05 2024-09-16T20:50:55.229 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.229 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.229 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:55.229 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929150+0000", 2024-09-16T20:50:55.229 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7465e-05 2024-09-16T20:50:55.229 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.229 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:55.230 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.230 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.230 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.230 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3574 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:55.230 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:50:55.230 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 125.73970775700001, 2024-09-16T20:50:55.230 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053579441999999998, 2024-09-16T20:50:55.230 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:55.230 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:55.230 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:55.230 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:55.230 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:55.231 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:55.231 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3574 2024-09-16T20:50:55.231 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.231 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:55.231 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.231 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:55.231 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:50:55.231 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.231 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.231 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.231 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:55.231 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:50:55.231 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.231 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.232 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.232 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:55.232 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:50:55.232 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2e-07 2024-09-16T20:50:55.232 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.232 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.232 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:55.232 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481292+0000", 2024-09-16T20:50:55.232 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2630000000000002e-06 2024-09-16T20:50:55.232 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.232 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.232 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:55.232 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481297+0000", 2024-09-16T20:50:55.232 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.296e-06 2024-09-16T20:50:55.232 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.233 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.233 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:55.233 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481300+0000", 2024-09-16T20:50:55.233 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7019999999999999e-06 2024-09-16T20:50:55.233 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.233 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.233 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:55.233 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525533+0000", 2024-09-16T20:50:55.233 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.044232318 2024-09-16T20:50:55.233 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.233 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.233 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:55.233 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525619+0000", 2024-09-16T20:50:55.234 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5892999999999999e-05 2024-09-16T20:50:55.234 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.234 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.234 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:55.234 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525987+0000", 2024-09-16T20:50:55.234 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00036836099999999999 2024-09-16T20:50:55.234 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.234 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.234 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:55.234 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531779+0000", 2024-09-16T20:50:55.234 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0057918240000000001 2024-09-16T20:50:55.234 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.234 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.234 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:55.235 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534747+0000", 2024-09-16T20:50:55.235 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0029681600000000001 2024-09-16T20:50:55.235 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.235 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.235 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:55.235 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534832+0000", 2024-09-16T20:50:55.235 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5180999999999996e-05 2024-09-16T20:50:55.235 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.235 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.235 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:55.235 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534869+0000", 2024-09-16T20:50:55.235 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7024000000000003e-05 2024-09-16T20:50:55.235 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.235 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:55.235 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.236 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.236 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.236 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3575 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:55.236 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:50:55.236 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 125.73969301699999, 2024-09-16T20:50:55.236 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053801463000000001, 2024-09-16T20:50:55.236 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:55.236 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:55.236 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:55.236 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:55.236 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:55.236 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:55.236 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3575 2024-09-16T20:50:55.236 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.237 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:55.237 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.237 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:55.237 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:50:55.237 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.237 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.237 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.237 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:55.237 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:50:55.237 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.237 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.237 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.237 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:55.237 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481305+0000", 2024-09-16T20:50:55.238 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5400000000000002e-07 2024-09-16T20:50:55.238 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.238 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.238 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:55.238 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481307+0000", 2024-09-16T20:50:55.238 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7800000000000001e-06 2024-09-16T20:50:55.238 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.238 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.238 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:55.238 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481312+0000", 2024-09-16T20:50:55.238 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6349999999999997e-06 2024-09-16T20:50:55.238 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.238 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.238 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:55.238 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481315+0000", 2024-09-16T20:50:55.239 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7829999999999999e-06 2024-09-16T20:50:55.239 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.239 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.239 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:55.239 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.526876+0000", 2024-09-16T20:50:55.239 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.045561070000000002 2024-09-16T20:50:55.239 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.239 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.239 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:55.239 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.526954+0000", 2024-09-16T20:50:55.239 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8108000000000005e-05 2024-09-16T20:50:55.239 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.239 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.239 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:55.239 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527282+0000", 2024-09-16T20:50:55.240 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000327483 2024-09-16T20:50:55.240 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.240 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.240 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:55.240 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531784+0000", 2024-09-16T20:50:55.240 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0045020169999999997 2024-09-16T20:50:55.240 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.240 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.240 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:55.240 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534921+0000", 2024-09-16T20:50:55.240 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.003137791 2024-09-16T20:50:55.240 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.240 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.241 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:55.241 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534976+0000", 2024-09-16T20:50:55.241 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4953000000000001e-05 2024-09-16T20:50:55.241 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.241 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.241 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:55.241 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535106+0000", 2024-09-16T20:50:55.241 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000129589 2024-09-16T20:50:55.241 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.241 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:55.241 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.241 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.241 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.242 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3576 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:55.242 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:50:55.242 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 125.73967910499999, 2024-09-16T20:50:55.242 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053955436000000002, 2024-09-16T20:50:55.242 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:55.242 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:55.242 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:55.242 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:55.242 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:55.242 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:55.242 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3576 2024-09-16T20:50:55.242 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.242 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:55.242 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.242 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:55.242 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:50:55.243 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.243 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.243 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.243 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:55.243 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:50:55.243 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.243 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.243 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.243 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:55.243 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481319+0000", 2024-09-16T20:50:55.243 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.53e-07 2024-09-16T20:50:55.243 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.243 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.243 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:55.244 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481320+0000", 2024-09-16T20:50:55.244 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5230000000000001e-06 2024-09-16T20:50:55.244 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.244 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.244 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:55.244 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481330+0000", 2024-09-16T20:50:55.244 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0190000000000001e-05 2024-09-16T20:50:55.244 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.244 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.244 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:55.244 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481333+0000", 2024-09-16T20:50:55.244 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.151e-06 2024-09-16T20:50:55.244 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.245 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.245 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:55.245 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527689+0000", 2024-09-16T20:50:55.245 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046355192000000003 2024-09-16T20:50:55.245 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.245 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.245 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:55.245 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527764+0000", 2024-09-16T20:50:55.245 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5776000000000001e-05 2024-09-16T20:50:55.245 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.245 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.245 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:55.245 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528141+0000", 2024-09-16T20:50:55.246 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00037647599999999999 2024-09-16T20:50:55.246 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.246 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.246 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:55.246 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531788+0000", 2024-09-16T20:50:55.246 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0036470679999999998 2024-09-16T20:50:55.246 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.246 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.246 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:55.246 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535174+0000", 2024-09-16T20:50:55.246 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0033862670000000001 2024-09-16T20:50:55.246 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.246 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.246 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:55.246 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535236+0000", 2024-09-16T20:50:55.246 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1797999999999993e-05 2024-09-16T20:50:55.247 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.247 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.247 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:55.247 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535274+0000", 2024-09-16T20:50:55.247 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7741999999999999e-05 2024-09-16T20:50:55.247 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.247 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:55.247 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.247 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.247 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.247 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3577 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:55.247 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:50:55.247 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 125.739657505, 2024-09-16T20:50:55.247 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054073807000000002, 2024-09-16T20:50:55.247 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:55.248 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:55.248 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:55.248 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:55.248 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:55.248 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:55.248 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3577 2024-09-16T20:50:55.248 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.248 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:55.248 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.248 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:55.248 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:50:55.248 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.248 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.249 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.249 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:55.249 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:50:55.249 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.249 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.249 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.249 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:55.249 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:50:55.249 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.96e-07 2024-09-16T20:50:55.249 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.249 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.249 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:55.249 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481342+0000", 2024-09-16T20:50:55.249 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.714e-06 2024-09-16T20:50:55.250 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.250 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.250 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:55.250 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481349+0000", 2024-09-16T20:50:55.250 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4930000000000001e-06 2024-09-16T20:50:55.250 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.250 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.250 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:55.250 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481353+0000", 2024-09-16T20:50:55.250 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4879999999999999e-06 2024-09-16T20:50:55.250 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.250 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.250 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:55.251 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528619+0000", 2024-09-16T20:50:55.251 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047266351999999998 2024-09-16T20:50:55.251 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.251 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.251 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:55.251 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528706+0000", 2024-09-16T20:50:55.251 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6796e-05 2024-09-16T20:50:55.251 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.251 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.251 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:55.251 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529148+0000", 2024-09-16T20:50:55.251 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00044182999999999999 2024-09-16T20:50:55.251 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.252 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.252 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:55.252 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531792+0000", 2024-09-16T20:50:55.252 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0026443119999999998 2024-09-16T20:50:55.252 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.252 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.252 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:55.252 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535319+0000", 2024-09-16T20:50:55.252 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0035266439999999998 2024-09-16T20:50:55.252 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.252 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.252 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:55.252 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535372+0000", 2024-09-16T20:50:55.252 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3207000000000001e-05 2024-09-16T20:50:55.253 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.253 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.253 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:55.253 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535414+0000", 2024-09-16T20:50:55.253 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1675000000000003e-05 2024-09-16T20:50:55.253 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.253 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:55.253 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.253 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.253 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.253 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3578 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:55.253 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:50:55.253 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 125.739636336, 2024-09-16T20:50:55.253 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054189586999999997, 2024-09-16T20:50:55.253 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:55.254 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:55.254 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:55.254 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:55.254 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:55.254 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:55.254 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3578 2024-09-16T20:50:55.254 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.254 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:55.254 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.254 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:55.254 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:50:55.254 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.254 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.254 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.255 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:55.255 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:50:55.255 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.255 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.255 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.255 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:55.255 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:50:55.255 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0800000000000001e-07 2024-09-16T20:50:55.255 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.255 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.255 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:55.255 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481363+0000", 2024-09-16T20:50:55.255 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0339999999999999e-06 2024-09-16T20:50:55.255 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.256 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.256 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:55.256 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481371+0000", 2024-09-16T20:50:55.256 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4440000000000002e-06 2024-09-16T20:50:55.256 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.256 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.256 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:55.256 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481374+0000", 2024-09-16T20:50:55.256 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2049999999999998e-06 2024-09-16T20:50:55.256 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.256 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.256 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:55.256 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529555+0000", 2024-09-16T20:50:55.256 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048181202999999999 2024-09-16T20:50:55.256 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.257 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.257 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:55.257 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529616+0000", 2024-09-16T20:50:55.257 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0912000000000002e-05 2024-09-16T20:50:55.257 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.257 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.257 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:55.257 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529939+0000", 2024-09-16T20:50:55.257 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00032269800000000001 2024-09-16T20:50:55.257 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.257 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.257 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:55.257 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532602+0000", 2024-09-16T20:50:55.257 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0026634940000000002 2024-09-16T20:50:55.257 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.258 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.258 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:55.258 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535465+0000", 2024-09-16T20:50:55.258 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.002862149 2024-09-16T20:50:55.258 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.258 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.258 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:55.258 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535516+0000", 2024-09-16T20:50:55.258 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1696000000000001e-05 2024-09-16T20:50:55.258 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.258 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.258 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:55.258 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535551+0000", 2024-09-16T20:50:55.258 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4443999999999998e-05 2024-09-16T20:50:55.259 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.259 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:55.259 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.259 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.259 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.259 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3579 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:55.259 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:50:55.259 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 125.739617897, 2024-09-16T20:50:55.259 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054295151, 2024-09-16T20:50:55.259 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:55.259 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:55.259 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:55.259 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:55.259 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:55.259 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:55.260 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3579 2024-09-16T20:50:55.260 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.260 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:55.260 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.260 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:55.260 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:50:55.260 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.260 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.260 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.260 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:55.260 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:50:55.260 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.261 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.261 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.261 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:55.261 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:50:55.261 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3300000000000001e-07 2024-09-16T20:50:55.261 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.261 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.261 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:55.261 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481381+0000", 2024-09-16T20:50:55.261 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.649e-06 2024-09-16T20:50:55.261 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.261 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.261 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:55.261 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481387+0000", 2024-09-16T20:50:55.261 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4519999999999996e-06 2024-09-16T20:50:55.262 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.262 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.262 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:55.262 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481390+0000", 2024-09-16T20:50:55.262 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0170000000000001e-06 2024-09-16T20:50:55.262 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.262 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.262 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:55.262 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530756+0000", 2024-09-16T20:50:55.262 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049365882999999999 2024-09-16T20:50:55.262 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.262 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.262 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:55.262 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530845+0000", 2024-09-16T20:50:55.262 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9197000000000007e-05 2024-09-16T20:50:55.263 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.263 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.263 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:55.263 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531191+0000", 2024-09-16T20:50:55.263 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00034562500000000001 2024-09-16T20:50:55.263 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.263 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.263 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:55.263 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534078+0000", 2024-09-16T20:50:55.263 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0028878179999999999 2024-09-16T20:50:55.263 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.263 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.263 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:55.263 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535589+0000", 2024-09-16T20:50:55.263 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00151085 2024-09-16T20:50:55.264 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.264 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.264 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:55.264 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535639+0000", 2024-09-16T20:50:55.264 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9697000000000002e-05 2024-09-16T20:50:55.264 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.264 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.264 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:55.264 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535675+0000", 2024-09-16T20:50:55.264 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5729999999999998e-05 2024-09-16T20:50:55.264 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.264 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:55.264 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.265 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.265 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.265 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3580 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:55.265 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:50:55.265 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 125.739604265, 2024-09-16T20:50:55.265 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054397361999999998, 2024-09-16T20:50:55.265 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:55.265 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:55.265 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:55.265 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:55.265 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:55.265 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:55.265 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3580 2024-09-16T20:50:55.265 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.266 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:55.266 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.266 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:55.266 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:50:55.266 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.266 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.266 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.266 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:55.266 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:50:55.266 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.266 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.266 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.266 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:55.266 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:50:55.267 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2600000000000001e-07 2024-09-16T20:50:55.267 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.267 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.267 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:55.267 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481395+0000", 2024-09-16T20:50:55.267 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.544e-06 2024-09-16T20:50:55.267 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.267 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.267 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:55.267 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481404+0000", 2024-09-16T20:50:55.267 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8850000000000005e-06 2024-09-16T20:50:55.267 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.267 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.268 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:55.268 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481407+0000", 2024-09-16T20:50:55.268 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1540000000000002e-06 2024-09-16T20:50:55.268 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.268 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.268 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:55.268 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531823+0000", 2024-09-16T20:50:55.268 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050415637999999999 2024-09-16T20:50:55.268 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.268 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.268 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:55.268 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531895+0000", 2024-09-16T20:50:55.268 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2179000000000001e-05 2024-09-16T20:50:55.269 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.269 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.269 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:55.269 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532236+0000", 2024-09-16T20:50:55.269 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00034073799999999999 2024-09-16T20:50:55.269 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.269 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.269 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:55.269 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534083+0000", 2024-09-16T20:50:55.269 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001847298 2024-09-16T20:50:55.269 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.269 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.269 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:55.269 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535714+0000", 2024-09-16T20:50:55.270 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0016316810000000001 2024-09-16T20:50:55.270 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.270 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.270 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:55.270 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535759+0000", 2024-09-16T20:50:55.270 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4478999999999999e-05 2024-09-16T20:50:55.270 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.270 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.270 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:55.270 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535791+0000", 2024-09-16T20:50:55.270 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.154e-05 2024-09-16T20:50:55.270 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.270 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:55.270 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.271 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.271 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.271 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3581 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:50:55.271 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:50:55.271 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 125.73958226800001, 2024-09-16T20:50:55.271 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.057094403000000002, 2024-09-16T20:50:55.271 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:55.271 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:55.271 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:55.271 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:55.271 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:50:55.271 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:55.271 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3581 2024-09-16T20:50:55.272 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.272 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:55.272 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.272 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:55.272 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:50:55.272 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.272 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.272 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.272 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:55.272 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:50:55.272 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.272 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.272 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.272 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:55.273 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:50:55.273 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8799999999999998e-07 2024-09-16T20:50:55.273 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.273 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.273 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:55.273 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481418+0000", 2024-09-16T20:50:55.273 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1859999999999999e-06 2024-09-16T20:50:55.273 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.273 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.273 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:55.273 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481423+0000", 2024-09-16T20:50:55.273 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8279999999999999e-06 2024-09-16T20:50:55.274 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.274 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.274 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:55.274 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481426+0000", 2024-09-16T20:50:55.274 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0110000000000001e-06 2024-09-16T20:50:55.274 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.274 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.274 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:55.274 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532631+0000", 2024-09-16T20:50:55.274 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051204514999999999 2024-09-16T20:50:55.274 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.274 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.274 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:55.274 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.533381+0000", 2024-09-16T20:50:55.274 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00075029000000000003 2024-09-16T20:50:55.275 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.275 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.275 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:55.275 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.533739+0000", 2024-09-16T20:50:55.275 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00035773100000000002 2024-09-16T20:50:55.275 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.275 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.275 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:55.275 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535123+0000", 2024-09-16T20:50:55.275 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0013836160000000001 2024-09-16T20:50:55.275 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.275 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.275 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:55.275 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538413+0000", 2024-09-16T20:50:55.276 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0032902510000000001 2024-09-16T20:50:55.276 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.276 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.276 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:55.276 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538470+0000", 2024-09-16T20:50:55.276 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6697999999999998e-05 2024-09-16T20:50:55.276 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.276 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.276 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:55.276 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538510+0000", 2024-09-16T20:50:55.276 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9988999999999999e-05 2024-09-16T20:50:55.276 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.276 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:55.276 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.277 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.277 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.277 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:12732 3.d 3:b752444a:::rbd_header.110e747f0779:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-16T20:50:55.277 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:50:55.277 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 110.45250969, 2024-09-16T20:50:55.277 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.065302467000000003, 2024-09-16T20:50:55.277 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:55.277 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:55.277 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:55.277 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:55.277 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:50:55.277 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:50:55.277 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 12732 2024-09-16T20:50:55.278 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.278 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:55.278 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.278 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:55.278 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:50:55.278 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.278 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.278 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.278 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:55.278 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:50:55.278 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.278 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.278 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.278 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:55.278 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768490+0000", 2024-09-16T20:50:55.279 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2859999999999998e-06 2024-09-16T20:50:55.279 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.279 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.279 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:55.279 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768501+0000", 2024-09-16T20:50:55.279 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1226999999999999e-05 2024-09-16T20:50:55.279 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.279 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.279 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:55.279 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768517+0000", 2024-09-16T20:50:55.279 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.592e-05 2024-09-16T20:50:55.279 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.279 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.279 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:55.280 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768528+0000", 2024-09-16T20:50:55.280 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0689999999999999e-05 2024-09-16T20:50:55.280 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.280 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.280 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:55.280 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768584+0000", 2024-09-16T20:50:55.280 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5889e-05 2024-09-16T20:50:55.280 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.280 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.280 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:55.280 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768720+0000", 2024-09-16T20:50:55.280 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000136718 2024-09-16T20:50:55.280 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.280 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.281 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:55.281 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.769066+0000", 2024-09-16T20:50:55.281 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000346013 2024-09-16T20:50:55.281 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.281 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.281 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:50:55.281 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833659+0000", 2024-09-16T20:50:55.281 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.064592822999999994 2024-09-16T20:50:55.281 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.281 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.281 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:55.281 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833719+0000", 2024-09-16T20:50:55.281 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9441000000000002e-05 2024-09-16T20:50:55.281 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.281 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.282 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:55.282 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833758+0000", 2024-09-16T20:50:55.282 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9122000000000003e-05 2024-09-16T20:50:55.282 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.282 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.282 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:55.282 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833790+0000", 2024-09-16T20:50:55.282 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2338000000000001e-05 2024-09-16T20:50:55.282 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.282 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:55.282 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.282 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.282 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.282 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.5089.0:12732 3.d e73/23 ondisk, result = 0)", 2024-09-16T20:50:55.283 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:50:55.283 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 110.45090752, 2024-09-16T20:50:55.283 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.063706456999999994, 2024-09-16T20:50:55.283 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:55.283 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:55.283 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:55.283 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:55.283 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.283 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:55.283 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:50:55.283 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.283 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.283 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.283 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:55.283 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:50:55.284 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.284 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.284 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.284 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:55.284 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770091+0000", 2024-09-16T20:50:55.284 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.53e-06 2024-09-16T20:50:55.284 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.284 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.284 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:55.284 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770104+0000", 2024-09-16T20:50:55.284 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2203e-05 2024-09-16T20:50:55.284 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.284 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.284 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:55.284 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833637+0000", 2024-09-16T20:50:55.285 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.063533201999999997 2024-09-16T20:50:55.285 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.285 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.285 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:55.285 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833655+0000", 2024-09-16T20:50:55.285 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8369e-05 2024-09-16T20:50:55.285 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.285 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.285 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:55.285 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833707+0000", 2024-09-16T20:50:55.285 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2061999999999997e-05 2024-09-16T20:50:55.285 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.285 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.285 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:55.285 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833715+0000", 2024-09-16T20:50:55.286 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5179999999999998e-06 2024-09-16T20:50:55.286 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.286 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.286 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:55.286 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833796+0000", 2024-09-16T20:50:55.286 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1573e-05 2024-09-16T20:50:55.286 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.286 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:55.286 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.286 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.286 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.286 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3286 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 245760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:55.286 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:50:55.286 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 65.179845583000002, 2024-09-16T20:50:55.286 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052523564000000002, 2024-09-16T20:50:55.287 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:55.287 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:55.287 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:55.287 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:55.287 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:55.287 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:55.287 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3286 2024-09-16T20:50:55.287 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.287 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:55.287 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.287 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:55.287 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:50:55.287 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.287 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.288 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.288 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:55.288 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:50:55.288 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.288 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.288 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.288 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:55.288 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:50:55.288 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3900000000000001e-07 2024-09-16T20:50:55.288 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.288 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.288 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:55.288 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041153+0000", 2024-09-16T20:50:55.288 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.119e-06 2024-09-16T20:50:55.288 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.289 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.289 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:55.289 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041156+0000", 2024-09-16T20:50:55.289 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0000000000000001e-06 2024-09-16T20:50:55.289 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.289 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.289 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:55.289 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041159+0000", 2024-09-16T20:50:55.289 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5859999999999999e-06 2024-09-16T20:50:55.289 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.289 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.289 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:55.289 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093529+0000", 2024-09-16T20:50:55.290 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052370591000000001 2024-09-16T20:50:55.290 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.290 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.290 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:55.290 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093570+0000", 2024-09-16T20:50:55.290 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0161999999999999e-05 2024-09-16T20:50:55.290 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.290 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.290 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:55.290 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093675+0000", 2024-09-16T20:50:55.290 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010586699999999999 2024-09-16T20:50:55.290 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.290 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:55.290 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.290 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.291 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.291 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3287 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 250880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:55.291 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:50:55.291 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 65.179835534000006, 2024-09-16T20:50:55.291 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052781203999999998, 2024-09-16T20:50:55.291 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:55.291 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:55.291 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:55.291 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:55.291 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:55.291 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:55.291 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3287 2024-09-16T20:50:55.291 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.292 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:55.292 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.292 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:55.292 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:50:55.292 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.292 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.292 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.292 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:55.292 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:50:55.292 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.292 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.292 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.292 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:55.292 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:50:55.293 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1e-07 2024-09-16T20:50:55.293 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.293 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.293 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:55.293 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041163+0000", 2024-09-16T20:50:55.293 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0690000000000001e-06 2024-09-16T20:50:55.293 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.293 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.293 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:55.293 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041166+0000", 2024-09-16T20:50:55.293 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.982e-06 2024-09-16T20:50:55.293 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.293 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.294 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:55.294 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041169+0000", 2024-09-16T20:50:55.294 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.593e-06 2024-09-16T20:50:55.294 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.294 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.294 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:55.294 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093709+0000", 2024-09-16T20:50:55.294 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052540492000000001 2024-09-16T20:50:55.294 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.294 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.294 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:55.294 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093792+0000", 2024-09-16T20:50:55.294 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2436999999999997e-05 2024-09-16T20:50:55.294 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.295 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.295 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:55.295 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093943+0000", 2024-09-16T20:50:55.295 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000151421 2024-09-16T20:50:55.295 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.295 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:55.295 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.295 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.295 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.295 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3288 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 256000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:55.295 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:50:55.295 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 65.179825660999995, 2024-09-16T20:50:55.295 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052928269, 2024-09-16T20:50:55.296 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:55.296 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:55.296 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:55.296 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:55.296 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:55.296 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:55.296 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3288 2024-09-16T20:50:55.296 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.296 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:55.296 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.296 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:55.296 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:50:55.296 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.297 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.297 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.297 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:55.297 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:50:55.297 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.297 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.297 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.297 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:55.297 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:50:55.297 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2700000000000001e-07 2024-09-16T20:50:55.297 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.297 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.297 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:55.298 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041173+0000", 2024-09-16T20:50:55.298 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.113e-06 2024-09-16T20:50:55.298 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.298 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.298 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:55.298 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041176+0000", 2024-09-16T20:50:55.298 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9799999999999998e-06 2024-09-16T20:50:55.298 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.298 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.298 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:55.298 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041179+0000", 2024-09-16T20:50:55.298 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.667e-06 2024-09-16T20:50:55.298 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.298 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.298 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:55.299 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093957+0000", 2024-09-16T20:50:55.299 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052778703000000003 2024-09-16T20:50:55.299 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.299 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.299 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:55.299 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093999+0000", 2024-09-16T20:50:55.299 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1300000000000001e-05 2024-09-16T20:50:55.299 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.299 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.299 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:55.299 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094100+0000", 2024-09-16T20:50:55.299 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101279 2024-09-16T20:50:55.299 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.299 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:55.299 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.299 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.300 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.300 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3289 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 261120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:55.300 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:50:55.300 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 65.179815665000007, 2024-09-16T20:50:55.300 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053182976, 2024-09-16T20:50:55.300 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:55.300 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:55.300 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:55.300 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:55.300 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:50:55.300 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:55.300 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3289 2024-09-16T20:50:55.300 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.300 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:55.300 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.301 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:55.301 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:50:55.301 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.301 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.301 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.301 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:55.301 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:50:55.301 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.301 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.301 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.301 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:55.301 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:50:55.301 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2999999999999999e-07 2024-09-16T20:50:55.301 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.301 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.302 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:55.302 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041183+0000", 2024-09-16T20:50:55.302 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0720000000000001e-06 2024-09-16T20:50:55.302 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.302 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.302 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:55.302 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041186+0000", 2024-09-16T20:50:55.302 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9869999999999999e-06 2024-09-16T20:50:55.302 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.302 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.302 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:55.302 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041189+0000", 2024-09-16T20:50:55.302 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.587e-06 2024-09-16T20:50:55.302 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.302 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.303 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:55.303 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094136+0000", 2024-09-16T20:50:55.303 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052947270999999997 2024-09-16T20:50:55.303 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.303 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.303 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:55.303 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094213+0000", 2024-09-16T20:50:55.303 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7552000000000006e-05 2024-09-16T20:50:55.303 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.303 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.303 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:55.303 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094365+0000", 2024-09-16T20:50:55.303 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00015127700000000001 2024-09-16T20:50:55.304 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.304 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:55.304 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.304 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.304 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.304 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:2196 3.d 3:b1fe01a0:::rbd_data.110e7ef3004.0000000000000000:2d [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e136)", 2024-09-16T20:50:55.304 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:22.739057+0000", 2024-09-16T20:50:55.304 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.481940866000002, 2024-09-16T20:50:55.304 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11999464899999999, 2024-09-16T20:50:55.304 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:55.304 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:55.304 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:55.304 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:55.304 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:50:55.305 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:55.305 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2196 2024-09-16T20:50:55.305 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.305 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:55.305 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.305 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:55.305 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.739057+0000", 2024-09-16T20:50:55.305 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.305 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.305 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.305 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:55.305 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.739057+0000", 2024-09-16T20:50:55.305 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.305 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.305 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.306 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:55.306 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.739057+0000", 2024-09-16T20:50:55.306 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1399999999999997e-07 2024-09-16T20:50:55.306 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.306 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.306 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:55.306 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.739060+0000", 2024-09-16T20:50:55.306 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6019999999999998e-06 2024-09-16T20:50:55.306 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.306 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.306 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:55.306 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.858755+0000", 2024-09-16T20:50:55.306 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.119695151 2024-09-16T20:50:55.306 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.306 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.307 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:55.307 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.858778+0000", 2024-09-16T20:50:55.307 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3184999999999999e-05 2024-09-16T20:50:55.307 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.307 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.307 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:55.307 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.858865+0000", 2024-09-16T20:50:55.307 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.7330000000000006e-05 2024-09-16T20:50:55.307 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.307 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.307 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:55.307 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.858952+0000", 2024-09-16T20:50:55.307 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6803000000000002e-05 2024-09-16T20:50:55.307 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.308 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.308 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:55.308 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.859051+0000", 2024-09-16T20:50:55.308 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8963999999999996e-05 2024-09-16T20:50:55.308 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.308 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:55.308 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.308 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.308 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.308 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6061.0:5401 3.13 3:c8554348:::rbd_data.110e16eafbd.0000000000000011:head [read 577536~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-16T20:50:55.308 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:27.761536+0000", 2024-09-16T20:50:55.308 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 27.459461052999998, 2024-09-16T20:50:55.308 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.003742712, 2024-09-16T20:50:55.309 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:55.309 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:55.309 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:50:55.309 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:55.309 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6061", 2024-09-16T20:50:55.309 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3930128440", 2024-09-16T20:50:55.309 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5401 2024-09-16T20:50:55.309 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.309 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:55.309 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.309 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:55.309 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761536+0000", 2024-09-16T20:50:55.309 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.309 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.310 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.310 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:55.310 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761536+0000", 2024-09-16T20:50:55.310 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.310 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.310 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.310 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:55.310 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761537+0000", 2024-09-16T20:50:55.310 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2900000000000003e-07 2024-09-16T20:50:55.310 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.310 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.310 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:55.310 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761540+0000", 2024-09-16T20:50:55.310 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2320000000000001e-06 2024-09-16T20:50:55.310 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.311 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.311 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:55.311 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761548+0000", 2024-09-16T20:50:55.311 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1289999999999996e-06 2024-09-16T20:50:55.311 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.311 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.311 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:55.311 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761554+0000", 2024-09-16T20:50:55.311 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0820000000000004e-06 2024-09-16T20:50:55.311 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.311 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.311 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:55.311 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761621+0000", 2024-09-16T20:50:55.311 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6266000000000006e-05 2024-09-16T20:50:55.311 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.312 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.312 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:55.312 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761626+0000", 2024-09-16T20:50:55.312 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.614e-06 2024-09-16T20:50:55.312 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.312 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.312 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:55.312 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:28.765163+0000", 2024-09-16T20:50:55.312 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.003536416 2024-09-16T20:50:55.312 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.312 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.312 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:55.312 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:28.765279+0000", 2024-09-16T20:50:55.312 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000116344 2024-09-16T20:50:55.312 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.313 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:55.313 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.313 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.313 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.313 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6061.0:5425 3.13 3:c8554348:::rbd_data.110e16eafbd.0000000000000011:head [read 602112~28672] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-16T20:50:55.313 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:27.767049+0000", 2024-09-16T20:50:55.313 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 27.453948394000001, 2024-09-16T20:50:55.313 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.99847897900000004, 2024-09-16T20:50:55.313 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:55.313 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:55.313 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:50:55.313 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:55.313 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6061", 2024-09-16T20:50:55.314 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3930128440", 2024-09-16T20:50:55.314 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5425 2024-09-16T20:50:55.314 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.314 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:55.314 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.314 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:55.314 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767049+0000", 2024-09-16T20:50:55.314 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.314 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.314 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.314 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:55.314 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767049+0000", 2024-09-16T20:50:55.314 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.314 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.315 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.315 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:55.315 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767050+0000", 2024-09-16T20:50:55.315 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6499999999999999e-07 2024-09-16T20:50:55.315 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.315 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.315 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:55.315 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767052+0000", 2024-09-16T20:50:55.315 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7350000000000001e-06 2024-09-16T20:50:55.315 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.315 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.315 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:55.315 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767060+0000", 2024-09-16T20:50:55.315 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5109999999999997e-06 2024-09-16T20:50:55.315 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.316 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.316 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:55.316 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767066+0000", 2024-09-16T20:50:55.316 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8139999999999997e-06 2024-09-16T20:50:55.316 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.316 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.316 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:55.316 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767128+0000", 2024-09-16T20:50:55.316 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2174999999999993e-05 2024-09-16T20:50:55.316 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.316 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.316 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:55.316 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767133+0000", 2024-09-16T20:50:55.317 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2140000000000004e-06 2024-09-16T20:50:55.317 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.317 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.317 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:55.317 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:28.765441+0000", 2024-09-16T20:50:55.317 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.99830808599999998 2024-09-16T20:50:55.317 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.317 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.317 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:55.317 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:28.765528+0000", 2024-09-16T20:50:55.317 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6879000000000006e-05 2024-09-16T20:50:55.317 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.317 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:55.317 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.318 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.318 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.318 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:25585 3.10 3:0c7c5b23:::rbd_header.110ebf0678ad:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:50:55.318 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:32.877132+0000", 2024-09-16T20:50:55.318 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 22.343865845, 2024-09-16T20:50:55.318 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054096699999999998, 2024-09-16T20:50:55.318 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:55.318 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:55.318 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:55.318 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:55.318 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:50:55.318 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:55.318 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 25585 2024-09-16T20:50:55.318 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.319 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:55.319 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.319 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:55.319 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.877132+0000", 2024-09-16T20:50:55.319 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.319 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.319 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.319 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:55.319 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.877132+0000", 2024-09-16T20:50:55.319 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.319 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.319 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.319 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:55.319 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931072+0000", 2024-09-16T20:50:55.320 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053939931000000003 2024-09-16T20:50:55.320 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.320 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.320 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:55.320 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931076+0000", 2024-09-16T20:50:55.320 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2860000000000001e-06 2024-09-16T20:50:55.320 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.320 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.320 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:55.320 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931085+0000", 2024-09-16T20:50:55.320 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.3370000000000006e-06 2024-09-16T20:50:55.320 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.320 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.321 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:55.321 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931092+0000", 2024-09-16T20:50:55.321 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9970000000000001e-06 2024-09-16T20:50:55.321 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.321 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.321 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:55.321 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931137+0000", 2024-09-16T20:50:55.321 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5160000000000001e-05 2024-09-16T20:50:55.321 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.321 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.321 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:55.321 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931178+0000", 2024-09-16T20:50:55.321 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0379000000000001e-05 2024-09-16T20:50:55.321 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.322 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.322 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:55.322 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931228+0000", 2024-09-16T20:50:55.322 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0609999999999998e-05 2024-09-16T20:50:55.322 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.322 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:55.322 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.322 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.322 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.322 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:26679 3.c 3:33e2f78f:::rbd_header.110e16eafbd:head [watch ping cookie 140213436013408 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e153)", 2024-09-16T20:50:55.322 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:41.440287+0000", 2024-09-16T20:50:55.322 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 13.78070997, 2024-09-16T20:50:55.322 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.13295633600000001, 2024-09-16T20:50:55.322 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:55.322 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:55.323 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:55.323 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:55.323 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:50:55.323 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:55.323 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 26679 2024-09-16T20:50:55.323 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.323 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:55.323 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.323 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:55.323 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440287+0000", 2024-09-16T20:50:55.323 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.323 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.323 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.323 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:55.324 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440287+0000", 2024-09-16T20:50:55.324 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.324 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.324 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.324 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:55.324 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440288+0000", 2024-09-16T20:50:55.324 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0900000000000001e-07 2024-09-16T20:50:55.324 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.324 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.324 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:55.324 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440292+0000", 2024-09-16T20:50:55.324 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3749999999999996e-06 2024-09-16T20:50:55.324 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.324 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.324 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:55.325 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440299+0000", 2024-09-16T20:50:55.325 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8249999999999999e-06 2024-09-16T20:50:55.325 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.325 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.325 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:55.325 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440307+0000", 2024-09-16T20:50:55.325 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4800000000000004e-06 2024-09-16T20:50:55.325 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.325 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.325 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:55.325 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440451+0000", 2024-09-16T20:50:55.325 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014414500000000001 2024-09-16T20:50:55.325 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.325 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.326 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:50:55.326 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440465+0000", 2024-09-16T20:50:55.326 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.402e-05 2024-09-16T20:50:55.326 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.326 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.326 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:55.326 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.573043+0000", 2024-09-16T20:50:55.326 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.13257855399999999 2024-09-16T20:50:55.326 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.326 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.326 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:55.326 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.573130+0000", 2024-09-16T20:50:55.326 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6168999999999994e-05 2024-09-16T20:50:55.326 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.326 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.327 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:55.327 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.573244+0000", 2024-09-16T20:50:55.327 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000114159 2024-09-16T20:50:55.327 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.327 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:55.327 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.327 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.327 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:55.327 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:55.329 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:50:55.329 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:55.328+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:55.429 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:55.428+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:55.478 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:55.478 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:50:55.478 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:50:55.478 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:55.486 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:50:55.530 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:55.529+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:55.568 INFO:teuthology.orchestra.run.smithi029.stderr:noscrub is set 2024-09-16T20:50:55.605 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:55.605 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:50:55.605 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:50:55.606 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:50:55.606 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:55.616 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:50:55.630 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:55.629+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:55.731 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:55.729+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:55.790 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:50:55.790 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:50:55.790 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:50:55.790 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:50:55.790 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.790 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3714 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1480704~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:55.790 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:50:55.790 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 153.14918076399999, 2024-09-16T20:50:55.790 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050152552000000003, 2024-09-16T20:50:55.790 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:55.790 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:55.791 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:55.791 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:55.791 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:55.791 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:55.791 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3714 2024-09-16T20:50:55.791 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.791 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:55.791 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.791 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:55.791 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:50:55.791 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.791 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.791 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.792 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:55.792 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:50:55.792 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.792 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.792 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.792 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:55.792 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:50:55.792 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0499999999999999e-07 2024-09-16T20:50:55.792 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.792 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.792 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:55.792 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639007+0000", 2024-09-16T20:50:55.792 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5340000000000001e-06 2024-09-16T20:50:55.793 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.793 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.793 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:55.793 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639013+0000", 2024-09-16T20:50:55.793 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3260000000000003e-06 2024-09-16T20:50:55.793 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.793 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.793 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:55.793 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639018+0000", 2024-09-16T20:50:55.793 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2850000000000002e-06 2024-09-16T20:50:55.793 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.793 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.793 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:55.794 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689026+0000", 2024-09-16T20:50:55.794 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.05000798 2024-09-16T20:50:55.794 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.794 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.794 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:55.794 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689077+0000", 2024-09-16T20:50:55.794 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1063999999999998e-05 2024-09-16T20:50:55.794 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.794 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.794 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:55.794 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689158+0000", 2024-09-16T20:50:55.794 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1057999999999995e-05 2024-09-16T20:50:55.794 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.794 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:55.795 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.795 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.795 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.795 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3716 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1488896~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:55.795 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:50:55.795 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 153.14916295399999, 2024-09-16T20:50:55.795 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050280142, 2024-09-16T20:50:55.795 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:55.795 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:55.795 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:55.795 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:55.795 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:55.795 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:55.795 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3716 2024-09-16T20:50:55.796 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.796 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:55.796 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.796 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:55.796 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:50:55.796 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.796 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.796 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.796 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:55.796 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:50:55.796 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.796 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.796 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.797 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:55.797 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639024+0000", 2024-09-16T20:50:55.797 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9999999999999997e-07 2024-09-16T20:50:55.797 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.797 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.797 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:55.797 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639026+0000", 2024-09-16T20:50:55.797 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.781e-06 2024-09-16T20:50:55.797 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.797 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.797 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:55.797 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639032+0000", 2024-09-16T20:50:55.797 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7780000000000004e-06 2024-09-16T20:50:55.797 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.798 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.798 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:55.798 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639036+0000", 2024-09-16T20:50:55.798 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6909999999999996e-06 2024-09-16T20:50:55.798 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.798 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.798 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:55.798 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689191+0000", 2024-09-16T20:50:55.798 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050155284000000001 2024-09-16T20:50:55.798 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.798 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.798 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:55.798 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689242+0000", 2024-09-16T20:50:55.799 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0662999999999999e-05 2024-09-16T20:50:55.799 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.799 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.799 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:55.799 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689304+0000", 2024-09-16T20:50:55.799 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1345000000000003e-05 2024-09-16T20:50:55.799 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.799 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:55.799 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.799 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.799 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.799 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3719 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1497088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:55.799 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:50:55.799 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 153.14913567599999, 2024-09-16T20:50:55.800 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050399715999999997, 2024-09-16T20:50:55.800 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:55.800 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:55.800 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:55.800 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:55.800 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:55.800 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:55.800 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3719 2024-09-16T20:50:55.800 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.800 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:55.800 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.800 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:55.801 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:50:55.801 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.801 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.801 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.801 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:55.801 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:50:55.801 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.801 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.801 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.801 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:55.801 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:50:55.801 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9700000000000002e-07 2024-09-16T20:50:55.801 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.802 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.802 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:55.802 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639053+0000", 2024-09-16T20:50:55.802 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2900000000000001e-06 2024-09-16T20:50:55.802 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.802 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.802 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:55.802 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639060+0000", 2024-09-16T20:50:55.802 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7750000000000004e-06 2024-09-16T20:50:55.802 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.802 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.802 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:55.802 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639066+0000", 2024-09-16T20:50:55.802 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0639999999999999e-06 2024-09-16T20:50:55.803 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.803 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.803 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:55.803 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689337+0000", 2024-09-16T20:50:55.803 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050270784999999998 2024-09-16T20:50:55.803 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.803 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.803 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:55.803 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689392+0000", 2024-09-16T20:50:55.803 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4920999999999998e-05 2024-09-16T20:50:55.803 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.803 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.803 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:55.803 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689450+0000", 2024-09-16T20:50:55.804 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8483999999999998e-05 2024-09-16T20:50:55.804 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.804 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:55.804 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.804 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.804 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.804 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3721 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1505280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:55.804 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:50:55.805 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 153.14911254399999, 2024-09-16T20:50:55.805 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050513608000000002, 2024-09-16T20:50:55.805 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:55.805 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:55.805 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:55.805 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:55.805 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:55.805 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:55.805 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3721 2024-09-16T20:50:55.805 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.805 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:55.805 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.805 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:55.806 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:50:55.806 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.806 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.806 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.806 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:55.806 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:50:55.806 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.806 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.806 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.806 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:55.806 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:50:55.806 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7e-07 2024-09-16T20:50:55.806 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.807 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.807 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:55.807 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639076+0000", 2024-09-16T20:50:55.807 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8899999999999999e-06 2024-09-16T20:50:55.807 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.807 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.807 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:55.807 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639083+0000", 2024-09-16T20:50:55.807 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1210000000000001e-06 2024-09-16T20:50:55.807 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.807 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.807 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:55.807 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639089+0000", 2024-09-16T20:50:55.808 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.643e-06 2024-09-16T20:50:55.808 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.808 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.808 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:55.808 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689474+0000", 2024-09-16T20:50:55.808 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050385679000000003 2024-09-16T20:50:55.808 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.808 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.808 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:55.808 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689533+0000", 2024-09-16T20:50:55.808 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8303000000000001e-05 2024-09-16T20:50:55.808 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.808 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.809 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:55.809 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689587+0000", 2024-09-16T20:50:55.809 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4601999999999997e-05 2024-09-16T20:50:55.809 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.809 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:55.809 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.809 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.809 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.809 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3723 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1513472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:55.809 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:50:55.809 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 153.14908174000001, 2024-09-16T20:50:55.809 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050699691999999998, 2024-09-16T20:50:55.809 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:55.809 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:55.810 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:55.810 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:55.810 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:55.810 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:55.810 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3723 2024-09-16T20:50:55.810 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.810 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:55.810 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.810 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:55.810 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:50:55.810 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.810 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.811 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.811 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:55.811 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:50:55.811 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.811 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.811 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.811 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:55.811 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:50:55.811 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3399999999999999e-07 2024-09-16T20:50:55.811 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.811 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.811 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:55.811 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639107+0000", 2024-09-16T20:50:55.811 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3860000000000001e-06 2024-09-16T20:50:55.812 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.812 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.812 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:55.812 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639114+0000", 2024-09-16T20:50:55.812 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9800000000000001e-06 2024-09-16T20:50:55.812 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.812 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.812 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:55.812 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639129+0000", 2024-09-16T20:50:55.812 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4887e-05 2024-09-16T20:50:55.812 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.812 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.812 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:55.813 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689625+0000", 2024-09-16T20:50:55.813 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050495848000000003 2024-09-16T20:50:55.813 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.813 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.813 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:55.813 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689720+0000", 2024-09-16T20:50:55.813 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4821000000000006e-05 2024-09-16T20:50:55.813 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.813 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.813 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:55.813 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689804+0000", 2024-09-16T20:50:55.813 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4235999999999998e-05 2024-09-16T20:50:55.813 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.813 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:55.813 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.814 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.814 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.814 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3725 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1521664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:55.814 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:50:55.814 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 153.14904814400001, 2024-09-16T20:50:55.814 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050797660000000001, 2024-09-16T20:50:55.814 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:55.814 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:55.814 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:55.814 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:55.814 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:55.814 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:55.814 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3725 2024-09-16T20:50:55.815 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.815 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:55.815 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.815 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:55.815 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:50:55.815 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.815 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.815 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.815 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:55.815 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:50:55.815 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.815 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.815 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.815 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:55.816 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639139+0000", 2024-09-16T20:50:55.816 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8999999999999997e-07 2024-09-16T20:50:55.816 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.816 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.816 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:55.816 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639141+0000", 2024-09-16T20:50:55.816 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.182e-06 2024-09-16T20:50:55.816 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.816 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.816 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:55.816 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639154+0000", 2024-09-16T20:50:55.816 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3626e-05 2024-09-16T20:50:55.816 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.816 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.816 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:55.817 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639160+0000", 2024-09-16T20:50:55.817 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6810000000000003e-06 2024-09-16T20:50:55.817 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.817 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.817 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:55.817 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689831+0000", 2024-09-16T20:50:55.817 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050670907000000001 2024-09-16T20:50:55.817 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.817 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.817 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:55.817 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689882+0000", 2024-09-16T20:50:55.817 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0445999999999998e-05 2024-09-16T20:50:55.817 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.817 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.818 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:55.818 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689936+0000", 2024-09-16T20:50:55.818 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4327999999999999e-05 2024-09-16T20:50:55.818 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.818 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:55.818 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.818 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.818 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.818 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3727 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1529856~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:55.818 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:50:55.818 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 153.149012567, 2024-09-16T20:50:55.818 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051016922999999999, 2024-09-16T20:50:55.818 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:55.818 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:55.818 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:55.818 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:55.819 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:55.819 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:55.819 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3727 2024-09-16T20:50:55.819 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.819 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:55.819 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.819 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:55.819 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:50:55.819 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.819 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.819 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.819 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:55.819 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:50:55.819 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.819 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.820 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.820 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:55.820 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:50:55.820 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9999999999999998e-07 2024-09-16T20:50:55.820 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.820 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.820 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:55.820 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639183+0000", 2024-09-16T20:50:55.820 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2399999999999996e-06 2024-09-16T20:50:55.820 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.820 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.820 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:55.820 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639195+0000", 2024-09-16T20:50:55.820 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1113e-05 2024-09-16T20:50:55.821 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.821 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.821 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:55.821 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639206+0000", 2024-09-16T20:50:55.821 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1503e-05 2024-09-16T20:50:55.821 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.821 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.821 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:55.821 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689969+0000", 2024-09-16T20:50:55.821 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050762676 2024-09-16T20:50:55.821 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.821 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.821 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:55.821 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690071+0000", 2024-09-16T20:50:55.821 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102163 2024-09-16T20:50:55.822 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.822 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.822 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:55.822 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690191+0000", 2024-09-16T20:50:55.822 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000119728 2024-09-16T20:50:55.822 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.822 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:55.822 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.822 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.822 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.822 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3730 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1538048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:55.822 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:50:55.822 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 153.148971004, 2024-09-16T20:50:55.822 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051131467999999999, 2024-09-16T20:50:55.823 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:55.823 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:55.823 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:55.823 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:55.823 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:55.823 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:55.823 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3730 2024-09-16T20:50:55.823 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.823 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:55.823 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.823 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:55.823 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:50:55.823 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.823 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.823 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.824 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:55.824 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:50:55.824 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.824 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.824 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.824 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:55.824 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639216+0000", 2024-09-16T20:50:55.824 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.39e-07 2024-09-16T20:50:55.824 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.824 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.824 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:55.824 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639218+0000", 2024-09-16T20:50:55.824 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.537e-06 2024-09-16T20:50:55.824 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.825 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.825 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:55.825 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639229+0000", 2024-09-16T20:50:55.825 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1122e-05 2024-09-16T20:50:55.825 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.825 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.825 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:55.825 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639237+0000", 2024-09-16T20:50:55.825 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8509999999999999e-06 2024-09-16T20:50:55.825 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.825 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.825 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:55.825 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690227+0000", 2024-09-16T20:50:55.825 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050990147 2024-09-16T20:50:55.825 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.826 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.826 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:55.826 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690278+0000", 2024-09-16T20:50:55.826 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0223999999999999e-05 2024-09-16T20:50:55.826 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.826 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.826 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:55.826 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690347+0000", 2024-09-16T20:50:55.826 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9147999999999996e-05 2024-09-16T20:50:55.826 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.826 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:55.826 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.826 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.826 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.827 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3731 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1546240~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:55.827 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:50:55.827 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 153.14894029600001, 2024-09-16T20:50:55.827 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051276744999999999, 2024-09-16T20:50:55.827 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:55.827 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:55.827 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:55.827 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:55.827 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:55.827 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:55.827 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3731 2024-09-16T20:50:55.827 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.827 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:55.827 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.827 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:55.828 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:50:55.828 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.828 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.828 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.828 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:55.828 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:50:55.828 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.828 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.828 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.828 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:55.828 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:50:55.828 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9900000000000001e-07 2024-09-16T20:50:55.828 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.828 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.829 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:55.829 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639249+0000", 2024-09-16T20:50:55.829 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3630000000000001e-06 2024-09-16T20:50:55.829 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.829 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.829 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:55.829 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639255+0000", 2024-09-16T20:50:55.829 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4960000000000001e-06 2024-09-16T20:50:55.829 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.829 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.829 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:55.829 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639261+0000", 2024-09-16T20:50:55.829 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2839999999999997e-06 2024-09-16T20:50:55.829 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.829 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.830 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:55.830 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690375+0000", 2024-09-16T20:50:55.830 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051114684000000001 2024-09-16T20:50:55.830 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.830 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.830 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:55.830 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690425+0000", 2024-09-16T20:50:55.830 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9379999999999998e-05 2024-09-16T20:50:55.830 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.830 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.830 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:55.830 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690523+0000", 2024-09-16T20:50:55.830 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8139000000000003e-05 2024-09-16T20:50:55.830 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.830 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:55.831 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.831 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.831 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.831 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3732 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1554432~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:55.831 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:50:55.831 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 153.14889451400001, 2024-09-16T20:50:55.831 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051380096, 2024-09-16T20:50:55.831 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:55.831 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:55.831 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:55.831 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:55.831 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:55.831 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:55.831 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3732 2024-09-16T20:50:55.832 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.832 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:55.832 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.832 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:55.832 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:50:55.832 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.832 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.832 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.832 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:55.832 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:50:55.832 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.832 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.832 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.832 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:55.833 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:50:55.833 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6100000000000001e-07 2024-09-16T20:50:55.833 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.833 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.833 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:55.833 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639295+0000", 2024-09-16T20:50:55.833 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.447e-06 2024-09-16T20:50:55.833 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.833 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.833 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:55.833 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639302+0000", 2024-09-16T20:50:55.833 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0570000000000004e-06 2024-09-16T20:50:55.833 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.833 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.833 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:55.834 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639307+0000", 2024-09-16T20:50:55.834 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1429999999999997e-06 2024-09-16T20:50:55.834 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.834 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.834 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:55.834 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690555+0000", 2024-09-16T20:50:55.834 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051247963000000001 2024-09-16T20:50:55.834 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.834 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.834 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:55.834 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690602+0000", 2024-09-16T20:50:55.834 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7602000000000003e-05 2024-09-16T20:50:55.834 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.834 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.834 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:55.835 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690672+0000", 2024-09-16T20:50:55.835 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9423000000000003e-05 2024-09-16T20:50:55.835 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.835 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:55.835 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.835 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.835 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.835 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3734 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1562624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:55.835 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:50:55.835 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 153.148872864, 2024-09-16T20:50:55.835 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051490610999999999, 2024-09-16T20:50:55.835 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:55.835 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:55.835 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:55.835 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:55.836 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:55.836 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:55.836 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3734 2024-09-16T20:50:55.836 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.836 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:55.836 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.836 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:55.836 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:50:55.836 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.836 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.836 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.836 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:55.836 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:50:55.836 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.836 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.837 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.837 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:55.837 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639314+0000", 2024-09-16T20:50:55.837 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4400000000000001e-07 2024-09-16T20:50:55.837 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.837 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.837 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:55.837 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639316+0000", 2024-09-16T20:50:55.837 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1459999999999999e-06 2024-09-16T20:50:55.837 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.837 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.837 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:55.837 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639322+0000", 2024-09-16T20:50:55.837 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7610000000000004e-06 2024-09-16T20:50:55.838 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.838 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.838 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:55.838 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639327+0000", 2024-09-16T20:50:55.838 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.186e-06 2024-09-16T20:50:55.838 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.838 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.838 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:55.838 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690698+0000", 2024-09-16T20:50:55.838 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051371223000000001 2024-09-16T20:50:55.838 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.838 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.838 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:55.838 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690748+0000", 2024-09-16T20:50:55.839 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0192999999999999e-05 2024-09-16T20:50:55.839 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.839 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.839 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:55.839 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690804+0000", 2024-09-16T20:50:55.839 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5658000000000001e-05 2024-09-16T20:50:55.839 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.839 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:55.839 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.839 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.839 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.839 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3736 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1570816~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:55.839 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:50:55.839 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 153.148853605, 2024-09-16T20:50:55.839 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051596528000000003, 2024-09-16T20:50:55.840 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:55.840 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:55.840 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:55.840 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:55.840 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:55.840 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:55.840 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3736 2024-09-16T20:50:55.840 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.840 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:55.840 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.840 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:55.840 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:50:55.840 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.841 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.841 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.841 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:55.841 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:50:55.841 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.841 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.841 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.841 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:55.841 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:50:55.841 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9999999999999999e-07 2024-09-16T20:50:55.841 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.841 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.841 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:55.842 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639335+0000", 2024-09-16T20:50:55.842 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.635e-06 2024-09-16T20:50:55.842 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.842 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.842 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:55.842 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639340+0000", 2024-09-16T20:50:55.842 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5840000000000002e-06 2024-09-16T20:50:55.842 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.842 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.842 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:55.842 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639346+0000", 2024-09-16T20:50:55.842 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1190000000000002e-06 2024-09-16T20:50:55.842 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.842 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.843 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:55.843 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690817+0000", 2024-09-16T20:50:55.843 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051470721999999997 2024-09-16T20:50:55.843 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.843 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.843 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:55.843 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690875+0000", 2024-09-16T20:50:55.843 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7441000000000001e-05 2024-09-16T20:50:55.843 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.843 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.843 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:55.843 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690929+0000", 2024-09-16T20:50:55.843 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4727e-05 2024-09-16T20:50:55.843 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.843 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:55.844 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.844 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.844 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.844 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5434 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:55.844 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:50:55.844 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 152.923592039, 2024-09-16T20:50:55.844 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099755831000000003, 2024-09-16T20:50:55.844 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:55.844 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:55.844 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:55.844 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:55.844 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:55.844 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:55.844 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5434 2024-09-16T20:50:55.844 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.845 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:55.845 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.845 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:55.845 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:50:55.845 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.845 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.845 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.845 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:55.845 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:50:55.845 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.845 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.845 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.845 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:55.845 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864595+0000", 2024-09-16T20:50:55.845 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1099999999999995e-07 2024-09-16T20:50:55.846 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.846 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.846 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:55.846 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864597+0000", 2024-09-16T20:50:55.846 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3989999999999998e-06 2024-09-16T20:50:55.846 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.846 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.846 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:55.846 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864603+0000", 2024-09-16T20:50:55.846 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9309999999999996e-06 2024-09-16T20:50:55.846 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.846 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.846 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:55.846 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864607+0000", 2024-09-16T20:50:55.846 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6200000000000001e-06 2024-09-16T20:50:55.847 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.847 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.847 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:55.847 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964164+0000", 2024-09-16T20:50:55.847 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099557162000000005 2024-09-16T20:50:55.847 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.847 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.847 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:55.847 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964244+0000", 2024-09-16T20:50:55.847 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9597000000000004e-05 2024-09-16T20:50:55.847 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.847 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.848 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:55.848 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964350+0000", 2024-09-16T20:50:55.848 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010641099999999999 2024-09-16T20:50:55.848 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.848 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:55.848 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.848 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.848 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.848 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5435 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 20480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:50:55.848 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:50:55.848 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 152.92356221700001, 2024-09-16T20:50:55.848 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100897102, 2024-09-16T20:50:55.848 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:55.848 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:55.848 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:55.849 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:55.849 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:50:55.849 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:50:55.849 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5435 2024-09-16T20:50:55.849 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.849 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:55.849 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.849 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:55.849 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:50:55.849 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.849 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.849 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.849 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:55.849 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:50:55.850 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.850 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.850 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.850 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:55.850 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:50:55.850 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8000000000000001e-07 2024-09-16T20:50:55.850 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.850 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.850 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:55.850 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864626+0000", 2024-09-16T20:50:55.851 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.745e-06 2024-09-16T20:50:55.851 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.851 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.851 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:55.851 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864633+0000", 2024-09-16T20:50:55.851 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3339999999999998e-06 2024-09-16T20:50:55.851 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.851 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.851 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:55.851 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864636+0000", 2024-09-16T20:50:55.851 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1200000000000002e-06 2024-09-16T20:50:55.851 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.851 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.851 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:55.852 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965299+0000", 2024-09-16T20:50:55.852 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10066352000000001 2024-09-16T20:50:55.852 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.852 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.852 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:55.852 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965390+0000", 2024-09-16T20:50:55.852 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0894000000000001e-05 2024-09-16T20:50:55.852 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.852 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.852 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:55.852 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965521+0000", 2024-09-16T20:50:55.852 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013110899999999999 2024-09-16T20:50:55.852 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.853 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:55.853 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.853 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.853 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.853 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:152 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 327680~32768 out=32768b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:50:55.853 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:50:55.853 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 151.625065275, 2024-09-16T20:50:55.853 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11869405800000001, 2024-09-16T20:50:55.853 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:55.853 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:55.853 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:55.853 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:55.853 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:50:55.854 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:50:55.854 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 152 2024-09-16T20:50:55.854 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.854 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:55.854 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.854 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:55.854 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:50:55.854 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.854 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.854 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.854 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:55.854 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:50:55.854 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.854 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.854 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.855 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:55.855 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163122+0000", 2024-09-16T20:50:55.855 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6400000000000003e-07 2024-09-16T20:50:55.855 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.855 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.855 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:55.855 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163124+0000", 2024-09-16T20:50:55.855 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.565e-06 2024-09-16T20:50:55.855 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.855 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.855 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:55.855 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163136+0000", 2024-09-16T20:50:55.855 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1717e-05 2024-09-16T20:50:55.855 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.855 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.855 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:55.856 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163142+0000", 2024-09-16T20:50:55.856 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1750000000000002e-06 2024-09-16T20:50:55.856 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.856 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.856 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:55.856 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163606+0000", 2024-09-16T20:50:55.856 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00046418799999999998 2024-09-16T20:50:55.856 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.856 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.856 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:55.856 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163656+0000", 2024-09-16T20:50:55.856 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9852999999999999e-05 2024-09-16T20:50:55.856 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.856 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.857 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:55.857 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281815+0000", 2024-09-16T20:50:55.857 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11815909600000001 2024-09-16T20:50:55.857 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.857 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:55.857 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.857 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.857 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.857 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:153 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:50:55.857 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:50:55.857 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 151.62503232200001, 2024-09-16T20:50:55.857 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11903462400000001, 2024-09-16T20:50:55.857 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:55.857 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:55.857 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:55.858 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:55.858 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:50:55.858 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:50:55.858 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 153 2024-09-16T20:50:55.858 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.858 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:55.858 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.858 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:55.858 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:50:55.858 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.858 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.858 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.858 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:55.858 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:50:55.859 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.859 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.859 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.859 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:55.859 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163155+0000", 2024-09-16T20:50:55.859 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.37e-07 2024-09-16T20:50:55.859 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.859 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.859 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:55.859 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163157+0000", 2024-09-16T20:50:55.859 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6010000000000002e-06 2024-09-16T20:50:55.859 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.859 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.859 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:55.859 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163165+0000", 2024-09-16T20:50:55.859 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2619999999999998e-06 2024-09-16T20:50:55.860 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.860 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.860 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:55.860 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163171+0000", 2024-09-16T20:50:55.860 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.711e-06 2024-09-16T20:50:55.860 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.860 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.860 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:55.860 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281868+0000", 2024-09-16T20:50:55.860 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118696622 2024-09-16T20:50:55.860 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.860 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.860 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:55.861 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281969+0000", 2024-09-16T20:50:55.861 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101371 2024-09-16T20:50:55.861 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.861 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.861 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:55.861 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282189+0000", 2024-09-16T20:50:55.861 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00021952000000000001 2024-09-16T20:50:55.861 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.861 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:55.861 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.861 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.861 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.861 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:154 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 360448~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:50:55.861 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:50:55.861 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 151.62500055000001, 2024-09-16T20:50:55.862 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11932035100000001, 2024-09-16T20:50:55.862 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:55.862 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:55.862 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:55.862 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:55.862 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:50:55.862 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:50:55.862 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 154 2024-09-16T20:50:55.862 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.862 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:55.862 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.862 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:55.862 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:50:55.862 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.863 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.863 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.863 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:55.863 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:50:55.863 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.863 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.863 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.863 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:55.863 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:50:55.863 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:50:55.863 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.863 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.863 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:55.863 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163189+0000", 2024-09-16T20:50:55.863 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6840000000000001e-06 2024-09-16T20:50:55.864 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.864 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.864 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:55.864 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163196+0000", 2024-09-16T20:50:55.864 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7920000000000004e-06 2024-09-16T20:50:55.864 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.864 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.864 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:55.864 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163201+0000", 2024-09-16T20:50:55.864 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2809999999999999e-06 2024-09-16T20:50:55.864 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.864 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.864 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:55.864 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282219+0000", 2024-09-16T20:50:55.865 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.119018343 2024-09-16T20:50:55.865 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.865 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.865 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:55.865 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282305+0000", 2024-09-16T20:50:55.865 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5865000000000005e-05 2024-09-16T20:50:55.865 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.865 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.865 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:55.865 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282506+0000", 2024-09-16T20:50:55.865 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000200927 2024-09-16T20:50:55.865 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.865 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:55.865 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.866 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.866 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.866 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop(client.4366.0:3088 3.1e e38/23 3:79e5ec01:::rbd_header.110e891a9b2b:head v 38'306, mlcod=36'304)", 2024-09-16T20:50:55.866 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:50:55.866 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 151.62398168199999, 2024-09-16T20:50:55.866 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118102759, 2024-09-16T20:50:55.866 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:55.866 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:55.866 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:55.866 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:55.866 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.866 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:55.866 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:50:55.866 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.866 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.867 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.867 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:55.867 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:50:55.867 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.867 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.867 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.867 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:55.867 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:50:55.867 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.85e-07 2024-09-16T20:50:55.867 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.867 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.867 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:55.867 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164212+0000", 2024-09-16T20:50:55.867 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3999999999999997e-06 2024-09-16T20:50:55.867 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.867 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.868 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:55.868 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281553+0000", 2024-09-16T20:50:55.868 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11734135499999999 2024-09-16T20:50:55.868 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.868 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.868 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:55.868 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281567+0000", 2024-09-16T20:50:55.868 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4188999999999999e-05 2024-09-16T20:50:55.868 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.868 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.868 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:55.868 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281653+0000", 2024-09-16T20:50:55.868 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5385999999999999e-05 2024-09-16T20:50:55.868 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.869 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.869 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:55.869 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281679+0000", 2024-09-16T20:50:55.869 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6689000000000001e-05 2024-09-16T20:50:55.869 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.869 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.869 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:50:55.869 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282277+0000", 2024-09-16T20:50:55.869 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00059798000000000002 2024-09-16T20:50:55.869 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.869 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.869 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:55.869 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282307+0000", 2024-09-16T20:50:55.869 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9975e-05 2024-09-16T20:50:55.869 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.870 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:55.870 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.870 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.870 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.870 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5255.0:44 3.16 3:6f5476df:::rbd_header.110e25db1f60:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:50:55.870 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:50:55.870 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 79.467678626999998, 2024-09-16T20:50:55.870 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.121007509, 2024-09-16T20:50:55.870 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:55.870 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:55.870 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:55.870 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:55.870 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5255", 2024-09-16T20:50:55.870 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:50:55.870 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 44 2024-09-16T20:50:55.870 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.871 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:55.871 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.871 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:55.871 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:50:55.871 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.871 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.871 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.871 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:55.871 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:50:55.871 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.871 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.871 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.871 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:55.871 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:50:55.871 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0500000000000003e-07 2024-09-16T20:50:55.872 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.872 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.872 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:55.872 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320512+0000", 2024-09-16T20:50:55.872 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2540000000000001e-06 2024-09-16T20:50:55.872 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.872 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.872 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:55.872 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320518+0000", 2024-09-16T20:50:55.872 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1689999999999997e-06 2024-09-16T20:50:55.872 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.872 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.872 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:55.872 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320521+0000", 2024-09-16T20:50:55.872 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5200000000000002e-06 2024-09-16T20:50:55.873 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.873 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.873 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:55.873 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320548+0000", 2024-09-16T20:50:55.873 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6662999999999999e-05 2024-09-16T20:50:55.873 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.873 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.873 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:55.873 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320587+0000", 2024-09-16T20:50:55.873 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8866999999999999e-05 2024-09-16T20:50:55.873 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.873 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.873 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:55.873 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.441515+0000", 2024-09-16T20:50:55.874 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.120928331 2024-09-16T20:50:55.874 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.874 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:55.874 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.874 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.874 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.874 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:20329 3.5 3:a4ea64b7:::rbd_header.110ede5ef857:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:55.874 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:01.061694+0000", 2024-09-16T20:50:55.874 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 54.726492059000002, 2024-09-16T20:50:55.874 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.071234077000000007, 2024-09-16T20:50:55.874 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:50:55.874 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:50:55.874 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:50:55.874 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:50:55.874 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:50:55.875 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:55.875 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 20329 2024-09-16T20:50:55.875 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.875 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:50:55.875 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.875 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:50:55.875 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.061694+0000", 2024-09-16T20:50:55.875 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.875 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.875 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.875 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:50:55.875 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.061694+0000", 2024-09-16T20:50:55.875 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:50:55.875 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.876 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.876 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:50:55.876 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089232+0000", 2024-09-16T20:50:55.876 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.027537275 2024-09-16T20:50:55.876 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.876 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.876 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:50:55.876 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089237+0000", 2024-09-16T20:50:55.876 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4709999999999998e-06 2024-09-16T20:50:55.876 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.876 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.876 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:50:55.876 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089246+0000", 2024-09-16T20:50:55.876 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8230000000000001e-06 2024-09-16T20:50:55.876 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.876 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.877 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:50:55.877 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089252+0000", 2024-09-16T20:50:55.877 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3060000000000003e-06 2024-09-16T20:50:55.877 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.877 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.877 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:50:55.877 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.132756+0000", 2024-09-16T20:50:55.877 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.043504334999999998 2024-09-16T20:50:55.877 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.877 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.877 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:50:55.877 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.132829+0000", 2024-09-16T20:50:55.877 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2849999999999995e-05 2024-09-16T20:50:55.877 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:50:55.877 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:50:55.878 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:50:55.878 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.132928+0000", 2024-09-16T20:50:55.878 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.9017000000000004e-05 2024-09-16T20:50:55.878 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.878 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:55.878 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.878 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:50:55.878 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:50:55.878 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:50:55.879 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:50:55.880 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:55.878+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:55.980 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:55.979+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:55.987 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:55.988 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:50:55.988 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:50:55.988 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:55.996 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:50:56.081 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:56.079+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:56.104 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:56.104 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:50:56.104 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:50:56.104 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:50:56.104 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:56.112 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:50:56.181 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:56.180+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:56.262 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:56.262 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:50:56.262 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:50:56.262 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:50:56.262 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.263 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:3088 3.1e 3:79e5ec01:::rbd_header.110e891a9b2b:head [watch watch cookie 140213437171488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:50:56.263 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:50:56.263 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 152.09648989999999, 2024-09-16T20:50:56.263 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.11895186200000001, 2024-09-16T20:50:56.263 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:56.263 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:56.263 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:56.263 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:56.263 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:50:56.263 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:56.263 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3088 2024-09-16T20:50:56.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.263 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:56.263 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.263 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:56.264 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:50:56.264 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.264 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.264 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:56.264 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:50:56.264 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.264 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.264 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:56.264 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163846+0000", 2024-09-16T20:50:56.264 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6799999999999999e-07 2024-09-16T20:50:56.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.265 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.265 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:56.265 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163850+0000", 2024-09-16T20:50:56.265 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.929e-06 2024-09-16T20:50:56.265 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.265 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.265 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:56.265 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163862+0000", 2024-09-16T20:50:56.265 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1972e-05 2024-09-16T20:50:56.265 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.265 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.265 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:56.265 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163867+0000", 2024-09-16T20:50:56.265 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.451e-06 2024-09-16T20:50:56.266 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.266 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.266 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:56.266 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163912+0000", 2024-09-16T20:50:56.266 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4456000000000003e-05 2024-09-16T20:50:56.266 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.266 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.266 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:56.266 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163962+0000", 2024-09-16T20:50:56.266 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0578000000000003e-05 2024-09-16T20:50:56.266 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.266 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.266 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-16T20:50:56.266 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164058+0000", 2024-09-16T20:50:56.267 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5934999999999995e-05 2024-09-16T20:50:56.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.267 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.267 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:50:56.267 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164570+0000", 2024-09-16T20:50:56.267 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00051241100000000005 2024-09-16T20:50:56.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.267 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.267 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:56.267 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282629+0000", 2024-09-16T20:50:56.267 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.118058986 2024-09-16T20:50:56.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.267 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.267 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:50:56.267 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282725+0000", 2024-09-16T20:50:56.268 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5063999999999996e-05 2024-09-16T20:50:56.268 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.268 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.268 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:56.268 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282797+0000", 2024-09-16T20:50:56.268 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2212000000000006e-05 2024-09-16T20:50:56.268 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.268 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:56.268 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.268 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.268 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.268 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:8039 3.1e 3.ac44ff9e (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:50:56.268 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:50:56.268 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 126.552662551, 2024-09-16T20:50:56.268 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0053310449999999, 2024-09-16T20:50:56.269 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:56.269 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:56.269 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:56.269 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:56.269 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:50:56.269 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:56.269 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8039 2024-09-16T20:50:56.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.269 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:56.269 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.269 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:56.269 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:50:56.270 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.270 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.270 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.270 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:56.270 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:50:56.270 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.270 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.270 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.270 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:56.270 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707674+0000", 2024-09-16T20:50:56.270 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6530000000000001e-06 2024-09-16T20:50:56.270 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.270 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.270 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:56.270 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707681+0000", 2024-09-16T20:50:56.271 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9639999999999997e-06 2024-09-16T20:50:56.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.271 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.271 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:56.271 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707700+0000", 2024-09-16T20:50:56.271 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8674000000000001e-05 2024-09-16T20:50:56.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.271 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.271 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:56.271 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707708+0000", 2024-09-16T20:50:56.271 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6810000000000004e-06 2024-09-16T20:50:56.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.271 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.271 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:56.271 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711293+0000", 2024-09-16T20:50:56.272 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035848189999999999 2024-09-16T20:50:56.272 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.272 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.272 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:56.272 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711301+0000", 2024-09-16T20:50:56.272 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4730000000000003e-06 2024-09-16T20:50:56.272 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.272 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.272 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:56.272 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.712990+0000", 2024-09-16T20:50:56.272 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.001689348 2024-09-16T20:50:56.272 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.272 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.272 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:56.273 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713003+0000", 2024-09-16T20:50:56.273 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3433e-05 2024-09-16T20:50:56.273 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.273 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:56.273 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.273 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.273 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.273 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:8040 3.1e 3.ac44ff9e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:50:56.273 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:50:56.273 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 126.55260215600001, 2024-09-16T20:50:56.273 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0053410780000001, 2024-09-16T20:50:56.273 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:56.273 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:56.273 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:56.274 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:56.274 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:50:56.274 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:56.274 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8040 2024-09-16T20:50:56.274 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.274 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:56.274 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.274 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:56.274 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:50:56.274 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.274 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.274 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.274 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:56.274 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:50:56.275 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.275 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.275 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.275 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:56.275 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:50:56.275 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8100000000000003e-07 2024-09-16T20:50:56.275 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.275 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.275 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:56.275 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707737+0000", 2024-09-16T20:50:56.275 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5020000000000001e-06 2024-09-16T20:50:56.275 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.275 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.276 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:56.276 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707745+0000", 2024-09-16T20:50:56.276 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2169999999999994e-06 2024-09-16T20:50:56.276 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.276 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.276 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:56.276 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707751+0000", 2024-09-16T20:50:56.276 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.263e-06 2024-09-16T20:50:56.276 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.276 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.276 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:56.276 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711336+0000", 2024-09-16T20:50:56.276 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035848360000000001 2024-09-16T20:50:56.276 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.277 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.277 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:56.277 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711340+0000", 2024-09-16T20:50:56.277 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1309999999999999e-06 2024-09-16T20:50:56.277 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.277 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.277 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:56.277 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713060+0000", 2024-09-16T20:50:56.277 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0017195059999999 2024-09-16T20:50:56.277 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.277 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.277 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:56.277 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713074+0000", 2024-09-16T20:50:56.278 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4042e-05 2024-09-16T20:50:56.278 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.278 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:56.278 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.278 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.278 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.278 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4927.0:4487 3.1e 3.ac44ff9e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:50:56.278 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:50:56.278 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 126.552081954, 2024-09-16T20:50:56.278 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0048840880000001, 2024-09-16T20:50:56.278 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:56.278 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:56.278 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:56.279 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:56.279 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4927", 2024-09-16T20:50:56.279 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:50:56.279 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4487 2024-09-16T20:50:56.279 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.279 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:56.279 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.279 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:56.279 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:50:56.279 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.279 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.279 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.279 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:56.280 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:50:56.280 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.280 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.280 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.280 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:56.280 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708254+0000", 2024-09-16T20:50:56.280 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1800000000000005e-07 2024-09-16T20:50:56.280 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.280 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.280 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:56.280 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708258+0000", 2024-09-16T20:50:56.280 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8689999999999997e-06 2024-09-16T20:50:56.280 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.281 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.281 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:56.281 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708267+0000", 2024-09-16T20:50:56.281 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9590000000000007e-06 2024-09-16T20:50:56.281 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.281 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.281 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:56.281 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708274+0000", 2024-09-16T20:50:56.281 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9970000000000001e-06 2024-09-16T20:50:56.281 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.281 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.281 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:56.281 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711366+0000", 2024-09-16T20:50:56.282 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030919530000000001 2024-09-16T20:50:56.282 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.282 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.282 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:56.282 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711372+0000", 2024-09-16T20:50:56.282 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9410000000000004e-06 2024-09-16T20:50:56.282 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.282 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.282 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:56.282 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713126+0000", 2024-09-16T20:50:56.282 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0017537249999999 2024-09-16T20:50:56.282 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.282 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.282 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:56.283 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713137+0000", 2024-09-16T20:50:56.283 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0825999999999999e-05 2024-09-16T20:50:56.283 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.283 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:56.283 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.283 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.283 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.283 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5089.0:11703 3.1b 3:dcd4684e:::rbd_data.110e747f0779.000000000000001b:head [set-alloc-hint object_size 262144 write_size 262144,write 139264~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:50:56.283 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:50:56.283 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 112.12234617999999, 2024-09-16T20:50:56.283 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.084231385000000006, 2024-09-16T20:50:56.283 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:56.283 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:56.284 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:56.284 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:56.284 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5089", 2024-09-16T20:50:56.284 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:50:56.284 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 11703 2024-09-16T20:50:56.284 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.284 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:56.284 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.284 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:56.284 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:50:56.284 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.284 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.284 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.284 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:56.285 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:50:56.285 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.285 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.285 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.285 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:56.285 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:50:56.285 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2600000000000001e-07 2024-09-16T20:50:56.285 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.285 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.285 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:56.285 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137998+0000", 2024-09-16T20:50:56.285 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9549999999999998e-06 2024-09-16T20:50:56.285 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.285 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.286 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:56.286 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138008+0000", 2024-09-16T20:50:56.286 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3789999999999995e-06 2024-09-16T20:50:56.286 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.286 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.286 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:56.286 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138013+0000", 2024-09-16T20:50:56.286 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2209999999999996e-06 2024-09-16T20:50:56.286 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.286 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.286 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:56.286 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138054+0000", 2024-09-16T20:50:56.286 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1022999999999997e-05 2024-09-16T20:50:56.286 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.287 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.287 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:56.287 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138124+0000", 2024-09-16T20:50:56.287 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0234999999999994e-05 2024-09-16T20:50:56.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.287 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.287 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:56.287 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138197+0000", 2024-09-16T20:50:56.287 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2451000000000001e-05 2024-09-16T20:50:56.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.287 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.287 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:56.287 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222010+0000", 2024-09-16T20:50:56.287 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.083813433000000007 2024-09-16T20:50:56.288 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.288 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.288 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:50:56.288 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222163+0000", 2024-09-16T20:50:56.288 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015274599999999999 2024-09-16T20:50:56.288 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.288 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.288 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:50:56.288 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222197+0000", 2024-09-16T20:50:56.288 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3683000000000003e-05 2024-09-16T20:50:56.288 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.288 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.288 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:56.289 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222220+0000", 2024-09-16T20:50:56.289 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3433000000000001e-05 2024-09-16T20:50:56.289 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.289 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:56.289 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.289 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.289 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.289 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.5089.0:11702 3.15 e73/64 3:acf33c8b:::rbd_data.110e747f0779.000000000000001a:head v 73'132, mlcod=73'130)", 2024-09-16T20:50:56.289 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:50:56.289 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 112.122078851, 2024-09-16T20:50:56.289 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.084079763000000002, 2024-09-16T20:50:56.289 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:56.289 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:56.290 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:56.290 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:56.290 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.290 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:56.290 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:50:56.290 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.290 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.290 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.290 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:56.290 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:50:56.290 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.290 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.290 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.290 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:56.290 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:50:56.291 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:50:56.291 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.291 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.291 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:56.291 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138263+0000", 2024-09-16T20:50:56.291 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1e-06 2024-09-16T20:50:56.291 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.291 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.291 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:56.291 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221589+0000", 2024-09-16T20:50:56.291 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.083326002999999996 2024-09-16T20:50:56.291 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.291 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.291 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:56.291 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221614+0000", 2024-09-16T20:50:56.292 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5593999999999999e-05 2024-09-16T20:50:56.292 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.292 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.292 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:56.292 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221714+0000", 2024-09-16T20:50:56.292 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9889000000000005e-05 2024-09-16T20:50:56.292 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.292 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.292 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:56.292 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221727+0000", 2024-09-16T20:50:56.292 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3125999999999999e-05 2024-09-16T20:50:56.292 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.292 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.293 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:50:56.293 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222294+0000", 2024-09-16T20:50:56.293 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00056712199999999998 2024-09-16T20:50:56.293 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.293 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.293 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:56.293 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222336+0000", 2024-09-16T20:50:56.293 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1470000000000001e-05 2024-09-16T20:50:56.293 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.293 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:56.293 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.293 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.294 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.294 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5138.0:9358 3.c 3:33996a43:::rbd_data.110e6372565d.0000000000000007:head [read 1343488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T20:50:56.294 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:50:56.294 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 88.407803440999999, 2024-09-16T20:50:56.294 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0167493569999999, 2024-09-16T20:50:56.294 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:56.294 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:56.294 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:56.294 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:56.294 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5138", 2024-09-16T20:50:56.294 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:50:56.294 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9358 2024-09-16T20:50:56.294 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.294 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:56.295 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.295 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:56.295 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:50:56.295 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.295 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.295 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.295 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:56.295 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:50:56.295 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.295 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.295 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.295 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:56.295 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852533+0000", 2024-09-16T20:50:56.295 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.646e-06 2024-09-16T20:50:56.296 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.296 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.296 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:56.296 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852539+0000", 2024-09-16T20:50:56.296 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3389999999999998e-06 2024-09-16T20:50:56.296 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.296 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.296 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:56.296 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852552+0000", 2024-09-16T20:50:56.296 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.271e-05 2024-09-16T20:50:56.296 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.296 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.296 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:56.296 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852563+0000", 2024-09-16T20:50:56.296 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1139e-05 2024-09-16T20:50:56.297 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.297 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.297 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:56.297 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852883+0000", 2024-09-16T20:50:56.297 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00032010600000000001 2024-09-16T20:50:56.297 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.297 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.297 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:56.297 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852890+0000", 2024-09-16T20:50:56.297 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7669999999999998e-06 2024-09-16T20:50:56.297 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.297 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.297 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:56.297 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869059+0000", 2024-09-16T20:50:56.298 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.016169353 2024-09-16T20:50:56.298 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.298 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.298 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:56.298 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869178+0000", 2024-09-16T20:50:56.298 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118193 2024-09-16T20:50:56.298 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.298 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.298 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:56.298 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869281+0000", 2024-09-16T20:50:56.298 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010310399999999999 2024-09-16T20:50:56.298 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.298 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:56.298 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.298 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.299 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.299 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5138.0:9382 3.c 3:33996a43:::rbd_data.110e6372565d.0000000000000007:head [read 0~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T20:50:56.299 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:50:56.299 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 88.407752564000006, 2024-09-16T20:50:56.299 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.017136034, 2024-09-16T20:50:56.299 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:56.299 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:56.299 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:56.299 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:56.299 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5138", 2024-09-16T20:50:56.299 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:50:56.299 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9382 2024-09-16T20:50:56.299 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.299 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:56.299 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.300 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:56.300 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:50:56.300 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.300 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.300 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:56.300 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:50:56.300 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.300 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.300 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:56.300 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852583+0000", 2024-09-16T20:50:56.300 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.51e-07 2024-09-16T20:50:56.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.300 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.301 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:56.301 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852586+0000", 2024-09-16T20:50:56.301 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0280000000000001e-06 2024-09-16T20:50:56.301 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.301 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.301 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:56.301 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852595+0000", 2024-09-16T20:50:56.301 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2799999999999992e-06 2024-09-16T20:50:56.301 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.301 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.301 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:56.301 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852601+0000", 2024-09-16T20:50:56.301 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3810000000000004e-06 2024-09-16T20:50:56.301 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.301 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.302 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:56.302 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852921+0000", 2024-09-16T20:50:56.302 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00031979699999999997 2024-09-16T20:50:56.302 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.302 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.302 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:56.302 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852930+0000", 2024-09-16T20:50:56.302 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6489999999999997e-06 2024-09-16T20:50:56.302 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.302 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.302 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:56.302 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869434+0000", 2024-09-16T20:50:56.302 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0165038829999999 2024-09-16T20:50:56.302 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.302 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.302 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:56.303 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869495+0000", 2024-09-16T20:50:56.303 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1509999999999999e-05 2024-09-16T20:50:56.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.303 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.303 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:56.303 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869718+0000", 2024-09-16T20:50:56.303 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000222955 2024-09-16T20:50:56.303 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.303 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:56.303 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.303 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.303 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9969 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 632832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:56.303 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642045+0000", 2024-09-16T20:50:56.304 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 65.618290107000007, 2024-09-16T20:50:56.304 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076903192999999995, 2024-09-16T20:50:56.304 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:56.304 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:56.304 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:56.304 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:56.304 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:56.304 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:56.304 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9969 2024-09-16T20:50:56.304 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.304 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:56.304 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.305 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:56.305 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642045+0000", 2024-09-16T20:50:56.305 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.305 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:56.305 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642045+0000", 2024-09-16T20:50:56.305 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.305 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:56.305 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642045+0000", 2024-09-16T20:50:56.305 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5100000000000001e-07 2024-09-16T20:50:56.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.306 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:56.306 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642047+0000", 2024-09-16T20:50:56.306 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8679999999999999e-06 2024-09-16T20:50:56.306 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.306 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.306 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:56.306 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642053+0000", 2024-09-16T20:50:56.306 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1720000000000004e-06 2024-09-16T20:50:56.306 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.306 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.306 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:56.306 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642060+0000", 2024-09-16T20:50:56.306 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0339999999999999e-06 2024-09-16T20:50:56.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.307 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.307 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:56.307 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.718562+0000", 2024-09-16T20:50:56.307 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076501969000000003 2024-09-16T20:50:56.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.307 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.307 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:56.307 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.718643+0000", 2024-09-16T20:50:56.307 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0539999999999998e-05 2024-09-16T20:50:56.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.307 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.307 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:56.307 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.718948+0000", 2024-09-16T20:50:56.308 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000305259 2024-09-16T20:50:56.308 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.308 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:56.308 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.308 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.308 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9970 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 637952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:56.308 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642067+0000", 2024-09-16T20:50:56.308 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 65.618268126999993, 2024-09-16T20:50:56.308 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077174928000000004, 2024-09-16T20:50:56.308 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:56.308 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:56.308 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:56.308 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:56.308 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:56.309 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:56.309 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9970 2024-09-16T20:50:56.309 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.309 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:56.309 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.309 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:56.309 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642067+0000", 2024-09-16T20:50:56.309 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.309 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.309 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.309 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:56.309 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642067+0000", 2024-09-16T20:50:56.309 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.309 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.309 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.310 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:56.310 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642067+0000", 2024-09-16T20:50:56.310 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2499999999999995e-07 2024-09-16T20:50:56.310 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.310 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.310 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:56.310 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642070+0000", 2024-09-16T20:50:56.310 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3199999999999998e-06 2024-09-16T20:50:56.310 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.310 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.310 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:56.310 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642076+0000", 2024-09-16T20:50:56.310 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.122e-06 2024-09-16T20:50:56.310 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.310 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.310 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:56.311 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642081+0000", 2024-09-16T20:50:56.311 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0869999999999997e-06 2024-09-16T20:50:56.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.311 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.311 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:56.311 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.718980+0000", 2024-09-16T20:50:56.311 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076899482000000005 2024-09-16T20:50:56.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.311 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.311 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:56.311 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719085+0000", 2024-09-16T20:50:56.311 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010446399999999999 2024-09-16T20:50:56.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.311 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.311 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:56.312 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719242+0000", 2024-09-16T20:50:56.312 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000156928 2024-09-16T20:50:56.312 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.312 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:56.312 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.312 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.312 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.312 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9971 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 643072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:56.312 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642102+0000", 2024-09-16T20:50:56.312 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 65.618232419999998, 2024-09-16T20:50:56.312 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077421016999999995, 2024-09-16T20:50:56.312 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:56.312 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:56.312 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:56.313 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:56.313 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:56.313 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:56.313 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9971 2024-09-16T20:50:56.313 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.313 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:56.313 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.313 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:56.313 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642102+0000", 2024-09-16T20:50:56.313 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.313 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.313 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.313 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:56.313 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642102+0000", 2024-09-16T20:50:56.314 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.314 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.314 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.314 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:56.314 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642103+0000", 2024-09-16T20:50:56.314 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7399999999999998e-07 2024-09-16T20:50:56.314 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.314 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.314 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:56.314 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642105+0000", 2024-09-16T20:50:56.314 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2649999999999999e-06 2024-09-16T20:50:56.314 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.314 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.314 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:56.315 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642112+0000", 2024-09-16T20:50:56.315 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.294e-06 2024-09-16T20:50:56.315 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.315 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.315 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:56.315 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642118+0000", 2024-09-16T20:50:56.315 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4500000000000003e-06 2024-09-16T20:50:56.315 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.315 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.315 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:56.315 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719285+0000", 2024-09-16T20:50:56.315 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077166883000000006 2024-09-16T20:50:56.315 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.315 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.315 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:56.315 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719368+0000", 2024-09-16T20:50:56.316 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2811000000000004e-05 2024-09-16T20:50:56.316 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.316 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.316 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:56.316 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719523+0000", 2024-09-16T20:50:56.316 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015584 2024-09-16T20:50:56.316 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.316 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:56.316 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.316 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.316 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.316 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9972 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 648192~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:56.316 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642125+0000", 2024-09-16T20:50:56.316 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 65.618209472999993, 2024-09-16T20:50:56.316 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077710324999999997, 2024-09-16T20:50:56.317 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:56.317 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:56.317 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:56.317 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:56.317 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:56.317 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:56.317 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9972 2024-09-16T20:50:56.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.317 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:56.317 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.317 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:56.317 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642125+0000", 2024-09-16T20:50:56.317 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.318 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.318 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:56.318 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642125+0000", 2024-09-16T20:50:56.318 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.318 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.318 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:56.318 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642126+0000", 2024-09-16T20:50:56.318 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1899999999999998e-07 2024-09-16T20:50:56.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.318 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.318 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:56.318 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642128+0000", 2024-09-16T20:50:56.318 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8619999999999999e-06 2024-09-16T20:50:56.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.318 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.319 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:56.319 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642134+0000", 2024-09-16T20:50:56.319 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5289999999999997e-06 2024-09-16T20:50:56.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.319 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.319 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:56.319 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642139+0000", 2024-09-16T20:50:56.319 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1250000000000001e-06 2024-09-16T20:50:56.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.319 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.319 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:56.319 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719570+0000", 2024-09-16T20:50:56.319 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077430516000000005 2024-09-16T20:50:56.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.319 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.320 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:56.320 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719655+0000", 2024-09-16T20:50:56.320 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5500000000000005e-05 2024-09-16T20:50:56.320 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.320 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.320 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:56.320 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719836+0000", 2024-09-16T20:50:56.320 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018037399999999999 2024-09-16T20:50:56.320 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.320 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:56.320 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.320 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.320 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.320 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9973 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 653312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:56.320 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:50:56.321 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 65.618188863, 2024-09-16T20:50:56.321 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077905262000000003, 2024-09-16T20:50:56.321 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:56.321 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:56.321 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:56.321 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:56.321 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:56.321 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:56.321 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9973 2024-09-16T20:50:56.321 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.321 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:56.321 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.321 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:56.321 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:50:56.322 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.322 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.322 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:56.322 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:50:56.322 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.322 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.322 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:56.322 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:50:56.322 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:50:56.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.322 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.322 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:56.322 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642148+0000", 2024-09-16T20:50:56.322 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0219999999999999e-06 2024-09-16T20:50:56.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.323 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.323 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:56.323 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642154+0000", 2024-09-16T20:50:56.323 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8540000000000002e-06 2024-09-16T20:50:56.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.323 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.323 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:56.323 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642159+0000", 2024-09-16T20:50:56.323 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.045e-06 2024-09-16T20:50:56.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.323 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.323 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:56.323 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719875+0000", 2024-09-16T20:50:56.324 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077716116000000002 2024-09-16T20:50:56.324 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.324 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.324 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:56.324 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719934+0000", 2024-09-16T20:50:56.324 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8819e-05 2024-09-16T20:50:56.324 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.324 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.324 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:56.324 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720051+0000", 2024-09-16T20:50:56.324 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000116947 2024-09-16T20:50:56.324 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.324 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:56.325 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.325 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.325 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.325 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9974 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 658432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:56.325 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642166+0000", 2024-09-16T20:50:56.325 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 65.618168948999994, 2024-09-16T20:50:56.325 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.078179786000000001, 2024-09-16T20:50:56.325 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:56.325 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:56.325 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:56.325 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:56.325 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:56.325 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:56.325 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9974 2024-09-16T20:50:56.326 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.326 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:56.326 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.326 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:56.326 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642166+0000", 2024-09-16T20:50:56.326 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.326 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.326 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.326 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:56.326 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642166+0000", 2024-09-16T20:50:56.326 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.326 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.326 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.326 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:56.326 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647343+0000", 2024-09-16T20:50:56.326 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0051769440000000002 2024-09-16T20:50:56.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.327 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.327 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:56.327 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647348+0000", 2024-09-16T20:50:56.327 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3619999999999997e-06 2024-09-16T20:50:56.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.327 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.327 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:56.327 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647359+0000", 2024-09-16T20:50:56.327 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0929e-05 2024-09-16T20:50:56.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.327 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.327 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:56.327 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647366+0000", 2024-09-16T20:50:56.327 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0910000000000004e-06 2024-09-16T20:50:56.328 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.328 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.328 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:56.328 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720094+0000", 2024-09-16T20:50:56.328 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.072727403999999995 2024-09-16T20:50:56.328 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.328 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.328 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:56.328 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720173+0000", 2024-09-16T20:50:56.328 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9382000000000007e-05 2024-09-16T20:50:56.328 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.328 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.328 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:56.328 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720346+0000", 2024-09-16T20:50:56.328 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017267399999999999 2024-09-16T20:50:56.329 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.329 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:56.329 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.329 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.329 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.329 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9990 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 740352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:56.329 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647725+0000", 2024-09-16T20:50:56.329 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 65.612609864999996, 2024-09-16T20:50:56.329 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077029919000000002, 2024-09-16T20:50:56.329 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:56.329 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:56.329 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:56.329 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:56.329 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:56.329 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:56.330 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9990 2024-09-16T20:50:56.330 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.330 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:56.330 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.330 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:56.330 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647725+0000", 2024-09-16T20:50:56.330 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.330 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.330 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.330 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:56.330 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647725+0000", 2024-09-16T20:50:56.330 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.330 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.330 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.330 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:56.331 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647725+0000", 2024-09-16T20:50:56.331 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2399999999999999e-07 2024-09-16T20:50:56.331 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.331 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.331 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:56.331 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647726+0000", 2024-09-16T20:50:56.331 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.116e-06 2024-09-16T20:50:56.331 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.331 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.331 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:56.331 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647729+0000", 2024-09-16T20:50:56.331 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.937e-06 2024-09-16T20:50:56.331 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.331 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.332 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:56.332 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647732+0000", 2024-09-16T20:50:56.332 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8739999999999999e-06 2024-09-16T20:50:56.332 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.332 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.332 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:56.332 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724612+0000", 2024-09-16T20:50:56.332 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076880102000000006 2024-09-16T20:50:56.332 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.332 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.332 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:56.332 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724649+0000", 2024-09-16T20:50:56.332 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6738e-05 2024-09-16T20:50:56.332 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.332 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.333 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:56.333 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724755+0000", 2024-09-16T20:50:56.333 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010592800000000001 2024-09-16T20:50:56.333 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.333 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:56.333 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.333 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.333 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.333 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9991 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 745472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:56.333 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:50:56.333 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 65.612599556000006, 2024-09-16T20:50:56.333 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077257820000000005, 2024-09-16T20:50:56.333 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:56.333 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:56.334 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:56.334 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:56.334 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:56.334 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:56.334 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9991 2024-09-16T20:50:56.334 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.334 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:56.334 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.334 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:56.334 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:50:56.334 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.334 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.334 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.334 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:56.334 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:50:56.335 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.335 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.335 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.335 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:56.335 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:50:56.335 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1199999999999999e-07 2024-09-16T20:50:56.335 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.335 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.335 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:56.335 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647736+0000", 2024-09-16T20:50:56.335 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3699999999999999e-07 2024-09-16T20:50:56.335 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.335 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.335 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:56.335 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647739+0000", 2024-09-16T20:50:56.336 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9359999999999999e-06 2024-09-16T20:50:56.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.336 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.336 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:56.336 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647742+0000", 2024-09-16T20:50:56.336 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5720000000000001e-06 2024-09-16T20:50:56.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.336 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.336 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:56.336 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724774+0000", 2024-09-16T20:50:56.336 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077032457999999998 2024-09-16T20:50:56.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.336 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.336 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:56.336 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724818+0000", 2024-09-16T20:50:56.337 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3803000000000001e-05 2024-09-16T20:50:56.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.337 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.337 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:56.337 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724993+0000", 2024-09-16T20:50:56.337 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000174902 2024-09-16T20:50:56.337 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.337 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:56.337 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.337 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.337 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9992 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 750592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:56.337 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:50:56.337 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 65.612589967000005, 2024-09-16T20:50:56.338 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077553227000000002, 2024-09-16T20:50:56.338 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:56.338 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:56.338 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:56.338 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:56.338 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:56.338 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:56.338 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9992 2024-09-16T20:50:56.338 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.338 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:56.338 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.338 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:56.338 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:50:56.338 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.339 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.339 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.339 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:56.339 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:50:56.339 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.339 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.339 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.339 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:56.339 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:50:56.339 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0200000000000001e-07 2024-09-16T20:50:56.339 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.339 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.339 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:56.339 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647746+0000", 2024-09-16T20:50:56.339 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.088e-06 2024-09-16T20:50:56.340 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.340 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.340 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:56.340 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647749+0000", 2024-09-16T20:50:56.340 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9170000000000002e-06 2024-09-16T20:50:56.340 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.340 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.340 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:56.340 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647752+0000", 2024-09-16T20:50:56.340 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.774e-06 2024-09-16T20:50:56.340 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.340 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.340 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:56.340 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725042+0000", 2024-09-16T20:50:56.340 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077289952999999995 2024-09-16T20:50:56.341 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.341 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.341 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:56.341 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725113+0000", 2024-09-16T20:50:56.341 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0920999999999994e-05 2024-09-16T20:50:56.341 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.341 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.341 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:56.341 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725298+0000", 2024-09-16T20:50:56.341 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000185372 2024-09-16T20:50:56.341 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.341 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:56.341 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.341 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.342 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.342 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9993 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 755712~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:56.342 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647755+0000", 2024-09-16T20:50:56.342 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 65.612579941000007, 2024-09-16T20:50:56.342 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077725674999999994, 2024-09-16T20:50:56.342 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:56.342 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:56.342 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:56.342 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:56.342 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:56.342 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:56.342 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9993 2024-09-16T20:50:56.342 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.342 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:56.342 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.343 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:56.343 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647755+0000", 2024-09-16T20:50:56.343 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.343 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.343 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.343 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:56.343 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647755+0000", 2024-09-16T20:50:56.343 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.343 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.343 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.343 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:56.343 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653406+0000", 2024-09-16T20:50:56.343 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.005650876 2024-09-16T20:50:56.343 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.343 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.344 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:56.344 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653411+0000", 2024-09-16T20:50:56.344 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8899999999999998e-06 2024-09-16T20:50:56.344 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.344 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.344 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:56.344 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653423+0000", 2024-09-16T20:50:56.344 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2447e-05 2024-09-16T20:50:56.344 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.344 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.344 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:56.344 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653431+0000", 2024-09-16T20:50:56.344 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5730000000000002e-06 2024-09-16T20:50:56.345 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.345 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.345 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:56.345 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725333+0000", 2024-09-16T20:50:56.345 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.071902743000000005 2024-09-16T20:50:56.345 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.345 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.345 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:56.345 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725385+0000", 2024-09-16T20:50:56.345 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1585000000000002e-05 2024-09-16T20:50:56.345 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.345 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.345 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:56.345 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725481+0000", 2024-09-16T20:50:56.346 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5561000000000002e-05 2024-09-16T20:50:56.346 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.346 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:56.346 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.346 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.346 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.346 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6061.0:5400 3.b 3:d353d652:::rbd_data.110e16eafbd.0000000000000010:head [read 577536~28672] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-16T20:50:56.346 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:27.761367+0000", 2024-09-16T20:50:56.346 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 28.498967995000001, 2024-09-16T20:50:56.346 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0029066630000001, 2024-09-16T20:50:56.346 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:56.346 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:56.346 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:56.346 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:56.347 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6061", 2024-09-16T20:50:56.347 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3930128440", 2024-09-16T20:50:56.347 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5400 2024-09-16T20:50:56.347 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.347 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:56.347 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.347 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:56.347 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761367+0000", 2024-09-16T20:50:56.347 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.347 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.347 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.347 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:56.347 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761367+0000", 2024-09-16T20:50:56.347 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.347 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.348 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.348 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:56.348 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761369+0000", 2024-09-16T20:50:56.348 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.934e-06 2024-09-16T20:50:56.348 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.348 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.348 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:56.348 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761374+0000", 2024-09-16T20:50:56.348 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7139999999999998e-06 2024-09-16T20:50:56.348 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.348 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.348 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:56.348 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761387+0000", 2024-09-16T20:50:56.349 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2349e-05 2024-09-16T20:50:56.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.349 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.349 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:56.349 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761393+0000", 2024-09-16T20:50:56.349 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.652e-06 2024-09-16T20:50:56.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.349 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.350 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:56.350 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761442+0000", 2024-09-16T20:50:56.350 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9011000000000002e-05 2024-09-16T20:50:56.350 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.350 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.350 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:56.350 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761451+0000", 2024-09-16T20:50:56.350 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1920000000000005e-06 2024-09-16T20:50:56.350 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.350 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.350 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:56.350 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:28.764169+0000", 2024-09-16T20:50:56.350 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.002718161 2024-09-16T20:50:56.350 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.351 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.351 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:56.351 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:28.764273+0000", 2024-09-16T20:50:56.351 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010465 2024-09-16T20:50:56.351 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.351 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:56.351 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.351 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.351 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.351 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6061.0:5424 3.b 3:d353d652:::rbd_data.110e16eafbd.0000000000000010:head [read 606208~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-16T20:50:56.351 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:27.766882+0000", 2024-09-16T20:50:56.351 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 28.493452632, 2024-09-16T20:50:56.351 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99753650500000002, 2024-09-16T20:50:56.351 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:56.352 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:56.352 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:56.352 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:56.352 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6061", 2024-09-16T20:50:56.352 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3930128440", 2024-09-16T20:50:56.352 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5424 2024-09-16T20:50:56.352 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.352 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:56.352 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.352 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:56.352 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766882+0000", 2024-09-16T20:50:56.352 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.352 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.353 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.353 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:56.353 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766882+0000", 2024-09-16T20:50:56.353 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.353 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.353 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.353 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:56.353 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766883+0000", 2024-09-16T20:50:56.353 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3289999999999999e-06 2024-09-16T20:50:56.353 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.353 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.353 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:56.353 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766888+0000", 2024-09-16T20:50:56.353 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5800000000000002e-06 2024-09-16T20:50:56.354 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.354 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.354 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:56.354 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766900+0000", 2024-09-16T20:50:56.354 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1861e-05 2024-09-16T20:50:56.354 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.354 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.354 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:56.354 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766909+0000", 2024-09-16T20:50:56.354 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5210000000000008e-06 2024-09-16T20:50:56.354 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.354 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.354 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:56.355 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766958+0000", 2024-09-16T20:50:56.355 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8718999999999999e-05 2024-09-16T20:50:56.355 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.355 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.355 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:56.355 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766975+0000", 2024-09-16T20:50:56.355 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7231000000000002e-05 2024-09-16T20:50:56.355 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.355 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.355 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:56.355 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:28.764327+0000", 2024-09-16T20:50:56.355 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99735112500000001 2024-09-16T20:50:56.355 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.355 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.356 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:56.356 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:28.764419+0000", 2024-09-16T20:50:56.356 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2139000000000007e-05 2024-09-16T20:50:56.356 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.356 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:56.356 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.356 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.356 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:56.356 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:56.357 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:50:56.358 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:56.357+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:56.458 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:56.456+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:56.464 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:56.464 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:50:56.465 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:50:56.465 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:56.473 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:50:56.559 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:56.559+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:56.621 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:56.621 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:50:56.621 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:50:56.621 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:50:56.621 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:56.630 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:50:56.638 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T20:50:56.674 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:56.673+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:56.746 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:56.747 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:50:56.747 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:50:56.747 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:50:56.747 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.747 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12548 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 628736~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:56.747 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757009+0000", 2024-09-16T20:50:56.747 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 65.988186976999998, 2024-09-16T20:50:56.747 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065743956000000006, 2024-09-16T20:50:56.747 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:56.747 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:56.747 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:56.747 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:56.748 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:56.748 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:56.748 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12548 2024-09-16T20:50:56.748 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.748 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:56.748 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.748 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:56.748 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757009+0000", 2024-09-16T20:50:56.748 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.748 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.748 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.748 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:56.748 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757009+0000", 2024-09-16T20:50:56.749 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.749 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.749 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.749 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:56.749 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757010+0000", 2024-09-16T20:50:56.749 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9900000000000001e-07 2024-09-16T20:50:56.749 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.749 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.749 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:56.749 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757012+0000", 2024-09-16T20:50:56.749 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2689999999999998e-06 2024-09-16T20:50:56.749 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.749 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.750 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:56.750 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757022+0000", 2024-09-16T20:50:56.750 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9699999999999994e-06 2024-09-16T20:50:56.750 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.750 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.750 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:56.750 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757028+0000", 2024-09-16T20:50:56.750 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3799999999999999e-06 2024-09-16T20:50:56.750 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.750 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.751 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:56.751 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822552+0000", 2024-09-16T20:50:56.751 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065524281000000004 2024-09-16T20:50:56.751 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.751 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.751 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:56.751 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822613+0000", 2024-09-16T20:50:56.751 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0430999999999998e-05 2024-09-16T20:50:56.751 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.751 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.751 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:56.751 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822753+0000", 2024-09-16T20:50:56.751 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000140226 2024-09-16T20:50:56.752 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.752 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:56.752 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.752 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.752 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.752 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12549 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 633856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:56.752 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:50:56.752 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 65.988158229999996, 2024-09-16T20:50:56.752 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065978518, 2024-09-16T20:50:56.752 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:56.752 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:56.752 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:56.752 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:56.752 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:56.752 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:56.753 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12549 2024-09-16T20:50:56.753 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.753 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:56.753 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.753 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:56.753 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:50:56.753 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.753 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.753 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.754 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:56.754 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:50:56.754 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.754 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.754 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.754 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:56.754 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:50:56.754 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9299999999999999e-07 2024-09-16T20:50:56.754 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.754 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.754 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:56.754 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757040+0000", 2024-09-16T20:50:56.754 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.677e-06 2024-09-16T20:50:56.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.755 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.755 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:56.755 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757045+0000", 2024-09-16T20:50:56.755 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8149999999999996e-06 2024-09-16T20:50:56.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.755 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.755 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:56.755 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757048+0000", 2024-09-16T20:50:56.755 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1e-06 2024-09-16T20:50:56.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.755 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.755 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:56.756 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822796+0000", 2024-09-16T20:50:56.756 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065748081 2024-09-16T20:50:56.756 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.756 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.756 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:56.756 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822867+0000", 2024-09-16T20:50:56.756 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0679000000000005e-05 2024-09-16T20:50:56.756 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.756 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.756 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:56.756 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823016+0000", 2024-09-16T20:50:56.756 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014987299999999999 2024-09-16T20:50:56.756 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.756 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:56.757 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.757 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.757 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.757 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12550 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 638976~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:56.757 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757060+0000", 2024-09-16T20:50:56.757 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 65.988135815000007, 2024-09-16T20:50:56.757 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066153564999999998, 2024-09-16T20:50:56.757 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:56.757 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:56.757 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:56.757 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:56.757 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:56.757 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:56.758 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12550 2024-09-16T20:50:56.758 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.758 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:56.758 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.758 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:56.758 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757060+0000", 2024-09-16T20:50:56.758 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.758 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.758 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.758 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:56.758 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757060+0000", 2024-09-16T20:50:56.758 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.758 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.758 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.758 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:56.759 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757061+0000", 2024-09-16T20:50:56.759 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.15e-07 2024-09-16T20:50:56.759 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.759 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.759 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:56.759 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757062+0000", 2024-09-16T20:50:56.759 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.663e-06 2024-09-16T20:50:56.759 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.759 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.759 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:56.759 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757066+0000", 2024-09-16T20:50:56.759 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.817e-06 2024-09-16T20:50:56.759 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.759 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.759 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:56.760 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757070+0000", 2024-09-16T20:50:56.760 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1509999999999997e-06 2024-09-16T20:50:56.760 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.760 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.760 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:56.760 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823044+0000", 2024-09-16T20:50:56.760 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065973992999999995 2024-09-16T20:50:56.760 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.760 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.760 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:56.760 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823082+0000", 2024-09-16T20:50:56.760 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8050999999999999e-05 2024-09-16T20:50:56.760 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.760 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.760 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:56.761 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823214+0000", 2024-09-16T20:50:56.761 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013157500000000001 2024-09-16T20:50:56.761 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.761 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:56.761 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.761 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.761 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.761 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12551 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 644096~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:56.761 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:50:56.761 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 65.988122117000003, 2024-09-16T20:50:56.761 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066391378000000001, 2024-09-16T20:50:56.761 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:56.761 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:56.762 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:56.762 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:56.762 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:56.762 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:56.762 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12551 2024-09-16T20:50:56.762 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.762 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:56.762 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.762 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:56.762 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:50:56.762 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.762 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.762 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.762 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:56.762 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:50:56.763 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.763 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.763 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.763 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:56.763 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:50:56.763 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2999999999999999e-07 2024-09-16T20:50:56.763 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.763 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.763 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:56.763 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757075+0000", 2024-09-16T20:50:56.763 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.097e-06 2024-09-16T20:50:56.763 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.763 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.763 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:56.764 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757082+0000", 2024-09-16T20:50:56.764 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9500000000000004e-06 2024-09-16T20:50:56.764 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.764 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.764 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:56.764 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757086+0000", 2024-09-16T20:50:56.764 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2229999999999999e-06 2024-09-16T20:50:56.764 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.764 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.764 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:56.764 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823262+0000", 2024-09-16T20:50:56.764 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066176078999999999 2024-09-16T20:50:56.764 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.765 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.765 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:56.765 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823336+0000", 2024-09-16T20:50:56.765 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3979000000000004e-05 2024-09-16T20:50:56.765 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.765 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.765 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:56.765 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823465+0000", 2024-09-16T20:50:56.765 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012982 2024-09-16T20:50:56.765 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.765 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:56.765 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.765 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.766 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.766 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12552 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 649216~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:56.766 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757089+0000", 2024-09-16T20:50:56.766 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 65.988106975999997, 2024-09-16T20:50:56.766 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066655617, 2024-09-16T20:50:56.766 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:56.766 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:56.766 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:56.766 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:56.766 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:56.766 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:56.766 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12552 2024-09-16T20:50:56.766 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.767 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:56.767 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.767 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:56.767 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757089+0000", 2024-09-16T20:50:56.767 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.767 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.767 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.767 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:56.767 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757089+0000", 2024-09-16T20:50:56.767 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.767 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.767 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.767 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:56.767 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759492+0000", 2024-09-16T20:50:56.768 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0024026109999999998 2024-09-16T20:50:56.768 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.768 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.768 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:56.768 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759497+0000", 2024-09-16T20:50:56.768 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1120000000000004e-06 2024-09-16T20:50:56.768 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.768 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.768 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:56.768 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759508+0000", 2024-09-16T20:50:56.768 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0981e-05 2024-09-16T20:50:56.768 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.768 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.769 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:56.769 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759516+0000", 2024-09-16T20:50:56.769 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7270000000000007e-06 2024-09-16T20:50:56.769 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.769 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.769 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:56.769 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823504+0000", 2024-09-16T20:50:56.769 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.063987984999999997 2024-09-16T20:50:56.769 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.769 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.769 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:56.769 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823579+0000", 2024-09-16T20:50:56.769 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5703000000000003e-05 2024-09-16T20:50:56.769 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.770 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.770 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:56.770 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823745+0000", 2024-09-16T20:50:56.770 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016549800000000001 2024-09-16T20:50:56.770 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.770 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:56.770 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.770 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.770 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.770 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12560 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 690176~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:56.770 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.759658+0000", 2024-09-16T20:50:56.770 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 65.985537773999994, 2024-09-16T20:50:56.770 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065858517000000005, 2024-09-16T20:50:56.771 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:56.771 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:56.771 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:56.771 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:56.771 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:56.771 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:56.771 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12560 2024-09-16T20:50:56.771 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.771 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:56.771 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.771 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:56.771 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759658+0000", 2024-09-16T20:50:56.771 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.771 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.772 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.772 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:56.772 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759658+0000", 2024-09-16T20:50:56.772 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.772 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.772 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.772 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:56.772 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759659+0000", 2024-09-16T20:50:56.772 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.17e-07 2024-09-16T20:50:56.772 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.772 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.772 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:56.772 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759660+0000", 2024-09-16T20:50:56.772 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0189999999999999e-06 2024-09-16T20:50:56.773 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.773 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.773 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:56.773 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759663+0000", 2024-09-16T20:50:56.773 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0010000000000002e-06 2024-09-16T20:50:56.773 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.773 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.773 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:56.773 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759665+0000", 2024-09-16T20:50:56.773 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5440000000000001e-06 2024-09-16T20:50:56.773 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.773 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.773 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:56.773 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825355+0000", 2024-09-16T20:50:56.774 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065690224000000005 2024-09-16T20:50:56.774 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.774 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.774 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:56.774 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825394+0000", 2024-09-16T20:50:56.774 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8810999999999998e-05 2024-09-16T20:50:56.774 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.774 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.774 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:56.774 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825517+0000", 2024-09-16T20:50:56.774 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012270099999999999 2024-09-16T20:50:56.774 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.774 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:56.774 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.775 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.775 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.775 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12561 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 695296~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:56.775 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.759668+0000", 2024-09-16T20:50:56.775 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 65.985527931999997, 2024-09-16T20:50:56.775 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066116999999999995, 2024-09-16T20:50:56.775 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:56.775 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:56.775 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:56.775 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:56.775 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:56.775 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:56.775 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12561 2024-09-16T20:50:56.776 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.776 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:56.776 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.776 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:56.776 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759668+0000", 2024-09-16T20:50:56.776 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.776 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.776 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.776 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:56.776 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759668+0000", 2024-09-16T20:50:56.776 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.776 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.776 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.777 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:56.777 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765379+0000", 2024-09-16T20:50:56.777 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057104410000000001 2024-09-16T20:50:56.777 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.777 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.777 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:56.777 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765383+0000", 2024-09-16T20:50:56.777 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2980000000000002e-06 2024-09-16T20:50:56.777 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.777 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.777 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:56.777 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765395+0000", 2024-09-16T20:50:56.777 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1671000000000001e-05 2024-09-16T20:50:56.777 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.778 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.778 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:56.778 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765403+0000", 2024-09-16T20:50:56.778 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1210000000000007e-06 2024-09-16T20:50:56.778 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.778 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.778 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:56.778 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825552+0000", 2024-09-16T20:50:56.778 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.060149498000000003 2024-09-16T20:50:56.778 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.778 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.778 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:56.778 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825625+0000", 2024-09-16T20:50:56.779 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2540999999999995e-05 2024-09-16T20:50:56.779 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.779 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.779 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:56.779 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825785+0000", 2024-09-16T20:50:56.779 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016043000000000001 2024-09-16T20:50:56.779 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.779 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:56.779 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.779 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.779 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.779 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12585 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 818176~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:56.779 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.765848+0000", 2024-09-16T20:50:56.779 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 65.979347961000002, 2024-09-16T20:50:56.780 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065843218999999995, 2024-09-16T20:50:56.780 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:56.780 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:56.780 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:56.780 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:56.780 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:56.780 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:56.780 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12585 2024-09-16T20:50:56.780 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.780 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:56.780 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.780 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:56.780 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765848+0000", 2024-09-16T20:50:56.780 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.781 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.781 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.781 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:56.781 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765848+0000", 2024-09-16T20:50:56.781 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.781 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.781 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.781 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:56.781 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768683+0000", 2024-09-16T20:50:56.781 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0028344519999999999 2024-09-16T20:50:56.781 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.781 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.781 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:56.782 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768686+0000", 2024-09-16T20:50:56.782 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8789999999999996e-06 2024-09-16T20:50:56.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.782 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.782 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:56.782 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768696+0000", 2024-09-16T20:50:56.782 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6709999999999994e-06 2024-09-16T20:50:56.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.782 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.782 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:56.782 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768702+0000", 2024-09-16T20:50:56.782 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2550000000000003e-06 2024-09-16T20:50:56.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.782 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.783 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:56.783 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.831499+0000", 2024-09-16T20:50:56.783 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.062797081000000005 2024-09-16T20:50:56.783 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.783 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.783 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:56.783 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.831538+0000", 2024-09-16T20:50:56.783 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8523999999999999e-05 2024-09-16T20:50:56.783 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.783 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.783 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:56.783 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.831691+0000", 2024-09-16T20:50:56.783 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015335699999999999 2024-09-16T20:50:56.784 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.784 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:56.784 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.784 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.784 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.784 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12597 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 879616~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:56.784 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.768883+0000", 2024-09-16T20:50:56.784 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 65.976313232999999, 2024-09-16T20:50:56.784 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065816527, 2024-09-16T20:50:56.784 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:56.784 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:56.784 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:56.784 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:56.784 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:56.785 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:56.785 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12597 2024-09-16T20:50:56.785 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.785 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:56.785 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.785 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:56.785 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768883+0000", 2024-09-16T20:50:56.785 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.785 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.785 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.785 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:56.785 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768883+0000", 2024-09-16T20:50:56.785 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.785 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.786 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:56.786 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771508+0000", 2024-09-16T20:50:56.786 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0026247589999999999 2024-09-16T20:50:56.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.786 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:56.786 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771513+0000", 2024-09-16T20:50:56.786 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9949999999999996e-06 2024-09-16T20:50:56.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.786 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:56.786 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771525+0000", 2024-09-16T20:50:56.786 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2033e-05 2024-09-16T20:50:56.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.787 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.787 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:56.787 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771532+0000", 2024-09-16T20:50:56.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4220000000000003e-06 2024-09-16T20:50:56.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.787 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.787 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:56.787 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.834532+0000", 2024-09-16T20:50:56.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.063000159 2024-09-16T20:50:56.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.787 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.787 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:56.787 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.834590+0000", 2024-09-16T20:50:56.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8218999999999999e-05 2024-09-16T20:50:56.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.788 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.788 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:56.788 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.834699+0000", 2024-09-16T20:50:56.788 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010894 2024-09-16T20:50:56.788 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.788 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:56.788 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.788 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.788 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12619 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 992256~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:56.788 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.774656+0000", 2024-09-16T20:50:56.788 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 65.970540459000006, 2024-09-16T20:50:56.788 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065715859000000001, 2024-09-16T20:50:56.789 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:56.789 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:56.789 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:56.789 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:56.789 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:56.789 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:56.789 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12619 2024-09-16T20:50:56.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.789 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:56.789 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.789 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:56.789 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774656+0000", 2024-09-16T20:50:56.789 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.790 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.790 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:56.790 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774656+0000", 2024-09-16T20:50:56.790 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.790 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.790 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:56.790 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774875+0000", 2024-09-16T20:50:56.790 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000218898 2024-09-16T20:50:56.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.790 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.790 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:56.790 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774879+0000", 2024-09-16T20:50:56.790 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0389999999999998e-06 2024-09-16T20:50:56.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.791 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.791 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:56.791 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774890+0000", 2024-09-16T20:50:56.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1250000000000001e-05 2024-09-16T20:50:56.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.791 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.791 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:56.791 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774897+0000", 2024-09-16T20:50:56.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7619999999999998e-06 2024-09-16T20:50:56.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.791 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.791 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:56.791 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840139+0000", 2024-09-16T20:50:56.792 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065242438 2024-09-16T20:50:56.792 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.792 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.792 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:56.792 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840186+0000", 2024-09-16T20:50:56.792 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6894999999999997e-05 2024-09-16T20:50:56.792 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.792 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.792 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:56.792 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840372+0000", 2024-09-16T20:50:56.792 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000185577 2024-09-16T20:50:56.792 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.792 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:56.793 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.793 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.793 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12620 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 997376~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:56.793 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.774908+0000", 2024-09-16T20:50:56.793 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 65.970288237999995, 2024-09-16T20:50:56.793 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065782754999999998, 2024-09-16T20:50:56.793 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:56.793 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:56.793 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:56.793 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:56.793 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:56.793 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:56.793 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12620 2024-09-16T20:50:56.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.794 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:56.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.794 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:56.794 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774908+0000", 2024-09-16T20:50:56.794 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.794 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:56.794 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774908+0000", 2024-09-16T20:50:56.794 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.795 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:56.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775102+0000", 2024-09-16T20:50:56.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000194098 2024-09-16T20:50:56.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.795 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.795 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:56.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775105+0000", 2024-09-16T20:50:56.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8739999999999999e-06 2024-09-16T20:50:56.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.795 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.795 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:56.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775111+0000", 2024-09-16T20:50:56.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9519999999999999e-06 2024-09-16T20:50:56.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.796 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.796 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:56.796 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775115+0000", 2024-09-16T20:50:56.796 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.002e-06 2024-09-16T20:50:56.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.796 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.796 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:56.796 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840409+0000", 2024-09-16T20:50:56.796 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065294195999999999 2024-09-16T20:50:56.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.796 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.796 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:56.796 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840507+0000", 2024-09-16T20:50:56.796 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7662000000000001e-05 2024-09-16T20:50:56.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.797 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.797 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:56.797 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840691+0000", 2024-09-16T20:50:56.797 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000183971 2024-09-16T20:50:56.797 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.797 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:56.797 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.797 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.797 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12621 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1002496~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:56.797 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775123+0000", 2024-09-16T20:50:56.797 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 65.970073541999994, 2024-09-16T20:50:56.798 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065829586999999995, 2024-09-16T20:50:56.798 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:56.798 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:56.798 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:56.798 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:56.798 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:56.798 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:56.798 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12621 2024-09-16T20:50:56.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.798 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:56.798 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.798 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:56.798 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775123+0000", 2024-09-16T20:50:56.798 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.799 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.799 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:56.799 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775123+0000", 2024-09-16T20:50:56.799 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.799 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.799 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:56.799 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775348+0000", 2024-09-16T20:50:56.799 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000225295 2024-09-16T20:50:56.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.799 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.799 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:56.799 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775352+0000", 2024-09-16T20:50:56.800 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0810000000000004e-06 2024-09-16T20:50:56.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.800 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:56.800 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775361+0000", 2024-09-16T20:50:56.800 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7399999999999993e-06 2024-09-16T20:50:56.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.800 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:56.800 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775367+0000", 2024-09-16T20:50:56.800 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5200000000000003e-06 2024-09-16T20:50:56.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.800 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:56.801 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840730+0000", 2024-09-16T20:50:56.801 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065362803999999997 2024-09-16T20:50:56.801 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.801 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.801 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:56.801 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840801+0000", 2024-09-16T20:50:56.801 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1006000000000004e-05 2024-09-16T20:50:56.801 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.801 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.801 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:56.801 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840952+0000", 2024-09-16T20:50:56.801 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015114100000000001 2024-09-16T20:50:56.801 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.802 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:56.802 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.802 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.802 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.802 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12622 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1007616~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:56.802 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775381+0000", 2024-09-16T20:50:56.802 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 65.969814708000001, 2024-09-16T20:50:56.802 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065822581000000005, 2024-09-16T20:50:56.802 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:56.802 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:56.802 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:56.802 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:56.802 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:56.802 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:56.803 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12622 2024-09-16T20:50:56.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.803 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:56.803 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.803 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:56.803 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775381+0000", 2024-09-16T20:50:56.803 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.803 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.803 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:56.803 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775381+0000", 2024-09-16T20:50:56.803 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.803 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.803 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:56.804 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775563+0000", 2024-09-16T20:50:56.804 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018153700000000001 2024-09-16T20:50:56.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.804 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:56.804 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775566+0000", 2024-09-16T20:50:56.804 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9620000000000002e-06 2024-09-16T20:50:56.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.804 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:56.804 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775572+0000", 2024-09-16T20:50:56.804 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8529999999999997e-06 2024-09-16T20:50:56.805 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.805 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.805 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:56.805 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775576+0000", 2024-09-16T20:50:56.805 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8920000000000002e-06 2024-09-16T20:50:56.805 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.805 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.805 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:56.805 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840990+0000", 2024-09-16T20:50:56.805 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065414599000000004 2024-09-16T20:50:56.805 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.805 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.805 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:56.805 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841068+0000", 2024-09-16T20:50:56.806 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8157000000000004e-05 2024-09-16T20:50:56.806 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.806 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.806 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:56.806 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841204+0000", 2024-09-16T20:50:56.806 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013558099999999999 2024-09-16T20:50:56.806 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.806 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:56.806 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.806 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.806 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.806 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12623 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1012736~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:56.806 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775583+0000", 2024-09-16T20:50:56.807 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 65.969613269999996, 2024-09-16T20:50:56.807 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065929087999999997, 2024-09-16T20:50:56.807 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:56.807 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:56.807 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:56.807 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:56.807 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:56.807 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:56.807 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12623 2024-09-16T20:50:56.807 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.807 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:56.807 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.807 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:56.808 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775583+0000", 2024-09-16T20:50:56.808 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.808 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.808 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.808 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:56.808 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775583+0000", 2024-09-16T20:50:56.808 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.808 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.808 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.808 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:56.808 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775804+0000", 2024-09-16T20:50:56.808 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00022116900000000001 2024-09-16T20:50:56.808 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.808 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.809 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:56.809 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775807+0000", 2024-09-16T20:50:56.809 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9529999999999999e-06 2024-09-16T20:50:56.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.809 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.809 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:56.809 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775814+0000", 2024-09-16T20:50:56.809 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7490000000000001e-06 2024-09-16T20:50:56.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.809 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.809 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:56.809 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775818+0000", 2024-09-16T20:50:56.809 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1999999999999996e-06 2024-09-16T20:50:56.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.810 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.810 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:56.810 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841267+0000", 2024-09-16T20:50:56.810 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065449127999999995 2024-09-16T20:50:56.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.810 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.810 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:56.810 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841344+0000", 2024-09-16T20:50:56.810 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6546999999999997e-05 2024-09-16T20:50:56.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.810 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.810 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:56.810 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841512+0000", 2024-09-16T20:50:56.811 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016834200000000001 2024-09-16T20:50:56.811 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.811 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:56.811 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.811 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.811 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.811 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12624 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1017856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:56.811 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:50:56.811 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 65.969370130000001, 2024-09-16T20:50:56.811 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065945866000000006, 2024-09-16T20:50:56.811 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:56.811 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:56.811 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:56.812 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:56.812 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:56.812 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:56.812 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12624 2024-09-16T20:50:56.812 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.812 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:56.812 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.812 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:56.812 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:50:56.812 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.812 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.812 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.812 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:56.812 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:50:56.812 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.813 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.813 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.813 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:56.813 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:50:56.813 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-16T20:50:56.813 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.813 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.813 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:56.813 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775828+0000", 2024-09-16T20:50:56.813 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3999999999999999e-06 2024-09-16T20:50:56.813 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.813 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.813 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:56.813 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775831+0000", 2024-09-16T20:50:56.814 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.478e-06 2024-09-16T20:50:56.814 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.814 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.814 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:56.814 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775834+0000", 2024-09-16T20:50:56.814 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9440000000000001e-06 2024-09-16T20:50:56.814 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.814 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.814 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:56.814 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841547+0000", 2024-09-16T20:50:56.814 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065712801000000001 2024-09-16T20:50:56.814 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.814 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.814 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:56.815 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841618+0000", 2024-09-16T20:50:56.815 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1151999999999999e-05 2024-09-16T20:50:56.815 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.815 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.815 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:56.815 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841772+0000", 2024-09-16T20:50:56.815 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000153721 2024-09-16T20:50:56.815 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.815 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:56.815 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.815 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.815 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.815 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12625 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1022976~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:56.815 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775840+0000", 2024-09-16T20:50:56.816 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 65.969356520999995, 2024-09-16T20:50:56.816 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066165451, 2024-09-16T20:50:56.816 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:56.816 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:56.816 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:56.816 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:56.816 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:56.816 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:56.816 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12625 2024-09-16T20:50:56.816 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.816 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:56.816 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.816 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:56.817 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775840+0000", 2024-09-16T20:50:56.817 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.817 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.817 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.817 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:56.817 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775840+0000", 2024-09-16T20:50:56.817 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.817 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.817 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.817 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:56.817 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776014+0000", 2024-09-16T20:50:56.817 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017441099999999999 2024-09-16T20:50:56.818 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.818 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.818 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:56.818 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776018+0000", 2024-09-16T20:50:56.818 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6509999999999999e-06 2024-09-16T20:50:56.818 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.818 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.818 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:56.818 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776026+0000", 2024-09-16T20:50:56.818 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.028e-06 2024-09-16T20:50:56.818 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.818 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.818 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:56.819 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776031+0000", 2024-09-16T20:50:56.819 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8400000000000002e-06 2024-09-16T20:50:56.819 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.819 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.819 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:56.819 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841801+0000", 2024-09-16T20:50:56.819 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065770472999999996 2024-09-16T20:50:56.819 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.819 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.819 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:56.819 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841847+0000", 2024-09-16T20:50:56.819 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5558e-05 2024-09-16T20:50:56.819 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.819 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.820 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:56.820 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842005+0000", 2024-09-16T20:50:56.820 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015849000000000001 2024-09-16T20:50:56.820 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.820 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:56.820 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.820 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.820 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.820 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12626 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1028096~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:56.820 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776041+0000", 2024-09-16T20:50:56.820 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 65.969155221999998, 2024-09-16T20:50:56.820 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066225260999999994, 2024-09-16T20:50:56.820 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:56.820 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:56.821 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:56.821 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:56.821 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:56.821 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:56.821 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12626 2024-09-16T20:50:56.821 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.821 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:56.821 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.821 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:56.821 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776041+0000", 2024-09-16T20:50:56.821 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.821 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.821 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.821 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:56.822 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776041+0000", 2024-09-16T20:50:56.822 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.822 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.822 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.822 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:56.822 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776263+0000", 2024-09-16T20:50:56.822 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00022193899999999999 2024-09-16T20:50:56.822 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.822 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.822 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:56.822 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776268+0000", 2024-09-16T20:50:56.822 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8049999999999997e-06 2024-09-16T20:50:56.822 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.822 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.822 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:56.823 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776280+0000", 2024-09-16T20:50:56.823 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2635e-05 2024-09-16T20:50:56.823 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.823 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.823 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:56.823 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776288+0000", 2024-09-16T20:50:56.823 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8199999999999997e-06 2024-09-16T20:50:56.823 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.823 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.823 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:56.823 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842041+0000", 2024-09-16T20:50:56.823 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065752506000000002 2024-09-16T20:50:56.823 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.824 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.824 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:56.824 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842123+0000", 2024-09-16T20:50:56.824 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2393999999999998e-05 2024-09-16T20:50:56.824 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.824 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.824 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:56.824 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842266+0000", 2024-09-16T20:50:56.824 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014316200000000001 2024-09-16T20:50:56.824 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.824 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:56.824 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.824 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.824 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.824 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12627 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1033216~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:56.825 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776301+0000", 2024-09-16T20:50:56.825 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 65.968895262000004, 2024-09-16T20:50:56.825 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066187814999999997, 2024-09-16T20:50:56.825 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:56.825 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:56.825 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:56.825 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:56.825 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:56.825 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:56.825 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12627 2024-09-16T20:50:56.825 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.825 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:56.825 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.825 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:56.826 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776301+0000", 2024-09-16T20:50:56.826 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.826 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.826 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.826 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:56.826 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776301+0000", 2024-09-16T20:50:56.826 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.826 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.826 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.826 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:56.826 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776473+0000", 2024-09-16T20:50:56.826 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017222400000000001 2024-09-16T20:50:56.826 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.826 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.827 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:56.827 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776477+0000", 2024-09-16T20:50:56.827 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7019999999999999e-06 2024-09-16T20:50:56.827 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.827 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.827 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:56.827 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776486+0000", 2024-09-16T20:50:56.827 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2119999999999992e-06 2024-09-16T20:50:56.827 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.827 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.827 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:56.827 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776492+0000", 2024-09-16T20:50:56.827 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8089999999999998e-06 2024-09-16T20:50:56.827 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.827 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.828 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:56.828 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842301+0000", 2024-09-16T20:50:56.828 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065809042999999998 2024-09-16T20:50:56.828 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.828 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.828 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:56.828 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842362+0000", 2024-09-16T20:50:56.828 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1260000000000006e-05 2024-09-16T20:50:56.828 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.828 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.828 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:56.828 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842489+0000", 2024-09-16T20:50:56.828 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012656500000000001 2024-09-16T20:50:56.828 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.829 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:56.829 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.829 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.829 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.829 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12628 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1038336~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:56.829 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776502+0000", 2024-09-16T20:50:56.829 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 65.968694615999993, 2024-09-16T20:50:56.829 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066283426000000006, 2024-09-16T20:50:56.829 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:56.829 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:56.829 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:56.829 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:56.829 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:56.829 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:56.829 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12628 2024-09-16T20:50:56.830 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.830 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:56.830 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.830 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:56.830 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776502+0000", 2024-09-16T20:50:56.830 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.830 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.830 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.830 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:56.830 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776502+0000", 2024-09-16T20:50:56.830 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.830 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.830 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.830 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:56.831 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776716+0000", 2024-09-16T20:50:56.831 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000214596 2024-09-16T20:50:56.831 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.831 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.831 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:56.831 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776723+0000", 2024-09-16T20:50:56.831 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4000000000000003e-06 2024-09-16T20:50:56.831 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.831 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.831 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:56.831 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776731+0000", 2024-09-16T20:50:56.831 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6129999999999998e-06 2024-09-16T20:50:56.831 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.831 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.831 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:56.832 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776737+0000", 2024-09-16T20:50:56.832 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1970000000000001e-06 2024-09-16T20:50:56.832 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.832 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.832 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:56.832 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842532+0000", 2024-09-16T20:50:56.832 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065795127999999994 2024-09-16T20:50:56.832 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.832 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.832 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:56.832 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842599+0000", 2024-09-16T20:50:56.832 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6711000000000006e-05 2024-09-16T20:50:56.832 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.832 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.833 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:56.833 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842785+0000", 2024-09-16T20:50:56.833 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018578099999999999 2024-09-16T20:50:56.833 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.833 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:56.833 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.833 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.833 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.833 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12629 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1043456~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:56.833 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776748+0000", 2024-09-16T20:50:56.833 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 65.968448238999997, 2024-09-16T20:50:56.833 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066303788000000002, 2024-09-16T20:50:56.833 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:56.834 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:56.834 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:56.834 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:56.834 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:56.834 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:56.834 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12629 2024-09-16T20:50:56.834 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.834 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:56.834 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.834 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:56.834 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776748+0000", 2024-09-16T20:50:56.834 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.834 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.834 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.835 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:56.835 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776748+0000", 2024-09-16T20:50:56.835 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:56.835 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.835 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.835 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:56.835 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776934+0000", 2024-09-16T20:50:56.835 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018598399999999999 2024-09-16T20:50:56.835 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.835 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.835 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:56.835 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776939+0000", 2024-09-16T20:50:56.835 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1000000000000003e-06 2024-09-16T20:50:56.835 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.836 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.836 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:56.836 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776954+0000", 2024-09-16T20:50:56.836 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5221e-05 2024-09-16T20:50:56.836 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.836 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.836 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:56.836 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776962+0000", 2024-09-16T20:50:56.836 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.554e-06 2024-09-16T20:50:56.836 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.836 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.836 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:56.836 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842813+0000", 2024-09-16T20:50:56.837 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065851006000000004 2024-09-16T20:50:56.837 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.837 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.837 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:56.837 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842886+0000", 2024-09-16T20:50:56.837 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3663000000000003e-05 2024-09-16T20:50:56.837 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:56.837 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:56.837 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:56.837 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.843052+0000", 2024-09-16T20:50:56.837 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016526 2024-09-16T20:50:56.837 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.837 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:56.837 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.838 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:56.838 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:56.838 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:56.839 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:50:56.839 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:50:56.840 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:50:56.840 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:50:56.840 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:50:56.841 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:50:56.841 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:50:56.841 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:50:56.842 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:50:56.842 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:56.837+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:56.855 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:50:56.855 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:50:56.872 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:50:56.872 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:50:56.940 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:56.939+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:56.988 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:56.988 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:56.997 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:50:57.081 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:57.080+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:57.142 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:57.140+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:57.145 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:57.145 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:57.154 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:50:57.243 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:57.242+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:57.302 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:57.302 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:57.311 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:50:57.344 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:57.342+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:57.431 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:57.431 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:57.441 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:50:57.444 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:57.443+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:57.545 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:50:57.543+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:50:57.549 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:57.549 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:57.558 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:50:57.584 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T20:50:57.645 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:57.644+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:57.678 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:57.679 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:57.688 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:50:57.747 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:57.746+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:57.812 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:57.812 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:57.821 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:50:57.862 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:57.861+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:57.948 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:57.947+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:57.968 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:57.968 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:57.977 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:50:58.061 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:58.060+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:58.125 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:58.125 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:58.133 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:50:58.174 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:58.173+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:58.252 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:58.251+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:58.280 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:58.280 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:58.289 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:50:58.374 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:58.373+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:58.437 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:58.437 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:58.445 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:50:58.486 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:58.485+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:58.554 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:58.553+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:58.592 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:58.593 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:58.601 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:50:58.635 INFO:teuthology.orchestra.run.smithi029.stderr:nodeep-scrub is set 2024-09-16T20:50:58.643 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 4, 5] out_osds: [0, 3] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-16T20:50:58.643 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T20:50:58.643 INFO:tasks.thrashosds.thrasher:Adding osd 0 2024-09-16T20:50:58.644 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 0 2024-09-16T20:50:58.658 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:58.656+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:58.737 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:58.738 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:58.747 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:50:58.790 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:58.789+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:58.857 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:58.856+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:58.857 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:58.857 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:58.866 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:50:58.883 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T20:50:58.958 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:58.956+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:58.965 INFO:tasks.ceph.mon.a.smithi029.stderr:2024-09-16T20:50:58.964+0000 7fe0ce96d640 -1 mon.a@0(leader).osd e171 definitely_dead 0 2024-09-16T20:50:59.006 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:50:59.006 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:50:59.014 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:50:59.014 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:50:59.023 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:50:59.024 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:50:59.024 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:50:59.039 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:50:59.039 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:50:59.056 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:50:59.057 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:50:59.064 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:59.063+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:59.073 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:50:59.073 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:50:59.090 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:50:59.090 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:50:59.160 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:59.159+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:59.170 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:59.171 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:50:59.171 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:50:59.171 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:59.179 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:50:59.261 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:59.260+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:59.286 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:59.286 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:50:59.286 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:50:59.286 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:50:59.286 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:59.295 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:50:59.362 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:50:59.361+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:50:59.404 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:59.404 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:50:59.404 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:50:59.404 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:50:59.404 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.404 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12548 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 628736~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:59.404 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757009+0000", 2024-09-16T20:50:59.404 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 68.645026688000002, 2024-09-16T20:50:59.404 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065743956000000006, 2024-09-16T20:50:59.405 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:59.405 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:59.405 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:59.405 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:59.405 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:59.405 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:59.405 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12548 2024-09-16T20:50:59.405 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.405 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:59.405 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.405 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:59.405 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757009+0000", 2024-09-16T20:50:59.405 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.405 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.406 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.406 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:59.406 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757009+0000", 2024-09-16T20:50:59.406 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.406 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.406 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:59.406 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757010+0000", 2024-09-16T20:50:59.406 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9900000000000001e-07 2024-09-16T20:50:59.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.406 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.406 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:59.406 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757012+0000", 2024-09-16T20:50:59.406 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2689999999999998e-06 2024-09-16T20:50:59.407 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.407 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.407 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:59.407 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757022+0000", 2024-09-16T20:50:59.407 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9699999999999994e-06 2024-09-16T20:50:59.407 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.407 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.407 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:59.407 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757028+0000", 2024-09-16T20:50:59.407 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3799999999999999e-06 2024-09-16T20:50:59.407 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.407 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.407 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:59.407 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822552+0000", 2024-09-16T20:50:59.407 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065524281000000004 2024-09-16T20:50:59.408 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.408 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.408 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:59.408 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822613+0000", 2024-09-16T20:50:59.408 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0430999999999998e-05 2024-09-16T20:50:59.408 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.408 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.408 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:59.408 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822753+0000", 2024-09-16T20:50:59.408 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000140226 2024-09-16T20:50:59.408 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.408 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:59.408 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.408 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.409 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.409 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12549 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 633856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:59.409 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:50:59.409 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 68.644997941, 2024-09-16T20:50:59.409 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065978518, 2024-09-16T20:50:59.409 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:59.409 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:59.409 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:59.409 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:59.409 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:59.409 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:59.409 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12549 2024-09-16T20:50:59.409 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.409 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:59.410 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.410 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:59.410 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:50:59.410 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.410 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.410 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:59.410 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:50:59.410 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.410 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.410 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:59.410 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:50:59.410 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9299999999999999e-07 2024-09-16T20:50:59.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.411 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.411 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:59.411 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757040+0000", 2024-09-16T20:50:59.411 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.677e-06 2024-09-16T20:50:59.411 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.411 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.411 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:59.411 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757045+0000", 2024-09-16T20:50:59.411 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8149999999999996e-06 2024-09-16T20:50:59.411 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.411 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.411 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:59.411 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757048+0000", 2024-09-16T20:50:59.411 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1e-06 2024-09-16T20:50:59.412 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.412 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.412 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:59.412 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822796+0000", 2024-09-16T20:50:59.412 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065748081 2024-09-16T20:50:59.412 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.412 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.412 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:59.412 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822867+0000", 2024-09-16T20:50:59.412 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0679000000000005e-05 2024-09-16T20:50:59.412 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.412 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.412 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:59.413 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823016+0000", 2024-09-16T20:50:59.413 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014987299999999999 2024-09-16T20:50:59.413 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.413 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:59.413 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.413 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.413 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.413 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12550 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 638976~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:59.413 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757060+0000", 2024-09-16T20:50:59.413 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 68.644975525999996, 2024-09-16T20:50:59.413 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066153564999999998, 2024-09-16T20:50:59.413 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:59.413 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:59.413 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:59.414 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:59.414 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:59.414 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:59.414 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12550 2024-09-16T20:50:59.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.414 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:59.414 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.414 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:59.414 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757060+0000", 2024-09-16T20:50:59.414 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.414 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.414 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:59.414 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757060+0000", 2024-09-16T20:50:59.414 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.415 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.415 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.415 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:59.415 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757061+0000", 2024-09-16T20:50:59.415 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.15e-07 2024-09-16T20:50:59.415 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.415 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.415 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:59.415 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757062+0000", 2024-09-16T20:50:59.415 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.663e-06 2024-09-16T20:50:59.415 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.415 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.415 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:59.415 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757066+0000", 2024-09-16T20:50:59.415 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.817e-06 2024-09-16T20:50:59.416 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.416 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.416 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:59.416 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757070+0000", 2024-09-16T20:50:59.416 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1509999999999997e-06 2024-09-16T20:50:59.416 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.416 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.416 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:59.416 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823044+0000", 2024-09-16T20:50:59.416 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065973992999999995 2024-09-16T20:50:59.416 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.416 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.416 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:59.416 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823082+0000", 2024-09-16T20:50:59.417 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8050999999999999e-05 2024-09-16T20:50:59.417 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.417 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.417 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:59.417 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823214+0000", 2024-09-16T20:50:59.417 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013157500000000001 2024-09-16T20:50:59.417 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.417 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:59.417 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.417 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.417 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.417 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12551 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 644096~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:59.417 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:50:59.417 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 68.644961828000007, 2024-09-16T20:50:59.418 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066391378000000001, 2024-09-16T20:50:59.418 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:59.418 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:59.418 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:59.418 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:59.418 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:59.418 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:59.418 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12551 2024-09-16T20:50:59.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.418 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:59.418 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.418 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:59.418 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:50:59.419 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.419 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.419 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.419 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:59.419 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:50:59.419 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.419 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.419 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.419 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:59.419 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:50:59.419 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2999999999999999e-07 2024-09-16T20:50:59.419 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.419 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.419 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:59.419 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757075+0000", 2024-09-16T20:50:59.420 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.097e-06 2024-09-16T20:50:59.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.420 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.420 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:59.420 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757082+0000", 2024-09-16T20:50:59.420 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9500000000000004e-06 2024-09-16T20:50:59.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.420 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.420 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:59.420 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757086+0000", 2024-09-16T20:50:59.420 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2229999999999999e-06 2024-09-16T20:50:59.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.420 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.420 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:59.420 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823262+0000", 2024-09-16T20:50:59.421 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066176078999999999 2024-09-16T20:50:59.421 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.421 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.421 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:59.421 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823336+0000", 2024-09-16T20:50:59.421 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3979000000000004e-05 2024-09-16T20:50:59.421 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.421 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.421 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:59.421 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823465+0000", 2024-09-16T20:50:59.421 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012982 2024-09-16T20:50:59.421 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.421 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:59.421 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.422 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.422 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.422 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12552 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 649216~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:59.422 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757089+0000", 2024-09-16T20:50:59.422 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 68.644946687000001, 2024-09-16T20:50:59.422 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066655617, 2024-09-16T20:50:59.422 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:59.422 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:59.422 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:59.422 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:59.422 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:59.422 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:59.422 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12552 2024-09-16T20:50:59.422 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.423 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:59.423 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.423 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:59.423 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757089+0000", 2024-09-16T20:50:59.423 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.423 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.423 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:59.423 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757089+0000", 2024-09-16T20:50:59.423 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.423 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.423 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:59.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759492+0000", 2024-09-16T20:50:59.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0024026109999999998 2024-09-16T20:50:59.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.424 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:59.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759497+0000", 2024-09-16T20:50:59.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1120000000000004e-06 2024-09-16T20:50:59.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.424 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:59.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759508+0000", 2024-09-16T20:50:59.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0981e-05 2024-09-16T20:50:59.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.425 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:59.425 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759516+0000", 2024-09-16T20:50:59.425 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7270000000000007e-06 2024-09-16T20:50:59.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.425 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:59.425 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823504+0000", 2024-09-16T20:50:59.425 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.063987984999999997 2024-09-16T20:50:59.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.425 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:59.425 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823579+0000", 2024-09-16T20:50:59.425 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5703000000000003e-05 2024-09-16T20:50:59.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.426 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.426 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:59.426 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823745+0000", 2024-09-16T20:50:59.426 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016549800000000001 2024-09-16T20:50:59.426 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.426 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:59.426 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.426 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.426 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.426 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12560 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 690176~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:59.426 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.759658+0000", 2024-09-16T20:50:59.426 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 68.642377484999997, 2024-09-16T20:50:59.426 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065858517000000005, 2024-09-16T20:50:59.427 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:59.427 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:59.427 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:59.427 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:59.427 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:59.427 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:59.427 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12560 2024-09-16T20:50:59.427 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.427 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:59.427 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.427 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:59.427 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759658+0000", 2024-09-16T20:50:59.427 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.428 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.428 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.428 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:59.428 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759658+0000", 2024-09-16T20:50:59.428 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.428 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.428 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.428 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:59.428 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759659+0000", 2024-09-16T20:50:59.428 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.17e-07 2024-09-16T20:50:59.428 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.428 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.428 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:59.428 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759660+0000", 2024-09-16T20:50:59.429 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0189999999999999e-06 2024-09-16T20:50:59.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.429 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.429 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:59.429 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759663+0000", 2024-09-16T20:50:59.429 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0010000000000002e-06 2024-09-16T20:50:59.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.429 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.429 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:59.429 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759665+0000", 2024-09-16T20:50:59.429 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5440000000000001e-06 2024-09-16T20:50:59.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.429 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.429 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:59.429 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825355+0000", 2024-09-16T20:50:59.430 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065690224000000005 2024-09-16T20:50:59.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.430 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.430 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:59.430 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825394+0000", 2024-09-16T20:50:59.430 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8810999999999998e-05 2024-09-16T20:50:59.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.430 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.430 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:59.430 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825517+0000", 2024-09-16T20:50:59.430 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012270099999999999 2024-09-16T20:50:59.430 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.430 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:59.430 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.431 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.431 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12561 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 695296~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:59.431 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.759668+0000", 2024-09-16T20:50:59.431 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 68.642367643, 2024-09-16T20:50:59.431 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066116999999999995, 2024-09-16T20:50:59.431 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:59.431 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:59.431 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:59.431 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:59.431 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:59.431 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:59.431 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12561 2024-09-16T20:50:59.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.431 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:59.432 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.432 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:59.432 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759668+0000", 2024-09-16T20:50:59.432 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.432 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.432 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:59.432 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759668+0000", 2024-09-16T20:50:59.432 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.432 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.432 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:59.432 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765379+0000", 2024-09-16T20:50:59.432 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057104410000000001 2024-09-16T20:50:59.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.433 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.433 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:59.433 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765383+0000", 2024-09-16T20:50:59.433 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2980000000000002e-06 2024-09-16T20:50:59.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.433 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.433 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:59.433 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765395+0000", 2024-09-16T20:50:59.433 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1671000000000001e-05 2024-09-16T20:50:59.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.433 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.433 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:59.433 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765403+0000", 2024-09-16T20:50:59.434 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1210000000000007e-06 2024-09-16T20:50:59.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.434 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.434 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:59.434 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825552+0000", 2024-09-16T20:50:59.434 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.060149498000000003 2024-09-16T20:50:59.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.434 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.434 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:59.434 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825625+0000", 2024-09-16T20:50:59.434 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2540999999999995e-05 2024-09-16T20:50:59.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.434 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.434 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:59.435 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825785+0000", 2024-09-16T20:50:59.435 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016043000000000001 2024-09-16T20:50:59.435 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.435 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:59.435 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.435 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.435 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12585 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 818176~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:59.435 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.765848+0000", 2024-09-16T20:50:59.435 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 68.636187672000005, 2024-09-16T20:50:59.435 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065843218999999995, 2024-09-16T20:50:59.435 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:59.435 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:59.436 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:59.436 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:59.436 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:59.436 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:59.436 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12585 2024-09-16T20:50:59.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.436 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:59.436 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.436 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:59.436 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765848+0000", 2024-09-16T20:50:59.436 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.436 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.436 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:59.436 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765848+0000", 2024-09-16T20:50:59.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.437 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:59.437 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768683+0000", 2024-09-16T20:50:59.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0028344519999999999 2024-09-16T20:50:59.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.437 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:59.437 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768686+0000", 2024-09-16T20:50:59.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8789999999999996e-06 2024-09-16T20:50:59.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:59.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768696+0000", 2024-09-16T20:50:59.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6709999999999994e-06 2024-09-16T20:50:59.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:59.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768702+0000", 2024-09-16T20:50:59.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2550000000000003e-06 2024-09-16T20:50:59.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:59.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.831499+0000", 2024-09-16T20:50:59.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.062797081000000005 2024-09-16T20:50:59.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:59.439 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.831538+0000", 2024-09-16T20:50:59.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8523999999999999e-05 2024-09-16T20:50:59.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.439 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.439 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:59.439 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.831691+0000", 2024-09-16T20:50:59.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015335699999999999 2024-09-16T20:50:59.439 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.439 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:59.439 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.439 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.439 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12597 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 879616~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:59.439 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.768883+0000", 2024-09-16T20:50:59.440 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 68.633152944000003, 2024-09-16T20:50:59.440 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065816527, 2024-09-16T20:50:59.440 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:59.440 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:59.440 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:59.440 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:59.440 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:59.440 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:59.440 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12597 2024-09-16T20:50:59.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.440 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:59.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.440 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:59.440 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768883+0000", 2024-09-16T20:50:59.440 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:59.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768883+0000", 2024-09-16T20:50:59.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:59.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771508+0000", 2024-09-16T20:50:59.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0026247589999999999 2024-09-16T20:50:59.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:59.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771513+0000", 2024-09-16T20:50:59.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9949999999999996e-06 2024-09-16T20:50:59.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:59.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771525+0000", 2024-09-16T20:50:59.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2033e-05 2024-09-16T20:50:59.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:59.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771532+0000", 2024-09-16T20:50:59.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4220000000000003e-06 2024-09-16T20:50:59.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:59.443 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.834532+0000", 2024-09-16T20:50:59.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.063000159 2024-09-16T20:50:59.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.443 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:59.443 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.834590+0000", 2024-09-16T20:50:59.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8218999999999999e-05 2024-09-16T20:50:59.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.443 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:59.443 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.834699+0000", 2024-09-16T20:50:59.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010894 2024-09-16T20:50:59.443 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.443 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:59.443 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.444 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12619 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 992256~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:59.444 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.774656+0000", 2024-09-16T20:50:59.444 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 68.627380169999995, 2024-09-16T20:50:59.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065715859000000001, 2024-09-16T20:50:59.444 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:59.444 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:59.444 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:59.444 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:59.444 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:59.444 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:59.444 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12619 2024-09-16T20:50:59.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.444 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:59.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:59.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774656+0000", 2024-09-16T20:50:59.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:59.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774656+0000", 2024-09-16T20:50:59.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:59.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774875+0000", 2024-09-16T20:50:59.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000218898 2024-09-16T20:50:59.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.446 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:59.446 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774879+0000", 2024-09-16T20:50:59.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0389999999999998e-06 2024-09-16T20:50:59.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.446 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:59.446 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774890+0000", 2024-09-16T20:50:59.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1250000000000001e-05 2024-09-16T20:50:59.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.446 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:59.446 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774897+0000", 2024-09-16T20:50:59.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7619999999999998e-06 2024-09-16T20:50:59.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:59.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840139+0000", 2024-09-16T20:50:59.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065242438 2024-09-16T20:50:59.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:59.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840186+0000", 2024-09-16T20:50:59.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6894999999999997e-05 2024-09-16T20:50:59.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:59.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840372+0000", 2024-09-16T20:50:59.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000185577 2024-09-16T20:50:59.448 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.448 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:59.448 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.448 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12620 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 997376~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:59.448 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.774908+0000", 2024-09-16T20:50:59.448 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 68.627127948999998, 2024-09-16T20:50:59.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065782754999999998, 2024-09-16T20:50:59.448 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:59.448 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:59.448 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:59.448 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:59.449 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:59.449 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:59.449 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12620 2024-09-16T20:50:59.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.449 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:59.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.449 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:59.449 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774908+0000", 2024-09-16T20:50:59.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.449 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:59.449 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774908+0000", 2024-09-16T20:50:59.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.450 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:59.450 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775102+0000", 2024-09-16T20:50:59.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000194098 2024-09-16T20:50:59.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.450 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:59.450 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775105+0000", 2024-09-16T20:50:59.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8739999999999999e-06 2024-09-16T20:50:59.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.450 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:59.450 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775111+0000", 2024-09-16T20:50:59.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9519999999999999e-06 2024-09-16T20:50:59.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.451 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:59.451 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775115+0000", 2024-09-16T20:50:59.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.002e-06 2024-09-16T20:50:59.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.451 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:59.451 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840409+0000", 2024-09-16T20:50:59.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065294195999999999 2024-09-16T20:50:59.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.451 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:59.451 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840507+0000", 2024-09-16T20:50:59.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7662000000000001e-05 2024-09-16T20:50:59.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:59.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840691+0000", 2024-09-16T20:50:59.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000183971 2024-09-16T20:50:59.452 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.452 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:59.452 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.452 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12621 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1002496~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:59.452 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775123+0000", 2024-09-16T20:50:59.453 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 68.626913252999998, 2024-09-16T20:50:59.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065829586999999995, 2024-09-16T20:50:59.453 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:59.453 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:59.453 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:59.453 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:59.453 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:59.453 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:59.453 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12621 2024-09-16T20:50:59.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.453 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:59.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:59.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775123+0000", 2024-09-16T20:50:59.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:59.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775123+0000", 2024-09-16T20:50:59.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:59.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775348+0000", 2024-09-16T20:50:59.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000225295 2024-09-16T20:50:59.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:59.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775352+0000", 2024-09-16T20:50:59.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0810000000000004e-06 2024-09-16T20:50:59.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:59.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775361+0000", 2024-09-16T20:50:59.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7399999999999993e-06 2024-09-16T20:50:59.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.456 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:59.456 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775367+0000", 2024-09-16T20:50:59.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5200000000000003e-06 2024-09-16T20:50:59.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.456 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:59.456 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840730+0000", 2024-09-16T20:50:59.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065362803999999997 2024-09-16T20:50:59.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.456 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:59.456 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840801+0000", 2024-09-16T20:50:59.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1006000000000004e-05 2024-09-16T20:50:59.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.457 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:59.457 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840952+0000", 2024-09-16T20:50:59.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015114100000000001 2024-09-16T20:50:59.457 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.457 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:59.457 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.457 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12622 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1007616~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:59.457 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775381+0000", 2024-09-16T20:50:59.457 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 68.626654419000005, 2024-09-16T20:50:59.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065822581000000005, 2024-09-16T20:50:59.458 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:59.458 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:59.458 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:59.458 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:59.458 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:59.458 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:59.458 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12622 2024-09-16T20:50:59.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.458 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:59.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:59.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775381+0000", 2024-09-16T20:50:59.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:59.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775381+0000", 2024-09-16T20:50:59.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:59.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775563+0000", 2024-09-16T20:50:59.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018153700000000001 2024-09-16T20:50:59.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:59.460 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775566+0000", 2024-09-16T20:50:59.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9620000000000002e-06 2024-09-16T20:50:59.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:59.460 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775572+0000", 2024-09-16T20:50:59.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8529999999999997e-06 2024-09-16T20:50:59.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:59.460 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775576+0000", 2024-09-16T20:50:59.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8920000000000002e-06 2024-09-16T20:50:59.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:59.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840990+0000", 2024-09-16T20:50:59.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065414599000000004 2024-09-16T20:50:59.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:59.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841068+0000", 2024-09-16T20:50:59.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8157000000000004e-05 2024-09-16T20:50:59.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:59.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841204+0000", 2024-09-16T20:50:59.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013558099999999999 2024-09-16T20:50:59.462 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.462 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:59.462 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.462 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12623 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1012736~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:59.462 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775583+0000", 2024-09-16T20:50:59.462 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 68.626452981, 2024-09-16T20:50:59.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065929087999999997, 2024-09-16T20:50:59.462 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:59.462 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:59.462 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:59.462 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:59.463 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:59.463 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:59.463 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12623 2024-09-16T20:50:59.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.463 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:59.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.463 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:59.463 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775583+0000", 2024-09-16T20:50:59.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.463 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:59.463 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775583+0000", 2024-09-16T20:50:59.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.464 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:59.464 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775804+0000", 2024-09-16T20:50:59.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00022116900000000001 2024-09-16T20:50:59.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.464 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:59.464 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775807+0000", 2024-09-16T20:50:59.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9529999999999999e-06 2024-09-16T20:50:59.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.464 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:59.464 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775814+0000", 2024-09-16T20:50:59.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7490000000000001e-06 2024-09-16T20:50:59.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.465 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:59.465 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775818+0000", 2024-09-16T20:50:59.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1999999999999996e-06 2024-09-16T20:50:59.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.465 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:59.465 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841267+0000", 2024-09-16T20:50:59.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065449127999999995 2024-09-16T20:50:59.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.465 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:59.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841344+0000", 2024-09-16T20:50:59.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6546999999999997e-05 2024-09-16T20:50:59.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:59.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841512+0000", 2024-09-16T20:50:59.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016834200000000001 2024-09-16T20:50:59.466 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.466 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:59.466 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.466 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12624 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1017856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:59.466 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:50:59.467 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 68.626209841000005, 2024-09-16T20:50:59.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065945866000000006, 2024-09-16T20:50:59.467 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:59.467 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:59.467 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:59.467 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:59.467 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:59.467 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:59.467 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12624 2024-09-16T20:50:59.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.467 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:59.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.467 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:59.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:50:59.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:59.468 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:50:59.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:59.468 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:50:59.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-16T20:50:59.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:59.469 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775828+0000", 2024-09-16T20:50:59.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3999999999999999e-06 2024-09-16T20:50:59.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.469 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:59.469 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775831+0000", 2024-09-16T20:50:59.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.478e-06 2024-09-16T20:50:59.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.469 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:59.469 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775834+0000", 2024-09-16T20:50:59.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9440000000000001e-06 2024-09-16T20:50:59.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.469 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:59.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841547+0000", 2024-09-16T20:50:59.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065712801000000001 2024-09-16T20:50:59.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:59.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841618+0000", 2024-09-16T20:50:59.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1151999999999999e-05 2024-09-16T20:50:59.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:59.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841772+0000", 2024-09-16T20:50:59.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000153721 2024-09-16T20:50:59.470 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.470 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:59.470 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.471 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12625 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1022976~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:59.471 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775840+0000", 2024-09-16T20:50:59.471 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 68.626196231999998, 2024-09-16T20:50:59.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066165451, 2024-09-16T20:50:59.471 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:59.471 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:59.471 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:59.471 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:59.471 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:59.471 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:59.471 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12625 2024-09-16T20:50:59.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.471 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:59.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:59.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775840+0000", 2024-09-16T20:50:59.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:59.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775840+0000", 2024-09-16T20:50:59.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:59.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776014+0000", 2024-09-16T20:50:59.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017441099999999999 2024-09-16T20:50:59.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:59.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776018+0000", 2024-09-16T20:50:59.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6509999999999999e-06 2024-09-16T20:50:59.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:59.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776026+0000", 2024-09-16T20:50:59.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.028e-06 2024-09-16T20:50:59.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:59.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776031+0000", 2024-09-16T20:50:59.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8400000000000002e-06 2024-09-16T20:50:59.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:59.474 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841801+0000", 2024-09-16T20:50:59.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065770472999999996 2024-09-16T20:50:59.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:59.474 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841847+0000", 2024-09-16T20:50:59.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5558e-05 2024-09-16T20:50:59.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:59.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842005+0000", 2024-09-16T20:50:59.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015849000000000001 2024-09-16T20:50:59.475 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.475 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:59.475 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.475 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12626 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1028096~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:59.475 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776041+0000", 2024-09-16T20:50:59.475 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 68.625994933000001, 2024-09-16T20:50:59.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066225260999999994, 2024-09-16T20:50:59.475 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:59.475 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:59.475 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:59.475 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:59.476 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:59.476 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:59.476 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12626 2024-09-16T20:50:59.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.476 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:59.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.476 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:59.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776041+0000", 2024-09-16T20:50:59.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.476 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:59.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776041+0000", 2024-09-16T20:50:59.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:59.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776263+0000", 2024-09-16T20:50:59.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00022193899999999999 2024-09-16T20:50:59.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:59.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776268+0000", 2024-09-16T20:50:59.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8049999999999997e-06 2024-09-16T20:50:59.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:59.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776280+0000", 2024-09-16T20:50:59.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2635e-05 2024-09-16T20:50:59.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:59.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776288+0000", 2024-09-16T20:50:59.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8199999999999997e-06 2024-09-16T20:50:59.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:59.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842041+0000", 2024-09-16T20:50:59.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065752506000000002 2024-09-16T20:50:59.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:59.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842123+0000", 2024-09-16T20:50:59.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2393999999999998e-05 2024-09-16T20:50:59.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:59.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842266+0000", 2024-09-16T20:50:59.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014316200000000001 2024-09-16T20:50:59.479 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.479 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:59.479 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.479 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12627 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1033216~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:59.479 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776301+0000", 2024-09-16T20:50:59.480 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 68.625734972999993, 2024-09-16T20:50:59.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066187814999999997, 2024-09-16T20:50:59.480 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:59.480 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:59.480 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:59.480 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:59.480 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:59.480 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:59.480 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12627 2024-09-16T20:50:59.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.480 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:59.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:59.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776301+0000", 2024-09-16T20:50:59.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.481 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:59.481 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776301+0000", 2024-09-16T20:50:59.481 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.481 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:59.481 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776473+0000", 2024-09-16T20:50:59.481 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017222400000000001 2024-09-16T20:50:59.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.481 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:59.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776477+0000", 2024-09-16T20:50:59.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7019999999999999e-06 2024-09-16T20:50:59.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.482 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:59.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776486+0000", 2024-09-16T20:50:59.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2119999999999992e-06 2024-09-16T20:50:59.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.482 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:59.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776492+0000", 2024-09-16T20:50:59.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8089999999999998e-06 2024-09-16T20:50:59.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.482 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:59.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842301+0000", 2024-09-16T20:50:59.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065809042999999998 2024-09-16T20:50:59.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:59.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842362+0000", 2024-09-16T20:50:59.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1260000000000006e-05 2024-09-16T20:50:59.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:59.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842489+0000", 2024-09-16T20:50:59.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012656500000000001 2024-09-16T20:50:59.483 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.483 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:59.483 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.484 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12628 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1038336~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:59.484 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776502+0000", 2024-09-16T20:50:59.484 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 68.625534326999997, 2024-09-16T20:50:59.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066283426000000006, 2024-09-16T20:50:59.484 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:59.484 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:59.484 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:59.484 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:59.484 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:59.484 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:59.484 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12628 2024-09-16T20:50:59.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.484 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:59.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.485 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:59.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776502+0000", 2024-09-16T20:50:59.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.485 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:59.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776502+0000", 2024-09-16T20:50:59.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.485 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:59.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776716+0000", 2024-09-16T20:50:59.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000214596 2024-09-16T20:50:59.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.486 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:59.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776723+0000", 2024-09-16T20:50:59.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4000000000000003e-06 2024-09-16T20:50:59.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.486 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:59.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776731+0000", 2024-09-16T20:50:59.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6129999999999998e-06 2024-09-16T20:50:59.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.486 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:59.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776737+0000", 2024-09-16T20:50:59.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1970000000000001e-06 2024-09-16T20:50:59.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:59.487 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842532+0000", 2024-09-16T20:50:59.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065795127999999994 2024-09-16T20:50:59.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:59.487 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842599+0000", 2024-09-16T20:50:59.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6711000000000006e-05 2024-09-16T20:50:59.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:59.487 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842785+0000", 2024-09-16T20:50:59.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018578099999999999 2024-09-16T20:50:59.487 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.488 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:59.488 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.488 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12629 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1043456~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:59.488 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776748+0000", 2024-09-16T20:50:59.488 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 68.625287950000001, 2024-09-16T20:50:59.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066303788000000002, 2024-09-16T20:50:59.488 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:59.488 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:59.488 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:59.488 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:59.488 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:59.488 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:59.489 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12629 2024-09-16T20:50:59.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.489 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:59.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:59.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776748+0000", 2024-09-16T20:50:59.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:59.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776748+0000", 2024-09-16T20:50:59.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:59.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776934+0000", 2024-09-16T20:50:59.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018598399999999999 2024-09-16T20:50:59.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:59.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776939+0000", 2024-09-16T20:50:59.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1000000000000003e-06 2024-09-16T20:50:59.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:59.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776954+0000", 2024-09-16T20:50:59.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5221e-05 2024-09-16T20:50:59.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:59.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776962+0000", 2024-09-16T20:50:59.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.554e-06 2024-09-16T20:50:59.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:59.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842813+0000", 2024-09-16T20:50:59.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065851006000000004 2024-09-16T20:50:59.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:59.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842886+0000", 2024-09-16T20:50:59.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3663000000000003e-05 2024-09-16T20:50:59.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:59.492 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.843052+0000", 2024-09-16T20:50:59.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016526 2024-09-16T20:50:59.492 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.492 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:59.492 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.492 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.492 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:59.492 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:59.493 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:50:59.494 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:59.492+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:59.571 INFO:teuthology.orchestra.run.smithi029.stderr:marked in osd.0. 2024-09-16T20:50:59.594 INFO:tasks.thrashosds.thrasher:Added osd 0 2024-09-16T20:50:59.594 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:50:59.592+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:50:59.647 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:59.647 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:50:59.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.648 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:29912 3.17 3.5b6ed7f7 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e172)", 2024-09-16T20:50:59.648 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:59.571233+0000", 2024-09-16T20:50:59.648 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.076061818000000003, 2024-09-16T20:50:59.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076092098999999996, 2024-09-16T20:50:59.648 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:59.648 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:59.648 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "waiting for peered", 2024-09-16T20:50:59.648 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:59.648 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:50:59.648 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:59.648 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 29912 2024-09-16T20:50:59.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.648 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:59.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:59.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:59.571233+0000", 2024-09-16T20:50:59.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:59.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:59.571233+0000", 2024-09-16T20:50:59.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:59.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:59.571234+0000", 2024-09-16T20:50:59.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6810000000000001e-06 2024-09-16T20:50:59.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.650 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:59.650 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:59.571240+0000", 2024-09-16T20:50:59.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9370000000000001e-06 2024-09-16T20:50:59.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.650 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:59.650 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:59.571252+0000", 2024-09-16T20:50:59.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1447999999999999e-05 2024-09-16T20:50:59.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.650 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:59.650 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:59.571262+0000", 2024-09-16T20:50:59.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0456e-05 2024-09-16T20:50:59.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.651 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:59.651 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:59.572012+0000", 2024-09-16T20:50:59.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00074957099999999998 2024-09-16T20:50:59.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.651 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:59.651 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:59.572023+0000", 2024-09-16T20:50:59.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1265e-05 2024-09-16T20:50:59.651 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.651 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:59.651 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.652 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:29913 3.17 3.5b6ed7f7 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e172)", 2024-09-16T20:50:59.652 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:59.571276+0000", 2024-09-16T20:50:59.652 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.076018817000000002, 2024-09-16T20:50:59.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076095328000000004, 2024-09-16T20:50:59.652 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:59.652 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:59.652 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "waiting for peered", 2024-09-16T20:50:59.652 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:59.652 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:50:59.652 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:59.652 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 29913 2024-09-16T20:50:59.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.652 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:59.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:59.653 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:59.571276+0000", 2024-09-16T20:50:59.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:59.653 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:59.571276+0000", 2024-09-16T20:50:59.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:59.653 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:59.571277+0000", 2024-09-16T20:50:59.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.06e-07 2024-09-16T20:50:59.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:59.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:59.571280+0000", 2024-09-16T20:50:59.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5159999999999999e-06 2024-09-16T20:50:59.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:59.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:59.571287+0000", 2024-09-16T20:50:59.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1799999999999999e-06 2024-09-16T20:50:59.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:59.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:59.571293+0000", 2024-09-16T20:50:59.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9739999999999999e-06 2024-09-16T20:50:59.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.655 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.655 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:59.655 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:59.572052+0000", 2024-09-16T20:50:59.655 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00075849299999999995 2024-09-16T20:50:59.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.655 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.655 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:59.655 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:59.572059+0000", 2024-09-16T20:50:59.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6410000000000002e-06 2024-09-16T20:50:59.656 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.656 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:59.656 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.656 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.656 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-16T20:50:59.656 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 2 2024-09-16T20:50:59.656 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:59.657 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:50:59.695 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:50:59.693+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:50:59.764 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:59.764 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:50:59.764 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:50:59.764 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:50:59.764 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:59.773 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:50:59.795 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:50:59.795+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:50:59.882 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:50:59.882 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:50:59.882 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:50:59.882 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:50:59.882 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.883 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5286.0:10643 3.17 3.f6273fb7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-16T20:50:59.883 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:50:59.883 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 111.089790529, 2024-09-16T20:50:59.883 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.76672347200000002, 2024-09-16T20:50:59.883 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:59.883 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:59.883 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:59.883 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:59.883 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5286", 2024-09-16T20:50:59.883 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/4135433989", 2024-09-16T20:50:59.883 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 10643 2024-09-16T20:50:59.883 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.883 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:59.883 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.883 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:59.884 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:50:59.884 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.884 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.884 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.884 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:59.884 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:50:59.884 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.884 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.884 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.884 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:59.884 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790489+0000", 2024-09-16T20:50:59.884 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5910000000000001e-06 2024-09-16T20:50:59.884 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.884 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.885 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:59.885 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790493+0000", 2024-09-16T20:50:59.885 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2669999999999998e-06 2024-09-16T20:50:59.885 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.885 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.885 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:59.885 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790500+0000", 2024-09-16T20:50:59.885 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3780000000000003e-06 2024-09-16T20:50:59.885 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.885 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.885 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:59.885 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790505+0000", 2024-09-16T20:50:59.885 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1560000000000003e-06 2024-09-16T20:50:59.885 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.886 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.886 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:59.886 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790559+0000", 2024-09-16T20:50:59.886 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3494e-05 2024-09-16T20:50:59.886 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.886 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.886 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:59.886 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790566+0000", 2024-09-16T20:50:59.886 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7649999999999993e-06 2024-09-16T20:50:59.886 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.886 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.886 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:59.886 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:09.557198+0000", 2024-09-16T20:50:59.886 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.76663136200000004 2024-09-16T20:50:59.886 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.887 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.887 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:59.887 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:09.557211+0000", 2024-09-16T20:50:59.887 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3458999999999999e-05 2024-09-16T20:50:59.887 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.887 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:59.887 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.887 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.887 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.887 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5512.0:450 3.2 3:44e0cffa:::rbd_header.110ef0d7a535:head [watch reconnect cookie 140263298018912 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:50:59.887 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:50:59.887 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 83.317767615999998, 2024-09-16T20:50:59.887 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0084581939999999, 2024-09-16T20:50:59.888 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:59.888 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:59.888 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:59.888 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:59.888 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5512", 2024-09-16T20:50:59.888 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:50:59.888 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 450 2024-09-16T20:50:59.888 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.888 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:59.888 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.888 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:59.888 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:50:59.888 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.888 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.889 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.889 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:59.889 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:50:59.889 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.889 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.889 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.889 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:59.889 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562512+0000", 2024-09-16T20:50:59.889 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.009e-06 2024-09-16T20:50:59.889 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.889 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.889 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:59.889 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562517+0000", 2024-09-16T20:50:59.889 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9339999999999997e-06 2024-09-16T20:50:59.889 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.890 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.890 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:59.890 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562530+0000", 2024-09-16T20:50:59.890 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3495e-05 2024-09-16T20:50:59.890 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.890 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.890 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:59.890 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562536+0000", 2024-09-16T20:50:59.890 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5690000000000004e-06 2024-09-16T20:50:59.890 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.890 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.890 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:59.890 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562591+0000", 2024-09-16T20:50:59.890 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5761999999999999e-05 2024-09-16T20:50:59.891 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.891 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.891 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:50:59.891 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562599+0000", 2024-09-16T20:50:59.891 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9179999999999994e-06 2024-09-16T20:50:59.891 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.891 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.891 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:59.891 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.565971+0000", 2024-09-16T20:50:59.891 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003371546 2024-09-16T20:50:59.891 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.891 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.891 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:59.891 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566019+0000", 2024-09-16T20:50:59.891 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7849000000000002e-05 2024-09-16T20:50:59.891 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.892 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.892 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:59.892 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.569382+0000", 2024-09-16T20:50:59.892 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.003362833 2024-09-16T20:50:59.892 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.892 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.892 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:59.892 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.569529+0000", 2024-09-16T20:50:59.892 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000147823 2024-09-16T20:50:59.892 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.892 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.892 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:59.892 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570969+0000", 2024-09-16T20:50:59.892 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001439456 2024-09-16T20:50:59.893 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.893 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:59.893 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.893 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.893 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.893 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:14996 3.2 3:44e0cffa:::rbd_header.110ef0d7a535:head [watch reconnect cookie 140213436269840 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:50:59.893 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:50:59.893 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 83.317719203999999, 2024-09-16T20:50:59.893 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.008813943, 2024-09-16T20:50:59.893 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:59.893 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:59.893 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:59.893 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:59.893 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:50:59.894 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:59.894 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 14996 2024-09-16T20:50:59.894 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.894 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:59.894 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.894 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:59.894 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:50:59.894 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.894 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.894 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.894 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:59.894 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:50:59.894 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.894 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.895 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.895 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:59.895 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562560+0000", 2024-09-16T20:50:59.895 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.06e-07 2024-09-16T20:50:59.895 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.895 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.895 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:59.895 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562563+0000", 2024-09-16T20:50:59.895 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.495e-06 2024-09-16T20:50:59.895 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.895 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.895 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:59.895 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562571+0000", 2024-09-16T20:50:59.895 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1950000000000003e-06 2024-09-16T20:50:59.895 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.896 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.896 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:59.896 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562579+0000", 2024-09-16T20:50:59.896 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2649999999999999e-06 2024-09-16T20:50:59.896 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.896 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.896 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:59.896 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562673+0000", 2024-09-16T20:50:59.896 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4118999999999998e-05 2024-09-16T20:50:59.896 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.896 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.896 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:50:59.896 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562677+0000", 2024-09-16T20:50:59.897 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5179999999999998e-06 2024-09-16T20:50:59.897 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.897 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.897 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:59.897 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566461+0000", 2024-09-16T20:50:59.897 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0037837259999999998 2024-09-16T20:50:59.897 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.897 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.897 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:59.897 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566468+0000", 2024-09-16T20:50:59.897 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3350000000000001e-06 2024-09-16T20:50:59.897 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.897 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.897 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:59.898 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570306+0000", 2024-09-16T20:50:59.898 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.003837997 2024-09-16T20:50:59.898 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.898 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.898 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:59.898 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570392+0000", 2024-09-16T20:50:59.898 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5872999999999996e-05 2024-09-16T20:50:59.898 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.898 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.898 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:59.899 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571373+0000", 2024-09-16T20:50:59.899 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00098071399999999993 2024-09-16T20:50:59.899 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.899 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:59.899 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.899 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.899 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.899 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5255.0:1931 3.2 3:47882819:::rbd_data.110eb5f95afc.0000000000000009:head [read 217088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:50:59.899 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:50:59.899 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 83.317680453999998, 2024-09-16T20:50:59.899 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0082752880000001, 2024-09-16T20:50:59.899 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:59.899 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:59.899 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:59.900 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:59.900 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5255", 2024-09-16T20:50:59.900 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:50:59.900 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1931 2024-09-16T20:50:59.900 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.900 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:59.900 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.901 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:59.901 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:50:59.901 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.901 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.901 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.901 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:59.901 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:50:59.901 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.901 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.901 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.902 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:59.902 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:50:59.902 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9699999999999996e-07 2024-09-16T20:50:59.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.902 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.902 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:59.902 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562601+0000", 2024-09-16T20:50:59.902 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8600000000000001e-06 2024-09-16T20:50:59.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.902 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.902 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:59.902 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562608+0000", 2024-09-16T20:50:59.902 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9e-06 2024-09-16T20:50:59.903 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.903 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.903 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:59.903 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562615+0000", 2024-09-16T20:50:59.903 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8179999999999998e-06 2024-09-16T20:50:59.903 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.903 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.903 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:59.903 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562703+0000", 2024-09-16T20:50:59.903 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7811000000000003e-05 2024-09-16T20:50:59.903 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.903 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.903 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:50:59.903 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562706+0000", 2024-09-16T20:50:59.904 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4920000000000002e-06 2024-09-16T20:50:59.904 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.904 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.904 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:59.904 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566685+0000", 2024-09-16T20:50:59.904 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0039792780000000002 2024-09-16T20:50:59.904 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.904 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.904 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:59.904 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566693+0000", 2024-09-16T20:50:59.904 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4730000000000003e-06 2024-09-16T20:50:59.904 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.905 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.905 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:59.905 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570737+0000", 2024-09-16T20:50:59.905 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0040436749999999 2024-09-16T20:50:59.905 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.905 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.905 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:59.905 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570873+0000", 2024-09-16T20:50:59.905 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000136484 2024-09-16T20:50:59.905 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.905 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:59.905 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.905 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.905 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.906 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5255.0:1939 3.2 3:47882819:::rbd_data.110eb5f95afc.0000000000000009:head [read 233472~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:50:59.906 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:50:59.906 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 83.317525587999995, 2024-09-16T20:50:59.906 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0084657589999999, 2024-09-16T20:50:59.906 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:59.906 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:59.906 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:59.906 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:59.906 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5255", 2024-09-16T20:50:59.906 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:50:59.906 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1939 2024-09-16T20:50:59.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.906 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:59.906 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.907 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:59.907 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:50:59.907 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.907 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.907 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.907 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:59.907 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:50:59.907 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.907 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.907 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.907 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:59.907 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:50:59.907 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5199999999999997e-07 2024-09-16T20:50:59.907 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.907 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.908 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:59.908 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562757+0000", 2024-09-16T20:50:59.908 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4979999999999998e-06 2024-09-16T20:50:59.908 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.908 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.908 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:59.908 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562764+0000", 2024-09-16T20:50:59.908 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1470000000000004e-06 2024-09-16T20:50:59.908 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.908 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.908 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:59.908 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562768+0000", 2024-09-16T20:50:59.909 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7929999999999999e-06 2024-09-16T20:50:59.909 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.909 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.909 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:59.909 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562805+0000", 2024-09-16T20:50:59.909 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7477e-05 2024-09-16T20:50:59.909 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.909 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.909 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting_for_map not empty", 2024-09-16T20:50:59.909 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562810+0000", 2024-09-16T20:50:59.909 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0819999999999998e-06 2024-09-16T20:50:59.910 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.910 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.910 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:59.910 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566717+0000", 2024-09-16T20:50:59.910 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0039072600000000001 2024-09-16T20:50:59.910 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.910 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.910 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:59.910 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566724+0000", 2024-09-16T20:50:59.910 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4880000000000004e-06 2024-09-16T20:50:59.910 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.910 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.910 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:59.910 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571009+0000", 2024-09-16T20:50:59.910 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.004284677 2024-09-16T20:50:59.910 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.911 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.911 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:59.911 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571218+0000", 2024-09-16T20:50:59.911 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00020978500000000001 2024-09-16T20:50:59.911 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.911 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:59.911 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.911 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.911 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.911 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:11020 3.1f 3:fee2d1dd:::rbd_data.110ea93691bb.000000000000000e:23 [read 955392~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:59.911 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.693811+0000", 2024-09-16T20:50:59.911 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 69.186466930999998, 2024-09-16T20:50:59.911 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056833005999999998, 2024-09-16T20:50:59.912 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:59.912 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:59.912 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:59.912 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:59.912 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:59.912 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:59.912 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 11020 2024-09-16T20:50:59.912 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.912 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:59.912 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.912 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:59.912 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693811+0000", 2024-09-16T20:50:59.912 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.912 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.913 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.913 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:59.913 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693811+0000", 2024-09-16T20:50:59.913 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.913 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.913 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.913 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:59.913 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693812+0000", 2024-09-16T20:50:59.913 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6100000000000001e-07 2024-09-16T20:50:59.913 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.913 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.913 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:59.913 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693815+0000", 2024-09-16T20:50:59.914 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7389999999999999e-06 2024-09-16T20:50:59.914 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.914 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.914 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:59.914 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693823+0000", 2024-09-16T20:50:59.914 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0439999999999996e-06 2024-09-16T20:50:59.914 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.914 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.914 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:59.914 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693828+0000", 2024-09-16T20:50:59.914 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7440000000000004e-06 2024-09-16T20:50:59.914 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.914 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.914 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:59.914 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750357+0000", 2024-09-16T20:50:59.915 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056529192999999998 2024-09-16T20:50:59.915 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.915 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.915 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:59.915 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750429+0000", 2024-09-16T20:50:59.915 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1557000000000006e-05 2024-09-16T20:50:59.915 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.915 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.915 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:59.915 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750644+0000", 2024-09-16T20:50:59.915 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00021526800000000001 2024-09-16T20:50:59.915 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.915 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:59.915 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.915 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.916 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.916 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:11021 3.1f 3:fee2d1dd:::rbd_data.110ea93691bb.000000000000000e:23 [read 960512~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:50:59.916 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.693837+0000", 2024-09-16T20:50:59.916 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 69.186441043000002, 2024-09-16T20:50:59.916 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057155138000000001, 2024-09-16T20:50:59.916 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:59.916 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:59.916 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:59.916 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:59.916 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:50:59.916 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:50:59.916 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 11021 2024-09-16T20:50:59.916 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.916 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:59.917 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.917 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:59.917 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693837+0000", 2024-09-16T20:50:59.917 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.917 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.917 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.917 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:59.917 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693837+0000", 2024-09-16T20:50:59.917 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.917 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.917 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.917 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:59.917 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699566+0000", 2024-09-16T20:50:59.917 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057283639999999997 2024-09-16T20:50:59.918 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.918 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.918 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:59.918 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699571+0000", 2024-09-16T20:50:59.918 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.604e-06 2024-09-16T20:50:59.918 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.918 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.918 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:59.918 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699583+0000", 2024-09-16T20:50:59.918 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1372999999999999e-05 2024-09-16T20:50:59.918 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.918 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.918 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:59.918 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699591+0000", 2024-09-16T20:50:59.918 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5259999999999993e-06 2024-09-16T20:50:59.919 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.919 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.919 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:59.919 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750674+0000", 2024-09-16T20:50:59.919 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051083154999999998 2024-09-16T20:50:59.919 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.919 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.919 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:59.919 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750745+0000", 2024-09-16T20:50:59.919 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0933999999999995e-05 2024-09-16T20:50:59.919 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.919 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.919 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:59.919 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750992+0000", 2024-09-16T20:50:59.919 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00024718199999999998 2024-09-16T20:50:59.919 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.920 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:59.920 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.920 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.920 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.920 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8791 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2332672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:59.920 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057279+0000", 2024-09-16T20:50:59.920 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 58.822999308, 2024-09-16T20:50:59.920 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056890067000000002, 2024-09-16T20:50:59.920 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:59.920 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:59.920 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:59.920 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:59.920 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:59.921 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:59.921 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8791 2024-09-16T20:50:59.921 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.921 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:59.921 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.921 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:59.921 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057279+0000", 2024-09-16T20:50:59.921 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.921 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.921 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.921 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:59.921 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057279+0000", 2024-09-16T20:50:59.921 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.921 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.922 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.922 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:59.922 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057279+0000", 2024-09-16T20:50:59.922 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.15e-07 2024-09-16T20:50:59.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.922 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.922 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:59.922 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057281+0000", 2024-09-16T20:50:59.922 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8080000000000001e-06 2024-09-16T20:50:59.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.922 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.922 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:59.922 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057287+0000", 2024-09-16T20:50:59.922 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5999999999999997e-06 2024-09-16T20:50:59.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.923 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.923 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:59.923 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057291+0000", 2024-09-16T20:50:59.923 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5990000000000004e-06 2024-09-16T20:50:59.923 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.923 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.923 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:59.923 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114078+0000", 2024-09-16T20:50:59.923 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056786246999999998 2024-09-16T20:50:59.923 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.923 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.923 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:59.923 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114118+0000", 2024-09-16T20:50:59.924 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9898999999999999e-05 2024-09-16T20:50:59.924 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.924 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.924 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:59.924 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114169+0000", 2024-09-16T20:50:59.924 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1499000000000003e-05 2024-09-16T20:50:59.924 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.924 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:59.924 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.924 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.924 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.924 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8792 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2609152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:59.924 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:50:59.924 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 58.822981325999997, 2024-09-16T20:50:59.925 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057003524999999999, 2024-09-16T20:50:59.925 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:59.925 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:59.925 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:59.925 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:59.925 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:59.925 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:59.925 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8792 2024-09-16T20:50:59.925 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.925 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:59.925 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.925 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:59.925 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:50:59.925 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.926 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.926 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.926 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:59.926 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:50:59.926 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.926 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.926 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.926 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:59.926 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:50:59.926 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0400000000000002e-07 2024-09-16T20:50:59.926 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.926 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.926 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:59.926 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057299+0000", 2024-09-16T20:50:59.926 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7460000000000001e-06 2024-09-16T20:50:59.927 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.927 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.927 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:59.927 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057304+0000", 2024-09-16T20:50:59.927 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3210000000000004e-06 2024-09-16T20:50:59.927 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.927 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.927 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:59.927 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057310+0000", 2024-09-16T20:50:59.927 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4360000000000001e-06 2024-09-16T20:50:59.927 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.927 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.927 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:59.927 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114200+0000", 2024-09-16T20:50:59.927 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056889954 2024-09-16T20:50:59.928 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.928 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.928 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:59.928 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114243+0000", 2024-09-16T20:50:59.928 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2998999999999999e-05 2024-09-16T20:50:59.928 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.928 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.928 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:59.928 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114300+0000", 2024-09-16T20:50:59.928 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7765e-05 2024-09-16T20:50:59.928 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.928 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:59.928 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.929 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.929 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.929 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8794 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2351104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:59.929 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057327+0000", 2024-09-16T20:50:59.929 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 58.822951002000003, 2024-09-16T20:50:59.929 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057101130999999999, 2024-09-16T20:50:59.929 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:59.929 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:59.929 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:59.929 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:59.929 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:59.929 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:59.929 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8794 2024-09-16T20:50:59.929 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.930 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:59.930 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.930 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:59.930 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057327+0000", 2024-09-16T20:50:59.930 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.930 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.930 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:59.930 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057327+0000", 2024-09-16T20:50:59.930 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.930 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.930 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:59.930 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057328+0000", 2024-09-16T20:50:59.931 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7399999999999999e-07 2024-09-16T20:50:59.931 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.931 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.931 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:59.931 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057330+0000", 2024-09-16T20:50:59.931 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2199999999999999e-06 2024-09-16T20:50:59.931 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.931 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.931 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:59.931 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057336+0000", 2024-09-16T20:50:59.931 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1240000000000002e-06 2024-09-16T20:50:59.931 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.931 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.931 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:59.931 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057341+0000", 2024-09-16T20:50:59.932 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8470000000000003e-06 2024-09-16T20:50:59.932 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.932 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.932 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:59.932 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114334+0000", 2024-09-16T20:50:59.932 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056993267 2024-09-16T20:50:59.932 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.932 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.932 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:59.932 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114376+0000", 2024-09-16T20:50:59.932 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1638999999999999e-05 2024-09-16T20:50:59.932 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.932 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.932 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:59.933 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114428+0000", 2024-09-16T20:50:59.933 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2660000000000001e-05 2024-09-16T20:50:59.933 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.933 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:59.933 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.933 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.933 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.933 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8798 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2297856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:59.933 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:50:59.933 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 58.822931275000002, 2024-09-16T20:50:59.933 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057277041000000001, 2024-09-16T20:50:59.933 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:59.933 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:59.934 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:59.934 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:59.934 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:59.934 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:59.934 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8798 2024-09-16T20:50:59.934 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.934 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:59.934 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.934 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:59.934 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:50:59.934 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.934 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.934 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.934 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:59.934 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:50:59.935 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.935 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.935 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:59.935 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:50:59.935 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2800000000000003e-07 2024-09-16T20:50:59.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.935 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.935 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:59.935 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057354+0000", 2024-09-16T20:50:59.935 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7780000000000002e-06 2024-09-16T20:50:59.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.935 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.935 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:59.936 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057360+0000", 2024-09-16T20:50:59.936 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0419999999999999e-06 2024-09-16T20:50:59.936 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.936 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.936 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:59.936 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057365+0000", 2024-09-16T20:50:59.936 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7929999999999997e-06 2024-09-16T20:50:59.936 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.936 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.936 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:59.936 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114460+0000", 2024-09-16T20:50:59.936 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057094662999999997 2024-09-16T20:50:59.936 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.936 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.937 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:59.937 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114532+0000", 2024-09-16T20:50:59.937 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2881999999999998e-05 2024-09-16T20:50:59.937 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.937 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.937 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:59.937 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114624+0000", 2024-09-16T20:50:59.937 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1555e-05 2024-09-16T20:50:59.937 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.937 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:59.937 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.937 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.937 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.937 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8800 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2316288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:59.937 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:50:59.938 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 58.822907336, 2024-09-16T20:50:59.938 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.05746946, 2024-09-16T20:50:59.938 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:59.938 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:59.938 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:59.938 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:59.938 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:59.938 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:59.938 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8800 2024-09-16T20:50:59.938 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.938 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:59.938 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.938 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:59.939 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:50:59.939 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.939 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:59.939 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:50:59.939 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.939 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:59.939 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:50:59.939 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5199999999999998e-07 2024-09-16T20:50:59.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.939 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:59.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057373+0000", 2024-09-16T20:50:59.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.79e-06 2024-09-16T20:50:59.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:59.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057379+0000", 2024-09-16T20:50:59.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6699999999999999e-06 2024-09-16T20:50:59.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:59.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057386+0000", 2024-09-16T20:50:59.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3170000000000004e-06 2024-09-16T20:50:59.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:59.941 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114655+0000", 2024-09-16T20:50:59.941 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057269420000000001 2024-09-16T20:50:59.941 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.941 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.941 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:59.941 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114734+0000", 2024-09-16T20:50:59.941 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8084000000000007e-05 2024-09-16T20:50:59.941 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.941 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.941 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:59.941 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114840+0000", 2024-09-16T20:50:59.941 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000106827 2024-09-16T20:50:59.941 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.941 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:59.941 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.942 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.942 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8801 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2334720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:59.942 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:50:59.942 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 58.822886304999997, 2024-09-16T20:50:59.942 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057594682000000001, 2024-09-16T20:50:59.942 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:59.942 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:59.942 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:59.942 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:59.942 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:59.942 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:59.942 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8801 2024-09-16T20:50:59.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.943 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:59.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.943 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:59.943 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:50:59.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.943 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:59.943 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:50:59.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.943 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:59.943 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:50:59.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6899999999999998e-07 2024-09-16T20:50:59.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.944 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:59.944 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057394+0000", 2024-09-16T20:50:59.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8300000000000001e-06 2024-09-16T20:50:59.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.944 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:59.944 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057400+0000", 2024-09-16T20:50:59.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6500000000000001e-06 2024-09-16T20:50:59.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.944 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:59.944 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057404+0000", 2024-09-16T20:50:59.945 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4669999999999998e-06 2024-09-16T20:50:59.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.945 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:59.945 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114874+0000", 2024-09-16T20:50:59.945 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057469602000000002 2024-09-16T20:50:59.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.945 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:59.945 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114933+0000", 2024-09-16T20:50:59.945 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9500000000000003e-05 2024-09-16T20:50:59.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.945 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:59.946 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114987+0000", 2024-09-16T20:50:59.946 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3263999999999997e-05 2024-09-16T20:50:59.946 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.946 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:59.946 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.946 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.946 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8802 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2353152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:59.946 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:50:59.946 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 58.822837620000001, 2024-09-16T20:50:59.946 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057714079000000001, 2024-09-16T20:50:59.946 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:59.946 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:59.946 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:59.947 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:59.947 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:59.947 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:59.947 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8802 2024-09-16T20:50:59.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.947 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:59.947 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.947 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:59.947 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:50:59.947 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.947 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.947 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:59.947 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:50:59.948 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.948 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:59.948 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:50:59.948 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4999999999999998e-07 2024-09-16T20:50:59.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.948 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:59.948 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057446+0000", 2024-09-16T20:50:59.948 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4240000000000003e-06 2024-09-16T20:50:59.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.948 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:59.949 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057452+0000", 2024-09-16T20:50:59.949 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8379999999999996e-06 2024-09-16T20:50:59.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.949 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.949 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:59.949 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057460+0000", 2024-09-16T20:50:59.949 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9629999999999998e-06 2024-09-16T20:50:59.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.949 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.949 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:59.949 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115015+0000", 2024-09-16T20:50:59.949 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057554389999999997 2024-09-16T20:50:59.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.949 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.949 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:59.950 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115082+0000", 2024-09-16T20:50:59.950 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7733999999999999e-05 2024-09-16T20:50:59.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.950 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.950 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:59.950 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115155+0000", 2024-09-16T20:50:59.950 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2280000000000006e-05 2024-09-16T20:50:59.950 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.950 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:59.950 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.950 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.950 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8804 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2555904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:59.950 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:50:59.951 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 58.822811303999998, 2024-09-16T20:50:59.951 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057833862, 2024-09-16T20:50:59.951 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:59.951 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:59.951 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:59.951 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:59.951 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:59.951 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:59.951 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8804 2024-09-16T20:50:59.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.951 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:59.951 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.951 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:59.951 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:50:59.952 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.952 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.952 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:59.952 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:50:59.952 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.952 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.952 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:59.952 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:50:59.952 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7899999999999999e-07 2024-09-16T20:50:59.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.952 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.952 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:59.953 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057469+0000", 2024-09-16T20:50:59.953 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.001e-06 2024-09-16T20:50:59.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.953 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.953 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:59.953 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057475+0000", 2024-09-16T20:50:59.953 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7629999999999997e-06 2024-09-16T20:50:59.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.953 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.953 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:59.953 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057480+0000", 2024-09-16T20:50:59.953 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8069999999999999e-06 2024-09-16T20:50:59.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.953 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.953 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:59.954 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115183+0000", 2024-09-16T20:50:59.954 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057703364 2024-09-16T20:50:59.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.954 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.954 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:59.954 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115227+0000", 2024-09-16T20:50:59.954 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3903999999999998e-05 2024-09-16T20:50:59.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.954 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.954 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:59.954 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115301+0000", 2024-09-16T20:50:59.954 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3543999999999999e-05 2024-09-16T20:50:59.954 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.954 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:59.954 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.955 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8805 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2574336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:59.955 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057487+0000", 2024-09-16T20:50:59.955 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 58.822790976, 2024-09-16T20:50:59.955 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058021027000000003, 2024-09-16T20:50:59.955 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:59.955 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:59.955 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:59.955 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:59.955 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:59.955 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:59.955 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8805 2024-09-16T20:50:59.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.956 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:59.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.956 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:59.956 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057487+0000", 2024-09-16T20:50:59.956 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.956 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:59.956 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057487+0000", 2024-09-16T20:50:59.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.957 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:59.957 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057488+0000", 2024-09-16T20:50:59.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5199999999999998e-07 2024-09-16T20:50:59.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.957 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:59.957 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057490+0000", 2024-09-16T20:50:59.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.939e-06 2024-09-16T20:50:59.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.957 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:59.958 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057495+0000", 2024-09-16T20:50:59.958 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6439999999999997e-06 2024-09-16T20:50:59.958 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.958 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.958 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:59.958 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057500+0000", 2024-09-16T20:50:59.958 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8119999999999998e-06 2024-09-16T20:50:59.958 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.958 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.958 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:59.958 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115341+0000", 2024-09-16T20:50:59.958 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057841341999999997 2024-09-16T20:50:59.958 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.959 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.959 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:59.959 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115422+0000", 2024-09-16T20:50:59.959 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1003999999999999e-05 2024-09-16T20:50:59.959 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.959 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.959 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:59.959 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115508+0000", 2024-09-16T20:50:59.959 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5933999999999994e-05 2024-09-16T20:50:59.959 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.959 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:59.959 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.959 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.959 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.960 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8807 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2592768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:59.960 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057505+0000", 2024-09-16T20:50:59.960 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 58.822772766, 2024-09-16T20:50:59.960 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058186633000000001, 2024-09-16T20:50:59.960 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:59.960 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:59.960 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:59.960 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:59.960 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:59.960 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:59.960 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8807 2024-09-16T20:50:59.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.960 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:59.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.961 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:59.961 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057505+0000", 2024-09-16T20:50:59.961 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.961 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.961 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:59.961 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057505+0000", 2024-09-16T20:50:59.961 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.961 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.962 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:59.962 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057506+0000", 2024-09-16T20:50:59.962 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.27e-07 2024-09-16T20:50:59.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.962 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:59.962 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057508+0000", 2024-09-16T20:50:59.962 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7400000000000001e-06 2024-09-16T20:50:59.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.962 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:59.962 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057513+0000", 2024-09-16T20:50:59.962 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.468e-06 2024-09-16T20:50:59.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.963 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.963 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:59.963 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057517+0000", 2024-09-16T20:50:59.963 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4329999999999997e-06 2024-09-16T20:50:59.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.963 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.963 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:59.963 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115538+0000", 2024-09-16T20:50:59.963 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058020355000000003 2024-09-16T20:50:59.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.963 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.963 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:59.963 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115616+0000", 2024-09-16T20:50:59.963 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7998999999999996e-05 2024-09-16T20:50:59.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.964 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.964 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:59.964 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115692+0000", 2024-09-16T20:50:59.964 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6310999999999995e-05 2024-09-16T20:50:59.964 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.964 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:59.964 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.964 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.964 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8808 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2611200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:50:59.964 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:50:59.964 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 58.822755505000003, 2024-09-16T20:50:59.965 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058354046, 2024-09-16T20:50:59.965 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:59.965 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:59.965 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:50:59.965 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:59.965 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:50:59.965 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:50:59.965 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8808 2024-09-16T20:50:59.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.965 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:59.965 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.965 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:59.965 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:50:59.965 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.966 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:59.966 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:50:59.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.966 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:59.966 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:50:59.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2500000000000001e-07 2024-09-16T20:50:59.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.966 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:59.966 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057525+0000", 2024-09-16T20:50:59.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5549999999999999e-06 2024-09-16T20:50:59.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.967 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:59.967 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057530+0000", 2024-09-16T20:50:59.967 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5489999999999997e-06 2024-09-16T20:50:59.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.967 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:59.967 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057535+0000", 2024-09-16T20:50:59.967 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5709999999999999e-06 2024-09-16T20:50:59.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.967 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:59.967 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115722+0000", 2024-09-16T20:50:59.968 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058187323999999999 2024-09-16T20:50:59.968 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.968 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.968 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:59.968 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115799+0000", 2024-09-16T20:50:59.968 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6445000000000004e-05 2024-09-16T20:50:59.968 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.968 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.968 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:59.968 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115877+0000", 2024-09-16T20:50:59.968 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8276999999999996e-05 2024-09-16T20:50:59.968 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.968 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:59.968 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.969 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.969 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.969 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:2182 3.17 3:edb5e5cd:::rbd_data.110e61479aee.0000000000000014:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write 0~5120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-16T20:50:59.969 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:22.734609+0000", 2024-09-16T20:50:59.969 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 37.145669257999998, 2024-09-16T20:50:59.969 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.124890894, 2024-09-16T20:50:59.969 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:59.969 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:59.969 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:50:59.969 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:59.969 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:50:59.969 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:50:59.969 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 2182 2024-09-16T20:50:59.969 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.969 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:59.970 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.970 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:59.970 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734609+0000", 2024-09-16T20:50:59.970 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.970 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.970 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:59.970 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734609+0000", 2024-09-16T20:50:59.970 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.970 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.970 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:59.970 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734611+0000", 2024-09-16T20:50:59.970 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9599999999999999e-06 2024-09-16T20:50:59.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.971 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.971 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:59.971 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734628+0000", 2024-09-16T20:50:59.971 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.715e-05 2024-09-16T20:50:59.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.971 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.971 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:59.971 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734656+0000", 2024-09-16T20:50:59.971 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7759999999999998e-05 2024-09-16T20:50:59.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.971 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.971 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:59.971 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734669+0000", 2024-09-16T20:50:59.971 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3315999999999999e-05 2024-09-16T20:50:59.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.972 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.972 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:59.972 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734826+0000", 2024-09-16T20:50:59.972 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015668499999999999 2024-09-16T20:50:59.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.972 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.972 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:50:59.972 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.735048+0000", 2024-09-16T20:50:59.972 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00022179 2024-09-16T20:50:59.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.972 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.972 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-16T20:50:59.972 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.735248+0000", 2024-09-16T20:50:59.972 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00020010399999999999 2024-09-16T20:50:59.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.973 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.973 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:50:59.973 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.736252+0000", 2024-09-16T20:50:59.973 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010039739999999999 2024-09-16T20:50:59.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.973 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.973 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:50:59.973 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.859340+0000", 2024-09-16T20:50:59.973 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.12308846599999999 2024-09-16T20:50:59.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.973 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.973 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:50:59.974 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.859431+0000", 2024-09-16T20:50:59.974 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1099000000000003e-05 2024-09-16T20:50:59.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.974 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:59.974 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.859500+0000", 2024-09-16T20:50:59.974 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8590000000000006e-05 2024-09-16T20:50:59.974 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.974 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:59.974 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.974 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:29157 3.1b 3:dbe69130:::journal.110e1b19a241:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e167)", 2024-09-16T20:50:59.974 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:54.663540+0000", 2024-09-16T20:50:59.974 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.2167379199999999, 2024-09-16T20:50:59.975 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.92912227999999997, 2024-09-16T20:50:59.975 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:50:59.975 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:50:59.975 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:50:59.975 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:50:59.975 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:50:59.975 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:50:59.975 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 29157 2024-09-16T20:50:59.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.975 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:50:59.975 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.975 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:50:59.975 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663540+0000", 2024-09-16T20:50:59.976 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.976 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.976 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:50:59.976 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663540+0000", 2024-09-16T20:50:59.976 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:50:59.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.976 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.976 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:50:59.976 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663541+0000", 2024-09-16T20:50:59.976 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1680000000000001e-06 2024-09-16T20:50:59.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.976 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.976 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:50:59.977 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663546+0000", 2024-09-16T20:50:59.977 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7129999999999996e-06 2024-09-16T20:50:59.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.977 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:50:59.977 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663558+0000", 2024-09-16T20:50:59.977 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1347e-05 2024-09-16T20:50:59.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.977 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:50:59.977 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663563+0000", 2024-09-16T20:50:59.977 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1250000000000001e-06 2024-09-16T20:50:59.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.978 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:59.978 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663612+0000", 2024-09-16T20:50:59.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9665e-05 2024-09-16T20:50:59.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.978 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.978 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:50:59.978 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663620+0000", 2024-09-16T20:50:59.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8439999999999998e-06 2024-09-16T20:50:59.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.978 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.978 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:50:59.978 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:55.592525+0000", 2024-09-16T20:50:59.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.92890456700000001 2024-09-16T20:50:59.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:50:59.978 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:50:59.979 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:50:59.979 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:55.592663+0000", 2024-09-16T20:50:59.979 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013785100000000001 2024-09-16T20:50:59.979 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.979 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:59.979 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.979 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:50:59.979 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:50:59.979 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:50:59.980 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:50:59.979+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:50:59.980 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:51:00.081 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:00.080+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:00.101 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:00.101 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:51:00.101 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:51:00.101 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:00.110 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:51:00.194 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:00.193+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:00.259 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:00.259 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:51:00.259 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:51:00.259 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:51:00.259 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:00.268 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:51:00.309 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:00.308+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:00.383 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:00.382+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:00.417 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:00.417 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:51:00.417 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:51:00.417 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:51:00.417 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.417 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5439 3.a 3:5789f5c3:::rbd_data.110ed9ca540.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 1181696~60416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:00.417 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:51:00.417 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 157.55057726499999, 2024-09-16T20:51:00.417 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10102045599999999, 2024-09-16T20:51:00.417 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:00.417 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:00.418 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:00.418 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:00.418 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:00.418 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:00.418 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5439 2024-09-16T20:51:00.418 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.418 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:00.418 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.418 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:00.418 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:51:00.418 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:00.418 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.418 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.418 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:00.419 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:51:00.419 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:00.419 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.419 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.419 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:00.419 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864711+0000", 2024-09-16T20:51:00.419 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7899999999999997e-07 2024-09-16T20:51:00.419 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.419 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.419 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:00.419 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864760+0000", 2024-09-16T20:51:00.419 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8803e-05 2024-09-16T20:51:00.419 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.419 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.420 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:00.420 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864769+0000", 2024-09-16T20:51:00.420 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0999999999999993e-06 2024-09-16T20:51:00.420 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.420 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.420 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:00.420 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864774+0000", 2024-09-16T20:51:00.420 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8919999999999999e-06 2024-09-16T20:51:00.420 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.420 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.420 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:00.420 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864815+0000", 2024-09-16T20:51:00.420 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1307000000000003e-05 2024-09-16T20:51:00.421 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.421 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.421 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:00.421 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864893+0000", 2024-09-16T20:51:00.421 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7581000000000001e-05 2024-09-16T20:51:00.421 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.421 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.421 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 1", 2024-09-16T20:51:00.421 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864996+0000", 2024-09-16T20:51:00.421 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102725 2024-09-16T20:51:00.421 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.421 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.422 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:00.422 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965442+0000", 2024-09-16T20:51:00.422 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100446012 2024-09-16T20:51:00.422 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.422 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.422 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:00.422 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965618+0000", 2024-09-16T20:51:00.422 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00017653100000000001 2024-09-16T20:51:00.422 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.422 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.422 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:00.422 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965682+0000", 2024-09-16T20:51:00.422 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3905999999999994e-05 2024-09-16T20:51:00.422 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.423 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.423 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:00.423 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965731+0000", 2024-09-16T20:51:00.423 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8819999999999997e-05 2024-09-16T20:51:00.423 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:00.423 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:00.423 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:00.423 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.423 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.423 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4600.0:5439 3.a e37/23 ondisk, result = 0)", 2024-09-16T20:51:00.423 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:51:00.423 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 157.549581117, 2024-09-16T20:51:00.423 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099742634999999996, 2024-09-16T20:51:00.424 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:00.424 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:00.424 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:00.424 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:00.424 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.424 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:00.424 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:51:00.424 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:00.424 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.424 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.424 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:00.424 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:51:00.424 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:00.424 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.424 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.425 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:00.425 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:51:00.425 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0900000000000001e-07 2024-09-16T20:51:00.425 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.425 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.425 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:00.425 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865711+0000", 2024-09-16T20:51:00.425 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3739999999999998e-06 2024-09-16T20:51:00.425 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.425 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.425 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:00.425 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965343+0000", 2024-09-16T20:51:00.425 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099632526999999999 2024-09-16T20:51:00.425 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.425 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.426 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:00.426 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965357+0000", 2024-09-16T20:51:00.426 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3713999999999999e-05 2024-09-16T20:51:00.426 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.426 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.426 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:00.426 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965426+0000", 2024-09-16T20:51:00.426 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8745999999999995e-05 2024-09-16T20:51:00.426 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.426 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.426 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:00.426 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965434+0000", 2024-09-16T20:51:00.426 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6910000000000003e-06 2024-09-16T20:51:00.426 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.426 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.427 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:00.427 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965449+0000", 2024-09-16T20:51:00.427 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4874e-05 2024-09-16T20:51:00.427 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:00.427 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:00.427 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:00.427 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.427 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.427 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4882.0:2508 3.12 3:4992771f:::rbd_object_map.110e7c8851c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T20:51:00.427 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:51:00.427 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 133.933777569, 2024-09-16T20:51:00.427 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.125007011, 2024-09-16T20:51:00.427 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:00.427 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:00.428 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:00.428 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:00.428 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4882", 2024-09-16T20:51:00.428 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2436923223", 2024-09-16T20:51:00.428 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2508 2024-09-16T20:51:00.428 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.428 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:00.428 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.428 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:00.428 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:51:00.428 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:00.428 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.428 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.429 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:00.429 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:51:00.429 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:00.429 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.429 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.429 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:00.429 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481511+0000", 2024-09-16T20:51:00.429 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0699999999999996e-07 2024-09-16T20:51:00.429 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.429 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.429 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:00.429 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481515+0000", 2024-09-16T20:51:00.429 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0609999999999997e-06 2024-09-16T20:51:00.429 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.429 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.430 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:00.430 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481523+0000", 2024-09-16T20:51:00.430 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1310000000000006e-06 2024-09-16T20:51:00.430 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.430 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.430 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:00.430 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481528+0000", 2024-09-16T20:51:00.430 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4900000000000002e-06 2024-09-16T20:51:00.430 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.430 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.430 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:00.430 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481565+0000", 2024-09-16T20:51:00.430 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7066000000000001e-05 2024-09-16T20:51:00.430 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.430 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.431 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:00.431 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481628+0000", 2024-09-16T20:51:00.431 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3118000000000001e-05 2024-09-16T20:51:00.431 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.431 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.431 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 2", 2024-09-16T20:51:00.431 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481822+0000", 2024-09-16T20:51:00.431 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019446200000000001 2024-09-16T20:51:00.431 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.431 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.431 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:00.431 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605988+0000", 2024-09-16T20:51:00.431 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.124165888 2024-09-16T20:51:00.431 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.431 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.432 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:00.432 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606427+0000", 2024-09-16T20:51:00.432 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00043854899999999999 2024-09-16T20:51:00.432 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.432 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.432 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:00.432 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606476+0000", 2024-09-16T20:51:00.432 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8971999999999998e-05 2024-09-16T20:51:00.432 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.432 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.432 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:00.432 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606517+0000", 2024-09-16T20:51:00.432 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1467000000000001e-05 2024-09-16T20:51:00.433 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:00.433 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:00.433 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:00.433 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.433 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.433 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4882.0:2508 3.12 e57/55 ondisk, result = 0)", 2024-09-16T20:51:00.433 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:51:00.433 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 133.93278906899999, 2024-09-16T20:51:00.433 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12349711400000001, 2024-09-16T20:51:00.433 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:00.433 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:00.433 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:00.433 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:00.433 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.433 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:00.434 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:51:00.434 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:00.434 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.434 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.434 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:00.434 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:51:00.434 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:00.434 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.434 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.434 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:00.434 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:51:00.434 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.06e-07 2024-09-16T20:51:00.434 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.434 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.434 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:00.435 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482502+0000", 2024-09-16T20:51:00.435 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.01e-06 2024-09-16T20:51:00.435 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.435 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.435 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:00.435 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605772+0000", 2024-09-16T20:51:00.435 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.123269687 2024-09-16T20:51:00.435 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.435 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.435 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:00.435 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605791+0000", 2024-09-16T20:51:00.435 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8912000000000001e-05 2024-09-16T20:51:00.435 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.435 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.435 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:00.435 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605974+0000", 2024-09-16T20:51:00.436 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000182836 2024-09-16T20:51:00.436 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.436 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.436 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:00.436 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605982+0000", 2024-09-16T20:51:00.436 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1699999999999997e-06 2024-09-16T20:51:00.436 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.436 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.436 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:00.436 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605996+0000", 2024-09-16T20:51:00.436 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3893000000000001e-05 2024-09-16T20:51:00.436 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:00.436 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:00.436 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:00.436 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.437 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.437 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4882.0:2512 3.f 3:f4d895b7:::rbd_data.110e7c8851c.0000000000000000:head [read 16384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T20:51:00.437 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:51:00.437 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 133.93235203899999, 2024-09-16T20:51:00.437 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12318483, 2024-09-16T20:51:00.437 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:00.437 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:00.437 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:00.437 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:00.437 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4882", 2024-09-16T20:51:00.437 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2436923223", 2024-09-16T20:51:00.437 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2512 2024-09-16T20:51:00.437 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.437 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:00.437 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.438 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:00.438 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:51:00.438 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:00.438 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.438 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.438 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:00.438 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:51:00.438 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:00.438 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.438 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.438 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:00.438 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:51:00.438 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3499999999999995e-07 2024-09-16T20:51:00.438 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.438 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.439 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:00.439 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482943+0000", 2024-09-16T20:51:00.439 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3450000000000002e-06 2024-09-16T20:51:00.439 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.439 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.439 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:00.439 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605772+0000", 2024-09-16T20:51:00.439 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.122829796 2024-09-16T20:51:00.439 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.439 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.439 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:00.439 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605792+0000", 2024-09-16T20:51:00.439 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9916999999999999e-05 2024-09-16T20:51:00.439 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.439 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.440 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:00.440 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605917+0000", 2024-09-16T20:51:00.440 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00012500799999999999 2024-09-16T20:51:00.440 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.440 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.440 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:00.440 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606004+0000", 2024-09-16T20:51:00.440 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6242999999999994e-05 2024-09-16T20:51:00.440 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.440 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.440 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:00.440 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606120+0000", 2024-09-16T20:51:00.440 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000116786 2024-09-16T20:51:00.440 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:00.440 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:00.440 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:00.441 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.441 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.441 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:11702 3.15 3:acf33c8b:::rbd_data.110e747f0779.000000000000001a:head [set-alloc-hint object_size 262144 write_size 262144,write 143360~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:51:00.441 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:51:00.441 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 116.277359617, 2024-09-16T20:51:00.441 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.084801043000000006, 2024-09-16T20:51:00.441 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:00.441 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:00.441 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:00.441 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:00.441 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:51:00.441 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:51:00.441 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 11702 2024-09-16T20:51:00.441 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.441 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:00.441 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.442 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:00.442 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:51:00.442 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:00.442 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.442 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.442 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:00.442 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:51:00.442 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:00.442 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.442 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.442 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:00.442 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137929+0000", 2024-09-16T20:51:00.442 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8400000000000002e-07 2024-09-16T20:51:00.442 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.442 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.443 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:00.443 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137937+0000", 2024-09-16T20:51:00.443 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0260000000000007e-06 2024-09-16T20:51:00.443 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.443 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.443 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:00.443 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137952+0000", 2024-09-16T20:51:00.443 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4659e-05 2024-09-16T20:51:00.443 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.443 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.443 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:00.443 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137960+0000", 2024-09-16T20:51:00.443 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0169999999999996e-06 2024-09-16T20:51:00.443 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.443 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.444 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:00.444 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138010+0000", 2024-09-16T20:51:00.444 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0021000000000001e-05 2024-09-16T20:51:00.444 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.444 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.444 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:00.444 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138114+0000", 2024-09-16T20:51:00.444 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000103797 2024-09-16T20:51:00.444 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.444 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.444 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:51:00.444 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138233+0000", 2024-09-16T20:51:00.444 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000118971 2024-09-16T20:51:00.444 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.444 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.445 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:00.445 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138953+0000", 2024-09-16T20:51:00.445 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00072026000000000002 2024-09-16T20:51:00.445 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.445 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.445 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:00.445 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222609+0000", 2024-09-16T20:51:00.445 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.083656650999999999 2024-09-16T20:51:00.445 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.445 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.445 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:00.445 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222684+0000", 2024-09-16T20:51:00.445 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4369999999999994e-05 2024-09-16T20:51:00.445 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.446 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.446 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:00.446 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222729+0000", 2024-09-16T20:51:00.446 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5287000000000002e-05 2024-09-16T20:51:00.446 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:00.446 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:00.446 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:00.446 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.446 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.446 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:11704 3.9 3:91220b75:::rbd_data.110e747f0779.000000000000001c:head [set-alloc-hint object_size 262144 write_size 262144,write 139264~6656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:51:00.446 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:51:00.446 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 116.277121616, 2024-09-16T20:51:00.446 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.084531133999999994, 2024-09-16T20:51:00.447 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:00.447 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:00.447 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:00.447 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:00.447 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:51:00.447 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:51:00.447 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 11704 2024-09-16T20:51:00.447 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.447 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:00.447 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.447 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:00.447 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:51:00.447 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:00.448 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.448 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.448 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:00.448 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:51:00.448 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:00.448 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.448 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.448 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:00.448 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138167+0000", 2024-09-16T20:51:00.448 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.33e-07 2024-09-16T20:51:00.448 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.448 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.449 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:00.449 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138179+0000", 2024-09-16T20:51:00.449 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2055999999999999e-05 2024-09-16T20:51:00.449 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.449 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.449 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:00.449 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138193+0000", 2024-09-16T20:51:00.449 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3961000000000001e-05 2024-09-16T20:51:00.449 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.449 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.449 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:00.449 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138200+0000", 2024-09-16T20:51:00.449 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4429999999999997e-06 2024-09-16T20:51:00.449 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.449 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.450 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:00.450 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138258+0000", 2024-09-16T20:51:00.450 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7225999999999997e-05 2024-09-16T20:51:00.450 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.450 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.450 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:00.450 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138364+0000", 2024-09-16T20:51:00.450 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000106013 2024-09-16T20:51:00.450 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.450 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.450 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:51:00.450 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138494+0000", 2024-09-16T20:51:00.450 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013059599999999999 2024-09-16T20:51:00.450 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.451 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.451 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:00.451 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.139220+0000", 2024-09-16T20:51:00.451 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00072539800000000004 2024-09-16T20:51:00.451 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.451 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.451 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:00.451 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222608+0000", 2024-09-16T20:51:00.451 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.083388620999999996 2024-09-16T20:51:00.451 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.451 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.451 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:00.451 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222651+0000", 2024-09-16T20:51:00.451 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2228000000000003e-05 2024-09-16T20:51:00.452 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.452 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.452 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:00.452 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222697+0000", 2024-09-16T20:51:00.452 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6659000000000002e-05 2024-09-16T20:51:00.452 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:00.452 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:00.452 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:00.452 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.452 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.452 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5138.0:9358 3.c 3.c25699cc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T20:51:00.452 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:51:00.452 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 93.564974311, 2024-09-16T20:51:00.452 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0018657580000001, 2024-09-16T20:51:00.452 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:00.453 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:00.453 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:51:00.453 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:00.453 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5138", 2024-09-16T20:51:00.453 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:51:00.453 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 9358 2024-09-16T20:51:00.453 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.453 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:00.453 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.453 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:00.453 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:51:00.453 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:00.453 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.453 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.453 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:00.454 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:51:00.454 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:00.454 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.454 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.454 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:00.454 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850314+0000", 2024-09-16T20:51:00.454 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4099999999999999e-07 2024-09-16T20:51:00.454 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.454 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.454 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:00.454 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850316+0000", 2024-09-16T20:51:00.454 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2469999999999998e-06 2024-09-16T20:51:00.454 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.454 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.455 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:00.455 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850322+0000", 2024-09-16T20:51:00.455 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3470000000000004e-06 2024-09-16T20:51:00.455 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.455 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.455 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:00.455 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850327+0000", 2024-09-16T20:51:00.455 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9450000000000001e-06 2024-09-16T20:51:00.455 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.455 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.455 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:00.455 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850376+0000", 2024-09-16T20:51:00.455 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8257000000000002e-05 2024-09-16T20:51:00.456 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.456 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.456 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:00.456 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850385+0000", 2024-09-16T20:51:00.456 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9800000000000004e-06 2024-09-16T20:51:00.456 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.456 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.456 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:00.456 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852167+0000", 2024-09-16T20:51:00.456 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0017825659999999 2024-09-16T20:51:00.456 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.456 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.456 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:00.456 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852179+0000", 2024-09-16T20:51:00.457 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1975e-05 2024-09-16T20:51:00.457 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:00.457 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:00.457 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:00.457 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.457 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.457 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5138.0:9382 3.c 3.c25699cc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T20:51:00.457 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:51:00.457 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 93.556627294999998, 2024-09-16T20:51:00.457 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.99358404700000003, 2024-09-16T20:51:00.458 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:00.458 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:00.458 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:51:00.458 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:00.458 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5138", 2024-09-16T20:51:00.458 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:51:00.458 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 9382 2024-09-16T20:51:00.458 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.458 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:00.458 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.458 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:00.458 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:51:00.459 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:00.459 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.459 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.459 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:00.459 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:51:00.459 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:00.459 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.459 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.459 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:00.459 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858661+0000", 2024-09-16T20:51:00.459 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0200000000000002e-07 2024-09-16T20:51:00.459 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.459 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.460 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:00.460 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858664+0000", 2024-09-16T20:51:00.460 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.038e-06 2024-09-16T20:51:00.460 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.460 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.460 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:00.460 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858671+0000", 2024-09-16T20:51:00.460 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0110000000000003e-06 2024-09-16T20:51:00.460 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.460 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.460 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:00.460 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858677+0000", 2024-09-16T20:51:00.460 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0959999999999997e-06 2024-09-16T20:51:00.460 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.461 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.461 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:00.461 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858732+0000", 2024-09-16T20:51:00.461 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4543000000000003e-05 2024-09-16T20:51:00.461 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.461 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.461 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:00.461 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858739+0000", 2024-09-16T20:51:00.461 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5429999999999996e-06 2024-09-16T20:51:00.461 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.461 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.461 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:00.461 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852236+0000", 2024-09-16T20:51:00.462 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.993496453 2024-09-16T20:51:00.462 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.462 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.462 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:00.462 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852244+0000", 2024-09-16T20:51:00.462 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8610000000000004e-06 2024-09-16T20:51:00.462 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:00.462 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:00.462 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:00.462 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.462 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.462 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5763.0:4895 3.a 3:5384606c:::rbd_data.110e1b19a241.0000000000000018:head [read 0~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-16T20:51:00.462 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:45.598212+0000", 2024-09-16T20:51:00.462 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 74.817075254000002, 2024-09-16T20:51:00.463 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0375864829999999, 2024-09-16T20:51:00.463 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:00.463 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:00.463 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:00.463 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:00.463 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5763", 2024-09-16T20:51:00.463 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/623897567", 2024-09-16T20:51:00.463 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4895 2024-09-16T20:51:00.464 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.464 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:00.464 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.464 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:00.464 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598212+0000", 2024-09-16T20:51:00.464 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:00.464 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.464 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.464 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:00.464 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598212+0000", 2024-09-16T20:51:00.464 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:00.464 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.464 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.465 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:00.465 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598214+0000", 2024-09-16T20:51:00.465 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0090000000000002e-06 2024-09-16T20:51:00.465 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.465 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.465 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:00.465 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598222+0000", 2024-09-16T20:51:00.465 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6699999999999994e-06 2024-09-16T20:51:00.465 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.465 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.465 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:00.465 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598241+0000", 2024-09-16T20:51:00.465 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8896999999999999e-05 2024-09-16T20:51:00.465 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.465 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.465 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:00.466 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598250+0000", 2024-09-16T20:51:00.466 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6649999999999992e-06 2024-09-16T20:51:00.466 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.466 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.466 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:00.466 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598325+0000", 2024-09-16T20:51:00.466 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5189000000000001e-05 2024-09-16T20:51:00.466 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.466 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.466 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op must wait for map", 2024-09-16T20:51:00.466 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598333+0000", 2024-09-16T20:51:00.466 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.3170000000000002e-06 2024-09-16T20:51:00.466 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.466 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.467 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:00.467 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601355+0000", 2024-09-16T20:51:00.467 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0030215400000000001 2024-09-16T20:51:00.467 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.467 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.467 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:00.467 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601365+0000", 2024-09-16T20:51:00.467 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0215e-05 2024-09-16T20:51:00.467 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.467 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.467 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:00.467 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635506+0000", 2024-09-16T20:51:00.467 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0341408240000001 2024-09-16T20:51:00.467 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.467 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.468 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:00.468 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635655+0000", 2024-09-16T20:51:00.468 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000149412 2024-09-16T20:51:00.468 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.468 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.468 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:00.468 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635799+0000", 2024-09-16T20:51:00.468 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000143745 2024-09-16T20:51:00.468 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:00.468 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:00.468 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:00.468 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.468 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.468 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5763.0:4900 3.a 3:5384606c:::rbd_data.110e1b19a241.0000000000000018:head [read 8192~24576] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-16T20:51:00.469 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:51:00.469 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 74.817016777999996, 2024-09-16T20:51:00.469 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0379340800000001, 2024-09-16T20:51:00.469 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:00.469 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:00.469 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:00.469 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:00.469 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5763", 2024-09-16T20:51:00.469 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/623897567", 2024-09-16T20:51:00.469 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4900 2024-09-16T20:51:00.469 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.469 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:00.469 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.469 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:00.470 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:51:00.470 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:00.470 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.470 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.470 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:00.470 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:51:00.470 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:00.470 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.470 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.470 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:00.470 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:51:00.470 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4300000000000003e-07 2024-09-16T20:51:00.470 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.470 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.470 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:00.470 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598274+0000", 2024-09-16T20:51:00.471 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.976e-06 2024-09-16T20:51:00.471 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.471 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.471 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:00.471 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598282+0000", 2024-09-16T20:51:00.471 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0029999999999994e-06 2024-09-16T20:51:00.471 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.471 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.471 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:00.471 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598289+0000", 2024-09-16T20:51:00.471 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2389999999999999e-06 2024-09-16T20:51:00.471 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.471 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.471 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:00.472 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598430+0000", 2024-09-16T20:51:00.472 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014119600000000001 2024-09-16T20:51:00.472 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.472 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.472 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting_for_map not empty", 2024-09-16T20:51:00.472 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598435+0000", 2024-09-16T20:51:00.472 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3489999999999999e-06 2024-09-16T20:51:00.472 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.472 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.472 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:00.472 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601429+0000", 2024-09-16T20:51:00.472 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0029942720000000001 2024-09-16T20:51:00.472 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.472 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.472 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:00.473 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601438+0000", 2024-09-16T20:51:00.473 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4759999999999998e-06 2024-09-16T20:51:00.473 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.473 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.473 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:00.473 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635987+0000", 2024-09-16T20:51:00.473 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0345495010000001 2024-09-16T20:51:00.473 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.473 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.473 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:00.473 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.636056+0000", 2024-09-16T20:51:00.473 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8610999999999998e-05 2024-09-16T20:51:00.473 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.473 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.474 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:00.474 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.636205+0000", 2024-09-16T20:51:00.474 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014891399999999999 2024-09-16T20:51:00.474 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:00.474 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:00.474 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:00.474 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.474 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.474 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop(client.6231.0:2182 3.17 e136/81 3:edb5e5cd:::rbd_data.110e61479aee.0000000000000014:head v 136'791, mlcod=135'789)", 2024-09-16T20:51:00.474 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:22.735499+0000", 2024-09-16T20:51:00.474 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 37.679788836999997, 2024-09-16T20:51:00.474 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12369608, 2024-09-16T20:51:00.474 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:00.474 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:00.474 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:00.475 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:00.475 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.475 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:00.475 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735499+0000", 2024-09-16T20:51:00.475 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:00.475 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.475 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.475 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:00.475 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735499+0000", 2024-09-16T20:51:00.475 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:00.475 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.475 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.475 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:00.475 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735499+0000", 2024-09-16T20:51:00.475 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8099999999999999e-07 2024-09-16T20:51:00.475 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.476 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.476 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:00.476 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735506+0000", 2024-09-16T20:51:00.476 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3579999999999997e-06 2024-09-16T20:51:00.476 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.476 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.476 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:00.476 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735517+0000", 2024-09-16T20:51:00.476 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0869999999999999e-05 2024-09-16T20:51:00.476 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.476 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.476 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:00.476 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735533+0000", 2024-09-16T20:51:00.477 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5858e-05 2024-09-16T20:51:00.477 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.477 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.477 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:00.477 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735584+0000", 2024-09-16T20:51:00.477 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0470999999999998e-05 2024-09-16T20:51:00.477 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.477 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.477 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:00.477 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735607+0000", 2024-09-16T20:51:00.477 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2875999999999999e-05 2024-09-16T20:51:00.477 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.477 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.477 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:00.478 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.859144+0000", 2024-09-16T20:51:00.478 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.123537701 2024-09-16T20:51:00.478 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.478 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.478 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:00.478 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.859195+0000", 2024-09-16T20:51:00.478 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0664999999999997e-05 2024-09-16T20:51:00.478 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:00.478 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:00.478 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:00.478 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.478 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.478 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5204 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 2523136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:51:00.479 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.883660+0000", 2024-09-16T20:51:00.479 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 36.531627307000001, 2024-09-16T20:51:00.479 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052517638999999998, 2024-09-16T20:51:00.479 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:00.479 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:00.479 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:00.479 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:00.479 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:00.479 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:00.479 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5204 2024-09-16T20:51:00.479 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.479 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:00.479 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.479 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:00.480 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883660+0000", 2024-09-16T20:51:00.480 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:00.480 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.480 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.480 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:00.480 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883660+0000", 2024-09-16T20:51:00.480 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:00.480 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.480 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.480 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:00.480 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883661+0000", 2024-09-16T20:51:00.480 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9999999999999998e-07 2024-09-16T20:51:00.480 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.480 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.481 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:00.481 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883663+0000", 2024-09-16T20:51:00.481 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.4480000000000001e-06 2024-09-16T20:51:00.481 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.481 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.481 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:00.481 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883669+0000", 2024-09-16T20:51:00.481 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.071e-06 2024-09-16T20:51:00.481 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.481 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.481 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:00.481 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883675+0000", 2024-09-16T20:51:00.481 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.366e-06 2024-09-16T20:51:00.481 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.481 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.481 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:00.482 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.935970+0000", 2024-09-16T20:51:00.482 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052295346999999999 2024-09-16T20:51:00.482 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.482 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.482 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:00.482 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936064+0000", 2024-09-16T20:51:00.482 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4438000000000006e-05 2024-09-16T20:51:00.482 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.482 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.482 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:00.482 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936178+0000", 2024-09-16T20:51:00.482 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000113569 2024-09-16T20:51:00.482 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:00.482 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:00.482 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:00.483 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.483 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.483 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5205 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 2543616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:51:00.483 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.883745+0000", 2024-09-16T20:51:00.483 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 36.531542549000001, 2024-09-16T20:51:00.483 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052692112999999999, 2024-09-16T20:51:00.483 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:00.483 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:00.483 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:00.483 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:00.483 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:00.483 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:00.483 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5205 2024-09-16T20:51:00.483 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.483 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:00.483 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.483 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:00.484 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883745+0000", 2024-09-16T20:51:00.484 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:00.484 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.484 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.484 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:00.484 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883745+0000", 2024-09-16T20:51:00.484 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:00.484 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.484 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.484 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:00.484 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883746+0000", 2024-09-16T20:51:00.484 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2699999999999999e-07 2024-09-16T20:51:00.484 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.484 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.485 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:00.485 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883750+0000", 2024-09-16T20:51:00.485 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7050000000000001e-06 2024-09-16T20:51:00.485 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.485 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.485 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:00.485 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883769+0000", 2024-09-16T20:51:00.485 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9165e-05 2024-09-16T20:51:00.485 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.485 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.485 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:00.485 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883778+0000", 2024-09-16T20:51:00.485 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.6690000000000001e-06 2024-09-16T20:51:00.485 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.485 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.486 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:00.486 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936228+0000", 2024-09-16T20:51:00.486 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052449587999999998 2024-09-16T20:51:00.486 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.486 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.486 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:00.486 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936333+0000", 2024-09-16T20:51:00.486 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010511800000000001 2024-09-16T20:51:00.486 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.486 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.486 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:00.486 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936437+0000", 2024-09-16T20:51:00.486 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000104141 2024-09-16T20:51:00.486 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:00.486 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:00.486 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:00.487 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.487 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.487 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5206 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 1167360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:51:00.487 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.883802+0000", 2024-09-16T20:51:00.487 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 36.531486055000002, 2024-09-16T20:51:00.487 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052800780999999998, 2024-09-16T20:51:00.487 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:00.487 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:00.487 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:00.487 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:00.487 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:00.487 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:00.487 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5206 2024-09-16T20:51:00.487 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.487 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:00.488 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.488 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:00.488 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883802+0000", 2024-09-16T20:51:00.488 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:00.488 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.488 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.488 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:00.488 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883802+0000", 2024-09-16T20:51:00.488 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:00.488 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.488 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.488 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:00.488 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883933+0000", 2024-09-16T20:51:00.488 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013182999999999999 2024-09-16T20:51:00.488 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.489 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.489 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:00.489 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883937+0000", 2024-09-16T20:51:00.489 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.771e-06 2024-09-16T20:51:00.489 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.489 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.489 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:00.489 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883953+0000", 2024-09-16T20:51:00.489 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5409000000000001e-05 2024-09-16T20:51:00.489 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.489 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.489 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:00.489 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883959+0000", 2024-09-16T20:51:00.489 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0700000000000003e-06 2024-09-16T20:51:00.489 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.490 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.490 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:00.490 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936476+0000", 2024-09-16T20:51:00.490 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052517434000000002 2024-09-16T20:51:00.490 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.490 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.490 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:00.490 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936529+0000", 2024-09-16T20:51:00.490 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2969000000000001e-05 2024-09-16T20:51:00.490 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.490 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.490 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:00.490 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936602+0000", 2024-09-16T20:51:00.490 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3298000000000002e-05 2024-09-16T20:51:00.490 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:00.490 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:00.491 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:00.491 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.491 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.491 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5207 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 1904640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:51:00.491 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.883980+0000", 2024-09-16T20:51:00.491 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 36.531307845999997, 2024-09-16T20:51:00.491 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052759922000000001, 2024-09-16T20:51:00.491 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:00.491 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:00.491 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:00.491 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:00.491 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:00.491 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:00.491 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5207 2024-09-16T20:51:00.491 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.492 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:00.492 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.492 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:00.492 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883980+0000", 2024-09-16T20:51:00.492 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:00.492 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.492 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.492 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:00.492 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883980+0000", 2024-09-16T20:51:00.492 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:00.492 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.492 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.492 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:00.492 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884126+0000", 2024-09-16T20:51:00.492 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014577399999999999 2024-09-16T20:51:00.492 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.493 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.493 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:00.493 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884130+0000", 2024-09-16T20:51:00.493 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9729999999999999e-06 2024-09-16T20:51:00.493 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.493 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.493 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:00.493 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884139+0000", 2024-09-16T20:51:00.493 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8660000000000001e-06 2024-09-16T20:51:00.493 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.493 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.493 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:00.493 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884144+0000", 2024-09-16T20:51:00.493 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3669999999999999e-06 2024-09-16T20:51:00.494 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.494 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.494 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:00.494 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936635+0000", 2024-09-16T20:51:00.494 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052491705 2024-09-16T20:51:00.494 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.494 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.494 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:00.494 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936679+0000", 2024-09-16T20:51:00.494 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3105999999999997e-05 2024-09-16T20:51:00.494 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.494 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.494 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:00.495 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936740+0000", 2024-09-16T20:51:00.495 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1130999999999995e-05 2024-09-16T20:51:00.495 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:00.495 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:00.495 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:00.495 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.495 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.495 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5208 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 131072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:51:00.495 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.884165+0000", 2024-09-16T20:51:00.495 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 36.531122549999999, 2024-09-16T20:51:00.495 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052746603000000003, 2024-09-16T20:51:00.495 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:00.495 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:00.495 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:00.496 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:00.496 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:00.496 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:00.496 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5208 2024-09-16T20:51:00.496 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.496 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:00.496 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.496 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:00.496 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884165+0000", 2024-09-16T20:51:00.496 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:00.496 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.496 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.496 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:00.496 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884165+0000", 2024-09-16T20:51:00.497 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:00.497 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.497 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.497 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:00.497 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884463+0000", 2024-09-16T20:51:00.497 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00029801699999999998 2024-09-16T20:51:00.497 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.497 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.497 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:00.497 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884467+0000", 2024-09-16T20:51:00.497 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8210000000000003e-06 2024-09-16T20:51:00.497 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.497 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.497 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:00.498 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884477+0000", 2024-09-16T20:51:00.498 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.7030000000000001e-06 2024-09-16T20:51:00.498 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.498 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.498 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:00.498 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884485+0000", 2024-09-16T20:51:00.498 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9710000000000004e-06 2024-09-16T20:51:00.498 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.498 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.498 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:00.498 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936785+0000", 2024-09-16T20:51:00.498 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052300665000000003 2024-09-16T20:51:00.498 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.498 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.499 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:00.499 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936851+0000", 2024-09-16T20:51:00.499 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6159000000000001e-05 2024-09-16T20:51:00.499 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.499 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.499 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:00.499 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936912+0000", 2024-09-16T20:51:00.499 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0266999999999997e-05 2024-09-16T20:51:00.499 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:00.499 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:00.499 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:00.499 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.499 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.500 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5209 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 1925120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:51:00.500 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.884504+0000", 2024-09-16T20:51:00.500 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 36.530783665000001, 2024-09-16T20:51:00.500 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052575445999999998, 2024-09-16T20:51:00.500 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:00.500 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:00.500 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:00.500 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:00.500 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:00.500 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:00.500 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5209 2024-09-16T20:51:00.500 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.500 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:00.500 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.501 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:00.501 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884504+0000", 2024-09-16T20:51:00.501 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:00.501 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.501 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.501 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:00.501 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884504+0000", 2024-09-16T20:51:00.501 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:00.501 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.501 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.501 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:00.501 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884741+0000", 2024-09-16T20:51:00.501 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00023679899999999999 2024-09-16T20:51:00.501 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.502 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.502 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:00.502 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884747+0000", 2024-09-16T20:51:00.502 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0009999999999998e-06 2024-09-16T20:51:00.502 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.502 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.502 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:00.502 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884764+0000", 2024-09-16T20:51:00.502 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7371999999999999e-05 2024-09-16T20:51:00.502 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.502 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.502 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:00.502 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884772+0000", 2024-09-16T20:51:00.503 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1349999999999992e-06 2024-09-16T20:51:00.503 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.503 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.503 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:00.503 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936956+0000", 2024-09-16T20:51:00.503 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052183754999999998 2024-09-16T20:51:00.503 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.503 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.503 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:00.503 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937003+0000", 2024-09-16T20:51:00.503 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6925000000000002e-05 2024-09-16T20:51:00.503 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.503 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.503 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:00.503 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937079+0000", 2024-09-16T20:51:00.504 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6458999999999994e-05 2024-09-16T20:51:00.504 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:00.504 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:00.504 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:00.504 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.504 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.504 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5210 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 151552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:51:00.504 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.884799+0000", 2024-09-16T20:51:00.504 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 36.530488396000003, 2024-09-16T20:51:00.504 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052462483999999997, 2024-09-16T20:51:00.504 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:00.504 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:00.504 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:00.505 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:00.505 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:00.505 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:00.505 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5210 2024-09-16T20:51:00.505 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.505 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:00.505 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.505 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:00.505 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884799+0000", 2024-09-16T20:51:00.505 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:00.505 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.505 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.505 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:00.505 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884799+0000", 2024-09-16T20:51:00.505 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:00.506 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.506 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.506 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:00.506 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884990+0000", 2024-09-16T20:51:00.506 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019051800000000001 2024-09-16T20:51:00.506 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.506 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.506 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:00.506 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884996+0000", 2024-09-16T20:51:00.506 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0689999999999998e-06 2024-09-16T20:51:00.506 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.506 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.506 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:00.506 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.885008+0000", 2024-09-16T20:51:00.506 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2153999999999999e-05 2024-09-16T20:51:00.506 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.507 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.507 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:00.507 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.885016+0000", 2024-09-16T20:51:00.507 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2770000000000005e-06 2024-09-16T20:51:00.507 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.507 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.507 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:00.507 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937128+0000", 2024-09-16T20:51:00.507 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052111377 2024-09-16T20:51:00.507 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.507 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.507 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:00.507 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937188+0000", 2024-09-16T20:51:00.507 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9933999999999999e-05 2024-09-16T20:51:00.507 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.508 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.508 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:00.508 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937262+0000", 2024-09-16T20:51:00.508 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4154999999999997e-05 2024-09-16T20:51:00.508 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:00.508 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:00.508 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:00.508 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.508 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.508 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5211 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 1945600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:51:00.508 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.885030+0000", 2024-09-16T20:51:00.508 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 36.530257311, 2024-09-16T20:51:00.508 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052369470000000001, 2024-09-16T20:51:00.508 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:00.509 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:00.509 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:00.509 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:00.509 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:00.509 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:00.509 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5211 2024-09-16T20:51:00.509 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.509 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:00.509 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.509 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:00.509 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.885030+0000", 2024-09-16T20:51:00.509 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:00.509 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.509 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.509 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:00.510 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.885030+0000", 2024-09-16T20:51:00.510 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:00.510 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.510 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.510 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:00.510 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.885219+0000", 2024-09-16T20:51:00.510 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000188278 2024-09-16T20:51:00.510 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.510 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.510 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:00.510 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.885223+0000", 2024-09-16T20:51:00.510 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7849999999999999e-06 2024-09-16T20:51:00.510 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.510 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.510 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:00.511 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.885233+0000", 2024-09-16T20:51:00.511 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.5060000000000001e-06 2024-09-16T20:51:00.511 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.511 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.511 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:00.511 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.885240+0000", 2024-09-16T20:51:00.511 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9630000000000001e-06 2024-09-16T20:51:00.511 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.511 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.511 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:00.511 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937304+0000", 2024-09-16T20:51:00.511 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052064619999999999 2024-09-16T20:51:00.511 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.511 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.511 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:00.511 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937347+0000", 2024-09-16T20:51:00.512 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2407000000000002e-05 2024-09-16T20:51:00.512 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.512 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.512 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:00.512 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937400+0000", 2024-09-16T20:51:00.512 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2911000000000002e-05 2024-09-16T20:51:00.512 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:00.512 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:00.512 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:00.512 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:00.512 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:00.512 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:00.514 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:51:00.514 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:00.513+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:00.614 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:00.613+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:00.659 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T20:51:00.668 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:00.668 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:51:00.668 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:51:00.668 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:00.677 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:51:00.718 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:00.717+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:00.816 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:00.815+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:00.830 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:00.830 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:51:00.830 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:51:00.830 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:51:00.830 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:00.839 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:51:00.923 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:00.922+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:00.988 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:00.988 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:51:00.988 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:51:00.988 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:51:00.988 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.989 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:51:00.989 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:51:00.989 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 187.73240058799999, 2024-09-16T20:51:00.989 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67502346999999996, 2024-09-16T20:51:00.989 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:00.989 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:00.989 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:00.989 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:00.989 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:51:00.989 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:51:00.989 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 1 2024-09-16T20:51:00.989 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.989 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:00.990 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.990 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:00.990 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:51:00.990 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:00.990 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.990 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.990 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:00.990 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:51:00.990 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:00.990 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.990 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.990 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:00.990 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254128+0000", 2024-09-16T20:51:00.991 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.463e-06 2024-09-16T20:51:00.991 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.991 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.991 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:00.991 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254135+0000", 2024-09-16T20:51:00.991 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1579999999999999e-06 2024-09-16T20:51:00.991 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.991 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.991 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:00.991 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254146+0000", 2024-09-16T20:51:00.991 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1341e-05 2024-09-16T20:51:00.991 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.991 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.991 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:00.991 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254155+0000", 2024-09-16T20:51:00.992 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.1549999999999996e-06 2024-09-16T20:51:00.992 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.992 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.992 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:00.992 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254221+0000", 2024-09-16T20:51:00.992 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5470999999999997e-05 2024-09-16T20:51:00.992 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.992 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.992 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:00.992 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254231+0000", 2024-09-16T20:51:00.992 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0592e-05 2024-09-16T20:51:00.992 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.992 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.992 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:00.993 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926711+0000", 2024-09-16T20:51:00.993 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67247946599999997 2024-09-16T20:51:00.993 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.993 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.993 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:00.993 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926910+0000", 2024-09-16T20:51:00.993 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019934 2024-09-16T20:51:00.993 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.993 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.993 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:51:00.993 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.927159+0000", 2024-09-16T20:51:00.993 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00024876599999999998 2024-09-16T20:51:00.993 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.993 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.994 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:00.994 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.928797+0000", 2024-09-16T20:51:00.994 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001638308 2024-09-16T20:51:00.994 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.994 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.994 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:00.994 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929026+0000", 2024-09-16T20:51:00.994 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00022876099999999999 2024-09-16T20:51:00.994 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.994 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.994 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:00.994 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929102+0000", 2024-09-16T20:51:00.994 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6184000000000001e-05 2024-09-16T20:51:00.994 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.995 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.995 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:00.995 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929150+0000", 2024-09-16T20:51:00.995 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7465e-05 2024-09-16T20:51:00.995 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:00.995 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:00.995 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:00.995 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.995 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.995 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3574 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:00.995 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:51:00.995 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 131.505237056, 2024-09-16T20:51:00.995 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053579441999999998, 2024-09-16T20:51:00.995 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:00.996 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:00.996 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:00.996 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:00.996 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:00.996 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:00.996 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3574 2024-09-16T20:51:00.996 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.996 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:00.996 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.996 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:00.996 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:51:00.996 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:00.996 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.996 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.997 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:00.997 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:51:00.997 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:00.997 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.997 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.997 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:00.997 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:51:00.997 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2e-07 2024-09-16T20:51:00.997 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.997 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.997 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:00.997 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481292+0000", 2024-09-16T20:51:00.997 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2630000000000002e-06 2024-09-16T20:51:00.998 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.998 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.998 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:00.998 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481297+0000", 2024-09-16T20:51:00.998 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.296e-06 2024-09-16T20:51:00.998 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.998 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.998 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:00.998 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481300+0000", 2024-09-16T20:51:00.998 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7019999999999999e-06 2024-09-16T20:51:00.998 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.998 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.998 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:00.999 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525533+0000", 2024-09-16T20:51:00.999 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.044232318 2024-09-16T20:51:00.999 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.999 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.999 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:00.999 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525619+0000", 2024-09-16T20:51:00.999 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5892999999999999e-05 2024-09-16T20:51:00.999 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:00.999 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:00.999 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:00.999 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525987+0000", 2024-09-16T20:51:00.999 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00036836099999999999 2024-09-16T20:51:01.000 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.000 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.000 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:01.000 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531779+0000", 2024-09-16T20:51:01.000 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0057918240000000001 2024-09-16T20:51:01.000 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.000 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.000 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:01.000 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534747+0000", 2024-09-16T20:51:01.000 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0029681600000000001 2024-09-16T20:51:01.000 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.000 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.000 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:01.000 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534832+0000", 2024-09-16T20:51:01.001 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5180999999999996e-05 2024-09-16T20:51:01.001 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.001 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.001 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:01.001 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534869+0000", 2024-09-16T20:51:01.001 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7024000000000003e-05 2024-09-16T20:51:01.001 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:01.001 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:01.001 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:01.001 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.001 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.001 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3575 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:01.001 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:51:01.002 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 131.50522231599999, 2024-09-16T20:51:01.002 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053801463000000001, 2024-09-16T20:51:01.002 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:01.002 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:01.002 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:01.002 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:01.002 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:01.002 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:01.002 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3575 2024-09-16T20:51:01.002 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.002 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:01.002 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.002 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:01.002 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:51:01.003 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:01.003 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.003 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.003 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:01.003 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:51:01.003 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:01.003 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.003 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.003 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:01.003 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481305+0000", 2024-09-16T20:51:01.003 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5400000000000002e-07 2024-09-16T20:51:01.003 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.003 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.004 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:01.004 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481307+0000", 2024-09-16T20:51:01.004 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7800000000000001e-06 2024-09-16T20:51:01.004 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.004 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.004 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:01.004 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481312+0000", 2024-09-16T20:51:01.004 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6349999999999997e-06 2024-09-16T20:51:01.004 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.004 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.004 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:01.004 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481315+0000", 2024-09-16T20:51:01.005 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7829999999999999e-06 2024-09-16T20:51:01.005 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.005 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.005 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:01.005 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.526876+0000", 2024-09-16T20:51:01.005 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.045561070000000002 2024-09-16T20:51:01.005 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.005 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.005 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:01.005 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.526954+0000", 2024-09-16T20:51:01.005 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8108000000000005e-05 2024-09-16T20:51:01.005 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.005 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.005 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:01.006 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527282+0000", 2024-09-16T20:51:01.006 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000327483 2024-09-16T20:51:01.006 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.006 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.006 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:01.006 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531784+0000", 2024-09-16T20:51:01.006 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0045020169999999997 2024-09-16T20:51:01.006 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.006 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.006 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:01.006 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534921+0000", 2024-09-16T20:51:01.006 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.003137791 2024-09-16T20:51:01.006 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.006 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.007 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:01.007 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534976+0000", 2024-09-16T20:51:01.007 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4953000000000001e-05 2024-09-16T20:51:01.007 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.007 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.007 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:01.007 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535106+0000", 2024-09-16T20:51:01.007 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000129589 2024-09-16T20:51:01.007 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:01.007 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:01.007 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:01.007 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.007 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.007 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3576 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:01.008 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:51:01.008 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 131.505208404, 2024-09-16T20:51:01.008 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053955436000000002, 2024-09-16T20:51:01.008 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:01.008 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:01.008 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:01.008 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:01.008 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:01.008 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:01.008 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3576 2024-09-16T20:51:01.008 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.008 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:01.008 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.009 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:01.009 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:51:01.009 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:01.009 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.009 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.009 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:01.009 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:51:01.009 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:01.009 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.009 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.009 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:01.009 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481319+0000", 2024-09-16T20:51:01.009 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.53e-07 2024-09-16T20:51:01.009 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.009 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.010 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:01.010 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481320+0000", 2024-09-16T20:51:01.010 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5230000000000001e-06 2024-09-16T20:51:01.010 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.010 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.010 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:01.010 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481330+0000", 2024-09-16T20:51:01.010 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0190000000000001e-05 2024-09-16T20:51:01.010 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.010 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.010 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:01.010 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481333+0000", 2024-09-16T20:51:01.010 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.151e-06 2024-09-16T20:51:01.010 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.010 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.010 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:01.011 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527689+0000", 2024-09-16T20:51:01.011 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046355192000000003 2024-09-16T20:51:01.011 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.011 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.011 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:01.011 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527764+0000", 2024-09-16T20:51:01.011 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5776000000000001e-05 2024-09-16T20:51:01.011 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.011 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.011 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:01.011 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528141+0000", 2024-09-16T20:51:01.011 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00037647599999999999 2024-09-16T20:51:01.011 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.011 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.011 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:01.012 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531788+0000", 2024-09-16T20:51:01.012 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0036470679999999998 2024-09-16T20:51:01.012 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.012 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.012 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:01.012 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535174+0000", 2024-09-16T20:51:01.012 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0033862670000000001 2024-09-16T20:51:01.012 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.012 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.012 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:01.012 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535236+0000", 2024-09-16T20:51:01.012 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1797999999999993e-05 2024-09-16T20:51:01.012 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.012 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.012 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:01.013 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535274+0000", 2024-09-16T20:51:01.013 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7741999999999999e-05 2024-09-16T20:51:01.013 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:01.013 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:01.013 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:01.013 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.013 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.013 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3577 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:01.013 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:51:01.013 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 131.505186804, 2024-09-16T20:51:01.013 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054073807000000002, 2024-09-16T20:51:01.013 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:01.013 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:01.013 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:01.013 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:01.014 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:01.014 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:01.014 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3577 2024-09-16T20:51:01.014 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.014 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:01.014 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.014 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:01.014 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:51:01.014 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:01.014 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.014 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.014 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:01.014 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:51:01.014 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:01.014 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.015 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.015 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:01.015 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:51:01.015 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.96e-07 2024-09-16T20:51:01.015 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.015 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.015 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:01.015 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481342+0000", 2024-09-16T20:51:01.015 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.714e-06 2024-09-16T20:51:01.015 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.015 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.015 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:01.015 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481349+0000", 2024-09-16T20:51:01.015 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4930000000000001e-06 2024-09-16T20:51:01.016 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.016 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.016 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:01.016 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481353+0000", 2024-09-16T20:51:01.016 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4879999999999999e-06 2024-09-16T20:51:01.016 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.016 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.016 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:01.016 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528619+0000", 2024-09-16T20:51:01.016 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047266351999999998 2024-09-16T20:51:01.016 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.016 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.016 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:01.016 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528706+0000", 2024-09-16T20:51:01.017 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6796e-05 2024-09-16T20:51:01.017 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.017 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.017 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:01.017 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529148+0000", 2024-09-16T20:51:01.017 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00044182999999999999 2024-09-16T20:51:01.017 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.017 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.017 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:01.017 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531792+0000", 2024-09-16T20:51:01.017 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0026443119999999998 2024-09-16T20:51:01.017 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.017 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.017 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:01.018 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535319+0000", 2024-09-16T20:51:01.018 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0035266439999999998 2024-09-16T20:51:01.018 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.018 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.018 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:01.018 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535372+0000", 2024-09-16T20:51:01.018 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3207000000000001e-05 2024-09-16T20:51:01.018 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.018 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.018 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:01.018 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535414+0000", 2024-09-16T20:51:01.018 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1675000000000003e-05 2024-09-16T20:51:01.018 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:01.018 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:01.019 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:01.019 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.019 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.019 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3578 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:01.019 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:51:01.019 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 131.505165635, 2024-09-16T20:51:01.019 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054189586999999997, 2024-09-16T20:51:01.019 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:01.019 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:01.019 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:01.019 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:01.019 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:01.019 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:01.019 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3578 2024-09-16T20:51:01.020 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.020 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:01.020 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.020 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:01.020 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:51:01.020 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:01.020 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.020 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.020 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:01.020 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:51:01.020 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:01.020 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.020 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.020 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:01.020 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:51:01.021 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0800000000000001e-07 2024-09-16T20:51:01.021 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.021 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.021 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:01.021 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481363+0000", 2024-09-16T20:51:01.021 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0339999999999999e-06 2024-09-16T20:51:01.021 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.021 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.021 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:01.021 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481371+0000", 2024-09-16T20:51:01.021 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4440000000000002e-06 2024-09-16T20:51:01.021 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.021 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.021 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:01.021 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481374+0000", 2024-09-16T20:51:01.022 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2049999999999998e-06 2024-09-16T20:51:01.022 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.022 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.022 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:01.022 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529555+0000", 2024-09-16T20:51:01.022 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048181202999999999 2024-09-16T20:51:01.022 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.022 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.022 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:01.022 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529616+0000", 2024-09-16T20:51:01.022 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0912000000000002e-05 2024-09-16T20:51:01.022 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.022 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.022 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:01.022 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529939+0000", 2024-09-16T20:51:01.023 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00032269800000000001 2024-09-16T20:51:01.023 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.023 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.023 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:01.023 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532602+0000", 2024-09-16T20:51:01.023 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0026634940000000002 2024-09-16T20:51:01.023 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.023 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.023 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:01.023 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535465+0000", 2024-09-16T20:51:01.023 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.002862149 2024-09-16T20:51:01.023 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.023 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.023 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:01.023 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535516+0000", 2024-09-16T20:51:01.024 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1696000000000001e-05 2024-09-16T20:51:01.024 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.024 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.024 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:01.024 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535551+0000", 2024-09-16T20:51:01.024 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4443999999999998e-05 2024-09-16T20:51:01.024 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:01.024 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:01.024 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:01.024 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.024 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.024 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3579 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:01.024 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:51:01.024 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 131.505147196, 2024-09-16T20:51:01.024 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054295151, 2024-09-16T20:51:01.025 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:01.025 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:01.025 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:01.025 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:01.025 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:01.025 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:01.025 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3579 2024-09-16T20:51:01.025 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.025 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:01.025 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.025 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:01.025 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:51:01.025 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:01.025 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.025 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.025 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:01.026 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:51:01.026 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:01.026 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.026 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.026 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:01.026 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:51:01.026 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3300000000000001e-07 2024-09-16T20:51:01.026 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.026 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.026 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:01.026 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481381+0000", 2024-09-16T20:51:01.026 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.649e-06 2024-09-16T20:51:01.026 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.026 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.026 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:01.027 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481387+0000", 2024-09-16T20:51:01.027 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4519999999999996e-06 2024-09-16T20:51:01.027 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.027 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.027 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:01.027 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481390+0000", 2024-09-16T20:51:01.027 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0170000000000001e-06 2024-09-16T20:51:01.027 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.027 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.027 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:01.027 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530756+0000", 2024-09-16T20:51:01.027 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049365882999999999 2024-09-16T20:51:01.027 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.027 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.027 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:01.028 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530845+0000", 2024-09-16T20:51:01.028 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9197000000000007e-05 2024-09-16T20:51:01.028 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.028 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.028 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:01.028 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531191+0000", 2024-09-16T20:51:01.028 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00034562500000000001 2024-09-16T20:51:01.028 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.028 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.028 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:01.028 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534078+0000", 2024-09-16T20:51:01.028 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0028878179999999999 2024-09-16T20:51:01.028 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.029 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.029 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:01.029 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535589+0000", 2024-09-16T20:51:01.029 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00151085 2024-09-16T20:51:01.029 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.029 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.029 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:01.029 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535639+0000", 2024-09-16T20:51:01.029 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9697000000000002e-05 2024-09-16T20:51:01.029 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.029 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.029 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:01.029 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535675+0000", 2024-09-16T20:51:01.030 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5729999999999998e-05 2024-09-16T20:51:01.030 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:01.030 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:01.030 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:01.030 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.030 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.030 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3580 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:01.030 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:51:01.030 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 131.505133564, 2024-09-16T20:51:01.030 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054397361999999998, 2024-09-16T20:51:01.030 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:01.030 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:01.032 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:01.032 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:01.032 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:01.032 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:01.032 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3580 2024-09-16T20:51:01.032 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.032 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:01.033 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.033 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:01.033 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:51:01.033 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:01.033 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.033 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.033 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:01.033 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:51:01.033 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:01.033 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.033 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.033 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:01.033 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:51:01.033 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2600000000000001e-07 2024-09-16T20:51:01.034 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.034 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.034 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:01.034 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481395+0000", 2024-09-16T20:51:01.034 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.544e-06 2024-09-16T20:51:01.034 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.034 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.034 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:01.034 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481404+0000", 2024-09-16T20:51:01.034 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8850000000000005e-06 2024-09-16T20:51:01.034 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.034 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.034 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:01.034 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481407+0000", 2024-09-16T20:51:01.035 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1540000000000002e-06 2024-09-16T20:51:01.035 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.035 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.035 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:01.035 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531823+0000", 2024-09-16T20:51:01.035 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050415637999999999 2024-09-16T20:51:01.035 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.035 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.035 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:01.035 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531895+0000", 2024-09-16T20:51:01.035 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2179000000000001e-05 2024-09-16T20:51:01.035 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.035 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.035 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:01.035 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532236+0000", 2024-09-16T20:51:01.035 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00034073799999999999 2024-09-16T20:51:01.036 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.036 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.036 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:01.036 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534083+0000", 2024-09-16T20:51:01.036 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001847298 2024-09-16T20:51:01.036 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.036 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.036 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:01.036 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535714+0000", 2024-09-16T20:51:01.036 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0016316810000000001 2024-09-16T20:51:01.036 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.036 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.036 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:01.036 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535759+0000", 2024-09-16T20:51:01.036 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4478999999999999e-05 2024-09-16T20:51:01.037 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.037 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.037 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:01.037 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535791+0000", 2024-09-16T20:51:01.037 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.154e-05 2024-09-16T20:51:01.037 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:01.037 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:01.037 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:01.037 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.037 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.037 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3581 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:01.037 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:51:01.037 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 131.505111567, 2024-09-16T20:51:01.037 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.057094403000000002, 2024-09-16T20:51:01.038 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:01.038 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:01.038 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:01.038 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:01.038 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:01.038 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:01.038 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3581 2024-09-16T20:51:01.038 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.038 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:01.038 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.038 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:01.038 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:51:01.038 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:01.038 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.038 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.039 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:01.039 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:51:01.039 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:01.039 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.039 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.039 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:01.039 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:51:01.039 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8799999999999998e-07 2024-09-16T20:51:01.039 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.039 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.039 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:01.039 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481418+0000", 2024-09-16T20:51:01.039 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1859999999999999e-06 2024-09-16T20:51:01.039 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.039 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.040 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:01.040 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481423+0000", 2024-09-16T20:51:01.040 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8279999999999999e-06 2024-09-16T20:51:01.040 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.040 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.040 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:01.040 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481426+0000", 2024-09-16T20:51:01.040 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0110000000000001e-06 2024-09-16T20:51:01.040 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.040 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.040 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:01.040 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532631+0000", 2024-09-16T20:51:01.040 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051204514999999999 2024-09-16T20:51:01.040 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.040 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.041 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:01.041 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.533381+0000", 2024-09-16T20:51:01.041 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00075029000000000003 2024-09-16T20:51:01.041 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.041 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.041 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:01.041 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.533739+0000", 2024-09-16T20:51:01.041 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00035773100000000002 2024-09-16T20:51:01.041 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.041 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.041 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:01.041 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535123+0000", 2024-09-16T20:51:01.041 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0013836160000000001 2024-09-16T20:51:01.041 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.042 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.042 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:01.042 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538413+0000", 2024-09-16T20:51:01.042 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0032902510000000001 2024-09-16T20:51:01.042 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.042 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.042 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:01.042 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538470+0000", 2024-09-16T20:51:01.042 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6697999999999998e-05 2024-09-16T20:51:01.042 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.042 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.042 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:01.042 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538510+0000", 2024-09-16T20:51:01.042 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9988999999999999e-05 2024-09-16T20:51:01.042 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:01.043 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:01.043 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:01.043 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.043 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.043 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:12732 3.d 3:b752444a:::rbd_header.110e747f0779:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-16T20:51:01.043 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:51:01.043 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 116.21803898899999, 2024-09-16T20:51:01.043 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.065302467000000003, 2024-09-16T20:51:01.043 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:01.043 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:01.043 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:01.043 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:01.043 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:51:01.043 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:51:01.043 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 12732 2024-09-16T20:51:01.044 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.044 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:01.044 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.044 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:01.044 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:51:01.044 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:01.044 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.044 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.044 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:01.044 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:51:01.044 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:01.044 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.044 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.044 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:01.044 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768490+0000", 2024-09-16T20:51:01.045 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2859999999999998e-06 2024-09-16T20:51:01.045 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.045 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.045 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:01.045 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768501+0000", 2024-09-16T20:51:01.045 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1226999999999999e-05 2024-09-16T20:51:01.045 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.045 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.045 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:01.045 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768517+0000", 2024-09-16T20:51:01.045 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.592e-05 2024-09-16T20:51:01.045 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.045 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.046 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:01.046 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768528+0000", 2024-09-16T20:51:01.046 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0689999999999999e-05 2024-09-16T20:51:01.046 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.046 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.046 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:01.046 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768584+0000", 2024-09-16T20:51:01.046 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5889e-05 2024-09-16T20:51:01.046 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.046 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.046 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:01.046 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768720+0000", 2024-09-16T20:51:01.046 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000136718 2024-09-16T20:51:01.046 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.047 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.047 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:01.047 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.769066+0000", 2024-09-16T20:51:01.047 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000346013 2024-09-16T20:51:01.047 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.047 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.047 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:01.047 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833659+0000", 2024-09-16T20:51:01.047 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.064592822999999994 2024-09-16T20:51:01.047 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.047 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.047 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:01.047 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833719+0000", 2024-09-16T20:51:01.047 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9441000000000002e-05 2024-09-16T20:51:01.047 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.048 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.048 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:01.048 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833758+0000", 2024-09-16T20:51:01.048 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9122000000000003e-05 2024-09-16T20:51:01.048 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.048 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.048 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:01.048 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833790+0000", 2024-09-16T20:51:01.048 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2338000000000001e-05 2024-09-16T20:51:01.048 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:01.048 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:01.048 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:01.048 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.048 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.049 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.5089.0:12732 3.d e73/23 ondisk, result = 0)", 2024-09-16T20:51:01.049 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:51:01.049 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 116.21643681899999, 2024-09-16T20:51:01.049 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.063706456999999994, 2024-09-16T20:51:01.049 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:01.049 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:01.049 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:01.049 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:01.049 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.049 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:01.049 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:51:01.049 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:01.050 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.050 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.050 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:01.050 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:51:01.050 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:01.050 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.050 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.050 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:01.050 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770091+0000", 2024-09-16T20:51:01.050 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.53e-06 2024-09-16T20:51:01.050 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.050 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.050 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:01.050 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770104+0000", 2024-09-16T20:51:01.050 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2203e-05 2024-09-16T20:51:01.051 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.051 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.051 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:01.051 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833637+0000", 2024-09-16T20:51:01.051 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.063533201999999997 2024-09-16T20:51:01.051 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.051 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.051 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:01.051 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833655+0000", 2024-09-16T20:51:01.051 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8369e-05 2024-09-16T20:51:01.051 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.051 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.051 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:01.051 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833707+0000", 2024-09-16T20:51:01.051 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2061999999999997e-05 2024-09-16T20:51:01.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.052 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.052 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:01.052 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833715+0000", 2024-09-16T20:51:01.052 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5179999999999998e-06 2024-09-16T20:51:01.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.052 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.052 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:01.052 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833796+0000", 2024-09-16T20:51:01.052 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1573e-05 2024-09-16T20:51:01.052 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:01.052 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:01.052 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:01.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.052 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.053 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3286 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 245760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:01.053 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:51:01.053 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 70.945374881999996, 2024-09-16T20:51:01.053 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052523564000000002, 2024-09-16T20:51:01.053 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:01.053 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:01.053 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:01.053 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:01.053 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:51:01.053 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:01.053 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3286 2024-09-16T20:51:01.053 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.053 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:01.053 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.054 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:01.054 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:51:01.054 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:01.054 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.054 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.054 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:01.054 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:51:01.054 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:01.054 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.054 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.054 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:01.054 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:51:01.054 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3900000000000001e-07 2024-09-16T20:51:01.054 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.054 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.054 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:01.055 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041153+0000", 2024-09-16T20:51:01.055 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.119e-06 2024-09-16T20:51:01.055 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.055 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.055 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:01.055 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041156+0000", 2024-09-16T20:51:01.055 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0000000000000001e-06 2024-09-16T20:51:01.055 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.055 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.055 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:01.055 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041159+0000", 2024-09-16T20:51:01.055 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5859999999999999e-06 2024-09-16T20:51:01.055 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.055 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.055 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:01.055 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093529+0000", 2024-09-16T20:51:01.056 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052370591000000001 2024-09-16T20:51:01.056 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.056 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.056 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:01.056 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093570+0000", 2024-09-16T20:51:01.056 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0161999999999999e-05 2024-09-16T20:51:01.056 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.056 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.056 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:01.056 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093675+0000", 2024-09-16T20:51:01.056 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010586699999999999 2024-09-16T20:51:01.056 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:01.056 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:01.057 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:01.057 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.057 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.057 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3287 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 250880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:01.057 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:51:01.057 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 70.945364832999999, 2024-09-16T20:51:01.057 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052781203999999998, 2024-09-16T20:51:01.057 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:01.057 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:01.057 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:01.057 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:01.057 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:51:01.057 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:01.057 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3287 2024-09-16T20:51:01.058 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.058 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:01.058 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.058 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:01.058 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:51:01.058 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:01.058 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.058 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.058 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:01.058 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:51:01.058 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:01.058 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.058 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.059 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:01.059 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:51:01.059 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1e-07 2024-09-16T20:51:01.059 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.059 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.059 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:01.059 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041163+0000", 2024-09-16T20:51:01.059 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0690000000000001e-06 2024-09-16T20:51:01.059 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.059 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.059 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:01.059 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041166+0000", 2024-09-16T20:51:01.060 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.982e-06 2024-09-16T20:51:01.060 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.060 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.060 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:01.060 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041169+0000", 2024-09-16T20:51:01.060 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.593e-06 2024-09-16T20:51:01.060 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.060 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.060 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:01.060 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093709+0000", 2024-09-16T20:51:01.060 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052540492000000001 2024-09-16T20:51:01.060 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.060 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.061 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:01.061 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093792+0000", 2024-09-16T20:51:01.061 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2436999999999997e-05 2024-09-16T20:51:01.061 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.061 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.061 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:01.061 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093943+0000", 2024-09-16T20:51:01.061 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000151421 2024-09-16T20:51:01.061 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:01.061 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:01.061 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:01.061 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.061 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.061 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3288 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 256000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:01.062 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:51:01.062 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 70.945354960000003, 2024-09-16T20:51:01.062 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052928269, 2024-09-16T20:51:01.062 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:01.062 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:01.062 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:01.062 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:01.062 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:51:01.062 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:01.062 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3288 2024-09-16T20:51:01.062 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.062 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:01.062 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.062 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:01.062 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:51:01.063 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:01.063 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.063 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.063 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:01.063 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:51:01.063 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:01.063 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.063 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.063 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:01.063 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:51:01.063 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2700000000000001e-07 2024-09-16T20:51:01.063 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.063 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.064 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:01.064 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041173+0000", 2024-09-16T20:51:01.064 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.113e-06 2024-09-16T20:51:01.064 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.064 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.064 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:01.064 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041176+0000", 2024-09-16T20:51:01.064 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9799999999999998e-06 2024-09-16T20:51:01.064 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.064 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.064 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:01.064 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041179+0000", 2024-09-16T20:51:01.065 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.667e-06 2024-09-16T20:51:01.065 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.065 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.065 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:01.065 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093957+0000", 2024-09-16T20:51:01.065 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052778703000000003 2024-09-16T20:51:01.065 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.065 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.065 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:01.065 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093999+0000", 2024-09-16T20:51:01.066 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1300000000000001e-05 2024-09-16T20:51:01.066 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.066 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.066 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:01.066 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094100+0000", 2024-09-16T20:51:01.066 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101279 2024-09-16T20:51:01.066 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:01.066 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:01.066 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:01.066 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.066 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.066 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3289 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 261120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:01.067 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:51:01.067 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 70.945344964, 2024-09-16T20:51:01.067 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053182976, 2024-09-16T20:51:01.067 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:01.067 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:01.067 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:01.067 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:01.067 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:51:01.067 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:01.067 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3289 2024-09-16T20:51:01.067 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.067 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:01.067 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.068 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:01.068 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:51:01.068 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:01.068 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.068 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.068 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:01.068 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:51:01.068 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:01.068 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.068 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.068 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:01.068 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:51:01.068 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2999999999999999e-07 2024-09-16T20:51:01.069 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.069 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.069 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:01.069 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041183+0000", 2024-09-16T20:51:01.069 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0720000000000001e-06 2024-09-16T20:51:01.069 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.069 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.069 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:01.069 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041186+0000", 2024-09-16T20:51:01.069 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9869999999999999e-06 2024-09-16T20:51:01.069 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.070 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.070 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:01.070 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041189+0000", 2024-09-16T20:51:01.070 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.587e-06 2024-09-16T20:51:01.070 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.070 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.070 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:01.070 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094136+0000", 2024-09-16T20:51:01.070 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052947270999999997 2024-09-16T20:51:01.070 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.070 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.071 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:01.071 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094213+0000", 2024-09-16T20:51:01.071 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7552000000000006e-05 2024-09-16T20:51:01.071 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.071 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.071 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:01.071 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094365+0000", 2024-09-16T20:51:01.071 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00015127700000000001 2024-09-16T20:51:01.071 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:01.071 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:01.071 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:01.071 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.071 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.071 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:2196 3.d 3:b1fe01a0:::rbd_data.110e7ef3004.0000000000000000:2d [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e136)", 2024-09-16T20:51:01.072 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:22.739057+0000", 2024-09-16T20:51:01.072 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 38.247470165000003, 2024-09-16T20:51:01.072 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11999464899999999, 2024-09-16T20:51:01.072 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:01.072 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:01.072 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:01.072 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:01.072 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:01.072 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:01.072 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2196 2024-09-16T20:51:01.072 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.072 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:01.072 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.072 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:01.073 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.739057+0000", 2024-09-16T20:51:01.073 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:01.073 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.073 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.073 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:01.073 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.739057+0000", 2024-09-16T20:51:01.073 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:01.073 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.073 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.073 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:01.073 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.739057+0000", 2024-09-16T20:51:01.073 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1399999999999997e-07 2024-09-16T20:51:01.073 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.074 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.074 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:01.074 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.739060+0000", 2024-09-16T20:51:01.074 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6019999999999998e-06 2024-09-16T20:51:01.074 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.074 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.074 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:01.074 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.858755+0000", 2024-09-16T20:51:01.074 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.119695151 2024-09-16T20:51:01.074 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.074 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.074 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:01.074 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.858778+0000", 2024-09-16T20:51:01.074 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3184999999999999e-05 2024-09-16T20:51:01.075 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.075 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.075 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:01.075 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.858865+0000", 2024-09-16T20:51:01.075 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.7330000000000006e-05 2024-09-16T20:51:01.075 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.075 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.075 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:01.075 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.858952+0000", 2024-09-16T20:51:01.075 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6803000000000002e-05 2024-09-16T20:51:01.075 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.075 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.075 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:01.075 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.859051+0000", 2024-09-16T20:51:01.075 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8963999999999996e-05 2024-09-16T20:51:01.076 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:01.076 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:01.076 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:01.076 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.076 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.076 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6061.0:5401 3.13 3:c8554348:::rbd_data.110e16eafbd.0000000000000011:head [read 577536~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-16T20:51:01.076 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:27.761536+0000", 2024-09-16T20:51:01.076 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 33.224990351999999, 2024-09-16T20:51:01.076 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.003742712, 2024-09-16T20:51:01.076 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:01.076 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:01.076 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:51:01.076 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:01.077 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6061", 2024-09-16T20:51:01.077 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3930128440", 2024-09-16T20:51:01.077 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5401 2024-09-16T20:51:01.077 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.077 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:01.077 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.077 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:01.077 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761536+0000", 2024-09-16T20:51:01.077 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:01.077 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.077 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.077 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:01.077 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761536+0000", 2024-09-16T20:51:01.078 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:01.078 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.078 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.078 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:01.078 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761537+0000", 2024-09-16T20:51:01.078 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2900000000000003e-07 2024-09-16T20:51:01.078 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.078 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.078 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:01.078 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761540+0000", 2024-09-16T20:51:01.078 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2320000000000001e-06 2024-09-16T20:51:01.078 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.078 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.078 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:01.079 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761548+0000", 2024-09-16T20:51:01.079 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1289999999999996e-06 2024-09-16T20:51:01.079 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.079 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.079 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:01.079 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761554+0000", 2024-09-16T20:51:01.079 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0820000000000004e-06 2024-09-16T20:51:01.079 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.079 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.079 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:01.079 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761621+0000", 2024-09-16T20:51:01.079 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6266000000000006e-05 2024-09-16T20:51:01.079 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.080 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.080 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:01.080 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761626+0000", 2024-09-16T20:51:01.080 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.614e-06 2024-09-16T20:51:01.080 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.080 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.080 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:01.080 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:28.765163+0000", 2024-09-16T20:51:01.080 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.003536416 2024-09-16T20:51:01.080 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.080 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.080 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:01.080 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:28.765279+0000", 2024-09-16T20:51:01.081 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000116344 2024-09-16T20:51:01.081 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:01.081 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:01.081 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:01.081 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.081 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.081 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6061.0:5425 3.13 3:c8554348:::rbd_data.110e16eafbd.0000000000000011:head [read 602112~28672] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-16T20:51:01.081 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:27.767049+0000", 2024-09-16T20:51:01.081 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 33.219477693000002, 2024-09-16T20:51:01.081 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.99847897900000004, 2024-09-16T20:51:01.081 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:01.081 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:01.081 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:51:01.081 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:01.082 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6061", 2024-09-16T20:51:01.082 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3930128440", 2024-09-16T20:51:01.082 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5425 2024-09-16T20:51:01.082 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.082 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:01.082 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.082 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:01.082 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767049+0000", 2024-09-16T20:51:01.082 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:01.082 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.082 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.082 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:01.082 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767049+0000", 2024-09-16T20:51:01.083 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:01.083 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.083 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.083 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:01.083 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767050+0000", 2024-09-16T20:51:01.083 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6499999999999999e-07 2024-09-16T20:51:01.083 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.083 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.083 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:01.083 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767052+0000", 2024-09-16T20:51:01.083 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7350000000000001e-06 2024-09-16T20:51:01.083 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.083 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.083 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:01.084 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767060+0000", 2024-09-16T20:51:01.084 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5109999999999997e-06 2024-09-16T20:51:01.084 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.084 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.084 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:01.084 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767066+0000", 2024-09-16T20:51:01.084 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8139999999999997e-06 2024-09-16T20:51:01.084 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.084 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.084 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:01.084 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767128+0000", 2024-09-16T20:51:01.084 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2174999999999993e-05 2024-09-16T20:51:01.084 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.084 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.085 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:01.085 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767133+0000", 2024-09-16T20:51:01.085 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2140000000000004e-06 2024-09-16T20:51:01.085 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.085 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.085 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:01.085 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:28.765441+0000", 2024-09-16T20:51:01.085 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.99830808599999998 2024-09-16T20:51:01.085 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.085 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.085 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:01.085 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:28.765528+0000", 2024-09-16T20:51:01.085 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6879000000000006e-05 2024-09-16T20:51:01.085 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:01.086 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:01.086 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:01.086 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.086 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.086 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:25585 3.10 3:0c7c5b23:::rbd_header.110ebf0678ad:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:51:01.086 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:32.877132+0000", 2024-09-16T20:51:01.086 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 28.109395144, 2024-09-16T20:51:01.086 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054096699999999998, 2024-09-16T20:51:01.086 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:01.086 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:01.086 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:01.086 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:01.086 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:51:01.087 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:01.087 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 25585 2024-09-16T20:51:01.087 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.087 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:01.087 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.087 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:01.087 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.877132+0000", 2024-09-16T20:51:01.087 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:01.087 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.087 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.087 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:01.087 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.877132+0000", 2024-09-16T20:51:01.087 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:01.087 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.088 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.088 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:01.088 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931072+0000", 2024-09-16T20:51:01.088 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053939931000000003 2024-09-16T20:51:01.088 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.088 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.088 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:01.088 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931076+0000", 2024-09-16T20:51:01.088 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2860000000000001e-06 2024-09-16T20:51:01.088 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.088 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.088 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:01.088 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931085+0000", 2024-09-16T20:51:01.089 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.3370000000000006e-06 2024-09-16T20:51:01.089 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.089 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.089 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:01.089 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931092+0000", 2024-09-16T20:51:01.089 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9970000000000001e-06 2024-09-16T20:51:01.089 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.089 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.089 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:01.089 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931137+0000", 2024-09-16T20:51:01.089 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5160000000000001e-05 2024-09-16T20:51:01.089 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.089 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.090 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:01.090 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931178+0000", 2024-09-16T20:51:01.090 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0379000000000001e-05 2024-09-16T20:51:01.090 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.090 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.090 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:01.090 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931228+0000", 2024-09-16T20:51:01.090 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0609999999999998e-05 2024-09-16T20:51:01.090 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:01.090 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:01.090 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:01.090 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.090 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.090 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:26679 3.c 3:33e2f78f:::rbd_header.110e16eafbd:head [watch ping cookie 140213436013408 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e153)", 2024-09-16T20:51:01.091 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:41.440287+0000", 2024-09-16T20:51:01.091 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 19.546239269000001, 2024-09-16T20:51:01.091 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.13295633600000001, 2024-09-16T20:51:01.091 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:01.091 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:01.091 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:01.091 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:01.091 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:51:01.091 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:01.091 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 26679 2024-09-16T20:51:01.091 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.091 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:01.091 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.091 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:01.092 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440287+0000", 2024-09-16T20:51:01.092 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:01.092 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.092 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.092 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:01.092 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440287+0000", 2024-09-16T20:51:01.092 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:01.092 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.092 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.092 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:01.092 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440288+0000", 2024-09-16T20:51:01.092 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0900000000000001e-07 2024-09-16T20:51:01.092 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.092 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.093 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:01.093 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440292+0000", 2024-09-16T20:51:01.093 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3749999999999996e-06 2024-09-16T20:51:01.093 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.093 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.093 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:01.093 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440299+0000", 2024-09-16T20:51:01.093 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8249999999999999e-06 2024-09-16T20:51:01.093 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.093 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.093 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:01.093 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440307+0000", 2024-09-16T20:51:01.093 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4800000000000004e-06 2024-09-16T20:51:01.093 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.094 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.094 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:01.094 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440451+0000", 2024-09-16T20:51:01.094 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014414500000000001 2024-09-16T20:51:01.094 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.094 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.094 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:01.094 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440465+0000", 2024-09-16T20:51:01.094 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.402e-05 2024-09-16T20:51:01.094 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.094 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.094 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:01.094 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.573043+0000", 2024-09-16T20:51:01.094 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.13257855399999999 2024-09-16T20:51:01.095 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.095 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.095 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:01.095 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.573130+0000", 2024-09-16T20:51:01.095 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6168999999999994e-05 2024-09-16T20:51:01.095 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:01.095 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:01.095 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:01.095 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.573244+0000", 2024-09-16T20:51:01.095 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000114159 2024-09-16T20:51:01.095 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:01.095 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:01.095 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:01.095 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:01.096 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:01.096 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:01.097 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:51:01.098 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:01.095+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:01.101 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T20:51:01.198 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:01.196+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:01.248 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:01.249 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:51:01.249 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:51:01.249 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:01.258 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:51:01.263 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:51:01.263 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:51:01.282 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:51:01.282 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:51:01.298 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:01.297+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:01.300 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:51:01.300 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:51:01.318 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:51:01.318 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:51:01.335 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:51:01.335 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:51:01.353 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:51:01.353 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:51:01.369 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:01.369 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:01.377 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:51:01.399 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:51:01.398+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:51:01.500 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:01.499+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:01.524 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:01.524 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:01.533 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:51:01.568 INFO:teuthology.orchestra.run.smithi029.stderr:noscrub is unset 2024-09-16T20:51:01.601 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:01.600+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:01.668 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:01.668 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:01.678 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:51:01.719 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:01.718+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:01.803 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:01.802+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:01.832 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:01.832 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:01.840 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:51:01.904 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:01.902+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:01.947 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:01.947 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:01.956 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:51:02.005 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:02.004+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:02.062 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:02.062 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:02.071 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:51:02.105 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:02.104+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:02.192 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:02.192 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:02.201 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:51:02.207 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:02.206+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:02.307 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:02.306+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:02.311 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:02.311 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:02.319 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:51:02.409 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:02.408+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:02.426 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:02.427 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:02.435 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:51:02.510 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:02.509+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:02.585 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:02.585 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:02.596 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:51:02.611 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:02.610+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:02.705 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:02.705 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:02.712 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:02.711+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:02.714 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:51:02.813 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:02.812+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:02.834 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:02.834 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:02.844 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:51:02.914 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:02.913+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:02.965 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:02.965 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:02.974 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:51:03.016 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:03.014+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:03.117 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:03.115+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:03.145 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:03.145 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:03.156 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:51:03.218 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:03.217+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:03.267 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:03.267 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:03.277 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:51:03.319 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:03.317+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:03.364 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T20:51:03.420 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:03.419+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:03.448 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:03.449 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:03.460 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:51:03.514 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:51:03.515 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:51:03.521 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:03.519+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:03.538 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:51:03.539 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:51:03.560 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:51:03.560 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:51:03.593 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:51:03.593 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:51:03.604 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T20:51:03.623 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:51:03.623 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:51:03.623 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:03.622+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:03.656 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:51:03.656 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:51:03.658 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:03.658 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:51:03.658 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:51:03.659 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:51:03.659 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.659 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:51:03.659 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:51:03.659 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 190.402016524, 2024-09-16T20:51:03.659 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67502346999999996, 2024-09-16T20:51:03.659 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:03.659 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:03.659 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:03.659 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:03.659 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:51:03.659 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:51:03.659 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 1 2024-09-16T20:51:03.660 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.660 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:03.660 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.660 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:03.660 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:51:03.660 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:03.660 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.660 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.660 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:03.660 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:51:03.660 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:03.660 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.660 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.661 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:03.661 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254128+0000", 2024-09-16T20:51:03.661 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.463e-06 2024-09-16T20:51:03.661 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.661 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.661 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:03.661 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254135+0000", 2024-09-16T20:51:03.661 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1579999999999999e-06 2024-09-16T20:51:03.661 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.661 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.661 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:03.661 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254146+0000", 2024-09-16T20:51:03.661 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1341e-05 2024-09-16T20:51:03.662 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.662 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.662 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:03.662 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254155+0000", 2024-09-16T20:51:03.662 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.1549999999999996e-06 2024-09-16T20:51:03.662 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.662 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.662 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:03.662 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254221+0000", 2024-09-16T20:51:03.662 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5470999999999997e-05 2024-09-16T20:51:03.662 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.662 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.662 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:03.662 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254231+0000", 2024-09-16T20:51:03.662 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0592e-05 2024-09-16T20:51:03.663 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.663 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.663 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:03.663 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926711+0000", 2024-09-16T20:51:03.663 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67247946599999997 2024-09-16T20:51:03.663 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.663 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.663 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:03.663 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926910+0000", 2024-09-16T20:51:03.663 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019934 2024-09-16T20:51:03.663 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.663 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.663 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:51:03.663 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.927159+0000", 2024-09-16T20:51:03.663 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00024876599999999998 2024-09-16T20:51:03.664 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.664 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.664 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:03.664 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.928797+0000", 2024-09-16T20:51:03.664 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001638308 2024-09-16T20:51:03.664 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.664 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.664 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:03.664 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929026+0000", 2024-09-16T20:51:03.664 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00022876099999999999 2024-09-16T20:51:03.664 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.664 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.664 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:03.665 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929102+0000", 2024-09-16T20:51:03.665 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6184000000000001e-05 2024-09-16T20:51:03.665 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.665 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.665 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:03.665 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929150+0000", 2024-09-16T20:51:03.665 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7465e-05 2024-09-16T20:51:03.665 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:03.665 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:03.665 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:03.665 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.665 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.665 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3574 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:03.666 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:51:03.666 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 134.17485299200001, 2024-09-16T20:51:03.666 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053579441999999998, 2024-09-16T20:51:03.666 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:03.666 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:03.666 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:03.666 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:03.666 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:03.666 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:03.666 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3574 2024-09-16T20:51:03.666 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.666 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:03.666 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.667 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:03.667 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:51:03.667 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:03.667 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.667 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.667 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:03.667 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:51:03.667 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:03.667 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.667 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.668 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:03.668 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:51:03.668 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2e-07 2024-09-16T20:51:03.668 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.668 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.668 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:03.668 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481292+0000", 2024-09-16T20:51:03.668 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2630000000000002e-06 2024-09-16T20:51:03.668 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.668 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.668 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:03.668 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481297+0000", 2024-09-16T20:51:03.668 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.296e-06 2024-09-16T20:51:03.668 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.669 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.669 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:03.669 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481300+0000", 2024-09-16T20:51:03.669 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7019999999999999e-06 2024-09-16T20:51:03.669 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.669 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.669 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:03.669 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525533+0000", 2024-09-16T20:51:03.669 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.044232318 2024-09-16T20:51:03.669 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.669 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.669 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:03.669 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525619+0000", 2024-09-16T20:51:03.669 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5892999999999999e-05 2024-09-16T20:51:03.670 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.670 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.670 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:03.670 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525987+0000", 2024-09-16T20:51:03.670 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00036836099999999999 2024-09-16T20:51:03.670 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.670 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.670 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:03.670 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531779+0000", 2024-09-16T20:51:03.670 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0057918240000000001 2024-09-16T20:51:03.670 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.670 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.670 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:03.670 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534747+0000", 2024-09-16T20:51:03.671 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0029681600000000001 2024-09-16T20:51:03.671 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.671 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.671 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:03.671 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534832+0000", 2024-09-16T20:51:03.671 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5180999999999996e-05 2024-09-16T20:51:03.671 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.671 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.671 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:03.671 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534869+0000", 2024-09-16T20:51:03.671 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7024000000000003e-05 2024-09-16T20:51:03.671 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:03.671 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:03.671 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:03.672 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.672 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.672 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3575 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:03.672 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:51:03.672 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 134.174838252, 2024-09-16T20:51:03.672 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053801463000000001, 2024-09-16T20:51:03.672 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:03.672 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:03.672 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:03.672 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:03.672 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:03.672 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:03.672 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3575 2024-09-16T20:51:03.673 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.673 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:03.673 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.673 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:03.673 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:51:03.673 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:03.673 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.673 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.673 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:03.673 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:51:03.673 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:03.673 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.673 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.673 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:03.674 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481305+0000", 2024-09-16T20:51:03.674 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5400000000000002e-07 2024-09-16T20:51:03.674 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.674 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.674 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:03.674 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481307+0000", 2024-09-16T20:51:03.674 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7800000000000001e-06 2024-09-16T20:51:03.674 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.674 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.674 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:03.674 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481312+0000", 2024-09-16T20:51:03.674 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6349999999999997e-06 2024-09-16T20:51:03.674 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.674 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.674 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:03.675 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481315+0000", 2024-09-16T20:51:03.675 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7829999999999999e-06 2024-09-16T20:51:03.675 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.675 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.675 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:03.675 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.526876+0000", 2024-09-16T20:51:03.676 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.045561070000000002 2024-09-16T20:51:03.676 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.676 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.676 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:03.676 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.526954+0000", 2024-09-16T20:51:03.676 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8108000000000005e-05 2024-09-16T20:51:03.676 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.676 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.676 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:03.676 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527282+0000", 2024-09-16T20:51:03.676 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000327483 2024-09-16T20:51:03.677 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.677 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.677 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:03.677 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531784+0000", 2024-09-16T20:51:03.677 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0045020169999999997 2024-09-16T20:51:03.677 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.677 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.677 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:03.677 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534921+0000", 2024-09-16T20:51:03.677 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.003137791 2024-09-16T20:51:03.677 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.677 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.677 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:03.677 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534976+0000", 2024-09-16T20:51:03.678 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4953000000000001e-05 2024-09-16T20:51:03.678 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.678 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.678 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:03.678 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535106+0000", 2024-09-16T20:51:03.678 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000129589 2024-09-16T20:51:03.678 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:03.678 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:03.678 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:03.678 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.678 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.678 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3576 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:03.678 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:51:03.679 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 134.17482433999999, 2024-09-16T20:51:03.679 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053955436000000002, 2024-09-16T20:51:03.679 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:03.679 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:03.679 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:03.679 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:03.679 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:03.679 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:03.679 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3576 2024-09-16T20:51:03.679 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.679 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:03.679 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.679 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:03.679 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:51:03.680 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:03.680 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.680 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.680 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:03.680 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:51:03.680 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:03.680 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.680 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.680 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:03.680 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481319+0000", 2024-09-16T20:51:03.680 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.53e-07 2024-09-16T20:51:03.680 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.680 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.680 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:03.681 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481320+0000", 2024-09-16T20:51:03.681 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5230000000000001e-06 2024-09-16T20:51:03.681 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.681 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.681 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:03.681 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481330+0000", 2024-09-16T20:51:03.681 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0190000000000001e-05 2024-09-16T20:51:03.681 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.681 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.681 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:03.681 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481333+0000", 2024-09-16T20:51:03.681 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.151e-06 2024-09-16T20:51:03.681 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.682 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.682 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:03.682 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527689+0000", 2024-09-16T20:51:03.682 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046355192000000003 2024-09-16T20:51:03.682 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.682 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.682 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:03.682 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527764+0000", 2024-09-16T20:51:03.682 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5776000000000001e-05 2024-09-16T20:51:03.682 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.682 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.682 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:03.682 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528141+0000", 2024-09-16T20:51:03.683 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00037647599999999999 2024-09-16T20:51:03.683 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.683 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.683 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:03.683 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531788+0000", 2024-09-16T20:51:03.683 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0036470679999999998 2024-09-16T20:51:03.683 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.683 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.683 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:03.683 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535174+0000", 2024-09-16T20:51:03.683 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0033862670000000001 2024-09-16T20:51:03.683 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.683 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.683 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:03.684 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535236+0000", 2024-09-16T20:51:03.684 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1797999999999993e-05 2024-09-16T20:51:03.684 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.684 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.684 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:03.684 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535274+0000", 2024-09-16T20:51:03.684 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7741999999999999e-05 2024-09-16T20:51:03.684 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:03.684 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:03.684 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:03.684 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.684 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.684 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3577 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:03.684 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:51:03.685 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 134.17480273999999, 2024-09-16T20:51:03.685 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054073807000000002, 2024-09-16T20:51:03.685 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:03.685 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:03.685 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:03.685 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:03.685 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:03.685 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:03.685 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3577 2024-09-16T20:51:03.685 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.685 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:03.685 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.685 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:03.685 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:51:03.686 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:03.686 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.686 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.686 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:03.686 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:51:03.686 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:03.686 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.686 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.686 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:03.686 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:51:03.686 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.96e-07 2024-09-16T20:51:03.686 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.686 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.686 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:03.687 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481342+0000", 2024-09-16T20:51:03.687 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.714e-06 2024-09-16T20:51:03.687 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.687 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.687 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:03.687 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481349+0000", 2024-09-16T20:51:03.687 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4930000000000001e-06 2024-09-16T20:51:03.687 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.687 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.687 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:03.687 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481353+0000", 2024-09-16T20:51:03.687 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4879999999999999e-06 2024-09-16T20:51:03.687 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.688 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.688 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:03.688 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528619+0000", 2024-09-16T20:51:03.688 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047266351999999998 2024-09-16T20:51:03.688 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.688 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.688 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:03.688 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528706+0000", 2024-09-16T20:51:03.688 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6796e-05 2024-09-16T20:51:03.688 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.688 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.688 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:03.688 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529148+0000", 2024-09-16T20:51:03.688 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00044182999999999999 2024-09-16T20:51:03.689 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.689 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.689 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:03.689 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531792+0000", 2024-09-16T20:51:03.689 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0026443119999999998 2024-09-16T20:51:03.689 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.689 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.689 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:03.689 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535319+0000", 2024-09-16T20:51:03.689 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0035266439999999998 2024-09-16T20:51:03.689 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.689 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.689 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:03.690 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535372+0000", 2024-09-16T20:51:03.690 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3207000000000001e-05 2024-09-16T20:51:03.690 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.690 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.690 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:03.690 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535414+0000", 2024-09-16T20:51:03.690 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1675000000000003e-05 2024-09-16T20:51:03.690 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:03.690 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:03.690 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:03.690 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.690 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.690 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3578 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:03.690 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:51:03.691 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 134.17478157100001, 2024-09-16T20:51:03.691 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054189586999999997, 2024-09-16T20:51:03.691 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:03.691 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:03.691 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:03.691 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:03.691 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:03.691 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:03.691 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3578 2024-09-16T20:51:03.691 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.691 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:03.691 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.691 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:03.691 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:51:03.691 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:03.692 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.692 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.692 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:03.692 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:51:03.692 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:03.692 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.692 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.692 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:03.692 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:51:03.692 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0800000000000001e-07 2024-09-16T20:51:03.692 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.692 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.692 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:03.692 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481363+0000", 2024-09-16T20:51:03.693 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0339999999999999e-06 2024-09-16T20:51:03.693 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.693 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.693 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:03.693 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481371+0000", 2024-09-16T20:51:03.693 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4440000000000002e-06 2024-09-16T20:51:03.693 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.693 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.693 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:03.693 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481374+0000", 2024-09-16T20:51:03.694 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2049999999999998e-06 2024-09-16T20:51:03.694 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.694 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.694 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:03.694 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529555+0000", 2024-09-16T20:51:03.694 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048181202999999999 2024-09-16T20:51:03.694 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.694 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.694 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:03.694 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529616+0000", 2024-09-16T20:51:03.694 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0912000000000002e-05 2024-09-16T20:51:03.694 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.694 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.694 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:03.695 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529939+0000", 2024-09-16T20:51:03.695 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00032269800000000001 2024-09-16T20:51:03.695 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.695 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.695 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:03.695 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532602+0000", 2024-09-16T20:51:03.695 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0026634940000000002 2024-09-16T20:51:03.695 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.695 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.695 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:03.695 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535465+0000", 2024-09-16T20:51:03.695 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.002862149 2024-09-16T20:51:03.695 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.695 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.695 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:03.696 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535516+0000", 2024-09-16T20:51:03.696 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1696000000000001e-05 2024-09-16T20:51:03.696 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.696 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.696 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:03.696 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535551+0000", 2024-09-16T20:51:03.696 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4443999999999998e-05 2024-09-16T20:51:03.696 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:03.696 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:03.696 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:03.696 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.696 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.696 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3579 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:03.696 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:51:03.697 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 134.17476313200001, 2024-09-16T20:51:03.697 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054295151, 2024-09-16T20:51:03.697 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:03.697 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:03.697 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:03.697 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:03.697 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:03.697 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:03.697 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3579 2024-09-16T20:51:03.697 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.698 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:03.698 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.698 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:03.698 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:51:03.698 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:03.698 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.699 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.699 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:03.699 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:51:03.699 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:03.699 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.699 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.699 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:03.700 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:51:03.700 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3300000000000001e-07 2024-09-16T20:51:03.700 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.700 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.700 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:03.700 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481381+0000", 2024-09-16T20:51:03.700 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.649e-06 2024-09-16T20:51:03.700 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.700 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.700 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:03.700 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481387+0000", 2024-09-16T20:51:03.701 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4519999999999996e-06 2024-09-16T20:51:03.701 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.701 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.701 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:03.701 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481390+0000", 2024-09-16T20:51:03.701 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0170000000000001e-06 2024-09-16T20:51:03.701 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.701 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.701 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:03.701 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530756+0000", 2024-09-16T20:51:03.701 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049365882999999999 2024-09-16T20:51:03.701 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.701 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.702 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:03.702 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530845+0000", 2024-09-16T20:51:03.702 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9197000000000007e-05 2024-09-16T20:51:03.702 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.702 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.702 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:03.702 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531191+0000", 2024-09-16T20:51:03.702 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00034562500000000001 2024-09-16T20:51:03.702 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.702 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.702 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:03.702 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534078+0000", 2024-09-16T20:51:03.702 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0028878179999999999 2024-09-16T20:51:03.703 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.703 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.703 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:03.703 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535589+0000", 2024-09-16T20:51:03.703 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00151085 2024-09-16T20:51:03.703 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.703 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.703 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:03.703 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535639+0000", 2024-09-16T20:51:03.703 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9697000000000002e-05 2024-09-16T20:51:03.703 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.703 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.703 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:03.703 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535675+0000", 2024-09-16T20:51:03.703 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5729999999999998e-05 2024-09-16T20:51:03.704 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:03.704 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:03.704 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:03.704 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.704 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.704 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3580 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:03.704 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:51:03.704 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 134.17474949999999, 2024-09-16T20:51:03.704 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054397361999999998, 2024-09-16T20:51:03.704 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:03.704 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:03.704 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:03.704 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:03.705 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:03.705 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:03.705 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3580 2024-09-16T20:51:03.705 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.705 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:03.705 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.705 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:03.705 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:51:03.705 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:03.705 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.705 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.705 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:03.705 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:51:03.705 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:03.706 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.706 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.706 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:03.706 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:51:03.706 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2600000000000001e-07 2024-09-16T20:51:03.706 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.706 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.706 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:03.707 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481395+0000", 2024-09-16T20:51:03.707 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.544e-06 2024-09-16T20:51:03.707 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.707 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.707 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:03.707 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481404+0000", 2024-09-16T20:51:03.708 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8850000000000005e-06 2024-09-16T20:51:03.708 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.708 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.708 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:03.708 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481407+0000", 2024-09-16T20:51:03.708 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1540000000000002e-06 2024-09-16T20:51:03.708 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.708 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.708 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:03.708 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531823+0000", 2024-09-16T20:51:03.709 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050415637999999999 2024-09-16T20:51:03.709 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.709 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.709 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:03.709 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531895+0000", 2024-09-16T20:51:03.709 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2179000000000001e-05 2024-09-16T20:51:03.709 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.709 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.709 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:03.709 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532236+0000", 2024-09-16T20:51:03.709 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00034073799999999999 2024-09-16T20:51:03.709 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.709 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.709 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:03.710 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534083+0000", 2024-09-16T20:51:03.710 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001847298 2024-09-16T20:51:03.710 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.710 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.710 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:03.710 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535714+0000", 2024-09-16T20:51:03.710 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0016316810000000001 2024-09-16T20:51:03.710 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.710 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.710 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:03.710 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535759+0000", 2024-09-16T20:51:03.710 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4478999999999999e-05 2024-09-16T20:51:03.710 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.711 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.711 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:03.711 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535791+0000", 2024-09-16T20:51:03.711 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.154e-05 2024-09-16T20:51:03.711 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:03.711 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:03.711 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:03.711 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.711 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.711 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3581 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:03.711 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:51:03.711 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 134.17472750300001, 2024-09-16T20:51:03.711 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.057094403000000002, 2024-09-16T20:51:03.711 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:03.711 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:03.712 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:03.712 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:03.712 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:03.712 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:03.712 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3581 2024-09-16T20:51:03.712 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.712 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:03.712 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.712 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:03.712 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:51:03.712 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:03.712 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.712 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.713 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:03.713 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:51:03.713 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:03.713 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.713 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.713 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:03.713 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:51:03.713 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8799999999999998e-07 2024-09-16T20:51:03.713 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.713 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.713 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:03.713 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481418+0000", 2024-09-16T20:51:03.713 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1859999999999999e-06 2024-09-16T20:51:03.713 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.714 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.714 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:03.714 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481423+0000", 2024-09-16T20:51:03.714 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8279999999999999e-06 2024-09-16T20:51:03.714 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.714 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.714 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:03.714 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481426+0000", 2024-09-16T20:51:03.714 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0110000000000001e-06 2024-09-16T20:51:03.714 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.714 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.714 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:03.714 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532631+0000", 2024-09-16T20:51:03.715 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051204514999999999 2024-09-16T20:51:03.715 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.715 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.715 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:03.715 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.533381+0000", 2024-09-16T20:51:03.715 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00075029000000000003 2024-09-16T20:51:03.715 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.715 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.715 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:03.715 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.533739+0000", 2024-09-16T20:51:03.715 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00035773100000000002 2024-09-16T20:51:03.715 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.715 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.716 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:03.716 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535123+0000", 2024-09-16T20:51:03.716 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0013836160000000001 2024-09-16T20:51:03.716 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.716 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.716 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:03.716 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538413+0000", 2024-09-16T20:51:03.716 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0032902510000000001 2024-09-16T20:51:03.716 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.716 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.716 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:03.716 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538470+0000", 2024-09-16T20:51:03.716 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6697999999999998e-05 2024-09-16T20:51:03.716 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.716 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.717 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:03.717 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538510+0000", 2024-09-16T20:51:03.717 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9988999999999999e-05 2024-09-16T20:51:03.717 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:03.717 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:03.717 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:03.717 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.717 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.717 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:12732 3.d 3:b752444a:::rbd_header.110e747f0779:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-16T20:51:03.717 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:51:03.717 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 118.88765492500001, 2024-09-16T20:51:03.717 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.065302467000000003, 2024-09-16T20:51:03.717 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:03.718 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:03.718 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:03.718 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:03.718 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:51:03.718 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:51:03.718 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 12732 2024-09-16T20:51:03.718 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.718 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:03.718 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.718 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:03.718 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:51:03.718 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:03.718 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.718 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.718 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:03.718 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:51:03.719 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:03.719 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.719 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.719 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:03.719 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768490+0000", 2024-09-16T20:51:03.719 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2859999999999998e-06 2024-09-16T20:51:03.719 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.719 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.719 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:03.719 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768501+0000", 2024-09-16T20:51:03.719 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1226999999999999e-05 2024-09-16T20:51:03.719 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.719 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.719 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:03.720 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768517+0000", 2024-09-16T20:51:03.720 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.592e-05 2024-09-16T20:51:03.720 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.720 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.720 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:03.720 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768528+0000", 2024-09-16T20:51:03.720 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0689999999999999e-05 2024-09-16T20:51:03.720 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.720 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.720 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:03.720 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768584+0000", 2024-09-16T20:51:03.720 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5889e-05 2024-09-16T20:51:03.720 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.720 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.721 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:03.721 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768720+0000", 2024-09-16T20:51:03.721 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000136718 2024-09-16T20:51:03.721 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.721 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.721 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:03.721 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.769066+0000", 2024-09-16T20:51:03.721 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000346013 2024-09-16T20:51:03.721 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.721 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.721 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:03.722 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833659+0000", 2024-09-16T20:51:03.722 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.064592822999999994 2024-09-16T20:51:03.722 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.722 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.722 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:03.722 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833719+0000", 2024-09-16T20:51:03.722 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9441000000000002e-05 2024-09-16T20:51:03.722 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.722 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.722 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:03.722 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833758+0000", 2024-09-16T20:51:03.722 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9122000000000003e-05 2024-09-16T20:51:03.722 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.722 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.722 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:03.723 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833790+0000", 2024-09-16T20:51:03.723 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2338000000000001e-05 2024-09-16T20:51:03.723 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:03.723 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:03.723 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:03.723 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.723 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.723 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.5089.0:12732 3.d e73/23 ondisk, result = 0)", 2024-09-16T20:51:03.723 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:51:03.723 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 118.88605275499999, 2024-09-16T20:51:03.723 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.063706456999999994, 2024-09-16T20:51:03.723 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:03.723 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:03.723 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:03.723 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:03.724 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.724 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:03.724 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:51:03.724 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:03.724 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.724 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.724 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:03.724 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:51:03.724 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:03.724 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.724 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.724 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:03.724 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770091+0000", 2024-09-16T20:51:03.724 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.53e-06 2024-09-16T20:51:03.724 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.725 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.725 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:03.725 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770104+0000", 2024-09-16T20:51:03.725 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2203e-05 2024-09-16T20:51:03.725 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.725 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.725 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:03.725 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833637+0000", 2024-09-16T20:51:03.725 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.063533201999999997 2024-09-16T20:51:03.725 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.725 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.725 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:03.725 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833655+0000", 2024-09-16T20:51:03.725 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8369e-05 2024-09-16T20:51:03.726 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.726 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.726 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:03.726 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833707+0000", 2024-09-16T20:51:03.726 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2061999999999997e-05 2024-09-16T20:51:03.726 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.726 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.726 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:03.726 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833715+0000", 2024-09-16T20:51:03.726 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5179999999999998e-06 2024-09-16T20:51:03.726 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.726 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.726 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:03.726 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833796+0000", 2024-09-16T20:51:03.727 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1573e-05 2024-09-16T20:51:03.727 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:03.727 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:03.727 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:03.727 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.727 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.727 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3286 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 245760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:03.727 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:51:03.727 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 73.614990817999995, 2024-09-16T20:51:03.727 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052523564000000002, 2024-09-16T20:51:03.727 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:03.727 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:03.727 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:03.727 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:03.727 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:51:03.727 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:03.728 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3286 2024-09-16T20:51:03.728 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.728 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:03.728 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.728 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:03.728 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:51:03.728 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:03.728 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.728 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.728 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:03.728 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:51:03.728 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:03.728 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.728 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.728 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:03.729 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:51:03.729 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3900000000000001e-07 2024-09-16T20:51:03.729 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.729 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.729 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:03.729 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041153+0000", 2024-09-16T20:51:03.729 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.119e-06 2024-09-16T20:51:03.729 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.729 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.729 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:03.729 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041156+0000", 2024-09-16T20:51:03.729 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0000000000000001e-06 2024-09-16T20:51:03.730 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.730 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.730 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:03.730 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041159+0000", 2024-09-16T20:51:03.730 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5859999999999999e-06 2024-09-16T20:51:03.730 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.730 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.730 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:03.730 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093529+0000", 2024-09-16T20:51:03.730 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052370591000000001 2024-09-16T20:51:03.730 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.730 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.730 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:03.730 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093570+0000", 2024-09-16T20:51:03.731 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0161999999999999e-05 2024-09-16T20:51:03.731 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.731 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.731 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:03.731 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093675+0000", 2024-09-16T20:51:03.731 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010586699999999999 2024-09-16T20:51:03.731 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:03.731 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:03.731 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:03.731 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.731 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.731 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3287 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 250880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:03.731 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:51:03.731 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 73.614980768999999, 2024-09-16T20:51:03.732 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052781203999999998, 2024-09-16T20:51:03.732 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:03.732 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:03.732 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:03.732 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:03.732 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:51:03.732 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:03.732 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3287 2024-09-16T20:51:03.732 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.732 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:03.732 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.732 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:03.732 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:51:03.732 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:03.733 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.733 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.733 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:03.733 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:51:03.733 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:03.733 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.733 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.733 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:03.733 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:51:03.733 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1e-07 2024-09-16T20:51:03.733 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.733 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.733 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:03.733 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041163+0000", 2024-09-16T20:51:03.734 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0690000000000001e-06 2024-09-16T20:51:03.734 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.734 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.734 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:03.734 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041166+0000", 2024-09-16T20:51:03.734 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.982e-06 2024-09-16T20:51:03.734 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.734 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.734 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:03.734 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041169+0000", 2024-09-16T20:51:03.734 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.593e-06 2024-09-16T20:51:03.734 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.734 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.734 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:03.734 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093709+0000", 2024-09-16T20:51:03.734 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052540492000000001 2024-09-16T20:51:03.735 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.735 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.735 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:03.735 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093792+0000", 2024-09-16T20:51:03.735 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2436999999999997e-05 2024-09-16T20:51:03.735 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.735 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.735 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:03.735 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093943+0000", 2024-09-16T20:51:03.735 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000151421 2024-09-16T20:51:03.735 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:03.735 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:03.735 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:03.735 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.735 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.735 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3288 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 256000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:03.736 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:51:03.736 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 73.614970896000003, 2024-09-16T20:51:03.736 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052928269, 2024-09-16T20:51:03.736 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:03.736 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:03.736 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:03.736 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:03.736 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:51:03.736 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:03.736 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3288 2024-09-16T20:51:03.736 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.736 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:03.736 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.736 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:03.736 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:51:03.737 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:03.737 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.737 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.737 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:03.737 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:51:03.737 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:03.737 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.737 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.737 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:03.737 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:51:03.737 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2700000000000001e-07 2024-09-16T20:51:03.737 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.737 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.737 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:03.738 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041173+0000", 2024-09-16T20:51:03.738 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.113e-06 2024-09-16T20:51:03.738 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.738 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.738 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:03.738 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041176+0000", 2024-09-16T20:51:03.738 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9799999999999998e-06 2024-09-16T20:51:03.738 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.738 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.738 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:03.738 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041179+0000", 2024-09-16T20:51:03.738 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.667e-06 2024-09-16T20:51:03.738 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.738 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.738 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:03.739 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093957+0000", 2024-09-16T20:51:03.739 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052778703000000003 2024-09-16T20:51:03.739 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.739 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.739 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:03.739 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093999+0000", 2024-09-16T20:51:03.739 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1300000000000001e-05 2024-09-16T20:51:03.739 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.739 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.739 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:03.739 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094100+0000", 2024-09-16T20:51:03.739 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101279 2024-09-16T20:51:03.739 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:03.739 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:03.739 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:03.739 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.740 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.740 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3289 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 261120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:03.740 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:51:03.740 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 73.6149609, 2024-09-16T20:51:03.740 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053182976, 2024-09-16T20:51:03.740 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:03.740 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:03.740 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:03.740 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:03.740 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:51:03.740 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:03.740 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3289 2024-09-16T20:51:03.740 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.740 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:03.740 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.741 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:03.741 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:51:03.741 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:03.741 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.741 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.741 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:03.741 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:51:03.741 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:03.741 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.741 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.741 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:03.741 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:51:03.741 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2999999999999999e-07 2024-09-16T20:51:03.741 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.741 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.742 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:03.742 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041183+0000", 2024-09-16T20:51:03.742 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0720000000000001e-06 2024-09-16T20:51:03.742 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.742 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.742 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:03.742 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041186+0000", 2024-09-16T20:51:03.742 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9869999999999999e-06 2024-09-16T20:51:03.742 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.742 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.742 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:03.742 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041189+0000", 2024-09-16T20:51:03.742 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.587e-06 2024-09-16T20:51:03.742 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.742 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.742 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:03.743 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094136+0000", 2024-09-16T20:51:03.743 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052947270999999997 2024-09-16T20:51:03.743 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.743 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.743 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:03.743 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094213+0000", 2024-09-16T20:51:03.743 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7552000000000006e-05 2024-09-16T20:51:03.743 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.743 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.743 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:03.743 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094365+0000", 2024-09-16T20:51:03.743 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00015127700000000001 2024-09-16T20:51:03.743 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:03.743 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:03.744 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:03.744 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.744 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.744 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:2196 3.d 3:b1fe01a0:::rbd_data.110e7ef3004.0000000000000000:2d [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e136)", 2024-09-16T20:51:03.744 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:22.739057+0000", 2024-09-16T20:51:03.744 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 40.917086101000002, 2024-09-16T20:51:03.744 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11999464899999999, 2024-09-16T20:51:03.744 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:03.744 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:03.744 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:03.744 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:03.744 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:03.744 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:03.745 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2196 2024-09-16T20:51:03.745 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.745 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:03.745 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.745 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:03.745 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.739057+0000", 2024-09-16T20:51:03.745 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:03.745 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.746 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.746 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:03.746 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.739057+0000", 2024-09-16T20:51:03.746 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:03.746 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.747 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.747 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:03.747 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.739057+0000", 2024-09-16T20:51:03.747 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1399999999999997e-07 2024-09-16T20:51:03.747 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.747 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.748 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:03.748 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.739060+0000", 2024-09-16T20:51:03.748 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6019999999999998e-06 2024-09-16T20:51:03.748 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.748 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.748 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:03.748 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.858755+0000", 2024-09-16T20:51:03.749 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.119695151 2024-09-16T20:51:03.749 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.749 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.749 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:03.749 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.858778+0000", 2024-09-16T20:51:03.749 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3184999999999999e-05 2024-09-16T20:51:03.749 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.750 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.750 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:03.750 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.858865+0000", 2024-09-16T20:51:03.750 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.7330000000000006e-05 2024-09-16T20:51:03.750 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.750 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.750 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:03.751 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.858952+0000", 2024-09-16T20:51:03.751 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6803000000000002e-05 2024-09-16T20:51:03.751 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.751 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.751 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:03.751 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.859051+0000", 2024-09-16T20:51:03.752 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8963999999999996e-05 2024-09-16T20:51:03.752 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:03.752 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:03.752 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:03.752 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.752 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.753 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6061.0:5401 3.13 3:c8554348:::rbd_data.110e16eafbd.0000000000000011:head [read 577536~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-16T20:51:03.753 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:27.761536+0000", 2024-09-16T20:51:03.753 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 35.894606287999999, 2024-09-16T20:51:03.753 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.003742712, 2024-09-16T20:51:03.753 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:03.753 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:03.754 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:51:03.754 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:03.754 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6061", 2024-09-16T20:51:03.754 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3930128440", 2024-09-16T20:51:03.754 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5401 2024-09-16T20:51:03.754 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.754 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:03.754 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.754 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:03.754 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761536+0000", 2024-09-16T20:51:03.755 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:03.755 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.755 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.755 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:03.755 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761536+0000", 2024-09-16T20:51:03.755 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:03.755 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.755 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.755 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:03.755 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761537+0000", 2024-09-16T20:51:03.755 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2900000000000003e-07 2024-09-16T20:51:03.755 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.755 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.755 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:03.756 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761540+0000", 2024-09-16T20:51:03.756 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2320000000000001e-06 2024-09-16T20:51:03.756 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.756 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.756 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:03.756 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761548+0000", 2024-09-16T20:51:03.756 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1289999999999996e-06 2024-09-16T20:51:03.756 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.756 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.756 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:03.756 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761554+0000", 2024-09-16T20:51:03.756 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0820000000000004e-06 2024-09-16T20:51:03.756 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.756 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.757 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:03.757 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761621+0000", 2024-09-16T20:51:03.757 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6266000000000006e-05 2024-09-16T20:51:03.757 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.757 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.757 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:03.757 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761626+0000", 2024-09-16T20:51:03.757 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.614e-06 2024-09-16T20:51:03.757 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.757 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.757 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:03.757 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:28.765163+0000", 2024-09-16T20:51:03.758 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.003536416 2024-09-16T20:51:03.758 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.758 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.758 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:03.758 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:28.765279+0000", 2024-09-16T20:51:03.758 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000116344 2024-09-16T20:51:03.758 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:03.758 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:03.758 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:03.758 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.758 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.758 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6061.0:5425 3.13 3:c8554348:::rbd_data.110e16eafbd.0000000000000011:head [read 602112~28672] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-16T20:51:03.758 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:27.767049+0000", 2024-09-16T20:51:03.758 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 35.889093629000001, 2024-09-16T20:51:03.758 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.99847897900000004, 2024-09-16T20:51:03.758 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:03.759 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:03.759 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:51:03.759 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:03.759 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6061", 2024-09-16T20:51:03.759 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3930128440", 2024-09-16T20:51:03.759 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5425 2024-09-16T20:51:03.759 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.759 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:03.759 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.759 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:03.759 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767049+0000", 2024-09-16T20:51:03.759 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:03.759 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.759 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.759 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:03.759 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767049+0000", 2024-09-16T20:51:03.760 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:03.760 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.760 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.760 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:03.760 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767050+0000", 2024-09-16T20:51:03.760 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6499999999999999e-07 2024-09-16T20:51:03.760 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.760 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.760 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:03.760 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767052+0000", 2024-09-16T20:51:03.760 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7350000000000001e-06 2024-09-16T20:51:03.760 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.760 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.760 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:03.760 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767060+0000", 2024-09-16T20:51:03.761 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5109999999999997e-06 2024-09-16T20:51:03.761 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.761 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.761 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:03.761 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767066+0000", 2024-09-16T20:51:03.761 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8139999999999997e-06 2024-09-16T20:51:03.761 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.761 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.761 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:03.761 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767128+0000", 2024-09-16T20:51:03.761 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2174999999999993e-05 2024-09-16T20:51:03.761 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.761 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.761 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:03.762 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767133+0000", 2024-09-16T20:51:03.762 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2140000000000004e-06 2024-09-16T20:51:03.762 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.762 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.762 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:03.762 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:28.765441+0000", 2024-09-16T20:51:03.762 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.99830808599999998 2024-09-16T20:51:03.762 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.762 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.762 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:03.762 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:28.765528+0000", 2024-09-16T20:51:03.762 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6879000000000006e-05 2024-09-16T20:51:03.762 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:03.762 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:03.762 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:03.763 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.763 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.763 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:25585 3.10 3:0c7c5b23:::rbd_header.110ebf0678ad:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:51:03.763 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:32.877132+0000", 2024-09-16T20:51:03.763 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 30.77901108, 2024-09-16T20:51:03.763 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054096699999999998, 2024-09-16T20:51:03.763 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:03.763 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:03.763 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:03.763 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:03.763 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:51:03.763 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:03.763 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 25585 2024-09-16T20:51:03.763 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.763 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:03.764 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.764 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:03.764 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.877132+0000", 2024-09-16T20:51:03.764 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:03.764 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.764 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.764 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:03.764 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.877132+0000", 2024-09-16T20:51:03.764 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:03.764 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.764 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.765 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:03.765 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931072+0000", 2024-09-16T20:51:03.765 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053939931000000003 2024-09-16T20:51:03.765 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.765 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.765 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:03.765 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931076+0000", 2024-09-16T20:51:03.765 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2860000000000001e-06 2024-09-16T20:51:03.765 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.765 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.765 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:03.765 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931085+0000", 2024-09-16T20:51:03.765 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.3370000000000006e-06 2024-09-16T20:51:03.765 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.766 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.766 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:03.766 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931092+0000", 2024-09-16T20:51:03.766 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9970000000000001e-06 2024-09-16T20:51:03.766 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.766 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.766 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:03.766 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931137+0000", 2024-09-16T20:51:03.766 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5160000000000001e-05 2024-09-16T20:51:03.766 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.766 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.766 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:03.766 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931178+0000", 2024-09-16T20:51:03.766 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0379000000000001e-05 2024-09-16T20:51:03.766 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.767 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.767 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:03.767 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931228+0000", 2024-09-16T20:51:03.767 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0609999999999998e-05 2024-09-16T20:51:03.767 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:03.767 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:03.767 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:03.767 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.767 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.767 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:26679 3.c 3:33e2f78f:::rbd_header.110e16eafbd:head [watch ping cookie 140213436013408 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e153)", 2024-09-16T20:51:03.767 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:41.440287+0000", 2024-09-16T20:51:03.767 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 22.215855205, 2024-09-16T20:51:03.767 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.13295633600000001, 2024-09-16T20:51:03.768 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:03.768 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:03.768 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:03.768 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:03.768 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:51:03.768 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:03.768 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 26679 2024-09-16T20:51:03.768 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.768 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:03.768 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.768 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:03.768 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440287+0000", 2024-09-16T20:51:03.769 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:03.769 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.769 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.769 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:03.769 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440287+0000", 2024-09-16T20:51:03.769 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:03.769 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.769 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.769 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:03.769 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440288+0000", 2024-09-16T20:51:03.769 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0900000000000001e-07 2024-09-16T20:51:03.769 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.769 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.770 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:03.770 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440292+0000", 2024-09-16T20:51:03.770 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3749999999999996e-06 2024-09-16T20:51:03.770 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.770 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.770 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:03.770 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440299+0000", 2024-09-16T20:51:03.770 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8249999999999999e-06 2024-09-16T20:51:03.770 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.770 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.770 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:03.770 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440307+0000", 2024-09-16T20:51:03.770 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4800000000000004e-06 2024-09-16T20:51:03.770 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.771 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.771 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:03.771 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440451+0000", 2024-09-16T20:51:03.771 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014414500000000001 2024-09-16T20:51:03.771 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.771 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.771 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:03.771 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440465+0000", 2024-09-16T20:51:03.771 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.402e-05 2024-09-16T20:51:03.771 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.772 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.772 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:03.772 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.573043+0000", 2024-09-16T20:51:03.772 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.13257855399999999 2024-09-16T20:51:03.772 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.772 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.772 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:03.772 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.573130+0000", 2024-09-16T20:51:03.772 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6168999999999994e-05 2024-09-16T20:51:03.772 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:03.772 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:03.772 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:03.773 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.573244+0000", 2024-09-16T20:51:03.773 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000114159 2024-09-16T20:51:03.773 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:03.773 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:03.773 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:03.773 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:03.773 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:03.773 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:03.775 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:51:03.776 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:03.773+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:03.875 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:03.874+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:03.945 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:03.945 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:51:03.945 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:51:03.945 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:03.955 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:51:03.976 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:03.974+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:04.077 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:04.076+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:04.111 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:04.111 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:51:04.111 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:51:04.112 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:51:04.112 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:04.121 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:51:04.178 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:04.177+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:04.240 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:04.240 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:51:04.240 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:51:04.240 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:51:04.240 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.240 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3714 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1480704~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:04.240 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:51:04.240 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 161.59922046599999, 2024-09-16T20:51:04.240 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050152552000000003, 2024-09-16T20:51:04.241 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:04.241 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:04.241 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:04.241 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:04.241 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:04.241 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:04.241 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3714 2024-09-16T20:51:04.241 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.241 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:04.241 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.241 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:04.241 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:51:04.241 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:04.241 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.242 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.242 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:04.242 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:51:04.242 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:04.242 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.242 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.242 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:04.242 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:51:04.242 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0499999999999999e-07 2024-09-16T20:51:04.242 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.242 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.242 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:04.243 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639007+0000", 2024-09-16T20:51:04.243 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5340000000000001e-06 2024-09-16T20:51:04.243 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.243 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.243 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:04.243 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639013+0000", 2024-09-16T20:51:04.243 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3260000000000003e-06 2024-09-16T20:51:04.243 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.243 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.243 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:04.243 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639018+0000", 2024-09-16T20:51:04.243 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2850000000000002e-06 2024-09-16T20:51:04.243 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.243 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.243 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:04.244 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689026+0000", 2024-09-16T20:51:04.244 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.05000798 2024-09-16T20:51:04.244 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.244 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.244 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:04.244 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689077+0000", 2024-09-16T20:51:04.244 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1063999999999998e-05 2024-09-16T20:51:04.244 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.244 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.244 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:04.244 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689158+0000", 2024-09-16T20:51:04.244 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1057999999999995e-05 2024-09-16T20:51:04.244 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:04.244 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:04.244 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:04.245 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.245 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.245 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3716 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1488896~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:04.245 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:51:04.245 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 161.59920265599999, 2024-09-16T20:51:04.245 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050280142, 2024-09-16T20:51:04.245 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:04.245 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:04.245 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:04.245 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:04.246 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:04.246 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:04.246 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3716 2024-09-16T20:51:04.246 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.246 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:04.246 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.246 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:04.246 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:51:04.246 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:04.246 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.246 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.246 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:04.247 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:51:04.247 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:04.247 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.247 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.247 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:04.247 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639024+0000", 2024-09-16T20:51:04.247 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9999999999999997e-07 2024-09-16T20:51:04.247 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.247 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.247 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:04.247 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639026+0000", 2024-09-16T20:51:04.247 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.781e-06 2024-09-16T20:51:04.247 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.248 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.248 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:04.248 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639032+0000", 2024-09-16T20:51:04.248 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7780000000000004e-06 2024-09-16T20:51:04.248 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.248 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.248 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:04.248 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639036+0000", 2024-09-16T20:51:04.248 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6909999999999996e-06 2024-09-16T20:51:04.248 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.248 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.248 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:04.249 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689191+0000", 2024-09-16T20:51:04.249 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050155284000000001 2024-09-16T20:51:04.249 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.249 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.249 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:04.249 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689242+0000", 2024-09-16T20:51:04.249 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0662999999999999e-05 2024-09-16T20:51:04.249 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.249 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.249 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:04.249 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689304+0000", 2024-09-16T20:51:04.249 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1345000000000003e-05 2024-09-16T20:51:04.249 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:04.249 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:04.250 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:04.250 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.250 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.250 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3719 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1497088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:04.250 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:51:04.250 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 161.59917537800001, 2024-09-16T20:51:04.250 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050399715999999997, 2024-09-16T20:51:04.250 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:04.250 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:04.250 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:04.250 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:04.250 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:04.250 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:04.250 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3719 2024-09-16T20:51:04.250 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.251 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:04.251 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.251 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:04.251 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:51:04.251 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:04.251 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.251 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.251 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:04.251 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:51:04.251 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:04.251 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.252 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.252 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:04.252 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:51:04.252 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9700000000000002e-07 2024-09-16T20:51:04.252 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.252 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.252 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:04.252 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639053+0000", 2024-09-16T20:51:04.252 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2900000000000001e-06 2024-09-16T20:51:04.252 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.252 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.252 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:04.252 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639060+0000", 2024-09-16T20:51:04.253 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7750000000000004e-06 2024-09-16T20:51:04.253 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.253 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.253 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:04.253 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639066+0000", 2024-09-16T20:51:04.253 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0639999999999999e-06 2024-09-16T20:51:04.253 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.253 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.253 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:04.253 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689337+0000", 2024-09-16T20:51:04.253 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050270784999999998 2024-09-16T20:51:04.253 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.253 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.254 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:04.254 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689392+0000", 2024-09-16T20:51:04.254 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4920999999999998e-05 2024-09-16T20:51:04.254 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.254 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.254 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:04.254 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689450+0000", 2024-09-16T20:51:04.254 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8483999999999998e-05 2024-09-16T20:51:04.254 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:04.254 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:04.254 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:04.254 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.254 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.254 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3721 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1505280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:04.256 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:51:04.256 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 161.59915224599999, 2024-09-16T20:51:04.256 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050513608000000002, 2024-09-16T20:51:04.256 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:04.256 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:04.257 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:04.257 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:04.257 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:04.257 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:04.257 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3721 2024-09-16T20:51:04.257 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.257 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:04.257 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.257 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:04.257 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:51:04.257 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:04.257 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.258 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.258 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:04.258 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:51:04.258 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:04.258 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.258 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.258 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:04.258 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:51:04.258 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7e-07 2024-09-16T20:51:04.258 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.258 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.258 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:04.258 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639076+0000", 2024-09-16T20:51:04.258 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8899999999999999e-06 2024-09-16T20:51:04.258 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.259 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.259 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:04.259 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639083+0000", 2024-09-16T20:51:04.259 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1210000000000001e-06 2024-09-16T20:51:04.259 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.259 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.259 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:04.259 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639089+0000", 2024-09-16T20:51:04.259 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.643e-06 2024-09-16T20:51:04.259 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.259 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.259 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:04.259 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689474+0000", 2024-09-16T20:51:04.259 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050385679000000003 2024-09-16T20:51:04.260 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.260 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.260 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:04.260 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689533+0000", 2024-09-16T20:51:04.260 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8303000000000001e-05 2024-09-16T20:51:04.260 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.260 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.260 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:04.260 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689587+0000", 2024-09-16T20:51:04.260 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4601999999999997e-05 2024-09-16T20:51:04.260 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:04.260 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:04.260 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:04.260 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.260 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.261 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3723 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1513472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:04.261 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:51:04.261 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 161.59912144200001, 2024-09-16T20:51:04.261 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050699691999999998, 2024-09-16T20:51:04.261 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:04.261 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:04.261 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:04.261 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:04.261 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:04.261 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:04.261 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3723 2024-09-16T20:51:04.261 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.261 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:04.261 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.262 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:04.262 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:51:04.262 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:04.262 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.262 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.262 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:04.262 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:51:04.262 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:04.262 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.262 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.262 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:04.262 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:51:04.262 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3399999999999999e-07 2024-09-16T20:51:04.262 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.262 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.263 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:04.263 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639107+0000", 2024-09-16T20:51:04.263 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3860000000000001e-06 2024-09-16T20:51:04.263 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.263 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.263 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:04.263 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639114+0000", 2024-09-16T20:51:04.263 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9800000000000001e-06 2024-09-16T20:51:04.263 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.263 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.263 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:04.263 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639129+0000", 2024-09-16T20:51:04.263 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4887e-05 2024-09-16T20:51:04.263 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.263 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.263 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:04.264 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689625+0000", 2024-09-16T20:51:04.264 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050495848000000003 2024-09-16T20:51:04.264 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.264 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.264 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:04.264 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689720+0000", 2024-09-16T20:51:04.264 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4821000000000006e-05 2024-09-16T20:51:04.264 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.264 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.264 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:04.264 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689804+0000", 2024-09-16T20:51:04.264 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4235999999999998e-05 2024-09-16T20:51:04.264 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:04.264 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:04.265 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:04.265 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.265 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.265 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3725 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1521664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:04.265 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:51:04.265 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 161.599087846, 2024-09-16T20:51:04.265 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050797660000000001, 2024-09-16T20:51:04.265 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:04.265 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:04.265 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:04.265 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:04.265 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:04.265 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:04.266 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3725 2024-09-16T20:51:04.266 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.266 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:04.266 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.266 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:04.266 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:51:04.266 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:04.266 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.266 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.266 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:04.266 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:51:04.266 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:04.266 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.266 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.267 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:04.267 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639139+0000", 2024-09-16T20:51:04.267 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8999999999999997e-07 2024-09-16T20:51:04.267 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.267 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.267 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:04.267 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639141+0000", 2024-09-16T20:51:04.267 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.182e-06 2024-09-16T20:51:04.267 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.267 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.267 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:04.267 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639154+0000", 2024-09-16T20:51:04.267 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3626e-05 2024-09-16T20:51:04.267 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.267 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.267 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:04.268 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639160+0000", 2024-09-16T20:51:04.268 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6810000000000003e-06 2024-09-16T20:51:04.268 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.268 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.268 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:04.268 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689831+0000", 2024-09-16T20:51:04.268 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050670907000000001 2024-09-16T20:51:04.268 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.268 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.268 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:04.268 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689882+0000", 2024-09-16T20:51:04.268 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0445999999999998e-05 2024-09-16T20:51:04.268 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.268 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.269 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:04.269 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689936+0000", 2024-09-16T20:51:04.269 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4327999999999999e-05 2024-09-16T20:51:04.269 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:04.269 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:04.269 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:04.269 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.269 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.269 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3727 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1529856~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:04.269 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:51:04.269 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 161.599052269, 2024-09-16T20:51:04.269 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051016922999999999, 2024-09-16T20:51:04.270 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:04.270 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:04.270 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:04.270 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:04.270 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:04.270 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:04.270 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3727 2024-09-16T20:51:04.270 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.270 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:04.270 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.270 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:04.270 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:51:04.270 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:04.270 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.270 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.271 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:04.271 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:51:04.271 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:04.271 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.271 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.271 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:04.271 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:51:04.271 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9999999999999998e-07 2024-09-16T20:51:04.271 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.271 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.271 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:04.272 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639183+0000", 2024-09-16T20:51:04.272 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2399999999999996e-06 2024-09-16T20:51:04.272 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.272 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.272 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:04.272 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639195+0000", 2024-09-16T20:51:04.272 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1113e-05 2024-09-16T20:51:04.272 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.272 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.272 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:04.272 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639206+0000", 2024-09-16T20:51:04.272 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1503e-05 2024-09-16T20:51:04.273 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.273 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.273 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:04.273 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689969+0000", 2024-09-16T20:51:04.273 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050762676 2024-09-16T20:51:04.273 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.273 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.273 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:04.273 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690071+0000", 2024-09-16T20:51:04.273 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102163 2024-09-16T20:51:04.273 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.273 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.273 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:04.273 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690191+0000", 2024-09-16T20:51:04.273 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000119728 2024-09-16T20:51:04.274 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:04.274 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:04.274 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:04.274 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.274 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.274 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3730 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1538048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:04.274 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:51:04.274 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 161.599010706, 2024-09-16T20:51:04.274 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051131467999999999, 2024-09-16T20:51:04.274 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:04.274 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:04.274 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:04.274 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:04.275 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:04.275 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:04.275 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3730 2024-09-16T20:51:04.275 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.275 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:04.275 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.275 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:04.275 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:51:04.275 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:04.275 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.275 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.275 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:04.275 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:51:04.276 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:04.276 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.276 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.276 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:04.276 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639216+0000", 2024-09-16T20:51:04.276 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.39e-07 2024-09-16T20:51:04.276 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.276 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.276 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:04.276 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639218+0000", 2024-09-16T20:51:04.276 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.537e-06 2024-09-16T20:51:04.276 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.276 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.277 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:04.277 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639229+0000", 2024-09-16T20:51:04.277 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1122e-05 2024-09-16T20:51:04.277 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.277 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.278 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:04.278 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639237+0000", 2024-09-16T20:51:04.278 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8509999999999999e-06 2024-09-16T20:51:04.278 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.278 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.278 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:04.278 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690227+0000", 2024-09-16T20:51:04.278 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050990147 2024-09-16T20:51:04.278 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.278 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.278 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:04.279 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690278+0000", 2024-09-16T20:51:04.279 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0223999999999999e-05 2024-09-16T20:51:04.279 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.279 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.279 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:04.279 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690347+0000", 2024-09-16T20:51:04.279 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9147999999999996e-05 2024-09-16T20:51:04.279 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:04.279 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:04.279 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:04.279 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.279 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.279 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3731 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1546240~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:04.279 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:51:04.280 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 161.598979998, 2024-09-16T20:51:04.280 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051276744999999999, 2024-09-16T20:51:04.280 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:04.280 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:04.280 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:04.280 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:04.280 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:04.280 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:04.280 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3731 2024-09-16T20:51:04.280 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.280 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:04.280 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.280 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:04.281 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:51:04.281 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:04.281 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.281 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.281 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:04.281 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:51:04.281 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:04.281 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.281 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.281 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:04.281 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:51:04.281 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9900000000000001e-07 2024-09-16T20:51:04.282 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.282 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.282 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:04.282 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639249+0000", 2024-09-16T20:51:04.282 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3630000000000001e-06 2024-09-16T20:51:04.282 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.282 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.282 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:04.283 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639255+0000", 2024-09-16T20:51:04.283 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4960000000000001e-06 2024-09-16T20:51:04.283 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.283 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.283 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:04.283 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639261+0000", 2024-09-16T20:51:04.283 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2839999999999997e-06 2024-09-16T20:51:04.283 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.283 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.283 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:04.283 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690375+0000", 2024-09-16T20:51:04.283 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051114684000000001 2024-09-16T20:51:04.283 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.283 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.284 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:04.284 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690425+0000", 2024-09-16T20:51:04.284 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9379999999999998e-05 2024-09-16T20:51:04.284 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.284 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.284 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:04.284 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690523+0000", 2024-09-16T20:51:04.284 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8139000000000003e-05 2024-09-16T20:51:04.284 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:04.284 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:04.284 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:04.284 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.285 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.285 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3732 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1554432~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:04.285 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:51:04.285 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 161.598934216, 2024-09-16T20:51:04.285 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051380096, 2024-09-16T20:51:04.285 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:04.285 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:04.285 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:04.285 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:04.285 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:04.285 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:04.285 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3732 2024-09-16T20:51:04.285 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.286 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:04.286 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.286 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:04.286 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:51:04.286 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:04.286 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.286 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.286 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:04.286 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:51:04.286 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:04.286 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.286 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.286 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:04.287 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:51:04.287 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6100000000000001e-07 2024-09-16T20:51:04.287 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.287 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.287 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:04.287 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639295+0000", 2024-09-16T20:51:04.287 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.447e-06 2024-09-16T20:51:04.287 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.287 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.287 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:04.287 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639302+0000", 2024-09-16T20:51:04.287 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0570000000000004e-06 2024-09-16T20:51:04.287 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.287 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.287 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:04.288 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639307+0000", 2024-09-16T20:51:04.288 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1429999999999997e-06 2024-09-16T20:51:04.288 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.288 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.288 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:04.288 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690555+0000", 2024-09-16T20:51:04.288 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051247963000000001 2024-09-16T20:51:04.288 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.288 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.288 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:04.288 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690602+0000", 2024-09-16T20:51:04.288 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7602000000000003e-05 2024-09-16T20:51:04.288 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.288 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.289 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:04.289 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690672+0000", 2024-09-16T20:51:04.289 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9423000000000003e-05 2024-09-16T20:51:04.289 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:04.289 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:04.289 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:04.289 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.289 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.289 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3734 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1562624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:04.289 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:51:04.289 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 161.598912566, 2024-09-16T20:51:04.289 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051490610999999999, 2024-09-16T20:51:04.289 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:04.289 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:04.290 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:04.290 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:04.290 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:04.290 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:04.290 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3734 2024-09-16T20:51:04.290 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.290 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:04.290 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.290 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:04.290 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:51:04.290 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:04.290 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.290 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.290 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:04.291 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:51:04.291 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:04.291 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.291 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.291 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:04.291 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639314+0000", 2024-09-16T20:51:04.291 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4400000000000001e-07 2024-09-16T20:51:04.291 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.291 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.292 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:04.292 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639316+0000", 2024-09-16T20:51:04.292 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1459999999999999e-06 2024-09-16T20:51:04.292 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.292 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.292 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:04.292 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639322+0000", 2024-09-16T20:51:04.292 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7610000000000004e-06 2024-09-16T20:51:04.292 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.292 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.292 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:04.292 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639327+0000", 2024-09-16T20:51:04.292 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.186e-06 2024-09-16T20:51:04.292 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.293 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.293 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:04.293 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690698+0000", 2024-09-16T20:51:04.293 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051371223000000001 2024-09-16T20:51:04.293 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.293 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.293 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:04.293 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690748+0000", 2024-09-16T20:51:04.293 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0192999999999999e-05 2024-09-16T20:51:04.293 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.293 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.293 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:04.293 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690804+0000", 2024-09-16T20:51:04.293 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5658000000000001e-05 2024-09-16T20:51:04.294 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:04.294 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:04.294 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:04.294 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.294 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.294 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3736 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1570816~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:04.294 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:51:04.294 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 161.598893307, 2024-09-16T20:51:04.294 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051596528000000003, 2024-09-16T20:51:04.294 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:04.294 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:04.294 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:04.294 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:04.294 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:04.294 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:04.295 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3736 2024-09-16T20:51:04.295 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.295 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:04.295 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.295 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:04.295 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:51:04.295 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:04.295 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.295 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.295 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:04.295 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:51:04.296 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:04.296 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.296 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.296 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:04.296 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:51:04.296 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9999999999999999e-07 2024-09-16T20:51:04.296 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.296 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.296 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:04.296 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639335+0000", 2024-09-16T20:51:04.296 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.635e-06 2024-09-16T20:51:04.296 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.296 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.296 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:04.297 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639340+0000", 2024-09-16T20:51:04.297 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5840000000000002e-06 2024-09-16T20:51:04.297 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.297 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.297 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:04.297 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639346+0000", 2024-09-16T20:51:04.297 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1190000000000002e-06 2024-09-16T20:51:04.297 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.297 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.297 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:04.297 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690817+0000", 2024-09-16T20:51:04.297 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051470721999999997 2024-09-16T20:51:04.297 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.297 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.297 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:04.298 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690875+0000", 2024-09-16T20:51:04.298 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7441000000000001e-05 2024-09-16T20:51:04.298 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.298 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.298 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:04.298 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690929+0000", 2024-09-16T20:51:04.298 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4727e-05 2024-09-16T20:51:04.298 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:04.298 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:04.298 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:04.298 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.298 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.298 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5434 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:04.298 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:51:04.298 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 161.373631741, 2024-09-16T20:51:04.298 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099755831000000003, 2024-09-16T20:51:04.299 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:04.299 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:04.299 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:04.299 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:04.299 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:04.299 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:04.299 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5434 2024-09-16T20:51:04.299 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.299 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:04.299 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.299 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:04.299 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:51:04.299 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:04.299 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.299 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.300 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:04.300 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:51:04.300 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:04.300 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.300 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.300 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:04.300 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864595+0000", 2024-09-16T20:51:04.300 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1099999999999995e-07 2024-09-16T20:51:04.300 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.300 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.300 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:04.300 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864597+0000", 2024-09-16T20:51:04.300 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3989999999999998e-06 2024-09-16T20:51:04.300 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.301 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.301 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:04.301 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864603+0000", 2024-09-16T20:51:04.301 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9309999999999996e-06 2024-09-16T20:51:04.301 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.301 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.301 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:04.301 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864607+0000", 2024-09-16T20:51:04.301 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6200000000000001e-06 2024-09-16T20:51:04.301 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.301 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.301 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:04.301 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964164+0000", 2024-09-16T20:51:04.301 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099557162000000005 2024-09-16T20:51:04.302 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.302 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.302 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:04.302 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964244+0000", 2024-09-16T20:51:04.302 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9597000000000004e-05 2024-09-16T20:51:04.302 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.302 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.302 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:04.302 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964350+0000", 2024-09-16T20:51:04.302 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010641099999999999 2024-09-16T20:51:04.302 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:04.302 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:04.302 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:04.302 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.302 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.303 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5435 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 20480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:04.303 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:51:04.303 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 161.37360191900001, 2024-09-16T20:51:04.303 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100897102, 2024-09-16T20:51:04.303 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:04.303 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:04.303 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:04.303 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:04.303 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:04.303 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:04.303 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5435 2024-09-16T20:51:04.303 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.303 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:04.304 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.304 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:04.304 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:51:04.304 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:04.304 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.304 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.304 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:04.304 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:51:04.304 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:04.304 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.304 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.304 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:04.304 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:51:04.304 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8000000000000001e-07 2024-09-16T20:51:04.305 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.305 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.305 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:04.305 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864626+0000", 2024-09-16T20:51:04.305 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.745e-06 2024-09-16T20:51:04.305 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.305 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.305 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:04.305 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864633+0000", 2024-09-16T20:51:04.305 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3339999999999998e-06 2024-09-16T20:51:04.305 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.305 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.305 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:04.305 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864636+0000", 2024-09-16T20:51:04.305 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1200000000000002e-06 2024-09-16T20:51:04.306 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.306 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.306 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:04.306 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965299+0000", 2024-09-16T20:51:04.306 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10066352000000001 2024-09-16T20:51:04.306 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.306 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.306 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:04.306 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965390+0000", 2024-09-16T20:51:04.306 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0894000000000001e-05 2024-09-16T20:51:04.306 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.306 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.306 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:04.306 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965521+0000", 2024-09-16T20:51:04.306 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013110899999999999 2024-09-16T20:51:04.307 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:04.307 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:04.307 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:04.307 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.307 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.307 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:152 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 327680~32768 out=32768b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:51:04.307 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:51:04.307 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 160.075104977, 2024-09-16T20:51:04.307 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11869405800000001, 2024-09-16T20:51:04.307 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:04.307 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:04.307 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:04.307 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:04.307 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:51:04.308 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:51:04.308 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 152 2024-09-16T20:51:04.308 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.308 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:04.308 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.308 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:04.308 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:51:04.308 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:04.308 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.308 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.308 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:04.308 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:51:04.308 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:04.309 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.309 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.309 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:04.309 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163122+0000", 2024-09-16T20:51:04.309 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6400000000000003e-07 2024-09-16T20:51:04.309 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.309 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.309 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:04.309 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163124+0000", 2024-09-16T20:51:04.309 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.565e-06 2024-09-16T20:51:04.309 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.309 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.310 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:04.310 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163136+0000", 2024-09-16T20:51:04.310 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1717e-05 2024-09-16T20:51:04.310 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.310 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.310 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:04.310 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163142+0000", 2024-09-16T20:51:04.310 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1750000000000002e-06 2024-09-16T20:51:04.310 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.310 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.310 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:04.310 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163606+0000", 2024-09-16T20:51:04.310 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00046418799999999998 2024-09-16T20:51:04.310 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.311 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.311 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:04.311 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163656+0000", 2024-09-16T20:51:04.311 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9852999999999999e-05 2024-09-16T20:51:04.311 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.311 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.311 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:04.311 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281815+0000", 2024-09-16T20:51:04.311 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11815909600000001 2024-09-16T20:51:04.311 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:04.311 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:04.311 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:04.311 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.311 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.312 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:153 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:51:04.312 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:51:04.312 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 160.07507202400001, 2024-09-16T20:51:04.312 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11903462400000001, 2024-09-16T20:51:04.312 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:04.312 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:04.312 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:04.312 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:04.312 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:51:04.312 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:51:04.312 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 153 2024-09-16T20:51:04.312 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.312 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:04.313 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.313 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:04.313 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:51:04.313 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:04.313 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.313 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.313 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:04.313 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:51:04.313 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:04.313 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.313 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.313 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:04.313 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163155+0000", 2024-09-16T20:51:04.313 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.37e-07 2024-09-16T20:51:04.314 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.314 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.314 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:04.314 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163157+0000", 2024-09-16T20:51:04.314 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6010000000000002e-06 2024-09-16T20:51:04.314 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.314 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.314 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:04.314 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163165+0000", 2024-09-16T20:51:04.314 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2619999999999998e-06 2024-09-16T20:51:04.314 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.314 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.315 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:04.315 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163171+0000", 2024-09-16T20:51:04.315 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.711e-06 2024-09-16T20:51:04.315 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.315 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.315 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:04.315 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281868+0000", 2024-09-16T20:51:04.315 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118696622 2024-09-16T20:51:04.315 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.315 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.315 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:04.315 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281969+0000", 2024-09-16T20:51:04.315 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101371 2024-09-16T20:51:04.315 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.315 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.316 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:04.316 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282189+0000", 2024-09-16T20:51:04.316 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00021952000000000001 2024-09-16T20:51:04.316 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:04.316 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:04.316 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:04.316 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.316 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.316 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:154 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 360448~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:51:04.316 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:51:04.316 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 160.07504025200001, 2024-09-16T20:51:04.316 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11932035100000001, 2024-09-16T20:51:04.316 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:04.316 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:04.317 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:04.317 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:04.317 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:51:04.317 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:51:04.317 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 154 2024-09-16T20:51:04.317 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.317 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:04.317 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.317 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:04.317 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:51:04.317 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:04.317 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.317 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.317 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:04.318 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:51:04.318 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:04.318 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.318 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.318 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:04.318 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:51:04.318 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:51:04.318 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.318 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.318 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:04.318 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163189+0000", 2024-09-16T20:51:04.318 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6840000000000001e-06 2024-09-16T20:51:04.318 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.319 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.319 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:04.319 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163196+0000", 2024-09-16T20:51:04.319 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7920000000000004e-06 2024-09-16T20:51:04.319 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.319 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.319 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:04.319 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163201+0000", 2024-09-16T20:51:04.319 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2809999999999999e-06 2024-09-16T20:51:04.319 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.319 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.319 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:04.319 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282219+0000", 2024-09-16T20:51:04.320 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.119018343 2024-09-16T20:51:04.320 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.320 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.320 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:04.320 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282305+0000", 2024-09-16T20:51:04.320 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5865000000000005e-05 2024-09-16T20:51:04.320 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.320 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.320 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:04.320 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282506+0000", 2024-09-16T20:51:04.320 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000200927 2024-09-16T20:51:04.320 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:04.320 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:04.321 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:04.321 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.321 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.321 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop(client.4366.0:3088 3.1e e38/23 3:79e5ec01:::rbd_header.110e891a9b2b:head v 38'306, mlcod=36'304)", 2024-09-16T20:51:04.321 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:51:04.321 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 160.07402138399999, 2024-09-16T20:51:04.321 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118102759, 2024-09-16T20:51:04.321 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:04.321 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:04.321 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:04.321 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:04.321 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.321 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:04.321 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:51:04.321 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:04.322 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.322 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.322 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:04.322 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:51:04.322 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:04.322 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.322 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.322 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:04.322 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:51:04.322 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.85e-07 2024-09-16T20:51:04.322 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.322 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.322 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:04.322 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164212+0000", 2024-09-16T20:51:04.322 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3999999999999997e-06 2024-09-16T20:51:04.323 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.323 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.323 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:04.323 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281553+0000", 2024-09-16T20:51:04.323 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11734135499999999 2024-09-16T20:51:04.323 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.323 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.323 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:04.323 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281567+0000", 2024-09-16T20:51:04.323 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4188999999999999e-05 2024-09-16T20:51:04.323 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.323 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.323 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:04.323 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281653+0000", 2024-09-16T20:51:04.323 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5385999999999999e-05 2024-09-16T20:51:04.324 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.324 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.324 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:04.324 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281679+0000", 2024-09-16T20:51:04.324 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6689000000000001e-05 2024-09-16T20:51:04.324 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.324 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.324 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:04.324 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282277+0000", 2024-09-16T20:51:04.324 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00059798000000000002 2024-09-16T20:51:04.324 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.324 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.324 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:04.324 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282307+0000", 2024-09-16T20:51:04.324 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9975e-05 2024-09-16T20:51:04.325 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:04.325 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:04.325 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:04.325 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.325 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.325 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5255.0:44 3.16 3:6f5476df:::rbd_header.110e25db1f60:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:51:04.325 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:51:04.325 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 87.917718328999996, 2024-09-16T20:51:04.325 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.121007509, 2024-09-16T20:51:04.325 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:04.325 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:04.325 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:04.325 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:04.326 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5255", 2024-09-16T20:51:04.326 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:51:04.326 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 44 2024-09-16T20:51:04.326 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.326 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:04.326 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.326 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:04.326 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:51:04.326 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:04.326 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.326 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.326 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:04.326 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:51:04.327 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:04.327 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.327 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.327 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:04.327 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:51:04.327 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0500000000000003e-07 2024-09-16T20:51:04.327 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.327 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.327 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:04.327 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320512+0000", 2024-09-16T20:51:04.327 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2540000000000001e-06 2024-09-16T20:51:04.327 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.327 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.328 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:04.328 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320518+0000", 2024-09-16T20:51:04.328 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1689999999999997e-06 2024-09-16T20:51:04.328 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.328 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.328 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:04.328 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320521+0000", 2024-09-16T20:51:04.328 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5200000000000002e-06 2024-09-16T20:51:04.328 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.328 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.328 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:04.328 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320548+0000", 2024-09-16T20:51:04.328 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6662999999999999e-05 2024-09-16T20:51:04.328 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.329 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.329 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:04.329 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320587+0000", 2024-09-16T20:51:04.329 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8866999999999999e-05 2024-09-16T20:51:04.329 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.329 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.329 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:04.329 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.441515+0000", 2024-09-16T20:51:04.329 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.120928331 2024-09-16T20:51:04.329 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:04.329 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:04.329 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:04.329 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.329 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.330 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:20329 3.5 3:a4ea64b7:::rbd_header.110ede5ef857:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:04.330 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:01.061694+0000", 2024-09-16T20:51:04.330 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 63.176531761, 2024-09-16T20:51:04.330 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.071234077000000007, 2024-09-16T20:51:04.330 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:04.330 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:04.330 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:04.330 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:04.330 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:51:04.330 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:04.330 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 20329 2024-09-16T20:51:04.330 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.330 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:04.330 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.331 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:04.331 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.061694+0000", 2024-09-16T20:51:04.331 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:04.331 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.331 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.331 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:04.331 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.061694+0000", 2024-09-16T20:51:04.331 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:04.331 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.331 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.331 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:04.331 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089232+0000", 2024-09-16T20:51:04.331 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.027537275 2024-09-16T20:51:04.331 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.331 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.332 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:04.332 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089237+0000", 2024-09-16T20:51:04.332 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4709999999999998e-06 2024-09-16T20:51:04.332 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.332 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.332 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:04.332 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089246+0000", 2024-09-16T20:51:04.332 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8230000000000001e-06 2024-09-16T20:51:04.332 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.332 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.332 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:04.332 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089252+0000", 2024-09-16T20:51:04.332 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3060000000000003e-06 2024-09-16T20:51:04.332 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.332 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.332 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:04.333 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.132756+0000", 2024-09-16T20:51:04.333 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.043504334999999998 2024-09-16T20:51:04.333 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.333 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.333 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:04.333 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.132829+0000", 2024-09-16T20:51:04.333 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2849999999999995e-05 2024-09-16T20:51:04.333 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:04.333 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:04.333 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:04.333 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.132928+0000", 2024-09-16T20:51:04.333 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.9017000000000004e-05 2024-09-16T20:51:04.333 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:04.333 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:04.334 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:04.334 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:04.334 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:04.334 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:04.335 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:51:04.336 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:04.334+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:04.436 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:04.434+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:04.456 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:04.456 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:51:04.456 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:51:04.457 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:04.465 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:51:04.537 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:51:04.535+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:51:04.566 INFO:teuthology.orchestra.run.smithi029.stderr:nodeep-scrub is unset 2024-09-16T20:51:04.575 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:04.575 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:51:04.575 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:51:04.575 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:51:04.576 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:04.588 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:51:04.594 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 4, 5, 0] out_osds: [3] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-16T20:51:04.595 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T20:51:04.595 INFO:tasks.thrashosds.thrasher:Removing osd 2, in_osds are: [1, 2, 4, 5, 0] 2024-09-16T20:51:04.595 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 2 2024-09-16T20:51:04.672 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:51:04.671+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:51:04.737 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:04.737 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:51:04.737 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:51:04.737 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:51:04.737 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.738 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:3088 3.1e 3:79e5ec01:::rbd_header.110e891a9b2b:head [watch watch cookie 140213437171488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:51:04.738 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:51:04.738 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 160.57167505300001, 2024-09-16T20:51:04.738 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.11895186200000001, 2024-09-16T20:51:04.738 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:04.738 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:04.738 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:04.738 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:04.738 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:04.738 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:04.738 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3088 2024-09-16T20:51:04.738 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.739 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:04.739 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.739 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:04.739 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:51:04.739 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:04.739 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.739 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.739 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:04.739 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:51:04.739 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:04.739 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.739 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.739 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:04.740 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163846+0000", 2024-09-16T20:51:04.740 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6799999999999999e-07 2024-09-16T20:51:04.740 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.740 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.740 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:04.740 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163850+0000", 2024-09-16T20:51:04.740 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.929e-06 2024-09-16T20:51:04.740 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.740 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.740 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:04.740 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163862+0000", 2024-09-16T20:51:04.740 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1972e-05 2024-09-16T20:51:04.740 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.740 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.741 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:04.741 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163867+0000", 2024-09-16T20:51:04.741 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.451e-06 2024-09-16T20:51:04.741 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.741 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.741 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:04.741 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163912+0000", 2024-09-16T20:51:04.741 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4456000000000003e-05 2024-09-16T20:51:04.741 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.741 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.741 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:04.741 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163962+0000", 2024-09-16T20:51:04.741 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0578000000000003e-05 2024-09-16T20:51:04.741 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.742 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.742 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-16T20:51:04.742 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164058+0000", 2024-09-16T20:51:04.742 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5934999999999995e-05 2024-09-16T20:51:04.742 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.742 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.742 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:51:04.742 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164570+0000", 2024-09-16T20:51:04.742 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00051241100000000005 2024-09-16T20:51:04.742 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.742 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.742 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:04.742 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282629+0000", 2024-09-16T20:51:04.742 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.118058986 2024-09-16T20:51:04.742 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.743 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.743 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:51:04.743 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282725+0000", 2024-09-16T20:51:04.743 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5063999999999996e-05 2024-09-16T20:51:04.743 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.743 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.743 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:04.743 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282797+0000", 2024-09-16T20:51:04.743 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2212000000000006e-05 2024-09-16T20:51:04.743 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:04.743 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:04.743 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:04.743 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.744 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.744 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:8039 3.1e 3.ac44ff9e (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:51:04.744 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:51:04.744 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 135.02784770400001, 2024-09-16T20:51:04.744 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0053310449999999, 2024-09-16T20:51:04.744 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:04.744 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:04.744 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:04.744 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:04.744 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:04.744 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:04.744 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8039 2024-09-16T20:51:04.745 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.745 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:04.745 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.745 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:04.745 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:51:04.745 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:04.745 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.745 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.745 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:04.745 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:51:04.745 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:04.745 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.746 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.746 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:04.746 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707674+0000", 2024-09-16T20:51:04.746 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6530000000000001e-06 2024-09-16T20:51:04.746 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.746 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.746 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:04.746 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707681+0000", 2024-09-16T20:51:04.746 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9639999999999997e-06 2024-09-16T20:51:04.746 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.746 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.746 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:04.746 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707700+0000", 2024-09-16T20:51:04.746 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8674000000000001e-05 2024-09-16T20:51:04.747 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.747 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.747 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:04.747 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707708+0000", 2024-09-16T20:51:04.747 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6810000000000004e-06 2024-09-16T20:51:04.747 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.747 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.747 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:04.747 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711293+0000", 2024-09-16T20:51:04.747 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035848189999999999 2024-09-16T20:51:04.747 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.747 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.747 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:04.747 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711301+0000", 2024-09-16T20:51:04.748 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4730000000000003e-06 2024-09-16T20:51:04.748 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.748 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.748 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:04.748 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.712990+0000", 2024-09-16T20:51:04.748 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.001689348 2024-09-16T20:51:04.748 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.748 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.748 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:04.748 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713003+0000", 2024-09-16T20:51:04.748 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3433e-05 2024-09-16T20:51:04.748 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:04.748 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:04.748 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:04.749 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.749 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.749 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:8040 3.1e 3.ac44ff9e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:51:04.749 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:51:04.749 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 135.02778730899999, 2024-09-16T20:51:04.749 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0053410780000001, 2024-09-16T20:51:04.749 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:04.749 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:04.749 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:04.749 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:04.749 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:04.749 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:04.749 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8040 2024-09-16T20:51:04.749 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.750 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:04.750 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.750 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:04.750 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:51:04.750 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:04.750 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.750 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.750 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:04.750 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:51:04.750 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:04.750 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.750 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.750 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:04.751 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:51:04.751 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8100000000000003e-07 2024-09-16T20:51:04.751 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.751 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.751 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:04.751 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707737+0000", 2024-09-16T20:51:04.751 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5020000000000001e-06 2024-09-16T20:51:04.751 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.751 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.751 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:04.751 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707745+0000", 2024-09-16T20:51:04.751 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2169999999999994e-06 2024-09-16T20:51:04.752 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.752 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.752 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:04.752 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707751+0000", 2024-09-16T20:51:04.752 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.263e-06 2024-09-16T20:51:04.752 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.752 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.752 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:04.752 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711336+0000", 2024-09-16T20:51:04.752 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035848360000000001 2024-09-16T20:51:04.752 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.752 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.752 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:04.752 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711340+0000", 2024-09-16T20:51:04.753 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1309999999999999e-06 2024-09-16T20:51:04.753 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.753 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.753 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:04.753 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713060+0000", 2024-09-16T20:51:04.753 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0017195059999999 2024-09-16T20:51:04.753 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.753 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.753 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:04.753 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713074+0000", 2024-09-16T20:51:04.753 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4042e-05 2024-09-16T20:51:04.753 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:04.753 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:04.754 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:04.754 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.754 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.754 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4927.0:4487 3.1e 3.ac44ff9e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:51:04.754 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:51:04.754 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 135.027267107, 2024-09-16T20:51:04.754 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0048840880000001, 2024-09-16T20:51:04.754 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:04.754 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:04.754 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:04.754 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:04.754 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4927", 2024-09-16T20:51:04.754 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:04.755 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4487 2024-09-16T20:51:04.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.755 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:04.755 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.755 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:04.755 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:51:04.755 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:04.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.755 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.755 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:04.755 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:51:04.755 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:04.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.756 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.756 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:04.756 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708254+0000", 2024-09-16T20:51:04.756 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1800000000000005e-07 2024-09-16T20:51:04.756 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.756 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.756 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:04.756 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708258+0000", 2024-09-16T20:51:04.756 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8689999999999997e-06 2024-09-16T20:51:04.756 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.756 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.756 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:04.756 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708267+0000", 2024-09-16T20:51:04.756 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9590000000000007e-06 2024-09-16T20:51:04.756 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.757 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.757 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:04.757 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708274+0000", 2024-09-16T20:51:04.757 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9970000000000001e-06 2024-09-16T20:51:04.757 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.757 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.757 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:04.757 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711366+0000", 2024-09-16T20:51:04.757 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030919530000000001 2024-09-16T20:51:04.757 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.757 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.757 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:04.757 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711372+0000", 2024-09-16T20:51:04.757 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9410000000000004e-06 2024-09-16T20:51:04.758 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.758 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.758 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:04.758 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713126+0000", 2024-09-16T20:51:04.758 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0017537249999999 2024-09-16T20:51:04.758 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.758 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.758 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:04.758 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713137+0000", 2024-09-16T20:51:04.758 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0825999999999999e-05 2024-09-16T20:51:04.758 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:04.758 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:04.758 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:04.758 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.759 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.759 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5089.0:11703 3.1b 3:dcd4684e:::rbd_data.110e747f0779.000000000000001b:head [set-alloc-hint object_size 262144 write_size 262144,write 139264~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:51:04.759 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:51:04.759 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 120.59753133300001, 2024-09-16T20:51:04.759 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.084231385000000006, 2024-09-16T20:51:04.759 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:04.759 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:04.759 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:04.759 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:04.759 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5089", 2024-09-16T20:51:04.759 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:51:04.759 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 11703 2024-09-16T20:51:04.759 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.759 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:04.759 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.760 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:04.760 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:51:04.760 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:04.760 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.760 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.760 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:04.760 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:51:04.760 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:04.760 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.760 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.760 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:04.760 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:51:04.760 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2600000000000001e-07 2024-09-16T20:51:04.761 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.761 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.761 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:04.761 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137998+0000", 2024-09-16T20:51:04.761 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9549999999999998e-06 2024-09-16T20:51:04.761 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.761 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.761 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:04.761 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138008+0000", 2024-09-16T20:51:04.761 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3789999999999995e-06 2024-09-16T20:51:04.761 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.761 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.761 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:04.761 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138013+0000", 2024-09-16T20:51:04.762 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2209999999999996e-06 2024-09-16T20:51:04.762 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.762 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.762 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:04.762 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138054+0000", 2024-09-16T20:51:04.762 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1022999999999997e-05 2024-09-16T20:51:04.762 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.762 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.762 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:04.762 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138124+0000", 2024-09-16T20:51:04.762 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0234999999999994e-05 2024-09-16T20:51:04.762 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.762 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.762 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:04.762 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138197+0000", 2024-09-16T20:51:04.763 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2451000000000001e-05 2024-09-16T20:51:04.763 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.763 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.763 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:04.763 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222010+0000", 2024-09-16T20:51:04.763 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.083813433000000007 2024-09-16T20:51:04.763 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.763 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.763 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:51:04.763 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222163+0000", 2024-09-16T20:51:04.763 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015274599999999999 2024-09-16T20:51:04.763 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.763 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.764 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:51:04.764 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222197+0000", 2024-09-16T20:51:04.764 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3683000000000003e-05 2024-09-16T20:51:04.764 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.764 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.764 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:04.764 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222220+0000", 2024-09-16T20:51:04.764 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3433000000000001e-05 2024-09-16T20:51:04.764 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:04.764 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:04.764 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:04.764 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.764 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.764 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.5089.0:11702 3.15 e73/64 3:acf33c8b:::rbd_data.110e747f0779.000000000000001a:head v 73'132, mlcod=73'130)", 2024-09-16T20:51:04.764 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:51:04.765 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 120.597264004, 2024-09-16T20:51:04.765 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.084079763000000002, 2024-09-16T20:51:04.765 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:04.765 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:04.765 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:04.765 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:04.765 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.765 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:04.765 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:51:04.765 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:04.765 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.765 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.765 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:04.765 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:51:04.765 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:04.765 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.766 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.766 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:04.766 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:51:04.766 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:51:04.766 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.766 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.766 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:04.766 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138263+0000", 2024-09-16T20:51:04.766 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1e-06 2024-09-16T20:51:04.766 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.766 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.766 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:04.766 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221589+0000", 2024-09-16T20:51:04.766 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.083326002999999996 2024-09-16T20:51:04.767 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.767 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.767 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:04.767 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221614+0000", 2024-09-16T20:51:04.767 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5593999999999999e-05 2024-09-16T20:51:04.767 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.767 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.767 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:04.767 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221714+0000", 2024-09-16T20:51:04.767 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9889000000000005e-05 2024-09-16T20:51:04.767 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.767 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.767 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:04.767 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221727+0000", 2024-09-16T20:51:04.767 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3125999999999999e-05 2024-09-16T20:51:04.768 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.768 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.768 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:51:04.768 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222294+0000", 2024-09-16T20:51:04.768 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00056712199999999998 2024-09-16T20:51:04.768 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.768 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.768 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:04.768 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222336+0000", 2024-09-16T20:51:04.768 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1470000000000001e-05 2024-09-16T20:51:04.768 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:04.768 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:04.768 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:04.768 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.769 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.769 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5138.0:9358 3.c 3:33996a43:::rbd_data.110e6372565d.0000000000000007:head [read 1343488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T20:51:04.769 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:51:04.769 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 96.882988593999997, 2024-09-16T20:51:04.769 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0167493569999999, 2024-09-16T20:51:04.769 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:04.769 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:04.769 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:04.769 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:04.769 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5138", 2024-09-16T20:51:04.769 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:51:04.769 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9358 2024-09-16T20:51:04.769 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.770 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:04.770 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.770 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:04.770 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:51:04.770 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:04.770 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.770 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.770 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:04.770 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:51:04.770 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:04.770 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.770 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.770 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:04.770 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852533+0000", 2024-09-16T20:51:04.771 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.646e-06 2024-09-16T20:51:04.771 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.771 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.771 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:04.771 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852539+0000", 2024-09-16T20:51:04.771 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3389999999999998e-06 2024-09-16T20:51:04.771 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.771 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.771 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:04.771 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852552+0000", 2024-09-16T20:51:04.771 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.271e-05 2024-09-16T20:51:04.771 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.771 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.771 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:04.772 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852563+0000", 2024-09-16T20:51:04.772 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1139e-05 2024-09-16T20:51:04.772 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.772 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.772 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:04.772 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852883+0000", 2024-09-16T20:51:04.772 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00032010600000000001 2024-09-16T20:51:04.772 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.772 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.772 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:04.772 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852890+0000", 2024-09-16T20:51:04.772 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7669999999999998e-06 2024-09-16T20:51:04.772 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.772 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.773 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:04.773 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869059+0000", 2024-09-16T20:51:04.773 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.016169353 2024-09-16T20:51:04.773 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.773 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.773 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:04.773 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869178+0000", 2024-09-16T20:51:04.773 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118193 2024-09-16T20:51:04.773 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.773 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.773 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:04.773 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869281+0000", 2024-09-16T20:51:04.774 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010310399999999999 2024-09-16T20:51:04.774 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:04.774 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:04.774 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:04.774 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.774 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.774 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5138.0:9382 3.c 3:33996a43:::rbd_data.110e6372565d.0000000000000007:head [read 0~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T20:51:04.774 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:51:04.774 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 96.882937717000004, 2024-09-16T20:51:04.774 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.017136034, 2024-09-16T20:51:04.774 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:04.774 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:04.774 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:04.774 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:04.775 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5138", 2024-09-16T20:51:04.775 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:51:04.775 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9382 2024-09-16T20:51:04.775 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.775 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:04.775 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.775 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:04.775 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:51:04.775 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:04.775 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.775 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.775 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:04.775 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:51:04.775 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:04.775 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.776 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.776 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:04.776 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852583+0000", 2024-09-16T20:51:04.776 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.51e-07 2024-09-16T20:51:04.776 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.776 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.776 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:04.776 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852586+0000", 2024-09-16T20:51:04.776 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0280000000000001e-06 2024-09-16T20:51:04.776 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.776 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.776 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:04.776 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852595+0000", 2024-09-16T20:51:04.776 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2799999999999992e-06 2024-09-16T20:51:04.777 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.777 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.777 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:04.777 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852601+0000", 2024-09-16T20:51:04.777 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3810000000000004e-06 2024-09-16T20:51:04.777 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.777 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.777 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:04.777 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852921+0000", 2024-09-16T20:51:04.777 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00031979699999999997 2024-09-16T20:51:04.777 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.777 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.777 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:04.777 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852930+0000", 2024-09-16T20:51:04.777 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6489999999999997e-06 2024-09-16T20:51:04.778 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.778 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.778 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:04.778 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869434+0000", 2024-09-16T20:51:04.778 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0165038829999999 2024-09-16T20:51:04.778 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.778 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.778 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:04.778 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869495+0000", 2024-09-16T20:51:04.778 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1509999999999999e-05 2024-09-16T20:51:04.778 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.778 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.778 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:04.778 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869718+0000", 2024-09-16T20:51:04.778 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000222955 2024-09-16T20:51:04.779 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:04.779 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:04.779 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:04.779 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.779 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.779 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9969 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 632832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:04.779 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642045+0000", 2024-09-16T20:51:04.779 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 74.093475260000005, 2024-09-16T20:51:04.779 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076903192999999995, 2024-09-16T20:51:04.779 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:04.780 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:04.780 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:04.780 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:04.780 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:04.780 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:04.780 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9969 2024-09-16T20:51:04.780 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.780 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:04.780 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.780 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:04.780 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642045+0000", 2024-09-16T20:51:04.780 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:04.780 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.781 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.781 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:04.781 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642045+0000", 2024-09-16T20:51:04.781 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:04.781 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.781 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.781 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:04.781 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642045+0000", 2024-09-16T20:51:04.781 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5100000000000001e-07 2024-09-16T20:51:04.781 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.781 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.781 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:04.781 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642047+0000", 2024-09-16T20:51:04.781 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8679999999999999e-06 2024-09-16T20:51:04.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.782 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.782 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:04.782 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642053+0000", 2024-09-16T20:51:04.782 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1720000000000004e-06 2024-09-16T20:51:04.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.782 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.782 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:04.782 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642060+0000", 2024-09-16T20:51:04.782 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0339999999999999e-06 2024-09-16T20:51:04.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.782 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.782 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:04.782 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.718562+0000", 2024-09-16T20:51:04.782 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076501969000000003 2024-09-16T20:51:04.783 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.783 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.783 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:04.783 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.718643+0000", 2024-09-16T20:51:04.783 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0539999999999998e-05 2024-09-16T20:51:04.783 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.783 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.783 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:04.783 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.718948+0000", 2024-09-16T20:51:04.783 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000305259 2024-09-16T20:51:04.783 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:04.783 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:04.783 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:04.784 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.784 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.784 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9970 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 637952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:04.784 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642067+0000", 2024-09-16T20:51:04.784 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 74.093453280000006, 2024-09-16T20:51:04.784 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077174928000000004, 2024-09-16T20:51:04.784 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:04.784 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:04.784 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:04.784 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:04.784 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:04.784 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:04.784 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9970 2024-09-16T20:51:04.784 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.785 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:04.785 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.785 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:04.785 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642067+0000", 2024-09-16T20:51:04.785 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:04.785 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.785 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.785 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:04.785 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642067+0000", 2024-09-16T20:51:04.785 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:04.785 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.785 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.785 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:04.785 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642067+0000", 2024-09-16T20:51:04.786 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2499999999999995e-07 2024-09-16T20:51:04.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.786 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:04.786 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642070+0000", 2024-09-16T20:51:04.786 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3199999999999998e-06 2024-09-16T20:51:04.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.786 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:04.786 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642076+0000", 2024-09-16T20:51:04.786 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.122e-06 2024-09-16T20:51:04.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.786 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:04.786 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642081+0000", 2024-09-16T20:51:04.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0869999999999997e-06 2024-09-16T20:51:04.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.787 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.787 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:04.787 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.718980+0000", 2024-09-16T20:51:04.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076899482000000005 2024-09-16T20:51:04.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.787 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.787 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:04.787 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719085+0000", 2024-09-16T20:51:04.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010446399999999999 2024-09-16T20:51:04.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.787 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.787 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:04.787 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719242+0000", 2024-09-16T20:51:04.788 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000156928 2024-09-16T20:51:04.788 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:04.788 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:04.788 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:04.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.788 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.788 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9971 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 643072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:04.788 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642102+0000", 2024-09-16T20:51:04.788 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 74.093417572999996, 2024-09-16T20:51:04.788 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077421016999999995, 2024-09-16T20:51:04.788 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:04.788 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:04.788 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:04.788 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:04.788 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:04.789 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:04.789 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9971 2024-09-16T20:51:04.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.789 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:04.789 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.789 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:04.789 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642102+0000", 2024-09-16T20:51:04.789 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:04.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.789 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.789 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:04.789 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642102+0000", 2024-09-16T20:51:04.789 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:04.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.790 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.790 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:04.790 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642103+0000", 2024-09-16T20:51:04.790 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7399999999999998e-07 2024-09-16T20:51:04.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.790 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.790 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:04.790 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642105+0000", 2024-09-16T20:51:04.790 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2649999999999999e-06 2024-09-16T20:51:04.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.790 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.790 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:04.790 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642112+0000", 2024-09-16T20:51:04.790 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.294e-06 2024-09-16T20:51:04.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.790 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.791 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:04.791 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642118+0000", 2024-09-16T20:51:04.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4500000000000003e-06 2024-09-16T20:51:04.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.791 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.791 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:04.791 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719285+0000", 2024-09-16T20:51:04.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077166883000000006 2024-09-16T20:51:04.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.791 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.791 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:04.791 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719368+0000", 2024-09-16T20:51:04.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2811000000000004e-05 2024-09-16T20:51:04.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.791 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.792 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:04.792 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719523+0000", 2024-09-16T20:51:04.792 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015584 2024-09-16T20:51:04.792 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:04.792 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:04.792 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:04.792 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.792 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.792 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9972 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 648192~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:04.792 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642125+0000", 2024-09-16T20:51:04.792 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 74.093394626000006, 2024-09-16T20:51:04.792 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077710324999999997, 2024-09-16T20:51:04.792 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:04.792 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:04.793 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:04.793 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:04.793 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:04.793 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:04.793 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9972 2024-09-16T20:51:04.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.793 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:04.793 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.793 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:04.793 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642125+0000", 2024-09-16T20:51:04.793 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:04.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.794 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:04.794 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642125+0000", 2024-09-16T20:51:04.794 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:04.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.794 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:04.794 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642126+0000", 2024-09-16T20:51:04.794 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1899999999999998e-07 2024-09-16T20:51:04.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.794 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:04.794 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642128+0000", 2024-09-16T20:51:04.794 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8619999999999999e-06 2024-09-16T20:51:04.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.795 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.795 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:04.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642134+0000", 2024-09-16T20:51:04.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5289999999999997e-06 2024-09-16T20:51:04.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.795 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.795 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:04.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642139+0000", 2024-09-16T20:51:04.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1250000000000001e-06 2024-09-16T20:51:04.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.795 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.795 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:04.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719570+0000", 2024-09-16T20:51:04.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077430516000000005 2024-09-16T20:51:04.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.796 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.796 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:04.796 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719655+0000", 2024-09-16T20:51:04.796 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5500000000000005e-05 2024-09-16T20:51:04.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.796 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.796 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:04.796 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719836+0000", 2024-09-16T20:51:04.796 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018037399999999999 2024-09-16T20:51:04.796 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:04.796 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:04.796 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:04.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.796 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.797 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9973 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 653312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:04.797 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:51:04.797 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 74.093374015999999, 2024-09-16T20:51:04.797 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077905262000000003, 2024-09-16T20:51:04.797 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:04.797 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:04.797 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:04.797 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:04.797 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:04.797 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:04.797 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9973 2024-09-16T20:51:04.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.797 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:04.798 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.798 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:04.798 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:51:04.798 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:04.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.798 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.798 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:04.798 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:51:04.798 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:04.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.798 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.798 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:04.798 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:51:04.798 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:51:04.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.799 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.799 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:04.799 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642148+0000", 2024-09-16T20:51:04.799 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0219999999999999e-06 2024-09-16T20:51:04.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.799 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.799 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:04.799 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642154+0000", 2024-09-16T20:51:04.799 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8540000000000002e-06 2024-09-16T20:51:04.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.799 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.799 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:04.799 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642159+0000", 2024-09-16T20:51:04.799 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.045e-06 2024-09-16T20:51:04.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.800 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:04.800 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719875+0000", 2024-09-16T20:51:04.800 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077716116000000002 2024-09-16T20:51:04.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.800 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:04.800 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719934+0000", 2024-09-16T20:51:04.800 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8819e-05 2024-09-16T20:51:04.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.800 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:04.800 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720051+0000", 2024-09-16T20:51:04.800 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000116947 2024-09-16T20:51:04.800 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:04.801 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:04.801 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:04.801 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.801 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.801 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9974 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 658432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:04.801 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642166+0000", 2024-09-16T20:51:04.801 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 74.093354102000006, 2024-09-16T20:51:04.801 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.078179786000000001, 2024-09-16T20:51:04.801 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:04.801 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:04.801 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:04.801 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:04.801 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:04.801 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:04.801 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9974 2024-09-16T20:51:04.802 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.802 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:04.802 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.802 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:04.802 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642166+0000", 2024-09-16T20:51:04.802 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:04.802 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.802 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.802 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:04.802 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642166+0000", 2024-09-16T20:51:04.802 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:04.802 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.802 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.802 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:04.802 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647343+0000", 2024-09-16T20:51:04.803 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0051769440000000002 2024-09-16T20:51:04.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.803 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.803 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:04.803 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647348+0000", 2024-09-16T20:51:04.803 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3619999999999997e-06 2024-09-16T20:51:04.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.803 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.803 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:04.803 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647359+0000", 2024-09-16T20:51:04.803 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0929e-05 2024-09-16T20:51:04.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.803 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.803 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:04.803 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647366+0000", 2024-09-16T20:51:04.804 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0910000000000004e-06 2024-09-16T20:51:04.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.804 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:04.804 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720094+0000", 2024-09-16T20:51:04.804 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.072727403999999995 2024-09-16T20:51:04.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.804 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:04.804 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720173+0000", 2024-09-16T20:51:04.804 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9382000000000007e-05 2024-09-16T20:51:04.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.804 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:04.804 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720346+0000", 2024-09-16T20:51:04.804 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017267399999999999 2024-09-16T20:51:04.805 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:04.805 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:04.805 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:04.805 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.805 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.805 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9990 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 740352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:04.805 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647725+0000", 2024-09-16T20:51:04.805 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 74.087795017999994, 2024-09-16T20:51:04.805 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077029919000000002, 2024-09-16T20:51:04.805 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:04.805 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:04.805 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:04.805 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:04.806 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:04.806 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:04.806 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9990 2024-09-16T20:51:04.806 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.806 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:04.806 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.806 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:04.806 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647725+0000", 2024-09-16T20:51:04.806 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:04.806 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.806 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.806 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:04.806 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647725+0000", 2024-09-16T20:51:04.806 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:04.806 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.807 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.807 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:04.807 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647725+0000", 2024-09-16T20:51:04.807 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2399999999999999e-07 2024-09-16T20:51:04.807 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.807 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.807 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:04.807 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647726+0000", 2024-09-16T20:51:04.807 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.116e-06 2024-09-16T20:51:04.807 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.807 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.807 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:04.807 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647729+0000", 2024-09-16T20:51:04.807 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.937e-06 2024-09-16T20:51:04.807 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.808 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.808 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:04.808 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647732+0000", 2024-09-16T20:51:04.808 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8739999999999999e-06 2024-09-16T20:51:04.808 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.808 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.808 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:04.808 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724612+0000", 2024-09-16T20:51:04.808 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076880102000000006 2024-09-16T20:51:04.808 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.808 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.808 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:04.808 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724649+0000", 2024-09-16T20:51:04.808 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6738e-05 2024-09-16T20:51:04.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.809 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.809 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:04.809 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724755+0000", 2024-09-16T20:51:04.809 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010592800000000001 2024-09-16T20:51:04.809 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:04.809 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:04.809 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:04.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.809 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.809 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9991 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 745472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:04.809 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:51:04.809 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 74.087784709000005, 2024-09-16T20:51:04.809 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077257820000000005, 2024-09-16T20:51:04.809 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:04.810 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:04.810 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:04.810 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:04.810 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:04.810 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:04.810 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9991 2024-09-16T20:51:04.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.810 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:04.810 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.810 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:04.810 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:51:04.810 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:04.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.810 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.810 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:04.811 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:51:04.811 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:04.811 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.811 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.811 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:04.811 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:51:04.811 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1199999999999999e-07 2024-09-16T20:51:04.811 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.811 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.811 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:04.811 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647736+0000", 2024-09-16T20:51:04.811 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3699999999999999e-07 2024-09-16T20:51:04.811 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.811 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.811 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:04.812 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647739+0000", 2024-09-16T20:51:04.812 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9359999999999999e-06 2024-09-16T20:51:04.812 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.812 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.812 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:04.812 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647742+0000", 2024-09-16T20:51:04.812 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5720000000000001e-06 2024-09-16T20:51:04.812 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.812 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.812 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:04.812 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724774+0000", 2024-09-16T20:51:04.812 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077032457999999998 2024-09-16T20:51:04.812 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.812 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.813 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:04.813 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724818+0000", 2024-09-16T20:51:04.813 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3803000000000001e-05 2024-09-16T20:51:04.813 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.813 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.813 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:04.813 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724993+0000", 2024-09-16T20:51:04.813 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000174902 2024-09-16T20:51:04.813 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:04.813 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:04.813 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:04.813 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.813 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.813 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9992 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 750592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:04.814 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:51:04.814 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 74.087775120000003, 2024-09-16T20:51:04.814 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077553227000000002, 2024-09-16T20:51:04.814 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:04.814 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:04.814 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:04.814 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:04.814 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:04.814 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:04.814 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9992 2024-09-16T20:51:04.814 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.814 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:04.814 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.814 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:04.815 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:51:04.815 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:04.815 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.815 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.815 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:04.815 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:51:04.815 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:04.815 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.815 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.815 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:04.815 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:51:04.815 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0200000000000001e-07 2024-09-16T20:51:04.815 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.815 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.816 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:04.816 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647746+0000", 2024-09-16T20:51:04.816 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.088e-06 2024-09-16T20:51:04.816 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.816 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.816 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:04.816 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647749+0000", 2024-09-16T20:51:04.816 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9170000000000002e-06 2024-09-16T20:51:04.816 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.816 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.816 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:04.816 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647752+0000", 2024-09-16T20:51:04.816 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.774e-06 2024-09-16T20:51:04.816 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.817 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.817 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:04.817 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725042+0000", 2024-09-16T20:51:04.817 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077289952999999995 2024-09-16T20:51:04.817 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.817 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.817 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:04.817 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725113+0000", 2024-09-16T20:51:04.817 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0920999999999994e-05 2024-09-16T20:51:04.817 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.817 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.817 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:04.818 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725298+0000", 2024-09-16T20:51:04.818 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000185372 2024-09-16T20:51:04.818 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:04.818 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:04.818 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:04.818 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.818 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.818 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9993 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 755712~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:04.818 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647755+0000", 2024-09-16T20:51:04.818 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 74.087765094000005, 2024-09-16T20:51:04.818 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077725674999999994, 2024-09-16T20:51:04.818 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:04.818 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:04.819 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:04.819 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:04.819 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:04.819 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:04.819 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9993 2024-09-16T20:51:04.819 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.819 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:04.819 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.819 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:04.819 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647755+0000", 2024-09-16T20:51:04.819 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:04.819 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.819 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.820 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:04.820 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647755+0000", 2024-09-16T20:51:04.820 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:04.820 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.820 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.820 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:04.820 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653406+0000", 2024-09-16T20:51:04.820 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.005650876 2024-09-16T20:51:04.820 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.820 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.820 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:04.820 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653411+0000", 2024-09-16T20:51:04.820 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8899999999999998e-06 2024-09-16T20:51:04.820 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.821 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.821 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:04.821 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653423+0000", 2024-09-16T20:51:04.821 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2447e-05 2024-09-16T20:51:04.821 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.821 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.821 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:04.821 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653431+0000", 2024-09-16T20:51:04.821 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5730000000000002e-06 2024-09-16T20:51:04.821 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.821 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.821 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:04.821 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725333+0000", 2024-09-16T20:51:04.821 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.071902743000000005 2024-09-16T20:51:04.822 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.822 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.822 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:04.822 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725385+0000", 2024-09-16T20:51:04.822 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1585000000000002e-05 2024-09-16T20:51:04.822 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.822 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.822 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:04.822 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725481+0000", 2024-09-16T20:51:04.822 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5561000000000002e-05 2024-09-16T20:51:04.822 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:04.822 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:04.822 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:04.822 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.822 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.823 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6061.0:5400 3.b 3:d353d652:::rbd_data.110e16eafbd.0000000000000010:head [read 577536~28672] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-16T20:51:04.823 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:27.761367+0000", 2024-09-16T20:51:04.823 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.974153147999999, 2024-09-16T20:51:04.823 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0029066630000001, 2024-09-16T20:51:04.823 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:04.823 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:04.823 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:04.823 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:04.823 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6061", 2024-09-16T20:51:04.823 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3930128440", 2024-09-16T20:51:04.823 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5400 2024-09-16T20:51:04.823 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.823 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:04.823 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.824 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:04.824 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761367+0000", 2024-09-16T20:51:04.824 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:04.824 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.824 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.824 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:04.824 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761367+0000", 2024-09-16T20:51:04.824 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:04.824 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.824 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.824 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:04.824 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761369+0000", 2024-09-16T20:51:04.824 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.934e-06 2024-09-16T20:51:04.824 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.825 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.825 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:04.825 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761374+0000", 2024-09-16T20:51:04.825 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7139999999999998e-06 2024-09-16T20:51:04.825 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.825 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.825 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:04.825 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761387+0000", 2024-09-16T20:51:04.825 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2349e-05 2024-09-16T20:51:04.825 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.825 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.825 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:04.825 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761393+0000", 2024-09-16T20:51:04.825 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.652e-06 2024-09-16T20:51:04.825 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.825 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.826 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:04.826 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761442+0000", 2024-09-16T20:51:04.826 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9011000000000002e-05 2024-09-16T20:51:04.826 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.826 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.826 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:04.826 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761451+0000", 2024-09-16T20:51:04.826 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1920000000000005e-06 2024-09-16T20:51:04.826 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.826 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.826 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:04.826 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:28.764169+0000", 2024-09-16T20:51:04.826 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.002718161 2024-09-16T20:51:04.826 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.826 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.827 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:04.827 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:28.764273+0000", 2024-09-16T20:51:04.827 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010465 2024-09-16T20:51:04.827 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:04.827 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:04.827 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:04.827 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.827 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.827 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6061.0:5424 3.b 3:d353d652:::rbd_data.110e16eafbd.0000000000000010:head [read 606208~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-16T20:51:04.827 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:27.766882+0000", 2024-09-16T20:51:04.827 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.968637784999999, 2024-09-16T20:51:04.827 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99753650500000002, 2024-09-16T20:51:04.827 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:04.827 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:04.827 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:04.828 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:04.828 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6061", 2024-09-16T20:51:04.828 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3930128440", 2024-09-16T20:51:04.828 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5424 2024-09-16T20:51:04.828 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.828 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:04.828 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.828 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:04.828 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766882+0000", 2024-09-16T20:51:04.828 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:04.828 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.828 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.828 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:04.828 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766882+0000", 2024-09-16T20:51:04.829 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:04.829 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.829 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.829 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:04.829 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766883+0000", 2024-09-16T20:51:04.829 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3289999999999999e-06 2024-09-16T20:51:04.829 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.829 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.829 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:04.829 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766888+0000", 2024-09-16T20:51:04.829 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5800000000000002e-06 2024-09-16T20:51:04.829 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.829 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.829 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:04.830 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766900+0000", 2024-09-16T20:51:04.830 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1861e-05 2024-09-16T20:51:04.830 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.830 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.830 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:04.830 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766909+0000", 2024-09-16T20:51:04.830 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5210000000000008e-06 2024-09-16T20:51:04.830 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.830 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.830 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:04.830 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766958+0000", 2024-09-16T20:51:04.830 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8718999999999999e-05 2024-09-16T20:51:04.830 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.831 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.831 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:04.831 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766975+0000", 2024-09-16T20:51:04.831 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7231000000000002e-05 2024-09-16T20:51:04.831 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.831 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.831 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:04.831 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:28.764327+0000", 2024-09-16T20:51:04.831 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99735112500000001 2024-09-16T20:51:04.831 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:04.831 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:04.831 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:04.831 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:28.764419+0000", 2024-09-16T20:51:04.831 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2139000000000007e-05 2024-09-16T20:51:04.831 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:04.832 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:04.832 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:04.832 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:04.832 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:04.832 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:04.833 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:51:04.834 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:04.832+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:04.934 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:04.933+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:04.943 INFO:tasks.ceph.mon.a.smithi029.stderr:2024-09-16T20:51:04.942+0000 7fe0ce96d640 -1 mon.a@0(leader).osd e177 definitely_dead 0 2024-09-16T20:51:04.982 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:04.983 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:51:04.983 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:51:04.983 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:04.991 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:51:05.035 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:05.035+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:05.136 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:05.135+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:05.138 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:05.139 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:51:05.139 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:51:05.139 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:51:05.139 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:05.147 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:51:05.237 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:05.236+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:05.256 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:05.256 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:51:05.256 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:51:05.256 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:51:05.256 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.256 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12548 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 628736~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:05.256 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757009+0000", 2024-09-16T20:51:05.257 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 74.497336641000004, 2024-09-16T20:51:05.257 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065743956000000006, 2024-09-16T20:51:05.257 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:05.257 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:05.257 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:05.257 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:05.257 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:05.257 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:05.257 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12548 2024-09-16T20:51:05.257 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.257 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:05.257 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.257 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:05.257 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757009+0000", 2024-09-16T20:51:05.258 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.258 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.258 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.258 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:05.258 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757009+0000", 2024-09-16T20:51:05.258 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.258 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.258 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.258 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:05.258 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757010+0000", 2024-09-16T20:51:05.258 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9900000000000001e-07 2024-09-16T20:51:05.258 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.258 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.258 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:05.259 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757012+0000", 2024-09-16T20:51:05.259 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2689999999999998e-06 2024-09-16T20:51:05.259 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.259 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.259 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:05.259 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757022+0000", 2024-09-16T20:51:05.259 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9699999999999994e-06 2024-09-16T20:51:05.259 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.259 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.259 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:05.259 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757028+0000", 2024-09-16T20:51:05.259 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3799999999999999e-06 2024-09-16T20:51:05.259 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.259 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.260 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:05.260 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822552+0000", 2024-09-16T20:51:05.260 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065524281000000004 2024-09-16T20:51:05.260 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.260 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.260 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:05.260 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822613+0000", 2024-09-16T20:51:05.260 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0430999999999998e-05 2024-09-16T20:51:05.260 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.260 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.260 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:05.260 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822753+0000", 2024-09-16T20:51:05.261 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000140226 2024-09-16T20:51:05.261 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.261 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:05.261 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.261 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.261 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.261 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12549 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 633856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:05.261 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:51:05.261 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 74.497307894000002, 2024-09-16T20:51:05.261 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065978518, 2024-09-16T20:51:05.261 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:05.261 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:05.261 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:05.262 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:05.262 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:05.262 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:05.262 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12549 2024-09-16T20:51:05.262 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.262 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:05.262 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.262 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:05.262 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:51:05.263 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.263 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.263 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:05.263 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:51:05.263 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.263 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.263 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:05.263 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:51:05.263 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9299999999999999e-07 2024-09-16T20:51:05.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.263 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.263 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:05.264 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757040+0000", 2024-09-16T20:51:05.264 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.677e-06 2024-09-16T20:51:05.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.264 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.264 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:05.264 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757045+0000", 2024-09-16T20:51:05.264 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8149999999999996e-06 2024-09-16T20:51:05.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.264 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.264 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:05.264 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757048+0000", 2024-09-16T20:51:05.264 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1e-06 2024-09-16T20:51:05.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.265 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.265 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:05.265 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822796+0000", 2024-09-16T20:51:05.265 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065748081 2024-09-16T20:51:05.265 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.265 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.265 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:05.265 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822867+0000", 2024-09-16T20:51:05.265 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0679000000000005e-05 2024-09-16T20:51:05.265 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.265 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.265 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:05.265 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823016+0000", 2024-09-16T20:51:05.265 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014987299999999999 2024-09-16T20:51:05.266 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.266 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:05.266 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.266 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.266 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.266 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12550 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 638976~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:05.266 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757060+0000", 2024-09-16T20:51:05.266 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 74.497285478999999, 2024-09-16T20:51:05.266 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066153564999999998, 2024-09-16T20:51:05.266 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:05.266 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:05.266 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:05.266 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:05.266 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:05.267 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:05.267 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12550 2024-09-16T20:51:05.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.267 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:05.267 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.267 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:05.267 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757060+0000", 2024-09-16T20:51:05.267 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.267 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.267 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:05.267 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757060+0000", 2024-09-16T20:51:05.267 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.268 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.268 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:05.268 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757061+0000", 2024-09-16T20:51:05.268 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.15e-07 2024-09-16T20:51:05.268 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.268 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.268 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:05.268 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757062+0000", 2024-09-16T20:51:05.268 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.663e-06 2024-09-16T20:51:05.268 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.268 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.268 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:05.268 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757066+0000", 2024-09-16T20:51:05.269 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.817e-06 2024-09-16T20:51:05.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.269 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.269 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:05.269 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757070+0000", 2024-09-16T20:51:05.269 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1509999999999997e-06 2024-09-16T20:51:05.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.269 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.269 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:05.269 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823044+0000", 2024-09-16T20:51:05.269 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065973992999999995 2024-09-16T20:51:05.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.269 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.270 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:05.270 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823082+0000", 2024-09-16T20:51:05.270 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8050999999999999e-05 2024-09-16T20:51:05.270 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.270 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.270 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:05.270 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823214+0000", 2024-09-16T20:51:05.270 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013157500000000001 2024-09-16T20:51:05.270 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.270 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:05.270 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.270 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.270 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.270 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12551 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 644096~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:05.271 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:51:05.271 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 74.497271780999995, 2024-09-16T20:51:05.271 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066391378000000001, 2024-09-16T20:51:05.271 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:05.271 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:05.271 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:05.271 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:05.271 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:05.271 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:05.271 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12551 2024-09-16T20:51:05.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.271 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:05.271 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.272 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:05.272 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:51:05.272 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.272 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.272 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.272 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:05.272 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:51:05.272 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.272 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.272 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.272 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:05.272 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:51:05.272 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2999999999999999e-07 2024-09-16T20:51:05.272 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.273 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.273 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:05.273 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757075+0000", 2024-09-16T20:51:05.273 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.097e-06 2024-09-16T20:51:05.273 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.273 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.273 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:05.273 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757082+0000", 2024-09-16T20:51:05.273 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9500000000000004e-06 2024-09-16T20:51:05.273 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.273 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.273 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:05.273 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757086+0000", 2024-09-16T20:51:05.273 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2229999999999999e-06 2024-09-16T20:51:05.274 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.274 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.274 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:05.274 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823262+0000", 2024-09-16T20:51:05.274 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066176078999999999 2024-09-16T20:51:05.274 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.274 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.274 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:05.274 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823336+0000", 2024-09-16T20:51:05.274 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3979000000000004e-05 2024-09-16T20:51:05.274 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.274 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.274 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:05.274 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823465+0000", 2024-09-16T20:51:05.275 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012982 2024-09-16T20:51:05.275 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.275 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:05.275 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.275 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.275 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.275 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12552 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 649216~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:05.275 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757089+0000", 2024-09-16T20:51:05.275 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 74.497256640000003, 2024-09-16T20:51:05.275 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066655617, 2024-09-16T20:51:05.275 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:05.275 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:05.275 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:05.276 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:05.276 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:05.276 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:05.276 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12552 2024-09-16T20:51:05.276 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.276 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:05.276 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.276 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:05.276 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757089+0000", 2024-09-16T20:51:05.276 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.276 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.276 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.276 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:05.277 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757089+0000", 2024-09-16T20:51:05.277 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.277 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.277 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.277 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:05.277 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759492+0000", 2024-09-16T20:51:05.277 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0024026109999999998 2024-09-16T20:51:05.277 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.277 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.277 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:05.277 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759497+0000", 2024-09-16T20:51:05.277 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1120000000000004e-06 2024-09-16T20:51:05.277 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.278 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.278 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:05.278 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759508+0000", 2024-09-16T20:51:05.278 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0981e-05 2024-09-16T20:51:05.278 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.278 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.278 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:05.278 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759516+0000", 2024-09-16T20:51:05.278 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7270000000000007e-06 2024-09-16T20:51:05.278 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.278 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.278 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:05.278 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823504+0000", 2024-09-16T20:51:05.278 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.063987984999999997 2024-09-16T20:51:05.278 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.279 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.279 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:05.279 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823579+0000", 2024-09-16T20:51:05.279 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5703000000000003e-05 2024-09-16T20:51:05.279 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.279 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.279 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:05.279 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823745+0000", 2024-09-16T20:51:05.279 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016549800000000001 2024-09-16T20:51:05.279 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.279 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:05.279 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.279 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.279 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.279 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12560 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 690176~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:05.279 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.759658+0000", 2024-09-16T20:51:05.280 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 74.494687438, 2024-09-16T20:51:05.280 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065858517000000005, 2024-09-16T20:51:05.280 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:05.280 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:05.280 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:05.280 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:05.280 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:05.280 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:05.280 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12560 2024-09-16T20:51:05.280 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.280 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:05.280 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.280 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:05.280 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759658+0000", 2024-09-16T20:51:05.280 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.281 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.281 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.281 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:05.281 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759658+0000", 2024-09-16T20:51:05.281 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.281 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.281 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.281 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:05.282 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759659+0000", 2024-09-16T20:51:05.282 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.17e-07 2024-09-16T20:51:05.282 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.282 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.282 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:05.282 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759660+0000", 2024-09-16T20:51:05.282 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0189999999999999e-06 2024-09-16T20:51:05.282 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.282 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.282 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:05.282 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759663+0000", 2024-09-16T20:51:05.282 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0010000000000002e-06 2024-09-16T20:51:05.282 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.282 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.282 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:05.283 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759665+0000", 2024-09-16T20:51:05.283 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5440000000000001e-06 2024-09-16T20:51:05.283 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.283 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.283 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:05.283 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825355+0000", 2024-09-16T20:51:05.283 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065690224000000005 2024-09-16T20:51:05.283 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.283 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.283 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:05.283 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825394+0000", 2024-09-16T20:51:05.283 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8810999999999998e-05 2024-09-16T20:51:05.283 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.283 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.283 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:05.283 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825517+0000", 2024-09-16T20:51:05.284 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012270099999999999 2024-09-16T20:51:05.284 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.284 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:05.284 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.284 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.284 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.284 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12561 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 695296~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:05.284 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.759668+0000", 2024-09-16T20:51:05.284 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 74.494677596000002, 2024-09-16T20:51:05.284 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066116999999999995, 2024-09-16T20:51:05.284 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:05.284 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:05.285 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:05.285 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:05.285 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:05.285 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:05.285 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12561 2024-09-16T20:51:05.285 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.285 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:05.285 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.285 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:05.285 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759668+0000", 2024-09-16T20:51:05.285 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.285 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.286 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.286 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:05.286 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759668+0000", 2024-09-16T20:51:05.286 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.286 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.286 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.286 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:05.286 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765379+0000", 2024-09-16T20:51:05.286 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057104410000000001 2024-09-16T20:51:05.286 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.286 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.286 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:05.286 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765383+0000", 2024-09-16T20:51:05.286 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2980000000000002e-06 2024-09-16T20:51:05.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.287 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.287 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:05.287 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765395+0000", 2024-09-16T20:51:05.287 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1671000000000001e-05 2024-09-16T20:51:05.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.287 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.287 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:05.287 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765403+0000", 2024-09-16T20:51:05.287 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1210000000000007e-06 2024-09-16T20:51:05.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.287 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.287 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:05.288 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825552+0000", 2024-09-16T20:51:05.288 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.060149498000000003 2024-09-16T20:51:05.288 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.288 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.288 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:05.288 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825625+0000", 2024-09-16T20:51:05.288 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2540999999999995e-05 2024-09-16T20:51:05.288 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.288 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.288 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:05.288 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825785+0000", 2024-09-16T20:51:05.288 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016043000000000001 2024-09-16T20:51:05.288 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.289 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:05.289 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.289 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.289 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.289 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12585 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 818176~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:05.289 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.765848+0000", 2024-09-16T20:51:05.289 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 74.488497624999994, 2024-09-16T20:51:05.289 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065843218999999995, 2024-09-16T20:51:05.289 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:05.289 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:05.289 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:05.289 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:05.290 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:05.290 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:05.290 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12585 2024-09-16T20:51:05.290 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.290 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:05.290 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.290 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:05.290 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765848+0000", 2024-09-16T20:51:05.290 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.290 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.290 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.290 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:05.290 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765848+0000", 2024-09-16T20:51:05.290 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.291 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.291 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.291 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:05.291 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768683+0000", 2024-09-16T20:51:05.291 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0028344519999999999 2024-09-16T20:51:05.291 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.291 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.291 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:05.291 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768686+0000", 2024-09-16T20:51:05.291 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8789999999999996e-06 2024-09-16T20:51:05.291 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.291 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.291 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:05.291 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768696+0000", 2024-09-16T20:51:05.292 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6709999999999994e-06 2024-09-16T20:51:05.292 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.292 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.292 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:05.292 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768702+0000", 2024-09-16T20:51:05.292 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2550000000000003e-06 2024-09-16T20:51:05.292 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.292 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.292 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:05.292 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.831499+0000", 2024-09-16T20:51:05.292 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.062797081000000005 2024-09-16T20:51:05.292 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.292 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.293 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:05.293 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.831538+0000", 2024-09-16T20:51:05.293 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8523999999999999e-05 2024-09-16T20:51:05.293 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.293 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.293 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:05.293 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.831691+0000", 2024-09-16T20:51:05.293 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015335699999999999 2024-09-16T20:51:05.293 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.293 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:05.293 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.293 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.294 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.294 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12597 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 879616~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:05.294 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.768883+0000", 2024-09-16T20:51:05.294 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 74.485462897000005, 2024-09-16T20:51:05.294 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065816527, 2024-09-16T20:51:05.294 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:05.294 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:05.294 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:05.294 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:05.294 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:05.294 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:05.294 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12597 2024-09-16T20:51:05.294 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.294 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:05.294 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.295 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:05.295 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768883+0000", 2024-09-16T20:51:05.295 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.295 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.295 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.295 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:05.295 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768883+0000", 2024-09-16T20:51:05.295 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.295 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.295 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.295 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:05.295 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771508+0000", 2024-09-16T20:51:05.295 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0026247589999999999 2024-09-16T20:51:05.295 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.296 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.296 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:05.296 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771513+0000", 2024-09-16T20:51:05.296 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9949999999999996e-06 2024-09-16T20:51:05.296 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.296 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.296 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:05.296 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771525+0000", 2024-09-16T20:51:05.296 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2033e-05 2024-09-16T20:51:05.296 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.296 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.296 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:05.296 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771532+0000", 2024-09-16T20:51:05.296 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4220000000000003e-06 2024-09-16T20:51:05.297 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.297 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.297 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:05.297 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.834532+0000", 2024-09-16T20:51:05.297 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.063000159 2024-09-16T20:51:05.297 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.297 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.297 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:05.297 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.834590+0000", 2024-09-16T20:51:05.297 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8218999999999999e-05 2024-09-16T20:51:05.297 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.297 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.297 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:05.297 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.834699+0000", 2024-09-16T20:51:05.297 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010894 2024-09-16T20:51:05.298 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.298 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:05.298 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.298 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.298 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.298 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12619 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 992256~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:05.298 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.774656+0000", 2024-09-16T20:51:05.298 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 74.479690122999997, 2024-09-16T20:51:05.298 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065715859000000001, 2024-09-16T20:51:05.298 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:05.298 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:05.298 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:05.298 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:05.298 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:05.298 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:05.298 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12619 2024-09-16T20:51:05.299 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.299 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:05.299 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.299 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:05.299 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774656+0000", 2024-09-16T20:51:05.299 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.299 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.299 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.299 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:05.299 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774656+0000", 2024-09-16T20:51:05.299 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.299 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.299 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.299 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:05.300 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774875+0000", 2024-09-16T20:51:05.300 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000218898 2024-09-16T20:51:05.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.300 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.300 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:05.300 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774879+0000", 2024-09-16T20:51:05.300 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0389999999999998e-06 2024-09-16T20:51:05.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.300 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.300 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:05.300 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774890+0000", 2024-09-16T20:51:05.300 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1250000000000001e-05 2024-09-16T20:51:05.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.300 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.300 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:05.300 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774897+0000", 2024-09-16T20:51:05.301 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7619999999999998e-06 2024-09-16T20:51:05.301 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.301 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.301 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:05.301 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840139+0000", 2024-09-16T20:51:05.301 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065242438 2024-09-16T20:51:05.301 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.301 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.301 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:05.301 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840186+0000", 2024-09-16T20:51:05.301 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6894999999999997e-05 2024-09-16T20:51:05.301 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.301 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.301 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:05.301 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840372+0000", 2024-09-16T20:51:05.301 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000185577 2024-09-16T20:51:05.302 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.302 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:05.302 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.302 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.302 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.302 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12620 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 997376~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:05.302 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.774908+0000", 2024-09-16T20:51:05.302 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 74.479437902000001, 2024-09-16T20:51:05.302 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065782754999999998, 2024-09-16T20:51:05.302 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:05.302 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:05.302 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:05.302 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:05.302 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:05.302 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:05.303 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12620 2024-09-16T20:51:05.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.303 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:05.303 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.303 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:05.303 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774908+0000", 2024-09-16T20:51:05.303 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.303 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.303 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:05.303 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774908+0000", 2024-09-16T20:51:05.303 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.303 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.303 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:05.303 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775102+0000", 2024-09-16T20:51:05.304 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000194098 2024-09-16T20:51:05.304 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.304 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.304 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:05.304 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775105+0000", 2024-09-16T20:51:05.304 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8739999999999999e-06 2024-09-16T20:51:05.304 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.304 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.304 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:05.304 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775111+0000", 2024-09-16T20:51:05.304 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9519999999999999e-06 2024-09-16T20:51:05.304 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.304 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.304 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:05.304 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775115+0000", 2024-09-16T20:51:05.305 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.002e-06 2024-09-16T20:51:05.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.305 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:05.305 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840409+0000", 2024-09-16T20:51:05.305 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065294195999999999 2024-09-16T20:51:05.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.305 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:05.305 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840507+0000", 2024-09-16T20:51:05.305 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7662000000000001e-05 2024-09-16T20:51:05.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.305 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:05.305 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840691+0000", 2024-09-16T20:51:05.306 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000183971 2024-09-16T20:51:05.306 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.306 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:05.306 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.306 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.306 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.306 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12621 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1002496~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:05.306 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775123+0000", 2024-09-16T20:51:05.306 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 74.479223206, 2024-09-16T20:51:05.306 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065829586999999995, 2024-09-16T20:51:05.306 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:05.306 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:05.306 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:05.307 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:05.307 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:05.307 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:05.307 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12621 2024-09-16T20:51:05.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.307 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:05.307 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.307 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:05.307 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775123+0000", 2024-09-16T20:51:05.307 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.307 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.307 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:05.307 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775123+0000", 2024-09-16T20:51:05.308 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.308 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.308 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:05.308 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775348+0000", 2024-09-16T20:51:05.308 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000225295 2024-09-16T20:51:05.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.308 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.308 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:05.308 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775352+0000", 2024-09-16T20:51:05.308 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0810000000000004e-06 2024-09-16T20:51:05.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.308 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.308 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:05.308 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775361+0000", 2024-09-16T20:51:05.309 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7399999999999993e-06 2024-09-16T20:51:05.309 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.309 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.309 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:05.309 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775367+0000", 2024-09-16T20:51:05.309 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5200000000000003e-06 2024-09-16T20:51:05.309 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.309 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.309 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:05.309 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840730+0000", 2024-09-16T20:51:05.309 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065362803999999997 2024-09-16T20:51:05.309 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.309 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.309 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:05.310 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840801+0000", 2024-09-16T20:51:05.310 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1006000000000004e-05 2024-09-16T20:51:05.310 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.310 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.310 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:05.310 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840952+0000", 2024-09-16T20:51:05.310 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015114100000000001 2024-09-16T20:51:05.310 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.310 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:05.310 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.310 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.310 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.310 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12622 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1007616~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:05.310 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775381+0000", 2024-09-16T20:51:05.310 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 74.478964371999993, 2024-09-16T20:51:05.310 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065822581000000005, 2024-09-16T20:51:05.311 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:05.311 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:05.311 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:05.311 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:05.311 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:05.311 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:05.311 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12622 2024-09-16T20:51:05.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.311 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:05.311 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.311 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:05.312 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775381+0000", 2024-09-16T20:51:05.312 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.312 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.312 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.312 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:05.312 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775381+0000", 2024-09-16T20:51:05.312 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.312 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.312 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.312 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:05.312 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775563+0000", 2024-09-16T20:51:05.312 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018153700000000001 2024-09-16T20:51:05.312 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.312 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.312 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:05.313 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775566+0000", 2024-09-16T20:51:05.313 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9620000000000002e-06 2024-09-16T20:51:05.313 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.313 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.313 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:05.313 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775572+0000", 2024-09-16T20:51:05.313 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8529999999999997e-06 2024-09-16T20:51:05.313 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.313 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.313 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:05.313 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775576+0000", 2024-09-16T20:51:05.313 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8920000000000002e-06 2024-09-16T20:51:05.313 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.313 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.314 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:05.314 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840990+0000", 2024-09-16T20:51:05.314 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065414599000000004 2024-09-16T20:51:05.314 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.314 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.314 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:05.314 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841068+0000", 2024-09-16T20:51:05.314 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8157000000000004e-05 2024-09-16T20:51:05.314 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.314 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.314 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:05.314 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841204+0000", 2024-09-16T20:51:05.314 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013558099999999999 2024-09-16T20:51:05.314 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.315 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:05.315 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.315 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.315 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.315 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12623 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1012736~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:05.315 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775583+0000", 2024-09-16T20:51:05.315 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 74.478762934000002, 2024-09-16T20:51:05.315 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065929087999999997, 2024-09-16T20:51:05.315 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:05.315 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:05.315 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:05.315 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:05.315 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:05.316 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:05.316 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12623 2024-09-16T20:51:05.316 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.316 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:05.316 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.316 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:05.316 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775583+0000", 2024-09-16T20:51:05.316 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.316 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.316 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.316 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:05.316 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775583+0000", 2024-09-16T20:51:05.317 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.317 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.317 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:05.317 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775804+0000", 2024-09-16T20:51:05.317 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00022116900000000001 2024-09-16T20:51:05.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.317 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.317 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:05.317 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775807+0000", 2024-09-16T20:51:05.317 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9529999999999999e-06 2024-09-16T20:51:05.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.317 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.317 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:05.318 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775814+0000", 2024-09-16T20:51:05.318 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7490000000000001e-06 2024-09-16T20:51:05.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.318 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.318 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:05.318 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775818+0000", 2024-09-16T20:51:05.318 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1999999999999996e-06 2024-09-16T20:51:05.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.318 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.318 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:05.318 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841267+0000", 2024-09-16T20:51:05.318 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065449127999999995 2024-09-16T20:51:05.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.318 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.318 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:05.318 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841344+0000", 2024-09-16T20:51:05.319 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6546999999999997e-05 2024-09-16T20:51:05.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.319 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.319 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:05.319 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841512+0000", 2024-09-16T20:51:05.319 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016834200000000001 2024-09-16T20:51:05.319 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.319 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:05.319 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.319 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.319 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12624 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1017856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:05.319 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:51:05.319 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 74.478519793999993, 2024-09-16T20:51:05.320 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065945866000000006, 2024-09-16T20:51:05.320 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:05.320 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:05.320 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:05.320 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:05.320 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:05.320 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:05.320 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12624 2024-09-16T20:51:05.320 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.320 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:05.320 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.320 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:05.320 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:51:05.320 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.321 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.321 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.321 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:05.321 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:51:05.321 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.321 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.321 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.321 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:05.321 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:51:05.321 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-16T20:51:05.321 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.321 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.321 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:05.321 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775828+0000", 2024-09-16T20:51:05.321 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3999999999999999e-06 2024-09-16T20:51:05.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.322 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.322 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:05.322 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775831+0000", 2024-09-16T20:51:05.322 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.478e-06 2024-09-16T20:51:05.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.322 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.322 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:05.322 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775834+0000", 2024-09-16T20:51:05.322 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9440000000000001e-06 2024-09-16T20:51:05.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.322 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.322 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:05.322 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841547+0000", 2024-09-16T20:51:05.322 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065712801000000001 2024-09-16T20:51:05.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.323 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.323 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:05.323 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841618+0000", 2024-09-16T20:51:05.323 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1151999999999999e-05 2024-09-16T20:51:05.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.323 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.323 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:05.323 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841772+0000", 2024-09-16T20:51:05.323 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000153721 2024-09-16T20:51:05.323 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.323 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:05.324 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.324 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.324 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.324 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12625 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1022976~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:05.324 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775840+0000", 2024-09-16T20:51:05.324 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 74.478506185000001, 2024-09-16T20:51:05.324 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066165451, 2024-09-16T20:51:05.324 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:05.324 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:05.324 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:05.324 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:05.324 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:05.324 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:05.324 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12625 2024-09-16T20:51:05.324 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.325 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:05.325 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.325 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:05.325 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775840+0000", 2024-09-16T20:51:05.325 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.325 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.325 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.325 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:05.325 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775840+0000", 2024-09-16T20:51:05.325 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.325 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.325 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.325 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:05.325 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776014+0000", 2024-09-16T20:51:05.325 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017441099999999999 2024-09-16T20:51:05.325 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.326 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.326 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:05.326 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776018+0000", 2024-09-16T20:51:05.326 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6509999999999999e-06 2024-09-16T20:51:05.326 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.326 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.326 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:05.326 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776026+0000", 2024-09-16T20:51:05.326 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.028e-06 2024-09-16T20:51:05.326 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.326 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.326 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:05.326 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776031+0000", 2024-09-16T20:51:05.327 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8400000000000002e-06 2024-09-16T20:51:05.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.327 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.327 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:05.327 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841801+0000", 2024-09-16T20:51:05.327 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065770472999999996 2024-09-16T20:51:05.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.327 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.327 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:05.327 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841847+0000", 2024-09-16T20:51:05.327 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5558e-05 2024-09-16T20:51:05.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.327 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.327 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:05.327 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842005+0000", 2024-09-16T20:51:05.328 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015849000000000001 2024-09-16T20:51:05.328 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.328 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:05.328 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.328 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.328 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.328 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12626 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1028096~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:05.328 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776041+0000", 2024-09-16T20:51:05.328 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 74.478304886000004, 2024-09-16T20:51:05.328 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066225260999999994, 2024-09-16T20:51:05.328 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:05.328 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:05.328 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:05.328 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:05.328 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:05.329 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:05.329 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12626 2024-09-16T20:51:05.329 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.329 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:05.329 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.329 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:05.329 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776041+0000", 2024-09-16T20:51:05.329 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.332 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.332 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.332 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:05.332 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776041+0000", 2024-09-16T20:51:05.332 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.332 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.332 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.332 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:05.332 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776263+0000", 2024-09-16T20:51:05.333 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00022193899999999999 2024-09-16T20:51:05.333 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.333 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.333 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:05.333 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776268+0000", 2024-09-16T20:51:05.333 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8049999999999997e-06 2024-09-16T20:51:05.333 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.333 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.333 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:05.333 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776280+0000", 2024-09-16T20:51:05.333 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2635e-05 2024-09-16T20:51:05.333 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.333 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.333 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:05.333 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776288+0000", 2024-09-16T20:51:05.334 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8199999999999997e-06 2024-09-16T20:51:05.334 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.334 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.334 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:05.334 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842041+0000", 2024-09-16T20:51:05.334 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065752506000000002 2024-09-16T20:51:05.334 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.334 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.334 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:05.334 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842123+0000", 2024-09-16T20:51:05.334 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2393999999999998e-05 2024-09-16T20:51:05.334 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.334 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.334 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:05.335 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842266+0000", 2024-09-16T20:51:05.335 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014316200000000001 2024-09-16T20:51:05.335 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.335 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:05.335 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.335 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.335 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.335 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12627 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1033216~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:05.335 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776301+0000", 2024-09-16T20:51:05.335 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 74.478044925999995, 2024-09-16T20:51:05.335 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066187814999999997, 2024-09-16T20:51:05.335 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:05.335 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:05.335 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:05.335 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:05.336 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:05.336 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:05.336 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12627 2024-09-16T20:51:05.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.336 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:05.336 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.336 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:05.336 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776301+0000", 2024-09-16T20:51:05.336 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.336 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.336 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:05.336 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776301+0000", 2024-09-16T20:51:05.337 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.337 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.337 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:05.337 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776473+0000", 2024-09-16T20:51:05.337 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017222400000000001 2024-09-16T20:51:05.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.337 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.337 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:05.337 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776477+0000", 2024-09-16T20:51:05.337 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7019999999999999e-06 2024-09-16T20:51:05.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.337 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.337 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:05.337 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776486+0000", 2024-09-16T20:51:05.338 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2119999999999992e-06 2024-09-16T20:51:05.338 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.338 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.338 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:05.338 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776492+0000", 2024-09-16T20:51:05.338 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8089999999999998e-06 2024-09-16T20:51:05.338 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.338 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.338 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:05.338 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842301+0000", 2024-09-16T20:51:05.338 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065809042999999998 2024-09-16T20:51:05.338 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.338 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.339 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:05.339 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842362+0000", 2024-09-16T20:51:05.339 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1260000000000006e-05 2024-09-16T20:51:05.339 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.339 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.339 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:05.339 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842489+0000", 2024-09-16T20:51:05.339 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012656500000000001 2024-09-16T20:51:05.339 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.339 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:05.339 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.339 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.339 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.339 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12628 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1038336~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:05.340 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776502+0000", 2024-09-16T20:51:05.340 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 74.477844279999999, 2024-09-16T20:51:05.340 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066283426000000006, 2024-09-16T20:51:05.340 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:05.340 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:05.340 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:05.340 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:05.340 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:05.340 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:05.340 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12628 2024-09-16T20:51:05.340 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.340 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:05.340 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.340 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:05.341 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776502+0000", 2024-09-16T20:51:05.341 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.341 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.341 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.341 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:05.341 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776502+0000", 2024-09-16T20:51:05.341 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.341 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.341 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.341 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:05.341 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776716+0000", 2024-09-16T20:51:05.341 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000214596 2024-09-16T20:51:05.341 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.341 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.342 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:05.342 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776723+0000", 2024-09-16T20:51:05.342 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4000000000000003e-06 2024-09-16T20:51:05.342 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.342 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.342 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:05.342 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776731+0000", 2024-09-16T20:51:05.342 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6129999999999998e-06 2024-09-16T20:51:05.342 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.342 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.342 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:05.342 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776737+0000", 2024-09-16T20:51:05.342 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1970000000000001e-06 2024-09-16T20:51:05.342 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.343 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.343 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:05.343 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842532+0000", 2024-09-16T20:51:05.343 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065795127999999994 2024-09-16T20:51:05.343 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.343 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.343 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:05.343 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842599+0000", 2024-09-16T20:51:05.343 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6711000000000006e-05 2024-09-16T20:51:05.343 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.343 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.343 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:05.343 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842785+0000", 2024-09-16T20:51:05.343 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018578099999999999 2024-09-16T20:51:05.344 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.344 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:05.344 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.344 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.344 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.344 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12629 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1043456~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:05.344 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776748+0000", 2024-09-16T20:51:05.344 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 74.477597903000003, 2024-09-16T20:51:05.344 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066303788000000002, 2024-09-16T20:51:05.344 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:05.344 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:05.344 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:05.344 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:05.345 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:05.345 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:05.345 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12629 2024-09-16T20:51:05.345 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.345 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:05.345 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.345 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:05.345 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776748+0000", 2024-09-16T20:51:05.345 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.345 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.345 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.345 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:05.345 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776748+0000", 2024-09-16T20:51:05.345 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.346 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.346 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.346 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:05.346 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776934+0000", 2024-09-16T20:51:05.346 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018598399999999999 2024-09-16T20:51:05.346 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.346 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.346 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:05.346 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776939+0000", 2024-09-16T20:51:05.346 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1000000000000003e-06 2024-09-16T20:51:05.346 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.346 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.347 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:05.347 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776954+0000", 2024-09-16T20:51:05.347 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5221e-05 2024-09-16T20:51:05.347 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.347 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.347 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:05.347 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776962+0000", 2024-09-16T20:51:05.347 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.554e-06 2024-09-16T20:51:05.347 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.347 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.347 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:05.347 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842813+0000", 2024-09-16T20:51:05.347 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065851006000000004 2024-09-16T20:51:05.347 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.348 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.348 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:05.348 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842886+0000", 2024-09-16T20:51:05.348 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3663000000000003e-05 2024-09-16T20:51:05.348 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.348 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.348 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:05.348 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.843052+0000", 2024-09-16T20:51:05.348 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016526 2024-09-16T20:51:05.348 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.348 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:05.348 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.348 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.348 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:05.349 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:05.350 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:51:05.351 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:05.349+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:05.451 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:05.450+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:05.458 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:05.458 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:51:05.458 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:51:05.458 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:05.467 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:51:05.552 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:05.552+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:05.573 INFO:teuthology.orchestra.run.smithi029.stderr:marked out osd.2. 2024-09-16T20:51:05.618 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:05.619 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:51:05.619 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:51:05.619 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:51:05.619 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:05.627 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:51:05.652 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:05.652+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:05.736 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:05.736 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:51:05.736 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:51:05.736 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:51:05.736 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.736 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5286.0:10643 3.17 3.f6273fb7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-16T20:51:05.736 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:51:05.736 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 116.943729664, 2024-09-16T20:51:05.737 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.76672347200000002, 2024-09-16T20:51:05.737 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:05.737 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:05.737 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:05.737 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:05.737 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5286", 2024-09-16T20:51:05.737 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/4135433989", 2024-09-16T20:51:05.737 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 10643 2024-09-16T20:51:05.737 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.737 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:05.737 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.737 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:05.737 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:51:05.738 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.738 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.738 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.738 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:05.738 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:51:05.738 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.738 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.738 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.738 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:05.738 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790489+0000", 2024-09-16T20:51:05.738 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5910000000000001e-06 2024-09-16T20:51:05.738 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.738 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.738 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:05.739 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790493+0000", 2024-09-16T20:51:05.739 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2669999999999998e-06 2024-09-16T20:51:05.739 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.739 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.739 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:05.739 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790500+0000", 2024-09-16T20:51:05.739 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3780000000000003e-06 2024-09-16T20:51:05.739 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.739 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.739 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:05.739 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790505+0000", 2024-09-16T20:51:05.739 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1560000000000003e-06 2024-09-16T20:51:05.740 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.740 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.740 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:05.740 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790559+0000", 2024-09-16T20:51:05.740 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3494e-05 2024-09-16T20:51:05.740 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.740 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.740 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:05.740 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790566+0000", 2024-09-16T20:51:05.740 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7649999999999993e-06 2024-09-16T20:51:05.740 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.740 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.740 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:05.740 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:09.557198+0000", 2024-09-16T20:51:05.741 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.76663136200000004 2024-09-16T20:51:05.741 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.741 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.741 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:05.741 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:09.557211+0000", 2024-09-16T20:51:05.741 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3458999999999999e-05 2024-09-16T20:51:05.741 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.741 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:05.741 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.741 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.741 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.741 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5512.0:450 3.2 3:44e0cffa:::rbd_header.110ef0d7a535:head [watch reconnect cookie 140263298018912 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:51:05.741 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:51:05.742 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 89.171706751000002, 2024-09-16T20:51:05.742 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0084581939999999, 2024-09-16T20:51:05.742 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:05.742 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:05.742 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:05.742 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:05.742 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5512", 2024-09-16T20:51:05.742 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:51:05.742 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 450 2024-09-16T20:51:05.742 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.742 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:05.742 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.742 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:05.742 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:51:05.743 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.743 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.743 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.743 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:05.743 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:51:05.743 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.743 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.743 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.743 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:05.743 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562512+0000", 2024-09-16T20:51:05.743 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.009e-06 2024-09-16T20:51:05.743 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.744 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.744 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:05.744 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562517+0000", 2024-09-16T20:51:05.744 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9339999999999997e-06 2024-09-16T20:51:05.744 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.744 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.744 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:05.744 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562530+0000", 2024-09-16T20:51:05.744 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3495e-05 2024-09-16T20:51:05.744 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.744 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.744 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:05.744 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562536+0000", 2024-09-16T20:51:05.744 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5690000000000004e-06 2024-09-16T20:51:05.744 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.745 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.745 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:05.745 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562591+0000", 2024-09-16T20:51:05.745 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5761999999999999e-05 2024-09-16T20:51:05.745 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.745 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.745 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:51:05.745 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562599+0000", 2024-09-16T20:51:05.745 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9179999999999994e-06 2024-09-16T20:51:05.745 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.745 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.745 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:05.745 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.565971+0000", 2024-09-16T20:51:05.745 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003371546 2024-09-16T20:51:05.746 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.746 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.746 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:05.746 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566019+0000", 2024-09-16T20:51:05.746 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7849000000000002e-05 2024-09-16T20:51:05.746 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.746 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.746 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:05.746 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.569382+0000", 2024-09-16T20:51:05.746 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.003362833 2024-09-16T20:51:05.746 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.746 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.746 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:05.747 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.569529+0000", 2024-09-16T20:51:05.747 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000147823 2024-09-16T20:51:05.747 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.747 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.747 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:05.747 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570969+0000", 2024-09-16T20:51:05.747 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001439456 2024-09-16T20:51:05.747 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.747 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:05.747 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.747 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.747 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.747 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:14996 3.2 3:44e0cffa:::rbd_header.110ef0d7a535:head [watch reconnect cookie 140213436269840 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:51:05.747 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:51:05.748 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 89.171658339000004, 2024-09-16T20:51:05.748 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.008813943, 2024-09-16T20:51:05.748 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:05.748 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:05.748 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:05.748 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:05.748 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:05.748 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:05.748 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 14996 2024-09-16T20:51:05.748 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.748 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:05.748 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.748 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:05.749 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:51:05.749 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.749 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.749 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.749 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:05.749 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:51:05.749 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.749 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.749 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.749 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:05.749 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562560+0000", 2024-09-16T20:51:05.749 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.06e-07 2024-09-16T20:51:05.749 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.750 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.750 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:05.750 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562563+0000", 2024-09-16T20:51:05.750 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.495e-06 2024-09-16T20:51:05.750 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.750 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.750 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:05.750 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562571+0000", 2024-09-16T20:51:05.750 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1950000000000003e-06 2024-09-16T20:51:05.750 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.750 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.750 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:05.750 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562579+0000", 2024-09-16T20:51:05.750 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2649999999999999e-06 2024-09-16T20:51:05.750 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.751 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.751 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:05.751 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562673+0000", 2024-09-16T20:51:05.751 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4118999999999998e-05 2024-09-16T20:51:05.751 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.751 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.751 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:51:05.751 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562677+0000", 2024-09-16T20:51:05.751 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5179999999999998e-06 2024-09-16T20:51:05.751 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.751 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.751 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:05.751 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566461+0000", 2024-09-16T20:51:05.751 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0037837259999999998 2024-09-16T20:51:05.751 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.752 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.752 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:05.752 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566468+0000", 2024-09-16T20:51:05.752 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3350000000000001e-06 2024-09-16T20:51:05.752 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.752 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.752 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:05.752 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570306+0000", 2024-09-16T20:51:05.752 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.003837997 2024-09-16T20:51:05.752 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.752 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.752 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:05.752 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570392+0000", 2024-09-16T20:51:05.753 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5872999999999996e-05 2024-09-16T20:51:05.753 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.753 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.753 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:05.753 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571373+0000", 2024-09-16T20:51:05.753 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00098071399999999993 2024-09-16T20:51:05.753 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.753 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:05.753 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.753 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.753 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.753 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5255.0:1931 3.2 3:47882819:::rbd_data.110eb5f95afc.0000000000000009:head [read 217088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:51:05.753 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:51:05.753 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 89.171619589000002, 2024-09-16T20:51:05.753 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0082752880000001, 2024-09-16T20:51:05.754 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:05.754 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:05.754 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:05.754 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:05.754 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5255", 2024-09-16T20:51:05.754 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:51:05.754 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1931 2024-09-16T20:51:05.754 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.754 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:05.754 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.754 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:05.754 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:51:05.754 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.754 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.755 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.755 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:05.755 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:51:05.755 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.755 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.755 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:05.755 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:51:05.755 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9699999999999996e-07 2024-09-16T20:51:05.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.755 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.755 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:05.755 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562601+0000", 2024-09-16T20:51:05.755 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8600000000000001e-06 2024-09-16T20:51:05.756 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.756 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.756 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:05.756 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562608+0000", 2024-09-16T20:51:05.756 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9e-06 2024-09-16T20:51:05.756 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.756 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.756 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:05.756 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562615+0000", 2024-09-16T20:51:05.756 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8179999999999998e-06 2024-09-16T20:51:05.756 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.756 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.756 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:05.757 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562703+0000", 2024-09-16T20:51:05.757 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7811000000000003e-05 2024-09-16T20:51:05.757 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.757 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.757 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:51:05.757 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562706+0000", 2024-09-16T20:51:05.757 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4920000000000002e-06 2024-09-16T20:51:05.757 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.757 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.757 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:05.757 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566685+0000", 2024-09-16T20:51:05.757 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0039792780000000002 2024-09-16T20:51:05.757 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.758 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.758 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:05.758 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566693+0000", 2024-09-16T20:51:05.758 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4730000000000003e-06 2024-09-16T20:51:05.758 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.758 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.758 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:05.758 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570737+0000", 2024-09-16T20:51:05.758 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0040436749999999 2024-09-16T20:51:05.758 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.758 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.758 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:05.758 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570873+0000", 2024-09-16T20:51:05.758 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000136484 2024-09-16T20:51:05.758 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.759 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:05.759 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.759 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.759 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.759 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5255.0:1939 3.2 3:47882819:::rbd_data.110eb5f95afc.0000000000000009:head [read 233472~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:51:05.759 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:51:05.759 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 89.171464723, 2024-09-16T20:51:05.759 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0084657589999999, 2024-09-16T20:51:05.759 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:05.759 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:05.759 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:05.759 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:05.759 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5255", 2024-09-16T20:51:05.759 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:51:05.759 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1939 2024-09-16T20:51:05.760 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.760 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:05.760 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.760 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:05.760 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:51:05.760 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.760 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.760 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.760 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:05.760 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:51:05.760 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.760 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.760 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.760 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:05.760 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:51:05.761 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5199999999999997e-07 2024-09-16T20:51:05.761 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.761 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.761 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:05.761 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562757+0000", 2024-09-16T20:51:05.761 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4979999999999998e-06 2024-09-16T20:51:05.761 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.761 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.761 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:05.761 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562764+0000", 2024-09-16T20:51:05.761 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1470000000000004e-06 2024-09-16T20:51:05.761 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.761 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.761 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:05.761 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562768+0000", 2024-09-16T20:51:05.762 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7929999999999999e-06 2024-09-16T20:51:05.762 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.762 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.762 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:05.762 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562805+0000", 2024-09-16T20:51:05.762 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7477e-05 2024-09-16T20:51:05.762 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.762 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.762 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting_for_map not empty", 2024-09-16T20:51:05.762 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562810+0000", 2024-09-16T20:51:05.762 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0819999999999998e-06 2024-09-16T20:51:05.762 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.762 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.762 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:05.763 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566717+0000", 2024-09-16T20:51:05.763 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0039072600000000001 2024-09-16T20:51:05.763 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.763 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.763 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:05.763 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566724+0000", 2024-09-16T20:51:05.763 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4880000000000004e-06 2024-09-16T20:51:05.763 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.763 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.763 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:05.763 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571009+0000", 2024-09-16T20:51:05.763 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.004284677 2024-09-16T20:51:05.763 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.763 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.763 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:05.764 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571218+0000", 2024-09-16T20:51:05.764 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00020978500000000001 2024-09-16T20:51:05.764 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.764 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:05.764 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.764 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.764 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.764 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:11020 3.1f 3:fee2d1dd:::rbd_data.110ea93691bb.000000000000000e:23 [read 955392~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:05.764 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.693811+0000", 2024-09-16T20:51:05.764 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 75.040406066000003, 2024-09-16T20:51:05.764 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056833005999999998, 2024-09-16T20:51:05.764 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:05.764 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:05.764 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:05.765 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:05.765 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:05.765 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:05.765 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 11020 2024-09-16T20:51:05.765 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.765 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:05.765 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.765 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:05.765 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693811+0000", 2024-09-16T20:51:05.765 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.765 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.765 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.766 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:05.766 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693811+0000", 2024-09-16T20:51:05.766 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.766 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.766 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.766 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:05.766 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693812+0000", 2024-09-16T20:51:05.766 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6100000000000001e-07 2024-09-16T20:51:05.766 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.766 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.766 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:05.766 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693815+0000", 2024-09-16T20:51:05.766 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7389999999999999e-06 2024-09-16T20:51:05.767 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.767 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.767 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:05.767 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693823+0000", 2024-09-16T20:51:05.767 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0439999999999996e-06 2024-09-16T20:51:05.767 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.767 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.767 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:05.767 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693828+0000", 2024-09-16T20:51:05.767 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7440000000000004e-06 2024-09-16T20:51:05.767 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.767 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.767 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:05.767 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750357+0000", 2024-09-16T20:51:05.767 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056529192999999998 2024-09-16T20:51:05.768 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.768 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.768 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:05.768 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750429+0000", 2024-09-16T20:51:05.768 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1557000000000006e-05 2024-09-16T20:51:05.768 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.768 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.768 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:05.768 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750644+0000", 2024-09-16T20:51:05.768 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00021526800000000001 2024-09-16T20:51:05.768 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.768 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:05.768 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.769 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.769 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.769 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:11021 3.1f 3:fee2d1dd:::rbd_data.110ea93691bb.000000000000000e:23 [read 960512~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:05.769 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.693837+0000", 2024-09-16T20:51:05.769 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 75.040380178000007, 2024-09-16T20:51:05.769 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057155138000000001, 2024-09-16T20:51:05.769 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:05.769 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:05.769 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:05.769 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:05.769 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:05.769 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:05.769 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 11021 2024-09-16T20:51:05.770 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.770 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:05.770 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.770 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:05.770 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693837+0000", 2024-09-16T20:51:05.770 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.770 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.770 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.770 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:05.770 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693837+0000", 2024-09-16T20:51:05.770 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.770 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.770 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.771 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:05.771 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699566+0000", 2024-09-16T20:51:05.771 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057283639999999997 2024-09-16T20:51:05.771 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.771 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.771 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:05.771 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699571+0000", 2024-09-16T20:51:05.771 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.604e-06 2024-09-16T20:51:05.771 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.771 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.771 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:05.771 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699583+0000", 2024-09-16T20:51:05.771 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1372999999999999e-05 2024-09-16T20:51:05.772 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.772 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.772 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:05.772 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699591+0000", 2024-09-16T20:51:05.772 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5259999999999993e-06 2024-09-16T20:51:05.772 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.772 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.772 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:05.772 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750674+0000", 2024-09-16T20:51:05.772 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051083154999999998 2024-09-16T20:51:05.772 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.772 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.772 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:05.773 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750745+0000", 2024-09-16T20:51:05.773 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0933999999999995e-05 2024-09-16T20:51:05.773 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.773 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.773 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:05.773 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750992+0000", 2024-09-16T20:51:05.773 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00024718199999999998 2024-09-16T20:51:05.773 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.773 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:05.773 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.773 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.773 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.773 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8791 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2332672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:05.773 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057279+0000", 2024-09-16T20:51:05.774 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 64.676938442999997, 2024-09-16T20:51:05.774 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056890067000000002, 2024-09-16T20:51:05.774 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:05.774 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:05.774 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:05.774 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:05.774 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:05.774 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:05.774 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8791 2024-09-16T20:51:05.774 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.774 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:05.774 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.774 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:05.774 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057279+0000", 2024-09-16T20:51:05.775 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.775 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.775 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.775 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:05.775 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057279+0000", 2024-09-16T20:51:05.775 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.775 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.775 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.775 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:05.775 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057279+0000", 2024-09-16T20:51:05.775 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.15e-07 2024-09-16T20:51:05.775 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.775 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.775 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:05.776 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057281+0000", 2024-09-16T20:51:05.776 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8080000000000001e-06 2024-09-16T20:51:05.776 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.776 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.776 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:05.776 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057287+0000", 2024-09-16T20:51:05.776 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5999999999999997e-06 2024-09-16T20:51:05.776 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.776 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.776 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:05.776 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057291+0000", 2024-09-16T20:51:05.776 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5990000000000004e-06 2024-09-16T20:51:05.776 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.776 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.777 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:05.777 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114078+0000", 2024-09-16T20:51:05.777 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056786246999999998 2024-09-16T20:51:05.777 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.777 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.777 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:05.777 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114118+0000", 2024-09-16T20:51:05.777 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9898999999999999e-05 2024-09-16T20:51:05.777 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.777 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.777 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:05.777 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114169+0000", 2024-09-16T20:51:05.777 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1499000000000003e-05 2024-09-16T20:51:05.778 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.778 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:05.778 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.778 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.778 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.778 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8792 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2609152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:05.778 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:51:05.778 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 64.676920460999995, 2024-09-16T20:51:05.778 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057003524999999999, 2024-09-16T20:51:05.778 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:05.778 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:05.778 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:05.778 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:05.778 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:05.779 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:05.779 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8792 2024-09-16T20:51:05.779 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.779 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:05.779 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.779 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:05.779 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:51:05.779 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.779 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.779 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.779 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:05.779 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:51:05.779 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.780 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.780 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.780 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:05.780 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:51:05.780 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0400000000000002e-07 2024-09-16T20:51:05.780 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.780 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.780 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:05.780 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057299+0000", 2024-09-16T20:51:05.780 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7460000000000001e-06 2024-09-16T20:51:05.780 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.780 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.780 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:05.780 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057304+0000", 2024-09-16T20:51:05.781 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3210000000000004e-06 2024-09-16T20:51:05.781 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.781 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.781 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:05.781 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057310+0000", 2024-09-16T20:51:05.781 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4360000000000001e-06 2024-09-16T20:51:05.781 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.781 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.781 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:05.781 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114200+0000", 2024-09-16T20:51:05.781 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056889954 2024-09-16T20:51:05.781 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.781 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.781 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:05.782 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114243+0000", 2024-09-16T20:51:05.782 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2998999999999999e-05 2024-09-16T20:51:05.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.782 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.782 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:05.782 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114300+0000", 2024-09-16T20:51:05.782 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7765e-05 2024-09-16T20:51:05.782 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.782 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:05.782 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.782 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.782 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8794 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2351104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:05.782 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057327+0000", 2024-09-16T20:51:05.783 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 64.676890137000001, 2024-09-16T20:51:05.783 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057101130999999999, 2024-09-16T20:51:05.783 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:05.783 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:05.783 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:05.783 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:05.783 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:05.783 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:05.783 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8794 2024-09-16T20:51:05.784 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.784 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:05.784 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.784 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:05.784 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057327+0000", 2024-09-16T20:51:05.784 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.784 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.784 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.784 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:05.784 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057327+0000", 2024-09-16T20:51:05.784 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.784 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.784 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.784 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:05.785 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057328+0000", 2024-09-16T20:51:05.785 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7399999999999999e-07 2024-09-16T20:51:05.785 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.785 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.785 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:05.785 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057330+0000", 2024-09-16T20:51:05.785 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2199999999999999e-06 2024-09-16T20:51:05.785 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.785 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.785 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:05.785 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057336+0000", 2024-09-16T20:51:05.785 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1240000000000002e-06 2024-09-16T20:51:05.785 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.786 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:05.786 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057341+0000", 2024-09-16T20:51:05.786 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8470000000000003e-06 2024-09-16T20:51:05.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.786 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:05.786 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114334+0000", 2024-09-16T20:51:05.786 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056993267 2024-09-16T20:51:05.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.786 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:05.786 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114376+0000", 2024-09-16T20:51:05.786 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1638999999999999e-05 2024-09-16T20:51:05.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.787 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.787 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:05.787 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114428+0000", 2024-09-16T20:51:05.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2660000000000001e-05 2024-09-16T20:51:05.787 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.787 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:05.787 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.787 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.787 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8798 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2297856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:05.787 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:51:05.787 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 64.676870410000006, 2024-09-16T20:51:05.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057277041000000001, 2024-09-16T20:51:05.788 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:05.788 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:05.788 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:05.788 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:05.788 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:05.788 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:05.788 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8798 2024-09-16T20:51:05.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.788 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:05.788 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.788 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:05.788 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:51:05.788 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.789 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.789 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:05.789 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:51:05.789 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.789 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.789 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:05.789 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:51:05.789 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2800000000000003e-07 2024-09-16T20:51:05.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.789 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.789 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:05.789 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057354+0000", 2024-09-16T20:51:05.790 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7780000000000002e-06 2024-09-16T20:51:05.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.790 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.790 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:05.790 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057360+0000", 2024-09-16T20:51:05.790 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0419999999999999e-06 2024-09-16T20:51:05.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.790 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.790 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:05.790 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057365+0000", 2024-09-16T20:51:05.790 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7929999999999997e-06 2024-09-16T20:51:05.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.790 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.790 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:05.790 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114460+0000", 2024-09-16T20:51:05.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057094662999999997 2024-09-16T20:51:05.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.791 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.791 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:05.791 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114532+0000", 2024-09-16T20:51:05.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2881999999999998e-05 2024-09-16T20:51:05.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.791 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.791 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:05.791 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114624+0000", 2024-09-16T20:51:05.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1555e-05 2024-09-16T20:51:05.791 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.791 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:05.791 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.791 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.792 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8800 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2316288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:05.792 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:51:05.792 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 64.676846471000005, 2024-09-16T20:51:05.792 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.05746946, 2024-09-16T20:51:05.792 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:05.792 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:05.792 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:05.792 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:05.792 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:05.792 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:05.792 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8800 2024-09-16T20:51:05.792 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.792 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:05.792 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.793 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:05.793 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:51:05.793 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.793 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.793 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:05.793 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:51:05.793 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.793 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.793 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:05.793 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:51:05.793 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5199999999999998e-07 2024-09-16T20:51:05.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.793 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.794 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:05.794 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057373+0000", 2024-09-16T20:51:05.794 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.79e-06 2024-09-16T20:51:05.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.794 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:05.794 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057379+0000", 2024-09-16T20:51:05.794 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6699999999999999e-06 2024-09-16T20:51:05.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.794 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:05.794 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057386+0000", 2024-09-16T20:51:05.794 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3170000000000004e-06 2024-09-16T20:51:05.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.795 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:05.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114655+0000", 2024-09-16T20:51:05.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057269420000000001 2024-09-16T20:51:05.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.795 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.795 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:05.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114734+0000", 2024-09-16T20:51:05.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8084000000000007e-05 2024-09-16T20:51:05.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.795 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.795 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:05.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114840+0000", 2024-09-16T20:51:05.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000106827 2024-09-16T20:51:05.795 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.795 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:05.795 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.796 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.796 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8801 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2334720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:05.796 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:51:05.796 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 64.676825440000002, 2024-09-16T20:51:05.796 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057594682000000001, 2024-09-16T20:51:05.796 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:05.796 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:05.796 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:05.796 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:05.796 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:05.796 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:05.797 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8801 2024-09-16T20:51:05.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.797 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:05.797 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.797 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:05.797 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:51:05.797 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.797 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.797 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:05.797 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:51:05.797 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.798 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.798 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:05.798 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:51:05.798 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6899999999999998e-07 2024-09-16T20:51:05.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.798 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.798 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:05.798 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057394+0000", 2024-09-16T20:51:05.798 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8300000000000001e-06 2024-09-16T20:51:05.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.798 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.798 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:05.798 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057400+0000", 2024-09-16T20:51:05.799 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6500000000000001e-06 2024-09-16T20:51:05.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.799 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.799 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:05.799 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057404+0000", 2024-09-16T20:51:05.799 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4669999999999998e-06 2024-09-16T20:51:05.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.799 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.799 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:05.799 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114874+0000", 2024-09-16T20:51:05.799 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057469602000000002 2024-09-16T20:51:05.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.799 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.799 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:05.800 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114933+0000", 2024-09-16T20:51:05.800 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9500000000000003e-05 2024-09-16T20:51:05.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.800 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:05.800 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114987+0000", 2024-09-16T20:51:05.800 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3263999999999997e-05 2024-09-16T20:51:05.800 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.800 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:05.800 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.800 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8802 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2353152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:05.800 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:51:05.801 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 64.676776755000006, 2024-09-16T20:51:05.801 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057714079000000001, 2024-09-16T20:51:05.801 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:05.801 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:05.801 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:05.801 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:05.801 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:05.801 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:05.801 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8802 2024-09-16T20:51:05.801 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.801 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:05.801 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.801 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:05.801 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:51:05.802 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.802 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.802 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.802 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:05.802 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:51:05.802 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.802 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.802 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.802 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:05.802 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:51:05.802 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4999999999999998e-07 2024-09-16T20:51:05.802 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.802 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.802 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:05.802 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057446+0000", 2024-09-16T20:51:05.803 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4240000000000003e-06 2024-09-16T20:51:05.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.803 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.803 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:05.803 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057452+0000", 2024-09-16T20:51:05.803 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8379999999999996e-06 2024-09-16T20:51:05.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.803 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.803 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:05.803 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057460+0000", 2024-09-16T20:51:05.803 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9629999999999998e-06 2024-09-16T20:51:05.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.803 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.803 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:05.803 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115015+0000", 2024-09-16T20:51:05.804 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057554389999999997 2024-09-16T20:51:05.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.804 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:05.804 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115082+0000", 2024-09-16T20:51:05.804 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7733999999999999e-05 2024-09-16T20:51:05.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.804 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:05.804 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115155+0000", 2024-09-16T20:51:05.804 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2280000000000006e-05 2024-09-16T20:51:05.804 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.804 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:05.804 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.805 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.805 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.805 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8804 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2555904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:05.805 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:51:05.805 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 64.676750439000003, 2024-09-16T20:51:05.805 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057833862, 2024-09-16T20:51:05.805 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:05.805 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:05.805 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:05.805 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:05.805 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:05.805 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:05.805 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8804 2024-09-16T20:51:05.805 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.806 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:05.806 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.806 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:05.806 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:51:05.806 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.806 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.806 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.806 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:05.806 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:51:05.806 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.806 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.806 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.806 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:05.806 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:51:05.806 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7899999999999999e-07 2024-09-16T20:51:05.806 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.807 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.807 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:05.807 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057469+0000", 2024-09-16T20:51:05.807 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.001e-06 2024-09-16T20:51:05.807 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.807 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.807 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:05.807 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057475+0000", 2024-09-16T20:51:05.807 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7629999999999997e-06 2024-09-16T20:51:05.807 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.807 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.807 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:05.807 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057480+0000", 2024-09-16T20:51:05.808 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8069999999999999e-06 2024-09-16T20:51:05.808 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.808 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.808 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:05.808 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115183+0000", 2024-09-16T20:51:05.808 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057703364 2024-09-16T20:51:05.808 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.808 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.808 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:05.808 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115227+0000", 2024-09-16T20:51:05.808 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3903999999999998e-05 2024-09-16T20:51:05.808 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.808 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.809 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:05.809 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115301+0000", 2024-09-16T20:51:05.809 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3543999999999999e-05 2024-09-16T20:51:05.809 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.809 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:05.809 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.809 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.809 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8805 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2574336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:05.809 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057487+0000", 2024-09-16T20:51:05.809 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 64.676730110999998, 2024-09-16T20:51:05.809 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058021027000000003, 2024-09-16T20:51:05.809 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:05.810 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:05.810 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:05.810 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:05.810 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:05.810 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:05.810 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8805 2024-09-16T20:51:05.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.810 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:05.810 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.810 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:05.810 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057487+0000", 2024-09-16T20:51:05.810 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.810 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.811 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:05.811 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057487+0000", 2024-09-16T20:51:05.811 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.811 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.811 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.811 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:05.811 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057488+0000", 2024-09-16T20:51:05.811 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5199999999999998e-07 2024-09-16T20:51:05.811 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.811 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.811 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:05.811 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057490+0000", 2024-09-16T20:51:05.811 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.939e-06 2024-09-16T20:51:05.811 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.811 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.811 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:05.812 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057495+0000", 2024-09-16T20:51:05.812 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6439999999999997e-06 2024-09-16T20:51:05.812 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.812 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.812 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:05.812 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057500+0000", 2024-09-16T20:51:05.812 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8119999999999998e-06 2024-09-16T20:51:05.812 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.812 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.812 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:05.812 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115341+0000", 2024-09-16T20:51:05.812 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057841341999999997 2024-09-16T20:51:05.812 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.813 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.813 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:05.813 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115422+0000", 2024-09-16T20:51:05.813 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1003999999999999e-05 2024-09-16T20:51:05.813 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.813 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.813 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:05.813 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115508+0000", 2024-09-16T20:51:05.813 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5933999999999994e-05 2024-09-16T20:51:05.813 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.813 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:05.813 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.813 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.813 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.814 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8807 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2592768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:05.814 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057505+0000", 2024-09-16T20:51:05.814 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 64.676711901000004, 2024-09-16T20:51:05.814 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058186633000000001, 2024-09-16T20:51:05.814 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:05.814 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:05.814 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:05.814 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:05.814 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:05.814 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:05.814 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8807 2024-09-16T20:51:05.814 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.814 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:05.815 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.815 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:05.815 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057505+0000", 2024-09-16T20:51:05.815 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.815 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.815 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.815 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:05.815 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057505+0000", 2024-09-16T20:51:05.815 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.815 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.815 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.815 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:05.815 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057506+0000", 2024-09-16T20:51:05.815 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.27e-07 2024-09-16T20:51:05.816 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.816 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.816 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:05.816 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057508+0000", 2024-09-16T20:51:05.816 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7400000000000001e-06 2024-09-16T20:51:05.816 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.816 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.816 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:05.816 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057513+0000", 2024-09-16T20:51:05.816 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.468e-06 2024-09-16T20:51:05.816 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.816 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.816 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:05.817 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057517+0000", 2024-09-16T20:51:05.817 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4329999999999997e-06 2024-09-16T20:51:05.817 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.817 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.817 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:05.817 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115538+0000", 2024-09-16T20:51:05.817 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058020355000000003 2024-09-16T20:51:05.817 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.817 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.817 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:05.817 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115616+0000", 2024-09-16T20:51:05.817 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7998999999999996e-05 2024-09-16T20:51:05.817 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.817 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.818 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:05.818 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115692+0000", 2024-09-16T20:51:05.818 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6310999999999995e-05 2024-09-16T20:51:05.818 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.818 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:05.818 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.818 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.818 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.818 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8808 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2611200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:05.818 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:51:05.818 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 64.676694639999994, 2024-09-16T20:51:05.818 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058354046, 2024-09-16T20:51:05.818 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:05.818 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:05.818 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:05.819 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:05.819 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:05.819 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:05.819 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8808 2024-09-16T20:51:05.819 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.819 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:05.819 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.819 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:05.819 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:51:05.819 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.819 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.819 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.819 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:05.819 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:51:05.819 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.819 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.820 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.820 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:05.820 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:51:05.820 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2500000000000001e-07 2024-09-16T20:51:05.820 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.820 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.820 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:05.820 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057525+0000", 2024-09-16T20:51:05.820 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5549999999999999e-06 2024-09-16T20:51:05.820 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.820 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.820 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:05.820 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057530+0000", 2024-09-16T20:51:05.820 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5489999999999997e-06 2024-09-16T20:51:05.821 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.821 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.821 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:05.821 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057535+0000", 2024-09-16T20:51:05.821 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5709999999999999e-06 2024-09-16T20:51:05.821 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.821 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.821 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:05.821 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115722+0000", 2024-09-16T20:51:05.821 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058187323999999999 2024-09-16T20:51:05.821 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.821 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.821 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:05.821 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115799+0000", 2024-09-16T20:51:05.821 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6445000000000004e-05 2024-09-16T20:51:05.822 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.822 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.822 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:05.822 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115877+0000", 2024-09-16T20:51:05.822 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8276999999999996e-05 2024-09-16T20:51:05.822 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.822 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:05.822 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.822 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.822 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.822 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:2182 3.17 3:edb5e5cd:::rbd_data.110e61479aee.0000000000000014:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write 0~5120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-16T20:51:05.822 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:22.734609+0000", 2024-09-16T20:51:05.822 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 42.999608393000003, 2024-09-16T20:51:05.822 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.124890894, 2024-09-16T20:51:05.823 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:05.823 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:05.823 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:05.823 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:05.823 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:51:05.823 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:05.823 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 2182 2024-09-16T20:51:05.823 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.823 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:05.823 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.823 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:05.823 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734609+0000", 2024-09-16T20:51:05.823 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.823 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.823 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.824 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:05.824 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734609+0000", 2024-09-16T20:51:05.824 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.824 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.824 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.824 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:05.824 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734611+0000", 2024-09-16T20:51:05.824 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9599999999999999e-06 2024-09-16T20:51:05.824 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.824 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.824 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:05.824 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734628+0000", 2024-09-16T20:51:05.824 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.715e-05 2024-09-16T20:51:05.825 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.825 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.825 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:05.825 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734656+0000", 2024-09-16T20:51:05.825 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7759999999999998e-05 2024-09-16T20:51:05.825 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.825 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.825 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:05.825 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734669+0000", 2024-09-16T20:51:05.825 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3315999999999999e-05 2024-09-16T20:51:05.825 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.825 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.825 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:05.826 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734826+0000", 2024-09-16T20:51:05.826 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015668499999999999 2024-09-16T20:51:05.826 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.826 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.826 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:05.826 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.735048+0000", 2024-09-16T20:51:05.826 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00022179 2024-09-16T20:51:05.826 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.826 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.826 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:05.826 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.735248+0000", 2024-09-16T20:51:05.826 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00020010399999999999 2024-09-16T20:51:05.826 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.826 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.827 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:51:05.827 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.736252+0000", 2024-09-16T20:51:05.827 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010039739999999999 2024-09-16T20:51:05.827 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.827 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.827 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:05.827 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.859340+0000", 2024-09-16T20:51:05.827 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.12308846599999999 2024-09-16T20:51:05.827 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.827 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.827 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:51:05.827 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.859431+0000", 2024-09-16T20:51:05.827 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1099000000000003e-05 2024-09-16T20:51:05.827 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.827 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.828 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:05.828 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.859500+0000", 2024-09-16T20:51:05.828 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8590000000000006e-05 2024-09-16T20:51:05.828 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.828 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:05.828 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.828 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.828 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.828 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:29157 3.1b 3:dbe69130:::journal.110e1b19a241:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e167)", 2024-09-16T20:51:05.828 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:54.663540+0000", 2024-09-16T20:51:05.828 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 11.070677055000001, 2024-09-16T20:51:05.828 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.92912227999999997, 2024-09-16T20:51:05.828 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:05.828 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:05.829 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:05.829 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:05.829 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:05.829 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:05.829 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 29157 2024-09-16T20:51:05.829 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.829 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:05.829 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.829 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:05.829 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663540+0000", 2024-09-16T20:51:05.829 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.829 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.829 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.829 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:05.829 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663540+0000", 2024-09-16T20:51:05.829 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:05.830 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.830 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.830 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:05.830 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663541+0000", 2024-09-16T20:51:05.830 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1680000000000001e-06 2024-09-16T20:51:05.830 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.830 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.830 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:05.830 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663546+0000", 2024-09-16T20:51:05.830 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7129999999999996e-06 2024-09-16T20:51:05.830 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.830 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.830 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:05.831 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663558+0000", 2024-09-16T20:51:05.831 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1347e-05 2024-09-16T20:51:05.831 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.831 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.831 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:05.831 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663563+0000", 2024-09-16T20:51:05.831 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1250000000000001e-06 2024-09-16T20:51:05.831 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.831 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.831 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:05.831 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663612+0000", 2024-09-16T20:51:05.831 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9665e-05 2024-09-16T20:51:05.831 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.831 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.831 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:05.832 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663620+0000", 2024-09-16T20:51:05.832 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8439999999999998e-06 2024-09-16T20:51:05.832 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.832 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.832 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:05.832 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:55.592525+0000", 2024-09-16T20:51:05.832 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.92890456700000001 2024-09-16T20:51:05.832 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:05.832 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:05.832 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:05.832 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:55.592663+0000", 2024-09-16T20:51:05.832 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013785100000000001 2024-09-16T20:51:05.832 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.832 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:05.832 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.833 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:05.833 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:05.833 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:05.833 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T20:51:05.834 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:05.834+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:05.835 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:51:05.935 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:05.934+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:05.999 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:05.999 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:51:06.000 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:51:06.000 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:06.000 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:51:06.001 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:51:06.001 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:51:06.001 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:51:06.007 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:51:06.007 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:51:06.024 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:51:06.024 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:51:06.035 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:06.035+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:06.041 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:51:06.041 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:51:06.058 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:51:06.058 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:51:06.115 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:06.115 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:06.124 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:51:06.169 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:06.169+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:06.231 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:06.231 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:06.237 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:51:06.236+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:51:06.240 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:51:06.338 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:06.337+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:06.387 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:06.387 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:06.396 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:51:06.439 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:06.438+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:06.502 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:06.503 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:06.511 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:51:06.555 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:06.554+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:06.590 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T20:51:06.630 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:06.630 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:06.639 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:51:06.680 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:06.679+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:06.741 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:06.740+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:06.788 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:06.789 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:06.798 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:51:06.841 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:06.840+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:06.942 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:06.942+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:06.945 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:06.945 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:06.954 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:51:07.043 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:07.043+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:07.101 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:07.102 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:07.110 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:51:07.154 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:51:07.153+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:51:07.229 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:07.230 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:07.239 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:51:07.245 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:07.245+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:07.346 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:07.344+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:07.387 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:07.387 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:07.396 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:51:07.446 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:07.446+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:07.544 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:07.545 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:07.546 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:07.545+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:07.553 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:51:07.574 INFO:teuthology.orchestra.run.smithi029.stderr:noscrub is set 2024-09-16T20:51:07.647 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:51:07.646+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:51:07.742 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:07.742 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:07.749 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:07.748+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:07.753 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:51:07.849 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:07.849+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:07.931 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:07.931 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:07.942 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:51:07.986 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:07.985+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:08.051 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:08.051+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:08.070 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T20:51:08.070 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:08.071 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:08.080 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:51:08.152 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:08.152+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:08.216 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:51:08.217 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:51:08.232 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:08.233 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:08.239 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:51:08.239 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:51:08.242 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:51:08.262 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:51:08.262 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:51:08.283 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:51:08.284 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:51:08.285 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:08.284+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:08.300 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:51:08.300 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:51:08.317 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:51:08.317 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:51:08.353 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:51:08.351+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:51:08.357 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:08.357 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:51:08.357 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:51:08.357 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:51:08.357 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:08.366 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:51:08.453 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:08.453+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:08.521 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:08.521 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:51:08.521 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:51:08.522 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:51:08.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.522 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5286.0:10643 3.17 3.f6273fb7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-16T20:51:08.522 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:51:08.522 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 119.729144675, 2024-09-16T20:51:08.522 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.76672347200000002, 2024-09-16T20:51:08.522 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:08.522 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:08.522 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:08.522 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:08.522 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5286", 2024-09-16T20:51:08.522 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/4135433989", 2024-09-16T20:51:08.522 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 10643 2024-09-16T20:51:08.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.523 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:08.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:08.523 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:51:08.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:08.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:08.523 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:51:08.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:08.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:08.523 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790489+0000", 2024-09-16T20:51:08.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5910000000000001e-06 2024-09-16T20:51:08.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.524 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:08.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790493+0000", 2024-09-16T20:51:08.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2669999999999998e-06 2024-09-16T20:51:08.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.524 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:08.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790500+0000", 2024-09-16T20:51:08.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3780000000000003e-06 2024-09-16T20:51:08.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.525 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:08.525 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790505+0000", 2024-09-16T20:51:08.525 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1560000000000003e-06 2024-09-16T20:51:08.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.525 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.525 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:08.525 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790559+0000", 2024-09-16T20:51:08.525 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3494e-05 2024-09-16T20:51:08.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.525 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.525 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:08.525 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790566+0000", 2024-09-16T20:51:08.525 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7649999999999993e-06 2024-09-16T20:51:08.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.526 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.526 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:08.526 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:09.557198+0000", 2024-09-16T20:51:08.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.76663136200000004 2024-09-16T20:51:08.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.526 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.526 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:08.526 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:09.557211+0000", 2024-09-16T20:51:08.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3458999999999999e-05 2024-09-16T20:51:08.526 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:08.526 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:08.526 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:08.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.527 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5512.0:450 3.2 3:44e0cffa:::rbd_header.110ef0d7a535:head [watch reconnect cookie 140263298018912 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:51:08.527 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:51:08.527 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 91.957121762, 2024-09-16T20:51:08.527 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0084581939999999, 2024-09-16T20:51:08.527 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:08.527 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:08.527 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:08.527 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:08.527 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5512", 2024-09-16T20:51:08.527 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:51:08.527 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 450 2024-09-16T20:51:08.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.527 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:08.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:08.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:51:08.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:08.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:08.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:51:08.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:08.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:08.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562512+0000", 2024-09-16T20:51:08.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.009e-06 2024-09-16T20:51:08.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.529 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:08.529 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562517+0000", 2024-09-16T20:51:08.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9339999999999997e-06 2024-09-16T20:51:08.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.529 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:08.529 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562530+0000", 2024-09-16T20:51:08.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3495e-05 2024-09-16T20:51:08.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.529 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:08.529 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562536+0000", 2024-09-16T20:51:08.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5690000000000004e-06 2024-09-16T20:51:08.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.530 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:08.530 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562591+0000", 2024-09-16T20:51:08.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5761999999999999e-05 2024-09-16T20:51:08.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.530 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:51:08.530 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562599+0000", 2024-09-16T20:51:08.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9179999999999994e-06 2024-09-16T20:51:08.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.530 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:08.530 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.565971+0000", 2024-09-16T20:51:08.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003371546 2024-09-16T20:51:08.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:08.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566019+0000", 2024-09-16T20:51:08.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7849000000000002e-05 2024-09-16T20:51:08.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:08.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.569382+0000", 2024-09-16T20:51:08.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.003362833 2024-09-16T20:51:08.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:08.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.569529+0000", 2024-09-16T20:51:08.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000147823 2024-09-16T20:51:08.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.532 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:08.532 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570969+0000", 2024-09-16T20:51:08.532 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001439456 2024-09-16T20:51:08.532 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:08.532 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:08.532 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:08.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.532 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:14996 3.2 3:44e0cffa:::rbd_header.110ef0d7a535:head [watch reconnect cookie 140213436269840 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:51:08.532 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:51:08.532 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 91.957073350000002, 2024-09-16T20:51:08.533 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.008813943, 2024-09-16T20:51:08.533 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:08.533 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:08.533 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:08.533 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:08.533 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:08.533 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:08.533 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 14996 2024-09-16T20:51:08.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.533 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:08.533 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.533 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:08.533 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:51:08.533 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:08.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.534 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:08.534 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:51:08.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:08.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.534 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:08.534 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562560+0000", 2024-09-16T20:51:08.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.06e-07 2024-09-16T20:51:08.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.534 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:08.534 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562563+0000", 2024-09-16T20:51:08.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.495e-06 2024-09-16T20:51:08.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:08.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562571+0000", 2024-09-16T20:51:08.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1950000000000003e-06 2024-09-16T20:51:08.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:08.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562579+0000", 2024-09-16T20:51:08.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2649999999999999e-06 2024-09-16T20:51:08.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:08.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562673+0000", 2024-09-16T20:51:08.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4118999999999998e-05 2024-09-16T20:51:08.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.536 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:51:08.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562677+0000", 2024-09-16T20:51:08.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5179999999999998e-06 2024-09-16T20:51:08.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.536 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:08.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566461+0000", 2024-09-16T20:51:08.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0037837259999999998 2024-09-16T20:51:08.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.536 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:08.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566468+0000", 2024-09-16T20:51:08.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3350000000000001e-06 2024-09-16T20:51:08.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:08.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570306+0000", 2024-09-16T20:51:08.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.003837997 2024-09-16T20:51:08.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:08.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570392+0000", 2024-09-16T20:51:08.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5872999999999996e-05 2024-09-16T20:51:08.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:08.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571373+0000", 2024-09-16T20:51:08.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00098071399999999993 2024-09-16T20:51:08.538 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:08.538 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:08.538 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:08.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.538 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5255.0:1931 3.2 3:47882819:::rbd_data.110eb5f95afc.0000000000000009:head [read 217088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:51:08.538 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:51:08.538 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 91.9570346, 2024-09-16T20:51:08.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0082752880000001, 2024-09-16T20:51:08.538 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:08.538 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:08.538 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:08.538 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:08.538 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5255", 2024-09-16T20:51:08.539 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:51:08.539 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1931 2024-09-16T20:51:08.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.539 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:08.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.539 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:08.539 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:51:08.539 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:08.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.539 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:08.539 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:51:08.539 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:08.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.539 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:08.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:51:08.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9699999999999996e-07 2024-09-16T20:51:08.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:08.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562601+0000", 2024-09-16T20:51:08.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8600000000000001e-06 2024-09-16T20:51:08.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:08.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562608+0000", 2024-09-16T20:51:08.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9e-06 2024-09-16T20:51:08.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.541 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:08.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562615+0000", 2024-09-16T20:51:08.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8179999999999998e-06 2024-09-16T20:51:08.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.541 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:08.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562703+0000", 2024-09-16T20:51:08.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7811000000000003e-05 2024-09-16T20:51:08.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.541 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:51:08.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562706+0000", 2024-09-16T20:51:08.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4920000000000002e-06 2024-09-16T20:51:08.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:08.542 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566685+0000", 2024-09-16T20:51:08.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0039792780000000002 2024-09-16T20:51:08.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:08.542 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566693+0000", 2024-09-16T20:51:08.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4730000000000003e-06 2024-09-16T20:51:08.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:08.543 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570737+0000", 2024-09-16T20:51:08.543 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0040436749999999 2024-09-16T20:51:08.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.543 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.543 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:08.543 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570873+0000", 2024-09-16T20:51:08.543 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000136484 2024-09-16T20:51:08.543 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:08.543 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:08.543 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:08.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.543 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.543 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5255.0:1939 3.2 3:47882819:::rbd_data.110eb5f95afc.0000000000000009:head [read 233472~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:51:08.543 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:51:08.544 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 91.956879733999997, 2024-09-16T20:51:08.544 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0084657589999999, 2024-09-16T20:51:08.544 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:08.544 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:08.544 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:08.544 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:08.544 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5255", 2024-09-16T20:51:08.544 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:51:08.544 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1939 2024-09-16T20:51:08.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.544 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:08.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.544 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:08.544 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:51:08.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:08.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.545 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:08.545 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:51:08.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:08.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.545 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:08.545 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:51:08.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5199999999999997e-07 2024-09-16T20:51:08.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:08.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562757+0000", 2024-09-16T20:51:08.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4979999999999998e-06 2024-09-16T20:51:08.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:08.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562764+0000", 2024-09-16T20:51:08.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1470000000000004e-06 2024-09-16T20:51:08.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:08.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562768+0000", 2024-09-16T20:51:08.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7929999999999999e-06 2024-09-16T20:51:08.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.547 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:08.547 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562805+0000", 2024-09-16T20:51:08.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7477e-05 2024-09-16T20:51:08.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.547 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting_for_map not empty", 2024-09-16T20:51:08.547 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562810+0000", 2024-09-16T20:51:08.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0819999999999998e-06 2024-09-16T20:51:08.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.547 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:08.547 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566717+0000", 2024-09-16T20:51:08.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0039072600000000001 2024-09-16T20:51:08.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.548 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.548 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:08.548 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566724+0000", 2024-09-16T20:51:08.548 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4880000000000004e-06 2024-09-16T20:51:08.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.548 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.548 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:08.548 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571009+0000", 2024-09-16T20:51:08.548 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.004284677 2024-09-16T20:51:08.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.548 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.548 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:08.548 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571218+0000", 2024-09-16T20:51:08.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00020978500000000001 2024-09-16T20:51:08.549 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:08.549 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:08.549 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:08.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.549 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:11020 3.1f 3:fee2d1dd:::rbd_data.110ea93691bb.000000000000000e:23 [read 955392~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:08.549 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.693811+0000", 2024-09-16T20:51:08.549 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 77.825821077000001, 2024-09-16T20:51:08.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056833005999999998, 2024-09-16T20:51:08.549 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:08.549 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:08.549 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:08.549 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:08.549 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:08.550 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:08.550 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 11020 2024-09-16T20:51:08.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.550 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:08.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.550 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:08.550 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693811+0000", 2024-09-16T20:51:08.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:08.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.550 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:08.550 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693811+0000", 2024-09-16T20:51:08.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:08.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.551 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:08.551 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693812+0000", 2024-09-16T20:51:08.551 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6100000000000001e-07 2024-09-16T20:51:08.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.551 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:08.551 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693815+0000", 2024-09-16T20:51:08.551 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7389999999999999e-06 2024-09-16T20:51:08.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.551 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:08.551 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693823+0000", 2024-09-16T20:51:08.551 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0439999999999996e-06 2024-09-16T20:51:08.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.551 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:08.552 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693828+0000", 2024-09-16T20:51:08.552 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7440000000000004e-06 2024-09-16T20:51:08.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.552 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.552 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:08.552 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750357+0000", 2024-09-16T20:51:08.552 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056529192999999998 2024-09-16T20:51:08.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.552 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.552 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:08.552 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750429+0000", 2024-09-16T20:51:08.552 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1557000000000006e-05 2024-09-16T20:51:08.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.553 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:08.553 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750644+0000", 2024-09-16T20:51:08.553 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00021526800000000001 2024-09-16T20:51:08.553 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:08.553 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:08.553 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:08.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.553 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:11021 3.1f 3:fee2d1dd:::rbd_data.110ea93691bb.000000000000000e:23 [read 960512~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:08.553 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.693837+0000", 2024-09-16T20:51:08.553 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 77.825795189000004, 2024-09-16T20:51:08.553 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057155138000000001, 2024-09-16T20:51:08.553 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:08.554 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:08.554 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:08.554 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:08.554 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:08.554 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:08.554 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 11021 2024-09-16T20:51:08.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.554 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:08.554 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.554 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:08.554 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693837+0000", 2024-09-16T20:51:08.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:08.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:08.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693837+0000", 2024-09-16T20:51:08.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:08.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:08.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699566+0000", 2024-09-16T20:51:08.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057283639999999997 2024-09-16T20:51:08.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:08.556 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699571+0000", 2024-09-16T20:51:08.556 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.604e-06 2024-09-16T20:51:08.556 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.556 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.556 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:08.556 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699583+0000", 2024-09-16T20:51:08.556 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1372999999999999e-05 2024-09-16T20:51:08.556 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.556 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.556 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:08.556 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699591+0000", 2024-09-16T20:51:08.556 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5259999999999993e-06 2024-09-16T20:51:08.556 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.557 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.557 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:08.557 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750674+0000", 2024-09-16T20:51:08.557 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051083154999999998 2024-09-16T20:51:08.557 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.557 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.557 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:08.557 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750745+0000", 2024-09-16T20:51:08.557 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0933999999999995e-05 2024-09-16T20:51:08.557 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.557 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.557 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:08.557 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750992+0000", 2024-09-16T20:51:08.557 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00024718199999999998 2024-09-16T20:51:08.558 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:08.558 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:08.558 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:08.558 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.558 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.558 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8791 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2332672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:08.558 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057279+0000", 2024-09-16T20:51:08.558 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 67.462353453999995, 2024-09-16T20:51:08.558 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056890067000000002, 2024-09-16T20:51:08.558 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:08.558 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:08.558 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:08.559 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:08.559 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:08.559 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:08.559 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8791 2024-09-16T20:51:08.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.559 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:08.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:08.559 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057279+0000", 2024-09-16T20:51:08.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:08.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:08.560 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057279+0000", 2024-09-16T20:51:08.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:08.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.560 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:08.560 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057279+0000", 2024-09-16T20:51:08.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.15e-07 2024-09-16T20:51:08.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.560 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:08.560 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057281+0000", 2024-09-16T20:51:08.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8080000000000001e-06 2024-09-16T20:51:08.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.561 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:08.561 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057287+0000", 2024-09-16T20:51:08.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5999999999999997e-06 2024-09-16T20:51:08.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.561 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.561 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:08.561 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057291+0000", 2024-09-16T20:51:08.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5990000000000004e-06 2024-09-16T20:51:08.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.561 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.561 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:08.561 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114078+0000", 2024-09-16T20:51:08.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056786246999999998 2024-09-16T20:51:08.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.562 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:08.562 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114118+0000", 2024-09-16T20:51:08.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9898999999999999e-05 2024-09-16T20:51:08.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.562 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:08.562 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114169+0000", 2024-09-16T20:51:08.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1499000000000003e-05 2024-09-16T20:51:08.562 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:08.562 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:08.562 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:08.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.563 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8792 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2609152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:08.563 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:51:08.563 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 67.462335472000007, 2024-09-16T20:51:08.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057003524999999999, 2024-09-16T20:51:08.563 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:08.563 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:08.563 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:08.563 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:08.563 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:08.563 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:08.563 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8792 2024-09-16T20:51:08.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.563 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:08.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:08.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:51:08.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:08.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:08.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:51:08.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:08.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:08.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:51:08.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0400000000000002e-07 2024-09-16T20:51:08.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.565 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:08.565 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057299+0000", 2024-09-16T20:51:08.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7460000000000001e-06 2024-09-16T20:51:08.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.565 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:08.565 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057304+0000", 2024-09-16T20:51:08.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3210000000000004e-06 2024-09-16T20:51:08.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.565 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:08.565 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057310+0000", 2024-09-16T20:51:08.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4360000000000001e-06 2024-09-16T20:51:08.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.566 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:08.566 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114200+0000", 2024-09-16T20:51:08.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056889954 2024-09-16T20:51:08.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.566 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.566 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:08.566 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114243+0000", 2024-09-16T20:51:08.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2998999999999999e-05 2024-09-16T20:51:08.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.566 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.566 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:08.566 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114300+0000", 2024-09-16T20:51:08.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7765e-05 2024-09-16T20:51:08.566 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:08.566 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:08.567 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:08.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.567 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8794 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2351104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:08.567 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057327+0000", 2024-09-16T20:51:08.567 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 67.462305147999999, 2024-09-16T20:51:08.567 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057101130999999999, 2024-09-16T20:51:08.567 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:08.567 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:08.567 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:08.567 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:08.567 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:08.567 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:08.567 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8794 2024-09-16T20:51:08.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.567 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:08.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.568 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:08.568 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057327+0000", 2024-09-16T20:51:08.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:08.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.568 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:08.568 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057327+0000", 2024-09-16T20:51:08.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:08.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.568 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:08.568 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057328+0000", 2024-09-16T20:51:08.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7399999999999999e-07 2024-09-16T20:51:08.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.569 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:08.569 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057330+0000", 2024-09-16T20:51:08.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2199999999999999e-06 2024-09-16T20:51:08.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.569 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:08.569 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057336+0000", 2024-09-16T20:51:08.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1240000000000002e-06 2024-09-16T20:51:08.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.569 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:08.569 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057341+0000", 2024-09-16T20:51:08.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8470000000000003e-06 2024-09-16T20:51:08.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.570 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:08.570 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114334+0000", 2024-09-16T20:51:08.570 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056993267 2024-09-16T20:51:08.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.570 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:08.570 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114376+0000", 2024-09-16T20:51:08.570 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1638999999999999e-05 2024-09-16T20:51:08.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.570 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:08.570 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114428+0000", 2024-09-16T20:51:08.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2660000000000001e-05 2024-09-16T20:51:08.571 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:08.571 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:08.571 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:08.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.571 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.571 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8798 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2297856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:08.571 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:51:08.571 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 67.462285421000004, 2024-09-16T20:51:08.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057277041000000001, 2024-09-16T20:51:08.571 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:08.571 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:08.571 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:08.571 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:08.571 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:08.572 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:08.572 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8798 2024-09-16T20:51:08.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.572 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:08.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.572 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:08.572 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:51:08.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:08.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.572 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:08.572 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:51:08.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:08.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.573 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:08.573 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:51:08.573 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2800000000000003e-07 2024-09-16T20:51:08.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.573 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.573 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:08.573 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057354+0000", 2024-09-16T20:51:08.573 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7780000000000002e-06 2024-09-16T20:51:08.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.573 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.573 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:08.573 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057360+0000", 2024-09-16T20:51:08.573 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0419999999999999e-06 2024-09-16T20:51:08.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.573 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.574 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:08.574 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057365+0000", 2024-09-16T20:51:08.574 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7929999999999997e-06 2024-09-16T20:51:08.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.574 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:08.574 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114460+0000", 2024-09-16T20:51:08.574 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057094662999999997 2024-09-16T20:51:08.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.574 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:08.574 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114532+0000", 2024-09-16T20:51:08.574 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2881999999999998e-05 2024-09-16T20:51:08.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.574 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:08.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114624+0000", 2024-09-16T20:51:08.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1555e-05 2024-09-16T20:51:08.575 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:08.575 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:08.575 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:08.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.575 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8800 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2316288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:08.575 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:51:08.575 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 67.462261482000002, 2024-09-16T20:51:08.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.05746946, 2024-09-16T20:51:08.575 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:08.575 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:08.575 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:08.575 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:08.575 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:08.576 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:08.576 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8800 2024-09-16T20:51:08.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.576 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:08.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:08.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:51:08.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:08.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:08.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:51:08.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:08.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.577 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:08.577 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:51:08.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5199999999999998e-07 2024-09-16T20:51:08.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.577 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:08.577 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057373+0000", 2024-09-16T20:51:08.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.79e-06 2024-09-16T20:51:08.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.577 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:08.577 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057379+0000", 2024-09-16T20:51:08.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6699999999999999e-06 2024-09-16T20:51:08.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.578 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:08.578 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057386+0000", 2024-09-16T20:51:08.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3170000000000004e-06 2024-09-16T20:51:08.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.578 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:08.578 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114655+0000", 2024-09-16T20:51:08.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057269420000000001 2024-09-16T20:51:08.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.578 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:08.578 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114734+0000", 2024-09-16T20:51:08.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8084000000000007e-05 2024-09-16T20:51:08.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:08.579 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114840+0000", 2024-09-16T20:51:08.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000106827 2024-09-16T20:51:08.579 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:08.579 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:08.579 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:08.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.579 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8801 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2334720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:08.579 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:51:08.579 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 67.462240451, 2024-09-16T20:51:08.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057594682000000001, 2024-09-16T20:51:08.579 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:08.580 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:08.580 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:08.580 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:08.580 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:08.580 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:08.580 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8801 2024-09-16T20:51:08.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.580 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:08.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:08.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:51:08.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:08.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:08.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:51:08.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:08.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:08.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:51:08.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6899999999999998e-07 2024-09-16T20:51:08.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:08.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057394+0000", 2024-09-16T20:51:08.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8300000000000001e-06 2024-09-16T20:51:08.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:08.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057400+0000", 2024-09-16T20:51:08.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6500000000000001e-06 2024-09-16T20:51:08.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:08.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057404+0000", 2024-09-16T20:51:08.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4669999999999998e-06 2024-09-16T20:51:08.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:08.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114874+0000", 2024-09-16T20:51:08.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057469602000000002 2024-09-16T20:51:08.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:08.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114933+0000", 2024-09-16T20:51:08.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9500000000000003e-05 2024-09-16T20:51:08.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:08.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114987+0000", 2024-09-16T20:51:08.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3263999999999997e-05 2024-09-16T20:51:08.583 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:08.583 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:08.583 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:08.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.583 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8802 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2353152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:08.583 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:51:08.584 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 67.462191766000004, 2024-09-16T20:51:08.584 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057714079000000001, 2024-09-16T20:51:08.584 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:08.584 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:08.584 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:08.584 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:08.584 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:08.584 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:08.584 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8802 2024-09-16T20:51:08.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.584 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:08.584 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.584 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:08.584 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:51:08.584 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:08.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:08.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:51:08.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:08.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:08.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:51:08.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4999999999999998e-07 2024-09-16T20:51:08.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:08.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057446+0000", 2024-09-16T20:51:08.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4240000000000003e-06 2024-09-16T20:51:08.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:08.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057452+0000", 2024-09-16T20:51:08.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8379999999999996e-06 2024-09-16T20:51:08.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:08.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057460+0000", 2024-09-16T20:51:08.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9629999999999998e-06 2024-09-16T20:51:08.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:08.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115015+0000", 2024-09-16T20:51:08.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057554389999999997 2024-09-16T20:51:08.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:08.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115082+0000", 2024-09-16T20:51:08.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7733999999999999e-05 2024-09-16T20:51:08.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:08.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115155+0000", 2024-09-16T20:51:08.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2280000000000006e-05 2024-09-16T20:51:08.587 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:08.587 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:08.587 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:08.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.587 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8804 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2555904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:08.588 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:51:08.588 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 67.462165450000001, 2024-09-16T20:51:08.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057833862, 2024-09-16T20:51:08.588 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:08.588 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:08.588 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:08.588 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:08.588 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:08.588 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:08.588 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8804 2024-09-16T20:51:08.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.588 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:08.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:08.589 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:51:08.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:08.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:08.589 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:51:08.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:08.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:08.589 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:51:08.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7899999999999999e-07 2024-09-16T20:51:08.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.590 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:08.590 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057469+0000", 2024-09-16T20:51:08.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.001e-06 2024-09-16T20:51:08.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.590 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:08.590 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057475+0000", 2024-09-16T20:51:08.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7629999999999997e-06 2024-09-16T20:51:08.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.590 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:08.590 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057480+0000", 2024-09-16T20:51:08.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8069999999999999e-06 2024-09-16T20:51:08.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.591 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:08.591 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115183+0000", 2024-09-16T20:51:08.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057703364 2024-09-16T20:51:08.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.591 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:08.591 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115227+0000", 2024-09-16T20:51:08.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3903999999999998e-05 2024-09-16T20:51:08.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.591 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:08.591 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115301+0000", 2024-09-16T20:51:08.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3543999999999999e-05 2024-09-16T20:51:08.592 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:08.592 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:08.592 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:08.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.592 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8805 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2574336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:08.592 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057487+0000", 2024-09-16T20:51:08.592 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 67.462145121999995, 2024-09-16T20:51:08.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058021027000000003, 2024-09-16T20:51:08.592 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:08.592 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:08.592 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:08.592 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:08.592 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:08.593 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:08.593 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8805 2024-09-16T20:51:08.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.593 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:08.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.593 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:08.593 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057487+0000", 2024-09-16T20:51:08.593 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:08.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.593 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:08.593 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057487+0000", 2024-09-16T20:51:08.593 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:08.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.594 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:08.594 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057488+0000", 2024-09-16T20:51:08.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5199999999999998e-07 2024-09-16T20:51:08.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.594 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:08.594 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057490+0000", 2024-09-16T20:51:08.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.939e-06 2024-09-16T20:51:08.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.594 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:08.594 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057495+0000", 2024-09-16T20:51:08.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6439999999999997e-06 2024-09-16T20:51:08.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.595 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:08.595 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057500+0000", 2024-09-16T20:51:08.595 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8119999999999998e-06 2024-09-16T20:51:08.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.595 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:08.595 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115341+0000", 2024-09-16T20:51:08.595 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057841341999999997 2024-09-16T20:51:08.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.595 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:08.595 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115422+0000", 2024-09-16T20:51:08.596 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1003999999999999e-05 2024-09-16T20:51:08.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.596 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.596 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:08.596 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115508+0000", 2024-09-16T20:51:08.596 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5933999999999994e-05 2024-09-16T20:51:08.596 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:08.596 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:08.596 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:08.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.597 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.597 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8807 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2592768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:08.597 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057505+0000", 2024-09-16T20:51:08.597 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 67.462126912000002, 2024-09-16T20:51:08.597 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058186633000000001, 2024-09-16T20:51:08.597 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:08.597 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:08.597 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:08.597 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:08.597 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:08.597 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:08.598 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8807 2024-09-16T20:51:08.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.598 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:08.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.598 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:08.598 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057505+0000", 2024-09-16T20:51:08.598 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:08.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.598 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:08.598 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057505+0000", 2024-09-16T20:51:08.598 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:08.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.599 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:08.599 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057506+0000", 2024-09-16T20:51:08.599 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.27e-07 2024-09-16T20:51:08.599 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.599 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:08.599 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057508+0000", 2024-09-16T20:51:08.599 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7400000000000001e-06 2024-09-16T20:51:08.599 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.600 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:08.600 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057513+0000", 2024-09-16T20:51:08.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.468e-06 2024-09-16T20:51:08.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.600 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.600 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:08.600 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057517+0000", 2024-09-16T20:51:08.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4329999999999997e-06 2024-09-16T20:51:08.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.600 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.600 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:08.600 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115538+0000", 2024-09-16T20:51:08.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058020355000000003 2024-09-16T20:51:08.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.601 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:08.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115616+0000", 2024-09-16T20:51:08.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7998999999999996e-05 2024-09-16T20:51:08.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.601 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:08.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115692+0000", 2024-09-16T20:51:08.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6310999999999995e-05 2024-09-16T20:51:08.601 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:08.601 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:08.601 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:08.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.602 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8808 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2611200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:08.602 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:51:08.602 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 67.462109651000006, 2024-09-16T20:51:08.602 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058354046, 2024-09-16T20:51:08.602 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:08.602 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:08.602 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:08.602 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:08.602 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:08.602 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:08.602 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8808 2024-09-16T20:51:08.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.602 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:08.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.603 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:08.603 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:51:08.603 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:08.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.603 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:08.603 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:51:08.603 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:08.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.603 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:08.603 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:51:08.603 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2500000000000001e-07 2024-09-16T20:51:08.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.604 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:08.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057525+0000", 2024-09-16T20:51:08.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5549999999999999e-06 2024-09-16T20:51:08.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.604 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:08.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057530+0000", 2024-09-16T20:51:08.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5489999999999997e-06 2024-09-16T20:51:08.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.604 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:08.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057535+0000", 2024-09-16T20:51:08.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5709999999999999e-06 2024-09-16T20:51:08.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:08.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115722+0000", 2024-09-16T20:51:08.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058187323999999999 2024-09-16T20:51:08.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:08.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115799+0000", 2024-09-16T20:51:08.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6445000000000004e-05 2024-09-16T20:51:08.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:08.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115877+0000", 2024-09-16T20:51:08.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8276999999999996e-05 2024-09-16T20:51:08.606 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:08.606 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:08.606 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:08.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.606 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:2182 3.17 3:edb5e5cd:::rbd_data.110e61479aee.0000000000000014:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write 0~5120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-16T20:51:08.606 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:22.734609+0000", 2024-09-16T20:51:08.606 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 45.785023404, 2024-09-16T20:51:08.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.124890894, 2024-09-16T20:51:08.606 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:08.606 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:08.606 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:08.606 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:08.606 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:51:08.607 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:08.607 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 2182 2024-09-16T20:51:08.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.607 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:08.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.607 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:08.607 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734609+0000", 2024-09-16T20:51:08.607 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:08.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.607 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:08.607 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734609+0000", 2024-09-16T20:51:08.607 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:08.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.608 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:08.608 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734611+0000", 2024-09-16T20:51:08.608 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9599999999999999e-06 2024-09-16T20:51:08.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.608 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:08.608 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734628+0000", 2024-09-16T20:51:08.608 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.715e-05 2024-09-16T20:51:08.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.608 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:08.608 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734656+0000", 2024-09-16T20:51:08.608 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7759999999999998e-05 2024-09-16T20:51:08.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:08.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734669+0000", 2024-09-16T20:51:08.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3315999999999999e-05 2024-09-16T20:51:08.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:08.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734826+0000", 2024-09-16T20:51:08.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015668499999999999 2024-09-16T20:51:08.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:08.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.735048+0000", 2024-09-16T20:51:08.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00022179 2024-09-16T20:51:08.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.610 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:08.610 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.735248+0000", 2024-09-16T20:51:08.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00020010399999999999 2024-09-16T20:51:08.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.610 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:51:08.610 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.736252+0000", 2024-09-16T20:51:08.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010039739999999999 2024-09-16T20:51:08.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.612 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:08.612 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.859340+0000", 2024-09-16T20:51:08.612 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.12308846599999999 2024-09-16T20:51:08.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.612 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:51:08.612 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.859431+0000", 2024-09-16T20:51:08.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1099000000000003e-05 2024-09-16T20:51:08.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.613 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:08.613 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.859500+0000", 2024-09-16T20:51:08.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8590000000000006e-05 2024-09-16T20:51:08.613 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:08.613 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:08.613 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:08.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.613 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:29157 3.1b 3:dbe69130:::journal.110e1b19a241:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e167)", 2024-09-16T20:51:08.613 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:54.663540+0000", 2024-09-16T20:51:08.613 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 13.856092066, 2024-09-16T20:51:08.614 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.92912227999999997, 2024-09-16T20:51:08.614 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:08.614 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:08.614 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:08.614 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:08.614 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:08.614 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:08.614 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 29157 2024-09-16T20:51:08.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.614 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:08.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.614 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:08.614 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663540+0000", 2024-09-16T20:51:08.614 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:08.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.615 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:08.615 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663540+0000", 2024-09-16T20:51:08.615 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:08.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.615 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.615 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:08.615 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663541+0000", 2024-09-16T20:51:08.615 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1680000000000001e-06 2024-09-16T20:51:08.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.615 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.615 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:08.615 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663546+0000", 2024-09-16T20:51:08.615 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7129999999999996e-06 2024-09-16T20:51:08.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.615 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.616 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:08.616 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663558+0000", 2024-09-16T20:51:08.616 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1347e-05 2024-09-16T20:51:08.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.616 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.616 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:08.616 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663563+0000", 2024-09-16T20:51:08.616 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1250000000000001e-06 2024-09-16T20:51:08.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.616 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.616 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:08.616 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663612+0000", 2024-09-16T20:51:08.616 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9665e-05 2024-09-16T20:51:08.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.617 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:08.617 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663620+0000", 2024-09-16T20:51:08.617 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8439999999999998e-06 2024-09-16T20:51:08.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.617 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:08.617 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:55.592525+0000", 2024-09-16T20:51:08.617 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.92890456700000001 2024-09-16T20:51:08.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:08.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:08.617 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:08.617 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:55.592663+0000", 2024-09-16T20:51:08.617 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013785100000000001 2024-09-16T20:51:08.617 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:08.618 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:08.618 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:08.618 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:08.618 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:08.618 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:08.619 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:51:08.620 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:08.619+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:08.720 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:08.720+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:08.726 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:08.726 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:51:08.726 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:51:08.726 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:08.735 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:51:08.820 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:08.820+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:08.882 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:08.883 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:51:08.883 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:51:08.883 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:51:08.883 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:08.891 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:51:08.921 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:08.920+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:09.023 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:09.022+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:09.039 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:09.039 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:51:09.039 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:51:09.039 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:51:09.039 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.040 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5439 3.a 3:5789f5c3:::rbd_data.110ed9ca540.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 1181696~60416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:09.040 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:51:09.040 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 166.173042256, 2024-09-16T20:51:09.040 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10102045599999999, 2024-09-16T20:51:09.040 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:09.040 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:09.040 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:09.040 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:09.040 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:09.040 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:09.040 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5439 2024-09-16T20:51:09.040 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.040 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:09.041 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.041 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:09.041 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:51:09.041 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.041 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.041 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.041 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:09.041 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:51:09.041 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.041 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.041 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.041 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:09.041 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864711+0000", 2024-09-16T20:51:09.042 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7899999999999997e-07 2024-09-16T20:51:09.042 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.042 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.042 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:09.042 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864760+0000", 2024-09-16T20:51:09.042 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8803e-05 2024-09-16T20:51:09.042 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.042 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.042 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:09.042 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864769+0000", 2024-09-16T20:51:09.042 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0999999999999993e-06 2024-09-16T20:51:09.042 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.042 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.042 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:09.043 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864774+0000", 2024-09-16T20:51:09.043 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8919999999999999e-06 2024-09-16T20:51:09.043 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.043 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.043 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:09.043 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864815+0000", 2024-09-16T20:51:09.043 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1307000000000003e-05 2024-09-16T20:51:09.043 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.043 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.043 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:09.043 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864893+0000", 2024-09-16T20:51:09.043 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7581000000000001e-05 2024-09-16T20:51:09.043 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.044 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.044 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 1", 2024-09-16T20:51:09.044 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864996+0000", 2024-09-16T20:51:09.044 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102725 2024-09-16T20:51:09.044 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.044 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.044 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:09.044 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965442+0000", 2024-09-16T20:51:09.044 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100446012 2024-09-16T20:51:09.044 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.044 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.044 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:09.044 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965618+0000", 2024-09-16T20:51:09.044 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00017653100000000001 2024-09-16T20:51:09.045 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.045 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.045 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:09.045 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965682+0000", 2024-09-16T20:51:09.045 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3905999999999994e-05 2024-09-16T20:51:09.045 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.045 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.045 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:09.045 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965731+0000", 2024-09-16T20:51:09.045 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8819999999999997e-05 2024-09-16T20:51:09.045 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.045 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:09.045 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.046 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.046 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.046 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4600.0:5439 3.a e37/23 ondisk, result = 0)", 2024-09-16T20:51:09.046 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:51:09.046 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 166.17204610799999, 2024-09-16T20:51:09.046 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099742634999999996, 2024-09-16T20:51:09.046 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:09.046 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:09.046 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:09.046 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:09.046 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.046 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:09.046 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:51:09.046 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.046 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.047 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.047 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:09.047 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:51:09.047 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.047 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.047 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.047 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:09.047 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:51:09.047 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0900000000000001e-07 2024-09-16T20:51:09.047 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.047 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.047 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:09.047 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865711+0000", 2024-09-16T20:51:09.047 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3739999999999998e-06 2024-09-16T20:51:09.048 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.048 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.048 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:09.048 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965343+0000", 2024-09-16T20:51:09.048 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099632526999999999 2024-09-16T20:51:09.048 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.048 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.048 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:09.048 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965357+0000", 2024-09-16T20:51:09.048 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3713999999999999e-05 2024-09-16T20:51:09.048 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.048 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.048 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:09.049 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965426+0000", 2024-09-16T20:51:09.049 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8745999999999995e-05 2024-09-16T20:51:09.049 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.049 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.049 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:09.049 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965434+0000", 2024-09-16T20:51:09.049 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6910000000000003e-06 2024-09-16T20:51:09.049 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.049 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.049 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:09.049 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965449+0000", 2024-09-16T20:51:09.049 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4874e-05 2024-09-16T20:51:09.049 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.049 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:09.050 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.050 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.050 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.050 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4882.0:2508 3.12 3:4992771f:::rbd_object_map.110e7c8851c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T20:51:09.050 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:51:09.050 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 142.55624255999999, 2024-09-16T20:51:09.050 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.125007011, 2024-09-16T20:51:09.050 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:09.050 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:09.050 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:09.050 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:09.050 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4882", 2024-09-16T20:51:09.050 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2436923223", 2024-09-16T20:51:09.050 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2508 2024-09-16T20:51:09.050 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.051 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:09.051 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.051 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:09.051 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:51:09.051 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.051 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.051 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.051 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:09.051 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:51:09.051 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.051 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.051 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.051 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:09.051 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481511+0000", 2024-09-16T20:51:09.052 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0699999999999996e-07 2024-09-16T20:51:09.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.052 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.052 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:09.052 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481515+0000", 2024-09-16T20:51:09.052 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0609999999999997e-06 2024-09-16T20:51:09.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.052 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.052 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:09.052 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481523+0000", 2024-09-16T20:51:09.052 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1310000000000006e-06 2024-09-16T20:51:09.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.052 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.052 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:09.053 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481528+0000", 2024-09-16T20:51:09.053 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4900000000000002e-06 2024-09-16T20:51:09.053 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.053 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.053 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:09.053 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481565+0000", 2024-09-16T20:51:09.053 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7066000000000001e-05 2024-09-16T20:51:09.053 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.053 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.053 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:09.053 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481628+0000", 2024-09-16T20:51:09.053 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3118000000000001e-05 2024-09-16T20:51:09.053 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.054 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.054 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 2", 2024-09-16T20:51:09.054 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481822+0000", 2024-09-16T20:51:09.054 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019446200000000001 2024-09-16T20:51:09.054 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.054 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.054 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:09.054 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605988+0000", 2024-09-16T20:51:09.054 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.124165888 2024-09-16T20:51:09.054 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.054 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.054 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:09.054 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606427+0000", 2024-09-16T20:51:09.055 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00043854899999999999 2024-09-16T20:51:09.055 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.055 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.055 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:09.055 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606476+0000", 2024-09-16T20:51:09.055 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8971999999999998e-05 2024-09-16T20:51:09.055 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.055 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.055 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:09.055 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606517+0000", 2024-09-16T20:51:09.055 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1467000000000001e-05 2024-09-16T20:51:09.055 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.055 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:09.055 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.056 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.056 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.056 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4882.0:2508 3.12 e57/55 ondisk, result = 0)", 2024-09-16T20:51:09.056 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:51:09.056 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 142.55525406000001, 2024-09-16T20:51:09.056 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12349711400000001, 2024-09-16T20:51:09.056 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:09.056 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:09.056 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:09.056 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:09.056 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.056 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:09.056 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:51:09.056 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.057 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.057 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.057 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:09.057 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:51:09.057 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.057 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.057 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.057 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:09.057 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:51:09.057 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.06e-07 2024-09-16T20:51:09.057 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.057 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.057 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:09.057 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482502+0000", 2024-09-16T20:51:09.058 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.01e-06 2024-09-16T20:51:09.058 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.058 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.058 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:09.058 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605772+0000", 2024-09-16T20:51:09.058 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.123269687 2024-09-16T20:51:09.058 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.058 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.058 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:09.058 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605791+0000", 2024-09-16T20:51:09.058 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8912000000000001e-05 2024-09-16T20:51:09.058 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.058 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.058 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:09.059 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605974+0000", 2024-09-16T20:51:09.059 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000182836 2024-09-16T20:51:09.059 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.059 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.059 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:09.059 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605982+0000", 2024-09-16T20:51:09.059 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1699999999999997e-06 2024-09-16T20:51:09.059 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.059 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.059 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:09.059 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605996+0000", 2024-09-16T20:51:09.059 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3893000000000001e-05 2024-09-16T20:51:09.059 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.059 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:09.059 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.060 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.060 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.060 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4882.0:2512 3.f 3:f4d895b7:::rbd_data.110e7c8851c.0000000000000000:head [read 16384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T20:51:09.060 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:51:09.060 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 142.55481703000001, 2024-09-16T20:51:09.060 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12318483, 2024-09-16T20:51:09.060 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:09.060 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:09.060 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:09.060 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:09.060 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4882", 2024-09-16T20:51:09.060 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2436923223", 2024-09-16T20:51:09.060 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2512 2024-09-16T20:51:09.060 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.061 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:09.061 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.061 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:09.061 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:51:09.061 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.061 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.061 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.061 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:09.061 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:51:09.061 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.061 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.061 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.061 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:09.062 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:51:09.062 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3499999999999995e-07 2024-09-16T20:51:09.062 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.062 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.062 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:09.062 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482943+0000", 2024-09-16T20:51:09.062 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3450000000000002e-06 2024-09-16T20:51:09.062 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.062 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.062 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:09.062 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605772+0000", 2024-09-16T20:51:09.062 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.122829796 2024-09-16T20:51:09.062 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.063 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.063 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:09.063 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605792+0000", 2024-09-16T20:51:09.063 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9916999999999999e-05 2024-09-16T20:51:09.063 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.063 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.063 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:09.063 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605917+0000", 2024-09-16T20:51:09.063 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00012500799999999999 2024-09-16T20:51:09.063 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.063 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.063 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:09.063 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606004+0000", 2024-09-16T20:51:09.064 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6242999999999994e-05 2024-09-16T20:51:09.064 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.064 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.064 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:09.064 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606120+0000", 2024-09-16T20:51:09.064 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000116786 2024-09-16T20:51:09.064 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.064 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:09.064 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.064 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.064 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.064 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:11702 3.15 3:acf33c8b:::rbd_data.110e747f0779.000000000000001a:head [set-alloc-hint object_size 262144 write_size 262144,write 143360~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:51:09.064 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:51:09.064 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 124.899824608, 2024-09-16T20:51:09.065 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.084801043000000006, 2024-09-16T20:51:09.065 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:09.065 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:09.065 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:09.065 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:09.065 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:51:09.065 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:51:09.065 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 11702 2024-09-16T20:51:09.065 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.065 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:09.065 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.065 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:09.065 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:51:09.065 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.066 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.066 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.066 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:09.066 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:51:09.066 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.066 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.066 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.066 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:09.066 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137929+0000", 2024-09-16T20:51:09.066 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8400000000000002e-07 2024-09-16T20:51:09.066 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.066 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.067 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:09.067 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137937+0000", 2024-09-16T20:51:09.067 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0260000000000007e-06 2024-09-16T20:51:09.067 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.067 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.067 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:09.067 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137952+0000", 2024-09-16T20:51:09.067 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4659e-05 2024-09-16T20:51:09.067 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.067 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.067 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:09.067 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137960+0000", 2024-09-16T20:51:09.067 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0169999999999996e-06 2024-09-16T20:51:09.067 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.068 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.068 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:09.068 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138010+0000", 2024-09-16T20:51:09.068 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0021000000000001e-05 2024-09-16T20:51:09.068 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.068 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.068 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:09.068 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138114+0000", 2024-09-16T20:51:09.068 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000103797 2024-09-16T20:51:09.068 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.068 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.068 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:51:09.068 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138233+0000", 2024-09-16T20:51:09.069 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000118971 2024-09-16T20:51:09.069 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.069 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.069 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:09.069 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138953+0000", 2024-09-16T20:51:09.069 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00072026000000000002 2024-09-16T20:51:09.069 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.069 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.069 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:09.069 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222609+0000", 2024-09-16T20:51:09.069 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.083656650999999999 2024-09-16T20:51:09.069 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.069 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.069 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:09.070 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222684+0000", 2024-09-16T20:51:09.070 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4369999999999994e-05 2024-09-16T20:51:09.070 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.070 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.070 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:09.070 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222729+0000", 2024-09-16T20:51:09.070 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5287000000000002e-05 2024-09-16T20:51:09.070 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.070 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:09.070 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.070 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.070 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.070 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:11704 3.9 3:91220b75:::rbd_data.110e747f0779.000000000000001c:head [set-alloc-hint object_size 262144 write_size 262144,write 139264~6656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:51:09.070 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:51:09.070 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 124.899586607, 2024-09-16T20:51:09.071 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.084531133999999994, 2024-09-16T20:51:09.071 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:09.071 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:09.071 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:09.071 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:09.071 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:51:09.071 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:51:09.071 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 11704 2024-09-16T20:51:09.071 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.071 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:09.071 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.071 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:09.071 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:51:09.071 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.071 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.072 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.072 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:09.072 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:51:09.072 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.072 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.072 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.072 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:09.072 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138167+0000", 2024-09-16T20:51:09.072 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.33e-07 2024-09-16T20:51:09.072 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.072 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.072 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:09.072 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138179+0000", 2024-09-16T20:51:09.073 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2055999999999999e-05 2024-09-16T20:51:09.073 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.073 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.073 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:09.073 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138193+0000", 2024-09-16T20:51:09.073 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3961000000000001e-05 2024-09-16T20:51:09.073 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.073 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.073 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:09.073 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138200+0000", 2024-09-16T20:51:09.073 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4429999999999997e-06 2024-09-16T20:51:09.073 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.073 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.073 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:09.074 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138258+0000", 2024-09-16T20:51:09.074 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7225999999999997e-05 2024-09-16T20:51:09.074 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.074 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.074 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:09.074 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138364+0000", 2024-09-16T20:51:09.074 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000106013 2024-09-16T20:51:09.074 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.074 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.074 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:51:09.074 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138494+0000", 2024-09-16T20:51:09.074 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013059599999999999 2024-09-16T20:51:09.074 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.074 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.074 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:09.075 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.139220+0000", 2024-09-16T20:51:09.075 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00072539800000000004 2024-09-16T20:51:09.075 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.075 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.075 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:09.075 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222608+0000", 2024-09-16T20:51:09.075 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.083388620999999996 2024-09-16T20:51:09.075 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.075 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.075 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:09.075 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222651+0000", 2024-09-16T20:51:09.075 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2228000000000003e-05 2024-09-16T20:51:09.075 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.076 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.076 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:09.076 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222697+0000", 2024-09-16T20:51:09.076 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6659000000000002e-05 2024-09-16T20:51:09.076 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.076 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:09.076 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.076 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.076 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.076 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5138.0:9358 3.c 3.c25699cc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T20:51:09.076 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:51:09.076 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 102.187439302, 2024-09-16T20:51:09.076 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0018657580000001, 2024-09-16T20:51:09.076 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:09.076 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:09.077 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:51:09.077 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:09.077 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5138", 2024-09-16T20:51:09.077 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:51:09.077 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 9358 2024-09-16T20:51:09.077 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.077 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:09.077 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.077 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:09.077 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:51:09.077 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.077 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.077 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.077 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:09.078 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:51:09.078 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.078 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.078 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.078 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:09.078 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850314+0000", 2024-09-16T20:51:09.078 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4099999999999999e-07 2024-09-16T20:51:09.078 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.078 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.078 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:09.078 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850316+0000", 2024-09-16T20:51:09.078 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2469999999999998e-06 2024-09-16T20:51:09.078 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.078 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.078 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:09.078 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850322+0000", 2024-09-16T20:51:09.079 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3470000000000004e-06 2024-09-16T20:51:09.079 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.079 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.079 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:09.079 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850327+0000", 2024-09-16T20:51:09.079 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9450000000000001e-06 2024-09-16T20:51:09.079 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.079 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.079 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:09.079 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850376+0000", 2024-09-16T20:51:09.079 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8257000000000002e-05 2024-09-16T20:51:09.079 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.079 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.079 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:09.079 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850385+0000", 2024-09-16T20:51:09.080 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9800000000000004e-06 2024-09-16T20:51:09.080 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.080 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.080 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:09.080 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852167+0000", 2024-09-16T20:51:09.080 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0017825659999999 2024-09-16T20:51:09.080 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.080 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.080 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:09.080 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852179+0000", 2024-09-16T20:51:09.080 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1975e-05 2024-09-16T20:51:09.080 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.080 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:09.081 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.081 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.081 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.081 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5138.0:9382 3.c 3.c25699cc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T20:51:09.081 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:51:09.081 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 102.179092286, 2024-09-16T20:51:09.081 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.99358404700000003, 2024-09-16T20:51:09.081 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:09.081 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:09.081 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:51:09.081 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:09.081 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5138", 2024-09-16T20:51:09.081 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:51:09.081 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 9382 2024-09-16T20:51:09.081 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.082 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:09.082 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.082 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:09.082 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:51:09.082 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.082 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.082 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.082 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:09.082 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:51:09.082 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.082 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.082 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.082 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:09.082 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858661+0000", 2024-09-16T20:51:09.083 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0200000000000002e-07 2024-09-16T20:51:09.083 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.083 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.083 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:09.083 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858664+0000", 2024-09-16T20:51:09.083 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.038e-06 2024-09-16T20:51:09.083 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.083 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.083 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:09.083 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858671+0000", 2024-09-16T20:51:09.083 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0110000000000003e-06 2024-09-16T20:51:09.083 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.083 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.083 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:09.083 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858677+0000", 2024-09-16T20:51:09.084 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0959999999999997e-06 2024-09-16T20:51:09.084 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.084 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.084 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:09.084 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858732+0000", 2024-09-16T20:51:09.084 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4543000000000003e-05 2024-09-16T20:51:09.084 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.084 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.084 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:09.084 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858739+0000", 2024-09-16T20:51:09.084 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5429999999999996e-06 2024-09-16T20:51:09.084 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.084 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.085 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:09.085 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852236+0000", 2024-09-16T20:51:09.085 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.993496453 2024-09-16T20:51:09.085 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.085 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.085 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:09.085 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852244+0000", 2024-09-16T20:51:09.085 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8610000000000004e-06 2024-09-16T20:51:09.085 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.085 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:09.085 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.085 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.085 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.085 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5763.0:4895 3.a 3:5384606c:::rbd_data.110e1b19a241.0000000000000018:head [read 0~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-16T20:51:09.085 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:45.598212+0000", 2024-09-16T20:51:09.086 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 83.439540245000003, 2024-09-16T20:51:09.086 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0375864829999999, 2024-09-16T20:51:09.086 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:09.086 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:09.086 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:09.086 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:09.086 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5763", 2024-09-16T20:51:09.086 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/623897567", 2024-09-16T20:51:09.086 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4895 2024-09-16T20:51:09.086 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.086 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:09.086 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.086 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:09.086 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598212+0000", 2024-09-16T20:51:09.086 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.087 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.087 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.087 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:09.087 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598212+0000", 2024-09-16T20:51:09.087 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.087 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.087 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.087 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:09.087 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598214+0000", 2024-09-16T20:51:09.087 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0090000000000002e-06 2024-09-16T20:51:09.087 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.087 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.087 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:09.087 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598222+0000", 2024-09-16T20:51:09.087 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6699999999999994e-06 2024-09-16T20:51:09.088 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.088 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.088 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:09.088 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598241+0000", 2024-09-16T20:51:09.088 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8896999999999999e-05 2024-09-16T20:51:09.088 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.088 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.088 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:09.088 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598250+0000", 2024-09-16T20:51:09.088 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6649999999999992e-06 2024-09-16T20:51:09.088 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.088 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.088 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:09.089 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598325+0000", 2024-09-16T20:51:09.089 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5189000000000001e-05 2024-09-16T20:51:09.089 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.089 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.089 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op must wait for map", 2024-09-16T20:51:09.089 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598333+0000", 2024-09-16T20:51:09.089 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.3170000000000002e-06 2024-09-16T20:51:09.089 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.089 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.089 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:09.089 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601355+0000", 2024-09-16T20:51:09.089 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0030215400000000001 2024-09-16T20:51:09.089 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.089 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.090 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:09.090 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601365+0000", 2024-09-16T20:51:09.090 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0215e-05 2024-09-16T20:51:09.090 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.090 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.090 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:09.090 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635506+0000", 2024-09-16T20:51:09.090 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0341408240000001 2024-09-16T20:51:09.090 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.090 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.090 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:09.090 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635655+0000", 2024-09-16T20:51:09.090 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000149412 2024-09-16T20:51:09.091 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.091 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.091 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:09.091 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635799+0000", 2024-09-16T20:51:09.091 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000143745 2024-09-16T20:51:09.091 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.091 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:09.091 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.091 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.091 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.091 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5763.0:4900 3.a 3:5384606c:::rbd_data.110e1b19a241.0000000000000018:head [read 8192~24576] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-16T20:51:09.091 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:51:09.091 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 83.439481768999997, 2024-09-16T20:51:09.091 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0379340800000001, 2024-09-16T20:51:09.092 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:09.092 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:09.092 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:09.092 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:09.092 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5763", 2024-09-16T20:51:09.092 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/623897567", 2024-09-16T20:51:09.092 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4900 2024-09-16T20:51:09.092 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.092 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:09.092 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.092 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:09.092 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:51:09.092 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.093 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.093 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.093 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:09.093 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:51:09.093 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.093 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.093 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.093 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:09.093 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:51:09.093 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4300000000000003e-07 2024-09-16T20:51:09.093 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.093 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.093 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:09.094 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598274+0000", 2024-09-16T20:51:09.094 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.976e-06 2024-09-16T20:51:09.094 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.094 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.094 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:09.094 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598282+0000", 2024-09-16T20:51:09.094 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0029999999999994e-06 2024-09-16T20:51:09.094 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.094 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.094 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:09.094 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598289+0000", 2024-09-16T20:51:09.094 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2389999999999999e-06 2024-09-16T20:51:09.094 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.094 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.095 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:09.095 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598430+0000", 2024-09-16T20:51:09.095 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014119600000000001 2024-09-16T20:51:09.095 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.095 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.095 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting_for_map not empty", 2024-09-16T20:51:09.095 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598435+0000", 2024-09-16T20:51:09.095 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3489999999999999e-06 2024-09-16T20:51:09.095 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.095 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.095 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:09.095 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601429+0000", 2024-09-16T20:51:09.095 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0029942720000000001 2024-09-16T20:51:09.096 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.096 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.096 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:09.096 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601438+0000", 2024-09-16T20:51:09.096 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4759999999999998e-06 2024-09-16T20:51:09.096 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.096 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.096 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:09.096 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635987+0000", 2024-09-16T20:51:09.096 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0345495010000001 2024-09-16T20:51:09.096 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.096 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.096 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:09.096 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.636056+0000", 2024-09-16T20:51:09.096 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8610999999999998e-05 2024-09-16T20:51:09.097 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.097 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.097 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:09.097 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.636205+0000", 2024-09-16T20:51:09.097 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014891399999999999 2024-09-16T20:51:09.097 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.097 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:09.097 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.097 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.097 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.097 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop(client.6231.0:2182 3.17 e136/81 3:edb5e5cd:::rbd_data.110e61479aee.0000000000000014:head v 136'791, mlcod=135'789)", 2024-09-16T20:51:09.097 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:22.735499+0000", 2024-09-16T20:51:09.098 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 46.302253827999998, 2024-09-16T20:51:09.098 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12369608, 2024-09-16T20:51:09.098 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:09.098 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:09.098 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:09.098 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:09.098 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.098 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:09.098 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735499+0000", 2024-09-16T20:51:09.098 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.098 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.098 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.099 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:09.099 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735499+0000", 2024-09-16T20:51:09.099 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.099 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.099 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.099 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:09.099 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735499+0000", 2024-09-16T20:51:09.099 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8099999999999999e-07 2024-09-16T20:51:09.099 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.099 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.099 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:09.099 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735506+0000", 2024-09-16T20:51:09.099 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3579999999999997e-06 2024-09-16T20:51:09.100 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.100 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.100 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:09.100 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735517+0000", 2024-09-16T20:51:09.100 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0869999999999999e-05 2024-09-16T20:51:09.100 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.100 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.100 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:09.100 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735533+0000", 2024-09-16T20:51:09.100 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5858e-05 2024-09-16T20:51:09.100 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.101 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.101 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:09.101 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735584+0000", 2024-09-16T20:51:09.101 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0470999999999998e-05 2024-09-16T20:51:09.101 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.101 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.101 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:09.101 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735607+0000", 2024-09-16T20:51:09.101 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2875999999999999e-05 2024-09-16T20:51:09.101 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.102 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.102 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:09.102 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.859144+0000", 2024-09-16T20:51:09.102 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.123537701 2024-09-16T20:51:09.102 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.102 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.102 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:09.102 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.859195+0000", 2024-09-16T20:51:09.102 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0664999999999997e-05 2024-09-16T20:51:09.102 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.102 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:09.102 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.102 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.103 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.103 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5204 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 2523136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:51:09.103 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.883660+0000", 2024-09-16T20:51:09.103 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 45.154092298000002, 2024-09-16T20:51:09.103 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052517638999999998, 2024-09-16T20:51:09.103 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:09.103 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:09.103 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:09.103 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:09.103 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:09.103 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:09.103 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5204 2024-09-16T20:51:09.103 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.104 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:09.104 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.104 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:09.104 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883660+0000", 2024-09-16T20:51:09.104 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.104 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.104 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.104 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:09.104 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883660+0000", 2024-09-16T20:51:09.104 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.104 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.104 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.104 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:09.104 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883661+0000", 2024-09-16T20:51:09.105 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9999999999999998e-07 2024-09-16T20:51:09.105 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.105 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.105 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:09.105 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883663+0000", 2024-09-16T20:51:09.105 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.4480000000000001e-06 2024-09-16T20:51:09.105 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.105 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.105 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:09.105 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883669+0000", 2024-09-16T20:51:09.105 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.071e-06 2024-09-16T20:51:09.105 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.105 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.105 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:09.106 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883675+0000", 2024-09-16T20:51:09.106 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.366e-06 2024-09-16T20:51:09.106 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.106 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.106 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:09.106 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.935970+0000", 2024-09-16T20:51:09.106 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052295346999999999 2024-09-16T20:51:09.106 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.106 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.106 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:09.106 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936064+0000", 2024-09-16T20:51:09.106 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4438000000000006e-05 2024-09-16T20:51:09.106 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.106 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.107 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:09.107 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936178+0000", 2024-09-16T20:51:09.107 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000113569 2024-09-16T20:51:09.107 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.107 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:09.107 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.107 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.107 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.107 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5205 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 2543616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:51:09.107 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.883745+0000", 2024-09-16T20:51:09.107 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 45.154007540000002, 2024-09-16T20:51:09.107 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052692112999999999, 2024-09-16T20:51:09.107 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:09.107 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:09.107 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:09.108 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:09.108 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:09.108 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:09.108 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5205 2024-09-16T20:51:09.108 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.108 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:09.108 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.108 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:09.108 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883745+0000", 2024-09-16T20:51:09.108 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.108 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.108 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.108 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:09.109 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883745+0000", 2024-09-16T20:51:09.109 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.109 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.109 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.109 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:09.109 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883746+0000", 2024-09-16T20:51:09.109 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2699999999999999e-07 2024-09-16T20:51:09.109 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.109 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.109 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:09.109 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883750+0000", 2024-09-16T20:51:09.109 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7050000000000001e-06 2024-09-16T20:51:09.109 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.109 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.109 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:09.110 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883769+0000", 2024-09-16T20:51:09.110 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9165e-05 2024-09-16T20:51:09.110 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.110 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.110 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:09.110 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883778+0000", 2024-09-16T20:51:09.110 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.6690000000000001e-06 2024-09-16T20:51:09.110 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.110 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.110 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:09.110 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936228+0000", 2024-09-16T20:51:09.110 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052449587999999998 2024-09-16T20:51:09.110 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.110 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.110 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:09.110 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936333+0000", 2024-09-16T20:51:09.111 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010511800000000001 2024-09-16T20:51:09.111 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.111 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.111 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:09.111 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936437+0000", 2024-09-16T20:51:09.111 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000104141 2024-09-16T20:51:09.111 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.111 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:09.111 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.111 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.111 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.111 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5206 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 1167360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:51:09.111 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.883802+0000", 2024-09-16T20:51:09.111 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 45.153951046000003, 2024-09-16T20:51:09.111 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052800780999999998, 2024-09-16T20:51:09.112 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:09.112 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:09.112 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:09.112 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:09.112 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:09.112 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:09.112 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5206 2024-09-16T20:51:09.112 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.112 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:09.112 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.112 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:09.112 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883802+0000", 2024-09-16T20:51:09.112 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.112 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.113 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.113 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:09.113 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883802+0000", 2024-09-16T20:51:09.113 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.113 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.113 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.113 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:09.113 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883933+0000", 2024-09-16T20:51:09.113 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013182999999999999 2024-09-16T20:51:09.113 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.113 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.113 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:09.113 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883937+0000", 2024-09-16T20:51:09.113 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.771e-06 2024-09-16T20:51:09.114 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.114 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.114 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:09.114 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883953+0000", 2024-09-16T20:51:09.114 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5409000000000001e-05 2024-09-16T20:51:09.114 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.114 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.114 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:09.114 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883959+0000", 2024-09-16T20:51:09.114 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0700000000000003e-06 2024-09-16T20:51:09.114 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.114 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.114 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:09.114 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936476+0000", 2024-09-16T20:51:09.115 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052517434000000002 2024-09-16T20:51:09.115 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.115 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.115 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:09.115 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936529+0000", 2024-09-16T20:51:09.115 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2969000000000001e-05 2024-09-16T20:51:09.115 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.115 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.115 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:09.115 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936602+0000", 2024-09-16T20:51:09.115 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3298000000000002e-05 2024-09-16T20:51:09.115 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.115 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:09.116 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.116 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.116 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.116 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5207 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 1904640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:51:09.116 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.883980+0000", 2024-09-16T20:51:09.116 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 45.153772836999998, 2024-09-16T20:51:09.116 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052759922000000001, 2024-09-16T20:51:09.116 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:09.116 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:09.116 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:09.116 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:09.116 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:09.116 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:09.116 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5207 2024-09-16T20:51:09.117 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.117 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:09.117 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.117 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:09.117 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883980+0000", 2024-09-16T20:51:09.117 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.117 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.117 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.117 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:09.117 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883980+0000", 2024-09-16T20:51:09.117 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.117 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.117 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.118 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:09.118 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884126+0000", 2024-09-16T20:51:09.118 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014577399999999999 2024-09-16T20:51:09.118 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.118 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.118 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:09.118 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884130+0000", 2024-09-16T20:51:09.118 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9729999999999999e-06 2024-09-16T20:51:09.118 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.118 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.118 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:09.118 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884139+0000", 2024-09-16T20:51:09.118 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8660000000000001e-06 2024-09-16T20:51:09.119 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.119 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.119 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:09.119 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884144+0000", 2024-09-16T20:51:09.119 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3669999999999999e-06 2024-09-16T20:51:09.119 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.119 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.119 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:09.119 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936635+0000", 2024-09-16T20:51:09.119 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052491705 2024-09-16T20:51:09.119 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.119 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.119 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:09.119 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936679+0000", 2024-09-16T20:51:09.119 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3105999999999997e-05 2024-09-16T20:51:09.120 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.120 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.120 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:09.120 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936740+0000", 2024-09-16T20:51:09.120 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1130999999999995e-05 2024-09-16T20:51:09.120 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.120 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:09.120 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.120 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.120 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.120 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5208 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 131072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:51:09.120 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.884165+0000", 2024-09-16T20:51:09.120 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 45.153587541, 2024-09-16T20:51:09.120 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052746603000000003, 2024-09-16T20:51:09.120 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:09.121 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:09.121 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:09.121 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:09.121 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:09.121 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:09.121 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5208 2024-09-16T20:51:09.121 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.121 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:09.121 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.121 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:09.121 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884165+0000", 2024-09-16T20:51:09.121 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.121 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.121 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.122 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:09.122 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884165+0000", 2024-09-16T20:51:09.122 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.122 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.122 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.122 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:09.122 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884463+0000", 2024-09-16T20:51:09.122 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00029801699999999998 2024-09-16T20:51:09.122 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.122 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.122 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:09.122 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884467+0000", 2024-09-16T20:51:09.122 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8210000000000003e-06 2024-09-16T20:51:09.122 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.122 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.122 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:09.123 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884477+0000", 2024-09-16T20:51:09.123 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.7030000000000001e-06 2024-09-16T20:51:09.123 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.123 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.123 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:09.123 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884485+0000", 2024-09-16T20:51:09.123 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9710000000000004e-06 2024-09-16T20:51:09.123 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.123 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.123 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:09.123 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936785+0000", 2024-09-16T20:51:09.123 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052300665000000003 2024-09-16T20:51:09.123 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.123 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.123 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:09.123 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936851+0000", 2024-09-16T20:51:09.124 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6159000000000001e-05 2024-09-16T20:51:09.124 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.124 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.124 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:09.124 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936912+0000", 2024-09-16T20:51:09.124 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0266999999999997e-05 2024-09-16T20:51:09.124 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.124 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:09.124 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.124 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.124 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.124 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5209 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 1925120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:51:09.124 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.884504+0000", 2024-09-16T20:51:09.125 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 45.153248656000002, 2024-09-16T20:51:09.125 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052575445999999998, 2024-09-16T20:51:09.125 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:09.125 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:09.125 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:09.125 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:09.125 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:09.125 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:09.125 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5209 2024-09-16T20:51:09.125 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.125 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:09.125 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.125 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:09.125 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884504+0000", 2024-09-16T20:51:09.125 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.125 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.126 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.126 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:09.126 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884504+0000", 2024-09-16T20:51:09.126 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.126 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.126 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.126 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:09.126 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884741+0000", 2024-09-16T20:51:09.126 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00023679899999999999 2024-09-16T20:51:09.126 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.126 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.126 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:09.126 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884747+0000", 2024-09-16T20:51:09.126 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0009999999999998e-06 2024-09-16T20:51:09.127 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.127 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.127 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:09.127 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884764+0000", 2024-09-16T20:51:09.127 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7371999999999999e-05 2024-09-16T20:51:09.127 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.127 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.127 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:09.127 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884772+0000", 2024-09-16T20:51:09.127 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1349999999999992e-06 2024-09-16T20:51:09.127 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.127 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.127 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:09.127 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936956+0000", 2024-09-16T20:51:09.127 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052183754999999998 2024-09-16T20:51:09.128 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.128 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.128 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:09.128 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937003+0000", 2024-09-16T20:51:09.128 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6925000000000002e-05 2024-09-16T20:51:09.128 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.128 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.128 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:09.128 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937079+0000", 2024-09-16T20:51:09.128 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6458999999999994e-05 2024-09-16T20:51:09.128 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.128 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:09.128 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.128 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.128 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.129 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:30174 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_remove in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e176)", 2024-09-16T20:51:09.129 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:51:03.615665+0000", 2024-09-16T20:51:09.129 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 5.4220871370000001, 2024-09-16T20:51:09.129 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.96831816599999998, 2024-09-16T20:51:09.129 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:09.129 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:09.129 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:09.129 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:09.129 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:51:09.129 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:09.129 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 30174 2024-09-16T20:51:09.129 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.129 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:09.130 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.130 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:09.130 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615665+0000", 2024-09-16T20:51:09.130 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.130 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.130 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.130 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:09.130 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615665+0000", 2024-09-16T20:51:09.130 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.130 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.130 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.130 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:09.130 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615667+0000", 2024-09-16T20:51:09.130 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0020000000000001e-06 2024-09-16T20:51:09.130 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.131 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.131 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:09.131 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615675+0000", 2024-09-16T20:51:09.131 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5120000000000002e-06 2024-09-16T20:51:09.131 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.131 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.131 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:09.131 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615692+0000", 2024-09-16T20:51:09.131 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.6977e-05 2024-09-16T20:51:09.131 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.131 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.131 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:09.131 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615701+0000", 2024-09-16T20:51:09.131 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6119999999999999e-06 2024-09-16T20:51:09.131 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.131 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.132 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:09.132 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615782+0000", 2024-09-16T20:51:09.132 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1804000000000005e-05 2024-09-16T20:51:09.132 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.132 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.132 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:09.132 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615795+0000", 2024-09-16T20:51:09.132 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2464e-05 2024-09-16T20:51:09.132 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.132 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.132 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:09.132 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.580071+0000", 2024-09-16T20:51:09.132 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.96427590100000005 2024-09-16T20:51:09.132 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.132 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.133 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:09.133 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.580247+0000", 2024-09-16T20:51:09.133 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000175754 2024-09-16T20:51:09.133 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.133 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.133 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 2", 2024-09-16T20:51:09.133 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.580436+0000", 2024-09-16T20:51:09.133 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00018945299999999999 2024-09-16T20:51:09.133 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.133 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.133 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:09.133 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.581045+0000", 2024-09-16T20:51:09.133 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00060937600000000001 2024-09-16T20:51:09.133 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.133 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.134 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:09.134 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.583883+0000", 2024-09-16T20:51:09.134 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0028372900000000001 2024-09-16T20:51:09.134 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.134 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.134 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:09.134 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.583944+0000", 2024-09-16T20:51:09.134 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1848999999999997e-05 2024-09-16T20:51:09.134 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.134 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.134 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:09.134 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.583984+0000", 2024-09-16T20:51:09.134 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9171999999999997e-05 2024-09-16T20:51:09.134 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.134 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:09.135 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.135 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.135 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.135 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:30201 3.f 3:f6903a53:::rbd_object_map.110eca9b190a.000000000000001a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e177)", 2024-09-16T20:51:09.135 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:51:04.603586+0000", 2024-09-16T20:51:09.135 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.4341670300000002, 2024-09-16T20:51:09.135 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.98028146800000004, 2024-09-16T20:51:09.135 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:09.135 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:09.135 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:09.135 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:09.135 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:51:09.135 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:09.135 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 30201 2024-09-16T20:51:09.136 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.136 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:09.136 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.136 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:09.136 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603586+0000", 2024-09-16T20:51:09.136 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.136 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.136 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.136 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:09.136 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603586+0000", 2024-09-16T20:51:09.136 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.136 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.136 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.136 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:09.137 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603587+0000", 2024-09-16T20:51:09.137 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.519e-06 2024-09-16T20:51:09.137 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.137 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.137 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:09.137 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603593+0000", 2024-09-16T20:51:09.137 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1449999999999996e-06 2024-09-16T20:51:09.137 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.137 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.137 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:09.137 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603606+0000", 2024-09-16T20:51:09.137 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2797e-05 2024-09-16T20:51:09.137 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.137 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.138 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:09.138 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603612+0000", 2024-09-16T20:51:09.138 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8710000000000002e-06 2024-09-16T20:51:09.138 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.138 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.138 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:09.138 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603675+0000", 2024-09-16T20:51:09.138 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2728e-05 2024-09-16T20:51:09.138 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.138 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.138 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:09.138 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603687+0000", 2024-09-16T20:51:09.138 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.253e-05 2024-09-16T20:51:09.138 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.139 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.139 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:09.139 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:05.583358+0000", 2024-09-16T20:51:09.139 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.97967074600000004 2024-09-16T20:51:09.139 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.139 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.139 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:09.139 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:05.583515+0000", 2024-09-16T20:51:09.139 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00015755700000000001 2024-09-16T20:51:09.139 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.139 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.139 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:09.139 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:05.583867+0000", 2024-09-16T20:51:09.140 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00035157500000000002 2024-09-16T20:51:09.140 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.140 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:09.140 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.140 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.140 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:09.140 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:09.141 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:51:09.142 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:09.141+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:09.241 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:09.241+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:09.288 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:09.289 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:51:09.289 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:51:09.289 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:09.297 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:51:09.382 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:09.382+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:09.443 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:51:09.442+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:51:09.444 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:09.444 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:51:09.444 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:51:09.444 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:51:09.444 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:09.453 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:51:09.544 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:09.543+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:09.607 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T20:51:09.619 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:09.619 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:51:09.619 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:51:09.619 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:51:09.619 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.619 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:51:09.619 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:51:09.619 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 196.36296381899999, 2024-09-16T20:51:09.619 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67502346999999996, 2024-09-16T20:51:09.619 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:09.619 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:09.619 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:09.620 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:09.620 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:51:09.620 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:51:09.620 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 1 2024-09-16T20:51:09.620 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.620 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:09.620 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.620 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:09.620 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:51:09.620 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.620 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.620 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.620 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:09.620 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:51:09.621 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.621 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.621 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.621 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:09.621 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254128+0000", 2024-09-16T20:51:09.621 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.463e-06 2024-09-16T20:51:09.621 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.621 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.621 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:09.621 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254135+0000", 2024-09-16T20:51:09.621 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1579999999999999e-06 2024-09-16T20:51:09.621 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.621 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.621 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:09.622 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254146+0000", 2024-09-16T20:51:09.622 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1341e-05 2024-09-16T20:51:09.622 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.622 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.622 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:09.622 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254155+0000", 2024-09-16T20:51:09.622 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.1549999999999996e-06 2024-09-16T20:51:09.622 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.622 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.622 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:09.622 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254221+0000", 2024-09-16T20:51:09.622 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5470999999999997e-05 2024-09-16T20:51:09.622 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.623 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.623 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:09.623 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254231+0000", 2024-09-16T20:51:09.623 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0592e-05 2024-09-16T20:51:09.623 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.623 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.623 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:09.623 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926711+0000", 2024-09-16T20:51:09.623 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67247946599999997 2024-09-16T20:51:09.623 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.623 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.623 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:09.623 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926910+0000", 2024-09-16T20:51:09.623 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019934 2024-09-16T20:51:09.624 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.624 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.624 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:51:09.624 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.927159+0000", 2024-09-16T20:51:09.624 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00024876599999999998 2024-09-16T20:51:09.624 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.624 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.624 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:09.624 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.928797+0000", 2024-09-16T20:51:09.624 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001638308 2024-09-16T20:51:09.624 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.624 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.624 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:09.625 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929026+0000", 2024-09-16T20:51:09.625 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00022876099999999999 2024-09-16T20:51:09.625 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.625 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.625 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:09.625 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929102+0000", 2024-09-16T20:51:09.625 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6184000000000001e-05 2024-09-16T20:51:09.625 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.625 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.625 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:09.625 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929150+0000", 2024-09-16T20:51:09.625 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7465e-05 2024-09-16T20:51:09.626 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.626 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:09.626 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.626 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.626 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.626 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3574 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:09.626 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:51:09.626 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 140.135800287, 2024-09-16T20:51:09.626 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053579441999999998, 2024-09-16T20:51:09.626 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:09.626 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:09.626 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:09.626 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:09.627 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:09.627 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:09.627 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3574 2024-09-16T20:51:09.627 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.627 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:09.627 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.627 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:09.627 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:51:09.627 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.627 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.627 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.627 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:09.627 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:51:09.628 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.628 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.628 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.628 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:09.628 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:51:09.628 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2e-07 2024-09-16T20:51:09.628 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.628 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.628 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:09.628 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481292+0000", 2024-09-16T20:51:09.628 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2630000000000002e-06 2024-09-16T20:51:09.628 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.628 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.629 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:09.629 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481297+0000", 2024-09-16T20:51:09.629 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.296e-06 2024-09-16T20:51:09.629 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.629 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.629 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:09.629 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481300+0000", 2024-09-16T20:51:09.629 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7019999999999999e-06 2024-09-16T20:51:09.629 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.629 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.629 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:09.629 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525533+0000", 2024-09-16T20:51:09.630 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.044232318 2024-09-16T20:51:09.630 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.630 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.630 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:09.630 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525619+0000", 2024-09-16T20:51:09.630 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5892999999999999e-05 2024-09-16T20:51:09.630 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.630 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.630 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:09.630 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525987+0000", 2024-09-16T20:51:09.630 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00036836099999999999 2024-09-16T20:51:09.630 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.630 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.630 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:09.630 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531779+0000", 2024-09-16T20:51:09.631 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0057918240000000001 2024-09-16T20:51:09.631 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.631 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.631 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:09.631 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534747+0000", 2024-09-16T20:51:09.631 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0029681600000000001 2024-09-16T20:51:09.631 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.631 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.631 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:09.631 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534832+0000", 2024-09-16T20:51:09.631 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5180999999999996e-05 2024-09-16T20:51:09.631 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.631 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.631 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:09.632 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534869+0000", 2024-09-16T20:51:09.632 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7024000000000003e-05 2024-09-16T20:51:09.632 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.632 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:09.632 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.632 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.632 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.632 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3575 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:09.632 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:51:09.632 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 140.13578554700001, 2024-09-16T20:51:09.632 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053801463000000001, 2024-09-16T20:51:09.632 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:09.632 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:09.632 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:09.632 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:09.633 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:09.633 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:09.633 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3575 2024-09-16T20:51:09.633 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.633 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:09.633 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.633 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:09.633 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:51:09.633 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.633 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.633 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.633 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:09.633 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:51:09.633 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.634 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.634 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.634 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:09.634 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481305+0000", 2024-09-16T20:51:09.635 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5400000000000002e-07 2024-09-16T20:51:09.635 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.636 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.636 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:09.636 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481307+0000", 2024-09-16T20:51:09.636 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7800000000000001e-06 2024-09-16T20:51:09.636 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.636 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.636 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:09.636 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481312+0000", 2024-09-16T20:51:09.636 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6349999999999997e-06 2024-09-16T20:51:09.636 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.636 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.636 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:09.636 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481315+0000", 2024-09-16T20:51:09.636 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7829999999999999e-06 2024-09-16T20:51:09.637 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.637 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.637 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:09.637 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.526876+0000", 2024-09-16T20:51:09.637 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.045561070000000002 2024-09-16T20:51:09.637 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.637 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.637 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:09.637 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.526954+0000", 2024-09-16T20:51:09.637 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8108000000000005e-05 2024-09-16T20:51:09.637 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.637 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.637 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:09.637 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527282+0000", 2024-09-16T20:51:09.637 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000327483 2024-09-16T20:51:09.638 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.638 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.638 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:09.638 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531784+0000", 2024-09-16T20:51:09.638 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0045020169999999997 2024-09-16T20:51:09.638 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.638 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.638 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:09.638 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534921+0000", 2024-09-16T20:51:09.638 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.003137791 2024-09-16T20:51:09.638 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.638 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.638 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:09.638 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534976+0000", 2024-09-16T20:51:09.639 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4953000000000001e-05 2024-09-16T20:51:09.639 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.639 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.639 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:09.639 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535106+0000", 2024-09-16T20:51:09.639 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000129589 2024-09-16T20:51:09.639 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.639 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:09.639 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.639 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.639 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.639 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3576 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:09.640 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:51:09.640 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 140.135771635, 2024-09-16T20:51:09.640 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053955436000000002, 2024-09-16T20:51:09.640 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:09.640 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:09.640 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:09.640 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:09.640 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:09.640 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:09.640 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3576 2024-09-16T20:51:09.640 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.640 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:09.640 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.640 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:09.641 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:51:09.641 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.641 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.641 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.641 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:09.641 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:51:09.641 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.641 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.641 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.641 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:09.641 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481319+0000", 2024-09-16T20:51:09.641 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.53e-07 2024-09-16T20:51:09.641 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.641 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.642 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:09.642 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481320+0000", 2024-09-16T20:51:09.642 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5230000000000001e-06 2024-09-16T20:51:09.642 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.642 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.642 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:09.642 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481330+0000", 2024-09-16T20:51:09.642 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0190000000000001e-05 2024-09-16T20:51:09.642 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.642 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.642 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:09.642 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481333+0000", 2024-09-16T20:51:09.642 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.151e-06 2024-09-16T20:51:09.642 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.642 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.643 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:09.643 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527689+0000", 2024-09-16T20:51:09.643 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046355192000000003 2024-09-16T20:51:09.643 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.643 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.643 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:09.643 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527764+0000", 2024-09-16T20:51:09.643 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5776000000000001e-05 2024-09-16T20:51:09.643 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.643 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.643 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:09.643 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528141+0000", 2024-09-16T20:51:09.643 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00037647599999999999 2024-09-16T20:51:09.643 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.643 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.643 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:09.644 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531788+0000", 2024-09-16T20:51:09.644 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0036470679999999998 2024-09-16T20:51:09.644 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.644 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.644 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:09.644 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535174+0000", 2024-09-16T20:51:09.644 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0033862670000000001 2024-09-16T20:51:09.644 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.644 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.644 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:09.644 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535236+0000", 2024-09-16T20:51:09.644 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1797999999999993e-05 2024-09-16T20:51:09.644 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.644 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.645 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:09.645 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535274+0000", 2024-09-16T20:51:09.645 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7741999999999999e-05 2024-09-16T20:51:09.645 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.645 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:09.645 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.645 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.645 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.645 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3577 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:09.645 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:51:09.645 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 140.135750035, 2024-09-16T20:51:09.645 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054073807000000002, 2024-09-16T20:51:09.645 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:09.645 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:09.645 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:09.646 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:09.646 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:09.646 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:09.646 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3577 2024-09-16T20:51:09.646 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.646 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:09.646 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.646 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:09.646 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:51:09.646 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.646 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.646 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.646 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:09.646 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:51:09.646 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.646 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.647 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.647 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:09.647 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:51:09.647 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.96e-07 2024-09-16T20:51:09.647 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.647 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.647 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:09.647 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481342+0000", 2024-09-16T20:51:09.647 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.714e-06 2024-09-16T20:51:09.647 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.647 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.647 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:09.647 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481349+0000", 2024-09-16T20:51:09.648 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4930000000000001e-06 2024-09-16T20:51:09.648 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.648 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.648 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:09.648 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481353+0000", 2024-09-16T20:51:09.648 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4879999999999999e-06 2024-09-16T20:51:09.648 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.648 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.648 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:09.648 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528619+0000", 2024-09-16T20:51:09.648 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047266351999999998 2024-09-16T20:51:09.648 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.648 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.649 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:09.649 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528706+0000", 2024-09-16T20:51:09.649 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6796e-05 2024-09-16T20:51:09.649 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.649 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.649 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:09.649 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529148+0000", 2024-09-16T20:51:09.649 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00044182999999999999 2024-09-16T20:51:09.649 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.649 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.649 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:09.649 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531792+0000", 2024-09-16T20:51:09.649 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0026443119999999998 2024-09-16T20:51:09.649 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.650 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.650 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:09.650 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535319+0000", 2024-09-16T20:51:09.650 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0035266439999999998 2024-09-16T20:51:09.650 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.650 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.650 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:09.650 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535372+0000", 2024-09-16T20:51:09.650 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3207000000000001e-05 2024-09-16T20:51:09.650 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.650 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.650 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:09.650 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535414+0000", 2024-09-16T20:51:09.650 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1675000000000003e-05 2024-09-16T20:51:09.651 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.651 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:09.651 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.651 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.651 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.651 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3578 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:09.651 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:51:09.651 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 140.13572886599999, 2024-09-16T20:51:09.651 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054189586999999997, 2024-09-16T20:51:09.651 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:09.651 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:09.651 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:09.651 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:09.651 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:09.652 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:09.652 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3578 2024-09-16T20:51:09.652 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.652 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:09.652 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.652 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:09.652 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:51:09.652 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.652 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.652 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.652 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:09.652 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:51:09.652 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.652 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.652 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.653 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:09.653 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:51:09.653 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0800000000000001e-07 2024-09-16T20:51:09.653 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.653 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.653 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:09.653 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481363+0000", 2024-09-16T20:51:09.653 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0339999999999999e-06 2024-09-16T20:51:09.653 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.653 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.653 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:09.653 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481371+0000", 2024-09-16T20:51:09.653 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4440000000000002e-06 2024-09-16T20:51:09.653 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.653 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.654 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:09.654 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481374+0000", 2024-09-16T20:51:09.654 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2049999999999998e-06 2024-09-16T20:51:09.654 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.654 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.654 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:09.654 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529555+0000", 2024-09-16T20:51:09.654 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048181202999999999 2024-09-16T20:51:09.654 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.654 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.654 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:09.654 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529616+0000", 2024-09-16T20:51:09.654 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0912000000000002e-05 2024-09-16T20:51:09.654 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.655 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.655 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:09.655 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529939+0000", 2024-09-16T20:51:09.655 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00032269800000000001 2024-09-16T20:51:09.655 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.655 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.655 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:09.655 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532602+0000", 2024-09-16T20:51:09.655 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0026634940000000002 2024-09-16T20:51:09.655 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.655 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.655 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:09.655 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535465+0000", 2024-09-16T20:51:09.655 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.002862149 2024-09-16T20:51:09.655 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.656 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.656 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:09.656 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535516+0000", 2024-09-16T20:51:09.656 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1696000000000001e-05 2024-09-16T20:51:09.656 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.656 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.656 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:09.656 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535551+0000", 2024-09-16T20:51:09.656 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4443999999999998e-05 2024-09-16T20:51:09.656 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.656 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:09.656 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.656 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.656 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.656 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3579 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:09.656 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:51:09.657 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 140.13571042699999, 2024-09-16T20:51:09.657 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054295151, 2024-09-16T20:51:09.657 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:09.657 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:09.657 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:09.657 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:09.657 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:09.657 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:09.657 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3579 2024-09-16T20:51:09.657 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.657 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:09.657 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.657 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:09.658 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:51:09.658 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.658 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.658 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.658 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:09.658 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:51:09.658 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.658 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.658 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.658 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:09.658 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:51:09.658 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3300000000000001e-07 2024-09-16T20:51:09.658 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.658 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.658 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:09.659 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481381+0000", 2024-09-16T20:51:09.659 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.649e-06 2024-09-16T20:51:09.659 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.659 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.659 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:09.659 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481387+0000", 2024-09-16T20:51:09.659 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4519999999999996e-06 2024-09-16T20:51:09.659 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.659 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.659 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:09.659 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481390+0000", 2024-09-16T20:51:09.659 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0170000000000001e-06 2024-09-16T20:51:09.659 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.659 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.659 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:09.660 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530756+0000", 2024-09-16T20:51:09.660 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049365882999999999 2024-09-16T20:51:09.660 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.660 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.660 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:09.660 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530845+0000", 2024-09-16T20:51:09.660 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9197000000000007e-05 2024-09-16T20:51:09.660 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.660 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.660 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:09.660 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531191+0000", 2024-09-16T20:51:09.660 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00034562500000000001 2024-09-16T20:51:09.660 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.660 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.660 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:09.661 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534078+0000", 2024-09-16T20:51:09.661 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0028878179999999999 2024-09-16T20:51:09.661 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.661 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.661 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:09.661 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535589+0000", 2024-09-16T20:51:09.661 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00151085 2024-09-16T20:51:09.661 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.661 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.661 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:09.661 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535639+0000", 2024-09-16T20:51:09.661 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9697000000000002e-05 2024-09-16T20:51:09.661 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.661 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.662 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:09.662 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535675+0000", 2024-09-16T20:51:09.662 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5729999999999998e-05 2024-09-16T20:51:09.662 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.662 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:09.662 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.662 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.662 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.662 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3580 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:09.662 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:51:09.662 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 140.135696795, 2024-09-16T20:51:09.662 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054397361999999998, 2024-09-16T20:51:09.662 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:09.662 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:09.662 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:09.663 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:09.663 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:09.663 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:09.663 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3580 2024-09-16T20:51:09.663 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.663 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:09.663 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.663 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:09.663 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:51:09.663 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.663 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.663 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.663 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:09.664 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:51:09.664 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.664 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.664 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.664 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:09.664 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:51:09.664 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2600000000000001e-07 2024-09-16T20:51:09.664 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.664 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.664 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:09.664 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481395+0000", 2024-09-16T20:51:09.664 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.544e-06 2024-09-16T20:51:09.664 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.664 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.664 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:09.665 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481404+0000", 2024-09-16T20:51:09.665 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8850000000000005e-06 2024-09-16T20:51:09.665 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.665 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.665 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:09.665 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481407+0000", 2024-09-16T20:51:09.665 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1540000000000002e-06 2024-09-16T20:51:09.665 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.665 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.665 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:09.665 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531823+0000", 2024-09-16T20:51:09.665 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050415637999999999 2024-09-16T20:51:09.665 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.665 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.666 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:09.666 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531895+0000", 2024-09-16T20:51:09.666 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2179000000000001e-05 2024-09-16T20:51:09.666 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.666 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.666 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:09.666 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532236+0000", 2024-09-16T20:51:09.666 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00034073799999999999 2024-09-16T20:51:09.666 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.666 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.666 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:09.666 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534083+0000", 2024-09-16T20:51:09.666 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001847298 2024-09-16T20:51:09.666 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.666 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.666 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:09.667 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535714+0000", 2024-09-16T20:51:09.667 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0016316810000000001 2024-09-16T20:51:09.667 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.667 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.667 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:09.667 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535759+0000", 2024-09-16T20:51:09.667 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4478999999999999e-05 2024-09-16T20:51:09.667 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.667 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.667 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:09.667 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535791+0000", 2024-09-16T20:51:09.667 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.154e-05 2024-09-16T20:51:09.667 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.667 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:09.668 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.668 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.668 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.668 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3581 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:09.668 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:51:09.668 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 140.135674798, 2024-09-16T20:51:09.668 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.057094403000000002, 2024-09-16T20:51:09.668 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:09.668 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:09.668 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:09.668 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:09.668 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:09.668 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:09.669 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3581 2024-09-16T20:51:09.669 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.669 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:09.669 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.669 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:09.669 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:51:09.669 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.669 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.669 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.669 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:09.669 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:51:09.669 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.669 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.670 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.670 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:09.670 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:51:09.670 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8799999999999998e-07 2024-09-16T20:51:09.670 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.670 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.670 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:09.670 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481418+0000", 2024-09-16T20:51:09.670 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1859999999999999e-06 2024-09-16T20:51:09.670 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.670 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.670 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:09.670 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481423+0000", 2024-09-16T20:51:09.670 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8279999999999999e-06 2024-09-16T20:51:09.670 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.671 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.671 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:09.671 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481426+0000", 2024-09-16T20:51:09.671 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0110000000000001e-06 2024-09-16T20:51:09.671 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.671 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.671 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:09.671 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532631+0000", 2024-09-16T20:51:09.671 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051204514999999999 2024-09-16T20:51:09.671 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.671 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.671 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:09.671 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.533381+0000", 2024-09-16T20:51:09.671 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00075029000000000003 2024-09-16T20:51:09.671 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.671 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.672 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:09.672 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.533739+0000", 2024-09-16T20:51:09.672 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00035773100000000002 2024-09-16T20:51:09.672 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.672 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.672 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:09.672 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535123+0000", 2024-09-16T20:51:09.672 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0013836160000000001 2024-09-16T20:51:09.672 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.672 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.672 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:09.672 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538413+0000", 2024-09-16T20:51:09.672 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0032902510000000001 2024-09-16T20:51:09.672 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.673 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.673 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:09.673 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538470+0000", 2024-09-16T20:51:09.673 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6697999999999998e-05 2024-09-16T20:51:09.673 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.673 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.673 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:09.673 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538510+0000", 2024-09-16T20:51:09.673 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9988999999999999e-05 2024-09-16T20:51:09.673 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.673 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:09.673 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.673 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.673 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.673 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:12732 3.d 3:b752444a:::rbd_header.110e747f0779:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-16T20:51:09.674 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:51:09.674 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 124.84860222, 2024-09-16T20:51:09.674 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.065302467000000003, 2024-09-16T20:51:09.674 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:09.674 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:09.674 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:09.674 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:09.674 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:51:09.674 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:51:09.674 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 12732 2024-09-16T20:51:09.674 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.674 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:09.674 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.674 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:09.675 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:51:09.675 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.675 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.675 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.675 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:09.675 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:51:09.675 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.675 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.675 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.675 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:09.675 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768490+0000", 2024-09-16T20:51:09.675 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2859999999999998e-06 2024-09-16T20:51:09.675 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.675 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.675 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:09.675 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768501+0000", 2024-09-16T20:51:09.676 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1226999999999999e-05 2024-09-16T20:51:09.676 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.676 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.676 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:09.676 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768517+0000", 2024-09-16T20:51:09.676 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.592e-05 2024-09-16T20:51:09.676 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.676 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.676 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:09.676 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768528+0000", 2024-09-16T20:51:09.676 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0689999999999999e-05 2024-09-16T20:51:09.676 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.676 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.676 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:09.677 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768584+0000", 2024-09-16T20:51:09.677 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5889e-05 2024-09-16T20:51:09.677 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.677 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.677 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:09.677 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768720+0000", 2024-09-16T20:51:09.677 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000136718 2024-09-16T20:51:09.677 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.677 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.677 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:09.677 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.769066+0000", 2024-09-16T20:51:09.677 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000346013 2024-09-16T20:51:09.677 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.677 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.677 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:09.678 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833659+0000", 2024-09-16T20:51:09.678 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.064592822999999994 2024-09-16T20:51:09.678 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.678 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.678 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:09.678 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833719+0000", 2024-09-16T20:51:09.678 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9441000000000002e-05 2024-09-16T20:51:09.678 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.678 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.678 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:09.678 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833758+0000", 2024-09-16T20:51:09.678 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9122000000000003e-05 2024-09-16T20:51:09.678 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.678 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.679 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:09.679 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833790+0000", 2024-09-16T20:51:09.679 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2338000000000001e-05 2024-09-16T20:51:09.679 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.679 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:09.679 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.679 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.679 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.679 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.5089.0:12732 3.d e73/23 ondisk, result = 0)", 2024-09-16T20:51:09.679 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:51:09.679 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 124.84700005000001, 2024-09-16T20:51:09.679 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.063706456999999994, 2024-09-16T20:51:09.679 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:09.679 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:09.679 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:09.680 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:09.680 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.680 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:09.680 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:51:09.680 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.680 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.680 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.680 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:09.680 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:51:09.680 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.680 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.680 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.680 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:09.680 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770091+0000", 2024-09-16T20:51:09.680 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.53e-06 2024-09-16T20:51:09.681 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.681 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.681 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:09.681 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770104+0000", 2024-09-16T20:51:09.681 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2203e-05 2024-09-16T20:51:09.681 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.681 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.681 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:09.681 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833637+0000", 2024-09-16T20:51:09.681 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.063533201999999997 2024-09-16T20:51:09.681 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.681 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.681 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:09.681 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833655+0000", 2024-09-16T20:51:09.681 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8369e-05 2024-09-16T20:51:09.682 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.682 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.682 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:09.682 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833707+0000", 2024-09-16T20:51:09.682 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2061999999999997e-05 2024-09-16T20:51:09.682 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.682 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.682 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:09.682 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833715+0000", 2024-09-16T20:51:09.682 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5179999999999998e-06 2024-09-16T20:51:09.682 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.682 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.682 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:09.682 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833796+0000", 2024-09-16T20:51:09.682 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1573e-05 2024-09-16T20:51:09.683 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.683 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:09.683 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.683 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.683 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.683 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3286 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 245760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:09.683 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:51:09.683 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 79.575938113000007, 2024-09-16T20:51:09.683 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052523564000000002, 2024-09-16T20:51:09.683 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:09.683 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:09.683 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:09.683 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:09.683 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:51:09.683 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:09.683 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3286 2024-09-16T20:51:09.684 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.684 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:09.684 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.684 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:09.684 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:51:09.684 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.684 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.684 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.684 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:09.684 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:51:09.684 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.684 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.684 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.684 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:09.684 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:51:09.685 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3900000000000001e-07 2024-09-16T20:51:09.685 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.685 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.685 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:09.685 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041153+0000", 2024-09-16T20:51:09.685 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.119e-06 2024-09-16T20:51:09.685 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.685 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.685 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:09.685 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041156+0000", 2024-09-16T20:51:09.685 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0000000000000001e-06 2024-09-16T20:51:09.685 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.685 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.685 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:09.685 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041159+0000", 2024-09-16T20:51:09.685 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5859999999999999e-06 2024-09-16T20:51:09.686 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.686 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.686 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:09.686 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093529+0000", 2024-09-16T20:51:09.686 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052370591000000001 2024-09-16T20:51:09.686 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.686 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.686 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:09.686 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093570+0000", 2024-09-16T20:51:09.686 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0161999999999999e-05 2024-09-16T20:51:09.686 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.686 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.686 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:09.686 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093675+0000", 2024-09-16T20:51:09.687 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010586699999999999 2024-09-16T20:51:09.687 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.687 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:09.687 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.687 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.687 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.687 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3287 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 250880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:09.687 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:51:09.687 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 79.575928063999996, 2024-09-16T20:51:09.687 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052781203999999998, 2024-09-16T20:51:09.687 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:09.687 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:09.687 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:09.687 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:09.687 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:51:09.688 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:09.688 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3287 2024-09-16T20:51:09.688 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.688 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:09.688 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.688 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:09.688 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:51:09.688 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.688 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.688 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.688 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:09.688 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:51:09.688 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.689 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.689 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.689 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:09.689 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:51:09.689 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1e-07 2024-09-16T20:51:09.689 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.689 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.689 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:09.689 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041163+0000", 2024-09-16T20:51:09.689 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0690000000000001e-06 2024-09-16T20:51:09.689 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.689 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.689 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:09.689 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041166+0000", 2024-09-16T20:51:09.689 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.982e-06 2024-09-16T20:51:09.690 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.690 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.690 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:09.690 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041169+0000", 2024-09-16T20:51:09.690 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.593e-06 2024-09-16T20:51:09.690 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.690 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.690 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:09.690 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093709+0000", 2024-09-16T20:51:09.690 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052540492000000001 2024-09-16T20:51:09.690 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.690 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.690 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:09.690 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093792+0000", 2024-09-16T20:51:09.691 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2436999999999997e-05 2024-09-16T20:51:09.691 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.691 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.691 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:09.691 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093943+0000", 2024-09-16T20:51:09.691 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000151421 2024-09-16T20:51:09.691 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.691 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:09.691 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.691 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.691 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.691 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3288 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 256000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:09.691 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:51:09.691 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 79.575918191, 2024-09-16T20:51:09.691 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052928269, 2024-09-16T20:51:09.692 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:09.692 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:09.692 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:09.692 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:09.692 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:51:09.692 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:09.692 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3288 2024-09-16T20:51:09.692 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.692 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:09.692 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.692 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:09.692 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:51:09.692 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.692 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.693 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.693 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:09.693 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:51:09.693 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.693 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.693 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.693 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:09.693 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:51:09.693 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2700000000000001e-07 2024-09-16T20:51:09.693 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.693 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.693 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:09.693 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041173+0000", 2024-09-16T20:51:09.693 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.113e-06 2024-09-16T20:51:09.694 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.694 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.694 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:09.694 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041176+0000", 2024-09-16T20:51:09.694 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9799999999999998e-06 2024-09-16T20:51:09.694 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.694 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.694 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:09.694 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041179+0000", 2024-09-16T20:51:09.694 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.667e-06 2024-09-16T20:51:09.694 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.694 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.694 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:09.694 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093957+0000", 2024-09-16T20:51:09.694 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052778703000000003 2024-09-16T20:51:09.695 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.695 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.695 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:09.695 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093999+0000", 2024-09-16T20:51:09.695 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1300000000000001e-05 2024-09-16T20:51:09.695 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.695 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.695 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:09.695 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094100+0000", 2024-09-16T20:51:09.695 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101279 2024-09-16T20:51:09.695 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.695 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:09.695 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.695 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.695 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.696 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3289 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 261120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:09.696 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:51:09.696 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 79.575908194999997, 2024-09-16T20:51:09.696 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053182976, 2024-09-16T20:51:09.696 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:09.696 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:09.696 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:09.696 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:09.696 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:51:09.696 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:09.696 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3289 2024-09-16T20:51:09.696 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.696 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:09.697 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.697 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:09.697 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:51:09.697 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.697 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.697 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.697 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:09.697 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:51:09.697 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.697 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.697 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.697 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:09.697 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:51:09.697 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2999999999999999e-07 2024-09-16T20:51:09.698 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.698 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.698 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:09.698 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041183+0000", 2024-09-16T20:51:09.698 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0720000000000001e-06 2024-09-16T20:51:09.698 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.698 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.698 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:09.698 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041186+0000", 2024-09-16T20:51:09.698 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9869999999999999e-06 2024-09-16T20:51:09.698 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.698 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.698 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:09.699 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041189+0000", 2024-09-16T20:51:09.699 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.587e-06 2024-09-16T20:51:09.699 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.699 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.699 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:09.699 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094136+0000", 2024-09-16T20:51:09.699 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052947270999999997 2024-09-16T20:51:09.699 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.699 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.699 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:09.699 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094213+0000", 2024-09-16T20:51:09.699 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7552000000000006e-05 2024-09-16T20:51:09.699 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.699 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.700 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:09.700 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094365+0000", 2024-09-16T20:51:09.700 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00015127700000000001 2024-09-16T20:51:09.700 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.700 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:09.700 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.700 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.700 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.700 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:2196 3.d 3:b1fe01a0:::rbd_data.110e7ef3004.0000000000000000:2d [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e136)", 2024-09-16T20:51:09.700 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:22.739057+0000", 2024-09-16T20:51:09.700 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 46.878033395999999, 2024-09-16T20:51:09.700 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11999464899999999, 2024-09-16T20:51:09.701 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:09.701 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:09.701 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:09.701 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:09.701 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:09.701 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:09.701 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2196 2024-09-16T20:51:09.701 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.701 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:09.701 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.701 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:09.701 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.739057+0000", 2024-09-16T20:51:09.701 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.701 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.702 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.702 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:09.702 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.739057+0000", 2024-09-16T20:51:09.702 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.702 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.702 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.702 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:09.702 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.739057+0000", 2024-09-16T20:51:09.702 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1399999999999997e-07 2024-09-16T20:51:09.702 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.702 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.702 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:09.702 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.739060+0000", 2024-09-16T20:51:09.702 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6019999999999998e-06 2024-09-16T20:51:09.702 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.703 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.703 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:09.703 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.858755+0000", 2024-09-16T20:51:09.703 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.119695151 2024-09-16T20:51:09.703 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.703 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.703 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:09.703 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.858778+0000", 2024-09-16T20:51:09.703 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3184999999999999e-05 2024-09-16T20:51:09.703 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.703 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.703 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:09.704 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.858865+0000", 2024-09-16T20:51:09.704 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.7330000000000006e-05 2024-09-16T20:51:09.704 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.704 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.704 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:09.704 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.858952+0000", 2024-09-16T20:51:09.704 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6803000000000002e-05 2024-09-16T20:51:09.704 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.704 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.704 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:09.704 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.859051+0000", 2024-09-16T20:51:09.704 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8963999999999996e-05 2024-09-16T20:51:09.704 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.704 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:09.704 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.705 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.705 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.705 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6061.0:5401 3.13 3:c8554348:::rbd_data.110e16eafbd.0000000000000011:head [read 577536~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-16T20:51:09.705 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:27.761536+0000", 2024-09-16T20:51:09.705 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 41.855553583000003, 2024-09-16T20:51:09.705 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.003742712, 2024-09-16T20:51:09.705 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:09.705 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:09.705 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:51:09.705 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:09.705 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6061", 2024-09-16T20:51:09.705 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3930128440", 2024-09-16T20:51:09.705 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5401 2024-09-16T20:51:09.705 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.705 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:09.706 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.706 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:09.706 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761536+0000", 2024-09-16T20:51:09.706 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.706 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.706 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.706 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:09.706 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761536+0000", 2024-09-16T20:51:09.706 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.706 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.706 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.706 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:09.706 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761537+0000", 2024-09-16T20:51:09.706 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2900000000000003e-07 2024-09-16T20:51:09.706 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.706 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.707 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:09.707 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761540+0000", 2024-09-16T20:51:09.707 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2320000000000001e-06 2024-09-16T20:51:09.707 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.707 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.707 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:09.707 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761548+0000", 2024-09-16T20:51:09.707 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1289999999999996e-06 2024-09-16T20:51:09.707 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.707 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.707 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:09.707 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761554+0000", 2024-09-16T20:51:09.707 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0820000000000004e-06 2024-09-16T20:51:09.707 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.707 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.708 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:09.708 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761621+0000", 2024-09-16T20:51:09.708 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6266000000000006e-05 2024-09-16T20:51:09.708 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.708 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.708 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:09.708 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761626+0000", 2024-09-16T20:51:09.708 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.614e-06 2024-09-16T20:51:09.708 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.708 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.708 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:09.708 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:28.765163+0000", 2024-09-16T20:51:09.708 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.003536416 2024-09-16T20:51:09.709 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.709 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.709 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:09.709 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:28.765279+0000", 2024-09-16T20:51:09.709 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000116344 2024-09-16T20:51:09.709 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.709 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:09.709 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.709 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.709 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.709 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6061.0:5425 3.13 3:c8554348:::rbd_data.110e16eafbd.0000000000000011:head [read 602112~28672] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-16T20:51:09.709 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:27.767049+0000", 2024-09-16T20:51:09.709 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 41.850040923999998, 2024-09-16T20:51:09.709 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.99847897900000004, 2024-09-16T20:51:09.709 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:09.709 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:09.710 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:51:09.710 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:09.710 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6061", 2024-09-16T20:51:09.710 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3930128440", 2024-09-16T20:51:09.710 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5425 2024-09-16T20:51:09.710 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.710 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:09.710 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.710 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:09.710 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767049+0000", 2024-09-16T20:51:09.710 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.710 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.710 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.710 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:09.710 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767049+0000", 2024-09-16T20:51:09.711 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.711 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.711 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.711 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:09.711 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767050+0000", 2024-09-16T20:51:09.711 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6499999999999999e-07 2024-09-16T20:51:09.711 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.711 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.711 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:09.711 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767052+0000", 2024-09-16T20:51:09.711 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7350000000000001e-06 2024-09-16T20:51:09.711 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.711 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.711 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:09.711 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767060+0000", 2024-09-16T20:51:09.711 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5109999999999997e-06 2024-09-16T20:51:09.712 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.712 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.712 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:09.712 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767066+0000", 2024-09-16T20:51:09.712 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8139999999999997e-06 2024-09-16T20:51:09.712 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.712 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.712 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:09.712 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767128+0000", 2024-09-16T20:51:09.712 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2174999999999993e-05 2024-09-16T20:51:09.712 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.712 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.712 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:09.712 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767133+0000", 2024-09-16T20:51:09.712 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2140000000000004e-06 2024-09-16T20:51:09.712 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.713 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.713 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:09.713 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:28.765441+0000", 2024-09-16T20:51:09.713 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.99830808599999998 2024-09-16T20:51:09.713 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.713 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.713 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:09.713 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:28.765528+0000", 2024-09-16T20:51:09.713 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6879000000000006e-05 2024-09-16T20:51:09.713 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.713 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:09.713 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.713 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.713 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.713 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:25585 3.10 3:0c7c5b23:::rbd_header.110ebf0678ad:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:51:09.714 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:32.877132+0000", 2024-09-16T20:51:09.714 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 36.739958375000001, 2024-09-16T20:51:09.714 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054096699999999998, 2024-09-16T20:51:09.714 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:09.714 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:09.714 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:09.714 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:09.714 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:51:09.714 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:09.714 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 25585 2024-09-16T20:51:09.714 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.714 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:09.714 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.714 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:09.714 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.877132+0000", 2024-09-16T20:51:09.714 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.715 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.715 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.715 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:09.715 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.877132+0000", 2024-09-16T20:51:09.715 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.715 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.715 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.715 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:09.715 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931072+0000", 2024-09-16T20:51:09.715 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053939931000000003 2024-09-16T20:51:09.715 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.715 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.715 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:09.715 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931076+0000", 2024-09-16T20:51:09.715 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2860000000000001e-06 2024-09-16T20:51:09.716 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.716 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.716 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:09.716 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931085+0000", 2024-09-16T20:51:09.716 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.3370000000000006e-06 2024-09-16T20:51:09.716 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.716 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.716 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:09.716 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931092+0000", 2024-09-16T20:51:09.716 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9970000000000001e-06 2024-09-16T20:51:09.716 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.716 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.716 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:09.716 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931137+0000", 2024-09-16T20:51:09.716 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5160000000000001e-05 2024-09-16T20:51:09.717 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.717 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.717 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:09.717 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931178+0000", 2024-09-16T20:51:09.717 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0379000000000001e-05 2024-09-16T20:51:09.717 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.717 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.717 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:09.717 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931228+0000", 2024-09-16T20:51:09.717 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0609999999999998e-05 2024-09-16T20:51:09.717 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.717 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:09.717 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.717 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.717 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.718 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:26679 3.c 3:33e2f78f:::rbd_header.110e16eafbd:head [watch ping cookie 140213436013408 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e153)", 2024-09-16T20:51:09.718 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:41.440287+0000", 2024-09-16T20:51:09.718 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 28.176802500000001, 2024-09-16T20:51:09.718 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.13295633600000001, 2024-09-16T20:51:09.718 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:09.718 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:09.718 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:09.718 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:09.718 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:51:09.718 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:09.718 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 26679 2024-09-16T20:51:09.718 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.718 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:09.718 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.718 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:09.719 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440287+0000", 2024-09-16T20:51:09.719 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.719 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.719 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.719 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:09.719 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440287+0000", 2024-09-16T20:51:09.719 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:09.719 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.719 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.719 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:09.719 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440288+0000", 2024-09-16T20:51:09.719 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0900000000000001e-07 2024-09-16T20:51:09.719 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.719 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.719 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:09.719 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440292+0000", 2024-09-16T20:51:09.720 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3749999999999996e-06 2024-09-16T20:51:09.720 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.720 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.720 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:09.720 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440299+0000", 2024-09-16T20:51:09.720 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8249999999999999e-06 2024-09-16T20:51:09.720 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.720 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.720 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:09.720 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440307+0000", 2024-09-16T20:51:09.720 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4800000000000004e-06 2024-09-16T20:51:09.720 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.720 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.720 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:09.721 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440451+0000", 2024-09-16T20:51:09.721 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014414500000000001 2024-09-16T20:51:09.721 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.721 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.721 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:09.721 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440465+0000", 2024-09-16T20:51:09.721 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.402e-05 2024-09-16T20:51:09.721 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.721 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.721 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:09.721 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.573043+0000", 2024-09-16T20:51:09.721 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.13257855399999999 2024-09-16T20:51:09.721 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.721 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.721 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:09.721 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.573130+0000", 2024-09-16T20:51:09.722 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6168999999999994e-05 2024-09-16T20:51:09.722 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:09.722 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:09.722 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:09.722 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.573244+0000", 2024-09-16T20:51:09.722 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000114159 2024-09-16T20:51:09.722 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.722 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:09.722 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.722 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:09.722 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:09.722 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:09.724 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:51:09.724 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:09.723+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:09.824 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:09.824+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:09.873 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:09.873 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:51:09.873 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:51:09.873 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:09.881 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:51:09.925 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:09.925+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:10.026 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:10.024+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:10.029 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:10.029 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:51:10.029 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:51:10.029 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:51:10.029 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:10.037 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:51:10.126 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:10.126+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:10.145 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:10.145 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:51:10.145 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:51:10.146 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:51:10.146 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.146 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3714 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1480704~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:10.146 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:51:10.146 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 167.50510830799999, 2024-09-16T20:51:10.146 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050152552000000003, 2024-09-16T20:51:10.146 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:10.146 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:10.146 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:10.146 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:10.146 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:10.146 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:10.146 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3714 2024-09-16T20:51:10.146 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.147 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:10.147 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.147 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:10.147 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:51:10.147 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:10.147 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.147 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.147 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:10.147 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:51:10.147 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:10.147 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.147 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.147 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:10.147 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:51:10.147 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0499999999999999e-07 2024-09-16T20:51:10.147 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.148 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.148 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:10.148 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639007+0000", 2024-09-16T20:51:10.148 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5340000000000001e-06 2024-09-16T20:51:10.148 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.148 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.148 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:10.148 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639013+0000", 2024-09-16T20:51:10.148 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3260000000000003e-06 2024-09-16T20:51:10.148 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.148 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.148 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:10.148 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639018+0000", 2024-09-16T20:51:10.148 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2850000000000002e-06 2024-09-16T20:51:10.149 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.149 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.149 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:10.149 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689026+0000", 2024-09-16T20:51:10.149 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.05000798 2024-09-16T20:51:10.149 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.149 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.149 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:10.149 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689077+0000", 2024-09-16T20:51:10.149 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1063999999999998e-05 2024-09-16T20:51:10.149 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.149 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.149 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:10.150 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689158+0000", 2024-09-16T20:51:10.150 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1057999999999995e-05 2024-09-16T20:51:10.150 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:10.150 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:10.150 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:10.150 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.150 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.150 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3716 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1488896~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:10.150 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:51:10.150 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 167.50509049799999, 2024-09-16T20:51:10.150 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050280142, 2024-09-16T20:51:10.150 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:10.150 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:10.151 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:10.151 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:10.151 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:10.151 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:10.151 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3716 2024-09-16T20:51:10.151 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.151 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:10.151 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.151 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:10.151 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:51:10.151 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:10.151 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.151 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.151 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:10.152 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:51:10.152 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:10.152 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.152 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.152 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:10.152 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639024+0000", 2024-09-16T20:51:10.152 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9999999999999997e-07 2024-09-16T20:51:10.152 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.152 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.152 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:10.152 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639026+0000", 2024-09-16T20:51:10.152 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.781e-06 2024-09-16T20:51:10.152 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.152 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.153 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:10.153 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639032+0000", 2024-09-16T20:51:10.153 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7780000000000004e-06 2024-09-16T20:51:10.153 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.153 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.153 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:10.153 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639036+0000", 2024-09-16T20:51:10.153 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6909999999999996e-06 2024-09-16T20:51:10.153 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.153 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.153 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:10.153 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689191+0000", 2024-09-16T20:51:10.153 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050155284000000001 2024-09-16T20:51:10.154 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.154 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.154 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:10.154 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689242+0000", 2024-09-16T20:51:10.154 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0662999999999999e-05 2024-09-16T20:51:10.154 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.154 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.154 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:10.154 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689304+0000", 2024-09-16T20:51:10.154 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1345000000000003e-05 2024-09-16T20:51:10.154 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:10.154 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:10.154 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:10.154 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.154 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.155 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3719 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1497088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:10.155 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:51:10.155 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 167.50506322000001, 2024-09-16T20:51:10.155 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050399715999999997, 2024-09-16T20:51:10.155 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:10.155 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:10.155 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:10.155 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:10.155 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:10.155 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:10.155 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3719 2024-09-16T20:51:10.155 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.155 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:10.155 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.156 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:10.156 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:51:10.156 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:10.156 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.156 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.156 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:10.156 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:51:10.156 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:10.156 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.156 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.156 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:10.156 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:51:10.156 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9700000000000002e-07 2024-09-16T20:51:10.156 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.157 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.157 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:10.157 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639053+0000", 2024-09-16T20:51:10.157 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2900000000000001e-06 2024-09-16T20:51:10.157 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.157 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.157 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:10.157 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639060+0000", 2024-09-16T20:51:10.157 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7750000000000004e-06 2024-09-16T20:51:10.157 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.157 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.157 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:10.157 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639066+0000", 2024-09-16T20:51:10.157 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0639999999999999e-06 2024-09-16T20:51:10.158 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.158 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.158 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:10.158 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689337+0000", 2024-09-16T20:51:10.158 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050270784999999998 2024-09-16T20:51:10.158 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.158 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.158 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:10.158 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689392+0000", 2024-09-16T20:51:10.158 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4920999999999998e-05 2024-09-16T20:51:10.158 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.158 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.158 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:10.158 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689450+0000", 2024-09-16T20:51:10.159 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8483999999999998e-05 2024-09-16T20:51:10.159 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:10.159 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:10.159 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:10.159 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.159 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.159 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3721 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1505280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:10.159 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:51:10.159 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 167.50504008799999, 2024-09-16T20:51:10.159 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050513608000000002, 2024-09-16T20:51:10.159 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:10.160 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:10.160 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:10.160 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:10.160 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:10.160 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:10.160 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3721 2024-09-16T20:51:10.160 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.160 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:10.160 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.160 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:10.160 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:51:10.160 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:10.160 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.160 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.161 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:10.161 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:51:10.161 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:10.161 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.161 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.161 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:10.161 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:51:10.161 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7e-07 2024-09-16T20:51:10.161 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.161 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.161 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:10.161 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639076+0000", 2024-09-16T20:51:10.161 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8899999999999999e-06 2024-09-16T20:51:10.161 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.161 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.162 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:10.162 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639083+0000", 2024-09-16T20:51:10.162 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1210000000000001e-06 2024-09-16T20:51:10.162 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.162 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.162 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:10.162 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639089+0000", 2024-09-16T20:51:10.162 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.643e-06 2024-09-16T20:51:10.162 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.162 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.162 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:10.162 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689474+0000", 2024-09-16T20:51:10.162 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050385679000000003 2024-09-16T20:51:10.163 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.163 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.163 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:10.163 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689533+0000", 2024-09-16T20:51:10.163 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8303000000000001e-05 2024-09-16T20:51:10.163 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.163 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.163 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:10.163 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689587+0000", 2024-09-16T20:51:10.163 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4601999999999997e-05 2024-09-16T20:51:10.163 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:10.163 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:10.163 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:10.163 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.163 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.164 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3723 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1513472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:10.164 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:51:10.164 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 167.50500928400001, 2024-09-16T20:51:10.164 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050699691999999998, 2024-09-16T20:51:10.164 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:10.164 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:10.164 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:10.164 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:10.164 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:10.164 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:10.164 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3723 2024-09-16T20:51:10.164 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.164 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:10.164 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.165 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:10.165 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:51:10.165 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:10.165 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.165 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.165 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:10.165 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:51:10.165 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:10.165 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.165 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.165 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:10.165 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:51:10.165 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3399999999999999e-07 2024-09-16T20:51:10.166 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.166 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.166 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:10.166 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639107+0000", 2024-09-16T20:51:10.166 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3860000000000001e-06 2024-09-16T20:51:10.166 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.166 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.166 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:10.166 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639114+0000", 2024-09-16T20:51:10.166 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9800000000000001e-06 2024-09-16T20:51:10.166 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.166 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.166 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:10.166 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639129+0000", 2024-09-16T20:51:10.166 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4887e-05 2024-09-16T20:51:10.167 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.167 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.167 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:10.167 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689625+0000", 2024-09-16T20:51:10.167 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050495848000000003 2024-09-16T20:51:10.167 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.167 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.167 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:10.167 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689720+0000", 2024-09-16T20:51:10.167 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4821000000000006e-05 2024-09-16T20:51:10.167 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.167 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.167 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:10.167 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689804+0000", 2024-09-16T20:51:10.167 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4235999999999998e-05 2024-09-16T20:51:10.168 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:10.168 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:10.168 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:10.168 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.168 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.168 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3725 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1521664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:10.168 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:51:10.168 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 167.504975688, 2024-09-16T20:51:10.168 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050797660000000001, 2024-09-16T20:51:10.168 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:10.168 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:10.168 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:10.168 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:10.169 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:10.169 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:10.169 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3725 2024-09-16T20:51:10.169 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.169 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:10.169 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.169 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:10.169 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:51:10.169 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:10.169 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.169 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.169 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:10.169 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:51:10.169 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:10.170 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.170 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.170 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:10.170 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639139+0000", 2024-09-16T20:51:10.170 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8999999999999997e-07 2024-09-16T20:51:10.170 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.170 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.170 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:10.170 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639141+0000", 2024-09-16T20:51:10.170 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.182e-06 2024-09-16T20:51:10.170 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.170 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.170 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:10.170 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639154+0000", 2024-09-16T20:51:10.170 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3626e-05 2024-09-16T20:51:10.171 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.171 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.171 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:10.171 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639160+0000", 2024-09-16T20:51:10.171 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6810000000000003e-06 2024-09-16T20:51:10.171 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.171 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.171 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:10.171 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689831+0000", 2024-09-16T20:51:10.171 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050670907000000001 2024-09-16T20:51:10.171 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.171 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.171 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:10.171 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689882+0000", 2024-09-16T20:51:10.171 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0445999999999998e-05 2024-09-16T20:51:10.171 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.172 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.172 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:10.172 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689936+0000", 2024-09-16T20:51:10.172 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4327999999999999e-05 2024-09-16T20:51:10.172 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:10.172 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:10.172 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:10.172 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.172 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.172 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3727 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1529856~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:10.172 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:51:10.172 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 167.504940111, 2024-09-16T20:51:10.172 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051016922999999999, 2024-09-16T20:51:10.173 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:10.173 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:10.173 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:10.173 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:10.173 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:10.173 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:10.173 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3727 2024-09-16T20:51:10.173 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.173 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:10.173 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.173 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:10.173 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:51:10.173 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:10.174 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.174 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.174 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:10.174 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:51:10.174 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:10.174 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.174 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.174 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:10.174 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:51:10.174 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9999999999999998e-07 2024-09-16T20:51:10.174 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.174 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.174 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:10.174 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639183+0000", 2024-09-16T20:51:10.174 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2399999999999996e-06 2024-09-16T20:51:10.175 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.175 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.175 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:10.175 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639195+0000", 2024-09-16T20:51:10.175 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1113e-05 2024-09-16T20:51:10.175 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.175 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.175 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:10.175 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639206+0000", 2024-09-16T20:51:10.175 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1503e-05 2024-09-16T20:51:10.175 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.175 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.175 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:10.175 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689969+0000", 2024-09-16T20:51:10.175 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050762676 2024-09-16T20:51:10.176 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.176 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.176 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:10.176 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690071+0000", 2024-09-16T20:51:10.176 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102163 2024-09-16T20:51:10.176 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.176 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.176 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:10.176 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690191+0000", 2024-09-16T20:51:10.176 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000119728 2024-09-16T20:51:10.176 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:10.176 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:10.176 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:10.177 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.177 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.177 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3730 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1538048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:10.177 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:51:10.177 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 167.504898548, 2024-09-16T20:51:10.177 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051131467999999999, 2024-09-16T20:51:10.177 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:10.177 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:10.177 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:10.177 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:10.177 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:10.177 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:10.177 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3730 2024-09-16T20:51:10.177 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.178 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:10.178 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.178 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:10.178 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:51:10.178 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:10.178 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.178 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.178 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:10.178 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:51:10.178 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:10.178 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.178 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.178 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:10.178 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639216+0000", 2024-09-16T20:51:10.179 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.39e-07 2024-09-16T20:51:10.179 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.179 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.179 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:10.179 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639218+0000", 2024-09-16T20:51:10.179 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.537e-06 2024-09-16T20:51:10.179 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.179 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.179 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:10.179 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639229+0000", 2024-09-16T20:51:10.179 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1122e-05 2024-09-16T20:51:10.179 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.179 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.179 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:10.179 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639237+0000", 2024-09-16T20:51:10.180 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8509999999999999e-06 2024-09-16T20:51:10.180 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.180 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.180 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:10.180 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690227+0000", 2024-09-16T20:51:10.180 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050990147 2024-09-16T20:51:10.180 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.180 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.180 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:10.180 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690278+0000", 2024-09-16T20:51:10.180 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0223999999999999e-05 2024-09-16T20:51:10.180 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.180 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.181 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:10.181 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690347+0000", 2024-09-16T20:51:10.181 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9147999999999996e-05 2024-09-16T20:51:10.181 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:10.181 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:10.181 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:10.181 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.181 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.181 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3731 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1546240~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:10.181 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:51:10.181 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 167.50486784, 2024-09-16T20:51:10.181 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051276744999999999, 2024-09-16T20:51:10.181 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:10.182 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:10.182 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:10.182 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:10.182 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:10.182 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:10.182 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3731 2024-09-16T20:51:10.182 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.182 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:10.182 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.182 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:10.182 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:51:10.182 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:10.182 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.183 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.183 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:10.183 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:51:10.183 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:10.183 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.183 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.183 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:10.183 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:51:10.183 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9900000000000001e-07 2024-09-16T20:51:10.183 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.183 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.183 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:10.183 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639249+0000", 2024-09-16T20:51:10.183 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3630000000000001e-06 2024-09-16T20:51:10.184 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.184 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.184 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:10.184 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639255+0000", 2024-09-16T20:51:10.184 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4960000000000001e-06 2024-09-16T20:51:10.184 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.184 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.184 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:10.184 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639261+0000", 2024-09-16T20:51:10.184 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2839999999999997e-06 2024-09-16T20:51:10.184 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.184 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.184 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:10.184 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690375+0000", 2024-09-16T20:51:10.184 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051114684000000001 2024-09-16T20:51:10.185 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.185 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.185 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:10.185 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690425+0000", 2024-09-16T20:51:10.185 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9379999999999998e-05 2024-09-16T20:51:10.185 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.185 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.185 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:10.185 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690523+0000", 2024-09-16T20:51:10.185 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8139000000000003e-05 2024-09-16T20:51:10.185 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:10.185 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:10.185 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:10.185 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.185 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.186 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3732 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1554432~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:10.186 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:51:10.186 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 167.504822058, 2024-09-16T20:51:10.186 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051380096, 2024-09-16T20:51:10.186 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:10.186 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:10.186 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:10.186 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:10.186 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:10.186 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:10.186 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3732 2024-09-16T20:51:10.186 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.186 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:10.186 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.186 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:10.187 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:51:10.187 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:10.187 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.187 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.187 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:10.187 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:51:10.187 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:10.187 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.187 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.187 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:10.187 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:51:10.187 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6100000000000001e-07 2024-09-16T20:51:10.187 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.187 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.187 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:10.188 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639295+0000", 2024-09-16T20:51:10.188 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.447e-06 2024-09-16T20:51:10.188 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.188 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.188 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:10.188 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639302+0000", 2024-09-16T20:51:10.188 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0570000000000004e-06 2024-09-16T20:51:10.188 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.188 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.188 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:10.188 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639307+0000", 2024-09-16T20:51:10.188 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1429999999999997e-06 2024-09-16T20:51:10.188 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.189 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.189 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:10.189 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690555+0000", 2024-09-16T20:51:10.189 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051247963000000001 2024-09-16T20:51:10.189 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.189 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.189 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:10.189 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690602+0000", 2024-09-16T20:51:10.189 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7602000000000003e-05 2024-09-16T20:51:10.189 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.189 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.189 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:10.189 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690672+0000", 2024-09-16T20:51:10.189 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9423000000000003e-05 2024-09-16T20:51:10.190 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:10.190 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:10.190 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:10.190 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.190 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.190 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3734 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1562624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:10.190 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:51:10.190 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 167.50480040799999, 2024-09-16T20:51:10.190 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051490610999999999, 2024-09-16T20:51:10.190 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:10.190 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:10.190 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:10.191 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:10.191 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:10.191 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:10.191 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3734 2024-09-16T20:51:10.191 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.191 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:10.191 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.191 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:10.191 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:51:10.191 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:10.191 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.191 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.191 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:10.192 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:51:10.192 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:10.192 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.192 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.192 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:10.192 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639314+0000", 2024-09-16T20:51:10.192 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4400000000000001e-07 2024-09-16T20:51:10.192 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.192 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.192 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:10.192 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639316+0000", 2024-09-16T20:51:10.192 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1459999999999999e-06 2024-09-16T20:51:10.192 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.193 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.193 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:10.193 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639322+0000", 2024-09-16T20:51:10.193 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7610000000000004e-06 2024-09-16T20:51:10.193 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.193 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.193 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:10.193 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639327+0000", 2024-09-16T20:51:10.193 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.186e-06 2024-09-16T20:51:10.193 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.193 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.193 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:10.193 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690698+0000", 2024-09-16T20:51:10.193 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051371223000000001 2024-09-16T20:51:10.193 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.194 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.194 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:10.194 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690748+0000", 2024-09-16T20:51:10.194 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0192999999999999e-05 2024-09-16T20:51:10.194 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.194 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.194 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:10.194 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690804+0000", 2024-09-16T20:51:10.194 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5658000000000001e-05 2024-09-16T20:51:10.194 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:10.194 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:10.194 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:10.194 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.194 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.194 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3736 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1570816~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:10.195 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:51:10.195 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 167.504781149, 2024-09-16T20:51:10.195 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051596528000000003, 2024-09-16T20:51:10.195 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:10.195 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:10.195 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:10.195 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:10.195 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:10.195 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:10.195 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3736 2024-09-16T20:51:10.195 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.195 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:10.195 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.196 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:10.196 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:51:10.196 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:10.196 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.196 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.196 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:10.196 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:51:10.196 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:10.196 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.196 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.196 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:10.196 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:51:10.196 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9999999999999999e-07 2024-09-16T20:51:10.197 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.197 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.197 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:10.197 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639335+0000", 2024-09-16T20:51:10.197 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.635e-06 2024-09-16T20:51:10.197 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.197 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.197 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:10.197 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639340+0000", 2024-09-16T20:51:10.197 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5840000000000002e-06 2024-09-16T20:51:10.197 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.197 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.197 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:10.198 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639346+0000", 2024-09-16T20:51:10.198 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1190000000000002e-06 2024-09-16T20:51:10.198 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.198 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.198 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:10.198 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690817+0000", 2024-09-16T20:51:10.198 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051470721999999997 2024-09-16T20:51:10.198 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.198 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.198 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:10.198 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690875+0000", 2024-09-16T20:51:10.198 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7441000000000001e-05 2024-09-16T20:51:10.198 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.199 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.199 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:10.199 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690929+0000", 2024-09-16T20:51:10.199 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4727e-05 2024-09-16T20:51:10.199 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:10.199 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:10.199 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:10.199 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.199 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.199 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5434 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:10.199 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:51:10.199 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 167.279519583, 2024-09-16T20:51:10.199 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099755831000000003, 2024-09-16T20:51:10.199 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:10.200 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:10.200 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:10.200 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:10.200 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:10.200 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:10.200 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5434 2024-09-16T20:51:10.200 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.200 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:10.200 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.200 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:10.200 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:51:10.200 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:10.200 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.200 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.200 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:10.200 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:51:10.201 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:10.201 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.201 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.201 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:10.201 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864595+0000", 2024-09-16T20:51:10.201 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1099999999999995e-07 2024-09-16T20:51:10.201 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.201 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.201 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:10.201 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864597+0000", 2024-09-16T20:51:10.201 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3989999999999998e-06 2024-09-16T20:51:10.201 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.201 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.201 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:10.201 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864603+0000", 2024-09-16T20:51:10.202 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9309999999999996e-06 2024-09-16T20:51:10.202 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.202 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.202 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:10.202 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864607+0000", 2024-09-16T20:51:10.202 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6200000000000001e-06 2024-09-16T20:51:10.202 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.202 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.202 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:10.202 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964164+0000", 2024-09-16T20:51:10.202 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099557162000000005 2024-09-16T20:51:10.202 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.202 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.203 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:10.203 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964244+0000", 2024-09-16T20:51:10.203 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9597000000000004e-05 2024-09-16T20:51:10.203 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.203 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.203 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:10.203 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964350+0000", 2024-09-16T20:51:10.203 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010641099999999999 2024-09-16T20:51:10.203 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:10.203 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:10.203 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:10.203 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.203 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.203 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5435 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 20480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:10.203 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:51:10.204 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 167.27948976100001, 2024-09-16T20:51:10.204 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100897102, 2024-09-16T20:51:10.204 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:10.204 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:10.204 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:10.204 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:10.204 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:10.204 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:10.204 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5435 2024-09-16T20:51:10.204 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.204 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:10.204 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.204 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:10.204 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:51:10.204 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:10.205 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.205 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.205 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:10.205 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:51:10.205 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:10.205 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.205 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.205 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:10.205 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:51:10.205 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8000000000000001e-07 2024-09-16T20:51:10.205 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.205 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.205 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:10.206 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864626+0000", 2024-09-16T20:51:10.206 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.745e-06 2024-09-16T20:51:10.206 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.206 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.206 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:10.206 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864633+0000", 2024-09-16T20:51:10.206 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3339999999999998e-06 2024-09-16T20:51:10.206 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.206 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.206 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:10.206 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864636+0000", 2024-09-16T20:51:10.206 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1200000000000002e-06 2024-09-16T20:51:10.206 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.206 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.206 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:10.206 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965299+0000", 2024-09-16T20:51:10.207 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10066352000000001 2024-09-16T20:51:10.207 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.207 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.207 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:10.207 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965390+0000", 2024-09-16T20:51:10.207 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0894000000000001e-05 2024-09-16T20:51:10.207 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.207 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.207 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:10.207 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965521+0000", 2024-09-16T20:51:10.207 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013110899999999999 2024-09-16T20:51:10.207 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:10.207 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:10.207 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:10.207 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.208 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.208 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:152 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 327680~32768 out=32768b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:51:10.208 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:51:10.208 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 165.98099281899999, 2024-09-16T20:51:10.208 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11869405800000001, 2024-09-16T20:51:10.208 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:10.208 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:10.208 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:10.208 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:10.208 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:51:10.208 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:51:10.208 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 152 2024-09-16T20:51:10.208 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.208 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:10.209 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.209 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:10.209 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:51:10.209 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:10.209 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.209 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.209 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:10.209 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:51:10.209 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:10.209 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.209 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.209 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:10.209 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163122+0000", 2024-09-16T20:51:10.209 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6400000000000003e-07 2024-09-16T20:51:10.209 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.210 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.210 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:10.210 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163124+0000", 2024-09-16T20:51:10.210 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.565e-06 2024-09-16T20:51:10.210 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.210 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.210 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:10.210 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163136+0000", 2024-09-16T20:51:10.210 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1717e-05 2024-09-16T20:51:10.210 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.210 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.210 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:10.210 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163142+0000", 2024-09-16T20:51:10.210 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1750000000000002e-06 2024-09-16T20:51:10.210 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.211 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.211 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:10.211 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163606+0000", 2024-09-16T20:51:10.211 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00046418799999999998 2024-09-16T20:51:10.211 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.211 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.211 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:10.211 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163656+0000", 2024-09-16T20:51:10.211 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9852999999999999e-05 2024-09-16T20:51:10.211 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.211 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.211 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:10.211 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281815+0000", 2024-09-16T20:51:10.211 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11815909600000001 2024-09-16T20:51:10.211 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:10.211 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:10.212 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:10.212 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.212 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.212 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:153 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:51:10.212 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:51:10.212 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 165.98095986600001, 2024-09-16T20:51:10.212 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11903462400000001, 2024-09-16T20:51:10.212 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:10.212 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:10.212 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:10.212 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:10.212 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:51:10.212 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:51:10.213 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 153 2024-09-16T20:51:10.213 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.213 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:10.213 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.213 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:10.213 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:51:10.213 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:10.213 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.213 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.213 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:10.213 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:51:10.213 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:10.213 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.213 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.213 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:10.214 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163155+0000", 2024-09-16T20:51:10.214 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.37e-07 2024-09-16T20:51:10.214 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.214 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.214 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:10.214 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163157+0000", 2024-09-16T20:51:10.214 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6010000000000002e-06 2024-09-16T20:51:10.214 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.214 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.214 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:10.214 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163165+0000", 2024-09-16T20:51:10.214 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2619999999999998e-06 2024-09-16T20:51:10.214 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.214 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.215 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:10.215 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163171+0000", 2024-09-16T20:51:10.215 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.711e-06 2024-09-16T20:51:10.215 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.215 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.215 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:10.215 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281868+0000", 2024-09-16T20:51:10.215 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118696622 2024-09-16T20:51:10.215 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.215 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.215 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:10.215 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281969+0000", 2024-09-16T20:51:10.215 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101371 2024-09-16T20:51:10.215 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.216 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.216 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:10.216 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282189+0000", 2024-09-16T20:51:10.216 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00021952000000000001 2024-09-16T20:51:10.216 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:10.216 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:10.216 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:10.216 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.216 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.216 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:154 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 360448~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:51:10.216 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:51:10.216 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 165.98092809400001, 2024-09-16T20:51:10.216 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11932035100000001, 2024-09-16T20:51:10.216 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:10.216 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:10.217 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:10.217 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:10.217 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:51:10.217 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:51:10.217 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 154 2024-09-16T20:51:10.217 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.217 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:10.217 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.217 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:10.217 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:51:10.217 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:10.217 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.217 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.217 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:10.218 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:51:10.218 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:10.218 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.218 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.218 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:10.218 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:51:10.218 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:51:10.218 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.218 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.218 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:10.218 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163189+0000", 2024-09-16T20:51:10.218 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6840000000000001e-06 2024-09-16T20:51:10.218 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.219 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.219 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:10.219 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163196+0000", 2024-09-16T20:51:10.219 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7920000000000004e-06 2024-09-16T20:51:10.219 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.219 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.219 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:10.219 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163201+0000", 2024-09-16T20:51:10.219 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2809999999999999e-06 2024-09-16T20:51:10.219 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.219 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.219 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:10.219 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282219+0000", 2024-09-16T20:51:10.219 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.119018343 2024-09-16T20:51:10.219 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.219 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.220 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:10.220 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282305+0000", 2024-09-16T20:51:10.220 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5865000000000005e-05 2024-09-16T20:51:10.220 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.220 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.220 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:10.220 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282506+0000", 2024-09-16T20:51:10.220 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000200927 2024-09-16T20:51:10.220 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:10.220 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:10.220 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:10.220 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.220 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.220 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop(client.4366.0:3088 3.1e e38/23 3:79e5ec01:::rbd_header.110e891a9b2b:head v 38'306, mlcod=36'304)", 2024-09-16T20:51:10.221 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:51:10.221 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 165.97990922599999, 2024-09-16T20:51:10.221 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118102759, 2024-09-16T20:51:10.221 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:10.221 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:10.221 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:10.221 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:10.221 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.221 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:10.221 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:51:10.221 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:10.221 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.221 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.221 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:10.221 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:51:10.222 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:10.222 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.222 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.222 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:10.222 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:51:10.222 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.85e-07 2024-09-16T20:51:10.222 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.222 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.222 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:10.222 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164212+0000", 2024-09-16T20:51:10.222 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3999999999999997e-06 2024-09-16T20:51:10.222 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.222 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.222 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:10.223 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281553+0000", 2024-09-16T20:51:10.223 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11734135499999999 2024-09-16T20:51:10.223 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.223 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.223 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:10.223 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281567+0000", 2024-09-16T20:51:10.223 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4188999999999999e-05 2024-09-16T20:51:10.223 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.223 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.223 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:10.223 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281653+0000", 2024-09-16T20:51:10.223 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5385999999999999e-05 2024-09-16T20:51:10.223 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.223 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.223 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:10.224 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281679+0000", 2024-09-16T20:51:10.224 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6689000000000001e-05 2024-09-16T20:51:10.224 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.224 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.224 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:10.224 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282277+0000", 2024-09-16T20:51:10.224 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00059798000000000002 2024-09-16T20:51:10.224 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.224 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.224 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:10.224 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282307+0000", 2024-09-16T20:51:10.224 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9975e-05 2024-09-16T20:51:10.224 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:10.225 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:10.225 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:10.225 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.225 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.225 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5255.0:44 3.16 3:6f5476df:::rbd_header.110e25db1f60:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:51:10.225 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:51:10.225 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 93.823606170999994, 2024-09-16T20:51:10.225 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.121007509, 2024-09-16T20:51:10.225 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:10.225 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:10.225 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:10.225 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:10.225 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5255", 2024-09-16T20:51:10.225 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:51:10.225 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 44 2024-09-16T20:51:10.226 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.226 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:10.226 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.226 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:10.226 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:51:10.226 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:10.226 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.226 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.226 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:10.226 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:51:10.226 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:10.226 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.226 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.226 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:10.227 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:51:10.227 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0500000000000003e-07 2024-09-16T20:51:10.227 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.227 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.227 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:10.227 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320512+0000", 2024-09-16T20:51:10.227 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2540000000000001e-06 2024-09-16T20:51:10.227 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.227 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.227 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:10.227 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320518+0000", 2024-09-16T20:51:10.227 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1689999999999997e-06 2024-09-16T20:51:10.227 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.227 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.227 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:10.228 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320521+0000", 2024-09-16T20:51:10.228 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5200000000000002e-06 2024-09-16T20:51:10.228 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.228 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.228 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:10.228 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320548+0000", 2024-09-16T20:51:10.228 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6662999999999999e-05 2024-09-16T20:51:10.228 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.228 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.228 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:10.228 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320587+0000", 2024-09-16T20:51:10.228 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8866999999999999e-05 2024-09-16T20:51:10.229 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.229 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.229 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:10.229 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.441515+0000", 2024-09-16T20:51:10.229 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.120928331 2024-09-16T20:51:10.229 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:10.229 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:10.229 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:10.229 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.229 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.229 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:20329 3.5 3:a4ea64b7:::rbd_header.110ede5ef857:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:10.229 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:01.061694+0000", 2024-09-16T20:51:10.229 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 69.082419603000005, 2024-09-16T20:51:10.229 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.071234077000000007, 2024-09-16T20:51:10.229 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:10.230 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:10.230 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:10.230 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:10.230 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:51:10.230 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:10.230 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 20329 2024-09-16T20:51:10.230 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.230 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:10.230 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.230 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:10.230 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.061694+0000", 2024-09-16T20:51:10.230 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:10.230 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.230 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.231 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:10.231 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.061694+0000", 2024-09-16T20:51:10.231 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:10.231 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.231 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.231 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:10.231 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089232+0000", 2024-09-16T20:51:10.231 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.027537275 2024-09-16T20:51:10.231 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.231 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.231 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:10.231 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089237+0000", 2024-09-16T20:51:10.231 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4709999999999998e-06 2024-09-16T20:51:10.231 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.231 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.232 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:10.232 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089246+0000", 2024-09-16T20:51:10.232 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8230000000000001e-06 2024-09-16T20:51:10.232 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.232 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.232 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:10.232 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089252+0000", 2024-09-16T20:51:10.232 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3060000000000003e-06 2024-09-16T20:51:10.232 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.232 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.232 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:10.232 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.132756+0000", 2024-09-16T20:51:10.232 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.043504334999999998 2024-09-16T20:51:10.232 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.233 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.233 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:10.233 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.132829+0000", 2024-09-16T20:51:10.233 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2849999999999995e-05 2024-09-16T20:51:10.233 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:10.233 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:10.233 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:10.233 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.132928+0000", 2024-09-16T20:51:10.233 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.9017000000000004e-05 2024-09-16T20:51:10.233 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:10.233 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:10.233 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:10.233 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:10.233 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:10.234 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:10.235 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:51:10.235 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:10.234+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:10.328 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T20:51:10.336 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:10.335+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:10.355 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:10.355 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:51:10.355 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:51:10.355 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:10.364 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:51:10.448 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:10.447+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:10.451 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:51:10.451 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:51:10.468 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:51:10.468 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:51:10.485 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:51:10.485 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:51:10.503 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:51:10.503 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:51:10.515 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:10.515 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:10.521 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:51:10.522 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:51:10.525 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:51:10.539 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:51:10.539 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:51:10.565 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:10.564+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:10.572 INFO:teuthology.orchestra.run.smithi029.stderr:nodeep-scrub is set 2024-09-16T20:51:10.600 INFO:tasks.thrashosds.thrasher:in_osds: [1, 4, 5, 0] out_osds: [3, 2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-16T20:51:10.600 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T20:51:10.601 INFO:tasks.thrashosds.thrasher:Adding osd 3 2024-09-16T20:51:10.601 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 3 2024-09-16T20:51:10.639 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:10.637+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:10.680 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:10.680 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:10.689 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:51:10.773 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:10.772+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:10.837 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:10.837 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:10.841 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:10.840+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:10.846 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:51:10.942 INFO:tasks.ceph.mon.a.smithi029.stderr:2024-09-16T20:51:10.942+0000 7fe0ce96d640 -1 mon.a@0(leader).osd e183 definitely_dead 0 2024-09-16T20:51:10.942 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:10.941+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:10.992 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:10.992 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:11.000 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:51:11.043 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:51:11.042+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:51:11.145 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:11.143+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:11.147 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:11.148 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:11.156 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:51:11.245 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:11.245+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:11.304 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:11.304 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:11.313 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:51:11.356 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:11.355+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:11.419 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:11.420 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:11.428 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:51:11.447 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:11.446+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:11.547 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:11.547+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:11.578 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:11.578 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:11.587 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:51:11.605 INFO:teuthology.orchestra.run.smithi029.stderr:marked in osd.3. 2024-09-16T20:51:11.632 INFO:tasks.thrashosds.thrasher:Added osd 3 2024-09-16T20:51:11.673 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:11.673+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:11.713 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:11.713 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:11.722 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:51:11.749 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:11.748+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:11.850 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:11.850+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:11.870 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:11.870 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:11.878 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:51:11.951 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:11.950+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:12.025 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:12.025 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:12.033 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:51:12.077 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:12.077+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:12.141 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:12.141 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:12.149 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:51:12.191 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:12.191+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:12.253 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:12.252+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:12.297 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:12.297 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:12.305 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:51:12.354 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:12.353+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:12.412 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:12.412 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:12.421 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:51:12.455 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:12.454+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:12.528 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:12.528 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:12.537 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:51:12.550 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T20:51:12.556 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:51:12.555+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:51:12.590 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T20:51:12.650 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:12.650 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:12.657 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:12.657+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:12.659 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:51:12.709 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:51:12.709 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:51:12.726 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:51:12.727 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:51:12.746 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:51:12.746 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:51:12.758 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:12.758+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:12.763 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:51:12.764 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:51:12.781 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:51:12.781 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:51:12.798 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:51:12.799 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:51:12.812 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:12.813 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:51:12.813 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:51:12.813 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:51:12.813 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.813 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3714 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1480704~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:12.813 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:51:12.813 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 170.17218846399999, 2024-09-16T20:51:12.813 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050152552000000003, 2024-09-16T20:51:12.813 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:12.813 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:12.813 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:12.813 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:12.813 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:12.814 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:12.814 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3714 2024-09-16T20:51:12.814 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.814 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:12.814 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.814 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:12.814 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:51:12.814 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:12.815 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.815 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.815 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:12.815 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:51:12.815 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:12.815 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.815 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.815 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:12.815 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:51:12.815 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0499999999999999e-07 2024-09-16T20:51:12.815 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.816 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.816 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:12.816 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639007+0000", 2024-09-16T20:51:12.816 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5340000000000001e-06 2024-09-16T20:51:12.816 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.816 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.816 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:12.816 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639013+0000", 2024-09-16T20:51:12.816 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3260000000000003e-06 2024-09-16T20:51:12.816 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.816 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.816 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:12.816 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639018+0000", 2024-09-16T20:51:12.817 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2850000000000002e-06 2024-09-16T20:51:12.817 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.817 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.817 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:12.817 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689026+0000", 2024-09-16T20:51:12.817 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.05000798 2024-09-16T20:51:12.817 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.817 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.817 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:12.817 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689077+0000", 2024-09-16T20:51:12.817 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1063999999999998e-05 2024-09-16T20:51:12.817 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.817 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.818 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:12.818 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689158+0000", 2024-09-16T20:51:12.818 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1057999999999995e-05 2024-09-16T20:51:12.818 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:12.818 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:12.818 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:12.818 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.818 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.818 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3716 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1488896~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:12.818 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:51:12.818 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 170.17217065400001, 2024-09-16T20:51:12.818 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050280142, 2024-09-16T20:51:12.818 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:12.818 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:12.819 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:12.819 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:12.819 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:12.819 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:12.819 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3716 2024-09-16T20:51:12.819 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.819 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:12.819 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.819 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:12.819 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:51:12.819 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:12.819 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.819 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.820 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:12.820 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:51:12.820 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:12.820 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.820 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.820 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:12.820 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639024+0000", 2024-09-16T20:51:12.820 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9999999999999997e-07 2024-09-16T20:51:12.820 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.820 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.820 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:12.820 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639026+0000", 2024-09-16T20:51:12.820 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.781e-06 2024-09-16T20:51:12.820 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.821 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.821 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:12.821 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639032+0000", 2024-09-16T20:51:12.821 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7780000000000004e-06 2024-09-16T20:51:12.821 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.821 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.821 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:12.821 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639036+0000", 2024-09-16T20:51:12.821 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6909999999999996e-06 2024-09-16T20:51:12.821 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.821 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.821 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:12.822 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689191+0000", 2024-09-16T20:51:12.822 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050155284000000001 2024-09-16T20:51:12.822 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.822 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.822 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:12.822 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689242+0000", 2024-09-16T20:51:12.822 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0662999999999999e-05 2024-09-16T20:51:12.822 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.822 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.822 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:12.822 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689304+0000", 2024-09-16T20:51:12.823 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1345000000000003e-05 2024-09-16T20:51:12.823 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:12.823 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:12.823 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:12.823 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.823 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.823 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3719 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1497088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:12.823 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:51:12.823 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 170.17214337600001, 2024-09-16T20:51:12.823 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050399715999999997, 2024-09-16T20:51:12.823 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:12.823 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:12.823 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:12.824 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:12.824 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:12.824 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:12.824 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3719 2024-09-16T20:51:12.824 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.824 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:12.824 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.824 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:12.824 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:51:12.824 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:12.824 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.824 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.824 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:12.824 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:51:12.825 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:12.825 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.825 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.825 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:12.825 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:51:12.825 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9700000000000002e-07 2024-09-16T20:51:12.825 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.825 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.825 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:12.825 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639053+0000", 2024-09-16T20:51:12.825 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2900000000000001e-06 2024-09-16T20:51:12.825 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.825 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.826 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:12.826 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639060+0000", 2024-09-16T20:51:12.826 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7750000000000004e-06 2024-09-16T20:51:12.826 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.826 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.826 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:12.826 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639066+0000", 2024-09-16T20:51:12.826 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0639999999999999e-06 2024-09-16T20:51:12.826 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.826 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.826 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:12.826 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689337+0000", 2024-09-16T20:51:12.826 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050270784999999998 2024-09-16T20:51:12.826 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.827 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.827 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:12.827 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689392+0000", 2024-09-16T20:51:12.827 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4920999999999998e-05 2024-09-16T20:51:12.827 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.827 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.827 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:12.827 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689450+0000", 2024-09-16T20:51:12.827 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8483999999999998e-05 2024-09-16T20:51:12.827 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:12.827 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:12.827 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:12.827 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.827 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.828 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3721 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1505280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:12.828 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:51:12.828 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 170.17212024400001, 2024-09-16T20:51:12.828 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050513608000000002, 2024-09-16T20:51:12.828 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:12.828 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:12.828 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:12.828 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:12.828 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:12.828 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:12.828 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3721 2024-09-16T20:51:12.828 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.829 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:12.829 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.829 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:12.829 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:51:12.829 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:12.829 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.829 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.829 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:12.829 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:51:12.829 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:12.829 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.829 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.829 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:12.829 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:51:12.830 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7e-07 2024-09-16T20:51:12.830 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.830 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.830 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:12.830 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639076+0000", 2024-09-16T20:51:12.830 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8899999999999999e-06 2024-09-16T20:51:12.830 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.830 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.830 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:12.830 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639083+0000", 2024-09-16T20:51:12.830 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1210000000000001e-06 2024-09-16T20:51:12.830 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.830 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.830 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:12.831 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639089+0000", 2024-09-16T20:51:12.831 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.643e-06 2024-09-16T20:51:12.831 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.831 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.831 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:12.831 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689474+0000", 2024-09-16T20:51:12.831 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050385679000000003 2024-09-16T20:51:12.831 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.831 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.831 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:12.831 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689533+0000", 2024-09-16T20:51:12.831 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8303000000000001e-05 2024-09-16T20:51:12.831 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.831 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.832 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:12.832 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689587+0000", 2024-09-16T20:51:12.832 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4601999999999997e-05 2024-09-16T20:51:12.832 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:12.832 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:12.832 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:12.832 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.832 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.832 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3723 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1513472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:12.832 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:51:12.832 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 170.17208944000001, 2024-09-16T20:51:12.832 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050699691999999998, 2024-09-16T20:51:12.832 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:12.833 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:12.833 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:12.833 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:12.833 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:12.833 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:12.833 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3723 2024-09-16T20:51:12.833 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.833 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:12.833 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.833 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:12.833 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:51:12.833 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:12.833 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.833 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.834 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:12.834 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:51:12.834 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:12.834 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.834 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.834 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:12.834 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:51:12.834 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3399999999999999e-07 2024-09-16T20:51:12.834 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.834 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.834 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:12.834 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639107+0000", 2024-09-16T20:51:12.834 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3860000000000001e-06 2024-09-16T20:51:12.834 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.835 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.835 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:12.835 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639114+0000", 2024-09-16T20:51:12.835 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9800000000000001e-06 2024-09-16T20:51:12.835 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.835 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.835 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:12.835 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639129+0000", 2024-09-16T20:51:12.835 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4887e-05 2024-09-16T20:51:12.835 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.835 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.835 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:12.835 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689625+0000", 2024-09-16T20:51:12.836 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050495848000000003 2024-09-16T20:51:12.836 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.836 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.836 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:12.836 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689720+0000", 2024-09-16T20:51:12.836 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4821000000000006e-05 2024-09-16T20:51:12.836 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.836 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.836 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:12.836 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689804+0000", 2024-09-16T20:51:12.836 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4235999999999998e-05 2024-09-16T20:51:12.836 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:12.836 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:12.836 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:12.837 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.837 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.837 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3725 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1521664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:12.837 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:51:12.837 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 170.172055844, 2024-09-16T20:51:12.837 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050797660000000001, 2024-09-16T20:51:12.837 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:12.837 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:12.837 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:12.837 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:12.837 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:12.837 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:12.837 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3725 2024-09-16T20:51:12.837 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.838 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:12.838 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.838 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:12.838 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:51:12.838 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:12.838 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.838 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.838 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:12.838 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:51:12.838 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:12.838 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.838 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.838 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:12.838 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639139+0000", 2024-09-16T20:51:12.839 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8999999999999997e-07 2024-09-16T20:51:12.839 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.839 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.839 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:12.839 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639141+0000", 2024-09-16T20:51:12.839 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.182e-06 2024-09-16T20:51:12.839 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.839 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.839 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:12.839 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639154+0000", 2024-09-16T20:51:12.839 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3626e-05 2024-09-16T20:51:12.839 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.839 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.839 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:12.839 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639160+0000", 2024-09-16T20:51:12.840 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6810000000000003e-06 2024-09-16T20:51:12.840 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.840 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.840 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:12.840 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689831+0000", 2024-09-16T20:51:12.840 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050670907000000001 2024-09-16T20:51:12.840 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.840 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.840 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:12.840 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689882+0000", 2024-09-16T20:51:12.840 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0445999999999998e-05 2024-09-16T20:51:12.840 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.840 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.841 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:12.841 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689936+0000", 2024-09-16T20:51:12.841 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4327999999999999e-05 2024-09-16T20:51:12.841 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:12.841 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:12.841 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:12.841 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.841 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.841 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3727 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1529856~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:12.841 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:51:12.841 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 170.17202026699999, 2024-09-16T20:51:12.841 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051016922999999999, 2024-09-16T20:51:12.841 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:12.841 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:12.841 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:12.842 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:12.842 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:12.842 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:12.842 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3727 2024-09-16T20:51:12.842 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.842 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:12.842 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.842 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:12.842 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:51:12.842 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:12.842 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.842 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.842 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:12.843 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:51:12.843 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:12.843 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.843 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.843 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:12.843 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:51:12.843 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9999999999999998e-07 2024-09-16T20:51:12.843 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.843 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.843 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:12.843 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639183+0000", 2024-09-16T20:51:12.843 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2399999999999996e-06 2024-09-16T20:51:12.843 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.843 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.844 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:12.844 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639195+0000", 2024-09-16T20:51:12.844 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1113e-05 2024-09-16T20:51:12.844 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.844 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.844 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:12.844 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639206+0000", 2024-09-16T20:51:12.844 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1503e-05 2024-09-16T20:51:12.844 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.844 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.844 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:12.844 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689969+0000", 2024-09-16T20:51:12.844 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050762676 2024-09-16T20:51:12.844 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.845 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.845 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:12.845 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690071+0000", 2024-09-16T20:51:12.845 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102163 2024-09-16T20:51:12.845 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.845 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.845 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:12.845 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690191+0000", 2024-09-16T20:51:12.845 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000119728 2024-09-16T20:51:12.845 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:12.845 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:12.845 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:12.845 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.846 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.846 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3730 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1538048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:12.846 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:51:12.846 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 170.171978704, 2024-09-16T20:51:12.846 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051131467999999999, 2024-09-16T20:51:12.846 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:12.846 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:12.846 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:12.846 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:12.846 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:12.846 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:12.846 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3730 2024-09-16T20:51:12.846 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.846 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:12.847 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.847 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:12.847 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:51:12.847 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:12.847 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.847 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.847 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:12.847 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:51:12.847 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:12.847 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.847 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.847 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:12.847 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639216+0000", 2024-09-16T20:51:12.847 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.39e-07 2024-09-16T20:51:12.848 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.848 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.848 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:12.848 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639218+0000", 2024-09-16T20:51:12.848 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.537e-06 2024-09-16T20:51:12.848 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.848 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.848 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:12.848 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639229+0000", 2024-09-16T20:51:12.848 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1122e-05 2024-09-16T20:51:12.848 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.848 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.848 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:12.848 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639237+0000", 2024-09-16T20:51:12.849 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8509999999999999e-06 2024-09-16T20:51:12.849 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.849 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.849 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:12.849 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690227+0000", 2024-09-16T20:51:12.849 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050990147 2024-09-16T20:51:12.849 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.849 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.849 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:12.849 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690278+0000", 2024-09-16T20:51:12.849 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0223999999999999e-05 2024-09-16T20:51:12.849 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.849 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.849 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:12.850 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690347+0000", 2024-09-16T20:51:12.850 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9147999999999996e-05 2024-09-16T20:51:12.850 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:12.850 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:12.850 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:12.850 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.850 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.850 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3731 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1546240~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:12.850 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:51:12.850 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 170.171947996, 2024-09-16T20:51:12.850 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051276744999999999, 2024-09-16T20:51:12.850 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:12.850 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:12.851 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:12.851 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:12.851 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:12.851 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:12.851 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3731 2024-09-16T20:51:12.851 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.851 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:12.851 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.851 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:12.851 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:51:12.851 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:12.851 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.851 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.852 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:12.852 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:51:12.852 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:12.852 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.852 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.852 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:12.852 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:51:12.852 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9900000000000001e-07 2024-09-16T20:51:12.852 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.852 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.852 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:12.852 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639249+0000", 2024-09-16T20:51:12.852 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3630000000000001e-06 2024-09-16T20:51:12.853 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.853 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.853 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:12.853 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639255+0000", 2024-09-16T20:51:12.853 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4960000000000001e-06 2024-09-16T20:51:12.853 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.853 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.853 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:12.853 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639261+0000", 2024-09-16T20:51:12.853 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2839999999999997e-06 2024-09-16T20:51:12.853 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.853 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.853 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:12.853 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690375+0000", 2024-09-16T20:51:12.854 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051114684000000001 2024-09-16T20:51:12.854 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.854 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.854 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:12.854 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690425+0000", 2024-09-16T20:51:12.854 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9379999999999998e-05 2024-09-16T20:51:12.854 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.854 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.854 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:12.854 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690523+0000", 2024-09-16T20:51:12.854 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8139000000000003e-05 2024-09-16T20:51:12.854 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:12.854 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:12.854 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:12.855 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.855 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.855 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3732 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1554432~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:12.855 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:51:12.855 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 170.171902214, 2024-09-16T20:51:12.855 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051380096, 2024-09-16T20:51:12.855 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:12.855 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:12.855 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:12.855 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:12.855 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:12.855 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:12.855 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3732 2024-09-16T20:51:12.856 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.856 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:12.856 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.856 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:12.856 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:51:12.856 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:12.856 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.856 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.856 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:12.856 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:51:12.856 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:12.856 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.856 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.857 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:12.857 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:51:12.857 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6100000000000001e-07 2024-09-16T20:51:12.857 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.857 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.857 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:12.857 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639295+0000", 2024-09-16T20:51:12.857 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.447e-06 2024-09-16T20:51:12.857 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.857 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.857 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:12.857 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639302+0000", 2024-09-16T20:51:12.857 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0570000000000004e-06 2024-09-16T20:51:12.857 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.858 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.858 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:12.858 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639307+0000", 2024-09-16T20:51:12.858 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1429999999999997e-06 2024-09-16T20:51:12.858 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.858 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.858 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:12.858 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690555+0000", 2024-09-16T20:51:12.858 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051247963000000001 2024-09-16T20:51:12.858 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.858 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.858 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:12.858 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690602+0000", 2024-09-16T20:51:12.859 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7602000000000003e-05 2024-09-16T20:51:12.859 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.859 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.859 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:12.859 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690672+0000", 2024-09-16T20:51:12.859 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9423000000000003e-05 2024-09-16T20:51:12.859 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:12.859 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:12.859 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:12.859 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.859 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.859 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3734 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1562624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:12.859 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:51:12.859 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 170.17188056399999, 2024-09-16T20:51:12.860 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051490610999999999, 2024-09-16T20:51:12.860 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:12.860 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:12.860 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:12.860 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:12.860 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:12.860 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:12.860 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3734 2024-09-16T20:51:12.860 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.860 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:12.860 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.860 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:12.860 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:51:12.861 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:12.861 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.861 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.861 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:12.861 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:51:12.861 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:12.861 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.861 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.861 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:12.861 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639314+0000", 2024-09-16T20:51:12.861 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4400000000000001e-07 2024-09-16T20:51:12.861 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.861 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.861 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:12.862 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639316+0000", 2024-09-16T20:51:12.862 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1459999999999999e-06 2024-09-16T20:51:12.862 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.862 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.862 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:12.862 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639322+0000", 2024-09-16T20:51:12.862 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7610000000000004e-06 2024-09-16T20:51:12.862 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.862 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.862 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:12.862 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639327+0000", 2024-09-16T20:51:12.862 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.186e-06 2024-09-16T20:51:12.862 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.862 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.863 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:12.863 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690698+0000", 2024-09-16T20:51:12.863 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051371223000000001 2024-09-16T20:51:12.863 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.863 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.863 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:12.863 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690748+0000", 2024-09-16T20:51:12.863 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0192999999999999e-05 2024-09-16T20:51:12.863 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.863 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.863 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:12.863 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690804+0000", 2024-09-16T20:51:12.863 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5658000000000001e-05 2024-09-16T20:51:12.863 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:12.864 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:12.864 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:12.864 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.864 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.864 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3736 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1570816~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:12.864 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:51:12.864 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 170.17186130499999, 2024-09-16T20:51:12.864 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051596528000000003, 2024-09-16T20:51:12.864 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:12.864 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:12.864 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:12.864 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:12.864 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:12.864 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:12.865 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3736 2024-09-16T20:51:12.865 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.865 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:12.865 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.865 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:12.865 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:51:12.865 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:12.865 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.865 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.865 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:12.865 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:51:12.865 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:12.865 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.865 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.866 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:12.866 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:51:12.866 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9999999999999999e-07 2024-09-16T20:51:12.866 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.866 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.866 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:12.866 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639335+0000", 2024-09-16T20:51:12.866 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.635e-06 2024-09-16T20:51:12.866 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.866 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.866 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:12.866 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639340+0000", 2024-09-16T20:51:12.866 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5840000000000002e-06 2024-09-16T20:51:12.866 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.867 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.867 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:12.867 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639346+0000", 2024-09-16T20:51:12.867 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1190000000000002e-06 2024-09-16T20:51:12.867 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.867 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.867 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:12.867 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690817+0000", 2024-09-16T20:51:12.867 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051470721999999997 2024-09-16T20:51:12.867 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.867 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.867 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:12.867 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690875+0000", 2024-09-16T20:51:12.867 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7441000000000001e-05 2024-09-16T20:51:12.868 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.868 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.868 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:12.868 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690929+0000", 2024-09-16T20:51:12.868 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4727e-05 2024-09-16T20:51:12.868 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:12.868 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:12.868 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:12.868 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.868 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.868 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5434 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:12.868 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:51:12.868 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 169.94659973899999, 2024-09-16T20:51:12.868 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099755831000000003, 2024-09-16T20:51:12.869 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:12.869 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:12.869 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:12.869 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:12.869 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:12.869 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:12.869 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5434 2024-09-16T20:51:12.869 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.869 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:12.869 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.869 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:12.869 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:51:12.869 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:12.870 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.870 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.870 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:12.870 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:51:12.870 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:12.870 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.870 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.870 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:12.870 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864595+0000", 2024-09-16T20:51:12.870 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1099999999999995e-07 2024-09-16T20:51:12.870 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.870 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.870 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:12.870 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864597+0000", 2024-09-16T20:51:12.871 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3989999999999998e-06 2024-09-16T20:51:12.871 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.871 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.871 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:12.871 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864603+0000", 2024-09-16T20:51:12.871 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9309999999999996e-06 2024-09-16T20:51:12.871 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.871 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.871 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:12.871 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864607+0000", 2024-09-16T20:51:12.871 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6200000000000001e-06 2024-09-16T20:51:12.871 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.871 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.872 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:12.872 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964164+0000", 2024-09-16T20:51:12.872 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099557162000000005 2024-09-16T20:51:12.872 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.872 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.872 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:12.872 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964244+0000", 2024-09-16T20:51:12.872 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9597000000000004e-05 2024-09-16T20:51:12.872 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.872 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.872 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:12.872 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964350+0000", 2024-09-16T20:51:12.872 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010641099999999999 2024-09-16T20:51:12.872 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:12.873 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:12.873 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:12.873 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.873 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.873 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5435 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 20480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:12.873 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:51:12.873 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 169.94656991700001, 2024-09-16T20:51:12.873 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100897102, 2024-09-16T20:51:12.873 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:12.873 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:12.873 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:12.873 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:12.873 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:12.873 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:12.874 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5435 2024-09-16T20:51:12.874 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.874 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:12.874 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.874 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:12.874 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:51:12.874 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:12.874 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.874 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.874 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:12.874 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:51:12.874 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:12.874 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.875 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.875 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:12.875 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:51:12.875 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8000000000000001e-07 2024-09-16T20:51:12.875 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.875 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.875 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:12.875 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864626+0000", 2024-09-16T20:51:12.875 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.745e-06 2024-09-16T20:51:12.875 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.875 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.875 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:12.875 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864633+0000", 2024-09-16T20:51:12.875 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3339999999999998e-06 2024-09-16T20:51:12.876 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.876 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.876 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:12.876 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864636+0000", 2024-09-16T20:51:12.876 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1200000000000002e-06 2024-09-16T20:51:12.876 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.876 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.876 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:12.876 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965299+0000", 2024-09-16T20:51:12.876 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10066352000000001 2024-09-16T20:51:12.876 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.876 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.876 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:12.877 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965390+0000", 2024-09-16T20:51:12.877 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0894000000000001e-05 2024-09-16T20:51:12.877 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.877 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.877 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:12.877 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965521+0000", 2024-09-16T20:51:12.877 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013110899999999999 2024-09-16T20:51:12.877 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:12.877 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:12.877 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:12.877 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.877 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.877 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:152 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 327680~32768 out=32768b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:51:12.878 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:51:12.878 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 168.64807297499999, 2024-09-16T20:51:12.878 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11869405800000001, 2024-09-16T20:51:12.878 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:12.878 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:12.878 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:12.878 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:12.878 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:51:12.878 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:51:12.878 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 152 2024-09-16T20:51:12.878 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.878 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:12.878 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.878 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:12.879 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:51:12.879 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:12.879 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.879 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.879 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:12.879 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:51:12.879 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:12.879 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.879 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.879 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:12.879 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163122+0000", 2024-09-16T20:51:12.879 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6400000000000003e-07 2024-09-16T20:51:12.879 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.879 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.880 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:12.880 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163124+0000", 2024-09-16T20:51:12.880 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.565e-06 2024-09-16T20:51:12.880 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.880 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.880 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:12.880 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163136+0000", 2024-09-16T20:51:12.880 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1717e-05 2024-09-16T20:51:12.880 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.880 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.880 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:12.880 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163142+0000", 2024-09-16T20:51:12.880 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1750000000000002e-06 2024-09-16T20:51:12.881 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.881 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.881 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:12.881 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163606+0000", 2024-09-16T20:51:12.881 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00046418799999999998 2024-09-16T20:51:12.881 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.881 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.881 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:12.881 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163656+0000", 2024-09-16T20:51:12.881 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9852999999999999e-05 2024-09-16T20:51:12.881 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.881 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.881 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:12.881 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281815+0000", 2024-09-16T20:51:12.882 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11815909600000001 2024-09-16T20:51:12.882 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:12.882 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:12.882 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:12.882 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.882 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.882 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:153 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:51:12.882 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:51:12.882 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 168.648040022, 2024-09-16T20:51:12.882 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11903462400000001, 2024-09-16T20:51:12.882 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:12.882 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:12.882 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:12.883 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:12.883 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:51:12.883 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:51:12.883 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 153 2024-09-16T20:51:12.883 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.883 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:12.883 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.883 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:12.883 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:51:12.883 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:12.883 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.883 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.883 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:12.884 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:51:12.884 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:12.884 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.884 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.884 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:12.884 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163155+0000", 2024-09-16T20:51:12.884 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.37e-07 2024-09-16T20:51:12.884 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.884 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.884 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:12.884 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163157+0000", 2024-09-16T20:51:12.884 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6010000000000002e-06 2024-09-16T20:51:12.884 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.885 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.885 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:12.885 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163165+0000", 2024-09-16T20:51:12.885 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2619999999999998e-06 2024-09-16T20:51:12.885 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.885 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.885 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:12.885 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163171+0000", 2024-09-16T20:51:12.885 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.711e-06 2024-09-16T20:51:12.885 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.885 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.885 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:12.885 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281868+0000", 2024-09-16T20:51:12.885 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118696622 2024-09-16T20:51:12.886 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.886 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.886 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:12.886 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281969+0000", 2024-09-16T20:51:12.886 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101371 2024-09-16T20:51:12.886 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.886 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.886 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:12.886 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282189+0000", 2024-09-16T20:51:12.886 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00021952000000000001 2024-09-16T20:51:12.886 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:12.886 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:12.886 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:12.887 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.887 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.887 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:154 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 360448~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:51:12.887 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:51:12.887 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 168.64800825, 2024-09-16T20:51:12.887 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11932035100000001, 2024-09-16T20:51:12.887 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:12.887 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:12.887 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:12.887 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:12.887 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:51:12.887 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:51:12.887 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 154 2024-09-16T20:51:12.887 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.887 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:12.888 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.888 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:12.888 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:51:12.888 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:12.888 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.888 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.888 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:12.888 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:51:12.888 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:12.888 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.888 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.888 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:12.889 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:51:12.889 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:51:12.889 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.889 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.889 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:12.889 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163189+0000", 2024-09-16T20:51:12.889 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6840000000000001e-06 2024-09-16T20:51:12.889 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.889 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.889 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:12.889 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163196+0000", 2024-09-16T20:51:12.889 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7920000000000004e-06 2024-09-16T20:51:12.889 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.890 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.890 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:12.890 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163201+0000", 2024-09-16T20:51:12.890 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2809999999999999e-06 2024-09-16T20:51:12.890 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.890 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.890 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:12.890 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282219+0000", 2024-09-16T20:51:12.890 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.119018343 2024-09-16T20:51:12.890 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.890 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.890 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:12.890 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282305+0000", 2024-09-16T20:51:12.890 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5865000000000005e-05 2024-09-16T20:51:12.891 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.891 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.891 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:12.891 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282506+0000", 2024-09-16T20:51:12.891 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000200927 2024-09-16T20:51:12.891 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:12.891 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:12.891 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:12.891 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.891 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.891 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop(client.4366.0:3088 3.1e e38/23 3:79e5ec01:::rbd_header.110e891a9b2b:head v 38'306, mlcod=36'304)", 2024-09-16T20:51:12.891 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:51:12.891 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 168.64698938199999, 2024-09-16T20:51:12.891 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118102759, 2024-09-16T20:51:12.892 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:12.892 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:12.892 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:12.892 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:12.892 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.892 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:12.892 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:51:12.892 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:12.892 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.892 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.892 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:12.892 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:51:12.892 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:12.892 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.893 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.893 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:12.893 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:51:12.893 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.85e-07 2024-09-16T20:51:12.893 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.893 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.893 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:12.893 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164212+0000", 2024-09-16T20:51:12.893 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3999999999999997e-06 2024-09-16T20:51:12.893 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.893 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.893 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:12.893 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281553+0000", 2024-09-16T20:51:12.893 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11734135499999999 2024-09-16T20:51:12.894 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.894 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.894 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:12.894 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281567+0000", 2024-09-16T20:51:12.894 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4188999999999999e-05 2024-09-16T20:51:12.894 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.894 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.894 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:12.894 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281653+0000", 2024-09-16T20:51:12.894 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5385999999999999e-05 2024-09-16T20:51:12.894 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.894 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.894 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:12.894 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281679+0000", 2024-09-16T20:51:12.895 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6689000000000001e-05 2024-09-16T20:51:12.895 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.895 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.895 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:12.895 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282277+0000", 2024-09-16T20:51:12.895 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00059798000000000002 2024-09-16T20:51:12.895 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.895 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.895 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:12.895 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282307+0000", 2024-09-16T20:51:12.895 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9975e-05 2024-09-16T20:51:12.895 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:12.895 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:12.895 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:12.896 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.896 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.896 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5255.0:44 3.16 3:6f5476df:::rbd_header.110e25db1f60:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:51:12.896 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:51:12.896 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 96.490686327000006, 2024-09-16T20:51:12.896 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.121007509, 2024-09-16T20:51:12.896 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:12.896 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:12.896 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:12.896 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:12.896 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5255", 2024-09-16T20:51:12.896 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:51:12.896 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 44 2024-09-16T20:51:12.896 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.897 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:12.897 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.897 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:12.897 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:51:12.897 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:12.897 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.897 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.897 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:12.897 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:51:12.897 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:12.900 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.900 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.900 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:12.900 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:51:12.900 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0500000000000003e-07 2024-09-16T20:51:12.900 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.900 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.900 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:12.900 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320512+0000", 2024-09-16T20:51:12.901 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2540000000000001e-06 2024-09-16T20:51:12.901 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.901 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.901 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:12.901 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320518+0000", 2024-09-16T20:51:12.901 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1689999999999997e-06 2024-09-16T20:51:12.901 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.901 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.901 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:12.901 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320521+0000", 2024-09-16T20:51:12.901 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5200000000000002e-06 2024-09-16T20:51:12.901 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.901 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.902 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:12.902 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320548+0000", 2024-09-16T20:51:12.902 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6662999999999999e-05 2024-09-16T20:51:12.902 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.902 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.902 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:12.902 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320587+0000", 2024-09-16T20:51:12.902 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8866999999999999e-05 2024-09-16T20:51:12.902 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.902 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.902 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:12.902 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.441515+0000", 2024-09-16T20:51:12.902 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.120928331 2024-09-16T20:51:12.903 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:12.903 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:12.903 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:12.903 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.903 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.903 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:20329 3.5 3:a4ea64b7:::rbd_header.110ede5ef857:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:12.903 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:01.061694+0000", 2024-09-16T20:51:12.903 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 71.749499759000003, 2024-09-16T20:51:12.903 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.071234077000000007, 2024-09-16T20:51:12.903 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:12.903 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:12.903 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:12.903 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:12.903 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:51:12.904 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:12.904 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 20329 2024-09-16T20:51:12.904 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.904 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:12.904 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.904 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:12.904 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.061694+0000", 2024-09-16T20:51:12.904 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:12.904 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.905 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.905 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:12.905 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.061694+0000", 2024-09-16T20:51:12.905 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:12.905 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.905 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.905 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:12.905 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089232+0000", 2024-09-16T20:51:12.905 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.027537275 2024-09-16T20:51:12.905 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.905 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.905 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:12.905 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089237+0000", 2024-09-16T20:51:12.905 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4709999999999998e-06 2024-09-16T20:51:12.906 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.906 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.906 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:12.906 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089246+0000", 2024-09-16T20:51:12.906 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8230000000000001e-06 2024-09-16T20:51:12.906 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.906 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.906 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:12.906 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089252+0000", 2024-09-16T20:51:12.906 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3060000000000003e-06 2024-09-16T20:51:12.906 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.906 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.906 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:12.907 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.132756+0000", 2024-09-16T20:51:12.907 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.043504334999999998 2024-09-16T20:51:12.907 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.907 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.907 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:12.907 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.132829+0000", 2024-09-16T20:51:12.907 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2849999999999995e-05 2024-09-16T20:51:12.907 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:12.907 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:12.907 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:12.907 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.132928+0000", 2024-09-16T20:51:12.907 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.9017000000000004e-05 2024-09-16T20:51:12.907 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:12.907 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:12.908 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:12.908 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:12.908 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:12.908 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:12.909 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:51:12.910 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:12.909+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:13.010 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:13.008+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:13.030 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:13.030 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:51:13.030 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:51:13.030 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:13.039 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:51:13.111 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:13.110+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:13.188 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:13.188 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:51:13.188 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:51:13.188 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:51:13.188 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:13.196 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:51:13.240 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:13.239+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:13.305 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:13.305 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:51:13.305 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:51:13.308 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:51:13.308 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.309 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:3088 3.1e 3:79e5ec01:::rbd_header.110e891a9b2b:head [watch watch cookie 140213437171488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:51:13.309 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:51:13.309 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 169.13906133699999, 2024-09-16T20:51:13.309 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.11895186200000001, 2024-09-16T20:51:13.309 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:13.309 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:13.309 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:13.309 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:13.309 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:13.309 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:13.309 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3088 2024-09-16T20:51:13.309 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.309 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:13.310 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.310 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:13.310 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:51:13.310 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.310 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.310 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.310 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:13.310 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:51:13.310 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.310 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.310 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.310 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:13.310 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163846+0000", 2024-09-16T20:51:13.311 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6799999999999999e-07 2024-09-16T20:51:13.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.311 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.311 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:13.311 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163850+0000", 2024-09-16T20:51:13.311 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.929e-06 2024-09-16T20:51:13.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.311 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.311 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:13.311 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163862+0000", 2024-09-16T20:51:13.311 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1972e-05 2024-09-16T20:51:13.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.311 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.312 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:13.312 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163867+0000", 2024-09-16T20:51:13.312 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.451e-06 2024-09-16T20:51:13.312 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.312 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.312 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:13.312 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163912+0000", 2024-09-16T20:51:13.312 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4456000000000003e-05 2024-09-16T20:51:13.312 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.312 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.312 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:13.312 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163962+0000", 2024-09-16T20:51:13.312 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0578000000000003e-05 2024-09-16T20:51:13.312 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.313 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.313 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-16T20:51:13.313 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164058+0000", 2024-09-16T20:51:13.313 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5934999999999995e-05 2024-09-16T20:51:13.313 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.314 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.314 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:51:13.314 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164570+0000", 2024-09-16T20:51:13.314 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00051241100000000005 2024-09-16T20:51:13.314 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.314 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.314 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:13.314 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282629+0000", 2024-09-16T20:51:13.314 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.118058986 2024-09-16T20:51:13.314 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.314 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.314 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:51:13.314 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282725+0000", 2024-09-16T20:51:13.314 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5063999999999996e-05 2024-09-16T20:51:13.315 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.315 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.315 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:13.315 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282797+0000", 2024-09-16T20:51:13.315 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2212000000000006e-05 2024-09-16T20:51:13.315 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.315 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:13.315 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.315 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.315 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.315 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:8039 3.1e 3.ac44ff9e (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:51:13.315 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:51:13.315 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 143.59523398799999, 2024-09-16T20:51:13.316 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0053310449999999, 2024-09-16T20:51:13.316 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:13.316 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:13.316 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:13.316 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:13.316 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:13.316 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:13.316 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8039 2024-09-16T20:51:13.316 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.317 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:13.317 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.317 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:13.317 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:51:13.317 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.317 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.317 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:13.317 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:51:13.317 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.317 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.317 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:13.318 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707674+0000", 2024-09-16T20:51:13.318 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6530000000000001e-06 2024-09-16T20:51:13.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.318 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.318 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:13.318 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707681+0000", 2024-09-16T20:51:13.318 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9639999999999997e-06 2024-09-16T20:51:13.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.318 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.318 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:13.318 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707700+0000", 2024-09-16T20:51:13.318 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8674000000000001e-05 2024-09-16T20:51:13.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.318 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.319 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:13.319 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707708+0000", 2024-09-16T20:51:13.319 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6810000000000004e-06 2024-09-16T20:51:13.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.319 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.319 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:13.319 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711293+0000", 2024-09-16T20:51:13.319 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035848189999999999 2024-09-16T20:51:13.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.319 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.319 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:13.319 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711301+0000", 2024-09-16T20:51:13.319 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4730000000000003e-06 2024-09-16T20:51:13.320 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.320 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.320 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:13.320 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.712990+0000", 2024-09-16T20:51:13.320 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.001689348 2024-09-16T20:51:13.320 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.320 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.320 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:13.320 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713003+0000", 2024-09-16T20:51:13.320 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3433e-05 2024-09-16T20:51:13.320 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.320 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:13.320 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.321 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.321 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.321 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:8040 3.1e 3.ac44ff9e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:51:13.321 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:51:13.321 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 143.595173593, 2024-09-16T20:51:13.321 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0053410780000001, 2024-09-16T20:51:13.321 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:13.321 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:13.321 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:13.321 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:13.321 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:13.321 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:13.321 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8040 2024-09-16T20:51:13.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.322 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:13.322 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.322 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:13.322 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:51:13.322 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.322 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.322 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:13.322 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:51:13.322 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.322 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.322 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:13.323 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:51:13.323 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8100000000000003e-07 2024-09-16T20:51:13.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.323 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.323 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:13.323 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707737+0000", 2024-09-16T20:51:13.323 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5020000000000001e-06 2024-09-16T20:51:13.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.323 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.323 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:13.323 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707745+0000", 2024-09-16T20:51:13.323 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2169999999999994e-06 2024-09-16T20:51:13.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.323 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.324 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:13.324 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707751+0000", 2024-09-16T20:51:13.324 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.263e-06 2024-09-16T20:51:13.324 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.324 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.324 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:13.324 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711336+0000", 2024-09-16T20:51:13.324 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035848360000000001 2024-09-16T20:51:13.324 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.324 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.324 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:13.324 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711340+0000", 2024-09-16T20:51:13.325 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1309999999999999e-06 2024-09-16T20:51:13.325 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.325 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.325 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:13.325 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713060+0000", 2024-09-16T20:51:13.325 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0017195059999999 2024-09-16T20:51:13.325 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.325 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.325 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:13.325 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713074+0000", 2024-09-16T20:51:13.325 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4042e-05 2024-09-16T20:51:13.325 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.325 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:13.326 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.326 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.326 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.326 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4927.0:4487 3.1e 3.ac44ff9e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:51:13.326 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:51:13.326 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 143.59465339100001, 2024-09-16T20:51:13.326 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0048840880000001, 2024-09-16T20:51:13.326 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:13.326 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:13.326 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:13.326 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:13.326 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4927", 2024-09-16T20:51:13.326 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:13.327 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4487 2024-09-16T20:51:13.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.327 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:13.327 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.327 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:13.327 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:51:13.327 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.327 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.327 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:13.327 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:51:13.327 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.327 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.328 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:13.328 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708254+0000", 2024-09-16T20:51:13.328 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1800000000000005e-07 2024-09-16T20:51:13.328 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.328 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.328 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:13.328 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708258+0000", 2024-09-16T20:51:13.328 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8689999999999997e-06 2024-09-16T20:51:13.328 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.328 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.328 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:13.328 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708267+0000", 2024-09-16T20:51:13.328 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9590000000000007e-06 2024-09-16T20:51:13.329 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.329 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.329 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:13.329 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708274+0000", 2024-09-16T20:51:13.329 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9970000000000001e-06 2024-09-16T20:51:13.329 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.329 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.329 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:13.329 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711366+0000", 2024-09-16T20:51:13.329 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030919530000000001 2024-09-16T20:51:13.329 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.329 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.329 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:13.329 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711372+0000", 2024-09-16T20:51:13.330 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9410000000000004e-06 2024-09-16T20:51:13.330 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.330 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.330 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:13.330 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713126+0000", 2024-09-16T20:51:13.330 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0017537249999999 2024-09-16T20:51:13.330 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.330 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.330 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:13.330 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713137+0000", 2024-09-16T20:51:13.330 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0825999999999999e-05 2024-09-16T20:51:13.330 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.330 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:13.331 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.331 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.331 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.331 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5089.0:11703 3.1b 3:dcd4684e:::rbd_data.110e747f0779.000000000000001b:head [set-alloc-hint object_size 262144 write_size 262144,write 139264~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:51:13.331 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:51:13.331 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 129.16491761699999, 2024-09-16T20:51:13.331 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.084231385000000006, 2024-09-16T20:51:13.331 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:13.331 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:13.331 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:13.331 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:13.331 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5089", 2024-09-16T20:51:13.331 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:51:13.332 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 11703 2024-09-16T20:51:13.332 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.332 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:13.332 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.332 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:13.332 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:51:13.332 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.332 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.332 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.332 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:13.332 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:51:13.332 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.332 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.332 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.333 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:13.333 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:51:13.333 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2600000000000001e-07 2024-09-16T20:51:13.333 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.333 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.333 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:13.333 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137998+0000", 2024-09-16T20:51:13.333 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9549999999999998e-06 2024-09-16T20:51:13.333 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.333 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.333 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:13.333 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138008+0000", 2024-09-16T20:51:13.334 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3789999999999995e-06 2024-09-16T20:51:13.334 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.334 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.334 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:13.334 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138013+0000", 2024-09-16T20:51:13.334 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2209999999999996e-06 2024-09-16T20:51:13.334 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.334 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.334 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:13.334 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138054+0000", 2024-09-16T20:51:13.334 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1022999999999997e-05 2024-09-16T20:51:13.334 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.334 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.334 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:13.335 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138124+0000", 2024-09-16T20:51:13.335 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0234999999999994e-05 2024-09-16T20:51:13.335 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.335 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.335 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:13.335 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138197+0000", 2024-09-16T20:51:13.335 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2451000000000001e-05 2024-09-16T20:51:13.335 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.335 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.335 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:13.335 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222010+0000", 2024-09-16T20:51:13.335 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.083813433000000007 2024-09-16T20:51:13.335 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.335 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.336 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:51:13.336 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222163+0000", 2024-09-16T20:51:13.336 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015274599999999999 2024-09-16T20:51:13.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.336 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.336 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:51:13.336 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222197+0000", 2024-09-16T20:51:13.336 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3683000000000003e-05 2024-09-16T20:51:13.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.336 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.336 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:13.336 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222220+0000", 2024-09-16T20:51:13.336 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3433000000000001e-05 2024-09-16T20:51:13.336 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.337 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:13.337 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.337 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.337 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.5089.0:11702 3.15 e73/64 3:acf33c8b:::rbd_data.110e747f0779.000000000000001a:head v 73'132, mlcod=73'130)", 2024-09-16T20:51:13.337 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:51:13.337 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 129.16465028799999, 2024-09-16T20:51:13.337 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.084079763000000002, 2024-09-16T20:51:13.337 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:13.337 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:13.337 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:13.337 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:13.337 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.337 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:13.338 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:51:13.338 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.338 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.338 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.338 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:13.338 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:51:13.338 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.338 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.338 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.338 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:13.338 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:51:13.338 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:51:13.338 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.338 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.339 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:13.339 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138263+0000", 2024-09-16T20:51:13.339 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1e-06 2024-09-16T20:51:13.339 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.339 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.339 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:13.339 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221589+0000", 2024-09-16T20:51:13.339 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.083326002999999996 2024-09-16T20:51:13.339 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.339 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.339 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:13.339 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221614+0000", 2024-09-16T20:51:13.340 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5593999999999999e-05 2024-09-16T20:51:13.340 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.340 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.340 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:13.340 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221714+0000", 2024-09-16T20:51:13.340 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9889000000000005e-05 2024-09-16T20:51:13.340 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.340 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.340 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:13.340 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221727+0000", 2024-09-16T20:51:13.340 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3125999999999999e-05 2024-09-16T20:51:13.340 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.340 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.340 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:51:13.341 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222294+0000", 2024-09-16T20:51:13.341 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00056712199999999998 2024-09-16T20:51:13.341 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.341 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.341 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:13.341 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222336+0000", 2024-09-16T20:51:13.341 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1470000000000001e-05 2024-09-16T20:51:13.341 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.341 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:13.341 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.341 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.341 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.341 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5138.0:9358 3.c 3:33996a43:::rbd_data.110e6372565d.0000000000000007:head [read 1343488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T20:51:13.342 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:51:13.342 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 105.45037487800001, 2024-09-16T20:51:13.342 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0167493569999999, 2024-09-16T20:51:13.342 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:13.342 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:13.342 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:13.342 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:13.342 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5138", 2024-09-16T20:51:13.342 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:51:13.342 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9358 2024-09-16T20:51:13.342 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.342 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:13.342 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.342 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:13.343 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:51:13.343 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.343 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.343 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.343 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:13.343 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:51:13.343 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.343 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.343 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.343 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:13.343 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852533+0000", 2024-09-16T20:51:13.343 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.646e-06 2024-09-16T20:51:13.343 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.344 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.344 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:13.344 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852539+0000", 2024-09-16T20:51:13.344 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3389999999999998e-06 2024-09-16T20:51:13.344 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.344 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.344 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:13.344 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852552+0000", 2024-09-16T20:51:13.344 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.271e-05 2024-09-16T20:51:13.344 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.344 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.344 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:13.344 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852563+0000", 2024-09-16T20:51:13.345 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1139e-05 2024-09-16T20:51:13.345 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.345 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.345 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:13.345 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852883+0000", 2024-09-16T20:51:13.345 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00032010600000000001 2024-09-16T20:51:13.345 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.345 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.345 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:13.345 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852890+0000", 2024-09-16T20:51:13.345 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7669999999999998e-06 2024-09-16T20:51:13.345 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.345 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.345 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:13.346 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869059+0000", 2024-09-16T20:51:13.346 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.016169353 2024-09-16T20:51:13.346 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.346 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.346 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:13.346 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869178+0000", 2024-09-16T20:51:13.346 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118193 2024-09-16T20:51:13.346 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.346 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.346 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:13.346 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869281+0000", 2024-09-16T20:51:13.346 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010310399999999999 2024-09-16T20:51:13.346 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.347 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:13.347 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.347 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.347 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.347 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5138.0:9382 3.c 3:33996a43:::rbd_data.110e6372565d.0000000000000007:head [read 0~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T20:51:13.347 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:51:13.347 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 105.450324001, 2024-09-16T20:51:13.347 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.017136034, 2024-09-16T20:51:13.347 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:13.347 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:13.347 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:13.347 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:13.347 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5138", 2024-09-16T20:51:13.347 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:51:13.348 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9382 2024-09-16T20:51:13.348 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.348 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:13.348 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.348 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:13.348 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:51:13.348 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.348 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.348 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.348 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:13.348 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:51:13.348 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.348 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.349 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.349 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:13.349 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852583+0000", 2024-09-16T20:51:13.349 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.51e-07 2024-09-16T20:51:13.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.349 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.349 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:13.349 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852586+0000", 2024-09-16T20:51:13.349 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0280000000000001e-06 2024-09-16T20:51:13.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.349 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.349 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:13.349 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852595+0000", 2024-09-16T20:51:13.349 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2799999999999992e-06 2024-09-16T20:51:13.350 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.350 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.350 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:13.350 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852601+0000", 2024-09-16T20:51:13.350 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3810000000000004e-06 2024-09-16T20:51:13.350 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.350 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.350 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:13.350 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852921+0000", 2024-09-16T20:51:13.350 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00031979699999999997 2024-09-16T20:51:13.350 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.350 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.350 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:13.351 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852930+0000", 2024-09-16T20:51:13.351 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6489999999999997e-06 2024-09-16T20:51:13.351 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.351 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.351 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:13.351 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869434+0000", 2024-09-16T20:51:13.351 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0165038829999999 2024-09-16T20:51:13.351 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.351 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.351 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:13.351 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869495+0000", 2024-09-16T20:51:13.351 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1509999999999999e-05 2024-09-16T20:51:13.351 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.351 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.351 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:13.352 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869718+0000", 2024-09-16T20:51:13.352 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000222955 2024-09-16T20:51:13.352 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.352 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:13.352 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.352 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.352 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.352 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9969 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 632832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:13.352 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642045+0000", 2024-09-16T20:51:13.352 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 82.660861543999999, 2024-09-16T20:51:13.352 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076903192999999995, 2024-09-16T20:51:13.352 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:13.352 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:13.352 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:13.353 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:13.353 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:13.353 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:13.353 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9969 2024-09-16T20:51:13.353 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.353 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:13.353 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.353 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:13.353 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642045+0000", 2024-09-16T20:51:13.353 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.353 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.353 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.354 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:13.354 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642045+0000", 2024-09-16T20:51:13.354 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.354 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.354 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.354 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:13.354 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642045+0000", 2024-09-16T20:51:13.354 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5100000000000001e-07 2024-09-16T20:51:13.354 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.354 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.354 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:13.354 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642047+0000", 2024-09-16T20:51:13.354 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8679999999999999e-06 2024-09-16T20:51:13.354 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.355 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.355 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:13.355 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642053+0000", 2024-09-16T20:51:13.355 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1720000000000004e-06 2024-09-16T20:51:13.355 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.355 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.355 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:13.355 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642060+0000", 2024-09-16T20:51:13.355 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0339999999999999e-06 2024-09-16T20:51:13.355 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.355 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.355 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:13.355 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.718562+0000", 2024-09-16T20:51:13.355 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076501969000000003 2024-09-16T20:51:13.356 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.356 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.356 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:13.356 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.718643+0000", 2024-09-16T20:51:13.356 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0539999999999998e-05 2024-09-16T20:51:13.356 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.356 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.356 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:13.356 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.718948+0000", 2024-09-16T20:51:13.356 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000305259 2024-09-16T20:51:13.356 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.356 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:13.356 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.357 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.357 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.357 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9970 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 637952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:13.357 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642067+0000", 2024-09-16T20:51:13.357 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 82.660839564, 2024-09-16T20:51:13.357 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077174928000000004, 2024-09-16T20:51:13.357 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:13.357 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:13.357 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:13.357 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:13.357 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:13.357 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:13.357 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9970 2024-09-16T20:51:13.358 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.358 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:13.358 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.358 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:13.358 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642067+0000", 2024-09-16T20:51:13.358 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.358 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.358 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.358 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:13.358 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642067+0000", 2024-09-16T20:51:13.358 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.358 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.358 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.359 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:13.359 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642067+0000", 2024-09-16T20:51:13.359 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2499999999999995e-07 2024-09-16T20:51:13.359 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.359 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.359 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:13.359 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642070+0000", 2024-09-16T20:51:13.359 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3199999999999998e-06 2024-09-16T20:51:13.359 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.359 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.359 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:13.359 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642076+0000", 2024-09-16T20:51:13.359 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.122e-06 2024-09-16T20:51:13.360 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.360 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.360 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:13.360 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642081+0000", 2024-09-16T20:51:13.360 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0869999999999997e-06 2024-09-16T20:51:13.360 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.360 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.360 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:13.360 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.718980+0000", 2024-09-16T20:51:13.360 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076899482000000005 2024-09-16T20:51:13.360 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.360 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.360 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:13.361 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719085+0000", 2024-09-16T20:51:13.361 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010446399999999999 2024-09-16T20:51:13.361 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.361 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.361 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:13.361 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719242+0000", 2024-09-16T20:51:13.361 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000156928 2024-09-16T20:51:13.361 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.361 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:13.361 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.361 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.361 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.361 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9971 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 643072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:13.361 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642102+0000", 2024-09-16T20:51:13.362 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 82.660803857000005, 2024-09-16T20:51:13.362 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077421016999999995, 2024-09-16T20:51:13.362 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:13.362 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:13.362 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:13.362 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:13.362 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:13.362 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:13.362 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9971 2024-09-16T20:51:13.362 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.362 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:13.362 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.362 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:13.362 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642102+0000", 2024-09-16T20:51:13.363 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.363 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.363 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.363 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:13.363 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642102+0000", 2024-09-16T20:51:13.363 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.363 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.363 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.363 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:13.363 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642103+0000", 2024-09-16T20:51:13.363 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7399999999999998e-07 2024-09-16T20:51:13.363 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.363 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.364 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:13.364 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642105+0000", 2024-09-16T20:51:13.364 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2649999999999999e-06 2024-09-16T20:51:13.364 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.364 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.364 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:13.364 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642112+0000", 2024-09-16T20:51:13.364 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.294e-06 2024-09-16T20:51:13.364 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.364 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.364 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:13.364 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642118+0000", 2024-09-16T20:51:13.364 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4500000000000003e-06 2024-09-16T20:51:13.364 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.365 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.365 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:13.365 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719285+0000", 2024-09-16T20:51:13.365 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077166883000000006 2024-09-16T20:51:13.365 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.365 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.365 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:13.365 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719368+0000", 2024-09-16T20:51:13.365 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2811000000000004e-05 2024-09-16T20:51:13.365 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.365 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.365 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:13.365 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719523+0000", 2024-09-16T20:51:13.365 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015584 2024-09-16T20:51:13.366 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.366 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:13.366 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.366 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.366 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.366 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9972 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 648192~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:13.366 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642125+0000", 2024-09-16T20:51:13.366 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 82.66078091, 2024-09-16T20:51:13.366 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077710324999999997, 2024-09-16T20:51:13.366 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:13.366 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:13.366 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:13.366 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:13.367 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:13.367 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:13.367 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9972 2024-09-16T20:51:13.367 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.367 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:13.367 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.367 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:13.367 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642125+0000", 2024-09-16T20:51:13.367 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.367 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.367 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.367 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:13.367 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642125+0000", 2024-09-16T20:51:13.368 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.368 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.368 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.368 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:13.368 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642126+0000", 2024-09-16T20:51:13.368 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1899999999999998e-07 2024-09-16T20:51:13.368 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.368 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.368 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:13.368 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642128+0000", 2024-09-16T20:51:13.368 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8619999999999999e-06 2024-09-16T20:51:13.368 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.369 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.369 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:13.369 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642134+0000", 2024-09-16T20:51:13.369 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5289999999999997e-06 2024-09-16T20:51:13.369 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.369 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.369 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:13.369 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642139+0000", 2024-09-16T20:51:13.369 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1250000000000001e-06 2024-09-16T20:51:13.369 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.369 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.369 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:13.369 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719570+0000", 2024-09-16T20:51:13.369 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077430516000000005 2024-09-16T20:51:13.370 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.370 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.370 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:13.370 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719655+0000", 2024-09-16T20:51:13.370 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5500000000000005e-05 2024-09-16T20:51:13.370 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.370 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.370 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:13.370 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719836+0000", 2024-09-16T20:51:13.370 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018037399999999999 2024-09-16T20:51:13.370 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.370 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:13.370 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.370 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.371 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.371 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9973 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 653312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:13.371 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:51:13.371 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 82.660760300000007, 2024-09-16T20:51:13.371 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077905262000000003, 2024-09-16T20:51:13.371 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:13.371 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:13.371 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:13.371 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:13.371 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:13.371 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:13.371 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9973 2024-09-16T20:51:13.371 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.372 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:13.372 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.372 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:13.372 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:51:13.372 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.372 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.372 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.372 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:13.372 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:51:13.372 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.372 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.372 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.372 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:13.373 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:51:13.373 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:51:13.373 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.373 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.373 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:13.373 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642148+0000", 2024-09-16T20:51:13.373 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0219999999999999e-06 2024-09-16T20:51:13.373 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.373 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.373 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:13.373 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642154+0000", 2024-09-16T20:51:13.373 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8540000000000002e-06 2024-09-16T20:51:13.373 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.373 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.374 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:13.374 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642159+0000", 2024-09-16T20:51:13.374 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.045e-06 2024-09-16T20:51:13.374 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.374 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.374 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:13.374 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719875+0000", 2024-09-16T20:51:13.374 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077716116000000002 2024-09-16T20:51:13.374 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.374 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.374 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:13.374 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719934+0000", 2024-09-16T20:51:13.374 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8819e-05 2024-09-16T20:51:13.375 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.375 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.375 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:13.375 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720051+0000", 2024-09-16T20:51:13.375 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000116947 2024-09-16T20:51:13.375 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.375 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:13.375 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.375 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.375 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.375 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9974 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 658432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:13.375 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642166+0000", 2024-09-16T20:51:13.375 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 82.660740386000001, 2024-09-16T20:51:13.375 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.078179786000000001, 2024-09-16T20:51:13.376 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:13.376 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:13.376 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:13.376 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:13.376 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:13.376 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:13.376 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9974 2024-09-16T20:51:13.376 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.376 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:13.376 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.376 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:13.376 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642166+0000", 2024-09-16T20:51:13.376 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.377 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.377 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.377 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:13.377 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642166+0000", 2024-09-16T20:51:13.377 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.377 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.377 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.377 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:13.377 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647343+0000", 2024-09-16T20:51:13.377 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0051769440000000002 2024-09-16T20:51:13.377 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.377 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.377 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:13.378 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647348+0000", 2024-09-16T20:51:13.378 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3619999999999997e-06 2024-09-16T20:51:13.378 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.378 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.378 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:13.378 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647359+0000", 2024-09-16T20:51:13.378 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0929e-05 2024-09-16T20:51:13.378 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.378 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.378 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:13.378 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647366+0000", 2024-09-16T20:51:13.378 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0910000000000004e-06 2024-09-16T20:51:13.378 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.379 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.379 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:13.379 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720094+0000", 2024-09-16T20:51:13.379 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.072727403999999995 2024-09-16T20:51:13.379 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.379 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.379 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:13.379 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720173+0000", 2024-09-16T20:51:13.379 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9382000000000007e-05 2024-09-16T20:51:13.379 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.379 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.379 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:13.379 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720346+0000", 2024-09-16T20:51:13.379 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017267399999999999 2024-09-16T20:51:13.380 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.380 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:13.380 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.380 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.380 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.380 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9990 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 740352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:13.380 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647725+0000", 2024-09-16T20:51:13.380 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 82.655181302000003, 2024-09-16T20:51:13.380 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077029919000000002, 2024-09-16T20:51:13.380 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:13.380 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:13.380 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:13.380 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:13.380 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:13.381 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:13.381 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9990 2024-09-16T20:51:13.381 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.381 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:13.381 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.381 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:13.381 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647725+0000", 2024-09-16T20:51:13.381 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.381 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.381 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.381 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:13.381 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647725+0000", 2024-09-16T20:51:13.381 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.382 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.382 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.382 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:13.382 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647725+0000", 2024-09-16T20:51:13.382 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2399999999999999e-07 2024-09-16T20:51:13.382 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.382 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.382 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:13.382 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647726+0000", 2024-09-16T20:51:13.382 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.116e-06 2024-09-16T20:51:13.382 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.382 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.382 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:13.383 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647729+0000", 2024-09-16T20:51:13.383 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.937e-06 2024-09-16T20:51:13.383 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.383 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.383 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:13.383 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647732+0000", 2024-09-16T20:51:13.383 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8739999999999999e-06 2024-09-16T20:51:13.383 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.383 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.383 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:13.383 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724612+0000", 2024-09-16T20:51:13.383 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076880102000000006 2024-09-16T20:51:13.383 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.383 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.384 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:13.384 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724649+0000", 2024-09-16T20:51:13.384 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6738e-05 2024-09-16T20:51:13.384 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.384 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.384 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:13.384 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724755+0000", 2024-09-16T20:51:13.384 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010592800000000001 2024-09-16T20:51:13.384 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.384 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:13.384 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.384 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.384 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.385 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9991 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 745472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:13.385 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:51:13.385 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 82.655170992999999, 2024-09-16T20:51:13.385 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077257820000000005, 2024-09-16T20:51:13.385 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:13.385 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:13.385 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:13.385 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:13.385 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:13.385 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:13.385 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9991 2024-09-16T20:51:13.385 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.385 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:13.386 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.386 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:13.386 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:51:13.386 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.386 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.386 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.386 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:13.386 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:51:13.386 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.386 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.386 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.386 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:13.386 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:51:13.387 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1199999999999999e-07 2024-09-16T20:51:13.387 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.387 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.387 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:13.387 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647736+0000", 2024-09-16T20:51:13.387 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3699999999999999e-07 2024-09-16T20:51:13.387 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.387 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.387 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:13.387 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647739+0000", 2024-09-16T20:51:13.387 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9359999999999999e-06 2024-09-16T20:51:13.387 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.387 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.387 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:13.388 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647742+0000", 2024-09-16T20:51:13.388 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5720000000000001e-06 2024-09-16T20:51:13.388 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.388 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.388 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:13.388 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724774+0000", 2024-09-16T20:51:13.388 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077032457999999998 2024-09-16T20:51:13.388 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.388 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.388 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:13.388 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724818+0000", 2024-09-16T20:51:13.388 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3803000000000001e-05 2024-09-16T20:51:13.388 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.389 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.389 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:13.389 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724993+0000", 2024-09-16T20:51:13.389 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000174902 2024-09-16T20:51:13.389 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.389 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:13.389 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.389 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.389 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.389 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9992 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 750592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:13.389 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:51:13.389 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 82.655161403999998, 2024-09-16T20:51:13.389 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077553227000000002, 2024-09-16T20:51:13.389 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:13.390 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:13.390 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:13.390 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:13.390 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:13.390 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:13.390 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9992 2024-09-16T20:51:13.390 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.390 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:13.390 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.390 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:13.390 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:51:13.390 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.390 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.391 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.391 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:13.391 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:51:13.391 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.391 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.391 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.391 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:13.391 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:51:13.391 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0200000000000001e-07 2024-09-16T20:51:13.391 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.391 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.391 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:13.391 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647746+0000", 2024-09-16T20:51:13.391 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.088e-06 2024-09-16T20:51:13.392 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.392 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.392 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:13.392 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647749+0000", 2024-09-16T20:51:13.392 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9170000000000002e-06 2024-09-16T20:51:13.392 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.392 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.392 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:13.392 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647752+0000", 2024-09-16T20:51:13.392 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.774e-06 2024-09-16T20:51:13.392 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.392 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.392 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:13.393 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725042+0000", 2024-09-16T20:51:13.393 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077289952999999995 2024-09-16T20:51:13.393 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.393 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.393 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:13.393 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725113+0000", 2024-09-16T20:51:13.393 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0920999999999994e-05 2024-09-16T20:51:13.393 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.393 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.393 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:13.393 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725298+0000", 2024-09-16T20:51:13.393 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000185372 2024-09-16T20:51:13.393 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.394 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:13.394 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.394 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.394 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.394 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9993 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 755712~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:13.394 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647755+0000", 2024-09-16T20:51:13.394 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 82.655151377999999, 2024-09-16T20:51:13.394 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077725674999999994, 2024-09-16T20:51:13.394 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:13.394 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:13.394 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:13.394 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:13.394 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:13.394 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:13.395 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9993 2024-09-16T20:51:13.395 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.395 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:13.395 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.395 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:13.395 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647755+0000", 2024-09-16T20:51:13.395 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.395 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.395 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.395 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:13.395 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647755+0000", 2024-09-16T20:51:13.395 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.395 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.395 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.396 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:13.396 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653406+0000", 2024-09-16T20:51:13.396 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.005650876 2024-09-16T20:51:13.396 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.396 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.396 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:13.396 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653411+0000", 2024-09-16T20:51:13.396 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8899999999999998e-06 2024-09-16T20:51:13.396 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.396 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.396 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:13.396 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653423+0000", 2024-09-16T20:51:13.396 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2447e-05 2024-09-16T20:51:13.396 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.397 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.397 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:13.397 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653431+0000", 2024-09-16T20:51:13.397 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5730000000000002e-06 2024-09-16T20:51:13.397 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.397 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.397 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:13.397 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725333+0000", 2024-09-16T20:51:13.397 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.071902743000000005 2024-09-16T20:51:13.397 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.397 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.397 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:13.397 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725385+0000", 2024-09-16T20:51:13.398 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1585000000000002e-05 2024-09-16T20:51:13.398 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.398 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.398 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:13.398 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725481+0000", 2024-09-16T20:51:13.398 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5561000000000002e-05 2024-09-16T20:51:13.398 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.398 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:13.398 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.398 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.398 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.398 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6061.0:5400 3.b 3:d353d652:::rbd_data.110e16eafbd.0000000000000010:head [read 577536~28672] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-16T20:51:13.398 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:27.761367+0000", 2024-09-16T20:51:13.399 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 45.541539432, 2024-09-16T20:51:13.399 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0029066630000001, 2024-09-16T20:51:13.399 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:13.399 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:13.399 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:13.399 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:13.399 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6061", 2024-09-16T20:51:13.399 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3930128440", 2024-09-16T20:51:13.399 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5400 2024-09-16T20:51:13.399 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.399 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:13.399 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.399 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:13.399 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761367+0000", 2024-09-16T20:51:13.400 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.400 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.400 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.400 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:13.400 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761367+0000", 2024-09-16T20:51:13.400 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.400 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.400 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.400 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:13.400 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761369+0000", 2024-09-16T20:51:13.400 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.934e-06 2024-09-16T20:51:13.400 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.400 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.400 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:13.401 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761374+0000", 2024-09-16T20:51:13.401 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7139999999999998e-06 2024-09-16T20:51:13.401 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.401 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.401 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:13.401 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761387+0000", 2024-09-16T20:51:13.401 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2349e-05 2024-09-16T20:51:13.401 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.401 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.401 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:13.401 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761393+0000", 2024-09-16T20:51:13.401 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.652e-06 2024-09-16T20:51:13.401 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.402 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.402 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:13.402 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761442+0000", 2024-09-16T20:51:13.402 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9011000000000002e-05 2024-09-16T20:51:13.402 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.402 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.402 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:13.402 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761451+0000", 2024-09-16T20:51:13.402 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1920000000000005e-06 2024-09-16T20:51:13.402 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.402 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.402 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:13.402 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:28.764169+0000", 2024-09-16T20:51:13.402 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.002718161 2024-09-16T20:51:13.403 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.403 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.403 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:13.403 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:28.764273+0000", 2024-09-16T20:51:13.403 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010465 2024-09-16T20:51:13.403 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.403 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:13.403 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.403 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.403 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.403 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6061.0:5424 3.b 3:d353d652:::rbd_data.110e16eafbd.0000000000000010:head [read 606208~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-16T20:51:13.403 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:27.766882+0000", 2024-09-16T20:51:13.403 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 45.536024069, 2024-09-16T20:51:13.404 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99753650500000002, 2024-09-16T20:51:13.404 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:13.404 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:13.404 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:13.404 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:13.404 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6061", 2024-09-16T20:51:13.404 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3930128440", 2024-09-16T20:51:13.404 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5424 2024-09-16T20:51:13.404 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.404 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:13.404 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.404 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:13.404 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766882+0000", 2024-09-16T20:51:13.405 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.405 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.405 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.405 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:13.405 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766882+0000", 2024-09-16T20:51:13.405 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.405 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.405 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.405 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:13.405 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766883+0000", 2024-09-16T20:51:13.405 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3289999999999999e-06 2024-09-16T20:51:13.405 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.405 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.405 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:13.406 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766888+0000", 2024-09-16T20:51:13.406 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5800000000000002e-06 2024-09-16T20:51:13.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.406 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.406 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:13.406 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766900+0000", 2024-09-16T20:51:13.406 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1861e-05 2024-09-16T20:51:13.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.406 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.406 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:13.406 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766909+0000", 2024-09-16T20:51:13.406 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5210000000000008e-06 2024-09-16T20:51:13.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.407 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.407 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:13.407 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766958+0000", 2024-09-16T20:51:13.407 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8718999999999999e-05 2024-09-16T20:51:13.407 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.407 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.407 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:13.407 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766975+0000", 2024-09-16T20:51:13.407 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7231000000000002e-05 2024-09-16T20:51:13.407 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.407 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.407 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:13.407 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:28.764327+0000", 2024-09-16T20:51:13.407 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99735112500000001 2024-09-16T20:51:13.408 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.408 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.408 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:13.408 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:28.764419+0000", 2024-09-16T20:51:13.408 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2139000000000007e-05 2024-09-16T20:51:13.408 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.408 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:13.408 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.408 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.408 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:13.408 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:13.410 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:51:13.410 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:13.408+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:13.510 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:13.509+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:13.559 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:13.559 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:51:13.559 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:51:13.559 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:13.567 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:51:13.614 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:13.610+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:13.614 INFO:teuthology.orchestra.run.smithi029.stderr:noscrub is unset 2024-09-16T20:51:13.712 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:13.712 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:51:13.712 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:51:13.712 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:51:13.712 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:13.713 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:13.711+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:13.723 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:51:13.813 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:13.812+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:13.850 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:13.850 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:51:13.850 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:51:13.851 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:51:13.851 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.851 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12548 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 628736~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:13.851 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757009+0000", 2024-09-16T20:51:13.851 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 83.091929465999996, 2024-09-16T20:51:13.851 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065743956000000006, 2024-09-16T20:51:13.852 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:13.852 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:13.852 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:13.852 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:13.852 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:13.852 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:13.852 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12548 2024-09-16T20:51:13.853 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.853 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:13.853 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.853 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:13.853 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757009+0000", 2024-09-16T20:51:13.853 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.853 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.853 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.853 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:13.853 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757009+0000", 2024-09-16T20:51:13.853 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.853 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.854 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.854 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:13.854 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757010+0000", 2024-09-16T20:51:13.854 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9900000000000001e-07 2024-09-16T20:51:13.854 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.854 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.854 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:13.854 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757012+0000", 2024-09-16T20:51:13.855 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2689999999999998e-06 2024-09-16T20:51:13.855 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.855 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.855 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:13.855 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757022+0000", 2024-09-16T20:51:13.855 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9699999999999994e-06 2024-09-16T20:51:13.855 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.856 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.856 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:13.856 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757028+0000", 2024-09-16T20:51:13.856 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3799999999999999e-06 2024-09-16T20:51:13.856 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.856 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.856 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:13.856 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822552+0000", 2024-09-16T20:51:13.856 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065524281000000004 2024-09-16T20:51:13.857 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.857 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.857 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:13.857 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822613+0000", 2024-09-16T20:51:13.857 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0430999999999998e-05 2024-09-16T20:51:13.857 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.857 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.857 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:13.858 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822753+0000", 2024-09-16T20:51:13.858 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000140226 2024-09-16T20:51:13.858 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.858 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:13.858 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.858 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.858 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.858 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12549 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 633856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:13.858 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:51:13.859 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 83.091900718999995, 2024-09-16T20:51:13.859 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065978518, 2024-09-16T20:51:13.859 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:13.859 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:13.859 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:13.859 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:13.860 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:13.860 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:13.860 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12549 2024-09-16T20:51:13.860 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.860 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:13.860 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.861 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:13.861 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:51:13.861 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.861 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.861 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.861 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:13.861 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:51:13.861 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.861 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.861 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.861 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:13.861 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:51:13.861 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9299999999999999e-07 2024-09-16T20:51:13.862 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.862 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.862 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:13.862 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757040+0000", 2024-09-16T20:51:13.862 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.677e-06 2024-09-16T20:51:13.862 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.862 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.862 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:13.862 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757045+0000", 2024-09-16T20:51:13.862 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8149999999999996e-06 2024-09-16T20:51:13.862 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.862 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.862 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:13.862 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757048+0000", 2024-09-16T20:51:13.862 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1e-06 2024-09-16T20:51:13.862 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.863 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.863 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:13.863 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822796+0000", 2024-09-16T20:51:13.863 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065748081 2024-09-16T20:51:13.863 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.863 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.863 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:13.863 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822867+0000", 2024-09-16T20:51:13.863 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0679000000000005e-05 2024-09-16T20:51:13.863 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.863 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.863 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:13.863 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823016+0000", 2024-09-16T20:51:13.863 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014987299999999999 2024-09-16T20:51:13.864 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.864 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:13.864 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.864 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.864 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.864 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12550 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 638976~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:13.864 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757060+0000", 2024-09-16T20:51:13.864 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 83.091878304000005, 2024-09-16T20:51:13.864 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066153564999999998, 2024-09-16T20:51:13.864 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:13.864 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:13.864 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:13.864 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:13.864 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:13.865 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:13.865 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12550 2024-09-16T20:51:13.865 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.865 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:13.865 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.865 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:13.865 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757060+0000", 2024-09-16T20:51:13.865 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.865 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.865 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.865 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:13.865 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757060+0000", 2024-09-16T20:51:13.865 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.865 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.866 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.866 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:13.866 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757061+0000", 2024-09-16T20:51:13.866 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.15e-07 2024-09-16T20:51:13.866 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.866 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.866 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:13.866 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757062+0000", 2024-09-16T20:51:13.866 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.663e-06 2024-09-16T20:51:13.866 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.866 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.866 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:13.866 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757066+0000", 2024-09-16T20:51:13.866 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.817e-06 2024-09-16T20:51:13.867 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.867 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.867 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:13.867 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757070+0000", 2024-09-16T20:51:13.867 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1509999999999997e-06 2024-09-16T20:51:13.867 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.867 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.868 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:13.868 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823044+0000", 2024-09-16T20:51:13.868 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065973992999999995 2024-09-16T20:51:13.868 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.868 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.868 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:13.868 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823082+0000", 2024-09-16T20:51:13.868 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8050999999999999e-05 2024-09-16T20:51:13.868 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.868 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.869 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:13.869 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823214+0000", 2024-09-16T20:51:13.869 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013157500000000001 2024-09-16T20:51:13.869 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.869 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:13.869 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.869 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.869 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.869 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12551 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 644096~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:13.870 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:51:13.870 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 83.091864606000001, 2024-09-16T20:51:13.870 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066391378000000001, 2024-09-16T20:51:13.870 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:13.870 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:13.870 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:13.870 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:13.871 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:13.871 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:13.871 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12551 2024-09-16T20:51:13.871 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.871 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:13.871 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.872 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:13.872 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:51:13.872 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.872 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.872 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.872 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:13.873 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:51:13.873 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.873 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.873 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.873 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:13.873 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:51:13.873 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2999999999999999e-07 2024-09-16T20:51:13.874 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.874 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.874 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:13.874 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757075+0000", 2024-09-16T20:51:13.874 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.097e-06 2024-09-16T20:51:13.874 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.874 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.874 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:13.874 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757082+0000", 2024-09-16T20:51:13.874 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9500000000000004e-06 2024-09-16T20:51:13.875 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.875 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.875 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:13.875 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757086+0000", 2024-09-16T20:51:13.875 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2229999999999999e-06 2024-09-16T20:51:13.875 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.875 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.875 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:13.875 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823262+0000", 2024-09-16T20:51:13.876 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066176078999999999 2024-09-16T20:51:13.876 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.876 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.876 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:13.876 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823336+0000", 2024-09-16T20:51:13.876 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3979000000000004e-05 2024-09-16T20:51:13.876 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.876 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.876 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:13.876 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823465+0000", 2024-09-16T20:51:13.876 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012982 2024-09-16T20:51:13.876 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.876 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:13.877 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.877 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.877 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.877 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12552 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 649216~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:13.877 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757089+0000", 2024-09-16T20:51:13.877 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 83.091849464999996, 2024-09-16T20:51:13.877 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066655617, 2024-09-16T20:51:13.877 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:13.877 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:13.877 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:13.878 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:13.878 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:13.878 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:13.878 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12552 2024-09-16T20:51:13.878 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.878 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:13.878 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.878 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:13.878 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757089+0000", 2024-09-16T20:51:13.878 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.879 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.879 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.879 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:13.879 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757089+0000", 2024-09-16T20:51:13.879 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.879 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.879 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.880 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:13.880 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759492+0000", 2024-09-16T20:51:13.880 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0024026109999999998 2024-09-16T20:51:13.880 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.880 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.880 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:13.880 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759497+0000", 2024-09-16T20:51:13.880 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1120000000000004e-06 2024-09-16T20:51:13.880 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.880 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.880 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:13.880 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759508+0000", 2024-09-16T20:51:13.881 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0981e-05 2024-09-16T20:51:13.881 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.881 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.881 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:13.881 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759516+0000", 2024-09-16T20:51:13.881 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7270000000000007e-06 2024-09-16T20:51:13.881 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.881 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.881 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:13.881 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823504+0000", 2024-09-16T20:51:13.881 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.063987984999999997 2024-09-16T20:51:13.882 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.882 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.882 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:13.882 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823579+0000", 2024-09-16T20:51:13.882 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5703000000000003e-05 2024-09-16T20:51:13.882 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.882 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.882 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:13.882 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823745+0000", 2024-09-16T20:51:13.882 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016549800000000001 2024-09-16T20:51:13.882 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.882 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:13.882 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.882 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.883 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.883 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12560 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 690176~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:13.883 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.759658+0000", 2024-09-16T20:51:13.883 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 83.089280263000006, 2024-09-16T20:51:13.883 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065858517000000005, 2024-09-16T20:51:13.883 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:13.883 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:13.883 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:13.883 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:13.883 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:13.883 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:13.883 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12560 2024-09-16T20:51:13.883 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.883 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:13.884 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.884 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:13.884 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759658+0000", 2024-09-16T20:51:13.884 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.884 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.884 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.884 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:13.884 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759658+0000", 2024-09-16T20:51:13.884 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.884 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.884 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.884 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:13.884 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759659+0000", 2024-09-16T20:51:13.884 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.17e-07 2024-09-16T20:51:13.885 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.885 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.885 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:13.885 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759660+0000", 2024-09-16T20:51:13.885 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0189999999999999e-06 2024-09-16T20:51:13.885 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.885 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.885 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:13.885 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759663+0000", 2024-09-16T20:51:13.885 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0010000000000002e-06 2024-09-16T20:51:13.885 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.885 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.885 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:13.885 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759665+0000", 2024-09-16T20:51:13.886 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5440000000000001e-06 2024-09-16T20:51:13.886 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.886 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.886 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:13.886 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825355+0000", 2024-09-16T20:51:13.886 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065690224000000005 2024-09-16T20:51:13.886 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.886 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.886 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:13.886 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825394+0000", 2024-09-16T20:51:13.886 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8810999999999998e-05 2024-09-16T20:51:13.886 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.886 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.886 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:13.887 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825517+0000", 2024-09-16T20:51:13.887 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012270099999999999 2024-09-16T20:51:13.887 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.887 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:13.887 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.887 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.887 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.887 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12561 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 695296~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:13.887 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.759668+0000", 2024-09-16T20:51:13.887 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 83.089270420999995, 2024-09-16T20:51:13.887 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066116999999999995, 2024-09-16T20:51:13.887 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:13.887 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:13.887 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:13.888 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:13.888 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:13.888 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:13.888 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12561 2024-09-16T20:51:13.888 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.888 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:13.888 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.888 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:13.888 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759668+0000", 2024-09-16T20:51:13.888 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.888 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.888 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.889 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:13.889 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759668+0000", 2024-09-16T20:51:13.889 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.889 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.889 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.889 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:13.889 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765379+0000", 2024-09-16T20:51:13.889 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057104410000000001 2024-09-16T20:51:13.889 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.889 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.889 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:13.890 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765383+0000", 2024-09-16T20:51:13.890 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2980000000000002e-06 2024-09-16T20:51:13.890 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.890 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.890 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:13.890 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765395+0000", 2024-09-16T20:51:13.891 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1671000000000001e-05 2024-09-16T20:51:13.891 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.891 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.891 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:13.891 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765403+0000", 2024-09-16T20:51:13.891 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1210000000000007e-06 2024-09-16T20:51:13.892 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.892 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.892 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:13.892 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825552+0000", 2024-09-16T20:51:13.892 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.060149498000000003 2024-09-16T20:51:13.892 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.893 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.893 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:13.893 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825625+0000", 2024-09-16T20:51:13.893 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2540999999999995e-05 2024-09-16T20:51:13.893 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.893 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.893 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:13.894 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825785+0000", 2024-09-16T20:51:13.894 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016043000000000001 2024-09-16T20:51:13.894 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.894 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:13.894 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.894 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.894 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.894 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12585 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 818176~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:13.894 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.765848+0000", 2024-09-16T20:51:13.895 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 83.08309045, 2024-09-16T20:51:13.895 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065843218999999995, 2024-09-16T20:51:13.895 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:13.895 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:13.895 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:13.895 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:13.895 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:13.895 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:13.895 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12585 2024-09-16T20:51:13.895 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.895 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:13.895 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.896 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:13.896 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765848+0000", 2024-09-16T20:51:13.896 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.896 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.896 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.896 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:13.896 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765848+0000", 2024-09-16T20:51:13.896 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.896 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.896 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.896 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:13.896 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768683+0000", 2024-09-16T20:51:13.896 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0028344519999999999 2024-09-16T20:51:13.897 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.897 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.897 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:13.897 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768686+0000", 2024-09-16T20:51:13.897 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8789999999999996e-06 2024-09-16T20:51:13.897 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.897 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.897 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:13.897 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768696+0000", 2024-09-16T20:51:13.897 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6709999999999994e-06 2024-09-16T20:51:13.897 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.897 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.898 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:13.898 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768702+0000", 2024-09-16T20:51:13.898 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2550000000000003e-06 2024-09-16T20:51:13.898 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.898 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.898 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:13.898 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.831499+0000", 2024-09-16T20:51:13.898 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.062797081000000005 2024-09-16T20:51:13.898 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.898 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.898 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:13.898 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.831538+0000", 2024-09-16T20:51:13.898 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8523999999999999e-05 2024-09-16T20:51:13.899 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.899 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.899 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:13.899 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.831691+0000", 2024-09-16T20:51:13.899 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015335699999999999 2024-09-16T20:51:13.899 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.899 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:13.899 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.899 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.899 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.899 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12597 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 879616~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:13.899 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.768883+0000", 2024-09-16T20:51:13.899 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 83.080055721999997, 2024-09-16T20:51:13.899 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065816527, 2024-09-16T20:51:13.900 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:13.900 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:13.900 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:13.900 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:13.900 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:13.900 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:13.900 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12597 2024-09-16T20:51:13.900 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.900 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:13.901 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.901 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:13.901 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768883+0000", 2024-09-16T20:51:13.901 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.901 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.901 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.901 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:13.901 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768883+0000", 2024-09-16T20:51:13.901 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.901 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.901 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.901 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:13.901 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771508+0000", 2024-09-16T20:51:13.902 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0026247589999999999 2024-09-16T20:51:13.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.902 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.902 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:13.902 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771513+0000", 2024-09-16T20:51:13.902 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9949999999999996e-06 2024-09-16T20:51:13.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.902 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.902 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:13.902 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771525+0000", 2024-09-16T20:51:13.902 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2033e-05 2024-09-16T20:51:13.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.902 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.902 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:13.902 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771532+0000", 2024-09-16T20:51:13.902 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4220000000000003e-06 2024-09-16T20:51:13.903 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.903 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.903 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:13.903 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.834532+0000", 2024-09-16T20:51:13.903 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.063000159 2024-09-16T20:51:13.903 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.903 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.903 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:13.903 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.834590+0000", 2024-09-16T20:51:13.903 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8218999999999999e-05 2024-09-16T20:51:13.903 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.903 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.903 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:13.903 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.834699+0000", 2024-09-16T20:51:13.903 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010894 2024-09-16T20:51:13.903 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.904 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:13.904 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.904 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.904 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.904 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12619 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 992256~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:13.904 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.774656+0000", 2024-09-16T20:51:13.904 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 83.074282948000004, 2024-09-16T20:51:13.904 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065715859000000001, 2024-09-16T20:51:13.904 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:13.904 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:13.904 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:13.905 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:13.905 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:13.905 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:13.905 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12619 2024-09-16T20:51:13.905 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.905 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:13.905 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.905 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:13.905 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774656+0000", 2024-09-16T20:51:13.905 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.905 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.905 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.905 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:13.906 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774656+0000", 2024-09-16T20:51:13.906 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.906 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.906 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:13.906 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774875+0000", 2024-09-16T20:51:13.906 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000218898 2024-09-16T20:51:13.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.906 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.906 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:13.906 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774879+0000", 2024-09-16T20:51:13.906 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0389999999999998e-06 2024-09-16T20:51:13.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.906 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.906 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:13.906 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774890+0000", 2024-09-16T20:51:13.907 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1250000000000001e-05 2024-09-16T20:51:13.907 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.907 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.907 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:13.907 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774897+0000", 2024-09-16T20:51:13.907 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7619999999999998e-06 2024-09-16T20:51:13.907 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.907 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.907 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:13.907 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840139+0000", 2024-09-16T20:51:13.907 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065242438 2024-09-16T20:51:13.907 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.907 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.907 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:13.908 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840186+0000", 2024-09-16T20:51:13.908 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6894999999999997e-05 2024-09-16T20:51:13.908 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.908 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.908 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:13.908 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840372+0000", 2024-09-16T20:51:13.908 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000185577 2024-09-16T20:51:13.908 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.908 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:13.908 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.908 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.908 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.908 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12620 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 997376~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:13.908 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.774908+0000", 2024-09-16T20:51:13.908 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 83.074030726999993, 2024-09-16T20:51:13.909 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065782754999999998, 2024-09-16T20:51:13.909 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:13.909 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:13.909 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:13.909 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:13.909 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:13.909 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:13.909 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12620 2024-09-16T20:51:13.909 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.909 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:13.909 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.909 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:13.909 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774908+0000", 2024-09-16T20:51:13.910 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.910 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.910 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.910 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:13.910 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774908+0000", 2024-09-16T20:51:13.910 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.910 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.910 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.910 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:13.910 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775102+0000", 2024-09-16T20:51:13.910 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000194098 2024-09-16T20:51:13.910 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.911 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.911 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:13.911 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775105+0000", 2024-09-16T20:51:13.911 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8739999999999999e-06 2024-09-16T20:51:13.911 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.911 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.911 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:13.911 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775111+0000", 2024-09-16T20:51:13.911 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9519999999999999e-06 2024-09-16T20:51:13.911 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.912 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.912 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:13.912 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775115+0000", 2024-09-16T20:51:13.912 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.002e-06 2024-09-16T20:51:13.912 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.912 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.912 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:13.912 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840409+0000", 2024-09-16T20:51:13.912 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065294195999999999 2024-09-16T20:51:13.912 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.912 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.912 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:13.913 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840507+0000", 2024-09-16T20:51:13.913 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7662000000000001e-05 2024-09-16T20:51:13.913 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.913 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.913 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:13.913 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840691+0000", 2024-09-16T20:51:13.913 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000183971 2024-09-16T20:51:13.913 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.913 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:13.913 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.913 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.913 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.913 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12621 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1002496~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:13.913 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775123+0000", 2024-09-16T20:51:13.914 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 83.073816031000007, 2024-09-16T20:51:13.914 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065829586999999995, 2024-09-16T20:51:13.914 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:13.914 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:13.914 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:13.914 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:13.914 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:13.914 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:13.914 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12621 2024-09-16T20:51:13.914 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.914 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:13.914 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.914 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:13.914 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775123+0000", 2024-09-16T20:51:13.915 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.915 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.915 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.915 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:13.915 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775123+0000", 2024-09-16T20:51:13.915 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.915 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.915 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.915 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:13.915 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775348+0000", 2024-09-16T20:51:13.915 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000225295 2024-09-16T20:51:13.915 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.916 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.916 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:13.916 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775352+0000", 2024-09-16T20:51:13.916 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0810000000000004e-06 2024-09-16T20:51:13.916 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.916 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.916 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:13.916 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775361+0000", 2024-09-16T20:51:13.916 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7399999999999993e-06 2024-09-16T20:51:13.916 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.916 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.916 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:13.916 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775367+0000", 2024-09-16T20:51:13.916 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5200000000000003e-06 2024-09-16T20:51:13.917 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.917 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.917 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:13.917 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840730+0000", 2024-09-16T20:51:13.917 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065362803999999997 2024-09-16T20:51:13.917 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.917 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.917 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:13.917 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840801+0000", 2024-09-16T20:51:13.917 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1006000000000004e-05 2024-09-16T20:51:13.918 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.918 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.918 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:13.918 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840952+0000", 2024-09-16T20:51:13.918 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015114100000000001 2024-09-16T20:51:13.918 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.918 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:13.918 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.918 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.918 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.918 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12622 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1007616~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:13.919 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775381+0000", 2024-09-16T20:51:13.919 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 83.073557197, 2024-09-16T20:51:13.919 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065822581000000005, 2024-09-16T20:51:13.919 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:13.919 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:13.919 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:13.919 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:13.919 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:13.919 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:13.919 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12622 2024-09-16T20:51:13.919 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.919 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:13.919 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.919 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:13.919 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775381+0000", 2024-09-16T20:51:13.920 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.920 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.920 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.920 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:13.920 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775381+0000", 2024-09-16T20:51:13.920 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.920 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.920 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.920 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:13.920 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775563+0000", 2024-09-16T20:51:13.920 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018153700000000001 2024-09-16T20:51:13.920 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.920 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.920 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:13.921 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775566+0000", 2024-09-16T20:51:13.921 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9620000000000002e-06 2024-09-16T20:51:13.921 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.921 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.921 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:13.921 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775572+0000", 2024-09-16T20:51:13.922 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8529999999999997e-06 2024-09-16T20:51:13.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.922 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.922 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:13.922 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775576+0000", 2024-09-16T20:51:13.922 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8920000000000002e-06 2024-09-16T20:51:13.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.922 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.922 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:13.922 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840990+0000", 2024-09-16T20:51:13.922 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065414599000000004 2024-09-16T20:51:13.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.922 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.922 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:13.923 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841068+0000", 2024-09-16T20:51:13.923 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8157000000000004e-05 2024-09-16T20:51:13.923 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.923 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.923 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:13.923 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841204+0000", 2024-09-16T20:51:13.923 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013558099999999999 2024-09-16T20:51:13.923 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.923 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:13.923 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.923 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.923 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.923 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12623 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1012736~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:13.923 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775583+0000", 2024-09-16T20:51:13.923 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 83.073355758999995, 2024-09-16T20:51:13.924 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065929087999999997, 2024-09-16T20:51:13.924 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:13.924 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:13.924 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:13.924 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:13.924 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:13.924 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:13.924 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12623 2024-09-16T20:51:13.924 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.924 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:13.924 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.924 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:13.924 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775583+0000", 2024-09-16T20:51:13.924 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.925 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.925 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.925 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:13.928 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775583+0000", 2024-09-16T20:51:13.928 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.928 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.928 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.928 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:13.928 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775804+0000", 2024-09-16T20:51:13.928 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00022116900000000001 2024-09-16T20:51:13.928 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.928 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.928 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:13.929 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775807+0000", 2024-09-16T20:51:13.929 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9529999999999999e-06 2024-09-16T20:51:13.929 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.929 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.929 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:13.929 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775814+0000", 2024-09-16T20:51:13.929 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7490000000000001e-06 2024-09-16T20:51:13.929 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.929 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.929 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:13.929 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775818+0000", 2024-09-16T20:51:13.929 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1999999999999996e-06 2024-09-16T20:51:13.929 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.930 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.930 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:13.930 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841267+0000", 2024-09-16T20:51:13.930 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065449127999999995 2024-09-16T20:51:13.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.930 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.930 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:13.930 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841344+0000", 2024-09-16T20:51:13.930 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6546999999999997e-05 2024-09-16T20:51:13.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.930 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.930 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:13.930 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841512+0000", 2024-09-16T20:51:13.930 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016834200000000001 2024-09-16T20:51:13.931 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.931 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:13.931 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.931 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.931 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.931 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12624 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1017856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:13.931 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:51:13.931 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 83.073112619, 2024-09-16T20:51:13.931 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065945866000000006, 2024-09-16T20:51:13.931 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:13.931 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:13.931 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:13.931 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:13.931 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:13.932 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:13.932 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12624 2024-09-16T20:51:13.932 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.932 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:13.932 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.932 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:13.932 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:51:13.932 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.932 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.933 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.933 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:13.933 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:51:13.933 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.933 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.933 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.933 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:13.933 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:51:13.933 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-16T20:51:13.933 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.933 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.934 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:13.934 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775828+0000", 2024-09-16T20:51:13.934 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3999999999999999e-06 2024-09-16T20:51:13.934 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.934 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.934 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:13.934 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775831+0000", 2024-09-16T20:51:13.934 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.478e-06 2024-09-16T20:51:13.934 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.934 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.934 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:13.934 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775834+0000", 2024-09-16T20:51:13.935 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9440000000000001e-06 2024-09-16T20:51:13.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.935 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.935 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:13.935 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841547+0000", 2024-09-16T20:51:13.935 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065712801000000001 2024-09-16T20:51:13.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.935 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.935 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:13.935 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841618+0000", 2024-09-16T20:51:13.935 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1151999999999999e-05 2024-09-16T20:51:13.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.935 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.935 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:13.935 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841772+0000", 2024-09-16T20:51:13.936 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000153721 2024-09-16T20:51:13.936 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.936 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:13.936 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.936 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.936 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.936 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12625 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1022976~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:13.936 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775840+0000", 2024-09-16T20:51:13.936 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 83.073099010000007, 2024-09-16T20:51:13.936 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066165451, 2024-09-16T20:51:13.936 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:13.937 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:13.937 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:13.937 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:13.937 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:13.937 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:13.937 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12625 2024-09-16T20:51:13.937 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.937 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:13.937 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.937 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:13.937 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775840+0000", 2024-09-16T20:51:13.937 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.938 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.938 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.938 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:13.938 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775840+0000", 2024-09-16T20:51:13.938 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.938 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.938 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.938 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:13.938 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776014+0000", 2024-09-16T20:51:13.938 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017441099999999999 2024-09-16T20:51:13.938 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.938 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.938 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:13.938 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776018+0000", 2024-09-16T20:51:13.938 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6509999999999999e-06 2024-09-16T20:51:13.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.939 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:13.939 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776026+0000", 2024-09-16T20:51:13.939 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.028e-06 2024-09-16T20:51:13.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.939 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:13.939 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776031+0000", 2024-09-16T20:51:13.939 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8400000000000002e-06 2024-09-16T20:51:13.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.939 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:13.939 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841801+0000", 2024-09-16T20:51:13.939 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065770472999999996 2024-09-16T20:51:13.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:13.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841847+0000", 2024-09-16T20:51:13.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5558e-05 2024-09-16T20:51:13.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:13.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842005+0000", 2024-09-16T20:51:13.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015849000000000001 2024-09-16T20:51:13.940 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.941 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:13.941 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.941 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.941 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.941 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12626 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1028096~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:13.941 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776041+0000", 2024-09-16T20:51:13.941 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 83.072897710999996, 2024-09-16T20:51:13.941 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066225260999999994, 2024-09-16T20:51:13.941 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:13.941 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:13.941 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:13.941 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:13.941 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:13.941 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:13.942 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12626 2024-09-16T20:51:13.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.942 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:13.942 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.942 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:13.942 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776041+0000", 2024-09-16T20:51:13.942 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.942 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.942 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:13.942 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776041+0000", 2024-09-16T20:51:13.942 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.943 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:13.943 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776263+0000", 2024-09-16T20:51:13.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00022193899999999999 2024-09-16T20:51:13.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.943 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:13.943 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776268+0000", 2024-09-16T20:51:13.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8049999999999997e-06 2024-09-16T20:51:13.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.943 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:13.943 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776280+0000", 2024-09-16T20:51:13.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2635e-05 2024-09-16T20:51:13.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.944 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:13.944 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776288+0000", 2024-09-16T20:51:13.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8199999999999997e-06 2024-09-16T20:51:13.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.944 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:13.944 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842041+0000", 2024-09-16T20:51:13.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065752506000000002 2024-09-16T20:51:13.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.945 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:13.945 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842123+0000", 2024-09-16T20:51:13.945 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2393999999999998e-05 2024-09-16T20:51:13.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.945 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:13.945 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842266+0000", 2024-09-16T20:51:13.945 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014316200000000001 2024-09-16T20:51:13.945 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.945 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:13.945 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.945 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12627 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1033216~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:13.946 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776301+0000", 2024-09-16T20:51:13.946 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 83.072637751000002, 2024-09-16T20:51:13.946 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066187814999999997, 2024-09-16T20:51:13.946 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:13.946 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:13.946 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:13.946 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:13.946 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:13.946 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:13.946 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12627 2024-09-16T20:51:13.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.946 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:13.946 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.946 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:13.946 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776301+0000", 2024-09-16T20:51:13.947 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.947 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.947 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:13.947 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776301+0000", 2024-09-16T20:51:13.947 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.948 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:13.948 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776473+0000", 2024-09-16T20:51:13.948 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017222400000000001 2024-09-16T20:51:13.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.949 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:13.949 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776477+0000", 2024-09-16T20:51:13.949 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7019999999999999e-06 2024-09-16T20:51:13.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.949 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.949 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:13.950 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776486+0000", 2024-09-16T20:51:13.950 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2119999999999992e-06 2024-09-16T20:51:13.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.950 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.950 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:13.950 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776492+0000", 2024-09-16T20:51:13.951 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8089999999999998e-06 2024-09-16T20:51:13.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.951 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.951 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:13.951 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842301+0000", 2024-09-16T20:51:13.951 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065809042999999998 2024-09-16T20:51:13.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.951 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.951 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:13.952 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842362+0000", 2024-09-16T20:51:13.952 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1260000000000006e-05 2024-09-16T20:51:13.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.952 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.952 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:13.952 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842489+0000", 2024-09-16T20:51:13.952 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012656500000000001 2024-09-16T20:51:13.952 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.952 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:13.953 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.953 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.953 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12628 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1038336~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:13.953 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776502+0000", 2024-09-16T20:51:13.953 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 83.072437105000006, 2024-09-16T20:51:13.953 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066283426000000006, 2024-09-16T20:51:13.953 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:13.954 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:13.954 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:13.954 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:13.954 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:13.954 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:13.954 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12628 2024-09-16T20:51:13.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.954 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:13.954 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.954 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:13.954 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776502+0000", 2024-09-16T20:51:13.954 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.955 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:13.955 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776502+0000", 2024-09-16T20:51:13.955 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.955 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:13.955 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776716+0000", 2024-09-16T20:51:13.955 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000214596 2024-09-16T20:51:13.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.955 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:13.955 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776723+0000", 2024-09-16T20:51:13.956 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4000000000000003e-06 2024-09-16T20:51:13.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.956 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:13.956 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776731+0000", 2024-09-16T20:51:13.956 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6129999999999998e-06 2024-09-16T20:51:13.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.956 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:13.956 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776737+0000", 2024-09-16T20:51:13.956 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1970000000000001e-06 2024-09-16T20:51:13.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.956 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:13.957 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842532+0000", 2024-09-16T20:51:13.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065795127999999994 2024-09-16T20:51:13.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.957 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:13.957 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842599+0000", 2024-09-16T20:51:13.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6711000000000006e-05 2024-09-16T20:51:13.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.957 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:13.957 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842785+0000", 2024-09-16T20:51:13.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018578099999999999 2024-09-16T20:51:13.957 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.958 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:13.958 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.958 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.958 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.958 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12629 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1043456~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:13.958 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776748+0000", 2024-09-16T20:51:13.958 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 83.072190727999995, 2024-09-16T20:51:13.958 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066303788000000002, 2024-09-16T20:51:13.958 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:13.958 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:13.958 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:13.958 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:13.958 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:13.958 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:13.958 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12629 2024-09-16T20:51:13.959 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.959 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:13.959 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.959 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:13.959 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776748+0000", 2024-09-16T20:51:13.959 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.959 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.959 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.959 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:13.959 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776748+0000", 2024-09-16T20:51:13.959 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:13.959 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.959 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.959 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:13.960 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776934+0000", 2024-09-16T20:51:13.960 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018598399999999999 2024-09-16T20:51:13.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.960 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:13.960 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776939+0000", 2024-09-16T20:51:13.960 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1000000000000003e-06 2024-09-16T20:51:13.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.960 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:13.960 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776954+0000", 2024-09-16T20:51:13.960 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5221e-05 2024-09-16T20:51:13.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.961 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:13.961 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776962+0000", 2024-09-16T20:51:13.961 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.554e-06 2024-09-16T20:51:13.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.961 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.961 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:13.961 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842813+0000", 2024-09-16T20:51:13.961 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065851006000000004 2024-09-16T20:51:13.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.961 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.961 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:13.961 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842886+0000", 2024-09-16T20:51:13.961 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3663000000000003e-05 2024-09-16T20:51:13.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:13.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:13.962 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:13.962 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.843052+0000", 2024-09-16T20:51:13.962 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016526 2024-09-16T20:51:13.962 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.962 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:13.962 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.962 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:13.962 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:13.962 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:13.964 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:51:13.964 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:13.962+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:14.064 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:14.062+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:14.123 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:14.123 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:51:14.123 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:51:14.123 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:14.134 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:51:14.174 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:14.173+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:14.266 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:51:14.264+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:51:14.289 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:14.289 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:51:14.289 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:51:14.289 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:51:14.289 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:14.299 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:51:14.383 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:14.382+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:14.459 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:14.459 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:51:14.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:51:14.460 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:51:14.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.460 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5286.0:10643 3.17 3.f6273fb7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-16T20:51:14.460 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:51:14.460 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 125.667197708, 2024-09-16T20:51:14.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.76672347200000002, 2024-09-16T20:51:14.460 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:14.460 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:14.460 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:14.460 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:14.460 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5286", 2024-09-16T20:51:14.461 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/4135433989", 2024-09-16T20:51:14.461 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 10643 2024-09-16T20:51:14.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.461 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:14.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:14.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:51:14.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:14.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:14.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:51:14.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:14.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.462 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:14.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790489+0000", 2024-09-16T20:51:14.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5910000000000001e-06 2024-09-16T20:51:14.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.462 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:14.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790493+0000", 2024-09-16T20:51:14.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2669999999999998e-06 2024-09-16T20:51:14.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.462 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:14.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790500+0000", 2024-09-16T20:51:14.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3780000000000003e-06 2024-09-16T20:51:14.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.463 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:14.463 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790505+0000", 2024-09-16T20:51:14.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1560000000000003e-06 2024-09-16T20:51:14.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.463 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:14.463 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790559+0000", 2024-09-16T20:51:14.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3494e-05 2024-09-16T20:51:14.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.463 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:14.464 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790566+0000", 2024-09-16T20:51:14.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7649999999999993e-06 2024-09-16T20:51:14.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.464 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:14.464 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:09.557198+0000", 2024-09-16T20:51:14.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.76663136200000004 2024-09-16T20:51:14.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.464 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:14.464 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:09.557211+0000", 2024-09-16T20:51:14.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3458999999999999e-05 2024-09-16T20:51:14.464 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:14.465 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:14.465 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:14.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.465 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5512.0:450 3.2 3:44e0cffa:::rbd_header.110ef0d7a535:head [watch reconnect cookie 140263298018912 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:51:14.465 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:51:14.465 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 97.895174795000003, 2024-09-16T20:51:14.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0084581939999999, 2024-09-16T20:51:14.465 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:14.465 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:14.465 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:14.465 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:14.465 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5512", 2024-09-16T20:51:14.465 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:51:14.466 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 450 2024-09-16T20:51:14.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.466 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:14.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:14.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:51:14.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:14.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:14.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:51:14.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:14.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.467 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:14.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562512+0000", 2024-09-16T20:51:14.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.009e-06 2024-09-16T20:51:14.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.467 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:14.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562517+0000", 2024-09-16T20:51:14.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9339999999999997e-06 2024-09-16T20:51:14.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.467 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:14.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562530+0000", 2024-09-16T20:51:14.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3495e-05 2024-09-16T20:51:14.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:14.468 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562536+0000", 2024-09-16T20:51:14.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5690000000000004e-06 2024-09-16T20:51:14.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:14.468 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562591+0000", 2024-09-16T20:51:14.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5761999999999999e-05 2024-09-16T20:51:14.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:51:14.469 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562599+0000", 2024-09-16T20:51:14.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9179999999999994e-06 2024-09-16T20:51:14.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.469 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:14.469 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.565971+0000", 2024-09-16T20:51:14.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003371546 2024-09-16T20:51:14.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.469 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:14.469 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566019+0000", 2024-09-16T20:51:14.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7849000000000002e-05 2024-09-16T20:51:14.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:14.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.569382+0000", 2024-09-16T20:51:14.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.003362833 2024-09-16T20:51:14.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:14.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.569529+0000", 2024-09-16T20:51:14.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000147823 2024-09-16T20:51:14.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:14.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570969+0000", 2024-09-16T20:51:14.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001439456 2024-09-16T20:51:14.471 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:14.471 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:14.471 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:14.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.471 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:14996 3.2 3:44e0cffa:::rbd_header.110ef0d7a535:head [watch reconnect cookie 140213436269840 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:51:14.471 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:51:14.471 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 97.895126383000004, 2024-09-16T20:51:14.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.008813943, 2024-09-16T20:51:14.471 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:14.471 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:14.471 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:14.471 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:14.471 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:14.472 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:14.472 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 14996 2024-09-16T20:51:14.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.472 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:14.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:14.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:51:14.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:14.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:14.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:51:14.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:14.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:14.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562560+0000", 2024-09-16T20:51:14.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.06e-07 2024-09-16T20:51:14.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:14.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562563+0000", 2024-09-16T20:51:14.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.495e-06 2024-09-16T20:51:14.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:14.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562571+0000", 2024-09-16T20:51:14.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1950000000000003e-06 2024-09-16T20:51:14.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:14.474 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562579+0000", 2024-09-16T20:51:14.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2649999999999999e-06 2024-09-16T20:51:14.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:14.474 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562673+0000", 2024-09-16T20:51:14.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4118999999999998e-05 2024-09-16T20:51:14.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:51:14.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562677+0000", 2024-09-16T20:51:14.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5179999999999998e-06 2024-09-16T20:51:14.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:14.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566461+0000", 2024-09-16T20:51:14.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0037837259999999998 2024-09-16T20:51:14.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:14.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566468+0000", 2024-09-16T20:51:14.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3350000000000001e-06 2024-09-16T20:51:14.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.476 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:14.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570306+0000", 2024-09-16T20:51:14.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.003837997 2024-09-16T20:51:14.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.476 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:14.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570392+0000", 2024-09-16T20:51:14.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5872999999999996e-05 2024-09-16T20:51:14.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.476 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:14.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571373+0000", 2024-09-16T20:51:14.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00098071399999999993 2024-09-16T20:51:14.477 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:14.477 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:14.477 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:14.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.477 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5255.0:1931 3.2 3:47882819:::rbd_data.110eb5f95afc.0000000000000009:head [read 217088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:51:14.477 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:51:14.477 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 97.895087633000003, 2024-09-16T20:51:14.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0082752880000001, 2024-09-16T20:51:14.477 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:14.477 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:14.477 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:14.478 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:14.478 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5255", 2024-09-16T20:51:14.478 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:51:14.478 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1931 2024-09-16T20:51:14.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.478 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:14.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:14.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:51:14.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:14.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:14.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:51:14.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:14.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:14.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:51:14.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9699999999999996e-07 2024-09-16T20:51:14.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:14.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562601+0000", 2024-09-16T20:51:14.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8600000000000001e-06 2024-09-16T20:51:14.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:14.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562608+0000", 2024-09-16T20:51:14.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9e-06 2024-09-16T20:51:14.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:14.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562615+0000", 2024-09-16T20:51:14.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8179999999999998e-06 2024-09-16T20:51:14.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:14.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562703+0000", 2024-09-16T20:51:14.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7811000000000003e-05 2024-09-16T20:51:14.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.481 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:51:14.481 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562706+0000", 2024-09-16T20:51:14.481 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4920000000000002e-06 2024-09-16T20:51:14.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.481 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:14.481 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566685+0000", 2024-09-16T20:51:14.481 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0039792780000000002 2024-09-16T20:51:14.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.481 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:14.481 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566693+0000", 2024-09-16T20:51:14.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4730000000000003e-06 2024-09-16T20:51:14.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.482 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:14.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570737+0000", 2024-09-16T20:51:14.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0040436749999999 2024-09-16T20:51:14.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.482 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:14.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570873+0000", 2024-09-16T20:51:14.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000136484 2024-09-16T20:51:14.482 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:14.482 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:14.482 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:14.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.483 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5255.0:1939 3.2 3:47882819:::rbd_data.110eb5f95afc.0000000000000009:head [read 233472~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:51:14.483 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:51:14.483 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 97.894932767, 2024-09-16T20:51:14.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0084657589999999, 2024-09-16T20:51:14.483 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:14.483 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:14.483 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:14.483 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:14.483 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5255", 2024-09-16T20:51:14.483 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:51:14.483 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1939 2024-09-16T20:51:14.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.484 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:14.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:14.484 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:51:14.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:14.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:14.484 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:51:14.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:14.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:14.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:51:14.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5199999999999997e-07 2024-09-16T20:51:14.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.485 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:14.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562757+0000", 2024-09-16T20:51:14.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4979999999999998e-06 2024-09-16T20:51:14.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.485 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:14.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562764+0000", 2024-09-16T20:51:14.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1470000000000004e-06 2024-09-16T20:51:14.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.486 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:14.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562768+0000", 2024-09-16T20:51:14.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7929999999999999e-06 2024-09-16T20:51:14.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.486 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:14.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562805+0000", 2024-09-16T20:51:14.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7477e-05 2024-09-16T20:51:14.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.486 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting_for_map not empty", 2024-09-16T20:51:14.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562810+0000", 2024-09-16T20:51:14.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0819999999999998e-06 2024-09-16T20:51:14.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:14.487 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566717+0000", 2024-09-16T20:51:14.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0039072600000000001 2024-09-16T20:51:14.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:14.487 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566724+0000", 2024-09-16T20:51:14.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4880000000000004e-06 2024-09-16T20:51:14.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:14.487 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571009+0000", 2024-09-16T20:51:14.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.004284677 2024-09-16T20:51:14.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.488 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:14.488 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571218+0000", 2024-09-16T20:51:14.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00020978500000000001 2024-09-16T20:51:14.488 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:14.488 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:14.488 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:14.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.488 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:11021 3.1f 3:fee2d1dd:::rbd_data.110ea93691bb.000000000000000e:23 [read 960512~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:14.488 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.693837+0000", 2024-09-16T20:51:14.488 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 83.763848222000007, 2024-09-16T20:51:14.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057155138000000001, 2024-09-16T20:51:14.489 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:14.489 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:14.489 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:14.489 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:14.489 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:14.489 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:14.489 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 11021 2024-09-16T20:51:14.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.489 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:14.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:14.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693837+0000", 2024-09-16T20:51:14.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:14.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:14.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693837+0000", 2024-09-16T20:51:14.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:14.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:14.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699566+0000", 2024-09-16T20:51:14.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057283639999999997 2024-09-16T20:51:14.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:14.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699571+0000", 2024-09-16T20:51:14.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.604e-06 2024-09-16T20:51:14.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:14.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699583+0000", 2024-09-16T20:51:14.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1372999999999999e-05 2024-09-16T20:51:14.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:14.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699591+0000", 2024-09-16T20:51:14.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5259999999999993e-06 2024-09-16T20:51:14.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:14.492 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750674+0000", 2024-09-16T20:51:14.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051083154999999998 2024-09-16T20:51:14.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.492 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:14.492 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750745+0000", 2024-09-16T20:51:14.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0933999999999995e-05 2024-09-16T20:51:14.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.492 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:14.492 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750992+0000", 2024-09-16T20:51:14.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00024718199999999998 2024-09-16T20:51:14.492 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:14.492 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:14.493 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:14.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.493 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8792 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2609152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:14.493 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:51:14.493 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 73.400388504999995, 2024-09-16T20:51:14.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057003524999999999, 2024-09-16T20:51:14.493 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:14.493 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:14.493 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:14.493 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:14.493 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:14.493 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:14.494 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8792 2024-09-16T20:51:14.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.494 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:14.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.494 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:14.494 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:51:14.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:14.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.494 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:14.494 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:51:14.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:14.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:14.495 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:51:14.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0400000000000002e-07 2024-09-16T20:51:14.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:14.495 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057299+0000", 2024-09-16T20:51:14.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7460000000000001e-06 2024-09-16T20:51:14.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:14.495 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057304+0000", 2024-09-16T20:51:14.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3210000000000004e-06 2024-09-16T20:51:14.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:14.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057310+0000", 2024-09-16T20:51:14.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4360000000000001e-06 2024-09-16T20:51:14.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:14.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114200+0000", 2024-09-16T20:51:14.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056889954 2024-09-16T20:51:14.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:14.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114243+0000", 2024-09-16T20:51:14.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2998999999999999e-05 2024-09-16T20:51:14.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:14.497 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114300+0000", 2024-09-16T20:51:14.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7765e-05 2024-09-16T20:51:14.497 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:14.497 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:14.497 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:14.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.497 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8794 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2351104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:14.497 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057327+0000", 2024-09-16T20:51:14.497 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 73.400358181000001, 2024-09-16T20:51:14.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057101130999999999, 2024-09-16T20:51:14.497 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:14.498 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:14.498 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:14.498 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:14.498 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:14.498 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:14.498 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8794 2024-09-16T20:51:14.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.498 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:14.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.498 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:14.498 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057327+0000", 2024-09-16T20:51:14.498 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:14.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.499 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:14.499 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057327+0000", 2024-09-16T20:51:14.499 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:14.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.499 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.499 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:14.499 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057328+0000", 2024-09-16T20:51:14.499 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7399999999999999e-07 2024-09-16T20:51:14.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.499 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.499 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:14.499 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057330+0000", 2024-09-16T20:51:14.499 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2199999999999999e-06 2024-09-16T20:51:14.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.499 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.500 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:14.500 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057336+0000", 2024-09-16T20:51:14.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1240000000000002e-06 2024-09-16T20:51:14.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.500 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:14.500 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057341+0000", 2024-09-16T20:51:14.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8470000000000003e-06 2024-09-16T20:51:14.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.500 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:14.500 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114334+0000", 2024-09-16T20:51:14.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056993267 2024-09-16T20:51:14.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:14.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114376+0000", 2024-09-16T20:51:14.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1638999999999999e-05 2024-09-16T20:51:14.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:14.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114428+0000", 2024-09-16T20:51:14.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2660000000000001e-05 2024-09-16T20:51:14.501 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:14.501 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:14.501 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:14.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.502 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8798 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2297856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:14.502 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:51:14.502 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 73.400338454000007, 2024-09-16T20:51:14.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057277041000000001, 2024-09-16T20:51:14.502 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:14.502 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:14.502 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:14.502 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:14.502 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:14.502 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:14.502 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8798 2024-09-16T20:51:14.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.503 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:14.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:14.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:51:14.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:14.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:14.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:51:14.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:14.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:14.504 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:51:14.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2800000000000003e-07 2024-09-16T20:51:14.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.504 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:14.504 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057354+0000", 2024-09-16T20:51:14.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7780000000000002e-06 2024-09-16T20:51:14.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.504 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:14.504 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057360+0000", 2024-09-16T20:51:14.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0419999999999999e-06 2024-09-16T20:51:14.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:14.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057365+0000", 2024-09-16T20:51:14.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7929999999999997e-06 2024-09-16T20:51:14.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:14.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114460+0000", 2024-09-16T20:51:14.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057094662999999997 2024-09-16T20:51:14.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:14.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114532+0000", 2024-09-16T20:51:14.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2881999999999998e-05 2024-09-16T20:51:14.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:14.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114624+0000", 2024-09-16T20:51:14.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1555e-05 2024-09-16T20:51:14.506 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:14.506 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:14.506 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:14.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.506 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8800 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2316288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:14.506 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:51:14.506 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 73.400314515000005, 2024-09-16T20:51:14.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.05746946, 2024-09-16T20:51:14.506 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:14.507 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:14.507 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:14.507 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:14.507 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:14.507 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:14.507 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8800 2024-09-16T20:51:14.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.507 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:14.507 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.507 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:14.507 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:51:14.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:14.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.507 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.507 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:14.508 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:51:14.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:14.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.508 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:14.508 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:51:14.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5199999999999998e-07 2024-09-16T20:51:14.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.508 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:14.508 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057373+0000", 2024-09-16T20:51:14.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.79e-06 2024-09-16T20:51:14.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:14.509 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057379+0000", 2024-09-16T20:51:14.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6699999999999999e-06 2024-09-16T20:51:14.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:14.509 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057386+0000", 2024-09-16T20:51:14.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3170000000000004e-06 2024-09-16T20:51:14.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:14.509 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114655+0000", 2024-09-16T20:51:14.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057269420000000001 2024-09-16T20:51:14.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:14.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114734+0000", 2024-09-16T20:51:14.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8084000000000007e-05 2024-09-16T20:51:14.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:14.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114840+0000", 2024-09-16T20:51:14.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000106827 2024-09-16T20:51:14.510 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:14.510 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:14.510 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:14.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.511 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8801 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2334720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:14.511 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:51:14.511 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 73.400293484000002, 2024-09-16T20:51:14.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057594682000000001, 2024-09-16T20:51:14.511 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:14.511 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:14.511 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:14.511 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:14.511 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:14.512 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:14.512 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8801 2024-09-16T20:51:14.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.512 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:14.512 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.512 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:14.512 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:51:14.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:14.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.512 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.513 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:14.513 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:51:14.513 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:14.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.513 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:14.513 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:51:14.513 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6899999999999998e-07 2024-09-16T20:51:14.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.513 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:14.513 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057394+0000", 2024-09-16T20:51:14.513 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8300000000000001e-06 2024-09-16T20:51:14.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:14.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057400+0000", 2024-09-16T20:51:14.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6500000000000001e-06 2024-09-16T20:51:14.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:14.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057404+0000", 2024-09-16T20:51:14.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4669999999999998e-06 2024-09-16T20:51:14.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:14.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114874+0000", 2024-09-16T20:51:14.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057469602000000002 2024-09-16T20:51:14.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.515 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.515 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:14.515 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114933+0000", 2024-09-16T20:51:14.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9500000000000003e-05 2024-09-16T20:51:14.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.515 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.515 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:14.515 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114987+0000", 2024-09-16T20:51:14.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3263999999999997e-05 2024-09-16T20:51:14.515 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:14.515 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:14.515 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:14.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.516 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.516 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8802 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2353152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:14.516 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:51:14.516 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 73.400244799000006, 2024-09-16T20:51:14.516 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057714079000000001, 2024-09-16T20:51:14.516 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:14.516 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:14.516 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:14.516 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:14.516 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:14.516 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:14.516 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8802 2024-09-16T20:51:14.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.517 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:14.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.517 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:14.517 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:51:14.517 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:14.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.517 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:14.517 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:51:14.517 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:14.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.517 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:14.518 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:51:14.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4999999999999998e-07 2024-09-16T20:51:14.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.518 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:14.518 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057446+0000", 2024-09-16T20:51:14.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4240000000000003e-06 2024-09-16T20:51:14.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.518 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:14.518 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057452+0000", 2024-09-16T20:51:14.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8379999999999996e-06 2024-09-16T20:51:14.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.519 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:14.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057460+0000", 2024-09-16T20:51:14.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9629999999999998e-06 2024-09-16T20:51:14.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.519 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:14.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115015+0000", 2024-09-16T20:51:14.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057554389999999997 2024-09-16T20:51:14.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.519 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:14.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115082+0000", 2024-09-16T20:51:14.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7733999999999999e-05 2024-09-16T20:51:14.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.520 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:14.520 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115155+0000", 2024-09-16T20:51:14.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2280000000000006e-05 2024-09-16T20:51:14.520 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:14.520 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:14.520 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:14.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.520 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8804 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2555904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:14.520 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:51:14.520 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 73.400218483000003, 2024-09-16T20:51:14.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057833862, 2024-09-16T20:51:14.521 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:14.521 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:14.521 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:14.521 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:14.521 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:14.521 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:14.521 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8804 2024-09-16T20:51:14.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.521 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:14.521 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.521 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:14.521 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:51:14.521 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:14.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.522 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:14.522 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:51:14.522 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:14.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.522 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:14.522 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:51:14.522 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7899999999999999e-07 2024-09-16T20:51:14.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.522 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:14.522 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057469+0000", 2024-09-16T20:51:14.522 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.001e-06 2024-09-16T20:51:14.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:14.523 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057475+0000", 2024-09-16T20:51:14.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7629999999999997e-06 2024-09-16T20:51:14.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:14.523 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057480+0000", 2024-09-16T20:51:14.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8069999999999999e-06 2024-09-16T20:51:14.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:14.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115183+0000", 2024-09-16T20:51:14.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057703364 2024-09-16T20:51:14.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.524 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:14.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115227+0000", 2024-09-16T20:51:14.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3903999999999998e-05 2024-09-16T20:51:14.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.524 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:14.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115301+0000", 2024-09-16T20:51:14.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3543999999999999e-05 2024-09-16T20:51:14.524 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:14.524 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:14.525 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:14.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.525 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.525 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8805 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2574336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:14.525 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057487+0000", 2024-09-16T20:51:14.525 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 73.400198154999998, 2024-09-16T20:51:14.525 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058021027000000003, 2024-09-16T20:51:14.525 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:14.525 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:14.525 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:14.525 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:14.525 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:14.525 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:14.525 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8805 2024-09-16T20:51:14.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.526 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:14.526 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.526 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:14.526 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057487+0000", 2024-09-16T20:51:14.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:14.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.526 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.526 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:14.526 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057487+0000", 2024-09-16T20:51:14.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:14.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.526 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.526 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:14.526 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057488+0000", 2024-09-16T20:51:14.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5199999999999998e-07 2024-09-16T20:51:14.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.527 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:14.527 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057490+0000", 2024-09-16T20:51:14.527 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.939e-06 2024-09-16T20:51:14.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.527 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:14.527 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057495+0000", 2024-09-16T20:51:14.527 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6439999999999997e-06 2024-09-16T20:51:14.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.527 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:14.527 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057500+0000", 2024-09-16T20:51:14.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8119999999999998e-06 2024-09-16T20:51:14.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:14.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115341+0000", 2024-09-16T20:51:14.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057841341999999997 2024-09-16T20:51:14.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:14.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115422+0000", 2024-09-16T20:51:14.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1003999999999999e-05 2024-09-16T20:51:14.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:14.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115508+0000", 2024-09-16T20:51:14.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5933999999999994e-05 2024-09-16T20:51:14.529 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:14.529 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:14.529 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:14.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.529 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8807 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2592768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:14.529 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057505+0000", 2024-09-16T20:51:14.529 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 73.400179945000005, 2024-09-16T20:51:14.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058186633000000001, 2024-09-16T20:51:14.529 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:14.529 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:14.529 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:14.529 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:14.530 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:14.530 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:14.530 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8807 2024-09-16T20:51:14.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.530 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:14.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.530 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:14.530 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057505+0000", 2024-09-16T20:51:14.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:14.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.530 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:14.530 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057505+0000", 2024-09-16T20:51:14.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:14.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:14.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057506+0000", 2024-09-16T20:51:14.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.27e-07 2024-09-16T20:51:14.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:14.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057508+0000", 2024-09-16T20:51:14.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7400000000000001e-06 2024-09-16T20:51:14.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:14.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057513+0000", 2024-09-16T20:51:14.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.468e-06 2024-09-16T20:51:14.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.532 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:14.532 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057517+0000", 2024-09-16T20:51:14.532 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4329999999999997e-06 2024-09-16T20:51:14.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.532 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:14.532 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115538+0000", 2024-09-16T20:51:14.532 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058020355000000003 2024-09-16T20:51:14.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.532 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:14.532 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115616+0000", 2024-09-16T20:51:14.532 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7998999999999996e-05 2024-09-16T20:51:14.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.533 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.533 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:14.533 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115692+0000", 2024-09-16T20:51:14.533 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6310999999999995e-05 2024-09-16T20:51:14.533 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:14.533 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:14.533 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:14.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.533 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.533 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8808 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2611200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:14.533 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:51:14.533 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 73.400162683999994, 2024-09-16T20:51:14.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058354046, 2024-09-16T20:51:14.534 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:14.534 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:14.534 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:14.534 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:14.534 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:14.534 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:14.534 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8808 2024-09-16T20:51:14.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.534 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:14.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.534 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:14.534 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:51:14.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:14.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:14.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:51:14.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:14.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:14.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:51:14.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2500000000000001e-07 2024-09-16T20:51:14.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.536 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:14.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057525+0000", 2024-09-16T20:51:14.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5549999999999999e-06 2024-09-16T20:51:14.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.536 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:14.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057530+0000", 2024-09-16T20:51:14.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5489999999999997e-06 2024-09-16T20:51:14.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.536 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:14.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057535+0000", 2024-09-16T20:51:14.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5709999999999999e-06 2024-09-16T20:51:14.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:14.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115722+0000", 2024-09-16T20:51:14.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058187323999999999 2024-09-16T20:51:14.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:14.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115799+0000", 2024-09-16T20:51:14.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6445000000000004e-05 2024-09-16T20:51:14.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.541 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:14.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115877+0000", 2024-09-16T20:51:14.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8276999999999996e-05 2024-09-16T20:51:14.541 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:14.541 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:14.541 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:14.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.541 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:2182 3.17 3:edb5e5cd:::rbd_data.110e61479aee.0000000000000014:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write 0~5120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-16T20:51:14.541 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:22.734609+0000", 2024-09-16T20:51:14.541 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.723076437000003, 2024-09-16T20:51:14.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.124890894, 2024-09-16T20:51:14.541 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:14.542 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:14.542 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:14.542 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:14.542 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:51:14.542 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:14.542 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 2182 2024-09-16T20:51:14.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.542 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:14.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:14.542 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734609+0000", 2024-09-16T20:51:14.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:14.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.543 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.543 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:14.543 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734609+0000", 2024-09-16T20:51:14.543 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:14.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.543 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.543 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:14.543 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734611+0000", 2024-09-16T20:51:14.543 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9599999999999999e-06 2024-09-16T20:51:14.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.543 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.543 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:14.543 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734628+0000", 2024-09-16T20:51:14.544 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.715e-05 2024-09-16T20:51:14.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.544 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:14.544 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734656+0000", 2024-09-16T20:51:14.544 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7759999999999998e-05 2024-09-16T20:51:14.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.544 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:14.544 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734669+0000", 2024-09-16T20:51:14.544 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3315999999999999e-05 2024-09-16T20:51:14.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.545 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:14.545 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734826+0000", 2024-09-16T20:51:14.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015668499999999999 2024-09-16T20:51:14.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.545 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:14.545 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.735048+0000", 2024-09-16T20:51:14.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00022179 2024-09-16T20:51:14.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.545 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:14.545 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.735248+0000", 2024-09-16T20:51:14.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00020010399999999999 2024-09-16T20:51:14.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:51:14.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.736252+0000", 2024-09-16T20:51:14.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010039739999999999 2024-09-16T20:51:14.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:14.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.859340+0000", 2024-09-16T20:51:14.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.12308846599999999 2024-09-16T20:51:14.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:51:14.547 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.859431+0000", 2024-09-16T20:51:14.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1099000000000003e-05 2024-09-16T20:51:14.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.547 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:14.547 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.859500+0000", 2024-09-16T20:51:14.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8590000000000006e-05 2024-09-16T20:51:14.547 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:14.547 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:14.547 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:14.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.547 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:29157 3.1b 3:dbe69130:::journal.110e1b19a241:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e167)", 2024-09-16T20:51:14.547 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:54.663540+0000", 2024-09-16T20:51:14.548 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.794145099000001, 2024-09-16T20:51:14.548 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.92912227999999997, 2024-09-16T20:51:14.548 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:14.548 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:14.548 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:14.548 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:14.548 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:14.548 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:14.548 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 29157 2024-09-16T20:51:14.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.548 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:14.548 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.548 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:14.549 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663540+0000", 2024-09-16T20:51:14.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:14.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.549 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:14.549 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663540+0000", 2024-09-16T20:51:14.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:14.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.549 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:14.549 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663541+0000", 2024-09-16T20:51:14.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1680000000000001e-06 2024-09-16T20:51:14.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.550 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:14.550 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663546+0000", 2024-09-16T20:51:14.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7129999999999996e-06 2024-09-16T20:51:14.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.550 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:14.550 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663558+0000", 2024-09-16T20:51:14.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1347e-05 2024-09-16T20:51:14.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.550 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:14.550 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663563+0000", 2024-09-16T20:51:14.551 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1250000000000001e-06 2024-09-16T20:51:14.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.551 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:14.551 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663612+0000", 2024-09-16T20:51:14.551 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9665e-05 2024-09-16T20:51:14.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.551 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:14.551 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663620+0000", 2024-09-16T20:51:14.551 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8439999999999998e-06 2024-09-16T20:51:14.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.551 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:14.552 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:55.592525+0000", 2024-09-16T20:51:14.552 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.92890456700000001 2024-09-16T20:51:14.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.552 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.552 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:14.552 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:55.592663+0000", 2024-09-16T20:51:14.552 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013785100000000001 2024-09-16T20:51:14.552 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:14.552 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:14.552 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:14.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.552 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.552 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:30229 3.16 3:6f5476df:::rbd_header.110e25db1f60:head [watch reconnect cookie 140213637350016 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e181)", 2024-09-16T20:51:14.552 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:51:08.603124+0000", 2024-09-16T20:51:14.553 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.8545614539999997, 2024-09-16T20:51:14.553 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.97765519899999997, 2024-09-16T20:51:14.553 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:14.553 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:14.553 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:14.553 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:14.553 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:14.553 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:14.553 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30229 2024-09-16T20:51:14.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.553 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:14.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.553 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:14.553 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603124+0000", 2024-09-16T20:51:14.554 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:14.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.554 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.554 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:14.554 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603124+0000", 2024-09-16T20:51:14.554 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:14.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.554 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.554 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:14.554 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603125+0000", 2024-09-16T20:51:14.554 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5090000000000001e-06 2024-09-16T20:51:14.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.554 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.554 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:14.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603131+0000", 2024-09-16T20:51:14.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9089999999999997e-06 2024-09-16T20:51:14.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:14.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603143+0000", 2024-09-16T20:51:14.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1416e-05 2024-09-16T20:51:14.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:14.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603151+0000", 2024-09-16T20:51:14.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7889999999999995e-06 2024-09-16T20:51:14.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.556 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:14.556 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603674+0000", 2024-09-16T20:51:14.556 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00052357 2024-09-16T20:51:14.556 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.556 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.556 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:14.556 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603684+0000", 2024-09-16T20:51:14.556 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0274e-05 2024-09-16T20:51:14.556 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.556 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.556 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:14.556 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.575380+0000", 2024-09-16T20:51:14.556 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.97169509799999998 2024-09-16T20:51:14.556 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.557 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.557 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:14.557 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.575591+0000", 2024-09-16T20:51:14.557 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00021126700000000001 2024-09-16T20:51:14.557 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.557 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.557 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:14.557 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.580779+0000", 2024-09-16T20:51:14.557 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0051883670000000002 2024-09-16T20:51:14.557 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:14.557 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:14.557 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:14.557 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.557 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.557 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:30230 3.16 3:6f5476df:::rbd_header.110e25db1f60:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e181)", 2024-09-16T20:51:14.557 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:51:08.603204+0000", 2024-09-16T20:51:14.558 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.8544815059999999, 2024-09-16T20:51:14.558 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.98129583300000001, 2024-09-16T20:51:14.558 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:14.558 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:14.558 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:14.558 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:14.558 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:14.558 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:14.558 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30230 2024-09-16T20:51:14.558 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.558 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:14.558 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.558 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:14.559 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603204+0000", 2024-09-16T20:51:14.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:14.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:14.559 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603204+0000", 2024-09-16T20:51:14.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:14.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:14.559 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603205+0000", 2024-09-16T20:51:14.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4900000000000005e-07 2024-09-16T20:51:14.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:14.560 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603209+0000", 2024-09-16T20:51:14.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5210000000000004e-06 2024-09-16T20:51:14.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.560 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:14.560 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603222+0000", 2024-09-16T20:51:14.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2733e-05 2024-09-16T20:51:14.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.560 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:14.560 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603228+0000", 2024-09-16T20:51:14.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4520000000000002e-06 2024-09-16T20:51:14.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.561 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:14.561 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603719+0000", 2024-09-16T20:51:14.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00049103600000000003 2024-09-16T20:51:14.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.561 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.561 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:14.561 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603730+0000", 2024-09-16T20:51:14.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.023e-05 2024-09-16T20:51:14.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.561 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.561 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:14.561 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.577394+0000", 2024-09-16T20:51:14.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.97366440099999996 2024-09-16T20:51:14.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.562 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:14.562 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.577499+0000", 2024-09-16T20:51:14.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000104509 2024-09-16T20:51:14.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.562 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:14.562 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.577920+0000", 2024-09-16T20:51:14.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00042173699999999999 2024-09-16T20:51:14.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.562 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:51:14.562 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.580795+0000", 2024-09-16T20:51:14.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.002874302 2024-09-16T20:51:14.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.563 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:14.563 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.584423+0000", 2024-09-16T20:51:14.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0036280010000000001 2024-09-16T20:51:14.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.563 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:51:14.563 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.584465+0000", 2024-09-16T20:51:14.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2590999999999999e-05 2024-09-16T20:51:14.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:14.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:14.563 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:14.563 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.584500+0000", 2024-09-16T20:51:14.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4570999999999998e-05 2024-09-16T20:51:14.564 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:14.564 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:14.564 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:14.564 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:14.564 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:14.564 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:14.565 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:51:14.563+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:51:14.566 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:51:14.666 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:14.665+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:14.678 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:14.679 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:51:14.679 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:51:14.679 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:14.687 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:51:14.766 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:14.765+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:14.798 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:14.798 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:51:14.798 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:51:14.798 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:51:14.798 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:14.807 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:51:14.811 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T20:51:14.892 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:14.891+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:14.964 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:14.964 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:51:14.964 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:51:14.964 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:51:14.964 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.964 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5439 3.a 3:5789f5c3:::rbd_data.110ed9ca540.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 1181696~60416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:14.964 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:51:14.964 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 172.0970465, 2024-09-16T20:51:14.964 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10102045599999999, 2024-09-16T20:51:14.964 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:14.964 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:14.965 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:14.965 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:14.965 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:14.965 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:14.965 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5439 2024-09-16T20:51:14.965 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.965 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:14.965 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.965 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:14.965 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:51:14.965 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:14.965 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.965 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.965 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:14.966 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:51:14.966 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:14.966 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.966 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.966 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:14.966 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864711+0000", 2024-09-16T20:51:14.966 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7899999999999997e-07 2024-09-16T20:51:14.966 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.966 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.966 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:14.966 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864760+0000", 2024-09-16T20:51:14.966 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8803e-05 2024-09-16T20:51:14.966 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.966 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.967 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:14.967 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864769+0000", 2024-09-16T20:51:14.967 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0999999999999993e-06 2024-09-16T20:51:14.967 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.967 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.967 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:14.967 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864774+0000", 2024-09-16T20:51:14.967 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8919999999999999e-06 2024-09-16T20:51:14.967 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.967 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.967 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:14.967 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864815+0000", 2024-09-16T20:51:14.967 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1307000000000003e-05 2024-09-16T20:51:14.967 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.968 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.968 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:14.968 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864893+0000", 2024-09-16T20:51:14.968 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7581000000000001e-05 2024-09-16T20:51:14.968 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.968 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.968 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 1", 2024-09-16T20:51:14.968 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864996+0000", 2024-09-16T20:51:14.968 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102725 2024-09-16T20:51:14.968 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.968 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.968 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:14.968 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965442+0000", 2024-09-16T20:51:14.968 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100446012 2024-09-16T20:51:14.969 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.969 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.969 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:14.969 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965618+0000", 2024-09-16T20:51:14.969 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00017653100000000001 2024-09-16T20:51:14.969 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.969 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.969 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:14.969 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965682+0000", 2024-09-16T20:51:14.969 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3905999999999994e-05 2024-09-16T20:51:14.969 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.969 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.969 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:14.969 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965731+0000", 2024-09-16T20:51:14.970 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8819999999999997e-05 2024-09-16T20:51:14.970 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:14.970 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:14.970 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:14.970 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.970 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.970 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4600.0:5439 3.a e37/23 ondisk, result = 0)", 2024-09-16T20:51:14.970 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:51:14.970 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 172.09605035199999, 2024-09-16T20:51:14.970 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099742634999999996, 2024-09-16T20:51:14.970 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:14.970 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:14.970 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:14.970 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:14.971 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.971 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:14.971 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:51:14.971 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:14.971 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.971 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.971 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:14.971 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:51:14.971 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:14.971 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.971 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.971 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:14.971 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:51:14.971 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0900000000000001e-07 2024-09-16T20:51:14.972 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.972 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.972 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:14.972 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865711+0000", 2024-09-16T20:51:14.972 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3739999999999998e-06 2024-09-16T20:51:14.972 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.972 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.972 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:14.972 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965343+0000", 2024-09-16T20:51:14.972 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099632526999999999 2024-09-16T20:51:14.972 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.972 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.972 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:14.972 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965357+0000", 2024-09-16T20:51:14.972 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3713999999999999e-05 2024-09-16T20:51:14.973 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.973 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.973 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:14.973 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965426+0000", 2024-09-16T20:51:14.973 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8745999999999995e-05 2024-09-16T20:51:14.973 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.973 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.973 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:14.973 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965434+0000", 2024-09-16T20:51:14.973 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6910000000000003e-06 2024-09-16T20:51:14.973 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.973 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.973 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:14.973 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965449+0000", 2024-09-16T20:51:14.974 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4874e-05 2024-09-16T20:51:14.974 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:14.974 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:14.974 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:14.974 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.974 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.974 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4882.0:2508 3.12 3:4992771f:::rbd_object_map.110e7c8851c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T20:51:14.974 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:51:14.974 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 148.48024680399999, 2024-09-16T20:51:14.974 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.125007011, 2024-09-16T20:51:14.974 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:14.974 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:14.974 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:14.974 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:14.975 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4882", 2024-09-16T20:51:14.975 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2436923223", 2024-09-16T20:51:14.975 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2508 2024-09-16T20:51:14.975 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.975 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:14.975 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.975 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:14.975 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:51:14.975 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:14.975 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.975 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.975 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:14.975 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:51:14.975 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:14.976 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.976 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.976 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:14.976 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481511+0000", 2024-09-16T20:51:14.976 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0699999999999996e-07 2024-09-16T20:51:14.976 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.976 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.976 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:14.976 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481515+0000", 2024-09-16T20:51:14.976 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0609999999999997e-06 2024-09-16T20:51:14.976 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.976 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.976 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:14.977 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481523+0000", 2024-09-16T20:51:14.977 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1310000000000006e-06 2024-09-16T20:51:14.977 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.977 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.977 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:14.977 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481528+0000", 2024-09-16T20:51:14.977 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4900000000000002e-06 2024-09-16T20:51:14.977 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.977 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.977 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:14.977 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481565+0000", 2024-09-16T20:51:14.977 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7066000000000001e-05 2024-09-16T20:51:14.977 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.977 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.978 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:14.978 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481628+0000", 2024-09-16T20:51:14.978 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3118000000000001e-05 2024-09-16T20:51:14.978 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.978 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.978 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 2", 2024-09-16T20:51:14.978 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481822+0000", 2024-09-16T20:51:14.978 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019446200000000001 2024-09-16T20:51:14.978 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.978 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.978 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:14.978 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605988+0000", 2024-09-16T20:51:14.978 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.124165888 2024-09-16T20:51:14.979 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.979 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.979 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:14.979 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606427+0000", 2024-09-16T20:51:14.979 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00043854899999999999 2024-09-16T20:51:14.979 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.979 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.979 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:14.979 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606476+0000", 2024-09-16T20:51:14.979 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8971999999999998e-05 2024-09-16T20:51:14.979 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.979 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.979 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:14.979 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606517+0000", 2024-09-16T20:51:14.980 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1467000000000001e-05 2024-09-16T20:51:14.980 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:14.980 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:14.980 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:14.980 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.980 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.980 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4882.0:2508 3.12 e57/55 ondisk, result = 0)", 2024-09-16T20:51:14.980 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:51:14.980 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 148.47925830400001, 2024-09-16T20:51:14.980 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12349711400000001, 2024-09-16T20:51:14.980 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:14.980 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:14.980 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:14.980 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:14.981 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.981 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:14.981 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:51:14.981 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:14.981 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.981 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.981 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:14.981 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:51:14.981 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:14.981 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.981 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.981 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:14.981 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:51:14.981 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.06e-07 2024-09-16T20:51:14.982 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.982 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.982 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:14.982 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482502+0000", 2024-09-16T20:51:14.982 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.01e-06 2024-09-16T20:51:14.982 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.982 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.982 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:14.982 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605772+0000", 2024-09-16T20:51:14.982 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.123269687 2024-09-16T20:51:14.982 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.982 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.982 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:14.982 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605791+0000", 2024-09-16T20:51:14.983 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8912000000000001e-05 2024-09-16T20:51:14.983 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.983 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.983 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:14.983 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605974+0000", 2024-09-16T20:51:14.983 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000182836 2024-09-16T20:51:14.983 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.983 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.983 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:14.983 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605982+0000", 2024-09-16T20:51:14.983 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1699999999999997e-06 2024-09-16T20:51:14.983 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.983 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.983 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:14.984 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605996+0000", 2024-09-16T20:51:14.984 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3893000000000001e-05 2024-09-16T20:51:14.984 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:14.984 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:14.984 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:14.984 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.984 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.984 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4882.0:2512 3.f 3:f4d895b7:::rbd_data.110e7c8851c.0000000000000000:head [read 16384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T20:51:14.984 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:51:14.984 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 148.47882127400001, 2024-09-16T20:51:14.984 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12318483, 2024-09-16T20:51:14.984 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:14.984 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:14.984 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:14.985 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:14.985 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4882", 2024-09-16T20:51:14.985 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2436923223", 2024-09-16T20:51:14.985 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2512 2024-09-16T20:51:14.985 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.985 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:14.985 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.985 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:14.985 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:51:14.985 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:14.985 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.985 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.985 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:14.985 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:51:14.986 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:14.986 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.986 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.986 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:14.986 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:51:14.986 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3499999999999995e-07 2024-09-16T20:51:14.986 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.986 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.986 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:14.986 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482943+0000", 2024-09-16T20:51:14.986 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3450000000000002e-06 2024-09-16T20:51:14.986 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.986 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.986 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:14.987 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605772+0000", 2024-09-16T20:51:14.987 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.122829796 2024-09-16T20:51:14.987 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.987 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.987 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:14.987 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605792+0000", 2024-09-16T20:51:14.987 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9916999999999999e-05 2024-09-16T20:51:14.987 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.987 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.987 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:14.987 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605917+0000", 2024-09-16T20:51:14.987 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00012500799999999999 2024-09-16T20:51:14.987 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.987 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.988 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:14.988 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606004+0000", 2024-09-16T20:51:14.988 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6242999999999994e-05 2024-09-16T20:51:14.988 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.988 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.988 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:14.988 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606120+0000", 2024-09-16T20:51:14.988 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000116786 2024-09-16T20:51:14.988 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:14.988 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:14.988 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:14.988 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.988 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.988 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:11702 3.15 3:acf33c8b:::rbd_data.110e747f0779.000000000000001a:head [set-alloc-hint object_size 262144 write_size 262144,write 143360~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:51:14.989 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:51:14.989 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 130.82382885199999, 2024-09-16T20:51:14.989 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.084801043000000006, 2024-09-16T20:51:14.989 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:14.989 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:14.989 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:14.989 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:14.989 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:51:14.989 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:51:14.989 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 11702 2024-09-16T20:51:14.989 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.989 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:14.989 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.989 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:14.989 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:51:14.990 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:14.990 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.990 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.990 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:14.990 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:51:14.990 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:14.990 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.990 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.990 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:14.990 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137929+0000", 2024-09-16T20:51:14.990 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8400000000000002e-07 2024-09-16T20:51:14.990 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.990 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.990 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:14.991 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137937+0000", 2024-09-16T20:51:14.991 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0260000000000007e-06 2024-09-16T20:51:14.991 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.991 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.991 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:14.991 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137952+0000", 2024-09-16T20:51:14.991 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4659e-05 2024-09-16T20:51:14.991 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.991 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.991 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:14.991 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137960+0000", 2024-09-16T20:51:14.991 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0169999999999996e-06 2024-09-16T20:51:14.991 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.992 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.992 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:14.992 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138010+0000", 2024-09-16T20:51:14.992 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0021000000000001e-05 2024-09-16T20:51:14.992 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.992 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.992 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:14.992 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138114+0000", 2024-09-16T20:51:14.992 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000103797 2024-09-16T20:51:14.992 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.992 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.992 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:51:14.992 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138233+0000", 2024-09-16T20:51:14.993 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000118971 2024-09-16T20:51:14.993 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.993 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.993 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:14.993 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138953+0000", 2024-09-16T20:51:14.993 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00072026000000000002 2024-09-16T20:51:14.993 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.993 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.993 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:14.993 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222609+0000", 2024-09-16T20:51:14.993 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.083656650999999999 2024-09-16T20:51:14.993 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.993 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.993 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:14.994 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222684+0000", 2024-09-16T20:51:14.994 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4369999999999994e-05 2024-09-16T20:51:14.994 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.994 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.994 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:14.994 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222729+0000", 2024-09-16T20:51:14.994 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5287000000000002e-05 2024-09-16T20:51:14.994 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:14.994 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:14.994 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:14.994 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.994 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.994 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:11704 3.9 3:91220b75:::rbd_data.110e747f0779.000000000000001c:head [set-alloc-hint object_size 262144 write_size 262144,write 139264~6656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:51:14.994 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:51:14.994 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 130.82359085100001, 2024-09-16T20:51:14.995 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.084531133999999994, 2024-09-16T20:51:14.995 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:14.995 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:14.995 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:14.995 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:14.995 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:51:14.995 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:51:14.995 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 11704 2024-09-16T20:51:14.995 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.995 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:14.995 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.995 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:14.995 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:51:14.995 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:14.996 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.996 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.996 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:14.996 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:51:14.996 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:14.996 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.996 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.996 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:14.996 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138167+0000", 2024-09-16T20:51:14.996 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.33e-07 2024-09-16T20:51:14.996 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.996 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.996 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:14.996 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138179+0000", 2024-09-16T20:51:14.997 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2055999999999999e-05 2024-09-16T20:51:14.997 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.997 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.997 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:14.997 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138193+0000", 2024-09-16T20:51:14.997 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3961000000000001e-05 2024-09-16T20:51:14.997 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.997 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.997 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:14.997 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138200+0000", 2024-09-16T20:51:14.997 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4429999999999997e-06 2024-09-16T20:51:14.997 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.997 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.997 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:14.998 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138258+0000", 2024-09-16T20:51:14.998 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7225999999999997e-05 2024-09-16T20:51:14.998 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.998 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.998 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:14.998 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138364+0000", 2024-09-16T20:51:14.998 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000106013 2024-09-16T20:51:14.998 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.998 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.998 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:51:14.998 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138494+0000", 2024-09-16T20:51:14.998 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013059599999999999 2024-09-16T20:51:14.998 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.998 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.998 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:14.999 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.139220+0000", 2024-09-16T20:51:14.999 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00072539800000000004 2024-09-16T20:51:14.999 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.999 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.999 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:14.999 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222608+0000", 2024-09-16T20:51:14.999 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.083388620999999996 2024-09-16T20:51:14.999 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.999 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:14.999 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:14.999 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222651+0000", 2024-09-16T20:51:14.999 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2228000000000003e-05 2024-09-16T20:51:14.999 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:14.999 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.000 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:15.000 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222697+0000", 2024-09-16T20:51:15.000 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6659000000000002e-05 2024-09-16T20:51:15.000 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:15.000 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:15.000 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:15.000 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.000 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.000 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5138.0:9358 3.c 3.c25699cc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T20:51:15.000 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:51:15.000 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 108.111443546, 2024-09-16T20:51:15.000 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0018657580000001, 2024-09-16T20:51:15.000 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:15.000 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:15.001 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:51:15.001 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:15.001 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5138", 2024-09-16T20:51:15.001 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:51:15.001 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 9358 2024-09-16T20:51:15.001 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.001 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:15.001 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.001 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:15.001 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:51:15.001 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:15.001 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.001 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.001 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:15.002 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:51:15.002 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:15.002 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.002 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.002 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:15.002 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850314+0000", 2024-09-16T20:51:15.002 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4099999999999999e-07 2024-09-16T20:51:15.002 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.002 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.002 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:15.002 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850316+0000", 2024-09-16T20:51:15.002 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2469999999999998e-06 2024-09-16T20:51:15.002 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.002 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.003 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:15.003 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850322+0000", 2024-09-16T20:51:15.003 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3470000000000004e-06 2024-09-16T20:51:15.003 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.003 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.003 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:15.003 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850327+0000", 2024-09-16T20:51:15.003 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9450000000000001e-06 2024-09-16T20:51:15.003 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.003 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.003 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:15.003 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850376+0000", 2024-09-16T20:51:15.003 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8257000000000002e-05 2024-09-16T20:51:15.003 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.004 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.004 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:15.004 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850385+0000", 2024-09-16T20:51:15.004 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9800000000000004e-06 2024-09-16T20:51:15.004 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.004 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.004 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:15.004 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852167+0000", 2024-09-16T20:51:15.004 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0017825659999999 2024-09-16T20:51:15.004 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.004 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.004 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:15.004 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852179+0000", 2024-09-16T20:51:15.005 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1975e-05 2024-09-16T20:51:15.005 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:15.005 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:15.005 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:15.005 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.005 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.005 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5138.0:9382 3.c 3.c25699cc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T20:51:15.005 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:51:15.005 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 108.10309653, 2024-09-16T20:51:15.005 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.99358404700000003, 2024-09-16T20:51:15.005 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:15.005 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:15.005 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:51:15.005 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:15.006 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5138", 2024-09-16T20:51:15.006 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:51:15.006 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 9382 2024-09-16T20:51:15.006 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.006 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:15.006 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.006 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:15.006 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:51:15.006 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:15.006 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.006 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.006 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:15.006 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:51:15.006 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:15.007 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.007 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.007 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:15.007 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858661+0000", 2024-09-16T20:51:15.007 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0200000000000002e-07 2024-09-16T20:51:15.007 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.007 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.007 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:15.007 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858664+0000", 2024-09-16T20:51:15.007 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.038e-06 2024-09-16T20:51:15.007 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.007 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.007 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:15.007 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858671+0000", 2024-09-16T20:51:15.008 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0110000000000003e-06 2024-09-16T20:51:15.008 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.008 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.008 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:15.008 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858677+0000", 2024-09-16T20:51:15.008 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0959999999999997e-06 2024-09-16T20:51:15.008 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.008 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.008 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:15.008 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858732+0000", 2024-09-16T20:51:15.008 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4543000000000003e-05 2024-09-16T20:51:15.008 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.008 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.009 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:15.009 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858739+0000", 2024-09-16T20:51:15.009 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5429999999999996e-06 2024-09-16T20:51:15.009 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.009 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.009 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:15.009 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852236+0000", 2024-09-16T20:51:15.009 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.993496453 2024-09-16T20:51:15.009 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.009 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.009 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:15.009 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852244+0000", 2024-09-16T20:51:15.009 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8610000000000004e-06 2024-09-16T20:51:15.009 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:15.010 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:15.010 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:15.010 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.010 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.010 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5763.0:4895 3.a 3:5384606c:::rbd_data.110e1b19a241.0000000000000018:head [read 0~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-16T20:51:15.010 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:45.598212+0000", 2024-09-16T20:51:15.010 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 89.363544489000006, 2024-09-16T20:51:15.010 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0375864829999999, 2024-09-16T20:51:15.010 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:15.010 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:15.010 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:15.010 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:15.010 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5763", 2024-09-16T20:51:15.010 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/623897567", 2024-09-16T20:51:15.011 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4895 2024-09-16T20:51:15.011 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.011 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:15.011 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.011 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:15.011 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598212+0000", 2024-09-16T20:51:15.011 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:15.011 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.011 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.011 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:15.011 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598212+0000", 2024-09-16T20:51:15.011 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:15.011 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.012 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.012 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:15.012 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598214+0000", 2024-09-16T20:51:15.012 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0090000000000002e-06 2024-09-16T20:51:15.012 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.012 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.012 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:15.012 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598222+0000", 2024-09-16T20:51:15.012 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6699999999999994e-06 2024-09-16T20:51:15.012 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.012 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.012 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:15.012 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598241+0000", 2024-09-16T20:51:15.012 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8896999999999999e-05 2024-09-16T20:51:15.013 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.013 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.013 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:15.013 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598250+0000", 2024-09-16T20:51:15.013 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6649999999999992e-06 2024-09-16T20:51:15.013 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.013 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.013 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:15.013 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598325+0000", 2024-09-16T20:51:15.013 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5189000000000001e-05 2024-09-16T20:51:15.013 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.013 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.013 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op must wait for map", 2024-09-16T20:51:15.014 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598333+0000", 2024-09-16T20:51:15.014 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.3170000000000002e-06 2024-09-16T20:51:15.014 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.014 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.014 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:15.014 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601355+0000", 2024-09-16T20:51:15.014 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0030215400000000001 2024-09-16T20:51:15.014 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.014 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.014 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:15.014 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601365+0000", 2024-09-16T20:51:15.014 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0215e-05 2024-09-16T20:51:15.014 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.014 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.015 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:15.015 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635506+0000", 2024-09-16T20:51:15.015 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0341408240000001 2024-09-16T20:51:15.015 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.015 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.015 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:15.015 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635655+0000", 2024-09-16T20:51:15.015 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000149412 2024-09-16T20:51:15.015 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.015 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.015 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:15.015 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635799+0000", 2024-09-16T20:51:15.015 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000143745 2024-09-16T20:51:15.015 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:15.016 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:15.016 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:15.016 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.016 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.016 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5763.0:4900 3.a 3:5384606c:::rbd_data.110e1b19a241.0000000000000018:head [read 8192~24576] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-16T20:51:15.016 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:51:15.016 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 89.363486012999999, 2024-09-16T20:51:15.016 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0379340800000001, 2024-09-16T20:51:15.016 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:15.016 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:15.016 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:15.016 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:15.016 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5763", 2024-09-16T20:51:15.017 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/623897567", 2024-09-16T20:51:15.017 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4900 2024-09-16T20:51:15.017 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.017 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:15.017 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.017 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:15.017 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:51:15.017 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:15.017 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.017 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.017 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:15.017 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:51:15.017 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:15.017 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.018 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.018 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:15.018 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:51:15.018 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4300000000000003e-07 2024-09-16T20:51:15.018 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.018 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.018 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:15.018 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598274+0000", 2024-09-16T20:51:15.018 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.976e-06 2024-09-16T20:51:15.018 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.018 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.018 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:15.018 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598282+0000", 2024-09-16T20:51:15.018 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0029999999999994e-06 2024-09-16T20:51:15.019 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.019 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.019 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:15.019 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598289+0000", 2024-09-16T20:51:15.019 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2389999999999999e-06 2024-09-16T20:51:15.019 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.019 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.019 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:15.019 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598430+0000", 2024-09-16T20:51:15.019 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014119600000000001 2024-09-16T20:51:15.019 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.019 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.019 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting_for_map not empty", 2024-09-16T20:51:15.020 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598435+0000", 2024-09-16T20:51:15.020 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3489999999999999e-06 2024-09-16T20:51:15.020 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.020 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.020 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:15.020 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601429+0000", 2024-09-16T20:51:15.020 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0029942720000000001 2024-09-16T20:51:15.020 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.020 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.020 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:15.020 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601438+0000", 2024-09-16T20:51:15.020 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4759999999999998e-06 2024-09-16T20:51:15.020 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.021 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.021 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:15.021 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635987+0000", 2024-09-16T20:51:15.021 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0345495010000001 2024-09-16T20:51:15.021 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.021 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.021 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:15.021 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.636056+0000", 2024-09-16T20:51:15.021 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8610999999999998e-05 2024-09-16T20:51:15.021 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.021 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.021 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:15.021 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.636205+0000", 2024-09-16T20:51:15.022 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014891399999999999 2024-09-16T20:51:15.022 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:15.022 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:15.022 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:15.022 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.022 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.022 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop(client.6231.0:2182 3.17 e136/81 3:edb5e5cd:::rbd_data.110e61479aee.0000000000000014:head v 136'791, mlcod=135'789)", 2024-09-16T20:51:15.022 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:22.735499+0000", 2024-09-16T20:51:15.022 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 52.226258072, 2024-09-16T20:51:15.022 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12369608, 2024-09-16T20:51:15.022 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:15.022 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:15.022 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:15.022 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:15.023 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.023 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:15.023 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735499+0000", 2024-09-16T20:51:15.023 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:15.023 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.023 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.023 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:15.023 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735499+0000", 2024-09-16T20:51:15.023 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:15.023 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.023 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.023 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:15.023 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735499+0000", 2024-09-16T20:51:15.024 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8099999999999999e-07 2024-09-16T20:51:15.024 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.024 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.024 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:15.024 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735506+0000", 2024-09-16T20:51:15.024 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3579999999999997e-06 2024-09-16T20:51:15.024 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.024 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.024 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:15.024 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735517+0000", 2024-09-16T20:51:15.024 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0869999999999999e-05 2024-09-16T20:51:15.024 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.024 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.025 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:15.025 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735533+0000", 2024-09-16T20:51:15.025 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5858e-05 2024-09-16T20:51:15.025 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.025 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.025 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:15.025 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735584+0000", 2024-09-16T20:51:15.025 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0470999999999998e-05 2024-09-16T20:51:15.025 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.025 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.025 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:15.026 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735607+0000", 2024-09-16T20:51:15.026 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2875999999999999e-05 2024-09-16T20:51:15.026 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.026 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.026 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:15.026 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.859144+0000", 2024-09-16T20:51:15.026 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.123537701 2024-09-16T20:51:15.026 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.026 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.026 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:15.026 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.859195+0000", 2024-09-16T20:51:15.026 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0664999999999997e-05 2024-09-16T20:51:15.026 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:15.026 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:15.026 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:15.027 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.027 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.027 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5204 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 2523136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:51:15.027 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.883660+0000", 2024-09-16T20:51:15.027 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 51.078096541999997, 2024-09-16T20:51:15.027 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052517638999999998, 2024-09-16T20:51:15.027 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:15.027 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:15.027 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:15.027 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:15.027 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:15.027 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:15.027 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5204 2024-09-16T20:51:15.027 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.028 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:15.028 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.028 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:15.028 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883660+0000", 2024-09-16T20:51:15.028 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:15.028 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.028 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.028 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:15.028 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883660+0000", 2024-09-16T20:51:15.028 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:15.028 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.028 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.028 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:15.029 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883661+0000", 2024-09-16T20:51:15.029 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9999999999999998e-07 2024-09-16T20:51:15.029 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.029 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.029 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:15.029 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883663+0000", 2024-09-16T20:51:15.029 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.4480000000000001e-06 2024-09-16T20:51:15.029 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.029 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.029 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:15.029 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883669+0000", 2024-09-16T20:51:15.029 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.071e-06 2024-09-16T20:51:15.029 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.029 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.029 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:15.030 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883675+0000", 2024-09-16T20:51:15.030 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.366e-06 2024-09-16T20:51:15.030 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.030 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.030 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:15.030 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.935970+0000", 2024-09-16T20:51:15.030 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052295346999999999 2024-09-16T20:51:15.030 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.030 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.030 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:15.030 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936064+0000", 2024-09-16T20:51:15.030 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4438000000000006e-05 2024-09-16T20:51:15.030 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.031 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.031 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:15.031 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936178+0000", 2024-09-16T20:51:15.031 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000113569 2024-09-16T20:51:15.031 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:15.031 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:15.031 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:15.031 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.031 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.031 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5205 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 2543616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:51:15.031 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.883745+0000", 2024-09-16T20:51:15.031 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 51.078011783999997, 2024-09-16T20:51:15.031 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052692112999999999, 2024-09-16T20:51:15.031 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:15.031 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:15.032 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:15.032 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:15.032 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:15.032 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:15.032 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5205 2024-09-16T20:51:15.032 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.032 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:15.032 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.032 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:15.032 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883745+0000", 2024-09-16T20:51:15.032 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:15.032 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.032 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.032 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:15.033 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883745+0000", 2024-09-16T20:51:15.033 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:15.033 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.033 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.033 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:15.033 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883746+0000", 2024-09-16T20:51:15.033 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2699999999999999e-07 2024-09-16T20:51:15.033 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.033 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.033 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:15.033 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883750+0000", 2024-09-16T20:51:15.033 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7050000000000001e-06 2024-09-16T20:51:15.033 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.033 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.034 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:15.034 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883769+0000", 2024-09-16T20:51:15.034 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9165e-05 2024-09-16T20:51:15.034 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.034 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.034 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:15.034 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883778+0000", 2024-09-16T20:51:15.034 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.6690000000000001e-06 2024-09-16T20:51:15.034 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.034 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.034 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:15.034 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936228+0000", 2024-09-16T20:51:15.034 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052449587999999998 2024-09-16T20:51:15.035 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.035 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.035 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:15.035 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936333+0000", 2024-09-16T20:51:15.035 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010511800000000001 2024-09-16T20:51:15.035 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.035 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.035 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:15.035 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936437+0000", 2024-09-16T20:51:15.035 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000104141 2024-09-16T20:51:15.035 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:15.035 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:15.035 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:15.035 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.035 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.036 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5206 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 1167360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:51:15.036 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.883802+0000", 2024-09-16T20:51:15.036 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 51.077955289999998, 2024-09-16T20:51:15.036 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052800780999999998, 2024-09-16T20:51:15.036 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:15.036 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:15.036 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:15.036 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:15.036 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:15.036 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:15.036 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5206 2024-09-16T20:51:15.036 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.036 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:15.037 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.037 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:15.037 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883802+0000", 2024-09-16T20:51:15.037 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:15.037 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.037 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.037 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:15.037 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883802+0000", 2024-09-16T20:51:15.037 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:15.037 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.037 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.037 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:15.037 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883933+0000", 2024-09-16T20:51:15.037 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013182999999999999 2024-09-16T20:51:15.038 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.038 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.038 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:15.038 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883937+0000", 2024-09-16T20:51:15.038 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.771e-06 2024-09-16T20:51:15.038 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.038 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.038 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:15.038 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883953+0000", 2024-09-16T20:51:15.038 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5409000000000001e-05 2024-09-16T20:51:15.038 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.038 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.038 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:15.038 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883959+0000", 2024-09-16T20:51:15.039 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0700000000000003e-06 2024-09-16T20:51:15.039 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.039 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.039 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:15.039 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936476+0000", 2024-09-16T20:51:15.039 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052517434000000002 2024-09-16T20:51:15.039 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.039 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.039 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:15.039 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936529+0000", 2024-09-16T20:51:15.039 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2969000000000001e-05 2024-09-16T20:51:15.039 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.039 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.039 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:15.040 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936602+0000", 2024-09-16T20:51:15.040 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3298000000000002e-05 2024-09-16T20:51:15.040 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:15.040 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:15.040 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:15.040 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.040 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.040 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5207 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 1904640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:51:15.040 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.883980+0000", 2024-09-16T20:51:15.040 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 51.077777081000001, 2024-09-16T20:51:15.040 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052759922000000001, 2024-09-16T20:51:15.040 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:15.040 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:15.040 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:15.041 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:15.041 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:15.041 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:15.041 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5207 2024-09-16T20:51:15.041 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.041 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:15.041 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.041 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:15.041 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883980+0000", 2024-09-16T20:51:15.041 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:15.041 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.041 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.041 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:15.042 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883980+0000", 2024-09-16T20:51:15.042 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:15.042 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.042 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.042 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:15.042 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884126+0000", 2024-09-16T20:51:15.042 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014577399999999999 2024-09-16T20:51:15.042 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.042 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.042 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:15.042 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884130+0000", 2024-09-16T20:51:15.042 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9729999999999999e-06 2024-09-16T20:51:15.042 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.042 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.042 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:15.043 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884139+0000", 2024-09-16T20:51:15.043 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8660000000000001e-06 2024-09-16T20:51:15.043 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.043 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.043 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:15.043 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884144+0000", 2024-09-16T20:51:15.043 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3669999999999999e-06 2024-09-16T20:51:15.043 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.043 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.043 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:15.043 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936635+0000", 2024-09-16T20:51:15.043 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052491705 2024-09-16T20:51:15.043 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.043 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.043 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:15.044 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936679+0000", 2024-09-16T20:51:15.044 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3105999999999997e-05 2024-09-16T20:51:15.044 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.044 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.044 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:15.044 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936740+0000", 2024-09-16T20:51:15.044 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1130999999999995e-05 2024-09-16T20:51:15.044 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:15.044 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:15.044 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:15.044 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.044 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.044 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5208 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 131072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:51:15.045 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.884165+0000", 2024-09-16T20:51:15.045 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 51.077591785000003, 2024-09-16T20:51:15.045 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052746603000000003, 2024-09-16T20:51:15.045 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:15.045 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:15.045 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:15.045 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:15.045 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:15.045 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:15.045 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5208 2024-09-16T20:51:15.045 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.045 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:15.045 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.045 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:15.046 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884165+0000", 2024-09-16T20:51:15.046 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:15.046 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.046 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.046 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:15.046 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884165+0000", 2024-09-16T20:51:15.046 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:15.046 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.046 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.046 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:15.046 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884463+0000", 2024-09-16T20:51:15.046 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00029801699999999998 2024-09-16T20:51:15.046 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.046 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.046 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:15.047 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884467+0000", 2024-09-16T20:51:15.047 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8210000000000003e-06 2024-09-16T20:51:15.047 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.047 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.047 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:15.047 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884477+0000", 2024-09-16T20:51:15.047 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.7030000000000001e-06 2024-09-16T20:51:15.047 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.047 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.047 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:15.047 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884485+0000", 2024-09-16T20:51:15.047 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9710000000000004e-06 2024-09-16T20:51:15.047 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.047 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.047 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:15.048 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936785+0000", 2024-09-16T20:51:15.048 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052300665000000003 2024-09-16T20:51:15.048 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.048 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.048 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:15.048 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936851+0000", 2024-09-16T20:51:15.048 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6159000000000001e-05 2024-09-16T20:51:15.048 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.048 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.048 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:15.048 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936912+0000", 2024-09-16T20:51:15.048 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0266999999999997e-05 2024-09-16T20:51:15.048 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:15.049 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:15.049 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:15.049 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.049 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.049 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5209 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 1925120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:51:15.049 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.884504+0000", 2024-09-16T20:51:15.049 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 51.077252899999998, 2024-09-16T20:51:15.049 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052575445999999998, 2024-09-16T20:51:15.049 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:15.049 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:15.049 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:15.049 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:15.049 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:15.049 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:15.050 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5209 2024-09-16T20:51:15.050 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.050 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:15.050 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.050 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:15.050 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884504+0000", 2024-09-16T20:51:15.050 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:15.050 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.050 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.050 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:15.050 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884504+0000", 2024-09-16T20:51:15.050 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:15.050 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.050 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.051 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:15.051 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884741+0000", 2024-09-16T20:51:15.051 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00023679899999999999 2024-09-16T20:51:15.051 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.051 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.051 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:15.051 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884747+0000", 2024-09-16T20:51:15.051 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0009999999999998e-06 2024-09-16T20:51:15.051 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.051 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.051 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:15.051 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884764+0000", 2024-09-16T20:51:15.051 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7371999999999999e-05 2024-09-16T20:51:15.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.052 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.052 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:15.052 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884772+0000", 2024-09-16T20:51:15.052 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1349999999999992e-06 2024-09-16T20:51:15.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.052 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.052 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:15.052 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936956+0000", 2024-09-16T20:51:15.052 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052183754999999998 2024-09-16T20:51:15.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.052 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.052 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:15.052 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937003+0000", 2024-09-16T20:51:15.053 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6925000000000002e-05 2024-09-16T20:51:15.053 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.053 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.053 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:15.053 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937079+0000", 2024-09-16T20:51:15.053 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6458999999999994e-05 2024-09-16T20:51:15.053 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:15.053 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:15.053 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:15.053 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.053 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.053 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:30174 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_remove in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e176)", 2024-09-16T20:51:15.053 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:51:03.615665+0000", 2024-09-16T20:51:15.053 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 11.346091381000001, 2024-09-16T20:51:15.054 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.96831816599999998, 2024-09-16T20:51:15.054 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:15.054 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:15.054 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:15.054 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:15.054 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:51:15.054 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:15.054 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 30174 2024-09-16T20:51:15.054 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.054 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:15.054 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.054 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:15.054 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615665+0000", 2024-09-16T20:51:15.054 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:15.055 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.055 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.055 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:15.055 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615665+0000", 2024-09-16T20:51:15.055 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:15.055 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.055 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.055 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:15.055 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615667+0000", 2024-09-16T20:51:15.055 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0020000000000001e-06 2024-09-16T20:51:15.055 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.055 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.055 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:15.056 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615675+0000", 2024-09-16T20:51:15.056 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5120000000000002e-06 2024-09-16T20:51:15.056 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.056 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.056 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:15.056 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615692+0000", 2024-09-16T20:51:15.056 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.6977e-05 2024-09-16T20:51:15.056 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.056 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.056 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:15.056 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615701+0000", 2024-09-16T20:51:15.056 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6119999999999999e-06 2024-09-16T20:51:15.056 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.056 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.057 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:15.057 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615782+0000", 2024-09-16T20:51:15.057 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1804000000000005e-05 2024-09-16T20:51:15.057 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.057 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.057 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:15.057 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615795+0000", 2024-09-16T20:51:15.057 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2464e-05 2024-09-16T20:51:15.057 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.057 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.057 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:15.057 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.580071+0000", 2024-09-16T20:51:15.057 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.96427590100000005 2024-09-16T20:51:15.057 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.058 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.058 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:15.058 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.580247+0000", 2024-09-16T20:51:15.058 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000175754 2024-09-16T20:51:15.058 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.058 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.058 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 2", 2024-09-16T20:51:15.058 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.580436+0000", 2024-09-16T20:51:15.058 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00018945299999999999 2024-09-16T20:51:15.058 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.058 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.058 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:15.058 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.581045+0000", 2024-09-16T20:51:15.058 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00060937600000000001 2024-09-16T20:51:15.059 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.059 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.059 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:15.059 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.583883+0000", 2024-09-16T20:51:15.059 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0028372900000000001 2024-09-16T20:51:15.059 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.059 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.059 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:15.059 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.583944+0000", 2024-09-16T20:51:15.059 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1848999999999997e-05 2024-09-16T20:51:15.059 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.059 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.059 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:15.060 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.583984+0000", 2024-09-16T20:51:15.060 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9171999999999997e-05 2024-09-16T20:51:15.060 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:15.060 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:15.060 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:15.060 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.060 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.060 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:30201 3.f 3:f6903a53:::rbd_object_map.110eca9b190a.000000000000001a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e177)", 2024-09-16T20:51:15.060 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:51:04.603586+0000", 2024-09-16T20:51:15.060 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 10.358171274, 2024-09-16T20:51:15.060 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.98028146800000004, 2024-09-16T20:51:15.060 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:15.060 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:15.060 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:15.061 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:15.061 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:51:15.061 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:15.061 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 30201 2024-09-16T20:51:15.061 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.061 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:15.061 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.061 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:15.061 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603586+0000", 2024-09-16T20:51:15.061 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:15.061 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.061 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.061 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:15.061 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603586+0000", 2024-09-16T20:51:15.062 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:15.062 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.062 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.062 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:15.062 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603587+0000", 2024-09-16T20:51:15.062 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.519e-06 2024-09-16T20:51:15.062 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.062 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.062 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:15.062 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603593+0000", 2024-09-16T20:51:15.062 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1449999999999996e-06 2024-09-16T20:51:15.062 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.062 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.062 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:15.063 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603606+0000", 2024-09-16T20:51:15.063 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2797e-05 2024-09-16T20:51:15.063 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.063 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.063 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:15.063 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603612+0000", 2024-09-16T20:51:15.063 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8710000000000002e-06 2024-09-16T20:51:15.063 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.063 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.063 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:15.063 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603675+0000", 2024-09-16T20:51:15.063 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2728e-05 2024-09-16T20:51:15.063 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.063 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.063 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:15.064 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603687+0000", 2024-09-16T20:51:15.064 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.253e-05 2024-09-16T20:51:15.064 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.064 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.064 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:15.064 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:05.583358+0000", 2024-09-16T20:51:15.064 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.97967074600000004 2024-09-16T20:51:15.064 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.064 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.064 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:15.064 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:05.583515+0000", 2024-09-16T20:51:15.064 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00015755700000000001 2024-09-16T20:51:15.064 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:15.064 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:15.065 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:15.065 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:05.583867+0000", 2024-09-16T20:51:15.065 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00035157500000000002 2024-09-16T20:51:15.065 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:15.065 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:15.065 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:15.065 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:15.065 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:15.065 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:15.067 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:51:15.067 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:51:15.067 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:51:15.068 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:51:15.068 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:51:15.068 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:51:15.068 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:51:15.069 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:51:15.069 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:51:15.069 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:51:15.070 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:51:15.070 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:15.065+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:15.070 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:51:15.070 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:51:15.167 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:15.166+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:15.186 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:15.187 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:15.196 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:51:15.268 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:15.267+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:15.343 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:15.343 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:15.352 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:51:15.395 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:15.394+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:15.466 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:15.466 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:15.469 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:51:15.468+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:51:15.475 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:51:15.570 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:15.569+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:15.637 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T20:51:15.638 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:15.638 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:15.648 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:51:15.671 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:51:15.670+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:51:15.772 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:15.771+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:15.802 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:15.802 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:15.812 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:51:15.874 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:15.872+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:15.965 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:15.965 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:15.974 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:51:15.975 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:15.974+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:16.075 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:51:16.074+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:51:16.081 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:16.081 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:16.090 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:51:16.177 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:16.176+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:16.236 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:16.236 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:16.245 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:51:16.288 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:51:16.287+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:51:16.352 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:16.353 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:16.362 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:51:16.378 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:16.377+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:16.479 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:16.478+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:16.509 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:16.509 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:16.517 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:51:16.570 INFO:teuthology.orchestra.run.smithi029.stderr:nodeep-scrub is unset 2024-09-16T20:51:16.579 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:16.578+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:16.633 INFO:tasks.thrashosds.thrasher:in_osds: [1, 4, 5, 0, 3] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-16T20:51:16.633 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T20:51:16.633 INFO:tasks.thrashosds.thrasher:inject_pause on osd.2 2024-09-16T20:51:16.633 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T20:51:16.633 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T20:51:16.634 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok config set filestore_inject_stall 3 2024-09-16T20:51:16.674 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:16.674 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:16.680 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:16.679+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:16.683 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:51:16.747 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:16.747 INFO:teuthology.orchestra.run.smithi029.stdout: "success": "" 2024-09-16T20:51:16.747 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:16.782 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:16.780+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:16.829 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:16.830 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:16.838 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:51:16.883 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:16.881+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:16.946 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:16.946 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:16.954 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:51:16.983 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:16.982+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:17.082 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T20:51:17.084 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:17.083+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:17.103 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:17.103 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:17.111 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:51:17.185 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:51:17.184+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:51:17.210 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:51:17.210 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:51:17.218 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:17.219 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:17.227 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:51:17.227 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:51:17.228 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:51:17.243 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:51:17.244 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:51:17.260 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:51:17.261 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:51:17.277 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:51:17.277 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:51:17.285 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:17.284+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:17.294 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:51:17.295 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:51:17.337 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:17.337 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:51:17.337 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:51:17.337 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:17.345 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:51:17.386 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:17.385+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:17.487 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:17.486+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:17.493 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:17.493 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:51:17.493 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:51:17.493 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:51:17.493 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:17.501 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:51:17.588 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:17.586+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:17.618 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:17.619 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:51:17.619 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:51:17.619 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:51:17.619 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.619 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5439 3.a 3:5789f5c3:::rbd_data.110ed9ca540.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 1181696~60416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:17.619 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:51:17.619 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 174.75224069800001, 2024-09-16T20:51:17.619 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10102045599999999, 2024-09-16T20:51:17.619 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:17.619 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:17.619 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:17.620 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:17.620 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:17.620 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:17.620 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5439 2024-09-16T20:51:17.620 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.620 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:17.620 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.620 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:17.620 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:51:17.620 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:17.620 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.620 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.621 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:17.621 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:51:17.621 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:17.621 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.621 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.621 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:17.621 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864711+0000", 2024-09-16T20:51:17.621 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7899999999999997e-07 2024-09-16T20:51:17.621 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.621 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.621 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:17.621 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864760+0000", 2024-09-16T20:51:17.622 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8803e-05 2024-09-16T20:51:17.622 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.622 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.622 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:17.622 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864769+0000", 2024-09-16T20:51:17.622 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0999999999999993e-06 2024-09-16T20:51:17.622 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.622 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.622 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:17.622 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864774+0000", 2024-09-16T20:51:17.622 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8919999999999999e-06 2024-09-16T20:51:17.622 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.622 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.623 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:17.623 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864815+0000", 2024-09-16T20:51:17.623 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1307000000000003e-05 2024-09-16T20:51:17.623 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.623 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.623 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:17.623 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864893+0000", 2024-09-16T20:51:17.623 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7581000000000001e-05 2024-09-16T20:51:17.623 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.623 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.623 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 1", 2024-09-16T20:51:17.623 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864996+0000", 2024-09-16T20:51:17.623 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102725 2024-09-16T20:51:17.623 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.623 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.624 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:17.624 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965442+0000", 2024-09-16T20:51:17.624 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100446012 2024-09-16T20:51:17.624 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.624 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.624 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:17.624 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965618+0000", 2024-09-16T20:51:17.624 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00017653100000000001 2024-09-16T20:51:17.624 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.624 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.624 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:17.624 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965682+0000", 2024-09-16T20:51:17.625 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3905999999999994e-05 2024-09-16T20:51:17.625 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.625 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.625 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:17.625 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965731+0000", 2024-09-16T20:51:17.625 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8819999999999997e-05 2024-09-16T20:51:17.625 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:17.625 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:17.625 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:17.625 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.625 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.625 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4600.0:5439 3.a e37/23 ondisk, result = 0)", 2024-09-16T20:51:17.625 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:51:17.625 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 174.75124455, 2024-09-16T20:51:17.626 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099742634999999996, 2024-09-16T20:51:17.626 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:17.626 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:17.626 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:17.626 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:17.626 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.626 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:17.626 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:51:17.626 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:17.626 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.626 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.626 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:17.626 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:51:17.627 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:17.627 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.627 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.627 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:17.627 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:51:17.627 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0900000000000001e-07 2024-09-16T20:51:17.627 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.627 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.627 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:17.627 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865711+0000", 2024-09-16T20:51:17.627 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3739999999999998e-06 2024-09-16T20:51:17.627 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.627 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.627 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:17.628 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965343+0000", 2024-09-16T20:51:17.628 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099632526999999999 2024-09-16T20:51:17.628 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.628 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.628 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:17.628 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965357+0000", 2024-09-16T20:51:17.628 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3713999999999999e-05 2024-09-16T20:51:17.628 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.628 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.628 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:17.628 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965426+0000", 2024-09-16T20:51:17.628 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8745999999999995e-05 2024-09-16T20:51:17.629 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.629 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.629 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:17.629 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965434+0000", 2024-09-16T20:51:17.629 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6910000000000003e-06 2024-09-16T20:51:17.629 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.629 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.629 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:17.629 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965449+0000", 2024-09-16T20:51:17.629 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4874e-05 2024-09-16T20:51:17.629 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:17.629 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:17.629 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:17.630 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.630 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.630 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4882.0:2508 3.12 3:4992771f:::rbd_object_map.110e7c8851c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T20:51:17.630 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:51:17.630 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 151.13544100199999, 2024-09-16T20:51:17.630 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.125007011, 2024-09-16T20:51:17.630 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:17.630 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:17.630 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:17.630 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:17.630 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4882", 2024-09-16T20:51:17.630 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2436923223", 2024-09-16T20:51:17.630 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2508 2024-09-16T20:51:17.630 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.631 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:17.631 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.631 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:17.631 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:51:17.631 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:17.631 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.631 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.631 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:17.631 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:51:17.632 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:17.632 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.632 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.632 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:17.632 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481511+0000", 2024-09-16T20:51:17.632 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0699999999999996e-07 2024-09-16T20:51:17.632 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.632 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.632 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:17.632 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481515+0000", 2024-09-16T20:51:17.632 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0609999999999997e-06 2024-09-16T20:51:17.632 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.633 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.633 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:17.633 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481523+0000", 2024-09-16T20:51:17.633 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1310000000000006e-06 2024-09-16T20:51:17.633 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.633 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.633 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:17.633 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481528+0000", 2024-09-16T20:51:17.633 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4900000000000002e-06 2024-09-16T20:51:17.633 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.633 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.633 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:17.633 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481565+0000", 2024-09-16T20:51:17.633 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7066000000000001e-05 2024-09-16T20:51:17.634 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.634 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.634 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:17.634 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481628+0000", 2024-09-16T20:51:17.634 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3118000000000001e-05 2024-09-16T20:51:17.634 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.634 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.634 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 2", 2024-09-16T20:51:17.634 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481822+0000", 2024-09-16T20:51:17.634 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019446200000000001 2024-09-16T20:51:17.634 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.635 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.635 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:17.635 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605988+0000", 2024-09-16T20:51:17.635 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.124165888 2024-09-16T20:51:17.635 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.635 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.635 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:17.635 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606427+0000", 2024-09-16T20:51:17.635 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00043854899999999999 2024-09-16T20:51:17.635 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.635 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.635 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:17.635 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606476+0000", 2024-09-16T20:51:17.636 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8971999999999998e-05 2024-09-16T20:51:17.636 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.636 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.636 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:17.636 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606517+0000", 2024-09-16T20:51:17.636 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1467000000000001e-05 2024-09-16T20:51:17.636 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:17.636 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:17.636 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:17.636 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.636 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.636 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4882.0:2508 3.12 e57/55 ondisk, result = 0)", 2024-09-16T20:51:17.636 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:51:17.637 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 151.13445250199999, 2024-09-16T20:51:17.637 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12349711400000001, 2024-09-16T20:51:17.637 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:17.637 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:17.637 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:17.637 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:17.637 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.637 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:17.637 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:51:17.637 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:17.637 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.637 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.637 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:17.638 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:51:17.638 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:17.638 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.638 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.638 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:17.638 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:51:17.638 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.06e-07 2024-09-16T20:51:17.638 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.638 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.638 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:17.638 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482502+0000", 2024-09-16T20:51:17.638 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.01e-06 2024-09-16T20:51:17.638 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.639 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.639 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:17.639 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605772+0000", 2024-09-16T20:51:17.639 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.123269687 2024-09-16T20:51:17.639 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.639 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.639 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:17.639 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605791+0000", 2024-09-16T20:51:17.639 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8912000000000001e-05 2024-09-16T20:51:17.639 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.639 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.639 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:17.639 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605974+0000", 2024-09-16T20:51:17.640 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000182836 2024-09-16T20:51:17.640 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.640 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.640 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:17.640 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605982+0000", 2024-09-16T20:51:17.640 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1699999999999997e-06 2024-09-16T20:51:17.640 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.640 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.640 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:17.640 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605996+0000", 2024-09-16T20:51:17.640 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3893000000000001e-05 2024-09-16T20:51:17.640 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:17.640 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:17.641 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:17.641 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.641 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.641 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4882.0:2512 3.f 3:f4d895b7:::rbd_data.110e7c8851c.0000000000000000:head [read 16384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T20:51:17.641 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:51:17.641 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 151.13401547199999, 2024-09-16T20:51:17.641 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12318483, 2024-09-16T20:51:17.641 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:17.641 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:17.641 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:17.641 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:17.641 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4882", 2024-09-16T20:51:17.642 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2436923223", 2024-09-16T20:51:17.642 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2512 2024-09-16T20:51:17.642 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.642 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:17.642 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.642 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:17.642 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:51:17.642 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:17.642 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.642 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.643 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:17.643 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:51:17.643 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:17.643 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.643 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.643 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:17.643 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:51:17.643 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3499999999999995e-07 2024-09-16T20:51:17.643 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.643 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.643 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:17.643 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482943+0000", 2024-09-16T20:51:17.643 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3450000000000002e-06 2024-09-16T20:51:17.644 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.644 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.644 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:17.644 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605772+0000", 2024-09-16T20:51:17.644 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.122829796 2024-09-16T20:51:17.644 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.644 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.644 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:17.644 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605792+0000", 2024-09-16T20:51:17.644 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9916999999999999e-05 2024-09-16T20:51:17.644 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.644 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.644 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:17.645 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605917+0000", 2024-09-16T20:51:17.645 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00012500799999999999 2024-09-16T20:51:17.645 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.645 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.645 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:17.645 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606004+0000", 2024-09-16T20:51:17.645 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6242999999999994e-05 2024-09-16T20:51:17.645 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.645 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.645 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:17.645 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606120+0000", 2024-09-16T20:51:17.645 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000116786 2024-09-16T20:51:17.645 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:17.646 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:17.646 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:17.646 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.646 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.646 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:11702 3.15 3:acf33c8b:::rbd_data.110e747f0779.000000000000001a:head [set-alloc-hint object_size 262144 write_size 262144,write 143360~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:51:17.646 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:51:17.646 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 133.47902305, 2024-09-16T20:51:17.646 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.084801043000000006, 2024-09-16T20:51:17.646 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:17.646 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:17.646 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:17.646 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:17.646 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:51:17.647 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:51:17.647 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 11702 2024-09-16T20:51:17.647 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.647 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:17.647 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.647 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:17.647 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:51:17.647 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:17.647 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.647 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.647 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:17.647 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:51:17.647 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:17.648 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.648 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.648 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:17.648 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137929+0000", 2024-09-16T20:51:17.648 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8400000000000002e-07 2024-09-16T20:51:17.648 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.648 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.648 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:17.648 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137937+0000", 2024-09-16T20:51:17.648 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0260000000000007e-06 2024-09-16T20:51:17.648 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.648 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.648 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:17.649 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137952+0000", 2024-09-16T20:51:17.649 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4659e-05 2024-09-16T20:51:17.649 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.649 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.649 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:17.649 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137960+0000", 2024-09-16T20:51:17.649 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0169999999999996e-06 2024-09-16T20:51:17.649 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.649 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.649 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:17.649 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138010+0000", 2024-09-16T20:51:17.649 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0021000000000001e-05 2024-09-16T20:51:17.649 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.650 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.650 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:17.650 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138114+0000", 2024-09-16T20:51:17.650 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000103797 2024-09-16T20:51:17.650 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.650 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.650 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:51:17.650 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138233+0000", 2024-09-16T20:51:17.650 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000118971 2024-09-16T20:51:17.650 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.650 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.650 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:17.650 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138953+0000", 2024-09-16T20:51:17.650 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00072026000000000002 2024-09-16T20:51:17.651 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.651 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.651 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:17.651 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222609+0000", 2024-09-16T20:51:17.651 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.083656650999999999 2024-09-16T20:51:17.651 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.651 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.651 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:17.651 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222684+0000", 2024-09-16T20:51:17.651 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4369999999999994e-05 2024-09-16T20:51:17.651 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.651 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.651 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:17.651 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222729+0000", 2024-09-16T20:51:17.652 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5287000000000002e-05 2024-09-16T20:51:17.652 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:17.652 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:17.652 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:17.652 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.652 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.652 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:11704 3.9 3:91220b75:::rbd_data.110e747f0779.000000000000001c:head [set-alloc-hint object_size 262144 write_size 262144,write 139264~6656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:51:17.652 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:51:17.652 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 133.47878504900001, 2024-09-16T20:51:17.652 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.084531133999999994, 2024-09-16T20:51:17.652 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:17.652 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:17.652 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:17.653 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:17.653 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:51:17.653 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:51:17.653 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 11704 2024-09-16T20:51:17.653 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.653 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:17.653 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.653 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:17.653 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:51:17.653 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:17.653 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.653 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.653 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:17.653 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:51:17.653 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:17.654 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.654 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.654 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:17.654 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138167+0000", 2024-09-16T20:51:17.654 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.33e-07 2024-09-16T20:51:17.654 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.654 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.654 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:17.654 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138179+0000", 2024-09-16T20:51:17.654 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2055999999999999e-05 2024-09-16T20:51:17.654 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.654 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.654 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:17.654 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138193+0000", 2024-09-16T20:51:17.655 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3961000000000001e-05 2024-09-16T20:51:17.655 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.655 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.655 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:17.655 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138200+0000", 2024-09-16T20:51:17.655 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4429999999999997e-06 2024-09-16T20:51:17.655 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.655 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.655 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:17.655 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138258+0000", 2024-09-16T20:51:17.655 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7225999999999997e-05 2024-09-16T20:51:17.655 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.655 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.655 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:17.655 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138364+0000", 2024-09-16T20:51:17.656 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000106013 2024-09-16T20:51:17.656 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.656 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.656 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:51:17.656 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138494+0000", 2024-09-16T20:51:17.656 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013059599999999999 2024-09-16T20:51:17.656 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.656 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.656 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:17.656 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.139220+0000", 2024-09-16T20:51:17.656 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00072539800000000004 2024-09-16T20:51:17.656 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.656 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.657 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:17.657 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222608+0000", 2024-09-16T20:51:17.657 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.083388620999999996 2024-09-16T20:51:17.657 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.657 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.657 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:17.657 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222651+0000", 2024-09-16T20:51:17.657 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2228000000000003e-05 2024-09-16T20:51:17.657 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.657 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.657 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:17.657 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222697+0000", 2024-09-16T20:51:17.657 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6659000000000002e-05 2024-09-16T20:51:17.658 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:17.658 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:17.658 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:17.658 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.658 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.658 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5138.0:9358 3.c 3.c25699cc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T20:51:17.658 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:51:17.658 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 110.76663774399999, 2024-09-16T20:51:17.658 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0018657580000001, 2024-09-16T20:51:17.658 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:17.658 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:17.658 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:51:17.659 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:17.659 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5138", 2024-09-16T20:51:17.659 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:51:17.659 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 9358 2024-09-16T20:51:17.659 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.659 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:17.659 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.659 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:17.659 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:51:17.659 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:17.659 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.659 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.659 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:17.659 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:51:17.660 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:17.660 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.660 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.660 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:17.660 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850314+0000", 2024-09-16T20:51:17.660 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4099999999999999e-07 2024-09-16T20:51:17.660 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.660 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.660 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:17.660 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850316+0000", 2024-09-16T20:51:17.660 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2469999999999998e-06 2024-09-16T20:51:17.660 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.660 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.661 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:17.661 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850322+0000", 2024-09-16T20:51:17.661 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3470000000000004e-06 2024-09-16T20:51:17.661 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.661 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.661 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:17.661 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850327+0000", 2024-09-16T20:51:17.661 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9450000000000001e-06 2024-09-16T20:51:17.661 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.661 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.661 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:17.661 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850376+0000", 2024-09-16T20:51:17.661 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8257000000000002e-05 2024-09-16T20:51:17.662 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.662 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.662 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:17.662 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850385+0000", 2024-09-16T20:51:17.662 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9800000000000004e-06 2024-09-16T20:51:17.662 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.662 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.662 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:17.662 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852167+0000", 2024-09-16T20:51:17.662 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0017825659999999 2024-09-16T20:51:17.662 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.662 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.662 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:17.663 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852179+0000", 2024-09-16T20:51:17.663 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1975e-05 2024-09-16T20:51:17.663 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:17.663 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:17.663 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:17.663 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.663 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.663 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5138.0:9382 3.c 3.c25699cc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T20:51:17.663 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:51:17.663 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 110.75829072800001, 2024-09-16T20:51:17.663 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.99358404700000003, 2024-09-16T20:51:17.663 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:17.663 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:17.663 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:51:17.664 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:17.664 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5138", 2024-09-16T20:51:17.664 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:51:17.664 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 9382 2024-09-16T20:51:17.664 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.664 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:17.664 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.664 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:17.664 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:51:17.664 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:17.664 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.664 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.665 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:17.665 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:51:17.665 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:17.665 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.665 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.665 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:17.665 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858661+0000", 2024-09-16T20:51:17.665 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0200000000000002e-07 2024-09-16T20:51:17.665 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.665 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.665 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:17.665 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858664+0000", 2024-09-16T20:51:17.665 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.038e-06 2024-09-16T20:51:17.666 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.666 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.666 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:17.666 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858671+0000", 2024-09-16T20:51:17.666 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0110000000000003e-06 2024-09-16T20:51:17.666 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.666 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.666 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:17.666 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858677+0000", 2024-09-16T20:51:17.666 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0959999999999997e-06 2024-09-16T20:51:17.666 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.666 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.666 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:17.666 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858732+0000", 2024-09-16T20:51:17.666 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4543000000000003e-05 2024-09-16T20:51:17.666 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.667 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.667 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:17.667 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858739+0000", 2024-09-16T20:51:17.667 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5429999999999996e-06 2024-09-16T20:51:17.667 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.667 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.667 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:17.667 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852236+0000", 2024-09-16T20:51:17.667 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.993496453 2024-09-16T20:51:17.667 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.667 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.667 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:17.667 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852244+0000", 2024-09-16T20:51:17.667 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8610000000000004e-06 2024-09-16T20:51:17.668 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:17.668 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:17.668 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:17.668 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.668 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.668 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5763.0:4895 3.a 3:5384606c:::rbd_data.110e1b19a241.0000000000000018:head [read 0~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-16T20:51:17.668 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:45.598212+0000", 2024-09-16T20:51:17.668 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 92.018738686999995, 2024-09-16T20:51:17.668 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0375864829999999, 2024-09-16T20:51:17.668 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:17.668 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:17.668 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:17.669 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:17.669 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5763", 2024-09-16T20:51:17.669 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/623897567", 2024-09-16T20:51:17.669 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4895 2024-09-16T20:51:17.669 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.669 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:17.669 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.669 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:17.669 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598212+0000", 2024-09-16T20:51:17.669 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:17.669 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.669 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.669 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:17.669 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598212+0000", 2024-09-16T20:51:17.669 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:17.670 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.670 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.670 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:17.670 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598214+0000", 2024-09-16T20:51:17.670 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0090000000000002e-06 2024-09-16T20:51:17.670 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.670 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.670 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:17.670 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598222+0000", 2024-09-16T20:51:17.670 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6699999999999994e-06 2024-09-16T20:51:17.670 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.670 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.670 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:17.670 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598241+0000", 2024-09-16T20:51:17.671 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8896999999999999e-05 2024-09-16T20:51:17.671 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.671 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.671 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:17.671 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598250+0000", 2024-09-16T20:51:17.671 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6649999999999992e-06 2024-09-16T20:51:17.671 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.671 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.671 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:17.671 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598325+0000", 2024-09-16T20:51:17.671 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5189000000000001e-05 2024-09-16T20:51:17.671 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.671 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.672 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op must wait for map", 2024-09-16T20:51:17.672 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598333+0000", 2024-09-16T20:51:17.672 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.3170000000000002e-06 2024-09-16T20:51:17.672 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.672 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.672 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:17.672 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601355+0000", 2024-09-16T20:51:17.672 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0030215400000000001 2024-09-16T20:51:17.672 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.672 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.672 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:17.672 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601365+0000", 2024-09-16T20:51:17.672 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0215e-05 2024-09-16T20:51:17.672 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.673 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.673 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:17.673 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635506+0000", 2024-09-16T20:51:17.673 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0341408240000001 2024-09-16T20:51:17.673 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.673 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.673 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:17.673 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635655+0000", 2024-09-16T20:51:17.673 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000149412 2024-09-16T20:51:17.673 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.673 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.673 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:17.673 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635799+0000", 2024-09-16T20:51:17.673 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000143745 2024-09-16T20:51:17.674 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:17.674 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:17.674 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:17.674 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.674 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.674 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5763.0:4900 3.a 3:5384606c:::rbd_data.110e1b19a241.0000000000000018:head [read 8192~24576] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-16T20:51:17.674 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:51:17.674 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 92.018680211000003, 2024-09-16T20:51:17.674 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0379340800000001, 2024-09-16T20:51:17.674 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:17.674 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:17.674 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:17.674 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:17.674 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5763", 2024-09-16T20:51:17.674 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/623897567", 2024-09-16T20:51:17.675 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4900 2024-09-16T20:51:17.675 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.675 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:17.675 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.675 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:17.675 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:51:17.675 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:17.675 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.675 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.675 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:17.675 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:51:17.675 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:17.675 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.675 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.675 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:17.676 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:51:17.676 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4300000000000003e-07 2024-09-16T20:51:17.676 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.676 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.676 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:17.676 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598274+0000", 2024-09-16T20:51:17.676 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.976e-06 2024-09-16T20:51:17.676 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.676 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.676 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:17.676 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598282+0000", 2024-09-16T20:51:17.676 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0029999999999994e-06 2024-09-16T20:51:17.676 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.676 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.677 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:17.677 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598289+0000", 2024-09-16T20:51:17.677 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2389999999999999e-06 2024-09-16T20:51:17.677 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.677 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.677 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:17.677 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598430+0000", 2024-09-16T20:51:17.677 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014119600000000001 2024-09-16T20:51:17.677 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.677 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.677 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting_for_map not empty", 2024-09-16T20:51:17.677 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598435+0000", 2024-09-16T20:51:17.677 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3489999999999999e-06 2024-09-16T20:51:17.677 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.678 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.678 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:17.678 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601429+0000", 2024-09-16T20:51:17.678 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0029942720000000001 2024-09-16T20:51:17.678 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.678 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.678 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:17.678 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601438+0000", 2024-09-16T20:51:17.678 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4759999999999998e-06 2024-09-16T20:51:17.678 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.678 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.678 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:17.678 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635987+0000", 2024-09-16T20:51:17.679 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0345495010000001 2024-09-16T20:51:17.679 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.679 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.679 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:17.679 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.636056+0000", 2024-09-16T20:51:17.679 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8610999999999998e-05 2024-09-16T20:51:17.679 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.679 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.679 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:17.679 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.636205+0000", 2024-09-16T20:51:17.679 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014891399999999999 2024-09-16T20:51:17.679 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:17.679 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:17.679 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:17.680 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.680 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.680 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop(client.6231.0:2182 3.17 e136/81 3:edb5e5cd:::rbd_data.110e61479aee.0000000000000014:head v 136'791, mlcod=135'789)", 2024-09-16T20:51:17.680 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:22.735499+0000", 2024-09-16T20:51:17.680 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 54.881452269999997, 2024-09-16T20:51:17.680 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12369608, 2024-09-16T20:51:17.680 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:17.680 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:17.680 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:17.680 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:17.680 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.680 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:17.680 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735499+0000", 2024-09-16T20:51:17.681 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:17.681 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.681 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.681 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:17.681 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735499+0000", 2024-09-16T20:51:17.681 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:17.681 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.681 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.681 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:17.681 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735499+0000", 2024-09-16T20:51:17.681 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8099999999999999e-07 2024-09-16T20:51:17.681 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.681 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.681 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:17.682 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735506+0000", 2024-09-16T20:51:17.682 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3579999999999997e-06 2024-09-16T20:51:17.682 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.682 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.682 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:17.682 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735517+0000", 2024-09-16T20:51:17.682 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0869999999999999e-05 2024-09-16T20:51:17.682 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.682 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.682 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:17.682 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735533+0000", 2024-09-16T20:51:17.682 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5858e-05 2024-09-16T20:51:17.682 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.682 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.683 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:17.683 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735584+0000", 2024-09-16T20:51:17.683 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0470999999999998e-05 2024-09-16T20:51:17.683 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.683 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.683 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:17.683 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735607+0000", 2024-09-16T20:51:17.683 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2875999999999999e-05 2024-09-16T20:51:17.683 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.683 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.683 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:17.683 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.859144+0000", 2024-09-16T20:51:17.683 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.123537701 2024-09-16T20:51:17.683 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.684 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.684 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:17.684 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.859195+0000", 2024-09-16T20:51:17.684 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0664999999999997e-05 2024-09-16T20:51:17.684 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:17.684 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:17.684 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:17.684 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.684 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.684 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5204 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 2523136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:51:17.684 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.883660+0000", 2024-09-16T20:51:17.684 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 53.733290740000001, 2024-09-16T20:51:17.684 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052517638999999998, 2024-09-16T20:51:17.684 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:17.685 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:17.685 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:17.685 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:17.685 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:17.685 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:17.685 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5204 2024-09-16T20:51:17.685 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.685 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:17.685 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.685 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:17.685 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883660+0000", 2024-09-16T20:51:17.685 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:17.685 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.685 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.686 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:17.686 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883660+0000", 2024-09-16T20:51:17.686 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:17.686 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.686 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.686 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:17.686 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883661+0000", 2024-09-16T20:51:17.686 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9999999999999998e-07 2024-09-16T20:51:17.686 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.686 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.686 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:17.686 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883663+0000", 2024-09-16T20:51:17.686 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.4480000000000001e-06 2024-09-16T20:51:17.686 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.686 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.687 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:17.687 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883669+0000", 2024-09-16T20:51:17.687 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.071e-06 2024-09-16T20:51:17.687 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.687 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.687 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:17.687 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883675+0000", 2024-09-16T20:51:17.687 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.366e-06 2024-09-16T20:51:17.687 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.687 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.687 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:17.687 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.935970+0000", 2024-09-16T20:51:17.687 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052295346999999999 2024-09-16T20:51:17.687 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.688 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.688 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:17.688 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936064+0000", 2024-09-16T20:51:17.688 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4438000000000006e-05 2024-09-16T20:51:17.688 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.688 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.688 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:17.688 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936178+0000", 2024-09-16T20:51:17.688 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000113569 2024-09-16T20:51:17.688 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:17.688 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:17.688 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:17.689 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.689 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.689 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5205 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 2543616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:51:17.689 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.883745+0000", 2024-09-16T20:51:17.689 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 53.733205982000001, 2024-09-16T20:51:17.689 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052692112999999999, 2024-09-16T20:51:17.689 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:17.689 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:17.689 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:17.689 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:17.689 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:17.689 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:17.689 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5205 2024-09-16T20:51:17.689 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.690 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:17.690 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.690 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:17.690 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883745+0000", 2024-09-16T20:51:17.690 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:17.690 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.690 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.690 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:17.690 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883745+0000", 2024-09-16T20:51:17.690 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:17.690 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.690 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.690 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:17.690 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883746+0000", 2024-09-16T20:51:17.691 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2699999999999999e-07 2024-09-16T20:51:17.691 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.691 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.691 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:17.691 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883750+0000", 2024-09-16T20:51:17.691 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7050000000000001e-06 2024-09-16T20:51:17.691 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.691 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.691 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:17.691 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883769+0000", 2024-09-16T20:51:17.691 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9165e-05 2024-09-16T20:51:17.691 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.691 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.692 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:17.692 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883778+0000", 2024-09-16T20:51:17.692 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.6690000000000001e-06 2024-09-16T20:51:17.692 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.692 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.692 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:17.692 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936228+0000", 2024-09-16T20:51:17.692 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052449587999999998 2024-09-16T20:51:17.692 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.692 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.692 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:17.692 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936333+0000", 2024-09-16T20:51:17.692 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010511800000000001 2024-09-16T20:51:17.693 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.693 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.693 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:17.693 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936437+0000", 2024-09-16T20:51:17.693 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000104141 2024-09-16T20:51:17.693 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:17.693 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:17.693 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:17.693 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.693 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.693 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5206 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 1167360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:51:17.693 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.883802+0000", 2024-09-16T20:51:17.693 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 53.733149488000002, 2024-09-16T20:51:17.693 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052800780999999998, 2024-09-16T20:51:17.693 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:17.694 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:17.694 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:17.694 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:17.694 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:17.694 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:17.694 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5206 2024-09-16T20:51:17.694 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.694 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:17.694 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.694 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:17.694 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883802+0000", 2024-09-16T20:51:17.694 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:17.694 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.694 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.695 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:17.695 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883802+0000", 2024-09-16T20:51:17.695 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:17.695 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.695 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.695 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:17.695 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883933+0000", 2024-09-16T20:51:17.695 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013182999999999999 2024-09-16T20:51:17.695 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.695 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.695 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:17.695 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883937+0000", 2024-09-16T20:51:17.695 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.771e-06 2024-09-16T20:51:17.695 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.696 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.696 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:17.696 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883953+0000", 2024-09-16T20:51:17.696 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5409000000000001e-05 2024-09-16T20:51:17.696 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.696 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.696 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:17.696 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883959+0000", 2024-09-16T20:51:17.696 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0700000000000003e-06 2024-09-16T20:51:17.696 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.696 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.696 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:17.696 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936476+0000", 2024-09-16T20:51:17.697 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052517434000000002 2024-09-16T20:51:17.697 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.697 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.697 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:17.697 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936529+0000", 2024-09-16T20:51:17.697 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2969000000000001e-05 2024-09-16T20:51:17.697 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.697 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.697 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:17.697 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936602+0000", 2024-09-16T20:51:17.697 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3298000000000002e-05 2024-09-16T20:51:17.697 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:17.697 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:17.698 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:17.698 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.698 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.698 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5207 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 1904640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:51:17.698 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.883980+0000", 2024-09-16T20:51:17.698 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 53.732971278999997, 2024-09-16T20:51:17.698 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052759922000000001, 2024-09-16T20:51:17.698 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:17.698 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:17.698 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:17.698 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:17.698 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:17.698 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:17.698 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5207 2024-09-16T20:51:17.699 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.699 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:17.699 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.699 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:17.699 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883980+0000", 2024-09-16T20:51:17.699 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:17.699 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.699 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.699 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:17.699 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883980+0000", 2024-09-16T20:51:17.699 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:17.699 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.699 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.700 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:17.700 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884126+0000", 2024-09-16T20:51:17.700 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014577399999999999 2024-09-16T20:51:17.700 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.700 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.700 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:17.700 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884130+0000", 2024-09-16T20:51:17.700 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9729999999999999e-06 2024-09-16T20:51:17.700 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.700 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.700 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:17.700 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884139+0000", 2024-09-16T20:51:17.700 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8660000000000001e-06 2024-09-16T20:51:17.700 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.701 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.701 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:17.701 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884144+0000", 2024-09-16T20:51:17.701 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3669999999999999e-06 2024-09-16T20:51:17.701 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.701 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.701 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:17.701 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936635+0000", 2024-09-16T20:51:17.701 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052491705 2024-09-16T20:51:17.701 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.701 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.701 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:17.701 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936679+0000", 2024-09-16T20:51:17.701 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3105999999999997e-05 2024-09-16T20:51:17.702 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.702 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.702 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:17.702 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936740+0000", 2024-09-16T20:51:17.702 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1130999999999995e-05 2024-09-16T20:51:17.702 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:17.702 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:17.702 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:17.702 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.702 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.702 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5208 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 131072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:51:17.702 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.884165+0000", 2024-09-16T20:51:17.702 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 53.732785982999999, 2024-09-16T20:51:17.703 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052746603000000003, 2024-09-16T20:51:17.703 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:17.703 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:17.703 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:17.703 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:17.703 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:17.703 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:17.703 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5208 2024-09-16T20:51:17.703 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.703 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:17.703 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.703 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:17.703 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884165+0000", 2024-09-16T20:51:17.703 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:17.704 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.704 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.704 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:17.704 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884165+0000", 2024-09-16T20:51:17.704 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:17.704 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.704 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.704 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:17.704 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884463+0000", 2024-09-16T20:51:17.704 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00029801699999999998 2024-09-16T20:51:17.704 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.704 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.704 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:17.705 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884467+0000", 2024-09-16T20:51:17.705 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8210000000000003e-06 2024-09-16T20:51:17.705 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.705 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.705 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:17.705 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884477+0000", 2024-09-16T20:51:17.705 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.7030000000000001e-06 2024-09-16T20:51:17.705 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.705 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.705 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:17.705 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884485+0000", 2024-09-16T20:51:17.705 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9710000000000004e-06 2024-09-16T20:51:17.705 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.705 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.706 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:17.706 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936785+0000", 2024-09-16T20:51:17.706 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052300665000000003 2024-09-16T20:51:17.706 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.706 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.706 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:17.706 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936851+0000", 2024-09-16T20:51:17.706 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6159000000000001e-05 2024-09-16T20:51:17.706 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.706 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.706 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:17.706 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936912+0000", 2024-09-16T20:51:17.706 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0266999999999997e-05 2024-09-16T20:51:17.707 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:17.707 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:17.707 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:17.707 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.707 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.707 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5209 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 1925120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:51:17.707 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.884504+0000", 2024-09-16T20:51:17.707 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 53.732447098000002, 2024-09-16T20:51:17.707 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052575445999999998, 2024-09-16T20:51:17.707 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:17.707 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:17.707 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:17.707 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:17.707 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:17.708 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:17.708 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5209 2024-09-16T20:51:17.708 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.708 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:17.708 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.708 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:17.708 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884504+0000", 2024-09-16T20:51:17.708 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:17.708 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.708 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.708 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:17.708 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884504+0000", 2024-09-16T20:51:17.708 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:17.709 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.709 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.709 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:17.709 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884741+0000", 2024-09-16T20:51:17.709 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00023679899999999999 2024-09-16T20:51:17.709 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.709 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.709 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:17.709 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884747+0000", 2024-09-16T20:51:17.709 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0009999999999998e-06 2024-09-16T20:51:17.709 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.709 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.709 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:17.709 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884764+0000", 2024-09-16T20:51:17.710 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7371999999999999e-05 2024-09-16T20:51:17.710 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.710 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.710 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:17.710 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884772+0000", 2024-09-16T20:51:17.710 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1349999999999992e-06 2024-09-16T20:51:17.710 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.710 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.710 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:17.710 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936956+0000", 2024-09-16T20:51:17.710 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052183754999999998 2024-09-16T20:51:17.710 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.710 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.710 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:17.711 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937003+0000", 2024-09-16T20:51:17.711 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6925000000000002e-05 2024-09-16T20:51:17.711 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.711 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.711 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:17.711 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937079+0000", 2024-09-16T20:51:17.711 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6458999999999994e-05 2024-09-16T20:51:17.711 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:17.711 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:17.711 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:17.711 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.711 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.711 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:30174 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_remove in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e176)", 2024-09-16T20:51:17.711 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:51:03.615665+0000", 2024-09-16T20:51:17.712 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 14.001285578999999, 2024-09-16T20:51:17.712 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.96831816599999998, 2024-09-16T20:51:17.712 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:17.712 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:17.712 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:17.712 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:17.712 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:51:17.712 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:17.712 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 30174 2024-09-16T20:51:17.712 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.712 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:17.712 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.713 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:17.713 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615665+0000", 2024-09-16T20:51:17.713 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:17.713 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.713 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.713 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:17.713 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615665+0000", 2024-09-16T20:51:17.713 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:17.713 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.713 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.713 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:17.713 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615667+0000", 2024-09-16T20:51:17.713 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0020000000000001e-06 2024-09-16T20:51:17.713 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.714 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.714 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:17.714 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615675+0000", 2024-09-16T20:51:17.714 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5120000000000002e-06 2024-09-16T20:51:17.714 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.714 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.714 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:17.714 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615692+0000", 2024-09-16T20:51:17.714 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.6977e-05 2024-09-16T20:51:17.714 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.714 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.714 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:17.714 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615701+0000", 2024-09-16T20:51:17.714 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6119999999999999e-06 2024-09-16T20:51:17.715 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.715 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.715 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:17.715 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615782+0000", 2024-09-16T20:51:17.715 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1804000000000005e-05 2024-09-16T20:51:17.715 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.715 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.715 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:17.715 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615795+0000", 2024-09-16T20:51:17.715 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2464e-05 2024-09-16T20:51:17.715 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.715 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.715 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:17.715 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.580071+0000", 2024-09-16T20:51:17.716 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.96427590100000005 2024-09-16T20:51:17.716 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.716 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.716 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:17.716 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.580247+0000", 2024-09-16T20:51:17.716 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000175754 2024-09-16T20:51:17.716 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.716 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.716 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 2", 2024-09-16T20:51:17.716 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.580436+0000", 2024-09-16T20:51:17.716 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00018945299999999999 2024-09-16T20:51:17.716 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.716 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.717 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:17.717 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.581045+0000", 2024-09-16T20:51:17.717 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00060937600000000001 2024-09-16T20:51:17.717 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.717 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.717 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:17.717 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.583883+0000", 2024-09-16T20:51:17.717 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0028372900000000001 2024-09-16T20:51:17.717 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.717 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.717 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:17.717 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.583944+0000", 2024-09-16T20:51:17.717 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1848999999999997e-05 2024-09-16T20:51:17.718 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.718 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.718 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:17.718 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.583984+0000", 2024-09-16T20:51:17.718 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9171999999999997e-05 2024-09-16T20:51:17.718 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:17.718 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:17.718 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:17.718 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.718 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.718 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:30201 3.f 3:f6903a53:::rbd_object_map.110eca9b190a.000000000000001a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e177)", 2024-09-16T20:51:17.718 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:51:04.603586+0000", 2024-09-16T20:51:17.718 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 13.013365472, 2024-09-16T20:51:17.718 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.98028146800000004, 2024-09-16T20:51:17.719 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:17.719 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:17.719 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:17.719 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:17.719 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:51:17.719 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:17.719 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 30201 2024-09-16T20:51:17.719 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.719 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:17.719 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.719 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:17.719 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603586+0000", 2024-09-16T20:51:17.719 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:17.719 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.720 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.720 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:17.720 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603586+0000", 2024-09-16T20:51:17.720 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:17.720 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.720 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.720 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:17.720 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603587+0000", 2024-09-16T20:51:17.720 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.519e-06 2024-09-16T20:51:17.720 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.720 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.720 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:17.720 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603593+0000", 2024-09-16T20:51:17.721 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1449999999999996e-06 2024-09-16T20:51:17.721 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.721 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.721 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:17.721 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603606+0000", 2024-09-16T20:51:17.721 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2797e-05 2024-09-16T20:51:17.721 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.721 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.721 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:17.721 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603612+0000", 2024-09-16T20:51:17.721 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8710000000000002e-06 2024-09-16T20:51:17.721 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.721 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.721 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:17.722 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603675+0000", 2024-09-16T20:51:17.722 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2728e-05 2024-09-16T20:51:17.722 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.722 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.722 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:17.722 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603687+0000", 2024-09-16T20:51:17.722 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.253e-05 2024-09-16T20:51:17.722 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.722 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.722 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:17.722 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:05.583358+0000", 2024-09-16T20:51:17.722 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.97967074600000004 2024-09-16T20:51:17.722 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.722 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.722 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:17.723 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:05.583515+0000", 2024-09-16T20:51:17.723 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00015755700000000001 2024-09-16T20:51:17.723 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:17.723 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:17.723 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:17.723 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:05.583867+0000", 2024-09-16T20:51:17.723 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00035157500000000002 2024-09-16T20:51:17.723 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:17.723 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:17.723 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:17.723 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:17.723 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:17.723 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:17.725 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:51:17.725 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:17.723+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:17.825 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:17.824+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:17.873 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:17.873 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:51:17.874 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:51:17.874 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:17.882 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:51:17.926 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:51:17.925+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:51:18.026 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:51:18.025+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:51:18.029 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:18.029 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:51:18.029 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:51:18.029 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:51:18.029 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:18.038 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:51:18.127 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:18.126+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:18.146 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:18.147 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:51:18.147 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:51:18.147 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:51:18.147 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.147 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:51:18.147 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:51:18.147 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 204.89088462199999, 2024-09-16T20:51:18.147 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67502346999999996, 2024-09-16T20:51:18.147 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:18.147 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:18.147 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:18.147 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:18.148 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:51:18.148 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:51:18.148 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 1 2024-09-16T20:51:18.148 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.148 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:18.148 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.148 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:18.148 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:51:18.148 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.148 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.148 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.148 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:18.148 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:51:18.148 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.149 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.149 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.149 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:18.149 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254128+0000", 2024-09-16T20:51:18.149 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.463e-06 2024-09-16T20:51:18.149 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.149 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.149 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:18.149 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254135+0000", 2024-09-16T20:51:18.149 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1579999999999999e-06 2024-09-16T20:51:18.149 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.149 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.149 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:18.149 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254146+0000", 2024-09-16T20:51:18.150 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1341e-05 2024-09-16T20:51:18.150 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.150 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.150 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:18.150 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254155+0000", 2024-09-16T20:51:18.150 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.1549999999999996e-06 2024-09-16T20:51:18.150 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.150 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.150 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:18.150 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254221+0000", 2024-09-16T20:51:18.150 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5470999999999997e-05 2024-09-16T20:51:18.150 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.150 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.150 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:18.151 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254231+0000", 2024-09-16T20:51:18.151 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0592e-05 2024-09-16T20:51:18.151 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.151 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.151 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:18.151 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926711+0000", 2024-09-16T20:51:18.151 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67247946599999997 2024-09-16T20:51:18.151 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.151 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.151 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:18.151 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926910+0000", 2024-09-16T20:51:18.151 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019934 2024-09-16T20:51:18.151 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.151 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.152 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:51:18.152 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.927159+0000", 2024-09-16T20:51:18.152 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00024876599999999998 2024-09-16T20:51:18.152 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.152 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.152 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:18.152 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.928797+0000", 2024-09-16T20:51:18.152 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001638308 2024-09-16T20:51:18.152 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.152 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.152 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:18.152 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929026+0000", 2024-09-16T20:51:18.152 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00022876099999999999 2024-09-16T20:51:18.152 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.153 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.153 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:18.153 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929102+0000", 2024-09-16T20:51:18.153 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6184000000000001e-05 2024-09-16T20:51:18.153 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.153 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.153 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:18.153 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929150+0000", 2024-09-16T20:51:18.153 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7465e-05 2024-09-16T20:51:18.153 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.153 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:18.153 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.153 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.153 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.154 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3574 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:18.154 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:51:18.154 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 148.66372109, 2024-09-16T20:51:18.154 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053579441999999998, 2024-09-16T20:51:18.154 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:18.154 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:18.154 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:18.154 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:18.154 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:18.154 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:18.154 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3574 2024-09-16T20:51:18.154 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.154 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:18.154 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.155 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:18.155 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:51:18.155 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.155 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.155 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.155 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:18.155 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:51:18.155 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.155 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.155 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.155 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:18.155 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:51:18.155 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2e-07 2024-09-16T20:51:18.155 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.156 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.156 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:18.156 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481292+0000", 2024-09-16T20:51:18.156 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2630000000000002e-06 2024-09-16T20:51:18.156 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.156 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.156 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:18.156 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481297+0000", 2024-09-16T20:51:18.156 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.296e-06 2024-09-16T20:51:18.156 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.156 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.156 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:18.156 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481300+0000", 2024-09-16T20:51:18.157 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7019999999999999e-06 2024-09-16T20:51:18.157 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.157 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.157 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:18.157 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525533+0000", 2024-09-16T20:51:18.157 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.044232318 2024-09-16T20:51:18.157 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.157 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.157 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:18.157 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525619+0000", 2024-09-16T20:51:18.157 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5892999999999999e-05 2024-09-16T20:51:18.157 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.157 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.157 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:18.158 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525987+0000", 2024-09-16T20:51:18.158 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00036836099999999999 2024-09-16T20:51:18.158 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.158 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.158 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:18.158 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531779+0000", 2024-09-16T20:51:18.158 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0057918240000000001 2024-09-16T20:51:18.158 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.158 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.158 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:18.158 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534747+0000", 2024-09-16T20:51:18.158 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0029681600000000001 2024-09-16T20:51:18.158 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.158 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.158 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:18.159 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534832+0000", 2024-09-16T20:51:18.159 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5180999999999996e-05 2024-09-16T20:51:18.159 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.159 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.159 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:18.159 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534869+0000", 2024-09-16T20:51:18.159 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7024000000000003e-05 2024-09-16T20:51:18.159 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.159 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:18.159 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.159 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.159 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.159 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3575 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:18.159 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:51:18.160 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 148.66370635000001, 2024-09-16T20:51:18.160 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053801463000000001, 2024-09-16T20:51:18.160 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:18.160 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:18.160 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:18.160 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:18.160 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:18.160 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:18.160 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3575 2024-09-16T20:51:18.160 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.160 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:18.160 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.160 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:18.161 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:51:18.161 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.161 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.161 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.161 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:18.161 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:51:18.161 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.161 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.161 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.161 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:18.161 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481305+0000", 2024-09-16T20:51:18.161 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5400000000000002e-07 2024-09-16T20:51:18.161 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.161 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.161 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:18.162 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481307+0000", 2024-09-16T20:51:18.162 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7800000000000001e-06 2024-09-16T20:51:18.162 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.162 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.162 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:18.162 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481312+0000", 2024-09-16T20:51:18.162 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6349999999999997e-06 2024-09-16T20:51:18.162 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.162 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.162 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:18.162 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481315+0000", 2024-09-16T20:51:18.162 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7829999999999999e-06 2024-09-16T20:51:18.162 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.162 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.163 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:18.163 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.526876+0000", 2024-09-16T20:51:18.163 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.045561070000000002 2024-09-16T20:51:18.163 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.163 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.163 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:18.163 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.526954+0000", 2024-09-16T20:51:18.163 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8108000000000005e-05 2024-09-16T20:51:18.163 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.163 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.163 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:18.163 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527282+0000", 2024-09-16T20:51:18.163 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000327483 2024-09-16T20:51:18.163 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.164 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.164 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:18.164 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531784+0000", 2024-09-16T20:51:18.164 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0045020169999999997 2024-09-16T20:51:18.164 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.164 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.164 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:18.164 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534921+0000", 2024-09-16T20:51:18.164 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.003137791 2024-09-16T20:51:18.164 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.164 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.164 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:18.164 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534976+0000", 2024-09-16T20:51:18.165 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4953000000000001e-05 2024-09-16T20:51:18.165 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.165 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.165 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:18.165 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535106+0000", 2024-09-16T20:51:18.165 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000129589 2024-09-16T20:51:18.165 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.165 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:18.165 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.165 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.165 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.165 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3576 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:18.165 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:51:18.165 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 148.663692438, 2024-09-16T20:51:18.166 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053955436000000002, 2024-09-16T20:51:18.166 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:18.166 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:18.166 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:18.166 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:18.166 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:18.166 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:18.166 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3576 2024-09-16T20:51:18.166 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.166 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:18.166 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.166 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:18.166 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:51:18.166 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.167 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.167 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.167 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:18.167 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:51:18.167 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.167 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.167 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.167 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:18.167 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481319+0000", 2024-09-16T20:51:18.167 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.53e-07 2024-09-16T20:51:18.167 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.167 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.167 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:18.167 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481320+0000", 2024-09-16T20:51:18.168 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5230000000000001e-06 2024-09-16T20:51:18.168 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.168 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.168 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:18.168 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481330+0000", 2024-09-16T20:51:18.168 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0190000000000001e-05 2024-09-16T20:51:18.168 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.168 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.168 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:18.168 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481333+0000", 2024-09-16T20:51:18.168 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.151e-06 2024-09-16T20:51:18.168 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.168 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.168 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:18.169 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527689+0000", 2024-09-16T20:51:18.169 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046355192000000003 2024-09-16T20:51:18.169 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.169 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.169 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:18.169 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527764+0000", 2024-09-16T20:51:18.169 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5776000000000001e-05 2024-09-16T20:51:18.169 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.169 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.169 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:18.169 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528141+0000", 2024-09-16T20:51:18.169 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00037647599999999999 2024-09-16T20:51:18.169 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.169 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.170 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:18.170 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531788+0000", 2024-09-16T20:51:18.170 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0036470679999999998 2024-09-16T20:51:18.170 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.170 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.170 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:18.170 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535174+0000", 2024-09-16T20:51:18.170 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0033862670000000001 2024-09-16T20:51:18.170 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.170 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.170 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:18.170 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535236+0000", 2024-09-16T20:51:18.170 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1797999999999993e-05 2024-09-16T20:51:18.171 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.171 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.171 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:18.171 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535274+0000", 2024-09-16T20:51:18.171 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7741999999999999e-05 2024-09-16T20:51:18.171 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.171 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:18.171 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.171 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.171 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.171 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3577 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:18.171 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:51:18.171 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 148.663670838, 2024-09-16T20:51:18.171 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054073807000000002, 2024-09-16T20:51:18.171 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:18.172 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:18.172 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:18.172 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:18.172 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:18.172 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:18.172 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3577 2024-09-16T20:51:18.172 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.172 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:18.172 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.172 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:18.172 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:51:18.172 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.172 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.173 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.173 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:18.173 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:51:18.173 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.173 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.173 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.173 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:18.173 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:51:18.173 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.96e-07 2024-09-16T20:51:18.173 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.173 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.173 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:18.173 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481342+0000", 2024-09-16T20:51:18.173 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.714e-06 2024-09-16T20:51:18.174 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.174 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.174 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:18.174 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481349+0000", 2024-09-16T20:51:18.174 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4930000000000001e-06 2024-09-16T20:51:18.174 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.174 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.174 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:18.174 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481353+0000", 2024-09-16T20:51:18.174 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4879999999999999e-06 2024-09-16T20:51:18.174 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.174 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.174 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:18.174 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528619+0000", 2024-09-16T20:51:18.175 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047266351999999998 2024-09-16T20:51:18.175 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.175 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.175 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:18.175 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528706+0000", 2024-09-16T20:51:18.175 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6796e-05 2024-09-16T20:51:18.175 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.175 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.175 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:18.175 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529148+0000", 2024-09-16T20:51:18.175 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00044182999999999999 2024-09-16T20:51:18.175 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.176 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.176 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:18.176 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531792+0000", 2024-09-16T20:51:18.176 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0026443119999999998 2024-09-16T20:51:18.176 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.176 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.176 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:18.176 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535319+0000", 2024-09-16T20:51:18.176 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0035266439999999998 2024-09-16T20:51:18.176 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.176 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.176 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:18.176 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535372+0000", 2024-09-16T20:51:18.176 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3207000000000001e-05 2024-09-16T20:51:18.177 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.177 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.177 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:18.177 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535414+0000", 2024-09-16T20:51:18.177 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1675000000000003e-05 2024-09-16T20:51:18.177 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.177 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:18.177 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.177 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.177 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.177 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3578 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:18.177 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:51:18.177 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 148.66364966899999, 2024-09-16T20:51:18.178 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054189586999999997, 2024-09-16T20:51:18.178 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:18.178 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:18.178 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:18.178 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:18.178 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:18.178 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:18.178 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3578 2024-09-16T20:51:18.178 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.178 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:18.178 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.178 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:18.178 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:51:18.178 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.179 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.179 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.179 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:18.179 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:51:18.179 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.179 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.179 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.179 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:18.179 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:51:18.179 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0800000000000001e-07 2024-09-16T20:51:18.179 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.179 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.179 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:18.179 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481363+0000", 2024-09-16T20:51:18.180 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0339999999999999e-06 2024-09-16T20:51:18.180 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.180 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.180 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:18.180 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481371+0000", 2024-09-16T20:51:18.180 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4440000000000002e-06 2024-09-16T20:51:18.180 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.180 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.180 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:18.180 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481374+0000", 2024-09-16T20:51:18.180 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2049999999999998e-06 2024-09-16T20:51:18.180 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.180 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.180 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:18.181 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529555+0000", 2024-09-16T20:51:18.181 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048181202999999999 2024-09-16T20:51:18.181 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.181 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.181 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:18.181 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529616+0000", 2024-09-16T20:51:18.181 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0912000000000002e-05 2024-09-16T20:51:18.181 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.181 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.181 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:18.181 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529939+0000", 2024-09-16T20:51:18.181 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00032269800000000001 2024-09-16T20:51:18.181 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.181 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.182 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:18.182 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532602+0000", 2024-09-16T20:51:18.182 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0026634940000000002 2024-09-16T20:51:18.182 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.182 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.182 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:18.182 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535465+0000", 2024-09-16T20:51:18.182 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.002862149 2024-09-16T20:51:18.182 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.182 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.182 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:18.182 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535516+0000", 2024-09-16T20:51:18.182 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1696000000000001e-05 2024-09-16T20:51:18.182 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.183 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.183 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:18.183 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535551+0000", 2024-09-16T20:51:18.183 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4443999999999998e-05 2024-09-16T20:51:18.183 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.183 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:18.183 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.183 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.183 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.183 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3579 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:18.183 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:51:18.183 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 148.66363122999999, 2024-09-16T20:51:18.183 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054295151, 2024-09-16T20:51:18.183 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:18.184 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:18.184 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:18.184 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:18.184 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:18.184 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:18.184 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3579 2024-09-16T20:51:18.184 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.184 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:18.184 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.184 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:18.184 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:51:18.184 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.184 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.184 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.185 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:18.185 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:51:18.185 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.185 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.185 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.185 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:18.185 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:51:18.185 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3300000000000001e-07 2024-09-16T20:51:18.185 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.185 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.185 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:18.185 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481381+0000", 2024-09-16T20:51:18.185 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.649e-06 2024-09-16T20:51:18.185 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.186 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.186 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:18.186 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481387+0000", 2024-09-16T20:51:18.186 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4519999999999996e-06 2024-09-16T20:51:18.186 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.186 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.186 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:18.186 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481390+0000", 2024-09-16T20:51:18.186 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0170000000000001e-06 2024-09-16T20:51:18.186 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.186 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.186 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:18.186 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530756+0000", 2024-09-16T20:51:18.186 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049365882999999999 2024-09-16T20:51:18.187 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.187 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.187 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:18.187 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530845+0000", 2024-09-16T20:51:18.187 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9197000000000007e-05 2024-09-16T20:51:18.187 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.187 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.187 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:18.187 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531191+0000", 2024-09-16T20:51:18.187 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00034562500000000001 2024-09-16T20:51:18.187 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.187 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.187 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:18.188 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534078+0000", 2024-09-16T20:51:18.188 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0028878179999999999 2024-09-16T20:51:18.188 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.188 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.188 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:18.188 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535589+0000", 2024-09-16T20:51:18.188 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00151085 2024-09-16T20:51:18.188 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.188 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.188 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:18.188 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535639+0000", 2024-09-16T20:51:18.188 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9697000000000002e-05 2024-09-16T20:51:18.188 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.188 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.189 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:18.189 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535675+0000", 2024-09-16T20:51:18.189 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5729999999999998e-05 2024-09-16T20:51:18.189 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.189 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:18.189 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.189 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.189 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.189 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3580 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:18.189 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:51:18.189 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 148.663617598, 2024-09-16T20:51:18.189 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054397361999999998, 2024-09-16T20:51:18.189 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:18.189 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:18.190 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:18.190 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:18.190 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:18.190 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:18.190 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3580 2024-09-16T20:51:18.190 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.190 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:18.190 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.190 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:18.190 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:51:18.190 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.190 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.190 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.190 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:18.191 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:51:18.191 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.191 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.191 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.191 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:18.191 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:51:18.191 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2600000000000001e-07 2024-09-16T20:51:18.191 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.191 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.191 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:18.191 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481395+0000", 2024-09-16T20:51:18.191 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.544e-06 2024-09-16T20:51:18.191 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.191 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.192 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:18.192 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481404+0000", 2024-09-16T20:51:18.192 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8850000000000005e-06 2024-09-16T20:51:18.192 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.192 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.192 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:18.192 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481407+0000", 2024-09-16T20:51:18.192 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1540000000000002e-06 2024-09-16T20:51:18.192 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.192 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.192 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:18.192 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531823+0000", 2024-09-16T20:51:18.192 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050415637999999999 2024-09-16T20:51:18.192 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.193 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.193 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:18.193 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531895+0000", 2024-09-16T20:51:18.193 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2179000000000001e-05 2024-09-16T20:51:18.193 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.193 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.193 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:18.193 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532236+0000", 2024-09-16T20:51:18.193 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00034073799999999999 2024-09-16T20:51:18.193 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.193 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.193 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:18.193 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534083+0000", 2024-09-16T20:51:18.193 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001847298 2024-09-16T20:51:18.194 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.194 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.194 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:18.194 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535714+0000", 2024-09-16T20:51:18.194 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0016316810000000001 2024-09-16T20:51:18.194 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.194 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.194 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:18.194 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535759+0000", 2024-09-16T20:51:18.194 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4478999999999999e-05 2024-09-16T20:51:18.194 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.194 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.194 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:18.194 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535791+0000", 2024-09-16T20:51:18.195 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.154e-05 2024-09-16T20:51:18.195 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.195 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:18.195 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.195 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.195 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.195 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3581 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:18.195 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:51:18.195 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 148.663595601, 2024-09-16T20:51:18.195 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.057094403000000002, 2024-09-16T20:51:18.195 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:18.195 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:18.195 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:18.195 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:18.195 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:18.196 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:18.196 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3581 2024-09-16T20:51:18.196 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.196 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:18.196 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.196 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:18.196 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:51:18.196 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.196 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.196 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.196 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:18.196 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:51:18.196 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.197 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.197 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.197 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:18.197 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:51:18.197 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8799999999999998e-07 2024-09-16T20:51:18.197 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.197 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.197 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:18.197 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481418+0000", 2024-09-16T20:51:18.197 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1859999999999999e-06 2024-09-16T20:51:18.197 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.197 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.197 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:18.197 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481423+0000", 2024-09-16T20:51:18.198 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8279999999999999e-06 2024-09-16T20:51:18.198 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.198 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.198 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:18.198 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481426+0000", 2024-09-16T20:51:18.198 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0110000000000001e-06 2024-09-16T20:51:18.198 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.198 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.198 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:18.198 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532631+0000", 2024-09-16T20:51:18.198 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051204514999999999 2024-09-16T20:51:18.198 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.199 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.199 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:18.199 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.533381+0000", 2024-09-16T20:51:18.199 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00075029000000000003 2024-09-16T20:51:18.199 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.199 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.199 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:18.199 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.533739+0000", 2024-09-16T20:51:18.199 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00035773100000000002 2024-09-16T20:51:18.199 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.199 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.200 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:18.200 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535123+0000", 2024-09-16T20:51:18.200 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0013836160000000001 2024-09-16T20:51:18.200 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.200 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.200 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:18.200 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538413+0000", 2024-09-16T20:51:18.200 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0032902510000000001 2024-09-16T20:51:18.200 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.200 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.200 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:18.200 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538470+0000", 2024-09-16T20:51:18.200 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6697999999999998e-05 2024-09-16T20:51:18.201 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.201 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.201 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:18.201 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538510+0000", 2024-09-16T20:51:18.201 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9988999999999999e-05 2024-09-16T20:51:18.201 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.201 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:18.201 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.201 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.201 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.201 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:12732 3.d 3:b752444a:::rbd_header.110e747f0779:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-16T20:51:18.201 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:51:18.201 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 133.376523023, 2024-09-16T20:51:18.202 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.065302467000000003, 2024-09-16T20:51:18.202 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:18.202 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:18.202 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:18.202 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:18.202 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:51:18.202 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:51:18.202 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 12732 2024-09-16T20:51:18.202 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.202 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:18.202 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.202 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:18.202 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:51:18.203 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.203 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.203 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.203 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:18.203 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:51:18.203 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.203 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.203 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.203 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:18.203 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768490+0000", 2024-09-16T20:51:18.203 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2859999999999998e-06 2024-09-16T20:51:18.203 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.203 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.204 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:18.204 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768501+0000", 2024-09-16T20:51:18.204 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1226999999999999e-05 2024-09-16T20:51:18.204 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.204 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.204 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:18.204 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768517+0000", 2024-09-16T20:51:18.204 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.592e-05 2024-09-16T20:51:18.204 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.204 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.204 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:18.204 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768528+0000", 2024-09-16T20:51:18.205 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0689999999999999e-05 2024-09-16T20:51:18.205 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.205 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.205 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:18.205 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768584+0000", 2024-09-16T20:51:18.205 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5889e-05 2024-09-16T20:51:18.205 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.205 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.205 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:18.205 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768720+0000", 2024-09-16T20:51:18.205 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000136718 2024-09-16T20:51:18.205 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.205 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.205 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:18.206 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.769066+0000", 2024-09-16T20:51:18.206 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000346013 2024-09-16T20:51:18.206 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.206 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.206 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:18.206 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833659+0000", 2024-09-16T20:51:18.206 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.064592822999999994 2024-09-16T20:51:18.206 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.206 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.206 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:18.206 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833719+0000", 2024-09-16T20:51:18.206 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9441000000000002e-05 2024-09-16T20:51:18.206 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.206 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.207 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:18.207 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833758+0000", 2024-09-16T20:51:18.207 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9122000000000003e-05 2024-09-16T20:51:18.207 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.207 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.207 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:18.207 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833790+0000", 2024-09-16T20:51:18.207 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2338000000000001e-05 2024-09-16T20:51:18.207 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.207 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:18.207 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.207 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.207 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.207 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.5089.0:12732 3.d e73/23 ondisk, result = 0)", 2024-09-16T20:51:18.208 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:51:18.208 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 133.37492085299999, 2024-09-16T20:51:18.208 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.063706456999999994, 2024-09-16T20:51:18.208 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:18.208 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:18.208 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:18.208 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:18.208 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.208 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:18.208 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:51:18.208 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.208 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.208 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.208 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:18.209 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:51:18.209 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.209 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.209 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.209 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:18.209 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770091+0000", 2024-09-16T20:51:18.209 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.53e-06 2024-09-16T20:51:18.209 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.209 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.209 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:18.209 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770104+0000", 2024-09-16T20:51:18.209 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2203e-05 2024-09-16T20:51:18.209 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.209 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.210 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:18.210 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833637+0000", 2024-09-16T20:51:18.210 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.063533201999999997 2024-09-16T20:51:18.210 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.210 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.210 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:18.210 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833655+0000", 2024-09-16T20:51:18.210 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8369e-05 2024-09-16T20:51:18.210 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.210 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.210 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:18.210 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833707+0000", 2024-09-16T20:51:18.210 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2061999999999997e-05 2024-09-16T20:51:18.210 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.211 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.211 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:18.211 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833715+0000", 2024-09-16T20:51:18.211 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5179999999999998e-06 2024-09-16T20:51:18.211 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.211 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.211 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:18.211 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833796+0000", 2024-09-16T20:51:18.211 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1573e-05 2024-09-16T20:51:18.211 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.211 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:18.211 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.211 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.211 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.211 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3286 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 245760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:18.212 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:51:18.212 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 88.103858915999993, 2024-09-16T20:51:18.212 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052523564000000002, 2024-09-16T20:51:18.212 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:18.212 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:18.212 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:18.212 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:18.212 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:51:18.212 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:18.212 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3286 2024-09-16T20:51:18.212 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.212 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:18.212 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.213 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:18.213 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:51:18.213 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.213 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.213 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.213 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:18.213 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:51:18.213 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.213 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.213 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.213 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:18.213 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:51:18.213 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3900000000000001e-07 2024-09-16T20:51:18.213 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.213 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.214 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:18.214 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041153+0000", 2024-09-16T20:51:18.214 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.119e-06 2024-09-16T20:51:18.214 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.214 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.214 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:18.214 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041156+0000", 2024-09-16T20:51:18.214 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0000000000000001e-06 2024-09-16T20:51:18.214 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.214 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.214 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:18.214 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041159+0000", 2024-09-16T20:51:18.214 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5859999999999999e-06 2024-09-16T20:51:18.214 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.215 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.215 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:18.215 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093529+0000", 2024-09-16T20:51:18.215 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052370591000000001 2024-09-16T20:51:18.215 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.215 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.215 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:18.215 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093570+0000", 2024-09-16T20:51:18.215 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0161999999999999e-05 2024-09-16T20:51:18.215 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.215 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.215 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:18.215 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093675+0000", 2024-09-16T20:51:18.215 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010586699999999999 2024-09-16T20:51:18.216 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.216 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:18.216 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.216 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.216 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.216 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3287 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 250880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:18.216 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:51:18.216 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 88.103848866999996, 2024-09-16T20:51:18.216 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052781203999999998, 2024-09-16T20:51:18.216 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:18.216 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:18.216 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:18.216 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:18.216 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:51:18.217 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:18.217 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3287 2024-09-16T20:51:18.217 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.217 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:18.217 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.217 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:18.220 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:51:18.220 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.221 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.221 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.221 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:18.221 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:51:18.221 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.221 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.221 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.221 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:18.221 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:51:18.221 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1e-07 2024-09-16T20:51:18.221 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.222 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.222 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:18.222 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041163+0000", 2024-09-16T20:51:18.222 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0690000000000001e-06 2024-09-16T20:51:18.222 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.222 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.222 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:18.222 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041166+0000", 2024-09-16T20:51:18.222 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.982e-06 2024-09-16T20:51:18.222 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.222 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.223 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:18.223 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041169+0000", 2024-09-16T20:51:18.223 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.593e-06 2024-09-16T20:51:18.223 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.223 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.223 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:18.223 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093709+0000", 2024-09-16T20:51:18.223 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052540492000000001 2024-09-16T20:51:18.223 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.223 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.223 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:18.224 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093792+0000", 2024-09-16T20:51:18.224 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2436999999999997e-05 2024-09-16T20:51:18.224 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.224 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.224 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:18.224 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093943+0000", 2024-09-16T20:51:18.224 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000151421 2024-09-16T20:51:18.224 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.224 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:18.224 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.224 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.224 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.225 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3288 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 256000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:18.225 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:51:18.225 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 88.103838994, 2024-09-16T20:51:18.225 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052928269, 2024-09-16T20:51:18.225 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:18.225 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:18.225 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:18.225 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:18.225 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:51:18.225 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:18.225 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3288 2024-09-16T20:51:18.226 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.226 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:18.226 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.226 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:18.226 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:51:18.226 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.226 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.226 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.227 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:18.227 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:51:18.227 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.227 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.227 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.227 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:18.227 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:51:18.227 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2700000000000001e-07 2024-09-16T20:51:18.227 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.227 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.227 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:18.228 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041173+0000", 2024-09-16T20:51:18.228 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.113e-06 2024-09-16T20:51:18.228 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.228 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.228 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:18.228 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041176+0000", 2024-09-16T20:51:18.228 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9799999999999998e-06 2024-09-16T20:51:18.228 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.228 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.228 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:18.228 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041179+0000", 2024-09-16T20:51:18.229 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.667e-06 2024-09-16T20:51:18.229 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.229 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.229 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:18.229 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093957+0000", 2024-09-16T20:51:18.229 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052778703000000003 2024-09-16T20:51:18.229 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.229 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.229 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:18.229 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093999+0000", 2024-09-16T20:51:18.229 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1300000000000001e-05 2024-09-16T20:51:18.230 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.230 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.230 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:18.230 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094100+0000", 2024-09-16T20:51:18.230 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101279 2024-09-16T20:51:18.230 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.230 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:18.230 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.230 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.230 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.230 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3289 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 261120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:18.231 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:51:18.231 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 88.103828997999997, 2024-09-16T20:51:18.231 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053182976, 2024-09-16T20:51:18.231 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:18.231 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:18.231 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:18.231 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:18.231 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:51:18.231 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:18.231 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3289 2024-09-16T20:51:18.231 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.232 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:18.232 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.232 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:18.232 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:51:18.232 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.232 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.232 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.232 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:18.232 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:51:18.232 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.232 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.232 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.233 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:18.233 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:51:18.233 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2999999999999999e-07 2024-09-16T20:51:18.233 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.233 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.233 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:18.233 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041183+0000", 2024-09-16T20:51:18.233 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0720000000000001e-06 2024-09-16T20:51:18.233 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.233 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.233 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:18.234 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041186+0000", 2024-09-16T20:51:18.234 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9869999999999999e-06 2024-09-16T20:51:18.234 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.234 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.234 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:18.234 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041189+0000", 2024-09-16T20:51:18.234 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.587e-06 2024-09-16T20:51:18.234 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.234 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.234 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:18.234 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094136+0000", 2024-09-16T20:51:18.235 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052947270999999997 2024-09-16T20:51:18.235 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.235 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.235 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:18.235 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094213+0000", 2024-09-16T20:51:18.235 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7552000000000006e-05 2024-09-16T20:51:18.235 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.235 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.235 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:18.235 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094365+0000", 2024-09-16T20:51:18.235 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00015127700000000001 2024-09-16T20:51:18.236 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.236 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:18.236 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.236 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.236 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.236 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:2196 3.d 3:b1fe01a0:::rbd_data.110e7ef3004.0000000000000000:2d [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e136)", 2024-09-16T20:51:18.236 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:22.739057+0000", 2024-09-16T20:51:18.236 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 55.405954199, 2024-09-16T20:51:18.236 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11999464899999999, 2024-09-16T20:51:18.236 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:18.236 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:18.237 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:18.237 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:18.237 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:18.237 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:18.237 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2196 2024-09-16T20:51:18.237 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.237 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:18.237 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.237 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:18.237 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.739057+0000", 2024-09-16T20:51:18.237 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.237 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.238 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.238 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:18.238 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.739057+0000", 2024-09-16T20:51:18.238 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.238 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.238 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.238 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:18.238 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.739057+0000", 2024-09-16T20:51:18.238 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1399999999999997e-07 2024-09-16T20:51:18.238 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.238 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.239 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:18.239 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.739060+0000", 2024-09-16T20:51:18.239 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6019999999999998e-06 2024-09-16T20:51:18.239 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.239 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.239 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:18.239 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.858755+0000", 2024-09-16T20:51:18.239 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.119695151 2024-09-16T20:51:18.239 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.239 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.239 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:18.239 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.858778+0000", 2024-09-16T20:51:18.240 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3184999999999999e-05 2024-09-16T20:51:18.240 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.240 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.240 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:18.240 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.858865+0000", 2024-09-16T20:51:18.240 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.7330000000000006e-05 2024-09-16T20:51:18.240 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.240 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.240 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:18.240 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.858952+0000", 2024-09-16T20:51:18.241 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6803000000000002e-05 2024-09-16T20:51:18.241 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.241 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.241 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:18.241 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.859051+0000", 2024-09-16T20:51:18.241 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8963999999999996e-05 2024-09-16T20:51:18.241 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.241 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:18.241 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.241 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.241 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.242 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6061.0:5401 3.13 3:c8554348:::rbd_data.110e16eafbd.0000000000000011:head [read 577536~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-16T20:51:18.242 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:27.761536+0000", 2024-09-16T20:51:18.242 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 50.383474386000003, 2024-09-16T20:51:18.242 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.003742712, 2024-09-16T20:51:18.242 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:18.242 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:18.242 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:51:18.242 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:18.242 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6061", 2024-09-16T20:51:18.242 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3930128440", 2024-09-16T20:51:18.242 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5401 2024-09-16T20:51:18.242 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.243 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:18.243 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.243 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:18.243 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761536+0000", 2024-09-16T20:51:18.243 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.243 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.243 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.243 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:18.243 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761536+0000", 2024-09-16T20:51:18.243 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.243 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.243 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.243 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:18.244 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761537+0000", 2024-09-16T20:51:18.244 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2900000000000003e-07 2024-09-16T20:51:18.244 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.244 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.244 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:18.245 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761540+0000", 2024-09-16T20:51:18.245 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2320000000000001e-06 2024-09-16T20:51:18.245 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.245 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.245 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:18.245 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761548+0000", 2024-09-16T20:51:18.245 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1289999999999996e-06 2024-09-16T20:51:18.245 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.245 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.245 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:18.245 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761554+0000", 2024-09-16T20:51:18.246 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0820000000000004e-06 2024-09-16T20:51:18.246 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.246 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.246 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:18.246 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761621+0000", 2024-09-16T20:51:18.246 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6266000000000006e-05 2024-09-16T20:51:18.246 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.246 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.246 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:18.246 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761626+0000", 2024-09-16T20:51:18.246 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.614e-06 2024-09-16T20:51:18.247 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.247 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.247 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:18.247 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:28.765163+0000", 2024-09-16T20:51:18.247 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.003536416 2024-09-16T20:51:18.247 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.247 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.247 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:18.247 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:28.765279+0000", 2024-09-16T20:51:18.247 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000116344 2024-09-16T20:51:18.248 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.248 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:18.248 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.248 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.248 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.248 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6061.0:5425 3.13 3:c8554348:::rbd_data.110e16eafbd.0000000000000011:head [read 602112~28672] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-16T20:51:18.248 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:27.767049+0000", 2024-09-16T20:51:18.248 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 50.377961726999999, 2024-09-16T20:51:18.248 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.99847897900000004, 2024-09-16T20:51:18.248 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:18.249 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:18.249 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:51:18.249 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:18.249 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6061", 2024-09-16T20:51:18.249 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3930128440", 2024-09-16T20:51:18.249 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5425 2024-09-16T20:51:18.249 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.249 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:18.249 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.249 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:18.249 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767049+0000", 2024-09-16T20:51:18.250 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.250 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.250 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.250 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:18.250 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767049+0000", 2024-09-16T20:51:18.250 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.250 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.250 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.250 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:18.250 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767050+0000", 2024-09-16T20:51:18.250 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6499999999999999e-07 2024-09-16T20:51:18.250 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.251 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.251 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:18.251 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767052+0000", 2024-09-16T20:51:18.251 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7350000000000001e-06 2024-09-16T20:51:18.251 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.251 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.251 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:18.251 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767060+0000", 2024-09-16T20:51:18.251 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5109999999999997e-06 2024-09-16T20:51:18.251 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.251 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.251 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:18.252 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767066+0000", 2024-09-16T20:51:18.252 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8139999999999997e-06 2024-09-16T20:51:18.252 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.252 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.252 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:18.252 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767128+0000", 2024-09-16T20:51:18.252 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2174999999999993e-05 2024-09-16T20:51:18.252 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.252 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.252 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:18.252 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767133+0000", 2024-09-16T20:51:18.253 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2140000000000004e-06 2024-09-16T20:51:18.253 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.253 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.253 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:18.253 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:28.765441+0000", 2024-09-16T20:51:18.253 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.99830808599999998 2024-09-16T20:51:18.253 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.253 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.253 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:18.253 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:28.765528+0000", 2024-09-16T20:51:18.254 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6879000000000006e-05 2024-09-16T20:51:18.254 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.254 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:18.254 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.254 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.254 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.254 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:25585 3.10 3:0c7c5b23:::rbd_header.110ebf0678ad:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:51:18.254 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:32.877132+0000", 2024-09-16T20:51:18.255 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 45.267879178000001, 2024-09-16T20:51:18.255 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054096699999999998, 2024-09-16T20:51:18.255 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:18.255 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:18.255 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:18.255 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:18.255 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:51:18.255 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:18.255 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 25585 2024-09-16T20:51:18.255 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.255 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:18.255 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.256 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:18.256 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.877132+0000", 2024-09-16T20:51:18.256 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.256 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.256 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.256 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:18.256 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.877132+0000", 2024-09-16T20:51:18.256 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.256 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.256 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.256 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:18.257 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931072+0000", 2024-09-16T20:51:18.257 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053939931000000003 2024-09-16T20:51:18.257 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.257 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.257 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:18.257 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931076+0000", 2024-09-16T20:51:18.257 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2860000000000001e-06 2024-09-16T20:51:18.257 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.257 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.257 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:18.257 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931085+0000", 2024-09-16T20:51:18.258 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.3370000000000006e-06 2024-09-16T20:51:18.258 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.258 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.258 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:18.258 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931092+0000", 2024-09-16T20:51:18.258 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9970000000000001e-06 2024-09-16T20:51:18.258 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.258 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.258 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:18.258 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931137+0000", 2024-09-16T20:51:18.258 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5160000000000001e-05 2024-09-16T20:51:18.259 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.259 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.259 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:18.259 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931178+0000", 2024-09-16T20:51:18.259 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0379000000000001e-05 2024-09-16T20:51:18.259 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.259 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.259 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:18.259 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931228+0000", 2024-09-16T20:51:18.259 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0609999999999998e-05 2024-09-16T20:51:18.259 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.259 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:18.259 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.260 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.260 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.260 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:26679 3.c 3:33e2f78f:::rbd_header.110e16eafbd:head [watch ping cookie 140213436013408 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e153)", 2024-09-16T20:51:18.260 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:41.440287+0000", 2024-09-16T20:51:18.260 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 36.704723303000002, 2024-09-16T20:51:18.260 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.13295633600000001, 2024-09-16T20:51:18.260 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:18.260 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:18.260 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:18.260 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:18.260 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:51:18.260 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:18.261 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 26679 2024-09-16T20:51:18.261 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.261 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:18.261 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.261 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:18.261 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440287+0000", 2024-09-16T20:51:18.261 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.261 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.261 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.261 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:18.261 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440287+0000", 2024-09-16T20:51:18.261 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.262 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.262 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.262 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:18.262 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440288+0000", 2024-09-16T20:51:18.262 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0900000000000001e-07 2024-09-16T20:51:18.262 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.262 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.262 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:18.262 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440292+0000", 2024-09-16T20:51:18.262 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3749999999999996e-06 2024-09-16T20:51:18.262 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.262 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.263 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:18.263 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440299+0000", 2024-09-16T20:51:18.263 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8249999999999999e-06 2024-09-16T20:51:18.263 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.263 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.263 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:18.263 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440307+0000", 2024-09-16T20:51:18.263 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4800000000000004e-06 2024-09-16T20:51:18.263 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.263 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.263 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:18.263 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440451+0000", 2024-09-16T20:51:18.264 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014414500000000001 2024-09-16T20:51:18.264 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.264 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.264 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:18.264 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440465+0000", 2024-09-16T20:51:18.264 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.402e-05 2024-09-16T20:51:18.264 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.264 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.264 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:18.264 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.573043+0000", 2024-09-16T20:51:18.264 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.13257855399999999 2024-09-16T20:51:18.264 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.265 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.265 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:18.265 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.573130+0000", 2024-09-16T20:51:18.265 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6168999999999994e-05 2024-09-16T20:51:18.265 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.265 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.265 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:18.265 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.573244+0000", 2024-09-16T20:51:18.265 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000114159 2024-09-16T20:51:18.265 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.265 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:18.265 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.266 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.266 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:18.266 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:18.268 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:51:18.268 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:18.266+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:18.369 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:18.368+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:18.375 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:18.376 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:51:18.376 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:51:18.376 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:18.384 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:51:18.470 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:18.469+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:18.491 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:18.491 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:51:18.491 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:51:18.491 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:51:18.491 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:18.499 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:51:18.571 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:18.570+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:18.594 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T20:51:18.614 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:18.615 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:51:18.615 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:51:18.615 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:51:18.615 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.615 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3714 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1480704~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:18.615 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:51:18.615 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 175.974086347, 2024-09-16T20:51:18.615 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050152552000000003, 2024-09-16T20:51:18.615 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:18.615 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:18.615 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:18.615 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:18.615 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:18.616 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:18.616 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3714 2024-09-16T20:51:18.616 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.616 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:18.616 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.616 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:18.616 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:51:18.616 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.616 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.616 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.616 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:18.616 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:51:18.616 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.617 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.617 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.617 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:18.617 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:51:18.617 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0499999999999999e-07 2024-09-16T20:51:18.617 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.617 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.617 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:18.617 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639007+0000", 2024-09-16T20:51:18.617 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5340000000000001e-06 2024-09-16T20:51:18.617 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.617 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.617 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:18.618 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639013+0000", 2024-09-16T20:51:18.618 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3260000000000003e-06 2024-09-16T20:51:18.618 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.618 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.618 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:18.618 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639018+0000", 2024-09-16T20:51:18.618 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2850000000000002e-06 2024-09-16T20:51:18.618 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.618 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.618 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:18.618 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689026+0000", 2024-09-16T20:51:18.618 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.05000798 2024-09-16T20:51:18.618 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.619 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.619 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:18.619 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689077+0000", 2024-09-16T20:51:18.619 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1063999999999998e-05 2024-09-16T20:51:18.619 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.619 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.619 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:18.619 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689158+0000", 2024-09-16T20:51:18.619 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1057999999999995e-05 2024-09-16T20:51:18.619 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.619 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:18.619 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.619 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.619 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.620 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3716 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1488896~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:18.620 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:51:18.620 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 175.97406853699999, 2024-09-16T20:51:18.620 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050280142, 2024-09-16T20:51:18.620 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:18.620 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:18.620 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:18.620 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:18.620 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:18.620 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:18.620 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3716 2024-09-16T20:51:18.620 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.620 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:18.620 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.621 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:18.621 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:51:18.621 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.621 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.621 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.621 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:18.621 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:51:18.621 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.621 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.621 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.621 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:18.621 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639024+0000", 2024-09-16T20:51:18.621 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9999999999999997e-07 2024-09-16T20:51:18.621 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.622 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.622 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:18.622 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639026+0000", 2024-09-16T20:51:18.622 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.781e-06 2024-09-16T20:51:18.622 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.622 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.622 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:18.622 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639032+0000", 2024-09-16T20:51:18.622 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7780000000000004e-06 2024-09-16T20:51:18.622 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.622 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.622 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:18.622 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639036+0000", 2024-09-16T20:51:18.623 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6909999999999996e-06 2024-09-16T20:51:18.623 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.623 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.623 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:18.623 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689191+0000", 2024-09-16T20:51:18.623 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050155284000000001 2024-09-16T20:51:18.623 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.623 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.623 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:18.623 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689242+0000", 2024-09-16T20:51:18.623 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0662999999999999e-05 2024-09-16T20:51:18.623 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.623 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.623 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:18.624 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689304+0000", 2024-09-16T20:51:18.624 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1345000000000003e-05 2024-09-16T20:51:18.624 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.624 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:18.624 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.624 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.624 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.624 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3719 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1497088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:18.624 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:51:18.624 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 175.97404125899999, 2024-09-16T20:51:18.624 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050399715999999997, 2024-09-16T20:51:18.625 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:18.625 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:18.625 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:18.625 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:18.625 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:18.625 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:18.625 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3719 2024-09-16T20:51:18.625 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.625 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:18.625 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.625 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:18.625 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:51:18.628 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.628 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.628 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.629 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:18.629 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:51:18.629 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.629 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.629 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.629 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:18.629 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:51:18.629 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9700000000000002e-07 2024-09-16T20:51:18.629 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.629 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.629 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:18.629 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639053+0000", 2024-09-16T20:51:18.629 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2900000000000001e-06 2024-09-16T20:51:18.630 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.630 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.630 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:18.630 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639060+0000", 2024-09-16T20:51:18.630 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7750000000000004e-06 2024-09-16T20:51:18.630 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.630 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.630 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:18.630 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639066+0000", 2024-09-16T20:51:18.630 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0639999999999999e-06 2024-09-16T20:51:18.630 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.630 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.630 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:18.631 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689337+0000", 2024-09-16T20:51:18.631 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050270784999999998 2024-09-16T20:51:18.631 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.631 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.631 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:18.631 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689392+0000", 2024-09-16T20:51:18.631 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4920999999999998e-05 2024-09-16T20:51:18.631 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.631 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.631 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:18.631 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689450+0000", 2024-09-16T20:51:18.631 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8483999999999998e-05 2024-09-16T20:51:18.631 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.632 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:18.632 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.632 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.632 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.632 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3721 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1505280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:18.632 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:51:18.632 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 175.97401812699999, 2024-09-16T20:51:18.632 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050513608000000002, 2024-09-16T20:51:18.632 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:18.632 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:18.632 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:18.632 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:18.633 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:18.633 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:18.633 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3721 2024-09-16T20:51:18.633 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.633 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:18.633 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.633 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:18.633 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:51:18.633 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.633 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.633 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.633 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:18.634 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:51:18.634 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.634 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.634 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.634 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:18.634 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:51:18.634 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7e-07 2024-09-16T20:51:18.634 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.634 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.634 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:18.634 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639076+0000", 2024-09-16T20:51:18.634 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8899999999999999e-06 2024-09-16T20:51:18.634 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.635 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.635 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:18.635 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639083+0000", 2024-09-16T20:51:18.635 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1210000000000001e-06 2024-09-16T20:51:18.635 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.635 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.635 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:18.635 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639089+0000", 2024-09-16T20:51:18.635 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.643e-06 2024-09-16T20:51:18.636 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.636 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.636 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:18.636 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689474+0000", 2024-09-16T20:51:18.636 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050385679000000003 2024-09-16T20:51:18.636 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.636 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.636 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:18.636 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689533+0000", 2024-09-16T20:51:18.636 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8303000000000001e-05 2024-09-16T20:51:18.636 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.636 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.637 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:18.637 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689587+0000", 2024-09-16T20:51:18.637 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4601999999999997e-05 2024-09-16T20:51:18.637 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.637 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:18.637 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.637 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.637 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.637 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3723 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1513472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:18.637 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:51:18.637 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 175.97398732299999, 2024-09-16T20:51:18.637 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050699691999999998, 2024-09-16T20:51:18.637 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:18.637 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:18.638 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:18.638 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:18.638 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:18.638 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:18.638 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3723 2024-09-16T20:51:18.638 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.638 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:18.638 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.638 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:18.638 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:51:18.638 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.638 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.638 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.638 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:18.639 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:51:18.639 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.639 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.639 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.639 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:18.639 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:51:18.639 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3399999999999999e-07 2024-09-16T20:51:18.639 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.639 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.639 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:18.639 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639107+0000", 2024-09-16T20:51:18.639 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3860000000000001e-06 2024-09-16T20:51:18.639 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.640 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.640 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:18.640 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639114+0000", 2024-09-16T20:51:18.640 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9800000000000001e-06 2024-09-16T20:51:18.640 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.640 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.640 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:18.640 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639129+0000", 2024-09-16T20:51:18.640 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4887e-05 2024-09-16T20:51:18.640 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.640 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.640 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:18.640 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689625+0000", 2024-09-16T20:51:18.640 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050495848000000003 2024-09-16T20:51:18.641 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.641 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.641 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:18.641 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689720+0000", 2024-09-16T20:51:18.641 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4821000000000006e-05 2024-09-16T20:51:18.641 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.641 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.641 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:18.641 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689804+0000", 2024-09-16T20:51:18.641 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4235999999999998e-05 2024-09-16T20:51:18.641 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.641 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:18.642 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.642 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.642 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.642 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3725 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1521664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:18.642 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:51:18.642 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 175.97395372700001, 2024-09-16T20:51:18.642 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050797660000000001, 2024-09-16T20:51:18.642 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:18.642 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:18.642 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:18.643 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:18.643 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:18.643 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:18.643 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3725 2024-09-16T20:51:18.643 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.643 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:18.643 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.643 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:18.643 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:51:18.643 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.643 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.643 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.643 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:18.644 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:51:18.644 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.644 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.644 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.644 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:18.644 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639139+0000", 2024-09-16T20:51:18.644 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8999999999999997e-07 2024-09-16T20:51:18.644 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.644 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.644 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:18.644 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639141+0000", 2024-09-16T20:51:18.644 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.182e-06 2024-09-16T20:51:18.644 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.645 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.645 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:18.645 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639154+0000", 2024-09-16T20:51:18.645 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3626e-05 2024-09-16T20:51:18.645 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.645 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.645 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:18.645 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639160+0000", 2024-09-16T20:51:18.645 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6810000000000003e-06 2024-09-16T20:51:18.645 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.645 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.645 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:18.645 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689831+0000", 2024-09-16T20:51:18.645 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050670907000000001 2024-09-16T20:51:18.646 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.646 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.646 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:18.646 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689882+0000", 2024-09-16T20:51:18.646 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0445999999999998e-05 2024-09-16T20:51:18.646 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.646 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.646 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:18.646 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689936+0000", 2024-09-16T20:51:18.647 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4327999999999999e-05 2024-09-16T20:51:18.647 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.647 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:18.647 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.647 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.647 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.647 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3727 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1529856~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:18.647 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:51:18.647 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 175.97391815, 2024-09-16T20:51:18.647 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051016922999999999, 2024-09-16T20:51:18.647 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:18.647 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:18.647 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:18.648 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:18.648 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:18.648 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:18.648 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3727 2024-09-16T20:51:18.648 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.648 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:18.648 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.648 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:18.648 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:51:18.648 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.648 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.648 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.648 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:18.649 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:51:18.649 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.649 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.649 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.649 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:18.649 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:51:18.649 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9999999999999998e-07 2024-09-16T20:51:18.649 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.649 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.649 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:18.649 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639183+0000", 2024-09-16T20:51:18.649 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2399999999999996e-06 2024-09-16T20:51:18.649 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.650 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.650 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:18.650 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639195+0000", 2024-09-16T20:51:18.650 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1113e-05 2024-09-16T20:51:18.650 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.650 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.650 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:18.651 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639206+0000", 2024-09-16T20:51:18.651 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1503e-05 2024-09-16T20:51:18.651 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.651 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.651 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:18.651 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689969+0000", 2024-09-16T20:51:18.651 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050762676 2024-09-16T20:51:18.651 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.651 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.651 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:18.651 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690071+0000", 2024-09-16T20:51:18.651 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102163 2024-09-16T20:51:18.651 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.652 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.652 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:18.652 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690191+0000", 2024-09-16T20:51:18.652 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000119728 2024-09-16T20:51:18.652 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.652 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:18.652 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.652 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.652 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.652 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3730 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1538048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:18.652 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:51:18.652 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 175.97387658700001, 2024-09-16T20:51:18.652 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051131467999999999, 2024-09-16T20:51:18.652 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:18.653 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:18.653 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:18.653 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:18.653 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:18.653 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:18.653 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3730 2024-09-16T20:51:18.653 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.653 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:18.653 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.653 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:18.653 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:51:18.653 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.653 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.653 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.654 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:18.654 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:51:18.654 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.654 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.654 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.654 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:18.654 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639216+0000", 2024-09-16T20:51:18.654 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.39e-07 2024-09-16T20:51:18.654 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.654 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.654 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:18.654 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639218+0000", 2024-09-16T20:51:18.654 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.537e-06 2024-09-16T20:51:18.654 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.655 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.655 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:18.655 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639229+0000", 2024-09-16T20:51:18.655 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1122e-05 2024-09-16T20:51:18.655 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.655 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.655 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:18.655 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639237+0000", 2024-09-16T20:51:18.655 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8509999999999999e-06 2024-09-16T20:51:18.655 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.655 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.655 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:18.655 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690227+0000", 2024-09-16T20:51:18.655 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050990147 2024-09-16T20:51:18.656 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.656 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.656 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:18.656 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690278+0000", 2024-09-16T20:51:18.656 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0223999999999999e-05 2024-09-16T20:51:18.656 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.656 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.656 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:18.656 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690347+0000", 2024-09-16T20:51:18.656 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9147999999999996e-05 2024-09-16T20:51:18.656 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.656 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:18.656 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.657 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.657 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.657 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3731 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1546240~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:18.657 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:51:18.657 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 175.97384587900001, 2024-09-16T20:51:18.657 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051276744999999999, 2024-09-16T20:51:18.657 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:18.657 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:18.657 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:18.657 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:18.657 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:18.657 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:18.657 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3731 2024-09-16T20:51:18.657 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.658 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:18.658 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.658 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:18.658 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:51:18.658 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.658 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.658 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.658 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:18.658 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:51:18.658 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.658 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.658 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.658 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:18.658 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:51:18.659 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9900000000000001e-07 2024-09-16T20:51:18.659 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.659 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.659 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:18.659 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639249+0000", 2024-09-16T20:51:18.659 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3630000000000001e-06 2024-09-16T20:51:18.659 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.659 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.659 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:18.659 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639255+0000", 2024-09-16T20:51:18.659 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4960000000000001e-06 2024-09-16T20:51:18.659 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.659 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.659 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:18.660 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639261+0000", 2024-09-16T20:51:18.660 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2839999999999997e-06 2024-09-16T20:51:18.660 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.660 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.660 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:18.660 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690375+0000", 2024-09-16T20:51:18.660 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051114684000000001 2024-09-16T20:51:18.660 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.660 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.660 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:18.660 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690425+0000", 2024-09-16T20:51:18.660 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9379999999999998e-05 2024-09-16T20:51:18.660 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.661 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.661 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:18.661 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690523+0000", 2024-09-16T20:51:18.661 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8139000000000003e-05 2024-09-16T20:51:18.661 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.661 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:18.661 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.661 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.661 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.661 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3732 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1554432~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:18.661 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:51:18.661 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 175.97380009700001, 2024-09-16T20:51:18.661 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051380096, 2024-09-16T20:51:18.662 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:18.662 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:18.662 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:18.662 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:18.662 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:18.662 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:18.662 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3732 2024-09-16T20:51:18.662 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.662 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:18.662 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.662 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:18.662 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:51:18.662 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.662 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.663 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.663 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:18.663 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:51:18.663 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.663 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.663 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.663 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:18.663 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:51:18.663 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6100000000000001e-07 2024-09-16T20:51:18.663 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.663 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.663 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:18.663 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639295+0000", 2024-09-16T20:51:18.663 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.447e-06 2024-09-16T20:51:18.664 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.664 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.664 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:18.664 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639302+0000", 2024-09-16T20:51:18.664 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0570000000000004e-06 2024-09-16T20:51:18.664 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.664 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.664 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:18.664 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639307+0000", 2024-09-16T20:51:18.664 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1429999999999997e-06 2024-09-16T20:51:18.664 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.664 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.664 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:18.664 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690555+0000", 2024-09-16T20:51:18.665 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051247963000000001 2024-09-16T20:51:18.665 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.665 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.665 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:18.665 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690602+0000", 2024-09-16T20:51:18.665 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7602000000000003e-05 2024-09-16T20:51:18.665 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.665 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.665 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:18.666 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690672+0000", 2024-09-16T20:51:18.666 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9423000000000003e-05 2024-09-16T20:51:18.666 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.666 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:18.666 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.666 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.666 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.666 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3734 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1562624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:18.666 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:51:18.666 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 175.973778447, 2024-09-16T20:51:18.666 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051490610999999999, 2024-09-16T20:51:18.666 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:18.666 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:18.666 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:18.666 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:18.667 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:18.667 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:18.667 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3734 2024-09-16T20:51:18.667 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.667 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:18.667 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.667 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:18.667 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:51:18.667 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.667 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.667 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.667 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:18.667 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:51:18.668 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.668 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.668 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.668 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:18.668 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639314+0000", 2024-09-16T20:51:18.668 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4400000000000001e-07 2024-09-16T20:51:18.668 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.668 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.668 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:18.668 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639316+0000", 2024-09-16T20:51:18.668 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1459999999999999e-06 2024-09-16T20:51:18.668 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.669 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.669 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:18.669 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639322+0000", 2024-09-16T20:51:18.669 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7610000000000004e-06 2024-09-16T20:51:18.669 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.669 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.669 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:18.669 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639327+0000", 2024-09-16T20:51:18.669 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.186e-06 2024-09-16T20:51:18.669 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.670 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.670 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:18.670 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690698+0000", 2024-09-16T20:51:18.670 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051371223000000001 2024-09-16T20:51:18.670 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.670 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.670 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:18.670 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690748+0000", 2024-09-16T20:51:18.670 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0192999999999999e-05 2024-09-16T20:51:18.670 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.670 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.670 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:18.670 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690804+0000", 2024-09-16T20:51:18.671 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5658000000000001e-05 2024-09-16T20:51:18.671 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.671 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:18.671 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.671 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.671 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.671 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3736 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1570816~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:18.671 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:51:18.671 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 175.973759188, 2024-09-16T20:51:18.671 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051596528000000003, 2024-09-16T20:51:18.671 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:18.671 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:18.671 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:18.672 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:18.672 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:18.672 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:18.672 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3736 2024-09-16T20:51:18.672 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.672 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:18.672 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.672 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:18.672 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:51:18.672 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.672 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.672 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.672 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:18.672 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:51:18.673 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.673 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.673 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.673 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:18.673 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:51:18.673 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9999999999999999e-07 2024-09-16T20:51:18.673 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.673 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.673 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:18.673 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639335+0000", 2024-09-16T20:51:18.674 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.635e-06 2024-09-16T20:51:18.674 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.674 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.674 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:18.674 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639340+0000", 2024-09-16T20:51:18.674 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5840000000000002e-06 2024-09-16T20:51:18.674 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.674 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.674 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:18.674 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639346+0000", 2024-09-16T20:51:18.674 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1190000000000002e-06 2024-09-16T20:51:18.674 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.674 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.674 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:18.674 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690817+0000", 2024-09-16T20:51:18.675 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051470721999999997 2024-09-16T20:51:18.675 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.675 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.675 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:18.675 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690875+0000", 2024-09-16T20:51:18.675 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7441000000000001e-05 2024-09-16T20:51:18.675 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.675 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.675 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:18.675 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690929+0000", 2024-09-16T20:51:18.675 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4727e-05 2024-09-16T20:51:18.675 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.675 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:18.675 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.676 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.676 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.676 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5434 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:18.676 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:51:18.676 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 175.748497622, 2024-09-16T20:51:18.676 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099755831000000003, 2024-09-16T20:51:18.676 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:18.676 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:18.676 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:18.676 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:18.676 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:18.676 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:18.676 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5434 2024-09-16T20:51:18.677 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.677 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:18.677 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.677 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:18.677 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:51:18.677 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.677 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.677 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.677 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:18.677 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:51:18.677 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.677 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.677 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.677 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:18.678 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864595+0000", 2024-09-16T20:51:18.678 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1099999999999995e-07 2024-09-16T20:51:18.678 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.678 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.678 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:18.678 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864597+0000", 2024-09-16T20:51:18.678 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3989999999999998e-06 2024-09-16T20:51:18.678 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.678 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.678 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:18.678 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864603+0000", 2024-09-16T20:51:18.678 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9309999999999996e-06 2024-09-16T20:51:18.678 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.679 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.679 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:18.679 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864607+0000", 2024-09-16T20:51:18.679 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6200000000000001e-06 2024-09-16T20:51:18.679 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.679 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.679 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:18.679 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964164+0000", 2024-09-16T20:51:18.679 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099557162000000005 2024-09-16T20:51:18.679 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.679 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.679 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:18.679 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964244+0000", 2024-09-16T20:51:18.679 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9597000000000004e-05 2024-09-16T20:51:18.680 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.680 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.680 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:18.680 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964350+0000", 2024-09-16T20:51:18.680 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010641099999999999 2024-09-16T20:51:18.680 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.680 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:18.680 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.680 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.680 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.680 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5435 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 20480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:18.680 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:51:18.680 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 175.74846779999999, 2024-09-16T20:51:18.680 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100897102, 2024-09-16T20:51:18.681 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:18.681 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:18.681 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:18.681 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:18.681 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:18.681 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:18.681 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5435 2024-09-16T20:51:18.681 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.681 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:18.681 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.681 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:18.681 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:51:18.681 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.682 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.682 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.682 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:18.682 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:51:18.682 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.682 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.682 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.682 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:18.682 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:51:18.682 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8000000000000001e-07 2024-09-16T20:51:18.682 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.682 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.682 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:18.682 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864626+0000", 2024-09-16T20:51:18.683 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.745e-06 2024-09-16T20:51:18.683 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.683 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.683 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:18.683 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864633+0000", 2024-09-16T20:51:18.683 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3339999999999998e-06 2024-09-16T20:51:18.683 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.683 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.683 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:18.683 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864636+0000", 2024-09-16T20:51:18.683 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1200000000000002e-06 2024-09-16T20:51:18.683 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.683 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.683 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:18.684 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965299+0000", 2024-09-16T20:51:18.684 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10066352000000001 2024-09-16T20:51:18.684 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.684 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.684 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:18.684 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965390+0000", 2024-09-16T20:51:18.684 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0894000000000001e-05 2024-09-16T20:51:18.684 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.684 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.684 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:18.684 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965521+0000", 2024-09-16T20:51:18.684 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013110899999999999 2024-09-16T20:51:18.684 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.685 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:18.685 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.685 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.685 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.685 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:152 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 327680~32768 out=32768b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:51:18.685 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:51:18.685 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 174.449970858, 2024-09-16T20:51:18.685 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11869405800000001, 2024-09-16T20:51:18.685 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:18.685 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:18.685 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:18.685 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:18.685 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:51:18.686 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:51:18.686 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 152 2024-09-16T20:51:18.686 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.686 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:18.686 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.686 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:18.686 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:51:18.686 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.686 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.686 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.686 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:18.686 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:51:18.686 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.686 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.687 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.687 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:18.687 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163122+0000", 2024-09-16T20:51:18.687 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6400000000000003e-07 2024-09-16T20:51:18.687 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.687 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.687 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:18.687 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163124+0000", 2024-09-16T20:51:18.687 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.565e-06 2024-09-16T20:51:18.687 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.687 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.687 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:18.687 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163136+0000", 2024-09-16T20:51:18.687 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1717e-05 2024-09-16T20:51:18.688 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.688 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.688 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:18.688 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163142+0000", 2024-09-16T20:51:18.688 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1750000000000002e-06 2024-09-16T20:51:18.688 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.688 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.688 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:18.688 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163606+0000", 2024-09-16T20:51:18.688 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00046418799999999998 2024-09-16T20:51:18.688 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.688 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.688 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:18.689 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163656+0000", 2024-09-16T20:51:18.689 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9852999999999999e-05 2024-09-16T20:51:18.689 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.689 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.689 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:18.689 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281815+0000", 2024-09-16T20:51:18.689 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11815909600000001 2024-09-16T20:51:18.689 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.689 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:18.689 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.689 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.689 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.689 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:153 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:51:18.689 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:51:18.690 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 174.44993790500001, 2024-09-16T20:51:18.690 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11903462400000001, 2024-09-16T20:51:18.690 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:18.690 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:18.690 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:18.690 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:18.690 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:51:18.690 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:51:18.690 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 153 2024-09-16T20:51:18.690 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.690 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:18.690 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.690 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:18.690 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:51:18.691 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.691 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.691 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.691 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:18.691 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:51:18.691 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.691 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.691 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.691 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:18.691 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163155+0000", 2024-09-16T20:51:18.691 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.37e-07 2024-09-16T20:51:18.691 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.691 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.691 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:18.692 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163157+0000", 2024-09-16T20:51:18.692 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6010000000000002e-06 2024-09-16T20:51:18.692 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.692 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.692 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:18.692 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163165+0000", 2024-09-16T20:51:18.692 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2619999999999998e-06 2024-09-16T20:51:18.692 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.692 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.692 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:18.692 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163171+0000", 2024-09-16T20:51:18.692 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.711e-06 2024-09-16T20:51:18.692 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.693 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.693 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:18.693 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281868+0000", 2024-09-16T20:51:18.693 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118696622 2024-09-16T20:51:18.693 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.693 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.693 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:18.693 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281969+0000", 2024-09-16T20:51:18.693 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101371 2024-09-16T20:51:18.693 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.693 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.693 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:18.693 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282189+0000", 2024-09-16T20:51:18.693 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00021952000000000001 2024-09-16T20:51:18.693 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.694 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:18.694 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.694 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.694 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.694 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:154 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 360448~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:51:18.694 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:51:18.694 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 174.44990613300001, 2024-09-16T20:51:18.694 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11932035100000001, 2024-09-16T20:51:18.694 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:18.694 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:18.694 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:18.694 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:18.694 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:51:18.695 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:51:18.695 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 154 2024-09-16T20:51:18.695 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.695 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:18.695 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.695 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:18.695 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:51:18.695 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.695 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.695 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.695 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:18.695 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:51:18.695 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.696 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.696 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.696 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:18.696 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:51:18.696 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:51:18.696 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.696 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.696 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:18.696 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163189+0000", 2024-09-16T20:51:18.696 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6840000000000001e-06 2024-09-16T20:51:18.696 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.696 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.696 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:18.696 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163196+0000", 2024-09-16T20:51:18.697 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7920000000000004e-06 2024-09-16T20:51:18.697 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.697 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.697 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:18.697 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163201+0000", 2024-09-16T20:51:18.697 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2809999999999999e-06 2024-09-16T20:51:18.697 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.697 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.697 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:18.697 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282219+0000", 2024-09-16T20:51:18.697 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.119018343 2024-09-16T20:51:18.697 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.697 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.697 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:18.698 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282305+0000", 2024-09-16T20:51:18.698 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5865000000000005e-05 2024-09-16T20:51:18.698 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.698 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.698 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:18.698 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282506+0000", 2024-09-16T20:51:18.698 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000200927 2024-09-16T20:51:18.698 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.698 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:18.698 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.698 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.698 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.698 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop(client.4366.0:3088 3.1e e38/23 3:79e5ec01:::rbd_header.110e891a9b2b:head v 38'306, mlcod=36'304)", 2024-09-16T20:51:18.698 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:51:18.699 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 174.448887265, 2024-09-16T20:51:18.699 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118102759, 2024-09-16T20:51:18.699 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:18.699 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:18.699 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:18.699 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:18.699 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.699 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:18.699 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:51:18.699 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.699 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.699 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.699 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:18.699 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:51:18.700 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.700 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.700 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.700 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:18.700 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:51:18.700 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.85e-07 2024-09-16T20:51:18.700 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.700 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.700 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:18.700 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164212+0000", 2024-09-16T20:51:18.700 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3999999999999997e-06 2024-09-16T20:51:18.700 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.700 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.700 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:18.701 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281553+0000", 2024-09-16T20:51:18.701 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11734135499999999 2024-09-16T20:51:18.701 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.701 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.701 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:18.701 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281567+0000", 2024-09-16T20:51:18.701 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4188999999999999e-05 2024-09-16T20:51:18.701 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.701 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.701 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:18.701 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281653+0000", 2024-09-16T20:51:18.701 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5385999999999999e-05 2024-09-16T20:51:18.701 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.702 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.702 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:18.702 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281679+0000", 2024-09-16T20:51:18.702 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6689000000000001e-05 2024-09-16T20:51:18.702 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.702 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.702 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:18.702 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282277+0000", 2024-09-16T20:51:18.702 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00059798000000000002 2024-09-16T20:51:18.702 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.702 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.702 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:18.702 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282307+0000", 2024-09-16T20:51:18.702 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9975e-05 2024-09-16T20:51:18.703 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.703 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:18.703 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.703 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.703 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.703 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5255.0:44 3.16 3:6f5476df:::rbd_header.110e25db1f60:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:51:18.703 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:51:18.703 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 102.29258421, 2024-09-16T20:51:18.703 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.121007509, 2024-09-16T20:51:18.703 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:18.703 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:18.703 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:18.703 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:18.703 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5255", 2024-09-16T20:51:18.704 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:51:18.704 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 44 2024-09-16T20:51:18.704 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.704 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:18.704 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.704 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:18.704 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:51:18.704 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.704 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.704 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.704 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:18.704 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:51:18.704 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.704 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.705 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.705 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:18.705 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:51:18.705 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0500000000000003e-07 2024-09-16T20:51:18.705 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.705 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.705 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:18.705 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320512+0000", 2024-09-16T20:51:18.705 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2540000000000001e-06 2024-09-16T20:51:18.705 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.705 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.705 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:18.705 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320518+0000", 2024-09-16T20:51:18.705 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1689999999999997e-06 2024-09-16T20:51:18.706 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.706 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.706 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:18.706 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320521+0000", 2024-09-16T20:51:18.706 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5200000000000002e-06 2024-09-16T20:51:18.706 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.706 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.706 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:18.706 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320548+0000", 2024-09-16T20:51:18.706 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6662999999999999e-05 2024-09-16T20:51:18.706 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.706 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.706 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:18.706 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320587+0000", 2024-09-16T20:51:18.707 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8866999999999999e-05 2024-09-16T20:51:18.707 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.707 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.707 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:18.707 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.441515+0000", 2024-09-16T20:51:18.707 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.120928331 2024-09-16T20:51:18.707 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.707 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:18.707 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.707 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.707 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.707 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:20329 3.5 3:a4ea64b7:::rbd_header.110ede5ef857:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:18.707 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:01.061694+0000", 2024-09-16T20:51:18.707 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 77.551397641999998, 2024-09-16T20:51:18.708 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.071234077000000007, 2024-09-16T20:51:18.708 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:18.708 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:18.708 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:18.708 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:18.708 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:51:18.708 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:18.708 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 20329 2024-09-16T20:51:18.708 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.708 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:18.708 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.708 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:18.708 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.061694+0000", 2024-09-16T20:51:18.708 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.709 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.709 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.709 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:18.709 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.061694+0000", 2024-09-16T20:51:18.709 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:18.709 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.709 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.709 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:18.709 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089232+0000", 2024-09-16T20:51:18.709 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.027537275 2024-09-16T20:51:18.709 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.709 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.709 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:18.709 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089237+0000", 2024-09-16T20:51:18.710 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4709999999999998e-06 2024-09-16T20:51:18.710 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.710 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.710 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:18.710 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089246+0000", 2024-09-16T20:51:18.710 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8230000000000001e-06 2024-09-16T20:51:18.710 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.710 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.710 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:18.710 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089252+0000", 2024-09-16T20:51:18.710 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3060000000000003e-06 2024-09-16T20:51:18.710 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.710 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.710 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:18.711 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.132756+0000", 2024-09-16T20:51:18.711 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.043504334999999998 2024-09-16T20:51:18.711 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.711 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.711 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:18.711 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.132829+0000", 2024-09-16T20:51:18.711 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2849999999999995e-05 2024-09-16T20:51:18.711 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:18.711 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:18.711 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:18.711 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.132928+0000", 2024-09-16T20:51:18.711 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.9017000000000004e-05 2024-09-16T20:51:18.711 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.711 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:18.711 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.712 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:18.712 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:18.712 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:18.713 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:51:18.714 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:18.712+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:18.814 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:18.813+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:18.826 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:18.826 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:51:18.826 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:51:18.826 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:18.835 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:51:18.915 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:18.914+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:18.943 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:18.943 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:51:18.944 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:51:18.944 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:51:18.944 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:18.953 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:51:19.016 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:19.015+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:19.062 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:19.062 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:51:19.062 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:51:19.062 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:51:19.062 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.063 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:3088 3.1e 3:79e5ec01:::rbd_header.110e891a9b2b:head [watch watch cookie 140213437171488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:51:19.063 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:51:19.063 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 174.896557887, 2024-09-16T20:51:19.063 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.11895186200000001, 2024-09-16T20:51:19.063 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:19.063 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:19.063 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:19.063 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:19.063 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:19.063 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:19.063 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3088 2024-09-16T20:51:19.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.063 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:19.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.064 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:19.064 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:51:19.064 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:19.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.064 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:19.064 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:51:19.064 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:19.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.065 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.065 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:19.065 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163846+0000", 2024-09-16T20:51:19.065 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6799999999999999e-07 2024-09-16T20:51:19.065 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.065 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.065 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:19.065 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163850+0000", 2024-09-16T20:51:19.065 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.929e-06 2024-09-16T20:51:19.065 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.065 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.065 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:19.065 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163862+0000", 2024-09-16T20:51:19.065 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1972e-05 2024-09-16T20:51:19.066 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.066 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.066 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:19.066 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163867+0000", 2024-09-16T20:51:19.066 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.451e-06 2024-09-16T20:51:19.066 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.066 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.066 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:19.066 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163912+0000", 2024-09-16T20:51:19.066 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4456000000000003e-05 2024-09-16T20:51:19.066 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.066 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.066 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:19.067 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163962+0000", 2024-09-16T20:51:19.067 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0578000000000003e-05 2024-09-16T20:51:19.067 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.067 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.067 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-16T20:51:19.067 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164058+0000", 2024-09-16T20:51:19.067 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5934999999999995e-05 2024-09-16T20:51:19.067 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.067 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.067 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:51:19.067 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164570+0000", 2024-09-16T20:51:19.067 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00051241100000000005 2024-09-16T20:51:19.068 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.068 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.068 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:19.068 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282629+0000", 2024-09-16T20:51:19.068 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.118058986 2024-09-16T20:51:19.068 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.068 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.068 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:51:19.068 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282725+0000", 2024-09-16T20:51:19.068 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5063999999999996e-05 2024-09-16T20:51:19.068 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.068 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.068 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:19.069 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282797+0000", 2024-09-16T20:51:19.069 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2212000000000006e-05 2024-09-16T20:51:19.069 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:19.069 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:19.069 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:19.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.069 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.069 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:8039 3.1e 3.ac44ff9e (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:51:19.069 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:51:19.069 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 149.352730538, 2024-09-16T20:51:19.069 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0053310449999999, 2024-09-16T20:51:19.069 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:19.069 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:19.070 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:19.070 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:19.070 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:19.070 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:19.070 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8039 2024-09-16T20:51:19.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.070 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:19.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.070 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:19.070 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:51:19.070 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:19.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.070 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:19.071 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:51:19.071 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:19.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.071 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.071 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:19.071 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707674+0000", 2024-09-16T20:51:19.071 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6530000000000001e-06 2024-09-16T20:51:19.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.071 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.071 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:19.071 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707681+0000", 2024-09-16T20:51:19.071 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9639999999999997e-06 2024-09-16T20:51:19.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.072 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.072 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:19.072 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707700+0000", 2024-09-16T20:51:19.072 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8674000000000001e-05 2024-09-16T20:51:19.072 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.072 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.072 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:19.072 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707708+0000", 2024-09-16T20:51:19.072 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6810000000000004e-06 2024-09-16T20:51:19.072 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.072 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.072 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:19.072 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711293+0000", 2024-09-16T20:51:19.073 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035848189999999999 2024-09-16T20:51:19.073 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.073 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.073 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:19.073 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711301+0000", 2024-09-16T20:51:19.073 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4730000000000003e-06 2024-09-16T20:51:19.073 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.073 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.073 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:19.073 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.712990+0000", 2024-09-16T20:51:19.073 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.001689348 2024-09-16T20:51:19.073 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.073 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.074 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:19.074 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713003+0000", 2024-09-16T20:51:19.074 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3433e-05 2024-09-16T20:51:19.074 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:19.074 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:19.074 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:19.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.074 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.074 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:8040 3.1e 3.ac44ff9e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:51:19.074 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:51:19.074 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 149.35267014300001, 2024-09-16T20:51:19.075 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0053410780000001, 2024-09-16T20:51:19.075 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:19.075 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:19.075 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:19.075 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:19.075 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:19.075 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:19.075 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8040 2024-09-16T20:51:19.075 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.075 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:19.075 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.075 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:19.075 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:51:19.075 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:19.076 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.076 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.076 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:19.076 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:51:19.076 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:19.076 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.076 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.076 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:19.076 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:51:19.076 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8100000000000003e-07 2024-09-16T20:51:19.076 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.076 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.076 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:19.076 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707737+0000", 2024-09-16T20:51:19.077 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5020000000000001e-06 2024-09-16T20:51:19.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.077 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.077 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:19.077 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707745+0000", 2024-09-16T20:51:19.077 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2169999999999994e-06 2024-09-16T20:51:19.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.077 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.077 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:19.077 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707751+0000", 2024-09-16T20:51:19.077 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.263e-06 2024-09-16T20:51:19.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.077 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.078 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:19.078 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711336+0000", 2024-09-16T20:51:19.078 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035848360000000001 2024-09-16T20:51:19.078 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.078 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.078 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:19.078 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711340+0000", 2024-09-16T20:51:19.078 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1309999999999999e-06 2024-09-16T20:51:19.078 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.078 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.078 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:19.079 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713060+0000", 2024-09-16T20:51:19.079 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0017195059999999 2024-09-16T20:51:19.079 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.079 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.079 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:19.079 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713074+0000", 2024-09-16T20:51:19.079 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4042e-05 2024-09-16T20:51:19.079 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:19.079 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:19.079 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:19.079 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.079 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.079 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4927.0:4487 3.1e 3.ac44ff9e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:51:19.080 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:51:19.080 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 149.35214994099999, 2024-09-16T20:51:19.080 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0048840880000001, 2024-09-16T20:51:19.080 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:19.080 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:19.080 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:19.080 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:19.080 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4927", 2024-09-16T20:51:19.080 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:19.080 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4487 2024-09-16T20:51:19.080 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.080 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:19.080 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.081 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:19.081 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:51:19.081 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:19.081 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.081 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.081 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:19.081 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:51:19.081 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:19.081 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.081 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.081 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:19.081 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708254+0000", 2024-09-16T20:51:19.081 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1800000000000005e-07 2024-09-16T20:51:19.082 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.082 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.082 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:19.082 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708258+0000", 2024-09-16T20:51:19.082 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8689999999999997e-06 2024-09-16T20:51:19.082 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.082 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.082 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:19.082 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708267+0000", 2024-09-16T20:51:19.082 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9590000000000007e-06 2024-09-16T20:51:19.082 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.082 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.082 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:19.082 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708274+0000", 2024-09-16T20:51:19.083 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9970000000000001e-06 2024-09-16T20:51:19.083 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.083 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.083 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:19.083 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711366+0000", 2024-09-16T20:51:19.083 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030919530000000001 2024-09-16T20:51:19.083 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.083 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.083 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:19.083 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711372+0000", 2024-09-16T20:51:19.083 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9410000000000004e-06 2024-09-16T20:51:19.083 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.084 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.084 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:19.084 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713126+0000", 2024-09-16T20:51:19.084 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0017537249999999 2024-09-16T20:51:19.084 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.084 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.084 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:19.084 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713137+0000", 2024-09-16T20:51:19.084 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0825999999999999e-05 2024-09-16T20:51:19.084 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:19.084 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:19.084 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:19.085 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.085 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.085 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5089.0:11703 3.1b 3:dcd4684e:::rbd_data.110e747f0779.000000000000001b:head [set-alloc-hint object_size 262144 write_size 262144,write 139264~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:51:19.085 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:51:19.085 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 134.922414167, 2024-09-16T20:51:19.085 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.084231385000000006, 2024-09-16T20:51:19.085 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:19.085 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:19.085 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:19.085 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:19.085 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5089", 2024-09-16T20:51:19.085 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:51:19.085 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 11703 2024-09-16T20:51:19.085 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.086 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:19.086 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.086 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:19.086 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:51:19.086 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:19.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.086 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.086 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:19.086 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:51:19.086 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:19.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.086 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.086 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:19.086 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:51:19.087 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2600000000000001e-07 2024-09-16T20:51:19.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.087 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.087 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:19.087 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137998+0000", 2024-09-16T20:51:19.087 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9549999999999998e-06 2024-09-16T20:51:19.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.087 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.087 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:19.087 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138008+0000", 2024-09-16T20:51:19.087 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3789999999999995e-06 2024-09-16T20:51:19.088 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.088 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.088 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:19.088 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138013+0000", 2024-09-16T20:51:19.088 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2209999999999996e-06 2024-09-16T20:51:19.088 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.088 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.088 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:19.088 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138054+0000", 2024-09-16T20:51:19.088 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1022999999999997e-05 2024-09-16T20:51:19.088 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.088 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.088 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:19.089 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138124+0000", 2024-09-16T20:51:19.089 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0234999999999994e-05 2024-09-16T20:51:19.089 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.089 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.089 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:19.089 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138197+0000", 2024-09-16T20:51:19.089 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2451000000000001e-05 2024-09-16T20:51:19.089 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.089 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.089 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:19.089 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222010+0000", 2024-09-16T20:51:19.089 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.083813433000000007 2024-09-16T20:51:19.089 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.089 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.090 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:51:19.090 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222163+0000", 2024-09-16T20:51:19.090 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015274599999999999 2024-09-16T20:51:19.090 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.090 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.090 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:51:19.090 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222197+0000", 2024-09-16T20:51:19.090 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3683000000000003e-05 2024-09-16T20:51:19.090 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.090 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.090 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:19.090 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222220+0000", 2024-09-16T20:51:19.090 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3433000000000001e-05 2024-09-16T20:51:19.090 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:19.091 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:19.091 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:19.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.091 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.5089.0:11702 3.15 e73/64 3:acf33c8b:::rbd_data.110e747f0779.000000000000001a:head v 73'132, mlcod=73'130)", 2024-09-16T20:51:19.091 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:51:19.091 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 134.922146838, 2024-09-16T20:51:19.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.084079763000000002, 2024-09-16T20:51:19.091 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:19.091 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:19.091 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:19.091 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:19.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:19.092 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:51:19.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:19.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.092 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.092 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:19.092 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:51:19.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:19.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.092 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.092 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:19.092 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:51:19.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:51:19.093 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.093 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.093 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:19.093 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138263+0000", 2024-09-16T20:51:19.093 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1e-06 2024-09-16T20:51:19.093 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.093 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.093 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:19.093 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221589+0000", 2024-09-16T20:51:19.093 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.083326002999999996 2024-09-16T20:51:19.094 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.094 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.094 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:19.094 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221614+0000", 2024-09-16T20:51:19.094 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5593999999999999e-05 2024-09-16T20:51:19.094 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.094 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.094 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:19.094 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221714+0000", 2024-09-16T20:51:19.094 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9889000000000005e-05 2024-09-16T20:51:19.094 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.094 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.094 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:19.094 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221727+0000", 2024-09-16T20:51:19.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3125999999999999e-05 2024-09-16T20:51:19.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.095 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:51:19.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222294+0000", 2024-09-16T20:51:19.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00056712199999999998 2024-09-16T20:51:19.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.095 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:19.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222336+0000", 2024-09-16T20:51:19.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1470000000000001e-05 2024-09-16T20:51:19.095 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:19.095 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:19.096 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:19.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.096 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.096 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5138.0:9358 3.c 3:33996a43:::rbd_data.110e6372565d.0000000000000007:head [read 1343488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T20:51:19.096 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:51:19.096 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 111.207871428, 2024-09-16T20:51:19.096 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0167493569999999, 2024-09-16T20:51:19.096 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:19.096 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:19.096 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:19.096 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:19.096 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5138", 2024-09-16T20:51:19.096 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:51:19.097 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9358 2024-09-16T20:51:19.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.097 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:19.097 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.097 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:19.097 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:51:19.097 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:19.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.097 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.097 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:19.097 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:51:19.097 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:19.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.097 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.098 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:19.098 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852533+0000", 2024-09-16T20:51:19.098 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.646e-06 2024-09-16T20:51:19.098 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.098 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.098 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:19.098 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852539+0000", 2024-09-16T20:51:19.098 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3389999999999998e-06 2024-09-16T20:51:19.098 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.098 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.098 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:19.098 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852552+0000", 2024-09-16T20:51:19.098 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.271e-05 2024-09-16T20:51:19.098 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.099 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.099 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:19.099 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852563+0000", 2024-09-16T20:51:19.099 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1139e-05 2024-09-16T20:51:19.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.099 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.099 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:19.099 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852883+0000", 2024-09-16T20:51:19.099 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00032010600000000001 2024-09-16T20:51:19.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.099 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.099 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:19.099 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852890+0000", 2024-09-16T20:51:19.100 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7669999999999998e-06 2024-09-16T20:51:19.100 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.100 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.100 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:19.100 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869059+0000", 2024-09-16T20:51:19.100 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.016169353 2024-09-16T20:51:19.100 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.100 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.100 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:19.100 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869178+0000", 2024-09-16T20:51:19.100 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118193 2024-09-16T20:51:19.100 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.101 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:19.101 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869281+0000", 2024-09-16T20:51:19.101 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010310399999999999 2024-09-16T20:51:19.101 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:19.101 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:19.102 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:19.102 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.102 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.102 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5138.0:9382 3.c 3:33996a43:::rbd_data.110e6372565d.0000000000000007:head [read 0~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T20:51:19.102 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:51:19.102 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 111.207820551, 2024-09-16T20:51:19.102 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.017136034, 2024-09-16T20:51:19.102 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:19.102 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:19.102 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:19.102 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:19.102 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5138", 2024-09-16T20:51:19.103 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:51:19.103 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9382 2024-09-16T20:51:19.103 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.103 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:19.103 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.103 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:19.103 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:51:19.103 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:19.103 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.103 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.103 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:19.104 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:51:19.104 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:19.104 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.104 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.104 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:19.104 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852583+0000", 2024-09-16T20:51:19.104 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.51e-07 2024-09-16T20:51:19.104 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.104 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.104 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:19.104 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852586+0000", 2024-09-16T20:51:19.104 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0280000000000001e-06 2024-09-16T20:51:19.104 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.104 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.105 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:19.105 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852595+0000", 2024-09-16T20:51:19.105 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2799999999999992e-06 2024-09-16T20:51:19.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.105 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:19.105 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852601+0000", 2024-09-16T20:51:19.105 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3810000000000004e-06 2024-09-16T20:51:19.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.105 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:19.105 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852921+0000", 2024-09-16T20:51:19.105 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00031979699999999997 2024-09-16T20:51:19.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.106 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:19.106 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852930+0000", 2024-09-16T20:51:19.106 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6489999999999997e-06 2024-09-16T20:51:19.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.106 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:19.106 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869434+0000", 2024-09-16T20:51:19.106 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0165038829999999 2024-09-16T20:51:19.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.106 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:19.106 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869495+0000", 2024-09-16T20:51:19.106 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1509999999999999e-05 2024-09-16T20:51:19.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.107 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:19.107 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869718+0000", 2024-09-16T20:51:19.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000222955 2024-09-16T20:51:19.107 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:19.107 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:19.107 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:19.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.107 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9969 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 632832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:19.108 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642045+0000", 2024-09-16T20:51:19.108 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 88.418358093999998, 2024-09-16T20:51:19.108 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076903192999999995, 2024-09-16T20:51:19.108 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:19.108 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:19.108 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:19.108 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:19.108 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:19.108 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:19.108 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9969 2024-09-16T20:51:19.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.108 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:19.109 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.109 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:19.109 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642045+0000", 2024-09-16T20:51:19.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:19.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.109 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.109 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:19.109 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642045+0000", 2024-09-16T20:51:19.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:19.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.109 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.109 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:19.109 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642045+0000", 2024-09-16T20:51:19.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5100000000000001e-07 2024-09-16T20:51:19.110 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.110 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.110 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:19.110 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642047+0000", 2024-09-16T20:51:19.110 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8679999999999999e-06 2024-09-16T20:51:19.110 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.110 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.110 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:19.110 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642053+0000", 2024-09-16T20:51:19.110 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1720000000000004e-06 2024-09-16T20:51:19.110 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.110 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.110 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:19.111 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642060+0000", 2024-09-16T20:51:19.111 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0339999999999999e-06 2024-09-16T20:51:19.111 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.111 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.111 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:19.111 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.718562+0000", 2024-09-16T20:51:19.111 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076501969000000003 2024-09-16T20:51:19.111 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.111 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.111 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:19.111 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.718643+0000", 2024-09-16T20:51:19.111 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0539999999999998e-05 2024-09-16T20:51:19.111 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.111 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:19.112 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.718948+0000", 2024-09-16T20:51:19.112 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000305259 2024-09-16T20:51:19.112 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:19.112 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:19.112 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:19.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.112 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.112 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9970 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 637952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:19.112 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642067+0000", 2024-09-16T20:51:19.112 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 88.418336113999999, 2024-09-16T20:51:19.112 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077174928000000004, 2024-09-16T20:51:19.112 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:19.113 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:19.113 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:19.113 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:19.113 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:19.113 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:19.113 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9970 2024-09-16T20:51:19.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.113 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:19.113 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.113 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:19.113 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642067+0000", 2024-09-16T20:51:19.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:19.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.113 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.114 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:19.114 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642067+0000", 2024-09-16T20:51:19.114 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:19.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.114 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.114 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:19.114 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642067+0000", 2024-09-16T20:51:19.114 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2499999999999995e-07 2024-09-16T20:51:19.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.114 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.114 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:19.114 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642070+0000", 2024-09-16T20:51:19.114 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3199999999999998e-06 2024-09-16T20:51:19.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.115 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.115 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:19.115 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642076+0000", 2024-09-16T20:51:19.115 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.122e-06 2024-09-16T20:51:19.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.115 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.115 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:19.115 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642081+0000", 2024-09-16T20:51:19.115 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0869999999999997e-06 2024-09-16T20:51:19.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.115 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.115 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:19.115 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.718980+0000", 2024-09-16T20:51:19.116 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076899482000000005 2024-09-16T20:51:19.116 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.116 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.116 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:19.116 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719085+0000", 2024-09-16T20:51:19.116 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010446399999999999 2024-09-16T20:51:19.116 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.116 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.116 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:19.116 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719242+0000", 2024-09-16T20:51:19.116 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000156928 2024-09-16T20:51:19.116 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:19.116 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:19.117 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:19.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.117 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9971 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 643072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:19.117 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642102+0000", 2024-09-16T20:51:19.117 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 88.418300407000004, 2024-09-16T20:51:19.117 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077421016999999995, 2024-09-16T20:51:19.117 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:19.117 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:19.117 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:19.118 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:19.118 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:19.118 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:19.118 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9971 2024-09-16T20:51:19.118 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.118 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:19.118 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.118 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:19.118 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642102+0000", 2024-09-16T20:51:19.118 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:19.118 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.118 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.118 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:19.118 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642102+0000", 2024-09-16T20:51:19.119 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:19.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.119 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.119 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:19.119 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642103+0000", 2024-09-16T20:51:19.119 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7399999999999998e-07 2024-09-16T20:51:19.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.119 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.119 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:19.119 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642105+0000", 2024-09-16T20:51:19.119 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2649999999999999e-06 2024-09-16T20:51:19.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.119 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.119 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:19.120 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642112+0000", 2024-09-16T20:51:19.120 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.294e-06 2024-09-16T20:51:19.120 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.120 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.120 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:19.120 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642118+0000", 2024-09-16T20:51:19.120 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4500000000000003e-06 2024-09-16T20:51:19.120 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.120 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.120 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:19.120 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719285+0000", 2024-09-16T20:51:19.120 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077166883000000006 2024-09-16T20:51:19.120 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.120 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.121 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:19.121 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719368+0000", 2024-09-16T20:51:19.121 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2811000000000004e-05 2024-09-16T20:51:19.121 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.121 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.121 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:19.121 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719523+0000", 2024-09-16T20:51:19.121 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015584 2024-09-16T20:51:19.121 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:19.121 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:19.121 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:19.121 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.121 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.121 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9972 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 648192~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:19.122 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642125+0000", 2024-09-16T20:51:19.122 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 88.418277459999999, 2024-09-16T20:51:19.122 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077710324999999997, 2024-09-16T20:51:19.122 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:19.122 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:19.122 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:19.122 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:19.122 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:19.122 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:19.122 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9972 2024-09-16T20:51:19.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.122 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:19.122 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.123 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:19.123 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642125+0000", 2024-09-16T20:51:19.123 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:19.123 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.123 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.123 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:19.123 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642125+0000", 2024-09-16T20:51:19.123 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:19.123 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.123 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.123 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:19.123 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642126+0000", 2024-09-16T20:51:19.124 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1899999999999998e-07 2024-09-16T20:51:19.124 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.124 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.124 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:19.124 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642128+0000", 2024-09-16T20:51:19.124 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8619999999999999e-06 2024-09-16T20:51:19.124 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.124 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.124 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:19.124 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642134+0000", 2024-09-16T20:51:19.124 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5289999999999997e-06 2024-09-16T20:51:19.124 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.124 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.125 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:19.125 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642139+0000", 2024-09-16T20:51:19.125 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1250000000000001e-06 2024-09-16T20:51:19.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.125 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.125 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:19.125 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719570+0000", 2024-09-16T20:51:19.125 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077430516000000005 2024-09-16T20:51:19.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.125 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.125 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:19.125 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719655+0000", 2024-09-16T20:51:19.126 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5500000000000005e-05 2024-09-16T20:51:19.126 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.126 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.126 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:19.126 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719836+0000", 2024-09-16T20:51:19.126 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018037399999999999 2024-09-16T20:51:19.126 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:19.127 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:19.127 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:19.127 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.127 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.127 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9973 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 653312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:19.127 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:51:19.127 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 88.418256850000006, 2024-09-16T20:51:19.127 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077905262000000003, 2024-09-16T20:51:19.127 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:19.127 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:19.127 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:19.127 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:19.127 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:19.127 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:19.128 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9973 2024-09-16T20:51:19.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.128 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:19.128 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.128 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:19.128 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:51:19.128 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:19.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.128 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.128 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:19.128 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:51:19.128 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:19.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.128 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.129 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:19.129 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:51:19.129 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:51:19.129 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.129 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.129 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:19.129 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642148+0000", 2024-09-16T20:51:19.129 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0219999999999999e-06 2024-09-16T20:51:19.129 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.129 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.129 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:19.129 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642154+0000", 2024-09-16T20:51:19.129 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8540000000000002e-06 2024-09-16T20:51:19.129 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.130 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.130 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:19.130 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642159+0000", 2024-09-16T20:51:19.130 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.045e-06 2024-09-16T20:51:19.130 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.130 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.130 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:19.130 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719875+0000", 2024-09-16T20:51:19.130 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077716116000000002 2024-09-16T20:51:19.131 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.131 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.131 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:19.131 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719934+0000", 2024-09-16T20:51:19.131 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8819e-05 2024-09-16T20:51:19.131 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.131 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.131 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:19.131 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720051+0000", 2024-09-16T20:51:19.131 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000116947 2024-09-16T20:51:19.131 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:19.132 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:19.132 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:19.132 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.132 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.132 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9974 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 658432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:19.132 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642166+0000", 2024-09-16T20:51:19.132 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 88.418236936, 2024-09-16T20:51:19.132 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.078179786000000001, 2024-09-16T20:51:19.132 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:19.132 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:19.132 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:19.132 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:19.132 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:19.132 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:19.133 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9974 2024-09-16T20:51:19.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.133 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:19.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.133 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:19.133 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642166+0000", 2024-09-16T20:51:19.133 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:19.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.133 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:19.133 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642166+0000", 2024-09-16T20:51:19.133 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:19.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.134 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:19.134 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647343+0000", 2024-09-16T20:51:19.134 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0051769440000000002 2024-09-16T20:51:19.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.134 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.134 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:19.134 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647348+0000", 2024-09-16T20:51:19.134 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3619999999999997e-06 2024-09-16T20:51:19.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.134 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.134 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:19.134 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647359+0000", 2024-09-16T20:51:19.134 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0929e-05 2024-09-16T20:51:19.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.134 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.135 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:19.135 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647366+0000", 2024-09-16T20:51:19.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0910000000000004e-06 2024-09-16T20:51:19.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.135 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.135 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:19.135 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720094+0000", 2024-09-16T20:51:19.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.072727403999999995 2024-09-16T20:51:19.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.135 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.135 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:19.135 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720173+0000", 2024-09-16T20:51:19.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9382000000000007e-05 2024-09-16T20:51:19.136 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.136 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.136 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:19.136 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720346+0000", 2024-09-16T20:51:19.136 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017267399999999999 2024-09-16T20:51:19.136 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:19.136 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:19.136 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:19.136 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.136 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.136 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9990 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 740352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:19.136 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647725+0000", 2024-09-16T20:51:19.136 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 88.412677852000002, 2024-09-16T20:51:19.136 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077029919000000002, 2024-09-16T20:51:19.137 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:19.137 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:19.137 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:19.137 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:19.137 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:19.137 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:19.137 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9990 2024-09-16T20:51:19.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.138 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:19.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.138 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:19.138 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647725+0000", 2024-09-16T20:51:19.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:19.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.138 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:19.138 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647725+0000", 2024-09-16T20:51:19.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:19.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.138 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:19.138 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647725+0000", 2024-09-16T20:51:19.139 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2399999999999999e-07 2024-09-16T20:51:19.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.139 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.139 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:19.139 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647726+0000", 2024-09-16T20:51:19.139 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.116e-06 2024-09-16T20:51:19.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.139 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.139 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:19.139 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647729+0000", 2024-09-16T20:51:19.139 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.937e-06 2024-09-16T20:51:19.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.140 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.140 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:19.140 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647732+0000", 2024-09-16T20:51:19.140 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8739999999999999e-06 2024-09-16T20:51:19.140 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.140 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.140 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:19.140 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724612+0000", 2024-09-16T20:51:19.140 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076880102000000006 2024-09-16T20:51:19.140 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.140 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.141 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:19.141 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724649+0000", 2024-09-16T20:51:19.141 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6738e-05 2024-09-16T20:51:19.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.141 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.141 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:19.141 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724755+0000", 2024-09-16T20:51:19.141 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010592800000000001 2024-09-16T20:51:19.141 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:19.141 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:19.141 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:19.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.141 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.141 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9991 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 745472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:19.142 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:51:19.142 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 88.412667542999998, 2024-09-16T20:51:19.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077257820000000005, 2024-09-16T20:51:19.142 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:19.142 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:19.142 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:19.142 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:19.142 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:19.142 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:19.142 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9991 2024-09-16T20:51:19.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.142 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:19.142 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.143 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:19.143 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:51:19.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:19.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.143 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:19.143 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:51:19.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:19.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.143 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:19.143 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:51:19.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1199999999999999e-07 2024-09-16T20:51:19.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.144 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.144 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:19.144 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647736+0000", 2024-09-16T20:51:19.144 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3699999999999999e-07 2024-09-16T20:51:19.144 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.144 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.144 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:19.144 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647739+0000", 2024-09-16T20:51:19.144 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9359999999999999e-06 2024-09-16T20:51:19.144 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.145 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.145 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:19.145 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647742+0000", 2024-09-16T20:51:19.145 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5720000000000001e-06 2024-09-16T20:51:19.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.145 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.145 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:19.145 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724774+0000", 2024-09-16T20:51:19.145 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077032457999999998 2024-09-16T20:51:19.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.145 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.145 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:19.145 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724818+0000", 2024-09-16T20:51:19.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3803000000000001e-05 2024-09-16T20:51:19.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.146 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:19.146 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724993+0000", 2024-09-16T20:51:19.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000174902 2024-09-16T20:51:19.146 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:19.146 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:19.146 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:19.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.147 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.147 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9992 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 750592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:19.147 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:51:19.147 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 88.412657953999997, 2024-09-16T20:51:19.147 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077553227000000002, 2024-09-16T20:51:19.147 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:19.147 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:19.147 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:19.148 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:19.148 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:19.148 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:19.148 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9992 2024-09-16T20:51:19.148 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.148 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:19.148 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.148 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:19.148 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:51:19.148 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:19.148 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.148 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.149 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:19.149 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:51:19.149 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:19.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.149 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.149 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:19.149 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:51:19.149 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0200000000000001e-07 2024-09-16T20:51:19.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.149 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.149 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:19.149 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647746+0000", 2024-09-16T20:51:19.149 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.088e-06 2024-09-16T20:51:19.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.150 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.150 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:19.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647749+0000", 2024-09-16T20:51:19.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9170000000000002e-06 2024-09-16T20:51:19.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.150 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.150 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:19.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647752+0000", 2024-09-16T20:51:19.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.774e-06 2024-09-16T20:51:19.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.150 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.150 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:19.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725042+0000", 2024-09-16T20:51:19.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077289952999999995 2024-09-16T20:51:19.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:19.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725113+0000", 2024-09-16T20:51:19.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0920999999999994e-05 2024-09-16T20:51:19.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.152 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.152 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:19.152 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725298+0000", 2024-09-16T20:51:19.152 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000185372 2024-09-16T20:51:19.152 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:19.152 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:19.152 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:19.153 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.153 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.153 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9993 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 755712~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:19.153 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647755+0000", 2024-09-16T20:51:19.153 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 88.412647927999998, 2024-09-16T20:51:19.153 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077725674999999994, 2024-09-16T20:51:19.153 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:19.153 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:19.153 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:19.153 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:19.153 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:19.153 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:19.153 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9993 2024-09-16T20:51:19.153 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.154 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:19.154 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.154 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:19.154 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647755+0000", 2024-09-16T20:51:19.154 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:19.154 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.154 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.154 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:19.154 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647755+0000", 2024-09-16T20:51:19.154 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:19.154 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.154 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.154 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:19.155 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653406+0000", 2024-09-16T20:51:19.155 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.005650876 2024-09-16T20:51:19.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.155 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.155 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:19.155 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653411+0000", 2024-09-16T20:51:19.155 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8899999999999998e-06 2024-09-16T20:51:19.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.155 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.155 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:19.155 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653423+0000", 2024-09-16T20:51:19.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2447e-05 2024-09-16T20:51:19.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.156 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.156 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:19.156 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653431+0000", 2024-09-16T20:51:19.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5730000000000002e-06 2024-09-16T20:51:19.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.156 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.156 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:19.156 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725333+0000", 2024-09-16T20:51:19.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.071902743000000005 2024-09-16T20:51:19.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.156 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.157 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:19.157 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725385+0000", 2024-09-16T20:51:19.157 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1585000000000002e-05 2024-09-16T20:51:19.157 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.157 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.157 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:19.157 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725481+0000", 2024-09-16T20:51:19.157 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5561000000000002e-05 2024-09-16T20:51:19.157 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:19.157 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:19.157 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:19.158 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.158 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.158 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6061.0:5400 3.b 3:d353d652:::rbd_data.110e16eafbd.0000000000000010:head [read 577536~28672] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-16T20:51:19.158 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:27.761367+0000", 2024-09-16T20:51:19.158 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.299035981999999, 2024-09-16T20:51:19.158 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0029066630000001, 2024-09-16T20:51:19.158 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:19.158 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:19.158 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:19.158 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:19.158 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6061", 2024-09-16T20:51:19.158 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3930128440", 2024-09-16T20:51:19.158 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5400 2024-09-16T20:51:19.158 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.159 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:19.159 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.159 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:19.159 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761367+0000", 2024-09-16T20:51:19.159 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:19.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.159 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.159 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:19.159 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761367+0000", 2024-09-16T20:51:19.159 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:19.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.159 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.159 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:19.159 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761369+0000", 2024-09-16T20:51:19.160 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.934e-06 2024-09-16T20:51:19.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.160 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.160 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:19.160 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761374+0000", 2024-09-16T20:51:19.160 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7139999999999998e-06 2024-09-16T20:51:19.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.160 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.160 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:19.160 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761387+0000", 2024-09-16T20:51:19.161 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2349e-05 2024-09-16T20:51:19.161 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.161 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.161 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:19.161 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761393+0000", 2024-09-16T20:51:19.161 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.652e-06 2024-09-16T20:51:19.162 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.162 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.162 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:19.162 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761442+0000", 2024-09-16T20:51:19.162 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9011000000000002e-05 2024-09-16T20:51:19.162 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.162 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.162 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:19.162 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761451+0000", 2024-09-16T20:51:19.162 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1920000000000005e-06 2024-09-16T20:51:19.162 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.162 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.163 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:19.163 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:28.764169+0000", 2024-09-16T20:51:19.163 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.002718161 2024-09-16T20:51:19.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.163 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.163 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:19.163 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:28.764273+0000", 2024-09-16T20:51:19.163 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010465 2024-09-16T20:51:19.163 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:19.163 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:19.163 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:19.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.164 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.164 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6061.0:5424 3.b 3:d353d652:::rbd_data.110e16eafbd.0000000000000010:head [read 606208~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-16T20:51:19.164 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:27.766882+0000", 2024-09-16T20:51:19.164 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.293520618999999, 2024-09-16T20:51:19.164 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99753650500000002, 2024-09-16T20:51:19.164 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:19.164 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:19.164 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:19.164 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:19.164 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6061", 2024-09-16T20:51:19.164 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3930128440", 2024-09-16T20:51:19.165 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5424 2024-09-16T20:51:19.165 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.165 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:19.165 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.165 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:19.165 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766882+0000", 2024-09-16T20:51:19.165 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:19.165 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.165 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.165 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:19.165 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766882+0000", 2024-09-16T20:51:19.165 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:19.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.166 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.166 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:19.166 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766883+0000", 2024-09-16T20:51:19.166 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3289999999999999e-06 2024-09-16T20:51:19.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.167 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.167 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:19.167 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766888+0000", 2024-09-16T20:51:19.167 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5800000000000002e-06 2024-09-16T20:51:19.167 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.167 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.167 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:19.167 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766900+0000", 2024-09-16T20:51:19.167 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1861e-05 2024-09-16T20:51:19.167 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.168 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.168 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:19.168 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766909+0000", 2024-09-16T20:51:19.168 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5210000000000008e-06 2024-09-16T20:51:19.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.168 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.168 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:19.168 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766958+0000", 2024-09-16T20:51:19.168 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8718999999999999e-05 2024-09-16T20:51:19.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.168 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.168 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:19.168 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766975+0000", 2024-09-16T20:51:19.169 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7231000000000002e-05 2024-09-16T20:51:19.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.169 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.169 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:19.169 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:28.764327+0000", 2024-09-16T20:51:19.169 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99735112500000001 2024-09-16T20:51:19.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:19.169 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:19.169 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:19.169 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:28.764419+0000", 2024-09-16T20:51:19.169 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2139000000000007e-05 2024-09-16T20:51:19.169 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:19.169 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:19.170 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:19.170 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:19.170 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:19.170 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:19.171 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:51:19.172 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:19.170+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:19.272 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:19.270+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:19.281 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:19.281 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:51:19.281 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:51:19.281 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:19.290 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:51:19.306 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T20:51:19.372 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:19.371+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:19.397 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:19.397 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:51:19.397 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:51:19.397 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:51:19.397 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:19.406 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:51:19.436 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:51:19.437 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:51:19.454 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:51:19.454 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:51:19.470 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:51:19.470 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:51:19.473 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:19.472+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:19.487 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:51:19.487 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:51:19.505 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:51:19.505 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:51:19.515 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:19.515 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:19.522 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:51:19.522 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:51:19.524 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:51:19.571 INFO:teuthology.orchestra.run.smithi029.stderr:noscrub is set 2024-09-16T20:51:19.574 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:19.573+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:19.675 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:19.674+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:19.686 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:19.686 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:19.696 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:51:19.775 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:19.774+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:19.803 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:19.803 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:19.812 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:51:19.876 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:19.875+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:19.961 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:19.961 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:19.970 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:51:19.977 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:19.976+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:20.079 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:20.077+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:20.079 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:20.080 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:20.088 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:51:20.180 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:20.179+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:20.195 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:20.195 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:20.204 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:51:20.281 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:20.280+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:20.310 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:20.311 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:20.320 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:51:20.382 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:20.381+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:20.427 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:20.427 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:20.435 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:51:20.483 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:20.482+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:20.582 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:20.582 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:20.584 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:20.583+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:20.591 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:51:20.686 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:20.684+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:20.745 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:20.745 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:20.753 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:51:20.797 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:20.796+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:20.860 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:20.861 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:20.869 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:51:20.887 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:51:20.886+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:51:20.988 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:20.987+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:21.017 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:21.017 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:21.026 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:51:21.110 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:21.109+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:21.173 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:21.173 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:21.182 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:51:21.227 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:51:21.225+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:51:21.291 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:21.290+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:21.303 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:21.303 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:21.312 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:51:21.392 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:21.390+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:21.461 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:21.461 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:21.470 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:51:21.492 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:21.491+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:21.533 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T20:51:21.577 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:21.577 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:21.585 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:51:21.589 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T20:51:21.633 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:21.632+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:21.671 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:51:21.672 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:51:21.694 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:21.692+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:21.695 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:51:21.695 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:51:21.709 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:21.709 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:21.712 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:51:21.712 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:51:21.715 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:51:21.730 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:51:21.731 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:51:21.749 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:51:21.749 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:51:21.756 INFO:tasks.thrashosds.thrasher:in_osds: [1, 4, 5, 0, 3] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-16T20:51:21.756 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T20:51:21.756 INFO:tasks.thrashosds.thrasher:Adding osd 2 2024-09-16T20:51:21.756 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 2 2024-09-16T20:51:21.766 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:51:21.767 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:51:21.822 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:21.821+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:21.823 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:21.823 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:51:21.824 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:51:21.824 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:51:21.824 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:21.832 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:51:21.895 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:21.894+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:21.941 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:21.942 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:51:21.942 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:51:21.942 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:51:21.942 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.942 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12548 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 628736~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:21.942 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757009+0000", 2024-09-16T20:51:21.942 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 91.182489326999999, 2024-09-16T20:51:21.942 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065743956000000006, 2024-09-16T20:51:21.943 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:21.943 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:21.943 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:21.943 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:21.943 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:21.943 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:21.943 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12548 2024-09-16T20:51:21.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.943 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:21.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.943 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:21.943 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757009+0000", 2024-09-16T20:51:21.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:21.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.944 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:21.944 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757009+0000", 2024-09-16T20:51:21.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:21.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.944 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:21.944 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757010+0000", 2024-09-16T20:51:21.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9900000000000001e-07 2024-09-16T20:51:21.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.944 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:21.945 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757012+0000", 2024-09-16T20:51:21.945 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2689999999999998e-06 2024-09-16T20:51:21.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.945 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:21.945 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757022+0000", 2024-09-16T20:51:21.945 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9699999999999994e-06 2024-09-16T20:51:21.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.945 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:21.945 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757028+0000", 2024-09-16T20:51:21.945 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3799999999999999e-06 2024-09-16T20:51:21.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.946 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.946 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:21.946 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822552+0000", 2024-09-16T20:51:21.946 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065524281000000004 2024-09-16T20:51:21.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.946 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.946 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:21.946 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822613+0000", 2024-09-16T20:51:21.946 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0430999999999998e-05 2024-09-16T20:51:21.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.946 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.946 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:21.946 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822753+0000", 2024-09-16T20:51:21.946 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000140226 2024-09-16T20:51:21.947 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:21.947 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:21.947 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:21.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.947 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.947 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12549 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 633856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:21.947 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:51:21.947 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 91.182460579999997, 2024-09-16T20:51:21.947 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065978518, 2024-09-16T20:51:21.948 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:21.948 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:21.948 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:21.948 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:21.948 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:21.948 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:21.948 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12549 2024-09-16T20:51:21.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.948 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:21.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.948 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:21.948 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:51:21.948 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:21.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.949 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.949 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:21.949 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:51:21.949 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:21.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.949 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.949 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:21.949 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:51:21.949 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9299999999999999e-07 2024-09-16T20:51:21.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.949 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.950 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:21.950 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757040+0000", 2024-09-16T20:51:21.950 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.677e-06 2024-09-16T20:51:21.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.950 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.950 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:21.950 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757045+0000", 2024-09-16T20:51:21.950 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8149999999999996e-06 2024-09-16T20:51:21.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.950 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.951 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:21.951 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757048+0000", 2024-09-16T20:51:21.951 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1e-06 2024-09-16T20:51:21.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.951 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.952 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:21.952 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822796+0000", 2024-09-16T20:51:21.952 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065748081 2024-09-16T20:51:21.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.952 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.952 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:21.952 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822867+0000", 2024-09-16T20:51:21.952 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0679000000000005e-05 2024-09-16T20:51:21.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.952 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.952 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:21.953 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823016+0000", 2024-09-16T20:51:21.953 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014987299999999999 2024-09-16T20:51:21.953 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:21.953 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:21.953 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:21.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.953 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.953 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12550 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 638976~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:21.953 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757060+0000", 2024-09-16T20:51:21.953 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 91.182438164999994, 2024-09-16T20:51:21.953 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066153564999999998, 2024-09-16T20:51:21.953 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:21.954 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:21.954 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:21.954 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:21.954 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:21.954 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:21.954 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12550 2024-09-16T20:51:21.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.954 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:21.954 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.954 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:21.954 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757060+0000", 2024-09-16T20:51:21.954 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:21.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.954 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.954 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:21.955 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757060+0000", 2024-09-16T20:51:21.955 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:21.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.955 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:21.955 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757061+0000", 2024-09-16T20:51:21.955 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.15e-07 2024-09-16T20:51:21.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.955 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:21.955 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757062+0000", 2024-09-16T20:51:21.955 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.663e-06 2024-09-16T20:51:21.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.956 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:21.956 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757066+0000", 2024-09-16T20:51:21.956 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.817e-06 2024-09-16T20:51:21.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.956 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:21.956 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757070+0000", 2024-09-16T20:51:21.956 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1509999999999997e-06 2024-09-16T20:51:21.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.956 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:21.956 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823044+0000", 2024-09-16T20:51:21.956 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065973992999999995 2024-09-16T20:51:21.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.957 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:21.957 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823082+0000", 2024-09-16T20:51:21.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8050999999999999e-05 2024-09-16T20:51:21.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.957 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:21.957 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823214+0000", 2024-09-16T20:51:21.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013157500000000001 2024-09-16T20:51:21.957 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:21.957 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:21.957 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:21.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.958 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.958 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12551 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 644096~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:21.958 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:51:21.958 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 91.182424467000004, 2024-09-16T20:51:21.958 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066391378000000001, 2024-09-16T20:51:21.958 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:21.958 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:21.958 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:21.958 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:21.958 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:21.958 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:21.958 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12551 2024-09-16T20:51:21.958 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.959 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:21.959 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.959 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:21.959 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:51:21.959 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:21.959 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.959 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.959 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:21.959 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:51:21.959 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:21.959 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.959 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.960 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:21.960 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:51:21.960 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2999999999999999e-07 2024-09-16T20:51:21.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.960 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:21.960 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757075+0000", 2024-09-16T20:51:21.960 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.097e-06 2024-09-16T20:51:21.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.960 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:21.961 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757082+0000", 2024-09-16T20:51:21.961 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9500000000000004e-06 2024-09-16T20:51:21.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.961 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.961 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:21.961 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757086+0000", 2024-09-16T20:51:21.961 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2229999999999999e-06 2024-09-16T20:51:21.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.961 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.961 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:21.961 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823262+0000", 2024-09-16T20:51:21.961 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066176078999999999 2024-09-16T20:51:21.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.962 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:21.962 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823336+0000", 2024-09-16T20:51:21.962 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3979000000000004e-05 2024-09-16T20:51:21.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.962 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:21.962 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823465+0000", 2024-09-16T20:51:21.962 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012982 2024-09-16T20:51:21.962 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:21.962 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:21.962 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:21.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.962 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12552 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 649216~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:21.963 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757089+0000", 2024-09-16T20:51:21.963 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 91.182409325999998, 2024-09-16T20:51:21.963 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066655617, 2024-09-16T20:51:21.963 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:21.963 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:21.963 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:21.963 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:21.963 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:21.963 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:21.963 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12552 2024-09-16T20:51:21.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.963 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:21.963 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.963 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:21.964 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757089+0000", 2024-09-16T20:51:21.964 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:21.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.964 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.964 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:21.964 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757089+0000", 2024-09-16T20:51:21.964 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:21.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.964 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.964 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:21.964 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759492+0000", 2024-09-16T20:51:21.964 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0024026109999999998 2024-09-16T20:51:21.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.964 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.965 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:21.965 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759497+0000", 2024-09-16T20:51:21.965 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1120000000000004e-06 2024-09-16T20:51:21.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.965 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.965 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:21.965 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759508+0000", 2024-09-16T20:51:21.965 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0981e-05 2024-09-16T20:51:21.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.965 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.965 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:21.966 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759516+0000", 2024-09-16T20:51:21.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7270000000000007e-06 2024-09-16T20:51:21.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.966 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:21.966 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823504+0000", 2024-09-16T20:51:21.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.063987984999999997 2024-09-16T20:51:21.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.966 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:21.966 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823579+0000", 2024-09-16T20:51:21.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5703000000000003e-05 2024-09-16T20:51:21.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.967 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:21.967 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823745+0000", 2024-09-16T20:51:21.967 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016549800000000001 2024-09-16T20:51:21.967 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:21.967 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:21.967 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:21.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.967 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12560 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 690176~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:21.967 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.759658+0000", 2024-09-16T20:51:21.967 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 91.179840123999995, 2024-09-16T20:51:21.967 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065858517000000005, 2024-09-16T20:51:21.967 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:21.967 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:21.968 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:21.968 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:21.968 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:21.968 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:21.968 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12560 2024-09-16T20:51:21.968 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.968 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:21.968 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.968 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:21.968 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759658+0000", 2024-09-16T20:51:21.968 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:21.968 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.969 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.969 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:21.969 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759658+0000", 2024-09-16T20:51:21.969 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:21.969 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.969 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.969 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:21.969 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759659+0000", 2024-09-16T20:51:21.969 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.17e-07 2024-09-16T20:51:21.969 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.969 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.969 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:21.969 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759660+0000", 2024-09-16T20:51:21.970 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0189999999999999e-06 2024-09-16T20:51:21.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.970 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.970 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:21.970 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759663+0000", 2024-09-16T20:51:21.970 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0010000000000002e-06 2024-09-16T20:51:21.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.970 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.970 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:21.970 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759665+0000", 2024-09-16T20:51:21.970 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5440000000000001e-06 2024-09-16T20:51:21.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.970 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.971 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:21.971 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825355+0000", 2024-09-16T20:51:21.971 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065690224000000005 2024-09-16T20:51:21.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.971 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.971 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:21.971 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825394+0000", 2024-09-16T20:51:21.971 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8810999999999998e-05 2024-09-16T20:51:21.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.971 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.972 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:21.972 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825517+0000", 2024-09-16T20:51:21.972 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012270099999999999 2024-09-16T20:51:21.972 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:21.972 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:21.972 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:21.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.972 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.973 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12561 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 695296~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:21.973 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.759668+0000", 2024-09-16T20:51:21.973 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 91.179830281999998, 2024-09-16T20:51:21.973 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066116999999999995, 2024-09-16T20:51:21.973 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:21.973 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:21.973 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:21.973 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:21.973 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:21.974 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:21.974 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12561 2024-09-16T20:51:21.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.974 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:21.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.974 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:21.974 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759668+0000", 2024-09-16T20:51:21.974 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:21.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.974 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:21.974 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759668+0000", 2024-09-16T20:51:21.974 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:21.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.975 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.975 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:21.975 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765379+0000", 2024-09-16T20:51:21.975 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057104410000000001 2024-09-16T20:51:21.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.975 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.975 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:21.976 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765383+0000", 2024-09-16T20:51:21.976 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2980000000000002e-06 2024-09-16T20:51:21.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.976 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.976 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:21.976 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765395+0000", 2024-09-16T20:51:21.976 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1671000000000001e-05 2024-09-16T20:51:21.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.976 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.976 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:21.976 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765403+0000", 2024-09-16T20:51:21.976 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1210000000000007e-06 2024-09-16T20:51:21.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.977 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:21.977 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825552+0000", 2024-09-16T20:51:21.977 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.060149498000000003 2024-09-16T20:51:21.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.977 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:21.977 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825625+0000", 2024-09-16T20:51:21.977 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2540999999999995e-05 2024-09-16T20:51:21.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.978 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:21.978 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825785+0000", 2024-09-16T20:51:21.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016043000000000001 2024-09-16T20:51:21.978 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:21.978 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:21.978 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:21.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.978 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.978 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12585 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 818176~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:21.978 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.765848+0000", 2024-09-16T20:51:21.978 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 91.173650311000003, 2024-09-16T20:51:21.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065843218999999995, 2024-09-16T20:51:21.978 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:21.979 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:21.979 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:21.979 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:21.979 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:21.979 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:21.979 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12585 2024-09-16T20:51:21.979 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.979 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:21.979 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.979 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:21.979 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765848+0000", 2024-09-16T20:51:21.979 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:21.979 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.980 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.980 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:21.980 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765848+0000", 2024-09-16T20:51:21.980 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:21.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.980 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.980 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:21.980 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768683+0000", 2024-09-16T20:51:21.980 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0028344519999999999 2024-09-16T20:51:21.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.980 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.980 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:21.980 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768686+0000", 2024-09-16T20:51:21.981 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8789999999999996e-06 2024-09-16T20:51:21.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.981 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.981 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:21.981 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768696+0000", 2024-09-16T20:51:21.981 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6709999999999994e-06 2024-09-16T20:51:21.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.981 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.981 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:21.981 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768702+0000", 2024-09-16T20:51:21.981 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2550000000000003e-06 2024-09-16T20:51:21.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.981 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.981 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:21.982 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.831499+0000", 2024-09-16T20:51:21.982 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.062797081000000005 2024-09-16T20:51:21.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.982 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.982 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:21.982 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.831538+0000", 2024-09-16T20:51:21.982 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8523999999999999e-05 2024-09-16T20:51:21.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.982 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.982 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:21.982 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.831691+0000", 2024-09-16T20:51:21.982 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015335699999999999 2024-09-16T20:51:21.982 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:21.982 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:21.983 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:21.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.983 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.983 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12597 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 879616~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:21.983 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.768883+0000", 2024-09-16T20:51:21.983 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 91.170615583, 2024-09-16T20:51:21.983 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065816527, 2024-09-16T20:51:21.983 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:21.983 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:21.983 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:21.983 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:21.983 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:21.983 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:21.983 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12597 2024-09-16T20:51:21.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.984 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:21.984 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.984 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:21.984 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768883+0000", 2024-09-16T20:51:21.984 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:21.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.984 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.984 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:21.984 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768883+0000", 2024-09-16T20:51:21.984 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:21.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.984 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.984 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:21.985 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771508+0000", 2024-09-16T20:51:21.985 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0026247589999999999 2024-09-16T20:51:21.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.985 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.985 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:21.985 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771513+0000", 2024-09-16T20:51:21.985 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9949999999999996e-06 2024-09-16T20:51:21.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.985 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.985 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:21.985 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771525+0000", 2024-09-16T20:51:21.985 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2033e-05 2024-09-16T20:51:21.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.986 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.986 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:21.986 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771532+0000", 2024-09-16T20:51:21.986 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4220000000000003e-06 2024-09-16T20:51:21.986 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.986 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.986 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:21.986 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.834532+0000", 2024-09-16T20:51:21.986 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.063000159 2024-09-16T20:51:21.986 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.986 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.986 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:21.986 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.834590+0000", 2024-09-16T20:51:21.986 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8218999999999999e-05 2024-09-16T20:51:21.987 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.987 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.987 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:21.987 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.834699+0000", 2024-09-16T20:51:21.987 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010894 2024-09-16T20:51:21.987 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:21.987 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:21.987 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:21.987 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.987 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.987 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12619 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 992256~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:21.987 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.774656+0000", 2024-09-16T20:51:21.987 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 91.164842809000007, 2024-09-16T20:51:21.988 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065715859000000001, 2024-09-16T20:51:21.988 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:21.988 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:21.988 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:21.988 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:21.988 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:21.988 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:21.988 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12619 2024-09-16T20:51:21.988 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.988 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:21.988 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.988 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:21.988 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774656+0000", 2024-09-16T20:51:21.988 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:21.989 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.989 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.989 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:21.989 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774656+0000", 2024-09-16T20:51:21.989 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:21.989 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.989 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.989 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:21.989 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774875+0000", 2024-09-16T20:51:21.989 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000218898 2024-09-16T20:51:21.989 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.989 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.990 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:21.990 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774879+0000", 2024-09-16T20:51:21.990 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0389999999999998e-06 2024-09-16T20:51:21.990 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.990 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.990 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:21.990 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774890+0000", 2024-09-16T20:51:21.990 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1250000000000001e-05 2024-09-16T20:51:21.990 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.990 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.990 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:21.990 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774897+0000", 2024-09-16T20:51:21.990 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7619999999999998e-06 2024-09-16T20:51:21.990 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.991 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.991 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:21.991 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840139+0000", 2024-09-16T20:51:21.991 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065242438 2024-09-16T20:51:21.991 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.991 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.991 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:21.991 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840186+0000", 2024-09-16T20:51:21.991 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6894999999999997e-05 2024-09-16T20:51:21.991 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.991 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.991 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:21.992 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840372+0000", 2024-09-16T20:51:21.992 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000185577 2024-09-16T20:51:21.992 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:21.992 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:21.992 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:21.992 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.992 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.992 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12620 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 997376~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:21.992 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.774908+0000", 2024-09-16T20:51:21.992 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 91.164590587999996, 2024-09-16T20:51:21.993 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065782754999999998, 2024-09-16T20:51:21.993 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:21.993 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:21.993 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:21.993 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:21.993 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:21.993 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:21.993 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12620 2024-09-16T20:51:21.993 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.993 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:21.993 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.993 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:21.994 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774908+0000", 2024-09-16T20:51:21.994 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:21.994 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.994 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.994 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:21.994 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774908+0000", 2024-09-16T20:51:21.994 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:21.994 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.994 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.994 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:21.994 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775102+0000", 2024-09-16T20:51:21.994 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000194098 2024-09-16T20:51:21.994 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.994 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.995 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:21.995 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775105+0000", 2024-09-16T20:51:21.995 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8739999999999999e-06 2024-09-16T20:51:21.995 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.995 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.995 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:21.995 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775111+0000", 2024-09-16T20:51:21.995 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9519999999999999e-06 2024-09-16T20:51:21.995 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.995 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.995 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:21.995 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775115+0000", 2024-09-16T20:51:21.995 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.002e-06 2024-09-16T20:51:21.996 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.996 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.996 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:21.996 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840409+0000", 2024-09-16T20:51:21.996 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065294195999999999 2024-09-16T20:51:21.996 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.996 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.996 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:21.996 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840507+0000", 2024-09-16T20:51:21.997 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7662000000000001e-05 2024-09-16T20:51:21.997 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.997 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.997 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:21.997 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840691+0000", 2024-09-16T20:51:21.997 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000183971 2024-09-16T20:51:21.997 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:21.997 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:21.997 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:21.997 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.997 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.997 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12621 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1002496~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:21.997 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775123+0000", 2024-09-16T20:51:21.998 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 91.164375891999995, 2024-09-16T20:51:21.998 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065829586999999995, 2024-09-16T20:51:21.998 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:21.998 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:21.998 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:21.998 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:21.998 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:21.998 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:21.998 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12621 2024-09-16T20:51:21.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.998 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:21.998 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.998 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:21.998 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775123+0000", 2024-09-16T20:51:21.998 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:21.999 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.999 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.999 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:21.999 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775123+0000", 2024-09-16T20:51:21.999 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:21.999 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.999 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.999 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:21.999 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775348+0000", 2024-09-16T20:51:21.999 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000225295 2024-09-16T20:51:21.999 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:21.999 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:21.999 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:21.999 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775352+0000", 2024-09-16T20:51:22.000 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0810000000000004e-06 2024-09-16T20:51:22.000 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.000 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.000 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:22.000 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775361+0000", 2024-09-16T20:51:22.000 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7399999999999993e-06 2024-09-16T20:51:22.000 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.000 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.000 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:22.000 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775367+0000", 2024-09-16T20:51:22.000 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5200000000000003e-06 2024-09-16T20:51:22.000 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.000 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.001 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:22.001 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840730+0000", 2024-09-16T20:51:22.001 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065362803999999997 2024-09-16T20:51:22.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.001 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.001 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:22.001 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840801+0000", 2024-09-16T20:51:22.001 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1006000000000004e-05 2024-09-16T20:51:22.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.001 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.001 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:22.001 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840952+0000", 2024-09-16T20:51:22.001 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015114100000000001 2024-09-16T20:51:22.002 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:22.002 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:22.002 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:22.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.002 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12622 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1007616~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:22.002 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775381+0000", 2024-09-16T20:51:22.002 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 91.164117058000002, 2024-09-16T20:51:22.002 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065822581000000005, 2024-09-16T20:51:22.002 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:22.002 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:22.002 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:22.003 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:22.003 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:22.003 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:22.003 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12622 2024-09-16T20:51:22.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.003 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:22.003 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.003 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:22.003 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775381+0000", 2024-09-16T20:51:22.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:22.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.003 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.004 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:22.004 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775381+0000", 2024-09-16T20:51:22.004 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:22.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.004 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.004 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:22.004 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775563+0000", 2024-09-16T20:51:22.004 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018153700000000001 2024-09-16T20:51:22.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.004 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.004 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:22.004 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775566+0000", 2024-09-16T20:51:22.004 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9620000000000002e-06 2024-09-16T20:51:22.005 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.005 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.005 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:22.005 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775572+0000", 2024-09-16T20:51:22.005 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8529999999999997e-06 2024-09-16T20:51:22.005 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.005 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.005 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:22.005 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775576+0000", 2024-09-16T20:51:22.005 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8920000000000002e-06 2024-09-16T20:51:22.005 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.005 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.005 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:22.006 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840990+0000", 2024-09-16T20:51:22.006 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065414599000000004 2024-09-16T20:51:22.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.006 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.006 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:22.006 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841068+0000", 2024-09-16T20:51:22.006 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8157000000000004e-05 2024-09-16T20:51:22.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.006 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.006 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:22.006 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841204+0000", 2024-09-16T20:51:22.006 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013558099999999999 2024-09-16T20:51:22.006 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:22.007 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:22.007 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:22.007 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.007 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.007 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12623 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1012736~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:22.007 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775583+0000", 2024-09-16T20:51:22.007 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 91.163915619999997, 2024-09-16T20:51:22.007 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065929087999999997, 2024-09-16T20:51:22.007 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:22.007 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:22.007 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:22.007 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:22.007 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:22.008 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:22.008 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12623 2024-09-16T20:51:22.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.008 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:22.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.008 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:22.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775583+0000", 2024-09-16T20:51:22.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:22.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.008 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:22.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775583+0000", 2024-09-16T20:51:22.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:22.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.009 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.009 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:22.009 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775804+0000", 2024-09-16T20:51:22.009 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00022116900000000001 2024-09-16T20:51:22.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.009 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.009 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:22.009 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775807+0000", 2024-09-16T20:51:22.009 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9529999999999999e-06 2024-09-16T20:51:22.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.009 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.009 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:22.010 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775814+0000", 2024-09-16T20:51:22.010 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7490000000000001e-06 2024-09-16T20:51:22.010 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.010 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.010 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:22.010 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775818+0000", 2024-09-16T20:51:22.010 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1999999999999996e-06 2024-09-16T20:51:22.010 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.010 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.010 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:22.010 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841267+0000", 2024-09-16T20:51:22.010 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065449127999999995 2024-09-16T20:51:22.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.011 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.011 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:22.011 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841344+0000", 2024-09-16T20:51:22.011 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6546999999999997e-05 2024-09-16T20:51:22.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.011 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.011 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:22.011 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841512+0000", 2024-09-16T20:51:22.011 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016834200000000001 2024-09-16T20:51:22.011 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:22.012 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:22.012 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:22.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.012 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.012 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12624 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1017856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:22.012 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:51:22.012 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 91.163672480000002, 2024-09-16T20:51:22.012 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065945866000000006, 2024-09-16T20:51:22.012 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:22.012 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:22.012 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:22.012 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:22.012 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:22.012 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:22.013 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12624 2024-09-16T20:51:22.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.013 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:22.013 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.013 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:22.013 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:51:22.013 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:22.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.013 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.013 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:22.013 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:51:22.013 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:22.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.013 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.014 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:22.014 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:51:22.014 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-16T20:51:22.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.014 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.014 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:22.014 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775828+0000", 2024-09-16T20:51:22.014 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3999999999999999e-06 2024-09-16T20:51:22.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.014 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.014 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:22.014 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775831+0000", 2024-09-16T20:51:22.014 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.478e-06 2024-09-16T20:51:22.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.014 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.015 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:22.015 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775834+0000", 2024-09-16T20:51:22.015 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9440000000000001e-06 2024-09-16T20:51:22.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.015 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.015 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:22.015 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841547+0000", 2024-09-16T20:51:22.015 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065712801000000001 2024-09-16T20:51:22.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.015 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.015 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:22.015 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841618+0000", 2024-09-16T20:51:22.015 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1151999999999999e-05 2024-09-16T20:51:22.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.016 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.016 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:22.016 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841772+0000", 2024-09-16T20:51:22.016 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000153721 2024-09-16T20:51:22.016 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:22.016 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:22.016 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:22.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.016 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.016 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12625 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1022976~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:22.016 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775840+0000", 2024-09-16T20:51:22.016 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 91.163658870999996, 2024-09-16T20:51:22.017 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066165451, 2024-09-16T20:51:22.017 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:22.017 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:22.017 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:22.017 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:22.017 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:22.017 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:22.017 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12625 2024-09-16T20:51:22.017 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.017 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:22.017 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.017 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:22.017 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775840+0000", 2024-09-16T20:51:22.018 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:22.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.018 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.018 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:22.018 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775840+0000", 2024-09-16T20:51:22.018 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:22.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.018 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.018 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:22.018 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776014+0000", 2024-09-16T20:51:22.018 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017441099999999999 2024-09-16T20:51:22.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.018 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.019 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:22.019 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776018+0000", 2024-09-16T20:51:22.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6509999999999999e-06 2024-09-16T20:51:22.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.019 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:22.019 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776026+0000", 2024-09-16T20:51:22.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.028e-06 2024-09-16T20:51:22.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.019 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:22.019 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776031+0000", 2024-09-16T20:51:22.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8400000000000002e-06 2024-09-16T20:51:22.020 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.020 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.020 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:22.020 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841801+0000", 2024-09-16T20:51:22.020 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065770472999999996 2024-09-16T20:51:22.020 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.020 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.020 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:22.020 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841847+0000", 2024-09-16T20:51:22.020 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5558e-05 2024-09-16T20:51:22.020 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.020 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.020 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:22.021 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842005+0000", 2024-09-16T20:51:22.021 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015849000000000001 2024-09-16T20:51:22.021 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:22.021 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:22.021 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:22.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.021 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.021 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12626 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1028096~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:22.022 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776041+0000", 2024-09-16T20:51:22.022 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 91.163457571999999, 2024-09-16T20:51:22.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066225260999999994, 2024-09-16T20:51:22.022 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:22.022 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:22.022 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:22.022 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:22.022 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:22.022 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:22.022 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12626 2024-09-16T20:51:22.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.022 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:22.023 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.023 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:22.023 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776041+0000", 2024-09-16T20:51:22.023 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:22.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.023 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.023 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:22.023 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776041+0000", 2024-09-16T20:51:22.023 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:22.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.023 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.023 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:22.023 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776263+0000", 2024-09-16T20:51:22.023 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00022193899999999999 2024-09-16T20:51:22.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.024 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:22.024 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776268+0000", 2024-09-16T20:51:22.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8049999999999997e-06 2024-09-16T20:51:22.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.024 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:22.024 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776280+0000", 2024-09-16T20:51:22.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2635e-05 2024-09-16T20:51:22.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.024 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:22.024 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776288+0000", 2024-09-16T20:51:22.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8199999999999997e-06 2024-09-16T20:51:22.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:22.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842041+0000", 2024-09-16T20:51:22.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065752506000000002 2024-09-16T20:51:22.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:22.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842123+0000", 2024-09-16T20:51:22.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2393999999999998e-05 2024-09-16T20:51:22.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:22.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842266+0000", 2024-09-16T20:51:22.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014316200000000001 2024-09-16T20:51:22.026 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:22.026 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:22.026 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:22.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.026 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12627 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1033216~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:22.026 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776301+0000", 2024-09-16T20:51:22.026 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 91.163197612000005, 2024-09-16T20:51:22.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066187814999999997, 2024-09-16T20:51:22.026 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:22.026 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:22.026 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:22.026 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:22.027 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:22.027 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:22.027 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12627 2024-09-16T20:51:22.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.027 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:22.027 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.027 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:22.027 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776301+0000", 2024-09-16T20:51:22.027 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:22.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.027 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.027 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:22.027 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776301+0000", 2024-09-16T20:51:22.027 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:22.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.028 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.028 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:22.028 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776473+0000", 2024-09-16T20:51:22.028 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017222400000000001 2024-09-16T20:51:22.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.028 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.028 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:22.028 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776477+0000", 2024-09-16T20:51:22.028 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7019999999999999e-06 2024-09-16T20:51:22.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.028 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.028 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:22.028 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776486+0000", 2024-09-16T20:51:22.028 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2119999999999992e-06 2024-09-16T20:51:22.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.029 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:22.029 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776492+0000", 2024-09-16T20:51:22.029 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8089999999999998e-06 2024-09-16T20:51:22.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.029 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:22.029 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842301+0000", 2024-09-16T20:51:22.029 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065809042999999998 2024-09-16T20:51:22.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.030 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:22.030 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842362+0000", 2024-09-16T20:51:22.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1260000000000006e-05 2024-09-16T20:51:22.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.030 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:22.030 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842489+0000", 2024-09-16T20:51:22.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012656500000000001 2024-09-16T20:51:22.030 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:22.030 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:22.030 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:22.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.031 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.031 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12628 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1038336~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:22.031 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776502+0000", 2024-09-16T20:51:22.031 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 91.162996965999994, 2024-09-16T20:51:22.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066283426000000006, 2024-09-16T20:51:22.031 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:22.031 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:22.031 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:22.031 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:22.031 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:22.031 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:22.032 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12628 2024-09-16T20:51:22.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.032 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:22.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.032 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:22.032 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776502+0000", 2024-09-16T20:51:22.032 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:22.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.032 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:22.032 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776502+0000", 2024-09-16T20:51:22.032 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:22.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.033 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.033 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:22.033 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776716+0000", 2024-09-16T20:51:22.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000214596 2024-09-16T20:51:22.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.033 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.033 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:22.033 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776723+0000", 2024-09-16T20:51:22.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4000000000000003e-06 2024-09-16T20:51:22.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.033 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.033 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:22.033 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776731+0000", 2024-09-16T20:51:22.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6129999999999998e-06 2024-09-16T20:51:22.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.034 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.034 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:22.034 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776737+0000", 2024-09-16T20:51:22.034 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1970000000000001e-06 2024-09-16T20:51:22.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.034 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.034 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:22.034 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842532+0000", 2024-09-16T20:51:22.034 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065795127999999994 2024-09-16T20:51:22.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.034 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.034 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:22.034 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842599+0000", 2024-09-16T20:51:22.035 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6711000000000006e-05 2024-09-16T20:51:22.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.035 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:22.035 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842785+0000", 2024-09-16T20:51:22.035 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018578099999999999 2024-09-16T20:51:22.035 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:22.035 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:22.035 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:22.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.035 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12629 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1043456~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:22.035 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776748+0000", 2024-09-16T20:51:22.035 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 91.162750588999998, 2024-09-16T20:51:22.036 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066303788000000002, 2024-09-16T20:51:22.036 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:22.036 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:22.036 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:22.036 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:22.036 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:22.036 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:22.036 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12629 2024-09-16T20:51:22.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.036 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:22.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.036 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:22.036 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776748+0000", 2024-09-16T20:51:22.037 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:22.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.037 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.037 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:22.037 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776748+0000", 2024-09-16T20:51:22.037 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:22.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.037 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.037 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:22.037 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776934+0000", 2024-09-16T20:51:22.037 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018598399999999999 2024-09-16T20:51:22.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.037 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.038 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:22.038 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776939+0000", 2024-09-16T20:51:22.038 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1000000000000003e-06 2024-09-16T20:51:22.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.038 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.038 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:22.038 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776954+0000", 2024-09-16T20:51:22.038 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5221e-05 2024-09-16T20:51:22.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.038 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.038 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:22.038 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776962+0000", 2024-09-16T20:51:22.038 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.554e-06 2024-09-16T20:51:22.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.039 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:22.039 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842813+0000", 2024-09-16T20:51:22.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065851006000000004 2024-09-16T20:51:22.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.039 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:22.039 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842886+0000", 2024-09-16T20:51:22.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3663000000000003e-05 2024-09-16T20:51:22.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.039 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:22.039 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.843052+0000", 2024-09-16T20:51:22.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016526 2024-09-16T20:51:22.039 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:22.040 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:22.040 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:22.040 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:22.040 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:22.040 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:22.042 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:51:22.042 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:22.039+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:22.065 INFO:tasks.ceph.mon.a.smithi029.stderr:2024-09-16T20:51:22.064+0000 7fe0ce96d640 -1 mon.a@0(leader).osd e194 definitely_dead 0 2024-09-16T20:51:22.142 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:51:22.140+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:51:22.190 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:22.190 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:51:22.190 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:51:22.190 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:22.199 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:51:22.243 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:22.242+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:22.344 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:22.342+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:22.345 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:22.345 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:51:22.345 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:51:22.345 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:51:22.345 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:22.354 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:51:22.445 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:51:22.443+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:51:22.463 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:22.463 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:51:22.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:51:22.463 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:51:22.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.463 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5286.0:10643 3.17 3.f6273fb7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-16T20:51:22.463 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:51:22.463 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 133.670422035, 2024-09-16T20:51:22.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.76672347200000002, 2024-09-16T20:51:22.463 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:22.463 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:22.464 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:22.464 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:22.464 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5286", 2024-09-16T20:51:22.464 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/4135433989", 2024-09-16T20:51:22.464 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 10643 2024-09-16T20:51:22.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.464 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:22.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.464 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:22.464 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:51:22.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:22.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.465 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:22.465 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:51:22.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:22.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.465 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:22.465 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790489+0000", 2024-09-16T20:51:22.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5910000000000001e-06 2024-09-16T20:51:22.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.465 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:22.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790493+0000", 2024-09-16T20:51:22.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2669999999999998e-06 2024-09-16T20:51:22.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:22.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790500+0000", 2024-09-16T20:51:22.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3780000000000003e-06 2024-09-16T20:51:22.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:22.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790505+0000", 2024-09-16T20:51:22.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1560000000000003e-06 2024-09-16T20:51:22.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.467 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:22.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790559+0000", 2024-09-16T20:51:22.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3494e-05 2024-09-16T20:51:22.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.467 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:22.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790566+0000", 2024-09-16T20:51:22.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7649999999999993e-06 2024-09-16T20:51:22.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:22.468 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:09.557198+0000", 2024-09-16T20:51:22.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.76663136200000004 2024-09-16T20:51:22.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:22.468 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:09.557211+0000", 2024-09-16T20:51:22.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3458999999999999e-05 2024-09-16T20:51:22.468 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:22.468 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:22.469 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:22.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.469 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5512.0:450 3.2 3:44e0cffa:::rbd_header.110ef0d7a535:head [watch reconnect cookie 140263298018912 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:51:22.469 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:51:22.469 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 105.898399122, 2024-09-16T20:51:22.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0084581939999999, 2024-09-16T20:51:22.469 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:22.469 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:22.469 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:22.469 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:22.469 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5512", 2024-09-16T20:51:22.469 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:51:22.470 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 450 2024-09-16T20:51:22.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.470 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:22.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:22.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:51:22.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:22.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:22.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:51:22.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:22.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:22.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562512+0000", 2024-09-16T20:51:22.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.009e-06 2024-09-16T20:51:22.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:22.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562517+0000", 2024-09-16T20:51:22.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9339999999999997e-06 2024-09-16T20:51:22.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:22.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562530+0000", 2024-09-16T20:51:22.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3495e-05 2024-09-16T20:51:22.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:22.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562536+0000", 2024-09-16T20:51:22.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5690000000000004e-06 2024-09-16T20:51:22.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:22.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562591+0000", 2024-09-16T20:51:22.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5761999999999999e-05 2024-09-16T20:51:22.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:51:22.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562599+0000", 2024-09-16T20:51:22.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9179999999999994e-06 2024-09-16T20:51:22.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:22.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.565971+0000", 2024-09-16T20:51:22.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003371546 2024-09-16T20:51:22.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:22.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566019+0000", 2024-09-16T20:51:22.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7849000000000002e-05 2024-09-16T20:51:22.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:22.474 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.569382+0000", 2024-09-16T20:51:22.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.003362833 2024-09-16T20:51:22.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:22.474 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.569529+0000", 2024-09-16T20:51:22.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000147823 2024-09-16T20:51:22.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:22.474 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570969+0000", 2024-09-16T20:51:22.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001439456 2024-09-16T20:51:22.475 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:22.475 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:22.475 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:22.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.475 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:14996 3.2 3:44e0cffa:::rbd_header.110ef0d7a535:head [watch reconnect cookie 140213436269840 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:51:22.475 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:51:22.475 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 105.89835071, 2024-09-16T20:51:22.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.008813943, 2024-09-16T20:51:22.475 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:22.475 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:22.475 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:22.475 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:22.475 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:22.476 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:22.476 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 14996 2024-09-16T20:51:22.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.476 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:22.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.476 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:22.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:51:22.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:22.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.476 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:22.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:51:22.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:22.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:22.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562560+0000", 2024-09-16T20:51:22.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.06e-07 2024-09-16T20:51:22.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:22.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562563+0000", 2024-09-16T20:51:22.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.495e-06 2024-09-16T20:51:22.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:22.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562571+0000", 2024-09-16T20:51:22.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1950000000000003e-06 2024-09-16T20:51:22.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:22.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562579+0000", 2024-09-16T20:51:22.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2649999999999999e-06 2024-09-16T20:51:22.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:22.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562673+0000", 2024-09-16T20:51:22.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4118999999999998e-05 2024-09-16T20:51:22.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:51:22.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562677+0000", 2024-09-16T20:51:22.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5179999999999998e-06 2024-09-16T20:51:22.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:22.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566461+0000", 2024-09-16T20:51:22.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0037837259999999998 2024-09-16T20:51:22.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:22.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566468+0000", 2024-09-16T20:51:22.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3350000000000001e-06 2024-09-16T20:51:22.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:22.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570306+0000", 2024-09-16T20:51:22.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.003837997 2024-09-16T20:51:22.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:22.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570392+0000", 2024-09-16T20:51:22.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5872999999999996e-05 2024-09-16T20:51:22.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:22.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571373+0000", 2024-09-16T20:51:22.481 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00098071399999999993 2024-09-16T20:51:22.481 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:22.481 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:22.481 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:22.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.481 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5255.0:1931 3.2 3:47882819:::rbd_data.110eb5f95afc.0000000000000009:head [read 217088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:51:22.481 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:51:22.481 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 105.89831196, 2024-09-16T20:51:22.481 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0082752880000001, 2024-09-16T20:51:22.481 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:22.481 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:22.481 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:22.482 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:22.482 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5255", 2024-09-16T20:51:22.482 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:51:22.482 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1931 2024-09-16T20:51:22.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.482 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:22.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.482 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:22.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:51:22.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:22.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.482 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:22.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:51:22.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:22.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:22.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:51:22.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9699999999999996e-07 2024-09-16T20:51:22.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:22.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562601+0000", 2024-09-16T20:51:22.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8600000000000001e-06 2024-09-16T20:51:22.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:22.484 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562608+0000", 2024-09-16T20:51:22.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9e-06 2024-09-16T20:51:22.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:22.484 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562615+0000", 2024-09-16T20:51:22.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8179999999999998e-06 2024-09-16T20:51:22.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:22.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562703+0000", 2024-09-16T20:51:22.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7811000000000003e-05 2024-09-16T20:51:22.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.485 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:51:22.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562706+0000", 2024-09-16T20:51:22.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4920000000000002e-06 2024-09-16T20:51:22.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.485 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:22.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566685+0000", 2024-09-16T20:51:22.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0039792780000000002 2024-09-16T20:51:22.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.486 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:22.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566693+0000", 2024-09-16T20:51:22.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4730000000000003e-06 2024-09-16T20:51:22.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.486 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:22.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570737+0000", 2024-09-16T20:51:22.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0040436749999999 2024-09-16T20:51:22.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.486 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:22.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570873+0000", 2024-09-16T20:51:22.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000136484 2024-09-16T20:51:22.486 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:22.487 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:22.487 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:22.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.487 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5255.0:1939 3.2 3:47882819:::rbd_data.110eb5f95afc.0000000000000009:head [read 233472~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:51:22.487 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:51:22.487 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 105.898157094, 2024-09-16T20:51:22.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0084657589999999, 2024-09-16T20:51:22.487 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:22.487 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:22.487 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:22.487 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:22.487 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5255", 2024-09-16T20:51:22.487 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:51:22.488 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1939 2024-09-16T20:51:22.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.488 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:22.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.488 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:22.488 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:51:22.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:22.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.488 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:22.488 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:51:22.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:22.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:22.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:51:22.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5199999999999997e-07 2024-09-16T20:51:22.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:22.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562757+0000", 2024-09-16T20:51:22.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4979999999999998e-06 2024-09-16T20:51:22.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:22.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562764+0000", 2024-09-16T20:51:22.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1470000000000004e-06 2024-09-16T20:51:22.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:22.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562768+0000", 2024-09-16T20:51:22.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7929999999999999e-06 2024-09-16T20:51:22.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:22.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562805+0000", 2024-09-16T20:51:22.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7477e-05 2024-09-16T20:51:22.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting_for_map not empty", 2024-09-16T20:51:22.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562810+0000", 2024-09-16T20:51:22.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0819999999999998e-06 2024-09-16T20:51:22.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:22.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566717+0000", 2024-09-16T20:51:22.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0039072600000000001 2024-09-16T20:51:22.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:22.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566724+0000", 2024-09-16T20:51:22.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4880000000000004e-06 2024-09-16T20:51:22.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.492 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:22.492 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571009+0000", 2024-09-16T20:51:22.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.004284677 2024-09-16T20:51:22.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.492 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:22.492 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571218+0000", 2024-09-16T20:51:22.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00020978500000000001 2024-09-16T20:51:22.492 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:22.492 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:22.492 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:22.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.493 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:11021 3.1f 3:fee2d1dd:::rbd_data.110ea93691bb.000000000000000e:23 [read 960512~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:22.493 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.693837+0000", 2024-09-16T20:51:22.493 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 91.767072549000005, 2024-09-16T20:51:22.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057155138000000001, 2024-09-16T20:51:22.493 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:22.493 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:22.493 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:22.493 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:22.493 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:22.494 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:22.494 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 11021 2024-09-16T20:51:22.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.494 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:22.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.494 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:22.494 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693837+0000", 2024-09-16T20:51:22.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:22.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:22.495 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693837+0000", 2024-09-16T20:51:22.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:22.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:22.495 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699566+0000", 2024-09-16T20:51:22.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057283639999999997 2024-09-16T20:51:22.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:22.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699571+0000", 2024-09-16T20:51:22.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.604e-06 2024-09-16T20:51:22.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:22.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699583+0000", 2024-09-16T20:51:22.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1372999999999999e-05 2024-09-16T20:51:22.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:22.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699591+0000", 2024-09-16T20:51:22.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5259999999999993e-06 2024-09-16T20:51:22.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:22.497 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750674+0000", 2024-09-16T20:51:22.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051083154999999998 2024-09-16T20:51:22.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:22.497 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750745+0000", 2024-09-16T20:51:22.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0933999999999995e-05 2024-09-16T20:51:22.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:22.497 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750992+0000", 2024-09-16T20:51:22.498 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00024718199999999998 2024-09-16T20:51:22.498 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:22.498 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:22.498 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:22.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.498 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8792 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2609152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:22.498 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:51:22.498 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.403612831999993, 2024-09-16T20:51:22.498 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057003524999999999, 2024-09-16T20:51:22.498 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:22.499 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:22.499 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:22.499 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:22.499 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:22.499 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:22.499 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8792 2024-09-16T20:51:22.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.499 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:22.499 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.499 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:22.499 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:51:22.499 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:22.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.500 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:22.500 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:51:22.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:22.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.500 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:22.500 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:51:22.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0400000000000002e-07 2024-09-16T20:51:22.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.500 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:22.500 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057299+0000", 2024-09-16T20:51:22.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7460000000000001e-06 2024-09-16T20:51:22.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:22.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057304+0000", 2024-09-16T20:51:22.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3210000000000004e-06 2024-09-16T20:51:22.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:22.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057310+0000", 2024-09-16T20:51:22.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4360000000000001e-06 2024-09-16T20:51:22.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:22.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114200+0000", 2024-09-16T20:51:22.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056889954 2024-09-16T20:51:22.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.502 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:22.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114243+0000", 2024-09-16T20:51:22.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2998999999999999e-05 2024-09-16T20:51:22.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.502 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:22.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114300+0000", 2024-09-16T20:51:22.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7765e-05 2024-09-16T20:51:22.502 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:22.502 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:22.503 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:22.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.503 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8794 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2351104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:22.503 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057327+0000", 2024-09-16T20:51:22.503 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.403582508, 2024-09-16T20:51:22.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057101130999999999, 2024-09-16T20:51:22.503 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:22.503 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:22.503 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:22.503 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:22.503 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:22.503 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:22.503 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8794 2024-09-16T20:51:22.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.504 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:22.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.504 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:22.504 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057327+0000", 2024-09-16T20:51:22.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:22.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:22.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057327+0000", 2024-09-16T20:51:22.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:22.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:22.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057328+0000", 2024-09-16T20:51:22.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7399999999999999e-07 2024-09-16T20:51:22.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:22.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057330+0000", 2024-09-16T20:51:22.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2199999999999999e-06 2024-09-16T20:51:22.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:22.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057336+0000", 2024-09-16T20:51:22.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1240000000000002e-06 2024-09-16T20:51:22.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:22.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057341+0000", 2024-09-16T20:51:22.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8470000000000003e-06 2024-09-16T20:51:22.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:22.507 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114334+0000", 2024-09-16T20:51:22.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056993267 2024-09-16T20:51:22.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.507 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.507 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:22.507 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114376+0000", 2024-09-16T20:51:22.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1638999999999999e-05 2024-09-16T20:51:22.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.507 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.507 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:22.507 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114428+0000", 2024-09-16T20:51:22.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2660000000000001e-05 2024-09-16T20:51:22.507 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:22.507 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:22.507 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:22.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.508 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8798 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2297856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:22.508 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:51:22.508 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.403562781000005, 2024-09-16T20:51:22.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057277041000000001, 2024-09-16T20:51:22.508 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:22.508 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:22.508 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:22.508 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:22.508 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:22.508 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:22.508 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8798 2024-09-16T20:51:22.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.508 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:22.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:22.509 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:51:22.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:22.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:22.509 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:51:22.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:22.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:22.509 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:51:22.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2800000000000003e-07 2024-09-16T20:51:22.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:22.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057354+0000", 2024-09-16T20:51:22.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7780000000000002e-06 2024-09-16T20:51:22.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:22.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057360+0000", 2024-09-16T20:51:22.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0419999999999999e-06 2024-09-16T20:51:22.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:22.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057365+0000", 2024-09-16T20:51:22.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7929999999999997e-06 2024-09-16T20:51:22.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:22.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114460+0000", 2024-09-16T20:51:22.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057094662999999997 2024-09-16T20:51:22.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:22.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114532+0000", 2024-09-16T20:51:22.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2881999999999998e-05 2024-09-16T20:51:22.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:22.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114624+0000", 2024-09-16T20:51:22.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1555e-05 2024-09-16T20:51:22.512 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:22.512 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:22.512 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:22.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.512 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.512 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8800 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2316288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:22.512 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:51:22.512 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.403538842000003, 2024-09-16T20:51:22.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.05746946, 2024-09-16T20:51:22.512 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:22.512 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:22.512 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:22.512 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:22.512 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:22.513 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:22.513 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8800 2024-09-16T20:51:22.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.513 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:22.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.513 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:22.513 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:51:22.513 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:22.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.513 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:22.513 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:51:22.513 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:22.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:22.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:51:22.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5199999999999998e-07 2024-09-16T20:51:22.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:22.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057373+0000", 2024-09-16T20:51:22.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.79e-06 2024-09-16T20:51:22.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:22.515 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057379+0000", 2024-09-16T20:51:22.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6699999999999999e-06 2024-09-16T20:51:22.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.515 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.515 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:22.515 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057386+0000", 2024-09-16T20:51:22.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3170000000000004e-06 2024-09-16T20:51:22.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.516 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.516 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:22.516 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114655+0000", 2024-09-16T20:51:22.516 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057269420000000001 2024-09-16T20:51:22.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.517 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:22.517 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114734+0000", 2024-09-16T20:51:22.517 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8084000000000007e-05 2024-09-16T20:51:22.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.518 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:22.518 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114840+0000", 2024-09-16T20:51:22.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000106827 2024-09-16T20:51:22.518 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:22.518 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:22.518 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:22.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.518 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8801 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2334720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:22.518 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:51:22.518 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.403517811, 2024-09-16T20:51:22.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057594682000000001, 2024-09-16T20:51:22.518 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:22.518 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:22.519 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:22.519 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:22.519 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:22.519 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:22.519 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8801 2024-09-16T20:51:22.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.519 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:22.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.519 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:22.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:51:22.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:22.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.519 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:22.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:51:22.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:22.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.520 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:22.520 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:51:22.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6899999999999998e-07 2024-09-16T20:51:22.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.520 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:22.520 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057394+0000", 2024-09-16T20:51:22.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8300000000000001e-06 2024-09-16T20:51:22.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.521 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:22.521 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057400+0000", 2024-09-16T20:51:22.521 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6500000000000001e-06 2024-09-16T20:51:22.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.521 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.525 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:22.525 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057404+0000", 2024-09-16T20:51:22.525 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4669999999999998e-06 2024-09-16T20:51:22.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.525 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.525 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:22.525 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114874+0000", 2024-09-16T20:51:22.525 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057469602000000002 2024-09-16T20:51:22.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.526 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.526 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:22.526 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114933+0000", 2024-09-16T20:51:22.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9500000000000003e-05 2024-09-16T20:51:22.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.526 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.526 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:22.526 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114987+0000", 2024-09-16T20:51:22.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3263999999999997e-05 2024-09-16T20:51:22.526 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:22.526 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:22.526 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:22.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.526 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.527 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8802 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2353152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:22.527 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:51:22.527 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.403469126000005, 2024-09-16T20:51:22.527 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057714079000000001, 2024-09-16T20:51:22.527 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:22.527 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:22.527 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:22.527 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:22.527 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:22.527 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:22.527 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8802 2024-09-16T20:51:22.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.527 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:22.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:22.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:51:22.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:22.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:22.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:51:22.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:22.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.529 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:22.529 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:51:22.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4999999999999998e-07 2024-09-16T20:51:22.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.530 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:22.530 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057446+0000", 2024-09-16T20:51:22.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4240000000000003e-06 2024-09-16T20:51:22.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:22.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057452+0000", 2024-09-16T20:51:22.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8379999999999996e-06 2024-09-16T20:51:22.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.532 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:22.532 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057460+0000", 2024-09-16T20:51:22.532 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9629999999999998e-06 2024-09-16T20:51:22.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.533 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.533 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:22.533 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115015+0000", 2024-09-16T20:51:22.533 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057554389999999997 2024-09-16T20:51:22.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.533 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.534 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:22.534 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115082+0000", 2024-09-16T20:51:22.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7733999999999999e-05 2024-09-16T20:51:22.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:22.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115155+0000", 2024-09-16T20:51:22.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2280000000000006e-05 2024-09-16T20:51:22.535 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:22.535 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:22.535 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:22.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.536 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8804 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2555904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:22.536 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:51:22.536 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.403442810000001, 2024-09-16T20:51:22.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057833862, 2024-09-16T20:51:22.537 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:22.537 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:22.537 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:22.537 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:22.537 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:22.537 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:22.537 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8804 2024-09-16T20:51:22.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.538 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:22.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.538 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:22.538 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:51:22.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:22.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.539 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:22.539 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:51:22.539 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:22.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.539 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:22.539 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:51:22.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7899999999999999e-07 2024-09-16T20:51:22.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:22.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057469+0000", 2024-09-16T20:51:22.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.001e-06 2024-09-16T20:51:22.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.541 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:22.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057475+0000", 2024-09-16T20:51:22.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7629999999999997e-06 2024-09-16T20:51:22.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:22.542 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057480+0000", 2024-09-16T20:51:22.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8069999999999999e-06 2024-09-16T20:51:22.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.543 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:22.543 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115183+0000", 2024-09-16T20:51:22.543 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057703364 2024-09-16T20:51:22.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.543 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.543 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:22.544 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115227+0000", 2024-09-16T20:51:22.544 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3903999999999998e-05 2024-09-16T20:51:22.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.544 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:22.544 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115301+0000", 2024-09-16T20:51:22.544 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3543999999999999e-05 2024-09-16T20:51:22.544 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:22.544 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:22.544 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:22.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.545 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8805 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2574336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:22.545 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057487+0000", 2024-09-16T20:51:22.545 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.403422481999996, 2024-09-16T20:51:22.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058021027000000003, 2024-09-16T20:51:22.545 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:22.545 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:22.545 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:22.545 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:22.545 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:22.545 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:22.546 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8805 2024-09-16T20:51:22.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.546 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:22.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:22.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057487+0000", 2024-09-16T20:51:22.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:22.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:22.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057487+0000", 2024-09-16T20:51:22.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:22.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.547 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:22.547 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057488+0000", 2024-09-16T20:51:22.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5199999999999998e-07 2024-09-16T20:51:22.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.547 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:22.547 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057490+0000", 2024-09-16T20:51:22.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.939e-06 2024-09-16T20:51:22.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.547 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:22.547 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057495+0000", 2024-09-16T20:51:22.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6439999999999997e-06 2024-09-16T20:51:22.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.548 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.548 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:22.548 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057500+0000", 2024-09-16T20:51:22.548 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8119999999999998e-06 2024-09-16T20:51:22.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.548 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.548 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:22.548 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115341+0000", 2024-09-16T20:51:22.548 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057841341999999997 2024-09-16T20:51:22.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.548 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.549 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:22.549 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115422+0000", 2024-09-16T20:51:22.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1003999999999999e-05 2024-09-16T20:51:22.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.549 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:22.549 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115508+0000", 2024-09-16T20:51:22.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5933999999999994e-05 2024-09-16T20:51:22.550 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:22.550 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:22.550 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:22.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.550 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8807 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2592768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:22.550 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057505+0000", 2024-09-16T20:51:22.550 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.403404272000003, 2024-09-16T20:51:22.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058186633000000001, 2024-09-16T20:51:22.550 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:22.550 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:22.550 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:22.551 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:22.551 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:22.551 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:22.551 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8807 2024-09-16T20:51:22.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.551 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:22.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.551 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:22.551 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057505+0000", 2024-09-16T20:51:22.551 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:22.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.552 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:22.552 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057505+0000", 2024-09-16T20:51:22.552 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:22.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.552 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.552 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:22.552 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057506+0000", 2024-09-16T20:51:22.552 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.27e-07 2024-09-16T20:51:22.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.552 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.552 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:22.552 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057508+0000", 2024-09-16T20:51:22.552 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7400000000000001e-06 2024-09-16T20:51:22.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.553 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:22.553 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057513+0000", 2024-09-16T20:51:22.553 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.468e-06 2024-09-16T20:51:22.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.553 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:22.553 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057517+0000", 2024-09-16T20:51:22.553 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4329999999999997e-06 2024-09-16T20:51:22.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.554 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.554 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:22.554 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115538+0000", 2024-09-16T20:51:22.554 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058020355000000003 2024-09-16T20:51:22.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.554 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.554 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:22.554 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115616+0000", 2024-09-16T20:51:22.554 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7998999999999996e-05 2024-09-16T20:51:22.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:22.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115692+0000", 2024-09-16T20:51:22.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6310999999999995e-05 2024-09-16T20:51:22.555 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:22.555 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:22.555 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:22.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.555 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8808 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2611200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:22.555 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:51:22.555 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.403387011000007, 2024-09-16T20:51:22.556 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058354046, 2024-09-16T20:51:22.556 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:22.556 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:22.556 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:22.556 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:22.556 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:22.556 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:22.556 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8808 2024-09-16T20:51:22.556 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.556 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:22.556 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.556 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:22.556 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:51:22.557 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:22.557 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.557 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.557 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:22.557 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:51:22.557 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:22.557 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.557 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.557 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:22.557 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:51:22.557 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2500000000000001e-07 2024-09-16T20:51:22.558 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.558 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.558 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:22.558 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057525+0000", 2024-09-16T20:51:22.558 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5549999999999999e-06 2024-09-16T20:51:22.558 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.558 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.558 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:22.558 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057530+0000", 2024-09-16T20:51:22.558 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5489999999999997e-06 2024-09-16T20:51:22.558 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.558 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.558 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:22.558 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057535+0000", 2024-09-16T20:51:22.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5709999999999999e-06 2024-09-16T20:51:22.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:22.559 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115722+0000", 2024-09-16T20:51:22.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058187323999999999 2024-09-16T20:51:22.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:22.559 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115799+0000", 2024-09-16T20:51:22.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6445000000000004e-05 2024-09-16T20:51:22.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:22.560 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115877+0000", 2024-09-16T20:51:22.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8276999999999996e-05 2024-09-16T20:51:22.560 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:22.560 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:22.560 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:22.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.560 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:2182 3.17 3:edb5e5cd:::rbd_data.110e61479aee.0000000000000014:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write 0~5120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-16T20:51:22.560 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:22.734609+0000", 2024-09-16T20:51:22.560 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 59.726300764000001, 2024-09-16T20:51:22.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.124890894, 2024-09-16T20:51:22.561 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:22.561 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:22.561 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:22.561 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:22.561 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:51:22.561 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:22.561 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 2182 2024-09-16T20:51:22.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.561 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:22.561 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.561 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:22.561 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734609+0000", 2024-09-16T20:51:22.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:22.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.562 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:22.562 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734609+0000", 2024-09-16T20:51:22.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:22.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.562 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:22.562 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734611+0000", 2024-09-16T20:51:22.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9599999999999999e-06 2024-09-16T20:51:22.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.563 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:22.563 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734628+0000", 2024-09-16T20:51:22.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.715e-05 2024-09-16T20:51:22.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.563 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:22.563 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734656+0000", 2024-09-16T20:51:22.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7759999999999998e-05 2024-09-16T20:51:22.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.563 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:22.563 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734669+0000", 2024-09-16T20:51:22.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3315999999999999e-05 2024-09-16T20:51:22.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:22.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734826+0000", 2024-09-16T20:51:22.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015668499999999999 2024-09-16T20:51:22.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:22.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.735048+0000", 2024-09-16T20:51:22.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00022179 2024-09-16T20:51:22.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:22.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.735248+0000", 2024-09-16T20:51:22.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00020010399999999999 2024-09-16T20:51:22.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.565 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:51:22.565 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.736252+0000", 2024-09-16T20:51:22.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010039739999999999 2024-09-16T20:51:22.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.565 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:22.565 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.859340+0000", 2024-09-16T20:51:22.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.12308846599999999 2024-09-16T20:51:22.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.565 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:51:22.566 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.859431+0000", 2024-09-16T20:51:22.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1099000000000003e-05 2024-09-16T20:51:22.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.566 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.566 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:22.566 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.859500+0000", 2024-09-16T20:51:22.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8590000000000006e-05 2024-09-16T20:51:22.566 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:22.566 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:22.566 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:22.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.566 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.566 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:29157 3.1b 3:dbe69130:::journal.110e1b19a241:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e167)", 2024-09-16T20:51:22.566 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:54.663540+0000", 2024-09-16T20:51:22.567 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.797369425999999, 2024-09-16T20:51:22.567 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.92912227999999997, 2024-09-16T20:51:22.567 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:22.567 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:22.567 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:22.567 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:22.567 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:22.567 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:22.567 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 29157 2024-09-16T20:51:22.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.567 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:22.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.568 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:22.568 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663540+0000", 2024-09-16T20:51:22.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:22.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.568 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:22.568 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663540+0000", 2024-09-16T20:51:22.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:22.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.569 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:22.569 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663541+0000", 2024-09-16T20:51:22.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1680000000000001e-06 2024-09-16T20:51:22.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.569 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:22.569 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663546+0000", 2024-09-16T20:51:22.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7129999999999996e-06 2024-09-16T20:51:22.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.569 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:22.570 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663558+0000", 2024-09-16T20:51:22.570 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1347e-05 2024-09-16T20:51:22.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.570 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:22.570 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663563+0000", 2024-09-16T20:51:22.570 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1250000000000001e-06 2024-09-16T20:51:22.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.570 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:22.570 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663612+0000", 2024-09-16T20:51:22.570 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9665e-05 2024-09-16T20:51:22.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.571 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:22.571 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663620+0000", 2024-09-16T20:51:22.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8439999999999998e-06 2024-09-16T20:51:22.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.571 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.571 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:22.571 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:55.592525+0000", 2024-09-16T20:51:22.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.92890456700000001 2024-09-16T20:51:22.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.571 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.571 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:22.571 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:55.592663+0000", 2024-09-16T20:51:22.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013785100000000001 2024-09-16T20:51:22.572 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:22.572 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:22.572 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:22.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.572 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:30229 3.16 3:6f5476df:::rbd_header.110e25db1f60:head [watch reconnect cookie 140213637350016 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e181)", 2024-09-16T20:51:22.572 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:51:08.603124+0000", 2024-09-16T20:51:22.572 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 13.857785781, 2024-09-16T20:51:22.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.97765519899999997, 2024-09-16T20:51:22.572 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:22.572 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:22.572 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:22.572 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:22.573 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:22.573 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:22.573 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30229 2024-09-16T20:51:22.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.573 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:22.573 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.573 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:22.573 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603124+0000", 2024-09-16T20:51:22.573 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:22.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.573 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.573 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:22.573 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603124+0000", 2024-09-16T20:51:22.574 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:22.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.574 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:22.574 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603125+0000", 2024-09-16T20:51:22.574 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5090000000000001e-06 2024-09-16T20:51:22.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.574 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:22.574 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603131+0000", 2024-09-16T20:51:22.574 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9089999999999997e-06 2024-09-16T20:51:22.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.574 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:22.574 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603143+0000", 2024-09-16T20:51:22.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1416e-05 2024-09-16T20:51:22.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.575 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:22.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603151+0000", 2024-09-16T20:51:22.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7889999999999995e-06 2024-09-16T20:51:22.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.575 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:22.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603674+0000", 2024-09-16T20:51:22.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00052357 2024-09-16T20:51:22.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.575 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:22.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603684+0000", 2024-09-16T20:51:22.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0274e-05 2024-09-16T20:51:22.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:22.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.575380+0000", 2024-09-16T20:51:22.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.97169509799999998 2024-09-16T20:51:22.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:22.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.575591+0000", 2024-09-16T20:51:22.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00021126700000000001 2024-09-16T20:51:22.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.577 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:22.577 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.580779+0000", 2024-09-16T20:51:22.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0051883670000000002 2024-09-16T20:51:22.577 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:22.577 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:22.577 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:22.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.577 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:30230 3.16 3:6f5476df:::rbd_header.110e25db1f60:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e181)", 2024-09-16T20:51:22.577 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:51:08.603204+0000", 2024-09-16T20:51:22.577 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 13.857705833000001, 2024-09-16T20:51:22.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.98129583300000001, 2024-09-16T20:51:22.577 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:22.578 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:22.578 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:22.578 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:22.578 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:22.578 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:22.578 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30230 2024-09-16T20:51:22.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.578 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:22.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.578 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:22.578 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603204+0000", 2024-09-16T20:51:22.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:22.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:22.579 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603204+0000", 2024-09-16T20:51:22.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:22.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:22.579 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603205+0000", 2024-09-16T20:51:22.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4900000000000005e-07 2024-09-16T20:51:22.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:22.579 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603209+0000", 2024-09-16T20:51:22.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5210000000000004e-06 2024-09-16T20:51:22.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:22.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603222+0000", 2024-09-16T20:51:22.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2733e-05 2024-09-16T20:51:22.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:22.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603228+0000", 2024-09-16T20:51:22.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4520000000000002e-06 2024-09-16T20:51:22.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:22.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603719+0000", 2024-09-16T20:51:22.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00049103600000000003 2024-09-16T20:51:22.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:22.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603730+0000", 2024-09-16T20:51:22.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.023e-05 2024-09-16T20:51:22.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:22.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.577394+0000", 2024-09-16T20:51:22.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.97366440099999996 2024-09-16T20:51:22.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:22.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.577499+0000", 2024-09-16T20:51:22.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000104509 2024-09-16T20:51:22.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:22.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.577920+0000", 2024-09-16T20:51:22.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00042173699999999999 2024-09-16T20:51:22.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:51:22.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.580795+0000", 2024-09-16T20:51:22.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.002874302 2024-09-16T20:51:22.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:22.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.584423+0000", 2024-09-16T20:51:22.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0036280010000000001 2024-09-16T20:51:22.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:51:22.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.584465+0000", 2024-09-16T20:51:22.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2590999999999999e-05 2024-09-16T20:51:22.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:22.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:22.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:22.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.584500+0000", 2024-09-16T20:51:22.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4570999999999998e-05 2024-09-16T20:51:22.583 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:22.584 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:22.584 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:22.584 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:22.584 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:22.584 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:22.585 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:22.584+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:22.586 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:51:22.599 INFO:teuthology.orchestra.run.smithi029.stderr:nodeep-scrub is set 2024-09-16T20:51:22.599 INFO:teuthology.orchestra.run.smithi029.stderr:marked in osd.2. 2024-09-16T20:51:22.629 INFO:tasks.thrashosds.thrasher:Added osd 2 2024-09-16T20:51:22.686 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:22.685+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:22.716 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:22.716 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:51:22.716 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:22.716 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:31707 3.3 3.5a749323 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e195)", 2024-09-16T20:51:22.716 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:51:22.668328+0000", 2024-09-16T20:51:22.716 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.047485605, 2024-09-16T20:51:22.716 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047503696999999998, 2024-09-16T20:51:22.716 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:22.716 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:22.717 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "waiting for peered", 2024-09-16T20:51:22.717 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:22.717 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:51:22.717 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:22.717 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 31707 2024-09-16T20:51:22.717 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:22.717 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:22.717 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:22.717 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:22.717 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:22.668328+0000", 2024-09-16T20:51:22.717 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:22.717 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:22.717 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:22.718 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:22.718 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:22.668328+0000", 2024-09-16T20:51:22.718 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:22.718 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:22.718 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:22.718 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:22.718 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:22.668329+0000", 2024-09-16T20:51:22.718 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7400000000000002e-07 2024-09-16T20:51:22.718 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:22.718 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:22.718 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:22.718 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:22.668333+0000", 2024-09-16T20:51:22.718 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0940000000000001e-06 2024-09-16T20:51:22.718 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:22.719 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:22.719 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:22.719 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:22.668345+0000", 2024-09-16T20:51:22.719 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2072000000000001e-05 2024-09-16T20:51:22.719 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:22.719 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:22.719 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:22.719 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:22.668350+0000", 2024-09-16T20:51:22.719 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4389999999999999e-06 2024-09-16T20:51:22.719 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:22.719 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:22.719 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:22.719 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:22.668421+0000", 2024-09-16T20:51:22.719 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1289000000000001e-05 2024-09-16T20:51:22.720 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:22.720 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:22.720 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:22.720 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:22.668435+0000", 2024-09-16T20:51:22.720 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3054000000000001e-05 2024-09-16T20:51:22.720 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:22.720 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:22.720 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:22.720 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:22.720 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-16T20:51:22.720 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 1 2024-09-16T20:51:22.720 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:22.725 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:51:22.787 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:22.786+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:22.832 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:22.832 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:51:22.832 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:51:22.833 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:51:22.833 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:22.841 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:51:22.925 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:22.924+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:22.990 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:22.988+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:22.990 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:22.990 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:51:22.991 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:51:22.991 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:51:22.991 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:22.991 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5439 3.a 3:5789f5c3:::rbd_data.110ed9ca540.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 1181696~60416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:22.991 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:51:22.991 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 180.12392743999999, 2024-09-16T20:51:22.991 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10102045599999999, 2024-09-16T20:51:22.991 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:22.991 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:22.991 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:22.991 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:22.991 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:22.991 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:22.991 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5439 2024-09-16T20:51:22.991 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:22.992 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:22.992 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:22.992 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:22.992 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:51:22.992 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:22.992 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:22.992 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:22.992 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:22.992 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:51:22.992 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:22.992 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:22.992 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:22.992 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:22.992 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864711+0000", 2024-09-16T20:51:22.992 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7899999999999997e-07 2024-09-16T20:51:22.993 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:22.993 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:22.993 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:22.993 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864760+0000", 2024-09-16T20:51:22.993 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8803e-05 2024-09-16T20:51:22.993 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:22.993 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:22.993 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:22.993 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864769+0000", 2024-09-16T20:51:22.993 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0999999999999993e-06 2024-09-16T20:51:22.993 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:22.993 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:22.993 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:22.993 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864774+0000", 2024-09-16T20:51:22.994 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8919999999999999e-06 2024-09-16T20:51:22.994 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:22.994 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:22.994 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:22.994 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864815+0000", 2024-09-16T20:51:22.994 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1307000000000003e-05 2024-09-16T20:51:22.994 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:22.994 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:22.994 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:22.994 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864893+0000", 2024-09-16T20:51:22.994 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7581000000000001e-05 2024-09-16T20:51:22.994 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:22.994 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:22.994 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 1", 2024-09-16T20:51:22.994 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864996+0000", 2024-09-16T20:51:22.995 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102725 2024-09-16T20:51:22.995 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:22.995 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:22.995 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:22.995 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965442+0000", 2024-09-16T20:51:22.995 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100446012 2024-09-16T20:51:22.995 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:22.995 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:22.995 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:22.995 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965618+0000", 2024-09-16T20:51:22.995 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00017653100000000001 2024-09-16T20:51:22.995 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:22.995 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:22.995 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:22.995 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965682+0000", 2024-09-16T20:51:22.996 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3905999999999994e-05 2024-09-16T20:51:22.996 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:22.996 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:22.996 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:22.996 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965731+0000", 2024-09-16T20:51:22.996 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8819999999999997e-05 2024-09-16T20:51:22.996 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:22.996 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:22.996 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:22.996 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:22.996 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:22.996 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4600.0:5439 3.a e37/23 ondisk, result = 0)", 2024-09-16T20:51:22.996 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:51:22.996 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 180.122931292, 2024-09-16T20:51:22.996 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099742634999999996, 2024-09-16T20:51:22.997 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:22.997 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:22.997 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:22.997 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:22.997 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:22.997 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:22.997 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:51:22.997 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:22.997 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:22.997 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:22.997 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:22.997 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:51:22.997 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:22.997 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:22.998 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:22.998 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:22.998 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:51:22.998 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0900000000000001e-07 2024-09-16T20:51:22.998 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:22.998 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:22.998 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:22.998 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865711+0000", 2024-09-16T20:51:22.998 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3739999999999998e-06 2024-09-16T20:51:22.998 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:22.998 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:22.998 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:22.998 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965343+0000", 2024-09-16T20:51:22.998 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099632526999999999 2024-09-16T20:51:22.998 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:22.999 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:22.999 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:22.999 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965357+0000", 2024-09-16T20:51:22.999 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3713999999999999e-05 2024-09-16T20:51:22.999 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:22.999 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:22.999 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:22.999 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965426+0000", 2024-09-16T20:51:22.999 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8745999999999995e-05 2024-09-16T20:51:22.999 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:22.999 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:22.999 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:22.999 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965434+0000", 2024-09-16T20:51:22.999 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6910000000000003e-06 2024-09-16T20:51:22.999 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.000 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.000 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:23.000 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965449+0000", 2024-09-16T20:51:23.000 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4874e-05 2024-09-16T20:51:23.000 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.000 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:23.000 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.000 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.000 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.000 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4882.0:2508 3.12 3:4992771f:::rbd_object_map.110e7c8851c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T20:51:23.000 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:51:23.000 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 156.507127744, 2024-09-16T20:51:23.000 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.125007011, 2024-09-16T20:51:23.000 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:23.000 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:23.001 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:23.001 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:23.001 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4882", 2024-09-16T20:51:23.001 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2436923223", 2024-09-16T20:51:23.001 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2508 2024-09-16T20:51:23.001 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.001 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:23.001 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.001 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:23.001 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:51:23.001 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.001 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.001 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.001 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:23.001 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:51:23.002 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.002 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.002 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.002 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:23.002 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481511+0000", 2024-09-16T20:51:23.002 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0699999999999996e-07 2024-09-16T20:51:23.002 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.002 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.002 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:23.002 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481515+0000", 2024-09-16T20:51:23.002 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0609999999999997e-06 2024-09-16T20:51:23.002 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.002 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.002 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:23.002 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481523+0000", 2024-09-16T20:51:23.002 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1310000000000006e-06 2024-09-16T20:51:23.003 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.003 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.003 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:23.003 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481528+0000", 2024-09-16T20:51:23.003 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4900000000000002e-06 2024-09-16T20:51:23.003 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.003 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.003 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:23.003 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481565+0000", 2024-09-16T20:51:23.003 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7066000000000001e-05 2024-09-16T20:51:23.003 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.003 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.003 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:23.003 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481628+0000", 2024-09-16T20:51:23.003 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3118000000000001e-05 2024-09-16T20:51:23.004 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.004 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.004 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 2", 2024-09-16T20:51:23.004 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481822+0000", 2024-09-16T20:51:23.004 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019446200000000001 2024-09-16T20:51:23.004 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.004 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.004 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:23.004 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605988+0000", 2024-09-16T20:51:23.004 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.124165888 2024-09-16T20:51:23.004 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.004 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.004 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:23.004 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606427+0000", 2024-09-16T20:51:23.005 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00043854899999999999 2024-09-16T20:51:23.005 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.005 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.005 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:23.005 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606476+0000", 2024-09-16T20:51:23.005 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8971999999999998e-05 2024-09-16T20:51:23.005 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.005 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.005 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:23.005 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606517+0000", 2024-09-16T20:51:23.005 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1467000000000001e-05 2024-09-16T20:51:23.005 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.005 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:23.005 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.005 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.006 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.006 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4882.0:2508 3.12 e57/55 ondisk, result = 0)", 2024-09-16T20:51:23.006 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:51:23.006 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 156.506139244, 2024-09-16T20:51:23.006 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12349711400000001, 2024-09-16T20:51:23.006 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:23.006 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:23.006 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:23.006 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:23.006 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.006 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:23.006 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:51:23.006 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.006 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.006 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.007 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:23.007 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:51:23.007 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.007 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.007 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.007 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:23.007 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:51:23.007 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.06e-07 2024-09-16T20:51:23.007 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.007 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.007 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:23.007 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482502+0000", 2024-09-16T20:51:23.007 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.01e-06 2024-09-16T20:51:23.007 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.008 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.008 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:23.008 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605772+0000", 2024-09-16T20:51:23.008 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.123269687 2024-09-16T20:51:23.008 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.008 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.008 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:23.009 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605791+0000", 2024-09-16T20:51:23.009 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8912000000000001e-05 2024-09-16T20:51:23.009 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.009 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.009 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:23.009 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605974+0000", 2024-09-16T20:51:23.009 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000182836 2024-09-16T20:51:23.009 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.009 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.009 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:23.009 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605982+0000", 2024-09-16T20:51:23.009 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1699999999999997e-06 2024-09-16T20:51:23.009 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.010 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.010 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:23.010 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605996+0000", 2024-09-16T20:51:23.010 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3893000000000001e-05 2024-09-16T20:51:23.010 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.010 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:23.010 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.010 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.010 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.010 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4882.0:2512 3.f 3:f4d895b7:::rbd_data.110e7c8851c.0000000000000000:head [read 16384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T20:51:23.010 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:51:23.010 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 156.505702214, 2024-09-16T20:51:23.010 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12318483, 2024-09-16T20:51:23.010 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:23.010 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:23.011 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:23.011 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:23.011 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4882", 2024-09-16T20:51:23.011 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2436923223", 2024-09-16T20:51:23.011 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2512 2024-09-16T20:51:23.011 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.011 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:23.011 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.011 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:23.011 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:51:23.011 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.011 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.011 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.011 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:23.011 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:51:23.011 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.012 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.012 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.012 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:23.012 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:51:23.012 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3499999999999995e-07 2024-09-16T20:51:23.012 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.012 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.012 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:23.012 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482943+0000", 2024-09-16T20:51:23.012 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3450000000000002e-06 2024-09-16T20:51:23.012 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.012 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.012 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:23.012 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605772+0000", 2024-09-16T20:51:23.012 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.122829796 2024-09-16T20:51:23.013 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.013 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.013 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:23.013 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605792+0000", 2024-09-16T20:51:23.013 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9916999999999999e-05 2024-09-16T20:51:23.013 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.013 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.013 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:23.013 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605917+0000", 2024-09-16T20:51:23.013 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00012500799999999999 2024-09-16T20:51:23.013 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.013 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.013 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:23.013 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606004+0000", 2024-09-16T20:51:23.014 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6242999999999994e-05 2024-09-16T20:51:23.014 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.014 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.014 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:23.014 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606120+0000", 2024-09-16T20:51:23.014 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000116786 2024-09-16T20:51:23.014 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.014 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:23.014 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.014 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.014 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.014 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:11702 3.15 3:acf33c8b:::rbd_data.110e747f0779.000000000000001a:head [set-alloc-hint object_size 262144 write_size 262144,write 143360~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:51:23.014 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:51:23.014 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 138.850709792, 2024-09-16T20:51:23.015 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.084801043000000006, 2024-09-16T20:51:23.015 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:23.015 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:23.015 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:23.015 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:23.015 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:51:23.015 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:51:23.015 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 11702 2024-09-16T20:51:23.015 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.015 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:23.015 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.015 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:23.015 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:51:23.015 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.015 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.016 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.016 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:23.016 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:51:23.016 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.016 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.016 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.016 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:23.016 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137929+0000", 2024-09-16T20:51:23.016 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8400000000000002e-07 2024-09-16T20:51:23.016 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.016 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.016 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:23.016 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137937+0000", 2024-09-16T20:51:23.016 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0260000000000007e-06 2024-09-16T20:51:23.016 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.017 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.017 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:23.017 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137952+0000", 2024-09-16T20:51:23.017 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4659e-05 2024-09-16T20:51:23.017 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.017 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.017 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:23.017 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137960+0000", 2024-09-16T20:51:23.017 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0169999999999996e-06 2024-09-16T20:51:23.017 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.017 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.017 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:23.017 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138010+0000", 2024-09-16T20:51:23.017 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0021000000000001e-05 2024-09-16T20:51:23.018 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.018 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.018 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:23.018 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138114+0000", 2024-09-16T20:51:23.018 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000103797 2024-09-16T20:51:23.018 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.018 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.018 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:51:23.018 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138233+0000", 2024-09-16T20:51:23.018 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000118971 2024-09-16T20:51:23.018 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.018 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.018 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:23.018 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138953+0000", 2024-09-16T20:51:23.018 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00072026000000000002 2024-09-16T20:51:23.019 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.019 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.019 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:23.019 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222609+0000", 2024-09-16T20:51:23.019 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.083656650999999999 2024-09-16T20:51:23.019 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.019 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.019 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:23.019 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222684+0000", 2024-09-16T20:51:23.019 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4369999999999994e-05 2024-09-16T20:51:23.019 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.019 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.019 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:23.020 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222729+0000", 2024-09-16T20:51:23.020 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5287000000000002e-05 2024-09-16T20:51:23.020 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.020 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:23.020 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.020 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.020 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.020 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:11704 3.9 3:91220b75:::rbd_data.110e747f0779.000000000000001c:head [set-alloc-hint object_size 262144 write_size 262144,write 139264~6656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:51:23.020 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:51:23.020 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 138.85047179099999, 2024-09-16T20:51:23.020 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.084531133999999994, 2024-09-16T20:51:23.020 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:23.020 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:23.020 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:23.020 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:23.021 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:51:23.021 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:51:23.021 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 11704 2024-09-16T20:51:23.021 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.021 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:23.021 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.021 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:23.021 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:51:23.021 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.021 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.021 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.021 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:23.021 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:51:23.021 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.022 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.022 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.022 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:23.022 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138167+0000", 2024-09-16T20:51:23.022 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.33e-07 2024-09-16T20:51:23.022 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.022 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.022 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:23.022 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138179+0000", 2024-09-16T20:51:23.022 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2055999999999999e-05 2024-09-16T20:51:23.022 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.022 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.022 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:23.022 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138193+0000", 2024-09-16T20:51:23.022 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3961000000000001e-05 2024-09-16T20:51:23.022 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.023 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.023 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:23.023 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138200+0000", 2024-09-16T20:51:23.023 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4429999999999997e-06 2024-09-16T20:51:23.023 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.023 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.023 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:23.023 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138258+0000", 2024-09-16T20:51:23.023 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7225999999999997e-05 2024-09-16T20:51:23.023 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.023 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.023 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:23.023 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138364+0000", 2024-09-16T20:51:23.023 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000106013 2024-09-16T20:51:23.023 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.024 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.024 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:51:23.024 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138494+0000", 2024-09-16T20:51:23.024 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013059599999999999 2024-09-16T20:51:23.024 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.024 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.024 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:23.024 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.139220+0000", 2024-09-16T20:51:23.024 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00072539800000000004 2024-09-16T20:51:23.024 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.024 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.024 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:23.024 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222608+0000", 2024-09-16T20:51:23.024 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.083388620999999996 2024-09-16T20:51:23.024 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.025 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.025 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:23.025 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222651+0000", 2024-09-16T20:51:23.025 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2228000000000003e-05 2024-09-16T20:51:23.025 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.025 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.025 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:23.025 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222697+0000", 2024-09-16T20:51:23.025 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6659000000000002e-05 2024-09-16T20:51:23.025 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.025 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:23.025 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.025 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.025 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.025 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5138.0:9358 3.c 3.c25699cc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T20:51:23.026 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:51:23.026 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 116.138324486, 2024-09-16T20:51:23.026 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0018657580000001, 2024-09-16T20:51:23.026 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:23.026 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:23.026 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:51:23.026 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:23.026 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5138", 2024-09-16T20:51:23.026 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:51:23.026 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 9358 2024-09-16T20:51:23.026 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.026 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:23.026 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.026 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:23.026 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:51:23.027 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.027 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.027 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.027 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:23.027 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:51:23.027 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.027 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.027 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.027 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:23.027 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850314+0000", 2024-09-16T20:51:23.027 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4099999999999999e-07 2024-09-16T20:51:23.027 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.027 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.027 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:23.027 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850316+0000", 2024-09-16T20:51:23.027 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2469999999999998e-06 2024-09-16T20:51:23.028 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.028 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.028 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:23.028 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850322+0000", 2024-09-16T20:51:23.028 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3470000000000004e-06 2024-09-16T20:51:23.028 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.028 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.028 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:23.028 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850327+0000", 2024-09-16T20:51:23.028 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9450000000000001e-06 2024-09-16T20:51:23.028 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.028 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.029 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:23.029 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850376+0000", 2024-09-16T20:51:23.029 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8257000000000002e-05 2024-09-16T20:51:23.029 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.029 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.029 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:23.029 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850385+0000", 2024-09-16T20:51:23.029 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9800000000000004e-06 2024-09-16T20:51:23.029 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.029 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.029 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:23.029 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852167+0000", 2024-09-16T20:51:23.029 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0017825659999999 2024-09-16T20:51:23.030 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.030 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.030 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:23.030 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852179+0000", 2024-09-16T20:51:23.030 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1975e-05 2024-09-16T20:51:23.030 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.030 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:23.030 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.030 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.030 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.030 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5138.0:9382 3.c 3.c25699cc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T20:51:23.030 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:51:23.031 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 116.12997747, 2024-09-16T20:51:23.031 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.99358404700000003, 2024-09-16T20:51:23.031 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:23.031 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:23.031 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:51:23.031 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:23.031 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5138", 2024-09-16T20:51:23.031 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:51:23.031 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 9382 2024-09-16T20:51:23.031 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.031 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:23.031 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.031 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:23.031 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:51:23.031 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.031 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.032 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.032 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:23.032 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:51:23.032 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.032 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.032 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.032 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:23.032 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858661+0000", 2024-09-16T20:51:23.032 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0200000000000002e-07 2024-09-16T20:51:23.032 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.032 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.032 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:23.032 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858664+0000", 2024-09-16T20:51:23.032 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.038e-06 2024-09-16T20:51:23.033 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.033 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.033 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:23.033 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858671+0000", 2024-09-16T20:51:23.033 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0110000000000003e-06 2024-09-16T20:51:23.033 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.033 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.033 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:23.033 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858677+0000", 2024-09-16T20:51:23.033 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0959999999999997e-06 2024-09-16T20:51:23.033 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.033 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.033 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:23.033 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858732+0000", 2024-09-16T20:51:23.034 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4543000000000003e-05 2024-09-16T20:51:23.034 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.034 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.034 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:23.034 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858739+0000", 2024-09-16T20:51:23.034 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5429999999999996e-06 2024-09-16T20:51:23.034 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.034 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.034 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:23.034 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852236+0000", 2024-09-16T20:51:23.034 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.993496453 2024-09-16T20:51:23.034 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.034 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.034 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:23.034 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852244+0000", 2024-09-16T20:51:23.035 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8610000000000004e-06 2024-09-16T20:51:23.035 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.035 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:23.035 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.035 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.035 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.035 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5763.0:4895 3.a 3:5384606c:::rbd_data.110e1b19a241.0000000000000018:head [read 0~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-16T20:51:23.035 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:45.598212+0000", 2024-09-16T20:51:23.035 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 97.390425429000004, 2024-09-16T20:51:23.035 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0375864829999999, 2024-09-16T20:51:23.035 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:23.035 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:23.035 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:23.035 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:23.035 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5763", 2024-09-16T20:51:23.036 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/623897567", 2024-09-16T20:51:23.036 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4895 2024-09-16T20:51:23.036 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.036 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:23.036 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.036 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:23.036 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598212+0000", 2024-09-16T20:51:23.036 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.036 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.036 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.036 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:23.036 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598212+0000", 2024-09-16T20:51:23.036 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.036 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.036 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.037 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:23.037 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598214+0000", 2024-09-16T20:51:23.037 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0090000000000002e-06 2024-09-16T20:51:23.037 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.037 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.037 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:23.037 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598222+0000", 2024-09-16T20:51:23.037 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6699999999999994e-06 2024-09-16T20:51:23.037 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.037 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.037 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:23.037 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598241+0000", 2024-09-16T20:51:23.037 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8896999999999999e-05 2024-09-16T20:51:23.037 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.038 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.038 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:23.038 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598250+0000", 2024-09-16T20:51:23.038 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6649999999999992e-06 2024-09-16T20:51:23.038 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.038 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.038 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:23.038 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598325+0000", 2024-09-16T20:51:23.038 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5189000000000001e-05 2024-09-16T20:51:23.038 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.038 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.038 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op must wait for map", 2024-09-16T20:51:23.038 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598333+0000", 2024-09-16T20:51:23.038 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.3170000000000002e-06 2024-09-16T20:51:23.038 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.039 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.039 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:23.039 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601355+0000", 2024-09-16T20:51:23.039 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0030215400000000001 2024-09-16T20:51:23.039 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.039 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.039 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:23.039 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601365+0000", 2024-09-16T20:51:23.039 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0215e-05 2024-09-16T20:51:23.039 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.039 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.039 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:23.039 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635506+0000", 2024-09-16T20:51:23.039 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0341408240000001 2024-09-16T20:51:23.039 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.039 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.040 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:23.040 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635655+0000", 2024-09-16T20:51:23.040 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000149412 2024-09-16T20:51:23.040 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.040 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.040 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:23.040 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635799+0000", 2024-09-16T20:51:23.040 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000143745 2024-09-16T20:51:23.040 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.040 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:23.040 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.040 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.040 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.040 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5763.0:4900 3.a 3:5384606c:::rbd_data.110e1b19a241.0000000000000018:head [read 8192~24576] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-16T20:51:23.040 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:51:23.041 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 97.390366952999997, 2024-09-16T20:51:23.041 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0379340800000001, 2024-09-16T20:51:23.041 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:23.041 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:23.041 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:23.041 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:23.041 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5763", 2024-09-16T20:51:23.041 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/623897567", 2024-09-16T20:51:23.041 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4900 2024-09-16T20:51:23.041 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.041 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:23.041 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.041 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:23.041 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:51:23.042 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.042 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.042 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.042 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:23.042 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:51:23.042 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.042 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.042 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.042 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:23.042 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:51:23.042 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4300000000000003e-07 2024-09-16T20:51:23.042 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.042 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.042 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:23.042 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598274+0000", 2024-09-16T20:51:23.042 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.976e-06 2024-09-16T20:51:23.043 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.043 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.043 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:23.043 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598282+0000", 2024-09-16T20:51:23.043 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0029999999999994e-06 2024-09-16T20:51:23.043 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.043 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.043 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:23.043 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598289+0000", 2024-09-16T20:51:23.043 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2389999999999999e-06 2024-09-16T20:51:23.043 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.043 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.043 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:23.043 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598430+0000", 2024-09-16T20:51:23.043 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014119600000000001 2024-09-16T20:51:23.044 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.044 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.044 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting_for_map not empty", 2024-09-16T20:51:23.044 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598435+0000", 2024-09-16T20:51:23.044 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3489999999999999e-06 2024-09-16T20:51:23.044 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.044 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.044 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:23.044 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601429+0000", 2024-09-16T20:51:23.044 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0029942720000000001 2024-09-16T20:51:23.044 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.044 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.044 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:23.044 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601438+0000", 2024-09-16T20:51:23.044 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4759999999999998e-06 2024-09-16T20:51:23.045 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.045 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.045 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:23.045 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635987+0000", 2024-09-16T20:51:23.045 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0345495010000001 2024-09-16T20:51:23.045 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.045 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.045 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:23.045 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.636056+0000", 2024-09-16T20:51:23.046 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8610999999999998e-05 2024-09-16T20:51:23.046 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.046 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.046 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:23.046 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.636205+0000", 2024-09-16T20:51:23.046 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014891399999999999 2024-09-16T20:51:23.046 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.046 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:23.046 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.046 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.046 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.046 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop(client.6231.0:2182 3.17 e136/81 3:edb5e5cd:::rbd_data.110e61479aee.0000000000000014:head v 136'791, mlcod=135'789)", 2024-09-16T20:51:23.046 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:22.735499+0000", 2024-09-16T20:51:23.046 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 60.253139011999998, 2024-09-16T20:51:23.046 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12369608, 2024-09-16T20:51:23.047 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:23.047 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:23.047 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:23.047 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:23.047 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.047 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:23.047 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735499+0000", 2024-09-16T20:51:23.047 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.047 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.047 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.047 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:23.047 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735499+0000", 2024-09-16T20:51:23.047 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.047 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.047 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.048 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:23.048 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735499+0000", 2024-09-16T20:51:23.048 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8099999999999999e-07 2024-09-16T20:51:23.048 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.048 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.048 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:23.048 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735506+0000", 2024-09-16T20:51:23.048 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3579999999999997e-06 2024-09-16T20:51:23.048 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.048 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.048 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:23.048 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735517+0000", 2024-09-16T20:51:23.048 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0869999999999999e-05 2024-09-16T20:51:23.048 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.048 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.049 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:23.049 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735533+0000", 2024-09-16T20:51:23.049 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5858e-05 2024-09-16T20:51:23.049 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.049 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.049 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:23.049 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735584+0000", 2024-09-16T20:51:23.049 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0470999999999998e-05 2024-09-16T20:51:23.049 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.049 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.049 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:23.049 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735607+0000", 2024-09-16T20:51:23.049 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2875999999999999e-05 2024-09-16T20:51:23.049 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.050 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.050 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:23.050 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.859144+0000", 2024-09-16T20:51:23.050 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.123537701 2024-09-16T20:51:23.050 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.050 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.050 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:23.050 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.859195+0000", 2024-09-16T20:51:23.050 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0664999999999997e-05 2024-09-16T20:51:23.050 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.050 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:23.050 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.050 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.050 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.050 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5204 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 2523136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:51:23.051 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.883660+0000", 2024-09-16T20:51:23.051 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 59.104977482000002, 2024-09-16T20:51:23.051 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052517638999999998, 2024-09-16T20:51:23.051 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:23.051 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:23.051 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:23.051 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:23.051 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:23.051 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:23.051 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5204 2024-09-16T20:51:23.051 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.051 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:23.051 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.052 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:23.052 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883660+0000", 2024-09-16T20:51:23.052 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.052 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.052 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:23.052 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883660+0000", 2024-09-16T20:51:23.052 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.052 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.052 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:23.052 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883661+0000", 2024-09-16T20:51:23.052 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9999999999999998e-07 2024-09-16T20:51:23.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.053 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.053 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:23.053 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883663+0000", 2024-09-16T20:51:23.053 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.4480000000000001e-06 2024-09-16T20:51:23.053 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.053 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.053 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:23.053 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883669+0000", 2024-09-16T20:51:23.053 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.071e-06 2024-09-16T20:51:23.053 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.053 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.053 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:23.053 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883675+0000", 2024-09-16T20:51:23.053 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.366e-06 2024-09-16T20:51:23.053 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.054 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.054 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:23.054 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.935970+0000", 2024-09-16T20:51:23.054 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052295346999999999 2024-09-16T20:51:23.054 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.054 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.054 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:23.054 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936064+0000", 2024-09-16T20:51:23.054 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4438000000000006e-05 2024-09-16T20:51:23.054 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.054 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.054 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:23.054 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936178+0000", 2024-09-16T20:51:23.054 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000113569 2024-09-16T20:51:23.055 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.055 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:23.055 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.055 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.055 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.055 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5205 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 2543616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:51:23.055 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.883745+0000", 2024-09-16T20:51:23.055 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 59.104892724000003, 2024-09-16T20:51:23.055 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052692112999999999, 2024-09-16T20:51:23.055 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:23.056 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:23.056 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:23.056 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:23.056 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:23.056 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:23.056 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5205 2024-09-16T20:51:23.056 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.056 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:23.056 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.056 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:23.056 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883745+0000", 2024-09-16T20:51:23.056 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.056 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.056 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.057 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:23.057 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883745+0000", 2024-09-16T20:51:23.057 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.057 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.057 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.057 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:23.057 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883746+0000", 2024-09-16T20:51:23.057 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2699999999999999e-07 2024-09-16T20:51:23.057 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.057 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.057 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:23.057 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883750+0000", 2024-09-16T20:51:23.057 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7050000000000001e-06 2024-09-16T20:51:23.057 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.058 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.058 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:23.058 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883769+0000", 2024-09-16T20:51:23.058 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9165e-05 2024-09-16T20:51:23.058 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.058 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.058 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:23.058 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883778+0000", 2024-09-16T20:51:23.058 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.6690000000000001e-06 2024-09-16T20:51:23.058 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.058 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.058 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:23.058 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936228+0000", 2024-09-16T20:51:23.058 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052449587999999998 2024-09-16T20:51:23.059 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.059 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.059 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:23.059 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936333+0000", 2024-09-16T20:51:23.059 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010511800000000001 2024-09-16T20:51:23.059 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.059 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.059 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:23.059 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936437+0000", 2024-09-16T20:51:23.059 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000104141 2024-09-16T20:51:23.059 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.059 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:23.059 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.059 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.060 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.060 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5206 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 1167360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:51:23.060 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.883802+0000", 2024-09-16T20:51:23.060 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 59.104836229999997, 2024-09-16T20:51:23.060 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052800780999999998, 2024-09-16T20:51:23.060 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:23.060 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:23.060 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:23.060 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:23.060 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:23.060 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:23.060 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5206 2024-09-16T20:51:23.060 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.060 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:23.061 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.061 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:23.061 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883802+0000", 2024-09-16T20:51:23.061 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.061 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.061 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.061 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:23.061 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883802+0000", 2024-09-16T20:51:23.061 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.061 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.061 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.061 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:23.061 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883933+0000", 2024-09-16T20:51:23.061 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013182999999999999 2024-09-16T20:51:23.062 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.062 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.062 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:23.062 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883937+0000", 2024-09-16T20:51:23.062 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.771e-06 2024-09-16T20:51:23.062 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.062 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.062 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:23.062 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883953+0000", 2024-09-16T20:51:23.062 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5409000000000001e-05 2024-09-16T20:51:23.062 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.062 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.062 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:23.062 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883959+0000", 2024-09-16T20:51:23.062 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0700000000000003e-06 2024-09-16T20:51:23.063 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.063 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.063 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:23.063 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936476+0000", 2024-09-16T20:51:23.063 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052517434000000002 2024-09-16T20:51:23.063 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.063 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.063 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:23.063 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936529+0000", 2024-09-16T20:51:23.063 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2969000000000001e-05 2024-09-16T20:51:23.063 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.063 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.063 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:23.064 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936602+0000", 2024-09-16T20:51:23.064 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3298000000000002e-05 2024-09-16T20:51:23.064 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.064 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:23.064 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.064 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.064 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.064 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5207 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 1904640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:51:23.064 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.883980+0000", 2024-09-16T20:51:23.064 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 59.104658020999999, 2024-09-16T20:51:23.064 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052759922000000001, 2024-09-16T20:51:23.064 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:23.064 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:23.064 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:23.064 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:23.065 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:23.065 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:23.065 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5207 2024-09-16T20:51:23.065 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.065 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:23.065 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.065 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:23.065 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883980+0000", 2024-09-16T20:51:23.065 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.065 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.065 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.065 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:23.065 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883980+0000", 2024-09-16T20:51:23.065 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.066 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.066 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.066 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:23.066 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884126+0000", 2024-09-16T20:51:23.066 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014577399999999999 2024-09-16T20:51:23.066 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.066 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.066 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:23.066 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884130+0000", 2024-09-16T20:51:23.066 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9729999999999999e-06 2024-09-16T20:51:23.066 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.066 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.066 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:23.066 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884139+0000", 2024-09-16T20:51:23.066 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8660000000000001e-06 2024-09-16T20:51:23.067 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.067 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.067 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:23.067 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884144+0000", 2024-09-16T20:51:23.067 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3669999999999999e-06 2024-09-16T20:51:23.067 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.067 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.067 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:23.067 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936635+0000", 2024-09-16T20:51:23.067 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052491705 2024-09-16T20:51:23.067 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.067 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.068 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:23.068 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936679+0000", 2024-09-16T20:51:23.068 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3105999999999997e-05 2024-09-16T20:51:23.068 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.068 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.068 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:23.068 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936740+0000", 2024-09-16T20:51:23.068 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1130999999999995e-05 2024-09-16T20:51:23.068 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.068 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:23.068 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.068 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.068 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.068 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5208 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 131072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:51:23.068 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.884165+0000", 2024-09-16T20:51:23.069 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 59.104472725000001, 2024-09-16T20:51:23.069 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052746603000000003, 2024-09-16T20:51:23.069 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:23.069 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:23.069 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:23.069 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:23.069 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:23.069 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:23.069 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5208 2024-09-16T20:51:23.069 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.069 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:23.069 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.069 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:23.070 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884165+0000", 2024-09-16T20:51:23.070 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.070 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.070 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.070 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:23.070 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884165+0000", 2024-09-16T20:51:23.070 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.070 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.070 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.070 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:23.070 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884463+0000", 2024-09-16T20:51:23.070 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00029801699999999998 2024-09-16T20:51:23.070 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.070 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.070 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:23.071 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884467+0000", 2024-09-16T20:51:23.071 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8210000000000003e-06 2024-09-16T20:51:23.071 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.071 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.071 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:23.071 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884477+0000", 2024-09-16T20:51:23.071 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.7030000000000001e-06 2024-09-16T20:51:23.071 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.071 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.071 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:23.071 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884485+0000", 2024-09-16T20:51:23.071 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9710000000000004e-06 2024-09-16T20:51:23.072 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.072 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.072 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:23.072 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936785+0000", 2024-09-16T20:51:23.072 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052300665000000003 2024-09-16T20:51:23.072 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.072 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.072 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:23.072 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936851+0000", 2024-09-16T20:51:23.072 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6159000000000001e-05 2024-09-16T20:51:23.072 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.072 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.072 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:23.072 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936912+0000", 2024-09-16T20:51:23.073 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0266999999999997e-05 2024-09-16T20:51:23.073 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.073 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:23.073 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.073 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.073 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.073 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5209 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 1925120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:51:23.073 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.884504+0000", 2024-09-16T20:51:23.073 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 59.104133840000003, 2024-09-16T20:51:23.073 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052575445999999998, 2024-09-16T20:51:23.073 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:23.073 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:23.073 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:23.073 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:23.073 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:23.074 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:23.074 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5209 2024-09-16T20:51:23.074 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.074 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:23.074 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.074 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:23.074 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884504+0000", 2024-09-16T20:51:23.074 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.074 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.074 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.074 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:23.074 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884504+0000", 2024-09-16T20:51:23.074 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.074 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.074 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.075 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:23.075 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884741+0000", 2024-09-16T20:51:23.075 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00023679899999999999 2024-09-16T20:51:23.075 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.075 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.075 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:23.075 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884747+0000", 2024-09-16T20:51:23.075 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0009999999999998e-06 2024-09-16T20:51:23.075 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.075 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.075 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:23.075 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884764+0000", 2024-09-16T20:51:23.075 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7371999999999999e-05 2024-09-16T20:51:23.075 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.075 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.076 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:23.076 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884772+0000", 2024-09-16T20:51:23.076 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1349999999999992e-06 2024-09-16T20:51:23.076 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.076 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.076 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:23.076 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936956+0000", 2024-09-16T20:51:23.076 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052183754999999998 2024-09-16T20:51:23.076 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.076 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.076 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:23.076 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937003+0000", 2024-09-16T20:51:23.076 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6925000000000002e-05 2024-09-16T20:51:23.076 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.076 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.077 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:23.077 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937079+0000", 2024-09-16T20:51:23.077 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6458999999999994e-05 2024-09-16T20:51:23.077 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.077 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:23.077 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.077 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.077 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.077 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:30174 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_remove in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e176)", 2024-09-16T20:51:23.077 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:51:03.615665+0000", 2024-09-16T20:51:23.077 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 19.372972320999999, 2024-09-16T20:51:23.077 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.96831816599999998, 2024-09-16T20:51:23.078 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:23.078 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:23.078 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:23.078 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:23.078 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:51:23.078 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:23.078 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 30174 2024-09-16T20:51:23.078 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.078 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:23.078 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.078 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:23.078 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615665+0000", 2024-09-16T20:51:23.078 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.078 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.079 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.079 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:23.079 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615665+0000", 2024-09-16T20:51:23.079 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.079 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.079 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.079 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:23.079 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615667+0000", 2024-09-16T20:51:23.079 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0020000000000001e-06 2024-09-16T20:51:23.079 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.079 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.079 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:23.079 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615675+0000", 2024-09-16T20:51:23.079 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5120000000000002e-06 2024-09-16T20:51:23.080 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.080 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.080 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:23.080 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615692+0000", 2024-09-16T20:51:23.080 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.6977e-05 2024-09-16T20:51:23.080 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.080 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.080 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:23.080 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615701+0000", 2024-09-16T20:51:23.080 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6119999999999999e-06 2024-09-16T20:51:23.080 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.080 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.080 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:23.080 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615782+0000", 2024-09-16T20:51:23.080 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1804000000000005e-05 2024-09-16T20:51:23.081 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.081 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.081 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:23.081 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615795+0000", 2024-09-16T20:51:23.081 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2464e-05 2024-09-16T20:51:23.081 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.081 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.081 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:23.081 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.580071+0000", 2024-09-16T20:51:23.081 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.96427590100000005 2024-09-16T20:51:23.081 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.081 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.081 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:23.081 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.580247+0000", 2024-09-16T20:51:23.081 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000175754 2024-09-16T20:51:23.082 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.082 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.082 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 2", 2024-09-16T20:51:23.082 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.580436+0000", 2024-09-16T20:51:23.082 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00018945299999999999 2024-09-16T20:51:23.082 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.082 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.082 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:23.082 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.581045+0000", 2024-09-16T20:51:23.082 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00060937600000000001 2024-09-16T20:51:23.082 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.082 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.082 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:23.082 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.583883+0000", 2024-09-16T20:51:23.082 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0028372900000000001 2024-09-16T20:51:23.082 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.083 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.083 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:23.083 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.583944+0000", 2024-09-16T20:51:23.083 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1848999999999997e-05 2024-09-16T20:51:23.083 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.083 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.083 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:23.083 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.583984+0000", 2024-09-16T20:51:23.083 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9171999999999997e-05 2024-09-16T20:51:23.083 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.083 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:23.083 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.083 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.083 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.083 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:30201 3.f 3:f6903a53:::rbd_object_map.110eca9b190a.000000000000001a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e177)", 2024-09-16T20:51:23.083 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:51:04.603586+0000", 2024-09-16T20:51:23.084 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 18.385052214000002, 2024-09-16T20:51:23.084 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.98028146800000004, 2024-09-16T20:51:23.084 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:23.084 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:23.084 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:23.084 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:23.084 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:51:23.084 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:23.084 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 30201 2024-09-16T20:51:23.084 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.084 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:23.084 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.084 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:23.084 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603586+0000", 2024-09-16T20:51:23.084 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.085 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.085 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.085 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:23.085 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603586+0000", 2024-09-16T20:51:23.085 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.085 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.085 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.085 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:23.085 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603587+0000", 2024-09-16T20:51:23.085 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.519e-06 2024-09-16T20:51:23.085 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.085 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.085 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:23.085 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603593+0000", 2024-09-16T20:51:23.085 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1449999999999996e-06 2024-09-16T20:51:23.086 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.086 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.086 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:23.086 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603606+0000", 2024-09-16T20:51:23.086 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2797e-05 2024-09-16T20:51:23.086 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.086 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.086 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:23.086 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603612+0000", 2024-09-16T20:51:23.086 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8710000000000002e-06 2024-09-16T20:51:23.086 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.086 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.086 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:23.086 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603675+0000", 2024-09-16T20:51:23.086 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2728e-05 2024-09-16T20:51:23.086 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.087 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.087 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:23.087 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603687+0000", 2024-09-16T20:51:23.087 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.253e-05 2024-09-16T20:51:23.087 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.087 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.087 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:23.087 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:05.583358+0000", 2024-09-16T20:51:23.087 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.97967074600000004 2024-09-16T20:51:23.087 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.087 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.087 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:23.087 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:05.583515+0000", 2024-09-16T20:51:23.087 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00015755700000000001 2024-09-16T20:51:23.087 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.088 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.088 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:23.088 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:05.583867+0000", 2024-09-16T20:51:23.088 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00035157500000000002 2024-09-16T20:51:23.088 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.088 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:23.088 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.088 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.088 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:23.088 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:23.089 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:51:23.090 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:23.089+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:23.191 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:23.190+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:23.238 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:23.238 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:51:23.238 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:51:23.238 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:23.247 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:51:23.292 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:23.291+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:23.353 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:23.353 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:51:23.353 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:51:23.353 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:51:23.353 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:23.361 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:51:23.393 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:23.391+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:23.470 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:23.470 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:51:23.470 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:51:23.470 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:51:23.470 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.470 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:51:23.471 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:51:23.471 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 210.21412310299999, 2024-09-16T20:51:23.471 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67502346999999996, 2024-09-16T20:51:23.471 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:23.471 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:23.471 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:23.471 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:23.471 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:51:23.471 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:51:23.471 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 1 2024-09-16T20:51:23.471 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.472 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:23.472 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.472 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:23.472 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:51:23.472 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.472 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.472 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.472 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:23.472 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:51:23.472 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.472 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.472 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.472 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:23.473 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254128+0000", 2024-09-16T20:51:23.473 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.463e-06 2024-09-16T20:51:23.473 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.473 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.473 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:23.473 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254135+0000", 2024-09-16T20:51:23.473 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1579999999999999e-06 2024-09-16T20:51:23.473 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.473 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.473 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:23.473 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254146+0000", 2024-09-16T20:51:23.474 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1341e-05 2024-09-16T20:51:23.474 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.474 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.474 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:23.474 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254155+0000", 2024-09-16T20:51:23.474 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.1549999999999996e-06 2024-09-16T20:51:23.474 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.474 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.474 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:23.474 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254221+0000", 2024-09-16T20:51:23.474 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5470999999999997e-05 2024-09-16T20:51:23.474 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.475 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.475 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:23.475 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254231+0000", 2024-09-16T20:51:23.475 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0592e-05 2024-09-16T20:51:23.475 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.475 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.475 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:23.475 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926711+0000", 2024-09-16T20:51:23.475 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67247946599999997 2024-09-16T20:51:23.475 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.475 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.475 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:23.475 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926910+0000", 2024-09-16T20:51:23.476 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019934 2024-09-16T20:51:23.476 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.476 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.476 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:51:23.476 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.927159+0000", 2024-09-16T20:51:23.476 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00024876599999999998 2024-09-16T20:51:23.476 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.476 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.476 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:23.476 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.928797+0000", 2024-09-16T20:51:23.476 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001638308 2024-09-16T20:51:23.476 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.476 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.476 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:23.477 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929026+0000", 2024-09-16T20:51:23.477 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00022876099999999999 2024-09-16T20:51:23.477 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.477 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.477 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:23.477 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929102+0000", 2024-09-16T20:51:23.477 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6184000000000001e-05 2024-09-16T20:51:23.477 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.477 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.477 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:23.477 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929150+0000", 2024-09-16T20:51:23.477 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7465e-05 2024-09-16T20:51:23.477 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.478 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:23.478 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.478 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.478 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.478 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3574 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:23.478 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:51:23.478 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 153.986959571, 2024-09-16T20:51:23.478 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053579441999999998, 2024-09-16T20:51:23.478 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:23.478 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:23.478 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:23.478 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:23.478 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:23.479 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:23.479 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3574 2024-09-16T20:51:23.479 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.479 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:23.479 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.479 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:23.479 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:51:23.479 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.479 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.479 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.479 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:23.479 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:51:23.479 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.479 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.480 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.480 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:23.480 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:51:23.480 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2e-07 2024-09-16T20:51:23.480 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.480 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.481 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:23.481 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481292+0000", 2024-09-16T20:51:23.481 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2630000000000002e-06 2024-09-16T20:51:23.481 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.481 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.481 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:23.481 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481297+0000", 2024-09-16T20:51:23.481 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.296e-06 2024-09-16T20:51:23.481 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.482 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.482 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:23.482 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481300+0000", 2024-09-16T20:51:23.482 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7019999999999999e-06 2024-09-16T20:51:23.482 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.482 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.482 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:23.482 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525533+0000", 2024-09-16T20:51:23.482 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.044232318 2024-09-16T20:51:23.482 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.482 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.482 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:23.482 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525619+0000", 2024-09-16T20:51:23.482 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5892999999999999e-05 2024-09-16T20:51:23.483 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.483 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.483 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:23.483 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525987+0000", 2024-09-16T20:51:23.483 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00036836099999999999 2024-09-16T20:51:23.483 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.483 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.483 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:23.483 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531779+0000", 2024-09-16T20:51:23.483 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0057918240000000001 2024-09-16T20:51:23.483 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.483 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.483 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:23.483 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534747+0000", 2024-09-16T20:51:23.484 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0029681600000000001 2024-09-16T20:51:23.484 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.484 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.484 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:23.484 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534832+0000", 2024-09-16T20:51:23.484 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5180999999999996e-05 2024-09-16T20:51:23.484 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.484 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.484 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:23.484 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534869+0000", 2024-09-16T20:51:23.484 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7024000000000003e-05 2024-09-16T20:51:23.484 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.484 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:23.484 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.485 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.485 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.485 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3575 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:23.485 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:51:23.485 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 153.98694483099999, 2024-09-16T20:51:23.485 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053801463000000001, 2024-09-16T20:51:23.485 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:23.485 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:23.485 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:23.485 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:23.485 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:23.485 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:23.485 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3575 2024-09-16T20:51:23.486 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.486 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:23.486 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.486 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:23.486 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:51:23.486 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.486 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.486 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.486 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:23.486 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:51:23.486 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.486 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.486 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.486 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:23.487 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481305+0000", 2024-09-16T20:51:23.487 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5400000000000002e-07 2024-09-16T20:51:23.487 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.487 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.487 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:23.487 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481307+0000", 2024-09-16T20:51:23.487 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7800000000000001e-06 2024-09-16T20:51:23.487 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.487 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.487 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:23.487 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481312+0000", 2024-09-16T20:51:23.487 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6349999999999997e-06 2024-09-16T20:51:23.487 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.487 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.488 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:23.488 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481315+0000", 2024-09-16T20:51:23.488 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7829999999999999e-06 2024-09-16T20:51:23.488 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.488 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.488 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:23.488 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.526876+0000", 2024-09-16T20:51:23.488 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.045561070000000002 2024-09-16T20:51:23.488 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.488 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.488 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:23.489 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.526954+0000", 2024-09-16T20:51:23.489 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8108000000000005e-05 2024-09-16T20:51:23.489 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.489 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.489 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:23.489 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527282+0000", 2024-09-16T20:51:23.489 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000327483 2024-09-16T20:51:23.489 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.489 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.489 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:23.489 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531784+0000", 2024-09-16T20:51:23.490 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0045020169999999997 2024-09-16T20:51:23.490 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.490 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.490 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:23.490 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534921+0000", 2024-09-16T20:51:23.490 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.003137791 2024-09-16T20:51:23.490 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.490 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.490 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:23.490 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534976+0000", 2024-09-16T20:51:23.490 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4953000000000001e-05 2024-09-16T20:51:23.490 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.490 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.490 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:23.491 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535106+0000", 2024-09-16T20:51:23.491 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000129589 2024-09-16T20:51:23.491 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.491 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:23.491 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.491 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.491 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.491 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3576 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:23.491 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:51:23.491 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 153.986930919, 2024-09-16T20:51:23.491 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053955436000000002, 2024-09-16T20:51:23.491 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:23.491 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:23.491 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:23.492 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:23.492 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:23.492 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:23.492 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3576 2024-09-16T20:51:23.492 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.492 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:23.492 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.492 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:23.492 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:51:23.492 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.492 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.492 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.492 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:23.493 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:51:23.493 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.493 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.493 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.493 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:23.493 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481319+0000", 2024-09-16T20:51:23.493 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.53e-07 2024-09-16T20:51:23.493 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.493 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.493 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:23.493 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481320+0000", 2024-09-16T20:51:23.493 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5230000000000001e-06 2024-09-16T20:51:23.493 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.494 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.494 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:23.494 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481330+0000", 2024-09-16T20:51:23.494 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0190000000000001e-05 2024-09-16T20:51:23.494 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.494 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.494 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:23.494 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481333+0000", 2024-09-16T20:51:23.494 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.151e-06 2024-09-16T20:51:23.494 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.494 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.494 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:23.494 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527689+0000", 2024-09-16T20:51:23.495 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046355192000000003 2024-09-16T20:51:23.495 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.495 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.495 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:23.495 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527764+0000", 2024-09-16T20:51:23.495 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5776000000000001e-05 2024-09-16T20:51:23.495 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.495 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.495 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:23.495 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528141+0000", 2024-09-16T20:51:23.495 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00037647599999999999 2024-09-16T20:51:23.495 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.495 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.495 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:23.496 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531788+0000", 2024-09-16T20:51:23.496 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0036470679999999998 2024-09-16T20:51:23.496 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.496 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.496 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:23.496 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535174+0000", 2024-09-16T20:51:23.496 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0033862670000000001 2024-09-16T20:51:23.496 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.496 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.496 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:23.496 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535236+0000", 2024-09-16T20:51:23.496 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1797999999999993e-05 2024-09-16T20:51:23.496 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.497 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.497 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:23.497 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535274+0000", 2024-09-16T20:51:23.497 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7741999999999999e-05 2024-09-16T20:51:23.497 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.497 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:23.497 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.497 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.497 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.497 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3577 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:23.497 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:51:23.497 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 153.98690931900001, 2024-09-16T20:51:23.497 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054073807000000002, 2024-09-16T20:51:23.497 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:23.498 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:23.498 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:23.498 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:23.498 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:23.498 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:23.498 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3577 2024-09-16T20:51:23.498 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.498 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:23.498 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.498 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:23.498 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:51:23.498 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.498 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.498 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.499 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:23.499 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:51:23.499 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.499 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.499 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.499 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:23.499 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:51:23.499 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.96e-07 2024-09-16T20:51:23.499 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.499 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.499 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:23.499 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481342+0000", 2024-09-16T20:51:23.500 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.714e-06 2024-09-16T20:51:23.500 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.500 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.500 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:23.500 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481349+0000", 2024-09-16T20:51:23.500 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4930000000000001e-06 2024-09-16T20:51:23.500 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.500 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.500 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:23.500 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481353+0000", 2024-09-16T20:51:23.500 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4879999999999999e-06 2024-09-16T20:51:23.500 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.500 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.500 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:23.501 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528619+0000", 2024-09-16T20:51:23.501 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047266351999999998 2024-09-16T20:51:23.501 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.501 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.501 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:23.501 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528706+0000", 2024-09-16T20:51:23.501 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6796e-05 2024-09-16T20:51:23.501 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.501 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.501 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:23.501 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529148+0000", 2024-09-16T20:51:23.501 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00044182999999999999 2024-09-16T20:51:23.501 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.502 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.502 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:23.502 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531792+0000", 2024-09-16T20:51:23.502 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0026443119999999998 2024-09-16T20:51:23.502 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.502 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.502 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:23.502 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535319+0000", 2024-09-16T20:51:23.502 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0035266439999999998 2024-09-16T20:51:23.502 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.502 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.502 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:23.502 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535372+0000", 2024-09-16T20:51:23.502 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3207000000000001e-05 2024-09-16T20:51:23.503 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.503 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.503 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:23.503 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535414+0000", 2024-09-16T20:51:23.503 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1675000000000003e-05 2024-09-16T20:51:23.503 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.503 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:23.503 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.503 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.503 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.503 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3578 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:23.503 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:51:23.503 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 153.98688815, 2024-09-16T20:51:23.504 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054189586999999997, 2024-09-16T20:51:23.504 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:23.504 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:23.504 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:23.504 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:23.504 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:23.504 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:23.504 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3578 2024-09-16T20:51:23.504 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.504 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:23.504 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.504 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:23.504 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:51:23.504 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.505 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.505 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.505 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:23.505 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:51:23.505 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.505 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.505 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.505 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:23.505 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:51:23.505 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0800000000000001e-07 2024-09-16T20:51:23.506 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.506 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.506 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:23.506 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481363+0000", 2024-09-16T20:51:23.506 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0339999999999999e-06 2024-09-16T20:51:23.506 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.506 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.506 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:23.506 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481371+0000", 2024-09-16T20:51:23.506 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4440000000000002e-06 2024-09-16T20:51:23.506 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.506 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.506 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:23.506 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481374+0000", 2024-09-16T20:51:23.507 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2049999999999998e-06 2024-09-16T20:51:23.507 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.507 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.507 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:23.507 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529555+0000", 2024-09-16T20:51:23.507 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048181202999999999 2024-09-16T20:51:23.507 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.507 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.507 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:23.507 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529616+0000", 2024-09-16T20:51:23.507 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0912000000000002e-05 2024-09-16T20:51:23.507 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.507 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.508 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:23.508 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529939+0000", 2024-09-16T20:51:23.508 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00032269800000000001 2024-09-16T20:51:23.508 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.508 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.508 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:23.508 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532602+0000", 2024-09-16T20:51:23.508 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0026634940000000002 2024-09-16T20:51:23.508 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.508 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.508 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:23.508 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535465+0000", 2024-09-16T20:51:23.508 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.002862149 2024-09-16T20:51:23.509 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.509 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.509 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:23.509 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535516+0000", 2024-09-16T20:51:23.509 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1696000000000001e-05 2024-09-16T20:51:23.509 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.509 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.509 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:23.509 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535551+0000", 2024-09-16T20:51:23.509 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4443999999999998e-05 2024-09-16T20:51:23.509 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.509 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:23.509 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.509 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.510 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.510 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3579 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:23.510 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:51:23.510 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 153.986869711, 2024-09-16T20:51:23.510 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054295151, 2024-09-16T20:51:23.510 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:23.510 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:23.510 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:23.510 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:23.511 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:23.511 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:23.511 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3579 2024-09-16T20:51:23.511 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.511 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:23.511 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.511 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:23.511 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:51:23.511 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.511 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.511 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.511 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:23.511 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:51:23.512 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.512 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.512 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.512 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:23.512 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:51:23.512 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3300000000000001e-07 2024-09-16T20:51:23.512 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.512 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.512 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:23.512 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481381+0000", 2024-09-16T20:51:23.512 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.649e-06 2024-09-16T20:51:23.512 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.512 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.512 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:23.513 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481387+0000", 2024-09-16T20:51:23.513 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4519999999999996e-06 2024-09-16T20:51:23.513 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.513 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.513 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:23.513 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481390+0000", 2024-09-16T20:51:23.513 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0170000000000001e-06 2024-09-16T20:51:23.513 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.513 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.513 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:23.513 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530756+0000", 2024-09-16T20:51:23.513 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049365882999999999 2024-09-16T20:51:23.513 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.513 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.514 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:23.514 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530845+0000", 2024-09-16T20:51:23.514 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9197000000000007e-05 2024-09-16T20:51:23.514 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.514 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.514 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:23.514 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531191+0000", 2024-09-16T20:51:23.514 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00034562500000000001 2024-09-16T20:51:23.514 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.514 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.514 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:23.514 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534078+0000", 2024-09-16T20:51:23.514 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0028878179999999999 2024-09-16T20:51:23.515 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.515 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.515 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:23.515 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535589+0000", 2024-09-16T20:51:23.515 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00151085 2024-09-16T20:51:23.515 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.515 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.515 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:23.515 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535639+0000", 2024-09-16T20:51:23.515 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9697000000000002e-05 2024-09-16T20:51:23.515 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.515 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.515 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:23.515 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535675+0000", 2024-09-16T20:51:23.516 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5729999999999998e-05 2024-09-16T20:51:23.516 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.516 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:23.516 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.516 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.516 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.516 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3580 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:23.516 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:51:23.516 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 153.98685607900001, 2024-09-16T20:51:23.516 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054397361999999998, 2024-09-16T20:51:23.516 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:23.516 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:23.516 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:23.517 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:23.517 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:23.517 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:23.517 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3580 2024-09-16T20:51:23.517 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.517 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:23.517 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.517 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:23.517 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:51:23.517 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.517 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.517 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.517 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:23.517 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:51:23.518 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.518 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.518 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.518 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:23.518 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:51:23.518 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2600000000000001e-07 2024-09-16T20:51:23.518 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.518 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.518 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:23.518 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481395+0000", 2024-09-16T20:51:23.519 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.544e-06 2024-09-16T20:51:23.519 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.519 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.519 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:23.519 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481404+0000", 2024-09-16T20:51:23.519 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8850000000000005e-06 2024-09-16T20:51:23.519 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.519 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.519 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:23.519 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481407+0000", 2024-09-16T20:51:23.519 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1540000000000002e-06 2024-09-16T20:51:23.519 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.519 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.520 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:23.520 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531823+0000", 2024-09-16T20:51:23.520 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050415637999999999 2024-09-16T20:51:23.520 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.520 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.520 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:23.520 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531895+0000", 2024-09-16T20:51:23.520 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2179000000000001e-05 2024-09-16T20:51:23.520 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.520 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.520 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:23.520 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532236+0000", 2024-09-16T20:51:23.520 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00034073799999999999 2024-09-16T20:51:23.520 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.521 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.521 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:23.521 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534083+0000", 2024-09-16T20:51:23.521 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001847298 2024-09-16T20:51:23.521 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.521 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.521 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:23.521 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535714+0000", 2024-09-16T20:51:23.521 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0016316810000000001 2024-09-16T20:51:23.521 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.522 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.522 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:23.522 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535759+0000", 2024-09-16T20:51:23.522 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4478999999999999e-05 2024-09-16T20:51:23.522 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.522 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.522 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:23.522 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535791+0000", 2024-09-16T20:51:23.522 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.154e-05 2024-09-16T20:51:23.522 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.522 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:23.522 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.522 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.523 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.523 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3581 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:23.523 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:51:23.523 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 153.986834082, 2024-09-16T20:51:23.523 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.057094403000000002, 2024-09-16T20:51:23.523 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:23.523 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:23.523 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:23.523 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:23.523 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:23.523 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:23.523 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3581 2024-09-16T20:51:23.523 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.523 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:23.524 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.524 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:23.524 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:51:23.524 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.524 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.524 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.524 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:23.524 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:51:23.524 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.524 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.524 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.524 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:23.525 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:51:23.525 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8799999999999998e-07 2024-09-16T20:51:23.525 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.525 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.525 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:23.525 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481418+0000", 2024-09-16T20:51:23.525 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1859999999999999e-06 2024-09-16T20:51:23.525 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.525 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.525 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:23.525 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481423+0000", 2024-09-16T20:51:23.525 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8279999999999999e-06 2024-09-16T20:51:23.525 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.525 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.526 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:23.526 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481426+0000", 2024-09-16T20:51:23.526 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0110000000000001e-06 2024-09-16T20:51:23.526 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.526 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.526 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:23.526 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532631+0000", 2024-09-16T20:51:23.526 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051204514999999999 2024-09-16T20:51:23.526 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.526 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.526 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:23.526 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.533381+0000", 2024-09-16T20:51:23.526 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00075029000000000003 2024-09-16T20:51:23.526 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.527 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.527 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:23.527 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.533739+0000", 2024-09-16T20:51:23.527 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00035773100000000002 2024-09-16T20:51:23.527 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.527 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.527 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:23.527 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535123+0000", 2024-09-16T20:51:23.527 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0013836160000000001 2024-09-16T20:51:23.527 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.527 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.527 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:23.527 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538413+0000", 2024-09-16T20:51:23.528 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0032902510000000001 2024-09-16T20:51:23.528 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.528 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.528 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:23.528 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538470+0000", 2024-09-16T20:51:23.528 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6697999999999998e-05 2024-09-16T20:51:23.528 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.528 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.528 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:23.528 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538510+0000", 2024-09-16T20:51:23.528 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9988999999999999e-05 2024-09-16T20:51:23.529 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.529 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:23.529 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.529 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.529 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.529 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:12732 3.d 3:b752444a:::rbd_header.110e747f0779:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-16T20:51:23.529 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:51:23.529 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 138.69976150400001, 2024-09-16T20:51:23.529 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.065302467000000003, 2024-09-16T20:51:23.529 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:23.529 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:23.530 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:23.530 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:23.530 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:51:23.530 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:51:23.530 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 12732 2024-09-16T20:51:23.530 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.530 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:23.530 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.530 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:23.530 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:51:23.530 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.530 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.530 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.531 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:23.531 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:51:23.531 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.531 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.531 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.531 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:23.531 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768490+0000", 2024-09-16T20:51:23.531 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2859999999999998e-06 2024-09-16T20:51:23.531 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.531 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.531 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:23.531 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768501+0000", 2024-09-16T20:51:23.531 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1226999999999999e-05 2024-09-16T20:51:23.531 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.531 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.532 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:23.532 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768517+0000", 2024-09-16T20:51:23.532 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.592e-05 2024-09-16T20:51:23.532 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.532 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.532 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:23.532 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768528+0000", 2024-09-16T20:51:23.532 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0689999999999999e-05 2024-09-16T20:51:23.532 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.532 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.532 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:23.532 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768584+0000", 2024-09-16T20:51:23.532 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5889e-05 2024-09-16T20:51:23.533 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.533 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.533 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:23.533 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768720+0000", 2024-09-16T20:51:23.533 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000136718 2024-09-16T20:51:23.533 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.533 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.533 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:23.533 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.769066+0000", 2024-09-16T20:51:23.533 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000346013 2024-09-16T20:51:23.533 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.533 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.533 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:23.534 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833659+0000", 2024-09-16T20:51:23.534 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.064592822999999994 2024-09-16T20:51:23.534 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.534 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.534 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:23.534 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833719+0000", 2024-09-16T20:51:23.534 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9441000000000002e-05 2024-09-16T20:51:23.534 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.534 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.534 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:23.534 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833758+0000", 2024-09-16T20:51:23.534 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9122000000000003e-05 2024-09-16T20:51:23.534 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.534 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.535 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:23.535 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833790+0000", 2024-09-16T20:51:23.535 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2338000000000001e-05 2024-09-16T20:51:23.535 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.535 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:23.535 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.535 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.535 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.535 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.5089.0:12732 3.d e73/23 ondisk, result = 0)", 2024-09-16T20:51:23.535 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:51:23.535 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 138.698159334, 2024-09-16T20:51:23.535 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.063706456999999994, 2024-09-16T20:51:23.536 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:23.536 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:23.536 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:23.536 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:23.536 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.536 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:23.536 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:51:23.536 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.536 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.536 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.536 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:23.536 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:51:23.536 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.537 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.537 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.537 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:23.537 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770091+0000", 2024-09-16T20:51:23.537 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.53e-06 2024-09-16T20:51:23.537 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.537 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.537 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:23.537 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770104+0000", 2024-09-16T20:51:23.537 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2203e-05 2024-09-16T20:51:23.537 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.537 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.538 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:23.538 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833637+0000", 2024-09-16T20:51:23.538 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.063533201999999997 2024-09-16T20:51:23.538 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.538 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.538 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:23.538 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833655+0000", 2024-09-16T20:51:23.538 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8369e-05 2024-09-16T20:51:23.538 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.538 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.538 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:23.538 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833707+0000", 2024-09-16T20:51:23.539 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2061999999999997e-05 2024-09-16T20:51:23.539 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.539 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.539 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:23.539 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833715+0000", 2024-09-16T20:51:23.539 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5179999999999998e-06 2024-09-16T20:51:23.539 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.539 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.539 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:23.539 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833796+0000", 2024-09-16T20:51:23.539 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1573e-05 2024-09-16T20:51:23.539 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.539 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:23.539 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.540 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.540 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.540 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3286 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 245760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:23.540 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:51:23.540 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 93.427097396999997, 2024-09-16T20:51:23.540 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052523564000000002, 2024-09-16T20:51:23.540 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:23.540 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:23.540 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:23.540 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:23.540 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:51:23.540 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:23.540 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3286 2024-09-16T20:51:23.541 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.541 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:23.541 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.541 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:23.541 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:51:23.541 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.541 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.541 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.541 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:23.541 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:51:23.541 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.542 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.542 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.542 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:23.542 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:51:23.542 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3900000000000001e-07 2024-09-16T20:51:23.542 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.542 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.542 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:23.542 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041153+0000", 2024-09-16T20:51:23.542 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.119e-06 2024-09-16T20:51:23.542 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.542 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.542 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:23.542 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041156+0000", 2024-09-16T20:51:23.543 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0000000000000001e-06 2024-09-16T20:51:23.543 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.543 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.543 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:23.543 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041159+0000", 2024-09-16T20:51:23.543 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5859999999999999e-06 2024-09-16T20:51:23.543 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.543 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.543 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:23.543 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093529+0000", 2024-09-16T20:51:23.543 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052370591000000001 2024-09-16T20:51:23.543 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.543 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.544 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:23.544 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093570+0000", 2024-09-16T20:51:23.544 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0161999999999999e-05 2024-09-16T20:51:23.544 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.544 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.544 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:23.544 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093675+0000", 2024-09-16T20:51:23.544 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010586699999999999 2024-09-16T20:51:23.544 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.544 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:23.544 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.544 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.544 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.545 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3287 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 250880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:23.545 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:51:23.545 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 93.427087348000001, 2024-09-16T20:51:23.545 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052781203999999998, 2024-09-16T20:51:23.545 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:23.548 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:23.548 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:23.548 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:23.548 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:51:23.548 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:23.548 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3287 2024-09-16T20:51:23.548 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.548 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:23.548 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.548 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:23.548 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:51:23.549 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.549 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.549 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.549 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:23.549 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:51:23.549 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.549 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.549 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.549 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:23.549 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:51:23.549 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1e-07 2024-09-16T20:51:23.549 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.549 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.550 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:23.550 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041163+0000", 2024-09-16T20:51:23.550 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0690000000000001e-06 2024-09-16T20:51:23.550 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.550 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.550 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:23.550 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041166+0000", 2024-09-16T20:51:23.550 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.982e-06 2024-09-16T20:51:23.550 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.550 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.550 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:23.550 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041169+0000", 2024-09-16T20:51:23.550 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.593e-06 2024-09-16T20:51:23.550 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.551 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.551 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:23.551 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093709+0000", 2024-09-16T20:51:23.551 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052540492000000001 2024-09-16T20:51:23.551 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.551 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.551 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:23.551 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093792+0000", 2024-09-16T20:51:23.551 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2436999999999997e-05 2024-09-16T20:51:23.551 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.551 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.551 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:23.551 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093943+0000", 2024-09-16T20:51:23.552 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000151421 2024-09-16T20:51:23.552 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.552 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:23.552 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.552 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.552 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.552 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3288 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 256000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:23.552 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:51:23.552 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 93.427077475000004, 2024-09-16T20:51:23.552 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052928269, 2024-09-16T20:51:23.552 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:23.552 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:23.552 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:23.553 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:23.553 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:51:23.553 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:23.553 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3288 2024-09-16T20:51:23.553 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.553 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:23.553 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.553 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:23.553 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:51:23.553 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.553 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.553 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.553 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:23.554 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:51:23.554 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.554 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.554 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.554 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:23.554 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:51:23.554 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2700000000000001e-07 2024-09-16T20:51:23.554 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.554 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.554 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:23.554 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041173+0000", 2024-09-16T20:51:23.554 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.113e-06 2024-09-16T20:51:23.554 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.555 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.555 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:23.555 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041176+0000", 2024-09-16T20:51:23.555 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9799999999999998e-06 2024-09-16T20:51:23.555 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.555 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.555 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:23.555 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041179+0000", 2024-09-16T20:51:23.555 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.667e-06 2024-09-16T20:51:23.555 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.555 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.555 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:23.555 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093957+0000", 2024-09-16T20:51:23.556 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052778703000000003 2024-09-16T20:51:23.556 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.556 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.556 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:23.556 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093999+0000", 2024-09-16T20:51:23.556 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1300000000000001e-05 2024-09-16T20:51:23.556 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.556 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.556 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:23.556 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094100+0000", 2024-09-16T20:51:23.556 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101279 2024-09-16T20:51:23.556 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.556 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:23.557 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.557 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.557 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.557 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3289 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 261120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:23.557 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:51:23.557 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 93.427067479000002, 2024-09-16T20:51:23.557 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053182976, 2024-09-16T20:51:23.557 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:23.557 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:23.557 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:23.557 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:23.557 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:51:23.557 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:23.558 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3289 2024-09-16T20:51:23.558 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.558 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:23.558 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.558 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:23.558 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:51:23.558 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.558 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.558 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.558 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:23.558 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:51:23.558 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.558 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.559 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.559 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:23.559 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:51:23.559 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2999999999999999e-07 2024-09-16T20:51:23.559 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.559 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.559 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:23.559 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041183+0000", 2024-09-16T20:51:23.559 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0720000000000001e-06 2024-09-16T20:51:23.559 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.559 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.559 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:23.559 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041186+0000", 2024-09-16T20:51:23.559 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9869999999999999e-06 2024-09-16T20:51:23.560 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.560 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.560 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:23.560 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041189+0000", 2024-09-16T20:51:23.560 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.587e-06 2024-09-16T20:51:23.560 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.560 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.560 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:23.560 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094136+0000", 2024-09-16T20:51:23.560 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052947270999999997 2024-09-16T20:51:23.560 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.560 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.560 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:23.561 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094213+0000", 2024-09-16T20:51:23.561 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7552000000000006e-05 2024-09-16T20:51:23.561 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.561 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.561 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:23.561 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094365+0000", 2024-09-16T20:51:23.561 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00015127700000000001 2024-09-16T20:51:23.561 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.561 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:23.561 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.561 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.561 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.561 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:2196 3.d 3:b1fe01a0:::rbd_data.110e7ef3004.0000000000000000:2d [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e136)", 2024-09-16T20:51:23.561 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:22.739057+0000", 2024-09-16T20:51:23.562 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 60.729192679999997, 2024-09-16T20:51:23.562 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11999464899999999, 2024-09-16T20:51:23.562 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:23.562 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:23.562 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:23.562 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:23.562 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:23.562 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:23.562 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2196 2024-09-16T20:51:23.562 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.562 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:23.562 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.562 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:23.563 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.739057+0000", 2024-09-16T20:51:23.563 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.563 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.563 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.563 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:23.563 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.739057+0000", 2024-09-16T20:51:23.563 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.563 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.564 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.564 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:23.564 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.739057+0000", 2024-09-16T20:51:23.564 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1399999999999997e-07 2024-09-16T20:51:23.564 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.564 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.564 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:23.564 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.739060+0000", 2024-09-16T20:51:23.564 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6019999999999998e-06 2024-09-16T20:51:23.564 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.564 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.564 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:23.564 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.858755+0000", 2024-09-16T20:51:23.565 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.119695151 2024-09-16T20:51:23.565 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.565 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.565 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:23.565 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.858778+0000", 2024-09-16T20:51:23.565 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3184999999999999e-05 2024-09-16T20:51:23.565 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.565 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.565 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:23.565 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.858865+0000", 2024-09-16T20:51:23.565 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.7330000000000006e-05 2024-09-16T20:51:23.565 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.565 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.566 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:23.566 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.858952+0000", 2024-09-16T20:51:23.566 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6803000000000002e-05 2024-09-16T20:51:23.566 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.566 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.566 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:23.566 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.859051+0000", 2024-09-16T20:51:23.566 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8963999999999996e-05 2024-09-16T20:51:23.566 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.566 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:23.566 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.566 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.566 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.567 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6061.0:5401 3.13 3:c8554348:::rbd_data.110e16eafbd.0000000000000011:head [read 577536~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-16T20:51:23.567 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:27.761536+0000", 2024-09-16T20:51:23.567 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 55.706712867, 2024-09-16T20:51:23.567 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.003742712, 2024-09-16T20:51:23.567 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:23.567 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:23.567 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:51:23.567 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:23.567 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6061", 2024-09-16T20:51:23.567 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3930128440", 2024-09-16T20:51:23.567 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5401 2024-09-16T20:51:23.567 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.567 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:23.567 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.568 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:23.568 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761536+0000", 2024-09-16T20:51:23.568 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.568 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.568 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.568 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:23.568 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761536+0000", 2024-09-16T20:51:23.568 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.568 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.568 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.568 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:23.568 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761537+0000", 2024-09-16T20:51:23.568 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2900000000000003e-07 2024-09-16T20:51:23.569 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.569 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.569 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:23.569 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761540+0000", 2024-09-16T20:51:23.569 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2320000000000001e-06 2024-09-16T20:51:23.569 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.569 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.569 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:23.569 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761548+0000", 2024-09-16T20:51:23.569 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1289999999999996e-06 2024-09-16T20:51:23.569 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.569 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.569 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:23.570 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761554+0000", 2024-09-16T20:51:23.570 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0820000000000004e-06 2024-09-16T20:51:23.570 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.570 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.570 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:23.570 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761621+0000", 2024-09-16T20:51:23.570 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6266000000000006e-05 2024-09-16T20:51:23.570 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.570 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.570 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:23.570 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761626+0000", 2024-09-16T20:51:23.570 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.614e-06 2024-09-16T20:51:23.570 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.570 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.571 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:23.571 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:28.765163+0000", 2024-09-16T20:51:23.571 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.003536416 2024-09-16T20:51:23.571 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.571 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.571 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:23.571 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:28.765279+0000", 2024-09-16T20:51:23.571 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000116344 2024-09-16T20:51:23.571 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.571 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:23.571 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.571 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.571 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.571 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6061.0:5425 3.13 3:c8554348:::rbd_data.110e16eafbd.0000000000000011:head [read 602112~28672] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-16T20:51:23.572 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:27.767049+0000", 2024-09-16T20:51:23.572 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 55.701200208000003, 2024-09-16T20:51:23.572 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.99847897900000004, 2024-09-16T20:51:23.572 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:23.572 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:23.572 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:51:23.572 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:23.572 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6061", 2024-09-16T20:51:23.572 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3930128440", 2024-09-16T20:51:23.572 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5425 2024-09-16T20:51:23.572 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.572 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:23.572 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.573 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:23.573 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767049+0000", 2024-09-16T20:51:23.573 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.573 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.573 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.573 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:23.573 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767049+0000", 2024-09-16T20:51:23.573 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.573 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.573 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.573 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:23.573 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767050+0000", 2024-09-16T20:51:23.573 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6499999999999999e-07 2024-09-16T20:51:23.573 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.574 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.574 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:23.574 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767052+0000", 2024-09-16T20:51:23.574 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7350000000000001e-06 2024-09-16T20:51:23.574 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.574 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.574 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:23.574 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767060+0000", 2024-09-16T20:51:23.574 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5109999999999997e-06 2024-09-16T20:51:23.574 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.574 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.574 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:23.574 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767066+0000", 2024-09-16T20:51:23.574 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8139999999999997e-06 2024-09-16T20:51:23.575 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.575 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.575 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:23.575 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767128+0000", 2024-09-16T20:51:23.575 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2174999999999993e-05 2024-09-16T20:51:23.575 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.575 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.575 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:23.575 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767133+0000", 2024-09-16T20:51:23.575 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2140000000000004e-06 2024-09-16T20:51:23.575 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.575 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.575 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:23.575 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:28.765441+0000", 2024-09-16T20:51:23.576 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.99830808599999998 2024-09-16T20:51:23.576 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.576 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.576 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:23.576 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:28.765528+0000", 2024-09-16T20:51:23.576 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6879000000000006e-05 2024-09-16T20:51:23.576 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.576 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:23.576 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.576 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.576 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.576 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:25585 3.10 3:0c7c5b23:::rbd_header.110ebf0678ad:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:51:23.576 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:32.877132+0000", 2024-09-16T20:51:23.577 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 50.591117658999998, 2024-09-16T20:51:23.577 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054096699999999998, 2024-09-16T20:51:23.577 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:23.577 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:23.577 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:23.577 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:23.577 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:51:23.577 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:23.577 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 25585 2024-09-16T20:51:23.577 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.577 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:23.577 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.577 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:23.578 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.877132+0000", 2024-09-16T20:51:23.578 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.578 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.578 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.578 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:23.578 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.877132+0000", 2024-09-16T20:51:23.578 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.578 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.578 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.578 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:23.578 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931072+0000", 2024-09-16T20:51:23.578 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053939931000000003 2024-09-16T20:51:23.578 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.578 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.579 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:23.579 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931076+0000", 2024-09-16T20:51:23.579 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2860000000000001e-06 2024-09-16T20:51:23.579 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.579 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.579 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:23.579 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931085+0000", 2024-09-16T20:51:23.579 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.3370000000000006e-06 2024-09-16T20:51:23.579 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.579 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.579 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:23.579 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931092+0000", 2024-09-16T20:51:23.579 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9970000000000001e-06 2024-09-16T20:51:23.579 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.580 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.580 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:23.580 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931137+0000", 2024-09-16T20:51:23.580 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5160000000000001e-05 2024-09-16T20:51:23.580 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.580 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.580 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:23.580 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931178+0000", 2024-09-16T20:51:23.580 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0379000000000001e-05 2024-09-16T20:51:23.580 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.580 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.580 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:23.580 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931228+0000", 2024-09-16T20:51:23.581 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0609999999999998e-05 2024-09-16T20:51:23.581 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.581 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:23.581 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.581 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.581 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.581 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:26679 3.c 3:33e2f78f:::rbd_header.110e16eafbd:head [watch ping cookie 140213436013408 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e153)", 2024-09-16T20:51:23.581 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:41.440287+0000", 2024-09-16T20:51:23.581 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 42.027961783999999, 2024-09-16T20:51:23.581 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.13295633600000001, 2024-09-16T20:51:23.581 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:23.581 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:23.581 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:23.581 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:23.582 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:51:23.582 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:23.582 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 26679 2024-09-16T20:51:23.582 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.582 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:23.582 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.582 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:23.582 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440287+0000", 2024-09-16T20:51:23.582 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.582 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.582 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.582 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:23.582 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440287+0000", 2024-09-16T20:51:23.582 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:23.583 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.583 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.583 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:23.583 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440288+0000", 2024-09-16T20:51:23.583 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0900000000000001e-07 2024-09-16T20:51:23.583 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.583 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.583 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:23.583 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440292+0000", 2024-09-16T20:51:23.583 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3749999999999996e-06 2024-09-16T20:51:23.583 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.583 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.583 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:23.583 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440299+0000", 2024-09-16T20:51:23.584 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8249999999999999e-06 2024-09-16T20:51:23.584 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.584 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.584 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:23.584 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440307+0000", 2024-09-16T20:51:23.584 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4800000000000004e-06 2024-09-16T20:51:23.584 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.584 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.584 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:23.584 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440451+0000", 2024-09-16T20:51:23.584 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014414500000000001 2024-09-16T20:51:23.584 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.584 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.584 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:23.585 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440465+0000", 2024-09-16T20:51:23.585 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.402e-05 2024-09-16T20:51:23.585 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.585 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.585 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:23.585 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.573043+0000", 2024-09-16T20:51:23.585 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.13257855399999999 2024-09-16T20:51:23.585 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.585 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.585 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:23.585 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.573130+0000", 2024-09-16T20:51:23.585 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6168999999999994e-05 2024-09-16T20:51:23.585 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:23.585 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:23.586 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:23.586 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.573244+0000", 2024-09-16T20:51:23.586 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000114159 2024-09-16T20:51:23.586 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.586 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:23.586 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.586 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:23.586 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:23.586 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:23.588 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:51:23.588 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:51:23.586+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:51:23.689 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:23.688+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:23.700 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:23.700 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:51:23.700 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:51:23.700 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:23.708 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:51:23.782 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T20:51:23.790 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:23.789+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:23.817 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:23.817 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:51:23.817 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:51:23.817 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:51:23.817 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:23.826 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:51:23.891 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:23.889+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:23.907 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:51:23.907 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:51:23.924 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:51:23.924 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:51:23.941 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:51:23.941 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:51:23.959 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:51:23.959 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:51:23.975 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:23.975 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:23.976 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:51:23.976 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:51:23.985 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:51:23.995 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:51:23.995 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:51:24.028 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:24.027+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:24.092 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:24.091+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:24.104 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:24.104 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:24.113 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:51:24.197 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:24.196+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:24.262 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:24.262 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:24.271 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:51:24.312 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:51:24.311+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:51:24.394 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:24.393+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:24.418 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:24.418 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:24.427 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:51:24.495 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:51:24.494+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:51:24.534 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:24.534 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:24.543 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:51:24.628 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:24.627+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:24.629 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T20:51:24.692 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:24.692 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:24.696 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:24.695+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:24.701 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:51:24.797 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:51:24.796+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:51:24.852 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:24.852 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:24.861 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:51:24.898 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:24.897+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:24.999 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:24.998+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:25.010 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:25.010 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:25.020 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:51:25.100 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:51:25.098+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:51:25.132 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:25.132 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:25.141 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:51:25.201 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:25.199+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:25.296 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:25.296 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:25.301 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:25.300+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:25.305 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:51:25.403 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:25.402+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:25.424 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:25.424 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:25.434 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:51:25.504 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:25.503+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:25.553 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:25.553 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:25.564 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:51:25.605 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:25.604+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:25.615 INFO:teuthology.orchestra.run.smithi029.stderr:noscrub is unset 2024-09-16T20:51:25.706 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:25.704+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:25.734 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:25.734 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:25.743 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:51:25.807 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:25.805+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:25.908 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:25.906+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:25.910 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:25.910 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:25.921 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:51:26.006 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T20:51:26.008 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:51:26.006+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:51:26.051 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:26.051 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:26.062 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:51:26.108 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:26.107+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:26.162 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:51:26.163 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:51:26.184 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:51:26.184 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:51:26.198 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:26.198 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:51:26.198 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:51:26.198 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:51:26.198 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.198 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:51:26.198 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:51:26.198 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 212.94181065199999, 2024-09-16T20:51:26.198 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67502346999999996, 2024-09-16T20:51:26.198 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:26.198 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:26.198 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:26.199 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:26.199 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:51:26.199 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:51:26.199 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 1 2024-09-16T20:51:26.199 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.199 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:26.199 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.199 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:26.199 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:51:26.199 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.199 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.199 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.199 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:26.199 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:51:26.200 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.200 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.200 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.200 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:26.200 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254128+0000", 2024-09-16T20:51:26.200 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.463e-06 2024-09-16T20:51:26.200 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.200 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.200 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:26.200 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254135+0000", 2024-09-16T20:51:26.200 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1579999999999999e-06 2024-09-16T20:51:26.200 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.200 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.201 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:26.201 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254146+0000", 2024-09-16T20:51:26.201 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1341e-05 2024-09-16T20:51:26.201 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.201 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.201 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:26.201 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254155+0000", 2024-09-16T20:51:26.201 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.1549999999999996e-06 2024-09-16T20:51:26.201 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.201 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.201 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:26.201 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254221+0000", 2024-09-16T20:51:26.201 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5470999999999997e-05 2024-09-16T20:51:26.202 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.202 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.202 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:26.202 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254231+0000", 2024-09-16T20:51:26.202 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0592e-05 2024-09-16T20:51:26.202 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.202 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.202 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:26.202 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926711+0000", 2024-09-16T20:51:26.202 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67247946599999997 2024-09-16T20:51:26.202 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.202 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.202 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:26.203 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926910+0000", 2024-09-16T20:51:26.203 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019934 2024-09-16T20:51:26.203 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.203 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.203 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:51:26.203 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.927159+0000", 2024-09-16T20:51:26.203 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00024876599999999998 2024-09-16T20:51:26.203 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.203 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.203 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:26.203 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.928797+0000", 2024-09-16T20:51:26.203 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001638308 2024-09-16T20:51:26.203 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.204 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.204 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:26.204 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929026+0000", 2024-09-16T20:51:26.204 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00022876099999999999 2024-09-16T20:51:26.204 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.204 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.204 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:26.204 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929102+0000", 2024-09-16T20:51:26.204 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6184000000000001e-05 2024-09-16T20:51:26.204 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.204 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.204 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:26.204 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929150+0000", 2024-09-16T20:51:26.204 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7465e-05 2024-09-16T20:51:26.205 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.205 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:26.205 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.205 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.205 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.205 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3574 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:26.205 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:51:26.205 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 156.71464712, 2024-09-16T20:51:26.205 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053579441999999998, 2024-09-16T20:51:26.205 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:26.205 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:26.205 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:26.205 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:26.206 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:26.206 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:26.206 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3574 2024-09-16T20:51:26.206 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.206 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:26.206 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.206 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:26.206 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:51:26.206 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.206 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.206 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.206 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:26.206 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:51:26.207 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.207 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.207 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.207 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:26.207 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:51:26.207 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2e-07 2024-09-16T20:51:26.207 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.207 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.207 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:26.207 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481292+0000", 2024-09-16T20:51:26.207 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2630000000000002e-06 2024-09-16T20:51:26.207 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.207 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.207 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:26.208 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481297+0000", 2024-09-16T20:51:26.208 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.296e-06 2024-09-16T20:51:26.208 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.208 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.208 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:26.208 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481300+0000", 2024-09-16T20:51:26.208 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7019999999999999e-06 2024-09-16T20:51:26.208 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.208 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.208 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:26.208 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525533+0000", 2024-09-16T20:51:26.208 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.044232318 2024-09-16T20:51:26.208 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.209 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.209 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:26.209 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525619+0000", 2024-09-16T20:51:26.209 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5892999999999999e-05 2024-09-16T20:51:26.209 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.209 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.209 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:26.209 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525987+0000", 2024-09-16T20:51:26.209 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00036836099999999999 2024-09-16T20:51:26.209 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.209 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.209 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:26.209 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531779+0000", 2024-09-16T20:51:26.209 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0057918240000000001 2024-09-16T20:51:26.210 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.210 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.210 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:26.210 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534747+0000", 2024-09-16T20:51:26.210 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0029681600000000001 2024-09-16T20:51:26.210 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.210 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.210 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:26.210 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534832+0000", 2024-09-16T20:51:26.210 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5180999999999996e-05 2024-09-16T20:51:26.210 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.210 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.210 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:26.211 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534869+0000", 2024-09-16T20:51:26.211 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7024000000000003e-05 2024-09-16T20:51:26.211 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.211 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:26.211 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.211 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.211 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.211 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3575 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:26.211 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:51:26.211 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 156.71463238000001, 2024-09-16T20:51:26.211 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053801463000000001, 2024-09-16T20:51:26.211 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:26.212 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:26.212 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:26.212 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:26.212 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:26.212 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:26.212 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3575 2024-09-16T20:51:26.212 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.212 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:26.212 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.212 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:26.212 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:51:26.212 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.212 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.212 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.213 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:26.213 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:51:26.213 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.213 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.213 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.213 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:26.213 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481305+0000", 2024-09-16T20:51:26.213 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5400000000000002e-07 2024-09-16T20:51:26.213 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.213 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.213 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:26.213 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481307+0000", 2024-09-16T20:51:26.213 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7800000000000001e-06 2024-09-16T20:51:26.214 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.214 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.214 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:26.214 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481312+0000", 2024-09-16T20:51:26.214 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6349999999999997e-06 2024-09-16T20:51:26.214 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.214 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.214 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:26.214 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481315+0000", 2024-09-16T20:51:26.214 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7829999999999999e-06 2024-09-16T20:51:26.214 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.214 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.214 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:26.214 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.526876+0000", 2024-09-16T20:51:26.215 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.045561070000000002 2024-09-16T20:51:26.215 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.215 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.215 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:26.215 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.526954+0000", 2024-09-16T20:51:26.215 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8108000000000005e-05 2024-09-16T20:51:26.215 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.215 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.215 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:26.215 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527282+0000", 2024-09-16T20:51:26.215 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000327483 2024-09-16T20:51:26.215 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.215 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.216 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:26.216 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531784+0000", 2024-09-16T20:51:26.216 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0045020169999999997 2024-09-16T20:51:26.216 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.216 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.216 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:26.216 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534921+0000", 2024-09-16T20:51:26.216 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.003137791 2024-09-16T20:51:26.216 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.216 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.216 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:26.216 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534976+0000", 2024-09-16T20:51:26.216 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4953000000000001e-05 2024-09-16T20:51:26.216 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.217 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.217 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:26.217 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535106+0000", 2024-09-16T20:51:26.217 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000129589 2024-09-16T20:51:26.217 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.217 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:26.217 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.217 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.217 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.217 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3576 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:26.217 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:51:26.217 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 156.714618468, 2024-09-16T20:51:26.217 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053955436000000002, 2024-09-16T20:51:26.217 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:26.218 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:26.218 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:26.218 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:26.218 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:26.218 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:26.218 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3576 2024-09-16T20:51:26.218 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.218 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:26.218 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.218 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:26.218 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:51:26.218 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.218 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.219 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.219 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:26.219 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:51:26.219 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.219 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.219 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.219 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:26.219 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481319+0000", 2024-09-16T20:51:26.219 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.53e-07 2024-09-16T20:51:26.219 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.219 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.219 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:26.219 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481320+0000", 2024-09-16T20:51:26.220 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5230000000000001e-06 2024-09-16T20:51:26.220 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.220 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.220 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:26.220 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481330+0000", 2024-09-16T20:51:26.220 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0190000000000001e-05 2024-09-16T20:51:26.220 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.220 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.220 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:26.220 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481333+0000", 2024-09-16T20:51:26.220 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.151e-06 2024-09-16T20:51:26.220 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.220 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.221 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:26.221 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527689+0000", 2024-09-16T20:51:26.221 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046355192000000003 2024-09-16T20:51:26.221 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.221 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.221 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:26.221 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527764+0000", 2024-09-16T20:51:26.221 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5776000000000001e-05 2024-09-16T20:51:26.221 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.221 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.221 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:26.221 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528141+0000", 2024-09-16T20:51:26.222 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00037647599999999999 2024-09-16T20:51:26.222 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.222 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.222 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:26.222 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531788+0000", 2024-09-16T20:51:26.222 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0036470679999999998 2024-09-16T20:51:26.222 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.222 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.222 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:26.222 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535174+0000", 2024-09-16T20:51:26.222 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0033862670000000001 2024-09-16T20:51:26.222 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.222 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.222 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:26.223 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535236+0000", 2024-09-16T20:51:26.223 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1797999999999993e-05 2024-09-16T20:51:26.223 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.223 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.223 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:26.223 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535274+0000", 2024-09-16T20:51:26.223 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7741999999999999e-05 2024-09-16T20:51:26.223 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.223 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:26.223 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.223 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.223 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.223 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3577 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:26.223 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:51:26.224 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 156.714596868, 2024-09-16T20:51:26.224 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054073807000000002, 2024-09-16T20:51:26.224 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:26.224 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:26.224 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:26.224 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:26.224 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:26.224 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:26.224 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3577 2024-09-16T20:51:26.224 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.224 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:26.224 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.224 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:26.224 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:51:26.225 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.225 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.225 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.225 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:26.225 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:51:26.225 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.225 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.225 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.225 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:26.225 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:51:26.225 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.96e-07 2024-09-16T20:51:26.225 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.225 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.225 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:26.226 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481342+0000", 2024-09-16T20:51:26.226 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.714e-06 2024-09-16T20:51:26.226 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.226 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.226 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:26.226 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481349+0000", 2024-09-16T20:51:26.226 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4930000000000001e-06 2024-09-16T20:51:26.226 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.226 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.226 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:26.226 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481353+0000", 2024-09-16T20:51:26.226 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4879999999999999e-06 2024-09-16T20:51:26.226 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.226 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.227 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:26.227 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528619+0000", 2024-09-16T20:51:26.227 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047266351999999998 2024-09-16T20:51:26.227 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.227 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.227 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:26.227 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528706+0000", 2024-09-16T20:51:26.227 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6796e-05 2024-09-16T20:51:26.227 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.227 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.227 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:26.227 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529148+0000", 2024-09-16T20:51:26.227 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00044182999999999999 2024-09-16T20:51:26.228 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.228 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.228 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:26.228 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531792+0000", 2024-09-16T20:51:26.228 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0026443119999999998 2024-09-16T20:51:26.228 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.228 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.228 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:26.228 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535319+0000", 2024-09-16T20:51:26.228 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0035266439999999998 2024-09-16T20:51:26.228 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.228 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.228 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:26.229 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535372+0000", 2024-09-16T20:51:26.229 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3207000000000001e-05 2024-09-16T20:51:26.229 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.229 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.229 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:26.229 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535414+0000", 2024-09-16T20:51:26.229 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1675000000000003e-05 2024-09-16T20:51:26.229 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.229 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:26.229 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.229 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.229 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.229 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3578 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:26.229 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:51:26.230 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 156.71457569899999, 2024-09-16T20:51:26.230 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054189586999999997, 2024-09-16T20:51:26.230 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:26.230 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:26.230 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:26.230 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:26.230 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:26.230 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:26.230 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3578 2024-09-16T20:51:26.230 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.230 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:26.230 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.230 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:26.230 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:51:26.231 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.231 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.231 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.231 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:26.231 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:51:26.231 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.231 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.231 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.231 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:26.231 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:51:26.231 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0800000000000001e-07 2024-09-16T20:51:26.231 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.231 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.231 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:26.232 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481363+0000", 2024-09-16T20:51:26.232 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0339999999999999e-06 2024-09-16T20:51:26.232 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.232 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.232 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:26.232 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481371+0000", 2024-09-16T20:51:26.232 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4440000000000002e-06 2024-09-16T20:51:26.232 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.232 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.232 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:26.232 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481374+0000", 2024-09-16T20:51:26.232 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2049999999999998e-06 2024-09-16T20:51:26.232 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.233 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.233 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:26.233 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529555+0000", 2024-09-16T20:51:26.233 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048181202999999999 2024-09-16T20:51:26.233 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.233 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.233 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:26.233 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529616+0000", 2024-09-16T20:51:26.233 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0912000000000002e-05 2024-09-16T20:51:26.233 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.233 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.233 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:26.233 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529939+0000", 2024-09-16T20:51:26.234 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00032269800000000001 2024-09-16T20:51:26.234 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.234 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.234 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:26.234 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532602+0000", 2024-09-16T20:51:26.234 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0026634940000000002 2024-09-16T20:51:26.234 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.234 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.234 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:26.234 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535465+0000", 2024-09-16T20:51:26.234 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.002862149 2024-09-16T20:51:26.234 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.234 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.234 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:26.235 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535516+0000", 2024-09-16T20:51:26.235 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1696000000000001e-05 2024-09-16T20:51:26.235 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.235 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.235 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:26.235 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535551+0000", 2024-09-16T20:51:26.235 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4443999999999998e-05 2024-09-16T20:51:26.235 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.235 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:26.235 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.235 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.235 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.235 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3579 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:26.235 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:51:26.236 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 156.71455725999999, 2024-09-16T20:51:26.236 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054295151, 2024-09-16T20:51:26.236 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:26.236 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:26.236 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:26.236 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:26.236 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:26.236 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:26.236 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3579 2024-09-16T20:51:26.236 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.236 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:26.236 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.236 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:26.236 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:51:26.237 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.237 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.237 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.237 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:26.237 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:51:26.237 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.237 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.237 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.237 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:26.237 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:51:26.237 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3300000000000001e-07 2024-09-16T20:51:26.237 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.237 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.237 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:26.238 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481381+0000", 2024-09-16T20:51:26.238 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.649e-06 2024-09-16T20:51:26.238 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.238 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.238 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:26.238 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481387+0000", 2024-09-16T20:51:26.238 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4519999999999996e-06 2024-09-16T20:51:26.238 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.238 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.238 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:26.238 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481390+0000", 2024-09-16T20:51:26.238 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0170000000000001e-06 2024-09-16T20:51:26.238 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.238 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.239 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:26.239 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530756+0000", 2024-09-16T20:51:26.239 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049365882999999999 2024-09-16T20:51:26.239 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.239 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.239 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:26.239 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530845+0000", 2024-09-16T20:51:26.239 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9197000000000007e-05 2024-09-16T20:51:26.239 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.239 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.239 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:26.239 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531191+0000", 2024-09-16T20:51:26.239 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00034562500000000001 2024-09-16T20:51:26.240 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.240 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.240 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:26.240 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534078+0000", 2024-09-16T20:51:26.240 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0028878179999999999 2024-09-16T20:51:26.240 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.240 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.240 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:26.240 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535589+0000", 2024-09-16T20:51:26.240 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00151085 2024-09-16T20:51:26.240 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.240 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.240 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:26.240 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535639+0000", 2024-09-16T20:51:26.241 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9697000000000002e-05 2024-09-16T20:51:26.241 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.241 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.241 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:26.241 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535675+0000", 2024-09-16T20:51:26.241 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5729999999999998e-05 2024-09-16T20:51:26.241 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.241 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:26.241 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.241 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.241 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.241 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3580 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:26.241 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:51:26.241 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 156.714543628, 2024-09-16T20:51:26.242 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054397361999999998, 2024-09-16T20:51:26.242 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:26.242 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:26.242 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:26.242 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:26.242 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:26.242 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:26.242 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3580 2024-09-16T20:51:26.242 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.242 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:26.242 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.242 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:26.242 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:51:26.242 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.243 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.243 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.243 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:26.243 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:51:26.243 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.243 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.243 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.243 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:26.243 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:51:26.243 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2600000000000001e-07 2024-09-16T20:51:26.243 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.243 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.243 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:26.243 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481395+0000", 2024-09-16T20:51:26.244 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.544e-06 2024-09-16T20:51:26.244 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.244 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.244 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:26.244 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481404+0000", 2024-09-16T20:51:26.244 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8850000000000005e-06 2024-09-16T20:51:26.244 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.244 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.244 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:26.244 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481407+0000", 2024-09-16T20:51:26.244 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1540000000000002e-06 2024-09-16T20:51:26.244 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.245 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.245 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:26.245 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531823+0000", 2024-09-16T20:51:26.245 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050415637999999999 2024-09-16T20:51:26.245 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.245 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.245 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:26.245 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531895+0000", 2024-09-16T20:51:26.245 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2179000000000001e-05 2024-09-16T20:51:26.245 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.245 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.245 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:26.245 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532236+0000", 2024-09-16T20:51:26.245 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00034073799999999999 2024-09-16T20:51:26.246 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.246 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.246 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:26.246 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534083+0000", 2024-09-16T20:51:26.246 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001847298 2024-09-16T20:51:26.246 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.246 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.246 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:26.246 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535714+0000", 2024-09-16T20:51:26.246 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0016316810000000001 2024-09-16T20:51:26.246 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.246 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.246 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:26.246 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535759+0000", 2024-09-16T20:51:26.247 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4478999999999999e-05 2024-09-16T20:51:26.247 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.247 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.247 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:26.247 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535791+0000", 2024-09-16T20:51:26.247 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.154e-05 2024-09-16T20:51:26.247 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.247 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:26.247 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.247 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.247 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.247 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3581 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:26.247 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:51:26.248 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 156.714521631, 2024-09-16T20:51:26.248 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.057094403000000002, 2024-09-16T20:51:26.248 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:26.248 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:26.248 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:26.248 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:26.248 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:26.248 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:26.248 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3581 2024-09-16T20:51:26.248 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.248 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:26.248 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.248 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:26.248 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:51:26.249 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.249 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.249 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.249 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:26.249 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:51:26.249 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.249 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.249 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.249 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:26.249 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:51:26.250 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8799999999999998e-07 2024-09-16T20:51:26.250 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.250 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.250 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:26.250 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481418+0000", 2024-09-16T20:51:26.250 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1859999999999999e-06 2024-09-16T20:51:26.250 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.250 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.250 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:26.250 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481423+0000", 2024-09-16T20:51:26.250 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8279999999999999e-06 2024-09-16T20:51:26.250 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.250 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.251 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:26.251 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481426+0000", 2024-09-16T20:51:26.251 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0110000000000001e-06 2024-09-16T20:51:26.251 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.251 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.251 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:26.251 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532631+0000", 2024-09-16T20:51:26.251 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051204514999999999 2024-09-16T20:51:26.251 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.251 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.251 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:26.251 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.533381+0000", 2024-09-16T20:51:26.251 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00075029000000000003 2024-09-16T20:51:26.252 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.252 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.252 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:26.252 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.533739+0000", 2024-09-16T20:51:26.252 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00035773100000000002 2024-09-16T20:51:26.252 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.252 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.252 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:26.252 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535123+0000", 2024-09-16T20:51:26.252 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0013836160000000001 2024-09-16T20:51:26.252 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.252 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.253 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:26.253 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538413+0000", 2024-09-16T20:51:26.253 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0032902510000000001 2024-09-16T20:51:26.253 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.253 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.253 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:26.253 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538470+0000", 2024-09-16T20:51:26.253 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6697999999999998e-05 2024-09-16T20:51:26.253 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.253 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.253 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:26.253 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538510+0000", 2024-09-16T20:51:26.253 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9988999999999999e-05 2024-09-16T20:51:26.254 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.254 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:26.254 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.254 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.254 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.254 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:12732 3.d 3:b752444a:::rbd_header.110e747f0779:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-16T20:51:26.254 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:51:26.254 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 141.427449053, 2024-09-16T20:51:26.254 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.065302467000000003, 2024-09-16T20:51:26.254 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:26.254 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:26.254 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:26.254 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:26.255 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:51:26.255 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:51:26.255 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 12732 2024-09-16T20:51:26.255 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.255 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:26.255 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.255 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:26.255 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:51:26.255 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.255 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.255 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.255 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:26.255 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:51:26.255 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.256 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.256 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.256 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:26.256 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768490+0000", 2024-09-16T20:51:26.256 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2859999999999998e-06 2024-09-16T20:51:26.256 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.256 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.256 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:26.256 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768501+0000", 2024-09-16T20:51:26.256 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1226999999999999e-05 2024-09-16T20:51:26.256 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.256 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.256 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:26.257 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768517+0000", 2024-09-16T20:51:26.257 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.592e-05 2024-09-16T20:51:26.257 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.257 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.257 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:26.257 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768528+0000", 2024-09-16T20:51:26.257 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0689999999999999e-05 2024-09-16T20:51:26.257 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.257 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.257 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:26.257 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768584+0000", 2024-09-16T20:51:26.257 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5889e-05 2024-09-16T20:51:26.257 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.257 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.258 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:26.258 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768720+0000", 2024-09-16T20:51:26.258 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000136718 2024-09-16T20:51:26.258 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.258 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.258 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:26.258 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.769066+0000", 2024-09-16T20:51:26.258 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000346013 2024-09-16T20:51:26.258 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.258 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.258 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:26.258 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833659+0000", 2024-09-16T20:51:26.258 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.064592822999999994 2024-09-16T20:51:26.259 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.259 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.259 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:26.259 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833719+0000", 2024-09-16T20:51:26.259 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9441000000000002e-05 2024-09-16T20:51:26.259 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.259 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.259 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:26.259 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833758+0000", 2024-09-16T20:51:26.259 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9122000000000003e-05 2024-09-16T20:51:26.259 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.259 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.259 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:26.260 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833790+0000", 2024-09-16T20:51:26.260 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2338000000000001e-05 2024-09-16T20:51:26.260 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.260 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:26.260 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.260 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.260 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.260 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.5089.0:12732 3.d e73/23 ondisk, result = 0)", 2024-09-16T20:51:26.260 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:51:26.260 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 141.42584688299999, 2024-09-16T20:51:26.260 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.063706456999999994, 2024-09-16T20:51:26.260 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:26.261 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:26.261 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:26.261 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:26.261 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.261 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:26.261 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:51:26.261 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.261 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.261 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.261 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:26.261 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:51:26.262 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.262 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.262 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.262 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:26.262 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770091+0000", 2024-09-16T20:51:26.262 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.53e-06 2024-09-16T20:51:26.262 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.262 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.262 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:26.262 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770104+0000", 2024-09-16T20:51:26.262 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2203e-05 2024-09-16T20:51:26.262 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.262 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.262 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:26.263 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833637+0000", 2024-09-16T20:51:26.263 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.063533201999999997 2024-09-16T20:51:26.263 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.263 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.263 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:26.263 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833655+0000", 2024-09-16T20:51:26.263 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8369e-05 2024-09-16T20:51:26.263 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.263 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.263 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:26.263 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833707+0000", 2024-09-16T20:51:26.263 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2061999999999997e-05 2024-09-16T20:51:26.263 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.263 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.264 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:26.264 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833715+0000", 2024-09-16T20:51:26.264 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5179999999999998e-06 2024-09-16T20:51:26.264 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.264 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.264 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:26.264 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833796+0000", 2024-09-16T20:51:26.264 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1573e-05 2024-09-16T20:51:26.264 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.264 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:26.264 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.264 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.264 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.265 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3286 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 245760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:26.265 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:51:26.265 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 96.154784946000007, 2024-09-16T20:51:26.265 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052523564000000002, 2024-09-16T20:51:26.265 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:26.265 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:26.265 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:26.265 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:26.265 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:51:26.265 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:26.265 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3286 2024-09-16T20:51:26.265 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.265 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:26.265 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.266 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:26.266 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:51:26.266 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.266 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.266 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.266 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:26.266 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:51:26.266 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.266 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.266 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.266 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:26.266 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:51:26.266 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3900000000000001e-07 2024-09-16T20:51:26.267 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.267 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.267 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:26.267 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041153+0000", 2024-09-16T20:51:26.267 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.119e-06 2024-09-16T20:51:26.267 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.267 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.267 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:26.267 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041156+0000", 2024-09-16T20:51:26.267 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0000000000000001e-06 2024-09-16T20:51:26.267 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.267 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.267 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:26.268 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041159+0000", 2024-09-16T20:51:26.268 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5859999999999999e-06 2024-09-16T20:51:26.268 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.268 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.268 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:26.268 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093529+0000", 2024-09-16T20:51:26.268 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052370591000000001 2024-09-16T20:51:26.268 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.268 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.268 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:26.268 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093570+0000", 2024-09-16T20:51:26.268 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0161999999999999e-05 2024-09-16T20:51:26.268 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.269 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.269 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:26.269 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093675+0000", 2024-09-16T20:51:26.269 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010586699999999999 2024-09-16T20:51:26.269 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.269 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:26.269 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.269 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.269 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.269 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3287 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 250880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:26.269 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:51:26.269 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 96.154774896999996, 2024-09-16T20:51:26.269 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052781203999999998, 2024-09-16T20:51:26.270 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:26.270 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:26.270 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:26.270 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:26.270 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:51:26.270 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:26.270 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3287 2024-09-16T20:51:26.270 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.270 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:26.270 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.270 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:26.270 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:51:26.270 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.271 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.271 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.271 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:26.271 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:51:26.271 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.271 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.271 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.271 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:26.271 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:51:26.271 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1e-07 2024-09-16T20:51:26.271 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.271 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.272 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:26.272 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041163+0000", 2024-09-16T20:51:26.272 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0690000000000001e-06 2024-09-16T20:51:26.272 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.272 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.272 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:26.272 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041166+0000", 2024-09-16T20:51:26.272 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.982e-06 2024-09-16T20:51:26.272 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.272 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.272 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:26.272 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041169+0000", 2024-09-16T20:51:26.272 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.593e-06 2024-09-16T20:51:26.273 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.273 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.273 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:26.273 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093709+0000", 2024-09-16T20:51:26.273 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052540492000000001 2024-09-16T20:51:26.273 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.273 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.273 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:26.273 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093792+0000", 2024-09-16T20:51:26.273 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2436999999999997e-05 2024-09-16T20:51:26.273 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.273 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.273 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:26.274 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093943+0000", 2024-09-16T20:51:26.274 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000151421 2024-09-16T20:51:26.274 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.274 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:26.274 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.274 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.274 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.274 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3288 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 256000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:26.274 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:51:26.274 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 96.154765024, 2024-09-16T20:51:26.274 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052928269, 2024-09-16T20:51:26.274 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:26.274 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:26.275 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:26.275 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:26.275 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:51:26.275 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:26.275 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3288 2024-09-16T20:51:26.275 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.275 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:26.275 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.275 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:26.275 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:51:26.275 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.275 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.275 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.276 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:26.276 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:51:26.276 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.276 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.276 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.276 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:26.276 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:51:26.276 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2700000000000001e-07 2024-09-16T20:51:26.276 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.276 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.276 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:26.276 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041173+0000", 2024-09-16T20:51:26.276 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.113e-06 2024-09-16T20:51:26.277 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.277 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.277 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:26.277 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041176+0000", 2024-09-16T20:51:26.277 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9799999999999998e-06 2024-09-16T20:51:26.277 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.277 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.277 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:26.277 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041179+0000", 2024-09-16T20:51:26.277 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.667e-06 2024-09-16T20:51:26.277 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.277 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.277 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:26.277 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093957+0000", 2024-09-16T20:51:26.278 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052778703000000003 2024-09-16T20:51:26.278 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.278 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.278 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:26.278 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093999+0000", 2024-09-16T20:51:26.278 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1300000000000001e-05 2024-09-16T20:51:26.278 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.278 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.278 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:26.278 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094100+0000", 2024-09-16T20:51:26.278 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101279 2024-09-16T20:51:26.278 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.278 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:26.279 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.279 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.279 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.279 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3289 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 261120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:26.279 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:51:26.279 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 96.154755027999997, 2024-09-16T20:51:26.279 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053182976, 2024-09-16T20:51:26.279 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:26.279 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:26.279 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:26.279 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:26.279 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:51:26.279 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:26.280 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3289 2024-09-16T20:51:26.280 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.280 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:26.280 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.280 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:26.280 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:51:26.280 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.280 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.280 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.280 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:26.280 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:51:26.280 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.280 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.280 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.281 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:26.281 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:51:26.281 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2999999999999999e-07 2024-09-16T20:51:26.281 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.281 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.281 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:26.281 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041183+0000", 2024-09-16T20:51:26.281 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0720000000000001e-06 2024-09-16T20:51:26.281 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.281 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.281 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:26.281 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041186+0000", 2024-09-16T20:51:26.282 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9869999999999999e-06 2024-09-16T20:51:26.282 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.282 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.282 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:26.282 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041189+0000", 2024-09-16T20:51:26.282 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.587e-06 2024-09-16T20:51:26.282 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.282 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.282 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:26.282 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094136+0000", 2024-09-16T20:51:26.282 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052947270999999997 2024-09-16T20:51:26.282 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.282 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.282 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:26.283 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094213+0000", 2024-09-16T20:51:26.283 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7552000000000006e-05 2024-09-16T20:51:26.283 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.283 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.283 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:26.283 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094365+0000", 2024-09-16T20:51:26.283 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00015127700000000001 2024-09-16T20:51:26.283 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.283 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:26.283 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.283 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.283 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.283 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:2196 3.d 3:b1fe01a0:::rbd_data.110e7ef3004.0000000000000000:2d [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e136)", 2024-09-16T20:51:26.283 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:22.739057+0000", 2024-09-16T20:51:26.284 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 63.456880228999999, 2024-09-16T20:51:26.284 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11999464899999999, 2024-09-16T20:51:26.284 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:26.284 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:26.284 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:26.284 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:26.284 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:26.284 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:26.284 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2196 2024-09-16T20:51:26.284 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.284 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:26.284 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.284 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:26.285 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.739057+0000", 2024-09-16T20:51:26.285 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.285 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.285 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.285 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:26.285 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.739057+0000", 2024-09-16T20:51:26.285 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.285 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.285 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.285 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:26.285 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.739057+0000", 2024-09-16T20:51:26.285 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1399999999999997e-07 2024-09-16T20:51:26.285 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.285 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.286 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:26.286 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.739060+0000", 2024-09-16T20:51:26.286 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6019999999999998e-06 2024-09-16T20:51:26.286 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.286 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.286 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:26.286 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.858755+0000", 2024-09-16T20:51:26.286 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.119695151 2024-09-16T20:51:26.286 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.286 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.286 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:26.286 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.858778+0000", 2024-09-16T20:51:26.287 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3184999999999999e-05 2024-09-16T20:51:26.287 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.287 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.287 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:26.287 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.858865+0000", 2024-09-16T20:51:26.287 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.7330000000000006e-05 2024-09-16T20:51:26.287 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.287 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.287 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:26.287 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.858952+0000", 2024-09-16T20:51:26.287 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6803000000000002e-05 2024-09-16T20:51:26.287 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.288 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.288 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:26.288 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.859051+0000", 2024-09-16T20:51:26.288 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8963999999999996e-05 2024-09-16T20:51:26.288 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.288 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:26.288 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.288 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.288 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.288 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6061.0:5401 3.13 3:c8554348:::rbd_data.110e16eafbd.0000000000000011:head [read 577536~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-16T20:51:26.288 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:27.761536+0000", 2024-09-16T20:51:26.288 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 58.434400416000003, 2024-09-16T20:51:26.288 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.003742712, 2024-09-16T20:51:26.289 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:26.289 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:26.289 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:51:26.289 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:26.289 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6061", 2024-09-16T20:51:26.289 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3930128440", 2024-09-16T20:51:26.289 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5401 2024-09-16T20:51:26.289 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.289 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:26.289 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.289 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:26.289 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761536+0000", 2024-09-16T20:51:26.289 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.290 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.290 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.290 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:26.290 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761536+0000", 2024-09-16T20:51:26.290 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.290 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.290 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.290 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:26.290 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761537+0000", 2024-09-16T20:51:26.290 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2900000000000003e-07 2024-09-16T20:51:26.290 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.290 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.290 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:26.291 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761540+0000", 2024-09-16T20:51:26.291 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2320000000000001e-06 2024-09-16T20:51:26.291 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.291 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.291 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:26.291 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761548+0000", 2024-09-16T20:51:26.291 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1289999999999996e-06 2024-09-16T20:51:26.291 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.291 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.291 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:26.291 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761554+0000", 2024-09-16T20:51:26.291 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0820000000000004e-06 2024-09-16T20:51:26.291 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.292 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.292 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:26.292 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761621+0000", 2024-09-16T20:51:26.292 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6266000000000006e-05 2024-09-16T20:51:26.292 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.292 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.292 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:26.292 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761626+0000", 2024-09-16T20:51:26.292 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.614e-06 2024-09-16T20:51:26.292 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.292 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.292 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:26.293 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:28.765163+0000", 2024-09-16T20:51:26.293 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.003536416 2024-09-16T20:51:26.293 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.293 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.293 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:26.293 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:28.765279+0000", 2024-09-16T20:51:26.293 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000116344 2024-09-16T20:51:26.293 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.293 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:26.293 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.293 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.293 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.293 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6061.0:5425 3.13 3:c8554348:::rbd_data.110e16eafbd.0000000000000011:head [read 602112~28672] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-16T20:51:26.293 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:27.767049+0000", 2024-09-16T20:51:26.294 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 58.428887756999998, 2024-09-16T20:51:26.294 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.99847897900000004, 2024-09-16T20:51:26.294 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:26.294 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:26.294 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:51:26.294 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:26.294 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6061", 2024-09-16T20:51:26.294 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3930128440", 2024-09-16T20:51:26.294 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5425 2024-09-16T20:51:26.294 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.294 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:26.294 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.294 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:26.294 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767049+0000", 2024-09-16T20:51:26.295 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.295 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.295 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.295 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:26.295 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767049+0000", 2024-09-16T20:51:26.295 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.295 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.295 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.295 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:26.295 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767050+0000", 2024-09-16T20:51:26.295 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6499999999999999e-07 2024-09-16T20:51:26.295 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.295 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.295 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:26.296 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767052+0000", 2024-09-16T20:51:26.296 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7350000000000001e-06 2024-09-16T20:51:26.296 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.296 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.296 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:26.296 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767060+0000", 2024-09-16T20:51:26.296 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5109999999999997e-06 2024-09-16T20:51:26.296 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.296 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.296 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:26.296 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767066+0000", 2024-09-16T20:51:26.296 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8139999999999997e-06 2024-09-16T20:51:26.297 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.297 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.297 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:26.297 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767128+0000", 2024-09-16T20:51:26.297 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2174999999999993e-05 2024-09-16T20:51:26.297 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.297 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.297 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:26.297 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767133+0000", 2024-09-16T20:51:26.297 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2140000000000004e-06 2024-09-16T20:51:26.297 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.297 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.297 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:26.297 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:28.765441+0000", 2024-09-16T20:51:26.298 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.99830808599999998 2024-09-16T20:51:26.298 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.298 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.298 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:26.298 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:28.765528+0000", 2024-09-16T20:51:26.298 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6879000000000006e-05 2024-09-16T20:51:26.298 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.298 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:26.298 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.298 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.298 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.298 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:25585 3.10 3:0c7c5b23:::rbd_header.110ebf0678ad:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:51:26.298 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:32.877132+0000", 2024-09-16T20:51:26.299 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 53.318805208000001, 2024-09-16T20:51:26.299 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054096699999999998, 2024-09-16T20:51:26.299 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:26.299 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:26.299 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:26.299 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:26.299 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:51:26.299 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:26.299 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 25585 2024-09-16T20:51:26.299 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.299 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:26.299 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.299 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:26.300 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.877132+0000", 2024-09-16T20:51:26.300 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.300 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.300 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.300 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:26.300 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.877132+0000", 2024-09-16T20:51:26.300 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.300 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.300 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.300 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:26.300 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931072+0000", 2024-09-16T20:51:26.300 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053939931000000003 2024-09-16T20:51:26.300 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.301 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.301 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:26.301 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931076+0000", 2024-09-16T20:51:26.301 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2860000000000001e-06 2024-09-16T20:51:26.301 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.301 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.301 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:26.301 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931085+0000", 2024-09-16T20:51:26.301 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.3370000000000006e-06 2024-09-16T20:51:26.301 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.301 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.301 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:26.301 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931092+0000", 2024-09-16T20:51:26.302 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9970000000000001e-06 2024-09-16T20:51:26.302 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.302 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.302 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:26.302 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931137+0000", 2024-09-16T20:51:26.302 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5160000000000001e-05 2024-09-16T20:51:26.302 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.302 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.302 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:26.302 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931178+0000", 2024-09-16T20:51:26.302 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0379000000000001e-05 2024-09-16T20:51:26.302 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.302 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.302 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:26.303 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931228+0000", 2024-09-16T20:51:26.303 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0609999999999998e-05 2024-09-16T20:51:26.303 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.303 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:26.303 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.303 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.303 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.303 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:26679 3.c 3:33e2f78f:::rbd_header.110e16eafbd:head [watch ping cookie 140213436013408 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e153)", 2024-09-16T20:51:26.303 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:41.440287+0000", 2024-09-16T20:51:26.303 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 44.755649333000001, 2024-09-16T20:51:26.303 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.13295633600000001, 2024-09-16T20:51:26.303 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:26.303 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:26.303 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:26.304 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:26.304 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:51:26.304 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:26.304 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 26679 2024-09-16T20:51:26.304 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.304 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:26.304 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.304 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:26.304 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440287+0000", 2024-09-16T20:51:26.304 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.304 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.304 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.304 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:26.305 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440287+0000", 2024-09-16T20:51:26.305 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.305 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.305 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.305 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:26.305 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440288+0000", 2024-09-16T20:51:26.305 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0900000000000001e-07 2024-09-16T20:51:26.305 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.305 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.305 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:26.305 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440292+0000", 2024-09-16T20:51:26.305 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3749999999999996e-06 2024-09-16T20:51:26.305 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.305 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.305 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:26.306 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440299+0000", 2024-09-16T20:51:26.306 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8249999999999999e-06 2024-09-16T20:51:26.306 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.306 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.306 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:26.306 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440307+0000", 2024-09-16T20:51:26.306 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4800000000000004e-06 2024-09-16T20:51:26.306 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.306 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.306 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:26.306 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440451+0000", 2024-09-16T20:51:26.306 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014414500000000001 2024-09-16T20:51:26.306 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.306 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.307 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:26.307 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440465+0000", 2024-09-16T20:51:26.307 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.402e-05 2024-09-16T20:51:26.307 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.307 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.307 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:26.307 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.573043+0000", 2024-09-16T20:51:26.307 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.13257855399999999 2024-09-16T20:51:26.307 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.307 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.307 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:26.307 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.573130+0000", 2024-09-16T20:51:26.307 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6168999999999994e-05 2024-09-16T20:51:26.308 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.308 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.308 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:26.308 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.573244+0000", 2024-09-16T20:51:26.308 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000114159 2024-09-16T20:51:26.308 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.308 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:26.308 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.308 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.308 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:26.308 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:26.309 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:51:26.309 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:51:26.310 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:51:26.311 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:51:26.311 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:51:26.311 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:51:26.312 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:51:26.312 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:51:26.313 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:51:26.313 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:51:26.308+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:51:26.410 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:26.409+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:26.463 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:26.463 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:51:26.463 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:51:26.463 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:26.472 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:51:26.514 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:26.513+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:26.613 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:26.611+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:26.627 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:26.627 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:51:26.627 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:51:26.627 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:51:26.627 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:26.640 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:51:26.714 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:26.713+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:26.789 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:26.790 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:51:26.790 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:51:26.790 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:51:26.790 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.790 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3714 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1480704~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:26.790 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:51:26.790 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 184.14918510499999, 2024-09-16T20:51:26.790 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050152552000000003, 2024-09-16T20:51:26.790 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:26.790 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:26.790 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:26.790 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:26.790 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:26.791 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:26.791 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3714 2024-09-16T20:51:26.791 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.791 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:26.791 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.791 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:26.791 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:51:26.791 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.791 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.791 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.791 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:26.791 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:51:26.791 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.791 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.792 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.792 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:26.792 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:51:26.792 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0499999999999999e-07 2024-09-16T20:51:26.792 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.792 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.792 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:26.792 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639007+0000", 2024-09-16T20:51:26.792 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5340000000000001e-06 2024-09-16T20:51:26.792 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.792 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.792 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:26.793 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639013+0000", 2024-09-16T20:51:26.793 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3260000000000003e-06 2024-09-16T20:51:26.793 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.793 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.793 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:26.793 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639018+0000", 2024-09-16T20:51:26.793 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2850000000000002e-06 2024-09-16T20:51:26.793 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.793 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.793 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:26.793 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689026+0000", 2024-09-16T20:51:26.793 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.05000798 2024-09-16T20:51:26.793 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.794 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.794 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:26.794 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689077+0000", 2024-09-16T20:51:26.794 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1063999999999998e-05 2024-09-16T20:51:26.794 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.794 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.794 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:26.794 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689158+0000", 2024-09-16T20:51:26.794 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1057999999999995e-05 2024-09-16T20:51:26.794 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.794 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:26.794 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.795 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.795 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.795 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3716 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1488896~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:26.795 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:51:26.795 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 184.14916729500001, 2024-09-16T20:51:26.795 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050280142, 2024-09-16T20:51:26.795 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:26.795 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:26.795 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:26.795 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:26.795 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:26.795 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:26.795 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3716 2024-09-16T20:51:26.795 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.796 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:26.796 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.796 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:26.796 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:51:26.796 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.796 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.796 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.796 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:26.796 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:51:26.796 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.796 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.796 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.796 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:26.797 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639024+0000", 2024-09-16T20:51:26.797 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9999999999999997e-07 2024-09-16T20:51:26.797 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.797 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.797 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:26.797 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639026+0000", 2024-09-16T20:51:26.797 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.781e-06 2024-09-16T20:51:26.797 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.797 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.797 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:26.797 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639032+0000", 2024-09-16T20:51:26.797 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7780000000000004e-06 2024-09-16T20:51:26.797 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.797 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.798 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:26.798 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639036+0000", 2024-09-16T20:51:26.798 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6909999999999996e-06 2024-09-16T20:51:26.798 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.798 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.798 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:26.798 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689191+0000", 2024-09-16T20:51:26.798 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050155284000000001 2024-09-16T20:51:26.798 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.798 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.798 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:26.798 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689242+0000", 2024-09-16T20:51:26.798 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0662999999999999e-05 2024-09-16T20:51:26.799 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.799 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.799 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:26.799 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689304+0000", 2024-09-16T20:51:26.799 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1345000000000003e-05 2024-09-16T20:51:26.799 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.799 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:26.799 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.799 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.799 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.799 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3719 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1497088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:26.799 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:51:26.799 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 184.14914001700001, 2024-09-16T20:51:26.800 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050399715999999997, 2024-09-16T20:51:26.800 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:26.800 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:26.800 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:26.800 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:26.800 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:26.800 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:26.800 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3719 2024-09-16T20:51:26.800 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.800 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:26.800 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.800 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:26.800 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:51:26.801 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.801 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.801 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.801 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:26.801 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:51:26.801 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.801 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.801 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.801 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:26.801 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:51:26.801 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9700000000000002e-07 2024-09-16T20:51:26.801 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.801 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.801 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:26.802 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639053+0000", 2024-09-16T20:51:26.802 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2900000000000001e-06 2024-09-16T20:51:26.802 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.802 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.802 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:26.802 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639060+0000", 2024-09-16T20:51:26.802 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7750000000000004e-06 2024-09-16T20:51:26.802 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.802 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.802 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:26.802 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639066+0000", 2024-09-16T20:51:26.802 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0639999999999999e-06 2024-09-16T20:51:26.802 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.802 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.803 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:26.803 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689337+0000", 2024-09-16T20:51:26.803 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050270784999999998 2024-09-16T20:51:26.803 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.803 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.803 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:26.803 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689392+0000", 2024-09-16T20:51:26.803 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4920999999999998e-05 2024-09-16T20:51:26.803 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.803 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.803 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:26.803 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689450+0000", 2024-09-16T20:51:26.803 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8483999999999998e-05 2024-09-16T20:51:26.803 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.804 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:26.804 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.804 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.804 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.804 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3721 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1505280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:26.804 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:51:26.804 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 184.14911688500001, 2024-09-16T20:51:26.804 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050513608000000002, 2024-09-16T20:51:26.804 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:26.804 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:26.804 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:26.805 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:26.805 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:26.805 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:26.805 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3721 2024-09-16T20:51:26.805 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.805 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:26.805 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.805 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:26.805 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:51:26.805 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.805 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.805 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.805 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:26.805 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:51:26.806 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.806 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.806 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.806 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:26.806 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:51:26.806 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7e-07 2024-09-16T20:51:26.806 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.806 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.806 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:26.806 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639076+0000", 2024-09-16T20:51:26.806 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8899999999999999e-06 2024-09-16T20:51:26.806 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.806 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.806 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:26.807 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639083+0000", 2024-09-16T20:51:26.807 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1210000000000001e-06 2024-09-16T20:51:26.807 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.807 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.807 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:26.807 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639089+0000", 2024-09-16T20:51:26.807 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.643e-06 2024-09-16T20:51:26.807 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.807 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.807 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:26.807 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689474+0000", 2024-09-16T20:51:26.807 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050385679000000003 2024-09-16T20:51:26.807 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.808 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.808 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:26.808 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689533+0000", 2024-09-16T20:51:26.808 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8303000000000001e-05 2024-09-16T20:51:26.808 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.808 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.808 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:26.808 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689587+0000", 2024-09-16T20:51:26.808 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4601999999999997e-05 2024-09-16T20:51:26.808 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.808 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:26.808 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.808 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.808 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.809 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3723 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1513472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:26.809 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:51:26.809 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 184.14908608100001, 2024-09-16T20:51:26.809 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050699691999999998, 2024-09-16T20:51:26.809 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:26.809 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:26.809 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:26.809 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:26.809 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:26.809 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:26.809 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3723 2024-09-16T20:51:26.809 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.809 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:26.810 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.810 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:26.810 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:51:26.810 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.810 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.810 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.810 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:26.810 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:51:26.810 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.810 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.810 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.810 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:26.810 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:51:26.810 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3399999999999999e-07 2024-09-16T20:51:26.811 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.811 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.811 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:26.811 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639107+0000", 2024-09-16T20:51:26.811 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3860000000000001e-06 2024-09-16T20:51:26.811 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.811 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.811 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:26.811 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639114+0000", 2024-09-16T20:51:26.811 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9800000000000001e-06 2024-09-16T20:51:26.811 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.811 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.811 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:26.811 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639129+0000", 2024-09-16T20:51:26.812 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4887e-05 2024-09-16T20:51:26.812 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.812 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.812 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:26.812 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689625+0000", 2024-09-16T20:51:26.812 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050495848000000003 2024-09-16T20:51:26.812 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.812 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.812 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:26.812 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689720+0000", 2024-09-16T20:51:26.812 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4821000000000006e-05 2024-09-16T20:51:26.812 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.812 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.813 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:26.813 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689804+0000", 2024-09-16T20:51:26.813 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4235999999999998e-05 2024-09-16T20:51:26.813 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.813 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:26.813 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.813 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.813 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.813 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3725 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1521664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:26.813 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:51:26.813 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 184.149052485, 2024-09-16T20:51:26.813 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050797660000000001, 2024-09-16T20:51:26.813 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:26.813 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:26.814 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:26.814 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:26.814 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:26.814 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:26.814 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3725 2024-09-16T20:51:26.814 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.814 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:26.814 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.814 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:26.814 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:51:26.814 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.814 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.814 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.815 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:26.815 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:51:26.815 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.815 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.815 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.815 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:26.815 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639139+0000", 2024-09-16T20:51:26.815 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8999999999999997e-07 2024-09-16T20:51:26.815 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.815 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.815 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:26.815 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639141+0000", 2024-09-16T20:51:26.815 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.182e-06 2024-09-16T20:51:26.815 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.816 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.816 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:26.816 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639154+0000", 2024-09-16T20:51:26.816 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3626e-05 2024-09-16T20:51:26.816 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.816 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.816 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:26.816 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639160+0000", 2024-09-16T20:51:26.816 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6810000000000003e-06 2024-09-16T20:51:26.816 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.816 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.816 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:26.816 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689831+0000", 2024-09-16T20:51:26.816 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050670907000000001 2024-09-16T20:51:26.817 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.817 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.817 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:26.817 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689882+0000", 2024-09-16T20:51:26.817 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0445999999999998e-05 2024-09-16T20:51:26.817 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.817 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.817 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:26.817 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689936+0000", 2024-09-16T20:51:26.817 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4327999999999999e-05 2024-09-16T20:51:26.817 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.817 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:26.817 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.817 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.818 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.818 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3727 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1529856~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:26.818 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:51:26.818 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 184.14901690799999, 2024-09-16T20:51:26.818 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051016922999999999, 2024-09-16T20:51:26.818 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:26.818 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:26.818 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:26.818 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:26.818 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:26.818 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:26.818 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3727 2024-09-16T20:51:26.818 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.819 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:26.819 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.819 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:26.819 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:51:26.819 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.819 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.819 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.819 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:26.819 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:51:26.819 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.819 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.819 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.819 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:26.819 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:51:26.820 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9999999999999998e-07 2024-09-16T20:51:26.820 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.820 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.820 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:26.820 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639183+0000", 2024-09-16T20:51:26.820 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2399999999999996e-06 2024-09-16T20:51:26.820 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.820 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.820 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:26.820 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639195+0000", 2024-09-16T20:51:26.820 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1113e-05 2024-09-16T20:51:26.820 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.820 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.821 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:26.821 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639206+0000", 2024-09-16T20:51:26.821 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1503e-05 2024-09-16T20:51:26.821 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.821 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.821 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:26.821 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689969+0000", 2024-09-16T20:51:26.821 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050762676 2024-09-16T20:51:26.821 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.821 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.821 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:26.821 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690071+0000", 2024-09-16T20:51:26.821 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102163 2024-09-16T20:51:26.821 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.822 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.822 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:26.822 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690191+0000", 2024-09-16T20:51:26.822 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000119728 2024-09-16T20:51:26.822 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.822 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:26.822 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.822 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.824 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.824 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3730 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1538048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:26.824 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:51:26.824 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 184.148975345, 2024-09-16T20:51:26.824 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051131467999999999, 2024-09-16T20:51:26.824 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:26.824 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:26.824 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:26.824 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:26.824 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:26.824 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:26.825 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3730 2024-09-16T20:51:26.825 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.825 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:26.825 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.825 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:26.825 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:51:26.825 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.825 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.825 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.825 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:26.825 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:51:26.825 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.825 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.826 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.826 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:26.826 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639216+0000", 2024-09-16T20:51:26.826 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.39e-07 2024-09-16T20:51:26.826 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.826 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.826 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:26.826 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639218+0000", 2024-09-16T20:51:26.826 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.537e-06 2024-09-16T20:51:26.826 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.826 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.826 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:26.826 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639229+0000", 2024-09-16T20:51:26.826 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1122e-05 2024-09-16T20:51:26.827 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.827 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.827 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:26.827 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639237+0000", 2024-09-16T20:51:26.827 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8509999999999999e-06 2024-09-16T20:51:26.827 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.827 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.827 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:26.827 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690227+0000", 2024-09-16T20:51:26.827 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050990147 2024-09-16T20:51:26.827 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.827 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.827 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:26.828 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690278+0000", 2024-09-16T20:51:26.828 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0223999999999999e-05 2024-09-16T20:51:26.828 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.828 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.828 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:26.828 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690347+0000", 2024-09-16T20:51:26.828 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9147999999999996e-05 2024-09-16T20:51:26.828 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.828 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:26.828 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.828 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.828 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.828 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3731 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1546240~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:26.829 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:51:26.829 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 184.148944637, 2024-09-16T20:51:26.829 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051276744999999999, 2024-09-16T20:51:26.829 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:26.829 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:26.829 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:26.829 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:26.829 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:26.829 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:26.829 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3731 2024-09-16T20:51:26.829 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.829 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:26.830 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.830 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:26.830 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:51:26.830 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.830 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.830 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.830 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:26.830 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:51:26.830 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.830 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.830 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.830 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:26.830 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:51:26.830 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9900000000000001e-07 2024-09-16T20:51:26.831 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.831 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.831 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:26.831 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639249+0000", 2024-09-16T20:51:26.831 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3630000000000001e-06 2024-09-16T20:51:26.831 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.831 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.831 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:26.831 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639255+0000", 2024-09-16T20:51:26.831 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4960000000000001e-06 2024-09-16T20:51:26.831 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.831 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.831 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:26.832 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639261+0000", 2024-09-16T20:51:26.832 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2839999999999997e-06 2024-09-16T20:51:26.832 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.832 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.832 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:26.832 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690375+0000", 2024-09-16T20:51:26.832 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051114684000000001 2024-09-16T20:51:26.832 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.832 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.832 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:26.832 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690425+0000", 2024-09-16T20:51:26.832 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9379999999999998e-05 2024-09-16T20:51:26.832 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.833 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.833 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:26.833 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690523+0000", 2024-09-16T20:51:26.833 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8139000000000003e-05 2024-09-16T20:51:26.833 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.833 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:26.833 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.833 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.833 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.833 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3732 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1554432~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:26.833 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:51:26.833 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 184.148898855, 2024-09-16T20:51:26.833 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051380096, 2024-09-16T20:51:26.833 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:26.834 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:26.834 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:26.834 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:26.834 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:26.834 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:26.834 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3732 2024-09-16T20:51:26.834 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.834 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:26.834 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.834 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:26.834 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:51:26.834 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.834 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.834 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.835 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:26.835 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:51:26.835 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.835 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.835 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.835 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:26.835 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:51:26.835 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6100000000000001e-07 2024-09-16T20:51:26.835 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.835 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.835 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:26.835 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639295+0000", 2024-09-16T20:51:26.835 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.447e-06 2024-09-16T20:51:26.836 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.836 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.836 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:26.836 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639302+0000", 2024-09-16T20:51:26.836 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0570000000000004e-06 2024-09-16T20:51:26.836 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.836 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.836 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:26.836 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639307+0000", 2024-09-16T20:51:26.836 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1429999999999997e-06 2024-09-16T20:51:26.836 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.836 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.836 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:26.836 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690555+0000", 2024-09-16T20:51:26.837 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051247963000000001 2024-09-16T20:51:26.837 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.837 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.837 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:26.837 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690602+0000", 2024-09-16T20:51:26.837 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7602000000000003e-05 2024-09-16T20:51:26.837 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.837 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.837 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:26.837 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690672+0000", 2024-09-16T20:51:26.837 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9423000000000003e-05 2024-09-16T20:51:26.837 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.837 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:26.838 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.838 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.838 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.838 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3734 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1562624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:26.838 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:51:26.838 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 184.14887720499999, 2024-09-16T20:51:26.838 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051490610999999999, 2024-09-16T20:51:26.838 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:26.838 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:26.838 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:26.838 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:26.838 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:26.838 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:26.838 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3734 2024-09-16T20:51:26.839 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.839 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:26.839 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.839 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:26.839 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:51:26.839 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.839 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.839 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.839 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:26.839 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:51:26.839 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.839 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.839 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.839 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:26.840 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639314+0000", 2024-09-16T20:51:26.840 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4400000000000001e-07 2024-09-16T20:51:26.840 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.840 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.840 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:26.840 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639316+0000", 2024-09-16T20:51:26.840 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1459999999999999e-06 2024-09-16T20:51:26.840 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.840 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.840 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:26.840 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639322+0000", 2024-09-16T20:51:26.840 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7610000000000004e-06 2024-09-16T20:51:26.840 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.841 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.841 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:26.841 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639327+0000", 2024-09-16T20:51:26.841 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.186e-06 2024-09-16T20:51:26.841 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.841 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.841 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:26.841 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690698+0000", 2024-09-16T20:51:26.841 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051371223000000001 2024-09-16T20:51:26.841 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.841 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.841 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:26.841 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690748+0000", 2024-09-16T20:51:26.841 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0192999999999999e-05 2024-09-16T20:51:26.842 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.842 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.842 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:26.842 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690804+0000", 2024-09-16T20:51:26.842 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5658000000000001e-05 2024-09-16T20:51:26.842 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.842 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:26.842 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.842 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.842 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.842 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3736 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1570816~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:26.842 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:51:26.842 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 184.14885794599999, 2024-09-16T20:51:26.843 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051596528000000003, 2024-09-16T20:51:26.843 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:26.843 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:26.843 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:26.843 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:26.843 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:26.843 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:26.843 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3736 2024-09-16T20:51:26.843 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.843 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:26.843 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.843 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:26.843 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:51:26.843 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.844 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.844 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.844 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:26.844 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:51:26.844 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.844 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.844 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.844 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:26.844 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:51:26.844 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9999999999999999e-07 2024-09-16T20:51:26.844 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.844 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.845 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:26.845 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639335+0000", 2024-09-16T20:51:26.845 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.635e-06 2024-09-16T20:51:26.845 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.845 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.845 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:26.845 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639340+0000", 2024-09-16T20:51:26.845 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5840000000000002e-06 2024-09-16T20:51:26.845 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.845 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.845 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:26.845 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639346+0000", 2024-09-16T20:51:26.845 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1190000000000002e-06 2024-09-16T20:51:26.845 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.846 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.846 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:26.846 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690817+0000", 2024-09-16T20:51:26.846 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051470721999999997 2024-09-16T20:51:26.846 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.846 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.846 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:26.846 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690875+0000", 2024-09-16T20:51:26.846 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7441000000000001e-05 2024-09-16T20:51:26.846 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.846 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.846 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:26.846 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690929+0000", 2024-09-16T20:51:26.846 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4727e-05 2024-09-16T20:51:26.847 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.847 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:26.847 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.847 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.847 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.847 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5434 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:26.847 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:51:26.847 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 183.92359637999999, 2024-09-16T20:51:26.847 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099755831000000003, 2024-09-16T20:51:26.847 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:26.847 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:26.847 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:26.847 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:26.847 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:26.848 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:26.848 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5434 2024-09-16T20:51:26.848 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.848 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:26.848 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.848 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:26.848 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:51:26.848 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.848 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.848 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.848 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:26.848 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:51:26.849 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.849 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.849 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.849 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:26.849 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864595+0000", 2024-09-16T20:51:26.849 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1099999999999995e-07 2024-09-16T20:51:26.849 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.849 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.849 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:26.849 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864597+0000", 2024-09-16T20:51:26.849 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3989999999999998e-06 2024-09-16T20:51:26.849 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.849 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.850 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:26.850 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864603+0000", 2024-09-16T20:51:26.850 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9309999999999996e-06 2024-09-16T20:51:26.850 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.850 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.850 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:26.850 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864607+0000", 2024-09-16T20:51:26.850 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6200000000000001e-06 2024-09-16T20:51:26.850 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.850 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.850 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:26.850 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964164+0000", 2024-09-16T20:51:26.850 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099557162000000005 2024-09-16T20:51:26.850 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.851 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.851 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:26.851 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964244+0000", 2024-09-16T20:51:26.851 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9597000000000004e-05 2024-09-16T20:51:26.851 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.851 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.851 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:26.851 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964350+0000", 2024-09-16T20:51:26.851 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010641099999999999 2024-09-16T20:51:26.851 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.851 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:26.851 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.851 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.851 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.852 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5435 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 20480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:26.852 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:51:26.852 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 183.923566558, 2024-09-16T20:51:26.852 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100897102, 2024-09-16T20:51:26.852 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:26.852 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:26.852 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:26.852 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:26.852 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:26.852 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:26.852 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5435 2024-09-16T20:51:26.852 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.852 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:26.853 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.853 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:26.853 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:51:26.853 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.853 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.853 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.853 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:26.853 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:51:26.853 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.853 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.853 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.853 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:26.853 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:51:26.853 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8000000000000001e-07 2024-09-16T20:51:26.854 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.854 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.854 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:26.854 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864626+0000", 2024-09-16T20:51:26.854 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.745e-06 2024-09-16T20:51:26.854 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.854 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.854 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:26.854 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864633+0000", 2024-09-16T20:51:26.854 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3339999999999998e-06 2024-09-16T20:51:26.854 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.854 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.854 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:26.854 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864636+0000", 2024-09-16T20:51:26.855 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1200000000000002e-06 2024-09-16T20:51:26.855 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.855 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.855 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:26.855 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965299+0000", 2024-09-16T20:51:26.855 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10066352000000001 2024-09-16T20:51:26.855 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.855 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.855 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:26.855 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965390+0000", 2024-09-16T20:51:26.855 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0894000000000001e-05 2024-09-16T20:51:26.855 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.855 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.856 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:26.856 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965521+0000", 2024-09-16T20:51:26.856 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013110899999999999 2024-09-16T20:51:26.856 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.856 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:26.856 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.856 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.856 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.856 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:152 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 327680~32768 out=32768b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:51:26.856 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:51:26.856 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 182.62506961599999, 2024-09-16T20:51:26.856 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11869405800000001, 2024-09-16T20:51:26.856 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:26.857 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:26.857 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:26.857 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:26.857 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:51:26.857 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:51:26.857 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 152 2024-09-16T20:51:26.857 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.857 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:26.857 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.857 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:26.857 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:51:26.857 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.857 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.858 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.858 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:26.858 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:51:26.858 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.858 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.858 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.858 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:26.858 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163122+0000", 2024-09-16T20:51:26.858 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6400000000000003e-07 2024-09-16T20:51:26.858 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.858 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.858 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:26.858 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163124+0000", 2024-09-16T20:51:26.858 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.565e-06 2024-09-16T20:51:26.859 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.859 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.859 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:26.859 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163136+0000", 2024-09-16T20:51:26.859 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1717e-05 2024-09-16T20:51:26.859 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.859 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.859 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:26.859 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163142+0000", 2024-09-16T20:51:26.859 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1750000000000002e-06 2024-09-16T20:51:26.859 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.859 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.859 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:26.860 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163606+0000", 2024-09-16T20:51:26.860 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00046418799999999998 2024-09-16T20:51:26.860 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.860 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.860 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:26.860 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163656+0000", 2024-09-16T20:51:26.860 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9852999999999999e-05 2024-09-16T20:51:26.860 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.860 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.860 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:26.860 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281815+0000", 2024-09-16T20:51:26.860 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11815909600000001 2024-09-16T20:51:26.860 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.861 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:26.861 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.861 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.861 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.861 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:153 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:51:26.861 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:51:26.861 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 182.625036663, 2024-09-16T20:51:26.861 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11903462400000001, 2024-09-16T20:51:26.861 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:26.861 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:26.861 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:26.861 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:26.861 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:51:26.861 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:51:26.862 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 153 2024-09-16T20:51:26.862 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.862 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:26.862 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.862 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:26.862 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:51:26.862 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.862 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.862 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.862 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:26.862 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:51:26.862 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.863 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.863 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.863 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:26.863 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163155+0000", 2024-09-16T20:51:26.863 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.37e-07 2024-09-16T20:51:26.863 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.863 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.863 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:26.863 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163157+0000", 2024-09-16T20:51:26.863 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6010000000000002e-06 2024-09-16T20:51:26.863 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.863 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.863 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:26.863 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163165+0000", 2024-09-16T20:51:26.864 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2619999999999998e-06 2024-09-16T20:51:26.864 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.864 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.864 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:26.864 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163171+0000", 2024-09-16T20:51:26.864 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.711e-06 2024-09-16T20:51:26.864 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.864 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.864 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:26.864 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281868+0000", 2024-09-16T20:51:26.864 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118696622 2024-09-16T20:51:26.864 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.865 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.865 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:26.865 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281969+0000", 2024-09-16T20:51:26.865 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101371 2024-09-16T20:51:26.865 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.865 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.865 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:26.865 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282189+0000", 2024-09-16T20:51:26.865 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00021952000000000001 2024-09-16T20:51:26.865 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.866 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:26.866 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.866 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.866 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.866 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:154 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 360448~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:51:26.866 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:51:26.866 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 182.625004891, 2024-09-16T20:51:26.866 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11932035100000001, 2024-09-16T20:51:26.866 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:26.866 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:26.866 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:26.866 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:26.866 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:51:26.867 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:51:26.867 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 154 2024-09-16T20:51:26.867 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.867 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:26.867 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.867 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:26.867 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:51:26.867 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.867 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.867 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.867 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:26.867 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:51:26.867 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.868 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.868 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.868 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:26.868 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:51:26.868 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:51:26.868 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.868 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.868 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:26.868 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163189+0000", 2024-09-16T20:51:26.868 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6840000000000001e-06 2024-09-16T20:51:26.868 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.868 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.868 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:26.869 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163196+0000", 2024-09-16T20:51:26.869 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7920000000000004e-06 2024-09-16T20:51:26.869 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.869 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.869 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:26.869 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163201+0000", 2024-09-16T20:51:26.869 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2809999999999999e-06 2024-09-16T20:51:26.869 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.869 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.869 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:26.869 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282219+0000", 2024-09-16T20:51:26.869 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.119018343 2024-09-16T20:51:26.869 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.869 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.870 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:26.870 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282305+0000", 2024-09-16T20:51:26.870 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5865000000000005e-05 2024-09-16T20:51:26.870 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.870 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.870 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:26.870 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282506+0000", 2024-09-16T20:51:26.870 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000200927 2024-09-16T20:51:26.870 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.870 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:26.870 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.870 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.871 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.871 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop(client.4366.0:3088 3.1e e38/23 3:79e5ec01:::rbd_header.110e891a9b2b:head v 38'306, mlcod=36'304)", 2024-09-16T20:51:26.871 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:51:26.871 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 182.62398602299999, 2024-09-16T20:51:26.871 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118102759, 2024-09-16T20:51:26.871 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:26.871 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:26.871 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:26.871 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:26.871 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.871 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:26.871 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:51:26.871 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.872 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.872 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.872 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:26.872 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:51:26.872 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.872 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.872 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.872 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:26.872 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:51:26.872 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.85e-07 2024-09-16T20:51:26.872 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.872 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.872 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:26.872 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164212+0000", 2024-09-16T20:51:26.873 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3999999999999997e-06 2024-09-16T20:51:26.873 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.873 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.873 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:26.873 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281553+0000", 2024-09-16T20:51:26.873 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11734135499999999 2024-09-16T20:51:26.873 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.873 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.873 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:26.873 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281567+0000", 2024-09-16T20:51:26.873 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4188999999999999e-05 2024-09-16T20:51:26.873 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.873 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.874 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:26.874 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281653+0000", 2024-09-16T20:51:26.874 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5385999999999999e-05 2024-09-16T20:51:26.874 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.874 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.874 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:26.874 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281679+0000", 2024-09-16T20:51:26.874 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6689000000000001e-05 2024-09-16T20:51:26.874 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.874 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.874 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:26.874 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282277+0000", 2024-09-16T20:51:26.874 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00059798000000000002 2024-09-16T20:51:26.875 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.875 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.875 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:26.875 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282307+0000", 2024-09-16T20:51:26.875 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9975e-05 2024-09-16T20:51:26.875 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.875 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:26.875 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.875 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.875 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.875 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5255.0:44 3.16 3:6f5476df:::rbd_header.110e25db1f60:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:51:26.875 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:51:26.875 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 110.46768296800001, 2024-09-16T20:51:26.875 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.121007509, 2024-09-16T20:51:26.876 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:26.876 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:26.876 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:26.876 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:26.876 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5255", 2024-09-16T20:51:26.876 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:51:26.876 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 44 2024-09-16T20:51:26.876 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.876 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:26.876 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.876 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:26.876 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:51:26.876 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.877 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.877 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.877 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:26.877 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:51:26.877 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.877 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.877 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.877 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:26.877 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:51:26.877 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0500000000000003e-07 2024-09-16T20:51:26.877 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.877 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.877 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:26.878 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320512+0000", 2024-09-16T20:51:26.878 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2540000000000001e-06 2024-09-16T20:51:26.878 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.878 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.878 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:26.878 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320518+0000", 2024-09-16T20:51:26.878 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1689999999999997e-06 2024-09-16T20:51:26.878 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.878 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.878 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:26.878 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320521+0000", 2024-09-16T20:51:26.878 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5200000000000002e-06 2024-09-16T20:51:26.878 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.878 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.879 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:26.879 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320548+0000", 2024-09-16T20:51:26.879 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6662999999999999e-05 2024-09-16T20:51:26.879 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.879 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.879 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:26.879 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320587+0000", 2024-09-16T20:51:26.879 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8866999999999999e-05 2024-09-16T20:51:26.879 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.879 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.879 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:26.879 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.441515+0000", 2024-09-16T20:51:26.879 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.120928331 2024-09-16T20:51:26.879 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.880 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:26.880 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.880 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.880 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.880 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:20329 3.5 3:a4ea64b7:::rbd_header.110ede5ef857:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:26.880 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:01.061694+0000", 2024-09-16T20:51:26.880 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 85.726496400000002, 2024-09-16T20:51:26.880 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.071234077000000007, 2024-09-16T20:51:26.880 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:26.880 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:26.880 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:26.880 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:26.881 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:51:26.881 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:26.881 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 20329 2024-09-16T20:51:26.881 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.881 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:26.881 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.881 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:26.881 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.061694+0000", 2024-09-16T20:51:26.881 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.881 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.881 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.881 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:26.881 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.061694+0000", 2024-09-16T20:51:26.881 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:26.882 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.882 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.882 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:26.882 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089232+0000", 2024-09-16T20:51:26.882 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.027537275 2024-09-16T20:51:26.882 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.882 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.882 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:26.882 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089237+0000", 2024-09-16T20:51:26.882 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4709999999999998e-06 2024-09-16T20:51:26.882 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.882 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.882 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:26.883 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089246+0000", 2024-09-16T20:51:26.883 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8230000000000001e-06 2024-09-16T20:51:26.883 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.883 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.883 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:26.883 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089252+0000", 2024-09-16T20:51:26.883 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3060000000000003e-06 2024-09-16T20:51:26.883 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.883 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.883 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:26.883 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.132756+0000", 2024-09-16T20:51:26.883 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.043504334999999998 2024-09-16T20:51:26.883 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.883 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.884 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:26.884 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.132829+0000", 2024-09-16T20:51:26.884 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2849999999999995e-05 2024-09-16T20:51:26.884 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:26.884 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:26.884 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:26.884 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.132928+0000", 2024-09-16T20:51:26.884 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.9017000000000004e-05 2024-09-16T20:51:26.884 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.884 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:26.884 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.884 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:26.884 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:26.884 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:26.886 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:51:26.886 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:26.885+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:26.987 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:51:26.985+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:51:26.994 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:51:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:31909 3.1e 3.ac44ff9e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e199)", 2024-09-16T20:51:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:51:26.618259+0000", 2024-09-16T20:51:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.37545779200000001, 2024-09-16T20:51:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.37547421800000003, 2024-09-16T20:51:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "waiting for peered", 2024-09-16T20:51:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:26.995 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:26.995 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:26.995 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 31909 2024-09-16T20:51:26.995 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:26.995 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:26.995 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:26.995 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:26.995 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.618259+0000", 2024-09-16T20:51:26.995 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:26.995 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:26.995 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:26.995 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:26.996 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.618259+0000", 2024-09-16T20:51:26.996 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:26.996 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:26.996 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:26.996 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:26.996 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.618261+0000", 2024-09-16T20:51:26.996 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4899999999999999e-06 2024-09-16T20:51:26.996 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:26.996 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:26.996 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:26.996 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.618267+0000", 2024-09-16T20:51:26.996 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8649999999999998e-06 2024-09-16T20:51:26.997 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:26.997 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:26.997 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:26.997 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.618277+0000", 2024-09-16T20:51:26.997 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0209e-05 2024-09-16T20:51:26.997 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:26.997 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:26.997 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:26.997 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.618282+0000", 2024-09-16T20:51:26.997 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.733e-06 2024-09-16T20:51:26.997 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:26.997 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:26.997 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:26.997 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.618341+0000", 2024-09-16T20:51:26.998 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8823000000000002e-05 2024-09-16T20:51:26.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:26.998 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:26.998 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:51:26.998 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.618346+0000", 2024-09-16T20:51:26.998 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8049999999999997e-06 2024-09-16T20:51:26.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:26.998 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:26.998 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:26.998 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.620745+0000", 2024-09-16T20:51:26.998 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0023987499999999998 2024-09-16T20:51:26.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:26.998 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:26.999 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:26.999 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.620755+0000", 2024-09-16T20:51:26.999 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0474e-05 2024-09-16T20:51:26.999 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:26.999 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:26.999 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:26.999 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:26.999 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:26.999 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:31914 3.1e 3.ac44ff9e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e199)", 2024-09-16T20:51:26.999 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:51:26.635328+0000", 2024-09-16T20:51:26.999 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.35838876400000003, 2024-09-16T20:51:26.999 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.35844678299999999, 2024-09-16T20:51:26.999 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:26.999 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:27.000 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "waiting for peered", 2024-09-16T20:51:27.000 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:27.000 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:27.000 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:27.000 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 31914 2024-09-16T20:51:27.000 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.000 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:27.000 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.000 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:27.000 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.635328+0000", 2024-09-16T20:51:27.000 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.000 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.000 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.000 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:27.001 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.635328+0000", 2024-09-16T20:51:27.001 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.001 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.001 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:27.001 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.635330+0000", 2024-09-16T20:51:27.001 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3379999999999999e-06 2024-09-16T20:51:27.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.001 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.001 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:27.001 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.635334+0000", 2024-09-16T20:51:27.001 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6280000000000004e-06 2024-09-16T20:51:27.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.002 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:27.002 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.635344+0000", 2024-09-16T20:51:27.002 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7229999999999999e-06 2024-09-16T20:51:27.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.002 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:27.002 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.635349+0000", 2024-09-16T20:51:27.002 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1719999999999998e-06 2024-09-16T20:51:27.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.002 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:27.002 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.635424+0000", 2024-09-16T20:51:27.002 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4499999999999995e-05 2024-09-16T20:51:27.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.003 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.003 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:27.003 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.635435+0000", 2024-09-16T20:51:27.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1355000000000001e-05 2024-09-16T20:51:27.003 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.003 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:27.003 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.003 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.003 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-16T20:51:27.003 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 2 2024-09-16T20:51:27.003 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:27.004 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:51:27.088 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:27.087+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:27.151 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:27.152 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:51:27.152 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:51:27.152 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:51:27.152 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:27.160 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:51:27.189 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:27.188+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:27.269 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:27.269 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:51:27.269 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:51:27.269 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:51:27.269 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.269 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:3088 3.1e 3:79e5ec01:::rbd_header.110e891a9b2b:head [watch watch cookie 140213437171488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:51:27.269 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:51:27.269 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 183.10324058899999, 2024-09-16T20:51:27.269 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.11895186200000001, 2024-09-16T20:51:27.269 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:27.269 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:27.270 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:27.270 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:27.270 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:27.270 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:27.270 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3088 2024-09-16T20:51:27.270 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.270 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:27.270 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.270 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:27.270 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:51:27.270 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.270 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.270 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.270 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:27.271 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:51:27.271 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.271 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.271 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:27.271 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163846+0000", 2024-09-16T20:51:27.271 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6799999999999999e-07 2024-09-16T20:51:27.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.271 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.271 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:27.271 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163850+0000", 2024-09-16T20:51:27.271 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.929e-06 2024-09-16T20:51:27.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.272 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.272 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:27.272 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163862+0000", 2024-09-16T20:51:27.272 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1972e-05 2024-09-16T20:51:27.272 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.272 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.272 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:27.272 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163867+0000", 2024-09-16T20:51:27.272 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.451e-06 2024-09-16T20:51:27.272 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.272 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.272 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:27.272 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163912+0000", 2024-09-16T20:51:27.273 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4456000000000003e-05 2024-09-16T20:51:27.273 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.273 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.273 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:27.273 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163962+0000", 2024-09-16T20:51:27.273 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0578000000000003e-05 2024-09-16T20:51:27.273 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.273 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.273 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-16T20:51:27.273 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164058+0000", 2024-09-16T20:51:27.273 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5934999999999995e-05 2024-09-16T20:51:27.273 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.273 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.273 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:51:27.274 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164570+0000", 2024-09-16T20:51:27.274 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00051241100000000005 2024-09-16T20:51:27.274 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.274 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.274 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:27.274 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282629+0000", 2024-09-16T20:51:27.274 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.118058986 2024-09-16T20:51:27.274 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.274 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.274 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:51:27.274 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282725+0000", 2024-09-16T20:51:27.274 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5063999999999996e-05 2024-09-16T20:51:27.274 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.274 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.275 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:27.275 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282797+0000", 2024-09-16T20:51:27.275 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2212000000000006e-05 2024-09-16T20:51:27.275 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.275 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:27.275 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.275 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.275 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.275 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:8039 3.1e 3.ac44ff9e (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:51:27.275 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:51:27.275 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 157.55941324, 2024-09-16T20:51:27.275 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0053310449999999, 2024-09-16T20:51:27.275 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:27.275 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:27.276 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:27.276 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:27.276 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:27.276 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:27.276 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8039 2024-09-16T20:51:27.276 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.276 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:27.276 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.276 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:27.276 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:51:27.276 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.276 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.276 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.277 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:27.277 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:51:27.277 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.277 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.277 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.277 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:27.277 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707674+0000", 2024-09-16T20:51:27.277 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6530000000000001e-06 2024-09-16T20:51:27.277 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.277 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.277 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:27.277 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707681+0000", 2024-09-16T20:51:27.277 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9639999999999997e-06 2024-09-16T20:51:27.277 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.278 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.278 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:27.278 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707700+0000", 2024-09-16T20:51:27.278 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8674000000000001e-05 2024-09-16T20:51:27.278 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.278 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.278 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:27.278 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707708+0000", 2024-09-16T20:51:27.278 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6810000000000004e-06 2024-09-16T20:51:27.278 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.278 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.278 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:27.278 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711293+0000", 2024-09-16T20:51:27.278 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035848189999999999 2024-09-16T20:51:27.279 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.279 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.279 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:27.279 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711301+0000", 2024-09-16T20:51:27.279 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4730000000000003e-06 2024-09-16T20:51:27.279 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.279 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.279 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:27.279 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.712990+0000", 2024-09-16T20:51:27.279 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.001689348 2024-09-16T20:51:27.279 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.279 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.280 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:27.280 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713003+0000", 2024-09-16T20:51:27.280 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3433e-05 2024-09-16T20:51:27.280 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.280 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:27.280 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.280 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.280 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.280 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:8040 3.1e 3.ac44ff9e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:51:27.280 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:51:27.280 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 157.55935284500001, 2024-09-16T20:51:27.280 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0053410780000001, 2024-09-16T20:51:27.280 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:27.280 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:27.281 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:27.281 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:27.281 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:27.281 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:27.281 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8040 2024-09-16T20:51:27.281 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.281 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:27.281 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.281 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:27.281 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:51:27.281 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.281 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.281 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.281 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:27.282 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:51:27.282 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.282 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.282 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.282 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:27.282 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:51:27.282 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8100000000000003e-07 2024-09-16T20:51:27.282 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.282 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.282 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:27.282 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707737+0000", 2024-09-16T20:51:27.282 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5020000000000001e-06 2024-09-16T20:51:27.282 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.283 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.283 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:27.283 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707745+0000", 2024-09-16T20:51:27.283 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2169999999999994e-06 2024-09-16T20:51:27.283 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.283 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.283 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:27.283 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707751+0000", 2024-09-16T20:51:27.283 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.263e-06 2024-09-16T20:51:27.283 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.283 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.283 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:27.283 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711336+0000", 2024-09-16T20:51:27.283 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035848360000000001 2024-09-16T20:51:27.284 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.284 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.284 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:27.284 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711340+0000", 2024-09-16T20:51:27.284 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1309999999999999e-06 2024-09-16T20:51:27.284 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.284 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.284 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:27.284 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713060+0000", 2024-09-16T20:51:27.284 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0017195059999999 2024-09-16T20:51:27.284 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.284 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.285 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:27.285 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713074+0000", 2024-09-16T20:51:27.285 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4042e-05 2024-09-16T20:51:27.285 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.285 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:27.285 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.285 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.285 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.285 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4927.0:4487 3.1e 3.ac44ff9e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:51:27.285 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:51:27.285 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 157.55883264299999, 2024-09-16T20:51:27.285 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0048840880000001, 2024-09-16T20:51:27.285 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:27.286 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:27.286 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:27.286 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:27.286 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4927", 2024-09-16T20:51:27.286 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:27.286 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4487 2024-09-16T20:51:27.286 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.286 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:27.286 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.286 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:27.286 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:51:27.286 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.286 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.286 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.287 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:27.287 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:51:27.287 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.287 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.287 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:27.287 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708254+0000", 2024-09-16T20:51:27.287 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1800000000000005e-07 2024-09-16T20:51:27.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.287 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.287 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:27.287 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708258+0000", 2024-09-16T20:51:27.287 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8689999999999997e-06 2024-09-16T20:51:27.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.288 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.288 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:27.288 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708267+0000", 2024-09-16T20:51:27.288 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9590000000000007e-06 2024-09-16T20:51:27.288 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.288 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.288 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:27.288 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708274+0000", 2024-09-16T20:51:27.288 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9970000000000001e-06 2024-09-16T20:51:27.288 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.288 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.288 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:27.288 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711366+0000", 2024-09-16T20:51:27.289 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030919530000000001 2024-09-16T20:51:27.289 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.289 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.289 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:27.289 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711372+0000", 2024-09-16T20:51:27.289 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9410000000000004e-06 2024-09-16T20:51:27.289 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.289 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.289 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:27.289 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713126+0000", 2024-09-16T20:51:27.289 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0017537249999999 2024-09-16T20:51:27.289 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.289 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.289 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:27.290 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713137+0000", 2024-09-16T20:51:27.290 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0825999999999999e-05 2024-09-16T20:51:27.290 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.290 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:27.290 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.290 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.290 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.290 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5089.0:11703 3.1b 3:dcd4684e:::rbd_data.110e747f0779.000000000000001b:head [set-alloc-hint object_size 262144 write_size 262144,write 139264~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:51:27.290 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:51:27.290 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 143.12909686899999, 2024-09-16T20:51:27.290 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.084231385000000006, 2024-09-16T20:51:27.290 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:27.290 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:27.290 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:27.291 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:27.291 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5089", 2024-09-16T20:51:27.291 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:51:27.291 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 11703 2024-09-16T20:51:27.291 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.291 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:27.291 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.291 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:27.291 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:51:27.291 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.291 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.291 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.291 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:27.292 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:51:27.292 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.292 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.292 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.292 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:27.292 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:51:27.292 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2600000000000001e-07 2024-09-16T20:51:27.292 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.292 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.292 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:27.292 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137998+0000", 2024-09-16T20:51:27.293 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9549999999999998e-06 2024-09-16T20:51:27.293 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.293 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.293 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:27.293 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138008+0000", 2024-09-16T20:51:27.293 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3789999999999995e-06 2024-09-16T20:51:27.293 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.293 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.293 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:27.293 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138013+0000", 2024-09-16T20:51:27.293 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2209999999999996e-06 2024-09-16T20:51:27.293 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.293 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.294 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:27.294 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138054+0000", 2024-09-16T20:51:27.294 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1022999999999997e-05 2024-09-16T20:51:27.294 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.294 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.294 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:27.294 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138124+0000", 2024-09-16T20:51:27.294 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0234999999999994e-05 2024-09-16T20:51:27.294 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.294 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.294 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:27.294 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138197+0000", 2024-09-16T20:51:27.294 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2451000000000001e-05 2024-09-16T20:51:27.294 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.295 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.295 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:27.295 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222010+0000", 2024-09-16T20:51:27.295 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.083813433000000007 2024-09-16T20:51:27.295 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.295 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.295 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:51:27.295 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222163+0000", 2024-09-16T20:51:27.295 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015274599999999999 2024-09-16T20:51:27.295 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.295 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.295 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:51:27.295 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222197+0000", 2024-09-16T20:51:27.295 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3683000000000003e-05 2024-09-16T20:51:27.295 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.296 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.296 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:27.296 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222220+0000", 2024-09-16T20:51:27.296 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3433000000000001e-05 2024-09-16T20:51:27.296 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.296 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:27.296 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.296 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.296 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.296 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.5089.0:11702 3.15 e73/64 3:acf33c8b:::rbd_data.110e747f0779.000000000000001a:head v 73'132, mlcod=73'130)", 2024-09-16T20:51:27.296 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:51:27.296 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 143.12882954, 2024-09-16T20:51:27.297 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.084079763000000002, 2024-09-16T20:51:27.297 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:27.297 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:27.297 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:27.297 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:27.297 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.297 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:27.297 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:51:27.297 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.297 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.297 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.297 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:27.297 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:51:27.297 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.297 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.298 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.298 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:27.298 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:51:27.298 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:51:27.298 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.298 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.298 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:27.298 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138263+0000", 2024-09-16T20:51:27.298 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1e-06 2024-09-16T20:51:27.298 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.298 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.298 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:27.299 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221589+0000", 2024-09-16T20:51:27.299 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.083326002999999996 2024-09-16T20:51:27.299 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.299 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.299 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:27.299 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221614+0000", 2024-09-16T20:51:27.299 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5593999999999999e-05 2024-09-16T20:51:27.299 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.299 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.299 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:27.299 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221714+0000", 2024-09-16T20:51:27.299 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9889000000000005e-05 2024-09-16T20:51:27.299 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.300 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.300 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:27.300 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221727+0000", 2024-09-16T20:51:27.300 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3125999999999999e-05 2024-09-16T20:51:27.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.300 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.300 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:51:27.300 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222294+0000", 2024-09-16T20:51:27.300 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00056712199999999998 2024-09-16T20:51:27.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.300 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.300 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:27.300 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222336+0000", 2024-09-16T20:51:27.300 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1470000000000001e-05 2024-09-16T20:51:27.301 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.301 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:27.301 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.301 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.301 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.301 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5138.0:9358 3.c 3:33996a43:::rbd_data.110e6372565d.0000000000000007:head [read 1343488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T20:51:27.301 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:51:27.301 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 119.41455413, 2024-09-16T20:51:27.301 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0167493569999999, 2024-09-16T20:51:27.301 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:27.301 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:27.301 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:27.301 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:27.301 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5138", 2024-09-16T20:51:27.302 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:51:27.302 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9358 2024-09-16T20:51:27.302 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.302 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:27.302 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.302 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:27.302 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:51:27.302 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.302 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.302 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.302 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:27.302 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:51:27.302 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.302 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.302 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.303 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:27.303 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852533+0000", 2024-09-16T20:51:27.303 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.646e-06 2024-09-16T20:51:27.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.303 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.303 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:27.303 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852539+0000", 2024-09-16T20:51:27.303 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3389999999999998e-06 2024-09-16T20:51:27.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.303 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.303 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:27.303 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852552+0000", 2024-09-16T20:51:27.303 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.271e-05 2024-09-16T20:51:27.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.304 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.304 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:27.304 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852563+0000", 2024-09-16T20:51:27.304 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1139e-05 2024-09-16T20:51:27.304 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.304 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.304 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:27.304 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852883+0000", 2024-09-16T20:51:27.304 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00032010600000000001 2024-09-16T20:51:27.304 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.304 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.304 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:27.304 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852890+0000", 2024-09-16T20:51:27.304 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7669999999999998e-06 2024-09-16T20:51:27.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.305 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:27.305 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869059+0000", 2024-09-16T20:51:27.305 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.016169353 2024-09-16T20:51:27.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.305 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:27.305 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869178+0000", 2024-09-16T20:51:27.305 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118193 2024-09-16T20:51:27.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.305 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:27.305 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869281+0000", 2024-09-16T20:51:27.306 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010310399999999999 2024-09-16T20:51:27.306 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.306 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:27.306 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.306 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.306 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.306 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5138.0:9382 3.c 3:33996a43:::rbd_data.110e6372565d.0000000000000007:head [read 0~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T20:51:27.306 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:51:27.306 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 119.41450325300001, 2024-09-16T20:51:27.306 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.017136034, 2024-09-16T20:51:27.306 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:27.306 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:27.306 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:27.306 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:27.307 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5138", 2024-09-16T20:51:27.307 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:51:27.307 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9382 2024-09-16T20:51:27.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.307 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:27.307 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.307 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:27.307 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:51:27.307 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.307 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.307 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:27.307 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:51:27.307 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.308 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.308 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:27.308 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852583+0000", 2024-09-16T20:51:27.308 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.51e-07 2024-09-16T20:51:27.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.308 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.308 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:27.308 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852586+0000", 2024-09-16T20:51:27.308 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0280000000000001e-06 2024-09-16T20:51:27.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.308 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.308 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:27.308 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852595+0000", 2024-09-16T20:51:27.309 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2799999999999992e-06 2024-09-16T20:51:27.309 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.309 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.309 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:27.309 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852601+0000", 2024-09-16T20:51:27.309 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3810000000000004e-06 2024-09-16T20:51:27.309 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.309 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.309 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:27.309 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852921+0000", 2024-09-16T20:51:27.309 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00031979699999999997 2024-09-16T20:51:27.309 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.309 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.309 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:27.309 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852930+0000", 2024-09-16T20:51:27.310 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6489999999999997e-06 2024-09-16T20:51:27.310 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.310 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.310 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:27.310 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869434+0000", 2024-09-16T20:51:27.310 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0165038829999999 2024-09-16T20:51:27.310 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.310 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.310 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:27.310 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869495+0000", 2024-09-16T20:51:27.310 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1509999999999999e-05 2024-09-16T20:51:27.310 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.310 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.310 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:27.310 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869718+0000", 2024-09-16T20:51:27.310 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000222955 2024-09-16T20:51:27.311 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.311 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:27.311 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.311 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.311 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9969 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 632832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:27.311 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642045+0000", 2024-09-16T20:51:27.311 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 96.625040795999993, 2024-09-16T20:51:27.311 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076903192999999995, 2024-09-16T20:51:27.311 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:27.311 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:27.311 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:27.311 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:27.311 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:27.312 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:27.312 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9969 2024-09-16T20:51:27.312 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.312 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:27.312 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.312 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:27.312 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642045+0000", 2024-09-16T20:51:27.312 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.312 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.312 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.312 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:27.312 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642045+0000", 2024-09-16T20:51:27.312 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.312 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.312 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.313 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:27.313 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642045+0000", 2024-09-16T20:51:27.313 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5100000000000001e-07 2024-09-16T20:51:27.313 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.313 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.313 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:27.313 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642047+0000", 2024-09-16T20:51:27.313 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8679999999999999e-06 2024-09-16T20:51:27.313 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.313 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.313 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:27.313 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642053+0000", 2024-09-16T20:51:27.313 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1720000000000004e-06 2024-09-16T20:51:27.314 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.314 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.314 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:27.314 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642060+0000", 2024-09-16T20:51:27.314 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0339999999999999e-06 2024-09-16T20:51:27.314 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.314 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.314 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:27.314 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.718562+0000", 2024-09-16T20:51:27.314 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076501969000000003 2024-09-16T20:51:27.314 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.314 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.314 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:27.314 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.718643+0000", 2024-09-16T20:51:27.314 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0539999999999998e-05 2024-09-16T20:51:27.314 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.315 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.315 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:27.315 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.718948+0000", 2024-09-16T20:51:27.315 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000305259 2024-09-16T20:51:27.315 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.315 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:27.315 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.315 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.315 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.315 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9970 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 637952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:27.315 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642067+0000", 2024-09-16T20:51:27.315 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 96.625018815999994, 2024-09-16T20:51:27.315 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077174928000000004, 2024-09-16T20:51:27.315 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:27.316 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:27.316 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:27.316 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:27.316 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:27.316 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:27.316 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9970 2024-09-16T20:51:27.316 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.316 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:27.316 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.316 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:27.316 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642067+0000", 2024-09-16T20:51:27.316 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.316 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.316 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.316 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:27.317 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642067+0000", 2024-09-16T20:51:27.317 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.317 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.317 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:27.317 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642067+0000", 2024-09-16T20:51:27.317 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2499999999999995e-07 2024-09-16T20:51:27.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.317 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.317 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:27.317 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642070+0000", 2024-09-16T20:51:27.317 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3199999999999998e-06 2024-09-16T20:51:27.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.317 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.317 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:27.318 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642076+0000", 2024-09-16T20:51:27.318 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.122e-06 2024-09-16T20:51:27.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.318 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.318 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:27.318 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642081+0000", 2024-09-16T20:51:27.318 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0869999999999997e-06 2024-09-16T20:51:27.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.318 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.318 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:27.318 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.718980+0000", 2024-09-16T20:51:27.318 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076899482000000005 2024-09-16T20:51:27.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.318 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.319 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:27.319 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719085+0000", 2024-09-16T20:51:27.319 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010446399999999999 2024-09-16T20:51:27.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.319 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.319 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:27.319 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719242+0000", 2024-09-16T20:51:27.319 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000156928 2024-09-16T20:51:27.319 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.319 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:27.319 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.319 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.319 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9971 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 643072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:27.320 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642102+0000", 2024-09-16T20:51:27.320 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 96.624983108999999, 2024-09-16T20:51:27.320 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077421016999999995, 2024-09-16T20:51:27.320 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:27.320 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:27.320 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:27.320 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:27.320 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:27.320 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:27.320 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9971 2024-09-16T20:51:27.320 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.320 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:27.320 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.320 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:27.321 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642102+0000", 2024-09-16T20:51:27.321 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.321 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.321 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.321 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:27.321 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642102+0000", 2024-09-16T20:51:27.321 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.321 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.321 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.321 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:27.321 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642103+0000", 2024-09-16T20:51:27.321 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7399999999999998e-07 2024-09-16T20:51:27.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.322 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.322 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:27.322 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642105+0000", 2024-09-16T20:51:27.322 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2649999999999999e-06 2024-09-16T20:51:27.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.322 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.322 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:27.322 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642112+0000", 2024-09-16T20:51:27.322 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.294e-06 2024-09-16T20:51:27.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.322 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.323 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:27.323 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642118+0000", 2024-09-16T20:51:27.323 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4500000000000003e-06 2024-09-16T20:51:27.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.323 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.323 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:27.323 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719285+0000", 2024-09-16T20:51:27.323 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077166883000000006 2024-09-16T20:51:27.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.323 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.323 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:27.323 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719368+0000", 2024-09-16T20:51:27.323 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2811000000000004e-05 2024-09-16T20:51:27.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.324 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.324 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:27.324 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719523+0000", 2024-09-16T20:51:27.324 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015584 2024-09-16T20:51:27.324 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.324 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:27.324 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.324 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.324 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.324 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9972 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 648192~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:27.324 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642125+0000", 2024-09-16T20:51:27.324 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 96.624960161999994, 2024-09-16T20:51:27.324 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077710324999999997, 2024-09-16T20:51:27.324 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:27.325 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:27.325 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:27.325 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:27.325 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:27.325 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:27.325 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9972 2024-09-16T20:51:27.325 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.325 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:27.325 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.325 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:27.325 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642125+0000", 2024-09-16T20:51:27.325 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.325 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.325 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.326 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:27.326 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642125+0000", 2024-09-16T20:51:27.326 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.326 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.326 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.326 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:27.326 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642126+0000", 2024-09-16T20:51:27.326 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1899999999999998e-07 2024-09-16T20:51:27.326 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.326 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.326 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:27.326 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642128+0000", 2024-09-16T20:51:27.326 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8619999999999999e-06 2024-09-16T20:51:27.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.327 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.327 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:27.327 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642134+0000", 2024-09-16T20:51:27.327 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5289999999999997e-06 2024-09-16T20:51:27.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.327 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.327 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:27.327 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642139+0000", 2024-09-16T20:51:27.327 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1250000000000001e-06 2024-09-16T20:51:27.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.327 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.327 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:27.327 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719570+0000", 2024-09-16T20:51:27.328 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077430516000000005 2024-09-16T20:51:27.328 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.328 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.328 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:27.328 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719655+0000", 2024-09-16T20:51:27.328 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5500000000000005e-05 2024-09-16T20:51:27.328 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.328 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.328 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:27.328 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719836+0000", 2024-09-16T20:51:27.328 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018037399999999999 2024-09-16T20:51:27.328 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.328 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:27.329 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.329 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.329 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.329 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9973 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 653312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:27.329 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:51:27.329 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 96.624939552000001, 2024-09-16T20:51:27.329 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077905262000000003, 2024-09-16T20:51:27.329 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:27.329 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:27.329 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:27.329 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:27.329 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:27.329 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:27.329 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9973 2024-09-16T20:51:27.330 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.330 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:27.330 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.330 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:27.330 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:51:27.330 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.330 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.330 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.330 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:27.330 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:51:27.330 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.330 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.330 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.330 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:27.330 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:51:27.331 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:51:27.331 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.331 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.331 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:27.331 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642148+0000", 2024-09-16T20:51:27.331 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0219999999999999e-06 2024-09-16T20:51:27.331 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.331 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.331 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:27.331 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642154+0000", 2024-09-16T20:51:27.331 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8540000000000002e-06 2024-09-16T20:51:27.331 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.331 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.331 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:27.331 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642159+0000", 2024-09-16T20:51:27.332 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.045e-06 2024-09-16T20:51:27.332 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.332 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.332 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:27.332 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719875+0000", 2024-09-16T20:51:27.332 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077716116000000002 2024-09-16T20:51:27.332 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.332 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.332 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:27.332 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719934+0000", 2024-09-16T20:51:27.332 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8819e-05 2024-09-16T20:51:27.332 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.332 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.332 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:27.332 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720051+0000", 2024-09-16T20:51:27.333 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000116947 2024-09-16T20:51:27.333 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.333 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:27.333 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.333 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.333 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.333 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9974 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 658432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:27.333 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642166+0000", 2024-09-16T20:51:27.333 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 96.624919637999994, 2024-09-16T20:51:27.333 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.078179786000000001, 2024-09-16T20:51:27.333 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:27.333 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:27.333 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:27.333 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:27.333 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:27.334 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:27.334 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9974 2024-09-16T20:51:27.334 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.334 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:27.334 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.334 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:27.334 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642166+0000", 2024-09-16T20:51:27.334 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.334 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.334 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.334 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:27.334 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642166+0000", 2024-09-16T20:51:27.334 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.334 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.334 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.335 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:27.335 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647343+0000", 2024-09-16T20:51:27.335 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0051769440000000002 2024-09-16T20:51:27.335 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.335 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.335 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:27.335 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647348+0000", 2024-09-16T20:51:27.335 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3619999999999997e-06 2024-09-16T20:51:27.335 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.335 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.335 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:27.335 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647359+0000", 2024-09-16T20:51:27.335 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0929e-05 2024-09-16T20:51:27.335 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.335 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.336 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:27.336 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647366+0000", 2024-09-16T20:51:27.336 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0910000000000004e-06 2024-09-16T20:51:27.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.336 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.336 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:27.336 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720094+0000", 2024-09-16T20:51:27.336 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.072727403999999995 2024-09-16T20:51:27.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.336 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.336 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:27.336 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720173+0000", 2024-09-16T20:51:27.336 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9382000000000007e-05 2024-09-16T20:51:27.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.337 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.337 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:27.337 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720346+0000", 2024-09-16T20:51:27.337 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017267399999999999 2024-09-16T20:51:27.337 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.337 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:27.337 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.337 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.337 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9990 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 740352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:27.337 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647725+0000", 2024-09-16T20:51:27.337 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 96.619360553999996, 2024-09-16T20:51:27.337 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077029919000000002, 2024-09-16T20:51:27.337 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:27.338 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:27.338 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:27.338 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:27.338 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:27.338 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:27.338 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9990 2024-09-16T20:51:27.338 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.338 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:27.338 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.338 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:27.338 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647725+0000", 2024-09-16T20:51:27.338 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.338 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.339 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.339 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:27.339 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647725+0000", 2024-09-16T20:51:27.339 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.339 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.339 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.339 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:27.339 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647725+0000", 2024-09-16T20:51:27.339 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2399999999999999e-07 2024-09-16T20:51:27.339 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.339 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.339 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:27.340 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647726+0000", 2024-09-16T20:51:27.340 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.116e-06 2024-09-16T20:51:27.340 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.340 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.340 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:27.340 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647729+0000", 2024-09-16T20:51:27.340 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.937e-06 2024-09-16T20:51:27.340 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.340 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.340 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:27.340 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647732+0000", 2024-09-16T20:51:27.340 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8739999999999999e-06 2024-09-16T20:51:27.340 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.341 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.341 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:27.341 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724612+0000", 2024-09-16T20:51:27.341 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076880102000000006 2024-09-16T20:51:27.341 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.341 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.341 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:27.341 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724649+0000", 2024-09-16T20:51:27.341 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6738e-05 2024-09-16T20:51:27.341 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.341 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.341 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:27.341 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724755+0000", 2024-09-16T20:51:27.341 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010592800000000001 2024-09-16T20:51:27.342 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.342 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:27.342 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.342 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.342 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.342 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9991 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 745472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:27.342 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:51:27.342 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 96.619350245000007, 2024-09-16T20:51:27.342 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077257820000000005, 2024-09-16T20:51:27.342 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:27.342 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:27.342 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:27.342 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:27.342 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:27.343 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:27.343 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9991 2024-09-16T20:51:27.343 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.343 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:27.343 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.343 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:27.343 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:51:27.343 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.343 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.343 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.343 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:27.343 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:51:27.343 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.344 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.344 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.344 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:27.344 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:51:27.344 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1199999999999999e-07 2024-09-16T20:51:27.344 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.344 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.344 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:27.344 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647736+0000", 2024-09-16T20:51:27.344 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3699999999999999e-07 2024-09-16T20:51:27.344 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.344 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.344 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:27.344 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647739+0000", 2024-09-16T20:51:27.345 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9359999999999999e-06 2024-09-16T20:51:27.345 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.345 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.345 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:27.345 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647742+0000", 2024-09-16T20:51:27.345 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5720000000000001e-06 2024-09-16T20:51:27.345 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.345 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.345 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:27.345 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724774+0000", 2024-09-16T20:51:27.345 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077032457999999998 2024-09-16T20:51:27.345 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.345 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.345 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:27.346 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724818+0000", 2024-09-16T20:51:27.346 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3803000000000001e-05 2024-09-16T20:51:27.346 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.346 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.346 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:27.346 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724993+0000", 2024-09-16T20:51:27.346 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000174902 2024-09-16T20:51:27.346 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.346 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:27.346 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.346 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.346 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.346 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9992 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 750592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:27.346 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:51:27.346 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 96.619340656000006, 2024-09-16T20:51:27.347 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077553227000000002, 2024-09-16T20:51:27.347 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:27.347 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:27.347 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:27.347 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:27.347 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:27.347 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:27.347 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9992 2024-09-16T20:51:27.347 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.347 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:27.347 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.347 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:27.347 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:51:27.347 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.348 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.348 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.348 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:27.348 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:51:27.348 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.348 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.348 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.348 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:27.348 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:51:27.348 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0200000000000001e-07 2024-09-16T20:51:27.348 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.348 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.348 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:27.349 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647746+0000", 2024-09-16T20:51:27.349 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.088e-06 2024-09-16T20:51:27.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.349 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.349 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:27.349 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647749+0000", 2024-09-16T20:51:27.349 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9170000000000002e-06 2024-09-16T20:51:27.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.349 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.349 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:27.349 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647752+0000", 2024-09-16T20:51:27.349 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.774e-06 2024-09-16T20:51:27.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.350 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.350 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:27.350 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725042+0000", 2024-09-16T20:51:27.350 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077289952999999995 2024-09-16T20:51:27.350 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.350 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.350 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:27.350 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725113+0000", 2024-09-16T20:51:27.350 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0920999999999994e-05 2024-09-16T20:51:27.350 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.350 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.350 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:27.351 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725298+0000", 2024-09-16T20:51:27.351 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000185372 2024-09-16T20:51:27.351 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.351 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:27.351 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.351 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.351 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.351 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9993 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 755712~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:27.351 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647755+0000", 2024-09-16T20:51:27.351 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 96.619330629999993, 2024-09-16T20:51:27.351 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077725674999999994, 2024-09-16T20:51:27.351 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:27.351 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:27.352 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:27.352 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:27.352 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:27.352 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:27.352 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9993 2024-09-16T20:51:27.352 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.352 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:27.352 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.352 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:27.352 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647755+0000", 2024-09-16T20:51:27.352 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.352 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.352 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.353 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:27.353 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647755+0000", 2024-09-16T20:51:27.353 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.353 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.353 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.353 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:27.353 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653406+0000", 2024-09-16T20:51:27.353 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.005650876 2024-09-16T20:51:27.353 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.353 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.353 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:27.353 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653411+0000", 2024-09-16T20:51:27.353 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8899999999999998e-06 2024-09-16T20:51:27.354 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.354 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.354 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:27.354 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653423+0000", 2024-09-16T20:51:27.354 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2447e-05 2024-09-16T20:51:27.354 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.354 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.354 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:27.354 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653431+0000", 2024-09-16T20:51:27.354 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5730000000000002e-06 2024-09-16T20:51:27.354 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.354 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.354 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:27.355 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725333+0000", 2024-09-16T20:51:27.355 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.071902743000000005 2024-09-16T20:51:27.355 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.355 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.355 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:27.355 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725385+0000", 2024-09-16T20:51:27.355 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1585000000000002e-05 2024-09-16T20:51:27.355 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.355 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.355 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:27.355 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725481+0000", 2024-09-16T20:51:27.355 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5561000000000002e-05 2024-09-16T20:51:27.355 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.356 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:27.356 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.356 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.356 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.356 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6061.0:5400 3.b 3:d353d652:::rbd_data.110e16eafbd.0000000000000010:head [read 577536~28672] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-16T20:51:27.356 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:27.761367+0000", 2024-09-16T20:51:27.356 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 59.505718684000001, 2024-09-16T20:51:27.356 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0029066630000001, 2024-09-16T20:51:27.356 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:27.356 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:27.356 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:27.356 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:27.356 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6061", 2024-09-16T20:51:27.357 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3930128440", 2024-09-16T20:51:27.357 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5400 2024-09-16T20:51:27.357 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.357 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:27.357 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.357 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:27.357 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761367+0000", 2024-09-16T20:51:27.357 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.357 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.357 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.357 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:27.357 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761367+0000", 2024-09-16T20:51:27.357 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.358 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.358 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.358 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:27.358 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761369+0000", 2024-09-16T20:51:27.358 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.934e-06 2024-09-16T20:51:27.358 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.358 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.358 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:27.358 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761374+0000", 2024-09-16T20:51:27.358 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7139999999999998e-06 2024-09-16T20:51:27.358 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.358 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.358 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:27.358 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761387+0000", 2024-09-16T20:51:27.359 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2349e-05 2024-09-16T20:51:27.359 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.359 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.359 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:27.359 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761393+0000", 2024-09-16T20:51:27.359 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.652e-06 2024-09-16T20:51:27.359 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.359 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.359 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:27.359 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761442+0000", 2024-09-16T20:51:27.359 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9011000000000002e-05 2024-09-16T20:51:27.359 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.359 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.359 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:27.360 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761451+0000", 2024-09-16T20:51:27.360 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1920000000000005e-06 2024-09-16T20:51:27.360 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.360 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.360 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:27.360 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:28.764169+0000", 2024-09-16T20:51:27.360 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.002718161 2024-09-16T20:51:27.360 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.360 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.360 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:27.360 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:28.764273+0000", 2024-09-16T20:51:27.360 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010465 2024-09-16T20:51:27.360 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.361 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:27.361 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.361 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.361 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.361 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6061.0:5424 3.b 3:d353d652:::rbd_data.110e16eafbd.0000000000000010:head [read 606208~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-16T20:51:27.361 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:27.766882+0000", 2024-09-16T20:51:27.361 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 59.500203321000001, 2024-09-16T20:51:27.361 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99753650500000002, 2024-09-16T20:51:27.361 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:27.361 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:27.361 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:27.361 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:27.361 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6061", 2024-09-16T20:51:27.361 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3930128440", 2024-09-16T20:51:27.362 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5424 2024-09-16T20:51:27.362 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.362 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:27.362 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.362 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:27.362 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766882+0000", 2024-09-16T20:51:27.362 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.362 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.362 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.362 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:27.362 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766882+0000", 2024-09-16T20:51:27.362 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.362 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.362 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.363 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:27.363 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766883+0000", 2024-09-16T20:51:27.363 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3289999999999999e-06 2024-09-16T20:51:27.363 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.363 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.363 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:27.363 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766888+0000", 2024-09-16T20:51:27.363 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5800000000000002e-06 2024-09-16T20:51:27.363 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.363 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.363 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:27.363 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766900+0000", 2024-09-16T20:51:27.364 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1861e-05 2024-09-16T20:51:27.364 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.364 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.364 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:27.364 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766909+0000", 2024-09-16T20:51:27.364 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5210000000000008e-06 2024-09-16T20:51:27.364 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.364 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.364 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:27.364 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766958+0000", 2024-09-16T20:51:27.364 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8718999999999999e-05 2024-09-16T20:51:27.364 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.365 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.365 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:27.365 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766975+0000", 2024-09-16T20:51:27.365 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7231000000000002e-05 2024-09-16T20:51:27.365 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.365 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.365 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:27.365 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:28.764327+0000", 2024-09-16T20:51:27.365 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99735112500000001 2024-09-16T20:51:27.365 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.365 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.365 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:27.365 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:28.764419+0000", 2024-09-16T20:51:27.365 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2139000000000007e-05 2024-09-16T20:51:27.366 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.366 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:27.366 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.366 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.366 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:27.366 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:27.367 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:51:27.368 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:27.366+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:27.469 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:27.467+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:27.475 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:27.475 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:51:27.475 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:51:27.476 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:27.484 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:51:27.570 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:27.568+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:27.591 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:27.591 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:51:27.591 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:51:27.591 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:51:27.592 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:27.600 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:51:27.629 INFO:tasks.thrashosds.thrasher:in_osds: [1, 4, 5, 0, 3, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-16T20:51:27.630 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T20:51:27.630 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-16T20:51:27.630 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-16T20:51:27.641 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T20:51:27.675 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:27.674+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:27.751 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:27.751 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:51:27.751 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:51:27.752 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:51:27.752 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.752 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12548 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 628736~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:27.752 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757009+0000", 2024-09-16T20:51:27.752 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 96.992837588, 2024-09-16T20:51:27.752 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065743956000000006, 2024-09-16T20:51:27.752 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:27.752 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:27.752 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:27.752 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:27.752 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:27.752 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:27.753 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12548 2024-09-16T20:51:27.753 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.753 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:27.753 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.753 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:27.753 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757009+0000", 2024-09-16T20:51:27.753 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.753 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.753 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.753 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:27.753 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757009+0000", 2024-09-16T20:51:27.753 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.753 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.753 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.754 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:27.754 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757010+0000", 2024-09-16T20:51:27.754 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9900000000000001e-07 2024-09-16T20:51:27.754 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.754 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.754 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:27.754 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757012+0000", 2024-09-16T20:51:27.754 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2689999999999998e-06 2024-09-16T20:51:27.754 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.754 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.754 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:27.754 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757022+0000", 2024-09-16T20:51:27.754 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9699999999999994e-06 2024-09-16T20:51:27.754 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.754 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.755 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:27.755 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757028+0000", 2024-09-16T20:51:27.755 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3799999999999999e-06 2024-09-16T20:51:27.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.755 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.755 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:27.755 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822552+0000", 2024-09-16T20:51:27.755 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065524281000000004 2024-09-16T20:51:27.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.755 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.755 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:27.755 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822613+0000", 2024-09-16T20:51:27.755 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0430999999999998e-05 2024-09-16T20:51:27.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.756 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.756 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:27.756 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822753+0000", 2024-09-16T20:51:27.756 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000140226 2024-09-16T20:51:27.756 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.756 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:27.756 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.756 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.756 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.756 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12549 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 633856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:27.756 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:51:27.756 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 96.992808840999999, 2024-09-16T20:51:27.756 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065978518, 2024-09-16T20:51:27.756 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:27.757 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:27.757 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:27.757 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:27.757 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:27.757 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:27.757 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12549 2024-09-16T20:51:27.757 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.757 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:27.757 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.757 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:27.757 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:51:27.757 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.757 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.757 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.758 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:27.758 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:51:27.758 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.758 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.758 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.758 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:27.758 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:51:27.758 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9299999999999999e-07 2024-09-16T20:51:27.758 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.758 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.758 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:27.758 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757040+0000", 2024-09-16T20:51:27.758 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.677e-06 2024-09-16T20:51:27.758 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.759 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.759 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:27.759 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757045+0000", 2024-09-16T20:51:27.759 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8149999999999996e-06 2024-09-16T20:51:27.759 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.759 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.759 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:27.759 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757048+0000", 2024-09-16T20:51:27.759 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1e-06 2024-09-16T20:51:27.759 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.759 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.759 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:27.759 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822796+0000", 2024-09-16T20:51:27.759 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065748081 2024-09-16T20:51:27.759 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.760 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.760 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:27.760 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822867+0000", 2024-09-16T20:51:27.760 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0679000000000005e-05 2024-09-16T20:51:27.760 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.760 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.760 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:27.760 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823016+0000", 2024-09-16T20:51:27.760 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014987299999999999 2024-09-16T20:51:27.760 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.760 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:27.760 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.760 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.761 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.761 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12550 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 638976~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:27.761 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757060+0000", 2024-09-16T20:51:27.761 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 96.992786425999995, 2024-09-16T20:51:27.761 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066153564999999998, 2024-09-16T20:51:27.761 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:27.761 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:27.761 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:27.761 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:27.761 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:27.761 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:27.761 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12550 2024-09-16T20:51:27.761 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.761 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:27.761 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.762 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:27.762 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757060+0000", 2024-09-16T20:51:27.762 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.762 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.762 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.762 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:27.762 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757060+0000", 2024-09-16T20:51:27.762 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.762 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.762 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.762 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:27.762 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757061+0000", 2024-09-16T20:51:27.762 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.15e-07 2024-09-16T20:51:27.762 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.762 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.763 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:27.763 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757062+0000", 2024-09-16T20:51:27.763 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.663e-06 2024-09-16T20:51:27.763 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.763 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.763 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:27.763 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757066+0000", 2024-09-16T20:51:27.763 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.817e-06 2024-09-16T20:51:27.763 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.763 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.763 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:27.763 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757070+0000", 2024-09-16T20:51:27.763 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1509999999999997e-06 2024-09-16T20:51:27.763 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.764 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.764 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:27.764 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823044+0000", 2024-09-16T20:51:27.764 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065973992999999995 2024-09-16T20:51:27.764 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.764 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.764 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:27.764 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823082+0000", 2024-09-16T20:51:27.764 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8050999999999999e-05 2024-09-16T20:51:27.764 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.764 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.764 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:27.765 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823214+0000", 2024-09-16T20:51:27.765 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013157500000000001 2024-09-16T20:51:27.765 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.765 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:27.765 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.765 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.765 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.765 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12551 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 644096~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:27.765 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:51:27.765 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 96.992772728000006, 2024-09-16T20:51:27.765 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066391378000000001, 2024-09-16T20:51:27.765 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:27.765 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:27.765 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:27.766 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:27.766 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:27.766 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:27.766 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12551 2024-09-16T20:51:27.766 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.766 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:27.766 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.766 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:27.766 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:51:27.766 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.766 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.766 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.766 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:27.766 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:51:27.766 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.766 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.767 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.767 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:27.767 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:51:27.767 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2999999999999999e-07 2024-09-16T20:51:27.767 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.767 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.767 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:27.767 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757075+0000", 2024-09-16T20:51:27.767 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.097e-06 2024-09-16T20:51:27.767 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.767 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.767 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:27.767 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757082+0000", 2024-09-16T20:51:27.768 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9500000000000004e-06 2024-09-16T20:51:27.768 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.768 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.768 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:27.768 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757086+0000", 2024-09-16T20:51:27.768 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2229999999999999e-06 2024-09-16T20:51:27.768 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.768 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.768 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:27.768 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823262+0000", 2024-09-16T20:51:27.768 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066176078999999999 2024-09-16T20:51:27.768 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.768 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.769 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:27.769 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823336+0000", 2024-09-16T20:51:27.769 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3979000000000004e-05 2024-09-16T20:51:27.769 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.769 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.769 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:27.769 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823465+0000", 2024-09-16T20:51:27.769 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012982 2024-09-16T20:51:27.769 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.769 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:27.769 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.769 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.769 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.770 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12552 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 649216~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:27.770 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757089+0000", 2024-09-16T20:51:27.770 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 96.992757587, 2024-09-16T20:51:27.770 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066655617, 2024-09-16T20:51:27.770 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:27.770 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:27.770 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:27.770 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:27.770 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:27.770 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:27.770 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12552 2024-09-16T20:51:27.770 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.770 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:27.770 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.771 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:27.771 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757089+0000", 2024-09-16T20:51:27.771 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.771 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.771 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.771 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:27.771 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757089+0000", 2024-09-16T20:51:27.771 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.771 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.771 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.771 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:27.771 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759492+0000", 2024-09-16T20:51:27.772 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0024026109999999998 2024-09-16T20:51:27.772 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.772 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.772 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:27.772 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759497+0000", 2024-09-16T20:51:27.772 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1120000000000004e-06 2024-09-16T20:51:27.772 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.772 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.772 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:27.772 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759508+0000", 2024-09-16T20:51:27.772 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0981e-05 2024-09-16T20:51:27.772 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.772 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.773 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:27.773 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759516+0000", 2024-09-16T20:51:27.773 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7270000000000007e-06 2024-09-16T20:51:27.773 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.773 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.773 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:27.773 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823504+0000", 2024-09-16T20:51:27.773 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.063987984999999997 2024-09-16T20:51:27.773 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.773 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.773 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:27.773 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823579+0000", 2024-09-16T20:51:27.773 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5703000000000003e-05 2024-09-16T20:51:27.773 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.774 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.774 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:27.774 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823745+0000", 2024-09-16T20:51:27.774 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016549800000000001 2024-09-16T20:51:27.774 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.774 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:27.774 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.774 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.774 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.774 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12560 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 690176~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:27.774 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.759658+0000", 2024-09-16T20:51:27.774 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 96.990188384999996, 2024-09-16T20:51:27.774 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065858517000000005, 2024-09-16T20:51:27.774 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:27.774 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:27.775 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:27.775 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:27.775 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:27.775 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:27.775 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12560 2024-09-16T20:51:27.775 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.775 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:27.775 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.775 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:27.775 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759658+0000", 2024-09-16T20:51:27.775 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.775 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.775 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.775 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:27.776 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759658+0000", 2024-09-16T20:51:27.776 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.776 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.776 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.776 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:27.776 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759659+0000", 2024-09-16T20:51:27.776 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.17e-07 2024-09-16T20:51:27.776 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.776 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.776 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:27.776 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759660+0000", 2024-09-16T20:51:27.776 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0189999999999999e-06 2024-09-16T20:51:27.776 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.777 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.777 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:27.777 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759663+0000", 2024-09-16T20:51:27.777 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0010000000000002e-06 2024-09-16T20:51:27.777 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.777 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.777 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:27.777 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759665+0000", 2024-09-16T20:51:27.777 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5440000000000001e-06 2024-09-16T20:51:27.777 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.777 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.777 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:27.778 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825355+0000", 2024-09-16T20:51:27.778 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065690224000000005 2024-09-16T20:51:27.778 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.778 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.778 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:27.778 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825394+0000", 2024-09-16T20:51:27.778 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8810999999999998e-05 2024-09-16T20:51:27.778 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.778 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.778 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:27.778 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825517+0000", 2024-09-16T20:51:27.778 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012270099999999999 2024-09-16T20:51:27.778 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.778 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:27.779 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.779 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.779 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.779 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12561 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 695296~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:27.779 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.759668+0000", 2024-09-16T20:51:27.779 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 96.990178542999999, 2024-09-16T20:51:27.779 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066116999999999995, 2024-09-16T20:51:27.779 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:27.779 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:27.779 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:27.779 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:27.779 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:27.779 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:27.779 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12561 2024-09-16T20:51:27.780 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.780 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:27.780 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.780 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:27.780 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759668+0000", 2024-09-16T20:51:27.780 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.780 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.780 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.780 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:27.780 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759668+0000", 2024-09-16T20:51:27.780 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.780 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.780 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.781 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:27.781 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765379+0000", 2024-09-16T20:51:27.781 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057104410000000001 2024-09-16T20:51:27.781 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.781 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.781 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:27.781 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765383+0000", 2024-09-16T20:51:27.781 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2980000000000002e-06 2024-09-16T20:51:27.781 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.781 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.781 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:27.781 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765395+0000", 2024-09-16T20:51:27.781 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1671000000000001e-05 2024-09-16T20:51:27.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.782 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.782 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:27.782 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765403+0000", 2024-09-16T20:51:27.782 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1210000000000007e-06 2024-09-16T20:51:27.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.782 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.782 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:27.782 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825552+0000", 2024-09-16T20:51:27.782 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.060149498000000003 2024-09-16T20:51:27.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.782 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.782 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:27.783 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825625+0000", 2024-09-16T20:51:27.783 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2540999999999995e-05 2024-09-16T20:51:27.783 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.783 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.783 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:27.783 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825785+0000", 2024-09-16T20:51:27.783 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016043000000000001 2024-09-16T20:51:27.783 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.783 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:27.783 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.783 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.783 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.783 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12585 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 818176~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:27.783 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.765848+0000", 2024-09-16T20:51:27.784 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 96.983998572000004, 2024-09-16T20:51:27.784 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065843218999999995, 2024-09-16T20:51:27.784 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:27.784 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:27.784 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:27.784 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:27.784 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:27.784 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:27.784 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12585 2024-09-16T20:51:27.784 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.784 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:27.784 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.784 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:27.784 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765848+0000", 2024-09-16T20:51:27.785 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.785 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.785 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.785 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:27.785 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765848+0000", 2024-09-16T20:51:27.785 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.785 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.785 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.785 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:27.785 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768683+0000", 2024-09-16T20:51:27.785 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0028344519999999999 2024-09-16T20:51:27.785 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.785 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.785 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:27.786 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768686+0000", 2024-09-16T20:51:27.786 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8789999999999996e-06 2024-09-16T20:51:27.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.786 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:27.786 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768696+0000", 2024-09-16T20:51:27.786 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6709999999999994e-06 2024-09-16T20:51:27.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.786 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:27.786 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768702+0000", 2024-09-16T20:51:27.786 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2550000000000003e-06 2024-09-16T20:51:27.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.787 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:27.787 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.831499+0000", 2024-09-16T20:51:27.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.062797081000000005 2024-09-16T20:51:27.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.787 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.787 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:27.787 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.831538+0000", 2024-09-16T20:51:27.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8523999999999999e-05 2024-09-16T20:51:27.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.787 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.787 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:27.787 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.831691+0000", 2024-09-16T20:51:27.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015335699999999999 2024-09-16T20:51:27.787 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.787 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:27.788 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.788 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.788 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12597 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 879616~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:27.788 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.768883+0000", 2024-09-16T20:51:27.788 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 96.980963844000001, 2024-09-16T20:51:27.788 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065816527, 2024-09-16T20:51:27.788 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:27.788 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:27.788 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:27.788 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:27.788 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:27.788 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:27.788 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12597 2024-09-16T20:51:27.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.789 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:27.789 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.789 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:27.789 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768883+0000", 2024-09-16T20:51:27.789 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.789 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.789 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:27.789 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768883+0000", 2024-09-16T20:51:27.789 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.789 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.789 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:27.790 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771508+0000", 2024-09-16T20:51:27.790 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0026247589999999999 2024-09-16T20:51:27.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.790 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.790 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:27.790 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771513+0000", 2024-09-16T20:51:27.790 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9949999999999996e-06 2024-09-16T20:51:27.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.790 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.790 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:27.790 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771525+0000", 2024-09-16T20:51:27.790 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2033e-05 2024-09-16T20:51:27.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.790 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.790 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:27.791 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771532+0000", 2024-09-16T20:51:27.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4220000000000003e-06 2024-09-16T20:51:27.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.791 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.791 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:27.791 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.834532+0000", 2024-09-16T20:51:27.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.063000159 2024-09-16T20:51:27.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.791 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.791 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:27.791 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.834590+0000", 2024-09-16T20:51:27.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8218999999999999e-05 2024-09-16T20:51:27.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.791 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.792 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:27.792 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.834699+0000", 2024-09-16T20:51:27.792 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010894 2024-09-16T20:51:27.792 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.792 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:27.792 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.792 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.792 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.792 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12619 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 992256~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:27.792 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.774656+0000", 2024-09-16T20:51:27.792 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 96.975191069999994, 2024-09-16T20:51:27.792 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065715859000000001, 2024-09-16T20:51:27.792 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:27.792 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:27.792 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:27.792 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:27.793 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:27.793 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:27.793 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12619 2024-09-16T20:51:27.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.793 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:27.793 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.793 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:27.793 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774656+0000", 2024-09-16T20:51:27.793 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.793 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.793 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:27.793 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774656+0000", 2024-09-16T20:51:27.793 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.794 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:27.794 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774875+0000", 2024-09-16T20:51:27.794 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000218898 2024-09-16T20:51:27.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.794 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:27.794 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774879+0000", 2024-09-16T20:51:27.794 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0389999999999998e-06 2024-09-16T20:51:27.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.794 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:27.794 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774890+0000", 2024-09-16T20:51:27.794 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1250000000000001e-05 2024-09-16T20:51:27.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.795 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.795 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:27.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774897+0000", 2024-09-16T20:51:27.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7619999999999998e-06 2024-09-16T20:51:27.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.795 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.795 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:27.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840139+0000", 2024-09-16T20:51:27.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065242438 2024-09-16T20:51:27.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.795 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.795 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:27.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840186+0000", 2024-09-16T20:51:27.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6894999999999997e-05 2024-09-16T20:51:27.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.796 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.796 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:27.796 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840372+0000", 2024-09-16T20:51:27.796 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000185577 2024-09-16T20:51:27.796 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.796 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:27.796 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.796 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.796 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12620 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 997376~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:27.796 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.774908+0000", 2024-09-16T20:51:27.796 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 96.974938848999997, 2024-09-16T20:51:27.796 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065782754999999998, 2024-09-16T20:51:27.796 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:27.797 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:27.797 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:27.797 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:27.797 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:27.797 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:27.797 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12620 2024-09-16T20:51:27.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.797 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:27.797 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.797 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:27.797 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774908+0000", 2024-09-16T20:51:27.797 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.797 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.798 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:27.798 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774908+0000", 2024-09-16T20:51:27.798 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.798 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.798 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:27.798 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775102+0000", 2024-09-16T20:51:27.798 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000194098 2024-09-16T20:51:27.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.798 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.798 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:27.798 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775105+0000", 2024-09-16T20:51:27.798 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8739999999999999e-06 2024-09-16T20:51:27.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.799 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.799 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:27.799 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775111+0000", 2024-09-16T20:51:27.799 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9519999999999999e-06 2024-09-16T20:51:27.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.799 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.799 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:27.799 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775115+0000", 2024-09-16T20:51:27.799 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.002e-06 2024-09-16T20:51:27.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.799 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.799 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:27.800 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840409+0000", 2024-09-16T20:51:27.800 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065294195999999999 2024-09-16T20:51:27.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.800 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:27.800 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840507+0000", 2024-09-16T20:51:27.800 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7662000000000001e-05 2024-09-16T20:51:27.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.800 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:27.800 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840691+0000", 2024-09-16T20:51:27.800 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000183971 2024-09-16T20:51:27.800 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.801 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:27.801 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.801 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.801 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.801 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12621 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1002496~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:27.801 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775123+0000", 2024-09-16T20:51:27.801 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 96.974724152999997, 2024-09-16T20:51:27.801 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065829586999999995, 2024-09-16T20:51:27.801 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:27.801 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:27.801 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:27.801 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:27.801 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:27.802 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:27.802 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12621 2024-09-16T20:51:27.802 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.802 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:27.802 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.802 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:27.802 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775123+0000", 2024-09-16T20:51:27.802 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.802 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.802 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.802 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:27.802 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775123+0000", 2024-09-16T20:51:27.803 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.803 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.803 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:27.803 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775348+0000", 2024-09-16T20:51:27.803 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000225295 2024-09-16T20:51:27.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.803 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.803 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:27.803 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775352+0000", 2024-09-16T20:51:27.803 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0810000000000004e-06 2024-09-16T20:51:27.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.803 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.803 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:27.803 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775361+0000", 2024-09-16T20:51:27.804 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7399999999999993e-06 2024-09-16T20:51:27.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.804 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:27.804 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775367+0000", 2024-09-16T20:51:27.804 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5200000000000003e-06 2024-09-16T20:51:27.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.804 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:27.804 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840730+0000", 2024-09-16T20:51:27.804 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065362803999999997 2024-09-16T20:51:27.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.805 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:27.805 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840801+0000", 2024-09-16T20:51:27.805 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1006000000000004e-05 2024-09-16T20:51:27.805 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.805 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.805 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:27.805 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840952+0000", 2024-09-16T20:51:27.805 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015114100000000001 2024-09-16T20:51:27.805 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.805 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:27.805 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.805 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.805 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.806 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12622 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1007616~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:27.806 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775381+0000", 2024-09-16T20:51:27.806 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 96.974465319000004, 2024-09-16T20:51:27.806 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065822581000000005, 2024-09-16T20:51:27.806 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:27.806 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:27.806 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:27.806 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:27.806 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:27.806 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:27.806 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12622 2024-09-16T20:51:27.806 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.806 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:27.807 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.807 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:27.807 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775381+0000", 2024-09-16T20:51:27.807 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.807 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.807 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.807 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:27.807 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775381+0000", 2024-09-16T20:51:27.807 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.807 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.807 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.807 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:27.808 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775563+0000", 2024-09-16T20:51:27.808 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018153700000000001 2024-09-16T20:51:27.808 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.808 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.808 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:27.808 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775566+0000", 2024-09-16T20:51:27.808 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9620000000000002e-06 2024-09-16T20:51:27.808 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.808 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.808 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:27.808 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775572+0000", 2024-09-16T20:51:27.808 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8529999999999997e-06 2024-09-16T20:51:27.808 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.808 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.809 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:27.809 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775576+0000", 2024-09-16T20:51:27.809 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8920000000000002e-06 2024-09-16T20:51:27.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.809 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.809 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:27.809 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840990+0000", 2024-09-16T20:51:27.809 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065414599000000004 2024-09-16T20:51:27.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.809 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.809 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:27.809 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841068+0000", 2024-09-16T20:51:27.809 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8157000000000004e-05 2024-09-16T20:51:27.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.810 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.810 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:27.810 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841204+0000", 2024-09-16T20:51:27.810 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013558099999999999 2024-09-16T20:51:27.810 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.810 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:27.810 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.810 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.810 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12623 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1012736~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:27.810 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775583+0000", 2024-09-16T20:51:27.811 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 96.974263880999999, 2024-09-16T20:51:27.811 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065929087999999997, 2024-09-16T20:51:27.811 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:27.811 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:27.811 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:27.811 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:27.811 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:27.811 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:27.811 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12623 2024-09-16T20:51:27.811 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.811 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:27.811 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.811 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:27.812 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775583+0000", 2024-09-16T20:51:27.812 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.812 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.812 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.812 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:27.812 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775583+0000", 2024-09-16T20:51:27.812 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.812 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.812 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.812 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:27.812 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775804+0000", 2024-09-16T20:51:27.812 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00022116900000000001 2024-09-16T20:51:27.812 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.813 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.813 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:27.813 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775807+0000", 2024-09-16T20:51:27.813 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9529999999999999e-06 2024-09-16T20:51:27.813 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.813 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.813 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:27.813 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775814+0000", 2024-09-16T20:51:27.813 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7490000000000001e-06 2024-09-16T20:51:27.813 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.813 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.813 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:27.813 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775818+0000", 2024-09-16T20:51:27.813 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1999999999999996e-06 2024-09-16T20:51:27.814 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.814 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.814 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:27.814 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841267+0000", 2024-09-16T20:51:27.814 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065449127999999995 2024-09-16T20:51:27.814 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.814 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.814 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:27.814 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841344+0000", 2024-09-16T20:51:27.814 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6546999999999997e-05 2024-09-16T20:51:27.814 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.814 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.814 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:27.814 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841512+0000", 2024-09-16T20:51:27.815 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016834200000000001 2024-09-16T20:51:27.815 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.815 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:27.815 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.815 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.815 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.815 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12624 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1017856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:27.815 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:51:27.815 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 96.974020741000004, 2024-09-16T20:51:27.815 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065945866000000006, 2024-09-16T20:51:27.815 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:27.815 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:27.815 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:27.816 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:27.816 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:27.816 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:27.816 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12624 2024-09-16T20:51:27.816 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.816 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:27.816 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.816 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:27.816 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:51:27.816 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.816 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.816 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.816 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:27.817 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:51:27.817 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.817 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.817 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.817 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:27.817 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:51:27.817 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-16T20:51:27.817 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.817 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.817 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:27.817 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775828+0000", 2024-09-16T20:51:27.817 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3999999999999999e-06 2024-09-16T20:51:27.817 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.817 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.818 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:27.818 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775831+0000", 2024-09-16T20:51:27.818 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.478e-06 2024-09-16T20:51:27.818 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.818 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.818 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:27.818 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775834+0000", 2024-09-16T20:51:27.818 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9440000000000001e-06 2024-09-16T20:51:27.818 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.818 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.818 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:27.818 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841547+0000", 2024-09-16T20:51:27.818 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065712801000000001 2024-09-16T20:51:27.819 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.819 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.819 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:27.819 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841618+0000", 2024-09-16T20:51:27.819 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1151999999999999e-05 2024-09-16T20:51:27.819 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.819 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.819 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:27.819 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841772+0000", 2024-09-16T20:51:27.819 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000153721 2024-09-16T20:51:27.819 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.819 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:27.819 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.819 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.819 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.820 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12625 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1022976~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:27.820 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775840+0000", 2024-09-16T20:51:27.820 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 96.974007131999997, 2024-09-16T20:51:27.820 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066165451, 2024-09-16T20:51:27.820 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:27.820 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:27.820 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:27.820 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:27.820 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:27.820 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:27.820 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12625 2024-09-16T20:51:27.820 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.820 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:27.820 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.820 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:27.821 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775840+0000", 2024-09-16T20:51:27.821 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.821 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.821 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.821 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:27.821 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775840+0000", 2024-09-16T20:51:27.821 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.821 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.821 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.821 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:27.821 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776014+0000", 2024-09-16T20:51:27.821 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017441099999999999 2024-09-16T20:51:27.821 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.821 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.822 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:27.822 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776018+0000", 2024-09-16T20:51:27.822 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6509999999999999e-06 2024-09-16T20:51:27.822 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.822 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.822 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:27.822 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776026+0000", 2024-09-16T20:51:27.822 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.028e-06 2024-09-16T20:51:27.822 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.822 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.822 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:27.822 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776031+0000", 2024-09-16T20:51:27.822 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8400000000000002e-06 2024-09-16T20:51:27.822 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.822 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.823 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:27.823 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841801+0000", 2024-09-16T20:51:27.823 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065770472999999996 2024-09-16T20:51:27.823 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.823 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.823 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:27.823 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841847+0000", 2024-09-16T20:51:27.823 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5558e-05 2024-09-16T20:51:27.823 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.823 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.823 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:27.823 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842005+0000", 2024-09-16T20:51:27.823 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015849000000000001 2024-09-16T20:51:27.823 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.823 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:27.824 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.824 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.824 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.824 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12626 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1028096~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:27.824 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776041+0000", 2024-09-16T20:51:27.824 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 96.973805833, 2024-09-16T20:51:27.824 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066225260999999994, 2024-09-16T20:51:27.824 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:27.824 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:27.824 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:27.824 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:27.824 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:27.824 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:27.825 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12626 2024-09-16T20:51:27.825 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.825 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:27.825 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.825 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:27.825 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776041+0000", 2024-09-16T20:51:27.825 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.825 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.825 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.825 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:27.825 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776041+0000", 2024-09-16T20:51:27.825 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.825 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.826 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.826 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:27.826 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776263+0000", 2024-09-16T20:51:27.826 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00022193899999999999 2024-09-16T20:51:27.826 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.826 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.826 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:27.826 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776268+0000", 2024-09-16T20:51:27.826 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8049999999999997e-06 2024-09-16T20:51:27.826 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.826 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.826 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:27.826 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776280+0000", 2024-09-16T20:51:27.826 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2635e-05 2024-09-16T20:51:27.827 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.827 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.827 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:27.827 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776288+0000", 2024-09-16T20:51:27.827 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8199999999999997e-06 2024-09-16T20:51:27.827 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.827 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.827 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:27.827 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842041+0000", 2024-09-16T20:51:27.827 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065752506000000002 2024-09-16T20:51:27.827 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.827 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.827 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:27.827 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842123+0000", 2024-09-16T20:51:27.827 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2393999999999998e-05 2024-09-16T20:51:27.827 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.828 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.828 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:27.828 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842266+0000", 2024-09-16T20:51:27.828 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014316200000000001 2024-09-16T20:51:27.828 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.828 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:27.828 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.828 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.828 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.828 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12627 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1033216~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:27.828 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776301+0000", 2024-09-16T20:51:27.828 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 96.973545873000006, 2024-09-16T20:51:27.828 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066187814999999997, 2024-09-16T20:51:27.829 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:27.829 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:27.829 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:27.829 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:27.829 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:27.829 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:27.829 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12627 2024-09-16T20:51:27.829 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.829 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:27.829 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.829 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:27.829 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776301+0000", 2024-09-16T20:51:27.829 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.829 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.830 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.830 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:27.830 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776301+0000", 2024-09-16T20:51:27.830 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.830 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.830 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.830 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:27.830 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776473+0000", 2024-09-16T20:51:27.830 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017222400000000001 2024-09-16T20:51:27.830 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.830 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.830 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:27.830 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776477+0000", 2024-09-16T20:51:27.830 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7019999999999999e-06 2024-09-16T20:51:27.830 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.831 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.831 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:27.831 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776486+0000", 2024-09-16T20:51:27.831 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2119999999999992e-06 2024-09-16T20:51:27.831 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.831 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.831 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:27.831 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776492+0000", 2024-09-16T20:51:27.831 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8089999999999998e-06 2024-09-16T20:51:27.831 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.831 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.831 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:27.831 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842301+0000", 2024-09-16T20:51:27.831 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065809042999999998 2024-09-16T20:51:27.832 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.832 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.832 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:27.832 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842362+0000", 2024-09-16T20:51:27.832 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1260000000000006e-05 2024-09-16T20:51:27.832 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.832 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.832 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:27.832 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842489+0000", 2024-09-16T20:51:27.832 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012656500000000001 2024-09-16T20:51:27.832 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.832 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:27.832 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.832 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.833 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.833 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12628 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1038336~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:27.833 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776502+0000", 2024-09-16T20:51:27.833 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 96.973345226999996, 2024-09-16T20:51:27.833 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066283426000000006, 2024-09-16T20:51:27.833 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:27.833 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:27.833 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:27.833 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:27.833 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:27.833 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:27.833 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12628 2024-09-16T20:51:27.833 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.833 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:27.834 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.834 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:27.834 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776502+0000", 2024-09-16T20:51:27.834 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.834 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.834 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.834 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:27.834 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776502+0000", 2024-09-16T20:51:27.834 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.834 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.834 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.834 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:27.834 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776716+0000", 2024-09-16T20:51:27.834 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000214596 2024-09-16T20:51:27.834 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.834 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.835 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:27.835 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776723+0000", 2024-09-16T20:51:27.835 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4000000000000003e-06 2024-09-16T20:51:27.835 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.835 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.835 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:27.835 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776731+0000", 2024-09-16T20:51:27.835 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6129999999999998e-06 2024-09-16T20:51:27.835 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.835 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.835 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:27.835 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776737+0000", 2024-09-16T20:51:27.835 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1970000000000001e-06 2024-09-16T20:51:27.835 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.835 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.835 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:27.836 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842532+0000", 2024-09-16T20:51:27.836 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065795127999999994 2024-09-16T20:51:27.836 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.836 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.836 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:27.836 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842599+0000", 2024-09-16T20:51:27.836 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6711000000000006e-05 2024-09-16T20:51:27.836 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.836 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.836 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:27.836 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842785+0000", 2024-09-16T20:51:27.836 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018578099999999999 2024-09-16T20:51:27.836 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.836 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:27.836 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.837 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.837 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.837 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12629 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1043456~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:27.837 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776748+0000", 2024-09-16T20:51:27.837 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 96.97309885, 2024-09-16T20:51:27.837 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066303788000000002, 2024-09-16T20:51:27.837 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:27.837 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:27.837 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:27.837 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:27.837 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:27.837 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:27.837 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12629 2024-09-16T20:51:27.837 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.837 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:27.838 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.838 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:27.838 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776748+0000", 2024-09-16T20:51:27.838 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.838 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.838 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.838 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:27.838 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776748+0000", 2024-09-16T20:51:27.838 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:27.838 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.838 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.838 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:27.838 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776934+0000", 2024-09-16T20:51:27.839 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018598399999999999 2024-09-16T20:51:27.839 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.839 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.839 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:27.839 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776939+0000", 2024-09-16T20:51:27.839 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1000000000000003e-06 2024-09-16T20:51:27.839 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.839 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.839 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:27.839 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776954+0000", 2024-09-16T20:51:27.839 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5221e-05 2024-09-16T20:51:27.839 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.839 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.839 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:27.839 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776962+0000", 2024-09-16T20:51:27.839 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.554e-06 2024-09-16T20:51:27.840 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.840 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.840 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:27.840 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842813+0000", 2024-09-16T20:51:27.840 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065851006000000004 2024-09-16T20:51:27.840 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.840 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.840 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:27.840 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842886+0000", 2024-09-16T20:51:27.840 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3663000000000003e-05 2024-09-16T20:51:27.840 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:27.840 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:27.840 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:27.840 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.843052+0000", 2024-09-16T20:51:27.840 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016526 2024-09-16T20:51:27.840 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.841 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:27.841 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.841 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:27.841 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:27.841 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:27.842 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:27.841+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:27.843 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:51:27.943 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:51:27.941+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:51:27.950 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:27.950 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:51:27.950 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:51:27.950 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:27.959 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:51:27.962 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:51:27.962 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:27.962 INFO:teuthology.orchestra.run.smithi029.stdout: "epoch": 200, 2024-09-16T20:51:27.962 INFO:teuthology.orchestra.run.smithi029.stdout: "fsid": "e28af4a7-5fcd-4de8-8f77-a2b575361bb1", 2024-09-16T20:51:27.962 INFO:teuthology.orchestra.run.smithi029.stdout: "created": "2024-09-16T20:47:39.760609+0000", 2024-09-16T20:51:27.962 INFO:teuthology.orchestra.run.smithi029.stdout: "modified": "2024-09-16T20:51:27.618971+0000", 2024-09-16T20:51:27.962 INFO:teuthology.orchestra.run.smithi029.stdout: "last_up_change": "2024-09-16T20:47:50.238013+0000", 2024-09-16T20:51:27.962 INFO:teuthology.orchestra.run.smithi029.stdout: "last_in_change": "2024-09-16T20:51:22.595086+0000", 2024-09-16T20:51:27.962 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-16T20:51:27.962 INFO:teuthology.orchestra.run.smithi029.stdout: "flags_num": 5804032, 2024-09-16T20:51:27.962 INFO:teuthology.orchestra.run.smithi029.stdout: "flags_set": [ 2024-09-16T20:51:27.962 INFO:teuthology.orchestra.run.smithi029.stdout: "nodeep-scrub", 2024-09-16T20:51:27.962 INFO:teuthology.orchestra.run.smithi029.stdout: "pglog_hardlimit", 2024-09-16T20:51:27.963 INFO:teuthology.orchestra.run.smithi029.stdout: "purged_snapdirs", 2024-09-16T20:51:27.963 INFO:teuthology.orchestra.run.smithi029.stdout: "recovery_deletes", 2024-09-16T20:51:27.963 INFO:teuthology.orchestra.run.smithi029.stdout: "sortbitwise" 2024-09-16T20:51:27.963 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-16T20:51:27.963 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_version": 5, 2024-09-16T20:51:27.963 INFO:teuthology.orchestra.run.smithi029.stdout: "full_ratio": 0.94999998807907104, 2024-09-16T20:51:27.963 INFO:teuthology.orchestra.run.smithi029.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-16T20:51:27.963 INFO:teuthology.orchestra.run.smithi029.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-16T20:51:27.963 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_snapshot": "", 2024-09-16T20:51:27.963 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_max": 3, 2024-09-16T20:51:27.963 INFO:teuthology.orchestra.run.smithi029.stdout: "max_osd": 6, 2024-09-16T20:51:27.963 INFO:teuthology.orchestra.run.smithi029.stdout: "require_min_compat_client": "luminous", 2024-09-16T20:51:27.963 INFO:teuthology.orchestra.run.smithi029.stdout: "min_compat_client": "luminous", 2024-09-16T20:51:27.964 INFO:teuthology.orchestra.run.smithi029.stdout: "require_osd_release": "squid", 2024-09-16T20:51:27.964 INFO:teuthology.orchestra.run.smithi029.stdout: "allow_crimson": false, 2024-09-16T20:51:27.964 INFO:teuthology.orchestra.run.smithi029.stdout: "pools": [ 2024-09-16T20:51:27.964 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:27.964 INFO:teuthology.orchestra.run.smithi029.stdout: "pool": 1, 2024-09-16T20:51:27.964 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_name": ".mgr", 2024-09-16T20:51:27.964 INFO:teuthology.orchestra.run.smithi029.stdout: "create_time": "2024-09-16T20:47:52.220750+0000", 2024-09-16T20:51:27.964 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-16T20:51:27.964 INFO:teuthology.orchestra.run.smithi029.stdout: "flags_names": "hashpspool", 2024-09-16T20:51:27.964 INFO:teuthology.orchestra.run.smithi029.stdout: "type": 1, 2024-09-16T20:51:27.964 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 2, 2024-09-16T20:51:27.964 INFO:teuthology.orchestra.run.smithi029.stdout: "min_size": 1, 2024-09-16T20:51:27.965 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_rule": 0, 2024-09-16T20:51:27.965 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_count": 0, 2024-09-16T20:51:27.965 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_target": 0, 2024-09-16T20:51:27.965 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T20:51:27.965 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T20:51:27.965 INFO:teuthology.orchestra.run.smithi029.stdout: "is_stretch_pool": false, 2024-09-16T20:51:27.965 INFO:teuthology.orchestra.run.smithi029.stdout: "object_hash": 2, 2024-09-16T20:51:27.965 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_autoscale_mode": "off", 2024-09-16T20:51:27.965 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num": 1, 2024-09-16T20:51:27.965 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num": 1, 2024-09-16T20:51:27.965 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num_target": 1, 2024-09-16T20:51:27.965 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_target": 1, 2024-09-16T20:51:27.965 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_pending": 1, 2024-09-16T20:51:27.965 INFO:teuthology.orchestra.run.smithi029.stdout: "last_pg_merge_meta": { 2024-09-16T20:51:27.966 INFO:teuthology.orchestra.run.smithi029.stdout: "source_pgid": "0.0", 2024-09-16T20:51:27.966 INFO:teuthology.orchestra.run.smithi029.stdout: "ready_epoch": 0, 2024-09-16T20:51:27.966 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_started": 0, 2024-09-16T20:51:27.966 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_clean": 0, 2024-09-16T20:51:27.966 INFO:teuthology.orchestra.run.smithi029.stdout: "source_version": "0'0", 2024-09-16T20:51:27.966 INFO:teuthology.orchestra.run.smithi029.stdout: "target_version": "0'0" 2024-09-16T20:51:27.966 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:27.966 INFO:teuthology.orchestra.run.smithi029.stdout: "last_change": "17", 2024-09-16T20:51:27.966 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend": "0", 2024-09-16T20:51:27.966 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T20:51:27.966 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T20:51:27.966 INFO:teuthology.orchestra.run.smithi029.stdout: "auid": 0, 2024-09-16T20:51:27.966 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_mode": "selfmanaged", 2024-09-16T20:51:27.966 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_seq": 0, 2024-09-16T20:51:27.967 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_epoch": 0, 2024-09-16T20:51:27.967 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_snaps": [], 2024-09-16T20:51:27.967 INFO:teuthology.orchestra.run.smithi029.stdout: "removed_snaps": "[]", 2024-09-16T20:51:27.967 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_bytes": 0, 2024-09-16T20:51:27.967 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_objects": 0, 2024-09-16T20:51:27.967 INFO:teuthology.orchestra.run.smithi029.stdout: "tiers": [], 2024-09-16T20:51:27.967 INFO:teuthology.orchestra.run.smithi029.stdout: "tier_of": -1, 2024-09-16T20:51:27.967 INFO:teuthology.orchestra.run.smithi029.stdout: "read_tier": -1, 2024-09-16T20:51:27.967 INFO:teuthology.orchestra.run.smithi029.stdout: "write_tier": -1, 2024-09-16T20:51:27.967 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_mode": "none", 2024-09-16T20:51:27.967 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_bytes": 0, 2024-09-16T20:51:27.967 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_objects": 0, 2024-09-16T20:51:27.967 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T20:51:27.967 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T20:51:27.968 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T20:51:27.968 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_flush_age": 0, 2024-09-16T20:51:27.968 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_evict_age": 0, 2024-09-16T20:51:27.968 INFO:teuthology.orchestra.run.smithi029.stdout: "erasure_code_profile": "", 2024-09-16T20:51:27.968 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_params": { 2024-09-16T20:51:27.968 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "none" 2024-09-16T20:51:27.968 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:27.968 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_period": 0, 2024-09-16T20:51:27.968 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_count": 0, 2024-09-16T20:51:27.968 INFO:teuthology.orchestra.run.smithi029.stdout: "use_gmt_hitset": true, 2024-09-16T20:51:27.968 INFO:teuthology.orchestra.run.smithi029.stdout: "min_read_recency_for_promote": 0, 2024-09-16T20:51:27.968 INFO:teuthology.orchestra.run.smithi029.stdout: "min_write_recency_for_promote": 0, 2024-09-16T20:51:27.968 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T20:51:27.969 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_search_last_n": 0, 2024-09-16T20:51:27.969 INFO:teuthology.orchestra.run.smithi029.stdout: "grade_table": [], 2024-09-16T20:51:27.969 INFO:teuthology.orchestra.run.smithi029.stdout: "stripe_width": 0, 2024-09-16T20:51:27.969 INFO:teuthology.orchestra.run.smithi029.stdout: "expected_num_objects": 0, 2024-09-16T20:51:27.969 INFO:teuthology.orchestra.run.smithi029.stdout: "fast_read": false, 2024-09-16T20:51:27.969 INFO:teuthology.orchestra.run.smithi029.stdout: "options": { 2024-09-16T20:51:27.969 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_max": 32, 2024-09-16T20:51:27.969 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_min": 1 2024-09-16T20:51:27.969 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:27.969 INFO:teuthology.orchestra.run.smithi029.stdout: "application_metadata": { 2024-09-16T20:51:27.969 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": {} 2024-09-16T20:51:27.969 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:27.969 INFO:teuthology.orchestra.run.smithi029.stdout: "read_balance": { 2024-09-16T20:51:27.970 INFO:teuthology.orchestra.run.smithi029.stdout: "score_type": "Fair distribution", 2024-09-16T20:51:27.970 INFO:teuthology.orchestra.run.smithi029.stdout: "score_acting": 6.059999942779541, 2024-09-16T20:51:27.970 INFO:teuthology.orchestra.run.smithi029.stdout: "score_stable": 6.059999942779541, 2024-09-16T20:51:27.970 INFO:teuthology.orchestra.run.smithi029.stdout: "optimal_score": 0.33000001311302185, 2024-09-16T20:51:27.970 INFO:teuthology.orchestra.run.smithi029.stdout: "raw_score_acting": 2, 2024-09-16T20:51:27.970 INFO:teuthology.orchestra.run.smithi029.stdout: "raw_score_stable": 2, 2024-09-16T20:51:27.970 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity_weighted": 1, 2024-09-16T20:51:27.970 INFO:teuthology.orchestra.run.smithi029.stdout: "average_primary_affinity": 1, 2024-09-16T20:51:27.970 INFO:teuthology.orchestra.run.smithi029.stdout: "average_primary_affinity_weighted": 1 2024-09-16T20:51:27.970 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:27.970 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:27.970 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:27.970 INFO:teuthology.orchestra.run.smithi029.stdout: "pool": 2, 2024-09-16T20:51:27.971 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_name": "rbd", 2024-09-16T20:51:27.971 INFO:teuthology.orchestra.run.smithi029.stdout: "create_time": "2024-09-16T20:47:56.756666+0000", 2024-09-16T20:51:27.971 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 8193, 2024-09-16T20:51:27.971 INFO:teuthology.orchestra.run.smithi029.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T20:51:27.971 INFO:teuthology.orchestra.run.smithi029.stdout: "type": 1, 2024-09-16T20:51:27.971 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 2, 2024-09-16T20:51:27.971 INFO:teuthology.orchestra.run.smithi029.stdout: "min_size": 1, 2024-09-16T20:51:27.971 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_rule": 0, 2024-09-16T20:51:27.971 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_count": 0, 2024-09-16T20:51:27.971 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_target": 0, 2024-09-16T20:51:27.971 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T20:51:27.971 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T20:51:27.971 INFO:teuthology.orchestra.run.smithi029.stdout: "is_stretch_pool": false, 2024-09-16T20:51:27.972 INFO:teuthology.orchestra.run.smithi029.stdout: "object_hash": 2, 2024-09-16T20:51:27.972 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_autoscale_mode": "off", 2024-09-16T20:51:27.972 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num": 8, 2024-09-16T20:51:27.972 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num": 8, 2024-09-16T20:51:27.972 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num_target": 8, 2024-09-16T20:51:27.972 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_target": 8, 2024-09-16T20:51:27.972 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_pending": 8, 2024-09-16T20:51:27.972 INFO:teuthology.orchestra.run.smithi029.stdout: "last_pg_merge_meta": { 2024-09-16T20:51:27.972 INFO:teuthology.orchestra.run.smithi029.stdout: "source_pgid": "0.0", 2024-09-16T20:51:27.972 INFO:teuthology.orchestra.run.smithi029.stdout: "ready_epoch": 0, 2024-09-16T20:51:27.972 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_started": 0, 2024-09-16T20:51:27.973 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_clean": 0, 2024-09-16T20:51:27.973 INFO:teuthology.orchestra.run.smithi029.stdout: "source_version": "0'0", 2024-09-16T20:51:27.973 INFO:teuthology.orchestra.run.smithi029.stdout: "target_version": "0'0" 2024-09-16T20:51:27.973 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:27.973 INFO:teuthology.orchestra.run.smithi029.stdout: "last_change": "21", 2024-09-16T20:51:27.973 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend": "0", 2024-09-16T20:51:27.973 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T20:51:27.973 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T20:51:27.973 INFO:teuthology.orchestra.run.smithi029.stdout: "auid": 0, 2024-09-16T20:51:27.973 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_mode": "selfmanaged", 2024-09-16T20:51:27.973 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_seq": 2, 2024-09-16T20:51:27.973 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_epoch": 21, 2024-09-16T20:51:27.973 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_snaps": [], 2024-09-16T20:51:27.973 INFO:teuthology.orchestra.run.smithi029.stdout: "removed_snaps": "[]", 2024-09-16T20:51:27.974 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_bytes": 0, 2024-09-16T20:51:27.974 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_objects": 0, 2024-09-16T20:51:27.974 INFO:teuthology.orchestra.run.smithi029.stdout: "tiers": [], 2024-09-16T20:51:27.974 INFO:teuthology.orchestra.run.smithi029.stdout: "tier_of": -1, 2024-09-16T20:51:27.974 INFO:teuthology.orchestra.run.smithi029.stdout: "read_tier": -1, 2024-09-16T20:51:27.974 INFO:teuthology.orchestra.run.smithi029.stdout: "write_tier": -1, 2024-09-16T20:51:27.974 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_mode": "none", 2024-09-16T20:51:27.974 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_bytes": 0, 2024-09-16T20:51:27.974 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_objects": 0, 2024-09-16T20:51:27.974 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T20:51:27.974 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T20:51:27.974 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T20:51:27.974 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_flush_age": 0, 2024-09-16T20:51:27.974 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_evict_age": 0, 2024-09-16T20:51:27.975 INFO:teuthology.orchestra.run.smithi029.stdout: "erasure_code_profile": "", 2024-09-16T20:51:27.975 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_params": { 2024-09-16T20:51:27.975 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "none" 2024-09-16T20:51:27.975 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:27.975 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_period": 0, 2024-09-16T20:51:27.975 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_count": 0, 2024-09-16T20:51:27.975 INFO:teuthology.orchestra.run.smithi029.stdout: "use_gmt_hitset": true, 2024-09-16T20:51:27.975 INFO:teuthology.orchestra.run.smithi029.stdout: "min_read_recency_for_promote": 0, 2024-09-16T20:51:27.975 INFO:teuthology.orchestra.run.smithi029.stdout: "min_write_recency_for_promote": 0, 2024-09-16T20:51:27.975 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T20:51:27.975 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_search_last_n": 0, 2024-09-16T20:51:27.975 INFO:teuthology.orchestra.run.smithi029.stdout: "grade_table": [], 2024-09-16T20:51:27.975 INFO:teuthology.orchestra.run.smithi029.stdout: "stripe_width": 0, 2024-09-16T20:51:27.976 INFO:teuthology.orchestra.run.smithi029.stdout: "expected_num_objects": 0, 2024-09-16T20:51:27.976 INFO:teuthology.orchestra.run.smithi029.stdout: "fast_read": false, 2024-09-16T20:51:27.976 INFO:teuthology.orchestra.run.smithi029.stdout: "options": {}, 2024-09-16T20:51:27.976 INFO:teuthology.orchestra.run.smithi029.stdout: "application_metadata": { 2024-09-16T20:51:27.976 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd": {} 2024-09-16T20:51:27.976 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:27.976 INFO:teuthology.orchestra.run.smithi029.stdout: "read_balance": { 2024-09-16T20:51:27.976 INFO:teuthology.orchestra.run.smithi029.stdout: "score_type": "Fair distribution", 2024-09-16T20:51:27.976 INFO:teuthology.orchestra.run.smithi029.stdout: "score_acting": 2.25, 2024-09-16T20:51:27.976 INFO:teuthology.orchestra.run.smithi029.stdout: "score_stable": 2.25, 2024-09-16T20:51:27.976 INFO:teuthology.orchestra.run.smithi029.stdout: "optimal_score": 1, 2024-09-16T20:51:27.976 INFO:teuthology.orchestra.run.smithi029.stdout: "raw_score_acting": 2.25, 2024-09-16T20:51:27.976 INFO:teuthology.orchestra.run.smithi029.stdout: "raw_score_stable": 2.25, 2024-09-16T20:51:27.976 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity_weighted": 1, 2024-09-16T20:51:27.977 INFO:teuthology.orchestra.run.smithi029.stdout: "average_primary_affinity": 1, 2024-09-16T20:51:27.977 INFO:teuthology.orchestra.run.smithi029.stdout: "average_primary_affinity_weighted": 1 2024-09-16T20:51:27.977 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:27.977 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:27.977 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:27.977 INFO:teuthology.orchestra.run.smithi029.stdout: "pool": 3, 2024-09-16T20:51:27.977 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_name": "pool_client.0", 2024-09-16T20:51:27.977 INFO:teuthology.orchestra.run.smithi029.stdout: "create_time": "2024-09-16T20:48:06.418889+0000", 2024-09-16T20:51:27.977 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 8193, 2024-09-16T20:51:27.977 INFO:teuthology.orchestra.run.smithi029.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T20:51:27.977 INFO:teuthology.orchestra.run.smithi029.stdout: "type": 1, 2024-09-16T20:51:27.977 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 2, 2024-09-16T20:51:27.977 INFO:teuthology.orchestra.run.smithi029.stdout: "min_size": 1, 2024-09-16T20:51:27.977 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_rule": 0, 2024-09-16T20:51:27.978 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_count": 0, 2024-09-16T20:51:27.978 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_target": 0, 2024-09-16T20:51:27.978 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T20:51:27.978 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T20:51:27.978 INFO:teuthology.orchestra.run.smithi029.stdout: "is_stretch_pool": false, 2024-09-16T20:51:27.978 INFO:teuthology.orchestra.run.smithi029.stdout: "object_hash": 2, 2024-09-16T20:51:27.978 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_autoscale_mode": "off", 2024-09-16T20:51:27.978 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num": 32, 2024-09-16T20:51:27.978 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num": 32, 2024-09-16T20:51:27.978 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num_target": 32, 2024-09-16T20:51:27.978 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_target": 32, 2024-09-16T20:51:27.978 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_pending": 32, 2024-09-16T20:51:27.978 INFO:teuthology.orchestra.run.smithi029.stdout: "last_pg_merge_meta": { 2024-09-16T20:51:27.979 INFO:teuthology.orchestra.run.smithi029.stdout: "source_pgid": "0.0", 2024-09-16T20:51:27.979 INFO:teuthology.orchestra.run.smithi029.stdout: "ready_epoch": 0, 2024-09-16T20:51:27.979 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_started": 0, 2024-09-16T20:51:27.979 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_clean": 0, 2024-09-16T20:51:27.979 INFO:teuthology.orchestra.run.smithi029.stdout: "source_version": "0'0", 2024-09-16T20:51:27.979 INFO:teuthology.orchestra.run.smithi029.stdout: "target_version": "0'0" 2024-09-16T20:51:27.979 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:27.979 INFO:teuthology.orchestra.run.smithi029.stdout: "last_change": "198", 2024-09-16T20:51:27.979 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend": "0", 2024-09-16T20:51:27.979 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T20:51:27.979 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T20:51:27.979 INFO:teuthology.orchestra.run.smithi029.stdout: "auid": 0, 2024-09-16T20:51:27.979 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_mode": "selfmanaged", 2024-09-16T20:51:27.979 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_seq": 48, 2024-09-16T20:51:27.980 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_epoch": 198, 2024-09-16T20:51:27.980 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_snaps": [], 2024-09-16T20:51:27.980 INFO:teuthology.orchestra.run.smithi029.stdout: "removed_snaps": "[]", 2024-09-16T20:51:27.980 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_bytes": 0, 2024-09-16T20:51:27.980 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_objects": 0, 2024-09-16T20:51:27.980 INFO:teuthology.orchestra.run.smithi029.stdout: "tiers": [], 2024-09-16T20:51:27.980 INFO:teuthology.orchestra.run.smithi029.stdout: "tier_of": -1, 2024-09-16T20:51:27.980 INFO:teuthology.orchestra.run.smithi029.stdout: "read_tier": -1, 2024-09-16T20:51:27.980 INFO:teuthology.orchestra.run.smithi029.stdout: "write_tier": -1, 2024-09-16T20:51:27.980 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_mode": "none", 2024-09-16T20:51:27.980 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_bytes": 0, 2024-09-16T20:51:27.980 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_objects": 0, 2024-09-16T20:51:27.981 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T20:51:27.981 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T20:51:27.981 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T20:51:27.981 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_flush_age": 0, 2024-09-16T20:51:27.981 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_evict_age": 0, 2024-09-16T20:51:27.981 INFO:teuthology.orchestra.run.smithi029.stdout: "erasure_code_profile": "", 2024-09-16T20:51:27.981 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_params": { 2024-09-16T20:51:27.981 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "none" 2024-09-16T20:51:27.981 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:27.981 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_period": 0, 2024-09-16T20:51:27.981 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_count": 0, 2024-09-16T20:51:27.981 INFO:teuthology.orchestra.run.smithi029.stdout: "use_gmt_hitset": true, 2024-09-16T20:51:27.981 INFO:teuthology.orchestra.run.smithi029.stdout: "min_read_recency_for_promote": 0, 2024-09-16T20:51:27.982 INFO:teuthology.orchestra.run.smithi029.stdout: "min_write_recency_for_promote": 0, 2024-09-16T20:51:27.982 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T20:51:27.982 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_search_last_n": 0, 2024-09-16T20:51:27.982 INFO:teuthology.orchestra.run.smithi029.stdout: "grade_table": [], 2024-09-16T20:51:27.982 INFO:teuthology.orchestra.run.smithi029.stdout: "stripe_width": 0, 2024-09-16T20:51:27.982 INFO:teuthology.orchestra.run.smithi029.stdout: "expected_num_objects": 0, 2024-09-16T20:51:27.982 INFO:teuthology.orchestra.run.smithi029.stdout: "fast_read": false, 2024-09-16T20:51:27.982 INFO:teuthology.orchestra.run.smithi029.stdout: "options": {}, 2024-09-16T20:51:27.982 INFO:teuthology.orchestra.run.smithi029.stdout: "application_metadata": { 2024-09-16T20:51:27.982 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd": {} 2024-09-16T20:51:27.982 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:27.982 INFO:teuthology.orchestra.run.smithi029.stdout: "read_balance": { 2024-09-16T20:51:27.982 INFO:teuthology.orchestra.run.smithi029.stdout: "score_type": "Fair distribution", 2024-09-16T20:51:27.983 INFO:teuthology.orchestra.run.smithi029.stdout: "score_acting": 1.309999942779541, 2024-09-16T20:51:27.983 INFO:teuthology.orchestra.run.smithi029.stdout: "score_stable": 1.309999942779541, 2024-09-16T20:51:27.983 INFO:teuthology.orchestra.run.smithi029.stdout: "optimal_score": 1, 2024-09-16T20:51:27.983 INFO:teuthology.orchestra.run.smithi029.stdout: "raw_score_acting": 1.309999942779541, 2024-09-16T20:51:27.983 INFO:teuthology.orchestra.run.smithi029.stdout: "raw_score_stable": 1.309999942779541, 2024-09-16T20:51:27.983 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity_weighted": 1, 2024-09-16T20:51:27.983 INFO:teuthology.orchestra.run.smithi029.stdout: "average_primary_affinity": 1, 2024-09-16T20:51:27.983 INFO:teuthology.orchestra.run.smithi029.stdout: "average_primary_affinity_weighted": 1 2024-09-16T20:51:27.983 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:27.983 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:27.983 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-16T20:51:27.983 INFO:teuthology.orchestra.run.smithi029.stdout: "osds": [ 2024-09-16T20:51:27.983 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:27.983 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 0, 2024-09-16T20:51:27.984 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "6c3cb2d7-7d23-4566-9ca1-d7b08e12360c", 2024-09-16T20:51:27.984 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-16T20:51:27.984 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-16T20:51:27.984 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-16T20:51:27.984 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-16T20:51:27.984 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-16T20:51:27.984 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-16T20:51:27.984 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 13, 2024-09-16T20:51:27.984 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 199, 2024-09-16T20:51:27.984 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-16T20:51:27.984 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-16T20:51:27.984 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-16T20:51:27.984 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:51:27.985 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:27.985 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:51:27.985 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6816", 2024-09-16T20:51:27.985 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:51:27.985 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:27.985 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:27.985 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:51:27.985 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6817", 2024-09-16T20:51:27.985 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:51:27.985 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:27.985 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:27.985 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:27.985 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-16T20:51:27.986 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:51:27.986 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:27.986 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:51:27.986 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6818", 2024-09-16T20:51:27.986 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:51:27.986 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:27.986 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:27.986 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:51:27.986 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6819", 2024-09-16T20:51:27.986 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:51:27.986 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:27.986 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:27.986 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:27.986 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-16T20:51:27.987 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:51:27.987 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:27.987 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:51:27.987 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6822", 2024-09-16T20:51:27.987 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:51:27.987 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:27.987 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:27.987 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:51:27.987 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6823", 2024-09-16T20:51:27.987 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:51:27.987 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:27.987 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:27.987 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:27.988 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-16T20:51:27.988 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:51:27.988 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:27.988 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:51:27.988 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6820", 2024-09-16T20:51:27.988 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:51:27.988 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:27.988 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:27.988 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:51:27.988 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6821", 2024-09-16T20:51:27.988 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2744025949 2024-09-16T20:51:27.988 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:27.988 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:27.989 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:27.989 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.29:6817/2744025949", 2024-09-16T20:51:27.989 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.29:6819/2744025949", 2024-09-16T20:51:27.989 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.29:6823/2744025949", 2024-09-16T20:51:27.989 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.29:6821/2744025949", 2024-09-16T20:51:27.989 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-16T20:51:27.989 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-16T20:51:27.989 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-16T20:51:27.989 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:27.989 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:27.989 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:27.989 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 1, 2024-09-16T20:51:27.989 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "5cf8c20c-1ed2-42ae-a8b0-c88cc4e36bce", 2024-09-16T20:51:27.990 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-16T20:51:27.990 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-16T20:51:27.990 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 0.8807220458984375, 2024-09-16T20:51:27.990 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-16T20:51:27.990 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-16T20:51:27.990 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-16T20:51:27.990 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 13, 2024-09-16T20:51:27.990 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 197, 2024-09-16T20:51:27.990 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-16T20:51:27.990 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-16T20:51:27.990 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-16T20:51:27.991 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:51:27.991 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:27.991 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:51:27.991 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6800", 2024-09-16T20:51:27.991 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:51:27.991 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:27.991 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:27.991 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:51:27.991 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6801", 2024-09-16T20:51:27.991 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:51:27.991 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:27.991 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:27.991 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:27.992 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-16T20:51:27.992 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:51:27.992 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:27.992 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:51:27.992 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6802", 2024-09-16T20:51:27.992 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:51:27.992 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:27.992 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:27.992 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:51:27.992 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6803", 2024-09-16T20:51:27.992 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:51:27.992 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:27.993 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:27.993 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:27.993 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-16T20:51:27.993 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:51:27.993 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:27.993 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:51:27.993 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6806", 2024-09-16T20:51:27.993 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:51:27.993 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:27.993 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:27.993 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:51:27.993 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6807", 2024-09-16T20:51:27.993 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:51:27.994 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:27.994 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:27.994 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:27.994 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-16T20:51:27.994 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:51:27.994 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:27.994 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:51:27.994 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6804", 2024-09-16T20:51:27.994 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:51:27.994 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:27.994 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:27.994 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:51:27.994 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6805", 2024-09-16T20:51:27.994 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 208628762 2024-09-16T20:51:27.995 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:27.995 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:27.995 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:27.995 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.29:6801/208628762", 2024-09-16T20:51:27.995 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.29:6803/208628762", 2024-09-16T20:51:27.995 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.29:6807/208628762", 2024-09-16T20:51:27.995 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.29:6805/208628762", 2024-09-16T20:51:27.995 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-16T20:51:27.995 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-16T20:51:27.995 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-16T20:51:27.995 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:27.995 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:27.995 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:27.995 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 2, 2024-09-16T20:51:27.996 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "075d16da-82bf-449b-9f6e-542c9e8d810b", 2024-09-16T20:51:27.996 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-16T20:51:27.996 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-16T20:51:27.996 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-16T20:51:27.996 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-16T20:51:27.996 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-16T20:51:27.996 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-16T20:51:27.996 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 13, 2024-09-16T20:51:27.996 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 199, 2024-09-16T20:51:27.996 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-16T20:51:27.996 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-16T20:51:27.996 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-16T20:51:27.996 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:51:27.997 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:27.997 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:51:27.997 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6808", 2024-09-16T20:51:27.997 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:51:27.997 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:27.997 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:27.997 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:51:27.997 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6809", 2024-09-16T20:51:27.997 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:51:27.997 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:27.997 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:27.997 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:27.997 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-16T20:51:27.997 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:51:27.998 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:27.998 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:51:27.998 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6810", 2024-09-16T20:51:27.998 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:51:27.998 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:27.998 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:27.998 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:51:27.998 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6811", 2024-09-16T20:51:27.998 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:51:27.998 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:27.998 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:27.998 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:27.998 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-16T20:51:27.999 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:51:27.999 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:27.999 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:51:27.999 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6814", 2024-09-16T20:51:27.999 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:51:27.999 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:27.999 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:27.999 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:51:27.999 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6815", 2024-09-16T20:51:28.000 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:51:28.000 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:28.000 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:28.000 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:28.000 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-16T20:51:28.000 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:51:28.000 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:28.000 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:51:28.000 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6812", 2024-09-16T20:51:28.000 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:51:28.000 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:28.000 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:28.000 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:51:28.001 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6813", 2024-09-16T20:51:28.001 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3447426536 2024-09-16T20:51:28.001 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:28.001 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:28.001 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:28.001 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.29:6809/3447426536", 2024-09-16T20:51:28.001 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.29:6811/3447426536", 2024-09-16T20:51:28.001 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.29:6815/3447426536", 2024-09-16T20:51:28.001 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.29:6813/3447426536", 2024-09-16T20:51:28.001 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-16T20:51:28.001 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-16T20:51:28.001 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-16T20:51:28.002 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:28.002 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:28.002 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:28.002 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 3, 2024-09-16T20:51:28.002 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "0612be38-3a24-41b6-aba4-5a2fe985165e", 2024-09-16T20:51:28.002 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-16T20:51:28.002 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-16T20:51:28.002 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-16T20:51:28.002 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-16T20:51:28.002 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-16T20:51:28.002 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-16T20:51:28.002 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 13, 2024-09-16T20:51:28.002 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 199, 2024-09-16T20:51:28.002 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-16T20:51:28.003 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-16T20:51:28.003 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-16T20:51:28.003 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:51:28.003 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:28.003 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:51:28.003 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6808", 2024-09-16T20:51:28.003 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:51:28.003 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:28.003 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:28.003 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:51:28.003 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6809", 2024-09-16T20:51:28.003 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:51:28.003 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:28.004 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:28.004 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:28.004 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-16T20:51:28.004 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:51:28.004 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:28.004 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:51:28.004 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6810", 2024-09-16T20:51:28.004 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:51:28.004 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:28.004 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:28.004 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:51:28.004 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6811", 2024-09-16T20:51:28.004 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:51:28.004 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:28.005 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:28.005 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:28.005 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-16T20:51:28.005 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:51:28.005 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:28.005 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:51:28.005 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6814", 2024-09-16T20:51:28.005 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:51:28.005 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:28.005 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:28.005 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:51:28.005 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6815", 2024-09-16T20:51:28.005 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:51:28.006 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:28.006 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:28.006 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:28.006 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-16T20:51:28.006 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:51:28.006 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:28.006 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:51:28.006 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6812", 2024-09-16T20:51:28.006 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:51:28.006 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:28.006 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:28.006 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:51:28.006 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6813", 2024-09-16T20:51:28.006 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3687583891 2024-09-16T20:51:28.007 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:28.007 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:28.007 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:28.007 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.43:6809/3687583891", 2024-09-16T20:51:28.007 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.43:6811/3687583891", 2024-09-16T20:51:28.007 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.43:6815/3687583891", 2024-09-16T20:51:28.007 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.43:6813/3687583891", 2024-09-16T20:51:28.007 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-16T20:51:28.007 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-16T20:51:28.007 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-16T20:51:28.007 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:28.007 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:28.007 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:28.007 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 4, 2024-09-16T20:51:28.008 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "be590bf8-e9fa-44ca-a471-6f6bcb087ca7", 2024-09-16T20:51:28.008 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-16T20:51:28.008 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-16T20:51:28.008 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-16T20:51:28.008 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-16T20:51:28.008 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-16T20:51:28.008 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-16T20:51:28.008 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 11, 2024-09-16T20:51:28.008 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 199, 2024-09-16T20:51:28.008 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-16T20:51:28.008 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-16T20:51:28.008 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-16T20:51:28.008 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:51:28.008 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:28.009 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:51:28.009 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6800", 2024-09-16T20:51:28.009 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:51:28.009 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:28.009 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:28.009 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:51:28.009 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6801", 2024-09-16T20:51:28.009 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:51:28.009 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:28.009 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:28.009 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:28.009 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-16T20:51:28.009 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:51:28.009 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:28.010 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:51:28.010 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6802", 2024-09-16T20:51:28.010 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:51:28.010 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:28.010 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:28.010 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:51:28.010 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6803", 2024-09-16T20:51:28.010 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:51:28.010 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:28.010 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:28.010 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:28.010 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-16T20:51:28.010 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:51:28.011 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:28.011 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:51:28.011 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6806", 2024-09-16T20:51:28.011 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:51:28.011 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:28.011 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:28.011 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:51:28.011 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6807", 2024-09-16T20:51:28.011 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:51:28.011 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:28.011 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:28.011 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:28.011 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-16T20:51:28.012 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:51:28.012 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:28.012 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:51:28.012 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6804", 2024-09-16T20:51:28.012 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:51:28.012 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:28.012 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:28.012 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:51:28.012 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6805", 2024-09-16T20:51:28.012 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2505220646 2024-09-16T20:51:28.012 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:28.012 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:28.012 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:28.012 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.43:6801/2505220646", 2024-09-16T20:51:28.013 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.43:6803/2505220646", 2024-09-16T20:51:28.013 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.43:6807/2505220646", 2024-09-16T20:51:28.013 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.43:6805/2505220646", 2024-09-16T20:51:28.013 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-16T20:51:28.013 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-16T20:51:28.013 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-16T20:51:28.013 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:28.013 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:28.013 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:28.013 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 5, 2024-09-16T20:51:28.013 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "4a208e30-031b-4a9f-af52-e26d4a039ea6", 2024-09-16T20:51:28.013 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-16T20:51:28.013 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-16T20:51:28.014 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-16T20:51:28.014 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-16T20:51:28.014 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-16T20:51:28.014 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-16T20:51:28.014 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 13, 2024-09-16T20:51:28.014 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 198, 2024-09-16T20:51:28.014 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-16T20:51:28.014 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-16T20:51:28.014 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-16T20:51:28.014 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:51:28.014 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:28.014 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:51:28.014 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6816", 2024-09-16T20:51:28.014 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:51:28.015 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:28.015 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:28.015 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:51:28.015 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6817", 2024-09-16T20:51:28.015 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:51:28.015 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:28.015 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:28.015 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:28.015 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-16T20:51:28.015 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:51:28.015 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:28.015 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:51:28.015 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6818", 2024-09-16T20:51:28.015 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:51:28.016 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:28.016 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:28.016 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:51:28.016 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6819", 2024-09-16T20:51:28.016 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:51:28.016 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:28.016 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:28.016 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:28.016 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-16T20:51:28.016 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:51:28.016 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:28.016 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:51:28.017 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6822", 2024-09-16T20:51:28.017 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:51:28.017 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:28.017 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:28.017 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:51:28.017 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6823", 2024-09-16T20:51:28.017 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:51:28.017 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:28.017 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:28.017 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:28.017 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-16T20:51:28.017 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-16T20:51:28.018 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:28.018 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-16T20:51:28.018 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6820", 2024-09-16T20:51:28.018 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:51:28.018 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:28.018 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:28.018 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-16T20:51:28.018 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.43:6821", 2024-09-16T20:51:28.018 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1517367307 2024-09-16T20:51:28.018 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:28.018 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:28.018 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:28.018 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.43:6817/1517367307", 2024-09-16T20:51:28.019 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.43:6819/1517367307", 2024-09-16T20:51:28.019 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.43:6823/1517367307", 2024-09-16T20:51:28.019 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.43:6821/1517367307", 2024-09-16T20:51:28.019 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-16T20:51:28.019 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-16T20:51:28.019 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-16T20:51:28.019 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:28.019 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:28.019 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-16T20:51:28.019 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_xinfo": [ 2024-09-16T20:51:28.019 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:28.019 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 0, 2024-09-16T20:51:28.019 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-16T20:51:28.020 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-16T20:51:28.020 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-16T20:51:28.020 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540719139924082687, 2024-09-16T20:51:28.020 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-16T20:51:28.020 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-16T20:47:47.887412+0000", 2024-09-16T20:51:28.020 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-16T20:51:28.020 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:28.020 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:28.020 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 1, 2024-09-16T20:51:28.020 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-16T20:51:28.020 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-16T20:51:28.020 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-16T20:51:28.020 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540719139924082687, 2024-09-16T20:51:28.021 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-16T20:51:28.021 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-16T20:47:47.658782+0000", 2024-09-16T20:51:28.021 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-16T20:51:28.021 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:28.021 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:28.021 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 2, 2024-09-16T20:51:28.021 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-16T20:51:28.021 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-16T20:51:28.021 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-16T20:51:28.021 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540719139924082687, 2024-09-16T20:51:28.021 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-16T20:51:28.021 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-16T20:47:47.453752+0000", 2024-09-16T20:51:28.021 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-16T20:51:28.021 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:28.022 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:28.022 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 3, 2024-09-16T20:51:28.022 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-16T20:51:28.022 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-16T20:51:28.022 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-16T20:51:28.022 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540719139924082687, 2024-09-16T20:51:28.022 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-16T20:51:28.022 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-16T20:47:47.487992+0000", 2024-09-16T20:51:28.022 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-16T20:51:28.022 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:28.022 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:28.022 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 4, 2024-09-16T20:51:28.022 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-16T20:51:28.022 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-16T20:51:28.023 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-16T20:51:28.023 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540719139924082687, 2024-09-16T20:51:28.023 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-16T20:51:28.023 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-16T20:47:46.943344+0000", 2024-09-16T20:51:28.023 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-16T20:51:28.023 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:28.023 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:28.023 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 5, 2024-09-16T20:51:28.023 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-16T20:51:28.023 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-16T20:51:28.023 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-16T20:51:28.023 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540719139924082687, 2024-09-16T20:51:28.023 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-16T20:51:28.023 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-16T20:47:47.901547+0000", 2024-09-16T20:51:28.024 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-16T20:51:28.024 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:28.024 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-16T20:51:28.024 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_upmap": [ 2024-09-16T20:51:28.024 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:28.024 INFO:teuthology.orchestra.run.smithi029.stdout: "pgid": "3.1b", 2024-09-16T20:51:28.024 INFO:teuthology.orchestra.run.smithi029.stdout: "osds": [ 2024-09-16T20:51:28.024 INFO:teuthology.orchestra.run.smithi029.stdout: 0, 2024-09-16T20:51:28.024 INFO:teuthology.orchestra.run.smithi029.stdout: 1 2024-09-16T20:51:28.024 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:28.024 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:28.024 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-16T20:51:28.024 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_upmap_items": [], 2024-09-16T20:51:28.025 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_upmap_primaries": [], 2024-09-16T20:51:28.025 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_temp": [], 2024-09-16T20:51:28.025 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_temp": [], 2024-09-16T20:51:28.025 INFO:teuthology.orchestra.run.smithi029.stdout: "blocklist": {}, 2024-09-16T20:51:28.025 INFO:teuthology.orchestra.run.smithi029.stdout: "range_blocklist": {}, 2024-09-16T20:51:28.025 INFO:teuthology.orchestra.run.smithi029.stdout: "erasure_code_profiles": { 2024-09-16T20:51:28.025 INFO:teuthology.orchestra.run.smithi029.stdout: "default": { 2024-09-16T20:51:28.025 INFO:teuthology.orchestra.run.smithi029.stdout: "crush-failure-domain": "osd", 2024-09-16T20:51:28.025 INFO:teuthology.orchestra.run.smithi029.stdout: "k": "2", 2024-09-16T20:51:28.025 INFO:teuthology.orchestra.run.smithi029.stdout: "m": "1", 2024-09-16T20:51:28.025 INFO:teuthology.orchestra.run.smithi029.stdout: "plugin": "jerasure", 2024-09-16T20:51:28.025 INFO:teuthology.orchestra.run.smithi029.stdout: "technique": "reed_sol_van" 2024-09-16T20:51:28.025 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:28.025 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:28.025 INFO:teuthology.orchestra.run.smithi029.stdout: "removed_snaps_queue": [ 2024-09-16T20:51:28.026 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:28.026 INFO:teuthology.orchestra.run.smithi029.stdout: "pool": 3, 2024-09-16T20:51:28.026 INFO:teuthology.orchestra.run.smithi029.stdout: "snaps": [ 2024-09-16T20:51:28.026 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:28.026 INFO:teuthology.orchestra.run.smithi029.stdout: "begin": 18, 2024-09-16T20:51:28.026 INFO:teuthology.orchestra.run.smithi029.stdout: "length": 6 2024-09-16T20:51:28.026 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:28.026 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:28.026 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:28.026 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-16T20:51:28.026 INFO:teuthology.orchestra.run.smithi029.stdout: "new_removed_snaps": [], 2024-09-16T20:51:28.026 INFO:teuthology.orchestra.run.smithi029.stdout: "new_purged_snaps": [], 2024-09-16T20:51:28.026 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_node_flags": {}, 2024-09-16T20:51:28.026 INFO:teuthology.orchestra.run.smithi029.stdout: "device_class_flags": {}, 2024-09-16T20:51:28.027 INFO:teuthology.orchestra.run.smithi029.stdout: "stretch_mode": { 2024-09-16T20:51:28.027 INFO:teuthology.orchestra.run.smithi029.stdout: "stretch_mode_enabled": false, 2024-09-16T20:51:28.027 INFO:teuthology.orchestra.run.smithi029.stdout: "stretch_bucket_count": 0, 2024-09-16T20:51:28.027 INFO:teuthology.orchestra.run.smithi029.stdout: "degraded_stretch_mode": 0, 2024-09-16T20:51:28.027 INFO:teuthology.orchestra.run.smithi029.stdout: "recovering_stretch_mode": 0, 2024-09-16T20:51:28.027 INFO:teuthology.orchestra.run.smithi029.stdout: "stretch_mode_bucket": 0 2024-09-16T20:51:28.027 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:28.027 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:28.029 INFO:tasks.thrashosds.thrasher:j is {'epoch': 200, 'fsid': 'e28af4a7-5fcd-4de8-8f77-a2b575361bb1', 'created': '2024-09-16T20:47:39.760609+0000', 'modified': '2024-09-16T20:51:27.618971+0000', 'last_up_change': '2024-09-16T20:47:50.238013+0000', 'last_in_change': '2024-09-16T20:51:22.595086+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T20:47:52.220750+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-16T20:47:56.756666+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-16T20:48:06.418889+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '198', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 48, 'snap_epoch': 198, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.309999942779541, 'score_stable': 1.309999942779541, 'optimal_score': 1, 'raw_score_acting': 1.309999942779541, 'raw_score_stable': 1.309999942779541, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '6c3cb2d7-7d23-4566-9ca1-d7b08e12360c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 199, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6816', 'nonce': 2744025949}, {'type': 'v1', 'addr': '172.21.15.29:6817', 'nonce': 2744025949}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6818', 'nonce': 2744025949}, {'type': 'v1', 'addr': '172.21.15.29:6819', 'nonce': 2744025949}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6822', 'nonce': 2744025949}, {'type': 'v1', 'addr': '172.21.15.29:6823', 'nonce': 2744025949}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6820', 'nonce': 2744025949}, {'type': 'v1', 'addr': '172.21.15.29:6821', 'nonce': 2744025949}]}, 'public_addr': '172.21.15.29:6817/2744025949', 'cluster_addr': '172.21.15.29:6819/2744025949', 'heartbeat_back_addr': '172.21.15.29:6823/2744025949', 'heartbeat_front_addr': '172.21.15.29:6821/2744025949', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '5cf8c20c-1ed2-42ae-a8b0-c88cc4e36bce', 'up': 1, 'in': 1, 'weight': 0.8807220458984375, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 197, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6800', 'nonce': 208628762}, {'type': 'v1', 'addr': '172.21.15.29:6801', 'nonce': 208628762}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6802', 'nonce': 208628762}, {'type': 'v1', 'addr': '172.21.15.29:6803', 'nonce': 208628762}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6806', 'nonce': 208628762}, {'type': 'v1', 'addr': '172.21.15.29:6807', 'nonce': 208628762}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6804', 'nonce': 208628762}, {'type': 'v1', 'addr': '172.21.15.29:6805', 'nonce': 208628762}]}, 'public_addr': '172.21.15.29:6801/208628762', 'cluster_addr': '172.21.15.29:6803/208628762', 'heartbeat_back_addr': '172.21.15.29:6807/208628762', 'heartbeat_front_addr': '172.21.15.29:6805/208628762', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '075d16da-82bf-449b-9f6e-542c9e8d810b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 199, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6808', 'nonce': 3447426536}, {'type': 'v1', 'addr': '172.21.15.29:6809', 'nonce': 3447426536}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6810', 'nonce': 3447426536}, {'type': 'v1', 'addr': '172.21.15.29:6811', 'nonce': 3447426536}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6814', 'nonce': 3447426536}, {'type': 'v1', 'addr': '172.21.15.29:6815', 'nonce': 3447426536}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6812', 'nonce': 3447426536}, {'type': 'v1', 'addr': '172.21.15.29:6813', 'nonce': 3447426536}]}, 'public_addr': '172.21.15.29:6809/3447426536', 'cluster_addr': '172.21.15.29:6811/3447426536', 'heartbeat_back_addr': '172.21.15.29:6815/3447426536', 'heartbeat_front_addr': '172.21.15.29:6813/3447426536', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '0612be38-3a24-41b6-aba4-5a2fe985165e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 199, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6808', 'nonce': 3687583891}, {'type': 'v1', 'addr': '172.21.15.43:6809', 'nonce': 3687583891}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6810', 'nonce': 3687583891}, {'type': 'v1', 'addr': '172.21.15.43:6811', 'nonce': 3687583891}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6814', 'nonce': 3687583891}, {'type': 'v1', 'addr': '172.21.15.43:6815', 'nonce': 3687583891}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6812', 'nonce': 3687583891}, {'type': 'v1', 'addr': '172.21.15.43:6813', 'nonce': 3687583891}]}, 'public_addr': '172.21.15.43:6809/3687583891', 'cluster_addr': '172.21.15.43:6811/3687583891', 'heartbeat_back_addr': '172.21.15.43:6815/3687583891', 'heartbeat_front_addr': '172.21.15.43:6813/3687583891', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'be590bf8-e9fa-44ca-a471-6f6bcb087ca7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 199, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6800', 'nonce': 2505220646}, {'type': 'v1', 'addr': '172.21.15.43:6801', 'nonce': 2505220646}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6802', 'nonce': 2505220646}, {'type': 'v1', 'addr': '172.21.15.43:6803', 'nonce': 2505220646}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6806', 'nonce': 2505220646}, {'type': 'v1', 'addr': '172.21.15.43:6807', 'nonce': 2505220646}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6804', 'nonce': 2505220646}, {'type': 'v1', 'addr': '172.21.15.43:6805', 'nonce': 2505220646}]}, 'public_addr': '172.21.15.43:6801/2505220646', 'cluster_addr': '172.21.15.43:6803/2505220646', 'heartbeat_back_addr': '172.21.15.43:6807/2505220646', 'heartbeat_front_addr': '172.21.15.43:6805/2505220646', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '4a208e30-031b-4a9f-af52-e26d4a039ea6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 198, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6816', 'nonce': 1517367307}, {'type': 'v1', 'addr': '172.21.15.43:6817', 'nonce': 1517367307}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6818', 'nonce': 1517367307}, {'type': 'v1', 'addr': '172.21.15.43:6819', 'nonce': 1517367307}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6822', 'nonce': 1517367307}, {'type': 'v1', 'addr': '172.21.15.43:6823', 'nonce': 1517367307}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6820', 'nonce': 1517367307}, {'type': 'v1', 'addr': '172.21.15.43:6821', 'nonce': 1517367307}]}, 'public_addr': '172.21.15.43:6817/1517367307', 'cluster_addr': '172.21.15.43:6819/1517367307', 'heartbeat_back_addr': '172.21.15.43:6823/1517367307', 'heartbeat_front_addr': '172.21.15.43:6821/1517367307', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T20:47:47.887412+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T20:47:47.658782+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T20:47:47.453752+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T20:47:47.487992+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T20:47:46.943344+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T20:47:47.901547+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.1b', 'osds': [0, 1]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 18, 'length': 6}]}], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-16T20:51:28.029 INFO:tasks.thrashosds.thrasher:Clearing pg_upmap on 3.1b 2024-09-16T20:51:28.029 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd rm-pg-upmap 3.1b 2024-09-16T20:51:28.071 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:28.070+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:28.133 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:28.133 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:51:28.133 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:51:28.134 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:51:28.134 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:28.142 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:51:28.144 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:28.143+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:28.244 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:51:28.243+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:51:28.291 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:28.291 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:51:28.291 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:51:28.291 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:51:28.291 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.292 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5286.0:10643 3.17 3.f6273fb7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-16T20:51:28.292 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:51:28.292 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 139.499114453, 2024-09-16T20:51:28.292 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.76672347200000002, 2024-09-16T20:51:28.292 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:28.292 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:28.292 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:28.292 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:28.292 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5286", 2024-09-16T20:51:28.292 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/4135433989", 2024-09-16T20:51:28.292 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 10643 2024-09-16T20:51:28.292 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.292 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:28.293 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.293 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:28.293 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:51:28.293 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:28.293 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.293 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.293 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:28.293 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:51:28.293 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:28.293 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.293 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.293 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:28.293 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790489+0000", 2024-09-16T20:51:28.293 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5910000000000001e-06 2024-09-16T20:51:28.293 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.294 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.294 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:28.294 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790493+0000", 2024-09-16T20:51:28.294 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2669999999999998e-06 2024-09-16T20:51:28.294 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.294 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.294 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:28.294 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790500+0000", 2024-09-16T20:51:28.294 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3780000000000003e-06 2024-09-16T20:51:28.294 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.294 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.294 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:28.294 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790505+0000", 2024-09-16T20:51:28.294 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1560000000000003e-06 2024-09-16T20:51:28.294 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.295 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.295 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:28.295 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790559+0000", 2024-09-16T20:51:28.295 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3494e-05 2024-09-16T20:51:28.295 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.295 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.295 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:28.295 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790566+0000", 2024-09-16T20:51:28.295 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7649999999999993e-06 2024-09-16T20:51:28.295 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.295 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.295 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:28.295 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:09.557198+0000", 2024-09-16T20:51:28.295 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.76663136200000004 2024-09-16T20:51:28.295 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.296 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.296 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:28.296 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:09.557211+0000", 2024-09-16T20:51:28.296 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3458999999999999e-05 2024-09-16T20:51:28.296 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:28.296 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:28.296 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:28.296 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.296 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.296 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5512.0:450 3.2 3:44e0cffa:::rbd_header.110ef0d7a535:head [watch reconnect cookie 140263298018912 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:51:28.296 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:51:28.296 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 111.72709154, 2024-09-16T20:51:28.296 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0084581939999999, 2024-09-16T20:51:28.296 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:28.297 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:28.297 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:28.297 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:28.297 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5512", 2024-09-16T20:51:28.297 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:51:28.297 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 450 2024-09-16T20:51:28.297 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.297 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:28.297 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.297 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:28.297 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:51:28.297 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:28.297 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.297 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.298 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:28.298 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:51:28.298 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:28.298 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.298 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.298 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:28.298 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562512+0000", 2024-09-16T20:51:28.298 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.009e-06 2024-09-16T20:51:28.298 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.298 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.298 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:28.298 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562517+0000", 2024-09-16T20:51:28.298 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9339999999999997e-06 2024-09-16T20:51:28.298 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.298 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.299 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:28.299 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562530+0000", 2024-09-16T20:51:28.299 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3495e-05 2024-09-16T20:51:28.299 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.299 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.299 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:28.299 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562536+0000", 2024-09-16T20:51:28.299 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5690000000000004e-06 2024-09-16T20:51:28.299 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.299 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.299 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:28.299 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562591+0000", 2024-09-16T20:51:28.299 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5761999999999999e-05 2024-09-16T20:51:28.299 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.300 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.300 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:51:28.300 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562599+0000", 2024-09-16T20:51:28.300 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9179999999999994e-06 2024-09-16T20:51:28.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.300 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.300 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:28.300 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.565971+0000", 2024-09-16T20:51:28.300 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003371546 2024-09-16T20:51:28.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.300 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.300 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:28.300 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566019+0000", 2024-09-16T20:51:28.300 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7849000000000002e-05 2024-09-16T20:51:28.301 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.301 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.301 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:28.301 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.569382+0000", 2024-09-16T20:51:28.301 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.003362833 2024-09-16T20:51:28.301 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.301 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.301 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:28.301 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.569529+0000", 2024-09-16T20:51:28.301 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000147823 2024-09-16T20:51:28.301 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.301 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.301 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:28.301 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570969+0000", 2024-09-16T20:51:28.302 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001439456 2024-09-16T20:51:28.302 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:28.302 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:28.302 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:28.302 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.302 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.302 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:14996 3.2 3:44e0cffa:::rbd_header.110ef0d7a535:head [watch reconnect cookie 140213436269840 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:51:28.302 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:51:28.302 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 111.72704312800001, 2024-09-16T20:51:28.302 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.008813943, 2024-09-16T20:51:28.302 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:28.302 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:28.302 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:28.302 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:28.302 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:28.303 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:28.303 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 14996 2024-09-16T20:51:28.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.303 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:28.303 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.303 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:28.303 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:51:28.303 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:28.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.303 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.303 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:28.303 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:51:28.303 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:28.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.303 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.304 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:28.304 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562560+0000", 2024-09-16T20:51:28.304 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.06e-07 2024-09-16T20:51:28.304 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.304 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.304 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:28.304 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562563+0000", 2024-09-16T20:51:28.304 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.495e-06 2024-09-16T20:51:28.304 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.304 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.304 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:28.304 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562571+0000", 2024-09-16T20:51:28.304 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1950000000000003e-06 2024-09-16T20:51:28.304 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.305 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:28.305 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562579+0000", 2024-09-16T20:51:28.305 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2649999999999999e-06 2024-09-16T20:51:28.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.305 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:28.305 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562673+0000", 2024-09-16T20:51:28.305 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4118999999999998e-05 2024-09-16T20:51:28.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.305 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:51:28.305 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562677+0000", 2024-09-16T20:51:28.305 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5179999999999998e-06 2024-09-16T20:51:28.306 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.306 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.306 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:28.306 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566461+0000", 2024-09-16T20:51:28.306 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0037837259999999998 2024-09-16T20:51:28.306 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.306 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.306 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:28.306 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566468+0000", 2024-09-16T20:51:28.306 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3350000000000001e-06 2024-09-16T20:51:28.306 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.306 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.306 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:28.306 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570306+0000", 2024-09-16T20:51:28.306 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.003837997 2024-09-16T20:51:28.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.307 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.307 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:28.307 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570392+0000", 2024-09-16T20:51:28.307 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5872999999999996e-05 2024-09-16T20:51:28.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.307 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.307 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:28.307 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571373+0000", 2024-09-16T20:51:28.307 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00098071399999999993 2024-09-16T20:51:28.307 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:28.307 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:28.307 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:28.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.307 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.308 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5255.0:1931 3.2 3:47882819:::rbd_data.110eb5f95afc.0000000000000009:head [read 217088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:51:28.308 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:51:28.308 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 111.727004378, 2024-09-16T20:51:28.308 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0082752880000001, 2024-09-16T20:51:28.308 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:28.308 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:28.308 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:28.308 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:28.308 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5255", 2024-09-16T20:51:28.308 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:51:28.308 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1931 2024-09-16T20:51:28.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.308 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:28.308 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.309 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:28.309 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:51:28.309 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:28.309 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.309 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.309 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:28.309 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:51:28.309 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:28.309 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.309 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.309 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:28.309 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:51:28.309 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9699999999999996e-07 2024-09-16T20:51:28.309 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.309 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.310 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:28.310 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562601+0000", 2024-09-16T20:51:28.310 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8600000000000001e-06 2024-09-16T20:51:28.310 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.310 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.310 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:28.310 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562608+0000", 2024-09-16T20:51:28.310 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9e-06 2024-09-16T20:51:28.310 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.310 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.310 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:28.310 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562615+0000", 2024-09-16T20:51:28.310 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8179999999999998e-06 2024-09-16T20:51:28.310 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.310 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.311 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:28.311 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562703+0000", 2024-09-16T20:51:28.311 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7811000000000003e-05 2024-09-16T20:51:28.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.311 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.311 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:51:28.311 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562706+0000", 2024-09-16T20:51:28.311 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4920000000000002e-06 2024-09-16T20:51:28.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.311 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.311 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:28.311 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566685+0000", 2024-09-16T20:51:28.311 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0039792780000000002 2024-09-16T20:51:28.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.311 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.312 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:28.312 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566693+0000", 2024-09-16T20:51:28.312 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4730000000000003e-06 2024-09-16T20:51:28.312 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.312 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.312 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:28.312 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570737+0000", 2024-09-16T20:51:28.312 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0040436749999999 2024-09-16T20:51:28.312 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.312 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.312 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:28.312 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570873+0000", 2024-09-16T20:51:28.312 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000136484 2024-09-16T20:51:28.313 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:28.313 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:28.313 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:28.313 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.313 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.313 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5255.0:1939 3.2 3:47882819:::rbd_data.110eb5f95afc.0000000000000009:head [read 233472~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:51:28.313 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:51:28.313 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 111.726849512, 2024-09-16T20:51:28.313 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0084657589999999, 2024-09-16T20:51:28.313 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:28.313 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:28.313 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:28.313 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:28.313 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5255", 2024-09-16T20:51:28.314 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:51:28.314 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1939 2024-09-16T20:51:28.314 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.314 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:28.314 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.314 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:28.314 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:51:28.314 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:28.314 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.314 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.314 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:28.314 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:51:28.314 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:28.314 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.314 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.315 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:28.315 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:51:28.315 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5199999999999997e-07 2024-09-16T20:51:28.315 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.315 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.315 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:28.315 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562757+0000", 2024-09-16T20:51:28.315 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4979999999999998e-06 2024-09-16T20:51:28.315 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.315 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.315 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:28.315 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562764+0000", 2024-09-16T20:51:28.315 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1470000000000004e-06 2024-09-16T20:51:28.315 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.316 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.316 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:28.316 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562768+0000", 2024-09-16T20:51:28.316 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7929999999999999e-06 2024-09-16T20:51:28.316 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.316 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.316 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:28.316 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562805+0000", 2024-09-16T20:51:28.316 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7477e-05 2024-09-16T20:51:28.316 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.316 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.316 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting_for_map not empty", 2024-09-16T20:51:28.316 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562810+0000", 2024-09-16T20:51:28.316 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0819999999999998e-06 2024-09-16T20:51:28.316 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.317 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.317 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:28.317 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566717+0000", 2024-09-16T20:51:28.317 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0039072600000000001 2024-09-16T20:51:28.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.317 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.317 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:28.317 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566724+0000", 2024-09-16T20:51:28.317 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4880000000000004e-06 2024-09-16T20:51:28.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.317 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.317 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:28.317 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571009+0000", 2024-09-16T20:51:28.317 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.004284677 2024-09-16T20:51:28.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.318 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.318 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:28.318 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571218+0000", 2024-09-16T20:51:28.318 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00020978500000000001 2024-09-16T20:51:28.318 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:28.318 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:28.318 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:28.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.318 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.318 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:11021 3.1f 3:fee2d1dd:::rbd_data.110ea93691bb.000000000000000e:23 [read 960512~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:28.318 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.693837+0000", 2024-09-16T20:51:28.318 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 97.595764966999994, 2024-09-16T20:51:28.318 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057155138000000001, 2024-09-16T20:51:28.318 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:28.318 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:28.319 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:28.319 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:28.319 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:28.319 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:28.319 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 11021 2024-09-16T20:51:28.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.319 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:28.319 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.319 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:28.319 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693837+0000", 2024-09-16T20:51:28.319 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:28.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.319 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.319 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:28.320 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693837+0000", 2024-09-16T20:51:28.320 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:28.320 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.320 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.320 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:28.320 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699566+0000", 2024-09-16T20:51:28.320 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057283639999999997 2024-09-16T20:51:28.320 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.320 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.320 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:28.320 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699571+0000", 2024-09-16T20:51:28.320 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.604e-06 2024-09-16T20:51:28.320 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.320 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.320 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:28.321 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699583+0000", 2024-09-16T20:51:28.321 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1372999999999999e-05 2024-09-16T20:51:28.321 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.321 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.321 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:28.321 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699591+0000", 2024-09-16T20:51:28.321 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5259999999999993e-06 2024-09-16T20:51:28.321 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.321 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.321 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:28.321 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750674+0000", 2024-09-16T20:51:28.321 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051083154999999998 2024-09-16T20:51:28.321 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.321 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.321 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:28.322 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750745+0000", 2024-09-16T20:51:28.322 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0933999999999995e-05 2024-09-16T20:51:28.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.322 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.322 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:28.322 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750992+0000", 2024-09-16T20:51:28.322 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00024718199999999998 2024-09-16T20:51:28.322 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:28.322 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:28.322 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:28.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.322 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.322 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8792 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2609152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:28.322 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:51:28.322 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 87.232305249999996, 2024-09-16T20:51:28.323 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057003524999999999, 2024-09-16T20:51:28.323 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:28.323 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:28.323 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:28.323 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:28.323 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:28.323 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:28.323 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8792 2024-09-16T20:51:28.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.323 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:28.323 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.323 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:28.324 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:51:28.324 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:28.324 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.324 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.324 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:28.324 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:51:28.324 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:28.324 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.324 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.324 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:28.324 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:51:28.324 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0400000000000002e-07 2024-09-16T20:51:28.324 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.324 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.325 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:28.325 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057299+0000", 2024-09-16T20:51:28.325 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7460000000000001e-06 2024-09-16T20:51:28.325 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.325 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.325 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:28.325 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057304+0000", 2024-09-16T20:51:28.326 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3210000000000004e-06 2024-09-16T20:51:28.326 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.326 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.326 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:28.326 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057310+0000", 2024-09-16T20:51:28.326 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4360000000000001e-06 2024-09-16T20:51:28.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.327 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.327 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:28.327 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114200+0000", 2024-09-16T20:51:28.327 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056889954 2024-09-16T20:51:28.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.327 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.328 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:28.328 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114243+0000", 2024-09-16T20:51:28.328 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2998999999999999e-05 2024-09-16T20:51:28.328 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.328 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.328 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:28.328 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114300+0000", 2024-09-16T20:51:28.328 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7765e-05 2024-09-16T20:51:28.328 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:28.328 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:28.328 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:28.328 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.329 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.329 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8794 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2351104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:28.329 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057327+0000", 2024-09-16T20:51:28.329 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 87.232274926000002, 2024-09-16T20:51:28.329 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057101130999999999, 2024-09-16T20:51:28.329 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:28.329 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:28.329 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:28.329 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:28.329 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:28.329 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:28.329 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8794 2024-09-16T20:51:28.329 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.330 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:28.330 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.330 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:28.330 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057327+0000", 2024-09-16T20:51:28.330 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:28.330 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.330 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.330 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:28.330 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057327+0000", 2024-09-16T20:51:28.330 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:28.330 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.330 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.330 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:28.330 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057328+0000", 2024-09-16T20:51:28.331 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7399999999999999e-07 2024-09-16T20:51:28.331 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.331 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.331 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:28.331 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057330+0000", 2024-09-16T20:51:28.331 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2199999999999999e-06 2024-09-16T20:51:28.331 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.331 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.331 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:28.331 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057336+0000", 2024-09-16T20:51:28.331 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1240000000000002e-06 2024-09-16T20:51:28.332 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.332 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.332 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:28.332 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057341+0000", 2024-09-16T20:51:28.332 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8470000000000003e-06 2024-09-16T20:51:28.332 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.332 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.332 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:28.332 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114334+0000", 2024-09-16T20:51:28.332 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056993267 2024-09-16T20:51:28.332 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.332 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.333 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:28.333 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114376+0000", 2024-09-16T20:51:28.333 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1638999999999999e-05 2024-09-16T20:51:28.333 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.333 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.333 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:28.333 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114428+0000", 2024-09-16T20:51:28.333 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2660000000000001e-05 2024-09-16T20:51:28.333 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:28.333 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:28.333 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:28.333 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.333 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.334 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8798 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2297856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:28.334 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:51:28.334 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 87.232255198999994, 2024-09-16T20:51:28.334 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057277041000000001, 2024-09-16T20:51:28.334 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:28.334 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:28.334 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:28.334 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:28.334 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:28.334 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:28.334 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8798 2024-09-16T20:51:28.334 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.334 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:28.335 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.335 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:28.335 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:51:28.335 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:28.335 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.335 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.335 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:28.335 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:51:28.335 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:28.335 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.335 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.335 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:28.335 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:51:28.335 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2800000000000003e-07 2024-09-16T20:51:28.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.336 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.336 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:28.336 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057354+0000", 2024-09-16T20:51:28.336 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7780000000000002e-06 2024-09-16T20:51:28.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.336 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.336 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:28.336 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057360+0000", 2024-09-16T20:51:28.336 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0419999999999999e-06 2024-09-16T20:51:28.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.336 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.336 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:28.337 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057365+0000", 2024-09-16T20:51:28.337 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7929999999999997e-06 2024-09-16T20:51:28.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.337 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.337 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:28.337 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114460+0000", 2024-09-16T20:51:28.337 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057094662999999997 2024-09-16T20:51:28.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.337 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.337 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:28.337 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114532+0000", 2024-09-16T20:51:28.337 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2881999999999998e-05 2024-09-16T20:51:28.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.338 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.338 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:28.338 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114624+0000", 2024-09-16T20:51:28.338 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1555e-05 2024-09-16T20:51:28.338 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:28.338 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:28.338 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:28.338 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.338 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.338 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8800 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2316288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:28.338 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:51:28.338 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 87.232231260000006, 2024-09-16T20:51:28.338 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.05746946, 2024-09-16T20:51:28.339 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:28.339 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:28.339 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:28.339 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:28.339 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:28.339 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:28.339 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8800 2024-09-16T20:51:28.339 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.339 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:28.339 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.339 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:28.339 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:51:28.339 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:28.340 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.340 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.340 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:28.340 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:51:28.340 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:28.340 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.340 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.340 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:28.340 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:51:28.340 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5199999999999998e-07 2024-09-16T20:51:28.340 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.340 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.341 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:28.341 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057373+0000", 2024-09-16T20:51:28.341 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.79e-06 2024-09-16T20:51:28.341 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.341 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.341 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:28.341 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057379+0000", 2024-09-16T20:51:28.341 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6699999999999999e-06 2024-09-16T20:51:28.341 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.341 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.341 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:28.341 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057386+0000", 2024-09-16T20:51:28.341 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3170000000000004e-06 2024-09-16T20:51:28.342 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.342 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.342 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:28.342 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114655+0000", 2024-09-16T20:51:28.342 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057269420000000001 2024-09-16T20:51:28.342 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.342 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.342 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:28.342 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114734+0000", 2024-09-16T20:51:28.342 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8084000000000007e-05 2024-09-16T20:51:28.342 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.342 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.342 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:28.342 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114840+0000", 2024-09-16T20:51:28.343 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000106827 2024-09-16T20:51:28.343 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:28.343 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:28.343 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:28.343 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.343 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.343 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8801 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2334720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:28.343 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:51:28.343 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 87.232210229000003, 2024-09-16T20:51:28.343 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057594682000000001, 2024-09-16T20:51:28.343 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:28.343 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:28.343 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:28.343 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:28.343 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:28.344 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:28.344 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8801 2024-09-16T20:51:28.344 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.344 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:28.344 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.344 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:28.344 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:51:28.344 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:28.344 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.344 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.344 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:28.344 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:51:28.344 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:28.344 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.345 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.345 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:28.345 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:51:28.345 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6899999999999998e-07 2024-09-16T20:51:28.345 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.345 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.345 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:28.345 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057394+0000", 2024-09-16T20:51:28.345 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8300000000000001e-06 2024-09-16T20:51:28.345 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.345 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.345 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:28.345 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057400+0000", 2024-09-16T20:51:28.345 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6500000000000001e-06 2024-09-16T20:51:28.345 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.346 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.346 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:28.346 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057404+0000", 2024-09-16T20:51:28.346 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4669999999999998e-06 2024-09-16T20:51:28.346 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.346 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.346 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:28.346 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114874+0000", 2024-09-16T20:51:28.346 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057469602000000002 2024-09-16T20:51:28.346 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.346 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.346 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:28.346 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114933+0000", 2024-09-16T20:51:28.346 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9500000000000003e-05 2024-09-16T20:51:28.346 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.347 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.347 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:28.347 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114987+0000", 2024-09-16T20:51:28.347 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3263999999999997e-05 2024-09-16T20:51:28.347 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:28.347 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:28.347 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:28.347 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.347 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.347 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8802 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2353152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:28.347 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:51:28.347 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 87.232161543999993, 2024-09-16T20:51:28.347 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057714079000000001, 2024-09-16T20:51:28.347 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:28.347 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:28.347 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:28.348 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:28.348 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:28.348 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:28.348 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8802 2024-09-16T20:51:28.348 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.348 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:28.348 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.348 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:28.348 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:51:28.348 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:28.348 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.348 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.349 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:28.349 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:51:28.349 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:28.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.349 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.349 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:28.349 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:51:28.349 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4999999999999998e-07 2024-09-16T20:51:28.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.349 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.349 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:28.349 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057446+0000", 2024-09-16T20:51:28.349 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4240000000000003e-06 2024-09-16T20:51:28.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.350 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.350 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:28.350 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057452+0000", 2024-09-16T20:51:28.350 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8379999999999996e-06 2024-09-16T20:51:28.350 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.350 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.350 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:28.350 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057460+0000", 2024-09-16T20:51:28.350 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9629999999999998e-06 2024-09-16T20:51:28.350 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.350 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.350 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:28.350 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115015+0000", 2024-09-16T20:51:28.350 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057554389999999997 2024-09-16T20:51:28.351 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.351 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.351 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:28.351 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115082+0000", 2024-09-16T20:51:28.351 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7733999999999999e-05 2024-09-16T20:51:28.351 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.351 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.351 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:28.351 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115155+0000", 2024-09-16T20:51:28.351 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2280000000000006e-05 2024-09-16T20:51:28.351 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:28.351 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:28.351 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:28.352 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.352 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.352 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8804 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2555904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:28.352 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:51:28.352 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 87.232135228000004, 2024-09-16T20:51:28.352 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057833862, 2024-09-16T20:51:28.352 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:28.352 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:28.352 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:28.352 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:28.352 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:28.353 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:28.353 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8804 2024-09-16T20:51:28.353 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.353 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:28.353 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.353 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:28.353 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:51:28.353 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:28.353 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.353 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.354 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:28.354 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:51:28.354 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:28.354 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.354 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.354 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:28.354 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:51:28.354 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7899999999999999e-07 2024-09-16T20:51:28.354 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.354 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.355 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:28.355 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057469+0000", 2024-09-16T20:51:28.355 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.001e-06 2024-09-16T20:51:28.355 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.355 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.355 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:28.355 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057475+0000", 2024-09-16T20:51:28.355 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7629999999999997e-06 2024-09-16T20:51:28.355 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.355 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.355 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:28.356 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057480+0000", 2024-09-16T20:51:28.356 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8069999999999999e-06 2024-09-16T20:51:28.356 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.356 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.356 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:28.356 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115183+0000", 2024-09-16T20:51:28.356 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057703364 2024-09-16T20:51:28.356 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.356 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.356 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:28.356 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115227+0000", 2024-09-16T20:51:28.356 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3903999999999998e-05 2024-09-16T20:51:28.357 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.357 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.357 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:28.357 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115301+0000", 2024-09-16T20:51:28.357 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3543999999999999e-05 2024-09-16T20:51:28.357 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:28.357 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:28.357 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:28.357 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.357 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.357 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8805 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2574336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:28.357 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057487+0000", 2024-09-16T20:51:28.357 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 87.232114899999999, 2024-09-16T20:51:28.357 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058021027000000003, 2024-09-16T20:51:28.358 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:28.358 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:28.358 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:28.358 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:28.358 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:28.358 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:28.358 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8805 2024-09-16T20:51:28.358 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.358 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:28.358 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.358 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:28.358 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057487+0000", 2024-09-16T20:51:28.359 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:28.359 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.359 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.359 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:28.359 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057487+0000", 2024-09-16T20:51:28.359 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:28.359 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.359 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.359 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:28.359 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057488+0000", 2024-09-16T20:51:28.359 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5199999999999998e-07 2024-09-16T20:51:28.359 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.359 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.359 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:28.359 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057490+0000", 2024-09-16T20:51:28.360 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.939e-06 2024-09-16T20:51:28.360 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.360 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.360 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:28.360 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057495+0000", 2024-09-16T20:51:28.360 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6439999999999997e-06 2024-09-16T20:51:28.360 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.360 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.360 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:28.360 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057500+0000", 2024-09-16T20:51:28.360 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8119999999999998e-06 2024-09-16T20:51:28.360 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.360 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.360 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:28.361 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115341+0000", 2024-09-16T20:51:28.361 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057841341999999997 2024-09-16T20:51:28.361 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.361 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.361 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:28.361 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115422+0000", 2024-09-16T20:51:28.361 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1003999999999999e-05 2024-09-16T20:51:28.361 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.361 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.361 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:28.361 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115508+0000", 2024-09-16T20:51:28.361 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5933999999999994e-05 2024-09-16T20:51:28.361 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:28.361 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:28.362 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:28.362 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.362 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.362 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8807 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2592768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:28.362 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057505+0000", 2024-09-16T20:51:28.362 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 87.232096690000006, 2024-09-16T20:51:28.362 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058186633000000001, 2024-09-16T20:51:28.362 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:28.362 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:28.362 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:28.362 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:28.362 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:28.362 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:28.362 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8807 2024-09-16T20:51:28.362 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.363 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:28.363 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.363 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:28.363 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057505+0000", 2024-09-16T20:51:28.363 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:28.363 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.363 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.363 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:28.363 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057505+0000", 2024-09-16T20:51:28.363 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:28.363 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.363 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.363 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:28.363 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057506+0000", 2024-09-16T20:51:28.364 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.27e-07 2024-09-16T20:51:28.364 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.364 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.364 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:28.364 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057508+0000", 2024-09-16T20:51:28.364 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7400000000000001e-06 2024-09-16T20:51:28.364 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.364 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.364 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:28.364 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057513+0000", 2024-09-16T20:51:28.364 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.468e-06 2024-09-16T20:51:28.364 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.364 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.364 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:28.364 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057517+0000", 2024-09-16T20:51:28.365 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4329999999999997e-06 2024-09-16T20:51:28.365 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.365 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.365 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:28.365 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115538+0000", 2024-09-16T20:51:28.365 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058020355000000003 2024-09-16T20:51:28.365 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.365 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.365 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:28.365 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115616+0000", 2024-09-16T20:51:28.365 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7998999999999996e-05 2024-09-16T20:51:28.366 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.366 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.366 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:28.366 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115692+0000", 2024-09-16T20:51:28.366 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6310999999999995e-05 2024-09-16T20:51:28.366 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:28.366 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:28.366 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:28.366 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.366 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.366 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8808 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2611200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:28.367 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:51:28.367 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 87.232079428999995, 2024-09-16T20:51:28.367 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058354046, 2024-09-16T20:51:28.367 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:28.367 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:28.367 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:28.367 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:28.367 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:28.367 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:28.367 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8808 2024-09-16T20:51:28.367 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.367 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:28.367 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.367 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:28.367 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:51:28.368 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:28.368 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.368 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.368 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:28.368 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:51:28.368 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:28.368 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.368 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.368 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:28.368 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:51:28.368 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2500000000000001e-07 2024-09-16T20:51:28.368 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.368 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.369 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:28.369 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057525+0000", 2024-09-16T20:51:28.369 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5549999999999999e-06 2024-09-16T20:51:28.369 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.369 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.369 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:28.369 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057530+0000", 2024-09-16T20:51:28.369 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5489999999999997e-06 2024-09-16T20:51:28.369 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.369 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.369 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:28.369 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057535+0000", 2024-09-16T20:51:28.369 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5709999999999999e-06 2024-09-16T20:51:28.369 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.370 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.370 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:28.370 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115722+0000", 2024-09-16T20:51:28.370 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058187323999999999 2024-09-16T20:51:28.370 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.370 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.370 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:28.370 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115799+0000", 2024-09-16T20:51:28.370 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6445000000000004e-05 2024-09-16T20:51:28.370 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.370 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.370 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:28.370 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115877+0000", 2024-09-16T20:51:28.370 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8276999999999996e-05 2024-09-16T20:51:28.370 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:28.371 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:28.371 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:28.371 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.371 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.371 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:2182 3.17 3:edb5e5cd:::rbd_data.110e61479aee.0000000000000014:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write 0~5120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-16T20:51:28.371 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:22.734609+0000", 2024-09-16T20:51:28.371 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 65.554993182000004, 2024-09-16T20:51:28.371 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.124890894, 2024-09-16T20:51:28.371 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:28.371 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:28.371 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:28.371 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:28.371 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:51:28.372 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:28.372 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 2182 2024-09-16T20:51:28.372 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.372 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:28.372 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.372 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:28.372 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734609+0000", 2024-09-16T20:51:28.372 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:28.372 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.372 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.372 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:28.373 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734609+0000", 2024-09-16T20:51:28.373 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:28.373 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.373 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.373 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:28.373 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734611+0000", 2024-09-16T20:51:28.373 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9599999999999999e-06 2024-09-16T20:51:28.373 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.373 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.373 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:28.373 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734628+0000", 2024-09-16T20:51:28.373 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.715e-05 2024-09-16T20:51:28.373 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.373 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.374 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:28.374 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734656+0000", 2024-09-16T20:51:28.374 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7759999999999998e-05 2024-09-16T20:51:28.374 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.374 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.374 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:28.374 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734669+0000", 2024-09-16T20:51:28.374 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3315999999999999e-05 2024-09-16T20:51:28.374 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.374 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.374 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:28.374 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734826+0000", 2024-09-16T20:51:28.374 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015668499999999999 2024-09-16T20:51:28.374 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.375 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.375 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:28.375 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.735048+0000", 2024-09-16T20:51:28.375 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00022179 2024-09-16T20:51:28.375 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.375 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.375 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:28.375 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.735248+0000", 2024-09-16T20:51:28.375 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00020010399999999999 2024-09-16T20:51:28.375 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.375 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.375 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:51:28.375 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.736252+0000", 2024-09-16T20:51:28.376 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010039739999999999 2024-09-16T20:51:28.376 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.376 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.376 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:28.376 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.859340+0000", 2024-09-16T20:51:28.376 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.12308846599999999 2024-09-16T20:51:28.376 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.376 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.376 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:51:28.376 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.859431+0000", 2024-09-16T20:51:28.376 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1099000000000003e-05 2024-09-16T20:51:28.376 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.376 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.376 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:28.376 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.859500+0000", 2024-09-16T20:51:28.377 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8590000000000006e-05 2024-09-16T20:51:28.377 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:28.377 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:28.377 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:28.377 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.377 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.377 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:29157 3.1b 3:dbe69130:::journal.110e1b19a241:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e167)", 2024-09-16T20:51:28.377 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:54.663540+0000", 2024-09-16T20:51:28.377 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 33.626061843999999, 2024-09-16T20:51:28.377 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.92912227999999997, 2024-09-16T20:51:28.377 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:28.377 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:28.378 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:28.378 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:28.378 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:28.378 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:28.378 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 29157 2024-09-16T20:51:28.378 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.378 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:28.378 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.378 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:28.378 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663540+0000", 2024-09-16T20:51:28.378 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:28.378 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.378 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.378 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:28.378 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663540+0000", 2024-09-16T20:51:28.379 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:28.379 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.379 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.379 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:28.379 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663541+0000", 2024-09-16T20:51:28.379 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1680000000000001e-06 2024-09-16T20:51:28.379 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.379 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.379 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:28.379 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663546+0000", 2024-09-16T20:51:28.379 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7129999999999996e-06 2024-09-16T20:51:28.379 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.379 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.380 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:28.380 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663558+0000", 2024-09-16T20:51:28.380 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1347e-05 2024-09-16T20:51:28.380 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.380 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.380 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:28.380 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663563+0000", 2024-09-16T20:51:28.380 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1250000000000001e-06 2024-09-16T20:51:28.380 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.380 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.380 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:28.380 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663612+0000", 2024-09-16T20:51:28.380 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9665e-05 2024-09-16T20:51:28.381 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.381 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.381 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:28.381 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663620+0000", 2024-09-16T20:51:28.381 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8439999999999998e-06 2024-09-16T20:51:28.381 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.381 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.381 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:28.381 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:55.592525+0000", 2024-09-16T20:51:28.381 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.92890456700000001 2024-09-16T20:51:28.381 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.381 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.381 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:28.382 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:55.592663+0000", 2024-09-16T20:51:28.382 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013785100000000001 2024-09-16T20:51:28.382 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:28.382 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:28.382 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:28.382 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.382 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.382 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:30229 3.16 3:6f5476df:::rbd_header.110e25db1f60:head [watch reconnect cookie 140213637350016 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e181)", 2024-09-16T20:51:28.382 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:51:08.603124+0000", 2024-09-16T20:51:28.382 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.686478199, 2024-09-16T20:51:28.382 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.97765519899999997, 2024-09-16T20:51:28.382 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:28.383 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:28.383 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:28.383 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:28.383 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:28.383 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:28.383 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30229 2024-09-16T20:51:28.383 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.383 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:28.383 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.383 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:28.383 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603124+0000", 2024-09-16T20:51:28.383 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:28.384 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.384 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.384 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:28.384 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603124+0000", 2024-09-16T20:51:28.384 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:28.384 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.384 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.384 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:28.384 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603125+0000", 2024-09-16T20:51:28.384 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5090000000000001e-06 2024-09-16T20:51:28.384 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.384 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.385 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:28.385 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603131+0000", 2024-09-16T20:51:28.385 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9089999999999997e-06 2024-09-16T20:51:28.385 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.385 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.385 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:28.385 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603143+0000", 2024-09-16T20:51:28.385 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1416e-05 2024-09-16T20:51:28.385 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.385 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.385 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:28.385 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603151+0000", 2024-09-16T20:51:28.386 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7889999999999995e-06 2024-09-16T20:51:28.386 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.386 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.386 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:28.386 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603674+0000", 2024-09-16T20:51:28.386 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00052357 2024-09-16T20:51:28.386 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.386 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.386 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:28.386 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603684+0000", 2024-09-16T20:51:28.386 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0274e-05 2024-09-16T20:51:28.386 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.386 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.386 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:28.387 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.575380+0000", 2024-09-16T20:51:28.387 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.97169509799999998 2024-09-16T20:51:28.387 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.387 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.387 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:28.387 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.575591+0000", 2024-09-16T20:51:28.387 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00021126700000000001 2024-09-16T20:51:28.387 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.387 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.387 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:28.387 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.580779+0000", 2024-09-16T20:51:28.387 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0051883670000000002 2024-09-16T20:51:28.388 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:28.388 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:28.388 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:28.388 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.388 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.388 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:30230 3.16 3:6f5476df:::rbd_header.110e25db1f60:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e181)", 2024-09-16T20:51:28.388 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:51:08.603204+0000", 2024-09-16T20:51:28.388 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.686398251, 2024-09-16T20:51:28.388 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.98129583300000001, 2024-09-16T20:51:28.388 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:28.388 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:28.388 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:28.389 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:28.389 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:28.389 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:28.389 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30230 2024-09-16T20:51:28.389 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.389 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:28.389 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.389 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:28.389 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603204+0000", 2024-09-16T20:51:28.389 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:28.389 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.389 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.389 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:28.390 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603204+0000", 2024-09-16T20:51:28.390 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:28.390 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.390 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.390 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:28.390 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603205+0000", 2024-09-16T20:51:28.390 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4900000000000005e-07 2024-09-16T20:51:28.390 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.390 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.390 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:28.390 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603209+0000", 2024-09-16T20:51:28.390 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5210000000000004e-06 2024-09-16T20:51:28.390 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.391 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.391 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:28.391 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603222+0000", 2024-09-16T20:51:28.391 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2733e-05 2024-09-16T20:51:28.391 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.391 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.391 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:28.391 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603228+0000", 2024-09-16T20:51:28.391 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4520000000000002e-06 2024-09-16T20:51:28.391 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.391 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.391 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:28.391 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603719+0000", 2024-09-16T20:51:28.392 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00049103600000000003 2024-09-16T20:51:28.392 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.392 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.392 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:28.392 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603730+0000", 2024-09-16T20:51:28.392 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.023e-05 2024-09-16T20:51:28.392 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.392 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.392 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:28.392 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.577394+0000", 2024-09-16T20:51:28.392 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.97366440099999996 2024-09-16T20:51:28.392 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.392 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.392 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:28.393 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.577499+0000", 2024-09-16T20:51:28.393 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000104509 2024-09-16T20:51:28.393 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.393 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.393 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:28.393 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.577920+0000", 2024-09-16T20:51:28.393 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00042173699999999999 2024-09-16T20:51:28.393 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.393 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.393 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:51:28.393 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.580795+0000", 2024-09-16T20:51:28.393 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.002874302 2024-09-16T20:51:28.394 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.394 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.394 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:28.394 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.584423+0000", 2024-09-16T20:51:28.394 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0036280010000000001 2024-09-16T20:51:28.394 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.394 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.394 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:51:28.394 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.584465+0000", 2024-09-16T20:51:28.394 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2590999999999999e-05 2024-09-16T20:51:28.394 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:28.394 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:28.394 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:28.394 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.584500+0000", 2024-09-16T20:51:28.395 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4570999999999998e-05 2024-09-16T20:51:28.395 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:28.395 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:28.395 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:28.395 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:28.395 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:28.395 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:28.395 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T20:51:28.397 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:51:28.398 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:28.395+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:28.497 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:28.496+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:28.505 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:28.506 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:51:28.506 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:51:28.506 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:28.514 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:51:28.519 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:51:28.520 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:51:28.537 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:51:28.537 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:51:28.554 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:51:28.554 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:51:28.571 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:51:28.571 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:51:28.588 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:51:28.589 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:51:28.599 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:28.597+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:28.606 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:51:28.606 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:51:28.624 INFO:teuthology.orchestra.run.smithi029.stderr:nodeep-scrub is unset 2024-09-16T20:51:28.624 INFO:teuthology.orchestra.run.smithi029.stderr:clear 3.1b pg_upmap mapping 2024-09-16T20:51:28.636 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:28.636 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:28.649 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:51:28.734 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:28.733+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:28.797 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:28.798 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:28.800 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:28.798+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:28.806 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:51:28.901 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:28.900+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:28.954 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:28.954 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:28.962 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:51:29.002 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:51:29.001+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:51:29.070 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:29.070 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:29.078 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:51:29.122 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:29.121+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:29.185 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:29.186 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:29.195 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:51:29.203 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:29.202+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:29.304 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:29.303+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:29.343 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:29.343 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:29.351 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:51:29.405 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:29.404+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:29.499 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:29.499 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:29.505 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:29.504+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:29.508 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:51:29.606 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:29.605+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:29.660 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:29.660 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:29.668 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:51:29.713 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:29.712+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:29.789 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:29.789 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:29.798 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:51:29.808 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:29.807+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:29.909 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:51:29.908+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:51:29.946 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:29.947 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:29.955 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:51:30.039 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:51:30.038+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:51:30.104 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:30.104 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:30.111 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:30.109+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:30.112 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:51:30.212 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:30.210+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:30.261 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:30.261 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:30.269 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:51:30.313 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:30.311+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:30.414 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:30.412+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:30.416 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:30.416 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:30.425 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:51:30.515 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:30.513+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:30.532 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:30.532 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:30.541 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:51:30.618 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T20:51:30.625 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:30.624+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:30.646 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T20:51:30.689 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:30.689 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:30.698 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:51:30.716 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:30.715+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:30.756 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:51:30.756 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:51:30.774 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:51:30.774 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:51:30.792 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:51:30.793 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:51:30.809 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:51:30.810 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:51:30.817 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:51:30.815+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:51:30.826 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:51:30.827 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:51:30.843 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:51:30.843 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:51:30.846 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:30.846 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:51:30.846 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:51:30.846 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:51:30.846 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:30.854 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:51:30.939 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:51:30.938+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:51:31.018 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:31.018 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:51:31.018 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:51:31.018 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:51:31.018 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.018 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5286.0:10643 3.17 3.f6273fb7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-16T20:51:31.018 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:51:31.018 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 142.22528069099999, 2024-09-16T20:51:31.018 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.76672347200000002, 2024-09-16T20:51:31.018 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:31.018 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:31.018 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:31.019 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:31.019 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5286", 2024-09-16T20:51:31.019 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/4135433989", 2024-09-16T20:51:31.019 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 10643 2024-09-16T20:51:31.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.019 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:31.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.019 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:31.019 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:51:31.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:31.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.019 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:31.019 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:51:31.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:31.020 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.020 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.020 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:31.020 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790489+0000", 2024-09-16T20:51:31.020 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5910000000000001e-06 2024-09-16T20:51:31.020 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.020 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.020 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:31.020 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790493+0000", 2024-09-16T20:51:31.020 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2669999999999998e-06 2024-09-16T20:51:31.020 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.020 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.020 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:31.021 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790500+0000", 2024-09-16T20:51:31.021 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3780000000000003e-06 2024-09-16T20:51:31.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.021 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.021 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:31.021 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790505+0000", 2024-09-16T20:51:31.021 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1560000000000003e-06 2024-09-16T20:51:31.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.021 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.021 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:31.021 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790559+0000", 2024-09-16T20:51:31.021 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3494e-05 2024-09-16T20:51:31.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.022 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.022 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:31.022 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790566+0000", 2024-09-16T20:51:31.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7649999999999993e-06 2024-09-16T20:51:31.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.022 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.022 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:31.022 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:09.557198+0000", 2024-09-16T20:51:31.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.76663136200000004 2024-09-16T20:51:31.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.022 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.022 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:31.022 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:09.557211+0000", 2024-09-16T20:51:31.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3458999999999999e-05 2024-09-16T20:51:31.022 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:31.023 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:31.023 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:31.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.023 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.023 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5512.0:450 3.2 3:44e0cffa:::rbd_header.110ef0d7a535:head [watch reconnect cookie 140263298018912 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:51:31.023 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:51:31.023 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 114.45325777799999, 2024-09-16T20:51:31.023 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0084581939999999, 2024-09-16T20:51:31.023 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:31.023 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:31.023 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:31.023 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:31.023 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5512", 2024-09-16T20:51:31.023 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:51:31.024 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 450 2024-09-16T20:51:31.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.024 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:31.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.024 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:31.024 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:51:31.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:31.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.024 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:31.024 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:51:31.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:31.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:31.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562512+0000", 2024-09-16T20:51:31.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.009e-06 2024-09-16T20:51:31.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:31.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562517+0000", 2024-09-16T20:51:31.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9339999999999997e-06 2024-09-16T20:51:31.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:31.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562530+0000", 2024-09-16T20:51:31.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3495e-05 2024-09-16T20:51:31.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.026 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:31.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562536+0000", 2024-09-16T20:51:31.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5690000000000004e-06 2024-09-16T20:51:31.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.026 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:31.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562591+0000", 2024-09-16T20:51:31.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5761999999999999e-05 2024-09-16T20:51:31.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.026 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:51:31.027 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562599+0000", 2024-09-16T20:51:31.027 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9179999999999994e-06 2024-09-16T20:51:31.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.027 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.027 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:31.027 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.565971+0000", 2024-09-16T20:51:31.027 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003371546 2024-09-16T20:51:31.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.027 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.027 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:31.027 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566019+0000", 2024-09-16T20:51:31.027 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7849000000000002e-05 2024-09-16T20:51:31.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.027 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.027 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:31.028 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.569382+0000", 2024-09-16T20:51:31.028 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.003362833 2024-09-16T20:51:31.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.028 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.028 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:31.028 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.569529+0000", 2024-09-16T20:51:31.028 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000147823 2024-09-16T20:51:31.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.028 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.028 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:31.028 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570969+0000", 2024-09-16T20:51:31.028 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001439456 2024-09-16T20:51:31.028 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:31.029 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:31.029 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:31.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.029 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:14996 3.2 3:44e0cffa:::rbd_header.110ef0d7a535:head [watch reconnect cookie 140213436269840 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:51:31.029 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:51:31.029 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 114.453209366, 2024-09-16T20:51:31.029 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.008813943, 2024-09-16T20:51:31.029 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:31.029 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:31.029 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:31.029 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:31.029 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:31.029 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:31.030 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 14996 2024-09-16T20:51:31.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.030 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:31.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.030 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:31.030 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:51:31.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:31.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.030 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:31.030 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:51:31.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:31.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.030 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:31.030 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562560+0000", 2024-09-16T20:51:31.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.06e-07 2024-09-16T20:51:31.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.031 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.031 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:31.031 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562563+0000", 2024-09-16T20:51:31.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.495e-06 2024-09-16T20:51:31.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.031 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.031 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:31.031 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562571+0000", 2024-09-16T20:51:31.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1950000000000003e-06 2024-09-16T20:51:31.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.031 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.031 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:31.031 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562579+0000", 2024-09-16T20:51:31.032 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2649999999999999e-06 2024-09-16T20:51:31.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.032 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:31.032 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562673+0000", 2024-09-16T20:51:31.032 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4118999999999998e-05 2024-09-16T20:51:31.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.032 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:51:31.032 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562677+0000", 2024-09-16T20:51:31.032 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5179999999999998e-06 2024-09-16T20:51:31.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.032 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:31.033 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566461+0000", 2024-09-16T20:51:31.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0037837259999999998 2024-09-16T20:51:31.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.033 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.033 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:31.033 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566468+0000", 2024-09-16T20:51:31.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3350000000000001e-06 2024-09-16T20:51:31.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.033 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.033 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:31.033 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570306+0000", 2024-09-16T20:51:31.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.003837997 2024-09-16T20:51:31.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.033 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.034 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:31.034 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570392+0000", 2024-09-16T20:51:31.034 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5872999999999996e-05 2024-09-16T20:51:31.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.034 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.034 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:31.034 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571373+0000", 2024-09-16T20:51:31.034 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00098071399999999993 2024-09-16T20:51:31.034 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:31.034 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:31.034 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:31.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.034 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.035 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5255.0:1931 3.2 3:47882819:::rbd_data.110eb5f95afc.0000000000000009:head [read 217088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:51:31.035 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:51:31.035 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 114.45317061599999, 2024-09-16T20:51:31.035 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0082752880000001, 2024-09-16T20:51:31.035 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:31.035 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:31.035 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:31.035 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:31.035 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5255", 2024-09-16T20:51:31.035 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:51:31.035 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1931 2024-09-16T20:51:31.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.035 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:31.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.035 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:31.035 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:51:31.036 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:31.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.036 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:31.036 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:51:31.036 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:31.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.036 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:31.036 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:51:31.036 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9699999999999996e-07 2024-09-16T20:51:31.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.036 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:31.037 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562601+0000", 2024-09-16T20:51:31.037 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8600000000000001e-06 2024-09-16T20:51:31.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.037 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.037 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:31.037 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562608+0000", 2024-09-16T20:51:31.037 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9e-06 2024-09-16T20:51:31.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.037 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.037 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:31.037 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562615+0000", 2024-09-16T20:51:31.037 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8179999999999998e-06 2024-09-16T20:51:31.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.037 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.037 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:31.038 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562703+0000", 2024-09-16T20:51:31.038 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7811000000000003e-05 2024-09-16T20:51:31.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.038 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.038 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:51:31.038 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562706+0000", 2024-09-16T20:51:31.038 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4920000000000002e-06 2024-09-16T20:51:31.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.038 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.038 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:31.038 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566685+0000", 2024-09-16T20:51:31.038 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0039792780000000002 2024-09-16T20:51:31.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.038 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.038 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:31.039 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566693+0000", 2024-09-16T20:51:31.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4730000000000003e-06 2024-09-16T20:51:31.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.039 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:31.039 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570737+0000", 2024-09-16T20:51:31.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0040436749999999 2024-09-16T20:51:31.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.039 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:31.039 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570873+0000", 2024-09-16T20:51:31.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000136484 2024-09-16T20:51:31.039 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:31.039 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:31.039 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:31.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.040 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.040 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5255.0:1939 3.2 3:47882819:::rbd_data.110eb5f95afc.0000000000000009:head [read 233472~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:51:31.040 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:51:31.040 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 114.45301575000001, 2024-09-16T20:51:31.040 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0084657589999999, 2024-09-16T20:51:31.040 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:31.040 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:31.040 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:31.040 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:31.040 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5255", 2024-09-16T20:51:31.040 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:51:31.040 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1939 2024-09-16T20:51:31.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.041 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:31.041 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.041 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:31.041 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:51:31.041 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:31.041 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.041 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.041 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:31.041 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:51:31.041 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:31.041 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.041 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.041 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:31.041 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:51:31.041 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5199999999999997e-07 2024-09-16T20:51:31.042 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.042 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.042 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:31.042 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562757+0000", 2024-09-16T20:51:31.042 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4979999999999998e-06 2024-09-16T20:51:31.042 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.042 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.042 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:31.042 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562764+0000", 2024-09-16T20:51:31.042 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1470000000000004e-06 2024-09-16T20:51:31.042 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.042 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.042 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:31.042 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562768+0000", 2024-09-16T20:51:31.042 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7929999999999999e-06 2024-09-16T20:51:31.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.043 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.043 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:31.043 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562805+0000", 2024-09-16T20:51:31.043 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7477e-05 2024-09-16T20:51:31.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.043 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.043 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting_for_map not empty", 2024-09-16T20:51:31.043 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562810+0000", 2024-09-16T20:51:31.043 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0819999999999998e-06 2024-09-16T20:51:31.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.044 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.044 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:31.044 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566717+0000", 2024-09-16T20:51:31.044 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0039072600000000001 2024-09-16T20:51:31.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.044 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.044 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:31.044 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566724+0000", 2024-09-16T20:51:31.044 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4880000000000004e-06 2024-09-16T20:51:31.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.044 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.045 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:31.045 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571009+0000", 2024-09-16T20:51:31.045 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.004284677 2024-09-16T20:51:31.045 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.045 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.045 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:31.045 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571218+0000", 2024-09-16T20:51:31.045 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00020978500000000001 2024-09-16T20:51:31.045 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:31.045 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:31.045 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:31.045 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.045 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.046 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:11021 3.1f 3:fee2d1dd:::rbd_data.110ea93691bb.000000000000000e:23 [read 960512~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:31.046 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.693837+0000", 2024-09-16T20:51:31.046 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 100.321931205, 2024-09-16T20:51:31.046 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057155138000000001, 2024-09-16T20:51:31.046 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:31.046 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:31.046 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:31.046 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:31.046 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:31.046 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:31.046 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 11021 2024-09-16T20:51:31.046 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.046 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:31.047 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.047 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:31.047 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693837+0000", 2024-09-16T20:51:31.047 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:31.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.047 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.047 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:31.047 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693837+0000", 2024-09-16T20:51:31.047 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:31.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.047 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.047 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:31.047 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699566+0000", 2024-09-16T20:51:31.048 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057283639999999997 2024-09-16T20:51:31.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.048 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.048 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:31.048 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699571+0000", 2024-09-16T20:51:31.048 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.604e-06 2024-09-16T20:51:31.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.048 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.048 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:31.048 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699583+0000", 2024-09-16T20:51:31.048 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1372999999999999e-05 2024-09-16T20:51:31.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.048 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.049 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:31.049 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699591+0000", 2024-09-16T20:51:31.049 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5259999999999993e-06 2024-09-16T20:51:31.049 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.049 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.049 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:31.049 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750674+0000", 2024-09-16T20:51:31.049 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051083154999999998 2024-09-16T20:51:31.049 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.049 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.049 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:31.049 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750745+0000", 2024-09-16T20:51:31.049 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0933999999999995e-05 2024-09-16T20:51:31.050 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.050 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.050 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:31.050 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750992+0000", 2024-09-16T20:51:31.050 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00024718199999999998 2024-09-16T20:51:31.050 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:31.050 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:31.050 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:31.050 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.050 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.050 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8792 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2609152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:31.050 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:51:31.050 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 89.958471488000001, 2024-09-16T20:51:31.050 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057003524999999999, 2024-09-16T20:51:31.051 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:31.051 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:31.051 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:31.051 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:31.051 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:31.051 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:31.051 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8792 2024-09-16T20:51:31.051 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.051 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:31.051 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.051 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:31.051 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:51:31.051 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:31.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.052 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.052 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:31.052 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:51:31.052 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:31.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.052 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.052 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:31.052 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:51:31.052 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0400000000000002e-07 2024-09-16T20:51:31.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.052 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.052 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:31.052 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057299+0000", 2024-09-16T20:51:31.053 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7460000000000001e-06 2024-09-16T20:51:31.053 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.053 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.053 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:31.053 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057304+0000", 2024-09-16T20:51:31.053 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3210000000000004e-06 2024-09-16T20:51:31.053 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.053 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.053 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:31.053 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057310+0000", 2024-09-16T20:51:31.053 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4360000000000001e-06 2024-09-16T20:51:31.053 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.053 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.053 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:31.053 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114200+0000", 2024-09-16T20:51:31.054 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056889954 2024-09-16T20:51:31.054 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.054 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.054 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:31.054 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114243+0000", 2024-09-16T20:51:31.054 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2998999999999999e-05 2024-09-16T20:51:31.054 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.054 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.054 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:31.054 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114300+0000", 2024-09-16T20:51:31.054 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7765e-05 2024-09-16T20:51:31.054 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:31.054 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:31.054 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:31.054 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.055 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.055 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8794 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2351104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:31.055 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057327+0000", 2024-09-16T20:51:31.055 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 89.958441164000007, 2024-09-16T20:51:31.055 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057101130999999999, 2024-09-16T20:51:31.055 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:31.055 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:31.055 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:31.055 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:31.055 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:31.055 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:31.055 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8794 2024-09-16T20:51:31.055 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.056 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:31.056 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.056 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:31.056 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057327+0000", 2024-09-16T20:51:31.056 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:31.056 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.056 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.056 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:31.056 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057327+0000", 2024-09-16T20:51:31.056 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:31.056 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.056 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.056 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:31.057 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057328+0000", 2024-09-16T20:51:31.057 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7399999999999999e-07 2024-09-16T20:51:31.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.057 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.057 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:31.057 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057330+0000", 2024-09-16T20:51:31.057 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2199999999999999e-06 2024-09-16T20:51:31.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.057 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.057 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:31.057 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057336+0000", 2024-09-16T20:51:31.057 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1240000000000002e-06 2024-09-16T20:51:31.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.058 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.058 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:31.058 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057341+0000", 2024-09-16T20:51:31.058 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8470000000000003e-06 2024-09-16T20:51:31.058 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.058 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.058 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:31.058 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114334+0000", 2024-09-16T20:51:31.058 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056993267 2024-09-16T20:51:31.058 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.058 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.058 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:31.058 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114376+0000", 2024-09-16T20:51:31.058 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1638999999999999e-05 2024-09-16T20:51:31.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.059 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.059 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:31.059 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114428+0000", 2024-09-16T20:51:31.059 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2660000000000001e-05 2024-09-16T20:51:31.059 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:31.059 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:31.059 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:31.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.059 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.059 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8798 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2297856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:31.059 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:51:31.059 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 89.958421436999998, 2024-09-16T20:51:31.059 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057277041000000001, 2024-09-16T20:51:31.060 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:31.060 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:31.060 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:31.060 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:31.060 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:31.060 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:31.060 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8798 2024-09-16T20:51:31.060 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.060 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:31.060 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.060 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:31.060 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:51:31.060 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:31.061 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.061 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.061 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:31.061 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:51:31.061 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:31.061 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.061 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.061 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:31.061 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:51:31.061 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2800000000000003e-07 2024-09-16T20:51:31.061 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.061 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.061 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:31.062 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057354+0000", 2024-09-16T20:51:31.062 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7780000000000002e-06 2024-09-16T20:51:31.062 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.062 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.062 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:31.062 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057360+0000", 2024-09-16T20:51:31.062 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0419999999999999e-06 2024-09-16T20:51:31.062 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.062 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.062 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:31.062 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057365+0000", 2024-09-16T20:51:31.062 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7929999999999997e-06 2024-09-16T20:51:31.062 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.062 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.063 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:31.063 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114460+0000", 2024-09-16T20:51:31.063 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057094662999999997 2024-09-16T20:51:31.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.063 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.063 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:31.063 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114532+0000", 2024-09-16T20:51:31.063 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2881999999999998e-05 2024-09-16T20:51:31.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.063 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.063 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:31.063 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114624+0000", 2024-09-16T20:51:31.063 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1555e-05 2024-09-16T20:51:31.064 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:31.064 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:31.064 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:31.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.064 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8800 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2316288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:31.064 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:51:31.064 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 89.958397497999997, 2024-09-16T20:51:31.064 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.05746946, 2024-09-16T20:51:31.064 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:31.064 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:31.064 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:31.064 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:31.065 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:31.065 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:31.065 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8800 2024-09-16T20:51:31.065 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.065 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:31.065 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.065 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:31.065 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:51:31.065 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:31.065 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.065 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.065 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:31.065 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:51:31.066 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:31.066 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.066 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.066 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:31.066 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:51:31.066 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5199999999999998e-07 2024-09-16T20:51:31.066 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.066 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.066 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:31.066 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057373+0000", 2024-09-16T20:51:31.066 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.79e-06 2024-09-16T20:51:31.066 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.066 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.066 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:31.067 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057379+0000", 2024-09-16T20:51:31.067 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6699999999999999e-06 2024-09-16T20:51:31.067 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.067 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.067 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:31.067 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057386+0000", 2024-09-16T20:51:31.067 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3170000000000004e-06 2024-09-16T20:51:31.067 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.067 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.067 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:31.067 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114655+0000", 2024-09-16T20:51:31.067 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057269420000000001 2024-09-16T20:51:31.067 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.067 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.068 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:31.068 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114734+0000", 2024-09-16T20:51:31.068 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8084000000000007e-05 2024-09-16T20:51:31.068 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.068 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.068 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:31.068 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114840+0000", 2024-09-16T20:51:31.068 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000106827 2024-09-16T20:51:31.068 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:31.068 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:31.068 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:31.068 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.068 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.069 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8801 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2334720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:31.069 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:51:31.069 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 89.958376466999994, 2024-09-16T20:51:31.069 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057594682000000001, 2024-09-16T20:51:31.069 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:31.069 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:31.069 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:31.069 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:31.069 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:31.069 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:31.069 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8801 2024-09-16T20:51:31.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.069 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:31.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.070 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:31.070 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:51:31.070 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:31.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.070 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:31.070 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:51:31.070 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:31.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.070 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:31.071 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:51:31.071 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6899999999999998e-07 2024-09-16T20:51:31.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.071 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.071 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:31.071 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057394+0000", 2024-09-16T20:51:31.071 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8300000000000001e-06 2024-09-16T20:51:31.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.071 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.071 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:31.071 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057400+0000", 2024-09-16T20:51:31.071 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6500000000000001e-06 2024-09-16T20:51:31.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.072 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.072 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:31.072 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057404+0000", 2024-09-16T20:51:31.072 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4669999999999998e-06 2024-09-16T20:51:31.072 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.072 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.072 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:31.072 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114874+0000", 2024-09-16T20:51:31.072 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057469602000000002 2024-09-16T20:51:31.072 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.072 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.072 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:31.072 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114933+0000", 2024-09-16T20:51:31.072 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9500000000000003e-05 2024-09-16T20:51:31.073 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.073 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.073 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:31.073 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114987+0000", 2024-09-16T20:51:31.073 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3263999999999997e-05 2024-09-16T20:51:31.073 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:31.073 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:31.073 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:31.073 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.073 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.073 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8802 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2353152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:31.073 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:51:31.073 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 89.958327781999998, 2024-09-16T20:51:31.074 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057714079000000001, 2024-09-16T20:51:31.074 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:31.074 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:31.074 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:31.074 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:31.074 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:31.074 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:31.074 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8802 2024-09-16T20:51:31.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.074 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:31.074 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.074 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:31.074 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:51:31.074 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:31.075 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.075 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.075 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:31.075 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:51:31.075 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:31.075 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.075 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.075 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:31.075 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:51:31.075 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4999999999999998e-07 2024-09-16T20:51:31.075 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.075 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.075 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:31.076 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057446+0000", 2024-09-16T20:51:31.076 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4240000000000003e-06 2024-09-16T20:51:31.076 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.076 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.076 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:31.076 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057452+0000", 2024-09-16T20:51:31.076 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8379999999999996e-06 2024-09-16T20:51:31.076 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.077 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.077 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:31.077 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057460+0000", 2024-09-16T20:51:31.077 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9629999999999998e-06 2024-09-16T20:51:31.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.077 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.077 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:31.077 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115015+0000", 2024-09-16T20:51:31.077 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057554389999999997 2024-09-16T20:51:31.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.077 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.077 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:31.077 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115082+0000", 2024-09-16T20:51:31.077 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7733999999999999e-05 2024-09-16T20:51:31.078 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.078 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.078 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:31.078 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115155+0000", 2024-09-16T20:51:31.078 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2280000000000006e-05 2024-09-16T20:51:31.078 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:31.078 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:31.078 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:31.078 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.078 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.078 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8804 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2555904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:31.078 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:51:31.078 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 89.958301465999995, 2024-09-16T20:51:31.079 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057833862, 2024-09-16T20:51:31.079 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:31.079 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:31.079 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:31.079 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:31.079 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:31.079 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:31.079 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8804 2024-09-16T20:51:31.079 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.079 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:31.079 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.079 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:31.079 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:51:31.080 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:31.080 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.080 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.080 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:31.080 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:51:31.080 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:31.080 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.080 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.080 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:31.080 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:51:31.080 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7899999999999999e-07 2024-09-16T20:51:31.080 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.080 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.081 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:31.081 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057469+0000", 2024-09-16T20:51:31.081 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.001e-06 2024-09-16T20:51:31.081 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.081 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.081 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:31.081 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057475+0000", 2024-09-16T20:51:31.081 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7629999999999997e-06 2024-09-16T20:51:31.081 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.081 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.081 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:31.081 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057480+0000", 2024-09-16T20:51:31.081 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8069999999999999e-06 2024-09-16T20:51:31.082 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.082 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.082 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:31.082 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115183+0000", 2024-09-16T20:51:31.082 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057703364 2024-09-16T20:51:31.082 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.082 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.082 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:31.082 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115227+0000", 2024-09-16T20:51:31.082 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3903999999999998e-05 2024-09-16T20:51:31.082 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.082 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.082 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:31.083 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115301+0000", 2024-09-16T20:51:31.083 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3543999999999999e-05 2024-09-16T20:51:31.083 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:31.083 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:31.083 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:31.083 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.083 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.083 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8805 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2574336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:31.083 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057487+0000", 2024-09-16T20:51:31.083 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 89.958281138000004, 2024-09-16T20:51:31.083 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058021027000000003, 2024-09-16T20:51:31.084 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:31.084 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:31.084 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:31.084 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:31.084 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:31.084 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:31.084 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8805 2024-09-16T20:51:31.084 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.084 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:31.084 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.084 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:31.084 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057487+0000", 2024-09-16T20:51:31.085 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:31.085 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.085 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.085 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:31.085 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057487+0000", 2024-09-16T20:51:31.085 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:31.085 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.085 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.085 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:31.085 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057488+0000", 2024-09-16T20:51:31.085 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5199999999999998e-07 2024-09-16T20:51:31.085 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.085 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.086 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:31.086 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057490+0000", 2024-09-16T20:51:31.086 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.939e-06 2024-09-16T20:51:31.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.086 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.086 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:31.086 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057495+0000", 2024-09-16T20:51:31.086 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6439999999999997e-06 2024-09-16T20:51:31.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.086 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.086 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:31.086 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057500+0000", 2024-09-16T20:51:31.086 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8119999999999998e-06 2024-09-16T20:51:31.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.087 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.087 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:31.087 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115341+0000", 2024-09-16T20:51:31.087 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057841341999999997 2024-09-16T20:51:31.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.087 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.087 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:31.087 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115422+0000", 2024-09-16T20:51:31.087 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1003999999999999e-05 2024-09-16T20:51:31.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.087 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.087 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:31.087 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115508+0000", 2024-09-16T20:51:31.088 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5933999999999994e-05 2024-09-16T20:51:31.088 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:31.088 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:31.088 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:31.088 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.088 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.088 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8807 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2592768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:31.088 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057505+0000", 2024-09-16T20:51:31.088 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 89.958262927999996, 2024-09-16T20:51:31.088 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058186633000000001, 2024-09-16T20:51:31.088 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:31.088 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:31.088 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:31.089 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:31.089 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:31.089 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:31.089 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8807 2024-09-16T20:51:31.089 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.089 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:31.089 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.089 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:31.089 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057505+0000", 2024-09-16T20:51:31.089 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:31.089 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.089 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.089 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:31.089 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057505+0000", 2024-09-16T20:51:31.090 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:31.090 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.090 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.090 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:31.090 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057506+0000", 2024-09-16T20:51:31.090 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.27e-07 2024-09-16T20:51:31.090 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.090 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.090 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:31.090 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057508+0000", 2024-09-16T20:51:31.090 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7400000000000001e-06 2024-09-16T20:51:31.090 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.090 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.090 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:31.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057513+0000", 2024-09-16T20:51:31.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.468e-06 2024-09-16T20:51:31.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:31.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057517+0000", 2024-09-16T20:51:31.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4329999999999997e-06 2024-09-16T20:51:31.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:31.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115538+0000", 2024-09-16T20:51:31.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058020355000000003 2024-09-16T20:51:31.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.092 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.092 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:31.092 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115616+0000", 2024-09-16T20:51:31.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7998999999999996e-05 2024-09-16T20:51:31.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.092 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.092 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:31.092 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115692+0000", 2024-09-16T20:51:31.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6310999999999995e-05 2024-09-16T20:51:31.092 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:31.092 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:31.092 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:31.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.093 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.093 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8808 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2611200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:31.093 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:51:31.093 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 89.958245667, 2024-09-16T20:51:31.093 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058354046, 2024-09-16T20:51:31.093 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:31.093 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:31.093 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:31.093 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:31.093 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:31.093 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:31.093 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8808 2024-09-16T20:51:31.093 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.094 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:31.094 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.094 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:31.094 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:51:31.094 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:31.094 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.094 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.094 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:31.094 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:51:31.094 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:31.094 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.094 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.094 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:31.094 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:51:31.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2500000000000001e-07 2024-09-16T20:51:31.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.095 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:31.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057525+0000", 2024-09-16T20:51:31.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5549999999999999e-06 2024-09-16T20:51:31.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.095 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:31.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057530+0000", 2024-09-16T20:51:31.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5489999999999997e-06 2024-09-16T20:51:31.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.095 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:31.096 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057535+0000", 2024-09-16T20:51:31.096 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5709999999999999e-06 2024-09-16T20:51:31.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.096 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.096 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:31.096 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115722+0000", 2024-09-16T20:51:31.096 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058187323999999999 2024-09-16T20:51:31.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.096 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.096 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:31.096 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115799+0000", 2024-09-16T20:51:31.096 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6445000000000004e-05 2024-09-16T20:51:31.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.097 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.097 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:31.097 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115877+0000", 2024-09-16T20:51:31.097 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8276999999999996e-05 2024-09-16T20:51:31.097 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:31.097 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:31.097 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:31.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.097 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.097 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:2182 3.17 3:edb5e5cd:::rbd_data.110e61479aee.0000000000000014:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write 0~5120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-16T20:51:31.097 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:22.734609+0000", 2024-09-16T20:51:31.098 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 68.281159419999994, 2024-09-16T20:51:31.098 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.124890894, 2024-09-16T20:51:31.098 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:31.098 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:31.098 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:31.098 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:31.098 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:51:31.098 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:31.098 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 2182 2024-09-16T20:51:31.098 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.098 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:31.098 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.098 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:31.099 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734609+0000", 2024-09-16T20:51:31.099 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:31.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.099 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.099 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:31.099 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734609+0000", 2024-09-16T20:51:31.099 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:31.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.099 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.099 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:31.099 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734611+0000", 2024-09-16T20:51:31.099 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9599999999999999e-06 2024-09-16T20:51:31.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.100 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.100 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:31.100 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734628+0000", 2024-09-16T20:51:31.100 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.715e-05 2024-09-16T20:51:31.100 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.100 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.100 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:31.100 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734656+0000", 2024-09-16T20:51:31.100 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7759999999999998e-05 2024-09-16T20:51:31.100 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.100 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.100 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:31.100 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734669+0000", 2024-09-16T20:51:31.101 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3315999999999999e-05 2024-09-16T20:51:31.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.101 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:31.101 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734826+0000", 2024-09-16T20:51:31.101 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015668499999999999 2024-09-16T20:51:31.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.101 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:31.101 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.735048+0000", 2024-09-16T20:51:31.101 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00022179 2024-09-16T20:51:31.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.102 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:31.102 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.735248+0000", 2024-09-16T20:51:31.102 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00020010399999999999 2024-09-16T20:51:31.102 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.102 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.102 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:51:31.102 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.736252+0000", 2024-09-16T20:51:31.102 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010039739999999999 2024-09-16T20:51:31.102 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.102 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.102 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:31.102 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.859340+0000", 2024-09-16T20:51:31.102 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.12308846599999999 2024-09-16T20:51:31.102 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.103 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.103 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:51:31.103 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.859431+0000", 2024-09-16T20:51:31.103 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1099000000000003e-05 2024-09-16T20:51:31.103 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.103 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.103 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:31.103 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.859500+0000", 2024-09-16T20:51:31.103 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8590000000000006e-05 2024-09-16T20:51:31.103 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:31.103 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:31.103 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:31.103 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.103 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.104 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:29157 3.1b 3:dbe69130:::journal.110e1b19a241:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e167)", 2024-09-16T20:51:31.104 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:54.663540+0000", 2024-09-16T20:51:31.104 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.352228082000003, 2024-09-16T20:51:31.104 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.92912227999999997, 2024-09-16T20:51:31.104 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:31.104 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:31.104 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:31.104 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:31.104 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:31.104 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:31.104 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 29157 2024-09-16T20:51:31.104 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.104 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:31.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.105 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:31.105 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663540+0000", 2024-09-16T20:51:31.105 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:31.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.105 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:31.105 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663540+0000", 2024-09-16T20:51:31.105 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:31.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.105 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:31.105 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663541+0000", 2024-09-16T20:51:31.105 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1680000000000001e-06 2024-09-16T20:51:31.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.106 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:31.106 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663546+0000", 2024-09-16T20:51:31.106 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7129999999999996e-06 2024-09-16T20:51:31.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.106 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:31.106 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663558+0000", 2024-09-16T20:51:31.106 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1347e-05 2024-09-16T20:51:31.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.106 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:31.107 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663563+0000", 2024-09-16T20:51:31.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1250000000000001e-06 2024-09-16T20:51:31.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.107 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:31.107 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663612+0000", 2024-09-16T20:51:31.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9665e-05 2024-09-16T20:51:31.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.107 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:31.107 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663620+0000", 2024-09-16T20:51:31.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8439999999999998e-06 2024-09-16T20:51:31.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:31.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:55.592525+0000", 2024-09-16T20:51:31.108 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.92890456700000001 2024-09-16T20:51:31.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:31.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:55.592663+0000", 2024-09-16T20:51:31.108 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013785100000000001 2024-09-16T20:51:31.108 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:31.108 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:31.108 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:31.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.109 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:30229 3.16 3:6f5476df:::rbd_header.110e25db1f60:head [watch reconnect cookie 140213637350016 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e181)", 2024-09-16T20:51:31.109 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:51:08.603124+0000", 2024-09-16T20:51:31.109 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.412644437000001, 2024-09-16T20:51:31.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.97765519899999997, 2024-09-16T20:51:31.109 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:31.109 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:31.109 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:31.109 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:31.109 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:31.109 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:31.109 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30229 2024-09-16T20:51:31.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.109 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:31.109 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.110 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:31.110 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603124+0000", 2024-09-16T20:51:31.110 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:31.110 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.110 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.110 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:31.110 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603124+0000", 2024-09-16T20:51:31.110 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:31.110 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.110 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.110 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:31.110 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603125+0000", 2024-09-16T20:51:31.110 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5090000000000001e-06 2024-09-16T20:51:31.110 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.111 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.111 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:31.111 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603131+0000", 2024-09-16T20:51:31.111 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9089999999999997e-06 2024-09-16T20:51:31.111 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.111 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.111 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:31.111 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603143+0000", 2024-09-16T20:51:31.111 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1416e-05 2024-09-16T20:51:31.111 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.111 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.111 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:31.111 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603151+0000", 2024-09-16T20:51:31.112 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7889999999999995e-06 2024-09-16T20:51:31.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.112 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:31.112 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603674+0000", 2024-09-16T20:51:31.112 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00052357 2024-09-16T20:51:31.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.112 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:31.112 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603684+0000", 2024-09-16T20:51:31.112 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0274e-05 2024-09-16T20:51:31.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.112 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:31.113 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.575380+0000", 2024-09-16T20:51:31.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.97169509799999998 2024-09-16T20:51:31.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.113 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.113 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:31.113 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.575591+0000", 2024-09-16T20:51:31.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00021126700000000001 2024-09-16T20:51:31.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.113 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.113 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:31.113 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.580779+0000", 2024-09-16T20:51:31.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0051883670000000002 2024-09-16T20:51:31.113 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:31.114 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:31.114 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:31.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.114 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.114 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:30230 3.16 3:6f5476df:::rbd_header.110e25db1f60:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e181)", 2024-09-16T20:51:31.114 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:51:08.603204+0000", 2024-09-16T20:51:31.114 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.412564489000001, 2024-09-16T20:51:31.114 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.98129583300000001, 2024-09-16T20:51:31.114 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:31.114 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:31.114 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:31.114 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:31.114 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:31.114 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:31.115 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30230 2024-09-16T20:51:31.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.115 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:31.115 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.115 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:31.115 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603204+0000", 2024-09-16T20:51:31.115 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:31.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.115 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.115 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:31.115 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603204+0000", 2024-09-16T20:51:31.115 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:31.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.115 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.116 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:31.116 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603205+0000", 2024-09-16T20:51:31.116 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4900000000000005e-07 2024-09-16T20:51:31.116 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.116 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.116 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:31.116 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603209+0000", 2024-09-16T20:51:31.116 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5210000000000004e-06 2024-09-16T20:51:31.116 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.116 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.116 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:31.116 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603222+0000", 2024-09-16T20:51:31.116 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2733e-05 2024-09-16T20:51:31.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.117 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:31.117 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603228+0000", 2024-09-16T20:51:31.117 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4520000000000002e-06 2024-09-16T20:51:31.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.117 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:31.117 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603719+0000", 2024-09-16T20:51:31.117 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00049103600000000003 2024-09-16T20:51:31.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.117 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:31.117 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603730+0000", 2024-09-16T20:51:31.118 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.023e-05 2024-09-16T20:51:31.118 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.118 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.118 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:31.118 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.577394+0000", 2024-09-16T20:51:31.118 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.97366440099999996 2024-09-16T20:51:31.118 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.118 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.118 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:31.118 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.577499+0000", 2024-09-16T20:51:31.118 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000104509 2024-09-16T20:51:31.118 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.118 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.119 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:31.119 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.577920+0000", 2024-09-16T20:51:31.119 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00042173699999999999 2024-09-16T20:51:31.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.119 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.119 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:51:31.119 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.580795+0000", 2024-09-16T20:51:31.119 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.002874302 2024-09-16T20:51:31.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.119 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.119 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:31.119 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.584423+0000", 2024-09-16T20:51:31.119 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0036280010000000001 2024-09-16T20:51:31.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.119 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.120 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:51:31.120 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.584465+0000", 2024-09-16T20:51:31.120 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2590999999999999e-05 2024-09-16T20:51:31.120 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:31.120 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:31.120 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:31.120 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.584500+0000", 2024-09-16T20:51:31.120 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4570999999999998e-05 2024-09-16T20:51:31.120 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:31.120 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:31.120 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:31.120 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:31.120 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:31.120 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:31.124 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:31.121+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:31.124 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:51:31.223 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:51:31.221+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:51:31.233 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:31.233 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:51:31.233 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:51:31.233 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:31.241 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:51:31.324 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:31.323+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:31.349 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:31.349 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:51:31.349 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:51:31.349 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:51:31.349 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:31.357 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:51:31.425 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:31.424+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:31.506 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:31.507 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:51:31.507 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:51:31.507 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:51:31.507 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.507 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5439 3.a 3:5789f5c3:::rbd_data.110ed9ca540.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 1181696~60416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:31.507 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:51:31.507 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 188.64009958299999, 2024-09-16T20:51:31.507 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10102045599999999, 2024-09-16T20:51:31.507 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:31.507 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:31.507 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:31.507 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:31.507 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:31.507 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:31.508 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5439 2024-09-16T20:51:31.508 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.508 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:31.508 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.508 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:31.508 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:51:31.508 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:31.508 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.508 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.508 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:31.508 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:51:31.508 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:31.508 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.509 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.509 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:31.509 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864711+0000", 2024-09-16T20:51:31.509 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7899999999999997e-07 2024-09-16T20:51:31.509 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.509 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.509 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:31.509 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864760+0000", 2024-09-16T20:51:31.509 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8803e-05 2024-09-16T20:51:31.509 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.509 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.509 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:31.509 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864769+0000", 2024-09-16T20:51:31.509 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0999999999999993e-06 2024-09-16T20:51:31.510 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.510 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.510 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:31.510 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864774+0000", 2024-09-16T20:51:31.510 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8919999999999999e-06 2024-09-16T20:51:31.510 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.510 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.510 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:31.510 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864815+0000", 2024-09-16T20:51:31.510 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1307000000000003e-05 2024-09-16T20:51:31.510 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.510 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.510 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:31.511 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864893+0000", 2024-09-16T20:51:31.511 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7581000000000001e-05 2024-09-16T20:51:31.511 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.511 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.511 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 1", 2024-09-16T20:51:31.511 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864996+0000", 2024-09-16T20:51:31.511 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102725 2024-09-16T20:51:31.511 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.511 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.511 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:31.511 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965442+0000", 2024-09-16T20:51:31.511 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100446012 2024-09-16T20:51:31.511 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.511 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.511 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:31.512 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965618+0000", 2024-09-16T20:51:31.512 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00017653100000000001 2024-09-16T20:51:31.512 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.512 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.512 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:31.512 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965682+0000", 2024-09-16T20:51:31.512 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3905999999999994e-05 2024-09-16T20:51:31.512 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.512 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.512 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:31.512 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965731+0000", 2024-09-16T20:51:31.512 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8819999999999997e-05 2024-09-16T20:51:31.512 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:31.513 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:31.513 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:31.513 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.513 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.513 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4600.0:5439 3.a e37/23 ondisk, result = 0)", 2024-09-16T20:51:31.513 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:51:31.513 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 188.63910343500001, 2024-09-16T20:51:31.513 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099742634999999996, 2024-09-16T20:51:31.513 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:31.513 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:31.513 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:31.513 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:31.513 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.514 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:31.514 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:51:31.514 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:31.514 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.514 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.514 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:31.514 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:51:31.514 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:31.514 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.514 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.514 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:31.514 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:51:31.514 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0900000000000001e-07 2024-09-16T20:51:31.514 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.515 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.515 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:31.515 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865711+0000", 2024-09-16T20:51:31.515 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3739999999999998e-06 2024-09-16T20:51:31.515 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.515 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.515 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:31.515 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965343+0000", 2024-09-16T20:51:31.515 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099632526999999999 2024-09-16T20:51:31.515 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.515 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.515 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:31.515 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965357+0000", 2024-09-16T20:51:31.515 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3713999999999999e-05 2024-09-16T20:51:31.516 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.516 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.516 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:31.516 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965426+0000", 2024-09-16T20:51:31.516 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8745999999999995e-05 2024-09-16T20:51:31.516 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.516 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.516 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:31.516 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965434+0000", 2024-09-16T20:51:31.516 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6910000000000003e-06 2024-09-16T20:51:31.516 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.516 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.516 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:31.516 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965449+0000", 2024-09-16T20:51:31.517 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4874e-05 2024-09-16T20:51:31.517 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:31.517 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:31.517 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:31.517 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.517 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.517 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4882.0:2508 3.12 3:4992771f:::rbd_object_map.110e7c8851c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T20:51:31.517 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:51:31.517 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 165.02329988700001, 2024-09-16T20:51:31.517 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.125007011, 2024-09-16T20:51:31.517 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:31.517 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:31.517 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:31.518 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:31.518 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4882", 2024-09-16T20:51:31.518 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2436923223", 2024-09-16T20:51:31.518 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2508 2024-09-16T20:51:31.518 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.518 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:31.518 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.518 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:31.518 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:51:31.518 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:31.519 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.519 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.519 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:31.519 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:51:31.519 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:31.519 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.519 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.519 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:31.519 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481511+0000", 2024-09-16T20:51:31.519 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0699999999999996e-07 2024-09-16T20:51:31.519 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.519 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.519 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:31.519 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481515+0000", 2024-09-16T20:51:31.519 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0609999999999997e-06 2024-09-16T20:51:31.520 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.520 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.520 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:31.520 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481523+0000", 2024-09-16T20:51:31.520 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1310000000000006e-06 2024-09-16T20:51:31.520 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.520 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.520 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:31.520 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481528+0000", 2024-09-16T20:51:31.520 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4900000000000002e-06 2024-09-16T20:51:31.520 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.520 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.520 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:31.521 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481565+0000", 2024-09-16T20:51:31.521 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7066000000000001e-05 2024-09-16T20:51:31.521 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.521 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.521 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:31.521 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481628+0000", 2024-09-16T20:51:31.521 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3118000000000001e-05 2024-09-16T20:51:31.521 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.521 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.521 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 2", 2024-09-16T20:51:31.521 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481822+0000", 2024-09-16T20:51:31.521 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019446200000000001 2024-09-16T20:51:31.521 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.522 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.522 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:31.522 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605988+0000", 2024-09-16T20:51:31.522 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.124165888 2024-09-16T20:51:31.522 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.522 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.522 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:31.522 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606427+0000", 2024-09-16T20:51:31.522 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00043854899999999999 2024-09-16T20:51:31.522 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.522 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.522 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:31.522 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606476+0000", 2024-09-16T20:51:31.522 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8971999999999998e-05 2024-09-16T20:51:31.522 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.523 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.523 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:31.523 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606517+0000", 2024-09-16T20:51:31.523 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1467000000000001e-05 2024-09-16T20:51:31.523 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:31.523 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:31.523 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:31.523 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.523 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.523 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4882.0:2508 3.12 e57/55 ondisk, result = 0)", 2024-09-16T20:51:31.523 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:51:31.523 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 165.022311387, 2024-09-16T20:51:31.524 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12349711400000001, 2024-09-16T20:51:31.524 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:31.524 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:31.524 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:31.524 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:31.524 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.524 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:31.524 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:51:31.524 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:31.524 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.524 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.524 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:31.524 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:51:31.524 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:31.524 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.525 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.525 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:31.525 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:51:31.525 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.06e-07 2024-09-16T20:51:31.525 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.525 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.525 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:31.525 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482502+0000", 2024-09-16T20:51:31.525 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.01e-06 2024-09-16T20:51:31.525 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.525 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.525 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:31.525 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605772+0000", 2024-09-16T20:51:31.525 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.123269687 2024-09-16T20:51:31.526 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.526 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.526 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:31.526 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605791+0000", 2024-09-16T20:51:31.526 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8912000000000001e-05 2024-09-16T20:51:31.526 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.526 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.526 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:31.526 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605974+0000", 2024-09-16T20:51:31.526 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000182836 2024-09-16T20:51:31.526 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.526 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.526 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:31.526 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605982+0000", 2024-09-16T20:51:31.526 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1699999999999997e-06 2024-09-16T20:51:31.527 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.527 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.527 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:31.527 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605996+0000", 2024-09-16T20:51:31.527 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3893000000000001e-05 2024-09-16T20:51:31.527 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:31.527 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:31.527 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:31.527 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.527 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.527 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4882.0:2512 3.f 3:f4d895b7:::rbd_data.110e7c8851c.0000000000000000:head [read 16384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T20:51:31.527 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:51:31.527 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 165.021874357, 2024-09-16T20:51:31.527 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12318483, 2024-09-16T20:51:31.528 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:31.528 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:31.528 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:31.528 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:31.528 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4882", 2024-09-16T20:51:31.528 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2436923223", 2024-09-16T20:51:31.528 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2512 2024-09-16T20:51:31.528 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.528 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:31.528 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.528 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:31.528 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:51:31.528 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:31.528 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.528 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.529 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:31.529 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:51:31.529 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:31.529 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.529 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.529 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:31.529 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:51:31.529 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3499999999999995e-07 2024-09-16T20:51:31.529 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.529 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.529 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:31.529 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482943+0000", 2024-09-16T20:51:31.529 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3450000000000002e-06 2024-09-16T20:51:31.530 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.530 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.530 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:31.530 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605772+0000", 2024-09-16T20:51:31.530 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.122829796 2024-09-16T20:51:31.530 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.530 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.530 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:31.530 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605792+0000", 2024-09-16T20:51:31.530 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9916999999999999e-05 2024-09-16T20:51:31.530 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.530 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.530 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:31.530 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605917+0000", 2024-09-16T20:51:31.531 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00012500799999999999 2024-09-16T20:51:31.531 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.531 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.531 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:31.531 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606004+0000", 2024-09-16T20:51:31.531 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6242999999999994e-05 2024-09-16T20:51:31.531 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.531 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.531 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:31.531 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606120+0000", 2024-09-16T20:51:31.531 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000116786 2024-09-16T20:51:31.531 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:31.531 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:31.531 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:31.532 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.532 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.532 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:11702 3.15 3:acf33c8b:::rbd_data.110e747f0779.000000000000001a:head [set-alloc-hint object_size 262144 write_size 262144,write 143360~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:51:31.532 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:51:31.532 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 147.36688193500001, 2024-09-16T20:51:31.532 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.084801043000000006, 2024-09-16T20:51:31.532 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:31.532 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:31.532 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:31.532 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:31.532 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:51:31.532 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:51:31.532 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 11702 2024-09-16T20:51:31.532 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.533 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:31.533 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.533 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:31.533 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:51:31.533 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:31.533 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.533 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.533 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:31.533 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:51:31.533 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:31.533 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.533 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.533 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:31.533 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137929+0000", 2024-09-16T20:51:31.533 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8400000000000002e-07 2024-09-16T20:51:31.534 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.534 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.534 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:31.534 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137937+0000", 2024-09-16T20:51:31.534 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0260000000000007e-06 2024-09-16T20:51:31.534 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.534 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.534 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:31.534 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137952+0000", 2024-09-16T20:51:31.534 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4659e-05 2024-09-16T20:51:31.534 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.534 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.534 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:31.534 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137960+0000", 2024-09-16T20:51:31.534 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0169999999999996e-06 2024-09-16T20:51:31.535 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.535 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.535 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:31.535 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138010+0000", 2024-09-16T20:51:31.535 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0021000000000001e-05 2024-09-16T20:51:31.535 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.535 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.535 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:31.535 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138114+0000", 2024-09-16T20:51:31.535 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000103797 2024-09-16T20:51:31.535 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.535 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.535 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:51:31.535 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138233+0000", 2024-09-16T20:51:31.535 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000118971 2024-09-16T20:51:31.536 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.536 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.536 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:31.536 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138953+0000", 2024-09-16T20:51:31.536 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00072026000000000002 2024-09-16T20:51:31.536 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.536 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.536 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:31.536 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222609+0000", 2024-09-16T20:51:31.536 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.083656650999999999 2024-09-16T20:51:31.536 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.536 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.536 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:31.536 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222684+0000", 2024-09-16T20:51:31.537 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4369999999999994e-05 2024-09-16T20:51:31.537 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.537 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.537 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:31.537 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222729+0000", 2024-09-16T20:51:31.537 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5287000000000002e-05 2024-09-16T20:51:31.537 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:31.537 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:31.537 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:31.537 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.537 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.537 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:11704 3.9 3:91220b75:::rbd_data.110e747f0779.000000000000001c:head [set-alloc-hint object_size 262144 write_size 262144,write 139264~6656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:51:31.537 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:51:31.537 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 147.366643934, 2024-09-16T20:51:31.537 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.084531133999999994, 2024-09-16T20:51:31.538 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:31.538 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:31.538 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:31.538 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:31.538 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:51:31.538 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:51:31.538 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 11704 2024-09-16T20:51:31.538 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.538 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:31.538 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.538 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:31.538 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:51:31.538 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:31.538 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.539 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.539 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:31.539 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:51:31.539 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:31.539 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.539 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.539 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:31.539 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138167+0000", 2024-09-16T20:51:31.539 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.33e-07 2024-09-16T20:51:31.539 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.539 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.539 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:31.539 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138179+0000", 2024-09-16T20:51:31.539 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2055999999999999e-05 2024-09-16T20:51:31.539 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.539 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.540 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:31.540 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138193+0000", 2024-09-16T20:51:31.540 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3961000000000001e-05 2024-09-16T20:51:31.540 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.540 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.540 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:31.540 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138200+0000", 2024-09-16T20:51:31.540 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4429999999999997e-06 2024-09-16T20:51:31.540 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.540 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.540 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:31.540 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138258+0000", 2024-09-16T20:51:31.540 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7225999999999997e-05 2024-09-16T20:51:31.541 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.541 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.541 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:31.541 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138364+0000", 2024-09-16T20:51:31.541 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000106013 2024-09-16T20:51:31.541 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.541 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.541 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:51:31.541 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138494+0000", 2024-09-16T20:51:31.541 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013059599999999999 2024-09-16T20:51:31.541 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.541 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.541 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:31.541 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.139220+0000", 2024-09-16T20:51:31.542 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00072539800000000004 2024-09-16T20:51:31.542 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.542 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.542 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:31.542 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222608+0000", 2024-09-16T20:51:31.542 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.083388620999999996 2024-09-16T20:51:31.542 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.542 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.542 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:31.542 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222651+0000", 2024-09-16T20:51:31.542 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2228000000000003e-05 2024-09-16T20:51:31.542 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.542 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.542 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:31.543 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222697+0000", 2024-09-16T20:51:31.543 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6659000000000002e-05 2024-09-16T20:51:31.543 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:31.543 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:31.543 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:31.543 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.543 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.543 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5138.0:9358 3.c 3.c25699cc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T20:51:31.543 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:51:31.543 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 124.65449662899999, 2024-09-16T20:51:31.543 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0018657580000001, 2024-09-16T20:51:31.543 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:31.543 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:31.543 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:51:31.543 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:31.544 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5138", 2024-09-16T20:51:31.544 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:51:31.544 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 9358 2024-09-16T20:51:31.544 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.544 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:31.544 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.544 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:31.544 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:51:31.544 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:31.544 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.544 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.544 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:31.544 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:51:31.544 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:31.545 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.545 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.545 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:31.545 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850314+0000", 2024-09-16T20:51:31.545 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4099999999999999e-07 2024-09-16T20:51:31.545 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.545 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.545 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:31.545 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850316+0000", 2024-09-16T20:51:31.545 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2469999999999998e-06 2024-09-16T20:51:31.545 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.545 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.545 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:31.545 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850322+0000", 2024-09-16T20:51:31.546 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3470000000000004e-06 2024-09-16T20:51:31.546 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.546 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.546 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:31.546 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850327+0000", 2024-09-16T20:51:31.546 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9450000000000001e-06 2024-09-16T20:51:31.546 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.546 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.546 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:31.546 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850376+0000", 2024-09-16T20:51:31.546 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8257000000000002e-05 2024-09-16T20:51:31.546 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.546 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.547 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:31.547 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850385+0000", 2024-09-16T20:51:31.547 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9800000000000004e-06 2024-09-16T20:51:31.547 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.547 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.547 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:31.547 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852167+0000", 2024-09-16T20:51:31.547 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0017825659999999 2024-09-16T20:51:31.547 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.547 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.547 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:31.547 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852179+0000", 2024-09-16T20:51:31.547 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1975e-05 2024-09-16T20:51:31.547 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:31.547 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:31.547 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:31.548 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.548 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.548 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5138.0:9382 3.c 3.c25699cc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T20:51:31.548 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:51:31.548 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 124.64614961300001, 2024-09-16T20:51:31.548 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.99358404700000003, 2024-09-16T20:51:31.548 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:31.548 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:31.548 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:51:31.548 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:31.548 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5138", 2024-09-16T20:51:31.548 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:51:31.548 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 9382 2024-09-16T20:51:31.549 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.549 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:31.549 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.549 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:31.549 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:51:31.549 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:31.549 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.549 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.549 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:31.549 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:51:31.549 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:31.549 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.549 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.549 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:31.550 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858661+0000", 2024-09-16T20:51:31.550 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0200000000000002e-07 2024-09-16T20:51:31.550 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.550 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.550 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:31.550 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858664+0000", 2024-09-16T20:51:31.550 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.038e-06 2024-09-16T20:51:31.550 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.550 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.550 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:31.550 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858671+0000", 2024-09-16T20:51:31.550 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0110000000000003e-06 2024-09-16T20:51:31.550 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.550 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.550 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:31.551 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858677+0000", 2024-09-16T20:51:31.551 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0959999999999997e-06 2024-09-16T20:51:31.551 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.551 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.551 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:31.551 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858732+0000", 2024-09-16T20:51:31.551 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4543000000000003e-05 2024-09-16T20:51:31.551 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.551 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.551 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:31.551 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858739+0000", 2024-09-16T20:51:31.551 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5429999999999996e-06 2024-09-16T20:51:31.551 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.551 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.551 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:31.552 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852236+0000", 2024-09-16T20:51:31.552 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.993496453 2024-09-16T20:51:31.552 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.552 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.552 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:31.552 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852244+0000", 2024-09-16T20:51:31.552 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8610000000000004e-06 2024-09-16T20:51:31.552 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:31.552 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:31.552 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:31.552 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.552 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.552 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5763.0:4895 3.a 3:5384606c:::rbd_data.110e1b19a241.0000000000000018:head [read 0~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-16T20:51:31.552 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:45.598212+0000", 2024-09-16T20:51:31.553 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 105.906597572, 2024-09-16T20:51:31.553 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0375864829999999, 2024-09-16T20:51:31.553 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:31.553 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:31.553 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:31.553 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:31.553 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5763", 2024-09-16T20:51:31.553 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/623897567", 2024-09-16T20:51:31.553 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4895 2024-09-16T20:51:31.553 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.553 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:31.553 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.553 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:31.554 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598212+0000", 2024-09-16T20:51:31.554 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:31.554 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.554 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.554 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:31.554 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598212+0000", 2024-09-16T20:51:31.554 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:31.554 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.554 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.554 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:31.554 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598214+0000", 2024-09-16T20:51:31.554 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0090000000000002e-06 2024-09-16T20:51:31.554 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.555 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.555 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:31.555 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598222+0000", 2024-09-16T20:51:31.555 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6699999999999994e-06 2024-09-16T20:51:31.555 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.555 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.555 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:31.555 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598241+0000", 2024-09-16T20:51:31.555 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8896999999999999e-05 2024-09-16T20:51:31.555 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.555 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.555 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:31.555 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598250+0000", 2024-09-16T20:51:31.556 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6649999999999992e-06 2024-09-16T20:51:31.556 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.556 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.556 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:31.556 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598325+0000", 2024-09-16T20:51:31.556 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5189000000000001e-05 2024-09-16T20:51:31.556 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.556 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.556 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op must wait for map", 2024-09-16T20:51:31.556 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598333+0000", 2024-09-16T20:51:31.556 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.3170000000000002e-06 2024-09-16T20:51:31.556 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.557 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.557 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:31.557 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601355+0000", 2024-09-16T20:51:31.557 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0030215400000000001 2024-09-16T20:51:31.557 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.557 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.557 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:31.557 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601365+0000", 2024-09-16T20:51:31.557 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0215e-05 2024-09-16T20:51:31.557 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.557 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.557 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:31.558 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635506+0000", 2024-09-16T20:51:31.558 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0341408240000001 2024-09-16T20:51:31.558 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.558 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.558 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:31.558 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635655+0000", 2024-09-16T20:51:31.558 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000149412 2024-09-16T20:51:31.558 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.558 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.558 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:31.558 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635799+0000", 2024-09-16T20:51:31.558 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000143745 2024-09-16T20:51:31.558 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:31.558 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:31.559 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:31.559 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.559 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.559 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5763.0:4900 3.a 3:5384606c:::rbd_data.110e1b19a241.0000000000000018:head [read 8192~24576] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-16T20:51:31.559 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:51:31.559 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 105.906539096, 2024-09-16T20:51:31.559 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0379340800000001, 2024-09-16T20:51:31.559 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:31.559 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:31.559 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:31.559 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:31.559 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5763", 2024-09-16T20:51:31.559 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/623897567", 2024-09-16T20:51:31.559 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4900 2024-09-16T20:51:31.560 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.560 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:31.560 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.560 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:31.560 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:51:31.560 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:31.560 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.560 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.560 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:31.560 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:51:31.560 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:31.560 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.560 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.561 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:31.561 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:51:31.561 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4300000000000003e-07 2024-09-16T20:51:31.561 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.561 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.561 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:31.561 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598274+0000", 2024-09-16T20:51:31.561 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.976e-06 2024-09-16T20:51:31.561 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.561 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.561 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:31.561 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598282+0000", 2024-09-16T20:51:31.561 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0029999999999994e-06 2024-09-16T20:51:31.561 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.562 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.562 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:31.562 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598289+0000", 2024-09-16T20:51:31.562 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2389999999999999e-06 2024-09-16T20:51:31.562 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.562 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.562 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:31.562 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598430+0000", 2024-09-16T20:51:31.562 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014119600000000001 2024-09-16T20:51:31.562 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.562 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.562 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting_for_map not empty", 2024-09-16T20:51:31.562 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598435+0000", 2024-09-16T20:51:31.563 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3489999999999999e-06 2024-09-16T20:51:31.563 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.563 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.563 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:31.563 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601429+0000", 2024-09-16T20:51:31.563 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0029942720000000001 2024-09-16T20:51:31.563 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.563 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.563 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:31.563 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601438+0000", 2024-09-16T20:51:31.563 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4759999999999998e-06 2024-09-16T20:51:31.563 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.563 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.564 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:31.564 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635987+0000", 2024-09-16T20:51:31.564 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0345495010000001 2024-09-16T20:51:31.564 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.564 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.564 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:31.564 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.636056+0000", 2024-09-16T20:51:31.564 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8610999999999998e-05 2024-09-16T20:51:31.564 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.564 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.564 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:31.564 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.636205+0000", 2024-09-16T20:51:31.564 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014891399999999999 2024-09-16T20:51:31.565 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:31.565 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:31.565 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:31.565 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.565 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.565 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop(client.6231.0:2182 3.17 e136/81 3:edb5e5cd:::rbd_data.110e61479aee.0000000000000014:head v 136'791, mlcod=135'789)", 2024-09-16T20:51:31.565 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:22.735499+0000", 2024-09-16T20:51:31.565 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 68.769311154999997, 2024-09-16T20:51:31.565 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12369608, 2024-09-16T20:51:31.565 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:31.565 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:31.565 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:31.565 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:31.565 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.566 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:31.566 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735499+0000", 2024-09-16T20:51:31.566 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:31.566 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.566 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.566 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:31.566 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735499+0000", 2024-09-16T20:51:31.566 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:31.566 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.566 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.566 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:31.566 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735499+0000", 2024-09-16T20:51:31.566 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8099999999999999e-07 2024-09-16T20:51:31.566 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.567 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.567 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:31.567 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735506+0000", 2024-09-16T20:51:31.567 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3579999999999997e-06 2024-09-16T20:51:31.567 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.567 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.567 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:31.567 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735517+0000", 2024-09-16T20:51:31.567 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0869999999999999e-05 2024-09-16T20:51:31.567 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.567 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.567 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:31.567 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735533+0000", 2024-09-16T20:51:31.567 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5858e-05 2024-09-16T20:51:31.567 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.568 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.568 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:31.568 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735584+0000", 2024-09-16T20:51:31.568 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0470999999999998e-05 2024-09-16T20:51:31.568 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.568 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.568 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:31.568 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735607+0000", 2024-09-16T20:51:31.568 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2875999999999999e-05 2024-09-16T20:51:31.568 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.568 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.568 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:31.568 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.859144+0000", 2024-09-16T20:51:31.568 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.123537701 2024-09-16T20:51:31.569 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.569 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.569 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:31.569 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.859195+0000", 2024-09-16T20:51:31.569 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0664999999999997e-05 2024-09-16T20:51:31.569 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:31.569 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:31.569 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:31.569 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.569 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.569 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5204 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 2523136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:51:31.569 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.883660+0000", 2024-09-16T20:51:31.569 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 67.621149625000001, 2024-09-16T20:51:31.569 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052517638999999998, 2024-09-16T20:51:31.570 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:31.570 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:31.570 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:31.570 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:31.570 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:31.570 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:31.570 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5204 2024-09-16T20:51:31.570 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.570 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:31.570 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.570 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:31.570 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883660+0000", 2024-09-16T20:51:31.570 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:31.570 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.570 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.571 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:31.571 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883660+0000", 2024-09-16T20:51:31.571 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:31.571 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.571 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.571 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:31.571 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883661+0000", 2024-09-16T20:51:31.571 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9999999999999998e-07 2024-09-16T20:51:31.571 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.571 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.571 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:31.571 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883663+0000", 2024-09-16T20:51:31.571 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.4480000000000001e-06 2024-09-16T20:51:31.572 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.572 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.572 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:31.572 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883669+0000", 2024-09-16T20:51:31.572 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.071e-06 2024-09-16T20:51:31.572 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.572 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.572 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:31.572 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883675+0000", 2024-09-16T20:51:31.572 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.366e-06 2024-09-16T20:51:31.572 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.572 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.572 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:31.572 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.935970+0000", 2024-09-16T20:51:31.573 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052295346999999999 2024-09-16T20:51:31.573 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.573 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.573 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:31.573 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936064+0000", 2024-09-16T20:51:31.573 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4438000000000006e-05 2024-09-16T20:51:31.573 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.573 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.573 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:31.573 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936178+0000", 2024-09-16T20:51:31.573 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000113569 2024-09-16T20:51:31.573 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:31.573 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:31.573 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:31.574 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.574 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.574 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5205 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 2543616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:51:31.574 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.883745+0000", 2024-09-16T20:51:31.574 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 67.621064867000001, 2024-09-16T20:51:31.574 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052692112999999999, 2024-09-16T20:51:31.574 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:31.574 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:31.574 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:31.574 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:31.574 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:31.574 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:31.574 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5205 2024-09-16T20:51:31.574 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.575 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:31.575 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.575 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:31.575 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883745+0000", 2024-09-16T20:51:31.575 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:31.575 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.575 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.575 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:31.575 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883745+0000", 2024-09-16T20:51:31.575 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:31.575 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.575 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.575 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:31.575 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883746+0000", 2024-09-16T20:51:31.576 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2699999999999999e-07 2024-09-16T20:51:31.576 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.576 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.576 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:31.576 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883750+0000", 2024-09-16T20:51:31.576 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7050000000000001e-06 2024-09-16T20:51:31.576 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.576 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.576 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:31.576 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883769+0000", 2024-09-16T20:51:31.576 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9165e-05 2024-09-16T20:51:31.576 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.576 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.577 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:31.577 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883778+0000", 2024-09-16T20:51:31.577 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.6690000000000001e-06 2024-09-16T20:51:31.577 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.577 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.577 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:31.577 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936228+0000", 2024-09-16T20:51:31.577 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052449587999999998 2024-09-16T20:51:31.577 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.577 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.577 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:31.577 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936333+0000", 2024-09-16T20:51:31.577 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010511800000000001 2024-09-16T20:51:31.578 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.578 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.578 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:31.578 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936437+0000", 2024-09-16T20:51:31.578 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000104141 2024-09-16T20:51:31.578 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:31.578 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:31.578 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:31.578 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.578 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.579 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5206 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 1167360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:51:31.579 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.883802+0000", 2024-09-16T20:51:31.579 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 67.621008372999995, 2024-09-16T20:51:31.579 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052800780999999998, 2024-09-16T20:51:31.579 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:31.579 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:31.579 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:31.579 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:31.579 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:31.579 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:31.579 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5206 2024-09-16T20:51:31.579 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.579 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:31.579 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.580 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:31.580 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883802+0000", 2024-09-16T20:51:31.580 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:31.580 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.580 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.580 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:31.580 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883802+0000", 2024-09-16T20:51:31.580 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:31.580 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.580 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.580 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:31.580 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883933+0000", 2024-09-16T20:51:31.580 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013182999999999999 2024-09-16T20:51:31.581 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.581 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.581 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:31.581 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883937+0000", 2024-09-16T20:51:31.581 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.771e-06 2024-09-16T20:51:31.581 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.581 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.581 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:31.581 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883953+0000", 2024-09-16T20:51:31.581 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5409000000000001e-05 2024-09-16T20:51:31.581 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.581 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.581 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:31.582 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883959+0000", 2024-09-16T20:51:31.582 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0700000000000003e-06 2024-09-16T20:51:31.582 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.582 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.582 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:31.582 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936476+0000", 2024-09-16T20:51:31.582 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052517434000000002 2024-09-16T20:51:31.582 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.582 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.582 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:31.582 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936529+0000", 2024-09-16T20:51:31.582 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2969000000000001e-05 2024-09-16T20:51:31.582 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.583 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.583 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:31.583 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936602+0000", 2024-09-16T20:51:31.583 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3298000000000002e-05 2024-09-16T20:51:31.583 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:31.583 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:31.583 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:31.583 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.583 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.583 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5207 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 1904640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:51:31.583 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.883980+0000", 2024-09-16T20:51:31.583 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 67.620830163999997, 2024-09-16T20:51:31.583 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052759922000000001, 2024-09-16T20:51:31.584 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:31.584 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:31.584 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:31.584 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:31.584 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:31.584 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:31.584 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5207 2024-09-16T20:51:31.584 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.584 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:31.584 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.584 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:31.584 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883980+0000", 2024-09-16T20:51:31.585 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:31.585 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.585 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.585 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:31.585 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883980+0000", 2024-09-16T20:51:31.585 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:31.585 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.585 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.585 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:31.585 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884126+0000", 2024-09-16T20:51:31.585 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014577399999999999 2024-09-16T20:51:31.585 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.586 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.586 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:31.586 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884130+0000", 2024-09-16T20:51:31.586 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9729999999999999e-06 2024-09-16T20:51:31.586 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.586 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.586 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:31.586 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884139+0000", 2024-09-16T20:51:31.586 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8660000000000001e-06 2024-09-16T20:51:31.586 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.586 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.586 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:31.586 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884144+0000", 2024-09-16T20:51:31.587 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3669999999999999e-06 2024-09-16T20:51:31.587 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.587 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.587 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:31.587 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936635+0000", 2024-09-16T20:51:31.587 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052491705 2024-09-16T20:51:31.587 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.587 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.587 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:31.587 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936679+0000", 2024-09-16T20:51:31.587 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3105999999999997e-05 2024-09-16T20:51:31.587 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.587 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.588 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:31.588 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936740+0000", 2024-09-16T20:51:31.588 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1130999999999995e-05 2024-09-16T20:51:31.588 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:31.588 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:31.588 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:31.588 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.588 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.588 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5208 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 131072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:51:31.588 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.884165+0000", 2024-09-16T20:51:31.588 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 67.620644867999999, 2024-09-16T20:51:31.588 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052746603000000003, 2024-09-16T20:51:31.588 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:31.588 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:31.589 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:31.589 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:31.589 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:31.589 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:31.589 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5208 2024-09-16T20:51:31.589 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.589 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:31.589 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.589 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:31.589 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884165+0000", 2024-09-16T20:51:31.589 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:31.589 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.589 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.590 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:31.590 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884165+0000", 2024-09-16T20:51:31.590 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:31.590 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.590 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.590 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:31.590 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884463+0000", 2024-09-16T20:51:31.590 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00029801699999999998 2024-09-16T20:51:31.590 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.590 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.590 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:31.590 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884467+0000", 2024-09-16T20:51:31.590 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8210000000000003e-06 2024-09-16T20:51:31.590 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.590 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.591 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:31.591 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884477+0000", 2024-09-16T20:51:31.591 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.7030000000000001e-06 2024-09-16T20:51:31.591 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.591 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.591 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:31.591 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884485+0000", 2024-09-16T20:51:31.591 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9710000000000004e-06 2024-09-16T20:51:31.591 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.591 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.591 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:31.591 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936785+0000", 2024-09-16T20:51:31.591 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052300665000000003 2024-09-16T20:51:31.592 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.592 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.592 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:31.592 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936851+0000", 2024-09-16T20:51:31.592 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6159000000000001e-05 2024-09-16T20:51:31.592 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.592 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.592 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:31.592 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936912+0000", 2024-09-16T20:51:31.592 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0266999999999997e-05 2024-09-16T20:51:31.592 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:31.592 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:31.592 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:31.592 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.593 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.593 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5209 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 1925120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:51:31.593 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.884504+0000", 2024-09-16T20:51:31.593 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 67.620305982999994, 2024-09-16T20:51:31.593 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052575445999999998, 2024-09-16T20:51:31.593 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:31.593 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:31.593 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:31.593 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:31.593 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:31.593 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:31.593 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5209 2024-09-16T20:51:31.593 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.594 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:31.594 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.594 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:31.594 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884504+0000", 2024-09-16T20:51:31.594 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:31.594 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.594 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.594 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:31.594 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884504+0000", 2024-09-16T20:51:31.594 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:31.594 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.594 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.594 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:31.594 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884741+0000", 2024-09-16T20:51:31.595 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00023679899999999999 2024-09-16T20:51:31.595 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.595 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.595 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:31.595 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884747+0000", 2024-09-16T20:51:31.595 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0009999999999998e-06 2024-09-16T20:51:31.595 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.595 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.595 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:31.595 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884764+0000", 2024-09-16T20:51:31.595 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7371999999999999e-05 2024-09-16T20:51:31.595 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.595 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.596 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:31.596 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884772+0000", 2024-09-16T20:51:31.596 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1349999999999992e-06 2024-09-16T20:51:31.596 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.596 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.596 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:31.596 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936956+0000", 2024-09-16T20:51:31.596 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052183754999999998 2024-09-16T20:51:31.596 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.596 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.596 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:31.596 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937003+0000", 2024-09-16T20:51:31.596 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6925000000000002e-05 2024-09-16T20:51:31.597 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.597 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.597 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:31.597 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937079+0000", 2024-09-16T20:51:31.597 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6458999999999994e-05 2024-09-16T20:51:31.597 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:31.597 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:31.597 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:31.597 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.597 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.597 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:30174 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_remove in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e176)", 2024-09-16T20:51:31.597 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:51:03.615665+0000", 2024-09-16T20:51:31.597 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 27.889144464000001, 2024-09-16T20:51:31.598 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.96831816599999998, 2024-09-16T20:51:31.598 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:31.598 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:31.598 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:31.598 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:31.598 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:51:31.598 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:31.598 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 30174 2024-09-16T20:51:31.598 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.598 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:31.598 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.598 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:31.598 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615665+0000", 2024-09-16T20:51:31.598 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:31.599 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.599 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.599 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:31.599 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615665+0000", 2024-09-16T20:51:31.599 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:31.599 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.599 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.599 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:31.599 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615667+0000", 2024-09-16T20:51:31.599 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0020000000000001e-06 2024-09-16T20:51:31.599 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.599 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.599 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:31.599 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615675+0000", 2024-09-16T20:51:31.600 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5120000000000002e-06 2024-09-16T20:51:31.600 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.600 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.600 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:31.600 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615692+0000", 2024-09-16T20:51:31.600 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.6977e-05 2024-09-16T20:51:31.600 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.600 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.600 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:31.600 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615701+0000", 2024-09-16T20:51:31.600 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6119999999999999e-06 2024-09-16T20:51:31.600 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.600 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.601 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:31.601 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615782+0000", 2024-09-16T20:51:31.601 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1804000000000005e-05 2024-09-16T20:51:31.601 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.601 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.601 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:31.601 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615795+0000", 2024-09-16T20:51:31.601 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2464e-05 2024-09-16T20:51:31.601 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.601 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.601 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:31.601 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.580071+0000", 2024-09-16T20:51:31.601 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.96427590100000005 2024-09-16T20:51:31.602 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.602 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.602 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:31.602 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.580247+0000", 2024-09-16T20:51:31.602 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000175754 2024-09-16T20:51:31.602 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.602 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.602 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 2", 2024-09-16T20:51:31.602 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.580436+0000", 2024-09-16T20:51:31.602 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00018945299999999999 2024-09-16T20:51:31.602 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.602 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.602 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:31.602 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.581045+0000", 2024-09-16T20:51:31.602 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00060937600000000001 2024-09-16T20:51:31.603 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.603 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.603 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:31.603 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.583883+0000", 2024-09-16T20:51:31.603 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0028372900000000001 2024-09-16T20:51:31.603 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.603 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.603 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:31.603 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.583944+0000", 2024-09-16T20:51:31.603 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1848999999999997e-05 2024-09-16T20:51:31.603 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.603 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.603 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:31.603 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.583984+0000", 2024-09-16T20:51:31.604 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9171999999999997e-05 2024-09-16T20:51:31.604 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:31.604 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:31.604 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:31.604 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.604 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.604 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:30201 3.f 3:f6903a53:::rbd_object_map.110eca9b190a.000000000000001a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e177)", 2024-09-16T20:51:31.604 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:51:04.603586+0000", 2024-09-16T20:51:31.604 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 26.901224357, 2024-09-16T20:51:31.604 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.98028146800000004, 2024-09-16T20:51:31.604 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:31.604 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:31.604 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:31.605 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:31.605 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:51:31.605 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:31.605 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 30201 2024-09-16T20:51:31.605 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.605 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:31.605 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.605 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:31.605 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603586+0000", 2024-09-16T20:51:31.605 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:31.605 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.605 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.605 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:31.605 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603586+0000", 2024-09-16T20:51:31.606 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:31.606 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.606 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.606 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:31.606 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603587+0000", 2024-09-16T20:51:31.606 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.519e-06 2024-09-16T20:51:31.606 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.606 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.606 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:31.606 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603593+0000", 2024-09-16T20:51:31.606 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1449999999999996e-06 2024-09-16T20:51:31.606 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.606 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.606 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:31.607 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603606+0000", 2024-09-16T20:51:31.607 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2797e-05 2024-09-16T20:51:31.607 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.607 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.607 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:31.607 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603612+0000", 2024-09-16T20:51:31.607 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8710000000000002e-06 2024-09-16T20:51:31.607 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.607 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.607 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:31.607 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603675+0000", 2024-09-16T20:51:31.607 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2728e-05 2024-09-16T20:51:31.607 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.608 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.608 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:31.608 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603687+0000", 2024-09-16T20:51:31.608 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.253e-05 2024-09-16T20:51:31.608 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.608 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.608 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:31.608 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:05.583358+0000", 2024-09-16T20:51:31.608 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.97967074600000004 2024-09-16T20:51:31.608 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.608 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.608 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:31.608 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:05.583515+0000", 2024-09-16T20:51:31.609 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00015755700000000001 2024-09-16T20:51:31.609 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.609 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.609 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:31.609 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:05.583867+0000", 2024-09-16T20:51:31.609 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00035157500000000002 2024-09-16T20:51:31.609 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:31.609 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:31.609 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:31.609 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:31.609 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:31.609 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:31.611 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:51:31.611 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:31.609+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:31.627 INFO:teuthology.orchestra.run.smithi029.stderr:noscrub is set 2024-09-16T20:51:31.711 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:31.709+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:31.732 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:31.732 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:51:31.732 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:51:31.732 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:31.741 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:51:31.812 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:51:31.811+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:51:31.848 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:31.848 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:51:31.848 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:51:31.848 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:51:31.849 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:31.857 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:51:31.912 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:31.911+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:31.965 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:31.966 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:51:31.966 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:51:31.966 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:51:31.966 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.966 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:51:31.966 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:51:31.966 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 218.70954368299999, 2024-09-16T20:51:31.966 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67502346999999996, 2024-09-16T20:51:31.966 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:31.966 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:31.966 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:31.966 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:31.967 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:51:31.967 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:51:31.967 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 1 2024-09-16T20:51:31.967 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.967 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:31.967 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.967 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:31.967 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:51:31.967 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:31.967 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.967 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.967 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:31.968 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:51:31.968 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:31.968 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.968 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.968 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:31.968 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254128+0000", 2024-09-16T20:51:31.968 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.463e-06 2024-09-16T20:51:31.968 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.968 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.968 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:31.968 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254135+0000", 2024-09-16T20:51:31.968 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1579999999999999e-06 2024-09-16T20:51:31.969 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.969 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.969 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:31.969 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254146+0000", 2024-09-16T20:51:31.969 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1341e-05 2024-09-16T20:51:31.969 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.969 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.969 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:31.969 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254155+0000", 2024-09-16T20:51:31.969 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.1549999999999996e-06 2024-09-16T20:51:31.969 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.970 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.970 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:31.970 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254221+0000", 2024-09-16T20:51:31.970 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5470999999999997e-05 2024-09-16T20:51:31.970 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.970 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.970 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:31.970 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254231+0000", 2024-09-16T20:51:31.970 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0592e-05 2024-09-16T20:51:31.970 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.970 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.970 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:31.970 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926711+0000", 2024-09-16T20:51:31.971 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67247946599999997 2024-09-16T20:51:31.971 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.971 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.971 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:31.971 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926910+0000", 2024-09-16T20:51:31.971 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019934 2024-09-16T20:51:31.971 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.971 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.971 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:51:31.971 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.927159+0000", 2024-09-16T20:51:31.971 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00024876599999999998 2024-09-16T20:51:31.971 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.971 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.972 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:31.972 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.928797+0000", 2024-09-16T20:51:31.972 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001638308 2024-09-16T20:51:31.972 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.972 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.972 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:31.972 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929026+0000", 2024-09-16T20:51:31.972 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00022876099999999999 2024-09-16T20:51:31.972 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.972 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.972 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:31.973 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929102+0000", 2024-09-16T20:51:31.973 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6184000000000001e-05 2024-09-16T20:51:31.973 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.973 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.973 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:31.973 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929150+0000", 2024-09-16T20:51:31.973 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7465e-05 2024-09-16T20:51:31.973 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:31.973 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:31.973 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:31.973 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.973 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.974 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3574 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:31.974 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:51:31.974 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 162.482380151, 2024-09-16T20:51:31.974 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053579441999999998, 2024-09-16T20:51:31.974 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:31.974 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:31.974 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:31.974 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:31.974 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:31.974 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:31.974 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3574 2024-09-16T20:51:31.974 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.974 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:31.975 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.975 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:31.975 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:51:31.975 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:31.975 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.975 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.975 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:31.975 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:51:31.975 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:31.975 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.975 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.975 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:31.976 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:51:31.976 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2e-07 2024-09-16T20:51:31.976 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.976 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.976 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:31.976 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481292+0000", 2024-09-16T20:51:31.976 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2630000000000002e-06 2024-09-16T20:51:31.976 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.976 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.976 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:31.976 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481297+0000", 2024-09-16T20:51:31.976 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.296e-06 2024-09-16T20:51:31.976 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.977 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.977 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:31.977 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481300+0000", 2024-09-16T20:51:31.977 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7019999999999999e-06 2024-09-16T20:51:31.977 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.977 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.977 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:31.977 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525533+0000", 2024-09-16T20:51:31.977 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.044232318 2024-09-16T20:51:31.977 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.977 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.978 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:31.978 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525619+0000", 2024-09-16T20:51:31.978 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5892999999999999e-05 2024-09-16T20:51:31.978 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.978 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.978 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:31.978 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525987+0000", 2024-09-16T20:51:31.978 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00036836099999999999 2024-09-16T20:51:31.979 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.979 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.979 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:31.979 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531779+0000", 2024-09-16T20:51:31.979 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0057918240000000001 2024-09-16T20:51:31.979 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.980 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.980 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:31.980 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534747+0000", 2024-09-16T20:51:31.980 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0029681600000000001 2024-09-16T20:51:31.980 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.980 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.980 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:31.980 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534832+0000", 2024-09-16T20:51:31.980 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5180999999999996e-05 2024-09-16T20:51:31.980 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.981 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.981 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:31.981 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534869+0000", 2024-09-16T20:51:31.981 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7024000000000003e-05 2024-09-16T20:51:31.981 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:31.981 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:31.981 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:31.981 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.981 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.981 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3575 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:31.981 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:51:31.982 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 162.48236541099999, 2024-09-16T20:51:31.982 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053801463000000001, 2024-09-16T20:51:31.982 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:31.982 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:31.982 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:31.982 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:31.982 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:31.982 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:31.983 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3575 2024-09-16T20:51:31.983 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.983 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:31.983 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.983 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:31.983 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:51:31.983 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:31.983 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.983 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.983 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:31.983 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:51:31.984 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:31.984 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.984 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.984 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:31.984 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481305+0000", 2024-09-16T20:51:31.984 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5400000000000002e-07 2024-09-16T20:51:31.984 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.984 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.984 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:31.984 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481307+0000", 2024-09-16T20:51:31.984 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7800000000000001e-06 2024-09-16T20:51:31.984 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.985 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.985 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:31.985 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481312+0000", 2024-09-16T20:51:31.985 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6349999999999997e-06 2024-09-16T20:51:31.985 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.985 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.985 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:31.985 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481315+0000", 2024-09-16T20:51:31.985 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7829999999999999e-06 2024-09-16T20:51:31.985 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.985 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.986 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:31.986 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.526876+0000", 2024-09-16T20:51:31.986 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.045561070000000002 2024-09-16T20:51:31.986 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.986 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.986 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:31.986 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.526954+0000", 2024-09-16T20:51:31.986 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8108000000000005e-05 2024-09-16T20:51:31.986 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.986 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.986 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:31.986 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527282+0000", 2024-09-16T20:51:31.987 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000327483 2024-09-16T20:51:31.987 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.987 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.987 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:31.987 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531784+0000", 2024-09-16T20:51:31.987 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0045020169999999997 2024-09-16T20:51:31.987 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.987 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.988 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:31.988 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534921+0000", 2024-09-16T20:51:31.988 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.003137791 2024-09-16T20:51:31.988 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.988 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.988 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:31.988 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534976+0000", 2024-09-16T20:51:31.988 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4953000000000001e-05 2024-09-16T20:51:31.988 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.988 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.988 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:31.988 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535106+0000", 2024-09-16T20:51:31.989 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000129589 2024-09-16T20:51:31.989 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:31.989 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:31.989 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:31.989 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.989 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.989 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3576 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:31.989 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:51:31.989 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 162.482351499, 2024-09-16T20:51:31.989 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053955436000000002, 2024-09-16T20:51:31.990 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:31.990 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:31.990 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:31.990 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:31.990 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:31.990 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:31.990 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3576 2024-09-16T20:51:31.990 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.990 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:31.990 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.990 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:31.990 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:51:31.991 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:31.991 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.991 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.991 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:31.991 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:51:31.991 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:31.991 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.991 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.991 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:31.991 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481319+0000", 2024-09-16T20:51:31.991 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.53e-07 2024-09-16T20:51:31.992 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.992 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.992 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:31.992 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481320+0000", 2024-09-16T20:51:31.992 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5230000000000001e-06 2024-09-16T20:51:31.992 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.992 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.992 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:31.992 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481330+0000", 2024-09-16T20:51:31.992 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0190000000000001e-05 2024-09-16T20:51:31.992 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.993 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.993 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:31.993 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481333+0000", 2024-09-16T20:51:31.993 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.151e-06 2024-09-16T20:51:31.993 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.993 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.993 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:31.993 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527689+0000", 2024-09-16T20:51:31.993 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046355192000000003 2024-09-16T20:51:31.993 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.993 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.994 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:31.994 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527764+0000", 2024-09-16T20:51:31.994 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5776000000000001e-05 2024-09-16T20:51:31.994 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.994 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.994 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:31.994 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528141+0000", 2024-09-16T20:51:31.994 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00037647599999999999 2024-09-16T20:51:31.994 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.994 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.994 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:31.995 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531788+0000", 2024-09-16T20:51:31.995 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0036470679999999998 2024-09-16T20:51:31.995 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.995 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.995 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:31.995 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535174+0000", 2024-09-16T20:51:31.995 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0033862670000000001 2024-09-16T20:51:31.995 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.995 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.995 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:31.995 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535236+0000", 2024-09-16T20:51:31.995 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1797999999999993e-05 2024-09-16T20:51:31.995 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.996 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.996 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:31.996 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535274+0000", 2024-09-16T20:51:31.996 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7741999999999999e-05 2024-09-16T20:51:31.996 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:31.996 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:31.996 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:31.996 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.996 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.996 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3577 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:31.996 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:51:31.996 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 162.48232989900001, 2024-09-16T20:51:31.996 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054073807000000002, 2024-09-16T20:51:31.997 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:31.997 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:31.997 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:31.997 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:31.997 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:31.997 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:31.997 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3577 2024-09-16T20:51:31.997 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.997 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:31.997 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.997 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:31.998 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:51:31.998 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:31.998 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.998 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.998 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:31.998 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:51:31.998 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:31.998 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.998 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.998 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:31.998 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:51:31.999 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.96e-07 2024-09-16T20:51:31.999 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.999 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.999 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:31.999 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481342+0000", 2024-09-16T20:51:31.999 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.714e-06 2024-09-16T20:51:31.999 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:31.999 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:31.999 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:31.999 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481349+0000", 2024-09-16T20:51:31.999 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4930000000000001e-06 2024-09-16T20:51:32.000 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.000 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.000 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:32.000 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481353+0000", 2024-09-16T20:51:32.000 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4879999999999999e-06 2024-09-16T20:51:32.000 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.000 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.000 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:32.000 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528619+0000", 2024-09-16T20:51:32.000 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047266351999999998 2024-09-16T20:51:32.000 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.001 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.001 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:32.001 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528706+0000", 2024-09-16T20:51:32.001 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6796e-05 2024-09-16T20:51:32.001 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.001 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.001 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:32.001 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529148+0000", 2024-09-16T20:51:32.001 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00044182999999999999 2024-09-16T20:51:32.001 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.001 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.002 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:32.002 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531792+0000", 2024-09-16T20:51:32.002 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0026443119999999998 2024-09-16T20:51:32.002 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.002 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.002 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:32.002 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535319+0000", 2024-09-16T20:51:32.002 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0035266439999999998 2024-09-16T20:51:32.002 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.002 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.002 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:32.003 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535372+0000", 2024-09-16T20:51:32.003 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3207000000000001e-05 2024-09-16T20:51:32.003 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.003 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.003 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:32.003 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535414+0000", 2024-09-16T20:51:32.003 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1675000000000003e-05 2024-09-16T20:51:32.003 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.003 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:32.003 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.003 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.003 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.004 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3578 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:32.004 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:51:32.004 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 162.48230873, 2024-09-16T20:51:32.004 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054189586999999997, 2024-09-16T20:51:32.004 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:32.004 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:32.004 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:32.004 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:32.004 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:32.004 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:32.004 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3578 2024-09-16T20:51:32.004 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.005 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:32.005 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.005 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:32.005 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:51:32.005 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.005 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.005 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.005 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:32.005 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:51:32.005 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.005 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.005 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.006 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:32.006 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:51:32.006 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0800000000000001e-07 2024-09-16T20:51:32.006 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.006 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.006 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:32.006 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481363+0000", 2024-09-16T20:51:32.006 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0339999999999999e-06 2024-09-16T20:51:32.006 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.006 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.007 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:32.007 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481371+0000", 2024-09-16T20:51:32.007 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4440000000000002e-06 2024-09-16T20:51:32.007 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.007 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.007 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:32.007 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481374+0000", 2024-09-16T20:51:32.007 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2049999999999998e-06 2024-09-16T20:51:32.007 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.007 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.007 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:32.007 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529555+0000", 2024-09-16T20:51:32.008 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048181202999999999 2024-09-16T20:51:32.008 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.008 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.008 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:32.008 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529616+0000", 2024-09-16T20:51:32.008 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0912000000000002e-05 2024-09-16T20:51:32.008 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.008 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.009 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:32.009 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529939+0000", 2024-09-16T20:51:32.009 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00032269800000000001 2024-09-16T20:51:32.009 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.009 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.009 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:32.009 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532602+0000", 2024-09-16T20:51:32.009 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0026634940000000002 2024-09-16T20:51:32.009 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.009 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.009 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:32.010 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535465+0000", 2024-09-16T20:51:32.010 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.002862149 2024-09-16T20:51:32.010 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.010 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.010 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:32.010 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535516+0000", 2024-09-16T20:51:32.010 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1696000000000001e-05 2024-09-16T20:51:32.010 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.010 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.010 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:32.011 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535551+0000", 2024-09-16T20:51:32.011 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4443999999999998e-05 2024-09-16T20:51:32.011 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.011 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:32.011 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.011 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.011 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.011 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3579 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:32.012 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:51:32.012 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 162.482290291, 2024-09-16T20:51:32.012 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054295151, 2024-09-16T20:51:32.012 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:32.012 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:32.012 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:32.012 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:32.012 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:32.012 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:32.012 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3579 2024-09-16T20:51:32.012 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.013 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:32.013 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.013 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:32.013 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:51:32.013 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.013 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.013 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.013 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:32.013 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:51:32.013 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.013 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.014 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.014 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:32.014 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:51:32.014 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3300000000000001e-07 2024-09-16T20:51:32.014 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.014 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.014 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:32.014 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481381+0000", 2024-09-16T20:51:32.014 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.649e-06 2024-09-16T20:51:32.014 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.014 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.015 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:32.015 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481387+0000", 2024-09-16T20:51:32.015 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4519999999999996e-06 2024-09-16T20:51:32.015 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.015 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.015 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:32.015 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481390+0000", 2024-09-16T20:51:32.015 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0170000000000001e-06 2024-09-16T20:51:32.015 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.015 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.015 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:32.015 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530756+0000", 2024-09-16T20:51:32.016 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049365882999999999 2024-09-16T20:51:32.016 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.016 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.016 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:32.016 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530845+0000", 2024-09-16T20:51:32.016 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9197000000000007e-05 2024-09-16T20:51:32.016 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.016 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.016 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:32.016 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531191+0000", 2024-09-16T20:51:32.016 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00034562500000000001 2024-09-16T20:51:32.016 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.017 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.017 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:32.017 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534078+0000", 2024-09-16T20:51:32.017 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0028878179999999999 2024-09-16T20:51:32.017 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.017 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.017 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:32.017 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535589+0000", 2024-09-16T20:51:32.017 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00151085 2024-09-16T20:51:32.017 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.017 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.018 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:32.018 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535639+0000", 2024-09-16T20:51:32.018 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9697000000000002e-05 2024-09-16T20:51:32.018 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.018 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.018 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:32.018 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535675+0000", 2024-09-16T20:51:32.018 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5729999999999998e-05 2024-09-16T20:51:32.018 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.018 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:32.018 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.019 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.019 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.019 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3580 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:32.019 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:51:32.019 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 162.48227665900001, 2024-09-16T20:51:32.019 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054397361999999998, 2024-09-16T20:51:32.019 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:32.019 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:32.019 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:32.019 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:32.019 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:32.020 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:32.020 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3580 2024-09-16T20:51:32.020 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.020 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:32.020 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.020 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:32.020 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:51:32.020 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.020 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.020 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.020 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:32.021 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:51:32.021 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.021 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.021 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.021 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:32.021 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:51:32.021 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2600000000000001e-07 2024-09-16T20:51:32.021 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.021 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.021 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:32.021 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481395+0000", 2024-09-16T20:51:32.021 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.544e-06 2024-09-16T20:51:32.022 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.022 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.022 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:32.022 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481404+0000", 2024-09-16T20:51:32.022 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8850000000000005e-06 2024-09-16T20:51:32.022 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.022 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.022 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:32.022 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481407+0000", 2024-09-16T20:51:32.022 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1540000000000002e-06 2024-09-16T20:51:32.022 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.022 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.023 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:32.023 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531823+0000", 2024-09-16T20:51:32.023 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050415637999999999 2024-09-16T20:51:32.023 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.023 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.023 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:32.023 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531895+0000", 2024-09-16T20:51:32.023 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2179000000000001e-05 2024-09-16T20:51:32.023 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.023 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.023 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:32.024 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532236+0000", 2024-09-16T20:51:32.024 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00034073799999999999 2024-09-16T20:51:32.024 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.024 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.024 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:32.024 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534083+0000", 2024-09-16T20:51:32.024 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001847298 2024-09-16T20:51:32.024 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.024 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.024 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:32.024 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535714+0000", 2024-09-16T20:51:32.025 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0016316810000000001 2024-09-16T20:51:32.025 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.025 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.025 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:32.025 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535759+0000", 2024-09-16T20:51:32.025 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4478999999999999e-05 2024-09-16T20:51:32.025 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.025 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.025 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:32.025 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535791+0000", 2024-09-16T20:51:32.025 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.154e-05 2024-09-16T20:51:32.025 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.026 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:32.026 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.026 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.026 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.026 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3581 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:32.026 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:51:32.026 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 162.482254662, 2024-09-16T20:51:32.026 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.057094403000000002, 2024-09-16T20:51:32.026 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:32.026 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:32.026 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:32.026 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:32.027 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:32.027 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:32.027 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3581 2024-09-16T20:51:32.027 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.027 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:32.027 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.027 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:32.027 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:51:32.027 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.027 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.027 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.027 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:32.028 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:51:32.028 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.028 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.028 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.028 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:32.028 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:51:32.028 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8799999999999998e-07 2024-09-16T20:51:32.028 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.028 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.028 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:32.028 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481418+0000", 2024-09-16T20:51:32.028 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1859999999999999e-06 2024-09-16T20:51:32.029 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.029 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.029 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:32.029 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481423+0000", 2024-09-16T20:51:32.029 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8279999999999999e-06 2024-09-16T20:51:32.029 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.029 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.029 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:32.029 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481426+0000", 2024-09-16T20:51:32.029 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0110000000000001e-06 2024-09-16T20:51:32.029 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.030 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.030 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:32.030 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532631+0000", 2024-09-16T20:51:32.030 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051204514999999999 2024-09-16T20:51:32.030 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.030 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.030 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:32.030 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.533381+0000", 2024-09-16T20:51:32.030 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00075029000000000003 2024-09-16T20:51:32.030 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.030 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.030 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:32.031 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.533739+0000", 2024-09-16T20:51:32.031 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00035773100000000002 2024-09-16T20:51:32.031 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.031 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.031 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:32.031 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535123+0000", 2024-09-16T20:51:32.031 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0013836160000000001 2024-09-16T20:51:32.031 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.031 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.031 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:32.031 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538413+0000", 2024-09-16T20:51:32.031 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0032902510000000001 2024-09-16T20:51:32.031 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.032 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.032 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:32.032 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538470+0000", 2024-09-16T20:51:32.032 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6697999999999998e-05 2024-09-16T20:51:32.032 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.032 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.032 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:32.032 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538510+0000", 2024-09-16T20:51:32.032 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9988999999999999e-05 2024-09-16T20:51:32.032 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.032 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:32.033 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.033 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.033 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.033 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:12732 3.d 3:b752444a:::rbd_header.110e747f0779:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-16T20:51:32.033 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:51:32.033 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 147.19518208400001, 2024-09-16T20:51:32.033 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.065302467000000003, 2024-09-16T20:51:32.033 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:32.033 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:32.033 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:32.033 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:32.033 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:51:32.034 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:51:32.034 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 12732 2024-09-16T20:51:32.034 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.034 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:32.034 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.034 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:32.034 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:51:32.034 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.034 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.034 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.034 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:32.034 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:51:32.034 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.035 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.035 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.035 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:32.035 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768490+0000", 2024-09-16T20:51:32.035 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2859999999999998e-06 2024-09-16T20:51:32.035 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.035 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.035 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:32.035 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768501+0000", 2024-09-16T20:51:32.035 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1226999999999999e-05 2024-09-16T20:51:32.035 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.036 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.036 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:32.036 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768517+0000", 2024-09-16T20:51:32.036 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.592e-05 2024-09-16T20:51:32.036 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.036 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.036 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:32.036 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768528+0000", 2024-09-16T20:51:32.036 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0689999999999999e-05 2024-09-16T20:51:32.036 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.037 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.037 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:32.037 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768584+0000", 2024-09-16T20:51:32.037 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5889e-05 2024-09-16T20:51:32.037 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.037 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.037 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:32.037 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768720+0000", 2024-09-16T20:51:32.037 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000136718 2024-09-16T20:51:32.037 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.037 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.037 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:32.038 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.769066+0000", 2024-09-16T20:51:32.038 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000346013 2024-09-16T20:51:32.038 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.038 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.038 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:32.038 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833659+0000", 2024-09-16T20:51:32.038 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.064592822999999994 2024-09-16T20:51:32.038 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.038 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.038 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:32.039 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833719+0000", 2024-09-16T20:51:32.039 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9441000000000002e-05 2024-09-16T20:51:32.039 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.039 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.039 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:32.039 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833758+0000", 2024-09-16T20:51:32.039 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9122000000000003e-05 2024-09-16T20:51:32.039 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.039 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.039 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:32.039 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833790+0000", 2024-09-16T20:51:32.039 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2338000000000001e-05 2024-09-16T20:51:32.040 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.040 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:32.040 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.040 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.040 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.040 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.5089.0:12732 3.d e73/23 ondisk, result = 0)", 2024-09-16T20:51:32.040 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:51:32.040 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 147.193579914, 2024-09-16T20:51:32.040 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.063706456999999994, 2024-09-16T20:51:32.040 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:32.040 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:32.040 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:32.040 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:32.041 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.041 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:32.041 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:51:32.041 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.041 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.041 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.041 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:32.041 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:51:32.041 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.041 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.041 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.041 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:32.042 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770091+0000", 2024-09-16T20:51:32.042 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.53e-06 2024-09-16T20:51:32.042 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.042 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.042 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:32.042 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770104+0000", 2024-09-16T20:51:32.042 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2203e-05 2024-09-16T20:51:32.042 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.042 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.042 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:32.042 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833637+0000", 2024-09-16T20:51:32.042 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.063533201999999997 2024-09-16T20:51:32.043 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.043 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.043 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:32.043 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833655+0000", 2024-09-16T20:51:32.043 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8369e-05 2024-09-16T20:51:32.043 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.043 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.043 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:32.043 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833707+0000", 2024-09-16T20:51:32.043 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2061999999999997e-05 2024-09-16T20:51:32.043 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.043 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.044 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:32.044 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833715+0000", 2024-09-16T20:51:32.044 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5179999999999998e-06 2024-09-16T20:51:32.044 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.044 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.044 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:32.044 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833796+0000", 2024-09-16T20:51:32.044 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1573e-05 2024-09-16T20:51:32.044 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.044 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:32.044 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.044 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.044 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.045 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3286 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 245760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:32.045 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:51:32.045 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 101.922517977, 2024-09-16T20:51:32.045 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052523564000000002, 2024-09-16T20:51:32.045 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:32.045 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:32.045 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:32.045 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:32.045 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:51:32.045 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:32.045 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3286 2024-09-16T20:51:32.046 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.046 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:32.046 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.046 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:32.046 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:51:32.046 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.046 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.046 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.046 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:32.046 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:51:32.046 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.046 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.047 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.047 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:32.047 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:51:32.047 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3900000000000001e-07 2024-09-16T20:51:32.047 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.047 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.047 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:32.047 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041153+0000", 2024-09-16T20:51:32.047 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.119e-06 2024-09-16T20:51:32.047 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.047 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.047 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:32.047 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041156+0000", 2024-09-16T20:51:32.048 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0000000000000001e-06 2024-09-16T20:51:32.048 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.048 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.048 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:32.048 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041159+0000", 2024-09-16T20:51:32.048 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5859999999999999e-06 2024-09-16T20:51:32.048 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.048 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.048 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:32.048 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093529+0000", 2024-09-16T20:51:32.048 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052370591000000001 2024-09-16T20:51:32.049 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.049 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.049 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:32.049 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093570+0000", 2024-09-16T20:51:32.049 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0161999999999999e-05 2024-09-16T20:51:32.049 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.049 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.049 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:32.049 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093675+0000", 2024-09-16T20:51:32.049 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010586699999999999 2024-09-16T20:51:32.049 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.049 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:32.050 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.050 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.050 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.050 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3287 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 250880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:32.050 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:51:32.050 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 101.922507928, 2024-09-16T20:51:32.050 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052781203999999998, 2024-09-16T20:51:32.050 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:32.050 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:32.050 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:32.050 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:32.050 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:51:32.050 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:32.051 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3287 2024-09-16T20:51:32.051 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.051 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:32.051 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.051 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:32.051 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:51:32.051 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.051 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.051 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.051 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:32.051 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:51:32.052 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.052 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.052 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:32.052 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:51:32.052 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1e-07 2024-09-16T20:51:32.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.052 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.052 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:32.052 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041163+0000", 2024-09-16T20:51:32.052 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0690000000000001e-06 2024-09-16T20:51:32.053 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.053 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.053 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:32.053 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041166+0000", 2024-09-16T20:51:32.053 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.982e-06 2024-09-16T20:51:32.053 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.053 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.053 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:32.053 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041169+0000", 2024-09-16T20:51:32.053 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.593e-06 2024-09-16T20:51:32.053 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.053 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.054 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:32.054 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093709+0000", 2024-09-16T20:51:32.054 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052540492000000001 2024-09-16T20:51:32.054 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.054 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.054 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:32.054 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093792+0000", 2024-09-16T20:51:32.054 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2436999999999997e-05 2024-09-16T20:51:32.054 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.054 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.054 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:32.054 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093943+0000", 2024-09-16T20:51:32.055 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000151421 2024-09-16T20:51:32.055 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.055 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:32.055 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.055 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.055 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.055 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3288 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 256000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:32.055 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:51:32.055 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 101.92249805500001, 2024-09-16T20:51:32.055 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052928269, 2024-09-16T20:51:32.055 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:32.055 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:32.055 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:32.056 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:32.056 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:51:32.056 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:32.056 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3288 2024-09-16T20:51:32.056 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.056 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:32.056 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.056 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:32.056 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:51:32.056 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.056 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.056 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.056 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:32.057 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:51:32.057 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.057 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.057 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.057 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:32.057 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:51:32.057 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2700000000000001e-07 2024-09-16T20:51:32.057 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.057 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.057 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:32.057 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041173+0000", 2024-09-16T20:51:32.057 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.113e-06 2024-09-16T20:51:32.057 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.058 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.058 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:32.058 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041176+0000", 2024-09-16T20:51:32.058 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9799999999999998e-06 2024-09-16T20:51:32.058 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.058 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.058 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:32.058 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041179+0000", 2024-09-16T20:51:32.058 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.667e-06 2024-09-16T20:51:32.058 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.058 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.059 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:32.059 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093957+0000", 2024-09-16T20:51:32.059 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052778703000000003 2024-09-16T20:51:32.059 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.059 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.059 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:32.059 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093999+0000", 2024-09-16T20:51:32.059 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1300000000000001e-05 2024-09-16T20:51:32.059 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.059 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.059 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:32.059 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094100+0000", 2024-09-16T20:51:32.059 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101279 2024-09-16T20:51:32.060 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.060 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:32.060 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.060 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.060 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.060 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3289 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 261120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:32.060 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:51:32.060 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 101.922488059, 2024-09-16T20:51:32.060 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053182976, 2024-09-16T20:51:32.060 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:32.060 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:32.060 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:32.060 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:32.061 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:51:32.061 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:32.061 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3289 2024-09-16T20:51:32.061 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.061 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:32.061 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.061 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:32.061 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:51:32.061 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.061 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.061 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.061 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:32.061 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:51:32.062 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.062 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.062 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.062 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:32.062 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:51:32.062 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2999999999999999e-07 2024-09-16T20:51:32.062 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.062 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.062 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:32.062 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041183+0000", 2024-09-16T20:51:32.062 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0720000000000001e-06 2024-09-16T20:51:32.062 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.062 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.063 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:32.063 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041186+0000", 2024-09-16T20:51:32.063 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9869999999999999e-06 2024-09-16T20:51:32.063 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.063 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.063 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:32.063 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041189+0000", 2024-09-16T20:51:32.063 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.587e-06 2024-09-16T20:51:32.063 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.063 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.063 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:32.063 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094136+0000", 2024-09-16T20:51:32.063 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052947270999999997 2024-09-16T20:51:32.063 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.064 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.064 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:32.064 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094213+0000", 2024-09-16T20:51:32.064 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7552000000000006e-05 2024-09-16T20:51:32.064 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.064 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.064 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:32.064 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094365+0000", 2024-09-16T20:51:32.064 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00015127700000000001 2024-09-16T20:51:32.064 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.064 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:32.064 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.064 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.065 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.065 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:2196 3.d 3:b1fe01a0:::rbd_data.110e7ef3004.0000000000000000:2d [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e136)", 2024-09-16T20:51:32.065 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:22.739057+0000", 2024-09-16T20:51:32.065 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 69.224613259999998, 2024-09-16T20:51:32.065 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11999464899999999, 2024-09-16T20:51:32.065 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:32.065 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:32.065 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:32.065 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:32.065 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:32.065 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:32.066 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2196 2024-09-16T20:51:32.066 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.066 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:32.066 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.066 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:32.066 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.739057+0000", 2024-09-16T20:51:32.066 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.066 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.066 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.066 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:32.066 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.739057+0000", 2024-09-16T20:51:32.066 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.066 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.067 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.067 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:32.067 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.739057+0000", 2024-09-16T20:51:32.067 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1399999999999997e-07 2024-09-16T20:51:32.067 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.067 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.067 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:32.067 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.739060+0000", 2024-09-16T20:51:32.067 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6019999999999998e-06 2024-09-16T20:51:32.067 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.067 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.067 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:32.067 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.858755+0000", 2024-09-16T20:51:32.068 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.119695151 2024-09-16T20:51:32.068 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.068 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.068 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:32.068 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.858778+0000", 2024-09-16T20:51:32.068 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3184999999999999e-05 2024-09-16T20:51:32.068 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.068 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.068 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:32.068 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.858865+0000", 2024-09-16T20:51:32.068 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.7330000000000006e-05 2024-09-16T20:51:32.069 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.069 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.069 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:32.069 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.858952+0000", 2024-09-16T20:51:32.069 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6803000000000002e-05 2024-09-16T20:51:32.069 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.069 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.069 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:32.069 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.859051+0000", 2024-09-16T20:51:32.069 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8963999999999996e-05 2024-09-16T20:51:32.069 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.069 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:32.070 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.070 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.070 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.070 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6061.0:5401 3.13 3:c8554348:::rbd_data.110e16eafbd.0000000000000011:head [read 577536~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-16T20:51:32.070 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:27.761536+0000", 2024-09-16T20:51:32.070 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 64.202133446999994, 2024-09-16T20:51:32.070 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.003742712, 2024-09-16T20:51:32.070 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:32.070 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:32.070 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:51:32.070 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:32.071 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6061", 2024-09-16T20:51:32.071 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3930128440", 2024-09-16T20:51:32.071 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5401 2024-09-16T20:51:32.071 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.071 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:32.071 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.071 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:32.071 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761536+0000", 2024-09-16T20:51:32.071 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.071 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.071 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.072 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:32.072 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761536+0000", 2024-09-16T20:51:32.072 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.072 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.072 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.072 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:32.072 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761537+0000", 2024-09-16T20:51:32.072 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2900000000000003e-07 2024-09-16T20:51:32.072 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.072 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.072 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:32.072 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761540+0000", 2024-09-16T20:51:32.073 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2320000000000001e-06 2024-09-16T20:51:32.073 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.073 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.073 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:32.073 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761548+0000", 2024-09-16T20:51:32.073 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1289999999999996e-06 2024-09-16T20:51:32.073 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.073 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.073 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:32.073 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761554+0000", 2024-09-16T20:51:32.073 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0820000000000004e-06 2024-09-16T20:51:32.073 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.073 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.074 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:32.074 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761621+0000", 2024-09-16T20:51:32.074 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6266000000000006e-05 2024-09-16T20:51:32.074 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.074 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.074 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:32.074 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761626+0000", 2024-09-16T20:51:32.074 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.614e-06 2024-09-16T20:51:32.074 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.074 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.074 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:32.074 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:28.765163+0000", 2024-09-16T20:51:32.075 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.003536416 2024-09-16T20:51:32.075 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.075 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.075 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:32.075 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:28.765279+0000", 2024-09-16T20:51:32.075 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000116344 2024-09-16T20:51:32.075 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.075 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:32.075 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.075 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.075 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.075 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6061.0:5425 3.13 3:c8554348:::rbd_data.110e16eafbd.0000000000000011:head [read 602112~28672] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-16T20:51:32.075 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:27.767049+0000", 2024-09-16T20:51:32.076 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 64.196620788000004, 2024-09-16T20:51:32.076 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.99847897900000004, 2024-09-16T20:51:32.076 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:32.076 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:32.076 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:51:32.076 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:32.076 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6061", 2024-09-16T20:51:32.076 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3930128440", 2024-09-16T20:51:32.076 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5425 2024-09-16T20:51:32.076 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.076 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:32.077 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.077 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:32.077 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767049+0000", 2024-09-16T20:51:32.077 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.077 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.077 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.077 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:32.077 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767049+0000", 2024-09-16T20:51:32.077 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.077 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.077 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.078 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:32.078 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767050+0000", 2024-09-16T20:51:32.078 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6499999999999999e-07 2024-09-16T20:51:32.078 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.078 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.078 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:32.078 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767052+0000", 2024-09-16T20:51:32.078 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7350000000000001e-06 2024-09-16T20:51:32.078 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.078 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.078 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:32.079 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767060+0000", 2024-09-16T20:51:32.079 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5109999999999997e-06 2024-09-16T20:51:32.079 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.079 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.079 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:32.079 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767066+0000", 2024-09-16T20:51:32.079 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8139999999999997e-06 2024-09-16T20:51:32.080 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.080 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.080 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:32.080 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767128+0000", 2024-09-16T20:51:32.080 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2174999999999993e-05 2024-09-16T20:51:32.080 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.080 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.080 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:32.080 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767133+0000", 2024-09-16T20:51:32.081 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2140000000000004e-06 2024-09-16T20:51:32.081 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.081 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.081 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:32.081 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:28.765441+0000", 2024-09-16T20:51:32.081 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.99830808599999998 2024-09-16T20:51:32.081 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.081 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.081 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:32.081 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:28.765528+0000", 2024-09-16T20:51:32.081 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6879000000000006e-05 2024-09-16T20:51:32.082 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.082 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:32.082 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.082 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.082 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.082 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:25585 3.10 3:0c7c5b23:::rbd_header.110ebf0678ad:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:51:32.082 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:32.877132+0000", 2024-09-16T20:51:32.082 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 59.086538238999999, 2024-09-16T20:51:32.082 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054096699999999998, 2024-09-16T20:51:32.083 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:32.083 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:32.083 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:32.083 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:32.083 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:51:32.083 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:32.083 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 25585 2024-09-16T20:51:32.083 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.084 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:32.084 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.084 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:32.084 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.877132+0000", 2024-09-16T20:51:32.084 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.084 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.084 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.084 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:32.084 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.877132+0000", 2024-09-16T20:51:32.084 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.084 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.085 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.085 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:32.085 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931072+0000", 2024-09-16T20:51:32.085 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053939931000000003 2024-09-16T20:51:32.085 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.085 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.085 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:32.085 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931076+0000", 2024-09-16T20:51:32.085 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2860000000000001e-06 2024-09-16T20:51:32.085 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.085 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.086 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:32.086 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931085+0000", 2024-09-16T20:51:32.086 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.3370000000000006e-06 2024-09-16T20:51:32.086 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.086 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.086 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:32.086 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931092+0000", 2024-09-16T20:51:32.086 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9970000000000001e-06 2024-09-16T20:51:32.086 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.086 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.086 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:32.086 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931137+0000", 2024-09-16T20:51:32.087 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5160000000000001e-05 2024-09-16T20:51:32.087 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.087 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.087 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:32.087 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931178+0000", 2024-09-16T20:51:32.087 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0379000000000001e-05 2024-09-16T20:51:32.087 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.087 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.088 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:32.088 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931228+0000", 2024-09-16T20:51:32.088 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0609999999999998e-05 2024-09-16T20:51:32.088 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.088 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:32.088 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.088 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.088 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.088 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:26679 3.c 3:33e2f78f:::rbd_header.110e16eafbd:head [watch ping cookie 140213436013408 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e153)", 2024-09-16T20:51:32.088 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:41.440287+0000", 2024-09-16T20:51:32.088 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 50.523382364, 2024-09-16T20:51:32.089 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.13295633600000001, 2024-09-16T20:51:32.089 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:32.089 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:32.089 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:32.089 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:32.089 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:51:32.089 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:32.089 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 26679 2024-09-16T20:51:32.089 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.089 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:32.089 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.090 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:32.090 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440287+0000", 2024-09-16T20:51:32.090 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.090 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.090 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.090 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:32.090 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440287+0000", 2024-09-16T20:51:32.090 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.090 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.090 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.090 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:32.090 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440288+0000", 2024-09-16T20:51:32.090 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0900000000000001e-07 2024-09-16T20:51:32.091 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.091 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.091 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:32.091 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440292+0000", 2024-09-16T20:51:32.091 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3749999999999996e-06 2024-09-16T20:51:32.091 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.091 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.091 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:32.091 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440299+0000", 2024-09-16T20:51:32.091 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8249999999999999e-06 2024-09-16T20:51:32.091 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.091 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.091 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:32.092 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440307+0000", 2024-09-16T20:51:32.092 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4800000000000004e-06 2024-09-16T20:51:32.092 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.092 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.092 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:32.092 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440451+0000", 2024-09-16T20:51:32.092 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014414500000000001 2024-09-16T20:51:32.092 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.092 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.092 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:32.092 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440465+0000", 2024-09-16T20:51:32.092 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.402e-05 2024-09-16T20:51:32.093 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.093 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.093 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:32.093 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.573043+0000", 2024-09-16T20:51:32.093 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.13257855399999999 2024-09-16T20:51:32.093 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.093 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.093 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:32.093 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.573130+0000", 2024-09-16T20:51:32.093 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6168999999999994e-05 2024-09-16T20:51:32.093 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.093 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.094 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:32.094 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.573244+0000", 2024-09-16T20:51:32.094 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000114159 2024-09-16T20:51:32.094 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.094 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:32.094 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.094 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.094 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:32.094 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:32.096 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:51:32.096 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:32.094+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:32.196 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:32.195+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:32.246 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:32.246 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:51:32.246 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:51:32.246 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:32.254 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:51:32.297 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:51:32.296+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:51:32.361 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:32.361 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:51:32.362 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:51:32.362 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:51:32.362 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:32.370 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:51:32.399 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:32.397+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:32.478 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:32.479 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:51:32.479 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:51:32.479 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:51:32.479 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.479 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3714 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1480704~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:32.479 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:51:32.479 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 189.83793702299999, 2024-09-16T20:51:32.479 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050152552000000003, 2024-09-16T20:51:32.479 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:32.479 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:32.479 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:32.480 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:32.480 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:32.480 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:32.480 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3714 2024-09-16T20:51:32.480 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.480 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:32.480 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.480 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:32.480 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:51:32.480 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.480 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.480 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.480 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:32.481 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:51:32.481 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.481 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.481 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.481 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:32.481 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:51:32.481 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0499999999999999e-07 2024-09-16T20:51:32.481 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.481 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.481 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:32.481 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639007+0000", 2024-09-16T20:51:32.481 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5340000000000001e-06 2024-09-16T20:51:32.481 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.482 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.482 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:32.482 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639013+0000", 2024-09-16T20:51:32.482 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3260000000000003e-06 2024-09-16T20:51:32.482 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.482 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.482 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:32.482 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639018+0000", 2024-09-16T20:51:32.482 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2850000000000002e-06 2024-09-16T20:51:32.483 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.484 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.484 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:32.484 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689026+0000", 2024-09-16T20:51:32.484 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.05000798 2024-09-16T20:51:32.484 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.484 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.484 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:32.484 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689077+0000", 2024-09-16T20:51:32.484 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1063999999999998e-05 2024-09-16T20:51:32.484 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.484 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.485 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:32.485 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689158+0000", 2024-09-16T20:51:32.485 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1057999999999995e-05 2024-09-16T20:51:32.485 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.485 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:32.485 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.485 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.485 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.485 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3716 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1488896~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:32.485 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:51:32.485 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 189.83791921299999, 2024-09-16T20:51:32.485 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050280142, 2024-09-16T20:51:32.486 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:32.486 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:32.486 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:32.486 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:32.486 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:32.486 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:32.486 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3716 2024-09-16T20:51:32.486 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.486 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:32.486 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.486 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:32.486 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:51:32.486 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.487 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.487 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.487 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:32.487 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:51:32.487 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.487 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.487 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.487 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:32.487 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639024+0000", 2024-09-16T20:51:32.487 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9999999999999997e-07 2024-09-16T20:51:32.487 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.487 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.487 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:32.488 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639026+0000", 2024-09-16T20:51:32.488 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.781e-06 2024-09-16T20:51:32.488 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.488 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.488 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:32.488 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639032+0000", 2024-09-16T20:51:32.488 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7780000000000004e-06 2024-09-16T20:51:32.488 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.488 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.488 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:32.488 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639036+0000", 2024-09-16T20:51:32.489 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6909999999999996e-06 2024-09-16T20:51:32.489 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.489 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.489 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:32.489 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689191+0000", 2024-09-16T20:51:32.489 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050155284000000001 2024-09-16T20:51:32.489 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.489 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.489 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:32.489 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689242+0000", 2024-09-16T20:51:32.490 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0662999999999999e-05 2024-09-16T20:51:32.490 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.490 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.490 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:32.490 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689304+0000", 2024-09-16T20:51:32.490 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1345000000000003e-05 2024-09-16T20:51:32.490 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.490 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:32.490 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.490 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.490 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.490 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3719 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1497088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:32.491 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:51:32.491 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 189.83789193499999, 2024-09-16T20:51:32.491 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050399715999999997, 2024-09-16T20:51:32.491 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:32.491 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:32.491 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:32.491 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:32.491 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:32.491 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:32.491 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3719 2024-09-16T20:51:32.491 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.491 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:32.491 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.491 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:32.492 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:51:32.492 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.492 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.492 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.492 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:32.492 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:51:32.492 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.492 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.492 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.492 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:32.492 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:51:32.492 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9700000000000002e-07 2024-09-16T20:51:32.492 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.493 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.493 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:32.493 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639053+0000", 2024-09-16T20:51:32.493 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2900000000000001e-06 2024-09-16T20:51:32.493 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.493 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.493 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:32.493 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639060+0000", 2024-09-16T20:51:32.493 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7750000000000004e-06 2024-09-16T20:51:32.493 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.493 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.493 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:32.493 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639066+0000", 2024-09-16T20:51:32.494 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0639999999999999e-06 2024-09-16T20:51:32.494 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.494 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.494 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:32.494 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689337+0000", 2024-09-16T20:51:32.494 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050270784999999998 2024-09-16T20:51:32.494 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.494 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.494 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:32.494 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689392+0000", 2024-09-16T20:51:32.494 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4920999999999998e-05 2024-09-16T20:51:32.494 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.494 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.494 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:32.494 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689450+0000", 2024-09-16T20:51:32.495 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8483999999999998e-05 2024-09-16T20:51:32.495 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.495 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:32.495 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.495 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.495 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.495 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3721 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1505280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:32.495 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:51:32.495 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 189.83786880299999, 2024-09-16T20:51:32.495 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050513608000000002, 2024-09-16T20:51:32.495 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:32.495 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:32.495 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:32.496 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:32.496 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:32.496 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:32.496 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3721 2024-09-16T20:51:32.496 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.496 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:32.496 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.496 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:32.496 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:51:32.496 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.496 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.496 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.496 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:32.496 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:51:32.496 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.497 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.497 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.497 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:32.497 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:51:32.497 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7e-07 2024-09-16T20:51:32.497 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.497 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.497 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:32.497 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639076+0000", 2024-09-16T20:51:32.497 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8899999999999999e-06 2024-09-16T20:51:32.497 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.497 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.497 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:32.498 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639083+0000", 2024-09-16T20:51:32.498 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1210000000000001e-06 2024-09-16T20:51:32.498 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.498 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.498 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:32.498 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639089+0000", 2024-09-16T20:51:32.498 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.643e-06 2024-09-16T20:51:32.498 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.498 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.498 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:32.498 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689474+0000", 2024-09-16T20:51:32.498 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050385679000000003 2024-09-16T20:51:32.498 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.498 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.499 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:32.499 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689533+0000", 2024-09-16T20:51:32.499 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8303000000000001e-05 2024-09-16T20:51:32.499 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.499 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.499 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:32.499 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689587+0000", 2024-09-16T20:51:32.499 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4601999999999997e-05 2024-09-16T20:51:32.499 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.499 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:32.499 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.499 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.499 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.499 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3723 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1513472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:32.500 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:51:32.500 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 189.83783799899999, 2024-09-16T20:51:32.500 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050699691999999998, 2024-09-16T20:51:32.500 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:32.500 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:32.500 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:32.500 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:32.500 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:32.500 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:32.500 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3723 2024-09-16T20:51:32.500 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.500 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:32.500 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.500 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:32.500 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:51:32.501 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.501 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.501 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.501 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:32.501 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:51:32.501 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.501 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.501 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.501 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:32.501 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:51:32.501 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3399999999999999e-07 2024-09-16T20:51:32.501 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.501 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.502 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:32.502 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639107+0000", 2024-09-16T20:51:32.502 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3860000000000001e-06 2024-09-16T20:51:32.502 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.502 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.502 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:32.502 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639114+0000", 2024-09-16T20:51:32.502 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9800000000000001e-06 2024-09-16T20:51:32.502 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.502 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.502 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:32.502 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639129+0000", 2024-09-16T20:51:32.502 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4887e-05 2024-09-16T20:51:32.502 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.502 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.503 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:32.503 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689625+0000", 2024-09-16T20:51:32.503 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050495848000000003 2024-09-16T20:51:32.503 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.503 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.503 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:32.503 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689720+0000", 2024-09-16T20:51:32.503 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4821000000000006e-05 2024-09-16T20:51:32.503 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.503 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.503 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:32.503 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689804+0000", 2024-09-16T20:51:32.503 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4235999999999998e-05 2024-09-16T20:51:32.503 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.504 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:32.504 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.504 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.504 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.504 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3725 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1521664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:32.504 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:51:32.504 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 189.83780440300001, 2024-09-16T20:51:32.504 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050797660000000001, 2024-09-16T20:51:32.504 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:32.504 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:32.504 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:32.504 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:32.504 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:32.504 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:32.505 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3725 2024-09-16T20:51:32.505 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.505 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:32.505 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.505 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:32.505 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:51:32.505 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.505 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.505 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.505 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:32.505 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:51:32.505 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.505 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.506 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.506 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:32.506 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639139+0000", 2024-09-16T20:51:32.506 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8999999999999997e-07 2024-09-16T20:51:32.506 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.506 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.506 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:32.506 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639141+0000", 2024-09-16T20:51:32.506 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.182e-06 2024-09-16T20:51:32.506 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.506 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.506 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:32.506 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639154+0000", 2024-09-16T20:51:32.506 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3626e-05 2024-09-16T20:51:32.507 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.507 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.507 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:32.507 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639160+0000", 2024-09-16T20:51:32.507 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6810000000000003e-06 2024-09-16T20:51:32.507 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.507 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.507 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:32.507 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689831+0000", 2024-09-16T20:51:32.507 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050670907000000001 2024-09-16T20:51:32.507 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.507 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.507 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:32.507 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689882+0000", 2024-09-16T20:51:32.507 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0445999999999998e-05 2024-09-16T20:51:32.507 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.508 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.508 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:32.508 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689936+0000", 2024-09-16T20:51:32.508 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4327999999999999e-05 2024-09-16T20:51:32.508 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.508 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:32.508 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.508 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.508 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.508 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3727 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1529856~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:32.508 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:51:32.508 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 189.837768826, 2024-09-16T20:51:32.508 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051016922999999999, 2024-09-16T20:51:32.509 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:32.509 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:32.509 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:32.509 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:32.509 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:32.509 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:32.509 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3727 2024-09-16T20:51:32.509 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.509 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:32.509 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.509 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:32.509 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:51:32.509 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.509 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.510 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.510 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:32.510 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:51:32.510 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.510 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.510 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.510 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:32.510 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:51:32.510 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9999999999999998e-07 2024-09-16T20:51:32.510 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.510 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.510 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:32.510 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639183+0000", 2024-09-16T20:51:32.510 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2399999999999996e-06 2024-09-16T20:51:32.510 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.511 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.511 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:32.511 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639195+0000", 2024-09-16T20:51:32.511 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1113e-05 2024-09-16T20:51:32.511 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.511 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.511 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:32.511 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639206+0000", 2024-09-16T20:51:32.511 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1503e-05 2024-09-16T20:51:32.511 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.511 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.511 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:32.511 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689969+0000", 2024-09-16T20:51:32.511 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050762676 2024-09-16T20:51:32.511 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.512 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.512 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:32.512 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690071+0000", 2024-09-16T20:51:32.512 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102163 2024-09-16T20:51:32.512 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.512 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.512 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:32.512 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690191+0000", 2024-09-16T20:51:32.512 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000119728 2024-09-16T20:51:32.512 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.512 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:32.512 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.512 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.513 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.513 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3730 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1538048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:32.513 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:51:32.513 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 189.837727263, 2024-09-16T20:51:32.513 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051131467999999999, 2024-09-16T20:51:32.513 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:32.513 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:32.513 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:32.513 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:32.513 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:32.513 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:32.513 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3730 2024-09-16T20:51:32.513 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.513 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:32.514 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.514 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:32.514 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:51:32.514 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.514 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.514 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.514 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:32.514 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:51:32.514 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.514 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.514 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.514 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:32.514 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639216+0000", 2024-09-16T20:51:32.514 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.39e-07 2024-09-16T20:51:32.515 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.515 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.515 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:32.515 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639218+0000", 2024-09-16T20:51:32.515 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.537e-06 2024-09-16T20:51:32.515 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.515 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.515 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:32.515 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639229+0000", 2024-09-16T20:51:32.515 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1122e-05 2024-09-16T20:51:32.515 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.515 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.515 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:32.515 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639237+0000", 2024-09-16T20:51:32.515 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8509999999999999e-06 2024-09-16T20:51:32.516 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.516 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.516 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:32.516 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690227+0000", 2024-09-16T20:51:32.516 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050990147 2024-09-16T20:51:32.516 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.516 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.516 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:32.516 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690278+0000", 2024-09-16T20:51:32.516 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0223999999999999e-05 2024-09-16T20:51:32.516 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.516 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.517 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:32.517 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690347+0000", 2024-09-16T20:51:32.517 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9147999999999996e-05 2024-09-16T20:51:32.517 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.517 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:32.517 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.517 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.517 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.517 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3731 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1546240~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:32.517 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:51:32.517 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 189.83769655500001, 2024-09-16T20:51:32.517 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051276744999999999, 2024-09-16T20:51:32.517 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:32.517 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:32.518 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:32.518 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:32.518 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:32.518 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:32.518 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3731 2024-09-16T20:51:32.518 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.518 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:32.518 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.518 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:32.518 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:51:32.518 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.518 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.518 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.518 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:32.519 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:51:32.519 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.519 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.519 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.519 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:32.519 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:51:32.519 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9900000000000001e-07 2024-09-16T20:51:32.519 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.519 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.519 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:32.519 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639249+0000", 2024-09-16T20:51:32.519 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3630000000000001e-06 2024-09-16T20:51:32.519 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.520 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.520 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:32.520 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639255+0000", 2024-09-16T20:51:32.520 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4960000000000001e-06 2024-09-16T20:51:32.520 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.520 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.520 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:32.520 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639261+0000", 2024-09-16T20:51:32.520 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2839999999999997e-06 2024-09-16T20:51:32.520 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.520 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.520 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:32.520 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690375+0000", 2024-09-16T20:51:32.520 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051114684000000001 2024-09-16T20:51:32.521 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.521 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.521 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:32.521 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690425+0000", 2024-09-16T20:51:32.521 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9379999999999998e-05 2024-09-16T20:51:32.521 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.521 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.521 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:32.521 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690523+0000", 2024-09-16T20:51:32.521 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8139000000000003e-05 2024-09-16T20:51:32.521 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.521 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:32.521 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.521 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.522 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.522 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3732 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1554432~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:32.522 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:51:32.522 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 189.83765077300001, 2024-09-16T20:51:32.522 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051380096, 2024-09-16T20:51:32.522 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:32.522 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:32.522 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:32.522 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:32.522 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:32.522 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:32.522 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3732 2024-09-16T20:51:32.522 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.522 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:32.523 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.523 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:32.523 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:51:32.523 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.523 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.523 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.523 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:32.523 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:51:32.523 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.523 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.523 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.523 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:32.523 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:51:32.523 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6100000000000001e-07 2024-09-16T20:51:32.524 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.524 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.524 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:32.524 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639295+0000", 2024-09-16T20:51:32.524 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.447e-06 2024-09-16T20:51:32.524 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.524 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.524 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:32.524 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639302+0000", 2024-09-16T20:51:32.524 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0570000000000004e-06 2024-09-16T20:51:32.524 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.524 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.524 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:32.525 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639307+0000", 2024-09-16T20:51:32.525 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1429999999999997e-06 2024-09-16T20:51:32.525 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.525 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.525 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:32.525 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690555+0000", 2024-09-16T20:51:32.525 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051247963000000001 2024-09-16T20:51:32.525 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.525 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.525 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:32.525 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690602+0000", 2024-09-16T20:51:32.525 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7602000000000003e-05 2024-09-16T20:51:32.525 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.525 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.526 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:32.526 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690672+0000", 2024-09-16T20:51:32.526 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9423000000000003e-05 2024-09-16T20:51:32.526 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.526 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:32.526 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.526 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.526 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.526 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3734 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1562624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:32.526 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:51:32.526 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 189.837629123, 2024-09-16T20:51:32.526 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051490610999999999, 2024-09-16T20:51:32.526 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:32.526 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:32.527 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:32.527 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:32.527 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:32.527 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:32.527 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3734 2024-09-16T20:51:32.527 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.527 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:32.527 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.527 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:32.527 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:51:32.527 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.527 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.527 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.527 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:32.528 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:51:32.528 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.528 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.528 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.528 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:32.528 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639314+0000", 2024-09-16T20:51:32.528 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4400000000000001e-07 2024-09-16T20:51:32.528 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.528 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.528 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:32.528 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639316+0000", 2024-09-16T20:51:32.528 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1459999999999999e-06 2024-09-16T20:51:32.528 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.529 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.529 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:32.529 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639322+0000", 2024-09-16T20:51:32.529 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7610000000000004e-06 2024-09-16T20:51:32.529 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.529 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.529 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:32.529 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639327+0000", 2024-09-16T20:51:32.529 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.186e-06 2024-09-16T20:51:32.529 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.529 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.529 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:32.529 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690698+0000", 2024-09-16T20:51:32.530 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051371223000000001 2024-09-16T20:51:32.530 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.530 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.530 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:32.530 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690748+0000", 2024-09-16T20:51:32.530 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0192999999999999e-05 2024-09-16T20:51:32.530 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.530 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.530 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:32.530 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690804+0000", 2024-09-16T20:51:32.530 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5658000000000001e-05 2024-09-16T20:51:32.530 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.530 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:32.530 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.530 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.531 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.531 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3736 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1570816~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:32.531 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:51:32.531 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 189.837609864, 2024-09-16T20:51:32.531 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051596528000000003, 2024-09-16T20:51:32.531 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:32.531 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:32.531 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:32.531 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:32.531 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:32.531 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:32.531 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3736 2024-09-16T20:51:32.531 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.531 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:32.531 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.532 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:32.532 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:51:32.532 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.532 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.532 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.532 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:32.532 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:51:32.532 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.532 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.532 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.532 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:32.532 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:51:32.532 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9999999999999999e-07 2024-09-16T20:51:32.533 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.533 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.533 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:32.533 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639335+0000", 2024-09-16T20:51:32.533 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.635e-06 2024-09-16T20:51:32.533 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.533 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.533 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:32.533 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639340+0000", 2024-09-16T20:51:32.533 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5840000000000002e-06 2024-09-16T20:51:32.533 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.533 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.533 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:32.534 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639346+0000", 2024-09-16T20:51:32.534 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1190000000000002e-06 2024-09-16T20:51:32.534 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.534 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.534 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:32.534 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690817+0000", 2024-09-16T20:51:32.534 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051470721999999997 2024-09-16T20:51:32.534 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.534 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.534 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:32.534 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690875+0000", 2024-09-16T20:51:32.534 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7441000000000001e-05 2024-09-16T20:51:32.534 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.535 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.535 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:32.535 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690929+0000", 2024-09-16T20:51:32.535 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4727e-05 2024-09-16T20:51:32.535 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.535 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:32.535 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.535 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.535 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.535 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5434 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:32.535 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:51:32.535 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 189.612348298, 2024-09-16T20:51:32.535 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099755831000000003, 2024-09-16T20:51:32.535 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:32.536 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:32.536 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:32.536 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:32.536 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:32.536 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:32.536 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5434 2024-09-16T20:51:32.536 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.536 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:32.536 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.536 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:32.536 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:51:32.536 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.536 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.536 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.537 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:32.537 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:51:32.537 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.537 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.537 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.537 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:32.537 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864595+0000", 2024-09-16T20:51:32.537 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1099999999999995e-07 2024-09-16T20:51:32.537 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.537 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.537 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:32.537 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864597+0000", 2024-09-16T20:51:32.537 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3989999999999998e-06 2024-09-16T20:51:32.538 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.538 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.538 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:32.538 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864603+0000", 2024-09-16T20:51:32.538 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9309999999999996e-06 2024-09-16T20:51:32.538 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.538 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.538 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:32.538 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864607+0000", 2024-09-16T20:51:32.538 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6200000000000001e-06 2024-09-16T20:51:32.538 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.538 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.538 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:32.538 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964164+0000", 2024-09-16T20:51:32.539 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099557162000000005 2024-09-16T20:51:32.539 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.539 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.539 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:32.539 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964244+0000", 2024-09-16T20:51:32.539 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9597000000000004e-05 2024-09-16T20:51:32.539 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.539 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.539 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:32.539 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964350+0000", 2024-09-16T20:51:32.539 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010641099999999999 2024-09-16T20:51:32.539 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.539 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:32.539 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.539 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.540 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.540 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5435 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 20480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:32.540 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:51:32.540 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 189.61231847600001, 2024-09-16T20:51:32.540 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100897102, 2024-09-16T20:51:32.540 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:32.540 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:32.540 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:32.540 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:32.540 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:32.540 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:32.540 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5435 2024-09-16T20:51:32.540 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.541 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:32.541 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.541 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:32.541 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:51:32.541 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.541 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.541 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.541 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:32.541 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:51:32.541 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.541 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.541 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.541 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:32.541 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:51:32.542 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8000000000000001e-07 2024-09-16T20:51:32.542 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.542 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.542 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:32.542 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864626+0000", 2024-09-16T20:51:32.542 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.745e-06 2024-09-16T20:51:32.542 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.542 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.542 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:32.542 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864633+0000", 2024-09-16T20:51:32.542 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3339999999999998e-06 2024-09-16T20:51:32.542 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.542 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.542 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:32.543 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864636+0000", 2024-09-16T20:51:32.543 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1200000000000002e-06 2024-09-16T20:51:32.543 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.543 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.543 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:32.543 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965299+0000", 2024-09-16T20:51:32.543 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10066352000000001 2024-09-16T20:51:32.543 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.543 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.543 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:32.543 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965390+0000", 2024-09-16T20:51:32.543 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0894000000000001e-05 2024-09-16T20:51:32.543 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.543 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.543 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:32.544 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965521+0000", 2024-09-16T20:51:32.544 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013110899999999999 2024-09-16T20:51:32.544 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.544 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:32.544 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.544 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.544 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.544 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:152 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 327680~32768 out=32768b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:51:32.544 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:51:32.544 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 188.313821534, 2024-09-16T20:51:32.544 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11869405800000001, 2024-09-16T20:51:32.544 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:32.544 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:32.545 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:32.545 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:32.545 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:51:32.545 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:51:32.545 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 152 2024-09-16T20:51:32.545 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.545 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:32.545 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.545 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:32.545 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:51:32.545 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.545 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.545 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.545 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:32.546 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:51:32.546 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.546 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.546 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.546 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:32.546 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163122+0000", 2024-09-16T20:51:32.546 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6400000000000003e-07 2024-09-16T20:51:32.546 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.546 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.546 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:32.546 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163124+0000", 2024-09-16T20:51:32.546 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.565e-06 2024-09-16T20:51:32.546 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.546 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.547 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:32.547 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163136+0000", 2024-09-16T20:51:32.547 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1717e-05 2024-09-16T20:51:32.547 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.547 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.547 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:32.547 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163142+0000", 2024-09-16T20:51:32.547 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1750000000000002e-06 2024-09-16T20:51:32.547 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.547 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.547 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:32.547 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163606+0000", 2024-09-16T20:51:32.547 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00046418799999999998 2024-09-16T20:51:32.547 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.548 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.548 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:32.548 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163656+0000", 2024-09-16T20:51:32.548 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9852999999999999e-05 2024-09-16T20:51:32.548 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.548 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.548 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:32.548 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281815+0000", 2024-09-16T20:51:32.548 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11815909600000001 2024-09-16T20:51:32.548 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.548 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:32.548 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.548 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.548 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.549 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:153 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:51:32.549 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:51:32.549 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 188.31378858100001, 2024-09-16T20:51:32.549 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11903462400000001, 2024-09-16T20:51:32.549 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:32.549 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:32.549 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:32.549 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:32.549 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:51:32.549 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:51:32.549 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 153 2024-09-16T20:51:32.549 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.549 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:32.550 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.550 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:32.550 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:51:32.550 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.550 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.550 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.550 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:32.550 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:51:32.550 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.550 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.550 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.550 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:32.550 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163155+0000", 2024-09-16T20:51:32.550 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.37e-07 2024-09-16T20:51:32.551 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.551 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.551 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:32.551 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163157+0000", 2024-09-16T20:51:32.551 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6010000000000002e-06 2024-09-16T20:51:32.551 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.551 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.551 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:32.551 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163165+0000", 2024-09-16T20:51:32.551 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2619999999999998e-06 2024-09-16T20:51:32.551 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.551 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.551 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:32.551 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163171+0000", 2024-09-16T20:51:32.552 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.711e-06 2024-09-16T20:51:32.552 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.552 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.552 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:32.552 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281868+0000", 2024-09-16T20:51:32.552 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118696622 2024-09-16T20:51:32.552 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.552 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.552 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:32.552 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281969+0000", 2024-09-16T20:51:32.552 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101371 2024-09-16T20:51:32.552 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.552 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.552 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:32.553 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282189+0000", 2024-09-16T20:51:32.553 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00021952000000000001 2024-09-16T20:51:32.553 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.553 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:32.553 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.553 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.553 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.553 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:154 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 360448~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:51:32.553 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:51:32.553 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 188.31375680900001, 2024-09-16T20:51:32.553 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11932035100000001, 2024-09-16T20:51:32.553 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:32.553 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:32.553 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:32.554 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:32.554 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:51:32.554 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:51:32.554 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 154 2024-09-16T20:51:32.554 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.554 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:32.554 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.554 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:32.554 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:51:32.554 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.554 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.554 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.554 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:32.555 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:51:32.555 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.555 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.555 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.555 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:32.555 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:51:32.555 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:51:32.555 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.555 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.555 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:32.555 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163189+0000", 2024-09-16T20:51:32.555 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6840000000000001e-06 2024-09-16T20:51:32.555 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.555 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.556 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:32.556 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163196+0000", 2024-09-16T20:51:32.556 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7920000000000004e-06 2024-09-16T20:51:32.556 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.556 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.556 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:32.556 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163201+0000", 2024-09-16T20:51:32.556 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2809999999999999e-06 2024-09-16T20:51:32.556 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.556 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.556 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:32.556 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282219+0000", 2024-09-16T20:51:32.556 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.119018343 2024-09-16T20:51:32.556 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.557 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.557 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:32.557 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282305+0000", 2024-09-16T20:51:32.557 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5865000000000005e-05 2024-09-16T20:51:32.557 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.557 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.557 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:32.557 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282506+0000", 2024-09-16T20:51:32.557 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000200927 2024-09-16T20:51:32.557 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.557 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:32.557 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.557 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.557 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.558 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop(client.4366.0:3088 3.1e e38/23 3:79e5ec01:::rbd_header.110e891a9b2b:head v 38'306, mlcod=36'304)", 2024-09-16T20:51:32.558 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:51:32.558 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 188.31273794099999, 2024-09-16T20:51:32.558 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118102759, 2024-09-16T20:51:32.558 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:32.558 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:32.558 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:32.558 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:32.558 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.558 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:32.558 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:51:32.559 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.559 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.559 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.559 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:32.559 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:51:32.559 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.559 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.559 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.559 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:32.559 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:51:32.559 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.85e-07 2024-09-16T20:51:32.559 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.559 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.560 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:32.560 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164212+0000", 2024-09-16T20:51:32.560 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3999999999999997e-06 2024-09-16T20:51:32.560 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.560 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.560 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:32.560 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281553+0000", 2024-09-16T20:51:32.560 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11734135499999999 2024-09-16T20:51:32.560 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.560 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.560 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:32.560 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281567+0000", 2024-09-16T20:51:32.560 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4188999999999999e-05 2024-09-16T20:51:32.561 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.561 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.561 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:32.561 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281653+0000", 2024-09-16T20:51:32.561 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5385999999999999e-05 2024-09-16T20:51:32.561 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.561 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.561 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:32.561 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281679+0000", 2024-09-16T20:51:32.561 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6689000000000001e-05 2024-09-16T20:51:32.561 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.561 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.561 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:32.562 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282277+0000", 2024-09-16T20:51:32.562 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00059798000000000002 2024-09-16T20:51:32.562 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.562 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.562 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:32.562 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282307+0000", 2024-09-16T20:51:32.562 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9975e-05 2024-09-16T20:51:32.562 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.562 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:32.562 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.562 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.562 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.562 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5255.0:44 3.16 3:6f5476df:::rbd_header.110e25db1f60:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:51:32.562 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:51:32.562 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 116.156434886, 2024-09-16T20:51:32.563 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.121007509, 2024-09-16T20:51:32.563 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:32.563 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:32.563 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:32.563 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:32.563 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5255", 2024-09-16T20:51:32.563 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:51:32.563 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 44 2024-09-16T20:51:32.563 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.563 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:32.563 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.563 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:32.563 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:51:32.564 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.564 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.564 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.564 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:32.564 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:51:32.564 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.564 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.564 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.564 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:32.564 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:51:32.564 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0500000000000003e-07 2024-09-16T20:51:32.564 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.564 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.564 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:32.565 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320512+0000", 2024-09-16T20:51:32.565 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2540000000000001e-06 2024-09-16T20:51:32.565 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.565 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.565 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:32.565 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320518+0000", 2024-09-16T20:51:32.565 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1689999999999997e-06 2024-09-16T20:51:32.565 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.565 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.565 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:32.565 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320521+0000", 2024-09-16T20:51:32.565 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5200000000000002e-06 2024-09-16T20:51:32.565 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.566 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.566 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:32.566 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320548+0000", 2024-09-16T20:51:32.566 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6662999999999999e-05 2024-09-16T20:51:32.566 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.566 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.566 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:32.566 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320587+0000", 2024-09-16T20:51:32.566 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8866999999999999e-05 2024-09-16T20:51:32.566 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.566 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.566 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:32.567 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.441515+0000", 2024-09-16T20:51:32.567 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.120928331 2024-09-16T20:51:32.567 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.567 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:32.567 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.567 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.567 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.567 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:20329 3.5 3:a4ea64b7:::rbd_header.110ede5ef857:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:32.567 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:01.061694+0000", 2024-09-16T20:51:32.567 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 91.415248317999996, 2024-09-16T20:51:32.567 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.071234077000000007, 2024-09-16T20:51:32.567 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:32.568 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:32.568 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:32.568 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:32.568 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:51:32.568 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:32.568 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 20329 2024-09-16T20:51:32.568 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.568 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:32.568 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.568 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:32.568 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.061694+0000", 2024-09-16T20:51:32.568 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.569 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.569 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.569 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:32.569 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.061694+0000", 2024-09-16T20:51:32.569 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:32.569 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.569 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.569 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:32.569 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089232+0000", 2024-09-16T20:51:32.569 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.027537275 2024-09-16T20:51:32.569 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.569 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.569 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:32.569 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089237+0000", 2024-09-16T20:51:32.570 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4709999999999998e-06 2024-09-16T20:51:32.570 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.570 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.570 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:32.570 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089246+0000", 2024-09-16T20:51:32.570 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8230000000000001e-06 2024-09-16T20:51:32.570 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.570 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.570 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:32.570 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089252+0000", 2024-09-16T20:51:32.570 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3060000000000003e-06 2024-09-16T20:51:32.570 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.570 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.570 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:32.570 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.132756+0000", 2024-09-16T20:51:32.571 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.043504334999999998 2024-09-16T20:51:32.571 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.571 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.571 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:32.571 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.132829+0000", 2024-09-16T20:51:32.571 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2849999999999995e-05 2024-09-16T20:51:32.571 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:32.571 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:32.571 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:32.571 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.132928+0000", 2024-09-16T20:51:32.571 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.9017000000000004e-05 2024-09-16T20:51:32.571 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.571 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:32.571 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.572 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:32.572 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:32.572 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:32.573 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:51:32.574 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:32.572+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:32.673 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:32.672+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:32.700 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:32.700 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:51:32.700 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:51:32.700 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:32.711 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:51:32.774 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:32.773+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:32.819 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:32.819 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:51:32.819 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:51:32.819 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:51:32.819 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:32.828 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:51:32.854 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T20:51:32.898 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:32.897+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:32.977 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:32.977 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:51:32.977 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:51:32.977 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:51:32.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:32.977 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:3088 3.1e 3:79e5ec01:::rbd_header.110e891a9b2b:head [watch watch cookie 140213437171488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:51:32.977 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:51:32.977 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 188.81089625000001, 2024-09-16T20:51:32.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.11895186200000001, 2024-09-16T20:51:32.978 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:32.978 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:32.978 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:32.978 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:32.978 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:32.978 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:32.978 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3088 2024-09-16T20:51:32.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:32.978 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:32.978 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:32.978 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:32.978 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:51:32.979 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:32.979 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:32.979 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:32.979 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:32.979 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:51:32.979 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:32.979 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:32.979 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:32.979 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:32.979 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163846+0000", 2024-09-16T20:51:32.979 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6799999999999999e-07 2024-09-16T20:51:32.979 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:32.980 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:32.980 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:32.980 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163850+0000", 2024-09-16T20:51:32.980 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.929e-06 2024-09-16T20:51:32.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:32.980 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:32.980 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:32.980 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163862+0000", 2024-09-16T20:51:32.980 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1972e-05 2024-09-16T20:51:32.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:32.980 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:32.981 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:32.981 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163867+0000", 2024-09-16T20:51:32.981 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.451e-06 2024-09-16T20:51:32.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:32.981 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:32.981 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:32.981 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163912+0000", 2024-09-16T20:51:32.981 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4456000000000003e-05 2024-09-16T20:51:32.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:32.981 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:32.981 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:32.981 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163962+0000", 2024-09-16T20:51:32.981 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0578000000000003e-05 2024-09-16T20:51:32.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:32.982 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:32.982 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-16T20:51:32.982 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164058+0000", 2024-09-16T20:51:32.982 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5934999999999995e-05 2024-09-16T20:51:32.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:32.982 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:32.982 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:51:32.982 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164570+0000", 2024-09-16T20:51:32.982 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00051241100000000005 2024-09-16T20:51:32.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:32.982 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:32.982 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:32.982 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282629+0000", 2024-09-16T20:51:32.982 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.118058986 2024-09-16T20:51:32.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:32.983 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:32.983 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:51:32.983 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282725+0000", 2024-09-16T20:51:32.983 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5063999999999996e-05 2024-09-16T20:51:32.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:32.983 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:32.983 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:32.983 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282797+0000", 2024-09-16T20:51:32.983 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2212000000000006e-05 2024-09-16T20:51:32.983 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:32.983 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:32.983 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:32.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:32.984 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:32.984 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:8039 3.1e 3.ac44ff9e (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:51:32.984 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:51:32.984 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 163.26706890099999, 2024-09-16T20:51:32.984 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0053310449999999, 2024-09-16T20:51:32.984 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:32.984 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:32.984 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:32.984 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:32.984 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:32.984 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:32.984 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8039 2024-09-16T20:51:32.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:32.985 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:32.985 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:32.985 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:32.985 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:51:32.985 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:32.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:32.985 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:32.985 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:32.985 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:51:32.985 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:32.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:32.985 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:32.985 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:32.985 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707674+0000", 2024-09-16T20:51:32.986 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6530000000000001e-06 2024-09-16T20:51:32.986 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:32.986 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:32.986 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:32.986 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707681+0000", 2024-09-16T20:51:32.986 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9639999999999997e-06 2024-09-16T20:51:32.986 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:32.986 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:32.986 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:32.986 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707700+0000", 2024-09-16T20:51:32.987 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8674000000000001e-05 2024-09-16T20:51:32.987 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:32.987 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:32.987 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:32.987 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707708+0000", 2024-09-16T20:51:32.987 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6810000000000004e-06 2024-09-16T20:51:32.987 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:32.987 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:32.987 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:32.987 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711293+0000", 2024-09-16T20:51:32.987 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035848189999999999 2024-09-16T20:51:32.987 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:32.987 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:32.987 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:32.988 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711301+0000", 2024-09-16T20:51:32.988 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4730000000000003e-06 2024-09-16T20:51:32.988 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:32.988 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:32.988 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:32.988 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.712990+0000", 2024-09-16T20:51:32.988 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.001689348 2024-09-16T20:51:32.988 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:32.988 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:32.988 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:32.988 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713003+0000", 2024-09-16T20:51:32.988 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3433e-05 2024-09-16T20:51:32.988 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:32.988 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:32.988 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:32.989 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:32.989 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:32.989 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:8040 3.1e 3.ac44ff9e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:51:32.989 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:51:32.989 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 163.267008506, 2024-09-16T20:51:32.989 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0053410780000001, 2024-09-16T20:51:32.989 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:32.989 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:32.989 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:32.989 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:32.989 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:32.989 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:32.989 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8040 2024-09-16T20:51:32.990 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:32.990 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:32.990 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:32.990 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:32.990 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:51:32.990 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:32.990 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:32.990 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:32.990 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:32.990 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:51:32.990 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:32.990 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:32.990 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:32.990 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:32.991 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:51:32.991 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8100000000000003e-07 2024-09-16T20:51:32.991 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:32.991 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:32.991 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:32.991 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707737+0000", 2024-09-16T20:51:32.991 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5020000000000001e-06 2024-09-16T20:51:32.991 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:32.991 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:32.991 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:32.991 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707745+0000", 2024-09-16T20:51:32.991 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2169999999999994e-06 2024-09-16T20:51:32.991 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:32.991 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:32.992 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:32.992 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707751+0000", 2024-09-16T20:51:32.992 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.263e-06 2024-09-16T20:51:32.992 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:32.992 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:32.992 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:32.992 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711336+0000", 2024-09-16T20:51:32.992 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035848360000000001 2024-09-16T20:51:32.992 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:32.992 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:32.992 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:32.992 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711340+0000", 2024-09-16T20:51:32.992 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1309999999999999e-06 2024-09-16T20:51:32.993 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:32.993 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:32.993 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:32.993 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713060+0000", 2024-09-16T20:51:32.993 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0017195059999999 2024-09-16T20:51:32.993 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:32.993 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:32.993 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:32.993 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713074+0000", 2024-09-16T20:51:32.993 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4042e-05 2024-09-16T20:51:32.993 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:32.993 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:32.994 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:32.994 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:32.994 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:32.994 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4927.0:4487 3.1e 3.ac44ff9e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:51:32.994 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:51:32.994 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 163.26648830400001, 2024-09-16T20:51:32.994 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0048840880000001, 2024-09-16T20:51:32.994 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:32.994 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:32.994 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:32.994 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:32.994 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4927", 2024-09-16T20:51:32.994 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:32.994 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4487 2024-09-16T20:51:32.995 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:32.995 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:32.995 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:32.995 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:32.995 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:51:32.995 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:32.995 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:32.995 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:32.995 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:32.995 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:51:32.995 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:32.995 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:32.995 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:32.996 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:32.996 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708254+0000", 2024-09-16T20:51:32.996 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1800000000000005e-07 2024-09-16T20:51:32.996 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:32.996 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:32.996 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:32.996 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708258+0000", 2024-09-16T20:51:32.996 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8689999999999997e-06 2024-09-16T20:51:32.996 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:32.996 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:32.996 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:32.996 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708267+0000", 2024-09-16T20:51:32.996 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9590000000000007e-06 2024-09-16T20:51:32.997 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:32.997 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:32.997 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:32.997 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708274+0000", 2024-09-16T20:51:32.997 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9970000000000001e-06 2024-09-16T20:51:32.997 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:32.997 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:32.997 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:32.997 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711366+0000", 2024-09-16T20:51:32.997 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030919530000000001 2024-09-16T20:51:32.997 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:32.997 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:32.997 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:32.997 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711372+0000", 2024-09-16T20:51:32.998 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9410000000000004e-06 2024-09-16T20:51:32.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:32.998 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:32.998 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:32.998 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713126+0000", 2024-09-16T20:51:32.998 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0017537249999999 2024-09-16T20:51:32.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:32.998 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:32.998 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:32.998 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713137+0000", 2024-09-16T20:51:32.998 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0825999999999999e-05 2024-09-16T20:51:32.998 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:32.998 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:32.999 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:32.999 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:32.999 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:32.999 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5089.0:11703 3.1b 3:dcd4684e:::rbd_data.110e747f0779.000000000000001b:head [set-alloc-hint object_size 262144 write_size 262144,write 139264~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:51:32.999 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:51:32.999 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 148.83675253000001, 2024-09-16T20:51:32.999 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.084231385000000006, 2024-09-16T20:51:32.999 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:32.999 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:32.999 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:32.999 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:32.999 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5089", 2024-09-16T20:51:32.999 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:51:33.000 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 11703 2024-09-16T20:51:33.000 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.000 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:33.000 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.000 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:33.000 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:51:33.000 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:33.000 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.000 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.000 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:33.000 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:51:33.000 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:33.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.001 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.001 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:33.001 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:51:33.001 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2600000000000001e-07 2024-09-16T20:51:33.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.001 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.001 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:33.001 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137998+0000", 2024-09-16T20:51:33.001 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9549999999999998e-06 2024-09-16T20:51:33.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.001 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.001 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:33.001 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138008+0000", 2024-09-16T20:51:33.002 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3789999999999995e-06 2024-09-16T20:51:33.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.002 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:33.002 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138013+0000", 2024-09-16T20:51:33.002 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2209999999999996e-06 2024-09-16T20:51:33.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.002 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:33.002 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138054+0000", 2024-09-16T20:51:33.002 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1022999999999997e-05 2024-09-16T20:51:33.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.002 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:33.003 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138124+0000", 2024-09-16T20:51:33.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0234999999999994e-05 2024-09-16T20:51:33.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.003 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.003 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:33.003 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138197+0000", 2024-09-16T20:51:33.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2451000000000001e-05 2024-09-16T20:51:33.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.003 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.003 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:33.003 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222010+0000", 2024-09-16T20:51:33.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.083813433000000007 2024-09-16T20:51:33.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.003 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.004 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:51:33.004 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222163+0000", 2024-09-16T20:51:33.004 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015274599999999999 2024-09-16T20:51:33.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.004 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.004 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:51:33.004 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222197+0000", 2024-09-16T20:51:33.004 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3683000000000003e-05 2024-09-16T20:51:33.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.004 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.004 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:33.004 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222220+0000", 2024-09-16T20:51:33.004 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3433000000000001e-05 2024-09-16T20:51:33.005 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:33.005 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:33.005 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:33.005 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.005 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.005 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.5089.0:11702 3.15 e73/64 3:acf33c8b:::rbd_data.110e747f0779.000000000000001a:head v 73'132, mlcod=73'130)", 2024-09-16T20:51:33.005 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:51:33.005 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 148.83648520099999, 2024-09-16T20:51:33.005 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.084079763000000002, 2024-09-16T20:51:33.005 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:33.005 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:33.005 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:33.005 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:33.006 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.006 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:33.006 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:51:33.006 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:33.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.006 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.006 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:33.006 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:51:33.006 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:33.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.006 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.006 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:33.006 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:51:33.006 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:51:33.007 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.007 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.007 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:33.007 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138263+0000", 2024-09-16T20:51:33.007 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1e-06 2024-09-16T20:51:33.007 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.007 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.007 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:33.007 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221589+0000", 2024-09-16T20:51:33.007 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.083326002999999996 2024-09-16T20:51:33.007 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.007 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.007 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:33.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221614+0000", 2024-09-16T20:51:33.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5593999999999999e-05 2024-09-16T20:51:33.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.008 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:33.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221714+0000", 2024-09-16T20:51:33.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9889000000000005e-05 2024-09-16T20:51:33.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.008 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:33.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221727+0000", 2024-09-16T20:51:33.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3125999999999999e-05 2024-09-16T20:51:33.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.009 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.009 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:51:33.009 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222294+0000", 2024-09-16T20:51:33.009 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00056712199999999998 2024-09-16T20:51:33.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.009 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.009 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:33.009 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222336+0000", 2024-09-16T20:51:33.009 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1470000000000001e-05 2024-09-16T20:51:33.009 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:33.009 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:33.009 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:33.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.010 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.010 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5138.0:9358 3.c 3:33996a43:::rbd_data.110e6372565d.0000000000000007:head [read 1343488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T20:51:33.010 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:51:33.010 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 125.122209791, 2024-09-16T20:51:33.010 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0167493569999999, 2024-09-16T20:51:33.010 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:33.010 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:33.010 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:33.010 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:33.010 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5138", 2024-09-16T20:51:33.010 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:51:33.010 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9358 2024-09-16T20:51:33.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.011 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:33.011 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.011 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:33.011 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:51:33.011 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:33.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.011 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.011 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:33.011 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:51:33.011 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:33.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.011 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.011 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:33.011 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852533+0000", 2024-09-16T20:51:33.012 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.646e-06 2024-09-16T20:51:33.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.012 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.012 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:33.012 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852539+0000", 2024-09-16T20:51:33.012 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3389999999999998e-06 2024-09-16T20:51:33.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.012 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.012 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:33.012 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852552+0000", 2024-09-16T20:51:33.012 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.271e-05 2024-09-16T20:51:33.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.012 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.012 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:33.013 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852563+0000", 2024-09-16T20:51:33.013 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1139e-05 2024-09-16T20:51:33.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.013 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.013 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:33.013 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852883+0000", 2024-09-16T20:51:33.013 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00032010600000000001 2024-09-16T20:51:33.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.013 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.013 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:33.013 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852890+0000", 2024-09-16T20:51:33.013 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7669999999999998e-06 2024-09-16T20:51:33.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.014 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.014 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:33.014 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869059+0000", 2024-09-16T20:51:33.014 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.016169353 2024-09-16T20:51:33.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.014 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.014 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:33.014 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869178+0000", 2024-09-16T20:51:33.014 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118193 2024-09-16T20:51:33.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.014 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.014 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:33.015 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869281+0000", 2024-09-16T20:51:33.015 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010310399999999999 2024-09-16T20:51:33.015 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:33.015 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:33.015 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:33.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.015 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.015 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5138.0:9382 3.c 3:33996a43:::rbd_data.110e6372565d.0000000000000007:head [read 0~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T20:51:33.015 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:51:33.015 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 125.122158914, 2024-09-16T20:51:33.015 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.017136034, 2024-09-16T20:51:33.015 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:33.015 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:33.015 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:33.016 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:33.016 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5138", 2024-09-16T20:51:33.016 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:51:33.016 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9382 2024-09-16T20:51:33.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.016 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:33.016 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.016 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:33.016 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:51:33.016 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:33.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.016 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.016 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:33.016 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:51:33.016 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:33.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.017 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.017 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:33.017 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852583+0000", 2024-09-16T20:51:33.017 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.51e-07 2024-09-16T20:51:33.017 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.017 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.017 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:33.017 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852586+0000", 2024-09-16T20:51:33.017 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0280000000000001e-06 2024-09-16T20:51:33.017 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.017 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.017 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:33.017 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852595+0000", 2024-09-16T20:51:33.017 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2799999999999992e-06 2024-09-16T20:51:33.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.018 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.018 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:33.018 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852601+0000", 2024-09-16T20:51:33.018 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3810000000000004e-06 2024-09-16T20:51:33.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.018 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.018 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:33.018 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852921+0000", 2024-09-16T20:51:33.018 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00031979699999999997 2024-09-16T20:51:33.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.018 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.018 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:33.018 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852930+0000", 2024-09-16T20:51:33.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6489999999999997e-06 2024-09-16T20:51:33.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.019 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:33.019 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869434+0000", 2024-09-16T20:51:33.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0165038829999999 2024-09-16T20:51:33.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.019 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:33.019 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869495+0000", 2024-09-16T20:51:33.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1509999999999999e-05 2024-09-16T20:51:33.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.019 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:33.019 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869718+0000", 2024-09-16T20:51:33.020 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000222955 2024-09-16T20:51:33.020 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:33.020 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:33.020 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:33.020 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.020 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.020 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9971 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 643072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:33.020 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642102+0000", 2024-09-16T20:51:33.020 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 102.33263877, 2024-09-16T20:51:33.020 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077421016999999995, 2024-09-16T20:51:33.020 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:33.020 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:33.020 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:33.020 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:33.020 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:33.021 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:33.021 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9971 2024-09-16T20:51:33.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.021 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:33.021 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.021 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:33.021 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642102+0000", 2024-09-16T20:51:33.021 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:33.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.021 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.021 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:33.021 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642102+0000", 2024-09-16T20:51:33.021 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:33.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.022 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.022 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:33.022 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642103+0000", 2024-09-16T20:51:33.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7399999999999998e-07 2024-09-16T20:51:33.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.022 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.022 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:33.022 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642105+0000", 2024-09-16T20:51:33.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2649999999999999e-06 2024-09-16T20:51:33.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.022 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.022 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:33.022 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642112+0000", 2024-09-16T20:51:33.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.294e-06 2024-09-16T20:51:33.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.023 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.023 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:33.023 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642118+0000", 2024-09-16T20:51:33.023 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4500000000000003e-06 2024-09-16T20:51:33.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.023 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.023 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:33.023 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719285+0000", 2024-09-16T20:51:33.023 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077166883000000006 2024-09-16T20:51:33.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.023 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.023 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:33.023 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719368+0000", 2024-09-16T20:51:33.023 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2811000000000004e-05 2024-09-16T20:51:33.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.024 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:33.024 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719523+0000", 2024-09-16T20:51:33.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015584 2024-09-16T20:51:33.024 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:33.024 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:33.024 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:33.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.024 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9972 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 648192~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:33.024 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642125+0000", 2024-09-16T20:51:33.024 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 102.332615823, 2024-09-16T20:51:33.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077710324999999997, 2024-09-16T20:51:33.025 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:33.025 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:33.025 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:33.025 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:33.025 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:33.025 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:33.025 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9972 2024-09-16T20:51:33.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.025 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:33.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:33.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642125+0000", 2024-09-16T20:51:33.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:33.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.026 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:33.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642125+0000", 2024-09-16T20:51:33.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:33.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.026 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:33.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642126+0000", 2024-09-16T20:51:33.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1899999999999998e-07 2024-09-16T20:51:33.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.026 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:33.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642128+0000", 2024-09-16T20:51:33.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8619999999999999e-06 2024-09-16T20:51:33.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.027 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.027 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:33.027 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642134+0000", 2024-09-16T20:51:33.027 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5289999999999997e-06 2024-09-16T20:51:33.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.027 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.027 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:33.027 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642139+0000", 2024-09-16T20:51:33.027 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1250000000000001e-06 2024-09-16T20:51:33.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.027 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.027 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:33.027 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719570+0000", 2024-09-16T20:51:33.027 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077430516000000005 2024-09-16T20:51:33.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.028 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.028 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:33.028 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719655+0000", 2024-09-16T20:51:33.028 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5500000000000005e-05 2024-09-16T20:51:33.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.028 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.028 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:33.028 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719836+0000", 2024-09-16T20:51:33.028 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018037399999999999 2024-09-16T20:51:33.028 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:33.028 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:33.028 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:33.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.029 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9973 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 653312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:33.029 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:51:33.029 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 102.332595213, 2024-09-16T20:51:33.029 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077905262000000003, 2024-09-16T20:51:33.029 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:33.029 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:33.029 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:33.029 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:33.029 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:33.029 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:33.029 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9973 2024-09-16T20:51:33.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.029 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:33.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.030 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:33.030 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:51:33.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:33.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.030 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:33.030 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:51:33.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:33.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.030 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:33.030 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:51:33.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:51:33.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.031 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.031 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:33.031 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642148+0000", 2024-09-16T20:51:33.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0219999999999999e-06 2024-09-16T20:51:33.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.031 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.031 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:33.031 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642154+0000", 2024-09-16T20:51:33.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8540000000000002e-06 2024-09-16T20:51:33.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.031 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.031 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:33.031 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642159+0000", 2024-09-16T20:51:33.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.045e-06 2024-09-16T20:51:33.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.031 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.032 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:33.032 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719875+0000", 2024-09-16T20:51:33.032 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077716116000000002 2024-09-16T20:51:33.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.032 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:33.032 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719934+0000", 2024-09-16T20:51:33.032 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8819e-05 2024-09-16T20:51:33.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.032 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:33.032 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720051+0000", 2024-09-16T20:51:33.032 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000116947 2024-09-16T20:51:33.032 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:33.033 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:33.033 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:33.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.033 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.033 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9974 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 658432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:33.033 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642166+0000", 2024-09-16T20:51:33.033 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 102.332575299, 2024-09-16T20:51:33.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.078179786000000001, 2024-09-16T20:51:33.033 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:33.033 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:33.033 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:33.033 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:33.033 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:33.033 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:33.033 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9974 2024-09-16T20:51:33.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.034 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:33.034 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.034 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:33.034 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642166+0000", 2024-09-16T20:51:33.034 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:33.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.034 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.034 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:33.034 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642166+0000", 2024-09-16T20:51:33.034 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:33.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.034 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.034 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:33.034 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647343+0000", 2024-09-16T20:51:33.035 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0051769440000000002 2024-09-16T20:51:33.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.035 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:33.035 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647348+0000", 2024-09-16T20:51:33.035 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3619999999999997e-06 2024-09-16T20:51:33.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.035 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:33.035 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647359+0000", 2024-09-16T20:51:33.035 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0929e-05 2024-09-16T20:51:33.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.035 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:33.035 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647366+0000", 2024-09-16T20:51:33.035 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0910000000000004e-06 2024-09-16T20:51:33.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.036 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:33.036 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720094+0000", 2024-09-16T20:51:33.036 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.072727403999999995 2024-09-16T20:51:33.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.036 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:33.036 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720173+0000", 2024-09-16T20:51:33.036 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9382000000000007e-05 2024-09-16T20:51:33.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.036 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:33.036 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720346+0000", 2024-09-16T20:51:33.036 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017267399999999999 2024-09-16T20:51:33.036 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:33.037 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:33.037 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:33.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.037 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.037 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9991 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 745472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:33.037 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:51:33.037 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 102.327005906, 2024-09-16T20:51:33.037 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077257820000000005, 2024-09-16T20:51:33.037 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:33.037 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:33.037 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:33.037 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:33.037 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:33.037 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:33.037 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9991 2024-09-16T20:51:33.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.038 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:33.038 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.038 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:33.038 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:51:33.038 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:33.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.038 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.038 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:33.038 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:51:33.038 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:33.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.038 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.039 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:33.039 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:51:33.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1199999999999999e-07 2024-09-16T20:51:33.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.039 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:33.039 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647736+0000", 2024-09-16T20:51:33.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3699999999999999e-07 2024-09-16T20:51:33.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.039 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:33.039 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647739+0000", 2024-09-16T20:51:33.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9359999999999999e-06 2024-09-16T20:51:33.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.040 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:33.040 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647742+0000", 2024-09-16T20:51:33.040 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5720000000000001e-06 2024-09-16T20:51:33.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.040 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.040 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:33.040 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724774+0000", 2024-09-16T20:51:33.040 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077032457999999998 2024-09-16T20:51:33.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.040 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.040 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:33.040 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724818+0000", 2024-09-16T20:51:33.040 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3803000000000001e-05 2024-09-16T20:51:33.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.040 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.041 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:33.041 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724993+0000", 2024-09-16T20:51:33.041 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000174902 2024-09-16T20:51:33.041 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:33.041 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:33.041 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:33.041 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.041 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.041 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9992 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 750592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:33.041 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:51:33.041 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 102.326996317, 2024-09-16T20:51:33.041 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077553227000000002, 2024-09-16T20:51:33.041 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:33.041 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:33.042 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:33.042 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:33.042 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:33.042 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:33.042 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9992 2024-09-16T20:51:33.042 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.042 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:33.042 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.042 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:33.042 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:51:33.042 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:33.042 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.043 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.043 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:33.043 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:51:33.043 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:33.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.043 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.043 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:33.043 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:51:33.043 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0200000000000001e-07 2024-09-16T20:51:33.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.043 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.043 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:33.043 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647746+0000", 2024-09-16T20:51:33.043 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.088e-06 2024-09-16T20:51:33.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.044 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.044 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:33.044 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647749+0000", 2024-09-16T20:51:33.044 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9170000000000002e-06 2024-09-16T20:51:33.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.044 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.044 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:33.044 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647752+0000", 2024-09-16T20:51:33.044 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.774e-06 2024-09-16T20:51:33.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.044 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.044 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:33.044 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725042+0000", 2024-09-16T20:51:33.044 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077289952999999995 2024-09-16T20:51:33.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.045 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.045 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:33.045 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725113+0000", 2024-09-16T20:51:33.045 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0920999999999994e-05 2024-09-16T20:51:33.045 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.045 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.045 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:33.045 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725298+0000", 2024-09-16T20:51:33.045 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000185372 2024-09-16T20:51:33.045 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:33.045 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:33.045 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:33.045 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.045 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.045 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9993 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 755712~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:33.046 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647755+0000", 2024-09-16T20:51:33.046 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 102.326986291, 2024-09-16T20:51:33.046 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077725674999999994, 2024-09-16T20:51:33.046 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:33.046 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:33.046 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:33.046 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:33.046 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:33.046 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:33.046 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9993 2024-09-16T20:51:33.046 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.046 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:33.046 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.046 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:33.047 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647755+0000", 2024-09-16T20:51:33.047 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:33.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.047 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.047 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:33.047 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647755+0000", 2024-09-16T20:51:33.047 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:33.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.047 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.047 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:33.047 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653406+0000", 2024-09-16T20:51:33.047 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.005650876 2024-09-16T20:51:33.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.047 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.048 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:33.048 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653411+0000", 2024-09-16T20:51:33.048 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8899999999999998e-06 2024-09-16T20:51:33.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.048 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.048 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:33.048 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653423+0000", 2024-09-16T20:51:33.048 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2447e-05 2024-09-16T20:51:33.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.048 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.048 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:33.048 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653431+0000", 2024-09-16T20:51:33.048 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5730000000000002e-06 2024-09-16T20:51:33.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.049 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.049 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:33.049 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725333+0000", 2024-09-16T20:51:33.049 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.071902743000000005 2024-09-16T20:51:33.049 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.049 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.049 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:33.049 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725385+0000", 2024-09-16T20:51:33.049 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1585000000000002e-05 2024-09-16T20:51:33.049 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.049 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.049 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:33.049 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725481+0000", 2024-09-16T20:51:33.049 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5561000000000002e-05 2024-09-16T20:51:33.049 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:33.050 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:33.050 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:33.050 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.050 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.050 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6061.0:5400 3.b 3:d353d652:::rbd_data.110e16eafbd.0000000000000010:head [read 577536~28672] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-16T20:51:33.050 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:27.761367+0000", 2024-09-16T20:51:33.050 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 65.213374345000005, 2024-09-16T20:51:33.050 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0029066630000001, 2024-09-16T20:51:33.050 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:33.050 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:33.050 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:33.050 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:33.050 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6061", 2024-09-16T20:51:33.051 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3930128440", 2024-09-16T20:51:33.051 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5400 2024-09-16T20:51:33.051 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.051 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:33.051 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.051 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:33.051 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761367+0000", 2024-09-16T20:51:33.051 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:33.051 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.051 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.051 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:33.051 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761367+0000", 2024-09-16T20:51:33.051 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:33.051 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.052 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.052 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:33.052 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761369+0000", 2024-09-16T20:51:33.052 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.934e-06 2024-09-16T20:51:33.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.052 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.052 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:33.052 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761374+0000", 2024-09-16T20:51:33.052 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7139999999999998e-06 2024-09-16T20:51:33.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.052 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.052 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:33.052 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761387+0000", 2024-09-16T20:51:33.053 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2349e-05 2024-09-16T20:51:33.053 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.053 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.053 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:33.053 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761393+0000", 2024-09-16T20:51:33.053 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.652e-06 2024-09-16T20:51:33.053 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.053 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.053 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:33.053 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761442+0000", 2024-09-16T20:51:33.053 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9011000000000002e-05 2024-09-16T20:51:33.053 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.053 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.053 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:33.054 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761451+0000", 2024-09-16T20:51:33.054 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1920000000000005e-06 2024-09-16T20:51:33.054 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.054 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.054 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:33.054 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:28.764169+0000", 2024-09-16T20:51:33.054 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.002718161 2024-09-16T20:51:33.054 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.054 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.054 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:33.054 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:28.764273+0000", 2024-09-16T20:51:33.054 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010465 2024-09-16T20:51:33.054 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:33.054 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:33.055 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:33.055 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.055 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.055 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6061.0:5424 3.b 3:d353d652:::rbd_data.110e16eafbd.0000000000000010:head [read 606208~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-16T20:51:33.055 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:27.766882+0000", 2024-09-16T20:51:33.055 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 65.207858982000005, 2024-09-16T20:51:33.055 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99753650500000002, 2024-09-16T20:51:33.055 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:33.055 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:33.055 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:33.055 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:33.055 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6061", 2024-09-16T20:51:33.055 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3930128440", 2024-09-16T20:51:33.055 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5424 2024-09-16T20:51:33.056 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.056 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:33.056 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.056 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:33.056 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766882+0000", 2024-09-16T20:51:33.056 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:33.056 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.056 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.056 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:33.056 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766882+0000", 2024-09-16T20:51:33.056 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:33.056 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.056 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.056 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:33.057 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766883+0000", 2024-09-16T20:51:33.057 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3289999999999999e-06 2024-09-16T20:51:33.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.057 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.057 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:33.057 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766888+0000", 2024-09-16T20:51:33.057 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5800000000000002e-06 2024-09-16T20:51:33.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.057 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.057 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:33.057 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766900+0000", 2024-09-16T20:51:33.057 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1861e-05 2024-09-16T20:51:33.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.058 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.058 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:33.058 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766909+0000", 2024-09-16T20:51:33.058 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5210000000000008e-06 2024-09-16T20:51:33.058 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.058 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.058 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:33.058 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766958+0000", 2024-09-16T20:51:33.058 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8718999999999999e-05 2024-09-16T20:51:33.058 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.058 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.058 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:33.058 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766975+0000", 2024-09-16T20:51:33.058 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7231000000000002e-05 2024-09-16T20:51:33.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.059 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.059 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:33.059 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:28.764327+0000", 2024-09-16T20:51:33.059 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99735112500000001 2024-09-16T20:51:33.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.059 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.059 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:33.059 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:28.764419+0000", 2024-09-16T20:51:33.059 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2139000000000007e-05 2024-09-16T20:51:33.059 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:33.059 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:33.059 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:33.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.059 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.060 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:31909 3.1e 3:79ff2235:::rbd_header.110ee9a2c138:head [watch reconnect cookie 140213436757440 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e199)", 2024-09-16T20:51:33.060 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:51:26.618259+0000", 2024-09-16T20:51:33.060 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 6.3564819559999997, 2024-09-16T20:51:33.060 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0148341999999999, 2024-09-16T20:51:33.060 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:33.060 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:33.060 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:33.060 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:33.060 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:33.060 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:33.060 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 31909 2024-09-16T20:51:33.060 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.060 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:33.060 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.060 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:33.061 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.618259+0000", 2024-09-16T20:51:33.061 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:33.061 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.061 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.061 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:33.061 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.618259+0000", 2024-09-16T20:51:33.061 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:33.061 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.061 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.061 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:33.061 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.618261+0000", 2024-09-16T20:51:33.061 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4899999999999999e-06 2024-09-16T20:51:33.061 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.061 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.062 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:33.062 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.618267+0000", 2024-09-16T20:51:33.062 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8649999999999998e-06 2024-09-16T20:51:33.062 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.062 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.062 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:33.062 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.618277+0000", 2024-09-16T20:51:33.062 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0209e-05 2024-09-16T20:51:33.062 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.062 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.062 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:33.062 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.618282+0000", 2024-09-16T20:51:33.062 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.733e-06 2024-09-16T20:51:33.062 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.062 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.063 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:33.063 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.618341+0000", 2024-09-16T20:51:33.063 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8823000000000002e-05 2024-09-16T20:51:33.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.063 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.063 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:51:33.063 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.618346+0000", 2024-09-16T20:51:33.063 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8049999999999997e-06 2024-09-16T20:51:33.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.063 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.063 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:33.063 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.620745+0000", 2024-09-16T20:51:33.063 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0023987499999999998 2024-09-16T20:51:33.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.064 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:33.064 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.620755+0000", 2024-09-16T20:51:33.064 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0474e-05 2024-09-16T20:51:33.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.064 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:33.064 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:27.629480+0000", 2024-09-16T20:51:33.064 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.008724403 2024-09-16T20:51:33.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.064 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:33.064 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:27.629590+0000", 2024-09-16T20:51:33.064 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011044600000000001 2024-09-16T20:51:33.065 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.065 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.065 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:33.065 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:27.633093+0000", 2024-09-16T20:51:33.065 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003503202 2024-09-16T20:51:33.065 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:33.065 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:33.065 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:33.065 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.065 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.065 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:31914 3.1e 3:79ff2235:::rbd_header.110ee9a2c138:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e199)", 2024-09-16T20:51:33.065 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:51:26.635328+0000", 2024-09-16T20:51:33.065 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 6.3394129279999998, 2024-09-16T20:51:33.065 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99790577499999999, 2024-09-16T20:51:33.065 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:33.066 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:33.066 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:33.066 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:33.066 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:33.066 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:33.066 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 31914 2024-09-16T20:51:33.066 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.066 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:33.066 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.066 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:33.066 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.635328+0000", 2024-09-16T20:51:33.066 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:33.066 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.066 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.066 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:33.066 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.635328+0000", 2024-09-16T20:51:33.067 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:33.067 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.067 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.067 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:33.067 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.635330+0000", 2024-09-16T20:51:33.067 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3379999999999999e-06 2024-09-16T20:51:33.067 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.067 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.067 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:33.067 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.635334+0000", 2024-09-16T20:51:33.067 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6280000000000004e-06 2024-09-16T20:51:33.067 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.067 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.068 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:33.068 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.635344+0000", 2024-09-16T20:51:33.068 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7229999999999999e-06 2024-09-16T20:51:33.068 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.068 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.068 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:33.068 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.635349+0000", 2024-09-16T20:51:33.068 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1719999999999998e-06 2024-09-16T20:51:33.068 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.068 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.068 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:33.068 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.635424+0000", 2024-09-16T20:51:33.068 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4499999999999995e-05 2024-09-16T20:51:33.068 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.069 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.069 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:33.069 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.635435+0000", 2024-09-16T20:51:33.069 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1355000000000001e-05 2024-09-16T20:51:33.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.069 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.069 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:33.069 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:27.630304+0000", 2024-09-16T20:51:33.069 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99486901500000002 2024-09-16T20:51:33.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.069 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.069 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:33.069 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:27.630370+0000", 2024-09-16T20:51:33.070 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6390999999999995e-05 2024-09-16T20:51:33.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.070 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-16T20:51:33.070 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:27.630488+0000", 2024-09-16T20:51:33.070 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011726 2024-09-16T20:51:33.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.070 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:51:33.070 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:27.632101+0000", 2024-09-16T20:51:33.070 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016134249999999999 2024-09-16T20:51:33.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.070 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:33.070 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:27.633138+0000", 2024-09-16T20:51:33.071 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010367499999999999 2024-09-16T20:51:33.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.071 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.071 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:51:33.071 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:27.633196+0000", 2024-09-16T20:51:33.071 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8399000000000001e-05 2024-09-16T20:51:33.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.071 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.071 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:33.071 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:27.633234+0000", 2024-09-16T20:51:33.071 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7818999999999998e-05 2024-09-16T20:51:33.071 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:33.071 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:33.071 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:33.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.072 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.072 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:32211 3.1b 3:df9cfa64:::journal.110e645639d:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e204)", 2024-09-16T20:51:33.072 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:51:31.721834+0000", 2024-09-16T20:51:33.072 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 1.2529072960000001, 2024-09-16T20:51:33.072 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.917244744, 2024-09-16T20:51:33.072 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:33.072 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:33.072 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:33.072 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:33.072 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:33.072 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:33.072 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 32211 2024-09-16T20:51:33.072 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.073 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:33.073 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.073 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:33.073 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:31.721834+0000", 2024-09-16T20:51:33.073 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:33.073 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.073 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.073 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:33.073 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:31.721834+0000", 2024-09-16T20:51:33.073 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:33.073 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.073 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.073 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:33.074 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:31.721836+0000", 2024-09-16T20:51:33.074 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.719e-06 2024-09-16T20:51:33.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.074 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.074 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:33.074 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:31.721841+0000", 2024-09-16T20:51:33.074 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.913e-06 2024-09-16T20:51:33.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.074 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.074 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:33.074 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:31.721855+0000", 2024-09-16T20:51:33.074 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3149000000000001e-05 2024-09-16T20:51:33.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.074 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.075 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:33.075 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:31.721861+0000", 2024-09-16T20:51:33.075 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0310000000000004e-06 2024-09-16T20:51:33.075 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.075 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.075 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:33.075 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:31.721930+0000", 2024-09-16T20:51:33.075 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9739000000000004e-05 2024-09-16T20:51:33.075 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.075 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.075 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:33.075 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:31.721939+0000", 2024-09-16T20:51:33.075 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6170000000000007e-06 2024-09-16T20:51:33.076 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.076 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.076 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:33.076 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:32.638931+0000", 2024-09-16T20:51:33.076 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.91699213700000004 2024-09-16T20:51:33.076 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:33.076 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:33.076 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:33.076 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:32.639079+0000", 2024-09-16T20:51:33.076 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000147439 2024-09-16T20:51:33.076 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:33.076 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:33.076 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:33.076 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:33.076 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:33.076 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:33.077 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:32.975+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:33.078 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:51:33.078 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:51:33.079 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:51:33.079 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:51:33.080 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:51:33.077+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:51:33.080 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:51:33.080 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:51:33.081 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:51:33.081 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:51:33.081 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:51:33.081 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:51:33.082 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:51:33.082 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:51:33.083 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:51:33.179 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:33.178+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:33.226 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:33.227 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:33.235 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:51:33.280 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:33.278+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:33.342 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:33.342 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:33.350 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:51:33.391 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:33.390+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:33.482 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:33.481+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:33.497 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:33.498 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:33.506 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:51:33.583 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:51:33.581+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:51:33.613 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:33.614 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:33.622 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:51:33.644 INFO:tasks.thrashosds.thrasher:in_osds: [1, 4, 5, 0, 3, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-16T20:51:33.644 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T20:51:33.644 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-16T20:51:33.644 INFO:tasks.thrashosds.thrasher:Setting osd 2 primary_affinity to 1.000000 2024-09-16T20:51:33.644 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-16T20:51:33.649 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T20:51:33.707 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:33.706+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:33.776 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:33.777 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:33.785 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:33.783+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:33.786 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:51:33.885 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:33.884+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:33.934 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:33.934 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:33.942 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:51:33.986 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:33.985+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:34.052 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:34.052 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:34.060 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:51:34.101 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:34.100+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:34.189 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:34.188+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:34.207 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:34.207 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:34.216 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:51:34.290 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:34.289+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:34.323 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:34.323 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:34.332 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:51:34.390 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:34.389+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:34.439 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:34.440 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:34.448 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:51:34.491 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:34.490+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:34.575 INFO:teuthology.orchestra.run.smithi029.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-16T20:51:34.576 INFO:teuthology.orchestra.run.smithi029.stderr:nodeep-scrub is set 2024-09-16T20:51:34.592 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:34.591+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:34.605 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:34.606 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:34.615 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:51:34.693 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:34.691+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:34.767 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:34.767 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:34.775 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:51:34.794 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:34.793+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:34.882 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:34.883 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:34.891 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:51:34.894 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:34.893+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:34.995 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:34.994+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:34.998 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:34.998 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:35.006 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:51:35.084 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T20:51:35.096 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:35.095+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:35.156 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:35.156 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:35.165 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:51:35.196 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:35.195+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:35.213 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:51:35.213 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:51:35.230 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:51:35.231 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:51:35.247 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:51:35.247 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:51:35.264 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:51:35.264 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:51:35.280 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:51:35.281 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:51:35.285 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:35.285 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:51:35.285 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:51:35.286 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:35.295 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:51:35.298 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:51:35.298 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:51:35.298 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:35.296+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:35.398 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:35.397+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:35.402 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:35.403 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:51:35.403 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:51:35.403 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:51:35.403 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:35.411 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:51:35.499 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:35.498+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:35.520 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:35.520 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:51:35.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:51:35.520 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:51:35.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.521 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:3088 3.1e 3:79e5ec01:::rbd_header.110e891a9b2b:head [watch watch cookie 140213437171488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:51:35.521 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:51:35.521 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 191.35462148600001, 2024-09-16T20:51:35.521 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.11895186200000001, 2024-09-16T20:51:35.521 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:35.521 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:35.521 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:35.521 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:35.521 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:35.521 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:35.521 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3088 2024-09-16T20:51:35.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.521 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:35.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.522 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:35.522 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:51:35.522 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:35.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.522 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:35.522 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:51:35.522 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:35.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.522 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:35.522 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163846+0000", 2024-09-16T20:51:35.522 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6799999999999999e-07 2024-09-16T20:51:35.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:35.523 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163850+0000", 2024-09-16T20:51:35.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.929e-06 2024-09-16T20:51:35.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:35.523 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163862+0000", 2024-09-16T20:51:35.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1972e-05 2024-09-16T20:51:35.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:35.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163867+0000", 2024-09-16T20:51:35.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.451e-06 2024-09-16T20:51:35.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.524 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:35.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163912+0000", 2024-09-16T20:51:35.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4456000000000003e-05 2024-09-16T20:51:35.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.524 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:35.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163962+0000", 2024-09-16T20:51:35.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0578000000000003e-05 2024-09-16T20:51:35.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.525 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.525 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-16T20:51:35.525 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164058+0000", 2024-09-16T20:51:35.525 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5934999999999995e-05 2024-09-16T20:51:35.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.525 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.525 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:51:35.525 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164570+0000", 2024-09-16T20:51:35.525 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00051241100000000005 2024-09-16T20:51:35.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.525 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.525 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:35.525 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282629+0000", 2024-09-16T20:51:35.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.118058986 2024-09-16T20:51:35.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.526 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.526 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:51:35.526 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282725+0000", 2024-09-16T20:51:35.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5063999999999996e-05 2024-09-16T20:51:35.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.526 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.526 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:35.526 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282797+0000", 2024-09-16T20:51:35.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2212000000000006e-05 2024-09-16T20:51:35.526 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:35.526 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:35.527 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:35.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.527 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:8039 3.1e 3.ac44ff9e (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:51:35.527 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:51:35.527 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 165.81079413699999, 2024-09-16T20:51:35.527 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0053310449999999, 2024-09-16T20:51:35.527 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:35.527 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:35.527 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:35.527 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:35.527 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:35.527 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:35.528 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8039 2024-09-16T20:51:35.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.528 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:35.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:35.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:51:35.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:35.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:35.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:51:35.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:35.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.529 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:35.529 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707674+0000", 2024-09-16T20:51:35.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6530000000000001e-06 2024-09-16T20:51:35.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.529 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:35.529 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707681+0000", 2024-09-16T20:51:35.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9639999999999997e-06 2024-09-16T20:51:35.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.529 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:35.529 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707700+0000", 2024-09-16T20:51:35.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8674000000000001e-05 2024-09-16T20:51:35.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.530 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:35.530 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707708+0000", 2024-09-16T20:51:35.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6810000000000004e-06 2024-09-16T20:51:35.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.530 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:35.530 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711293+0000", 2024-09-16T20:51:35.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035848189999999999 2024-09-16T20:51:35.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.530 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:35.530 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711301+0000", 2024-09-16T20:51:35.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4730000000000003e-06 2024-09-16T20:51:35.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:35.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.712990+0000", 2024-09-16T20:51:35.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.001689348 2024-09-16T20:51:35.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:35.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713003+0000", 2024-09-16T20:51:35.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3433e-05 2024-09-16T20:51:35.531 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:35.531 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:35.531 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:35.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.532 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:8040 3.1e 3.ac44ff9e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:51:35.532 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:51:35.532 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 165.810733742, 2024-09-16T20:51:35.532 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0053410780000001, 2024-09-16T20:51:35.532 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:35.532 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:35.532 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:35.532 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:35.532 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:35.532 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:35.532 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8040 2024-09-16T20:51:35.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.532 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:35.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.533 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:35.533 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:51:35.533 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:35.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.533 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.533 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:35.533 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:51:35.533 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:35.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.533 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.533 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:35.533 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:51:35.533 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8100000000000003e-07 2024-09-16T20:51:35.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.534 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:35.534 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707737+0000", 2024-09-16T20:51:35.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5020000000000001e-06 2024-09-16T20:51:35.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.534 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:35.534 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707745+0000", 2024-09-16T20:51:35.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2169999999999994e-06 2024-09-16T20:51:35.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.534 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:35.534 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707751+0000", 2024-09-16T20:51:35.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.263e-06 2024-09-16T20:51:35.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:35.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711336+0000", 2024-09-16T20:51:35.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035848360000000001 2024-09-16T20:51:35.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:35.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711340+0000", 2024-09-16T20:51:35.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1309999999999999e-06 2024-09-16T20:51:35.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:35.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713060+0000", 2024-09-16T20:51:35.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0017195059999999 2024-09-16T20:51:35.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.536 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:35.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713074+0000", 2024-09-16T20:51:35.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4042e-05 2024-09-16T20:51:35.536 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:35.536 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:35.536 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:35.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.536 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4927.0:4487 3.1e 3.ac44ff9e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:51:35.536 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:51:35.536 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 165.81021354000001, 2024-09-16T20:51:35.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0048840880000001, 2024-09-16T20:51:35.537 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:35.537 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:35.537 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:35.537 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:35.537 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4927", 2024-09-16T20:51:35.537 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:35.537 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4487 2024-09-16T20:51:35.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.537 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:35.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:35.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:51:35.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:35.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.538 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:35.538 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:51:35.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:35.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.538 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:35.538 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708254+0000", 2024-09-16T20:51:35.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1800000000000005e-07 2024-09-16T20:51:35.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.538 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:35.539 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708258+0000", 2024-09-16T20:51:35.539 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8689999999999997e-06 2024-09-16T20:51:35.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.539 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:35.539 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708267+0000", 2024-09-16T20:51:35.539 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9590000000000007e-06 2024-09-16T20:51:35.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.539 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:35.539 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708274+0000", 2024-09-16T20:51:35.539 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9970000000000001e-06 2024-09-16T20:51:35.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:35.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711366+0000", 2024-09-16T20:51:35.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030919530000000001 2024-09-16T20:51:35.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:35.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711372+0000", 2024-09-16T20:51:35.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9410000000000004e-06 2024-09-16T20:51:35.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:35.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713126+0000", 2024-09-16T20:51:35.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0017537249999999 2024-09-16T20:51:35.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.541 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:35.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713137+0000", 2024-09-16T20:51:35.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0825999999999999e-05 2024-09-16T20:51:35.541 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:35.541 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:35.541 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:35.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.541 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5089.0:11703 3.1b 3:dcd4684e:::rbd_data.110e747f0779.000000000000001b:head [set-alloc-hint object_size 262144 write_size 262144,write 139264~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:51:35.541 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:51:35.542 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 151.38047776600001, 2024-09-16T20:51:35.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.084231385000000006, 2024-09-16T20:51:35.542 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:35.542 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:35.542 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:35.542 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:35.542 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5089", 2024-09-16T20:51:35.542 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:51:35.542 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 11703 2024-09-16T20:51:35.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.542 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:35.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:35.543 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:51:35.543 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:35.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.543 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.543 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:35.543 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:51:35.543 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:35.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.543 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.543 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:35.543 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:51:35.543 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2600000000000001e-07 2024-09-16T20:51:35.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.543 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.544 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:35.544 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137998+0000", 2024-09-16T20:51:35.544 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9549999999999998e-06 2024-09-16T20:51:35.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.544 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:35.544 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138008+0000", 2024-09-16T20:51:35.544 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3789999999999995e-06 2024-09-16T20:51:35.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.544 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:35.544 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138013+0000", 2024-09-16T20:51:35.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2209999999999996e-06 2024-09-16T20:51:35.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.545 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:35.545 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138054+0000", 2024-09-16T20:51:35.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1022999999999997e-05 2024-09-16T20:51:35.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.545 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:35.545 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138124+0000", 2024-09-16T20:51:35.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0234999999999994e-05 2024-09-16T20:51:35.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:35.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138197+0000", 2024-09-16T20:51:35.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2451000000000001e-05 2024-09-16T20:51:35.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:35.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222010+0000", 2024-09-16T20:51:35.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.083813433000000007 2024-09-16T20:51:35.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:51:35.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222163+0000", 2024-09-16T20:51:35.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015274599999999999 2024-09-16T20:51:35.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.547 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:51:35.547 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222197+0000", 2024-09-16T20:51:35.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3683000000000003e-05 2024-09-16T20:51:35.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.547 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:35.547 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222220+0000", 2024-09-16T20:51:35.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3433000000000001e-05 2024-09-16T20:51:35.547 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:35.547 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:35.547 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:35.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.548 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.548 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.5089.0:11702 3.15 e73/64 3:acf33c8b:::rbd_data.110e747f0779.000000000000001a:head v 73'132, mlcod=73'130)", 2024-09-16T20:51:35.548 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:51:35.548 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 151.38021043699999, 2024-09-16T20:51:35.548 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.084079763000000002, 2024-09-16T20:51:35.548 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:35.548 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:35.548 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:35.548 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:35.548 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.548 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:35.548 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:51:35.548 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:35.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.549 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:35.549 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:51:35.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:35.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.549 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:35.549 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:51:35.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:51:35.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.549 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:35.549 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138263+0000", 2024-09-16T20:51:35.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1e-06 2024-09-16T20:51:35.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.550 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:35.550 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221589+0000", 2024-09-16T20:51:35.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.083326002999999996 2024-09-16T20:51:35.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.550 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:35.550 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221614+0000", 2024-09-16T20:51:35.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5593999999999999e-05 2024-09-16T20:51:35.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.550 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:35.551 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221714+0000", 2024-09-16T20:51:35.551 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9889000000000005e-05 2024-09-16T20:51:35.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.551 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:35.551 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221727+0000", 2024-09-16T20:51:35.551 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3125999999999999e-05 2024-09-16T20:51:35.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.551 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:51:35.551 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222294+0000", 2024-09-16T20:51:35.551 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00056712199999999998 2024-09-16T20:51:35.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.552 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.552 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:35.552 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222336+0000", 2024-09-16T20:51:35.552 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1470000000000001e-05 2024-09-16T20:51:35.552 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:35.552 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:35.552 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:35.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.552 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.552 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5138.0:9358 3.c 3:33996a43:::rbd_data.110e6372565d.0000000000000007:head [read 1343488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T20:51:35.552 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:51:35.552 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 127.665935027, 2024-09-16T20:51:35.552 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0167493569999999, 2024-09-16T20:51:35.553 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:35.553 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:35.553 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:35.553 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:35.553 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5138", 2024-09-16T20:51:35.553 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:51:35.553 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9358 2024-09-16T20:51:35.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.553 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:35.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.553 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:35.553 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:51:35.554 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:35.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.554 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.554 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:35.554 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:51:35.554 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:35.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.554 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.554 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:35.554 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852533+0000", 2024-09-16T20:51:35.554 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.646e-06 2024-09-16T20:51:35.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:35.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852539+0000", 2024-09-16T20:51:35.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3389999999999998e-06 2024-09-16T20:51:35.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:35.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852552+0000", 2024-09-16T20:51:35.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.271e-05 2024-09-16T20:51:35.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:35.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852563+0000", 2024-09-16T20:51:35.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1139e-05 2024-09-16T20:51:35.556 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.556 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.556 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:35.556 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852883+0000", 2024-09-16T20:51:35.556 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00032010600000000001 2024-09-16T20:51:35.556 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.556 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.556 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:35.556 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852890+0000", 2024-09-16T20:51:35.556 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7669999999999998e-06 2024-09-16T20:51:35.556 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.556 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.557 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:35.557 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869059+0000", 2024-09-16T20:51:35.557 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.016169353 2024-09-16T20:51:35.557 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.557 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.557 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:35.557 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869178+0000", 2024-09-16T20:51:35.557 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118193 2024-09-16T20:51:35.557 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.557 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.557 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:35.557 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869281+0000", 2024-09-16T20:51:35.557 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010310399999999999 2024-09-16T20:51:35.558 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:35.558 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:35.558 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:35.558 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.558 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.558 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5138.0:9382 3.c 3:33996a43:::rbd_data.110e6372565d.0000000000000007:head [read 0~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T20:51:35.558 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:51:35.558 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 127.66588415, 2024-09-16T20:51:35.558 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.017136034, 2024-09-16T20:51:35.558 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:35.558 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:35.558 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:35.559 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:35.559 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5138", 2024-09-16T20:51:35.559 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:51:35.559 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9382 2024-09-16T20:51:35.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.559 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:35.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:35.559 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:51:35.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:35.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.560 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:35.560 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:51:35.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:35.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.560 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:35.560 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852583+0000", 2024-09-16T20:51:35.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.51e-07 2024-09-16T20:51:35.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.560 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:35.560 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852586+0000", 2024-09-16T20:51:35.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0280000000000001e-06 2024-09-16T20:51:35.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.561 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.561 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:35.561 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852595+0000", 2024-09-16T20:51:35.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2799999999999992e-06 2024-09-16T20:51:35.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.561 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.561 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:35.561 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852601+0000", 2024-09-16T20:51:35.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3810000000000004e-06 2024-09-16T20:51:35.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.561 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.562 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:35.562 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852921+0000", 2024-09-16T20:51:35.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00031979699999999997 2024-09-16T20:51:35.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.562 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:35.562 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852930+0000", 2024-09-16T20:51:35.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6489999999999997e-06 2024-09-16T20:51:35.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.562 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:35.562 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869434+0000", 2024-09-16T20:51:35.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0165038829999999 2024-09-16T20:51:35.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.563 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:35.563 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869495+0000", 2024-09-16T20:51:35.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1509999999999999e-05 2024-09-16T20:51:35.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.563 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:35.563 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869718+0000", 2024-09-16T20:51:35.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000222955 2024-09-16T20:51:35.563 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:35.563 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:35.563 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:35.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.564 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9971 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 643072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:35.564 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642102+0000", 2024-09-16T20:51:35.564 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 104.876364006, 2024-09-16T20:51:35.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077421016999999995, 2024-09-16T20:51:35.564 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:35.564 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:35.564 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:35.564 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:35.564 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:35.564 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:35.564 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9971 2024-09-16T20:51:35.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.565 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:35.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.565 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:35.565 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642102+0000", 2024-09-16T20:51:35.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:35.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.565 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:35.565 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642102+0000", 2024-09-16T20:51:35.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:35.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.566 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:35.566 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642103+0000", 2024-09-16T20:51:35.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7399999999999998e-07 2024-09-16T20:51:35.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.566 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.566 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:35.566 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642105+0000", 2024-09-16T20:51:35.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2649999999999999e-06 2024-09-16T20:51:35.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.566 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.566 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:35.566 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642112+0000", 2024-09-16T20:51:35.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.294e-06 2024-09-16T20:51:35.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.567 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:35.567 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642118+0000", 2024-09-16T20:51:35.567 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4500000000000003e-06 2024-09-16T20:51:35.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.567 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:35.567 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719285+0000", 2024-09-16T20:51:35.567 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077166883000000006 2024-09-16T20:51:35.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.567 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:35.568 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719368+0000", 2024-09-16T20:51:35.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2811000000000004e-05 2024-09-16T20:51:35.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.568 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:35.568 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719523+0000", 2024-09-16T20:51:35.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015584 2024-09-16T20:51:35.568 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:35.568 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:35.568 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:35.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.569 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9972 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 648192~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:35.569 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642125+0000", 2024-09-16T20:51:35.569 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 104.876341059, 2024-09-16T20:51:35.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077710324999999997, 2024-09-16T20:51:35.569 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:35.569 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:35.569 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:35.569 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:35.569 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:35.569 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:35.569 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9972 2024-09-16T20:51:35.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.569 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:35.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.570 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:35.570 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642125+0000", 2024-09-16T20:51:35.570 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:35.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.570 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:35.570 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642125+0000", 2024-09-16T20:51:35.570 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:35.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.570 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:35.570 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642126+0000", 2024-09-16T20:51:35.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1899999999999998e-07 2024-09-16T20:51:35.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.571 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.571 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:35.571 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642128+0000", 2024-09-16T20:51:35.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8619999999999999e-06 2024-09-16T20:51:35.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.571 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.571 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:35.571 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642134+0000", 2024-09-16T20:51:35.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5289999999999997e-06 2024-09-16T20:51:35.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.572 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:35.572 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642139+0000", 2024-09-16T20:51:35.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1250000000000001e-06 2024-09-16T20:51:35.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.572 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:35.572 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719570+0000", 2024-09-16T20:51:35.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077430516000000005 2024-09-16T20:51:35.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.572 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:35.573 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719655+0000", 2024-09-16T20:51:35.573 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5500000000000005e-05 2024-09-16T20:51:35.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.573 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.573 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:35.573 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719836+0000", 2024-09-16T20:51:35.573 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018037399999999999 2024-09-16T20:51:35.573 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:35.573 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:35.573 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:35.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.574 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9973 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 653312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:35.574 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:51:35.574 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 104.876320449, 2024-09-16T20:51:35.574 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077905262000000003, 2024-09-16T20:51:35.574 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:35.574 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:35.574 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:35.574 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:35.574 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:35.574 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:35.574 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9973 2024-09-16T20:51:35.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.574 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:35.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.575 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:35.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:51:35.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:35.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.575 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:35.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:51:35.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:35.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.575 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:35.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:51:35.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:51:35.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:35.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642148+0000", 2024-09-16T20:51:35.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0219999999999999e-06 2024-09-16T20:51:35.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:35.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642154+0000", 2024-09-16T20:51:35.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8540000000000002e-06 2024-09-16T20:51:35.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:35.577 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642159+0000", 2024-09-16T20:51:35.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.045e-06 2024-09-16T20:51:35.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.577 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:35.577 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719875+0000", 2024-09-16T20:51:35.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077716116000000002 2024-09-16T20:51:35.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.577 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:35.577 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719934+0000", 2024-09-16T20:51:35.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8819e-05 2024-09-16T20:51:35.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.578 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:35.578 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720051+0000", 2024-09-16T20:51:35.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000116947 2024-09-16T20:51:35.578 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:35.578 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:35.578 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:35.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.578 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9974 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 658432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:35.578 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642166+0000", 2024-09-16T20:51:35.578 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 104.876300535, 2024-09-16T20:51:35.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.078179786000000001, 2024-09-16T20:51:35.578 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:35.579 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:35.579 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:35.579 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:35.579 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:35.579 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:35.579 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9974 2024-09-16T20:51:35.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.579 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:35.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:35.579 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642166+0000", 2024-09-16T20:51:35.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:35.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:35.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642166+0000", 2024-09-16T20:51:35.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:35.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:35.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647343+0000", 2024-09-16T20:51:35.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0051769440000000002 2024-09-16T20:51:35.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:35.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647348+0000", 2024-09-16T20:51:35.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3619999999999997e-06 2024-09-16T20:51:35.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:35.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647359+0000", 2024-09-16T20:51:35.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0929e-05 2024-09-16T20:51:35.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:35.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647366+0000", 2024-09-16T20:51:35.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0910000000000004e-06 2024-09-16T20:51:35.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:35.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720094+0000", 2024-09-16T20:51:35.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.072727403999999995 2024-09-16T20:51:35.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:35.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720173+0000", 2024-09-16T20:51:35.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9382000000000007e-05 2024-09-16T20:51:35.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:35.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720346+0000", 2024-09-16T20:51:35.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017267399999999999 2024-09-16T20:51:35.583 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:35.583 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:35.583 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:35.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.583 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9991 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 745472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:35.583 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:51:35.583 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 104.870731142, 2024-09-16T20:51:35.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077257820000000005, 2024-09-16T20:51:35.583 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:35.584 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:35.584 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:35.584 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:35.584 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:35.584 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:35.584 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9991 2024-09-16T20:51:35.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.584 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:35.584 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.584 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:35.584 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:51:35.584 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:35.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:35.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:51:35.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:35.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:35.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:51:35.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1199999999999999e-07 2024-09-16T20:51:35.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:35.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647736+0000", 2024-09-16T20:51:35.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3699999999999999e-07 2024-09-16T20:51:35.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:35.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647739+0000", 2024-09-16T20:51:35.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9359999999999999e-06 2024-09-16T20:51:35.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:35.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647742+0000", 2024-09-16T20:51:35.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5720000000000001e-06 2024-09-16T20:51:35.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:35.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724774+0000", 2024-09-16T20:51:35.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077032457999999998 2024-09-16T20:51:35.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:35.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724818+0000", 2024-09-16T20:51:35.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3803000000000001e-05 2024-09-16T20:51:35.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:35.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724993+0000", 2024-09-16T20:51:35.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000174902 2024-09-16T20:51:35.587 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:35.588 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:35.588 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:35.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.588 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9992 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 750592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:35.588 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:51:35.588 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 104.870721553, 2024-09-16T20:51:35.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077553227000000002, 2024-09-16T20:51:35.588 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:35.588 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:35.588 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:35.588 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:35.588 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:35.589 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:35.589 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9992 2024-09-16T20:51:35.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.589 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:35.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:35.589 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:51:35.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:35.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:35.589 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:51:35.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:35.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.590 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:35.590 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:51:35.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0200000000000001e-07 2024-09-16T20:51:35.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.590 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:35.590 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647746+0000", 2024-09-16T20:51:35.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.088e-06 2024-09-16T20:51:35.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.590 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:35.590 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647749+0000", 2024-09-16T20:51:35.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9170000000000002e-06 2024-09-16T20:51:35.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.591 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:35.591 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647752+0000", 2024-09-16T20:51:35.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.774e-06 2024-09-16T20:51:35.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.591 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:35.591 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725042+0000", 2024-09-16T20:51:35.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077289952999999995 2024-09-16T20:51:35.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.591 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:35.592 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725113+0000", 2024-09-16T20:51:35.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0920999999999994e-05 2024-09-16T20:51:35.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.592 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:35.592 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725298+0000", 2024-09-16T20:51:35.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000185372 2024-09-16T20:51:35.592 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:35.592 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:35.592 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:35.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.592 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9993 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 755712~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:35.592 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647755+0000", 2024-09-16T20:51:35.593 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 104.870711527, 2024-09-16T20:51:35.593 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077725674999999994, 2024-09-16T20:51:35.593 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:35.593 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:35.593 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:35.593 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:35.593 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:35.593 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:35.594 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9993 2024-09-16T20:51:35.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.594 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:35.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.594 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:35.594 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647755+0000", 2024-09-16T20:51:35.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:35.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.594 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:35.594 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647755+0000", 2024-09-16T20:51:35.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:35.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.595 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:35.595 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653406+0000", 2024-09-16T20:51:35.595 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.005650876 2024-09-16T20:51:35.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.595 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:35.595 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653411+0000", 2024-09-16T20:51:35.595 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8899999999999998e-06 2024-09-16T20:51:35.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.595 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:35.595 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653423+0000", 2024-09-16T20:51:35.596 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2447e-05 2024-09-16T20:51:35.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.596 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.596 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:35.596 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653431+0000", 2024-09-16T20:51:35.596 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5730000000000002e-06 2024-09-16T20:51:35.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.596 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.596 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:35.596 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725333+0000", 2024-09-16T20:51:35.596 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.071902743000000005 2024-09-16T20:51:35.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.596 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.597 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:35.597 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725385+0000", 2024-09-16T20:51:35.597 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1585000000000002e-05 2024-09-16T20:51:35.597 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.597 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.597 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:35.597 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725481+0000", 2024-09-16T20:51:35.597 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5561000000000002e-05 2024-09-16T20:51:35.597 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:35.597 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:35.597 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:35.597 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.597 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.598 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6061.0:5400 3.b 3:d353d652:::rbd_data.110e16eafbd.0000000000000010:head [read 577536~28672] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-16T20:51:35.598 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:27.761367+0000", 2024-09-16T20:51:35.598 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 67.757099581000006, 2024-09-16T20:51:35.598 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0029066630000001, 2024-09-16T20:51:35.598 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:35.598 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:35.598 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:35.598 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:35.598 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6061", 2024-09-16T20:51:35.598 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3930128440", 2024-09-16T20:51:35.598 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5400 2024-09-16T20:51:35.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.598 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:35.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.599 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:35.599 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761367+0000", 2024-09-16T20:51:35.599 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:35.599 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.599 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:35.599 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761367+0000", 2024-09-16T20:51:35.599 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:35.599 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.599 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:35.599 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761369+0000", 2024-09-16T20:51:35.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.934e-06 2024-09-16T20:51:35.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.600 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.600 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:35.600 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761374+0000", 2024-09-16T20:51:35.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7139999999999998e-06 2024-09-16T20:51:35.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.600 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.600 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:35.600 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761387+0000", 2024-09-16T20:51:35.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2349e-05 2024-09-16T20:51:35.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.600 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.601 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:35.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761393+0000", 2024-09-16T20:51:35.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.652e-06 2024-09-16T20:51:35.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.601 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:35.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761442+0000", 2024-09-16T20:51:35.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9011000000000002e-05 2024-09-16T20:51:35.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.601 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:35.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761451+0000", 2024-09-16T20:51:35.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1920000000000005e-06 2024-09-16T20:51:35.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.602 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:35.602 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:28.764169+0000", 2024-09-16T20:51:35.602 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.002718161 2024-09-16T20:51:35.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.602 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:35.602 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:28.764273+0000", 2024-09-16T20:51:35.602 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010465 2024-09-16T20:51:35.602 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:35.602 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:35.602 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:35.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.603 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6061.0:5424 3.b 3:d353d652:::rbd_data.110e16eafbd.0000000000000010:head [read 606208~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-16T20:51:35.603 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:27.766882+0000", 2024-09-16T20:51:35.603 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 67.751584218000005, 2024-09-16T20:51:35.603 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99753650500000002, 2024-09-16T20:51:35.603 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:35.603 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:35.603 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:35.603 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:35.603 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6061", 2024-09-16T20:51:35.603 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3930128440", 2024-09-16T20:51:35.603 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5424 2024-09-16T20:51:35.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.604 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:35.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.604 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:35.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766882+0000", 2024-09-16T20:51:35.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:35.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.604 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:35.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766882+0000", 2024-09-16T20:51:35.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:35.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.604 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:35.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766883+0000", 2024-09-16T20:51:35.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3289999999999999e-06 2024-09-16T20:51:35.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:35.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766888+0000", 2024-09-16T20:51:35.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5800000000000002e-06 2024-09-16T20:51:35.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:35.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766900+0000", 2024-09-16T20:51:35.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1861e-05 2024-09-16T20:51:35.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.606 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:35.606 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766909+0000", 2024-09-16T20:51:35.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5210000000000008e-06 2024-09-16T20:51:35.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.606 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:35.606 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766958+0000", 2024-09-16T20:51:35.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8718999999999999e-05 2024-09-16T20:51:35.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.606 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:35.606 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766975+0000", 2024-09-16T20:51:35.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7231000000000002e-05 2024-09-16T20:51:35.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.607 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:35.607 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:28.764327+0000", 2024-09-16T20:51:35.607 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99735112500000001 2024-09-16T20:51:35.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.607 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:35.607 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:28.764419+0000", 2024-09-16T20:51:35.607 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2139000000000007e-05 2024-09-16T20:51:35.607 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:35.607 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:35.607 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:35.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.608 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:31909 3.1e 3:79ff2235:::rbd_header.110ee9a2c138:head [watch reconnect cookie 140213436757440 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e199)", 2024-09-16T20:51:35.608 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:51:26.618259+0000", 2024-09-16T20:51:35.608 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.9002071919999999, 2024-09-16T20:51:35.608 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0148341999999999, 2024-09-16T20:51:35.608 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:35.608 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:35.608 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:35.608 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:35.608 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:35.608 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:35.608 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 31909 2024-09-16T20:51:35.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.608 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:35.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:35.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.618259+0000", 2024-09-16T20:51:35.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:35.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:35.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.618259+0000", 2024-09-16T20:51:35.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:35.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:35.610 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.618261+0000", 2024-09-16T20:51:35.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4899999999999999e-06 2024-09-16T20:51:35.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.610 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:35.610 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.618267+0000", 2024-09-16T20:51:35.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8649999999999998e-06 2024-09-16T20:51:35.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.610 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:35.610 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.618277+0000", 2024-09-16T20:51:35.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0209e-05 2024-09-16T20:51:35.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.611 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:35.611 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.618282+0000", 2024-09-16T20:51:35.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.733e-06 2024-09-16T20:51:35.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.611 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:35.611 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.618341+0000", 2024-09-16T20:51:35.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8823000000000002e-05 2024-09-16T20:51:35.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.611 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:51:35.611 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.618346+0000", 2024-09-16T20:51:35.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8049999999999997e-06 2024-09-16T20:51:35.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.612 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:35.612 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.620745+0000", 2024-09-16T20:51:35.612 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0023987499999999998 2024-09-16T20:51:35.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.612 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:35.612 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.620755+0000", 2024-09-16T20:51:35.612 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0474e-05 2024-09-16T20:51:35.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.612 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:35.612 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:27.629480+0000", 2024-09-16T20:51:35.612 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.008724403 2024-09-16T20:51:35.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.613 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:35.613 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:27.629590+0000", 2024-09-16T20:51:35.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011044600000000001 2024-09-16T20:51:35.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.613 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:35.613 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:27.633093+0000", 2024-09-16T20:51:35.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003503202 2024-09-16T20:51:35.613 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:35.613 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:35.613 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:35.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.614 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:31914 3.1e 3:79ff2235:::rbd_header.110ee9a2c138:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e199)", 2024-09-16T20:51:35.614 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:51:26.635328+0000", 2024-09-16T20:51:35.614 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.883138164, 2024-09-16T20:51:35.614 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99790577499999999, 2024-09-16T20:51:35.614 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:35.614 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:35.614 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:35.614 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:35.614 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:35.614 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:35.614 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 31914 2024-09-16T20:51:35.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.615 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:35.615 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.615 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:35.615 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.635328+0000", 2024-09-16T20:51:35.615 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:35.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.615 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.615 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:35.615 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.635328+0000", 2024-09-16T20:51:35.615 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:35.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.615 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.616 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:35.616 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.635330+0000", 2024-09-16T20:51:35.616 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3379999999999999e-06 2024-09-16T20:51:35.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.616 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.616 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:35.616 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.635334+0000", 2024-09-16T20:51:35.616 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6280000000000004e-06 2024-09-16T20:51:35.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.616 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.616 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:35.616 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.635344+0000", 2024-09-16T20:51:35.616 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7229999999999999e-06 2024-09-16T20:51:35.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.617 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:35.617 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.635349+0000", 2024-09-16T20:51:35.617 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1719999999999998e-06 2024-09-16T20:51:35.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.617 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:35.617 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.635424+0000", 2024-09-16T20:51:35.617 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4499999999999995e-05 2024-09-16T20:51:35.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:35.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.635435+0000", 2024-09-16T20:51:35.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1355000000000001e-05 2024-09-16T20:51:35.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:35.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:27.630304+0000", 2024-09-16T20:51:35.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99486901500000002 2024-09-16T20:51:35.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:35.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:27.630370+0000", 2024-09-16T20:51:35.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6390999999999995e-05 2024-09-16T20:51:35.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.619 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-16T20:51:35.619 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:27.630488+0000", 2024-09-16T20:51:35.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011726 2024-09-16T20:51:35.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.619 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:51:35.619 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:27.632101+0000", 2024-09-16T20:51:35.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016134249999999999 2024-09-16T20:51:35.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.619 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:35.619 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:27.633138+0000", 2024-09-16T20:51:35.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010367499999999999 2024-09-16T20:51:35.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.620 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:51:35.620 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:27.633196+0000", 2024-09-16T20:51:35.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8399000000000001e-05 2024-09-16T20:51:35.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.620 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:35.620 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:27.633234+0000", 2024-09-16T20:51:35.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7818999999999998e-05 2024-09-16T20:51:35.620 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:35.620 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:35.620 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:35.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.621 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:32211 3.1b 3:df9cfa64:::journal.110e645639d:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e204)", 2024-09-16T20:51:35.621 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:51:31.721834+0000", 2024-09-16T20:51:35.621 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.7966325319999998, 2024-09-16T20:51:35.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.917244744, 2024-09-16T20:51:35.621 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:35.621 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:35.621 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:35.621 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:35.621 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:35.621 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:35.621 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 32211 2024-09-16T20:51:35.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.621 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:35.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.622 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:35.622 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:31.721834+0000", 2024-09-16T20:51:35.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:35.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.622 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:35.622 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:31.721834+0000", 2024-09-16T20:51:35.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:35.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.622 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:35.622 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:31.721836+0000", 2024-09-16T20:51:35.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.719e-06 2024-09-16T20:51:35.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:35.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:31.721841+0000", 2024-09-16T20:51:35.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.913e-06 2024-09-16T20:51:35.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:35.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:31.721855+0000", 2024-09-16T20:51:35.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3149000000000001e-05 2024-09-16T20:51:35.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:35.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:31.721861+0000", 2024-09-16T20:51:35.624 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0310000000000004e-06 2024-09-16T20:51:35.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.624 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.624 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:35.624 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:31.721930+0000", 2024-09-16T20:51:35.624 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9739000000000004e-05 2024-09-16T20:51:35.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.624 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.624 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:35.624 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:31.721939+0000", 2024-09-16T20:51:35.624 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6170000000000007e-06 2024-09-16T20:51:35.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.624 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.624 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:35.625 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:32.638931+0000", 2024-09-16T20:51:35.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.91699213700000004 2024-09-16T20:51:35.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.625 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:35.625 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:32.639079+0000", 2024-09-16T20:51:35.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000147439 2024-09-16T20:51:35.625 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:35.625 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:35.625 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:35.625 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:35.625 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:35.625 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:35.627 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:35.625+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:35.627 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:51:35.728 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:35.727+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:35.741 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:35.741 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:51:35.742 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:51:35.742 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:35.751 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:51:35.828 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:35.827+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:35.859 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:35.859 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:51:35.859 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:51:35.859 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:51:35.859 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:35.867 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:51:35.929 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:35.927+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:35.976 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:35.976 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:51:35.976 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:51:35.976 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:51:35.976 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.976 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12548 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 628736~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:35.976 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757009+0000", 2024-09-16T20:51:35.977 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 105.217413611, 2024-09-16T20:51:35.977 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065743956000000006, 2024-09-16T20:51:35.977 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:35.977 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:35.977 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:35.977 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:35.977 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:35.977 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:35.977 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12548 2024-09-16T20:51:35.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.977 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:35.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.977 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:35.978 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757009+0000", 2024-09-16T20:51:35.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:35.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.978 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.978 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:35.978 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757009+0000", 2024-09-16T20:51:35.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:35.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.978 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.978 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:35.978 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757010+0000", 2024-09-16T20:51:35.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9900000000000001e-07 2024-09-16T20:51:35.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.979 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.979 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:35.979 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757012+0000", 2024-09-16T20:51:35.979 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2689999999999998e-06 2024-09-16T20:51:35.979 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.979 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.979 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:35.979 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757022+0000", 2024-09-16T20:51:35.979 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9699999999999994e-06 2024-09-16T20:51:35.979 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.979 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.979 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:35.979 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757028+0000", 2024-09-16T20:51:35.979 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3799999999999999e-06 2024-09-16T20:51:35.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.980 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.980 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:35.980 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822552+0000", 2024-09-16T20:51:35.980 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065524281000000004 2024-09-16T20:51:35.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.980 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.980 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:35.980 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822613+0000", 2024-09-16T20:51:35.980 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0430999999999998e-05 2024-09-16T20:51:35.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.980 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.980 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:35.981 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822753+0000", 2024-09-16T20:51:35.981 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000140226 2024-09-16T20:51:35.981 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:35.981 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:35.981 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:35.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.981 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.981 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12549 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 633856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:35.981 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:51:35.981 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 105.217384864, 2024-09-16T20:51:35.981 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065978518, 2024-09-16T20:51:35.981 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:35.981 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:35.982 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:35.982 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:35.982 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:35.982 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:35.982 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12549 2024-09-16T20:51:35.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.982 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:35.982 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.982 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:35.982 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:51:35.982 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:35.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.982 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.983 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:35.983 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:51:35.983 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:35.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.983 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.983 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:35.983 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:51:35.983 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9299999999999999e-07 2024-09-16T20:51:35.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.983 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.983 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:35.983 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757040+0000", 2024-09-16T20:51:35.983 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.677e-06 2024-09-16T20:51:35.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.984 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.984 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:35.984 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757045+0000", 2024-09-16T20:51:35.984 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8149999999999996e-06 2024-09-16T20:51:35.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.984 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.984 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:35.984 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757048+0000", 2024-09-16T20:51:35.984 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1e-06 2024-09-16T20:51:35.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.984 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.984 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:35.984 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822796+0000", 2024-09-16T20:51:35.985 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065748081 2024-09-16T20:51:35.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.985 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.985 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:35.985 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822867+0000", 2024-09-16T20:51:35.985 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0679000000000005e-05 2024-09-16T20:51:35.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.985 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.985 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:35.985 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823016+0000", 2024-09-16T20:51:35.985 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014987299999999999 2024-09-16T20:51:35.985 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:35.985 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:35.985 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:35.986 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.986 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.986 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12550 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 638976~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:35.986 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757060+0000", 2024-09-16T20:51:35.986 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 105.21736244900001, 2024-09-16T20:51:35.986 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066153564999999998, 2024-09-16T20:51:35.986 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:35.986 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:35.986 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:35.986 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:35.986 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:35.986 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:35.986 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12550 2024-09-16T20:51:35.987 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.987 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:35.987 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.987 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:35.987 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757060+0000", 2024-09-16T20:51:35.987 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:35.987 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.987 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.987 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:35.987 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757060+0000", 2024-09-16T20:51:35.987 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:35.987 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.987 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.987 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:35.988 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757061+0000", 2024-09-16T20:51:35.988 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.15e-07 2024-09-16T20:51:35.988 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.988 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.988 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:35.988 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757062+0000", 2024-09-16T20:51:35.988 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.663e-06 2024-09-16T20:51:35.988 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.988 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.988 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:35.988 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757066+0000", 2024-09-16T20:51:35.988 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.817e-06 2024-09-16T20:51:35.989 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.989 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.989 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:35.989 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757070+0000", 2024-09-16T20:51:35.989 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1509999999999997e-06 2024-09-16T20:51:35.989 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.989 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.989 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:35.989 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823044+0000", 2024-09-16T20:51:35.989 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065973992999999995 2024-09-16T20:51:35.989 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.989 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.989 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:35.989 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823082+0000", 2024-09-16T20:51:35.990 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8050999999999999e-05 2024-09-16T20:51:35.990 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.990 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.990 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:35.990 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823214+0000", 2024-09-16T20:51:35.990 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013157500000000001 2024-09-16T20:51:35.990 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:35.990 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:35.990 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:35.990 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.990 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.990 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12551 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 644096~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:35.990 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:51:35.990 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 105.217348751, 2024-09-16T20:51:35.991 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066391378000000001, 2024-09-16T20:51:35.991 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:35.991 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:35.991 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:35.991 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:35.991 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:35.991 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:35.991 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12551 2024-09-16T20:51:35.991 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.991 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:35.991 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.991 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:35.991 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:51:35.992 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:35.992 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.992 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.992 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:35.992 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:51:35.992 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:35.992 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.992 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.992 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:35.992 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:51:35.992 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2999999999999999e-07 2024-09-16T20:51:35.992 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.992 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.992 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:35.993 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757075+0000", 2024-09-16T20:51:35.993 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.097e-06 2024-09-16T20:51:35.993 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.993 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.993 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:35.993 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757082+0000", 2024-09-16T20:51:35.993 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9500000000000004e-06 2024-09-16T20:51:35.993 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.993 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.993 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:35.993 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757086+0000", 2024-09-16T20:51:35.993 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2229999999999999e-06 2024-09-16T20:51:35.993 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.994 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.994 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:35.994 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823262+0000", 2024-09-16T20:51:35.994 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066176078999999999 2024-09-16T20:51:35.994 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.994 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.994 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:35.994 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823336+0000", 2024-09-16T20:51:35.994 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3979000000000004e-05 2024-09-16T20:51:35.994 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.995 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.995 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:35.995 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823465+0000", 2024-09-16T20:51:35.995 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012982 2024-09-16T20:51:35.995 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:35.995 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:35.995 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:35.995 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.995 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.995 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12552 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 649216~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:35.995 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757089+0000", 2024-09-16T20:51:35.995 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 105.21733361, 2024-09-16T20:51:35.995 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066655617, 2024-09-16T20:51:35.995 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:35.996 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:35.996 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:35.996 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:35.996 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:35.996 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:35.996 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12552 2024-09-16T20:51:35.996 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.996 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:35.996 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.996 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:35.996 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757089+0000", 2024-09-16T20:51:35.996 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:35.996 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.997 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.997 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:35.997 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757089+0000", 2024-09-16T20:51:35.997 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:35.997 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.997 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.997 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:35.997 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759492+0000", 2024-09-16T20:51:35.997 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0024026109999999998 2024-09-16T20:51:35.997 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.997 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.997 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:35.997 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759497+0000", 2024-09-16T20:51:35.998 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1120000000000004e-06 2024-09-16T20:51:35.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.998 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.998 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:35.998 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759508+0000", 2024-09-16T20:51:35.998 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0981e-05 2024-09-16T20:51:35.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.998 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.998 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:35.998 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759516+0000", 2024-09-16T20:51:35.998 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7270000000000007e-06 2024-09-16T20:51:35.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.998 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.998 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:35.999 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823504+0000", 2024-09-16T20:51:35.999 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.063987984999999997 2024-09-16T20:51:35.999 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.999 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.999 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:35.999 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823579+0000", 2024-09-16T20:51:35.999 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5703000000000003e-05 2024-09-16T20:51:35.999 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:35.999 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:35.999 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:35.999 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823745+0000", 2024-09-16T20:51:35.999 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016549800000000001 2024-09-16T20:51:35.999 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.000 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:36.000 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.000 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.000 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.000 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12560 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 690176~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:36.000 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.759658+0000", 2024-09-16T20:51:36.000 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 105.21476440799999, 2024-09-16T20:51:36.000 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065858517000000005, 2024-09-16T20:51:36.000 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:36.000 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:36.000 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:36.000 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:36.000 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:36.001 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:36.001 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12560 2024-09-16T20:51:36.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.001 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:36.001 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.001 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:36.001 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759658+0000", 2024-09-16T20:51:36.001 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.001 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.001 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:36.001 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759658+0000", 2024-09-16T20:51:36.001 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.002 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:36.002 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759659+0000", 2024-09-16T20:51:36.002 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.17e-07 2024-09-16T20:51:36.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.002 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:36.002 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759660+0000", 2024-09-16T20:51:36.002 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0189999999999999e-06 2024-09-16T20:51:36.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.002 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:36.002 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759663+0000", 2024-09-16T20:51:36.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0010000000000002e-06 2024-09-16T20:51:36.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.003 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.003 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:36.003 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759665+0000", 2024-09-16T20:51:36.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5440000000000001e-06 2024-09-16T20:51:36.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.003 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.003 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:36.003 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825355+0000", 2024-09-16T20:51:36.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065690224000000005 2024-09-16T20:51:36.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.003 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.004 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:36.004 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825394+0000", 2024-09-16T20:51:36.004 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8810999999999998e-05 2024-09-16T20:51:36.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.004 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.004 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:36.004 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825517+0000", 2024-09-16T20:51:36.004 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012270099999999999 2024-09-16T20:51:36.004 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.004 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:36.004 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.004 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.005 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12561 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 695296~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:36.005 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.759668+0000", 2024-09-16T20:51:36.005 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 105.214754566, 2024-09-16T20:51:36.005 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066116999999999995, 2024-09-16T20:51:36.005 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:36.005 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:36.005 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:36.005 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:36.005 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:36.005 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:36.005 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12561 2024-09-16T20:51:36.005 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.005 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:36.006 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.006 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:36.006 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759668+0000", 2024-09-16T20:51:36.006 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.006 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.006 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:36.006 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759668+0000", 2024-09-16T20:51:36.006 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.006 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.006 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:36.006 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765379+0000", 2024-09-16T20:51:36.007 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057104410000000001 2024-09-16T20:51:36.007 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.007 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.007 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:36.007 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765383+0000", 2024-09-16T20:51:36.007 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2980000000000002e-06 2024-09-16T20:51:36.007 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.007 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.007 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:36.007 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765395+0000", 2024-09-16T20:51:36.007 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1671000000000001e-05 2024-09-16T20:51:36.007 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.007 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.007 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:36.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765403+0000", 2024-09-16T20:51:36.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1210000000000007e-06 2024-09-16T20:51:36.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.008 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:36.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825552+0000", 2024-09-16T20:51:36.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.060149498000000003 2024-09-16T20:51:36.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.008 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:36.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825625+0000", 2024-09-16T20:51:36.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2540999999999995e-05 2024-09-16T20:51:36.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.009 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.009 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:36.009 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825785+0000", 2024-09-16T20:51:36.009 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016043000000000001 2024-09-16T20:51:36.009 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.009 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:36.009 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.009 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.009 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12585 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 818176~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:36.009 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.765848+0000", 2024-09-16T20:51:36.009 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 105.208574595, 2024-09-16T20:51:36.010 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065843218999999995, 2024-09-16T20:51:36.010 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:36.010 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:36.010 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:36.010 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:36.010 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:36.010 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:36.010 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12585 2024-09-16T20:51:36.010 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.010 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:36.010 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.010 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:36.010 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765848+0000", 2024-09-16T20:51:36.011 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.011 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.011 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:36.011 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765848+0000", 2024-09-16T20:51:36.011 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.011 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.011 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:36.011 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768683+0000", 2024-09-16T20:51:36.011 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0028344519999999999 2024-09-16T20:51:36.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.011 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.012 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:36.012 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768686+0000", 2024-09-16T20:51:36.012 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8789999999999996e-06 2024-09-16T20:51:36.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.012 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.012 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:36.012 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768696+0000", 2024-09-16T20:51:36.012 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6709999999999994e-06 2024-09-16T20:51:36.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.012 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.012 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:36.012 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768702+0000", 2024-09-16T20:51:36.012 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2550000000000003e-06 2024-09-16T20:51:36.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.013 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.013 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:36.013 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.831499+0000", 2024-09-16T20:51:36.013 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.062797081000000005 2024-09-16T20:51:36.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.013 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.013 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:36.013 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.831538+0000", 2024-09-16T20:51:36.013 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8523999999999999e-05 2024-09-16T20:51:36.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.013 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.013 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:36.014 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.831691+0000", 2024-09-16T20:51:36.014 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015335699999999999 2024-09-16T20:51:36.014 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.014 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:36.014 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.014 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.014 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12597 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 879616~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:36.014 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.768883+0000", 2024-09-16T20:51:36.014 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 105.205539867, 2024-09-16T20:51:36.014 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065816527, 2024-09-16T20:51:36.014 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:36.014 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:36.015 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:36.015 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:36.015 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:36.015 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:36.015 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12597 2024-09-16T20:51:36.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.015 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:36.015 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.015 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:36.015 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768883+0000", 2024-09-16T20:51:36.015 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.015 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.015 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:36.016 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768883+0000", 2024-09-16T20:51:36.016 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.016 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.016 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:36.016 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771508+0000", 2024-09-16T20:51:36.016 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0026247589999999999 2024-09-16T20:51:36.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.016 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.016 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:36.016 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771513+0000", 2024-09-16T20:51:36.016 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9949999999999996e-06 2024-09-16T20:51:36.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.016 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.017 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:36.017 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771525+0000", 2024-09-16T20:51:36.017 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2033e-05 2024-09-16T20:51:36.017 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.017 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.017 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:36.017 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771532+0000", 2024-09-16T20:51:36.017 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4220000000000003e-06 2024-09-16T20:51:36.017 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.017 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.017 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:36.017 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.834532+0000", 2024-09-16T20:51:36.017 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.063000159 2024-09-16T20:51:36.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.018 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.018 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:36.018 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.834590+0000", 2024-09-16T20:51:36.018 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8218999999999999e-05 2024-09-16T20:51:36.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.018 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.018 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:36.018 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.834699+0000", 2024-09-16T20:51:36.018 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010894 2024-09-16T20:51:36.018 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.018 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:36.018 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.019 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12619 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 992256~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:36.019 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.774656+0000", 2024-09-16T20:51:36.019 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 105.19976709300001, 2024-09-16T20:51:36.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065715859000000001, 2024-09-16T20:51:36.019 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:36.019 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:36.019 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:36.019 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:36.019 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:36.019 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:36.019 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12619 2024-09-16T20:51:36.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.020 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:36.020 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.020 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:36.020 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774656+0000", 2024-09-16T20:51:36.020 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.020 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.020 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.020 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:36.020 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774656+0000", 2024-09-16T20:51:36.020 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.020 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.020 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.020 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:36.021 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774875+0000", 2024-09-16T20:51:36.021 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000218898 2024-09-16T20:51:36.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.021 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.021 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:36.021 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774879+0000", 2024-09-16T20:51:36.021 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0389999999999998e-06 2024-09-16T20:51:36.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.021 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.021 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:36.021 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774890+0000", 2024-09-16T20:51:36.021 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1250000000000001e-05 2024-09-16T20:51:36.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.021 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.022 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:36.022 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774897+0000", 2024-09-16T20:51:36.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7619999999999998e-06 2024-09-16T20:51:36.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.022 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.022 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:36.022 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840139+0000", 2024-09-16T20:51:36.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065242438 2024-09-16T20:51:36.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.022 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.022 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:36.022 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840186+0000", 2024-09-16T20:51:36.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6894999999999997e-05 2024-09-16T20:51:36.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.023 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.023 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:36.023 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840372+0000", 2024-09-16T20:51:36.023 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000185577 2024-09-16T20:51:36.023 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.023 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:36.023 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.023 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.023 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12620 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 997376~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:36.023 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.774908+0000", 2024-09-16T20:51:36.023 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 105.19951487199999, 2024-09-16T20:51:36.023 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065782754999999998, 2024-09-16T20:51:36.024 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:36.024 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:36.024 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:36.024 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:36.024 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:36.024 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:36.024 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12620 2024-09-16T20:51:36.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.024 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:36.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.024 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:36.024 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774908+0000", 2024-09-16T20:51:36.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:36.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774908+0000", 2024-09-16T20:51:36.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:36.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775102+0000", 2024-09-16T20:51:36.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000194098 2024-09-16T20:51:36.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:36.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775105+0000", 2024-09-16T20:51:36.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8739999999999999e-06 2024-09-16T20:51:36.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.026 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:36.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775111+0000", 2024-09-16T20:51:36.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9519999999999999e-06 2024-09-16T20:51:36.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.026 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:36.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775115+0000", 2024-09-16T20:51:36.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.002e-06 2024-09-16T20:51:36.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.026 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:36.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840409+0000", 2024-09-16T20:51:36.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065294195999999999 2024-09-16T20:51:36.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.027 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.027 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:36.027 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840507+0000", 2024-09-16T20:51:36.027 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7662000000000001e-05 2024-09-16T20:51:36.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.027 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.027 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:36.027 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840691+0000", 2024-09-16T20:51:36.027 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000183971 2024-09-16T20:51:36.027 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.027 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:36.027 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.028 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.028 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12621 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1002496~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:36.028 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775123+0000", 2024-09-16T20:51:36.028 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 105.19930017599999, 2024-09-16T20:51:36.028 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065829586999999995, 2024-09-16T20:51:36.028 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:36.028 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:36.028 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:36.028 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:36.028 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:36.028 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:36.028 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12621 2024-09-16T20:51:36.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.029 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:36.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.029 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:36.029 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775123+0000", 2024-09-16T20:51:36.029 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.029 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:36.029 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775123+0000", 2024-09-16T20:51:36.029 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.029 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:36.030 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775348+0000", 2024-09-16T20:51:36.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000225295 2024-09-16T20:51:36.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.030 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:36.030 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775352+0000", 2024-09-16T20:51:36.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0810000000000004e-06 2024-09-16T20:51:36.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.030 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:36.030 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775361+0000", 2024-09-16T20:51:36.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7399999999999993e-06 2024-09-16T20:51:36.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.031 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.031 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:36.031 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775367+0000", 2024-09-16T20:51:36.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5200000000000003e-06 2024-09-16T20:51:36.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.031 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.031 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:36.031 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840730+0000", 2024-09-16T20:51:36.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065362803999999997 2024-09-16T20:51:36.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.031 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.031 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:36.031 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840801+0000", 2024-09-16T20:51:36.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1006000000000004e-05 2024-09-16T20:51:36.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.032 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:36.032 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840952+0000", 2024-09-16T20:51:36.032 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015114100000000001 2024-09-16T20:51:36.032 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.032 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:36.032 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.032 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12622 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1007616~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:36.032 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775381+0000", 2024-09-16T20:51:36.032 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 105.199041342, 2024-09-16T20:51:36.032 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065822581000000005, 2024-09-16T20:51:36.033 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:36.033 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:36.033 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:36.033 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:36.033 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:36.033 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:36.033 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12622 2024-09-16T20:51:36.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.033 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:36.033 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.033 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:36.033 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775381+0000", 2024-09-16T20:51:36.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.033 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.034 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:36.034 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775381+0000", 2024-09-16T20:51:36.034 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.034 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.034 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:36.034 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775563+0000", 2024-09-16T20:51:36.034 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018153700000000001 2024-09-16T20:51:36.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.034 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.034 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:36.034 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775566+0000", 2024-09-16T20:51:36.034 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9620000000000002e-06 2024-09-16T20:51:36.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.034 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.035 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:36.035 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775572+0000", 2024-09-16T20:51:36.035 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8529999999999997e-06 2024-09-16T20:51:36.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.035 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:36.035 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775576+0000", 2024-09-16T20:51:36.035 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8920000000000002e-06 2024-09-16T20:51:36.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.035 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:36.035 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840990+0000", 2024-09-16T20:51:36.035 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065414599000000004 2024-09-16T20:51:36.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.036 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:36.036 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841068+0000", 2024-09-16T20:51:36.036 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8157000000000004e-05 2024-09-16T20:51:36.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.036 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:36.036 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841204+0000", 2024-09-16T20:51:36.036 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013558099999999999 2024-09-16T20:51:36.036 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.036 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:36.036 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.037 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.037 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12623 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1012736~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:36.037 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775583+0000", 2024-09-16T20:51:36.037 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 105.198839904, 2024-09-16T20:51:36.037 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065929087999999997, 2024-09-16T20:51:36.037 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:36.037 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:36.037 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:36.037 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:36.037 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:36.037 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:36.037 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12623 2024-09-16T20:51:36.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.037 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:36.038 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.038 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:36.038 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775583+0000", 2024-09-16T20:51:36.038 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.038 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.038 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:36.038 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775583+0000", 2024-09-16T20:51:36.038 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.038 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.038 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:36.038 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775804+0000", 2024-09-16T20:51:36.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00022116900000000001 2024-09-16T20:51:36.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.039 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:36.039 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775807+0000", 2024-09-16T20:51:36.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9529999999999999e-06 2024-09-16T20:51:36.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.039 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:36.039 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775814+0000", 2024-09-16T20:51:36.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7490000000000001e-06 2024-09-16T20:51:36.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.039 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:36.040 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775818+0000", 2024-09-16T20:51:36.040 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1999999999999996e-06 2024-09-16T20:51:36.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.040 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.040 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:36.040 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841267+0000", 2024-09-16T20:51:36.040 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065449127999999995 2024-09-16T20:51:36.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.040 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.040 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:36.040 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841344+0000", 2024-09-16T20:51:36.040 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6546999999999997e-05 2024-09-16T20:51:36.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.040 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.041 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:36.041 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841512+0000", 2024-09-16T20:51:36.041 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016834200000000001 2024-09-16T20:51:36.041 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.041 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:36.041 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.041 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.041 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.041 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12624 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1017856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:36.041 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:51:36.041 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 105.198596764, 2024-09-16T20:51:36.041 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065945866000000006, 2024-09-16T20:51:36.041 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:36.041 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:36.042 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:36.042 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:36.042 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:36.042 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:36.042 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12624 2024-09-16T20:51:36.042 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.042 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:36.042 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.042 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:36.042 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:51:36.042 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.042 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.042 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.042 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:36.042 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:51:36.043 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.043 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.043 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:36.043 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:51:36.043 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-16T20:51:36.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.043 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.043 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:36.043 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775828+0000", 2024-09-16T20:51:36.043 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3999999999999999e-06 2024-09-16T20:51:36.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.043 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.043 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:36.044 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775831+0000", 2024-09-16T20:51:36.044 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.478e-06 2024-09-16T20:51:36.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.044 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.044 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:36.044 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775834+0000", 2024-09-16T20:51:36.044 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9440000000000001e-06 2024-09-16T20:51:36.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.044 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.044 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:36.044 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841547+0000", 2024-09-16T20:51:36.044 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065712801000000001 2024-09-16T20:51:36.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.044 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.044 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:36.045 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841618+0000", 2024-09-16T20:51:36.045 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1151999999999999e-05 2024-09-16T20:51:36.045 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.045 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.045 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:36.045 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841772+0000", 2024-09-16T20:51:36.045 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000153721 2024-09-16T20:51:36.045 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.045 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:36.045 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.045 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.045 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.045 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12625 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1022976~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:36.045 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775840+0000", 2024-09-16T20:51:36.045 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 105.19858315499999, 2024-09-16T20:51:36.046 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066165451, 2024-09-16T20:51:36.046 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:36.046 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:36.046 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:36.046 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:36.046 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:36.046 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:36.046 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12625 2024-09-16T20:51:36.046 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.046 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:36.046 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.046 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:36.046 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775840+0000", 2024-09-16T20:51:36.046 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.046 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.047 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.047 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:36.047 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775840+0000", 2024-09-16T20:51:36.047 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.047 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.047 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:36.047 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776014+0000", 2024-09-16T20:51:36.047 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017441099999999999 2024-09-16T20:51:36.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.047 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.047 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:36.047 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776018+0000", 2024-09-16T20:51:36.047 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6509999999999999e-06 2024-09-16T20:51:36.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.048 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.048 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:36.048 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776026+0000", 2024-09-16T20:51:36.048 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.028e-06 2024-09-16T20:51:36.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.048 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.048 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:36.048 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776031+0000", 2024-09-16T20:51:36.048 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8400000000000002e-06 2024-09-16T20:51:36.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.048 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.048 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:36.049 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841801+0000", 2024-09-16T20:51:36.049 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065770472999999996 2024-09-16T20:51:36.049 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.049 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.049 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:36.049 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841847+0000", 2024-09-16T20:51:36.049 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5558e-05 2024-09-16T20:51:36.049 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.049 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.049 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:36.049 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842005+0000", 2024-09-16T20:51:36.049 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015849000000000001 2024-09-16T20:51:36.049 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.050 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:36.050 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.050 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.050 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.050 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12626 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1028096~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:36.050 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776041+0000", 2024-09-16T20:51:36.050 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 105.198381856, 2024-09-16T20:51:36.050 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066225260999999994, 2024-09-16T20:51:36.050 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:36.050 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:36.050 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:36.050 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:36.050 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:36.050 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:36.051 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12626 2024-09-16T20:51:36.051 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.051 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:36.051 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.051 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:36.051 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776041+0000", 2024-09-16T20:51:36.051 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.051 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.051 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.051 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:36.051 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776041+0000", 2024-09-16T20:51:36.051 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.051 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.051 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.051 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:36.052 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776263+0000", 2024-09-16T20:51:36.052 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00022193899999999999 2024-09-16T20:51:36.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.052 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.052 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:36.052 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776268+0000", 2024-09-16T20:51:36.052 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8049999999999997e-06 2024-09-16T20:51:36.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.052 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.052 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:36.052 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776280+0000", 2024-09-16T20:51:36.052 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2635e-05 2024-09-16T20:51:36.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.052 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.053 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:36.053 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776288+0000", 2024-09-16T20:51:36.053 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8199999999999997e-06 2024-09-16T20:51:36.053 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.053 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.053 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:36.053 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842041+0000", 2024-09-16T20:51:36.053 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065752506000000002 2024-09-16T20:51:36.053 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.053 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.053 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:36.053 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842123+0000", 2024-09-16T20:51:36.053 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2393999999999998e-05 2024-09-16T20:51:36.053 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.053 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.054 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:36.054 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842266+0000", 2024-09-16T20:51:36.054 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014316200000000001 2024-09-16T20:51:36.054 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.054 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:36.054 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.054 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.054 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.054 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12627 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1033216~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:36.054 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776301+0000", 2024-09-16T20:51:36.054 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 105.198121896, 2024-09-16T20:51:36.054 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066187814999999997, 2024-09-16T20:51:36.054 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:36.054 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:36.054 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:36.055 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:36.055 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:36.055 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:36.055 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12627 2024-09-16T20:51:36.055 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.055 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:36.055 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.055 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:36.055 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776301+0000", 2024-09-16T20:51:36.055 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.055 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.055 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.055 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:36.055 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776301+0000", 2024-09-16T20:51:36.055 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.056 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.056 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.056 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:36.056 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776473+0000", 2024-09-16T20:51:36.056 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017222400000000001 2024-09-16T20:51:36.056 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.056 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.056 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:36.056 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776477+0000", 2024-09-16T20:51:36.056 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7019999999999999e-06 2024-09-16T20:51:36.056 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.056 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.056 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:36.056 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776486+0000", 2024-09-16T20:51:36.057 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2119999999999992e-06 2024-09-16T20:51:36.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.057 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.057 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:36.057 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776492+0000", 2024-09-16T20:51:36.057 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8089999999999998e-06 2024-09-16T20:51:36.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.057 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.057 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:36.057 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842301+0000", 2024-09-16T20:51:36.057 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065809042999999998 2024-09-16T20:51:36.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.057 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.058 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:36.058 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842362+0000", 2024-09-16T20:51:36.058 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1260000000000006e-05 2024-09-16T20:51:36.058 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.058 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.058 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:36.058 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842489+0000", 2024-09-16T20:51:36.058 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012656500000000001 2024-09-16T20:51:36.058 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.058 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:36.058 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.058 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.058 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.059 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12628 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1038336~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:36.059 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776502+0000", 2024-09-16T20:51:36.059 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 105.19792124999999, 2024-09-16T20:51:36.059 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066283426000000006, 2024-09-16T20:51:36.059 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:36.059 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:36.059 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:36.059 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:36.059 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:36.059 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:36.059 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12628 2024-09-16T20:51:36.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.059 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:36.059 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.060 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:36.060 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776502+0000", 2024-09-16T20:51:36.060 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.060 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.060 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.060 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:36.060 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776502+0000", 2024-09-16T20:51:36.060 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.060 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.060 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.060 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:36.060 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776716+0000", 2024-09-16T20:51:36.060 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000214596 2024-09-16T20:51:36.060 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.061 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.061 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:36.061 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776723+0000", 2024-09-16T20:51:36.061 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4000000000000003e-06 2024-09-16T20:51:36.061 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.061 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.061 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:36.061 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776731+0000", 2024-09-16T20:51:36.061 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6129999999999998e-06 2024-09-16T20:51:36.061 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.061 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.061 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:36.062 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776737+0000", 2024-09-16T20:51:36.062 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1970000000000001e-06 2024-09-16T20:51:36.062 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.062 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.062 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:36.062 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842532+0000", 2024-09-16T20:51:36.062 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065795127999999994 2024-09-16T20:51:36.062 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.062 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.062 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:36.062 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842599+0000", 2024-09-16T20:51:36.062 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6711000000000006e-05 2024-09-16T20:51:36.062 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.062 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.063 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:36.063 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842785+0000", 2024-09-16T20:51:36.063 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018578099999999999 2024-09-16T20:51:36.063 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.063 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:36.063 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.063 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.063 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12629 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1043456~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:36.063 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776748+0000", 2024-09-16T20:51:36.063 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 105.197674873, 2024-09-16T20:51:36.063 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066303788000000002, 2024-09-16T20:51:36.063 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:36.063 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:36.064 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:36.064 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:36.064 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:36.064 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:36.064 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12629 2024-09-16T20:51:36.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.064 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:36.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.064 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:36.064 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776748+0000", 2024-09-16T20:51:36.064 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.064 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:36.065 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776748+0000", 2024-09-16T20:51:36.065 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.065 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.065 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.065 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:36.065 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776934+0000", 2024-09-16T20:51:36.065 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018598399999999999 2024-09-16T20:51:36.065 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.065 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.065 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:36.065 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776939+0000", 2024-09-16T20:51:36.065 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1000000000000003e-06 2024-09-16T20:51:36.065 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.066 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.066 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:36.066 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776954+0000", 2024-09-16T20:51:36.066 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5221e-05 2024-09-16T20:51:36.066 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.066 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.066 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:36.066 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776962+0000", 2024-09-16T20:51:36.066 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.554e-06 2024-09-16T20:51:36.066 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.066 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.066 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:36.066 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842813+0000", 2024-09-16T20:51:36.066 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065851006000000004 2024-09-16T20:51:36.066 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.067 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.067 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:36.067 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842886+0000", 2024-09-16T20:51:36.067 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3663000000000003e-05 2024-09-16T20:51:36.067 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.067 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.067 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:36.067 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.843052+0000", 2024-09-16T20:51:36.067 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016526 2024-09-16T20:51:36.067 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.067 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:36.067 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.067 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.068 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:36.068 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:36.069 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:51:36.070 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:36.068+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:36.170 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:51:36.169+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:51:36.217 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:36.217 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:51:36.217 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:51:36.218 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:36.227 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:51:36.271 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:36.270+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:36.372 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:36.371+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:36.373 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:36.374 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:51:36.374 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:51:36.374 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:51:36.374 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:36.382 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:51:36.473 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:51:36.471+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:51:36.491 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:36.491 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:51:36.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:51:36.492 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:51:36.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.492 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5286.0:10643 3.17 3.f6273fb7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-16T20:51:36.492 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:51:36.492 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 147.699494996, 2024-09-16T20:51:36.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.76672347200000002, 2024-09-16T20:51:36.492 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:36.492 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:36.492 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:36.492 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:36.493 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5286", 2024-09-16T20:51:36.493 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/4135433989", 2024-09-16T20:51:36.493 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 10643 2024-09-16T20:51:36.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.493 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:36.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.493 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:36.493 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:51:36.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.493 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:36.493 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:51:36.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.494 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:36.494 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790489+0000", 2024-09-16T20:51:36.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5910000000000001e-06 2024-09-16T20:51:36.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.494 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:36.494 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790493+0000", 2024-09-16T20:51:36.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2669999999999998e-06 2024-09-16T20:51:36.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.494 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:36.494 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790500+0000", 2024-09-16T20:51:36.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3780000000000003e-06 2024-09-16T20:51:36.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:36.495 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790505+0000", 2024-09-16T20:51:36.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1560000000000003e-06 2024-09-16T20:51:36.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:36.495 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790559+0000", 2024-09-16T20:51:36.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3494e-05 2024-09-16T20:51:36.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:36.495 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790566+0000", 2024-09-16T20:51:36.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7649999999999993e-06 2024-09-16T20:51:36.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:36.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:09.557198+0000", 2024-09-16T20:51:36.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.76663136200000004 2024-09-16T20:51:36.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:36.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:09.557211+0000", 2024-09-16T20:51:36.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3458999999999999e-05 2024-09-16T20:51:36.496 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.496 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:36.497 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.497 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5512.0:450 3.2 3:44e0cffa:::rbd_header.110ef0d7a535:head [watch reconnect cookie 140263298018912 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:51:36.497 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:51:36.497 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 119.927472083, 2024-09-16T20:51:36.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0084581939999999, 2024-09-16T20:51:36.497 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:36.497 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:36.497 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:36.497 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:36.497 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5512", 2024-09-16T20:51:36.497 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:51:36.498 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 450 2024-09-16T20:51:36.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.498 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:36.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.498 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:36.498 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:51:36.498 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.498 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:36.498 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:51:36.498 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.499 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.499 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:36.499 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562512+0000", 2024-09-16T20:51:36.499 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.009e-06 2024-09-16T20:51:36.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.499 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.499 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:36.499 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562517+0000", 2024-09-16T20:51:36.499 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9339999999999997e-06 2024-09-16T20:51:36.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.499 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.499 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:36.499 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562530+0000", 2024-09-16T20:51:36.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3495e-05 2024-09-16T20:51:36.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.500 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:36.500 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562536+0000", 2024-09-16T20:51:36.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5690000000000004e-06 2024-09-16T20:51:36.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.500 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:36.500 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562591+0000", 2024-09-16T20:51:36.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5761999999999999e-05 2024-09-16T20:51:36.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:51:36.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562599+0000", 2024-09-16T20:51:36.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9179999999999994e-06 2024-09-16T20:51:36.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:36.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.565971+0000", 2024-09-16T20:51:36.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003371546 2024-09-16T20:51:36.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:36.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566019+0000", 2024-09-16T20:51:36.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7849000000000002e-05 2024-09-16T20:51:36.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.502 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:36.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.569382+0000", 2024-09-16T20:51:36.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.003362833 2024-09-16T20:51:36.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.502 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:36.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.569529+0000", 2024-09-16T20:51:36.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000147823 2024-09-16T20:51:36.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.502 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:36.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570969+0000", 2024-09-16T20:51:36.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001439456 2024-09-16T20:51:36.503 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.503 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:36.503 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.503 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:14996 3.2 3:44e0cffa:::rbd_header.110ef0d7a535:head [watch reconnect cookie 140213436269840 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:51:36.503 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:51:36.503 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 119.927423671, 2024-09-16T20:51:36.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.008813943, 2024-09-16T20:51:36.503 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:36.503 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:36.503 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:36.503 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:36.504 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:36.504 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:36.504 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 14996 2024-09-16T20:51:36.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.504 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:36.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.504 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:36.504 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:51:36.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.504 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:36.504 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:51:36.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:36.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562560+0000", 2024-09-16T20:51:36.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.06e-07 2024-09-16T20:51:36.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:36.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562563+0000", 2024-09-16T20:51:36.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.495e-06 2024-09-16T20:51:36.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:36.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562571+0000", 2024-09-16T20:51:36.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1950000000000003e-06 2024-09-16T20:51:36.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:36.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562579+0000", 2024-09-16T20:51:36.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2649999999999999e-06 2024-09-16T20:51:36.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:36.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562673+0000", 2024-09-16T20:51:36.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4118999999999998e-05 2024-09-16T20:51:36.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.507 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.507 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:51:36.507 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562677+0000", 2024-09-16T20:51:36.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5179999999999998e-06 2024-09-16T20:51:36.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.507 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.507 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:36.507 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566461+0000", 2024-09-16T20:51:36.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0037837259999999998 2024-09-16T20:51:36.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.507 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.507 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:36.507 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566468+0000", 2024-09-16T20:51:36.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3350000000000001e-06 2024-09-16T20:51:36.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.508 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:36.508 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570306+0000", 2024-09-16T20:51:36.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.003837997 2024-09-16T20:51:36.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.508 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:36.508 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570392+0000", 2024-09-16T20:51:36.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5872999999999996e-05 2024-09-16T20:51:36.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:36.509 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571373+0000", 2024-09-16T20:51:36.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00098071399999999993 2024-09-16T20:51:36.509 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.509 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:36.509 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.509 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5255.0:1931 3.2 3:47882819:::rbd_data.110eb5f95afc.0000000000000009:head [read 217088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:51:36.509 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:51:36.509 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 119.927384921, 2024-09-16T20:51:36.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0082752880000001, 2024-09-16T20:51:36.509 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:36.509 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:36.510 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:36.510 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:36.510 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5255", 2024-09-16T20:51:36.510 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:51:36.510 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1931 2024-09-16T20:51:36.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.510 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:36.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:36.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:51:36.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:36.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:51:36.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:36.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:51:36.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9699999999999996e-07 2024-09-16T20:51:36.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:36.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562601+0000", 2024-09-16T20:51:36.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8600000000000001e-06 2024-09-16T20:51:36.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:36.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562608+0000", 2024-09-16T20:51:36.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9e-06 2024-09-16T20:51:36.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.512 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.512 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:36.512 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562615+0000", 2024-09-16T20:51:36.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8179999999999998e-06 2024-09-16T20:51:36.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.512 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.512 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:36.512 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562703+0000", 2024-09-16T20:51:36.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7811000000000003e-05 2024-09-16T20:51:36.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.512 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.513 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:51:36.513 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562706+0000", 2024-09-16T20:51:36.513 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4920000000000002e-06 2024-09-16T20:51:36.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.513 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:36.513 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566685+0000", 2024-09-16T20:51:36.513 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0039792780000000002 2024-09-16T20:51:36.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.513 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:36.513 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566693+0000", 2024-09-16T20:51:36.513 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4730000000000003e-06 2024-09-16T20:51:36.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:36.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570737+0000", 2024-09-16T20:51:36.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0040436749999999 2024-09-16T20:51:36.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:36.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570873+0000", 2024-09-16T20:51:36.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000136484 2024-09-16T20:51:36.514 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.514 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:36.514 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.515 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5255.0:1939 3.2 3:47882819:::rbd_data.110eb5f95afc.0000000000000009:head [read 233472~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:51:36.515 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:51:36.515 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 119.927230055, 2024-09-16T20:51:36.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0084657589999999, 2024-09-16T20:51:36.515 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:36.515 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:36.515 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:36.515 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:36.515 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5255", 2024-09-16T20:51:36.516 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:51:36.516 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1939 2024-09-16T20:51:36.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.516 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:36.516 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.516 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:36.516 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:51:36.516 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.516 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.516 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:36.516 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:51:36.517 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.517 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:36.517 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:51:36.517 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5199999999999997e-07 2024-09-16T20:51:36.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.517 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:36.517 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562757+0000", 2024-09-16T20:51:36.517 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4979999999999998e-06 2024-09-16T20:51:36.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.517 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:36.517 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562764+0000", 2024-09-16T20:51:36.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1470000000000004e-06 2024-09-16T20:51:36.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.518 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:36.518 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562768+0000", 2024-09-16T20:51:36.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7929999999999999e-06 2024-09-16T20:51:36.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.518 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:36.518 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562805+0000", 2024-09-16T20:51:36.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7477e-05 2024-09-16T20:51:36.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.518 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting_for_map not empty", 2024-09-16T20:51:36.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562810+0000", 2024-09-16T20:51:36.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0819999999999998e-06 2024-09-16T20:51:36.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.519 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:36.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566717+0000", 2024-09-16T20:51:36.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0039072600000000001 2024-09-16T20:51:36.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.519 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:36.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566724+0000", 2024-09-16T20:51:36.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4880000000000004e-06 2024-09-16T20:51:36.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.519 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:36.520 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571009+0000", 2024-09-16T20:51:36.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.004284677 2024-09-16T20:51:36.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.520 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:36.520 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571218+0000", 2024-09-16T20:51:36.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00020978500000000001 2024-09-16T20:51:36.520 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.520 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:36.520 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.520 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:11021 3.1f 3:fee2d1dd:::rbd_data.110ea93691bb.000000000000000e:23 [read 960512~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:36.521 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.693837+0000", 2024-09-16T20:51:36.521 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 105.79614551, 2024-09-16T20:51:36.521 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057155138000000001, 2024-09-16T20:51:36.521 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:36.521 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:36.521 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:36.521 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:36.521 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:36.521 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:36.521 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 11021 2024-09-16T20:51:36.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.521 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:36.521 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.521 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:36.522 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693837+0000", 2024-09-16T20:51:36.522 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.522 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:36.522 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693837+0000", 2024-09-16T20:51:36.522 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.522 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:36.522 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699566+0000", 2024-09-16T20:51:36.522 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057283639999999997 2024-09-16T20:51:36.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:36.523 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699571+0000", 2024-09-16T20:51:36.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.604e-06 2024-09-16T20:51:36.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:36.523 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699583+0000", 2024-09-16T20:51:36.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1372999999999999e-05 2024-09-16T20:51:36.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:36.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699591+0000", 2024-09-16T20:51:36.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5259999999999993e-06 2024-09-16T20:51:36.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.524 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:36.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750674+0000", 2024-09-16T20:51:36.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051083154999999998 2024-09-16T20:51:36.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.524 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:36.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750745+0000", 2024-09-16T20:51:36.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0933999999999995e-05 2024-09-16T20:51:36.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.525 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.525 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:36.525 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750992+0000", 2024-09-16T20:51:36.525 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00024718199999999998 2024-09-16T20:51:36.525 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.525 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:36.525 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.525 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.525 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8792 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2609152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:36.525 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:51:36.526 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 95.432685793000005, 2024-09-16T20:51:36.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057003524999999999, 2024-09-16T20:51:36.526 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:36.526 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:36.526 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:36.526 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:36.526 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:36.526 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:36.526 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8792 2024-09-16T20:51:36.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.526 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:36.526 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.526 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:36.526 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:51:36.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.527 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:36.527 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:51:36.527 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.527 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:36.527 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:51:36.527 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0400000000000002e-07 2024-09-16T20:51:36.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.527 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:36.527 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057299+0000", 2024-09-16T20:51:36.527 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7460000000000001e-06 2024-09-16T20:51:36.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:36.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057304+0000", 2024-09-16T20:51:36.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3210000000000004e-06 2024-09-16T20:51:36.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:36.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057310+0000", 2024-09-16T20:51:36.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4360000000000001e-06 2024-09-16T20:51:36.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:36.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114200+0000", 2024-09-16T20:51:36.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056889954 2024-09-16T20:51:36.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.529 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:36.529 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114243+0000", 2024-09-16T20:51:36.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2998999999999999e-05 2024-09-16T20:51:36.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.529 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:36.529 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114300+0000", 2024-09-16T20:51:36.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7765e-05 2024-09-16T20:51:36.529 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.529 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:36.529 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.530 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8794 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2351104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:36.530 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057327+0000", 2024-09-16T20:51:36.530 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 95.432655468999997, 2024-09-16T20:51:36.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057101130999999999, 2024-09-16T20:51:36.530 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:36.530 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:36.530 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:36.530 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:36.530 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:36.530 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:36.530 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8794 2024-09-16T20:51:36.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.530 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:36.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:36.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057327+0000", 2024-09-16T20:51:36.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:36.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057327+0000", 2024-09-16T20:51:36.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:36.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057328+0000", 2024-09-16T20:51:36.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7399999999999999e-07 2024-09-16T20:51:36.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.532 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:36.532 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057330+0000", 2024-09-16T20:51:36.532 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2199999999999999e-06 2024-09-16T20:51:36.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.532 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:36.532 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057336+0000", 2024-09-16T20:51:36.532 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1240000000000002e-06 2024-09-16T20:51:36.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.532 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:36.532 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057341+0000", 2024-09-16T20:51:36.532 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8470000000000003e-06 2024-09-16T20:51:36.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.533 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.533 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:36.533 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114334+0000", 2024-09-16T20:51:36.533 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056993267 2024-09-16T20:51:36.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.533 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.533 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:36.533 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114376+0000", 2024-09-16T20:51:36.533 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1638999999999999e-05 2024-09-16T20:51:36.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.533 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.533 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:36.533 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114428+0000", 2024-09-16T20:51:36.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2660000000000001e-05 2024-09-16T20:51:36.534 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.534 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:36.534 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.534 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8798 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2297856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:36.534 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:51:36.534 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 95.432635742000002, 2024-09-16T20:51:36.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057277041000000001, 2024-09-16T20:51:36.534 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:36.534 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:36.534 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:36.534 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:36.535 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:36.535 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:36.535 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8798 2024-09-16T20:51:36.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.535 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:36.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:36.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:51:36.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:36.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:51:36.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.536 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:36.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:51:36.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2800000000000003e-07 2024-09-16T20:51:36.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.536 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:36.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057354+0000", 2024-09-16T20:51:36.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7780000000000002e-06 2024-09-16T20:51:36.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.536 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:36.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057360+0000", 2024-09-16T20:51:36.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0419999999999999e-06 2024-09-16T20:51:36.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:36.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057365+0000", 2024-09-16T20:51:36.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7929999999999997e-06 2024-09-16T20:51:36.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:36.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114460+0000", 2024-09-16T20:51:36.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057094662999999997 2024-09-16T20:51:36.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:36.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114532+0000", 2024-09-16T20:51:36.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2881999999999998e-05 2024-09-16T20:51:36.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.538 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:36.538 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114624+0000", 2024-09-16T20:51:36.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1555e-05 2024-09-16T20:51:36.538 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.538 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:36.538 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.538 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8800 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2316288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:36.538 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:51:36.538 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 95.432611803, 2024-09-16T20:51:36.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.05746946, 2024-09-16T20:51:36.538 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:36.538 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:36.539 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:36.539 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:36.539 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:36.539 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:36.539 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8800 2024-09-16T20:51:36.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.539 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:36.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.539 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:36.539 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:51:36.539 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:36.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:51:36.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:36.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:51:36.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5199999999999998e-07 2024-09-16T20:51:36.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:36.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057373+0000", 2024-09-16T20:51:36.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.79e-06 2024-09-16T20:51:36.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.541 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:36.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057379+0000", 2024-09-16T20:51:36.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6699999999999999e-06 2024-09-16T20:51:36.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.541 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:36.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057386+0000", 2024-09-16T20:51:36.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3170000000000004e-06 2024-09-16T20:51:36.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.541 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:36.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114655+0000", 2024-09-16T20:51:36.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057269420000000001 2024-09-16T20:51:36.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:36.542 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114734+0000", 2024-09-16T20:51:36.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8084000000000007e-05 2024-09-16T20:51:36.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:36.542 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114840+0000", 2024-09-16T20:51:36.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000106827 2024-09-16T20:51:36.542 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.542 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:36.542 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.543 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.543 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8801 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2334720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:36.543 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:51:36.543 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 95.432590771999998, 2024-09-16T20:51:36.543 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057594682000000001, 2024-09-16T20:51:36.543 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:36.543 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:36.543 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:36.543 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:36.543 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:36.543 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:36.543 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8801 2024-09-16T20:51:36.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.543 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:36.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.544 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:36.544 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:51:36.544 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.544 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:36.544 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:51:36.544 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.544 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:36.544 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:51:36.544 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6899999999999998e-07 2024-09-16T20:51:36.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.545 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:36.545 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057394+0000", 2024-09-16T20:51:36.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8300000000000001e-06 2024-09-16T20:51:36.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.545 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:36.545 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057400+0000", 2024-09-16T20:51:36.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6500000000000001e-06 2024-09-16T20:51:36.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.545 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:36.545 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057404+0000", 2024-09-16T20:51:36.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4669999999999998e-06 2024-09-16T20:51:36.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:36.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114874+0000", 2024-09-16T20:51:36.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057469602000000002 2024-09-16T20:51:36.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:36.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114933+0000", 2024-09-16T20:51:36.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9500000000000003e-05 2024-09-16T20:51:36.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:36.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114987+0000", 2024-09-16T20:51:36.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3263999999999997e-05 2024-09-16T20:51:36.546 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.547 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:36.547 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.547 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8802 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2353152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:36.547 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:51:36.547 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 95.432542087000002, 2024-09-16T20:51:36.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057714079000000001, 2024-09-16T20:51:36.547 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:36.547 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:36.547 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:36.547 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:36.547 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:36.547 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:36.547 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8802 2024-09-16T20:51:36.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.548 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:36.548 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.548 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:36.548 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:51:36.548 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.548 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.548 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:36.548 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:51:36.548 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.548 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.548 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:36.548 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:51:36.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4999999999999998e-07 2024-09-16T20:51:36.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.549 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:36.549 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057446+0000", 2024-09-16T20:51:36.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4240000000000003e-06 2024-09-16T20:51:36.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.549 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:36.549 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057452+0000", 2024-09-16T20:51:36.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8379999999999996e-06 2024-09-16T20:51:36.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.549 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:36.550 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057460+0000", 2024-09-16T20:51:36.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9629999999999998e-06 2024-09-16T20:51:36.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.550 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:36.550 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115015+0000", 2024-09-16T20:51:36.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057554389999999997 2024-09-16T20:51:36.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.550 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:36.550 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115082+0000", 2024-09-16T20:51:36.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7733999999999999e-05 2024-09-16T20:51:36.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.550 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:36.551 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115155+0000", 2024-09-16T20:51:36.551 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2280000000000006e-05 2024-09-16T20:51:36.551 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.551 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:36.551 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.551 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8804 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2555904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:36.551 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:51:36.551 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 95.432515770999998, 2024-09-16T20:51:36.551 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057833862, 2024-09-16T20:51:36.551 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:36.551 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:36.552 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:36.552 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:36.552 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:36.552 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:36.552 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8804 2024-09-16T20:51:36.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.552 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:36.552 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.552 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:36.552 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:51:36.552 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.552 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.552 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:36.553 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:51:36.553 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.553 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:36.553 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:51:36.553 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7899999999999999e-07 2024-09-16T20:51:36.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.553 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:36.553 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057469+0000", 2024-09-16T20:51:36.553 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.001e-06 2024-09-16T20:51:36.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.554 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.554 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:36.554 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057475+0000", 2024-09-16T20:51:36.554 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7629999999999997e-06 2024-09-16T20:51:36.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.554 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.554 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:36.554 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057480+0000", 2024-09-16T20:51:36.554 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8069999999999999e-06 2024-09-16T20:51:36.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.554 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.554 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:36.554 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115183+0000", 2024-09-16T20:51:36.554 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057703364 2024-09-16T20:51:36.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:36.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115227+0000", 2024-09-16T20:51:36.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3903999999999998e-05 2024-09-16T20:51:36.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:36.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115301+0000", 2024-09-16T20:51:36.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3543999999999999e-05 2024-09-16T20:51:36.555 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.555 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:36.555 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.556 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.556 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.556 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8805 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2574336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:36.556 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057487+0000", 2024-09-16T20:51:36.556 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 95.432495442999993, 2024-09-16T20:51:36.556 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058021027000000003, 2024-09-16T20:51:36.556 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:36.556 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:36.556 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:36.556 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:36.556 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:36.557 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:36.557 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8805 2024-09-16T20:51:36.557 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.557 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:36.557 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.557 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:36.557 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057487+0000", 2024-09-16T20:51:36.557 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.557 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.557 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.557 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:36.557 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057487+0000", 2024-09-16T20:51:36.557 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.558 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.558 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.558 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:36.558 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057488+0000", 2024-09-16T20:51:36.558 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5199999999999998e-07 2024-09-16T20:51:36.558 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.558 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.558 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:36.558 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057490+0000", 2024-09-16T20:51:36.558 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.939e-06 2024-09-16T20:51:36.558 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.558 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.558 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:36.558 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057495+0000", 2024-09-16T20:51:36.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6439999999999997e-06 2024-09-16T20:51:36.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:36.559 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057500+0000", 2024-09-16T20:51:36.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8119999999999998e-06 2024-09-16T20:51:36.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:36.559 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115341+0000", 2024-09-16T20:51:36.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057841341999999997 2024-09-16T20:51:36.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:36.559 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115422+0000", 2024-09-16T20:51:36.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1003999999999999e-05 2024-09-16T20:51:36.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.560 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:36.560 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115508+0000", 2024-09-16T20:51:36.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5933999999999994e-05 2024-09-16T20:51:36.560 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.560 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:36.560 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.560 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8807 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2592768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:36.560 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057505+0000", 2024-09-16T20:51:36.560 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 95.432477233, 2024-09-16T20:51:36.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058186633000000001, 2024-09-16T20:51:36.561 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:36.561 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:36.561 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:36.561 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:36.561 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:36.561 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:36.561 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8807 2024-09-16T20:51:36.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.561 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:36.561 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.561 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:36.561 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057505+0000", 2024-09-16T20:51:36.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.561 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.561 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:36.562 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057505+0000", 2024-09-16T20:51:36.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.562 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:36.562 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057506+0000", 2024-09-16T20:51:36.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.27e-07 2024-09-16T20:51:36.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.562 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:36.562 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057508+0000", 2024-09-16T20:51:36.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7400000000000001e-06 2024-09-16T20:51:36.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.562 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:36.563 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057513+0000", 2024-09-16T20:51:36.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.468e-06 2024-09-16T20:51:36.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.563 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:36.563 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057517+0000", 2024-09-16T20:51:36.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4329999999999997e-06 2024-09-16T20:51:36.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.563 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:36.563 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115538+0000", 2024-09-16T20:51:36.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058020355000000003 2024-09-16T20:51:36.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.563 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:36.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115616+0000", 2024-09-16T20:51:36.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7998999999999996e-05 2024-09-16T20:51:36.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:36.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115692+0000", 2024-09-16T20:51:36.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6310999999999995e-05 2024-09-16T20:51:36.564 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.564 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:36.564 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.564 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8808 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2611200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:36.564 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:51:36.564 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 95.432459972000004, 2024-09-16T20:51:36.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058354046, 2024-09-16T20:51:36.565 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:36.565 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:36.565 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:36.565 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:36.565 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:36.565 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:36.565 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8808 2024-09-16T20:51:36.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.565 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:36.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.565 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:36.565 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:51:36.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.566 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:36.566 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:51:36.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.566 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.566 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:36.566 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:51:36.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2500000000000001e-07 2024-09-16T20:51:36.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.566 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.566 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:36.566 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057525+0000", 2024-09-16T20:51:36.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5549999999999999e-06 2024-09-16T20:51:36.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.567 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:36.567 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057530+0000", 2024-09-16T20:51:36.567 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5489999999999997e-06 2024-09-16T20:51:36.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.567 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:36.567 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057535+0000", 2024-09-16T20:51:36.567 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5709999999999999e-06 2024-09-16T20:51:36.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.567 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:36.568 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115722+0000", 2024-09-16T20:51:36.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058187323999999999 2024-09-16T20:51:36.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.568 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:36.568 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115799+0000", 2024-09-16T20:51:36.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6445000000000004e-05 2024-09-16T20:51:36.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.568 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:36.568 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115877+0000", 2024-09-16T20:51:36.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8276999999999996e-05 2024-09-16T20:51:36.568 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.569 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:36.569 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.569 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:2182 3.17 3:edb5e5cd:::rbd_data.110e61479aee.0000000000000014:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write 0~5120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-16T20:51:36.569 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:22.734609+0000", 2024-09-16T20:51:36.569 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 73.755373724999998, 2024-09-16T20:51:36.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.124890894, 2024-09-16T20:51:36.569 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:36.569 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:36.569 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:36.569 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:36.570 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:51:36.570 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:36.570 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 2182 2024-09-16T20:51:36.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.570 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:36.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.570 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:36.570 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734609+0000", 2024-09-16T20:51:36.570 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.570 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:36.570 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734609+0000", 2024-09-16T20:51:36.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.571 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.571 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:36.571 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734611+0000", 2024-09-16T20:51:36.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9599999999999999e-06 2024-09-16T20:51:36.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.571 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.571 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:36.571 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734628+0000", 2024-09-16T20:51:36.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.715e-05 2024-09-16T20:51:36.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.571 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.572 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:36.572 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734656+0000", 2024-09-16T20:51:36.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7759999999999998e-05 2024-09-16T20:51:36.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.572 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:36.572 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734669+0000", 2024-09-16T20:51:36.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3315999999999999e-05 2024-09-16T20:51:36.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.572 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:36.572 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734826+0000", 2024-09-16T20:51:36.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015668499999999999 2024-09-16T20:51:36.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.573 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.573 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:36.573 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.735048+0000", 2024-09-16T20:51:36.573 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00022179 2024-09-16T20:51:36.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.573 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.573 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:36.573 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.735248+0000", 2024-09-16T20:51:36.573 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00020010399999999999 2024-09-16T20:51:36.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.573 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.573 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:51:36.574 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.736252+0000", 2024-09-16T20:51:36.574 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010039739999999999 2024-09-16T20:51:36.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.574 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:36.574 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.859340+0000", 2024-09-16T20:51:36.574 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.12308846599999999 2024-09-16T20:51:36.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.574 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:51:36.574 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.859431+0000", 2024-09-16T20:51:36.574 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1099000000000003e-05 2024-09-16T20:51:36.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.575 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:36.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.859500+0000", 2024-09-16T20:51:36.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8590000000000006e-05 2024-09-16T20:51:36.575 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.575 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:36.575 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.575 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:29157 3.1b 3:dbe69130:::journal.110e1b19a241:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e167)", 2024-09-16T20:51:36.575 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:54.663540+0000", 2024-09-16T20:51:36.575 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 41.826442387, 2024-09-16T20:51:36.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.92912227999999997, 2024-09-16T20:51:36.575 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:36.576 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:36.576 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:36.576 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:36.576 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:36.576 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:36.576 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 29157 2024-09-16T20:51:36.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.576 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:36.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:36.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663540+0000", 2024-09-16T20:51:36.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.577 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:36.577 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663540+0000", 2024-09-16T20:51:36.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.577 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:36.577 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663541+0000", 2024-09-16T20:51:36.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1680000000000001e-06 2024-09-16T20:51:36.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.577 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:36.577 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663546+0000", 2024-09-16T20:51:36.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7129999999999996e-06 2024-09-16T20:51:36.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.578 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:36.578 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663558+0000", 2024-09-16T20:51:36.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1347e-05 2024-09-16T20:51:36.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.578 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:36.578 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663563+0000", 2024-09-16T20:51:36.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1250000000000001e-06 2024-09-16T20:51:36.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.578 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:36.578 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663612+0000", 2024-09-16T20:51:36.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9665e-05 2024-09-16T20:51:36.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:36.579 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663620+0000", 2024-09-16T20:51:36.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8439999999999998e-06 2024-09-16T20:51:36.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:36.579 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:55.592525+0000", 2024-09-16T20:51:36.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.92890456700000001 2024-09-16T20:51:36.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:36.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:55.592663+0000", 2024-09-16T20:51:36.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013785100000000001 2024-09-16T20:51:36.580 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.580 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:36.580 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.580 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:30229 3.16 3:6f5476df:::rbd_header.110e25db1f60:head [watch reconnect cookie 140213637350016 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e181)", 2024-09-16T20:51:36.580 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:51:08.603124+0000", 2024-09-16T20:51:36.580 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.886858742000001, 2024-09-16T20:51:36.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.97765519899999997, 2024-09-16T20:51:36.580 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:36.580 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:36.580 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:36.581 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:36.581 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:36.581 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:36.581 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30229 2024-09-16T20:51:36.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.581 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:36.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:36.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603124+0000", 2024-09-16T20:51:36.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:36.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603124+0000", 2024-09-16T20:51:36.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:36.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603125+0000", 2024-09-16T20:51:36.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5090000000000001e-06 2024-09-16T20:51:36.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:36.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603131+0000", 2024-09-16T20:51:36.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9089999999999997e-06 2024-09-16T20:51:36.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:36.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603143+0000", 2024-09-16T20:51:36.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1416e-05 2024-09-16T20:51:36.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:36.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603151+0000", 2024-09-16T20:51:36.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7889999999999995e-06 2024-09-16T20:51:36.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:36.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603674+0000", 2024-09-16T20:51:36.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00052357 2024-09-16T20:51:36.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:36.584 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603684+0000", 2024-09-16T20:51:36.584 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0274e-05 2024-09-16T20:51:36.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.584 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.584 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:36.584 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.575380+0000", 2024-09-16T20:51:36.584 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.97169509799999998 2024-09-16T20:51:36.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.584 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.584 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:36.584 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.575591+0000", 2024-09-16T20:51:36.584 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00021126700000000001 2024-09-16T20:51:36.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.584 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.584 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:36.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.580779+0000", 2024-09-16T20:51:36.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0051883670000000002 2024-09-16T20:51:36.585 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.585 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:36.585 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.585 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:30230 3.16 3:6f5476df:::rbd_header.110e25db1f60:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e181)", 2024-09-16T20:51:36.585 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:51:08.603204+0000", 2024-09-16T20:51:36.585 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.886778794000001, 2024-09-16T20:51:36.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.98129583300000001, 2024-09-16T20:51:36.585 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:36.585 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:36.585 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:36.585 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:36.586 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:36.586 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:36.586 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30230 2024-09-16T20:51:36.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.586 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:36.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:36.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603204+0000", 2024-09-16T20:51:36.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:36.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603204+0000", 2024-09-16T20:51:36.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:36.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:36.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603205+0000", 2024-09-16T20:51:36.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4900000000000005e-07 2024-09-16T20:51:36.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:36.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603209+0000", 2024-09-16T20:51:36.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5210000000000004e-06 2024-09-16T20:51:36.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:36.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603222+0000", 2024-09-16T20:51:36.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2733e-05 2024-09-16T20:51:36.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:36.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603228+0000", 2024-09-16T20:51:36.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4520000000000002e-06 2024-09-16T20:51:36.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:36.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603719+0000", 2024-09-16T20:51:36.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00049103600000000003 2024-09-16T20:51:36.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:36.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603730+0000", 2024-09-16T20:51:36.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.023e-05 2024-09-16T20:51:36.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:36.589 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.577394+0000", 2024-09-16T20:51:36.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.97366440099999996 2024-09-16T20:51:36.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:36.589 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.577499+0000", 2024-09-16T20:51:36.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000104509 2024-09-16T20:51:36.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:36.589 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.577920+0000", 2024-09-16T20:51:36.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00042173699999999999 2024-09-16T20:51:36.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.590 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:51:36.590 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.580795+0000", 2024-09-16T20:51:36.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.002874302 2024-09-16T20:51:36.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.590 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:36.590 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.584423+0000", 2024-09-16T20:51:36.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0036280010000000001 2024-09-16T20:51:36.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.590 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:51:36.590 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.584465+0000", 2024-09-16T20:51:36.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2590999999999999e-05 2024-09-16T20:51:36.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:36.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:36.591 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:36.591 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.584500+0000", 2024-09-16T20:51:36.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4570999999999998e-05 2024-09-16T20:51:36.591 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.591 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:36.591 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.591 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:36.591 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:36.591 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:36.593 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:51:36.593 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:36.592+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:36.599 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T20:51:36.694 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:36.693+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:37.062 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:37.062 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:51:37.062 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:51:37.062 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:37.063 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:51:37.063 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:37.061+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:37.163 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:37.162+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:37.210 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:37.210 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:51:37.210 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:51:37.210 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:51:37.210 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:37.218 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:51:37.264 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:37.263+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:37.309 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T20:51:37.327 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:37.327 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:51:37.327 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:51:37.327 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:51:37.327 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.327 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5439 3.a 3:5789f5c3:::rbd_data.110ed9ca540.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 1181696~60416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:37.327 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:51:37.327 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 194.460539098, 2024-09-16T20:51:37.327 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10102045599999999, 2024-09-16T20:51:37.327 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:37.327 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:37.327 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:37.328 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:37.328 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:37.328 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:37.328 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5439 2024-09-16T20:51:37.328 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.328 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:37.328 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.328 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:37.328 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:51:37.328 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:37.328 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.328 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.328 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:37.328 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:51:37.329 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:37.329 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.329 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.329 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:37.329 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864711+0000", 2024-09-16T20:51:37.329 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7899999999999997e-07 2024-09-16T20:51:37.329 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.329 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.329 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:37.329 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864760+0000", 2024-09-16T20:51:37.329 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8803e-05 2024-09-16T20:51:37.329 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.329 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.329 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:37.329 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864769+0000", 2024-09-16T20:51:37.330 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0999999999999993e-06 2024-09-16T20:51:37.330 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.330 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.330 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:37.330 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864774+0000", 2024-09-16T20:51:37.330 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8919999999999999e-06 2024-09-16T20:51:37.330 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.330 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.330 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:37.330 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864815+0000", 2024-09-16T20:51:37.330 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1307000000000003e-05 2024-09-16T20:51:37.330 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.330 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.330 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:37.331 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864893+0000", 2024-09-16T20:51:37.331 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7581000000000001e-05 2024-09-16T20:51:37.331 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.331 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.331 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 1", 2024-09-16T20:51:37.331 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864996+0000", 2024-09-16T20:51:37.331 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102725 2024-09-16T20:51:37.331 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.331 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.331 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:37.331 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965442+0000", 2024-09-16T20:51:37.331 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100446012 2024-09-16T20:51:37.332 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.332 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.332 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:37.332 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965618+0000", 2024-09-16T20:51:37.332 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00017653100000000001 2024-09-16T20:51:37.332 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.332 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.332 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:37.332 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965682+0000", 2024-09-16T20:51:37.332 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3905999999999994e-05 2024-09-16T20:51:37.332 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.332 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.333 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:37.333 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965731+0000", 2024-09-16T20:51:37.333 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8819999999999997e-05 2024-09-16T20:51:37.333 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:37.333 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:37.333 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:37.333 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.333 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.333 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4600.0:5439 3.a e37/23 ondisk, result = 0)", 2024-09-16T20:51:37.333 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:51:37.333 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 194.45954295000001, 2024-09-16T20:51:37.333 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099742634999999996, 2024-09-16T20:51:37.333 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:37.334 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:37.334 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:37.334 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:37.334 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.334 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:37.334 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:51:37.334 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:37.334 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.334 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.334 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:37.334 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:51:37.334 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:37.334 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.334 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.334 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:37.335 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:51:37.335 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0900000000000001e-07 2024-09-16T20:51:37.335 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.335 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.335 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:37.335 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865711+0000", 2024-09-16T20:51:37.335 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3739999999999998e-06 2024-09-16T20:51:37.335 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.335 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.335 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:37.335 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965343+0000", 2024-09-16T20:51:37.335 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099632526999999999 2024-09-16T20:51:37.335 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.335 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.336 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:37.336 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965357+0000", 2024-09-16T20:51:37.336 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3713999999999999e-05 2024-09-16T20:51:37.336 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.336 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.336 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:37.336 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965426+0000", 2024-09-16T20:51:37.336 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8745999999999995e-05 2024-09-16T20:51:37.336 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.336 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.336 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:37.336 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965434+0000", 2024-09-16T20:51:37.336 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6910000000000003e-06 2024-09-16T20:51:37.336 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.337 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.337 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:37.337 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965449+0000", 2024-09-16T20:51:37.337 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4874e-05 2024-09-16T20:51:37.337 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:37.337 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:37.337 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:37.337 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.337 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.337 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4882.0:2508 3.12 3:4992771f:::rbd_object_map.110e7c8851c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T20:51:37.337 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:51:37.337 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 170.84373940200001, 2024-09-16T20:51:37.337 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.125007011, 2024-09-16T20:51:37.337 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:37.338 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:37.338 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:37.338 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:37.338 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4882", 2024-09-16T20:51:37.338 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2436923223", 2024-09-16T20:51:37.338 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2508 2024-09-16T20:51:37.338 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.338 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:37.338 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.338 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:37.338 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:51:37.338 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:37.338 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.338 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.338 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:37.339 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:51:37.339 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:37.339 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.339 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.339 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:37.339 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481511+0000", 2024-09-16T20:51:37.339 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0699999999999996e-07 2024-09-16T20:51:37.339 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.339 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.339 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:37.339 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481515+0000", 2024-09-16T20:51:37.339 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0609999999999997e-06 2024-09-16T20:51:37.339 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.339 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.339 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:37.340 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481523+0000", 2024-09-16T20:51:37.340 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1310000000000006e-06 2024-09-16T20:51:37.340 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.340 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.340 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:37.340 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481528+0000", 2024-09-16T20:51:37.340 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4900000000000002e-06 2024-09-16T20:51:37.340 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.340 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.340 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:37.340 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481565+0000", 2024-09-16T20:51:37.340 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7066000000000001e-05 2024-09-16T20:51:37.340 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.340 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.341 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:37.341 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481628+0000", 2024-09-16T20:51:37.341 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3118000000000001e-05 2024-09-16T20:51:37.341 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.341 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.341 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 2", 2024-09-16T20:51:37.341 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481822+0000", 2024-09-16T20:51:37.341 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019446200000000001 2024-09-16T20:51:37.341 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.341 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.341 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:37.341 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605988+0000", 2024-09-16T20:51:37.341 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.124165888 2024-09-16T20:51:37.341 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.341 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.342 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:37.342 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606427+0000", 2024-09-16T20:51:37.342 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00043854899999999999 2024-09-16T20:51:37.342 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.342 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.342 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:37.342 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606476+0000", 2024-09-16T20:51:37.342 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8971999999999998e-05 2024-09-16T20:51:37.342 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.342 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.342 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:37.342 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606517+0000", 2024-09-16T20:51:37.342 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1467000000000001e-05 2024-09-16T20:51:37.343 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:37.343 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:37.343 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:37.343 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.343 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.343 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4882.0:2508 3.12 e57/55 ondisk, result = 0)", 2024-09-16T20:51:37.343 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:51:37.343 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 170.84275090200001, 2024-09-16T20:51:37.343 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12349711400000001, 2024-09-16T20:51:37.343 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:37.343 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:37.343 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:37.343 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:37.343 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.344 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:37.344 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:51:37.344 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:37.344 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.344 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.344 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:37.344 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:51:37.344 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:37.344 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.344 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.344 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:37.344 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:51:37.344 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.06e-07 2024-09-16T20:51:37.344 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.345 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.345 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:37.345 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482502+0000", 2024-09-16T20:51:37.345 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.01e-06 2024-09-16T20:51:37.345 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.345 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.345 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:37.345 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605772+0000", 2024-09-16T20:51:37.345 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.123269687 2024-09-16T20:51:37.345 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.345 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.345 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:37.345 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605791+0000", 2024-09-16T20:51:37.345 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8912000000000001e-05 2024-09-16T20:51:37.345 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.346 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.346 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:37.346 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605974+0000", 2024-09-16T20:51:37.346 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000182836 2024-09-16T20:51:37.346 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.346 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.346 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:37.346 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605982+0000", 2024-09-16T20:51:37.346 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1699999999999997e-06 2024-09-16T20:51:37.346 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.346 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.346 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:37.346 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605996+0000", 2024-09-16T20:51:37.346 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3893000000000001e-05 2024-09-16T20:51:37.346 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:37.347 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:37.347 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:37.347 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.347 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.347 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4882.0:2512 3.f 3:f4d895b7:::rbd_data.110e7c8851c.0000000000000000:head [read 16384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T20:51:37.347 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:51:37.347 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 170.84231387200001, 2024-09-16T20:51:37.347 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12318483, 2024-09-16T20:51:37.347 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:37.347 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:37.347 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:37.347 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:37.347 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4882", 2024-09-16T20:51:37.347 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2436923223", 2024-09-16T20:51:37.347 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2512 2024-09-16T20:51:37.348 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.348 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:37.348 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.348 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:37.348 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:51:37.348 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:37.348 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.348 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.348 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:37.348 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:51:37.348 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:37.348 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.348 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.348 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:37.348 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:51:37.349 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3499999999999995e-07 2024-09-16T20:51:37.349 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.349 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.349 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:37.349 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482943+0000", 2024-09-16T20:51:37.349 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3450000000000002e-06 2024-09-16T20:51:37.349 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.349 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.349 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:37.349 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605772+0000", 2024-09-16T20:51:37.349 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.122829796 2024-09-16T20:51:37.349 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.349 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.349 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:37.350 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605792+0000", 2024-09-16T20:51:37.350 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9916999999999999e-05 2024-09-16T20:51:37.350 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.350 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.350 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:37.350 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605917+0000", 2024-09-16T20:51:37.350 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00012500799999999999 2024-09-16T20:51:37.350 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.350 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.350 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:37.350 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606004+0000", 2024-09-16T20:51:37.350 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6242999999999994e-05 2024-09-16T20:51:37.350 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.350 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.350 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:37.351 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606120+0000", 2024-09-16T20:51:37.351 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000116786 2024-09-16T20:51:37.351 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:37.351 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:37.351 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:37.351 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.351 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.351 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:11702 3.15 3:acf33c8b:::rbd_data.110e747f0779.000000000000001a:head [set-alloc-hint object_size 262144 write_size 262144,write 143360~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:51:37.351 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:51:37.351 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 153.18732145000001, 2024-09-16T20:51:37.351 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.084801043000000006, 2024-09-16T20:51:37.351 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:37.351 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:37.351 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:37.352 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:37.352 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:51:37.352 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:51:37.352 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 11702 2024-09-16T20:51:37.352 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.352 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:37.352 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.352 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:37.352 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:51:37.352 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:37.352 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.352 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.352 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:37.352 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:51:37.352 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:37.353 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.353 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.353 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:37.353 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137929+0000", 2024-09-16T20:51:37.353 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8400000000000002e-07 2024-09-16T20:51:37.353 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.353 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.353 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:37.353 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137937+0000", 2024-09-16T20:51:37.353 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0260000000000007e-06 2024-09-16T20:51:37.353 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.353 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.353 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:37.353 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137952+0000", 2024-09-16T20:51:37.354 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4659e-05 2024-09-16T20:51:37.354 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.354 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.354 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:37.354 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137960+0000", 2024-09-16T20:51:37.354 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0169999999999996e-06 2024-09-16T20:51:37.354 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.354 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.354 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:37.354 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138010+0000", 2024-09-16T20:51:37.354 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0021000000000001e-05 2024-09-16T20:51:37.354 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.354 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.354 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:37.355 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138114+0000", 2024-09-16T20:51:37.355 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000103797 2024-09-16T20:51:37.355 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.355 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.355 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:51:37.355 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138233+0000", 2024-09-16T20:51:37.355 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000118971 2024-09-16T20:51:37.355 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.355 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.355 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:37.355 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138953+0000", 2024-09-16T20:51:37.355 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00072026000000000002 2024-09-16T20:51:37.355 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.355 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.355 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:37.356 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222609+0000", 2024-09-16T20:51:37.356 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.083656650999999999 2024-09-16T20:51:37.356 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.356 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.356 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:37.356 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222684+0000", 2024-09-16T20:51:37.356 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4369999999999994e-05 2024-09-16T20:51:37.356 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.356 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.356 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:37.356 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222729+0000", 2024-09-16T20:51:37.356 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5287000000000002e-05 2024-09-16T20:51:37.356 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:37.356 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:37.357 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:37.357 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.357 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.357 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:11704 3.9 3:91220b75:::rbd_data.110e747f0779.000000000000001c:head [set-alloc-hint object_size 262144 write_size 262144,write 139264~6656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:51:37.357 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:51:37.357 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 153.187083449, 2024-09-16T20:51:37.357 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.084531133999999994, 2024-09-16T20:51:37.357 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:37.357 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:37.357 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:37.357 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:37.357 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:51:37.357 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:51:37.357 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 11704 2024-09-16T20:51:37.358 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.358 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:37.358 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.358 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:37.358 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:51:37.358 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:37.358 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.358 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.358 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:37.358 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:51:37.358 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:37.358 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.358 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.358 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:37.358 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138167+0000", 2024-09-16T20:51:37.359 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.33e-07 2024-09-16T20:51:37.359 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.359 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.359 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:37.359 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138179+0000", 2024-09-16T20:51:37.359 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2055999999999999e-05 2024-09-16T20:51:37.359 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.359 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.359 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:37.359 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138193+0000", 2024-09-16T20:51:37.359 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3961000000000001e-05 2024-09-16T20:51:37.359 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.359 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.359 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:37.360 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138200+0000", 2024-09-16T20:51:37.360 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4429999999999997e-06 2024-09-16T20:51:37.360 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.360 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.360 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:37.360 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138258+0000", 2024-09-16T20:51:37.360 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7225999999999997e-05 2024-09-16T20:51:37.360 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.360 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.360 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:37.360 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138364+0000", 2024-09-16T20:51:37.360 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000106013 2024-09-16T20:51:37.360 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.360 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.360 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:51:37.361 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138494+0000", 2024-09-16T20:51:37.361 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013059599999999999 2024-09-16T20:51:37.361 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.361 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.361 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:37.361 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.139220+0000", 2024-09-16T20:51:37.361 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00072539800000000004 2024-09-16T20:51:37.361 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.361 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.361 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:37.361 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222608+0000", 2024-09-16T20:51:37.361 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.083388620999999996 2024-09-16T20:51:37.361 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.361 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.361 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:37.362 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222651+0000", 2024-09-16T20:51:37.362 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2228000000000003e-05 2024-09-16T20:51:37.362 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.362 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.362 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:37.362 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222697+0000", 2024-09-16T20:51:37.362 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6659000000000002e-05 2024-09-16T20:51:37.362 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:37.362 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:37.362 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:37.362 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.362 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.362 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5138.0:9358 3.c 3.c25699cc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T20:51:37.362 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:51:37.362 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 130.474936144, 2024-09-16T20:51:37.363 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0018657580000001, 2024-09-16T20:51:37.363 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:37.363 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:37.363 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:51:37.363 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:37.363 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5138", 2024-09-16T20:51:37.363 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:51:37.363 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 9358 2024-09-16T20:51:37.363 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.363 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:37.363 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.363 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:37.363 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:51:37.363 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:37.363 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.364 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.364 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:37.364 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:51:37.364 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:37.364 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.364 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.364 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:37.364 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850314+0000", 2024-09-16T20:51:37.364 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4099999999999999e-07 2024-09-16T20:51:37.364 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.364 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.364 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:37.364 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850316+0000", 2024-09-16T20:51:37.364 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2469999999999998e-06 2024-09-16T20:51:37.364 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.365 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.365 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:37.365 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850322+0000", 2024-09-16T20:51:37.365 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3470000000000004e-06 2024-09-16T20:51:37.365 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.365 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.365 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:37.365 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850327+0000", 2024-09-16T20:51:37.365 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9450000000000001e-06 2024-09-16T20:51:37.365 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.365 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.365 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:37.365 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850376+0000", 2024-09-16T20:51:37.365 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8257000000000002e-05 2024-09-16T20:51:37.365 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.366 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.366 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:37.366 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850385+0000", 2024-09-16T20:51:37.366 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9800000000000004e-06 2024-09-16T20:51:37.366 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.366 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.366 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:37.366 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852167+0000", 2024-09-16T20:51:37.366 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0017825659999999 2024-09-16T20:51:37.366 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.366 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.366 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:37.366 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852179+0000", 2024-09-16T20:51:37.366 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1975e-05 2024-09-16T20:51:37.367 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:37.367 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:37.367 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:37.367 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.367 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.367 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5138.0:9382 3.c 3.c25699cc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T20:51:37.367 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:51:37.367 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 130.46658912800001, 2024-09-16T20:51:37.367 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.99358404700000003, 2024-09-16T20:51:37.367 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:37.368 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:37.368 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:51:37.368 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:37.368 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5138", 2024-09-16T20:51:37.368 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:51:37.368 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 9382 2024-09-16T20:51:37.368 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.368 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:37.368 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.368 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:37.368 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:51:37.368 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:37.369 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.369 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.369 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:37.369 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:51:37.369 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:37.369 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.369 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.369 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:37.369 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858661+0000", 2024-09-16T20:51:37.369 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0200000000000002e-07 2024-09-16T20:51:37.369 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.369 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.369 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:37.370 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858664+0000", 2024-09-16T20:51:37.370 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.038e-06 2024-09-16T20:51:37.370 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.370 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.370 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:37.370 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858671+0000", 2024-09-16T20:51:37.370 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0110000000000003e-06 2024-09-16T20:51:37.370 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.370 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.370 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:37.370 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858677+0000", 2024-09-16T20:51:37.370 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0959999999999997e-06 2024-09-16T20:51:37.370 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.370 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.370 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:37.371 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858732+0000", 2024-09-16T20:51:37.371 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4543000000000003e-05 2024-09-16T20:51:37.371 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.371 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.371 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:37.371 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858739+0000", 2024-09-16T20:51:37.371 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5429999999999996e-06 2024-09-16T20:51:37.371 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.371 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.371 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:37.371 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852236+0000", 2024-09-16T20:51:37.371 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.993496453 2024-09-16T20:51:37.371 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.371 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.371 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:37.372 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852244+0000", 2024-09-16T20:51:37.372 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8610000000000004e-06 2024-09-16T20:51:37.372 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:37.372 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:37.372 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:37.372 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.372 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.372 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5763.0:4895 3.a 3:5384606c:::rbd_data.110e1b19a241.0000000000000018:head [read 0~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-16T20:51:37.372 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:45.598212+0000", 2024-09-16T20:51:37.372 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 111.727037087, 2024-09-16T20:51:37.372 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0375864829999999, 2024-09-16T20:51:37.372 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:37.372 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:37.372 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:37.372 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:37.373 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5763", 2024-09-16T20:51:37.373 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/623897567", 2024-09-16T20:51:37.373 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4895 2024-09-16T20:51:37.373 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.373 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:37.373 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.373 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:37.373 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598212+0000", 2024-09-16T20:51:37.373 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:37.373 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.373 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.373 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:37.373 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598212+0000", 2024-09-16T20:51:37.374 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:37.374 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.374 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.374 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:37.374 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598214+0000", 2024-09-16T20:51:37.374 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0090000000000002e-06 2024-09-16T20:51:37.374 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.374 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.374 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:37.374 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598222+0000", 2024-09-16T20:51:37.375 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6699999999999994e-06 2024-09-16T20:51:37.375 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.375 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.375 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:37.375 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598241+0000", 2024-09-16T20:51:37.375 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8896999999999999e-05 2024-09-16T20:51:37.375 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.375 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.375 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:37.375 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598250+0000", 2024-09-16T20:51:37.375 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6649999999999992e-06 2024-09-16T20:51:37.375 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.375 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.375 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:37.375 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598325+0000", 2024-09-16T20:51:37.376 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5189000000000001e-05 2024-09-16T20:51:37.376 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.376 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.376 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op must wait for map", 2024-09-16T20:51:37.376 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598333+0000", 2024-09-16T20:51:37.376 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.3170000000000002e-06 2024-09-16T20:51:37.376 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.376 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.376 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:37.376 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601355+0000", 2024-09-16T20:51:37.376 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0030215400000000001 2024-09-16T20:51:37.376 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.376 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.376 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:37.376 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601365+0000", 2024-09-16T20:51:37.377 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0215e-05 2024-09-16T20:51:37.377 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.377 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.377 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:37.377 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635506+0000", 2024-09-16T20:51:37.377 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0341408240000001 2024-09-16T20:51:37.377 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.377 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.377 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:37.377 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635655+0000", 2024-09-16T20:51:37.377 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000149412 2024-09-16T20:51:37.377 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.377 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.378 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:37.378 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635799+0000", 2024-09-16T20:51:37.378 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000143745 2024-09-16T20:51:37.378 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:37.378 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:37.378 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:37.378 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.378 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.378 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5763.0:4900 3.a 3:5384606c:::rbd_data.110e1b19a241.0000000000000018:head [read 8192~24576] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-16T20:51:37.378 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:51:37.378 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 111.72697861100001, 2024-09-16T20:51:37.378 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0379340800000001, 2024-09-16T20:51:37.378 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:37.378 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:37.379 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:37.379 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:37.379 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5763", 2024-09-16T20:51:37.379 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/623897567", 2024-09-16T20:51:37.379 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4900 2024-09-16T20:51:37.379 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.379 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:37.379 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.379 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:37.379 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:51:37.379 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:37.379 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.379 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.379 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:37.379 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:51:37.380 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:37.380 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.380 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.380 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:37.380 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:51:37.380 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4300000000000003e-07 2024-09-16T20:51:37.380 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.380 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.380 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:37.380 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598274+0000", 2024-09-16T20:51:37.380 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.976e-06 2024-09-16T20:51:37.380 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.380 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.380 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:37.380 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598282+0000", 2024-09-16T20:51:37.381 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0029999999999994e-06 2024-09-16T20:51:37.381 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.381 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.381 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:37.381 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598289+0000", 2024-09-16T20:51:37.381 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2389999999999999e-06 2024-09-16T20:51:37.381 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.381 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.381 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:37.381 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598430+0000", 2024-09-16T20:51:37.381 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014119600000000001 2024-09-16T20:51:37.381 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.381 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.381 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting_for_map not empty", 2024-09-16T20:51:37.382 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598435+0000", 2024-09-16T20:51:37.382 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3489999999999999e-06 2024-09-16T20:51:37.382 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.382 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.382 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:37.382 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601429+0000", 2024-09-16T20:51:37.382 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0029942720000000001 2024-09-16T20:51:37.382 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.382 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.382 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:37.382 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601438+0000", 2024-09-16T20:51:37.382 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4759999999999998e-06 2024-09-16T20:51:37.382 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.382 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.382 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:37.383 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635987+0000", 2024-09-16T20:51:37.383 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0345495010000001 2024-09-16T20:51:37.383 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.383 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.383 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:37.383 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.636056+0000", 2024-09-16T20:51:37.383 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8610999999999998e-05 2024-09-16T20:51:37.383 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.383 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.383 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:37.383 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.636205+0000", 2024-09-16T20:51:37.383 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014891399999999999 2024-09-16T20:51:37.383 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:37.383 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:37.383 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:37.384 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.384 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.384 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop(client.6231.0:2182 3.17 e136/81 3:edb5e5cd:::rbd_data.110e61479aee.0000000000000014:head v 136'791, mlcod=135'789)", 2024-09-16T20:51:37.384 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:22.735499+0000", 2024-09-16T20:51:37.384 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 74.589750670000001, 2024-09-16T20:51:37.384 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12369608, 2024-09-16T20:51:37.384 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:37.384 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:37.384 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:37.384 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:37.384 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.384 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:37.384 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735499+0000", 2024-09-16T20:51:37.384 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:37.384 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.385 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.385 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:37.385 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735499+0000", 2024-09-16T20:51:37.385 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:37.385 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.385 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.385 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:37.385 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735499+0000", 2024-09-16T20:51:37.385 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8099999999999999e-07 2024-09-16T20:51:37.385 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.385 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.385 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:37.385 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735506+0000", 2024-09-16T20:51:37.385 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3579999999999997e-06 2024-09-16T20:51:37.385 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.386 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.386 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:37.386 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735517+0000", 2024-09-16T20:51:37.386 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0869999999999999e-05 2024-09-16T20:51:37.386 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.386 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.386 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:37.386 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735533+0000", 2024-09-16T20:51:37.386 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5858e-05 2024-09-16T20:51:37.386 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.386 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.386 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:37.386 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735584+0000", 2024-09-16T20:51:37.386 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0470999999999998e-05 2024-09-16T20:51:37.386 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.387 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.387 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:37.387 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735607+0000", 2024-09-16T20:51:37.387 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2875999999999999e-05 2024-09-16T20:51:37.387 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.387 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.387 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:37.387 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.859144+0000", 2024-09-16T20:51:37.387 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.123537701 2024-09-16T20:51:37.387 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.387 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.387 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:37.387 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.859195+0000", 2024-09-16T20:51:37.387 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0664999999999997e-05 2024-09-16T20:51:37.387 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:37.387 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:37.388 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:37.388 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.388 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.388 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5204 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 2523136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:51:37.388 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.883660+0000", 2024-09-16T20:51:37.388 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 73.441589140000005, 2024-09-16T20:51:37.388 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052517638999999998, 2024-09-16T20:51:37.388 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:37.388 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:37.388 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:37.388 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:37.388 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:37.388 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:37.388 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5204 2024-09-16T20:51:37.388 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.389 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:37.389 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.389 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:37.389 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883660+0000", 2024-09-16T20:51:37.389 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:37.389 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.389 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.389 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:37.389 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883660+0000", 2024-09-16T20:51:37.389 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:37.389 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.389 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.389 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:37.389 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883661+0000", 2024-09-16T20:51:37.390 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9999999999999998e-07 2024-09-16T20:51:37.390 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.390 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.390 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:37.390 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883663+0000", 2024-09-16T20:51:37.390 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.4480000000000001e-06 2024-09-16T20:51:37.390 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.390 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.390 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:37.390 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883669+0000", 2024-09-16T20:51:37.390 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.071e-06 2024-09-16T20:51:37.390 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.390 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.390 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:37.390 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883675+0000", 2024-09-16T20:51:37.391 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.366e-06 2024-09-16T20:51:37.391 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.391 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.391 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:37.391 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.935970+0000", 2024-09-16T20:51:37.391 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052295346999999999 2024-09-16T20:51:37.391 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.391 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.391 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:37.391 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936064+0000", 2024-09-16T20:51:37.391 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4438000000000006e-05 2024-09-16T20:51:37.391 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.391 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.392 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:37.392 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936178+0000", 2024-09-16T20:51:37.392 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000113569 2024-09-16T20:51:37.392 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:37.392 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:37.392 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:37.392 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.392 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.392 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5205 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 2543616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:51:37.392 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.883745+0000", 2024-09-16T20:51:37.392 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 73.441504382000005, 2024-09-16T20:51:37.392 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052692112999999999, 2024-09-16T20:51:37.392 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:37.392 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:37.393 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:37.393 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:37.393 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:37.393 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:37.393 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5205 2024-09-16T20:51:37.393 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.393 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:37.393 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.393 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:37.393 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883745+0000", 2024-09-16T20:51:37.393 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:37.393 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.393 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.394 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:37.394 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883745+0000", 2024-09-16T20:51:37.394 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:37.394 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.394 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.394 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:37.394 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883746+0000", 2024-09-16T20:51:37.394 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2699999999999999e-07 2024-09-16T20:51:37.394 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.394 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.394 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:37.394 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883750+0000", 2024-09-16T20:51:37.394 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7050000000000001e-06 2024-09-16T20:51:37.394 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.395 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.395 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:37.395 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883769+0000", 2024-09-16T20:51:37.395 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9165e-05 2024-09-16T20:51:37.395 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.395 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.395 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:37.395 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883778+0000", 2024-09-16T20:51:37.395 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.6690000000000001e-06 2024-09-16T20:51:37.395 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.395 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.395 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:37.395 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936228+0000", 2024-09-16T20:51:37.396 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052449587999999998 2024-09-16T20:51:37.396 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.396 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.396 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:37.396 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936333+0000", 2024-09-16T20:51:37.396 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010511800000000001 2024-09-16T20:51:37.396 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.396 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.396 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:37.396 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936437+0000", 2024-09-16T20:51:37.396 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000104141 2024-09-16T20:51:37.396 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:37.396 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:37.397 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:37.397 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.397 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.397 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5206 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 1167360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:51:37.397 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.883802+0000", 2024-09-16T20:51:37.397 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 73.441447887999999, 2024-09-16T20:51:37.397 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052800780999999998, 2024-09-16T20:51:37.397 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:37.397 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:37.397 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:37.397 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:37.397 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:37.397 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:37.397 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5206 2024-09-16T20:51:37.398 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.398 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:37.398 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.398 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:37.398 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883802+0000", 2024-09-16T20:51:37.398 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:37.398 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.398 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.398 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:37.398 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883802+0000", 2024-09-16T20:51:37.398 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:37.398 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.398 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.399 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:37.399 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883933+0000", 2024-09-16T20:51:37.399 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013182999999999999 2024-09-16T20:51:37.399 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.399 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.399 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:37.399 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883937+0000", 2024-09-16T20:51:37.399 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.771e-06 2024-09-16T20:51:37.399 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.399 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.399 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:37.399 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883953+0000", 2024-09-16T20:51:37.399 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5409000000000001e-05 2024-09-16T20:51:37.400 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.400 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.400 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:37.400 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883959+0000", 2024-09-16T20:51:37.400 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0700000000000003e-06 2024-09-16T20:51:37.400 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.400 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.400 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:37.400 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936476+0000", 2024-09-16T20:51:37.400 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052517434000000002 2024-09-16T20:51:37.400 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.400 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.400 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:37.400 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936529+0000", 2024-09-16T20:51:37.401 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2969000000000001e-05 2024-09-16T20:51:37.401 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.401 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.401 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:37.401 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936602+0000", 2024-09-16T20:51:37.401 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3298000000000002e-05 2024-09-16T20:51:37.401 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:37.401 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:37.401 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:37.401 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.401 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.401 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5207 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 1904640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:51:37.402 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.883980+0000", 2024-09-16T20:51:37.402 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 73.441269679000001, 2024-09-16T20:51:37.402 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052759922000000001, 2024-09-16T20:51:37.402 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:37.402 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:37.402 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:37.402 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:37.402 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:37.402 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:37.402 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5207 2024-09-16T20:51:37.402 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.402 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:37.402 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.402 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:37.402 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883980+0000", 2024-09-16T20:51:37.403 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:37.403 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.403 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.403 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:37.403 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883980+0000", 2024-09-16T20:51:37.403 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:37.403 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.403 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.403 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:37.403 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884126+0000", 2024-09-16T20:51:37.403 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014577399999999999 2024-09-16T20:51:37.403 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.403 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.403 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:37.403 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884130+0000", 2024-09-16T20:51:37.404 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9729999999999999e-06 2024-09-16T20:51:37.404 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.404 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.404 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:37.404 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884139+0000", 2024-09-16T20:51:37.404 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8660000000000001e-06 2024-09-16T20:51:37.404 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.404 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.404 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:37.404 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884144+0000", 2024-09-16T20:51:37.404 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3669999999999999e-06 2024-09-16T20:51:37.404 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.404 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.405 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:37.405 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936635+0000", 2024-09-16T20:51:37.405 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052491705 2024-09-16T20:51:37.405 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.405 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.405 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:37.405 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936679+0000", 2024-09-16T20:51:37.405 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3105999999999997e-05 2024-09-16T20:51:37.405 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.405 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.405 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:37.405 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936740+0000", 2024-09-16T20:51:37.405 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1130999999999995e-05 2024-09-16T20:51:37.405 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:37.406 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:37.406 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:37.406 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.406 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.406 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5208 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 131072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:51:37.406 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.884165+0000", 2024-09-16T20:51:37.406 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 73.441084383000003, 2024-09-16T20:51:37.406 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052746603000000003, 2024-09-16T20:51:37.406 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:37.406 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:37.406 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:37.406 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:37.406 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:37.406 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:37.407 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5208 2024-09-16T20:51:37.407 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.407 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:37.407 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.407 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:37.407 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884165+0000", 2024-09-16T20:51:37.407 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:37.407 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.407 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.407 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:37.407 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884165+0000", 2024-09-16T20:51:37.407 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:37.407 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.407 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.407 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:37.408 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884463+0000", 2024-09-16T20:51:37.408 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00029801699999999998 2024-09-16T20:51:37.408 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.408 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.408 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:37.408 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884467+0000", 2024-09-16T20:51:37.408 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8210000000000003e-06 2024-09-16T20:51:37.408 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.408 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.408 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:37.408 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884477+0000", 2024-09-16T20:51:37.408 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.7030000000000001e-06 2024-09-16T20:51:37.408 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.409 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.409 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:37.409 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884485+0000", 2024-09-16T20:51:37.409 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9710000000000004e-06 2024-09-16T20:51:37.409 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.409 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.409 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:37.409 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936785+0000", 2024-09-16T20:51:37.409 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052300665000000003 2024-09-16T20:51:37.409 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.409 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.409 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:37.409 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936851+0000", 2024-09-16T20:51:37.410 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6159000000000001e-05 2024-09-16T20:51:37.410 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.410 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.410 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:37.410 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936912+0000", 2024-09-16T20:51:37.410 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0266999999999997e-05 2024-09-16T20:51:37.410 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:37.410 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:37.410 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:37.410 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.410 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.410 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5209 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 1925120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:51:37.410 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.884504+0000", 2024-09-16T20:51:37.410 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 73.440745497999998, 2024-09-16T20:51:37.410 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052575445999999998, 2024-09-16T20:51:37.411 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:37.411 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:37.411 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:37.411 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:37.411 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:37.411 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:37.411 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5209 2024-09-16T20:51:37.411 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.411 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:37.411 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.411 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:37.411 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884504+0000", 2024-09-16T20:51:37.411 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:37.411 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.412 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.412 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:37.412 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884504+0000", 2024-09-16T20:51:37.412 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:37.412 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.412 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.412 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:37.412 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884741+0000", 2024-09-16T20:51:37.412 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00023679899999999999 2024-09-16T20:51:37.412 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.412 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.412 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:37.412 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884747+0000", 2024-09-16T20:51:37.413 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0009999999999998e-06 2024-09-16T20:51:37.413 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.413 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.413 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:37.413 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884764+0000", 2024-09-16T20:51:37.413 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7371999999999999e-05 2024-09-16T20:51:37.413 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.413 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.413 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:37.413 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884772+0000", 2024-09-16T20:51:37.413 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1349999999999992e-06 2024-09-16T20:51:37.413 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.413 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.413 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:37.414 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936956+0000", 2024-09-16T20:51:37.414 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052183754999999998 2024-09-16T20:51:37.414 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.414 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.414 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:37.414 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937003+0000", 2024-09-16T20:51:37.414 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6925000000000002e-05 2024-09-16T20:51:37.414 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.414 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.414 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:37.414 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937079+0000", 2024-09-16T20:51:37.414 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6458999999999994e-05 2024-09-16T20:51:37.414 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:37.414 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:37.415 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:37.415 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.415 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.415 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:30174 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_remove in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e176)", 2024-09-16T20:51:37.415 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:51:03.615665+0000", 2024-09-16T20:51:37.415 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 33.709583979000001, 2024-09-16T20:51:37.415 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.96831816599999998, 2024-09-16T20:51:37.415 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:37.415 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:37.415 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:37.415 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:37.415 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:51:37.415 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:37.415 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 30174 2024-09-16T20:51:37.415 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.416 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:37.416 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.416 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:37.416 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615665+0000", 2024-09-16T20:51:37.416 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:37.416 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.416 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.416 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:37.416 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615665+0000", 2024-09-16T20:51:37.416 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:37.416 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.416 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.416 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:37.417 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615667+0000", 2024-09-16T20:51:37.417 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0020000000000001e-06 2024-09-16T20:51:37.417 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.417 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.417 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:37.417 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615675+0000", 2024-09-16T20:51:37.417 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5120000000000002e-06 2024-09-16T20:51:37.417 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.417 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.417 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:37.417 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615692+0000", 2024-09-16T20:51:37.417 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.6977e-05 2024-09-16T20:51:37.417 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.417 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.417 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:37.418 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615701+0000", 2024-09-16T20:51:37.418 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6119999999999999e-06 2024-09-16T20:51:37.418 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.418 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.418 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:37.418 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615782+0000", 2024-09-16T20:51:37.418 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1804000000000005e-05 2024-09-16T20:51:37.418 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.418 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.418 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:37.418 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615795+0000", 2024-09-16T20:51:37.418 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2464e-05 2024-09-16T20:51:37.418 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.418 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.418 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:37.419 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.580071+0000", 2024-09-16T20:51:37.419 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.96427590100000005 2024-09-16T20:51:37.419 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.419 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.419 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:37.419 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.580247+0000", 2024-09-16T20:51:37.419 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000175754 2024-09-16T20:51:37.419 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.419 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.419 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 2", 2024-09-16T20:51:37.419 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.580436+0000", 2024-09-16T20:51:37.419 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00018945299999999999 2024-09-16T20:51:37.419 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.419 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.419 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:37.420 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.581045+0000", 2024-09-16T20:51:37.420 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00060937600000000001 2024-09-16T20:51:37.420 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.420 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.420 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:37.420 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.583883+0000", 2024-09-16T20:51:37.420 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0028372900000000001 2024-09-16T20:51:37.420 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.420 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.420 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:37.420 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.583944+0000", 2024-09-16T20:51:37.420 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1848999999999997e-05 2024-09-16T20:51:37.420 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.420 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.421 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:37.421 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.583984+0000", 2024-09-16T20:51:37.421 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9171999999999997e-05 2024-09-16T20:51:37.421 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:37.421 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:37.421 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:37.421 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.421 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.421 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:30201 3.f 3:f6903a53:::rbd_object_map.110eca9b190a.000000000000001a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e177)", 2024-09-16T20:51:37.421 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:51:04.603586+0000", 2024-09-16T20:51:37.421 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.721663872000001, 2024-09-16T20:51:37.421 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.98028146800000004, 2024-09-16T20:51:37.421 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:37.422 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:37.422 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:37.422 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:37.422 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:51:37.422 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:37.422 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 30201 2024-09-16T20:51:37.422 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.422 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:37.422 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.422 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:37.422 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603586+0000", 2024-09-16T20:51:37.422 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:37.422 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.423 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.423 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:37.423 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603586+0000", 2024-09-16T20:51:37.423 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:37.423 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.423 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.423 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:37.423 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603587+0000", 2024-09-16T20:51:37.423 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.519e-06 2024-09-16T20:51:37.423 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.423 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.423 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:37.423 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603593+0000", 2024-09-16T20:51:37.423 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1449999999999996e-06 2024-09-16T20:51:37.423 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.424 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.424 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:37.424 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603606+0000", 2024-09-16T20:51:37.424 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2797e-05 2024-09-16T20:51:37.424 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.424 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.424 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:37.424 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603612+0000", 2024-09-16T20:51:37.424 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8710000000000002e-06 2024-09-16T20:51:37.424 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.424 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.424 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:37.424 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603675+0000", 2024-09-16T20:51:37.424 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2728e-05 2024-09-16T20:51:37.425 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.425 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.425 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:37.425 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603687+0000", 2024-09-16T20:51:37.425 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.253e-05 2024-09-16T20:51:37.425 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.425 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.425 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:37.425 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:05.583358+0000", 2024-09-16T20:51:37.425 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.97967074600000004 2024-09-16T20:51:37.425 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.425 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.425 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:37.425 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:05.583515+0000", 2024-09-16T20:51:37.426 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00015755700000000001 2024-09-16T20:51:37.426 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:37.426 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:37.426 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:37.426 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:05.583867+0000", 2024-09-16T20:51:37.426 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00035157500000000002 2024-09-16T20:51:37.426 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:37.426 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:37.426 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:37.426 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:37.426 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:37.426 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:37.428 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:51:37.428 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:37.426+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:37.432 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:51:37.432 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:51:37.449 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:51:37.449 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:51:37.466 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:51:37.467 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:51:37.486 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:51:37.504 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:51:37.504 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:51:37.522 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:51:37.522 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:51:37.529 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:37.528+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:37.542 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:37.542 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:37.551 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:51:37.635 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:37.634+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:37.648 INFO:teuthology.orchestra.run.smithi029.stderr:noscrub is unset 2024-09-16T20:51:37.705 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:37.705 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:37.715 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:51:37.730 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:37.729+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:37.831 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:37.829+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:37.865 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:37.865 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:37.874 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:51:37.931 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:51:37.930+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:51:37.981 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:37.981 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:37.990 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:51:38.033 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:38.032+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:38.097 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:38.097 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:38.106 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:51:38.132 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:38.132+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:38.234 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:38.233+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:38.253 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:38.253 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:38.262 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:51:38.335 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:38.334+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:38.382 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:38.382 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:38.391 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:51:38.436 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:38.435+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:38.500 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:38.500 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:38.508 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:51:38.536 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:38.536+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:38.615 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:38.615 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:38.624 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:51:38.668 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:38.667+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:38.737 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:38.737 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:38.738 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:38.737+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:38.746 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:51:38.839 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:38.837+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:38.894 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:38.894 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:38.902 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:51:38.943 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:38.942+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:39.041 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:39.040+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:39.049 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:39.050 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:39.058 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:51:39.142 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:39.141+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:39.166 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:39.166 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:39.175 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:51:39.243 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:39.242+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:39.282 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:39.282 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:39.292 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:51:39.344 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:39.343+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:39.439 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:39.439 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:39.445 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:51:39.443+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:51:39.448 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:51:39.534 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T20:51:39.545 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:39.544+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:39.569 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:39.569 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:39.583 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:51:39.599 INFO:tasks.thrashosds.thrasher:in_osds: [1, 4, 5, 0, 3, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-16T20:51:39.599 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T20:51:39.599 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2024-09-16T20:51:39.599 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T20:51:39.599 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T20:51:39.599 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok config set filestore_inject_stall 3 2024-09-16T20:51:39.645 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:39.644+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:39.666 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T20:51:39.685 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:51:39.685 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:51:39.708 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:51:39.708 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:51:39.710 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:39.711 INFO:teuthology.orchestra.run.smithi043.stdout: "success": "" 2024-09-16T20:51:39.711 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:39.726 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:51:39.727 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:51:39.746 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:51:39.746 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:51:39.751 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:39.751 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:51:39.751 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:51:39.752 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:51:39.752 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:39.761 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:51:39.762 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:39.759+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:39.764 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:51:39.764 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:51:39.783 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:51:39.784 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:51:39.846 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:39.845+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:39.885 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:39.885 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:51:39.885 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:51:39.885 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:51:39.885 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.885 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5439 3.a 3:5789f5c3:::rbd_data.110ed9ca540.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 1181696~60416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:39.885 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:51:39.885 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 197.01840619999999, 2024-09-16T20:51:39.885 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10102045599999999, 2024-09-16T20:51:39.885 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:39.886 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:39.886 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:39.886 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:39.886 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:39.886 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:39.886 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5439 2024-09-16T20:51:39.886 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.886 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:39.886 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.886 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:39.886 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:51:39.886 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:39.886 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.886 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.886 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:39.887 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:51:39.887 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:39.887 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.887 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.887 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:39.887 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864711+0000", 2024-09-16T20:51:39.887 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7899999999999997e-07 2024-09-16T20:51:39.887 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.887 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.887 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:39.887 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864760+0000", 2024-09-16T20:51:39.887 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8803e-05 2024-09-16T20:51:39.887 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.888 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.888 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:39.888 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864769+0000", 2024-09-16T20:51:39.888 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0999999999999993e-06 2024-09-16T20:51:39.888 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.888 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.888 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:39.888 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864774+0000", 2024-09-16T20:51:39.888 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8919999999999999e-06 2024-09-16T20:51:39.888 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.888 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.888 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:39.888 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864815+0000", 2024-09-16T20:51:39.888 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1307000000000003e-05 2024-09-16T20:51:39.889 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.889 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.889 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:39.889 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864893+0000", 2024-09-16T20:51:39.889 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7581000000000001e-05 2024-09-16T20:51:39.889 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.889 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.889 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 1", 2024-09-16T20:51:39.889 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864996+0000", 2024-09-16T20:51:39.889 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102725 2024-09-16T20:51:39.889 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.890 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.890 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:39.890 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965442+0000", 2024-09-16T20:51:39.890 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100446012 2024-09-16T20:51:39.890 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.890 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.890 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:39.890 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965618+0000", 2024-09-16T20:51:39.890 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00017653100000000001 2024-09-16T20:51:39.890 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.890 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.890 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:39.890 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965682+0000", 2024-09-16T20:51:39.890 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3905999999999994e-05 2024-09-16T20:51:39.890 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.891 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.891 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:39.891 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965731+0000", 2024-09-16T20:51:39.891 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8819999999999997e-05 2024-09-16T20:51:39.891 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:39.891 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:39.891 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:39.891 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.891 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.891 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4600.0:5439 3.a e37/23 ondisk, result = 0)", 2024-09-16T20:51:39.891 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:51:39.891 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 197.017410052, 2024-09-16T20:51:39.891 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099742634999999996, 2024-09-16T20:51:39.892 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:39.892 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:39.892 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:39.892 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:39.892 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.892 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:39.892 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:51:39.892 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:39.892 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.892 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.892 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:39.892 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:51:39.892 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:39.892 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.893 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.893 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:39.893 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:51:39.893 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0900000000000001e-07 2024-09-16T20:51:39.893 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.893 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.893 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:39.893 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865711+0000", 2024-09-16T20:51:39.893 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3739999999999998e-06 2024-09-16T20:51:39.893 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.893 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.893 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:39.893 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965343+0000", 2024-09-16T20:51:39.894 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099632526999999999 2024-09-16T20:51:39.894 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.894 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.894 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:39.894 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965357+0000", 2024-09-16T20:51:39.894 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3713999999999999e-05 2024-09-16T20:51:39.894 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.894 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.894 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:39.894 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965426+0000", 2024-09-16T20:51:39.894 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8745999999999995e-05 2024-09-16T20:51:39.894 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.894 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.894 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:39.895 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965434+0000", 2024-09-16T20:51:39.895 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6910000000000003e-06 2024-09-16T20:51:39.895 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.895 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.895 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:39.895 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965449+0000", 2024-09-16T20:51:39.895 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4874e-05 2024-09-16T20:51:39.895 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:39.895 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:39.895 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:39.895 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.895 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.895 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4882.0:2508 3.12 3:4992771f:::rbd_object_map.110e7c8851c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T20:51:39.895 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:51:39.895 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 173.401606504, 2024-09-16T20:51:39.896 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.125007011, 2024-09-16T20:51:39.896 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:39.896 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:39.896 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:39.896 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:39.896 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4882", 2024-09-16T20:51:39.896 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2436923223", 2024-09-16T20:51:39.896 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2508 2024-09-16T20:51:39.896 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.896 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:39.896 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.896 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:39.896 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:51:39.897 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:39.897 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.897 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.897 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:39.897 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:51:39.897 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:39.897 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.897 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.897 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:39.897 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481511+0000", 2024-09-16T20:51:39.897 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0699999999999996e-07 2024-09-16T20:51:39.897 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.897 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.897 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:39.897 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481515+0000", 2024-09-16T20:51:39.898 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0609999999999997e-06 2024-09-16T20:51:39.898 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.898 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.898 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:39.898 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481523+0000", 2024-09-16T20:51:39.898 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1310000000000006e-06 2024-09-16T20:51:39.898 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.898 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.898 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:39.898 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481528+0000", 2024-09-16T20:51:39.898 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4900000000000002e-06 2024-09-16T20:51:39.898 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.898 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.898 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:39.899 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481565+0000", 2024-09-16T20:51:39.899 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7066000000000001e-05 2024-09-16T20:51:39.899 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.899 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.899 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:39.899 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481628+0000", 2024-09-16T20:51:39.899 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3118000000000001e-05 2024-09-16T20:51:39.899 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.899 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.899 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 2", 2024-09-16T20:51:39.899 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481822+0000", 2024-09-16T20:51:39.899 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019446200000000001 2024-09-16T20:51:39.899 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.899 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.900 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:39.900 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605988+0000", 2024-09-16T20:51:39.900 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.124165888 2024-09-16T20:51:39.900 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.900 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.900 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:39.900 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606427+0000", 2024-09-16T20:51:39.900 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00043854899999999999 2024-09-16T20:51:39.900 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.900 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.900 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:39.900 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606476+0000", 2024-09-16T20:51:39.900 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8971999999999998e-05 2024-09-16T20:51:39.901 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.901 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.901 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:39.901 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606517+0000", 2024-09-16T20:51:39.901 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1467000000000001e-05 2024-09-16T20:51:39.901 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:39.901 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:39.901 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:39.901 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.901 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.901 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4882.0:2508 3.12 e57/55 ondisk, result = 0)", 2024-09-16T20:51:39.901 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:51:39.901 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 173.40061800399999, 2024-09-16T20:51:39.902 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12349711400000001, 2024-09-16T20:51:39.902 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:39.902 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:39.902 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:39.902 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:39.902 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.902 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:39.902 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:51:39.902 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:39.902 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.902 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.902 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:39.902 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:51:39.902 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:39.903 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.903 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.903 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:39.903 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:51:39.903 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.06e-07 2024-09-16T20:51:39.903 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.903 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.903 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:39.903 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482502+0000", 2024-09-16T20:51:39.903 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.01e-06 2024-09-16T20:51:39.903 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.903 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.903 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:39.904 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605772+0000", 2024-09-16T20:51:39.904 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.123269687 2024-09-16T20:51:39.904 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.904 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.904 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:39.904 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605791+0000", 2024-09-16T20:51:39.904 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8912000000000001e-05 2024-09-16T20:51:39.904 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.904 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.904 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:39.904 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605974+0000", 2024-09-16T20:51:39.904 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000182836 2024-09-16T20:51:39.904 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.905 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.905 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:39.905 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605982+0000", 2024-09-16T20:51:39.905 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1699999999999997e-06 2024-09-16T20:51:39.905 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.905 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.905 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:39.905 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605996+0000", 2024-09-16T20:51:39.905 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3893000000000001e-05 2024-09-16T20:51:39.905 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:39.905 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:39.905 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:39.905 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.906 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.906 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4882.0:2512 3.f 3:f4d895b7:::rbd_data.110e7c8851c.0000000000000000:head [read 16384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T20:51:39.906 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:51:39.906 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 173.40018097399999, 2024-09-16T20:51:39.906 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12318483, 2024-09-16T20:51:39.906 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:39.906 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:39.906 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:39.906 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:39.906 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4882", 2024-09-16T20:51:39.906 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2436923223", 2024-09-16T20:51:39.906 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2512 2024-09-16T20:51:39.906 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.906 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:39.906 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.906 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:39.907 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:51:39.907 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:39.907 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.907 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.907 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:39.907 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:51:39.907 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:39.907 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.907 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.907 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:39.907 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:51:39.907 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3499999999999995e-07 2024-09-16T20:51:39.907 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.907 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.907 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:39.908 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482943+0000", 2024-09-16T20:51:39.908 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3450000000000002e-06 2024-09-16T20:51:39.908 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.908 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.908 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:39.908 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605772+0000", 2024-09-16T20:51:39.908 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.122829796 2024-09-16T20:51:39.908 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.908 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.908 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:39.908 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605792+0000", 2024-09-16T20:51:39.908 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9916999999999999e-05 2024-09-16T20:51:39.908 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.908 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.909 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:39.909 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605917+0000", 2024-09-16T20:51:39.909 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00012500799999999999 2024-09-16T20:51:39.909 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.909 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.909 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:39.909 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606004+0000", 2024-09-16T20:51:39.909 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6242999999999994e-05 2024-09-16T20:51:39.909 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.909 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.909 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:39.909 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606120+0000", 2024-09-16T20:51:39.909 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000116786 2024-09-16T20:51:39.909 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:39.910 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:39.910 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:39.910 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.910 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.910 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:11702 3.15 3:acf33c8b:::rbd_data.110e747f0779.000000000000001a:head [set-alloc-hint object_size 262144 write_size 262144,write 143360~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:51:39.910 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:51:39.910 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 155.745188552, 2024-09-16T20:51:39.910 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.084801043000000006, 2024-09-16T20:51:39.910 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:39.910 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:39.910 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:39.910 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:39.910 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:51:39.911 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:51:39.911 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 11702 2024-09-16T20:51:39.911 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.911 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:39.911 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.911 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:39.911 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:51:39.911 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:39.911 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.911 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.911 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:39.911 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:51:39.911 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:39.911 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.911 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.912 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:39.912 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137929+0000", 2024-09-16T20:51:39.912 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8400000000000002e-07 2024-09-16T20:51:39.912 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.912 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.912 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:39.912 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137937+0000", 2024-09-16T20:51:39.912 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0260000000000007e-06 2024-09-16T20:51:39.912 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.912 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.912 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:39.912 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137952+0000", 2024-09-16T20:51:39.912 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4659e-05 2024-09-16T20:51:39.912 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.913 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.913 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:39.913 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137960+0000", 2024-09-16T20:51:39.913 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0169999999999996e-06 2024-09-16T20:51:39.913 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.913 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.913 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:39.913 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138010+0000", 2024-09-16T20:51:39.913 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0021000000000001e-05 2024-09-16T20:51:39.913 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.913 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.913 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:39.914 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138114+0000", 2024-09-16T20:51:39.914 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000103797 2024-09-16T20:51:39.914 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.914 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.914 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:51:39.914 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138233+0000", 2024-09-16T20:51:39.914 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000118971 2024-09-16T20:51:39.914 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.914 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.914 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:39.914 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138953+0000", 2024-09-16T20:51:39.914 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00072026000000000002 2024-09-16T20:51:39.914 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.914 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.915 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:39.915 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222609+0000", 2024-09-16T20:51:39.915 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.083656650999999999 2024-09-16T20:51:39.915 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.915 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.915 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:39.915 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222684+0000", 2024-09-16T20:51:39.915 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4369999999999994e-05 2024-09-16T20:51:39.915 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.915 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.915 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:39.915 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222729+0000", 2024-09-16T20:51:39.915 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5287000000000002e-05 2024-09-16T20:51:39.915 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:39.915 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:39.916 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:39.916 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.916 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.916 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:11704 3.9 3:91220b75:::rbd_data.110e747f0779.000000000000001c:head [set-alloc-hint object_size 262144 write_size 262144,write 139264~6656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:51:39.916 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:51:39.916 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 155.74495055099999, 2024-09-16T20:51:39.916 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.084531133999999994, 2024-09-16T20:51:39.916 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:39.916 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:39.916 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:39.916 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:39.916 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:51:39.916 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:51:39.917 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 11704 2024-09-16T20:51:39.917 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.917 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:39.917 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.917 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:39.917 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:51:39.917 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:39.917 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.917 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.917 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:39.917 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:51:39.917 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:39.917 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.918 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.918 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:39.918 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138167+0000", 2024-09-16T20:51:39.918 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.33e-07 2024-09-16T20:51:39.918 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.918 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.918 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:39.918 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138179+0000", 2024-09-16T20:51:39.918 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2055999999999999e-05 2024-09-16T20:51:39.918 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.918 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.918 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:39.918 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138193+0000", 2024-09-16T20:51:39.918 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3961000000000001e-05 2024-09-16T20:51:39.919 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.919 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.919 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:39.919 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138200+0000", 2024-09-16T20:51:39.919 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4429999999999997e-06 2024-09-16T20:51:39.919 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.919 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.919 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:39.919 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138258+0000", 2024-09-16T20:51:39.919 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7225999999999997e-05 2024-09-16T20:51:39.919 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.919 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.919 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:39.919 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138364+0000", 2024-09-16T20:51:39.919 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000106013 2024-09-16T20:51:39.920 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.920 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.920 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:51:39.920 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138494+0000", 2024-09-16T20:51:39.920 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013059599999999999 2024-09-16T20:51:39.920 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.920 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.920 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:39.920 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.139220+0000", 2024-09-16T20:51:39.920 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00072539800000000004 2024-09-16T20:51:39.920 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.920 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.920 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:39.920 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222608+0000", 2024-09-16T20:51:39.921 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.083388620999999996 2024-09-16T20:51:39.921 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.921 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.921 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:39.921 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222651+0000", 2024-09-16T20:51:39.921 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2228000000000003e-05 2024-09-16T20:51:39.921 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.921 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.921 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:39.921 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222697+0000", 2024-09-16T20:51:39.921 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6659000000000002e-05 2024-09-16T20:51:39.921 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:39.921 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:39.921 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:39.922 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.922 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.922 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5138.0:9358 3.c 3.c25699cc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T20:51:39.922 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:51:39.922 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 133.03280324599999, 2024-09-16T20:51:39.922 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0018657580000001, 2024-09-16T20:51:39.922 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:39.922 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:39.922 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:51:39.922 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:39.922 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5138", 2024-09-16T20:51:39.922 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:51:39.922 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 9358 2024-09-16T20:51:39.923 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.923 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:39.923 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.923 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:39.923 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:51:39.923 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:39.923 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.923 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.923 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:39.923 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:51:39.923 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:39.923 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.923 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.924 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:39.924 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850314+0000", 2024-09-16T20:51:39.924 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4099999999999999e-07 2024-09-16T20:51:39.924 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.924 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.924 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:39.924 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850316+0000", 2024-09-16T20:51:39.924 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2469999999999998e-06 2024-09-16T20:51:39.924 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.924 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.924 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:39.925 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850322+0000", 2024-09-16T20:51:39.925 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3470000000000004e-06 2024-09-16T20:51:39.925 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.925 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.925 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:39.925 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850327+0000", 2024-09-16T20:51:39.925 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9450000000000001e-06 2024-09-16T20:51:39.925 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.925 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.925 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:39.925 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850376+0000", 2024-09-16T20:51:39.925 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8257000000000002e-05 2024-09-16T20:51:39.925 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.925 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.925 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:39.926 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850385+0000", 2024-09-16T20:51:39.926 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9800000000000004e-06 2024-09-16T20:51:39.926 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.926 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.926 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:39.926 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852167+0000", 2024-09-16T20:51:39.926 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0017825659999999 2024-09-16T20:51:39.926 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.926 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.926 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:39.926 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852179+0000", 2024-09-16T20:51:39.926 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1975e-05 2024-09-16T20:51:39.926 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:39.926 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:39.926 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:39.927 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.927 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.927 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5138.0:9382 3.c 3.c25699cc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T20:51:39.927 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:51:39.927 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 133.02445623, 2024-09-16T20:51:39.927 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.99358404700000003, 2024-09-16T20:51:39.927 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:39.927 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:39.927 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:51:39.927 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:39.927 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5138", 2024-09-16T20:51:39.927 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:51:39.927 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 9382 2024-09-16T20:51:39.927 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.927 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:39.928 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.928 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:39.928 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:51:39.928 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:39.928 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.928 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.928 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:39.928 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:51:39.928 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:39.928 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.928 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.928 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:39.928 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858661+0000", 2024-09-16T20:51:39.929 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0200000000000002e-07 2024-09-16T20:51:39.929 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.929 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.929 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:39.929 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858664+0000", 2024-09-16T20:51:39.929 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.038e-06 2024-09-16T20:51:39.929 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.929 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.929 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:39.929 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858671+0000", 2024-09-16T20:51:39.929 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0110000000000003e-06 2024-09-16T20:51:39.929 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.929 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.929 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:39.930 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858677+0000", 2024-09-16T20:51:39.930 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0959999999999997e-06 2024-09-16T20:51:39.930 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.930 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.930 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:39.930 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858732+0000", 2024-09-16T20:51:39.930 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4543000000000003e-05 2024-09-16T20:51:39.930 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.930 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.930 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:39.930 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858739+0000", 2024-09-16T20:51:39.930 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5429999999999996e-06 2024-09-16T20:51:39.930 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.930 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.931 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:39.931 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852236+0000", 2024-09-16T20:51:39.931 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.993496453 2024-09-16T20:51:39.931 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.931 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.931 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:39.931 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852244+0000", 2024-09-16T20:51:39.931 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8610000000000004e-06 2024-09-16T20:51:39.931 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:39.931 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:39.931 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:39.931 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.931 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.931 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5763.0:4895 3.a 3:5384606c:::rbd_data.110e1b19a241.0000000000000018:head [read 0~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-16T20:51:39.931 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:45.598212+0000", 2024-09-16T20:51:39.932 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 114.284904189, 2024-09-16T20:51:39.932 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0375864829999999, 2024-09-16T20:51:39.932 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:39.932 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:39.932 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:39.932 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:39.932 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5763", 2024-09-16T20:51:39.932 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/623897567", 2024-09-16T20:51:39.932 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4895 2024-09-16T20:51:39.932 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.932 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:39.932 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.932 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:39.933 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598212+0000", 2024-09-16T20:51:39.933 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:39.933 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.933 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.933 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:39.933 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598212+0000", 2024-09-16T20:51:39.933 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:39.933 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.933 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.933 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:39.933 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598214+0000", 2024-09-16T20:51:39.933 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0090000000000002e-06 2024-09-16T20:51:39.933 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.934 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.934 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:39.934 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598222+0000", 2024-09-16T20:51:39.934 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6699999999999994e-06 2024-09-16T20:51:39.934 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.934 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.934 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:39.934 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598241+0000", 2024-09-16T20:51:39.934 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8896999999999999e-05 2024-09-16T20:51:39.934 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.934 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.934 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:39.934 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598250+0000", 2024-09-16T20:51:39.934 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6649999999999992e-06 2024-09-16T20:51:39.935 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.935 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.935 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:39.935 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598325+0000", 2024-09-16T20:51:39.935 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5189000000000001e-05 2024-09-16T20:51:39.935 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.935 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.935 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op must wait for map", 2024-09-16T20:51:39.935 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598333+0000", 2024-09-16T20:51:39.935 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.3170000000000002e-06 2024-09-16T20:51:39.935 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.935 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.936 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:39.936 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601355+0000", 2024-09-16T20:51:39.936 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0030215400000000001 2024-09-16T20:51:39.936 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.936 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.936 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:39.936 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601365+0000", 2024-09-16T20:51:39.936 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0215e-05 2024-09-16T20:51:39.936 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.936 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.936 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:39.936 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635506+0000", 2024-09-16T20:51:39.936 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0341408240000001 2024-09-16T20:51:39.936 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.937 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.937 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:39.937 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635655+0000", 2024-09-16T20:51:39.937 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000149412 2024-09-16T20:51:39.937 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.937 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.937 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:39.937 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635799+0000", 2024-09-16T20:51:39.937 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000143745 2024-09-16T20:51:39.937 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:39.937 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:39.937 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:39.938 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.938 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.938 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5763.0:4900 3.a 3:5384606c:::rbd_data.110e1b19a241.0000000000000018:head [read 8192~24576] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-16T20:51:39.938 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:51:39.938 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 114.284845713, 2024-09-16T20:51:39.938 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0379340800000001, 2024-09-16T20:51:39.938 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:39.938 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:39.938 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:39.938 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:39.938 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5763", 2024-09-16T20:51:39.938 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/623897567", 2024-09-16T20:51:39.938 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4900 2024-09-16T20:51:39.938 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.939 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:39.939 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.939 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:39.939 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:51:39.939 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:39.939 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.939 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.939 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:39.939 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:51:39.939 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:39.939 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.939 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.939 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:39.940 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:51:39.940 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4300000000000003e-07 2024-09-16T20:51:39.940 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.940 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.940 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:39.940 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598274+0000", 2024-09-16T20:51:39.940 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.976e-06 2024-09-16T20:51:39.940 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.940 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.940 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:39.940 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598282+0000", 2024-09-16T20:51:39.940 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0029999999999994e-06 2024-09-16T20:51:39.940 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.940 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.940 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:39.941 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598289+0000", 2024-09-16T20:51:39.941 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2389999999999999e-06 2024-09-16T20:51:39.941 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.941 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.941 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:39.941 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598430+0000", 2024-09-16T20:51:39.941 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014119600000000001 2024-09-16T20:51:39.941 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.941 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.941 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting_for_map not empty", 2024-09-16T20:51:39.941 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598435+0000", 2024-09-16T20:51:39.941 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3489999999999999e-06 2024-09-16T20:51:39.941 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.941 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.941 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:39.942 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601429+0000", 2024-09-16T20:51:39.942 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0029942720000000001 2024-09-16T20:51:39.942 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.942 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.942 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:39.942 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601438+0000", 2024-09-16T20:51:39.942 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4759999999999998e-06 2024-09-16T20:51:39.942 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.942 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.942 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:39.942 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635987+0000", 2024-09-16T20:51:39.942 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0345495010000001 2024-09-16T20:51:39.942 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.942 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.942 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:39.943 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.636056+0000", 2024-09-16T20:51:39.943 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8610999999999998e-05 2024-09-16T20:51:39.943 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.943 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.943 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:39.943 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.636205+0000", 2024-09-16T20:51:39.943 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014891399999999999 2024-09-16T20:51:39.943 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:39.943 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:39.943 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:39.943 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.943 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.943 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop(client.6231.0:2182 3.17 e136/81 3:edb5e5cd:::rbd_data.110e61479aee.0000000000000014:head v 136'791, mlcod=135'789)", 2024-09-16T20:51:39.943 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:22.735499+0000", 2024-09-16T20:51:39.944 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 77.147617772000004, 2024-09-16T20:51:39.944 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12369608, 2024-09-16T20:51:39.944 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:39.944 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:39.944 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:39.944 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:39.944 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.944 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:39.944 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735499+0000", 2024-09-16T20:51:39.944 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:39.944 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.944 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.944 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:39.944 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735499+0000", 2024-09-16T20:51:39.945 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:39.945 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.945 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.945 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:39.945 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735499+0000", 2024-09-16T20:51:39.945 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8099999999999999e-07 2024-09-16T20:51:39.945 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.945 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.945 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:39.945 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735506+0000", 2024-09-16T20:51:39.945 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3579999999999997e-06 2024-09-16T20:51:39.945 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.945 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.945 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:39.945 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735517+0000", 2024-09-16T20:51:39.945 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0869999999999999e-05 2024-09-16T20:51:39.946 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.946 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.946 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:39.946 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735533+0000", 2024-09-16T20:51:39.946 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5858e-05 2024-09-16T20:51:39.946 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.946 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.946 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:39.946 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735584+0000", 2024-09-16T20:51:39.946 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0470999999999998e-05 2024-09-16T20:51:39.946 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.946 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.946 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:39.946 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735607+0000", 2024-09-16T20:51:39.946 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2875999999999999e-05 2024-09-16T20:51:39.946 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.947 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.947 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:39.947 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.859144+0000", 2024-09-16T20:51:39.947 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.123537701 2024-09-16T20:51:39.947 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.947 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.947 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:39.947 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.859195+0000", 2024-09-16T20:51:39.947 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0664999999999997e-05 2024-09-16T20:51:39.947 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:39.947 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:39.947 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:39.947 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.947 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.948 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5204 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 2523136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:51:39.948 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.883660+0000", 2024-09-16T20:51:39.948 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 75.999456241999994, 2024-09-16T20:51:39.948 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052517638999999998, 2024-09-16T20:51:39.948 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:39.948 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:39.948 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:39.948 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:39.948 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:39.948 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:39.948 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5204 2024-09-16T20:51:39.948 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.949 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:39.949 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.949 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:39.949 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883660+0000", 2024-09-16T20:51:39.949 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:39.949 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.949 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.949 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:39.949 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883660+0000", 2024-09-16T20:51:39.949 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:39.949 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.949 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.949 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:39.949 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883661+0000", 2024-09-16T20:51:39.950 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9999999999999998e-07 2024-09-16T20:51:39.950 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.950 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.950 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:39.950 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883663+0000", 2024-09-16T20:51:39.950 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.4480000000000001e-06 2024-09-16T20:51:39.950 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.950 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.950 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:39.950 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883669+0000", 2024-09-16T20:51:39.950 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.071e-06 2024-09-16T20:51:39.950 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.950 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.950 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:39.951 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883675+0000", 2024-09-16T20:51:39.951 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.366e-06 2024-09-16T20:51:39.951 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.951 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.951 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:39.951 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.935970+0000", 2024-09-16T20:51:39.951 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052295346999999999 2024-09-16T20:51:39.951 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.951 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.951 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:39.951 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936064+0000", 2024-09-16T20:51:39.951 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4438000000000006e-05 2024-09-16T20:51:39.951 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.952 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.952 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:39.952 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936178+0000", 2024-09-16T20:51:39.952 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000113569 2024-09-16T20:51:39.952 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:39.952 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:39.952 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:39.952 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.952 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.952 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5205 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 2543616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:51:39.952 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.883745+0000", 2024-09-16T20:51:39.952 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 75.999371483999994, 2024-09-16T20:51:39.952 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052692112999999999, 2024-09-16T20:51:39.952 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:39.953 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:39.953 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:39.953 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:39.953 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:39.953 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:39.953 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5205 2024-09-16T20:51:39.953 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.953 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:39.953 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.953 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:39.953 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883745+0000", 2024-09-16T20:51:39.953 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:39.953 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.953 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.954 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:39.954 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883745+0000", 2024-09-16T20:51:39.954 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:39.954 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.954 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.954 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:39.954 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883746+0000", 2024-09-16T20:51:39.954 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2699999999999999e-07 2024-09-16T20:51:39.954 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.954 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.954 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:39.954 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883750+0000", 2024-09-16T20:51:39.954 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7050000000000001e-06 2024-09-16T20:51:39.954 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.954 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.954 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:39.955 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883769+0000", 2024-09-16T20:51:39.955 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9165e-05 2024-09-16T20:51:39.955 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.955 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.955 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:39.955 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883778+0000", 2024-09-16T20:51:39.955 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.6690000000000001e-06 2024-09-16T20:51:39.955 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.955 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.955 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:39.955 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936228+0000", 2024-09-16T20:51:39.955 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052449587999999998 2024-09-16T20:51:39.955 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.955 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.956 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:39.956 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936333+0000", 2024-09-16T20:51:39.956 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010511800000000001 2024-09-16T20:51:39.956 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.956 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.956 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:39.956 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936437+0000", 2024-09-16T20:51:39.956 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000104141 2024-09-16T20:51:39.956 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:39.956 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:39.956 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:39.956 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.956 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.956 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5206 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 1167360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:51:39.956 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.883802+0000", 2024-09-16T20:51:39.957 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 75.999314990000002, 2024-09-16T20:51:39.957 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052800780999999998, 2024-09-16T20:51:39.957 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:39.957 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:39.957 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:39.957 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:39.957 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:39.957 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:39.957 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5206 2024-09-16T20:51:39.957 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.957 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:39.957 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.957 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:39.957 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883802+0000", 2024-09-16T20:51:39.957 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:39.957 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.958 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.958 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:39.958 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883802+0000", 2024-09-16T20:51:39.958 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:39.958 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.958 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.958 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:39.958 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883933+0000", 2024-09-16T20:51:39.958 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013182999999999999 2024-09-16T20:51:39.958 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.958 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.958 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:39.958 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883937+0000", 2024-09-16T20:51:39.958 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.771e-06 2024-09-16T20:51:39.959 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.959 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.959 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:39.959 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883953+0000", 2024-09-16T20:51:39.959 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5409000000000001e-05 2024-09-16T20:51:39.959 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.959 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.959 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:39.959 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883959+0000", 2024-09-16T20:51:39.959 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0700000000000003e-06 2024-09-16T20:51:39.959 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.959 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.959 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:39.959 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936476+0000", 2024-09-16T20:51:39.960 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052517434000000002 2024-09-16T20:51:39.960 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.960 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.960 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:39.960 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936529+0000", 2024-09-16T20:51:39.960 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2969000000000001e-05 2024-09-16T20:51:39.960 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.960 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.960 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:39.960 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936602+0000", 2024-09-16T20:51:39.960 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3298000000000002e-05 2024-09-16T20:51:39.960 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:39.960 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:39.960 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:39.960 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.961 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.961 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5207 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 1904640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:51:39.961 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.883980+0000", 2024-09-16T20:51:39.961 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 75.999136781000004, 2024-09-16T20:51:39.961 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052759922000000001, 2024-09-16T20:51:39.961 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:39.961 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:39.961 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:39.961 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:39.961 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:39.961 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:39.961 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5207 2024-09-16T20:51:39.961 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.961 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:39.961 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.962 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:39.962 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883980+0000", 2024-09-16T20:51:39.962 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:39.962 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.962 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.962 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:39.962 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883980+0000", 2024-09-16T20:51:39.962 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:39.962 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.962 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.962 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:39.962 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884126+0000", 2024-09-16T20:51:39.962 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014577399999999999 2024-09-16T20:51:39.962 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.962 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.962 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:39.963 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884130+0000", 2024-09-16T20:51:39.963 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9729999999999999e-06 2024-09-16T20:51:39.963 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.963 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.963 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:39.963 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884139+0000", 2024-09-16T20:51:39.963 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8660000000000001e-06 2024-09-16T20:51:39.963 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.963 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.963 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:39.963 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884144+0000", 2024-09-16T20:51:39.963 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3669999999999999e-06 2024-09-16T20:51:39.963 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.963 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.963 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:39.963 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936635+0000", 2024-09-16T20:51:39.964 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052491705 2024-09-16T20:51:39.964 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.964 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.964 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:39.964 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936679+0000", 2024-09-16T20:51:39.964 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3105999999999997e-05 2024-09-16T20:51:39.964 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.964 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.964 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:39.964 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936740+0000", 2024-09-16T20:51:39.964 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1130999999999995e-05 2024-09-16T20:51:39.964 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:39.964 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:39.965 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:39.965 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.965 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.965 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5208 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 131072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:51:39.965 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.884165+0000", 2024-09-16T20:51:39.965 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 75.998951485000006, 2024-09-16T20:51:39.965 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052746603000000003, 2024-09-16T20:51:39.965 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:39.965 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:39.965 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:39.965 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:39.965 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:39.965 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:39.965 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5208 2024-09-16T20:51:39.965 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.966 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:39.966 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.966 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:39.966 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884165+0000", 2024-09-16T20:51:39.966 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:39.966 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.966 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.966 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:39.966 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884165+0000", 2024-09-16T20:51:39.966 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:39.966 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.966 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.966 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:39.966 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884463+0000", 2024-09-16T20:51:39.966 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00029801699999999998 2024-09-16T20:51:39.966 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.967 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.967 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:39.967 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884467+0000", 2024-09-16T20:51:39.967 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8210000000000003e-06 2024-09-16T20:51:39.967 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.967 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.967 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:39.967 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884477+0000", 2024-09-16T20:51:39.967 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.7030000000000001e-06 2024-09-16T20:51:39.967 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.967 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.967 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:39.967 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884485+0000", 2024-09-16T20:51:39.967 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9710000000000004e-06 2024-09-16T20:51:39.967 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.967 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.968 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:39.968 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936785+0000", 2024-09-16T20:51:39.968 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052300665000000003 2024-09-16T20:51:39.968 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.968 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.968 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:39.968 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936851+0000", 2024-09-16T20:51:39.968 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6159000000000001e-05 2024-09-16T20:51:39.968 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.968 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.968 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:39.968 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936912+0000", 2024-09-16T20:51:39.968 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0266999999999997e-05 2024-09-16T20:51:39.969 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:39.969 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:39.969 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:39.969 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.969 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.969 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5209 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 1925120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:51:39.969 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.884504+0000", 2024-09-16T20:51:39.969 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 75.998612600000001, 2024-09-16T20:51:39.969 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052575445999999998, 2024-09-16T20:51:39.969 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:39.969 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:39.969 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:39.969 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:39.969 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:39.969 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:39.970 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5209 2024-09-16T20:51:39.970 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.970 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:39.970 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.970 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:39.970 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884504+0000", 2024-09-16T20:51:39.970 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:39.970 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.970 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.970 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:39.970 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884504+0000", 2024-09-16T20:51:39.970 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:39.970 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.971 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.971 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:39.971 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884741+0000", 2024-09-16T20:51:39.971 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00023679899999999999 2024-09-16T20:51:39.971 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.971 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.971 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:39.971 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884747+0000", 2024-09-16T20:51:39.971 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0009999999999998e-06 2024-09-16T20:51:39.971 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.971 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.971 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:39.971 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884764+0000", 2024-09-16T20:51:39.972 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7371999999999999e-05 2024-09-16T20:51:39.972 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.972 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.972 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:39.972 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884772+0000", 2024-09-16T20:51:39.972 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1349999999999992e-06 2024-09-16T20:51:39.972 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.972 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.972 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:39.972 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936956+0000", 2024-09-16T20:51:39.972 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052183754999999998 2024-09-16T20:51:39.972 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.972 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.972 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:39.973 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937003+0000", 2024-09-16T20:51:39.973 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6925000000000002e-05 2024-09-16T20:51:39.973 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.973 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.973 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:39.973 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937079+0000", 2024-09-16T20:51:39.973 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6458999999999994e-05 2024-09-16T20:51:39.973 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:39.973 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:39.973 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:39.973 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.973 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.973 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:30174 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_remove in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e176)", 2024-09-16T20:51:39.974 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:51:03.615665+0000", 2024-09-16T20:51:39.974 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 36.267451080999997, 2024-09-16T20:51:39.974 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.96831816599999998, 2024-09-16T20:51:39.974 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:39.974 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:39.974 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:39.974 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:39.974 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:51:39.974 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:39.974 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 30174 2024-09-16T20:51:39.974 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.974 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:39.975 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.975 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:39.975 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615665+0000", 2024-09-16T20:51:39.975 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:39.975 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.975 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.975 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:39.975 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615665+0000", 2024-09-16T20:51:39.975 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:39.975 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.975 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.975 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:39.975 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615667+0000", 2024-09-16T20:51:39.976 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0020000000000001e-06 2024-09-16T20:51:39.976 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.976 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.976 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:39.976 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615675+0000", 2024-09-16T20:51:39.976 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5120000000000002e-06 2024-09-16T20:51:39.976 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.976 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.976 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:39.976 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615692+0000", 2024-09-16T20:51:39.976 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.6977e-05 2024-09-16T20:51:39.976 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.976 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.977 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:39.977 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615701+0000", 2024-09-16T20:51:39.977 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6119999999999999e-06 2024-09-16T20:51:39.977 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.977 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.977 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:39.977 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615782+0000", 2024-09-16T20:51:39.977 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1804000000000005e-05 2024-09-16T20:51:39.977 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.977 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.977 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:39.977 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615795+0000", 2024-09-16T20:51:39.978 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2464e-05 2024-09-16T20:51:39.978 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.978 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.978 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:39.978 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.580071+0000", 2024-09-16T20:51:39.978 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.96427590100000005 2024-09-16T20:51:39.978 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.978 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.978 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:39.978 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.580247+0000", 2024-09-16T20:51:39.978 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000175754 2024-09-16T20:51:39.978 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.978 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.979 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 2", 2024-09-16T20:51:39.979 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.580436+0000", 2024-09-16T20:51:39.979 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00018945299999999999 2024-09-16T20:51:39.979 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.979 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.979 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:39.979 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.581045+0000", 2024-09-16T20:51:39.979 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00060937600000000001 2024-09-16T20:51:39.979 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.979 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.979 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:39.979 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.583883+0000", 2024-09-16T20:51:39.979 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0028372900000000001 2024-09-16T20:51:39.979 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.980 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.980 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:39.980 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.583944+0000", 2024-09-16T20:51:39.980 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1848999999999997e-05 2024-09-16T20:51:39.980 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.980 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.980 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:39.980 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.583984+0000", 2024-09-16T20:51:39.980 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9171999999999997e-05 2024-09-16T20:51:39.980 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:39.980 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:39.980 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:39.980 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.980 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.980 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:30201 3.f 3:f6903a53:::rbd_object_map.110eca9b190a.000000000000001a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e177)", 2024-09-16T20:51:39.981 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:51:04.603586+0000", 2024-09-16T20:51:39.981 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 35.279530973999996, 2024-09-16T20:51:39.981 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.98028146800000004, 2024-09-16T20:51:39.981 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:39.981 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:39.981 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:39.981 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:39.981 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:51:39.981 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:39.981 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 30201 2024-09-16T20:51:39.981 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.981 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:39.981 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.981 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:39.982 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603586+0000", 2024-09-16T20:51:39.982 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:39.982 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.982 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.982 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:39.982 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603586+0000", 2024-09-16T20:51:39.982 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:39.982 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.982 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.982 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:39.982 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603587+0000", 2024-09-16T20:51:39.982 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.519e-06 2024-09-16T20:51:39.982 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.982 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.982 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:39.983 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603593+0000", 2024-09-16T20:51:39.983 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1449999999999996e-06 2024-09-16T20:51:39.983 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.983 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.983 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:39.983 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603606+0000", 2024-09-16T20:51:39.983 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2797e-05 2024-09-16T20:51:39.983 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.983 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.983 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:39.983 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603612+0000", 2024-09-16T20:51:39.983 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8710000000000002e-06 2024-09-16T20:51:39.983 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.983 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.984 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:39.984 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603675+0000", 2024-09-16T20:51:39.984 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2728e-05 2024-09-16T20:51:39.984 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.984 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.984 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:39.984 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603687+0000", 2024-09-16T20:51:39.984 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.253e-05 2024-09-16T20:51:39.984 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.984 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.984 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:39.984 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:05.583358+0000", 2024-09-16T20:51:39.984 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.97967074600000004 2024-09-16T20:51:39.984 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.985 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.985 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:39.985 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:05.583515+0000", 2024-09-16T20:51:39.985 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00015755700000000001 2024-09-16T20:51:39.985 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:39.985 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:39.985 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:39.985 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:05.583867+0000", 2024-09-16T20:51:39.985 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00035157500000000002 2024-09-16T20:51:39.985 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:39.985 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:39.985 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:39.985 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:39.985 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:39.985 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:39.987 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:51:39.987 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:39.986+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:40.087 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:51:40.087+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:51:40.094 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:40.094 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:51:40.094 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:51:40.094 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:40.103 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:51:40.188 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:40.187+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:40.210 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:40.210 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:51:40.210 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:51:40.210 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:51:40.210 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:40.219 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:51:40.303 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:40.302+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:40.368 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:40.369 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:51:40.369 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:51:40.369 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:51:40.369 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.369 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:51:40.369 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:51:40.369 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 227.11258537399999, 2024-09-16T20:51:40.369 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67502346999999996, 2024-09-16T20:51:40.369 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:40.369 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:40.369 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:40.369 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:40.370 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:51:40.370 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:51:40.370 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 1 2024-09-16T20:51:40.370 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.370 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:40.370 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.370 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:40.370 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:51:40.370 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.370 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.370 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.370 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:40.371 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:51:40.371 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.371 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.371 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.371 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:40.371 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254128+0000", 2024-09-16T20:51:40.371 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.463e-06 2024-09-16T20:51:40.371 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.371 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.371 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:40.371 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254135+0000", 2024-09-16T20:51:40.371 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1579999999999999e-06 2024-09-16T20:51:40.371 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.371 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.371 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:40.372 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254146+0000", 2024-09-16T20:51:40.372 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1341e-05 2024-09-16T20:51:40.372 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.372 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.372 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:40.372 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254155+0000", 2024-09-16T20:51:40.372 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.1549999999999996e-06 2024-09-16T20:51:40.372 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.372 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.372 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:40.372 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254221+0000", 2024-09-16T20:51:40.372 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5470999999999997e-05 2024-09-16T20:51:40.372 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.372 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.373 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:40.373 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254231+0000", 2024-09-16T20:51:40.373 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0592e-05 2024-09-16T20:51:40.373 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.373 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.373 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:40.373 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926711+0000", 2024-09-16T20:51:40.373 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67247946599999997 2024-09-16T20:51:40.373 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.373 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.373 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:40.373 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926910+0000", 2024-09-16T20:51:40.373 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019934 2024-09-16T20:51:40.373 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.373 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.374 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:51:40.374 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.927159+0000", 2024-09-16T20:51:40.374 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00024876599999999998 2024-09-16T20:51:40.374 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.374 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.374 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:40.374 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.928797+0000", 2024-09-16T20:51:40.374 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001638308 2024-09-16T20:51:40.374 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.374 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.374 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:40.374 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929026+0000", 2024-09-16T20:51:40.374 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00022876099999999999 2024-09-16T20:51:40.374 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.374 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.375 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:40.375 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929102+0000", 2024-09-16T20:51:40.375 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6184000000000001e-05 2024-09-16T20:51:40.375 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.375 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.375 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:40.375 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929150+0000", 2024-09-16T20:51:40.375 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7465e-05 2024-09-16T20:51:40.375 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.375 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:40.375 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.375 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.375 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.375 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3574 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:40.376 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:51:40.376 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 170.885421842, 2024-09-16T20:51:40.376 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053579441999999998, 2024-09-16T20:51:40.376 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:40.376 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:40.376 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:40.376 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:40.376 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:40.376 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:40.376 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3574 2024-09-16T20:51:40.376 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.376 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:40.376 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.376 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:40.377 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:51:40.377 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.377 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.377 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.377 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:40.377 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:51:40.377 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.377 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.377 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.377 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:40.377 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:51:40.377 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2e-07 2024-09-16T20:51:40.377 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.377 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.378 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:40.378 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481292+0000", 2024-09-16T20:51:40.378 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2630000000000002e-06 2024-09-16T20:51:40.378 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.378 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.378 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:40.378 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481297+0000", 2024-09-16T20:51:40.378 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.296e-06 2024-09-16T20:51:40.378 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.378 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.378 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:40.378 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481300+0000", 2024-09-16T20:51:40.378 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7019999999999999e-06 2024-09-16T20:51:40.378 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.378 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.379 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:40.379 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525533+0000", 2024-09-16T20:51:40.379 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.044232318 2024-09-16T20:51:40.379 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.379 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.379 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:40.379 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525619+0000", 2024-09-16T20:51:40.379 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5892999999999999e-05 2024-09-16T20:51:40.379 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.379 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.379 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:40.379 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525987+0000", 2024-09-16T20:51:40.379 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00036836099999999999 2024-09-16T20:51:40.379 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.380 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.380 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:40.380 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531779+0000", 2024-09-16T20:51:40.380 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0057918240000000001 2024-09-16T20:51:40.380 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.380 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.380 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:40.380 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534747+0000", 2024-09-16T20:51:40.380 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0029681600000000001 2024-09-16T20:51:40.380 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.380 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.380 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:40.380 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534832+0000", 2024-09-16T20:51:40.380 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5180999999999996e-05 2024-09-16T20:51:40.381 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.381 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.381 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:40.381 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534869+0000", 2024-09-16T20:51:40.381 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7024000000000003e-05 2024-09-16T20:51:40.381 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.381 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:40.381 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.381 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.381 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.381 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3575 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:40.381 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:51:40.381 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 170.88540710199999, 2024-09-16T20:51:40.382 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053801463000000001, 2024-09-16T20:51:40.382 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:40.382 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:40.382 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:40.382 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:40.382 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:40.382 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:40.382 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3575 2024-09-16T20:51:40.382 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.382 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:40.382 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.382 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:40.382 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:51:40.382 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.383 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.383 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.383 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:40.383 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:51:40.383 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.383 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.383 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.383 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:40.383 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481305+0000", 2024-09-16T20:51:40.383 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5400000000000002e-07 2024-09-16T20:51:40.383 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.383 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.383 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:40.383 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481307+0000", 2024-09-16T20:51:40.384 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7800000000000001e-06 2024-09-16T20:51:40.384 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.384 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.384 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:40.384 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481312+0000", 2024-09-16T20:51:40.384 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6349999999999997e-06 2024-09-16T20:51:40.384 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.384 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.384 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:40.384 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481315+0000", 2024-09-16T20:51:40.384 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7829999999999999e-06 2024-09-16T20:51:40.384 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.384 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.384 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:40.384 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.526876+0000", 2024-09-16T20:51:40.385 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.045561070000000002 2024-09-16T20:51:40.385 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.385 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.385 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:40.385 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.526954+0000", 2024-09-16T20:51:40.385 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8108000000000005e-05 2024-09-16T20:51:40.385 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.385 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.385 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:40.385 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527282+0000", 2024-09-16T20:51:40.385 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000327483 2024-09-16T20:51:40.385 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.385 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.385 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:40.386 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531784+0000", 2024-09-16T20:51:40.386 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0045020169999999997 2024-09-16T20:51:40.386 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.386 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.386 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:40.386 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534921+0000", 2024-09-16T20:51:40.386 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.003137791 2024-09-16T20:51:40.386 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.386 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.386 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:40.386 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534976+0000", 2024-09-16T20:51:40.386 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4953000000000001e-05 2024-09-16T20:51:40.386 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.387 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.387 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:40.387 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535106+0000", 2024-09-16T20:51:40.387 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000129589 2024-09-16T20:51:40.387 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.387 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:40.387 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.387 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.388 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.388 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3576 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:40.388 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:51:40.388 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 170.88539319, 2024-09-16T20:51:40.388 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053955436000000002, 2024-09-16T20:51:40.389 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:40.389 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:40.389 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:40.389 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:40.389 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:40.389 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:40.389 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3576 2024-09-16T20:51:40.389 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.389 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:40.389 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.389 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:40.389 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:51:40.390 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.390 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.390 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.390 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:40.390 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:51:40.390 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.390 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.390 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.390 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:40.390 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481319+0000", 2024-09-16T20:51:40.390 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.53e-07 2024-09-16T20:51:40.390 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.391 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.391 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:40.391 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481320+0000", 2024-09-16T20:51:40.391 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5230000000000001e-06 2024-09-16T20:51:40.391 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.391 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.391 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:40.391 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481330+0000", 2024-09-16T20:51:40.391 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0190000000000001e-05 2024-09-16T20:51:40.391 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.391 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.391 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:40.391 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481333+0000", 2024-09-16T20:51:40.391 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.151e-06 2024-09-16T20:51:40.392 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.392 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.392 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:40.392 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527689+0000", 2024-09-16T20:51:40.392 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046355192000000003 2024-09-16T20:51:40.392 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.392 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.392 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:40.392 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527764+0000", 2024-09-16T20:51:40.392 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5776000000000001e-05 2024-09-16T20:51:40.392 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.392 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.392 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:40.392 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528141+0000", 2024-09-16T20:51:40.393 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00037647599999999999 2024-09-16T20:51:40.393 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.393 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.393 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:40.393 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531788+0000", 2024-09-16T20:51:40.393 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0036470679999999998 2024-09-16T20:51:40.393 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.393 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.393 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:40.393 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535174+0000", 2024-09-16T20:51:40.393 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0033862670000000001 2024-09-16T20:51:40.393 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.393 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.394 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:40.394 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535236+0000", 2024-09-16T20:51:40.394 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1797999999999993e-05 2024-09-16T20:51:40.394 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.394 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.394 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:40.394 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535274+0000", 2024-09-16T20:51:40.394 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7741999999999999e-05 2024-09-16T20:51:40.394 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.394 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:40.394 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.394 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.394 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.394 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3577 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:40.395 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:51:40.395 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 170.88537159000001, 2024-09-16T20:51:40.395 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054073807000000002, 2024-09-16T20:51:40.395 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:40.395 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:40.395 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:40.395 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:40.395 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:40.395 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:40.395 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3577 2024-09-16T20:51:40.395 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.395 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:40.395 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.395 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:40.395 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:51:40.396 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.396 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.396 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.396 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:40.396 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:51:40.396 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.396 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.396 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.396 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:40.396 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:51:40.396 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.96e-07 2024-09-16T20:51:40.396 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.396 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.397 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:40.397 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481342+0000", 2024-09-16T20:51:40.397 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.714e-06 2024-09-16T20:51:40.397 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.397 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.397 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:40.397 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481349+0000", 2024-09-16T20:51:40.397 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4930000000000001e-06 2024-09-16T20:51:40.397 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.397 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.397 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:40.398 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481353+0000", 2024-09-16T20:51:40.398 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4879999999999999e-06 2024-09-16T20:51:40.398 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.398 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.398 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:40.398 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528619+0000", 2024-09-16T20:51:40.398 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047266351999999998 2024-09-16T20:51:40.398 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.398 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.398 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:40.398 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528706+0000", 2024-09-16T20:51:40.398 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6796e-05 2024-09-16T20:51:40.398 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.399 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.399 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:40.399 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529148+0000", 2024-09-16T20:51:40.399 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00044182999999999999 2024-09-16T20:51:40.399 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.399 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.399 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:40.399 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531792+0000", 2024-09-16T20:51:40.399 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0026443119999999998 2024-09-16T20:51:40.399 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.399 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.399 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:40.400 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535319+0000", 2024-09-16T20:51:40.400 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0035266439999999998 2024-09-16T20:51:40.400 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.400 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.400 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:40.400 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535372+0000", 2024-09-16T20:51:40.400 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3207000000000001e-05 2024-09-16T20:51:40.400 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.400 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.400 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:40.400 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535414+0000", 2024-09-16T20:51:40.400 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1675000000000003e-05 2024-09-16T20:51:40.400 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.401 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:40.401 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.401 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.401 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.401 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3578 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:40.401 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:51:40.401 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 170.885350421, 2024-09-16T20:51:40.401 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054189586999999997, 2024-09-16T20:51:40.401 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:40.401 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:40.401 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:40.401 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:40.401 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:40.401 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:40.402 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3578 2024-09-16T20:51:40.402 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.402 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:40.402 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.402 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:40.402 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:51:40.402 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.402 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.402 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.402 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:40.402 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:51:40.402 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.402 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.402 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.403 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:40.403 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:51:40.403 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0800000000000001e-07 2024-09-16T20:51:40.403 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.403 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.403 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:40.403 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481363+0000", 2024-09-16T20:51:40.403 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0339999999999999e-06 2024-09-16T20:51:40.403 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.403 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.403 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:40.403 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481371+0000", 2024-09-16T20:51:40.403 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4440000000000002e-06 2024-09-16T20:51:40.403 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.404 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.404 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:40.404 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481374+0000", 2024-09-16T20:51:40.404 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2049999999999998e-06 2024-09-16T20:51:40.404 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.404 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.404 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:40.404 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529555+0000", 2024-09-16T20:51:40.404 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048181202999999999 2024-09-16T20:51:40.404 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.404 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.404 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:40.405 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529616+0000", 2024-09-16T20:51:40.405 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0912000000000002e-05 2024-09-16T20:51:40.405 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.405 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.405 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:40.405 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529939+0000", 2024-09-16T20:51:40.405 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00032269800000000001 2024-09-16T20:51:40.405 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.405 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.405 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:40.405 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532602+0000", 2024-09-16T20:51:40.405 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0026634940000000002 2024-09-16T20:51:40.406 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.406 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.406 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:40.406 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535465+0000", 2024-09-16T20:51:40.406 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.002862149 2024-09-16T20:51:40.406 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.406 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.406 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:40.406 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535516+0000", 2024-09-16T20:51:40.406 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1696000000000001e-05 2024-09-16T20:51:40.406 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.406 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.407 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:40.407 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535551+0000", 2024-09-16T20:51:40.407 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4443999999999998e-05 2024-09-16T20:51:40.407 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.407 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:40.407 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.407 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.407 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.407 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3579 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:40.407 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:51:40.407 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 170.885331982, 2024-09-16T20:51:40.407 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054295151, 2024-09-16T20:51:40.408 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:40.408 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:40.408 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:40.408 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:40.408 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:40.408 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:40.408 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3579 2024-09-16T20:51:40.408 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.408 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:40.408 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.408 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:40.408 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:51:40.408 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.409 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.409 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.409 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:40.409 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:51:40.409 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.409 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.409 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.409 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:40.409 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:51:40.409 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3300000000000001e-07 2024-09-16T20:51:40.409 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.409 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.409 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:40.409 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481381+0000", 2024-09-16T20:51:40.410 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.649e-06 2024-09-16T20:51:40.410 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.410 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.410 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:40.410 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481387+0000", 2024-09-16T20:51:40.410 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4519999999999996e-06 2024-09-16T20:51:40.410 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.410 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.410 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:40.411 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481390+0000", 2024-09-16T20:51:40.411 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0170000000000001e-06 2024-09-16T20:51:40.411 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.411 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.411 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:40.411 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530756+0000", 2024-09-16T20:51:40.411 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049365882999999999 2024-09-16T20:51:40.411 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.411 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.411 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:40.411 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530845+0000", 2024-09-16T20:51:40.411 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9197000000000007e-05 2024-09-16T20:51:40.411 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.411 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.411 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:40.411 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531191+0000", 2024-09-16T20:51:40.412 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00034562500000000001 2024-09-16T20:51:40.412 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.412 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.412 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:40.412 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534078+0000", 2024-09-16T20:51:40.412 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0028878179999999999 2024-09-16T20:51:40.412 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.412 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.412 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:40.412 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535589+0000", 2024-09-16T20:51:40.412 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00151085 2024-09-16T20:51:40.412 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.412 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.412 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:40.413 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535639+0000", 2024-09-16T20:51:40.413 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9697000000000002e-05 2024-09-16T20:51:40.413 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.413 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.413 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:40.413 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535675+0000", 2024-09-16T20:51:40.413 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5729999999999998e-05 2024-09-16T20:51:40.413 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.413 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:40.413 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.413 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.413 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.413 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3580 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:40.413 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:51:40.414 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 170.88531835000001, 2024-09-16T20:51:40.414 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054397361999999998, 2024-09-16T20:51:40.414 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:40.414 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:40.414 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:40.414 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:40.414 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:40.414 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:40.414 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3580 2024-09-16T20:51:40.414 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.414 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:40.414 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.414 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:40.414 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:51:40.414 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.415 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.415 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.415 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:40.415 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:51:40.415 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.415 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.415 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.415 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:40.415 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:51:40.415 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2600000000000001e-07 2024-09-16T20:51:40.415 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.415 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.415 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:40.416 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481395+0000", 2024-09-16T20:51:40.416 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.544e-06 2024-09-16T20:51:40.416 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.416 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.416 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:40.416 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481404+0000", 2024-09-16T20:51:40.416 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8850000000000005e-06 2024-09-16T20:51:40.416 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.416 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.416 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:40.416 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481407+0000", 2024-09-16T20:51:40.416 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1540000000000002e-06 2024-09-16T20:51:40.416 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.417 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.417 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:40.417 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531823+0000", 2024-09-16T20:51:40.417 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050415637999999999 2024-09-16T20:51:40.417 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.417 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.417 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:40.417 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531895+0000", 2024-09-16T20:51:40.417 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2179000000000001e-05 2024-09-16T20:51:40.417 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.417 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.417 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:40.417 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532236+0000", 2024-09-16T20:51:40.418 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00034073799999999999 2024-09-16T20:51:40.418 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.418 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.418 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:40.418 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534083+0000", 2024-09-16T20:51:40.418 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001847298 2024-09-16T20:51:40.418 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.418 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.418 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:40.418 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535714+0000", 2024-09-16T20:51:40.418 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0016316810000000001 2024-09-16T20:51:40.418 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.418 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.419 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:40.419 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535759+0000", 2024-09-16T20:51:40.419 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4478999999999999e-05 2024-09-16T20:51:40.419 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.419 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.419 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:40.419 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535791+0000", 2024-09-16T20:51:40.419 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.154e-05 2024-09-16T20:51:40.419 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.419 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:40.419 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.419 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.420 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.420 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3581 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:40.420 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:51:40.420 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 170.885296353, 2024-09-16T20:51:40.420 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.057094403000000002, 2024-09-16T20:51:40.420 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:40.420 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:40.420 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:40.420 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:40.420 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:40.420 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:40.420 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3581 2024-09-16T20:51:40.420 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.420 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:40.420 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.421 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:40.421 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:51:40.421 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.421 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.421 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.421 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:40.421 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:51:40.421 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.421 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.421 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.421 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:40.421 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:51:40.421 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8799999999999998e-07 2024-09-16T20:51:40.421 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.422 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.422 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:40.422 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481418+0000", 2024-09-16T20:51:40.422 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1859999999999999e-06 2024-09-16T20:51:40.422 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.422 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.422 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:40.422 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481423+0000", 2024-09-16T20:51:40.422 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8279999999999999e-06 2024-09-16T20:51:40.422 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.422 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.422 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:40.422 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481426+0000", 2024-09-16T20:51:40.422 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0110000000000001e-06 2024-09-16T20:51:40.423 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.423 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.423 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:40.423 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532631+0000", 2024-09-16T20:51:40.423 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051204514999999999 2024-09-16T20:51:40.423 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.423 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.423 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:40.423 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.533381+0000", 2024-09-16T20:51:40.423 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00075029000000000003 2024-09-16T20:51:40.423 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.423 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.423 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:40.424 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.533739+0000", 2024-09-16T20:51:40.424 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00035773100000000002 2024-09-16T20:51:40.424 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.424 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.424 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:40.424 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535123+0000", 2024-09-16T20:51:40.424 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0013836160000000001 2024-09-16T20:51:40.424 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.424 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.424 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:40.424 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538413+0000", 2024-09-16T20:51:40.424 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0032902510000000001 2024-09-16T20:51:40.424 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.425 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.425 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:40.425 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538470+0000", 2024-09-16T20:51:40.425 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6697999999999998e-05 2024-09-16T20:51:40.425 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.425 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.425 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:40.425 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538510+0000", 2024-09-16T20:51:40.425 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9988999999999999e-05 2024-09-16T20:51:40.425 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.425 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:40.425 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.425 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.426 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.426 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:12732 3.d 3:b752444a:::rbd_header.110e747f0779:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-16T20:51:40.426 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:51:40.426 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 155.59822377500001, 2024-09-16T20:51:40.426 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.065302467000000003, 2024-09-16T20:51:40.426 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:40.426 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:40.426 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:40.426 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:40.426 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:51:40.426 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:51:40.426 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 12732 2024-09-16T20:51:40.426 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.427 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:40.427 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.427 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:40.427 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:51:40.427 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.427 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.427 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.427 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:40.427 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:51:40.427 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.427 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.427 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.427 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:40.428 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768490+0000", 2024-09-16T20:51:40.428 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2859999999999998e-06 2024-09-16T20:51:40.428 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.428 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.428 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:40.428 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768501+0000", 2024-09-16T20:51:40.428 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1226999999999999e-05 2024-09-16T20:51:40.428 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.428 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.428 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:40.428 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768517+0000", 2024-09-16T20:51:40.428 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.592e-05 2024-09-16T20:51:40.428 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.428 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.429 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:40.429 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768528+0000", 2024-09-16T20:51:40.429 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0689999999999999e-05 2024-09-16T20:51:40.429 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.429 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.429 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:40.429 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768584+0000", 2024-09-16T20:51:40.429 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5889e-05 2024-09-16T20:51:40.429 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.429 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.429 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:40.429 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768720+0000", 2024-09-16T20:51:40.429 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000136718 2024-09-16T20:51:40.430 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.430 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.430 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:40.430 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.769066+0000", 2024-09-16T20:51:40.430 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000346013 2024-09-16T20:51:40.430 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.430 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.430 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:40.430 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833659+0000", 2024-09-16T20:51:40.430 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.064592822999999994 2024-09-16T20:51:40.430 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.430 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.430 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:40.431 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833719+0000", 2024-09-16T20:51:40.431 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9441000000000002e-05 2024-09-16T20:51:40.431 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.431 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.431 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:40.431 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833758+0000", 2024-09-16T20:51:40.431 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9122000000000003e-05 2024-09-16T20:51:40.431 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.431 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.431 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:40.431 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833790+0000", 2024-09-16T20:51:40.431 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2338000000000001e-05 2024-09-16T20:51:40.431 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.431 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:40.432 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.432 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.432 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.432 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.5089.0:12732 3.d e73/23 ondisk, result = 0)", 2024-09-16T20:51:40.432 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:51:40.432 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 155.596621605, 2024-09-16T20:51:40.432 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.063706456999999994, 2024-09-16T20:51:40.432 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:40.432 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:40.432 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:40.432 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:40.432 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.432 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:40.433 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:51:40.433 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.433 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.433 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.433 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:40.433 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:51:40.433 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.433 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.433 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.433 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:40.433 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770091+0000", 2024-09-16T20:51:40.433 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.53e-06 2024-09-16T20:51:40.433 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.434 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.434 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:40.434 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770104+0000", 2024-09-16T20:51:40.434 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2203e-05 2024-09-16T20:51:40.434 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.434 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.434 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:40.434 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833637+0000", 2024-09-16T20:51:40.434 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.063533201999999997 2024-09-16T20:51:40.434 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.434 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.434 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:40.434 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833655+0000", 2024-09-16T20:51:40.434 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8369e-05 2024-09-16T20:51:40.435 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.435 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.435 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:40.435 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833707+0000", 2024-09-16T20:51:40.435 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2061999999999997e-05 2024-09-16T20:51:40.435 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.435 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.435 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:40.435 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833715+0000", 2024-09-16T20:51:40.435 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5179999999999998e-06 2024-09-16T20:51:40.435 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.435 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.435 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:40.435 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833796+0000", 2024-09-16T20:51:40.436 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1573e-05 2024-09-16T20:51:40.436 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.436 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:40.436 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.436 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.436 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.436 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3286 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 245760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:40.436 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:51:40.436 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 110.325559668, 2024-09-16T20:51:40.436 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052523564000000002, 2024-09-16T20:51:40.436 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:40.436 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:40.436 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:40.437 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:40.437 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:51:40.437 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:40.437 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3286 2024-09-16T20:51:40.437 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.437 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:40.437 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.437 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:40.437 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:51:40.437 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.437 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.437 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.437 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:40.438 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:51:40.438 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.438 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.438 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.438 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:40.438 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:51:40.438 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3900000000000001e-07 2024-09-16T20:51:40.438 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.438 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.438 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:40.438 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041153+0000", 2024-09-16T20:51:40.438 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.119e-06 2024-09-16T20:51:40.438 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.439 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.439 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:40.439 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041156+0000", 2024-09-16T20:51:40.439 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0000000000000001e-06 2024-09-16T20:51:40.439 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.439 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.439 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:40.439 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041159+0000", 2024-09-16T20:51:40.439 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5859999999999999e-06 2024-09-16T20:51:40.439 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.439 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.439 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:40.439 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093529+0000", 2024-09-16T20:51:40.439 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052370591000000001 2024-09-16T20:51:40.440 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.440 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.440 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:40.440 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093570+0000", 2024-09-16T20:51:40.440 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0161999999999999e-05 2024-09-16T20:51:40.440 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.440 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.440 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:40.440 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093675+0000", 2024-09-16T20:51:40.440 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010586699999999999 2024-09-16T20:51:40.440 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.440 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:40.440 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.441 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.441 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.441 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3287 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 250880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:40.441 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:51:40.441 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 110.325549619, 2024-09-16T20:51:40.441 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052781203999999998, 2024-09-16T20:51:40.441 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:40.441 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:40.441 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:40.441 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:40.441 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:51:40.441 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:40.441 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3287 2024-09-16T20:51:40.442 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.442 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:40.442 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.442 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:40.442 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:51:40.442 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.442 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.442 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.442 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:40.442 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:51:40.442 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.442 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.442 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.442 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:40.442 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:51:40.443 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1e-07 2024-09-16T20:51:40.443 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.443 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.443 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:40.443 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041163+0000", 2024-09-16T20:51:40.443 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0690000000000001e-06 2024-09-16T20:51:40.443 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.443 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.443 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:40.443 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041166+0000", 2024-09-16T20:51:40.443 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.982e-06 2024-09-16T20:51:40.443 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.443 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.443 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:40.443 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041169+0000", 2024-09-16T20:51:40.443 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.593e-06 2024-09-16T20:51:40.444 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.444 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.444 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:40.444 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093709+0000", 2024-09-16T20:51:40.444 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052540492000000001 2024-09-16T20:51:40.444 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.444 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.444 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:40.444 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093792+0000", 2024-09-16T20:51:40.444 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2436999999999997e-05 2024-09-16T20:51:40.444 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.444 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.444 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:40.444 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093943+0000", 2024-09-16T20:51:40.444 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000151421 2024-09-16T20:51:40.445 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.445 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:40.445 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.445 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.445 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.445 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3288 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 256000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:40.445 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:51:40.445 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 110.325539746, 2024-09-16T20:51:40.445 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052928269, 2024-09-16T20:51:40.445 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:40.445 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:40.445 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:40.445 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:40.445 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:51:40.446 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:40.446 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3288 2024-09-16T20:51:40.446 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.446 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:40.446 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.446 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:40.446 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:51:40.446 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.446 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.446 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.446 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:40.446 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:51:40.446 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.446 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.446 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.446 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:40.447 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:51:40.447 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2700000000000001e-07 2024-09-16T20:51:40.447 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.447 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.447 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:40.447 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041173+0000", 2024-09-16T20:51:40.447 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.113e-06 2024-09-16T20:51:40.447 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.447 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.447 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:40.447 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041176+0000", 2024-09-16T20:51:40.447 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9799999999999998e-06 2024-09-16T20:51:40.447 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.447 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.448 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:40.448 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041179+0000", 2024-09-16T20:51:40.448 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.667e-06 2024-09-16T20:51:40.448 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.448 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.448 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:40.448 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093957+0000", 2024-09-16T20:51:40.448 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052778703000000003 2024-09-16T20:51:40.448 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.448 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.448 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:40.448 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093999+0000", 2024-09-16T20:51:40.448 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1300000000000001e-05 2024-09-16T20:51:40.449 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.449 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.449 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:40.449 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094100+0000", 2024-09-16T20:51:40.449 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101279 2024-09-16T20:51:40.449 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.449 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:40.449 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.449 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.449 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.449 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3289 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 261120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:40.449 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:51:40.449 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 110.32552975, 2024-09-16T20:51:40.449 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053182976, 2024-09-16T20:51:40.450 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:40.450 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:40.450 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:40.450 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:40.450 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:51:40.450 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:40.450 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3289 2024-09-16T20:51:40.450 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.450 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:40.450 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.450 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:40.450 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:51:40.450 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.450 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.451 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.451 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:40.451 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:51:40.451 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.451 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.451 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.451 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:40.451 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:51:40.451 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2999999999999999e-07 2024-09-16T20:51:40.451 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.451 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.451 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:40.451 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041183+0000", 2024-09-16T20:51:40.451 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0720000000000001e-06 2024-09-16T20:51:40.451 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.452 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.452 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:40.452 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041186+0000", 2024-09-16T20:51:40.452 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9869999999999999e-06 2024-09-16T20:51:40.452 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.452 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.452 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:40.452 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041189+0000", 2024-09-16T20:51:40.452 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.587e-06 2024-09-16T20:51:40.452 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.452 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.452 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:40.452 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094136+0000", 2024-09-16T20:51:40.452 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052947270999999997 2024-09-16T20:51:40.453 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.453 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.453 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:40.453 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094213+0000", 2024-09-16T20:51:40.453 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7552000000000006e-05 2024-09-16T20:51:40.453 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.453 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.453 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:40.453 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094365+0000", 2024-09-16T20:51:40.453 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00015127700000000001 2024-09-16T20:51:40.453 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.453 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:40.453 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.453 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.453 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.454 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:2196 3.d 3:b1fe01a0:::rbd_data.110e7ef3004.0000000000000000:2d [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e136)", 2024-09-16T20:51:40.454 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:22.739057+0000", 2024-09-16T20:51:40.454 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 77.627654950999997, 2024-09-16T20:51:40.454 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11999464899999999, 2024-09-16T20:51:40.454 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:40.454 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:40.454 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:40.454 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:40.454 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:40.454 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:40.454 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2196 2024-09-16T20:51:40.454 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.454 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:40.454 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.454 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:40.454 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.739057+0000", 2024-09-16T20:51:40.455 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.455 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.455 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.455 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:40.455 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.739057+0000", 2024-09-16T20:51:40.455 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.455 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.455 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.455 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:40.455 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.739057+0000", 2024-09-16T20:51:40.455 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1399999999999997e-07 2024-09-16T20:51:40.455 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.455 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.455 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:40.456 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.739060+0000", 2024-09-16T20:51:40.456 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6019999999999998e-06 2024-09-16T20:51:40.456 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.456 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.456 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:40.456 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.858755+0000", 2024-09-16T20:51:40.456 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.119695151 2024-09-16T20:51:40.456 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.456 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.456 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:40.456 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.858778+0000", 2024-09-16T20:51:40.456 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3184999999999999e-05 2024-09-16T20:51:40.456 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.456 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.457 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:40.457 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.858865+0000", 2024-09-16T20:51:40.457 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.7330000000000006e-05 2024-09-16T20:51:40.457 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.457 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.457 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:40.457 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.858952+0000", 2024-09-16T20:51:40.457 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6803000000000002e-05 2024-09-16T20:51:40.457 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.457 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.457 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:40.457 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.859051+0000", 2024-09-16T20:51:40.457 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8963999999999996e-05 2024-09-16T20:51:40.457 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.457 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:40.458 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.458 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.458 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.458 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6061.0:5401 3.13 3:c8554348:::rbd_data.110e16eafbd.0000000000000011:head [read 577536~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-16T20:51:40.458 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:27.761536+0000", 2024-09-16T20:51:40.458 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 72.605175138000007, 2024-09-16T20:51:40.458 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.003742712, 2024-09-16T20:51:40.458 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:40.458 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:40.458 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:51:40.458 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:40.458 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6061", 2024-09-16T20:51:40.458 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3930128440", 2024-09-16T20:51:40.458 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5401 2024-09-16T20:51:40.459 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.459 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:40.459 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.459 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:40.459 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761536+0000", 2024-09-16T20:51:40.459 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.459 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.459 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.459 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:40.459 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761536+0000", 2024-09-16T20:51:40.459 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.459 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.459 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.459 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:40.459 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761537+0000", 2024-09-16T20:51:40.460 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2900000000000003e-07 2024-09-16T20:51:40.460 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.460 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.460 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:40.460 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761540+0000", 2024-09-16T20:51:40.460 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2320000000000001e-06 2024-09-16T20:51:40.460 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.460 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.460 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:40.460 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761548+0000", 2024-09-16T20:51:40.460 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1289999999999996e-06 2024-09-16T20:51:40.460 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.460 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.460 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:40.460 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761554+0000", 2024-09-16T20:51:40.461 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0820000000000004e-06 2024-09-16T20:51:40.461 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.461 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.461 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:40.461 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761621+0000", 2024-09-16T20:51:40.461 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6266000000000006e-05 2024-09-16T20:51:40.461 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.461 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.461 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:40.461 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761626+0000", 2024-09-16T20:51:40.461 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.614e-06 2024-09-16T20:51:40.461 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.461 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.461 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:40.461 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:28.765163+0000", 2024-09-16T20:51:40.462 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.003536416 2024-09-16T20:51:40.462 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.462 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.462 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:40.462 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:28.765279+0000", 2024-09-16T20:51:40.462 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000116344 2024-09-16T20:51:40.462 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.462 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:40.462 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.462 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.462 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.462 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6061.0:5425 3.13 3:c8554348:::rbd_data.110e16eafbd.0000000000000011:head [read 602112~28672] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-16T20:51:40.462 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:27.767049+0000", 2024-09-16T20:51:40.462 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 72.599662479000003, 2024-09-16T20:51:40.462 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.99847897900000004, 2024-09-16T20:51:40.463 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:40.463 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:40.463 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:51:40.463 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:40.463 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6061", 2024-09-16T20:51:40.463 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3930128440", 2024-09-16T20:51:40.463 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5425 2024-09-16T20:51:40.463 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.463 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:40.463 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.463 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:40.463 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767049+0000", 2024-09-16T20:51:40.463 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.463 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.463 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.464 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:40.464 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767049+0000", 2024-09-16T20:51:40.464 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.464 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.464 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.464 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:40.464 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767050+0000", 2024-09-16T20:51:40.464 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6499999999999999e-07 2024-09-16T20:51:40.464 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.464 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.464 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:40.464 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767052+0000", 2024-09-16T20:51:40.464 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7350000000000001e-06 2024-09-16T20:51:40.464 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.464 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.465 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:40.465 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767060+0000", 2024-09-16T20:51:40.465 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5109999999999997e-06 2024-09-16T20:51:40.465 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.465 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.465 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:40.465 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767066+0000", 2024-09-16T20:51:40.465 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8139999999999997e-06 2024-09-16T20:51:40.465 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.465 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.465 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:40.465 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767128+0000", 2024-09-16T20:51:40.465 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2174999999999993e-05 2024-09-16T20:51:40.465 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.466 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.466 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:40.466 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767133+0000", 2024-09-16T20:51:40.466 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2140000000000004e-06 2024-09-16T20:51:40.466 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.466 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.466 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:40.466 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:28.765441+0000", 2024-09-16T20:51:40.466 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.99830808599999998 2024-09-16T20:51:40.466 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.466 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.466 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:40.466 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:28.765528+0000", 2024-09-16T20:51:40.466 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6879000000000006e-05 2024-09-16T20:51:40.466 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.467 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:40.467 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.467 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.467 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.467 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:25585 3.10 3:0c7c5b23:::rbd_header.110ebf0678ad:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:51:40.467 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:32.877132+0000", 2024-09-16T20:51:40.467 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 67.489579930000005, 2024-09-16T20:51:40.467 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054096699999999998, 2024-09-16T20:51:40.467 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:40.467 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:40.467 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:40.467 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:40.467 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:51:40.467 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:40.468 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 25585 2024-09-16T20:51:40.468 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.468 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:40.468 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.468 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:40.468 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.877132+0000", 2024-09-16T20:51:40.468 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.468 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.468 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.468 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:40.468 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.877132+0000", 2024-09-16T20:51:40.468 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.468 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.469 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.469 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:40.469 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931072+0000", 2024-09-16T20:51:40.469 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053939931000000003 2024-09-16T20:51:40.469 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.469 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.469 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:40.469 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931076+0000", 2024-09-16T20:51:40.469 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2860000000000001e-06 2024-09-16T20:51:40.469 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.469 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.469 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:40.470 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931085+0000", 2024-09-16T20:51:40.470 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.3370000000000006e-06 2024-09-16T20:51:40.470 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.470 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.470 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:40.470 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931092+0000", 2024-09-16T20:51:40.470 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9970000000000001e-06 2024-09-16T20:51:40.470 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.470 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.470 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:40.470 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931137+0000", 2024-09-16T20:51:40.470 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5160000000000001e-05 2024-09-16T20:51:40.471 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.471 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.471 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:40.471 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931178+0000", 2024-09-16T20:51:40.471 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0379000000000001e-05 2024-09-16T20:51:40.471 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.471 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.471 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:40.471 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931228+0000", 2024-09-16T20:51:40.471 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0609999999999998e-05 2024-09-16T20:51:40.471 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.471 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:40.471 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.471 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.472 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.472 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:26679 3.c 3:33e2f78f:::rbd_header.110e16eafbd:head [watch ping cookie 140213436013408 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e153)", 2024-09-16T20:51:40.472 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:41.440287+0000", 2024-09-16T20:51:40.472 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 58.926424054999998, 2024-09-16T20:51:40.472 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.13295633600000001, 2024-09-16T20:51:40.472 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:40.472 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:40.472 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:40.472 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:40.472 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:51:40.472 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:40.472 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 26679 2024-09-16T20:51:40.472 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.473 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:40.473 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.473 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:40.473 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440287+0000", 2024-09-16T20:51:40.473 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.473 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.473 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.473 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:40.473 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440287+0000", 2024-09-16T20:51:40.473 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.473 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.473 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.473 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:40.473 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440288+0000", 2024-09-16T20:51:40.473 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0900000000000001e-07 2024-09-16T20:51:40.474 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.474 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.474 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:40.474 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440292+0000", 2024-09-16T20:51:40.474 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3749999999999996e-06 2024-09-16T20:51:40.474 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.474 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.474 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:40.474 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440299+0000", 2024-09-16T20:51:40.474 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8249999999999999e-06 2024-09-16T20:51:40.474 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.474 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.474 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:40.474 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440307+0000", 2024-09-16T20:51:40.475 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4800000000000004e-06 2024-09-16T20:51:40.475 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.475 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.475 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:40.475 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440451+0000", 2024-09-16T20:51:40.475 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014414500000000001 2024-09-16T20:51:40.475 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.475 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.475 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:40.475 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440465+0000", 2024-09-16T20:51:40.475 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.402e-05 2024-09-16T20:51:40.475 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.475 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.475 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:40.475 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.573043+0000", 2024-09-16T20:51:40.476 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.13257855399999999 2024-09-16T20:51:40.476 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.476 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.476 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:40.476 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.573130+0000", 2024-09-16T20:51:40.476 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6168999999999994e-05 2024-09-16T20:51:40.476 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.476 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.476 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:40.476 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.573244+0000", 2024-09-16T20:51:40.476 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000114159 2024-09-16T20:51:40.476 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.476 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:40.476 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.477 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.477 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:40.477 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:40.478 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:51:40.479 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:40.477+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:40.579 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:40.578+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:40.627 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:40.627 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:51:40.627 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:51:40.627 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:40.635 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:51:40.657 INFO:teuthology.orchestra.run.smithi029.stderr:nodeep-scrub is unset 2024-09-16T20:51:40.680 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:40.679+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:40.754 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:40.754 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:51:40.754 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:51:40.755 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:51:40.755 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:40.766 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:51:40.810 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:40.809+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:40.876 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:40.877 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:51:40.877 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:51:40.877 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:51:40.877 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.877 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3714 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1480704~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:40.877 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:51:40.877 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 198.23588807100001, 2024-09-16T20:51:40.877 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050152552000000003, 2024-09-16T20:51:40.877 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:40.877 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:40.877 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:40.877 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:40.878 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:40.878 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:40.878 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3714 2024-09-16T20:51:40.878 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.878 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:40.878 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.878 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:40.878 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:51:40.878 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.878 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.878 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.878 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:40.879 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:51:40.879 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.879 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.879 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.879 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:40.879 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:51:40.879 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0499999999999999e-07 2024-09-16T20:51:40.879 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.879 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.879 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:40.879 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639007+0000", 2024-09-16T20:51:40.879 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5340000000000001e-06 2024-09-16T20:51:40.879 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.879 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.880 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:40.880 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639013+0000", 2024-09-16T20:51:40.880 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3260000000000003e-06 2024-09-16T20:51:40.880 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.880 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.880 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:40.880 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639018+0000", 2024-09-16T20:51:40.880 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2850000000000002e-06 2024-09-16T20:51:40.880 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.880 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.880 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:40.880 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689026+0000", 2024-09-16T20:51:40.881 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.05000798 2024-09-16T20:51:40.881 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.881 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.881 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:40.881 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689077+0000", 2024-09-16T20:51:40.881 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1063999999999998e-05 2024-09-16T20:51:40.881 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.881 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.881 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:40.881 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689158+0000", 2024-09-16T20:51:40.881 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1057999999999995e-05 2024-09-16T20:51:40.881 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.881 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:40.882 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.882 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.882 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.882 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3716 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1488896~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:40.882 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:51:40.882 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 198.235870261, 2024-09-16T20:51:40.882 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050280142, 2024-09-16T20:51:40.882 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:40.882 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:40.882 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:40.882 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:40.882 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:40.882 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:40.882 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3716 2024-09-16T20:51:40.883 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.883 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:40.883 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.883 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:40.883 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:51:40.883 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.883 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.883 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.883 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:40.883 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:51:40.883 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.883 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.883 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.884 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:40.884 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639024+0000", 2024-09-16T20:51:40.884 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9999999999999997e-07 2024-09-16T20:51:40.884 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.884 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.884 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:40.884 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639026+0000", 2024-09-16T20:51:40.884 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.781e-06 2024-09-16T20:51:40.884 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.884 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.884 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:40.884 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639032+0000", 2024-09-16T20:51:40.885 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7780000000000004e-06 2024-09-16T20:51:40.885 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.885 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.885 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:40.885 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639036+0000", 2024-09-16T20:51:40.885 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6909999999999996e-06 2024-09-16T20:51:40.885 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.885 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.885 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:40.885 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689191+0000", 2024-09-16T20:51:40.885 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050155284000000001 2024-09-16T20:51:40.885 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.885 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.885 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:40.885 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689242+0000", 2024-09-16T20:51:40.886 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0662999999999999e-05 2024-09-16T20:51:40.886 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.886 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.886 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:40.886 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689304+0000", 2024-09-16T20:51:40.886 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1345000000000003e-05 2024-09-16T20:51:40.886 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.886 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:40.886 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.886 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.886 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.886 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3719 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1497088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:40.886 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:51:40.886 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 198.235842983, 2024-09-16T20:51:40.887 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050399715999999997, 2024-09-16T20:51:40.887 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:40.887 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:40.887 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:40.887 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:40.887 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:40.887 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:40.887 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3719 2024-09-16T20:51:40.887 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.887 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:40.887 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.887 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:40.887 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:51:40.887 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.888 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.888 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.888 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:40.888 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:51:40.888 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.888 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.888 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.888 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:40.888 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:51:40.888 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9700000000000002e-07 2024-09-16T20:51:40.888 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.888 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.888 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:40.888 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639053+0000", 2024-09-16T20:51:40.889 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2900000000000001e-06 2024-09-16T20:51:40.889 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.889 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.889 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:40.889 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639060+0000", 2024-09-16T20:51:40.889 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7750000000000004e-06 2024-09-16T20:51:40.889 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.889 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.889 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:40.889 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639066+0000", 2024-09-16T20:51:40.889 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0639999999999999e-06 2024-09-16T20:51:40.889 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.889 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.890 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:40.890 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689337+0000", 2024-09-16T20:51:40.890 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050270784999999998 2024-09-16T20:51:40.890 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.890 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.890 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:40.890 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689392+0000", 2024-09-16T20:51:40.890 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4920999999999998e-05 2024-09-16T20:51:40.890 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.890 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.890 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:40.890 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689450+0000", 2024-09-16T20:51:40.890 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8483999999999998e-05 2024-09-16T20:51:40.890 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.890 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:40.891 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.891 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.891 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.891 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3721 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1505280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:40.891 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:51:40.891 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 198.235819851, 2024-09-16T20:51:40.891 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050513608000000002, 2024-09-16T20:51:40.891 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:40.891 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:40.891 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:40.891 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:40.891 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:40.891 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:40.891 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3721 2024-09-16T20:51:40.892 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.892 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:40.892 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.892 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:40.892 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:51:40.892 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.892 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.892 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.892 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:40.892 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:51:40.892 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.892 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.892 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.892 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:40.893 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:51:40.893 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7e-07 2024-09-16T20:51:40.893 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.893 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.893 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:40.893 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639076+0000", 2024-09-16T20:51:40.893 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8899999999999999e-06 2024-09-16T20:51:40.893 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.893 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.893 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:40.893 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639083+0000", 2024-09-16T20:51:40.893 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1210000000000001e-06 2024-09-16T20:51:40.893 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.893 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.893 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:40.894 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639089+0000", 2024-09-16T20:51:40.894 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.643e-06 2024-09-16T20:51:40.894 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.894 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.894 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:40.894 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689474+0000", 2024-09-16T20:51:40.894 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050385679000000003 2024-09-16T20:51:40.894 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.894 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.894 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:40.894 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689533+0000", 2024-09-16T20:51:40.894 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8303000000000001e-05 2024-09-16T20:51:40.894 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.894 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.895 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:40.895 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689587+0000", 2024-09-16T20:51:40.895 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4601999999999997e-05 2024-09-16T20:51:40.895 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.895 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:40.895 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.895 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.895 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.895 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3723 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1513472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:40.895 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:51:40.895 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 198.235789047, 2024-09-16T20:51:40.895 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050699691999999998, 2024-09-16T20:51:40.896 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:40.896 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:40.896 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:40.896 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:40.896 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:40.896 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:40.896 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3723 2024-09-16T20:51:40.896 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.896 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:40.896 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.896 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:40.896 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:51:40.897 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.897 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.897 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.897 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:40.897 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:51:40.897 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.897 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.897 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.897 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:40.897 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:51:40.897 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3399999999999999e-07 2024-09-16T20:51:40.897 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.897 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.897 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:40.898 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639107+0000", 2024-09-16T20:51:40.898 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3860000000000001e-06 2024-09-16T20:51:40.898 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.898 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.898 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:40.898 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639114+0000", 2024-09-16T20:51:40.898 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9800000000000001e-06 2024-09-16T20:51:40.898 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.898 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.898 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:40.898 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639129+0000", 2024-09-16T20:51:40.898 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4887e-05 2024-09-16T20:51:40.898 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.898 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.899 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:40.899 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689625+0000", 2024-09-16T20:51:40.899 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050495848000000003 2024-09-16T20:51:40.899 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.899 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.899 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:40.899 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689720+0000", 2024-09-16T20:51:40.899 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4821000000000006e-05 2024-09-16T20:51:40.899 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.899 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.899 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:40.899 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689804+0000", 2024-09-16T20:51:40.899 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4235999999999998e-05 2024-09-16T20:51:40.899 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.900 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:40.900 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.900 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.900 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.900 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3725 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1521664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:40.900 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:51:40.900 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 198.23575545099999, 2024-09-16T20:51:40.900 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050797660000000001, 2024-09-16T20:51:40.900 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:40.900 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:40.900 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:40.900 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:40.900 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:40.900 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:40.901 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3725 2024-09-16T20:51:40.901 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.901 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:40.901 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.901 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:40.901 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:51:40.901 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.901 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.901 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.901 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:40.901 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:51:40.901 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.901 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.901 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.902 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:40.902 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639139+0000", 2024-09-16T20:51:40.902 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8999999999999997e-07 2024-09-16T20:51:40.902 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.902 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.902 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:40.902 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639141+0000", 2024-09-16T20:51:40.902 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.182e-06 2024-09-16T20:51:40.902 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.902 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.902 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:40.902 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639154+0000", 2024-09-16T20:51:40.903 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3626e-05 2024-09-16T20:51:40.903 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.903 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.903 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:40.903 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639160+0000", 2024-09-16T20:51:40.903 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6810000000000003e-06 2024-09-16T20:51:40.903 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.903 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.903 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:40.903 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689831+0000", 2024-09-16T20:51:40.904 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050670907000000001 2024-09-16T20:51:40.904 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.904 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.904 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:40.904 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689882+0000", 2024-09-16T20:51:40.904 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0445999999999998e-05 2024-09-16T20:51:40.904 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.904 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.904 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:40.904 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689936+0000", 2024-09-16T20:51:40.904 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4327999999999999e-05 2024-09-16T20:51:40.904 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.905 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:40.905 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.905 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.905 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.905 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3727 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1529856~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:40.905 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:51:40.905 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 198.23571987400001, 2024-09-16T20:51:40.905 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051016922999999999, 2024-09-16T20:51:40.905 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:40.905 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:40.905 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:40.905 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:40.905 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:40.906 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:40.906 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3727 2024-09-16T20:51:40.906 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.906 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:40.906 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.906 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:40.906 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:51:40.906 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.906 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.906 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.906 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:40.906 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:51:40.906 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.907 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.907 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.907 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:40.907 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:51:40.907 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9999999999999998e-07 2024-09-16T20:51:40.907 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.907 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.907 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:40.907 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639183+0000", 2024-09-16T20:51:40.907 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2399999999999996e-06 2024-09-16T20:51:40.907 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.907 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.907 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:40.907 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639195+0000", 2024-09-16T20:51:40.908 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1113e-05 2024-09-16T20:51:40.908 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.908 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.908 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:40.908 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639206+0000", 2024-09-16T20:51:40.908 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1503e-05 2024-09-16T20:51:40.908 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.908 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.908 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:40.908 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689969+0000", 2024-09-16T20:51:40.908 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050762676 2024-09-16T20:51:40.908 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.908 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.909 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:40.909 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690071+0000", 2024-09-16T20:51:40.909 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102163 2024-09-16T20:51:40.909 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.909 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.909 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:40.909 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690191+0000", 2024-09-16T20:51:40.909 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000119728 2024-09-16T20:51:40.909 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.909 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:40.909 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.909 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.909 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.909 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3730 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1538048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:40.910 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:51:40.910 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 198.23567831099999, 2024-09-16T20:51:40.910 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051131467999999999, 2024-09-16T20:51:40.910 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:40.910 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:40.910 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:40.910 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:40.910 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:40.910 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:40.910 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3730 2024-09-16T20:51:40.910 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.910 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:40.911 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.911 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:40.911 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:51:40.911 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.911 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.911 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.911 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:40.911 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:51:40.911 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.912 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.912 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.912 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:40.912 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639216+0000", 2024-09-16T20:51:40.912 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.39e-07 2024-09-16T20:51:40.912 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.912 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.912 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:40.912 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639218+0000", 2024-09-16T20:51:40.912 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.537e-06 2024-09-16T20:51:40.912 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.912 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.913 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:40.913 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639229+0000", 2024-09-16T20:51:40.913 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1122e-05 2024-09-16T20:51:40.913 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.913 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.913 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:40.913 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639237+0000", 2024-09-16T20:51:40.913 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8509999999999999e-06 2024-09-16T20:51:40.913 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.913 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.913 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:40.913 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690227+0000", 2024-09-16T20:51:40.913 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050990147 2024-09-16T20:51:40.914 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.914 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.914 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:40.914 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690278+0000", 2024-09-16T20:51:40.914 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0223999999999999e-05 2024-09-16T20:51:40.914 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.914 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.914 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:40.914 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690347+0000", 2024-09-16T20:51:40.914 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9147999999999996e-05 2024-09-16T20:51:40.914 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.914 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:40.914 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.915 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.915 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.915 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3731 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1546240~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:40.915 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:51:40.915 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 198.23564760299999, 2024-09-16T20:51:40.915 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051276744999999999, 2024-09-16T20:51:40.915 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:40.915 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:40.915 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:40.915 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:40.915 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:40.915 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:40.915 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3731 2024-09-16T20:51:40.915 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.916 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:40.916 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.916 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:40.916 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:51:40.916 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.916 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.916 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.916 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:40.916 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:51:40.916 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.916 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.916 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.916 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:40.916 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:51:40.917 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9900000000000001e-07 2024-09-16T20:51:40.917 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.917 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.917 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:40.917 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639249+0000", 2024-09-16T20:51:40.917 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3630000000000001e-06 2024-09-16T20:51:40.917 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.917 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.917 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:40.917 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639255+0000", 2024-09-16T20:51:40.917 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4960000000000001e-06 2024-09-16T20:51:40.917 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.917 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.917 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:40.918 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639261+0000", 2024-09-16T20:51:40.918 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2839999999999997e-06 2024-09-16T20:51:40.918 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.918 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.918 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:40.918 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690375+0000", 2024-09-16T20:51:40.918 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051114684000000001 2024-09-16T20:51:40.918 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.918 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.918 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:40.918 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690425+0000", 2024-09-16T20:51:40.918 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9379999999999998e-05 2024-09-16T20:51:40.918 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.919 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.919 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:40.919 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690523+0000", 2024-09-16T20:51:40.919 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8139000000000003e-05 2024-09-16T20:51:40.919 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.919 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:40.919 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.919 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.919 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.919 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3732 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1554432~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:40.920 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:51:40.920 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 198.23560182099999, 2024-09-16T20:51:40.920 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051380096, 2024-09-16T20:51:40.920 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:40.920 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:40.920 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:40.920 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:40.920 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:40.920 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:40.920 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3732 2024-09-16T20:51:40.920 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.920 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:40.920 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.921 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:40.921 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:51:40.921 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.921 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.921 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.921 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:40.921 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:51:40.921 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.921 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.921 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.921 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:40.921 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:51:40.921 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6100000000000001e-07 2024-09-16T20:51:40.922 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.922 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.922 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:40.922 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639295+0000", 2024-09-16T20:51:40.922 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.447e-06 2024-09-16T20:51:40.922 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.922 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.922 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:40.922 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639302+0000", 2024-09-16T20:51:40.922 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0570000000000004e-06 2024-09-16T20:51:40.922 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.922 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.922 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:40.922 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639307+0000", 2024-09-16T20:51:40.922 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1429999999999997e-06 2024-09-16T20:51:40.923 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.923 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.923 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:40.923 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690555+0000", 2024-09-16T20:51:40.923 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051247963000000001 2024-09-16T20:51:40.923 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.923 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.923 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:40.923 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690602+0000", 2024-09-16T20:51:40.923 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7602000000000003e-05 2024-09-16T20:51:40.923 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.923 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.923 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:40.924 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690672+0000", 2024-09-16T20:51:40.924 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9423000000000003e-05 2024-09-16T20:51:40.924 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.924 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:40.924 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.924 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.924 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.925 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3734 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1562624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:40.925 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:51:40.925 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 198.23558017100001, 2024-09-16T20:51:40.925 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051490610999999999, 2024-09-16T20:51:40.925 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:40.925 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:40.925 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:40.925 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:40.925 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:40.925 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:40.925 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3734 2024-09-16T20:51:40.925 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.925 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:40.926 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.926 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:40.926 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:51:40.926 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.926 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.926 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.926 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:40.926 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:51:40.926 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.926 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.926 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.926 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:40.926 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639314+0000", 2024-09-16T20:51:40.926 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4400000000000001e-07 2024-09-16T20:51:40.926 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.927 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.927 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:40.927 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639316+0000", 2024-09-16T20:51:40.927 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1459999999999999e-06 2024-09-16T20:51:40.927 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.927 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.927 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:40.927 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639322+0000", 2024-09-16T20:51:40.927 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7610000000000004e-06 2024-09-16T20:51:40.927 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.927 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.927 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:40.927 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639327+0000", 2024-09-16T20:51:40.927 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.186e-06 2024-09-16T20:51:40.928 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.928 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.928 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:40.928 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690698+0000", 2024-09-16T20:51:40.928 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051371223000000001 2024-09-16T20:51:40.928 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.928 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.928 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:40.928 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690748+0000", 2024-09-16T20:51:40.928 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0192999999999999e-05 2024-09-16T20:51:40.928 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.928 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.929 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:40.929 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690804+0000", 2024-09-16T20:51:40.929 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5658000000000001e-05 2024-09-16T20:51:40.929 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.929 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:40.929 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.929 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.929 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.929 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3736 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1570816~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:40.929 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:51:40.929 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 198.23556091200001, 2024-09-16T20:51:40.929 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051596528000000003, 2024-09-16T20:51:40.929 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:40.929 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:40.930 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:40.930 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:40.930 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:40.930 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:40.930 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3736 2024-09-16T20:51:40.930 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.930 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:40.930 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.930 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:40.930 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:51:40.930 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.930 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.930 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.930 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:40.931 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:51:40.931 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.931 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.931 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.931 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:40.931 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:51:40.931 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9999999999999999e-07 2024-09-16T20:51:40.931 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.931 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.931 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:40.931 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639335+0000", 2024-09-16T20:51:40.931 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.635e-06 2024-09-16T20:51:40.931 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.931 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.931 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:40.932 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639340+0000", 2024-09-16T20:51:40.932 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5840000000000002e-06 2024-09-16T20:51:40.932 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.932 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.932 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:40.932 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639346+0000", 2024-09-16T20:51:40.932 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1190000000000002e-06 2024-09-16T20:51:40.932 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.932 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.932 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:40.932 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690817+0000", 2024-09-16T20:51:40.932 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051470721999999997 2024-09-16T20:51:40.932 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.933 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.933 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:40.933 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690875+0000", 2024-09-16T20:51:40.933 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7441000000000001e-05 2024-09-16T20:51:40.933 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.933 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.933 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:40.933 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690929+0000", 2024-09-16T20:51:40.933 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4727e-05 2024-09-16T20:51:40.933 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.933 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:40.933 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.933 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.934 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.934 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5434 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:40.934 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:51:40.934 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 198.01029934600001, 2024-09-16T20:51:40.934 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099755831000000003, 2024-09-16T20:51:40.934 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:40.934 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:40.934 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:40.934 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:40.934 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:40.934 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:40.934 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5434 2024-09-16T20:51:40.934 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.934 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:40.935 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.935 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:40.935 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:51:40.935 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.935 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.935 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.935 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:40.935 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:51:40.935 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.935 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.935 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.935 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:40.935 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864595+0000", 2024-09-16T20:51:40.935 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1099999999999995e-07 2024-09-16T20:51:40.936 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.936 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.936 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:40.936 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864597+0000", 2024-09-16T20:51:40.936 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3989999999999998e-06 2024-09-16T20:51:40.936 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.936 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.936 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:40.936 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864603+0000", 2024-09-16T20:51:40.936 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9309999999999996e-06 2024-09-16T20:51:40.936 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.936 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.936 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:40.937 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864607+0000", 2024-09-16T20:51:40.937 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6200000000000001e-06 2024-09-16T20:51:40.937 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.937 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.937 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:40.937 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964164+0000", 2024-09-16T20:51:40.937 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099557162000000005 2024-09-16T20:51:40.937 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.937 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.937 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:40.937 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964244+0000", 2024-09-16T20:51:40.937 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9597000000000004e-05 2024-09-16T20:51:40.937 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.938 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.938 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:40.938 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964350+0000", 2024-09-16T20:51:40.938 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010641099999999999 2024-09-16T20:51:40.938 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.938 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:40.938 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.938 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.938 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.938 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5435 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 20480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:40.938 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:51:40.938 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 198.01026952399999, 2024-09-16T20:51:40.939 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100897102, 2024-09-16T20:51:40.939 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:40.939 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:40.939 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:40.939 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:40.939 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:40.939 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:40.939 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5435 2024-09-16T20:51:40.939 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.939 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:40.939 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.939 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:40.940 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:51:40.940 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.940 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.940 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.940 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:40.940 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:51:40.940 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.940 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.940 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.940 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:40.940 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:51:40.940 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8000000000000001e-07 2024-09-16T20:51:40.940 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.940 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.941 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:40.941 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864626+0000", 2024-09-16T20:51:40.941 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.745e-06 2024-09-16T20:51:40.941 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.941 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.941 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:40.941 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864633+0000", 2024-09-16T20:51:40.941 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3339999999999998e-06 2024-09-16T20:51:40.941 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.941 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.941 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:40.941 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864636+0000", 2024-09-16T20:51:40.941 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1200000000000002e-06 2024-09-16T20:51:40.942 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.942 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.942 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:40.942 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965299+0000", 2024-09-16T20:51:40.942 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10066352000000001 2024-09-16T20:51:40.942 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.942 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.942 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:40.942 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965390+0000", 2024-09-16T20:51:40.942 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0894000000000001e-05 2024-09-16T20:51:40.942 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.942 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.942 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:40.942 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965521+0000", 2024-09-16T20:51:40.943 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013110899999999999 2024-09-16T20:51:40.943 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.943 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:40.943 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.943 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.943 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.943 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:152 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 327680~32768 out=32768b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:51:40.943 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:51:40.943 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 196.71177258200001, 2024-09-16T20:51:40.943 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11869405800000001, 2024-09-16T20:51:40.943 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:40.943 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:40.943 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:40.943 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:40.944 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:51:40.944 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:51:40.944 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 152 2024-09-16T20:51:40.944 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.944 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:40.944 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.944 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:40.944 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:51:40.944 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.944 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.944 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.944 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:40.945 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:51:40.945 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.945 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.945 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.945 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:40.945 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163122+0000", 2024-09-16T20:51:40.945 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6400000000000003e-07 2024-09-16T20:51:40.945 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.945 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.945 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:40.945 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163124+0000", 2024-09-16T20:51:40.945 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.565e-06 2024-09-16T20:51:40.945 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.945 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.946 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:40.946 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163136+0000", 2024-09-16T20:51:40.946 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1717e-05 2024-09-16T20:51:40.946 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.946 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.946 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:40.946 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163142+0000", 2024-09-16T20:51:40.946 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1750000000000002e-06 2024-09-16T20:51:40.946 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.946 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.946 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:40.946 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163606+0000", 2024-09-16T20:51:40.946 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00046418799999999998 2024-09-16T20:51:40.946 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.946 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.947 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:40.947 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163656+0000", 2024-09-16T20:51:40.947 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9852999999999999e-05 2024-09-16T20:51:40.947 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.947 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.947 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:40.947 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281815+0000", 2024-09-16T20:51:40.947 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11815909600000001 2024-09-16T20:51:40.947 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.947 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:40.947 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.947 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.947 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.947 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:153 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:51:40.948 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:51:40.948 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 196.71173962899999, 2024-09-16T20:51:40.948 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11903462400000001, 2024-09-16T20:51:40.948 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:40.948 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:40.948 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:40.948 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:40.948 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:51:40.948 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:51:40.948 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 153 2024-09-16T20:51:40.948 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.948 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:40.949 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.949 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:40.949 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:51:40.949 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.949 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.949 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.949 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:40.949 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:51:40.949 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.949 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.949 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.949 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:40.949 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163155+0000", 2024-09-16T20:51:40.949 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.37e-07 2024-09-16T20:51:40.949 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.950 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.950 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:40.950 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163157+0000", 2024-09-16T20:51:40.950 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6010000000000002e-06 2024-09-16T20:51:40.950 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.950 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.950 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:40.950 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163165+0000", 2024-09-16T20:51:40.950 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2619999999999998e-06 2024-09-16T20:51:40.950 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.950 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.950 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:40.950 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163171+0000", 2024-09-16T20:51:40.950 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.711e-06 2024-09-16T20:51:40.951 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.951 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.951 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:40.951 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281868+0000", 2024-09-16T20:51:40.951 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118696622 2024-09-16T20:51:40.951 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.951 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.951 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:40.951 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281969+0000", 2024-09-16T20:51:40.951 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101371 2024-09-16T20:51:40.951 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.951 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.951 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:40.951 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282189+0000", 2024-09-16T20:51:40.952 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00021952000000000001 2024-09-16T20:51:40.952 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.952 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:40.952 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.952 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.952 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.952 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:154 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 360448~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:51:40.952 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:51:40.952 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 196.71170785699999, 2024-09-16T20:51:40.952 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11932035100000001, 2024-09-16T20:51:40.952 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:40.952 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:40.952 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:40.953 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:40.953 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:51:40.953 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:51:40.953 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 154 2024-09-16T20:51:40.953 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.953 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:40.953 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.953 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:40.953 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:51:40.953 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.953 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.954 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.954 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:40.954 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:51:40.954 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.954 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.954 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.954 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:40.954 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:51:40.954 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:51:40.954 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.954 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.954 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:40.954 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163189+0000", 2024-09-16T20:51:40.955 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6840000000000001e-06 2024-09-16T20:51:40.955 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.955 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.955 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:40.955 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163196+0000", 2024-09-16T20:51:40.955 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7920000000000004e-06 2024-09-16T20:51:40.955 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.955 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.955 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:40.955 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163201+0000", 2024-09-16T20:51:40.955 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2809999999999999e-06 2024-09-16T20:51:40.955 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.955 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.956 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:40.956 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282219+0000", 2024-09-16T20:51:40.956 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.119018343 2024-09-16T20:51:40.956 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.956 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.956 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:40.956 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282305+0000", 2024-09-16T20:51:40.956 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5865000000000005e-05 2024-09-16T20:51:40.956 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.956 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.956 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:40.956 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282506+0000", 2024-09-16T20:51:40.957 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000200927 2024-09-16T20:51:40.957 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.957 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:40.957 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.957 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.957 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.957 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop(client.4366.0:3088 3.1e e38/23 3:79e5ec01:::rbd_header.110e891a9b2b:head v 38'306, mlcod=36'304)", 2024-09-16T20:51:40.957 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:51:40.957 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 196.710688989, 2024-09-16T20:51:40.957 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118102759, 2024-09-16T20:51:40.957 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:40.957 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:40.957 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:40.957 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:40.958 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.958 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:40.958 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:51:40.958 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.958 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.958 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.958 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:40.958 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:51:40.958 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.958 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.958 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.958 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:40.958 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:51:40.959 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.85e-07 2024-09-16T20:51:40.959 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.959 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.959 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:40.959 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164212+0000", 2024-09-16T20:51:40.959 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3999999999999997e-06 2024-09-16T20:51:40.959 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.959 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.959 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:40.959 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281553+0000", 2024-09-16T20:51:40.959 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11734135499999999 2024-09-16T20:51:40.959 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.959 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.960 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:40.960 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281567+0000", 2024-09-16T20:51:40.960 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4188999999999999e-05 2024-09-16T20:51:40.960 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.960 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.960 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:40.960 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281653+0000", 2024-09-16T20:51:40.960 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5385999999999999e-05 2024-09-16T20:51:40.960 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.960 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.960 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:40.960 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281679+0000", 2024-09-16T20:51:40.960 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6689000000000001e-05 2024-09-16T20:51:40.960 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.961 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.961 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:40.961 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282277+0000", 2024-09-16T20:51:40.961 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00059798000000000002 2024-09-16T20:51:40.961 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.961 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.961 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:40.961 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282307+0000", 2024-09-16T20:51:40.961 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9975e-05 2024-09-16T20:51:40.961 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.961 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:40.961 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.961 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.961 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.961 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5255.0:44 3.16 3:6f5476df:::rbd_header.110e25db1f60:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:51:40.962 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:51:40.962 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 124.554385934, 2024-09-16T20:51:40.962 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.121007509, 2024-09-16T20:51:40.962 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:40.962 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:40.962 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:40.962 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:40.962 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5255", 2024-09-16T20:51:40.962 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:51:40.962 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 44 2024-09-16T20:51:40.962 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.962 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:40.962 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.962 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:40.962 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:51:40.963 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.963 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.963 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.963 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:40.963 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:51:40.963 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.963 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.963 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.963 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:40.963 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:51:40.963 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0500000000000003e-07 2024-09-16T20:51:40.963 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.963 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.963 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:40.964 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320512+0000", 2024-09-16T20:51:40.964 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2540000000000001e-06 2024-09-16T20:51:40.964 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.964 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.964 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:40.964 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320518+0000", 2024-09-16T20:51:40.964 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1689999999999997e-06 2024-09-16T20:51:40.964 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.964 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.964 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:40.964 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320521+0000", 2024-09-16T20:51:40.964 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5200000000000002e-06 2024-09-16T20:51:40.964 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.964 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.964 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:40.965 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320548+0000", 2024-09-16T20:51:40.965 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6662999999999999e-05 2024-09-16T20:51:40.965 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.965 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.965 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:40.965 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320587+0000", 2024-09-16T20:51:40.965 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8866999999999999e-05 2024-09-16T20:51:40.965 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.965 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.965 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:40.965 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.441515+0000", 2024-09-16T20:51:40.965 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.120928331 2024-09-16T20:51:40.965 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.965 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:40.966 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.966 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.966 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.966 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:20329 3.5 3:a4ea64b7:::rbd_header.110ede5ef857:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:40.966 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:01.061694+0000", 2024-09-16T20:51:40.966 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 99.813199366000006, 2024-09-16T20:51:40.966 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.071234077000000007, 2024-09-16T20:51:40.966 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:40.966 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:40.966 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:40.966 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:40.966 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:51:40.966 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:40.966 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 20329 2024-09-16T20:51:40.967 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.967 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:40.967 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.967 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:40.967 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.061694+0000", 2024-09-16T20:51:40.967 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.967 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.967 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.967 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:40.967 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.061694+0000", 2024-09-16T20:51:40.967 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:40.967 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.967 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.968 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:40.968 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089232+0000", 2024-09-16T20:51:40.968 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.027537275 2024-09-16T20:51:40.968 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.968 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.968 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:40.968 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089237+0000", 2024-09-16T20:51:40.968 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4709999999999998e-06 2024-09-16T20:51:40.968 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.968 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.968 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:40.969 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089246+0000", 2024-09-16T20:51:40.969 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8230000000000001e-06 2024-09-16T20:51:40.969 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.969 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.969 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:40.969 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089252+0000", 2024-09-16T20:51:40.969 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3060000000000003e-06 2024-09-16T20:51:40.969 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.969 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.969 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:40.969 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.132756+0000", 2024-09-16T20:51:40.969 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.043504334999999998 2024-09-16T20:51:40.970 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.970 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.970 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:40.970 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.132829+0000", 2024-09-16T20:51:40.970 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2849999999999995e-05 2024-09-16T20:51:40.970 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:40.970 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:40.970 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:40.970 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.132928+0000", 2024-09-16T20:51:40.970 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.9017000000000004e-05 2024-09-16T20:51:40.970 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.970 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:40.970 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.971 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:40.971 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:40.971 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:40.972 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:51:40.973 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:40.971+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:41.073 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:41.072+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:41.087 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:41.087 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:51:41.087 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:51:41.087 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:41.096 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:51:41.174 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:41.172+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:41.203 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:41.203 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:51:41.203 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:51:41.203 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:51:41.203 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:41.211 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:51:41.274 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:41.273+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:41.361 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:41.361 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:51:41.361 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:51:41.361 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:51:41.361 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.361 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:3088 3.1e 3:79e5ec01:::rbd_header.110e891a9b2b:head [watch watch cookie 140213437171488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:51:41.361 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:51:41.361 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 197.19513209900001, 2024-09-16T20:51:41.361 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.11895186200000001, 2024-09-16T20:51:41.362 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:41.362 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:41.362 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:41.362 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:41.362 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:41.362 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:41.362 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3088 2024-09-16T20:51:41.362 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.362 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:41.362 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.362 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:41.362 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:51:41.362 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.362 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.362 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.363 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:41.363 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:51:41.363 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.363 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.363 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.363 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:41.363 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163846+0000", 2024-09-16T20:51:41.363 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6799999999999999e-07 2024-09-16T20:51:41.363 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.363 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.363 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:41.363 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163850+0000", 2024-09-16T20:51:41.363 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.929e-06 2024-09-16T20:51:41.363 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.364 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.364 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:41.364 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163862+0000", 2024-09-16T20:51:41.364 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1972e-05 2024-09-16T20:51:41.364 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.364 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.364 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:41.364 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163867+0000", 2024-09-16T20:51:41.364 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.451e-06 2024-09-16T20:51:41.364 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.364 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.364 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:41.364 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163912+0000", 2024-09-16T20:51:41.365 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4456000000000003e-05 2024-09-16T20:51:41.365 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.365 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.365 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:41.365 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163962+0000", 2024-09-16T20:51:41.365 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0578000000000003e-05 2024-09-16T20:51:41.365 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.365 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.365 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-16T20:51:41.365 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164058+0000", 2024-09-16T20:51:41.365 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5934999999999995e-05 2024-09-16T20:51:41.365 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.365 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.366 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:51:41.366 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164570+0000", 2024-09-16T20:51:41.366 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00051241100000000005 2024-09-16T20:51:41.366 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.366 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.366 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:41.366 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282629+0000", 2024-09-16T20:51:41.366 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.118058986 2024-09-16T20:51:41.366 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.366 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.366 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:51:41.366 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282725+0000", 2024-09-16T20:51:41.366 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5063999999999996e-05 2024-09-16T20:51:41.367 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.367 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.367 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:41.367 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282797+0000", 2024-09-16T20:51:41.367 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2212000000000006e-05 2024-09-16T20:51:41.367 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.367 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:41.367 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.367 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.367 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.367 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:8039 3.1e 3.ac44ff9e (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:51:41.367 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:51:41.368 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 171.65130475000001, 2024-09-16T20:51:41.368 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0053310449999999, 2024-09-16T20:51:41.368 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:41.368 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:41.368 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:41.368 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:41.368 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:41.368 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:41.368 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8039 2024-09-16T20:51:41.368 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.368 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:41.368 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.368 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:41.369 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:51:41.369 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.369 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.369 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.369 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:41.369 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:51:41.369 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.369 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.369 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.369 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:41.369 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707674+0000", 2024-09-16T20:51:41.369 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6530000000000001e-06 2024-09-16T20:51:41.369 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.370 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.370 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:41.370 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707681+0000", 2024-09-16T20:51:41.370 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9639999999999997e-06 2024-09-16T20:51:41.370 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.370 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.370 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:41.370 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707700+0000", 2024-09-16T20:51:41.370 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8674000000000001e-05 2024-09-16T20:51:41.370 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.370 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.370 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:41.370 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707708+0000", 2024-09-16T20:51:41.371 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6810000000000004e-06 2024-09-16T20:51:41.371 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.371 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.371 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:41.371 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711293+0000", 2024-09-16T20:51:41.371 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035848189999999999 2024-09-16T20:51:41.371 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.371 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.371 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:41.371 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711301+0000", 2024-09-16T20:51:41.371 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4730000000000003e-06 2024-09-16T20:51:41.371 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.371 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.371 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:41.372 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.712990+0000", 2024-09-16T20:51:41.372 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.001689348 2024-09-16T20:51:41.372 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.372 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.372 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:41.372 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713003+0000", 2024-09-16T20:51:41.372 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3433e-05 2024-09-16T20:51:41.372 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.372 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:41.372 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.372 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.372 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.372 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:8040 3.1e 3.ac44ff9e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:51:41.372 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:51:41.373 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 171.65124435499999, 2024-09-16T20:51:41.373 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0053410780000001, 2024-09-16T20:51:41.373 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:41.373 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:41.373 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:41.373 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:41.373 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:41.373 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:41.373 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8040 2024-09-16T20:51:41.373 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.373 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:41.373 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.373 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:41.374 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:51:41.374 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.374 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.374 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.374 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:41.374 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:51:41.374 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.374 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.374 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.374 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:41.374 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:51:41.374 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8100000000000003e-07 2024-09-16T20:51:41.374 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.375 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.375 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:41.375 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707737+0000", 2024-09-16T20:51:41.375 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5020000000000001e-06 2024-09-16T20:51:41.375 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.375 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.375 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:41.375 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707745+0000", 2024-09-16T20:51:41.375 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2169999999999994e-06 2024-09-16T20:51:41.375 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.375 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.375 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:41.375 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707751+0000", 2024-09-16T20:51:41.375 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.263e-06 2024-09-16T20:51:41.376 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.376 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.376 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:41.376 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711336+0000", 2024-09-16T20:51:41.376 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035848360000000001 2024-09-16T20:51:41.376 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.376 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.376 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:41.376 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711340+0000", 2024-09-16T20:51:41.376 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1309999999999999e-06 2024-09-16T20:51:41.376 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.376 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.377 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:41.377 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713060+0000", 2024-09-16T20:51:41.377 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0017195059999999 2024-09-16T20:51:41.377 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.377 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.377 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:41.377 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713074+0000", 2024-09-16T20:51:41.377 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4042e-05 2024-09-16T20:51:41.377 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.377 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:41.377 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.377 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.377 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.377 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4927.0:4487 3.1e 3.ac44ff9e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:51:41.378 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:51:41.378 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 171.650724153, 2024-09-16T20:51:41.378 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0048840880000001, 2024-09-16T20:51:41.378 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:41.378 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:41.378 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:41.378 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:41.378 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4927", 2024-09-16T20:51:41.378 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:41.378 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4487 2024-09-16T20:51:41.378 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.378 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:41.378 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.378 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:41.379 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:51:41.379 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.379 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.379 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.379 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:41.379 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:51:41.379 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.379 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.379 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.379 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:41.379 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708254+0000", 2024-09-16T20:51:41.379 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1800000000000005e-07 2024-09-16T20:51:41.379 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.379 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.380 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:41.380 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708258+0000", 2024-09-16T20:51:41.380 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8689999999999997e-06 2024-09-16T20:51:41.380 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.380 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.380 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:41.380 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708267+0000", 2024-09-16T20:51:41.380 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9590000000000007e-06 2024-09-16T20:51:41.380 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.380 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.380 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:41.380 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708274+0000", 2024-09-16T20:51:41.380 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9970000000000001e-06 2024-09-16T20:51:41.381 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.381 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.381 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:41.381 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711366+0000", 2024-09-16T20:51:41.381 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030919530000000001 2024-09-16T20:51:41.381 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.381 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.381 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:41.381 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711372+0000", 2024-09-16T20:51:41.381 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9410000000000004e-06 2024-09-16T20:51:41.381 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.381 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.381 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:41.381 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713126+0000", 2024-09-16T20:51:41.382 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0017537249999999 2024-09-16T20:51:41.382 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.382 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.382 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:41.382 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713137+0000", 2024-09-16T20:51:41.382 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0825999999999999e-05 2024-09-16T20:51:41.382 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.382 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:41.382 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.382 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.382 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.382 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5089.0:11703 3.1b 3:dcd4684e:::rbd_data.110e747f0779.000000000000001b:head [set-alloc-hint object_size 262144 write_size 262144,write 139264~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:51:41.382 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:51:41.383 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 157.220988379, 2024-09-16T20:51:41.383 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.084231385000000006, 2024-09-16T20:51:41.383 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:41.383 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:41.383 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:41.383 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:41.383 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5089", 2024-09-16T20:51:41.383 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:51:41.383 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 11703 2024-09-16T20:51:41.383 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.383 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:41.383 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.383 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:41.383 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:51:41.384 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.384 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.384 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.384 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:41.384 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:51:41.384 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.384 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.384 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.384 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:41.384 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:51:41.384 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2600000000000001e-07 2024-09-16T20:51:41.384 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.384 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.384 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:41.385 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137998+0000", 2024-09-16T20:51:41.385 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9549999999999998e-06 2024-09-16T20:51:41.385 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.385 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.385 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:41.385 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138008+0000", 2024-09-16T20:51:41.385 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3789999999999995e-06 2024-09-16T20:51:41.385 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.385 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.385 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:41.385 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138013+0000", 2024-09-16T20:51:41.385 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2209999999999996e-06 2024-09-16T20:51:41.385 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.385 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.386 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:41.386 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138054+0000", 2024-09-16T20:51:41.386 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1022999999999997e-05 2024-09-16T20:51:41.386 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.386 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.386 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:41.386 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138124+0000", 2024-09-16T20:51:41.386 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0234999999999994e-05 2024-09-16T20:51:41.386 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.386 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.386 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:41.386 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138197+0000", 2024-09-16T20:51:41.386 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2451000000000001e-05 2024-09-16T20:51:41.386 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.387 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.387 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:41.387 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222010+0000", 2024-09-16T20:51:41.387 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.083813433000000007 2024-09-16T20:51:41.387 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.387 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.387 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:51:41.387 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222163+0000", 2024-09-16T20:51:41.387 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015274599999999999 2024-09-16T20:51:41.387 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.387 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.387 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:51:41.387 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222197+0000", 2024-09-16T20:51:41.387 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3683000000000003e-05 2024-09-16T20:51:41.388 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.388 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.388 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:41.388 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222220+0000", 2024-09-16T20:51:41.388 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3433000000000001e-05 2024-09-16T20:51:41.388 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.388 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:41.388 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.388 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.388 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.388 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.5089.0:11702 3.15 e73/64 3:acf33c8b:::rbd_data.110e747f0779.000000000000001a:head v 73'132, mlcod=73'130)", 2024-09-16T20:51:41.388 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:51:41.388 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 157.22072105000001, 2024-09-16T20:51:41.388 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.084079763000000002, 2024-09-16T20:51:41.389 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:41.389 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:41.389 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:41.389 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:41.389 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.389 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:41.389 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:51:41.389 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.389 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.389 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.389 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:41.389 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:51:41.389 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.389 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.390 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.390 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:41.390 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:51:41.390 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:51:41.390 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.390 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.390 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:41.390 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138263+0000", 2024-09-16T20:51:41.390 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1e-06 2024-09-16T20:51:41.390 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.390 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.390 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:41.390 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221589+0000", 2024-09-16T20:51:41.391 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.083326002999999996 2024-09-16T20:51:41.391 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.391 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.391 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:41.391 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221614+0000", 2024-09-16T20:51:41.391 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5593999999999999e-05 2024-09-16T20:51:41.391 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.391 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.391 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:41.391 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221714+0000", 2024-09-16T20:51:41.391 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9889000000000005e-05 2024-09-16T20:51:41.391 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.391 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.391 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:41.391 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221727+0000", 2024-09-16T20:51:41.392 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3125999999999999e-05 2024-09-16T20:51:41.392 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.392 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.392 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:51:41.392 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222294+0000", 2024-09-16T20:51:41.392 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00056712199999999998 2024-09-16T20:51:41.392 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.392 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.392 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:41.392 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222336+0000", 2024-09-16T20:51:41.392 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1470000000000001e-05 2024-09-16T20:51:41.392 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.392 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:41.393 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.393 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.393 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.393 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5138.0:9358 3.c 3:33996a43:::rbd_data.110e6372565d.0000000000000007:head [read 1343488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T20:51:41.393 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:51:41.393 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 133.50644564000001, 2024-09-16T20:51:41.393 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0167493569999999, 2024-09-16T20:51:41.393 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:41.393 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:41.393 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:41.393 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:41.393 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5138", 2024-09-16T20:51:41.393 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:51:41.393 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9358 2024-09-16T20:51:41.393 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.394 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:41.394 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.394 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:41.394 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:51:41.394 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.394 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.394 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.394 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:41.394 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:51:41.394 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.394 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.394 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.394 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:41.394 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852533+0000", 2024-09-16T20:51:41.394 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.646e-06 2024-09-16T20:51:41.394 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.395 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.395 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:41.395 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852539+0000", 2024-09-16T20:51:41.395 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3389999999999998e-06 2024-09-16T20:51:41.395 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.395 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.395 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:41.395 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852552+0000", 2024-09-16T20:51:41.395 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.271e-05 2024-09-16T20:51:41.395 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.395 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.395 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:41.395 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852563+0000", 2024-09-16T20:51:41.395 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1139e-05 2024-09-16T20:51:41.395 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.395 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.396 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:41.396 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852883+0000", 2024-09-16T20:51:41.396 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00032010600000000001 2024-09-16T20:51:41.396 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.396 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.396 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:41.396 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852890+0000", 2024-09-16T20:51:41.396 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7669999999999998e-06 2024-09-16T20:51:41.396 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.396 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.396 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:41.396 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869059+0000", 2024-09-16T20:51:41.396 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.016169353 2024-09-16T20:51:41.396 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.396 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.397 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:41.397 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869178+0000", 2024-09-16T20:51:41.397 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118193 2024-09-16T20:51:41.397 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.397 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.397 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:41.397 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869281+0000", 2024-09-16T20:51:41.397 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010310399999999999 2024-09-16T20:51:41.397 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.397 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:41.397 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.397 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.397 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.398 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5138.0:9382 3.c 3:33996a43:::rbd_data.110e6372565d.0000000000000007:head [read 0~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T20:51:41.398 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:51:41.398 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 133.506394763, 2024-09-16T20:51:41.398 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.017136034, 2024-09-16T20:51:41.398 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:41.398 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:41.398 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:41.398 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:41.398 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5138", 2024-09-16T20:51:41.398 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:51:41.398 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9382 2024-09-16T20:51:41.398 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.398 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:41.399 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.399 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:41.399 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:51:41.399 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.399 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.399 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.399 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:41.399 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:51:41.399 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.399 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.399 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.399 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:41.399 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852583+0000", 2024-09-16T20:51:41.399 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.51e-07 2024-09-16T20:51:41.399 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.400 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.400 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:41.400 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852586+0000", 2024-09-16T20:51:41.400 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0280000000000001e-06 2024-09-16T20:51:41.400 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.400 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.400 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:41.400 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852595+0000", 2024-09-16T20:51:41.400 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2799999999999992e-06 2024-09-16T20:51:41.400 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.400 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.400 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:41.401 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852601+0000", 2024-09-16T20:51:41.401 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3810000000000004e-06 2024-09-16T20:51:41.401 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.401 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.401 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:41.401 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852921+0000", 2024-09-16T20:51:41.401 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00031979699999999997 2024-09-16T20:51:41.401 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.401 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.401 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:41.401 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852930+0000", 2024-09-16T20:51:41.401 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6489999999999997e-06 2024-09-16T20:51:41.401 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.401 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.402 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:41.402 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869434+0000", 2024-09-16T20:51:41.402 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0165038829999999 2024-09-16T20:51:41.402 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.402 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.402 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:41.402 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869495+0000", 2024-09-16T20:51:41.402 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1509999999999999e-05 2024-09-16T20:51:41.402 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.402 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.402 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:41.402 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869718+0000", 2024-09-16T20:51:41.402 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000222955 2024-09-16T20:51:41.402 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.403 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:41.403 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.403 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.403 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.403 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9971 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 643072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:41.403 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642102+0000", 2024-09-16T20:51:41.403 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 110.716874619, 2024-09-16T20:51:41.403 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077421016999999995, 2024-09-16T20:51:41.403 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:41.403 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:41.403 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:41.403 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:41.403 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:41.403 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:41.404 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9971 2024-09-16T20:51:41.404 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.404 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:41.404 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.404 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:41.404 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642102+0000", 2024-09-16T20:51:41.404 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.404 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.404 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.404 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:41.404 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642102+0000", 2024-09-16T20:51:41.404 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.404 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.405 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.405 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:41.405 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642103+0000", 2024-09-16T20:51:41.405 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7399999999999998e-07 2024-09-16T20:51:41.405 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.405 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.405 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:41.405 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642105+0000", 2024-09-16T20:51:41.405 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2649999999999999e-06 2024-09-16T20:51:41.405 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.405 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.405 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:41.405 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642112+0000", 2024-09-16T20:51:41.406 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.294e-06 2024-09-16T20:51:41.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.406 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.406 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:41.406 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642118+0000", 2024-09-16T20:51:41.406 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4500000000000003e-06 2024-09-16T20:51:41.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.406 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.406 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:41.406 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719285+0000", 2024-09-16T20:51:41.406 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077166883000000006 2024-09-16T20:51:41.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.406 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.406 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:41.407 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719368+0000", 2024-09-16T20:51:41.407 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2811000000000004e-05 2024-09-16T20:51:41.407 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.407 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.407 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:41.407 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719523+0000", 2024-09-16T20:51:41.407 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015584 2024-09-16T20:51:41.407 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.407 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:41.407 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.407 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.407 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.407 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9972 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 648192~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:41.407 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642125+0000", 2024-09-16T20:51:41.407 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 110.716851672, 2024-09-16T20:51:41.408 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077710324999999997, 2024-09-16T20:51:41.408 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:41.408 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:41.408 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:41.408 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:41.408 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:41.408 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:41.408 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9972 2024-09-16T20:51:41.408 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.408 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:41.408 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.408 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:41.408 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642125+0000", 2024-09-16T20:51:41.409 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.409 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.409 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.409 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:41.409 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642125+0000", 2024-09-16T20:51:41.409 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.409 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.409 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.409 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:41.409 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642126+0000", 2024-09-16T20:51:41.409 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1899999999999998e-07 2024-09-16T20:51:41.409 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.409 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.409 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:41.410 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642128+0000", 2024-09-16T20:51:41.410 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8619999999999999e-06 2024-09-16T20:51:41.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.410 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.410 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:41.410 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642134+0000", 2024-09-16T20:51:41.410 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5289999999999997e-06 2024-09-16T20:51:41.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.410 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.410 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:41.410 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642139+0000", 2024-09-16T20:51:41.410 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1250000000000001e-06 2024-09-16T20:51:41.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.410 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.411 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:41.411 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719570+0000", 2024-09-16T20:51:41.411 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077430516000000005 2024-09-16T20:51:41.411 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.411 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.411 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:41.411 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719655+0000", 2024-09-16T20:51:41.411 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5500000000000005e-05 2024-09-16T20:51:41.411 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.411 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.411 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:41.411 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719836+0000", 2024-09-16T20:51:41.411 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018037399999999999 2024-09-16T20:51:41.411 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.412 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:41.412 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.412 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.412 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.412 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9973 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 653312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:41.412 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:51:41.412 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 110.716831062, 2024-09-16T20:51:41.412 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077905262000000003, 2024-09-16T20:51:41.412 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:41.412 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:41.412 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:41.412 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:41.412 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:41.413 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:41.413 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9973 2024-09-16T20:51:41.413 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.413 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:41.413 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.413 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:41.413 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:51:41.413 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.413 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.413 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.413 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:41.413 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:51:41.413 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.413 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.414 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.414 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:41.414 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:51:41.414 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:51:41.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.414 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.414 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:41.414 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642148+0000", 2024-09-16T20:51:41.414 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0219999999999999e-06 2024-09-16T20:51:41.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.414 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.414 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:41.414 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642154+0000", 2024-09-16T20:51:41.414 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8540000000000002e-06 2024-09-16T20:51:41.415 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.415 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.415 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:41.415 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642159+0000", 2024-09-16T20:51:41.415 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.045e-06 2024-09-16T20:51:41.415 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.415 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.415 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:41.415 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719875+0000", 2024-09-16T20:51:41.415 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077716116000000002 2024-09-16T20:51:41.415 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.415 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.415 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:41.415 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719934+0000", 2024-09-16T20:51:41.415 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8819e-05 2024-09-16T20:51:41.415 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.416 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.416 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:41.416 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720051+0000", 2024-09-16T20:51:41.416 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000116947 2024-09-16T20:51:41.416 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.416 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:41.416 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.416 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.416 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.416 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9974 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 658432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:41.416 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642166+0000", 2024-09-16T20:51:41.416 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 110.71681114800001, 2024-09-16T20:51:41.416 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.078179786000000001, 2024-09-16T20:51:41.416 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:41.416 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:41.417 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:41.417 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:41.417 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:41.417 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:41.417 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9974 2024-09-16T20:51:41.417 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.417 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:41.417 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.417 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:41.417 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642166+0000", 2024-09-16T20:51:41.417 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.417 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.417 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.417 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:41.418 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642166+0000", 2024-09-16T20:51:41.418 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.418 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.418 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:41.418 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647343+0000", 2024-09-16T20:51:41.418 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0051769440000000002 2024-09-16T20:51:41.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.418 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.418 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:41.418 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647348+0000", 2024-09-16T20:51:41.418 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3619999999999997e-06 2024-09-16T20:51:41.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.418 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.419 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:41.419 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647359+0000", 2024-09-16T20:51:41.419 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0929e-05 2024-09-16T20:51:41.419 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.419 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.419 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:41.419 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647366+0000", 2024-09-16T20:51:41.419 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0910000000000004e-06 2024-09-16T20:51:41.419 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.419 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.419 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:41.419 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720094+0000", 2024-09-16T20:51:41.419 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.072727403999999995 2024-09-16T20:51:41.419 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.419 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.420 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:41.420 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720173+0000", 2024-09-16T20:51:41.420 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9382000000000007e-05 2024-09-16T20:51:41.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.420 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.420 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:41.420 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720346+0000", 2024-09-16T20:51:41.420 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017267399999999999 2024-09-16T20:51:41.420 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.420 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:41.420 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.420 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.421 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9991 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 745472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:41.421 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:51:41.421 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 110.711241755, 2024-09-16T20:51:41.421 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077257820000000005, 2024-09-16T20:51:41.421 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:41.421 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:41.421 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:41.421 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:41.421 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:41.422 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:41.422 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9991 2024-09-16T20:51:41.422 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.422 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:41.422 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.422 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:41.422 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:51:41.422 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.422 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.422 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.422 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:41.422 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:51:41.423 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.423 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.423 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:41.423 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:51:41.423 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1199999999999999e-07 2024-09-16T20:51:41.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.423 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.423 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:41.423 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647736+0000", 2024-09-16T20:51:41.423 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3699999999999999e-07 2024-09-16T20:51:41.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.423 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.423 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:41.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647739+0000", 2024-09-16T20:51:41.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9359999999999999e-06 2024-09-16T20:51:41.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.424 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:41.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647742+0000", 2024-09-16T20:51:41.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5720000000000001e-06 2024-09-16T20:51:41.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.424 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:41.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724774+0000", 2024-09-16T20:51:41.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077032457999999998 2024-09-16T20:51:41.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.425 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:41.425 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724818+0000", 2024-09-16T20:51:41.425 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3803000000000001e-05 2024-09-16T20:51:41.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.425 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:41.425 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724993+0000", 2024-09-16T20:51:41.425 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000174902 2024-09-16T20:51:41.425 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.425 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:41.425 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.426 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9992 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 750592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:41.426 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:51:41.426 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 110.711232166, 2024-09-16T20:51:41.426 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077553227000000002, 2024-09-16T20:51:41.426 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:41.426 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:41.426 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:41.426 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:41.426 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:41.426 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:41.426 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9992 2024-09-16T20:51:41.426 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.426 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:41.426 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.426 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:41.427 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:51:41.427 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.427 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.427 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.427 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:41.427 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:51:41.427 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.427 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.427 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.427 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:41.427 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:51:41.427 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0200000000000001e-07 2024-09-16T20:51:41.427 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.428 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.428 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:41.428 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647746+0000", 2024-09-16T20:51:41.428 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.088e-06 2024-09-16T20:51:41.428 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.428 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.428 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:41.428 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647749+0000", 2024-09-16T20:51:41.428 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9170000000000002e-06 2024-09-16T20:51:41.428 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.428 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.428 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:41.428 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647752+0000", 2024-09-16T20:51:41.429 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.774e-06 2024-09-16T20:51:41.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.429 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.429 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:41.429 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725042+0000", 2024-09-16T20:51:41.429 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077289952999999995 2024-09-16T20:51:41.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.429 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.429 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:41.429 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725113+0000", 2024-09-16T20:51:41.429 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0920999999999994e-05 2024-09-16T20:51:41.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.430 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.430 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:41.430 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725298+0000", 2024-09-16T20:51:41.430 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000185372 2024-09-16T20:51:41.430 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.430 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:41.430 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.430 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.430 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9993 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 755712~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:41.430 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647755+0000", 2024-09-16T20:51:41.430 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 110.71122214, 2024-09-16T20:51:41.431 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077725674999999994, 2024-09-16T20:51:41.431 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:41.431 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:41.431 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:41.431 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:41.431 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:41.431 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:41.431 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9993 2024-09-16T20:51:41.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.431 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:41.431 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.431 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:41.432 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647755+0000", 2024-09-16T20:51:41.432 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.432 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.432 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:41.432 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647755+0000", 2024-09-16T20:51:41.432 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.432 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.432 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:41.432 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653406+0000", 2024-09-16T20:51:41.432 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.005650876 2024-09-16T20:51:41.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.433 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.433 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:41.433 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653411+0000", 2024-09-16T20:51:41.433 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8899999999999998e-06 2024-09-16T20:51:41.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.433 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.433 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:41.433 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653423+0000", 2024-09-16T20:51:41.433 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2447e-05 2024-09-16T20:51:41.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.433 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.433 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:41.433 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653431+0000", 2024-09-16T20:51:41.433 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5730000000000002e-06 2024-09-16T20:51:41.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.434 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.434 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:41.434 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725333+0000", 2024-09-16T20:51:41.434 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.071902743000000005 2024-09-16T20:51:41.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.434 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.434 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:41.434 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725385+0000", 2024-09-16T20:51:41.434 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1585000000000002e-05 2024-09-16T20:51:41.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.434 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.434 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:41.434 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725481+0000", 2024-09-16T20:51:41.434 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5561000000000002e-05 2024-09-16T20:51:41.435 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.435 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:41.435 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.435 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.435 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6061.0:5400 3.b 3:d353d652:::rbd_data.110e16eafbd.0000000000000010:head [read 577536~28672] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-16T20:51:41.435 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:27.761367+0000", 2024-09-16T20:51:41.435 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 73.597610193999998, 2024-09-16T20:51:41.435 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0029066630000001, 2024-09-16T20:51:41.435 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:41.435 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:41.435 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:41.435 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:41.435 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6061", 2024-09-16T20:51:41.436 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3930128440", 2024-09-16T20:51:41.436 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5400 2024-09-16T20:51:41.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.436 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:41.436 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.436 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:41.436 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761367+0000", 2024-09-16T20:51:41.436 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.436 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.436 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:41.436 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761367+0000", 2024-09-16T20:51:41.436 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.436 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.437 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:41.437 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761369+0000", 2024-09-16T20:51:41.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.934e-06 2024-09-16T20:51:41.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.437 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:41.437 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761374+0000", 2024-09-16T20:51:41.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7139999999999998e-06 2024-09-16T20:51:41.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.437 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:41.437 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761387+0000", 2024-09-16T20:51:41.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2349e-05 2024-09-16T20:51:41.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:41.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761393+0000", 2024-09-16T20:51:41.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.652e-06 2024-09-16T20:51:41.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:41.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761442+0000", 2024-09-16T20:51:41.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9011000000000002e-05 2024-09-16T20:51:41.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:41.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761451+0000", 2024-09-16T20:51:41.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1920000000000005e-06 2024-09-16T20:51:41.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.439 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.439 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:41.439 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:28.764169+0000", 2024-09-16T20:51:41.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.002718161 2024-09-16T20:51:41.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.439 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.439 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:41.439 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:28.764273+0000", 2024-09-16T20:51:41.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010465 2024-09-16T20:51:41.439 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.439 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:41.439 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.439 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.440 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6061.0:5424 3.b 3:d353d652:::rbd_data.110e16eafbd.0000000000000010:head [read 606208~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-16T20:51:41.440 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:27.766882+0000", 2024-09-16T20:51:41.440 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 73.592094830999997, 2024-09-16T20:51:41.440 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99753650500000002, 2024-09-16T20:51:41.440 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:41.440 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:41.440 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:41.440 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:41.440 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6061", 2024-09-16T20:51:41.440 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3930128440", 2024-09-16T20:51:41.440 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5424 2024-09-16T20:51:41.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.440 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:41.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.440 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:41.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766882+0000", 2024-09-16T20:51:41.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:41.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766882+0000", 2024-09-16T20:51:41.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:41.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766883+0000", 2024-09-16T20:51:41.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3289999999999999e-06 2024-09-16T20:51:41.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:41.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766888+0000", 2024-09-16T20:51:41.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5800000000000002e-06 2024-09-16T20:51:41.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:41.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766900+0000", 2024-09-16T20:51:41.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1861e-05 2024-09-16T20:51:41.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:41.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766909+0000", 2024-09-16T20:51:41.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5210000000000008e-06 2024-09-16T20:51:41.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.443 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:41.443 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766958+0000", 2024-09-16T20:51:41.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8718999999999999e-05 2024-09-16T20:51:41.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.443 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:41.443 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766975+0000", 2024-09-16T20:51:41.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7231000000000002e-05 2024-09-16T20:51:41.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.443 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:41.443 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:28.764327+0000", 2024-09-16T20:51:41.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99735112500000001 2024-09-16T20:51:41.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:41.444 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:28.764419+0000", 2024-09-16T20:51:41.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2139000000000007e-05 2024-09-16T20:51:41.444 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.444 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:41.444 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.444 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:31909 3.1e 3:79ff2235:::rbd_header.110ee9a2c138:head [watch reconnect cookie 140213436757440 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e199)", 2024-09-16T20:51:41.444 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:51:26.618259+0000", 2024-09-16T20:51:41.444 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 14.740717804999999, 2024-09-16T20:51:41.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0148341999999999, 2024-09-16T20:51:41.445 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:41.445 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:41.445 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:41.445 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:41.445 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:41.445 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:41.445 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 31909 2024-09-16T20:51:41.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.445 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:41.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:41.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.618259+0000", 2024-09-16T20:51:41.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.446 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:41.446 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.618259+0000", 2024-09-16T20:51:41.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.446 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:41.446 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.618261+0000", 2024-09-16T20:51:41.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4899999999999999e-06 2024-09-16T20:51:41.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.446 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:41.446 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.618267+0000", 2024-09-16T20:51:41.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8649999999999998e-06 2024-09-16T20:51:41.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:41.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.618277+0000", 2024-09-16T20:51:41.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0209e-05 2024-09-16T20:51:41.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:41.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.618282+0000", 2024-09-16T20:51:41.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.733e-06 2024-09-16T20:51:41.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:41.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.618341+0000", 2024-09-16T20:51:41.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8823000000000002e-05 2024-09-16T20:51:41.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:51:41.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.618346+0000", 2024-09-16T20:51:41.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8049999999999997e-06 2024-09-16T20:51:41.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:41.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.620745+0000", 2024-09-16T20:51:41.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0023987499999999998 2024-09-16T20:51:41.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:41.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.620755+0000", 2024-09-16T20:51:41.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0474e-05 2024-09-16T20:51:41.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.449 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:41.449 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:27.629480+0000", 2024-09-16T20:51:41.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.008724403 2024-09-16T20:51:41.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.449 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:41.449 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:27.629590+0000", 2024-09-16T20:51:41.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011044600000000001 2024-09-16T20:51:41.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.449 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:41.450 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:27.633093+0000", 2024-09-16T20:51:41.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003503202 2024-09-16T20:51:41.450 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.450 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:41.450 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.450 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:31914 3.1e 3:79ff2235:::rbd_header.110ee9a2c138:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e199)", 2024-09-16T20:51:41.450 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:51:26.635328+0000", 2024-09-16T20:51:41.450 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 14.723648776999999, 2024-09-16T20:51:41.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99790577499999999, 2024-09-16T20:51:41.450 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:41.450 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:41.450 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:41.450 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:41.451 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:41.451 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:41.451 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 31914 2024-09-16T20:51:41.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.451 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:41.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.451 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:41.451 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.635328+0000", 2024-09-16T20:51:41.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.451 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:41.451 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.635328+0000", 2024-09-16T20:51:41.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:41.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.635330+0000", 2024-09-16T20:51:41.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3379999999999999e-06 2024-09-16T20:51:41.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:41.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.635334+0000", 2024-09-16T20:51:41.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6280000000000004e-06 2024-09-16T20:51:41.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:41.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.635344+0000", 2024-09-16T20:51:41.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7229999999999999e-06 2024-09-16T20:51:41.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:41.453 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.635349+0000", 2024-09-16T20:51:41.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1719999999999998e-06 2024-09-16T20:51:41.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:41.453 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.635424+0000", 2024-09-16T20:51:41.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4499999999999995e-05 2024-09-16T20:51:41.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:41.453 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.635435+0000", 2024-09-16T20:51:41.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1355000000000001e-05 2024-09-16T20:51:41.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:41.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:27.630304+0000", 2024-09-16T20:51:41.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99486901500000002 2024-09-16T20:51:41.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:41.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:27.630370+0000", 2024-09-16T20:51:41.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6390999999999995e-05 2024-09-16T20:51:41.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-16T20:51:41.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:27.630488+0000", 2024-09-16T20:51:41.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011726 2024-09-16T20:51:41.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:51:41.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:27.632101+0000", 2024-09-16T20:51:41.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016134249999999999 2024-09-16T20:51:41.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:41.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:27.633138+0000", 2024-09-16T20:51:41.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010367499999999999 2024-09-16T20:51:41.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:51:41.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:27.633196+0000", 2024-09-16T20:51:41.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8399000000000001e-05 2024-09-16T20:51:41.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.456 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:41.456 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:27.633234+0000", 2024-09-16T20:51:41.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7818999999999998e-05 2024-09-16T20:51:41.456 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.456 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:41.456 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.456 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:32211 3.1b 3:df9cfa64:::journal.110e645639d:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e204)", 2024-09-16T20:51:41.456 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:51:31.721834+0000", 2024-09-16T20:51:41.456 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 9.6371431449999996, 2024-09-16T20:51:41.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.917244744, 2024-09-16T20:51:41.457 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:41.457 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:41.457 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:41.457 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:41.457 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:41.457 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:41.457 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 32211 2024-09-16T20:51:41.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.457 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:41.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.457 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:41.457 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:31.721834+0000", 2024-09-16T20:51:41.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:41.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:31.721834+0000", 2024-09-16T20:51:41.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:41.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:31.721836+0000", 2024-09-16T20:51:41.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.719e-06 2024-09-16T20:51:41.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:41.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:31.721841+0000", 2024-09-16T20:51:41.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.913e-06 2024-09-16T20:51:41.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:41.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:31.721855+0000", 2024-09-16T20:51:41.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3149000000000001e-05 2024-09-16T20:51:41.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:41.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:31.721861+0000", 2024-09-16T20:51:41.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0310000000000004e-06 2024-09-16T20:51:41.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:41.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:31.721930+0000", 2024-09-16T20:51:41.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9739000000000004e-05 2024-09-16T20:51:41.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:41.460 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:31.721939+0000", 2024-09-16T20:51:41.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6170000000000007e-06 2024-09-16T20:51:41.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:41.460 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:32.638931+0000", 2024-09-16T20:51:41.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.91699213700000004 2024-09-16T20:51:41.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:41.460 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:32.639079+0000", 2024-09-16T20:51:41.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000147439 2024-09-16T20:51:41.460 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.461 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:41.461 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.461 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.461 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:41.461 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:41.462 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:51:41.463 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:41.461+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:41.563 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:41.562+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:41.570 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:41.571 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:51:41.571 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:51:41.571 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:41.579 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:51:41.664 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:41.663+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:41.735 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:41.735 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:51:41.735 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:51:41.735 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:51:41.735 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:41.744 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:51:41.789 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:51:41.788+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:51:41.799 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T20:51:41.857 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:41.857 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:51:41.857 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:51:41.857 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:51:41.857 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.857 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12548 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 628736~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:41.858 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757009+0000", 2024-09-16T20:51:41.858 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 111.09797594299999, 2024-09-16T20:51:41.858 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065743956000000006, 2024-09-16T20:51:41.858 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:41.858 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:41.858 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:41.858 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:41.859 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:41.859 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:41.859 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12548 2024-09-16T20:51:41.859 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.859 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:41.859 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.859 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:41.859 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757009+0000", 2024-09-16T20:51:41.859 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.859 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.860 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.860 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:41.860 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757009+0000", 2024-09-16T20:51:41.860 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.860 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.860 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.860 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:41.860 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757010+0000", 2024-09-16T20:51:41.860 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9900000000000001e-07 2024-09-16T20:51:41.860 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.861 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.861 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:41.861 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757012+0000", 2024-09-16T20:51:41.861 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2689999999999998e-06 2024-09-16T20:51:41.861 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.861 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.861 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:41.861 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757022+0000", 2024-09-16T20:51:41.862 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9699999999999994e-06 2024-09-16T20:51:41.862 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.862 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.862 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:41.862 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757028+0000", 2024-09-16T20:51:41.862 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3799999999999999e-06 2024-09-16T20:51:41.862 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.862 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.862 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:41.863 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822552+0000", 2024-09-16T20:51:41.863 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065524281000000004 2024-09-16T20:51:41.863 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.863 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.863 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:41.863 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822613+0000", 2024-09-16T20:51:41.863 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0430999999999998e-05 2024-09-16T20:51:41.863 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.863 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.863 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:41.863 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822753+0000", 2024-09-16T20:51:41.864 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000140226 2024-09-16T20:51:41.864 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.864 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:41.864 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.864 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.864 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.864 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12549 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 633856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:41.864 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:51:41.864 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 111.09794719600001, 2024-09-16T20:51:41.864 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065978518, 2024-09-16T20:51:41.865 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:41.865 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:41.865 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:41.865 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:41.865 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:41.865 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:41.865 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12549 2024-09-16T20:51:41.865 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.865 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:41.865 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.865 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:41.866 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:51:41.866 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.866 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.866 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.866 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:41.866 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:51:41.866 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.866 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.866 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.867 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:41.867 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:51:41.867 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9299999999999999e-07 2024-09-16T20:51:41.867 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.867 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.867 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:41.867 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757040+0000", 2024-09-16T20:51:41.867 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.677e-06 2024-09-16T20:51:41.867 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.867 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.867 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:41.868 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757045+0000", 2024-09-16T20:51:41.868 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8149999999999996e-06 2024-09-16T20:51:41.868 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.868 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.868 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:41.868 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757048+0000", 2024-09-16T20:51:41.868 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1e-06 2024-09-16T20:51:41.868 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.868 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.868 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:41.869 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822796+0000", 2024-09-16T20:51:41.869 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065748081 2024-09-16T20:51:41.869 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.869 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.869 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:41.869 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822867+0000", 2024-09-16T20:51:41.869 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0679000000000005e-05 2024-09-16T20:51:41.870 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.870 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.870 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:41.870 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823016+0000", 2024-09-16T20:51:41.870 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014987299999999999 2024-09-16T20:51:41.870 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.870 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:41.870 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.870 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.870 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.871 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12550 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 638976~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:41.871 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757060+0000", 2024-09-16T20:51:41.871 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 111.097924781, 2024-09-16T20:51:41.871 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066153564999999998, 2024-09-16T20:51:41.871 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:41.871 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:41.871 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:41.871 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:41.871 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:41.871 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:41.871 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12550 2024-09-16T20:51:41.871 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.872 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:41.872 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.872 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:41.872 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757060+0000", 2024-09-16T20:51:41.872 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.872 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.872 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.872 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:41.872 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757060+0000", 2024-09-16T20:51:41.872 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.872 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.872 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.873 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:41.873 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757061+0000", 2024-09-16T20:51:41.873 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.15e-07 2024-09-16T20:51:41.873 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.873 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.873 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:41.873 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757062+0000", 2024-09-16T20:51:41.873 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.663e-06 2024-09-16T20:51:41.873 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.873 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.874 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:41.874 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757066+0000", 2024-09-16T20:51:41.874 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.817e-06 2024-09-16T20:51:41.874 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.874 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.874 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:41.874 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757070+0000", 2024-09-16T20:51:41.874 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1509999999999997e-06 2024-09-16T20:51:41.874 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.875 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.875 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:41.875 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823044+0000", 2024-09-16T20:51:41.875 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065973992999999995 2024-09-16T20:51:41.875 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.875 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.875 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:41.875 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823082+0000", 2024-09-16T20:51:41.875 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8050999999999999e-05 2024-09-16T20:51:41.875 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.875 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.876 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:41.876 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823214+0000", 2024-09-16T20:51:41.876 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013157500000000001 2024-09-16T20:51:41.876 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.876 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:41.876 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.876 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.876 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.876 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12551 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 644096~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:41.876 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:51:41.877 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 111.097911083, 2024-09-16T20:51:41.877 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066391378000000001, 2024-09-16T20:51:41.877 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:41.877 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:41.877 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:41.877 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:41.877 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:41.877 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:41.877 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12551 2024-09-16T20:51:41.877 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.877 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:41.878 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.878 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:41.878 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:51:41.878 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.878 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.878 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.878 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:41.878 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:51:41.879 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.879 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.879 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.879 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:41.879 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:51:41.879 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2999999999999999e-07 2024-09-16T20:51:41.879 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.879 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.879 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:41.879 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757075+0000", 2024-09-16T20:51:41.879 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.097e-06 2024-09-16T20:51:41.879 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.879 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.880 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:41.880 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757082+0000", 2024-09-16T20:51:41.880 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9500000000000004e-06 2024-09-16T20:51:41.880 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.880 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.880 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:41.880 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757086+0000", 2024-09-16T20:51:41.880 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2229999999999999e-06 2024-09-16T20:51:41.880 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.880 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.880 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:41.880 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823262+0000", 2024-09-16T20:51:41.880 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066176078999999999 2024-09-16T20:51:41.881 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.881 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.881 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:41.881 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823336+0000", 2024-09-16T20:51:41.881 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3979000000000004e-05 2024-09-16T20:51:41.881 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.881 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.881 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:41.881 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823465+0000", 2024-09-16T20:51:41.881 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012982 2024-09-16T20:51:41.881 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.881 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:41.881 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.881 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.882 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.882 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12552 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 649216~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:41.882 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757089+0000", 2024-09-16T20:51:41.882 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 111.09789594199999, 2024-09-16T20:51:41.882 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066655617, 2024-09-16T20:51:41.882 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:41.882 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:41.882 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:41.882 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:41.882 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:41.882 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:41.882 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12552 2024-09-16T20:51:41.882 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.882 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:41.883 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.883 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:41.883 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757089+0000", 2024-09-16T20:51:41.883 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.883 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.883 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.883 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:41.883 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757089+0000", 2024-09-16T20:51:41.883 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.883 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.883 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.883 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:41.883 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759492+0000", 2024-09-16T20:51:41.883 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0024026109999999998 2024-09-16T20:51:41.884 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.884 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.884 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:41.884 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759497+0000", 2024-09-16T20:51:41.884 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1120000000000004e-06 2024-09-16T20:51:41.884 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.884 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.884 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:41.884 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759508+0000", 2024-09-16T20:51:41.884 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0981e-05 2024-09-16T20:51:41.884 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.884 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.884 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:41.885 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759516+0000", 2024-09-16T20:51:41.885 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7270000000000007e-06 2024-09-16T20:51:41.885 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.885 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.885 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:41.885 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823504+0000", 2024-09-16T20:51:41.885 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.063987984999999997 2024-09-16T20:51:41.885 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.885 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.885 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:41.885 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823579+0000", 2024-09-16T20:51:41.885 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5703000000000003e-05 2024-09-16T20:51:41.886 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.886 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.886 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:41.886 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823745+0000", 2024-09-16T20:51:41.886 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016549800000000001 2024-09-16T20:51:41.886 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.886 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:41.886 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.886 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.886 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.886 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12560 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 690176~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:41.886 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.759658+0000", 2024-09-16T20:51:41.886 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 111.09532674, 2024-09-16T20:51:41.886 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065858517000000005, 2024-09-16T20:51:41.887 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:41.887 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:41.887 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:41.887 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:41.887 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:41.887 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:41.887 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12560 2024-09-16T20:51:41.887 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.887 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:41.887 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.887 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:41.887 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759658+0000", 2024-09-16T20:51:41.887 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.887 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.888 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.888 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:41.888 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759658+0000", 2024-09-16T20:51:41.888 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.888 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.888 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.888 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:41.888 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759659+0000", 2024-09-16T20:51:41.888 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.17e-07 2024-09-16T20:51:41.888 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.888 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.888 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:41.888 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759660+0000", 2024-09-16T20:51:41.889 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0189999999999999e-06 2024-09-16T20:51:41.889 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.889 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.889 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:41.889 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759663+0000", 2024-09-16T20:51:41.889 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0010000000000002e-06 2024-09-16T20:51:41.889 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.889 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.889 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:41.889 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759665+0000", 2024-09-16T20:51:41.889 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5440000000000001e-06 2024-09-16T20:51:41.889 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.889 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.889 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:41.890 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825355+0000", 2024-09-16T20:51:41.890 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065690224000000005 2024-09-16T20:51:41.890 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.890 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.890 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:41.890 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825394+0000", 2024-09-16T20:51:41.890 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8810999999999998e-05 2024-09-16T20:51:41.890 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.890 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.890 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:41.890 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825517+0000", 2024-09-16T20:51:41.890 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012270099999999999 2024-09-16T20:51:41.890 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.891 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:41.891 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.891 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.891 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.891 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12561 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 695296~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:41.891 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.759668+0000", 2024-09-16T20:51:41.891 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 111.09531689799999, 2024-09-16T20:51:41.891 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066116999999999995, 2024-09-16T20:51:41.891 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:41.891 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:41.891 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:41.891 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:41.891 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:41.892 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:41.892 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12561 2024-09-16T20:51:41.892 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.892 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:41.892 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.892 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:41.892 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759668+0000", 2024-09-16T20:51:41.892 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.892 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.892 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.892 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:41.892 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759668+0000", 2024-09-16T20:51:41.893 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.893 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.893 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.893 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:41.893 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765379+0000", 2024-09-16T20:51:41.893 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057104410000000001 2024-09-16T20:51:41.893 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.893 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.893 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:41.893 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765383+0000", 2024-09-16T20:51:41.893 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2980000000000002e-06 2024-09-16T20:51:41.893 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.893 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.893 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:41.894 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765395+0000", 2024-09-16T20:51:41.894 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1671000000000001e-05 2024-09-16T20:51:41.894 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.894 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.894 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:41.894 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765403+0000", 2024-09-16T20:51:41.894 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1210000000000007e-06 2024-09-16T20:51:41.894 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.894 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.894 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:41.894 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825552+0000", 2024-09-16T20:51:41.894 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.060149498000000003 2024-09-16T20:51:41.894 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.894 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.895 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:41.895 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825625+0000", 2024-09-16T20:51:41.895 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2540999999999995e-05 2024-09-16T20:51:41.895 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.895 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.895 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:41.895 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825785+0000", 2024-09-16T20:51:41.895 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016043000000000001 2024-09-16T20:51:41.895 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.895 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:41.895 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.895 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.895 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.895 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12585 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 818176~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:41.896 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.765848+0000", 2024-09-16T20:51:41.896 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 111.089136927, 2024-09-16T20:51:41.896 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065843218999999995, 2024-09-16T20:51:41.896 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:41.896 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:41.896 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:41.896 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:41.896 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:41.896 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:41.896 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12585 2024-09-16T20:51:41.896 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.896 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:41.896 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.896 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:41.897 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765848+0000", 2024-09-16T20:51:41.897 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.897 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.897 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.897 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:41.897 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765848+0000", 2024-09-16T20:51:41.897 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.897 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.897 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.897 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:41.897 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768683+0000", 2024-09-16T20:51:41.897 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0028344519999999999 2024-09-16T20:51:41.897 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.898 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.898 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:41.898 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768686+0000", 2024-09-16T20:51:41.898 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8789999999999996e-06 2024-09-16T20:51:41.898 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.898 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.898 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:41.898 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768696+0000", 2024-09-16T20:51:41.898 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6709999999999994e-06 2024-09-16T20:51:41.898 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.898 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.898 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:41.898 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768702+0000", 2024-09-16T20:51:41.898 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2550000000000003e-06 2024-09-16T20:51:41.898 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.899 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.899 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:41.899 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.831499+0000", 2024-09-16T20:51:41.899 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.062797081000000005 2024-09-16T20:51:41.899 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.899 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.899 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:41.899 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.831538+0000", 2024-09-16T20:51:41.899 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8523999999999999e-05 2024-09-16T20:51:41.899 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.899 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.899 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:41.899 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.831691+0000", 2024-09-16T20:51:41.900 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015335699999999999 2024-09-16T20:51:41.900 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.900 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:41.900 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.900 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.900 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.900 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12597 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 879616~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:41.900 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.768883+0000", 2024-09-16T20:51:41.900 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 111.086102199, 2024-09-16T20:51:41.900 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065816527, 2024-09-16T20:51:41.900 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:41.900 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:41.900 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:41.900 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:41.901 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:41.901 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:41.901 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12597 2024-09-16T20:51:41.901 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.901 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:41.901 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.901 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:41.901 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768883+0000", 2024-09-16T20:51:41.901 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.901 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.901 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.901 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:41.901 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768883+0000", 2024-09-16T20:51:41.901 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.902 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.902 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:41.902 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771508+0000", 2024-09-16T20:51:41.902 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0026247589999999999 2024-09-16T20:51:41.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.902 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.902 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:41.902 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771513+0000", 2024-09-16T20:51:41.902 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9949999999999996e-06 2024-09-16T20:51:41.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.902 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.902 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:41.902 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771525+0000", 2024-09-16T20:51:41.903 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2033e-05 2024-09-16T20:51:41.903 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.903 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.903 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:41.903 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771532+0000", 2024-09-16T20:51:41.903 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4220000000000003e-06 2024-09-16T20:51:41.903 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.903 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.903 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:41.903 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.834532+0000", 2024-09-16T20:51:41.903 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.063000159 2024-09-16T20:51:41.903 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.903 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.904 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:41.904 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.834590+0000", 2024-09-16T20:51:41.904 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8218999999999999e-05 2024-09-16T20:51:41.904 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.904 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.904 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:41.904 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.834699+0000", 2024-09-16T20:51:41.904 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010894 2024-09-16T20:51:41.904 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.904 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:41.904 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.904 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.904 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.904 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12619 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 992256~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:41.905 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.774656+0000", 2024-09-16T20:51:41.905 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 111.080329425, 2024-09-16T20:51:41.905 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065715859000000001, 2024-09-16T20:51:41.905 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:41.905 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:41.905 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:41.905 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:41.905 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:41.905 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:41.905 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12619 2024-09-16T20:51:41.905 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.905 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:41.905 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.905 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:41.906 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774656+0000", 2024-09-16T20:51:41.906 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.906 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.906 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:41.906 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774656+0000", 2024-09-16T20:51:41.906 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.906 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.906 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:41.906 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774875+0000", 2024-09-16T20:51:41.906 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000218898 2024-09-16T20:51:41.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.907 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.907 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:41.907 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774879+0000", 2024-09-16T20:51:41.907 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0389999999999998e-06 2024-09-16T20:51:41.907 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.907 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.907 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:41.907 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774890+0000", 2024-09-16T20:51:41.907 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1250000000000001e-05 2024-09-16T20:51:41.907 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.907 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.907 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:41.907 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774897+0000", 2024-09-16T20:51:41.907 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7619999999999998e-06 2024-09-16T20:51:41.908 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.908 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.908 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:41.908 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840139+0000", 2024-09-16T20:51:41.908 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065242438 2024-09-16T20:51:41.908 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.908 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.908 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:41.908 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840186+0000", 2024-09-16T20:51:41.908 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6894999999999997e-05 2024-09-16T20:51:41.908 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.908 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.909 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:41.909 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840372+0000", 2024-09-16T20:51:41.909 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000185577 2024-09-16T20:51:41.909 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.909 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:41.909 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.909 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.909 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.909 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12620 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 997376~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:41.909 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.774908+0000", 2024-09-16T20:51:41.909 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 111.08007720400001, 2024-09-16T20:51:41.909 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065782754999999998, 2024-09-16T20:51:41.909 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:41.910 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:41.910 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:41.910 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:41.910 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:41.910 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:41.910 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12620 2024-09-16T20:51:41.910 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.910 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:41.910 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.910 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:41.910 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774908+0000", 2024-09-16T20:51:41.910 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.910 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.911 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.911 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:41.911 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774908+0000", 2024-09-16T20:51:41.911 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.911 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.911 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.911 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:41.911 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775102+0000", 2024-09-16T20:51:41.911 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000194098 2024-09-16T20:51:41.911 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.911 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.911 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:41.911 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775105+0000", 2024-09-16T20:51:41.911 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8739999999999999e-06 2024-09-16T20:51:41.911 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.912 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.912 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:41.912 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775111+0000", 2024-09-16T20:51:41.912 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9519999999999999e-06 2024-09-16T20:51:41.912 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.912 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.912 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:41.912 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775115+0000", 2024-09-16T20:51:41.912 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.002e-06 2024-09-16T20:51:41.912 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.912 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.912 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:41.912 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840409+0000", 2024-09-16T20:51:41.913 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065294195999999999 2024-09-16T20:51:41.913 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.913 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.913 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:41.913 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840507+0000", 2024-09-16T20:51:41.913 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7662000000000001e-05 2024-09-16T20:51:41.913 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.913 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.913 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:41.913 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840691+0000", 2024-09-16T20:51:41.913 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000183971 2024-09-16T20:51:41.913 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.913 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:41.913 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.914 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.914 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.914 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12621 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1002496~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:41.914 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775123+0000", 2024-09-16T20:51:41.914 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 111.07986250800001, 2024-09-16T20:51:41.914 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065829586999999995, 2024-09-16T20:51:41.914 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:41.914 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:41.914 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:41.914 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:41.914 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:41.914 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:41.914 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12621 2024-09-16T20:51:41.914 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.915 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:41.915 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.915 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:41.915 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775123+0000", 2024-09-16T20:51:41.915 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.915 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.915 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.915 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:41.915 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775123+0000", 2024-09-16T20:51:41.915 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.915 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.915 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.915 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:41.915 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775348+0000", 2024-09-16T20:51:41.916 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000225295 2024-09-16T20:51:41.916 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.916 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.916 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:41.916 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775352+0000", 2024-09-16T20:51:41.916 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0810000000000004e-06 2024-09-16T20:51:41.916 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.916 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.916 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:41.916 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775361+0000", 2024-09-16T20:51:41.916 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7399999999999993e-06 2024-09-16T20:51:41.916 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.916 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.917 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:41.917 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775367+0000", 2024-09-16T20:51:41.917 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5200000000000003e-06 2024-09-16T20:51:41.917 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.917 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.917 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:41.917 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840730+0000", 2024-09-16T20:51:41.917 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065362803999999997 2024-09-16T20:51:41.917 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.917 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.917 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:41.917 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840801+0000", 2024-09-16T20:51:41.917 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1006000000000004e-05 2024-09-16T20:51:41.917 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.918 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.918 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:41.918 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840952+0000", 2024-09-16T20:51:41.918 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015114100000000001 2024-09-16T20:51:41.918 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.918 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:41.918 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.918 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.918 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.918 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12622 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1007616~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:41.918 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775381+0000", 2024-09-16T20:51:41.918 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 111.079603674, 2024-09-16T20:51:41.918 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065822581000000005, 2024-09-16T20:51:41.919 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:41.919 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:41.919 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:41.919 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:41.919 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:41.919 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:41.919 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12622 2024-09-16T20:51:41.919 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.919 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:41.919 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.919 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:41.919 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775381+0000", 2024-09-16T20:51:41.919 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.919 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.920 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.920 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:41.920 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775381+0000", 2024-09-16T20:51:41.920 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.920 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.920 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.920 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:41.920 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775563+0000", 2024-09-16T20:51:41.920 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018153700000000001 2024-09-16T20:51:41.920 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.920 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.920 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:41.920 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775566+0000", 2024-09-16T20:51:41.921 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9620000000000002e-06 2024-09-16T20:51:41.921 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.921 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.921 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:41.921 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775572+0000", 2024-09-16T20:51:41.921 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8529999999999997e-06 2024-09-16T20:51:41.921 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.921 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.921 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:41.921 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775576+0000", 2024-09-16T20:51:41.921 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8920000000000002e-06 2024-09-16T20:51:41.921 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.921 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.922 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:41.922 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840990+0000", 2024-09-16T20:51:41.922 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065414599000000004 2024-09-16T20:51:41.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.922 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.922 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:41.922 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841068+0000", 2024-09-16T20:51:41.922 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8157000000000004e-05 2024-09-16T20:51:41.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.922 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.922 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:41.922 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841204+0000", 2024-09-16T20:51:41.922 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013558099999999999 2024-09-16T20:51:41.922 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.923 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:41.923 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.923 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.923 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.923 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12623 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1012736~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:41.923 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775583+0000", 2024-09-16T20:51:41.923 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 111.07940223600001, 2024-09-16T20:51:41.923 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065929087999999997, 2024-09-16T20:51:41.923 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:41.923 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:41.923 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:41.923 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:41.923 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:41.923 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:41.924 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12623 2024-09-16T20:51:41.924 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.924 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:41.924 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.924 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:41.924 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775583+0000", 2024-09-16T20:51:41.924 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.924 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.924 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.924 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:41.924 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775583+0000", 2024-09-16T20:51:41.925 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.925 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.925 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.925 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:41.925 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775804+0000", 2024-09-16T20:51:41.925 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00022116900000000001 2024-09-16T20:51:41.925 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.925 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.925 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:41.925 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775807+0000", 2024-09-16T20:51:41.925 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9529999999999999e-06 2024-09-16T20:51:41.925 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.925 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.926 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:41.926 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775814+0000", 2024-09-16T20:51:41.926 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7490000000000001e-06 2024-09-16T20:51:41.926 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.926 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.926 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:41.926 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775818+0000", 2024-09-16T20:51:41.926 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1999999999999996e-06 2024-09-16T20:51:41.926 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.926 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.926 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:41.926 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841267+0000", 2024-09-16T20:51:41.926 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065449127999999995 2024-09-16T20:51:41.926 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.927 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.927 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:41.927 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841344+0000", 2024-09-16T20:51:41.927 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6546999999999997e-05 2024-09-16T20:51:41.927 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.927 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.927 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:41.927 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841512+0000", 2024-09-16T20:51:41.927 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016834200000000001 2024-09-16T20:51:41.927 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.927 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:41.927 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.927 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.928 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.928 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12624 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1017856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:41.928 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:51:41.928 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 111.079159096, 2024-09-16T20:51:41.928 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065945866000000006, 2024-09-16T20:51:41.928 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:41.928 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:41.928 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:41.928 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:41.928 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:41.928 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:41.928 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12624 2024-09-16T20:51:41.928 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.929 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:41.929 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.929 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:41.929 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:51:41.929 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.929 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.929 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.929 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:41.929 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:51:41.929 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.929 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.929 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.929 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:41.929 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:51:41.930 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-16T20:51:41.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.930 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.930 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:41.930 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775828+0000", 2024-09-16T20:51:41.930 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3999999999999999e-06 2024-09-16T20:51:41.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.930 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.930 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:41.930 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775831+0000", 2024-09-16T20:51:41.930 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.478e-06 2024-09-16T20:51:41.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.930 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.930 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:41.931 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775834+0000", 2024-09-16T20:51:41.931 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9440000000000001e-06 2024-09-16T20:51:41.931 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.931 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.931 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:41.931 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841547+0000", 2024-09-16T20:51:41.931 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065712801000000001 2024-09-16T20:51:41.931 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.931 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.931 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:41.931 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841618+0000", 2024-09-16T20:51:41.931 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1151999999999999e-05 2024-09-16T20:51:41.931 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.931 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.932 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:41.932 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841772+0000", 2024-09-16T20:51:41.932 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000153721 2024-09-16T20:51:41.932 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.932 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:41.932 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.932 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.932 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.932 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12625 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1022976~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:41.932 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775840+0000", 2024-09-16T20:51:41.932 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 111.07914548700001, 2024-09-16T20:51:41.932 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066165451, 2024-09-16T20:51:41.932 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:41.933 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:41.933 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:41.933 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:41.933 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:41.933 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:41.933 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12625 2024-09-16T20:51:41.933 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.933 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:41.933 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.933 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:41.933 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775840+0000", 2024-09-16T20:51:41.933 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.933 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.933 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.934 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:41.934 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775840+0000", 2024-09-16T20:51:41.934 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.934 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.934 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.934 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:41.934 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776014+0000", 2024-09-16T20:51:41.934 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017441099999999999 2024-09-16T20:51:41.934 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.934 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.934 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:41.934 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776018+0000", 2024-09-16T20:51:41.934 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6509999999999999e-06 2024-09-16T20:51:41.934 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.935 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.935 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:41.935 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776026+0000", 2024-09-16T20:51:41.935 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.028e-06 2024-09-16T20:51:41.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.935 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.935 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:41.935 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776031+0000", 2024-09-16T20:51:41.935 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8400000000000002e-06 2024-09-16T20:51:41.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.935 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.935 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:41.935 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841801+0000", 2024-09-16T20:51:41.935 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065770472999999996 2024-09-16T20:51:41.936 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.936 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.936 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:41.936 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841847+0000", 2024-09-16T20:51:41.936 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5558e-05 2024-09-16T20:51:41.936 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.936 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.936 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:41.936 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842005+0000", 2024-09-16T20:51:41.936 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015849000000000001 2024-09-16T20:51:41.936 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.936 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:41.936 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.937 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.937 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.937 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12626 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1028096~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:41.937 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776041+0000", 2024-09-16T20:51:41.937 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 111.07894418799999, 2024-09-16T20:51:41.937 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066225260999999994, 2024-09-16T20:51:41.937 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:41.937 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:41.937 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:41.937 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:41.937 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:41.937 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:41.937 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12626 2024-09-16T20:51:41.937 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.938 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:41.938 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.938 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:41.938 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776041+0000", 2024-09-16T20:51:41.938 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.938 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.938 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.938 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:41.938 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776041+0000", 2024-09-16T20:51:41.938 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.938 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.938 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.938 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:41.939 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776263+0000", 2024-09-16T20:51:41.939 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00022193899999999999 2024-09-16T20:51:41.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.939 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:41.939 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776268+0000", 2024-09-16T20:51:41.939 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8049999999999997e-06 2024-09-16T20:51:41.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.939 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:41.939 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776280+0000", 2024-09-16T20:51:41.939 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2635e-05 2024-09-16T20:51:41.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:41.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776288+0000", 2024-09-16T20:51:41.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8199999999999997e-06 2024-09-16T20:51:41.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:41.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842041+0000", 2024-09-16T20:51:41.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065752506000000002 2024-09-16T20:51:41.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:41.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842123+0000", 2024-09-16T20:51:41.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2393999999999998e-05 2024-09-16T20:51:41.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.941 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.941 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:41.941 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842266+0000", 2024-09-16T20:51:41.941 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014316200000000001 2024-09-16T20:51:41.941 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.941 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:41.941 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.941 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.941 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.941 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12627 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1033216~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:41.941 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776301+0000", 2024-09-16T20:51:41.941 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 111.078684228, 2024-09-16T20:51:41.941 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066187814999999997, 2024-09-16T20:51:41.942 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:41.942 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:41.942 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:41.942 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:41.942 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:41.942 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:41.942 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12627 2024-09-16T20:51:41.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.942 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:41.942 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.942 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:41.942 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776301+0000", 2024-09-16T20:51:41.942 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.943 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:41.943 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776301+0000", 2024-09-16T20:51:41.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.943 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:41.943 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776473+0000", 2024-09-16T20:51:41.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017222400000000001 2024-09-16T20:51:41.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.943 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:41.943 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776477+0000", 2024-09-16T20:51:41.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7019999999999999e-06 2024-09-16T20:51:41.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.944 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:41.944 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776486+0000", 2024-09-16T20:51:41.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2119999999999992e-06 2024-09-16T20:51:41.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.944 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:41.944 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776492+0000", 2024-09-16T20:51:41.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8089999999999998e-06 2024-09-16T20:51:41.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.945 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:41.945 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842301+0000", 2024-09-16T20:51:41.945 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065809042999999998 2024-09-16T20:51:41.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.945 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:41.945 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842362+0000", 2024-09-16T20:51:41.945 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1260000000000006e-05 2024-09-16T20:51:41.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.945 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:41.945 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842489+0000", 2024-09-16T20:51:41.946 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012656500000000001 2024-09-16T20:51:41.946 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.946 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:41.946 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.946 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.946 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12628 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1038336~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:41.946 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776502+0000", 2024-09-16T20:51:41.946 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 111.078483582, 2024-09-16T20:51:41.946 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066283426000000006, 2024-09-16T20:51:41.946 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:41.946 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:41.946 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:41.946 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:41.947 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:41.947 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:41.947 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12628 2024-09-16T20:51:41.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.947 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:41.947 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.947 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:41.947 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776502+0000", 2024-09-16T20:51:41.947 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.947 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.947 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:41.947 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776502+0000", 2024-09-16T20:51:41.947 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.948 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:41.948 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776716+0000", 2024-09-16T20:51:41.948 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000214596 2024-09-16T20:51:41.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.948 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:41.948 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776723+0000", 2024-09-16T20:51:41.948 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4000000000000003e-06 2024-09-16T20:51:41.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.948 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:41.948 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776731+0000", 2024-09-16T20:51:41.948 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6129999999999998e-06 2024-09-16T20:51:41.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.949 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.949 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:41.949 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776737+0000", 2024-09-16T20:51:41.949 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1970000000000001e-06 2024-09-16T20:51:41.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.949 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.949 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:41.949 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842532+0000", 2024-09-16T20:51:41.949 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065795127999999994 2024-09-16T20:51:41.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.949 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.949 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:41.950 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842599+0000", 2024-09-16T20:51:41.950 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6711000000000006e-05 2024-09-16T20:51:41.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.950 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.950 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:41.950 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842785+0000", 2024-09-16T20:51:41.950 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018578099999999999 2024-09-16T20:51:41.950 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.950 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:41.950 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.950 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.950 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12629 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1043456~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:41.950 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776748+0000", 2024-09-16T20:51:41.950 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 111.07823720499999, 2024-09-16T20:51:41.950 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066303788000000002, 2024-09-16T20:51:41.951 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:41.951 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:41.951 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:41.951 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:41.951 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:41.951 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:41.951 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12629 2024-09-16T20:51:41.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.951 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:41.951 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.951 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:41.951 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776748+0000", 2024-09-16T20:51:41.951 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.952 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.952 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:41.952 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776748+0000", 2024-09-16T20:51:41.952 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:41.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.952 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.952 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:41.952 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776934+0000", 2024-09-16T20:51:41.952 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018598399999999999 2024-09-16T20:51:41.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.952 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.952 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:41.952 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776939+0000", 2024-09-16T20:51:41.952 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1000000000000003e-06 2024-09-16T20:51:41.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.953 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.953 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:41.953 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776954+0000", 2024-09-16T20:51:41.953 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5221e-05 2024-09-16T20:51:41.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.953 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.953 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:41.953 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776962+0000", 2024-09-16T20:51:41.953 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.554e-06 2024-09-16T20:51:41.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.953 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.953 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:41.953 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842813+0000", 2024-09-16T20:51:41.953 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065851006000000004 2024-09-16T20:51:41.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.955 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:41.955 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842886+0000", 2024-09-16T20:51:41.956 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3663000000000003e-05 2024-09-16T20:51:41.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:41.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:41.956 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:41.956 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.843052+0000", 2024-09-16T20:51:41.956 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016526 2024-09-16T20:51:41.956 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.956 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:41.956 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.956 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:41.956 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:41.956 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:41.958 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:51:41.959 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:51:41.959 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:51:41.959 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:51:41.960 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:51:41.960 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:41.956+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:41.961 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:51:41.961 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:51:41.975 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:51:41.975 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:51:41.992 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:51:41.992 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:51:42.009 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:51:42.010 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:51:42.061 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:42.060+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:42.108 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:42.108 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:42.117 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:51:42.160 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:42.160+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:42.224 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:42.225 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:42.233 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:51:42.261 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:42.260+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:42.340 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:42.340 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:42.349 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:51:42.393 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:42.392+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:42.460 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:42.460 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:42.463 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:42.462+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:42.469 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:51:42.564 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:42.563+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:42.616 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:42.616 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:42.625 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:51:42.665 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:51:42.663+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:51:42.682 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T20:51:42.743 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:42.743 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:42.752 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:51:42.795 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:42.794+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:42.866 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:42.865+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:42.867 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:42.867 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:42.876 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:51:42.967 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:51:42.966+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:51:43.024 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:43.024 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:43.032 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:51:43.076 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:43.075+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:43.139 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:43.139 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:43.148 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:51:43.169 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:51:43.168+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:51:43.270 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:43.269+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:43.295 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:43.296 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:43.304 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:51:43.388 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:43.387+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:43.451 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:43.451 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:43.460 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:51:43.501 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:43.500+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:43.572 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:43.571+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:43.608 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:43.608 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:43.616 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:51:43.669 INFO:teuthology.orchestra.run.smithi029.stderr:noscrub is set 2024-09-16T20:51:43.673 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:43.672+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:43.745 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:43.745 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:43.755 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:51:43.798 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:43.797+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:43.866 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:43.867 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:43.875 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:51:43.875 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-16T20:51:43.873+0000 7f2cca4f1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32260) UID: 0 2024-09-16T20:51:43.976 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:43.975+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:44.021 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T20:51:44.022 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:44.023 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:44.031 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:51:44.076 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:44.075+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:44.138 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-16T20:51:44.138 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T20:51:44.144 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-16T20:51:44.144 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T20:51:44.147 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:51:44.161 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-16T20:51:44.161 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T20:51:44.177 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-16T20:51:44.177 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T20:51:44.190 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:44.189+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:44.194 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-16T20:51:44.194 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T20:51:44.211 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-16T20:51:44.211 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T20:51:44.254 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:44.254 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:51:44.254 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:51:44.254 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:51:44.254 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:44.263 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:51:44.278 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:44.277+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:44.379 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:44.378+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:44.412 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:44.412 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:51:44.412 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:51:44.413 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:51:44.413 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.413 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12548 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 628736~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:44.413 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757009+0000", 2024-09-16T20:51:44.413 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 113.65384207300001, 2024-09-16T20:51:44.413 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065743956000000006, 2024-09-16T20:51:44.413 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:44.413 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:44.413 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:44.413 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:44.413 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:44.413 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:44.413 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12548 2024-09-16T20:51:44.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.414 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:44.414 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.414 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:44.414 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757009+0000", 2024-09-16T20:51:44.414 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.414 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.414 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:44.414 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757009+0000", 2024-09-16T20:51:44.414 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.414 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.414 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:44.415 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757010+0000", 2024-09-16T20:51:44.415 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9900000000000001e-07 2024-09-16T20:51:44.415 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.415 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.415 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:44.415 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757012+0000", 2024-09-16T20:51:44.415 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2689999999999998e-06 2024-09-16T20:51:44.415 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.415 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.415 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:44.415 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757022+0000", 2024-09-16T20:51:44.415 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9699999999999994e-06 2024-09-16T20:51:44.415 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.415 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.416 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:44.416 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757028+0000", 2024-09-16T20:51:44.416 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3799999999999999e-06 2024-09-16T20:51:44.416 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.416 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.416 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:44.416 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822552+0000", 2024-09-16T20:51:44.416 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065524281000000004 2024-09-16T20:51:44.416 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.416 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.416 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:44.417 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822613+0000", 2024-09-16T20:51:44.417 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0430999999999998e-05 2024-09-16T20:51:44.417 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.417 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.417 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:44.417 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822753+0000", 2024-09-16T20:51:44.417 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000140226 2024-09-16T20:51:44.417 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.417 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:44.418 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.418 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.418 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12549 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 633856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:44.418 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:51:44.418 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 113.65381332600001, 2024-09-16T20:51:44.418 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065978518, 2024-09-16T20:51:44.418 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:44.418 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:44.418 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:44.418 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:44.418 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:44.418 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:44.418 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12549 2024-09-16T20:51:44.419 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.419 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:44.419 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.419 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:44.419 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:51:44.419 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.419 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.419 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.419 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:44.419 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:51:44.419 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.419 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.419 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.419 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:44.420 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:51:44.420 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9299999999999999e-07 2024-09-16T20:51:44.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.420 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.420 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:44.420 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757040+0000", 2024-09-16T20:51:44.420 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.677e-06 2024-09-16T20:51:44.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.420 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.420 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:44.420 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757045+0000", 2024-09-16T20:51:44.420 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8149999999999996e-06 2024-09-16T20:51:44.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.420 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.421 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:44.421 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757048+0000", 2024-09-16T20:51:44.421 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1e-06 2024-09-16T20:51:44.421 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.421 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.421 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:44.421 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822796+0000", 2024-09-16T20:51:44.421 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065748081 2024-09-16T20:51:44.421 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.421 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.421 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:44.421 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822867+0000", 2024-09-16T20:51:44.422 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0679000000000005e-05 2024-09-16T20:51:44.422 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.422 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.422 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:44.422 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823016+0000", 2024-09-16T20:51:44.422 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014987299999999999 2024-09-16T20:51:44.422 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.422 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:44.422 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.422 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.422 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.422 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12550 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 638976~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:44.423 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757060+0000", 2024-09-16T20:51:44.423 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 113.653790911, 2024-09-16T20:51:44.423 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066153564999999998, 2024-09-16T20:51:44.423 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:44.423 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:44.423 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:44.423 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:44.423 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:44.423 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:44.423 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12550 2024-09-16T20:51:44.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.423 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:44.424 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:44.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757060+0000", 2024-09-16T20:51:44.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.424 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:44.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757060+0000", 2024-09-16T20:51:44.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.424 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:44.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757061+0000", 2024-09-16T20:51:44.425 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.15e-07 2024-09-16T20:51:44.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.425 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:44.425 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757062+0000", 2024-09-16T20:51:44.425 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.663e-06 2024-09-16T20:51:44.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.425 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:44.425 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757066+0000", 2024-09-16T20:51:44.425 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.817e-06 2024-09-16T20:51:44.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.426 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:44.426 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757070+0000", 2024-09-16T20:51:44.426 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1509999999999997e-06 2024-09-16T20:51:44.426 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.426 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.426 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:44.426 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823044+0000", 2024-09-16T20:51:44.426 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065973992999999995 2024-09-16T20:51:44.426 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.426 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.426 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:44.426 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823082+0000", 2024-09-16T20:51:44.426 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8050999999999999e-05 2024-09-16T20:51:44.426 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.426 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.427 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:44.427 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823214+0000", 2024-09-16T20:51:44.427 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013157500000000001 2024-09-16T20:51:44.427 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.427 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:44.427 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.427 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.427 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.427 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12551 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 644096~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:44.427 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:51:44.427 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 113.653777213, 2024-09-16T20:51:44.427 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066391378000000001, 2024-09-16T20:51:44.427 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:44.428 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:44.428 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:44.428 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:44.428 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:44.428 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:44.428 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12551 2024-09-16T20:51:44.428 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.428 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:44.428 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.428 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:44.428 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:51:44.428 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.429 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.429 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:44.429 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:51:44.429 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.429 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.429 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:44.429 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:51:44.429 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2999999999999999e-07 2024-09-16T20:51:44.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.429 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.429 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:44.429 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757075+0000", 2024-09-16T20:51:44.430 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.097e-06 2024-09-16T20:51:44.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.430 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.430 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:44.430 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757082+0000", 2024-09-16T20:51:44.430 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9500000000000004e-06 2024-09-16T20:51:44.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.430 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.430 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:44.430 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757086+0000", 2024-09-16T20:51:44.430 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2229999999999999e-06 2024-09-16T20:51:44.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.430 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.430 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:44.431 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823262+0000", 2024-09-16T20:51:44.431 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066176078999999999 2024-09-16T20:51:44.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.431 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.431 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:44.431 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823336+0000", 2024-09-16T20:51:44.431 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3979000000000004e-05 2024-09-16T20:51:44.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.431 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.431 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:44.431 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823465+0000", 2024-09-16T20:51:44.431 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012982 2024-09-16T20:51:44.431 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.431 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:44.432 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.432 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.432 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12552 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 649216~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:44.432 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757089+0000", 2024-09-16T20:51:44.432 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 113.65376207200001, 2024-09-16T20:51:44.432 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066655617, 2024-09-16T20:51:44.432 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:44.432 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:44.433 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:44.433 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:44.433 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:44.433 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:44.433 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12552 2024-09-16T20:51:44.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.433 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:44.433 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.433 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:44.433 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757089+0000", 2024-09-16T20:51:44.433 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.433 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.434 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:44.434 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757089+0000", 2024-09-16T20:51:44.434 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.434 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.434 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:44.434 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759492+0000", 2024-09-16T20:51:44.434 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0024026109999999998 2024-09-16T20:51:44.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.434 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.434 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:44.434 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759497+0000", 2024-09-16T20:51:44.434 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1120000000000004e-06 2024-09-16T20:51:44.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.435 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.435 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:44.435 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759508+0000", 2024-09-16T20:51:44.435 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0981e-05 2024-09-16T20:51:44.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.435 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.435 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:44.435 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759516+0000", 2024-09-16T20:51:44.435 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7270000000000007e-06 2024-09-16T20:51:44.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.435 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.435 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:44.435 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823504+0000", 2024-09-16T20:51:44.436 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.063987984999999997 2024-09-16T20:51:44.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.436 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.436 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:44.436 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823579+0000", 2024-09-16T20:51:44.436 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5703000000000003e-05 2024-09-16T20:51:44.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.436 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.436 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:44.436 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823745+0000", 2024-09-16T20:51:44.436 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016549800000000001 2024-09-16T20:51:44.436 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.436 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:44.436 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.437 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12560 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 690176~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:44.437 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.759658+0000", 2024-09-16T20:51:44.437 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 113.65119287, 2024-09-16T20:51:44.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065858517000000005, 2024-09-16T20:51:44.437 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:44.437 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:44.437 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:44.437 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:44.437 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:44.437 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:44.438 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12560 2024-09-16T20:51:44.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.438 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:44.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:44.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759658+0000", 2024-09-16T20:51:44.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:44.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759658+0000", 2024-09-16T20:51:44.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.439 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.439 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:44.439 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759659+0000", 2024-09-16T20:51:44.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.17e-07 2024-09-16T20:51:44.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.439 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.439 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:44.439 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759660+0000", 2024-09-16T20:51:44.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0189999999999999e-06 2024-09-16T20:51:44.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.439 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.439 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:44.439 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759663+0000", 2024-09-16T20:51:44.440 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0010000000000002e-06 2024-09-16T20:51:44.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.440 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:44.440 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759665+0000", 2024-09-16T20:51:44.440 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5440000000000001e-06 2024-09-16T20:51:44.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.440 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:44.440 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825355+0000", 2024-09-16T20:51:44.440 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065690224000000005 2024-09-16T20:51:44.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.440 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:44.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825394+0000", 2024-09-16T20:51:44.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8810999999999998e-05 2024-09-16T20:51:44.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:44.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825517+0000", 2024-09-16T20:51:44.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012270099999999999 2024-09-16T20:51:44.441 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.441 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:44.441 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.441 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12561 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 695296~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:44.442 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.759668+0000", 2024-09-16T20:51:44.442 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 113.65118302800001, 2024-09-16T20:51:44.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066116999999999995, 2024-09-16T20:51:44.442 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:44.442 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:44.442 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:44.442 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:44.442 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:44.442 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:44.442 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12561 2024-09-16T20:51:44.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.442 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:44.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:44.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759668+0000", 2024-09-16T20:51:44.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.443 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:44.443 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759668+0000", 2024-09-16T20:51:44.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.443 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:44.443 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765379+0000", 2024-09-16T20:51:44.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057104410000000001 2024-09-16T20:51:44.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:44.444 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765383+0000", 2024-09-16T20:51:44.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2980000000000002e-06 2024-09-16T20:51:44.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:44.444 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765395+0000", 2024-09-16T20:51:44.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1671000000000001e-05 2024-09-16T20:51:44.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:44.444 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765403+0000", 2024-09-16T20:51:44.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1210000000000007e-06 2024-09-16T20:51:44.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:44.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825552+0000", 2024-09-16T20:51:44.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.060149498000000003 2024-09-16T20:51:44.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:44.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825625+0000", 2024-09-16T20:51:44.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2540999999999995e-05 2024-09-16T20:51:44.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:44.446 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825785+0000", 2024-09-16T20:51:44.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016043000000000001 2024-09-16T20:51:44.446 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.446 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:44.446 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.446 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12585 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 818176~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:44.446 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.765848+0000", 2024-09-16T20:51:44.446 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 113.645003057, 2024-09-16T20:51:44.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065843218999999995, 2024-09-16T20:51:44.446 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:44.446 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:44.446 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:44.447 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:44.447 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:44.447 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:44.447 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12585 2024-09-16T20:51:44.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.447 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:44.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:44.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765848+0000", 2024-09-16T20:51:44.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:44.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765848+0000", 2024-09-16T20:51:44.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:44.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768683+0000", 2024-09-16T20:51:44.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0028344519999999999 2024-09-16T20:51:44.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:44.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768686+0000", 2024-09-16T20:51:44.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8789999999999996e-06 2024-09-16T20:51:44.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.449 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:44.449 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768696+0000", 2024-09-16T20:51:44.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6709999999999994e-06 2024-09-16T20:51:44.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.449 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:44.449 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768702+0000", 2024-09-16T20:51:44.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2550000000000003e-06 2024-09-16T20:51:44.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.449 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:44.449 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.831499+0000", 2024-09-16T20:51:44.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.062797081000000005 2024-09-16T20:51:44.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.450 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:44.450 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.831538+0000", 2024-09-16T20:51:44.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8523999999999999e-05 2024-09-16T20:51:44.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.450 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:44.450 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.831691+0000", 2024-09-16T20:51:44.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015335699999999999 2024-09-16T20:51:44.450 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.450 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:44.450 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.451 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12597 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 879616~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:44.451 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.768883+0000", 2024-09-16T20:51:44.451 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 113.64196832899999, 2024-09-16T20:51:44.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065816527, 2024-09-16T20:51:44.451 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:44.451 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:44.451 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:44.451 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:44.451 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:44.451 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:44.451 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12597 2024-09-16T20:51:44.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.452 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:44.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:44.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768883+0000", 2024-09-16T20:51:44.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:44.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768883+0000", 2024-09-16T20:51:44.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:44.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771508+0000", 2024-09-16T20:51:44.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0026247589999999999 2024-09-16T20:51:44.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:44.453 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771513+0000", 2024-09-16T20:51:44.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9949999999999996e-06 2024-09-16T20:51:44.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:44.453 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771525+0000", 2024-09-16T20:51:44.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2033e-05 2024-09-16T20:51:44.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:44.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771532+0000", 2024-09-16T20:51:44.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4220000000000003e-06 2024-09-16T20:51:44.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:44.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.834532+0000", 2024-09-16T20:51:44.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.063000159 2024-09-16T20:51:44.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:44.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.834590+0000", 2024-09-16T20:51:44.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8218999999999999e-05 2024-09-16T20:51:44.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:44.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.834699+0000", 2024-09-16T20:51:44.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010894 2024-09-16T20:51:44.455 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.455 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:44.455 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.455 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12619 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 992256~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:44.455 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.774656+0000", 2024-09-16T20:51:44.455 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 113.636195555, 2024-09-16T20:51:44.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065715859000000001, 2024-09-16T20:51:44.456 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:44.456 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:44.456 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:44.456 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:44.456 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:44.456 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:44.456 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12619 2024-09-16T20:51:44.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.456 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:44.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.456 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:44.456 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774656+0000", 2024-09-16T20:51:44.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.457 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:44.457 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774656+0000", 2024-09-16T20:51:44.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.457 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:44.457 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774875+0000", 2024-09-16T20:51:44.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000218898 2024-09-16T20:51:44.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.457 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:44.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774879+0000", 2024-09-16T20:51:44.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0389999999999998e-06 2024-09-16T20:51:44.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:44.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774890+0000", 2024-09-16T20:51:44.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1250000000000001e-05 2024-09-16T20:51:44.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:44.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774897+0000", 2024-09-16T20:51:44.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7619999999999998e-06 2024-09-16T20:51:44.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:44.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840139+0000", 2024-09-16T20:51:44.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065242438 2024-09-16T20:51:44.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:44.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840186+0000", 2024-09-16T20:51:44.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6894999999999997e-05 2024-09-16T20:51:44.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:44.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840372+0000", 2024-09-16T20:51:44.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000185577 2024-09-16T20:51:44.460 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.460 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:44.460 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.460 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12620 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 997376~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:44.460 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.774908+0000", 2024-09-16T20:51:44.460 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 113.635943334, 2024-09-16T20:51:44.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065782754999999998, 2024-09-16T20:51:44.460 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:44.460 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:44.460 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:44.461 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:44.461 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:44.461 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:44.461 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12620 2024-09-16T20:51:44.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.461 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:44.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:44.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774908+0000", 2024-09-16T20:51:44.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:44.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774908+0000", 2024-09-16T20:51:44.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.462 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:44.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775102+0000", 2024-09-16T20:51:44.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000194098 2024-09-16T20:51:44.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.462 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:44.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775105+0000", 2024-09-16T20:51:44.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8739999999999999e-06 2024-09-16T20:51:44.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.463 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:44.463 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775111+0000", 2024-09-16T20:51:44.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9519999999999999e-06 2024-09-16T20:51:44.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.463 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:44.463 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775115+0000", 2024-09-16T20:51:44.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.002e-06 2024-09-16T20:51:44.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.463 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:44.463 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840409+0000", 2024-09-16T20:51:44.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065294195999999999 2024-09-16T20:51:44.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.464 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:44.464 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840507+0000", 2024-09-16T20:51:44.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7662000000000001e-05 2024-09-16T20:51:44.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.464 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:44.464 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840691+0000", 2024-09-16T20:51:44.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000183971 2024-09-16T20:51:44.464 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.464 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:44.464 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.465 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12621 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1002496~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:44.465 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775123+0000", 2024-09-16T20:51:44.465 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 113.635728638, 2024-09-16T20:51:44.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065829586999999995, 2024-09-16T20:51:44.465 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:44.465 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:44.465 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:44.465 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:44.465 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:44.465 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:44.465 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12621 2024-09-16T20:51:44.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.465 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:44.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:44.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775123+0000", 2024-09-16T20:51:44.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:44.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775123+0000", 2024-09-16T20:51:44.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:44.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775348+0000", 2024-09-16T20:51:44.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000225295 2024-09-16T20:51:44.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.467 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:44.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775352+0000", 2024-09-16T20:51:44.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0810000000000004e-06 2024-09-16T20:51:44.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.467 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:44.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775361+0000", 2024-09-16T20:51:44.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7399999999999993e-06 2024-09-16T20:51:44.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.467 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:44.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775367+0000", 2024-09-16T20:51:44.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5200000000000003e-06 2024-09-16T20:51:44.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:44.468 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840730+0000", 2024-09-16T20:51:44.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065362803999999997 2024-09-16T20:51:44.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:44.468 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840801+0000", 2024-09-16T20:51:44.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1006000000000004e-05 2024-09-16T20:51:44.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:44.469 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840952+0000", 2024-09-16T20:51:44.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015114100000000001 2024-09-16T20:51:44.469 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.469 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:44.469 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.469 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12622 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1007616~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:44.469 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775381+0000", 2024-09-16T20:51:44.469 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 113.635469804, 2024-09-16T20:51:44.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065822581000000005, 2024-09-16T20:51:44.469 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:44.469 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:44.470 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:44.470 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:44.470 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:44.470 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:44.470 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12622 2024-09-16T20:51:44.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.470 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:44.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:44.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775381+0000", 2024-09-16T20:51:44.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:44.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775381+0000", 2024-09-16T20:51:44.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:44.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775563+0000", 2024-09-16T20:51:44.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018153700000000001 2024-09-16T20:51:44.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:44.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775566+0000", 2024-09-16T20:51:44.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9620000000000002e-06 2024-09-16T20:51:44.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:44.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775572+0000", 2024-09-16T20:51:44.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8529999999999997e-06 2024-09-16T20:51:44.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:44.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775576+0000", 2024-09-16T20:51:44.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8920000000000002e-06 2024-09-16T20:51:44.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:44.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840990+0000", 2024-09-16T20:51:44.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065414599000000004 2024-09-16T20:51:44.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:44.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841068+0000", 2024-09-16T20:51:44.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8157000000000004e-05 2024-09-16T20:51:44.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:44.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841204+0000", 2024-09-16T20:51:44.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013558099999999999 2024-09-16T20:51:44.473 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.474 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:44.474 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.474 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12623 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1012736~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:44.474 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775583+0000", 2024-09-16T20:51:44.474 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 113.63526836600001, 2024-09-16T20:51:44.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065929087999999997, 2024-09-16T20:51:44.474 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:44.474 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:44.474 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12623 2024-09-16T20:51:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775583+0000", 2024-09-16T20:51:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775583+0000", 2024-09-16T20:51:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.476 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:44.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775804+0000", 2024-09-16T20:51:44.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00022116900000000001 2024-09-16T20:51:44.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.476 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:44.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775807+0000", 2024-09-16T20:51:44.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9529999999999999e-06 2024-09-16T20:51:44.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.476 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:44.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775814+0000", 2024-09-16T20:51:44.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7490000000000001e-06 2024-09-16T20:51:44.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:44.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775818+0000", 2024-09-16T20:51:44.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1999999999999996e-06 2024-09-16T20:51:44.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:44.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841267+0000", 2024-09-16T20:51:44.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065449127999999995 2024-09-16T20:51:44.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:44.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841344+0000", 2024-09-16T20:51:44.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6546999999999997e-05 2024-09-16T20:51:44.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:44.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841512+0000", 2024-09-16T20:51:44.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016834200000000001 2024-09-16T20:51:44.478 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.478 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:44.478 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.478 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12624 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1017856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:44.478 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:51:44.479 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 113.635025226, 2024-09-16T20:51:44.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065945866000000006, 2024-09-16T20:51:44.479 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:44.479 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:44.479 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:44.479 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:44.479 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:44.479 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:44.479 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12624 2024-09-16T20:51:44.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.479 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:44.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:44.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:51:44.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:44.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:51:44.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:44.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:51:44.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-16T20:51:44.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:44.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775828+0000", 2024-09-16T20:51:44.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3999999999999999e-06 2024-09-16T20:51:44.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.481 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:44.481 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775831+0000", 2024-09-16T20:51:44.481 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.478e-06 2024-09-16T20:51:44.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.481 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:44.481 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775834+0000", 2024-09-16T20:51:44.481 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9440000000000001e-06 2024-09-16T20:51:44.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.482 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:44.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841547+0000", 2024-09-16T20:51:44.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065712801000000001 2024-09-16T20:51:44.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.482 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:44.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841618+0000", 2024-09-16T20:51:44.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1151999999999999e-05 2024-09-16T20:51:44.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.482 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:44.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841772+0000", 2024-09-16T20:51:44.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000153721 2024-09-16T20:51:44.482 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.483 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:44.483 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.483 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12625 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1022976~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:44.483 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775840+0000", 2024-09-16T20:51:44.483 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 113.635011617, 2024-09-16T20:51:44.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066165451, 2024-09-16T20:51:44.483 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:44.483 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:44.483 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:44.483 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:44.483 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:44.483 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:44.483 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12625 2024-09-16T20:51:44.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.484 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:44.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:44.484 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775840+0000", 2024-09-16T20:51:44.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:44.484 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775840+0000", 2024-09-16T20:51:44.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.485 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:44.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776014+0000", 2024-09-16T20:51:44.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017441099999999999 2024-09-16T20:51:44.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.485 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:44.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776018+0000", 2024-09-16T20:51:44.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6509999999999999e-06 2024-09-16T20:51:44.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.485 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:44.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776026+0000", 2024-09-16T20:51:44.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.028e-06 2024-09-16T20:51:44.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.486 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:44.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776031+0000", 2024-09-16T20:51:44.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8400000000000002e-06 2024-09-16T20:51:44.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.486 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:44.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841801+0000", 2024-09-16T20:51:44.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065770472999999996 2024-09-16T20:51:44.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.486 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:44.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841847+0000", 2024-09-16T20:51:44.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5558e-05 2024-09-16T20:51:44.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:44.487 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842005+0000", 2024-09-16T20:51:44.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015849000000000001 2024-09-16T20:51:44.487 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.487 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:44.487 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.487 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12626 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1028096~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:44.487 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776041+0000", 2024-09-16T20:51:44.487 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 113.63481031800001, 2024-09-16T20:51:44.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066225260999999994, 2024-09-16T20:51:44.488 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:44.488 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:44.488 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:44.488 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:44.488 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:44.488 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:44.488 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12626 2024-09-16T20:51:44.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.488 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:44.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.488 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:44.488 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776041+0000", 2024-09-16T20:51:44.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:44.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776041+0000", 2024-09-16T20:51:44.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:44.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776263+0000", 2024-09-16T20:51:44.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00022193899999999999 2024-09-16T20:51:44.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:44.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776268+0000", 2024-09-16T20:51:44.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8049999999999997e-06 2024-09-16T20:51:44.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:44.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776280+0000", 2024-09-16T20:51:44.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2635e-05 2024-09-16T20:51:44.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:44.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776288+0000", 2024-09-16T20:51:44.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8199999999999997e-06 2024-09-16T20:51:44.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:44.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842041+0000", 2024-09-16T20:51:44.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065752506000000002 2024-09-16T20:51:44.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:44.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842123+0000", 2024-09-16T20:51:44.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2393999999999998e-05 2024-09-16T20:51:44.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:44.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842266+0000", 2024-09-16T20:51:44.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014316200000000001 2024-09-16T20:51:44.492 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.492 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:44.492 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.492 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12627 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1033216~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:44.492 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776301+0000", 2024-09-16T20:51:44.492 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 113.634550358, 2024-09-16T20:51:44.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066187814999999997, 2024-09-16T20:51:44.492 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:44.492 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:44.492 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:44.492 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:44.493 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:44.493 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:44.493 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12627 2024-09-16T20:51:44.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.493 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:44.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.493 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:44.493 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776301+0000", 2024-09-16T20:51:44.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.493 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:44.493 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776301+0000", 2024-09-16T20:51:44.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.494 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:44.494 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776473+0000", 2024-09-16T20:51:44.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017222400000000001 2024-09-16T20:51:44.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.494 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:44.494 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776477+0000", 2024-09-16T20:51:44.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7019999999999999e-06 2024-09-16T20:51:44.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:44.495 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776486+0000", 2024-09-16T20:51:44.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2119999999999992e-06 2024-09-16T20:51:44.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:44.495 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776492+0000", 2024-09-16T20:51:44.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8089999999999998e-06 2024-09-16T20:51:44.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:44.495 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842301+0000", 2024-09-16T20:51:44.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065809042999999998 2024-09-16T20:51:44.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:44.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842362+0000", 2024-09-16T20:51:44.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1260000000000006e-05 2024-09-16T20:51:44.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:44.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842489+0000", 2024-09-16T20:51:44.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012656500000000001 2024-09-16T20:51:44.496 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.496 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:44.496 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.497 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12628 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1038336~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:44.497 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776502+0000", 2024-09-16T20:51:44.497 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 113.634349712, 2024-09-16T20:51:44.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066283426000000006, 2024-09-16T20:51:44.497 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:44.497 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:44.497 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:44.497 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:44.497 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:44.497 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:44.497 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12628 2024-09-16T20:51:44.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.498 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:44.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.498 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:44.498 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776502+0000", 2024-09-16T20:51:44.498 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.498 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:44.498 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776502+0000", 2024-09-16T20:51:44.498 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.498 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:44.498 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776716+0000", 2024-09-16T20:51:44.499 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000214596 2024-09-16T20:51:44.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.499 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.499 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:44.499 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776723+0000", 2024-09-16T20:51:44.499 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4000000000000003e-06 2024-09-16T20:51:44.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.499 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.499 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:44.499 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776731+0000", 2024-09-16T20:51:44.499 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6129999999999998e-06 2024-09-16T20:51:44.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.500 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:44.500 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776737+0000", 2024-09-16T20:51:44.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1970000000000001e-06 2024-09-16T20:51:44.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.500 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:44.500 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842532+0000", 2024-09-16T20:51:44.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065795127999999994 2024-09-16T20:51:44.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.500 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:44.500 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842599+0000", 2024-09-16T20:51:44.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6711000000000006e-05 2024-09-16T20:51:44.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:44.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842785+0000", 2024-09-16T20:51:44.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018578099999999999 2024-09-16T20:51:44.501 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.501 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:44.501 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.501 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12629 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1043456~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:44.501 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776748+0000", 2024-09-16T20:51:44.502 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 113.63410333500001, 2024-09-16T20:51:44.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066303788000000002, 2024-09-16T20:51:44.502 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:44.502 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:44.502 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:44.502 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:44.502 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:44.502 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:44.502 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12629 2024-09-16T20:51:44.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.502 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:44.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.502 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:44.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776748+0000", 2024-09-16T20:51:44.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:44.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776748+0000", 2024-09-16T20:51:44.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:44.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776934+0000", 2024-09-16T20:51:44.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018598399999999999 2024-09-16T20:51:44.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.504 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:44.504 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776939+0000", 2024-09-16T20:51:44.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1000000000000003e-06 2024-09-16T20:51:44.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.504 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:44.504 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776954+0000", 2024-09-16T20:51:44.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5221e-05 2024-09-16T20:51:44.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.504 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:44.504 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776962+0000", 2024-09-16T20:51:44.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.554e-06 2024-09-16T20:51:44.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:44.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842813+0000", 2024-09-16T20:51:44.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065851006000000004 2024-09-16T20:51:44.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:44.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842886+0000", 2024-09-16T20:51:44.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3663000000000003e-05 2024-09-16T20:51:44.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:44.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.843052+0000", 2024-09-16T20:51:44.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016526 2024-09-16T20:51:44.506 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.506 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:44.506 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.506 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.506 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:44.506 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:44.507 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-16T20:51:44.508 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:51:44.509 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T20:51:44.509 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:44.506+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:44.608 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:44.607+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:44.626 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:44.626 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:51:44.626 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:51:44.626 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:44.636 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:51:44.708 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:44.707+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:44.720 INFO:tasks.thrashosds.thrasher:in_osds: [1, 4, 5, 0, 3, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-16T20:51:44.720 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T20:51:44.720 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-16T20:51:44.720 INFO:tasks.thrashosds.thrasher:Setting osd 5 primary_affinity to 1.000000 2024-09-16T20:51:44.720 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-16T20:51:44.746 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:44.746 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:51:44.747 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:51:44.747 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:51:44.747 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:44.755 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:51:44.809 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:51:44.808+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:51:44.864 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:44.864 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:51:44.864 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:51:44.864 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:51:44.865 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.865 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5286.0:10643 3.17 3.f6273fb7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-16T20:51:44.865 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:51:44.865 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 156.072147266, 2024-09-16T20:51:44.865 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.76672347200000002, 2024-09-16T20:51:44.865 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:44.865 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:44.865 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:44.865 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:44.865 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5286", 2024-09-16T20:51:44.865 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/4135433989", 2024-09-16T20:51:44.865 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 10643 2024-09-16T20:51:44.865 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.866 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:44.866 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.866 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:44.866 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:51:44.866 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.866 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.866 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.866 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:44.866 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:51:44.866 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.866 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.866 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.866 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:44.866 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790489+0000", 2024-09-16T20:51:44.867 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5910000000000001e-06 2024-09-16T20:51:44.867 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.867 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.867 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:44.867 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790493+0000", 2024-09-16T20:51:44.867 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2669999999999998e-06 2024-09-16T20:51:44.867 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.867 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.867 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:44.867 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790500+0000", 2024-09-16T20:51:44.867 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3780000000000003e-06 2024-09-16T20:51:44.867 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.867 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.867 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:44.867 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790505+0000", 2024-09-16T20:51:44.868 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1560000000000003e-06 2024-09-16T20:51:44.868 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.868 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.868 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:44.868 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790559+0000", 2024-09-16T20:51:44.868 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3494e-05 2024-09-16T20:51:44.868 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.868 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.868 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:44.868 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790566+0000", 2024-09-16T20:51:44.868 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7649999999999993e-06 2024-09-16T20:51:44.868 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.868 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.868 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:44.869 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:09.557198+0000", 2024-09-16T20:51:44.869 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.76663136200000004 2024-09-16T20:51:44.869 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.869 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.869 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:44.869 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:09.557211+0000", 2024-09-16T20:51:44.869 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3458999999999999e-05 2024-09-16T20:51:44.869 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.869 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:44.869 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.869 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.869 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.869 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5512.0:450 3.2 3:44e0cffa:::rbd_header.110ef0d7a535:head [watch reconnect cookie 140263298018912 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:51:44.869 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:51:44.870 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 128.300124353, 2024-09-16T20:51:44.870 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0084581939999999, 2024-09-16T20:51:44.870 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:44.870 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:44.870 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:44.870 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:44.870 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5512", 2024-09-16T20:51:44.870 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:51:44.870 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 450 2024-09-16T20:51:44.870 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.870 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:44.870 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.870 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:44.870 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:51:44.870 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.871 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.871 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.871 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:44.871 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:51:44.871 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.871 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.871 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.871 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:44.871 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562512+0000", 2024-09-16T20:51:44.871 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.009e-06 2024-09-16T20:51:44.871 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.871 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.871 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:44.871 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562517+0000", 2024-09-16T20:51:44.871 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9339999999999997e-06 2024-09-16T20:51:44.872 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.872 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.872 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:44.872 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562530+0000", 2024-09-16T20:51:44.872 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3495e-05 2024-09-16T20:51:44.872 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.872 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.872 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:44.872 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562536+0000", 2024-09-16T20:51:44.872 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5690000000000004e-06 2024-09-16T20:51:44.872 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.872 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.872 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:44.872 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562591+0000", 2024-09-16T20:51:44.872 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5761999999999999e-05 2024-09-16T20:51:44.873 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.873 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.873 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:51:44.873 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562599+0000", 2024-09-16T20:51:44.873 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9179999999999994e-06 2024-09-16T20:51:44.873 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.873 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.873 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:44.873 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.565971+0000", 2024-09-16T20:51:44.873 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003371546 2024-09-16T20:51:44.873 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.873 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.873 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:44.873 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566019+0000", 2024-09-16T20:51:44.873 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7849000000000002e-05 2024-09-16T20:51:44.874 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.874 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.874 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:44.874 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.569382+0000", 2024-09-16T20:51:44.874 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.003362833 2024-09-16T20:51:44.874 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.874 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.874 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:44.874 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.569529+0000", 2024-09-16T20:51:44.874 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000147823 2024-09-16T20:51:44.874 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.874 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.874 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:44.874 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570969+0000", 2024-09-16T20:51:44.874 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001439456 2024-09-16T20:51:44.875 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.875 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:44.875 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.875 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.875 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.875 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:14996 3.2 3:44e0cffa:::rbd_header.110ef0d7a535:head [watch reconnect cookie 140213436269840 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:51:44.875 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:51:44.875 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 128.30007594099999, 2024-09-16T20:51:44.875 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.008813943, 2024-09-16T20:51:44.875 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:44.875 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:44.875 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:44.875 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:44.875 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:44.875 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:44.875 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 14996 2024-09-16T20:51:44.876 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.876 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:44.876 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.876 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:44.876 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:51:44.876 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.876 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.876 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.876 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:44.876 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:51:44.876 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.876 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.876 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.876 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:44.876 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562560+0000", 2024-09-16T20:51:44.877 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.06e-07 2024-09-16T20:51:44.877 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.877 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.877 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:44.877 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562563+0000", 2024-09-16T20:51:44.877 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.495e-06 2024-09-16T20:51:44.877 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.877 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.877 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:44.877 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562571+0000", 2024-09-16T20:51:44.877 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1950000000000003e-06 2024-09-16T20:51:44.877 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.877 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.877 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:44.878 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562579+0000", 2024-09-16T20:51:44.878 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2649999999999999e-06 2024-09-16T20:51:44.878 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.878 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.878 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:44.878 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562673+0000", 2024-09-16T20:51:44.878 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4118999999999998e-05 2024-09-16T20:51:44.878 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.878 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.878 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:51:44.878 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562677+0000", 2024-09-16T20:51:44.878 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5179999999999998e-06 2024-09-16T20:51:44.878 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.879 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.879 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:44.879 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566461+0000", 2024-09-16T20:51:44.879 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0037837259999999998 2024-09-16T20:51:44.879 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.879 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.879 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:44.879 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566468+0000", 2024-09-16T20:51:44.879 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3350000000000001e-06 2024-09-16T20:51:44.879 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.879 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.879 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:44.879 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570306+0000", 2024-09-16T20:51:44.879 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.003837997 2024-09-16T20:51:44.879 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.880 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.880 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:44.880 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570392+0000", 2024-09-16T20:51:44.880 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5872999999999996e-05 2024-09-16T20:51:44.880 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.880 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.880 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:44.880 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571373+0000", 2024-09-16T20:51:44.880 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00098071399999999993 2024-09-16T20:51:44.880 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.880 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:44.880 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.880 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.880 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.881 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5255.0:1931 3.2 3:47882819:::rbd_data.110eb5f95afc.0000000000000009:head [read 217088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:51:44.881 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:51:44.881 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 128.300037191, 2024-09-16T20:51:44.881 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0082752880000001, 2024-09-16T20:51:44.881 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:44.881 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:44.881 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:44.881 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:44.881 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5255", 2024-09-16T20:51:44.881 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:51:44.881 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1931 2024-09-16T20:51:44.881 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.881 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:44.881 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.882 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:44.882 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:51:44.882 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.882 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.882 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.882 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:44.882 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:51:44.882 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.882 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.882 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.882 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:44.882 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:51:44.882 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9699999999999996e-07 2024-09-16T20:51:44.882 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.882 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.882 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:44.883 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562601+0000", 2024-09-16T20:51:44.883 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8600000000000001e-06 2024-09-16T20:51:44.883 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.883 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.883 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:44.883 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562608+0000", 2024-09-16T20:51:44.883 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9e-06 2024-09-16T20:51:44.883 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.883 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.883 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:44.883 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562615+0000", 2024-09-16T20:51:44.883 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8179999999999998e-06 2024-09-16T20:51:44.883 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.883 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.883 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:44.884 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562703+0000", 2024-09-16T20:51:44.884 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7811000000000003e-05 2024-09-16T20:51:44.884 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.884 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.884 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:51:44.884 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562706+0000", 2024-09-16T20:51:44.884 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4920000000000002e-06 2024-09-16T20:51:44.884 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.884 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.884 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:44.884 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566685+0000", 2024-09-16T20:51:44.884 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0039792780000000002 2024-09-16T20:51:44.884 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.884 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.885 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:44.885 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566693+0000", 2024-09-16T20:51:44.885 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4730000000000003e-06 2024-09-16T20:51:44.885 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.885 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.885 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:44.885 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570737+0000", 2024-09-16T20:51:44.885 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0040436749999999 2024-09-16T20:51:44.885 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.885 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.885 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:44.885 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570873+0000", 2024-09-16T20:51:44.885 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000136484 2024-09-16T20:51:44.885 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.886 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:44.886 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.886 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.886 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.886 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5255.0:1939 3.2 3:47882819:::rbd_data.110eb5f95afc.0000000000000009:head [read 233472~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:51:44.886 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:51:44.886 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 128.299882325, 2024-09-16T20:51:44.886 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0084657589999999, 2024-09-16T20:51:44.886 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:44.886 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:44.886 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:44.886 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:44.886 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5255", 2024-09-16T20:51:44.886 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:51:44.887 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1939 2024-09-16T20:51:44.887 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.887 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:44.887 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.887 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:44.887 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:51:44.887 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.887 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.887 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.887 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:44.887 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:51:44.887 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.887 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.887 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.887 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:44.888 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:51:44.888 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5199999999999997e-07 2024-09-16T20:51:44.888 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.888 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.888 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:44.888 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562757+0000", 2024-09-16T20:51:44.888 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4979999999999998e-06 2024-09-16T20:51:44.888 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.888 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.888 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:44.888 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562764+0000", 2024-09-16T20:51:44.888 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1470000000000004e-06 2024-09-16T20:51:44.888 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.888 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.889 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:44.889 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562768+0000", 2024-09-16T20:51:44.889 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7929999999999999e-06 2024-09-16T20:51:44.889 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.889 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.889 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:44.889 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562805+0000", 2024-09-16T20:51:44.889 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7477e-05 2024-09-16T20:51:44.889 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.889 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.889 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting_for_map not empty", 2024-09-16T20:51:44.889 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562810+0000", 2024-09-16T20:51:44.889 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0819999999999998e-06 2024-09-16T20:51:44.889 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.889 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.890 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:44.890 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566717+0000", 2024-09-16T20:51:44.890 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0039072600000000001 2024-09-16T20:51:44.890 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.890 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.890 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:44.890 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566724+0000", 2024-09-16T20:51:44.890 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4880000000000004e-06 2024-09-16T20:51:44.890 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.890 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.890 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:44.890 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571009+0000", 2024-09-16T20:51:44.890 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.004284677 2024-09-16T20:51:44.890 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.890 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.890 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:44.891 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571218+0000", 2024-09-16T20:51:44.891 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00020978500000000001 2024-09-16T20:51:44.891 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.891 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:44.891 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.891 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.891 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.891 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:11021 3.1f 3:fee2d1dd:::rbd_data.110ea93691bb.000000000000000e:23 [read 960512~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:44.891 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.693837+0000", 2024-09-16T20:51:44.891 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 114.16879778000001, 2024-09-16T20:51:44.891 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057155138000000001, 2024-09-16T20:51:44.891 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:44.891 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:44.891 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:44.891 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:44.892 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:44.892 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:44.892 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 11021 2024-09-16T20:51:44.892 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.892 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:44.892 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.892 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:44.892 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693837+0000", 2024-09-16T20:51:44.892 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.892 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.892 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.892 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:44.892 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693837+0000", 2024-09-16T20:51:44.892 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.893 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.893 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.893 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:44.893 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699566+0000", 2024-09-16T20:51:44.893 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057283639999999997 2024-09-16T20:51:44.893 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.893 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.893 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:44.893 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699571+0000", 2024-09-16T20:51:44.893 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.604e-06 2024-09-16T20:51:44.893 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.893 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.893 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:44.894 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699583+0000", 2024-09-16T20:51:44.894 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1372999999999999e-05 2024-09-16T20:51:44.894 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.894 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.894 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:44.894 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699591+0000", 2024-09-16T20:51:44.894 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5259999999999993e-06 2024-09-16T20:51:44.894 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.894 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.894 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:44.894 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750674+0000", 2024-09-16T20:51:44.894 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051083154999999998 2024-09-16T20:51:44.894 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.895 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.895 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:44.895 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750745+0000", 2024-09-16T20:51:44.895 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0933999999999995e-05 2024-09-16T20:51:44.895 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.895 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.895 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:44.895 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750992+0000", 2024-09-16T20:51:44.895 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00024718199999999998 2024-09-16T20:51:44.895 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.895 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:44.895 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.895 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.896 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.896 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8792 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2609152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:44.896 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:51:44.896 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 103.80533806299999, 2024-09-16T20:51:44.896 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057003524999999999, 2024-09-16T20:51:44.896 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:44.896 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:44.896 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:44.896 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:44.896 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:44.896 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:44.896 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8792 2024-09-16T20:51:44.896 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.896 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:44.897 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.897 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:44.897 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:51:44.897 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.897 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.897 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.897 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:44.897 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:51:44.897 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.897 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.897 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.897 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:44.897 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:51:44.898 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0400000000000002e-07 2024-09-16T20:51:44.898 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.898 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.898 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:44.898 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057299+0000", 2024-09-16T20:51:44.898 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7460000000000001e-06 2024-09-16T20:51:44.898 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.898 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.898 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:44.898 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057304+0000", 2024-09-16T20:51:44.898 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3210000000000004e-06 2024-09-16T20:51:44.898 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.899 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.899 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:44.899 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057310+0000", 2024-09-16T20:51:44.899 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4360000000000001e-06 2024-09-16T20:51:44.899 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.899 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.899 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:44.899 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114200+0000", 2024-09-16T20:51:44.899 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056889954 2024-09-16T20:51:44.899 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.899 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.899 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:44.899 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114243+0000", 2024-09-16T20:51:44.899 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2998999999999999e-05 2024-09-16T20:51:44.900 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.900 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.900 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:44.900 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114300+0000", 2024-09-16T20:51:44.900 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7765e-05 2024-09-16T20:51:44.900 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.900 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:44.900 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.900 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.900 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.900 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8794 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2351104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:44.900 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057327+0000", 2024-09-16T20:51:44.900 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 103.805307739, 2024-09-16T20:51:44.901 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057101130999999999, 2024-09-16T20:51:44.901 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:44.901 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:44.901 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:44.901 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:44.901 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:44.901 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:44.901 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8794 2024-09-16T20:51:44.901 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.901 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:44.901 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.901 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:44.902 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057327+0000", 2024-09-16T20:51:44.902 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.902 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.902 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:44.902 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057327+0000", 2024-09-16T20:51:44.902 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.902 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.902 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:44.902 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057328+0000", 2024-09-16T20:51:44.902 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7399999999999999e-07 2024-09-16T20:51:44.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.903 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.903 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:44.903 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057330+0000", 2024-09-16T20:51:44.903 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2199999999999999e-06 2024-09-16T20:51:44.903 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.903 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.903 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:44.903 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057336+0000", 2024-09-16T20:51:44.903 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1240000000000002e-06 2024-09-16T20:51:44.903 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.903 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.903 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:44.904 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057341+0000", 2024-09-16T20:51:44.904 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8470000000000003e-06 2024-09-16T20:51:44.904 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.904 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.904 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:44.904 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114334+0000", 2024-09-16T20:51:44.904 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056993267 2024-09-16T20:51:44.904 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.904 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.904 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:44.904 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114376+0000", 2024-09-16T20:51:44.904 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1638999999999999e-05 2024-09-16T20:51:44.904 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.905 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.905 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:44.905 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114428+0000", 2024-09-16T20:51:44.905 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2660000000000001e-05 2024-09-16T20:51:44.905 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.905 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:44.905 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.905 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.905 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.905 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8798 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2297856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:44.905 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:51:44.905 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 103.80528801200001, 2024-09-16T20:51:44.905 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057277041000000001, 2024-09-16T20:51:44.906 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:44.906 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:44.906 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:44.906 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:44.906 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:44.906 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:44.906 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8798 2024-09-16T20:51:44.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.906 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:44.906 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.907 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:44.907 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:51:44.907 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.907 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.907 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.907 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:44.907 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:51:44.907 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.907 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.907 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.907 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:44.907 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:51:44.907 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2800000000000003e-07 2024-09-16T20:51:44.908 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.908 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.908 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:44.908 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057354+0000", 2024-09-16T20:51:44.908 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7780000000000002e-06 2024-09-16T20:51:44.908 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.908 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.908 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:44.908 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057360+0000", 2024-09-16T20:51:44.908 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0419999999999999e-06 2024-09-16T20:51:44.908 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.908 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.909 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:44.909 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057365+0000", 2024-09-16T20:51:44.909 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7929999999999997e-06 2024-09-16T20:51:44.909 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.909 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.909 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:44.909 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114460+0000", 2024-09-16T20:51:44.909 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057094662999999997 2024-09-16T20:51:44.909 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.909 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.909 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:44.909 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114532+0000", 2024-09-16T20:51:44.909 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2881999999999998e-05 2024-09-16T20:51:44.910 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.910 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.910 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:44.910 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114624+0000", 2024-09-16T20:51:44.910 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1555e-05 2024-09-16T20:51:44.910 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.910 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:44.910 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.910 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.910 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.910 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8800 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2316288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:44.910 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:51:44.910 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 103.805264073, 2024-09-16T20:51:44.911 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.05746946, 2024-09-16T20:51:44.911 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:44.911 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:44.911 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:44.911 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:44.911 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:44.911 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:44.911 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8800 2024-09-16T20:51:44.911 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.911 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:44.911 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.911 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:44.911 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:51:44.911 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.912 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.912 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.912 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:44.912 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:51:44.912 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.912 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.912 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.912 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:44.912 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:51:44.912 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5199999999999998e-07 2024-09-16T20:51:44.912 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.912 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.912 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:44.913 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057373+0000", 2024-09-16T20:51:44.913 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.79e-06 2024-09-16T20:51:44.913 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.913 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.913 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:44.913 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057379+0000", 2024-09-16T20:51:44.913 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6699999999999999e-06 2024-09-16T20:51:44.913 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.913 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.913 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:44.913 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057386+0000", 2024-09-16T20:51:44.913 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3170000000000004e-06 2024-09-16T20:51:44.913 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.913 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.914 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:44.914 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114655+0000", 2024-09-16T20:51:44.914 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057269420000000001 2024-09-16T20:51:44.914 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.914 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.914 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:44.914 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114734+0000", 2024-09-16T20:51:44.914 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8084000000000007e-05 2024-09-16T20:51:44.914 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.914 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.914 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:44.914 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114840+0000", 2024-09-16T20:51:44.914 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000106827 2024-09-16T20:51:44.915 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.915 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:44.915 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.915 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.915 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.915 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8801 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2334720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:44.915 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:51:44.915 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 103.805243042, 2024-09-16T20:51:44.915 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057594682000000001, 2024-09-16T20:51:44.915 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:44.915 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:44.915 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:44.915 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:44.915 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:44.915 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:44.916 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8801 2024-09-16T20:51:44.916 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.916 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:44.916 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.916 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:44.916 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:51:44.916 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.916 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.916 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.916 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:44.916 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:51:44.916 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.917 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.917 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.917 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:44.917 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:51:44.917 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6899999999999998e-07 2024-09-16T20:51:44.917 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.917 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.917 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:44.917 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057394+0000", 2024-09-16T20:51:44.917 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8300000000000001e-06 2024-09-16T20:51:44.917 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.917 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.917 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:44.918 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057400+0000", 2024-09-16T20:51:44.918 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6500000000000001e-06 2024-09-16T20:51:44.918 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.918 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.918 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:44.918 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057404+0000", 2024-09-16T20:51:44.918 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4669999999999998e-06 2024-09-16T20:51:44.918 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.918 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.918 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:44.918 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114874+0000", 2024-09-16T20:51:44.918 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057469602000000002 2024-09-16T20:51:44.918 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.919 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.919 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:44.919 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114933+0000", 2024-09-16T20:51:44.919 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9500000000000003e-05 2024-09-16T20:51:44.919 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.919 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.919 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:44.919 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114987+0000", 2024-09-16T20:51:44.919 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3263999999999997e-05 2024-09-16T20:51:44.919 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.919 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:44.919 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.919 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.920 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.920 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8802 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2353152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:44.920 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:51:44.920 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 103.805194357, 2024-09-16T20:51:44.920 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057714079000000001, 2024-09-16T20:51:44.920 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:44.920 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:44.920 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:44.920 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:44.920 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:44.920 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:44.920 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8802 2024-09-16T20:51:44.920 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.921 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:44.921 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.921 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:44.921 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:51:44.921 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.921 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.921 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.921 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:44.921 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:51:44.921 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.921 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.921 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.922 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:44.922 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:51:44.922 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4999999999999998e-07 2024-09-16T20:51:44.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.922 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.922 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:44.922 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057446+0000", 2024-09-16T20:51:44.922 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4240000000000003e-06 2024-09-16T20:51:44.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.922 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.922 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:44.922 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057452+0000", 2024-09-16T20:51:44.922 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8379999999999996e-06 2024-09-16T20:51:44.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.922 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.923 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:44.923 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057460+0000", 2024-09-16T20:51:44.923 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9629999999999998e-06 2024-09-16T20:51:44.923 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.923 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.923 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:44.923 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115015+0000", 2024-09-16T20:51:44.923 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057554389999999997 2024-09-16T20:51:44.923 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.923 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.923 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:44.923 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115082+0000", 2024-09-16T20:51:44.923 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7733999999999999e-05 2024-09-16T20:51:44.923 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.924 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.924 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:44.924 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115155+0000", 2024-09-16T20:51:44.924 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2280000000000006e-05 2024-09-16T20:51:44.924 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.924 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:44.924 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.924 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.924 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.924 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8804 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2555904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:44.924 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:51:44.924 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 103.805168041, 2024-09-16T20:51:44.924 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057833862, 2024-09-16T20:51:44.925 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:44.925 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:44.925 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:44.925 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:44.925 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:44.925 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:44.925 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8804 2024-09-16T20:51:44.925 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.925 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:44.925 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.925 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:44.925 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:51:44.925 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.926 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.926 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.926 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:44.926 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:51:44.926 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.926 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.926 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.926 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:44.926 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:51:44.926 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7899999999999999e-07 2024-09-16T20:51:44.926 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.926 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.927 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:44.927 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057469+0000", 2024-09-16T20:51:44.927 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.001e-06 2024-09-16T20:51:44.927 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.927 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.927 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:44.927 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057475+0000", 2024-09-16T20:51:44.927 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7629999999999997e-06 2024-09-16T20:51:44.927 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.927 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.927 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:44.927 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057480+0000", 2024-09-16T20:51:44.927 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8069999999999999e-06 2024-09-16T20:51:44.927 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.927 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.928 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:44.928 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115183+0000", 2024-09-16T20:51:44.928 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057703364 2024-09-16T20:51:44.928 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.928 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.928 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:44.928 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115227+0000", 2024-09-16T20:51:44.928 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3903999999999998e-05 2024-09-16T20:51:44.928 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.928 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.928 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:44.928 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115301+0000", 2024-09-16T20:51:44.928 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3543999999999999e-05 2024-09-16T20:51:44.929 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.929 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:44.929 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.929 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.929 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.929 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8805 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2574336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:44.929 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057487+0000", 2024-09-16T20:51:44.929 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 103.805147713, 2024-09-16T20:51:44.929 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058021027000000003, 2024-09-16T20:51:44.929 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:44.929 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:44.929 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:44.929 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:44.930 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:44.930 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:44.930 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8805 2024-09-16T20:51:44.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.930 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:44.930 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.930 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:44.930 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057487+0000", 2024-09-16T20:51:44.930 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.930 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.930 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:44.930 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057487+0000", 2024-09-16T20:51:44.930 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.930 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.931 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:44.931 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057488+0000", 2024-09-16T20:51:44.931 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5199999999999998e-07 2024-09-16T20:51:44.931 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.931 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.931 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:44.931 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057490+0000", 2024-09-16T20:51:44.931 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.939e-06 2024-09-16T20:51:44.931 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.931 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.931 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:44.931 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057495+0000", 2024-09-16T20:51:44.931 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6439999999999997e-06 2024-09-16T20:51:44.931 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.931 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.932 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:44.932 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057500+0000", 2024-09-16T20:51:44.932 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8119999999999998e-06 2024-09-16T20:51:44.932 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.932 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.932 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:44.932 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115341+0000", 2024-09-16T20:51:44.932 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057841341999999997 2024-09-16T20:51:44.932 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.932 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.932 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:44.932 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115422+0000", 2024-09-16T20:51:44.932 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1003999999999999e-05 2024-09-16T20:51:44.932 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.933 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.933 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:44.933 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115508+0000", 2024-09-16T20:51:44.933 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5933999999999994e-05 2024-09-16T20:51:44.933 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.933 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:44.933 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.933 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.933 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.933 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8807 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2592768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:44.933 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057505+0000", 2024-09-16T20:51:44.933 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 103.805129503, 2024-09-16T20:51:44.933 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058186633000000001, 2024-09-16T20:51:44.934 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:44.934 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:44.934 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:44.934 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:44.934 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:44.934 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:44.934 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8807 2024-09-16T20:51:44.934 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.934 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:44.934 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.935 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:44.935 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057505+0000", 2024-09-16T20:51:44.935 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.935 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.935 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:44.935 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057505+0000", 2024-09-16T20:51:44.935 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.935 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.935 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:44.935 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057506+0000", 2024-09-16T20:51:44.935 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.27e-07 2024-09-16T20:51:44.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.936 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.936 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:44.936 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057508+0000", 2024-09-16T20:51:44.936 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7400000000000001e-06 2024-09-16T20:51:44.936 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.936 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.936 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:44.936 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057513+0000", 2024-09-16T20:51:44.936 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.468e-06 2024-09-16T20:51:44.936 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.936 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.936 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:44.937 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057517+0000", 2024-09-16T20:51:44.937 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4329999999999997e-06 2024-09-16T20:51:44.937 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.937 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.937 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:44.937 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115538+0000", 2024-09-16T20:51:44.937 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058020355000000003 2024-09-16T20:51:44.937 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.937 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.937 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:44.937 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115616+0000", 2024-09-16T20:51:44.937 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7998999999999996e-05 2024-09-16T20:51:44.937 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.937 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.938 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:44.938 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115692+0000", 2024-09-16T20:51:44.938 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6310999999999995e-05 2024-09-16T20:51:44.938 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.938 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:44.938 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.938 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.938 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.938 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8808 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2611200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:44.938 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:51:44.938 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 103.80511224200001, 2024-09-16T20:51:44.938 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058354046, 2024-09-16T20:51:44.938 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:44.938 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:44.939 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:44.939 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:44.939 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:44.939 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:44.939 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8808 2024-09-16T20:51:44.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.939 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:44.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.939 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:44.939 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:51:44.939 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.939 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:44.939 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:51:44.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:44.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:51:44.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2500000000000001e-07 2024-09-16T20:51:44.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:44.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057525+0000", 2024-09-16T20:51:44.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5549999999999999e-06 2024-09-16T20:51:44.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:44.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057530+0000", 2024-09-16T20:51:44.941 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5489999999999997e-06 2024-09-16T20:51:44.941 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.941 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.941 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:44.941 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057535+0000", 2024-09-16T20:51:44.941 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5709999999999999e-06 2024-09-16T20:51:44.941 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.941 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.941 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:44.941 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115722+0000", 2024-09-16T20:51:44.941 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058187323999999999 2024-09-16T20:51:44.941 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.941 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.941 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:44.942 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115799+0000", 2024-09-16T20:51:44.942 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6445000000000004e-05 2024-09-16T20:51:44.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.942 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.942 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:44.942 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115877+0000", 2024-09-16T20:51:44.942 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8276999999999996e-05 2024-09-16T20:51:44.942 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.942 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:44.942 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.942 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.942 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:2182 3.17 3:edb5e5cd:::rbd_data.110e61479aee.0000000000000014:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write 0~5120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-16T20:51:44.942 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:22.734609+0000", 2024-09-16T20:51:44.942 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 82.128025995000002, 2024-09-16T20:51:44.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.124890894, 2024-09-16T20:51:44.943 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:44.943 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:44.943 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:44.943 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:44.943 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:51:44.943 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:44.943 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 2182 2024-09-16T20:51:44.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.943 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:44.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.943 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:44.943 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734609+0000", 2024-09-16T20:51:44.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.944 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:44.944 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734609+0000", 2024-09-16T20:51:44.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.944 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:44.944 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734611+0000", 2024-09-16T20:51:44.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9599999999999999e-06 2024-09-16T20:51:44.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.944 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:44.944 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734628+0000", 2024-09-16T20:51:44.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.715e-05 2024-09-16T20:51:44.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.945 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:44.945 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734656+0000", 2024-09-16T20:51:44.945 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7759999999999998e-05 2024-09-16T20:51:44.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.945 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:44.945 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734669+0000", 2024-09-16T20:51:44.945 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3315999999999999e-05 2024-09-16T20:51:44.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.945 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:44.946 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734826+0000", 2024-09-16T20:51:44.946 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015668499999999999 2024-09-16T20:51:44.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.946 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.946 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:44.946 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.735048+0000", 2024-09-16T20:51:44.946 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00022179 2024-09-16T20:51:44.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.946 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.946 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:44.946 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.735248+0000", 2024-09-16T20:51:44.946 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00020010399999999999 2024-09-16T20:51:44.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.947 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.947 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:51:44.947 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.736252+0000", 2024-09-16T20:51:44.947 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010039739999999999 2024-09-16T20:51:44.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.947 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.947 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:44.947 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.859340+0000", 2024-09-16T20:51:44.947 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.12308846599999999 2024-09-16T20:51:44.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.947 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.947 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:51:44.947 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.859431+0000", 2024-09-16T20:51:44.948 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1099000000000003e-05 2024-09-16T20:51:44.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.948 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:44.948 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.859500+0000", 2024-09-16T20:51:44.948 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8590000000000006e-05 2024-09-16T20:51:44.948 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.948 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:44.948 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.948 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:29157 3.1b 3:dbe69130:::journal.110e1b19a241:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e167)", 2024-09-16T20:51:44.948 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:54.663540+0000", 2024-09-16T20:51:44.949 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 50.199094657000003, 2024-09-16T20:51:44.949 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.92912227999999997, 2024-09-16T20:51:44.949 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:44.949 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:44.949 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:44.949 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:44.949 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:44.949 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:44.949 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 29157 2024-09-16T20:51:44.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.949 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:44.949 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.949 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:44.950 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663540+0000", 2024-09-16T20:51:44.950 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.950 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.950 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:44.950 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663540+0000", 2024-09-16T20:51:44.950 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.950 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.950 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:44.950 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663541+0000", 2024-09-16T20:51:44.950 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1680000000000001e-06 2024-09-16T20:51:44.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.951 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.951 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:44.951 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663546+0000", 2024-09-16T20:51:44.951 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7129999999999996e-06 2024-09-16T20:51:44.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.951 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.951 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:44.951 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663558+0000", 2024-09-16T20:51:44.951 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1347e-05 2024-09-16T20:51:44.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.951 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.951 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:44.951 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663563+0000", 2024-09-16T20:51:44.952 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1250000000000001e-06 2024-09-16T20:51:44.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.952 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.952 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:44.952 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663612+0000", 2024-09-16T20:51:44.952 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9665e-05 2024-09-16T20:51:44.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.952 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.952 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:44.952 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663620+0000", 2024-09-16T20:51:44.952 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8439999999999998e-06 2024-09-16T20:51:44.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.953 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.953 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:44.953 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:55.592525+0000", 2024-09-16T20:51:44.953 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.92890456700000001 2024-09-16T20:51:44.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.953 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.953 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:44.953 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:55.592663+0000", 2024-09-16T20:51:44.953 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013785100000000001 2024-09-16T20:51:44.953 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.953 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:44.953 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.953 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.954 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:30229 3.16 3:6f5476df:::rbd_header.110e25db1f60:head [watch reconnect cookie 140213637350016 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e181)", 2024-09-16T20:51:44.954 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:51:08.603124+0000", 2024-09-16T20:51:44.954 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.259511011999997, 2024-09-16T20:51:44.954 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.97765519899999997, 2024-09-16T20:51:44.954 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:44.954 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:44.954 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:44.954 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:44.954 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:44.954 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:44.954 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30229 2024-09-16T20:51:44.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.954 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:44.954 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.954 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:44.955 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603124+0000", 2024-09-16T20:51:44.955 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.955 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:44.955 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603124+0000", 2024-09-16T20:51:44.955 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.955 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:44.955 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603125+0000", 2024-09-16T20:51:44.955 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5090000000000001e-06 2024-09-16T20:51:44.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.955 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:44.956 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603131+0000", 2024-09-16T20:51:44.956 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9089999999999997e-06 2024-09-16T20:51:44.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.956 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:44.956 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603143+0000", 2024-09-16T20:51:44.956 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1416e-05 2024-09-16T20:51:44.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.956 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:44.956 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603151+0000", 2024-09-16T20:51:44.956 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7889999999999995e-06 2024-09-16T20:51:44.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.957 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:44.957 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603674+0000", 2024-09-16T20:51:44.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00052357 2024-09-16T20:51:44.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.957 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:44.957 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603684+0000", 2024-09-16T20:51:44.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0274e-05 2024-09-16T20:51:44.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.957 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:44.957 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.575380+0000", 2024-09-16T20:51:44.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.97169509799999998 2024-09-16T20:51:44.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.958 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.958 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:44.958 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.575591+0000", 2024-09-16T20:51:44.958 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00021126700000000001 2024-09-16T20:51:44.958 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.958 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.958 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:44.958 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.580779+0000", 2024-09-16T20:51:44.958 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0051883670000000002 2024-09-16T20:51:44.958 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.958 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:44.958 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.958 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.958 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.958 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:30230 3.16 3:6f5476df:::rbd_header.110e25db1f60:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e181)", 2024-09-16T20:51:44.959 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:51:08.603204+0000", 2024-09-16T20:51:44.959 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.259431063999997, 2024-09-16T20:51:44.959 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.98129583300000001, 2024-09-16T20:51:44.959 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:44.959 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:44.959 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:44.959 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:44.959 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:44.959 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:44.959 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30230 2024-09-16T20:51:44.959 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.959 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:44.959 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.959 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:44.960 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603204+0000", 2024-09-16T20:51:44.960 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.960 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:44.960 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603204+0000", 2024-09-16T20:51:44.960 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:44.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.960 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:44.960 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603205+0000", 2024-09-16T20:51:44.960 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4900000000000005e-07 2024-09-16T20:51:44.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.960 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:44.961 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603209+0000", 2024-09-16T20:51:44.961 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5210000000000004e-06 2024-09-16T20:51:44.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.961 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.961 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:44.961 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603222+0000", 2024-09-16T20:51:44.961 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2733e-05 2024-09-16T20:51:44.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.961 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.961 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:44.961 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603228+0000", 2024-09-16T20:51:44.961 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4520000000000002e-06 2024-09-16T20:51:44.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.961 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.961 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:44.962 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603719+0000", 2024-09-16T20:51:44.962 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00049103600000000003 2024-09-16T20:51:44.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.962 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:44.962 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603730+0000", 2024-09-16T20:51:44.962 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.023e-05 2024-09-16T20:51:44.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.962 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:44.962 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.577394+0000", 2024-09-16T20:51:44.962 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.97366440099999996 2024-09-16T20:51:44.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.963 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:44.963 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.577499+0000", 2024-09-16T20:51:44.963 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000104509 2024-09-16T20:51:44.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.963 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.963 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:44.963 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.577920+0000", 2024-09-16T20:51:44.963 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00042173699999999999 2024-09-16T20:51:44.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.963 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.963 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:51:44.963 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.580795+0000", 2024-09-16T20:51:44.963 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.002874302 2024-09-16T20:51:44.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.964 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.964 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:44.964 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.584423+0000", 2024-09-16T20:51:44.964 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0036280010000000001 2024-09-16T20:51:44.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.964 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.964 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:51:44.964 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.584465+0000", 2024-09-16T20:51:44.964 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2590999999999999e-05 2024-09-16T20:51:44.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:44.964 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:44.964 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:44.964 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.584500+0000", 2024-09-16T20:51:44.964 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4570999999999998e-05 2024-09-16T20:51:44.965 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.965 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:44.965 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.965 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:44.965 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:44.965 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:44.966 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:51:44.965+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:51:44.967 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T20:51:45.067 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:45.066+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:45.074 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:45.075 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:51:45.075 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:51:45.075 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:45.083 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T20:51:45.168 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-16T20:51:45.167+0000 7fb277493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31982) UID: 0 2024-09-16T20:51:45.230 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:45.231 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:51:45.231 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:51:45.231 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:51:45.231 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:45.239 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T20:51:45.269 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-16T20:51:45.268+0000 7f8ac329b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32257) UID: 0 2024-09-16T20:51:45.370 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-16T20:51:45.369+0000 7f09b0b2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32261) UID: 0 2024-09-16T20:51:45.387 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:45.388 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:51:45.388 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:51:45.388 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:51:45.388 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.388 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5439 3.a 3:5789f5c3:::rbd_data.110ed9ca540.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 1181696~60416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:45.388 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:51:45.388 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 202.52128892100001, 2024-09-16T20:51:45.388 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10102045599999999, 2024-09-16T20:51:45.388 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:45.388 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:45.388 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:45.388 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:45.389 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:45.389 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:45.389 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5439 2024-09-16T20:51:45.389 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.389 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:45.389 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.389 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:45.389 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:51:45.389 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:45.389 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.389 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.389 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:45.390 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864710+0000", 2024-09-16T20:51:45.390 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:45.390 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.390 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.390 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:45.390 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864711+0000", 2024-09-16T20:51:45.390 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7899999999999997e-07 2024-09-16T20:51:45.390 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.390 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.390 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:45.390 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864760+0000", 2024-09-16T20:51:45.391 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8803e-05 2024-09-16T20:51:45.391 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.391 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.391 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:45.391 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864769+0000", 2024-09-16T20:51:45.391 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0999999999999993e-06 2024-09-16T20:51:45.391 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.391 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.391 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:45.391 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864774+0000", 2024-09-16T20:51:45.391 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8919999999999999e-06 2024-09-16T20:51:45.391 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.391 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.392 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:45.392 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864815+0000", 2024-09-16T20:51:45.392 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1307000000000003e-05 2024-09-16T20:51:45.392 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.392 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.392 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:45.392 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864893+0000", 2024-09-16T20:51:45.392 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7581000000000001e-05 2024-09-16T20:51:45.392 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.393 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.393 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 1", 2024-09-16T20:51:45.393 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864996+0000", 2024-09-16T20:51:45.393 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102725 2024-09-16T20:51:45.393 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.393 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.393 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:45.393 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965442+0000", 2024-09-16T20:51:45.393 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100446012 2024-09-16T20:51:45.393 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.393 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.393 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:45.393 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965618+0000", 2024-09-16T20:51:45.394 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00017653100000000001 2024-09-16T20:51:45.394 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.394 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.394 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:45.394 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965682+0000", 2024-09-16T20:51:45.394 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3905999999999994e-05 2024-09-16T20:51:45.394 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.394 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.394 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:45.394 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965731+0000", 2024-09-16T20:51:45.394 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8819999999999997e-05 2024-09-16T20:51:45.394 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:45.395 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:45.395 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:45.395 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.395 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.395 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4600.0:5439 3.a e37/23 ondisk, result = 0)", 2024-09-16T20:51:45.395 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:51:45.395 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 202.52029277299999, 2024-09-16T20:51:45.395 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099742634999999996, 2024-09-16T20:51:45.395 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:45.395 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:45.395 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:45.396 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:45.396 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.396 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:45.396 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:51:45.396 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:45.396 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.396 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.396 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:45.397 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:51:45.397 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:45.397 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.397 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.397 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:45.397 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865707+0000", 2024-09-16T20:51:45.397 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0900000000000001e-07 2024-09-16T20:51:45.397 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.397 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.397 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:45.397 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.865711+0000", 2024-09-16T20:51:45.397 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3739999999999998e-06 2024-09-16T20:51:45.398 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.398 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.398 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:45.398 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965343+0000", 2024-09-16T20:51:45.398 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099632526999999999 2024-09-16T20:51:45.398 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.398 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.398 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:45.398 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965357+0000", 2024-09-16T20:51:45.398 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3713999999999999e-05 2024-09-16T20:51:45.398 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.398 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.399 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:45.399 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965426+0000", 2024-09-16T20:51:45.399 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8745999999999995e-05 2024-09-16T20:51:45.399 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.399 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.399 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:45.399 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965434+0000", 2024-09-16T20:51:45.399 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6910000000000003e-06 2024-09-16T20:51:45.399 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.399 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.399 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:45.399 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965449+0000", 2024-09-16T20:51:45.399 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4874e-05 2024-09-16T20:51:45.399 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:45.400 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:45.400 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:45.400 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.400 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.400 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4882.0:2508 3.12 3:4992771f:::rbd_object_map.110e7c8851c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T20:51:45.400 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:51:45.400 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 178.90448922499999, 2024-09-16T20:51:45.400 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.125007011, 2024-09-16T20:51:45.400 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:45.400 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:45.400 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:45.400 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:45.400 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4882", 2024-09-16T20:51:45.401 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2436923223", 2024-09-16T20:51:45.401 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2508 2024-09-16T20:51:45.401 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.401 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:45.401 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.401 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:45.401 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:51:45.401 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:45.401 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.401 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.401 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:45.401 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481510+0000", 2024-09-16T20:51:45.401 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:45.402 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.402 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.402 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:45.402 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481511+0000", 2024-09-16T20:51:45.402 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0699999999999996e-07 2024-09-16T20:51:45.402 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.402 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.402 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:45.402 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481515+0000", 2024-09-16T20:51:45.402 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0609999999999997e-06 2024-09-16T20:51:45.402 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.402 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.402 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:45.402 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481523+0000", 2024-09-16T20:51:45.403 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1310000000000006e-06 2024-09-16T20:51:45.403 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.403 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.403 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:45.403 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481528+0000", 2024-09-16T20:51:45.403 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4900000000000002e-06 2024-09-16T20:51:45.403 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.403 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.403 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:45.403 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481565+0000", 2024-09-16T20:51:45.403 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7066000000000001e-05 2024-09-16T20:51:45.403 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.403 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.403 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:45.403 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481628+0000", 2024-09-16T20:51:45.404 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3118000000000001e-05 2024-09-16T20:51:45.404 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.404 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.404 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 2", 2024-09-16T20:51:45.404 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.481822+0000", 2024-09-16T20:51:45.404 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019446200000000001 2024-09-16T20:51:45.404 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.404 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.404 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:45.404 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605988+0000", 2024-09-16T20:51:45.404 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.124165888 2024-09-16T20:51:45.404 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.405 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.405 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:45.405 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606427+0000", 2024-09-16T20:51:45.405 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00043854899999999999 2024-09-16T20:51:45.405 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.405 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.405 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:45.405 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606476+0000", 2024-09-16T20:51:45.405 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8971999999999998e-05 2024-09-16T20:51:45.405 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.405 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.405 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:45.405 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606517+0000", 2024-09-16T20:51:45.406 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1467000000000001e-05 2024-09-16T20:51:45.406 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:45.406 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:45.406 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:45.406 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.406 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.406 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.4882.0:2508 3.12 e57/55 ondisk, result = 0)", 2024-09-16T20:51:45.406 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:51:45.406 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 178.90350072499999, 2024-09-16T20:51:45.406 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12349711400000001, 2024-09-16T20:51:45.406 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:45.406 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:45.406 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:45.407 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:45.407 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.407 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:45.407 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:51:45.407 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:45.407 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.407 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.407 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:45.407 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:51:45.407 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:45.407 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.407 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.407 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:45.407 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482499+0000", 2024-09-16T20:51:45.408 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.06e-07 2024-09-16T20:51:45.408 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.408 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.408 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:45.408 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482502+0000", 2024-09-16T20:51:45.408 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.01e-06 2024-09-16T20:51:45.408 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.408 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.408 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:45.408 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605772+0000", 2024-09-16T20:51:45.408 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.123269687 2024-09-16T20:51:45.408 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.408 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.408 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:45.409 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605791+0000", 2024-09-16T20:51:45.409 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8912000000000001e-05 2024-09-16T20:51:45.409 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.409 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.409 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:45.409 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605974+0000", 2024-09-16T20:51:45.409 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000182836 2024-09-16T20:51:45.409 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.409 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.409 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:45.409 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605982+0000", 2024-09-16T20:51:45.409 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1699999999999997e-06 2024-09-16T20:51:45.410 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.410 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.410 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:45.410 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605996+0000", 2024-09-16T20:51:45.410 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3893000000000001e-05 2024-09-16T20:51:45.410 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:45.410 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:45.410 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:45.410 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.410 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.410 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4882.0:2512 3.f 3:f4d895b7:::rbd_data.110e7c8851c.0000000000000000:head [read 16384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T20:51:45.410 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:51:45.410 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 178.90306369499999, 2024-09-16T20:51:45.411 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12318483, 2024-09-16T20:51:45.411 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:45.411 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:45.411 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:45.411 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:45.411 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4882", 2024-09-16T20:51:45.411 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2436923223", 2024-09-16T20:51:45.411 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2512 2024-09-16T20:51:45.411 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.411 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:45.411 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.411 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:45.411 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:51:45.412 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:45.412 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.412 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.412 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:45.412 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:51:45.412 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:45.412 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.412 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.412 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:45.412 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482936+0000", 2024-09-16T20:51:45.412 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3499999999999995e-07 2024-09-16T20:51:45.412 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.413 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.413 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:45.413 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.482943+0000", 2024-09-16T20:51:45.413 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3450000000000002e-06 2024-09-16T20:51:45.413 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.413 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.413 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:45.413 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605772+0000", 2024-09-16T20:51:45.413 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.122829796 2024-09-16T20:51:45.413 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.413 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.413 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:45.413 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605792+0000", 2024-09-16T20:51:45.413 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9916999999999999e-05 2024-09-16T20:51:45.414 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.414 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.414 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:45.414 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.605917+0000", 2024-09-16T20:51:45.414 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00012500799999999999 2024-09-16T20:51:45.414 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.414 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.414 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:45.414 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606004+0000", 2024-09-16T20:51:45.414 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6242999999999994e-05 2024-09-16T20:51:45.414 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.414 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.414 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:45.415 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:46.606120+0000", 2024-09-16T20:51:45.415 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000116786 2024-09-16T20:51:45.415 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:45.415 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:45.415 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:45.415 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.415 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.415 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:11702 3.15 3:acf33c8b:::rbd_data.110e747f0779.000000000000001a:head [set-alloc-hint object_size 262144 write_size 262144,write 143360~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:51:45.415 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:51:45.415 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 161.24807127299999, 2024-09-16T20:51:45.415 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.084801043000000006, 2024-09-16T20:51:45.415 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:45.415 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:45.416 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:45.416 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:45.416 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:51:45.416 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:51:45.416 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 11702 2024-09-16T20:51:45.416 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.416 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:45.416 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.416 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:45.416 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:51:45.416 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:45.416 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.416 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.417 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:45.417 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137928+0000", 2024-09-16T20:51:45.417 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:45.417 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.417 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.417 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:45.417 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137929+0000", 2024-09-16T20:51:45.417 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8400000000000002e-07 2024-09-16T20:51:45.417 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.417 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.417 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:45.417 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137937+0000", 2024-09-16T20:51:45.417 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0260000000000007e-06 2024-09-16T20:51:45.417 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.418 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.418 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:45.418 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137952+0000", 2024-09-16T20:51:45.418 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4659e-05 2024-09-16T20:51:45.418 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.418 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.418 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:45.418 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.137960+0000", 2024-09-16T20:51:45.418 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0169999999999996e-06 2024-09-16T20:51:45.418 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.418 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.418 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:45.418 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138010+0000", 2024-09-16T20:51:45.419 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0021000000000001e-05 2024-09-16T20:51:45.419 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.419 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.419 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:45.419 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138114+0000", 2024-09-16T20:51:45.419 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000103797 2024-09-16T20:51:45.419 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.419 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.419 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:51:45.419 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138233+0000", 2024-09-16T20:51:45.419 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000118971 2024-09-16T20:51:45.419 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.419 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.420 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:45.420 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138953+0000", 2024-09-16T20:51:45.420 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00072026000000000002 2024-09-16T20:51:45.420 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.420 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.420 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:45.420 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222609+0000", 2024-09-16T20:51:45.420 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.083656650999999999 2024-09-16T20:51:45.420 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.420 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.420 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:45.420 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222684+0000", 2024-09-16T20:51:45.421 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4369999999999994e-05 2024-09-16T20:51:45.421 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.421 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.421 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:45.421 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222729+0000", 2024-09-16T20:51:45.421 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5287000000000002e-05 2024-09-16T20:51:45.421 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:45.421 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:45.421 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:45.421 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.421 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.421 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:11704 3.9 3:91220b75:::rbd_data.110e747f0779.000000000000001c:head [set-alloc-hint object_size 262144 write_size 262144,write 139264~6656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:51:45.421 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:51:45.422 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 161.24783327200001, 2024-09-16T20:51:45.422 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.084531133999999994, 2024-09-16T20:51:45.422 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:45.422 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:45.422 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:45.422 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:45.422 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:51:45.422 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:51:45.422 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 11704 2024-09-16T20:51:45.422 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.422 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:45.422 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.422 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:45.422 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:51:45.423 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:45.423 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.423 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.423 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:45.423 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138166+0000", 2024-09-16T20:51:45.423 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:45.423 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.423 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.423 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:45.423 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138167+0000", 2024-09-16T20:51:45.423 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.33e-07 2024-09-16T20:51:45.423 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.423 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.423 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:45.423 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138179+0000", 2024-09-16T20:51:45.424 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2055999999999999e-05 2024-09-16T20:51:45.424 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.424 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.424 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:45.424 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138193+0000", 2024-09-16T20:51:45.424 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3961000000000001e-05 2024-09-16T20:51:45.424 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.424 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.424 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:45.424 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138200+0000", 2024-09-16T20:51:45.424 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4429999999999997e-06 2024-09-16T20:51:45.424 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.424 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.424 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:45.425 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138258+0000", 2024-09-16T20:51:45.425 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7225999999999997e-05 2024-09-16T20:51:45.425 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.425 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.425 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:45.425 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138364+0000", 2024-09-16T20:51:45.425 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000106013 2024-09-16T20:51:45.425 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.425 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.425 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 3", 2024-09-16T20:51:45.425 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.138494+0000", 2024-09-16T20:51:45.425 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013059599999999999 2024-09-16T20:51:45.425 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.426 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.426 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:45.426 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.139220+0000", 2024-09-16T20:51:45.426 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00072539800000000004 2024-09-16T20:51:45.426 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.426 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.426 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:45.426 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222608+0000", 2024-09-16T20:51:45.426 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.083388620999999996 2024-09-16T20:51:45.426 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.426 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.426 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:45.426 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222651+0000", 2024-09-16T20:51:45.426 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2228000000000003e-05 2024-09-16T20:51:45.427 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.427 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.427 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:45.427 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.222697+0000", 2024-09-16T20:51:45.427 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6659000000000002e-05 2024-09-16T20:51:45.427 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:45.427 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:45.427 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:45.427 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.427 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.427 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5138.0:9358 3.c 3.c25699cc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T20:51:45.427 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:51:45.427 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 138.53568596700001, 2024-09-16T20:51:45.427 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0018657580000001, 2024-09-16T20:51:45.428 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:45.428 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:45.428 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:51:45.428 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:45.428 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5138", 2024-09-16T20:51:45.428 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:51:45.428 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 9358 2024-09-16T20:51:45.428 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.428 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:45.428 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.428 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:45.428 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:51:45.428 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:45.429 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.429 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.429 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:45.429 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850313+0000", 2024-09-16T20:51:45.429 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:45.429 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.429 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.429 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:45.429 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850314+0000", 2024-09-16T20:51:45.429 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4099999999999999e-07 2024-09-16T20:51:45.429 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.429 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.429 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:45.430 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850316+0000", 2024-09-16T20:51:45.430 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2469999999999998e-06 2024-09-16T20:51:45.430 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.430 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.430 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:45.430 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850322+0000", 2024-09-16T20:51:45.430 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3470000000000004e-06 2024-09-16T20:51:45.430 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.430 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.430 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:45.430 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850327+0000", 2024-09-16T20:51:45.430 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9450000000000001e-06 2024-09-16T20:51:45.430 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.430 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.430 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:45.431 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850376+0000", 2024-09-16T20:51:45.431 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8257000000000002e-05 2024-09-16T20:51:45.431 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.431 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.431 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:45.431 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.850385+0000", 2024-09-16T20:51:45.431 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9800000000000004e-06 2024-09-16T20:51:45.431 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.431 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.431 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:45.431 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852167+0000", 2024-09-16T20:51:45.431 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0017825659999999 2024-09-16T20:51:45.431 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.432 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.432 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:45.432 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852179+0000", 2024-09-16T20:51:45.432 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1975e-05 2024-09-16T20:51:45.432 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:45.432 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:45.432 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:45.432 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.432 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.432 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5138.0:9382 3.c 3.c25699cc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T20:51:45.432 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:51:45.432 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 138.52733895099999, 2024-09-16T20:51:45.432 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.99358404700000003, 2024-09-16T20:51:45.432 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:45.432 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:45.433 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:51:45.433 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:45.433 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5138", 2024-09-16T20:51:45.433 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:51:45.433 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 9382 2024-09-16T20:51:45.433 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.433 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:45.433 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.433 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:45.433 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:51:45.433 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:45.433 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.434 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.434 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:45.434 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858660+0000", 2024-09-16T20:51:45.434 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:45.434 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.434 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.434 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:45.434 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858661+0000", 2024-09-16T20:51:45.434 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0200000000000002e-07 2024-09-16T20:51:45.434 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.434 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.434 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:45.434 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858664+0000", 2024-09-16T20:51:45.434 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.038e-06 2024-09-16T20:51:45.435 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.435 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.435 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:45.435 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858671+0000", 2024-09-16T20:51:45.435 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0110000000000003e-06 2024-09-16T20:51:45.435 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.435 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.435 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:45.435 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858677+0000", 2024-09-16T20:51:45.435 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0959999999999997e-06 2024-09-16T20:51:45.435 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.435 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.435 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:45.435 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858732+0000", 2024-09-16T20:51:45.436 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4543000000000003e-05 2024-09-16T20:51:45.436 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.436 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.436 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:45.436 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:26.858739+0000", 2024-09-16T20:51:45.436 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5429999999999996e-06 2024-09-16T20:51:45.436 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.436 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.437 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:45.437 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852236+0000", 2024-09-16T20:51:45.437 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.993496453 2024-09-16T20:51:45.437 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.437 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.437 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:45.437 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:27.852244+0000", 2024-09-16T20:51:45.437 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8610000000000004e-06 2024-09-16T20:51:45.437 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:45.437 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:45.437 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:45.437 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.438 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.438 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5763.0:4895 3.a 3:5384606c:::rbd_data.110e1b19a241.0000000000000018:head [read 0~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-16T20:51:45.438 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:45.598212+0000", 2024-09-16T20:51:45.438 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 119.78778690999999, 2024-09-16T20:51:45.438 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0375864829999999, 2024-09-16T20:51:45.438 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:45.438 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:45.438 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:45.438 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:45.438 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5763", 2024-09-16T20:51:45.438 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/623897567", 2024-09-16T20:51:45.438 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4895 2024-09-16T20:51:45.438 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.439 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:45.439 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.439 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:45.439 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598212+0000", 2024-09-16T20:51:45.439 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:45.439 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.439 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.439 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:45.439 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598212+0000", 2024-09-16T20:51:45.439 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:45.439 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.439 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.439 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:45.439 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598214+0000", 2024-09-16T20:51:45.440 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0090000000000002e-06 2024-09-16T20:51:45.440 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.440 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.440 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:45.440 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598222+0000", 2024-09-16T20:51:45.440 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6699999999999994e-06 2024-09-16T20:51:45.440 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.440 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.440 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:45.440 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598241+0000", 2024-09-16T20:51:45.440 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8896999999999999e-05 2024-09-16T20:51:45.440 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.440 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.441 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:45.441 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598250+0000", 2024-09-16T20:51:45.441 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6649999999999992e-06 2024-09-16T20:51:45.441 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.441 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.441 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:45.441 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598325+0000", 2024-09-16T20:51:45.441 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5189000000000001e-05 2024-09-16T20:51:45.441 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.441 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.441 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op must wait for map", 2024-09-16T20:51:45.441 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598333+0000", 2024-09-16T20:51:45.442 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.3170000000000002e-06 2024-09-16T20:51:45.442 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.442 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.442 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:45.442 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601355+0000", 2024-09-16T20:51:45.442 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0030215400000000001 2024-09-16T20:51:45.442 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.442 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.442 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:45.442 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601365+0000", 2024-09-16T20:51:45.442 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0215e-05 2024-09-16T20:51:45.442 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.443 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.443 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:45.443 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635506+0000", 2024-09-16T20:51:45.443 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0341408240000001 2024-09-16T20:51:45.443 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.443 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.443 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:45.443 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635655+0000", 2024-09-16T20:51:45.443 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000149412 2024-09-16T20:51:45.443 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.443 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.443 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:45.443 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635799+0000", 2024-09-16T20:51:45.443 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000143745 2024-09-16T20:51:45.444 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:45.444 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:45.444 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:45.444 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.444 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.444 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5763.0:4900 3.a 3:5384606c:::rbd_data.110e1b19a241.0000000000000018:head [read 8192~24576] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-16T20:51:45.444 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:51:45.444 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 119.787728434, 2024-09-16T20:51:45.444 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0379340800000001, 2024-09-16T20:51:45.444 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:45.444 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:45.444 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:45.444 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:45.444 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5763", 2024-09-16T20:51:45.444 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/623897567", 2024-09-16T20:51:45.445 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 4900 2024-09-16T20:51:45.445 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.445 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:45.445 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.445 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:45.445 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:51:45.445 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:45.445 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.445 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.445 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:45.445 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:51:45.445 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:45.445 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.445 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.446 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:45.446 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598271+0000", 2024-09-16T20:51:45.446 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4300000000000003e-07 2024-09-16T20:51:45.446 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.446 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.446 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:45.446 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598274+0000", 2024-09-16T20:51:45.446 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.976e-06 2024-09-16T20:51:45.446 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.446 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.446 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:45.446 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598282+0000", 2024-09-16T20:51:45.446 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0029999999999994e-06 2024-09-16T20:51:45.446 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.446 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.447 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:45.447 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598289+0000", 2024-09-16T20:51:45.447 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2389999999999999e-06 2024-09-16T20:51:45.447 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.447 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.447 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:45.447 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598430+0000", 2024-09-16T20:51:45.447 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014119600000000001 2024-09-16T20:51:45.447 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.447 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.447 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting_for_map not empty", 2024-09-16T20:51:45.447 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.598435+0000", 2024-09-16T20:51:45.447 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3489999999999999e-06 2024-09-16T20:51:45.447 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.448 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.448 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:45.448 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601429+0000", 2024-09-16T20:51:45.448 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0029942720000000001 2024-09-16T20:51:45.448 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.448 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.448 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:45.448 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:45.601438+0000", 2024-09-16T20:51:45.448 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4759999999999998e-06 2024-09-16T20:51:45.448 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.448 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.448 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:45.448 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.635987+0000", 2024-09-16T20:51:45.448 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0345495010000001 2024-09-16T20:51:45.449 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.449 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.449 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:45.449 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.636056+0000", 2024-09-16T20:51:45.449 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8610999999999998e-05 2024-09-16T20:51:45.449 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.449 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.449 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:45.449 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:46.636205+0000", 2024-09-16T20:51:45.449 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014891399999999999 2024-09-16T20:51:45.449 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:45.449 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:45.450 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:45.450 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.450 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.450 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop(client.6231.0:2182 3.17 e136/81 3:edb5e5cd:::rbd_data.110e61479aee.0000000000000014:head v 136'791, mlcod=135'789)", 2024-09-16T20:51:45.450 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:22.735499+0000", 2024-09-16T20:51:45.450 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 82.650500492999996, 2024-09-16T20:51:45.450 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.12369608, 2024-09-16T20:51:45.450 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:45.450 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:45.450 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:45.450 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:45.451 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.451 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:45.451 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735499+0000", 2024-09-16T20:51:45.451 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:45.451 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.451 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.451 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:45.451 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735499+0000", 2024-09-16T20:51:45.451 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:45.451 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.451 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.451 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:45.451 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735499+0000", 2024-09-16T20:51:45.451 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8099999999999999e-07 2024-09-16T20:51:45.451 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.452 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.452 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:45.452 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735506+0000", 2024-09-16T20:51:45.452 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3579999999999997e-06 2024-09-16T20:51:45.452 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.452 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.452 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:45.452 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735517+0000", 2024-09-16T20:51:45.452 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0869999999999999e-05 2024-09-16T20:51:45.452 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.452 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.452 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:45.452 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735533+0000", 2024-09-16T20:51:45.452 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5858e-05 2024-09-16T20:51:45.453 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.453 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.453 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:45.453 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735584+0000", 2024-09-16T20:51:45.453 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0470999999999998e-05 2024-09-16T20:51:45.453 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.453 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.453 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:45.453 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.735607+0000", 2024-09-16T20:51:45.453 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2875999999999999e-05 2024-09-16T20:51:45.453 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.453 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.453 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:45.454 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.859144+0000", 2024-09-16T20:51:45.454 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.123537701 2024-09-16T20:51:45.454 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.454 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.454 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:45.454 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.859195+0000", 2024-09-16T20:51:45.454 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0664999999999997e-05 2024-09-16T20:51:45.454 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:45.454 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:45.454 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:45.454 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.454 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.454 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5204 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 2523136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:51:45.454 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.883660+0000", 2024-09-16T20:51:45.455 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 81.502338963, 2024-09-16T20:51:45.455 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052517638999999998, 2024-09-16T20:51:45.455 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:45.455 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:45.455 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:45.455 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:45.455 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:45.455 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:45.455 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5204 2024-09-16T20:51:45.455 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.455 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:45.455 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.455 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:45.455 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883660+0000", 2024-09-16T20:51:45.455 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:45.456 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.456 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.456 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:45.456 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883660+0000", 2024-09-16T20:51:45.456 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:45.456 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.456 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.456 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:45.456 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883661+0000", 2024-09-16T20:51:45.456 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9999999999999998e-07 2024-09-16T20:51:45.456 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.456 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.456 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:45.456 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883663+0000", 2024-09-16T20:51:45.457 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.4480000000000001e-06 2024-09-16T20:51:45.457 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.457 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.457 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:45.457 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883669+0000", 2024-09-16T20:51:45.457 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.071e-06 2024-09-16T20:51:45.457 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.457 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.457 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:45.457 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883675+0000", 2024-09-16T20:51:45.457 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.366e-06 2024-09-16T20:51:45.457 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.457 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.458 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:45.458 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.935970+0000", 2024-09-16T20:51:45.458 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052295346999999999 2024-09-16T20:51:45.458 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.458 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.458 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:45.458 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936064+0000", 2024-09-16T20:51:45.458 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4438000000000006e-05 2024-09-16T20:51:45.458 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.458 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.458 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:45.458 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936178+0000", 2024-09-16T20:51:45.458 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000113569 2024-09-16T20:51:45.458 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:45.459 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:45.459 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:45.459 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.459 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.459 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5205 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 2543616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:51:45.459 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.883745+0000", 2024-09-16T20:51:45.459 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 81.502254205, 2024-09-16T20:51:45.459 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052692112999999999, 2024-09-16T20:51:45.459 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:45.459 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:45.459 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:45.459 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:45.459 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:45.459 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:45.459 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5205 2024-09-16T20:51:45.460 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.460 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:45.460 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.460 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:45.460 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883745+0000", 2024-09-16T20:51:45.460 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:45.460 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.460 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.460 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:45.460 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883745+0000", 2024-09-16T20:51:45.460 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:45.460 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.460 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.460 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:45.461 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883746+0000", 2024-09-16T20:51:45.461 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2699999999999999e-07 2024-09-16T20:51:45.461 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.461 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.461 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:45.461 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883750+0000", 2024-09-16T20:51:45.461 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7050000000000001e-06 2024-09-16T20:51:45.461 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.461 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.461 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:45.461 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883769+0000", 2024-09-16T20:51:45.461 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9165e-05 2024-09-16T20:51:45.461 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.462 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.462 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:45.462 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883778+0000", 2024-09-16T20:51:45.462 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.6690000000000001e-06 2024-09-16T20:51:45.462 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.462 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.462 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:45.462 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936228+0000", 2024-09-16T20:51:45.462 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052449587999999998 2024-09-16T20:51:45.462 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.462 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.462 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:45.462 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936333+0000", 2024-09-16T20:51:45.463 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010511800000000001 2024-09-16T20:51:45.463 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.463 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.463 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:45.463 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936437+0000", 2024-09-16T20:51:45.463 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000104141 2024-09-16T20:51:45.463 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:45.463 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:45.463 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:45.463 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.463 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.463 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5206 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 1167360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:51:45.463 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.883802+0000", 2024-09-16T20:51:45.463 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 81.502197710999994, 2024-09-16T20:51:45.464 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052800780999999998, 2024-09-16T20:51:45.464 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:45.464 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:45.464 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:45.464 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:45.464 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:45.464 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:45.464 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5206 2024-09-16T20:51:45.464 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.464 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:45.464 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.464 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:45.464 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883802+0000", 2024-09-16T20:51:45.465 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:45.465 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.465 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.465 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:45.465 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883802+0000", 2024-09-16T20:51:45.465 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:45.465 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.465 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.465 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:45.465 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883933+0000", 2024-09-16T20:51:45.465 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013182999999999999 2024-09-16T20:51:45.465 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.465 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.465 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:45.466 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883937+0000", 2024-09-16T20:51:45.466 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.771e-06 2024-09-16T20:51:45.466 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.466 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.466 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:45.466 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883953+0000", 2024-09-16T20:51:45.466 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5409000000000001e-05 2024-09-16T20:51:45.466 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.466 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.466 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:45.466 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883959+0000", 2024-09-16T20:51:45.466 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0700000000000003e-06 2024-09-16T20:51:45.466 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.466 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.466 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:45.467 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936476+0000", 2024-09-16T20:51:45.467 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052517434000000002 2024-09-16T20:51:45.467 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.467 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.467 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:45.467 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936529+0000", 2024-09-16T20:51:45.467 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2969000000000001e-05 2024-09-16T20:51:45.467 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.467 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.467 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:45.467 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936602+0000", 2024-09-16T20:51:45.467 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3298000000000002e-05 2024-09-16T20:51:45.467 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:45.467 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:45.468 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:45.468 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.468 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.468 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5207 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 1904640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:51:45.468 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.883980+0000", 2024-09-16T20:51:45.468 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 81.502019501999996, 2024-09-16T20:51:45.468 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052759922000000001, 2024-09-16T20:51:45.468 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:45.468 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:45.468 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:45.468 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:45.468 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:45.468 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:45.468 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5207 2024-09-16T20:51:45.469 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.469 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:45.469 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.469 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:45.469 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883980+0000", 2024-09-16T20:51:45.469 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:45.469 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.469 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.469 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:45.469 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.883980+0000", 2024-09-16T20:51:45.469 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:45.469 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.469 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.470 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:45.470 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884126+0000", 2024-09-16T20:51:45.470 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014577399999999999 2024-09-16T20:51:45.470 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.470 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.470 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:45.470 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884130+0000", 2024-09-16T20:51:45.470 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9729999999999999e-06 2024-09-16T20:51:45.470 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.470 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.470 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:45.470 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884139+0000", 2024-09-16T20:51:45.470 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8660000000000001e-06 2024-09-16T20:51:45.471 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.471 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.471 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:45.471 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884144+0000", 2024-09-16T20:51:45.471 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3669999999999999e-06 2024-09-16T20:51:45.471 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.471 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.471 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:45.471 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936635+0000", 2024-09-16T20:51:45.471 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052491705 2024-09-16T20:51:45.471 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.471 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.471 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:45.472 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936679+0000", 2024-09-16T20:51:45.472 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3105999999999997e-05 2024-09-16T20:51:45.472 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.472 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.472 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:45.472 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936740+0000", 2024-09-16T20:51:45.472 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1130999999999995e-05 2024-09-16T20:51:45.472 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:45.472 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:45.472 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:45.472 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.472 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.472 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5208 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 131072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:51:45.472 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.884165+0000", 2024-09-16T20:51:45.473 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 81.501834205999998, 2024-09-16T20:51:45.473 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052746603000000003, 2024-09-16T20:51:45.473 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:45.473 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:45.473 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:45.473 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:45.473 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:45.473 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:45.473 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5208 2024-09-16T20:51:45.473 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.473 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:45.473 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.473 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:45.474 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884165+0000", 2024-09-16T20:51:45.474 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:45.474 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.474 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.474 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:45.474 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884165+0000", 2024-09-16T20:51:45.474 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:45.474 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.474 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.474 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:45.474 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884463+0000", 2024-09-16T20:51:45.474 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00029801699999999998 2024-09-16T20:51:45.474 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.474 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.474 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:45.475 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884467+0000", 2024-09-16T20:51:45.475 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8210000000000003e-06 2024-09-16T20:51:45.475 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.475 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.475 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:45.475 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884477+0000", 2024-09-16T20:51:45.475 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.7030000000000001e-06 2024-09-16T20:51:45.475 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.475 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.475 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:45.475 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884485+0000", 2024-09-16T20:51:45.475 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9710000000000004e-06 2024-09-16T20:51:45.475 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.475 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.476 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:45.476 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936785+0000", 2024-09-16T20:51:45.476 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052300665000000003 2024-09-16T20:51:45.476 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.476 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.476 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:45.476 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936851+0000", 2024-09-16T20:51:45.476 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6159000000000001e-05 2024-09-16T20:51:45.476 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.476 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.476 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:45.476 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936912+0000", 2024-09-16T20:51:45.476 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0266999999999997e-05 2024-09-16T20:51:45.476 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:45.476 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:45.477 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:45.477 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.477 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.477 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:5209 3.1b 3:db07e6dd:::rbd_data.110e7ef3004.0000000000000003:2d [read 1925120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-16T20:51:45.477 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:23.884504+0000", 2024-09-16T20:51:45.477 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 81.501495320999993, 2024-09-16T20:51:45.477 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052575445999999998, 2024-09-16T20:51:45.477 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:45.477 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:45.477 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:45.477 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:45.477 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:45.477 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:45.477 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5209 2024-09-16T20:51:45.478 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.478 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:45.478 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.478 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:45.478 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884504+0000", 2024-09-16T20:51:45.478 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:45.478 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.478 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.478 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:45.478 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884504+0000", 2024-09-16T20:51:45.478 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:45.478 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.478 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.479 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:45.479 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884741+0000", 2024-09-16T20:51:45.479 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00023679899999999999 2024-09-16T20:51:45.479 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.479 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.479 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:45.479 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884747+0000", 2024-09-16T20:51:45.479 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0009999999999998e-06 2024-09-16T20:51:45.479 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.479 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.479 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:45.479 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884764+0000", 2024-09-16T20:51:45.479 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7371999999999999e-05 2024-09-16T20:51:45.480 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.480 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.480 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:45.480 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.884772+0000", 2024-09-16T20:51:45.480 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1349999999999992e-06 2024-09-16T20:51:45.480 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.480 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.480 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:45.480 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.936956+0000", 2024-09-16T20:51:45.480 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052183754999999998 2024-09-16T20:51:45.480 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.480 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.480 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:45.480 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937003+0000", 2024-09-16T20:51:45.480 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6925000000000002e-05 2024-09-16T20:51:45.481 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.481 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.481 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:45.481 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:23.937079+0000", 2024-09-16T20:51:45.481 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6458999999999994e-05 2024-09-16T20:51:45.481 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:45.481 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:45.481 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:45.481 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.481 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.481 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:30174 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_remove in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e176)", 2024-09-16T20:51:45.481 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:51:03.615665+0000", 2024-09-16T20:51:45.481 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 41.770333802000003, 2024-09-16T20:51:45.481 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.96831816599999998, 2024-09-16T20:51:45.482 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:45.482 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:45.482 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:45.482 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:45.482 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:51:45.482 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:45.482 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 30174 2024-09-16T20:51:45.482 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.482 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:45.482 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.482 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:45.482 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615665+0000", 2024-09-16T20:51:45.482 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:45.482 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.483 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.483 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:45.483 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615665+0000", 2024-09-16T20:51:45.483 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:45.483 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.483 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.483 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:45.483 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615667+0000", 2024-09-16T20:51:45.483 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0020000000000001e-06 2024-09-16T20:51:45.483 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.483 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.483 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:45.483 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615675+0000", 2024-09-16T20:51:45.483 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5120000000000002e-06 2024-09-16T20:51:45.483 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.484 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.484 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:45.484 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615692+0000", 2024-09-16T20:51:45.484 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.6977e-05 2024-09-16T20:51:45.484 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.484 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.484 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:45.484 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615701+0000", 2024-09-16T20:51:45.484 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6119999999999999e-06 2024-09-16T20:51:45.484 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.484 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.484 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:45.484 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615782+0000", 2024-09-16T20:51:45.485 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1804000000000005e-05 2024-09-16T20:51:45.485 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.485 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.485 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:45.485 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:03.615795+0000", 2024-09-16T20:51:45.485 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2464e-05 2024-09-16T20:51:45.485 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.485 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.485 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:45.485 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.580071+0000", 2024-09-16T20:51:45.485 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.96427590100000005 2024-09-16T20:51:45.485 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.485 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.485 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:45.486 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.580247+0000", 2024-09-16T20:51:45.486 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000175754 2024-09-16T20:51:45.486 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.486 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.486 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 2", 2024-09-16T20:51:45.486 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.580436+0000", 2024-09-16T20:51:45.486 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00018945299999999999 2024-09-16T20:51:45.486 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.486 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.486 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:45.486 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.581045+0000", 2024-09-16T20:51:45.486 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00060937600000000001 2024-09-16T20:51:45.486 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.486 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.486 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:45.487 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.583883+0000", 2024-09-16T20:51:45.487 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0028372900000000001 2024-09-16T20:51:45.487 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.487 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.487 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:45.487 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.583944+0000", 2024-09-16T20:51:45.487 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1848999999999997e-05 2024-09-16T20:51:45.487 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.487 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.487 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:45.487 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.583984+0000", 2024-09-16T20:51:45.487 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9171999999999997e-05 2024-09-16T20:51:45.487 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:45.487 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:45.488 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:45.488 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.488 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.488 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:30201 3.f 3:f6903a53:::rbd_object_map.110eca9b190a.000000000000001a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e177)", 2024-09-16T20:51:45.488 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:51:04.603586+0000", 2024-09-16T20:51:45.488 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 40.782413695000002, 2024-09-16T20:51:45.488 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.98028146800000004, 2024-09-16T20:51:45.488 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:45.488 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:45.488 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:45.488 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:45.488 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:51:45.488 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:45.488 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 30201 2024-09-16T20:51:45.489 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.489 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:45.489 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.489 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:45.489 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603586+0000", 2024-09-16T20:51:45.489 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:45.489 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.489 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.489 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:45.489 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603586+0000", 2024-09-16T20:51:45.489 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:45.489 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.489 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.489 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:45.490 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603587+0000", 2024-09-16T20:51:45.490 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.519e-06 2024-09-16T20:51:45.490 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.490 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.490 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:45.490 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603593+0000", 2024-09-16T20:51:45.490 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1449999999999996e-06 2024-09-16T20:51:45.490 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.490 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.490 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:45.490 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603606+0000", 2024-09-16T20:51:45.490 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2797e-05 2024-09-16T20:51:45.491 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.491 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.491 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:45.491 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603612+0000", 2024-09-16T20:51:45.491 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8710000000000002e-06 2024-09-16T20:51:45.491 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.491 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.491 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:45.491 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603675+0000", 2024-09-16T20:51:45.491 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2728e-05 2024-09-16T20:51:45.491 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.491 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.491 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:45.492 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:04.603687+0000", 2024-09-16T20:51:45.492 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.253e-05 2024-09-16T20:51:45.492 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.492 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.492 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:45.492 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:05.583358+0000", 2024-09-16T20:51:45.492 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.97967074600000004 2024-09-16T20:51:45.492 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.492 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.492 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:45.492 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:05.583515+0000", 2024-09-16T20:51:45.492 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00015755700000000001 2024-09-16T20:51:45.492 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.493 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.493 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:45.493 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:05.583867+0000", 2024-09-16T20:51:45.493 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00035157500000000002 2024-09-16T20:51:45.493 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:45.493 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:45.493 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:45.493 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:45.493 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:45.493 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:45.495 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T20:51:45.495 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-09-16T20:51:45.493+0000 7f2618246640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31988) UID: 0 2024-09-16T20:51:45.574 INFO:teuthology.orchestra.run.smithi029.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-16T20:51:45.636 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-09-16T20:51:45.635+0000 7f4b2604f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31987) UID: 0 2024-09-16T20:51:45.656 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:45.656 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:51:45.656 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.656 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:34516 3.d 3:b4d4919c:::rbd_header.110e7f949337:head [watch watch cookie 140213636978240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e218)", 2024-09-16T20:51:45.657 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:51:45.655644+0000", 2024-09-16T20:51:45.657 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.00027622600000000002, 2024-09-16T20:51:45.657 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00030228500000000001, 2024-09-16T20:51:45.657 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:45.657 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:45.657 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "waiting for sub ops", 2024-09-16T20:51:45.657 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:45.657 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:51:45.657 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:45.657 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 34516 2024-09-16T20:51:45.657 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.658 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:45.658 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.658 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:45.658 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:45.655644+0000", 2024-09-16T20:51:45.658 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:45.658 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.658 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.658 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:45.658 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:45.655644+0000", 2024-09-16T20:51:45.658 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:45.658 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.658 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.659 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:45.659 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:45.655644+0000", 2024-09-16T20:51:45.659 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1099999999999995e-07 2024-09-16T20:51:45.659 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.659 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.659 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:45.659 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:45.655648+0000", 2024-09-16T20:51:45.659 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1530000000000001e-06 2024-09-16T20:51:45.659 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.659 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.659 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:45.659 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:45.655657+0000", 2024-09-16T20:51:45.660 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.1770000000000004e-06 2024-09-16T20:51:45.660 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.660 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.660 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:45.660 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:45.655662+0000", 2024-09-16T20:51:45.660 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3319999999999999e-06 2024-09-16T20:51:45.660 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.660 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.660 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:45.660 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:45.655695+0000", 2024-09-16T20:51:45.660 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2669000000000002e-05 2024-09-16T20:51:45.661 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.661 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.661 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:45.661 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:45.655735+0000", 2024-09-16T20:51:45.661 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0213999999999998e-05 2024-09-16T20:51:45.661 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.661 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.661 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:45.661 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:51:45.655815+0000", 2024-09-16T20:51:45.661 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9748999999999998e-05 2024-09-16T20:51:45.661 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:45.661 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:45.662 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:45.662 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:45.662 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-16T20:51:45.662 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 1 2024-09-16T20:51:45.662 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:45.666 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T20:51:45.684 INFO:teuthology.orchestra.run.smithi139.stdout:All operations completed A-OK! 2024-09-16T20:51:45.690 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T20:51:45.695 DEBUG:teuthology.parallel:result is None 2024-09-16T20:51:45.695 DEBUG:teuthology.run_tasks:Unwinding manager rbd_fsx 2024-09-16T20:51:45.705 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-16T20:51:45.740 INFO:tasks.thrashosds:joining thrashosds 2024-09-16T20:51:45.850 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:45.850 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:51:45.850 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:51:45.850 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:51:45.850 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:45.859 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T20:51:45.968 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:45.968 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:51:45.969 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:51:45.969 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:51:45.969 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.969 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4159.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T20:51:45.969 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:51:45.969 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 232.712519929, 2024-09-16T20:51:45.969 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67502346999999996, 2024-09-16T20:51:45.969 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:45.969 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:45.969 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:45.969 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:45.969 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4159", 2024-09-16T20:51:45.970 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.43:0/1277892741", 2024-09-16T20:51:45.970 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 1 2024-09-16T20:51:45.970 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.970 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:45.970 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.970 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:45.970 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:51:45.970 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:45.970 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.970 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.970 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:45.970 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254126+0000", 2024-09-16T20:51:45.970 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:45.971 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.971 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.971 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:45.971 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254128+0000", 2024-09-16T20:51:45.971 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.463e-06 2024-09-16T20:51:45.971 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.971 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.971 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:45.971 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254135+0000", 2024-09-16T20:51:45.971 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1579999999999999e-06 2024-09-16T20:51:45.971 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.971 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.971 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:45.972 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254146+0000", 2024-09-16T20:51:45.972 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1341e-05 2024-09-16T20:51:45.972 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.972 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.972 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:45.972 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254155+0000", 2024-09-16T20:51:45.972 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.1549999999999996e-06 2024-09-16T20:51:45.972 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.972 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.972 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:45.972 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254221+0000", 2024-09-16T20:51:45.972 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5470999999999997e-05 2024-09-16T20:51:45.973 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.973 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.973 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:45.973 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.254231+0000", 2024-09-16T20:51:45.973 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0592e-05 2024-09-16T20:51:45.973 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.973 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.973 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:45.973 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926711+0000", 2024-09-16T20:51:45.973 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.67247946599999997 2024-09-16T20:51:45.973 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.973 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.974 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:45.974 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.926910+0000", 2024-09-16T20:51:45.974 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019934 2024-09-16T20:51:45.974 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.974 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.974 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 5", 2024-09-16T20:51:45.974 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.927159+0000", 2024-09-16T20:51:45.974 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00024876599999999998 2024-09-16T20:51:45.974 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.974 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.974 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:45.974 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.928797+0000", 2024-09-16T20:51:45.975 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001638308 2024-09-16T20:51:45.975 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.975 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.975 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:45.975 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929026+0000", 2024-09-16T20:51:45.975 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00022876099999999999 2024-09-16T20:51:45.975 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.975 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.975 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:45.975 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929102+0000", 2024-09-16T20:51:45.975 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6184000000000001e-05 2024-09-16T20:51:45.975 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.975 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.975 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:45.975 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:47:53.929150+0000", 2024-09-16T20:51:45.976 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7465e-05 2024-09-16T20:51:45.976 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:45.976 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:45.976 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:45.976 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.976 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.976 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3574 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:45.976 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:51:45.976 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 176.485356397, 2024-09-16T20:51:45.976 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053579441999999998, 2024-09-16T20:51:45.976 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:45.976 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:45.977 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:45.977 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:45.977 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:45.977 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:45.977 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3574 2024-09-16T20:51:45.977 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.977 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:45.977 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.977 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:45.977 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:51:45.977 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:45.977 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.977 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.978 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:45.978 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:51:45.978 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:45.978 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.978 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.978 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:45.978 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481290+0000", 2024-09-16T20:51:45.978 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2e-07 2024-09-16T20:51:45.978 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.978 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.978 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:45.978 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481292+0000", 2024-09-16T20:51:45.978 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2630000000000002e-06 2024-09-16T20:51:45.978 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.979 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.979 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:45.979 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481297+0000", 2024-09-16T20:51:45.979 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.296e-06 2024-09-16T20:51:45.979 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.979 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.979 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:45.979 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481300+0000", 2024-09-16T20:51:45.979 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7019999999999999e-06 2024-09-16T20:51:45.979 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.979 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.979 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:45.979 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525533+0000", 2024-09-16T20:51:45.980 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.044232318 2024-09-16T20:51:45.980 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.980 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.980 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:45.980 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525619+0000", 2024-09-16T20:51:45.980 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5892999999999999e-05 2024-09-16T20:51:45.980 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.980 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.980 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:45.980 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.525987+0000", 2024-09-16T20:51:45.980 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00036836099999999999 2024-09-16T20:51:45.980 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.980 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.980 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:45.981 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531779+0000", 2024-09-16T20:51:45.981 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0057918240000000001 2024-09-16T20:51:45.981 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.981 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.981 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:45.981 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534747+0000", 2024-09-16T20:51:45.981 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0029681600000000001 2024-09-16T20:51:45.981 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.981 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.981 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:45.981 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534832+0000", 2024-09-16T20:51:45.981 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5180999999999996e-05 2024-09-16T20:51:45.981 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.982 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.982 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:45.982 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534869+0000", 2024-09-16T20:51:45.982 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7024000000000003e-05 2024-09-16T20:51:45.982 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:45.982 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:45.982 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:45.982 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.982 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.982 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3575 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:45.982 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:51:45.982 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 176.48534165699999, 2024-09-16T20:51:45.982 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053801463000000001, 2024-09-16T20:51:45.983 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:45.983 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:45.983 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:45.983 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:45.983 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:45.983 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:45.983 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3575 2024-09-16T20:51:45.983 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.983 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:45.983 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.983 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:45.983 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:51:45.983 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:45.984 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.984 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.984 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:45.984 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481304+0000", 2024-09-16T20:51:45.984 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:45.984 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.984 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.984 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:45.984 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481305+0000", 2024-09-16T20:51:45.984 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5400000000000002e-07 2024-09-16T20:51:45.984 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.984 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.984 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:45.985 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481307+0000", 2024-09-16T20:51:45.985 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7800000000000001e-06 2024-09-16T20:51:45.985 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.985 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.985 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:45.985 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481312+0000", 2024-09-16T20:51:45.985 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6349999999999997e-06 2024-09-16T20:51:45.985 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.985 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.985 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:45.985 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481315+0000", 2024-09-16T20:51:45.985 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7829999999999999e-06 2024-09-16T20:51:45.985 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.985 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.986 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:45.986 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.526876+0000", 2024-09-16T20:51:45.986 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.045561070000000002 2024-09-16T20:51:45.986 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.986 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.986 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:45.986 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.526954+0000", 2024-09-16T20:51:45.986 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8108000000000005e-05 2024-09-16T20:51:45.986 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.986 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.986 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:45.986 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527282+0000", 2024-09-16T20:51:45.986 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000327483 2024-09-16T20:51:45.986 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.987 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.987 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:45.987 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531784+0000", 2024-09-16T20:51:45.987 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0045020169999999997 2024-09-16T20:51:45.987 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.987 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.987 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:45.987 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534921+0000", 2024-09-16T20:51:45.987 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.003137791 2024-09-16T20:51:45.987 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.987 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.987 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:45.987 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534976+0000", 2024-09-16T20:51:45.988 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4953000000000001e-05 2024-09-16T20:51:45.988 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.988 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.988 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:45.988 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535106+0000", 2024-09-16T20:51:45.988 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000129589 2024-09-16T20:51:45.988 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:45.988 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:45.988 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:45.988 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.988 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.988 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3576 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:45.988 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:51:45.989 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 176.48532774500001, 2024-09-16T20:51:45.989 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053955436000000002, 2024-09-16T20:51:45.989 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:45.989 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:45.989 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:45.989 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:45.989 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:45.989 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:45.989 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3576 2024-09-16T20:51:45.989 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.989 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:45.989 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.989 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:45.990 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:51:45.990 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:45.990 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.990 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.990 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:45.990 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481318+0000", 2024-09-16T20:51:45.990 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:45.990 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.990 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.990 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:45.990 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481319+0000", 2024-09-16T20:51:45.990 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.53e-07 2024-09-16T20:51:45.990 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.990 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.991 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:45.991 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481320+0000", 2024-09-16T20:51:45.991 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5230000000000001e-06 2024-09-16T20:51:45.991 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.991 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.991 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:45.991 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481330+0000", 2024-09-16T20:51:45.991 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0190000000000001e-05 2024-09-16T20:51:45.991 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.991 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.991 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:45.991 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481333+0000", 2024-09-16T20:51:45.991 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.151e-06 2024-09-16T20:51:45.991 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.991 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.992 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:45.992 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527689+0000", 2024-09-16T20:51:45.992 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.046355192000000003 2024-09-16T20:51:45.992 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.992 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.992 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:45.992 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.527764+0000", 2024-09-16T20:51:45.992 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5776000000000001e-05 2024-09-16T20:51:45.992 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.992 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.992 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:45.992 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528141+0000", 2024-09-16T20:51:45.992 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00037647599999999999 2024-09-16T20:51:45.993 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.993 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.993 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:45.993 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531788+0000", 2024-09-16T20:51:45.993 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0036470679999999998 2024-09-16T20:51:45.993 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.993 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.993 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:45.993 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535174+0000", 2024-09-16T20:51:45.994 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0033862670000000001 2024-09-16T20:51:45.994 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.994 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.994 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:45.994 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535236+0000", 2024-09-16T20:51:45.994 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1797999999999993e-05 2024-09-16T20:51:45.994 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.994 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.994 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:45.994 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535274+0000", 2024-09-16T20:51:45.994 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7741999999999999e-05 2024-09-16T20:51:45.994 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:45.994 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:45.994 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:45.994 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.995 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.995 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3577 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:45.995 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:51:45.995 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 176.48530614500001, 2024-09-16T20:51:45.995 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054073807000000002, 2024-09-16T20:51:45.995 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:45.995 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:45.995 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:45.995 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:45.995 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:45.995 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:45.995 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3577 2024-09-16T20:51:45.995 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.995 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:45.995 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.996 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:45.996 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:51:45.996 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:45.996 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.996 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.996 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:45.996 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:51:45.996 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:45.996 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.996 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.996 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:45.996 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481340+0000", 2024-09-16T20:51:45.996 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.96e-07 2024-09-16T20:51:45.996 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.997 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.997 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:45.997 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481342+0000", 2024-09-16T20:51:45.997 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.714e-06 2024-09-16T20:51:45.997 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.997 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.997 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:45.997 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481349+0000", 2024-09-16T20:51:45.997 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4930000000000001e-06 2024-09-16T20:51:45.997 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.997 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.997 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:45.997 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481353+0000", 2024-09-16T20:51:45.998 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4879999999999999e-06 2024-09-16T20:51:45.998 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.998 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.998 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:45.998 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528619+0000", 2024-09-16T20:51:45.998 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.047266351999999998 2024-09-16T20:51:45.998 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.998 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.998 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:45.998 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.528706+0000", 2024-09-16T20:51:45.998 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6796e-05 2024-09-16T20:51:45.998 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.998 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.999 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:45.999 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529148+0000", 2024-09-16T20:51:45.999 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00044182999999999999 2024-09-16T20:51:45.999 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.999 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.999 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:45.999 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531792+0000", 2024-09-16T20:51:45.999 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0026443119999999998 2024-09-16T20:51:45.999 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:45.999 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:45.999 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:45.999 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535319+0000", 2024-09-16T20:51:45.999 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0035266439999999998 2024-09-16T20:51:46.000 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.000 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.000 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:46.000 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535372+0000", 2024-09-16T20:51:46.000 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3207000000000001e-05 2024-09-16T20:51:46.000 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.000 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.000 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:46.000 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535414+0000", 2024-09-16T20:51:46.000 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1675000000000003e-05 2024-09-16T20:51:46.000 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.000 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:46.000 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.000 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.001 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.001 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3578 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:46.001 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:51:46.001 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 176.485284976, 2024-09-16T20:51:46.001 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054189586999999997, 2024-09-16T20:51:46.001 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:46.001 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:46.001 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:46.001 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:46.001 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:46.001 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:46.001 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3578 2024-09-16T20:51:46.002 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.002 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:46.002 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.002 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:46.002 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:51:46.002 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.002 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.002 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.002 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:46.002 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:51:46.002 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.002 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.002 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.002 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:46.003 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481361+0000", 2024-09-16T20:51:46.003 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0800000000000001e-07 2024-09-16T20:51:46.003 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.003 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.003 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:46.003 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481363+0000", 2024-09-16T20:51:46.003 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0339999999999999e-06 2024-09-16T20:51:46.003 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.003 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.003 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:46.003 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481371+0000", 2024-09-16T20:51:46.003 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4440000000000002e-06 2024-09-16T20:51:46.003 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.004 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.004 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:46.004 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481374+0000", 2024-09-16T20:51:46.004 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2049999999999998e-06 2024-09-16T20:51:46.004 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.004 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.004 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:46.004 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529555+0000", 2024-09-16T20:51:46.004 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.048181202999999999 2024-09-16T20:51:46.004 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.004 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.004 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:46.004 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529616+0000", 2024-09-16T20:51:46.005 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0912000000000002e-05 2024-09-16T20:51:46.005 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.005 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.005 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:46.005 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.529939+0000", 2024-09-16T20:51:46.005 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00032269800000000001 2024-09-16T20:51:46.005 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.005 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.005 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:46.005 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532602+0000", 2024-09-16T20:51:46.005 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0026634940000000002 2024-09-16T20:51:46.005 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.005 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.006 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:46.006 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535465+0000", 2024-09-16T20:51:46.006 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.002862149 2024-09-16T20:51:46.006 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.006 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.006 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:46.006 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535516+0000", 2024-09-16T20:51:46.006 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1696000000000001e-05 2024-09-16T20:51:46.006 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.006 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.006 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:46.006 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535551+0000", 2024-09-16T20:51:46.006 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4443999999999998e-05 2024-09-16T20:51:46.006 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.007 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:46.007 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.007 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.007 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.007 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3579 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:46.007 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:51:46.007 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 176.485266537, 2024-09-16T20:51:46.007 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054295151, 2024-09-16T20:51:46.007 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:46.007 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:46.007 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:46.007 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:46.007 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:46.008 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:46.008 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3579 2024-09-16T20:51:46.008 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.008 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:46.008 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.008 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:46.008 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:51:46.008 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.008 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.008 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.008 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:46.008 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:51:46.008 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.008 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.009 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.009 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:46.009 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481380+0000", 2024-09-16T20:51:46.009 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3300000000000001e-07 2024-09-16T20:51:46.009 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.009 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.009 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:46.009 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481381+0000", 2024-09-16T20:51:46.009 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.649e-06 2024-09-16T20:51:46.009 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.009 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.009 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:46.009 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481387+0000", 2024-09-16T20:51:46.009 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4519999999999996e-06 2024-09-16T20:51:46.010 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.010 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.010 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:46.010 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481390+0000", 2024-09-16T20:51:46.010 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0170000000000001e-06 2024-09-16T20:51:46.010 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.010 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.010 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:46.010 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530756+0000", 2024-09-16T20:51:46.010 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.049365882999999999 2024-09-16T20:51:46.010 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.010 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.010 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:46.010 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.530845+0000", 2024-09-16T20:51:46.011 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9197000000000007e-05 2024-09-16T20:51:46.011 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.011 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.011 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:46.011 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531191+0000", 2024-09-16T20:51:46.011 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00034562500000000001 2024-09-16T20:51:46.011 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.011 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.011 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:46.011 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534078+0000", 2024-09-16T20:51:46.011 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0028878179999999999 2024-09-16T20:51:46.011 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.011 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.012 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:46.012 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535589+0000", 2024-09-16T20:51:46.012 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00151085 2024-09-16T20:51:46.012 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.012 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.012 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:46.012 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535639+0000", 2024-09-16T20:51:46.012 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9697000000000002e-05 2024-09-16T20:51:46.012 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.012 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.012 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:46.012 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535675+0000", 2024-09-16T20:51:46.012 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5729999999999998e-05 2024-09-16T20:51:46.013 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.013 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:46.013 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.013 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.013 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.013 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3580 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:46.013 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:51:46.013 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 176.48525290500001, 2024-09-16T20:51:46.013 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054397361999999998, 2024-09-16T20:51:46.013 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:46.013 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:46.013 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:46.013 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:46.014 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:46.014 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:46.014 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3580 2024-09-16T20:51:46.014 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.014 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:46.014 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.014 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:46.014 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:51:46.014 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.014 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.014 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.014 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:46.014 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:51:46.014 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.015 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.015 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.015 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:46.015 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481393+0000", 2024-09-16T20:51:46.015 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2600000000000001e-07 2024-09-16T20:51:46.015 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.015 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.015 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:46.015 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481395+0000", 2024-09-16T20:51:46.015 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.544e-06 2024-09-16T20:51:46.015 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.015 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.015 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:46.016 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481404+0000", 2024-09-16T20:51:46.016 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8850000000000005e-06 2024-09-16T20:51:46.016 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.016 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.016 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:46.016 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481407+0000", 2024-09-16T20:51:46.016 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1540000000000002e-06 2024-09-16T20:51:46.016 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.016 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.016 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:46.016 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531823+0000", 2024-09-16T20:51:46.016 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050415637999999999 2024-09-16T20:51:46.016 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.017 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.017 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:46.017 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.531895+0000", 2024-09-16T20:51:46.017 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2179000000000001e-05 2024-09-16T20:51:46.017 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.017 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.017 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:46.017 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532236+0000", 2024-09-16T20:51:46.017 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00034073799999999999 2024-09-16T20:51:46.017 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.017 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.017 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:46.017 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.534083+0000", 2024-09-16T20:51:46.018 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001847298 2024-09-16T20:51:46.018 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.018 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.018 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:46.018 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535714+0000", 2024-09-16T20:51:46.018 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0016316810000000001 2024-09-16T20:51:46.018 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.018 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.018 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:46.018 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535759+0000", 2024-09-16T20:51:46.018 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4478999999999999e-05 2024-09-16T20:51:46.018 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.018 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.019 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:46.019 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535791+0000", 2024-09-16T20:51:46.019 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.154e-05 2024-09-16T20:51:46.019 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.019 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:46.019 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.019 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.019 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.019 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4927.0:3581 3.1c 3:3c335954:::rbd_object_map.110ee9a2c138:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T20:51:46.019 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:51:46.019 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 176.48523090800001, 2024-09-16T20:51:46.019 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.057094403000000002, 2024-09-16T20:51:46.019 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:46.019 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:46.020 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:46.020 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:46.020 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4927", 2024-09-16T20:51:46.020 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:46.020 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3581 2024-09-16T20:51:46.020 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.020 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:46.020 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.020 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:46.020 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:51:46.020 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.020 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.020 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.021 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:46.021 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:51:46.021 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.021 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.021 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.021 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:46.021 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481415+0000", 2024-09-16T20:51:46.021 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8799999999999998e-07 2024-09-16T20:51:46.021 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.021 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.021 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:46.021 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481418+0000", 2024-09-16T20:51:46.021 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1859999999999999e-06 2024-09-16T20:51:46.022 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.022 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.022 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:46.022 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481423+0000", 2024-09-16T20:51:46.022 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8279999999999999e-06 2024-09-16T20:51:46.022 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.022 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.022 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:46.022 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.481426+0000", 2024-09-16T20:51:46.022 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0110000000000001e-06 2024-09-16T20:51:46.022 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.022 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.022 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:46.022 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.532631+0000", 2024-09-16T20:51:46.023 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051204514999999999 2024-09-16T20:51:46.023 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.023 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.023 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:46.023 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.533381+0000", 2024-09-16T20:51:46.023 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00075029000000000003 2024-09-16T20:51:46.023 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.023 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.023 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:46.023 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.533739+0000", 2024-09-16T20:51:46.023 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00035773100000000002 2024-09-16T20:51:46.023 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.023 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.023 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:46.023 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.535123+0000", 2024-09-16T20:51:46.024 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0013836160000000001 2024-09-16T20:51:46.024 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.024 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.024 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:46.024 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538413+0000", 2024-09-16T20:51:46.024 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0032902510000000001 2024-09-16T20:51:46.024 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.024 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.024 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:46.024 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538470+0000", 2024-09-16T20:51:46.024 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6697999999999998e-05 2024-09-16T20:51:46.024 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.024 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.025 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:46.025 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:49.538510+0000", 2024-09-16T20:51:46.025 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9988999999999999e-05 2024-09-16T20:51:46.025 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.025 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:46.025 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.025 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.025 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.025 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5089.0:12732 3.d 3:b752444a:::rbd_header.110e747f0779:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-16T20:51:46.025 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:51:46.025 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 161.19815833000001, 2024-09-16T20:51:46.025 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.065302467000000003, 2024-09-16T20:51:46.025 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:46.026 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:46.026 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:46.026 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:46.026 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5089", 2024-09-16T20:51:46.026 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:51:46.026 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 12732 2024-09-16T20:51:46.026 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.026 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:46.026 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.026 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:46.026 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:51:46.026 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.026 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.026 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.027 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:46.027 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768488+0000", 2024-09-16T20:51:46.027 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.027 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.027 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.027 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:46.027 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768490+0000", 2024-09-16T20:51:46.027 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2859999999999998e-06 2024-09-16T20:51:46.027 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.027 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.027 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:46.027 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768501+0000", 2024-09-16T20:51:46.027 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1226999999999999e-05 2024-09-16T20:51:46.028 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.028 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.028 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:46.028 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768517+0000", 2024-09-16T20:51:46.028 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.592e-05 2024-09-16T20:51:46.028 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.028 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.028 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:46.028 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768528+0000", 2024-09-16T20:51:46.028 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0689999999999999e-05 2024-09-16T20:51:46.028 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.028 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.028 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:46.029 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768584+0000", 2024-09-16T20:51:46.029 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5889e-05 2024-09-16T20:51:46.029 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.029 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.029 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:46.029 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.768720+0000", 2024-09-16T20:51:46.029 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000136718 2024-09-16T20:51:46.029 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.029 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.029 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:46.029 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.769066+0000", 2024-09-16T20:51:46.029 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000346013 2024-09-16T20:51:46.029 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.030 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.030 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op_commit", 2024-09-16T20:51:46.030 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833659+0000", 2024-09-16T20:51:46.030 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.064592822999999994 2024-09-16T20:51:46.030 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.030 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.030 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:46.030 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833719+0000", 2024-09-16T20:51:46.030 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9441000000000002e-05 2024-09-16T20:51:46.030 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.030 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.030 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:46.030 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833758+0000", 2024-09-16T20:51:46.030 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9122000000000003e-05 2024-09-16T20:51:46.031 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.031 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.031 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:46.031 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833790+0000", 2024-09-16T20:51:46.031 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2338000000000001e-05 2024-09-16T20:51:46.031 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.031 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:46.031 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.031 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.031 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.031 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop_reply(client.5089.0:12732 3.d e73/23 ondisk, result = 0)", 2024-09-16T20:51:46.031 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:51:46.031 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 161.19655616, 2024-09-16T20:51:46.032 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.063706456999999994, 2024-09-16T20:51:46.032 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:46.032 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:46.032 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:46.032 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:46.032 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.032 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:46.032 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:51:46.032 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.032 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.032 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.032 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:46.032 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770090+0000", 2024-09-16T20:51:46.032 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.033 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.033 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.033 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:46.033 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770091+0000", 2024-09-16T20:51:46.033 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.53e-06 2024-09-16T20:51:46.033 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.033 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.033 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:46.033 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.770104+0000", 2024-09-16T20:51:46.033 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2203e-05 2024-09-16T20:51:46.033 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.033 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.033 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:46.034 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833637+0000", 2024-09-16T20:51:46.034 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.063533201999999997 2024-09-16T20:51:46.034 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.034 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.034 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:46.034 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833655+0000", 2024-09-16T20:51:46.034 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8369e-05 2024-09-16T20:51:46.034 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.034 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.034 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:46.034 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833707+0000", 2024-09-16T20:51:46.034 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2061999999999997e-05 2024-09-16T20:51:46.034 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.035 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.035 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:46.035 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833715+0000", 2024-09-16T20:51:46.035 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5179999999999998e-06 2024-09-16T20:51:46.035 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.035 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.035 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:46.035 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:04.833796+0000", 2024-09-16T20:51:46.035 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1573e-05 2024-09-16T20:51:46.035 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.035 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:46.035 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.035 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.035 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.036 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3286 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 245760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:46.036 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:51:46.036 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 115.925494223, 2024-09-16T20:51:46.036 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052523564000000002, 2024-09-16T20:51:46.036 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:46.036 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:46.036 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:46.036 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:46.036 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:51:46.036 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:46.036 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3286 2024-09-16T20:51:46.036 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.036 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:46.037 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.037 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:46.037 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:51:46.037 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.037 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.037 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.037 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:46.037 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:51:46.037 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.037 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.037 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.037 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:46.037 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041152+0000", 2024-09-16T20:51:46.037 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3900000000000001e-07 2024-09-16T20:51:46.038 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.038 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.038 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:46.038 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041153+0000", 2024-09-16T20:51:46.038 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.119e-06 2024-09-16T20:51:46.038 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.038 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.038 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:46.038 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041156+0000", 2024-09-16T20:51:46.039 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0000000000000001e-06 2024-09-16T20:51:46.039 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.039 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.039 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:46.039 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041159+0000", 2024-09-16T20:51:46.039 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5859999999999999e-06 2024-09-16T20:51:46.039 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.039 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.039 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:46.039 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093529+0000", 2024-09-16T20:51:46.039 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052370591000000001 2024-09-16T20:51:46.039 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.039 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.040 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:46.040 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093570+0000", 2024-09-16T20:51:46.040 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0161999999999999e-05 2024-09-16T20:51:46.040 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.040 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.040 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:46.040 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093675+0000", 2024-09-16T20:51:46.040 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010586699999999999 2024-09-16T20:51:46.040 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.040 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:46.040 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.040 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.040 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.041 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3287 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 250880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:46.041 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:51:46.041 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 115.925484174, 2024-09-16T20:51:46.041 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052781203999999998, 2024-09-16T20:51:46.041 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:46.041 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:46.041 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:46.041 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:46.041 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:51:46.041 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:46.041 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3287 2024-09-16T20:51:46.041 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.041 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:46.042 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.042 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:46.042 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:51:46.042 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.042 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.042 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.042 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:46.042 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:51:46.042 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.042 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.042 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.042 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:46.042 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041162+0000", 2024-09-16T20:51:46.043 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1e-07 2024-09-16T20:51:46.043 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.043 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.043 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:46.043 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041163+0000", 2024-09-16T20:51:46.043 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0690000000000001e-06 2024-09-16T20:51:46.043 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.043 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.043 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:46.043 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041166+0000", 2024-09-16T20:51:46.043 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.982e-06 2024-09-16T20:51:46.043 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.043 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.043 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:46.044 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041169+0000", 2024-09-16T20:51:46.044 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.593e-06 2024-09-16T20:51:46.044 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.044 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.044 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:46.044 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093709+0000", 2024-09-16T20:51:46.044 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052540492000000001 2024-09-16T20:51:46.044 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.044 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.044 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:46.044 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093792+0000", 2024-09-16T20:51:46.044 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2436999999999997e-05 2024-09-16T20:51:46.044 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.045 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.045 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:46.045 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093943+0000", 2024-09-16T20:51:46.045 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000151421 2024-09-16T20:51:46.045 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.045 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:46.045 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.045 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.045 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.045 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3288 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 256000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:46.045 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:51:46.045 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 115.92547430099999, 2024-09-16T20:51:46.045 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052928269, 2024-09-16T20:51:46.045 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:46.046 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:46.046 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:46.046 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:46.046 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:51:46.046 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:46.046 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3288 2024-09-16T20:51:46.046 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.046 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:46.046 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.046 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:46.046 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:51:46.046 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.046 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.046 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.047 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:46.047 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:51:46.047 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.047 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.047 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.047 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:46.047 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041172+0000", 2024-09-16T20:51:46.047 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2700000000000001e-07 2024-09-16T20:51:46.047 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.047 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.047 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:46.047 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041173+0000", 2024-09-16T20:51:46.047 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.113e-06 2024-09-16T20:51:46.048 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.048 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.048 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:46.048 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041176+0000", 2024-09-16T20:51:46.048 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9799999999999998e-06 2024-09-16T20:51:46.048 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.048 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.048 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:46.048 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041179+0000", 2024-09-16T20:51:46.048 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.667e-06 2024-09-16T20:51:46.048 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.048 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.048 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:46.048 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093957+0000", 2024-09-16T20:51:46.049 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052778703000000003 2024-09-16T20:51:46.049 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.049 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.049 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:46.049 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.093999+0000", 2024-09-16T20:51:46.049 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1300000000000001e-05 2024-09-16T20:51:46.049 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.049 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.049 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:46.049 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094100+0000", 2024-09-16T20:51:46.049 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101279 2024-09-16T20:51:46.049 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.049 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:46.050 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.050 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.050 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.050 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5799.0:3289 3.10 3:0c6e800a:::rbd_data.110ea93691bb.0000000000000007:23 [read 261120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:46.050 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:51:46.050 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 115.92546430500001, 2024-09-16T20:51:46.050 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053182976, 2024-09-16T20:51:46.050 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:46.050 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:46.050 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:46.050 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:46.050 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5799", 2024-09-16T20:51:46.050 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:46.050 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3289 2024-09-16T20:51:46.051 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.051 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:46.051 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.051 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:46.051 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:51:46.051 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.051 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.051 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.051 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:46.051 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:51:46.051 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.051 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.051 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.052 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:46.052 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041182+0000", 2024-09-16T20:51:46.052 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2999999999999999e-07 2024-09-16T20:51:46.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.052 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.052 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:46.052 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041183+0000", 2024-09-16T20:51:46.052 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0720000000000001e-06 2024-09-16T20:51:46.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.052 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.052 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:46.052 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041186+0000", 2024-09-16T20:51:46.052 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9869999999999999e-06 2024-09-16T20:51:46.053 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.053 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.053 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:46.053 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.041189+0000", 2024-09-16T20:51:46.053 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.587e-06 2024-09-16T20:51:46.053 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.053 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.053 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:46.053 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094136+0000", 2024-09-16T20:51:46.053 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.052947270999999997 2024-09-16T20:51:46.053 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.053 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.053 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:46.053 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094213+0000", 2024-09-16T20:51:46.054 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7552000000000006e-05 2024-09-16T20:51:46.054 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.054 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.054 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:46.054 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:50.094365+0000", 2024-09-16T20:51:46.054 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00015127700000000001 2024-09-16T20:51:46.054 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.054 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:46.054 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.054 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.054 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.054 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6231.0:2196 3.d 3:b1fe01a0:::rbd_data.110e7ef3004.0000000000000000:2d [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e136)", 2024-09-16T20:51:46.054 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:22.739057+0000", 2024-09-16T20:51:46.055 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 83.227589506000001, 2024-09-16T20:51:46.055 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11999464899999999, 2024-09-16T20:51:46.055 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:46.055 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:46.055 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:46.055 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:46.055 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6231", 2024-09-16T20:51:46.055 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:46.055 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 2196 2024-09-16T20:51:46.055 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.055 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:46.055 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.055 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:46.056 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.739057+0000", 2024-09-16T20:51:46.056 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.056 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.056 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.056 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:46.056 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.739057+0000", 2024-09-16T20:51:46.056 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.056 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.056 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.056 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:46.056 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.739057+0000", 2024-09-16T20:51:46.056 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1399999999999997e-07 2024-09-16T20:51:46.056 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.056 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.057 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:46.057 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.739060+0000", 2024-09-16T20:51:46.057 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6019999999999998e-06 2024-09-16T20:51:46.057 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.057 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.057 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:46.057 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.858755+0000", 2024-09-16T20:51:46.057 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.119695151 2024-09-16T20:51:46.057 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.057 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.057 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:46.057 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.858778+0000", 2024-09-16T20:51:46.057 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3184999999999999e-05 2024-09-16T20:51:46.058 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.058 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.058 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:46.058 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.858865+0000", 2024-09-16T20:51:46.058 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.7330000000000006e-05 2024-09-16T20:51:46.058 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.058 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.058 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:46.058 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.858952+0000", 2024-09-16T20:51:46.058 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6803000000000002e-05 2024-09-16T20:51:46.058 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.058 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.058 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:46.058 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:22.859051+0000", 2024-09-16T20:51:46.059 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8963999999999996e-05 2024-09-16T20:51:46.059 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.059 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:46.059 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.059 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.059 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.059 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6061.0:5401 3.13 3:c8554348:::rbd_data.110e16eafbd.0000000000000011:head [read 577536~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-16T20:51:46.059 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:27.761536+0000", 2024-09-16T20:51:46.059 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 78.205109692999997, 2024-09-16T20:51:46.059 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.003742712, 2024-09-16T20:51:46.059 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:46.059 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:46.059 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:51:46.059 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:46.060 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6061", 2024-09-16T20:51:46.060 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3930128440", 2024-09-16T20:51:46.060 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5401 2024-09-16T20:51:46.060 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.060 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:46.060 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.060 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:46.060 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761536+0000", 2024-09-16T20:51:46.060 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.060 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.060 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.060 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:46.060 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761536+0000", 2024-09-16T20:51:46.060 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.061 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.061 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.061 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:46.061 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761537+0000", 2024-09-16T20:51:46.061 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2900000000000003e-07 2024-09-16T20:51:46.061 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.061 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.061 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:46.061 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761540+0000", 2024-09-16T20:51:46.061 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2320000000000001e-06 2024-09-16T20:51:46.061 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.061 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.062 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:46.062 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761548+0000", 2024-09-16T20:51:46.062 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1289999999999996e-06 2024-09-16T20:51:46.062 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.062 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.062 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:46.062 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761554+0000", 2024-09-16T20:51:46.062 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0820000000000004e-06 2024-09-16T20:51:46.062 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.062 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.062 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:46.062 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761621+0000", 2024-09-16T20:51:46.062 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6266000000000006e-05 2024-09-16T20:51:46.063 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.063 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.063 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:46.063 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.761626+0000", 2024-09-16T20:51:46.063 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.614e-06 2024-09-16T20:51:46.063 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.063 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.063 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:46.063 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:28.765163+0000", 2024-09-16T20:51:46.063 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.003536416 2024-09-16T20:51:46.063 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.063 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.063 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:46.063 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:28.765279+0000", 2024-09-16T20:51:46.064 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000116344 2024-09-16T20:51:46.064 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.064 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:46.064 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.064 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.064 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.064 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.6061.0:5425 3.13 3:c8554348:::rbd_data.110e16eafbd.0000000000000011:head [read 602112~28672] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-16T20:51:46.064 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:27.767049+0000", 2024-09-16T20:51:46.064 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 78.199597034000007, 2024-09-16T20:51:46.064 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.99847897900000004, 2024-09-16T20:51:46.064 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:46.064 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:46.065 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-16T20:51:46.065 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:46.065 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.6061", 2024-09-16T20:51:46.065 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3930128440", 2024-09-16T20:51:46.065 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5425 2024-09-16T20:51:46.065 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.065 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:46.065 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.065 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:46.065 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767049+0000", 2024-09-16T20:51:46.065 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.065 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.065 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.065 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:46.066 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767049+0000", 2024-09-16T20:51:46.066 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.066 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.066 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.066 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:46.066 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767050+0000", 2024-09-16T20:51:46.066 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6499999999999999e-07 2024-09-16T20:51:46.066 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.066 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.066 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:46.066 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767052+0000", 2024-09-16T20:51:46.066 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7350000000000001e-06 2024-09-16T20:51:46.066 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.067 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.067 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:46.067 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767060+0000", 2024-09-16T20:51:46.067 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.5109999999999997e-06 2024-09-16T20:51:46.067 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.067 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.067 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:46.067 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767066+0000", 2024-09-16T20:51:46.067 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8139999999999997e-06 2024-09-16T20:51:46.067 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.067 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.067 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:46.068 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767128+0000", 2024-09-16T20:51:46.068 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2174999999999993e-05 2024-09-16T20:51:46.068 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.068 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.068 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:46.068 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:27.767133+0000", 2024-09-16T20:51:46.068 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2140000000000004e-06 2024-09-16T20:51:46.068 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.068 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.068 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:46.068 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:28.765441+0000", 2024-09-16T20:51:46.068 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.99830808599999998 2024-09-16T20:51:46.068 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.068 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.069 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:46.069 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:28.765528+0000", 2024-09-16T20:51:46.069 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6879000000000006e-05 2024-09-16T20:51:46.069 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.069 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:46.069 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.069 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.069 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.069 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:25585 3.10 3:0c7c5b23:::rbd_header.110ebf0678ad:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e145)", 2024-09-16T20:51:46.069 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:32.877132+0000", 2024-09-16T20:51:46.069 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 73.089514484999995, 2024-09-16T20:51:46.069 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.054096699999999998, 2024-09-16T20:51:46.069 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:46.070 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:46.070 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:46.070 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:46.070 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:51:46.070 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:46.070 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 25585 2024-09-16T20:51:46.070 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.070 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:46.070 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.070 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:46.070 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.877132+0000", 2024-09-16T20:51:46.070 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.071 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.071 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.071 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:46.071 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.877132+0000", 2024-09-16T20:51:46.071 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.071 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.071 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.071 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:46.071 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931072+0000", 2024-09-16T20:51:46.071 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.053939931000000003 2024-09-16T20:51:46.071 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.071 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.071 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:46.072 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931076+0000", 2024-09-16T20:51:46.072 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2860000000000001e-06 2024-09-16T20:51:46.072 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.072 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.072 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:46.072 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931085+0000", 2024-09-16T20:51:46.072 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.3370000000000006e-06 2024-09-16T20:51:46.072 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.072 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.072 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:46.072 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931092+0000", 2024-09-16T20:51:46.072 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9970000000000001e-06 2024-09-16T20:51:46.072 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.073 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.073 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:46.073 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931137+0000", 2024-09-16T20:51:46.073 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5160000000000001e-05 2024-09-16T20:51:46.073 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.073 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.073 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:46.073 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931178+0000", 2024-09-16T20:51:46.073 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0379000000000001e-05 2024-09-16T20:51:46.073 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.073 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.073 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:46.073 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:32.931228+0000", 2024-09-16T20:51:46.074 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0609999999999998e-05 2024-09-16T20:51:46.074 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.074 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:46.074 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.074 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.074 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.074 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:26679 3.c 3:33e2f78f:::rbd_header.110e16eafbd:head [watch ping cookie 140213436013408 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e153)", 2024-09-16T20:51:46.074 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:41.440287+0000", 2024-09-16T20:51:46.074 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 64.526358610000003, 2024-09-16T20:51:46.074 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.13295633600000001, 2024-09-16T20:51:46.074 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:46.074 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:46.074 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:46.075 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:46.075 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:51:46.075 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:46.075 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 26679 2024-09-16T20:51:46.075 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.075 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:46.075 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.075 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:46.075 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440287+0000", 2024-09-16T20:51:46.075 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.075 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.075 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.075 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:46.076 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440287+0000", 2024-09-16T20:51:46.076 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.076 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.076 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.076 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:46.076 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440288+0000", 2024-09-16T20:51:46.076 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0900000000000001e-07 2024-09-16T20:51:46.076 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.076 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.076 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:46.076 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440292+0000", 2024-09-16T20:51:46.076 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3749999999999996e-06 2024-09-16T20:51:46.076 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.076 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.076 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:46.077 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440299+0000", 2024-09-16T20:51:46.077 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8249999999999999e-06 2024-09-16T20:51:46.077 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.077 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.077 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:46.077 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440307+0000", 2024-09-16T20:51:46.077 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4800000000000004e-06 2024-09-16T20:51:46.077 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.077 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.077 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:46.077 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440451+0000", 2024-09-16T20:51:46.077 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014414500000000001 2024-09-16T20:51:46.077 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.077 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.078 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for peered", 2024-09-16T20:51:46.078 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.440465+0000", 2024-09-16T20:51:46.078 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.402e-05 2024-09-16T20:51:46.078 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.078 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.078 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:46.078 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.573043+0000", 2024-09-16T20:51:46.078 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.13257855399999999 2024-09-16T20:51:46.078 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.078 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.078 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:46.078 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.573130+0000", 2024-09-16T20:51:46.078 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6168999999999994e-05 2024-09-16T20:51:46.078 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.078 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.079 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:46.079 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:41.573244+0000", 2024-09-16T20:51:46.079 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000114159 2024-09-16T20:51:46.079 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.079 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:46.079 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.079 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.079 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:46.079 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:46.080 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T20:51:46.187 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:46.188 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:51:46.188 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-16T20:51:46.188 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:46.196 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T20:51:46.303 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:46.303 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-16T20:51:46.303 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-16T20:51:46.303 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-16T20:51:46.303 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:46.312 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T20:51:46.420 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:51:46.420 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-16T20:51:46.420 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-16T20:51:46.420 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-16T20:51:46.420 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.421 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3714 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1480704~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:46.421 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:51:46.421 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 203.77981057299999, 2024-09-16T20:51:46.421 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050152552000000003, 2024-09-16T20:51:46.421 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:46.421 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:46.421 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:46.421 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:46.421 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:46.421 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:46.421 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3714 2024-09-16T20:51:46.421 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.421 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:46.422 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.422 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:46.422 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:51:46.422 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.422 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.422 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.422 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:46.422 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:51:46.422 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.422 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.422 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.422 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:46.422 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639006+0000", 2024-09-16T20:51:46.422 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0499999999999999e-07 2024-09-16T20:51:46.423 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.423 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.423 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:46.423 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639007+0000", 2024-09-16T20:51:46.423 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5340000000000001e-06 2024-09-16T20:51:46.423 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.423 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.423 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:46.423 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639013+0000", 2024-09-16T20:51:46.423 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3260000000000003e-06 2024-09-16T20:51:46.423 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.423 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.423 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:46.423 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639018+0000", 2024-09-16T20:51:46.424 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2850000000000002e-06 2024-09-16T20:51:46.424 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.424 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.424 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:46.424 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689026+0000", 2024-09-16T20:51:46.424 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.05000798 2024-09-16T20:51:46.424 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.424 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.424 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:46.424 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689077+0000", 2024-09-16T20:51:46.424 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1063999999999998e-05 2024-09-16T20:51:46.424 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.424 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.425 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:46.425 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689158+0000", 2024-09-16T20:51:46.425 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1057999999999995e-05 2024-09-16T20:51:46.425 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.425 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:46.425 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.425 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.425 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.425 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3716 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1488896~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:46.425 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:51:46.425 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 203.77979276299999, 2024-09-16T20:51:46.425 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050280142, 2024-09-16T20:51:46.425 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:46.425 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:46.426 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:46.426 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:46.426 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:46.426 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:46.426 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3716 2024-09-16T20:51:46.426 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.426 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:46.426 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.426 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:46.426 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:51:46.426 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.426 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.426 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.426 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:46.427 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639023+0000", 2024-09-16T20:51:46.427 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.427 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.427 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.427 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:46.427 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639024+0000", 2024-09-16T20:51:46.427 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9999999999999997e-07 2024-09-16T20:51:46.427 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.427 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.427 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:46.427 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639026+0000", 2024-09-16T20:51:46.427 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.781e-06 2024-09-16T20:51:46.427 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.427 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.428 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:46.428 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639032+0000", 2024-09-16T20:51:46.428 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7780000000000004e-06 2024-09-16T20:51:46.428 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.428 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.428 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:46.428 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639036+0000", 2024-09-16T20:51:46.428 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6909999999999996e-06 2024-09-16T20:51:46.428 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.428 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.428 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:46.428 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689191+0000", 2024-09-16T20:51:46.428 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050155284000000001 2024-09-16T20:51:46.429 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.429 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.429 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:46.429 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689242+0000", 2024-09-16T20:51:46.429 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0662999999999999e-05 2024-09-16T20:51:46.429 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.429 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.429 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:46.429 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689304+0000", 2024-09-16T20:51:46.429 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1345000000000003e-05 2024-09-16T20:51:46.429 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.429 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:46.430 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.430 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.430 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.430 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3719 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1497088~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:46.430 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:51:46.430 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 203.77976548500001, 2024-09-16T20:51:46.430 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050399715999999997, 2024-09-16T20:51:46.430 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:46.430 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:46.430 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:46.430 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:46.430 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:46.430 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:46.430 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3719 2024-09-16T20:51:46.431 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.431 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:46.431 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.431 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:46.431 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:51:46.431 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.431 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.431 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.431 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:46.431 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:51:46.431 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.431 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.431 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.432 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:46.432 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639051+0000", 2024-09-16T20:51:46.432 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9700000000000002e-07 2024-09-16T20:51:46.432 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.432 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.432 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:46.432 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639053+0000", 2024-09-16T20:51:46.432 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2900000000000001e-06 2024-09-16T20:51:46.432 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.432 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.432 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:46.432 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639060+0000", 2024-09-16T20:51:46.432 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7750000000000004e-06 2024-09-16T20:51:46.433 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.433 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.433 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:46.433 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639066+0000", 2024-09-16T20:51:46.433 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0639999999999999e-06 2024-09-16T20:51:46.433 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.433 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.433 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:46.433 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689337+0000", 2024-09-16T20:51:46.433 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050270784999999998 2024-09-16T20:51:46.433 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.433 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.433 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:46.434 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689392+0000", 2024-09-16T20:51:46.434 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4920999999999998e-05 2024-09-16T20:51:46.434 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.434 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.434 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:46.434 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689450+0000", 2024-09-16T20:51:46.434 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8483999999999998e-05 2024-09-16T20:51:46.434 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.434 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:46.434 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.434 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.434 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.434 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3721 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1505280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:46.435 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:51:46.435 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 203.77974235299999, 2024-09-16T20:51:46.435 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050513608000000002, 2024-09-16T20:51:46.435 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:46.435 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:46.435 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:46.435 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:46.435 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:46.435 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:46.435 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3721 2024-09-16T20:51:46.435 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.435 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:46.435 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.436 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:46.436 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:51:46.436 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.436 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.436 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.436 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:46.436 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:51:46.436 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.436 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.436 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.436 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:46.436 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639074+0000", 2024-09-16T20:51:46.436 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7e-07 2024-09-16T20:51:46.436 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.437 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.437 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:46.437 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639076+0000", 2024-09-16T20:51:46.437 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8899999999999999e-06 2024-09-16T20:51:46.437 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.437 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.437 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:46.437 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639083+0000", 2024-09-16T20:51:46.437 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1210000000000001e-06 2024-09-16T20:51:46.437 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.437 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.437 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:46.437 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639089+0000", 2024-09-16T20:51:46.438 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.643e-06 2024-09-16T20:51:46.438 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.438 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.438 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:46.438 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689474+0000", 2024-09-16T20:51:46.438 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050385679000000003 2024-09-16T20:51:46.438 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.438 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.438 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:46.438 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689533+0000", 2024-09-16T20:51:46.438 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8303000000000001e-05 2024-09-16T20:51:46.438 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.438 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.439 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:46.439 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689587+0000", 2024-09-16T20:51:46.439 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4601999999999997e-05 2024-09-16T20:51:46.439 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.439 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:46.439 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.439 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.439 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.439 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3723 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1513472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:46.439 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:51:46.440 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 203.77971154900001, 2024-09-16T20:51:46.440 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050699691999999998, 2024-09-16T20:51:46.440 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:46.440 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:46.440 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:46.440 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:46.440 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:46.440 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:46.440 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3723 2024-09-16T20:51:46.440 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.440 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:46.440 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.441 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:46.441 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:51:46.441 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.441 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.441 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.441 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:46.441 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:51:46.441 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.441 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.441 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.441 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:46.441 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639105+0000", 2024-09-16T20:51:46.441 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3399999999999999e-07 2024-09-16T20:51:46.441 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.442 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.442 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:46.442 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639107+0000", 2024-09-16T20:51:46.442 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3860000000000001e-06 2024-09-16T20:51:46.442 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.442 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.442 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:46.442 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639114+0000", 2024-09-16T20:51:46.442 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9800000000000001e-06 2024-09-16T20:51:46.442 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.442 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.442 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:46.443 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639129+0000", 2024-09-16T20:51:46.443 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4887e-05 2024-09-16T20:51:46.443 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.443 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.443 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:46.443 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689625+0000", 2024-09-16T20:51:46.443 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050495848000000003 2024-09-16T20:51:46.443 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.443 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.443 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:46.443 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689720+0000", 2024-09-16T20:51:46.443 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4821000000000006e-05 2024-09-16T20:51:46.443 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.444 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.444 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:46.444 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689804+0000", 2024-09-16T20:51:46.444 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4235999999999998e-05 2024-09-16T20:51:46.444 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.444 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:46.444 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.444 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.444 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.444 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3725 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1521664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:46.444 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:51:46.444 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 203.779677953, 2024-09-16T20:51:46.444 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050797660000000001, 2024-09-16T20:51:46.445 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:46.445 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:46.445 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:46.445 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:46.445 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:46.445 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:46.445 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3725 2024-09-16T20:51:46.445 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.445 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:46.445 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.445 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:46.445 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:51:46.445 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.446 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.446 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.446 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:46.446 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639138+0000", 2024-09-16T20:51:46.446 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.446 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.446 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.446 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:46.446 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639139+0000", 2024-09-16T20:51:46.446 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8999999999999997e-07 2024-09-16T20:51:46.446 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.446 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.446 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:46.446 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639141+0000", 2024-09-16T20:51:46.447 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.182e-06 2024-09-16T20:51:46.447 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.447 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.447 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:46.447 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639154+0000", 2024-09-16T20:51:46.447 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3626e-05 2024-09-16T20:51:46.447 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.447 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.447 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:46.447 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639160+0000", 2024-09-16T20:51:46.447 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6810000000000003e-06 2024-09-16T20:51:46.447 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.447 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.447 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:46.448 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689831+0000", 2024-09-16T20:51:46.448 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050670907000000001 2024-09-16T20:51:46.448 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.448 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.448 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:46.448 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689882+0000", 2024-09-16T20:51:46.448 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0445999999999998e-05 2024-09-16T20:51:46.448 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.448 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.448 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:46.448 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689936+0000", 2024-09-16T20:51:46.448 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4327999999999999e-05 2024-09-16T20:51:46.448 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.449 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:46.449 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.449 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.449 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.449 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3727 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1529856~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:46.449 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:51:46.449 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 203.779642376, 2024-09-16T20:51:46.449 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051016922999999999, 2024-09-16T20:51:46.449 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:46.449 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:46.449 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:46.449 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:46.449 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:46.450 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:46.450 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3727 2024-09-16T20:51:46.450 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.450 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:46.450 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.450 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:46.450 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:51:46.450 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.450 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.450 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.450 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:46.450 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:51:46.450 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.451 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.451 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.451 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:46.451 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639174+0000", 2024-09-16T20:51:46.451 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9999999999999998e-07 2024-09-16T20:51:46.451 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.451 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.451 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:46.451 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639183+0000", 2024-09-16T20:51:46.451 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2399999999999996e-06 2024-09-16T20:51:46.451 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.451 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.451 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:46.451 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639195+0000", 2024-09-16T20:51:46.451 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1113e-05 2024-09-16T20:51:46.451 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.452 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.452 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:46.452 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639206+0000", 2024-09-16T20:51:46.452 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1503e-05 2024-09-16T20:51:46.452 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.452 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.452 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:46.452 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.689969+0000", 2024-09-16T20:51:46.452 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050762676 2024-09-16T20:51:46.452 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.452 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.452 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:46.452 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690071+0000", 2024-09-16T20:51:46.452 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000102163 2024-09-16T20:51:46.452 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.453 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.453 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:46.453 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690191+0000", 2024-09-16T20:51:46.453 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000119728 2024-09-16T20:51:46.453 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.453 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:46.453 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.453 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.453 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.453 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3730 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1538048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:46.453 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:51:46.453 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 203.779600813, 2024-09-16T20:51:46.453 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051131467999999999, 2024-09-16T20:51:46.453 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:46.454 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:46.454 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:46.454 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:46.454 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:46.454 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:46.454 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3730 2024-09-16T20:51:46.454 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.454 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:46.454 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.454 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:46.454 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:51:46.454 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.454 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.454 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.454 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:46.454 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639215+0000", 2024-09-16T20:51:46.455 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.455 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.455 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.455 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:46.455 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639216+0000", 2024-09-16T20:51:46.455 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.39e-07 2024-09-16T20:51:46.455 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.455 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.455 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:46.455 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639218+0000", 2024-09-16T20:51:46.455 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.537e-06 2024-09-16T20:51:46.455 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.455 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.456 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:46.456 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639229+0000", 2024-09-16T20:51:46.456 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1122e-05 2024-09-16T20:51:46.456 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.456 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.456 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:46.456 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639237+0000", 2024-09-16T20:51:46.456 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8509999999999999e-06 2024-09-16T20:51:46.456 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.456 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.456 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:46.456 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690227+0000", 2024-09-16T20:51:46.456 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.050990147 2024-09-16T20:51:46.456 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.456 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.457 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:46.457 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690278+0000", 2024-09-16T20:51:46.457 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0223999999999999e-05 2024-09-16T20:51:46.457 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.457 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.457 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:46.457 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690347+0000", 2024-09-16T20:51:46.457 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9147999999999996e-05 2024-09-16T20:51:46.457 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.457 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:46.457 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.457 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.457 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.457 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3731 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1546240~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:46.457 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:51:46.458 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 203.779570105, 2024-09-16T20:51:46.458 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051276744999999999, 2024-09-16T20:51:46.458 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:46.458 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:46.458 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:46.458 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:46.458 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:46.458 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:46.458 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3731 2024-09-16T20:51:46.458 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.458 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:46.458 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.458 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:46.458 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:51:46.459 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.459 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.459 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.459 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:46.459 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:51:46.459 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.459 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.459 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.459 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:46.459 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639246+0000", 2024-09-16T20:51:46.459 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9900000000000001e-07 2024-09-16T20:51:46.459 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.459 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.459 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:46.460 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639249+0000", 2024-09-16T20:51:46.460 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3630000000000001e-06 2024-09-16T20:51:46.460 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.460 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.460 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:46.460 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639255+0000", 2024-09-16T20:51:46.460 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4960000000000001e-06 2024-09-16T20:51:46.460 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.460 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.460 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:46.460 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639261+0000", 2024-09-16T20:51:46.460 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2839999999999997e-06 2024-09-16T20:51:46.460 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.460 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.460 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:46.461 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690375+0000", 2024-09-16T20:51:46.461 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051114684000000001 2024-09-16T20:51:46.461 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.461 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.461 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:46.461 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690425+0000", 2024-09-16T20:51:46.461 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9379999999999998e-05 2024-09-16T20:51:46.461 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.461 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.461 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:46.461 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690523+0000", 2024-09-16T20:51:46.461 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8139000000000003e-05 2024-09-16T20:51:46.461 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.461 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:46.461 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.461 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.462 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.462 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3732 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1554432~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:46.462 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:51:46.462 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 203.779524323, 2024-09-16T20:51:46.462 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051380096, 2024-09-16T20:51:46.462 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:46.462 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:46.462 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:46.462 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:46.462 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:46.462 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:46.462 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3732 2024-09-16T20:51:46.462 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.462 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:46.462 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.463 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:46.463 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:51:46.463 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.463 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.463 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.463 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:46.463 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:51:46.463 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.463 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.463 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.463 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:46.463 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639292+0000", 2024-09-16T20:51:46.463 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6100000000000001e-07 2024-09-16T20:51:46.463 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.463 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.464 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:46.464 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639295+0000", 2024-09-16T20:51:46.464 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.447e-06 2024-09-16T20:51:46.464 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.464 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.464 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:46.464 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639302+0000", 2024-09-16T20:51:46.464 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0570000000000004e-06 2024-09-16T20:51:46.464 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.464 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.464 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:46.464 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639307+0000", 2024-09-16T20:51:46.464 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1429999999999997e-06 2024-09-16T20:51:46.464 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.465 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.465 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:46.465 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690555+0000", 2024-09-16T20:51:46.465 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051247963000000001 2024-09-16T20:51:46.465 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.465 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.465 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:46.465 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690602+0000", 2024-09-16T20:51:46.465 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7602000000000003e-05 2024-09-16T20:51:46.465 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.465 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.465 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:46.465 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690672+0000", 2024-09-16T20:51:46.465 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9423000000000003e-05 2024-09-16T20:51:46.466 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.466 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:46.466 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.466 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.466 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.466 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3734 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1562624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:46.466 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:51:46.466 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 203.779502673, 2024-09-16T20:51:46.466 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051490610999999999, 2024-09-16T20:51:46.466 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:46.466 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:46.466 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:46.466 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:46.466 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:46.466 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:46.467 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3734 2024-09-16T20:51:46.467 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.467 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:46.467 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.467 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:46.467 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:51:46.467 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.467 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.467 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.467 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:46.467 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639313+0000", 2024-09-16T20:51:46.467 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.467 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.467 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.468 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:46.468 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639314+0000", 2024-09-16T20:51:46.468 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4400000000000001e-07 2024-09-16T20:51:46.468 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.468 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.468 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:46.468 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639316+0000", 2024-09-16T20:51:46.468 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1459999999999999e-06 2024-09-16T20:51:46.468 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.468 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.468 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:46.468 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639322+0000", 2024-09-16T20:51:46.468 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7610000000000004e-06 2024-09-16T20:51:46.469 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.469 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.469 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:46.469 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639327+0000", 2024-09-16T20:51:46.469 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.186e-06 2024-09-16T20:51:46.469 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.469 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.469 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:46.469 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690698+0000", 2024-09-16T20:51:46.469 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051371223000000001 2024-09-16T20:51:46.469 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.469 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.469 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:46.470 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690748+0000", 2024-09-16T20:51:46.470 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0192999999999999e-05 2024-09-16T20:51:46.470 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.470 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.470 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:46.470 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690804+0000", 2024-09-16T20:51:46.470 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5658000000000001e-05 2024-09-16T20:51:46.470 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.470 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:46.470 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.470 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.470 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.470 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:3736 3.e 3:778bba30:::rbd_data.110e3f481dff.0000000000000004:7 [read 1570816~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:46.471 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:51:46.471 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 203.779483414, 2024-09-16T20:51:46.471 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051596528000000003, 2024-09-16T20:51:46.471 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:46.471 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:46.471 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:46.471 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:46.471 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:46.471 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:46.471 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 3736 2024-09-16T20:51:46.471 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.471 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:46.471 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.472 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:46.472 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:51:46.472 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.472 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.472 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.472 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:46.472 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:51:46.472 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.472 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.472 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.472 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:46.472 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639333+0000", 2024-09-16T20:51:46.473 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9999999999999999e-07 2024-09-16T20:51:46.473 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.473 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.473 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:46.473 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639335+0000", 2024-09-16T20:51:46.473 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.635e-06 2024-09-16T20:51:46.473 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.473 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.473 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:46.473 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639340+0000", 2024-09-16T20:51:46.473 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.5840000000000002e-06 2024-09-16T20:51:46.473 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.473 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.473 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:46.474 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.639346+0000", 2024-09-16T20:51:46.474 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1190000000000002e-06 2024-09-16T20:51:46.474 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.474 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.474 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:46.474 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690817+0000", 2024-09-16T20:51:46.474 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.051470721999999997 2024-09-16T20:51:46.474 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.474 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.474 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:46.474 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690875+0000", 2024-09-16T20:51:46.474 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7441000000000001e-05 2024-09-16T20:51:46.474 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.474 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.474 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:46.475 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.690929+0000", 2024-09-16T20:51:46.475 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4727e-05 2024-09-16T20:51:46.475 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.475 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:46.475 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.475 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.475 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.475 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5434 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:46.475 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:51:46.475 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 203.554221848, 2024-09-16T20:51:46.475 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099755831000000003, 2024-09-16T20:51:46.475 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:46.475 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:46.475 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:46.475 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:46.476 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:46.476 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:46.476 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5434 2024-09-16T20:51:46.476 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.476 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:46.476 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.476 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:46.476 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:51:46.476 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.476 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.476 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.476 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:46.476 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864594+0000", 2024-09-16T20:51:46.476 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.476 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.477 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.477 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:46.477 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864595+0000", 2024-09-16T20:51:46.477 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1099999999999995e-07 2024-09-16T20:51:46.477 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.477 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.477 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:46.477 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864597+0000", 2024-09-16T20:51:46.477 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3989999999999998e-06 2024-09-16T20:51:46.477 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.477 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.477 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:46.477 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864603+0000", 2024-09-16T20:51:46.477 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9309999999999996e-06 2024-09-16T20:51:46.477 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.478 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.478 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:46.478 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864607+0000", 2024-09-16T20:51:46.478 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6200000000000001e-06 2024-09-16T20:51:46.478 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.478 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.478 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:46.478 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964164+0000", 2024-09-16T20:51:46.478 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099557162000000005 2024-09-16T20:51:46.478 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.478 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.478 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:46.478 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964244+0000", 2024-09-16T20:51:46.478 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9597000000000004e-05 2024-09-16T20:51:46.479 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.479 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.479 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:46.479 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.964350+0000", 2024-09-16T20:51:46.479 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010641099999999999 2024-09-16T20:51:46.479 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.479 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:46.479 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.479 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.479 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.479 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4600.0:5435 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:head [read 20480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T20:51:46.479 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:51:46.479 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 203.55419202600001, 2024-09-16T20:51:46.480 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100897102, 2024-09-16T20:51:46.480 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:46.480 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:46.480 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:46.480 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:46.480 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4600", 2024-09-16T20:51:46.480 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/185864356", 2024-09-16T20:51:46.480 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 5435 2024-09-16T20:51:46.480 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.480 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:46.480 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.480 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:46.480 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:51:46.480 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.480 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.481 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.481 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:46.481 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:51:46.481 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.481 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.481 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.481 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:46.481 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864624+0000", 2024-09-16T20:51:46.481 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8000000000000001e-07 2024-09-16T20:51:46.481 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.481 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.481 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:46.481 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864626+0000", 2024-09-16T20:51:46.481 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.745e-06 2024-09-16T20:51:46.481 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.482 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.482 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:46.482 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864633+0000", 2024-09-16T20:51:46.482 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3339999999999998e-06 2024-09-16T20:51:46.482 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.482 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.482 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:46.482 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.864636+0000", 2024-09-16T20:51:46.482 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1200000000000002e-06 2024-09-16T20:51:46.482 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.482 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.482 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:46.482 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965299+0000", 2024-09-16T20:51:46.482 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10066352000000001 2024-09-16T20:51:46.483 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.483 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.483 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:46.483 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965390+0000", 2024-09-16T20:51:46.483 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0894000000000001e-05 2024-09-16T20:51:46.483 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.483 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.483 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:46.483 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:22.965521+0000", 2024-09-16T20:51:46.483 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013110899999999999 2024-09-16T20:51:46.483 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.483 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:46.483 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.483 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.483 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.484 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:152 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 327680~32768 out=32768b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:51:46.484 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:51:46.484 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 202.255695084, 2024-09-16T20:51:46.484 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11869405800000001, 2024-09-16T20:51:46.484 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:46.484 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:46.484 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:46.484 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:46.484 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:51:46.484 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:51:46.484 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 152 2024-09-16T20:51:46.484 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.484 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:46.484 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.485 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:46.485 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:51:46.485 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.485 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.485 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.485 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:46.485 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163121+0000", 2024-09-16T20:51:46.485 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.485 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.485 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.485 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:46.485 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163122+0000", 2024-09-16T20:51:46.485 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6400000000000003e-07 2024-09-16T20:51:46.485 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.486 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.486 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:46.486 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163124+0000", 2024-09-16T20:51:46.486 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.565e-06 2024-09-16T20:51:46.486 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.486 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.486 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:46.486 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163136+0000", 2024-09-16T20:51:46.486 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1717e-05 2024-09-16T20:51:46.486 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.486 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.486 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:46.486 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163142+0000", 2024-09-16T20:51:46.486 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1750000000000002e-06 2024-09-16T20:51:46.487 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.487 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.487 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:46.487 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163606+0000", 2024-09-16T20:51:46.487 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00046418799999999998 2024-09-16T20:51:46.487 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.487 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.487 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:46.487 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163656+0000", 2024-09-16T20:51:46.487 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9852999999999999e-05 2024-09-16T20:51:46.487 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.487 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.487 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:46.487 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281815+0000", 2024-09-16T20:51:46.488 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11815909600000001 2024-09-16T20:51:46.488 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.488 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:46.488 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.488 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.488 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.488 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:153 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:51:46.488 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:51:46.488 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 202.25566213100001, 2024-09-16T20:51:46.488 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11903462400000001, 2024-09-16T20:51:46.488 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:46.488 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:46.488 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:46.489 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:46.489 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:51:46.489 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:51:46.489 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 153 2024-09-16T20:51:46.489 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.489 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:46.489 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.489 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:46.489 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:51:46.489 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.489 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.489 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.489 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:46.489 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163154+0000", 2024-09-16T20:51:46.490 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.490 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.490 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.490 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:46.490 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163155+0000", 2024-09-16T20:51:46.490 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.37e-07 2024-09-16T20:51:46.490 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.490 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.490 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:46.490 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163157+0000", 2024-09-16T20:51:46.490 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6010000000000002e-06 2024-09-16T20:51:46.490 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.490 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.490 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:46.491 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163165+0000", 2024-09-16T20:51:46.491 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2619999999999998e-06 2024-09-16T20:51:46.491 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.491 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.491 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:46.491 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163171+0000", 2024-09-16T20:51:46.491 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.711e-06 2024-09-16T20:51:46.491 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.491 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.491 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:46.491 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281868+0000", 2024-09-16T20:51:46.491 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118696622 2024-09-16T20:51:46.491 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.492 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.492 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:46.492 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281969+0000", 2024-09-16T20:51:46.492 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101371 2024-09-16T20:51:46.492 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.492 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.492 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:46.492 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282189+0000", 2024-09-16T20:51:46.492 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00021952000000000001 2024-09-16T20:51:46.492 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.492 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:46.492 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.492 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.492 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.492 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4627.0:154 3.1d 3:b82ab2e6:::rbd_data.110ed9ca540.0000000000000000:9 [read 360448~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:51:46.493 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:51:46.493 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 202.25563035900001, 2024-09-16T20:51:46.493 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11932035100000001, 2024-09-16T20:51:46.493 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:46.493 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:46.493 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:46.493 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:46.493 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4627", 2024-09-16T20:51:46.493 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/3312459822", 2024-09-16T20:51:46.493 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 154 2024-09-16T20:51:46.493 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.493 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:46.493 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.493 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:46.494 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:51:46.494 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.494 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.494 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.494 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:46.494 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:51:46.494 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.494 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.494 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.494 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:46.494 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163186+0000", 2024-09-16T20:51:46.494 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:51:46.494 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.495 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.495 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:46.495 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163189+0000", 2024-09-16T20:51:46.495 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6840000000000001e-06 2024-09-16T20:51:46.495 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.495 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.495 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:46.495 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163196+0000", 2024-09-16T20:51:46.495 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7920000000000004e-06 2024-09-16T20:51:46.495 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.495 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.495 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:46.495 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.163201+0000", 2024-09-16T20:51:46.496 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2809999999999999e-06 2024-09-16T20:51:46.496 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.496 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.496 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:46.496 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282219+0000", 2024-09-16T20:51:46.496 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.119018343 2024-09-16T20:51:46.496 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.496 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.496 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:46.496 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282305+0000", 2024-09-16T20:51:46.496 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5865000000000005e-05 2024-09-16T20:51:46.496 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.496 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.497 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:46.497 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282506+0000", 2024-09-16T20:51:46.497 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000200927 2024-09-16T20:51:46.497 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.497 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:46.497 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.497 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.497 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.497 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_repop(client.4366.0:3088 3.1e e38/23 3:79e5ec01:::rbd_header.110e891a9b2b:head v 38'306, mlcod=36'304)", 2024-09-16T20:51:46.497 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:51:46.497 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 202.25461149099999, 2024-09-16T20:51:46.497 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.118102759, 2024-09-16T20:51:46.497 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:46.498 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:46.498 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:46.498 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:46.498 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.498 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:46.498 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:51:46.498 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.498 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.498 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.498 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:46.498 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:51:46.498 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.498 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.499 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.499 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:46.499 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164205+0000", 2024-09-16T20:51:46.499 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.85e-07 2024-09-16T20:51:46.499 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.499 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.499 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:46.499 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.164212+0000", 2024-09-16T20:51:46.499 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3999999999999997e-06 2024-09-16T20:51:46.499 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.499 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.499 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:46.499 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281553+0000", 2024-09-16T20:51:46.500 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.11734135499999999 2024-09-16T20:51:46.500 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.500 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.500 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:46.500 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281567+0000", 2024-09-16T20:51:46.500 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4188999999999999e-05 2024-09-16T20:51:46.500 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.500 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.500 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:46.500 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281653+0000", 2024-09-16T20:51:46.500 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5385999999999999e-05 2024-09-16T20:51:46.500 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.500 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.501 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:46.501 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.281679+0000", 2024-09-16T20:51:46.501 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6689000000000001e-05 2024-09-16T20:51:46.501 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.501 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.501 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-16T20:51:46.501 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282277+0000", 2024-09-16T20:51:46.501 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00059798000000000002 2024-09-16T20:51:46.501 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.501 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.501 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:46.501 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:48:24.282307+0000", 2024-09-16T20:51:46.501 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9975e-05 2024-09-16T20:51:46.502 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.502 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:46.502 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.502 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.502 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.502 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.5255.0:44 3.16 3:6f5476df:::rbd_header.110e25db1f60:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-16T20:51:46.502 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:51:46.502 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 130.098308436, 2024-09-16T20:51:46.502 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.121007509, 2024-09-16T20:51:46.502 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:46.502 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:46.502 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:46.502 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:46.503 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.5255", 2024-09-16T20:51:46.503 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:51:46.503 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 44 2024-09-16T20:51:46.503 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.503 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:46.503 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.503 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:46.503 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:51:46.503 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.503 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.503 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.503 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:46.503 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:51:46.504 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.504 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.504 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.504 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:46.504 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320508+0000", 2024-09-16T20:51:46.504 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0500000000000003e-07 2024-09-16T20:51:46.504 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.504 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.504 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:46.504 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320512+0000", 2024-09-16T20:51:46.504 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2540000000000001e-06 2024-09-16T20:51:46.504 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.505 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.505 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:46.505 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320518+0000", 2024-09-16T20:51:46.505 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1689999999999997e-06 2024-09-16T20:51:46.505 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.505 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.505 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:46.505 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320521+0000", 2024-09-16T20:51:46.505 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5200000000000002e-06 2024-09-16T20:51:46.505 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.505 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.505 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:46.505 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320548+0000", 2024-09-16T20:51:46.506 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6662999999999999e-05 2024-09-16T20:51:46.506 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.506 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.506 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:46.506 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.320587+0000", 2024-09-16T20:51:46.506 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8866999999999999e-05 2024-09-16T20:51:46.506 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.506 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.506 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:46.506 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:49:36.441515+0000", 2024-09-16T20:51:46.506 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.120928331 2024-09-16T20:51:46.506 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.506 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:46.506 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.507 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.507 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.507 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4366.0:20329 3.5 3:a4ea64b7:::rbd_header.110ede5ef857:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:46.507 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-16T20:50:01.061694+0000", 2024-09-16T20:51:46.507 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 105.35712186799999, 2024-09-16T20:51:46.507 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.071234077000000007, 2024-09-16T20:51:46.507 INFO:teuthology.orchestra.run.smithi029.stdout: "continuous": false, 2024-09-16T20:51:46.507 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-16T20:51:46.507 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-16T20:51:46.507 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-16T20:51:46.507 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4366", 2024-09-16T20:51:46.507 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:46.507 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 20329 2024-09-16T20:51:46.508 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.508 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-16T20:51:46.508 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.508 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-16T20:51:46.508 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.061694+0000", 2024-09-16T20:51:46.508 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.508 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.508 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.508 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-16T20:51:46.508 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.061694+0000", 2024-09-16T20:51:46.508 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-16T20:51:46.508 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.509 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.509 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-16T20:51:46.509 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089232+0000", 2024-09-16T20:51:46.509 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.027537275 2024-09-16T20:51:46.509 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.509 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.509 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-16T20:51:46.509 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089237+0000", 2024-09-16T20:51:46.509 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4709999999999998e-06 2024-09-16T20:51:46.509 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.509 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.509 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-16T20:51:46.509 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089246+0000", 2024-09-16T20:51:46.510 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8230000000000001e-06 2024-09-16T20:51:46.510 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.510 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.510 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-16T20:51:46.510 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.089252+0000", 2024-09-16T20:51:46.510 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3060000000000003e-06 2024-09-16T20:51:46.510 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.510 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.510 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-16T20:51:46.510 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.132756+0000", 2024-09-16T20:51:46.510 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.043504334999999998 2024-09-16T20:51:46.510 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.510 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.510 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-16T20:51:46.510 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.132829+0000", 2024-09-16T20:51:46.511 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2849999999999995e-05 2024-09-16T20:51:46.511 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-16T20:51:46.511 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-16T20:51:46.511 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-16T20:51:46.511 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-16T20:50:01.132928+0000", 2024-09-16T20:51:46.511 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.9017000000000004e-05 2024-09-16T20:51:46.511 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.511 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:46.511 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.511 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-16T20:51:46.511 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-16T20:51:46.511 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:51:46.513 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T20:51:46.575 INFO:teuthology.orchestra.run.smithi029.stderr:nodeep-scrub is set 2024-09-16T20:51:46.634 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:46.634 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:51:46.634 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:51:46.634 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:46.643 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T20:51:46.753 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:46.753 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:51:46.753 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:51:46.753 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:51:46.753 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:46.761 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T20:51:46.871 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:46.871 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:51:46.871 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:51:46.871 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:51:46.871 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.871 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:3088 3.1e 3:79e5ec01:::rbd_header.110e891a9b2b:head [watch watch cookie 140213437171488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T20:51:46.871 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:51:46.871 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 202.70529042199999, 2024-09-16T20:51:46.872 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.11895186200000001, 2024-09-16T20:51:46.872 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:46.872 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:46.872 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:46.872 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:46.872 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:46.872 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:46.872 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3088 2024-09-16T20:51:46.872 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.872 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:46.872 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.872 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:46.872 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:51:46.872 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:46.873 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.873 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.873 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:46.873 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163845+0000", 2024-09-16T20:51:46.873 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:46.873 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.873 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.873 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:46.873 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163846+0000", 2024-09-16T20:51:46.873 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6799999999999999e-07 2024-09-16T20:51:46.873 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.873 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.873 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:46.873 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163850+0000", 2024-09-16T20:51:46.873 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.929e-06 2024-09-16T20:51:46.874 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.874 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.874 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:46.874 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163862+0000", 2024-09-16T20:51:46.874 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1972e-05 2024-09-16T20:51:46.874 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.874 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.874 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:46.874 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163867+0000", 2024-09-16T20:51:46.874 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.451e-06 2024-09-16T20:51:46.874 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.874 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.874 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:46.874 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163912+0000", 2024-09-16T20:51:46.874 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4456000000000003e-05 2024-09-16T20:51:46.875 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.875 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.875 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:46.875 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.163962+0000", 2024-09-16T20:51:46.875 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0578000000000003e-05 2024-09-16T20:51:46.875 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.875 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.875 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-16T20:51:46.875 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164058+0000", 2024-09-16T20:51:46.875 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5934999999999995e-05 2024-09-16T20:51:46.875 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.875 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.875 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:51:46.875 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.164570+0000", 2024-09-16T20:51:46.876 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00051241100000000005 2024-09-16T20:51:46.876 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.876 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.876 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:46.876 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282629+0000", 2024-09-16T20:51:46.876 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.118058986 2024-09-16T20:51:46.876 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.876 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.876 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:51:46.876 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282725+0000", 2024-09-16T20:51:46.876 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5063999999999996e-05 2024-09-16T20:51:46.876 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.876 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.877 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:46.877 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:24.282797+0000", 2024-09-16T20:51:46.877 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2212000000000006e-05 2024-09-16T20:51:46.877 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:46.877 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:46.877 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:46.877 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.877 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.877 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:8039 3.1e 3.ac44ff9e (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:51:46.877 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:51:46.877 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 177.16146307299999, 2024-09-16T20:51:46.877 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0053310449999999, 2024-09-16T20:51:46.877 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:46.877 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:46.878 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:46.878 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:46.878 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:46.878 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:46.878 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8039 2024-09-16T20:51:46.878 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.878 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:46.878 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.878 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:46.878 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:51:46.878 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:46.878 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.878 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.879 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:46.879 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707672+0000", 2024-09-16T20:51:46.879 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:46.879 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.879 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.879 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:46.879 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707674+0000", 2024-09-16T20:51:46.879 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6530000000000001e-06 2024-09-16T20:51:46.879 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.879 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.879 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:46.879 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707681+0000", 2024-09-16T20:51:46.879 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9639999999999997e-06 2024-09-16T20:51:46.879 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.880 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.880 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:46.880 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707700+0000", 2024-09-16T20:51:46.880 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8674000000000001e-05 2024-09-16T20:51:46.880 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.880 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.880 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:46.880 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707708+0000", 2024-09-16T20:51:46.880 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6810000000000004e-06 2024-09-16T20:51:46.880 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.880 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.880 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:46.880 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711293+0000", 2024-09-16T20:51:46.880 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035848189999999999 2024-09-16T20:51:46.881 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.881 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.881 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:46.881 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711301+0000", 2024-09-16T20:51:46.881 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4730000000000003e-06 2024-09-16T20:51:46.881 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.881 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.881 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:46.881 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.712990+0000", 2024-09-16T20:51:46.881 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.001689348 2024-09-16T20:51:46.881 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.881 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.881 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:46.881 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713003+0000", 2024-09-16T20:51:46.882 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3433e-05 2024-09-16T20:51:46.882 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:46.882 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:46.882 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:46.882 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.882 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.882 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:8040 3.1e 3.ac44ff9e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:51:46.882 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:51:46.882 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 177.161402678, 2024-09-16T20:51:46.882 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0053410780000001, 2024-09-16T20:51:46.882 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:46.882 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:46.882 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:46.882 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:46.882 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:46.883 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:46.883 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8040 2024-09-16T20:51:46.883 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.883 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:46.883 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.883 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:46.883 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:51:46.883 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:46.883 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.883 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.883 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:46.883 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:51:46.883 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:46.883 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.883 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.884 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:46.884 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707733+0000", 2024-09-16T20:51:46.884 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8100000000000003e-07 2024-09-16T20:51:46.884 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.884 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.884 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:46.884 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707737+0000", 2024-09-16T20:51:46.884 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5020000000000001e-06 2024-09-16T20:51:46.884 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.884 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.884 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:46.884 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707745+0000", 2024-09-16T20:51:46.884 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2169999999999994e-06 2024-09-16T20:51:46.884 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.885 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.885 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:46.885 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.707751+0000", 2024-09-16T20:51:46.885 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.263e-06 2024-09-16T20:51:46.885 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.885 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.885 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:46.885 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711336+0000", 2024-09-16T20:51:46.885 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035848360000000001 2024-09-16T20:51:46.885 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.885 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.885 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:46.885 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711340+0000", 2024-09-16T20:51:46.886 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1309999999999999e-06 2024-09-16T20:51:46.886 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.886 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.886 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:46.886 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713060+0000", 2024-09-16T20:51:46.886 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0017195059999999 2024-09-16T20:51:46.886 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.886 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.886 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:46.886 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713074+0000", 2024-09-16T20:51:46.886 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4042e-05 2024-09-16T20:51:46.886 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:46.886 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:46.886 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:46.887 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.887 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.887 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4927.0:4487 3.1e 3.ac44ff9e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T20:51:46.887 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:51:46.887 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 177.16088247600001, 2024-09-16T20:51:46.887 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0048840880000001, 2024-09-16T20:51:46.887 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:46.887 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:46.887 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:46.887 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:46.887 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4927", 2024-09-16T20:51:46.887 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1291565550", 2024-09-16T20:51:46.887 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4487 2024-09-16T20:51:46.888 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.888 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:46.888 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.888 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:46.888 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:51:46.888 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:46.888 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.888 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.888 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:46.888 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708253+0000", 2024-09-16T20:51:46.888 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:46.888 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.888 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.889 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:46.889 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708254+0000", 2024-09-16T20:51:46.889 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1800000000000005e-07 2024-09-16T20:51:46.889 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.889 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.889 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:46.889 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708258+0000", 2024-09-16T20:51:46.889 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8689999999999997e-06 2024-09-16T20:51:46.889 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.889 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.889 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:46.889 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708267+0000", 2024-09-16T20:51:46.890 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9590000000000007e-06 2024-09-16T20:51:46.890 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.890 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.890 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:46.890 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.708274+0000", 2024-09-16T20:51:46.890 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9970000000000001e-06 2024-09-16T20:51:46.890 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.890 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.890 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:46.890 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711366+0000", 2024-09-16T20:51:46.890 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030919530000000001 2024-09-16T20:51:46.890 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.890 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.890 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:46.891 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:49.711372+0000", 2024-09-16T20:51:46.891 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9410000000000004e-06 2024-09-16T20:51:46.891 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.891 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.891 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:46.891 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713126+0000", 2024-09-16T20:51:46.891 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0017537249999999 2024-09-16T20:51:46.891 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.891 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.891 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:46.891 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:48:50.713137+0000", 2024-09-16T20:51:46.891 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0825999999999999e-05 2024-09-16T20:51:46.891 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:46.892 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:46.892 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:46.892 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.892 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.892 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5089.0:11703 3.1b 3:dcd4684e:::rbd_data.110e747f0779.000000000000001b:head [set-alloc-hint object_size 262144 write_size 262144,write 139264~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T20:51:46.892 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:51:46.892 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 162.73114670199999, 2024-09-16T20:51:46.892 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.084231385000000006, 2024-09-16T20:51:46.892 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:46.892 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:46.892 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:46.892 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:46.892 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5089", 2024-09-16T20:51:46.892 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1656248868", 2024-09-16T20:51:46.893 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 11703 2024-09-16T20:51:46.893 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.893 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:46.893 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.893 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:46.893 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:51:46.893 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:46.893 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.893 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.893 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:46.893 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:51:46.893 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:46.893 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.893 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.894 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:46.894 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137989+0000", 2024-09-16T20:51:46.894 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2600000000000001e-07 2024-09-16T20:51:46.894 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.894 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.894 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:46.894 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.137998+0000", 2024-09-16T20:51:46.894 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9549999999999998e-06 2024-09-16T20:51:46.894 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.894 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.894 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:46.894 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138008+0000", 2024-09-16T20:51:46.894 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3789999999999995e-06 2024-09-16T20:51:46.895 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.895 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.895 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:46.895 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138013+0000", 2024-09-16T20:51:46.895 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2209999999999996e-06 2024-09-16T20:51:46.895 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.895 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.895 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:46.895 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138054+0000", 2024-09-16T20:51:46.895 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1022999999999997e-05 2024-09-16T20:51:46.895 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.895 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.895 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:46.895 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138124+0000", 2024-09-16T20:51:46.896 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0234999999999994e-05 2024-09-16T20:51:46.896 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.896 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.896 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:46.896 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138197+0000", 2024-09-16T20:51:46.896 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2451000000000001e-05 2024-09-16T20:51:46.896 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.896 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.896 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:46.896 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222010+0000", 2024-09-16T20:51:46.896 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.083813433000000007 2024-09-16T20:51:46.896 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.896 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.897 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:51:46.897 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222163+0000", 2024-09-16T20:51:46.897 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015274599999999999 2024-09-16T20:51:46.897 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.897 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.897 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:51:46.897 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222197+0000", 2024-09-16T20:51:46.897 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3683000000000003e-05 2024-09-16T20:51:46.897 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.897 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.897 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:46.897 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222220+0000", 2024-09-16T20:51:46.897 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3433000000000001e-05 2024-09-16T20:51:46.898 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:46.898 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:46.898 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:46.898 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.898 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.898 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.5089.0:11702 3.15 e73/64 3:acf33c8b:::rbd_data.110e747f0779.000000000000001a:head v 73'132, mlcod=73'130)", 2024-09-16T20:51:46.898 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:51:46.898 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 162.73087937299999, 2024-09-16T20:51:46.898 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.084079763000000002, 2024-09-16T20:51:46.898 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:46.898 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:46.898 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:46.898 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:46.899 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.899 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:46.899 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:51:46.899 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:46.899 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.899 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.899 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:46.899 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:51:46.899 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:46.899 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.899 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.899 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:46.899 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138256+0000", 2024-09-16T20:51:46.900 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:51:46.900 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.900 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.900 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:46.900 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.138263+0000", 2024-09-16T20:51:46.900 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1e-06 2024-09-16T20:51:46.900 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.900 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.900 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:46.900 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221589+0000", 2024-09-16T20:51:46.900 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.083326002999999996 2024-09-16T20:51:46.900 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.900 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.901 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:46.901 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221614+0000", 2024-09-16T20:51:46.901 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5593999999999999e-05 2024-09-16T20:51:46.901 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.901 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.901 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:46.901 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221714+0000", 2024-09-16T20:51:46.901 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9889000000000005e-05 2024-09-16T20:51:46.901 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.901 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.901 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:46.901 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.221727+0000", 2024-09-16T20:51:46.901 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3125999999999999e-05 2024-09-16T20:51:46.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.902 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.902 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:51:46.902 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222294+0000", 2024-09-16T20:51:46.902 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00056712199999999998 2024-09-16T20:51:46.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.902 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.902 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:46.902 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:04.222336+0000", 2024-09-16T20:51:46.902 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1470000000000001e-05 2024-09-16T20:51:46.902 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:46.902 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:46.902 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:46.903 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.903 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.903 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5138.0:9358 3.c 3:33996a43:::rbd_data.110e6372565d.0000000000000007:head [read 1343488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T20:51:46.903 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:51:46.903 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 139.01660396299999, 2024-09-16T20:51:46.903 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0167493569999999, 2024-09-16T20:51:46.903 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:46.903 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:46.903 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:46.903 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:46.903 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5138", 2024-09-16T20:51:46.903 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:51:46.903 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9358 2024-09-16T20:51:46.904 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.904 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:46.904 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.904 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:46.904 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:51:46.904 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:46.904 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.904 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.904 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:46.904 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852531+0000", 2024-09-16T20:51:46.904 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:46.904 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.904 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.905 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:46.905 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852533+0000", 2024-09-16T20:51:46.905 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.646e-06 2024-09-16T20:51:46.905 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.905 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.905 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:46.905 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852539+0000", 2024-09-16T20:51:46.905 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3389999999999998e-06 2024-09-16T20:51:46.905 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.905 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.905 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:46.905 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852552+0000", 2024-09-16T20:51:46.905 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.271e-05 2024-09-16T20:51:46.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.906 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.906 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:46.906 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852563+0000", 2024-09-16T20:51:46.906 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1139e-05 2024-09-16T20:51:46.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.906 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.906 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:46.906 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852883+0000", 2024-09-16T20:51:46.906 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00032010600000000001 2024-09-16T20:51:46.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.906 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.906 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:46.907 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852890+0000", 2024-09-16T20:51:46.907 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7669999999999998e-06 2024-09-16T20:51:46.907 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.907 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.907 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:46.907 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869059+0000", 2024-09-16T20:51:46.907 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.016169353 2024-09-16T20:51:46.907 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.907 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.907 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:46.907 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869178+0000", 2024-09-16T20:51:46.907 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118193 2024-09-16T20:51:46.907 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.908 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.908 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:46.908 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869281+0000", 2024-09-16T20:51:46.908 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010310399999999999 2024-09-16T20:51:46.908 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:46.908 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:46.908 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:46.908 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.908 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.908 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5138.0:9382 3.c 3:33996a43:::rbd_data.110e6372565d.0000000000000007:head [read 0~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T20:51:46.908 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:51:46.908 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 139.01655308599999, 2024-09-16T20:51:46.908 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.017136034, 2024-09-16T20:51:46.909 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:46.909 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:46.909 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:46.909 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:46.909 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5138", 2024-09-16T20:51:46.909 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3677246376", 2024-09-16T20:51:46.909 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9382 2024-09-16T20:51:46.909 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.909 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:46.909 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.909 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:46.909 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:51:46.909 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:46.910 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.910 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.910 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:46.910 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852582+0000", 2024-09-16T20:51:46.910 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:46.910 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.910 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.910 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:46.910 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852583+0000", 2024-09-16T20:51:46.910 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.51e-07 2024-09-16T20:51:46.910 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.910 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.910 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:46.910 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852586+0000", 2024-09-16T20:51:46.911 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0280000000000001e-06 2024-09-16T20:51:46.911 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.911 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.911 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:46.911 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852595+0000", 2024-09-16T20:51:46.911 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2799999999999992e-06 2024-09-16T20:51:46.911 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.911 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.911 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:46.911 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852601+0000", 2024-09-16T20:51:46.911 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3810000000000004e-06 2024-09-16T20:51:46.911 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.911 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.911 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:46.912 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852921+0000", 2024-09-16T20:51:46.912 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00031979699999999997 2024-09-16T20:51:46.912 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.912 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.912 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:46.912 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:27.852930+0000", 2024-09-16T20:51:46.912 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6489999999999997e-06 2024-09-16T20:51:46.912 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.912 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.912 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:46.912 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869434+0000", 2024-09-16T20:51:46.912 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0165038829999999 2024-09-16T20:51:46.912 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.912 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.912 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:46.913 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869495+0000", 2024-09-16T20:51:46.913 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1509999999999999e-05 2024-09-16T20:51:46.913 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.913 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.913 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:46.913 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:28.869718+0000", 2024-09-16T20:51:46.913 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000222955 2024-09-16T20:51:46.913 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:46.913 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:46.913 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:46.913 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.913 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.913 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9971 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 643072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:46.913 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642102+0000", 2024-09-16T20:51:46.913 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 116.22703294199999, 2024-09-16T20:51:46.914 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077421016999999995, 2024-09-16T20:51:46.914 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:46.914 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:46.914 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:46.914 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:46.914 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:46.914 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:46.914 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9971 2024-09-16T20:51:46.914 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.914 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:46.914 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.914 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:46.914 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642102+0000", 2024-09-16T20:51:46.915 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:46.915 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.915 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.915 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:46.915 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642102+0000", 2024-09-16T20:51:46.915 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:46.915 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.915 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.915 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:46.915 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642103+0000", 2024-09-16T20:51:46.915 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7399999999999998e-07 2024-09-16T20:51:46.915 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.915 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.915 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:46.915 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642105+0000", 2024-09-16T20:51:46.916 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2649999999999999e-06 2024-09-16T20:51:46.916 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.916 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.916 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:46.916 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642112+0000", 2024-09-16T20:51:46.916 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.294e-06 2024-09-16T20:51:46.916 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.916 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.916 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:46.916 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642118+0000", 2024-09-16T20:51:46.916 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4500000000000003e-06 2024-09-16T20:51:46.916 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.916 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.916 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:46.916 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719285+0000", 2024-09-16T20:51:46.917 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077166883000000006 2024-09-16T20:51:46.917 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.917 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.917 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:46.917 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719368+0000", 2024-09-16T20:51:46.917 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2811000000000004e-05 2024-09-16T20:51:46.917 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.917 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.917 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:46.917 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719523+0000", 2024-09-16T20:51:46.917 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015584 2024-09-16T20:51:46.917 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:46.917 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:46.917 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:46.918 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.918 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.918 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9972 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 648192~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:46.918 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642125+0000", 2024-09-16T20:51:46.918 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 116.227009995, 2024-09-16T20:51:46.918 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077710324999999997, 2024-09-16T20:51:46.918 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:46.918 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:46.918 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:46.918 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:46.918 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:46.918 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:46.918 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9972 2024-09-16T20:51:46.919 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.919 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:46.919 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.919 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:46.919 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642125+0000", 2024-09-16T20:51:46.919 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:46.919 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.919 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.919 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:46.919 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642125+0000", 2024-09-16T20:51:46.919 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:46.919 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.919 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.919 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:46.920 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642126+0000", 2024-09-16T20:51:46.920 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1899999999999998e-07 2024-09-16T20:51:46.920 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.920 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.920 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:46.920 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642128+0000", 2024-09-16T20:51:46.920 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8619999999999999e-06 2024-09-16T20:51:46.920 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.920 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.920 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:46.920 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642134+0000", 2024-09-16T20:51:46.920 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5289999999999997e-06 2024-09-16T20:51:46.920 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.920 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.920 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:46.921 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642139+0000", 2024-09-16T20:51:46.921 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1250000000000001e-06 2024-09-16T20:51:46.921 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.921 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.921 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:46.921 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719570+0000", 2024-09-16T20:51:46.921 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077430516000000005 2024-09-16T20:51:46.921 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.921 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.921 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:46.921 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719655+0000", 2024-09-16T20:51:46.921 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5500000000000005e-05 2024-09-16T20:51:46.921 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.922 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.922 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:46.922 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719836+0000", 2024-09-16T20:51:46.922 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018037399999999999 2024-09-16T20:51:46.922 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:46.922 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:46.922 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:46.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.922 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.922 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9973 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 653312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:46.922 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:51:46.922 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 116.226989385, 2024-09-16T20:51:46.922 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077905262000000003, 2024-09-16T20:51:46.922 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:46.922 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:46.923 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:46.923 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:46.923 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:46.923 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:46.923 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9973 2024-09-16T20:51:46.923 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.923 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:46.923 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.923 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:46.923 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:51:46.923 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:46.923 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.923 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.923 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:46.923 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:51:46.924 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:46.924 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.924 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.924 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:46.924 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642146+0000", 2024-09-16T20:51:46.924 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5900000000000002e-07 2024-09-16T20:51:46.924 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.924 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.924 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:46.924 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642148+0000", 2024-09-16T20:51:46.924 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0219999999999999e-06 2024-09-16T20:51:46.924 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.924 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.924 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:46.925 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642154+0000", 2024-09-16T20:51:46.925 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8540000000000002e-06 2024-09-16T20:51:46.925 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.925 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.925 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:46.925 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642159+0000", 2024-09-16T20:51:46.925 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.045e-06 2024-09-16T20:51:46.925 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.925 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.925 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:46.925 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719875+0000", 2024-09-16T20:51:46.925 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077716116000000002 2024-09-16T20:51:46.925 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.926 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.926 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:46.926 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.719934+0000", 2024-09-16T20:51:46.926 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8819e-05 2024-09-16T20:51:46.926 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.926 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.926 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:46.926 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720051+0000", 2024-09-16T20:51:46.926 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000116947 2024-09-16T20:51:46.926 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:46.926 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:46.926 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:46.926 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.926 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.926 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9974 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 658432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:46.927 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.642166+0000", 2024-09-16T20:51:46.927 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 116.226969471, 2024-09-16T20:51:46.927 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.078179786000000001, 2024-09-16T20:51:46.927 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:46.927 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:46.927 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:46.927 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:46.927 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:46.927 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:46.927 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9974 2024-09-16T20:51:46.927 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.927 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:46.927 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.928 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:46.928 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642166+0000", 2024-09-16T20:51:46.928 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:46.928 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.928 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.928 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:46.928 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.642166+0000", 2024-09-16T20:51:46.928 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:46.928 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.928 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.928 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:46.928 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647343+0000", 2024-09-16T20:51:46.928 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0051769440000000002 2024-09-16T20:51:46.929 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.929 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.929 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:46.929 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647348+0000", 2024-09-16T20:51:46.929 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3619999999999997e-06 2024-09-16T20:51:46.929 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.929 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.929 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:46.929 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647359+0000", 2024-09-16T20:51:46.929 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0929e-05 2024-09-16T20:51:46.929 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.929 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.929 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:46.929 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647366+0000", 2024-09-16T20:51:46.930 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0910000000000004e-06 2024-09-16T20:51:46.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.930 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.930 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:46.930 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720094+0000", 2024-09-16T20:51:46.930 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.072727403999999995 2024-09-16T20:51:46.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.930 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.930 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:46.930 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720173+0000", 2024-09-16T20:51:46.930 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9382000000000007e-05 2024-09-16T20:51:46.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.930 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.930 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:46.930 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.720346+0000", 2024-09-16T20:51:46.931 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017267399999999999 2024-09-16T20:51:46.931 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:46.931 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:46.931 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:46.931 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.931 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.931 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9991 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 745472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:46.931 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:51:46.931 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 116.221400078, 2024-09-16T20:51:46.931 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077257820000000005, 2024-09-16T20:51:46.931 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:46.931 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:46.931 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:46.931 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:46.932 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:46.932 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:46.932 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9991 2024-09-16T20:51:46.932 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.932 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:46.932 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.932 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:46.932 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:51:46.932 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:46.932 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.932 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.932 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:46.932 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:51:46.933 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:46.933 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.933 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.933 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:46.933 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647735+0000", 2024-09-16T20:51:46.933 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1199999999999999e-07 2024-09-16T20:51:46.933 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.933 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.933 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:46.933 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647736+0000", 2024-09-16T20:51:46.933 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3699999999999999e-07 2024-09-16T20:51:46.933 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.933 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.934 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:46.934 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647739+0000", 2024-09-16T20:51:46.934 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9359999999999999e-06 2024-09-16T20:51:46.934 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.934 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.934 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:46.934 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647742+0000", 2024-09-16T20:51:46.934 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5720000000000001e-06 2024-09-16T20:51:46.934 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.934 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.934 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:46.934 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724774+0000", 2024-09-16T20:51:46.934 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077032457999999998 2024-09-16T20:51:46.934 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.935 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.935 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:46.935 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724818+0000", 2024-09-16T20:51:46.935 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3803000000000001e-05 2024-09-16T20:51:46.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.935 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.935 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:46.935 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.724993+0000", 2024-09-16T20:51:46.935 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000174902 2024-09-16T20:51:46.935 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:46.935 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:46.935 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:46.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.936 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.936 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9992 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 750592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:46.936 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:51:46.936 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 116.221390489, 2024-09-16T20:51:46.936 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077553227000000002, 2024-09-16T20:51:46.936 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:46.936 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:46.936 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:46.936 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:46.936 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:46.936 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:46.936 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9992 2024-09-16T20:51:46.936 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.937 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:46.937 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.937 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:46.937 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:51:46.937 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:46.937 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.937 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.937 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:46.937 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:51:46.937 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:46.937 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.937 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.937 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:46.938 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647745+0000", 2024-09-16T20:51:46.938 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0200000000000001e-07 2024-09-16T20:51:46.938 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.938 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.938 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:46.938 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647746+0000", 2024-09-16T20:51:46.938 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.088e-06 2024-09-16T20:51:46.938 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.938 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.938 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:46.938 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647749+0000", 2024-09-16T20:51:46.938 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9170000000000002e-06 2024-09-16T20:51:46.938 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.939 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:46.939 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647752+0000", 2024-09-16T20:51:46.939 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.774e-06 2024-09-16T20:51:46.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.939 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:46.939 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725042+0000", 2024-09-16T20:51:46.939 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077289952999999995 2024-09-16T20:51:46.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.939 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:46.939 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725113+0000", 2024-09-16T20:51:46.939 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0920999999999994e-05 2024-09-16T20:51:46.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:46.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725298+0000", 2024-09-16T20:51:46.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000185372 2024-09-16T20:51:46.940 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:46.940 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:46.940 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:46.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.940 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:9993 3.1e 3:7dd419ec:::rbd_data.110ea93691bb.000000000000000b:23 [read 755712~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:46.940 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.647755+0000", 2024-09-16T20:51:46.940 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 116.221380463, 2024-09-16T20:51:46.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077725674999999994, 2024-09-16T20:51:46.941 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:46.941 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:46.941 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:46.941 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:46.941 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:46.941 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:46.941 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 9993 2024-09-16T20:51:46.941 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.941 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:46.941 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.942 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:46.942 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647755+0000", 2024-09-16T20:51:46.942 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:46.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.942 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.942 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:46.942 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.647755+0000", 2024-09-16T20:51:46.942 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:46.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.942 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.942 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:46.942 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653406+0000", 2024-09-16T20:51:46.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.005650876 2024-09-16T20:51:46.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.943 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:46.943 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653411+0000", 2024-09-16T20:51:46.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8899999999999998e-06 2024-09-16T20:51:46.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.943 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:46.943 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653423+0000", 2024-09-16T20:51:46.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2447e-05 2024-09-16T20:51:46.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.943 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:46.943 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.653431+0000", 2024-09-16T20:51:46.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5730000000000002e-06 2024-09-16T20:51:46.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.944 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:46.944 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725333+0000", 2024-09-16T20:51:46.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.071902743000000005 2024-09-16T20:51:46.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.944 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:46.944 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725385+0000", 2024-09-16T20:51:46.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1585000000000002e-05 2024-09-16T20:51:46.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.945 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:46.945 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.725481+0000", 2024-09-16T20:51:46.945 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5561000000000002e-05 2024-09-16T20:51:46.945 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:46.945 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:46.945 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:46.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.945 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6061.0:5400 3.b 3:d353d652:::rbd_data.110e16eafbd.0000000000000010:head [read 577536~28672] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-16T20:51:46.945 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:27.761367+0000", 2024-09-16T20:51:46.945 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 79.107768516999997, 2024-09-16T20:51:46.945 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0029066630000001, 2024-09-16T20:51:46.945 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:46.945 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:46.946 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:46.946 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:46.946 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6061", 2024-09-16T20:51:46.946 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3930128440", 2024-09-16T20:51:46.946 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5400 2024-09-16T20:51:46.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.946 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:46.946 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.946 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:46.946 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761367+0000", 2024-09-16T20:51:46.946 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:46.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.946 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.946 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:46.947 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761367+0000", 2024-09-16T20:51:46.947 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:46.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.947 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.947 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:46.947 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761369+0000", 2024-09-16T20:51:46.947 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.934e-06 2024-09-16T20:51:46.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.947 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.947 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:46.947 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761374+0000", 2024-09-16T20:51:46.947 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7139999999999998e-06 2024-09-16T20:51:46.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.947 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.948 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:46.948 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761387+0000", 2024-09-16T20:51:46.948 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2349e-05 2024-09-16T20:51:46.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.948 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:46.948 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761393+0000", 2024-09-16T20:51:46.948 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.652e-06 2024-09-16T20:51:46.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.948 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:46.948 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761442+0000", 2024-09-16T20:51:46.948 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9011000000000002e-05 2024-09-16T20:51:46.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.949 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.949 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:46.949 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.761451+0000", 2024-09-16T20:51:46.949 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1920000000000005e-06 2024-09-16T20:51:46.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.949 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.949 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:46.949 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:28.764169+0000", 2024-09-16T20:51:46.949 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.002718161 2024-09-16T20:51:46.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.949 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.949 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:46.949 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:28.764273+0000", 2024-09-16T20:51:46.950 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010465 2024-09-16T20:51:46.950 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:46.950 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:46.950 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:46.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.950 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.950 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6061.0:5424 3.b 3:d353d652:::rbd_data.110e16eafbd.0000000000000010:head [read 606208~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-16T20:51:46.950 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:27.766882+0000", 2024-09-16T20:51:46.950 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 79.102253153999996, 2024-09-16T20:51:46.950 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99753650500000002, 2024-09-16T20:51:46.950 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:46.950 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:46.950 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:46.951 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:46.951 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6061", 2024-09-16T20:51:46.951 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3930128440", 2024-09-16T20:51:46.951 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5424 2024-09-16T20:51:46.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.951 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:46.951 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.951 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:46.951 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766882+0000", 2024-09-16T20:51:46.951 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:46.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.951 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.951 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:46.952 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766882+0000", 2024-09-16T20:51:46.952 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:46.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.952 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.952 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:46.952 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766883+0000", 2024-09-16T20:51:46.952 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3289999999999999e-06 2024-09-16T20:51:46.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.952 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.952 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:46.952 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766888+0000", 2024-09-16T20:51:46.952 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5800000000000002e-06 2024-09-16T20:51:46.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.953 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.953 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:46.953 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766900+0000", 2024-09-16T20:51:46.953 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1861e-05 2024-09-16T20:51:46.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.953 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.953 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:46.953 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766909+0000", 2024-09-16T20:51:46.953 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5210000000000008e-06 2024-09-16T20:51:46.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.953 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.953 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:46.953 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766958+0000", 2024-09-16T20:51:46.953 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8718999999999999e-05 2024-09-16T20:51:46.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.954 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.954 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:46.954 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:27.766975+0000", 2024-09-16T20:51:46.954 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7231000000000002e-05 2024-09-16T20:51:46.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.954 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.954 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:46.954 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:28.764327+0000", 2024-09-16T20:51:46.954 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99735112500000001 2024-09-16T20:51:46.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.954 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.954 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:46.954 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:28.764419+0000", 2024-09-16T20:51:46.955 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2139000000000007e-05 2024-09-16T20:51:46.955 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:46.955 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:46.955 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:46.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.955 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:31909 3.1e 3:79ff2235:::rbd_header.110ee9a2c138:head [watch reconnect cookie 140213436757440 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e199)", 2024-09-16T20:51:46.955 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:51:26.618259+0000", 2024-09-16T20:51:46.955 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 20.250876128000002, 2024-09-16T20:51:46.955 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0148341999999999, 2024-09-16T20:51:46.955 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:46.955 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:46.955 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:46.955 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:46.956 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:46.956 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:46.956 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 31909 2024-09-16T20:51:46.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.956 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:46.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.956 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:46.956 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.618259+0000", 2024-09-16T20:51:46.956 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:46.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.956 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:46.956 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.618259+0000", 2024-09-16T20:51:46.956 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:46.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.957 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:46.957 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.618261+0000", 2024-09-16T20:51:46.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4899999999999999e-06 2024-09-16T20:51:46.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.957 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:46.957 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.618267+0000", 2024-09-16T20:51:46.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8649999999999998e-06 2024-09-16T20:51:46.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.957 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:46.957 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.618277+0000", 2024-09-16T20:51:46.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0209e-05 2024-09-16T20:51:46.958 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.958 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.958 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:46.958 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.618282+0000", 2024-09-16T20:51:46.958 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.733e-06 2024-09-16T20:51:46.958 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.958 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.958 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:46.958 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.618341+0000", 2024-09-16T20:51:46.958 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8823000000000002e-05 2024-09-16T20:51:46.958 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.958 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.958 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:51:46.958 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.618346+0000", 2024-09-16T20:51:46.959 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8049999999999997e-06 2024-09-16T20:51:46.959 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.959 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.959 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:46.959 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.620745+0000", 2024-09-16T20:51:46.959 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0023987499999999998 2024-09-16T20:51:46.959 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.959 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.959 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:46.959 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.620755+0000", 2024-09-16T20:51:46.959 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0474e-05 2024-09-16T20:51:46.959 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.959 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.959 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:46.959 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:27.629480+0000", 2024-09-16T20:51:46.960 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.008724403 2024-09-16T20:51:46.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.960 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:46.960 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:27.629590+0000", 2024-09-16T20:51:46.960 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011044600000000001 2024-09-16T20:51:46.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.960 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:46.960 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:27.633093+0000", 2024-09-16T20:51:46.960 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003503202 2024-09-16T20:51:46.960 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:46.960 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:46.961 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:46.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.961 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.961 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:31914 3.1e 3:79ff2235:::rbd_header.110ee9a2c138:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e199)", 2024-09-16T20:51:46.961 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:51:26.635328+0000", 2024-09-16T20:51:46.961 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 20.2338071, 2024-09-16T20:51:46.961 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99790577499999999, 2024-09-16T20:51:46.961 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:46.961 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:46.961 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:46.961 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:46.961 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:46.961 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:46.962 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 31914 2024-09-16T20:51:46.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.962 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:46.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.962 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:46.962 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.635328+0000", 2024-09-16T20:51:46.962 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:46.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.962 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:46.962 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.635328+0000", 2024-09-16T20:51:46.962 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:46.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.963 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.963 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:46.963 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.635330+0000", 2024-09-16T20:51:46.963 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3379999999999999e-06 2024-09-16T20:51:46.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.963 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.963 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:46.963 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.635334+0000", 2024-09-16T20:51:46.963 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6280000000000004e-06 2024-09-16T20:51:46.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.963 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.963 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:46.963 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.635344+0000", 2024-09-16T20:51:46.963 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7229999999999999e-06 2024-09-16T20:51:46.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.964 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.964 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:46.964 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.635349+0000", 2024-09-16T20:51:46.964 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1719999999999998e-06 2024-09-16T20:51:46.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.964 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.964 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:46.964 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.635424+0000", 2024-09-16T20:51:46.964 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4499999999999995e-05 2024-09-16T20:51:46.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.964 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.965 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:46.965 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:26.635435+0000", 2024-09-16T20:51:46.965 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1355000000000001e-05 2024-09-16T20:51:46.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.965 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.965 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:46.965 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:27.630304+0000", 2024-09-16T20:51:46.965 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99486901500000002 2024-09-16T20:51:46.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.965 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.965 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:46.965 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:27.630370+0000", 2024-09-16T20:51:46.965 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6390999999999995e-05 2024-09-16T20:51:46.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.966 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-16T20:51:46.966 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:27.630488+0000", 2024-09-16T20:51:46.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011726 2024-09-16T20:51:46.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.966 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:51:46.966 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:27.632101+0000", 2024-09-16T20:51:46.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016134249999999999 2024-09-16T20:51:46.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.966 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:46.966 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:27.633138+0000", 2024-09-16T20:51:46.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010367499999999999 2024-09-16T20:51:46.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.967 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:51:46.967 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:27.633196+0000", 2024-09-16T20:51:46.967 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8399000000000001e-05 2024-09-16T20:51:46.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.967 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:46.967 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:27.633234+0000", 2024-09-16T20:51:46.967 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7818999999999998e-05 2024-09-16T20:51:46.967 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:46.967 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:46.967 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:46.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.968 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.968 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:32211 3.1b 3:df9cfa64:::journal.110e645639d:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e204)", 2024-09-16T20:51:46.968 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:51:31.721834+0000", 2024-09-16T20:51:46.968 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 15.147301468, 2024-09-16T20:51:46.968 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.917244744, 2024-09-16T20:51:46.968 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:46.968 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:46.968 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:46.968 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:46.968 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:46.968 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:46.968 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 32211 2024-09-16T20:51:46.968 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.968 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:46.969 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.969 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:46.969 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:31.721834+0000", 2024-09-16T20:51:46.969 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:46.969 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.969 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.969 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:46.969 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:31.721834+0000", 2024-09-16T20:51:46.969 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:46.969 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.969 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.969 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:46.969 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:31.721836+0000", 2024-09-16T20:51:46.969 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.719e-06 2024-09-16T20:51:46.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.970 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.970 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:46.970 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:31.721841+0000", 2024-09-16T20:51:46.970 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.913e-06 2024-09-16T20:51:46.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.970 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.970 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:46.970 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:31.721855+0000", 2024-09-16T20:51:46.970 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3149000000000001e-05 2024-09-16T20:51:46.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.970 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.970 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:46.971 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:31.721861+0000", 2024-09-16T20:51:46.971 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0310000000000004e-06 2024-09-16T20:51:46.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.971 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.971 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:46.971 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:31.721930+0000", 2024-09-16T20:51:46.971 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9739000000000004e-05 2024-09-16T20:51:46.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.971 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.971 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:46.971 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:31.721939+0000", 2024-09-16T20:51:46.971 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6170000000000007e-06 2024-09-16T20:51:46.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.971 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.972 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:46.972 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:32.638931+0000", 2024-09-16T20:51:46.972 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.91699213700000004 2024-09-16T20:51:46.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:46.972 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:46.972 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:46.972 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:32.639079+0000", 2024-09-16T20:51:46.972 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000147439 2024-09-16T20:51:46.972 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:46.972 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:46.972 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:46.972 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:46.972 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:46.972 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:46.974 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T20:51:47.082 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:47.082 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:51:47.082 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:51:47.082 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:47.091 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T20:51:47.197 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:47.198 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:51:47.198 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:51:47.198 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:51:47.198 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:47.206 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T20:51:47.315 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:47.315 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:51:47.316 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:51:47.316 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:51:47.316 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.316 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12548 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 628736~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:47.316 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757009+0000", 2024-09-16T20:51:47.316 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 116.55686529499999, 2024-09-16T20:51:47.316 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065743956000000006, 2024-09-16T20:51:47.316 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:47.316 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:47.316 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:47.316 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:47.316 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:47.317 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:47.317 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12548 2024-09-16T20:51:47.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.317 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:47.317 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.317 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:47.317 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757009+0000", 2024-09-16T20:51:47.317 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.317 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.317 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:47.317 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757009+0000", 2024-09-16T20:51:47.317 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.318 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.318 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:47.318 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757010+0000", 2024-09-16T20:51:47.318 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9900000000000001e-07 2024-09-16T20:51:47.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.318 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.318 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:47.318 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757012+0000", 2024-09-16T20:51:47.318 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2689999999999998e-06 2024-09-16T20:51:47.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.318 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.318 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:47.318 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757022+0000", 2024-09-16T20:51:47.318 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9699999999999994e-06 2024-09-16T20:51:47.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.319 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.319 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:47.319 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757028+0000", 2024-09-16T20:51:47.319 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3799999999999999e-06 2024-09-16T20:51:47.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.319 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.319 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:47.319 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822552+0000", 2024-09-16T20:51:47.319 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065524281000000004 2024-09-16T20:51:47.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.319 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.319 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:47.320 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822613+0000", 2024-09-16T20:51:47.320 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0430999999999998e-05 2024-09-16T20:51:47.320 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.320 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.320 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:47.320 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822753+0000", 2024-09-16T20:51:47.320 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000140226 2024-09-16T20:51:47.320 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.320 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:47.320 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.320 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.320 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.320 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12549 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 633856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:47.320 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:51:47.321 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 116.55683654800001, 2024-09-16T20:51:47.321 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065978518, 2024-09-16T20:51:47.321 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:47.321 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:47.321 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:47.321 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:47.321 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:47.321 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:47.321 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12549 2024-09-16T20:51:47.321 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.321 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:47.321 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.321 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:47.321 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:51:47.321 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.322 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.322 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:47.322 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:51:47.322 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.322 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.322 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:47.322 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757038+0000", 2024-09-16T20:51:47.322 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9299999999999999e-07 2024-09-16T20:51:47.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.322 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.322 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:47.322 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757040+0000", 2024-09-16T20:51:47.323 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.677e-06 2024-09-16T20:51:47.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.323 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.323 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:47.323 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757045+0000", 2024-09-16T20:51:47.323 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8149999999999996e-06 2024-09-16T20:51:47.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.323 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.323 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:47.323 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757048+0000", 2024-09-16T20:51:47.323 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1e-06 2024-09-16T20:51:47.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.323 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.323 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:47.323 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822796+0000", 2024-09-16T20:51:47.324 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065748081 2024-09-16T20:51:47.324 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.324 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.324 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:47.324 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.822867+0000", 2024-09-16T20:51:47.324 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0679000000000005e-05 2024-09-16T20:51:47.324 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.324 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.324 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:47.324 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823016+0000", 2024-09-16T20:51:47.324 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014987299999999999 2024-09-16T20:51:47.324 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.324 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:47.324 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.325 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.325 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.325 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12550 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 638976~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:47.325 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757060+0000", 2024-09-16T20:51:47.325 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 116.556814133, 2024-09-16T20:51:47.325 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066153564999999998, 2024-09-16T20:51:47.325 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:47.325 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:47.325 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:47.325 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:47.325 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:47.325 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:47.325 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12550 2024-09-16T20:51:47.325 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.326 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:47.326 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.326 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:47.326 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757060+0000", 2024-09-16T20:51:47.326 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.326 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.326 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.326 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:47.326 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757060+0000", 2024-09-16T20:51:47.326 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.326 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.326 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.326 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:47.326 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757061+0000", 2024-09-16T20:51:47.326 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.15e-07 2024-09-16T20:51:47.326 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.327 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.327 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:47.327 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757062+0000", 2024-09-16T20:51:47.327 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.663e-06 2024-09-16T20:51:47.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.327 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.327 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:47.327 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757066+0000", 2024-09-16T20:51:47.327 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.817e-06 2024-09-16T20:51:47.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.327 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.327 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:47.327 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757070+0000", 2024-09-16T20:51:47.327 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1509999999999997e-06 2024-09-16T20:51:47.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.328 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.328 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:47.328 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823044+0000", 2024-09-16T20:51:47.328 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065973992999999995 2024-09-16T20:51:47.328 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.328 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.328 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:47.328 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823082+0000", 2024-09-16T20:51:47.328 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8050999999999999e-05 2024-09-16T20:51:47.328 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.328 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.328 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:47.328 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823214+0000", 2024-09-16T20:51:47.328 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013157500000000001 2024-09-16T20:51:47.329 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.329 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:47.329 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.329 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.329 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.329 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12551 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 644096~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:47.329 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:51:47.329 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 116.556800435, 2024-09-16T20:51:47.329 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066391378000000001, 2024-09-16T20:51:47.329 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:47.329 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:47.329 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:47.329 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:47.329 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:47.329 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:47.330 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12551 2024-09-16T20:51:47.330 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.330 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:47.330 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.330 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:47.330 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:51:47.330 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.330 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.330 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.330 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:47.330 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:51:47.330 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.330 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.331 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.331 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:47.331 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757074+0000", 2024-09-16T20:51:47.331 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2999999999999999e-07 2024-09-16T20:51:47.331 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.331 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.331 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:47.331 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757075+0000", 2024-09-16T20:51:47.331 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.097e-06 2024-09-16T20:51:47.331 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.331 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.331 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:47.331 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757082+0000", 2024-09-16T20:51:47.331 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9500000000000004e-06 2024-09-16T20:51:47.331 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.332 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.332 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:47.332 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757086+0000", 2024-09-16T20:51:47.332 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2229999999999999e-06 2024-09-16T20:51:47.332 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.332 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.332 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:47.332 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823262+0000", 2024-09-16T20:51:47.332 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066176078999999999 2024-09-16T20:51:47.332 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.332 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.332 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:47.332 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823336+0000", 2024-09-16T20:51:47.332 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3979000000000004e-05 2024-09-16T20:51:47.332 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.332 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.333 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:47.333 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823465+0000", 2024-09-16T20:51:47.333 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012982 2024-09-16T20:51:47.333 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.333 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:47.333 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.333 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.333 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.333 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12552 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 649216~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:47.333 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.757089+0000", 2024-09-16T20:51:47.333 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 116.55678529399999, 2024-09-16T20:51:47.333 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066655617, 2024-09-16T20:51:47.333 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:47.333 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:47.334 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:47.334 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:47.334 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:47.334 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:47.334 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12552 2024-09-16T20:51:47.334 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.334 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:47.334 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.334 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:47.334 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757089+0000", 2024-09-16T20:51:47.334 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.334 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.334 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.335 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:47.335 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.757089+0000", 2024-09-16T20:51:47.335 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.335 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.335 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.335 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:47.335 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759492+0000", 2024-09-16T20:51:47.335 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0024026109999999998 2024-09-16T20:51:47.335 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.335 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.335 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:47.335 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759497+0000", 2024-09-16T20:51:47.335 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1120000000000004e-06 2024-09-16T20:51:47.335 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.336 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.336 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:47.336 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759508+0000", 2024-09-16T20:51:47.336 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0981e-05 2024-09-16T20:51:47.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.336 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.336 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:47.336 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759516+0000", 2024-09-16T20:51:47.336 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7270000000000007e-06 2024-09-16T20:51:47.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.336 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.336 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:47.336 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823504+0000", 2024-09-16T20:51:47.336 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.063987984999999997 2024-09-16T20:51:47.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.337 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.337 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:47.337 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823579+0000", 2024-09-16T20:51:47.337 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5703000000000003e-05 2024-09-16T20:51:47.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.337 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.337 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:47.337 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.823745+0000", 2024-09-16T20:51:47.337 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016549800000000001 2024-09-16T20:51:47.337 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.337 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:47.337 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.337 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.338 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12560 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 690176~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:47.338 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.759658+0000", 2024-09-16T20:51:47.338 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 116.554216092, 2024-09-16T20:51:47.338 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065858517000000005, 2024-09-16T20:51:47.338 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:47.338 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:47.338 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:47.338 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:47.338 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:47.338 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:47.338 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12560 2024-09-16T20:51:47.338 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.338 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:47.338 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.339 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:47.339 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759658+0000", 2024-09-16T20:51:47.339 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.339 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.339 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.339 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:47.339 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759658+0000", 2024-09-16T20:51:47.339 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.339 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.339 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.339 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:47.339 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759659+0000", 2024-09-16T20:51:47.339 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.17e-07 2024-09-16T20:51:47.340 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.340 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.340 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:47.340 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759660+0000", 2024-09-16T20:51:47.340 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0189999999999999e-06 2024-09-16T20:51:47.340 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.340 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.340 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:47.340 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759663+0000", 2024-09-16T20:51:47.340 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0010000000000002e-06 2024-09-16T20:51:47.340 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.340 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.340 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:47.340 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759665+0000", 2024-09-16T20:51:47.341 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5440000000000001e-06 2024-09-16T20:51:47.341 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.341 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.341 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:47.341 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825355+0000", 2024-09-16T20:51:47.341 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065690224000000005 2024-09-16T20:51:47.341 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.341 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.341 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:47.341 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825394+0000", 2024-09-16T20:51:47.341 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8810999999999998e-05 2024-09-16T20:51:47.341 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.341 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.342 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:47.342 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825517+0000", 2024-09-16T20:51:47.342 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012270099999999999 2024-09-16T20:51:47.342 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.342 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:47.342 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.342 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.342 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.342 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12561 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 695296~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:47.342 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.759668+0000", 2024-09-16T20:51:47.342 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 116.55420624999999, 2024-09-16T20:51:47.342 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066116999999999995, 2024-09-16T20:51:47.342 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:47.343 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:47.343 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:47.343 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:47.343 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:47.343 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:47.343 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12561 2024-09-16T20:51:47.343 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.343 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:47.343 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.343 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:47.343 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759668+0000", 2024-09-16T20:51:47.343 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.343 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.344 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.344 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:47.344 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.759668+0000", 2024-09-16T20:51:47.344 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.344 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.344 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.344 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:47.344 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765379+0000", 2024-09-16T20:51:47.344 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057104410000000001 2024-09-16T20:51:47.344 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.344 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.344 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:47.344 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765383+0000", 2024-09-16T20:51:47.344 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2980000000000002e-06 2024-09-16T20:51:47.344 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.344 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.345 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:47.345 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765395+0000", 2024-09-16T20:51:47.345 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1671000000000001e-05 2024-09-16T20:51:47.345 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.345 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.345 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:47.345 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765403+0000", 2024-09-16T20:51:47.345 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1210000000000007e-06 2024-09-16T20:51:47.345 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.345 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.345 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:47.345 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825552+0000", 2024-09-16T20:51:47.345 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.060149498000000003 2024-09-16T20:51:47.345 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.345 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.346 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:47.346 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825625+0000", 2024-09-16T20:51:47.346 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2540999999999995e-05 2024-09-16T20:51:47.346 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.346 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.346 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:47.346 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.825785+0000", 2024-09-16T20:51:47.346 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016043000000000001 2024-09-16T20:51:47.346 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.346 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:47.346 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.346 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.346 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.346 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12585 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 818176~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:47.346 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.765848+0000", 2024-09-16T20:51:47.346 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 116.548026279, 2024-09-16T20:51:47.347 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065843218999999995, 2024-09-16T20:51:47.347 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:47.347 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:47.347 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:47.347 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:47.347 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:47.347 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:47.347 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12585 2024-09-16T20:51:47.347 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.347 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:47.347 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.347 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:47.347 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765848+0000", 2024-09-16T20:51:47.347 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.347 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.348 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.348 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:47.348 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.765848+0000", 2024-09-16T20:51:47.348 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.348 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.348 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.348 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:47.348 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768683+0000", 2024-09-16T20:51:47.348 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0028344519999999999 2024-09-16T20:51:47.348 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.348 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.348 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:47.348 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768686+0000", 2024-09-16T20:51:47.348 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8789999999999996e-06 2024-09-16T20:51:47.348 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.349 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.349 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:47.349 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768696+0000", 2024-09-16T20:51:47.349 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6709999999999994e-06 2024-09-16T20:51:47.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.349 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.349 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:47.349 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768702+0000", 2024-09-16T20:51:47.349 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2550000000000003e-06 2024-09-16T20:51:47.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.349 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.349 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:47.349 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.831499+0000", 2024-09-16T20:51:47.349 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.062797081000000005 2024-09-16T20:51:47.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.350 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.350 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:47.350 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.831538+0000", 2024-09-16T20:51:47.350 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8523999999999999e-05 2024-09-16T20:51:47.350 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.350 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.350 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:47.350 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.831691+0000", 2024-09-16T20:51:47.350 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015335699999999999 2024-09-16T20:51:47.350 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.350 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:47.350 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.350 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.350 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.350 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12597 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 879616~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:47.350 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.768883+0000", 2024-09-16T20:51:47.351 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 116.544991551, 2024-09-16T20:51:47.351 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065816527, 2024-09-16T20:51:47.351 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:47.351 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:47.351 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:47.351 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:47.351 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:47.351 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:47.351 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12597 2024-09-16T20:51:47.351 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.351 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:47.351 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.351 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:47.351 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768883+0000", 2024-09-16T20:51:47.351 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.351 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.352 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.352 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:47.352 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.768883+0000", 2024-09-16T20:51:47.352 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.352 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.352 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.352 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:47.352 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771508+0000", 2024-09-16T20:51:47.352 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0026247589999999999 2024-09-16T20:51:47.352 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.352 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.352 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:47.352 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771513+0000", 2024-09-16T20:51:47.352 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9949999999999996e-06 2024-09-16T20:51:47.353 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.353 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.353 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:47.353 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771525+0000", 2024-09-16T20:51:47.353 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2033e-05 2024-09-16T20:51:47.353 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.353 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.353 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:47.353 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.771532+0000", 2024-09-16T20:51:47.353 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4220000000000003e-06 2024-09-16T20:51:47.353 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.353 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.353 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:47.353 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.834532+0000", 2024-09-16T20:51:47.353 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.063000159 2024-09-16T20:51:47.353 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.354 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.354 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:47.354 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.834590+0000", 2024-09-16T20:51:47.354 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8218999999999999e-05 2024-09-16T20:51:47.354 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.354 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.354 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:47.354 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.834699+0000", 2024-09-16T20:51:47.354 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010894 2024-09-16T20:51:47.354 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.354 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:47.354 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.354 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.354 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.354 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12619 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 992256~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:47.355 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.774656+0000", 2024-09-16T20:51:47.355 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 116.539218777, 2024-09-16T20:51:47.355 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065715859000000001, 2024-09-16T20:51:47.355 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:47.355 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:47.355 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:47.355 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:47.355 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:47.355 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:47.355 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12619 2024-09-16T20:51:47.355 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.355 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:47.355 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.355 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:47.356 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774656+0000", 2024-09-16T20:51:47.356 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.356 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.356 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.356 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:47.356 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774656+0000", 2024-09-16T20:51:47.356 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.356 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.356 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.356 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:47.356 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774875+0000", 2024-09-16T20:51:47.356 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000218898 2024-09-16T20:51:47.356 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.356 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.357 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:47.357 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774879+0000", 2024-09-16T20:51:47.357 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0389999999999998e-06 2024-09-16T20:51:47.357 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.357 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.357 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:47.357 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774890+0000", 2024-09-16T20:51:47.357 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1250000000000001e-05 2024-09-16T20:51:47.357 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.357 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.357 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:47.357 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774897+0000", 2024-09-16T20:51:47.357 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7619999999999998e-06 2024-09-16T20:51:47.357 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.358 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.358 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:47.358 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840139+0000", 2024-09-16T20:51:47.358 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065242438 2024-09-16T20:51:47.358 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.358 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.358 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:47.358 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840186+0000", 2024-09-16T20:51:47.358 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6894999999999997e-05 2024-09-16T20:51:47.358 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.358 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.358 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:47.358 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840372+0000", 2024-09-16T20:51:47.359 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000185577 2024-09-16T20:51:47.359 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.359 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:47.359 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.359 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.359 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.359 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12620 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 997376~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:47.359 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.774908+0000", 2024-09-16T20:51:47.359 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 116.53896655600001, 2024-09-16T20:51:47.359 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065782754999999998, 2024-09-16T20:51:47.359 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:47.359 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:47.359 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:47.359 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:47.360 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:47.360 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:47.360 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12620 2024-09-16T20:51:47.360 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.360 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:47.360 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.360 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:47.360 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774908+0000", 2024-09-16T20:51:47.360 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.360 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.360 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.360 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:47.360 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.774908+0000", 2024-09-16T20:51:47.360 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.360 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.361 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.361 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:47.361 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775102+0000", 2024-09-16T20:51:47.361 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000194098 2024-09-16T20:51:47.361 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.361 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.361 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:47.361 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775105+0000", 2024-09-16T20:51:47.361 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8739999999999999e-06 2024-09-16T20:51:47.361 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.361 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.361 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:47.361 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775111+0000", 2024-09-16T20:51:47.361 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9519999999999999e-06 2024-09-16T20:51:47.361 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.362 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.362 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:47.362 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775115+0000", 2024-09-16T20:51:47.362 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.002e-06 2024-09-16T20:51:47.362 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.362 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.362 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:47.362 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840409+0000", 2024-09-16T20:51:47.362 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065294195999999999 2024-09-16T20:51:47.362 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.362 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.362 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:47.362 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840507+0000", 2024-09-16T20:51:47.363 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7662000000000001e-05 2024-09-16T20:51:47.363 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.363 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.363 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:47.363 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840691+0000", 2024-09-16T20:51:47.363 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000183971 2024-09-16T20:51:47.363 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.363 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:47.363 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.363 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.363 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.363 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12621 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1002496~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:47.364 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775123+0000", 2024-09-16T20:51:47.364 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 116.53875186, 2024-09-16T20:51:47.364 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065829586999999995, 2024-09-16T20:51:47.364 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:47.364 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:47.364 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:47.364 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:47.364 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:47.364 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:47.364 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12621 2024-09-16T20:51:47.364 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.364 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:47.364 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.365 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:47.365 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775123+0000", 2024-09-16T20:51:47.365 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.365 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.365 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.365 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:47.365 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775123+0000", 2024-09-16T20:51:47.365 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.365 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.365 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.365 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:47.365 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775348+0000", 2024-09-16T20:51:47.365 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000225295 2024-09-16T20:51:47.365 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.366 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.366 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:47.366 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775352+0000", 2024-09-16T20:51:47.366 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0810000000000004e-06 2024-09-16T20:51:47.366 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.366 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.366 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:47.366 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775361+0000", 2024-09-16T20:51:47.366 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7399999999999993e-06 2024-09-16T20:51:47.366 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.366 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.366 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:47.366 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775367+0000", 2024-09-16T20:51:47.366 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5200000000000003e-06 2024-09-16T20:51:47.367 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.367 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.367 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:47.367 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840730+0000", 2024-09-16T20:51:47.367 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065362803999999997 2024-09-16T20:51:47.367 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.367 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.367 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:47.367 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840801+0000", 2024-09-16T20:51:47.367 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1006000000000004e-05 2024-09-16T20:51:47.367 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.367 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.367 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:47.367 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840952+0000", 2024-09-16T20:51:47.367 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015114100000000001 2024-09-16T20:51:47.368 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.368 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:47.368 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.368 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.368 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.368 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12622 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1007616~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:47.368 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775381+0000", 2024-09-16T20:51:47.368 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 116.538493026, 2024-09-16T20:51:47.368 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065822581000000005, 2024-09-16T20:51:47.368 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:47.368 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:47.368 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:47.368 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:47.368 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:47.368 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:47.369 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12622 2024-09-16T20:51:47.369 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.369 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:47.369 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.369 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:47.369 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775381+0000", 2024-09-16T20:51:47.369 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.369 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.369 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.369 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:47.369 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775381+0000", 2024-09-16T20:51:47.369 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.369 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.369 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.369 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:47.370 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775563+0000", 2024-09-16T20:51:47.370 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018153700000000001 2024-09-16T20:51:47.370 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.370 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.370 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:47.370 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775566+0000", 2024-09-16T20:51:47.370 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9620000000000002e-06 2024-09-16T20:51:47.370 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.370 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.370 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:47.370 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775572+0000", 2024-09-16T20:51:47.370 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8529999999999997e-06 2024-09-16T20:51:47.370 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.370 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.370 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:47.371 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775576+0000", 2024-09-16T20:51:47.371 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8920000000000002e-06 2024-09-16T20:51:47.371 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.371 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.371 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:47.371 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.840990+0000", 2024-09-16T20:51:47.371 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065414599000000004 2024-09-16T20:51:47.371 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.371 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.371 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:47.371 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841068+0000", 2024-09-16T20:51:47.371 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8157000000000004e-05 2024-09-16T20:51:47.371 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.371 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.371 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:47.372 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841204+0000", 2024-09-16T20:51:47.372 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013558099999999999 2024-09-16T20:51:47.372 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.372 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:47.372 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.372 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.372 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.372 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12623 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1012736~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:47.372 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775583+0000", 2024-09-16T20:51:47.372 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 116.53829158800001, 2024-09-16T20:51:47.372 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065929087999999997, 2024-09-16T20:51:47.372 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:47.372 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:47.372 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:47.373 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:47.373 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:47.373 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:47.373 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12623 2024-09-16T20:51:47.373 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.373 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:47.373 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.373 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:47.373 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775583+0000", 2024-09-16T20:51:47.373 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.373 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.373 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.373 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:47.373 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775583+0000", 2024-09-16T20:51:47.374 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.374 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.374 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.374 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:47.374 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775804+0000", 2024-09-16T20:51:47.374 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00022116900000000001 2024-09-16T20:51:47.374 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.374 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.374 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:47.374 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775807+0000", 2024-09-16T20:51:47.374 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9529999999999999e-06 2024-09-16T20:51:47.374 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.374 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.374 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:47.374 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775814+0000", 2024-09-16T20:51:47.374 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7490000000000001e-06 2024-09-16T20:51:47.375 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.375 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.375 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:47.375 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775818+0000", 2024-09-16T20:51:47.375 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1999999999999996e-06 2024-09-16T20:51:47.375 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.375 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.375 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:47.375 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841267+0000", 2024-09-16T20:51:47.375 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065449127999999995 2024-09-16T20:51:47.375 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.375 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.375 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:47.375 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841344+0000", 2024-09-16T20:51:47.376 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6546999999999997e-05 2024-09-16T20:51:47.376 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.376 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.376 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:47.376 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841512+0000", 2024-09-16T20:51:47.376 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016834200000000001 2024-09-16T20:51:47.376 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.376 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:47.376 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.376 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.376 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.376 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12624 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1017856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:47.376 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:51:47.376 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 116.538048448, 2024-09-16T20:51:47.376 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065945866000000006, 2024-09-16T20:51:47.376 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:47.377 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:47.377 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:47.377 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:47.377 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:47.377 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:47.377 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12624 2024-09-16T20:51:47.377 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.377 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:47.377 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.377 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:47.377 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:51:47.377 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.377 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.377 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.378 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:47.378 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:51:47.378 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.378 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.378 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.378 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:47.378 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775826+0000", 2024-09-16T20:51:47.378 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-16T20:51:47.378 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.378 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.378 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:47.378 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775828+0000", 2024-09-16T20:51:47.378 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3999999999999999e-06 2024-09-16T20:51:47.379 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.379 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.379 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:47.379 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775831+0000", 2024-09-16T20:51:47.379 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.478e-06 2024-09-16T20:51:47.379 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.379 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.379 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:47.379 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775834+0000", 2024-09-16T20:51:47.379 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9440000000000001e-06 2024-09-16T20:51:47.379 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.379 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.379 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:47.379 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841547+0000", 2024-09-16T20:51:47.379 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065712801000000001 2024-09-16T20:51:47.379 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.380 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.380 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:47.380 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841618+0000", 2024-09-16T20:51:47.380 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1151999999999999e-05 2024-09-16T20:51:47.380 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.380 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.380 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:47.380 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841772+0000", 2024-09-16T20:51:47.380 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000153721 2024-09-16T20:51:47.380 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.380 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:47.380 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.380 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.380 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.380 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12625 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1022976~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:47.381 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.775840+0000", 2024-09-16T20:51:47.381 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 116.53803483900001, 2024-09-16T20:51:47.381 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066165451, 2024-09-16T20:51:47.381 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:47.381 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:47.381 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:47.381 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:47.381 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:47.381 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:47.381 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12625 2024-09-16T20:51:47.381 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.381 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:47.381 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.381 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:47.381 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775840+0000", 2024-09-16T20:51:47.382 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.382 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.382 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.382 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:47.382 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.775840+0000", 2024-09-16T20:51:47.382 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.382 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.382 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.382 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:47.382 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776014+0000", 2024-09-16T20:51:47.382 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017441099999999999 2024-09-16T20:51:47.382 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.382 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.382 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:47.383 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776018+0000", 2024-09-16T20:51:47.383 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6509999999999999e-06 2024-09-16T20:51:47.383 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.383 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.383 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:47.383 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776026+0000", 2024-09-16T20:51:47.383 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.028e-06 2024-09-16T20:51:47.383 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.383 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.383 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:47.383 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776031+0000", 2024-09-16T20:51:47.383 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8400000000000002e-06 2024-09-16T20:51:47.383 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.383 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.384 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:47.384 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841801+0000", 2024-09-16T20:51:47.384 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065770472999999996 2024-09-16T20:51:47.384 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.384 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.384 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:47.384 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.841847+0000", 2024-09-16T20:51:47.384 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5558e-05 2024-09-16T20:51:47.384 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.384 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.384 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:47.384 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842005+0000", 2024-09-16T20:51:47.384 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015849000000000001 2024-09-16T20:51:47.384 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.384 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:47.385 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.385 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.385 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.385 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12626 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1028096~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:47.385 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776041+0000", 2024-09-16T20:51:47.385 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 116.53783353999999, 2024-09-16T20:51:47.385 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066225260999999994, 2024-09-16T20:51:47.385 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:47.385 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:47.385 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:47.385 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:47.385 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:47.385 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:47.385 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12626 2024-09-16T20:51:47.386 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.386 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:47.386 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.386 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:47.386 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776041+0000", 2024-09-16T20:51:47.386 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.386 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.386 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.386 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:47.386 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776041+0000", 2024-09-16T20:51:47.386 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.386 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.386 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.386 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:47.387 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776263+0000", 2024-09-16T20:51:47.387 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00022193899999999999 2024-09-16T20:51:47.387 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.387 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.387 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:47.387 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776268+0000", 2024-09-16T20:51:47.387 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8049999999999997e-06 2024-09-16T20:51:47.387 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.387 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.387 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:47.387 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776280+0000", 2024-09-16T20:51:47.387 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2635e-05 2024-09-16T20:51:47.387 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.387 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.387 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:47.388 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776288+0000", 2024-09-16T20:51:47.388 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8199999999999997e-06 2024-09-16T20:51:47.388 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.388 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.388 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:47.388 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842041+0000", 2024-09-16T20:51:47.388 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065752506000000002 2024-09-16T20:51:47.388 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.388 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.388 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:47.388 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842123+0000", 2024-09-16T20:51:47.388 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2393999999999998e-05 2024-09-16T20:51:47.388 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.389 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.389 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:47.389 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842266+0000", 2024-09-16T20:51:47.389 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014316200000000001 2024-09-16T20:51:47.389 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.389 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:47.389 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.389 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.389 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.389 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12627 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1033216~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:47.389 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776301+0000", 2024-09-16T20:51:47.389 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 116.53757358, 2024-09-16T20:51:47.389 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066187814999999997, 2024-09-16T20:51:47.390 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:47.390 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:47.390 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:47.390 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:47.390 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:47.390 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:47.390 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12627 2024-09-16T20:51:47.390 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.390 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:47.390 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.390 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:47.390 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776301+0000", 2024-09-16T20:51:47.390 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.390 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.391 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.391 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:47.391 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776301+0000", 2024-09-16T20:51:47.391 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.391 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.391 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.391 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:47.391 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776473+0000", 2024-09-16T20:51:47.391 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017222400000000001 2024-09-16T20:51:47.391 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.391 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.391 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:47.391 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776477+0000", 2024-09-16T20:51:47.391 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7019999999999999e-06 2024-09-16T20:51:47.392 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.392 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.392 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:47.392 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776486+0000", 2024-09-16T20:51:47.392 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2119999999999992e-06 2024-09-16T20:51:47.392 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.392 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.392 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:47.392 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776492+0000", 2024-09-16T20:51:47.392 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8089999999999998e-06 2024-09-16T20:51:47.392 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.392 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.392 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:47.393 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842301+0000", 2024-09-16T20:51:47.393 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065809042999999998 2024-09-16T20:51:47.393 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.393 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.393 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:47.393 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842362+0000", 2024-09-16T20:51:47.393 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1260000000000006e-05 2024-09-16T20:51:47.393 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.393 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.393 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:47.393 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842489+0000", 2024-09-16T20:51:47.393 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012656500000000001 2024-09-16T20:51:47.393 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.394 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:47.394 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.394 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.394 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.394 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12628 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1038336~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:47.394 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776502+0000", 2024-09-16T20:51:47.394 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 116.537372934, 2024-09-16T20:51:47.394 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066283426000000006, 2024-09-16T20:51:47.394 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:47.394 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:47.394 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:47.394 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:47.394 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:47.394 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:47.394 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12628 2024-09-16T20:51:47.395 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.395 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:47.395 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.395 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:47.395 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776502+0000", 2024-09-16T20:51:47.395 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.395 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.395 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.395 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:47.395 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776502+0000", 2024-09-16T20:51:47.395 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.395 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.395 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.395 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:47.396 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776716+0000", 2024-09-16T20:51:47.396 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000214596 2024-09-16T20:51:47.396 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.396 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.396 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:47.396 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776723+0000", 2024-09-16T20:51:47.396 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4000000000000003e-06 2024-09-16T20:51:47.396 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.396 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.396 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:47.396 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776731+0000", 2024-09-16T20:51:47.396 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6129999999999998e-06 2024-09-16T20:51:47.396 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.397 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.397 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:47.397 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776737+0000", 2024-09-16T20:51:47.397 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1970000000000001e-06 2024-09-16T20:51:47.397 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.397 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.397 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:47.397 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842532+0000", 2024-09-16T20:51:47.397 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065795127999999994 2024-09-16T20:51:47.397 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.397 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.397 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:47.397 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842599+0000", 2024-09-16T20:51:47.398 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6711000000000006e-05 2024-09-16T20:51:47.398 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.398 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.398 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:47.398 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842785+0000", 2024-09-16T20:51:47.398 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018578099999999999 2024-09-16T20:51:47.398 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.398 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:47.398 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.398 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.398 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.398 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:12629 3.8 3:15ac614e:::rbd_data.110ea93691bb.0000000000000010:23 [read 1043456~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:47.398 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.776748+0000", 2024-09-16T20:51:47.398 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 116.53712655699999, 2024-09-16T20:51:47.399 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066303788000000002, 2024-09-16T20:51:47.399 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:47.399 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:47.399 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:47.399 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:47.399 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:47.399 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:47.399 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12629 2024-09-16T20:51:47.399 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.399 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:47.399 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.399 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:47.399 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776748+0000", 2024-09-16T20:51:47.400 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.400 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.400 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.400 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:47.400 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776748+0000", 2024-09-16T20:51:47.400 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.400 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.400 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.400 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:47.400 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776934+0000", 2024-09-16T20:51:47.400 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018598399999999999 2024-09-16T20:51:47.400 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.400 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.400 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:47.401 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776939+0000", 2024-09-16T20:51:47.401 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1000000000000003e-06 2024-09-16T20:51:47.401 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.401 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.401 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:47.401 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776954+0000", 2024-09-16T20:51:47.401 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5221e-05 2024-09-16T20:51:47.401 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.401 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.401 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:47.401 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.776962+0000", 2024-09-16T20:51:47.401 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.554e-06 2024-09-16T20:51:47.401 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.401 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.402 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:47.402 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842813+0000", 2024-09-16T20:51:47.402 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065851006000000004 2024-09-16T20:51:47.402 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.402 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.402 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:47.402 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.842886+0000", 2024-09-16T20:51:47.402 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3663000000000003e-05 2024-09-16T20:51:47.402 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.402 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.402 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:47.402 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.843052+0000", 2024-09-16T20:51:47.402 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016526 2024-09-16T20:51:47.402 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.402 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:47.402 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.403 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.403 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:47.403 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:47.404 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T20:51:47.512 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:47.512 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:51:47.512 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-16T20:51:47.512 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:47.521 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T20:51:47.628 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:47.629 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-16T20:51:47.629 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-16T20:51:47.629 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-16T20:51:47.629 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:47.637 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T20:51:47.746 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T20:51:47.746 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-16T20:51:47.746 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-16T20:51:47.746 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-16T20:51:47.746 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.746 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5286.0:10643 3.17 3.f6273fb7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-16T20:51:47.746 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:51:47.746 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 158.95379358700001, 2024-09-16T20:51:47.746 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.76672347200000002, 2024-09-16T20:51:47.746 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:47.746 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:47.747 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:47.747 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:47.747 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5286", 2024-09-16T20:51:47.747 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/4135433989", 2024-09-16T20:51:47.747 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 10643 2024-09-16T20:51:47.747 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.747 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:47.747 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.747 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:47.747 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:51:47.747 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.747 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.747 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.747 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:47.748 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790488+0000", 2024-09-16T20:51:47.748 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.748 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.748 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.748 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:47.748 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790489+0000", 2024-09-16T20:51:47.748 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5910000000000001e-06 2024-09-16T20:51:47.748 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.748 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.748 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:47.748 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790493+0000", 2024-09-16T20:51:47.748 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2669999999999998e-06 2024-09-16T20:51:47.748 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.749 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.749 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:47.749 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790500+0000", 2024-09-16T20:51:47.749 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3780000000000003e-06 2024-09-16T20:51:47.749 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.749 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.749 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:47.749 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790505+0000", 2024-09-16T20:51:47.749 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1560000000000003e-06 2024-09-16T20:51:47.749 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.749 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.749 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:47.749 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790559+0000", 2024-09-16T20:51:47.749 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3494e-05 2024-09-16T20:51:47.750 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.750 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.750 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:47.750 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:08.790566+0000", 2024-09-16T20:51:47.750 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7649999999999993e-06 2024-09-16T20:51:47.750 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.750 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.750 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:47.750 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:09.557198+0000", 2024-09-16T20:51:47.750 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.76663136200000004 2024-09-16T20:51:47.750 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.750 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.750 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:47.751 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:09.557211+0000", 2024-09-16T20:51:47.751 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3458999999999999e-05 2024-09-16T20:51:47.751 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.751 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:47.751 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.751 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.751 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.751 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5512.0:450 3.2 3:44e0cffa:::rbd_header.110ef0d7a535:head [watch reconnect cookie 140263298018912 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:51:47.751 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:51:47.751 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 131.18177067400001, 2024-09-16T20:51:47.751 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0084581939999999, 2024-09-16T20:51:47.751 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:47.751 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:47.751 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:47.752 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:47.752 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5512", 2024-09-16T20:51:47.752 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/3827507434", 2024-09-16T20:51:47.752 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 450 2024-09-16T20:51:47.752 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.752 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:47.752 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.752 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:47.752 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:51:47.752 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.752 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.752 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.752 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:47.752 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562511+0000", 2024-09-16T20:51:47.753 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.753 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.753 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.753 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:47.753 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562512+0000", 2024-09-16T20:51:47.753 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.009e-06 2024-09-16T20:51:47.753 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.753 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.753 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:47.753 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562517+0000", 2024-09-16T20:51:47.753 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9339999999999997e-06 2024-09-16T20:51:47.753 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.753 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.754 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:47.754 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562530+0000", 2024-09-16T20:51:47.754 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3495e-05 2024-09-16T20:51:47.754 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.754 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.754 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:47.754 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562536+0000", 2024-09-16T20:51:47.754 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5690000000000004e-06 2024-09-16T20:51:47.754 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.754 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.754 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:47.754 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562591+0000", 2024-09-16T20:51:47.754 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5761999999999999e-05 2024-09-16T20:51:47.754 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.755 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.755 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:51:47.755 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562599+0000", 2024-09-16T20:51:47.755 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9179999999999994e-06 2024-09-16T20:51:47.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.755 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.755 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:47.755 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.565971+0000", 2024-09-16T20:51:47.755 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003371546 2024-09-16T20:51:47.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.755 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.755 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:47.755 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566019+0000", 2024-09-16T20:51:47.755 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7849000000000002e-05 2024-09-16T20:51:47.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.756 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.756 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:47.756 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.569382+0000", 2024-09-16T20:51:47.756 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.003362833 2024-09-16T20:51:47.756 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.756 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.756 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:47.756 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.569529+0000", 2024-09-16T20:51:47.756 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000147823 2024-09-16T20:51:47.756 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.756 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.756 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:47.756 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570969+0000", 2024-09-16T20:51:47.756 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001439456 2024-09-16T20:51:47.756 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.757 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:47.757 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.757 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.757 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.757 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:14996 3.2 3:44e0cffa:::rbd_header.110ef0d7a535:head [watch reconnect cookie 140213436269840 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:51:47.757 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:51:47.757 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 131.18172226199999, 2024-09-16T20:51:47.757 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.008813943, 2024-09-16T20:51:47.757 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:47.757 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:47.757 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:47.757 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:47.757 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:47.757 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:47.757 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 14996 2024-09-16T20:51:47.758 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.758 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:47.758 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.758 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:47.758 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:51:47.758 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.758 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.758 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.758 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:47.758 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562559+0000", 2024-09-16T20:51:47.758 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.758 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.758 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.759 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:47.759 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562560+0000", 2024-09-16T20:51:47.759 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.06e-07 2024-09-16T20:51:47.759 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.759 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.759 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:47.759 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562563+0000", 2024-09-16T20:51:47.759 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.495e-06 2024-09-16T20:51:47.759 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.759 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.759 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:47.759 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562571+0000", 2024-09-16T20:51:47.759 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1950000000000003e-06 2024-09-16T20:51:47.759 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.759 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.760 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:47.760 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562579+0000", 2024-09-16T20:51:47.760 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2649999999999999e-06 2024-09-16T20:51:47.760 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.760 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.760 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:47.760 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562673+0000", 2024-09-16T20:51:47.760 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4118999999999998e-05 2024-09-16T20:51:47.760 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.760 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.760 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:51:47.760 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562677+0000", 2024-09-16T20:51:47.760 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5179999999999998e-06 2024-09-16T20:51:47.761 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.761 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.761 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:47.761 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566461+0000", 2024-09-16T20:51:47.761 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0037837259999999998 2024-09-16T20:51:47.761 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.761 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.761 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:47.761 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566468+0000", 2024-09-16T20:51:47.761 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3350000000000001e-06 2024-09-16T20:51:47.761 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.761 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.762 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:47.762 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570306+0000", 2024-09-16T20:51:47.762 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.003837997 2024-09-16T20:51:47.762 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.762 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.762 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:47.762 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570392+0000", 2024-09-16T20:51:47.762 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5872999999999996e-05 2024-09-16T20:51:47.762 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.762 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.762 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:47.762 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571373+0000", 2024-09-16T20:51:47.762 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00098071399999999993 2024-09-16T20:51:47.762 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.763 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:47.763 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.763 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.763 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.763 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5255.0:1931 3.2 3:47882819:::rbd_data.110eb5f95afc.0000000000000009:head [read 217088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:51:47.763 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:51:47.763 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 131.18168351200001, 2024-09-16T20:51:47.763 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0082752880000001, 2024-09-16T20:51:47.763 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:47.763 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:47.763 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:47.763 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:47.763 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5255", 2024-09-16T20:51:47.764 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:51:47.764 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1931 2024-09-16T20:51:47.764 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.764 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:47.764 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.764 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:47.764 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:51:47.764 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.764 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.764 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.764 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:47.764 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:51:47.764 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.764 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.764 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.765 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:47.765 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562598+0000", 2024-09-16T20:51:47.765 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9699999999999996e-07 2024-09-16T20:51:47.765 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.765 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.765 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:47.765 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562601+0000", 2024-09-16T20:51:47.765 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8600000000000001e-06 2024-09-16T20:51:47.765 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.765 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.765 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:47.765 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562608+0000", 2024-09-16T20:51:47.765 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9e-06 2024-09-16T20:51:47.765 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.765 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.766 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:47.766 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562615+0000", 2024-09-16T20:51:47.766 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8179999999999998e-06 2024-09-16T20:51:47.766 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.766 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.766 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:47.766 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562703+0000", 2024-09-16T20:51:47.766 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7811000000000003e-05 2024-09-16T20:51:47.766 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.766 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.766 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-16T20:51:47.766 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562706+0000", 2024-09-16T20:51:47.766 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4920000000000002e-06 2024-09-16T20:51:47.766 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.767 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.767 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:47.767 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566685+0000", 2024-09-16T20:51:47.767 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0039792780000000002 2024-09-16T20:51:47.767 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.767 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.767 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:47.767 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566693+0000", 2024-09-16T20:51:47.767 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4730000000000003e-06 2024-09-16T20:51:47.767 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.767 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.767 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:47.767 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570737+0000", 2024-09-16T20:51:47.767 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0040436749999999 2024-09-16T20:51:47.767 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.767 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.768 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:47.768 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.570873+0000", 2024-09-16T20:51:47.768 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000136484 2024-09-16T20:51:47.768 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.768 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:47.768 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.768 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.768 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.768 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5255.0:1939 3.2 3:47882819:::rbd_data.110eb5f95afc.0000000000000009:head [read 233472~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T20:51:47.768 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:51:47.768 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 131.181528646, 2024-09-16T20:51:47.768 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0084657589999999, 2024-09-16T20:51:47.768 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:47.769 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:47.769 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:47.769 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:47.769 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5255", 2024-09-16T20:51:47.769 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/951241811", 2024-09-16T20:51:47.769 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1939 2024-09-16T20:51:47.769 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.769 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:47.769 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.769 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:47.769 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:51:47.769 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.769 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.769 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.769 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:47.770 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:51:47.770 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.770 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.770 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.770 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:47.770 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562753+0000", 2024-09-16T20:51:47.770 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5199999999999997e-07 2024-09-16T20:51:47.770 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.770 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.770 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:47.770 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562757+0000", 2024-09-16T20:51:47.770 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4979999999999998e-06 2024-09-16T20:51:47.770 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.770 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.771 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:47.771 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562764+0000", 2024-09-16T20:51:47.771 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1470000000000004e-06 2024-09-16T20:51:47.771 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.771 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.771 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:47.771 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562768+0000", 2024-09-16T20:51:47.771 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7929999999999999e-06 2024-09-16T20:51:47.771 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.771 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.771 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:47.771 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562805+0000", 2024-09-16T20:51:47.771 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7477e-05 2024-09-16T20:51:47.771 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.771 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.771 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting_for_map not empty", 2024-09-16T20:51:47.772 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.562810+0000", 2024-09-16T20:51:47.772 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0819999999999998e-06 2024-09-16T20:51:47.772 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.772 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.772 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:47.772 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566717+0000", 2024-09-16T20:51:47.772 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0039072600000000001 2024-09-16T20:51:47.772 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.772 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.772 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:47.772 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:36.566724+0000", 2024-09-16T20:51:47.772 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4880000000000004e-06 2024-09-16T20:51:47.772 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.772 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.773 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:47.773 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571009+0000", 2024-09-16T20:51:47.773 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.004284677 2024-09-16T20:51:47.773 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.773 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.773 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:47.773 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:37.571218+0000", 2024-09-16T20:51:47.773 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00020978500000000001 2024-09-16T20:51:47.773 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.773 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:47.773 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.773 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.773 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.773 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5799.0:11021 3.1f 3:fee2d1dd:::rbd_data.110ea93691bb.000000000000000e:23 [read 960512~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-16T20:51:47.773 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:49:50.693837+0000", 2024-09-16T20:51:47.774 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 117.050444101, 2024-09-16T20:51:47.774 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057155138000000001, 2024-09-16T20:51:47.774 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:47.774 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:47.774 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:47.774 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:47.774 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5799", 2024-09-16T20:51:47.774 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1448674546", 2024-09-16T20:51:47.774 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 11021 2024-09-16T20:51:47.774 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.774 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:47.774 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.774 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:47.774 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693837+0000", 2024-09-16T20:51:47.774 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.775 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.775 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.775 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:47.775 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.693837+0000", 2024-09-16T20:51:47.775 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.775 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.775 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.775 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:47.775 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699566+0000", 2024-09-16T20:51:47.775 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057283639999999997 2024-09-16T20:51:47.775 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.775 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.775 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:47.775 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699571+0000", 2024-09-16T20:51:47.776 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.604e-06 2024-09-16T20:51:47.776 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.776 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.776 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:47.776 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699583+0000", 2024-09-16T20:51:47.776 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1372999999999999e-05 2024-09-16T20:51:47.776 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.776 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.776 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:47.776 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.699591+0000", 2024-09-16T20:51:47.776 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5259999999999993e-06 2024-09-16T20:51:47.776 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.776 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.777 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:47.777 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750674+0000", 2024-09-16T20:51:47.777 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051083154999999998 2024-09-16T20:51:47.777 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.777 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.777 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:47.777 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750745+0000", 2024-09-16T20:51:47.777 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0933999999999995e-05 2024-09-16T20:51:47.777 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.777 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.777 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:47.777 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:49:50.750992+0000", 2024-09-16T20:51:47.777 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00024718199999999998 2024-09-16T20:51:47.777 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.778 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:47.778 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.778 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.778 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.778 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8792 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2609152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:47.778 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:51:47.778 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 106.686984384, 2024-09-16T20:51:47.778 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057003524999999999, 2024-09-16T20:51:47.778 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:47.778 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:47.778 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:47.778 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:47.778 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:47.779 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:47.779 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8792 2024-09-16T20:51:47.779 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.779 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:47.779 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.779 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:47.779 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:51:47.779 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.779 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.779 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.779 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:47.779 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:51:47.779 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.779 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.780 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.780 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:47.780 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057297+0000", 2024-09-16T20:51:47.780 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0400000000000002e-07 2024-09-16T20:51:47.780 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.780 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.780 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:47.780 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057299+0000", 2024-09-16T20:51:47.780 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7460000000000001e-06 2024-09-16T20:51:47.780 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.780 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.780 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:47.780 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057304+0000", 2024-09-16T20:51:47.781 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3210000000000004e-06 2024-09-16T20:51:47.781 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.781 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.781 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:47.781 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057310+0000", 2024-09-16T20:51:47.781 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4360000000000001e-06 2024-09-16T20:51:47.781 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.781 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.781 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:47.781 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114200+0000", 2024-09-16T20:51:47.781 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056889954 2024-09-16T20:51:47.781 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.781 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.781 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:47.782 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114243+0000", 2024-09-16T20:51:47.782 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2998999999999999e-05 2024-09-16T20:51:47.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.782 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.782 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:47.782 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114300+0000", 2024-09-16T20:51:47.782 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7765e-05 2024-09-16T20:51:47.782 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.782 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:47.782 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.782 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.782 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8794 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2351104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:47.782 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057327+0000", 2024-09-16T20:51:47.783 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 106.68695406000001, 2024-09-16T20:51:47.783 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057101130999999999, 2024-09-16T20:51:47.783 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:47.783 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:47.783 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:47.783 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:47.783 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:47.783 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:47.783 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8794 2024-09-16T20:51:47.783 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.783 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:47.783 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.783 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:47.783 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057327+0000", 2024-09-16T20:51:47.784 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.784 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.784 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.784 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:47.784 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057327+0000", 2024-09-16T20:51:47.784 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.784 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.784 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.784 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:47.784 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057328+0000", 2024-09-16T20:51:47.784 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7399999999999999e-07 2024-09-16T20:51:47.784 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.784 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.784 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:47.785 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057330+0000", 2024-09-16T20:51:47.785 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2199999999999999e-06 2024-09-16T20:51:47.785 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.785 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.785 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:47.785 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057336+0000", 2024-09-16T20:51:47.785 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1240000000000002e-06 2024-09-16T20:51:47.785 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.785 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.785 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:47.785 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057341+0000", 2024-09-16T20:51:47.785 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8470000000000003e-06 2024-09-16T20:51:47.785 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.786 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:47.786 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114334+0000", 2024-09-16T20:51:47.786 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056993267 2024-09-16T20:51:47.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.786 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:47.786 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114376+0000", 2024-09-16T20:51:47.786 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1638999999999999e-05 2024-09-16T20:51:47.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.786 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:47.786 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114428+0000", 2024-09-16T20:51:47.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2660000000000001e-05 2024-09-16T20:51:47.787 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.787 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:47.787 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.787 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.787 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8798 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2297856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:47.787 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:51:47.787 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 106.686934333, 2024-09-16T20:51:47.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057277041000000001, 2024-09-16T20:51:47.787 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:47.787 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:47.787 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:47.787 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:47.788 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:47.788 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:47.788 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8798 2024-09-16T20:51:47.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.788 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:47.788 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.788 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:47.788 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:51:47.788 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.788 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.788 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:47.788 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:51:47.789 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.789 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.789 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:47.789 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057347+0000", 2024-09-16T20:51:47.789 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2800000000000003e-07 2024-09-16T20:51:47.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.789 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.789 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:47.789 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057354+0000", 2024-09-16T20:51:47.789 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7780000000000002e-06 2024-09-16T20:51:47.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.789 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.790 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:47.790 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057360+0000", 2024-09-16T20:51:47.790 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0419999999999999e-06 2024-09-16T20:51:47.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.790 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.790 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:47.790 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057365+0000", 2024-09-16T20:51:47.790 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7929999999999997e-06 2024-09-16T20:51:47.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.790 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.790 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:47.790 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114460+0000", 2024-09-16T20:51:47.790 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057094662999999997 2024-09-16T20:51:47.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.791 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.791 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:47.791 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114532+0000", 2024-09-16T20:51:47.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2881999999999998e-05 2024-09-16T20:51:47.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.791 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.791 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:47.791 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114624+0000", 2024-09-16T20:51:47.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1555e-05 2024-09-16T20:51:47.791 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.791 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:47.791 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.791 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.792 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8800 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2316288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:47.792 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:51:47.792 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 106.68691039399999, 2024-09-16T20:51:47.792 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.05746946, 2024-09-16T20:51:47.792 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:47.792 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:47.792 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:47.792 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:47.792 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:47.792 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:47.792 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8800 2024-09-16T20:51:47.792 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.792 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:47.792 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.793 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:47.793 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:51:47.793 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.793 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.793 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:47.793 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:51:47.793 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.793 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.793 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:47.793 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057371+0000", 2024-09-16T20:51:47.793 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5199999999999998e-07 2024-09-16T20:51:47.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.794 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:47.794 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057373+0000", 2024-09-16T20:51:47.794 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.79e-06 2024-09-16T20:51:47.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.794 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:47.794 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057379+0000", 2024-09-16T20:51:47.794 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6699999999999999e-06 2024-09-16T20:51:47.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.794 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:47.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057386+0000", 2024-09-16T20:51:47.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3170000000000004e-06 2024-09-16T20:51:47.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.795 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.795 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:47.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114655+0000", 2024-09-16T20:51:47.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057269420000000001 2024-09-16T20:51:47.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.795 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.795 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:47.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114734+0000", 2024-09-16T20:51:47.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8084000000000007e-05 2024-09-16T20:51:47.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.795 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.796 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:47.796 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114840+0000", 2024-09-16T20:51:47.796 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000106827 2024-09-16T20:51:47.796 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.796 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:47.796 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.796 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.796 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8801 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2334720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:47.796 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:51:47.796 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 106.68688936300001, 2024-09-16T20:51:47.796 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057594682000000001, 2024-09-16T20:51:47.796 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:47.796 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:47.797 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:47.797 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:47.797 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:47.797 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:47.797 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8801 2024-09-16T20:51:47.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.797 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:47.797 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.797 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:47.797 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:51:47.797 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.797 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.798 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:47.798 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:51:47.798 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.798 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.798 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:47.798 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057392+0000", 2024-09-16T20:51:47.798 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6899999999999998e-07 2024-09-16T20:51:47.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.798 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.798 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:47.798 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057394+0000", 2024-09-16T20:51:47.798 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8300000000000001e-06 2024-09-16T20:51:47.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.799 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.799 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:47.799 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057400+0000", 2024-09-16T20:51:47.799 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6500000000000001e-06 2024-09-16T20:51:47.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.799 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.799 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:47.799 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057404+0000", 2024-09-16T20:51:47.799 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4669999999999998e-06 2024-09-16T20:51:47.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.799 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.799 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:47.799 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114874+0000", 2024-09-16T20:51:47.799 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057469602000000002 2024-09-16T20:51:47.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.800 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:47.800 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114933+0000", 2024-09-16T20:51:47.800 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9500000000000003e-05 2024-09-16T20:51:47.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.800 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:47.800 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.114987+0000", 2024-09-16T20:51:47.800 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3263999999999997e-05 2024-09-16T20:51:47.800 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.800 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:47.800 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.801 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.801 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8802 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2353152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:47.801 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:51:47.801 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 106.686840678, 2024-09-16T20:51:47.801 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057714079000000001, 2024-09-16T20:51:47.801 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:47.801 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:47.801 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:47.801 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:47.801 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:47.801 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:47.801 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8802 2024-09-16T20:51:47.801 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.801 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:47.802 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.802 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:47.802 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:51:47.802 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.802 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.802 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.802 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:47.802 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:51:47.802 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.802 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.802 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.802 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:47.802 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057441+0000", 2024-09-16T20:51:47.803 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4999999999999998e-07 2024-09-16T20:51:47.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.803 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.803 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:47.803 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057446+0000", 2024-09-16T20:51:47.803 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4240000000000003e-06 2024-09-16T20:51:47.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.803 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.803 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:47.803 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057452+0000", 2024-09-16T20:51:47.803 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8379999999999996e-06 2024-09-16T20:51:47.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.803 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.804 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:47.804 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057460+0000", 2024-09-16T20:51:47.804 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9629999999999998e-06 2024-09-16T20:51:47.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.804 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:47.804 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115015+0000", 2024-09-16T20:51:47.804 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057554389999999997 2024-09-16T20:51:47.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.804 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:47.804 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115082+0000", 2024-09-16T20:51:47.804 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7733999999999999e-05 2024-09-16T20:51:47.805 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.805 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.805 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:47.805 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115155+0000", 2024-09-16T20:51:47.805 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2280000000000006e-05 2024-09-16T20:51:47.805 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.805 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:47.805 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.805 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.805 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.805 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8804 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2555904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:47.805 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:51:47.805 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 106.68681436200001, 2024-09-16T20:51:47.806 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057833862, 2024-09-16T20:51:47.806 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:47.806 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:47.806 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:47.806 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:47.806 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:47.806 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:47.806 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8804 2024-09-16T20:51:47.806 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.806 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:47.806 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.806 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:47.806 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:51:47.806 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.806 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.807 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.807 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:47.807 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:51:47.807 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.807 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.807 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.807 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:47.807 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057467+0000", 2024-09-16T20:51:47.807 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7899999999999999e-07 2024-09-16T20:51:47.807 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.807 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.807 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:47.807 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057469+0000", 2024-09-16T20:51:47.807 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.001e-06 2024-09-16T20:51:47.807 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.808 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.808 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:47.808 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057475+0000", 2024-09-16T20:51:47.808 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7629999999999997e-06 2024-09-16T20:51:47.808 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.808 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.808 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:47.808 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057480+0000", 2024-09-16T20:51:47.808 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8069999999999999e-06 2024-09-16T20:51:47.808 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.808 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.808 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:47.808 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115183+0000", 2024-09-16T20:51:47.808 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057703364 2024-09-16T20:51:47.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.809 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.809 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:47.809 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115227+0000", 2024-09-16T20:51:47.809 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3903999999999998e-05 2024-09-16T20:51:47.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.809 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.809 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:47.809 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115301+0000", 2024-09-16T20:51:47.809 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3543999999999999e-05 2024-09-16T20:51:47.809 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.809 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:47.810 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.810 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.810 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8805 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2574336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:47.810 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057487+0000", 2024-09-16T20:51:47.810 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 106.686794034, 2024-09-16T20:51:47.810 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058021027000000003, 2024-09-16T20:51:47.810 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:47.810 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:47.810 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:47.810 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:47.810 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:47.810 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:47.810 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8805 2024-09-16T20:51:47.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.811 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:47.811 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.811 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:47.811 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057487+0000", 2024-09-16T20:51:47.811 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.811 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.811 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.811 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:47.811 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057487+0000", 2024-09-16T20:51:47.811 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.811 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.811 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.811 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:47.811 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057488+0000", 2024-09-16T20:51:47.811 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5199999999999998e-07 2024-09-16T20:51:47.811 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.812 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.812 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:47.812 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057490+0000", 2024-09-16T20:51:47.812 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.939e-06 2024-09-16T20:51:47.812 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.812 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.812 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:47.812 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057495+0000", 2024-09-16T20:51:47.812 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6439999999999997e-06 2024-09-16T20:51:47.812 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.812 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.812 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:47.812 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057500+0000", 2024-09-16T20:51:47.813 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8119999999999998e-06 2024-09-16T20:51:47.813 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.813 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.813 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:47.813 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115341+0000", 2024-09-16T20:51:47.813 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.057841341999999997 2024-09-16T20:51:47.813 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.813 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.813 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:47.813 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115422+0000", 2024-09-16T20:51:47.813 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1003999999999999e-05 2024-09-16T20:51:47.813 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.813 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.813 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:47.814 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115508+0000", 2024-09-16T20:51:47.814 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5933999999999994e-05 2024-09-16T20:51:47.814 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.814 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:47.814 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.814 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.814 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.814 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8807 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2592768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:47.814 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057505+0000", 2024-09-16T20:51:47.814 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 106.68677582399999, 2024-09-16T20:51:47.814 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058186633000000001, 2024-09-16T20:51:47.814 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:47.814 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:47.815 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:47.815 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:47.815 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:47.815 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:47.815 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8807 2024-09-16T20:51:47.815 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.815 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:47.815 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.815 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:47.815 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057505+0000", 2024-09-16T20:51:47.815 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.815 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.815 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.815 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:47.815 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057505+0000", 2024-09-16T20:51:47.815 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.816 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.816 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.816 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:47.816 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057506+0000", 2024-09-16T20:51:47.816 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.27e-07 2024-09-16T20:51:47.816 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.816 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.816 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:47.816 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057508+0000", 2024-09-16T20:51:47.816 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7400000000000001e-06 2024-09-16T20:51:47.816 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.816 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.817 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:47.817 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057513+0000", 2024-09-16T20:51:47.817 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.468e-06 2024-09-16T20:51:47.817 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.817 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.817 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:47.817 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057517+0000", 2024-09-16T20:51:47.817 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4329999999999997e-06 2024-09-16T20:51:47.817 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.817 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.817 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:47.817 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115538+0000", 2024-09-16T20:51:47.817 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058020355000000003 2024-09-16T20:51:47.817 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.818 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.818 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:47.818 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115616+0000", 2024-09-16T20:51:47.818 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7998999999999996e-05 2024-09-16T20:51:47.818 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.818 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.818 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:47.818 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115692+0000", 2024-09-16T20:51:47.818 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6310999999999995e-05 2024-09-16T20:51:47.818 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.818 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:47.818 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.818 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.818 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.818 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.5752.0:8808 3.0 3:00eb20d3:::rbd_data.110eb19ad55b.0000000000000004:25 [read 2611200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T20:51:47.819 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:51:47.819 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 106.686758563, 2024-09-16T20:51:47.819 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058354046, 2024-09-16T20:51:47.819 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:47.819 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:47.819 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:47.819 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:47.819 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.5752", 2024-09-16T20:51:47.819 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/134424556", 2024-09-16T20:51:47.819 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8808 2024-09-16T20:51:47.819 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.819 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:47.819 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.820 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:47.820 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:51:47.820 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.820 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.820 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.820 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:47.820 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:51:47.820 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.820 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.820 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.820 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:47.820 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057523+0000", 2024-09-16T20:51:47.820 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2500000000000001e-07 2024-09-16T20:51:47.820 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.821 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.821 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:47.821 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057525+0000", 2024-09-16T20:51:47.821 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5549999999999999e-06 2024-09-16T20:51:47.821 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.821 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.821 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:47.821 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057530+0000", 2024-09-16T20:51:47.821 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5489999999999997e-06 2024-09-16T20:51:47.821 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.821 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.821 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:47.821 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.057535+0000", 2024-09-16T20:51:47.822 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5709999999999999e-06 2024-09-16T20:51:47.822 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.822 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.822 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:47.822 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115722+0000", 2024-09-16T20:51:47.822 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058187323999999999 2024-09-16T20:51:47.822 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.822 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.822 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:47.822 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115799+0000", 2024-09-16T20:51:47.822 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6445000000000004e-05 2024-09-16T20:51:47.822 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.822 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.822 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:47.823 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:01.115877+0000", 2024-09-16T20:51:47.823 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8276999999999996e-05 2024-09-16T20:51:47.823 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.823 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:47.823 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.823 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.823 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.823 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.6231.0:2182 3.17 3:edb5e5cd:::rbd_data.110e61479aee.0000000000000014:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write 0~5120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-16T20:51:47.823 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:22.734609+0000", 2024-09-16T20:51:47.823 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 85.009672316000007, 2024-09-16T20:51:47.823 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.124890894, 2024-09-16T20:51:47.823 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:47.823 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:47.823 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:47.824 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:47.824 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.6231", 2024-09-16T20:51:47.824 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/1062987766", 2024-09-16T20:51:47.824 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 2182 2024-09-16T20:51:47.824 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.824 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:47.824 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.824 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:47.824 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734609+0000", 2024-09-16T20:51:47.824 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.824 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.824 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.824 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:47.825 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734609+0000", 2024-09-16T20:51:47.825 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.825 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.825 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.825 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:47.825 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734611+0000", 2024-09-16T20:51:47.825 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9599999999999999e-06 2024-09-16T20:51:47.825 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.825 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.825 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:47.825 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734628+0000", 2024-09-16T20:51:47.825 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.715e-05 2024-09-16T20:51:47.825 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.825 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.826 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:47.826 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734656+0000", 2024-09-16T20:51:47.826 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7759999999999998e-05 2024-09-16T20:51:47.826 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.826 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.826 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:47.826 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734669+0000", 2024-09-16T20:51:47.826 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3315999999999999e-05 2024-09-16T20:51:47.826 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.826 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.826 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:47.826 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.734826+0000", 2024-09-16T20:51:47.826 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015668499999999999 2024-09-16T20:51:47.826 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.827 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.827 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:47.827 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.735048+0000", 2024-09-16T20:51:47.827 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00022179 2024-09-16T20:51:47.827 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.827 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.827 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:47.827 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.735248+0000", 2024-09-16T20:51:47.827 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00020010399999999999 2024-09-16T20:51:47.827 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.827 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.827 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:51:47.827 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.736252+0000", 2024-09-16T20:51:47.828 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010039739999999999 2024-09-16T20:51:47.828 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.828 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.828 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:47.828 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.859340+0000", 2024-09-16T20:51:47.828 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.12308846599999999 2024-09-16T20:51:47.828 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.828 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.828 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:51:47.828 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.859431+0000", 2024-09-16T20:51:47.828 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1099000000000003e-05 2024-09-16T20:51:47.828 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.828 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.829 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:47.829 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:22.859500+0000", 2024-09-16T20:51:47.829 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8590000000000006e-05 2024-09-16T20:51:47.829 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.829 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:47.829 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.829 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.829 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.829 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:29157 3.1b 3:dbe69130:::journal.110e1b19a241:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e167)", 2024-09-16T20:51:47.829 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:50:54.663540+0000", 2024-09-16T20:51:47.829 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 53.080740978000001, 2024-09-16T20:51:47.829 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.92912227999999997, 2024-09-16T20:51:47.829 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:47.829 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:47.830 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-16T20:51:47.830 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:47.830 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:47.830 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:47.830 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 29157 2024-09-16T20:51:47.830 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.830 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:47.830 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.830 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:47.830 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663540+0000", 2024-09-16T20:51:47.830 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.830 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.830 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.830 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:47.831 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663540+0000", 2024-09-16T20:51:47.831 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.831 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.831 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.831 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:47.831 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663541+0000", 2024-09-16T20:51:47.831 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1680000000000001e-06 2024-09-16T20:51:47.831 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.831 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.831 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:47.831 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663546+0000", 2024-09-16T20:51:47.831 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7129999999999996e-06 2024-09-16T20:51:47.831 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.831 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.831 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:47.832 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663558+0000", 2024-09-16T20:51:47.832 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1347e-05 2024-09-16T20:51:47.832 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.832 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.832 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:47.832 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663563+0000", 2024-09-16T20:51:47.832 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1250000000000001e-06 2024-09-16T20:51:47.832 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.832 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.832 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:47.832 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663612+0000", 2024-09-16T20:51:47.832 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9665e-05 2024-09-16T20:51:47.832 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.832 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.833 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:47.833 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:54.663620+0000", 2024-09-16T20:51:47.833 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8439999999999998e-06 2024-09-16T20:51:47.833 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.833 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.833 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:47.833 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:55.592525+0000", 2024-09-16T20:51:47.833 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.92890456700000001 2024-09-16T20:51:47.833 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.833 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.833 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:47.833 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:50:55.592663+0000", 2024-09-16T20:51:47.833 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013785100000000001 2024-09-16T20:51:47.833 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.834 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:47.834 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.834 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.834 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.834 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:30229 3.16 3:6f5476df:::rbd_header.110e25db1f60:head [watch reconnect cookie 140213637350016 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e181)", 2024-09-16T20:51:47.834 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:51:08.603124+0000", 2024-09-16T20:51:47.834 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 39.141157333000002, 2024-09-16T20:51:47.834 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.97765519899999997, 2024-09-16T20:51:47.834 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:47.834 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:47.834 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-16T20:51:47.834 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:47.834 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:47.835 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:47.835 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30229 2024-09-16T20:51:47.835 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.835 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:47.835 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.835 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:47.835 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603124+0000", 2024-09-16T20:51:47.835 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.835 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.835 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.835 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:47.835 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603124+0000", 2024-09-16T20:51:47.835 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.835 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.836 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.836 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:47.836 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603125+0000", 2024-09-16T20:51:47.836 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5090000000000001e-06 2024-09-16T20:51:47.836 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.836 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.836 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:47.836 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603131+0000", 2024-09-16T20:51:47.836 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9089999999999997e-06 2024-09-16T20:51:47.836 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.836 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.836 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:47.836 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603143+0000", 2024-09-16T20:51:47.836 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1416e-05 2024-09-16T20:51:47.837 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.837 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.837 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:47.837 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603151+0000", 2024-09-16T20:51:47.837 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7889999999999995e-06 2024-09-16T20:51:47.837 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.837 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.837 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:47.837 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603674+0000", 2024-09-16T20:51:47.837 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00052357 2024-09-16T20:51:47.837 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.837 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.837 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:47.837 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603684+0000", 2024-09-16T20:51:47.838 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0274e-05 2024-09-16T20:51:47.838 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.838 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.838 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:47.838 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.575380+0000", 2024-09-16T20:51:47.838 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.97169509799999998 2024-09-16T20:51:47.838 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.838 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.838 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:47.838 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.575591+0000", 2024-09-16T20:51:47.838 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00021126700000000001 2024-09-16T20:51:47.838 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.838 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.839 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:47.839 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.580779+0000", 2024-09-16T20:51:47.839 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0051883670000000002 2024-09-16T20:51:47.839 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.839 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:47.839 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.839 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.839 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.839 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4366.0:30230 3.16 3:6f5476df:::rbd_header.110e25db1f60:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e181)", 2024-09-16T20:51:47.839 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-16T20:51:08.603204+0000", 2024-09-16T20:51:47.839 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 39.141077385000003, 2024-09-16T20:51:47.839 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.98129583300000001, 2024-09-16T20:51:47.839 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-16T20:51:47.840 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-16T20:51:47.840 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T20:51:47.840 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-16T20:51:47.840 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4366", 2024-09-16T20:51:47.840 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.139:0/2473492414", 2024-09-16T20:51:47.840 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30230 2024-09-16T20:51:47.840 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.840 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-16T20:51:47.840 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.840 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-16T20:51:47.840 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603204+0000", 2024-09-16T20:51:47.840 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.840 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.841 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.841 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-16T20:51:47.841 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603204+0000", 2024-09-16T20:51:47.841 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-16T20:51:47.841 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.841 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.841 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-16T20:51:47.841 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603205+0000", 2024-09-16T20:51:47.841 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4900000000000005e-07 2024-09-16T20:51:47.841 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.841 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.841 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-16T20:51:47.842 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603209+0000", 2024-09-16T20:51:47.842 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5210000000000004e-06 2024-09-16T20:51:47.842 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.842 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.842 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-16T20:51:47.842 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603222+0000", 2024-09-16T20:51:47.842 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2733e-05 2024-09-16T20:51:47.842 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.842 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.842 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-16T20:51:47.842 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603228+0000", 2024-09-16T20:51:47.842 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4520000000000002e-06 2024-09-16T20:51:47.842 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.843 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.843 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:47.843 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603719+0000", 2024-09-16T20:51:47.843 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00049103600000000003 2024-09-16T20:51:47.843 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.843 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.843 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-16T20:51:47.843 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:08.603730+0000", 2024-09-16T20:51:47.843 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.023e-05 2024-09-16T20:51:47.843 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.843 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.843 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-16T20:51:47.843 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.577394+0000", 2024-09-16T20:51:47.843 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.97366440099999996 2024-09-16T20:51:47.844 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.844 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.844 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-16T20:51:47.844 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.577499+0000", 2024-09-16T20:51:47.844 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000104509 2024-09-16T20:51:47.844 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.844 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.844 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-16T20:51:47.844 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.577920+0000", 2024-09-16T20:51:47.844 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00042173699999999999 2024-09-16T20:51:47.844 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.844 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.844 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-16T20:51:47.845 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.580795+0000", 2024-09-16T20:51:47.845 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.002874302 2024-09-16T20:51:47.845 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.845 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.845 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-16T20:51:47.845 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.584423+0000", 2024-09-16T20:51:47.845 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0036280010000000001 2024-09-16T20:51:47.845 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.845 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.845 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-16T20:51:47.845 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.584465+0000", 2024-09-16T20:51:47.845 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2590999999999999e-05 2024-09-16T20:51:47.846 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T20:51:47.846 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-16T20:51:47.846 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-16T20:51:47.846 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-16T20:51:09.584500+0000", 2024-09-16T20:51:47.846 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4570999999999998e-05 2024-09-16T20:51:47.846 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.846 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:47.846 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.846 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T20:51:47.846 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T20:51:47.846 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T20:51:48.591 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T20:51:49.574 INFO:teuthology.orchestra.run.smithi029.stderr:noscrub is unset 2024-09-16T20:51:49.592 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T20:51:50.595 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-16T20:51:50.685 INFO:teuthology.orchestra.run.smithi029.stderr:nodeep-scrub is unset 2024-09-16T20:51:50.725 INFO:teuthology.orchestra.run.smithi029.stdout:mon.a: {} 2024-09-16T20:51:50.726 INFO:teuthology.orchestra.run.smithi029.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-16T20:51:50.735 INFO:teuthology.orchestra.run.smithi029.stdout:mon.b: {} 2024-09-16T20:51:50.735 INFO:teuthology.orchestra.run.smithi029.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-16T20:51:50.744 INFO:teuthology.orchestra.run.smithi029.stdout:mon.c: {} 2024-09-16T20:51:50.744 INFO:teuthology.orchestra.run.smithi029.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-16T20:51:50.755 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-16T20:51:51.687 INFO:teuthology.orchestra.run.smithi029.stderr:reweighted osd.0 to 1 (10000) 2024-09-16T20:51:51.704 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-16T20:51:52.696 INFO:teuthology.orchestra.run.smithi029.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-16T20:51:52.713 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-16T20:51:53.698 INFO:teuthology.orchestra.run.smithi029.stderr:reweighted osd.1 to 1 (10000) 2024-09-16T20:51:53.717 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-16T20:51:54.578 INFO:teuthology.orchestra.run.smithi029.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-16T20:51:54.596 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-16T20:51:55.577 INFO:teuthology.orchestra.run.smithi029.stderr:reweighted osd.2 to 1 (10000) 2024-09-16T20:51:55.595 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-16T20:51:56.712 INFO:teuthology.orchestra.run.smithi029.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-16T20:51:56.731 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-16T20:51:57.713 INFO:teuthology.orchestra.run.smithi029.stderr:reweighted osd.3 to 1 (10000) 2024-09-16T20:51:57.732 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-16T20:51:58.721 INFO:teuthology.orchestra.run.smithi029.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-16T20:51:58.738 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-16T20:51:59.723 INFO:teuthology.orchestra.run.smithi029.stderr:reweighted osd.4 to 1 (10000) 2024-09-16T20:51:59.739 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-16T20:52:00.728 INFO:teuthology.orchestra.run.smithi029.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-16T20:52:00.745 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-16T20:52:01.737 INFO:teuthology.orchestra.run.smithi029.stderr:reweighted osd.5 to 1 (10000) 2024-09-16T20:52:01.754 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-16T20:52:02.742 INFO:teuthology.orchestra.run.smithi029.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-16T20:52:02.760 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-16T20:52:02.760 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-16T20:52:02.760 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-16T20:52:02.760 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-16T20:52:02.760 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-16T20:52:02.760 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T20:52:03.069 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:52:03.069 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":235,"fsid":"e28af4a7-5fcd-4de8-8f77-a2b575361bb1","created":"2024-09-16T20:47:39.760609+0000","modified":"2024-09-16T20:52:02.737578+0000","last_up_change":"2024-09-16T20:47:50.238013+0000","last_in_change":"2024-09-16T20:51:22.595086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T20:47:52.220750+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T20:47:56.756666+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-16T20:48:06.418889+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"218","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":48,"snap_epoch":218,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c3cb2d7-7d23-4566-9ca1-d7b08e12360c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":228,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":2744025949},{"type":"v1","addr":"172.21.15.29:6817","nonce":2744025949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":2744025949},{"type":"v1","addr":"172.21.15.29:6819","nonce":2744025949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":2744025949},{"type":"v1","addr":"172.21.15.29:6823","nonce":2744025949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":2744025949},{"type":"v1","addr":"172.21.15.29:6821","nonce":2744025949}]},"public_addr":"172.21.15.29:6817/2744025949","cluster_addr":"172.21.15.29:6819/2744025949","heartbeat_back_addr":"172.21.15.29:6823/2744025949","heartbeat_front_addr":"172.21.15.29:6821/2744025949","state":["exists","up"]},{"osd":1,"uuid":"5cf8c20c-1ed2-42ae-a8b0-c88cc4e36bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":208628762},{"type":"v1","addr":"172.21.15.29:6801","nonce":208628762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":208628762},{"type":"v1","addr":"172.21.15.29:6803","nonce":208628762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":208628762},{"type":"v1","addr":"172.21.15.29:6807","nonce":208628762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":208628762},{"type":"v1","addr":"172.21.15.29:6805","nonce":208628762}]},"public_addr":"172.21.15.29:6801/208628762","cluster_addr":"172.21.15.29:6803/208628762","heartbeat_back_addr":"172.21.15.29:6807/208628762","heartbeat_front_addr":"172.21.15.29:6805/208628762","state":["exists","up"]},{"osd":2,"uuid":"075d16da-82bf-449b-9f6e-542c9e8d810b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3447426536},{"type":"v1","addr":"172.21.15.29:6809","nonce":3447426536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3447426536},{"type":"v1","addr":"172.21.15.29:6811","nonce":3447426536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3447426536},{"type":"v1","addr":"172.21.15.29:6815","nonce":3447426536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":3447426536},{"type":"v1","addr":"172.21.15.29:6813","nonce":3447426536}]},"public_addr":"172.21.15.29:6809/3447426536","cluster_addr":"172.21.15.29:6811/3447426536","heartbeat_back_addr":"172.21.15.29:6815/3447426536","heartbeat_front_addr":"172.21.15.29:6813/3447426536","state":["exists","up"]},{"osd":3,"uuid":"0612be38-3a24-41b6-aba4-5a2fe985165e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3687583891},{"type":"v1","addr":"172.21.15.43:6809","nonce":3687583891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3687583891},{"type":"v1","addr":"172.21.15.43:6811","nonce":3687583891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3687583891},{"type":"v1","addr":"172.21.15.43:6815","nonce":3687583891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3687583891},{"type":"v1","addr":"172.21.15.43:6813","nonce":3687583891}]},"public_addr":"172.21.15.43:6809/3687583891","cluster_addr":"172.21.15.43:6811/3687583891","heartbeat_back_addr":"172.21.15.43:6815/3687583891","heartbeat_front_addr":"172.21.15.43:6813/3687583891","state":["exists","up"]},{"osd":4,"uuid":"be590bf8-e9fa-44ca-a471-6f6bcb087ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":2505220646},{"type":"v1","addr":"172.21.15.43:6801","nonce":2505220646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":2505220646},{"type":"v1","addr":"172.21.15.43:6803","nonce":2505220646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":2505220646},{"type":"v1","addr":"172.21.15.43:6807","nonce":2505220646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":2505220646},{"type":"v1","addr":"172.21.15.43:6805","nonce":2505220646}]},"public_addr":"172.21.15.43:6801/2505220646","cluster_addr":"172.21.15.43:6803/2505220646","heartbeat_back_addr":"172.21.15.43:6807/2505220646","heartbeat_front_addr":"172.21.15.43:6805/2505220646","state":["exists","up"]},{"osd":5,"uuid":"4a208e30-031b-4a9f-af52-e26d4a039ea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":198,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1517367307},{"type":"v1","addr":"172.21.15.43:6817","nonce":1517367307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":1517367307},{"type":"v1","addr":"172.21.15.43:6819","nonce":1517367307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":1517367307},{"type":"v1","addr":"172.21.15.43:6823","nonce":1517367307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1517367307},{"type":"v1","addr":"172.21.15.43:6821","nonce":1517367307}]},"public_addr":"172.21.15.43:6817/1517367307","cluster_addr":"172.21.15.43:6819/1517367307","heartbeat_back_addr":"172.21.15.43:6823/1517367307","heartbeat_front_addr":"172.21.15.43:6821/1517367307","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T20:47:47.887412+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T20:47:47.658782+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T20:47:47.453752+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T20:47:47.487992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T20:47:46.943344+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T20:47:47.901547+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1b","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T20:52:03.082 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-16T20:52:03.082 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T20:52:03.383 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:52:03.383 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":235,"fsid":"e28af4a7-5fcd-4de8-8f77-a2b575361bb1","created":"2024-09-16T20:47:39.760609+0000","modified":"2024-09-16T20:52:02.737578+0000","last_up_change":"2024-09-16T20:47:50.238013+0000","last_in_change":"2024-09-16T20:51:22.595086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T20:47:52.220750+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T20:47:56.756666+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-16T20:48:06.418889+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"218","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":48,"snap_epoch":218,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c3cb2d7-7d23-4566-9ca1-d7b08e12360c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":228,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":2744025949},{"type":"v1","addr":"172.21.15.29:6817","nonce":2744025949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":2744025949},{"type":"v1","addr":"172.21.15.29:6819","nonce":2744025949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":2744025949},{"type":"v1","addr":"172.21.15.29:6823","nonce":2744025949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":2744025949},{"type":"v1","addr":"172.21.15.29:6821","nonce":2744025949}]},"public_addr":"172.21.15.29:6817/2744025949","cluster_addr":"172.21.15.29:6819/2744025949","heartbeat_back_addr":"172.21.15.29:6823/2744025949","heartbeat_front_addr":"172.21.15.29:6821/2744025949","state":["exists","up"]},{"osd":1,"uuid":"5cf8c20c-1ed2-42ae-a8b0-c88cc4e36bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":208628762},{"type":"v1","addr":"172.21.15.29:6801","nonce":208628762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":208628762},{"type":"v1","addr":"172.21.15.29:6803","nonce":208628762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":208628762},{"type":"v1","addr":"172.21.15.29:6807","nonce":208628762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":208628762},{"type":"v1","addr":"172.21.15.29:6805","nonce":208628762}]},"public_addr":"172.21.15.29:6801/208628762","cluster_addr":"172.21.15.29:6803/208628762","heartbeat_back_addr":"172.21.15.29:6807/208628762","heartbeat_front_addr":"172.21.15.29:6805/208628762","state":["exists","up"]},{"osd":2,"uuid":"075d16da-82bf-449b-9f6e-542c9e8d810b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3447426536},{"type":"v1","addr":"172.21.15.29:6809","nonce":3447426536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3447426536},{"type":"v1","addr":"172.21.15.29:6811","nonce":3447426536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3447426536},{"type":"v1","addr":"172.21.15.29:6815","nonce":3447426536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":3447426536},{"type":"v1","addr":"172.21.15.29:6813","nonce":3447426536}]},"public_addr":"172.21.15.29:6809/3447426536","cluster_addr":"172.21.15.29:6811/3447426536","heartbeat_back_addr":"172.21.15.29:6815/3447426536","heartbeat_front_addr":"172.21.15.29:6813/3447426536","state":["exists","up"]},{"osd":3,"uuid":"0612be38-3a24-41b6-aba4-5a2fe985165e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3687583891},{"type":"v1","addr":"172.21.15.43:6809","nonce":3687583891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3687583891},{"type":"v1","addr":"172.21.15.43:6811","nonce":3687583891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3687583891},{"type":"v1","addr":"172.21.15.43:6815","nonce":3687583891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3687583891},{"type":"v1","addr":"172.21.15.43:6813","nonce":3687583891}]},"public_addr":"172.21.15.43:6809/3687583891","cluster_addr":"172.21.15.43:6811/3687583891","heartbeat_back_addr":"172.21.15.43:6815/3687583891","heartbeat_front_addr":"172.21.15.43:6813/3687583891","state":["exists","up"]},{"osd":4,"uuid":"be590bf8-e9fa-44ca-a471-6f6bcb087ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":2505220646},{"type":"v1","addr":"172.21.15.43:6801","nonce":2505220646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":2505220646},{"type":"v1","addr":"172.21.15.43:6803","nonce":2505220646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":2505220646},{"type":"v1","addr":"172.21.15.43:6807","nonce":2505220646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":2505220646},{"type":"v1","addr":"172.21.15.43:6805","nonce":2505220646}]},"public_addr":"172.21.15.43:6801/2505220646","cluster_addr":"172.21.15.43:6803/2505220646","heartbeat_back_addr":"172.21.15.43:6807/2505220646","heartbeat_front_addr":"172.21.15.43:6805/2505220646","state":["exists","up"]},{"osd":5,"uuid":"4a208e30-031b-4a9f-af52-e26d4a039ea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":198,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1517367307},{"type":"v1","addr":"172.21.15.43:6817","nonce":1517367307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":1517367307},{"type":"v1","addr":"172.21.15.43:6819","nonce":1517367307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":1517367307},{"type":"v1","addr":"172.21.15.43:6823","nonce":1517367307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1517367307},{"type":"v1","addr":"172.21.15.43:6821","nonce":1517367307}]},"public_addr":"172.21.15.43:6817/1517367307","cluster_addr":"172.21.15.43:6819/1517367307","heartbeat_back_addr":"172.21.15.43:6823/1517367307","heartbeat_front_addr":"172.21.15.43:6821/1517367307","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T20:47:47.887412+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T20:47:47.658782+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T20:47:47.453752+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T20:47:47.487992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T20:47:46.943344+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T20:47:47.901547+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1b","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T20:52:03.397 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-16T20:52:03.397 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-16T20:52:03.397 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-16T20:52:03.398 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-16T20:52:03.398 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-16T20:52:03.399 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-16T20:52:03.576 INFO:teuthology.orchestra.run.smithi029.stdout:55834574903 2024-09-16T20:52:03.577 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-16T20:52:03.580 INFO:teuthology.orchestra.run.smithi029.stdout:47244640311 2024-09-16T20:52:03.580 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-16T20:52:03.602 INFO:teuthology.orchestra.run.smithi029.stdout:55834574903 2024-09-16T20:52:03.602 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-16T20:52:03.606 INFO:teuthology.orchestra.run.smithi029.stdout:55834574902 2024-09-16T20:52:03.606 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-16T20:52:03.627 INFO:teuthology.orchestra.run.smithi029.stdout:55834574903 2024-09-16T20:52:03.627 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-16T20:52:03.628 INFO:teuthology.orchestra.run.smithi029.stdout:55834574903 2024-09-16T20:52:03.628 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T20:52:03.985 INFO:teuthology.orchestra.run.smithi029.stdout:55834574901 2024-09-16T20:52:03.999 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574902 got 55834574901 for osd.1 2024-09-16T20:52:04.035 INFO:teuthology.orchestra.run.smithi029.stdout:55834574902 2024-09-16T20:52:04.048 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574903 got 55834574902 for osd.0 2024-09-16T20:52:04.115 INFO:teuthology.orchestra.run.smithi029.stdout:55834574902 2024-09-16T20:52:04.128 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574903 got 55834574902 for osd.3 2024-09-16T20:52:04.128 INFO:teuthology.orchestra.run.smithi029.stdout:55834574902 2024-09-16T20:52:04.142 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574903 got 55834574902 for osd.2 2024-09-16T20:52:04.143 INFO:teuthology.orchestra.run.smithi029.stdout:47244640309 2024-09-16T20:52:04.156 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640311 got 47244640309 for osd.4 2024-09-16T20:52:04.169 INFO:teuthology.orchestra.run.smithi029.stdout:55834574902 2024-09-16T20:52:04.181 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574903 got 55834574902 for osd.5 2024-09-16T20:52:04.999 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-16T20:52:05.049 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T20:52:05.129 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-16T20:52:05.144 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-16T20:52:05.156 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-16T20:52:05.182 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-16T20:52:05.398 INFO:teuthology.orchestra.run.smithi029.stdout:55834574902 2024-09-16T20:52:05.413 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574902 got 55834574902 for osd.1 2024-09-16T20:52:05.414 DEBUG:teuthology.parallel:result is None 2024-09-16T20:52:05.449 INFO:teuthology.orchestra.run.smithi029.stdout:55834574903 2024-09-16T20:52:05.469 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574903 got 55834574903 for osd.0 2024-09-16T20:52:05.469 DEBUG:teuthology.parallel:result is None 2024-09-16T20:52:05.552 INFO:teuthology.orchestra.run.smithi029.stdout:55834574903 2024-09-16T20:52:05.563 INFO:teuthology.orchestra.run.smithi029.stdout:47244640311 2024-09-16T20:52:05.569 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574903 got 55834574903 for osd.3 2024-09-16T20:52:05.569 DEBUG:teuthology.parallel:result is None 2024-09-16T20:52:05.578 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640311 got 47244640311 for osd.4 2024-09-16T20:52:05.578 DEBUG:teuthology.parallel:result is None 2024-09-16T20:52:05.583 INFO:teuthology.orchestra.run.smithi029.stdout:55834574903 2024-09-16T20:52:05.596 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574903 got 55834574903 for osd.2 2024-09-16T20:52:05.596 DEBUG:teuthology.parallel:result is None 2024-09-16T20:52:05.660 INFO:teuthology.orchestra.run.smithi029.stdout:55834574903 2024-09-16T20:52:05.673 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574903 got 55834574903 for osd.5 2024-09-16T20:52:05.673 DEBUG:teuthology.parallel:result is None 2024-09-16T20:52:05.673 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-16T20:52:05.674 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T20:52:05.970 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:52:05.970 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-09-16T20:52:05.982 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":358,"stamp":"2024-09-16T20:52:04.249430+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":251,"num_object_clones":0,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":243,"num_read":448663,"num_read_kb":1798210,"num_write":23565,"num_write_kb":471301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12652,"num_bytes_recovered":607622711,"num_keys_recovered":1497,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22224,"ondisk_log_size":22224,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":423996,"kb_used_data":17712,"kb_used_omap":9,"kb_used_meta":406134,"kb_avail":565807044,"statfs":{"total":579820584960,"available":579386413056,"internally_reserved":0,"allocated":18137088,"data_stored":8272688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9567,"internal_metadata":415881889},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.526082"},"pg_stats":[{"pgid":"3.1f","version":"218'359","reported_seq":10822,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911515+0000","last_change":"2024-09-16T20:51:45.642922+0000","last_active":"2024-09-16T20:52:02.911515+0000","last_peered":"2024-09-16T20:52:02.911515+0000","last_clean":"2024-09-16T20:52:02.911515+0000","last_became_active":"2024-09-16T20:49:12.801566+0000","last_became_peered":"2024-09-16T20:49:12.801566+0000","last_unstale":"2024-09-16T20:52:02.911515+0000","last_undegraded":"2024-09-16T20:52:02.911515+0000","last_fullsized":"2024-09-16T20:52:02.911515+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:20:11.666141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016075530000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10091,"num_read_kb":31685,"num_write":315,"num_write_kb":5043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":2125998,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"218'905","reported_seq":28772,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.912100+0000","last_change":"2024-09-16T20:51:45.580852+0000","last_active":"2024-09-16T20:52:02.912100+0000","last_peered":"2024-09-16T20:52:02.912100+0000","last_clean":"2024-09-16T20:52:02.912100+0000","last_became_active":"2024-09-16T20:51:27.628085+0000","last_became_peered":"2024-09-16T20:51:27.628085+0000","last_unstale":"2024-09-16T20:52:02.912100+0000","last_undegraded":"2024-09-16T20:52:02.912100+0000","last_fullsized":"2024-09-16T20:52:02.912100+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":905,"log_dups_size":0,"ondisk_log_size":905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:59:41.269032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001444383,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":29337,"num_read_kb":83641,"num_write":1040,"num_write_kb":10328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":562,"num_bytes_recovered":11317531,"num_keys_recovered":154,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"214'707","reported_seq":12942,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.741297+0000","last_change":"2024-09-16T20:51:45.580224+0000","last_active":"2024-09-16T20:52:02.741297+0000","last_peered":"2024-09-16T20:52:02.741297+0000","last_clean":"2024-09-16T20:52:02.741297+0000","last_became_active":"2024-09-16T20:51:26.624314+0000","last_became_peered":"2024-09-16T20:51:26.624314+0000","last_unstale":"2024-09-16T20:52:02.741297+0000","last_undegraded":"2024-09-16T20:52:02.741297+0000","last_fullsized":"2024-09-16T20:52:02.741297+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":707,"log_dups_size":0,"ondisk_log_size":707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:47:19.682479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014726419999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":13066,"num_read_kb":103238,"num_write":750,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":387,"num_bytes_recovered":35619030,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"218'714","reported_seq":4372,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.742231+0000","last_change":"2024-09-16T20:51:45.586449+0000","last_active":"2024-09-16T20:52:02.742231+0000","last_peered":"2024-09-16T20:52:02.742231+0000","last_clean":"2024-09-16T20:52:02.742231+0000","last_became_active":"2024-09-16T20:51:04.577608+0000","last_became_peered":"2024-09-16T20:51:04.577608+0000","last_unstale":"2024-09-16T20:52:02.742231+0000","last_undegraded":"2024-09-16T20:52:02.742231+0000","last_fullsized":"2024-09-16T20:52:02.742231+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:15:00.043848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034574200000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":4615,"num_read_kb":24234,"num_write":996,"num_write_kb":15567,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":25643873,"num_keys_recovered":142,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"217'872","reported_seq":11160,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911930+0000","last_change":"2024-09-16T20:51:49.582917+0000","last_active":"2024-09-16T20:52:02.911930+0000","last_peered":"2024-09-16T20:52:02.911930+0000","last_clean":"2024-09-16T20:52:02.911930+0000","last_became_active":"2024-09-16T20:51:49.579980+0000","last_became_peered":"2024-09-16T20:51:49.579980+0000","last_unstale":"2024-09-16T20:52:02.911930+0000","last_undegraded":"2024-09-16T20:52:02.911930+0000","last_fullsized":"2024-09-16T20:52:02.911930+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":872,"log_dups_size":0,"ondisk_log_size":872,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:13:38.941261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001184378,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":10970,"num_read_kb":44437,"num_write":1151,"num_write_kb":16781,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":956,"num_bytes_recovered":37659232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"217'1221","reported_seq":19862,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.742042+0000","last_change":"2024-09-16T20:51:45.585643+0000","last_active":"2024-09-16T20:52:02.742042+0000","last_peered":"2024-09-16T20:52:02.742042+0000","last_clean":"2024-09-16T20:52:02.742042+0000","last_became_active":"2024-09-16T20:49:12.802869+0000","last_became_peered":"2024-09-16T20:49:12.802869+0000","last_unstale":"2024-09-16T20:52:02.742042+0000","last_undegraded":"2024-09-16T20:52:02.742042+0000","last_fullsized":"2024-09-16T20:52:02.742042+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":1221,"log_dups_size":0,"ondisk_log_size":1221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:17:13.191250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026611790000000001,"stat_sum":{"num_bytes":19,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":25740,"num_read_kb":67599,"num_write":1269,"num_write_kb":16392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":1051004,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"218'770","reported_seq":12989,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.912288+0000","last_change":"2024-09-16T20:51:45.584848+0000","last_active":"2024-09-16T20:52:02.912288+0000","last_peered":"2024-09-16T20:52:02.912288+0000","last_clean":"2024-09-16T20:52:02.912288+0000","last_became_active":"2024-09-16T20:51:16.577039+0000","last_became_peered":"2024-09-16T20:51:16.577039+0000","last_unstale":"2024-09-16T20:52:02.912288+0000","last_undegraded":"2024-09-16T20:52:02.912288+0000","last_fullsized":"2024-09-16T20:52:02.912288+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":770,"log_dups_size":0,"ondisk_log_size":770,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:26:39.219016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023604640000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":10011,"num_read_kb":47438,"num_write":753,"num_write_kb":12180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":12922788,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"216'458","reported_seq":13762,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911501+0000","last_change":"2024-09-16T20:51:45.581009+0000","last_active":"2024-09-16T20:52:02.911501+0000","last_peered":"2024-09-16T20:52:02.911501+0000","last_clean":"2024-09-16T20:52:02.911501+0000","last_became_active":"2024-09-16T20:51:15.575429+0000","last_became_peered":"2024-09-16T20:51:15.575429+0000","last_unstale":"2024-09-16T20:52:02.911501+0000","last_undegraded":"2024-09-16T20:52:02.911501+0000","last_fullsized":"2024-09-16T20:52:02.911501+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":458,"log_dups_size":0,"ondisk_log_size":458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:42:42.274481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001906629,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":14274,"num_read_kb":52840,"num_write":349,"num_write_kb":9355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":231,"num_bytes_recovered":11122736,"num_keys_recovered":66,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"217'981","reported_seq":8058,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911740+0000","last_change":"2024-09-16T20:51:45.642868+0000","last_active":"2024-09-16T20:52:02.911740+0000","last_peered":"2024-09-16T20:52:02.911740+0000","last_clean":"2024-09-16T20:52:02.911740+0000","last_became_active":"2024-09-16T20:51:03.592284+0000","last_became_peered":"2024-09-16T20:51:03.592284+0000","last_unstale":"2024-09-16T20:52:02.911740+0000","last_undegraded":"2024-09-16T20:52:02.911740+0000","last_fullsized":"2024-09-16T20:52:02.911740+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":981,"log_dups_size":0,"ondisk_log_size":981,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:35:23.178350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015146339999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":11609,"num_read_kb":15811,"num_write":975,"num_write_kb":5184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":280,"num_bytes_recovered":1580920,"num_keys_recovered":144,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"217'1277","reported_seq":11828,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.741549+0000","last_change":"2024-09-16T20:51:45.581650+0000","last_active":"2024-09-16T20:52:02.741549+0000","last_peered":"2024-09-16T20:52:02.741549+0000","last_clean":"2024-09-16T20:52:02.741549+0000","last_became_active":"2024-09-16T20:51:27.624341+0000","last_became_peered":"2024-09-16T20:51:27.624341+0000","last_unstale":"2024-09-16T20:52:02.741549+0000","last_undegraded":"2024-09-16T20:52:02.741549+0000","last_fullsized":"2024-09-16T20:52:02.741549+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":1277,"log_dups_size":0,"ondisk_log_size":1277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:01:31.923277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011831579999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":17170,"num_read_kb":43312,"num_write":1584,"num_write_kb":10211,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":542,"num_bytes_recovered":21909352,"num_keys_recovered":70,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"218'595","reported_seq":14001,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911281+0000","last_change":"2024-09-16T20:51:45.579668+0000","last_active":"2024-09-16T20:52:02.911281+0000","last_peered":"2024-09-16T20:52:02.911281+0000","last_clean":"2024-09-16T20:52:02.911281+0000","last_became_active":"2024-09-16T20:51:16.577620+0000","last_became_peered":"2024-09-16T20:51:16.577620+0000","last_unstale":"2024-09-16T20:52:02.911281+0000","last_undegraded":"2024-09-16T20:52:02.911281+0000","last_fullsized":"2024-09-16T20:52:02.911281+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":595,"log_dups_size":0,"ondisk_log_size":595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:23:49.832618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022662979999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":12843,"num_read_kb":77657,"num_write":690,"num_write_kb":15118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":562,"num_bytes_recovered":17171700,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"218'370","reported_seq":19983,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.742221+0000","last_change":"2024-09-16T20:51:57.720592+0000","last_active":"2024-09-16T20:52:02.742221+0000","last_peered":"2024-09-16T20:52:02.742221+0000","last_clean":"2024-09-16T20:52:02.742221+0000","last_became_active":"2024-09-16T20:51:57.718930+0000","last_became_peered":"2024-09-16T20:51:57.718930+0000","last_unstale":"2024-09-16T20:52:02.742221+0000","last_undegraded":"2024-09-16T20:52:02.742221+0000","last_fullsized":"2024-09-16T20:52:02.742221+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:08:04.217338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014936070000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":18755,"num_read_kb":45565,"num_write":465,"num_write_kb":13208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":540,"num_bytes_recovered":30162312,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":460,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.742331+0000","last_change":"2024-09-16T20:51:00.578451+0000","last_active":"2024-09-16T20:52:02.742331+0000","last_peered":"2024-09-16T20:52:02.742331+0000","last_clean":"2024-09-16T20:52:02.742331+0000","last_became_active":"2024-09-16T20:51:00.578081+0000","last_became_peered":"2024-09-16T20:51:00.578081+0000","last_unstale":"2024-09-16T20:52:02.742331+0000","last_undegraded":"2024-09-16T20:52:02.742331+0000","last_fullsized":"2024-09-16T20:52:02.742331+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:35:02.412998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091339500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"218'1028","reported_seq":19427,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.912471+0000","last_change":"2024-09-16T20:51:45.582985+0000","last_active":"2024-09-16T20:52:02.912471+0000","last_peered":"2024-09-16T20:52:02.912471+0000","last_clean":"2024-09-16T20:52:02.912471+0000","last_became_active":"2024-09-16T20:51:16.577104+0000","last_became_peered":"2024-09-16T20:51:16.577104+0000","last_unstale":"2024-09-16T20:52:02.912471+0000","last_undegraded":"2024-09-16T20:52:02.912471+0000","last_fullsized":"2024-09-16T20:52:02.912471+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":1028,"log_dups_size":0,"ondisk_log_size":1028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:56:34.623379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013907780000000001,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":20870,"num_read_kb":85867,"num_write":1218,"num_write_kb":25908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":29601482,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":463,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.912490+0000","last_change":"2024-09-16T20:51:00.577927+0000","last_active":"2024-09-16T20:52:02.912490+0000","last_peered":"2024-09-16T20:52:02.912490+0000","last_clean":"2024-09-16T20:52:02.912490+0000","last_became_active":"2024-09-16T20:51:00.577653+0000","last_became_peered":"2024-09-16T20:51:00.577653+0000","last_unstale":"2024-09-16T20:52:02.912490+0000","last_undegraded":"2024-09-16T20:52:02.912490+0000","last_fullsized":"2024-09-16T20:52:02.912490+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:12:13.831668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043659299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"214'653","reported_seq":25408,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.912290+0000","last_change":"2024-09-16T20:51:45.583929+0000","last_active":"2024-09-16T20:52:02.912290+0000","last_peered":"2024-09-16T20:52:02.912290+0000","last_clean":"2024-09-16T20:52:02.912290+0000","last_became_active":"2024-09-16T20:51:15.575607+0000","last_became_peered":"2024-09-16T20:51:15.575607+0000","last_unstale":"2024-09-16T20:52:02.912290+0000","last_undegraded":"2024-09-16T20:52:02.912290+0000","last_fullsized":"2024-09-16T20:52:02.912290+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":653,"log_dups_size":0,"ondisk_log_size":653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:42:18.817016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022924479999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":24178,"num_read_kb":81081,"num_write":502,"num_write_kb":17573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":20407955,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":448,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.742362+0000","last_change":"2024-09-16T20:48:00.278054+0000","last_active":"2024-09-16T20:52:02.742362+0000","last_peered":"2024-09-16T20:52:02.742362+0000","last_clean":"2024-09-16T20:52:02.742362+0000","last_became_active":"2024-09-16T20:47:58.272861+0000","last_became_peered":"2024-09-16T20:47:58.272861+0000","last_unstale":"2024-09-16T20:52:02.742362+0000","last_undegraded":"2024-09-16T20:52:02.742362+0000","last_fullsized":"2024-09-16T20:52:02.742362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:13:59.135547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012429489999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"213'491","reported_seq":3884,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.912098+0000","last_change":"2024-09-16T20:51:45.583107+0000","last_active":"2024-09-16T20:52:02.912098+0000","last_peered":"2024-09-16T20:52:02.912098+0000","last_clean":"2024-09-16T20:52:02.912098+0000","last_became_active":"2024-09-16T20:49:12.801810+0000","last_became_peered":"2024-09-16T20:49:12.801810+0000","last_unstale":"2024-09-16T20:52:02.912098+0000","last_undegraded":"2024-09-16T20:52:02.912098+0000","last_fullsized":"2024-09-16T20:52:02.912098+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":491,"log_dups_size":0,"ondisk_log_size":491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:48:19.573269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015370990000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":3741,"num_read_kb":11405,"num_write":512,"num_write_kb":6561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":2071114,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":340,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911584+0000","last_change":"2024-09-16T20:51:12.615838+0000","last_active":"2024-09-16T20:52:02.911584+0000","last_peered":"2024-09-16T20:52:02.911584+0000","last_clean":"2024-09-16T20:52:02.911584+0000","last_became_active":"2024-09-16T20:51:12.615412+0000","last_became_peered":"2024-09-16T20:51:12.615412+0000","last_unstale":"2024-09-16T20:52:02.911584+0000","last_undegraded":"2024-09-16T20:52:02.911584+0000","last_fullsized":"2024-09-16T20:52:02.911584+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:17:12.150193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"217'668","reported_seq":19715,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.741476+0000","last_change":"2024-09-16T20:51:45.580833+0000","last_active":"2024-09-16T20:52:02.741476+0000","last_peered":"2024-09-16T20:52:02.741476+0000","last_clean":"2024-09-16T20:52:02.741476+0000","last_became_active":"2024-09-16T20:51:27.624519+0000","last_became_peered":"2024-09-16T20:51:27.624519+0000","last_unstale":"2024-09-16T20:52:02.741476+0000","last_undegraded":"2024-09-16T20:52:02.741476+0000","last_fullsized":"2024-09-16T20:52:02.741476+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":668,"log_dups_size":0,"ondisk_log_size":668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:44:59.098443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001777441,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":18081,"num_read_kb":57429,"num_write":654,"num_write_kb":12584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":564,"num_bytes_recovered":27092048,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"218'728","reported_seq":8639,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911256+0000","last_change":"2024-09-16T20:51:45.580666+0000","last_active":"2024-09-16T20:52:02.911256+0000","last_peered":"2024-09-16T20:52:02.911256+0000","last_clean":"2024-09-16T20:52:02.911256+0000","last_became_active":"2024-09-16T20:51:27.626032+0000","last_became_peered":"2024-09-16T20:51:27.626032+0000","last_unstale":"2024-09-16T20:52:02.911256+0000","last_undegraded":"2024-09-16T20:52:02.911256+0000","last_fullsized":"2024-09-16T20:52:02.911256+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":728,"log_dups_size":0,"ondisk_log_size":728,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:28:16.242279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025014909999999998,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":7673,"num_read_kb":36418,"num_write":666,"num_write_kb":13145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":523,"num_bytes_recovered":11083665,"num_keys_recovered":14,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":137,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911158+0000","last_change":"2024-09-16T20:51:00.578241+0000","last_active":"2024-09-16T20:52:02.911158+0000","last_peered":"2024-09-16T20:52:02.911158+0000","last_clean":"2024-09-16T20:52:02.911158+0000","last_became_active":"2024-09-16T20:51:00.574693+0000","last_became_peered":"2024-09-16T20:51:00.574693+0000","last_unstale":"2024-09-16T20:52:02.911158+0000","last_undegraded":"2024-09-16T20:52:02.911158+0000","last_fullsized":"2024-09-16T20:52:02.911158+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:35:20.550930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062077499999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"216'498","reported_seq":18808,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911630+0000","last_change":"2024-09-16T20:51:45.643842+0000","last_active":"2024-09-16T20:52:02.911630+0000","last_peered":"2024-09-16T20:52:02.911630+0000","last_clean":"2024-09-16T20:52:02.911630+0000","last_became_active":"2024-09-16T20:51:26.626185+0000","last_became_peered":"2024-09-16T20:51:26.626185+0000","last_unstale":"2024-09-16T20:52:02.911630+0000","last_undegraded":"2024-09-16T20:52:02.911630+0000","last_fullsized":"2024-09-16T20:52:02.911630+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":498,"log_dups_size":0,"ondisk_log_size":498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:57:30.480666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023862639999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":19429,"num_read_kb":77391,"num_write":450,"num_write_kb":15765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":3862081,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":109,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911799+0000","last_change":"2024-09-16T20:51:12.615208+0000","last_active":"2024-09-16T20:52:02.911799+0000","last_peered":"2024-09-16T20:52:02.911799+0000","last_clean":"2024-09-16T20:52:02.911799+0000","last_became_active":"2024-09-16T20:51:12.614812+0000","last_became_peered":"2024-09-16T20:51:12.614812+0000","last_unstale":"2024-09-16T20:52:02.911799+0000","last_undegraded":"2024-09-16T20:52:02.911799+0000","last_fullsized":"2024-09-16T20:52:02.911799+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:37:11.741478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"215'312","reported_seq":5919,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.912408+0000","last_change":"2024-09-16T20:51:45.584820+0000","last_active":"2024-09-16T20:52:02.912408+0000","last_peered":"2024-09-16T20:52:02.912408+0000","last_clean":"2024-09-16T20:52:02.912408+0000","last_became_active":"2024-09-16T20:51:26.624469+0000","last_became_peered":"2024-09-16T20:51:26.624469+0000","last_unstale":"2024-09-16T20:52:02.912408+0000","last_undegraded":"2024-09-16T20:52:02.912408+0000","last_fullsized":"2024-09-16T20:52:02.912408+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:52:43.183901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025837220000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4323,"num_read_kb":33093,"num_write":273,"num_write_kb":12351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":324,"num_bytes_recovered":16931328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":89,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.741294+0000","last_change":"2024-09-16T20:51:23.608479+0000","last_active":"2024-09-16T20:52:02.741294+0000","last_peered":"2024-09-16T20:52:02.741294+0000","last_clean":"2024-09-16T20:52:02.741294+0000","last_became_active":"2024-09-16T20:51:23.608159+0000","last_became_peered":"2024-09-16T20:51:23.608159+0000","last_unstale":"2024-09-16T20:52:02.741294+0000","last_undegraded":"2024-09-16T20:52:02.741294+0000","last_fullsized":"2024-09-16T20:52:02.741294+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:06:21.085498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037636000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"217'609","reported_seq":7615,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911699+0000","last_change":"2024-09-16T20:51:45.582309+0000","last_active":"2024-09-16T20:52:02.911699+0000","last_peered":"2024-09-16T20:52:02.911699+0000","last_clean":"2024-09-16T20:52:02.911699+0000","last_became_active":"2024-09-16T20:51:26.622629+0000","last_became_peered":"2024-09-16T20:51:26.622629+0000","last_unstale":"2024-09-16T20:52:02.911699+0000","last_undegraded":"2024-09-16T20:52:02.911699+0000","last_fullsized":"2024-09-16T20:52:02.911699+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:40:49.603343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029005419999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":9084,"num_read_kb":37091,"num_write":448,"num_write_kb":13187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":848,"num_bytes_recovered":24298320,"num_keys_recovered":219,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":469,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.742037+0000","last_change":"2024-09-16T20:51:23.608488+0000","last_active":"2024-09-16T20:52:02.742037+0000","last_peered":"2024-09-16T20:52:02.742037+0000","last_clean":"2024-09-16T20:52:02.742037+0000","last_became_active":"2024-09-16T20:51:23.605321+0000","last_became_peered":"2024-09-16T20:51:23.605321+0000","last_unstale":"2024-09-16T20:52:02.742037+0000","last_undegraded":"2024-09-16T20:52:02.742037+0000","last_fullsized":"2024-09-16T20:52:02.742037+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:50:46.322344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095771799999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":564,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.741984+0000","last_change":"2024-09-16T20:49:09.565188+0000","last_active":"2024-09-16T20:52:02.741984+0000","last_peered":"2024-09-16T20:52:02.741984+0000","last_clean":"2024-09-16T20:52:02.741984+0000","last_became_active":"2024-09-16T20:49:09.559571+0000","last_became_peered":"2024-09-16T20:49:09.559571+0000","last_unstale":"2024-09-16T20:52:02.741984+0000","last_undegraded":"2024-09-16T20:52:02.741984+0000","last_fullsized":"2024-09-16T20:52:02.741984+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:52.241408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:52.241408+0000","last_clean_scrub_stamp":"2024-09-16T20:47:52.241408+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:01:53.651981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"214'641","reported_seq":7064,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.912231+0000","last_change":"2024-09-16T20:51:45.583936+0000","last_active":"2024-09-16T20:52:02.912231+0000","last_peered":"2024-09-16T20:52:02.912231+0000","last_clean":"2024-09-16T20:52:02.912231+0000","last_became_active":"2024-09-16T20:51:27.624220+0000","last_became_peered":"2024-09-16T20:51:27.624220+0000","last_unstale":"2024-09-16T20:52:02.912231+0000","last_undegraded":"2024-09-16T20:52:02.912231+0000","last_fullsized":"2024-09-16T20:52:02.912231+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":641,"log_dups_size":0,"ondisk_log_size":641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:13:47.571246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022745209999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6921,"num_read_kb":32984,"num_write":827,"num_write_kb":7103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":4839295,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"217'639","reported_seq":16470,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911021+0000","last_change":"2024-09-16T20:51:45.578750+0000","last_active":"2024-09-16T20:52:02.911021+0000","last_peered":"2024-09-16T20:52:02.911021+0000","last_clean":"2024-09-16T20:52:02.911021+0000","last_became_active":"2024-09-16T20:51:17.575136+0000","last_became_peered":"2024-09-16T20:51:17.575136+0000","last_unstale":"2024-09-16T20:52:02.911021+0000","last_undegraded":"2024-09-16T20:52:02.911021+0000","last_fullsized":"2024-09-16T20:52:02.911021+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":639,"log_dups_size":0,"ondisk_log_size":639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:32:32.550808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022726539999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":15159,"num_read_kb":101301,"num_write":561,"num_write_kb":29560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":790,"num_bytes_recovered":48497898,"num_keys_recovered":60,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"217'568","reported_seq":9847,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911042+0000","last_change":"2024-09-16T20:51:45.578068+0000","last_active":"2024-09-16T20:52:02.911042+0000","last_peered":"2024-09-16T20:52:02.911042+0000","last_clean":"2024-09-16T20:52:02.911042+0000","last_became_active":"2024-09-16T20:51:03.590152+0000","last_became_peered":"2024-09-16T20:51:03.590152+0000","last_unstale":"2024-09-16T20:52:02.911042+0000","last_undegraded":"2024-09-16T20:52:02.911042+0000","last_fullsized":"2024-09-16T20:52:02.911042+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":568,"log_dups_size":0,"ondisk_log_size":568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:21:38.569104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017317439999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":9605,"num_read_kb":51536,"num_write":618,"num_write_kb":12681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":14274951,"num_keys_recovered":91,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"218'468","reported_seq":8097,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.912037+0000","last_change":"2024-09-16T20:51:45.582215+0000","last_active":"2024-09-16T20:52:02.912037+0000","last_peered":"2024-09-16T20:52:02.912037+0000","last_clean":"2024-09-16T20:52:02.912037+0000","last_became_active":"2024-09-16T20:51:16.575698+0000","last_became_peered":"2024-09-16T20:51:16.575698+0000","last_unstale":"2024-09-16T20:52:02.912037+0000","last_undegraded":"2024-09-16T20:52:02.912037+0000","last_fullsized":"2024-09-16T20:52:02.912037+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:15:28.632535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020731130000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":7052,"num_read_kb":42310,"num_write":428,"num_write_kb":14897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":445,"num_bytes_recovered":8390493,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"216'726","reported_seq":22773,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911294+0000","last_change":"2024-09-16T20:51:45.579437+0000","last_active":"2024-09-16T20:52:02.911294+0000","last_peered":"2024-09-16T20:52:02.911294+0000","last_clean":"2024-09-16T20:52:02.911294+0000","last_became_active":"2024-09-16T20:51:17.575222+0000","last_became_peered":"2024-09-16T20:51:17.575222+0000","last_unstale":"2024-09-16T20:52:02.911294+0000","last_undegraded":"2024-09-16T20:52:02.911294+0000","last_fullsized":"2024-09-16T20:52:02.911294+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":726,"log_dups_size":0,"ondisk_log_size":726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:44:38.139123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018824849999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":22418,"num_read_kb":91945,"num_write":909,"num_write_kb":19380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":627,"num_bytes_recovered":71963834,"num_keys_recovered":119,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"218'1128","reported_seq":18644,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.742261+0000","last_change":"2024-09-16T20:51:45.585311+0000","last_active":"2024-09-16T20:52:02.742261+0000","last_peered":"2024-09-16T20:52:02.742261+0000","last_clean":"2024-09-16T20:52:02.742261+0000","last_became_active":"2024-09-16T20:51:05.581172+0000","last_became_peered":"2024-09-16T20:51:05.581172+0000","last_unstale":"2024-09-16T20:52:02.742261+0000","last_undegraded":"2024-09-16T20:52:02.742261+0000","last_fullsized":"2024-09-16T20:52:02.742261+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:27:15.820301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002500038,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":19435,"num_read_kb":70139,"num_write":1420,"num_write_kb":21576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":16987444,"num_keys_recovered":40,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"217'581","reported_seq":9275,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.741548+0000","last_change":"2024-09-16T20:51:45.581620+0000","last_active":"2024-09-16T20:52:02.741548+0000","last_peered":"2024-09-16T20:52:02.741548+0000","last_clean":"2024-09-16T20:52:02.741548+0000","last_became_active":"2024-09-16T20:51:27.627520+0000","last_became_peered":"2024-09-16T20:51:27.627520+0000","last_unstale":"2024-09-16T20:52:02.741548+0000","last_undegraded":"2024-09-16T20:52:02.741548+0000","last_fullsized":"2024-09-16T20:52:02.741548+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":581,"log_dups_size":0,"ondisk_log_size":581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:30:31.740673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012113930000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":8329,"num_read_kb":54907,"num_write":641,"num_write_kb":11423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":479,"num_bytes_recovered":14706083,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"217'534","reported_seq":13756,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911354+0000","last_change":"2024-09-16T20:51:45.580500+0000","last_active":"2024-09-16T20:52:02.911354+0000","last_peered":"2024-09-16T20:52:02.911354+0000","last_clean":"2024-09-16T20:52:02.911354+0000","last_became_active":"2024-09-16T20:51:05.581058+0000","last_became_peered":"2024-09-16T20:51:05.581058+0000","last_unstale":"2024-09-16T20:52:02.911354+0000","last_undegraded":"2024-09-16T20:52:02.911354+0000","last_fullsized":"2024-09-16T20:52:02.911354+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:55:20.810743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026424640000000002,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":13544,"num_read_kb":49574,"num_write":426,"num_write_kb":14576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":284,"num_bytes_recovered":9641443,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"218'691","reported_seq":13753,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.741972+0000","last_change":"2024-09-16T20:51:45.584709+0000","last_active":"2024-09-16T20:52:02.741972+0000","last_peered":"2024-09-16T20:52:02.741972+0000","last_clean":"2024-09-16T20:52:02.741972+0000","last_became_active":"2024-09-16T20:49:12.801013+0000","last_became_peered":"2024-09-16T20:49:12.801013+0000","last_unstale":"2024-09-16T20:52:02.741972+0000","last_undegraded":"2024-09-16T20:52:02.741972+0000","last_fullsized":"2024-09-16T20:52:02.741972+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":691,"log_dups_size":0,"ondisk_log_size":691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:13:59.069699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001757113,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":15597,"num_read_kb":58377,"num_write":523,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":99,"num_bytes_recovered":5210742,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"218'888","reported_seq":14337,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.741292+0000","last_change":"2024-09-16T20:51:45.579465+0000","last_active":"2024-09-16T20:52:02.741292+0000","last_peered":"2024-09-16T20:52:02.741292+0000","last_clean":"2024-09-16T20:52:02.741292+0000","last_became_active":"2024-09-16T20:51:26.624407+0000","last_became_peered":"2024-09-16T20:51:26.624407+0000","last_unstale":"2024-09-16T20:52:02.741292+0000","last_undegraded":"2024-09-16T20:52:02.741292+0000","last_fullsized":"2024-09-16T20:52:02.741292+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:59:50.323236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029313379999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":16446,"num_read_kb":75991,"num_write":874,"num_write_kb":24549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":351,"num_bytes_recovered":38747648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"211'608","reported_seq":20163,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911705+0000","last_change":"2024-09-16T20:51:45.643842+0000","last_active":"2024-09-16T20:52:02.911705+0000","last_peered":"2024-09-16T20:52:02.911705+0000","last_clean":"2024-09-16T20:52:02.911705+0000","last_became_active":"2024-09-16T20:51:03.589742+0000","last_became_peered":"2024-09-16T20:51:03.589742+0000","last_unstale":"2024-09-16T20:52:02.911705+0000","last_undegraded":"2024-09-16T20:52:02.911705+0000","last_fullsized":"2024-09-16T20:52:02.911705+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":608,"log_dups_size":0,"ondisk_log_size":608,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:49:22.579735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024335089999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":16969,"num_read_kb":76220,"num_write":613,"num_write_kb":13841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":486,"num_bytes_recovered":15329756,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"216'461","reported_seq":12004,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.741986+0000","last_change":"2024-09-16T20:51:45.586711+0000","last_active":"2024-09-16T20:52:02.741986+0000","last_peered":"2024-09-16T20:52:02.741986+0000","last_clean":"2024-09-16T20:52:02.741986+0000","last_became_active":"2024-09-16T20:51:03.595360+0000","last_became_peered":"2024-09-16T20:51:03.595360+0000","last_unstale":"2024-09-16T20:52:02.741986+0000","last_undegraded":"2024-09-16T20:52:02.741986+0000","last_fullsized":"2024-09-16T20:52:02.741986+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":461,"log_dups_size":0,"ondisk_log_size":461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:47:40.600095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035295309999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":11262,"num_read_kb":35639,"num_write":562,"num_write_kb":13645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":198,"num_bytes_recovered":14217900,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":247,"num_object_clones":0,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":243,"num_read":448597,"num_read_kb":1798155,"num_write":23462,"num_write_kb":469671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12645,"num_bytes_recovered":606441956,"num_keys_recovered":1497,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22149,"ondisk_log_size":22149,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574903,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52588,"kb_used_data":3344,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319252,"statfs":{"total":96636764160,"available":96582914048,"internally_reserved":0,"allocated":3424256,"data_stored":1774023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574903,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74120,"kb_used_data":2736,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94297720,"statfs":{"total":96636764160,"available":96560865280,"internally_reserved":0,"allocated":2801664,"data_stored":1173696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73071047},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574903,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69980,"kb_used_data":2760,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94301860,"statfs":{"total":96636764160,"available":96565104640,"internally_reserved":0,"allocated":2826240,"data_stored":1183636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574902,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70568,"kb_used_data":3344,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94301272,"statfs":{"total":96636764160,"available":96564502528,"internally_reserved":0,"allocated":3424256,"data_stored":1774023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574903,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":96420,"kb_used_data":2764,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":94275420,"statfs":{"total":96636764160,"available":96538030080,"internally_reserved":0,"allocated":2830336,"data_stored":1183655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":95877574},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640311,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60320,"kb_used_data":2764,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311520,"statfs":{"total":96636764160,"available":96574996480,"internally_reserved":0,"allocated":2830336,"data_stored":1183655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T20:52:05.984 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T20:52:06.281 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:52:06.282 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-09-16T20:52:06.293 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":359,"stamp":"2024-09-16T20:52:06.249925+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":251,"num_object_clones":0,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":243,"num_read":448663,"num_read_kb":1798210,"num_write":23565,"num_write_kb":471301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12652,"num_bytes_recovered":607622711,"num_keys_recovered":1497,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22224,"ondisk_log_size":22224,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":424016,"kb_used_data":17736,"kb_used_omap":9,"kb_used_meta":406134,"kb_avail":565807024,"statfs":{"total":579820584960,"available":579386392576,"internally_reserved":0,"allocated":18161664,"data_stored":8282628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9567,"internal_metadata":415881889},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001317"},"pg_stats":[{"pgid":"3.1f","version":"218'359","reported_seq":10822,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911515+0000","last_change":"2024-09-16T20:51:45.642922+0000","last_active":"2024-09-16T20:52:02.911515+0000","last_peered":"2024-09-16T20:52:02.911515+0000","last_clean":"2024-09-16T20:52:02.911515+0000","last_became_active":"2024-09-16T20:49:12.801566+0000","last_became_peered":"2024-09-16T20:49:12.801566+0000","last_unstale":"2024-09-16T20:52:02.911515+0000","last_undegraded":"2024-09-16T20:52:02.911515+0000","last_fullsized":"2024-09-16T20:52:02.911515+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:20:11.666141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016075530000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10091,"num_read_kb":31685,"num_write":315,"num_write_kb":5043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":2125998,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"218'905","reported_seq":28772,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.912100+0000","last_change":"2024-09-16T20:51:45.580852+0000","last_active":"2024-09-16T20:52:02.912100+0000","last_peered":"2024-09-16T20:52:02.912100+0000","last_clean":"2024-09-16T20:52:02.912100+0000","last_became_active":"2024-09-16T20:51:27.628085+0000","last_became_peered":"2024-09-16T20:51:27.628085+0000","last_unstale":"2024-09-16T20:52:02.912100+0000","last_undegraded":"2024-09-16T20:52:02.912100+0000","last_fullsized":"2024-09-16T20:52:02.912100+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":905,"log_dups_size":0,"ondisk_log_size":905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:59:41.269032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001444383,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":29337,"num_read_kb":83641,"num_write":1040,"num_write_kb":10328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":562,"num_bytes_recovered":11317531,"num_keys_recovered":154,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"214'707","reported_seq":12942,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.741297+0000","last_change":"2024-09-16T20:51:45.580224+0000","last_active":"2024-09-16T20:52:02.741297+0000","last_peered":"2024-09-16T20:52:02.741297+0000","last_clean":"2024-09-16T20:52:02.741297+0000","last_became_active":"2024-09-16T20:51:26.624314+0000","last_became_peered":"2024-09-16T20:51:26.624314+0000","last_unstale":"2024-09-16T20:52:02.741297+0000","last_undegraded":"2024-09-16T20:52:02.741297+0000","last_fullsized":"2024-09-16T20:52:02.741297+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":707,"log_dups_size":0,"ondisk_log_size":707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:47:19.682479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014726419999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":13066,"num_read_kb":103238,"num_write":750,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":387,"num_bytes_recovered":35619030,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"218'714","reported_seq":4372,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.742231+0000","last_change":"2024-09-16T20:51:45.586449+0000","last_active":"2024-09-16T20:52:02.742231+0000","last_peered":"2024-09-16T20:52:02.742231+0000","last_clean":"2024-09-16T20:52:02.742231+0000","last_became_active":"2024-09-16T20:51:04.577608+0000","last_became_peered":"2024-09-16T20:51:04.577608+0000","last_unstale":"2024-09-16T20:52:02.742231+0000","last_undegraded":"2024-09-16T20:52:02.742231+0000","last_fullsized":"2024-09-16T20:52:02.742231+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:15:00.043848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034574200000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":4615,"num_read_kb":24234,"num_write":996,"num_write_kb":15567,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":25643873,"num_keys_recovered":142,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"217'872","reported_seq":11160,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911930+0000","last_change":"2024-09-16T20:51:49.582917+0000","last_active":"2024-09-16T20:52:02.911930+0000","last_peered":"2024-09-16T20:52:02.911930+0000","last_clean":"2024-09-16T20:52:02.911930+0000","last_became_active":"2024-09-16T20:51:49.579980+0000","last_became_peered":"2024-09-16T20:51:49.579980+0000","last_unstale":"2024-09-16T20:52:02.911930+0000","last_undegraded":"2024-09-16T20:52:02.911930+0000","last_fullsized":"2024-09-16T20:52:02.911930+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":872,"log_dups_size":0,"ondisk_log_size":872,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:13:38.941261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001184378,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":10970,"num_read_kb":44437,"num_write":1151,"num_write_kb":16781,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":956,"num_bytes_recovered":37659232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"217'1221","reported_seq":19862,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.742042+0000","last_change":"2024-09-16T20:51:45.585643+0000","last_active":"2024-09-16T20:52:02.742042+0000","last_peered":"2024-09-16T20:52:02.742042+0000","last_clean":"2024-09-16T20:52:02.742042+0000","last_became_active":"2024-09-16T20:49:12.802869+0000","last_became_peered":"2024-09-16T20:49:12.802869+0000","last_unstale":"2024-09-16T20:52:02.742042+0000","last_undegraded":"2024-09-16T20:52:02.742042+0000","last_fullsized":"2024-09-16T20:52:02.742042+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":1221,"log_dups_size":0,"ondisk_log_size":1221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:17:13.191250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026611790000000001,"stat_sum":{"num_bytes":19,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":25740,"num_read_kb":67599,"num_write":1269,"num_write_kb":16392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":1051004,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"218'770","reported_seq":12989,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.912288+0000","last_change":"2024-09-16T20:51:45.584848+0000","last_active":"2024-09-16T20:52:02.912288+0000","last_peered":"2024-09-16T20:52:02.912288+0000","last_clean":"2024-09-16T20:52:02.912288+0000","last_became_active":"2024-09-16T20:51:16.577039+0000","last_became_peered":"2024-09-16T20:51:16.577039+0000","last_unstale":"2024-09-16T20:52:02.912288+0000","last_undegraded":"2024-09-16T20:52:02.912288+0000","last_fullsized":"2024-09-16T20:52:02.912288+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":770,"log_dups_size":0,"ondisk_log_size":770,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:26:39.219016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023604640000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":10011,"num_read_kb":47438,"num_write":753,"num_write_kb":12180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":12922788,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"216'458","reported_seq":13762,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911501+0000","last_change":"2024-09-16T20:51:45.581009+0000","last_active":"2024-09-16T20:52:02.911501+0000","last_peered":"2024-09-16T20:52:02.911501+0000","last_clean":"2024-09-16T20:52:02.911501+0000","last_became_active":"2024-09-16T20:51:15.575429+0000","last_became_peered":"2024-09-16T20:51:15.575429+0000","last_unstale":"2024-09-16T20:52:02.911501+0000","last_undegraded":"2024-09-16T20:52:02.911501+0000","last_fullsized":"2024-09-16T20:52:02.911501+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":458,"log_dups_size":0,"ondisk_log_size":458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:42:42.274481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001906629,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":14274,"num_read_kb":52840,"num_write":349,"num_write_kb":9355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":231,"num_bytes_recovered":11122736,"num_keys_recovered":66,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"217'981","reported_seq":8058,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911740+0000","last_change":"2024-09-16T20:51:45.642868+0000","last_active":"2024-09-16T20:52:02.911740+0000","last_peered":"2024-09-16T20:52:02.911740+0000","last_clean":"2024-09-16T20:52:02.911740+0000","last_became_active":"2024-09-16T20:51:03.592284+0000","last_became_peered":"2024-09-16T20:51:03.592284+0000","last_unstale":"2024-09-16T20:52:02.911740+0000","last_undegraded":"2024-09-16T20:52:02.911740+0000","last_fullsized":"2024-09-16T20:52:02.911740+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":981,"log_dups_size":0,"ondisk_log_size":981,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:35:23.178350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015146339999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":11609,"num_read_kb":15811,"num_write":975,"num_write_kb":5184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":280,"num_bytes_recovered":1580920,"num_keys_recovered":144,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"217'1277","reported_seq":11828,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.741549+0000","last_change":"2024-09-16T20:51:45.581650+0000","last_active":"2024-09-16T20:52:02.741549+0000","last_peered":"2024-09-16T20:52:02.741549+0000","last_clean":"2024-09-16T20:52:02.741549+0000","last_became_active":"2024-09-16T20:51:27.624341+0000","last_became_peered":"2024-09-16T20:51:27.624341+0000","last_unstale":"2024-09-16T20:52:02.741549+0000","last_undegraded":"2024-09-16T20:52:02.741549+0000","last_fullsized":"2024-09-16T20:52:02.741549+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":1277,"log_dups_size":0,"ondisk_log_size":1277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:01:31.923277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011831579999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":17170,"num_read_kb":43312,"num_write":1584,"num_write_kb":10211,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":542,"num_bytes_recovered":21909352,"num_keys_recovered":70,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"218'595","reported_seq":14001,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911281+0000","last_change":"2024-09-16T20:51:45.579668+0000","last_active":"2024-09-16T20:52:02.911281+0000","last_peered":"2024-09-16T20:52:02.911281+0000","last_clean":"2024-09-16T20:52:02.911281+0000","last_became_active":"2024-09-16T20:51:16.577620+0000","last_became_peered":"2024-09-16T20:51:16.577620+0000","last_unstale":"2024-09-16T20:52:02.911281+0000","last_undegraded":"2024-09-16T20:52:02.911281+0000","last_fullsized":"2024-09-16T20:52:02.911281+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":595,"log_dups_size":0,"ondisk_log_size":595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:23:49.832618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022662979999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":12843,"num_read_kb":77657,"num_write":690,"num_write_kb":15118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":562,"num_bytes_recovered":17171700,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"218'370","reported_seq":19983,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.742221+0000","last_change":"2024-09-16T20:51:57.720592+0000","last_active":"2024-09-16T20:52:02.742221+0000","last_peered":"2024-09-16T20:52:02.742221+0000","last_clean":"2024-09-16T20:52:02.742221+0000","last_became_active":"2024-09-16T20:51:57.718930+0000","last_became_peered":"2024-09-16T20:51:57.718930+0000","last_unstale":"2024-09-16T20:52:02.742221+0000","last_undegraded":"2024-09-16T20:52:02.742221+0000","last_fullsized":"2024-09-16T20:52:02.742221+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:08:04.217338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014936070000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":18755,"num_read_kb":45565,"num_write":465,"num_write_kb":13208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":540,"num_bytes_recovered":30162312,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":460,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.742331+0000","last_change":"2024-09-16T20:51:00.578451+0000","last_active":"2024-09-16T20:52:02.742331+0000","last_peered":"2024-09-16T20:52:02.742331+0000","last_clean":"2024-09-16T20:52:02.742331+0000","last_became_active":"2024-09-16T20:51:00.578081+0000","last_became_peered":"2024-09-16T20:51:00.578081+0000","last_unstale":"2024-09-16T20:52:02.742331+0000","last_undegraded":"2024-09-16T20:52:02.742331+0000","last_fullsized":"2024-09-16T20:52:02.742331+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:35:02.412998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091339500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"218'1028","reported_seq":19427,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.912471+0000","last_change":"2024-09-16T20:51:45.582985+0000","last_active":"2024-09-16T20:52:02.912471+0000","last_peered":"2024-09-16T20:52:02.912471+0000","last_clean":"2024-09-16T20:52:02.912471+0000","last_became_active":"2024-09-16T20:51:16.577104+0000","last_became_peered":"2024-09-16T20:51:16.577104+0000","last_unstale":"2024-09-16T20:52:02.912471+0000","last_undegraded":"2024-09-16T20:52:02.912471+0000","last_fullsized":"2024-09-16T20:52:02.912471+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":1028,"log_dups_size":0,"ondisk_log_size":1028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:56:34.623379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013907780000000001,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":20870,"num_read_kb":85867,"num_write":1218,"num_write_kb":25908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":29601482,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":463,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.912490+0000","last_change":"2024-09-16T20:51:00.577927+0000","last_active":"2024-09-16T20:52:02.912490+0000","last_peered":"2024-09-16T20:52:02.912490+0000","last_clean":"2024-09-16T20:52:02.912490+0000","last_became_active":"2024-09-16T20:51:00.577653+0000","last_became_peered":"2024-09-16T20:51:00.577653+0000","last_unstale":"2024-09-16T20:52:02.912490+0000","last_undegraded":"2024-09-16T20:52:02.912490+0000","last_fullsized":"2024-09-16T20:52:02.912490+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:12:13.831668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043659299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"214'653","reported_seq":25408,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.912290+0000","last_change":"2024-09-16T20:51:45.583929+0000","last_active":"2024-09-16T20:52:02.912290+0000","last_peered":"2024-09-16T20:52:02.912290+0000","last_clean":"2024-09-16T20:52:02.912290+0000","last_became_active":"2024-09-16T20:51:15.575607+0000","last_became_peered":"2024-09-16T20:51:15.575607+0000","last_unstale":"2024-09-16T20:52:02.912290+0000","last_undegraded":"2024-09-16T20:52:02.912290+0000","last_fullsized":"2024-09-16T20:52:02.912290+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":653,"log_dups_size":0,"ondisk_log_size":653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:42:18.817016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022924479999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":24178,"num_read_kb":81081,"num_write":502,"num_write_kb":17573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":20407955,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":448,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.742362+0000","last_change":"2024-09-16T20:48:00.278054+0000","last_active":"2024-09-16T20:52:02.742362+0000","last_peered":"2024-09-16T20:52:02.742362+0000","last_clean":"2024-09-16T20:52:02.742362+0000","last_became_active":"2024-09-16T20:47:58.272861+0000","last_became_peered":"2024-09-16T20:47:58.272861+0000","last_unstale":"2024-09-16T20:52:02.742362+0000","last_undegraded":"2024-09-16T20:52:02.742362+0000","last_fullsized":"2024-09-16T20:52:02.742362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:13:59.135547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012429489999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"213'491","reported_seq":3884,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.912098+0000","last_change":"2024-09-16T20:51:45.583107+0000","last_active":"2024-09-16T20:52:02.912098+0000","last_peered":"2024-09-16T20:52:02.912098+0000","last_clean":"2024-09-16T20:52:02.912098+0000","last_became_active":"2024-09-16T20:49:12.801810+0000","last_became_peered":"2024-09-16T20:49:12.801810+0000","last_unstale":"2024-09-16T20:52:02.912098+0000","last_undegraded":"2024-09-16T20:52:02.912098+0000","last_fullsized":"2024-09-16T20:52:02.912098+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":491,"log_dups_size":0,"ondisk_log_size":491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:48:19.573269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015370990000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":3741,"num_read_kb":11405,"num_write":512,"num_write_kb":6561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":2071114,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":340,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911584+0000","last_change":"2024-09-16T20:51:12.615838+0000","last_active":"2024-09-16T20:52:02.911584+0000","last_peered":"2024-09-16T20:52:02.911584+0000","last_clean":"2024-09-16T20:52:02.911584+0000","last_became_active":"2024-09-16T20:51:12.615412+0000","last_became_peered":"2024-09-16T20:51:12.615412+0000","last_unstale":"2024-09-16T20:52:02.911584+0000","last_undegraded":"2024-09-16T20:52:02.911584+0000","last_fullsized":"2024-09-16T20:52:02.911584+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:17:12.150193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"217'668","reported_seq":19715,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.741476+0000","last_change":"2024-09-16T20:51:45.580833+0000","last_active":"2024-09-16T20:52:02.741476+0000","last_peered":"2024-09-16T20:52:02.741476+0000","last_clean":"2024-09-16T20:52:02.741476+0000","last_became_active":"2024-09-16T20:51:27.624519+0000","last_became_peered":"2024-09-16T20:51:27.624519+0000","last_unstale":"2024-09-16T20:52:02.741476+0000","last_undegraded":"2024-09-16T20:52:02.741476+0000","last_fullsized":"2024-09-16T20:52:02.741476+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":668,"log_dups_size":0,"ondisk_log_size":668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:44:59.098443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001777441,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":18081,"num_read_kb":57429,"num_write":654,"num_write_kb":12584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":564,"num_bytes_recovered":27092048,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"218'728","reported_seq":8639,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911256+0000","last_change":"2024-09-16T20:51:45.580666+0000","last_active":"2024-09-16T20:52:02.911256+0000","last_peered":"2024-09-16T20:52:02.911256+0000","last_clean":"2024-09-16T20:52:02.911256+0000","last_became_active":"2024-09-16T20:51:27.626032+0000","last_became_peered":"2024-09-16T20:51:27.626032+0000","last_unstale":"2024-09-16T20:52:02.911256+0000","last_undegraded":"2024-09-16T20:52:02.911256+0000","last_fullsized":"2024-09-16T20:52:02.911256+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":728,"log_dups_size":0,"ondisk_log_size":728,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:28:16.242279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025014909999999998,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":7673,"num_read_kb":36418,"num_write":666,"num_write_kb":13145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":523,"num_bytes_recovered":11083665,"num_keys_recovered":14,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":137,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911158+0000","last_change":"2024-09-16T20:51:00.578241+0000","last_active":"2024-09-16T20:52:02.911158+0000","last_peered":"2024-09-16T20:52:02.911158+0000","last_clean":"2024-09-16T20:52:02.911158+0000","last_became_active":"2024-09-16T20:51:00.574693+0000","last_became_peered":"2024-09-16T20:51:00.574693+0000","last_unstale":"2024-09-16T20:52:02.911158+0000","last_undegraded":"2024-09-16T20:52:02.911158+0000","last_fullsized":"2024-09-16T20:52:02.911158+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:35:20.550930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062077499999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"216'498","reported_seq":18808,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911630+0000","last_change":"2024-09-16T20:51:45.643842+0000","last_active":"2024-09-16T20:52:02.911630+0000","last_peered":"2024-09-16T20:52:02.911630+0000","last_clean":"2024-09-16T20:52:02.911630+0000","last_became_active":"2024-09-16T20:51:26.626185+0000","last_became_peered":"2024-09-16T20:51:26.626185+0000","last_unstale":"2024-09-16T20:52:02.911630+0000","last_undegraded":"2024-09-16T20:52:02.911630+0000","last_fullsized":"2024-09-16T20:52:02.911630+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":498,"log_dups_size":0,"ondisk_log_size":498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:57:30.480666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023862639999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":19429,"num_read_kb":77391,"num_write":450,"num_write_kb":15765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":3862081,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":109,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911799+0000","last_change":"2024-09-16T20:51:12.615208+0000","last_active":"2024-09-16T20:52:02.911799+0000","last_peered":"2024-09-16T20:52:02.911799+0000","last_clean":"2024-09-16T20:52:02.911799+0000","last_became_active":"2024-09-16T20:51:12.614812+0000","last_became_peered":"2024-09-16T20:51:12.614812+0000","last_unstale":"2024-09-16T20:52:02.911799+0000","last_undegraded":"2024-09-16T20:52:02.911799+0000","last_fullsized":"2024-09-16T20:52:02.911799+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:37:11.741478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"215'312","reported_seq":5919,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.912408+0000","last_change":"2024-09-16T20:51:45.584820+0000","last_active":"2024-09-16T20:52:02.912408+0000","last_peered":"2024-09-16T20:52:02.912408+0000","last_clean":"2024-09-16T20:52:02.912408+0000","last_became_active":"2024-09-16T20:51:26.624469+0000","last_became_peered":"2024-09-16T20:51:26.624469+0000","last_unstale":"2024-09-16T20:52:02.912408+0000","last_undegraded":"2024-09-16T20:52:02.912408+0000","last_fullsized":"2024-09-16T20:52:02.912408+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:52:43.183901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025837220000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4323,"num_read_kb":33093,"num_write":273,"num_write_kb":12351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":324,"num_bytes_recovered":16931328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":89,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.741294+0000","last_change":"2024-09-16T20:51:23.608479+0000","last_active":"2024-09-16T20:52:02.741294+0000","last_peered":"2024-09-16T20:52:02.741294+0000","last_clean":"2024-09-16T20:52:02.741294+0000","last_became_active":"2024-09-16T20:51:23.608159+0000","last_became_peered":"2024-09-16T20:51:23.608159+0000","last_unstale":"2024-09-16T20:52:02.741294+0000","last_undegraded":"2024-09-16T20:52:02.741294+0000","last_fullsized":"2024-09-16T20:52:02.741294+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:06:21.085498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037636000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"217'609","reported_seq":7615,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911699+0000","last_change":"2024-09-16T20:51:45.582309+0000","last_active":"2024-09-16T20:52:02.911699+0000","last_peered":"2024-09-16T20:52:02.911699+0000","last_clean":"2024-09-16T20:52:02.911699+0000","last_became_active":"2024-09-16T20:51:26.622629+0000","last_became_peered":"2024-09-16T20:51:26.622629+0000","last_unstale":"2024-09-16T20:52:02.911699+0000","last_undegraded":"2024-09-16T20:52:02.911699+0000","last_fullsized":"2024-09-16T20:52:02.911699+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:40:49.603343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029005419999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":9084,"num_read_kb":37091,"num_write":448,"num_write_kb":13187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":848,"num_bytes_recovered":24298320,"num_keys_recovered":219,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":469,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.742037+0000","last_change":"2024-09-16T20:51:23.608488+0000","last_active":"2024-09-16T20:52:02.742037+0000","last_peered":"2024-09-16T20:52:02.742037+0000","last_clean":"2024-09-16T20:52:02.742037+0000","last_became_active":"2024-09-16T20:51:23.605321+0000","last_became_peered":"2024-09-16T20:51:23.605321+0000","last_unstale":"2024-09-16T20:52:02.742037+0000","last_undegraded":"2024-09-16T20:52:02.742037+0000","last_fullsized":"2024-09-16T20:52:02.742037+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:50:46.322344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095771799999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":564,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.741984+0000","last_change":"2024-09-16T20:49:09.565188+0000","last_active":"2024-09-16T20:52:02.741984+0000","last_peered":"2024-09-16T20:52:02.741984+0000","last_clean":"2024-09-16T20:52:02.741984+0000","last_became_active":"2024-09-16T20:49:09.559571+0000","last_became_peered":"2024-09-16T20:49:09.559571+0000","last_unstale":"2024-09-16T20:52:02.741984+0000","last_undegraded":"2024-09-16T20:52:02.741984+0000","last_fullsized":"2024-09-16T20:52:02.741984+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:52.241408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:52.241408+0000","last_clean_scrub_stamp":"2024-09-16T20:47:52.241408+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:01:53.651981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"214'641","reported_seq":7064,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.912231+0000","last_change":"2024-09-16T20:51:45.583936+0000","last_active":"2024-09-16T20:52:02.912231+0000","last_peered":"2024-09-16T20:52:02.912231+0000","last_clean":"2024-09-16T20:52:02.912231+0000","last_became_active":"2024-09-16T20:51:27.624220+0000","last_became_peered":"2024-09-16T20:51:27.624220+0000","last_unstale":"2024-09-16T20:52:02.912231+0000","last_undegraded":"2024-09-16T20:52:02.912231+0000","last_fullsized":"2024-09-16T20:52:02.912231+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":641,"log_dups_size":0,"ondisk_log_size":641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:13:47.571246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022745209999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6921,"num_read_kb":32984,"num_write":827,"num_write_kb":7103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":4839295,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"217'639","reported_seq":16470,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911021+0000","last_change":"2024-09-16T20:51:45.578750+0000","last_active":"2024-09-16T20:52:02.911021+0000","last_peered":"2024-09-16T20:52:02.911021+0000","last_clean":"2024-09-16T20:52:02.911021+0000","last_became_active":"2024-09-16T20:51:17.575136+0000","last_became_peered":"2024-09-16T20:51:17.575136+0000","last_unstale":"2024-09-16T20:52:02.911021+0000","last_undegraded":"2024-09-16T20:52:02.911021+0000","last_fullsized":"2024-09-16T20:52:02.911021+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":639,"log_dups_size":0,"ondisk_log_size":639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:32:32.550808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022726539999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":15159,"num_read_kb":101301,"num_write":561,"num_write_kb":29560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":790,"num_bytes_recovered":48497898,"num_keys_recovered":60,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"217'568","reported_seq":9847,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911042+0000","last_change":"2024-09-16T20:51:45.578068+0000","last_active":"2024-09-16T20:52:02.911042+0000","last_peered":"2024-09-16T20:52:02.911042+0000","last_clean":"2024-09-16T20:52:02.911042+0000","last_became_active":"2024-09-16T20:51:03.590152+0000","last_became_peered":"2024-09-16T20:51:03.590152+0000","last_unstale":"2024-09-16T20:52:02.911042+0000","last_undegraded":"2024-09-16T20:52:02.911042+0000","last_fullsized":"2024-09-16T20:52:02.911042+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":568,"log_dups_size":0,"ondisk_log_size":568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:21:38.569104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017317439999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":9605,"num_read_kb":51536,"num_write":618,"num_write_kb":12681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":14274951,"num_keys_recovered":91,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"218'468","reported_seq":8097,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.912037+0000","last_change":"2024-09-16T20:51:45.582215+0000","last_active":"2024-09-16T20:52:02.912037+0000","last_peered":"2024-09-16T20:52:02.912037+0000","last_clean":"2024-09-16T20:52:02.912037+0000","last_became_active":"2024-09-16T20:51:16.575698+0000","last_became_peered":"2024-09-16T20:51:16.575698+0000","last_unstale":"2024-09-16T20:52:02.912037+0000","last_undegraded":"2024-09-16T20:52:02.912037+0000","last_fullsized":"2024-09-16T20:52:02.912037+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:15:28.632535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020731130000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":7052,"num_read_kb":42310,"num_write":428,"num_write_kb":14897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":445,"num_bytes_recovered":8390493,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"216'726","reported_seq":22773,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911294+0000","last_change":"2024-09-16T20:51:45.579437+0000","last_active":"2024-09-16T20:52:02.911294+0000","last_peered":"2024-09-16T20:52:02.911294+0000","last_clean":"2024-09-16T20:52:02.911294+0000","last_became_active":"2024-09-16T20:51:17.575222+0000","last_became_peered":"2024-09-16T20:51:17.575222+0000","last_unstale":"2024-09-16T20:52:02.911294+0000","last_undegraded":"2024-09-16T20:52:02.911294+0000","last_fullsized":"2024-09-16T20:52:02.911294+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":726,"log_dups_size":0,"ondisk_log_size":726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:44:38.139123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018824849999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":22418,"num_read_kb":91945,"num_write":909,"num_write_kb":19380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":627,"num_bytes_recovered":71963834,"num_keys_recovered":119,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"218'1128","reported_seq":18644,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.742261+0000","last_change":"2024-09-16T20:51:45.585311+0000","last_active":"2024-09-16T20:52:02.742261+0000","last_peered":"2024-09-16T20:52:02.742261+0000","last_clean":"2024-09-16T20:52:02.742261+0000","last_became_active":"2024-09-16T20:51:05.581172+0000","last_became_peered":"2024-09-16T20:51:05.581172+0000","last_unstale":"2024-09-16T20:52:02.742261+0000","last_undegraded":"2024-09-16T20:52:02.742261+0000","last_fullsized":"2024-09-16T20:52:02.742261+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:27:15.820301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002500038,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":19435,"num_read_kb":70139,"num_write":1420,"num_write_kb":21576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":16987444,"num_keys_recovered":40,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"217'581","reported_seq":9275,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.741548+0000","last_change":"2024-09-16T20:51:45.581620+0000","last_active":"2024-09-16T20:52:02.741548+0000","last_peered":"2024-09-16T20:52:02.741548+0000","last_clean":"2024-09-16T20:52:02.741548+0000","last_became_active":"2024-09-16T20:51:27.627520+0000","last_became_peered":"2024-09-16T20:51:27.627520+0000","last_unstale":"2024-09-16T20:52:02.741548+0000","last_undegraded":"2024-09-16T20:52:02.741548+0000","last_fullsized":"2024-09-16T20:52:02.741548+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":581,"log_dups_size":0,"ondisk_log_size":581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:30:31.740673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012113930000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":8329,"num_read_kb":54907,"num_write":641,"num_write_kb":11423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":479,"num_bytes_recovered":14706083,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"217'534","reported_seq":13756,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911354+0000","last_change":"2024-09-16T20:51:45.580500+0000","last_active":"2024-09-16T20:52:02.911354+0000","last_peered":"2024-09-16T20:52:02.911354+0000","last_clean":"2024-09-16T20:52:02.911354+0000","last_became_active":"2024-09-16T20:51:05.581058+0000","last_became_peered":"2024-09-16T20:51:05.581058+0000","last_unstale":"2024-09-16T20:52:02.911354+0000","last_undegraded":"2024-09-16T20:52:02.911354+0000","last_fullsized":"2024-09-16T20:52:02.911354+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:55:20.810743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026424640000000002,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":13544,"num_read_kb":49574,"num_write":426,"num_write_kb":14576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":284,"num_bytes_recovered":9641443,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"218'691","reported_seq":13753,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.741972+0000","last_change":"2024-09-16T20:51:45.584709+0000","last_active":"2024-09-16T20:52:02.741972+0000","last_peered":"2024-09-16T20:52:02.741972+0000","last_clean":"2024-09-16T20:52:02.741972+0000","last_became_active":"2024-09-16T20:49:12.801013+0000","last_became_peered":"2024-09-16T20:49:12.801013+0000","last_unstale":"2024-09-16T20:52:02.741972+0000","last_undegraded":"2024-09-16T20:52:02.741972+0000","last_fullsized":"2024-09-16T20:52:02.741972+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":691,"log_dups_size":0,"ondisk_log_size":691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:13:59.069699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001757113,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":15597,"num_read_kb":58377,"num_write":523,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":99,"num_bytes_recovered":5210742,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"218'888","reported_seq":14337,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.741292+0000","last_change":"2024-09-16T20:51:45.579465+0000","last_active":"2024-09-16T20:52:02.741292+0000","last_peered":"2024-09-16T20:52:02.741292+0000","last_clean":"2024-09-16T20:52:02.741292+0000","last_became_active":"2024-09-16T20:51:26.624407+0000","last_became_peered":"2024-09-16T20:51:26.624407+0000","last_unstale":"2024-09-16T20:52:02.741292+0000","last_undegraded":"2024-09-16T20:52:02.741292+0000","last_fullsized":"2024-09-16T20:52:02.741292+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:59:50.323236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029313379999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":16446,"num_read_kb":75991,"num_write":874,"num_write_kb":24549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":351,"num_bytes_recovered":38747648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"211'608","reported_seq":20163,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911705+0000","last_change":"2024-09-16T20:51:45.643842+0000","last_active":"2024-09-16T20:52:02.911705+0000","last_peered":"2024-09-16T20:52:02.911705+0000","last_clean":"2024-09-16T20:52:02.911705+0000","last_became_active":"2024-09-16T20:51:03.589742+0000","last_became_peered":"2024-09-16T20:51:03.589742+0000","last_unstale":"2024-09-16T20:52:02.911705+0000","last_undegraded":"2024-09-16T20:52:02.911705+0000","last_fullsized":"2024-09-16T20:52:02.911705+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":608,"log_dups_size":0,"ondisk_log_size":608,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:49:22.579735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024335089999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":16969,"num_read_kb":76220,"num_write":613,"num_write_kb":13841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":486,"num_bytes_recovered":15329756,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"216'461","reported_seq":12004,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.741986+0000","last_change":"2024-09-16T20:51:45.586711+0000","last_active":"2024-09-16T20:52:02.741986+0000","last_peered":"2024-09-16T20:52:02.741986+0000","last_clean":"2024-09-16T20:52:02.741986+0000","last_became_active":"2024-09-16T20:51:03.595360+0000","last_became_peered":"2024-09-16T20:51:03.595360+0000","last_unstale":"2024-09-16T20:52:02.741986+0000","last_undegraded":"2024-09-16T20:52:02.741986+0000","last_fullsized":"2024-09-16T20:52:02.741986+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":461,"log_dups_size":0,"ondisk_log_size":461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:47:40.600095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035295309999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":11262,"num_read_kb":35639,"num_write":562,"num_write_kb":13645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":198,"num_bytes_recovered":14217900,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":247,"num_object_clones":0,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":243,"num_read":448597,"num_read_kb":1798155,"num_write":23462,"num_write_kb":469671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12645,"num_bytes_recovered":606441956,"num_keys_recovered":1497,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22149,"ondisk_log_size":22149,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574904,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52588,"kb_used_data":3344,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319252,"statfs":{"total":96636764160,"available":96582914048,"internally_reserved":0,"allocated":3424256,"data_stored":1774023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574904,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74140,"kb_used_data":2760,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94297700,"statfs":{"total":96636764160,"available":96560844800,"internally_reserved":0,"allocated":2826240,"data_stored":1183636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73071047},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574904,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69980,"kb_used_data":2760,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94301860,"statfs":{"total":96636764160,"available":96565104640,"internally_reserved":0,"allocated":2826240,"data_stored":1183636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574903,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70568,"kb_used_data":3344,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94301272,"statfs":{"total":96636764160,"available":96564502528,"internally_reserved":0,"allocated":3424256,"data_stored":1774023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574904,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":96420,"kb_used_data":2764,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":94275420,"statfs":{"total":96636764160,"available":96538030080,"internally_reserved":0,"allocated":2830336,"data_stored":1183655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":95877574},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640311,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60320,"kb_used_data":2764,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311520,"statfs":{"total":96636764160,"available":96574996480,"internally_reserved":0,"allocated":2830336,"data_stored":1183655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T20:52:06.295 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-16T20:52:06.295 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-16T20:52:06.612 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-16T20:52:06.622 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T20:52:06.622 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T20:52:06.923 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:52:06.924 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-09-16T20:52:06.935 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":359,"stamp":"2024-09-16T20:52:06.249925+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":251,"num_object_clones":0,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":243,"num_read":448663,"num_read_kb":1798210,"num_write":23565,"num_write_kb":471301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12652,"num_bytes_recovered":607622711,"num_keys_recovered":1497,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22224,"ondisk_log_size":22224,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":424016,"kb_used_data":17736,"kb_used_omap":9,"kb_used_meta":406134,"kb_avail":565807024,"statfs":{"total":579820584960,"available":579386392576,"internally_reserved":0,"allocated":18161664,"data_stored":8282628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9567,"internal_metadata":415881889},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001317"},"pg_stats":[{"pgid":"3.1f","version":"218'359","reported_seq":10822,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911515+0000","last_change":"2024-09-16T20:51:45.642922+0000","last_active":"2024-09-16T20:52:02.911515+0000","last_peered":"2024-09-16T20:52:02.911515+0000","last_clean":"2024-09-16T20:52:02.911515+0000","last_became_active":"2024-09-16T20:49:12.801566+0000","last_became_peered":"2024-09-16T20:49:12.801566+0000","last_unstale":"2024-09-16T20:52:02.911515+0000","last_undegraded":"2024-09-16T20:52:02.911515+0000","last_fullsized":"2024-09-16T20:52:02.911515+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:20:11.666141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016075530000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10091,"num_read_kb":31685,"num_write":315,"num_write_kb":5043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":2125998,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"218'905","reported_seq":28772,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.912100+0000","last_change":"2024-09-16T20:51:45.580852+0000","last_active":"2024-09-16T20:52:02.912100+0000","last_peered":"2024-09-16T20:52:02.912100+0000","last_clean":"2024-09-16T20:52:02.912100+0000","last_became_active":"2024-09-16T20:51:27.628085+0000","last_became_peered":"2024-09-16T20:51:27.628085+0000","last_unstale":"2024-09-16T20:52:02.912100+0000","last_undegraded":"2024-09-16T20:52:02.912100+0000","last_fullsized":"2024-09-16T20:52:02.912100+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":905,"log_dups_size":0,"ondisk_log_size":905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:59:41.269032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001444383,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":29337,"num_read_kb":83641,"num_write":1040,"num_write_kb":10328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":562,"num_bytes_recovered":11317531,"num_keys_recovered":154,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"214'707","reported_seq":12942,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.741297+0000","last_change":"2024-09-16T20:51:45.580224+0000","last_active":"2024-09-16T20:52:02.741297+0000","last_peered":"2024-09-16T20:52:02.741297+0000","last_clean":"2024-09-16T20:52:02.741297+0000","last_became_active":"2024-09-16T20:51:26.624314+0000","last_became_peered":"2024-09-16T20:51:26.624314+0000","last_unstale":"2024-09-16T20:52:02.741297+0000","last_undegraded":"2024-09-16T20:52:02.741297+0000","last_fullsized":"2024-09-16T20:52:02.741297+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":707,"log_dups_size":0,"ondisk_log_size":707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:47:19.682479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014726419999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":13066,"num_read_kb":103238,"num_write":750,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":387,"num_bytes_recovered":35619030,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"218'714","reported_seq":4372,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.742231+0000","last_change":"2024-09-16T20:51:45.586449+0000","last_active":"2024-09-16T20:52:02.742231+0000","last_peered":"2024-09-16T20:52:02.742231+0000","last_clean":"2024-09-16T20:52:02.742231+0000","last_became_active":"2024-09-16T20:51:04.577608+0000","last_became_peered":"2024-09-16T20:51:04.577608+0000","last_unstale":"2024-09-16T20:52:02.742231+0000","last_undegraded":"2024-09-16T20:52:02.742231+0000","last_fullsized":"2024-09-16T20:52:02.742231+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:15:00.043848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034574200000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":4615,"num_read_kb":24234,"num_write":996,"num_write_kb":15567,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":25643873,"num_keys_recovered":142,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"217'872","reported_seq":11160,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911930+0000","last_change":"2024-09-16T20:51:49.582917+0000","last_active":"2024-09-16T20:52:02.911930+0000","last_peered":"2024-09-16T20:52:02.911930+0000","last_clean":"2024-09-16T20:52:02.911930+0000","last_became_active":"2024-09-16T20:51:49.579980+0000","last_became_peered":"2024-09-16T20:51:49.579980+0000","last_unstale":"2024-09-16T20:52:02.911930+0000","last_undegraded":"2024-09-16T20:52:02.911930+0000","last_fullsized":"2024-09-16T20:52:02.911930+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":872,"log_dups_size":0,"ondisk_log_size":872,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:13:38.941261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001184378,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":10970,"num_read_kb":44437,"num_write":1151,"num_write_kb":16781,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":956,"num_bytes_recovered":37659232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"217'1221","reported_seq":19862,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.742042+0000","last_change":"2024-09-16T20:51:45.585643+0000","last_active":"2024-09-16T20:52:02.742042+0000","last_peered":"2024-09-16T20:52:02.742042+0000","last_clean":"2024-09-16T20:52:02.742042+0000","last_became_active":"2024-09-16T20:49:12.802869+0000","last_became_peered":"2024-09-16T20:49:12.802869+0000","last_unstale":"2024-09-16T20:52:02.742042+0000","last_undegraded":"2024-09-16T20:52:02.742042+0000","last_fullsized":"2024-09-16T20:52:02.742042+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":1221,"log_dups_size":0,"ondisk_log_size":1221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:17:13.191250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026611790000000001,"stat_sum":{"num_bytes":19,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":25740,"num_read_kb":67599,"num_write":1269,"num_write_kb":16392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":1051004,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"218'770","reported_seq":12989,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.912288+0000","last_change":"2024-09-16T20:51:45.584848+0000","last_active":"2024-09-16T20:52:02.912288+0000","last_peered":"2024-09-16T20:52:02.912288+0000","last_clean":"2024-09-16T20:52:02.912288+0000","last_became_active":"2024-09-16T20:51:16.577039+0000","last_became_peered":"2024-09-16T20:51:16.577039+0000","last_unstale":"2024-09-16T20:52:02.912288+0000","last_undegraded":"2024-09-16T20:52:02.912288+0000","last_fullsized":"2024-09-16T20:52:02.912288+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":770,"log_dups_size":0,"ondisk_log_size":770,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:26:39.219016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023604640000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":10011,"num_read_kb":47438,"num_write":753,"num_write_kb":12180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":12922788,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"216'458","reported_seq":13762,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911501+0000","last_change":"2024-09-16T20:51:45.581009+0000","last_active":"2024-09-16T20:52:02.911501+0000","last_peered":"2024-09-16T20:52:02.911501+0000","last_clean":"2024-09-16T20:52:02.911501+0000","last_became_active":"2024-09-16T20:51:15.575429+0000","last_became_peered":"2024-09-16T20:51:15.575429+0000","last_unstale":"2024-09-16T20:52:02.911501+0000","last_undegraded":"2024-09-16T20:52:02.911501+0000","last_fullsized":"2024-09-16T20:52:02.911501+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":458,"log_dups_size":0,"ondisk_log_size":458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:42:42.274481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001906629,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":14274,"num_read_kb":52840,"num_write":349,"num_write_kb":9355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":231,"num_bytes_recovered":11122736,"num_keys_recovered":66,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"217'981","reported_seq":8058,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911740+0000","last_change":"2024-09-16T20:51:45.642868+0000","last_active":"2024-09-16T20:52:02.911740+0000","last_peered":"2024-09-16T20:52:02.911740+0000","last_clean":"2024-09-16T20:52:02.911740+0000","last_became_active":"2024-09-16T20:51:03.592284+0000","last_became_peered":"2024-09-16T20:51:03.592284+0000","last_unstale":"2024-09-16T20:52:02.911740+0000","last_undegraded":"2024-09-16T20:52:02.911740+0000","last_fullsized":"2024-09-16T20:52:02.911740+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":981,"log_dups_size":0,"ondisk_log_size":981,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:35:23.178350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015146339999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":11609,"num_read_kb":15811,"num_write":975,"num_write_kb":5184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":280,"num_bytes_recovered":1580920,"num_keys_recovered":144,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"217'1277","reported_seq":11828,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.741549+0000","last_change":"2024-09-16T20:51:45.581650+0000","last_active":"2024-09-16T20:52:02.741549+0000","last_peered":"2024-09-16T20:52:02.741549+0000","last_clean":"2024-09-16T20:52:02.741549+0000","last_became_active":"2024-09-16T20:51:27.624341+0000","last_became_peered":"2024-09-16T20:51:27.624341+0000","last_unstale":"2024-09-16T20:52:02.741549+0000","last_undegraded":"2024-09-16T20:52:02.741549+0000","last_fullsized":"2024-09-16T20:52:02.741549+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":1277,"log_dups_size":0,"ondisk_log_size":1277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:01:31.923277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011831579999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":17170,"num_read_kb":43312,"num_write":1584,"num_write_kb":10211,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":542,"num_bytes_recovered":21909352,"num_keys_recovered":70,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"218'595","reported_seq":14001,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911281+0000","last_change":"2024-09-16T20:51:45.579668+0000","last_active":"2024-09-16T20:52:02.911281+0000","last_peered":"2024-09-16T20:52:02.911281+0000","last_clean":"2024-09-16T20:52:02.911281+0000","last_became_active":"2024-09-16T20:51:16.577620+0000","last_became_peered":"2024-09-16T20:51:16.577620+0000","last_unstale":"2024-09-16T20:52:02.911281+0000","last_undegraded":"2024-09-16T20:52:02.911281+0000","last_fullsized":"2024-09-16T20:52:02.911281+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":595,"log_dups_size":0,"ondisk_log_size":595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:23:49.832618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022662979999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":12843,"num_read_kb":77657,"num_write":690,"num_write_kb":15118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":562,"num_bytes_recovered":17171700,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"218'370","reported_seq":19983,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.742221+0000","last_change":"2024-09-16T20:51:57.720592+0000","last_active":"2024-09-16T20:52:02.742221+0000","last_peered":"2024-09-16T20:52:02.742221+0000","last_clean":"2024-09-16T20:52:02.742221+0000","last_became_active":"2024-09-16T20:51:57.718930+0000","last_became_peered":"2024-09-16T20:51:57.718930+0000","last_unstale":"2024-09-16T20:52:02.742221+0000","last_undegraded":"2024-09-16T20:52:02.742221+0000","last_fullsized":"2024-09-16T20:52:02.742221+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:08:04.217338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014936070000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":18755,"num_read_kb":45565,"num_write":465,"num_write_kb":13208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":540,"num_bytes_recovered":30162312,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":460,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.742331+0000","last_change":"2024-09-16T20:51:00.578451+0000","last_active":"2024-09-16T20:52:02.742331+0000","last_peered":"2024-09-16T20:52:02.742331+0000","last_clean":"2024-09-16T20:52:02.742331+0000","last_became_active":"2024-09-16T20:51:00.578081+0000","last_became_peered":"2024-09-16T20:51:00.578081+0000","last_unstale":"2024-09-16T20:52:02.742331+0000","last_undegraded":"2024-09-16T20:52:02.742331+0000","last_fullsized":"2024-09-16T20:52:02.742331+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:35:02.412998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091339500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"218'1028","reported_seq":19427,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.912471+0000","last_change":"2024-09-16T20:51:45.582985+0000","last_active":"2024-09-16T20:52:02.912471+0000","last_peered":"2024-09-16T20:52:02.912471+0000","last_clean":"2024-09-16T20:52:02.912471+0000","last_became_active":"2024-09-16T20:51:16.577104+0000","last_became_peered":"2024-09-16T20:51:16.577104+0000","last_unstale":"2024-09-16T20:52:02.912471+0000","last_undegraded":"2024-09-16T20:52:02.912471+0000","last_fullsized":"2024-09-16T20:52:02.912471+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":1028,"log_dups_size":0,"ondisk_log_size":1028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:56:34.623379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013907780000000001,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":20870,"num_read_kb":85867,"num_write":1218,"num_write_kb":25908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":29601482,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":463,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.912490+0000","last_change":"2024-09-16T20:51:00.577927+0000","last_active":"2024-09-16T20:52:02.912490+0000","last_peered":"2024-09-16T20:52:02.912490+0000","last_clean":"2024-09-16T20:52:02.912490+0000","last_became_active":"2024-09-16T20:51:00.577653+0000","last_became_peered":"2024-09-16T20:51:00.577653+0000","last_unstale":"2024-09-16T20:52:02.912490+0000","last_undegraded":"2024-09-16T20:52:02.912490+0000","last_fullsized":"2024-09-16T20:52:02.912490+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:12:13.831668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043659299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"214'653","reported_seq":25408,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.912290+0000","last_change":"2024-09-16T20:51:45.583929+0000","last_active":"2024-09-16T20:52:02.912290+0000","last_peered":"2024-09-16T20:52:02.912290+0000","last_clean":"2024-09-16T20:52:02.912290+0000","last_became_active":"2024-09-16T20:51:15.575607+0000","last_became_peered":"2024-09-16T20:51:15.575607+0000","last_unstale":"2024-09-16T20:52:02.912290+0000","last_undegraded":"2024-09-16T20:52:02.912290+0000","last_fullsized":"2024-09-16T20:52:02.912290+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":653,"log_dups_size":0,"ondisk_log_size":653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:42:18.817016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022924479999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":24178,"num_read_kb":81081,"num_write":502,"num_write_kb":17573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":20407955,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":448,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.742362+0000","last_change":"2024-09-16T20:48:00.278054+0000","last_active":"2024-09-16T20:52:02.742362+0000","last_peered":"2024-09-16T20:52:02.742362+0000","last_clean":"2024-09-16T20:52:02.742362+0000","last_became_active":"2024-09-16T20:47:58.272861+0000","last_became_peered":"2024-09-16T20:47:58.272861+0000","last_unstale":"2024-09-16T20:52:02.742362+0000","last_undegraded":"2024-09-16T20:52:02.742362+0000","last_fullsized":"2024-09-16T20:52:02.742362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:13:59.135547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012429489999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"213'491","reported_seq":3884,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.912098+0000","last_change":"2024-09-16T20:51:45.583107+0000","last_active":"2024-09-16T20:52:02.912098+0000","last_peered":"2024-09-16T20:52:02.912098+0000","last_clean":"2024-09-16T20:52:02.912098+0000","last_became_active":"2024-09-16T20:49:12.801810+0000","last_became_peered":"2024-09-16T20:49:12.801810+0000","last_unstale":"2024-09-16T20:52:02.912098+0000","last_undegraded":"2024-09-16T20:52:02.912098+0000","last_fullsized":"2024-09-16T20:52:02.912098+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":491,"log_dups_size":0,"ondisk_log_size":491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:48:19.573269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015370990000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":3741,"num_read_kb":11405,"num_write":512,"num_write_kb":6561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":2071114,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":340,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911584+0000","last_change":"2024-09-16T20:51:12.615838+0000","last_active":"2024-09-16T20:52:02.911584+0000","last_peered":"2024-09-16T20:52:02.911584+0000","last_clean":"2024-09-16T20:52:02.911584+0000","last_became_active":"2024-09-16T20:51:12.615412+0000","last_became_peered":"2024-09-16T20:51:12.615412+0000","last_unstale":"2024-09-16T20:52:02.911584+0000","last_undegraded":"2024-09-16T20:52:02.911584+0000","last_fullsized":"2024-09-16T20:52:02.911584+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:17:12.150193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"217'668","reported_seq":19715,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.741476+0000","last_change":"2024-09-16T20:51:45.580833+0000","last_active":"2024-09-16T20:52:02.741476+0000","last_peered":"2024-09-16T20:52:02.741476+0000","last_clean":"2024-09-16T20:52:02.741476+0000","last_became_active":"2024-09-16T20:51:27.624519+0000","last_became_peered":"2024-09-16T20:51:27.624519+0000","last_unstale":"2024-09-16T20:52:02.741476+0000","last_undegraded":"2024-09-16T20:52:02.741476+0000","last_fullsized":"2024-09-16T20:52:02.741476+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":668,"log_dups_size":0,"ondisk_log_size":668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:44:59.098443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001777441,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":18081,"num_read_kb":57429,"num_write":654,"num_write_kb":12584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":564,"num_bytes_recovered":27092048,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"218'728","reported_seq":8639,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911256+0000","last_change":"2024-09-16T20:51:45.580666+0000","last_active":"2024-09-16T20:52:02.911256+0000","last_peered":"2024-09-16T20:52:02.911256+0000","last_clean":"2024-09-16T20:52:02.911256+0000","last_became_active":"2024-09-16T20:51:27.626032+0000","last_became_peered":"2024-09-16T20:51:27.626032+0000","last_unstale":"2024-09-16T20:52:02.911256+0000","last_undegraded":"2024-09-16T20:52:02.911256+0000","last_fullsized":"2024-09-16T20:52:02.911256+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":728,"log_dups_size":0,"ondisk_log_size":728,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:28:16.242279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025014909999999998,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":7673,"num_read_kb":36418,"num_write":666,"num_write_kb":13145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":523,"num_bytes_recovered":11083665,"num_keys_recovered":14,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":137,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911158+0000","last_change":"2024-09-16T20:51:00.578241+0000","last_active":"2024-09-16T20:52:02.911158+0000","last_peered":"2024-09-16T20:52:02.911158+0000","last_clean":"2024-09-16T20:52:02.911158+0000","last_became_active":"2024-09-16T20:51:00.574693+0000","last_became_peered":"2024-09-16T20:51:00.574693+0000","last_unstale":"2024-09-16T20:52:02.911158+0000","last_undegraded":"2024-09-16T20:52:02.911158+0000","last_fullsized":"2024-09-16T20:52:02.911158+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:35:20.550930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062077499999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"216'498","reported_seq":18808,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911630+0000","last_change":"2024-09-16T20:51:45.643842+0000","last_active":"2024-09-16T20:52:02.911630+0000","last_peered":"2024-09-16T20:52:02.911630+0000","last_clean":"2024-09-16T20:52:02.911630+0000","last_became_active":"2024-09-16T20:51:26.626185+0000","last_became_peered":"2024-09-16T20:51:26.626185+0000","last_unstale":"2024-09-16T20:52:02.911630+0000","last_undegraded":"2024-09-16T20:52:02.911630+0000","last_fullsized":"2024-09-16T20:52:02.911630+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":498,"log_dups_size":0,"ondisk_log_size":498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:57:30.480666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023862639999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":19429,"num_read_kb":77391,"num_write":450,"num_write_kb":15765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":3862081,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":109,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911799+0000","last_change":"2024-09-16T20:51:12.615208+0000","last_active":"2024-09-16T20:52:02.911799+0000","last_peered":"2024-09-16T20:52:02.911799+0000","last_clean":"2024-09-16T20:52:02.911799+0000","last_became_active":"2024-09-16T20:51:12.614812+0000","last_became_peered":"2024-09-16T20:51:12.614812+0000","last_unstale":"2024-09-16T20:52:02.911799+0000","last_undegraded":"2024-09-16T20:52:02.911799+0000","last_fullsized":"2024-09-16T20:52:02.911799+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:37:11.741478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"215'312","reported_seq":5919,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.912408+0000","last_change":"2024-09-16T20:51:45.584820+0000","last_active":"2024-09-16T20:52:02.912408+0000","last_peered":"2024-09-16T20:52:02.912408+0000","last_clean":"2024-09-16T20:52:02.912408+0000","last_became_active":"2024-09-16T20:51:26.624469+0000","last_became_peered":"2024-09-16T20:51:26.624469+0000","last_unstale":"2024-09-16T20:52:02.912408+0000","last_undegraded":"2024-09-16T20:52:02.912408+0000","last_fullsized":"2024-09-16T20:52:02.912408+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:52:43.183901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025837220000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4323,"num_read_kb":33093,"num_write":273,"num_write_kb":12351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":324,"num_bytes_recovered":16931328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":89,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.741294+0000","last_change":"2024-09-16T20:51:23.608479+0000","last_active":"2024-09-16T20:52:02.741294+0000","last_peered":"2024-09-16T20:52:02.741294+0000","last_clean":"2024-09-16T20:52:02.741294+0000","last_became_active":"2024-09-16T20:51:23.608159+0000","last_became_peered":"2024-09-16T20:51:23.608159+0000","last_unstale":"2024-09-16T20:52:02.741294+0000","last_undegraded":"2024-09-16T20:52:02.741294+0000","last_fullsized":"2024-09-16T20:52:02.741294+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:06:21.085498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037636000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"217'609","reported_seq":7615,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911699+0000","last_change":"2024-09-16T20:51:45.582309+0000","last_active":"2024-09-16T20:52:02.911699+0000","last_peered":"2024-09-16T20:52:02.911699+0000","last_clean":"2024-09-16T20:52:02.911699+0000","last_became_active":"2024-09-16T20:51:26.622629+0000","last_became_peered":"2024-09-16T20:51:26.622629+0000","last_unstale":"2024-09-16T20:52:02.911699+0000","last_undegraded":"2024-09-16T20:52:02.911699+0000","last_fullsized":"2024-09-16T20:52:02.911699+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:40:49.603343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029005419999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":9084,"num_read_kb":37091,"num_write":448,"num_write_kb":13187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":848,"num_bytes_recovered":24298320,"num_keys_recovered":219,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":469,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.742037+0000","last_change":"2024-09-16T20:51:23.608488+0000","last_active":"2024-09-16T20:52:02.742037+0000","last_peered":"2024-09-16T20:52:02.742037+0000","last_clean":"2024-09-16T20:52:02.742037+0000","last_became_active":"2024-09-16T20:51:23.605321+0000","last_became_peered":"2024-09-16T20:51:23.605321+0000","last_unstale":"2024-09-16T20:52:02.742037+0000","last_undegraded":"2024-09-16T20:52:02.742037+0000","last_fullsized":"2024-09-16T20:52:02.742037+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:50:46.322344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095771799999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":564,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.741984+0000","last_change":"2024-09-16T20:49:09.565188+0000","last_active":"2024-09-16T20:52:02.741984+0000","last_peered":"2024-09-16T20:52:02.741984+0000","last_clean":"2024-09-16T20:52:02.741984+0000","last_became_active":"2024-09-16T20:49:09.559571+0000","last_became_peered":"2024-09-16T20:49:09.559571+0000","last_unstale":"2024-09-16T20:52:02.741984+0000","last_undegraded":"2024-09-16T20:52:02.741984+0000","last_fullsized":"2024-09-16T20:52:02.741984+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:52.241408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:52.241408+0000","last_clean_scrub_stamp":"2024-09-16T20:47:52.241408+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:01:53.651981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"214'641","reported_seq":7064,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.912231+0000","last_change":"2024-09-16T20:51:45.583936+0000","last_active":"2024-09-16T20:52:02.912231+0000","last_peered":"2024-09-16T20:52:02.912231+0000","last_clean":"2024-09-16T20:52:02.912231+0000","last_became_active":"2024-09-16T20:51:27.624220+0000","last_became_peered":"2024-09-16T20:51:27.624220+0000","last_unstale":"2024-09-16T20:52:02.912231+0000","last_undegraded":"2024-09-16T20:52:02.912231+0000","last_fullsized":"2024-09-16T20:52:02.912231+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":641,"log_dups_size":0,"ondisk_log_size":641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:13:47.571246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022745209999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6921,"num_read_kb":32984,"num_write":827,"num_write_kb":7103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":4839295,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"217'639","reported_seq":16470,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911021+0000","last_change":"2024-09-16T20:51:45.578750+0000","last_active":"2024-09-16T20:52:02.911021+0000","last_peered":"2024-09-16T20:52:02.911021+0000","last_clean":"2024-09-16T20:52:02.911021+0000","last_became_active":"2024-09-16T20:51:17.575136+0000","last_became_peered":"2024-09-16T20:51:17.575136+0000","last_unstale":"2024-09-16T20:52:02.911021+0000","last_undegraded":"2024-09-16T20:52:02.911021+0000","last_fullsized":"2024-09-16T20:52:02.911021+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":639,"log_dups_size":0,"ondisk_log_size":639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:32:32.550808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022726539999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":15159,"num_read_kb":101301,"num_write":561,"num_write_kb":29560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":790,"num_bytes_recovered":48497898,"num_keys_recovered":60,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"217'568","reported_seq":9847,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911042+0000","last_change":"2024-09-16T20:51:45.578068+0000","last_active":"2024-09-16T20:52:02.911042+0000","last_peered":"2024-09-16T20:52:02.911042+0000","last_clean":"2024-09-16T20:52:02.911042+0000","last_became_active":"2024-09-16T20:51:03.590152+0000","last_became_peered":"2024-09-16T20:51:03.590152+0000","last_unstale":"2024-09-16T20:52:02.911042+0000","last_undegraded":"2024-09-16T20:52:02.911042+0000","last_fullsized":"2024-09-16T20:52:02.911042+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":568,"log_dups_size":0,"ondisk_log_size":568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:21:38.569104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017317439999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":9605,"num_read_kb":51536,"num_write":618,"num_write_kb":12681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":14274951,"num_keys_recovered":91,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"218'468","reported_seq":8097,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.912037+0000","last_change":"2024-09-16T20:51:45.582215+0000","last_active":"2024-09-16T20:52:02.912037+0000","last_peered":"2024-09-16T20:52:02.912037+0000","last_clean":"2024-09-16T20:52:02.912037+0000","last_became_active":"2024-09-16T20:51:16.575698+0000","last_became_peered":"2024-09-16T20:51:16.575698+0000","last_unstale":"2024-09-16T20:52:02.912037+0000","last_undegraded":"2024-09-16T20:52:02.912037+0000","last_fullsized":"2024-09-16T20:52:02.912037+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:15:28.632535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020731130000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":7052,"num_read_kb":42310,"num_write":428,"num_write_kb":14897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":445,"num_bytes_recovered":8390493,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"216'726","reported_seq":22773,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911294+0000","last_change":"2024-09-16T20:51:45.579437+0000","last_active":"2024-09-16T20:52:02.911294+0000","last_peered":"2024-09-16T20:52:02.911294+0000","last_clean":"2024-09-16T20:52:02.911294+0000","last_became_active":"2024-09-16T20:51:17.575222+0000","last_became_peered":"2024-09-16T20:51:17.575222+0000","last_unstale":"2024-09-16T20:52:02.911294+0000","last_undegraded":"2024-09-16T20:52:02.911294+0000","last_fullsized":"2024-09-16T20:52:02.911294+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":726,"log_dups_size":0,"ondisk_log_size":726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:44:38.139123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018824849999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":22418,"num_read_kb":91945,"num_write":909,"num_write_kb":19380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":627,"num_bytes_recovered":71963834,"num_keys_recovered":119,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"218'1128","reported_seq":18644,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.742261+0000","last_change":"2024-09-16T20:51:45.585311+0000","last_active":"2024-09-16T20:52:02.742261+0000","last_peered":"2024-09-16T20:52:02.742261+0000","last_clean":"2024-09-16T20:52:02.742261+0000","last_became_active":"2024-09-16T20:51:05.581172+0000","last_became_peered":"2024-09-16T20:51:05.581172+0000","last_unstale":"2024-09-16T20:52:02.742261+0000","last_undegraded":"2024-09-16T20:52:02.742261+0000","last_fullsized":"2024-09-16T20:52:02.742261+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:27:15.820301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002500038,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":19435,"num_read_kb":70139,"num_write":1420,"num_write_kb":21576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":16987444,"num_keys_recovered":40,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"217'581","reported_seq":9275,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.741548+0000","last_change":"2024-09-16T20:51:45.581620+0000","last_active":"2024-09-16T20:52:02.741548+0000","last_peered":"2024-09-16T20:52:02.741548+0000","last_clean":"2024-09-16T20:52:02.741548+0000","last_became_active":"2024-09-16T20:51:27.627520+0000","last_became_peered":"2024-09-16T20:51:27.627520+0000","last_unstale":"2024-09-16T20:52:02.741548+0000","last_undegraded":"2024-09-16T20:52:02.741548+0000","last_fullsized":"2024-09-16T20:52:02.741548+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":581,"log_dups_size":0,"ondisk_log_size":581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:30:31.740673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012113930000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":8329,"num_read_kb":54907,"num_write":641,"num_write_kb":11423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":479,"num_bytes_recovered":14706083,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"217'534","reported_seq":13756,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911354+0000","last_change":"2024-09-16T20:51:45.580500+0000","last_active":"2024-09-16T20:52:02.911354+0000","last_peered":"2024-09-16T20:52:02.911354+0000","last_clean":"2024-09-16T20:52:02.911354+0000","last_became_active":"2024-09-16T20:51:05.581058+0000","last_became_peered":"2024-09-16T20:51:05.581058+0000","last_unstale":"2024-09-16T20:52:02.911354+0000","last_undegraded":"2024-09-16T20:52:02.911354+0000","last_fullsized":"2024-09-16T20:52:02.911354+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:55:20.810743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026424640000000002,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":13544,"num_read_kb":49574,"num_write":426,"num_write_kb":14576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":284,"num_bytes_recovered":9641443,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"218'691","reported_seq":13753,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.741972+0000","last_change":"2024-09-16T20:51:45.584709+0000","last_active":"2024-09-16T20:52:02.741972+0000","last_peered":"2024-09-16T20:52:02.741972+0000","last_clean":"2024-09-16T20:52:02.741972+0000","last_became_active":"2024-09-16T20:49:12.801013+0000","last_became_peered":"2024-09-16T20:49:12.801013+0000","last_unstale":"2024-09-16T20:52:02.741972+0000","last_undegraded":"2024-09-16T20:52:02.741972+0000","last_fullsized":"2024-09-16T20:52:02.741972+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":691,"log_dups_size":0,"ondisk_log_size":691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:13:59.069699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001757113,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":15597,"num_read_kb":58377,"num_write":523,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":99,"num_bytes_recovered":5210742,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"218'888","reported_seq":14337,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.741292+0000","last_change":"2024-09-16T20:51:45.579465+0000","last_active":"2024-09-16T20:52:02.741292+0000","last_peered":"2024-09-16T20:52:02.741292+0000","last_clean":"2024-09-16T20:52:02.741292+0000","last_became_active":"2024-09-16T20:51:26.624407+0000","last_became_peered":"2024-09-16T20:51:26.624407+0000","last_unstale":"2024-09-16T20:52:02.741292+0000","last_undegraded":"2024-09-16T20:52:02.741292+0000","last_fullsized":"2024-09-16T20:52:02.741292+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:59:50.323236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029313379999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":16446,"num_read_kb":75991,"num_write":874,"num_write_kb":24549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":351,"num_bytes_recovered":38747648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"211'608","reported_seq":20163,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911705+0000","last_change":"2024-09-16T20:51:45.643842+0000","last_active":"2024-09-16T20:52:02.911705+0000","last_peered":"2024-09-16T20:52:02.911705+0000","last_clean":"2024-09-16T20:52:02.911705+0000","last_became_active":"2024-09-16T20:51:03.589742+0000","last_became_peered":"2024-09-16T20:51:03.589742+0000","last_unstale":"2024-09-16T20:52:02.911705+0000","last_undegraded":"2024-09-16T20:52:02.911705+0000","last_fullsized":"2024-09-16T20:52:02.911705+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":608,"log_dups_size":0,"ondisk_log_size":608,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:49:22.579735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024335089999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":16969,"num_read_kb":76220,"num_write":613,"num_write_kb":13841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":486,"num_bytes_recovered":15329756,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"216'461","reported_seq":12004,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.741986+0000","last_change":"2024-09-16T20:51:45.586711+0000","last_active":"2024-09-16T20:52:02.741986+0000","last_peered":"2024-09-16T20:52:02.741986+0000","last_clean":"2024-09-16T20:52:02.741986+0000","last_became_active":"2024-09-16T20:51:03.595360+0000","last_became_peered":"2024-09-16T20:51:03.595360+0000","last_unstale":"2024-09-16T20:52:02.741986+0000","last_undegraded":"2024-09-16T20:52:02.741986+0000","last_fullsized":"2024-09-16T20:52:02.741986+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":461,"log_dups_size":0,"ondisk_log_size":461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:47:40.600095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035295309999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":11262,"num_read_kb":35639,"num_write":562,"num_write_kb":13645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":198,"num_bytes_recovered":14217900,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":247,"num_object_clones":0,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":243,"num_read":448597,"num_read_kb":1798155,"num_write":23462,"num_write_kb":469671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12645,"num_bytes_recovered":606441956,"num_keys_recovered":1497,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22149,"ondisk_log_size":22149,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574904,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52588,"kb_used_data":3344,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319252,"statfs":{"total":96636764160,"available":96582914048,"internally_reserved":0,"allocated":3424256,"data_stored":1774023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574904,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74140,"kb_used_data":2760,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94297700,"statfs":{"total":96636764160,"available":96560844800,"internally_reserved":0,"allocated":2826240,"data_stored":1183636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73071047},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574904,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69980,"kb_used_data":2760,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94301860,"statfs":{"total":96636764160,"available":96565104640,"internally_reserved":0,"allocated":2826240,"data_stored":1183636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574903,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70568,"kb_used_data":3344,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94301272,"statfs":{"total":96636764160,"available":96564502528,"internally_reserved":0,"allocated":3424256,"data_stored":1774023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574904,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":96420,"kb_used_data":2764,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":94275420,"statfs":{"total":96636764160,"available":96538030080,"internally_reserved":0,"allocated":2830336,"data_stored":1183655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":95877574},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640311,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60320,"kb_used_data":2764,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311520,"statfs":{"total":96636764160,"available":96574996480,"internally_reserved":0,"allocated":2830336,"data_stored":1183655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T20:52:06.937 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T20:52:07.232 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:52:07.232 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-09-16T20:52:07.244 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":359,"stamp":"2024-09-16T20:52:06.249925+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":251,"num_object_clones":0,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":243,"num_read":448663,"num_read_kb":1798210,"num_write":23565,"num_write_kb":471301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12652,"num_bytes_recovered":607622711,"num_keys_recovered":1497,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22224,"ondisk_log_size":22224,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":424016,"kb_used_data":17736,"kb_used_omap":9,"kb_used_meta":406134,"kb_avail":565807024,"statfs":{"total":579820584960,"available":579386392576,"internally_reserved":0,"allocated":18161664,"data_stored":8282628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9567,"internal_metadata":415881889},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001317"},"pg_stats":[{"pgid":"3.1f","version":"218'359","reported_seq":10822,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911515+0000","last_change":"2024-09-16T20:51:45.642922+0000","last_active":"2024-09-16T20:52:02.911515+0000","last_peered":"2024-09-16T20:52:02.911515+0000","last_clean":"2024-09-16T20:52:02.911515+0000","last_became_active":"2024-09-16T20:49:12.801566+0000","last_became_peered":"2024-09-16T20:49:12.801566+0000","last_unstale":"2024-09-16T20:52:02.911515+0000","last_undegraded":"2024-09-16T20:52:02.911515+0000","last_fullsized":"2024-09-16T20:52:02.911515+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:20:11.666141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016075530000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10091,"num_read_kb":31685,"num_write":315,"num_write_kb":5043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":2125998,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"218'905","reported_seq":28772,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.912100+0000","last_change":"2024-09-16T20:51:45.580852+0000","last_active":"2024-09-16T20:52:02.912100+0000","last_peered":"2024-09-16T20:52:02.912100+0000","last_clean":"2024-09-16T20:52:02.912100+0000","last_became_active":"2024-09-16T20:51:27.628085+0000","last_became_peered":"2024-09-16T20:51:27.628085+0000","last_unstale":"2024-09-16T20:52:02.912100+0000","last_undegraded":"2024-09-16T20:52:02.912100+0000","last_fullsized":"2024-09-16T20:52:02.912100+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":905,"log_dups_size":0,"ondisk_log_size":905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:59:41.269032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001444383,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":29337,"num_read_kb":83641,"num_write":1040,"num_write_kb":10328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":562,"num_bytes_recovered":11317531,"num_keys_recovered":154,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"214'707","reported_seq":12942,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.741297+0000","last_change":"2024-09-16T20:51:45.580224+0000","last_active":"2024-09-16T20:52:02.741297+0000","last_peered":"2024-09-16T20:52:02.741297+0000","last_clean":"2024-09-16T20:52:02.741297+0000","last_became_active":"2024-09-16T20:51:26.624314+0000","last_became_peered":"2024-09-16T20:51:26.624314+0000","last_unstale":"2024-09-16T20:52:02.741297+0000","last_undegraded":"2024-09-16T20:52:02.741297+0000","last_fullsized":"2024-09-16T20:52:02.741297+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":707,"log_dups_size":0,"ondisk_log_size":707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:47:19.682479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014726419999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":13066,"num_read_kb":103238,"num_write":750,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":387,"num_bytes_recovered":35619030,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"218'714","reported_seq":4372,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.742231+0000","last_change":"2024-09-16T20:51:45.586449+0000","last_active":"2024-09-16T20:52:02.742231+0000","last_peered":"2024-09-16T20:52:02.742231+0000","last_clean":"2024-09-16T20:52:02.742231+0000","last_became_active":"2024-09-16T20:51:04.577608+0000","last_became_peered":"2024-09-16T20:51:04.577608+0000","last_unstale":"2024-09-16T20:52:02.742231+0000","last_undegraded":"2024-09-16T20:52:02.742231+0000","last_fullsized":"2024-09-16T20:52:02.742231+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:15:00.043848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034574200000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":4615,"num_read_kb":24234,"num_write":996,"num_write_kb":15567,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":25643873,"num_keys_recovered":142,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"217'872","reported_seq":11160,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911930+0000","last_change":"2024-09-16T20:51:49.582917+0000","last_active":"2024-09-16T20:52:02.911930+0000","last_peered":"2024-09-16T20:52:02.911930+0000","last_clean":"2024-09-16T20:52:02.911930+0000","last_became_active":"2024-09-16T20:51:49.579980+0000","last_became_peered":"2024-09-16T20:51:49.579980+0000","last_unstale":"2024-09-16T20:52:02.911930+0000","last_undegraded":"2024-09-16T20:52:02.911930+0000","last_fullsized":"2024-09-16T20:52:02.911930+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":872,"log_dups_size":0,"ondisk_log_size":872,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:13:38.941261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001184378,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":10970,"num_read_kb":44437,"num_write":1151,"num_write_kb":16781,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":956,"num_bytes_recovered":37659232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"217'1221","reported_seq":19862,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.742042+0000","last_change":"2024-09-16T20:51:45.585643+0000","last_active":"2024-09-16T20:52:02.742042+0000","last_peered":"2024-09-16T20:52:02.742042+0000","last_clean":"2024-09-16T20:52:02.742042+0000","last_became_active":"2024-09-16T20:49:12.802869+0000","last_became_peered":"2024-09-16T20:49:12.802869+0000","last_unstale":"2024-09-16T20:52:02.742042+0000","last_undegraded":"2024-09-16T20:52:02.742042+0000","last_fullsized":"2024-09-16T20:52:02.742042+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":1221,"log_dups_size":0,"ondisk_log_size":1221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:17:13.191250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026611790000000001,"stat_sum":{"num_bytes":19,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":25740,"num_read_kb":67599,"num_write":1269,"num_write_kb":16392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":1051004,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"218'770","reported_seq":12989,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.912288+0000","last_change":"2024-09-16T20:51:45.584848+0000","last_active":"2024-09-16T20:52:02.912288+0000","last_peered":"2024-09-16T20:52:02.912288+0000","last_clean":"2024-09-16T20:52:02.912288+0000","last_became_active":"2024-09-16T20:51:16.577039+0000","last_became_peered":"2024-09-16T20:51:16.577039+0000","last_unstale":"2024-09-16T20:52:02.912288+0000","last_undegraded":"2024-09-16T20:52:02.912288+0000","last_fullsized":"2024-09-16T20:52:02.912288+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":770,"log_dups_size":0,"ondisk_log_size":770,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:26:39.219016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023604640000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":10011,"num_read_kb":47438,"num_write":753,"num_write_kb":12180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":12922788,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"216'458","reported_seq":13762,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911501+0000","last_change":"2024-09-16T20:51:45.581009+0000","last_active":"2024-09-16T20:52:02.911501+0000","last_peered":"2024-09-16T20:52:02.911501+0000","last_clean":"2024-09-16T20:52:02.911501+0000","last_became_active":"2024-09-16T20:51:15.575429+0000","last_became_peered":"2024-09-16T20:51:15.575429+0000","last_unstale":"2024-09-16T20:52:02.911501+0000","last_undegraded":"2024-09-16T20:52:02.911501+0000","last_fullsized":"2024-09-16T20:52:02.911501+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":458,"log_dups_size":0,"ondisk_log_size":458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:42:42.274481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001906629,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":14274,"num_read_kb":52840,"num_write":349,"num_write_kb":9355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":231,"num_bytes_recovered":11122736,"num_keys_recovered":66,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"217'981","reported_seq":8058,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911740+0000","last_change":"2024-09-16T20:51:45.642868+0000","last_active":"2024-09-16T20:52:02.911740+0000","last_peered":"2024-09-16T20:52:02.911740+0000","last_clean":"2024-09-16T20:52:02.911740+0000","last_became_active":"2024-09-16T20:51:03.592284+0000","last_became_peered":"2024-09-16T20:51:03.592284+0000","last_unstale":"2024-09-16T20:52:02.911740+0000","last_undegraded":"2024-09-16T20:52:02.911740+0000","last_fullsized":"2024-09-16T20:52:02.911740+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":981,"log_dups_size":0,"ondisk_log_size":981,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:35:23.178350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015146339999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":11609,"num_read_kb":15811,"num_write":975,"num_write_kb":5184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":280,"num_bytes_recovered":1580920,"num_keys_recovered":144,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"217'1277","reported_seq":11828,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.741549+0000","last_change":"2024-09-16T20:51:45.581650+0000","last_active":"2024-09-16T20:52:02.741549+0000","last_peered":"2024-09-16T20:52:02.741549+0000","last_clean":"2024-09-16T20:52:02.741549+0000","last_became_active":"2024-09-16T20:51:27.624341+0000","last_became_peered":"2024-09-16T20:51:27.624341+0000","last_unstale":"2024-09-16T20:52:02.741549+0000","last_undegraded":"2024-09-16T20:52:02.741549+0000","last_fullsized":"2024-09-16T20:52:02.741549+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":1277,"log_dups_size":0,"ondisk_log_size":1277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:01:31.923277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011831579999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":17170,"num_read_kb":43312,"num_write":1584,"num_write_kb":10211,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":542,"num_bytes_recovered":21909352,"num_keys_recovered":70,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"218'595","reported_seq":14001,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911281+0000","last_change":"2024-09-16T20:51:45.579668+0000","last_active":"2024-09-16T20:52:02.911281+0000","last_peered":"2024-09-16T20:52:02.911281+0000","last_clean":"2024-09-16T20:52:02.911281+0000","last_became_active":"2024-09-16T20:51:16.577620+0000","last_became_peered":"2024-09-16T20:51:16.577620+0000","last_unstale":"2024-09-16T20:52:02.911281+0000","last_undegraded":"2024-09-16T20:52:02.911281+0000","last_fullsized":"2024-09-16T20:52:02.911281+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":595,"log_dups_size":0,"ondisk_log_size":595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:23:49.832618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022662979999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":12843,"num_read_kb":77657,"num_write":690,"num_write_kb":15118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":562,"num_bytes_recovered":17171700,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"218'370","reported_seq":19983,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.742221+0000","last_change":"2024-09-16T20:51:57.720592+0000","last_active":"2024-09-16T20:52:02.742221+0000","last_peered":"2024-09-16T20:52:02.742221+0000","last_clean":"2024-09-16T20:52:02.742221+0000","last_became_active":"2024-09-16T20:51:57.718930+0000","last_became_peered":"2024-09-16T20:51:57.718930+0000","last_unstale":"2024-09-16T20:52:02.742221+0000","last_undegraded":"2024-09-16T20:52:02.742221+0000","last_fullsized":"2024-09-16T20:52:02.742221+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:08:04.217338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014936070000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":18755,"num_read_kb":45565,"num_write":465,"num_write_kb":13208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":540,"num_bytes_recovered":30162312,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":460,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.742331+0000","last_change":"2024-09-16T20:51:00.578451+0000","last_active":"2024-09-16T20:52:02.742331+0000","last_peered":"2024-09-16T20:52:02.742331+0000","last_clean":"2024-09-16T20:52:02.742331+0000","last_became_active":"2024-09-16T20:51:00.578081+0000","last_became_peered":"2024-09-16T20:51:00.578081+0000","last_unstale":"2024-09-16T20:52:02.742331+0000","last_undegraded":"2024-09-16T20:52:02.742331+0000","last_fullsized":"2024-09-16T20:52:02.742331+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:35:02.412998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091339500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"218'1028","reported_seq":19427,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.912471+0000","last_change":"2024-09-16T20:51:45.582985+0000","last_active":"2024-09-16T20:52:02.912471+0000","last_peered":"2024-09-16T20:52:02.912471+0000","last_clean":"2024-09-16T20:52:02.912471+0000","last_became_active":"2024-09-16T20:51:16.577104+0000","last_became_peered":"2024-09-16T20:51:16.577104+0000","last_unstale":"2024-09-16T20:52:02.912471+0000","last_undegraded":"2024-09-16T20:52:02.912471+0000","last_fullsized":"2024-09-16T20:52:02.912471+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":1028,"log_dups_size":0,"ondisk_log_size":1028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:56:34.623379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013907780000000001,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":20870,"num_read_kb":85867,"num_write":1218,"num_write_kb":25908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":29601482,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":463,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.912490+0000","last_change":"2024-09-16T20:51:00.577927+0000","last_active":"2024-09-16T20:52:02.912490+0000","last_peered":"2024-09-16T20:52:02.912490+0000","last_clean":"2024-09-16T20:52:02.912490+0000","last_became_active":"2024-09-16T20:51:00.577653+0000","last_became_peered":"2024-09-16T20:51:00.577653+0000","last_unstale":"2024-09-16T20:52:02.912490+0000","last_undegraded":"2024-09-16T20:52:02.912490+0000","last_fullsized":"2024-09-16T20:52:02.912490+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:12:13.831668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043659299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"214'653","reported_seq":25408,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.912290+0000","last_change":"2024-09-16T20:51:45.583929+0000","last_active":"2024-09-16T20:52:02.912290+0000","last_peered":"2024-09-16T20:52:02.912290+0000","last_clean":"2024-09-16T20:52:02.912290+0000","last_became_active":"2024-09-16T20:51:15.575607+0000","last_became_peered":"2024-09-16T20:51:15.575607+0000","last_unstale":"2024-09-16T20:52:02.912290+0000","last_undegraded":"2024-09-16T20:52:02.912290+0000","last_fullsized":"2024-09-16T20:52:02.912290+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":653,"log_dups_size":0,"ondisk_log_size":653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:42:18.817016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022924479999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":24178,"num_read_kb":81081,"num_write":502,"num_write_kb":17573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":20407955,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":448,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.742362+0000","last_change":"2024-09-16T20:48:00.278054+0000","last_active":"2024-09-16T20:52:02.742362+0000","last_peered":"2024-09-16T20:52:02.742362+0000","last_clean":"2024-09-16T20:52:02.742362+0000","last_became_active":"2024-09-16T20:47:58.272861+0000","last_became_peered":"2024-09-16T20:47:58.272861+0000","last_unstale":"2024-09-16T20:52:02.742362+0000","last_undegraded":"2024-09-16T20:52:02.742362+0000","last_fullsized":"2024-09-16T20:52:02.742362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:13:59.135547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012429489999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"213'491","reported_seq":3884,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.912098+0000","last_change":"2024-09-16T20:51:45.583107+0000","last_active":"2024-09-16T20:52:02.912098+0000","last_peered":"2024-09-16T20:52:02.912098+0000","last_clean":"2024-09-16T20:52:02.912098+0000","last_became_active":"2024-09-16T20:49:12.801810+0000","last_became_peered":"2024-09-16T20:49:12.801810+0000","last_unstale":"2024-09-16T20:52:02.912098+0000","last_undegraded":"2024-09-16T20:52:02.912098+0000","last_fullsized":"2024-09-16T20:52:02.912098+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":491,"log_dups_size":0,"ondisk_log_size":491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:48:19.573269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015370990000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":3741,"num_read_kb":11405,"num_write":512,"num_write_kb":6561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":2071114,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":340,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911584+0000","last_change":"2024-09-16T20:51:12.615838+0000","last_active":"2024-09-16T20:52:02.911584+0000","last_peered":"2024-09-16T20:52:02.911584+0000","last_clean":"2024-09-16T20:52:02.911584+0000","last_became_active":"2024-09-16T20:51:12.615412+0000","last_became_peered":"2024-09-16T20:51:12.615412+0000","last_unstale":"2024-09-16T20:52:02.911584+0000","last_undegraded":"2024-09-16T20:52:02.911584+0000","last_fullsized":"2024-09-16T20:52:02.911584+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:17:12.150193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"217'668","reported_seq":19715,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.741476+0000","last_change":"2024-09-16T20:51:45.580833+0000","last_active":"2024-09-16T20:52:02.741476+0000","last_peered":"2024-09-16T20:52:02.741476+0000","last_clean":"2024-09-16T20:52:02.741476+0000","last_became_active":"2024-09-16T20:51:27.624519+0000","last_became_peered":"2024-09-16T20:51:27.624519+0000","last_unstale":"2024-09-16T20:52:02.741476+0000","last_undegraded":"2024-09-16T20:52:02.741476+0000","last_fullsized":"2024-09-16T20:52:02.741476+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":668,"log_dups_size":0,"ondisk_log_size":668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:44:59.098443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001777441,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":18081,"num_read_kb":57429,"num_write":654,"num_write_kb":12584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":564,"num_bytes_recovered":27092048,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"218'728","reported_seq":8639,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911256+0000","last_change":"2024-09-16T20:51:45.580666+0000","last_active":"2024-09-16T20:52:02.911256+0000","last_peered":"2024-09-16T20:52:02.911256+0000","last_clean":"2024-09-16T20:52:02.911256+0000","last_became_active":"2024-09-16T20:51:27.626032+0000","last_became_peered":"2024-09-16T20:51:27.626032+0000","last_unstale":"2024-09-16T20:52:02.911256+0000","last_undegraded":"2024-09-16T20:52:02.911256+0000","last_fullsized":"2024-09-16T20:52:02.911256+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":728,"log_dups_size":0,"ondisk_log_size":728,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:28:16.242279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025014909999999998,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":7673,"num_read_kb":36418,"num_write":666,"num_write_kb":13145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":523,"num_bytes_recovered":11083665,"num_keys_recovered":14,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":137,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911158+0000","last_change":"2024-09-16T20:51:00.578241+0000","last_active":"2024-09-16T20:52:02.911158+0000","last_peered":"2024-09-16T20:52:02.911158+0000","last_clean":"2024-09-16T20:52:02.911158+0000","last_became_active":"2024-09-16T20:51:00.574693+0000","last_became_peered":"2024-09-16T20:51:00.574693+0000","last_unstale":"2024-09-16T20:52:02.911158+0000","last_undegraded":"2024-09-16T20:52:02.911158+0000","last_fullsized":"2024-09-16T20:52:02.911158+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:35:20.550930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062077499999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"216'498","reported_seq":18808,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911630+0000","last_change":"2024-09-16T20:51:45.643842+0000","last_active":"2024-09-16T20:52:02.911630+0000","last_peered":"2024-09-16T20:52:02.911630+0000","last_clean":"2024-09-16T20:52:02.911630+0000","last_became_active":"2024-09-16T20:51:26.626185+0000","last_became_peered":"2024-09-16T20:51:26.626185+0000","last_unstale":"2024-09-16T20:52:02.911630+0000","last_undegraded":"2024-09-16T20:52:02.911630+0000","last_fullsized":"2024-09-16T20:52:02.911630+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":498,"log_dups_size":0,"ondisk_log_size":498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:57:30.480666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023862639999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":19429,"num_read_kb":77391,"num_write":450,"num_write_kb":15765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":3862081,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":109,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911799+0000","last_change":"2024-09-16T20:51:12.615208+0000","last_active":"2024-09-16T20:52:02.911799+0000","last_peered":"2024-09-16T20:52:02.911799+0000","last_clean":"2024-09-16T20:52:02.911799+0000","last_became_active":"2024-09-16T20:51:12.614812+0000","last_became_peered":"2024-09-16T20:51:12.614812+0000","last_unstale":"2024-09-16T20:52:02.911799+0000","last_undegraded":"2024-09-16T20:52:02.911799+0000","last_fullsized":"2024-09-16T20:52:02.911799+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:37:11.741478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"215'312","reported_seq":5919,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.912408+0000","last_change":"2024-09-16T20:51:45.584820+0000","last_active":"2024-09-16T20:52:02.912408+0000","last_peered":"2024-09-16T20:52:02.912408+0000","last_clean":"2024-09-16T20:52:02.912408+0000","last_became_active":"2024-09-16T20:51:26.624469+0000","last_became_peered":"2024-09-16T20:51:26.624469+0000","last_unstale":"2024-09-16T20:52:02.912408+0000","last_undegraded":"2024-09-16T20:52:02.912408+0000","last_fullsized":"2024-09-16T20:52:02.912408+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:52:43.183901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025837220000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4323,"num_read_kb":33093,"num_write":273,"num_write_kb":12351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":324,"num_bytes_recovered":16931328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":89,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.741294+0000","last_change":"2024-09-16T20:51:23.608479+0000","last_active":"2024-09-16T20:52:02.741294+0000","last_peered":"2024-09-16T20:52:02.741294+0000","last_clean":"2024-09-16T20:52:02.741294+0000","last_became_active":"2024-09-16T20:51:23.608159+0000","last_became_peered":"2024-09-16T20:51:23.608159+0000","last_unstale":"2024-09-16T20:52:02.741294+0000","last_undegraded":"2024-09-16T20:52:02.741294+0000","last_fullsized":"2024-09-16T20:52:02.741294+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:06:21.085498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037636000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"217'609","reported_seq":7615,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911699+0000","last_change":"2024-09-16T20:51:45.582309+0000","last_active":"2024-09-16T20:52:02.911699+0000","last_peered":"2024-09-16T20:52:02.911699+0000","last_clean":"2024-09-16T20:52:02.911699+0000","last_became_active":"2024-09-16T20:51:26.622629+0000","last_became_peered":"2024-09-16T20:51:26.622629+0000","last_unstale":"2024-09-16T20:52:02.911699+0000","last_undegraded":"2024-09-16T20:52:02.911699+0000","last_fullsized":"2024-09-16T20:52:02.911699+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:40:49.603343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029005419999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":9084,"num_read_kb":37091,"num_write":448,"num_write_kb":13187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":848,"num_bytes_recovered":24298320,"num_keys_recovered":219,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":469,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.742037+0000","last_change":"2024-09-16T20:51:23.608488+0000","last_active":"2024-09-16T20:52:02.742037+0000","last_peered":"2024-09-16T20:52:02.742037+0000","last_clean":"2024-09-16T20:52:02.742037+0000","last_became_active":"2024-09-16T20:51:23.605321+0000","last_became_peered":"2024-09-16T20:51:23.605321+0000","last_unstale":"2024-09-16T20:52:02.742037+0000","last_undegraded":"2024-09-16T20:52:02.742037+0000","last_fullsized":"2024-09-16T20:52:02.742037+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:50:46.322344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095771799999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":564,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.741984+0000","last_change":"2024-09-16T20:49:09.565188+0000","last_active":"2024-09-16T20:52:02.741984+0000","last_peered":"2024-09-16T20:52:02.741984+0000","last_clean":"2024-09-16T20:52:02.741984+0000","last_became_active":"2024-09-16T20:49:09.559571+0000","last_became_peered":"2024-09-16T20:49:09.559571+0000","last_unstale":"2024-09-16T20:52:02.741984+0000","last_undegraded":"2024-09-16T20:52:02.741984+0000","last_fullsized":"2024-09-16T20:52:02.741984+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:52.241408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:52.241408+0000","last_clean_scrub_stamp":"2024-09-16T20:47:52.241408+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:01:53.651981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"214'641","reported_seq":7064,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.912231+0000","last_change":"2024-09-16T20:51:45.583936+0000","last_active":"2024-09-16T20:52:02.912231+0000","last_peered":"2024-09-16T20:52:02.912231+0000","last_clean":"2024-09-16T20:52:02.912231+0000","last_became_active":"2024-09-16T20:51:27.624220+0000","last_became_peered":"2024-09-16T20:51:27.624220+0000","last_unstale":"2024-09-16T20:52:02.912231+0000","last_undegraded":"2024-09-16T20:52:02.912231+0000","last_fullsized":"2024-09-16T20:52:02.912231+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":641,"log_dups_size":0,"ondisk_log_size":641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:13:47.571246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022745209999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6921,"num_read_kb":32984,"num_write":827,"num_write_kb":7103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":4839295,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"217'639","reported_seq":16470,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911021+0000","last_change":"2024-09-16T20:51:45.578750+0000","last_active":"2024-09-16T20:52:02.911021+0000","last_peered":"2024-09-16T20:52:02.911021+0000","last_clean":"2024-09-16T20:52:02.911021+0000","last_became_active":"2024-09-16T20:51:17.575136+0000","last_became_peered":"2024-09-16T20:51:17.575136+0000","last_unstale":"2024-09-16T20:52:02.911021+0000","last_undegraded":"2024-09-16T20:52:02.911021+0000","last_fullsized":"2024-09-16T20:52:02.911021+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":639,"log_dups_size":0,"ondisk_log_size":639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:32:32.550808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022726539999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":15159,"num_read_kb":101301,"num_write":561,"num_write_kb":29560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":790,"num_bytes_recovered":48497898,"num_keys_recovered":60,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"217'568","reported_seq":9847,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911042+0000","last_change":"2024-09-16T20:51:45.578068+0000","last_active":"2024-09-16T20:52:02.911042+0000","last_peered":"2024-09-16T20:52:02.911042+0000","last_clean":"2024-09-16T20:52:02.911042+0000","last_became_active":"2024-09-16T20:51:03.590152+0000","last_became_peered":"2024-09-16T20:51:03.590152+0000","last_unstale":"2024-09-16T20:52:02.911042+0000","last_undegraded":"2024-09-16T20:52:02.911042+0000","last_fullsized":"2024-09-16T20:52:02.911042+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":568,"log_dups_size":0,"ondisk_log_size":568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:21:38.569104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017317439999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":9605,"num_read_kb":51536,"num_write":618,"num_write_kb":12681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":14274951,"num_keys_recovered":91,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"218'468","reported_seq":8097,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.912037+0000","last_change":"2024-09-16T20:51:45.582215+0000","last_active":"2024-09-16T20:52:02.912037+0000","last_peered":"2024-09-16T20:52:02.912037+0000","last_clean":"2024-09-16T20:52:02.912037+0000","last_became_active":"2024-09-16T20:51:16.575698+0000","last_became_peered":"2024-09-16T20:51:16.575698+0000","last_unstale":"2024-09-16T20:52:02.912037+0000","last_undegraded":"2024-09-16T20:52:02.912037+0000","last_fullsized":"2024-09-16T20:52:02.912037+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:15:28.632535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020731130000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":7052,"num_read_kb":42310,"num_write":428,"num_write_kb":14897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":445,"num_bytes_recovered":8390493,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"216'726","reported_seq":22773,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911294+0000","last_change":"2024-09-16T20:51:45.579437+0000","last_active":"2024-09-16T20:52:02.911294+0000","last_peered":"2024-09-16T20:52:02.911294+0000","last_clean":"2024-09-16T20:52:02.911294+0000","last_became_active":"2024-09-16T20:51:17.575222+0000","last_became_peered":"2024-09-16T20:51:17.575222+0000","last_unstale":"2024-09-16T20:52:02.911294+0000","last_undegraded":"2024-09-16T20:52:02.911294+0000","last_fullsized":"2024-09-16T20:52:02.911294+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":726,"log_dups_size":0,"ondisk_log_size":726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:44:38.139123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018824849999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":22418,"num_read_kb":91945,"num_write":909,"num_write_kb":19380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":627,"num_bytes_recovered":71963834,"num_keys_recovered":119,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"218'1128","reported_seq":18644,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.742261+0000","last_change":"2024-09-16T20:51:45.585311+0000","last_active":"2024-09-16T20:52:02.742261+0000","last_peered":"2024-09-16T20:52:02.742261+0000","last_clean":"2024-09-16T20:52:02.742261+0000","last_became_active":"2024-09-16T20:51:05.581172+0000","last_became_peered":"2024-09-16T20:51:05.581172+0000","last_unstale":"2024-09-16T20:52:02.742261+0000","last_undegraded":"2024-09-16T20:52:02.742261+0000","last_fullsized":"2024-09-16T20:52:02.742261+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:27:15.820301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002500038,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":19435,"num_read_kb":70139,"num_write":1420,"num_write_kb":21576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":16987444,"num_keys_recovered":40,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"217'581","reported_seq":9275,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.741548+0000","last_change":"2024-09-16T20:51:45.581620+0000","last_active":"2024-09-16T20:52:02.741548+0000","last_peered":"2024-09-16T20:52:02.741548+0000","last_clean":"2024-09-16T20:52:02.741548+0000","last_became_active":"2024-09-16T20:51:27.627520+0000","last_became_peered":"2024-09-16T20:51:27.627520+0000","last_unstale":"2024-09-16T20:52:02.741548+0000","last_undegraded":"2024-09-16T20:52:02.741548+0000","last_fullsized":"2024-09-16T20:52:02.741548+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":581,"log_dups_size":0,"ondisk_log_size":581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:30:31.740673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012113930000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":8329,"num_read_kb":54907,"num_write":641,"num_write_kb":11423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":479,"num_bytes_recovered":14706083,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"217'534","reported_seq":13756,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911354+0000","last_change":"2024-09-16T20:51:45.580500+0000","last_active":"2024-09-16T20:52:02.911354+0000","last_peered":"2024-09-16T20:52:02.911354+0000","last_clean":"2024-09-16T20:52:02.911354+0000","last_became_active":"2024-09-16T20:51:05.581058+0000","last_became_peered":"2024-09-16T20:51:05.581058+0000","last_unstale":"2024-09-16T20:52:02.911354+0000","last_undegraded":"2024-09-16T20:52:02.911354+0000","last_fullsized":"2024-09-16T20:52:02.911354+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:55:20.810743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026424640000000002,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":13544,"num_read_kb":49574,"num_write":426,"num_write_kb":14576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":284,"num_bytes_recovered":9641443,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"218'691","reported_seq":13753,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.741972+0000","last_change":"2024-09-16T20:51:45.584709+0000","last_active":"2024-09-16T20:52:02.741972+0000","last_peered":"2024-09-16T20:52:02.741972+0000","last_clean":"2024-09-16T20:52:02.741972+0000","last_became_active":"2024-09-16T20:49:12.801013+0000","last_became_peered":"2024-09-16T20:49:12.801013+0000","last_unstale":"2024-09-16T20:52:02.741972+0000","last_undegraded":"2024-09-16T20:52:02.741972+0000","last_fullsized":"2024-09-16T20:52:02.741972+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":691,"log_dups_size":0,"ondisk_log_size":691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:13:59.069699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001757113,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":15597,"num_read_kb":58377,"num_write":523,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":99,"num_bytes_recovered":5210742,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"218'888","reported_seq":14337,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.741292+0000","last_change":"2024-09-16T20:51:45.579465+0000","last_active":"2024-09-16T20:52:02.741292+0000","last_peered":"2024-09-16T20:52:02.741292+0000","last_clean":"2024-09-16T20:52:02.741292+0000","last_became_active":"2024-09-16T20:51:26.624407+0000","last_became_peered":"2024-09-16T20:51:26.624407+0000","last_unstale":"2024-09-16T20:52:02.741292+0000","last_undegraded":"2024-09-16T20:52:02.741292+0000","last_fullsized":"2024-09-16T20:52:02.741292+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:59:50.323236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029313379999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":16446,"num_read_kb":75991,"num_write":874,"num_write_kb":24549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":351,"num_bytes_recovered":38747648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"211'608","reported_seq":20163,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911705+0000","last_change":"2024-09-16T20:51:45.643842+0000","last_active":"2024-09-16T20:52:02.911705+0000","last_peered":"2024-09-16T20:52:02.911705+0000","last_clean":"2024-09-16T20:52:02.911705+0000","last_became_active":"2024-09-16T20:51:03.589742+0000","last_became_peered":"2024-09-16T20:51:03.589742+0000","last_unstale":"2024-09-16T20:52:02.911705+0000","last_undegraded":"2024-09-16T20:52:02.911705+0000","last_fullsized":"2024-09-16T20:52:02.911705+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":608,"log_dups_size":0,"ondisk_log_size":608,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:49:22.579735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024335089999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":16969,"num_read_kb":76220,"num_write":613,"num_write_kb":13841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":486,"num_bytes_recovered":15329756,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"216'461","reported_seq":12004,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.741986+0000","last_change":"2024-09-16T20:51:45.586711+0000","last_active":"2024-09-16T20:52:02.741986+0000","last_peered":"2024-09-16T20:52:02.741986+0000","last_clean":"2024-09-16T20:52:02.741986+0000","last_became_active":"2024-09-16T20:51:03.595360+0000","last_became_peered":"2024-09-16T20:51:03.595360+0000","last_unstale":"2024-09-16T20:52:02.741986+0000","last_undegraded":"2024-09-16T20:52:02.741986+0000","last_fullsized":"2024-09-16T20:52:02.741986+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":461,"log_dups_size":0,"ondisk_log_size":461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:47:40.600095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035295309999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":11262,"num_read_kb":35639,"num_write":562,"num_write_kb":13645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":198,"num_bytes_recovered":14217900,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":247,"num_object_clones":0,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":243,"num_read":448597,"num_read_kb":1798155,"num_write":23462,"num_write_kb":469671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12645,"num_bytes_recovered":606441956,"num_keys_recovered":1497,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22149,"ondisk_log_size":22149,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574904,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52588,"kb_used_data":3344,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319252,"statfs":{"total":96636764160,"available":96582914048,"internally_reserved":0,"allocated":3424256,"data_stored":1774023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574904,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74140,"kb_used_data":2760,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94297700,"statfs":{"total":96636764160,"available":96560844800,"internally_reserved":0,"allocated":2826240,"data_stored":1183636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73071047},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574904,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69980,"kb_used_data":2760,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94301860,"statfs":{"total":96636764160,"available":96565104640,"internally_reserved":0,"allocated":2826240,"data_stored":1183636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574903,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70568,"kb_used_data":3344,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94301272,"statfs":{"total":96636764160,"available":96564502528,"internally_reserved":0,"allocated":3424256,"data_stored":1774023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574904,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":96420,"kb_used_data":2764,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":94275420,"statfs":{"total":96636764160,"available":96538030080,"internally_reserved":0,"allocated":2830336,"data_stored":1183655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":95877574},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640311,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60320,"kb_used_data":2764,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311520,"statfs":{"total":96636764160,"available":96574996480,"internally_reserved":0,"allocated":2830336,"data_stored":1183655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T20:52:07.246 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T20:52:07.246 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T20:52:07.545 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:52:07.546 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-09-16T20:52:07.557 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":359,"stamp":"2024-09-16T20:52:06.249925+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":251,"num_object_clones":0,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":243,"num_read":448663,"num_read_kb":1798210,"num_write":23565,"num_write_kb":471301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12652,"num_bytes_recovered":607622711,"num_keys_recovered":1497,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22224,"ondisk_log_size":22224,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":424016,"kb_used_data":17736,"kb_used_omap":9,"kb_used_meta":406134,"kb_avail":565807024,"statfs":{"total":579820584960,"available":579386392576,"internally_reserved":0,"allocated":18161664,"data_stored":8282628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9567,"internal_metadata":415881889},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001317"},"pg_stats":[{"pgid":"3.1f","version":"218'359","reported_seq":10822,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911515+0000","last_change":"2024-09-16T20:51:45.642922+0000","last_active":"2024-09-16T20:52:02.911515+0000","last_peered":"2024-09-16T20:52:02.911515+0000","last_clean":"2024-09-16T20:52:02.911515+0000","last_became_active":"2024-09-16T20:49:12.801566+0000","last_became_peered":"2024-09-16T20:49:12.801566+0000","last_unstale":"2024-09-16T20:52:02.911515+0000","last_undegraded":"2024-09-16T20:52:02.911515+0000","last_fullsized":"2024-09-16T20:52:02.911515+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:20:11.666141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016075530000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10091,"num_read_kb":31685,"num_write":315,"num_write_kb":5043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":2125998,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"218'905","reported_seq":28772,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.912100+0000","last_change":"2024-09-16T20:51:45.580852+0000","last_active":"2024-09-16T20:52:02.912100+0000","last_peered":"2024-09-16T20:52:02.912100+0000","last_clean":"2024-09-16T20:52:02.912100+0000","last_became_active":"2024-09-16T20:51:27.628085+0000","last_became_peered":"2024-09-16T20:51:27.628085+0000","last_unstale":"2024-09-16T20:52:02.912100+0000","last_undegraded":"2024-09-16T20:52:02.912100+0000","last_fullsized":"2024-09-16T20:52:02.912100+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":905,"log_dups_size":0,"ondisk_log_size":905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:59:41.269032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001444383,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":29337,"num_read_kb":83641,"num_write":1040,"num_write_kb":10328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":562,"num_bytes_recovered":11317531,"num_keys_recovered":154,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"214'707","reported_seq":12942,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.741297+0000","last_change":"2024-09-16T20:51:45.580224+0000","last_active":"2024-09-16T20:52:02.741297+0000","last_peered":"2024-09-16T20:52:02.741297+0000","last_clean":"2024-09-16T20:52:02.741297+0000","last_became_active":"2024-09-16T20:51:26.624314+0000","last_became_peered":"2024-09-16T20:51:26.624314+0000","last_unstale":"2024-09-16T20:52:02.741297+0000","last_undegraded":"2024-09-16T20:52:02.741297+0000","last_fullsized":"2024-09-16T20:52:02.741297+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":707,"log_dups_size":0,"ondisk_log_size":707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:47:19.682479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014726419999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":13066,"num_read_kb":103238,"num_write":750,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":387,"num_bytes_recovered":35619030,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"218'714","reported_seq":4372,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.742231+0000","last_change":"2024-09-16T20:51:45.586449+0000","last_active":"2024-09-16T20:52:02.742231+0000","last_peered":"2024-09-16T20:52:02.742231+0000","last_clean":"2024-09-16T20:52:02.742231+0000","last_became_active":"2024-09-16T20:51:04.577608+0000","last_became_peered":"2024-09-16T20:51:04.577608+0000","last_unstale":"2024-09-16T20:52:02.742231+0000","last_undegraded":"2024-09-16T20:52:02.742231+0000","last_fullsized":"2024-09-16T20:52:02.742231+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:15:00.043848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034574200000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":4615,"num_read_kb":24234,"num_write":996,"num_write_kb":15567,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":25643873,"num_keys_recovered":142,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"217'872","reported_seq":11160,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911930+0000","last_change":"2024-09-16T20:51:49.582917+0000","last_active":"2024-09-16T20:52:02.911930+0000","last_peered":"2024-09-16T20:52:02.911930+0000","last_clean":"2024-09-16T20:52:02.911930+0000","last_became_active":"2024-09-16T20:51:49.579980+0000","last_became_peered":"2024-09-16T20:51:49.579980+0000","last_unstale":"2024-09-16T20:52:02.911930+0000","last_undegraded":"2024-09-16T20:52:02.911930+0000","last_fullsized":"2024-09-16T20:52:02.911930+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":872,"log_dups_size":0,"ondisk_log_size":872,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:13:38.941261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001184378,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":10970,"num_read_kb":44437,"num_write":1151,"num_write_kb":16781,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":956,"num_bytes_recovered":37659232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"217'1221","reported_seq":19862,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.742042+0000","last_change":"2024-09-16T20:51:45.585643+0000","last_active":"2024-09-16T20:52:02.742042+0000","last_peered":"2024-09-16T20:52:02.742042+0000","last_clean":"2024-09-16T20:52:02.742042+0000","last_became_active":"2024-09-16T20:49:12.802869+0000","last_became_peered":"2024-09-16T20:49:12.802869+0000","last_unstale":"2024-09-16T20:52:02.742042+0000","last_undegraded":"2024-09-16T20:52:02.742042+0000","last_fullsized":"2024-09-16T20:52:02.742042+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":1221,"log_dups_size":0,"ondisk_log_size":1221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:17:13.191250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026611790000000001,"stat_sum":{"num_bytes":19,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":25740,"num_read_kb":67599,"num_write":1269,"num_write_kb":16392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":1051004,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"218'770","reported_seq":12989,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.912288+0000","last_change":"2024-09-16T20:51:45.584848+0000","last_active":"2024-09-16T20:52:02.912288+0000","last_peered":"2024-09-16T20:52:02.912288+0000","last_clean":"2024-09-16T20:52:02.912288+0000","last_became_active":"2024-09-16T20:51:16.577039+0000","last_became_peered":"2024-09-16T20:51:16.577039+0000","last_unstale":"2024-09-16T20:52:02.912288+0000","last_undegraded":"2024-09-16T20:52:02.912288+0000","last_fullsized":"2024-09-16T20:52:02.912288+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":770,"log_dups_size":0,"ondisk_log_size":770,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:26:39.219016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023604640000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":10011,"num_read_kb":47438,"num_write":753,"num_write_kb":12180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":12922788,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"216'458","reported_seq":13762,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911501+0000","last_change":"2024-09-16T20:51:45.581009+0000","last_active":"2024-09-16T20:52:02.911501+0000","last_peered":"2024-09-16T20:52:02.911501+0000","last_clean":"2024-09-16T20:52:02.911501+0000","last_became_active":"2024-09-16T20:51:15.575429+0000","last_became_peered":"2024-09-16T20:51:15.575429+0000","last_unstale":"2024-09-16T20:52:02.911501+0000","last_undegraded":"2024-09-16T20:52:02.911501+0000","last_fullsized":"2024-09-16T20:52:02.911501+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":458,"log_dups_size":0,"ondisk_log_size":458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:42:42.274481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001906629,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":14274,"num_read_kb":52840,"num_write":349,"num_write_kb":9355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":231,"num_bytes_recovered":11122736,"num_keys_recovered":66,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"217'981","reported_seq":8058,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911740+0000","last_change":"2024-09-16T20:51:45.642868+0000","last_active":"2024-09-16T20:52:02.911740+0000","last_peered":"2024-09-16T20:52:02.911740+0000","last_clean":"2024-09-16T20:52:02.911740+0000","last_became_active":"2024-09-16T20:51:03.592284+0000","last_became_peered":"2024-09-16T20:51:03.592284+0000","last_unstale":"2024-09-16T20:52:02.911740+0000","last_undegraded":"2024-09-16T20:52:02.911740+0000","last_fullsized":"2024-09-16T20:52:02.911740+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":981,"log_dups_size":0,"ondisk_log_size":981,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:35:23.178350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015146339999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":11609,"num_read_kb":15811,"num_write":975,"num_write_kb":5184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":280,"num_bytes_recovered":1580920,"num_keys_recovered":144,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"217'1277","reported_seq":11828,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.741549+0000","last_change":"2024-09-16T20:51:45.581650+0000","last_active":"2024-09-16T20:52:02.741549+0000","last_peered":"2024-09-16T20:52:02.741549+0000","last_clean":"2024-09-16T20:52:02.741549+0000","last_became_active":"2024-09-16T20:51:27.624341+0000","last_became_peered":"2024-09-16T20:51:27.624341+0000","last_unstale":"2024-09-16T20:52:02.741549+0000","last_undegraded":"2024-09-16T20:52:02.741549+0000","last_fullsized":"2024-09-16T20:52:02.741549+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":1277,"log_dups_size":0,"ondisk_log_size":1277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:01:31.923277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011831579999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":17170,"num_read_kb":43312,"num_write":1584,"num_write_kb":10211,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":542,"num_bytes_recovered":21909352,"num_keys_recovered":70,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"218'595","reported_seq":14001,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911281+0000","last_change":"2024-09-16T20:51:45.579668+0000","last_active":"2024-09-16T20:52:02.911281+0000","last_peered":"2024-09-16T20:52:02.911281+0000","last_clean":"2024-09-16T20:52:02.911281+0000","last_became_active":"2024-09-16T20:51:16.577620+0000","last_became_peered":"2024-09-16T20:51:16.577620+0000","last_unstale":"2024-09-16T20:52:02.911281+0000","last_undegraded":"2024-09-16T20:52:02.911281+0000","last_fullsized":"2024-09-16T20:52:02.911281+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":595,"log_dups_size":0,"ondisk_log_size":595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:23:49.832618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022662979999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":12843,"num_read_kb":77657,"num_write":690,"num_write_kb":15118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":562,"num_bytes_recovered":17171700,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"218'370","reported_seq":19983,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.742221+0000","last_change":"2024-09-16T20:51:57.720592+0000","last_active":"2024-09-16T20:52:02.742221+0000","last_peered":"2024-09-16T20:52:02.742221+0000","last_clean":"2024-09-16T20:52:02.742221+0000","last_became_active":"2024-09-16T20:51:57.718930+0000","last_became_peered":"2024-09-16T20:51:57.718930+0000","last_unstale":"2024-09-16T20:52:02.742221+0000","last_undegraded":"2024-09-16T20:52:02.742221+0000","last_fullsized":"2024-09-16T20:52:02.742221+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:08:04.217338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014936070000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":18755,"num_read_kb":45565,"num_write":465,"num_write_kb":13208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":540,"num_bytes_recovered":30162312,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":460,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.742331+0000","last_change":"2024-09-16T20:51:00.578451+0000","last_active":"2024-09-16T20:52:02.742331+0000","last_peered":"2024-09-16T20:52:02.742331+0000","last_clean":"2024-09-16T20:52:02.742331+0000","last_became_active":"2024-09-16T20:51:00.578081+0000","last_became_peered":"2024-09-16T20:51:00.578081+0000","last_unstale":"2024-09-16T20:52:02.742331+0000","last_undegraded":"2024-09-16T20:52:02.742331+0000","last_fullsized":"2024-09-16T20:52:02.742331+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:35:02.412998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091339500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"218'1028","reported_seq":19427,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.912471+0000","last_change":"2024-09-16T20:51:45.582985+0000","last_active":"2024-09-16T20:52:02.912471+0000","last_peered":"2024-09-16T20:52:02.912471+0000","last_clean":"2024-09-16T20:52:02.912471+0000","last_became_active":"2024-09-16T20:51:16.577104+0000","last_became_peered":"2024-09-16T20:51:16.577104+0000","last_unstale":"2024-09-16T20:52:02.912471+0000","last_undegraded":"2024-09-16T20:52:02.912471+0000","last_fullsized":"2024-09-16T20:52:02.912471+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":1028,"log_dups_size":0,"ondisk_log_size":1028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:56:34.623379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013907780000000001,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":20870,"num_read_kb":85867,"num_write":1218,"num_write_kb":25908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":29601482,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":463,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.912490+0000","last_change":"2024-09-16T20:51:00.577927+0000","last_active":"2024-09-16T20:52:02.912490+0000","last_peered":"2024-09-16T20:52:02.912490+0000","last_clean":"2024-09-16T20:52:02.912490+0000","last_became_active":"2024-09-16T20:51:00.577653+0000","last_became_peered":"2024-09-16T20:51:00.577653+0000","last_unstale":"2024-09-16T20:52:02.912490+0000","last_undegraded":"2024-09-16T20:52:02.912490+0000","last_fullsized":"2024-09-16T20:52:02.912490+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:12:13.831668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043659299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"214'653","reported_seq":25408,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.912290+0000","last_change":"2024-09-16T20:51:45.583929+0000","last_active":"2024-09-16T20:52:02.912290+0000","last_peered":"2024-09-16T20:52:02.912290+0000","last_clean":"2024-09-16T20:52:02.912290+0000","last_became_active":"2024-09-16T20:51:15.575607+0000","last_became_peered":"2024-09-16T20:51:15.575607+0000","last_unstale":"2024-09-16T20:52:02.912290+0000","last_undegraded":"2024-09-16T20:52:02.912290+0000","last_fullsized":"2024-09-16T20:52:02.912290+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":653,"log_dups_size":0,"ondisk_log_size":653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:42:18.817016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022924479999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":24178,"num_read_kb":81081,"num_write":502,"num_write_kb":17573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":20407955,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":448,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.742362+0000","last_change":"2024-09-16T20:48:00.278054+0000","last_active":"2024-09-16T20:52:02.742362+0000","last_peered":"2024-09-16T20:52:02.742362+0000","last_clean":"2024-09-16T20:52:02.742362+0000","last_became_active":"2024-09-16T20:47:58.272861+0000","last_became_peered":"2024-09-16T20:47:58.272861+0000","last_unstale":"2024-09-16T20:52:02.742362+0000","last_undegraded":"2024-09-16T20:52:02.742362+0000","last_fullsized":"2024-09-16T20:52:02.742362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:13:59.135547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012429489999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"213'491","reported_seq":3884,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.912098+0000","last_change":"2024-09-16T20:51:45.583107+0000","last_active":"2024-09-16T20:52:02.912098+0000","last_peered":"2024-09-16T20:52:02.912098+0000","last_clean":"2024-09-16T20:52:02.912098+0000","last_became_active":"2024-09-16T20:49:12.801810+0000","last_became_peered":"2024-09-16T20:49:12.801810+0000","last_unstale":"2024-09-16T20:52:02.912098+0000","last_undegraded":"2024-09-16T20:52:02.912098+0000","last_fullsized":"2024-09-16T20:52:02.912098+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":491,"log_dups_size":0,"ondisk_log_size":491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:48:19.573269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015370990000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":3741,"num_read_kb":11405,"num_write":512,"num_write_kb":6561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":2071114,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":340,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911584+0000","last_change":"2024-09-16T20:51:12.615838+0000","last_active":"2024-09-16T20:52:02.911584+0000","last_peered":"2024-09-16T20:52:02.911584+0000","last_clean":"2024-09-16T20:52:02.911584+0000","last_became_active":"2024-09-16T20:51:12.615412+0000","last_became_peered":"2024-09-16T20:51:12.615412+0000","last_unstale":"2024-09-16T20:52:02.911584+0000","last_undegraded":"2024-09-16T20:52:02.911584+0000","last_fullsized":"2024-09-16T20:52:02.911584+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:17:12.150193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"217'668","reported_seq":19715,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.741476+0000","last_change":"2024-09-16T20:51:45.580833+0000","last_active":"2024-09-16T20:52:02.741476+0000","last_peered":"2024-09-16T20:52:02.741476+0000","last_clean":"2024-09-16T20:52:02.741476+0000","last_became_active":"2024-09-16T20:51:27.624519+0000","last_became_peered":"2024-09-16T20:51:27.624519+0000","last_unstale":"2024-09-16T20:52:02.741476+0000","last_undegraded":"2024-09-16T20:52:02.741476+0000","last_fullsized":"2024-09-16T20:52:02.741476+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":668,"log_dups_size":0,"ondisk_log_size":668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:44:59.098443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001777441,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":18081,"num_read_kb":57429,"num_write":654,"num_write_kb":12584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":564,"num_bytes_recovered":27092048,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"218'728","reported_seq":8639,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911256+0000","last_change":"2024-09-16T20:51:45.580666+0000","last_active":"2024-09-16T20:52:02.911256+0000","last_peered":"2024-09-16T20:52:02.911256+0000","last_clean":"2024-09-16T20:52:02.911256+0000","last_became_active":"2024-09-16T20:51:27.626032+0000","last_became_peered":"2024-09-16T20:51:27.626032+0000","last_unstale":"2024-09-16T20:52:02.911256+0000","last_undegraded":"2024-09-16T20:52:02.911256+0000","last_fullsized":"2024-09-16T20:52:02.911256+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":728,"log_dups_size":0,"ondisk_log_size":728,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:28:16.242279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025014909999999998,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":7673,"num_read_kb":36418,"num_write":666,"num_write_kb":13145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":523,"num_bytes_recovered":11083665,"num_keys_recovered":14,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":137,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911158+0000","last_change":"2024-09-16T20:51:00.578241+0000","last_active":"2024-09-16T20:52:02.911158+0000","last_peered":"2024-09-16T20:52:02.911158+0000","last_clean":"2024-09-16T20:52:02.911158+0000","last_became_active":"2024-09-16T20:51:00.574693+0000","last_became_peered":"2024-09-16T20:51:00.574693+0000","last_unstale":"2024-09-16T20:52:02.911158+0000","last_undegraded":"2024-09-16T20:52:02.911158+0000","last_fullsized":"2024-09-16T20:52:02.911158+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:35:20.550930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062077499999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"216'498","reported_seq":18808,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911630+0000","last_change":"2024-09-16T20:51:45.643842+0000","last_active":"2024-09-16T20:52:02.911630+0000","last_peered":"2024-09-16T20:52:02.911630+0000","last_clean":"2024-09-16T20:52:02.911630+0000","last_became_active":"2024-09-16T20:51:26.626185+0000","last_became_peered":"2024-09-16T20:51:26.626185+0000","last_unstale":"2024-09-16T20:52:02.911630+0000","last_undegraded":"2024-09-16T20:52:02.911630+0000","last_fullsized":"2024-09-16T20:52:02.911630+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":498,"log_dups_size":0,"ondisk_log_size":498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:57:30.480666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023862639999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":19429,"num_read_kb":77391,"num_write":450,"num_write_kb":15765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":3862081,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":109,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911799+0000","last_change":"2024-09-16T20:51:12.615208+0000","last_active":"2024-09-16T20:52:02.911799+0000","last_peered":"2024-09-16T20:52:02.911799+0000","last_clean":"2024-09-16T20:52:02.911799+0000","last_became_active":"2024-09-16T20:51:12.614812+0000","last_became_peered":"2024-09-16T20:51:12.614812+0000","last_unstale":"2024-09-16T20:52:02.911799+0000","last_undegraded":"2024-09-16T20:52:02.911799+0000","last_fullsized":"2024-09-16T20:52:02.911799+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:37:11.741478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"215'312","reported_seq":5919,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.912408+0000","last_change":"2024-09-16T20:51:45.584820+0000","last_active":"2024-09-16T20:52:02.912408+0000","last_peered":"2024-09-16T20:52:02.912408+0000","last_clean":"2024-09-16T20:52:02.912408+0000","last_became_active":"2024-09-16T20:51:26.624469+0000","last_became_peered":"2024-09-16T20:51:26.624469+0000","last_unstale":"2024-09-16T20:52:02.912408+0000","last_undegraded":"2024-09-16T20:52:02.912408+0000","last_fullsized":"2024-09-16T20:52:02.912408+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:52:43.183901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025837220000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4323,"num_read_kb":33093,"num_write":273,"num_write_kb":12351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":324,"num_bytes_recovered":16931328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":89,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.741294+0000","last_change":"2024-09-16T20:51:23.608479+0000","last_active":"2024-09-16T20:52:02.741294+0000","last_peered":"2024-09-16T20:52:02.741294+0000","last_clean":"2024-09-16T20:52:02.741294+0000","last_became_active":"2024-09-16T20:51:23.608159+0000","last_became_peered":"2024-09-16T20:51:23.608159+0000","last_unstale":"2024-09-16T20:52:02.741294+0000","last_undegraded":"2024-09-16T20:52:02.741294+0000","last_fullsized":"2024-09-16T20:52:02.741294+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:06:21.085498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037636000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"217'609","reported_seq":7615,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911699+0000","last_change":"2024-09-16T20:51:45.582309+0000","last_active":"2024-09-16T20:52:02.911699+0000","last_peered":"2024-09-16T20:52:02.911699+0000","last_clean":"2024-09-16T20:52:02.911699+0000","last_became_active":"2024-09-16T20:51:26.622629+0000","last_became_peered":"2024-09-16T20:51:26.622629+0000","last_unstale":"2024-09-16T20:52:02.911699+0000","last_undegraded":"2024-09-16T20:52:02.911699+0000","last_fullsized":"2024-09-16T20:52:02.911699+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:40:49.603343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029005419999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":9084,"num_read_kb":37091,"num_write":448,"num_write_kb":13187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":848,"num_bytes_recovered":24298320,"num_keys_recovered":219,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":469,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.742037+0000","last_change":"2024-09-16T20:51:23.608488+0000","last_active":"2024-09-16T20:52:02.742037+0000","last_peered":"2024-09-16T20:52:02.742037+0000","last_clean":"2024-09-16T20:52:02.742037+0000","last_became_active":"2024-09-16T20:51:23.605321+0000","last_became_peered":"2024-09-16T20:51:23.605321+0000","last_unstale":"2024-09-16T20:52:02.742037+0000","last_undegraded":"2024-09-16T20:52:02.742037+0000","last_fullsized":"2024-09-16T20:52:02.742037+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:50:46.322344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095771799999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":564,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.741984+0000","last_change":"2024-09-16T20:49:09.565188+0000","last_active":"2024-09-16T20:52:02.741984+0000","last_peered":"2024-09-16T20:52:02.741984+0000","last_clean":"2024-09-16T20:52:02.741984+0000","last_became_active":"2024-09-16T20:49:09.559571+0000","last_became_peered":"2024-09-16T20:49:09.559571+0000","last_unstale":"2024-09-16T20:52:02.741984+0000","last_undegraded":"2024-09-16T20:52:02.741984+0000","last_fullsized":"2024-09-16T20:52:02.741984+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:52.241408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:52.241408+0000","last_clean_scrub_stamp":"2024-09-16T20:47:52.241408+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:01:53.651981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"214'641","reported_seq":7064,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.912231+0000","last_change":"2024-09-16T20:51:45.583936+0000","last_active":"2024-09-16T20:52:02.912231+0000","last_peered":"2024-09-16T20:52:02.912231+0000","last_clean":"2024-09-16T20:52:02.912231+0000","last_became_active":"2024-09-16T20:51:27.624220+0000","last_became_peered":"2024-09-16T20:51:27.624220+0000","last_unstale":"2024-09-16T20:52:02.912231+0000","last_undegraded":"2024-09-16T20:52:02.912231+0000","last_fullsized":"2024-09-16T20:52:02.912231+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":641,"log_dups_size":0,"ondisk_log_size":641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:13:47.571246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022745209999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6921,"num_read_kb":32984,"num_write":827,"num_write_kb":7103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":4839295,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"217'639","reported_seq":16470,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911021+0000","last_change":"2024-09-16T20:51:45.578750+0000","last_active":"2024-09-16T20:52:02.911021+0000","last_peered":"2024-09-16T20:52:02.911021+0000","last_clean":"2024-09-16T20:52:02.911021+0000","last_became_active":"2024-09-16T20:51:17.575136+0000","last_became_peered":"2024-09-16T20:51:17.575136+0000","last_unstale":"2024-09-16T20:52:02.911021+0000","last_undegraded":"2024-09-16T20:52:02.911021+0000","last_fullsized":"2024-09-16T20:52:02.911021+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":639,"log_dups_size":0,"ondisk_log_size":639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:32:32.550808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022726539999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":15159,"num_read_kb":101301,"num_write":561,"num_write_kb":29560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":790,"num_bytes_recovered":48497898,"num_keys_recovered":60,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"217'568","reported_seq":9847,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911042+0000","last_change":"2024-09-16T20:51:45.578068+0000","last_active":"2024-09-16T20:52:02.911042+0000","last_peered":"2024-09-16T20:52:02.911042+0000","last_clean":"2024-09-16T20:52:02.911042+0000","last_became_active":"2024-09-16T20:51:03.590152+0000","last_became_peered":"2024-09-16T20:51:03.590152+0000","last_unstale":"2024-09-16T20:52:02.911042+0000","last_undegraded":"2024-09-16T20:52:02.911042+0000","last_fullsized":"2024-09-16T20:52:02.911042+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":568,"log_dups_size":0,"ondisk_log_size":568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:21:38.569104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017317439999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":9605,"num_read_kb":51536,"num_write":618,"num_write_kb":12681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":14274951,"num_keys_recovered":91,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"218'468","reported_seq":8097,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.912037+0000","last_change":"2024-09-16T20:51:45.582215+0000","last_active":"2024-09-16T20:52:02.912037+0000","last_peered":"2024-09-16T20:52:02.912037+0000","last_clean":"2024-09-16T20:52:02.912037+0000","last_became_active":"2024-09-16T20:51:16.575698+0000","last_became_peered":"2024-09-16T20:51:16.575698+0000","last_unstale":"2024-09-16T20:52:02.912037+0000","last_undegraded":"2024-09-16T20:52:02.912037+0000","last_fullsized":"2024-09-16T20:52:02.912037+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:15:28.632535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020731130000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":7052,"num_read_kb":42310,"num_write":428,"num_write_kb":14897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":445,"num_bytes_recovered":8390493,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"216'726","reported_seq":22773,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911294+0000","last_change":"2024-09-16T20:51:45.579437+0000","last_active":"2024-09-16T20:52:02.911294+0000","last_peered":"2024-09-16T20:52:02.911294+0000","last_clean":"2024-09-16T20:52:02.911294+0000","last_became_active":"2024-09-16T20:51:17.575222+0000","last_became_peered":"2024-09-16T20:51:17.575222+0000","last_unstale":"2024-09-16T20:52:02.911294+0000","last_undegraded":"2024-09-16T20:52:02.911294+0000","last_fullsized":"2024-09-16T20:52:02.911294+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":726,"log_dups_size":0,"ondisk_log_size":726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:44:38.139123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018824849999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":22418,"num_read_kb":91945,"num_write":909,"num_write_kb":19380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":627,"num_bytes_recovered":71963834,"num_keys_recovered":119,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"218'1128","reported_seq":18644,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.742261+0000","last_change":"2024-09-16T20:51:45.585311+0000","last_active":"2024-09-16T20:52:02.742261+0000","last_peered":"2024-09-16T20:52:02.742261+0000","last_clean":"2024-09-16T20:52:02.742261+0000","last_became_active":"2024-09-16T20:51:05.581172+0000","last_became_peered":"2024-09-16T20:51:05.581172+0000","last_unstale":"2024-09-16T20:52:02.742261+0000","last_undegraded":"2024-09-16T20:52:02.742261+0000","last_fullsized":"2024-09-16T20:52:02.742261+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:27:15.820301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002500038,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":19435,"num_read_kb":70139,"num_write":1420,"num_write_kb":21576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":16987444,"num_keys_recovered":40,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"217'581","reported_seq":9275,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.741548+0000","last_change":"2024-09-16T20:51:45.581620+0000","last_active":"2024-09-16T20:52:02.741548+0000","last_peered":"2024-09-16T20:52:02.741548+0000","last_clean":"2024-09-16T20:52:02.741548+0000","last_became_active":"2024-09-16T20:51:27.627520+0000","last_became_peered":"2024-09-16T20:51:27.627520+0000","last_unstale":"2024-09-16T20:52:02.741548+0000","last_undegraded":"2024-09-16T20:52:02.741548+0000","last_fullsized":"2024-09-16T20:52:02.741548+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":581,"log_dups_size":0,"ondisk_log_size":581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:30:31.740673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012113930000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":8329,"num_read_kb":54907,"num_write":641,"num_write_kb":11423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":479,"num_bytes_recovered":14706083,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"217'534","reported_seq":13756,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911354+0000","last_change":"2024-09-16T20:51:45.580500+0000","last_active":"2024-09-16T20:52:02.911354+0000","last_peered":"2024-09-16T20:52:02.911354+0000","last_clean":"2024-09-16T20:52:02.911354+0000","last_became_active":"2024-09-16T20:51:05.581058+0000","last_became_peered":"2024-09-16T20:51:05.581058+0000","last_unstale":"2024-09-16T20:52:02.911354+0000","last_undegraded":"2024-09-16T20:52:02.911354+0000","last_fullsized":"2024-09-16T20:52:02.911354+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:55:20.810743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026424640000000002,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":13544,"num_read_kb":49574,"num_write":426,"num_write_kb":14576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":284,"num_bytes_recovered":9641443,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"218'691","reported_seq":13753,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.741972+0000","last_change":"2024-09-16T20:51:45.584709+0000","last_active":"2024-09-16T20:52:02.741972+0000","last_peered":"2024-09-16T20:52:02.741972+0000","last_clean":"2024-09-16T20:52:02.741972+0000","last_became_active":"2024-09-16T20:49:12.801013+0000","last_became_peered":"2024-09-16T20:49:12.801013+0000","last_unstale":"2024-09-16T20:52:02.741972+0000","last_undegraded":"2024-09-16T20:52:02.741972+0000","last_fullsized":"2024-09-16T20:52:02.741972+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":691,"log_dups_size":0,"ondisk_log_size":691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:13:59.069699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001757113,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":15597,"num_read_kb":58377,"num_write":523,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":99,"num_bytes_recovered":5210742,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"218'888","reported_seq":14337,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.741292+0000","last_change":"2024-09-16T20:51:45.579465+0000","last_active":"2024-09-16T20:52:02.741292+0000","last_peered":"2024-09-16T20:52:02.741292+0000","last_clean":"2024-09-16T20:52:02.741292+0000","last_became_active":"2024-09-16T20:51:26.624407+0000","last_became_peered":"2024-09-16T20:51:26.624407+0000","last_unstale":"2024-09-16T20:52:02.741292+0000","last_undegraded":"2024-09-16T20:52:02.741292+0000","last_fullsized":"2024-09-16T20:52:02.741292+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:59:50.323236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029313379999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":16446,"num_read_kb":75991,"num_write":874,"num_write_kb":24549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":351,"num_bytes_recovered":38747648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"211'608","reported_seq":20163,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911705+0000","last_change":"2024-09-16T20:51:45.643842+0000","last_active":"2024-09-16T20:52:02.911705+0000","last_peered":"2024-09-16T20:52:02.911705+0000","last_clean":"2024-09-16T20:52:02.911705+0000","last_became_active":"2024-09-16T20:51:03.589742+0000","last_became_peered":"2024-09-16T20:51:03.589742+0000","last_unstale":"2024-09-16T20:52:02.911705+0000","last_undegraded":"2024-09-16T20:52:02.911705+0000","last_fullsized":"2024-09-16T20:52:02.911705+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":608,"log_dups_size":0,"ondisk_log_size":608,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:49:22.579735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024335089999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":16969,"num_read_kb":76220,"num_write":613,"num_write_kb":13841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":486,"num_bytes_recovered":15329756,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"216'461","reported_seq":12004,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.741986+0000","last_change":"2024-09-16T20:51:45.586711+0000","last_active":"2024-09-16T20:52:02.741986+0000","last_peered":"2024-09-16T20:52:02.741986+0000","last_clean":"2024-09-16T20:52:02.741986+0000","last_became_active":"2024-09-16T20:51:03.595360+0000","last_became_peered":"2024-09-16T20:51:03.595360+0000","last_unstale":"2024-09-16T20:52:02.741986+0000","last_undegraded":"2024-09-16T20:52:02.741986+0000","last_fullsized":"2024-09-16T20:52:02.741986+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":461,"log_dups_size":0,"ondisk_log_size":461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:47:40.600095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035295309999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":11262,"num_read_kb":35639,"num_write":562,"num_write_kb":13645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":198,"num_bytes_recovered":14217900,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":247,"num_object_clones":0,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":243,"num_read":448597,"num_read_kb":1798155,"num_write":23462,"num_write_kb":469671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12645,"num_bytes_recovered":606441956,"num_keys_recovered":1497,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22149,"ondisk_log_size":22149,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574904,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52588,"kb_used_data":3344,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319252,"statfs":{"total":96636764160,"available":96582914048,"internally_reserved":0,"allocated":3424256,"data_stored":1774023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574904,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74140,"kb_used_data":2760,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94297700,"statfs":{"total":96636764160,"available":96560844800,"internally_reserved":0,"allocated":2826240,"data_stored":1183636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73071047},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574904,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69980,"kb_used_data":2760,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94301860,"statfs":{"total":96636764160,"available":96565104640,"internally_reserved":0,"allocated":2826240,"data_stored":1183636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574903,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70568,"kb_used_data":3344,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94301272,"statfs":{"total":96636764160,"available":96564502528,"internally_reserved":0,"allocated":3424256,"data_stored":1774023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574904,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":96420,"kb_used_data":2764,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":94275420,"statfs":{"total":96636764160,"available":96538030080,"internally_reserved":0,"allocated":2830336,"data_stored":1183655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":95877574},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640311,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60320,"kb_used_data":2764,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311520,"statfs":{"total":96636764160,"available":96574996480,"internally_reserved":0,"allocated":2830336,"data_stored":1183655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T20:52:07.559 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T20:52:07.858 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:52:07.858 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":235,"fsid":"e28af4a7-5fcd-4de8-8f77-a2b575361bb1","created":"2024-09-16T20:47:39.760609+0000","modified":"2024-09-16T20:52:02.737578+0000","last_up_change":"2024-09-16T20:47:50.238013+0000","last_in_change":"2024-09-16T20:51:22.595086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T20:47:52.220750+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T20:47:56.756666+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-16T20:48:06.418889+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"218","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":48,"snap_epoch":218,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c3cb2d7-7d23-4566-9ca1-d7b08e12360c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":228,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":2744025949},{"type":"v1","addr":"172.21.15.29:6817","nonce":2744025949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":2744025949},{"type":"v1","addr":"172.21.15.29:6819","nonce":2744025949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":2744025949},{"type":"v1","addr":"172.21.15.29:6823","nonce":2744025949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":2744025949},{"type":"v1","addr":"172.21.15.29:6821","nonce":2744025949}]},"public_addr":"172.21.15.29:6817/2744025949","cluster_addr":"172.21.15.29:6819/2744025949","heartbeat_back_addr":"172.21.15.29:6823/2744025949","heartbeat_front_addr":"172.21.15.29:6821/2744025949","state":["exists","up"]},{"osd":1,"uuid":"5cf8c20c-1ed2-42ae-a8b0-c88cc4e36bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":208628762},{"type":"v1","addr":"172.21.15.29:6801","nonce":208628762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":208628762},{"type":"v1","addr":"172.21.15.29:6803","nonce":208628762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":208628762},{"type":"v1","addr":"172.21.15.29:6807","nonce":208628762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":208628762},{"type":"v1","addr":"172.21.15.29:6805","nonce":208628762}]},"public_addr":"172.21.15.29:6801/208628762","cluster_addr":"172.21.15.29:6803/208628762","heartbeat_back_addr":"172.21.15.29:6807/208628762","heartbeat_front_addr":"172.21.15.29:6805/208628762","state":["exists","up"]},{"osd":2,"uuid":"075d16da-82bf-449b-9f6e-542c9e8d810b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3447426536},{"type":"v1","addr":"172.21.15.29:6809","nonce":3447426536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3447426536},{"type":"v1","addr":"172.21.15.29:6811","nonce":3447426536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3447426536},{"type":"v1","addr":"172.21.15.29:6815","nonce":3447426536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":3447426536},{"type":"v1","addr":"172.21.15.29:6813","nonce":3447426536}]},"public_addr":"172.21.15.29:6809/3447426536","cluster_addr":"172.21.15.29:6811/3447426536","heartbeat_back_addr":"172.21.15.29:6815/3447426536","heartbeat_front_addr":"172.21.15.29:6813/3447426536","state":["exists","up"]},{"osd":3,"uuid":"0612be38-3a24-41b6-aba4-5a2fe985165e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3687583891},{"type":"v1","addr":"172.21.15.43:6809","nonce":3687583891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3687583891},{"type":"v1","addr":"172.21.15.43:6811","nonce":3687583891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3687583891},{"type":"v1","addr":"172.21.15.43:6815","nonce":3687583891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3687583891},{"type":"v1","addr":"172.21.15.43:6813","nonce":3687583891}]},"public_addr":"172.21.15.43:6809/3687583891","cluster_addr":"172.21.15.43:6811/3687583891","heartbeat_back_addr":"172.21.15.43:6815/3687583891","heartbeat_front_addr":"172.21.15.43:6813/3687583891","state":["exists","up"]},{"osd":4,"uuid":"be590bf8-e9fa-44ca-a471-6f6bcb087ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":2505220646},{"type":"v1","addr":"172.21.15.43:6801","nonce":2505220646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":2505220646},{"type":"v1","addr":"172.21.15.43:6803","nonce":2505220646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":2505220646},{"type":"v1","addr":"172.21.15.43:6807","nonce":2505220646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":2505220646},{"type":"v1","addr":"172.21.15.43:6805","nonce":2505220646}]},"public_addr":"172.21.15.43:6801/2505220646","cluster_addr":"172.21.15.43:6803/2505220646","heartbeat_back_addr":"172.21.15.43:6807/2505220646","heartbeat_front_addr":"172.21.15.43:6805/2505220646","state":["exists","up"]},{"osd":5,"uuid":"4a208e30-031b-4a9f-af52-e26d4a039ea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":198,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1517367307},{"type":"v1","addr":"172.21.15.43:6817","nonce":1517367307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":1517367307},{"type":"v1","addr":"172.21.15.43:6819","nonce":1517367307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":1517367307},{"type":"v1","addr":"172.21.15.43:6823","nonce":1517367307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1517367307},{"type":"v1","addr":"172.21.15.43:6821","nonce":1517367307}]},"public_addr":"172.21.15.43:6817/1517367307","cluster_addr":"172.21.15.43:6819/1517367307","heartbeat_back_addr":"172.21.15.43:6823/1517367307","heartbeat_front_addr":"172.21.15.43:6821/1517367307","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T20:47:47.887412+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T20:47:47.658782+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T20:47:47.453752+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T20:47:47.487992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T20:47:46.943344+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T20:47:47.901547+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1b","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T20:52:08.872 INFO:tasks.ceph:Scrubbing osd.0 2024-09-16T20:52:08.872 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-16T20:52:09.012 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:52:09.012 INFO:teuthology.orchestra.run.smithi029.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T20:52:09.012 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:52:09.023 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-16T20:52:09.323 INFO:teuthology.orchestra.run.smithi029.stderr:instructed osd(s) 0 to deep-scrub 2024-09-16T20:52:09.336 INFO:tasks.ceph:Scrubbing osd.1 2024-09-16T20:52:09.336 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-16T20:52:09.461 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:52:09.461 INFO:teuthology.orchestra.run.smithi029.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T20:52:09.461 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:52:09.472 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-16T20:52:09.769 INFO:teuthology.orchestra.run.smithi029.stderr:instructed osd(s) 1 to deep-scrub 2024-09-16T20:52:09.782 INFO:tasks.ceph:Scrubbing osd.2 2024-09-16T20:52:09.782 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-16T20:52:09.908 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:52:09.908 INFO:teuthology.orchestra.run.smithi029.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T20:52:09.908 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:52:09.918 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-16T20:52:10.213 INFO:teuthology.orchestra.run.smithi029.stderr:instructed osd(s) 2 to deep-scrub 2024-09-16T20:52:10.227 INFO:tasks.ceph:Scrubbing osd.3 2024-09-16T20:52:10.227 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-16T20:52:10.352 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:52:10.352 INFO:teuthology.orchestra.run.smithi029.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T20:52:10.352 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:52:10.363 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-16T20:52:10.660 INFO:teuthology.orchestra.run.smithi029.stderr:instructed osd(s) 3 to deep-scrub 2024-09-16T20:52:10.673 INFO:tasks.ceph:Scrubbing osd.4 2024-09-16T20:52:10.674 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-16T20:52:10.799 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:52:10.800 INFO:teuthology.orchestra.run.smithi029.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T20:52:10.800 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:52:10.810 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-16T20:52:11.106 INFO:teuthology.orchestra.run.smithi029.stderr:instructed osd(s) 4 to deep-scrub 2024-09-16T20:52:11.118 INFO:tasks.ceph:Scrubbing osd.5 2024-09-16T20:52:11.119 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-16T20:52:11.249 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-16T20:52:11.249 INFO:teuthology.orchestra.run.smithi029.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T20:52:11.249 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-16T20:52:11.260 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-16T20:52:11.557 INFO:teuthology.orchestra.run.smithi029.stderr:instructed osd(s) 5 to deep-scrub 2024-09-16T20:52:11.570 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T20:52:11.867 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:52:11.868 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-09-16T20:52:11.879 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":361,"stamp":"2024-09-16T20:52:10.250608+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":251,"num_object_clones":0,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":243,"num_read":448663,"num_read_kb":1798210,"num_write":23565,"num_write_kb":471301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12652,"num_bytes_recovered":607622711,"num_keys_recovered":1497,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22224,"ondisk_log_size":22224,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":424016,"kb_used_data":17736,"kb_used_omap":9,"kb_used_meta":406134,"kb_avail":565807024,"statfs":{"total":579820584960,"available":579386392576,"internally_reserved":0,"allocated":18161664,"data_stored":8282628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9567,"internal_metadata":415881889},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.516518"},"pg_stats":[{"pgid":"3.1f","version":"218'359","reported_seq":10822,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911515+0000","last_change":"2024-09-16T20:51:45.642922+0000","last_active":"2024-09-16T20:52:02.911515+0000","last_peered":"2024-09-16T20:52:02.911515+0000","last_clean":"2024-09-16T20:52:02.911515+0000","last_became_active":"2024-09-16T20:49:12.801566+0000","last_became_peered":"2024-09-16T20:49:12.801566+0000","last_unstale":"2024-09-16T20:52:02.911515+0000","last_undegraded":"2024-09-16T20:52:02.911515+0000","last_fullsized":"2024-09-16T20:52:02.911515+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:20:11.666141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016075530000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10091,"num_read_kb":31685,"num_write":315,"num_write_kb":5043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":2125998,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"218'905","reported_seq":28772,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.912100+0000","last_change":"2024-09-16T20:51:45.580852+0000","last_active":"2024-09-16T20:52:02.912100+0000","last_peered":"2024-09-16T20:52:02.912100+0000","last_clean":"2024-09-16T20:52:02.912100+0000","last_became_active":"2024-09-16T20:51:27.628085+0000","last_became_peered":"2024-09-16T20:51:27.628085+0000","last_unstale":"2024-09-16T20:52:02.912100+0000","last_undegraded":"2024-09-16T20:52:02.912100+0000","last_fullsized":"2024-09-16T20:52:02.912100+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":905,"log_dups_size":0,"ondisk_log_size":905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:59:41.269032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001444383,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":29337,"num_read_kb":83641,"num_write":1040,"num_write_kb":10328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":562,"num_bytes_recovered":11317531,"num_keys_recovered":154,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"214'707","reported_seq":12942,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.741297+0000","last_change":"2024-09-16T20:51:45.580224+0000","last_active":"2024-09-16T20:52:02.741297+0000","last_peered":"2024-09-16T20:52:02.741297+0000","last_clean":"2024-09-16T20:52:02.741297+0000","last_became_active":"2024-09-16T20:51:26.624314+0000","last_became_peered":"2024-09-16T20:51:26.624314+0000","last_unstale":"2024-09-16T20:52:02.741297+0000","last_undegraded":"2024-09-16T20:52:02.741297+0000","last_fullsized":"2024-09-16T20:52:02.741297+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":707,"log_dups_size":0,"ondisk_log_size":707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:47:19.682479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014726419999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":13066,"num_read_kb":103238,"num_write":750,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":387,"num_bytes_recovered":35619030,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"218'714","reported_seq":4372,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.742231+0000","last_change":"2024-09-16T20:51:45.586449+0000","last_active":"2024-09-16T20:52:02.742231+0000","last_peered":"2024-09-16T20:52:02.742231+0000","last_clean":"2024-09-16T20:52:02.742231+0000","last_became_active":"2024-09-16T20:51:04.577608+0000","last_became_peered":"2024-09-16T20:51:04.577608+0000","last_unstale":"2024-09-16T20:52:02.742231+0000","last_undegraded":"2024-09-16T20:52:02.742231+0000","last_fullsized":"2024-09-16T20:52:02.742231+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:15:00.043848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034574200000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":4615,"num_read_kb":24234,"num_write":996,"num_write_kb":15567,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":25643873,"num_keys_recovered":142,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"217'872","reported_seq":11160,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911930+0000","last_change":"2024-09-16T20:51:49.582917+0000","last_active":"2024-09-16T20:52:02.911930+0000","last_peered":"2024-09-16T20:52:02.911930+0000","last_clean":"2024-09-16T20:52:02.911930+0000","last_became_active":"2024-09-16T20:51:49.579980+0000","last_became_peered":"2024-09-16T20:51:49.579980+0000","last_unstale":"2024-09-16T20:52:02.911930+0000","last_undegraded":"2024-09-16T20:52:02.911930+0000","last_fullsized":"2024-09-16T20:52:02.911930+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":872,"log_dups_size":0,"ondisk_log_size":872,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:13:38.941261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001184378,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":10970,"num_read_kb":44437,"num_write":1151,"num_write_kb":16781,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":956,"num_bytes_recovered":37659232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"217'1221","reported_seq":19862,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.742042+0000","last_change":"2024-09-16T20:51:45.585643+0000","last_active":"2024-09-16T20:52:02.742042+0000","last_peered":"2024-09-16T20:52:02.742042+0000","last_clean":"2024-09-16T20:52:02.742042+0000","last_became_active":"2024-09-16T20:49:12.802869+0000","last_became_peered":"2024-09-16T20:49:12.802869+0000","last_unstale":"2024-09-16T20:52:02.742042+0000","last_undegraded":"2024-09-16T20:52:02.742042+0000","last_fullsized":"2024-09-16T20:52:02.742042+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":1221,"log_dups_size":0,"ondisk_log_size":1221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:17:13.191250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026611790000000001,"stat_sum":{"num_bytes":19,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":25740,"num_read_kb":67599,"num_write":1269,"num_write_kb":16392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":1051004,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"218'770","reported_seq":12989,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.912288+0000","last_change":"2024-09-16T20:51:45.584848+0000","last_active":"2024-09-16T20:52:02.912288+0000","last_peered":"2024-09-16T20:52:02.912288+0000","last_clean":"2024-09-16T20:52:02.912288+0000","last_became_active":"2024-09-16T20:51:16.577039+0000","last_became_peered":"2024-09-16T20:51:16.577039+0000","last_unstale":"2024-09-16T20:52:02.912288+0000","last_undegraded":"2024-09-16T20:52:02.912288+0000","last_fullsized":"2024-09-16T20:52:02.912288+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":770,"log_dups_size":0,"ondisk_log_size":770,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:26:39.219016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023604640000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":10011,"num_read_kb":47438,"num_write":753,"num_write_kb":12180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":12922788,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"216'458","reported_seq":13762,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911501+0000","last_change":"2024-09-16T20:51:45.581009+0000","last_active":"2024-09-16T20:52:02.911501+0000","last_peered":"2024-09-16T20:52:02.911501+0000","last_clean":"2024-09-16T20:52:02.911501+0000","last_became_active":"2024-09-16T20:51:15.575429+0000","last_became_peered":"2024-09-16T20:51:15.575429+0000","last_unstale":"2024-09-16T20:52:02.911501+0000","last_undegraded":"2024-09-16T20:52:02.911501+0000","last_fullsized":"2024-09-16T20:52:02.911501+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":458,"log_dups_size":0,"ondisk_log_size":458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:42:42.274481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001906629,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":14274,"num_read_kb":52840,"num_write":349,"num_write_kb":9355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":231,"num_bytes_recovered":11122736,"num_keys_recovered":66,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"217'981","reported_seq":8058,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911740+0000","last_change":"2024-09-16T20:51:45.642868+0000","last_active":"2024-09-16T20:52:02.911740+0000","last_peered":"2024-09-16T20:52:02.911740+0000","last_clean":"2024-09-16T20:52:02.911740+0000","last_became_active":"2024-09-16T20:51:03.592284+0000","last_became_peered":"2024-09-16T20:51:03.592284+0000","last_unstale":"2024-09-16T20:52:02.911740+0000","last_undegraded":"2024-09-16T20:52:02.911740+0000","last_fullsized":"2024-09-16T20:52:02.911740+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":981,"log_dups_size":0,"ondisk_log_size":981,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:35:23.178350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015146339999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":11609,"num_read_kb":15811,"num_write":975,"num_write_kb":5184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":280,"num_bytes_recovered":1580920,"num_keys_recovered":144,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"217'1277","reported_seq":11828,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.741549+0000","last_change":"2024-09-16T20:51:45.581650+0000","last_active":"2024-09-16T20:52:02.741549+0000","last_peered":"2024-09-16T20:52:02.741549+0000","last_clean":"2024-09-16T20:52:02.741549+0000","last_became_active":"2024-09-16T20:51:27.624341+0000","last_became_peered":"2024-09-16T20:51:27.624341+0000","last_unstale":"2024-09-16T20:52:02.741549+0000","last_undegraded":"2024-09-16T20:52:02.741549+0000","last_fullsized":"2024-09-16T20:52:02.741549+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":1277,"log_dups_size":0,"ondisk_log_size":1277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:01:31.923277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011831579999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":17170,"num_read_kb":43312,"num_write":1584,"num_write_kb":10211,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":542,"num_bytes_recovered":21909352,"num_keys_recovered":70,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"218'595","reported_seq":14001,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911281+0000","last_change":"2024-09-16T20:51:45.579668+0000","last_active":"2024-09-16T20:52:02.911281+0000","last_peered":"2024-09-16T20:52:02.911281+0000","last_clean":"2024-09-16T20:52:02.911281+0000","last_became_active":"2024-09-16T20:51:16.577620+0000","last_became_peered":"2024-09-16T20:51:16.577620+0000","last_unstale":"2024-09-16T20:52:02.911281+0000","last_undegraded":"2024-09-16T20:52:02.911281+0000","last_fullsized":"2024-09-16T20:52:02.911281+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":595,"log_dups_size":0,"ondisk_log_size":595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:23:49.832618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022662979999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":12843,"num_read_kb":77657,"num_write":690,"num_write_kb":15118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":562,"num_bytes_recovered":17171700,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"218'370","reported_seq":19983,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.742221+0000","last_change":"2024-09-16T20:51:57.720592+0000","last_active":"2024-09-16T20:52:02.742221+0000","last_peered":"2024-09-16T20:52:02.742221+0000","last_clean":"2024-09-16T20:52:02.742221+0000","last_became_active":"2024-09-16T20:51:57.718930+0000","last_became_peered":"2024-09-16T20:51:57.718930+0000","last_unstale":"2024-09-16T20:52:02.742221+0000","last_undegraded":"2024-09-16T20:52:02.742221+0000","last_fullsized":"2024-09-16T20:52:02.742221+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:08:04.217338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014936070000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":18755,"num_read_kb":45565,"num_write":465,"num_write_kb":13208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":540,"num_bytes_recovered":30162312,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":460,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.742331+0000","last_change":"2024-09-16T20:51:00.578451+0000","last_active":"2024-09-16T20:52:02.742331+0000","last_peered":"2024-09-16T20:52:02.742331+0000","last_clean":"2024-09-16T20:52:02.742331+0000","last_became_active":"2024-09-16T20:51:00.578081+0000","last_became_peered":"2024-09-16T20:51:00.578081+0000","last_unstale":"2024-09-16T20:52:02.742331+0000","last_undegraded":"2024-09-16T20:52:02.742331+0000","last_fullsized":"2024-09-16T20:52:02.742331+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:35:02.412998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091339500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"218'1028","reported_seq":19427,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.912471+0000","last_change":"2024-09-16T20:51:45.582985+0000","last_active":"2024-09-16T20:52:02.912471+0000","last_peered":"2024-09-16T20:52:02.912471+0000","last_clean":"2024-09-16T20:52:02.912471+0000","last_became_active":"2024-09-16T20:51:16.577104+0000","last_became_peered":"2024-09-16T20:51:16.577104+0000","last_unstale":"2024-09-16T20:52:02.912471+0000","last_undegraded":"2024-09-16T20:52:02.912471+0000","last_fullsized":"2024-09-16T20:52:02.912471+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":1028,"log_dups_size":0,"ondisk_log_size":1028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:56:34.623379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013907780000000001,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":20870,"num_read_kb":85867,"num_write":1218,"num_write_kb":25908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":29601482,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":463,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.912490+0000","last_change":"2024-09-16T20:51:00.577927+0000","last_active":"2024-09-16T20:52:02.912490+0000","last_peered":"2024-09-16T20:52:02.912490+0000","last_clean":"2024-09-16T20:52:02.912490+0000","last_became_active":"2024-09-16T20:51:00.577653+0000","last_became_peered":"2024-09-16T20:51:00.577653+0000","last_unstale":"2024-09-16T20:52:02.912490+0000","last_undegraded":"2024-09-16T20:52:02.912490+0000","last_fullsized":"2024-09-16T20:52:02.912490+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:12:13.831668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043659299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"214'653","reported_seq":25408,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.912290+0000","last_change":"2024-09-16T20:51:45.583929+0000","last_active":"2024-09-16T20:52:02.912290+0000","last_peered":"2024-09-16T20:52:02.912290+0000","last_clean":"2024-09-16T20:52:02.912290+0000","last_became_active":"2024-09-16T20:51:15.575607+0000","last_became_peered":"2024-09-16T20:51:15.575607+0000","last_unstale":"2024-09-16T20:52:02.912290+0000","last_undegraded":"2024-09-16T20:52:02.912290+0000","last_fullsized":"2024-09-16T20:52:02.912290+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":653,"log_dups_size":0,"ondisk_log_size":653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:42:18.817016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022924479999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":24178,"num_read_kb":81081,"num_write":502,"num_write_kb":17573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":20407955,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":448,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.742362+0000","last_change":"2024-09-16T20:48:00.278054+0000","last_active":"2024-09-16T20:52:02.742362+0000","last_peered":"2024-09-16T20:52:02.742362+0000","last_clean":"2024-09-16T20:52:02.742362+0000","last_became_active":"2024-09-16T20:47:58.272861+0000","last_became_peered":"2024-09-16T20:47:58.272861+0000","last_unstale":"2024-09-16T20:52:02.742362+0000","last_undegraded":"2024-09-16T20:52:02.742362+0000","last_fullsized":"2024-09-16T20:52:02.742362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:13:59.135547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012429489999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"213'491","reported_seq":3884,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.912098+0000","last_change":"2024-09-16T20:51:45.583107+0000","last_active":"2024-09-16T20:52:02.912098+0000","last_peered":"2024-09-16T20:52:02.912098+0000","last_clean":"2024-09-16T20:52:02.912098+0000","last_became_active":"2024-09-16T20:49:12.801810+0000","last_became_peered":"2024-09-16T20:49:12.801810+0000","last_unstale":"2024-09-16T20:52:02.912098+0000","last_undegraded":"2024-09-16T20:52:02.912098+0000","last_fullsized":"2024-09-16T20:52:02.912098+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":491,"log_dups_size":0,"ondisk_log_size":491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:48:19.573269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015370990000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":3741,"num_read_kb":11405,"num_write":512,"num_write_kb":6561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":2071114,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":340,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911584+0000","last_change":"2024-09-16T20:51:12.615838+0000","last_active":"2024-09-16T20:52:02.911584+0000","last_peered":"2024-09-16T20:52:02.911584+0000","last_clean":"2024-09-16T20:52:02.911584+0000","last_became_active":"2024-09-16T20:51:12.615412+0000","last_became_peered":"2024-09-16T20:51:12.615412+0000","last_unstale":"2024-09-16T20:52:02.911584+0000","last_undegraded":"2024-09-16T20:52:02.911584+0000","last_fullsized":"2024-09-16T20:52:02.911584+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:17:12.150193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"217'668","reported_seq":19715,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.741476+0000","last_change":"2024-09-16T20:51:45.580833+0000","last_active":"2024-09-16T20:52:02.741476+0000","last_peered":"2024-09-16T20:52:02.741476+0000","last_clean":"2024-09-16T20:52:02.741476+0000","last_became_active":"2024-09-16T20:51:27.624519+0000","last_became_peered":"2024-09-16T20:51:27.624519+0000","last_unstale":"2024-09-16T20:52:02.741476+0000","last_undegraded":"2024-09-16T20:52:02.741476+0000","last_fullsized":"2024-09-16T20:52:02.741476+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":668,"log_dups_size":0,"ondisk_log_size":668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:44:59.098443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001777441,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":18081,"num_read_kb":57429,"num_write":654,"num_write_kb":12584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":564,"num_bytes_recovered":27092048,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"218'728","reported_seq":8639,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911256+0000","last_change":"2024-09-16T20:51:45.580666+0000","last_active":"2024-09-16T20:52:02.911256+0000","last_peered":"2024-09-16T20:52:02.911256+0000","last_clean":"2024-09-16T20:52:02.911256+0000","last_became_active":"2024-09-16T20:51:27.626032+0000","last_became_peered":"2024-09-16T20:51:27.626032+0000","last_unstale":"2024-09-16T20:52:02.911256+0000","last_undegraded":"2024-09-16T20:52:02.911256+0000","last_fullsized":"2024-09-16T20:52:02.911256+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":728,"log_dups_size":0,"ondisk_log_size":728,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:28:16.242279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025014909999999998,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":7673,"num_read_kb":36418,"num_write":666,"num_write_kb":13145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":523,"num_bytes_recovered":11083665,"num_keys_recovered":14,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":137,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911158+0000","last_change":"2024-09-16T20:51:00.578241+0000","last_active":"2024-09-16T20:52:02.911158+0000","last_peered":"2024-09-16T20:52:02.911158+0000","last_clean":"2024-09-16T20:52:02.911158+0000","last_became_active":"2024-09-16T20:51:00.574693+0000","last_became_peered":"2024-09-16T20:51:00.574693+0000","last_unstale":"2024-09-16T20:52:02.911158+0000","last_undegraded":"2024-09-16T20:52:02.911158+0000","last_fullsized":"2024-09-16T20:52:02.911158+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:35:20.550930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062077499999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"216'498","reported_seq":18808,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911630+0000","last_change":"2024-09-16T20:51:45.643842+0000","last_active":"2024-09-16T20:52:02.911630+0000","last_peered":"2024-09-16T20:52:02.911630+0000","last_clean":"2024-09-16T20:52:02.911630+0000","last_became_active":"2024-09-16T20:51:26.626185+0000","last_became_peered":"2024-09-16T20:51:26.626185+0000","last_unstale":"2024-09-16T20:52:02.911630+0000","last_undegraded":"2024-09-16T20:52:02.911630+0000","last_fullsized":"2024-09-16T20:52:02.911630+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":498,"log_dups_size":0,"ondisk_log_size":498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:57:30.480666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023862639999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":19429,"num_read_kb":77391,"num_write":450,"num_write_kb":15765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":3862081,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":109,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911799+0000","last_change":"2024-09-16T20:51:12.615208+0000","last_active":"2024-09-16T20:52:02.911799+0000","last_peered":"2024-09-16T20:52:02.911799+0000","last_clean":"2024-09-16T20:52:02.911799+0000","last_became_active":"2024-09-16T20:51:12.614812+0000","last_became_peered":"2024-09-16T20:51:12.614812+0000","last_unstale":"2024-09-16T20:52:02.911799+0000","last_undegraded":"2024-09-16T20:52:02.911799+0000","last_fullsized":"2024-09-16T20:52:02.911799+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:37:11.741478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"215'312","reported_seq":5919,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.912408+0000","last_change":"2024-09-16T20:51:45.584820+0000","last_active":"2024-09-16T20:52:02.912408+0000","last_peered":"2024-09-16T20:52:02.912408+0000","last_clean":"2024-09-16T20:52:02.912408+0000","last_became_active":"2024-09-16T20:51:26.624469+0000","last_became_peered":"2024-09-16T20:51:26.624469+0000","last_unstale":"2024-09-16T20:52:02.912408+0000","last_undegraded":"2024-09-16T20:52:02.912408+0000","last_fullsized":"2024-09-16T20:52:02.912408+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:52:43.183901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025837220000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4323,"num_read_kb":33093,"num_write":273,"num_write_kb":12351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":324,"num_bytes_recovered":16931328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":89,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.741294+0000","last_change":"2024-09-16T20:51:23.608479+0000","last_active":"2024-09-16T20:52:02.741294+0000","last_peered":"2024-09-16T20:52:02.741294+0000","last_clean":"2024-09-16T20:52:02.741294+0000","last_became_active":"2024-09-16T20:51:23.608159+0000","last_became_peered":"2024-09-16T20:51:23.608159+0000","last_unstale":"2024-09-16T20:52:02.741294+0000","last_undegraded":"2024-09-16T20:52:02.741294+0000","last_fullsized":"2024-09-16T20:52:02.741294+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:06:21.085498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037636000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"217'609","reported_seq":7615,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911699+0000","last_change":"2024-09-16T20:51:45.582309+0000","last_active":"2024-09-16T20:52:02.911699+0000","last_peered":"2024-09-16T20:52:02.911699+0000","last_clean":"2024-09-16T20:52:02.911699+0000","last_became_active":"2024-09-16T20:51:26.622629+0000","last_became_peered":"2024-09-16T20:51:26.622629+0000","last_unstale":"2024-09-16T20:52:02.911699+0000","last_undegraded":"2024-09-16T20:52:02.911699+0000","last_fullsized":"2024-09-16T20:52:02.911699+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:40:49.603343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029005419999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":9084,"num_read_kb":37091,"num_write":448,"num_write_kb":13187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":848,"num_bytes_recovered":24298320,"num_keys_recovered":219,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":469,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.742037+0000","last_change":"2024-09-16T20:51:23.608488+0000","last_active":"2024-09-16T20:52:02.742037+0000","last_peered":"2024-09-16T20:52:02.742037+0000","last_clean":"2024-09-16T20:52:02.742037+0000","last_became_active":"2024-09-16T20:51:23.605321+0000","last_became_peered":"2024-09-16T20:51:23.605321+0000","last_unstale":"2024-09-16T20:52:02.742037+0000","last_undegraded":"2024-09-16T20:52:02.742037+0000","last_fullsized":"2024-09-16T20:52:02.742037+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:57.257926+0000","last_clean_scrub_stamp":"2024-09-16T20:47:57.257926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:50:46.322344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095771799999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":564,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.741984+0000","last_change":"2024-09-16T20:49:09.565188+0000","last_active":"2024-09-16T20:52:02.741984+0000","last_peered":"2024-09-16T20:52:02.741984+0000","last_clean":"2024-09-16T20:52:02.741984+0000","last_became_active":"2024-09-16T20:49:09.559571+0000","last_became_peered":"2024-09-16T20:49:09.559571+0000","last_unstale":"2024-09-16T20:52:02.741984+0000","last_undegraded":"2024-09-16T20:52:02.741984+0000","last_fullsized":"2024-09-16T20:52:02.741984+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:47:52.241408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:47:52.241408+0000","last_clean_scrub_stamp":"2024-09-16T20:47:52.241408+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:01:53.651981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"214'641","reported_seq":7064,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.912231+0000","last_change":"2024-09-16T20:51:45.583936+0000","last_active":"2024-09-16T20:52:02.912231+0000","last_peered":"2024-09-16T20:52:02.912231+0000","last_clean":"2024-09-16T20:52:02.912231+0000","last_became_active":"2024-09-16T20:51:27.624220+0000","last_became_peered":"2024-09-16T20:51:27.624220+0000","last_unstale":"2024-09-16T20:52:02.912231+0000","last_undegraded":"2024-09-16T20:52:02.912231+0000","last_fullsized":"2024-09-16T20:52:02.912231+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":641,"log_dups_size":0,"ondisk_log_size":641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:13:47.571246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022745209999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6921,"num_read_kb":32984,"num_write":827,"num_write_kb":7103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":4839295,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"217'639","reported_seq":16470,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911021+0000","last_change":"2024-09-16T20:51:45.578750+0000","last_active":"2024-09-16T20:52:02.911021+0000","last_peered":"2024-09-16T20:52:02.911021+0000","last_clean":"2024-09-16T20:52:02.911021+0000","last_became_active":"2024-09-16T20:51:17.575136+0000","last_became_peered":"2024-09-16T20:51:17.575136+0000","last_unstale":"2024-09-16T20:52:02.911021+0000","last_undegraded":"2024-09-16T20:52:02.911021+0000","last_fullsized":"2024-09-16T20:52:02.911021+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":639,"log_dups_size":0,"ondisk_log_size":639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:32:32.550808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022726539999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":15159,"num_read_kb":101301,"num_write":561,"num_write_kb":29560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":790,"num_bytes_recovered":48497898,"num_keys_recovered":60,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"217'568","reported_seq":9847,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911042+0000","last_change":"2024-09-16T20:51:45.578068+0000","last_active":"2024-09-16T20:52:02.911042+0000","last_peered":"2024-09-16T20:52:02.911042+0000","last_clean":"2024-09-16T20:52:02.911042+0000","last_became_active":"2024-09-16T20:51:03.590152+0000","last_became_peered":"2024-09-16T20:51:03.590152+0000","last_unstale":"2024-09-16T20:52:02.911042+0000","last_undegraded":"2024-09-16T20:52:02.911042+0000","last_fullsized":"2024-09-16T20:52:02.911042+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":568,"log_dups_size":0,"ondisk_log_size":568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:21:38.569104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017317439999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":9605,"num_read_kb":51536,"num_write":618,"num_write_kb":12681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":14274951,"num_keys_recovered":91,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"218'468","reported_seq":8097,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.912037+0000","last_change":"2024-09-16T20:51:45.582215+0000","last_active":"2024-09-16T20:52:02.912037+0000","last_peered":"2024-09-16T20:52:02.912037+0000","last_clean":"2024-09-16T20:52:02.912037+0000","last_became_active":"2024-09-16T20:51:16.575698+0000","last_became_peered":"2024-09-16T20:51:16.575698+0000","last_unstale":"2024-09-16T20:52:02.912037+0000","last_undegraded":"2024-09-16T20:52:02.912037+0000","last_fullsized":"2024-09-16T20:52:02.912037+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:15:28.632535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020731130000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":7052,"num_read_kb":42310,"num_write":428,"num_write_kb":14897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":445,"num_bytes_recovered":8390493,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"216'726","reported_seq":22773,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911294+0000","last_change":"2024-09-16T20:51:45.579437+0000","last_active":"2024-09-16T20:52:02.911294+0000","last_peered":"2024-09-16T20:52:02.911294+0000","last_clean":"2024-09-16T20:52:02.911294+0000","last_became_active":"2024-09-16T20:51:17.575222+0000","last_became_peered":"2024-09-16T20:51:17.575222+0000","last_unstale":"2024-09-16T20:52:02.911294+0000","last_undegraded":"2024-09-16T20:52:02.911294+0000","last_fullsized":"2024-09-16T20:52:02.911294+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":726,"log_dups_size":0,"ondisk_log_size":726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:44:38.139123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018824849999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":22418,"num_read_kb":91945,"num_write":909,"num_write_kb":19380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":627,"num_bytes_recovered":71963834,"num_keys_recovered":119,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"218'1128","reported_seq":18644,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.742261+0000","last_change":"2024-09-16T20:51:45.585311+0000","last_active":"2024-09-16T20:52:02.742261+0000","last_peered":"2024-09-16T20:52:02.742261+0000","last_clean":"2024-09-16T20:52:02.742261+0000","last_became_active":"2024-09-16T20:51:05.581172+0000","last_became_peered":"2024-09-16T20:51:05.581172+0000","last_unstale":"2024-09-16T20:52:02.742261+0000","last_undegraded":"2024-09-16T20:52:02.742261+0000","last_fullsized":"2024-09-16T20:52:02.742261+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:27:15.820301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002500038,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":19435,"num_read_kb":70139,"num_write":1420,"num_write_kb":21576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":16987444,"num_keys_recovered":40,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"217'581","reported_seq":9275,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.741548+0000","last_change":"2024-09-16T20:51:45.581620+0000","last_active":"2024-09-16T20:52:02.741548+0000","last_peered":"2024-09-16T20:52:02.741548+0000","last_clean":"2024-09-16T20:52:02.741548+0000","last_became_active":"2024-09-16T20:51:27.627520+0000","last_became_peered":"2024-09-16T20:51:27.627520+0000","last_unstale":"2024-09-16T20:52:02.741548+0000","last_undegraded":"2024-09-16T20:52:02.741548+0000","last_fullsized":"2024-09-16T20:52:02.741548+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":581,"log_dups_size":0,"ondisk_log_size":581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:30:31.740673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012113930000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":8329,"num_read_kb":54907,"num_write":641,"num_write_kb":11423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":479,"num_bytes_recovered":14706083,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"217'534","reported_seq":13756,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911354+0000","last_change":"2024-09-16T20:51:45.580500+0000","last_active":"2024-09-16T20:52:02.911354+0000","last_peered":"2024-09-16T20:52:02.911354+0000","last_clean":"2024-09-16T20:52:02.911354+0000","last_became_active":"2024-09-16T20:51:05.581058+0000","last_became_peered":"2024-09-16T20:51:05.581058+0000","last_unstale":"2024-09-16T20:52:02.911354+0000","last_undegraded":"2024-09-16T20:52:02.911354+0000","last_fullsized":"2024-09-16T20:52:02.911354+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:55:20.810743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026424640000000002,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":13544,"num_read_kb":49574,"num_write":426,"num_write_kb":14576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":284,"num_bytes_recovered":9641443,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"218'691","reported_seq":13753,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.741972+0000","last_change":"2024-09-16T20:51:45.584709+0000","last_active":"2024-09-16T20:52:02.741972+0000","last_peered":"2024-09-16T20:52:02.741972+0000","last_clean":"2024-09-16T20:52:02.741972+0000","last_became_active":"2024-09-16T20:49:12.801013+0000","last_became_peered":"2024-09-16T20:49:12.801013+0000","last_unstale":"2024-09-16T20:52:02.741972+0000","last_undegraded":"2024-09-16T20:52:02.741972+0000","last_fullsized":"2024-09-16T20:52:02.741972+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":691,"log_dups_size":0,"ondisk_log_size":691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:13:59.069699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001757113,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":15597,"num_read_kb":58377,"num_write":523,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":99,"num_bytes_recovered":5210742,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"218'888","reported_seq":14337,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.741292+0000","last_change":"2024-09-16T20:51:45.579465+0000","last_active":"2024-09-16T20:52:02.741292+0000","last_peered":"2024-09-16T20:52:02.741292+0000","last_clean":"2024-09-16T20:52:02.741292+0000","last_became_active":"2024-09-16T20:51:26.624407+0000","last_became_peered":"2024-09-16T20:51:26.624407+0000","last_unstale":"2024-09-16T20:52:02.741292+0000","last_undegraded":"2024-09-16T20:52:02.741292+0000","last_fullsized":"2024-09-16T20:52:02.741292+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:59:50.323236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029313379999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":16446,"num_read_kb":75991,"num_write":874,"num_write_kb":24549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":351,"num_bytes_recovered":38747648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"211'608","reported_seq":20163,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.911705+0000","last_change":"2024-09-16T20:51:45.643842+0000","last_active":"2024-09-16T20:52:02.911705+0000","last_peered":"2024-09-16T20:52:02.911705+0000","last_clean":"2024-09-16T20:52:02.911705+0000","last_became_active":"2024-09-16T20:51:03.589742+0000","last_became_peered":"2024-09-16T20:51:03.589742+0000","last_unstale":"2024-09-16T20:52:02.911705+0000","last_undegraded":"2024-09-16T20:52:02.911705+0000","last_fullsized":"2024-09-16T20:52:02.911705+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":608,"log_dups_size":0,"ondisk_log_size":608,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:49:22.579735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024335089999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":16969,"num_read_kb":76220,"num_write":613,"num_write_kb":13841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":486,"num_bytes_recovered":15329756,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"216'461","reported_seq":12004,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:02.741986+0000","last_change":"2024-09-16T20:51:45.586711+0000","last_active":"2024-09-16T20:52:02.741986+0000","last_peered":"2024-09-16T20:52:02.741986+0000","last_clean":"2024-09-16T20:52:02.741986+0000","last_became_active":"2024-09-16T20:51:03.595360+0000","last_became_peered":"2024-09-16T20:51:03.595360+0000","last_unstale":"2024-09-16T20:52:02.741986+0000","last_undegraded":"2024-09-16T20:52:02.741986+0000","last_fullsized":"2024-09-16T20:52:02.741986+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:48:06.547314+0000","last_clean_scrub_stamp":"2024-09-16T20:48:06.547314+0000","objects_scrubbed":0,"log_size":461,"log_dups_size":0,"ondisk_log_size":461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:47:40.600095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035295309999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":11262,"num_read_kb":35639,"num_write":562,"num_write_kb":13645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":198,"num_bytes_recovered":14217900,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":247,"num_object_clones":0,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":243,"num_read":448597,"num_read_kb":1798155,"num_write":23462,"num_write_kb":469671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12645,"num_bytes_recovered":606441956,"num_keys_recovered":1497,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22149,"ondisk_log_size":22149,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574904,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52588,"kb_used_data":3344,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319252,"statfs":{"total":96636764160,"available":96582914048,"internally_reserved":0,"allocated":3424256,"data_stored":1774023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574904,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74140,"kb_used_data":2760,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94297700,"statfs":{"total":96636764160,"available":96560844800,"internally_reserved":0,"allocated":2826240,"data_stored":1183636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73071047},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574904,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69980,"kb_used_data":2760,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94301860,"statfs":{"total":96636764160,"available":96565104640,"internally_reserved":0,"allocated":2826240,"data_stored":1183636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574903,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70568,"kb_used_data":3344,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94301272,"statfs":{"total":96636764160,"available":96564502528,"internally_reserved":0,"allocated":3424256,"data_stored":1774023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574904,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":96420,"kb_used_data":2764,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":94275420,"statfs":{"total":96636764160,"available":96538030080,"internally_reserved":0,"allocated":2830336,"data_stored":1183655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":95877574},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640312,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60320,"kb_used_data":2764,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311520,"statfs":{"total":96636764160,"available":96574996480,"internally_reserved":0,"allocated":2830336,"data_stored":1183655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T20:52:11.885 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-16T20:48:06.547314+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=48, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=52, tm_sec=7, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T20:52:11.885 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-16T20:48:06.547314+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=48, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=52, tm_sec=7, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T20:52:11.885 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-16T20:48:06.547314+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=48, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=52, tm_sec=7, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T20:52:11.885 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-16T20:48:06.547314+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=48, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=52, tm_sec=7, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T20:52:11.885 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-16T20:48:06.547314+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=48, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=52, tm_sec=7, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T20:52:11.885 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-16T20:48:06.547314+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=48, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=52, tm_sec=7, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T20:52:11.885 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-16T20:48:06.547314+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=48, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=52, tm_sec=7, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T20:52:11.885 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-16T20:48:06.547314+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=48, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=52, tm_sec=7, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T20:52:11.886 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-16T20:48:06.547314+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=48, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=52, tm_sec=7, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T20:52:11.886 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-16T20:48:06.547314+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=48, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=52, tm_sec=7, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T20:52:11.886 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-16T20:48:06.547314+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=48, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=52, tm_sec=7, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T20:52:11.886 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-16T20:48:06.547314+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=48, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=52, tm_sec=7, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T20:52:11.886 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-16T20:47:57.257926+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=47, tm_sec=57, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=52, tm_sec=7, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T20:52:11.886 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-09-16T20:48:06.547314+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=48, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=52, tm_sec=7, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T20:52:11.886 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-16T20:47:57.257926+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=47, tm_sec=57, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=52, tm_sec=7, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T20:52:11.886 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-16T20:48:06.547314+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=48, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=52, tm_sec=7, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T20:52:11.886 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-16T20:47:57.257926+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=47, tm_sec=57, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=52, tm_sec=7, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T20:52:11.886 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-16T20:48:06.547314+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=48, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=52, tm_sec=7, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T20:52:11.887 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-16T20:47:57.257926+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=47, tm_sec=57, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=52, tm_sec=7, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T20:52:11.887 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-16T20:48:06.547314+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=48, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=52, tm_sec=7, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T20:52:11.887 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-16T20:48:06.547314+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=48, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=52, tm_sec=7, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T20:52:11.887 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-16T20:47:57.257926+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=47, tm_sec=57, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=52, tm_sec=7, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T20:52:11.887 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-16T20:48:06.547314+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=48, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=52, tm_sec=7, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T20:52:11.887 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-16T20:47:57.257926+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=47, tm_sec=57, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=52, tm_sec=7, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T20:52:11.887 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-16T20:48:06.547314+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=48, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=52, tm_sec=7, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T20:52:11.887 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-16T20:47:57.257926+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=47, tm_sec=57, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=52, tm_sec=7, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T20:52:11.887 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-16T20:48:06.547314+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=48, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=52, tm_sec=7, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T20:52:11.888 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-16T20:47:57.257926+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=47, tm_sec=57, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=52, tm_sec=7, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T20:52:11.888 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-16T20:47:52.241408+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=47, tm_sec=52, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=52, tm_sec=7, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T20:52:11.888 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-16T20:48:06.547314+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=48, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=52, tm_sec=7, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T20:52:11.888 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-16T20:48:06.547314+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=48, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=52, tm_sec=7, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T20:52:11.888 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-16T20:48:06.547314+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=48, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=52, tm_sec=7, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T20:52:11.888 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-16T20:48:06.547314+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=48, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=52, tm_sec=7, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T20:52:11.888 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-16T20:48:06.547314+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=48, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=52, tm_sec=7, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T20:52:11.888 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-16T20:48:06.547314+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=48, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=52, tm_sec=7, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T20:52:11.888 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-16T20:48:06.547314+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=48, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=52, tm_sec=7, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T20:52:11.889 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-16T20:48:06.547314+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=48, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=52, tm_sec=7, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T20:52:11.889 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-16T20:48:06.547314+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=48, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=52, tm_sec=7, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T20:52:11.889 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-16T20:48:06.547314+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=48, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=52, tm_sec=7, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T20:52:11.889 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-16T20:48:06.547314+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=48, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=52, tm_sec=7, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T20:52:11.889 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-16T20:48:06.547314+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=48, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=20, tm_min=52, tm_sec=7, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T20:52:11.889 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-16T20:52:31.890 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T20:52:32.200 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-16T20:52:32.201 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-09-16T20:52:32.212 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":371,"stamp":"2024-09-16T20:52:30.255106+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":251,"num_object_clones":0,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":243,"num_read":448663,"num_read_kb":1798210,"num_write":23565,"num_write_kb":471301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12652,"num_bytes_recovered":607622711,"num_keys_recovered":1497,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22473,"ondisk_log_size":22473,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":423976,"kb_used_data":17736,"kb_used_omap":9,"kb_used_meta":406134,"kb_avail":565807064,"statfs":{"total":579820584960,"available":579386433536,"internally_reserved":0,"allocated":18161664,"data_stored":8282628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9567,"internal_metadata":415881889},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003005"},"pg_stats":[{"pgid":"3.1f","version":"235'368","reported_seq":10839,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:16.229705+0000","last_change":"2024-09-16T20:52:16.229705+0000","last_active":"2024-09-16T20:52:16.229705+0000","last_peered":"2024-09-16T20:52:16.229705+0000","last_clean":"2024-09-16T20:52:16.229705+0000","last_became_active":"2024-09-16T20:49:12.801566+0000","last_became_peered":"2024-09-16T20:49:12.801566+0000","last_unstale":"2024-09-16T20:52:16.229705+0000","last_undegraded":"2024-09-16T20:52:16.229705+0000","last_fullsized":"2024-09-16T20:52:16.229705+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"235'368","last_scrub_stamp":"2024-09-16T20:52:16.229607+0000","last_deep_scrub":"235'368","last_deep_scrub_stamp":"2024-09-16T20:52:16.229607+0000","last_clean_scrub_stamp":"2024-09-16T20:52:16.229607+0000","objects_scrubbed":9,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:32:16.475709+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.0016075530000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10091,"num_read_kb":31685,"num_write":315,"num_write_kb":5043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":2125998,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"235'909","reported_seq":28784,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:15.876469+0000","last_change":"2024-09-16T20:52:15.876469+0000","last_active":"2024-09-16T20:52:15.876469+0000","last_peered":"2024-09-16T20:52:15.876469+0000","last_clean":"2024-09-16T20:52:15.876469+0000","last_became_active":"2024-09-16T20:51:27.628085+0000","last_became_peered":"2024-09-16T20:51:27.628085+0000","last_unstale":"2024-09-16T20:52:15.876469+0000","last_undegraded":"2024-09-16T20:52:15.876469+0000","last_fullsized":"2024-09-16T20:52:15.876469+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"235'909","last_scrub_stamp":"2024-09-16T20:52:15.876374+0000","last_deep_scrub":"235'909","last_deep_scrub_stamp":"2024-09-16T20:52:15.876374+0000","last_clean_scrub_stamp":"2024-09-16T20:52:15.876374+0000","objects_scrubbed":4,"log_size":909,"log_dups_size":0,"ondisk_log_size":909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:13:24.639907+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.001444383,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":29337,"num_read_kb":83641,"num_write":1040,"num_write_kb":10328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":562,"num_bytes_recovered":11317531,"num_keys_recovered":154,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"235'716","reported_seq":12959,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:14.342586+0000","last_change":"2024-09-16T20:52:14.342586+0000","last_active":"2024-09-16T20:52:14.342586+0000","last_peered":"2024-09-16T20:52:14.342586+0000","last_clean":"2024-09-16T20:52:14.342586+0000","last_became_active":"2024-09-16T20:51:26.624314+0000","last_became_peered":"2024-09-16T20:51:26.624314+0000","last_unstale":"2024-09-16T20:52:14.342586+0000","last_undegraded":"2024-09-16T20:52:14.342586+0000","last_fullsized":"2024-09-16T20:52:14.342586+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"235'716","last_scrub_stamp":"2024-09-16T20:52:14.342487+0000","last_deep_scrub":"235'716","last_deep_scrub_stamp":"2024-09-16T20:52:14.342487+0000","last_clean_scrub_stamp":"2024-09-16T20:52:14.342487+0000","objects_scrubbed":9,"log_size":716,"log_dups_size":0,"ondisk_log_size":716,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:27:06.359453+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.0014726419999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":13066,"num_read_kb":103238,"num_write":750,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":387,"num_bytes_recovered":35619030,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"235'725","reported_seq":4391,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:19.602492+0000","last_change":"2024-09-16T20:52:19.602492+0000","last_active":"2024-09-16T20:52:19.602492+0000","last_peered":"2024-09-16T20:52:19.602492+0000","last_clean":"2024-09-16T20:52:19.602492+0000","last_became_active":"2024-09-16T20:51:04.577608+0000","last_became_peered":"2024-09-16T20:51:04.577608+0000","last_unstale":"2024-09-16T20:52:19.602492+0000","last_undegraded":"2024-09-16T20:52:19.602492+0000","last_fullsized":"2024-09-16T20:52:19.602492+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"235'725","last_scrub_stamp":"2024-09-16T20:52:19.602380+0000","last_deep_scrub":"235'725","last_deep_scrub_stamp":"2024-09-16T20:52:19.602380+0000","last_clean_scrub_stamp":"2024-09-16T20:52:19.602380+0000","objects_scrubbed":11,"log_size":725,"log_dups_size":0,"ondisk_log_size":725,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:13:39.547810+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0.0034574200000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":4615,"num_read_kb":24234,"num_write":996,"num_write_kb":15567,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":25643873,"num_keys_recovered":142,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"235'883","reported_seq":11179,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:14.850035+0000","last_change":"2024-09-16T20:52:14.850035+0000","last_active":"2024-09-16T20:52:14.850035+0000","last_peered":"2024-09-16T20:52:14.850035+0000","last_clean":"2024-09-16T20:52:14.850035+0000","last_became_active":"2024-09-16T20:51:49.579980+0000","last_became_peered":"2024-09-16T20:51:49.579980+0000","last_unstale":"2024-09-16T20:52:14.850035+0000","last_undegraded":"2024-09-16T20:52:14.850035+0000","last_fullsized":"2024-09-16T20:52:14.850035+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"235'883","last_scrub_stamp":"2024-09-16T20:52:14.849920+0000","last_deep_scrub":"235'883","last_deep_scrub_stamp":"2024-09-16T20:52:14.849920+0000","last_clean_scrub_stamp":"2024-09-16T20:52:14.849920+0000","objects_scrubbed":11,"log_size":883,"log_dups_size":0,"ondisk_log_size":883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:40:03.375170+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0.001184378,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":10970,"num_read_kb":44437,"num_write":1151,"num_write_kb":16781,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":956,"num_bytes_recovered":37659232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"235'1230","reported_seq":19879,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:18.585082+0000","last_change":"2024-09-16T20:52:18.585082+0000","last_active":"2024-09-16T20:52:18.585082+0000","last_peered":"2024-09-16T20:52:18.585082+0000","last_clean":"2024-09-16T20:52:18.585082+0000","last_became_active":"2024-09-16T20:49:12.802869+0000","last_became_peered":"2024-09-16T20:49:12.802869+0000","last_unstale":"2024-09-16T20:52:18.585082+0000","last_undegraded":"2024-09-16T20:52:18.585082+0000","last_fullsized":"2024-09-16T20:52:18.585082+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"235'1230","last_scrub_stamp":"2024-09-16T20:52:18.584988+0000","last_deep_scrub":"235'1230","last_deep_scrub_stamp":"2024-09-16T20:52:18.584988+0000","last_clean_scrub_stamp":"2024-09-16T20:52:18.584988+0000","objects_scrubbed":9,"log_size":1230,"log_dups_size":0,"ondisk_log_size":1230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:40:50.663439+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.0026611790000000001,"stat_sum":{"num_bytes":19,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":25740,"num_read_kb":67599,"num_write":1269,"num_write_kb":16392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":1051004,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"235'781","reported_seq":13008,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:17.363933+0000","last_change":"2024-09-16T20:52:17.363933+0000","last_active":"2024-09-16T20:52:17.363933+0000","last_peered":"2024-09-16T20:52:17.363933+0000","last_clean":"2024-09-16T20:52:17.363933+0000","last_became_active":"2024-09-16T20:51:16.577039+0000","last_became_peered":"2024-09-16T20:51:16.577039+0000","last_unstale":"2024-09-16T20:52:17.363933+0000","last_undegraded":"2024-09-16T20:52:17.363933+0000","last_fullsized":"2024-09-16T20:52:17.363933+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"235'781","last_scrub_stamp":"2024-09-16T20:52:17.363819+0000","last_deep_scrub":"235'781","last_deep_scrub_stamp":"2024-09-16T20:52:17.363819+0000","last_clean_scrub_stamp":"2024-09-16T20:52:17.363819+0000","objects_scrubbed":11,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:23:49.542095+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0.0023604640000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":10011,"num_read_kb":47438,"num_write":753,"num_write_kb":12180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":12922788,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"235'462","reported_seq":13774,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:12.832888+0000","last_change":"2024-09-16T20:52:12.832888+0000","last_active":"2024-09-16T20:52:12.832888+0000","last_peered":"2024-09-16T20:52:12.832888+0000","last_clean":"2024-09-16T20:52:12.832888+0000","last_became_active":"2024-09-16T20:51:15.575429+0000","last_became_peered":"2024-09-16T20:51:15.575429+0000","last_unstale":"2024-09-16T20:52:12.832888+0000","last_undegraded":"2024-09-16T20:52:12.832888+0000","last_fullsized":"2024-09-16T20:52:12.832888+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"235'462","last_scrub_stamp":"2024-09-16T20:52:12.832784+0000","last_deep_scrub":"235'462","last_deep_scrub_stamp":"2024-09-16T20:52:12.832784+0000","last_clean_scrub_stamp":"2024-09-16T20:52:12.832784+0000","objects_scrubbed":4,"log_size":462,"log_dups_size":0,"ondisk_log_size":462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:50:29.991765+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.001906629,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":14274,"num_read_kb":52840,"num_write":349,"num_write_kb":9355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":231,"num_bytes_recovered":11122736,"num_keys_recovered":66,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"235'990","reported_seq":8075,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:15.187910+0000","last_change":"2024-09-16T20:52:15.187910+0000","last_active":"2024-09-16T20:52:15.187910+0000","last_peered":"2024-09-16T20:52:15.187910+0000","last_clean":"2024-09-16T20:52:15.187910+0000","last_became_active":"2024-09-16T20:51:03.592284+0000","last_became_peered":"2024-09-16T20:51:03.592284+0000","last_unstale":"2024-09-16T20:52:15.187910+0000","last_undegraded":"2024-09-16T20:52:15.187910+0000","last_fullsized":"2024-09-16T20:52:15.187910+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"235'990","last_scrub_stamp":"2024-09-16T20:52:15.187794+0000","last_deep_scrub":"235'990","last_deep_scrub_stamp":"2024-09-16T20:52:15.187794+0000","last_clean_scrub_stamp":"2024-09-16T20:52:15.187794+0000","objects_scrubbed":9,"log_size":990,"log_dups_size":0,"ondisk_log_size":990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:29:44.220363+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.0015146339999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":11609,"num_read_kb":15811,"num_write":975,"num_write_kb":5184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":280,"num_bytes_recovered":1580920,"num_keys_recovered":144,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"235'1286","reported_seq":11845,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:15.293889+0000","last_change":"2024-09-16T20:52:15.293889+0000","last_active":"2024-09-16T20:52:15.293889+0000","last_peered":"2024-09-16T20:52:15.293889+0000","last_clean":"2024-09-16T20:52:15.293889+0000","last_became_active":"2024-09-16T20:51:27.624341+0000","last_became_peered":"2024-09-16T20:51:27.624341+0000","last_unstale":"2024-09-16T20:52:15.293889+0000","last_undegraded":"2024-09-16T20:52:15.293889+0000","last_fullsized":"2024-09-16T20:52:15.293889+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"235'1286","last_scrub_stamp":"2024-09-16T20:52:15.293784+0000","last_deep_scrub":"235'1286","last_deep_scrub_stamp":"2024-09-16T20:52:15.293784+0000","last_clean_scrub_stamp":"2024-09-16T20:52:15.293784+0000","objects_scrubbed":9,"log_size":1286,"log_dups_size":0,"ondisk_log_size":1286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:56:03.162663+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.0011831579999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":17170,"num_read_kb":43312,"num_write":1584,"num_write_kb":10211,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":542,"num_bytes_recovered":21909352,"num_keys_recovered":70,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"235'603","reported_seq":14017,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:16.198659+0000","last_change":"2024-09-16T20:52:16.198659+0000","last_active":"2024-09-16T20:52:16.198659+0000","last_peered":"2024-09-16T20:52:16.198659+0000","last_clean":"2024-09-16T20:52:16.198659+0000","last_became_active":"2024-09-16T20:51:16.577620+0000","last_became_peered":"2024-09-16T20:51:16.577620+0000","last_unstale":"2024-09-16T20:52:16.198659+0000","last_undegraded":"2024-09-16T20:52:16.198659+0000","last_fullsized":"2024-09-16T20:52:16.198659+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"235'603","last_scrub_stamp":"2024-09-16T20:52:16.198565+0000","last_deep_scrub":"235'603","last_deep_scrub_stamp":"2024-09-16T20:52:16.198565+0000","last_clean_scrub_stamp":"2024-09-16T20:52:16.198565+0000","objects_scrubbed":8,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:00:15.711330+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.0022662979999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":12843,"num_read_kb":77657,"num_write":690,"num_write_kb":15118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":562,"num_bytes_recovered":17171700,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"235'374","reported_seq":19995,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:17.552071+0000","last_change":"2024-09-16T20:52:17.552071+0000","last_active":"2024-09-16T20:52:17.552071+0000","last_peered":"2024-09-16T20:52:17.552071+0000","last_clean":"2024-09-16T20:52:17.552071+0000","last_became_active":"2024-09-16T20:51:57.718930+0000","last_became_peered":"2024-09-16T20:51:57.718930+0000","last_unstale":"2024-09-16T20:52:17.552071+0000","last_undegraded":"2024-09-16T20:52:17.552071+0000","last_fullsized":"2024-09-16T20:52:17.552071+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"235'374","last_scrub_stamp":"2024-09-16T20:52:17.551959+0000","last_deep_scrub":"235'374","last_deep_scrub_stamp":"2024-09-16T20:52:17.551959+0000","last_clean_scrub_stamp":"2024-09-16T20:52:17.551959+0000","objects_scrubbed":4,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:29:25.686886+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.0014936070000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":18755,"num_read_kb":45565,"num_write":465,"num_write_kb":13208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":540,"num_bytes_recovered":30162312,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":469,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:13.435878+0000","last_change":"2024-09-16T20:52:13.435815+0000","last_active":"2024-09-16T20:52:13.435878+0000","last_peered":"2024-09-16T20:52:13.435878+0000","last_clean":"2024-09-16T20:52:13.435878+0000","last_became_active":"2024-09-16T20:51:00.578081+0000","last_became_peered":"2024-09-16T20:51:00.578081+0000","last_unstale":"2024-09-16T20:52:13.435878+0000","last_undegraded":"2024-09-16T20:52:13.435878+0000","last_fullsized":"2024-09-16T20:52:13.435878+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:52:13.435728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:52:13.435728+0000","last_clean_scrub_stamp":"2024-09-16T20:52:13.435728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:21:11.065314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091339500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"235'1033","reported_seq":19441,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:14.255362+0000","last_change":"2024-09-16T20:52:14.255288+0000","last_active":"2024-09-16T20:52:14.255362+0000","last_peered":"2024-09-16T20:52:14.255362+0000","last_clean":"2024-09-16T20:52:14.255362+0000","last_became_active":"2024-09-16T20:51:16.577104+0000","last_became_peered":"2024-09-16T20:51:16.577104+0000","last_unstale":"2024-09-16T20:52:14.255362+0000","last_undegraded":"2024-09-16T20:52:14.255362+0000","last_fullsized":"2024-09-16T20:52:14.255362+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"235'1033","last_scrub_stamp":"2024-09-16T20:52:14.255186+0000","last_deep_scrub":"235'1033","last_deep_scrub_stamp":"2024-09-16T20:52:14.255186+0000","last_clean_scrub_stamp":"2024-09-16T20:52:14.255186+0000","objects_scrubbed":5,"log_size":1033,"log_dups_size":0,"ondisk_log_size":1033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:18:51.132909+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.0013907780000000001,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":20870,"num_read_kb":85867,"num_write":1218,"num_write_kb":25908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":29601482,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":472,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:13.242530+0000","last_change":"2024-09-16T20:52:13.242465+0000","last_active":"2024-09-16T20:52:13.242530+0000","last_peered":"2024-09-16T20:52:13.242530+0000","last_clean":"2024-09-16T20:52:13.242530+0000","last_became_active":"2024-09-16T20:51:00.577653+0000","last_became_peered":"2024-09-16T20:51:00.577653+0000","last_unstale":"2024-09-16T20:52:13.242530+0000","last_undegraded":"2024-09-16T20:52:13.242530+0000","last_fullsized":"2024-09-16T20:52:13.242530+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:52:13.242381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:52:13.242381+0000","last_clean_scrub_stamp":"2024-09-16T20:52:13.242381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:13:34.645798+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00043659299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"235'657","reported_seq":25421,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:15.296310+0000","last_change":"2024-09-16T20:52:15.296248+0000","last_active":"2024-09-16T20:52:15.296310+0000","last_peered":"2024-09-16T20:52:15.296310+0000","last_clean":"2024-09-16T20:52:15.296310+0000","last_became_active":"2024-09-16T20:51:15.575607+0000","last_became_peered":"2024-09-16T20:51:15.575607+0000","last_unstale":"2024-09-16T20:52:15.296310+0000","last_undegraded":"2024-09-16T20:52:15.296310+0000","last_fullsized":"2024-09-16T20:52:15.296310+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"235'657","last_scrub_stamp":"2024-09-16T20:52:15.296153+0000","last_deep_scrub":"235'657","last_deep_scrub_stamp":"2024-09-16T20:52:15.296153+0000","last_clean_scrub_stamp":"2024-09-16T20:52:15.296153+0000","objects_scrubbed":4,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:31:06.095451+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.0022924479999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":24178,"num_read_kb":81081,"num_write":502,"num_write_kb":17573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":20407955,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":457,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:12.478731+0000","last_change":"2024-09-16T20:52:12.478731+0000","last_active":"2024-09-16T20:52:12.478731+0000","last_peered":"2024-09-16T20:52:12.478731+0000","last_clean":"2024-09-16T20:52:12.478731+0000","last_became_active":"2024-09-16T20:47:58.272861+0000","last_became_peered":"2024-09-16T20:47:58.272861+0000","last_unstale":"2024-09-16T20:52:12.478731+0000","last_undegraded":"2024-09-16T20:52:12.478731+0000","last_fullsized":"2024-09-16T20:52:12.478731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:52:12.478660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:52:12.478660+0000","last_clean_scrub_stamp":"2024-09-16T20:52:12.478660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:18:36.674695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012429489999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"235'494","reported_seq":3896,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:11.201620+0000","last_change":"2024-09-16T20:52:11.201557+0000","last_active":"2024-09-16T20:52:11.201620+0000","last_peered":"2024-09-16T20:52:11.201620+0000","last_clean":"2024-09-16T20:52:11.201620+0000","last_became_active":"2024-09-16T20:49:12.801810+0000","last_became_peered":"2024-09-16T20:49:12.801810+0000","last_unstale":"2024-09-16T20:52:11.201620+0000","last_undegraded":"2024-09-16T20:52:11.201620+0000","last_fullsized":"2024-09-16T20:52:11.201620+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"235'494","last_scrub_stamp":"2024-09-16T20:52:11.201459+0000","last_deep_scrub":"235'494","last_deep_scrub_stamp":"2024-09-16T20:52:11.201459+0000","last_clean_scrub_stamp":"2024-09-16T20:52:11.201459+0000","objects_scrubbed":3,"log_size":494,"log_dups_size":0,"ondisk_log_size":494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:27:56.000279+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.0015370990000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":3741,"num_read_kb":11405,"num_write":512,"num_write_kb":6561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":2071114,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":348,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:12.217225+0000","last_change":"2024-09-16T20:52:12.217225+0000","last_active":"2024-09-16T20:52:12.217225+0000","last_peered":"2024-09-16T20:52:12.217225+0000","last_clean":"2024-09-16T20:52:12.217225+0000","last_became_active":"2024-09-16T20:51:12.615412+0000","last_became_peered":"2024-09-16T20:51:12.615412+0000","last_unstale":"2024-09-16T20:52:12.217225+0000","last_undegraded":"2024-09-16T20:52:12.217225+0000","last_fullsized":"2024-09-16T20:52:12.217225+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:52:12.217140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:52:12.217140+0000","last_clean_scrub_stamp":"2024-09-16T20:52:12.217140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:27:27.046612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"235'674","reported_seq":19729,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:11.269177+0000","last_change":"2024-09-16T20:52:11.269177+0000","last_active":"2024-09-16T20:52:11.269177+0000","last_peered":"2024-09-16T20:52:11.269177+0000","last_clean":"2024-09-16T20:52:11.269177+0000","last_became_active":"2024-09-16T20:51:27.624519+0000","last_became_peered":"2024-09-16T20:51:27.624519+0000","last_unstale":"2024-09-16T20:52:11.269177+0000","last_undegraded":"2024-09-16T20:52:11.269177+0000","last_fullsized":"2024-09-16T20:52:11.269177+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"235'674","last_scrub_stamp":"2024-09-16T20:52:11.269081+0000","last_deep_scrub":"235'674","last_deep_scrub_stamp":"2024-09-16T20:52:11.269081+0000","last_clean_scrub_stamp":"2024-09-16T20:52:11.269081+0000","objects_scrubbed":6,"log_size":674,"log_dups_size":0,"ondisk_log_size":674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:04:03.621553+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.001777441,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":18081,"num_read_kb":57429,"num_write":654,"num_write_kb":12584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":564,"num_bytes_recovered":27092048,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"235'739","reported_seq":8658,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:11.240133+0000","last_change":"2024-09-16T20:52:11.240133+0000","last_active":"2024-09-16T20:52:11.240133+0000","last_peered":"2024-09-16T20:52:11.240133+0000","last_clean":"2024-09-16T20:52:11.240133+0000","last_became_active":"2024-09-16T20:51:27.626032+0000","last_became_peered":"2024-09-16T20:51:27.626032+0000","last_unstale":"2024-09-16T20:52:11.240133+0000","last_undegraded":"2024-09-16T20:52:11.240133+0000","last_fullsized":"2024-09-16T20:52:11.240133+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"235'739","last_scrub_stamp":"2024-09-16T20:52:11.240051+0000","last_deep_scrub":"235'739","last_deep_scrub_stamp":"2024-09-16T20:52:11.240051+0000","last_clean_scrub_stamp":"2024-09-16T20:52:11.240051+0000","objects_scrubbed":12,"log_size":739,"log_dups_size":0,"ondisk_log_size":739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:21:54.980119+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.0025014909999999998,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":7673,"num_read_kb":36418,"num_write":666,"num_write_kb":13145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":523,"num_bytes_recovered":11083665,"num_keys_recovered":14,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"235'3","reported_seq":146,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:10.214144+0000","last_change":"2024-09-16T20:52:10.214144+0000","last_active":"2024-09-16T20:52:10.214144+0000","last_peered":"2024-09-16T20:52:10.214144+0000","last_clean":"2024-09-16T20:52:10.214144+0000","last_became_active":"2024-09-16T20:51:00.574693+0000","last_became_peered":"2024-09-16T20:51:00.574693+0000","last_unstale":"2024-09-16T20:52:10.214144+0000","last_undegraded":"2024-09-16T20:52:10.214144+0000","last_fullsized":"2024-09-16T20:52:10.214144+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"235'3","last_scrub_stamp":"2024-09-16T20:52:10.214058+0000","last_deep_scrub":"235'3","last_deep_scrub_stamp":"2024-09-16T20:52:10.214058+0000","last_clean_scrub_stamp":"2024-09-16T20:52:10.214058+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:39:46.647527+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00062077499999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"235'506","reported_seq":18824,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:13.218092+0000","last_change":"2024-09-16T20:52:13.218092+0000","last_active":"2024-09-16T20:52:13.218092+0000","last_peered":"2024-09-16T20:52:13.218092+0000","last_clean":"2024-09-16T20:52:13.218092+0000","last_became_active":"2024-09-16T20:51:26.626185+0000","last_became_peered":"2024-09-16T20:51:26.626185+0000","last_unstale":"2024-09-16T20:52:13.218092+0000","last_undegraded":"2024-09-16T20:52:13.218092+0000","last_fullsized":"2024-09-16T20:52:13.218092+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"235'506","last_scrub_stamp":"2024-09-16T20:52:13.217991+0000","last_deep_scrub":"235'506","last_deep_scrub_stamp":"2024-09-16T20:52:13.217991+0000","last_clean_scrub_stamp":"2024-09-16T20:52:13.217991+0000","objects_scrubbed":8,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:19:22.766472+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.0023862639999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":19429,"num_read_kb":77391,"num_write":450,"num_write_kb":15765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":3862081,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":117,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:11.822619+0000","last_change":"2024-09-16T20:52:11.822619+0000","last_active":"2024-09-16T20:52:11.822619+0000","last_peered":"2024-09-16T20:52:11.822619+0000","last_clean":"2024-09-16T20:52:11.822619+0000","last_became_active":"2024-09-16T20:51:12.614812+0000","last_became_peered":"2024-09-16T20:51:12.614812+0000","last_unstale":"2024-09-16T20:52:11.822619+0000","last_undegraded":"2024-09-16T20:52:11.822619+0000","last_fullsized":"2024-09-16T20:52:11.822619+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:52:11.822518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:52:11.822518+0000","last_clean_scrub_stamp":"2024-09-16T20:52:11.822518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:44:27.546819+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"235'320","reported_seq":5936,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:12.242678+0000","last_change":"2024-09-16T20:52:12.242620+0000","last_active":"2024-09-16T20:52:12.242678+0000","last_peered":"2024-09-16T20:52:12.242678+0000","last_clean":"2024-09-16T20:52:12.242678+0000","last_became_active":"2024-09-16T20:51:26.624469+0000","last_became_peered":"2024-09-16T20:51:26.624469+0000","last_unstale":"2024-09-16T20:52:12.242678+0000","last_undegraded":"2024-09-16T20:52:12.242678+0000","last_fullsized":"2024-09-16T20:52:12.242678+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"235'320","last_scrub_stamp":"2024-09-16T20:52:12.242517+0000","last_deep_scrub":"235'320","last_deep_scrub_stamp":"2024-09-16T20:52:12.242517+0000","last_clean_scrub_stamp":"2024-09-16T20:52:12.242517+0000","objects_scrubbed":8,"log_size":320,"log_dups_size":0,"ondisk_log_size":320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:54:14.023982+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.0025837220000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4323,"num_read_kb":33093,"num_write":273,"num_write_kb":12351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":324,"num_bytes_recovered":16931328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":97,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:10.220703+0000","last_change":"2024-09-16T20:52:10.220703+0000","last_active":"2024-09-16T20:52:10.220703+0000","last_peered":"2024-09-16T20:52:10.220703+0000","last_clean":"2024-09-16T20:52:10.220703+0000","last_became_active":"2024-09-16T20:51:23.608159+0000","last_became_peered":"2024-09-16T20:51:23.608159+0000","last_unstale":"2024-09-16T20:52:10.220703+0000","last_undegraded":"2024-09-16T20:52:10.220703+0000","last_fullsized":"2024-09-16T20:52:10.220703+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T20:52:10.220629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T20:52:10.220629+0000","last_clean_scrub_stamp":"2024-09-16T20:52:10.220629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:23:02.510784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037636000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"235'614","reported_seq":7628,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:10.830002+0000","last_change":"2024-09-16T20:52:10.830002+0000","last_active":"2024-09-16T20:52:10.830002+0000","last_peered":"2024-09-16T20:52:10.830002+0000","last_clean":"2024-09-16T20:52:10.830002+0000","last_became_active":"2024-09-16T20:51:26.622629+0000","last_became_peered":"2024-09-16T20:51:26.622629+0000","last_unstale":"2024-09-16T20:52:10.830002+0000","last_undegraded":"2024-09-16T20:52:10.830002+0000","last_fullsized":"2024-09-16T20:52:10.830002+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"235'614","last_scrub_stamp":"2024-09-16T20:52:10.829911+0000","last_deep_scrub":"235'614","last_deep_scrub_stamp":"2024-09-16T20:52:10.829911+0000","last_clean_scrub_stamp":"2024-09-16T20:52:10.829911+0000","objects_scrubbed":5,"log_size":614,"log_dups_size":0,"ondisk_log_size":614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:06:30.697721+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.0029005419999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":9084,"num_read_kb":37091,"num_write":448,"num_write_kb":13187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":848,"num_bytes_recovered":24298320,"num_keys_recovered":219,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"235'2","reported_seq":479,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:11.463339+0000","last_change":"2024-09-16T20:52:11.463339+0000","last_active":"2024-09-16T20:52:11.463339+0000","last_peered":"2024-09-16T20:52:11.463339+0000","last_clean":"2024-09-16T20:52:11.463339+0000","last_became_active":"2024-09-16T20:51:23.605321+0000","last_became_peered":"2024-09-16T20:51:23.605321+0000","last_unstale":"2024-09-16T20:52:11.463339+0000","last_undegraded":"2024-09-16T20:52:11.463339+0000","last_fullsized":"2024-09-16T20:52:11.463339+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"235'2","last_scrub_stamp":"2024-09-16T20:52:11.463274+0000","last_deep_scrub":"235'2","last_deep_scrub_stamp":"2024-09-16T20:52:11.463274+0000","last_clean_scrub_stamp":"2024-09-16T20:52:11.463274+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:13:18.406431+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00095771799999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"235'74","reported_seq":574,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:10.468094+0000","last_change":"2024-09-16T20:52:10.468094+0000","last_active":"2024-09-16T20:52:10.468094+0000","last_peered":"2024-09-16T20:52:10.468094+0000","last_clean":"2024-09-16T20:52:10.468094+0000","last_became_active":"2024-09-16T20:49:09.559571+0000","last_became_peered":"2024-09-16T20:49:09.559571+0000","last_unstale":"2024-09-16T20:52:10.468094+0000","last_undegraded":"2024-09-16T20:52:10.468094+0000","last_fullsized":"2024-09-16T20:52:10.468094+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"235'74","last_scrub_stamp":"2024-09-16T20:52:10.468015+0000","last_deep_scrub":"235'74","last_deep_scrub_stamp":"2024-09-16T20:52:10.468015+0000","last_clean_scrub_stamp":"2024-09-16T20:52:10.468015+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:03:59.884743+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"235'647","reported_seq":7079,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:16.336281+0000","last_change":"2024-09-16T20:52:16.336184+0000","last_active":"2024-09-16T20:52:16.336281+0000","last_peered":"2024-09-16T20:52:16.336281+0000","last_clean":"2024-09-16T20:52:16.336281+0000","last_became_active":"2024-09-16T20:51:27.624220+0000","last_became_peered":"2024-09-16T20:51:27.624220+0000","last_unstale":"2024-09-16T20:52:16.336281+0000","last_undegraded":"2024-09-16T20:52:16.336281+0000","last_fullsized":"2024-09-16T20:52:16.336281+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"235'647","last_scrub_stamp":"2024-09-16T20:52:16.336078+0000","last_deep_scrub":"235'647","last_deep_scrub_stamp":"2024-09-16T20:52:16.336078+0000","last_clean_scrub_stamp":"2024-09-16T20:52:16.336078+0000","objects_scrubbed":6,"log_size":647,"log_dups_size":0,"ondisk_log_size":647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:04:41.333025+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.0022745209999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6921,"num_read_kb":32984,"num_write":827,"num_write_kb":7103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":4839295,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"235'646","reported_seq":16485,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:12.196030+0000","last_change":"2024-09-16T20:52:12.196030+0000","last_active":"2024-09-16T20:52:12.196030+0000","last_peered":"2024-09-16T20:52:12.196030+0000","last_clean":"2024-09-16T20:52:12.196030+0000","last_became_active":"2024-09-16T20:51:17.575136+0000","last_became_peered":"2024-09-16T20:51:17.575136+0000","last_unstale":"2024-09-16T20:52:12.196030+0000","last_undegraded":"2024-09-16T20:52:12.196030+0000","last_fullsized":"2024-09-16T20:52:12.196030+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"235'646","last_scrub_stamp":"2024-09-16T20:52:12.195955+0000","last_deep_scrub":"235'646","last_deep_scrub_stamp":"2024-09-16T20:52:12.195955+0000","last_clean_scrub_stamp":"2024-09-16T20:52:12.195955+0000","objects_scrubbed":7,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:06:19.106949+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.0022726539999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":15159,"num_read_kb":101301,"num_write":561,"num_write_kb":29560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":790,"num_bytes_recovered":48497898,"num_keys_recovered":60,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"235'577","reported_seq":9864,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:13.171231+0000","last_change":"2024-09-16T20:52:13.171231+0000","last_active":"2024-09-16T20:52:13.171231+0000","last_peered":"2024-09-16T20:52:13.171231+0000","last_clean":"2024-09-16T20:52:13.171231+0000","last_became_active":"2024-09-16T20:51:03.590152+0000","last_became_peered":"2024-09-16T20:51:03.590152+0000","last_unstale":"2024-09-16T20:52:13.171231+0000","last_undegraded":"2024-09-16T20:52:13.171231+0000","last_fullsized":"2024-09-16T20:52:13.171231+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"235'577","last_scrub_stamp":"2024-09-16T20:52:13.171118+0000","last_deep_scrub":"235'577","last_deep_scrub_stamp":"2024-09-16T20:52:13.171118+0000","last_clean_scrub_stamp":"2024-09-16T20:52:13.171118+0000","objects_scrubbed":10,"log_size":577,"log_dups_size":0,"ondisk_log_size":577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:42:38.500496+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.0017317439999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":9605,"num_read_kb":51536,"num_write":618,"num_write_kb":12681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":14274951,"num_keys_recovered":91,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"235'479","reported_seq":8116,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:13.826297+0000","last_change":"2024-09-16T20:52:13.826297+0000","last_active":"2024-09-16T20:52:13.826297+0000","last_peered":"2024-09-16T20:52:13.826297+0000","last_clean":"2024-09-16T20:52:13.826297+0000","last_became_active":"2024-09-16T20:51:16.575698+0000","last_became_peered":"2024-09-16T20:51:16.575698+0000","last_unstale":"2024-09-16T20:52:13.826297+0000","last_undegraded":"2024-09-16T20:52:13.826297+0000","last_fullsized":"2024-09-16T20:52:13.826297+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"235'479","last_scrub_stamp":"2024-09-16T20:52:13.826205+0000","last_deep_scrub":"235'479","last_deep_scrub_stamp":"2024-09-16T20:52:13.826205+0000","last_clean_scrub_stamp":"2024-09-16T20:52:13.826205+0000","objects_scrubbed":11,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:53:59.079570+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.0020731130000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":7052,"num_read_kb":42310,"num_write":428,"num_write_kb":14897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":445,"num_bytes_recovered":8390493,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"235'733","reported_seq":22788,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:14.195512+0000","last_change":"2024-09-16T20:52:14.195512+0000","last_active":"2024-09-16T20:52:14.195512+0000","last_peered":"2024-09-16T20:52:14.195512+0000","last_clean":"2024-09-16T20:52:14.195512+0000","last_became_active":"2024-09-16T20:51:17.575222+0000","last_became_peered":"2024-09-16T20:51:17.575222+0000","last_unstale":"2024-09-16T20:52:14.195512+0000","last_undegraded":"2024-09-16T20:52:14.195512+0000","last_fullsized":"2024-09-16T20:52:14.195512+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"235'733","last_scrub_stamp":"2024-09-16T20:52:14.195379+0000","last_deep_scrub":"235'733","last_deep_scrub_stamp":"2024-09-16T20:52:14.195379+0000","last_clean_scrub_stamp":"2024-09-16T20:52:14.195379+0000","objects_scrubbed":7,"log_size":733,"log_dups_size":0,"ondisk_log_size":733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:38:08.019258+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.0018824849999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":22418,"num_read_kb":91945,"num_write":909,"num_write_kb":19380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":627,"num_bytes_recovered":71963834,"num_keys_recovered":119,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"235'1140","reported_seq":18664,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:15.492148+0000","last_change":"2024-09-16T20:52:15.492148+0000","last_active":"2024-09-16T20:52:15.492148+0000","last_peered":"2024-09-16T20:52:15.492148+0000","last_clean":"2024-09-16T20:52:15.492148+0000","last_became_active":"2024-09-16T20:51:05.581172+0000","last_became_peered":"2024-09-16T20:51:05.581172+0000","last_unstale":"2024-09-16T20:52:15.492148+0000","last_undegraded":"2024-09-16T20:52:15.492148+0000","last_fullsized":"2024-09-16T20:52:15.492148+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"235'1140","last_scrub_stamp":"2024-09-16T20:52:15.492032+0000","last_deep_scrub":"235'1140","last_deep_scrub_stamp":"2024-09-16T20:52:15.492032+0000","last_clean_scrub_stamp":"2024-09-16T20:52:15.492032+0000","objects_scrubbed":12,"log_size":1140,"log_dups_size":0,"ondisk_log_size":1140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:57:39.275597+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.002500038,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":19435,"num_read_kb":70139,"num_write":1420,"num_write_kb":21576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":16987444,"num_keys_recovered":40,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"235'587","reported_seq":9289,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:12.314291+0000","last_change":"2024-09-16T20:52:12.314291+0000","last_active":"2024-09-16T20:52:12.314291+0000","last_peered":"2024-09-16T20:52:12.314291+0000","last_clean":"2024-09-16T20:52:12.314291+0000","last_became_active":"2024-09-16T20:51:27.627520+0000","last_became_peered":"2024-09-16T20:51:27.627520+0000","last_unstale":"2024-09-16T20:52:12.314291+0000","last_undegraded":"2024-09-16T20:52:12.314291+0000","last_fullsized":"2024-09-16T20:52:12.314291+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"235'587","last_scrub_stamp":"2024-09-16T20:52:12.314212+0000","last_deep_scrub":"235'587","last_deep_scrub_stamp":"2024-09-16T20:52:12.314212+0000","last_clean_scrub_stamp":"2024-09-16T20:52:12.314212+0000","objects_scrubbed":6,"log_size":587,"log_dups_size":0,"ondisk_log_size":587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:05:40.056408+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.0012113930000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":8329,"num_read_kb":54907,"num_write":641,"num_write_kb":11423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":479,"num_bytes_recovered":14706083,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"235'542","reported_seq":13772,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:15.187523+0000","last_change":"2024-09-16T20:52:15.187523+0000","last_active":"2024-09-16T20:52:15.187523+0000","last_peered":"2024-09-16T20:52:15.187523+0000","last_clean":"2024-09-16T20:52:15.187523+0000","last_became_active":"2024-09-16T20:51:05.581058+0000","last_became_peered":"2024-09-16T20:51:05.581058+0000","last_unstale":"2024-09-16T20:52:15.187523+0000","last_undegraded":"2024-09-16T20:52:15.187523+0000","last_fullsized":"2024-09-16T20:52:15.187523+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"235'542","last_scrub_stamp":"2024-09-16T20:52:15.187364+0000","last_deep_scrub":"235'542","last_deep_scrub_stamp":"2024-09-16T20:52:15.187364+0000","last_clean_scrub_stamp":"2024-09-16T20:52:15.187364+0000","objects_scrubbed":8,"log_size":542,"log_dups_size":0,"ondisk_log_size":542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:56:13.815765+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0026424640000000002,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":13544,"num_read_kb":49574,"num_write":426,"num_write_kb":14576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":284,"num_bytes_recovered":9641443,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"235'698","reported_seq":13768,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:14.444317+0000","last_change":"2024-09-16T20:52:14.444317+0000","last_active":"2024-09-16T20:52:14.444317+0000","last_peered":"2024-09-16T20:52:14.444317+0000","last_clean":"2024-09-16T20:52:14.444317+0000","last_became_active":"2024-09-16T20:49:12.801013+0000","last_became_peered":"2024-09-16T20:49:12.801013+0000","last_unstale":"2024-09-16T20:52:14.444317+0000","last_undegraded":"2024-09-16T20:52:14.444317+0000","last_fullsized":"2024-09-16T20:52:14.444317+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"235'698","last_scrub_stamp":"2024-09-16T20:52:14.444213+0000","last_deep_scrub":"235'698","last_deep_scrub_stamp":"2024-09-16T20:52:14.444213+0000","last_clean_scrub_stamp":"2024-09-16T20:52:14.444213+0000","objects_scrubbed":7,"log_size":698,"log_dups_size":0,"ondisk_log_size":698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:27:16.866424+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.001757113,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":15597,"num_read_kb":58377,"num_write":523,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":99,"num_bytes_recovered":5210742,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"235'894","reported_seq":14351,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:13.290264+0000","last_change":"2024-09-16T20:52:13.290264+0000","last_active":"2024-09-16T20:52:13.290264+0000","last_peered":"2024-09-16T20:52:13.290264+0000","last_clean":"2024-09-16T20:52:13.290264+0000","last_became_active":"2024-09-16T20:51:26.624407+0000","last_became_peered":"2024-09-16T20:51:26.624407+0000","last_unstale":"2024-09-16T20:52:13.290264+0000","last_undegraded":"2024-09-16T20:52:13.290264+0000","last_fullsized":"2024-09-16T20:52:13.290264+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"235'894","last_scrub_stamp":"2024-09-16T20:52:13.290166+0000","last_deep_scrub":"235'894","last_deep_scrub_stamp":"2024-09-16T20:52:13.290166+0000","last_clean_scrub_stamp":"2024-09-16T20:52:13.290166+0000","objects_scrubbed":6,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:24:43.944787+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.0029313379999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":16446,"num_read_kb":75991,"num_write":874,"num_write_kb":24549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":351,"num_bytes_recovered":38747648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"235'617","reported_seq":20180,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:14.217470+0000","last_change":"2024-09-16T20:52:14.217470+0000","last_active":"2024-09-16T20:52:14.217470+0000","last_peered":"2024-09-16T20:52:14.217470+0000","last_clean":"2024-09-16T20:52:14.217470+0000","last_became_active":"2024-09-16T20:51:03.589742+0000","last_became_peered":"2024-09-16T20:51:03.589742+0000","last_unstale":"2024-09-16T20:52:14.217470+0000","last_undegraded":"2024-09-16T20:52:14.217470+0000","last_fullsized":"2024-09-16T20:52:14.217470+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"235'617","last_scrub_stamp":"2024-09-16T20:52:14.217358+0000","last_deep_scrub":"235'617","last_deep_scrub_stamp":"2024-09-16T20:52:14.217358+0000","last_clean_scrub_stamp":"2024-09-16T20:52:14.217358+0000","objects_scrubbed":9,"log_size":617,"log_dups_size":0,"ondisk_log_size":617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:59:12.507993+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.0024335089999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":16969,"num_read_kb":76220,"num_write":613,"num_write_kb":13841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":486,"num_bytes_recovered":15329756,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"235'470","reported_seq":12021,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-16T20:52:16.536320+0000","last_change":"2024-09-16T20:52:16.536320+0000","last_active":"2024-09-16T20:52:16.536320+0000","last_peered":"2024-09-16T20:52:16.536320+0000","last_clean":"2024-09-16T20:52:16.536320+0000","last_became_active":"2024-09-16T20:51:03.595360+0000","last_became_peered":"2024-09-16T20:51:03.595360+0000","last_unstale":"2024-09-16T20:52:16.536320+0000","last_undegraded":"2024-09-16T20:52:16.536320+0000","last_fullsized":"2024-09-16T20:52:16.536320+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"235'470","last_scrub_stamp":"2024-09-16T20:52:16.536212+0000","last_deep_scrub":"235'470","last_deep_scrub_stamp":"2024-09-16T20:52:16.536212+0000","last_clean_scrub_stamp":"2024-09-16T20:52:16.536212+0000","objects_scrubbed":9,"log_size":470,"log_dups_size":0,"ondisk_log_size":470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:54:02.471208+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.0035295309999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":11262,"num_read_kb":35639,"num_write":562,"num_write_kb":13645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":198,"num_bytes_recovered":14217900,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":247,"num_object_clones":0,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":243,"num_read":448597,"num_read_kb":1798155,"num_write":23462,"num_write_kb":469671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12645,"num_bytes_recovered":606441956,"num_keys_recovered":1497,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22394,"ondisk_log_size":22394,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574908,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52576,"kb_used_data":3344,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319264,"statfs":{"total":96636764160,"available":96582926336,"internally_reserved":0,"allocated":3424256,"data_stored":1774023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574908,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74136,"kb_used_data":2760,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94297704,"statfs":{"total":96636764160,"available":96560848896,"internally_reserved":0,"allocated":2826240,"data_stored":1183636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73071047},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574908,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69976,"kb_used_data":2760,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94301864,"statfs":{"total":96636764160,"available":96565108736,"internally_reserved":0,"allocated":2826240,"data_stored":1183636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574907,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70560,"kb_used_data":3344,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94301280,"statfs":{"total":96636764160,"available":96564510720,"internally_reserved":0,"allocated":3424256,"data_stored":1774023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574908,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":96412,"kb_used_data":2764,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":94275428,"statfs":{"total":96636764160,"available":96538038272,"internally_reserved":0,"allocated":2830336,"data_stored":1183655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":95877574},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640316,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60316,"kb_used_data":2764,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311524,"statfs":{"total":96636764160,"available":96575000576,"internally_reserved":0,"allocated":2830336,"data_stored":1183655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T20:52:32.218 DEBUG:teuthology.orchestra.run.smithi029:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-16T20:52:32.531 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-16T20:52:32.531 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-16T20:52:32.532 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-16T20:52:32.532 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T20:52:32.578 INFO:tasks.ceph.osd.0:Stopped 2024-09-16T20:52:32.578 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-16T20:52:32.578 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T20:52:32.621 INFO:tasks.ceph.osd.1:Stopped 2024-09-16T20:52:32.621 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-16T20:52:32.621 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T20:52:32.664 INFO:tasks.ceph.osd.2:Stopped 2024-09-16T20:52:32.665 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-16T20:52:32.665 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T20:52:32.701 INFO:tasks.ceph.osd.3:Stopped 2024-09-16T20:52:32.702 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-16T20:52:32.702 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T20:52:32.778 INFO:tasks.ceph.osd.4:Stopped 2024-09-16T20:52:32.778 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-16T20:52:32.778 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T20:52:32.857 INFO:tasks.ceph.osd.5:Stopped 2024-09-16T20:52:32.858 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-16T20:52:32.858 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-16T20:52:32.858 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T20:52:32.928 INFO:tasks.ceph.mgr.x:Stopped 2024-09-16T20:52:32.928 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-16T20:52:32.928 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-16T20:52:32.928 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T20:52:32.940 INFO:tasks.ceph.mon.a:Stopped 2024-09-16T20:52:32.940 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-16T20:52:32.940 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T20:52:32.999 INFO:tasks.ceph.mon.c:Stopped 2024-09-16T20:52:33.288 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-16T20:52:33.288 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T20:52:33.303 INFO:tasks.ceph.mon.b:Stopped 2024-09-16T20:52:33.304 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-16T20:52:33.304 DEBUG:teuthology.orchestra.run.smithi029:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v SLOW_OPS | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v noscrub | egrep -v nodeep-scrub | head -n 1 2024-09-16T20:52:33.344 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi029.front.sepia.ceph.com 2024-09-16T20:52:33.344 DEBUG:teuthology.orchestra.run.smithi029:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-16T20:52:33.663 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi029.front.sepia.ceph.com 2024-09-16T20:52:33.663 DEBUG:teuthology.orchestra.run.smithi029:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-16T20:52:33.745 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi029.front.sepia.ceph.com 2024-09-16T20:52:33.745 DEBUG:teuthology.orchestra.run.smithi029:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-16T20:52:33.834 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi043.front.sepia.ceph.com 2024-09-16T20:52:33.834 DEBUG:teuthology.orchestra.run.smithi043:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-16T20:52:33.952 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi043.front.sepia.ceph.com 2024-09-16T20:52:33.952 DEBUG:teuthology.orchestra.run.smithi043:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-16T20:52:34.014 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi043.front.sepia.ceph.com 2024-09-16T20:52:34.015 DEBUG:teuthology.orchestra.run.smithi043:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-16T20:52:34.072 INFO:tasks.ceph:Archiving mon data... 2024-09-16T20:52:34.073 DEBUG:teuthology.misc:Transferring archived files from smithi029:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/dis-2024-09-16_19:45:14-rbd-wip-dis-testing-distro-default-smithi/7907507/data/mon.a.tgz 2024-09-16T20:52:34.073 DEBUG:teuthology.orchestra.run.smithi029:> mktemp 2024-09-16T20:52:34.092 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/tmp.tCEorTcxAf 2024-09-16T20:52:34.092 DEBUG:teuthology.orchestra.run.smithi029:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.tCEorTcxAf 2024-09-16T20:52:34.268 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod 0666 /tmp/tmp.tCEorTcxAf 2024-09-16T20:52:34.354 DEBUG:teuthology.orchestra.remote:smithi029:/tmp/tmp.tCEorTcxAf is 405KB 2024-09-16T20:52:34.451 DEBUG:teuthology.orchestra.run.smithi029:> rm -fr /tmp/tmp.tCEorTcxAf 2024-09-16T20:52:34.466 DEBUG:teuthology.misc:Transferring archived files from smithi029:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/dis-2024-09-16_19:45:14-rbd-wip-dis-testing-distro-default-smithi/7907507/data/mon.c.tgz 2024-09-16T20:52:34.467 DEBUG:teuthology.orchestra.run.smithi029:> mktemp 2024-09-16T20:52:34.521 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/tmp.5ViWR81tmg 2024-09-16T20:52:34.522 DEBUG:teuthology.orchestra.run.smithi029:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.5ViWR81tmg 2024-09-16T20:52:34.706 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod 0666 /tmp/tmp.5ViWR81tmg 2024-09-16T20:52:34.790 DEBUG:teuthology.orchestra.remote:smithi029:/tmp/tmp.5ViWR81tmg is 534KB 2024-09-16T20:52:34.891 DEBUG:teuthology.orchestra.run.smithi029:> rm -fr /tmp/tmp.5ViWR81tmg 2024-09-16T20:52:34.907 DEBUG:teuthology.misc:Transferring archived files from smithi043:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/dis-2024-09-16_19:45:14-rbd-wip-dis-testing-distro-default-smithi/7907507/data/mon.b.tgz 2024-09-16T20:52:34.907 DEBUG:teuthology.orchestra.run.smithi043:> mktemp 2024-09-16T20:52:34.926 INFO:teuthology.orchestra.run.smithi043.stdout:/tmp/tmp.VX3XAjIvUO 2024-09-16T20:52:34.926 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.VX3XAjIvUO 2024-09-16T20:52:35.128 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 0666 /tmp/tmp.VX3XAjIvUO 2024-09-16T20:52:35.213 DEBUG:teuthology.orchestra.remote:smithi043:/tmp/tmp.VX3XAjIvUO is 535KB 2024-09-16T20:52:35.309 DEBUG:teuthology.orchestra.run.smithi043:> rm -fr /tmp/tmp.VX3XAjIvUO 2024-09-16T20:52:35.325 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-16T20:52:35.325 DEBUG:teuthology.orchestra.run.smithi029:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-16T20:52:35.329 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-16T20:52:35.367 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-16T20:52:35.414 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-16T20:52:35.445 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-16T20:52:35.463 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-16T20:52:35.464 INFO:tasks.ceph:Archiving crash dumps... 2024-09-16T20:52:35.465 DEBUG:teuthology.misc:Transferring archived files from smithi029:/var/lib/ceph/crash to /home/teuthworker/archive/dis-2024-09-16_19:45:14-rbd-wip-dis-testing-distro-default-smithi/7907507/remote/smithi029/crash 2024-09-16T20:52:35.466 DEBUG:teuthology.orchestra.run.smithi029:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-16T20:52:35.492 DEBUG:teuthology.misc:Transferring archived files from smithi043:/var/lib/ceph/crash to /home/teuthworker/archive/dis-2024-09-16_19:45:14-rbd-wip-dis-testing-distro-default-smithi/7907507/remote/smithi043/crash 2024-09-16T20:52:35.492 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-16T20:52:35.518 DEBUG:teuthology.misc:Transferring archived files from smithi139:/var/lib/ceph/crash to /home/teuthworker/archive/dis-2024-09-16_19:45:14-rbd-wip-dis-testing-distro-default-smithi/7907507/remote/smithi139/crash 2024-09-16T20:52:35.519 DEBUG:teuthology.orchestra.run.smithi139:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-16T20:52:35.556 INFO:tasks.ceph:Compressing logs... 2024-09-16T20:52:35.556 DEBUG:teuthology.orchestra.run.smithi029:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-16T20:52:35.559 DEBUG:teuthology.orchestra.run.smithi043:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-16T20:52:35.563 DEBUG:teuthology.orchestra.run.smithi139:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-16T20:52:35.581 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-16T20:52:35.581 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38267.log 2024-09-16T20:52:35.595 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-16T20:52:35.595 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-16T20:52:35.595 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31973.log 2024-09-16T20:52:35.596 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-osd.4.log: /var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32093.log 2024-09-16T20:52:35.596 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.31973.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-16T20:52:35.596 INFO:teuthology.orchestra.run.smithi043.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31973.log.gz 2024-09-16T20:52:35.596 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-16T20:52:35.596 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.32093.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32093.log.gz 2024-09-16T20:52:35.596 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-16T20:52:35.596 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32013.log 2024-09-16T20:52:35.596 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-mgr.x.log: 89.1% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-16T20:52:35.596 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32133.log 2024-09-16T20:52:35.596 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.32013.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-16T20:52:35.596 INFO:teuthology.orchestra.run.smithi043.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32013.log.gz 2024-09-16T20:52:35.596 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.32133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32053.log 2024-09-16T20:52:35.597 INFO:teuthology.orchestra.run.smithi043.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32133.log.gz 2024-09-16T20:52:35.597 INFO:teuthology.orchestra.run.smithi043.stderr: 84.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-16T20:52:35.597 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32173.log 2024-09-16T20:52:35.597 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-16T20:52:35.597 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.32053.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32053.log.gz 2024-09-16T20:52:35.597 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.32173.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32173.log.gz 2024-09-16T20:52:35.654 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-osd.3.log: 91.8% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-16T20:52:35.759 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30245.log 2024-09-16T20:52:35.759 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27618.log 2024-09-16T20:52:35.759 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.30245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28212.log 2024-09-16T20:52:35.759 INFO:teuthology.orchestra.run.smithi139.stderr: 53.6% -- replaced with /var/log/ceph/ceph-client.admin.30245.log.gz 2024-09-16T20:52:35.759 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27469.log 2024-09-16T20:52:35.759 INFO:teuthology.orchestra.run.smithi139.stderr: 93.9% -- replaced with /var/log/ceph/ceph-client.admin.27618.log.gz 2024-09-16T20:52:35.759 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28178.log 2024-09-16T20:52:35.759 INFO:teuthology.orchestra.run.smithi139.stderr: 55.7% -- replaced with /var/log/ceph/ceph-client.admin.28212.log.gz 2024-09-16T20:52:35.759 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28529.log 2024-09-16T20:52:35.760 INFO:teuthology.orchestra.run.smithi139.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.27469.log.gz 2024-09-16T20:52:35.760 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28703.log 2024-09-16T20:52:35.760 INFO:teuthology.orchestra.run.smithi139.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.28178.log.gz 2024-09-16T20:52:35.760 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29361.log 2024-09-16T20:52:35.760 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28529.log: 53.0% -- replaced with /var/log/ceph/ceph-client.admin.28529.log.gz 2024-09-16T20:52:35.760 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30492.log 2024-09-16T20:52:35.760 INFO:teuthology.orchestra.run.smithi139.stderr: 25.3% -- replaced with /var/log/ceph/ceph-client.admin.28703.log.gz 2024-09-16T20:52:35.760 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29264.log 2024-09-16T20:52:35.760 INFO:teuthology.orchestra.run.smithi139.stderr: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.29361.log.gz 2024-09-16T20:52:35.760 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.30492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27713.log 2024-09-16T20:52:35.760 INFO:teuthology.orchestra.run.smithi139.stderr: 93.8% -- replaced with /var/log/ceph/ceph-client.admin.30492.log.gz 2024-09-16T20:52:35.760 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27853.log 2024-09-16T20:52:35.761 INFO:teuthology.orchestra.run.smithi139.stderr: 53.1% -- replaced with /var/log/ceph/ceph-client.admin.29264.log.gz 2024-09-16T20:52:35.761 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28871.log 2024-09-16T20:52:35.761 INFO:teuthology.orchestra.run.smithi139.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.27713.log.gz 2024-09-16T20:52:35.761 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29745.log 2024-09-16T20:52:35.761 INFO:teuthology.orchestra.run.smithi139.stderr: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.27853.log.gz 2024-09-16T20:52:35.761 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30236.log 2024-09-16T20:52:35.761 INFO:teuthology.orchestra.run.smithi139.stderr: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.28871.log.gz 2024-09-16T20:52:35.761 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27688.log 2024-09-16T20:52:35.761 INFO:teuthology.orchestra.run.smithi139.stderr: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.29745.log.gz 2024-09-16T20:52:35.761 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.30236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29614.log 2024-09-16T20:52:35.761 INFO:teuthology.orchestra.run.smithi139.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.30236.log.gz 2024-09-16T20:52:35.762 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30071.log 2024-09-16T20:52:35.762 INFO:teuthology.orchestra.run.smithi139.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.27688.log.gz 2024-09-16T20:52:35.762 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28343.log 2024-09-16T20:52:35.762 INFO:teuthology.orchestra.run.smithi139.stderr: 76.5% -- replaced with /var/log/ceph/ceph-client.admin.29614.log.gz 2024-09-16T20:52:35.762 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.30071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27758.log 2024-09-16T20:52:35.762 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29781.log 2024-09-16T20:52:35.762 INFO:teuthology.orchestra.run.smithi139.stderr: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.28343.log.gz 2024-09-16T20:52:35.762 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27758.log: 96.8% -- replaced with /var/log/ceph/ceph-client.admin.30071.log.gz 2024-09-16T20:52:35.762 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28773.log 2024-09-16T20:52:35.762 INFO:teuthology.orchestra.run.smithi139.stderr: 77.8% -- replaced with /var/log/ceph/ceph-client.admin.27758.log.gz 2024-09-16T20:52:35.762 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28389.log 2024-09-16T20:52:35.762 INFO:teuthology.orchestra.run.smithi139.stderr: 18.4% -- replaced with /var/log/ceph/ceph-client.admin.29781.log.gz 2024-09-16T20:52:35.763 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27819.log 2024-09-16T20:52:35.763 INFO:teuthology.orchestra.run.smithi139.stderr: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.28773.log.gz 2024-09-16T20:52:35.763 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29544.log 2024-09-16T20:52:35.763 INFO:teuthology.orchestra.run.smithi139.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.28389.log.gz 2024-09-16T20:52:35.763 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29510.log 2024-09-16T20:52:35.763 INFO:teuthology.orchestra.run.smithi139.stderr: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.27819.log.gz 2024-09-16T20:52:35.763 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29641.log 2024-09-16T20:52:35.763 INFO:teuthology.orchestra.run.smithi139.stderr: 78.7% -- replaced with /var/log/ceph/ceph-client.admin.29544.log.gz 2024-09-16T20:52:35.763 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29931.log 2024-09-16T20:52:35.763 INFO:teuthology.orchestra.run.smithi139.stderr: 76.2% -- replaced with /var/log/ceph/ceph-client.admin.29510.log.gz 2024-09-16T20:52:35.763 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30342.log 2024-09-16T20:52:35.763 INFO:teuthology.orchestra.run.smithi139.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.29641.log.gz 2024-09-16T20:52:35.763 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28739.log 2024-09-16T20:52:35.763 INFO:teuthology.orchestra.run.smithi139.stderr: 94.0% -- replaced with /var/log/ceph/ceph-client.admin.29931.log.gz 2024-09-16T20:52:35.764 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.30342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29124.log 2024-09-16T20:52:35.764 INFO:teuthology.orchestra.run.smithi139.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.30342.log.gz 2024-09-16T20:52:35.764 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28063.log 2024-09-16T20:52:35.764 INFO:teuthology.orchestra.run.smithi139.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.28739.log.gz 2024-09-16T20:52:35.764 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30026.log 2024-09-16T20:52:35.764 INFO:teuthology.orchestra.run.smithi139.stderr: 54.8% -- replaced with /var/log/ceph/ceph-client.admin.29124.log.gz 2024-09-16T20:52:35.764 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27269.log 2024-09-16T20:52:35.764 INFO:teuthology.orchestra.run.smithi139.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.28063.log.gz 2024-09-16T20:52:35.764 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.30026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29395.log 2024-09-16T20:52:35.764 INFO:teuthology.orchestra.run.smithi139.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.30026.log.gz 2024-09-16T20:52:35.764 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27783.log 2024-09-16T20:52:35.764 INFO:teuthology.orchestra.run.smithi139.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.27269.log.gz 2024-09-16T20:52:35.765 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29895.log 2024-09-16T20:52:35.765 INFO:teuthology.orchestra.run.smithi139.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.29395.log.gz 2024-09-16T20:52:35.765 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27432.log 2024-09-16T20:52:35.765 INFO:teuthology.orchestra.run.smithi139.stderr: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.27783.log.gz 2024-09-16T20:52:35.765 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29151.log 2024-09-16T20:52:35.765 INFO:teuthology.orchestra.run.smithi139.stderr: 53.9% -- replaced with /var/log/ceph/ceph-client.admin.29895.log.gz 2024-09-16T20:52:35.765 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29815.log 2024-09-16T20:52:35.765 INFO:teuthology.orchestra.run.smithi139.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.27432.log.gz 2024-09-16T20:52:35.765 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29535.log 2024-09-16T20:52:35.765 INFO:teuthology.orchestra.run.smithi139.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.29151.log.gz 2024-09-16T20:52:35.765 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29291.log 2024-09-16T20:52:35.765 INFO:teuthology.orchestra.run.smithi139.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.29815.log.gz 2024-09-16T20:52:35.766 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28108.log 2024-09-16T20:52:35.766 INFO:teuthology.orchestra.run.smithi139.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.29535.log.gz 2024-09-16T20:52:35.766 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27749.log 2024-09-16T20:52:35.766 INFO:teuthology.orchestra.run.smithi139.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.29291.log.gz 2024-09-16T20:52:35.766 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29571.log 2024-09-16T20:52:35.766 INFO:teuthology.orchestra.run.smithi139.stderr: 95.1% -- replaced with /var/log/ceph/ceph-client.admin.28108.log.gz 2024-09-16T20:52:35.766 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28352.log 2024-09-16T20:52:35.766 INFO:teuthology.orchestra.run.smithi139.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.27749.log.gz 2024-09-16T20:52:35.766 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29571.log: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.29571.log.gz 2024-09-16T20:52:35.766 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27582.log 2024-09-16T20:52:35.766 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28624.log 2024-09-16T20:52:35.766 INFO:teuthology.orchestra.run.smithi139.stderr: 25.3% -- replaced with /var/log/ceph/ceph-client.admin.28352.log.gz 2024-09-16T20:52:35.766 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29230.log 2024-09-16T20:52:35.766 INFO:teuthology.orchestra.run.smithi139.stderr: 56.8% -- replaced with /var/log/ceph/ceph-client.admin.27582.log.gz 2024-09-16T20:52:35.767 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27503.log 2024-09-16T20:52:35.767 INFO:teuthology.orchestra.run.smithi139.stderr: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.28624.log.gz 2024-09-16T20:52:35.767 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28764.log 2024-09-16T20:52:35.767 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27503.log: 96.1% -- replaced with /var/log/ceph/ceph-client.admin.29230.log.gz 2024-09-16T20:52:35.767 INFO:teuthology.orchestra.run.smithi139.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.27503.log.gz 2024-09-16T20:52:35.767 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29824.log 2024-09-16T20:52:35.767 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28764.log: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.28764.log.gz 2024-09-16T20:52:35.767 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27898.log 2024-09-16T20:52:35.767 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29824.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29255.log 2024-09-16T20:52:35.767 INFO:teuthology.orchestra.run.smithi139.stderr: 56.3% -- replaced with /var/log/ceph/ceph-client.admin.29824.log.gz 2024-09-16T20:52:35.767 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28599.log 2024-09-16T20:52:35.767 INFO:teuthology.orchestra.run.smithi139.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.27898.log.gz 2024-09-16T20:52:35.767 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27548.log 2024-09-16T20:52:35.768 INFO:teuthology.orchestra.run.smithi139.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.29255.log.gz 2024-09-16T20:52:35.768 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28273.log 2024-09-16T20:52:35.768 INFO:teuthology.orchestra.run.smithi139.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.28599.log.gz 2024-09-16T20:52:35.768 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28459.log 2024-09-16T20:52:35.768 INFO:teuthology.orchestra.run.smithi139.stderr: 94.8% -- replaced with /var/log/ceph/ceph-client.admin.27548.log.gz 2024-09-16T20:52:35.768 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27372.log 2024-09-16T20:52:35.768 INFO:teuthology.orchestra.run.smithi139.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.28273.log.gz 2024-09-16T20:52:35.768 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29684.log 2024-09-16T20:52:35.768 INFO:teuthology.orchestra.run.smithi139.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.28459.log.gz 2024-09-16T20:52:35.768 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29404.log 2024-09-16T20:52:35.768 INFO:teuthology.orchestra.run.smithi139.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.27372.log.gz 2024-09-16T20:52:35.768 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29675.log 2024-09-16T20:52:35.768 INFO:teuthology.orchestra.run.smithi139.stderr: 54.2% -- replaced with /var/log/ceph/ceph-client.admin.29684.log.gz 2024-09-16T20:52:35.769 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28423.log 2024-09-16T20:52:35.769 INFO:teuthology.orchestra.run.smithi139.stderr: 75.5% -- replaced with /var/log/ceph/ceph-client.admin.29404.log.gz 2024-09-16T20:52:35.769 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27407.log 2024-09-16T20:52:35.769 INFO:teuthology.orchestra.run.smithi139.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.29675.log.gz 2024-09-16T20:52:35.769 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28660.log 2024-09-16T20:52:35.769 INFO:teuthology.orchestra.run.smithi139.stderr: 53.7% -- replaced with /var/log/ceph/ceph-client.admin.28423.log.gz 2024-09-16T20:52:35.769 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28520.log 2024-09-16T20:52:35.769 INFO:teuthology.orchestra.run.smithi139.stderr: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.27407.log.gz 2024-09-16T20:52:35.769 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27278.log 2024-09-16T20:52:35.769 INFO:teuthology.orchestra.run.smithi139.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.28660.log.gz 2024-09-16T20:52:35.769 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29081.log 2024-09-16T20:52:35.769 INFO:teuthology.orchestra.run.smithi139.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.28520.log.gz 2024-09-16T20:52:35.769 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29334.log 2024-09-16T20:52:35.770 INFO:teuthology.orchestra.run.smithi139.stderr: 56.5% -- replaced with /var/log/ceph/ceph-client.admin.27278.log.gz 2024-09-16T20:52:35.770 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28380.log 2024-09-16T20:52:35.770 INFO:teuthology.orchestra.run.smithi139.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.29081.log.gz 2024-09-16T20:52:35.770 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28072.log 2024-09-16T20:52:35.770 INFO:teuthology.orchestra.run.smithi139.stderr: 84.2% -- replaced with /var/log/ceph/ceph-client.admin.29334.log.gz 2024-09-16T20:52:35.770 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29011.log 2024-09-16T20:52:35.770 INFO:teuthology.orchestra.run.smithi139.stderr: 20.9% -- replaced with /var/log/ceph/ceph-client.admin.28380.log.gz 2024-09-16T20:52:35.770 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30351.log 2024-09-16T20:52:35.770 INFO:teuthology.orchestra.run.smithi139.stderr: 53.3% -- replaced with /var/log/ceph/ceph-client.admin.28072.log.gz 2024-09-16T20:52:35.770 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29185.log 2024-09-16T20:52:35.770 INFO:teuthology.orchestra.run.smithi139.stderr: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.29011.log.gz 2024-09-16T20:52:35.770 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.30351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28484.log 2024-09-16T20:52:35.770 INFO:teuthology.orchestra.run.smithi139.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.30351.log.gz 2024-09-16T20:52:35.771 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29886.log 2024-09-16T20:52:35.771 INFO:teuthology.orchestra.run.smithi139.stderr: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.29185.log.gz 2024-09-16T20:52:35.771 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30455.log 2024-09-16T20:52:35.771 INFO:teuthology.orchestra.run.smithi139.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.28484.log.gz 2024-09-16T20:52:35.771 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27652.log 2024-09-16T20:52:35.771 INFO:teuthology.orchestra.run.smithi139.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.29886.log.gz 2024-09-16T20:52:35.771 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.30455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27643.log 2024-09-16T20:52:35.771 INFO:teuthology.orchestra.run.smithi139.stderr: 91.8% -- replaced with /var/log/ceph/ceph-client.admin.30455.log.gz 2024-09-16T20:52:35.771 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29370.log 2024-09-16T20:52:35.771 INFO:teuthology.orchestra.run.smithi139.stderr: 53.9% -- replaced with /var/log/ceph/ceph-client.admin.27652.log.gz 2024-09-16T20:52:35.771 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27679.log 2024-09-16T20:52:35.771 INFO:teuthology.orchestra.run.smithi139.stderr: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.27643.log.gz 2024-09-16T20:52:35.771 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28038.log 2024-09-16T20:52:35.772 INFO:teuthology.orchestra.run.smithi139.stderr: 53.2% -- replaced with /var/log/ceph/ceph-client.admin.29370.log.gz 2024-09-16T20:52:35.772 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29440.log 2024-09-16T20:52:35.772 INFO:teuthology.orchestra.run.smithi139.stderr: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.27679.log.gz 2024-09-16T20:52:35.772 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29194.log 2024-09-16T20:52:35.772 INFO:teuthology.orchestra.run.smithi139.stderr: 95.9% -- replaced with /var/log/ceph/ceph-client.admin.28038.log.gz 2024-09-16T20:52:35.772 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29754.log 2024-09-16T20:52:35.772 INFO:teuthology.orchestra.run.smithi139.stderr: 52.7% -- replaced with /var/log/ceph/ceph-client.admin.29440.log.gz 2024-09-16T20:52:35.772 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30315.log 2024-09-16T20:52:35.772 INFO:teuthology.orchestra.run.smithi139.stderr: 25.3% -- replaced with /var/log/ceph/ceph-client.admin.29194.log.gz 2024-09-16T20:52:35.772 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28941.log 2024-09-16T20:52:35.772 INFO:teuthology.orchestra.run.smithi139.stderr: 81.9% -- replaced with /var/log/ceph/ceph-client.admin.29754.log.gz 2024-09-16T20:52:35.772 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.30315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30562.log 2024-09-16T20:52:35.772 INFO:teuthology.orchestra.run.smithi139.stderr: 83.0% -- replaced with /var/log/ceph/ceph-client.admin.30315.log.gz 2024-09-16T20:52:35.773 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27722.log 2024-09-16T20:52:35.773 INFO:teuthology.orchestra.run.smithi139.stderr: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.28941.log.gz 2024-09-16T20:52:35.773 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.30562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27959.log 2024-09-16T20:52:35.773 INFO:teuthology.orchestra.run.smithi139.stderr: 56.9% -- replaced with /var/log/ceph/ceph-client.admin.30562.log.gz 2024-09-16T20:52:35.773 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28002.log 2024-09-16T20:52:35.773 INFO:teuthology.orchestra.run.smithi139.stderr: 53.2% -- replaced with /var/log/ceph/ceph-client.admin.27722.log.gz 2024-09-16T20:52:35.773 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29020.log 2024-09-16T20:52:35.773 INFO:teuthology.orchestra.run.smithi139.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.27959.log.gz 2024-09-16T20:52:35.773 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29956.log 2024-09-16T20:52:35.773 INFO:teuthology.orchestra.run.smithi139.stderr: 84.5% -- replaced with /var/log/ceph/ceph-client.admin.28002.log.gz 2024-09-16T20:52:35.773 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29045.log 2024-09-16T20:52:35.773 INFO:teuthology.orchestra.run.smithi139.stderr: 94.8% -- replaced with /var/log/ceph/ceph-client.admin.29020.log.gz 2024-09-16T20:52:35.773 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30202.log 2024-09-16T20:52:35.774 INFO:teuthology.orchestra.run.smithi139.stderr: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.29956.log.gz 2024-09-16T20:52:35.774 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28493.log 2024-09-16T20:52:35.774 INFO:teuthology.orchestra.run.smithi139.stderr: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.29045.log.gz 2024-09-16T20:52:35.774 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.30202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28975.log 2024-09-16T20:52:35.774 INFO:teuthology.orchestra.run.smithi139.stderr: 20.9% -- replaced with /var/log/ceph/ceph-client.admin.30202.log.gz 2024-09-16T20:52:35.774 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30001.log 2024-09-16T20:52:35.774 INFO:teuthology.orchestra.run.smithi139.stderr: 25.9% -- replaced with /var/log/ceph/ceph-client.admin.28493.log.gz 2024-09-16T20:52:35.774 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27968.log 2024-09-16T20:52:35.774 INFO:teuthology.orchestra.run.smithi139.stderr: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.28975.log.gz 2024-09-16T20:52:35.774 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.30001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30526.log 2024-09-16T20:52:35.774 INFO:teuthology.orchestra.run.smithi139.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.30001.log.gz 2024-09-16T20:52:35.774 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29790.log 2024-09-16T20:52:35.774 INFO:teuthology.orchestra.run.smithi139.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.27968.log.gz 2024-09-16T20:52:35.774 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.30526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29580.log 2024-09-16T20:52:35.775 INFO:teuthology.orchestra.run.smithi139.stderr: 55.0% -- replaced with /var/log/ceph/ceph-client.admin.30526.log.gz 2024-09-16T20:52:35.775 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29465.log 2024-09-16T20:52:35.775 INFO:teuthology.orchestra.run.smithi139.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.29790.log.gz 2024-09-16T20:52:35.775 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27512.log 2024-09-16T20:52:35.775 INFO:teuthology.orchestra.run.smithi139.stderr: 94.8% -- replaced with /var/log/ceph/ceph-client.admin.29580.log.gz 2024-09-16T20:52:35.775 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28800.log 2024-09-16T20:52:35.775 INFO:teuthology.orchestra.run.smithi139.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.29465.log.gz 2024-09-16T20:52:35.775 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28029.log 2024-09-16T20:52:35.775 INFO:teuthology.orchestra.run.smithi139.stderr: 54.6% -- replaced with /var/log/ceph/ceph-client.admin.27512.log.gz 2024-09-16T20:52:35.775 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30517.log 2024-09-16T20:52:35.775 INFO:teuthology.orchestra.run.smithi139.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.28800.log.gz 2024-09-16T20:52:35.775 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30421.log 2024-09-16T20:52:35.775 INFO:teuthology.orchestra.run.smithi139.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.28029.log.gz 2024-09-16T20:52:35.775 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.30517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29720.log 2024-09-16T20:52:35.776 INFO:teuthology.orchestra.run.smithi139.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.30517.log.gz 2024-09-16T20:52:35.776 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.30421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29711.log 2024-09-16T20:52:35.776 INFO:teuthology.orchestra.run.smithi139.stderr: 92.9% -- replaced with /var/log/ceph/ceph-client.admin.30421.log.gz 2024-09-16T20:52:35.776 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27932.log 2024-09-16T20:52:35.776 INFO:teuthology.orchestra.run.smithi139.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.admin.29720.log.gz 2024-09-16T20:52:35.776 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28694.log 2024-09-16T20:52:35.776 INFO:teuthology.orchestra.run.smithi139.stderr: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.29711.log.gz 2024-09-16T20:52:35.776 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29992.log 2024-09-16T20:52:35.776 INFO:teuthology.orchestra.run.smithi139.stderr: 53.5% -- replaced with /var/log/ceph/ceph-client.admin.27932.log.gz 2024-09-16T20:52:35.776 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28563.log 2024-09-16T20:52:35.776 INFO:teuthology.orchestra.run.smithi139.stderr: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.28694.log.gz 2024-09-16T20:52:35.776 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28282.log 2024-09-16T20:52:35.776 INFO:teuthology.orchestra.run.smithi139.stderr: 21.5% -- replaced with /var/log/ceph/ceph-client.admin.29992.log.gz 2024-09-16T20:52:35.776 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30035.log 2024-09-16T20:52:35.777 INFO:teuthology.orchestra.run.smithi139.stderr: 57.2% -- replaced with /var/log/ceph/ceph-client.admin.28563.log.gz 2024-09-16T20:52:35.777 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28450.log 2024-09-16T20:52:35.777 INFO:teuthology.orchestra.run.smithi139.stderr: 52.9% -- replaced with /var/log/ceph/ceph-client.admin.28282.log.gz 2024-09-16T20:52:35.777 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.30035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30062.log 2024-09-16T20:52:35.777 INFO:teuthology.orchestra.run.smithi139.stderr: 55.0% -- replaced with /var/log/ceph/ceph-client.admin.30035.log.gz 2024-09-16T20:52:35.777 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30553.log 2024-09-16T20:52:35.777 INFO:teuthology.orchestra.run.smithi139.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.28450.log.gz 2024-09-16T20:52:35.777 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.30062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27993.log 2024-09-16T20:52:35.777 INFO:teuthology.orchestra.run.smithi139.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.30062.log.gz 2024-09-16T20:52:35.777 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.30553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28318.log 2024-09-16T20:52:35.777 INFO:teuthology.orchestra.run.smithi139.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.30553.log.gz 2024-09-16T20:52:35.777 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30141.log 2024-09-16T20:52:35.777 INFO:teuthology.orchestra.run.smithi139.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.27993.log.gz 2024-09-16T20:52:35.778 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27441.log 2024-09-16T20:52:35.778 INFO:teuthology.orchestra.run.smithi139.stderr: 91.6% -- replaced with /var/log/ceph/ceph-client.admin.28318.log.gz 2024-09-16T20:52:35.778 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.30141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29300.log 2024-09-16T20:52:35.778 INFO:teuthology.orchestra.run.smithi139.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.30141.log.gz 2024-09-16T20:52:35.778 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28905.log 2024-09-16T20:52:35.778 INFO:teuthology.orchestra.run.smithi139.stderr: 56.8% -- replaced with /var/log/ceph/ceph-client.admin.27441.log.gz 2024-09-16T20:52:35.778 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29431.log 2024-09-16T20:52:35.778 INFO:teuthology.orchestra.run.smithi139.stderr: 91.6% -- replaced with /var/log/ceph/ceph-client.admin.29300.log.gz 2024-09-16T20:52:35.778 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30281.log 2024-09-16T20:52:35.778 INFO:teuthology.orchestra.run.smithi139.stderr: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.28905.log.gz 2024-09-16T20:52:35.778 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28239.log 2024-09-16T20:52:35.778 INFO:teuthology.orchestra.run.smithi139.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.29431.log.gz 2024-09-16T20:52:35.778 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.30281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28730.log 2024-09-16T20:52:35.779 INFO:teuthology.orchestra.run.smithi139.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.30281.log.gz 2024-09-16T20:52:35.779 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28633.log 2024-09-16T20:52:35.779 INFO:teuthology.orchestra.run.smithi139.stderr: 20.9% -- replaced with /var/log/ceph/ceph-client.admin.28239.log.gz 2024-09-16T20:52:35.779 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30385.log 2024-09-16T20:52:35.779 INFO:teuthology.orchestra.run.smithi139.stderr: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.28730.log.gz 2024-09-16T20:52:35.779 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29922.log 2024-09-16T20:52:35.779 INFO:teuthology.orchestra.run.smithi139.stderr: 84.5% -- replaced with /var/log/ceph/ceph-client.admin.28633.log.gz 2024-09-16T20:52:35.779 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.30385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28834.log 2024-09-16T20:52:35.779 INFO:teuthology.orchestra.run.smithi139.stderr: 55.4% -- replaced with /var/log/ceph/ceph-client.admin.30385.log.gz 2024-09-16T20:52:35.779 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28099.log 2024-09-16T20:52:35.779 INFO:teuthology.orchestra.run.smithi139.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.29922.log.gz 2024-09-16T20:52:35.779 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28414.log 2024-09-16T20:52:35.779 INFO:teuthology.orchestra.run.smithi139.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.28834.log.gz 2024-09-16T20:52:35.780 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30105.log 2024-09-16T20:52:35.780 INFO:teuthology.orchestra.run.smithi139.stderr: 20.9% -- replaced with /var/log/ceph/ceph-client.admin.28099.log.gz 2024-09-16T20:52:35.780 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29852.log 2024-09-16T20:52:35.780 INFO:teuthology.orchestra.run.smithi139.stderr: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.28414.log.gz 2024-09-16T20:52:35.780 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.30105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28133.log 2024-09-16T20:52:35.780 INFO:teuthology.orchestra.run.smithi139.stderr: 25.3% -- replaced with /var/log/ceph/ceph-client.admin.30105.log.gz 2024-09-16T20:52:35.780 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27363.log 2024-09-16T20:52:35.780 INFO:teuthology.orchestra.run.smithi139.stderr: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.29852.log.gz 2024-09-16T20:52:35.780 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29115.log 2024-09-16T20:52:35.780 INFO:teuthology.orchestra.run.smithi139.stderr: 20.9% -- replaced with /var/log/ceph/ceph-client.admin.28133.log.gz 2024-09-16T20:52:35.780 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30483.log 2024-09-16T20:52:35.780 INFO:teuthology.orchestra.run.smithi139.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.27363.log.gz 2024-09-16T20:52:35.780 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29965.log 2024-09-16T20:52:35.781 INFO:teuthology.orchestra.run.smithi139.stderr: 32.6% -- replaced with /var/log/ceph/ceph-client.admin.29115.log.gz 2024-09-16T20:52:35.781 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.30483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29054.log 2024-09-16T20:52:35.781 INFO:teuthology.orchestra.run.smithi139.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.30483.log.gz 2024-09-16T20:52:35.781 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27862.log 2024-09-16T20:52:35.781 INFO:teuthology.orchestra.run.smithi139.stderr: 53.9% -- replaced with /var/log/ceph/ceph-client.admin.29965.log.gz 2024-09-16T20:52:35.781 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29650.log 2024-09-16T20:52:35.781 INFO:teuthology.orchestra.run.smithi139.stderr: 76.2% -- replaced with /var/log/ceph/ceph-client.admin.29054.log.gz 2024-09-16T20:52:35.781 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28590.log 2024-09-16T20:52:35.781 INFO:teuthology.orchestra.run.smithi139.stderr: 53.3% -- replaced with /var/log/ceph/ceph-client.admin.27862.log.gz 2024-09-16T20:52:35.781 INFO:teuthology.orchestra.run.smithi139.stderr:gzip/var/log/ceph/ceph-client.admin.29650.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.29474.log 2024-09-16T20:52:35.781 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28590.log: 95.3% -- replaced with /var/log/ceph/ceph-client.admin.29650.log.gz 2024-09-16T20:52:35.781 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29501.log 2024-09-16T20:52:35.782 INFO:teuthology.orchestra.run.smithi139.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.28590.log.gz 2024-09-16T20:52:35.782 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30175.log 2024-09-16T20:52:35.782 INFO:teuthology.orchestra.run.smithi139.stderr: 78.6% -- replaced with /var/log/ceph/ceph-client.admin.29474.log.gz 2024-09-16T20:52:35.782 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28248.log 2024-09-16T20:52:35.782 INFO:teuthology.orchestra.run.smithi139.stderr: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.29501.log.gz 2024-09-16T20:52:35.782 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.30175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30132.log 2024-09-16T20:52:35.782 INFO:teuthology.orchestra.run.smithi139.stderr: 53.9% -- replaced with /var/log/ceph/ceph-client.admin.30175.log.gz 2024-09-16T20:52:35.782 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27792.log 2024-09-16T20:52:35.782 INFO:teuthology.orchestra.run.smithi139.stderr: 94.9% -- replaced with /var/log/ceph/ceph-client.admin.28248.log.gz 2024-09-16T20:52:35.782 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.30132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30306.log 2024-09-16T20:52:35.782 INFO:teuthology.orchestra.run.smithi139.stderr: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.30132.log.gz 2024-09-16T20:52:35.782 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30272.log 2024-09-16T20:52:35.782 INFO:teuthology.orchestra.run.smithi139.stderr: 79.4% -- replaced with /var/log/ceph/ceph-client.admin.27792.log.gz 2024-09-16T20:52:35.783 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.30306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28950.log 2024-09-16T20:52:35.783 INFO:teuthology.orchestra.run.smithi139.stderr: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.30306.log.gz 2024-09-16T20:52:35.783 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.30272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.27248.log 2024-09-16T20:52:35.783 INFO:teuthology.orchestra.run.smithi139.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.30272.log.gz 2024-09-16T20:52:35.783 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29325.log 2024-09-16T20:52:35.783 INFO:teuthology.orchestra.run.smithi139.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.28950.log.gz 2024-09-16T20:52:35.783 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.0.27248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30166.log 2024-09-16T20:52:35.783 INFO:teuthology.orchestra.run.smithi139.stderr: 93.2% -- replaced with /var/log/ceph/ceph-client.0.27248.log.gz 2024-09-16T20:52:35.783 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28914.log 2024-09-16T20:52:35.783 INFO:teuthology.orchestra.run.smithi139.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.29325.log.gz 2024-09-16T20:52:35.783 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.30166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29605.log 2024-09-16T20:52:35.783 INFO:teuthology.orchestra.run.smithi139.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.30166.log.gz 2024-09-16T20:52:35.783 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28142.log 2024-09-16T20:52:35.783 INFO:teuthology.orchestra.run.smithi139.stderr: 54.0% -- replaced with /var/log/ceph/ceph-client.admin.28914.log.gz 2024-09-16T20:52:35.783 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28880.log 2024-09-16T20:52:35.784 INFO:teuthology.orchestra.run.smithi139.stderr: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.29605.log.gz 2024-09-16T20:52:35.784 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27478.log 2024-09-16T20:52:35.784 INFO:teuthology.orchestra.run.smithi139.stderr: 54.9% -- replaced with /var/log/ceph/ceph-client.admin.28142.log.gz 2024-09-16T20:52:35.784 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28984.log 2024-09-16T20:52:35.784 INFO:teuthology.orchestra.run.smithi139.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.admin.28880.log.gz 2024-09-16T20:52:35.784 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27398.log 2024-09-16T20:52:35.784 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28984.log: 96.2% -- replaced with /var/log/ceph/ceph-client.admin.27478.log.gz 2024-09-16T20:52:35.784 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27609.log 2024-09-16T20:52:35.784 INFO:teuthology.orchestra.run.smithi139.stderr: 55.6% -- replaced with /var/log/ceph/ceph-client.admin.28984.log.gz 2024-09-16T20:52:35.784 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28203.log 2024-09-16T20:52:35.784 INFO:teuthology.orchestra.run.smithi139.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.27398.log.gz 2024-09-16T20:52:35.784 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28669.log 2024-09-16T20:52:35.784 INFO:teuthology.orchestra.run.smithi139.stderr: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.27609.log.gz 2024-09-16T20:52:35.785 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29221.log 2024-09-16T20:52:35.785 INFO:teuthology.orchestra.run.smithi139.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.28203.log.gz 2024-09-16T20:52:35.785 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28843.log 2024-09-16T20:52:35.785 INFO:teuthology.orchestra.run.smithi139.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.28669.log.gz 2024-09-16T20:52:35.785 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27889.log 2024-09-16T20:52:35.785 INFO:teuthology.orchestra.run.smithi139.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.29221.log.gz 2024-09-16T20:52:35.785 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30096.log 2024-09-16T20:52:35.785 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28843.log: /var/log/ceph/ceph-client.admin.27889.log: 92.0% -- replaced with /var/log/ceph/ceph-client.admin.28843.log.gz 2024-09-16T20:52:35.785 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28809.log 2024-09-16T20:52:35.785 INFO:teuthology.orchestra.run.smithi139.stderr: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.27889.log.gz 2024-09-16T20:52:35.785 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.30096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30211.log 2024-09-16T20:52:35.785 INFO:teuthology.orchestra.run.smithi139.stderr: 20.9% -- replaced with /var/log/ceph/ceph-client.admin.30096.log.gz 2024-09-16T20:52:35.785 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27828.log 2024-09-16T20:52:35.786 INFO:teuthology.orchestra.run.smithi139.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.28809.log.gz 2024-09-16T20:52:35.786 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.30211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27539.log 2024-09-16T20:52:35.786 INFO:teuthology.orchestra.run.smithi139.stderr: 53.7% -- replaced with /var/log/ceph/ceph-client.admin.30211.log.gz 2024-09-16T20:52:35.786 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29861.log 2024-09-16T20:52:35.786 INFO:teuthology.orchestra.run.smithi139.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.27828.log.gz 2024-09-16T20:52:35.786 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28309.log 2024-09-16T20:52:35.786 INFO:teuthology.orchestra.run.smithi139.stderr: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.27539.log.gz 2024-09-16T20:52:35.786 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27573.log 2024-09-16T20:52:35.786 INFO:teuthology.orchestra.run.smithi139.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.29861.log.gz 2024-09-16T20:52:35.786 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29090.log 2024-09-16T20:52:35.786 INFO:teuthology.orchestra.run.smithi139.stderr: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.28309.log.gz 2024-09-16T20:52:35.786 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28169.log 2024-09-16T20:52:35.786 INFO:teuthology.orchestra.run.smithi139.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.27573.log.gz 2024-09-16T20:52:35.787 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30412.log 2024-09-16T20:52:35.787 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30446.log 2024-09-16T20:52:35.787 INFO:teuthology.orchestra.run.smithi139.stderr: 95.7% -- replaced with /var/log/ceph/ceph-client.admin.29090.log.gz 2024-09-16T20:52:35.787 INFO:teuthology.orchestra.run.smithi139.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.28169.log.gz 2024-09-16T20:52:35.787 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.30412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28554.log 2024-09-16T20:52:35.787 INFO:teuthology.orchestra.run.smithi139.stderr: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.30412.log.gz 2024-09-16T20:52:35.787 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.30446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29160.log 2024-09-16T20:52:35.787 INFO:teuthology.orchestra.run.smithi139.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.30446.log.gz 2024-09-16T20:52:35.787 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27923.log 2024-09-16T20:52:35.787 INFO:teuthology.orchestra.run.smithi139.stderr: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.28554.log.gz 2024-09-16T20:52:35.787 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30376.log 2024-09-16T20:52:35.787 INFO:teuthology.orchestra.run.smithi139.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.29160.log.gz 2024-09-16T20:52:35.788 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27923.log: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.27923.log.gz 2024-09-16T20:52:35.788 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.30376.log: 20.2% -- replaced with /var/log/ceph/ceph-client.admin.30376.log.gz 2024-09-16T20:52:35.788 INFO:teuthology.orchestra.run.smithi139.stderr: 2024-09-16T20:52:35.788 INFO:teuthology.orchestra.run.smithi139.stderr:real 0m0.069s 2024-09-16T20:52:35.788 INFO:teuthology.orchestra.run.smithi139.stderr:user 0m0.070s 2024-09-16T20:52:35.788 INFO:teuthology.orchestra.run.smithi139.stderr:sys 0m0.099s 2024-09-16T20:52:35.803 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57886.log 2024-09-16T20:52:35.803 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-osd.0.log: /var/log/ceph/ceph-client.admin.38267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51042.log 2024-09-16T20:52:35.804 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38267.log.gz 2024-09-16T20:52:35.804 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.57886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50333.log 2024-09-16T20:52:35.804 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57886.log.gz 2024-09-16T20:52:35.804 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.51042.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51042.log.gz 2024-09-16T20:52:35.804 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57988.log 2024-09-16T20:52:35.804 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.50333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39670.log 2024-09-16T20:52:35.804 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50333.log.gz 2024-09-16T20:52:35.804 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.57988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61329.log 2024-09-16T20:52:35.804 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57988.log.gz 2024-09-16T20:52:35.804 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.39670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58221.log 2024-09-16T20:52:35.804 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39670.log.gz 2024-09-16T20:52:35.804 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.61329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49689.log 2024-09-16T20:52:35.804 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61329.log.gz 2024-09-16T20:52:35.805 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.58221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46858.log 2024-09-16T20:52:35.805 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58221.log.gz 2024-09-16T20:52:35.805 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.49689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47625.log 2024-09-16T20:52:35.805 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49689.log.gz 2024-09-16T20:52:35.805 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.46858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47175.log 2024-09-16T20:52:35.805 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46858.log.gz 2024-09-16T20:52:35.805 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.47625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45421.log 2024-09-16T20:52:35.805 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47625.log.gz 2024-09-16T20:52:35.805 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.47175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40508.log 2024-09-16T20:52:35.805 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47175.log.gz 2024-09-16T20:52:35.805 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.45421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48175.log 2024-09-16T20:52:35.805 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45421.log.gz 2024-09-16T20:52:35.806 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.40508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54851.log 2024-09-16T20:52:35.806 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40508.log.gz 2024-09-16T20:52:35.806 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.48175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36215.log 2024-09-16T20:52:35.806 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48175.log.gz 2024-09-16T20:52:35.806 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.54851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51084.log 2024-09-16T20:52:35.806 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54851.log.gz 2024-09-16T20:52:35.806 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.36215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48684.log 2024-09-16T20:52:35.806 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36215.log.gz 2024-09-16T20:52:35.806 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.51084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50845.log 2024-09-16T20:52:35.806 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51084.log.gz 2024-09-16T20:52:35.806 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.48684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59008.log 2024-09-16T20:52:35.806 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48684.log.gz 2024-09-16T20:52:35.806 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.50845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41988.log 2024-09-16T20:52:35.806 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50845.log.gz 2024-09-16T20:52:35.807 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.59008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40550.log 2024-09-16T20:52:35.807 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59008.log.gz 2024-09-16T20:52:35.807 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.41988.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.41988.log.gz 2024-09-16T20:52:35.807 INFO:teuthology.orchestra.run.smithi029.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.63476.log 2024-09-16T20:52:35.807 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.40550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37664.log 2024-09-16T20:52:35.807 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40550.log.gz 2024-09-16T20:52:35.807 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.63476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60440.log 2024-09-16T20:52:35.807 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63476.log.gz 2024-09-16T20:52:35.807 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.37664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44359.log 2024-09-16T20:52:35.807 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37664.log.gz 2024-09-16T20:52:35.808 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.60440.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.60440.log.gz 2024-09-16T20:52:35.808 INFO:teuthology.orchestra.run.smithi029.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.63014.log 2024-09-16T20:52:35.808 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.44359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50038.log 2024-09-16T20:52:35.808 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44359.log.gz 2024-09-16T20:52:35.808 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.63014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32062.log 2024-09-16T20:52:35.808 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63014.log.gz 2024-09-16T20:52:35.808 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.50038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44257.log 2024-09-16T20:52:35.808 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50038.log.gz 2024-09-16T20:52:35.808 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.32062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43423.log 2024-09-16T20:52:35.808 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32062.log.gz 2024-09-16T20:52:35.808 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.44257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55556.log 2024-09-16T20:52:35.808 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44257.log.gz 2024-09-16T20:52:35.809 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.43423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60970.log 2024-09-16T20:52:35.809 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43423.log.gz 2024-09-16T20:52:35.809 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.55556.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55556.log.gz 2024-09-16T20:52:35.809 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35621.log 2024-09-16T20:52:35.809 INFO:teuthology.orchestra.run.smithi029.stderr:gzip/var/log/ceph/ceph-client.admin.60970.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.47919.log 2024-09-16T20:52:35.809 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60970.log.gz 2024-09-16T20:52:35.809 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.35621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53087.log 2024-09-16T20:52:35.809 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35621.log.gz 2024-09-16T20:52:35.809 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.47919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35958.log 2024-09-16T20:52:35.809 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47919.log.gz 2024-09-16T20:52:35.809 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.53087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38626.log 2024-09-16T20:52:35.809 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53087.log.gz 2024-09-16T20:52:35.809 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.35958.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.27382.log 2024-09-16T20:52:35.810 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35958.log.gz 2024-09-16T20:52:35.810 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.38626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57048.log 2024-09-16T20:52:35.810 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38626.log.gz 2024-09-16T20:52:35.810 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph.tmp-client.admin.27382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51000.log 2024-09-16T20:52:35.810 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.27382.log.gz 2024-09-16T20:52:35.810 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.57048.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-16T20:52:35.810 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57048.log.gz 2024-09-16T20:52:35.810 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.51000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41847.log 2024-09-16T20:52:35.810 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51000.log.gz 2024-09-16T20:52:35.810 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43878.log 2024-09-16T20:52:35.810 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.41847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56792.log 2024-09-16T20:52:35.810 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41847.log.gz 2024-09-16T20:52:35.810 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.43878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37168.log 2024-09-16T20:52:35.811 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43878.log.gz 2024-09-16T20:52:35.811 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.56792.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56792.log.gz 2024-09-16T20:52:35.811 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46129.log 2024-09-16T20:52:35.811 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.37168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45083.log 2024-09-16T20:52:35.811 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37168.log.gz 2024-09-16T20:52:35.811 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.46129.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46129.log.gz 2024-09-16T20:52:35.811 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62510.log 2024-09-16T20:52:35.811 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62762.log 2024-09-16T20:52:35.811 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.45083.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45083.log.gz 2024-09-16T20:52:35.811 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.62510.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62510.log.gz 2024-09-16T20:52:35.811 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62930.log 2024-09-16T20:52:35.811 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.62762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40734.log 2024-09-16T20:52:35.811 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62762.log.gz 2024-09-16T20:52:35.812 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.62930.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62930.log.gz 2024-09-16T20:52:35.812 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54811.log 2024-09-16T20:52:35.812 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.40734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62104.log 2024-09-16T20:52:35.812 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40734.log.gz 2024-09-16T20:52:35.812 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.54811.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54811.log.gz 2024-09-16T20:52:35.812 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63602.log 2024-09-16T20:52:35.812 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36621.log 2024-09-16T20:52:35.812 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.62104.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62104.log.gz 2024-09-16T20:52:35.812 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.63602.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63602.log.gz 2024-09-16T20:52:35.812 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61670.log 2024-09-16T20:52:35.812 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.36621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35964.log 2024-09-16T20:52:35.812 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36621.log.gz 2024-09-16T20:52:35.812 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.61670.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61670.log.gz 2024-09-16T20:52:35.812 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45812.log 2024-09-16T20:52:35.813 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56556.log 2024-09-16T20:52:35.813 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.35964.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35964.log.gz 2024-09-16T20:52:35.813 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.45812.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45812.log.gz 2024-09-16T20:52:35.813 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36495.log 2024-09-16T20:52:35.813 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.56556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48390.log 2024-09-16T20:52:35.813 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56556.log.gz 2024-09-16T20:52:35.813 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.36495.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36495.log.gz 2024-09-16T20:52:35.813 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40968.log 2024-09-16T20:52:35.813 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.48390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53886.log 2024-09-16T20:52:35.813 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48390.log.gz 2024-09-16T20:52:35.813 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.40968.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40968.log.gz 2024-09-16T20:52:35.814 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46919.log 2024-09-16T20:52:35.814 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.53886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52024.log 2024-09-16T20:52:35.814 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53886.log.gz 2024-09-16T20:52:35.814 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.46919.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46919.log.gz 2024-09-16T20:52:35.814 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50652.log 2024-09-16T20:52:35.814 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.52024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62318.log 2024-09-16T20:52:35.814 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52024.log.gz 2024-09-16T20:52:35.814 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.50652.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50652.log.gz 2024-09-16T20:52:35.814 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50450.log 2024-09-16T20:52:35.814 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.62318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54872.log 2024-09-16T20:52:35.814 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62318.log.gz 2024-09-16T20:52:35.814 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.50450.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50450.log.gz 2024-09-16T20:52:35.814 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37980.log 2024-09-16T20:52:35.814 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40138.log 2024-09-16T20:52:35.815 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.54872.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54872.log.gz 2024-09-16T20:52:35.815 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.37980.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37980.log.gz 2024-09-16T20:52:35.815 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39218.log 2024-09-16T20:52:35.815 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.40138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62105.log 2024-09-16T20:52:35.815 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40138.log.gz 2024-09-16T20:52:35.815 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.39218.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39218.log.gz 2024-09-16T20:52:35.815 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54751.log 2024-09-16T20:52:35.815 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.62105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63308.log 2024-09-16T20:52:35.815 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62105.log.gz 2024-09-16T20:52:35.815 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.54751.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54751.log.gz 2024-09-16T20:52:35.815 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47033.log 2024-09-16T20:52:35.815 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.63308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54221.log 2024-09-16T20:52:35.815 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63308.log.gz 2024-09-16T20:52:35.816 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.47033.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47033.log.gz 2024-09-16T20:52:35.816 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36369.log 2024-09-16T20:52:35.816 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.54221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37288.log 2024-09-16T20:52:35.816 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54221.log.gz 2024-09-16T20:52:35.816 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.36369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62846.log 2024-09-16T20:52:35.816 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36369.log.gz 2024-09-16T20:52:35.816 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.37288.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54691.log 2024-09-16T20:52:35.816 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37288.log.gz 2024-09-16T20:52:35.816 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.62846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37877.log 2024-09-16T20:52:35.816 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62846.log.gz 2024-09-16T20:52:35.816 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.54691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51909.log 2024-09-16T20:52:35.816 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54691.log.gz 2024-09-16T20:52:35.817 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.37877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46284.log 2024-09-16T20:52:35.817 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37877.log.gz 2024-09-16T20:52:35.817 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.51909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53282.log 2024-09-16T20:52:35.817 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51909.log.gz 2024-09-16T20:52:35.817 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.46284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58589.log 2024-09-16T20:52:35.817 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46284.log.gz 2024-09-16T20:52:35.817 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.53282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37043.log 2024-09-16T20:52:35.817 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53282.log.gz 2024-09-16T20:52:35.817 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.58589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36579.log 2024-09-16T20:52:35.817 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58589.log.gz 2024-09-16T20:52:35.817 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36929.log 2024-09-16T20:52:35.817 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.37043.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37043.log.gz 2024-09-16T20:52:35.817 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.36579.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36579.log.gz 2024-09-16T20:52:35.818 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60808.log 2024-09-16T20:52:35.818 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.36929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39289.log 2024-09-16T20:52:35.818 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36929.log.gz 2024-09-16T20:52:35.818 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.60808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41734.log 2024-09-16T20:52:35.818 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60808.log.gz 2024-09-16T20:52:35.818 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.39289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35962.log 2024-09-16T20:52:35.818 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39289.log.gz 2024-09-16T20:52:35.818 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.41734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54007.log 2024-09-16T20:52:35.818 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41734.log.gz 2024-09-16T20:52:35.818 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.35962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48603.log 2024-09-16T20:52:35.818 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35962.log.gz 2024-09-16T20:52:35.818 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.54007.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.53793.log 2024-09-16T20:52:35.819 INFO:teuthology.orchestra.run.smithi029.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54007.log.gz 2024-09-16T20:52:35.819 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.48603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54528.log 2024-09-16T20:52:35.819 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48603.log.gz 2024-09-16T20:52:35.819 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.53793.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53793.log.gz 2024-09-16T20:52:35.819 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63056.log 2024-09-16T20:52:35.819 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.54528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39475.log 2024-09-16T20:52:35.819 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54528.log.gz 2024-09-16T20:52:35.819 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.63056.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63056.log.gz 2024-09-16T20:52:35.819 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53516.log 2024-09-16T20:52:35.819 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.39475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41246.log 2024-09-16T20:52:35.819 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39475.log.gz 2024-09-16T20:52:35.820 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.53516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44808.log 2024-09-16T20:52:35.820 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53516.log.gz 2024-09-16T20:52:35.820 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.41246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50725.log 2024-09-16T20:52:35.820 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41246.log.gz 2024-09-16T20:52:35.820 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.44808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36930.log 2024-09-16T20:52:35.820 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44808.log.gz 2024-09-16T20:52:35.820 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.50725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61922.log 2024-09-16T20:52:35.820 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50725.log.gz 2024-09-16T20:52:35.820 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.36930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36931.log 2024-09-16T20:52:35.820 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36930.log.gz 2024-09-16T20:52:35.820 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.61922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49804.log 2024-09-16T20:52:35.820 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61922.log.gz 2024-09-16T20:52:35.820 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.36931.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36931.log.gz 2024-09-16T20:52:35.820 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-16T20:52:35.821 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.49804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53611.log 2024-09-16T20:52:35.821 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49804.log.gz 2024-09-16T20:52:35.821 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46599.log 2024-09-16T20:52:35.821 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.53611.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.53611.log.gz -- 2024-09-16T20:52:35.821 INFO:teuthology.orchestra.run.smithi029.stderr: /var/log/ceph/ceph-client.admin.60503.log 2024-09-16T20:52:35.821 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.46599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48837.log 2024-09-16T20:52:35.821 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46599.log.gz 2024-09-16T20:52:35.821 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.60503.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60503.log.gz 2024-09-16T20:52:35.821 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56361.log 2024-09-16T20:52:35.821 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58866.log 2024-09-16T20:52:35.821 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.48837.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48837.log.gz 2024-09-16T20:52:35.821 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.56361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62106.log 2024-09-16T20:52:35.822 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56361.log.gz 2024-09-16T20:52:35.822 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.58866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37228.log 2024-09-16T20:52:35.822 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58866.log.gz 2024-09-16T20:52:35.822 INFO:teuthology.orchestra.run.smithi029.stderr: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-16T20:52:35.822 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.62106.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62106.log.gz -5 2024-09-16T20:52:35.822 INFO:teuthology.orchestra.run.smithi029.stderr: --verbose -- /var/log/ceph/ceph-client.admin.49186.log 2024-09-16T20:52:35.822 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.37228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50313.log 2024-09-16T20:52:35.822 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37228.log.gz 2024-09-16T20:52:35.822 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.49186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39263.log 2024-09-16T20:52:35.822 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49186.log.gz 2024-09-16T20:52:35.822 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.50313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55924.log 2024-09-16T20:52:35.822 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50313.log.gz 2024-09-16T20:52:35.822 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.39263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47308.log 2024-09-16T20:52:35.823 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39263.log.gz 2024-09-16T20:52:35.823 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.55924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43422.log 2024-09-16T20:52:35.823 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55924.log.gz 2024-09-16T20:52:35.823 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.47308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47450.log 2024-09-16T20:52:35.823 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47308.log.gz 2024-09-16T20:52:35.823 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.43422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59828.log 2024-09-16T20:52:35.823 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43422.log.gz 2024-09-16T20:52:35.823 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.47450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38544.log 2024-09-16T20:52:35.823 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47450.log.gz 2024-09-16T20:52:35.823 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.59828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43980.log 2024-09-16T20:52:35.823 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59828.log.gz 2024-09-16T20:52:35.823 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.38544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59775.log 2024-09-16T20:52:35.823 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38544.log.gz 2024-09-16T20:52:35.823 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.43980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36747.log 2024-09-16T20:52:35.824 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43980.log.gz 2024-09-16T20:52:35.824 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.59775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46659.log 2024-09-16T20:52:35.824 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59775.log.gz 2024-09-16T20:52:35.824 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.36747.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36747.log.gz 2024-09-16T20:52:35.824 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39863.log 2024-09-16T20:52:35.824 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.46659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52649.log 2024-09-16T20:52:35.824 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46659.log.gz 2024-09-16T20:52:35.824 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.39863.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39863.log.gz 2024-09-16T20:52:35.824 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61628.log 2024-09-16T20:52:35.824 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44612.log 2024-09-16T20:52:35.824 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.52649.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52649.log.gz 2024-09-16T20:52:35.825 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.61628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48307.log 2024-09-16T20:52:35.825 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61628.log.gz 2024-09-16T20:52:35.825 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.44612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61023.log 2024-09-16T20:52:35.825 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44612.log.gz 2024-09-16T20:52:35.825 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.48307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62804.log 2024-09-16T20:52:35.825 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48307.log.gz 2024-09-16T20:52:35.825 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.61023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62378.log 2024-09-16T20:52:35.825 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61023.log.gz 2024-09-16T20:52:35.825 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.62804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36389.log 2024-09-16T20:52:35.825 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62804.log.gz 2024-09-16T20:52:35.825 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.62378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51401.log 2024-09-16T20:52:35.825 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62378.log.gz 2024-09-16T20:52:35.825 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.36389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57221.log 2024-09-16T20:52:35.826 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36389.log.gz 2024-09-16T20:52:35.826 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.51401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60695.log 2024-09-16T20:52:35.826 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51401.log.gz 2024-09-16T20:52:35.826 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.57221.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57221.log.gz 2024-09-16T20:52:35.826 INFO:teuthology.orchestra.run.smithi029.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.52374.log 2024-09-16T20:52:35.826 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.60695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57376.log 2024-09-16T20:52:35.826 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60695.log.gz 2024-09-16T20:52:35.826 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.52374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61165.log 2024-09-16T20:52:35.826 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52374.log.gz 2024-09-16T20:52:35.826 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.57376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37581.log 2024-09-16T20:52:35.826 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57376.log.gz 2024-09-16T20:52:35.826 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.61165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36705.log 2024-09-16T20:52:35.827 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61165.log.gz 2024-09-16T20:52:35.827 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.37581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36132.log 2024-09-16T20:52:35.827 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37581.log.gz 2024-09-16T20:52:35.827 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.36705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63434.log 2024-09-16T20:52:35.827 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36705.log.gz 2024-09-16T20:52:35.827 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.36132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37108.log 2024-09-16T20:52:35.827 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36132.log.gz 2024-09-16T20:52:35.827 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.63434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56946.log 2024-09-16T20:52:35.827 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63434.log.gz 2024-09-16T20:52:35.827 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.37108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55453.log 2024-09-16T20:52:35.827 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37108.log.gz 2024-09-16T20:52:35.827 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.56946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54181.log 2024-09-16T20:52:35.827 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56946.log.gz 2024-09-16T20:52:35.827 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.55453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42552.log 2024-09-16T20:52:35.827 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55453.log.gz 2024-09-16T20:52:35.828 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.54181.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54181.log.gz 2024-09-16T20:52:35.828 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46086.log 2024-09-16T20:52:35.828 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49414.log 2024-09-16T20:52:35.828 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.42552.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42552.log.gz 2024-09-16T20:52:35.828 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.46086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55146.log 2024-09-16T20:52:35.828 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46086.log.gz 2024-09-16T20:52:35.828 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.49414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40590.log 2024-09-16T20:52:35.828 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49414.log.gz 2024-09-16T20:52:35.828 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.55146.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.55146.log.gz --verbose 2024-09-16T20:52:35.828 INFO:teuthology.orchestra.run.smithi029.stderr: -- /var/log/ceph/ceph-mon.c.log 2024-09-16T20:52:35.828 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.40590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61880.log 2024-09-16T20:52:35.828 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40590.log.gz 2024-09-16T20:52:35.828 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46816.log 2024-09-16T20:52:35.828 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.61880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41195.log 2024-09-16T20:52:35.829 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61880.log.gz 2024-09-16T20:52:35.829 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.46816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59141.log 2024-09-16T20:52:35.829 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46816.log.gz 2024-09-16T20:52:35.829 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.41195.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.41195.log.gz -- 2024-09-16T20:52:35.829 INFO:teuthology.orchestra.run.smithi029.stderr: /var/log/ceph/ceph-client.admin.59500.log 2024-09-16T20:52:35.829 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61796.log 2024-09-16T20:52:35.829 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.59141.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59141.log.gz 2024-09-16T20:52:35.829 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.59500.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.59500.log.gz -- 2024-09-16T20:52:35.829 INFO:teuthology.orchestra.run.smithi029.stderr: /var/log/ceph/ceph-client.admin.38943.log 2024-09-16T20:52:35.829 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47503.log 2024-09-16T20:52:35.829 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.61796.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61796.log.gz 2024-09-16T20:52:35.829 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.38943.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38943.log.gz 2024-09-16T20:52:35.830 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43782.log 2024-09-16T20:52:35.830 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58793.log 2024-09-16T20:52:35.830 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.47503.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47503.log.gz 2024-09-16T20:52:35.830 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.43782.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.43782.log.gz -5 2024-09-16T20:52:35.830 INFO:teuthology.orchestra.run.smithi029.stderr: --verbose -- /var/log/ceph/ceph-client.admin.57578.log 2024-09-16T20:52:35.830 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.58793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53556.log 2024-09-16T20:52:35.830 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58793.log.gz 2024-09-16T20:52:35.830 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.57578.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57578.log.gz 2024-09-16T20:52:35.830 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35823.log 2024-09-16T20:52:35.830 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47920.log 2024-09-16T20:52:35.830 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.53556.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53556.log.gz 2024-09-16T20:52:35.830 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.35823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52830.log 2024-09-16T20:52:35.830 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35823.log.gz 2024-09-16T20:52:35.830 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.47920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55811.log 2024-09-16T20:52:35.831 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47920.log.gz 2024-09-16T20:52:35.831 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.52830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43162.log 2024-09-16T20:52:35.831 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52830.log.gz 2024-09-16T20:52:35.831 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38174.log 2024-09-16T20:52:35.831 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.55811.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55811.log.gz 2024-09-16T20:52:35.831 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.43162.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43162.log.gz 2024-09-16T20:52:35.831 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61544.log 2024-09-16T20:52:35.831 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41459.log 2024-09-16T20:52:35.831 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.38174.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38174.log.gz 2024-09-16T20:52:35.831 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.61544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60165.log 2024-09-16T20:52:35.831 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61544.log.gz 2024-09-16T20:52:35.831 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.41459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52596.log 2024-09-16T20:52:35.831 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41459.log.gz 2024-09-16T20:52:35.831 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.60165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35739.log 2024-09-16T20:52:35.832 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60165.log.gz 2024-09-16T20:52:35.832 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.52596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55339.log 2024-09-16T20:52:35.832 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52596.log.gz 2024-09-16T20:52:35.832 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40867.log 2024-09-16T20:52:35.832 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.35739.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35739.log.gz 2024-09-16T20:52:35.832 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.55339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62572.log 2024-09-16T20:52:35.832 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55339.log.gz 2024-09-16T20:52:35.832 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62101.log 2024-09-16T20:52:35.832 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.40867.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40867.log.gz 2024-09-16T20:52:35.832 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.62572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57322.log 2024-09-16T20:52:35.832 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62572.log.gz 2024-09-16T20:52:35.832 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.62101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40191.log 2024-09-16T20:52:35.832 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62101.log.gz 2024-09-16T20:52:35.833 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38984.log 2024-09-16T20:52:35.833 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.57322.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57322.log.gz 2024-09-16T20:52:35.833 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.40191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62530.log 2024-09-16T20:52:35.833 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40191.log.gz 2024-09-16T20:52:35.833 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.38984.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-16T20:52:35.833 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38984.log.gz 2024-09-16T20:52:35.833 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.62530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31994.log 2024-09-16T20:52:35.833 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62530.log.gz 2024-09-16T20:52:35.833 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45905.log 2024-09-16T20:52:35.833 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-mon.a.log: /var/log/ceph/ceph-client.admin.31994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63098.log 2024-09-16T20:52:35.833 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31994.log.gz 2024-09-16T20:52:35.833 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.45905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45656.log 2024-09-16T20:52:35.833 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45905.log.gz 2024-09-16T20:52:35.834 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43484.log 2024-09-16T20:52:35.834 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.63098.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63098.log.gz 2024-09-16T20:52:35.834 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.45656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31765.log 2024-09-16T20:52:35.834 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45656.log.gz 2024-09-16T20:52:35.834 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.43484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62629.log 2024-09-16T20:52:35.834 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43484.log.gz 2024-09-16T20:52:35.834 INFO:teuthology.orchestra.run.smithi029.stderr:gzip/var/log/ceph/ceph-client.admin.31765.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.61502.log 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31765.log.gz 2024-09-16T20:52:35.834 INFO:teuthology.orchestra.run.smithi029.stderr: 2024-09-16T20:52:35.834 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.62629.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62629.log.gz 2024-09-16T20:52:35.834 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51123.log 2024-09-16T20:52:35.834 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62270.log 2024-09-16T20:52:35.834 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.61502.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61502.log.gz 2024-09-16T20:52:35.834 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.51123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63560.log 2024-09-16T20:52:35.835 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51123.log.gz 2024-09-16T20:52:35.835 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.62270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58496.log 2024-09-16T20:52:35.835 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62270.log.gz 2024-09-16T20:52:35.835 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45176.log 2024-09-16T20:52:35.835 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.63560.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63560.log.gz 2024-09-16T20:52:35.835 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.58496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43163.log 2024-09-16T20:52:35.835 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58496.log.gz 2024-09-16T20:52:35.835 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.45176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62630.log 2024-09-16T20:52:35.835 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45176.log.gz 2024-09-16T20:52:35.835 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.43163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56126.log 2024-09-16T20:52:35.835 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43163.log.gz 2024-09-16T20:52:35.835 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.62630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57753.log 2024-09-16T20:52:35.835 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62630.log.gz 2024-09-16T20:52:35.836 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.56126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35965.log 2024-09-16T20:52:35.836 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56126.log.gz 2024-09-16T20:52:35.836 INFO:teuthology.orchestra.run.smithi029.stderr:gzip/var/log/ceph/ceph-client.admin.57753.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.45319.log 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57753.log.gz 2024-09-16T20:52:35.836 INFO:teuthology.orchestra.run.smithi029.stderr: 2024-09-16T20:52:35.836 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.35965.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35965.log.gz 2024-09-16T20:52:35.836 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43164.log 2024-09-16T20:52:35.836 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55649.log 2024-09-16T20:52:35.836 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.45319.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45319.log.gz 2024-09-16T20:52:35.836 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.43164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37937.log 2024-09-16T20:52:35.836 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43164.log.gz 2024-09-16T20:52:35.836 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.55649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63518.log 2024-09-16T20:52:35.836 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55649.log.gz 2024-09-16T20:52:35.836 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.37937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44083.log 2024-09-16T20:52:35.836 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37937.log.gz 2024-09-16T20:52:35.837 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.63518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63266.log 2024-09-16T20:52:35.837 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63518.log.gz 2024-09-16T20:52:35.837 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-16T20:52:35.837 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.44083.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44083.log.gz 2024-09-16T20:52:35.837 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.63266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62972.log 2024-09-16T20:52:35.837 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63266.log.gz 2024-09-16T20:52:35.837 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41572.log 2024-09-16T20:52:35.837 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.62972.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62972.log.gz 2024-09-16T20:52:35.837 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42532.log 2024-09-16T20:52:35.837 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.41572.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41572.log.gz 2024-09-16T20:52:35.837 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47746.log 2024-09-16T20:52:35.837 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.42532.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42532.log.gz 2024-09-16T20:52:35.837 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41907.log 2024-09-16T20:52:35.837 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.47746.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47746.log.gz 2024-09-16T20:52:35.838 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51676.log 2024-09-16T20:52:35.838 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.41907.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41907.log.gz 2024-09-16T20:52:35.838 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56627.log 2024-09-16T20:52:35.838 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.51676.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51676.log.gz 2024-09-16T20:52:35.838 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61370.log 2024-09-16T20:52:35.838 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.56627.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56627.log.gz 2024-09-16T20:52:35.838 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56086.log 2024-09-16T20:52:35.838 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.61370.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61370.log.gz 2024-09-16T20:52:35.838 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49248.log 2024-09-16T20:52:35.838 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.56086.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56086.log.gz 2024-09-16T20:52:35.838 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60092.log 2024-09-16T20:52:35.838 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.49248.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49248.log.gz 2024-09-16T20:52:35.839 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49126.log 2024-09-16T20:52:35.839 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.60092.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60092.log.gz 2024-09-16T20:52:35.839 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51454.log 2024-09-16T20:52:35.839 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.49126.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49126.log.gz 2024-09-16T20:52:35.839 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42207.log 2024-09-16T20:52:35.839 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.51454.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51454.log.gz 2024-09-16T20:52:35.839 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49330.log 2024-09-16T20:52:35.839 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.42207.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42207.log.gz 2024-09-16T20:52:35.839 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46169.log 2024-09-16T20:52:35.839 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.49330.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49330.log.gz 2024-09-16T20:52:35.839 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59284.log 2024-09-16T20:52:35.839 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.46169.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46169.log.gz 2024-09-16T20:52:35.839 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51749.log 2024-09-16T20:52:35.839 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.59284.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59284.log.gz 2024-09-16T20:52:35.839 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38666.log 2024-09-16T20:52:35.840 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.51749.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51749.log.gz 2024-09-16T20:52:35.840 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44553.log 2024-09-16T20:52:35.840 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.38666.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38666.log.gz 2024-09-16T20:52:35.840 INFO:teuthology.orchestra.run.smithi029.stderr: -5 --verbose -- /var/log/ceph/ceph.log 2024-09-16T20:52:35.840 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.44553.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44553.log.gz 2024-09-16T20:52:35.840 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36217.log 2024-09-16T20:52:35.840 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60246.log 2024-09-16T20:52:35.840 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.36217.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.36217.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.53241.log 2024-09-16T20:52:35.840 INFO:teuthology.orchestra.run.smithi029.stderr: 2024-09-16T20:52:35.840 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49507.log 2024-09-16T20:52:35.840 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.60246.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60246.log.gz 2024-09-16T20:52:35.840 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.53241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36537.log 2024-09-16T20:52:35.840 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53241.log.gz 2024-09-16T20:52:35.840 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.49507.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49507.log.gz 2024-09-16T20:52:35.841 INFO:teuthology.orchestra.run.smithi029.stderr: 91.5% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-16T20:52:35.841 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35697.log 2024-09-16T20:52:35.841 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.36537.log: -- /var/log/ceph/ceph-client.admin.52105.log 2024-09-16T20:52:35.841 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36537.log.gz 2024-09-16T20:52:35.841 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.35697.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35697.log.gz 2024-09-16T20:52:35.841 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44183.log 2024-09-16T20:52:35.841 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36208.log 2024-09-16T20:52:35.841 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.52105.log: /var/log/ceph/ceph-client.admin.44183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35959.log 2024-09-16T20:52:35.841 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44183.log.gz 2024-09-16T20:52:35.841 INFO:teuthology.orchestra.run.smithi029.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52105.log.gz 2024-09-16T20:52:35.841 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.36208.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36208.log.gz 2024-09-16T20:52:35.841 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54456.log 2024-09-16T20:52:35.841 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.35959.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35959.log.gz 2024-09-16T20:52:35.841 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52986.log 2024-09-16T20:52:35.842 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.54456.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54456.log.gz 2024-09-16T20:52:35.842 INFO:teuthology.orchestra.run.smithi029.stderr: --verbose -- /var/log/ceph/ceph-client.admin.42379.log 2024-09-16T20:52:35.842 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.52986.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52986.log.gz 2024-09-16T20:52:35.842 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59520.log 2024-09-16T20:52:35.842 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.42379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62380.log 2024-09-16T20:52:35.842 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42379.log.gz 2024-09-16T20:52:35.842 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.59520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58262.log 2024-09-16T20:52:35.842 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59520.log.gz 2024-09-16T20:52:35.842 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61964.log 2024-09-16T20:52:35.842 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.62380.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62380.log.gz 2024-09-16T20:52:35.842 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36809.log 2024-09-16T20:52:35.842 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.58262.log: /var/log/ceph/ceph-client.admin.61964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61712.log 2024-09-16T20:52:35.842 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58262.log.gz 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61964.log.gz 2024-09-16T20:52:35.843 INFO:teuthology.orchestra.run.smithi029.stderr: 2024-09-16T20:52:35.843 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.36809.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36809.log.gz 2024-09-16T20:52:35.843 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36216.log 2024-09-16T20:52:35.843 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.61712.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61712.log.gz 2024-09-16T20:52:35.843 INFO:teuthology.orchestra.run.smithi029.stderr: --verbose -- /var/log/ceph/ceph-client.admin.36663.log 2024-09-16T20:52:35.843 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.36216.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36216.log.gz 2024-09-16T20:52:35.843 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45577.log 2024-09-16T20:52:35.843 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.36663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51146.log 2024-09-16T20:52:35.843 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36663.log.gz 2024-09-16T20:52:35.843 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.45577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42163.log 2024-09-16T20:52:35.843 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45577.log.gz 2024-09-16T20:52:35.843 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.51146.log: /var/log/ceph/ceph-client.admin.42849.log 2024-09-16T20:52:35.843 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51146.log.gz 2024-09-16T20:52:35.844 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.42163.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42163.log.gz 2024-09-16T20:52:35.844 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49053.log 2024-09-16T20:52:35.844 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.42849.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42849.log.gz 2024-09-16T20:52:35.844 INFO:teuthology.orchestra.run.smithi029.stderr: -- /var/log/ceph/ceph-client.admin.44881.log/var/log/ceph/ceph-client.admin.49053.log: 2024-09-16T20:52:35.844 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49053.log.gz 2024-09-16T20:52:35.844 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43341.log 2024-09-16T20:52:35.844 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.44881.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44881.log.gz 2024-09-16T20:52:35.844 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35484.log 2024-09-16T20:52:35.844 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.43341.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43341.log.gz 2024-09-16T20:52:35.844 INFO:teuthology.orchestra.run.smithi029.stderr: --verbose -- /var/log/ceph/ceph-client.admin.63140.log 2024-09-16T20:52:35.844 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.35484.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35484.log.gz 2024-09-16T20:52:35.844 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48838.log 2024-09-16T20:52:35.844 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.63140.log: gzip -5 --verbose -- 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63140.log.gz 2024-09-16T20:52:35.844 INFO:teuthology.orchestra.run.smithi029.stderr: /var/log/ceph/ceph-client.admin.35565.log 2024-09-16T20:52:35.845 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.48838.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48838.log.gz 2024-09-16T20:52:35.845 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38430.log 2024-09-16T20:52:35.845 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.35565.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35565.log.gz 2024-09-16T20:52:35.845 INFO:teuthology.orchestra.run.smithi029.stderr: --verbose -- /var/log/ceph/ceph-client.admin.63182.log 2024-09-16T20:52:35.845 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.38430.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38430.log.gz 2024-09-16T20:52:35.845 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43742.log 2024-09-16T20:52:35.845 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.63182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40413.log 2024-09-16T20:52:35.845 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63182.log.gz 2024-09-16T20:52:35.845 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.43742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52320.log 2024-09-16T20:52:35.845 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43742.log.gz 2024-09-16T20:52:35.845 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62316.log 2024-09-16T20:52:35.845 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.40413.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40413.log.gz 2024-09-16T20:52:35.845 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35781.log 2024-09-16T20:52:35.846 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.52320.log: /var/log/ceph/ceph-client.admin.62316.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62316.log.gz 2024-09-16T20:52:35.846 INFO:teuthology.orchestra.run.smithi029.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.38409.log 2024-09-16T20:52:35.846 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.35781.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52320.log.gz 2024-09-16T20:52:35.846 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35781.log.gz 2024-09-16T20:52:35.846 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61460.log 2024-09-16T20:52:35.846 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45236.log 2024-09-16T20:52:35.846 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.38409.log: /var/log/ceph/ceph-client.admin.61460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52279.log 2024-09-16T20:52:35.846 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61460.log.gz 2024-09-16T20:52:35.846 INFO:teuthology.orchestra.run.smithi029.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38409.log.gz 2024-09-16T20:52:35.846 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.45236.log: gzip 0.0% -5 --verbose -- /var/log/ceph/ceph-client.admin.63224.log 2024-09-16T20:52:35.846 INFO:teuthology.orchestra.run.smithi029.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45236.log.gz 2024-09-16T20:52:35.846 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.52279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36137.log 2024-09-16T20:52:35.846 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52279.log.gz 2024-09-16T20:52:35.846 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61287.log 2024-09-16T20:52:35.846 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.63224.log: /var/log/ceph/ceph-client.admin.36137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42696.log 2024-09-16T20:52:35.847 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36137.log.gz 2024-09-16T20:52:35.847 INFO:teuthology.orchestra.run.smithi029.stderr: -- replaced with /var/log/ceph/ceph-client.admin.63224.log.gz 2024-09-16T20:52:35.847 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.61287.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61287.log.gz 2024-09-16T20:52:35.847 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41142.log 2024-09-16T20:52:35.847 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56261.log 2024-09-16T20:52:35.847 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.42696.log: /var/log/ceph/ceph-client.admin.41142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59161.log 2024-09-16T20:52:35.847 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42696.log.gz 2024-09-16T20:52:35.847 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41142.log.gz 2024-09-16T20:52:35.847 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.56261.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56261.log.gz 2024-09-16T20:52:35.847 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50087.log 2024-09-16T20:52:35.847 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54944.log 2024-09-16T20:52:35.847 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.50087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58516.log 2024-09-16T20:52:35.847 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.59161.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50087.log.gz 2024-09-16T20:52:35.848 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59161.log.gz/var/log/ceph/ceph-client.admin.54944.log: 2024-09-16T20:52:35.848 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54944.log.gz 2024-09-16T20:52:35.848 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62305.log 2024-09-16T20:52:35.848 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61754.log 2024-09-16T20:52:35.848 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.58516.log: gzip/var/log/ceph/ceph-client.admin.62305.log: -5 --verbose 0.0% -- /var/log/ceph/ceph-client.admin.42970.log 2024-09-16T20:52:35.848 INFO:teuthology.orchestra.run.smithi029.stderr: -- replaced with /var/log/ceph/ceph-client.admin.62305.log.gz 2024-09-16T20:52:35.848 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58516.log.gz 2024-09-16T20:52:35.848 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.61754.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61754.log.gz 2024-09-16T20:52:35.848 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50408.log 2024-09-16T20:52:35.848 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.42970.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42970.log.gz 2024-09-16T20:52:35.848 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46774.log 2024-09-16T20:52:35.848 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.50408.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50408.log.gz 2024-09-16T20:52:35.849 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63642.log 2024-09-16T20:52:35.849 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.46774.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46774.log.gz 2024-09-16T20:52:35.849 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63392.log 2024-09-16T20:52:35.849 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.63642.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63642.log.gz 2024-09-16T20:52:35.849 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61418.log 2024-09-16T20:52:35.849 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.63392.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63392.log.gz 2024-09-16T20:52:35.849 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61838.log 2024-09-16T20:52:35.849 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.61418.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35646.log 2024-09-16T20:52:35.849 INFO:teuthology.orchestra.run.smithi029.stderr: -- replaced with /var/log/ceph/ceph-client.admin.61418.log.gz 2024-09-16T20:52:35.849 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.61838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62098.log 2024-09-16T20:52:35.849 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61838.log.gz 2024-09-16T20:52:35.849 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57598.log 2024-09-16T20:52:35.849 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.35646.log: /var/log/ceph/ceph-client.admin.62098.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62098.log.gz 2024-09-16T20:52:35.850 INFO:teuthology.orchestra.run.smithi029.stderr: --verbose -- /var/log/ceph/ceph-client.admin.59848.log 2024-09-16T20:52:35.850 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.57598.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57598.log.gz 2024-09-16T20:52:35.850 INFO:teuthology.orchestra.run.smithi029.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35646.log.gz 2024-09-16T20:52:35.850 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45420.log 2024-09-16T20:52:35.850 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.59848.log: /var/log/ceph/ceph-client.admin.35960.log 2024-09-16T20:52:35.850 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59848.log.gz 2024-09-16T20:52:35.850 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.45420.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45420.log.gz 2024-09-16T20:52:35.850 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40457.log 2024-09-16T20:52:35.850 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.35960.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35960.log.gz 2024-09-16T20:52:35.850 INFO:teuthology.orchestra.run.smithi029.stderr: --verbose -- /var/log/ceph/ceph-client.admin.59224.log 2024-09-16T20:52:35.850 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.40457.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40457.log.gz 2024-09-16T20:52:35.850 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38710.log 2024-09-16T20:52:35.850 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.59224.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59224.log.gz 2024-09-16T20:52:35.851 INFO:teuthology.orchestra.run.smithi029.stderr: --verbose -- /var/log/ceph/ceph-client.admin.37389.log 2024-09-16T20:52:35.851 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.38710.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38710.log.gz 2024-09-16T20:52:35.851 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62102.log 2024-09-16T20:52:35.851 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.37389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43857.log 2024-09-16T20:52:35.851 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37389.log.gz 2024-09-16T20:52:35.851 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56731.log 2024-09-16T20:52:35.851 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.62102.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62102.log.gz 2024-09-16T20:52:35.851 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55239.log 2024-09-16T20:52:35.851 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.43857.log: /var/log/ceph/ceph-client.admin.56731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39588.log 2024-09-16T20:52:35.851 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56731.log.gz 2024-09-16T20:52:35.851 INFO:teuthology.orchestra.run.smithi029.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43857.log.gz 2024-09-16T20:52:35.851 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.55239.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55239.log.gz 2024-09-16T20:52:35.852 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39904.log 2024-09-16T20:52:35.852 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.39588.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39588.log.gz 2024-09-16T20:52:35.852 INFO:teuthology.orchestra.run.smithi029.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.52924.log 2024-09-16T20:52:35.852 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.39904.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39904.log.gz 2024-09-16T20:52:35.852 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46425.log 2024-09-16T20:52:35.852 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.52924.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52924.log.gz 2024-09-16T20:52:35.852 INFO:teuthology.orchestra.run.smithi029.stderr: --verbose -- /var/log/ceph/ceph-client.admin.61586.log 2024-09-16T20:52:35.852 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.46425.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46425.log.gz 2024-09-16T20:52:35.852 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62650.log 2024-09-16T20:52:35.852 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.61586.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61586.log.gz 2024-09-16T20:52:35.852 INFO:teuthology.orchestra.run.smithi029.stderr: --verbose -- /var/log/ceph/ceph-client.admin.62888.log 2024-09-16T20:52:35.852 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.62650.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62650.log.gz 2024-09-16T20:52:35.852 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43659.log 2024-09-16T20:52:35.852 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.62888.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62888.log.gz 2024-09-16T20:52:35.853 INFO:teuthology.orchestra.run.smithi029.stderr: --verbose -- /var/log/ceph/ceph-client.admin.49763.log 2024-09-16T20:52:35.853 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.43659.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43659.log.gz 2024-09-16T20:52:35.853 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36429.log 2024-09-16T20:52:35.853 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.49763.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49763.log.gz 2024-09-16T20:52:35.853 INFO:teuthology.orchestra.run.smithi029.stderr: --verbose -- /var/log/ceph/ceph-client.admin.63350.log 2024-09-16T20:52:35.853 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.36429.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36429.log.gz 2024-09-16T20:52:35.853 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.63350.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63350.log.gz 2024-09-16T20:52:35.854 INFO:teuthology.orchestra.run.smithi029.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-16T20:52:35.854 INFO:teuthology.orchestra.run.smithi043.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-16T20:52:36.025 INFO:teuthology.orchestra.run.smithi029.stderr: 90.4% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-16T20:52:41.176 INFO:teuthology.orchestra.run.smithi043.stderr: 95.9% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-16T20:52:42.129 INFO:teuthology.orchestra.run.smithi043.stderr: 95.9% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-16T20:52:43.823 INFO:teuthology.orchestra.run.smithi043.stderr: 95.9% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-16T20:52:43.825 INFO:teuthology.orchestra.run.smithi043.stderr: 2024-09-16T20:52:43.825 INFO:teuthology.orchestra.run.smithi043.stderr:real 0m8.250s 2024-09-16T20:52:43.825 INFO:teuthology.orchestra.run.smithi043.stderr:user 0m19.902s 2024-09-16T20:52:43.825 INFO:teuthology.orchestra.run.smithi043.stderr:sys 0m0.733s 2024-09-16T20:52:44.148 INFO:teuthology.orchestra.run.smithi029.stderr: 95.6% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-16T20:52:44.858 INFO:teuthology.orchestra.run.smithi029.stderr: 95.7% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-16T20:52:45.744 INFO:teuthology.orchestra.run.smithi029.stderr: 95.7% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-16T20:52:45.746 INFO:teuthology.orchestra.run.smithi029.stderr: 2024-09-16T20:52:45.747 INFO:teuthology.orchestra.run.smithi029.stderr:real 0m10.175s 2024-09-16T20:52:45.747 INFO:teuthology.orchestra.run.smithi029.stderr:user 0m27.601s 2024-09-16T20:52:45.747 INFO:teuthology.orchestra.run.smithi029.stderr:sys 0m1.162s 2024-09-16T20:52:45.747 INFO:tasks.ceph:Archiving logs... 2024-09-16T20:52:45.747 DEBUG:teuthology.misc:Transferring archived files from smithi029:/var/log/ceph to /home/teuthworker/archive/dis-2024-09-16_19:45:14-rbd-wip-dis-testing-distro-default-smithi/7907507/remote/smithi029/log 2024-09-16T20:52:45.748 DEBUG:teuthology.orchestra.run.smithi029:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T20:52:48.990 DEBUG:teuthology.misc:Transferring archived files from smithi043:/var/log/ceph to /home/teuthworker/archive/dis-2024-09-16_19:45:14-rbd-wip-dis-testing-distro-default-smithi/7907507/remote/smithi043/log 2024-09-16T20:52:48.990 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T20:52:51.112 DEBUG:teuthology.misc:Transferring archived files from smithi139:/var/log/ceph to /home/teuthworker/archive/dis-2024-09-16_19:45:14-rbd-wip-dis-testing-distro-default-smithi/7907507/remote/smithi139/log 2024-09-16T20:52:51.113 DEBUG:teuthology.orchestra.run.smithi139:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T20:52:51.275 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-16T20:52:51.296 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-16T20:52:51.296 DEBUG:teuthology.orchestra.run.smithi029:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T20:52:51.299 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T20:52:51.302 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T20:52:51.394 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-16T20:52:51.403 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-16T20:52:51.403 DEBUG:teuthology.orchestra.run.smithi029:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T20:52:51.406 DEBUG:teuthology.orchestra.run.smithi043:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T20:52:51.408 DEBUG:teuthology.orchestra.run.smithi139:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T20:52:51.421 INFO:teuthology.orchestra.run.smithi029.stderr:bash: line 1: ntpq: command not found 2024-09-16T20:52:51.423 INFO:teuthology.orchestra.run.smithi043.stderr:bash: line 1: ntpq: command not found 2024-09-16T20:52:51.425 INFO:teuthology.orchestra.run.smithi139.stderr:bash: line 1: ntpq: command not found 2024-09-16T20:52:51.426 INFO:teuthology.orchestra.run.smithi029.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T20:52:51.426 INFO:teuthology.orchestra.run.smithi029.stdout:=============================================================================== 2024-09-16T20:52:51.426 INFO:teuthology.orchestra.run.smithi029.stdout:^* hv01.front.sepia.ceph.com 3 6 377 53 +1134us[+1178us] +/- 40ms 2024-09-16T20:52:51.427 INFO:teuthology.orchestra.run.smithi029.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 54 +587us[ +631us] +/- 88ms 2024-09-16T20:52:51.427 INFO:teuthology.orchestra.run.smithi029.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 54 -3107us[-3063us] +/- 86ms 2024-09-16T20:52:51.427 INFO:teuthology.orchestra.run.smithi029.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T20:52:51.429 INFO:teuthology.orchestra.run.smithi043.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T20:52:51.429 INFO:teuthology.orchestra.run.smithi043.stdout:=============================================================================== 2024-09-16T20:52:51.429 INFO:teuthology.orchestra.run.smithi043.stdout:^* hv01.front.sepia.ceph.com 3 6 377 55 +1156us[+1185us] +/- 40ms 2024-09-16T20:52:51.429 INFO:teuthology.orchestra.run.smithi043.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 55 +618us[ +647us] +/- 88ms 2024-09-16T20:52:51.429 INFO:teuthology.orchestra.run.smithi043.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 53 -3093us[-3093us] +/- 86ms 2024-09-16T20:52:51.429 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T20:52:51.430 INFO:teuthology.orchestra.run.smithi139.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T20:52:51.430 INFO:teuthology.orchestra.run.smithi139.stdout:=============================================================================== 2024-09-16T20:52:51.430 INFO:teuthology.orchestra.run.smithi139.stdout:^* hv01.front.sepia.ceph.com 3 6 377 54 +1169us[+1185us] +/- 40ms 2024-09-16T20:52:51.430 INFO:teuthology.orchestra.run.smithi139.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 54 +566us[ +582us] +/- 88ms 2024-09-16T20:52:51.430 INFO:teuthology.orchestra.run.smithi139.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 120 -3068us[-2994us] +/- 85ms 2024-09-16T20:52:51.431 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T20:52:51.431 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-16T20:52:51.439 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-16T20:52:51.439 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-16T20:52:51.454 DEBUG:teuthology.orchestra.run.smithi029:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T20:52:51.557 DEBUG:teuthology.orchestra.run.smithi043:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T20:52:51.642 DEBUG:teuthology.orchestra.run.smithi139:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T20:52:51.708 DEBUG:teuthology.orchestra.run.smithi029:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-16T20:52:51.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T20:52:51.741 DEBUG:teuthology.orchestra.run.smithi043:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-16T20:52:51.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T20:52:51.772 DEBUG:teuthology.orchestra.run.smithi139:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-16T20:52:51.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T20:52:51.802 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-16T20:52:51.811 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-16T20:52:51.856 INFO:teuthology.task.internal:Duration was 750.383192 seconds 2024-09-16T20:52:51.856 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-16T20:52:51.872 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-16T20:52:51.872 DEBUG:teuthology.orchestra.run.smithi029:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T20:52:51.875 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T20:52:51.877 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T20:52:51.917 INFO:teuthology.orchestra.run.smithi029.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T20:52:51.917 INFO:teuthology.orchestra.run.smithi043.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T20:52:51.918 INFO:teuthology.orchestra.run.smithi139.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T20:52:52.269 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-16T20:52:52.270 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi029.front.sepia.ceph.com 2024-09-16T20:52:52.270 DEBUG:teuthology.orchestra.run.smithi029:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T20:52:52.293 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi043.front.sepia.ceph.com 2024-09-16T20:52:52.293 DEBUG:teuthology.orchestra.run.smithi043:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T20:52:52.339 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi139.front.sepia.ceph.com 2024-09-16T20:52:52.340 DEBUG:teuthology.orchestra.run.smithi139:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T20:52:52.377 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-16T20:52:52.377 DEBUG:teuthology.orchestra.run.smithi029:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T20:52:52.380 DEBUG:teuthology.orchestra.run.smithi043:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T20:52:52.382 DEBUG:teuthology.orchestra.run.smithi139:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T20:52:52.445 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-16T20:52:52.445 DEBUG:teuthology.orchestra.run.smithi029:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T20:52:52.451 DEBUG:teuthology.orchestra.run.smithi043:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T20:52:52.453 DEBUG:teuthology.orchestra.run.smithi139:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T20:52:52.590 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-16T20:52:52.601 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-16T20:52:52.601 DEBUG:teuthology.orchestra.run.smithi029:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T20:52:52.626 DEBUG:teuthology.orchestra.run.smithi043:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T20:52:52.652 DEBUG:teuthology.orchestra.run.smithi139:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T20:52:52.676 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-16T20:52:52.689 DEBUG:teuthology.orchestra.run.smithi029:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T20:52:52.691 DEBUG:teuthology.orchestra.run.smithi043:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T20:52:52.695 DEBUG:teuthology.orchestra.run.smithi139:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T20:52:52.713 INFO:teuthology.orchestra.run.smithi029.stdout:kernel.core_pattern = core 2024-09-16T20:52:52.716 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern = core 2024-09-16T20:52:52.740 INFO:teuthology.orchestra.run.smithi139.stdout:kernel.core_pattern = core 2024-09-16T20:52:52.779 DEBUG:teuthology.orchestra.run.smithi029:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T20:52:52.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T20:52:52.807 DEBUG:teuthology.orchestra.run.smithi043:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T20:52:52.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T20:52:52.824 DEBUG:teuthology.orchestra.run.smithi139:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T20:52:52.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T20:52:52.838 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-16T20:52:52.850 INFO:teuthology.task.internal:Transferring archived files... 2024-09-16T20:52:52.852 DEBUG:teuthology.misc:Transferring archived files from smithi029:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/dis-2024-09-16_19:45:14-rbd-wip-dis-testing-distro-default-smithi/7907507/remote/smithi029 2024-09-16T20:52:52.852 DEBUG:teuthology.orchestra.run.smithi029:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T20:52:52.889 DEBUG:teuthology.misc:Transferring archived files from smithi043:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/dis-2024-09-16_19:45:14-rbd-wip-dis-testing-distro-default-smithi/7907507/remote/smithi043 2024-09-16T20:52:52.889 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T20:52:52.923 DEBUG:teuthology.misc:Transferring archived files from smithi139:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/dis-2024-09-16_19:45:14-rbd-wip-dis-testing-distro-default-smithi/7907507/remote/smithi139 2024-09-16T20:52:52.923 DEBUG:teuthology.orchestra.run.smithi139:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T20:52:52.959 INFO:teuthology.task.internal:Removing archive directory... 2024-09-16T20:52:52.959 DEBUG:teuthology.orchestra.run.smithi029:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T20:52:52.961 DEBUG:teuthology.orchestra.run.smithi043:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T20:52:52.964 DEBUG:teuthology.orchestra.run.smithi139:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T20:52:53.007 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-16T20:52:53.018 INFO:teuthology.task.internal:Not uploading archives. 2024-09-16T20:52:53.018 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-16T20:52:53.030 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-16T20:52:53.031 DEBUG:teuthology.orchestra.run.smithi029:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T20:52:53.033 DEBUG:teuthology.orchestra.run.smithi043:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T20:52:53.035 DEBUG:teuthology.orchestra.run.smithi139:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T20:52:53.045 INFO:teuthology.orchestra.run.smithi029.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 20:52 /home/ubuntu/cephtest 2024-09-16T20:52:53.048 INFO:teuthology.orchestra.run.smithi043.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 20:52 /home/ubuntu/cephtest 2024-09-16T20:52:53.062 INFO:teuthology.orchestra.run.smithi139.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 20:52 /home/ubuntu/cephtest 2024-09-16T20:52:53.063 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-16T20:52:53.073 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-16T20:52:53.106 INFO:teuthology.run:Summary data: description: rbd/device/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-bitmap supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nbd} duration: 750.3831920623779 flavor: default owner: scheduled_dis@teuthology success: true 2024-09-16T20:52:53.106 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T20:52:53.217 INFO:teuthology.run:pass